{
    "1998": [
        {
            "url": "http://arxiv.org/abs/cs/9809020v1",
            "date": "1998-09-15 23:49:32+00:00",
            "title": "Linear Segmentation and Segment Significance",
            "authors": [
                "Min-Yen Kan",
                "Judith L. Klavans",
                "Kathleen R. McKeown"
            ],
            "abstract": "We present a new method for discovering a segmental discourse structure of a\ndocument while categorizing segment function. We demonstrate how retrieval of\nnoun phrases and pronominal forms, along with a zero-sum weighting scheme,\ndetermines topicalized segmentation. Futhermore, we use term distribution to\naid in identifying the role that the segment performs in the document. Finally,\nwe present results of evaluation in terms of precision and recall which surpass\nearlier approaches.",
            "journal ref": "Proceedings of 6th International Workshop of Very Large Corpora\n  (WVLC-6), Montreal, Quebec, Canada: Aug. 1998. pp. 197-205",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809022v1",
            "date": "1998-09-17 11:10:14+00:00",
            "title": "Modelling Users, Intentions, and Structure in Spoken Dialog",
            "authors": [
                "Bernd Ludwig",
                "Guenther Goerz",
                "Heinrich Niemann"
            ],
            "abstract": "We outline how utterances in dialogs can be interpreted using a partial first\norder logic. We exploit the capability of this logic to talk about the truth\nstatus of formulae to define a notion of coherence between utterances and\nexplain how this coherence relation can serve for the construction of AND/OR\ntrees that represent the segmentation of the dialog. In a BDI model we\nformalize basic assumptions about dialog and cooperative behaviour of\nparticipants. These assumptions provide a basis for inferring speech acts from\ncoherence relations between utterances and attitudes of dialog participants.\nSpeech acts prove to be useful for determining dialog segments defined on the\nnotion of completing expectations of dialog participants. Finally, we sketch\nhow explicit segmentation signalled by cue phrases and performatives is covered\nby our dialog model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809024v2",
            "date": "1998-09-18 00:33:47+00:00",
            "title": "A Lexicalized Tree Adjoining Grammar for English",
            "authors": [
                "XTAG Research Group"
            ],
            "abstract": "This document describes a sizable grammar of English written in the TAG\nformalism and implemented for use with the XTAG system. This report and the\ngrammar described herein supersedes the TAG grammar described in an earlier\n1995 XTAG technical report. The English grammar described in this report is\nbased on the TAG formalism which has been extended to include lexicalization,\nand unification-based feature structures. The range of syntactic phenomena that\ncan be handled is large and includes auxiliaries (including inversion), copula,\nraising and small clause constructions, topicalization, relative clauses,\ninfinitives, gerunds, passives, adjuncts, it-clefts, wh-clefts, PRO\nconstructions, noun-noun modifications, extraposition, determiner sequences,\ngenitives, negation, noun-verb contractions, sentential adjuncts and\nimperatives. This technical report corresponds to the XTAG Release 8/31/98. The\nXTAG grammar is continuously updated with the addition of new analyses and\nmodification of old ones, and an online version of this report can be found at\nthe XTAG web page at http://www.cis.upenn.edu/~xtag/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809026v1",
            "date": "1998-09-18 03:45:45+00:00",
            "title": "Prefix Probabilities from Stochastic Tree Adjoining Grammars",
            "authors": [
                "Mark-Jan Nederhof",
                "Anoop Sarkar",
                "Giorgio Satta"
            ],
            "abstract": "Language models for speech recognition typically use a probability model of\nthe form Pr(a_n | a_1, a_2, ..., a_{n-1}). Stochastic grammars, on the other\nhand, are typically used to assign structure to utterances. A language model of\nthe above form is constructed from such grammars by computing the prefix\nprobability Sum_{w in Sigma*} Pr(a_1 ... a_n w), where w represents all\npossible terminations of the prefix a_1 ... a_n. The main result in this paper\nis an algorithm to compute such prefix probabilities given a stochastic Tree\nAdjoining Grammar (TAG). The algorithm achieves the required computation in\nO(n^6) time. The probability of subderivations that do not derive any words in\nthe prefix, but contribute structurally to its derivation, are precomputed to\nachieve termination. This algorithm enables existing corpus-based estimation\ntechniques for stochastic TAGs to be used for language modelling.",
            "journal ref": "In Proceedings of COLING-ACL '98 (Montreal)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809027v1",
            "date": "1998-09-18 03:58:57+00:00",
            "title": "Conditions on Consistency of Probabilistic Tree Adjoining Grammars",
            "authors": [
                "Anoop Sarkar"
            ],
            "abstract": "Much of the power of probabilistic methods in modelling language comes from\ntheir ability to compare several derivations for the same string in the\nlanguage. An important starting point for the study of such cross-derivational\nproperties is the notion of _consistency_. The probability model defined by a\nprobabilistic grammar is said to be _consistent_ if the probabilities assigned\nto all the strings in the language sum to one. From the literature on\nprobabilistic context-free grammars (CFGs), we know precisely the conditions\nwhich ensure that consistency is true for a given CFG. This paper derives the\nconditions under which a given probabilistic Tree Adjoining Grammar (TAG) can\nbe shown to be consistent. It gives a simple algorithm for checking consistency\nand gives the formal justification for its correctness. The conditions derived\nhere can be used to ensure that probability models that use TAGs can be checked\nfor _deficiency_ (i.e. whether any probability mass is assigned to strings that\ncannot be generated).",
            "journal ref": "In Proceedings of COLING-ACL '98 (Montreal)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809028v1",
            "date": "1998-09-18 04:44:02+00:00",
            "title": "Separating Dependency from Constituency in a Tree Rewriting System",
            "authors": [
                "Anoop Sarkar"
            ],
            "abstract": "In this paper we present a new tree-rewriting formalism called Link-Sharing\nTree Adjoining Grammar (LSTAG) which is a variant of synchronous TAGs. Using\nLSTAG we define an approach towards coordination where linguistic dependency is\ndistinguished from the notion of constituency. Such an approach towards\ncoordination that explicitly distinguishes dependencies from constituency gives\na better formal understanding of its representation when compared to previous\napproaches that use tree-rewriting systems which conflate the two issues.",
            "journal ref": "In Proceedings of the Fifth Meeting on Mathematics of Language,\n  Saarbruecken, August 1997",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809029v1",
            "date": "1998-09-18 05:03:48+00:00",
            "title": "Incremental Parser Generation for Tree Adjoining Grammars",
            "authors": [
                "Anoop Sarkar"
            ],
            "abstract": "This paper describes the incremental generation of parse tables for the\nLR-type parsing of Tree Adjoining Languages (TALs). The algorithm presented\nhandles modifications to the input grammar by updating the parser generated so\nfar. In this paper, a lazy generation of LR-type parsers for TALs is defined in\nwhich parse tables are created by need while parsing. We then describe an\nincremental parser generator for TALs which responds to modification of the\ninput grammar by updating parse tables built so far.",
            "journal ref": "Longer version of paper in Proceedings of the 34th Meeting of the\n  ACL, Student Session. Santa Cruz, June 1996",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809050v1",
            "date": "1998-09-23 12:59:39+00:00",
            "title": "A Freely Available Morphological Analyzer, Disambiguator and Context Sensitive Lemmatizer for German",
            "authors": [
                "Wolfgang Lezius",
                "Reinhard Rapp",
                "Manfred Wettler"
            ],
            "abstract": "In this paper we present Morphy, an integrated tool for German morphology,\npart-of-speech tagging and context-sensitive lemmatization. Its large lexicon\nof more than 320,000 word forms plus its ability to process German compound\nnouns guarantee a wide morphological coverage. Syntactic ambiguities can be\nresolved with a standard statistical part-of-speech tagger. By using the output\nof the tagger, the lemmatizer can determine the correct root even for ambiguous\nword forms. The complete package is freely available and can be downloaded from\nthe World Wide Web.",
            "journal ref": "Proceedings of the COLING-ACL 1998, pp. 743-748",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809106v1",
            "date": "1998-09-25 11:02:08+00:00",
            "title": "Processing Unknown Words in HPSG",
            "authors": [
                "Petra Barg",
                "Markus Walther"
            ],
            "abstract": "The lexical acquisition system presented in this paper incrementally updates\nlinguistic properties of unknown words inferred from their surrounding context\nby parsing sentences with an HPSG grammar for German. We employ a gradual,\ninformation-based concept of ``unknownness'' providing a uniform treatment for\nthe range of completely known to maximally unknown lexical entries. ``Unknown''\ninformation is viewed as revisable information, which is either generalizable\nor specializable. Updating takes place after parsing, which only requires a\nmodified lexical lookup. Revisable pieces of information are identified by\ngrammar-specified declarations which provide access paths into the parse\nfeature structure. The updating mechanism revises the corresponding places in\nthe lexical feature structures iff the context actually provides new\ninformation. For revising generalizable information, type union is required. A\nworked-out example demonstrates the inferential capacity of our implemented\nsystem.",
            "journal ref": "Proceedings COLING-ACL'98, vol.I, 91-95",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809107v1",
            "date": "1998-09-25 14:32:38+00:00",
            "title": "Computing Declarative Prosodic Morphology",
            "authors": [
                "Markus Walther"
            ],
            "abstract": "This paper describes a computational, declarative approach to prosodic\nmorphology that uses inviolable constraints to denote small finite candidate\nsets which are filtered by a restrictive incremental optimization mechanism.\nThe new approach is illustrated with an implemented fragment of Modern Hebrew\nverbs couched in MicroCUF, an expressive constraint logic formalism. For\ngeneration and parsing of word forms, I propose a novel off-line technique to\neliminate run-time optimization. It produces a finite-state oracle that\nefficiently restricts the constraint interpreter's search space. As a\nbyproduct, unknown words can be analyzed without special mechanisms. Unlike\npure finite-state transducer approaches, this hybrid setup allows for more\nexpressivity in constraints to specify e.g. token identity for reduplication or\narithmetic constraints for phonetics.",
            "journal ref": "Proceedings of SIGPHON'98, pp. 11-20 (COLING-ACL'98\n  Post-Conference Workshop on The Computation of Phonological Constraints)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809112v1",
            "date": "1998-09-28 07:49:11+00:00",
            "title": "On the Evaluation and Comparison of Taggers: The Effect of Noise in Testing Corpora",
            "authors": [
                "L. Padro",
                "L. Marquez"
            ],
            "abstract": "This paper addresses the issue of {\\sc pos} tagger evaluation. Such\nevaluation is usually performed by comparing the tagger output with a reference\ntest corpus, which is assumed to be error-free. Currently used corpora contain\nnoise which causes the obtained performance to be a distortion of the real\nvalue. We analyze to what extent this distortion may invalidate the comparison\nbetween taggers or the measure of the improvement given by a new system. The\nmain conclusion is that a more rigorous testing experimentation\nsetting/designing is needed to reliably evaluate and compare tagger accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809113v1",
            "date": "1998-09-28 07:50:55+00:00",
            "title": "Improving Tagging Performance by Using Voting Taggers",
            "authors": [
                "L. Marquez",
                "L. Padro",
                "H. Rodriguez"
            ],
            "abstract": "We present a bootstrapping method to develop an annotated corpus, which is\nspecially useful for languages with few available resources. The method is\nbeing applied to develop a corpus of Spanish of over 5Mw. The method consists\non taking advantage of the collaboration of two different POS taggers. The\ncases in which both taggers agree present a higher accuracy and are used to\nretrain the taggers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9810014v1",
            "date": "1998-10-13 20:33:05+00:00",
            "title": "Resources for Evaluation of Summarization Techniques",
            "authors": [
                "Judith L. Klavans",
                "Kathleen R. McKeown",
                "Min-Yen Kan",
                "Susan Lee"
            ],
            "abstract": "We report on two corpora to be used in the evaluation of component systems\nfor the tasks of (1) linear segmentation of text and (2) summary-directed\nsentence extraction. We present characteristics of the corpora, methods used in\nthe collection of user judgments, and an overview of the application of the\ncorpora to evaluating the component system. Finally, we discuss the problems\nand issues with construction of the test set which apply broadly to the\nconstruction of evaluation resources for language technologies.",
            "journal ref": "in Proc. of First International Conference on Language Resources\n  and Evaluation, Rubio, Gallardo, Castro, and Tejada (eds.), Granada, Spain,\n  1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9810015v1",
            "date": "1998-10-13 21:17:13+00:00",
            "title": "Restrictions on Tree Adjoining Languages",
            "authors": [
                "Giorgio Satta",
                "William Schuler"
            ],
            "abstract": "Several methods are known for parsing languages generated by Tree Adjoining\nGrammars (TAGs) in O(n^6) worst case running time. In this paper we investigate\nwhich restrictions on TAGs and TAG derivations are needed in order to lower\nthis O(n^6) time complexity, without introducing large runtime constants, and\nwithout losing any of the generative power needed to capture the syntactic\nconstructions in natural language that can be handled by unrestricted TAGs. In\nparticular, we describe an algorithm for parsing a strict subclass of TAG in\nO(n^5), and attempt to show that this subclass retains enough generative power\nto make it useful in the general case.",
            "journal ref": "Proceedings of COLING-ACL'98",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811008v1",
            "date": "1998-11-02 21:29:41+00:00",
            "title": "Translating near-synonyms: Possibilities and preferences in the interlingua",
            "authors": [
                "Philip Edmonds"
            ],
            "abstract": "This paper argues that an interlingual representation must explicitly\nrepresent some parts of the meaning of a situation as possibilities (or\npreferences), not as necessary or definite components of meaning (or\nconstraints). Possibilities enable the analysis and generation of nuance,\nsomething required for faithful translation. Furthermore, the representation of\nthe meaning of words, especially of near-synonyms, is crucial, because it\nspecifies which nuances words can convey in which contexts.",
            "journal ref": "Proceedings of the AMTA/SIG-IL Second Workshop on Interlinguas,\n  October 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811009v1",
            "date": "1998-11-02 23:06:19+00:00",
            "title": "Choosing the Word Most Typical in Context Using a Lexical Co-occurrence Network",
            "authors": [
                "Philip Edmonds"
            ],
            "abstract": "This paper presents a partial solution to a component of the problem of\nlexical choice: choosing the synonym most typical, or expected, in context. We\napply a new statistical approach to representing the context of a word through\nlexical co-occurrence networks. The implementation was trained and evaluated on\na large corpus, and results show that the inclusion of second-order\nco-occurrence relations improves the performance of our implemented lexical\nchoice program.",
            "journal ref": "Proceedings of ACL-EACL '97, student session",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811016v1",
            "date": "1998-11-11 11:06:34+00:00",
            "title": "Comparing a statistical and a rule-based tagger for German",
            "authors": [
                "Martin Volk",
                "Gerold Schneider"
            ],
            "abstract": "In this paper we present the results of comparing a statistical tagger for\nGerman based on decision trees and a rule-based Brill-Tagger for German. We\nused the same training corpus (and therefore the same tag-set) to train both\ntaggers. We then applied the taggers to the same test corpus and compared their\nrespective behavior and in particular their error rates. Both taggers perform\nsimilarly with an error rate of around 5%. From the detailed error analysis it\ncan be seen that the rule-based tagger has more problems with unknown words\nthan the statistical tagger. But the results are opposite for tokens that are\nmany-ways ambiguous. If the unknown words are fed into the taggers with the\nhelp of an external lexicon (such as the Gertwol system) the error rate of the\nrule-based tagger drops to 4.7%, and the respective rate of the statistical\ntaggers drops to around 3.7%. Combining the taggers by using the output of one\ntagger to help the other did not lead to any further improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811022v2",
            "date": "1998-11-12 17:31:17+00:00",
            "title": "Expoiting Syntactic Structure for Language Modeling",
            "authors": [
                "Ciprian Chelba",
                "Frederick Jelinek"
            ],
            "abstract": "The paper presents a language model that develops syntactic structure and\nuses it to extract meaningful information from the word history, thus enabling\nthe use of long distance dependencies. The model assigns probability to every\njoint sequence of words--binary-parse-structure with headword annotation and\noperates in a left-to-right manner --- therefore usable for automatic speech\nrecognition. The model, its probabilistic parameterization, and a set of\nexperiments meant to evaluate its predictive power are presented; an\nimprovement over standard trigram modeling is achieved.",
            "journal ref": "Proceedings of ACL'98, Montreal, Canada",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811025v2",
            "date": "1998-11-13 16:53:15+00:00",
            "title": "A Structured Language Model",
            "authors": [
                "Ciprian Chelba"
            ],
            "abstract": "The paper presents a language model that develops syntactic structure and\nuses it to extract meaningful information from the word history, thus enabling\nthe use of long distance dependencies. The model assigns probability to every\njoint sequence of words - binary-parse-structure with headword annotation. The\nmodel, its probabilistic parametrization, and a set of experiments meant to\nevaluate its predictive power are presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9812001v3",
            "date": "1998-12-01 11:43:32+00:00",
            "title": "A Probabilistic Approach to Lexical Semantic Knowledge Acquisition and S tructural Disambiguation",
            "authors": [
                "Hang LI"
            ],
            "abstract": "In this thesis, I address the problem of automatically acquiring lexical\nsemantic knowledge, especially that of case frame patterns, from large corpus\ndata and using the acquired knowledge in structural disambiguation. The\napproach I adopt has the following characteristics: (1) dividing the problem\ninto three subproblems: case slot generalization, case dependency learning, and\nword clustering (thesaurus construction). (2) viewing each subproblem as that\nof statistical estimation and defining probability models for each subproblem,\n(3) adopting the Minimum Description Length (MDL) principle as learning\nstrategy, (4) employing efficient learning algorithms, and (5) viewing the\ndisambiguation problem as that of statistical prediction. Major contributions\nof this thesis include: (1) formalization of the lexical knowledge acquisition\nproblem, (2) development of a number of learning methods for lexical knowledge\nacquisition, and (3) development of a high-performance disambiguation method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9812005v1",
            "date": "1998-12-04 16:16:35+00:00",
            "title": "Optimal Multi-Paragraph Text Segmentation by Dynamic Programming",
            "authors": [
                "Oskari Heinonen"
            ],
            "abstract": "There exist several methods of calculating a similarity curve, or a sequence\nof similarity values, representing the lexical cohesion of successive text\nconstituents, e.g., paragraphs. Methods for deciding the locations of fragment\nboundaries are, however, scarce. We propose a fragmentation method based on\ndynamic programming. The method is theoretically sound and guaranteed to\nprovide an optimal splitting on the basis of a similarity curve, a preferred\nfragment length, and a cost function defined. The method is especially useful\nwhen control on fragment size is of importance.",
            "journal ref": "Proceedings of COLING-ACL '98, pp. 1484-1486, Montreal, Canada",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9812018v1",
            "date": "1998-12-16 16:37:01+00:00",
            "title": "A Flexible Shallow Approach to Text Generation",
            "authors": [
                "Stephan Busemann",
                "Helmut Horacek"
            ],
            "abstract": "In order to support the efficient development of NL generation systems, two\northogonal methods are currently pursued with emphasis: (1) reusable, general,\nand linguistically motivated surface realization components, and (2) simple,\ntask-oriented template-based techniques. In this paper we argue that, from an\napplication-oriented perspective, the benefits of both are still limited. In\norder to improve this situation, we suggest and evaluate shallow generation\nmethods associated with increased flexibility. We advise a close connection\nbetween domain-motivated and linguistic ontologies that supports the quick\nadaptation to new tasks and domains, rather than the reuse of general\nresources. Our method is especially designed for generating reports with\nlimited linguistic variations.",
            "journal ref": "Proc. 9th International Workshop on Natural Language Generation,\n  Niagara-on-the-Lake, Canada, August 1998, 238-247",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9801001v3",
            "date": "1998-01-14 23:20:23+00:00",
            "title": "Hierarchical Non-Emitting Markov Models",
            "authors": [
                "Eric Sven Ristad",
                "Robert G. Thomas"
            ],
            "abstract": "We describe a simple variant of the interpolated Markov model with\nnon-emitting state transitions and prove that it is strictly more powerful than\nany Markov model. More importantly, the non-emitting model outperforms the\nclassic interpolated model on the natural language texts under a wide range of\nexperimental conditions, with only a modest increase in computational\nrequirements. The non-emitting model is also much less prone to overfitting.\n  Keywords: Markov model, interpolated Markov model, hidden Markov model,\nmixture modeling, non-emitting state transitions, state-conditional\ninterpolation, statistical language model, discrete time series, Brown corpus,\nWall Street Journal.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9801002v1",
            "date": "1998-01-17 00:57:29+00:00",
            "title": "Identifying Discourse Markers in Spoken Dialog",
            "authors": [
                "Peter A. Heeman",
                "Donna Byron",
                "James F. Allen"
            ],
            "abstract": "In this paper, we present a method for identifying discourse marker usage in\nspontaneous speech based on machine learning. Discourse markers are denoted by\nspecial POS tags, and thus the process of POS tagging can be used to identify\ndiscourse markers. By incorporating POS tagging into language modeling,\ndiscourse markers can be identified during speech recognition, in which the\ntimeliness of the information can be used to help predict the following words.\nWe contrast this approach with an alternative machine learning approach\nproposed by Litman (1996). This paper also argues that discourse markers can be\nused to help the hearer predict the role that the upcoming utterance plays in\nthe dialog. Thus discourse markers should provide valuable evidence for\nautomatic dialog act prediction.",
            "journal ref": "AAAI 1998 Spring Symposium on Applying Machine Learning to\n  Discourse Processing",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9801003v1",
            "date": "1998-01-26 13:51:59+00:00",
            "title": "Do not forget: Full memory in memory-based learning of word pronunciation",
            "authors": [
                "Antal van den Bosch",
                "Walter Daelemans"
            ],
            "abstract": "Memory-based learning, keeping full memory of learning material, appears a\nviable approach to learning NLP tasks, and is often superior in generalisation\naccuracy to eager learning approaches that abstract from learning material.\nHere we investigate three partial memory-based learning approaches which remove\nfrom memory specific task instance types estimated to be exceptional. The three\napproaches each implement one heuristic function for estimating exceptionality\nof instance types: (i) typicality, (ii) class prediction strength, and (iii)\nfriendly-neighbourhood size. Experiments are performed with the memory-based\nlearning algorithm IB1-IG trained on English word pronunciation. We find that\nremoving instance types with low prediction strength (ii) is the only tested\nmethod which does not seriously harm generalisation accuracy. We conclude that\nkeeping full memory of types rather than tokens, and excluding minority\nambiguities appear to be the only performance-preserving optimisations of\nmemory-based learning.",
            "journal ref": "Proceedings of NeMLaP3/CoNLL98, 195-204",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9801004v1",
            "date": "1998-01-26 14:42:24+00:00",
            "title": "Modularity in inductively-learned word pronunciation systems",
            "authors": [
                "Antal van den Bosch",
                "Ton Weijters",
                "Walter Daelemans"
            ],
            "abstract": "In leading morpho-phonological theories and state-of-the-art text-to-speech\nsystems it is assumed that word pronunciation cannot be learned or performed\nwithout in-between analyses at several abstraction levels (e.g., morphological,\ngraphemic, phonemic, syllabic, and stress levels). We challenge this assumption\nfor the case of English word pronunciation. Using IGTree, an inductive-learning\ndecision-tree algorithms, we train and test three word-pronunciation systems in\nwhich the number of abstraction levels (implemented as sequenced modules) is\nreduced from five, via three, to one. The latter system, classifying letter\nstrings directly as mapping to phonemes with stress markers, yields\nsignificantly better generalisation accuracies than the two multi-module\nsystems. Analyses of empirical results indicate that positive utility effects\nof sequencing modules are outweighed by cascading errors passed on between\nmodules.",
            "journal ref": "Proceedings of NeMLaP3/CoNLL98, 185-194",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9801005v1",
            "date": "1998-01-26 16:36:27+00:00",
            "title": "A General, Sound and Efficient Natural Language Parsing Algorithm based on Syntactic Constraints Propagation",
            "authors": [
                "Jose F. Quesada"
            ],
            "abstract": "This paper presents a new context-free parsing algorithm based on a\nbidirectional strictly horizontal strategy which incorporates strong top-down\npredictions (derivations and adjacencies). From a functional point of view, the\nparser is able to propagate syntactic constraints reducing parsing ambiguity.\n  From a computational perspective, the algorithm includes different techniques\naimed at the improvement of the manipulation and representation of the\nstructures used.",
            "journal ref": "Proceedings CAEPIA'97, Malaga, Spain. pp. 775-786",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9802001v1",
            "date": "1998-02-02 08:36:16+00:00",
            "title": "Look-Back and Look-Ahead in the Conversion of Hidden Markov Models into Finite State Transducers",
            "authors": [
                "Andre Kempe"
            ],
            "abstract": "This paper describes the conversion of a Hidden Markov Model into a finite\nstate transducer that closely approximates the behavior of the stochastic\nmodel. In some cases the transducer is equivalent to the HMM. This conversion\nis especially advantageous for part-of-speech tagging because the resulting\ntransducer can be composed with other transducers that encode correction rules\nfor the most frequent tagging errors. The speed of tagging is also improved.\nThe described methods have been implemented and successfully tested.",
            "journal ref": "NeMLaP3/CoNLL'98, pp.29-37, Sydney, Australia. January 15-17, 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9802002v1",
            "date": "1998-02-11 19:04:29+00:00",
            "title": "A Hybrid Environment for Syntax-Semantic Tagging",
            "authors": [
                "Lluis Padro"
            ],
            "abstract": "The thesis describes the application of the relaxation labelling algorithm to\nNLP disambiguation. Language is modelled through context constraint inspired on\nConstraint Grammars. The constraints enable the use of a real value statind\n\"compatibility\". The technique is applied to POS tagging, Shallow Parsing and\nWord Sense Disambigation. Experiments and results are reported. The proposed\napproach enables the use of multi-feature constraint models, the simultaneous\nresolution of several NL disambiguation tasks, and the collaboration of\nlinguistic and statistical models.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9803001v1",
            "date": "1998-03-02 20:32:58+00:00",
            "title": "Automating Coreference: The Role of Annotated Training Data",
            "authors": [
                "Lynette Hirschman",
                "Patricia Robinson",
                "John Burger",
                "Marc Vilain"
            ],
            "abstract": "We report here on a study of interannotator agreement in the coreference task\nas defined by the Message Understanding Conference (MUC-6 and MUC-7). Based on\nfeedback from annotators, we clarified and simplified the annotation\nspecification. We then performed an analysis of disagreement among several\nannotators, concluding that only 16% of the disagreements represented genuine\ndisagreement about coreference; the remainder of the cases were mostly\ntypographical errors or omissions, easily reconciled. Initially, we measured\ninterannotator agreement in the low 80s for precision and recall. To try to\nimprove upon this, we ran several experiments. In our final experiment, we\nseparated the tagging of candidate noun phrases from the linking of actual\ncoreferring expressions. This method shows promise - interannotator agreement\nclimbed to the low 90s - but it needs more extensive validation. These results\nposition the research community to broaden the coreference task to multiple\nlanguages, and possibly to different kinds of coreference.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9803002v1",
            "date": "1998-03-22 15:05:40+00:00",
            "title": "Time, Tense and Aspect in Natural Language Database Interfaces",
            "authors": [
                "I. Androutsopoulos",
                "G. D. Ritchie",
                "P. Thanisch"
            ],
            "abstract": "Most existing natural language database interfaces (NLDBs) were designed to\nbe used with database systems that provide very limited facilities for\nmanipulating time-dependent data, and they do not support adequately temporal\nlinguistic mechanisms (verb tenses, temporal adverbials, temporal subordinate\nclauses, etc.). The database community is becoming increasingly interested in\ntemporal database systems, that are intended to store and manipulate in a\nprincipled manner information not only about the present, but also about the\npast and future. When interfacing to temporal databases, supporting temporal\nlinguistic mechanisms becomes crucial.\n  We present a framework for constructing natural language interfaces for\ntemporal databases (NLTDBs), that draws on research in tense and aspect\ntheories, temporal logics, and temporal databases. The framework consists of a\ntemporal intermediate representation language, called TOP, an HPSG grammar that\nmaps a wide range of questions involving temporal mechanisms to appropriate TOP\nexpressions, and a provably correct method for translating from TOP to TSQL2,\nTSQL2 being a recently proposed temporal extension of the SQL database\nlanguage. This framework was employed to implement a prototype NLTDB using ALE\nand Prolog.",
            "journal ref": "Natural Language Engineering, 4(3), pp. 229-276, Sept. 1998,\n  Cambridge Univ. Press.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9803003v1",
            "date": "1998-03-27 15:38:37+00:00",
            "title": "Nymble: a High-Performance Learning Name-finder",
            "authors": [
                "Daniel M. Bikel",
                "Scott Miller",
                "Richard Schwartz",
                "Ralph Weischedel"
            ],
            "abstract": "This paper presents a statistical, learned approach to finding names and\nother non-recursive entities in text (as per the MUC-6 definition of the NE\ntask), using a variant of the standard hidden Markov model. We present our\njustification for the problem and our approach, a detailed discussion of the\nmodel itself and finally the successful results of this new approach.",
            "journal ref": "Proceedings of the Fifth Conference on Applied Natural Language\n  Processing, 1997, pp. 194-201",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9804001v1",
            "date": "1998-04-02 14:47:07+00:00",
            "title": "Graph Interpolation Grammars: a Rule-based Approach to the Incremental Parsing of Natural Languages",
            "authors": [
                "John Larcheveque"
            ],
            "abstract": "Graph Interpolation Grammars are a declarative formalism with an operational\nsemantics. Their goal is to emulate salient features of the human parser, and\nnotably incrementality. The parsing process defined by GIGs incrementally\nbuilds a syntactic representation of a sentence as each successive lexeme is\nread. A GIG rule specifies a set of parse configurations that trigger its\napplication and an operation to perform on a matching configuration. Rules are\npartly context-sensitive; furthermore, they are reversible, meaning that their\noperations can be undone, which allows the parsing process to be\nnondeterministic. These two factors confer enough expressive power to the\nformalism for parsing natural languages.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9804002v2",
            "date": "1998-04-26 22:50:01+00:00",
            "title": "The Proper Treatment of Optimality in Computational Phonology",
            "authors": [
                "Lauri Karttunen"
            ],
            "abstract": "This paper presents a novel formalization of optimality theory. Unlike\nprevious treatments of optimality in computational linguistics, starting with\nEllison (1994), the new approach does not require any explicit marking and\ncounting of constraint violations. It is based on the notion of \"lenient\ncomposition,\" defined as the combination of ordinary composition and priority\nunion. If an underlying form has outputs that can meet a given constraint,\nlenient composition enforces the constraint; if none of the output candidates\nmeet the constraint, lenient composition allows all of them. For the sake of\ngreater efficiency, we may \"leniently compose\" the GEN relation and all the\nconstraints into a single finite-state transducer that maps each underlying\nform directly into its optimal surface realizations, and vice versa, without\never producing any failing candidates. Seen from this perspective, optimality\ntheory is surprisingly similar to the two older strains of finite-state\nphonology: classical rewrite systems and two-level models. In particular, the\nranking of optimality constraints corresponds to the ordering of rewrite rules.",
            "journal ref": "Proceedings of FSMNLP'98. International Workshop on Finite-State\n  Methods in Natural Language Processing, pages 1-12, June 29 - July 1, 1998.\n  Bilkent University. Ankara, Turkey",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9804003v1",
            "date": "1998-04-28 08:29:54+00:00",
            "title": "Treatment of Epsilon-Moves in Subset Construction",
            "authors": [
                "Gertjan van Noord"
            ],
            "abstract": "The paper discusses the problem of determinising finite-state automata\ncontaining large numbers of epsilon-moves. Experiments with finite-state\napproximations of natural language grammars often give rise to very large\nautomata with a very large number of epsilon-moves. The paper identifies three\nsubset construction algorithms which treat epsilon-moves. A number of\nexperiments has been performed which indicate that the algorithms differ\nconsiderably in practice. Furthermore, the experiments suggest that the average\nnumber of epsilon-moves per state can be used to predict which algorithm is\nlikely to perform best for a given input automaton.",
            "journal ref": "Proceedings of FSMNLP'98. International Workshop on Finite-State\n  Methods in Natural Language Processing, pages 1-12, June 29 - July 1, 1998.\n  Bilkent University. Ankara, Turkey.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9804004v1",
            "date": "1998-04-29 12:01:24+00:00",
            "title": "Corpus-Based Word Sense Disambiguation",
            "authors": [
                "Atsushi Fujii"
            ],
            "abstract": "Resolution of lexical ambiguity, commonly termed ``word sense\ndisambiguation'', is expected to improve the analytical accuracy for tasks\nwhich are sensitive to lexical semantics. Such tasks include machine\ntranslation, information retrieval, parsing, natural language understanding and\nlexicography. Reflecting the growth in utilization of machine readable texts,\nword sense disambiguation techniques have been explored variously in the\ncontext of corpus-based approaches. Within one corpus-based framework, that is\nthe similarity-based method, systems use a database, in which example sentences\nare manually annotated with correct word senses. Given an input, systems search\nthe database for the most similar example to the input. The lexical ambiguity\nof a word contained in the input is resolved by selecting the sense annotation\nof the retrieved example. In this research, we apply this method of resolution\nof verbal polysemy, in which the similarity between two examples is computed as\nthe weighted average of the similarity between complements governed by a target\npolysemous verb. We explore similarity-based verb sense disambiguation focusing\non the following three methods. First, we propose a weighting schema for each\nverb complement in the similarity computation. Second, in similarity-based\ntechniques, the overhead for manual supervision and searching the large-sized\ndatabase can be prohibitive. To resolve this problem, we propose a method to\nselect a small number of effective examples, for system usage. Finally, the\nefficiency of our system is highly dependent on the similarity computation\nused. To maximize efficiency, we propose a method which integrates the\nadvantages of previous methods for similarity computation.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9804005v1",
            "date": "1998-04-30 17:08:57+00:00",
            "title": "On the existence of certain total recursive functions in nontrivial axiom systems, I",
            "authors": [
                "N. C. A. da Costa",
                "F. A. Doria"
            ],
            "abstract": "We investigate the existence of a class of ZFC-provably total recursive unary\nfunctions, given certain constraints, and apply some of those results to show\nthat, for $\\Sigma_1$-sound set theory, ZFC$\\not\\vdash P<NP$.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805001v1",
            "date": "1998-05-05 10:18:17+00:00",
            "title": "Valence Induction with a Head-Lexicalized PCFG",
            "authors": [
                "Glenn Carroll",
                "Mats Rooth"
            ],
            "abstract": "This paper presents an experiment in learning valences (subcategorization\nframes) from a 50 million word text corpus, based on a lexicalized\nprobabilistic context free grammar. Distributions are estimated using a\nmodified EM algorithm. We evaluate the acquired lexicon both by comparison with\na dictionary and by entropy measures. Results show that our model produces\nhighly accurate frame distributions.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805002v1",
            "date": "1998-05-07 10:58:06+00:00",
            "title": "Group Theory and Grammatical Description",
            "authors": [
                "Marc Dymetman"
            ],
            "abstract": "This paper presents a model for linguistic description based on group theory.\nA grammar in this model, or \"G-grammar\", is a collection of lexical expressions\nwhich are products of logical forms, phonological forms, and their inverses.\nPhrasal descriptions are obtained by forming products of lexical expressions\nand by cancelling contiguous elements which are inverses of each other. We show\napplications of this model to parsing and generation, long-distance movement,\nand quantifier scoping. We believe that by moving from the free monoid over a\nvocabulary V --- standard in formal language studies --- to the free group over\nV, deep affinities between linguistic phenomena and classical algebra come to\nthe surface, and that the consequences of tapping the mathematical connections\nthus established could be considerable.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805003v1",
            "date": "1998-05-08 18:02:19+00:00",
            "title": "Models of Co-occurrence",
            "authors": [
                "I. Dan Melamed"
            ],
            "abstract": "A model of co-occurrence in bitext is a boolean predicate that indicates\nwhether a given pair of word tokens co-occur in corresponding regions of the\nbitext space. Co-occurrence is a precondition for the possibility that two\ntokens might be mutual translations. Models of co-occurrence are the glue that\nbinds methods for mapping bitext correspondence with methods for estimating\ntranslation models into an integrated system for exploiting parallel texts.\nDifferent models of co-occurrence are possible, depending on the kind of bitext\nmap that is available, the language-specific information that is available, and\nthe assumptions made about the nature of translational equivalence. Although\nmost statistical translation models are based on models of co-occurrence,\nmodeling co-occurrence correctly is more difficult than it may at first appear.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805004v1",
            "date": "1998-05-08 16:36:56+00:00",
            "title": "Annotation Style Guide for the Blinker Project",
            "authors": [
                "I. Dan Melamed"
            ],
            "abstract": "This annotation style guide was created by and for the Blinker project at the\nUniversity of Pennsylvania. The Blinker project was so named after the\n``bilingual linker'' GUI, which was created to enable bilingual annotators to\n``link'' word tokens that are mutual translations in parallel texts. The\nparallel text chosen for this project was the Bible, because it is probably the\neasiest text to obtain in electronic form in multiple languages. The languages\ninvolved were English and French, because, of the languages with which the\nproject co-ordinator was familiar, these were the two for which a sufficient\nnumber of annotators was likely to be found.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805005v2",
            "date": "1998-05-08 16:57:08+00:00",
            "title": "Manual Annotation of Translational Equivalence: The Blinker Project",
            "authors": [
                "I. Dan Melamed"
            ],
            "abstract": "Bilingual annotators were paid to link roughly sixteen thousand corresponding\nwords between on-line versions of the Bible in modern French and modern\nEnglish. These annotations are freely available to the research community from\nhttp://www.cis.upenn.edu/~melamed . The annotations can be used for several\npurposes. First, they can be used as a standard data set for developing and\ntesting translation lexicons and statistical translation models. Second,\nresearchers in lexical semantics will be able to mine the annotations for\ninsights about cross-linguistic lexicalization patterns. Third, the annotations\ncan be used in research into certain recently proposed methods for monolingual\nword-sense disambiguation. This paper describes the annotated texts, the\nspecially-designed annotation tool, and the strategies employed to increase the\nconsistency of the annotations. The annotation process was repeated five times\nby different annotators. Inter-annotator agreement rates indicate that the\nannotations are reasonably reliable and that the method is easy to replicate.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805006v1",
            "date": "1998-05-11 16:07:46+00:00",
            "title": "Word-to-Word Models of Translational Equivalence",
            "authors": [
                "I. Dan Melamed"
            ],
            "abstract": "Parallel texts (bitexts) have properties that distinguish them from other\nkinds of parallel data. First, most words translate to only one other word.\nSecond, bitext correspondence is noisy. This article presents methods for\nbiasing statistical translation models to reflect these properties. Analysis of\nthe expected behavior of these biases in the presence of sparse data predicts\nthat they will result in more accurate models. The prediction is confirmed by\nevaluation with respect to a gold standard -- translation models that are\nbiased in this fashion are significantly more accurate than a baseline\nknowledge-poor model. This article also shows how a statistical translation\nmodel can take advantage of various kinds of pre-existing knowledge that might\nbe available about particular language pairs. Even the simplest kinds of\nlanguage-specific knowledge, such as the distinction between content words and\nfunction words, is shown to reliably boost translation model performance on\nsome tasks. Statistical models that are informed by pre-existing knowledge\nabout the model domain combine the best of both the rationalist and empiricist\ntraditions.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805007v1",
            "date": "1998-05-19 16:43:26+00:00",
            "title": "Parsing Inside-Out",
            "authors": [
                "Joshua Goodman"
            ],
            "abstract": "The inside-outside probabilities are typically used for reestimating\nProbabilistic Context Free Grammars (PCFGs), just as the forward-backward\nprobabilities are typically used for reestimating HMMs. I show several novel\nuses, including improving parser accuracy by matching parsing algorithms to\nevaluation criteria; speeding up DOP parsing by 500 times; and 30 times faster\nPCFG thresholding at a given accuracy level. I also give an elegant,\nstate-of-the-art grammar formalism, which can be used to compute inside-outside\nprobabilities; and a parser description formalism, which makes it easy to\nderive inside-outside formulas and many others.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805008v1",
            "date": "1998-05-21 03:39:19+00:00",
            "title": "A Descriptive Characterization of Tree-Adjoining Languages (Full Version)",
            "authors": [
                "James Rogers"
            ],
            "abstract": "Since the early Sixties and Seventies it has been known that the regular and\ncontext-free languages are characterized by definability in the monadic\nsecond-order theory of certain structures. More recently, these descriptive\ncharacterizations have been used to obtain complexity results for constraint-\nand principle-based theories of syntax and to provide a uniform model-theoretic\nframework for exploring the relationship between theories expressed in\ndisparate formal terms. These results have been limited, to an extent, by the\nlack of descriptive characterizations of language classes beyond the\ncontext-free. Recently, we have shown that tree-adjoining languages (in a\nmildly generalized form) can be characterized by recognition by automata\noperating on three-dimensional tree manifolds, a three-dimensional analog of\ntrees. In this paper, we exploit these automata-theoretic results to obtain a\ncharacterization of the tree-adjoining languages by definability in the monadic\nsecond-order theory of these three-dimensional tree manifolds. This not only\nopens the way to extending the tools of model-theoretic syntax to the level of\nTALs, but provides a highly flexible mechanism for defining TAGs in terms of\nlogical constraints.\n  This is the full version of a paper to appear in the proceedings of\nCOLING-ACL'98 as a project note.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805009v1",
            "date": "1998-05-27 23:40:44+00:00",
            "title": "Discovery of Linguistic Relations Using Lexical Attraction",
            "authors": [
                "Deniz Yuret"
            ],
            "abstract": "This work has been motivated by two long term goals: to understand how humans\nlearn language and to build programs that can understand language. Using a\nrepresentation that makes the relevant features explicit is a prerequisite for\nsuccessful learning and understanding. Therefore, I chose to represent\nrelations between individual words explicitly in my model. Lexical attraction\nis defined as the likelihood of such relations. I introduce a new class of\nprobabilistic language models named lexical attraction models which can\nrepresent long distance relations between words and I formalize this new class\nof models using information theory.\n  Within the framework of lexical attraction, I developed an unsupervised\nlanguage acquisition program that learns to identify linguistic relations in a\ngiven sentence. The only explicitly represented linguistic knowledge in the\nprogram is lexical attraction. There is no initial grammar or lexicon built in\nand the only input is raw text. Learning and processing are interdigitated. The\nprocessor uses the regularities detected by the learner to impose structure on\nthe input. This structure enables the learner to detect higher level\nregularities. Using this bootstrapping procedure, the program was trained on\n100 million words of Associated Press material and was able to achieve 60%\nprecision and 50% recall in finding relations between content-words. Using\nknowledge of lexical attraction, the program can identify the correct relations\nin syntactically ambiguous sentences such as ``I saw the Statue of Liberty\nflying over New York.''",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805010v1",
            "date": "1998-05-28 18:14:12+00:00",
            "title": "Integrating Text Plans for Conciseness and Coherence",
            "authors": [
                "Terrence Harvey",
                "Sandra Carberry"
            ],
            "abstract": "Our experience with a critiquing system shows that when the system detects\nproblems with the user's performance, multiple critiques are often produced.\nAnalysis of a corpus of actual critiques revealed that even though each\nindividual critique is concise and coherent, the set of critiques as a whole\nmay exhibit several problems that detract from conciseness and coherence, and\nconsequently assimilation. Thus a text planner was needed that could integrate\nthe text plans for individual communicative goals to produce an overall text\nplan representing a concise, coherent message.\n  This paper presents our general rule-based system for accomplishing this\ntask. The system takes as input a \\emph{set} of individual text plans\nrepresented as RST-style trees, and produces a smaller set of more complex\ntrees representing integrated messages that still achieve the multiple\ncommunicative goals of the individual text plans. Domain-independent rules are\nused to capture strategies across domains, while the facility for addition of\ndomain-dependent rules enables the system to be tuned to the requirements of a\nparticular domain. The system has been tested on a corpus of critiques in the\ndomain of trauma care.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL '98)",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805011v1",
            "date": "1998-05-29 10:57:56+00:00",
            "title": "Automatic summarising: factors and directions",
            "authors": [
                "Karen Sparck Jones"
            ],
            "abstract": "This position paper suggests that progress with automatic summarising demands\na better research methodology and a carefully focussed research strategy. In\norder to develop effective procedures it is necessary to identify and respond\nto the context factors, i.e. input, purpose, and output factors, that bear on\nsummarising and its evaluation. The paper analyses and illustrates these\nfactors and their implications for evaluation. It then argues that this\nanalysis, together with the state of the art and the intrinsic difficulty of\nsummarising, imply a nearer-term strategy concentrating on shallow, but not\nsurface, text analysis and on indicative summarising. This is illustrated with\ncurrent work, from which a potentially productive research programme can be\ndeveloped.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9805012v1",
            "date": "1998-05-29 17:15:39+00:00",
            "title": "Recognizing Syntactic Errors in the Writing of Second Language Learners",
            "authors": [
                "David A. Schneider",
                "Kathleen F. McCoy"
            ],
            "abstract": "This paper reports on the recognition component of an intelligent tutoring\nsystem that is designed to help foreign language speakers learn standard\nEnglish. The system models the grammar of the learner, with this instantiation\nof the system tailored to signers of American Sign Language (ASL). We discuss\nthe theoretical motivations for the system, various difficulties that have been\nencountered in the implementation, as well as the methods we have used to\novercome these problems. Our method of capturing ungrammaticalities involves\nusing mal-rules (also called 'error productions'). However, the straightforward\naddition of some mal-rules causes significant performance problems with the\nparser. For instance, the ASL population has a strong tendency to drop pronouns\nand the auxiliary verb `to be'. Being able to account for these as sentences\nresults in an explosion in the number of possible parses for each sentence.\nThis explosion, left unchecked, greatly hampers the performance of the system.\nWe discuss how this is handled by taking into account expectations from the\nspecific population (some of which are captured in our unique user model). The\ndifferent representations of lexical items at various points in the acquisition\nprocess are modeled by using mal-rules, which obviates the need for multiple\nlexicons. The grammar is evaluated on its ability to correctly diagnose\nagreement problems in actual sentences produced by ASL native speakers.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL '98)",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806001v1",
            "date": "1998-05-31 21:10:44+00:00",
            "title": "Learning Correlations between Linguistic Indicators and Semantic Constraints: Reuse of Context-Dependent Descriptions of Entities",
            "authors": [
                "Dragomir R. Radev"
            ],
            "abstract": "This paper presents the results of a study on the semantic constraints\nimposed on lexical choice by certain contextual indicators. We show how such\nindicators are computed and how correlations between them and the choice of a\nnoun phrase description of a named entity can be automatically established\nusing supervised learning. Based on this correlation, we have developed a\ntechnique for automatic lexical choice of descriptions of entities in text\ngeneration. We discuss the underlying relationship between the pragmatics of\nchoosing an appropriate description that serves a specific purpose in the\nautomatically generated text and the semantics of the description itself. We\npresent our work in the framework of the more general concept of reuse of\nlinguistic structures that are automatically extracted from large corpora. We\npresent a formal evaluation of our approach and we conclude with some thoughts\non potential applications of our method.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806002v1",
            "date": "1998-06-02 14:28:17+00:00",
            "title": "Computing Dialogue Acts from Features with Transformation-Based Learning",
            "authors": [
                "Ken Samuel",
                "Sandra Carberry",
                "K. Vijay-Shanker"
            ],
            "abstract": "To interpret natural language at the discourse level, it is very useful to\naccurately recognize dialogue acts, such as SUGGEST, in identifying speaker\nintentions. Our research explores the utility of a machine learning method\ncalled Transformation-Based Learning (TBL) in computing dialogue acts, because\nTBL has a number of advantages over alternative approaches for this\napplication. We have identified some extensions to TBL that are necessary in\norder to address the limitations of the original algorithm and the particular\ndemands of discourse processing. We use a Monte Carlo strategy to increase the\napplicability of the TBL method, and we select features of utterances that can\nbe used as input to improve the performance of TBL. Our system is currently\nbeing tested on the VerbMobil corpora of spoken dialogues, producing promising\npreliminary results.",
            "journal ref": "Applying Machine Learning to Discourse Processing: Papers from the\n  1998 AAAI Spring Symposium",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806003v1",
            "date": "1998-06-03 16:47:37+00:00",
            "title": "Lazy Transformation-Based Learning",
            "authors": [
                "Ken Samuel"
            ],
            "abstract": "We introduce a significant improvement for a relatively new machine learning\nmethod called Transformation-Based Learning. By applying a Monte Carlo strategy\nto randomly sample from the space of rules, rather than exhaustively analyzing\nall possible rules, we drastically reduce the memory and time costs of the\nalgorithm, without compromising accuracy on unseen data. This enables\nTransformation- Based Learning to apply to a wider range of domains, as it can\neffectively consider a larger number of different features and feature\ninteractions in the data. In addition, the Monte Carlo improvement decreases\nthe labor demands on the human developer, who no longer needs to develop a\nminimal set of rule templates to maintain tractability.",
            "journal ref": "Proceedings of the 11th International Florida Artificial\n  Intelligence Research Symposium Conference",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806004v1",
            "date": "1998-06-05 10:52:33+00:00",
            "title": "Rationality, Cooperation and Conversational Implicature",
            "authors": [
                "Mark Lee"
            ],
            "abstract": "Conversational implicatures are usually described as being licensed by the\ndisobeying or flouting of a Principle of Cooperation. However, the\nspecification of this principle has proved computationally elusive. In this\npaper we suggest that a more useful concept is rationality. Such a concept can\nbe specified explicitely in planning terms and we argue that speakers perform\nutterances as part of the optimal plan for their particular communicative\ngoals. Such an assumption can be used by the hearer to infer conversational\nimplicatures implicit in the speaker's utterance.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806005v1",
            "date": "1998-06-05 16:38:28+00:00",
            "title": "Eliminating deceptions and mistaken belief to infer conversational implicature",
            "authors": [
                "Mark Lee",
                "Yorick Wilks"
            ],
            "abstract": "Conversational implicatures are usually described as being licensed by the\ndisobeying or flouting of some principle by the speaker in cooperative\ndialogue. However, such work has failed to distinguish cases of the speaker\nflouting such a principle from cases where the speaker is either deceptive or\nholds a mistaken belief. In this paper, we demonstrate how the three different\ncases can be distinguished in terms of the beliefs ascribed to the speaker of\nthe utterance. We argue that in the act of distinguishing the speaker's\nintention and ascribing such beliefs, the intended inference can be made by the\nhearer. This theory is implemented in ViewGen, a pre-existing belief modelling\nsystem used in a medical counselling domain.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806006v1",
            "date": "1998-06-08 16:06:59+00:00",
            "title": "Dialogue Act Tagging with Transformation-Based Learning",
            "authors": [
                "Ken Samuel",
                "Sandra Carberry",
                "K. Vijay-Shanker"
            ],
            "abstract": "For the task of recognizing dialogue acts, we are applying the\nTransformation-Based Learning (TBL) machine learning algorithm. To circumvent a\nsparse data problem, we extract values of well-motivated features of\nutterances, such as speaker direction, punctuation marks, and a new feature,\ncalled dialogue act cues, which we find to be more effective than cue phrases\nand word n-grams in practice. We present strategies for constructing a set of\ndialogue act cues automatically by minimizing the entropy of the distribution\nof dialogue acts in a training corpus, filtering out irrelevant dialogue act\ncues, and clustering semantically-related words. In addition, to address\nlimitations of TBL, we introduce a Monte Carlo strategy for training\nefficiently and a committee method for computing confidence measures. These\nideas are combined in our working implementation, which labels held-out data as\naccurately as any other reported system for the dialogue act tagging task.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL '98)",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806007v1",
            "date": "1998-06-09 23:10:27+00:00",
            "title": "An Investigation of Transformation-Based Learning in Discourse",
            "authors": [
                "Ken Samuel",
                "Sandra Carberry",
                "K. Vijay-Shanker"
            ],
            "abstract": "This paper presents results from the first attempt to apply\nTransformation-Based Learning to a discourse-level Natural Language Processing\ntask. To address two limitations of the standard algorithm, we developed a\nMonte Carlo version of Transformation-Based Learning to make the method\ntractable for a wider range of problems without degradation in accuracy, and we\ndevised a committee method for assigning confidence measures to tags produced\nby Transformation-Based Learning. The paper describes these advances, presents\nexperimental evidence that Transformation-Based Learning is as effective as\nalternative approaches (such as Decision Trees and N-Grams) for a discourse\ntask called Dialogue Act Tagging, and argues that Transformation-Based Learning\nhas desirable features that make it particularly appealing for the Dialogue Act\nTagging task.",
            "journal ref": "Machine Learning: Proceedings of the 15th International Conference",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806008v1",
            "date": "1998-06-10 13:23:54+00:00",
            "title": "Unlimited Vocabulary Grapheme to Phoneme Conversion for Korean TTS",
            "authors": [
                "Byeongchang Kim",
                "WonIl Lee",
                "Geunbae Lee",
                "Jong-Hyeok Lee"
            ],
            "abstract": "This paper describes a grapheme-to-phoneme conversion method using phoneme\nconnectivity and CCV conversion rules. The method consists of mainly four\nmodules including morpheme normalization, phrase-break detection, morpheme to\nphoneme conversion and phoneme connectivity check.\n  The morpheme normalization is to replace non-Korean symbols into standard\nKorean graphemes. The phrase-break detector assigns phrase breaks using\npart-of-speech (POS) information. In the morpheme-to-phoneme conversion module,\neach morpheme in the phrase is converted into phonetic patterns by looking up\nthe morpheme phonetic pattern dictionary which contains candidate phonological\nchanges in boundaries of the morphemes. Graphemes within a morpheme are grouped\ninto CCV patterns and converted into phonemes by the CCV conversion rules. The\nphoneme connectivity table supports grammaticality checking of the adjacent two\nphonetic morphemes.\n  In the experiments with a corpus of 4,973 sentences, we achieved 99.9% of the\ngrapheme-to-phoneme conversion performance and 97.5% of the sentence conversion\nperformance. The full Korean TTS system is now being implemented using this\nconversion method.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806009v1",
            "date": "1998-06-11 11:05:19+00:00",
            "title": "Methods and Tools for Building the Catalan WordNet",
            "authors": [
                "Laura Benitez",
                "Sergi Cervell",
                "Gerard Escudero",
                "Monica Lopez",
                "German Rigau",
                "Mariona Taule"
            ],
            "abstract": "In this paper we introduce the methodology used and the basic phases we\nfollowed to develop the Catalan WordNet, and shich lexical resources have been\nemployed in its building. This methodology, as well as the tools we made use\nof, have been thought in a general way so that they could be applied to any\nother language.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806010v1",
            "date": "1998-06-15 19:38:11+00:00",
            "title": "Towards a single proposal is spelling correction",
            "authors": [
                "E. Agirre",
                "K. Gojenola",
                "K. Sarasola"
            ],
            "abstract": "The study presented here relies on the integrated use of different kinds of\nknowledge in order to improve first-guess accuracy in non-word\ncontext-sensitive correction for general unrestricted texts. State of the art\nspelling correction systems, e.g. ispell, apart from detecting spelling errors,\nalso assist the user by offering a set of candidate corrections that are close\nto the misspelled word. Based on the correction proposals of ispell, we built\nseveral guessers, which were combined in different ways. Firstly, we evaluated\nall possibilities and selected the best ones in a corpus with artificially\ngenerated typing errors. Secondly, the best combinations were tested on texts\nwith genuine spelling errors. The results for the latter suggest that we can\nexpect automatic non-word correction for all the errors in a free running text\nwith 80% precision and a single proposal 98% of the times (1.02 proposals on\naverage).",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806011v3",
            "date": "1998-06-16 11:17:41+00:00",
            "title": "A Memory-Based Approach to Learning Shallow Natural Language Patterns",
            "authors": [
                "Shlomo Argamon",
                "Ido Dagan",
                "Yuval Krymolowski"
            ],
            "abstract": "Recognizing shallow linguistic patterns, such as basic syntactic\nrelationships between words, is a common task in applied natural language and\ntext processing. The common practice for approaching this task is by tedious\nmanual definition of possible pattern structures, often in the form of regular\nexpressions or finite automata. This paper presents a novel memory-based\nlearning method that recognizes shallow patterns in new text based on a\nbracketed training corpus. The training data are stored as-is, in efficient\nsuffix-tree data structures. Generalization is performed on-line at recognition\ntime by comparing subsequences of the new text to positive and negative\nevidence in the corpus. This way, no information in the training is lost, as\ncan happen in other learning systems that construct a single generalized model\nat the time of training. The paper presents experimental results for\nrecognizing noun phrase, subject-verb and verb-object patterns in English.\nSince the learning approach enables easy porting to new domains, we plan to\napply it to syntactic patterns in other languages and to sub-language patterns\nfor information extraction.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806012v1",
            "date": "1998-06-19 20:58:37+00:00",
            "title": "Bayesian Stratified Sampling to Assess Corpus Utility",
            "authors": [
                "Judith Hochberg",
                "Clint Scovel",
                "Timothy Thomas",
                "Sam Hall"
            ],
            "abstract": "This paper describes a method for asking statistical questions about a large\ntext corpus. We exemplify the method by addressing the question, \"What\npercentage of Federal Register documents are real documents, of possible\ninterest to a text researcher or analyst?\" We estimate an answer to this\nquestion by evaluating 200 documents selected from a corpus of 45,820 Federal\nRegister documents. Stratified sampling is used to reduce the sampling\nuncertainty of the estimate from over 3100 documents to fewer than 1000. The\nstratification is based on observed characteristics of real documents, while\nthe sampling procedure incorporates a Bayesian version of Neyman allocation. A\npossible application of the method is to establish baseline statistics used to\nestimate recall rates for information retrieval systems.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806013v1",
            "date": "1998-06-21 05:35:53+00:00",
            "title": "Can Subcategorisation Probabilities Help a Statistical Parser?",
            "authors": [
                "John Carroll",
                "Guido Minnen",
                "Ted Briscoe"
            ],
            "abstract": "Research into the automatic acquisition of lexical information from corpora\nis starting to produce large-scale computational lexicons containing data on\nthe relative frequencies of subcategorisation alternatives for individual\nverbal predicates. However, the empirical question of whether this type of\nfrequency information can in practice improve the accuracy of a statistical\nparser has not yet been answered. In this paper we describe an experiment with\na wide-coverage statistical grammar and parser for English and\nsubcategorisation frequencies acquired from ten million words of text which\nshows that this information can significantly improve parse accuracy.",
            "journal ref": "6th Workshop on Very Large Corpora, Montreal, Canada, 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806014v1",
            "date": "1998-06-22 16:20:22+00:00",
            "title": "Word Sense Disambiguation using Optimised Combinations of Knowledge Sources",
            "authors": [
                "Yorick Wilks",
                "Mark Stevenson"
            ],
            "abstract": "Word sense disambiguation algorithms, with few exceptions, have made use of\nonly one lexical knowledge source. We describe a system which performs\nunrestricted word sense disambiguation (on all content words in free text) by\ncombining different knowledge sources: semantic preferences, dictionary\ndefinitions and subject/domain codes along with part-of-speech tags. The\nusefulness of these sources is optimised by means of a learning algorithm. We\nalso describe the creation of a new sense tagged corpus by combining existing\nresources. Tested accuracy of our approach on this corpus exceeds 92%,\ndemonstrating the viability of all-word disambiguation rather than restricting\noneself to a small sample.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806015v1",
            "date": "1998-06-23 15:41:13+00:00",
            "title": "Building Accurate Semantic Taxonomies from Monolingual MRDs",
            "authors": [
                "German Rigau",
                "Horacio Rodriguez",
                "Eneko Agirre"
            ],
            "abstract": "This paper presents a method that combines a set of unsupervised algorithms\nin order to accurately build large taxonomies from any machine-readable\ndictionary (MRD). Our aim is to profit from conventional MRDs, with no explicit\nsemantic coding. We propose a system that 1) performs fully automatic exraction\nof taxonomic links from MRD entries and 2) ranks the extracted relations in a\nway that selective manual refinement is allowed. Tested accuracy can reach\naround 100% depending on the degree of coverage selected, showing that taxonomy\nbuilding is not limited to structured dictionaries such as LDOCE.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806016v1",
            "date": "1998-06-23 17:26:41+00:00",
            "title": "Using WordNet for Building WordNets",
            "authors": [
                "Xavier Farreres",
                "German Rigau",
                "Horacio Rodriguez"
            ],
            "abstract": "This paper summarises a set of methodologies and techniques for the fast\nconstruction of multilingual WordNets. The English WordNet is used in this\napproach as a backbone for Catalan and Spanish WordNets and as a lexical\nknowledge resource for several subtasks.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806017v1",
            "date": "1998-06-24 21:22:15+00:00",
            "title": "Anchoring a Lexicalized Tree-Adjoining Grammar for Discourse",
            "authors": [
                "Bonnie Lynn Webber",
                "Aravind K. Joshi"
            ],
            "abstract": "We here explore a ``fully'' lexicalized Tree-Adjoining Grammar for discourse\nthat takes the basic elements of a (monologic) discourse to be not simply\nclauses, but larger structures that are anchored on variously realized\ndiscourse cues. This link with intra-sentential grammar suggests an account for\ndifferent patterns of discourse cues, while the different structures and\noperations suggest three separate sources for elements of discourse meaning:\n(1) a compositional semantics tied to the basic trees and operations; (2) a\npresuppositional semantics carried by cue phrases that freely adjoin to trees;\nand (3) general inference, that draws additional, defeasible conclusions that\nflesh out what is conveyed compositionally.",
            "journal ref": "Proceedings of COLING-ACL'98 Workshop on Discourse Relations and\n  Discourse Markers. (Reproduced with permission of the Universite de Montreal",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806018v1",
            "date": "1998-06-25 17:31:47+00:00",
            "title": "Never Look Back: An Alternative to Centering",
            "authors": [
                "Michael Strube"
            ],
            "abstract": "I propose a model for determining the hearer's attentional state which\ndepends solely on a list of salient discourse entities (S-list). The ordering\namong the elements of the S-list covers also the function of the\nbackward-looking center in the centering model. The ranking criteria for the\nS-list are based on the distinction between hearer-old and hearer-new discourse\nentities and incorporate preferences for inter- and intra-sentential anaphora.\nThe model is the basis for an algorithm which operates incrementally, word by\nword.",
            "journal ref": "Proceedings of COLING-ACL '98",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806019v1",
            "date": "1998-06-25 17:55:20+00:00",
            "title": "An Empirical Investigation of Proposals in Collaborative Dialogues",
            "authors": [
                "Barbara Di Eugenio",
                "Pamela W. Jordan",
                "Johanna D. Moore",
                "Richmond H. Thomason"
            ],
            "abstract": "We describe a corpus-based investigation of proposals in dialogue. First, we\ndescribe our DRI compliant coding scheme and report our inter-coder reliability\nresults. Next, we test several hypotheses about what constitutes a well-formed\nproposal.",
            "journal ref": "Proceedings of COLING-ACL 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9806020v1",
            "date": "1998-06-29 16:54:17+00:00",
            "title": "Textual Economy through Close Coupling of Syntax and Semantics",
            "authors": [
                "Matthew Stone",
                "Bonnie Webber"
            ],
            "abstract": "We focus on the production of efficient descriptions of objects, actions and\nevents. We define a type of efficiency, textual economy, that exploits the\nhearer's recognition of inferential links to material elsewhere within a\nsentence. Textual economy leads to efficient descriptions because the material\nthat supports such inferences has been included to satisfy independent\ncommunicative goals, and is therefore overloaded in Pollack's sense. We argue\nthat achieving textual economy imposes strong requirements on the\nrepresentation and reasoning used in generating sentences. The representation\nmust support the generator's simultaneous consideration of syntax and\nsemantics. Reasoning must enable the generator to assess quickly and reliably\nat any stage how the hearer will interpret the current sentence, with its\n(incomplete) syntax and semantics. We show that these representational and\nreasoning requirements are met in the SPUD system for sentence planning and\nrealization.",
            "journal ref": "Proceedings 1998 Int'l Workshop on Natural Language Generation,\n  Niagara-on-the-Lake, Canada, August 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807001v1",
            "date": "1998-07-06 17:28:35+00:00",
            "title": "Evaluating a Focus-Based Approach to Anaphora Resolution",
            "authors": [
                "Saliha Azzam",
                "Kevin Humphreys",
                "Robert Gaizauskas"
            ],
            "abstract": "We present an approach to anaphora resolution based on a focusing algorithm,\nand implemented within an existing MUC (Message Understanding Conference)\nInformation Extraction system, allowing quantitative evaluation against a\nsubstantial corpus of annotated real-world texts. Extensions to the basic\nfocusing mechanism can be easily tested, resulting in refinements to the\nmechanism and resolution rules. Results are compared with the results of a\nsimpler heuristic-based approach.",
            "journal ref": "Proceedings of COLING-ACL '98",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807002v1",
            "date": "1998-07-13 13:29:26+00:00",
            "title": "The Role of Verbs in Document Analysis",
            "authors": [
                "Judith L. Klavans",
                "Min-Yen Kan"
            ],
            "abstract": "We present results of two methods for assessing the event profile of news\narticles as a function of verb type. The unique contribution of this research\nis the focus on the role of verbs, rather than nouns. Two algorithms are\npresented and evaluated, one of which is shown to accurately discriminate\ndocuments by type and semantic properties, i.e. the event profile. The initial\nmethod, using WordNet (Miller et al. 1990), produced multiple\ncross-classification of articles, primarily due to the bushy nature of the verb\ntree coupled with the sense disambiguation problem. Our second approach using\nEnglish Verb Classes and Alternations (EVCA) Levin (1993) showed that\nmonosemous categorization of the frequent verbs in WSJ made it possible to\nusefully discriminate documents. For example, our results show that articles in\nwhich communication verbs predominate tend to be opinion pieces, whereas\narticles with a high percentage of agreement verbs tend to be about mergers or\nlegal cases. An evaluation is performed on the results using Kendall's Tau. We\npresent convincing evidence for using verb semantic classes as a discriminant\nin document classification.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL '98), Montreal, Canada: Aug. 1998. pp. 680-686.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807003v1",
            "date": "1998-07-14 09:08:33+00:00",
            "title": "Centering in Dynamic Semantics",
            "authors": [
                "Daniel Hardt"
            ],
            "abstract": "Centering theory posits a discourse center, a distinguished discourse entity\nthat is the topic of a discourse. A simplified version of this theory is\ndeveloped in a Dynamic Semantics framework. In the resulting system, the\nmechanism of center shift allows a simple, elegant analysis of a variety of\nphenomena involving sloppy identity in ellipsis and ``paycheck pronouns''.",
            "journal ref": "Proceedings of COLING 96, Copenhagen, Denmark",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807004v1",
            "date": "1998-07-17 08:22:43+00:00",
            "title": "Word Clustering and Disambiguation Based on Co-occurrence Data",
            "authors": [
                "Hang Li",
                "Naoki Abe"
            ],
            "abstract": "We address the problem of clustering words (or constructing a thesaurus)\nbased on co-occurrence data, and using the acquired word classes to improve the\naccuracy of syntactic disambiguation. We view this problem as that of\nestimating a joint probability distribution specifying the joint probabilities\nof word pairs, such as noun verb pairs. We propose an efficient algorithm based\non the Minimum Description Length (MDL) principle for estimating such a\nprobability distribution. Our method is a natural extension of those proposed\nin (Brown et al 92) and (Li & Abe 96), and overcomes their drawbacks while\nretaining their advantages. We then combined this clustering method with the\ndisambiguation method of (Li & Abe 95) to derive a disambiguation method that\nmakes use of both automatically constructed thesauruses and a hand-made\nthesaurus. The overall disambiguation accuracy achieved by our method is 85.2%,\nwhich compares favorably against the accuracy (82.4%) obtained by the\nstate-of-the-art disambiguation method of (Brill & Resnik 94).",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807005v1",
            "date": "1998-07-17 12:31:51+00:00",
            "title": "Graph Interpolation Grammars as Context-Free Automata",
            "authors": [
                "John Larcheveque"
            ],
            "abstract": "A derivation step in a Graph Interpolation Grammar has the effect of scanning\nan input token. This feature, which aims at emulating the incrementality of the\nnatural parser, restricts the formal power of GIGs. This contrasts with the\nfact that the derivation mechanism involves a context-sensitive device similar\nto tree adjunction in TAGs. The combined effect of input-driven derivation and\nrestricted context-sensitiveness would be conceivably unfortunate if it turned\nout that Graph Interpolation Languages did not subsume Context Free Languages\nwhile being partially context-sensitive. This report sets about examining\nrelations between CFGs and GIGs, and shows that GILs are a proper superclass of\nCFLs. It also brings out a strong equivalence between CFGs and GIGs for the\nclass of CFLs. Thus, it lays the basis for meaningfully investigating the\namount of context-sensitiveness supported by GIGs, but leaves this\ninvestigation for further research.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807006v1",
            "date": "1998-07-17 13:56:23+00:00",
            "title": "A Maximum-Entropy Partial Parser for Unrestricted Text",
            "authors": [
                "Wojciech Skut",
                "Thorsten Brants"
            ],
            "abstract": "This paper describes a partial parser that assigns syntactic structures to\nsequences of part-of-speech tags. The program uses the maximum entropy\nparameter estimation method, which allows a flexible combination of different\nknowledge sources: the hierarchical structure, parts of speech and phrasal\ncategories. In effect, the parser goes beyond simple bracketing and recognises\neven fairly complex structures. We give accuracy figures for different\napplications of the parser.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807007v1",
            "date": "1998-07-17 14:23:10+00:00",
            "title": "Chunk Tagger - Statistical Recognition of Noun Phrases",
            "authors": [
                "Wojciech Skut",
                "Thorsten Brants"
            ],
            "abstract": "We describe a stochastic approach to partial parsing, i.e., the recognition\nof syntactic structures of limited depth. The technique utilises Markov Models,\nbut goes beyond usual bracketing approaches, since it is capable of recognising\nnot only the boundaries, but also the internal structure and syntactic category\nof simple as well as complex NP's, PP's, AP's and adverbials. We compare\ntagging accuracy for different applications and encoding schemes.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807008v1",
            "date": "1998-07-17 18:04:07+00:00",
            "title": "A Linguistically Interpreted Corpus of German Newspaper Text",
            "authors": [
                "Wojciech Skut",
                "Thorsten Brants",
                "Brigitte Krenn",
                "Hans Uszkoreit"
            ],
            "abstract": "In this paper, we report on the development of an annotation scheme and\nannotation tools for unrestricted German text. Our representation format is\nbased on argument structure, but also permits the extraction of other kinds of\nrepresentations. We discuss several methodological issues and the analysis of\nsome phenomena. Additional focus is on the tools developed in our project and\ntheir applications.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807009v2",
            "date": "1998-07-20 15:34:57+00:00",
            "title": "A Projection Architecture for Dependency Grammar and How it Compares to LFG",
            "authors": [
                "Norbert Broeker"
            ],
            "abstract": "This paper explores commonalities and differences between \\dachs, a variant\nof Dependency Grammar, and Lexical-Functional Grammar. \\dachs\\ is based on\ntraditional linguistic insights, but on modern mathematical tools, aiming to\nintegrate different knowledge systems (from syntax and semantics) via their\ncoupling to an abstract syntactic primitive, the dependency relation. These\nknowledge systems correspond rather closely to projections in LFG. We will\ninvestigate commonalities arising from the usage of the projection approach in\nboth theories, and point out differences due to the incompatible linguistic\npremises. The main difference to LFG lies in the motivation and status of the\ndimensions, and the information coded there. We will argue that LFG confounds\ndifferent information in one projection, preventing it to achieve a good\nseparation of alternatives and calling the motivation of the projection into\nquestion.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807010v1",
            "date": "1998-07-20 22:34:56+00:00",
            "title": "Automatically Creating Bilingual Lexicons for Machine Translation from Bilingual Text",
            "authors": [
                "Davide Turcato"
            ],
            "abstract": "A method is presented for automatically augmenting the bilingual lexicon of\nan existing Machine Translation system, by extracting bilingual entries from\naligned bilingual text. The proposed method only relies on the resources\nalready available in the MT system itself. It is based on the use of bilingual\nlexical templates to match the terminal symbols in the parses of the aligned\nsentences.",
            "journal ref": "Proceedings of COLING-ACL'98",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807011v1",
            "date": "1998-07-22 22:05:04+00:00",
            "title": "Statistical Models for Unsupervised Prepositional Phrase Attachment",
            "authors": [
                "Adwait Ratnaparkhi"
            ],
            "abstract": "We present several unsupervised statistical models for the prepositional\nphrase attachment task that approach the accuracy of the best supervised\nmethods for this task. Our unsupervised approach uses a heuristic based on\nattachment proximity and trains from raw text that is annotated with only\npart-of-speech tags and morphological base forms, as opposed to attachment\ninformation. It is therefore less resource-intensive and more portable than\nprevious corpus-based algorithms proposed for this task. We present results for\nprepositional phrase attachment in both English and Spanish.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL '98)",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807012v1",
            "date": "1998-07-31 06:50:56+00:00",
            "title": "Character design for soccer commmentary",
            "authors": [
                "Kim Binsted"
            ],
            "abstract": "In this paper we present early work on an animated talking head commentary\nsystem called {\\bf Byrne}\\footnote{David Byrne is the lead singer of the\nTalking Heads.}. The goal of this project is to develop a system which can take\nthe output from the RoboCup soccer simulator, and generate appropriate\naffective speech and facial expressions, based on the character's personality,\nemotional state, and the state of play. Here we describe a system which takes\npre-analysed simulator output as input, and which generates text marked-up for\nuse by a speech generator and a face animation system. We make heavy use of\ninter-system standards, so that future versions of Byrne will be able to take\nadvantage of advances in the technologies that it incorporates.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9807013v1",
            "date": "1998-07-31 10:30:08+00:00",
            "title": "Improving Data Driven Wordclass Tagging by System Combination",
            "authors": [
                "Hans van Halteren",
                "Jakub Zavrel",
                "Walter Daelemans"
            ],
            "abstract": "In this paper we examine how the differences in modelling between different\ndata driven systems performing the same NLP task can be exploited to yield a\nhigher accuracy than the best individual system. We do this by means of an\nexperiment involving the task of morpho-syntactic wordclass tagging. Four\nwell-known tagger generators (Hidden Markov Model, Memory-Based, Transformation\nRules and Maximum Entropy) are trained on the same corpus data. After\ncomparison, their outputs are combined using several voting strategies and\nsecond stage classifiers. All combination taggers outperform their best\ncomponent, with the best combination showing a 19.1% lower error rate than the\nbest individual tagger.",
            "journal ref": "Proceedings of the 17th International Conference on Computational\n  Linguistics (COLING-ACL'98)",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808001v1",
            "date": "1998-08-04 02:15:25+00:00",
            "title": "An Empirical Evaluation of Probabilistic Lexicalized Tree Insertion Grammars",
            "authors": [
                "Rebecca Hwa"
            ],
            "abstract": "We present an empirical study of the applicability of Probabilistic\nLexicalized Tree Insertion Grammars (PLTIG), a lexicalized counterpart to\nProbabilistic Context-Free Grammars (PCFG), to problems in stochastic\nnatural-language processing. Comparing the performance of PLTIGs with\nnon-hierarchical N-gram models and PCFGs, we show that PLTIG combines the best\naspects of both, with language modeling capability comparable to N-grams, and\nimproved parsing performance over its non-lexicalized counterpart. Furthermore,\ntraining of PLTIGs displays faster convergence than PCFGs.",
            "journal ref": "Proceedings of COLING-ACL'98",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808002v1",
            "date": "1998-08-05 14:13:08+00:00",
            "title": "Indexing with WordNet synsets can improve Text Retrieval",
            "authors": [
                "Julio Gonzalo",
                "Felisa Verdejo",
                "Irina Chugur",
                "Juan Cigarran"
            ],
            "abstract": "The classical, vector space model for text retrieval is shown to give better\nresults (up to 29% better in our experiments) if WordNet synsets are chosen as\nthe indexing space, instead of word forms. This result is obtained for a\nmanually disambiguated test collection (of queries and documents) derived from\nthe Semcor semantic concordance. The sensitivity of retrieval performance to\n(automatic) disambiguation errors when indexing documents is also measured.\nFinally, it is observed that if queries are not disambiguated, indexing by\nsynsets performs (at best) only as good as standard word indexing.",
            "journal ref": "Proceedings of the COLING/ACL'98 Workshop on Usage of WordNet for\n  NLP, Montreal, 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808003v1",
            "date": "1998-08-07 20:16:09+00:00",
            "title": "Parallel Strands: A Preliminary Investigation into Mining the Web for Bilingual Text",
            "authors": [
                "Philip Resnik"
            ],
            "abstract": "Parallel corpora are a valuable resource for machine translation, but at\npresent their availability and utility is limited by genre- and\ndomain-specificity, licensing restrictions, and the basic difficulty of\nlocating parallel texts in all but the most dominant of the world's languages.\nA parallel corpus resource not yet explored is the World Wide Web, which hosts\nan abundance of pages in parallel translation, offering a potential solution to\nsome of these problems and unique opportunities of its own. This paper presents\nthe necessary first step in that exploration: a method for automatically\nfinding parallel translated documents on the Web. The technique is conceptually\nsimple, fully language independent, and scalable, and preliminary evaluation\nresults indicate that the method may be accurate enough to apply without human\nintervention.",
            "journal ref": "Proceedings of AMTA-98",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808004v2",
            "date": "1998-08-12 09:07:46+00:00",
            "title": "Word Length Frequency and Distribution in English: Observations, Theory, and Implications for the Construction of Verse Lines",
            "authors": [
                "Hideaki Aoyama",
                "John Constable"
            ],
            "abstract": "Recent observations in the theory of verse and empirical metrics have\nsuggested that constructing a verse line involves a pattern-matching search\nthrough a source text, and that the number of found elements (complete words\ntotaling a specified number of syllables) is given by dividing the total number\nof words by the mean number of syllables per word in the source text. This\npaper makes this latter point explicit mathematically, and in the course of\nthis demonstration shows that the word length frequency totals in English\noutput are distributed geometrically (previous researchers reported an adjusted\nPoisson distribution), and that the sequential distribution is random at the\nglobal level, with significant non-randomness in the fine structure. Data from\na corpus of just under two million words, and a syllable-count lexicon of\n71,000 word-forms is reported. The pattern-matching theory is shown to be\ninternally coherent, and it is observed that some of the analytic techniques\ndescribed here form a satisfactory test for regular (isometric) lineation in a\ntext.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808005v1",
            "date": "1998-08-13 07:25:22+00:00",
            "title": "Combining Expression and Content in Domains for Dialog Managers",
            "authors": [
                "Bernd Ludwig",
                "Guenther Goerz",
                "Heinrich Niemann"
            ],
            "abstract": "We present work in progress on abstracting dialog managers from their domain\nin order to implement a dialog manager development tool which takes (among\nother data) a domain description as input and delivers a new dialog manager for\nthe described domain as output. Thereby we will focus on two topics; firstly,\nthe construction of domain descriptions with description logics and secondly,\nthe interpretation of utterances in a given domain.",
            "journal ref": "Proceedings of DL '98, pp. 126-130, Trento, Italy",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808006v1",
            "date": "1998-08-14 00:44:26+00:00",
            "title": "Isometric Lineation in English Texts: An Empirical and Mathematical Examination of its Character and Consequences",
            "authors": [
                "Hideaki Aoyama",
                "John Constable"
            ],
            "abstract": "In this paper we build on earlier observations and theory regarding word\nlength frequency and sequential distribution to develop a mathematical\ncharacterization of some of the language features distinguishing isometrically\nlineated text from unlineated text, in other words the features distinguishing\nisometrical verse from prose. It is shown that the frequency of syllables\nmaking complete words produces a flat distribution for prose, while that for\nverse exhibits peaks at the line length position and subsequent multiples of\nthat position. Data from several verse authors is presented, including a\ndetailed mathematical analysis of the dynamics underlying peak creation, and\ncomments are offered on the processes by which authors construct lines. We note\nthat the word-length sequence of prose is random, whereas lineation\nnecessitates non-random word-length sequencing, and that this has the probable\nconsequence of introducing a degree of randomness into the otherwise highly\nordered grammatical sequence. In addition we observe that this effect can be\nameliorated by a reduction in the mean word length of the text (confirming\nearlier observations that verse tends to use shorter words) and the use of\nlines varying from the core isometrical set. The frequency of variant lines is\nshown to be coincident with the frequency of polysyllables, suggesting that the\nuse of variant lines is motivated by polysyllabic word placement. The\nrestrictive effects of different line lengths, the relationship between\nmetrical restriction and poetic effect, and the general character of metrical\nrules are also discussed.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808007v2",
            "date": "1998-08-19 23:53:22+00:00",
            "title": "Some Properties of Preposition and Subordinate Conjunction Attachments",
            "authors": [
                "Alexander S. Yeh",
                "Marc B. Vilain"
            ],
            "abstract": "Determining the attachments of prepositions and subordinate conjunctions is a\nkey problem in parsing natural language. This paper presents a trainable\napproach to making these attachments through transformation sequences and\nerror-driven learning. Our approach is broad coverage, and accounts for roughly\nthree times the attachment cases that have previously been handled by\ncorpus-based techniques. In addition, our approach is based on a simplified\nmodel of syntax that is more consistent with the practice in current\nstate-of-the-art language processing systems. This paper sketches syntactic and\nalgorithmic details, and presents experimental results on data sets derived\nfrom the Penn Treebank. We obtain an attachment accuracy of 75.4% for the\ngeneral case, the first such corpus-based result to be reported. For the\nrestricted cases previously studied with corpus-based methods, our approach\nyields an accuracy comparable to current work (83.1%).",
            "journal ref": "Proceedings of COLING-ACL '98: 36th Annual Meeting of the\n  Association for Computational Linguistics and 17th International Conference\n  on Computational Linguistics, Montreal, Canada, 1998. Pages 1436-1442.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808008v1",
            "date": "1998-08-20 14:26:11+00:00",
            "title": "Deriving the Predicate-Argument Structure for a Free Word Order Language",
            "authors": [
                "Cem Bozsahin"
            ],
            "abstract": "In relatively free word order languages, grammatical functions are\nintricately related to case marking. Assuming an ordered representation of the\npredicate-argument structure, this work proposes a Combinatory Categorial\nGrammar formulation of relating surface case cues to categories and types for\ncorrectly placing the arguments in the predicate-argument structure. This is\nachieved by assigning case markers GF-encoding categories. Unlike other CG\nformulations, type shifting does not proliferate or cause spurious ambiguity.\nCategories of all argument-encoding grammatical functions follow from the same\nprinciple of category assignment. Normal order evaluation of the combinatory\nform reveals the predicate-argument structure. Application of the method to\nTurkish is shown.",
            "journal ref": "Proceedings of COLING-ACL '98, 167-173",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808009v1",
            "date": "1998-08-21 14:04:30+00:00",
            "title": "How to define a context-free backbone for DGs: Implementing a DG in the LFG formalism",
            "authors": [
                "Norbert Broeker"
            ],
            "abstract": "This paper presents a multidimensional Dependency Grammar (DG), which\ndecouples the dependency tree from word order, such that surface ordering is\nnot determined by traversing the dependency tree. We develop the notion of a\n\\emph{word order domain structure}, which is linked but structurally dissimilar\nto the syntactic dependency tree. We then discuss the implementation of such a\nDG using constructs from a unification-based phrase-structure approach, namely\nLexical-Functional Grammar (LFG). Particular attention is given to the analysis\nof discontinuities in DG in terms of LFG's functional uncertainty.",
            "journal ref": "Proc. COLING-ACL'98 Workshop ``Processing of Dependency-Based\n  Grammars'', pp 29-38",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808010v1",
            "date": "1998-08-21 15:26:44+00:00",
            "title": "Letter to Sound Rules for Accented Lexicon Compression",
            "authors": [
                "V. Pagel",
                "K. Lenzo",
                "A. Black"
            ],
            "abstract": "This paper presents trainable methods for generating letter to sound rules\nfrom a given lexicon for use in pronouncing out-of-vocabulary words and as a\nmethod for lexicon compression.\n  As the relationship between a string of letters and a string of phonemes\nrepresenting its pronunciation for many languages is not trivial, we discuss\ntwo alignment procedures, one fully automatic and one hand-seeded which produce\nreasonable alignments of letters to phones.\n  Top Down Induction Tree models are trained on the aligned entries. We show\nhow combined phoneme/stress prediction is better than separate prediction\nprocesses, and still better when including in the model the last phonemes\ntranscribed and part of speech information. For the lexicons we have tested,\nour models have a word accuracy (including stress) of 78% for OALD, 62% for CMU\nand 94% for BRULEX. The extremely high scores on the training sets allow\nsubstantial size reductions (more than 1/20).\n  WWW site: http://tcts.fpms.ac.be/synthesis/mbrdico",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808011v1",
            "date": "1998-08-23 20:14:21+00:00",
            "title": "Primitive Part-of-Speech Tagging using Word Length and Sentential Structure",
            "authors": [
                "Simon Cozens"
            ],
            "abstract": "It has been argued that, when learning a first language, babies use a series\nof small clues to aid recognition and comprehension, and that one of these\nclues is word length. In this paper we present a statistical part of speech\ntagger which trains itself solely on the number of letters in each word in a\nsentence.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808012v1",
            "date": "1998-08-25 08:06:45+00:00",
            "title": "Separating Surface Order and Syntactic Relations in a Dependency Grammar",
            "authors": [
                "Norbert Broeker"
            ],
            "abstract": "This paper proposes decoupling the dependency tree from word order, such that\nsurface ordering is not determined by traversing the dependency tree. We\ndevelop the notion of a \\emph{word order domain structure}, which is linked but\nstructurally dissimilar to the syntactic dependency tree. The proposal results\nin a lexicalized, declarative, and formally precise description of word order;\nfeatures which lack previous proposals for dependency grammars. Contrary to\nother lexicalized approaches to word order, our proposal does not require\nlexical ambiguities for ordering alternatives.",
            "journal ref": "Proc. COLING-ACL'98, pp 174-180",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808013v1",
            "date": "1998-08-25 16:52:31+00:00",
            "title": "Partial Evaluation for Efficient Access to Inheritance Lexicons",
            "authors": [
                "Sven Hartrumpf"
            ],
            "abstract": "Multiple default inheritance formalisms for lexicons have attracted much\ninterest in recent years. I propose a new efficient method to access such\nlexicons. After showing two basic strategies for lookup in inheritance\nlexicons, a compromise is developed which combines to a large degree (from a\npractical point of view) the advantages of both strategies and avoids their\ndisadvantages. The method is a kind of (off-line) partial evaluation that makes\na subset of inherited information explicit before using the lexicon. I identify\nthe parts of a lexicon which should be evaluated, and show how partial\nevaluation works for inheritance lexicons. Finally, the theoretical results are\nconfirmed by a complete implementation. Speedups by a factor of 10-100 are\nreached.",
            "journal ref": "Proceedings of the 2nd International Conference on Recent Advances\n  in Natural Language Processing (RANLP-97), pp. 43-50, Tzigov Chark, Bulgaria,\n  September 1997.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808014v1",
            "date": "1998-08-26 11:12:30+00:00",
            "title": "Spotting Prosodic Boundaries in Continuous Speech in French",
            "authors": [
                "V. Pagel",
                "N. Carbonell",
                "Y. Laprie",
                "J. Vaissiere"
            ],
            "abstract": "A radio speech corpus of 9mn has been prosodically marked by a phonetician\nexpert, and non expert listeners. this corpus is large enough to train and test\nan automatic boundary spotting system, namely a time delay neural network fed\nwith F0 values, vowels and pseudo-syllable durations. Results validate both\nprosodic marking and automatic spotting of prosodic events.",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808015v1",
            "date": "1998-08-26 17:39:07+00:00",
            "title": "Error-Driven Pruning of Treebank Grammars for Base Noun Phrase Identification",
            "authors": [
                "Claire Cardie",
                "David Pierce"
            ],
            "abstract": "Finding simple, non-recursive, base noun phrases is an important subtask for\nmany natural language processing applications. While previous empirical methods\nfor base NP identification have been rather complex, this paper instead\nproposes a very simple algorithm that is tailored to the relative simplicity of\nthe task. In particular, we present a corpus-based approach for finding base\nNPs by matching part-of-speech tag sequences. The training phase of the\nalgorithm is based on two successful techniques: first the base NP grammar is\nread from a ``treebank'' corpus; then the grammar is improved by selecting\nrules with high ``benefit'' scores. Using this simple algorithm with a naive\nheuristic for matching rules, we achieve surprising accuracy in an evaluation\non the Penn Treebank Wall Street Journal.",
            "journal ref": "Proceedings of COLING-ACL'98, pages 218-224.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808016v1",
            "date": "1998-08-27 19:59:49+00:00",
            "title": "Segregatory Coordination and Ellipsis in Text Generation",
            "authors": [
                "James Shaw"
            ],
            "abstract": "In this paper, we provide an account of how to generate sentences with\ncoordination constructions from clause-sized semantic representations. An\nalgorithm is developed to generate sentences with ellipsis, gapping,\nright-node-raising, and non-constituent coordination constructions. Various\nexamples from linguistic literature will be used to demonstrate that the\nalgorithm does its job well.",
            "journal ref": "COLING-ACL'98, pp. 1220-1226",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9808017v1",
            "date": "1998-08-31 11:59:40+00:00",
            "title": "A Variant of Earley Parsing",
            "authors": [
                "Mark-Jan Nederhof",
                "Giorgio Satta"
            ],
            "abstract": "The Earley algorithm is a widely used parsing method in natural language\nprocessing applications. We introduce a variant of Earley parsing that is based\non a ``delayed'' recognition of constituents. This allows us to start the\nrecognition of a constituent only in cases in which all of its subconstituents\nhave been found within the input string. This is particularly advantageous in\nseveral cases in which partial analysis of a constituent cannot be completed\nand in general in all cases of productions sharing some suffix of their\nright-hand sides (even for different left-hand side nonterminals). Although the\ntwo algorithms result in the same asymptotic time and space complexity, from a\npractical perspective our algorithm improves the time and space requirements of\nthe original method, as shown by reported experimental results.",
            "journal ref": "AI*IA 97: Advances in Artificial Intelligence, 5th Congress of the\n  Italian Association for Artificial Intelligence, LNAI 1321, Springer Verlag,\n  pages 84-95, 1997.",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9809001v2",
            "date": "1998-09-01 18:26:06+00:00",
            "title": "Towards an implementable dependency grammar",
            "authors": [
                "Timo Jarvinen",
                "Pasi Tapanainen"
            ],
            "abstract": "The aim of this paper is to define a dependency grammar framework which is\nboth linguistically motivated and computationally parsable. See the demo at\nhttp://www.conexor.fi/analysers.html#testing",
            "journal ref": "in CoLing-ACL'98 workshop 'Processing of Dependency-Based\n  Grammars', Kahane and Polguere (eds), p. 1-10, Montreal, Canada, 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9809002v1",
            "date": "1998-09-09 14:49:01+00:00",
            "title": "Some Ontological Principles for Designing Upper Level Lexical Resources",
            "authors": [
                "Nicola Guarino"
            ],
            "abstract": "The purpose of this paper is to explore some semantic problems related to the\nuse of linguistic ontologies in information systems, and to suggest some\norganizing principles aimed to solve such problems. The taxonomic structure of\ncurrent ontologies is unfortunately quite complicated and hard to understand,\nespecially for what concerns the upper levels. I will focus here on the problem\nof ISA overloading, which I believe is the main responsible of these\ndifficulties. To this purpose, I will carefully analyze the ontological nature\nof the categories used in current upper-level structures, considering the\nnecessity of splitting them according to more subtle distinctions or the\nopportunity of excluding them because of their limited organizational role.",
            "journal ref": "in Proc. of First International Conference on Language Resources\n  and Evaluation, Rubio, Gallardo, Castro, and Tejada (eds.), p. 527-534,\n  Granada, Spain, 1998",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cmp-lg/9809003v1",
            "date": "1998-09-14 15:46:40+00:00",
            "title": "A Comparison of WordNet and Roget's Taxonomy for Measuring Semantic Similarity",
            "authors": [
                "Michael Mc Hale"
            ],
            "abstract": "This paper presents the results of using Roget's International Thesaurus as\nthe taxonomy in a semantic similarity measurement task. Four similarity metrics\nwere taken from the literature and applied to Roget's The experimental\nevaluation suggests that the traditional edge counting approach does\nsurprisingly well (a correlation of r=0.88 with a benchmark set of human\nsimilarity judgements, with an upper bound of r=0.90 for human subjects\nperforming the same task.)",
            "journal ref": "None",
            "category": "cmp-lg"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809021v1",
            "date": "1998-09-16 14:22:35+00:00",
            "title": "Producing NLP-based On-line Contentware",
            "authors": [
                "Francis Wolinski",
                "Frantz Vichot",
                "Olivier Gremont"
            ],
            "abstract": "For its internal needs as well as for commercial purposes, CDC Group has\nproduced several NLP-based on-line contentware applications for years. The\ndevelopment process of such applications is subject to numerous constraints\nsuch as quality of service, integration of new advances in NLP, direct\nreactions from users, continuous versioning, short delivery deadlines and cost\ncontrol. Following this industrial and commercial experience, malleability of\nthe applications, their openness towards foreign components, efficiency of\napplications and their ease of exploitation have appeared to be key points. In\nthis paper, we describe TalLab, a powerful architecture for on-line contentware\nwhich fulfils these requirements.",
            "journal ref": "Natural Language Processing & Industrial Applications, Moncton,\n  NB, Canada, Aug. 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809051v1",
            "date": "1998-09-23 14:47:49+00:00",
            "title": "Spoken Language Dialogue Systems and Components: Best practice in development and evaluation (DISC 24823) - Periodic Progress Report 1: Basic Details of the Action",
            "authors": [
                "Niels Ole Bernsen",
                "Laila Dybkjaer",
                "eds."
            ],
            "abstract": "The DISC project aims to (a) build an in-depth understanding of the\nstate-of-the-art in spoken language dialogue systems (SLDSs) and components\ndevelopment and evaluation with the purpose of (b) developing a first best\npractice methodology in the field. The methodology will be accompanied by (c) a\nseries of development and evaluation support tools. To the limited extent\npossible within the duration of the project, the draft versions of the\nmethodology and the tools will be (d) tested by SLDS developers from industry\nand research, and will be (e) packaged to best suit their needs. In the first\nyear of DISC, (a) has been accomplished, and (b) and (c) have started. A\nproposal to complete the work proposed above by adding 12 months to the 18\nmonths of the present project, has been submitted to Esprit Long-Term Research\nin March 1998.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9810012v4",
            "date": "1998-10-13 10:24:56+00:00",
            "title": "Ultrametric Distance in Syntax",
            "authors": [
                "Mark D. Roberts"
            ],
            "abstract": "Phrase structure trees have a hierarchical structure. In many subjects, most\nnotably in Taxonomy such tree structures have been studied using ultrametrics.\nHere syntactical hierarchical phrase trees are subject to a similar analysis,\nwhich is much siompler as the branching structure is more readily discernible\nand switched. The occurence of hierarchical structure elsewhere in linguistics\nis mentioned. The phrase tree can be represented by a matrix and the elements\nof the matrix can be represented by triangles. The height at which branching\noccurs is not prescribed in previous syntatic models, but it is by using the\nultrametric matrix. The ambiguity of which branching height to choose is\nresolved by postulating that branching occurs at the lowest height available.\nAn ultrametric produces a measure of the complexity of sentences: presumably\nthe complexity of sentence increases as a language is aquired so that this can\nbe tested. A All ultrametric triangles are equilateral or isocles, here it is\nshown that X structur implies that there are no equilateral triangles.\nRestricting attention to simple syntax a minium ultrametric distance between\nlexical categories is calculatex. This ultrametric distance is shown to be\ndifferent than the matrix obtasined from feaures. It is shown that the\ndefinition of c-commabnd can be replaced by an equivalent ultrametric\ndefinition. The new definition invokes a minimum distance between nodes and\nthis is more aesthetically satisfing than previouv varieties of definitions.\n  From the new definition of c-command follows a new definition of government.",
            "journal ref": "Prague Bulletin of Mathematical Linguistics 103 (2015) 111-130",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811003v1",
            "date": "1998-10-31 19:33:50+00:00",
            "title": "A Winnow-Based Approach to Context-Sensitive Spelling Correction",
            "authors": [
                "Andrew R. Golding",
                "Dan Roth"
            ],
            "abstract": "A large class of machine-learning problems in natural language require the\ncharacterization of linguistic context. Two characteristic properties of such\nproblems are that their feature space is of very high dimensionality, and their\ntarget concepts refer to only a small subset of the features in the space.\nUnder such conditions, multiplicative weight-update algorithms such as Winnow\nhave been shown to have exceptionally good theoretical properties. We present\nan algorithm combining variants of Winnow and weighted-majority voting, and\napply it to a problem in the aforementioned class: context-sensitive spelling\ncorrection. This is the task of fixing spelling errors that happen to result in\nvalid words, such as substituting \"to\" for \"too\", \"casual\" for \"causal\", etc.\nWe evaluate our algorithm, WinSpell, by comparing it against BaySpell, a\nstatistics-based method representing the state of the art for this task. We\nfind: (1) When run with a full (unpruned) set of features, WinSpell achieves\naccuracies significantly higher than BaySpell was able to achieve in either the\npruned or unpruned condition; (2) When compared with other systems in the\nliterature, WinSpell exhibits the highest performance; (3) The primary reason\nthat WinSpell outperforms BaySpell is that WinSpell learns a better linear\nseparator; (4) When run on a test set drawn from a different corpus than the\ntraining set was drawn from, WinSpell is better able than BaySpell to adapt,\nusing a strategy we will present that combines supervised learning on the\ntraining set with unsupervised learning on the (noisy) test set.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811004v4",
            "date": "1998-11-01 11:18:22+00:00",
            "title": "Does Meaning Evolve?",
            "authors": [
                "Mark D. Roberts"
            ],
            "abstract": "A common method of making a theory more understandable, is by comparing it to\nanother theory which has been better developed. Radical interpretation is a\ntheory which attempts to explain how communication has meaning. Radical\ninterpretation is treated as another time-dependent theory and compared to the\ntime dependent theory of biological evolution. The main reason for doing this\nis to find the nature of the time dependence; producing analogs between the two\ntheories is a necessary prerequisite to this and brings up many problems. Once\nthe nature of the time dependence is better known it might allow the underlying\nmechanism to be uncovered. Several similarities and differences are uncovered,\nthere appear to be more differences than similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811006v1",
            "date": "1998-11-02 18:57:23+00:00",
            "title": "Machine Learning of Generic and User-Focused Summarization",
            "authors": [
                "Inderjeet Mani",
                "Eric Bloedorn"
            ],
            "abstract": "A key problem in text summarization is finding a salience function which\ndetermines what information in the source should be included in the summary.\nThis paper describes the use of machine learning on a training corpus of\ndocuments and their abstracts to discover salience functions which describe\nwhat combination of features is optimal for a given summarization task. The\nmethod addresses both \"generic\" and user-focused summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811010v1",
            "date": "1998-11-03 21:14:32+00:00",
            "title": "Learning to Resolve Natural Language Ambiguities: A Unified Approach",
            "authors": [
                "Dan Roth"
            ],
            "abstract": "We analyze a few of the commonly used statistics based and machine learning\nalgorithms for natural language disambiguation tasks and observe that they can\nbe re-cast as learning linear separators in the feature space. Each of the\nmethods makes a priori assumptions, which it employs, given the data, when\nsearching for its hypothesis. Nevertheless, as we show, it searches a space\nthat is as rich as the space of all linear separators. We use this to build an\nargument for a data driven approach which merely searches for a good linear\nseparator in the feature space, without further assumptions on the domain or a\nspecific problem.\n  We present such an approach - a sparse network of linear separators,\nutilizing the Winnow learning algorithm - and show how to use it in a variety\nof ambiguity resolution problems. The learning approach presented is\nattribute-efficient and, therefore, appropriate for domains having very large\nnumber of attributes.\n  In particular, we present an extensive experimental comparison of our\napproach with other methods on several well studied lexical disambiguation\ntasks such as context-sensitive spelling correction, prepositional phrase\nattachment and part of speech tagging. In all cases we show that our approach\neither outperforms other methods tried for these tasks or performs comparably\nto the best.",
            "journal ref": "Proceedings of of AAAI'98 pp. 806--813",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9811018v3",
            "date": "1998-11-11 14:17:09+00:00",
            "title": "P-model Alternative to the T-model",
            "authors": [
                "Mark D. Roberts"
            ],
            "abstract": "Standard linguistic analysis of syntax uses the T-model. This model requires\nthe ordering: D-structure $>$ S-structure $>$ LF. Between each of these\nrepresentations there is movement which alters the order of the constituent\nwords; movement is achieved using the principles and parameters of syntactic\ntheory. Psychological serial models do not accommodate the T-model immediately\nso that here a new model called the P-model is introduced. Here it is argued\nthat the LF representation should be replaced by a variant of Frege's three\nqualities. In the F-representation the order of elements is not necessarily the\nsame as that in LF and it is suggested that the correct ordering is:\nF-representation $>$ D-structure $>$ S-structure. Within this framework\nmovement originates as the outcome of emphasis applied to the sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9812021v1",
            "date": "1998-12-22 16:33:19+00:00",
            "title": "Forgetting Exceptions is Harmful in Language Learning",
            "authors": [
                "Walter Daelemans",
                "Antal van den Bosch",
                "Jakub Zavrel"
            ],
            "abstract": "We show that in language learning, contrary to received wisdom, keeping\nexceptional training instances in memory can be beneficial for generalization\naccuracy. We investigate this phenomenon empirically on a selection of\nbenchmark natural language processing tasks: grapheme-to-phoneme conversion,\npart-of-speech tagging, prepositional-phrase attachment, and base noun phrase\nchunking. In a first series of experiments we combine memory-based learning\nwith training set editing techniques, in which instances are edited based on\ntheir typicality and class prediction strength. Results show that editing\nexceptional instances (with low typicality or low class prediction strength)\ntends to harm generalization accuracy. In a second series of experiments we\ncompare memory-based learning and decision-tree learning methods on the same\nselection of tasks, and find that decision-tree learning often performs worse\nthan memory-based learning. Moreover, the decrease in performance can be linked\nto the degree of abstraction from exceptions (i.e., pruning or eagerness). We\nprovide explanations for both results in terms of the properties of the natural\nlanguage processing tasks and the learning algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9809110v1",
            "date": "1998-09-27 18:42:51+00:00",
            "title": "Similarity-Based Models of Word Cooccurrence Probabilities",
            "authors": [
                "Ido Dagan",
                "Lillian Lee",
                "Fernando C. N. Pereira"
            ],
            "abstract": "In many applications of natural language processing (NLP) it is necessary to\ndetermine the likelihood of a given word combination. For example, a speech\nrecognizer may need to determine which of the two word combinations ``eat a\npeach'' and ``eat a beach'' is more likely. Statistical NLP methods determine\nthe likelihood of a word combination from its frequency in a training corpus.\nHowever, the nature of language is such that many word combinations are\ninfrequent and do not occur in any given corpus. In this work we propose a\nmethod for estimating the probability of such previously unseen word\ncombinations using available information on ``most similar'' words.\n  We describe probabilistic word association models based on distributional\nword similarity, and apply them to two tasks, language modeling and pseudo-word\ndisambiguation. In the language modeling task, a similarity-based model is used\nto improve probability estimates for unseen bigrams in a back-off language\nmodel. The similarity-based method yields a 20% perplexity improvement in the\nprediction of unseen bigrams and statistically significant reductions in\nspeech-recognition error.\n  We also compare four similarity-based estimation methods against back-off and\nmaximum-likelihood estimation methods on a pseudo-word sense disambiguation\ntask in which we controlled for both unigram and bigram frequency to avoid\ngiving too much weight to easy-to-disambiguate high-frequency configurations.\nThe similarity-based methods perform up to 40% better on this particular task.",
            "journal ref": "Machine Learning, 34, 43-69 (1999)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9812004v1",
            "date": "1998-12-04 12:28:19+00:00",
            "title": "Name Strategy: Its Existence and Implications",
            "authors": [
                "Mark D. Roberts"
            ],
            "abstract": "It is argued that colour name strategy, object name strategy, and chunking\nstrategy in memory are all aspects of the same general phenomena, called\nstereotyping. It is pointed out that the Berlin-Kay universal partial ordering\nof colours and the frequency of traffic accidents classified by colour are\nsurprisingly similar. Some consequences of the existence of a name strategy for\nthe philosophy of language and mathematics are discussed. It is argued that\nreal valued quantities occur {\\it ab initio}. The implication of real valued\ntruth quantities is that the {\\bf Continuum Hypothesis} of pure mathematics is\nside-stepped. The existence of name strategy shows that thought/sememes and\ntalk/phonemes can be separate, and this vindicates the assumption of thought\noccurring before talk used in psycholinguistic speech production models.",
            "journal ref": "Int.J.Computational Cognition Volume 3 Pages 1-14 (2005).",
            "category": "cs.CL"
        }
    ],
    "1999": [
        {
            "url": "http://arxiv.org/abs/cs/9901005v1",
            "date": "1999-01-13 17:37:00+00:00",
            "title": "An Empirical Approach to Temporal Reference Resolution (journal version)",
            "authors": [
                "Janyce Wiebe",
                "Thomas P. O'Hara",
                "Thorsten Ohrstrom-Sandgren",
                "Kenneth K. McKeever"
            ],
            "abstract": "Scheduling dialogs, during which people negotiate the times of appointments,\nare common in everyday life. This paper reports the results of an in-depth\nempirical investigation of resolving explicit temporal references in scheduling\ndialogs. There are four phases of this work: data annotation and evaluation,\nmodel development, system implementation and evaluation, and model evaluation\nand analysis. The system and model were developed primarily on one set of data,\nand then applied later to a much more complex data set, to assess the\ngeneralizability of the model for the task being performed. Many different\ntypes of empirical methods are applied to pinpoint the strengths and weaknesses\nof the approach. Detailed annotation instructions were developed and an\nintercoder reliability study was performed, showing that naive annotators can\nreliably perform the targeted annotations. A fully automatic system has been\ndeveloped and evaluated on unseen test data, with good results on both data\nsets. We adopt a pure realization of a recency-based focus model to identify\nprecisely when it is and is not adequate for the task being addressed. In\naddition to system results, an in-depth evaluation of the model itself is\npresented, based on detailed manual annotations. The results are that few\nerrors occur specifically due to the model of focus being used, and the set of\nanaphoric relations defined in the model are low in ambiguity for both data\nsets.",
            "journal ref": "Journal of Artificial Intelligence Research (JAIR), 9:247-293",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9902001v1",
            "date": "1999-01-31 18:57:45+00:00",
            "title": "Compacting the Penn Treebank Grammar",
            "authors": [
                "Alexander Krotov",
                "Mark Hepple",
                "Robert Gaizauskas",
                "Yorick Wilks"
            ],
            "abstract": "Treebanks, such as the Penn Treebank (PTB), offer a simple approach to\nobtaining a broad coverage grammar: one can simply read the grammar off the\nparse trees in the treebank. While such a grammar is easy to obtain, a\nsquare-root rate of growth of the rule set with corpus size suggests that the\nderived grammar is far from complete and that much more treebanked text would\nbe required to obtain a complete grammar, if one exists at some limit. However,\nwe offer an alternative explanation in terms of the underspecification of\nstructures within the treebank. This hypothesis is explored by applying an\nalgorithm to compact the derived grammar by eliminating redundant rules --\nrules whose right hand sides can be parsed by other rules. The size of the\nresulting compacted grammar, which is significantly less than that of the full\ntreebank grammar, is shown to approach a limit. However, such a compacted\ngrammar does not yield very good performance figures. A version of the\ncompaction algorithm taking rule probabilities into account is proposed, which\nis argued to be more linguistically motivated. Combined with simple\nthresholding, this method can be used to give a 58% reduction in grammar size\nwithout significant change in parsing performance, and can produce a 69%\nreduction with some gain in recall, but a loss in precision.",
            "journal ref": "In Proceedings of COLING-98 (Montreal), pages 699-703",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9902029v1",
            "date": "1999-02-25 14:41:24+00:00",
            "title": "The \"Fodor\"-FODOR fallacy bites back",
            "authors": [
                "Yorick Wilks"
            ],
            "abstract": "The paper argues that Fodor and Lepore are misguided in their attack on\nPustejovsky's Generative Lexicon, largely because their argument rests on a\ntraditional, but implausible and discredited, view of the lexicon on which it\nis effectively empty of content, a view that stands in the long line of\nexplaining word meaning (a) by ostension and then (b) explaining it by means of\na vacuous symbol in a lexicon, often the word itself after typographic\ntransmogrification. (a) and (b) both share the wrong belief that to a word must\ncorrespond a simple entity that is its meaning. I then turn to the semantic\nrules that Pustejovsky uses and argue first that, although they have novel\nfeatures, they are in a well-established Artificial Intelligence tradition of\nexplaining meaning by reference to structures that mention other structures\nassigned to words that may occur in close proximity to the first. It is argued\nthat Fodor and Lepore's view that there cannot be such rules is without\nfoundation, and indeed systems using such rules have proved their practical\nworth in computational systems. Their justification descends from line of\nargument, whose high points were probably Wittgenstein and Quine that meaning\nis not to be understood by simple links to the world, ostensive or otherwise,\nbut by the relationship of whole cultural representational structures to each\nother and to the world as a whole.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9902030v1",
            "date": "1999-02-25 14:41:32+00:00",
            "title": "Is Word Sense Disambiguation just one more NLP task?",
            "authors": [
                "Yorick Wilks"
            ],
            "abstract": "This paper compares the tasks of part-of-speech (POS) tagging and\nword-sense-tagging or disambiguation (WSD), and argues that the tasks are not\nrelated by fineness of grain or anything like that, but are quite different\nkinds of task, particularly becuase there is nothing in POS corresponding to\nsense novelty. The paper also argues for the reintegration of sub-tasks that\nare being separated for evaluation",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9903003v1",
            "date": "1999-03-02 12:30:55+00:00",
            "title": "A Formal Framework for Linguistic Annotation",
            "authors": [
                "Steven Bird",
                "Mark Liberman"
            ],
            "abstract": "`Linguistic annotation' covers any descriptive or analytic notations applied\nto raw language data. The basic data may be in the form of time functions --\naudio, video and/or physiological recordings -- or it may be textual. The added\nnotations may include transcriptions of all sorts (from phonetic features to\ndiscourse structures), part-of-speech and sense tagging, syntactic analysis,\n`named entity' identification, co-reference annotation, and so on. While there\nare several ongoing efforts to provide formats and tools for such annotations\nand to publish annotated linguistic databases, the lack of widely accepted\nstandards is becoming a critical problem. Proposed standards, to the extent\nthey exist, have focussed on file formats. This paper focuses instead on the\nlogical structure of linguistic annotations. We survey a wide variety of\nexisting annotation formats and demonstrate a common conceptual core, the\nannotation graph. This provides a formal framework for constructing,\nmaintaining and searching linguistic annotations, while remaining consistent\nwith many alternative data structures and file formats.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9903008v1",
            "date": "1999-03-05 22:03:13+00:00",
            "title": "Empirically Evaluating an Adaptable Spoken Dialogue System",
            "authors": [
                "Diane J. Litman",
                "Shimei Pan"
            ],
            "abstract": "Recent technological advances have made it possible to build real-time,\ninteractive spoken dialogue systems for a wide variety of applications.\nHowever, when users do not respect the limitations of such systems, performance\ntypically degrades. Although users differ with respect to their knowledge of\nsystem limitations, and although different dialogue strategies make system\nlimitations more apparent to users, most current systems do not try to improve\nperformance by adapting dialogue behavior to individual users. This paper\npresents an empirical evaluation of TOOT, an adaptable spoken dialogue system\nfor retrieving train schedules on the web. We conduct an experiment in which 20\nusers carry out 4 tasks with both adaptable and non-adaptable versions of TOOT,\nresulting in a corpus of 80 dialogues. The values for a wide range of\nevaluation measures are then extracted from this corpus. Our results show that\nadaptable TOOT generally outperforms non-adaptable TOOT, and that the utility\nof adaptation depends on TOOT's initial dialogue strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9904008v1",
            "date": "1999-04-15 14:00:41+00:00",
            "title": "Transducers from Rewrite Rules with Backreferences",
            "authors": [
                "Dale Gerdemann",
                "Gertjan van Noord"
            ],
            "abstract": "Context sensitive rewrite rules have been widely used in several areas of\nnatural language processing, including syntax, morphology, phonology and speech\nprocessing. Kaplan and Kay, Karttunen, and Mohri & Sproat have given various\nalgorithms to compile such rewrite rules into finite-state transducers. The\npresent paper extends this work by allowing a limited form of backreferencing\nin such rules. The explicit use of backreferencing leads to more elegant and\ngeneral solutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9904009v1",
            "date": "1999-04-15 16:03:27+00:00",
            "title": "An ascription-based approach to speech acts",
            "authors": [
                "Mark Lee",
                "Yorick Wilks"
            ],
            "abstract": "The two principal areas of natural language processing research in pragmatics\nare belief modelling and speech act processing. Belief modelling is the\ndevelopment of techniques to represent the mental attitudes of a dialogue\nparticipant. The latter approach, speech act processing, based on speech act\ntheory, involves viewing dialogue in planning terms. Utterances in a dialogue\nare modelled as steps in a plan where understanding an utterance involves\nderiving the complete plan a speaker is attempting to achieve. However,\nprevious speech act based approaches have been limited by a reliance upon\nrelatively simplistic belief modelling techniques and their relationship to\nplanning and plan recognition. In particular, such techniques assume\nprecomputed nested belief structures. In this paper, we will present an\napproach to speech act processing based on novel belief modelling techniques\nwhere nested beliefs are propagated on demand.",
            "journal ref": "Proceedings of COLING`96, Copenhagen. (1996)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9904018v1",
            "date": "1999-04-24 23:45:26+00:00",
            "title": "A Computational Memory and Processing Model for Processing",
            "authors": [
                "Janet E. Cahn"
            ],
            "abstract": "This paper links prosody to the information in a text and how it is processed\nby the speaker. It describes the operation and output of LOQ, a text-to-speech\nimplementation that includes a model of limited attention and working memory.\nAttentional limitations are key. Varying the attentional parameter in the\nsimulations varies in turn what counts as given and new in a text, and\ntherefore, the intonational contours with which it is uttered. Currently, the\nsystem produces prosody in three different styles: child-like, adult\nexpressive, and knowledgeable. This prosody also exhibits differences within\neach style -- no two simulations are alike. The limited resource approach\ncaptures some of the stylistic and individual variety found in natural prosody.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9905001v1",
            "date": "1999-05-02 20:48:21+00:00",
            "title": "Supervised Grammar Induction Using Training Data with Limited Constituent Information",
            "authors": [
                "Rebecca Hwa"
            ],
            "abstract": "Corpus-based grammar induction generally relies on hand-parsed training data\nto learn the structure of the language. Unfortunately, the cost of building\nlarge annotated corpora is prohibitively expensive. This work aims to improve\nthe induction strategy when there are few labels in the training data. We show\nthat the most informative linguistic constituents are the higher nodes in the\nparse trees, typically denoting complex noun phrases and sentential clauses.\nThey account for only 20% of all constituents. For inducing grammars from\nsparsely labeled training data (e.g., only higher-level constituent labels), we\npropose an adaptation strategy, which produces grammars that parse almost as\nwell as grammars induced from fully labeled corpora. Our results suggest that\nfor a partial parser to replace human annotators, it must be able to\nautomatically extract higher-level constituents rather than base noun phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906003v1",
            "date": "1999-06-02 12:03:14+00:00",
            "title": "The syntactic processing of particles in Japanese spoken language",
            "authors": [
                "Melanie Siegel"
            ],
            "abstract": "Particles fullfill several distinct central roles in the Japanese language.\nThey can mark arguments as well as adjuncts, can be functional or have semantic\nfuntions. There is, however, no straightforward matching from particles to\nfunctions, as, e.g., GA can mark the subject, the object or an adjunct of a\nsentence. Particles can cooccur. Verbal arguments that could be identified by\nparticles can be eliminated in the Japanese sentence. And finally, in spoken\nlanguage particles are often omitted. A proper treatment of particles is thus\nnecessary to make an analysis of Japanese sentences possible. Our treatment is\nbased on an empirical investigation of 800 dialogues. We set up a type\nhierarchy of particles motivated by their subcategorizational and\nmodificational behaviour. This type hierarchy is part of the Japanese syntax in\nVERBMOBIL.",
            "journal ref": "Proceedings of the 13th Pacific Asia Conference on Language,\n  Information and Computation. 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906009v1",
            "date": "1999-06-06 17:36:34+00:00",
            "title": "Cascaded Markov Models",
            "authors": [
                "Thorsten Brants"
            ],
            "abstract": "This paper presents a new approach to partial parsing of context-free\nstructures. The approach is based on Markov Models. Each layer of the resulting\nstructure is represented by its own Markov Model, and output of a lower layer\nis passed as input to the next higher layer. An empirical evaluation of the\nmethod yields very good results for NP/PP chunking of German newspaper texts.",
            "journal ref": "Proceedings of EACL-99, Bergen, Norway",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906014v1",
            "date": "1999-06-14 10:06:31+00:00",
            "title": "Evaluation of the NLP Components of the OVIS2 Spoken Dialogue System",
            "authors": [
                "Gert Veldhuijzen van Zanten",
                "Gosse Bouma",
                "Khalil Sima'an",
                "Gertjan van Noord",
                "Remko Bonnema"
            ],
            "abstract": "The NWO Priority Programme Language and Speech Technology is a 5-year\nresearch programme aiming at the development of spoken language information\nsystems. In the Programme, two alternative natural language processing (NLP)\nmodules are developed in parallel: a grammar-based (conventional, rule-based)\nmodule and a data-oriented (memory-based, stochastic, DOP) module. In order to\ncompare the NLP modules, a formal evaluation has been carried out three years\nafter the start of the Programme. This paper describes the evaluation procedure\nand the evaluation results. The grammar-based component performs much better\nthan the data-oriented one in this comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906015v1",
            "date": "1999-06-14 22:06:24+00:00",
            "title": "Learning Transformation Rules to Find Grammatical Relations",
            "authors": [
                "Lisa Ferro",
                "Marc Vilain",
                "Alexander Yeh"
            ],
            "abstract": "Grammatical relationships are an important level of natural language\nprocessing. We present a trainable approach to find these relationships through\ntransformation sequences and error-driven learning. Our approach finds\ngrammatical relationships between core syntax groups and bypasses much of the\nparsing phase. On our training and test set, our procedure achieves 63.6%\nrecall and 77.3% precision (f-score = 69.8).",
            "journal ref": "Computational Natural Language Learning (CoNLL-99), pages 43-52,\n  June, 1999. Bergen, Norway",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906020v1",
            "date": "1999-06-22 08:28:26+00:00",
            "title": "Temporal Meaning Representations in a Natural Language Front-End",
            "authors": [
                "I. Androutsopoulos"
            ],
            "abstract": "Previous work in the context of natural language querying of temporal\ndatabases has established a method to map automatically from a large subset of\nEnglish time-related questions to suitable expressions of a temporal logic-like\nlanguage, called TOP. An algorithm to translate from TOP to the TSQL2 temporal\ndatabase language has also been defined. This paper shows how TOP expressions\ncould be translated into a simpler logic-like language, called BOT. BOT is very\nclose to traditional first-order predicate logic (FOPL), and hence existing\nmethods to manipulate FOPL expressions can be exploited to interface to\ntime-sensitive applications other than TSQL2 databases, maintaining the\nexisting English-to-TOP mapping.",
            "journal ref": "In Gergatsoulis, M. and Rondogiannis, P. (Eds.), Intensional\n  Programming II (Proceedings of the 12th International Symposium on Languages\n  for Intensional Programming, Athens, Greece, 1999), pp. 197-213, World\n  Scientific, 2000.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906025v1",
            "date": "1999-06-24 16:56:45+00:00",
            "title": "Mapping Multilingual Hierarchies Using Relaxation Labeling",
            "authors": [
                "J. Daude",
                "L. Padro",
                "G. Rigau"
            ],
            "abstract": "This paper explores the automatic construction of a multilingual Lexical\nKnowledge Base from pre-existing lexical resources. We present a new and robust\napproach for linking already existing lexical/semantic hierarchies. We used a\nconstraint satisfaction algorithm (relaxation labeling) to select --among all\nthe candidate translations proposed by a bilingual dictionary-- the right\nEnglish WordNet synset for each sense in a taxonomy automatically derived from\na Spanish monolingual dictionary. Although on average, there are 15 possible\nWordNet connections for each sense in the taxonomy, the method achieves an\naccuracy over 80%. Finally, we also propose several ways in which this\ntechnique could be applied to enrich and improve existing lexical databases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906026v1",
            "date": "1999-06-25 08:16:23+00:00",
            "title": "Robust Grammatical Analysis for Spoken Dialogue Systems",
            "authors": [
                "Gertjan van Noord",
                "Gosse Bouma",
                "Rob Koeling",
                "Mark-Jan Nederhof"
            ],
            "abstract": "We argue that grammatical analysis is a viable alternative to concept\nspotting for processing spoken input in a practical spoken dialogue system. We\ndiscuss the structure of the grammar, and a model for robust parsing which\ncombines linguistic sources of information and statistical sources of\ninformation. We discuss test results suggesting that grammatical processing\nallows fast and accurate processing of spoken input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906034v1",
            "date": "1999-06-30 23:06:09+00:00",
            "title": "A Unified Example-Based and Lexicalist Approach to Machine Translation",
            "authors": [
                "Davide Turcato",
                "Paul McFetridge",
                "Fred Popowich",
                "Janine Toole"
            ],
            "abstract": "We present an approach to Machine Translation that combines the ideas and\nmethodologies of the Example-Based and Lexicalist theoretical frameworks. The\napproach has been implemented in a multilingual Machine Translation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907003v1",
            "date": "1999-07-05 14:51:26+00:00",
            "title": "Annotation graphs as a framework for multidimensional linguistic data analysis",
            "authors": [
                "Steven Bird",
                "Mark Liberman"
            ],
            "abstract": "In recent work we have presented a formal framework for linguistic annotation\nbased on labeled acyclic digraphs. These `annotation graphs' offer a simple yet\npowerful method for representing complex annotation structures incorporating\nhierarchy and overlap. Here, we motivate and illustrate our approach using\ndiscourse-level annotations of text and speech data drawn from the CALLHOME,\nCOCONUT, MUC-7, DAMSL and TRAINS annotation schemes. With the help of domain\nspecialists, we have constructed a hybrid multi-level annotation for a fragment\nof the Boston University Radio Speech Corpus which includes the following\nlevels: segment, word, breath, ToBI, Tilt, Treebank, coreference and named\nentity. We show how annotation graphs can represent hybrid multi-level\nstructures which derive from a diverse set of file formats. We also show how\nthe approach facilitates substantive comparison of multiple annotations of a\nsingle signal based on different theoretical models. The discussion shows how\nannotation graphs open the door to wide-ranging integration of tools, formats\nand corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907006v1",
            "date": "1999-07-06 12:44:20+00:00",
            "title": "Representing Text Chunks",
            "authors": [
                "Erik F. Tjong Kim Sang",
                "Jorn Veenstra"
            ],
            "abstract": "Dividing sentences in chunks of words is a useful preprocessing step for\nparsing, information extraction and information retrieval. (Ramshaw and Marcus,\n1995) have introduced a \"convenient\" data representation for chunking by\nconverting it to a tagging task. In this paper we will examine seven different\ndata representations for the problem of recognizing noun phrase chunks. We will\nshow that the the data representation choice has a minor influence on chunking\nperformance. However, equipped with the most suitable data representation, our\nmemory-based learning chunker was able to improve the best published chunking\nresults for a standard data set.",
            "journal ref": "EACL'99, Bergen",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907007v2",
            "date": "1999-07-06 16:25:46+00:00",
            "title": "Cross-Language Information Retrieval for Technical Documents",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "This paper proposes a Japanese/English cross-language information retrieval\n(CLIR) system targeting technical documents. Our system first translates a\ngiven query containing technical terms into the target language, and then\nretrieves documents relevant to the translated query. The translation of\ntechnical terms is still problematic in that technical terms are often compound\nwords, and thus new terms can be progressively created simply by combining\nexisting base words. In addition, Japanese often represents loanwords based on\nits phonogram. Consequently, existing dictionaries find it difficult to achieve\nsufficient coverage. To counter the first problem, we use a compound word\ntranslation method, which uses a bilingual dictionary for base words and\ncollocational statistics to resolve translation ambiguity. For the second\nproblem, we propose a transliteration method, which identifies phonetic\nequivalents in the target language. We also show the effectiveness of our\nsystem using a test collection for CLIR.",
            "journal ref": "Proceedings of the Joint ACL SIGDAT Conference on Empirical\n  Methods in Natural Language Processing and Very Large Corpora, pp.29-37, 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907008v1",
            "date": "1999-07-06 18:35:41+00:00",
            "title": "Explanation-based Learning for Machine Translation",
            "authors": [
                "Janine Toole",
                "Fred Popowich",
                "Devlan Nicholson",
                "Davide Turcato",
                "Paul McFetridge"
            ],
            "abstract": "In this paper we present an application of explanation-based learning (EBL)\nin the parsing module of a real-time English-Spanish machine translation system\ndesigned to translate closed captions. We discuss the efficiency/coverage\ntrade-offs available in EBL and introduce the techniques we use to increase\ncoverage while maintaining a high level of space and time efficiency. Our\nperformance results indicate that this approach is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907010v1",
            "date": "1999-07-07 09:28:40+00:00",
            "title": "Language Identification With Confidence Limits",
            "authors": [
                "David Elworthy"
            ],
            "abstract": "A statistical classification algorithm and its application to language\nidentification from noisy input are described. The main innovation is to\ncompute confidence limits on the classification, so that the algorithm\nterminates when enough evidence to make a clear decision has been made, and so\navoiding problems with categories that have similar characteristics. A second\napplication, to genre identification, is briefly examined. The results show\nthat some of the problems of other language identification techniques can be\navoided, and illustrate a more important point: that a statistical language\nprocess can be used to provide feedback about its own success rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907012v1",
            "date": "1999-07-08 09:46:37+00:00",
            "title": "Selective Magic HPSG Parsing",
            "authors": [
                "Guido Minnen"
            ],
            "abstract": "We propose a parser for constraint-logic grammars implementing HPSG that\ncombines the advantages of dynamic bottom-up and advanced top-down control. The\nparser allows the user to apply magic compilation to specific constraints in a\ngrammar which as a result can be processed dynamically in a bottom-up and\ngoal-directed fashion. State of the art top-down processing techniques are used\nto deal with the remaining constraints. We discuss various aspects concerning\nthe implementation of the parser as part of a grammar development system.",
            "journal ref": "Proceedings of EACL99, Bergen, Norway, June 8-11",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907013v1",
            "date": "1999-07-08 10:08:59+00:00",
            "title": "Corpus Annotation for Parser Evaluation",
            "authors": [
                "John Carroll",
                "Guido Minnen",
                "Ted Briscoe"
            ],
            "abstract": "We describe a recently developed corpus annotation scheme for evaluating\nparsers that avoids shortcomings of current methods. The scheme encodes\ngrammatical relations between heads and dependents, and has been used to mark\nup a new public-domain corpus of naturally occurring English text. We show how\nthe corpus can be used to evaluate the accuracy of a robust parser, and relate\nthe corpus to extant resources.",
            "journal ref": "Proceedings of the EACL99 workshop on Linguistically Interpreted\n  Corpora (LINC), Bergen, Norway, June 12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907017v1",
            "date": "1999-07-09 22:39:52+00:00",
            "title": "A Bootstrap Approach to Automatically Generating Lexical Transfer Rules",
            "authors": [
                "Davide Turcato",
                "Paul McFetridge",
                "Fred Popowich",
                "Janine Toole"
            ],
            "abstract": "We describe a method for automatically generating Lexical Transfer Rules\n(LTRs) from word equivalences using transfer rule templates. Templates are\nskeletal LTRs, unspecified for words. New LTRs are created by instantiating a\ntemplate with words, provided that the words belong to the appropriate lexical\ncategories required by the template. We define two methods for creating an\ninventory of templates and using them to generate new LTRs. A simpler method\nconsists of extracting a finite set of templates from a sample of hand coded\nLTRs and directly using them in the generation process. A further method\nconsists of abstracting over the initial finite set of templates to define\nhigher level templates, where bilingual equivalences are defined in terms of\ncorrespondences involving phrasal categories. Phrasal templates are then mapped\nonto sets of lexical templates with the aid of grammars. In this way an\ninfinite set of lexical templates is recursively defined. New LTRs are created\nby parsing input words, matching a template at the phrasal level and using the\ncorresponding lexical categories to instantiate the lexical template. The\ndefinition of an infinite set of templates enables the automatic creation of\nLTRs for multi-word, non-compositional word equivalences of any cardinality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907021v1",
            "date": "1999-07-14 09:21:16+00:00",
            "title": "Architectural Considerations for Conversational Systems -- The Verbmobil/INTARC Experience",
            "authors": [
                "Guenther Goerz",
                "Joerg Spilker",
                "Volker Strom",
                "Hans Weber"
            ],
            "abstract": "The paper describes the speech to speech translation system INTARC, developed\nduring the first phase of the Verbmobil project. The general design goals of\nthe INTARC system architecture were time synchronous processing as well as\nincrementality and interactivity as a means to achieve a higher degree of\nrobustness and scalability. Interactivity means that in addition to the\nbottom-up (in terms of processing levels) data flow the ability to process\ntop-down restrictions considering the same signal segment for all processing\nlevels. The construction of INTARC 2.0, which has been operational since fall\n1996, followed an engineering approach focussing on the integration of symbolic\n(linguistic) and stochastic (recognition) techniques which led to a\ngeneralization of the concept of a ``one pass'' beam search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9908001v1",
            "date": "1999-08-01 14:02:57+00:00",
            "title": "Detecting Sub-Topic Correspondence through Bipartite Term Clustering",
            "authors": [
                "Zvika Marx",
                "Ido Dagan",
                "Eli Shamir"
            ],
            "abstract": "This paper addresses a novel task of detecting sub-topic correspondence in a\npair of text fragments, enhancing common notions of text similarity. This task\nis addressed by coupling corresponding term subsets through bipartite\nclustering. The paper presents a cost-based clustering scheme and compares it\nwith a bipartite version of the single-link method, providing illustrating\nresults.",
            "journal ref": "Proceedings of ACL'99 Workshop on Unsupervised Learning in Natural\n  Language Processing, 1999, pp 45-51",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9909002v1",
            "date": "1999-09-02 15:53:07+00:00",
            "title": "Semantic robust parsing for noun extraction from natural language queries",
            "authors": [
                "Afzal Ballim",
                "Vincenzo Pallotta"
            ],
            "abstract": "This paper describes how robust parsing techniques can be fruitful applied\nfor building a query generation module which is part of a pipelined NLP\narchitecture aimed at process natural language queries in a restricted domain.\nWe want to show that semantic robustness represents a key issue in those NLP\nsystems where it is more likely to have partial and ill-formed utterances due\nto various factors (e.g. noisy environments, low quality of speech recognition\nmodules, etc...) and where it is necessary to succeed, even if partially, in\nextracting some meaningful information.",
            "journal ref": "Proceedings of WPDI'99 (Workshop on Procedures in Discourse\n  Interpretation),1999, Iasi - Romania",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9910020v1",
            "date": "1999-10-23 11:19:35+00:00",
            "title": "Selective Sampling for Example-based Word Sense Disambiguation",
            "authors": [
                "Atsushi Fujii",
                "Kentaro Inui",
                "Takenobu Tokunaga",
                "Hozumi Tanaka"
            ],
            "abstract": "This paper proposes an efficient example sampling method for example-based\nword sense disambiguation systems. To construct a database of practical size, a\nconsiderable overhead for manual sense disambiguation (overhead for\nsupervision) is required. In addition, the time complexity of searching a\nlarge-sized database poses a considerable problem (overhead for search). To\ncounter these problems, our method selectively samples a smaller-sized\neffective subset from a given example set for use in word sense disambiguation.\nOur method is characterized by the reliance on the notion of training utility:\nthe degree to which each example is informative for future example sampling\nwhen used for the training of the system. The system progressively collects\nexamples by selecting those with greatest utility. The paper reports the\neffectiveness of our method through experiments on about one thousand\nsentences. Compared to experiments with other example sampling methods, our\nmethod reduced both the overhead for supervision and the overhead for search,\nwithout the degeneration of the performance of the system.",
            "journal ref": "Computational Linguistics, Vol.24, No.4, pp.573-597, 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9910022v1",
            "date": "1999-10-25 15:00:52+00:00",
            "title": "Practical experiments with regular approximation of context-free languages",
            "authors": [
                "Mark-Jan Nederhof"
            ],
            "abstract": "Several methods are discussed that construct a finite automaton given a\ncontext-free grammar, including both methods that lead to subsets and those\nthat lead to supersets of the original context-free language. Some of these\nmethods of regular approximation are new, and some others are presented here in\na more refined form with respect to existing literature. Practical experiments\nwith the different methods of regular approximation are performed for\nspoken-language input: hypotheses from a speech recognizer are filtered through\na finite automaton.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9911006v1",
            "date": "1999-11-15 05:48:03+00:00",
            "title": "Question Answering System Using Syntactic Information",
            "authors": [
                "M. Murata",
                "M. Utiyama",
                "H. Isahara"
            ],
            "abstract": "Question answering task is now being done in TREC8 using English documents.\nWe examined question answering task in Japanese sentences. Our method selects\nthe answer by matching the question sentence with knowledge-based data written\nin natural language. We use syntactic information to obtain highly accurate\nanswers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9911011v1",
            "date": "1999-11-19 16:10:51+00:00",
            "title": "One-Level Prosodic Morphology",
            "authors": [
                "Markus Walther"
            ],
            "abstract": "Recent developments in theoretical linguistics have lead to a widespread\nacceptance of constraint-based analyses of prosodic morphology phenomena such\nas truncation, infixation, floating morphemes and reduplication. Of these,\nreduplication is particularly challenging for state-of-the-art computational\nmorphology, since it involves copying of some part of a phonological string. In\nthis paper I argue for certain extensions to the one-level model of phonology\nand morphology (Bird & Ellison 1994) to cover the computational aspects of\nprosodic morphology using finite-state methods. In a nutshell, enriched lexical\nrepresentations provide additional automaton arcs to repeat or skip sounds and\nalso to allow insertion of additional material. A kind of resource\nconsciousness is introduced to control this additional freedom, distinguishing\nbetween producer and consumer arcs. The non-finite-state copying aspect of\nreduplication is mapped to automata intersection, itself a non-finite-state\noperation. Bounded local optimization prunes certain automaton arcs that fail\nto contribute to linguistic optimisation criteria. The paper then presents\nimplemented case studies of Ulwa construct state infixation, German\nhypocoristic truncation and Tagalog over-applying reduplication that illustrate\nthe expressive power of this approach, before its merits and limitations are\ndiscussed and possible extensions are sketched. I conclude that the one-level\napproach to prosodic morphology presents an attractive way of extending\nfinite-state techniques to difficult phenomena that hitherto resisted elegant\ncomputational analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912003v1",
            "date": "1999-12-13 04:42:25+00:00",
            "title": "Resolution of Indirect Anaphora in Japanese Sentences Using Examples 'X no Y (Y of X)'",
            "authors": [
                "M. Murata",
                "H. Isahara",
                "M. Nagao"
            ],
            "abstract": "A noun phrase can indirectly refer to an entity that has already been\nmentioned. For example, ``I went into an old house last night. The roof was\nleaking badly and ...'' indicates that ``the roof'' is associated with `` an\nold house}'', which was mentioned in the previous sentence. This kind of\nreference (indirect anaphora) has not been studied well in natural language\nprocessing, but is important for coherence resolution, language understanding,\nand machine translation. In order to analyze indirect anaphora, we need a case\nframe dictionary for nouns that contains knowledge of the relationships between\ntwo nouns but no such dictionary presently exists. Therefore, we are forced to\nuse examples of ``X no Y'' (Y of X) and a verb case frame dictionary instead.\nWe tried estimating indirect anaphora using this information and obtained a\nrecall rate of 63% and a precision rate of 68% on test sentences. This\nindicates that the information of ``X no Y'' is useful to a certain extent when\nwe cannot make use of a noun case frame dictionary. We estimated the results\nthat would be given by a noun case frame dictionary, and obtained recall and\nprecision rates of 71% and 82% respectively. Finally, we proposed a way to\nconstruct a noun case frame dictionary by using examples of ``X no Y.''",
            "journal ref": "ACL'99 Workshop on 'Coreference and Its Applications', Maryland,\n  USA, June 22, 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912004v1",
            "date": "1999-12-13 04:46:20+00:00",
            "title": "Pronoun Resolution in Japanese Sentences Using Surface Expressions and Examples",
            "authors": [
                "M. Murata",
                "H. Isahara",
                "M. Nagao"
            ],
            "abstract": "In this paper, we present a method of estimating referents of demonstrative\npronouns, personal pronouns, and zero pronouns in Japanese sentences using\nexamples, surface expressions, topics and foci. Unlike conventional work which\nwas semantic markers for semantic constraints, we used examples for semantic\nconstraints and showed in our experiments that examples are as useful as\nsemantic markers. We also propose many new methods for estimating referents of\npronouns. For example, we use the form ``X of Y'' for estimating referents of\ndemonstrative adjectives. In addition to our new methods, we used many\nconventional methods. As a result, experiments using these methods obtained a\nprecision rate of 87% in estimating referents of demonstrative pronouns,\npersonal pronouns, and zero pronouns for training sentences, and obtained a\nprecision rate of 78% for test sentences.",
            "journal ref": "ACL'99 Workshop on 'Coreference and Its Applications', Maryland,\n  USA, June 22, 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912005v1",
            "date": "1999-12-13 05:20:40+00:00",
            "title": "An Estimate of Referent of Noun Phrases in Japanese Sentences",
            "authors": [
                "M. Murata",
                "M. Nagao"
            ],
            "abstract": "In machine translation and man-machine dialogue, it is important to clarify\nreferents of noun phrases. We present a method for determining the referents of\nnoun phrases in Japanese sentences by using the referential properties,\nmodifiers, and possessors of noun phrases. Since the Japanese language has no\narticles, it is difficult to decide whether a noun phrase has an antecedent or\nnot. We had previously estimated the referential properties of noun phrases\nthat correspond to articles by using clue words in the sentences. By using\nthese referential properties, our system determined the referents of noun\nphrases in Japanese sentences. Furthermore we used the modifiers and possessors\nof noun phrases in determining the referents of noun phrases. As a result, on\ntraining sentences we obtained a precision rate of 82% and a recall rate of 85%\nin the determination of the referents of noun phrases that have antecedents. On\ntest sentences, we obtained a precision rate of 79% and a recall rate of 77%.",
            "journal ref": "Coling-ACL '98, Montrial, Canada, August 10, 1998 p912-916",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912006v1",
            "date": "1999-12-13 05:19:46+00:00",
            "title": "Resolution of Verb Ellipsis in Japanese Sentence using Surface Expressions and Examples",
            "authors": [
                "M. Murata",
                "M. Nagao"
            ],
            "abstract": "Verbs are sometimes omitted in Japanese sentences. It is necessary to recover\nomitted verbs for purposes of language understanding, machine translation, and\nconversational processing. This paper describes a practical way to recover\nomitted verbs by using surface expressions and examples. We experimented the\nresolution of verb ellipses by using this information, and obtained a recall\nrate of 73% and a precision rate of 66% on test sentences.",
            "journal ref": "Natural Language Processing Pacific Rim Symposium 1997 (NLPRS'97),\n  Cape Panwa Hotel, Phuket, Thailand, December 2-4, 1997 p75-80",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912007v1",
            "date": "1999-12-13 06:01:19+00:00",
            "title": "An Example-Based Approach to Japanese-to-English Translation of Tense, Aspect, and Modality",
            "authors": [
                "M. Murata",
                "Q. Ma",
                "K. Uchimoto",
                "H. Isahara"
            ],
            "abstract": "We have developed a new method for Japanese-to-English translation of tense,\naspect, and modality that uses an example-based method. In this method the\nsimilarity between input and example sentences is defined as the degree of\nsemantic matching between the expressions at the ends of the sentences. Our\nmethod also uses the k-nearest neighbor method in order to exclude the effects\nof noise; for example, wrongly tagged data in the bilingual corpora.\nExperiments show that our method can translate tenses, aspects, and modalities\nmore accurately than the top-level MT software currently available on the\nmarket can. Moreover, it does not require hand-craft rules.",
            "journal ref": "TMI'99, Chester, UK, August 23, 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912009v1",
            "date": "1999-12-15 11:02:22+00:00",
            "title": "Deduction over Mixed-Level Logic Representations for Text Passage Retrieval",
            "authors": [
                "Michael Hess"
            ],
            "abstract": "A system is described that uses a mixed-level representation of (part of)\nmeaning of natural language documents (based on standard Horn Clause Logic) and\na variable-depth search strategy that distinguishes between the different\nlevels of abstraction in the knowledge representation to locate specific\npassages in the documents. Mixed-level representations as well as\nvariable-depth search strategies are applicable in fields outside that of NLP.",
            "journal ref": "IEEE Computer Society Press, 1996. 383-390",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912017v1",
            "date": "1999-12-23 15:48:26+00:00",
            "title": "Mixed-Level Knowledge Representation and Variable-Depth Inference in Natural Language Processing",
            "authors": [
                "Michael Hess"
            ],
            "abstract": "A system is described that uses a mixed-level knowledge representation based\non standard Horn Clause Logic to represent (part of) the meaning of natural\nlanguage documents. A variable-depth search strategy is outlined that\ndistinguishes between the different levels of abstraction in the knowledge\nrepresentation to locate specific passages in the documents. A detailed\ndescription of the linguistic aspects of the system is given. Mixed-level\nrepresentations as well as variable-depth search strategies are applicable in\nfields outside that of NLP.",
            "journal ref": "International Journal on Artificial Intelligence Tools (IJAIT),\n  vol 6, no 4, 1997. 481-509",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9902002v1",
            "date": "1999-02-01 11:01:23+00:00",
            "title": "Automatic Identification of Subjects for Textual Documents in Digital Libraries",
            "authors": [
                "Kuang-hua Chen"
            ],
            "abstract": "The amount of electronic documents in the Internet grows very quickly. How to\neffectively identify subjects for documents becomes an important issue. In\npast, the researches focus on the behavior of nouns in documents. Although\nsubjects are composed of nouns, the constituents that determine which nouns are\nsubjects are not only nouns. Based on the assumption that texts are\nwell-organized and event-driven, nouns and verbs together contribute the\nprocess of subject identification. This paper considers four factors: 1) word\nimportance, 2) word frequency, 3) word co-occurrence, and 4) word distance and\nproposes a model to identify subjects for textual documents. The preliminary\nexperiments show that the performance of the proposed model is close to that of\nhuman beings.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9903007v1",
            "date": "1999-03-05 18:25:11+00:00",
            "title": "Some Remarks on the Geometry of Grammar",
            "authors": [
                "Marc Dymetman"
            ],
            "abstract": "This paper, following (Dymetman:1998), presents an approach to grammar\ndescription and processing based on the geometry of cancellation diagrams, a\nconcept which plays a central role in combinatorial group theory\n(Lyndon-Schuppe:1977). The focus here is on the geometric intuitions and on\nrelating group-theoretical diagrams to the traditional charts associated with\ncontext-free grammars and type-0 rewriting systems. The paper is structured as\nfollows. We begin in Section 1 by analyzing charts in terms of constructs\ncalled cells, which are a geometrical counterpart to rules. Then we move in\nSection 2 to a presentation of cancellation diagrams and show how they can be\nused computationally. In Section 3 we give a formal algebraic presentation of\nthe concept of group computation structure, which is based on the standard\nnotions of free group and conjugacy. We then relate in Section 4 the geometric\nand the algebraic views of computation by using the fundamental theorem of\ncombinatorial group theory (Rotman:1994). In Section 5 we study in more detail\nthe relationship between the two views on the basis of a simple grammar stated\nas a group computation structure. In section 6 we extend this grammar to handle\nnon-local constructs such as relative pronouns and quantifiers. We conclude in\nSection 7 with some brief notes on the differences between normal submonoids\nand normal subgroups, group computation versus rewriting systems, and the use\nof group morphisms to study the computational complexity of parsing and\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9904004v1",
            "date": "1999-04-12 11:37:49+00:00",
            "title": "Mixing Metaphors",
            "authors": [
                "Mark Lee",
                "John Barnden"
            ],
            "abstract": "Mixed metaphors have been neglected in recent metaphor research. This paper\nsuggests that such neglect is short-sighted. Though mixing is a more complex\nphenomenon than straight metaphors, the same kinds of reasoning and knowledge\nstructures are required. This paper provides an analysis of both parallel and\nserial mixed metaphors within the framework of an AI system which is already\ncapable of reasoning about straight metaphorical manifestations and argues that\nthe processes underlying mixing are central to metaphorical meaning. Therefore,\nany theory of metaphors must be able to account for mixing.",
            "journal ref": "Proceedings of the AISB'99 Symposium on Metaphor, Artificial\n  Intelligence, and Cognition, pages 11-16, Edinburgh",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9905007v1",
            "date": "1999-05-12 14:25:40+00:00",
            "title": "An Efficient, Probabilistically Sound Algorithm for Segmentation and Word Discovery",
            "authors": [
                "Michael R. Brent"
            ],
            "abstract": "This paper presents a model-based, unsupervised algorithm for recovering word\nboundaries in a natural-language text from which they have been deleted. The\nalgorithm is derived from a probability model of the source that generated the\ntext. The fundamental structure of the model is specified abstractly so that\nthe detailed component models of phonology, word-order, and word frequency can\nbe replaced in a modular fashion. The model yields a language-independent,\nprior probability distribution on all possible sequences of all possible words\nover a given alphabet, based on the assumption that the input was generated by\nconcatenating words from a fixed but unknown lexicon. The model is unusual in\nthat it treats the generation of a complete corpus, regardless of length, as a\nsingle event in the probability space. Accordingly, the algorithm does not\nestimate a probability distribution on words; instead, it attempts to calculate\nthe prior probabilities of various word sequences that could underlie the\nobserved text. Experiments on phonemic transcripts of spontaneous speech by\nparents to young children suggest that this algorithm is more effective than\nother proposed algorithms, at least when utterance boundaries are given and the\ntext includes a substantial number of short utterances.\n  Keywords: Bayesian grammar induction, probability models, minimum description\nlength (MDL), unsupervised learning, cognitive modeling, language acquisition,\nsegmentation",
            "journal ref": "Brent, M. R. (1999). An efficient, probabilistically sound\n  algorithm for segmentation and word discovery. Machine Learning 34, 71-105",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9905009v1",
            "date": "1999-05-19 14:47:21+00:00",
            "title": "Inside-Outside Estimation of a Lexicalized PCFG for German",
            "authors": [
                "Franz Beil",
                "Glenn Carroll",
                "Detlef Prescher",
                "Stefan Riezler",
                "Mats Rooth"
            ],
            "abstract": "The paper describes an extensive experiment in inside-outside estimation of a\nlexicalized probabilistic context free grammar for German verb-final clauses.\nGrammar and formalism features which make the experiment feasible are\ndescribed. Successive models are evaluated on precision and recall of phrase\nmarkup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9905010v1",
            "date": "1999-05-19 16:03:05+00:00",
            "title": "Statistical Inference and Probabilistic Modelling for Constraint-Based NLP",
            "authors": [
                "Stefan Riezler"
            ],
            "abstract": "We present a probabilistic model for constraint-based grammars and a method\nfor estimating the parameters of such models from incomplete, i.e., unparsed\ndata. Whereas methods exist to estimate the parameters of probabilistic\ncontext-free grammars from incomplete data (Baum 1970), so far for\nprobabilistic grammars involving context-dependencies only parameter estimation\ntechniques from complete, i.e., fully parsed data have been presented (Abney\n1997). However, complete-data estimation requires labor-intensive, error-prone,\nand grammar-specific hand-annotating of large language corpora. We present a\nlog-linear probability model for constraint logic programming, and a general\nalgorithm to estimate the parameters of such models from incomplete data by\nextending the estimation algorithm of Della-Pietra, Della-Pietra, and Lafferty\n(1997) to incomplete data settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906004v1",
            "date": "1999-06-02 13:41:51+00:00",
            "title": "Cascaded Grammatical Relation Assignment",
            "authors": [
                "Sabine Buchholz",
                "Jorn Veenstra",
                "Walter Daelemans"
            ],
            "abstract": "In this paper we discuss cascaded Memory-Based grammatical relations\nassignment. In the first stages of the cascade, we find chunks of several types\n(NP,VP,ADJP,ADVP,PP) and label them with their adverbial function (e.g. local,\ntemporal). In the last stage, we assign grammatical relations to pairs of\nchunks. We studied the effect of adding several levels to this cascaded\nclassifier and we found that even the less performing chunkers enhanced the\nperformance of the relation finder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906005v1",
            "date": "1999-06-02 13:48:48+00:00",
            "title": "Memory-Based Shallow Parsing",
            "authors": [
                "Walter Daelemans",
                "Sabine Buchholz",
                "Jorn Veenstra"
            ],
            "abstract": "We present a memory-based learning (MBL) approach to shallow parsing in which\nPOS tagging, chunking, and identification of syntactic relations are formulated\nas memory-based modules. The experiments reported in this paper show\ncompetitive results, the F-value for the Wall Street Journal (WSJ) treebank is:\n93.8% for NP chunking, 94.7% for VP chunking, 77.1% for subject detection and\n79.0% for object detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906006v2",
            "date": "1999-06-02 15:50:26+00:00",
            "title": "Learning Efficient Disambiguation",
            "authors": [
                "Khalil Sima'an"
            ],
            "abstract": "This dissertation analyses the computational properties of current\nperformance-models of natural language parsing, in particular Data Oriented\nParsing (DOP), points out some of their major shortcomings and suggests\nsuitable solutions. It provides proofs that various problems of probabilistic\ndisambiguation are NP-Complete under instances of these performance-models, and\nit argues that none of these models accounts for attractive efficiency\nproperties of human language processing in limited domains, e.g. that frequent\ninputs are usually processed faster than infrequent ones. The central\nhypothesis of this dissertation is that these shortcomings can be eliminated by\nspecializing the performance-models to the limited domains. The dissertation\naddresses \"grammar and model specialization\" and presents a new framework, the\nAmbiguity-Reduction Specialization (ARS) framework, that formulates the\nnecessary and sufficient conditions for successful specialization. The\nframework is instantiated into specialization algorithms and applied to\nspecializing DOP. Novelties of these learning algorithms are 1) they limit the\nhypotheses-space to include only \"safe\" models, 2) are expressed as constrained\noptimization formulae that minimize the entropy of the training tree-bank given\nthe specialized grammar, under the constraint that the size of the specialized\nmodel does not exceed a predefined maximum, and 3) they enable integrating the\nspecialized model with the original one in a complementary manner. The\ndissertation provides experiments with initial implementations and compares the\nresulting Specialized DOP (SDOP) models to the original DOP models with\nencouraging results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906019v2",
            "date": "1999-06-22 07:41:24+00:00",
            "title": "Resolving Part-of-Speech Ambiguity in the Greek Language Using Learning Techniques",
            "authors": [
                "G. Petasis",
                "G. Paliouras",
                "V. Karkaletsis",
                "C. D. Spyropoulos",
                "I. Androutsopoulos"
            ],
            "abstract": "This article investigates the use of Transformation-Based Error-Driven\nlearning for resolving part-of-speech ambiguity in the Greek language. The aim\nis not only to study the performance, but also to examine its dependence on\ndifferent thematic domains. Results are presented here for two different test\ncases: a corpus on \"management succession events\" and a general-theme corpus.\nThe two experiments show that the performance of this method does not depend on\nthe thematic domain of the corpus, and its accuracy for the Greek language is\naround 95%.",
            "journal ref": "In Fakotakis, N. et al. (Eds.), Machine Learning in Human Language\n  Technology (Proceedings of the ACAI Workshop), pp. 29-34, Chania, Greece,\n  1999.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906027v1",
            "date": "1999-06-25 11:44:42+00:00",
            "title": "Human-Computer Conversation",
            "authors": [
                "Yorick Wilks",
                "Roberta Catizone"
            ],
            "abstract": "The article surveys a little of the history of the technology, sets out the\nmain current theoretical approaches in brief, and discusses the on-going\nopposition between theoretical and empirical approaches. It illustrates the\nsituation with some discussion of CONVERSE, a system that won the Loebner prize\nin 1997 and which displays features of both approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907004v2",
            "date": "1999-07-06 01:44:00+00:00",
            "title": "MAP Lexicon is useful for segmentation and word discovery in child-directed speech",
            "authors": [
                "Anand Venkataraman"
            ],
            "abstract": "Because of rather fundamental changes to the underlying model proposed in the\npaper, it has been withdrawn from the archive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9907026v1",
            "date": "1999-07-16 15:43:45+00:00",
            "title": "Mixing representation levels: The hybrid approach to automatic text generation",
            "authors": [
                "Emanuele Pianta",
                "Lucia M. Tovena"
            ],
            "abstract": "Natural language generation systems (NLG) map non-linguistic representations\ninto strings of words through a number of steps using intermediate\nrepresentations of various levels of abstraction. Template based systems, by\ncontrast, tend to use only one representation level, i.e. fixed strings, which\nare combined, possibly in a sophisticated way, to generate the final text.\n  In some circumstances, it may be profitable to combine NLG and template based\ntechniques. The issue of combining generation techniques can be seen in more\nabstract terms as the issue of mixing levels of representation of different\ndegrees of linguistic abstraction. This paper aims at defining a reference\narchitecture for systems using mixed representations. We argue that mixed\nrepresentations can be used without abandoning a linguistically grounded\napproach to language generation.",
            "journal ref": "Proceedings of the AISB'99 Workshop on ``Reference Architectures\n  and Data Standards for NLP'', Edinburgh Scotland, April 1999, 8-13",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9910011v1",
            "date": "1999-10-13 03:25:33+00:00",
            "title": "A statistical model for word discovery in child directed speech",
            "authors": [
                "Anand Venkataraman"
            ],
            "abstract": "A statistical model for segmentation and word discovery in child directed\nspeech is presented. An incremental unsupervised learning algorithm to infer\nword boundaries based on this model is described and results of empirical tests\nshowing that the algorithm is competitive with other models that have been used\nfor similar tasks are also presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9912016v1",
            "date": "1999-12-23 01:07:33+00:00",
            "title": "HMM Specialization with Selective Lexicalization",
            "authors": [
                "Jin-Dong Kim",
                "Sang-Zoo Lee",
                "Hae-Chang Rim"
            ],
            "abstract": "We present a technique which complements Hidden Markov Models by\nincorporating some lexicalized states representing syntactically uncommon\nwords. Our approach examines the distribution of transitions, selects the\nuncommon words, and makes lexicalized states for the words. We performed a\npart-of-speech tagging experiment on the Brown corpus to evaluate the resultant\nlanguage model and discovered that this technique improved the tagging accuracy\nby 0.21% at the 95% level of confidence.",
            "journal ref": "Proceedings of the 1999 Joint SIGDAT Conference on Empirical\n  Methods in Natural Language Processing and Very Large Corpora, pp.121-127,\n  1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9902027v1",
            "date": "1999-02-16 15:34:02+00:00",
            "title": "Autocatalytic Theory of Meaning",
            "authors": [
                "Mark D. Roberts"
            ],
            "abstract": "Recently it has been argued that autocatalytic theory could be applied to the\norigin of culture. Here possible application to a theory of meaning in the\nphilosophy of language, called radical interpretation, is commented upon and\ncompared to previous applications.",
            "journal ref": "PSYCHOLOQUY.99.10.014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9905008v1",
            "date": "1999-05-19 14:52:33+00:00",
            "title": "Inducing a Semantically Annotated Lexicon via EM-Based Clustering",
            "authors": [
                "Mats Rooth",
                "Stefan Riezler",
                "Detlef Prescher",
                "Glenn Carroll",
                "Franz Beil"
            ],
            "abstract": "We present a technique for automatic induction of slot annotations for\nsubcategorization frames, based on induction of hidden classes in the EM\nframework of statistical estimation. The models are empirically evalutated by a\ngeneral decision test. Induction of slot labeling for subcategorization frames\nis accomplished by a further application of EM, and applied experimentally on\nframe observations derived from parsing large corpora. We outline an\ninterpretation of the learned representations as theoretical-linguistic\ndecompositional lexical entries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/9906029v2",
            "date": "1999-06-28 17:06:51+00:00",
            "title": "Events in Property Patterns",
            "authors": [
                "M. Chechik",
                "D. Paun"
            ],
            "abstract": "A pattern-based approach to the presentation, codification and reuse of\nproperty specifications for finite-state verification was proposed by Dwyer and\nhis collegues. The patterns enable non-experts to read and write formal\nspecifications for realistic systems and facilitate easy conversion of\nspecifications between formalisms, such as LTL, CTL, QRE. In this paper, we\nextend the pattern system with events - changes of values of variables in the\ncontext of LTL.",
            "journal ref": "Lecture notes in Computer Science (Proceedings of 6 Spin'99\n  Workshop)",
            "category": "cs.SE"
        }
    ],
    "2000": [
        {
            "url": "http://arxiv.org/abs/cs/0001010v1",
            "date": "2000-01-14 16:01:31+00:00",
            "title": "A Real World Implementation of Answer Extraction",
            "authors": [
                "D. Molla",
                "J. Berri",
                "M. Hess"
            ],
            "abstract": "In this paper we describe ExtrAns, an answer extraction system. Answer\nextraction (AE) aims at retrieving those exact passages of a document that\ndirectly answer a given user question. AE is more ambitious than information\nretrieval and information extraction in that the retrieval results are phrases,\nnot entire documents, and in that the queries may be arbitrarily specific. It\nis less ambitious than full-fledged question answering in that the answers are\nnot generated from a knowledge base but looked up in the text of documents. The\ncurrent version of ExtrAns is able to parse unedited Unix \"man pages\", and\nderive the logical form of their sentences. User queries are also translated\ninto logical forms. A theorem prover then retrieves the relevant phrases, which\nare presented through selective highlighting in their context.",
            "journal ref": "Proc. of 9th International Conference and Workshop on Database and\n  Expert Systems. Workshop \"Natural Language and Information Systems\"\n  (NLIS'98). Vienna: 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001012v1",
            "date": "2000-01-18 23:19:22+00:00",
            "title": "Measures of Distributional Similarity",
            "authors": [
                "Lillian Lee"
            ],
            "abstract": "We study distributional similarity measures for the purpose of improving\nprobability estimation for unseen cooccurrences. Our contributions are\nthree-fold: an empirical comparison of a broad range of measures; a\nclassification of similarity functions based on the information that they\nincorporate; and the introduction of a novel function that is superior at\nevaluating potential proxy distributions.",
            "journal ref": "37th Annual Meeting of the ACL, 1999, pp. 25-32",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001020v1",
            "date": "2000-01-24 20:18:16+00:00",
            "title": "Exploiting Syntactic Structure for Natural Language Modeling",
            "authors": [
                "Ciprian Chelba"
            ],
            "abstract": "The thesis presents an attempt at using the syntactic structure in natural\nlanguage for improved language models for speech recognition. The structured\nlanguage model merges techniques in automatic parsing and language modeling\nusing an original probabilistic parameterization of a shift-reduce parser. A\nmaximum likelihood reestimation procedure belonging to the class of\nexpectation-maximization algorithms is employed for training the model.\nExperiments on the Wall Street Journal, Switchboard and Broadcast News corpora\nshow improvement in both perplexity and word error rate - word lattice\nrescoring - over the standard 3-gram language model. The significance of the\nthesis lies in presenting an original approach to language modeling that uses\nthe hierarchical - syntactic - structure in natural language to improve on\ncurrent 3-gram modeling techniques for large vocabulary speech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001021v1",
            "date": "2000-01-24 20:55:17+00:00",
            "title": "Refinement of a Structured Language Model",
            "authors": [
                "Ciprian Chelba",
                "Frederick Jelinek"
            ],
            "abstract": "A new language model for speech recognition inspired by linguistic analysis\nis presented. The model develops hidden hierarchical structure incrementally\nand uses it to extract meaningful information from the word history - thus\nenabling the use of extended distance dependencies - in an attempt to\ncomplement the locality of currently used n-gram Markov models. The model, its\nprobabilistic parametrization, a reestimation algorithm for the model\nparameters and a set of experiments meant to evaluate its potential for speech\nrecognition are presented.",
            "journal ref": "Proceedings of the International Conference on Advances in Pattern\n  Recognition, 1998, pp. 275-284, Plymouth, UK",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001022v1",
            "date": "2000-01-24 21:18:37+00:00",
            "title": "Recognition Performance of a Structured Language Model",
            "authors": [
                "Ciprian Chelba",
                "Frederick Jelinek"
            ],
            "abstract": "A new language model for speech recognition inspired by linguistic analysis\nis presented. The model develops hidden hierarchical structure incrementally\nand uses it to extract meaningful information from the word history - thus\nenabling the use of extended distance dependencies - in an attempt to\ncomplement the locality of currently used trigram models. The structured\nlanguage model, its probabilistic parameterization and performance in a\ntwo-pass speech recognizer are presented. Experiments on the SWITCHBOARD corpus\nshow an improvement in both perplexity and word error rate over conventional\ntrigram models.",
            "journal ref": "Proceedings of Eurospeech, 1999, pp. 1567-1570, Budapest, Hungary",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001023v1",
            "date": "2000-01-25 19:35:01+00:00",
            "title": "Structured Language Modeling for Speech Recognition",
            "authors": [
                "Ciprian Chelba",
                "Frederick Jelinek"
            ],
            "abstract": "A new language model for speech recognition is presented. The model develops\nhidden hierarchical syntactic-like structure incrementally and uses it to\nextract meaningful information from the word history, thus complementing the\nlocality of currently used trigram models. The structured language model (SLM)\nand its performance in a two-pass speech recognizer --- lattice decoding ---\nare presented. Experiments on the WSJ corpus show an improvement in both\nperplexity (PPL) and word error rate (WER) over conventional trigram models.",
            "journal ref": "Proceedings of NLDB'99, Klagenfurt, Austria",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0002007v1",
            "date": "2000-02-11 19:17:18+00:00",
            "title": "Requirements of Text Processing Lexicons",
            "authors": [
                "K. Litkowski"
            ],
            "abstract": "As text processing systems expand in scope, they will require ever larger\nlexicons along with a parsing capability for discriminating among many senses\nof a word. Existing systems do not incorporate such subtleties in meaning for\ntheir lexicons. Ordinary dictionaries contain such information, but are largely\nuntapped. When the contents of dictionaries are scrutinized, they reveal many\nrequirements that must be satisfied in representing meaning and in developing\nsemantic parsers. These requirements were identified in research designed to\nfind primitive verb concepts. The requirements are outlined and general\nprocedures for satisfying them through the use of ordinary dictionaries are\ndescribed, illustrated by building frames for and examining the definitions of\n\"change\" and its uses as a hypernym in other definitions.",
            "journal ref": "Proceedings of the 18th Annual Meeting of the Association for\n  Computational Linguistics, Philadelphia, PA (1980), pp. 153-4",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0002017v1",
            "date": "2000-02-27 04:09:59+00:00",
            "title": "An Usage Measure Based on Psychophysical Relations",
            "authors": [
                "V. Kromer"
            ],
            "abstract": "A new word usage measure is proposed. It is based on psychophysical relations\nand allows to reveal words by its degree of \"importance\" for making basic\ndictionaries of sublanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003055v1",
            "date": "2000-03-13 09:55:08+00:00",
            "title": "TnT - A Statistical Part-of-Speech Tagger",
            "authors": [
                "Thorsten Brants"
            ],
            "abstract": "Trigrams'n'Tags (TnT) is an efficient statistical part-of-speech tagger.\nContrary to claims found elsewhere in the literature, we argue that a tagger\nbased on Markov models performs at least as well as other current approaches,\nincluding the Maximum Entropy framework. A recent comparison has even shown\nthat TnT performs significantly better for the tested corpora. We describe the\nbasic model of TnT, the techniques used for smoothing and for handling unknown\nwords. Furthermore, we present evaluations on two corpora.",
            "journal ref": "Proceedings of ANLP-2000, Seattle, WA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003060v1",
            "date": "2000-03-14 13:09:28+00:00",
            "title": "Message Classification in the Call Center",
            "authors": [
                "Stephan Busemann",
                "Sven Schmeier",
                "Roman G. Arens"
            ],
            "abstract": "Customer care in technical domains is increasingly based on e-mail\ncommunication, allowing for the reproduction of approved solutions. Identifying\nthe customer's problem is often time-consuming, as the problem space changes if\nnew products are launched. This paper describes a new approach to the\nclassification of e-mail requests based on shallow text processing and machine\nlearning techniques. It is implemented within an assistance system for call\ncenter agents that is used in a commercial setting.",
            "journal ref": "Proceedings of ANLP-2000, Seattle, WA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003074v1",
            "date": "2000-03-23 11:29:15+00:00",
            "title": "A Finite State and Data-Oriented Method for Grapheme to Phoneme Conversion",
            "authors": [
                "Gosse Bouma"
            ],
            "abstract": "A finite-state method, based on leftmost longest-match replacement, is\npresented for segmenting words into graphemes, and for converting graphemes\ninto phonemes. A small set of hand-crafted conversion rules for Dutch achieves\na phoneme accuracy of over 93%. The accuracy of the system is further improved\nby using transformation-based learning. The phoneme accuracy of the best system\n(using a large set of rule templates and a `lazy' variant of Brill's algoritm),\ntrained on only 40K words, reaches 99% accuracy.",
            "journal ref": "Proceedings of NAACL-2000, Seattle, WA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003081v1",
            "date": "2000-03-29 16:35:58+00:00",
            "title": "Variable Word Rate N-grams",
            "authors": [
                "Yoshihiko Gotoh",
                "Steve Renals"
            ],
            "abstract": "The rate of occurrence of words is not uniform but varies from document to\ndocument. Despite this observation, parameters for conventional n-gram language\nmodels are usually derived using the assumption of a constant word rate. In\nthis paper we investigate the use of variable word rate assumption, modelled by\na Poisson distribution or a continuous mixture of Poissons. We present an\napproach to estimating the relative frequencies of words or n-grams taking\nprior information of their occurrences into account. Discounting and smoothing\nschemes are also considered. Using the Broadcast News task, the approach\ndemonstrates a reduction of perplexity up to 10%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003083v1",
            "date": "2000-03-30 16:56:02+00:00",
            "title": "Advances in domain independent linear text segmentation",
            "authors": [
                "Freddy Y. Y. Choi"
            ],
            "abstract": "This paper describes a method for linear text segmentation which is twice as\naccurate and over seven times as fast as the state-of-the-art (Reynar, 1998).\nInter-sentence similarity is replaced by rank in the local context. Boundary\nlocations are discovered by divisive clustering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003084v1",
            "date": "2000-03-30 16:52:50+00:00",
            "title": "Information Extraction from Broadcast News",
            "authors": [
                "Yoshihiko Gotoh",
                "Steve Renals"
            ],
            "abstract": "This paper discusses the development of trainable statistical models for\nextracting content from television and radio news broadcasts. In particular we\nconcentrate on statistical finite state models for identifying proper names and\nother named entities in broadcast speech. Two models are presented: the first\nrepresents name class information as a word attribute; the second represents\nboth word-word and class-class transitions explicitly. A common n-gram based\nformulation is used for both models. The task of named entity identification is\ncharacterized by relatively sparse training data and issues related to\nsmoothing are discussed. Experiments are reported using the DARPA/NIST Hub-4E\nevaluation for North American Broadcast News.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0004016v1",
            "date": "2000-04-28 03:18:09+00:00",
            "title": "Looking at discourse in a corpus: The role of lexical cohesion",
            "authors": [
                "Tony Berber Sardinha"
            ],
            "abstract": "This paper is aimed at reporting on the development and application of a\ncomputer model for discourse analysis through segmentation. Segmentation refers\nto the principled division of texts into contiguous constituents. Other studies\nhave looked at the application of a number of models to the analysis of\ndiscourse by computer. The segmentation procedure developed for the present\ninvestigation is called LSM ('Link Set Median'). It was applied to three corpus\nof 300 texts from three different genres. The results obtained by application\nof the LSM procedure on the corpus were then compared to segmentation carried\nout at random. Statistical analyses suggested that LSM significantly\noutperformed random segmentation, thus indicating that the segmentation was\nmeaningful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005006v1",
            "date": "2000-05-07 00:15:59+00:00",
            "title": "A Simple Approach to Building Ensembles of Naive Bayesian Classifiers for Word Sense Disambiguation",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper presents a corpus-based approach to word sense disambiguation that\nbuilds an ensemble of Naive Bayesian classifiers, each of which is based on\nlexical features that represent co--occurring words in varying sized windows of\ncontext. Despite the simplicity of this approach, empirical results\ndisambiguating the widely studied nouns line and interest show that such an\nensemble achieves accuracy rivaling the best previously published results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005015v1",
            "date": "2000-05-10 11:58:12+00:00",
            "title": "Noun Phrase Recognition by System Combination",
            "authors": [
                "Erik F. Tjong Kim Sang"
            ],
            "abstract": "The performance of machine learning algorithms can be improved by combining\nthe output of different systems. In this paper we apply this idea to the\nrecognition of noun phrases.We generate different classifiers by using\ndifferent representations of the data. By combining the results with voting\ntechniques described in (Van Halteren et.al. 1998) we manage to improve the\nbest reported performances on standard data sets for base noun phrases and\narbitrary noun phrases.",
            "journal ref": "Proceedings of NAACL 2000, Seattle, WA, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005016v1",
            "date": "2000-05-10 14:51:14+00:00",
            "title": "Improving Testsuites via Instrumentation",
            "authors": [
                "Norbert Broeker"
            ],
            "abstract": "This paper explores the usefulness of a technique from software engineering,\nnamely code instrumentation, for the development of large-scale natural\nlanguage grammars. Information about the usage of grammar rules in test\nsentences is used to detect untested rules, redundant test sentences, and\nlikely causes of overgeneration. Results show that less than half of a\nlarge-coverage grammar for German is actually tested by two large testsuites,\nand that 10-30% of testing time is redundant. The methodology applied can be\nseen as a re-use of grammar writing knowledge for testsuite compilation.",
            "journal ref": "Proc. ANLP--NAACL, Seattle/WA, Apr29--May4 2000, pp.325-330",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005019v1",
            "date": "2000-05-12 14:12:15+00:00",
            "title": "On the Scalability of the Answer Extraction System \"ExtrAns\"",
            "authors": [
                "Diego Moll'a Aliod",
                "Michael Hess"
            ],
            "abstract": "This paper reports on the scalability of the answer extraction system\nExtrAns. An answer extraction system locates the exact phrases in the documents\nthat contain the explicit answers to the user queries. Answer extraction\nsystems are therefore more convenient than document retrieval systems in\nsituations where the user wants to find specific information in limited time.\n  ExtrAns performs answer extraction over UNIX manpages. It has been\nconstructed by combining available linguistic resources and implementing only a\nfew modules from scratch. A resolution procedure between the minimal logical\nform of the user query and the minimal logical forms of the manpage sentences\nfinds the answers to the queries. These answers are displayed to the user,\ntogether with pointers to the respective manpages, and the exact phrases that\ncontribute to the answer are highlighted.\n  This paper shows that the increase in response times is not a big issue when\nscaling the system up from 30 to 500 documents, and that the response times for\n500 documents are still acceptable for a real-time answer extraction system.",
            "journal ref": "Applications of Natural Language to Information Systems (NLDB'99).\n  Klagenfurt, Austria, 1999, 219-224",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005025v1",
            "date": "2000-05-22 14:07:28+00:00",
            "title": "Finite-State Reduplication in One-Level Prosodic Morphology",
            "authors": [
                "Markus Walther"
            ],
            "abstract": "Reduplication, a central instance of prosodic morphology, is particularly\nchallenging for state-of-the-art computational morphology, since it involves\ncopying of some part of a phonological string. In this paper I advocate a\nfinite-state method that combines enriched lexical representations via\nintersection to implement the copying. The proposal includes a\nresource-conscious variant of automata and can benefit from the existence of\nlazy algorithms. Finally, the implementation of a complex case from Koasati is\npresented.",
            "journal ref": "Proc. NAACL-2000, Seattle/WA, pp.296-302",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005029v1",
            "date": "2000-05-30 17:10:33+00:00",
            "title": "Ranking suspected answers to natural language questions using predictive annotation",
            "authors": [
                "Dragomir R. Radev",
                "John Prager",
                "Valerie Samn"
            ],
            "abstract": "In this paper, we describe a system to rank suspected answers to natural\nlanguage questions. We process both corpus and query using a new technique,\npredictive annotation, which augments phrases in texts with labels anticipating\ntheir being targets of certain kinds of questions. Given a natural language\nquestion, an IR system returns a set of matching passages, which are then\nanalyzed and ranked according to various criteria described in this paper. We\nprovide an evaluation of the techniques based on results from the TREC Q&A\nevaluation in which our system participated.",
            "journal ref": "ANLP'00, Seattle, WA, May 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006003v1",
            "date": "2000-06-01 18:42:24+00:00",
            "title": "Exploiting Diversity in Natural Language Processing: Combining Parsers",
            "authors": [
                "John C. Henderson",
                "Eric Brill"
            ],
            "abstract": "Three state-of-the-art statistical parsers are combined to produce more\naccurate parses, as well as new bounds on achievable Treebank parsing accuracy.\nTwo general approaches are presented and two combination techniques are\ndescribed for each approach. Both parametric and non-parametric models are\nexplored. The resulting parsers surpass the best previously published\nperformance results for the Penn Treebank.",
            "journal ref": "Proceedings of the Fourth Conference on Empirical Methods in\n  Natural Language Processing (EMNLP-99), pages 187-194. College Park,\n  Maryland, USA. June, 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006011v1",
            "date": "2000-06-05 18:04:51+00:00",
            "title": "Bagging and Boosting a Treebank Parser",
            "authors": [
                "John C. Henderson",
                "Eric Brill"
            ],
            "abstract": "Bagging and boosting, two effective machine learning techniques, are applied\nto natural language parsing. Experiments using these techniques with a\ntrainable statistical parser are described. The best resulting system provides\nroughly as large of a gain in F-measure as doubling the corpus size. Error\nanalysis of the result of the boosting technique reveals some inconsistent\nannotations in the Penn Treebank, suggesting a semi-automatic method for\nfinding inconsistent treebank annotations.",
            "journal ref": "Proceedings of the 1st Meeting of the North American Chapter of\n  the Association for Computational Linguistics (NAACL-2000), pages 34-41",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006012v1",
            "date": "2000-06-05 21:33:03+00:00",
            "title": "Exploiting Diversity for Natural Language Parsing",
            "authors": [
                "John C. Henderson"
            ],
            "abstract": "The popularity of applying machine learning methods to computational\nlinguistics problems has produced a large supply of trainable natural language\nprocessing systems. Most problems of interest have an array of off-the-shelf\nproducts or downloadable code implementing solutions using various techniques.\nWhere these solutions are developed independently, it is observed that their\nerrors tend to be independently distributed. This thesis is concerned with\napproaches for capitalizing on this situation in a sample problem domain, Penn\nTreebank-style parsing.\n  The machine learning community provides techniques for combining outputs of\nclassifiers, but parser output is more structured and interdependent than\nclassifications. To address this discrepancy, two novel strategies for\ncombining parsers are used: learning to control a switch between parsers and\nconstructing a hybrid parse from multiple parsers' outputs.\n  Off-the-shelf parsers are not developed with an intention to perform well in\na collaborative ensemble. Two techniques are presented for producing an\nensemble of parsers that collaborate. All of the ensemble members are created\nusing the same underlying parser induction algorithm, and the method for\nproducing complementary parsers is only loosely constrained by that chosen\nalgorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006017v1",
            "date": "2000-06-09 17:28:40+00:00",
            "title": "Turning Speech Into Scripts",
            "authors": [
                "Manny Rayner",
                "Beth Ann Hockey",
                "Frankie James"
            ],
            "abstract": "We describe an architecture for implementing spoken natural language dialogue\ninterfaces to semi-autonomous systems, in which the central idea is to\ntransform the input speech signal through successive levels of representation\ncorresponding roughly to linguistic knowledge, dialogue knowledge, and domain\nknowledge. The final representation is an executable program in a simple\nscripting language equivalent to a subset of Cshell. At each stage of the\ntranslation process, an input is transformed into an output, producing as a\nbyproduct a \"meta-output\" which describes the nature of the transformation\nperformed. We show how consistent use of the output/meta-output distinction\npermits a simple and perspicuous treatment of apparently diverse topics\nincluding resolution of pronouns, correction of user misconceptions, and\noptimization of scripts. The methods described have been concretely realized in\na prototype speech interface to a simulation of the Personal Satellite\nAssistant.",
            "journal ref": "AAAI Spring Symposium on Natural Dialogues with Practical Robotic\n  Devices, March 20-22, 2000. Stanford, CA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006019v1",
            "date": "2000-06-09 21:41:54+00:00",
            "title": "A Compact Architecture for Dialogue Management Based on Scripts and Meta-Outputs",
            "authors": [
                "Manny Rayner",
                "Beth Ann Hockey",
                "Frankie James"
            ],
            "abstract": "We describe an architecture for spoken dialogue interfaces to semi-autonomous\nsystems that transforms speech signals through successive representations of\nlinguistic, dialogue, and domain knowledge. Each step produces an output, and a\nmeta-output describing the transformation, with an executable program in a\nsimple scripting language as the final result. The output/meta-output\ndistinction permits perspicuous treatment of diverse tasks such as resolving\npronouns, correcting user misconceptions, and optimizing scripts.",
            "journal ref": "Language Technology Joint Conference ANLP-NAACL 2000. 29 April - 4\n  May 2000, Seattle, WA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006020v1",
            "date": "2000-06-09 20:49:03+00:00",
            "title": "A Comparison of the XTAG and CLE Grammars for English",
            "authors": [
                "Beth Ann Hockey",
                "Manny Rayner",
                "Frankie James"
            ],
            "abstract": "When people develop something intended as a large broad-coverage grammar,\nthey usually have a more specific goal in mind. Sometimes this goal is covering\na corpus; sometimes the developers have theoretical ideas they wish to\ninvestigate; most often, work is driven by a combination of these two main\ntypes of goal. What tends to happen after a while is that the community of\npeople working with the grammar starts thinking of some phenomena as\n``central'', and makes serious efforts to deal with them; other phenomena are\nlabelled ``marginal'', and ignored. Before long, the distinction between\n``central'' and ``marginal'' becomes so ingrained that it is automatic, and\npeople virtually stop thinking about the ``marginal'' phenomena. In practice,\nthe only way to bring the marginal things back into focus is to look at what\nother people are doing and compare it with one's own work. In this paper, we\nwill take two large grammars, XTAG and the CLE, and examine each of them from\nthe other's point of view. We will find in both cases not only that important\nthings are missing, but that the perspective offered by the other grammar\nsuggests simple and practical ways of filling in the holes. It turns out that\nthere is a pleasing symmetry to the picture. XTAG has a very good treatment of\ncomplement structure, which the CLE to some extent lacks; conversely, the CLE\noffers a powerful and general account of adjuncts, which the XTAG grammar does\nnot fully duplicate. If we examine the way in which each grammar does the thing\nit is good at, we find that the relevant methods are quite easy to port to the\nother framework, and in fact only involve generalization and systematization of\nexisting mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006021v1",
            "date": "2000-06-09 22:03:10+00:00",
            "title": "Compiling Language Models from a Linguistically Motivated Unification Grammar",
            "authors": [
                "Manny Rayner",
                "Beth Ann Hockey",
                "Frankie James",
                "Elizabeth O. Bratt",
                "Sharon Goldwater",
                "Mark Gawron"
            ],
            "abstract": "Systems now exist which are able to compile unification grammars into\nlanguage models that can be included in a speech recognizer, but it is so far\nunclear whether non-trivial linguistically principled grammars can be used for\nthis purpose. We describe a series of experiments which investigate the\nquestion empirically, by incrementally constructing a grammar and discovering\nwhat problems emerge when successively larger versions are compiled into finite\nstate graph representations and used as language models for a medium-vocabulary\nrecognition task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006023v2",
            "date": "2000-06-11 06:06:10+00:00",
            "title": "Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech",
            "authors": [
                "A. Stolcke",
                "K. Ries",
                "N. Coccaro",
                "E. Shriberg",
                "R. Bates",
                "D. Jurafsky",
                "P. Taylor",
                "R. Martin",
                "C. Van Ess-Dykema",
                "M. Meteer"
            ],
            "abstract": "We describe a statistical approach for modeling dialogue acts in\nconversational speech, i.e., speech-act-like units such as Statement, Question,\nBackchannel, Agreement, Disagreement, and Apology. Our model detects and\npredicts dialogue acts based on lexical, collocational, and prosodic cues, as\nwell as on the discourse coherence of the dialogue act sequence. The dialogue\nmodel is based on treating the discourse structure of a conversation as a\nhidden Markov model and the individual dialogue acts as observations emanating\nfrom the model states. Constraints on the likely sequence of dialogue acts are\nmodeled via a dialogue act n-gram. The statistical dialogue grammar is combined\nwith word n-grams, decision trees, and neural networks modeling the\nidiosyncratic lexical and prosodic manifestations of each dialogue act. We\ndevelop a probabilistic integration of speech recognition with dialogue\nmodeling, to improve both speech recognition and dialogue act classification\naccuracy. Models are trained and evaluated using a large hand-labeled database\nof 1,155 conversations from the Switchboard corpus of spontaneous\nhuman-to-human telephone speech. We achieved good dialogue act labeling\naccuracy (65% based on errorful, automatically recognized words and prosody,\nand 71% based on word transcripts, compared to a chance baseline accuracy of\n35% and human accuracy of 84%) and a small reduction in word recognition error.",
            "journal ref": "Computational Linguistics 26(3), 339-373, September 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006024v1",
            "date": "2000-06-11 06:00:11+00:00",
            "title": "Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?",
            "authors": [
                "E. Shriberg",
                "R. Bates",
                "A. Stolcke",
                "P. Taylor",
                "D. Jurafsky",
                "K. Ries",
                "N. Coccaro",
                "R. Martin",
                "M. Meteer",
                "C. Van Ess-Dykema"
            ],
            "abstract": "Identifying whether an utterance is a statement, question, greeting, and so\nforth is integral to effective automatic understanding of natural dialog.\nLittle is known, however, about how such dialog acts (DAs) can be automatically\nclassified in truly natural conversation. This study asks whether current\napproaches, which use mainly word information, could be improved by adding\nprosodic information. The study is based on more than 1000 conversations from\nthe Switchboard corpus. DAs were hand-annotated, and prosodic features\n(duration, pause, F0, energy, and speaking rate) were automatically extracted\nfor each DA. In training, decision trees based on these features were inferred;\ntrees were then applied to unseen test data to evaluate performance.\nPerformance was evaluated for prosody models alone, and after combining the\nprosody models with word information -- either from true words or from the\noutput of an automatic speech recognizer. For an overall classification task,\nas well as three subtasks, prosody made significant contributions to\nclassification. Feature-specific analyses further revealed that although\ncanonical features (such as F0 for questions) were important, less obvious\nfeatures could compensate if canonical features were removed. Finally, in each\ntask, integrating the prosodic model with a DA-specific statistical language\nmodel improved performance over that of the language model alone, especially\nfor the case of recognized words. Results suggest that DAs are redundantly\nmarked in natural conversation, and that a variety of automatically extractable\nprosodic features could aid dialog processing in speech applications.",
            "journal ref": "Language and Speech 41(3-4), 439-487, 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006025v1",
            "date": "2000-06-11 18:20:20+00:00",
            "title": "Entropy-based Pruning of Backoff Language Models",
            "authors": [
                "A. Stolcke"
            ],
            "abstract": "A criterion for pruning parameters from N-gram backoff language models is\ndeveloped, based on the relative entropy between the original and the pruned\nmodel. It is shown that the relative entropy resulting from pruning a single\nN-gram can be computed exactly and efficiently for backoff models. The relative\nentropy measure can be expressed as a relative change in training set\nperplexity. This leads to a simple pruning criterion whereby all N-grams that\nchange perplexity by less than a threshold are removed from the model.\nExperiments show that a production-quality Hub4 LM can be reduced to 26% its\noriginal size without increasing recognition error. We also compare the\napproach to a heuristic pruning criterion by Seymore and Rosenfeld (1996), and\nshow that their approach can be interpreted as an approximation to the relative\nentropy criterion. Experimentally, both approaches select similar sets of\nN-grams (about 85% overlap), with the exact relative entropy criterion giving\nmarginally better performance.",
            "journal ref": "Proceedings DARPA Broadcast News Transcription and Understanding\n  Workshop, pp. 270-274, Lansdowne, VA, 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006028v1",
            "date": "2000-06-13 21:06:27+00:00",
            "title": "Trainable Methods for Surface Natural Language Generation",
            "authors": [
                "Adwait Ratnaparkhi"
            ],
            "abstract": "We present three systems for surface natural language generation that are\ntrainable from annotated corpora. The first two systems, called NLG1 and NLG2,\nrequire a corpus marked only with domain-specific semantic attributes, while\nthe last system, called NLG3, requires a corpus marked with both semantic\nattributes and syntactic dependency information. All systems attempt to produce\na grammatical natural language phrase from a domain-specific semantic\nrepresentation. NLG1 serves a baseline system and uses phrase frequencies to\ngenerate a whole phrase in one step, while NLG2 and NLG3 use maximum entropy\nprobability models to individually generate each word in the phrase. The\nsystems NLG2 and NLG3 learn to determine both the word choice and the word\norder of the phrase. We present experiments in which we generate phrases to\ndescribe flights in the air travel domain.",
            "journal ref": "Proceedings of the 1st Meeting of the North American Chapter of\n  the Association for Computational Linguistics (NAACL 2000). Pages 194--201",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006036v1",
            "date": "2000-06-27 04:39:57+00:00",
            "title": "Prosody-Based Automatic Segmentation of Speech into Sentences and Topics",
            "authors": [
                "E. Shriberg",
                "A. Stolcke",
                "D. Hakkani-Tur",
                "G. Tur"
            ],
            "abstract": "A crucial step in processing speech audio data for information extraction,\ntopic detection, or browsing/playback is to segment the input into sentence and\ntopic units. Speech segmentation is challenging, since the cues typically\npresent for segmenting text (headers, paragraphs, punctuation) are absent in\nspoken language. We investigate the use of prosody (information gleaned from\nthe timing and melody of speech) for these tasks. Using decision tree and\nhidden Markov modeling techniques, we combine prosodic cues with word-based\napproaches, and evaluate performance on two speech corpora, Broadcast News and\nSwitchboard. Results show that the prosodic model alone performs on par with,\nor better than, word-based statistical language models -- for both true and\nautomatically recognized words in news speech. The prosodic model achieves\ncomparable performance with significantly less training data, and requires no\nhand-labeling of prosodic events. Across tasks and corpora, we obtain a\nsignificant improvement over word-only models using a probabilistic combination\nof prosodic and lexical information. Inspection reveals that the prosodic\nmodels capture language-independent boundary indicators described in the\nliterature. Finally, cue usage is task and corpus dependent. For example, pause\nand pitch features are highly informative for segmenting news speech, whereas\npause, duration and word-based cues dominate for natural conversation.",
            "journal ref": "Speech Communication 32(1-2), 127-154, September 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006038v1",
            "date": "2000-06-28 10:06:02+00:00",
            "title": "Approximation and Exactness in Finite State Optimality Theory",
            "authors": [
                "Dale Gerdemann",
                "Gertjan van Noord"
            ],
            "abstract": "Previous work (Frank and Satta 1998; Karttunen, 1998) has shown that\nOptimality Theory with gradient constraints generally is not finite state. A\nnew finite-state treatment of gradient constraints is presented which improves\nupon the approximation of Karttunen (1998). The method turns out to be exact,\nand very compact, for the syllabification analysis of Prince and Smolensky\n(1993).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006044v1",
            "date": "2000-06-30 13:22:33+00:00",
            "title": "Finite-State Non-Concatenative Morphotactics",
            "authors": [
                "Kenneth R. Beesley",
                "Lauri Karttunen"
            ],
            "abstract": "Finite-state morphology in the general tradition of the Two-Level and Xerox\nimplementations has proved very successful in the production of robust\nmorphological analyzer-generators, including many large-scale commercial\nsystems. However, it has long been recognized that these implementations have\nserious limitations in handling non-concatenative phenomena. We describe a new\ntechnique for constructing finite-state transducers that involves reapplying\nthe regular-expression compiler to its own output. Implemented in an algorithm\ncalled compile-replace, this technique has proved useful for handling\nnon-concatenative phenomena; and we demonstrate it on Malay full-stem\nreduplication and Arabic stem interdigitation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007009v1",
            "date": "2000-07-06 14:15:26+00:00",
            "title": "Incremental construction of minimal acyclic finite-state automata",
            "authors": [
                "Jan Daciuk",
                "Stoyan Mihov",
                "Bruce Watson",
                "Richard Watson"
            ],
            "abstract": "In this paper, we describe a new method for constructing minimal,\ndeterministic, acyclic finite-state automata from a set of strings. Traditional\nmethods consist of two phases: the first to construct a trie, the second one to\nminimize it. Our approach is to construct a minimal automaton in a single phase\nby adding new strings one by one and minimizing the resulting automaton\non-the-fly. We present a general algorithm as well as a specialization that\nrelies upon the lexicographical ordering of the input strings.",
            "journal ref": "Computational Linguistics, Vol. 26, Number 1, March 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007018v1",
            "date": "2000-07-13 12:46:00+00:00",
            "title": "Bootstrapping a Tagged Corpus through Combination of Existing Heterogeneous Taggers",
            "authors": [
                "Jakub Zavrel",
                "Walter Daelemans"
            ],
            "abstract": "This paper describes a new method, Combi-bootstrap, to exploit existing\ntaggers and lexical resources for the annotation of corpora with new tagsets.\nCombi-bootstrap uses existing resources as features for a second level machine\nlearning module, that is trained to make the mapping to the new tagset on a\nvery small sample of annotated corpus material. Experiments show that\nCombi-bootstrap: i) can integrate a wide variety of existing resources, and ii)\nachieves much higher accuracy (up to 44.7 % error reduction) than both the best\nsingle tagger and an ensemble tagger constructed out of the same small training\nsample.",
            "journal ref": "Proceedings of the 2nd International Conference on Language\n  Resources and Evaluation (LREC 2000), pp. 17--20",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007022v1",
            "date": "2000-07-13 18:26:38+00:00",
            "title": "ATLAS: A flexible and extensible architecture for linguistic annotation",
            "authors": [
                "Steven Bird",
                "David Day",
                "John Garofolo",
                "John Henderson",
                "Christophe Laprun",
                "Mark Liberman"
            ],
            "abstract": "We describe a formal model for annotating linguistic artifacts, from which we\nderive an application programming interface (API) to a suite of tools for\nmanipulating these annotations. The abstract logical model provides for a range\nof storage formats and promotes the reuse of tools that interact through this\nAPI. We focus first on ``Annotation Graphs,'' a graph model for annotations on\nlinear signals (such as text and speech) indexed by intervals, for which\nefficient database storage and querying techniques are applicable. We note how\na wide range of existing annotated corpora can be mapped to this annotation\ngraph model. This model is then generalized to encompass a wider variety of\nlinguistic ``signals,'' including both naturally occuring phenomena (as\nrecorded in images, video, multi-modal interactions, etc.), as well as the\nderived resources that are increasingly important to the engineering of natural\nlanguage processing systems (such as word lists, dictionaries, aligned\nbilingual corpora, etc.). We conclude with a review of the current efforts\ntowards implementing key pieces of this architecture.",
            "journal ref": "Proceedings of the Second International Conference on Language\n  Resources and Evaluation, pp. 1699-1706, Paris: European Language Resources\n  Association, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007024v1",
            "date": "2000-07-13 18:51:48+00:00",
            "title": "Many uses, many annotations for large speech corpora: Switchboard and TDT as case studies",
            "authors": [
                "David Graff",
                "Steven Bird"
            ],
            "abstract": "This paper discusses the challenges that arise when large speech corpora\nreceive an ever-broadening range of diverse and distinct annotations. Two case\nstudies of this process are presented: the Switchboard Corpus of telephone\nconversations and the TDT2 corpus of broadcast news. Switchboard has undergone\ntwo independent transcriptions and various types of additional annotation, all\ncarried out as separate projects that were dispersed both geographically and\nchronologically. The TDT2 corpus has also received a variety of annotations,\nbut all directly created or managed by a core group. In both cases, issues\narise involving the propagation of repairs, consistency of references, and the\nability to integrate annotations having different formats and levels of detail.\nWe describe a general framework whereby these issues can be addressed\nsuccessfully.",
            "journal ref": "Proceedings of the Second International Conference on Language\n  Resources and Evaluation, pp. 427-433, Paris: European Language Resources\n  Association, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007031v2",
            "date": "2000-07-21 06:07:31+00:00",
            "title": "Parameter-free Model of Rank Polysemantic Distribution",
            "authors": [
                "Victor Kromer"
            ],
            "abstract": "A model of rank polysemantic distribution with a minimal number of fitting\nparameters is offered. In an ideal case a parameter-free description of the\ndependence on the basis of one or several immediate features of the\ndistribution is possible.",
            "journal ref": "Proceedings of the 4th conference of the International\n  Quantitative Linguistics Association (QUALICO 2000). Prague, August 24-26,\n  2000. P. 21-22.The full version (in Russian) is available in Web Journal\n  FCCL. See URL http://fccl.ksu.ru/fcclpap.htm",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007035v1",
            "date": "2000-07-25 17:20:47+00:00",
            "title": "Mapping WordNets Using Structural Information",
            "authors": [
                "J. Daude",
                "L. Padro",
                "G. Rigau"
            ],
            "abstract": "We present a robust approach for linking already existing lexical/semantic\nhierarchies. We used a constraint satisfaction algorithm (relaxation labeling)\nto select --among a set of candidates-- the node in a target taxonomy that\nbests matches each node in a source taxonomy. In particular, we use it to map\nthe nominal part of WordNet 1.5 onto WordNet 1.6, with a very high precision\nand a very low remaining ambiguity.",
            "journal ref": "38th Anual Meeting of the Association for Computational\n  Linguistics (ACL'2000). Hong Kong, October 2000.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007036v1",
            "date": "2000-07-25 17:40:00+00:00",
            "title": "Language identification of controlled systems: Modelling, control and anomaly detection",
            "authors": [
                "J. F. Martins",
                "J. A. Dente",
                "A. J. Pires",
                "R. Vilela Mendes"
            ],
            "abstract": "Formal language techniques have been used in the past to study autonomous\ndynamical systems. However, for controlled systems, new features are needed to\ndistinguish between information generated by the system and input control. We\nshow how the modelling framework for controlled dynamical systems leads\nnaturally to a formulation in terms of context-dependent grammars. A learning\nalgorithm is proposed for on-line generation of the grammar productions, this\nformulation being then used for modelling, control and anomaly detection.\nPractical applications are described for electromechanical drives. Grammatical\ninterpolation techniques yield accurate results and the pattern detection\ncapabilities of the language-based formulation makes it a promising technique\nfor the early detection of anomalies or faulty behaviour.",
            "journal ref": "IEEE Trans. in Systems, Man and Cybernetics 31 (2001) 234",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008003v1",
            "date": "2000-08-07 16:06:15+00:00",
            "title": "Interfacing Constraint-Based Grammars and Generation Algorithms",
            "authors": [
                "Stephan Busemann"
            ],
            "abstract": "Constraint-based grammars can, in principle, serve as the major linguistic\nknowledge source for both parsing and generation. Surface generation starts\nfrom input semantics representations that may vary across grammars. For many\ndeclarative grammars, the concept of derivation implicitly built in is that of\nparsing. They may thus not be interpretable by a generation algorithm. We show\nthat linguistically plausible semantic analyses can cause severe problems for\nsemantic-head-driven approaches for generation (SHDG). We use SeReal, a variant\nof SHDG and the DISCO grammar of German as our source of examples. We propose a\nnew, general approach that explicitly accounts for the interface between the\ngrammar and the generation algorithm by adding a control-oriented layer to the\nlinguistic knowledge base that reorganizes the semantics in a way suitable for\ngeneration.",
            "journal ref": "Proc. Workshop on Analysis for Generation, 1st International\n  Natural Language Generation Conference, Mitzpe Ramon, Israel, June 12, 2000.\n  pp. 14-21",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008004v1",
            "date": "2000-08-08 22:42:51+00:00",
            "title": "Comparing two trainable grammatical relations finders",
            "authors": [
                "Alexander Yeh"
            ],
            "abstract": "Grammatical relationships (GRs) form an important level of natural language\nprocessing, but different sets of GRs are useful for different purposes.\nTherefore, one may often only have time to obtain a small training corpus with\nthe desired GR annotations. On such a small training corpus, we compare two\nsystems. They use different learning techniques, but we find that this\ndifference by itself only has a minor effect. A larger factor is that in\nEnglish, a different GR length measure appears better suited for finding simple\nargument GRs than for finding modifier GRs. We also find that partitioning the\ndata may help memory-based learning.",
            "journal ref": "18th International Conference on Computational Linguistics (COLING\n  2000), pages 1146-1150, Saarbruecken, Germany, July, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008005v1",
            "date": "2000-08-08 23:52:02+00:00",
            "title": "More accurate tests for the statistical significance of result differences",
            "authors": [
                "Alexander Yeh"
            ],
            "abstract": "Statistical significance testing of differences in values of metrics like\nrecall, precision and balanced F-score is a necessary part of empirical natural\nlanguage processing. Unfortunately, we find in a set of experiments that many\ncommonly used tests often underestimate the significance and so are less likely\nto detect differences that exist between different techniques. This\nunderestimation comes from an independence assumption that is often violated.\nWe point out some useful tests that do not make this assumption, including\ncomputationally-intensive randomization tests.",
            "journal ref": "18th International Conference on Computational Linguistics (COLING\n  2000), pages 947-953, Saarbruecken, Germany, July, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008007v1",
            "date": "2000-08-10 03:21:33+00:00",
            "title": "Tagger Evaluation Given Hierarchical Tag Sets",
            "authors": [
                "I. Dan Melamed",
                "Philip Resnik"
            ],
            "abstract": "We present methods for evaluating human and automatic taggers that extend\ncurrent practice in three ways. First, we show how to evaluate taggers that\nassign multiple tags to each test instance, even if they do not assign\nprobabilities. Second, we show how to accommodate a common property of manually\nconstructed ``gold standards'' that are typically used for objective\nevaluation, namely that there is often more than one correct answer. Third, we\nshow how to measure performance when the set of possible tags is\ntree-structured in an IS-A hierarchy. To illustrate how our methods can be used\nto measure inter-annotator agreement, we show how to compute the kappa\ncoefficient over hierarchical tag sets.",
            "journal ref": "Computers and the Humanities 34(1-2). Special issue on SENSEVAL.\n  pp. 79-84",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008012v1",
            "date": "2000-08-17 13:13:42+00:00",
            "title": "Applying System Combination to Base Noun Phrase Identification",
            "authors": [
                "Erik F. Tjong Kim Sang",
                "Walter Daelemans",
                "Herve Dejean",
                "Rob Koeling",
                "Yuval Krymolowski",
                "Vasin Punyakanok",
                "Dan Roth"
            ],
            "abstract": "We use seven machine learning algorithms for one task: identifying base noun\nphrases. The results have been processed by different system combination\nmethods and all of these outperformed the best individual result. We have\napplied the seven learners with the best combinator, a majority vote of the top\nfive systems, to a standard data set and managed to improve the best published\nresult for this data set.",
            "journal ref": "Proceedings of COLING 2000, Saarbruecken, Germany",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008013v1",
            "date": "2000-08-18 12:06:36+00:00",
            "title": "Meta-Learning for Phonemic Annotation of Corpora",
            "authors": [
                "Veronique Hoste",
                "Walter Daelemans",
                "Erik Tjong Kim Sang",
                "Steven Gillis"
            ],
            "abstract": "We apply rule induction, classifier combination and meta-learning (stacked\nclassifiers) to the problem of bootstrapping high accuracy automatic annotation\nof corpora with pronunciation information. The task we address in this paper\nconsists of generating phonemic representations reflecting the Flemish and\nDutch pronunciations of a word on the basis of its orthographic representation\n(which in turn is based on the actual speech recordings). We compare several\npossible approaches to achieve the text-to-pronunciation mapping task:\nmemory-based learning, transformation-based learning, rule induction, maximum\nentropy modeling, combination of classifiers in stacked learning, and stacking\nof meta-learners. We are interested both in optimal accuracy and in obtaining\ninsight into the linguistic regularities involved. As far as accuracy is\nconcerned, an already high accuracy level (93% for Celex and 86% for Fonilex at\nword level) for single classifiers is boosted significantly with additional\nerror reductions of 31% and 38% respectively using combination of classifiers,\nand a further 5% using combination of meta-learners, bringing overall word\nlevel accuracy to 96% for the Dutch variant and 92% for the Flemish variant. We\nalso show that the application of machine learning methods indeed leads to\nincreased insight into the linguistic regularities determining the variation\nbetween the two pronunciation variants studied.",
            "journal ref": "Proceedings of ICML-2000, Stanford University, CA, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008014v1",
            "date": "2000-08-18 13:56:10+00:00",
            "title": "Aspects of Pattern-Matching in Data-Oriented Parsing",
            "authors": [
                "Guy De Pauw"
            ],
            "abstract": "Data-Oriented Parsing (dop) ranks among the best parsing schemes, pairing\nstate-of-the art parsing accuracy to the psycholinguistic insight that larger\nchunks of syntactic structures are relevant grammatical and probabilistic\nunits. Parsing with the dop-model, however, seems to involve a lot of CPU\ncycles and a considerable amount of double work, brought on by the concept of\nmultiple derivations, which is necessary for probabilistic processing, but\nwhich is not convincingly related to a proper linguistic backbone. It is\nhowever possible to re-interpret the dop-model as a pattern-matching model,\nwhich tries to maximize the size of the substructures that construct the parse,\nrather than the probability of the parse. By emphasizing this memory-based\naspect of the dop-model, it is possible to do away with multiple derivations,\nopening up possibilities for efficient Viterbi-style optimizations, while still\nretaining acceptable parsing accuracy through enhanced context-sensitivity.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008015v1",
            "date": "2000-08-18 16:07:17+00:00",
            "title": "Temiar Reduplication in One-Level Prosodic Morphology",
            "authors": [
                "Markus Walther"
            ],
            "abstract": "Temiar reduplication is a difficult piece of prosodic morphology. This paper\npresents the first computational analysis of Temiar reduplication, using the\nnovel finite-state approach of One-Level Prosodic Morphology originally\ndeveloped by Walther (1999b, 2000). After reviewing both the data and the basic\ntenets of One-level Prosodic Morphology, the analysis is laid out in some\ndetail, using the notation of the FSA Utilities finite-state toolkit (van Noord\n1997). One important discovery is that in this approach one can easily define a\nregular expression operator which ambiguously scans a string in the left- or\nrightward direction for a certain prosodic property. This yields an elegant\naccount of base-length-dependent triggering of reduplication as found in\nTemiar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008017v1",
            "date": "2000-08-21 19:27:18+00:00",
            "title": "Efficient probabilistic top-down and left-corner parsing",
            "authors": [
                "Brian Roark",
                "Mark Johnson"
            ],
            "abstract": "This paper examines efficient predictive broad-coverage parsing without\ndynamic programming. In contrast to bottom-up methods, depth-first top-down\nparsing produces partial parses that are fully connected trees spanning the\nentire left context, from which any kind of non-local dependency or partial\nsemantic interpretation can in principle be read. We contrast two predictive\nparsing approaches, top-down and left-corner parsing, and find both to be\nviable. In addition, we find that enhancement with non-local information not\nonly improves parser accuracy, but also substantially improves the search\nefficiency.",
            "journal ref": "Proceedings of the 37th Annual Meeting of the Association for\n  Computational Linguistics, 1999, pages 421-428",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008021v1",
            "date": "2000-08-22 15:16:22+00:00",
            "title": "Compact non-left-recursive grammars using the selective left-corner transform and factoring",
            "authors": [
                "Mark Johnson",
                "Brian Roark"
            ],
            "abstract": "The left-corner transform removes left-recursion from (probabilistic)\ncontext-free grammars and unification grammars, permitting simple top-down\nparsing techniques to be used. Unfortunately the grammars produced by the\nstandard left-corner transform are usually much larger than the original. The\nselective left-corner transform described in this paper produces a transformed\ngrammar which simulates left-corner recognition of a user-specified set of the\noriginal productions, and top-down recognition of the others. Combined with two\nfactorizations, it produces non-left-recursive grammars that are not much\nlarger than the original.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics (COLING), 2000, pages 355-361",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008023v1",
            "date": "2000-08-23 07:38:22+00:00",
            "title": "Selectional Restrictions in HPSG",
            "authors": [
                "Ion Androutsopoulos",
                "Robert Dale"
            ],
            "abstract": "Selectional restrictions are semantic sortal constraints imposed on the\nparticipants of linguistic constructions to capture contextually-dependent\nconstraints on interpretation. Despite their limitations, selectional\nrestrictions have proven very useful in natural language applications, where\nthey have been used frequently in word sense disambiguation, syntactic\ndisambiguation, and anaphora resolution. Given their practical value, we\nexplore two methods to incorporate selectional restrictions in the HPSG theory,\nassuming that the reader is familiar with HPSG. The first method employs HPSG's\nBackground feature and a constraint-satisfaction component pipe-lined after the\nparser. The second method uses subsorts of referential indices, and blocks\nreadings that violate selectional restrictions during parsing. While\ntheoretically less satisfactory, we have found the second method particularly\nuseful in the development of practical systems.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics (COLING), Saarbrucken, Germany, 31 July - 4 August 2000, pages\n  15-20",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008024v1",
            "date": "2000-08-23 12:38:08+00:00",
            "title": "Estimation of Stochastic Attribute-Value Grammars using an Informative Sample",
            "authors": [
                "Miles Osborne"
            ],
            "abstract": "We argue that some of the computational complexity associated with estimation\nof stochastic attribute-value grammars can be reduced by training upon an\ninformative subset of the full training set. Results using the parsed Wall\nStreet Journal corpus show that in some circumstances, it is possible to obtain\nbetter estimation results using an informative sample than when training upon\nall the available material. Further experimentation demonstrates that with\nunlexicalised models, a Gaussian Prior can reduce overfitting. However, when\nmodels are lexicalised and contain overlapping features, overfitting does not\nseem to be a problem, and a Gaussian Prior makes minimal difference to\nperformance. Our approach is applicable for situations when there are an\ninfeasibly large number of parses in the training set, or else for when\nrecovery of these parses from a packed representation is itself computationally\nexpensive.",
            "journal ref": "Coling 2000, Saarbr\\\"{u}cken, Germany. pp 586--592",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008026v1",
            "date": "2000-08-24 13:28:25+00:00",
            "title": "Noun-phrase co-occurrence statistics for semi-automatic semantic lexicon construction",
            "authors": [
                "Brian Roark",
                "Eugene Charniak"
            ],
            "abstract": "Generating semantic lexicons semi-automatically could be a great time saver,\nrelative to creating them by hand. In this paper, we present an algorithm for\nextracting potential entries for a category from an on-line corpus, based upon\na small set of exemplars. Our algorithm finds more correct terms and fewer\nincorrect ones than previous work in this area. Additionally, the entries that\nare generated potentially provide broader coverage of the category than would\noccur to an individual coding them by hand. Our algorithm finds many terms not\nincluded within Wordnet (many more than previous algorithms), and could be\nviewed as an ``enhancer'' of existing broad-coverage resources.",
            "journal ref": "Proceedings of the 36th Annual Meeting of the Association for\n  Computational Linguistics and 17th International Conference on Computational\n  Linguistics (COLING-ACL), 1998, pages 1110-1116",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008027v1",
            "date": "2000-08-24 16:38:53+00:00",
            "title": "Measuring efficiency in high-accuracy, broad-coverage statistical parsing",
            "authors": [
                "Brian Roark",
                "Eugene Charniak"
            ],
            "abstract": "Very little attention has been paid to the comparison of efficiency between\nhigh accuracy statistical parsers. This paper proposes one machine-independent\nmetric that is general enough to allow comparisons across very different\nparsing architectures. This metric, which we call ``events considered'',\nmeasures the number of ``events'', however they are defined for a particular\nparser, for which a probability must be calculated, in order to find the parse.\nIt is applicable to single-pass or multi-stage parsers. We discuss the\nadvantages of the metric, and demonstrate its usefulness by using it to compare\ntwo parsers which differ in several fundamental ways.",
            "journal ref": "Proceedings of the COLING 2000 Workshop on Efficiency in\n  Large-Scale Parsing Systems, 2000, pages 29-36",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008028v1",
            "date": "2000-08-25 17:23:07+00:00",
            "title": "Estimators for Stochastic ``Unification-Based'' Grammars",
            "authors": [
                "Mark Johnson",
                "Stuart Geman",
                "Stephen Canon",
                "Zhiyi Chi",
                "Stefan Riezler"
            ],
            "abstract": "Log-linear models provide a statistically sound framework for Stochastic\n``Unification-Based'' Grammars (SUBGs) and stochastic versions of other kinds\nof grammars. We describe two computationally-tractable ways of estimating the\nparameters of such grammars from a training corpus of syntactic analyses, and\napply these to estimate a stochastic version of Lexical-Functional Grammar.",
            "journal ref": "Proc 37th Annual Conference of the Association for Computational\n  Linguistics, 1999, pages 535-541",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008029v1",
            "date": "2000-08-25 17:31:53+00:00",
            "title": "Exploiting auxiliary distributions in stochastic unification-based grammars",
            "authors": [
                "Mark Johnson",
                "Stefan Riezler"
            ],
            "abstract": "This paper describes a method for estimating conditional probability\ndistributions over the parses of ``unification-based'' grammars which can\nutilize auxiliary distributions that are estimated by other means. We show how\nthis can be used to incorporate information about lexical selectional\npreferences gathered from other sources into Stochastic ``Unification-based''\nGrammars (SUBGs). While we apply this estimator to a Stochastic\nLexical-Functional Grammar, the method is general, and should be applicable to\nstochastic versions of HPSGs, categorial grammars and transformational\ngrammars.",
            "journal ref": "Proc 1st NAACL, 2000, pages 154-161",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008030v1",
            "date": "2000-08-28 08:10:14+00:00",
            "title": "Metonymy Interpretation Using X NO Y Examples",
            "authors": [
                "Masaki Murata",
                "Qing Ma",
                "Atsumu Yamamoto",
                "Hitoshi Isahara"
            ],
            "abstract": "We developed on example-based method of metonymy interpretation. One\nadvantages of this method is that a hand-built database of metonymy is not\nnecessary because it instead uses examples in the form ``Noun X no Noun Y (Noun\nY of Noun X).'' Another advantage is that we will be able to interpret\nnewly-coined metonymic sentences by using a new corpus. We experimented with\nmetonymy interpretation and obtained a precision rate of 66% when using this\nmethod.",
            "journal ref": "SNLP2000, Chiang Mai, Thailand, May 10, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008031v1",
            "date": "2000-08-28 08:17:18+00:00",
            "title": "Bunsetsu Identification Using Category-Exclusive Rules",
            "authors": [
                "Masaki Murata",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "This paper describes two new bunsetsu identification methods using supervised\nlearning. Since Japanese syntactic analysis is usually done after bunsetsu\nidentification, bunsetsu identification is important for analyzing Japanese\nsentences. In experiments comparing the four previously available\nmachine-learning methods (decision tree, maximum-entropy method, example-based\napproach and decision list) and two new methods using category-exclusive rules,\nthe new method using the category-exclusive rules with the highest similarity\nperformed best.",
            "journal ref": "COLING'2000, Saarbrucken, Germany, August, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008032v1",
            "date": "2000-08-28 08:27:22+00:00",
            "title": "Japanese Probabilistic Information Retrieval Using Location and Category Information",
            "authors": [
                "Masaki Murata",
                "Qing Ma",
                "Kiyotaka Uchimoto",
                "Hiromi Ozaku",
                "Masao Utiyama",
                "Hitoshi Isahara"
            ],
            "abstract": "Robertson's 2-poisson information retrieve model does not use location and\ncategory information. We constructed a framework using location and category\ninformation in a 2-poisson model. We submitted two systems based on this\nframework to the IREX contest, Japanese language information retrieval contest\nheld in Japan in 1999. For precision in the A-judgement measure they scored\n0.4926 and 0.4827, the highest values among the 15 teams and 22 systems that\nparticipated in the IREX contest. We describe our systems and the comparative\nexperiments done when various parameters were changed. These experiments\nconfirmed the effectiveness of using location and category information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008033v1",
            "date": "2000-08-28 19:51:32+00:00",
            "title": "Temporal Expressions in Japanese-to-English Machine Translation",
            "authors": [
                "Francis Bond",
                "Kentaro Ogura",
                "Hajime Uchino"
            ],
            "abstract": "This paper describes in outline a method for translating Japanese temporal\nexpressions into English. We argue that temporal expressions form a special\nsubset of language that is best handled as a special module in machine\ntranslation. The paper deals with problems of lexical idiosyncrasy as well as\nthe choice of articles and prepositions within temporal expressions. In\naddition temporal expressions are considered as parts of larger structures, and\nthe question of whether to translate them as noun phrases or adverbials is\naddressed.",
            "journal ref": "Seventh International Conference on Theoretical and Methodological\n  Issues in Machine Translation: TMI-97, Santa Fe, July 1997, pp 55--62",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008034v1",
            "date": "2000-08-30 13:14:58+00:00",
            "title": "Lexicalized Stochastic Modeling of Constraint-Based Grammars using Log-Linear Measures and EM Training",
            "authors": [
                "Stefan Riezler",
                "Detlef Prescher",
                "Jonas Kuhn",
                "Mark Johnson"
            ],
            "abstract": "We present a new approach to stochastic modeling of constraint-based grammars\nthat is based on log-linear models and uses EM for estimation from unannotated\ndata. The techniques are applied to an LFG grammar for German. Evaluation on an\nexact match task yields 86% precision for an ambiguity rate of 5.4, and 90%\nprecision on a subcat frame match for an ambiguity rate of 25. Experimental\ncomparison to training from a parsebank shows a 10% gain from EM training.\nAlso, a new class-based grammar lexicalization is presented, showing a 10% gain\nover unlexicalized models.",
            "journal ref": "Proceedings of the 38th Annual Meeting of the ACL, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008035v1",
            "date": "2000-08-30 13:24:06+00:00",
            "title": "Using a Probabilistic Class-Based Lexicon for Lexical Ambiguity Resolution",
            "authors": [
                "Detlef Prescher",
                "Stefan Riezler",
                "Mats Rooth"
            ],
            "abstract": "This paper presents the use of probabilistic class-based lexica for\ndisambiguation in target-word selection. Our method employs minimal but precise\ncontextual information for disambiguation. That is, only information provided\nby the target-verb, enriched by the condensed information of a probabilistic\nclass-based lexicon, is used. Induction of classes and fine-tuning to verbal\narguments is done in an unsupervised manner by EM-based clustering techniques.\nThe method shows promising results in an evaluation on real-world translations.",
            "journal ref": "Proceedings of the 18th COLING, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008036v1",
            "date": "2000-08-30 13:57:19+00:00",
            "title": "Probabilistic Constraint Logic Programming. Formal Foundations of Quantitative and Statistical Inference in Constraint-Based Natural Language Processing",
            "authors": [
                "Stefan Riezler"
            ],
            "abstract": "In this thesis, we present two approaches to a rigorous mathematical and\nalgorithmic foundation of quantitative and statistical inference in\nconstraint-based natural language processing. The first approach, called\nquantitative constraint logic programming, is conceptualized in a clear logical\nframework, and presents a sound and complete system of quantitative inference\nfor definite clauses annotated with subjective weights. This approach combines\na rigorous formal semantics for quantitative inference based on subjective\nweights with efficient weight-based pruning for constraint-based systems. The\nsecond approach, called probabilistic constraint logic programming, introduces\na log-linear probability distribution on the proof trees of a constraint logic\nprogram and an algorithm for statistical inference of the parameters and\nproperties of such probability models from incomplete, i.e., unparsed data. The\npossibility of defining arbitrary properties of proof trees as properties of\nthe log-linear probability model and efficiently estimating appropriate\nparameter values for them permits the probabilistic modeling of arbitrary\ncontext-dependencies in constraint logic programs. The usefulness of these\nideas is evaluated empirically in a small-scale experiment on finding the\ncorrect parses of a constraint-based grammar. In addition, we address the\nproblem of computational intractability of the calculation of expectations in\nthe inference task and present various techniques to approximately solve this\ntask. Moreover, we present an approximate heuristic technique for searching for\nthe most probable analysis in probabilistic constraint logic programs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009003v1",
            "date": "2000-09-08 15:48:53+00:00",
            "title": "Automatic Extraction of Subcategorization Frames for Czech",
            "authors": [
                "Anoop Sarkar",
                "Daniel Zeman"
            ],
            "abstract": "We present some novel machine learning techniques for the identification of\nsubcategorization information for verbs in Czech. We compare three different\nstatistical techniques applied to this problem. We show how the learning\nalgorithm can be used to discover previously unknown subcategorization frames\nfrom the Czech Prague Dependency Treebank. The algorithm can then be used to\nlabel dependents of a verb in the Czech treebank as either arguments or\nadjuncts. Using our techniques, we ar able to achieve 88% precision on unseen\nparsed text.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics (Coling 2000), Universit",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009008v1",
            "date": "2000-09-18 12:08:54+00:00",
            "title": "Introduction to the CoNLL-2000 Shared Task: Chunking",
            "authors": [
                "Erik F. Tjong Kim Sang",
                "Sabine Buchholz"
            ],
            "abstract": "We describe the CoNLL-2000 shared task: dividing text into syntactically\nrelated non-overlapping groups of words, so-called text chunking. We give\nbackground information on the data sets, present a general overview of the\nsystems that have taken part in the shared task and briefly discuss their\nperformance.",
            "journal ref": "Proceedings of CoNLL-2000 and LLL-2000, Lisbon, Portugal",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009011v1",
            "date": "2000-09-19 00:44:47+00:00",
            "title": "Anaphora Resolution in Japanese Sentences Using Surface Expressions and Examples",
            "authors": [
                "Masaki Murata"
            ],
            "abstract": "Anaphora resolution is one of the major problems in natural language\nprocessing. It is also one of the important tasks in machine translation and\nman/machine dialogue. We solve the problem by using surface expressions and\nexamples. Surface expressions are the words in sentences which provide clues\nfor anaphora resolution. Examples are linguistic data which are actually used\nin conversations and texts. The method using surface expressions and examples\nis a practical method. This thesis handles almost all kinds of anaphora: i. The\nreferential property and number of a noun phrase ii. Noun phrase direct\nanaphora iii. Noun phrase indirect anaphora iv. Pronoun anaphora v. Verb phrase\nellipsis",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009015v1",
            "date": "2000-09-20 13:23:17+00:00",
            "title": "A Tableaux Calculus for Ambiguous Quantification",
            "authors": [
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "Coping with ambiguity has recently received a lot of attention in natural\nlanguage processing. Most work focuses on the semantic representation of\nambiguous expressions. In this paper we complement this work in two ways.\nFirst, we provide an entailment relation for a language with ambiguous\nexpressions. Second, we give a sound and complete tableaux calculus for\nreasoning with statements involving ambiguous quantification. The calculus\ninterleaves partial disambiguation steps with steps in a traditional deductive\nprocess, so as to minimize and postpone branching in the proof process, and\nthereby increases its efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009025v1",
            "date": "2000-09-27 13:22:54+00:00",
            "title": "Parsing with the Shortest Derivation",
            "authors": [
                "Rens Bod"
            ],
            "abstract": "Common wisdom has it that the bias of stochastic grammars in favor of shorter\nderivations of a sentence is harmful and should be redressed. We show that the\ncommon wisdom is wrong for stochastic grammars that use elementary trees\ninstead of context-free rules, such as Stochastic Tree-Substitution Grammars\nused by Data-Oriented Parsing models. For such grammars a non-probabilistic\nmetric based on the shortest derivation outperforms a probabilistic metric on\nthe ATIS and OVIS corpora, while it obtains very competitive results on the\nWall Street Journal corpus. This paper also contains the first published\nexperiments with DOP on the Wall Street Journal.",
            "journal ref": "Proceedings COLING'2000, with a minor correction",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009026v1",
            "date": "2000-09-27 13:38:31+00:00",
            "title": "An improved parser for data-oriented lexical-functional analysis",
            "authors": [
                "Rens Bod"
            ],
            "abstract": "We present an LFG-DOP parser which uses fragments from LFG-annotated\nsentences to parse new sentences. Experiments with the Verbmobil and Homecentre\ncorpora show that (1) Viterbi n best search performs about 100 times faster\nthan Monte Carlo search while both achieve the same accuracy; (2) the DOP\nhypothesis which states that parse accuracy increases with increasing fragment\nsize is confirmed for LFG-DOP; (3) LFG-DOP's relative frequency estimator\nperforms worse than a discounted frequency estimator; and (4) LFG-DOP\nsignificantly outperforms Tree-DOP is evaluated on tree structures only.",
            "journal ref": "Proceedings ACL'2000, Hong Kong",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010012v1",
            "date": "2000-10-07 02:24:21+00:00",
            "title": "Finding consensus in speech recognition: word error minimization and other applications of confusion networks",
            "authors": [
                "L. Mangu",
                "E. Brill",
                "A. Stolcke"
            ],
            "abstract": "We describe a new framework for distilling information from word lattices to\nimprove the accuracy of speech recognition and obtain a more perspicuous\nrepresentation of a set of alternative hypotheses. In the standard MAP decoding\napproach the recognizer outputs the string of words corresponding to the path\nwith the highest posterior probability given the acoustics and a language\nmodel. However, even given optimal models, the MAP decoder does not necessarily\nminimize the commonly used performance metric, word error rate (WER). We\ndescribe a method for explicitly minimizing WER by extracting word hypotheses\nwith the highest posterior probabilities from word lattices. We change the\nstandard problem formulation by replacing global search over a large set of\nsentence hypotheses with local search over a small set of word candidates. In\naddition to improving the accuracy of the recognizer, our method produces a new\nrepresentation of the set of candidate hypotheses that specifies the sequence\nof word-level confusions in a compact lattice format. We study the properties\nof confusion networks and examine their use for other tasks, such as lattice\ncompression, word spotting, confidence annotation, and reevaluation of\nrecognition hypotheses using higher-level knowledge sources.",
            "journal ref": "Computer Speech and Language 14(4), 373-400, October 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010020v1",
            "date": "2000-10-11 22:30:09+00:00",
            "title": "Using existing systems to supplement small amounts of annotated grammatical relations training data",
            "authors": [
                "Alexander Yeh"
            ],
            "abstract": "Grammatical relationships (GRs) form an important level of natural language\nprocessing, but different sets of GRs are useful for different purposes.\nTherefore, one may often only have time to obtain a small training corpus with\nthe desired GR annotations. To boost the performance from using such a small\ntraining corpus on a transformation rule learner, we use existing systems that\nfind related types of annotations.",
            "journal ref": "38th Annual Meeting of the Association for Computational\n  Linguistics (ACL-2000), pages 126-132, Hong Kong, October, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010024v1",
            "date": "2000-10-17 08:31:48+00:00",
            "title": "Exploring automatic word sense disambiguation with decision lists and the Web",
            "authors": [
                "Eneko Agirre",
                "David Martinez"
            ],
            "abstract": "The most effective paradigm for word sense disambiguation, supervised\nlearning, seems to be stuck because of the knowledge acquisition bottleneck. In\nthis paper we take an in-depth study of the performance of decision lists on\ntwo publicly available corpora and an additional corpus automatically acquired\nfrom the Web, using the fine-grained highly polysemous senses in WordNet.\nDecision lists are shown a versatile state-of-the-art technique. The\nexperiments reveal, among other facts, that SemCor can be an acceptable (0.7\nprecision for polysemous words) starting point for an all-words system. The\nresults on the DSO corpus show that for some highly polysemous words 0.7\nprecision seems to be the current state-of-the-art limit. On the other hand,\nindependently constructed hand-tagged corpora are not mutually useful, and a\ncorpus automatically acquired from the Web is shown to fail.",
            "journal ref": "Procedings of the COLING 2000 Workshop on Semantic Annotation and\n  Intelligent Content",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010025v1",
            "date": "2000-10-17 09:05:27+00:00",
            "title": "Extraction of semantic relations from a Basque monolingual dictionary using Constraint Grammar",
            "authors": [
                "Eneko Agirre",
                "Olatz Ansa",
                "Xabier Arregi",
                "Xabier Artola",
                "Arantza Diaz de Ilarraza",
                "Mikel Lersundi",
                "David Martinez",
                "Kepa Sarasola",
                "Ruben Urizar"
            ],
            "abstract": "This paper deals with the exploitation of dictionaries for the semi-automatic\nconstruction of lexicons and lexical knowledge bases. The final goal of our\nresearch is to enrich the Basque Lexical Database with semantic information\nsuch as senses, definitions, semantic relations, etc., extracted from a Basque\nmonolingual dictionary. The work here presented focuses on the extraction of\nthe semantic relations that best characterise the headword, that is, those of\nsynonymy, antonymy, hypernymy, and other relations marked by specific relators\nand derivation. All nominal, verbal and adjectival entries were treated. Basque\nuses morphological inflection to mark case, and therefore semantic relations\nhave to be inferred from suffixes rather than from prepositions. Our approach\ncombines a morphological analyser and surface syntax parsing (based on\nConstraint Grammar), and has proven very successful for highly inflected\nlanguages such as Basque. Both the effort to write the rules and the actual\nprocessing time of the dictionary have been very low. At present we have\nextracted 42,533 relations, leaving only 2,943 (9%) definitions without any\nextracted relation. The error rate is extremely low, as only 2.2% of the\nextracted relations are wrong.",
            "journal ref": "Proceedings of EURALEX 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010026v1",
            "date": "2000-10-17 09:56:54+00:00",
            "title": "Enriching very large ontologies using the WWW",
            "authors": [
                "Eneko Agirre",
                "Olatz Ansa",
                "Eduard Hovy",
                "David Martinez"
            ],
            "abstract": "This paper explores the possibility to exploit text on the world wide web in\norder to enrich the concepts in existing ontologies. First, a method to\nretrieve documents from the WWW related to a concept is described. These\ndocument collections are used 1) to construct topic signatures (lists of\ntopically related words) for each concept in WordNet, and 2) to build\nhierarchical clusters of the concepts (the word senses) that lexicalize a given\nword. The overall goal is to overcome two shortcomings of WordNet: the lack of\ntopical links among concepts, and the proliferation of senses. Topic signatures\nare validated on a word sense disambiguation task with good results, which are\nimproved when the hierarchical clusters are used.",
            "journal ref": "Procedings of the ECAI 2000 Workshop on Ontology Learning",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010027v1",
            "date": "2000-10-17 10:26:33+00:00",
            "title": "One Sense per Collocation and Genre/Topic Variations",
            "authors": [
                "David Martinez",
                "Eneko Agirre"
            ],
            "abstract": "This paper revisits the one sense per collocation hypothesis using\nfine-grained sense distinctions and two different corpora. We show that the\nhypothesis is weaker for fine-grained sense distinctions (70% vs. 99% reported\nearlier on 2-way ambiguities). We also show that one sense per collocation does\nhold across corpora, but that collocations vary from one corpus to the other,\nfollowing genre and topic variations. This explains the low results when\nperforming word sense disambiguation across corpora. In fact, we demonstrate\nthat when two independent corpora share a related genre/topic, the word sense\ndisambiguation results would be better. Future work on word sense\ndisambiguation will have to take into account genre and topic as important\nparameters on their models.",
            "journal ref": "Proceedings of the Joint SIGDAT Conference on Empirical Methods in\n  Natural Language Processing and Very Large Corpora 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010030v1",
            "date": "2000-10-23 15:14:02+00:00",
            "title": "Reduction of Intermediate Alphabets in Finite-State Transducer Cascades",
            "authors": [
                "Andre Kempe"
            ],
            "abstract": "This article describes an algorithm for reducing the intermediate alphabets\nin cascades of finite-state transducers (FSTs). Although the method modifies\nthe component FSTs, there is no change in the overall relation described by the\nwhole cascade. No additional information or special algorithm, that could\ndecelerate the processing of input, is required at runtime. Two examples from\nNatural Language Processing are used to illustrate the effect of the algorithm\non the sizes of the FSTs and their alphabets. With some FSTs the number of arcs\nand symbols shrank considerably.",
            "journal ref": "Proc. TALN 2000, pp. 207-215, Lausanne, Switzerland. October 16-18",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011001v1",
            "date": "2000-11-02 09:02:12+00:00",
            "title": "Utilizing the World Wide Web as an Encyclopedia: Extracting Term Descriptions from Semi-Structured Texts",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "In this paper, we propose a method to extract descriptions of technical terms\nfrom Web pages in order to utilize the World Wide Web as an encyclopedia. We\nuse linguistic patterns and HTML text structures to extract text fragments\ncontaining term descriptions. We also use a language model to discard\nextraneous descriptions, and a clustering method to summarize resultant\ndescriptions. We show the effectiveness of our method by way of experiments.",
            "journal ref": "Proceedings of the 38th Annual Meeting of the Association for\n  Computational Linguistics (ACL-2000), pp.488-495, Oct. 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011002v1",
            "date": "2000-11-02 10:00:27+00:00",
            "title": "A Novelty-based Evaluation Method for Information Retrieval",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "In information retrieval research, precision and recall have long been used\nto evaluate IR systems. However, given that a number of retrieval systems\nresembling one another are already available to the public, it is valuable to\nretrieve novel relevant documents, i.e., documents that cannot be retrieved by\nthose existing systems. In view of this problem, we propose an evaluation\nmethod that favors systems retrieving as many novel documents as possible. We\nalso used our method to evaluate systems that participated in the IREX\nworkshop.",
            "journal ref": "Proceedings of the 2nd International Conference on Language\n  Resources and Evaluation (LREC-2000), pp.1637-1641, Jun. 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011003v1",
            "date": "2000-11-02 10:32:08+00:00",
            "title": "Applying Machine Translation to Two-Stage Cross-Language Information Retrieval",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "Cross-language information retrieval (CLIR), where queries and documents are\nin different languages, needs a translation of queries and/or documents, so as\nto standardize both of them into a common representation. For this purpose, the\nuse of machine translation is an effective approach. However, computational\ncost is prohibitive in translating large-scale document collections. To resolve\nthis problem, we propose a two-stage CLIR method. First, we translate a given\nquery into the document language, and retrieve a limited number of foreign\ndocuments. Second, we machine translate only those documents into the user\nlanguage, and re-rank them based on the translation result. We also show the\neffectiveness of our method by way of experiments using Japanese queries and\nEnglish technical documents.",
            "journal ref": "Proceedings of the 4th Conference of the Association for Machine\n  Translation in the Americas (AMTA-2000), pp.13-24, Oct. 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011020v2",
            "date": "2000-11-16 17:40:10+00:00",
            "title": "The Use of Instrumentation in Grammar Engineering",
            "authors": [
                "Norbert Broeker"
            ],
            "abstract": "This paper explores the usefulness of a technique from software engineering,\ncode instrumentation, for the development of large-scale natural language\ngrammars. Information about the usage of grammar rules in test and corpus\nsentences is used to improve grammar and testsuite, as well as adapting a\ngrammar to a specific genre. Results show that less than half of a\nlarge-coverage grammar for German is actually tested by two large testsuites,\nand that 10--30% of testing time is redundant. This methodology applied can be\nseen as a re-use of grammar writing knowledge for testsuite compilation.",
            "journal ref": "adapted from COLING2000, Saarbruecken/FRG, July31--Aug4 2000,\n  pp.118-124",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011034v1",
            "date": "2000-11-22 15:35:46+00:00",
            "title": "Semantic interpretation of temporal information by abductive inference",
            "authors": [
                "Sven Verdoolaege",
                "Marc Denecker",
                "Ness Schelkens",
                "Danny De Schreye",
                "Frank Van Eynde"
            ],
            "abstract": "Besides temporal information explicitly available in verbs and adjuncts, the\ntemporal interpretation of a text also depends on general world knowledge and\ndefault assumptions. We will present a theory for describing the relation\nbetween, on the one hand, verbs, their tenses and adjuncts and, on the other,\nthe eventualities and periods of time they represent and their relative\ntemporal locations.\n  The theory is formulated in logic and is a practical implementation of the\nconcepts described in Ness Schelkens et al. We will show how an abductive\nresolution procedure can be used on this representation to extract temporal\ninformation from texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011035v1",
            "date": "2000-11-23 08:41:52+00:00",
            "title": "Abductive reasoning with temporal information",
            "authors": [
                "Sven Verdoolaege",
                "Marc Denecker",
                "Frank Van Eynde"
            ],
            "abstract": "Texts in natural language contain a lot of temporal information, both\nexplicit and implicit. Verbs and temporal adjuncts carry most of the explicit\ninformation, but for a full understanding general world knowledge and default\nassumptions have to be taken into account. We will present a theory for\ndescribing the relation between, on the one hand, verbs, their tenses and\nadjuncts and, on the other, the eventualities and periods of time they\nrepresent and their relative temporal locations, while allowing interaction\nwith general world knowledge.\n  The theory is formulated in an extension of first order logic and is a\npractical implementation of the concepts described in Van Eynde 2001 and\nSchelkens et al. 2000. We will show how an abductive resolution procedure can\nbe used on this representation to extract temporal information from texts. The\ntheory presented here is an extension of that in Verdoolaege et al. 2000,\nadapted to VanEynde 2001, with a simplified and extended analysis of adjuncts\nand with more emphasis on how a model can be constructed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011040v1",
            "date": "2000-11-24 23:51:21+00:00",
            "title": "Do All Fragments Count?",
            "authors": [
                "Rens Bod"
            ],
            "abstract": "We aim at finding the minimal set of fragments which achieves maximal parse\naccuracy in Data Oriented Parsing. Experiments with the Penn Wall Street\nJournal treebank show that counts of almost arbitrary fragments within parse\ntrees are important, leading to improved parse accuracy over previous models\ntested on this treebank. We isolate a number of dependency relations which\nprevious models neglect but which contribute to higher parse accuracy.",
            "journal ref": "Technical Report COMP-11-12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001002v1",
            "date": "2000-01-04 21:46:29+00:00",
            "title": "Minimum Description Length and Compositionality",
            "authors": [
                "Wlodek Zadrozny"
            ],
            "abstract": "We present a non-vacuous definition of compositionality. It is based on the\nidea of combining the minimum description length principle with the original\ndefinition of compositionality (that is, that the meaning of the whole is a\nfunction of the meaning of the parts).\n  The new definition is intuitive and allows us to distinguish between\ncompositional and non-compositional semantics, and between idiomatic and\nnon-idiomatic expressions. It is not ad hoc, since it does not make any\nreferences to non-intrinsic properties of meaning functions (like being a\npolynomial). Moreover, it allows us to compare different meaning functions with\nrespect to how compositional they are. It bridges linguistic and corpus-based,\nstatistical approaches to natural language understanding.",
            "journal ref": "H.Bunt and R.Muskens(Eds.) \"Computing Meaning\" Vol.1. Kluwer 1999.\n  pp.113-128",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0001006v1",
            "date": "2000-01-09 23:54:15+00:00",
            "title": "Compositionality, Synonymy, and the Systematic Representation of Meaning",
            "authors": [
                "Shalom Lappin",
                "Wlodek Zadrozny"
            ],
            "abstract": "In a recent issue of Linguistics and Philosophy Kasmi and Pelletier (1998)\n(K&P), and Westerstahl (1998) criticize Zadrozny's (1994) argument that any\nsemantics can be represented compositionally. The argument is based upon\nZadrozny's theorem that every meaning function m can be encoded by a function\n\\mu such that (i) for any expression E of a specified language L, m(E) can be\nrecovered from \\mu(E), and (ii) \\mu is a homomorphism from the syntactic\nstructures of L to interpretations of L.\n  In both cases, the primary motivation for the objections brought against\nZadrozny's argument is the view that his encoding of the original meaning\nfunction does not properly reflect the synonymy relations posited for the\nlanguage.\n  In this paper, we argue that these technical criticisms do not go through. In\nparticular, we prove that \\mu properly encodes synonymy relations, i.e. if two\nexpressions are synonymous, then their compositional meanings are identical.\nThis corrects some misconceptions about the function \\mu, e.g. Janssen (1997).\n  We suggest that the reason that semanticists have been anxious to preserve\ncompositionality as a significant constraint on semantic theory is that it has\nbeen mistakenly regarded as a condition that must be satisfied by any theory\nthat sustains a systematic connection between the meaning of an expression and\nthe meanings of its parts. Recent developments in formal and computational\nsemantics show that systematic theories of meanings need not be compositional.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003003v1",
            "date": "2000-03-01 18:01:06+00:00",
            "title": "Prospects for in-depth story understanding by computer",
            "authors": [
                "Erik T. Mueller"
            ],
            "abstract": "While much research on the hard problem of in-depth story understanding by\ncomputer was performed starting in the 1970s, interest shifted in the 1990s to\ninformation extraction and word sense disambiguation. Now that a degree of\nsuccess has been achieved on these easier problems, I propose it is time to\nreturn to in-depth story understanding. In this paper I examine the shift away\nfrom story understanding, discuss some of the major problems in building a\nstory understanding system, present some possible solutions involving a set of\ninteracting understanding agents, and provide pointers to useful tools and\nresources for building story understanding systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003004v1",
            "date": "2000-03-01 18:07:02+00:00",
            "title": "A database and lexicon of scripts for ThoughtTreasure",
            "authors": [
                "Erik T. Mueller"
            ],
            "abstract": "Since scripts were proposed in the 1970's as an inferencing mechanism for AI\nand natural language processing programs, there have been few attempts to build\na database of scripts. This paper describes a database and lexicon of scripts\nthat has been added to the ThoughtTreasure commonsense platform. The database\nprovides the following information about scripts: sequence of events, roles,\nprops, entry conditions, results, goals, emotions, places, duration, frequency,\nand cost. English and French words and phrases are linked to script concepts.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0003022v1",
            "date": "2000-03-08 16:06:58+00:00",
            "title": "Hypothetical revision and matter-of-fact supposition",
            "authors": [
                "Horacio Arlo-Costa"
            ],
            "abstract": "The paper studies the notion of supposition encoded in non-Archimedean\nconditional probability (and revealed in the acceptance of the so-called\nindicative conditionals). The notion of qualitative change of view that thus\narises is axiomatized and compared with standard notions like AGM and UPDATE.\nApplications in the following fields are discussed: (1) theory of games and\ndecisions, (2) causal models, (3) non-monotonic logic.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0004008v1",
            "date": "2000-04-17 19:29:51+00:00",
            "title": "How to Evaluate your Question Answering System Every Day and Still Get Real Work Done",
            "authors": [
                "Eric Breck",
                "John D. Burger",
                "Lisa Ferro",
                "Lynette Hirschman",
                "David House",
                "Marc Light",
                "Inderjeet Mani"
            ],
            "abstract": "In this paper, we report on Qaviar, an experimental automated evaluation\nsystem for question answering applications. The goal of our research was to\nfind an automatically calculated measure that correlates well with human\njudges' assessment of answer correctness in the context of question answering\ntasks. Qaviar judges the response by computing recall against the stemmed\ncontent words in the human-generated answer key. It counts the answer correct\nif it exceeds agiven recall threshold. We determined that the answer\ncorrectness predicted by Qaviar agreed with the human 93% to 95% of the time.\n41 question-answering systems were ranked by both Qaviar and human assessors,\nand these rankings correlated with a Kendall's Tau measure of 0.920, compared\nto a correlation of 0.956 between human assessors on the same data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006013v1",
            "date": "2000-06-07 11:10:50+00:00",
            "title": "An evaluation of Naive Bayesian anti-spam filtering",
            "authors": [
                "Ion Androutsopoulos",
                "John Koutsias",
                "Konstantinos V. Chandrinos",
                "George Paliouras",
                "Constantine D. Spyropoulos"
            ],
            "abstract": "It has recently been argued that a Naive Bayesian classifier can be used to\nfilter unsolicited bulk e-mail (\"spam\"). We conduct a thorough evaluation of\nthis proposal on a corpus that we make publicly available, contributing towards\nstandard benchmarks. At the same time we investigate the effect of\nattribute-set size, training-corpus size, lemmatization, and stop-lists on the\nfilter's performance, issues that had not been previously explored. After\nintroducing appropriate cost-sensitive evaluation measures, we reach the\nconclusion that additional safety nets are needed for the Naive Bayesian\nanti-spam filter to be viable in practice.",
            "journal ref": "Proceedings of the workshop on Machine Learning in the New\n  Information Age, G. Potamias, V. Moustakis and M. van Someren (eds.), 11th\n  European Conference on Machine Learning, Barcelona, Spain, pp. 9-17, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006018v1",
            "date": "2000-06-09 18:10:58+00:00",
            "title": "Accuracy, Coverage, and Speed: What Do They Mean to Users?",
            "authors": [
                "Frankie James",
                "Manny Rayner",
                "Beth Ann Hockey"
            ],
            "abstract": "Speech is becoming increasingly popular as an interface modality, especially\nin hands- and eyes-busy situations where the use of a keyboard or mouse is\ndifficult. However, despite the fact that many have hailed speech as being\ninherently usable (since everyone already knows how to talk), most users of\nspeech input are left feeling disappointed by the quality of the interaction.\nClearly, there is much work to be done on the design of usable spoken\ninterfaces. We believe that there are two major problems in the design of\nspeech interfaces, namely, (a) the people who are currently working on the\ndesign of speech interfaces are, for the most part, not interface designers and\ntherefore do not have as much experience with usability issues as we in the CHI\ncommunity do, and (b) speech, as an interface modality, has vastly different\nproperties than other modalities, and therefore requires different usability\nmeasures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006027v1",
            "date": "2000-06-13 09:50:03+00:00",
            "title": "Verbal Interactions in Virtual Worlds",
            "authors": [
                "Pierre Nugues"
            ],
            "abstract": "We first discuss respective advantages of language interaction in virtual\nworlds and of using 3D images in dialogue systems. Then, we describe an example\nof a verbal interaction system in virtual reality: Ulysse. Ulysse is a\nconversational agent that helps a user navigate in virtual worlds. It has been\ndesigned to be embedded in the representation of a participant of a virtual\nconference and it responds positively to motion orders. Ulysse navigates the\nuser's viewpoint on his/her behalf in the virtual world. On tests we carried\nout, we discovered that users, novices as well as experienced ones have\ndifficulties moving in a 3D environment. Agents such as Ulysse enable a user to\ncarry out navigation motions that would have been impossible with classical\ninteraction devices. From the whole Ulysse system, we have stripped off a\nskeleton architecture that we have ported to VRML, Java, and Prolog. We hope\nthis skeleton helps the design of language applications in virtual worlds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006041v1",
            "date": "2000-06-29 07:44:16+00:00",
            "title": "Using a Diathesis Model for Semantic Parsing",
            "authors": [
                "Jordi Atserias",
                "Irene Castellon",
                "Montse Civit",
                "German Rigau"
            ],
            "abstract": "This paper presents a semantic parsing approach for unrestricted texts.\nSemantic parsing is one of the major bottlenecks of Natural Language\nUnderstanding (NLU) systems and usually requires building expensive resources\nnot easily portable to other domains. Our approach obtains a case-role\nanalysis, in which the semantic roles of the verb are identified. In order to\ncover all the possible syntactic realisations of a verb, our system combines\ntheir argument structure with a set of general semantic labelled diatheses\nmodels. Combining them, the system builds a set of syntactic-semantic patterns\nwith their own role-case representation. Once the patterns are build, we use an\napproximate tree pattern-matching algorithm to identify the most reliable\npattern for a sentence. The pattern matching is performed between the\nsyntactic-semantic patterns and the feature-structure tree representing the\nmorphological, syntactical and semantic information of the analysed sentence.\nFor sentences assigned to the correct model, the semantic parsing system we are\npresenting identifies correctly more than 73% of possible semantic case-roles.",
            "journal ref": "Proceedins of VEXTAL.1999 pg 385-392",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006042v1",
            "date": "2000-06-29 09:17:45+00:00",
            "title": "Semantic Parsing based on Verbal Subcategorization",
            "authors": [
                "Jordi Atserias",
                "Irene Castellon",
                "Montse Civit",
                "German Rigau"
            ],
            "abstract": "The aim of this work is to explore new methodologies on Semantic Parsing for\nunrestricted texts. Our approach follows the current trends in Information\nExtraction (IE) and is based on the application of a verbal subcategorization\nlexicon (LEXPIR) by means of complex pattern recognition techniques. LEXPIR is\nframed on the theoretical model of the verbal subcategorization developed in\nthe Pirapides project.",
            "journal ref": "Conference on Intelligence text Processing and Computational\n  Linguistics, CICLing 2000. pg 330-340",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007010v1",
            "date": "2000-07-07 14:10:05+00:00",
            "title": "Boosting Applied to Word Sense Disambiguation",
            "authors": [
                "Gerard Escudero",
                "Lluis Marquez",
                "German Rigau"
            ],
            "abstract": "In this paper Schapire and Singer's AdaBoost.MH boosting algorithm is applied\nto the Word Sense Disambiguation (WSD) problem. Initial experiments on a set of\n15 selected polysemous words show that the boosting approach surpasses Naive\nBayes and Exemplar-based approaches, which represent state-of-the-art accuracy\non supervised WSD. In order to make boosting practical for a real learning\ndomain of thousands of words, several ways of accelerating the algorithm by\nreducing the feature space are studied. The best variant, which we call\nLazyBoosting, is tested on the largest sense-tagged corpus available containing\n192,800 examples of the 191 most frequent and ambiguous English words. Again,\nboosting compares favourably to the other benchmark algorithms.",
            "journal ref": "Proceedings of the 11th European Conference on Machine Learning,\n  ECML'2000 pp. 129-141",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007011v1",
            "date": "2000-07-07 15:00:44+00:00",
            "title": "Naive Bayes and Exemplar-Based approaches to Word Sense Disambiguation Revisited",
            "authors": [
                "Gerard Escudero",
                "Lluis Marquez",
                "German Rigau"
            ],
            "abstract": "This paper describes an experimental comparison between two standard\nsupervised learning methods, namely Naive Bayes and Exemplar-based\nclassification, on the Word Sense Disambiguation (WSD) problem. The aim of the\nwork is twofold. Firstly, it attempts to contribute to clarify some confusing\ninformation about the comparison between both methods appearing in the related\nliterature. In doing so, several directions have been explored, including:\ntesting several modifications of the basic learning algorithms and varying the\nfeature space. Secondly, an improvement of both algorithms is proposed, in\norder to deal with large attribute sets. This modification, which basically\nconsists in using only the positive information appearing in the examples,\nallows to improve greatly the efficiency of the methods, with no loss in\naccuracy. The experiments have been performed on the largest sense-tagged\ncorpus available containing the most frequent and ambiguous English words.\nResults show that the Exemplar-based approach to WSD is generally superior to\nthe Bayesian approach, especially when a specific metric for dealing with\nsymbolic attributes is used.",
            "journal ref": "Proceedings of the 14th European Conference on Artificial\n  Intelligence, ECAI'2000 pp. 421-425",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007012v1",
            "date": "2000-07-07 15:13:09+00:00",
            "title": "Using Learning-based Filters to Detect Rule-based Filtering Obsolescence",
            "authors": [
                "Francis Wolinski",
                "Frantz Vichot",
                "Mathieu Stricker"
            ],
            "abstract": "For years, Caisse des Depots et Consignations has produced information\nfiltering applications. To be operational, these applications require high\nfiltering performances which are achieved by using rule-based filters. With\nthis technique, an administrator has to tune a set of rules for each topic.\nHowever, filters become obsolescent over time. The decrease of their\nperformances is due to diachronic polysemy of terms that involves a loss of\nprecision and to diachronic polymorphism of concepts that involves a loss of\nrecall.\n  To help the administrator to maintain his filters, we have developed a method\nwhich automatically detects filtering obsolescence. It consists in making a\nlearning-based control filter using a set of documents which have already been\ncategorised as relevant or not relevant by the rule-based filter. The idea is\nto supervise this filter by processing a differential comparison of its\noutcomes with those of the control one.\n  This method has many advantages. It is simple to implement since the training\nset used by the learning is supplied by the rule-based filter. Thus, both the\nmaking and the use of the control filter are fully automatic. With automatic\ndetection of obsolescence, learning-based filtering finds a rich application\nwhich offers interesting prospects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007013v1",
            "date": "2000-07-07 19:09:25+00:00",
            "title": "Applying Constraint Handling Rules to HPSG",
            "authors": [
                "Gerald Penn"
            ],
            "abstract": "Constraint Handling Rules (CHR) have provided a realistic solution to an\nover-arching problem in many fields that deal with constraint logic\nprogramming: how to combine recursive functions or relations with constraints\nwhile avoiding non-termination problems. This paper focuses on some other\nbenefits that CHR, specifically their implementation in SICStus Prolog, have\nprovided to computational linguists working on grammar design tools. CHR rules\nare applied by means of a subsumption check and this check is made only when\ntheir variables are instantiated or bound. The former functionality is at best\ndifficult to simulate using more primitive coroutining statements such as\nSICStus when/2, and the latter simply did not exist in any form before CHR.\n  For the sake of providing a case study in how these can be applied to grammar\ndevelopment, we consider the Attribute Logic Engine (ALE), a Prolog\npreprocessor for logic programming with typed feature structures, and its\nextension to a complete grammar development system for Head-driven Phrase\nStructure Grammar (HPSG), a popular constraint-based linguistic theory that\nuses typed feature structures. In this context, CHR can be used not only to\nextend the constraint language of feature structure descriptions to include\nrelations in a declarative way, but also to provide support for constraints\nwith complex antecedents and constraints on the co-occurrence of feature values\nthat are necessary to interpret the type system of HPSG properly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007016v1",
            "date": "2000-07-11 13:21:03+00:00",
            "title": "Two Steps Feature Selection and Neural Network Classification for the TREC-8 Routing",
            "authors": [
                "Mathieu Stricker",
                "Frantz Vichot",
                "Gerard Dreyfus",
                "Francis Wolinski"
            ],
            "abstract": "For the TREC-8 routing, one specific filter is built for each topic. Each\nfilter is a classifier trained to recognize the documents that are relevant to\nthe topic. When presented with a document, each classifier estimates the\nprobability for the document to be relevant to the topic for which it has been\ntrained. Since the procedure for building a filter is topic-independent, the\nsystem is fully automatic.\n  By making use of a sample of documents that have previously been evaluated as\nrelevant or not relevant to a particular topic, a term selection is performed,\nand a neural network is trained. Each document is represented by a vector of\nfrequencies of a list of selected terms. This list depends on the topic to be\nfiltered; it is constructed in two steps. The first step defines the\ncharacteristic words used in the relevant documents of the corpus; the second\none chooses, among the previous list, the most discriminant ones. The length of\nthe vector is optimized automatically for each topic. At the end of the term\nselection, a vector of typically 25 words is defined for the topic, so that\neach document which has to be processed is represented by a vector of term\nfrequencies.\n  This vector is subsequently input to a classifier that is trained from the\nsame sample. After training, the classifier estimates for each document of a\ntest set its probability of being relevant; for submission to TREC, the top\n1000 documents are ranked in order of decreasing relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0007023v1",
            "date": "2000-07-13 18:40:15+00:00",
            "title": "Towards a query language for annotation graphs",
            "authors": [
                "Steven Bird",
                "Peter Buneman",
                "Wang-Chiew Tan"
            ],
            "abstract": "The multidimensional, heterogeneous, and temporal nature of speech databases\nraises interesting challenges for representation and query. Recently,\nannotation graphs have been proposed as a general-purpose representational\nframework for speech databases. Typical queries on annotation graphs require\npath expressions similar to those used in semistructured query languages.\nHowever, the underlying model is rather different from the customary graph\nmodels for semistructured data: the graph is acyclic and unrooted, and both\ntemporal and inclusion relationships are important. We develop a query language\nand describe optimization techniques for an underlying relational\nrepresentation.",
            "journal ref": "Proceedings of the Second International Conference on Language\n  Resources and Evaluation, pp. 807-814, Paris: European Language Resources\n  Association, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008016v1",
            "date": "2000-08-21 10:54:11+00:00",
            "title": "Processing Self Corrections in a speech to speech system",
            "authors": [
                "Joerg Spilker",
                "Martin Klarner",
                "Guenther Goerz"
            ],
            "abstract": "Speech repairs occur often in spontaneous spoken dialogues. The ability to\ndetect and correct those repairs is necessary for any spoken language system.\nWe present a framework to detect and correct speech repairs where all relevant\nlevels of information, i.e., acoustics, lexis, syntax and semantics can be\nintegrated. The basic idea is to reduce the search space for repairs as soon as\npossible by cascading filters that involve more and more features. At first an\nacoustic module generates hypotheses about the existence of a repair. Second a\nstochastic model suggests a correction for every hypothesis. Well scored\ncorrections are inserted as new paths in the word lattice. Finally a lattice\nparser decides on accepting the rep air.",
            "journal ref": "Proceedings of COLING 2000, Saarbruecken, Germany; 31.7-4.8; pp\n  1116-1120",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008020v1",
            "date": "2000-08-22 15:01:21+00:00",
            "title": "Explaining away ambiguity: Learning verb selectional preference with Bayesian networks",
            "authors": [
                "Massimiliano Ciaramita",
                "Mark Johnson"
            ],
            "abstract": "This paper presents a Bayesian model for unsupervised learning of verb\nselectional preferences. For each verb the model creates a Bayesian network\nwhose architecture is determined by the lexical hierarchy of Wordnet and whose\nparameters are estimated from a list of verb-object pairs found from a corpus.\n``Explaining away'', a well-known property of Bayesian networks, helps the\nmodel deal in a natural fashion with word sense ambiguity in the training data.\nOn a word sense disambiguation test our model performed better than other state\nof the art systems for unsupervised learning of selectional preferences.\nComputational complexity problems, ways of improving this approach and methods\nfor implementing ``explaining away'' in other graphical frameworks are\ndiscussed.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics, Saarbrucken, Germany, Vol.1, 2000, p.187",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008022v1",
            "date": "2000-08-22 21:37:50+00:00",
            "title": "A Learning Approach to Shallow Parsing",
            "authors": [
                "Marcia Mu\u00f1oz",
                "Vasin Punyakanok",
                "Dan Roth",
                "Dav Zimak"
            ],
            "abstract": "A SNoW based learning approach to shallow parsing tasks is presented and\nstudied experimentally. The approach learns to identify syntactic patterns by\ncombining simple predictors to produce a coherent inference. Two instantiations\nof this approach are studied and experimental results for Noun-Phrases (NP) and\nSubject-Verb (SV) phrases that compare favorably with the best published\nresults are presented. In doing that, we compare two ways of modeling the\nproblem of learning to recognize patterns and suggest that shallow parsing\npatterns are better learned using open/close predictors than using\ninside/outside predictors.",
            "journal ref": "Proceedings of EMNLP-VLC'99, pages 168-178",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009014v1",
            "date": "2000-09-20 13:04:11+00:00",
            "title": "Combining Linguistic and Spatial Information for Document Analysis",
            "authors": [
                "Marco Aiello",
                "Christof Monz",
                "Leon Todoran"
            ],
            "abstract": "We present a framework to analyze color documents of complex layout. In\naddition, no assumption is made on the layout. Our framework combines in a\ncontent-driven bottom-up approach two different sources of information: textual\nand spatial. To analyze the text, shallow natural language processing tools,\nsuch as taggers and partial parsers, are used. To infer relations of the\nlogical layout we resort to a qualitative spatial calculus closely related to\nAllen's calculus. We evaluate the system against documents from a color journal\nand present the results of extracting the reading order from the journal's\npages. In this case, our analysis is successful as it extracts the intended\nreading order from the document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009016v1",
            "date": "2000-09-20 13:41:06+00:00",
            "title": "Contextual Inference in Computational Semantics",
            "authors": [
                "Christof Monz"
            ],
            "abstract": "In this paper, an application of automated theorem proving techniques to\ncomputational semantics is considered. In order to compute the presuppositions\nof a natural language discourse, several inference tasks arise. Instead of\ntreating these inferences independently of each other, we show how integrating\ntechniques from formal approaches to context into deduction can help to compute\npresuppositions more efficiently. Contexts are represented as Discourse\nRepresentation Structures and the way they are nested is made explicit. In\naddition, a tableau calculus is present which keeps track of contextual\ninformation, and thereby allows to avoid carrying out redundant inference steps\nas it happens in approaches that neglect explicit nesting of contexts.",
            "journal ref": "In: P. Bouquet, P. Brezillon, L. Serafini, M. Benerecetti, F.\n  Castellani (Eds.) 2nd International and Interdisciplinary Conference on\n  Modeling and Using Context (CONTEXT'99). Lecture Notes in Artificial\n  Intelligence 1688, Springer, 1999, pages 242-255",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009017v1",
            "date": "2000-09-21 14:49:19+00:00",
            "title": "A Tableau Calculus for Pronoun Resolution",
            "authors": [
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "We present a tableau calculus for reasoning in fragments of natural language.\nWe focus on the problem of pronoun resolution and the way in which it\ncomplicates automated theorem proving for natural language processing. A method\nfor explicitly manipulating contextual information during deduction is\nproposed, where pronouns are resolved against this context during deduction. As\na result, pronoun resolution and deduction can be interleaved in such a way\nthat pronouns are only resolved if this is licensed by a deduction rule; this\nhelps us to avoid the combinatorial complexity of total pronoun disambiguation.",
            "journal ref": "In: N.V. Murray (ed.) Automated Reasoning with Analytic Tableaux\n  and Related Methods. Lecture Notes in Artificial Intelligence 1617, Springer,\n  1999, pages 247-262",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009018v1",
            "date": "2000-09-21 15:21:01+00:00",
            "title": "A Resolution Calculus for Dynamic Semantics",
            "authors": [
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "This paper applies resolution theorem proving to natural language semantics.\nThe aim is to circumvent the computational complexity triggered by natural\nlanguage ambiguities like pronoun binding, by interleaving pronoun binding with\nresolution deduction. Therefore disambiguation is only applied to expression\nthat actually occur during derivations.",
            "journal ref": "In: J. Dix, L. Farinas del Cerro, and U. Furbach (eds.) Logics in\n  Artificial Intelligence (JELIA'98). Lecture Notes in Artificial Intelligence\n  1489, Springer, 1998, pp. 184-198",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009019v1",
            "date": "2000-09-21 15:32:17+00:00",
            "title": "Computing Presuppositions by Contextual Reasoning",
            "authors": [
                "Christof Monz"
            ],
            "abstract": "This paper describes how automated deduction methods for natural language\nprocessing can be applied more efficiently by encoding context in a more\nelaborate way. Our work is based on formal approaches to context, and we\nprovide a tableau calculus for contextual reasoning. This is explained by\nconsidering an example from the problem area of presupposition projection.",
            "journal ref": "In: P. Brezillon, R. Turner, J-C. Pomerol and E. Turner (Eds.)\n  Proceedings of the AAAI-99 Workshop on Reasoning in Context for AI\n  Applications, AAAI Press, 1999, pp. 75-79",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009022v1",
            "date": "2000-09-22 15:02:26+00:00",
            "title": "A Comparison between Supervised Learning Algorithms for Word Sense Disambiguation",
            "authors": [
                "Gerard Escudero",
                "Lluis Marquez",
                "German Rigau"
            ],
            "abstract": "This paper describes a set of comparative experiments, including cross-corpus\nevaluation, between five alternative algorithms for supervised Word Sense\nDisambiguation (WSD), namely Naive Bayes, Exemplar-based learning, SNoW,\nDecision Lists, and Boosting. Two main conclusions can be drawn: 1) The\nLazyBoosting algorithm outperforms the other four state-of-the-art algorithms\nin terms of accuracy and ability to tune to new domains; 2) The domain\ndependence of WSD systems seems very strong and suggests that some kind of\nadaptation or tuning is required for cross-corpus application.",
            "journal ref": "Proceedings of the 4th Conference on Computational Natural\n  Language Learning, CoNLL'2000, pp. 31-36",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011011v1",
            "date": "2000-11-07 10:09:15+00:00",
            "title": "Formal Properties of XML Grammars and Languages",
            "authors": [
                "Jean Berstel",
                "Luc Boasson"
            ],
            "abstract": "XML documents are described by a document type definition (DTD). An\nXML-grammar is a formal grammar that captures the syntactic features of a DTD.\nWe investigate properties of this family of grammars. We show that every\nXML-language basically has a unique XML-grammar. We give two characterizations\nof languages generated by XML-grammars, one is set-theoretic, the other is by a\nkind of saturation property. We investigate decidability problems and prove\nthat some properties that are undecidable for general context-free languages\nbecome decidable for XML-languages. We also characterize those XML-grammars\nthat generate regular XML-languages.",
            "journal ref": "None",
            "category": "cs.DM"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011028v1",
            "date": "2000-11-20 15:36:09+00:00",
            "title": "Retrieval from Captioned Image Databases Using Natural Language Processing",
            "authors": [
                "David Elworthy"
            ],
            "abstract": "It might appear that natural language processing should improve the accuracy\nof information retrieval systems, by making available a more detailed analysis\nof queries and documents. Although past results appear to show that this is not\nso, if the focus is shifted to short phrases rather than full documents, the\nsituation becomes somewhat different. The ANVIL system uses a natural language\ntechnique to obtain high accuracy retrieval of images which have been annotated\nwith a descriptive textual caption. The natural language techniques also allow\nadditional contextual information to be derived from the relation between the\nquery and the caption, which can help users to understand the overall\ncollection of retrieval results. The techniques have been successfully used in\na information retrieval system which forms both a testbed for research and the\nbasis of a commercial system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0006032v1",
            "date": "2000-06-23 09:43:27+00:00",
            "title": "Estimation of English and non-English Language Use on the WWW",
            "authors": [
                "Gregory Grefenstette",
                "Julien Nioche"
            ],
            "abstract": "The World Wide Web has grown so big, in such an anarchic fashion, that it is\ndifficult to describe. One of the evident intrinsic characteristics of the\nWorld Wide Web is its multilinguality. Here, we present a technique for\nestimating the size of a language-specific corpus given the frequency of\ncommonly occurring words in the corpus. We apply this technique to estimating\nthe number of words available through Web browsers for given languages.\nComparing data from 1996 to data from 1999 and 2000, we calculate the growth of\na number of European languages on the Web. As expected, non-English languages\nare growing at a faster pace than English, though the position of English is\nstill dominant.",
            "journal ref": "Proceedings of RIAO'2000, \"Content-Based Multimedia Information\n  Access\", Paris, April 12-14,2000, pp. 237-246",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0008019v1",
            "date": "2000-08-22 11:20:14+00:00",
            "title": "An Experimental Comparison of Naive Bayesian and Keyword-Based Anti-Spam Filtering with Personal E-mail Messages",
            "authors": [
                "Ion Androutsopoulos",
                "John Koutsias",
                "Konstantinos V. Chandrinos",
                "Constantine D. Spyropoulos"
            ],
            "abstract": "The growing problem of unsolicited bulk e-mail, also known as \"spam\", has\ngenerated a need for reliable anti-spam e-mail filters. Filters of this type\nhave so far been based mostly on manually constructed keyword patterns. An\nalternative approach has recently been proposed, whereby a Naive Bayesian\nclassifier is trained automatically to detect spam messages. We test this\napproach on a large collection of personal e-mail messages, which we make\npublicly available in \"encrypted\" form contributing towards standard\nbenchmarks. We introduce appropriate cost-sensitive measures, investigating at\nthe same time the effect of attribute-set size, training-corpus size,\nlemmatization, and stop lists, issues that have not been explored in previous\nexperiments. Finally, the Naive Bayesian filter is compared, in terms of\nperformance, to a filter that uses keyword patterns, and which is part of a\nwidely used e-mail reader.",
            "journal ref": "Proceedings of the 23rd Annual International ACM SIGIR Conference\n  on Research and Development in Information Retrieval, N.J. Belkin, P.\n  Ingwersen and M.-K. Leong (Eds.), Athens, Greece, July 24-28, 2000, pages\n  160-167",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009009v1",
            "date": "2000-09-18 14:05:13+00:00",
            "title": "Learning to Filter Spam E-Mail: A Comparison of a Naive Bayesian and a Memory-Based Approach",
            "authors": [
                "Ion Androutsopoulos",
                "Georgios Paliouras",
                "Vangelis Karkaletsis",
                "Georgios Sakkis",
                "Constantine D. Spyropoulos",
                "Panagiotis Stamatopoulos"
            ],
            "abstract": "We investigate the performance of two machine learning algorithms in the\ncontext of anti-spam filtering. The increasing volume of unsolicited bulk\ne-mail (spam) has generated a need for reliable anti-spam filters. Filters of\nthis type have so far been based mostly on keyword patterns that are\nconstructed by hand and perform poorly. The Naive Bayesian classifier has\nrecently been suggested as an effective method to construct automatically\nanti-spam filters with superior performance. We investigate thoroughly the\nperformance of the Naive Bayesian filter on a publicly available corpus,\ncontributing towards standard benchmarks. At the same time, we compare the\nperformance of the Naive Bayesian filter to an alternative memory-based\nlearning approach, after introducing suitable cost-sensitive evaluation\nmeasures. Both methods achieve very accurate spam filtering, outperforming\nclearly the keyword-based filter of a widely used e-mail reader.",
            "journal ref": "Proceedings of the workshop \"Machine Learning and Textual\n  Information Access\", 4th European Conference on Principles and Practice of\n  Knowledge Discovery in Databases (PKDD-2000), H. Zaragoza, P. Gallinari and\n  M. Rajman (Eds.), Lyon, France, September 2000, pp. 1-13",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009012v1",
            "date": "2000-09-19 15:43:18+00:00",
            "title": "Modeling Ambiguity in a Multi-Agent System",
            "authors": [
                "Christof Monz"
            ],
            "abstract": "This paper investigates the formal pragmatics of ambiguous expressions by\nmodeling ambiguity in a multi-agent system. Such a framework allows us to give\na more refined notion of the kind of information that is conveyed by ambiguous\nexpressions. We analyze how ambiguity affects the knowledge of the dialog\nparticipants and, especially, what they know about each other after an\nambiguous sentence has been uttered. The agents communicate with each other by\nmeans of a TELL-function, whose application is constrained by an implementation\nof some of Grice's maxims. The information states of the multi-agent system\nitself are represented as a Kripke structures and TELL is an update function on\nthose structures. This framework enables us to distinguish between the\ninformation conveyed by ambiguous sentences vs. the information conveyed by\ndisjunctions, and between semantic ambiguity vs. perceived ambiguity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0009027v1",
            "date": "2000-09-28 14:25:51+00:00",
            "title": "A Classification Approach to Word Prediction",
            "authors": [
                "Yair Even-Zohar",
                "Dan Roth"
            ],
            "abstract": "The eventual goal of a language model is to accurately predict the value of a\nmissing word given its context. We present an approach to word prediction that\nis based on learning a representation for each word as a function of words and\nlinguistics predicates in its context. This approach raises a few new questions\nthat we address. First, in order to learn good word representations it is\nnecessary to use an expressive representation of the context. We present a way\nthat uses external knowledge to generate expressive context representations,\nalong with a learning method capable of handling the large number of features\ngenerated this way that can, potentially, contribute to each prediction.\nSecond, since the number of words ``competing'' for each prediction is large,\nthere is a need to ``focus the attention'' on a smaller subset of these. We\nexhibit the contribution of a ``focus of attention'' mechanism to the\nperformance of the word predictor. Finally, we describe a large scale\nexperimental study in which the approach presented is shown to yield\nsignificant improvements in word prediction tasks.",
            "journal ref": "NAACL 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010014v1",
            "date": "2000-10-10 17:33:02+00:00",
            "title": "On a cepstrum-based speech detector robust to white noise",
            "authors": [
                "Sergei Skorik",
                "Frederic Berthommier"
            ],
            "abstract": "We study effects of additive white noise on the cepstral representation of\nspeech signals. Distribution of each individual cepstrum coefficient of speech\nis shown to depend strongly on noise and to overlap significantly with the\ncepstrum distribution of noise. Based on these studies, we suggest a scalar\nquantity, V, equal to the sum of weighted cepstral coefficients, which is able\nto classify frames containing speech against noise-like frames. The\ndistributions of V for speech and noise frames are reasonably well separated\nabove SNR = 5 dB, demonstrating the feasibility of robust speech detector based\non V.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0010033v1",
            "date": "2000-10-26 17:42:30+00:00",
            "title": "A Formal Framework for Linguistic Annotation (revised version)",
            "authors": [
                "Steven Bird",
                "Mark Liberman"
            ],
            "abstract": "`Linguistic annotation' covers any descriptive or analytic notations applied\nto raw language data. The basic data may be in the form of time functions -\naudio, video and/or physiological recordings - or it may be textual. The added\nnotations may include transcriptions of all sorts (from phonetic features to\ndiscourse structures), part-of-speech and sense tagging, syntactic analysis,\n`named entity' identification, co-reference annotation, and so on. While there\nare several ongoing efforts to provide formats and tools for such annotations\nand to publish annotated linguistic databases, the lack of widely accepted\nstandards is becoming a critical problem. Proposed standards, to the extent\nthey exist, have focused on file formats. This paper focuses instead on the\nlogical structure of linguistic annotations. We survey a wide variety of\nexisting annotation formats and demonstrate a common conceptual core, the\nannotation graph. This provides a formal framework for constructing,\nmaintaining and searching linguistic annotations, while remaining consistent\nwith many alternative data structures and file formats.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0011007v1",
            "date": "2000-11-06 13:56:42+00:00",
            "title": "Tree-gram Parsing: Lexical Dependencies and Structural Relations",
            "authors": [
                "Khalil Sima'an"
            ],
            "abstract": "This paper explores the kinds of probabilistic relations that are important\nin syntactic disambiguation. It proposes that two widely used kinds of\nrelations, lexical dependencies and structural relations, have complementary\ndisambiguation capabilities. It presents a new model based on structural\nrelations, the Tree-gram model, and reports experiments showing that structural\nrelations should benefit from enrichment by lexical dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0005020v1",
            "date": "2000-05-12 17:24:06+00:00",
            "title": "Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies",
            "authors": [
                "Dragomir R. Radev",
                "Hongyan Jing",
                "Malgorzata Budzikowska"
            ],
            "abstract": "We present a multi-document summarizer, called MEAD, which generates\nsummaries using cluster centroids produced by a topic detection and tracking\nsystem. We also describe two new techniques, based on sentence utility and\nsubsumption, which we have applied to the evaluation of both single and\nmultiple document summaries. Finally, we describe two user studies that test\nour models of multi-document summarization.",
            "journal ref": "NAACL/ANLP Workshop on Automatic Summarization, Seattle, WA, April\n  30, 2000",
            "category": "cs.CL"
        }
    ],
    "2001": [
        {
            "url": "http://arxiv.org/abs/cs/0102020v1",
            "date": "2001-02-22 21:05:01+00:00",
            "title": "Multi-Syllable Phonotactic Modelling",
            "authors": [
                "Anja Belz"
            ],
            "abstract": "This paper describes a novel approach to constructing phonotactic models. The\nunderlying theoretical approach to phonological description is the\nmultisyllable approach in which multiple syllable classes are defined that\nreflect phonotactically idiosyncratic syllable subcategories. A new\nfinite-state formalism, OFS Modelling, is used as a tool for encoding,\nautomatically constructing and generalising phonotactic descriptions.\nLanguage-independent prototype models are constructed which are instantiated on\nthe basis of data sets of phonological strings, and generalised with a\nclustering algorithm. The resulting approach enables the automatic construction\nof phonotactic models that encode arbitrarily close approximations of a\nlanguage's set of attested phonological forms. The approach is applied to the\nconstruction of multi-syllable word-level phonotactic models for German,\nEnglish and Dutch.",
            "journal ref": "Jason Eisner, Lauri Karttunen and Alain Theriault (eds.),\n  Finite-State Phonology: Proceedings of the 5th Workshop of the ACL Special\n  Interest Group in Computational Phonology (SIGPHON), pp. 46-56. Luxembourg,\n  August 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0102021v1",
            "date": "2001-02-22 13:09:25+00:00",
            "title": "Taking Primitive Optimality Theory Beyond the Finite State",
            "authors": [
                "Daniel Albro"
            ],
            "abstract": "Primitive Optimality Theory (OTP) (Eisner, 1997a; Albro, 1998), a\ncomputational model of Optimality Theory (Prince and Smolensky, 1993), employs\na finite state machine to represent the set of active candidates at each stage\nof an Optimality Theoretic derivation, as well as weighted finite state\nmachines to represent the constraints themselves. For some purposes, however,\nit would be convenient if the set of candidates were limited by some set of\ncriteria capable of being described only in a higher-level grammar formalism,\nsuch as a Context Free Grammar, a Context Sensitive Grammar, or a Multiple\nContext Free Grammar (Seki et al., 1991). Examples include reduplication and\nphrasal stress models. Here we introduce a mechanism for OTP-like Optimality\nTheory in which the constraints remain weighted finite state machines, but sets\nof candidates are represented by higher-level grammars. In particular, we use\nmultiple context-free grammars to model reduplication in the manner of\nCorrespondence Theory (McCarthy and Prince, 1995), and develop an extended\nversion of the Earley Algorithm (Earley, 1970) to apply the constraints to a\nreduplicating candidate set.",
            "journal ref": "Jason Eisner, Lauri Karttunen and Alain Theriault (eds.),\n  Finite-State Phonology: Proceedings of the 5th Workshop of the ACL Special\n  Interest Group in Computational Phonology (SIGPHON), pp. 57-67. Luxembourg,\n  August 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0102022v2",
            "date": "2001-02-22 14:10:20+00:00",
            "title": "Finite-State Phonology: Proceedings of the 5th Workshop of the ACL Special Interest Group in Computational Phonology (SIGPHON)",
            "authors": [
                "Jason Eisner",
                "Lauri Karttunen",
                "Alain Theriault"
            ],
            "abstract": "Home page of the workshop proceedings, with pointers to the individually\narchived papers. Includes front matter from the printed version of the\nproceedings.",
            "journal ref": "Jason Eisner, Lauri Karttunen and Alain Theriault (eds.),\n  Finite-State Phonology: Proceedings of the 5th Workshop of the ACL Special\n  Interest Group in Computational Phonology (SIGPHON). Luxembourg, August 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0102026v1",
            "date": "2001-02-24 06:56:43+00:00",
            "title": "Mathematical Model of Word Length on the Basis of the Cebanov-Fucks Distribution with Uniform Parameter Distribution",
            "authors": [
                "Victor Kromer"
            ],
            "abstract": "The data on 13 typologically different languages have been processed using a\ntwo-parameter word length model, based on 1-displaced uniform Poisson\ndistribution. Statistical dependencies of the 2nd parameter on the 1st one are\nrevealed for the German texts and genre of letters.",
            "journal ref": "Kromer V.W. Matematiceskaja model' dliny slova na osnove\n  raspredelenija Cebanova-Fuksa s ravnomernym raspredeleniem parametra //\n  Informatika i problemy telekommunikacij: Mezdunarodnaja naucno-techniceskaja\n  konferencija (SibGUTI, 26-27 aprelja 2001 g.) Materialy konferencii. -\n  Novosibirsk: Isd-vo SibGUTI, 2001. - S. 74-75.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103007v1",
            "date": "2001-03-08 06:30:27+00:00",
            "title": "Two-parameter Model of Word Length \"Language - Genre\"",
            "authors": [
                "Victor Kromer"
            ],
            "abstract": "A two-parameter model of word length measured by the number of syllables\ncomprising it is proposed. The first parameter is dependent on language type,\nthe second one - on text genre and reflects the degree of completion of\nsynergetic processes of language optimization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103010v1",
            "date": "2001-03-12 08:23:19+00:00",
            "title": "Magical Number Seven Plus or Minus Two: Syntactic Structure Recognition in Japanese and English Sentences",
            "authors": [
                "Masaki Murata",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "George A. Miller said that human beings have only seven chunks in short-term\nmemory, plus or minus two. We counted the number of bunsetsus (phrases) whose\nmodifiees are undetermined in each step of an analysis of the dependency\nstructure of Japanese sentences, and which therefore must be stored in\nshort-term memory. The number was roughly less than nine, the upper bound of\nseven plus or minus two. We also obtained similar results with English\nsentences under the assumption that human beings recognize a series of words,\nsuch as a noun phrase (NP), as a unit. This indicates that if we assume that\nthe human cognitive units in Japanese and English are bunsetsu and NP\nrespectively, analysis will support Miller's $7 \\pm 2$ theory.",
            "journal ref": "CICLing'2001, Mexico City, February 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103011v1",
            "date": "2001-03-12 08:29:59+00:00",
            "title": "A Machine-Learning Approach to Estimating the Referential Properties of Japanese Noun Phrases",
            "authors": [
                "Masaki Murata",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "The referential properties of noun phrases in the Japanese language, which\nhas no articles, are useful for article generation in Japanese-English machine\ntranslation and for anaphora resolution in Japanese noun phrases. They are\ngenerally classified as generic noun phrases, definite noun phrases, and\nindefinite noun phrases. In the previous work, referential properties were\nestimated by developing rules that used clue words. If two or more rules were\nin conflict with each other, the category having the maximum total score given\nby the rules was selected as the desired category. The score given by each rule\nwas established by hand, so the manpower cost was high. In this work, we\nautomatically adjusted these scores by using a machine-learning method and\nsucceeded in reducing the amount of manpower needed to adjust these scores.",
            "journal ref": "CICLing'2001, Mexico City, February 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103012v1",
            "date": "2001-03-12 08:43:11+00:00",
            "title": "Meaning Sort - Three examples: dictionary construction, tagged corpus construction, and information presentation system",
            "authors": [
                "Masaki Murata",
                "Kyoko Kanzaki",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "It is often useful to sort words into an order that reflects relations among\ntheir meanings as obtained by using a thesaurus. In this paper, we introduce a\nmethod of arranging words semantically by using several types of `{\\sf is-a}'\nthesauri and a multi-dimensional thesaurus. We also describe three major\napplications where a meaning sort is useful and show the effectiveness of a\nmeaning sort. Since there is no doubt that a word list in meaning-order is\neasier to use than a word list in some random order, a meaning sort, which can\neasily produce a word list in meaning-order, must be useful and effective.",
            "journal ref": "CICLing'2001, Mexico City, February 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103013v1",
            "date": "2001-03-12 09:36:50+00:00",
            "title": "CRL at Ntcir2",
            "authors": [
                "Masaki Murata",
                "Masao Utiyama",
                "Qing Ma",
                "Hiromi Ozaku",
                "Hitoshi Isahara"
            ],
            "abstract": "We have developed systems of two types for NTCIR2. One is an enhenced version\nof the system we developed for NTCIR1 and IREX. It submitted retrieval results\nfor JJ and CC tasks. A variety of parameters were tried with the system. It\nused such characteristics of newspapers as locational information in the CC\ntasks. The system got good results for both of the tasks. The other system is a\nportable system which avoids free parameters as much as possible. The system\nsubmitted retrieval results for JJ, JE, EE, EJ, and CC tasks. The system\nautomatically determined the number of top documents and the weight of the\noriginal query used in automatic-feedback retrieval. It also determined\nrelevant terms quite robustly. For EJ and JE tasks, it used document expansion\nto augment the initial queries. It achieved good results, except on the CC\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103026v1",
            "date": "2001-03-29 23:08:33+00:00",
            "title": "A Decision Tree of Bigrams is an Accurate Predictor of Word Sense",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper presents a corpus-based approach to word sense disambiguation\nwhere a decision tree assigns a sense to an ambiguous word based on the bigrams\nthat occur nearby. This approach is evaluated using the sense-tagged corpora\nfrom the 1998 SENSEVAL word sense disambiguation exercise. It is more accurate\nthan the average results reported for 30 of 36 words, and is more accurate than\nthe best results for 19 of 36 words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104010v1",
            "date": "2001-04-03 23:22:17+00:00",
            "title": "Type Arithmetics: Computation based on the theory of types",
            "authors": [
                "Oleg Kiselyov"
            ],
            "abstract": "The present paper shows meta-programming turn programming, which is rich\nenough to express arbitrary arithmetic computations. We demonstrate a type\nsystem that implements Peano arithmetics, slightly generalized to negative\nnumbers. Certain types in this system denote numerals. Arithmetic operations on\nsuch types-numerals - addition, subtraction, and even division - are expressed\nas type reduction rules executed by a compiler. A remarkable trait is that\ndivision by zero becomes a type error - and reported as such by a compiler.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104019v1",
            "date": "2001-04-27 22:50:31+00:00",
            "title": "Dynamic Nonlocal Language Modeling via Hierarchical Topic-Based Adaptation",
            "authors": [
                "Radu Florian",
                "David Yarowsky"
            ],
            "abstract": "This paper presents a novel method of generating and applying hierarchical,\ndynamic topic-based language models. It proposes and evaluates new cluster\ngeneration, hierarchical smoothing and adaptive topic-probability estimation\ntechniques. These combined models help capture long-distance lexical\ndependencies. Experiments on the Broadcast News corpus show significant\nimprovement in perplexity (10.5% overall and 33.5% on target vocabulary).",
            "journal ref": "Proceedings of the 37th Annual Meeting of the ACL, pages 167-174,\n  College Park, Maryland",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104022v1",
            "date": "2001-04-30 15:12:52+00:00",
            "title": "Microplanning with Communicative Intentions: The SPUD System",
            "authors": [
                "Matthew Stone",
                "Christine Doran",
                "Bonnie Webber",
                "Tonia Bleam",
                "Martha Palmer"
            ],
            "abstract": "The process of microplanning encompasses a range of problems in Natural\nLanguage Generation (NLG), such as referring expression generation, lexical\nchoice, and aggregation, problems in which a generator must bridge underlying\ndomain-specific representations and general linguistic representations. In this\npaper, we describe a uniform approach to microplanning based on declarative\nrepresentations of a generator's communicative intent. These representations\ndescribe the results of NLG: communicative intent associates the concrete\nlinguistic structure planned by the generator with inferences that show how the\nmeaning of that structure communicates needed information about some\napplication domain in the current discourse context. Our approach, implemented\nin the SPUD (sentence planning using description) microplanner, uses the\nlexicalized tree-adjoining grammar formalism (LTAG) to connect structure to\nmeaning and uses modal logic programming to connect meaning to context. At the\nsame time, communicative intent representations provide a resource for the\nprocess of NLG. Using representations of communicative intent, a generator can\naugment the syntax, semantics and pragmatics of an incomplete sentence\nsimultaneously, and can assess its progress on the various problems of\nmicroplanning incrementally. The declarative formulation of communicative\nintent translates into a well-defined methodology for designing grammatical and\nconceptual resources which the generator can use to achieve desired\nmicroplanning behavior in a specified domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105001v1",
            "date": "2001-05-02 05:29:27+00:00",
            "title": "Correction of Errors in a Modality Corpus Used for Machine Translation by Using Machine-learning Method",
            "authors": [
                "Masaki Murata",
                "Masao Utiyama",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "We performed corpus correction on a modality corpus for machine translation\nby using such machine-learning methods as the maximum-entropy method. We thus\nconstructed a high-quality modality corpus based on corpus correction. We\ncompared several kinds of methods for corpus correction in our experiments and\ndeveloped a good method for corpus correction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105002v1",
            "date": "2001-05-02 07:49:14+00:00",
            "title": "Man [and Woman] vs. Machine: A Case Study in Base Noun Phrase Learning",
            "authors": [
                "Eric Brill",
                "Grace Ngai"
            ],
            "abstract": "A great deal of work has been done demonstrating the ability of machine\nlearning algorithms to automatically extract linguistic knowledge from\nannotated corpora. Very little work has gone into quantifying the difference in\nability at this task between a person and a machine. This paper is a first step\nin that direction.",
            "journal ref": "Proceedings of the 37th Annual Meeting of the Association of\n  Computational Linguistics, pages 65-72, College Park, MD, USA (1999)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105005v1",
            "date": "2001-05-04 08:55:02+00:00",
            "title": "A Complete WordNet1.5 to WordNet1.6 Mapping",
            "authors": [
                "J. Daud\u00e9",
                "L. Padr\u00f3",
                "G. Rigau"
            ],
            "abstract": "We describe a robust approach for linking already existing lexical/semantic\nhierarchies. We use a constraint satisfaction algorithm (relaxation labelling)\nto select --among a set of candidates-- the node in a target taxonomy that\nbests matches each node in a source taxonomy. In this paper we present the\ncomplete mapping of the nominal, verbal, adjectival and adverbial parts of\nWordNet 1.5 onto WordNet 1.6.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105012v1",
            "date": "2001-05-07 14:25:22+00:00",
            "title": "Joint and conditional estimation of tagging and parsing models",
            "authors": [
                "Mark Johnson"
            ],
            "abstract": "This paper compares two different ways of estimating statistical language\nmodels. Many statistical NLP tagging and parsing models are estimated by\nmaximizing the (joint) likelihood of the fully-observed training data. However,\nsince these applications only require the conditional probability\ndistributions, these distributions can in principle be learnt by maximizing the\nconditional likelihood of the training data. Perhaps somewhat surprisingly,\nmodels estimated by maximizing the joint were superior to models estimated by\nmaximizing the conditional, even though some of the latter models intuitively\nhad access to ``more information''.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105016v1",
            "date": "2001-05-08 15:35:07+00:00",
            "title": "Probabilistic top-down parsing and language modeling",
            "authors": [
                "Brian Roark"
            ],
            "abstract": "This paper describes the functioning of a broad-coverage probabilistic\ntop-down parser, and its application to the problem of language modeling for\nspeech recognition. The paper first introduces key notions in language modeling\nand probabilistic parsing, and briefly reviews some previous approaches to\nusing syntactic structure for language modeling. A lexicalized probabilistic\ntop-down parser is then presented, which performs very well, in terms of both\nthe accuracy of returned parses and the efficiency with which they are found,\nrelative to the best broad-coverage statistical parsers. A new language model\nwhich utilizes probabilistic top-down parsing is then outlined, and empirical\nresults show that it improves upon previous work in test corpus perplexity.\nInterpolation with a trigram model yields an exceptional improvement relative\nto the improvement observed by other models, demonstrating the degree to which\nthe information captured by our parsing model is orthogonal to that captured by\na trigram model. A small recognition experiment also demonstrates the utility\nof the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105019v1",
            "date": "2001-05-09 17:01:10+00:00",
            "title": "Robust Probabilistic Predictive Syntactic Processing",
            "authors": [
                "Brian Roark"
            ],
            "abstract": "This thesis presents a broad-coverage probabilistic top-down parser, and its\napplication to the problem of language modeling for speech recognition. The\nparser builds fully connected derivations incrementally, in a single pass from\nleft-to-right across the string. We argue that the parsing approach that we\nhave adopted is well-motivated from a psycholinguistic perspective, as a model\nthat captures probabilistic dependencies between lexical items, as part of the\nprocess of building connected syntactic structures. The basic parser and\nconditional probability models are presented, and empirical results are\nprovided for its parsing accuracy on both newspaper text and spontaneous\ntelephone conversations. Modifications to the probability model are presented\nthat lead to improved performance. A new language model which uses the output\nof the parser is then defined. Perplexity and word error rate reduction are\ndemonstrated over trigram models, even when the trigram is trained on\nsignificantly more data. Interpolation on a word-by-word basis with a trigram\nmodel yields additional improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105023v1",
            "date": "2001-05-14 09:05:45+00:00",
            "title": "Generating a 3D Simulation of a Car Accident from a Written Description in Natural Language: the CarSim System",
            "authors": [
                "Sylvain Dupuy",
                "Arjan Egges",
                "Vincent Legendre",
                "Pierre Nugues"
            ],
            "abstract": "This paper describes a prototype system to visualize and animate 3D scenes\nfrom car accident reports, written in French. The problem of generating such a\n3D simulation can be divided into two subtasks: the linguistic analysis and the\nvirtual scene generation. As a means of communication between these two\nmodules, we first designed a template formalism to represent a written accident\nreport. The CarSim system first processes written reports, gathers relevant\ninformation, and converts it into a formal description. Then, it creates the\ncorresponding 3D scene and animates the vehicles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105035v1",
            "date": "2001-05-30 03:55:23+00:00",
            "title": "Historical Dynamics of Lexical System as Random Walk Process",
            "authors": [
                "Victor Kromer"
            ],
            "abstract": "It is offered to consider word meanings changes in diachrony as\nsemicontinuous random walk with reflecting and swallowing screens. The basic\ncharacteristics of word life cycle are defined. Verification of the model has\nbeen realized on the data of Russian words distribution on various age periods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105037v1",
            "date": "2001-05-31 18:08:57+00:00",
            "title": "Integrating Prosodic and Lexical Cues for Automatic Topic Segmentation",
            "authors": [
                "G. Tur",
                "D. Hakkani-Tur",
                "A. Stolcke",
                "E. Shriberg"
            ],
            "abstract": "We present a probabilistic model that uses both prosodic and lexical cues for\nthe automatic segmentation of speech into topically coherent units. We propose\ntwo methods for combining lexical and prosodic information using hidden Markov\nmodels and decision trees. Lexical information is obtained from a speech\nrecognizer, and prosodic features are extracted automatically from speech\nwaveforms. We evaluate our approach on the Broadcast News corpus, using the\nDARPA-TDT evaluation metrics. Results show that the prosodic model alone is\ncompetitive with word-based segmentation methods. Furthermore, we achieve a\nsignificant reduction in error by combining the prosodic and word-based\nknowledge sources.",
            "journal ref": "Computation Linguistics 27(1), 31-57, March 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106015v1",
            "date": "2001-06-10 06:14:09+00:00",
            "title": "Organizing Encyclopedic Knowledge based on the Web and its Application to Question Answering",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "We propose a method to generate large-scale encyclopedic knowledge, which is\nvaluable for much NLP research, based on the Web. We first search the Web for\npages containing a term in question. Then we use linguistic patterns and HTML\nstructures to extract text fragments describing the term. Finally, we organize\nextracted term descriptions based on word senses and domains. In addition, we\napply an automatically generated encyclopedia to a question answering system\ntargeting the Japanese Information-Technology Engineers Examination.",
            "journal ref": "Proceedings of the 39th Annual Meeting of the Association for\n  Computational Linguistics (ACL-EACL 2001), pp.196-203, July. 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106043v1",
            "date": "2001-06-20 14:16:17+00:00",
            "title": "Using the Distribution of Performance for Studying Statistical NLP Systems and Corpora",
            "authors": [
                "Yuval Krymolowski"
            ],
            "abstract": "Statistical NLP systems are frequently evaluated and compared on the basis of\ntheir performances on a single split of training and test data. Results\nobtained using a single split are, however, subject to sampling noise. In this\npaper we argue in favour of reporting a distribution of performance figures,\nobtained by resampling the training data, rather than a single number. The\nadditional information from distributions can be used to make statistically\nquantified statements about differences across parameter settings, systems, and\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106047v1",
            "date": "2001-06-21 20:37:43+00:00",
            "title": "Modeling informational novelty in a conversational system with a hybrid statistical and grammar-based approach to natural language generation",
            "authors": [
                "Adwait Ratnaparkhi"
            ],
            "abstract": "We present a hybrid statistical and grammar-based system for surface natural\nlanguage generation (NLG) that uses grammar rules, conditions on using those\ngrammar rules, and corpus statistics to determine the word order. We also\ndescribe how this surface NLG module is implemented in a prototype\nconversational system, and how it attempts to model informational novelty by\nvarying the word order. Using a combination of rules and statistical\ninformation, the conversational system expresses the novel information\ndifferently than the given information, based on the run-time dialog state. We\nalso discuss our plans for evaluating the generation strategy.",
            "journal ref": "Proceedings of the NAACL Workshop on Adaptation in Dialogue\n  Systems, June 4, 2001, Pittsburgh, PA, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107005v1",
            "date": "2001-07-03 10:27:44+00:00",
            "title": "The Role of Conceptual Relations in Word Sense Disambiguation",
            "authors": [
                "David Fernandez-Amoros",
                "Julio Gonzalo",
                "Felisa Verdejo"
            ],
            "abstract": "We explore many ways of using conceptual distance measures in Word Sense\nDisambiguation, starting with the Agirre-Rigau conceptual density measure. We\nuse a generalized form of this measure, introducing many (parameterized)\nrefinements and performing an exhaustive evaluation of all meaningful\ncombinations. We finally obtain a 42% improvement over the original algorithm,\nand show that measures of conceptual distance are not worse indicators for\nsense disambiguation than measures based on word-coocurrence (exemplified by\nthe Lesk algorithm). Our results, however, reinforce the idea that only a\ncombination of different sources of knowledge might eventually lead to accurate\nword sense disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107006v1",
            "date": "2001-07-03 18:06:05+00:00",
            "title": "Looking Under the Hood : Tools for Diagnosing your Question Answering Engine",
            "authors": [
                "Eric Breck",
                "Marc Light",
                "Gideon S. Mann",
                "Ellen Riloff",
                "Brianne Brown Pranav Anand",
                "Mats Rooth",
                "Michael Thelen"
            ],
            "abstract": "In this paper we analyze two question answering tasks : the TREC-8 question\nanswering task and a set of reading comprehension exams. First, we show that\nQ/A systems perform better when there are multiple answer opportunities per\nquestion. Next, we analyze common approaches to two subproblems: term overlap\nfor answer sentence identification, and answer typing for short answer\nextraction. We present general tools for analyzing the strengths and\nlimitations of techniques for these subproblems. Our results quantify the\nlimitations of both term overlap and answer typing to distinguish between\ncompeting answer candidates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107016v1",
            "date": "2001-07-15 12:51:01+00:00",
            "title": "Introduction to the CoNLL-2001 Shared Task: Clause Identification",
            "authors": [
                "Erik F. Tjong Kim Sang",
                "Herve Dejean"
            ],
            "abstract": "We describe the CoNLL-2001 shared task: dividing text into clauses. We give\nbackground information on the data sets, present a general overview of the\nsystems that have taken part in the shared task and briefly discuss their\nperformance.",
            "journal ref": "In: Walter Daelemans and Remi Zajac (eds.), Proceedings of\n  CoNLL-2001, Toulouse, France, 2001, pp. 53-57",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107017v1",
            "date": "2001-07-15 13:21:48+00:00",
            "title": "Learning Computational Grammars",
            "authors": [
                "John Nerbonne",
                "Anja Belz",
                "Nicola Cancedda",
                "Herve Dejean",
                "James Hammerton",
                "Rob Koeling",
                "Stasinos Konstantopoulos",
                "Miles Osborne",
                "Franck Thollard",
                "Erik F. Tjong Kim Sang"
            ],
            "abstract": "This paper reports on the \"Learning Computational Grammars\" (LCG) project, a\npostdoc network devoted to studying the application of machine learning\ntechniques to grammars suitable for computational use. We were interested in a\nmore systematic survey to understand the relevance of many factors to the\nsuccess of learning, esp. the availability of annotated data, the kind of\ndependencies in the data, and the availability of knowledge bases (grammars).\nWe focused on syntax, esp. noun phrase (NP) syntax.",
            "journal ref": "In: Walter Daelemans and Remi Zajac (eds.), Proceedings of\n  CoNLL-2001, Toulouse, France, 2001, pp. 97-104",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107018v1",
            "date": "2001-07-15 13:32:36+00:00",
            "title": "Combining a self-organising map with memory-based learning",
            "authors": [
                "James Hammerton",
                "Erik F. Tjong Kim Sang"
            ],
            "abstract": "Memory-based learning (MBL) has enjoyed considerable success in corpus-based\nnatural language processing (NLP) tasks and is thus a reliable method of\ngetting a high-level of performance when building corpus-based NLP systems.\nHowever there is a bottleneck in MBL whereby any novel testing item has to be\ncompared against all the training items in memory base. For this reason there\nhas been some interest in various forms of memory editing whereby some method\nof selecting a subset of the memory base is employed to reduce the number of\ncomparisons. This paper investigates the use of a modified self-organising map\n(SOM) to select a subset of the memory items for comparison. This method\ninvolves reducing the number of comparisons to a value proportional to the\nsquare root of the number of training items. The method is tested on the\nidentification of base noun-phrases in the Wall Street Journal corpus, using\nsections 15 to 18 for training and section 20 for testing.",
            "journal ref": "In: Walter Daelemans and Remi Zajac (eds.), Proceedings of\n  CoNLL-2001, Toulouse, France, 2001, pp. 9-14",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107019v2",
            "date": "2001-07-16 22:59:15+00:00",
            "title": "Applying Natural Language Generation to Indicative Summarization",
            "authors": [
                "Min-Yen Kan",
                "Kathleen R. McKeown",
                "Judith L. Klavans"
            ],
            "abstract": "The task of creating indicative summaries that help a searcher decide whether\nto read a particular document is a difficult task. This paper examines the\nindicative summarization task from a generation perspective, by first analyzing\nits required content via published guidelines and corpus analysis. We show how\nthese summaries can be factored into a set of document features, and how an\nimplemented content planner uses the topicality document feature to create\nindicative multidocument query-based summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107020v1",
            "date": "2001-07-17 15:26:13+00:00",
            "title": "Transformation-Based Learning in the Fast Lane",
            "authors": [
                "Grace Ngai",
                "Radu Florian"
            ],
            "abstract": "Transformation-based learning has been successfully employed to solve many\nnatural language processing problems. It achieves state-of-the-art performance\non many natural language processing tasks and does not overtrain easily.\nHowever, it does have a serious drawback: the training time is often\nintorelably long, especially on the large corpora which are often used in NLP.\nIn this paper, we present a novel and realistic method for speeding up the\ntraining time of a transformation-based learner without sacrificing\nperformance. The paper compares and contrasts the training time needed and\nperformance achieved by our modified learner with two other systems: a standard\ntransformation-based learner, and the ICA system \\cite{hepple00:tbl}. The\nresults of these experiments show that our system is able to achieve a\nsignificant improvement in training time while still achieving the same\nperformance as a standard transformation-based learner. This is a valuable\ncontribution to systems and algorithms which utilize transformation-based\nlearning at any part of the execution.",
            "journal ref": "Proceedings of the Second Conference of the North American Chapter\n  of the Association for Computational Linguistics, pages 40-47, Pittsburgh,\n  PA, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107021v1",
            "date": "2001-07-17 15:43:03+00:00",
            "title": "Multidimensional Transformation-Based Learning",
            "authors": [
                "Radu Florian",
                "Grace Ngai"
            ],
            "abstract": "This paper presents a novel method that allows a machine learning algorithm\nfollowing the transformation-based learning paradigm \\cite{brill95:tagging} to\nbe applied to multiple classification tasks by training jointly and\nsimultaneously on all fields. The motivation for constructing such a system\nstems from the observation that many tasks in natural language processing are\nnaturally composed of multiple subtasks which need to be resolved\nsimultaneously; also tasks usually learned in isolation can possibly benefit\nfrom being learned in a joint framework, as the signals for the extra tasks\nusually constitute inductive bias.\n  The proposed algorithm is evaluated in two experiments: in one, the system is\nused to jointly predict the part-of-speech and text chunks/baseNP chunks of an\nEnglish corpus; and in the second it is used to learn the joint prediction of\nword segment boundaries and part-of-speech tagging for Chinese. The results\nshow that the simultaneous learning of multiple tasks does achieve an\nimprovement in each task upon training the same tasks sequentially. The\npart-of-speech tagging result of 96.63% is state-of-the-art for individual\nsystems on the particular train/test split.",
            "journal ref": "Proceedings of the 5th Computational Natural Language Learning\n  Workshop (CoNNL-2001), pages 1-8, Toulouse, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0108005v1",
            "date": "2001-08-09 19:24:28+00:00",
            "title": "A Bit of Progress in Language Modeling",
            "authors": [
                "Joshua Goodman"
            ],
            "abstract": "In the past several years, a number of different language modeling\nimprovements over simple trigram models have been found, including caching,\nhigher-order n-grams, skipping, interpolated Kneser-Ney smoothing, and\nclustering. We present explorations of variations on, or of the limits of, each\nof these techniques, including showing that sentence mixture models may have\nmore potential. While all of these techniques have been studied separately,\nthey have rarely been studied in combination. We find some significant\ninteractions, especially with smoothing and clustering techniques. We compare a\ncombination of all techniques together to a Katz smoothed trigram model with no\ncount cutoffs. We achieve perplexity reductions between 38% and 50% (1 bit of\nentropy), depending on training data size, as well as a word error rate\nreduction of 8.9%. Our perplexity reductions are perhaps the highest reported\ncompared to a fair baseline. This is the extended version of the paper; it\ncontains additional details and proofs, and is designed to be a good\nintroduction to the state of the art in language modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0108006v1",
            "date": "2001-08-09 19:17:58+00:00",
            "title": "Classes for Fast Maximum Entropy Training",
            "authors": [
                "Joshua Goodman"
            ],
            "abstract": "Maximum entropy models are considered by many to be one of the most promising\navenues of language modeling research. Unfortunately, long training times make\nmaximum entropy research difficult. We present a novel speedup technique: we\nchange the form of the model to use classes. Our speedup works by creating two\nmaximum entropy models, the first of which predicts the class of each word, and\nthe second of which predicts the word itself. This factoring of the model leads\nto fewer non-zero indicator functions, and faster normalization, achieving\nspeedups of up to a factor of 35 over one of the best previous techniques. It\nalso results in typically slightly lower perplexities. The same trick can be\nused to speed training of other machine learning techniques, e.g. neural\nnetworks, applied to any problem with a large number of outputs, such as\nlanguage modeling.",
            "journal ref": "Proceedings of ICASSP-2001, Utah, May 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0108022v1",
            "date": "2001-08-29 03:59:31+00:00",
            "title": "Portability of Syntactic Structure for Language Modeling",
            "authors": [
                "Ciprian Chelba"
            ],
            "abstract": "The paper presents a study on the portability of statistical syntactic\nknowledge in the framework of the structured language model (SLM). We\ninvestigate the impact of porting SLM statistics from the Wall Street Journal\n(WSJ) to the Air Travel Information System (ATIS) domain. We compare this\napproach to applying the Microsoft rule-based parser (NLPwin) for the ATIS data\nand to using a small amount of data manually parsed at UPenn for gathering the\nintial SLM statistics. Surprisingly, despite the fact that it performs modestly\nin perplexity (PPL), the model initialized on WSJ parses outperforms the other\ninitialization methods based on in-domain annotated data, achieving a\nsignificant 0.4% absolute and 7% relative reduction in word error rate (WER)\nover a baseline system whose word error rate is 5.8%; the improvement measured\nrelative to the minimum WER achievable on the N-best lists we worked with is\n12%.",
            "journal ref": "ICASSP 2001 Proceedings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109010v2",
            "date": "2001-09-09 16:41:59+00:00",
            "title": "Anaphora and Discourse Structure",
            "authors": [
                "Bonnie Webber",
                "Matthew Stone",
                "Aravind Joshi",
                "Alistair Knott"
            ],
            "abstract": "We argue in this paper that many common adverbial phrases generally taken to\nsignal a discourse relation between syntactically connected units within\ndiscourse structure, instead work anaphorically to contribute relational\nmeaning, with only indirect dependence on discourse structure. This allows a\nsimpler discourse structure to provide scaffolding for compositional semantics,\nand reveals multiple ways in which the relational meaning conveyed by adverbial\nconnectives can interact with that associated with discourse structure. We\nconclude by sketching out a lexicalised grammar for discourse that facilitates\ndiscourse interpretation as a product of compositional rules, anaphor\nresolution and inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109015v1",
            "date": "2001-09-13 14:52:41+00:00",
            "title": "Boosting Trees for Anti-Spam Email Filtering",
            "authors": [
                "Xavier Carreras",
                "Lluis Marquez"
            ],
            "abstract": "This paper describes a set of comparative experiments for the problem of\nautomatically filtering unwanted electronic mail messages. Several variants of\nthe AdaBoost algorithm with confidence-rated predictions [Schapire & Singer,\n99] have been applied, which differ in the complexity of the base learners\nconsidered. Two main conclusions can be drawn from our experiments: a) The\nboosting-based methods clearly outperform the baseline learning algorithms\n(Naive Bayes and Induction of Decision Trees) on the PU1 corpus, achieving very\nhigh levels of the F1 measure; b) Increasing the complexity of the base\nlearners allows to obtain better ``high-precision'' classifiers, which is a\nvery important issue when misclassification costs are considered.",
            "journal ref": "Proceedings of RANLP-2001, pp. 58-64, Bulgaria, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109020v1",
            "date": "2001-09-15 22:44:55+00:00",
            "title": "Modelling Semantic Association and Conceptual Inheritance for Semantic Analysis",
            "authors": [
                "Pascal Vaillant"
            ],
            "abstract": "Allowing users to interact through language borders is an interesting\nchallenge for information technology. For the purpose of a computer assisted\nlanguage learning system, we have chosen icons for representing meaning on the\ninput interface, since icons do not depend on a particular language. However, a\nkey limitation of this type of communication is the expression of articulated\nideas instead of isolated concepts. We propose a method to interpret sequences\nof icons as complex messages by reconstructing the relations between concepts,\nso as to build conceptual graphs able to represent meaning and to be used for\nnatural language sentence generation. This method is based on an electronic\ndictionary containing semantic information.",
            "journal ref": "Springer LNCS (LNAI) 2166 (2001), 54-61",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109029v1",
            "date": "2001-09-18 14:00:27+00:00",
            "title": "Learning class-to-class selectional preferences",
            "authors": [
                "E. Agirre",
                "D. Martinez"
            ],
            "abstract": "Selectional preference learning methods have usually focused on word-to-class\nrelations, e.g., a verb selects as its subject a given nominal class. This\npapers extends previous statistical models to class-to-class preferences, and\npresents a model that learns selectional preferences for classes of verbs. The\nmotivation is twofold: different senses of a verb may have different\npreferences, and some classes of verbs can share preferences. The model is\ntested on a word sense disambiguation task which uses subject-verb and\nobject-verb relationships extracted from a small sense-disambiguated corpus.",
            "journal ref": "Proceedings of the Workshop \"Computational Natural Language\n  Learning\" (CoNLL-2001). In conjunction with ACL'2001/EACL'2001. Toulouse,\n  France. 6-7th July 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109030v1",
            "date": "2001-09-18 14:14:52+00:00",
            "title": "Knowledge Sources for Word Sense Disambiguation",
            "authors": [
                "Eneko Agirre",
                "David Martinez"
            ],
            "abstract": "Two kinds of systems have been defined during the long history of WSD:\nprincipled systems that define which knowledge types are useful for WSD, and\nrobust systems that use the information sources at hand, such as, dictionaries,\nlight-weight ontologies or hand-tagged corpora. This paper tries to systematize\nthe relation between desired knowledge types and actual information sources. We\nalso compare the results for a wide range of algorithms that have been\nevaluated on a common test setting in our research group. We hope that this\nanalysis will help change the shift from systems based on information sources\nto systems based on knowledge sources. This study might also shed some light on\nsemi-automatic acquisition of desired knowledge types from existing resources.",
            "journal ref": "Proceedings of the Fourth International Conference TSD 2001, Plzen\n  (Pilsen), Czech Republic, September 2001. Published in the Springer Verlag\n  Lecture Notes in Computer Science series. Vaclav Matousek, Pavel Mautner,\n  Roman Moucek, Karel Tauser (eds.)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109031v2",
            "date": "2001-09-18 14:18:58+00:00",
            "title": "Enriching WordNet concepts with topic signatures",
            "authors": [
                "Eneko Agirre",
                "Olatz Ansa",
                "Eduard Hovy",
                "David Martinez"
            ],
            "abstract": "This paper explores the possibility of enriching the content of existing\nontologies. The overall goal is to overcome the lack of topical links among\nconcepts in WordNet. Each concept is to be associated to a topic signature,\ni.e., a set of related words with associated weights. The signatures can be\nautomatically constructed from the WWW or from sense-tagged corpora. Both\napproaches are compared and evaluated on a word sense disambiguation task. The\nresults show that it is possible to construct clean signatures from the WWW\nusing some filtering techniques.",
            "journal ref": "Proceedings of the NAACL workshop on WordNet and Other lexical\n  Resources: Applications, Extensions and Customizations. Pittsburg, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109039v1",
            "date": "2001-09-20 06:42:11+00:00",
            "title": "Testing for Mathematical Lineation in Jim Crace's \"Quarantine\" and T. S. Eliot's \"Four Quartets\"",
            "authors": [
                "John Constable",
                "Hideaki Aoyama"
            ],
            "abstract": "The mathematical distinction between prose and verse may be detected in\nwritings that are not apparently lineated, for example in T. S. Eliot's \"Burnt\nNorton\", and Jim Crace's \"Quarantine\". In this paper we offer comments on\nappropriate statistical methods for such work, and also on the nature of formal\ninnovation in these two texts. Additional remarks are made on the roots of\nlineation as a metrical form, and on the prose-verse continuum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110015v1",
            "date": "2001-10-03 18:34:36+00:00",
            "title": "Richer Syntactic Dependencies for Structured Language Modeling",
            "authors": [
                "Ciprian Chelba",
                "Peng Xu"
            ],
            "abstract": "The paper investigates the use of richer syntactic dependencies in the\nstructured language model (SLM). We present two simple methods of enriching the\ndependencies in the syntactic parse trees used for intializing the SLM. We\nevaluate the impact of both methods on the perplexity (PPL) and\nword-error-rate(WER, N-best rescoring) performance of the SLM. We show that the\nnew model achieves an improvement in PPL and WER over the baseline results\nreported using the SLM on the UPenn Treebank and Wall Street Journal (WSJ)\ncorpora, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110027v1",
            "date": "2001-10-11 11:29:49+00:00",
            "title": "Part-of-Speech Tagging with Two Sequential Transducers",
            "authors": [
                "Andre Kempe"
            ],
            "abstract": "We present a method of constructing and using a cascade consisting of a left-\nand a right-sequential finite-state transducer (FST), T1 and T2, for\npart-of-speech (POS) disambiguation. Compared to an HMM, this FST cascade has\nthe advantage of significantly higher processing speed, but at the cost of\nslightly lower accuracy. Applications such as Information Retrieval, where the\nspeed can be more important than accuracy, could benefit from this approach.\n  In the process of tagging, we first assign every word a unique ambiguity\nclass c_i that can be looked up in a lexicon encoded by a sequential FST. Every\nc_i is denoted by a single symbol, e.g. [ADJ_NOUN], although it represents a\nset of alternative tags that a given word can occur with. The sequence of the\nc_i of all words of one sentence is the input to our FST cascade. It is mapped\nby T1, from left to right, to a sequence of reduced ambiguity classes r_i.\nEvery r_i is denoted by a single symbol, although it represents a set of\nalternative tags. Intuitively, T1 eliminates the less likely tags from c_i,\nthus creating r_i. Finally, T2 maps the sequence of r_i, from right to left, to\na sequence of single POS tags t_i. Intuitively, T2 selects the most likely t_i\nfrom every r_i.\n  The probabilities of all t_i, r_i, and c_i are used only at compile time, not\nat run time. They do not (directly) occur in the FSTs, but are \"implicitly\ncontained\" in their structure.",
            "journal ref": "Proc. CLIN 2000, pp. 88-96, Tilburg, The Netherlands. November 3",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110050v1",
            "date": "2001-10-24 11:01:08+00:00",
            "title": "What is the minimal set of fragments that achieves maximal parse accuracy?",
            "authors": [
                "Rens Bod"
            ],
            "abstract": "We aim at finding the minimal set of fragments which achieves maximal parse\naccuracy in Data Oriented Parsing. Experiments with the Penn Wall Street\nJournal treebank show that counts of almost arbitrary fragments within parse\ntrees are important, leading to improved parse accuracy over previous models\ntested on this treebank (a precision of 90.8% and a recall of 90.6%). We\nisolate some dependency relations which previous models neglect but which\ncontribute to higher parse accuracy.",
            "journal ref": "Proceedings ACL'2001, Toulouse, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110051v1",
            "date": "2001-10-24 11:30:50+00:00",
            "title": "Combining semantic and syntactic structure for language modeling",
            "authors": [
                "Rens Bod"
            ],
            "abstract": "Structured language models for speech recognition have been shown to remedy\nthe weaknesses of n-gram models. All current structured language models are,\nhowever, limited in that they do not take into account dependencies between\nnon-headwords. We show that non-headword dependencies contribute to\nsignificantly improved word error rate, and that a data-oriented parsing model\ntrained on semantically and syntactically annotated data can exploit these\ndependencies. This paper also contains the first DOP model trained by means of\na maximum likelihood reestimation procedure, which solves some of the\ntheoretical shortcomings of previous DOP models.",
            "journal ref": "Proceedings ICSLP'2000, Beijing, China",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0111064v1",
            "date": "2001-11-30 20:30:52+00:00",
            "title": "A procedure for unsupervised lexicon learning",
            "authors": [
                "Anand Venkataraman"
            ],
            "abstract": "We describe an incremental unsupervised procedure to learn words from\ntranscribed continuous speech. The algorithm is based on a conservative and\ntraditional statistical model, and results of empirical tests show that it is\ncompetitive with other algorithms that have been proposed recently for this\ntask.",
            "journal ref": "Proceedings of the eighteenth international conference on machine\n  learning, ICML-01, pp.569--576, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0111065v1",
            "date": "2001-11-30 20:40:50+00:00",
            "title": "A Statistical Model for Word Discovery in Transcribed Speech",
            "authors": [
                "Anand Venkataraman"
            ],
            "abstract": "A statistical model for segmentation and word discovery in continuous speech\nis presented. An incremental unsupervised learning algorithm to infer word\nboundaries based on this model is described. Results of empirical tests showing\nthat the algorithm is competitive with other models that have been used for\nsimilar tasks are also presented.",
            "journal ref": "Computational Linguistics, 27(3), pp.352--372, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0112003v1",
            "date": "2001-12-05 05:35:07+00:00",
            "title": "Using a Support-Vector Machine for Japanese-to-English Translation of Tense, Aspect, and Modality",
            "authors": [
                "Masaki Murata",
                "Kiyotaka Uchimoto",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "This paper describes experiments carried out using a variety of\nmachine-learning methods, including the k-nearest neighborhood method that was\nused in a previous study, for the translation of tense, aspect, and modality.\nIt was found that the support-vector machine method was the most precise of all\nthe methods tested.",
            "journal ref": "ACL Workshop, the Data-Driven Machine Translation, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0112004v1",
            "date": "2001-12-05 05:48:21+00:00",
            "title": "Part of Speech Tagging in Thai Language Using Support Vector Machine",
            "authors": [
                "Masaki Murata",
                "Qing Ma",
                "Hitoshi Isahara"
            ],
            "abstract": "The elastic-input neuro tagger and hybrid tagger, combined with a neural\nnetwork and Brill's error-driven learning, have already been proposed for the\npurpose of constructing a practical tagger using as little training data as\npossible. When a small Thai corpus is used for training, these taggers have\ntagging accuracies of 94.4% and 95.5% (accounting only for the ambiguous words\nin terms of the part of speech), respectively. In this study, in order to\nconstruct more accurate taggers we developed new tagging methods using three\nmachine learning methods: the decision-list, maximum entropy, and support\nvector machine methods. We then performed tagging experiments by using these\nmethods. Our results showed that the support vector machine method has the best\nprecision (96.1%), and that it is capable of improving the accuracy of tagging\nin the Thai language. Finally, we theoretically examined all these methods and\ndiscussed how the improvements were achived.",
            "journal ref": "NLPRS'2001 Workshop, the Second Workshop on Natural Language\n  Processing and Neural Networks (NLPNN2001)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0112005v1",
            "date": "2001-12-05 05:56:13+00:00",
            "title": "Universal Model for Paraphrasing -- Using Transformation Based on a Defined Criteria --",
            "authors": [
                "Masaki Murata",
                "Hitoshi Isahara"
            ],
            "abstract": "This paper describes a universal model for paraphrasing that transforms\naccording to defined criteria. We showed that by using different criteria we\ncould construct different kinds of paraphrasing systems including one for\nanswering questions, one for compressing sentences, one for polishing up, and\none for transforming written language to spoken language.",
            "journal ref": "NLPRS'2001, Workshop on Automatic Paraphrasing: Theories and\n  Applications",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0109218v1",
            "date": "2001-09-12 18:08:07+00:00",
            "title": "Entropic analysis of the role of words in literary texts",
            "authors": [
                "Marcelo A. Montemurro",
                "Damian H. Zanette"
            ],
            "abstract": "Beyond the local constraints imposed by grammar, words concatenated in long\nsequences carrying a complex message show statistical regularities that may\nreflect their linguistic role in the message. In this paper, we perform a\nsystematic statistical analysis of the use of words in literary English\ncorpora. We show that there is a quantitative relation between the role of\ncontent words in literary English and the Shannon information entropy defined\nover an appropriate probability distribution. Without assuming any previous\nknowledge about the syntactic structure of language, we are able to cluster\ncertain groups of words according to their specific role in the text.",
            "journal ref": "None",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/cs/0102019v1",
            "date": "2001-02-22 03:50:56+00:00",
            "title": "Easy and Hard Constraint Ranking in OT: Algorithms and Complexity",
            "authors": [
                "Jason Eisner"
            ],
            "abstract": "We consider the problem of ranking a set of OT constraints in a manner\nconsistent with data.\n  We speed up Tesar and Smolensky's RCD algorithm to be linear on the number of\nconstraints. This finds a ranking so each attested form x_i beats or ties a\nparticular competitor y_i. We also generalize RCD so each x_i beats or ties all\npossible competitors.\n  Alas, this more realistic version of learning has no polynomial algorithm\nunless P=NP! Indeed, not even generation does. So one cannot improve\nqualitatively upon brute force:\n  Merely checking that a single (given) ranking is consistent with given forms\nis coNP-complete if the surface forms are fully observed and Delta_2^p-complete\nif not. Indeed, OT generation is OptP-complete. As for ranking, determining\nwhether any consistent ranking exists is coNP-hard (but in Delta_2^p) if the\nforms are fully observed, and Sigma_2^p-complete if not.\n  Finally, we show that generation and ranking are easier in derivational\ntheories: in P, and NP-complete.",
            "journal ref": "Jason Eisner, Lauri Karttunen and Alain Theriault (eds.),\n  Finite-State Phonology: Proceedings of the 5th Workshop of the ACL Special\n  Interest Group in Computational Phonology (SIGPHON), pp. 22-33. Luxembourg,\n  August 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104005v1",
            "date": "2001-04-03 14:09:12+00:00",
            "title": "Bootstrapping Structure using Similarity",
            "authors": [
                "Menno van Zaanen"
            ],
            "abstract": "In this paper a new similarity-based learning algorithm, inspired by string\nedit-distance (Wagner and Fischer, 1974), is applied to the problem of\nbootstrapping structure from scratch. The algorithm takes a corpus of\nunannotated sentences as input and returns a corpus of bracketed sentences. The\nmethod works on pairs of unstructured sentences or sentences partially\nbracketed by the algorithm that have one or more words in common. It finds\nparts of sentences that are interchangeable (i.e. the parts of the sentences\nthat are different in both sentences). These parts are taken as possible\nconstituents of the same type. While this corresponds to the basic\nbootstrapping step of the algorithm, further structure may be learned from\ncomparison with other (similar) sentences.\n  We used this method for bootstrapping structure from the flat sentences of\nthe Penn Treebank ATIS corpus, and compared the resulting structured sentences\nto the structured sentences in the ATIS corpus. Similarly, the algorithm was\ntested on the OVIS corpus. We obtained 86.04 % non-crossing brackets precision\non the ATIS corpus and 89.39 % non-crossing brackets precision on the OVIS\ncorpus.",
            "journal ref": "Computational Linguistics in the Netherlands 1999 - Selected\n  Papers from the Tenth CLIN Meeting, pages 235-245",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104006v1",
            "date": "2001-04-03 14:20:26+00:00",
            "title": "ABL: Alignment-Based Learning",
            "authors": [
                "Menno van Zaanen"
            ],
            "abstract": "This paper introduces a new type of grammar learning algorithm, inspired by\nstring edit distance (Wagner and Fischer, 1974). The algorithm takes a corpus\nof flat sentences as input and returns a corpus of labelled, bracketed\nsentences. The method works on pairs of unstructured sentences that have one or\nmore words in common. When two sentences are divided into parts that are the\nsame in both sentences and parts that are different, this information is used\nto find parts that are interchangeable. These parts are taken as possible\nconstituents of the same type. After this alignment learning step, the\nselection learning step selects the most probable constituents from all\npossible constituents.\n  This method was used to bootstrap structure on the ATIS corpus (Marcus et\nal., 1993) and on the OVIS (Openbaar Vervoer Informatie Systeem (OVIS) stands\nfor Public Transport Information System.) corpus (Bonnema et al., 1997). While\nthe results are encouraging (we obtained up to 89.25 % non-crossing brackets\nprecision), this paper will point out some of the shortcomings of our approach\nand will suggest possible solutions.",
            "journal ref": "Proceedings of the 18th International Conference on Computational\n  Linguistics (COLING); Saarbrucken, Germany. pages 961-967",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104007v1",
            "date": "2001-04-03 15:03:16+00:00",
            "title": "Bootstrapping Syntax and Recursion using Alignment-Based Learning",
            "authors": [
                "Menno van Zaanen"
            ],
            "abstract": "This paper introduces a new type of unsupervised learning algorithm, based on\nthe alignment of sentences and Harris's (1951) notion of interchangeability.\nThe algorithm is applied to an untagged, unstructured corpus of natural\nlanguage sentences, resulting in a labelled, bracketed version of the corpus.\nFirstly, the algorithm aligns all sentences in the corpus in pairs, resulting\nin a partition of the sentences consisting of parts of the sentences that are\nsimilar in both sentences and parts that are dissimilar. This information is\nused to find (possibly overlapping) constituents. Next, the algorithm selects\n(non-overlapping) constituents. Several instances of the algorithm are applied\nto the ATIS corpus (Marcus et al., 1993) and the OVIS (Openbaar Vervoer\nInformatie Systeem (OVIS) stands for Public Transport Information System.)\ncorpus (Bonnema et al., 1997). Apart from the promising numerical results, the\nmost striking result is that even the simplest algorithm based on alignment\nlearns recursion.",
            "journal ref": "Proceedings of the Seventeenth International Conference on Machine\n  Learning. pages 1063-1070",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0104020v1",
            "date": "2001-04-27 23:16:21+00:00",
            "title": "Coaxing Confidences from an Old Friend: Probabilistic Classifications from Transformation Rule Lists",
            "authors": [
                "Radu Florian",
                "John C. Henderson",
                "Grace Ngai"
            ],
            "abstract": "Transformation-based learning has been successfully employed to solve many\nnatural language processing problems. It has many positive features, but one\ndrawback is that it does not provide estimates of class membership\nprobabilities.\n  In this paper, we present a novel method for obtaining class membership\nprobabilities from a transformation-based rule list classifier. Three\nexperiments are presented which measure the modeling accuracy and cross-entropy\nof the probabilistic classifier on unseen data and the degree to which the\noutput probabilities from the classifier can be used to estimate confidences in\nits classification decisions.\n  The results of these experiments show that, for the task of text chunking,\nthe estimates produced by this technique are more informative than those\ngenerated by a state-of-the-art decision tree.",
            "journal ref": "Proceedings of the Fifth Conference on Empirical Methods in\n  Natural Language Processing, pages 26-34, Hong Kong (2000)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105003v1",
            "date": "2001-05-02 08:39:32+00:00",
            "title": "Rule Writing or Annotation: Cost-efficient Resource Usage for Base Noun Phrase Chunking",
            "authors": [
                "Grace Ngai",
                "David Yarowsky"
            ],
            "abstract": "This paper presents a comprehensive empirical comparison between two\napproaches for developing a base noun phrase chunker: human rule writing and\nactive learning using interactive real-time human annotation. Several novel\nvariations on active learning are investigated, and underlying cost models for\ncross-modal machine learning comparison are presented and explored. Results\nshow that it is more efficient and more successful by several measures to train\na system using active learning annotation rather than hand-crafted rule writing\nat a comparable level of human labor investment.",
            "journal ref": "Proceedings of the 38th Annual Meeting of the Association for\n  Computational Linguistics, pages 117-125, Hong Kong (2000)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0105030v1",
            "date": "2001-05-21 16:53:55+00:00",
            "title": "The OLAC Metadata Set and Controlled Vocabularies",
            "authors": [
                "Steven Bird",
                "Gary Simons"
            ],
            "abstract": "As language data and associated technologies proliferate and as the language\nresources community rapidly expands, it has become difficult to locate and\nreuse existing resources. Are there any lexical resources for such-and-such a\nlanguage? What tool can work with transcripts in this particular format? What\nis a good format to use for linguistic data of this type? Questions like these\ndominate many mailing lists, since web search engines are an unreliable way to\nfind language resources. This paper describes a new digital infrastructure for\nlanguage resource discovery, based on the Open Archives Initiative, and called\nOLAC -- the Open Language Archives Community. The OLAC Metadata Set and the\nassociated controlled vocabularies facilitate consistent description and\nfocussed searching. We report progress on the metadata set and controlled\nvocabularies, describing current issues and soliciting input from the language\nresources community.",
            "journal ref": "Proceedings of the ACL/EACL Workshop on Sharing Tools and\n  Resources for Research and Education, Toulouse, July 2001, Association for\n  Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106011v1",
            "date": "2001-06-07 15:47:31+00:00",
            "title": "Computational properties of environment-based disambiguation",
            "authors": [
                "William Schuler"
            ],
            "abstract": "The standard pipeline approach to semantic processing, in which sentences are\nmorphologically and syntactically resolved to a single tree before they are\ninterpreted, is a poor fit for applications such as natural language\ninterfaces. This is because the environment information, in the form of the\nobjects and events in the application's run-time environment, cannot be used to\ninform parsing decisions unless the input sentence is semantically analyzed,\nbut this does not occur until after parsing in the single-tree semantic\narchitecture. This paper describes the computational properties of an\nalternative architecture, in which semantic analysis is performed on all\npossible interpretations during parsing, in polynomial time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106039v1",
            "date": "2001-06-17 20:45:37+00:00",
            "title": "Iterative Residual Rescaling: An Analysis and Generalization of LSI",
            "authors": [
                "Rie Kubota Ando",
                "Lillian Lee"
            ],
            "abstract": "We consider the problem of creating document representations in which\ninter-document similarity measurements correspond to semantic similarity. We\nfirst present a novel subspace-based framework for formalizing this task. Using\nthis framework, we derive a new analysis of Latent Semantic Indexing (LSI),\nshowing a precise relationship between its performance and the uniformity of\nthe underlying distribution of documents over topics. This analysis helps\nexplain the improvements gained by Ando's (2000) Iterative Residual Rescaling\n(IRR) algorithm: IRR can compensate for distributional non-uniformity. A\nfurther benefit of our framework is that it provides a well-motivated,\neffective method for automatically determining the rescaling factor IRR depends\non, leading to further improvements. A series of experiments over various\nsettings and with several evaluation metrics validates our claims.",
            "journal ref": "Proceedings of the 24th SIGIR, pp. 154--162, 2001.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106040v1",
            "date": "2001-06-19 14:56:02+00:00",
            "title": "Stacking classifiers for anti-spam filtering of e-mail",
            "authors": [
                "G. Sakkis",
                "I. Androutsopoulos",
                "G. Paliouras",
                "V. Karkaletsis",
                "C. D. Spyropoulos",
                "P. Stamatopoulos"
            ],
            "abstract": "We evaluate empirically a scheme for combining classifiers, known as stacked\ngeneralization, in the context of anti-spam filtering, a novel cost-sensitive\napplication of text categorization. Unsolicited commercial e-mail, or \"spam\",\nfloods mailboxes, causing frustration, wasting bandwidth, and exposing minors\nto unsuitable content. Using a public corpus, we show that stacking can improve\nthe efficiency of automatically induced anti-spam filters, and that such\nfilters can be used in real-life applications.",
            "journal ref": "Proceedings of \"Empirical Methods in Natural Language Processing\"\n  (EMNLP 2001), L. Lee and D. Harman (Eds.), pp. 44-50, Carnegie Mellon\n  University, Pittsburgh, PA, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106059v1",
            "date": "2001-06-29 08:41:02+00:00",
            "title": "CHR as grammar formalism. A first report",
            "authors": [
                "Henning Christiansen"
            ],
            "abstract": "Grammars written as Constraint Handling Rules (CHR) can be executed as\nefficient and robust bottom-up parsers that provide a straightforward,\nnon-backtracking treatment of ambiguity. Abduction with integrity constraints\nas well as other dynamic hypothesis generation techniques fit naturally into\nsuch grammars and are exemplified for anaphora resolution, coordination and\ntext interpretation.",
            "journal ref": "Proc. of ERCIM Workshop on Constraints, Prague, Czech Republic,\n  June 18-20, 2001",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0108023v1",
            "date": "2001-08-29 04:00:24+00:00",
            "title": "Information Extraction Using the Structured Language Model",
            "authors": [
                "Ciprian Chelba",
                "Milind Mahajan"
            ],
            "abstract": "The paper presents a data-driven approach to information extraction (viewed\nas template filling) using the structured language model (SLM) as a statistical\nparser. The task of template filling is cast as constrained parsing using the\nSLM. The model is automatically trained from a set of sentences annotated with\nframe/slot labels and spans. Training proceeds in stages: first a constrained\nsyntactic parser is trained such that the parses on training data meet the\nspecified semantic spans, then the non-terminal labels are enriched to contain\nsemantic information and finally a constrained syntactic+semantic parser is\ntrained on the parse trees resulting from the previous stage. Despite the small\namount of training data used, the model is shown to outperform the slot level\naccuracy of a simple semantic grammar authored manually for the MiPad ---\npersonal information management --- task.",
            "journal ref": "EMNLP/NAACL 2001 Conference Proceedings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109013v1",
            "date": "2001-09-11 12:17:04+00:00",
            "title": "Conceptual Analysis of Lexical Taxonomies: The Case of WordNet Top-Level",
            "authors": [
                "Aldo Gangemi",
                "Nicola Guarino",
                "Alessandro Oltramari"
            ],
            "abstract": "In this paper we propose an analysis and an upgrade of WordNet's top-level\nsynset taxonomy. We briefly review WordNet and identify its main semantic\nlimitations. Some principles from a forthcoming OntoClean methodology are\napplied to the ontological analysis of WordNet. A revised top-level taxonomy is\nproposed, which is meant to be more conceptually rigorous, cognitively\ntransparent, and efficiently exploitable in several applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0109023v1",
            "date": "2001-09-17 14:41:14+00:00",
            "title": "Integrating Multiple Knowledge Sources for Robust Semantic Parsing",
            "authors": [
                "Jordi Atserias",
                "Lluis Padro",
                "German Rigau"
            ],
            "abstract": "This work explores a new robust approach for Semantic Parsing of unrestricted\ntexts. Our approach considers Semantic Parsing as a Consistent Labelling\nProblem (CLP), allowing the integration of several knowledge types (syntactic\nand semantic) obtained from different sources (linguistic and statistic). The\ncurrent implementation obtains 95% accuracy in model identification and 72% in\ncase-role filling.",
            "journal ref": "Proceedings of Euroconference on Recent Advances in Natural\n  Language Processing (RANLP'01), p.8-14. Tzigov Chark, Bulgaria. Sept. 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110014v1",
            "date": "2001-10-03 12:45:41+00:00",
            "title": "The Open Language Archives Community and Asian Language Resources",
            "authors": [
                "Steven Bird",
                "Gary Simons",
                "Chu-Ren Huang"
            ],
            "abstract": "The Open Language Archives Community (OLAC) is a new project to build a\nworldwide system of federated language archives based on the Open Archives\nInitiative and the Dublin Core Metadata Initiative. This paper aims to\ndisseminate the OLAC vision to the language resources community in Asia, and to\nshow language technologists and linguists how they can document their tools and\ndata in such a way that others can easily discover them. We describe OLAC and\nthe OLAC Metadata Set, then discuss two key issues in the Asian context:\nlanguage classification and multilingual resource classification.",
            "journal ref": "Proceedings of the Workshop on Language Resources in Asia, 6th\n  Natural Language Processing Pacific Rim Symposium (NLPRS), Tokyo, November\n  2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110057v1",
            "date": "2001-10-29 16:55:32+00:00",
            "title": "Generating Multilingual Personalized Descriptions of Museum Exhibits - The M-PIRO Project",
            "authors": [
                "Ion Androutsopoulos",
                "Vassiliki Kokkinaki",
                "Aggeliki Dimitromanolaki",
                "Jo Calder",
                "Jon Oberlander",
                "Elena Not"
            ],
            "abstract": "This paper provides an overall presentation of the M-PIRO project. M-PIRO is\ndeveloping technology that will allow museums to generate automatically textual\nor spoken descriptions of exhibits for collections available over the Web or in\nvirtual reality environments. The descriptions are generated in several\nlanguages from information in a language-independent database and small\nfragments of text, and they can be tailored according to the backgrounds of the\nusers, their ages, and their previous interaction with the system. An authoring\ntool allows museum curators to update the system's database and to control the\nlanguage and content of the resulting descriptions. Although the project is\nstill in progress, a Web-based demonstrator that supports English, Greek and\nItalian is already available, and it is used throughout the paper to highlight\nthe capabilities of the emerging technology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0111003v1",
            "date": "2001-11-01 03:02:19+00:00",
            "title": "The Use of Classifiers in Sequential Inference",
            "authors": [
                "Vasin Punyakanok",
                "Dan Roth"
            ],
            "abstract": "We study the problem of combining the outcomes of several different\nclassifiers in a way that provides a coherent inference that satisfies some\nconstraints. In particular, we develop two general approaches for an important\nsubproblem-identifying phrase structure. The first is a Markovian approach that\nextends standard HMMs to allow the use of a rich observation structure and of\ngeneral classifiers to model state-observation dependencies. The second is an\nextension of constraint satisfaction formalisms. We develop efficient\ncombination algorithms under both models and study them experimentally in the\ncontext of shallow parsing.",
            "journal ref": "Advances in Neural Information Processing Systems 13",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0112010v1",
            "date": "2001-12-10 13:01:21+00:00",
            "title": "A Straightforward Approach to Morphological Analysis and Synthesis",
            "authors": [
                "Kyriakos N. Sgarbas",
                "Nikos D. Fakotakis",
                "George K. Kokkinakis"
            ],
            "abstract": "In this paper we present a lexicon-based approach to the problem of\nmorphological processing. Full-form words, lemmas and grammatical tags are\ninterconnected in a DAWG. Thus, the process of analysis/synthesis is reduced to\na search in the graph, which is very fast and can be performed even if several\npieces of information are missing from the input. The contents of the DAWG are\nupdated using an on-line incremental process. The proposed approach is language\nindependent and it does not utilize any morphophonetic rules or any other\nspecial linguistic information.",
            "journal ref": "Proc. COMLEX 2000, Workshop on Computational Lexicography and\n  Multimedia Dictionaries, pp.31-34, Kato Achaia, Greece, 22-23 September 2000.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0112018v1",
            "date": "2001-12-15 19:43:17+00:00",
            "title": "Fast Context-Free Grammar Parsing Requires Fast Boolean Matrix Multiplication",
            "authors": [
                "Lillian Lee"
            ],
            "abstract": "In 1975, Valiant showed that Boolean matrix multiplication can be used for\nparsing context-free grammars (CFGs), yielding the asympotically fastest\n(although not practical) CFG parsing algorithm known. We prove a dual result:\nany CFG parser with time complexity $O(g n^{3 - \\epsilson})$, where $g$ is the\nsize of the grammar and $n$ is the length of the input string, can be\nefficiently converted into an algorithm to multiply $m \\times m$ Boolean\nmatrices in time $O(m^{3 - \\epsilon/3})$.\n  Given that practical, substantially sub-cubic Boolean matrix multiplication\nalgorithms have been quite difficult to find, we thus explain why there has\nbeen little progress in developing practical, substantially sub-cubic general\nCFG parsers. In proving this result, we also develop a formalization of the\nnotion of parsing.",
            "journal ref": "Journal of the ACM 49(1), pp. 1--15, January 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0104066v2",
            "date": "2001-04-04 02:01:01+00:00",
            "title": "Beyond the Zipf-Mandelbrot law in quantitative linguistics",
            "authors": [
                "Marcelo A. Montemurro"
            ],
            "abstract": "In this paper the Zipf-Mandelbrot law is revisited in the context of\nlinguistics. Despite its widespread popularity the Zipf--Mandelbrot law can\nonly describe the statistical behaviour of a rather restricted fraction of the\ntotal number of words contained in some given corpus. In particular, we focus\nour attention on the important deviations that become statistically relevant as\nlarger corpora are considered and that ultimately could be understood as\nsalient features of the underlying complex process of language generation.\nFinally, it is shown that all the different observed regimes can be accurately\nencompassed within a single mathematical framework recently introduced by C.\nTsallis.",
            "journal ref": "None",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/cs/0103002v1",
            "date": "2001-03-02 00:20:01+00:00",
            "title": "Quantitative Neural Network Model of the Tip-of-the-Tongue Phenomenon Based on Synthesized Memory-Psycholinguistic-Metacognitive Approach",
            "authors": [
                "Petro M. Gopych"
            ],
            "abstract": "A new three-stage computer artificial neural network model of the\ntip-of-the-tongue phenomenon is proposed. Each word's node is build from some\ninterconnected learned auto-associative two-layer neural networks each of which\nrepresents separate word's semantic, lexical, or phonological components. The\nmodel synthesizes memory, psycholinguistic, and metamemory approaches, bridges\nspeech errors and naming chronometry research traditions, and can explain\nquantitatively many tip-of-the-tongue effects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106044v1",
            "date": "2001-06-20 19:01:41+00:00",
            "title": "A Sequential Model for Multi-Class Classification",
            "authors": [
                "Yair Even-Zohar",
                "Dan Roth"
            ],
            "abstract": "Many classification problems require decisions among a large number of\ncompeting classes. These tasks, however, are not handled well by general\npurpose learning methods and are usually addressed in an ad-hoc fashion. We\nsuggest a general approach -- a sequential learning model that utilizes\nclassifiers to sequentially restrict the number of competing classes while\nmaintaining, with high probability, the presence of the true outcome in the\ncandidates set. Some theoretical and computational properties of the model are\ndiscussed and we argue that these are important in NLP-like domains. The\nadvantages of the model are illustrated in an experiment in part-of-speech\ntagging.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107012v1",
            "date": "2001-07-09 21:00:42+00:00",
            "title": "Three-Stage Quantitative Neural Network Model of the Tip-of-the-Tongue Phenomenon",
            "authors": [
                "Petro M. Gopych"
            ],
            "abstract": "A new three-stage computer artificial neural network model of the\ntip-of-the-tongue phenomenon is shortly described, and its stochastic nature\nwas demonstrated. A way to calculate strength and appearance probability of\ntip-of-the-tongue states, neural network mechanism of feeling-of-knowing\nphenomenon are proposed. The model synthesizes memory, psycholinguistic, and\nmetamemory approaches, bridges speech errors and naming chronometry research\ntraditions. A model analysis of a tip-of-the-tongue case from Anton Chekhov's\nshort story 'A Horsey Name' is performed. A new 'throw-up-one's-arms effect' is\ndefined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0107032v1",
            "date": "2001-07-23 11:06:45+00:00",
            "title": "Coupled Clustering: a Method for Detecting Structural Correspondence",
            "authors": [
                "Zvika Marx",
                "Ido Dagan",
                "Joachim Buhmann"
            ],
            "abstract": "This paper proposes a new paradigm and computational framework for\nidentification of correspondences between sub-structures of distinct composite\nsystems. For this, we define and investigate a variant of traditional data\nclustering, termed coupled clustering, which simultaneously identifies\ncorresponding clusters within two data sets. The presented method is\ndemonstrated and evaluated for detecting topical correspondences in textual\ncorpora.",
            "journal ref": "In: C. E. Brodley and A. P. Danyluk (eds.), Proceedings of the\n  18th International Conference on Machine Learning (ICML 2001), pp. 353-360",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0110041v1",
            "date": "2001-10-18 19:39:22+00:00",
            "title": "Towards Solving the Interdisciplinary Language Barrier Problem",
            "authors": [
                "Sebastien Paquet"
            ],
            "abstract": "This work aims to make it easier for a specialist in one field to find and\nexplore ideas from another field which may be useful in solving a new problem\narising in his practice. It presents a methodology which serves to represent\nthe relationships that exist between concepts, problems, and solution patterns\nfrom different fields of human activity in the form of a graph. Our approach is\nbased upon generalization and specialization relationships and problem solving.\nIt is simple enough to be understood quite easily, and general enough to enable\ncoherent integration of concepts and problems from virtually any field. We have\nbuilt an implementation which uses the World Wide Web as a support to allow\nnavigation between graph nodes and collaborative development of the graph.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/cs/0106016v1",
            "date": "2001-06-10 14:56:51+00:00",
            "title": "File mapping Rule-based DBMS and Natural Language Processing",
            "authors": [
                "Vjacheslav M. Novikov"
            ],
            "abstract": "This paper describes the system of storage, extract and processing of\ninformation structured similarly to the natural language. For recursive\ninference the system uses the rules having the same representation, as the\ndata. The environment of storage of information is provided with the File\nMapping (SHM) mechanism of operating system. In the paper the main principles\nof construction of dynamic data structure and language for record of the\ninference rules are stated; the features of available implementation are\nconsidered and the description of the application realizing semantic\ninformation retrieval on the natural language is given.",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2002": [
        {
            "url": "http://arxiv.org/abs/cs/0204003v1",
            "date": "2002-04-02 21:14:40+00:00",
            "title": "Blind Normalization of Speech From Different Channels and Speakers",
            "authors": [
                "David N. Levin"
            ],
            "abstract": "This paper describes representations of time-dependent signals that are\ninvariant under any invertible time-independent transformation of the signal\ntime series. Such a representation is created by rescaling the signal in a\nnon-linear dynamic manner that is determined by recently encountered signal\nlevels. This technique may make it possible to normalize signals that are\nrelated by channel-dependent and speaker-dependent transformations, without\nhaving to characterize the form of the signal transformations, which remain\nunknown. The technique is illustrated by applying it to the time-dependent\nspectra of speech that has been filtered to simulate the effects of different\nchannels. The experimental results show that the rescaled speech\nrepresentations are largely normalized (i.e., channel-independent), despite the\nchannel-dependence of the raw (unrescaled) speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204007v1",
            "date": "2002-04-03 18:55:01+00:00",
            "title": "An Integrated Framework for Treebanks and Multilayer Annotations",
            "authors": [
                "Scott Cotton",
                "Steven Bird"
            ],
            "abstract": "Treebank formats and associated software tools are proliferating rapidly,\nwith little consideration for interoperability. We survey a wide variety of\ntreebank structures and operations, and show how they can be mapped onto the\nannotation graph model, and leading to an integrated framework encompassing\ntree and non-tree annotations alike. This development opens up new\npossibilities for managing and exploiting multilayer annotations.",
            "journal ref": "Proceedings of the Third International Conference on Language\n  Resources and Evaluation, Paris: European Language Resources Association,\n  2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204022v1",
            "date": "2002-04-10 15:37:26+00:00",
            "title": "Annotation Graphs and Servers and Multi-Modal Resources: Infrastructure for Interdisciplinary Education, Research and Development",
            "authors": [
                "Christopher Cieri",
                "Steven Bird"
            ],
            "abstract": "Annotation graphs and annotation servers offer infrastructure to support the\nanalysis of human language resources in the form of time-series data such as\ntext, audio and video. This paper outlines areas of common need among empirical\nlinguists and computational linguists. After reviewing examples of data and\ntools used or under development for each of several areas, it proposes a common\nframework for future tool development, data annotation and resource sharing\nbased upon annotation graphs and servers.",
            "journal ref": "Proceedings of ACL Workshop on Sharing Tools and Resources for\n  Research and Education, Toulouse, July 2001, pp 23-30",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204023v1",
            "date": "2002-04-10 15:49:24+00:00",
            "title": "Computational Phonology",
            "authors": [
                "Steven Bird"
            ],
            "abstract": "Phonology, as it is practiced, is deeply computational. Phonological analysis\nis data-intensive and the resulting models are nothing other than specialized\ndata structures and algorithms. In the past, phonological computation -\nmanaging data and developing analyses - was done manually with pencil and\npaper. Increasingly, with the proliferation of affordable computers, IPA fonts\nand drawing software, phonologists are seeking to move their computation work\nonline. Computational Phonology provides the theoretical and technological\nframework for this migration, building on methodologies and tools from\ncomputational linguistics. This piece consists of an apology for computational\nphonology, a history, and an overview of current research.",
            "journal ref": "Oxford International Encyclopedia of Linguistics, 2nd Edition,\n  2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204025v1",
            "date": "2002-04-11 11:22:43+00:00",
            "title": "Phonology",
            "authors": [
                "Steven Bird"
            ],
            "abstract": "Phonology is the systematic study of the sounds used in language, their\ninternal structure, and their composition into syllables, words and phrases.\nComputational phonology is the application of formal and computational\ntechniques to the representation and processing of phonological information.\nThis chapter will present the fundamentals of descriptive phonology along with\na brief overview of computational phonology.",
            "journal ref": "In Ruslan Mitkov (ed) (2002). Oxford Handbook of Computational\n  Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204027v1",
            "date": "2002-04-11 16:41:28+00:00",
            "title": "Integrating selectional preferences in WordNet",
            "authors": [
                "Eneko Agirre",
                "David Martinez"
            ],
            "abstract": "Selectional preference learning methods have usually focused on word-to-class\nrelations, e.g., a verb selects as its subject a given nominal class. This\npaper extends previous statistical models to class-to-class preferences, and\npresents a model that learns selectional preferences for classes of verbs,\ntogether with an algorithm to integrate the learned preferences in WordNet. The\ntheoretical motivation is twofold: different senses of a verb may have\ndifferent preferences, and classes of verbs may share preferences. On the\npractical side, class-to-class selectional preferences can be learned from\nuntagged corpora (the same as word-to-class), they provide selectional\npreferences for less frequent word senses via inheritance, and more important,\nthey allow for easy integration in WordNet. The model is trained on\nsubject-verb and object-verb relationships extracted from a small corpus\ndisambiguated with WordNet senses. Examples are provided illustrating that the\ntheoretical motivations are well founded, and showing that the approach is\nfeasible. Experimental results on a word sense disambiguation task are also\nprovided.",
            "journal ref": "Proceedings of First International WordNet Conference. Mysore\n  (India). 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204028v1",
            "date": "2002-04-12 07:41:55+00:00",
            "title": "Decision Lists for English and Basque",
            "authors": [
                "Eneko Agirre",
                "David Martinez"
            ],
            "abstract": "In this paper we describe the systems we developed for the English (lexical\nand all-words) and Basque tasks. They were all supervised systems based on\nYarowsky's Decision Lists. We used Semcor for training in the English all-words\ntask. We defined different feature sets for each language. For Basque, in order\nto extract all the information from the text, we defined features that have not\nbeen used before in the literature, using a morphological analyzer. We also\nimplemented systems that selected automatically good features and were able to\nobtain a prefixed precision (85%) at the cost of coverage. The systems that\nused all the features were identified as BCU-ehu-dlist-all and the systems that\nselected some features as BCU-ehu-dlist-best.",
            "journal ref": "Proceedings of the SENSEVAL-2 Workshop. In conjunction with\n  ACL'2001/EACL'2001. Toulouse",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204029v1",
            "date": "2002-04-12 07:49:32+00:00",
            "title": "The Basque task: did systems perform in the upperbound?",
            "authors": [
                "Eneko Agirre",
                "Elena Garcia",
                "Mikel Lersundi",
                "David Martinez",
                "Eli Pociello"
            ],
            "abstract": "In this paper we describe the Senseval 2 Basque lexical-sample task. The task\ncomprised 40 words (15 nouns, 15 verbs and 10 adjectives) selected from Euskal\nHiztegia, the main Basque dictionary. Most examples were taken from the\nEgunkaria newspaper. The method used to hand-tag the examples produced low\ninter-tagger agreement (75%) before arbitration. The four competing systems\nattained results well above the most frequent baseline and the best system\nscored 75% precision at 100% coverage. The paper includes an analysis of the\ntagging procedure used, as well as the performance of the competing systems. In\nparticular, we argue that inter-tagger agreement is not a real upperbound for\nthe Basque WSD task.",
            "journal ref": "Proceedings of the SENSEVAL-2 Workshop. In conjunction with\n  ACL'2001/EACL'2001. Toulouse",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204049v1",
            "date": "2002-04-24 14:48:31+00:00",
            "title": "Memory-Based Shallow Parsing",
            "authors": [
                "Erik F. Tjong Kim Sang"
            ],
            "abstract": "We present memory-based learning approaches to shallow parsing and apply\nthese to five tasks: base noun phrase identification, arbitrary base phrase\nrecognition, clause detection, noun phrase parsing and full parsing. We use\nfeature selection techniques and system combination methods for improving the\nperformance of the memory-based learner. Our approach is evaluated on standard\ndata sets and the results are compared with that of other systems. This reveals\nthat our approach works well for base phrase identification while its\napplication towards recognizing embedded structures leaves some room for\nimprovement.",
            "journal ref": "Journal of Machine Learning Research, volume 2 (March), 2002, pp.\n  559-594",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205006v1",
            "date": "2002-05-08 14:39:19+00:00",
            "title": "Unsupervised discovery of morphologically related words based on orthographic and semantic similarity",
            "authors": [
                "Marco Baroni",
                "Johannes Matiasek",
                "Harald Trost"
            ],
            "abstract": "We present an algorithm that takes an unannotated corpus as its input, and\nreturns a ranked list of probable morphologically related pairs as its output.\nThe algorithm tries to discover morphologically related pairs by looking for\npairs that are both orthographically and semantically similar, where\northographic similarity is measured in terms of minimum edit distance, and\nsemantic similarity is measured in terms of mutual information. The procedure\ndoes not rely on a morpheme concatenation model, nor on distributional\nproperties of word substrings (such as affix frequency). Experiments with\nGerman and English input give encouraging results, both in terms of precision\n(proportion of good pairs found at various cutoff points of the ranked list),\nand in terms of a qualitative analysis of the types of morphological patterns\ndiscovered by the algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205009v1",
            "date": "2002-05-10 18:55:25+00:00",
            "title": "Mostly-Unsupervised Statistical Segmentation of Japanese Kanji Sequences",
            "authors": [
                "Rie Kubota Ando",
                "Lillian Lee"
            ],
            "abstract": "Given the lack of word delimiters in written Japanese, word segmentation is\ngenerally considered a crucial first step in processing Japanese texts. Typical\nJapanese segmentation algorithms rely either on a lexicon and syntactic\nanalysis or on pre-segmented data; but these are labor-intensive, and the\nlexico-syntactic techniques are vulnerable to the unknown word problem. In\ncontrast, we introduce a novel, more robust statistical method utilizing\nunsegmented training data. Despite its simplicity, the algorithm yields\nperformance on long kanji sequences comparable to and sometimes surpassing that\nof state-of-the-art morphological analyzers over a variety of error metrics.\nThe algorithm also outperforms another mostly-unsupervised statistical\nalgorithm previously proposed for Chinese.\n  Additionally, we present a two-level annotation scheme for Japanese to\nincorporate multiple segmentation granularities, and introduce two novel\nevaluation metrics, both based on the notion of a compatible bracket, that can\naccount for multiple granularities simultaneously.",
            "journal ref": "Natural Language Engineering 9 (2), pp. 127--149, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205017v1",
            "date": "2002-05-13 11:18:08+00:00",
            "title": "Ellogon: A New Text Engineering Platform",
            "authors": [
                "Georgios Petasis",
                "Vangelis Karkaletsis",
                "Georgios Paliouras",
                "Ion Androutsopoulos",
                "Constantine D. Spyropoulos"
            ],
            "abstract": "This paper presents Ellogon, a multi-lingual, cross-platform, general-purpose\ntext engineering environment. Ellogon was designed in order to aid both\nresearchers in natural language processing, as well as companies that produce\nlanguage engineering systems for the end-user. Ellogon provides a powerful\nTIPSTER-based infrastructure for managing, storing and exchanging textual data,\nembedding and managing text processing components as well as visualising\ntextual data and their associated linguistic information. Among its key\nfeatures are full Unicode support, an extensive multi-lingual graphical user\ninterface, its modular architecture and the reduced hardware requirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205027v1",
            "date": "2002-05-17 09:33:52+00:00",
            "title": "A variable-free dynamic semantics",
            "authors": [
                "Chung-chieh Shan"
            ],
            "abstract": "I propose a variable-free treatment of dynamic semantics. By \"dynamic\nsemantics\" I mean analyses of donkey sentences (\"Every farmer who owns a donkey\nbeats it\") and other binding and anaphora phenomena in natural language where\nmeanings of constituents are updates to information states, for instance as\nproposed by Groenendijk and Stokhof. By \"variable-free\" I mean denotational\nsemantics in which functional combinators replace variable indices and\nassignment functions, for instance as advocated by Jacobson.\n  The new theory presented here achieves a compositional treatment of dynamic\nanaphora that does not involve assignment functions, and separates the\ncombinatorics of variable-free semantics from the particular linguistic\nphenomena it treats. Integrating variable-free semantics and dynamic semantics\ngives rise to interactions that make new empirical predictions, for example\n\"donkey weak crossover\" effects.",
            "journal ref": "Proceedings of the 13th Amsterdam Colloquium, ed. Robert van Rooy\n  and Martin Stokhof, 204-209 (2001)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205028v1",
            "date": "2002-05-17 12:51:00+00:00",
            "title": "NLTK: The Natural Language Toolkit",
            "authors": [
                "Edward Loper",
                "Steven Bird"
            ],
            "abstract": "NLTK, the Natural Language Toolkit, is a suite of open source program\nmodules, tutorials and problem sets, providing ready-to-use computational\nlinguistics courseware. NLTK covers symbolic and statistical natural language\nprocessing, and is interfaced to annotated corpora. Students augment and\nreplace existing components, learn structured programming by example, and\nmanipulate sophisticated models from the outset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205057v1",
            "date": "2002-05-21 14:37:22+00:00",
            "title": "Unsupervised Discovery of Morphemes",
            "authors": [
                "Mathias Creutz",
                "Krista Lagus"
            ],
            "abstract": "We present two methods for unsupervised segmentation of words into\nmorpheme-like units. The model utilized is especially suited for languages with\na rich morphology, such as Finnish. The first method is based on the Minimum\nDescription Length (MDL) principle and works online. In the second method,\nMaximum Likelihood (ML) optimization is used. The quality of the segmentations\nis measured using an evaluation method that compares the segmentations produced\nto an existing morphological analysis. Experiments on both Finnish and English\ncorpora show that the presented methods perform well compared to a current\nstate-of-the-art system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205065v1",
            "date": "2002-05-25 21:32:09+00:00",
            "title": "Bootstrapping Lexical Choice via Multiple-Sequence Alignment",
            "authors": [
                "Regina Barzilay",
                "Lillian Lee"
            ],
            "abstract": "An important component of any generation system is the mapping dictionary, a\nlexicon of elementary semantic expressions and corresponding natural language\nrealizations. Typically, labor-intensive knowledge-based methods are used to\nconstruct the dictionary. We instead propose to acquire it automatically via a\nnovel multiple-pass algorithm employing multiple-sequence alignment, a\ntechnique commonly used in bioinformatics. Crucially, our method leverages\nlatent information contained in multi-parallel corpora -- datasets that supply\nseveral verbalizations of the corresponding semantics rather than just one.\n  We used our techniques to generate natural language versions of\ncomputer-generated mathematical proofs, with good results on both a\nper-component and overall-output basis. For example, in evaluations involving a\ndozen human judges, our system produced output whose readability and\nfaithfulness to the semantic input rivaled that of a traditional generation\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205067v1",
            "date": "2002-05-27 18:42:10+00:00",
            "title": "Evaluating the Effectiveness of Ensembles of Decision Trees in Disambiguating Senseval Lexical Samples",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper presents an evaluation of an ensemble--based system that\nparticipated in the English and Spanish lexical sample tasks of Senseval-2. The\nsystem combines decision trees of unigrams, bigrams, and co--occurrences into a\nsingle classifier. The analysis is extended to include the Senseval-1 data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205068v1",
            "date": "2002-05-27 18:49:01+00:00",
            "title": "Assessing System Agreement and Instance Difficulty in the Lexical Sample Tasks of Senseval-2",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper presents a comparative evaluation among the systems that\nparticipated in the Spanish and English lexical sample tasks of Senseval-2. The\nfocus is on pairwise comparisons among systems to assess the degree to which\nthey agree, and on measuring the difficulty of the test instances included in\nthese tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205069v1",
            "date": "2002-05-27 18:57:11+00:00",
            "title": "Machine Learning with Lexical Features: The Duluth Approach to Senseval-2",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the sixteen Duluth entries in the Senseval-2 comparative\nexercise among word sense disambiguation systems. There were eight pairs of\nDuluth systems entered in the Spanish and English lexical sample tasks. These\nare all based on standard machine learning algorithms that induce classifiers\nfrom sense-tagged training text where the context in which ambiguous words\noccur are represented by simple lexical features. These are highly portable,\nrobust methods that can serve as a foundation for more tailored approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206014v1",
            "date": "2002-06-09 10:07:36+00:00",
            "title": "A Method for Open-Vocabulary Speech-Driven Text Retrieval",
            "authors": [
                "Atsushi Fujii",
                "Katunobu Itou",
                "Tetsuya Ishikawa"
            ],
            "abstract": "While recent retrieval techniques do not limit the number of index terms,\nout-of-vocabulary (OOV) words are crucial in speech recognition. Aiming at\nretrieving information with spoken queries, we fill the gap between speech\nrecognition and text retrieval in terms of the vocabulary size. Given a spoken\nquery, we generate a transcription and detect OOV words through speech\nrecognition. We then correspond detected OOV words to terms indexed in a target\ncollection to complete the transcription, and search the collection for\ndocuments relevant to the completed transcription. We show the effectiveness of\nour method by way of experiments.",
            "journal ref": "Proceedings of the 2002 Conference on Empirical Methods in Natural\n  Language Processing (EMNLP-2002), pp.188-195, July. 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206015v1",
            "date": "2002-06-09 10:53:49+00:00",
            "title": "Japanese/English Cross-Language Information Retrieval: Exploration of Query Translation and Transliteration",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "Cross-language information retrieval (CLIR), where queries and documents are\nin different languages, has of late become one of the major topics within the\ninformation retrieval community. This paper proposes a Japanese/English CLIR\nsystem, where we combine a query translation and retrieval modules. We\ncurrently target the retrieval of technical documents, and therefore the\nperformance of our system is highly dependent on the quality of the translation\nof technical terms. However, the technical term translation is still\nproblematic in that technical terms are often compound words, and thus new\nterms are progressively created by combining existing base words. In addition,\nJapanese often represents loanwords based on its special phonogram.\nConsequently, existing dictionaries find it difficult to achieve sufficient\ncoverage. To counter the first problem, we produce a Japanese/English\ndictionary for base words, and translate compound words on a word-by-word\nbasis. We also use a probabilistic method to resolve translation ambiguity. For\nthe second problem, we use a transliteration method, which corresponds words\nunlisted in the base word dictionary to their phonetic equivalents in the\ntarget language. We evaluate our system using a test collection for CLIR, and\nshow that both the compound word translation and transliteration methods\nimprove the system performance.",
            "journal ref": "Computers and the Humanities, Vol.35, No.4, pp.389-420, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206030v1",
            "date": "2002-06-20 07:13:59+00:00",
            "title": "A Probabilistic Method for Analyzing Japanese Anaphora Integrating Zero Pronoun Detection and Resolution",
            "authors": [
                "Kazuhiro Seki",
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "This paper proposes a method to analyze Japanese anaphora, in which zero\npronouns (omitted obligatory cases) are used to refer to preceding entities\n(antecedents). Unlike the case of general coreference resolution, zero pronouns\nhave to be detected prior to resolution because they are not expressed in\ndiscourse. Our method integrates two probability parameters to perform zero\npronoun detection and resolution in a single framework. The first parameter\nquantifies the degree to which a given case is a zero pronoun. The second\nparameter quantifies the degree to which a given entity is the antecedent for a\ndetected zero pronoun. To compute these parameters efficiently, we use corpora\nwith/without annotations of anaphoric relations. We show the effectiveness of\nour method by way of experiments.",
            "journal ref": "Proceedings of the 19th International Conference on Computational\n  Linguistics (COLING 2002), pp.911-917, Aug. 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206034v1",
            "date": "2002-06-24 07:46:06+00:00",
            "title": "Applying a Hybrid Query Translation Method to Japanese/English Cross-Language Patent Retrieval",
            "authors": [
                "Masatoshi Fukui",
                "Shigeto Higuchi",
                "Youichi Nakatani",
                "Masao Tanaka",
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "This paper applies an existing query translation method to cross-language\npatent retrieval. In our method, multiple dictionaries are used to derive all\npossible translations for an input query, and collocational statistics are used\nto resolve translation ambiguity. We used Japanese/English parallel patent\nabstracts to perform comparative experiments, where our method outperformed a\nsimple dictionary-based query translation method, and achieved 76% of\nmonolingual retrieval in terms of average precision.",
            "journal ref": "ACM SIGIR 2000 Workshop on Patent Retrieval, July, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206035v1",
            "date": "2002-06-24 08:00:45+00:00",
            "title": "PRIME: A System for Multi-lingual Patent Retrieval",
            "authors": [
                "Shigeto Higuchi",
                "Masatoshi Fukui",
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "Given the growing number of patents filed in multiple countries, users are\ninterested in retrieving patents across languages. We propose a multi-lingual\npatent retrieval system, which translates a user query into the target\nlanguage, searches a multilingual database for patents relevant to the query,\nand improves the browsing efficiency by way of machine translation and\nclustering. Our system also extracts new translations from patent families\nconsisting of comparable patents, to enhance the translation dictionary.",
            "journal ref": "Proceedings of MT Summit VIII, pp.163-167, Sep. 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206036v1",
            "date": "2002-06-24 08:38:36+00:00",
            "title": "Language Modeling for Multi-Domain Speech-Driven Text Retrieval",
            "authors": [
                "Katunobu Itou",
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "We report experimental results associated with speech-driven text retrieval,\nwhich facilitates retrieving information in multiple domains with spoken\nqueries. Since users speak contents related to a target collection, we produce\nlanguage models used for speech recognition based on the target collection, so\nas to improve both the recognition and retrieval accuracy. Experiments using\nexisting test collections combined with dictated queries showed the\neffectiveness of our method.",
            "journal ref": "IEEE Automatic Speech Recognition and Understanding Workshop, Dec.\n  2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206037v1",
            "date": "2002-06-24 10:28:02+00:00",
            "title": "Speech-Driven Text Retrieval: Using Target IR Collections for Statistical Language Model Adaptation in Speech Recognition",
            "authors": [
                "Atsushi Fujii",
                "Katunobu Itou",
                "Tetsuya Ishikawa"
            ],
            "abstract": "Speech recognition has of late become a practical technology for real world\napplications. Aiming at speech-driven text retrieval, which facilitates\nretrieving information with spoken queries, we propose a method to integrate\nspeech recognition and retrieval methods. Since users speak contents related to\na target collection, we adapt statistical language models used for speech\nrecognition based on the target collection, so as to improve both the\nrecognition and retrieval accuracy. Experiments using existing test collections\ncombined with dictated queries showed the effectiveness of our method.",
            "journal ref": "Anni R. Coden and Eric W. Brown and Savitha Srinivasan (Eds.),\n  Information Retrieval Techniques for Speech Applications (LNCS 2273),\n  pp.94-104, Springer, 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0207002v1",
            "date": "2002-07-02 01:15:33+00:00",
            "title": "Using eigenvectors of the bigram graph to infer morpheme identity",
            "authors": [
                "Mikhail Belkin",
                "John Goldsmith"
            ],
            "abstract": "This paper describes the results of some experiments exploring statistical\nmethods to infer syntactic behavior of words and morphemes from a raw corpus in\nan unsupervised fashion. It shares certain points in common with Brown et al\n(1992) and work that has grown out of that: it employs statistical techniques\nto analyze syntactic behavior based on what words occur adjacent to a given\nword. However, we use an eigenvector decomposition of a nearest-neighbor graph\nto produce a two-dimensional rendering of the words of a corpus in which words\nof the same syntactic category tend to form neighborhoods. We exploit this\ntechnique for extending the value of automatic learning of morphology. In\nparticular, we look at the suffixes derived from a corpus by unsupervised\nlearning of morphology, and we ask which of these suffixes have a consistent\nsyntactic function (e.g., in English, -tion is primarily a mark of nouns, but\n-s marks both noun plurals and 3rd person present on verbs), and we determine\nthat this method works well for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0207003v1",
            "date": "2002-07-02 15:08:44+00:00",
            "title": "Analysis of Titles and Readers For Title Generation Centered on the Readers",
            "authors": [
                "Yasuko Senda",
                "Yasusi Sinohara"
            ],
            "abstract": "The title of a document has two roles, to give a compact summary and to lead\nthe reader to read the document. Conventional title generation focuses on\nfinding key expressions from the author's wording in the document to give a\ncompact summary and pays little attention to the reader's interest. To make the\ntitle play its second role properly, it is indispensable to clarify the content\n(``what to say'') and wording (``how to say'') of titles that are effective to\nattract the target reader's interest. In this article, we first identify\ntypical content and wording of titles aimed at general readers in a comparative\nstudy between titles of technical papers and headlines rewritten for\nnewspapers. Next, we describe the results of a questionnaire survey on the\neffects of the content and wording of titles on the reader's interest. The\nsurvey of general and knowledgeable readers shows both common and different\ntendencies in interest.",
            "journal ref": "COLING'2002(The 19TH International Conference on Computational\n  Linguistics)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0207005v1",
            "date": "2002-07-03 11:54:21+00:00",
            "title": "Efficient Deep Processing of Japanese",
            "authors": [
                "Melanie Siegel",
                "Emily M. Bender"
            ],
            "abstract": "We present a broad coverage Japanese grammar written in the HPSG formalism\nwith MRS semantics. The grammar is created for use in real world applications,\nsuch that robustness and performance issues play an important role. It is\nconnected to a POS tagging and word segmentation tool. This grammar is being\ndeveloped in a multilingual context, requiring MRS structures that are easily\ncomparable across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0208020v1",
            "date": "2002-08-13 03:39:20+00:00",
            "title": "Using the DIFF Command for Natural Language Processing",
            "authors": [
                "Masaki Murata",
                "Hitoshi Isahara"
            ],
            "abstract": "Diff is a software program that detects differences between two data sets and\nis useful in natural language processing. This paper shows several examples of\nthe application of diff. They include the detection of differences between two\ndifferent datasets, extraction of rewriting rules, merging of two different\ndatasets, and the optimal matching of two different data sets. Since diff comes\nwith any standard UNIX system, it is readily available and very easy to use.\nOur studies showed that diff is a practical tool for research into natural\nlanguage processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0208035v1",
            "date": "2002-08-21 14:09:48+00:00",
            "title": "Evaluation of Coreference Rules on Complex Narrative Texts",
            "authors": [
                "Andrei Popescu-Belis",
                "Isabelle Robba"
            ],
            "abstract": "This article studies the problem of assessing relevance to each of the rules\nof a reference resolution system. The reference solver described here stems\nfrom a formal model of reference and is integrated in a reference processing\nworkbench. Evaluation of the reference resolution is essential, as it enables\ndifferential evaluation of individual rules. Numerical values of these measures\nare given, and discussed, for simple selection rules and other processing\nrules; such measures are then studied for numerical parameters.",
            "journal ref": "Proceedings of DAARC2 (Discourse Anaphora and Anaphor Resolution\n  Colloquium), Lancaster, UK, 1998, p.178-185",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0208036v1",
            "date": "2002-08-21 14:28:51+00:00",
            "title": "Three New Methods for Evaluating Reference Resolution",
            "authors": [
                "Andrei Popescu-Belis",
                "Isabelle Robba"
            ],
            "abstract": "Reference resolution on extended texts (several thousand references) cannot\nbe evaluated manually. An evaluation algorithm has been proposed for the MUC\ntests, using equivalence classes for the coreference relation. However, we show\nhere that this algorithm is too indulgent, yielding good scores even for poor\nresolution strategies. We elaborate on the same formalism to propose two new\nevaluation algorithms, comparing them first with the MUC algorithm and giving\nthen results on a variety of examples. A third algorithm using only\ndistributional comparison of equivalence classes is finally described; it\nassesses the relative importance of the recall vs. precision errors.",
            "journal ref": "Proceedings of the LREC'98 Workshop on Linguistic Coreference,\n  Madrid, Spain, 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0208037v1",
            "date": "2002-08-21 14:36:13+00:00",
            "title": "Cooperation between Pronoun and Reference Resolution for Unrestricted Texts",
            "authors": [
                "Andrei Popescu-Belis",
                "Isabelle Robba"
            ],
            "abstract": "Anaphora resolution is envisaged in this paper as part of the reference\nresolution process. A general open architecture is proposed, which can be\nparticularized and configured in order to simulate some classic anaphora\nresolution methods. With the aim of improving pronoun resolution, the system\ntakes advantage of elementary cues about characters of the text, which are\nrepresented through a particular data structure. In its most robust\nconfiguration, the system uses only a general lexicon, a local morpho-syntactic\nparser and a dictionary of synonyms. A short comparative corpus analysis shows\nthat narrative texts are the most suitable for testing such a system.",
            "journal ref": "Proceedings of the ACL'97 Workshop on Operational Factors in\n  Practical, Robust Anaphora Resolution for Unrestricted Texts, Madrid, Spain,\n  1998, p.94-99",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0208038v1",
            "date": "2002-08-21 14:43:18+00:00",
            "title": "Reference Resolution Beyond Coreference: a Conceptual Frame and its Application",
            "authors": [
                "Andrei Popescu-Belis",
                "Isabelle Robba",
                "Gerard Sabah"
            ],
            "abstract": "A model for reference use in communication is proposed, from a\nrepresentationist point of view. Both the sender and the receiver of a message\nhandle representations of their common environment, including mental\nrepresentations of objects. Reference resolution by a computer is viewed as the\nconstruction of object representations using referring expressions from the\ndiscourse, whereas often only coreference links between such expressions are\nlooked for. Differences between these two approaches are discussed. The model\nhas been implemented with elementary rules, and tested on complex narrative\ntexts (hundreds to thousands of referring expressions). The results support the\nmental representations paradigm.",
            "journal ref": "Proceedings of COLING-ACL'98, Montreal, Canada, 1998, p.1046-1052",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0209002v1",
            "date": "2002-09-02 16:55:34+00:00",
            "title": "A Chart-Parsing Algorithm for Efficient Semantic Analysis",
            "authors": [
                "Pascal Vaillant"
            ],
            "abstract": "In some contexts, well-formed natural language cannot be expected as input to\ninformation or communication systems. In these contexts, the use of\ngrammar-independent input (sequences of uninflected semantic units like e.g.\nlanguage-independent icons) can be an answer to the users' needs. A semantic\nanalysis can be performed, based on lexical semantic knowledge: it is\nequivalent to a dependency analysis with no syntactic or morphological clues.\nHowever, this requires that an intelligent system should be able to interpret\nthis input with reasonable accuracy and in reasonable time. Here we propose a\nmethod allowing a purely semantic-based analysis of sequences of semantic\nunits. It uses an algorithm inspired by the idea of ``chart parsing'' known in\nNatural Language Processing, which stores intermediate parsing results in order\nto bring the calculation time down. In comparison with using declarative logic\nprogramming - where the calculation time, left to a prolog engine, is\nhyperexponential -, this method brings the calculation time down to a\npolynomial time, where the order depends on the valency of the predicates.",
            "journal ref": "COLING 2002, Proceedings of the main Conference; The Association\n  for Computational Linguistics and Chinese Language Processing; vol. 2, p.\n  1044-1050",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0209003v1",
            "date": "2002-09-03 05:28:56+00:00",
            "title": "Rerendering Semantic Ontologies: Automatic Extensions to UMLS through Corpus Analytics",
            "authors": [
                "J. Pustejovsky",
                "A. Rumshisky",
                "J. Castano"
            ],
            "abstract": "In this paper, we discuss the utility and deficiencies of existing ontology\nresources for a number of language processing applications. We describe a\ntechnique for increasing the semantic type coverage of a specific ontology, the\nNational Library of Medicine's UMLS, with the use of robust finite state\nmethods used in conjunction with large-scale corpus analytics of the domain\ncorpus. We call this technique \"semantic rerendering\" of the ontology. This\nresearch has been done in the context of Medstract, a joint Brandeis-Tufts\neffort aimed at developing tools for analyzing biomedical language (i.e.,\nMedline), as well as creating targeted databases of bio-entities, biological\nrelations, and pathway data for biological researchers. Motivating the current\nresearch is the need to have robust and reliable semantic typing of syntactic\nelements in the Medline corpus, in order to improve the overall performance of\nthe information extraction applications mentioned above.",
            "journal ref": "LREC 2002 Workshop on Ontologies and Lexical Knowledge Bases",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0209010v1",
            "date": "2002-09-05 19:03:06+00:00",
            "title": "Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition",
            "authors": [
                "Erik F. Tjong Kim Sang"
            ],
            "abstract": "We describe the CoNLL-2002 shared task: language-independent named entity\nrecognition. We give background information on the data sets and the evaluation\nmethod, present a general overview of the systems that have taken part in the\ntask and discuss their performance.",
            "journal ref": "Dan Roth and Antal van den Bosch (eds.), Proceedings of\n  CoNLL-2002, Taipei, Taiwan, 2002, pp. 155-158",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0211017v1",
            "date": "2002-11-14 16:16:44+00:00",
            "title": "Probabilistic Parsing Strategies",
            "authors": [
                "Mark-Jan Nederhof",
                "Giorgio Satta"
            ],
            "abstract": "We present new results on the relation between purely symbolic context-free\nparsing strategies and their probabilistic counter-parts. Such parsing\nstrategies are seen as constructions of push-down devices from grammars. We\nshow that preservation of probability distribution is possible under two\nconditions, viz. the correct-prefix property and the property of strong\npredictiveness. These results generalize existing results in the literature\nthat were obtained by considering parsing strategies in isolation. From our\ngeneral results we also derive negative results on so-called generalized LR\nparsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212015v1",
            "date": "2002-12-09 13:09:10+00:00",
            "title": "Answering Subcognitive Turing Test Questions: A Reply to French",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Robert French has argued that a disembodied computer is incapable of passing\na Turing Test that includes subcognitive questions. Subcognitive questions are\ndesigned to probe the network of cultural and perceptual associations that\nhumans naturally develop as we live, embodied and embedded in the world. In\nthis paper, I show how it is possible for a disembodied computer to answer\nsubcognitive questions appropriately, contrary to French's claim. My approach\nto answering subcognitive questions is to use statistical information extracted\nfrom a very large collection of text. In particular, I show how it is possible\nto answer a sample of subcognitive questions taken from French, by issuing\nqueries to a search engine that indexes about 350 million Web pages. This\nsimple algorithm may shed light on the nature of human (sub-) cognition, but\nthe scope of this paper is limited to demonstrating that French is mistaken: a\ndisembodied computer can answer subcognitive questions.",
            "journal ref": "Journal of Experimental and Theoretical Artificial Intelligence,\n  (2001), 13 (4), 409-419",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0201002v1",
            "date": "2002-01-04 17:01:14+00:00",
            "title": "Incremental Construction of Compact Acyclic NFAs",
            "authors": [
                "Kyriakos N. Sgarbas",
                "Nikos D. Fakotakis",
                "George K. Kokkinakis"
            ],
            "abstract": "This paper presents and analyzes an incremental algorithm for the\nconstruction of Acyclic Non-deterministic Finite-state Automata (NFA). Automata\nof this type are quite useful in computational linguistics, especially for\nstoring lexicons. The proposed algorithm produces compact NFAs, i.e. NFAs that\ndo not contain equivalent states. Unlike Deterministic Finite-state Automata\n(DFA), this property is not sufficient to ensure minimality, but still the\nresulting NFAs are considerably smaller than the minimal DFAs for the same\nlanguages.",
            "journal ref": "Proc. ACL-2001, 39th Annual Meeting of the Association for\n  Computational Linguistics, pp.474-481, Toulouse, France, 6-11 July 2001",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/cs/0201008v1",
            "date": "2002-01-11 17:59:17+00:00",
            "title": "Using Tree Automata and Regular Expressions to Manipulate Hierarchically Structured Data",
            "authors": [
                "Nikita Schmidt",
                "Ahmed Patel"
            ],
            "abstract": "Information, stored or transmitted in digital form, is often structured.\nIndividual data records are usually represented as hierarchies of their\nelements. Together, records form larger structures. Information processing\napplications have to take account of this structuring, which assigns different\nsemantics to different data elements or records. Big variety of structural\nschemata in use today often requires much flexibility from applications--for\nexample, to process information coming from different sources. To ensure\napplication interoperability, translators are needed that can convert one\nstructure into another. This paper puts forward a formal data model aimed at\nsupporting hierarchical data processing in a simple and flexible way. The model\nis based on and extends results of two classical theories, studying finite\nstring and tree automata. The concept of finite automata and regular languages\nis applied to the case of arbitrarily structured tree-like hierarchical data\nrecords, represented as \"structured strings.\" These automata are compared with\nclassical string and tree automata; the model is shown to be a superset of the\nclassical models. Regular grammars and expressions over structured strings are\nintroduced. Regular expression matching and substitution has been widely used\nfor efficient unstructured text processing; the model described here brings the\npower of this proven technique to applications that deal with information\ntrees. A simple generic alternative is offered to replace today's specialised\nad-hoc approaches. The model unifies structural and content transformations,\nproviding applications with a single data type. An example scenario of how to\nbuild applications based on this theory is discussed. Further research\ndirections are outlined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204004v1",
            "date": "2002-04-03 17:25:39+00:00",
            "title": "Models and Tools for Collaborative Annotation",
            "authors": [
                "Xiaoyi Ma",
                "Haejoong Lee",
                "Steven Bird",
                "Kazuaki Maeda"
            ],
            "abstract": "The Annotation Graph Toolkit (AGTK) is a collection of software which\nfacilitates development of linguistic annotation tools. AGTK provides a\ndatabase interface which allows applications to use a database server for\npersistent storage. This paper discusses various modes of collaborative\nannotation and how they can be supported with tools built using AGTK and its\ndatabase interface. We describe the relational database schema and API, and\ndescribe a version of the TableTrans tool which supports collaborative\nannotation. The remainder of the paper discusses a high-level query language\nfor annotation graphs, along with optimizations, in support of expressive and\nefficient access to the annotations held on a large central server. The paper\ndemonstrates that it is straightforward to support a variety of different\nlevels of collaborative annotation with existing AGTK-based tools, with a\nminimum of additional programming effort.",
            "journal ref": "Proceedings of the Third International Conference on Language\n  Resources and Evaluation, Paris: European Language Resources Association,\n  2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204005v1",
            "date": "2002-04-03 17:32:53+00:00",
            "title": "Creating Annotation Tools with the Annotation Graph Toolkit",
            "authors": [
                "Kazuaki Maeda",
                "Steven Bird",
                "Xiaoyi Ma",
                "Haejoong Lee"
            ],
            "abstract": "The Annotation Graph Toolkit is a collection of software supporting the\ndevelopment of annotation tools based on the annotation graph model. The\ntoolkit includes application programming interfaces for manipulating annotation\ngraph data and for importing data from other formats. There are interfaces for\nthe scripting languages Tcl and Python, a database interface, specialized\ngraphical user interfaces for a variety of annotation tasks, and several sample\napplications. This paper describes all the toolkit components for the benefit\nof would-be application developers.",
            "journal ref": "Proceedings of the Third International Conference on Language\n  Resources and Evaluation, Paris: European Language Resources Association,\n  2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204006v1",
            "date": "2002-04-03 17:18:49+00:00",
            "title": "TableTrans, MultiTrans, InterTrans and TreeTrans: Diverse Tools Built on the Annotation Graph Toolkit",
            "authors": [
                "Steven Bird",
                "Kazuaki Maeda",
                "Xiaoyi Ma",
                "Haejoong Lee",
                "Beth Randall",
                "Salim Zayat"
            ],
            "abstract": "Four diverse tools built on the Annotation Graph Toolkit are described. Each\ntool associates linguistic codes and structures with time-series data. All are\nbased on the same software library and tool architecture. TableTrans is for\nobservational coding, using a spreadsheet whose rows are aligned to a signal.\nMultiTrans is for transcribing multi-party communicative interactions recorded\nusing multi-channel signals. InterTrans is for creating interlinear text\naligned to audio. TreeTrans is for creating and manipulating syntactic trees.\nThis work demonstrates that the development of diverse tools and re-use of\nsoftware components is greatly facilitated by a common high-level application\nprogramming interface for representing the data and managing input/output,\ntogether with a common architecture for managing the interaction of multiple\ncomponents.",
            "journal ref": "Proceedings of the Third International Conference on Language\n  Resources and Evaluation, Paris: European Language Resources Association,\n  2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204020v1",
            "date": "2002-04-10 13:52:19+00:00",
            "title": "Seven Dimensions of Portability for Language Documentation and Description",
            "authors": [
                "Steven Bird",
                "Gary Simons"
            ],
            "abstract": "The process of documenting and describing the world's languages is undergoing\nradical transformation with the rapid uptake of new digital technologies for\ncapture, storage, annotation and dissemination. However, uncritical adoption of\nnew tools and technologies is leading to resources that are difficult to reuse\nand which are less portable than the conventional printed resources they\nreplace. We begin by reviewing current uses of software tools and digital\ntechnologies for language documentation and description. This sheds light on\nhow digital language documentation and description are created and managed,\nleading to an analysis of seven portability problems under the following\nheadings: content, format, discovery, access, citation, preservation and\nrights. After characterizing each problem we provide a series of value\nstatements, and this provides the framework for a broad range of best practice\nrecommendations.",
            "journal ref": "Proceedings of the Workshop on Portability Issues in Human\n  Language Technologies, Third International Conference on Language Resources\n  and Evaluation, Paris: European Language Resources Association, 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204026v1",
            "date": "2002-04-11 16:43:57+00:00",
            "title": "Querying Databases of Annotated Speech",
            "authors": [
                "Steve Cassidy",
                "Steven Bird"
            ],
            "abstract": "Annotated speech corpora are databases consisting of signal data along with\ntime-aligned symbolic `transcriptions'. Such databases are typically\nmultidimensional, heterogeneous and dynamic. These properties present a number\nof tough challenges for representation and query. The temporal nature of the\ndata adds an additional layer of complexity. This paper presents and harmonises\ntwo independent efforts to model annotated speech databases, one at Macquarie\nUniversity and one at the University of Pennsylvania. Various query languages\nare described, along with illustrative applications to a variety of analytical\nproblems. The research reported here forms a part of several ongoing projects\nto develop platform-independent open-source tools for creating, browsing,\nsearching, querying and transforming linguistic databases, and to disseminate\nlarge linguistic databases over the internet.",
            "journal ref": "Database Technologies: Proceedings of the Eleventh Australasian\n  Database Conference, pp. 12-20, IEEE Computer Society, 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205025v1",
            "date": "2002-05-16 12:35:00+00:00",
            "title": "Bootstrapping Structure into Language: Alignment-Based Learning",
            "authors": [
                "Menno M. van Zaanen"
            ],
            "abstract": "This thesis introduces a new unsupervised learning framework, called\nAlignment-Based Learning, which is based on the alignment of sentences and\nHarris's (1951) notion of substitutability. Instances of the framework can be\napplied to an untagged, unstructured corpus of natural language sentences,\nresulting in a labelled, bracketed version of that corpus.\n  Firstly, the framework aligns all sentences in the corpus in pairs, resulting\nin a partition of the sentences consisting of parts of the sentences that are\nequal in both sentences and parts that are unequal. Unequal parts of sentences\ncan be seen as being substitutable for each other, since substituting one\nunequal part for the other results in another valid sentence. The unequal parts\nof the sentences are thus considered to be possible (possibly overlapping)\nconstituents, called hypotheses.\n  Secondly, the selection learning phase considers all hypotheses found by the\nalignment learning phase and selects the best of these. The hypotheses are\nselected based on the order in which they were found, or based on a\nprobabilistic function.\n  The framework can be extended with a grammar extraction phase. This extended\nframework is called parseABL. Instead of returning a structured version of the\nunstructured input corpus, like the ABL system, this system also returns a\nstochastic context-free or tree substitution grammar.\n  Different instances of the framework have been tested on the English ATIS\ncorpus, the Dutch OVIS corpus and the Wall Street Journal corpus. One of the\ninteresting results, apart from the encouraging numerical results, is that all\ninstances can (and do) learn recursive structures.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205026v1",
            "date": "2002-05-17 08:24:56+00:00",
            "title": "Monads for natural language semantics",
            "authors": [
                "Chung-chieh Shan"
            ],
            "abstract": "Accounts of semantic phenomena often involve extending types of meanings and\nrevising composition rules at the same time. The concept of monads allows many\nsuch accounts -- for intensionality, variable binding, quantification and focus\n-- to be stated uniformly and compositionally.",
            "journal ref": "Proceedings of the 2001 European Summer School in Logic, Language\n  and Information student session, ed. Kristina Striegnitz, 285-298",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205070v1",
            "date": "2002-05-28 02:01:55+00:00",
            "title": "Thumbs up? Sentiment Classification using Machine Learning Techniques",
            "authors": [
                "Bo Pang",
                "Lillian Lee",
                "Shivakumar Vaithyanathan"
            ],
            "abstract": "We consider the problem of classifying documents not by topic, but by overall\nsentiment, e.g., determining whether a review is positive or negative. Using\nmovie reviews as data, we find that standard machine learning techniques\ndefinitively outperform human-produced baselines. However, the three machine\nlearning methods we employed (Naive Bayes, maximum entropy classification, and\nsupport vector machines) do not perform as well on sentiment classification as\non traditional topic-based categorization. We conclude by examining factors\nthat make the sentiment classification problem more challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0205072v1",
            "date": "2002-05-29 17:48:48+00:00",
            "title": "Unsupervised Learning of Morphology without Morphemes",
            "authors": [
                "Sylvain Neuvel",
                "Sean A. Fulop"
            ],
            "abstract": "The first morphological learner based upon the theory of Whole Word\nMorphology Ford et al. (1997) is outlined, and preliminary evaluation results\nare presented. The program, Whole Word Morphologizer, takes a POS-tagged\nlexicon as input, induces morphological relationships without attempting to\ndiscover or identify morphemes, and is then able to generate new words beyond\nthe learning sample. The accuracy (precision) of the generated new words is as\nhigh as 80% using the pure Whole Word theory, and 92% after a post-hoc\nadjustment is added to the routine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206007v1",
            "date": "2002-06-04 14:48:48+00:00",
            "title": "Using the Annotated Bibliography as a Resource for Indicative Summarization",
            "authors": [
                "Min-Yen Kan",
                "Judith L. Klavans",
                "Kathleen R. McKeown"
            ],
            "abstract": "We report on a language resource consisting of 2000 annotated bibliography\nentries, which is being analyzed as part of our research on indicative document\nsummarization. We show how annotated bibliographies cover certain aspects of\nsummarization that have not been well-covered by other summary corpora, and\nmotivate why they constitute an important form to study for information\nretrieval. We detail our methodology for collecting the corpus, and overview\nour document feature markup that we introduced to facilitate summary analysis.\nWe present the characteristics of the corpus, methods of collection, and show\nits use in finding the distribution of types of information included in\nindicative summaries and their relative ordering within the summaries.",
            "journal ref": "Proceedings of LREC 2002, Las Palmas, Spain. pp. 1746-1752",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0206026v1",
            "date": "2002-06-18 15:11:52+00:00",
            "title": "Interleaved semantic interpretation in environment-based parsing",
            "authors": [
                "William Schuler"
            ],
            "abstract": "This paper extends a polynomial-time parsing algorithm that resolves\nstructural ambiguity in input to a speech-based user interface by calculating\nand comparing the denotations of rival constituents, given some model of the\ninterfaced application environment (Schuler 2001). The algorithm is extended to\nincorporate a full set of logical operators, including quantifiers and\nconjunctions, into this calculation without increasing the complexity of the\noverall algorithm beyond polynomial time, both in terms of the length of the\ninput and the number of entities in the environment model.",
            "journal ref": "Proceedings of the 19th International Conference on Computational\n  Linguistics (COLING 2002)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0207058v2",
            "date": "2002-07-14 21:04:43+00:00",
            "title": "Question Answering over Unstructured Data without Domain Restrictions",
            "authors": [
                "Jochen L. Leidner"
            ],
            "abstract": "Information needs are naturally represented as questions. Automatic\nNatural-Language Question Answering (NLQA) has only recently become a practical\ntask on a larger scale and without domain constraints.\n  This paper gives a brief introduction to the field, its history and the\nimpact of systematic evaluation competitions.\n  It is then demonstrated that an NLQA system for English can be built and\nevaluated in a very short time using off-the-shelf parsers and thesauri. The\nsystem is based on Robust Minimal Recursion Semantics (RMRS) and is portable\nwith respect to the parser used as a frontend. It applies atomic term\nunification supported by question classification and WordNet lookup for\nsemantic similarity matching of parsed question representation and free text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0207070v2",
            "date": "2002-07-18 23:27:01+00:00",
            "title": "A continuation semantics of interrogatives that accounts for Baker's ambiguity",
            "authors": [
                "Chung-chieh Shan"
            ],
            "abstract": "Wh-phrases in English can appear both raised and in-situ. However, only\nin-situ wh-phrases can take semantic scope beyond the immediately enclosing\nclause. I present a denotational semantics of interrogatives that naturally\naccounts for these two properties. It neither invokes movement or economy, nor\nposits lexical ambiguity between raised and in-situ occurrences of the same\nwh-phrase. My analysis is based on the concept of continuations. It uses a\nnovel type system for higher-order continuations to handle wide-scope\nwh-phrases while remaining strictly compositional. This treatment sheds light\non the combinatorics of interrogatives as well as other kinds of so-called\nA'-movement.",
            "journal ref": "Proceedings of SALT XII: Semantics and Linguistic Theory, ed.\n  Brendan Jackson, 246-265 (2002)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0210025v3",
            "date": "2002-10-29 00:33:26+00:00",
            "title": "An Algorithm for Pattern Discovery in Time Series",
            "authors": [
                "Cosma Rohilla Shalizi",
                "Kristina Lisa Shalizi",
                "James P. Crutchfield"
            ],
            "abstract": "We present a new algorithm for discovering patterns in time series and other\nsequential data. We exhibit a reliable procedure for building the minimal set\nof hidden, Markovian states that is statistically capable of producing the\nbehavior exhibited in the data -- the underlying process's causal states.\nUnlike conventional methods for fitting hidden Markov models (HMMs) to data,\nour algorithm makes no assumptions about the process's causal architecture (the\nnumber of hidden states and their transition structure), but rather infers it\nfrom the data. It starts with assumptions of minimal structure and introduces\ncomplexity only when the data demand it. Moreover, the causal states it infers\nhave important predictive optimality properties that conventional HMM states\nlack. We introduce the algorithm, review the theory behind it, prove its\nasymptotic reliability, use large deviation theory to estimate its rate of\nconvergence, and compare it to other algorithms which also construct HMMs from\ndata. We also illustrate its behavior on an example process, and report\nselected numerical results from an implementation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212018v1",
            "date": "2002-12-10 12:46:25+00:00",
            "title": "Real numbers having ultimately periodic representations in abstract numeration systems",
            "authors": [
                "P. Lecomte",
                "M. Rigo"
            ],
            "abstract": "Using a genealogically ordered infinite regular language, we know how to\nrepresent an interval of R. Numbers having an ultimately periodic\nrepresentation play a special role in classical numeration systems. The aim of\nthis paper is to characterize the numbers having an ultimately periodic\nrepresentation in generalized systems built on a regular language. The\nsyntactical properties of these words are also investigated. Finally, we show\nthe equivalence of the classical \"theta\"-expansions with our generalized\nrepresentations in some special case related to a Pisot number \"theta\".",
            "journal ref": "None",
            "category": "cs.CC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212024v1",
            "date": "2002-12-10 21:59:15+00:00",
            "title": "Unsupervised Language Acquisition: Theory and Practice",
            "authors": [
                "Alexander Clark"
            ],
            "abstract": "In this thesis I present various algorithms for the unsupervised machine\nlearning of aspects of natural languages using a variety of statistical models.\nThe scientific object of the work is to examine the validity of the so-called\nArgument from the Poverty of the Stimulus advanced in favour of the proposition\nthat humans have language-specific innate knowledge. I start by examining an a\npriori argument based on Gold's theorem, that purports to prove that natural\nlanguages cannot be learned, and some formal issues related to the choice of\nstatistical grammars rather than symbolic grammars. I present three novel\nalgorithms for learning various parts of natural languages: first, an algorithm\nfor the induction of syntactic categories from unlabelled text using\ndistributional information, that can deal with ambiguous and rare words;\nsecondly, a set of algorithms for learning morphological processes in a variety\nof languages, including languages such as Arabic with non-concatenative\nmorphology; thirdly an algorithm for the unsupervised induction of a\ncontext-free grammar from tagged text. I carefully examine the interaction\nbetween the various components, and show how these algorithms can form the\nbasis for a empiricist model of language acquisition. I therefore conclude that\nthe Argument from the Poverty of the Stimulus is unsupported by the evidence.",
            "journal ref": "D. Phil., School of Cognitive and Computing Sciences, University\n  of Sussex, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0201139v1",
            "date": "2002-01-09 18:55:22+00:00",
            "title": "Long-range fractal correlations in literary corpora",
            "authors": [
                "Marcelo A. Montemurro",
                "Pedro A. Pury"
            ],
            "abstract": "In this paper we analyse the fractal structure of long human-language records\nby mapping large samples of texts onto time series. The particular mapping set\nup in this work is inspired on linguistic basis in the sense that is retains\n{\\em the word} as the fundamental unit of communication. The results confirm\nthat beyond the short-range correlations resulting from syntactic rules acting\nat sentence level, long-range structures emerge in large written language\nsamples that give rise to long-range correlations in the use of words.",
            "journal ref": "Fractals 10(4), 451-461 (2002)",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0202383v1",
            "date": "2002-02-21 18:25:29+00:00",
            "title": "Extended Comment on Language Trees and Zipping",
            "authors": [
                "Joshua Goodman"
            ],
            "abstract": "This is the extended version of a Comment submitted to Physical Review\nLetters. I first point out the inappropriateness of publishing a Letter\nunrelated to physics. Next, I give experimental results showing that the\ntechnique used in the Letter is 3 times worse and 17 times slower than a simple\nbaseline. And finally, I review the literature, showing that the ideas of the\nLetter are not novel. I conclude by suggesting that Physical Review Letters\nshould not publish Letters unrelated to physics.",
            "journal ref": "None",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/cs/0204008v1",
            "date": "2002-04-04 18:59:49+00:00",
            "title": "The tip-of-the-tongue phenomenon: Irrelevant neural network localization or disruption of its interneuron links ?",
            "authors": [
                "Petro M. Gopych"
            ],
            "abstract": "On the base of recently proposed three-stage quantitative neural network\nmodel of the tip-of-the-tongue (TOT) phenomenon a possibility to occur of TOT\nstates coursed by neural network interneuron links' disruption has been\nstudied. Using a numerical example it was found that TOTs coursed by interneron\nlinks' disruption are in (1.5 + - 0.3)x1000 times less probable then those\ncoursed by irrelevant (incomplete) neural network localization. It was shown\nthat delayed TOT states' etiology cannot be related to neural network\ninterneuron links' disruption.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0209008v1",
            "date": "2002-09-04 20:11:14+00:00",
            "title": "The partition semantics of questions, syntactically",
            "authors": [
                "Chung-chieh Shan",
                "Balder D. ten Cate"
            ],
            "abstract": "Groenendijk and Stokhof (1984, 1996; Groenendijk 1999) provide a logically\nattractive theory of the semantics of natural language questions, commonly\nreferred to as the partition theory. Two central notions in this theory are\nentailment between questions and answerhood. For example, the question \"Who is\ngoing to the party?\" entails the question \"Is John going to the party?\", and\n\"John is going to the party\" counts as an answer to both. Groenendijk and\nStokhof define these two notions in terms of partitions of a set of possible\nworlds.\n  We provide a syntactic characterization of entailment between questions and\nanswerhood . We show that answers are, in some sense, exactly those formulas\nthat are built up from instances of the question. This result lets us compare\nthe partition theory with other approaches to interrogation -- both linguistic\nanalyses, such as Hamblin's and Karttunen's semantics, and computational\nsystems, such as Prolog. Our comparison separates a notion of answerhood into\nthree aspects: equivalence (when two questions or answers are interchangeable),\natomic answers (what instances of a question count as answers), and compound\nanswers (how answers compose).",
            "journal ref": "Proceedings of the 2002 European Summer School in Logic, Language\n  and Information student session, ed. Malvina Nissim",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0209009v1",
            "date": "2002-09-04 20:46:59+00:00",
            "title": "Question answering: from partitions to Prolog",
            "authors": [
                "Balder D. ten Cate",
                "Chung-chieh Shan"
            ],
            "abstract": "We implement Groenendijk and Stokhof's partition semantics of questions in a\nsimple question answering algorithm. The algorithm is sound, complete, and\nbased on tableau theorem proving. The algorithm relies on a syntactic\ncharacterization of answerhood: Any answer to a question is equivalent to some\nformula built up only from instances of the question. We prove this\ncharacterization by translating the logic of interrogation to classical\npredicate logic and applying Craig's interpolation theorem.",
            "journal ref": "Proceedings of TABLEAUX 2002: Automated Reasoning with Analytic\n  Tableaux and Related Methods, ed. Uwe Egly and Christian G. Fermueller,\n  Lecture Notes in Artificial Intelligence 2381, 251-265; also in Proceedings\n  of NLULP 2002, ed. Shuly Wintner",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212020v1",
            "date": "2002-12-10 15:30:56+00:00",
            "title": "Learning Algorithms for Keyphrase Extraction",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Many academic journals ask their authors to provide a list of about five to\nfifteen keywords, to appear on the first page of each article. Since these key\nwords are often phrases of two or more words, we prefer to call them\nkeyphrases. There is a wide variety of tasks for which keyphrases are useful,\nas we discuss in this paper. We approach the problem of automatically\nextracting keyphrases from text as a supervised learning task. We treat a\ndocument as a set of phrases, which the learning algorithm must learn to\nclassify as positive or negative examples of keyphrases. Our first set of\nexperiments applies the C4.5 decision tree induction algorithm to this learning\ntask. We evaluate the performance of nine different configurations of C4.5. The\nsecond set of experiments applies the GenEx algorithm to the task. We developed\nthe GenEx algorithm specifically for automatically extracting keyphrases from\ntext. The experimental results support the claim that a custom-designed\nalgorithm (GenEx), incorporating specialized procedural domain knowledge, can\ngenerate better keyphrases than a generalpurpose algorithm (C4.5). Subjective\nhuman evaluation of the keyphrases generated by Extractor suggests that about\n80% of the keyphrases are acceptable to human readers. This level of\nperformance should be satisfactory for a wide variety of applications.",
            "journal ref": "Information Retrieval, (2000), 2 (4), 303-336",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212032v1",
            "date": "2002-12-11 18:57:42+00:00",
            "title": "Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "This paper presents a simple unsupervised learning algorithm for classifying\nreviews as recommended (thumbs up) or not recommended (thumbs down). The\nclassification of a review is predicted by the average semantic orientation of\nthe phrases in the review that contain adjectives or adverbs. A phrase has a\npositive semantic orientation when it has good associations (e.g., \"subtle\nnuances\") and a negative semantic orientation when it has bad associations\n(e.g., \"very cavalier\"). In this paper, the semantic orientation of a phrase is\ncalculated as the mutual information between the given phrase and the word\n\"excellent\" minus the mutual information between the given phrase and the word\n\"poor\". A review is classified as recommended if the average semantic\norientation of its phrases is positive. The algorithm achieves an average\naccuracy of 74% when evaluated on 410 reviews from Epinions, sampled from four\ndifferent domains (reviews of automobiles, banks, movies, and travel\ndestinations). The accuracy ranges from 84% for automobile reviews to 66% for\nmovie reviews.",
            "journal ref": "Proceedings of the 40th Annual Meeting of the Association for\n  Computational Linguistics, (2002), Philadelphia, Pennsylvania, 417-424",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0212033v1",
            "date": "2002-12-11 19:17:06+00:00",
            "title": "Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "This paper presents a simple unsupervised learning algorithm for recognizing\nsynonyms, based on statistical data acquired by querying a Web search engine.\nThe algorithm, called PMI-IR, uses Pointwise Mutual Information (PMI) and\nInformation Retrieval (IR) to measure the similarity of pairs of words. PMI-IR\nis empirically evaluated using 80 synonym test questions from the Test of\nEnglish as a Foreign Language (TOEFL) and 50 synonym test questions from a\ncollection of tests for students of English as a Second Language (ESL). On both\ntests, the algorithm obtains a score of 74%. PMI-IR is contrasted with Latent\nSemantic Analysis (LSA), which achieves a score of 64% on the same 80 TOEFL\nquestions. The paper discusses potential applications of the new unsupervised\nlearning algorithm and some implications of the results for LSA and LSI (Latent\nSemantic Indexing).",
            "journal ref": "Proceedings of the Twelfth European Conference on Machine\n  Learning, (2001), Freiburg, Germany, 491-502",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0203436v1",
            "date": "2002-03-21 11:24:56+00:00",
            "title": "Entropy estimation of symbol sequences",
            "authors": [
                "Thomas Sch\u00fcrmann",
                "Peter Grassberger"
            ],
            "abstract": "We discuss algorithms for estimating the Shannon entropy h of finite symbol\nsequences with long range correlations. In particular, we consider algorithms\nwhich estimate h from the code lengths produced by some compression algorithm.\nOur interest is in describing their convergence with sequence length, assuming\nno limits for the space and time complexities of the compression algorithms. A\nscaling law is proposed for extrapolation from finite sample lengths. This is\napplied to sequences of dynamical systems in non-trivial chaotic regimes, a 1-D\ncellular automaton, and to written English texts.",
            "journal ref": "CHAOS Vol. 6, No. 3 (1996) 414-427",
            "category": "cond-mat.stat-mech"
        }
    ],
    "2003": [
        {
            "url": "http://arxiv.org/abs/cs/0302014v1",
            "date": "2003-02-12 06:31:54+00:00",
            "title": "An Algorithm for Aligning Sentences in Bilingual Corpora Using Lexical Information",
            "authors": [
                "Akshar Bharati",
                "V. Sriram",
                "A. Vamshi Krishna",
                "Rajeev Sangal",
                "S. M. Bendre"
            ],
            "abstract": "In this paper we describe an algorithm for aligning sentences with their\ntranslations in a bilingual corpus using lexical information of the languages.\nExisting efficient algorithms ignore word identities and consider only the\nsentence lengths (Brown, 1991; Gale and Church, 1993). For a sentence in the\nsource language text, the proposed algorithm picks the most likely translation\nfrom the target language text using lexical information and certain heuristics.\nIt does not do statistical analysis using sentence lengths. The algorithm is\nlanguage independent. It also aids in detecting addition and deletion of text\nin translations. The algorithm gives comparable results with the existing\nalgorithms in most of the cases while it does better in cases where statistical\nalgorithms do not give good results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0302032v1",
            "date": "2003-02-22 23:37:26+00:00",
            "title": "Empirical Methods for Compound Splitting",
            "authors": [
                "Philipp Koehn",
                "Kevin Knight"
            ],
            "abstract": "Compounded words are a challenge for NLP applications such as machine\ntranslation (MT). We introduce methods to learn splitting rules from\nmonolingual and parallel corpora. We evaluate them against a gold standard and\nmeasure their impact on performance of statistical MT systems. Results show\naccuracy of 99.1% and performance gains for MT of 0.039 BLEU on a\nGerman-English noun phrase translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0303002v2",
            "date": "2003-03-05 08:50:42+00:00",
            "title": "About compression of vocabulary in computer oriented languages",
            "authors": [
                "V. P. Maslov"
            ],
            "abstract": "The author uses the entropy of the ideal Bose-Einstein gas to minimize losses\nin computer-oriented languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0303007v1",
            "date": "2003-03-14 05:15:20+00:00",
            "title": "Glottochronology and problems of protolanguage reconstruction",
            "authors": [
                "Kromer Victor"
            ],
            "abstract": "A method of languages genealogical trees construction is proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304006v1",
            "date": "2003-04-02 23:02:44+00:00",
            "title": "Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment",
            "authors": [
                "Regina Barzilay",
                "Lillian Lee"
            ],
            "abstract": "We address the text-to-text generation problem of sentence-level paraphrasing\n-- a phenomenon distinct from and more difficult than word- or phrase-level\nparaphrasing. Our approach applies multiple-sequence alignment to sentences\ngathered from unannotated comparable corpora: it learns a set of paraphrasing\npatterns represented by word lattice pairs and automatically determines how to\napply these patterns to rewrite new sentences. The results of our evaluation\nexperiments show that the system derives accurate paraphrases, outperforming\nbaseline systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304019v1",
            "date": "2003-04-10 22:25:01+00:00",
            "title": "Blind Normalization of Speech From Different Channels",
            "authors": [
                "David N. Levin"
            ],
            "abstract": "We show how to construct a channel-independent representation of speech that\nhas propagated through a noisy reverberant channel. This is done by blindly\nrescaling the cepstral time series by a non-linear function, with the form of\nthis scale function being determined by previously encountered cepstra from\nthat channel. The rescaled form of the time series is an invariant property of\nit in the following sense: it is unaffected if the time series is transformed\nby any time-independent invertible distortion. Because a linear channel with\nstationary noise and impulse response transforms cepstra in this way, the new\ntechnique can be used to remove the channel dependence of a cepstral time\nseries. In experiments, the method achieved greater channel-independence than\ncepstral mean normalization, and it was comparable to the combination of\ncepstral mean normalization and spectral subtraction, despite the fact that no\nmeasurements of channel noise or reverberations were required (unlike spectral\nsubtraction).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304024v1",
            "date": "2003-04-17 02:22:06+00:00",
            "title": "Glottochronologic Retrognostic of Language System",
            "authors": [
                "Kromer Victor"
            ],
            "abstract": "A glottochronologic retrognostic of language system is proposed",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304027v1",
            "date": "2003-04-21 22:10:21+00:00",
            "title": "\"I'm sorry Dave, I'm afraid I can't do that\": Linguistics, Statistics, and Natural Language Processing circa 2001",
            "authors": [
                "Lillian Lee"
            ],
            "abstract": "A brief, general-audience overview of the history of natural language\nprocessing, focusing on data-driven approaches.Topics include \"Ambiguity and\nlanguage analysis\", \"Firth things first\", \"A 'C' change\", and \"The empiricists\nstrike back\".",
            "journal ref": "In \"Computer Science: Reflections on the Field, Reflections from\n  the Field\" (report of the National Academies' Study on the Fundamentals of\n  Computer Science), pp. 111--118, 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304029v1",
            "date": "2003-04-22 13:45:37+00:00",
            "title": "An XML based Document Suite",
            "authors": [
                "Dietmar Roesner",
                "Manuela Kunze"
            ],
            "abstract": "We report about the current state of development of a document suite and its\napplications. This collection of tools for the flexible and robust processing\nof documents in German is based on the use of XML as unifying formalism for\nencoding input and output data as well as process information. It is organized\nin modules with limited responsibilities that can easily be combined into\npipelines to solve complex tasks. Strong emphasis is laid on a number of\ntechniques to deal with lexical and conceptual gaps that are typical when\nstarting a new application.",
            "journal ref": "Proceedings of COLING 2002; p. 1278-1282",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304035v1",
            "date": "2003-04-23 08:02:53+00:00",
            "title": "Exploiting Sublanguage and Domain Characteristics in a Bootstrapping Approach to Lexicon and Ontology Creation",
            "authors": [
                "Dietmar Roesner",
                "Manuela Kunze"
            ],
            "abstract": "It is very costly to build up lexical resources and domain ontologies.\nEspecially when confronted with a new application domain lexical gaps and a\npoor coverage of domain concepts are a problem for the successful exploitation\nof natural language document analysis systems that need and exploit such\nknowledge sources. In this paper we report about ongoing experiments with\n`bootstrapping techniques' for lexicon and ontology creation.",
            "journal ref": "Workshop-Proceedings of the OntoLex 2002 - Ontologies and Lexical\n  Knowledge Bases at the LREC 2002, p. 68-73",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304036v1",
            "date": "2003-04-23 08:32:19+00:00",
            "title": "An Approach for Resource Sharing in Multilingual NLP",
            "authors": [
                "Manuela Kunze",
                "Chun Xiao"
            ],
            "abstract": "In this paper we describe an approach for the analysis of documents in German\nand English with a shared pool of resources. For the analysis of German\ndocuments we use a document suite, which supports the user in tasks like\ninformation retrieval and information extraction. The core of the document\nsuite is based on our tool XDOC. Now we want to exploit these methods for the\nanalysis of English documents as well. For this aim we need a multilingual\npresentation format of the resources. These resources must be transformed into\nan unified format, in which we can set additional information about linguistic\ncharacteristics of the language depending on the analyzed documents. In this\npaper we describe our approach for such an exchange model for multilingual\nresources based on XML.",
            "journal ref": "STAIRS 2002 - STarting Artificial Intelligence Researchers\n  Symposium at the ECAI 2002. Lyon, France. ISBN 158603 259 3. IOS Press\n  Amsterdam, p. 123-124",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0305041v2",
            "date": "2003-05-23 19:38:22+00:00",
            "title": "Factorization of Language Models through Backing-Off Lattices",
            "authors": [
                "Wei Wang"
            ],
            "abstract": "Factorization of statistical language models is the task that we resolve the\nmost discriminative model into factored models and determine a new model by\ncombining them so as to provide better estimate. Most of previous works mainly\nfocus on factorizing models of sequential events, each of which allows only one\nfactorization manner. To enable parallel factorization, which allows a model\nevent to be resolved in more than one ways at the same time, we propose a\ngeneral framework, where we adopt a backing-off lattice to reflect parallel\nfactorizations and to define the paths along which a model is resolved into\nfactored models, we use a mixture model to combine parallel paths in the\nlattice, and generalize Katz's backing-off method to integrate all the mixture\nmodels got by traversing the entire lattice. Based on this framework, we\nformulate two types of model factorizations that are used in natural language\nmodeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306050v1",
            "date": "2003-06-12 12:35:00+00:00",
            "title": "Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition",
            "authors": [
                "Erik F. Tjong Kim Sang",
                "Fien De Meulder"
            ],
            "abstract": "We describe the CoNLL-2003 shared task: language-independent named entity\nrecognition. We give background information on the data sets (English and\nGerman) and the evaluation method, present a general overview of the systems\nthat have taken part in the task and discuss their performance.",
            "journal ref": "Proceedings of CoNLL-2003, Edmonton, Canada, 2003, pp. 142-147",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306062v1",
            "date": "2003-06-13 09:05:10+00:00",
            "title": "Learning to Order Facts for Discourse Planning in Natural Language Generation",
            "authors": [
                "Aggeliki Dimitromanolaki",
                "Ion Androutsopoulos"
            ],
            "abstract": "This paper presents a machine learning approach to discourse planning in\nnatural language generation. More specifically, we address the problem of\nlearning the most natural ordering of facts in discourse plans for a specific\ndomain. We discuss our methodology and how it was instantiated using two\ndifferent machine learning algorithms. A quantitative evaluation performed in\nthe domain of museum exhibit descriptions indicates that our approach performs\nsignificantly better than manually constructed ordering rules. Being\nretrainable, the resulting planners can be ported easily to other similar\ndomains, without requiring language technology expertise.",
            "journal ref": "Proceedings of EACL 2003 Workshop on Natural Language Generation",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306099v1",
            "date": "2003-06-16 13:54:03+00:00",
            "title": "An Improved k-Nearest Neighbor Algorithm for Text Categorization",
            "authors": [
                "Baoli Li",
                "Shiwen Yu",
                "Qin Lu"
            ],
            "abstract": "k is the most important parameter in a text categorization system based on\nk-Nearest Neighbor algorithm (kNN).In the classification process, k nearest\ndocuments to the test one in the training set are determined firstly. Then, the\npredication can be made according to the category distribution among these k\nnearest neighbors. Generally speaking, the class distribution in the training\nset is uneven. Some classes may have more samples than others. Therefore, the\nsystem performance is very sensitive to the choice of the parameter k. And it\nis very likely that a fixed k value will result in a bias on large categories.\nTo deal with these problems, we propose an improved kNN algorithm, which uses\ndifferent numbers of nearest neighbors for different categories, rather than a\nfixed number across all categories. More samples (nearest neighbors) will be\nused for deciding whether a test document should be classified to a category,\nwhich has more samples in the training set. Preliminary experiments on Chinese\ntext categorization show that our method is less sensitive to the parameter k\nthan the traditional one, and it can properly classify documents belonging to\nsmaller classes with a large k. The method is promising for some cases, where\nestimating the parameter k via cross-validation is not allowed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0307028v1",
            "date": "2003-07-11 14:43:51+00:00",
            "title": "Issues in Communication Game",
            "authors": [
                "Koiti Hasida"
            ],
            "abstract": "As interaction between autonomous agents, communication can be analyzed in\ngame-theoretic terms. Meaning game is proposed to formalize the core of\nintended communication in which the sender sends a message and the receiver\nattempts to infer its meaning intended by the sender. Basic issues involved in\nthe game of natural language communication are discussed, such as salience,\ngrammaticality, common sense, and common belief, together with some\ndemonstration of the feasibility of game-theoretic account of language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0307030v1",
            "date": "2003-07-11 15:42:51+00:00",
            "title": "Parsing and Generation with Tabulation and Compilation",
            "authors": [
                "Koiti Hasida",
                "Takashi Miyata"
            ],
            "abstract": "The standard tabulation techniques for logic programming presuppose fixed\norder of computation. Some data-driven control should be introduced in order to\ndeal with diverse contexts. The present paper describes a data-driven method of\nconstraint transformation with a sort of compilation which subsumes\naccessibility check and last-call optimization, which characterize standard\nnatural-language parsing techniques, semantic-head-driven generation, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0307044v1",
            "date": "2003-07-19 12:24:33+00:00",
            "title": "The Linguistic DS: Linguisitic Description in MPEG-7",
            "authors": [
                "Koiti Hasida"
            ],
            "abstract": "MPEG-7 (Moving Picture Experts Group Phase 7) is an XML-based international\nstandard on semantic description of multimedia content. This document discusses\nthe Linguistic DS and related tools. The linguistic DS is a tool, based on the\nGDA tag set (http://i-content.org/GDA/tagset.html), for semantic annotation of\nlinguistic data in or associated with multimedia content. The current document\ntext reflects `Study of FPDAM - MPEG-7 MDS Extensions' issued in March 2003,\nand not most part of MPEG-7 MDS, for which the readers are referred to the\nfirst version of MPEG-7 MDS document available from ISO (http://www.iso.org).\nWithout that reference, however, this document should be mostly intelligible to\nthose who are familiar with XML and linguistic theories. Comments are welcome\nand will be considered in the standardization process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308016v1",
            "date": "2003-08-07 09:01:56+00:00",
            "title": "Collaborative Creation of Digital Content in Indian Languages",
            "authors": [
                "Akshar Bharati",
                "Rajeev Sangal"
            ],
            "abstract": "The world is passing through a major revolution called the information\nrevolution, in which information and knowledge is becoming available to people\nin unprecedented amounts wherever and whenever they need it. Those societies\nwhich fail to take advantage of the new technology will be left behind, just\nlike in the industrial revolution.\n  The information revolution is based on two major technologies: computers and\ncommunication. These technologies have to be delivered in a COST EFFECTIVE\nmanner, and in LANGUAGES accessible to people.\n  One way to deliver them in cost effective manner is to make suitable\ntechnology choices, and to allow people to access through shared resources.\nThis could be done throuch street corner shops (for computer usage, e-mail\netc.), schools, community centres and local library centres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308017v1",
            "date": "2003-08-07 09:17:16+00:00",
            "title": "Information Revolution",
            "authors": [
                "Akshar Bharati",
                "Vineet Chaitanya",
                "Rajeev Sangal"
            ],
            "abstract": "The world is passing through a major revolution called the information\nrevolution, in which information and knowledge is becoming available to people\nin unprecedented amounts wherever and whenever they need it. Those societies\nwhich fail to take advantage of the new technology will be left behind, just\nlike in the industrial revolution.\n  The information revolution is based on two major technologies: computers and\ncommunication. These technologies have to be delivered in a COST EFFECTIVE\nmanner, and in LANGUAGES accessible to people.\n  One way to deliver them in cost effective manner is to make suitable\ntechnology choices (discussed later), and to allow people to access through\nshared resources. This could be done throuch street corner shops (for computer\nusage, e-mail etc.), schools, community centers and local library centres.",
            "journal ref": "Published as a keynote lecture in IRIL-99: Information Revolution\n  and Indian Languages, 12-14 Nov 1999",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308018v1",
            "date": "2003-08-07 09:24:46+00:00",
            "title": "Anusaaraka: Overcoming the Language Barrier in India",
            "authors": [
                "Akshar Bharati",
                "Vineet Chaitanya",
                "Amba P. Kulkarni",
                "Rajeev Sangal",
                "G Umamaheshwara Rao"
            ],
            "abstract": "The anusaaraka system makes text in one Indian language accessible in another\nIndian language. In the anusaaraka approach, the load is so divided between man\nand computer that the language load is taken by the machine, and the\ninterpretation of the text is left to the man. The machine presents an image of\nthe source text in a language close to the target language.In the image, some\nconstructions of the source language (which do not have equivalents) spill over\nto the output. Some special notation is also devised. The user after some\ntraining learns to read and understand the output. Because the Indian languages\nare close, the learning time of the output language is short, and is expected\nto be around 2 weeks.\n  The output can also be post-edited by a trained user to make it grammatically\ncorrect in the target language. Style can also be changed, if necessary. Thus,\nin this scenario, it can function as a human assisted translation system.\n  Currently, anusaarakas are being built from Telugu, Kannada, Marathi, Bengali\nand Punjabi to Hindi. They can be built for all Indian languages in the near\nfuture. Everybody must pitch in to build such systems connecting all Indian\nlanguages, using the free software model.",
            "journal ref": "Published in \"Anuvad: Approaches to Translation\", Rukmini Bhaya\n  Nair, (editor), Sage, New Delhi, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308019v1",
            "date": "2003-08-07 09:40:04+00:00",
            "title": "Language Access: An Information Based Approach",
            "authors": [
                "Akshar Bharati",
                "Vineet Chaitanya",
                "Amba P. Kulkarni",
                "Rajeev Sangal"
            ],
            "abstract": "The anusaaraka system (a kind of machine translation system) makes text in\none Indian language accessible through another Indian language. The machine\npresents an image of the source text in a language close to the target\nlanguage. In the image, some constructions of the source language (which do not\nhave equivalents in the target language) spill over to the output. Some special\nnotation is also devised.\n  Anusaarakas have been built from five pairs of languages: Telugu,Kannada,\nMarathi, Bengali and Punjabi to Hindi. They are available for use through Email\nservers.\n  Anusaarkas follows the principle of substitutibility and reversibility of\nstrings produced. This implies preservation of information while going from a\nsource language to a target language.\n  For narrow subject areas, specialized modules can be built by putting subject\ndomain knowledge into the system, which produce good quality grammatical\noutput. However, it should be remembered, that such modules will work only in\nnarrow areas, and will sometimes go wrong. In such a situation, anusaaraka\noutput will still remain useful.",
            "journal ref": "Published in the proceedings of Knowledge Based Computer Systems\n  Conference, 2000, Tata McGraw Hill, New Delhi 2000",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308020v1",
            "date": "2003-08-07 10:08:43+00:00",
            "title": "LERIL : Collaborative Effort for Creating Lexical Resources",
            "authors": [
                "Akshar Bharati",
                "Dipti M Sharma",
                "Vineet Chaitanya",
                "Amba P Kulkarni",
                "Rajeev Sangal",
                "Durgesh D Rao"
            ],
            "abstract": "The paper reports on efforts taken to create lexical resources pertaining to\nIndian languages, using the collaborative model. The lexical resources being\ndeveloped are: (1) Transfer lexicon and grammar from English to several Indian\nlanguages. (2) Dependencey tree bank of annotated corpora for several Indian\nlanguages. The dependency trees are based on the Paninian model. (3) Bilingual\ndictionary of 'core meanings'.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0309019v1",
            "date": "2003-09-12 12:43:00+00:00",
            "title": "Building a Test Collection for Speech-Driven Web Retrieval",
            "authors": [
                "Atsushi Fujii",
                "Katunobu Itou"
            ],
            "abstract": "This paper describes a test collection (benchmark data) for retrieval systems\ndriven by spoken queries. This collection was produced in the subtask of the\nNTCIR-3 Web retrieval task, which was performed in a TREC-style evaluation\nworkshop. The search topics and document collection for the Web retrieval task\nwere used to produce spoken queries and language models for speech recognition,\nrespectively. We used this collection to evaluate the performance of our\nretrieval system. Experimental results showed that (a) the use of target\ndocuments for language modeling and (b) enhancement of the vocabulary size in\nspeech recognition were effective in improving the system performance.",
            "journal ref": "Proceedings of the 8th European Conference on Speech Communication\n  and Technology (Eurospeech 2003), pp.1153-1156, Sep. 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0309021v1",
            "date": "2003-09-13 06:54:58+00:00",
            "title": "A Cross-media Retrieval System for Lecture Videos",
            "authors": [
                "Atsushi Fujii",
                "Katunobu Itou",
                "Tomoyosi Akiba",
                "Tetsuya Ishikawa"
            ],
            "abstract": "We propose a cross-media lecture-on-demand system, in which users can\nselectively view specific segments of lecture videos by submitting text\nqueries. Users can easily formulate queries by using the textbook associated\nwith a target lecture, even if they cannot come up with effective keywords. Our\nsystem extracts the audio track from a target lecture video, generates a\ntranscription by large vocabulary continuous speech recognition, and produces a\ntext index. Experimental results showed that by adapting speech recognition to\nthe topic of the lecture, the recognition accuracy increased and the retrieval\naccuracy was comparable with that obtained by human transcription.",
            "journal ref": "Proceedings of the 8th European Conference on Speech Communication\n  and Technology (Eurospeech 2003), pp.1149-1152, Sep. 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0310014v1",
            "date": "2003-10-08 16:40:33+00:00",
            "title": "Effective XML Representation for Spoken Language in Organisations",
            "authors": [
                "Rodney J. Clarke",
                "Philip C. Windridge",
                "Dali Dong"
            ],
            "abstract": "Spoken Language can be used to provide insights into organisational\nprocesses, unfortunately transcription and coding stages are very time\nconsuming and expensive. The concept of partial transcription and coding is\nproposed in which spoken language is indexed prior to any subsequent\nprocessing. The functional linguistic theory of texture is used to describe the\neffects of partial transcription on observational records. The standard used to\nencode transcript context and metadata is called CHAT, but a previous XML\nschema developed to implement it contains design assumptions that make it\ndifficult to support partial transcription for example. This paper describes a\nmore effective XML schema that overcomes many of these problems and is intended\nfor use in applications that support the rapid development of spoken language\ndeliverables.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0310058v1",
            "date": "2003-10-29 20:13:30+00:00",
            "title": "Application Architecture for Spoken Language Resources in Organisational Settings",
            "authors": [
                "Rodney J. Clarke",
                "Dali Dong",
                "Philip C. Windridge"
            ],
            "abstract": "Special technologies need to be used to take advantage of, and overcome, the\nchallenges associated with acquiring, transforming, storing, processing, and\ndistributing spoken language resources in organisations. This paper introduces\nan application architecture consisting of tools and supporting utilities for\nindexing and transcription, and describes how these tools, together with\ndownstream processing and distribution systems, can be integrated into a\nworkflow. Two sample applications for this architecture are outlined- the\nanalysis of decision-making processes in organisations and the deployment of\nsystems development methods by designers in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0311033v1",
            "date": "2003-11-21 19:48:17+00:00",
            "title": "The Rank-Frequency Analysis for the Functional Style Corpora in the Ukrainian Language",
            "authors": [
                "Solomija N. Buk",
                "Andrij A. Rovenchak"
            ],
            "abstract": "We use the rank-frequency analysis for the estimation of Kernel Vocabulary\nsize within specific corpora of Ukrainian. The extrapolation of high-rank\nbehaviour is utilized for estimation of the total vocabulary size.",
            "journal ref": "Journal of Quantitative Linguistics, Vol. 11, No. 3, P. 161-171\n  (2004)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0311036v1",
            "date": "2003-11-24 18:05:40+00:00",
            "title": "Measuring the Functional Load of Phonological Contrasts",
            "authors": [
                "Dinoj Surendran",
                "Partha Niyogi"
            ],
            "abstract": "Frequency counts are a measure of how much use a language makes of a\nlinguistic unit, such as a phoneme or word. However, what is often important is\nnot the units themselves, but the contrasts between them. A measure is\ntherefore needed for how much use a language makes of a contrast, i.e. the\nfunctional load (FL) of the contrast. We generalize previous work in\nlinguistics and speech recognition and propose a family of measures for the FL\nof several phonological contrasts, including phonemic oppositions, distinctive\nfeatures, suprasegmentals, and phonological rules. We then test it for\nrobustness to changes of corpora. Finally, we provide examples in Cantonese,\nDutch, English, German and Mandarin, in the context of historical linguistics,\nlanguage acquisition and speech recognition. More information can be found at\nhttp://dinoj.info/research/fload",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0302021v1",
            "date": "2003-02-14 07:11:50+00:00",
            "title": "Building an Open Language Archives Community on the OAI Foundation",
            "authors": [
                "Gary Simons",
                "Steven Bird"
            ],
            "abstract": "The Open Language Archives Community (OLAC) is an international partnership\nof institutions and individuals who are creating a worldwide virtual library of\nlanguage resources. The Dublin Core (DC) Element Set and the OAI Protocol have\nprovided a solid foundation for the OLAC framework. However, we need more\nprecision in community-specific aspects of resource description than is offered\nby DC. Furthermore, many of the institutions and individuals who might\nparticipate in OLAC do not have the technical resources to support the OAI\nprotocol. This paper presents our solutions to these two problems. To address\nthe first, we have developed an extensible application profile for language\nresource metadata. To address the second, we have implemented Vida (the virtual\ndata provider) and Viser (the virtual service provider), which permit community\nmembers to provide data and services without having to implement the OAI\nprotocol. These solutions are generic and could be adopted by other specialized\nsubcommunities.",
            "journal ref": "Library Hi Tech 21(2), 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0304028v1",
            "date": "2003-04-22 02:51:51+00:00",
            "title": "Grid-Enabling Natural Language Engineering By Stealth",
            "authors": [
                "Baden Hughes",
                "Steven Bird"
            ],
            "abstract": "We describe a proposal for an extensible, component-based software\narchitecture for natural language engineering applications. Our model leverages\nexisting linguistic resource description and discovery mechanisms based on\nextended Dublin Core metadata. In addition, the application design is flexible,\nallowing disparate components to be combined to suit the overall application\nfunctionality. An application specification language provides abstraction from\nthe programming environment and allows ease of interface with computational\ngrids via a broker.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0305004v1",
            "date": "2003-05-06 14:06:48+00:00",
            "title": "Approximate Grammar for Information Extraction",
            "authors": [
                "V. Sriram",
                "B. Ravi Sekar Reddy",
                "R. Sangal"
            ],
            "abstract": "In this paper, we present the concept of Approximate grammar and how it can\nbe used to extract information from a documemt. As the structure of\ninformational strings cannot be defined well in a document, we cannot use the\nconventional grammar rules to represent the information. Hence, the need arises\nto design an approximate grammar that can be used effectively to accomplish the\ntask of Information extraction. Approximate grammars are a novel step in this\ndirection. The rules of an approximate grammar can be given by a user or the\nmachine can learn the rules from an annotated document. We have performed our\nexperiments in both the above areas and the results have been impressive.",
            "journal ref": "Conference on Universal Knowledge and Language, Goa'2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0305053v1",
            "date": "2003-05-29 12:12:19+00:00",
            "title": "Developing Open Data Models for Linguistic Field Data",
            "authors": [
                "Baden Hughes"
            ],
            "abstract": "The UQ Flint Archive houses the field notes and elicitation recordings made\nby Elwyn Flint in the 1950's and 1960's during extensive linguistic survey work\nacross Queensland, Australia.\n  The process of digitizing the contents of the UQ Flint Archive provides a\nnumber of interesting challenges in the context of EMELD. Firstly, all of the\nlinguistic data is for languages which are either endangered or extinct, and as\nsuch forms a valuable ethnographic repository. Secondly, the physical format of\nthe data is itself in danger of decline, and as such digitization is an\nimportant preservation task in the short to medium term. Thirdly, the adoption\nof open standards for the encoding and presentation of text and audio data for\nlinguistic field data, whilst enabling preservation, represents a new field of\nresearch in itself where best practice has yet to be formalised. Fourthly, the\nprovision of this linguistic data online as a new data source for future\nresearch introduces concerns of data portability and longevity.\n  This paper will outline the origins of the data model, the content creation\ncomponents, presentation forms based on the data model, data capture tools and\nmedia conversion components. It will also address some of the larger questions\nregarding the digitization and annotation of linguistic field work based on\nexperience gained through work with the Flint Archive contents.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306022v1",
            "date": "2003-06-04 23:08:03+00:00",
            "title": "Techniques for effective vocabulary selection",
            "authors": [
                "Anand Venkataraman",
                "Wen Wang"
            ],
            "abstract": "The vocabulary of a continuous speech recognition (CSR) system is a\nsignificant factor in determining its performance. In this paper, we present\nthree principled approaches to select the target vocabulary for a particular\ndomain by trading off between the target out-of-vocabulary (OOV) rate and\nvocabulary size. We evaluate these approaches against an ad-hoc baseline\nstrategy. Results are presented in the form of OOV rate graphs plotted against\nincreasing vocabulary size for each technique.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306040v1",
            "date": "2003-06-10 07:33:32+00:00",
            "title": "The Open Language Archives Community: An infrastructure for distributed archiving of language resources",
            "authors": [
                "Gary Simons",
                "Steven Bird"
            ],
            "abstract": "New ways of documenting and describing language via electronic media coupled\nwith new ways of distributing the results via the World-Wide Web offer a degree\nof access to language resources that is unparalleled in history. At the same\ntime, the proliferation of approaches to using these new technologies is\ncausing serious problems relating to resource discovery and resource creation.\nThis article describes the infrastructure that the Open Language Archives\nCommunity (OLAC) has built in order to address these problems. Its technical\nand usage infrastructures address problems of resource discovery by\nconstructing a single virtual library of distributed resources. Its governance\ninfrastructure addresses problems of resource creation by providing a mechanism\nthrough which the language-resource community can express its consensus on\nrecommended best practices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306130v1",
            "date": "2003-06-25 10:26:29+00:00",
            "title": "Anusaaraka: Machine Translation in Stages",
            "authors": [
                "Akshar Bharati",
                "Vineet Chaitanya",
                "Amba P. Kulkarni",
                "Rajeev Sangal"
            ],
            "abstract": "Fully-automatic general-purpose high-quality machine translation systems\n(FGH-MT) are extremely difficult to build. In fact, there is no system in the\nworld for any pair of languages which qualifies to be called FGH-MT. The\nreasons are not far to seek. Translation is a creative process which involves\ninterpretation of the given text by the translator. Translation would also vary\ndepending on the audience and the purpose for which it is meant. This would\nexplain the difficulty of building a machine translation system. Since, the\nmachine is not capable of interpreting a general text with sufficient accuracy\nautomatically at present - let alone re-expressing it for a given audience, it\nfails to perform as FGH-MT. FOOTNOTE{The major difficulty that the machine\nfaces in interpreting a given text is the lack of general world knowledge or\ncommon sense knowledge.}",
            "journal ref": "Vivek, A Quarterly in Artificial Intelligence, 10, 3, July 1997,\n  pp. 22-25",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0307014v1",
            "date": "2003-07-07 08:47:53+00:00",
            "title": "Syntax, Parsing and Production of Natural Language in a Framework of Information Compression by Multiple Alignment, Unification and Search",
            "authors": [
                "J Gerard Wolff"
            ],
            "abstract": "This article introduces the idea that \"information compression by multiple\nalignment, unification and search\" (ICMAUS) provides a framework within which\nnatural language syntax may be represented in a simple format and the parsing\nand production of natural language may be performed in a transparent manner.\n  The ICMAUS concepts are embodied in a software model, SP61. The organisation\nand operation of the model are described and a simple example is presented\nshowing how the model can achieve parsing of natural language.\n  Notwithstanding the apparent paradox of 'decompression by compression', the\nICMAUS framework, without any modification, can produce a sentence by decoding\na compressed code for the sentence. This is illustrated with output from the\nSP61 model.\n  The article includes four other examples - one of the parsing of a sentence\nin French and three from the domain of English auxiliary verbs. These examples\nshow how the ICMAUS framework and the SP61 model can accommodate 'context\nsensitive' features of syntax in a relatively simple and direct manner.",
            "journal ref": "Journal of Universal Computer Science 6(8), 781--829, 2000",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308008v1",
            "date": "2003-08-05 00:46:08+00:00",
            "title": "A Grid Based Architecture for High-Performance NLP",
            "authors": [
                "Baden Hughes",
                "Steven Bird"
            ],
            "abstract": "We describe the design and early implementation of an extensible,\ncomponent-based software architecture for natural language engineering\napplications which interfaces with high performance distributed computing\nservices. The architecture leverages existing linguistic resource description\nand discovery mechanisms based on metadata descriptions, combining these in a\ncompatible fashion with other software definition abstractions. Within this\narchitecture, application design is highly flexible, allowing disparate\ncomponents to be combined to suit the overall application functionality, and\nformally described independently of processing concerns. An application\nspecification language provides abstraction from the programming environment\nand allows ease of interface with high performance computational grids via a\nbroker.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308022v1",
            "date": "2003-08-14 23:28:21+00:00",
            "title": "Extending Dublin Core Metadata to Support the Description and Discovery of Language Resources",
            "authors": [
                "Steven Bird",
                "Gary Simons"
            ],
            "abstract": "As language data and associated technologies proliferate and as the language\nresources community expands, it is becoming increasingly difficult to locate\nand reuse existing resources. Are there any lexical resources for such-and-such\na language? What tool works with transcripts in this particular format? What is\na good format to use for linguistic data of this type? Questions like these\ndominate many mailing lists, since web search engines are an unreliable way to\nfind language resources. This paper reports on a new digital infrastructure for\ndiscovering language resources being developed by the Open Language Archives\nCommunity (OLAC). At the core of OLAC is its metadata format, which is designed\nto facilitate description and discovery of all kinds of language resources,\nincluding data, tools, or advice. The paper describes OLAC metadata, its\nrelationship to Dublin Core metadata, and its dissemination using the metadata\nharvesting protocol of the Open Archives Initiative.",
            "journal ref": "Computing and the Humanities, 37 (4), 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308032v1",
            "date": "2003-08-20 18:40:39+00:00",
            "title": "Evaluation of text data mining for database curation: lessons learned from the KDD Challenge Cup",
            "authors": [
                "Alexander S. Yeh",
                "Lynette Hirschman",
                "Alexander A. Morgan"
            ],
            "abstract": "MOTIVATION: The biological literature is a major repository of knowledge.\nMany biological databases draw much of their content from a careful curation of\nthis literature. However, as the volume of literature increases, the burden of\ncuration increases. Text mining may provide useful tools to assist in the\ncuration process. To date, the lack of standards has made it impossible to\ndetermine whether text mining techniques are sufficiently mature to be useful.\n  RESULTS: We report on a Challenge Evaluation task that we created for the\nKnowledge Discovery and Data Mining (KDD) Challenge Cup. We provided a training\ncorpus of 862 articles consisting of journal articles curated in FlyBase, along\nwith the associated lists of genes and gene products, as well as the relevant\ndata fields from FlyBase. For the test, we provided a corpus of 213 new\n(`blind') articles; the 18 participating groups provided systems that flagged\narticles for curation, based on whether the article contained experimental\nevidence for gene expression products. We report on the the evaluation results\nand describe the techniques used by the top performing groups.\n  CONTACT: asy@mitre.org\n  KEYWORDS: text mining, evaluation, curation, genomics, data management",
            "journal ref": "Bioinformatics Vol. 19 Suppl. 1 2003, pages i331-i339",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0310018v1",
            "date": "2003-10-10 15:25:57+00:00",
            "title": "The Study of the Application of a Keywords-based Chatbot System on the Teaching of Foreign Languages",
            "authors": [
                "Jiyou Jia"
            ],
            "abstract": "This paper reports the findings of a study conducted on the application of an\non-line human-computer dialog system with natural language (chatbot) on the\nteaching of foreign languages. A keywords-based human-computer dialog system\nmakes it possible that the user could chat with the computer using a natural\nlanguage, i.e. in English or in German to some extent. So an experiment has\nbeen made using this system online to work as a chat partner with the users\nlearning the foreign languages. Dialogs between the users and the chatbot are\ncollected. Findings indicate that the dialogs between the human and the\ncomputer are mostly very short because the user finds the responses from the\ncomputer are mostly repeated and irrelevant with the topics and context and the\nprogram does not understand the language at all. With analysis of the keywords\nor pattern-matching mechanism used in this chatbot it can be concluded that\nthis kind of system can not work as a teaching assistant program in foreign\nlanguage learning.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/cs/0310041v1",
            "date": "2003-10-21 18:19:52+00:00",
            "title": "A Dynamic Programming Algorithm for the Segmentation of Greek Texts",
            "authors": [
                "Pavlina Fragkou"
            ],
            "abstract": "In this paper we introduce a dynamic programming algorithm to perform linear\ntext segmentation by global minimization of a segmentation cost function which\nconsists of: (a) within-segment word similarity and (b) prior information about\nsegment length. The evaluation of the segmentation accuracy of the algorithm on\na text collection consisting of Greek texts showed that the algorithm achieves\nhigh segmentation accuracy and appears to be very innovating and promissing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312008v1",
            "date": "2003-12-03 13:09:28+00:00",
            "title": "Embedding Web-based Statistical Translation Models in Cross-Language Information Retrieval",
            "authors": [
                "Wessel Kraaij",
                "Jian-Yun Nie",
                "Michel Simard"
            ],
            "abstract": "Although more and more language pairs are covered by machine translation\nservices, there are still many pairs that lack translation resources.\nCross-language information retrieval (CLIR) is an application which needs\ntranslation functionality of a relatively low level of sophistication since\ncurrent models for information retrieval (IR) are still based on a\nbag-of-words. The Web provides a vast resource for the automatic construction\nof parallel corpora which can be used to train statistical translation models\nautomatically. The resulting translation models can be embedded in several ways\nin a retrieval model. In this paper, we will investigate the problem of\nautomatically mining parallel texts from the Web and different ways of\nintegrating the translation models within the retrieval process. Our\nexperiments on standard test collections for CLIR show that the Web-based\ntranslation models can surpass commercial MT systems in CLIR tasks. These\nresults open the perspective of constructing a fully automatic query\ntranslation device for CLIR at a very low cost.",
            "journal ref": "Computational Linguistics 29(3) september 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312050v1",
            "date": "2003-12-22 16:23:34+00:00",
            "title": "A Flexible Pragmatics-driven Language Generator for Animated Agents",
            "authors": [
                "Paul Piwek"
            ],
            "abstract": "This paper describes the NECA MNLG; a fully implemented Multimodal Natural\nLanguage Generation module. The MNLG is deployed as part of the NECA system\nwhich generates dialogues between animated agents. The generation module\nsupports the seamless integration of full grammar rules, templates and canned\ntext. The generator takes input which allows for the specification of\nsyntactic, semantic and pragmatic constraints on the output.",
            "journal ref": "Proceedings of the Research Note Sessions of the 10th Conference\n  of the European Chapter of the Association for Computational Linguistics\n  (EACL'03), 2003, pp. 151-154",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312051v1",
            "date": "2003-12-22 16:51:53+00:00",
            "title": "Towards Automated Generation of Scripted Dialogue: Some Time-Honoured Strategies",
            "authors": [
                "Paul Piwek",
                "Kees van Deemter"
            ],
            "abstract": "The main aim of this paper is to introduce automated generation of scripted\ndialogue as a worthwhile topic of investigation. In particular the fact that\nscripted dialogue involves two layers of communication, i.e., uni-directional\ncommunication between the author and the audience of a scripted dialogue and\nbi-directional pretended communication between the characters featuring in the\ndialogue, is argued to raise some interesting issues. Our hope is that the\ncombined study of the two layers will forge links between research in text\ngeneration and dialogue processing. The paper presents a first attempt at\ncreating such links by studying three types of strategies for the automated\ngeneration of scripted dialogue. The strategies are derived from examples of\nhuman-authored and naturally occurring dialogue.",
            "journal ref": "Proceedings of EDILOG: 6th Workshop on the Semantics and\n  Pragmatics of Dialogue, 2002, pp. 141-148",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312052v1",
            "date": "2003-12-22 17:07:31+00:00",
            "title": "Dialogue as Discourse: Controlling Global Properties of Scripted Dialogue",
            "authors": [
                "Paul Piwek",
                "Kees van Deemter"
            ],
            "abstract": "This paper explains why scripted dialogue shares some crucial properties with\ndiscourse. In particular, when scripted dialogues are generated by a Natural\nLanguage Generation system, the generator can apply revision strategies that\ncannot normally be used when the dialogue results from an interaction between\nautonomous agents (i.e., when the dialogue is not scripted). The paper explains\nthat the relevant revision operators are best applied at the level of a\ndialogue plan and discusses how the generator may decide when to apply a given\nrevision operator.",
            "journal ref": "Proceedings of AAAI Spring Symposium on Natural Language\n  Generation in Spoken and Written Dialogue, Stanford, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312060v1",
            "date": "2003-12-27 21:21:48+00:00",
            "title": "Part-of-Speech Tagging with Minimal Lexicalization",
            "authors": [
                "Virginia Savova",
                "Leonid Peshkin"
            ],
            "abstract": "We use a Dynamic Bayesian Network to represent compactly a variety of\nsublexical and contextual features relevant to Part-of-Speech (PoS) tagging.\nThe outcome is a flexible tagger (LegoTag) with state-of-the-art performance\n(3.6% error on a benchmark corpus). We explore the effect of eliminating\nredundancy and radically reducing the size of feature vocabularies. We find\nthat a small but linguistically motivated set of suffixes results in improved\ncross-corpora generalization. We also show that a minimal lexicon limited to\nfunction words is sufficient to ensure reasonable performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/quant-ph/0309022v4",
            "date": "2003-09-01 19:10:55+00:00",
            "title": "Quantum Aspects of Semantic Analysis and Symbolic Artificial Intelligence",
            "authors": [
                "Diederik Aerts",
                "Marek Czachor"
            ],
            "abstract": "Modern approaches to semanic analysis if reformulated as Hilbert-space\nproblems reveal formal structures known from quantum mechanics. Similar\nsituation is found in distributed representations of cognitive structures\ndeveloped for the purposes of neural networks. We take a closer look at\nsimilarites and differences between the above two fields and quantum\ninformation theory.",
            "journal ref": "J. Phys. A: Math. Gen. 37 (2004) L123-L132",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/cs/0306039v1",
            "date": "2003-06-10 04:40:45+00:00",
            "title": "Bayesian Information Extraction Network",
            "authors": [
                "Leonid Peshkin",
                "Avi Pfeffer"
            ],
            "abstract": "Dynamic Bayesian networks (DBNs) offer an elegant way to integrate various\naspects of language in one model. Many existing algorithms developed for\nlearning and inference in DBNs are applicable to probabilistic language\nmodeling. To demonstrate the potential of DBNs for natural language processing,\nwe employ a DBN in an information extraction task. We show how to assemble\nwealth of emerging linguistic instruments for shallow parsing, syntactic and\nsemantic tagging, morphological decomposition, named entity recognition etc. in\norder to incrementally build a robust information extraction system. Our method\noutperforms previously published results on an established benchmark domain.",
            "journal ref": "Intl. Joint Conference on Artificial Intelligence, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0307055v1",
            "date": "2003-07-24 21:09:43+00:00",
            "title": "Learning Analogies and Semantic Relations",
            "authors": [
                "Peter D. Turney",
                "Michael L. Littman"
            ],
            "abstract": "We present an algorithm for learning from unlabeled text, based on the Vector\nSpace Model (VSM) of information retrieval, that can solve verbal analogy\nquestions of the kind found in the Scholastic Aptitude Test (SAT). A verbal\nanalogy has the form A:B::C:D, meaning \"A is to B as C is to D\"; for example,\nmason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B,\nand the problem is to select the most analogous word pair, C:D, from a set of\nfive choices. The VSM algorithm correctly answers 47% of a collection of 374\ncollege-level analogy questions (random guessing would yield 20% correct). We\nmotivate this research by relating it to work in cognitive science and\nlinguistics, and by applying it to a difficult problem in natural language\nprocessing, determining semantic relations in noun-modifier pairs. The problem\nis to classify a noun-modifier pair, such as \"laser printer\", according to the\nsemantic relation between the noun (printer) and the modifier (laser). We use a\nsupervised nearest-neighbour algorithm that assigns a class to a given\nnoun-modifier pair by finding the most analogous noun-modifier pair in the\ntraining data. With 30 classes of semantic relations, on a collection of 600\nlabeled noun-modifier pairs, the learning algorithm attains an F value of 26.5%\n(random guessing: 3.3%). With 5 classes of semantic relations, the F value is\n43.2% (random: 20%). The performance is state-of-the-art for these challenging\nproblems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0308033v1",
            "date": "2003-08-20 20:42:19+00:00",
            "title": "Coherent Keyphrase Extraction via Web Mining",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Keyphrases are useful for a variety of purposes, including summarizing,\nindexing, labeling, categorizing, clustering, highlighting, browsing, and\nsearching. The task of automatic keyphrase extraction is to select keyphrases\nfrom within the text of a given document. Automatic keyphrase extraction makes\nit feasible to generate keyphrases for the huge number of documents that do not\nhave manually assigned keyphrases. A limitation of previous keyphrase\nextraction algorithms is that the selected keyphrases are occasionally\nincoherent. That is, the majority of the output keyphrases may fit together\nwell, but there may be a minority that appear to be outliers, with no clear\nsemantic relation to the majority or to each other. This paper presents\nenhancements to the Kea keyphrase extraction algorithm that are designed to\nincrease the coherence of the extracted keyphrases. The approach is to use the\ndegree of statistical association among candidate keyphrases as evidence that\nthey may be semantically related. The statistical association is measured using\nweb mining. Experiments demonstrate that the enhancements improve the quality\nof the extracted keyphrases. Furthermore, the enhancements are not\ndomain-specific: the algorithm generalizes well when it is trained on one\ndomain (computer science documents) and tested on another (physics documents).",
            "journal ref": "Proceedings of the Eighteenth International Joint Conference on\n  Artificial Intelligence (IJCAI-03), (2003), Acapulco, Mexico, 434-439",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0309034v1",
            "date": "2003-09-19 16:30:55+00:00",
            "title": "Measuring Praise and Criticism: Inference of Semantic Orientation from Association",
            "authors": [
                "Peter D. Turney",
                "Michael L. Littman"
            ],
            "abstract": "The evaluative character of a word is called its semantic orientation.\nPositive semantic orientation indicates praise (e.g., \"honest\", \"intrepid\") and\nnegative semantic orientation indicates criticism (e.g., \"disturbing\",\n\"superfluous\"). Semantic orientation varies in both direction (positive or\nnegative) and degree (mild to strong). An automated system for measuring\nsemantic orientation would have application in text classification, text\nfiltering, tracking opinions in online discussions, analysis of survey\nresponses, and automated chat systems (chatbots). This paper introduces a\nmethod for inferring the semantic orientation of a word from its statistical\nassociation with a set of positive and negative paradigm words. Two instances\nof this approach are evaluated, based on two different statistical measures of\nword association: pointwise mutual information (PMI) and latent semantic\nanalysis (LSA). The method is experimentally tested with 3,596 words (including\nadjectives, adverbs, nouns, and verbs) that have been manually labeled positive\n(1,614 words) and negative (1,982 words). The method attains an accuracy of\n82.8% on the full test set, but the accuracy rises above 95% when the algorithm\nis allowed to abstain from classifying mild words.",
            "journal ref": "ACM Transactions on Information Systems (TOIS), (2003), 21 (4),\n  315-346",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0309035v1",
            "date": "2003-09-19 20:13:07+00:00",
            "title": "Combining Independent Modules to Solve Multiple-choice Synonym and Analogy Problems",
            "authors": [
                "Peter D. Turney",
                "Michael L. Littman",
                "Jeffrey Bigham",
                "Victor Shnayder"
            ],
            "abstract": "Existing statistical approaches to natural language problems are very coarse\napproximations to the true complexity of language processing. As such, no\nsingle technique will be best for all problem instances. Many researchers are\nexamining ensemble methods that combine the output of successful, separately\ndeveloped modules to create more accurate solutions. This paper examines three\nmerging rules for combining probability distributions: the well known mixture\nrule, the logarithmic rule, and a novel product rule. These rules were applied\nwith state-of-the-art results to two problems commonly used to assess human\nmastery of lexical semantics -- synonym questions and analogy questions. All\nthree merging rules result in ensembles that are more accurate than any of\ntheir component modules. The differences among the three rules are not\nstatistically significant, but it is suggestive that the popular mixture rule\nis not the best rule for either of the two problems.",
            "journal ref": "Proceedings of the International Conference on Recent Advances in\n  Natural Language Processing (RANLP-03), (2003), Borovets, Bulgaria, 482-489",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0312058v1",
            "date": "2003-12-25 16:45:20+00:00",
            "title": "Acquiring Lexical Paraphrases from a Single Corpus",
            "authors": [
                "Oren Glickman",
                "Ido Dagan"
            ],
            "abstract": "This paper studies the potential of identifying lexical paraphrases within a\nsingle corpus, focusing on the extraction of verb paraphrases. Most previous\napproaches detect individual paraphrase instances within a pair (or set) of\ncomparable corpora, each of them containing roughly the same information, and\nrely on the substantial level of correspondence of such corpora. We present a\nnovel method that successfully detects isolated paraphrase instances within a\nsingle corpus without relying on any a-priori structure and information. A\ncomparison suggests that an instance-based approach may be combined with a\nvector based approach in order to assess better the paraphrase likelihood for\nmany verb pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/physics/0307117v1",
            "date": "2003-07-23 15:51:34+00:00",
            "title": "Symbolic stochastic dynamical systems viewed as binary N-step Markov chains",
            "authors": [
                "O. V. Usatenko",
                "V. A. Yampol'skii",
                "K. E. Kechedzhy",
                "S. S. Mel'nyk"
            ],
            "abstract": "A theory of systems with long-range correlations based on the consideration\nof binary N-step Markov chains is developed. In the model, the conditional\nprobability that the i-th symbol in the chain equals zero (or unity) is a\nlinear function of the number of unities among the preceding N symbols. The\ncorrelation and distribution functions as well as the variance of number of\nsymbols in the words of arbitrary length L are obtained analytically and\nnumerically. A self-similarity of the studied stochastic process is revealed\nand the similarity group transformation of the chain parameters is presented.\nThe diffusion Fokker-Planck equation governing the distribution function of the\nL-words is explored. If the persistent correlations are not extremely strong,\nthe distribution function is shown to be the Gaussian with the variance being\nnonlinearly dependent on L. The applicability of the developed theory to the\ncoarse-grained written and DNA texts is discussed.",
            "journal ref": "None",
            "category": "physics.data-an"
        }
    ],
    "2004": [
        {
            "url": "http://arxiv.org/abs/cs/0402055v1",
            "date": "2004-02-24 09:34:16+00:00",
            "title": "Lexical Base as a Compressed Language Model of the World (on the material of the Ukrainian language)",
            "authors": [
                "Solomiya Buk"
            ],
            "abstract": "In the article the fact is verified that the list of words selected by formal\nstatistical methods (frequency and functional genre unrestrictedness) is not a\nconglomerate of non-related words. It creates a system of interrelated items\nand it can be named \"lexical base of language\". This selected list of words\ncovers all the spheres of human activities. To verify this statement the\ninvariant synoptical scheme common for ideographic dictionaries of different\nlanguage was determined.",
            "journal ref": "Psychology of Language and Communication. 2009, vol. 13, no. 2,\n  pp. 35-44",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404007v1",
            "date": "2004-04-05 02:14:50+00:00",
            "title": "Polarity sensitivity and evaluation order in type-logical grammar",
            "authors": [
                "Chung-chieh Shan"
            ],
            "abstract": "We present a novel, type-logical analysis of_polarity sensitivity_: how\nnegative polarity items (like \"any\" and \"ever\") or positive ones (like \"some\")\nare licensed or prohibited. It takes not just scopal relations but also linear\norder into account, using the programming-language notions of delimited\ncontinuations and evaluation order, respectively. It thus achieves greater\nempirical coverage than previous proposals.",
            "journal ref": "Proceedings of the 2004 Human Language Technology Conference of\n  the North American Chapter of the Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404009v1",
            "date": "2004-04-05 11:51:43+00:00",
            "title": "Tabular Parsing",
            "authors": [
                "Mark-Jan Nederhof",
                "Giorgio Satta"
            ],
            "abstract": "This is a tutorial on tabular parsing, on the basis of tabulation of\nnondeterministic push-down automata. Discussed are Earley's algorithm, the\nCocke-Kasami-Younger algorithm, tabular LR parsing, the construction of parse\ntrees, and further issues.",
            "journal ref": "M.-J. Nederhof and G. Satta. Tabular Parsing. In C. Martin-Vide,\n  V. Mitrana, and G. Paun, editors, Formal Languages and Applications, Studies\n  in Fuzziness and Soft Computing 148, pages 529-549. Springer, 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404025v1",
            "date": "2004-04-10 08:43:31+00:00",
            "title": "Test Collections for Patent-to-Patent Retrieval and Patent Map Generation in NTCIR-4 Workshop",
            "authors": [
                "Atsushi Fujii",
                "Makoto Iwayama",
                "Noriko Kando"
            ],
            "abstract": "This paper describes the Patent Retrieval Task in the Fourth NTCIR Workshop,\nand the test collections produced in this task. We perform the invalidity\nsearch task, in which each participant group searches a patent collection for\nthe patents that can invalidate the demand in an existing claim. We also\nperform the automatic patent map generation task, in which the patents\nassociated with a specific topic are organized in a multi-dimensional matrix.",
            "journal ref": "Proceedings of the 4th International Conference on Language\n  Resources and Evaluation (LREC-2004), pp.1643-1646, May. 2004.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0405037v1",
            "date": "2004-05-10 16:05:23+00:00",
            "title": "A Probabilistic Model of Machine Translation",
            "authors": [
                "G. E. Miram",
                "V. K. Petrov"
            ],
            "abstract": "A probabilistic model for computer-based generation of a machine translation\nsystem on the basis of English-Russian parallel text corpora is suggested. The\nmodel is trained using parallel text corpora with pre-aligned source and target\nsentences. The training of the model results in a bilingual dictionary of words\nand \"word blocks\" with relevant translation probability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0405039v1",
            "date": "2004-05-12 14:14:52+00:00",
            "title": "Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization",
            "authors": [
                "Regina Barzilay",
                "Lillian Lee"
            ],
            "abstract": "We consider the problem of modeling the content structure of texts within a\nspecific domain, in terms of the topics the texts address and the order in\nwhich these topics appear. We first present an effective knowledge-lean method\nfor learning content models from un-annotated documents, utilizing a novel\nadaptation of algorithms for Hidden Markov Models. We then apply our method to\ntwo complementary tasks: information ordering and extractive summarization. Our\nexperiments show that incorporating content models in these applications yields\nsubstantial improvement over previously-proposed methods.",
            "journal ref": "HLT-NAACL 2004: Proceedings of the Main Conference, pp. 113--120",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0406031v1",
            "date": "2004-06-17 12:29:29+00:00",
            "title": "A Public Reference Implementation of the RAP Anaphora Resolution Algorithm",
            "authors": [
                "Long Qiu",
                "Min-Yen Kan",
                "Tat-Seng Chua"
            ],
            "abstract": "This paper describes a standalone, publicly-available implementation of the\nResolution of Anaphora Procedure (RAP) given by Lappin and Leass (1994). The\nRAP algorithm resolves third person pronouns, lexical anaphors, and identifies\npleonastic pronouns. Our implementation, JavaRAP, fills a current need in\nanaphora resolution research by providing a reference implementation that can\nbe benchmarked against current algorithms. The implementation uses the\nstandard, publicly available Charniak (2000) parser as input, and generates a\nlist of anaphora-antecedent pairs as output. Alternately, an in-place\nannotation or substitution of the anaphors with their antecedents can be\nproduced. Evaluation on the MUC-6 co-reference task shows that JavaRAP has an\naccuracy of 57.9%, similar to the performance given previously in the\nliterature (e.g., Preiss 2002).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0406054v1",
            "date": "2004-06-28 10:25:22+00:00",
            "title": "Building a linguistic corpus from bee dance data",
            "authors": [
                "J. J. Paijmans"
            ],
            "abstract": "This paper discusses the problems and possibility of collecting bee dance\ndata in a linguistic \\textit{corpus} and use linguistic instruments such as\nZipf's law and entropy statistics to decide on the question whether the dance\ncarries information of any kind. We describe this against the historical\nbackground of attempts to analyse non-human communication systems.",
            "journal ref": "Proceedings of the first international congres of bioinformatics,\n  Havana (Cuba), 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407002v1",
            "date": "2004-07-01 16:18:52+00:00",
            "title": "Annotating Predicate-Argument Structure for a Parallel Treebank",
            "authors": [
                "Lea Cyrus",
                "Hendrik Feddes",
                "Frank Schumacher"
            ],
            "abstract": "We report on a recently initiated project which aims at building a\nmulti-layered parallel treebank of English and German. Particular attention is\ndevoted to a dedicated predicate-argument layer which is used for aligning\ntranslationally equivalent sentences of the two languages. We describe both our\nconceptual decisions and aspects of their technical realisation. We discuss\nsome selected problems and conclude with a few remarks on how this project\nrelates to similar projects in the field.",
            "journal ref": "Proceedings of the LREC 2004 Workshop on Building Lexical\n  Resources from Semantically Annotated Corpora, Lisbon, May 30, 2004, pp.\n  39-46",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407005v3",
            "date": "2004-07-01 22:02:10+00:00",
            "title": "Statistical Machine Translation by Generalized Parsing",
            "authors": [
                "I. Dan Melamed",
                "Wei Wang"
            ],
            "abstract": "Designers of statistical machine translation (SMT) systems have begun to\nemploy tree-structured translation models. Systems involving tree-structured\ntranslation models tend to be complex. This article aims to reduce the\nconceptual complexity of such systems, in order to make them easier to design,\nimplement, debug, use, study, understand, explain, modify, and improve. In\nservice of this goal, the article extends the theory of semiring parsing to\narrive at a novel abstract parsing algorithm with five functional parameters: a\nlogic, a grammar, a semiring, a search strategy, and a termination condition.\nThe article then shows that all the common algorithms that revolve around\ntree-structured translation models, including hierarchical alignment, inference\nfor parameter estimation, translation, and structured evaluation, can be\nderived by generalizing two of these parameters -- the grammar and the logic.\nThe article culminates with a recipe for using such generalized parsers to\ntrain, apply, and evaluate an SMT system that is driven by tree-structured\ntranslation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407026v1",
            "date": "2004-07-10 11:18:42+00:00",
            "title": "Summarizing Encyclopedic Term Descriptions on the Web",
            "authors": [
                "Atsushi Fujii",
                "Tetsuya Ishikawa"
            ],
            "abstract": "We are developing an automatic method to compile an encyclopedic corpus from\nthe Web. In our previous work, paragraph-style descriptions for a term are\nextracted from Web pages and organized based on domains. However, these\ndescriptions are independent and do not comprise a condensed text as in\nhand-crafted encyclopedias. To resolve this problem, we propose a summarization\nmethod, which produces a single text from multiple descriptions. The resultant\nsummary concisely describes a term from different viewpoints. We also show the\neffectiveness of our method by means of experiments.",
            "journal ref": "Proceedings of the 20th International Conference on Computational\n  Linguistics (COLING 2004), pp.645-651, Aug. 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407027v1",
            "date": "2004-07-10 11:45:57+00:00",
            "title": "Unsupervised Topic Adaptation for Lecture Speech Retrieval",
            "authors": [
                "Atsushi Fujii",
                "Katunobu Itou",
                "Tomoyosi Akiba",
                "Tetsuya Ishikawa"
            ],
            "abstract": "We are developing a cross-media information retrieval system, in which users\ncan view specific segments of lecture videos by submitting text queries. To\nproduce a text index, the audio track is extracted from a lecture video and a\ntranscription is generated by automatic speech recognition. In this paper, to\nimprove the quality of our retrieval system, we extensively investigate the\neffects of adapting acoustic and language models on speech recognition. We\nperform an MLLR-based method to adapt an acoustic model. To obtain a corpus for\nlanguage model adaptation, we use the textbook for a target lecture to search a\nWeb collection for the pages associated with the lecture topic. We show the\neffectiveness of our method by means of experiments.",
            "journal ref": "Proceedings of the 8th International Conference on Spoken Language\n  Processing (ICSLP 2004), pp.2957-2960, Oct. 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407028v1",
            "date": "2004-07-10 11:57:17+00:00",
            "title": "Effects of Language Modeling on Speech-driven Question Answering",
            "authors": [
                "Tomoyosi Akiba",
                "Atsushi Fujii",
                "Katunobu Itou"
            ],
            "abstract": "We integrate automatic speech recognition (ASR) and question answering (QA)\nto realize a speech-driven QA system, and evaluate its performance. We adapt an\nN-gram language model to natural language questions, so that the input of our\nsystem can be recognized with a high accuracy. We target WH-questions which\nconsist of the topic part and fixed phrase used to ask about something. We\nfirst produce a general N-gram model intended to recognize the topic and\nemphasize the counts of the N-grams that correspond to the fixed phrases. Given\na transcription by the ASR engine, the QA engine extracts the answer candidates\nfrom target documents. We propose a passage retrieval method robust against\nrecognition errors in the transcription. We use the QA test collection produced\nin NTCIR, which is a TREC-style evaluation workshop, and show the effectiveness\nof our method by means of experiments.",
            "journal ref": "Proceedings of the 8th International Conference on Spoken Language\n  Processing (ICSLP 2004), pp.1053-1056, Oct. 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407046v1",
            "date": "2004-07-19 11:57:42+00:00",
            "title": "A Bimachine Compiler for Ranked Tagging Rules",
            "authors": [
                "Wojciech Skut",
                "Stefan Ulrich",
                "Kathrine Hammervold"
            ],
            "abstract": "This paper describes a novel method of compiling ranked tagging rules into a\ndeterministic finite-state device called a bimachine. The rules are formulated\nin the framework of regular rewrite operations and allow unrestricted regular\nexpressions in both left and right rule contexts. The compiler is illustrated\nby an application within a speech synthesis system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408052v1",
            "date": "2004-08-22 19:32:48+00:00",
            "title": "Application of the Double Metaphone Algorithm to Amharic Orthography",
            "authors": [
                "Daniel Yacob"
            ],
            "abstract": "The Metaphone algorithm applies the phonetic encoding of orthographic\nsequences to simplify words prior to comparison. While Metaphone has been\nhighly successful for the English language, for which it was designed, it may\nnot be applied directly to Ethiopian languages. The paper details how the\nprinciples of Metaphone can be applied to Ethiopic script and uses Amharic as a\ncase study. Match results improve as specific considerations are made for\nAmharic writing practices. Results are shown to improve further when common\nerrors from Amharic input methods are considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408059v1",
            "date": "2004-08-26 10:50:45+00:00",
            "title": "Proofing Tools Technology at Neurosoft S.A.",
            "authors": [
                "Ch. Tsalidis",
                "G. Orphanos",
                "A. Iordanidou",
                "A. Vagelatos"
            ],
            "abstract": "The aim of this paper is to present the R&D activities carried out at\nNeurosoft S.A. regarding the development of proofing tools for Modern Greek.\nFirstly, we focus on infrastructure issues that we faced during our initial\nsteps. Subsequently, we describe the most important insights of three proofing\ntools developed by Neurosoft, i.e. the spelling checker, the hyphenator and the\nthesaurus, outlining their efficiencies and inefficiencies. Finally, we discuss\nsome improvement ideas and give our future directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408060v1",
            "date": "2004-08-26 12:44:15+00:00",
            "title": "Verbal chunk extraction in French using limited resources",
            "authors": [
                "Gabriel G. Bes",
                "Lionel Lamadon",
                "Francois Trouilleux"
            ],
            "abstract": "A way of extracting French verbal chunks, inflected and infinitive, is\nexplored and tested on effective corpus. Declarative morphological and local\ngrammar rules specifying chunks and some simple contextual structures are used,\nrelying on limited lexical information and some simple heuristic/statistic\nproperties obtained from restricted corpora. The specific goals, the\narchitecture and the formalism of the system, the linguistic information on\nwhich it relies and the obtained results on effective corpus are presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408061v1",
            "date": "2004-08-26 13:17:38+00:00",
            "title": "An electronic dictionary as a basis for NLP tools: The Greek case",
            "authors": [
                "Ch. Tsalidis",
                "A. Vagelatos",
                "G. Orphanos"
            ],
            "abstract": "The existence of a Dictionary in electronic form for Modern Greek (MG) is\nmandatory if one is to process MG at the morphological and syntactic levels\nsince MG is a highly inflectional language with marked stress and a spelling\nsystem with many characteristics carried over from Ancient Greek. Moreover,\nsuch a tool becomes necessary if one is to create efficient and sophisticated\nNLP applications with substantial linguistic backing and coverage. The present\npaper will focus on the deployment of such an electronic dictionary for Modern\nGreek, which was built in two phases: first it was constructed to be the basis\nfor a spelling correction schema and then it was reconstructed in order to\nbecome the platform for the deployment of a wider spectrum of NLP tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0409008v1",
            "date": "2004-09-07 13:46:50+00:00",
            "title": "A Model for Fine-Grained Alignment of Multilingual Texts",
            "authors": [
                "Lea Cyrus",
                "Hendrik Feddes"
            ],
            "abstract": "While alignment of texts on the sentential level is often seen as being too\ncoarse, and word alignment as being too fine-grained, bi- or multilingual texts\nwhich are aligned on a level in-between are a useful resource for many\npurposes. Starting from a number of examples of non-literal translations, which\ntend to make alignment difficult, we describe an alignment model which copes\nwith these cases by explicitly coding them. The model is based on\npredicate-argument structures and thus covers the middle ground between\nsentence and word alignment. The model is currently used in a recently\ninitiated project of a parallel English-German treebank (FuSe), which can in\nprinciple be extended with additional languages.",
            "journal ref": "Proc. COLING 2004 Workshop on Multilingual Linguistic Resources\n  (MLR2004), Geneva, August 28, 2004, pp. 15-22",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0409058v1",
            "date": "2004-09-29 20:34:04+00:00",
            "title": "A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts",
            "authors": [
                "Bo Pang",
                "Lillian Lee"
            ],
            "abstract": "Sentiment analysis seeks to identify the viewpoint(s) underlying a text span;\nan example application is classifying a movie review as \"thumbs up\" or \"thumbs\ndown\". To determine this sentiment polarity, we propose a novel\nmachine-learning method that applies text-categorization techniques to just the\nsubjective portions of the document. Extracting these portions can be\nimplemented using efficient techniques for finding minimum cuts in graphs; this\ngreatly facilitates incorporation of cross-sentence contextual constraints.",
            "journal ref": "Proceedings of the 42nd ACL, pp. 271--278, 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412015v2",
            "date": "2004-12-03 17:10:17+00:00",
            "title": "A Tutorial on the Expectation-Maximization Algorithm Including Maximum-Likelihood Estimation and EM Training of Probabilistic Context-Free Grammars",
            "authors": [
                "Detlef Prescher"
            ],
            "abstract": "The paper gives a brief review of the expectation-maximization algorithm\n(Dempster 1977) in the comprehensible framework of discrete mathematics. In\nSection 2, two prominent estimation methods, the relative-frequency estimation\nand the maximum-likelihood estimation are presented. Section 3 is dedicated to\nthe expectation-maximization algorithm and a simpler variant, the generalized\nexpectation-maximization algorithm. In Section 4, two loaded dice are rolled. A\nmore interesting example is presented in Section 5: The estimation of\nprobabilistic context-free grammars.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412016v1",
            "date": "2004-12-03 18:10:17+00:00",
            "title": "Inside-Outside Estimation Meets Dynamic EM",
            "authors": [
                "Detlef Prescher"
            ],
            "abstract": "We briefly review the inside-outside and EM algorithm for probabilistic\ncontext-free grammars. As a result, we formally prove that inside-outside\nestimation is a dynamic-programming variant of EM. This is interesting in its\nown right, but even more when considered in a theoretical context since the\nwell-known convergence behavior of inside-outside estimation has been confirmed\nby many experiments but apparently has never been formally proved. However,\nbeing a version of EM, inside-outside estimation also inherits the good\nconvergence behavior of EM. Therefore, the as yet imperfect line of\nargumentation can be transformed into a coherent proof.",
            "journal ref": "Proceedings of IWPT 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412114v1",
            "date": "2004-12-29 15:19:03+00:00",
            "title": "State of the Art, Evaluation and Recommendations regarding \"Document Processing and Visualization Techniques\"",
            "authors": [
                "Martin Rajman",
                "Martin Vesely",
                "Pierre Andrews"
            ],
            "abstract": "Several Networks of Excellence have been set up in the framework of the\nEuropean FP5 research program. Among these Networks of Excellence, the NEMIS\nproject focuses on the field of Text Mining.\n  Within this field, document processing and visualization was identified as\none of the key topics and the WG1 working group was created in the NEMIS\nproject, to carry out a detailed survey of techniques associated with the text\nmining process and to identify the relevant research topics in related research\nareas.\n  In this document we present the results of this comprehensive survey. The\nreport includes a description of the current state-of-the-art and practice, a\nroadmap for follow-up research in the identified areas, and recommendations for\nanticipated technological development in the domain of text mining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412117v1",
            "date": "2004-12-30 02:01:45+00:00",
            "title": "Thematic Annotation: extracting concepts out of documents",
            "authors": [
                "Pierre Andrews",
                "Martin Rajman"
            ],
            "abstract": "Contrarily to standard approaches to topic annotation, the technique used in\nthis work does not centrally rely on some sort of -- possibly statistical --\nkeyword extraction. In fact, the proposed annotation algorithm uses a large\nscale semantic database -- the EDR Electronic Dictionary -- that provides a\nconcept hierarchy based on hyponym and hypernym relations. This concept\nhierarchy is used to generate a synthetic representation of the document by\naggregating the words present in topically homogeneous document segments into a\nset of concepts best preserving the document's content.\n  This new extraction technique uses an unexplored approach to topic selection.\nInstead of using semantic similarity measures based on a semantic resource, the\nlater is processed to extract the part of the conceptual hierarchy relevant to\nthe document content. Then this conceptual hierarchy is searched to extract the\nmost relevant set of concepts to represent the topics discussed in the\ndocument. Notice that this algorithm is able to extract generic concepts that\nare not directly present in the document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0403039v1",
            "date": "2004-03-23 21:06:11+00:00",
            "title": "A Flexible Rule Compiler for Speech Synthesis",
            "authors": [
                "Wojciech Skut",
                "Stefan Ulrich",
                "Kathrine Hammervold"
            ],
            "abstract": "We present a flexible rule compiler developed for a text-to-speech (TTS)\nsystem. The compiler converts a set of rules into a finite-state transducer\n(FST). The input and output of the FST are subject to parameterization, so that\nthe system can be applied to strings and sequences of feature-structures. The\nresulting transducer is guaranteed to realize a function (as opposed to a\nrelation), and therefore can be implemented as a deterministic device (either a\ndeterministic FST or a bimachine).",
            "journal ref": "In: Klopotek, Mieczyslaw A.; Wierzchon, Slawomir T.; Trojanowski,\n  Krzysztof (Eds.): \"Intelligent Information Processing and Web Mining -\n  Proceedings of the International IIS:IIPWM?04 Conference\"; Springer Verlag,\n  2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404006v1",
            "date": "2004-04-05 01:53:46+00:00",
            "title": "Delimited continuations in natural language: quantification and polarity sensitivity",
            "authors": [
                "Chung-chieh Shan"
            ],
            "abstract": "Making a linguistic theory is like making a programming language: one\ntypically devises a type system to delineate the acceptable utterances and a\ndenotational semantics to explain observations on their behavior. Via this\nconnection, the programming language concept of delimited continuations can\nhelp analyze natural language phenomena such as quantification and polarity\nsensitivity. Using a logical metalanguage whose syntax includes control\noperators and whose semantics involves evaluation order, these analyses can be\nexpressed in direct style rather than continuation-passing style, and these\nphenomena can be thought of as computational side effects.",
            "journal ref": "In CW'04: Proceedings of the 4th ACM SIGPLAN workshop on\n  continuations, ed. Hayo Thielecke, 55-64. Technical report CSR-04-1, School\n  of Computer Science, University of Birmingham (2004)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404018v1",
            "date": "2004-04-07 06:54:45+00:00",
            "title": "NLML--a Markup Language to Describe the Unlimited English Grammar",
            "authors": [
                "Jiyou Jia"
            ],
            "abstract": "In this paper we present NLML (Natural Language Markup Language), a markup\nlanguage to describe the syntactic and semantic structure of any grammatically\ncorrect English expression. At first the related works are analyzed to\ndemonstrate the necessity of the NLML: simple form, easy management and direct\nstorage. Then the description of the English grammar with NLML is introduced in\ndetails in three levels: sentences (with different complexities, voices, moods,\nand tenses), clause (relative clause and noun clause) and phrase (noun phrase,\nverb phrase, prepositional phrase, adjective phrase, adverb phrase and\npredicate phrase). At last the application fields of the NLML in NLP are shown\nwith two typical examples: NLOJM (Natural Language Object Modal in Java) and\nNLDB (Natural Language Database).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404026v1",
            "date": "2004-04-11 08:36:04+00:00",
            "title": "DAB Content Annotation and Receiver Hardware Control with XML",
            "authors": [
                "Darran Nathan",
                "Eva Rosdiana",
                "Chua Beng Koon"
            ],
            "abstract": "The Eureka-147 Digital Audio Broadcasting (DAB) standard defines the 'dynamic\nlabels' data field for holding information about the transmission content.\nHowever, this information does not follow a well-defined structure since it is\ndesigned to carry text for direct output to displays, for human interpretation.\nThis poses a problem when machine interpretation of DAB content information is\ndesired. Extensible Markup Language (XML) was developed to allow for the\nwell-defined, structured machine-to-machine exchange of data over computer\nnetworks. This article proposes a novel technique of machine-interpretable DAB\ncontent annotation and receiver hardware control, involving the utilisation of\nXML as metadata in the transmitted DAB frames.",
            "journal ref": "None",
            "category": "cs.GL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404041v2",
            "date": "2004-04-21 06:30:28+00:00",
            "title": "NLOMJ--Natural Language Object Model in Java",
            "authors": [
                "Jiyou Jia"
            ],
            "abstract": "In this paper we present NLOMJ--a natural language object model in Java with\nEnglish as the experiment language. This modal describes the grammar elements\nof any permissible expression in a natural language and their complicated\nrelations with each other with the concept \"Object\" in OOP(Object Oriented\nProgramming). Directly mapped to the syntax and semantics of the natural\nlanguage, it can be used in information retrieval as a linguistic method.\nAround the UML diagram of the NLOMJ the important classes(Sentence, Clause and\nPhrase) and their sub classes are introduced and their syntactic and semantic\nmeanings are explained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0404049v1",
            "date": "2004-04-23 10:44:24+00:00",
            "title": "Exploiting Cross-Document Relations for Multi-document Evolving Summarization",
            "authors": [
                "Stergos D. Afantenos",
                "Irene Doura",
                "Eleni Kapellou",
                "Vangelis Karkaletsis"
            ],
            "abstract": "This paper presents a methodology for summarization from multiple documents\nwhich are about a specific topic. It is based on the specification and\nidentification of the cross-document relations that occur among textual\nelements within those documents. Our methodology involves the specification of\nthe topic-specific entities, the messages conveyed for the specific entities by\ncertain textual elements and the specification of the relations that can hold\namong these messages. The above resources are necessary for setting up a\nspecific topic for our query-based summarization approach which uses these\nresources to identify the query-specific messages within the documents and the\nquery-specific relations that connect these messages across documents.",
            "journal ref": "Methods and Applications of Artificial Intelligence, Volume 3025\n  of Lecture Notes in Computer Science. Springer-Verlag Heidelberg 2004. pp\n  410-419.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0405044v1",
            "date": "2004-05-12 20:18:51+00:00",
            "title": "Corpus structure, language models, and ad hoc information retrieval",
            "authors": [
                "Oren Kurland",
                "Lillian Lee"
            ],
            "abstract": "Most previous work on the recently developed language-modeling approach to\ninformation retrieval focuses on document-specific characteristics, and\ntherefore does not take into account the structure of the surrounding corpus.\nWe propose a novel algorithmic framework in which information provided by\ndocument-based language models is enhanced by the incorporation of information\ndrawn from clusters of similar documents. Using this framework, we develop a\nsuite of new algorithms. Even the simplest typically outperforms the standard\nlanguage-modeling approach in precision and recall, and our new interpolation\nalgorithm posts statistically significant improvements for both metrics over\nall three corpora tested.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0406003v1",
            "date": "2004-06-02 18:51:52+00:00",
            "title": "Algorithms for weighted multi-tape automata",
            "authors": [
                "Andre Kempe",
                "Franck Guingne",
                "Florent Nicart"
            ],
            "abstract": "This report defines various operations for weighted multi-tape automata\n(WMTAs) and describes algorithms that have been implemented for those\noperations in the WFSC toolkit. Some algorithms are new, others are known or\nsimilar to known algorithms. The latter will be recalled to make this report\nmore complete and self-standing. We present a new approach to multi-tape\nintersection, meaning the intersection of a number of tapes of one WMTA with\nthe same number of tapes of another WMTA. In our approach, multi-tape\nintersection is not considered as an atomic operation but rather as a sequence\nof more elementary ones, which facilitates its implementation. We show an\nexample of multi-tape intersection, actually transducer intersection, that can\nbe compiled with our approach but not with several other methods that we\nanalysed. To show the practical relavance of our work, we include an example of\napplication: the preservation of intermediate results in transduction cascades.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0406015v1",
            "date": "2004-06-07 17:52:07+00:00",
            "title": "Zipf's law and the creation of musical context",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "This article discusses the extension of the notion of context from\nlinguistics to the domain of music. In language, the statistical regularity\nknown as Zipf's law -which concerns the frequency of usage of different words-\nhas been quantitatively related to the process of text generation. This\nconnection is established by Simon's model, on the basis of a few assumptions\nregarding the accompanying creation of context. Here, it is shown that the\nstatistics of note usage in musical compositions are compatible with the\npredictions of Simon's model. This result, which gives objective support to the\nconceptual likeness of context in language and music, is obtained through\nautomatic analysis of the digital versions of several compositions. As a\nby-product, a quantitative measure of context definiteness is introduced and\nused to compare tonal and atonal works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408026v1",
            "date": "2004-08-10 11:09:48+00:00",
            "title": "Incremental Construction of Minimal Acyclic Sequential Transducers from Unsorted Data",
            "authors": [
                "Wojciech Skut"
            ],
            "abstract": "This paper presents an efficient algorithm for the incremental construction\nof a minimal acyclic sequential transducer (ST) for a dictionary consisting of\na list of input and output strings. The algorithm generalises a known method of\nconstructing minimal finite-state automata (Daciuk et al. 2000). Unlike the\nalgorithm published by Mihov and Maurel (2001), it does not require the input\nstrings to be sorted. The new method is illustrated by an application to\npronunciation dictionaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408027v1",
            "date": "2004-08-12 11:15:17+00:00",
            "title": "CHR Grammars",
            "authors": [
                "Henning Christiansen"
            ],
            "abstract": "A grammar formalism based upon CHR is proposed analogously to the way\nDefinite Clause Grammars are defined and implemented on top of Prolog. These\ngrammars execute as robust bottom-up parsers with an inherent treatment of\nambiguity and a high flexibility to model various linguistic phenomena. The\nformalism extends previous logic programming based grammars with a form of\ncontext-sensitive rules and the possibility to include extra-grammatical\nhypotheses in both head and body of grammar rules. Among the applications are\nstraightforward implementations of Assumption Grammars and abduction under\nintegrity constraints for language analysis. CHR grammars appear as a powerful\ntool for specification and implementation of language processors and may be\nproposed as a new standard for bottom-up grammars in logic programming.\n  To appear in Theory and Practice of Logic Programming (TPLP), 2005",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408041v1",
            "date": "2004-08-17 10:14:22+00:00",
            "title": "Fractal geometry of literature: first attempt to Shakespeare's works",
            "authors": [
                "Ali Eftekhari"
            ],
            "abstract": "It was demonstrated that there is a geometrical order in the structure of\nliterature. Fractal geometry as a modern mathematical approach and a new\ngeometrical viewpoint on natural objects including both processes and\nstructures was employed for analysis of literature. As the first study, the\nworks of William Shakespeare were chosen as the most important items in western\nliterature. By counting the number of letters applied in a manuscript, it is\npossible to study the whole manuscript statistically. A novel method based on\nbasic assumption of fractal geometry was proposed for the calculation of\nfractal dimensions of the literature. The results were compared with Zipf's\nlaw. Zipf's law was successfully used for letters instead of words. Two new\nconcepts namely Zipf's dimension and Zipf's order were also introduced. It was\nfound that changes of both fractal dimension and Zipf's dimension are similar\nand dependent on the manuscript length. Interestingly, direct plotting the data\nobtained in semi-logarithmic and logarithmic forms also led to a power-law.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0409042v1",
            "date": "2004-09-23 19:29:26+00:00",
            "title": "A new architecture for making highly scalable applications",
            "authors": [
                "Harry Fiti\u00e9"
            ],
            "abstract": "An application is a logical image of the world on a computer. A scalable\napplication is an application that allows one to update that logical image at\nrun time. To put it in operational terms: an application is scalable if a\nclient can change between time T1 and time T2 - the logic of the application as\nexpressed by language L;\n  - the structure and volume of the stored knowledge;\n  - the user interface of the application; while clients working with the\napplication at time T1 will work with the changed application at time T2\nwithout performing any special action between T1 and T2. In order to realize\nsuch a scalable application a new architecture has been developed that fully\norbits around language. In order to verify the soundness of that architecture a\nprogram has been build. Both architecture and program are called CommunSENS.\nThe main purpose of this paper is: - to list the relevant elements of the\narchitecture; - to give a visual presentation of how the program and its image\nof the world look like; - to give a visual presentation of how the image can be\nupdated. Some relevant philosophical and practical backgrounds are included in\nthe appendixes.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410063v1",
            "date": "2004-10-25 02:17:35+00:00",
            "title": "INSPIRE: Evaluation of a Smart-Home System for Infotainment Management and Device Control",
            "authors": [
                "Sebastian Moeller",
                "Jan Krebber",
                "Alexander Raake",
                "Paula Smeele",
                "Martin Rajman",
                "Mirek Melichar",
                "Vincenzo Pallotta",
                "Gianna Tsakou",
                "Basilis Kladis",
                "Anestis Vovos",
                "Jettie Hoonhout",
                "Dietmar Schuchardt",
                "Nikos Fakotakis",
                "Todor Ganchev",
                "Ilyas Potamitis"
            ],
            "abstract": "This paper gives an overview of the assessment and evaluation methods which\nhave been used to determine the quality of the INSPIRE smart home system. The\nsystem allows different home appliances to be controlled via speech, and\nconsists of speech and speaker recognition, speech understanding, dialogue\nmanagement, and speech output components. The performance of these components\nis first assessed individually, and then the entire system is evaluated in an\ninteraction experiment with test users. Initial results of the assessment and\nevaluation are given, in particular with respect to the transmission channel\nimpact on speech and speaker recognition, and the assessment of speech output\nfor different system metaphors.",
            "journal ref": "Procedings of the LREC 2004 international conference, 26-28 May\n  2004, Lisbon, Portugal. Pages 1603-1606",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410072v1",
            "date": "2004-10-27 17:21:52+00:00",
            "title": "Temporal logic with predicate abstraction",
            "authors": [
                "Alexei Lisitsa",
                "Igor Potapov"
            ],
            "abstract": "A predicate linear temporal logic LTL_{\\lambda,=} without quantifiers but\nwith predicate abstraction mechanism and equality is considered. The models of\nLTL_{\\lambda,=} can be naturally seen as the systems of pebbles (flexible\nconstants) moving over the elements of some (possibly infinite) domain. This\nallows to use LTL_{\\lambda,=} for the specification of dynamic systems using\nsome resources, such as processes using memory locations, mobile agents\noccupying some sites, etc. On the other hand we show that LTL_{\\lambda,=} is\nnot recursively axiomatizable and, therefore, fully automated verification of\nLTL_{\\lambda,=} specifications is not, in general, possible.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/cs/0411074v1",
            "date": "2004-11-20 00:22:30+00:00",
            "title": "Building Chinese Lexicons from Scratch by Unsupervised Short Document Self-Segmentation",
            "authors": [
                "Daniel Gayo-Avello"
            ],
            "abstract": "Chinese text segmentation is a well-known and difficult problem. On one side,\nthere is not a simple notion of \"word\" in Chinese language making really hard\nto implement rule-based systems to segment written texts, thus lexicons and\nstatistical information are usually employed to achieve such a task. On the\nother side, any piece of Chinese text usually includes segments present neither\nin the lexicons nor in the training data. Even worse, such unseen sequences can\nbe segmented into a number of totally unrelated words making later processing\nphases difficult. For instance, using a lexicon-based system the sequence\n???(Baluozuo, Barroso, current president-designate of the European Commission)\ncan be segmented into ?(ba, to hope, to wish) and ??(luozuo, an undefined word)\nchanging completely the meaning of the sentence. A new and extremely simple\nalgorithm specially suited to work over short Chinese documents is introduced.\nThis new algorithm performs text \"self-segmentation\" producing results\ncomparable to those achieved by native speakers without using either lexicons\nor any statistical information beyond the obtained from the input text.\nFurthermore, it is really robust for finding new \"words\", especially proper\nnouns, and it is well suited to build lexicons from scratch. Some preliminary\nresults are provided in addition to examples of its employment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412065v1",
            "date": "2004-12-17 03:24:51+00:00",
            "title": "A Framework for Creating Natural Language User Interfaces for Action-Based Applications",
            "authors": [
                "Stephen Chong",
                "Riccardo Pucella"
            ],
            "abstract": "In this paper we present a framework for creating natural language interfaces\nto action-based applications. Our framework uses a number of reusable\napplication-independent components, in order to reduce the effort of creating a\nnatural language interface for a given application. Using a type-logical\ngrammar, we first translate natural language sentences into expressions in an\nextended higher-order logic. These expressions can be seen as executable\nspecifications corresponding to the original sentences. The executable\nspecifications are then interpreted by invoking appropriate procedures provided\nby the application for which a natural language interface is being created.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0407065v1",
            "date": "2004-07-29 19:46:01+00:00",
            "title": "Word Sense Disambiguation by Web Mining for Word Co-occurrence Probabilities",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "This paper describes the National Research Council (NRC) Word Sense\nDisambiguation (WSD) system, as applied to the English Lexical Sample (ELS)\ntask in Senseval-3. The NRC system approaches WSD as a classical supervised\nmachine learning problem, using familiar tools such as the Weka machine\nlearning software and Brill's rule-based part-of-speech tagger. Head words are\nrepresented as feature vectors with several hundred features. Approximately\nhalf of the features are syntactic and the other half are semantic. The main\nnovelty in the system is the method for generating the semantic features, based\non word \\hbox{co-occurrence} probabilities. The probabilities are estimated\nusing the Waterloo MultiText System with a corpus of about one terabyte of\nunlabeled text, collected by a web crawler.",
            "journal ref": "Proceedings of the Third International Workshop on the Evaluation\n  of Systems for the Semantic Analysis of Text (SENSEVAL-3), (2004), Barcelona,\n  Spain, 239-242",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408037v1",
            "date": "2004-08-15 08:51:19+00:00",
            "title": "Multi-dimensional Type Theory: Rules, Categories, and Combinators for Syntax and Semantics",
            "authors": [
                "J\u00f8rgen Villadsen"
            ],
            "abstract": "We investigate the possibility of modelling the syntax and semantics of\nnatural language by constraints, or rules, imposed by the multi-dimensional\ntype theory Nabla. The only multiplicity we explicitly consider is two, namely\none dimension for the syntax and one dimension for the semantics, but the\ngeneral perspective is important. For example, issues of pragmatics could be\nhandled as additional dimensions.\n  One of the main problems addressed is the rather complicated repertoire of\noperations that exists besides the notion of categories in traditional Montague\ngrammar. For the syntax we use a categorial grammar along the lines of Lambek.\nFor the semantics we use so-called lexical and logical combinators inspired by\nwork in natural logic. Nabla provides a concise interpretation and a sequent\ncalculus as the basis for implementations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0408057v1",
            "date": "2004-08-25 19:37:59+00:00",
            "title": "The role of robust semantic analysis in spoken language dialogue systems",
            "authors": [
                "Afzal Ballim",
                "Vincenzo Pallotta"
            ],
            "abstract": "In this paper we summarized a framework for designing grammar-based procedure\nfor the automatic extraction of the semantic content from spoken queries.\nStarting with a case study and following an approach which combines the notions\nof fuzziness and robustness in sentence parsing, we showed we built practical\ndomain-dependent rules which can be applied whenever it is possible to\nsuperimpose a sentence-level semantic structure to a text without relying on a\nprevious deep syntactical analysis. This kind of procedure can be also\nprofitably used as a pre-processing tool in order to cut out part of the\nsentence which have been recognized to have no relevance in the understanding\nprocess. In the case of particular dialogue applications where there is no need\nto build a complex semantic structure (e.g. word spotting or excerpting) the\npresented methodology may represent an efficient alternative solution to a\nsequential composition of deep linguistic analysis modules. Even if the query\ngeneration problem may not seem a critical application it should be held in\nmind that the sentence processing must be done on-line. Having this kind of\nconstraints we cannot design our system without caring for efficiency and thus\nprovide an immediate response. Another critical issue is related to whole\nrobustness of the system. In our case study we tried to make experiences on how\nit is possible to deal with an unreliable and noisy input without asking the\nuser for any repetition or clarification. This may correspond to a similar\nproblem one may have when processing text coming from informal writing such as\ne-mails, news and in many cases Web pages where it is often the case to have\nirrelevant surrounding information.",
            "journal ref": "Proceedings of the 3rd International Workshop on Human-Computer\n  Conversation, July 3-5, 2000, Bellagio, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410027v1",
            "date": "2004-10-13 02:28:10+00:00",
            "title": "Detecting User Engagement in Everyday Conversations",
            "authors": [
                "Chen Yu",
                "Paul M. Aoki",
                "Allison Woodruff"
            ],
            "abstract": "This paper presents a novel application of speech emotion recognition:\nestimation of the level of conversational engagement between users of a voice\ncommunication system. We begin by using machine learning techniques, such as\nthe support vector machine (SVM), to classify users' emotions as expressed in\nindividual utterances. However, this alone fails to model the temporal and\ninteractive aspects of conversational engagement. We therefore propose the use\nof a multilevel structure based on coupled hidden Markov models (HMM) to\nestimate engagement levels in continuous natural speech. The first level is\ncomprised of SVM-based classifiers that recognize emotional states, which could\nbe (e.g.) discrete emotion types or arousal/valence levels. A high-level HMM\nthen uses these emotional states as input, estimating users' engagement in\nconversation by decoding the internal states of the HMM. We report experimental\nresults obtained by applying our algorithms to the LDC Emotional Prosody and\nCallFriend speech corpora.",
            "journal ref": "Proc. 8th Int'l Conf. on Spoken Language Processing (ICSLP) (Vol.\n  2), Jeju Island, Republic of Korea, Oct. 2004, 1329-1332. ISCA.",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410059v1",
            "date": "2004-10-23 00:06:19+00:00",
            "title": "A knowledge-based approach to semi-automatic annotation of multimedia documents via user adaptation",
            "authors": [
                "Afzal Ballim",
                "Nastaran Fatemi",
                "Hatem Ghorbel",
                "Vincenzo Pallotta"
            ],
            "abstract": "Current approaches to the annotation process focus on annotation schemas,\nlanguages for annotation, or are very application driven. In this paper it is\nproposed that a more flexible architecture for annotation requires a knowledge\ncomponent to allow for flexible search and navigation of the annotated\nmaterial. In particular, it is claimed that a general approach must take into\naccount the needs, competencies, and goals of the producers, annotators, and\nconsumers of the annotated material. We propose that a user-model based\napproach is, therefore, necessary.",
            "journal ref": "First EAGLES/ISLE Workshop on Meta-Descriptions and Annotation\n  Schemas for Multimodal/Multimedia Language Resources (LREC 2000\n  Pre-Conference Workshop), 29-30 May 2000, Athens, Greece",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410060v1",
            "date": "2004-10-23 00:20:06+00:00",
            "title": "Semantic filtering by inference on domain knowledge in spoken dialogue systems",
            "authors": [
                "Afzal Ballim",
                "Vincenzo Pallotta"
            ],
            "abstract": "General natural dialogue processing requires large amounts of domain\nknowledge as well as linguistic knowledge in order to ensure acceptable\ncoverage and understanding. There are several ways of integrating lexical\nresources (e.g. dictionaries, thesauri) and knowledge bases or ontologies at\ndifferent levels of dialogue processing. We concentrate in this paper on how to\nexploit domain knowledge for filtering interpretation hypotheses generated by a\nrobust semantic parser. We use domain knowledge to semantically constrain the\nhypothesis space. Moreover, adding an inference mechanism allows us to complete\nthe interpretation when information is not explicitly available. Further, we\ndiscuss briefly how this can be generalized towards a predictive natural\ninteractive system.",
            "journal ref": "Proceedings of the LREC 2000 Workshop \"From spoken dialogue to\n  full natural interactive dialogue. Theory, empirical analysis and\n  evaluation\", May 29th, 2000 Athen, Greece",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410061v1",
            "date": "2004-10-25 01:38:07+00:00",
            "title": "An argumentative annotation schema for meeting discussions",
            "authors": [
                "Vincenzo Pallotta",
                "Hatem Ghorbel",
                "Patrick Ruch",
                "Giovanni Coray"
            ],
            "abstract": "In this article, we are interested in the annotation of transcriptions of\nhuman-human dialogue taken from meeting records. We first propose a meeting\ncontent model where conversational acts are interpreted with respect to their\nargumentative force and their role in building the argumentative structure of\nthe meeting discussion. Argumentation in dialogue describes the way\nparticipants take part in the discussion and argue their standpoints. Then, we\npropose an annotation scheme based on such an argumentative dialogue model as\nwell as the evaluation of its adequacy. The obtained higher-level semantic\nannotations are exploited in the conceptual indexing of the information\ncontained in meeting discussions.",
            "journal ref": "Procedings of the LREC 2004 international conference, 26-28 May\n  2004, Lisbon, Portugal. Pages 1003-1006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410062v1",
            "date": "2004-10-25 01:50:03+00:00",
            "title": "Automatic Keyword Extraction from Spoken Text. A Comparison of two Lexical Resources: the EDR and WordNet",
            "authors": [
                "Lonneke van der Plas",
                "Vincenzo Pallotta",
                "Martin Rajman",
                "Hatem Ghorbel"
            ],
            "abstract": "Lexical resources such as WordNet and the EDR electronic dictionary have been\nused in several NLP tasks. Probably, partly due to the fact that the EDR is not\nfreely available, WordNet has been used far more often than the EDR. We have\nused both resources on the same task in order to make a comparison possible.\nThe task is automatic assignment of keywords to multi-party dialogue episodes\n(i.e. thematically coherent stretches of spoken text). We show that the use of\nlexical resources in such a task results in slightly higher performances than\nthe use of a purely statistically based method.",
            "journal ref": "Procedings of the LREC 2004 international conference, 26-28 May\n  2004, Lisbon, Portugal. Pages 2205-2208",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412024v1",
            "date": "2004-12-06 21:50:18+00:00",
            "title": "Human-Level Performance on Word Analogy Questions by Latent Relational Analysis",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "This paper introduces Latent Relational Analysis (LRA), a method for\nmeasuring relational similarity. LRA has potential applications in many areas,\nincluding information extraction, word sense disambiguation, machine\ntranslation, and information retrieval. Relational similarity is correspondence\nbetween relations, in contrast with attributional similarity, which is\ncorrespondence between attributes. When two words have a high degree of\nattributional similarity, we call them synonyms. When two pairs of words have a\nhigh degree of relational similarity, we say that their relations are\nanalogous. For example, the word pair mason/stone is analogous to the pair\ncarpenter/wood. Past work on semantic similarity measures has mainly been\nconcerned with attributional similarity. Recently the Vector Space Model (VSM)\nof information retrieval has been adapted to the task of measuring relational\nsimilarity, achieving a score of 47% on a collection of 374 college-level\nmultiple-choice word analogy questions. In the VSM approach, the relation\nbetween a pair of words is characterized by a vector of frequencies of\npredefined patterns in a large corpus. LRA extends the VSM approach in three\nways: (1) the patterns are derived automatically from the corpus (they are not\npredefined), (2) the Singular Value Decomposition (SVD) is used to smooth the\nfrequency data (it is also used this way in Latent Semantic Analysis), and (3)\nautomatically generated synonyms are used to explore reformulations of the word\npairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent\nto the average human score of 57%. On the related problem of classifying\nnoun-modifier relations, LRA achieves similar gains over the VSM, while using a\nsmaller corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cond-mat/0403233v2",
            "date": "2004-03-09 09:56:19+00:00",
            "title": "Artificial Sequences and Complexity Measures",
            "authors": [
                "Andrea Baronchelli",
                "Emanuele Caglioti",
                "Vittorio Loreto"
            ],
            "abstract": "In this paper we exploit concepts of information theory to address the\nfundamental problem of identifying and defining the most suitable tools to\nextract, in a automatic and agnostic way, information from a generic string of\ncharacters. We introduce in particular a class of methods which use in a\ncrucial way data compression techniques in order to define a measure of\nremoteness and distance between pairs of sequences of characters (e.g. texts)\nbased on their relative information content. We also discuss in detail how\nspecific features of data compression techniques could be used to introduce the\nnotion of dictionary of a given sequence and of Artificial Text and we show how\nthese new tools can be used for information extraction purposes. We point out\nthe versatility and generality of our method that applies to any kind of\ncorpora of character strings independently of the type of coding behind them.\nWe consider as a case study linguistic motivated problems and we present\nresults for automatic language recognition, authorship attribution and self\nconsistent-classification.",
            "journal ref": "J. Stat. Mech., P04002 (2005)",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410058v1",
            "date": "2004-10-22 23:41:52+00:00",
            "title": "Robust Dialogue Understanding in HERALD",
            "authors": [
                "Vincenzo Pallotta",
                "Afzal Ballim"
            ],
            "abstract": "We tackle the problem of robust dialogue processing from the perspective of\nlanguage engineering. We propose an agent-oriented architecture that allows us\na flexible way of composing robust processors. Our approach is based on\nShoham's Agent Oriented Programming (AOP) paradigm. We will show how the AOP\nagent model can be enriched with special features and components that allow us\nto deal with classical problems of dialogue understanding.",
            "journal ref": "Proceedings of RANLP 2001 - EuroConference on Recent Advances in\n  Natural Language Processing, September 5-7, 2001, Tzigov-Chark, Bulgaria",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0412098v3",
            "date": "2004-12-21 16:05:36+00:00",
            "title": "The Google Similarity Distance",
            "authors": [
                "Rudi Cilibrasi",
                "Paul M. B. Vitanyi"
            ],
            "abstract": "Words and phrases acquire meaning from the way they are used in society, from\ntheir relative semantics to other words and phrases. For computers the\nequivalent of `society' is `database,' and the equivalent of `use' is `way to\nsearch the database.' We present a new theory of similarity between words and\nphrases based on information distance and Kolmogorov complexity. To fix\nthoughts we use the world-wide-web as database, and Google as search engine.\nThe method is also applicable to other search engines and databases. This\ntheory is then applied to construct a method to automatically extract\nsimilarity, the Google similarity distance, of words and phrases from the\nworld-wide-web using Google page counts. The world-wide-web is the largest\ndatabase on earth, and the context information entered by millions of\nindependent users averages out to provide automatic semantics of useful\nquality. We give applications in hierarchical clustering, classification, and\nlanguage translation. We give examples to distinguish between colors and\nnumbers, cluster names of paintings by 17th century Dutch masters and names of\nbooks by English novelists, the ability to understand emergencies, and primes,\nand we demonstrate the ability to do a simple automatic English-Spanish\ntranslation. Finally, we use the WordNet database as an objective baseline\nagainst which to judge the performance of our method. We conduct a massive\nrandomized trial in binary classification using support vector machines to\nlearn categories based on our Google distance, resulting in an a mean agreement\nof 87% with the expert crafted WordNet categories.",
            "journal ref": "R.L. Cilibrasi, P.M.B. Vitanyi, The Google Similarity Distance,\n  IEEE Trans. Knowledge and Data Engineering, 19:3(2007), 370-383",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0410017v1",
            "date": "2004-10-07 17:20:56+00:00",
            "title": "Automated Pattern Detection--An Algorithm for Constructing Optimally Synchronizing Multi-Regular Language Filters",
            "authors": [
                "Carl S. McTague",
                "James P. Crutchfield"
            ],
            "abstract": "In the computational-mechanics structural analysis of one-dimensional\ncellular automata the following automata-theoretic analogue of the\n\\emph{change-point problem} from time series analysis arises: \\emph{Given a\nstring $\\sigma$ and a collection $\\{\\mc{D}_i\\}$ of finite automata, identify\nthe regions of $\\sigma$ that belong to each $\\mc{D}_i$ and, in particular, the\nboundaries separating them.} We present two methods for solving this\n\\emph{multi-regular language filtering problem}. The first, although providing\nthe ideal solution, requires a stack, has a worst-case compute time that grows\nquadratically in $\\sigma$'s length and conditions its output at any point on\narbitrarily long windows of future input. The second method is to\nalgorithmically construct a transducer that approximates the first algorithm.\nIn contrast to the stack-based algorithm, however, the transducer requires only\na finite amount of memory, runs in linear time, and gives immediate output for\neach letter read; it is, moreover, the best possible finite-state approximation\nwith these three features.",
            "journal ref": "None",
            "category": "cs.CV"
        }
    ],
    "2005": [
        {
            "url": "http://arxiv.org/abs/cs/0501078v1",
            "date": "2005-01-26 22:43:17+00:00",
            "title": "Multi-document Biography Summarization",
            "authors": [
                "Liang Zhou",
                "Miruna Ticrea",
                "Eduard Hovy"
            ],
            "abstract": "In this paper we describe a biography summarization system using sentence\nclassification and ideas from information retrieval. Although the individual\ntechniques are not new, assembling and applying them to generate multi-document\nbiographies is new. Our system was evaluated in DUC2004. It is among the top\nperformers in task 5-short summaries focused by person questions.",
            "journal ref": "Proceedings of EMNLP, pp. 434-441, 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0504022v1",
            "date": "2005-04-06 20:04:55+00:00",
            "title": "A Matter of Opinion: Sentiment Analysis and Business Intelligence (position paper)",
            "authors": [
                "Lillian Lee"
            ],
            "abstract": "A general-audience introduction to the area of \"sentiment analysis\", the\ncomputational treatment of subjective, opinion-oriented language (an example\napplication is determining whether a review is \"thumbs up\" or \"thumbs down\").\nSome challenges, applications to business-intelligence tasks, and potential\nfuture directions are described.",
            "journal ref": "Presented at the IBM Faculty Summit on the Architecture of\n  On-Demand Business, May 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0510015v1",
            "date": "2005-10-05 14:23:19+00:00",
            "title": "Word sense disambiguation criteria: a systematic study",
            "authors": [
                "Laurent Audibert"
            ],
            "abstract": "This article describes the results of a systematic in-depth study of the\ncriteria used for word sense disambiguation. Our study is based on 60 target\nwords: 20 nouns, 20 adjectives and 20 verbs. Our results are not always in line\nwith some practices in the field. For example, we show that omitting\nnon-content words decreases performance and that bigrams yield better results\nthan unigrams.",
            "journal ref": "20th International Conference on Computational Linguistics\n  (COLING-2004) (2004) pp. 910-916",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0511076v1",
            "date": "2005-11-21 14:50:52+00:00",
            "title": "Using phonetic constraints in acoustic-to-articulatory inversion",
            "authors": [
                "Blaise Potard",
                "Yves Laprie"
            ],
            "abstract": "The goal of this work is to recover articulatory information from the speech\nsignal by acoustic-to-articulatory inversion. One of the main difficulties with\ninversion is that the problem is underdetermined and inversion methods\ngenerally offer no guarantee on the phonetical realism of the inverse\nsolutions. A way to adress this issue is to use additional phonetic\nconstraints. Knowledge of the phonetic caracteristics of French vowels enable\nthe derivation of reasonable articulatory domains in the space of Maeda\nparameters: given the formants frequencies (F1,F2,F3) of a speech sample, and\nthus the vowel identity, an \"ideal\" articulatory domain can be derived. The\nspace of formants frequencies is partitioned into vowels, using either\nspeaker-specific data or generic information on formants. Then, to each\narticulatory vector can be associated a phonetic score varying with the\ndistance to the \"ideal domain\" associated with the corresponding vowel.\nInversion experiments were conducted on isolated vowels and vowel-to-vowel\ntransitions. Articulatory parameters were compared with those obtained without\nusing these constraints and those measured from X-ray data.",
            "journal ref": "Proceedings of Interspeech, 9th European Conference on Speech\n  Communication and Technology (2005) 3217-3220",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0511079v1",
            "date": "2005-11-22 07:06:43+00:00",
            "title": "An elitist approach for extracting automatically well-realized speech sounds with high confidence",
            "authors": [
                "Jean-Baptiste Maj",
                "Anne Bonneau",
                "Dominique Fohr",
                "Yves Laprie"
            ],
            "abstract": "This paper presents an \"elitist approach\" for extracting automatically\nwell-realized speech sounds with high confidence. The elitist approach uses a\nspeech recognition system based on Hidden Markov Models (HMM). The HMM are\ntrained on speech sounds which are systematically well-detected in an iterative\nprocedure. The results show that, by using the HMM models defined in the\ntraining phase, the speech recognizer detects reliably specific speech sounds\nwith a small rate of errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0512102v1",
            "date": "2005-12-28 13:45:54+00:00",
            "title": "Statistical Parameters of the Novel \"Perekhresni stezhky\" (\"The Cross-Paths\") by Ivan Franko",
            "authors": [
                "Solomija Buk",
                "Andrij Rovenchak"
            ],
            "abstract": "In the paper, a complex statistical characteristics of a Ukrainian novel is\ngiven for the first time. The distribution of word-forms with respect to their\nsize is studied. The linguistic laws by Zipf-Mandelbrot and Altmann-Menzerath\nare analyzed.",
            "journal ref": "Quantitative Linguistics 62: Exact methods in the study of\n  language and text: dedicated to Professor Gabriel Altmann on the occasion of\n  his 75th birthday / Ed. by P. Grzybek and R. Kohler (Berlin; New York: de\n  Gruyter), 39-48 (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0501077v2",
            "date": "2005-01-26 14:11:38+00:00",
            "title": "Ontology-Based Users & Requests Clustering in Customer Service Management System",
            "authors": [
                "Alexander Smirnov",
                "Mikhail Pashkin",
                "Nikolai Chilov",
                "Tatiana Levashova",
                "Andrew Krizhanovsky",
                "Alexey Kashevnik"
            ],
            "abstract": "Customer Service Management is one of major business activities to better\nserve company customers through the introduction of reliable processes and\nprocedures. Today this kind of activities is implemented through e-services to\ndirectly involve customers into business processes. Traditionally Customer\nService Management involves application of data mining techniques to discover\nusage patterns from the company knowledge memory. Hence grouping of\ncustomers/requests to clusters is one of major technique to improve the level\nof company customization. The goal of this paper is to present an efficient for\nimplementation approach for clustering users and their requests. The approach\nuses ontology as knowledge representation model to improve the semantic\ninteroperability between units of the company and customers. Some fragments of\nthe approach tested in an industrial company are also presented in the paper.",
            "journal ref": "Smirnov A., Pashkin M., Chilov N., Levashova T., Krizhanovsky A.,\n  Kashevnik A. 2005. Ontology-Based Users and Requests Clustering in Customer\n  Service Management System. Springer-Verlag GmbH, Lecture Notes in Computer\n  Science, 3505: 231-246",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0503024v1",
            "date": "2005-03-10 11:49:51+00:00",
            "title": "Fine-Grained Word Sense Disambiguation Based on Parallel Corpora, Word Alignment, Word Clustering and Aligned Wordnets",
            "authors": [
                "Dan Tufis",
                "Radu Ion",
                "Nancy Ide"
            ],
            "abstract": "The paper presents a method for word sense disambiguation based on parallel\ncorpora. The method exploits recent advances in word alignment and word\nclustering based on automatic extraction of translation equivalents and being\nsupported by available aligned wordnets for the languages in the corpus. The\nwordnets are aligned to the Princeton Wordnet, according to the principles\nestablished by EuroWordNet. The evaluation of the WSD system, implementing the\nmethod described herein showed very encouraging results. The same system used\nin a validation mode, can be used to check and spot alignment errors in\nmultilingually aligned wordnets as BalkaNet and EuroWordNet.",
            "journal ref": "In proceedings of the 20th International Conference on\n  Computational Linguistics, COLING2004, Geneva, 2004, pp. 1312-1318",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0503030v2",
            "date": "2005-03-14 18:12:03+00:00",
            "title": "A Suffix Tree Approach to Email Filtering",
            "authors": [
                "Rajesh M. Pampapathi",
                "Boris Mirkin",
                "Mark Levene"
            ],
            "abstract": "We present an approach to email filtering based on the suffix tree data\nstructure. A method for the scoring of emails using the suffix tree is\ndeveloped and a number of scoring and score normalisation functions are tested.\nOur results show that the character level representation of emails and classes\nfacilitated by the suffix tree can significantly improve classification\naccuracy when compared with the currently popular methods, such as naive Bayes.\nWe believe the method can be extended to the classification of documents in\nother domains.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0503033v1",
            "date": "2005-03-15 15:31:14+00:00",
            "title": "An Introduction to the Summarization of Evolving Events: Linear and Non-linear Evolution",
            "authors": [
                "Stergos D. Afantenos",
                "Konstantina Liontou",
                "Maria Salapata",
                "Vangelis Karkaletsis"
            ],
            "abstract": "This paper examines the summarization of events that evolve through time. It\ndiscusses different types of evolution taking into account the time in which\nthe incidents of an event are happening and the different sources reporting on\nthe specific event. It proposes an approach for multi-document summarization\nwhich employs ``messages'' for representing the incidents of an event and\ncross-document relations that hold between messages according to certain\nconditions. The paper also outlines the current version of the summarization\nsystem we are implementing to realize this approach.",
            "journal ref": "Edited by Bernadete Sharp, Proceedings of the 2nd International\n  Workshop on Natural Language Understanding and Cognitive Science, NLUCS 2005.\n  Maiami, Florida, USA: INSTICC Press. pp 91-99.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0503077v1",
            "date": "2005-03-29 04:59:50+00:00",
            "title": "Weighted Automata in Text and Speech Processing",
            "authors": [
                "Mehryar Mohri",
                "Fernando Pereira",
                "Michael Riley"
            ],
            "abstract": "Finite-state automata are a very effective tool in natural language\nprocessing. However, in a variety of applications and especially in speech\nprecessing, it is necessary to consider more general machines in which arcs are\nassigned weights or costs. We briefly describe some of the main theoretical and\nalgorithmic aspects of these machines. In particular, we describe an efficient\ncomposition algorithm for weighted transducers, and give examples illustrating\nthe value of determinization and minimization algorithms for weighted automata.",
            "journal ref": "Mehryar Mohri, Fernando Pereira, and Michael Riley. Weighted\n  Automata in Text and Speech Processing. In Proceedings of the 12th biennial\n  European Conference on Artificial Intelligence (ECAI-96), Workshop on\n  Extended finite state models of language. Budapest, Hungary, 1996. John Wiley\n  and Sons, Chichester",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0504061v1",
            "date": "2005-04-13 20:02:25+00:00",
            "title": "Summarization from Medical Documents: A Survey",
            "authors": [
                "Stergos D. Afantenos",
                "Vangelis Karkaletsis",
                "Panagiotis Stamatopoulos"
            ],
            "abstract": "Objective:\n  The aim of this paper is to survey the recent work in medical documents\nsummarization.\n  Background:\n  During the last decade, documents summarization got increasing attention by\nthe AI research community. More recently it also attracted the interest of the\nmedical research community as well, due to the enormous growth of information\nthat is available to the physicians and researchers in medicine, through the\nlarge and growing number of published journals, conference proceedings, medical\nsites and portals on the World Wide Web, electronic medical records, etc.\n  Methodology:\n  This survey gives first a general background on documents summarization,\npresenting the factors that summarization depends upon, discussing evaluation\nissues and describing briefly the various types of summarization techniques. It\nthen examines the characteristics of the medical domain through the different\ntypes of medical documents. Finally, it presents and discusses the\nsummarization techniques used so far in the medical domain, referring to the\ncorresponding systems and their characteristics.\n  Discussion and conclusions:\n  The paper discusses thoroughly the promising paths for future research in\nmedical documents summarization. It mainly focuses on the issue of scaling to\nlarge collections of documents in various languages and from different media,\non personalization issues, on portability to new sub-domains, and on the\nintegration of summarization technology in practical applications",
            "journal ref": "Artificial Intelligence in Medicine, Volume 33, Issue 2, February\n  2005, Pages 157-177",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0506075v1",
            "date": "2005-06-17 20:10:43+00:00",
            "title": "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales",
            "authors": [
                "Bo Pang",
                "Lillian Lee"
            ],
            "abstract": "We address the rating-inference problem, wherein rather than simply decide\nwhether a review is \"thumbs up\" or \"thumbs down\", as in previous sentiment\nanalysis work, one must determine an author's evaluation with respect to a\nmulti-point scale (e.g., one to five \"stars\"). This task represents an\ninteresting twist on standard multi-class text categorization because there are\nseveral different degrees of similarity between class labels; for example,\n\"three stars\" is intuitively closer to \"four stars\" than to \"one star\". We\nfirst evaluate human performance at the task. Then, we apply a meta-algorithm,\nbased on a metric labeling formulation of the problem, that alters a given\nn-ary classifier's output in an explicit attempt to ensure that similar items\nreceive similar labels. We show that the meta-algorithm can provide significant\nimprovements over both multi-class and regression versions of SVMs when we\nemploy a novel similarity measure appropriate to the problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0506101v1",
            "date": "2005-06-29 20:26:33+00:00",
            "title": "Efficient Multiclass Implementations of L1-Regularized Maximum Entropy",
            "authors": [
                "Patrick Haffner",
                "Steven Phillips",
                "Rob Schapire"
            ],
            "abstract": "This paper discusses the application of L1-regularized maximum entropy\nmodeling or SL1-Max [9] to multiclass categorization problems. A new\nmodification to the SL1-Max fast sequential learning algorithm is proposed to\nhandle conditional distributions. Furthermore, unlike most previous studies,\nthe present research goes beyond a single type of conditional distribution. It\ndescribes and compares a variety of modeling assumptions about the class\ndistribution (independent or exclusive) and various types of joint or\nconditional distributions. It results in a new methodology for combining binary\nregularized classifiers to achieve multiclass categorization. In this context,\nMaximum Entropy can be considered as a generic and efficient regularized\nclassification tool that matches or outperforms the state-of-the art\nrepresented by AdaBoost and SVMs.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0508092v1",
            "date": "2005-08-22 12:56:32+00:00",
            "title": "Summarizing Reports on Evolving Events; Part I: Linear Evolution",
            "authors": [
                "Stergos D. Afantenos",
                "Vangelis Karkaletsis",
                "Panagiotis Stamatopoulos"
            ],
            "abstract": "We present an approach for summarization from multiple documents which report\non events that evolve through time, taking into account the different document\nsources. We distinguish the evolution of an event into linear and non-linear.\nAccording to our approach, each document is represented by a collection of\nmessages which are then used in order to instantiate the cross-document\nrelations that determine the summary content. The paper presents the\nsummarization system that implements this approach through a case study on\nlinear evolution.",
            "journal ref": "Edited by Galia Angelova, Kalina Bontcheva, Ruslan Mitkov, Nicolas\n  Nicolov, and Nikolai Nikolov, Recent Advances in Natural Language Processing\n  (RANLP 2005). Borovets, Bulgaria: INCOMA, 18-24.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0509092v1",
            "date": "2005-09-28 16:15:27+00:00",
            "title": "Automatic extraction of paraphrastic phrases from medium size corpora",
            "authors": [
                "Thierry Poibeau"
            ],
            "abstract": "This paper presents a versatile system intended to acquire paraphrastic\nphrases from a representative corpus. In order to decrease the time spent on\nthe elaboration of resources for NLP system (for example Information\nExtraction, IE hereafter), we suggest to use a machine learning system that\nhelps defining new templates and associated resources. This knowledge is\nautomatically derived from the text collection, in interaction with a large\nsemantic network.",
            "journal ref": "Actes de la conf\\'{e}rence Computational Linguisitcs (COLING 2004)\n  (2004) 638-644",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0510054v1",
            "date": "2005-10-19 14:56:48+00:00",
            "title": "The Nature of Novelty Detection",
            "authors": [
                "Le Zhao",
                "Min Zhang",
                "Shaoping Ma"
            ],
            "abstract": "Sentence level novelty detection aims at reducing redundant sentences from a\nsentence list. In the task, sentences appearing later in the list with no new\nmeanings are eliminated. Aiming at a better accuracy for detecting redundancy,\nthis paper reveals the nature of the novelty detection task currently\noverlooked by the Novelty community $-$ Novelty as a combination of the partial\noverlap (PO, two sentences sharing common facts) and complete overlap (CO, the\nfirst sentence covers all the facts of the second sentence) relations. By\nformalizing novelty detection as a combination of the two relations between\nsentences, new viewpoints toward techniques dealing with Novelty are proposed.\nAmong the methods discussed, the similarity, overlap, pool and language\nmodeling approaches are commonly used. Furthermore, a novel approach, selected\npool method is provided, which is immediate following the nature of the task.\nExperimental results obtained on all the three currently available novelty\ndatasets showed that selected pool is significantly better or no worse than the\ncurrent methods. Knowledge about the nature of the task also affects the\nevaluation methodologies. We propose new evaluation measures for Novelty\naccording to the nature of the task, as well as possible directions for future\nstudy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0501018v1",
            "date": "2005-01-10 21:03:14+00:00",
            "title": "Combining Independent Modules in Lexical Multiple-Choice Problems",
            "authors": [
                "Peter D. Turney",
                "Michael L. Littman",
                "Jeffrey Bigham",
                "Victor Shnayder"
            ],
            "abstract": "Existing statistical approaches to natural language problems are very coarse\napproximations to the true complexity of language processing. As such, no\nsingle technique will be best for all problem instances. Many researchers are\nexamining ensemble methods that combine the output of multiple modules to\ncreate more accurate solutions. This paper examines three merging rules for\ncombining probability distributions: the familiar mixture rule, the logarithmic\nrule, and a novel product rule. These rules were applied with state-of-the-art\nresults to two problems used to assess human mastery of lexical semantics --\nsynonym questions and analogy questions. All three merging rules result in\nensembles that are more accurate than any of their component modules. The\ndifferences among the three rules are not statistically significant, but it is\nsuggestive that the popular mixture rule is not the best rule for either of the\ntwo problems.",
            "journal ref": "Recent Advances in Natural Language Processing III: Selected\n  Papers from RANLP 2003, Eds: N. Nicolov, K. Botcheva, G. Angelova, and R.\n  Mitkov, (2004), Current Issues in Linguistic Theory (CILT), 260, John\n  Benjamins, 101-110",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0504074v1",
            "date": "2005-04-15 20:10:53+00:00",
            "title": "Metalinguistic Information Extraction for Terminology",
            "authors": [
                "Carlos Rodriguez"
            ],
            "abstract": "This paper describes and evaluates the Metalinguistic Operation Processor\n(MOP) system for automatic compilation of metalinguistic information from\ntechnical and scientific documents. This system is designed to extract\nnon-standard terminological resources that we have called Metalinguistic\nInformation Databases (or MIDs), in order to help update changing glossaries,\nknowledge bases and ontologies, as well as to reflect the metastable dynamics\nof special-domain knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0504089v2",
            "date": "2005-04-20 17:40:55+00:00",
            "title": "Universal Similarity",
            "authors": [
                "Paul Vitanyi"
            ],
            "abstract": "We survey a new area of parameter-free similarity distance measures useful in\ndata-mining, pattern recognition, learning and automatic semantics extraction.\nGiven a family of distances on a set of objects, a distance is universal up to\na certain precision for that family if it minorizes every distance in the\nfamily between every two objects in the set, up to the stated precision (we do\nnot require the universal distance to be an element of the family). We consider\nsimilarity distances for two types of objects: literal objects that as such\ncontain all of their meaning, like genomes or books, and names for objects. The\nlatter may have literal embodyments like the first type, but may also be\nabstract like ``red'' or ``christianity.'' For the first type we consider a\nfamily of computable distance measures corresponding to parameters expressing\nsimilarity according to particular features between pairs of literal objects.\nFor the second type we consider similarity distances generated by web users\ncorresponding to particular semantic relations between the (names for) the\ndesignated objects. For both families we give universal similarity distance\nmeasures, incorporating all particular distance measures in the family. In the\nfirst case the universal distance is based on compression and in the second\ncase it is based on Google page counts related to search terms. In both cases\nexperiments on a massive scale give evidence of the viability of the\napproaches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0507022v1",
            "date": "2005-07-07 18:53:14+00:00",
            "title": "On Hilberg's Law and Its Links with Guiraud's Law",
            "authors": [
                "\u0141ukasz D\u0229bowski"
            ],
            "abstract": "Hilberg (1990) supposed that finite-order excess entropy of a random human\ntext is proportional to the square root of the text length. Assuming that\nHilberg's hypothesis is true, we derive Guiraud's law, which states that the\nnumber of word types in a text is greater than proportional to the square root\nof the text length. Our derivation is based on some mathematical conjecture in\ncoding theory and on several experiments suggesting that words can be defined\napproximately as the nonterminals of the shortest context-free grammar for the\ntext. Such operational definition of words can be applied even to texts\ndeprived of spaces, which do not allow for Mandelbrot's ``intermittent\nsilence'' explanation of Zipf's and Guiraud's laws. In contrast to\nMandelbrot's, our model assumes some probabilistic long-memory effects in human\nnarration and might be capable of explaining Menzerath's law.",
            "journal ref": "Journal of Quantitative Linguistics 13(1):81-109, 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0507056v1",
            "date": "2005-07-21 21:56:34+00:00",
            "title": "Explorations in engagement for humans and robots",
            "authors": [
                "Candace L. Sidner",
                "Christopher Lee",
                "Cory Kidd",
                "Neal Lesh",
                "Charles Rich"
            ],
            "abstract": "This paper explores the concept of engagement, the process by which\nindividuals in an interaction start, maintain and end their perceived\nconnection to one another. The paper reports on one aspect of engagement among\nhuman interactors--the effect of tracking faces during an interaction. It also\ndescribes the architecture of a robot that can participate in conversational,\ncollaborative interactions with engagement gestures. Finally, the paper reports\non findings of experiments with human participants who interacted with a robot\nwhen it either performed or did not perform engagement gestures. Results of the\nhuman-robot studies indicate that people become engaged with robots: they\ndirect their attention to the robot more often in interactions where engagement\ngestures are present, and they find interactions more appropriate when\nengagement gestures are present than when they are not.",
            "journal ref": "Artificial Intelligence, volume 166, issues 1-2, August 2005, pp.\n  140-164",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0508053v1",
            "date": "2005-08-10 19:35:57+00:00",
            "title": "Measuring Semantic Similarity by Latent Relational Analysis",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "This paper introduces Latent Relational Analysis (LRA), a method for\nmeasuring semantic similarity. LRA measures similarity in the semantic\nrelations between two pairs of words. When two pairs have a high degree of\nrelational similarity, they are analogous. For example, the pair cat:meow is\nanalogous to the pair dog:bark. There is evidence from cognitive science that\nrelational similarity is fundamental to many cognitive and linguistic tasks\n(e.g., analogical reasoning). In the Vector Space Model (VSM) approach to\nmeasuring relational similarity, the similarity between two pairs is calculated\nby the cosine of the angle between the vectors that represent the two pairs.\nThe elements in the vectors are based on the frequencies of manually\nconstructed patterns in a large corpus. LRA extends the VSM approach in three\nways: (1) patterns are derived automatically from the corpus, (2) Singular\nValue Decomposition is used to smooth the frequency data, and (3) synonyms are\nused to reformulate word pairs. This paper describes the LRA algorithm and\nexperimentally compares LRA to VSM on two tasks, answering college-level\nmultiple-choice word analogy questions and classifying semantic relations in\nnoun-modifier expressions. LRA achieves state-of-the-art results, reaching\nhuman-level performance on the analogy questions and significantly exceeding\nVSM performance on both tasks.",
            "journal ref": "Proceedings of the Nineteenth International Joint Conference on\n  Artificial Intelligence (IJCAI-05), (2005), Edinburgh, Scotland, 1136-1141",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0508103v1",
            "date": "2005-08-23 20:21:56+00:00",
            "title": "Corpus-based Learning of Analogies and Semantic Relations",
            "authors": [
                "Peter D. Turney",
                "Michael L. Littman"
            ],
            "abstract": "We present an algorithm for learning from unlabeled text, based on the Vector\nSpace Model (VSM) of information retrieval, that can solve verbal analogy\nquestions of the kind found in the SAT college entrance exam. A verbal analogy\nhas the form A:B::C:D, meaning \"A is to B as C is to D\"; for example,\nmason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B,\nand the problem is to select the most analogous word pair, C:D, from a set of\nfive choices. The VSM algorithm correctly answers 47% of a collection of 374\ncollege-level analogy questions (random guessing would yield 20% correct; the\naverage college-bound senior high school student answers about 57% correctly).\nWe motivate this research by applying it to a difficult problem in natural\nlanguage processing, determining semantic relations in noun-modifier pairs. The\nproblem is to classify a noun-modifier pair, such as \"laser printer\", according\nto the semantic relation between the noun (printer) and the modifier (laser).\nWe use a supervised nearest-neighbour algorithm that assigns a class to a given\nnoun-modifier pair by finding the most analogous noun-modifier pair in the\ntraining data. With 30 classes of semantic relations, on a collection of 600\nlabeled noun-modifier pairs, the learning algorithm attains an F value of 26.5%\n(random guessing: 3.3%). With 5 classes of semantic relations, the F value is\n43.2% (random: 20%). The performance is state-of-the-art for both verbal\nanalogies and noun-modifier relations.",
            "journal ref": "Machine Learning, (2005), 60(1-3), 251-278",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0502086v1",
            "date": "2005-02-22 09:51:16+00:00",
            "title": "The Self-Organization of Speech Sounds",
            "authors": [
                "Pierre-Yves Oudeyer"
            ],
            "abstract": "The speech code is a vehicle of language: it defines a set of forms used by a\ncommunity to carry information. Such a code is necessary to support the\nlinguistic interactions that allow humans to communicate. How then may a speech\ncode be formed prior to the existence of linguistic interactions? Moreover, the\nhuman speech code is discrete and compositional, shared by all the individuals\nof a community but different across communities, and phoneme inventories are\ncharacterized by statistical regularities. How can a speech code with these\nproperties form? We try to approach these questions in the paper, using the\n\"methodology of the artificial\". We build a society of artificial agents, and\ndetail a mechanism that shows the formation of a discrete speech code without\npre-supposing the existence of linguistic capacities or of coordinated\ninteractions. The mechanism is based on a low-level model of sensory-motor\ninteractions. We show that the integration of certain very simple and non\nlanguage-specific neural devices leads to the formation of a speech code that\nhas properties similar to the human speech code. This result relies on the\nself-organizing properties of a generic coupling between perception and\nproduction within agents, and on the interactions between agents. The\nartificial system helps us to develop better intuitions on how speech might\nhave appeared, by showing how self-organization might have helped natural\nselection to find speech.",
            "journal ref": "Journal of Theoretical Biology 233 (2005) Issue 3, Pages 435-449",
            "category": "cs.LG"
        }
    ],
    "2006": [
        {
            "url": "http://arxiv.org/abs/cs/0604027v1",
            "date": "2006-04-07 13:10:30+00:00",
            "title": "Unification of multi-lingual scientific terminological resources using the ISO 16642 standard. The TermSciences initiative",
            "authors": [
                "Majid Khayari",
                "St\u00e9phane Schneider",
                "Isabelle Kramer",
                "Laurent Romary",
                "the termsciences Collaboration"
            ],
            "abstract": "This paper presents the TermSciences portal, which deals with the\nimplementation of a conceptual model that uses the recent ISO 16642 standard\n(Terminological Markup Framework). This standard turns out to be suitable for\nconcept modeling since it allowed for organizing the original resources by\nconcepts and to associate the various terms for a given concept. Additional\nstructuring is produced by sharing conceptual relationships, that is,\ncross-linking of resource results through the introduction of semantic\nrelations which may have initially be missing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606006v1",
            "date": "2006-06-01 09:14:51+00:00",
            "title": "Foundations of Modern Language Resource Archives",
            "authors": [
                "Peter Wittenburg",
                "Daan Broeder",
                "Wolfgang Klein",
                "Stephen Levinson",
                "Laurent Romary"
            ],
            "abstract": "A number of serious reasons will convince an increasing amount of researchers\nto store their relevant material in centers which we will call \"language\nresource archives\". They combine the duty of taking care of long-term\npreservation as well as the task to give access to their material to different\nuser groups. Access here is meant in the sense that an active interaction with\nthe data will be made possible to support the integration of new data, new\nversions or commentaries of all sort. Modern Language Resource Archives will\nhave to adhere to a number of basic principles to fulfill all requirements and\nthey will have to be involved in federations to create joint language resource\ndomains making it even more simple for the researchers to access the data. This\npaper makes an attempt to formulate the essential pillars language resource\narchives have to adhere to.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606096v1",
            "date": "2006-06-22 13:26:52+00:00",
            "title": "Building a resource for studying translation shifts",
            "authors": [
                "Lea Cyrus"
            ],
            "abstract": "This paper describes an interdisciplinary approach which brings together the\nfields of corpus linguistics and translation studies. It presents ongoing work\non the creation of a corpus resource in which translation shifts are explicitly\nannotated. Translation shifts denote departures from formal correspondence\nbetween source and target text, i.e. deviations that have occurred during the\ntranslation process. A resource in which such shifts are annotated in a\nsystematic way will make it possible to study those phenomena that need to be\naddressed if machine translation output is to resemble human translation. The\nresource described in this paper contains English source texts (parliamentary\nproceedings) and their German translations. The shift annotation is based on\npredicate-argument structures and proceeds in two steps: first, predicates and\ntheir arguments are annotated monolingually in a straightforward manner. Then,\nthe corresponding English and German predicates and arguments are aligned with\neach other. Whenever a shift - mainly grammatical or semantic -has occurred,\nthe alignment is tagged accordingly.",
            "journal ref": "Proc. LREC 2006, Genoa, May 24-26, 2006; pp. 1240-1245",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607051v1",
            "date": "2006-07-11 19:13:25+00:00",
            "title": "Raisonner avec des diagrammes : perspectives cognitives et computationnelles",
            "authors": [
                "Catherine Recanati"
            ],
            "abstract": "Diagrammatic, analogical or iconic representations are often contrasted with\nlinguistic or logical representations, in which the shape of the symbols is\narbitrary. The aim of this paper is to make a case for the usefulness of\ndiagrams in inferential knowledge representation systems. Although commonly\nused, diagrams have for a long time suffered from the reputation of being only\na heuristic tool or a mere support for intuition. The first part of this paper\nis an historical background paying tribute to the logicians, psychologists and\ncomputer scientists who put an end to this formal prejudice against diagrams.\nThe second part is a discussion of their characteristics as opposed to those of\nlinguistic forms. The last part is aimed at reviving the interest for\nheterogeneous representation systems including both linguistic and diagrammatic\nrepresentations.",
            "journal ref": "Intellectica 40 (2005) 9-42",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609019v1",
            "date": "2006-09-06 11:41:27+00:00",
            "title": "Improving Term Extraction with Terminological Resources",
            "authors": [
                "Sophie Aubin",
                "Thierry Hamon"
            ],
            "abstract": "Studies of different term extractors on a corpus of the biomedical domain\nrevealed decreasing performances when applied to highly technical texts. The\ndifficulty or impossibility of customising them to new domains is an additional\nlimitation. In this paper, we propose to use external terminologies to\ninfluence generic linguistic data in order to augment the quality of the\nextraction. The tool we implemented exploits testified terms at different steps\nof the process: chunking, parsing and extraction of term candidates.\nExperiments reported here show that, using this method, more term candidates\ncan be acquired with a higher level of reliability. We further describe the\nextraction process involving endogenous disambiguation implemented in the term\nextractor YaTeA.",
            "journal ref": "Advances in Natural Language Processing 5th International\n  Conference on NLP, FinTAL 2006 (2006) 380",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609043v1",
            "date": "2006-09-08 14:01:57+00:00",
            "title": "Challenging the principle of compositionality in interpreting natural language texts",
            "authors": [
                "Fran\u00e7oise Gayral",
                "Daniel Kayser",
                "Fran\u00e7ois L\u00e9vy"
            ],
            "abstract": "The paper aims at emphasizing that, even relaxed, the hypothesis of\ncompositionality has to face many problems when used for interpreting natural\nlanguage texts. Rather than fixing these problems within the compositional\nframework, we believe that a more radical change is necessary, and propose\nanother approach.",
            "journal ref": "conference on Compositionality, Concepts and Cognition, Allemagne\n  (2004)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609044v1",
            "date": "2006-09-08 14:11:50+00:00",
            "title": "The role of time in considering collections",
            "authors": [
                "Fran\u00e7oise Gayral",
                "Daniel Kayser",
                "Fran\u00e7ois L\u00e9vy"
            ],
            "abstract": "The paper concerns the understanding of plurals in the framework of\nArtificial Intelligence and emphasizes the role of time. The construction of\ncollection(s) and their evolution across time is often crucial and has to be\naccounted for. The paper contrasts a \"de dicto\" collection where the collection\ncan be considered as persisting over these situations even if its members\nchange with a \"de re\" collection whose composition does not vary through time.\nIt expresses different criteria of choice between the two interpretations (de\nre and de dicto) depending on the context of enunciation.",
            "journal ref": "Journ\\'{e}es de S\\'{e}mantique et Mod\\'{e}lisation, France (2004)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609058v1",
            "date": "2006-09-12 07:10:15+00:00",
            "title": "The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages",
            "authors": [
                "Ralf Steinberger",
                "Bruno Pouliquen",
                "Anna Widiger",
                "Camelia Ignat",
                "Tomaz Erjavec",
                "Dan Tufis",
                "Daniel Varga"
            ],
            "abstract": "We present a new, unique and freely available parallel corpus containing\nEuropean Union (EU) documents of mostly legal nature. It is available in all 20\nofficial EUanguages, with additional documents being available in the languages\nof the EU candidate countries. The corpus consists of almost 8,000 documents\nper language, with an average size of nearly 9 million words per language.\nPair-wise paragraph alignment information produced by two different aligners\n(Vanilla and HunAlign) is available for all 190+ language pair combinations.\nMost texts have been manually classified according to the EUROVOC subject\ndomains so that the collection can also be used to train and test multi-label\nclassification algorithms and keyword-assignment software. The corpus is\nencoded in XML, according to the Text Encoding Initiative Guidelines. Due to\nthe large number of parallel texts in many languages, the JRC-Acquis is\nparticularly suitable to carry out all types of cross-language research, as\nwell as to test and benchmark text analysis software across different languages\n(for instance for alignment, sentence splitting and term extraction).",
            "journal ref": "Proceedings of the 5th International Conference on Language\n  Resources and Evaluation (LREC'2006), pp. 2142-2147. Genoa, Italy, 24-26 May\n  2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610116v1",
            "date": "2006-10-19 17:42:57+00:00",
            "title": "DepAnn - An Annotation Tool for Dependency Treebanks",
            "authors": [
                "Tuomo Kakkonen"
            ],
            "abstract": "DepAnn is an interactive annotation tool for dependency treebanks, providing\nboth graphical and text-based annotation interfaces. The tool is aimed for\nsemi-automatic creation of treebanks. It aids the manual inspection and\ncorrection of automatically created parses, making the annotation process\nfaster and less error-prone. A novel feature of the tool is that it enables the\nuser to view outputs from several parsers as the basis for creating the final\ntree to be saved to the treebank. DepAnn uses TIGER-XML, an XML-based general\nencoding format for both, representing the parser outputs and saving the\nannotated treebank. The tool includes an automatic consistency checker for\nsentence structures. In addition, the tool enables users to build structures\nmanually, add comments on the annotations, modify the tagsets, and mark\nsentences for further revision.",
            "journal ref": "Proceedings of the 11th ESSLLI Student Session at the 18th\n  European Summer School in Logic, Language and Information (ESSLLI 2006), pp.\n  214-225. Malaga, Spain, 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610124v1",
            "date": "2006-10-20 11:48:38+00:00",
            "title": "Dependency Treebanks: Methods, Annotation Schemes and Tools",
            "authors": [
                "Tuomo Kakkonen"
            ],
            "abstract": "In this paper, current dependencybased treebanks are introduced and analyzed.\nThe methods used for building the resources, the annotation schemes applied,\nand the tools used (such as POS taggers, parsers and annotation software) are\ndiscussed.",
            "journal ref": "Proceedings of the 15th Nordic Conference of Computational\n  Linguistics (NODALIDA 2005), pp. 94-104. Joensuu, Finland, 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0611026v1",
            "date": "2006-11-06 14:37:27+00:00",
            "title": "Un mod\u00e8le g\u00e9n\u00e9rique d'organisation de corpus en ligne: application \u00e0 la FReeBank",
            "authors": [
                "Susanne Salmon-Alt",
                "Laurent Romary",
                "Jean-Marie Pierrel"
            ],
            "abstract": "The few available French resources for evaluating linguistic models or\nalgorithms on other linguistic levels than morpho-syntax are either\ninsufficient from quantitative as well as qualitative point of view or not\nfreely accessible. Based on this fact, the FREEBANK project intends to create\nFrench corpora constructed using manually revised output from a hybrid\nConstraint Grammar parser and annotated on several linguistic levels\n(structure, morpho-syntax, syntax, coreference), with the objective to make\nthem available on-line for research purposes. Therefore, we will focus on using\nstandard annotation schemes, integration of existing resources and maintenance\nallowing for continuous enrichment of the annotations. Prior to the actual\npresentation of the prototype that has been implemented, this paper describes a\ngeneric model for the organization and deployment of a linguistic resource\narchive, in compliance with the various works currently conducted within\ninternational standardization initiatives (TEI and ISO/TC 37/SC 4).",
            "journal ref": "Traitement Automatique des Langues (TAL) 45 (2006) 145-169",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0611069v1",
            "date": "2006-11-15 12:35:45+00:00",
            "title": "Scaling Construction Grammar up to Production Systems: the SCIM",
            "authors": [
                "Guillaume Pitel"
            ],
            "abstract": "While a great effort has concerned the development of fully integrated\nmodular understanding systems, few researches have focused on the problem of\nunifying existing linguistic formalisms with cognitive processing models. The\nSituated Constructional Interpretation Model is one of these attempts. In this\nmodel, the notion of \"construction\" has been adapted in order to be able to\nmimic the behavior of Production Systems. The Construction Grammar approach\nestablishes a model of the relations between linguistic forms and meaning, by\nthe mean of constructions. The latter can be considered as pairings from a\ntopologically structured space to an unstructured space, in some way a special\nkind of production rules.",
            "journal ref": "Dans Scalable Natural Language Understanding 2006 (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0611113v1",
            "date": "2006-11-22 03:24:54+00:00",
            "title": "An Anthological Review of Research Utilizing MontyLingua, a Python-Based End-to-End Text Processor",
            "authors": [
                "Maurice HT Ling"
            ],
            "abstract": "MontyLingua, an integral part of ConceptNet which is currently the largest\ncommonsense knowledge base, is an English text processor developed using Python\nprogramming language in MIT Media Lab. The main feature of MontyLingua is the\ncoverage for all aspects of English text processing from raw input text to\nsemantic meanings and summary generation, yet each component in MontyLingua is\nloosely-coupled to each other at the architectural and code level, which\nenabled individual components to be used independently or substituted. However,\nthere has been no review exploring the role of MontyLingua in recent research\nwork utilizing it. This paper aims to review the use of and roles played by\nMontyLingua and its components in research work published in 19 articles\nbetween October 2004 and August 2006. We had observed a diversified use of\nMontyLingua in many different areas, both generic and domain-specific. Although\nthe use of text summarizing component had not been observe, we are optimistic\nthat it will have a crucial role in managing the current trend of information\noverload in future research.",
            "journal ref": "Ling, Maurice HT. 2006. An Anthological Review of Research\n  Utilizing MontyLingua, a Python-Based End-to-End Text Processor. The Python\n  Papers 1 (1): 5-13",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0601005v1",
            "date": "2006-01-04 18:33:53+00:00",
            "title": "Analyzing language development from a network approach",
            "authors": [
                "J-Y Ke",
                "Y. Yao"
            ],
            "abstract": "In this paper we propose some new measures of language development using\nnetwork analyses, which is inspired by the recent surge of interests in network\nstudies of many real-world systems. Children's and care-takers' speech data\nfrom a longitudinal study are represented as a series of networks, word forms\nbeing taken as nodes and collocation of words as links. Measures on the\nproperties of the networks, such as size, connectivity, hub and authority\nanalyses, etc., allow us to make quantitative comparison so as to reveal\ndifferent paths of development. For example, the asynchrony of development in\nnetwork size and average degree suggests that children cannot be simply\nclassified as early talkers or late talkers by one or two measures. Children\nfollow different paths in a multi-dimensional space. They may develop faster in\none dimension but slower in another dimension. The network approach requires\nlittle preprocessing of words and analyses on sentence structures, and the\ncharacteristics of words and their usage emerge from the network and are\nindependent of any grammatical presumptions. We show that the change of the two\narticles \"the\" and \"a\" in their roles as important nodes in the network\nreflects the progress of children's syntactic development: the two articles\noften start in children's networks as hubs and later shift to authorities,\nwhile they are authorities constantly in the adult's networks. The network\nanalyses provide a new approach to study language development, and at the same\ntime language development also presents a rich area for network theories to\nexplore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0602093v1",
            "date": "2006-02-27 10:08:26+00:00",
            "title": "Rational stochastic languages",
            "authors": [
                "Fran\u00e7ois Denis",
                "Yann Esposito"
            ],
            "abstract": "The goal of the present paper is to provide a systematic and comprehensive\nstudy of rational stochastic languages over a semiring K \\in {Q, Q +, R, R+}. A\nrational stochastic language is a probability distribution over a free monoid\n\\Sigma^* which is rational over K, that is which can be generated by a\nmultiplicity automata with parameters in K. We study the relations between the\nclasses of rational stochastic languages S rat K (\\Sigma). We define the notion\nof residual of a stochastic language and we use it to investigate properties of\nseveral subclasses of rational stochastic languages. Lastly, we study the\nrepresentation of rational stochastic languages by means of multiplicity\nautomata.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/cs/0604087v1",
            "date": "2006-04-23 02:58:51+00:00",
            "title": "Probabilistic Automata for Computing with Words",
            "authors": [
                "Yongzhi Cao",
                "Lirong Xia",
                "Mingsheng Ying"
            ],
            "abstract": "Usually, probabilistic automata and probabilistic grammars have crisp symbols\nas inputs, which can be viewed as the formal models of computing with values.\nIn this paper, we first introduce probabilistic automata and probabilistic\ngrammars for computing with (some special) words in a probabilistic framework,\nwhere the words are interpreted as probabilistic distributions or possibility\ndistributions over a set of crisp symbols. By probabilistic conditioning, we\nthen establish a retraction principle from computing with words to computing\nwith values for handling crisp inputs and a generalized extension principle\nfrom computing with words to computing with all words for handling arbitrary\ninputs. These principles show that computing with values and computing with all\nwords can be respectively implemented by computing with some special words. To\ncompare the transition probabilities of two near inputs, we also examine some\nanalytical properties of the transition probability functions of generalized\nextensions. Moreover, the retractions and the generalized extensions are shown\nto be equivalence-preserving. Finally, we clarify some relationships among the\nretractions, the generalized extensions, and the extensions studied recently by\nQiu and Wang.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0605076v1",
            "date": "2006-05-17 17:12:25+00:00",
            "title": "Numeration-automatic sequences",
            "authors": [
                "J. F. J. Laros"
            ],
            "abstract": "We present a base class of automata that induce a numeration system and we\ngive an algorithm to give the n-th word in the language of the automaton when\nthe expansion of n in the induced numeration system is feeded to the automaton.\nFurthermore we give some algorithms for reverse reading of this expansion and a\nway to combine automata to other automata having the same properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0605147v1",
            "date": "2006-05-30 15:32:41+00:00",
            "title": "Utilisation de la linguistique en reconnaissance de la parole : un \u00e9tat de l'art",
            "authors": [
                "St\u00e9phane Huet",
                "Pascale S\u00e9billot",
                "Guillaume Gravier"
            ],
            "abstract": "To transcribe speech, automatic speech recognition systems use statistical\nmethods, particularly hidden Markov model and N-gram models. Although these\ntechniques perform well and lead to efficient systems, they approach their\nmaximum possibilities. It seems thus necessary, in order to outperform current\nresults, to use additional information, especially bound to language. However,\nintroducing such knowledge must be realized taking into account specificities\nof spoken language (hesitations for example) and being robust to possible\nmisrecognized words. This document presents a state of the art of these\nresearches, evaluating the impact of the insertion of linguistic information on\nthe quality of the transcription.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606069v1",
            "date": "2006-06-14 14:44:06+00:00",
            "title": "Inference and Evaluation of the Multinomial Mixture Model for Text Clustering",
            "authors": [
                "Lo\u00efs Rigouste",
                "Olivier Capp\u00e9",
                "Fran\u00e7ois Yvon"
            ],
            "abstract": "In this article, we investigate the use of a probabilistic model for\nunsupervised clustering in text collections. Unsupervised clustering has become\na basic module for many intelligent text processing applications, such as\ninformation retrieval, text classification or information extraction. The model\nconsidered in this contribution consists of a mixture of multinomial\ndistributions over the word counts, each component corresponding to a different\ntheme. We present and contrast various estimation procedures, which apply both\nin supervised and unsupervised contexts. In supervised learning, this work\nsuggests a criterion for evaluating the posterior odds of new documents which\nis more statistically sound than the \"naive Bayes\" approach. In an unsupervised\ncontext, we propose measures to set up a systematic evaluation framework and\nstart with examining the Expectation-Maximization (EM) algorithm as the basic\ntool for inference. We discuss the importance of initialization and the\ninfluence of other features such as the smoothing strategy or the size of the\nvocabulary, thereby illustrating the difficulties incurred by the high\ndimensionality of the parameter space. We also propose a heuristic algorithm\nbased on iterative EM with vocabulary reduction to solve this problem. Using\nthe fact that the latent variables can be analytically integrated out, we\nfinally show that Gibbs sampling algorithm is tractable and compares favorably\nto the basic expectation maximization approach.",
            "journal ref": "Information Processing & Management 43, 5 (01/09/2007) 1260?1280",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606118v1",
            "date": "2006-06-28 14:43:42+00:00",
            "title": "Adapting a general parser to a sublanguage",
            "authors": [
                "Sophie Aubin",
                "Adeline Nazarenko",
                "Claire N\u00e9dellec"
            ],
            "abstract": "In this paper, we propose a method to adapt a general parser (Link Parser) to\nsublanguages, focusing on the parsing of texts in biology. Our main proposal is\nthe use of terminology (identication and analysis of terms) in order to reduce\nthe complexity of the text to be parsed. Several other strategies are explored\nand finally combined among which text normalization, lexicon and\nmorpho-guessing module extensions and grammar rules adaptation. We compare the\nparsing results before and after these adaptations.",
            "journal ref": "Proceedings of the International Conference on Recent Advances in\n  Natural Language Processing (RANLP'05) (2005) 89-93",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606119v1",
            "date": "2006-06-28 14:44:48+00:00",
            "title": "Lexical Adaptation of Link Grammar to the Biomedical Sublanguage: a Comparative Evaluation of Three Approaches",
            "authors": [
                "Sampo Pyysalo",
                "Tapio Salakoski",
                "Sophie Aubin",
                "Adeline Nazarenko"
            ],
            "abstract": "We study the adaptation of Link Grammar Parser to the biomedical sublanguage\nwith a focus on domain terms not found in a general parser lexicon. Using two\nbiomedical corpora, we implement and evaluate three approaches to addressing\nunknown words: automatic lexicon expansion, the use of morphological clues, and\ndisambiguation using a part-of-speech tagger. We evaluate each approach\nseparately for its effect on parsing performance and consider combinations of\nthese approaches. In addition to a 45% increase in parsing efficiency, we find\nthat the best approach, incorporating information from a domain part-of-speech\ntagger, offers a statistically signicant 10% relative decrease in error. The\nadapted parser is available under an open-source license at\nhttp://www.it.utu.fi/biolg.",
            "journal ref": "Proceedings of the Second International Symposium on Semantic\n  Mining in Biomedicine (SMBM 2006) (2006) 60-67",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607052v1",
            "date": "2006-07-11 19:14:49+00:00",
            "title": "Dealing with Metonymic Readings of Named Entities",
            "authors": [
                "Thierry Poibeau"
            ],
            "abstract": "The aim of this paper is to propose a method for tagging named entities (NE),\nusing natural language processing techniques. Beyond their literal meaning,\nnamed entities are frequently subject to metonymy. We show the limits of\ncurrent NE type hierarchies and detail a new proposal aiming at dynamically\ncapturing the semantics of entities in context. This model can analyze complex\nlinguistic phenomena like metonymy, which are known to be difficult for natural\nlanguage processing but crucial for most applications. We present an\nimplementation and some test using the French ESTER corpus and give significant\nresults.",
            "journal ref": "Dans Actes de The 28th Annual Conference of the Cognitive Science\n  Society (CogSci 2006) - The 28th Annual Conference of the Cognitive Science\n  Society (CogSci 2006), Vancouver : Canada (2006)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607053v1",
            "date": "2006-07-11 19:17:13+00:00",
            "title": "Linguistically Grounded Models of Language Change",
            "authors": [
                "Thierry Poibeau"
            ],
            "abstract": "Questions related to the evolution of language have recently known an\nimpressive increase of interest (Briscoe, 2002). This short paper aims at\nquestioning the scientific status of these models and their relations to\nattested data. We show that one cannot directly model non-linguistic factors\n(exogenous factors) even if they play a crucial role in language evolution. We\nthen examine the relation between linguistic models and attested language data,\nas well as their contribution to cognitive linguistics.",
            "journal ref": "The 28th Annual Conference of the Cognitive Science Society\n  (CogSci 2006), Canada (2006)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607088v1",
            "date": "2006-07-18 07:43:07+00:00",
            "title": "Using Answer Set Programming in an Inference-Based approach to Natural Language Semantics",
            "authors": [
                "Farid Nouioua",
                "Pascal Nicolas"
            ],
            "abstract": "Using Answer Set Programming in an Inference-Based approach to Natural\nLanguage Semantics",
            "journal ref": "Inference in Computational Semantics ICoS-5, France (2006) 77-86",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609051v1",
            "date": "2006-09-11 11:08:23+00:00",
            "title": "Multilingual person name recognition and transliteration",
            "authors": [
                "Bruno Pouliquen",
                "Ralf Steinberger",
                "Camelia Ignat",
                "Irina Temnikova",
                "Anna Widiger",
                "Wajdi Zaghouani",
                "Jan Zizka"
            ],
            "abstract": "We present an exploratory tool that extracts person names from multilingual\nnews collections, matches name variants referring to the same person, and\ninfers relationships between people based on the co-occurrence of their names\nin related news. A novel feature is the matching of name variants across\nlanguages and writing systems, including names written with the Greek, Cyrillic\nand Arabic writing system. Due to our highly multilingual setting, we use an\ninternal standard representation for name representation and matching, instead\nof adopting the traditional bilingual approach to transliteration. This work is\npart of the news analysis system NewsExplorer that clusters an average of\n25,000 news articles per day to detect related news within the same and across\ndifferent languages.",
            "journal ref": "Journal CORELA - Cognition, Representation, Langage. Numeros\n  speciaux, Le traitement lexicographique des noms propres. December 2005. ISSN\n  1638-5748",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609053v1",
            "date": "2006-09-11 11:54:41+00:00",
            "title": "Navigating multilingual news collections using automatically extracted information",
            "authors": [
                "Ralf Steinberger",
                "Bruno Pouliquen",
                "Camelia Ignat"
            ],
            "abstract": "We are presenting a text analysis tool set that allows analysts in various\nfields to sieve through large collections of multilingual news items quickly\nand to find information that is of relevance to them. For a given document\ncollection, the tool set automatically clusters the texts into groups of\nsimilar articles, extracts names of places, people and organisations, lists the\nuser-defined specialist terms found, links clusters and entities, and generates\nhyperlinks. Through its daily news analysis operating on thousands of articles\nper day, the tool also learns relationships between people and other entities.\nThe fully functional prototype system allows users to explore and navigate\nmultilingual document collections across languages and time.",
            "journal ref": "Proceedings of the 27th International Conference 'Information\n  Technology Interfaces' (ITI'2005). Cavtat / Dubrovnik",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609059v1",
            "date": "2006-09-12 07:24:01+00:00",
            "title": "Automatic annotation of multilingual text collections with a conceptual thesaurus",
            "authors": [
                "Bruno Pouliquen",
                "Ralf Steinberger",
                "Camelia Ignat"
            ],
            "abstract": "Automatic annotation of documents with controlled vocabulary terms\n(descriptors) from a conceptual thesaurus is not only useful for document\nindexing and retrieval. The mapping of texts onto the same thesaurus\nfurthermore allows to establish links between similar documents. This is also a\nsubstantial requirement of the Semantic Web. This paper presents an almost\nlanguage-independent system that maps documents written in different languages\nonto the same multilingual conceptual thesaurus, EUROVOC. Conceptual thesauri\ndiffer from Natural Language Thesauri in that they consist of relatively small\ncontrolled lists of words or phrases with a rather abstract meaning. To\nautomatically identify which thesaurus descriptors describe the contents of a\ndocument best, we developed a statistical, associative system that is trained\non texts that have previously been indexed manually. In addition to describing\nthe large number of empirically optimised parameters of the fully functional\napplication, we present the performance of the software according to a human\nevaluation by professional indexers.",
            "journal ref": "Proceedings of the Workshop 'Ontologies and Information\n  Extraction' at the Summer School 'The Semantic Web and Language Technology -\n  Its Potential and Practicalities' (EUROLAN'2003), pp 9-28. Bucharest,\n  Romania, 28 July - 8 August 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609060v1",
            "date": "2006-09-12 08:44:53+00:00",
            "title": "Automatic Identification of Document Translations in Large Multilingual Document Collections",
            "authors": [
                "Bruno Pouliquen",
                "Ralf Steinberger",
                "Camelia Ignat"
            ],
            "abstract": "Texts and their translations are a rich linguistic resource that can be used\nto train and test statistics-based Machine Translation systems and many other\napplications. In this paper, we present a working system that can identify\ntranslations and other very similar documents among a large number of\ncandidates, by representing the document contents with a vector of thesaurus\nterms from a multilingual thesaurus, and by then measuring the semantic\nsimilarity between the vectors. Tests on different text types have shown that\nthe system can detect translations with over 96% precision in a large search\nspace of 820 documents or more. The system was tuned to ignore\nlanguage-specific similarities and to give similar documents in a second\nlanguage the same similarity score as equivalent documents in the same\nlanguage. The application can also be used to detect cross-lingual document\nplagiarism.",
            "journal ref": "Proceedings of the International Conference 'Recent Advances in\n  Natural Language Processing' (RANLP'2003), pp. 401-408. Borovets, Bulgaria,\n  10 - 12 September 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609061v1",
            "date": "2006-09-12 09:29:57+00:00",
            "title": "Cross-lingual keyword assignment",
            "authors": [
                "Ralf Steinberger"
            ],
            "abstract": "This paper presents a language-independent approach to controlled vocabulary\nkeyword assignment using the EUROVOC thesaurus. Due to the multilingual nature\nof EUROVOC, the keywords for a document written in one language can be\ndisplayed in all eleven official European Union languages. The mapping of\ndocuments written in different languages to the same multilingual thesaurus\nfurthermore allows cross-language document comparison. The assignment of the\ncontrolled vocabulary thesaurus descriptors is achieved by applying a\nstatistical method that uses a collection of manually indexed documents to\nidentify, for each thesaurus descriptor, a large number of lemmas that are\nstatistically associated to the descriptor. These associated words are then\nused during the assignment procedure to identify a ranked list of those EUROVOC\nterms that are most likely to be good keywords for a given document. The paper\nalso describes the challenges of this task and discusses the achieved results\nof the fully functional prototype.",
            "journal ref": "Proceedings of the XVII Conference of the Spanish Society for\n  Natural Language Processing (SEPLN-2001). Procesamiento del Lenguaje Natural,\n  Revista No. 27, pp. 273-280. Jaen, Spain, 12-14 September 2001. ISSN\n  1135-5948",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609063v1",
            "date": "2006-09-12 12:29:17+00:00",
            "title": "Extending an Information Extraction tool set to Central and Eastern European languages",
            "authors": [
                "Camelia Ignat",
                "Bruno Pouliquen",
                "Antonio Ribeiro",
                "Ralf Steinberger"
            ],
            "abstract": "In a highly multilingual and multicultural environment such as in the\nEuropean Commission with soon over twenty official languages, there is an\nurgent need for text analysis tools that use minimal linguistic knowledge so\nthat they can be adapted to many languages without much human effort. We are\npresenting two such Information Extraction tools that have already been adapted\nto various Western and Eastern European languages: one for the recognition of\ndate expressions in text, and one for the detection of geographical place names\nand the visualisation of the results in geographical maps. An evaluation of the\nperformance has produced very satisfying results.",
            "journal ref": "Proceedings of the International Workshop 'Information Extraction\n  for Slavonic and other Central and Eastern European Languages' (IESL-2003),\n  held at RANLP-2003, pp. 33-39. Borovets, Bulgaria, 8 - 9 September 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609064v1",
            "date": "2006-09-12 12:43:37+00:00",
            "title": "Exploiting multilingual nomenclatures and language-independent text features as an interlingua for cross-lingual text analysis applications",
            "authors": [
                "Ralf Steinberger",
                "Bruno Pouliquen",
                "Camelia Ignat"
            ],
            "abstract": "We are proposing a simple, but efficient basic approach for a number of\nmultilingual and cross-lingual language technology applications that are not\nlimited to the usual two or three languages, but that can be applied with\nrelatively little effort to larger sets of languages. The approach consists of\nusing existing multilingual linguistic resources such as thesauri,\nnomenclatures and gazetteers, as well as exploiting the existence of additional\nmore or less language-independent text items such as dates, currency\nexpressions, numbers, names and cognates. Mapping texts onto the multilingual\nresources and identifying word token links between texts in different languages\nare basic ingredients for applications such as cross-lingual document\nsimilarity calculation, multilingual clustering and categorisation,\ncross-lingual document retrieval, and tools to provide cross-lingual\ninformation access.",
            "journal ref": "Information Society 2004 (IS-2004) - Proceedings B of the 7th\n  International Multiconference - Language Technologies, pages 2-12. Ljubljana,\n  Slovenia, 13-14 October 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609065v1",
            "date": "2006-09-12 12:57:38+00:00",
            "title": "Geocoding multilingual texts: Recognition, disambiguation and visualisation",
            "authors": [
                "Bruno Pouliquen",
                "Marco Kimler",
                "Ralf Steinberger",
                "Camelia Ignat",
                "Tamara Oellinger",
                "Ken Blackler",
                "Flavio Fuart",
                "Wajdi Zaghouani",
                "Anna Widiger",
                "Ann-Charlotte Forslund",
                "Clive Best"
            ],
            "abstract": "We are presenting a method to recognise geographical references in free text.\nOur tool must work on various languages with a minimum of language-dependent\nresources, except a gazetteer. The main difficulty is to disambiguate these\nplace names by distinguishing places from persons and by selecting the most\nlikely place out of a list of homographic place names world-wide. The system\nuses a number of language-independent clues and heuristics to disambiguate\nplace name homographs. The final aim is to index texts with the countries and\ncities they mention and to automatically visualise this information on\ngeographical maps using various tools.",
            "journal ref": "Proceedings of the 5th International Conference on Language\n  Resources and Evaluation (LREC-2006), pp. 53-58. Genoa, Italy, 24-26 May 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609066v1",
            "date": "2006-09-12 13:09:37+00:00",
            "title": "Building and displaying name relations using automatic unsupervised analysis of newspaper articles",
            "authors": [
                "Bruno Pouliquen",
                "Ralf Steinberger",
                "Camelia Ignat",
                "Tamara Oellinger"
            ],
            "abstract": "We present a tool that, from automatically recognised names, tries to infer\ninter-person relations in order to present associated people on maps. Based on\nan in-house Named Entity Recognition tool, applied on clusters of an average of\n15,000 news articles per day, in 15 different languages, we build a knowledge\nbase that allows extracting statistical co-occurrences of persons and\nvisualising them on a per-person page or in various graphs.",
            "journal ref": "Proceedings of the 8th International Conference on the Statistical\n  Analysis of Textual Data (JADT-2006). Besancon, 19-21 April 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0609067v1",
            "date": "2006-09-12 13:30:01+00:00",
            "title": "A tool set for the quick and efficient exploration of large document collections",
            "authors": [
                "Camelia Ignat",
                "Bruno Pouliquen",
                "Ralf Steinberger",
                "Tomaz Erjavec"
            ],
            "abstract": "We are presenting a set of multilingual text analysis tools that can help\nanalysts in any field to explore large document collections quickly in order to\ndetermine whether the documents contain information of interest, and to find\nthe relevant text passages. The automatic tool, which currently exists as a\nfully functional prototype, is expected to be particularly useful when users\nrepeatedly have to sieve through large collections of documents such as those\ndownloaded automatically from the internet. The proposed system takes a whole\ndocument collection as input. It first carries out some automatic analysis\ntasks (named entity recognition, geo-coding, clustering, term extraction),\nannotates the texts with the generated meta-information and stores the\nmeta-information in a database. The system then generates a zoomable and\nhyperlinked geographic map enhanced with information on entities and terms\nfound. When the system is used on a regular basis, it builds up a historical\ndatabase that contains information on which names have been mentioned together\nwith which other names or places, and users can query this database to retrieve\ninformation extracted in the past.",
            "journal ref": "Proceedings of the Symposium on Safeguards and Nuclear Material\n  Management. 27th Annual Meeting of the European SAfeguards Research and\n  Development Association (ESARDA-2005). London, UK, 10-12 May 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610004v1",
            "date": "2006-10-01 17:39:44+00:00",
            "title": "Rapport technique du projet OGRE",
            "authors": [
                "G\u00e9rard B\u00e9cher",
                "Patrice Enjalbert",
                "Estelle Fiev\u00e9",
                "Laurent Gosselin",
                "Fran\u00e7ois L\u00e9vy",
                "G\u00e9rard Ligozat"
            ],
            "abstract": "This repport concerns automatic understanding of (french) iterative\nsentences, i.e. sentences where one single verb has to be interpreted by a more\nor less regular plurality of events. A linguistic analysis is proposed along an\nextension of Reichenbach's theory, several formal representations are\nconsidered and a corpus of 18000 newspaper extracts is described.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610010v4",
            "date": "2006-10-03 18:04:22+00:00",
            "title": "One-Pass, One-Hash n-Gram Statistics Estimation",
            "authors": [
                "Daniel Lemire",
                "Owen Kaser"
            ],
            "abstract": "In multimedia, text or bioinformatics databases, applications query sequences\nof n consecutive symbols called n-grams. Estimating the number of distinct\nn-grams is a view-size estimation problem. While view sizes can be estimated by\nsampling under statistical assumptions, we desire an unassuming algorithm with\nuniversally valid accuracy bounds. Most related work has focused on repeatedly\nhashing the data, which is prohibitive for large data sources. We prove that a\none-pass one-hash algorithm is sufficient for accurate estimates if the hashing\nis sufficiently independent. To reduce costs further, we investigate recursive\nrandom hashing algorithms and show that they are sufficiently independent in\npractice. We compare our running times with exact counts using suffix arrays\nand show that, while we use hardly any storage, we are an order of magnitude\nfaster. The approach further is extended to a one-pass/one-hash computation of\nn-gram entropy and iceberg counts. The experiments use a large collection of\nEnglish text from the Gutenberg Project as well as synthetic data.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610016v1",
            "date": "2006-10-04 11:33:02+00:00",
            "title": "Norm Based Causal Reasoning in Textual Corpus",
            "authors": [
                "Farid Nouioua"
            ],
            "abstract": "Truth based entailments are not sufficient for a good comprehension of NL. In\nfact, it can not deduce implicit information necessary to understand a text. On\nthe other hand, norm based entailments are able to reach this goal. This idea\nwas behind the development of Frames (Minsky 75) and Scripts (Schank 77, Schank\n79) in the 70's. But these theories are not formalized enough and their\nadaptation to new situations is far from being obvious. In this paper, we\npresent a reasoning system which uses norms in a causal reasoning process in\norder to find the cause of an accident from a text describing it.",
            "journal ref": "Proceedings of the Sixth International Workshop on Computational\n  Semantics IWCS-6, France (2005) 396-400",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610018v1",
            "date": "2006-10-04 13:09:48+00:00",
            "title": "Raisonnement stratifi\u00e9 \u00e0 base de normes pour inf\u00e9rer les causes dans un corpus textuel",
            "authors": [
                "Farid Nouioua"
            ],
            "abstract": "To understand texts written in natural language (LN), we use our knowledge\nabout the norms of the domain. Norms allow to infer more implicit information\nfrom the text. This kind of information can, in general, be defeasible, but it\nremains useful and acceptable while the text do not contradict it explicitly.\nIn this paper we describe a non-monotonic reasoning system based on the norms\nof the car crash domain. The system infers the cause of an accident from its\ntextual description. The cause of an accident is seen as the most specific norm\nwhich has been violated. The predicates and the rules of the system are\nstratified: organized on layers in order to obtain an efficient reasoning.",
            "journal ref": "The Seventh International Symposium On Programming and\n  SystemsUSTHB d'Alger (Ed.) (2005) 81-92",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0610118v1",
            "date": "2006-10-19 17:50:57+00:00",
            "title": "Applying Part-of-Seech Enhanced LSA to Automatic Essay Grading",
            "authors": [
                "Tuomo Kakkonen",
                "Niko Myller",
                "Erkki Sutinen"
            ],
            "abstract": "Latent Semantic Analysis (LSA) is a widely used Information Retrieval method\nbased on \"bag-of-words\" assumption. However, according to general conception,\nsyntax plays a role in representing meaning of sentences. Thus, enhancing LSA\nwith part-of-speech (POS) information to capture the context of word\noccurrences appears to be theoretically feasible extension. The approach is\ntested empirically on a automatic essay grading system using LSA for document\nsimilarity comparisons. A comparison on several POS-enhanced LSA models is\nreported. Our findings show that the addition of contextual information in the\nform of POS tags can raise the accuracy of the LSA-based scoring models up to\n10.77 per cent.",
            "journal ref": "Proceedings of the 4th IEEE International Conference on\n  Information Technology: Research and Education (ITRE 2006). Tel Aviv, Israel,\n  2006",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0611148v3",
            "date": "2006-11-29 10:49:23+00:00",
            "title": "Next Generation Language Resources using GRID",
            "authors": [
                "Federico Calzolari",
                "Eva Sassolini",
                "Manuela Sassi",
                "Sebastiana Cucurullo",
                "Eugenio Picchi",
                "Francesca Bertagna",
                "Alessandro Enea",
                "Monica Monachini",
                "Claudia Soria",
                "Nicoletta Calzolari"
            ],
            "abstract": "This paper presents a case study concerning the challenges and requirements\nposed by next generation language resources, realized as an overall model of\nopen, distributed and collaborative language infrastructure. If a sort of \"new\nparadigm\" is required, we think that the emerging and still evolving technology\nconnected to Grid computing is a very interesting and suitable one for a\nconcrete realization of this vision. Given the current limitations of Grid\ncomputing, it is very important to test the new environment on basic language\nanalysis tools, in order to get the feeling of what are the potentialities and\npossible limitations connected to its use in NLP. For this reason, we have done\nsome experiments on a module of Linguistic Miner, i.e. the extraction of\nlinguistic patterns from restricted domain corpora.",
            "journal ref": "Language Resources and Evaluation Conference LREC 2006 proceedings\n  pp.1858-1861, Genoa [Italy]",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0601037v1",
            "date": "2006-01-10 12:57:46+00:00",
            "title": "Constraint-based verification of abstract models of multitreaded programs",
            "authors": [
                "Giorgio Delzanno"
            ],
            "abstract": "We present a technique for the automated verification of abstract models of\nmultithreaded programs providing fresh name generation, name mobility, and\nunbounded control.\n  As high level specification language we adopt here an extension of\ncommunication finite-state machines with local variables ranging over an\ninfinite name domain, called TDL programs. Communication machines have been\nproved very effective for representing communication protocols as well as for\nrepresenting abstractions of multithreaded software.\n  The verification method that we propose is based on the encoding of TDL\nprograms into a low level language based on multiset rewriting and constraints\nthat can be viewed as an extension of Petri Nets. By means of this encoding,\nthe symbolic verification procedure developed for the low level language in our\nprevious work can now be applied to TDL programs. Furthermore, the encoding\nallows us to isolate a decidable class of verification problems for TDL\nprograms that still provide fresh name generation, name mobility, and unbounded\ncontrol. Our syntactic restrictions are in fact defined on the internal\nstructure of threads: In order to obtain a complete and terminating method,\nthreads are only allowed to have at most one local variable (ranging over an\ninfinite domain of names).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0601045v1",
            "date": "2006-01-11 21:27:28+00:00",
            "title": "PageRank without hyperlinks: Structural re-ranking using links induced by language models",
            "authors": [
                "Oren Kurland",
                "Lillian Lee"
            ],
            "abstract": "Inspired by the PageRank and HITS (hubs and authorities) algorithms for Web\nsearch, we propose a structural re-ranking approach to ad hoc information\nretrieval: we reorder the documents in an initially retrieved set by exploiting\nasymmetric relationships between them. Specifically, we consider generation\nlinks, which indicate that the language model induced from one document assigns\nhigh probability to the text of another; in doing so, we take care to prevent\nbias against long documents. We study a number of re-ranking criteria based on\nmeasures of centrality in the graphs formed by generation links, and show that\nintegrating centrality into standard language-model-based retrieval is quite\neffective at improving precision at top ranks.",
            "journal ref": "Proceedings of SIGIR 2005, pp. 306--313",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0601046v1",
            "date": "2006-01-11 21:47:44+00:00",
            "title": "Better than the real thing? Iterative pseudo-query processing using cluster-based language models",
            "authors": [
                "Oren Kurland",
                "Lillian Lee",
                "Carmel Domshlak"
            ],
            "abstract": "We present a novel approach to pseudo-feedback-based ad hoc retrieval that\nuses language models induced from both documents and clusters. First, we treat\nthe pseudo-feedback documents produced in response to the original query as a\nset of pseudo-queries that themselves can serve as input to the retrieval\nprocess. Observing that the documents returned in response to the\npseudo-queries can then act as pseudo-queries for subsequent rounds, we arrive\nat a formulation of pseudo-query-based retrieval as an iterative process.\nExperiments show that several concrete instantiations of this idea, when\napplied in conjunction with techniques designed to heighten precision, yield\nperformance results rivaling those of a number of previously-proposed\nalgorithms, including the standard language-modeling approach. The use of\ncluster-based language models is a key contributing factor to our algorithms'\nsuccess.",
            "journal ref": "Proceedings of SIGIR 2005, pp. 19--26",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/cs/0605121v1",
            "date": "2006-05-25 11:54:35+00:00",
            "title": "Communication of Social Agents and the Digital City - A Semiotic Perspective",
            "authors": [
                "Victor V. Kryssanov",
                "Masayuki Okabe",
                "Koh Kakusho",
                "Michihiko Minoh"
            ],
            "abstract": "This paper investigates the concept of digital city. First, a functional\nanalysis of a digital city is made in the light of the modern study of\nurbanism; similarities between the virtual and urban constructions are pointed\nout. Next, a semiotic perspective on the subject matter is elaborated, and a\nterminological basis is introduced to treat a digital city as a self-organizing\nmeaning-producing system intended to support social or spatial navigation. An\nexplicit definition of a digital city is formulated. Finally, the proposed\napproach is discussed, conclusions are given, and future work is outlined.",
            "journal ref": "Lecture Notes in Computer Science. 2002, Vol. 2362, 56-70",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/cs/0606017v1",
            "date": "2006-06-05 03:39:09+00:00",
            "title": "From semiotics of hypermedia to physics of semiosis: A view from system theory",
            "authors": [
                "V. V. Kryssanov",
                "K. Kakusho"
            ],
            "abstract": "Given that theoretical analysis and empirical validation is fundamental to\nany model, whether conceptual or formal, it is surprising that these two tools\nof scientific discovery are so often ignored in the contemporary studies of\ncommunication. In this paper, we pursued the ideas of a) correcting and\nexpanding the modeling approaches of linguistics, which are otherwise\ninapplicable (more precisely, which should not but are widely applied), to the\ngeneral case of hypermedia-based communication, and b) developing techniques\nfor empirical validation of semiotic models, which are nowadays routinely used\nto explore (in fact, to conjecture about) internal mechanisms of complex\nsystems, yet on a purely speculative basis. This study thus offers two\nexperimentally tested substantive contributions: the formal representation of\ncommunication as the mutually-orienting behavior of coupled autonomous systems,\nand the mathematical interpretation of the semiosis of communication, which\ntogether offer a concrete and parsimonious understanding of diverse\ncommunication phenomena.",
            "journal ref": "Semiotica. 2005, Vol.154-1/4, 11-38",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607062v3",
            "date": "2006-07-12 20:03:52+00:00",
            "title": "Get out the vote: Determining support or opposition from Congressional floor-debate transcripts",
            "authors": [
                "Matt Thomas",
                "Bo Pang",
                "Lillian Lee"
            ],
            "abstract": "We investigate whether one can determine from the transcripts of U.S.\nCongressional floor debates whether the speeches represent support of or\nopposition to proposed legislation. To address this problem, we exploit the\nfact that these speeches occur as part of a discussion; this allows us to use\nsources of information regarding relationships between discourse segments, such\nas whether a given utterance indicates agreement with the opinion expressed by\nanother. We find that the incorporation of such information yields substantial\nimprovements over classifying speeches in isolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0607120v1",
            "date": "2006-07-27 18:23:45+00:00",
            "title": "Expressing Implicit Semantic Relations without Supervision",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "We present an unsupervised learning algorithm that mines large text corpora\nfor patterns that express implicit semantic relations. For a given input word\npair X:Y with some unspecified semantic relations, the corresponding output\nlist of patterns <P1,...,Pm> is ranked according to how well each pattern Pi\nexpresses the relations between X and Y. For example, given X=ostrich and\nY=bird, the two highest ranking output patterns are \"X is the largest Y\" and \"Y\nsuch as the X\". The output patterns are intended to be useful for finding\nfurther pairs with the same relations, to support the construction of lexicons,\nontologies, and semantic networks. The patterns are sorted by pertinence, where\nthe pertinence of a pattern Pi for a word pair X:Y is the expected relational\nsimilarity between the given pair and typical pairs for Pi. The algorithm is\nempirically evaluated on two tasks, solving multiple-choice SAT word analogy\nquestions and classifying semantic relations in noun-modifier pairs. On both\ntasks, the algorithm achieves state-of-the-art results, performing\nsignificantly better than several alternative pattern ranking algorithms, based\non tf-idf.",
            "journal ref": "Proceedings of the 21st International Conference on Computational\n  Linguistics and 44th Annual Meeting of the Association for Computational\n  Linguistics (ACL-06), (2006), Sydney, Australia, 313-320",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0608100v1",
            "date": "2006-08-25 14:35:11+00:00",
            "title": "Similarity of Semantic Relations",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "There are at least two kinds of similarity. Relational similarity is\ncorrespondence between relations, in contrast with attributional similarity,\nwhich is correspondence between attributes. When two words have a high degree\nof attributional similarity, we call them synonyms. When two pairs of words\nhave a high degree of relational similarity, we say that their relations are\nanalogous. For example, the word pair mason:stone is analogous to the pair\ncarpenter:wood. This paper introduces Latent Relational Analysis (LRA), a\nmethod for measuring relational similarity. LRA has potential applications in\nmany areas, including information extraction, word sense disambiguation, and\ninformation retrieval. Recently the Vector Space Model (VSM) of information\nretrieval has been adapted to measuring relational similarity, achieving a\nscore of 47% on a collection of 374 college-level multiple-choice word analogy\nquestions. In the VSM approach, the relation between a pair of words is\ncharacterized by a vector of frequencies of predefined patterns in a large\ncorpus. LRA extends the VSM approach in three ways: (1) the patterns are\nderived automatically from the corpus, (2) the Singular Value Decomposition\n(SVD) is used to smooth the frequency data, and (3) automatically generated\nsynonyms are used to explore variations of the word pairs. LRA achieves 56% on\nthe 374 analogy questions, statistically equivalent to the average human score\nof 57%. On the related problem of classifying semantic relations, LRA achieves\nsimilar gains over the VSM.",
            "journal ref": "Computational Linguistics, (2006), 32(3), 379-416",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0612033v1",
            "date": "2006-12-06 10:13:12+00:00",
            "title": "Acronym-Meaning Extraction from Corpora Using Multi-Tape Weighted Finite-State Machines",
            "authors": [
                "Andr\u00e9 Kempe"
            ],
            "abstract": "The automatic extraction of acronyms and their meaning from corpora is an\nimportant sub-task of text mining. It can be seen as a special case of string\nalignment, where a text chunk is aligned with an acronym. Alternative\nalignments have different cost, and ideally the least costly one should give\nthe correct meaning of the acronym. We show how this approach can be\nimplemented by means of a 3-tape weighted finite-state machine (3-WFSM) which\nreads a text chunk on tape 1 and an acronym on tape 2, and generates all\nalternative alignments on tape 3. The 3-WFSM can be automatically generated\nfrom a simple regular expression. No additional algorithms are required at any\nstage. Our 3-WFSM has a size of 27 states and 64 transitions, and finds the\nbest analysis of an acronym in a few milliseconds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0612041v1",
            "date": "2006-12-07 08:42:46+00:00",
            "title": "Viterbi Algorithm Generalized for n-Tape Best-Path Search",
            "authors": [
                "Andr\u00e9 Kempe"
            ],
            "abstract": "We present a generalization of the Viterbi algorithm for identifying the path\nwith minimal (resp. maximal) weight in a n-tape weighted finite-state machine\n(n-WFSM), that accepts a given n-tuple of input strings (s_1,... s_n). It also\nallows us to compile the best transduction of a given input n-tuple by a\nweighted (n+m)-WFSM (transducer) with n input and m output tapes. Our algorithm\nhas a worst-case time complexity of O(|s|^n |E| log (|s|^n |Q|)), where n and\n|s| are the number and average length of the strings in the n-tuple, and |Q|\nand |E| the number of states and transitions in the n-WFSM, respectively. A\nstraight forward alternative, consisting in intersection followed by classical\nshortest-distance search, operates in O(|s|^n (|E|+|Q|) log (|s|^n |Q|)) time.",
            "journal ref": "Proc. FSMNLP 2009, Pretoria, South Africa. July 21-24. (improved\n  version).",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0602018v1",
            "date": "2006-02-06 15:17:34+00:00",
            "title": "Improving the CSIEC Project and Adapting It to the English Teaching and Learning in China",
            "authors": [
                "Jiyou Jia",
                "Shufen Hou",
                "Weichao Chen"
            ],
            "abstract": "In this paper after short review of the CSIEC project initialized by us in\n2003 we present the continuing development and improvement of the CSIEC project\nin details, including the design of five new Microsoft agent characters\nrepresenting different virtual chatting partners and the limitation of\nsimulated dialogs in specific practical scenarios like graduate job application\ninterview, then briefly analyze the actual conditions and features of its\napplication field: web-based English education in China. Finally we introduce\nour efforts to adapt this system to the requirements of English teaching and\nlearning in China and point out the work next to do.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/cs/0605101v1",
            "date": "2006-05-24 02:13:13+00:00",
            "title": "Modeling the Dynamics of Social Networks",
            "authors": [
                "Victor V. Kryssanov",
                "Frank J. Rinaldo",
                "Evgeny L. Kuleshov",
                "Hitoshi Ogawa"
            ],
            "abstract": "Modeling human dynamics responsible for the formation and evolution of the\nso-called social networks - structures comprised of individuals or\norganizations and indicating connectivities existing in a community - is a\ntopic recently attracting a significant research interest. It has been claimed\nthat these dynamics are scale-free in many practically important cases, such as\nimpersonal and personal communication, auctioning in a market, accessing sites\non the WWW, etc., and that human response times thus conform to the power law.\nWhile a certain amount of progress has recently been achieved in predicting the\ngeneral response rate of a human population, existing formal theories of human\nbehavior can hardly be found satisfactory to accommodate and comprehensively\nexplain the scaling observed in social networks. In the presented study, a\nnovel system-theoretic modeling approach is proposed and successfully applied\nto determine important characteristics of a communication network and to\nanalyze consumer behavior on the WWW.",
            "journal ref": "None",
            "category": "cs.CY"
        }
    ],
    "2007": [
        {
            "url": "http://arxiv.org/abs/cs/0701135v1",
            "date": "2007-01-22 00:45:31+00:00",
            "title": "Complex networks and human language",
            "authors": [
                "Jinyun KE"
            ],
            "abstract": "This paper introduces how human languages can be studied in light of recent\ndevelopment of network theories. There are two directions of exploration. One\nis to study networks existing in the language system. Various lexical networks\ncan be built based on different relationships between words, being semantic or\nsyntactic. Recent studies have shown that these lexical networks exhibit\nsmall-world and scale-free features. The other direction of exploration is to\nstudy networks of language users (i.e. social networks of people in the\nlinguistic community), and their role in language evolution. Social networks\nalso show small-world and scale-free features, which cannot be captured by\nrandom or regular network models. In the past, computational models of language\nchange and language emergence often assume a population to have a random or\nregular structure, and there has been little discussion how network structures\nmay affect the dynamics. In the second part of the paper, a series of\nsimulation models of diffusion of linguistic innovation are used to illustrate\nthe importance of choosing realistic conditions of population structure for\nmodeling language change. Four types of social networks are compared, which\nexhibit two categories of diffusion dynamics. While the questions about which\ntype of networks are more appropriate for modeling still remains, we give some\npreliminary suggestions for choosing the type of social networks for modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0701181v1",
            "date": "2007-01-27 19:09:53+00:00",
            "title": "A Note on Local Ultrametricity in Text",
            "authors": [
                "Fionn Murtagh"
            ],
            "abstract": "High dimensional, sparsely populated data spaces have been characterized in\nterms of ultrametric topology. This implies that there are natural, not\nnecessarily unique, tree or hierarchy structures defined by the ultrametric\ntopology. In this note we study the extent of local ultrametric topology in\ntexts, with the aim of finding unique ``fingerprints'' for a text or corpus,\ndiscriminating between texts from different domains, and opening up the\npossibility of exploiting hierarchical structures in the data. We use coherent\nand meaningful collections of over 1000 texts, comprising over 1.3 million\nwords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0701194v1",
            "date": "2007-01-30 16:58:07+00:00",
            "title": "Menzerath-Altmann Law for Syntactic Structures in Ukrainian",
            "authors": [
                "Solomija Buk",
                "Andrij Rovenchak"
            ],
            "abstract": "In the paper, the definition of clause suitable for an automated processing\nof a Ukrainian text is proposed. The Menzerath-Altmann law is verified on the\nsentence level and the parameters for the dependences of the clause length\ncounted in words and syllables on the sentence length counted in clauses are\ncalculated for \"Perekhresni Stezhky\" (\"The Cross-Paths\"), a novel by Ivan\nFranko.",
            "journal ref": "Glottotheory. Vol. 1, No. 1, pp 10-17 (2008)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0702081v1",
            "date": "2007-02-14 06:05:20+00:00",
            "title": "Random Sentences from a Generalized Phrase-Structure Grammar Interpreter",
            "authors": [
                "Rick Dale"
            ],
            "abstract": "In numerous domains in cognitive science it is often useful to have a source\nfor randomly generated corpora. These corpora may serve as a foundation for\nartificial stimuli in a learning experiment (e.g., Ellefson & Christiansen,\n2000), or as input into computational models (e.g., Christiansen & Dale, 2001).\nThe following compact and general C program interprets a phrase-structure\ngrammar specified in a text file. It follows parameters set at a Unix or\nUnix-based command-line and generates a corpus of random sentences from that\ngrammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0708.1564v1",
            "date": "2007-08-11 13:09:27+00:00",
            "title": "Learning Phonotactics Using ILP",
            "authors": [
                "Stasinos Konstantopoulos"
            ],
            "abstract": "This paper describes experiments on learning Dutch phonotactic rules using\nInductive Logic Programming, a machine learning discipline based on inductive\nlogical operators. Two different ways of approaching the problem are\nexperimented with, and compared against each other as well as with related work\non the task. The results show a direct correspondence between the quality and\ninformedness of the background knowledge and the constructed theory,\ndemonstrating the ability of ILP to take good advantage of the prior domain\nknowledge available. Further research is outlined.",
            "journal ref": "Special Issue of the WEB-SLS Journal: The Language Sections of the\n  ESSLLI-01 Student Session. 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0709.2401v1",
            "date": "2007-09-15 01:37:21+00:00",
            "title": "Bootstrapping Deep Lexical Resources: Resources for Courses",
            "authors": [
                "Timothy Baldwin"
            ],
            "abstract": "We propose a range of deep lexical acquisition methods which make use of\nmorphological, syntactic and ontological language resources to model word\nsimilarity and bootstrap from a seed lexicon. The different methods are\ndeployed in learning lexical items for a precision grammar, and shown to each\nhave strengths and weaknesses over different word classes. A particular focus\nof this paper is the relative accessibility of different language resource\ntypes, and predicted ``bang for the buck'' associated with each in deep lexical\nacquisition applications.",
            "journal ref": "In Proceedings of the ACL-SIGLEX 2005 Workshop on Deep Lexical\n  Acquisition, Ann Arbor, USA, pp. 67-76",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.0225v1",
            "date": "2007-10-01 08:23:24+00:00",
            "title": "On the role of autocorrelations in texts",
            "authors": [
                "D. V. Lande",
                "A. A. Snarskii"
            ],
            "abstract": "The task of finding a criterion allowing to distinguish a text from an\narbitrary set of words is rather relevant in itself, for instance, in the\naspect of development of means for internet-content indexing or separating\nsignals and noise in communication channels. The Zipf law is currently\nconsidered to be the most reliable criterion of this kind [3]. At any rate,\nconventional stochastic word sets do not meet this law. The present paper deals\nwith one of possible criteria based on the determination of the degree of data\ncompression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.0228v1",
            "date": "2007-10-01 08:31:56+00:00",
            "title": "On the fractal nature of mutual relevance sequences in the Internet news message flows",
            "authors": [
                "S. Braichevsky",
                "D. Lande",
                "A. Snarskii"
            ],
            "abstract": "In the task of information retrieval the term relevance is taken to mean\nformal conformity of a document given by the retrieval system to user's\ninformation query. As a rule, the documents found by the retrieval system\nshould be submitted to the user in a certain order. Therefore, a retrieval\nperceived as a selection of documents formally solving the user's query, should\nbe supplemented with a certain procedure of processing a relevant set. It would\nbe natural to introduce a quantitative measure of document conformity to query,\ni.e. the relevance measure. Since no single rule exists for the determination\nof the relevance measure, we shall consider two of them which are the simplest\nin our opinion. The proposed approach does not suppose any restrictions and can\nbe applied to other relevance measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.2852v1",
            "date": "2007-10-15 15:45:13+00:00",
            "title": "Generating models for temporal representations",
            "authors": [
                "Patrick Blackburn",
                "S\u00e9bastien Hinderer"
            ],
            "abstract": "We discuss the use of model building for temporal representations. We chose\nPolish to illustrate our discussion because it has an interesting aspectual\nsystem, but the points we wish to make are not language specific. Rather, our\ngoal is to develop theoretical and computational tools for temporal model\nbuilding tasks in computational semantics. To this end, we present a\nfirst-order theory of time and events which is rich enough to capture\ninteresting semantic distinctions, and an algorithm which takes minimal models\nfor first-order theories and systematically attempts to ``perturb'' their\ntemporal component to provide non-minimal, but semantically significant,\nmodels.",
            "journal ref": "Dans Recent Advances in Natural Language Processing (2007) 69-75",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.2988v1",
            "date": "2007-10-16 09:16:24+00:00",
            "title": "Using Description Logics for Recognising Textual Entailment",
            "authors": [
                "Paul Bedaride"
            ],
            "abstract": "The aim of this paper is to show how we can handle the Recognising Textual\nEntailment (RTE) task by using Description Logics (DLs). To do this, we propose\na representation of natural language semantics in DLs inspired by existing\nrepresentations in first-order logic. But our most significant contribution is\nthe definition of two novel inference tasks: A-Box saturation and subgraph\ndetection which are crucial for our approach to RTE.",
            "journal ref": "Dans 19th European Summer School in Logic, Language and\n  Information (2007) 11-21",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.5382v1",
            "date": "2007-10-29 10:48:48+00:00",
            "title": "Some Reflections on the Task of Content Determination in the Context of Multi-Document Summarization of Evolving Events",
            "authors": [
                "Stergos D. Afantenos"
            ],
            "abstract": "Despite its importance, the task of summarizing evolving events has received\nsmall attention by researchers in the field of multi-document summariztion. In\na previous paper (Afantenos et al. 2007) we have presented a methodology for\nthe automatic summarization of documents, emitted by multiple sources, which\ndescribe the evolution of an event. At the heart of this methodology lies the\nidentification of similarities and differences between the various documents,\nin two axes: the synchronic and the diachronic. This is achieved by the\nintroduction of the notion of Synchronic and Diachronic Relations. Those\nrelations connect the messages that are found in the documents, resulting thus\nin a graph which we call grid. Although the creation of the grid completes the\nDocument Planning phase of a typical NLG architecture, it can be the case that\nthe number of messages contained in a grid is very large, exceeding thus the\nrequired compression rate. In this paper we provide some initial thoughts on a\nprobabilistic model which can be applied at the Content Determination stage,\nand which tries to alleviate this problem.",
            "journal ref": "Edited by Galia Angelova, Kalina Bontcheva, Ruslan Mitkov, Nicolas\n  Nicolov, and Nikolai Nikolov, Recent Advances in Natural Language Processing\n  (RANLP 2007). Borovets, Bulgaria: INCOMA, 12-16",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.0666v1",
            "date": "2007-11-05 15:20:47+00:00",
            "title": "Discriminative Phoneme Sequences Extraction for Non-Native Speaker's Origin Classification",
            "authors": [
                "Ghazi Bouselmi",
                "Dominique Fohr",
                "Irina Illina",
                "Jean-Paul Haton"
            ],
            "abstract": "In this paper we present an automated method for the classification of the\norigin of non-native speakers. The origin of non-native speakers could be\nidentified by a human listener based on the detection of typical pronunciations\nfor each nationality. Thus we suppose the existence of several phoneme\nsequences that might allow the classification of the origin of non-native\nspeakers. Our new method is based on the extraction of discriminative sequences\nof phonemes from a non-native English speech database. These sequences are used\nto construct a probabilistic classifier for the speakers' origin. The existence\nof discriminative phone sequences in non-native speech is a significant result\nof this work. The system that we have developed achieved a significant correct\nclassification rate of 96.3% and a significant error reduction compared to some\nother tested techniques.",
            "journal ref": "Dans ISSPA, International Symposium on Signal Processing and its\n  Applications (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.0811v1",
            "date": "2007-11-06 08:23:49+00:00",
            "title": "Combined Acoustic and Pronunciation Modelling for Non-Native Speech Recognition",
            "authors": [
                "Ghazi Bouselmi",
                "Dominique Fohr",
                "Irina Illina"
            ],
            "abstract": "In this paper, we present several adaptation methods for non-native speech\nrecognition. We have tested pronunciation modelling, MLLR and MAP non-native\npronunciation adaptation and HMM models retraining on the HIWIRE foreign\naccented English speech database. The ``phonetic confusion'' scheme we have\ndeveloped consists in associating to each spoken phone several sequences of\nconfused phones. In our experiments, we have used different combinations of\nacoustic models representing the canonical and the foreign pronunciations:\nspoken and native models, models adapted to the non-native accent with MAP and\nMLLR. The joint use of pronunciation modelling and acoustic adaptation led to\nfurther improvements in recognition accuracy. The best combination of the above\nmentioned techniques resulted in a relative word error reduction ranging from\n46% to 71%.",
            "journal ref": "Dans InterSpeech 2007 (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.1038v1",
            "date": "2007-11-07 08:51:09+00:00",
            "title": "Am\u00e9lioration des Performances des Syst\u00e8mes Automatiques de Reconnaissance de la Parole pour la Parole Non Native",
            "authors": [
                "Ghazi Bouselmi",
                "Dominique Fohr",
                "Irina Illina",
                "Jean-Paul Haton"
            ],
            "abstract": "In this article, we present an approach for non native automatic speech\nrecognition (ASR). We propose two methods to adapt existing ASR systems to the\nnon-native accents. The first method is based on the modification of acoustic\nmodels through integration of acoustic models from the mother tong. The\nphonemes of the target language are pronounced in a similar manner to the\nnative language of speakers. We propose to combine the models of confused\nphonemes so that the ASR system could recognize both concurrent\npronounciations. The second method we propose is a refinment of the\npronounciation error detection through the introduction of graphemic\nconstraints. Indeed, non native speakers may rely on the writing of words in\ntheir uttering. Thus, the pronounctiation errors might depend on the characters\ncomposing the words. The average error rate reduction that we observed is\n(22.5%) relative for the sentence error rate, and 34.5% (relative) in word\nerror rate.",
            "journal ref": "Dans TAIMA'07, Traitement et Analyse de l'Information : M\\'ethodes\n  et Applications (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.2444v1",
            "date": "2007-11-15 15:39:48+00:00",
            "title": "Proof nets for display logic",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "This paper explores several extensions of proof nets for the Lambek calculus\nin order to handle the different connectives of display logic in a natural way.\nThe new proof net calculus handles some recent additions to the Lambek\nvocabulary such as Galois connections and Grishin interactions. It concludes\nwith an exploration of the generative capacity of the Lambek-Grishin calculus,\npresenting an embedding of lexicalized tree adjoining grammars into the\nLambek-Grishin calculus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3412v1",
            "date": "2007-11-21 16:47:57+00:00",
            "title": "Morphological annotation of Korean with Directly Maintainable Resources",
            "authors": [
                "Ivan Berlocher",
                "Hyun-Gue Huh",
                "Eric Laporte",
                "Jee-Sun Nam"
            ],
            "abstract": "This article describes an exclusively resource-based method of morphological\nannotation of written Korean text. Korean is an agglutinative language. Our\nannotator is designed to process text before the operation of a syntactic\nparser. In its present state, it annotates one-stem words only. The output is a\ngraph of morphemes annotated with accurate linguistic information. The\ngranularity of the tagset is 3 to 5 times higher than usual tagsets. A\ncomparison with a reference annotated corpus showed that it achieves 89% recall\nwithout any corpus training. The language resources used by the system are\nlexicons of stems, transducers of suffixes and transducers of generation of\nallomorphs. All can be easily updated, which allows users to control the\nevolution of the performances of the system. It has been claimed that\nmorphological annotation of Korean text could only be performed by a\nmorphological analysis module accessing a lexicon of morphemes. We show that it\ncan also be performed directly with a lexicon of words and without applying\nmorphological rules at annotation time, which speeds up annotation to 1,210\nword/s. The lexicon of words is obtained from the maintainable language\nresources through a fully automated compilation process.",
            "journal ref": "Dans Proceedings of the Language Resource and Evaluation\n  Consference (LREC) - Morphological annotation of Korean with Directly\n  Maintainable Resources, Genoa : Italie (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3449v1",
            "date": "2007-11-21 20:34:08+00:00",
            "title": "Lexicon management and standard formats",
            "authors": [
                "Eric Laporte"
            ],
            "abstract": "International standards for lexicon formats are in preparation. To a certain\nextent, the proposed formats converge with prior results of standardization\nprojects. However, their adequacy for (i) lexicon management and (ii)\nlexicon-driven applications have been little debated in the past, nor are they\nas a part of the present standardization effort. We examine these issues. IGM\nhas developed XML formats compatible with the emerging international standards,\nand we report experimental results on large-coverage lexica.",
            "journal ref": "Archives of Control Sciences 15, 3 (2005) 329-340",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3452v1",
            "date": "2007-11-21 20:38:29+00:00",
            "title": "In memoriam Maurice Gross",
            "authors": [
                "Eric Laporte"
            ],
            "abstract": "Maurice Gross (1934-2001) was both a great linguist and a pioneer in natural\nlanguage processing. This article is written in homage to his memory",
            "journal ref": "Archives of Control Sciences 15, 3 (2005) 257-278",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3453v1",
            "date": "2007-11-21 20:41:59+00:00",
            "title": "A resource-based Korean morphological annotation system",
            "authors": [
                "Hyun-Gue Huh",
                "Eric Laporte"
            ],
            "abstract": "We describe a resource-based method of morphological annotation of written\nKorean text. Korean is an agglutinative language. The output of our system is a\ngraph of morphemes annotated with accurate linguistic information. The language\nresources used by the system can be easily updated, which allows us-ers to\ncontrol the evolution of the per-formances of the system. We show that\nmorphological annotation of Korean text can be performed directly with a\nlexicon of words and without morpho-logical rules.",
            "journal ref": "Dans Proceedings of the International Joint Conference on Natural\n  Language Processing (IJCNLP) - A resource-based Korean morphological\n  annotation system, Jeju : Cor\\'ee, R\\'epublique de (2005)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3454v1",
            "date": "2007-11-21 20:44:04+00:00",
            "title": "Graphes param\u00e9tr\u00e9s et outils de lexicalisation",
            "authors": [
                "Eric Laporte",
                "S\u00e9bastien Paumier"
            ],
            "abstract": "Shifting to a lexicalized grammar reduces the number of parsing errors and\nimproves application results. However, such an operation affects a syntactic\nparser in all its aspects. One of our research objectives is to design a\nrealistic model for grammar lexicalization. We carried out experiments for\nwhich we used a grammar with a very simple content and formalism, and a very\ninformative syntactic lexicon, the lexicon-grammar of French elaborated by the\nLADL. Lexicalization was performed by applying the parameterized-graph\napproach. Our results tend to show that most information in the lexicon-grammar\ncan be transferred into a grammar and exploited successfully for the syntactic\nparsing of sentences.",
            "journal ref": "Dans Verbum ex machina. Proceedings of TALN - Graphes\n  param\\'etr\\'es et outils de lexicalisation, Louvain : Belgique (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3457v1",
            "date": "2007-11-21 20:49:21+00:00",
            "title": "Evaluation of a Grammar of French Determiners",
            "authors": [
                "Eric Laporte"
            ],
            "abstract": "Existing syntactic grammars of natural languages, even with a far from\ncomplete coverage, are complex objects. Assessments of the quality of parts of\nsuch grammars are useful for the validation of their construction. We evaluated\nthe quality of a grammar of French determiners that takes the form of a\nrecursive transition network. The result of the application of this local\ngrammar gives deeper syntactic information than chunking or information\navailable in treebanks. We performed the evaluation by comparison with a corpus\nindependently annotated with information on determiners. We obtained 86%\nprecision and 92% recall on text not tagged for parts of speech.",
            "journal ref": "Dans Annals of the 27th Congress of the Brazilian Society of\n  Computation - Evaluation of a Grammar of French Determiners, Rio de Janeiro :\n  Br\\'esil (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3605v1",
            "date": "2007-11-22 15:54:31+00:00",
            "title": "Very strict selectional restrictions",
            "authors": [
                "Eric Laporte",
                "Christian Lecl\u00e8re",
                "Maria Carmelita P. Dias"
            ],
            "abstract": "We discuss the characteristics and behaviour of two parallel classes of verbs\nin two Romance languages, French and Portuguese. Examples of these verbs are\nPort. abater [gado] and Fr. abattre [b\\'etail], both meaning \"slaughter\n[cattle]\". In both languages, the definition of the class of verbs includes\nseveral features: - They have only one essential complement, which is a direct\nobject. - The nominal distribution of the complement is very limited, i.e., few\nnouns can be selected as head nouns of the complement. However, this selection\nis not restricted to a single noun, as would be the case for verbal idioms such\nas Fr. monter la garde \"mount guard\". - We excluded from the class\nconstructions which are reductions of more complex constructions, e.g. Port.\nafinar [instrumento] com \"tune [instrument] with\".",
            "journal ref": "Dans Proceedings - Very strict selectional restrictions. A\n  Comparison between Portuguese and French, Itatiaia : Br\\'esil (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3691v2",
            "date": "2007-11-23 09:45:13+00:00",
            "title": "Outilex, plate-forme logicielle de traitement de textes \u00e9crits",
            "authors": [
                "Olivier Blanc",
                "Matthieu Constant",
                "Eric Laporte"
            ],
            "abstract": "The Outilex software platform, which will be made available to research,\ndevelopment and industry, comprises software components implementing all the\nfundamental operations of written text processing: processing without lexicons,\nexploitation of lexicons and grammars, language resource management. All data\nare structured in XML formats, and also in more compact formats, either\nreadable or binary, whenever necessary; the required format converters are\nincluded in the platform; the grammar formats allow for combining statistical\napproaches with resource-based approaches. Manually constructed lexicons for\nFrench and English, originating from the LADL, and of substantial coverage,\nwill be distributed with the platform under LGPL-LR license.",
            "journal ref": "Dans Verbum ex machina. Proceedings of TALN - Outilex, plate-forme\n  logicielle de traitement de textes \\'ecrits, Louvain : Belgique (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3726v1",
            "date": "2007-11-23 13:44:55+00:00",
            "title": "Let's get the student into the driver's seat",
            "authors": [
                "Michael Zock",
                "Stergos D. Afantenos"
            ],
            "abstract": "Speaking a language and achieving proficiency in another one is a highly\ncomplex process which requires the acquisition of various kinds of knowledge\nand skills, like the learning of words, rules and patterns and their connection\nto communicative goals (intentions), the usual starting point. To help the\nlearner to acquire these skills we propose an enhanced, electronic version of\nan age old method: pattern drills (henceforth PDs). While being highly regarded\nin the fifties, PDs have become unpopular since then, partially because of\ntheir lack of grounding (natural context) and rigidity. Despite these\nshortcomings we do believe in the virtues of this approach, at least with\nregard to the acquisition of basic linguistic reflexes or skills (automatisms),\nnecessary to survive in the new language. Of course, the method needs\nimprovement, and we will show here how this can be achieved. Unlike tapes or\nbooks, computers are open media, allowing for dynamic changes, taking users'\nperformances and preferences into account. Building an electronic version of\nPDs amounts to building an open resource, accomodatable to the users' ever\nchanging needs.",
            "journal ref": "The Seventh International Symposium on Natural Language Processing\n  (SNLP 2007). Chonburi, Thailand",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.4475v6",
            "date": "2007-11-28 12:16:08+00:00",
            "title": "Valence extraction using EM selection and co-occurrence matrices",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "This paper discusses two new procedures for extracting verb valences from raw\ntexts, with an application to the Polish language. The first novel technique,\nthe EM selection algorithm, performs unsupervised disambiguation of valence\nframe forests, obtained by applying a non-probabilistic deep grammar parser and\nsome post-processing to the text. The second new idea concerns filtering of\nincorrect frames detected in the parsed text and is motivated by an observation\nthat verbs which take similar arguments tend to have similar frames. This\nphenomenon is described in terms of newly introduced co-occurrence matrices.\nUsing co-occurrence matrices, we split filtering into two steps. The list of\nvalid arguments is first determined for each verb, whereas the pattern\naccording to which the arguments are combined into frames is computed in the\nfollowing stage. Our best extracted dictionary reaches an $F$-score of 45%,\ncompared to an $F$-score of 39% for the standard frame-based BHT filtering.",
            "journal ref": "Language Resources and Evaluation 43:301-327, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0712.3705v1",
            "date": "2007-12-21 08:55:17+00:00",
            "title": "Framework and Resources for Natural Language Parser Evaluation",
            "authors": [
                "Tuomo Kakkonen"
            ],
            "abstract": "Because of the wide variety of contemporary practices used in the automatic\nsyntactic parsing of natural languages, it has become necessary to analyze and\nevaluate the strengths and weaknesses of different approaches. This research is\nall the more necessary because there are currently no genre- and\ndomain-independent parsers that are able to analyze unrestricted text with 100%\npreciseness (I use this term to refer to the correctness of analyses assigned\nby a parser). All these factors create a need for methods and resources that\ncan be used to evaluate and compare parsing systems. This research describes:\n(1) A theoretical analysis of current achievements in parsing and parser\nevaluation. (2) A framework (called FEPa) that can be used to carry out\npractical parser evaluations and comparisons. (3) A set of new evaluation\nresources: FiEval is a Finnish treebank under construction, and MGTS and RobSet\nare parser evaluation resources in English. (4) The results of experiments in\nwhich the developed evaluation framework and the two resources for English were\nused for evaluating a set of selected parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0704.3708v2",
            "date": "2007-04-27 17:13:37+00:00",
            "title": "Network statistics on early English Syntax: Structural criteria",
            "authors": [
                "Bernat Corominas-Murtra"
            ],
            "abstract": "This paper includes a reflection on the role of networks in the study of\nEnglish language acquisition, as well as a collection of practical criteria to\nannotate free-speech corpora from children utterances. At the theoretical\nlevel, the main claim of this paper is that syntactic networks should be\ninterpreted as the outcome of the use of the syntactic machinery. Thus, the\nintrinsic features of such machinery are not accessible directly from (known)\nnetwork properties. Rather, what one can see are the global patterns of its use\nand, thus, a global view of the power and organization of the underlying\ngrammar. Taking a look into more practical issues, the paper examines how to\nbuild a net from the projection of syntactic relations. Recall that, as opposed\nto adult grammars, early-child language has not a well-defined concept of\nstructure. To overcome such difficulty, we develop a set of systematic criteria\nassuming constituency hierarchy and a grammar based on lexico-thematic\nrelations. At the end, what we obtain is a well defined corpora annotation that\nenables us i) to perform statistics on the size of structures and ii) to build\na network from syntactic relations over which we can perform the standard\nmeasures of complexity. We also provide a detailed example.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0707.3269v1",
            "date": "2007-07-22 15:24:48+00:00",
            "title": "International Standard for a Linguistic Annotation Framework",
            "authors": [
                "Laurent Romary",
                "Nancy Ide"
            ],
            "abstract": "This paper describes the Linguistic Annotation Framework under development\nwithin ISO TC37 SC4 WG1. The Linguistic Annotation Framework is intended to\nserve as a basis for harmonizing existing language resources as well as\ndeveloping new ones.",
            "journal ref": "Natural Language Engineering 10, 3-4 (09/2004) 211-225",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0707.3270v1",
            "date": "2007-07-22 15:25:27+00:00",
            "title": "A Formal Model of Dictionary Structure and Content",
            "authors": [
                "Laurent Romary",
                "Nancy Ide",
                "Adam Kilgarriff"
            ],
            "abstract": "We show that a general model of lexical information conforms to an abstract\nmodel that reflects the hierarchy of information found in a typical dictionary\nentry. We show that this model can be mapped into a well-formed XML document,\nand how the XSL transformation language can be used to implement a semantics\ndefined over the abstract model to enable extraction and manipulation of the\ninformation in any format.",
            "journal ref": "Dans Euralex 2000 Euralex 2000, Stuttgart : Allemagne (2000)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0703027v2",
            "date": "2007-03-06 19:50:37+00:00",
            "title": "Interroger un corpus par le sens",
            "authors": [
                "Bernard Jacquemin"
            ],
            "abstract": "In textual knowledge management, statistical methods prevail. Nonetheless,\nsome difficulties cannot be overcome by these methodologies. I propose a\nsymbolic approach using a complete textual analysis to identify which analysis\nlevel can improve the the answers provided by a system. The approach identifies\nword senses and relation between words and generates as many rephrasings as\npossible. Using synonyms and derivative, the system provides new utterances\nwithout changing the original meaning of the sentences. Such a way, an\ninformation can be retrieved whatever the question or answer's wording may be.",
            "journal ref": "Dans \"Mots, termes et contextes\", Actes des septi\\`emes Journ\\'ees\n  scientifiques du r\\'eseau de chercheurs Lexicologie, Terminologie, Traduction\n  - Bruxelles : Belgique (2005)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0703049v1",
            "date": "2007-03-10 23:59:55+00:00",
            "title": "Algorithm of Segment-Syllabic Synthesis in Speech Recognition Problem",
            "authors": [
                "Oleg N. Karpov",
                "Olga A. Savenkova"
            ],
            "abstract": "Speech recognition based on the syllable segment is discussed in this paper.\nThe principal search methods in space of states for the speech recognition\nproblem by segment-syllabic parameters trajectory synthesis are investigated.\nRecognition as comparison the parameters trajectories in chosen speech units on\nthe sections of the segmented speech is realized. Some experimental results are\ngiven and discussed.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/cs/0703135v1",
            "date": "2007-03-27 22:12:25+00:00",
            "title": "Dependency Parsing with Dynamic Bayesian Network",
            "authors": [
                "Virginia Savova",
                "Leonid Peshkin"
            ],
            "abstract": "Exact parsing with finite state automata is deemed inappropriate because of\nthe unbounded non-locality languages overwhelmingly exhibit. We propose a way\nto structure the parsing task in order to make it amenable to local\nclassification methods. This allows us to build a Dynamic Bayesian Network\nwhich uncovers the syntactic dependency structure of English sentences.\nExperiments with the Wall Street Journal demonstrate that the model\nsuccessfully learns from labeled data.",
            "journal ref": "In proceedings of American Association for Artificial Intelligence\n  AAAI 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0705.0462v1",
            "date": "2007-05-03 13:44:54+00:00",
            "title": "Resource modalities in game semantics",
            "authors": [
                "Paul-Andr\u00e9 Melli\u00e8s",
                "Nicolas Tabareau"
            ],
            "abstract": "The description of resources in game semantics has never achieved the\nsimplicity and precision of linear logic, because of a misleading conception:\nthe belief that linear logic is more primitive than game semantics. We advocate\ninstead the contrary: that game semantics is conceptually more primitive than\nlinear logic. Starting from this revised point of view, we design a categorical\nmodel of resources in game semantics, and construct an arena game model where\nthe usual notion of bracketing is extended to multi- bracketing in order to\ncapture various resource policies: linear, af&#64257;ne and exponential.",
            "journal ref": "None",
            "category": "math.CT"
        },
        {
            "url": "http://arxiv.org/abs/0705.1161v1",
            "date": "2007-05-08 20:08:13+00:00",
            "title": "IDF revisited: A simple new derivation within the Robertson-Sp\u00e4rck Jones probabilistic model",
            "authors": [
                "Lillian Lee"
            ],
            "abstract": "There have been a number of prior attempts to theoretically justify the\neffectiveness of the inverse document frequency (IDF). Those that take as their\nstarting point Robertson and Sparck Jones's probabilistic model are based on\nstrong or complex assumptions. We show that a more intuitively plausible\nassumption suffices. Moreover, the new assumption, while conceptually very\nsimple, provides a solution to an estimation problem that had been deemed\nintractable by Robertson and Walker (1997).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0705.4676v8",
            "date": "2007-05-31 18:41:28+00:00",
            "title": "Recursive n-gram hashing is pairwise independent, at best",
            "authors": [
                "Daniel Lemire",
                "Owen Kaser"
            ],
            "abstract": "Many applications use sequences of n consecutive symbols (n-grams). Hashing\nthese n-grams can be a performance bottleneck. For more speed, recursive hash\nfamilies compute hash values by updating previous values. We prove that\nrecursive hash families cannot be more than pairwise independent. While hashing\nby irreducible polynomials is pairwise independent, our implementations either\nrun in time O(n) or use an exponential amount of memory. As a more scalable\nalternative, we make hashing by cyclic polynomials pairwise independent by\nignoring n-1 bits. Experimentally, we show that hashing by cyclic polynomials\nis is twice as fast as hashing by irreducible polynomials. We also show that\nrandomized Karp-Rabin hash families are not pairwise independent.",
            "journal ref": "Computer Speech & Language 24(4): 698-710 (2010)",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/0708.2303v2",
            "date": "2007-08-17 01:15:11+00:00",
            "title": "Compositional Semantics Grounded in Commonsense Metaphysics",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "We argue for a compositional semantics grounded in a strongly typed ontology\nthat reflects our commonsense view of the world and the way we talk about it in\nordinary language. Assuming the existence of such a structure, we show that the\nsemantics of various natural language phenomena may become nearly trivial.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0709.0116v2",
            "date": "2007-09-02 17:00:40+00:00",
            "title": "On Ultrametric Algorithmic Information",
            "authors": [
                "Fionn Murtagh"
            ],
            "abstract": "How best to quantify the information of an object, whether natural or\nartifact, is a problem of wide interest. A related problem is the computability\nof an object. We present practical examples of a new way to address this\nproblem. By giving an appropriate representation to our objects, based on a\nhierarchical coding of information, we exemplify how it is remarkably easy to\ncompute complex objects. Our algorithmic complexity is related to the length of\nthe class of objects, rather than to the length of the object.",
            "journal ref": "Computer Journal, 53, 405-416, 2010",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0710.0105v2",
            "date": "2007-09-30 03:21:54+00:00",
            "title": "Zipf's Law and Avoidance of Excessive Synonymy",
            "authors": [
                "Dmitrii Manin"
            ],
            "abstract": "Zipf's law states that if words of language are ranked in the order of\ndecreasing frequency in texts, the frequency of a word is inversely\nproportional to its rank. It is very robust as an experimental observation, but\nto date it escaped satisfactory theoretical explanation. We suggest that Zipf's\nlaw may arise from the evolution of word semantics dominated by expansion of\nmeanings and competition of synonyms.",
            "journal ref": "Main text in Cognitive Science, 32 (7) 2008, pp. 1075 - 1098;\n  Appendix A TBP separately in J. Quant. Ling.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.0169v6",
            "date": "2007-10-01 16:04:52+00:00",
            "title": "Evaluation experiments on related terms search in Wikipedia: Information Content and Adapted HITS (In Russian)",
            "authors": [
                "A. A. Krizhanovsky"
            ],
            "abstract": "The classification of metrics and algorithms search for related terms via\nWordNet, Roget's Thesaurus, and Wikipedia was extended to include adapted HITS\nalgorithm. Evaluation experiments on Information Content and adapted HITS\nalgorithm are described. The test collection of Russian word pairs with\nhuman-assigned similarity judgments is proposed.\n  -----\n  Klassifikacija metrik i algoritmov poiska semanticheski blizkih slov v\ntezaurusah WordNet, Rozhe i jenciklopedii Vikipedija rasshirena adaptirovannym\nHITS algoritmom. S pomow'ju jeksperimentov v Vikipedii oceneny metrika\nInformation Content i adaptirovannyj algoritm HITS. Predlozhen resurs dlja\nocenki semanticheskoj blizosti russkih slov.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0710.1481v1",
            "date": "2007-10-08 08:36:32+00:00",
            "title": "What's in a Name?",
            "authors": [
                "Stasinos Konstantopoulos"
            ],
            "abstract": "This paper describes experiments on identifying the language of a single name\nin isolation or in a document written in a different language. A new corpus has\nbeen compiled and made available, matching names against languages. This corpus\nis used in a series of experiments measuring the performance of general\nlanguage models and names-only language models on the language identification\ntask. Conclusions are drawn from the comparison between using general language\nmodels and names-only language models and between identifying the language of\nisolated names and the language of very short document fragments. Future\nresearch directions are outlined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.3285v2",
            "date": "2007-10-17 12:42:39+00:00",
            "title": "Nontraditional Scoring of C-tests",
            "authors": [
                "Tretjakova Tamara"
            ],
            "abstract": "In C-tests the hypothesis of items local independence is violated, which\ndoesn't permit to consider them as real tests. It is suggested to determine the\ndistances between separate C-test items (blanks) and to combine items into\nclusters. Weights, inversely proportional to the number of items in\ncorresponding clusters, are assigned to items. As a result, the C-test\nstructure becomes similar to the structure of classical tests, without\nviolation of local independence hypothesis.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/0710.3502v1",
            "date": "2007-10-18 13:24:26+00:00",
            "title": "Using Synchronic and Diachronic Relations for Summarizing Multiple Documents Describing Evolving Events",
            "authors": [
                "Stergos D. Afantenos",
                "V. Karkaletsis",
                "P. Stamatopoulos",
                "C. Halatsis"
            ],
            "abstract": "In this paper we present a fresh look at the problem of summarizing evolving\nevents from multiple sources. After a discussion concerning the nature of\nevolving events we introduce a distinction between linearly and non-linearly\nevolving events. We present then a general methodology for the automatic\ncreation of summaries from evolving events. At its heart lie the notions of\nSynchronic and Diachronic cross-document Relations (SDRs), whose aim is the\nidentification of similarities and differences between sources, from a\nsynchronical and diachronical perspective. SDRs do not connect documents or\ntextual elements found therein, but structures one might call messages.\nApplying this methodology will yield a set of messages and relations, SDRs,\nconnecting them, that is a graph which we call grid. We will show how such a\ngrid can be considered as the starting point of a Natural Language Generation\nSystem. The methodology is evaluated in two case-studies, one for linearly\nevolving events (descriptions of football matches) and another one for\nnon-linearly evolving events (terrorist incidents involving hostages). In both\ncases we evaluate the results produced by our computational systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.1360v1",
            "date": "2007-11-08 21:05:38+00:00",
            "title": "Analytical approach to bit-string models of language evolution",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "A formulation of bit-string models of language evolution, based on\ndifferential equations for the population speaking each language, is introduced\nand preliminarily studied. Connections with replicator dynamics and diffusion\nprocesses are pointed out. The stability of the dominance state, where most of\nthe population speaks a single language, is analyzed within a mean-field-like\napproximation, while the homogeneous state, where the population is evenly\ndistributed among languages, can be exactly studied. This analysis discloses\nthe existence of a bistability region, where dominance coexists with\nhomogeneity as possible asymptotic states. Numerical resolution of the\ndifferential system validates these findings.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0712.1529v2",
            "date": "2007-12-01 14:27:12+00:00",
            "title": "Ontology and Formal Semantics - Integration Overdue",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "In this note we suggest that difficulties encountered in natural language\nsemantics are, for the most part, due to the use of mere symbol manipulation\nsystems that are devoid of any content. In such systems, where there is hardly\nany link with our common-sense view of the world, and it is quite difficult to\nenvision how one can formally account for the considerable amount of content\nthat is often implicit, but almost never explicitly stated in our everyday\ndiscourse. The solution, in our opinion, is a compositional semantics grounded\nin an ontology that reflects our commonsense view of the world and the way we\ntalk about it in ordinary language. In the compositional logic we envision\nthere are ontological (or first-intension) concepts, and logical (or\nsecond-intension) concepts, and where the ontological concepts include not only\nDavidsonian events, but other abstract objects as well (e.g., states,\nprocesses, properties, activities, attributes, etc.) It will be demonstrated\nhere that in such a framework, a number of challenges in the semantics of\nnatural language (e.g., metonymy, intensionality, metaphor, etc.) can be\nproperly and uniformly addressed.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0712.3298v1",
            "date": "2007-12-19 22:20:40+00:00",
            "title": "CLAIRLIB Documentation v1.03",
            "authors": [
                "Dragomir Radev",
                "Mark Hodges",
                "Anthony Fader",
                "Mark Joseph",
                "Joshua Gerrish",
                "Mark Schaller",
                "Jonathan dePeri",
                "Bryan Gibson"
            ],
            "abstract": "The Clair library is intended to simplify a number of generic tasks in\nNatural Language Processing (NLP), Information Retrieval (IR), and Network\nAnalysis. Its architecture also allows for external software to be plugged in\nwith very little effort. Functionality native to Clairlib includes\nTokenization, Summarization, LexRank, Biased LexRank, Document Clustering,\nDocument Indexing, PageRank, Biased PageRank, Web Graph Analysis, Network\nGeneration, Power Law Distribution Analysis, Network Analysis (clustering\ncoefficient, degree distribution plotting, average shortest path, diameter,\ntriangles, shortest path matrices, connected components), Cosine Similarity,\nRandom Walks on Graphs, Statistics (distributions, tests), Tf, Idf, Community\nFinding.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0704.2083v1",
            "date": "2007-04-17 01:04:01+00:00",
            "title": "Introduction to Arabic Speech Recognition Using CMUSphinx System",
            "authors": [
                "H. Satori",
                "M. Harti",
                "N. Chenfour"
            ],
            "abstract": "In this paper Arabic was investigated from the speech recognition problem\npoint of view. We propose a novel approach to build an Arabic Automated Speech\nRecognition System (ASR). This system is based on the open source CMU Sphinx-4,\nfrom the Carnegie Mellon University. CMU Sphinx is a large-vocabulary;\nspeaker-independent, continuous speech recognition system based on discrete\nHidden Markov Models (HMMs). We build a model using utilities from the\nOpenSource CMU Sphinx. We will demonstrate the possible adaptability of this\nsystem to Arabic voice recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0704.2201v1",
            "date": "2007-04-17 17:04:26+00:00",
            "title": "Arabic Speech Recognition System using CMU-Sphinx4",
            "authors": [
                "H. Satori",
                "M. Harti",
                "N. Chenfour"
            ],
            "abstract": "In this paper we present the creation of an Arabic version of Automated\nSpeech Recognition System (ASR). This system is based on the open source\nSphinx-4, from the Carnegie Mellon University. Which is a speech recognition\nsystem based on discrete hidden Markov models (HMMs). We investigate the\nchanges that must be made to the model to adapt Arabic voice recognition.\n  Keywords: Speech recognition, Acoustic model, Arabic language, HMMs,\nCMUSphinx-4, Artificial intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0704.3662v1",
            "date": "2007-04-27 05:34:10+00:00",
            "title": "An Automated Evaluation Metric for Chinese Text Entry",
            "authors": [
                "Mike Tian-Jian Jiang",
                "James Zhan",
                "Jaimie Lin",
                "Jerry Lin",
                "Wen-Lien Hsu"
            ],
            "abstract": "In this paper, we propose an automated evaluation metric for text entry. We\nalso consider possible improvements to existing text entry evaluation metrics,\nsuch as the minimum string distance error rate, keystrokes per character, cost\nper correction, and a unified approach proposed by MacKenzie, so they can\naccommodate the special characteristics of Chinese text. Current methods lack\nan integrated concern about both typing speed and accuracy for Chinese text\nentry evaluation. Our goal is to remove the bias that arises due to human\nfactors. First, we propose a new metric, called the correction penalty (P),\nbased on Fitts' law and Hick's law. Next, we transform it into the approximate\namortized cost (AAC) of information theory. An analysis of the AAC of Chinese\ntext input methods with different context lengths is also presented.",
            "journal ref": "Jiang, Mike Tian-Jian, et al. \"Robustness analysis of adaptive\n  chinese input methods.\" Advances in Text Input Methods (WTIM 2011) (2011): 53",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/0704.3665v1",
            "date": "2007-04-27 05:58:32+00:00",
            "title": "On the Development of Text Input Method - Lessons Learned",
            "authors": [
                "Mike Tian-Jian Jiang",
                "Deng Liu",
                "Meng-Juei Hsieh",
                "Wen-Lien Hsu"
            ],
            "abstract": "Intelligent Input Methods (IM) are essential for making text entries in many\nEast Asian scripts, but their application to other languages has not been fully\nexplored. This paper discusses how such tools can contribute to the development\nof computer processing of other oriental languages. We propose a design\nphilosophy that regards IM as a text service platform, and treats the study of\nIM as a cross disciplinary subject from the perspectives of software\nengineering, human-computer interaction (HCI), and natural language processing\n(NLP). We discuss these three perspectives and indicate a number of possible\nfuture research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0704.3886v6",
            "date": "2007-04-30 17:55:39+00:00",
            "title": "A Note on Ontology and Ordinary Language",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "We argue for a compositional semantics grounded in a strongly typed ontology\nthat reflects our commonsense view of the world and the way we talk about it.\nAssuming such a structure we show that the semantics of various natural\nlanguage phenomena may become nearly trivial.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0707.0895v1",
            "date": "2007-07-06 01:45:05+00:00",
            "title": "Segmentation and Context of Literary and Musical Sequences",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "We test a segmentation algorithm, based on the calculation of the\nJensen-Shannon divergence between probability distributions, to two symbolic\nsequences of literary and musical origin. The first sequence represents the\nsuccessive appearance of characters in a theatrical play, and the second\nrepresents the succession of tones from the twelve-tone scale in a keyboard\nsonata. The algorithm divides the sequences into segments of maximal\ncompositional divergence between them. For the play, these segments are related\nto changes in the frequency of appearance of different characters and in the\ngeographical setting of the action. For the sonata, the segments correspond to\ntonal domains and reveal in detail the characteristic tonal progression of such\nkind of musical composition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0707.1913v3",
            "date": "2007-07-13 02:30:10+00:00",
            "title": "Removing Manually-Generated Boilerplate from Electronic Texts: Experiments with Project Gutenberg e-Books",
            "authors": [
                "Owen Kaser",
                "Daniel Lemire"
            ],
            "abstract": "Collaborative work on unstructured or semi-structured documents, such as in\nliterature corpora or source code, often involves agreed upon templates\ncontaining metadata. These templates are not consistent across users and over\ntime. Rule-based parsing of these templates is expensive to maintain and tends\nto fail as new documents are added. Statistical techniques based on frequent\noccurrences have the potential to identify automatically a large fraction of\nthe templates, thus reducing the burden on the programmers. We investigate the\ncase of the Project Gutenberg corpus, where most documents are in ASCII format\nwith preambles and epilogues that are often copied and pasted or manually\ntyped. We show that a statistical approach can solve most cases though some\ndocuments require knowledge of English. We also survey various technical\nsolutions that make our approach applicable to large data sets.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/0707.3972v1",
            "date": "2007-07-26 17:02:40+00:00",
            "title": "Learning Probabilistic Models of Word Sense Disambiguation",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This dissertation presents several new methods of supervised and unsupervised\nlearning of word sense disambiguation models. The supervised methods focus on\nperforming model searches through a space of probabilistic models, and the\nunsupervised methods rely on the use of Gibbs Sampling and the Expectation\nMaximization (EM) algorithm. In both the supervised and unsupervised case, the\nNaive Bayesian model is found to perform well. An explanation for this success\nis presented in terms of learning rates and bias-variance decompositions.",
            "journal ref": "PhD dissertation, May 1998, Department of Computer Science and\n  Engineering, Southern Methodist University",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0701028v2",
            "date": "2007-01-05 06:11:16+00:00",
            "title": "Statistical keyword detection in literary corpora",
            "authors": [
                "Juan P. Herrera",
                "Pedro A. Pury"
            ],
            "abstract": "Understanding the complexity of human language requires an appropriate\nanalysis of the statistical distribution of words in texts. We consider the\ninformation retrieval problem of detecting and ranking the relevant words of a\ntext by means of statistical information referring to the \"spatial\" use of the\nwords. Shannon's entropy of information is used as a tool for automatic keyword\nextraction. By using The Origin of Species by Charles Darwin as a\nrepresentative text sample, we show the performance of our detector and compare\nit with another proposals in the literature. The random shuffled text receives\nspecial attention as a tool for calibrating the ranking indices.",
            "journal ref": "Eur. Phys. J. B 63, 135-146 (2008)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/cs/0701047v2",
            "date": "2007-01-08 13:00:23+00:00",
            "title": "On vocabulary size of grammar-based codes",
            "authors": [
                "Lukasz Debowski"
            ],
            "abstract": "We discuss inequalities holding between the vocabulary size, i.e., the number\nof distinct nonterminal symbols in a grammar-based compression for a string,\nand the excess length of the respective universal code, i.e., the code-based\nanalog of algorithmic mutual information. The aim is to strengthen inequalities\nwhich were discussed in a weaker form in linguistics but shed some light on\nredundancy of efficiently computable codes. The main contribution of the paper\nis a construction of universal grammar-based codes for which the excess lengths\ncan be bounded easily.",
            "journal ref": "2007 IEEE International Symposium on Information Theory, 91-95",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/0708.0694v1",
            "date": "2007-08-06 01:22:46+00:00",
            "title": "Reconstruction of Protein-Protein Interaction Pathways by Mining Subject-Verb-Objects Intermediates",
            "authors": [
                "Maurice HT Ling",
                "Christophe Lefevre",
                "Kevin R. Nicholas",
                "Feng Lin"
            ],
            "abstract": "The exponential increase in publication rate of new articles is limiting\naccess of researchers to relevant literature. This has prompted the use of text\nmining tools to extract key biological information. Previous studies have\nreported extensive modification of existing generic text processors to process\nbiological text. However, this requirement for modification had not been\nexamined. In this study, we have constructed Muscorian, using MontyLingua, a\ngeneric text processor. It uses a two-layered generalization-specialization\nparadigm previously proposed where text was generically processed to a suitable\nintermediate format before domain-specific data extraction techniques are\napplied at the specialization layer. Evaluation using a corpus and experts\nindicated 86-90% precision and approximately 30% recall in extracting\nprotein-protein interactions, which was comparable to previous studies using\neither specialized biological text processing tools or modified existing tools.\nOur study had also demonstrated the flexibility of the two-layered\ngeneralization-specialization paradigm by using the same generalization layer\nfor two specialized information extraction tasks.",
            "journal ref": "Ling, Maurice HT, Lefevre, Christophe, Nicholas, Kevin R, Lin,\n  Feng. 2007. In J.C. Ragapakse, B. Schmidt, and G. Volkert (Eds.), PRIB 2007.\n  Lecture Notes in Bioinformatics 4774: 286-299. Springer-Verlag.",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0710.1511v1",
            "date": "2007-10-08 11:16:51+00:00",
            "title": "Demographic growth and the distribution of language sizes",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "It is argued that the present log-normal distribution of language sizes is,\nto a large extent, a consequence of demographic dynamics within the population\nof speakers of each language. A two-parameter stochastic multiplicative process\nis proposed as a model for the population dynamics of individual languages, and\napplied over a period spanning the last ten centuries. The model disregards\nlanguage birth and death. A straightforward fitting of the two parameters,\nwhich statistically characterize the population growth rate, predicts a\ndistribution of language sizes in excellent agreement with empirical data.\nNumerical simulations, and the study of the size distribution within language\nfamilies, validate the assumptions at the basis of the model.",
            "journal ref": "None",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/0710.2446v1",
            "date": "2007-10-12 12:44:11+00:00",
            "title": "The structure of verbal sequences analyzed with unsupervised learning techniques",
            "authors": [
                "Catherine Recanati",
                "Nicoleta Rogovschi",
                "Youn\u00e8s Bennani"
            ],
            "abstract": "Data mining allows the exploration of sequences of phenomena, whereas one\nusually tends to focus on isolated phenomena or on the relation between two\nphenomena. It offers invaluable tools for theoretical analyses and exploration\nof the structure of sentences, texts, dialogues, and speech. We report here the\nresults of an attempt at using it for inspecting sequences of verbs from French\naccounts of road accidents. This analysis comes from an original approach of\nunsupervised training allowing the discovery of the structure of sequential\ndata. The entries of the analyzer were only made of the verbs appearing in the\nsentences. It provided a classification of the links between two successive\nverbs into four distinct clusters, allowing thus text segmentation. We give\nhere an interpretation of these clusters by applying a statistical analysis to\nindependent semantic annotations.",
            "journal ref": "Dans Proceedings - The 3rd Language & Technology Conference: Human\n  Language Technologies as a Challenge for Computer Science and Linguistics,\n  Poznan : Pologne (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.2674v1",
            "date": "2007-10-14 16:09:53+00:00",
            "title": "Linguistic Information Energy",
            "authors": [
                "James Ford"
            ],
            "abstract": "In this treatment a text is considered to be a series of word impulses which\nare read at a constant rate. The brain then assembles these units of\ninformation into higher units of meaning. A classical systems approach is used\nto model an initial part of this assembly process. The concepts of linguistic\nsystem response, information energy, and ordering energy are defined and\nanalyzed. Finally, as a demonstration, information energy is used to estimate\nthe publication dates of a series of texts and the similarity of a set of\ntexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.4516v2",
            "date": "2007-10-24 17:23:13+00:00",
            "title": "The predictability of letters in written english",
            "authors": [
                "Thomas Sch\u00fcrmann",
                "Peter Grassberger"
            ],
            "abstract": "We show that the predictability of letters in written English texts depends\nstrongly on their position in the word. The first letters are usually the least\neasy to predict. This agrees with the intuitive notion that words are well\ndefined subunits in written languages, with much weaker correlations across\nthese units than within them. It implies that the average entropy of a letter\ndeep inside a word is roughly 4 times smaller than the entropy of the first\nletter.",
            "journal ref": "Fractals, Vol. 4, No. 1 (1996) 1-5",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0711.2023v1",
            "date": "2007-11-13 16:28:47+00:00",
            "title": "Empirical Evaluation of Four Tensor Decomposition Algorithms",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Higher-order tensor decompositions are analogous to the familiar Singular\nValue Decomposition (SVD), but they transcend the limitations of matrices\n(second-order tensors). SVD is a powerful tool that has achieved impressive\nresults in information retrieval, collaborative filtering, computational\nlinguistics, computational vision, and other fields. However, SVD is limited to\ntwo-dimensional arrays of data (two modes), and many potential applications\nhave three or more modes, which require higher-order tensor decompositions.\nThis paper evaluates four algorithms for higher-order tensor decomposition:\nHigher-Order Singular Value Decomposition (HO-SVD), Higher-Order Orthogonal\nIteration (HOOI), Slice Projection (SP), and Multislice Projection (MP). We\nmeasure the time (elapsed run time), space (RAM and disk space requirements),\nand fit (tensor reconstruction accuracy) of the four algorithms, under a\nvariety of conditions. We find that standard implementations of HO-SVD and HOOI\ndo not scale up to larger tensors, due to increasing RAM requirements. We\nrecommend HOOI for tensors that are small enough for the available RAM and MP\nfor larger tensors.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0711.2270v1",
            "date": "2007-11-14 18:32:09+00:00",
            "title": "Can a Computer Laugh ?",
            "authors": [
                "I. M. Suslov"
            ],
            "abstract": "A computer model of \"a sense of humour\" suggested previously\n[arXiv:0711.2058,0711.2061], relating the humorous effect with a specific\nmalfunction in information processing, is given in somewhat different\nexposition. Psychological aspects of humour are elaborated more thoroughly. The\nmechanism of laughter is formulated on the more general level. Detailed\ndiscussion is presented for the higher levels of information processing, which\nare responsible for a perception of complex samples of humour. Development of a\nsense of humour in the process of evolution is discussed.",
            "journal ref": "Computer Chronicle (Moscow), 1994, issue 1, p.1",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0711.3197v1",
            "date": "2007-11-20 19:57:23+00:00",
            "title": "How to realize \"a sense of humour\" in computers ?",
            "authors": [
                "I. M. Suslov"
            ],
            "abstract": "Computer model of a \"sense of humour\" suggested previously [arXiv:0711.2058,\n0711.2061, 0711.2270] is raised to the level of a realistic algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.0253v1",
            "date": "2007-12-31 23:51:51+00:00",
            "title": "Toward a statistical mechanics of four letter words",
            "authors": [
                "Greg J. Stephens",
                "William Bialek"
            ],
            "abstract": "We consider words as a network of interacting letters, and approximate the\nprobability distribution of states taken on by this network. Despite the\nintuition that the rules of English spelling are highly combinatorial (and\narbitrary), we find that maximum entropy models consistent with pairwise\ncorrelations among letters provide a surprisingly good approximation to the\nfull statistics of four letter words, capturing ~92% of the multi-information\namong letters and even \"discovering\" real words that were not represented in\nthe data from which the pairwise correlations were estimated. The maximum\nentropy model defines an energy landscape on the space of possible words, and\nlocal minima in this landscape account for nearly two-thirds of words used in\nwritten English.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/0707.3559v1",
            "date": "2007-07-24 14:30:27+00:00",
            "title": "Practical Approach to Knowledge-based Question Answering with Natural Language Understanding and Advanced Reasoning",
            "authors": [
                "Wilson Wong"
            ],
            "abstract": "This research hypothesized that a practical approach in the form of a\nsolution framework known as Natural Language Understanding and Reasoning for\nIntelligence (NaLURI), which combines full-discourse natural language\nunderstanding, powerful representation formalism capable of exploiting\nontological information and reasoning approach with advanced features, will\nsolve the following problems without compromising practicality factors: 1)\nrestriction on the nature of question and response, and 2) limitation to scale\nacross domains and to real-life natural language text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0710.0009v2",
            "date": "2007-10-01 09:39:49+00:00",
            "title": "Bio-linguistic transition and Baldwin effect in an evolutionary naming-game model",
            "authors": [
                "Adam Lipowski",
                "Dorota Lipowska"
            ],
            "abstract": "We examine an evolutionary naming-game model where communicating agents are\nequipped with an evolutionarily selected learning ability. Such a coupling of\nbiological and linguistic ingredients results in an abrupt transition: upon a\nsmall change of a model control parameter a poorly communicating group of\nlinguistically unskilled agents transforms into almost perfectly communicating\ngroup with large learning abilities. When learning ability is kept fixed, the\ntransition appears to be continuous. Genetic imprinting of the learning\nabilities proceeds via Baldwin effect: initially unskilled communicating agents\nlearn a language and that creates a niche in which there is an evolutionary\npressure for the increase of learning ability.Our model suggests that when\nlinguistic (or cultural) processes became intensive enough, a transition took\nplace where both linguistic performance and biological endowment of our species\nexperienced an abrupt change that perhaps triggered the rapid expansion of\nhuman civilization.",
            "journal ref": "Int.J.Mod.Phys. C vol.19, pp. 399-407 (2008)",
            "category": "cs.CL"
        }
    ],
    "2008": [
        {
            "url": "http://arxiv.org/abs/0801.3817v1",
            "date": "2008-01-24 18:41:01+00:00",
            "title": "Robustness Evaluation of Two CCG, a PCFG and a Link Grammar Parsers",
            "authors": [
                "Tuomo Kakkonen"
            ],
            "abstract": "Robustness in a parser refers to an ability to deal with exceptional\nphenomena. A parser is robust if it deals with phenomena outside its normal\nrange of inputs. This paper reports on a series of robustness evaluations of\nstate-of-the-art parsers in which we concentrated on one aspect of robustness:\nits ability to parse sentences containing misspelled words. We propose two\nmeasures for robustness evaluation based on a comparison of a parser's output\nfor grammatical input sentences and their noisy counterparts. In this paper, we\nuse these measures to compare the overall robustness of the four evaluated\nparsers, and we present an analysis of the decline in parser performance with\nincreasing error levels. Our results indicate that performance typically\ndeclines tens of percentage units when parsers are presented with texts\ncontaining misspellings. When it was tested on our purpose-built test set of\n443 sentences, the best parser in the experiment (C&C parser) was able to\nreturn exactly the same parse tree for the grammatical and ungrammatical\nsentences for 60.8%, 34.0% and 14.9% of the sentences with one, two or three\nmisspelled words respectively.",
            "journal ref": "Proceedings of the 3rd Language & Technology Conference: Human\n  Language Technologies as a Challenge for Computer Science and Linguistics.\n  Poznan, Poland, 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.4716v1",
            "date": "2008-01-30 17:10:24+00:00",
            "title": "Methods to integrate a language model with semantic information for a word prediction component",
            "authors": [
                "Tonio Wandmacher",
                "Jean-Yves Antoine"
            ],
            "abstract": "Most current word prediction systems make use of n-gram language models (LM)\nto estimate the probability of the following word in a phrase. In the past\nyears there have been many attempts to enrich such language models with further\nsyntactic or semantic information. We want to explore the predictive powers of\nLatent Semantic Analysis (LSA), a method that has been shown to provide\nreliable information on long-distance semantic dependencies between words in a\ncontext. We present and evaluate here several methods that integrate LSA-based\ninformation with a standard language model: a semantic cache, partial\nreranking, and different forms of interpolation. We found that all methods show\nsignificant improvements, compared to the 4-gram baseline, and most of them to\na simple cache model as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0802.4198v1",
            "date": "2008-02-28 12:58:49+00:00",
            "title": "Some properties of the Ukrainian writing system",
            "authors": [
                "Solomija Buk",
                "J\u00e1n Ma\u010dutek",
                "Andrij Rovenchak"
            ],
            "abstract": "We investigate the grapheme-phoneme relation in Ukrainian and some properties\nof the Ukrainian version of the Cyrillic alphabet.",
            "journal ref": "Glottometrics 16, 63-79 (2008)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.0143v1",
            "date": "2008-04-01 11:33:39+00:00",
            "title": "Effects of High-Order Co-occurrences on Word Semantic Similarities",
            "authors": [
                "Beno\u00eet Lemaire",
                "Guy Denhi\u00e8re"
            ],
            "abstract": "A computational model of the construction of word meaning through exposure to\ntexts is built in order to simulate the effects of co-occurrence values on word\nsemantic similarities, paragraph by paragraph. Semantic similarity is here\nviewed as association. It turns out that the similarity between two words W1\nand W2 strongly increases with a co-occurrence, decreases with the occurrence\nof W1 without W2 or W2 without W1, and slightly increases with high-order\nco-occurrences. Therefore, operationalizing similarity as a frequency of\nco-occurrence probably introduces a bias: first, there are cases in which there\nis similarity without co-occurrence and, second, the frequency of co-occurrence\noverestimates similarity.",
            "journal ref": "Current Psychology Letters - Behaviour, Brain and Cognition 18, 1\n  (2006) 1",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.4584v1",
            "date": "2008-04-29 11:39:18+00:00",
            "title": "Feature Unification in TAG Derivation Trees",
            "authors": [
                "Sylvain Schmitz",
                "Joseph Le Roux"
            ],
            "abstract": "The derivation trees of a tree adjoining grammar provide a first insight into\nthe sentence semantics, and are thus prime targets for generation systems. We\ndefine a formalism, feature-based regular tree grammars, and a translation from\nfeature based tree adjoining grammars into this new formalism. The translation\npreserves the derivation structures of the original grammar, and accounts for\nfeature unification.",
            "journal ref": "In TAG+9, Ninth International Workshop on Tree Adjoining Grammars\n  and Related Formalisms, 2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.2303v1",
            "date": "2008-05-15 13:30:08+00:00",
            "title": "Graph Algorithms for Improving Type-Logical Proof Search",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "Proof nets are a graph theoretical representation of proofs in various\nfragments of type-logical grammar. In spite of this basis in graph theory,\nthere has been relatively little attention to the use of graph theoretic\nalgorithms for type-logical proof search. In this paper we will look at several\nways in which standard graph theoretic algorithms can be used to restrict the\nsearch space. In particular, we will provide an O(n4) algorithm for selecting\nan optimal axiom link at any stage in the proof search as well as a O(kn3)\nalgorithm for selecting the k best proof candidates.",
            "journal ref": "Dans Categorial grammars - an efficient tool for natural language\n  processing - June 2004 - Categorial grammars - an efficient tool for natural\n  language processing, Montpellier : France (2004)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.2537v1",
            "date": "2008-05-16 13:58:44+00:00",
            "title": "A toolkit for a generative lexicon",
            "authors": [
                "Patrick Henry",
                "Christian Bassac"
            ],
            "abstract": "In this paper we describe the conception of a software toolkit designed for\nthe construction, maintenance and collaborative use of a Generative Lexicon. In\norder to ease its portability and spreading use, this tool was built with free\nand open source products. We eventually tested the toolkit and showed it\nfilters the adequate form of anaphoric reference to the modifier in endocentric\ncompounds.",
            "journal ref": "Dans A toolkit for a Generative Lexicon - Fourth International\n  Workshop on Generative Approaches to the Lexicon, PARIS : France (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.3366v1",
            "date": "2008-05-21 23:44:06+00:00",
            "title": "Computational Representation of Linguistic Structures using Domain-Specific Languages",
            "authors": [
                "Fabian Steeg",
                "Christoph Benden",
                "Paul O. Samuelsdorff"
            ],
            "abstract": "We describe a modular system for generating sentences from formal definitions\nof underlying linguistic structures using domain-specific languages. The system\nuses Java in general, Prolog for lexical entries and custom domain-specific\nlanguages based on Functional Grammar and Functional Discourse Grammar\nnotation, implemented using the ANTLR parser generator. We show how linguistic\nand technological parts can be brought together in a natural language\nprocessing system and how domain-specific languages can be used as a tool for\nconsistent formal notation in linguistic description.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.3410v1",
            "date": "2008-05-22 08:48:28+00:00",
            "title": "Exploring a type-theoretic approach to accessibility constraint modelling",
            "authors": [
                "Sylvain Pogodalla"
            ],
            "abstract": "The type-theoretic modelling of DRT that [degroote06] proposed features\ncontinuations for the management of the context in which a clause has to be\ninterpreted. This approach, while keeping the standard definitions of\nquantifier scope, translates the rules of the accessibility constraints of\ndiscourse referents inside the semantic recipes. In this paper, we deal with\nadditional rules for these accessibility constraints. In particular in the case\nof discourse referents introduced by proper nouns, that negation does not\nblock, and in the case of rhetorical relations that structure discourses. We\nshow how this continuation-based approach applies to those accessibility\nconstraints and how we can consider the parallel management of various\nprinciples.",
            "journal ref": "Dans Journ\\'ees S\\'emantiques et Mod\\'elisation (2008)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.4369v1",
            "date": "2008-05-28 14:56:18+00:00",
            "title": "A semantic space for modeling children's semantic memory",
            "authors": [
                "Guy Denhi\u00e8re",
                "Beno\u00eet Lemaire",
                "C\u00e9drick Bellissens",
                "Sandra Jhean"
            ],
            "abstract": "The goal of this paper is to present a model of children's semantic memory,\nwhich is based on a corpus reproducing the kinds of texts children are exposed\nto. After presenting the literature in the development of the semantic memory,\na preliminary French corpus of 3.2 million words is described. Similarities in\nthe resulting semantic space are compared to human data on four tests:\nassociation norms, vocabulary test, semantic judgments and memory tasks. A\nsecond corpus is described, which is composed of subcorpora corresponding to\nvarious ages. This stratified corpus is intended as a basis for developmental\nstudies. Finally, two applications of these models of semantic memory are\npresented: the first one aims at tracing the development of semantic\nsimilarities paragraph by paragraph; the second one describes an implementation\nof a model of text comprehension derived from the Construction-integration\nmodel (Kintsch, 1988, 1998) and based on such models of semantic memory.",
            "journal ref": "The handbook of Latent Semantic Analysis, Lawrence Erlbaum\n  Associates (Ed.) (2007) 143-165",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.4521v1",
            "date": "2008-05-29 11:53:39+00:00",
            "title": "Textual Entailment Recognizing by Theorem Proving Approach",
            "authors": [
                "Doina Tatar",
                "Militon Frentiu"
            ],
            "abstract": "In this paper we present two original methods for recognizing textual\ninference. First one is a modified resolution method such that some linguistic\nconsiderations are introduced in the unification of two atoms. The approach is\npossible due to the recent methods of transforming texts in logic formulas.\nSecond one is based on semantic relations in text, as presented in WordNet.\nSome similarities between these two methods are remarked.",
            "journal ref": "Studia Univ.Babes-Bolyai, Informatica, Vol.LI, Number 2, 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0806.2581v1",
            "date": "2008-06-16 13:48:55+00:00",
            "title": "A chain dictionary method for Word Sense Disambiguation and applications",
            "authors": [
                "Doina Tatar",
                "Gabriela Serban",
                "Andreea Mihis",
                "Mihaiela Lupea",
                "Dana Lupsa",
                "Militon Frentiu"
            ],
            "abstract": "A large class of unsupervised algorithms for Word Sense Disambiguation (WSD)\nis that of dictionary-based methods. Various algorithms have as the root Lesk's\nalgorithm, which exploits the sense definitions in the dictionary directly. Our\napproach uses the lexical base WordNet for a new algorithm originated in\nLesk's, namely \"chain algorithm for disambiguation of all words\", CHAD. We show\nhow translation from a language into another one and also text entailment\nverification could be accomplished by this disambiguation.",
            "journal ref": "Studia Universitatis Babes-Bolyai, Special Issue, KEPT 2007,\n  Knowledge Engineering: Principles and Technologies, Cluj-Napoca, June 6-8,\n  2007, pp 33-40,",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0806.3787v2",
            "date": "2008-06-23 23:27:20+00:00",
            "title": "Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "Measuring the similarity of short written contexts is a fundamental problem\nin Natural Language Processing. This article provides a unifying framework by\nwhich short context problems can be categorized both by their intended\napplication and proposed solution. The goal is to show that various problems\nand methodologies that appear quite different on the surface are in fact very\nclosely related. The axes by which these categorizations are made include the\nformat of the contexts (headed versus headless), the way in which the contexts\nare to be measured (first-order versus second-order similarity), and the\ninformation used to represent the features in the contexts (micro versus macro\nviews). The unifying thread that binds together many short context applications\nand methods is the fact that similarity decisions must be made between contexts\nthat share few (if any) words in common.",
            "journal ref": "University of Minnesota Supercomputing Institute Research Report\n  UMSI 2010/118, October 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0807.0311v1",
            "date": "2008-07-02 09:49:14+00:00",
            "title": "About the creation of a parallel bilingual corpora of web-publications",
            "authors": [
                "D. V. Lande",
                "V. V. Zhygalo"
            ],
            "abstract": "The algorithm of the creation texts parallel corpora was presented. The\nalgorithm is based on the use of \"key words\" in text documents, and on the\nmeans of their automated translation. Key words were singled out by means of\nusing Russian and Ukrainian morphological dictionaries, as well as dictionaries\nof the translation of nouns for the Russian and Ukrainianlanguages. Besides, to\ncalculate the weights of the terms in the documents, empiric-statistic rules\nwere used. The algorithm under consideration was realized in the form of a\nprogram complex, integrated into the content-monitoring InfoStream system. As a\nresult, a parallel bilingual corpora of web-publications containing about 30\nthousand documents, was created",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0807.3622v1",
            "date": "2008-07-23 09:05:50+00:00",
            "title": "TuLiPA: Towards a Multi-Formalism Parsing Environment for Grammar Engineering",
            "authors": [
                "Laura Kallmeyer",
                "Timm Lichte",
                "Wolfgang Maier",
                "Yannick Parmentier",
                "Johannes Dellert",
                "Kilian Evang"
            ],
            "abstract": "In this paper, we present an open-source parsing environment (Tuebingen\nLinguistic Parsing Architecture, TuLiPA) which uses Range Concatenation Grammar\n(RCG) as a pivot formalism, thus opening the way to the parsing of several\nmildly context-sensitive formalisms. This environment currently supports\ntree-based grammars (namely Tree-Adjoining Grammars, TAG) and Multi-Component\nTree-Adjoining Grammars with Tree Tuples (TT-MCTAG)) and allows computation not\nonly of syntactic structures, but also of the corresponding semantic\nrepresentations. It is used for the development of a tree-based grammar for\nGerman.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0808.2904v1",
            "date": "2008-08-21 10:54:54+00:00",
            "title": "Investigation of the Zipf-plot of the extinct Meroitic language",
            "authors": [
                "Reginald D. Smith"
            ],
            "abstract": "The ancient and extinct language Meroitic is investigated using Zipf's Law.\nIn particular, since Meroitic is still undeciphered, the Zipf law analysis\nallows us to assess the quality of current texts and possible avenues for\nfuture investigation using statistical techniques.",
            "journal ref": "Glottometrics 15, 2007, 53-61",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0808.3563v1",
            "date": "2008-08-26 18:17:44+00:00",
            "title": "What It Feels Like To Hear Voices: Fond Memories of Julian Jaynes",
            "authors": [
                "Stevan Harnad"
            ],
            "abstract": "Julian Jaynes's profound humanitarian convictions not only prevented him from\ngoing to war, but would have prevented him from ever kicking a dog. Yet\naccording to his theory, not only are language-less dogs unconscious, but so\ntoo were the speaking/hearing Greeks in the Bicameral Era, when they heard\ngods' voices telling them what to do rather than thinking for themselves. I\nargue that to be conscious is to be able to feel, and that all mammals (and\nprobably lower vertebrates and invertebrates too) feel, hence are conscious.\nJulian Jaynes's brilliant analysis of our concepts of consciousness\nnevertheless keeps inspiring ever more inquiry and insights into the age-old\nmind/body problem and its relation to cognition and language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0808.3616v3",
            "date": "2008-08-27 02:02:40+00:00",
            "title": "Constructing word similarities in Meroitic as an aid to decipherment",
            "authors": [
                "Reginald D. Smith"
            ],
            "abstract": "Meroitic is the still undeciphered language of the ancient civilization of\nKush. Over the years, various techniques for decipherment such as finding a\nbilingual text or cognates from modern or other ancient languages in the Sudan\nand surrounding areas has not been successful. Using techniques borrowed from\ninformation theory and natural language statistics, similar words are paired\nand attempts are made to use currently defined words to extract at least\npartial meaning from unknown words.",
            "journal ref": "British Museum Studies in Ancient Egypt and Sudan, 12, 1-10 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0808.3889v1",
            "date": "2008-08-28 11:59:34+00:00",
            "title": "Open architecture for multilingual parallel texts",
            "authors": [
                "M. T. Carrasco Benitez"
            ],
            "abstract": "Multilingual parallel texts (abbreviated to parallel texts) are linguistic\nversions of the same content (\"translations\"); e.g., the Maastricht Treaty in\nEnglish and Spanish are parallel texts. This document is about creating an open\narchitecture for the whole Authoring, Translation and Publishing Chain\n(ATP-chain) for the processing of parallel texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0809.3250v1",
            "date": "2008-09-18 20:48:13+00:00",
            "title": "Using descriptive mark-up to formalize translation quality assessment",
            "authors": [
                "Andrey Kutuzov"
            ],
            "abstract": "The paper deals with using descriptive mark-up to emphasize translation\nmistakes. The author postulates the necessity to develop a standard and formal\nXML-based way of describing translation mistakes. It is considered to be\nimportant for achieving impersonal translation quality assessment. Marked-up\ntranslations can be used in corpus translation studies; moreover, automatic\ntranslation assessment based on marked-up mistakes is possible. The paper\nconcludes with setting up guidelines for further activity within the described\nfield.",
            "journal ref": "Published in Russian in 'Translation industry and information\n  supply in international business activities: materials of international\n  conference' - Perm, 2008, pp. 90-101",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.0200v1",
            "date": "2008-10-01 15:53:36+00:00",
            "title": "Distribution of complexities in the Vai script",
            "authors": [
                "Andrij Rovenchak",
                "J\u00e1n Ma\u010dutek",
                "Charles Riley"
            ],
            "abstract": "In the paper, we analyze the distribution of complexities in the Vai script,\nan indigenous syllabic writing system from Liberia. It is found that the\nuniformity hypothesis for complexities fails for this script. The models using\nPoisson distribution for the number of components and hyper-Poisson\ndistribution for connections provide good fits in the case of the Vai script.",
            "journal ref": "Glottometrics 18, 1-12 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.1199v1",
            "date": "2008-10-07 14:40:19+00:00",
            "title": "Une grammaire formelle du cr\u00e9ole martiniquais pour la g\u00e9n\u00e9ration automatique",
            "authors": [
                "Pascal Vaillant"
            ],
            "abstract": "In this article, some first elements of a computational modelling of the\ngrammar of the Martiniquese French Creole dialect are presented. The sources of\ninspiration for the modelling is the functional description given by Damoiseau\n(1984), and Pinalie's & Bernabe's (1999) grammar manual. Based on earlier works\nin text generation (Vaillant, 1997), a unification grammar formalism, namely\nTree Adjoining Grammars (TAG), and a modelling of lexical functional categories\nbased on syntactic and semantic properties, are used to implement a grammar of\nMartiniquese Creole which is used in a prototype of text generation system. One\nof the main applications of the system could be its use as a tool software\nsupporting the task of learning Creole as a second language. -- Nous\npr\\'esenterons dans cette communication les premiers travaux de mod\\'elisation\ninformatique d'une grammaire de la langue cr\\'eole martiniquaise, en nous\ninspirant des descriptions fonctionnelles de Damoiseau (1984) ainsi que du\nmanuel de Pinalie & Bernab\\'e (1999). Prenant appui sur des travaux\nant\\'erieurs en g\\'en\\'eration de texte (Vaillant, 1997), nous utilisons un\nformalisme de grammaires d'unification, les grammaires d'adjonction d'arbres\n(TAG d'apr\\`es l'acronyme anglais), ainsi qu'une mod\\'elisation de cat\\'egories\nlexicales fonctionnelles \\`a base syntaxico-s\\'emantique, pour mettre en oeuvre\nune grammaire du cr\\'eole martiniquais utilisable dans une maquette de\nsyst\\`eme de g\\'en\\'eration automatique. L'un des int\\'er\\^ets principaux de ce\nsyst\\`eme pourrait \\^etre son utilisation comme logiciel outil pour l'aide \\`a\nl'apprentissage du cr\\'eole en tant que langue seconde.",
            "journal ref": "Actes de la 10eme conference annuelle sur le Traitement\n  Automatique des Langues Naturelles (TALN 2003), p. 255-264. Batz-sur-mer,\n  France, 10-14 juin 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.1207v1",
            "date": "2008-10-07 14:50:59+00:00",
            "title": "A Layered Grammar Model: Using Tree-Adjoining Grammars to Build a Common Syntactic Kernel for Related Dialects",
            "authors": [
                "Pascal Vaillant"
            ],
            "abstract": "This article describes the design of a common syntactic description for the\ncore grammar of a group of related dialects. The common description does not\nrely on an abstract sub-linguistic structure like a metagrammar: it consists in\na single FS-LTAG where the actual specific language is included as one of the\nattributes in the set of attribute types defined for the features. When the\nlang attribute is instantiated, the selected subset of the grammar is\nequivalent to the grammar of one dialect. When it is not, we have a model of a\nhybrid multidialectal linguistic system. This principle is used for a group of\ncreole languages of the West-Atlantic area, namely the French-based Creoles of\nHaiti, Guadeloupe, Martinique and French Guiana.",
            "journal ref": "Proceedings of the Ninth International Workshop on Tree Adjoining\n  Grammars and Related Formalisms (TAG+9 2008), p. 157-164. Tuebingen,\n  Baden-Wurttemberg, Germany, 6-8 June 2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0811.0579v1",
            "date": "2008-11-04 19:31:58+00:00",
            "title": "UNL-French deconversion as transfer & generation from an interlingua with possible quality enhancement through offline human interaction",
            "authors": [
                "Gilles s\u00e9rasset",
                "Christian Boitet"
            ],
            "abstract": "We present the architecture of the UNL-French deconverter, which \"generates\"\nfrom the UNL interlingua by first\"localizing\" the UNL form for French, within\nUNL, and then applying slightly adapted but classical transfer and generation\ntechniques, implemented in GETA's Ariane-G5 environment, supplemented by some\nUNL-specific tools. Online interaction can be used during deconversion to\nenhance output quality and is now used for development purposes. We show how\ninteraction could be delayed and embedded in the postedition phase, which would\nthen interact not directly with the output text, but indirectly with several\ncomponents of the deconverter. Interacting online or offline can improve the\nquality not only of the utterance at hand, but also of the utterances processed\nlater, as various preferences may be automatically changed to let the\ndeconverter \"learn\".",
            "journal ref": "MACHINE TRANSLATION SUMMIT VII, Singapour : Singapour (1999)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0811.1260v1",
            "date": "2008-11-08 10:44:43+00:00",
            "title": "The Application of Fuzzy Logic to Collocation Extraction",
            "authors": [
                "Raj Kishor Bisht",
                "H. S. Dhami"
            ],
            "abstract": "Collocations are important for many tasks of Natural language processing such\nas information retrieval, machine translation, computational lexicography etc.\nSo far many statistical methods have been used for collocation extraction.\nAlmost all the methods form a classical crisp set of collocation. We propose a\nfuzzy logic approach of collocation extraction to form a fuzzy set of\ncollocations in which each word combination has a certain grade of membership\nfor being collocation. Fuzzy logic provides an easy way to express natural\nlanguage into fuzzy logic rules. Two existing methods; Mutual information and\nt-test have been utilized for the input of the fuzzy inference system. The\nresulting membership function could be easily seen and demonstrated. To show\nthe utility of the fuzzy logic some word pairs have been examined as an\nexample. The working data has been based on a corpus of about one million words\ncontained in different novels constituting project Gutenberg available on\nwww.gutenberg.org. The proposed method has all the advantages of the two\nmethods, while overcoming their drawbacks. Hence it provides a better result\nthan the two methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.1179v2",
            "date": "2008-01-08 08:21:26+00:00",
            "title": "Corpus sp{\u00e9}cialis{\u00e9} et ressource de sp{\u00e9}cialit{\u00e9}",
            "authors": [
                "Bernard Jacquemin",
                "Sabine Ploux"
            ],
            "abstract": "\"Semantic Atlas\" is a mathematic and statistic model to visualise word senses\naccording to relations between words. The model, that has been applied to\nproximity relations from a corpus, has shown its ability to distinguish word\nsenses as the corpus' contributors comprehend them. We propose to use the model\nand a specialised corpus in order to create automatically a specialised\ndictionary relative to the corpus' domain. A morpho-syntactic analysis\nperformed on the corpus makes it possible to create the dictionary from\nsyntactic relations between lexical units. The semantic resource can be used to\nnavigate semantically - and not only lexically - through the corpus, to create\nclassical dictionaries or for diachronic studies of the language.",
            "journal ref": "Appears in Fran\\c{c}ois Maniez; Pascaline Dury; Nathalie Arlin;\n  Claire Rougemont. Corpus et dictionnaires de langues de sp{\\'e}cialit{\\'e},\n  Presses Universitaires de Granoble, pp.197-212, 2008",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0801.1415v1",
            "date": "2008-01-09 12:34:40+00:00",
            "title": "The emerging field of language dynamics",
            "authors": [
                "S. Wichmann"
            ],
            "abstract": "A simple review by a linguist, citing many articles by physicists:\nQuantitative methods, agent-based computer simulations, language dynamics,\nlanguage typology, historical linguistics",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.2510v1",
            "date": "2008-01-16 14:07:33+00:00",
            "title": "A Comparison of natural (english) and artificial (esperanto) languages. A Multifractal method based analysis",
            "authors": [
                "J. Gillet",
                "M. Ausloos"
            ],
            "abstract": "We present a comparison of two english texts, written by Lewis Carroll, one\n(Alice in wonderland) and the other (Through a looking glass), the former\ntranslated into esperanto, in order to observe whether natural and artificial\nlanguages significantly differ from each other. We construct one dimensional\ntime series like signals using either word lengths or word frequencies. We use\nthe multifractal ideas for sorting out correlations in the writings. In order\nto check the robustness of the methods we also write the corresponding shuffled\ntexts. We compare characteristic functions and e.g. observe marked differences\nin the (far from parabolic) f(alpha) curves, differences which we attribute to\nTsallis non extensive statistical features in the ''frequency time series'' and\n''length time series''. The esperanto text has more extreme vallues. A very\nrough approximation consists in modeling the texts as a random Cantor set if\nresulting from a binomial cascade of long and short words (or words and\nblanks). This leads to parameters characterizing the text style, and most\nlikely in fine the author writings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.3239v1",
            "date": "2008-01-21 17:41:57+00:00",
            "title": "Online-concordance \"Perekhresni stezhky\" (\"The Cross-Paths\"), a novel by Ivan Franko",
            "authors": [
                "Solomiya Buk",
                "Andrij Rovenchak"
            ],
            "abstract": "In the article, theoretical principles and practical realization for the\ncompilation of the concordance to \"Perekhresni stezhky\" (\"The Cross-Paths\"), a\nnovel by Ivan Franko, are described. Two forms for the context presentation are\nproposed. The electronic version of this lexicographic work is available\nonline.",
            "journal ref": "Ivan Franko: Spirit, Science, Thought, Will (Proceedings of the\n  International Scientific Congress dedicated to the 150th anniversary (Lviv,\n  27 September -- 1 October 2006, Lviv University Press, Vol. 2, pp. 203-211,\n  2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.3864v1",
            "date": "2008-01-25 01:09:47+00:00",
            "title": "Between conjecture and memento: shaping a collective emotional perception of the future",
            "authors": [
                "Alberto Pepe",
                "Johan Bollen"
            ],
            "abstract": "Large scale surveys of public mood are costly and often impractical to\nperform. However, the web is awash with material indicative of public mood such\nas blogs, emails, and web queries. Inexpensive content analysis on such\nextensive corpora can be used to assess public mood fluctuations. The work\npresented here is concerned with the analysis of the public mood towards the\nfuture. Using an extension of the Profile of Mood States questionnaire, we have\nextracted mood indicators from 10,741 emails submitted in 2006 to futureme.org,\na web service that allows its users to send themselves emails to be delivered\nat a later date. Our results indicate long-term optimism toward the future, but\nmedium-term apprehension and confusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0801.4746v5",
            "date": "2008-01-30 19:40:45+00:00",
            "title": "Concerning Olga, the Beautiful Little Street Dancer (Adjectives as Higher-Order Polymorphic Functions)",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "In this paper we suggest a typed compositional seman-tics for nominal\ncompounds of the form [Adj Noun] that models adjectives as higher-order\npolymorphic functions, and where types are assumed to represent concepts in an\nontology that reflects our commonsense view of the world and the way we talk\nabout it in or-dinary language. In addition to [Adj Noun] compounds our\nproposal seems also to suggest a plausible explana-tion for well known\nadjective ordering restrictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0802.2234v1",
            "date": "2008-02-15 16:14:09+00:00",
            "title": "Textual Fingerprinting with Texts from Parkin, Bassewitz, and Leander",
            "authors": [
                "Christoph Schommer",
                "Conny Uhde"
            ],
            "abstract": "Current research in author profiling to discover a legal author's fingerprint\ndoes not only follow examinations based on statistical parameters only but\ninclude more and more dynamic methods that can learn and that react adaptable\nto the specific behavior of an author. But the question on how to appropriately\nrepresent a text is still one of the fundamental tasks, and the problem of\nwhich attribute should be used to fingerprint the author's style is still not\nexactly defined. In this work, we focus on linguistic selection of attributes\nto fingerprint the style of the authors Parkin, Bassewitz and Leander. We use\ntexts of the genre Fairy Tale as it has a clear style and texts of a shorter\nsize with a straightforward story-line and a simple language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0803.2856v1",
            "date": "2008-03-19 18:00:19+00:00",
            "title": "Figuring out Actors in Text Streams: Using Collocations to establish Incremental Mind-maps",
            "authors": [
                "T. Rothenberger",
                "S. Oez",
                "E. Tahirovic",
                "C. Schommer"
            ],
            "abstract": "The recognition, involvement, and description of main actors influences the\nstory line of the whole text. This is of higher importance as the text per se\nrepresents a flow of words and expressions that once it is read it is lost. In\nthis respect, the understanding of a text and moreover on how the actor exactly\nbehaves is not only a major concern: as human beings try to store a given input\non short-term memory while associating diverse aspects and actors with\nincidents, the following approach represents a virtual architecture, where\ncollocations are concerned and taken as the associative completion of the\nactors' acting. Once that collocations are discovered, they become managed in\nseparated memory blocks broken down by the actors. As for human beings, the\nmemory blocks refer to associative mind-maps. We then present several priority\nfunctions to represent the actual temporal situation inside a mind-map to\nenable the user to reconstruct the recent events from the discovered temporal\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.0317v1",
            "date": "2008-04-02 09:34:13+00:00",
            "title": "Parts-of-Speech Tagger Errors Do Not Necessarily Degrade Accuracy in Extracting Information from Biomedical Text",
            "authors": [
                "Maurice HT Ling",
                "Christophe Lefevre",
                "Kevin R. Nicholas"
            ],
            "abstract": "A recent study reported development of Muscorian, a generic text processing\ntool for extracting protein-protein interactions from text that achieved\ncomparable performance to biomedical-specific text processing tools. This\nresult was unexpected since potential errors from a series of text analysis\nprocesses is likely to adversely affect the outcome of the entire process. Most\nbiomedical entity relationship extraction tools have used biomedical-specific\nparts-of-speech (POS) tagger as errors in POS tagging and are likely to affect\nsubsequent semantic analysis of the text, such as shallow parsing. This study\naims to evaluate the parts-of-speech (POS) tagging accuracy and attempts to\nexplore whether a comparable performance is obtained when a generic POS tagger,\nMontyTagger, was used in place of MedPost, a tagger trained in biomedical text.\nOur results demonstrated that MontyTagger, Muscorian's POS tagger, has a POS\ntagging accuracy of 83.1% when tested on biomedical text. Replacing MontyTagger\nwith MedPost did not result in a significant improvement in entity relationship\nextraction from text; precision of 55.6% from MontyTagger versus 56.8% from\nMedPost on directional relationships and 86.1% from MontyTagger compared to\n81.8% from MedPost on nondirectional relationships. This is unexpected as the\npotential for poor POS tagging by MontyTagger is likely to affect the outcome\nof the information extraction. An analysis of POS tagging errors demonstrated\nthat 78.5% of tagging errors are being compensated by shallow parsing. Thus,\ndespite 83.1% tagging accuracy, MontyTagger has a functional tagging accuracy\nof 94.6%.",
            "journal ref": "Ling, Maurice HT, Lefevre, Christophe, Nicholas, Kevin R. 2008.\n  Parts-of-Speech Tagger Errors Do Not Necessarily Degrade Accuracy in\n  Extracting Information from Biomedical Text. The Python Papers 3 (1): 65-80",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.1033v1",
            "date": "2008-04-07 14:13:27+00:00",
            "title": "A Semi-Automatic Framework to Discover Epistemic Modalities in Scientific Articles",
            "authors": [
                "Sviatlana Danilava",
                "Christoph Schommer"
            ],
            "abstract": "Documents in scientific newspapers are often marked by attitudes and opinions\nof the author and/or other persons, who contribute with objective and\nsubjective statements and arguments as well. In this respect, the attitude is\noften accomplished by a linguistic modality. As in languages like english,\nfrench and german, the modality is expressed by special verbs like can, must,\nmay, etc. and the subjunctive mood, an occurrence of modalities often induces\nthat these verbs take over the role of modality. This is not correct as it is\nproven that modality is the instrument of the whole sentence where both the\nadverbs, modal particles, punctuation marks, and the intonation of a sentence\ncontribute. Often, a combination of all these instruments are necessary to\nexpress a modality. In this work, we concern with the finding of modal verbs in\nscientific texts as a pre-step towards the discovery of the attitude of an\nauthor. Whereas the input will be an arbitrary text, the output consists of\nzones representing modalities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.2354v2",
            "date": "2008-04-15 11:05:59+00:00",
            "title": "Information filtering based on wiki index database",
            "authors": [
                "A. V. Smirnov",
                "A. A. Krizhanovsky"
            ],
            "abstract": "In this paper we present a profile-based approach to information filtering by\nan analysis of the content of text documents. The Wikipedia index database is\ncreated and used to automatically generate the user profile from the user\ndocument collection. The problem-oriented Wikipedia subcorpora are created\n(using knowledge extracted from the user profile) for each topic of user\ninterests. The index databases of these subcorpora are applied to filtering\ninformation flow (e.g., mails, news). Thus, the analyzed texts are classified\ninto several topics explicitly presented in the user profile. The paper\nconcentrates on the indexing part of the approach. The architecture of an\napplication implementing the Wikipedia indexing is described. The indexing\nmethod is evaluated using the Russian and Simple English Wikipedia.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0804.3269v1",
            "date": "2008-04-21 15:38:45+00:00",
            "title": "Phoneme recognition in TIMIT with BLSTM-CTC",
            "authors": [
                "Santiago Fern\u00e1ndez",
                "Alex Graves",
                "Juergen Schmidhuber"
            ],
            "abstract": "We compare the performance of a recurrent neural network with the best\nresults published so far on phoneme recognition in the TIMIT database. These\npublished results have been obtained with a combination of classifiers.\nHowever, in this paper we apply a single recurrent neural network to the same\ntask. Our recurrent neural network attains an error rate of 24.6%. This result\nis not significantly different from that obtained by the other best methods,\nbut they rely on a combination of classifiers for achieving comparable\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0804.3599v1",
            "date": "2008-04-22 20:02:14+00:00",
            "title": "Respect My Authority! HITS Without Hyperlinks, Utilizing Cluster-Based Language Models",
            "authors": [
                "Oren Kurland",
                "Lillian Lee"
            ],
            "abstract": "We present an approach to improving the precision of an initial document\nranking wherein we utilize cluster information within a graph-based framework.\nThe main idea is to perform re-ranking based on centrality within bipartite\ngraphs of documents (on one side) and clusters (on the other side), on the\npremise that these are mutually reinforcing entities. Links between entities\nare created via consideration of language models induced from them.\n  We find that our cluster-document graphs give rise to much better retrieval\nperformance than previously proposed document-only graphs do. For example,\nauthority-based re-ranking of documents via a HITS-style cluster-based approach\noutperforms a previously-proposed PageRank-inspired algorithm applied to\nsolely-document graphs. Moreover, we also show that computing authority scores\nfor clusters constitutes an effective method for identifying clusters\ncontaining a large percentage of relevant documents.",
            "journal ref": "Proceedings of SIGIR 2006, pp 83--90",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0805.1030v1",
            "date": "2008-05-07 17:41:47+00:00",
            "title": "Decomposition Techniques for Subgraph Matching",
            "authors": [
                "Stephane Zampelli",
                "Martin Mann",
                "Yves Deville",
                "Rolf Backofen"
            ],
            "abstract": "In the constraint programming framework, state-of-the-art static and dynamic\ndecomposition techniques are hard to apply to problems with complete initial\nconstraint graphs. For such problems, we propose a hybrid approach of these\ntechniques in the presence of global constraints. In particular, we solve the\nsubgraph isomorphism problem. Further we design specific heuristics for this\nhard problem, exploiting its special structure to achieve decomposition. The\nunderlying idea is to precompute a static heuristic on a subset of its\nconstraint network, to follow this static ordering until a first problem\ndecomposition is available, and to switch afterwards to a fully propagated,\ndynamically decomposing search. Experimental results show that, for sparse\ngraphs, our decomposition method solves more instances than dedicated,\nstate-of-the-art matching algorithms or standard constraint programming\napproaches.",
            "journal ref": "None",
            "category": "cs.CC"
        },
        {
            "url": "http://arxiv.org/abs/0805.4101v1",
            "date": "2008-05-27 12:16:12+00:00",
            "title": "Goal-oriented Dialog as a Collaborative Subordinated Activity involving Collective Acceptance",
            "authors": [
                "Sylvie Saget",
                "Marc Guyomard"
            ],
            "abstract": "Modeling dialog as a collaborative activity consists notably in specifying\nthe content of the Conversational Common Ground and the kind of social mental\nstate involved. In previous work (Saget, 2006), we claim that Collective\nAcceptance is the proper social attitude for modeling Conversational Common\nGround in the particular case of goal-oriented dialog. In this paper, a\nformalization of Collective Acceptance is shown, besides elements in order to\nintegrate this attitude in a rational model of dialog are provided; and\nfinally, a model of referential acts as being part of a collaborative activity\nis presented. The particular case of reference has been chosen in order to\nexemplify our claims.",
            "journal ref": "Dans Proceedings of the 10th Workshop on the Semantics and the\n  Pragmatics of Dialogue (Brandial 2006) - 10th Workshop on the Semantics and\n  the Pragmatics of Dialogue (Brandial 2006), Potsdam : Allemagne (2006)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0806.3710v2",
            "date": "2008-06-23 15:53:05+00:00",
            "title": "How Is Meaning Grounded in Dictionary Definitions?",
            "authors": [
                "A. Blondin Masse",
                "G. Chicoisne",
                "Y. Gargouri",
                "S. Harnad",
                "O. Picard",
                "O. Marcotte"
            ],
            "abstract": "Meaning cannot be based on dictionary definitions all the way down: at some\npoint the circularity of definitions must be broken in some way, by grounding\nthe meanings of certain words in sensorimotor categories learned from\nexperience or shaped by evolution. This is the \"symbol grounding problem.\" We\nintroduce the concept of a reachable set -- a larger vocabulary whose meanings\ncan be learned from a smaller vocabulary through definition alone, as long as\nthe meanings of the smaller vocabulary are themselves already grounded. We\nprovide simple algorithms to compute reachable sets for any given dictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0807.0565v1",
            "date": "2008-07-03 13:42:00+00:00",
            "title": "Music, Complexity, Information",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "These are the preparatory notes for a Science & Music essay, \"Playing by\nnumbers\", appeared in Nature 453 (2008) 988-989.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0807.1560v1",
            "date": "2008-07-10 00:01:20+00:00",
            "title": "Scientific Paper Summarization Using Citation Summary Networks",
            "authors": [
                "Vahed Qazvinian",
                "Dragomir R. Radev"
            ],
            "abstract": "Quickly moving to a new area of research is painful for researchers due to\nthe vast amount of scientific literature in each field of study. One possible\nway to overcome this problem is to summarize a scientific topic. In this paper,\nwe propose a model of summarizing a single article, which can be further used\nto summarize an entire topic. Our model is based on analyzing others' viewpoint\nof the target article's contributions and the study of its citation summary\nnetwork using a clustering approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0808.1211v1",
            "date": "2008-08-08 14:37:45+00:00",
            "title": "Commonsense Knowledge, Ontology and Ordinary Language",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "Over two decades ago a \"quite revolution\" overwhelmingly replaced\nknowledgebased approaches in natural language processing (NLP) by quantitative\n(e.g., statistical, corpus-based, machine learning) methods. Although it is our\nfirm belief that purely quantitative approaches cannot be the only paradigm for\nNLP, dissatisfaction with purely engineering approaches to the construction of\nlarge knowledge bases for NLP are somewhat justified. In this paper we hope to\ndemonstrate that both trends are partly misguided and that the time has come to\nenrich logical semantics with an ontological structure that reflects our\ncommonsense view of the world and the way we talk about in ordinary language.\nIn this paper it will be demonstrated that assuming such an ontological\nstructure a number of challenges in the semantics of natural language (e.g.,\nmetonymy, intensionality, copredication, nominal compounds, etc.) can be\nproperly and uniformly addressed.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0808.1753v2",
            "date": "2008-08-12 23:47:21+00:00",
            "title": "Index wiki database: design and experiments",
            "authors": [
                "A. A. Krizhanovsky"
            ],
            "abstract": "With the fantastic growth of Internet usage, information search in documents\nof a special type called a \"wiki page\" that is written using a simple markup\nlanguage, has become an important problem. This paper describes the software\narchitectural model for indexing wiki texts in three languages (Russian,\nEnglish, and German) and the interaction between the software components (GATE,\nLemmatizer, and Synarcher). The inverted file index database was designed using\nvisual tool DBDesigner. The rules for parsing Wikipedia texts are illustrated\nby examples. Two index databases of Russian Wikipedia (RW) and Simple English\nWikipedia (SEW) are built and compared. The size of RW is by order of magnitude\nhigher than SEW (number of words, lexemes), though the growth rate of number of\npages in SEW was found to be 14% higher than in Russian, and the rate of\nacquisition of new words in SEW lexicon was 7% higher during a period of five\nmonths (from September 2007 to February 2008). The Zipf's law was tested with\nboth Russian and Simple Wikipedias. The entire source code of the indexing\nsoftware and the generated index databases are freely available under GPL (GNU\nGeneral Public License).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0808.3569v3",
            "date": "2008-08-26 19:15:24+00:00",
            "title": "Offloading Cognition onto Cognitive Technology",
            "authors": [
                "Itiel Dror",
                "Stevan Harnad"
            ],
            "abstract": "\"Cognizing\" (e.g., thinking, understanding, and knowing) is a mental state.\nSystems without mental states, such as cognitive technology, can sometimes\ncontribute to human cognition, but that does not make them cognizers. Cognizers\ncan offload some of their cognitive functions onto cognitive technology,\nthereby extending their performance capacity beyond the limits of their own\nbrain power. Language itself is a form of cognitive technology that allows\ncognizers to offload some of their cognitive functions onto the brains of other\ncognizers. Language also extends cognizers' individual and joint performance\npowers, distributing the load through interactive and collaborative cognition.\nReading, writing, print, telecommunications and computing further extend\ncognizers' capacities. And now the web, with its network of cognizers, digital\ndatabases and software agents, all accessible anytime, anywhere, has become our\n'Cognitive Commons,' in which distributed cognizers and cognitive technology\ncan interoperate globally with a speed, scope and degree of interactivity\ninconceivable through local individual cognition alone. And as with language,\nthe cognitive tool par excellence, such technological changes are not merely\ninstrumental and quantitative: they can have profound effects on how we think\nand encode information, on how we communicate with one another, on our mental\nstates, and on our very nature.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/0809.0360v2",
            "date": "2008-09-02 07:51:04+00:00",
            "title": "The Complexity of Enriched Mu-Calculi",
            "authors": [
                "Piero A. Bonatti",
                "Carsten Lutz",
                "Aniello Murano",
                "Moshe Y. Vardi"
            ],
            "abstract": "The fully enriched &mu;-calculus is the extension of the propositional\n&mu;-calculus with inverse programs, graded modalities, and nominals. While\nsatisfiability in several expressive fragments of the fully enriched\n&mu;-calculus is known to be decidable and ExpTime-complete, it has recently\nbeen proved that the full calculus is undecidable. In this paper, we study the\nfragments of the fully enriched &mu;-calculus that are obtained by dropping at\nleast one of the additional constructs. We show that, in all fragments obtained\nin this way, satisfiability is decidable and ExpTime-complete. Thus, we\nidentify a family of decidable logics that are maximal (and incomparable) in\nexpressive power. Our results are obtained by introducing two new automata\nmodels, showing that their emptiness problems are ExpTime-complete, and then\nreducing satisfiability in the relevant logics to these problems. The automata\nmodels we introduce are two-way graded alternating parity automata over\ninfinite trees (2GAPTs) and fully enriched automata (FEAs) over infinite\nforests. The former are a common generalization of two incomparable automata\nmodels from the literature. The latter extend alternating automata in a similar\nway as the fully enriched &mu;-calculus extends the standard &mu;-calculus.",
            "journal ref": "Logical Methods in Computer Science, Volume 4, Issue 3 (September\n  22, 2008) lmcs:993",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/0810.1212v1",
            "date": "2008-10-07 15:25:31+00:00",
            "title": "Analyse spectrale des textes: d\u00e9tection automatique des fronti\u00e8res de langue et de discours",
            "authors": [
                "Pascal Vaillant",
                "Richard Nock",
                "Claudia Henry"
            ],
            "abstract": "We propose a theoretical framework within which information on the vocabulary\nof a given corpus can be inferred on the basis of statistical information\ngathered on that corpus. Inferences can be made on the categories of the words\nin the vocabulary, and on their syntactical properties within particular\nlanguages. Based on the same statistical data, it is possible to build matrices\nof syntagmatic similarity (bigram transition matrices) or paradigmatic\nsimilarity (probability for any pair of words to share common contexts). When\nclustered with respect to their syntagmatic similarity, words tend to group\ninto sublanguage vocabularies, and when clustered with respect to their\nparadigmatic similarity, into syntactic or semantic classes. Experiments have\nexplored the first of these two possibilities. Their results are interpreted in\nthe frame of a Markov chain modelling of the corpus' generative processe(s): we\nshow that the results of a spectral analysis of the transition matrix can be\ninterpreted as probability distributions of words within clusters. This method\nyields a soft clustering of the vocabulary into sublanguages which contribute\nto the generation of heterogeneous corpora. As an application, we show how\nmultilingual texts can be visually segmented into linguistically homogeneous\nsegments. Our method is specifically useful in the case of related languages\nwhich happened to be mixed in corpora.",
            "journal ref": "Verbum ex machina: Actes de la 13eme conference annuelle sur le\n  Traitement Automatique des Langues Naturelles (TALN 2006), p. 619-629.\n  Louvain (Leuven), Belgique, 10-13 avril 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.1261v1",
            "date": "2008-10-07 18:09:07+00:00",
            "title": "Soft Uncoupling of Markov Chains for Permeable Language Distinction: A New Algorithm",
            "authors": [
                "Richard Nock",
                "Pascal Vaillant",
                "Frank Nielsen",
                "Claudia Henry"
            ],
            "abstract": "Without prior knowledge, distinguishing different languages may be a hard\ntask, especially when their borders are permeable. We develop an extension of\nspectral clustering -- a powerful unsupervised classification toolbox -- that\nis shown to resolve accurately the task of soft language distinction. At the\nheart of our approach, we replace the usual hard membership assignment of\nspectral clustering by a soft, probabilistic assignment, which also presents\nthe advantage to bypass a well-known complexity bottleneck of the method.\nFurthermore, our approach relies on a novel, convenient construction of a\nMarkov chain out of a corpus. Extensive experiments with a readily available\nsystem clearly display the potential of the method, which brings a visually\nappealing soft distinction of languages that may define altogether a whole\ncorpus.",
            "journal ref": "ECAI 2006: 17th European Conference on Artificial Intelligence.\n  Riva del Garda, Italy, 29 August - 1st September 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.3416v1",
            "date": "2008-10-19 17:34:33+00:00",
            "title": "Text as Statistical Mechanics Object",
            "authors": [
                "K. Koroutchev",
                "E. Korutcheva"
            ],
            "abstract": "In this article we present a model of human written text based on statistical\nmechanics approach by deriving the potential energy for different parts of the\ntext using large text corpus. We have checked the results numerically and found\nthat the specific heat parameter effectively separates the closed class words\nfrom the specific terms used in the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.4616v1",
            "date": "2008-10-25 16:00:08+00:00",
            "title": "Assembling Actor-based Mind-Maps from Text Stream",
            "authors": [
                "Claudine Brucks",
                "Christoph Schommer"
            ],
            "abstract": "For human beings, the processing of text streams of unknown size leads\ngenerally to problems because e.g. noise must be selected out, information be\ntested for its relevance or redundancy, and linguistic phenomenon like\nambiguity or the resolution of pronouns be advanced. Putting this into\nsimulation by using an artificial mind-map is a challenge, which offers the\ngate for a wide field of applications like automatic text summarization or\npunctual retrieval. In this work we present a framework that is a first step\ntowards an automatic intellect. It aims at assembling a mind-map based on\nincoming text streams and on a subject-verb-object strategy, having the verb as\nan interconnection between the adjacent nouns. The mind-map's performance is\nenriched by a pronoun resolution engine that bases on the work of D. Klein, and\nC. D. Manning.",
            "journal ref": "Summary of the Master Thesis \"Actor-based Mind-map learning from\n  Text Streams\". Dept. of Computer Science and Communication, University of\n  Luxembourg, 2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0811.0453v1",
            "date": "2008-11-04 09:08:32+00:00",
            "title": "CoZo+ - A Content Zoning Engine for textual documents",
            "authors": [
                "Cynthia Wagner",
                "Christoph Schommer"
            ],
            "abstract": "Content zoning can be understood as a segmentation of textual documents into\nzones. This is inspired by [6] who initially proposed an approach for the\nargumentative zoning of textual documents. With the prototypical CoZo+ engine,\nwe focus on content zoning towards an automatic processing of textual streams\nwhile considering only the actors as the zones. We gain information that can be\nused to realize an automatic recognition of content for pre-defined actors. We\nunderstand CoZo+ as a necessary pre-step towards an automatic generation of\nsummaries and to make intellectual ownership of documents detectable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0811.4717v1",
            "date": "2008-11-28 13:30:23+00:00",
            "title": "Prospective Study for Semantic Inter-Media Fusion in Content-Based Medical Image Retrieval",
            "authors": [
                "Roxana Teodorescu",
                "Daniel Racoceanu",
                "Wee-Kheng Leow",
                "Vladimir Cretu"
            ],
            "abstract": "One important challenge in modern Content-Based Medical Image Retrieval\n(CBMIR) approaches is represented by the semantic gap, related to the\ncomplexity of the medical knowledge. Among the methods that are able to close\nthis gap in CBMIR, the use of medical thesauri/ontologies has interesting\nperspectives due to the possibility of accessing on-line updated relevant\nwebservices and to extract real-time medical semantic structured information.\nThe CBMIR approach proposed in this paper uses the Unified Medical Language\nSystem's (UMLS) Metathesaurus to perform a semantic indexing and fusion of\nmedical media. This fusion operates before the query processing (retrieval) and\nworks at an UMLS-compliant conceptual indexing level. Our purpose is to study\nvarious techniques related to semantic data alignment, preprocessing, fusion,\nclustering and retrieval, by evaluating the various techniques and highlighting\nfuture research directions. The alignment and the preprocessing are based on\npartial text/image retrieval feedback and on the data structure. We analyze\nvarious probabilistic, fuzzy and evidence-based approaches for the fusion\nprocess and different similarity functions for the retrieval process. All the\nproposed methods are evaluated on the Cross Language Evaluation Forum's (CLEF)\nmedical image retrieval benchmark, by focusing also on a more homogeneous\ncomponent medical image database: the Pathology Education Instructional\nResource (PEIR).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0801.1658v3",
            "date": "2008-01-10 19:45:25+00:00",
            "title": "Computational approach to the emergence and evolution of language - evolutionary naming game model",
            "authors": [
                "Adam Lipowski",
                "Dorota Lipowska"
            ],
            "abstract": "Computational modelling with multi-agent systems is becoming an important\ntechnique of studying language evolution. We present a brief introduction into\nthis rapidly developing field, as well as our own contributions that include an\nanalysis of the evolutionary naming-game model. In this model communicating\nagents, that try to establish a common vocabulary, are equipped with an\nevolutionarily selected learning ability. Such a coupling of biological and\nlinguistic ingredients results in an abrupt transition: upon a small change of\nthe model control parameter a poorly communicating group of linguistically\nunskilled agents transforms into almost perfectly communicating group with\nlarge learning abilities. Genetic imprinting of the learning abilities proceeds\nvia Baldwin effect: initially unskilled communicating agents learn a language\nand that creates a niche in which there is an evolutionary pressure for the\nincrease of learning ability. Under the assumption that communication intensity\nincreases continuously with finite speed, the transition is split into several\ntransition-like changes. It shows that the speed of cultural changes, that sets\nan additional characteristic timescale, might be yet another factor affecting\nthe evolution of language. In our opinion, this model shows that linguistic and\nbiological processes have a strong influence on each other and this effect\ncertainly has contributed to an explosive development of our species.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0802.4112v1",
            "date": "2008-02-28 00:15:54+00:00",
            "title": "Hubs in Languages: Scale Free Networks of Synonyms",
            "authors": [
                "Hanna E. Makaruk",
                "Robert Owczarek"
            ],
            "abstract": "Natural languages are described in this paper in terms of networks of\nsynonyms: a word is identified with a node, and synonyms are connected by\nundirected links. Our statistical analysis of the network of synonyms in Polish\nlanguage showed it is scale-free; similar to what is known for English. The\nstatistical properties of the networks are also similar. Thus, the statistical\naspects of the networks are good candidates for culture independent elements of\nhuman language. We hypothesize that optimization for robustness and efficiency\nis responsible for this universality. Despite the statistical similarity, there\nis no one-to-one mapping between networks of these two languages. Although many\nhubs in Polish are translated into similarly highly connected hubs in English,\nthere are also hubs specific to one of these languages only: a single word in\none language is equivalent to many different and disconnected words in the\nother, in accordance with the Whorf hypothesis about language relativity.\nIdentifying language-specific hubs is vitally important for automatic\ntranslation, and for understanding contextual, culturally related messages that\nare frequently missed or twisted in a naive, literary translation.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0802.4215v1",
            "date": "2008-02-28 11:49:48+00:00",
            "title": "Equilibrium (Zipf) and Dynamic (Grasseberg-Procaccia) method based analyses of human texts. A comparison of natural (english) and artificial (esperanto) languages",
            "authors": [
                "M. Ausloos"
            ],
            "abstract": "A comparison of two english texts from Lewis Carroll, one (Alice in\nwonderland), also translated into esperanto, the other (Through a looking\nglass) are discussed in order to observe whether natural and artificial\nlanguages significantly differ from each other. One dimensional time series\nlike signals are constructed using only word frequencies (FTS) or word lengths\n(LTS). The data is studied through (i) a Zipf method for sorting out\ncorrelations in the FTS and (ii) a Grassberger-Procaccia (GP) technique based\nmethod for finding correlations in LTS. Features are compared : different power\nlaws are observed with characteristic exponents for the ranking properties, and\nthe {\\it phase space attractor dimensionality}. The Zipf exponent can take\nvalues much less than unity ($ca.$ 0.50 or 0.30) depending on how a sentence is\ndefined. This non-universality is conjectured to be a measure of the author\n$style$. Moreover the attractor dimension $r$ is a simple function of the so\ncalled phase space dimension $n$, i.e., $r = n^{\\lambda}$, with $\\lambda =\n0.79$. Such an exponent should also conjecture to be a measure of the author\n$creativity$. However, even though there are quantitative differences between\nthe original english text and its esperanto translation, the qualitative\ndifferences are very minutes, indicating in this case a translation relatively\nwell respecting, along our analysis lines, the content of the author writing.",
            "journal ref": "Physica A 387 (25) 6411-6420 (2008)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0802.4326v1",
            "date": "2008-02-29 06:16:29+00:00",
            "title": "The Generation of Textual Entailment with NLML in an Intelligent Dialogue system for Language Learning CSIEC",
            "authors": [
                "Jiyou Jia"
            ],
            "abstract": "This research report introduces the generation of textual entailment within\nthe project CSIEC (Computer Simulation in Educational Communication), an\ninteractive web-based human-computer dialogue system with natural language for\nEnglish instruction. The generation of textual entailment (GTE) is critical to\nthe further improvement of CSIEC project. Up to now we have found few\nliteratures related with GTE. Simulating the process that a human being learns\nEnglish as a foreign language we explore our naive approach to tackle the GTE\nproblem and its algorithm within the framework of CSIEC, i.e. rule annotation\nin NLML, pattern recognition (matching), and entailment transformation. The\ntime and space complexity of our algorithm is tested with some entailment\nexamples. Further works include the rules annotation based on the English\ntextbooks and a GUI interface for normal users to edit the entailment rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0805.4722v1",
            "date": "2008-05-30 10:45:42+00:00",
            "title": "La fiabilit\u00e9 des informations sur le web",
            "authors": [
                "Bernard Jacquemin",
                "Aur\u00e9lien Lauf",
                "C\u00e9line Poudat",
                "Martine Hurault-Plantet",
                "Nicolas Auray"
            ],
            "abstract": "Online IR tools have to take into account new phenomena linked to the\nappearance of blogs, wiki and other collaborative publications. Among these\ncollaborative sites, Wikipedia represents a crucial source of information.\nHowever, the quality of this information has been recently questionned. A\nbetter knowledge of the contributors' behaviors should help users navigate\nthrough information whose quality may vary from one source to another. In order\nto explore this idea, we present an analysis of the role of different types of\ncontributors in the control of the publication of conflictual articles.",
            "journal ref": "Dans Actes de la Conf\\'erence en Recherche d'Information et\n  Applications CORIA 2008 - Conf\\'erence en Recherche d'Information et\n  Applications 2008, Tr\\'egastel : France (2008)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0805.4754v1",
            "date": "2008-05-30 13:20:42+00:00",
            "title": "Managing conflicts between users in Wikipedia",
            "authors": [
                "Bernard Jacquemin",
                "Aur\u00e9lien Lauf",
                "C\u00e9line Poudat",
                "Martine Hurault-Plantet",
                "Nicolas Auray"
            ],
            "abstract": "Wikipedia is nowadays a widely used encyclopedia, and one of the most visible\nsites on the Internet. Its strong principle of collaborative work and free\nediting sometimes generates disputes due to disagreements between users. In\nthis article we study how the wikipedian community resolves the conflicts and\nwhich roles do wikipedian choose in this process. We observed the users\nbehavior both in the article talk pages, and in the Arbitration Committee pages\nspecifically dedicated to serious disputes. We first set up a users typology\naccording to their involvement in conflicts and their publishing and management\nactivity in the encyclopedia. We then used those user types to describe users\nbehavior in contributing to articles that are tagged by the wikipedian\ncommunity as being in conflict with the official guidelines of Wikipedia, or\nconversely as being well featured.",
            "journal ref": "Dans BIS 2008 Workshop proceedings - 11th Conference on Business\n  Information Systems (BIS 2008), Social Aspects of the Web Workshop (SAW\n  2008), Innsbruck : Autriche (2008)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0807.3845v1",
            "date": "2008-07-24 10:31:55+00:00",
            "title": "Formal semantics of language and the Richard-Berry paradox",
            "authors": [
                "Stefano Crespi Reghizzi"
            ],
            "abstract": "The classical logical antinomy known as Richard-Berry paradox is combined\nwith plausible assumptions about the size i.e. the descriptional complexity of\nTuring machines formalizing certain sentences, to show that formalization of\nlanguage leads to contradiction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0808.0521v1",
            "date": "2008-08-04 22:26:38+00:00",
            "title": "Logics for the Relational Syllogistic",
            "authors": [
                "Ian Pratt-Hartmann",
                "Lawrence S. Moss"
            ],
            "abstract": "The Aristotelian syllogistic cannot account for the validity of many\ninferences involving relational facts. In this paper, we investigate the\nprospects for providing a relational syllogistic. We identify several fragments\nbased on (a) whether negation is permitted on all nouns, including those in the\nsubject of a sentence; and (b) whether the subject noun phrase may contain a\nrelative clause. The logics we present are extensions of the classical\nsyllogistic, and we pay special attention to the question of whether reductio\nad absurdum is needed. Thus our main goal is to derive results on the existence\n(or non-existence) of syllogistic proof systems for relational fragments. We\nalso determine the computational complexity of all our fragments.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/0808.4122v2",
            "date": "2008-08-29 16:09:08+00:00",
            "title": "Swapping Lemmas for Regular and Context-Free Languages",
            "authors": [
                "Tomoyuki Yamakami"
            ],
            "abstract": "In formal language theory, one of the most fundamental tools, known as\npumping lemmas, is extremely useful for regular and context-free languages.\nHowever, there are natural properties for which the pumping lemmas are of\nlittle use. One of such examples concerns a notion of advice, which depends\nonly on the size of an underlying input. A standard pumping lemma encounters\ndifficulty in proving that a given language is not regular in the presence of\nadvice. We develop its substitution, called a swapping lemma for regular\nlanguages, to demonstrate the non-regularity of a target language with advice.\nFor context-free languages, we also present a similar form of swapping lemma,\nwhich serves as a technical tool to show that certain languages are not\ncontext-free with advice.",
            "journal ref": "None",
            "category": "cs.CC"
        },
        {
            "url": "http://arxiv.org/abs/0809.0103v1",
            "date": "2008-08-31 06:08:15+00:00",
            "title": "On the nature of long-range letter correlations in texts",
            "authors": [
                "Dmitrii Y. Manin"
            ],
            "abstract": "The origin of long-range letter correlations in natural texts is studied\nusing random walk analysis and Jensen-Shannon divergence. It is concluded that\nthey result from slow variations in letter frequency distribution, which are a\nconsequence of slow variations in lexical composition within the text. These\ncorrelations are preserved by random letter shuffling within a moving window.\nAs such, they do reflect structural properties of the text, but in a very\nindirect manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0809.0124v1",
            "date": "2008-08-31 14:00:26+00:00",
            "title": "A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Recognizing analogies, synonyms, antonyms, and associations appear to be four\ndistinct tasks, requiring distinct NLP algorithms. In the past, the four tasks\nhave been treated independently, using a wide variety of algorithms. These four\nsemantic classes, however, are a tiny sample of the full range of semantic\nphenomena, and we cannot afford to create ad hoc algorithms for each semantic\nphenomenon; we need to seek a unified approach. We propose to subsume a broad\nrange of phenomena under analogies. To limit the scope of this paper, we\nrestrict our attention to the subsumption of synonyms, antonyms, and\nassociations. We introduce a supervised corpus-based machine learning algorithm\nfor classifying analogous word pairs, and we show that it can solve\nmultiple-choice SAT analogy questions, TOEFL synonym questions, ESL\nsynonym-antonym questions, and similar-associated-both questions from cognitive\npsychology.",
            "journal ref": "Proceedings of the 22nd International Conference on Computational\n  Linguistics (Coling 2008), August 2008, Manchester, UK, Pages 905-912",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0809.4530v2",
            "date": "2008-09-26 04:47:19+00:00",
            "title": "Mining Meaning from Wikipedia",
            "authors": [
                "Olena Medelyan",
                "David Milne",
                "Catherine Legg",
                "Ian H. Witten"
            ],
            "abstract": "Wikipedia is a goldmine of information; not just for its many readers, but\nalso for the growing community of researchers who recognize it as a resource of\nexceptional scale and utility. It represents a vast investment of manual effort\nand judgment: a huge, constantly evolving tapestry of concepts and relations\nthat is being applied to a host of tasks.\n  This article provides a comprehensive description of this work. It focuses on\nresearch that extracts and makes use of the concepts, relations, facts and\ndescriptions found in Wikipedia, and organizes the work into four broad\ncategories: applying Wikipedia to natural language processing; using it to\nfacilitate information retrieval and information extraction; and as a resource\nfor ontology building. The article addresses how Wikipedia is being used as is,\nhow it is being improved and adapted, and how it is being combined with other\nstructures to create entirely new resources. We identify the research groups\nand individuals involved, and how their work has developed in the last few\nyears. We provide a comprehensive list of the open-source software they have\nproduced.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/0812.3070v1",
            "date": "2008-12-16 14:24:23+00:00",
            "title": "A Computational Model to Disentangle Semantic Information Embedded in Word Association Norms",
            "authors": [
                "J. Borge",
                "A. Arenas"
            ],
            "abstract": "Two well-known databases of semantic relationships between pairs of words\nused in psycholinguistics, feature-based and association-based, are studied as\ncomplex networks. We propose an algorithm to disentangle feature based\nrelationships from free association semantic networks. The algorithm uses the\nrich topology of the free association semantic network to produce a new set of\nrelationships between words similar to those observed in feature production\nnorms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0812.4446v1",
            "date": "2008-12-23 20:08:53+00:00",
            "title": "The Latent Relation Mapping Engine: Algorithm and Experiments",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Many AI researchers and cognitive scientists have argued that analogy is the\ncore of cognition. The most influential work on computational modeling of\nanalogy-making is Structure Mapping Theory (SMT) and its implementation in the\nStructure Mapping Engine (SME). A limitation of SME is the requirement for\ncomplex hand-coded representations. We introduce the Latent Relation Mapping\nEngine (LRME), which combines ideas from SME and Latent Relational Analysis\n(LRA) in order to remove the requirement for hand-coded representations. LRME\nbuilds analogical mappings between lists of words, using a large corpus of raw\ntext to automatically discover the semantic relations among the words. We\nevaluate LRME on a set of twenty analogical mapping problems, ten based on\nscientific analogies and ten based on common metaphors. LRME achieves\nhuman-level performance on the twenty problems. We compare LRME with a variety\nof alternative approaches and find that they are not able to reach the same\nlevel of performance.",
            "journal ref": "Journal of Artificial Intelligence Research, (2008), 33, 615-655",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.3125v5",
            "date": "2008-10-17 16:32:17+00:00",
            "title": "On the Vocabulary of Grammar-Based Codes and the Logical Consistency of Texts",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "The article presents a new interpretation for Zipf-Mandelbrot's law in\nnatural language which rests on two areas of information theory. Firstly, we\nconstruct a new class of grammar-based codes and, secondly, we investigate\nproperties of strongly nonergodic stationary processes. The motivation for the\njoint discussion is to prove a proposition with a simple informal statement: If\na text of length $n$ describes $n^\\beta$ independent facts in a repetitive way\nthen the text contains at least $n^\\beta/\\log n$ different words, under\nsuitable conditions on $n$. In the formal statement, two modeling postulates\nare adopted. Firstly, the words are understood as nonterminal symbols of the\nshortest grammar-based encoding of the text. Secondly, the text is assumed to\nbe emitted by a finite-energy strongly nonergodic source whereas the facts are\nbinary IID variables predictable in a shift-invariant way.",
            "journal ref": "IEEE Transactions on Information Theory 57:4589-4599, 2011",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/0810.3442v2",
            "date": "2008-10-19 23:59:37+00:00",
            "title": "Language structure in the n-object naming game",
            "authors": [
                "Adam Lipowski",
                "Dorota Lipowska"
            ],
            "abstract": "We examine a naming game with two agents trying to establish a common\nvocabulary for n objects. Such efforts lead to the emergence of language that\nallows for an efficient communication and exhibits some degree of homonymy and\nsynonymy. Although homonymy reduces the communication efficiency, it seems to\nbe a dynamical trap that persists for a long, and perhaps indefinite, time. On\nthe other hand, synonymy does not reduce the efficiency of communication, but\nappears to be only a transient feature of the language. Thus, in our model the\nrole of synonymy decreases and in the long-time limit it becomes negligible. A\nsimilar rareness of synonymy is observed in present natural languages. The role\nof noise, that distorts the communicated words, is also examined. Although, in\ngeneral, the noise reduces the communication efficiency, it also regroups the\nwords so that they are more evenly distributed within the available \"verbal\"\nspace.",
            "journal ref": "Phys. Rev. E 80, 056107 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0810.4952v1",
            "date": "2008-10-27 23:20:11+00:00",
            "title": "Computational modelling of evolution: ecosystems and language",
            "authors": [
                "Adam Lipowski",
                "Dorota Lipowska"
            ],
            "abstract": "Recently, computational modelling became a very important research tool that\nenables us to study problems that for decades evaded scientific analysis.\nEvolutionary systems are certainly examples of such problems: they are composed\nof many units that might reproduce, diffuse, mutate, die, or in some cases for\nexample communicate. These processes might be of some adaptive value, they\ninfluence each other and occur on various time scales. That is why such systems\nare so difficult to study. In this paper we briefly review some computational\napproaches, as well as our contributions, to the evolution of ecosystems and\nlanguage. We start from Lotka-Volterra equations and the modelling of simple\ntwo-species prey-predator systems. Such systems are canonical example for\nstudying oscillatory behaviour in competitive populations. Then we describe\nvarious approaches to study long-term evolution of multi-species ecosystems. We\nemphasize the need to use models that take into account both ecological and\nevolutionary processes. Finally, we address the problem of the emergence and\ndevelopment of language. It is becoming more and more evident that any theory\nof language origin and development must be consistent with darwinian principles\nof evolution. Consequently, a number of techniques developed for modelling\nevolution of complex ecosystems are being applied to the problem of language.\nWe briefly review some of these approaches.",
            "journal ref": "Series on Advances in Mathematics for Applied Sciences - vol.79\n  (2009)",
            "category": "q-bio.PE"
        }
    ],
    "2009": [
        {
            "url": "http://arxiv.org/abs/0901.3017v1",
            "date": "2009-01-20 12:55:55+00:00",
            "title": "Statistical analysis of the Indus script using $n$-grams",
            "authors": [
                "Nisha Yadav",
                "Hrishikesh Joglekar",
                "Rajesh P. N. Rao",
                "M. N. Vahia",
                "Iravatham Mahadevan",
                "R. Adhikari"
            ],
            "abstract": "The Indus script is one of the major undeciphered scripts of the ancient\nworld. The small size of the corpus, the absence of bilingual texts, and the\nlack of definite knowledge of the underlying language has frustrated efforts at\ndecipherment since the discovery of the remains of the Indus civilisation.\nRecently, some researchers have questioned the premise that the Indus script\nencodes spoken language. Building on previous statistical approaches, we apply\nthe tools of statistical language processing, specifically $n$-gram Markov\nchains, to analyse the Indus script for syntax. Our main results are that the\nscript has well-defined signs which begin and end texts, that there is\ndirectionality and strong correlations in the sign order, and that there are\ngroups of signs which appear to have identical syntactic function. All these\nrequire no {\\it a priori} suppositions regarding the syntactic or semantic\ncontent of the signs, but follow directly from the statistical analysis. Using\ninformation theoretic measures, we find the information in the script to be\nintermediate between that of a completely random and a completely fixed\nordering of signs. Our study reveals that the Indus script is a structured sign\nsystem showing features of a formal language, but, at present, cannot\nconclusively establish that it encodes {\\it natural} language. Our $n$-gram\nMarkov model is useful for predicting signs which are missing or illegible in a\ncorpus of Indus texts. This work forms the basis for the development of a\nstochastic grammar which can be used to explore the syntax of the Indus script\nin greater detail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.4180v2",
            "date": "2009-01-27 06:29:10+00:00",
            "title": "Google distance between words",
            "authors": [
                "Bj\u00f8rn Kjos-Hanssen",
                "Alberto J. Evangelista"
            ],
            "abstract": "Cilibrasi and Vitanyi have demonstrated that it is possible to extract the\nmeaning of words from the world-wide web. To achieve this, they rely on the\nnumber of webpages that are found through a Google search containing a given\nword and they associate the page count to the probability that the word appears\non a webpage. Thus, conditional probabilities allow them to correlate one word\nwith another word's meaning. Furthermore, they have developed a similarity\ndistance function that gauges how closely related a pair of words is. We\npresent a specific counterexample to the triangle inequality for this\nsimilarity distance function.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.1033v1",
            "date": "2009-02-06 09:28:58+00:00",
            "title": "New Confidence Measures for Statistical Machine Translation",
            "authors": [
                "Sylvain Raybaud",
                "Caroline Lavecchia",
                "David Langlois",
                "Kamel Sma\u00efli"
            ],
            "abstract": "A confidence measure is able to estimate the reliability of an hypothesis\nprovided by a machine translation system. The problem of confidence measure can\nbe seen as a process of testing : we want to decide whether the most probable\nsequence of words provided by the machine translation system is correct or not.\nIn the following we describe several original word-level confidence measures\nfor machine translation, based on mutual information, n-gram language model and\nlexical features language model. We evaluate how well they perform individually\nor together, and show that using a combination of confidence measures based on\nmutual information yields a classification error rate as low as 25.1% with an\nF-measure of 0.708.",
            "journal ref": "International Conference On Agents and Artificial Intelligence -\n  ICAART 09 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.2345v1",
            "date": "2009-02-13 17:08:10+00:00",
            "title": "What's in a Message?",
            "authors": [
                "Stergos D. Afantenos",
                "Nicolas Hernandez"
            ],
            "abstract": "In this paper we present the first step in a larger series of experiments for\nthe induction of predicate/argument structures. The structures that we are\ninducing are very similar to the conceptual structures that are used in Frame\nSemantics (such as FrameNet). Those structures are called messages and they\nwere previously used in the context of a multi-document summarization system of\nevolving events. The series of experiments that we are proposing are\nessentially composed from two stages. In the first stage we are trying to\nextract a representative vocabulary of words. This vocabulary is later used in\nthe second stage, during which we apply to it various clustering approaches in\norder to identify the clusters of predicates and arguments--or frames and\nsemantic roles, to use the jargon of Frame Semantics. This paper presents in\ndetail and evaluates the first stage.",
            "journal ref": "12th Conference of the European Chapter of the Association for\n  Computational Linguistics (EACL 2009), workshop on Cognitive Aspects of\n  Computational Language Acquisition. Athens, Greece",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.3072v1",
            "date": "2009-02-18 08:51:28+00:00",
            "title": "Syntactic variation of support verb constructions",
            "authors": [
                "Eric Laporte",
                "Elisabete Ranchhod",
                "Anastasia Yannacopoulou"
            ],
            "abstract": "We report experiments about the syntactic variations of support verb\nconstructions, a special type of multiword expressions (MWEs) containing\npredicative nouns. In these expressions, the noun can occur with or without the\nverb, with no clear-cut semantic difference. We extracted from a large French\ncorpus a set of examples of the two situations and derived statistical results\nfrom these data. The extraction involved large-coverage language resources and\nfinite-state techniques. The results show that, most frequently, predicative\nnouns occur without a support verb. This fact has consequences on methods of\nextracting or recognising MWEs.",
            "journal ref": "Lingvisticae Investigationes 31, 2 (2008) 173-185",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0903.5168v1",
            "date": "2009-03-30 09:45:20+00:00",
            "title": "Mathematical Model for Transformation of Sentences from Active Voice to Passive Voice",
            "authors": [
                "Rakesh Pandey",
                "H. S. Dhami"
            ],
            "abstract": "Formal work in linguistics has both produced and used important mathematical\ntools. Motivated by a survey of models for context and word meaning, syntactic\ncategories, phrase structure rules and trees, an attempt is being made in the\npresent paper to present a mathematical model for structuring of sentences from\nactive voice to passive voice, which is is the form of a transitive verb whose\ngrammatical subject serves as the patient, receiving the action of the verb.\n  For this purpose we have parsed all sentences of a corpus and have generated\nBoolean groups for each of them. It has been observed that when we take\nconstituents of the sentences as subgroups, the sequences of phrases form\npermutation roups. Application of isomorphism property yields permutation\nmapping between the important subgroups. It has resulted in a model for\ntransformation of sentences from active voice to passive voice. A computer\nprogram has been written to enable the software developers to evolve grammar\nsoftware for sentence transformations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0905.1609v1",
            "date": "2009-05-11 12:17:36+00:00",
            "title": "Acquisition of morphological families and derivational series from a machine readable dictionary",
            "authors": [
                "Nabil Hathout"
            ],
            "abstract": "The paper presents a linguistic and computational model aiming at making the\nmorphological structure of the lexicon emerge from the formal and semantic\nregularities of the words it contains. The model is word-based. The proposed\nmorphological structure consists of (1) binary relations that connect each\nheadword with words that are morphologically related, and especially with the\nmembers of its morphological family and its derivational series, and of (2) the\nanalogies that hold between the words. The model has been tested on the lexicon\nof French using the TLFi machine readable dictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.0785v1",
            "date": "2009-07-04 18:43:16+00:00",
            "title": "A Bayesian Model for Discovering Typological Implications",
            "authors": [
                "Hal Daum\u00e9 III",
                "Lyle Campbell"
            ],
            "abstract": "A standard form of analysis for linguistic typology is the universal\nimplication. These implications state facts about the range of extant\nlanguages, such as ``if objects come after verbs, then adjectives come after\nnouns.'' Such implications are typically discovered by painstaking hand\nanalysis over a small sample of languages. We propose a computational model for\nassisting at this process. Our model is able to discover both well-known\nimplications as well as some novel implications that deserve further study.\nMoreover, through a careful application of hierarchical analysis, we are able\nto cope with the well-known sampling problem: languages are not independent.",
            "journal ref": "ACL 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.0804v1",
            "date": "2009-07-04 22:57:26+00:00",
            "title": "Induction of Word and Phrase Alignments for Automatic Document Summarization",
            "authors": [
                "Hal Daum\u00e9 III",
                "Daniel Marcu"
            ],
            "abstract": "Current research in automatic single document summarization is dominated by\ntwo effective, yet naive approaches: summarization by sentence extraction, and\nheadline generation via bag-of-words models. While successful in some tasks,\nneither of these models is able to adequately capture the large set of\nlinguistic devices utilized by humans when they produce summaries. One possible\nexplanation for the widespread use of these models is that good techniques have\nbeen developed to extract appropriate training data for them from existing\ndocument/abstract and document/headline corpora. We believe that future\nprogress in automatic summarization will be driven both by the development of\nmore sophisticated, linguistically informed models, as well as a more effective\nleveraging of document/abstract corpora. In order to open the doors to\nsimultaneously achieving both of these goals, we have developed techniques for\nautomatically producing word-to-word and phrase-to-phrase alignments between\ndocuments and their human-written abstracts. These alignments make explicit the\ncorrespondences that exist in such document/abstract pairs, and create a\npotentially rich data source from which complex summarization algorithms may\nlearn. This paper describes experiments we have carried out to analyze the\nability of humans to perform such alignments, and based on these analyses, we\ndescribe experiments for creating them automatically. Our model for the\nalignment task is based on an extension of the standard hidden Markov model,\nand learns to create alignments in a completely unsupervised fashion. We\ndescribe our model in detail and present experimental results that show that\nour model is able to learn to reliably identify word- and phrase-level\nalignments in a corpus of <document,abstract> pairs.",
            "journal ref": "Computational Linguistics, 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.0806v1",
            "date": "2009-07-04 22:26:47+00:00",
            "title": "A Noisy-Channel Model for Document Compression",
            "authors": [
                "Hal Daum\u00e9 III",
                "Daniel Marcu"
            ],
            "abstract": "We present a document compression system that uses a hierarchical\nnoisy-channel model of text production. Our compression system first\nautomatically derives the syntactic structure of each sentence and the overall\ndiscourse structure of the text given as input. The system then uses a\nstatistical hierarchical model of text production in order to drop\nnon-important syntactic and discourse constituents so as to generate coherent,\ngrammatical document compressions of arbitrary length. The system outperforms\nboth a baseline and a sentence-based compression system that operates by\nsimplifying sequentially all sentences in a text. Our results support the claim\nthat discourse knowledge plays an important role in document summarization.",
            "journal ref": "ACL 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.0807v1",
            "date": "2009-07-04 22:28:15+00:00",
            "title": "A Large-Scale Exploration of Effective Global Features for a Joint Entity Detection and Tracking Model",
            "authors": [
                "Hal Daum\u00e9 III",
                "Daniel Marcu"
            ],
            "abstract": "Entity detection and tracking (EDT) is the task of identifying textual\nmentions of real-world entities in documents, extending the named entity\ndetection and coreference resolution task by considering mentions other than\nnames (pronouns, definite descriptions, etc.). Like NE tagging and coreference\nresolution, most solutions to the EDT task separate out the mention detection\naspect from the coreference aspect. By doing so, these solutions are limited to\nusing only local features for learning. In contrast, by modeling both aspects\nof the EDT task simultaneously, we are able to learn using highly complex,\nnon-local features. We develop a new joint EDT model and explore the utility of\nmany features, demonstrating their effectiveness on this task.",
            "journal ref": "HLT/EMNLP 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.2452v1",
            "date": "2009-07-14 21:02:14+00:00",
            "title": "Pattern Based Term Extraction Using ACABIT System",
            "authors": [
                "Koichi Takeuchi",
                "Kyo Kageura",
                "Teruo Koyama",
                "B\u00e9atrice Daille",
                "Laurent Romary"
            ],
            "abstract": "In this paper, we propose a pattern-based term extraction approach for\nJapanese, applying ACABIT system originally developed for French. The proposed\napproach evaluates termhood using morphological patterns of basic terms and\nterm variants. After extracting term candidates, ACABIT system filters out\nnon-terms from the candidates based on log-likelihood. This approach is\nsuitable for Japanese term extraction because most of Japanese terms are\ncompound nouns or simple phrasal patterns.",
            "journal ref": "IEIC Technical Report 103, 280 (2003) 31-36",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.3781v1",
            "date": "2009-07-22 06:25:59+00:00",
            "title": "Un syst\u00e8me modulaire d'acquisition automatique de traductions \u00e0 partir du Web",
            "authors": [
                "St\u00e9phanie L\u00e9on"
            ],
            "abstract": "We present a method of automatic translation (French/English) of Complex\nLexical Units (CLU) for aiming at extracting a bilingual lexicon. Our modular\nsystem is based on linguistic properties (compositionality, polysemy, etc.).\nDifferent aspects of the multilingual Web are used to validate candidate\ntranslations and collect new terms. We first build a French corpus of Web pages\nto collect CLU. Three adapted processing stages are applied for each linguistic\nproperty : compositional and non polysemous translations, compositional\npolysemous translations and non compositional translations. Our evaluation on a\nsample of CLU shows that our technique based on the Web can reach a very high\nprecision.",
            "journal ref": "TALN'09 (Traitement Automatique des Langues Naturelles), France\n  (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0908.4413v1",
            "date": "2009-08-30 18:50:19+00:00",
            "title": "Multiple Retrieval Models and Regression Models for Prior Art Search",
            "authors": [
                "Patrice Lopez",
                "Laurent Romary"
            ],
            "abstract": "This paper presents the system called PATATRAS (PATent and Article Tracking,\nRetrieval and AnalysiS) realized for the IP track of CLEF 2009. Our approach\npresents three main characteristics: 1. The usage of multiple retrieval models\n(KL, Okapi) and term index definitions (lemma, phrase, concept) for the three\nlanguages considered in the present track (English, French, German) producing\nten different sets of ranked results. 2. The merging of the different results\nbased on multiple regression models using an additional validation set created\nfrom the patent collection. 3. The exploitation of patent metadata and of the\ncitation structures for creating restricted initial working sets of patents and\nfor producing a final re-ranking regression model. As we exploit specific\nmetadata of the patent documents and the citation relations only at the\ncreation of initial working sets and during the final post ranking step, our\narchitecture remains generic and easy to extend.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0908.4431v1",
            "date": "2009-08-30 23:20:41+00:00",
            "title": "An OLAC Extension for Dravidian Languages",
            "authors": [
                "B Prabhulla Chandran Pillai"
            ],
            "abstract": "OLAC was founded in 2000 for creating online databases of language resources.\nThis paper intends to review the bottom-up distributed character of the project\nand proposes an extension of the architecture for Dravidian languages. An\nontological structure is considered for effective natural language processing\n(NLP) and its advantages over statistical methods are reviewed",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.1147v1",
            "date": "2009-09-07 06:29:51+00:00",
            "title": "Empowering OLAC Extension using Anusaaraka and Effective text processing using Double Byte coding",
            "authors": [
                "B Prabhulla Chandran Pillai"
            ],
            "abstract": "The paper reviews the hurdles while trying to implement the OLAC extension\nfor Dravidian / Indian languages. The paper further explores the possibilities\nwhich could minimise or solve these problems. In this context, the Chinese\nsystem of text processing and the anusaaraka system are scrutinised.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.2379v1",
            "date": "2009-09-12 22:27:57+00:00",
            "title": "Implementation of Rule Based Algorithm for Sandhi-Vicheda Of Compound Hindi Words",
            "authors": [
                "Priyanka Gupta",
                "Vishal Goyal"
            ],
            "abstract": "Sandhi means to join two or more words to coin new word. Sandhi literally\nmeans `putting together' or combining (of sounds), It denotes all combinatory\nsound-changes effected (spontaneously) for ease of pronunciation.\nSandhi-vicheda describes [5] the process by which one letter (whether single or\ncojoined) is broken to form two words. Part of the broken letter remains as the\nlast letter of the first word and part of the letter forms the first letter of\nthe next letter. Sandhi- Vicheda is an easy and interesting way that can give\nentirely new dimension that add new way to traditional approach to Hindi\nTeaching. In this paper using the Rule based algorithm we have reported an\naccuracy of 60-80% depending upon the number of rules to be implemented.",
            "journal ref": "P.Gupta and V.Goyal, \" Implementation of Rule Based Algorithm for\n  Sandhi-Vicheda Of Compound Hindi Words\", International Journal of Computer\n  Science Issues (IJCSI), Volume 3, pp45-49, August 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.2626v1",
            "date": "2009-09-14 19:10:59+00:00",
            "title": "Reference Resolution within the Framework of Cognitive Grammar",
            "authors": [
                "Susanne Salmon-Alt",
                "Laurent Romary"
            ],
            "abstract": "Following the principles of Cognitive Grammar, we concentrate on a model for\nreference resolution that attempts to overcome the difficulties previous\napproaches, based on the fundamental assumption that all reference (independent\non the type of the referring expression) is accomplished via access to and\nrestructuring of domains of reference rather than by direct linkage to the\nentities themselves. The model accounts for entities not explicitly mentioned\nbut understood in a discourse, and enables exploitation of discursive and\nperceptual context to limit the set of potential referents for a given\nreferring expression. As the most important feature, we note that a single\nmechanism is required to handle what are typically treated as diverse\nphenomena. Our approach, then, provides a fresh perspective on the relations\nbetween Cognitive Grammar and the problem of reference.",
            "journal ref": "International Colloqium on Cognitive Science, San Sebastian :\n  Spain (2001)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.2715v1",
            "date": "2009-09-15 06:02:56+00:00",
            "title": "Marking-up multiple views of a Text: Discourse and Reference",
            "authors": [
                "Dan Cristea",
                "Nancy Ide",
                "Laurent Romary"
            ],
            "abstract": "We describe an encoding scheme for discourse structure and reference, based\non the TEI Guidelines and the recommendations of the Corpus Encoding\nSpecification (CES). A central feature of the scheme is a CES-based data\narchitecture enabling the encoding of and access to multiple views of a\nmarked-up document. We describe a tool architecture that supports the encoding\nscheme, and then show how we have used the encoding scheme and the tools to\nperform a discourse analytic task in support of a model of global discourse\ncohesion called Veins Theory (Cristea & Ide, 1998).",
            "journal ref": "First International Language Resources and Evaluation Conference,\n  Grenada, Espagne : France (1998)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.2718v1",
            "date": "2009-09-15 06:09:47+00:00",
            "title": "A Common XML-based Framework for Syntactic Annotations",
            "authors": [
                "Nancy Ide",
                "Laurent Romary",
                "Tomaz Erjavec"
            ],
            "abstract": "It is widely recognized that the proliferation of annotation schemes runs\ncounter to the need to re-use language resources, and that standards for\nlinguistic annotation are becoming increasingly mandatory. To answer this need,\nwe have developed a framework comprised of an abstract model for a variety of\ndifferent annotation types (e.g., morpho-syntactic tagging, syntactic\nannotation, co-reference annotation, etc.), which can be instantiated in\ndifferent ways depending on the annotator's approach and goals. In this paper\nwe provide an overview of the framework, demonstrate its applicability to\nsyntactic annotation, and show how it can contribute to comparative evaluation\nof parser output and diverse syntactic annotation schemes.",
            "journal ref": "1st NLP and XML Workshop, Tokyo, Japan : Japan (2001)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.2719v1",
            "date": "2009-09-15 06:10:45+00:00",
            "title": "Standards for Language Resources",
            "authors": [
                "Nancy Ide",
                "Laurent Romary"
            ],
            "abstract": "This paper presents an abstract data model for linguistic annotations and its\nimplementation using XML, RDF and related standards; and to outline the work of\na newly formed committee of the International Standards Organization (ISO),\nISO/TC 37/SC 4 Language Resource Management, which will use this work as its\nstarting point. The primary motive for presenting the latter is to solicit the\nparticipation of members of the research community to contribute to the work of\nthe committee.",
            "journal ref": "Third International Conference on Language Resources and\n  Evaluation - LREC 2002, Las Palmas, Spain : France (2002)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.3027v1",
            "date": "2009-09-16 14:38:19+00:00",
            "title": "Language Models for Handwritten Short Message Services",
            "authors": [
                "Emmanuel Ep Prochasson",
                "Christian Viard-Gaudin",
                "Emmanuel Morin"
            ],
            "abstract": "Handwriting is an alternative method for entering texts composing Short\nMessage Services. However, a whole new language features the texts which are\nproduced. They include for instance abbreviations and other consonantal writing\nwhich sprung up for time saving and fashion. We have collected and processed a\nsignificant number of such handwriting SMS, and used various strategies to\ntackle this challenging area of handwriting recognition. We proposed to study\nmore specifically three different phenomena: consonant skeleton, rebus, and\nphonetic writing. For each of them, we compare the rough results produced by a\nstandard recognition system with those obtained when using a specific language\nmodel.",
            "journal ref": "International Conference on Document Analysis and Recognition,\n  Brazil (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.3028v1",
            "date": "2009-09-16 14:39:05+00:00",
            "title": "Vers la reconnaissance de mini-messages manuscrits",
            "authors": [
                "Emmanuel Prochasson",
                "Emmanuel Morin",
                "Christian Viard-Gaudin"
            ],
            "abstract": "Handwriting is an alternative method for entering texts which composed Short\nMessage Services. However, a whole new language features the texts which are\nproduced. They include for instance abbreviations and other consonantal writing\nwhich sprung up for time saving and fashion. We have collected and processed a\nsignificant number of such handwritten SMS, and used various strategies to\ntackle this challenging area of handwriting recognition. We proposed to study\nmore specifically three different phenomena: consonant skeleton, rebus, and\nphonetic writing. For each of them, we compare the rough results produced by a\nstandard recognition system with those obtained when using a specific language\nmodel to take care of them.",
            "journal ref": "Colloque International sur le Lexique et la Grammaire, Bonifacio :\n  France (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.3444v1",
            "date": "2009-09-18 14:23:05+00:00",
            "title": "Analyse en d\u00e9pendances \u00e0 l'aide des grammaires d'interaction",
            "authors": [
                "Jonathan Marchand",
                "Bruno Guillaume",
                "Guy Perrier"
            ],
            "abstract": "This article proposes a method to extract dependency structures from\nphrase-structure level parsing with Interaction Grammars. Interaction Grammars\nare a formalism which expresses interactions among words using a polarity\nsystem. Syntactical composition is led by the saturation of polarities.\nInteractions take place between constituents, but as grammars are lexicalized,\nthese interactions can be translated at the level of words. Dependency\nrelations are extracted from the parsing process: every dependency is the\nconsequence of a polarity saturation. The dependency relations we obtain can be\nseen as a refinement of the usual dependency tree. Generally speaking, this\nwork sheds new light on links between phrase structure and dependency parsing.",
            "journal ref": "Traitement Automatique des Langues Naturelles, Senlis : France\n  (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.3445v1",
            "date": "2009-09-18 14:25:29+00:00",
            "title": "Grouping Synonyms by Definitions",
            "authors": [
                "Ingrid Falk",
                "Claire Gardent",
                "Evelyne Jacquey",
                "Fabienne Venant"
            ],
            "abstract": "We present a method for grouping the synonyms of a lemma according to its\ndictionary senses. The senses are defined by a large machine readable\ndictionary for French, the TLFi (Tr\\'esor de la langue fran\\c{c}aise\ninformatis\\'e) and the synonyms are given by 5 synonym dictionaries (also for\nFrench). To evaluate the proposed method, we manually constructed a gold\nstandard where for each (word, definition) pair and given the set of synonyms\ndefined for that word by the 5 synonym dictionaries, 4 lexicographers specified\nthe set of synonyms they judge adequate. While inter-annotator agreement ranges\non that task from 67% to at best 88% depending on the annotator pair and on the\nsynonym dictionary being considered, the automatic procedure we propose scores\na precision of 67% and a recall of 71%. The proposed method is compared with\nrelated work namely, word sense disambiguation, synonym lexicon acquisition and\nWordNet construction.",
            "journal ref": "Recent Advances in Natural Language Processing (RANLP), Borovets :\n  Bulgaria (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.3591v1",
            "date": "2009-09-19 15:38:55+00:00",
            "title": "Mathematics, Recursion, and Universals in Human Languages",
            "authors": [
                "P. Gilkey",
                "S. Lopez Ornat",
                "A. Karousou"
            ],
            "abstract": "There are many scientific problems generated by the multiple and conflicting\nalternative definitions of linguistic recursion and human recursive processing\nthat exist in the literature. The purpose of this article is to make available\nto the linguistic community the standard mathematical definition of recursion\nand to apply it to discuss linguistic recursion. As a byproduct, we obtain an\ninsight into certain \"soft universals\" of human languages, which are related to\ncognitive constructs necessary to implement mathematical reasoning, i.e.\nmathematical model theory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0909.4280v1",
            "date": "2009-09-23 18:55:53+00:00",
            "title": "Towards Multimodal Content Representation",
            "authors": [
                "Harry Bunt",
                "Laurent Romary"
            ],
            "abstract": "Multimodal interfaces, combining the use of speech, graphics, gestures, and\nfacial expressions in input and output, promise to provide new possibilities to\ndeal with information in more effective and efficient ways, supporting for\ninstance: - the understanding of possibly imprecise, partial or ambiguous\nmultimodal input; - the generation of coordinated, cohesive, and coherent\nmultimodal presentations; - the management of multimodal interaction (e.g.,\ntask completion, adapting the interface, error prevention) by representing and\nexploiting models of the user, the domain, the task, the interactive context,\nand the media (e.g. text, audio, video). The present document is intended to\nsupport the discussion on multimodal content representation, its possible\nobjectives and basic constraints, and how the definition of a generic\nrepresentation framework for multimodal content representation may be\napproached. It takes into account the results of the Dagstuhl workshop, in\nparticular those of the informal working group on multimodal meaning\nrepresentation that was active during the workshop (see\nhttp://www.dfki.de/~wahlster/Dagstuhl_Multi_Modality, Working Group 4).",
            "journal ref": "LREC Workshop on International Standards of Terminology and\n  Language Resources Management, Las Palams : Spain (2002)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.0894v1",
            "date": "2009-11-04 19:33:21+00:00",
            "title": "A New Computational Schema for Euphonic Conjunctions in Sanskrit Processing",
            "authors": [
                "N. Rama",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "Automated language processing is central to the drive to enable facilitated\nreferencing of increasingly available Sanskrit E texts. The first step towards\nprocessing Sanskrit text involves the handling of Sanskrit compound words that\nare an integral part of Sanskrit texts. This firstly necessitates the\nprocessing of euphonic conjunctions or sandhis, which are points in words or\nbetween words, at which adjacent letters coalesce and transform. The ancient\nSanskrit grammarian Panini's codification of the Sanskrit grammar is the\naccepted authority in the subject. His famed sutras or aphorisms, numbering\napproximately four thousand, tersely, precisely and comprehensively codify the\nrules of the grammar, including all the rules pertaining to sandhis. This work\npresents a fresh new approach to processing sandhis in terms of a computational\nschema. This new computational model is based on Panini's complex codification\nof the rules of grammar. The model has simple beginnings and is yet powerful,\ncomprehensive and computationally lean.",
            "journal ref": "Rama N. and M. Lakshmanan, \"A New Computational Schema for\n  Euphonic Conjunctions in Sanskrit Processing\", International Journal of\n  Computer Science Issues, IJCSI, Volume 5, pp43-51, October 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.0907v1",
            "date": "2009-11-04 18:32:08+00:00",
            "title": "ANN-based Innovative Segmentation Method for Handwritten text in Assamese",
            "authors": [
                "Kaustubh Bhattacharyya",
                "Kandarpa Kumar Sarma"
            ],
            "abstract": "Artificial Neural Network (ANN) s has widely been used for recognition of\noptically scanned character, which partially emulates human thinking in the\ndomain of the Artificial Intelligence. But prior to recognition, it is\nnecessary to segment the character from the text to sentences, words etc.\nSegmentation of words into individual letters has been one of the major\nproblems in handwriting recognition. Despite several successful works all over\nthe work, development of such tools in specific languages is still an ongoing\nprocess especially in the Indian context. This work explores the application of\nANN as an aid to segmentation of handwritten characters in Assamese- an\nimportant language in the North Eastern part of India. The work explores the\nperformance difference obtained in applying an ANN-based dynamic segmentation\nalgorithm compared to projection- based static segmentation. The algorithm\ninvolves, first training of an ANN with individual handwritten characters\nrecorded from different individuals. Handwritten sentences are separated out\nfrom text using a static segmentation method. From the segmented line,\nindividual characters are separated out by first over segmenting the entire\nline. Each of the segments thus obtained, next, is fed to the trained ANN. The\npoint of segmentation at which the ANN recognizes a segment or a combination of\nseveral segments to be similar to a handwritten character, a segmentation\nboundary for the character is assumed to exist and segmentation performed. The\nsegmented character is next compared to the best available match and the\nsegmentation boundary confirmed.",
            "journal ref": "K. Bhattacharyya and K. K. Sarma, \"ANN-based Innovative\n  Segmentation Method for Handwritten text in Assamese\", International Journal\n  of Computer Science Issues, Volume 5, pp9-16, October 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.1516v2",
            "date": "2009-11-08 10:57:04+00:00",
            "title": "A Discourse-based Approach in Text-based Machine Translation",
            "authors": [
                "Sana Ullah",
                "M. A. Khan",
                "Kyung Sup Kwak"
            ],
            "abstract": "This paper presents a theoretical research based approach to ellipsis\nresolution in machine translation. The formula of discourse is applied in order\nto resolve ellipses. The validity of the discourse formula is analyzed by\napplying it to the real world text, i.e., newspaper fragments. The source text\nis converted into mono-sentential discourses where complex discourses require\nfurther dissection either directly into primitive discourses or first into\ncompound discourses and later into primitive ones. The procedure of dissection\nneeds further improvement, i.e., discovering as many primitive discourse forms\nas possible. An attempt has been made to investigate new primitive discourses\nor patterns from the given text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.1517v2",
            "date": "2009-11-09 13:52:05+00:00",
            "title": "Resolution of Unidentified Words in Machine Translation",
            "authors": [
                "Sana Ullah",
                "M. Asdaque Hussain",
                "Kyung Sup Kwak"
            ],
            "abstract": "This paper presents a mechanism of resolving unidentified lexical units in\nText-based Machine Translation (TBMT). In a Machine Translation (MT) system it\nis unlikely to have a complete lexicon and hence there is intense need of a new\nmechanism to handle the problem of unidentified words. These unknown words\ncould be abbreviations, names, acronyms and newly introduced terms. We have\nproposed an algorithm for the resolution of the unidentified words. This\nalgorithm takes discourse unit (primitive discourse) as a unit of analysis and\nprovides real time updates to the lexicon. We have manually applied the\nalgorithm to news paper fragments. Along with anaphora and cataphora\nresolution, many unknown words especially names and abbreviations were updated\nto the lexicon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.1842v1",
            "date": "2009-11-10 07:12:03+00:00",
            "title": "Standards for Language Resources",
            "authors": [
                "Nancy Ide",
                "Laurent Romary"
            ],
            "abstract": "The goal of this paper is two-fold: to present an abstract data model for\nlinguistic annotations and its implementation using XML, RDF and related\nstandards; and to outline the work of a newly formed committee of the\nInternational Standards Organization (ISO), ISO/TC 37/SC 4 Language Resource\nManagement, which will use this work as its starting point.",
            "journal ref": "IRCS Workshop on Linguistic Databases, Philadelphia : United\n  States (2001)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.2284v2",
            "date": "2009-11-12 01:48:12+00:00",
            "title": "A New Look at the Classical Entropy of Written English",
            "authors": [
                "Fabio G. Guerrero"
            ],
            "abstract": "A simple method for finding the entropy and redundancy of a reasonable long\nsample of English text by direct computer processing and from first principles\naccording to Shannon theory is presented. As an example, results on the entropy\nof the English language have been obtained based on a total of 20.3 million\ncharacters of written English, considering symbols from one to five hundred\ncharacters in length. Besides a more realistic value of the entropy of English,\na new perspective on some classic entropy-related concepts is presented. This\nmethod can also be extended to other Latin languages. Some implications for\npractical applications such as plagiarism-detection software, and the minimum\nnumber of words that should be used in social Internet network messaging, are\ndiscussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.5116v1",
            "date": "2009-11-26 16:27:58+00:00",
            "title": "Standardization of the formal representation of lexical information for NLP",
            "authors": [
                "Laurent Romary"
            ],
            "abstract": "A survey of dictionary models and formats is presented as well as a\npresentation of corresponding recent standardisation activities.",
            "journal ref": "Dictionarie. An International Encyclopedia of Lexicography.\n  Supplementary volume: Recent developments with special focus on computational\n  lexicography (2010) -",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.1820v1",
            "date": "2009-12-09 18:03:20+00:00",
            "title": "Parsing of part-of-speech tagged Assamese Texts",
            "authors": [
                "Mirzanur Rahman",
                "Sufal Das",
                "Utpal Sharma"
            ],
            "abstract": "A natural language (or ordinary language) is a language that is spoken,\nwritten, or signed by humans for general-purpose communication, as\ndistinguished from formal languages (such as computer-programming languages or\nthe \"languages\" used in the study of formal logic). The computational\nactivities required for enabling a computer to carry out information processing\nusing natural language is called natural language processing. We have taken\nAssamese language to check the grammars of the input sentence. Our aim is to\nproduce a technique to check the grammatical structures of the sentences in\nAssamese text. We have made grammar rules by analyzing the structures of\nAssamese sentences. Our parsing program finds the grammatical errors, if any,\nin the Assamese sentence. If there is no error, the program will generate the\nparse tree for the Assamese sentence",
            "journal ref": "M. Rahman, S. Das and U. Sharma, \"Parsing of part-of-speech tagged\n  Assamese Texts\", International Journal of Computer Science Issues, IJCSI,\n  Volume 6, Issue 1, pp28-34, November 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.2881v2",
            "date": "2009-12-15 13:30:14+00:00",
            "title": "Representing human and machine dictionaries in Markup languages",
            "authors": [
                "Lothar Lemnitzer",
                "Laurent Romary",
                "Andreas Witt"
            ],
            "abstract": "In this chapter we present the main issues in representing machine readable\ndictionaries in XML, and in particular according to the Text Encoding\nDictionary (TEI) guidelines.",
            "journal ref": "Dictionaries. An International Encyclopedia of Lexicography.\n  Supplementary volume: Recent developments with special focus on computational\n  lexicography, Ulrich Heid (Ed.) (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.0675v1",
            "date": "2009-06-03 09:53:12+00:00",
            "title": "Encoding models for scholarly literature",
            "authors": [
                "Martin Holmes",
                "Laurent Romary"
            ],
            "abstract": "We examine the issue of digital formats for document encoding, archiving and\npublishing, through the specific example of \"born-digital\" scholarly journal\narticles. We will begin by looking at the traditional workflow of journal\nediting and publication, and how these practices have made the transition into\nthe online domain. We will examine the range of different file formats in which\nelectronic articles are currently stored and published. We will argue strongly\nthat, despite the prevalence of binary and proprietary formats such as PDF and\nMS Word, XML is a far superior encoding choice for journal articles. Next, we\nlook at the range of XML document structures (DTDs, Schemas) which are in\ncommon use for encoding journal articles, and consider some of their strengths\nand weaknesses. We will suggest that, despite the existence of specialized\nschemas intended specifically for journal articles (such as NLM), and more\nbroadly-used publication-oriented schemas such as DocBook, there are strong\narguments in favour of developing a subset or customization of the Text\nEncoding Initiative (TEI) schema for the purpose of journal-article encoding;\nTEI is already in use in a number of journal publication projects, and the\nscale and precision of the TEI tagset makes it particularly appropriate for\nencoding scholarly articles. We will outline the document structure of a\nTEI-encoded journal article, and look in detail at suggested markup patterns\nfor specific features of journal articles.",
            "journal ref": "Publishing and digital libraries: Legal and organizational issues,\n  Ioannis Iglezakis, Tatiana-Eleni Synodinou, Sarantos Kapidakis (Ed.) (2010) -",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.2415v1",
            "date": "2009-06-12 23:50:06+00:00",
            "title": "Without a 'doubt'? Unsupervised discovery of downward-entailing operators",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee",
                "Richard Ducott"
            ],
            "abstract": "An important part of textual inference is making deductions involving\nmonotonicity, that is, determining whether a given assertion entails\nrestrictions or relaxations of that assertion. For instance, the statement 'We\nknow the epidemic spread quickly' does not entail 'We know the epidemic spread\nquickly via fleas', but 'We doubt the epidemic spread quickly' entails 'We\ndoubt the epidemic spread quickly via fleas'. Here, we present the first\nalgorithm for the challenging lexical-semantics problem of learning linguistic\nconstructions that, like 'doubt', are downward entailing (DE). Our algorithm is\nunsupervised, resource-lean, and effective, accurately recovering many DE\noperators that are missing from the hand-constructed lists that\ntextual-inference systems currently use.",
            "journal ref": "Proceedings of NAACL HLT, pp. 137--145, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.5114v1",
            "date": "2009-06-28 02:32:53+00:00",
            "title": "Non-Parametric Bayesian Areal Linguistics",
            "authors": [
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We describe a statistical model over linguistic areas and phylogeny.\n  Our model recovers known areas and identifies a plausible hierarchy of areal\nfeatures. The use of areas improves genetic reconstruction of languages both\nqualitatively and quantitatively according to a variety of metrics. We model\nlinguistic areas by a Pitman-Yor process and linguistic phylogeny by Kingman's\ncoalescent.",
            "journal ref": "Proceedings of the Conference of the North American Association\n  for Computational Linguistics, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0910.0537v1",
            "date": "2009-10-03 12:26:47+00:00",
            "title": "A Note On Higher Order Grammar",
            "authors": [
                "Victor Gluzberg"
            ],
            "abstract": "Both syntax-phonology and syntax-semantics interfaces in Higher Order Grammar\n(HOG) are expressed as axiomatic theories in higher-order logic (HOL), i.e. a\nlanguage is defined entirely in terms of provability in the single logical\nsystem. An important implication of this elegant architecture is that the\nmeaning of a valid expression turns out to be represented not by a single, nor\neven by a few \"discrete\" terms (in case of ambiguity), but by a \"continuous\"\nset of logically equivalent terms. The note is devoted to precise formulation\nand proof of this observation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0910.1484v1",
            "date": "2009-10-08 12:21:22+00:00",
            "title": "Ludics and its Applications to natural Language Semantics",
            "authors": [
                "Alain Lecomte",
                "Myriam Quatrini"
            ],
            "abstract": "Proofs, in Ludics, have an interpretation provided by their counter-proofs,\nthat is the objects they interact with. We follow the same idea by proposing\nthat sentence meanings are given by the counter-meanings they are opposed to in\na dialectical interaction. The conception is at the intersection of a\nproof-theoretic and a game-theoretic accounts of semantics, but it enlarges\nthem by allowing to deal with possibly infinite processes.",
            "journal ref": "Lecture Notes in Artificial Intelligence LNAI, 5514 (2009) pp\n  242--255",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0910.1868v1",
            "date": "2009-10-09 21:40:26+00:00",
            "title": "Evaluation of Hindi to Punjabi Machine Translation System",
            "authors": [
                "Vishal Goyal",
                "Gurpreet Singh Lehal"
            ],
            "abstract": "Machine Translation in India is relatively young. The earliest efforts date\nfrom the late 80s and early 90s. The success of every system is judged from its\nevaluation experimental results. Number of machine translation systems has been\nstarted for development but to the best of author knowledge, no high quality\nsystem has been completed which can be used in real applications. Recently,\nPunjabi University, Patiala, India has developed Punjabi to Hindi Machine\ntranslation system with high accuracy of about 92%. Both the systems i.e.\nsystem under question and developed system are between same closely related\nlanguages. Thus, this paper presents the evaluation results of Hindi to Punjabi\nmachine translation system. It makes sense to use same evaluation criteria as\nthat of Punjabi to Hindi Punjabi Machine Translation System. After evaluation,\nthe accuracy of the system is found to be about 95%.",
            "journal ref": "V.Goyal and G. Singh Lehal, \"Evaluation of Hindi to Punjabi\n  Machine Translation System\", International Journal of Computer Science\n  Issues, IJCSI, Volume 4, Issue 1, pp36-39, September 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.2216v1",
            "date": "2009-01-15 10:22:28+00:00",
            "title": "Discovering Global Patterns in Linguistic Networks through Spectral Analysis: A Case Study of the Consonant Inventories",
            "authors": [
                "Animesh Mukherjee",
                "Monojit Choudhury",
                "Ravi Kannan"
            ],
            "abstract": "Recent research has shown that language and the socio-cognitive phenomena\nassociated with it can be aptly modeled and visualized through networks of\nlinguistic entities. However, most of the existing works on linguistic networks\nfocus only on the local properties of the networks. This study is an attempt to\nanalyze the structure of languages via a purely structural technique, namely\nspectral analysis, which is ideally suited for discovering the global\ncorrelations in a network. Application of this technique to PhoNet, the\nco-occurrence network of consonants, not only reveals several natural\nlinguistic principles governing the structure of the consonant inventories, but\nis also able to quantify their relative importance. We believe that this\npowerful technique can be successfully applied, in general, to study the\nstructure of natural languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.2924v1",
            "date": "2009-01-19 21:19:55+00:00",
            "title": "Universal Complex Structures in Written Language",
            "authors": [
                "Alvaro Corral",
                "Ramon Ferrer-i-Cancho",
                "Gemma Boleda",
                "Albert Diaz-Guilera",
                "."
            ],
            "abstract": "Quantitative linguistics has provided us with a number of empirical laws that\ncharacterise the evolution of languages and competition amongst them. In terms\nof language usage, one of the most influential results is Zipf's law of word\nfrequencies. Zipf's law appears to be universal, and may not even be unique to\nhuman language. However, there is ongoing controversy over whether Zipf's law\nis a good indicator of complexity. Here we present an alternative approach that\nputs Zipf's law in the context of critical phenomena (the cornerstone of\ncomplexity in physics) and establishes the presence of a large scale\n\"attraction\" between successive repetitions of words. Moreover, this phenomenon\nis scale-invariant and universal -- the pattern is independent of word\nfrequency and is observed in texts by different authors and written in\ndifferent languages. There is evidence, however, that the shape of the scaling\nrelation changes for words that play a key role in the text, implying the\nexistence of different \"universality classes\" in the repetition of words. These\nbehaviours exhibit striking parallels with complex catastrophic phenomena.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0901.3291v1",
            "date": "2009-01-21 15:24:59+00:00",
            "title": "Approaching the linguistic complexity",
            "authors": [
                "Stanislaw Drozdz",
                "Jaroslaw Kwapien",
                "Adam Orczyk"
            ],
            "abstract": "We analyze the rank-frequency distributions of words in selected English and\nPolish texts. We compare scaling properties of these distributions in both\nlanguages. We also study a few small corpora of Polish literary texts and find\nthat for a corpus consisting of texts written by different authors the basic\nscaling regime is broken more strongly than in the case of comparable corpus\nconsisting of texts written by the same author. Similarly, for a corpus\nconsisting of texts translated into Polish from other languages the scaling\nregime is broken more strongly than for a comparable corpus of native Polish\ntexts. Moreover, based on the British National Corpus, we consider the\nrank-frequency distributions of the grammatically basic forms of words (lemmas)\ntagged with their proper part of speech. We find that these distributions do\nnot scale if each part of speech is analyzed separately. The only part of\nspeech that independently develops a trace of scaling is verbs.",
            "journal ref": "Complex Sciences, Lect. Notes ICST vol.4, 1044-1050 (Springer,\n  2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.3990v1",
            "date": "2009-01-26 15:52:21+00:00",
            "title": "Du corpus au dictionnaire",
            "authors": [
                "Bernard Jacquemin",
                "Sabine Ploux"
            ],
            "abstract": "In this article, we propose an automatic process to build multi-lingual\nlexico-semantic resources. The goal of these resources is to browse\nsemantically textual information contained in texts of different languages.\nThis method uses a mathematical model called Atlas s\\'emantiques in order to\nrepresent the different senses of each word. It uses the linguistic relations\nbetween words to create graphs that are projected into a semantic space. These\nprojections constitute semantic maps that denote the sense trends of each given\nword. This model is fed with syntactic relations between words extracted from a\ncorpus. Therefore, the lexico-semantic resource produced describes all the\nwords and all their meanings observed in the corpus. The sense trends are\nexpressed by syntactic contexts, typical for a given meaning. The link between\neach sense trend and the utterances used to build the sense trend are also\nstored in an index. Thus all the instances of a word in a particular sense are\nlinked and can be browsed easily. And by using several corpora of different\nlanguages, several resources are built that correspond with each other through\nlanguages. It makes it possible to browse information through languages thanks\nto syntactic contexts translations (even if some of them are partial).",
            "journal ref": "Cahiers de Linguistique. Revue de sociolinguistique et de\n  sociologie de la langue fran\\c{c}aise 33, 1 (2008) 63-84",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.0606v1",
            "date": "2009-02-03 21:04:33+00:00",
            "title": "Beyond Zipf's law: Modeling the structure of human language",
            "authors": [
                "M. Angeles Serrano",
                "Alessandro Flammini",
                "Filippo Menczer"
            ],
            "abstract": "Human language, the most powerful communication system in history, is closely\nassociated with cognition. Written text is one of the fundamental\nmanifestations of language, and the study of its universal regularities can\ngive clues about how our brains process information and how we, as a society,\norganize and share it. Still, only classical patterns such as Zipf's law have\nbeen explored in depth. In contrast, other basic properties like the existence\nof bursts of rare words in specific documents, the topical organization of\ncollections, or the sublinear growth of vocabulary size with the length of a\ndocument, have only been studied one by one and mainly applying heuristic\nmethodologies rather than basic principles and general mechanisms. As a\nconsequence, there is a lack of understanding of linguistic processes as\ncomplex emergent phenomena. Beyond Zipf's law for word frequencies, here we\nfocus on Heaps' law, burstiness, and the topicality of document collections,\nwhich encode correlations within and across documents absent in random null\nmodels. We introduce and validate a generative model that explains the\nsimultaneous emergence of all these patterns from simple rules. As a result, we\nfind a connection between the bursty nature of rare words and the topical\norganization of texts and identify dynamic word ranking and memory across\ndocuments as key mechanisms explaining the non trivial organization of written\ntext. Our research can have broad implications and practical applications in\ncomputer science, cognitive science, and linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.2230v1",
            "date": "2009-02-12 23:02:06+00:00",
            "title": "BagPack: A general framework to represent semantic relations",
            "authors": [
                "Ama\u00e7 Herda\u011fdelen",
                "Marco Baroni"
            ],
            "abstract": "We introduce a way to represent word pairs instantiating arbitrary semantic\nrelations that keeps track of the contexts in which the words in the pair occur\nboth together and independently. The resulting features are of sufficient\ngenerality to allow us, with the help of a standard supervised machine learning\nalgorithm, to tackle a variety of unrelated semantic tasks with good results\nand almost no task-specific tailoring.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0902.4060v1",
            "date": "2009-02-24 04:53:49+00:00",
            "title": "Network of two-Chinese-character compound words in Japanese language",
            "authors": [
                "Ken Yamamoto",
                "Yoshihiro Yamazaki"
            ],
            "abstract": "Some statistical properties of a network of two-Chinese-character compound\nwords in Japanese language are reported. In this network, a node represents a\nChinese character and an edge represents a two-Chinese-character compound word.\nIt is found that this network has properties of \"small-world\" and \"scale-free.\"\nA network formed by only Chinese characters for common use ({\\it joyo-kanji} in\nJapanese), which is regarded as a subclass of the original network, also has\nsmall-world property. However, a degree distribution of the network exhibits no\nclear power law. In order to reproduce disappearance of the power-law property,\na model for a selecting process of the Chinese characters for common use is\nproposed.",
            "journal ref": "Physica A 388, 2555-2560 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0905.0740v1",
            "date": "2009-05-06 04:29:51+00:00",
            "title": "A FORTRAN coded regular expression Compiler for IBM 1130 Computing System",
            "authors": [
                "Gerardo Cisneros"
            ],
            "abstract": "REC (Regular Expression Compiler) is a concise programming language which\nallows students to write programs without knowledge of the complicated syntax\nof languages like FORTRAN and ALGOL. The language is recursive and contains\nonly four elements for control. This paper describes an interpreter of REC\nwritten in FORTRAN.",
            "journal ref": "Acta Mexicana de Ciencia y Tecnologia Vol. IV No. 1, page 30-86,\n  1970",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0905.1130v1",
            "date": "2009-05-07 20:21:45+00:00",
            "title": "Statistical Automatic Summarization in Organic Chemistry",
            "authors": [
                "Florian Boudin",
                "Patricia Velazquez-Morales",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "We present an oriented numerical summarizer algorithm, applied to producing\nautomatic summaries of scientific documents in Organic Chemistry. We present\nits implementation named Yachs (Yet Another Chemistry Summarizer) that combines\na specific document pre-processing with a sentence scoring method relying on\nthe statistical properties of documents. We show that Yachs achieves the best\nresults among several other summarizers on a corpus of Organic Chemistry\narticles.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0905.2990v1",
            "date": "2009-05-18 21:35:52+00:00",
            "title": "Automatic Summarization System coupled with a Question-Answering System (QAAS)",
            "authors": [
                "Juan-Manuel Torres-Moreno",
                "Pier-Luc St-Onge",
                "Michel Gagnon",
                "Marc El-B\u00e8ze",
                "Patrice Bellot"
            ],
            "abstract": "To select the most relevant sentences of a document, it uses an optimal\ndecision algorithm that combines several metrics. The metrics processes,\nweighting and extract pertinence sentences by statistical and informational\nalgorithms. This technique might improve a Question-Answering system, whose\nfunction is to provide an exact answer to a question in natural language. In\nthis paper, we present the results obtained by coupling the Cortex summarizer\nwith a Question-Answering system (QAAS). Two configurations have been\nevaluated. In the first one, a low compression level is selected and the\nsummarization system is only used as a noise filter. In the second\nconfiguration, the system actually functions as a summarizer, with a very high\nlevel of compression. Our results on French corpus demonstrate that the\ncoupling of Automatic Summarization system with a Question-Answering system is\npromising. Then the system has been adapted to generate a customized summary\ndepending on the specific question. Tests on a french multi-document corpus\nhave been realized, and the personalized QAAS system obtains the best\nperformances.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0905.4039v1",
            "date": "2009-05-25 15:42:40+00:00",
            "title": "Normalized Web Distance and Word Similarity",
            "authors": [
                "Rudi L. Cilibrasi",
                "Paul M. B. Vitanyi"
            ],
            "abstract": "There is a great deal of work in cognitive psychology, linguistics, and\ncomputer science, about using word (or phrase) frequencies in context in text\ncorpora to develop measures for word similarity or word association, going back\nto at least the 1960s. The goal of this chapter is to introduce the\nnormalizedis a general way to tap the amorphous low-grade knowledge available\nfor free on the Internet, typed in by local users aiming at personal\ngratification of diverse objectives, and yet globally achieving what is\neffectively the largest semantic electronic database in the world. Moreover,\nthis database is available for all by using any search engine that can return\naggregate page-count estimates for a large range of search-queries. In the\npaper introducing the NWD it was called `normalized Google distance (NGD),' but\nsince Google doesn't allow computer searches anymore, we opt for the more\nneutral and descriptive NWD. web distance (NWD) method to determine similarity\nbetween words and phrases. It",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.0784v1",
            "date": "2009-07-04 18:42:01+00:00",
            "title": "Cross-Task Knowledge-Constrained Self Training",
            "authors": [
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We present an algorithmic framework for learning multiple related tasks. Our\nframework exploits a form of prior knowledge that relates the output spaces of\nthese tasks. We present PAC learning results that analyze the conditions under\nwhich such learning is possible. We present results on learning a shallow\nparser and named-entity recognition system that exploits our framework, showing\nconsistent improvements over baseline methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0907.0786v1",
            "date": "2009-07-04 18:48:34+00:00",
            "title": "Search-based Structured Prediction",
            "authors": [
                "Hal Daum\u00e9 III",
                "John Langford",
                "Daniel Marcu"
            ],
            "abstract": "We present Searn, an algorithm for integrating search and learning to solve\ncomplex structured prediction problems such as those that occur in natural\nlanguage, speech, computational biology, and vision. Searn is a meta-algorithm\nthat transforms these complex problems into simple classification problems to\nwhich any binary classifier may be applied. Unlike current algorithms for\nstructured learning that require decomposition of both the loss function and\nthe feature functions over the predicted structure, Searn is able to learn\nprediction functions for any loss function and any class of features. Moreover,\nSearn comes with a strong, natural theoretical guarantee: good performance on\nthe derived classification problems implies good performance on the structured\nprediction problem.",
            "journal ref": "Machine Learning Journal 2009",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0907.0809v1",
            "date": "2009-07-04 22:34:25+00:00",
            "title": "Learning as Search Optimization: Approximate Large Margin Methods for Structured Prediction",
            "authors": [
                "Hal Daum\u00e9 III",
                "Daniel Marcu"
            ],
            "abstract": "Mappings to structured output spaces (strings, trees, partitions, etc.) are\ntypically learned using extensions of classification algorithms to simple\ngraphical structures (eg., linear chains) in which search and parameter\nestimation can be performed exactly. Unfortunately, in many complex problems,\nit is rare that exact search or parameter estimation is tractable. Instead of\nlearning exact models and searching via heuristic means, we embrace this\ndifficulty and treat the structured output problem in terms of approximate\nsearch. We present a framework for learning as search optimization, and two\nparameter updates with convergence theorems and bounds. Empirical evidence\nshows that our integrated approach to learning and decoding can outperform\nexact models at smaller computational cost.",
            "journal ref": "ICML 2005",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0907.1815v1",
            "date": "2009-07-10 13:25:48+00:00",
            "title": "Frustratingly Easy Domain Adaptation",
            "authors": [
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We describe an approach to domain adaptation that is appropriate exactly in\nthe case when one has enough ``target'' data to do slightly better than just\nusing only ``source'' data. Our approach is incredibly simple, easy to\nimplement as a preprocessing step (10 lines of Perl!) and outperforms\nstate-of-the-art approaches on a range of datasets. Moreover, it is trivially\nextended to a multi-domain adaptation problem, where one has data from a\nvariety of different domains.",
            "journal ref": "ACL 2007",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0907.4960v1",
            "date": "2009-07-28 19:24:51+00:00",
            "title": "Ezhil: A Tamil Programming Language",
            "authors": [
                "Muthiah Annamalai"
            ],
            "abstract": "Ezhil is a Tamil language based interpreted procedural programming language.\nTamil keywords and grammar are chosen to make the native Tamil speaker write\nprograms in the Ezhil system. Ezhil allows easy representation of computer\nprogram closer to the Tamil language logical constructs equivalent to the\nconditional, branch and loop statements in modern English based programming\nlanguages. Ezhil is a compact programming language aimed towards Tamil speaking\nnovice computer users. Grammar for Ezhil and a few example programs are\nreported here, from the initial proof-of-concept implementation using the\nPython programming language1. To the best of our knowledge, Ezhil language is\nthe first freely available Tamil programming language.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/0909.1308v2",
            "date": "2009-09-07 18:48:42+00:00",
            "title": "Efficient Learning of Sparse Conditional Random Fields for Supervised Sequence Labelling",
            "authors": [
                "Nataliya Sokolovska",
                "Thomas Lavergne",
                "Olivier Capp\u00e9",
                "Fran\u00e7ois Yvon"
            ],
            "abstract": "Conditional Random Fields (CRFs) constitute a popular and efficient approach\nfor supervised sequence labelling. CRFs can cope with large description spaces\nand can integrate some form of structural dependency between labels. In this\ncontribution, we address the issue of efficient feature selection for CRFs\nbased on imposing sparsity through an L1 penalty. We first show how sparsity of\nthe parameter set can be exploited to significantly speed up training and\nlabelling. We then introduce coordinate descent parameter update schemes for\nCRFs with L1 regularization. We finally provide some empirical comparisons of\nthe proposed approach with state-of-the-art CRF training strategies. In\nparticular, it is shown that the proposed approach is able to take profit of\nthe sparsity to speed up processing and hence potentially handle larger\ndimensional models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/0911.1451v1",
            "date": "2009-11-07 19:15:33+00:00",
            "title": "Co-word Analysis using the Chinese Character Set",
            "authors": [
                "Loet Leydesdorff",
                "Ping Zhou"
            ],
            "abstract": "Until recently, Chinese texts could not be studied using co-word analysis\nbecause the words are not separated by spaces in Chinese (and Japanese). A word\ncan be composed of one or more characters. The online availability of programs\nthat separate Chinese texts makes it possible to analyze them using semantic\nmaps. Chinese characters contain not only information, but also meaning. This\nmay enhance the readability of semantic maps. In this study, we analyze 58\nwords which occur ten or more times in the 1652 journal titles of the China\nScientific and Technical Papers and Citations Database. The word occurrence\nmatrix is visualized and factor-analyzed.",
            "journal ref": "Co-Word Analysis using the Chinese Character Set, Journal of the\n  American Society for Information Science and Technology 59(9), 1528-1530,\n  2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.1965v1",
            "date": "2009-11-10 19:52:01+00:00",
            "title": "Active Learning for Mention Detection: A Comparison of Sentence Selection Strategies",
            "authors": [
                "Nitin Madnani",
                "Hongyan Jing",
                "Nanda Kambhatla",
                "Salim Roukos"
            ],
            "abstract": "We propose and compare various sentence selection strategies for active\nlearning for the task of detecting mentions of entities. The best strategy\nemploys the sum of confidences of two statistical classifiers trained on\ndifferent views of the data. Our experimental results show that, compared to\nthe random selection strategy, this strategy reduces the amount of required\nlabeled training data by over 50% while achieving the same performance. The\neffect is even more significant when only named mentions are considered: the\nsystem achieves the same performance by using only 42% of the training data\nrequired by the random selection strategy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.5568v1",
            "date": "2009-11-30 08:07:16+00:00",
            "title": "Acquisition d'informations lexicales \u00e0 partir de corpus C\u00e9dric Messiant et Thierry Poibeau",
            "authors": [
                "C\u00e9dric Messiant",
                "Thierry Poibeau"
            ],
            "abstract": "This paper is about automatic acquisition of lexical information from\ncorpora, especially subcategorization acquisition.",
            "journal ref": "Troisi\\`eme colloque international de l'Association Fran\\c{c}aise\n  de Linguistique Cognitive (AFLICO), Nanterre : France (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.5703v1",
            "date": "2009-11-30 18:15:35+00:00",
            "title": "Hierarchies in Dictionary Definition Space",
            "authors": [
                "Olivier Picard",
                "Alexandre Blondin-Masse",
                "Stevan Harnad",
                "Odile Marcotte",
                "Guillaume Chicoisne",
                "Yassine Gargouri"
            ],
            "abstract": "A dictionary defines words in terms of other words. Definitions can tell you\nthe meanings of words you don't know, but only if you know the meanings of the\ndefining words. How many words do you need to know (and which ones) in order to\nbe able to learn all the rest from definitions? We reduced dictionaries to\ntheir \"grounding kernels\" (GKs), about 10% of the dictionary, from which all\nthe other words could be defined. The GK words turned out to have\npsycholinguistic correlates: they were learned at an earlier age and more\nconcrete than the rest of the dictionary. But one can compress still more: the\nGK turns out to have internal structure, with a strongly connected \"kernel\ncore\" (KC) and a surrounding layer, from which a hierarchy of definitional\ndistances can be derived, all the way out to the periphery of the full\ndictionary. These definitional distances, too, are correlated with\npsycholinguistic variables (age of acquisition, concreteness, imageability,\noral and written frequency) and hence perhaps with the \"mental lexicon\" in each\nof our heads.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.0821v2",
            "date": "2009-12-04 11:33:54+00:00",
            "title": "Lexical evolution rates by automated stability measure",
            "authors": [
                "Filippo Petroni",
                "Maurizio Serva"
            ],
            "abstract": "Phylogenetic trees can be reconstructed from the matrix which contains the\ndistances between all pairs of languages in a family. Recently, we proposed a\nnew method which uses normalized Levenshtein distances among words with same\nmeaning and averages on all the items of a given list. Decisions about the\nnumber of items in the input lists for language comparison have been debated\nsince the beginning of glottochronology. The point is that words associated to\nsome of the meanings have a rapid lexical evolution. Therefore, a large\nvocabulary comparison is only apparently more accurate then a smaller one since\nmany of the words do not carry any useful information. In principle, one should\nfind the optimal length of the input lists studying the stability of the\ndifferent items. In this paper we tackle the problem with an automated\nmethodology only based on our normalized Levenshtein distance. With this\napproach, the program of an automated reconstruction of languages relationships\nis completed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.0884v3",
            "date": "2009-12-04 15:53:24+00:00",
            "title": "Measures of lexical distance between languages",
            "authors": [
                "Filippo Petroni",
                "Maurizio Serva"
            ],
            "abstract": "The idea of measuring distance between languages seems to have its roots in\nthe work of the French explorer Dumont D'Urville \\cite{Urv}. He collected\ncomparative words lists of various languages during his voyages aboard the\nAstrolabe from 1826 to 1829 and, in his work about the geographical division of\nthe Pacific, he proposed a method to measure the degree of relation among\nlanguages. The method used by modern glottochronology, developed by Morris\nSwadesh in the 1950s, measures distances from the percentage of shared\ncognates, which are words with a common historical origin. Recently, we\nproposed a new automated method which uses normalized Levenshtein distance\namong words with the same meaning and averages on the words contained in a\nlist. Recently another group of scholars \\cite{Bak, Hol} proposed a refined of\nour definition including a second normalization. In this paper we compare the\ninformation content of our definition with the refined version in order to\ndecide which of the two can be applied with greater success to resolve\nrelationships among languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.1829v1",
            "date": "2009-12-09 18:38:03+00:00",
            "title": "Document Searching System based on Natural Language Query Processing for Vietnam Open Courseware Library",
            "authors": [
                "Dang Tuan Nguyen",
                "Ha Quy-Tinh Luong"
            ],
            "abstract": "The necessary of buiding the searching system being able to support users\nexpressing their searching by natural language queries is very important and\nopens the researching direction with many potential. It combines the\ntraditional methods of information retrieval and the researching of Question\nAnswering (QA). In this paper, we introduce a searching system built by us for\nsearching courses on the Vietnam OpenCourseWare Program (VOCW). It can be\nconsidered as the first tool to be able to perform the user's Vietnamese\nquestions. The experiment results are rather good when we evaluate this system\non the precision",
            "journal ref": "D. T. NGUYEN and H. Q. LUONG, \"Document Searching System based on\n  Natural Language Query Processing for Vietnam Open Courseware Library\",\n  IJCSI, Volume 6, Issue 2, pp7-13, November 2009",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0912.3747v3",
            "date": "2009-12-18 17:34:45+00:00",
            "title": "A Survey of Paraphrasing and Textual Entailment Methods",
            "authors": [
                "Ion Androutsopoulos",
                "Prodromos Malakasiotis"
            ],
            "abstract": "Paraphrasing methods recognize, generate, or extract phrases, sentences, or\nlonger natural language expressions that convey almost the same information.\nTextual entailment methods, on the other hand, recognize, generate, or extract\npairs of natural language expressions, such that a human who reads (and trusts)\nthe first element of a pair would most likely infer that the other element is\nalso true. Paraphrasing can be seen as bidirectional textual entailment and\nmethods from the two areas are often similar. Both kinds of methods are useful,\nat least in principle, in a wide range of natural language processing\napplications, including question answering, summarization, text generation, and\nmachine translation. We summarize key ideas from the two areas by considering\nin turn recognition, generation, and extraction methods, also pointing to\nprominent articles and resources.",
            "journal ref": "I. Androutsopoulos and P. Malakasiotis, \"A Survey of Paraphrasing\n  and Textual Entailment Methods\". Journal of Artificial Intelligence Research,\n  38:135-187, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0912.3917v1",
            "date": "2009-12-19 18:29:43+00:00",
            "title": "Speech Recognition Oriented Vowel Classification Using Temporal Radial Basis Functions",
            "authors": [
                "Mustapha Guezouri",
                "Larbi Mesbahi",
                "Abdelkader Benyettou"
            ],
            "abstract": "The recent resurgence of interest in spatio-temporal neural network as speech\nrecognition tool motivates the present investigation. In this paper an approach\nwas developed based on temporal radial basis function \"TRBF\" looking to many\nadvantages: few parameters, speed convergence and time invariance. This\napplication aims to identify vowels taken from natural speech samples from the\nTimit corpus of American speech. We report a recognition accuracy of 98.06\npercent in training and 90.13 in test on a subset of 6 vowel phonemes, with the\npossibility to expend the vowel sets in future.",
            "journal ref": "Journal of Computing, Volume 1, Issue 1, pp 162-167, December 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.0716v1",
            "date": "2009-06-03 13:52:30+00:00",
            "title": "Size dependent word frequencies and translational invariance of books",
            "authors": [
                "Sebastian Bernhardsson",
                "Luis Enrique Correa da Rocha",
                "Petter Minnhagen"
            ],
            "abstract": "It is shown that a real novel shares many characteristic features with a null\nmodel in which the words are randomly distributed throughout the text. Such a\ncommon feature is a certain translational invariance of the text. Another is\nthat the functional form of the word-frequency distribution of a novel depends\non the length of the text in the same way as the null model. This means that an\napproximate power-law tail ascribed to the data will have an exponent which\nchanges with the size of the text-section which is analyzed. A further\nconsequence is that a novel cannot be described by text-evolution models like\nthe Simon model. The size-transformation of a novel is found to be well\ndescribed by a specific Random Book Transformation. This size transformation in\naddition enables a more precise determination of the functional form of the\nword-frequency distribution. The implications of the results are discussed.",
            "journal ref": "Physica A 389:2, pp. 330-341 (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.1467v1",
            "date": "2009-06-08 11:20:30+00:00",
            "title": "Syntax is from Mars while Semantics from Venus! Insights from Spectral Analysis of Distributional Similarity Networks",
            "authors": [
                "Chris Biemann",
                "Monojit Choudhury",
                "Animesh Mukherjee"
            ],
            "abstract": "We study the global topology of the syntactic and semantic distributional\nsimilarity networks for English through the technique of spectral analysis. We\nobserve that while the syntactic network has a hierarchical structure with\nstrong communities and their mixtures, the semantic network has several tightly\nknit communities along with a large core without any such well-defined\ncommunity structure.",
            "journal ref": "None",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/0906.2369v1",
            "date": "2009-06-12 16:34:56+00:00",
            "title": "Properties of quasi-alphabetic tree bimorphisms",
            "authors": [
                "Andreas Maletti",
                "Catalin Ionut Tirnauca"
            ],
            "abstract": "We study the class of quasi-alphabetic relations, i.e., tree transformations\ndefined by tree bimorphisms with two quasi-alphabetic tree homomorphisms and a\nregular tree language. We present a canonical representation of these\nrelations; as an immediate consequence, we get the closure under union. Also,\nwe show that they are not closed under intersection and complement, and do not\npreserve most common operations on trees (branches, subtrees, v-product,\nv-quotient, f-top-catenation). Moreover, we prove that the translations defined\nby quasi-alphabetic tree bimorphism are exactly products of context-free string\nlanguages. We conclude by presenting the connections between quasi-alphabetic\nrelations, alphabetic relations and classes of tree transformations defined by\nseveral types of top-down tree transducers. Furthermore, we get that\nquasi-alphabetic relations preserve the recognizable and algebraic tree\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0906.2835v1",
            "date": "2009-06-16 02:09:27+00:00",
            "title": "Employing Wikipedia's Natural Intelligence For Cross Language Information Retrieval",
            "authors": [
                "Mikhail Basilyan"
            ],
            "abstract": "In this paper we present a novel method for retrieving information in\nlanguages other than that of the query. We use this technique in combination\nwith existing traditional Cross Language Information Retrieval (CLIR)\ntechniques to improve their results. This method has a number of advantages\nover traditional techniques that rely on machine translation to translate the\nquery and then search the target document space using a machine translation.\nThis method is not limited to the availability of a machine translation\nalgorithm for the desired language and uses already existing sources of readily\navailable translated information on the internet as a \"middle-man\" approach. In\nthis paper we use Wikipedia; however, any similar multilingual, cross\nreferenced body of documents can be used. For evaluation and comparison\npurposes we also implemented a traditional machine translation approach\nseparately as well as the Wikipedia approach separately.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/0910.5410v1",
            "date": "2009-10-28 16:58:27+00:00",
            "title": "The Uned systems at Senseval-2",
            "authors": [
                "David Fernandez-Amoros",
                "Julio Gonzalo",
                "Felisa Verdejo"
            ],
            "abstract": "We have participated in the SENSEVAL-2 English tasks (all words and lexical\nsample) with an unsupervised system based on mutual information measured over a\nlarge corpus (277 million words) and some additional heuristics. A supervised\nextension of the system was also presented to the lexical sample task.\n  Our system scored first among unsupervised systems in both tasks: 56.9%\nrecall in all words, 40.2% in lexical sample. This is slightly worse than the\nfirst sense heuristic for all words and 3.6% better for the lexical sample, a\nstrong indication that unsupervised Word Sense Disambiguation remains being a\nstrong challenge.",
            "journal ref": "In Proceedings of the Second International Workshop on Evaluating\n  Word Sense Disambiguation Systems (SENSEVAL), Toulouse 2002",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0910.5419v1",
            "date": "2009-10-28 17:03:00+00:00",
            "title": "Word Sense Disambiguation Based on Mutual Information and Syntactic Patterns",
            "authors": [
                "David Fernandez-Amoros"
            ],
            "abstract": "This paper describes a hybrid system for WSD, presented to the English\nall-words and lexical-sample tasks, that relies on two different unsupervised\napproaches. The first one selects the senses according to mutual information\nproximity between a context word a variant of the sense. The second heuristic\nanalyzes the examples of use in the glosses of the senses so that simple\nsyntactic patterns are inferred. This patterns are matched against the\ndisambiguation contexts. We show that the first heuristic obtains a precision\nand recall of .58 and .35 respectively in the all words task while the second\nobtains .80 and .25. The high precision obtained recommends deeper research of\nthe techniques. Results for the lexical sample task are also provided.",
            "journal ref": "In proceedings of the Third International Workshop on Evaluating\n  Word Sense Disambiguation Systems (SENSEVAL) 2004, Barcelona",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0910.5682v1",
            "date": "2009-10-29 17:19:01+00:00",
            "title": "Word Sense Disambiguation Using English-Spanish Aligned Phrases over Comparable Corpora",
            "authors": [
                "David Fernandez-Amoros"
            ],
            "abstract": "In this paper we describe a WSD experiment based on bilingual English-Spanish\ncomparable corpora in which individual noun phrases have been identified and\naligned with their respective counterparts in the other language. The\nevaluation of the experiment has been carried out against SemCor.\n  We show that, with the alignment algorithm employed, potential precision is\nhigh (74.3%), however the coverage of the method is low (2.7%), due to\nalignments being far less frequent than we expected.\n  Contrary to our intuition, precision does not rise consistently with the\nnumber of alignments. The coverage is low due to several factors; there are\nimportant domain differences, and English and Spanish are too close languages\nfor this approach to be able to discriminate efficiently between senses,\nrendering it unsuitable for WSD, although the method may prove more productive\nin machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.2349v2",
            "date": "2009-01-15 21:12:12+00:00",
            "title": "Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words",
            "authors": [
                "Eduardo G. Altmann",
                "Janet B. Pierrehumbert",
                "Adilson E. Motter"
            ],
            "abstract": "Background: Zipf's discovery that word frequency distributions obey a power\nlaw established parallels between biological and physical processes, and\nlanguage, laying the groundwork for a complex systems perspective on human\ncommunication. More recent research has also identified scaling regularities in\nthe dynamics underlying the successive occurrences of events, suggesting the\npossibility of similar findings for language as well.\n  Methodology/Principal Findings: By considering frequent words in USENET\ndiscussion groups and in disparate databases where the language has different\nlevels of formality, here we show that the distributions of distances between\nsuccessive occurrences of the same word display bursty deviations from a\nPoisson process and are well characterized by a stretched exponential (Weibull)\nscaling. The extent of this deviation depends strongly on semantic type -- a\nmeasure of the logicality of each word -- and less strongly on frequency. We\ndevelop a generative model of this behavior that fully determines the dynamics\nof word usage.\n  Conclusions/Significance: Recurrence patterns of words are well described by\na stretched exponential distribution of recurrence times, an empirical scaling\nthat cannot be anticipated from Zipf's law. Because the use of words provides a\nuniquely precise and powerful lens on human thought and activity, our findings\nalso have implications for other overt manifestations of collective human\ndynamics.",
            "journal ref": "PLoS ONE 4 (11): e7678 (2009)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0901.4375v1",
            "date": "2009-01-27 23:59:59+00:00",
            "title": "Extracting Spooky-activation-at-a-distance from Considerations of Entanglement",
            "authors": [
                "P. D. Bruza",
                "K. Kitto",
                "D. Nelson",
                "C. McEvoy"
            ],
            "abstract": "Following an early claim by Nelson & McEvoy \\cite{Nelson:McEvoy:2007}\nsuggesting that word associations can display `spooky action at a distance\nbehaviour', a serious investigation of the potentially quantum nature of such\nassociations is currently underway. This paper presents a simple quantum model\nof a word association system. It is shown that a quantum model of word\nentanglement can recover aspects of both the Spreading Activation equation and\nthe Spooky-activation-at-a-distance equation, both of which are used to model\nthe activation level of words in human memory.",
            "journal ref": "None",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/0901.4784v1",
            "date": "2009-01-30 03:46:01+00:00",
            "title": "On the Entropy of Written Spanish",
            "authors": [
                "Fabio G. Guerrero"
            ],
            "abstract": "This paper reports on results on the entropy of the Spanish language. They\nare based on an analysis of natural language for n-word symbols (n = 1 to 18),\ntrigrams, digrams, and characters. The results obtained in this work are based\non the analysis of twelve different literary works in Spanish, as well as a\n279917 word news file provided by the Spanish press agency EFE. Entropy values\nare calculated by a direct method using computer processing and the probability\nlaw of large numbers. Three samples of artificial Spanish language produced by\na first-order model software source are also analyzed and compared with natural\nSpanish language.",
            "journal ref": "Revista Colombiana de Estadistica (RCE), Vol. 35, No. 3, Dec.\n  2012, pp 423-440",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0903.2792v3",
            "date": "2009-03-16 16:16:42+00:00",
            "title": "Thermodynamics of Information Retrieval",
            "authors": [
                "Kostadin Koroutchev",
                "Jian Shen",
                "Elka Koroutcheva",
                "Manuel Cebrian"
            ],
            "abstract": "In this work, we suggest a parameterized statistical model (the gamma\ndistribution) for the frequency of word occurrences in long strings of English\ntext and use this model to build a corresponding thermodynamic picture by\nconstructing the partition function. We then use our partition function to\ncompute thermodynamic quantities such as the free energy and the specific heat.\nIn this approach, the parameters of the word frequency model vary from word to\nword so that each word has a different corresponding thermodynamics and we\nsuggest that differences in the specific heat reflect differences in how the\nwords are used in language, differentiating keywords from common and function\nwords. Finally, we apply our thermodynamic picture to the problem of retrieval\nof texts based on keywords and suggest some advantages over traditional\ninformation retrieval methods.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/0904.1289v1",
            "date": "2009-04-08 08:57:43+00:00",
            "title": "Language Diversity across the Consonant Inventories: A Study in the Framework of Complex Networks",
            "authors": [
                "Monojit Choudhury",
                "Animesh Mukherjee",
                "Anupam Basu",
                "Niloy Ganguly",
                "Ashish Garg",
                "Vaibhav Jalan"
            ],
            "abstract": "n this paper, we attempt to explain the emergence of the linguistic diversity\nthat exists across the consonant inventories of some of the major language\nfamilies of the world through a complex network based growth model. There is\nonly a single parameter for this model that is meant to introduce a small\namount of randomness in the otherwise preferential attachment based growth\nprocess. The experiments with this model parameter indicates that the choice of\nconsonants among the languages within a family are far more preferential than\nit is across the families. The implications of this result are twofold -- (a)\nthere is an innate preference of the speakers towards acquiring certain\nlinguistic structures over others and (b) shared ancestry propels the stronger\npreferential connection between the languages within a family than across them.\nFurthermore, our observations indicate that this parameter might bear a\ncorrelation with the period of existence of the language families under\ninvestigation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.1558v3",
            "date": "2009-07-09 14:36:47+00:00",
            "title": "Towards the quantification of the semantic information encoded in written language",
            "authors": [
                "Marcelo A. Montemurro",
                "Damian Zanette"
            ],
            "abstract": "Written language is a complex communication signal capable of conveying\ninformation encoded in the form of ordered sequences of words. Beyond the local\norder ruled by grammar, semantic and thematic structures affect long-range\npatterns in word usage. Here, we show that a direct application of information\ntheory quantifies the relationship between the statistical distribution of\nwords and the semantic content of the text. We show that there is a\ncharacteristic scale, roughly around a few thousand words, which establishes\nthe typical size of the most informative segments in written language.\nMoreover, we find that the words whose contributions to the overall information\nis larger, are the ones more closely associated with the main subjects and\ntopics of the text. This scenario can be explained by a model of word usage\nthat assumes that words are distributed along the text in domains of a\ncharacteristic size where their frequency is higher than elsewhere. Our\nconclusions are based on the analysis of a large database of written language,\ndiverse in subjects and styles, and thus are likely to be applicable to general\nlanguage sequences encoding complex information.",
            "journal ref": "Advances in Complex Systems, Volume 13, Issue 2 (2010), pp.\n  135-153",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0907.1814v1",
            "date": "2009-07-10 13:24:55+00:00",
            "title": "Bayesian Query-Focused Summarization",
            "authors": [
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We present BayeSum (for ``Bayesian summarization''), a model for sentence\nextraction in query-focused summarization. BayeSum leverages the common case in\nwhich multiple documents are relevant to a single query. Using these documents\nas reinforcement for query terms, BayeSum is not afflicted by the paucity of\ninformation in short queries. We show that approximate inference in BayeSum is\npossible on large data sets and results in a state-of-the-art summarization\nsystem. Furthermore, we show how BayeSum can be understood as a justified query\nexpansion technique in the language modeling for IR framework.",
            "journal ref": "ACL 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0907.5083v1",
            "date": "2009-07-29 09:45:32+00:00",
            "title": "Serializing the Parallelism in Parallel Communicating Pushdown Automata Systems",
            "authors": [
                "M. Sakthi Balan"
            ],
            "abstract": "We consider parallel communicating pushdown automata systems (PCPA) and\ndefine a property called known communication for it. We use this property to\nprove that the power of a variant of PCPA, called returning centralized\nparallel communicating pushdown automata (RCPCPA), is equivalent to that of\nmulti-head pushdown automata. The above result presents a new sub-class of\nreturning parallel communicating pushdown automata systems (RPCPA) called\nsimple-RPCPA and we show that it can be written as a finite intersection of\nmulti-head pushdown automata systems.",
            "journal ref": "EPTCS 3, 2009, pp. 59-68",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/0909.4385v1",
            "date": "2009-09-24 08:50:49+00:00",
            "title": "The meta book and size-dependent properties of written language",
            "authors": [
                "Sebastian Bernhardsson",
                "Luis Enrique Correa da Rocha",
                "Petter Minnhagen"
            ],
            "abstract": "Evidence is given for a systematic text-length dependence of the power-law\nindex gamma of a single book. The estimated gamma values are consistent with a\nmonotonic decrease from 2 to 1 with increasing length of a text. A direct\nconnection to an extended Heap's law is explored. The infinite book limit is,\nas a consequence, proposed to be given by gamma = 1 instead of the value\ngamma=2 expected if the Zipf's law was ubiquitously applicable. In addition we\nexplore the idea that the systematic text-length dependence can be described by\na meta book concept, which is an abstract representation reflecting the\nword-frequency structure of a text. According to this concept the\nword-frequency distribution of a text, with a certain length written by a\nsingle author, has the same characteristics as a text of the same length pulled\nout from an imaginary complete infinite corpus written by the same author.",
            "journal ref": "New J. Phys. 11 (2009) 123015",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/0911.3280v7",
            "date": "2009-11-17 12:17:22+00:00",
            "title": "Automated languages phylogeny from Levenshtein distance",
            "authors": [
                "Maurizio Serva"
            ],
            "abstract": "Languages evolve over time in a process in which reproduction, mutation and\nextinction are all possible, similar to what happens to living organisms. Using\nthis similarity it is possible, in principle, to build family trees which show\nthe degree of relatedness between languages.\n  The method used by modern glottochronology, developed by Swadesh in the\n1950s, measures distances from the percentage of words with a common historical\norigin. The weak point of this method is that subjective judgment plays a\nrelevant role.\n  Recently we proposed an automated method that avoids the subjectivity, whose\nresults can be replicated by studies that use the same database and that\ndoesn't require a specific linguistic knowledge. Moreover, the method allows a\nquick comparison of a large number of languages.\n  We applied our method to the Indo-European and Austronesian families,\nconsidering in both cases, fifty different languages. The resulting trees are\nsimilar to those of previous studies, but with some important differences in\nthe position of few languages and subgroups. We believe that these differences\ncarry new information on the structure of the tree and on the phylogenetic\nrelationships within families.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.3292v2",
            "date": "2009-11-17 12:47:44+00:00",
            "title": "Automated words stability and languages phylogeny",
            "authors": [
                "Filippo Petroni",
                "Maurizio Serva"
            ],
            "abstract": "The idea of measuring distance between languages seems to have its roots in\nthe work of the French explorer Dumont D'Urville (D'Urville 1832). He collected\ncomparative words lists of various languages during his voyages aboard the\nAstrolabe from 1826 to1829 and, in his work about the geographical division of\nthe Pacific, he proposed a method to measure the degree of relation among\nlanguages. The method used by modern glottochronology, developed by Morris\nSwadesh in the 1950s (Swadesh 1952), measures distances from the percentage of\nshared cognates, which are words with a common historical origin. Recently, we\nproposed a new automated method which uses normalized Levenshtein distance\namong words with the same meaning and averages on the words contained in a\nlist. Another classical problem in glottochronology is the study of the\nstability of words corresponding to different meanings. Words, in fact, evolve\nbecause of lexical changes, borrowings and replacement at a rate which is not\nthe same for all of them. The speed of lexical evolution is different for\ndifferent meanings and it is probably related to the frequency of use of the\nassociated words (Pagel et al. 2007). This problem is tackled here by an\nautomated methodology only based on normalized Levenshtein distance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.3411v1",
            "date": "2009-11-17 22:03:56+00:00",
            "title": "Measuring the Meaning of Words in Contexts: An automated analysis of controversies about Monarch butterflies, Frankenfoods, and stem cells",
            "authors": [
                "Loet Leydesdorff",
                "Iina Hellsten"
            ],
            "abstract": "Co-words have been considered as carriers of meaning across different domains\nin studies of science, technology, and society. Words and co-words, however,\nobtain meaning in sentences, and sentences obtain meaning in their contexts of\nuse. At the science/society interface, words can be expected to have different\nmeanings: the codes of communication that provide meaning to words differ on\nthe varying sides of the interface. Furthermore, meanings and interfaces may\nchange over time. Given this structuring of meaning across interfaces and over\ntime, we distinguish between metaphors and diaphors as reflexive mechanisms\nthat facilitate the translation between contexts. Our empirical focus is on\nthree recent scientific controversies: Monarch butterflies, Frankenfoods, and\nstem-cell therapies. This study explores new avenues that relate the study of\nco-word analysis in context with the sociological quest for the analysis and\nprocessing of meaning.",
            "journal ref": "Scientometrics 67(2), 2006, 231-258",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0911.3944v1",
            "date": "2009-11-20 01:30:36+00:00",
            "title": "Likelihood-based semi-supervised model selection with applications to speech processing",
            "authors": [
                "Christopher M. White",
                "Sanjeev P. Khudanpur",
                "Patrick J. Wolfe"
            ],
            "abstract": "In conventional supervised pattern recognition tasks, model selection is\ntypically accomplished by minimizing the classification error rate on a set of\nso-called development data, subject to ground-truth labeling by human experts\nor some other means. In the context of speech processing systems and other\nlarge-scale practical applications, however, such labeled development data are\ntypically costly and difficult to obtain. This article proposes an alternative\nsemi-supervised framework for likelihood-based model selection that leverages\nunlabeled data by using trained classifiers representing each model to\nautomatically generate putative labels. The errors that result from this\nautomatic labeling are shown to be amenable to results from robust statistics,\nwhich in turn provide for minimax-optimal censored likelihood ratio tests that\nrecover the nonparametric sign test as a limiting case. This approach is then\nvalidated experimentally using a state-of-the-art automatic speech recognition\nsystem to select between candidate word pronunciations using unlabeled speech\ndata that only potentially contain instances of the words under test. Results\nprovide supporting evidence for the utility of this approach, and suggest that\nit may also find use in other applications of machine learning.",
            "journal ref": "IEEE Journal of Selected Topics in Signal Processing, vol. 4, pp.\n  1016-1026, 2010",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/0906.3741v1",
            "date": "2009-06-21 01:59:21+00:00",
            "title": "How opinions are received by online communities: A case study on Amazon.com helpfulness votes",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Gueorgi Kossinets",
                "Jon Kleinberg",
                "Lillian Lee"
            ],
            "abstract": "There are many on-line settings in which users publicly express opinions. A\nnumber of these offer mechanisms for other users to evaluate these opinions; a\ncanonical example is Amazon.com, where reviews come with annotations like \"26\nof 32 people found the following review helpful.\" Opinion evaluation appears in\nmany off-line settings as well, including market research and political\ncampaigns. Reasoning about the evaluation of an opinion is fundamentally\ndifferent from reasoning about the opinion itself: rather than asking, \"What\ndid Y think of X?\", we are asking, \"What did Z think of Y's opinion of X?\" Here\nwe develop a framework for analyzing and modeling opinion evaluation, using a\nlarge-scale collection of Amazon book reviews as a dataset. We find that the\nperceived helpfulness of a review depends not just on its content but also but\nalso in subtle ways on how the expressed evaluation relates to other\nevaluations of the same product. As part of our approach, we develop novel\nmethods that take advantage of the phenomenon of review \"plagiarism\" to control\nfor the effects of text in opinion evaluation, and we provide a simple and\nnatural mathematical model consistent with our findings. Our analysis also\nallows us to distinguish among the predictions of competing theories from\nsociology and social psychology, and to discover unexpected differences in the\ncollective opinion-evaluation behavior of user populations from different\ncountries.",
            "journal ref": "Proceedings of WWW, pp. 141--150, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/0905.1235v2",
            "date": "2009-05-08 14:42:03+00:00",
            "title": "The Modular Audio Recognition Framework (MARF) and its Applications: Scientific and Software Engineering Notes",
            "authors": [
                "Serguei A. Mokhov",
                "Stephen Sinclair",
                "Ian Cl\u00e9ment",
                "Dimitrios Nicolacopoulos"
            ],
            "abstract": "MARF is an open-source research platform and a collection of\nvoice/sound/speech/text and natural language processing (NLP) algorithms\nwritten in Java and arranged into a modular and extensible framework\nfacilitating addition of new algorithms. MARF can run distributively over the\nnetwork and may act as a library in applications or be used as a source for\nlearning and extension. A few example applications are provided to show how to\nuse the framework. There is an API reference in the Javadoc format as well as\nthis set of accompanying notes with the detailed description of the\narchitectural design, algorithms, and applications. MARF and its applications\nare released under a BSD-style license and is hosted at SourceForge.net. This\ndocument provides the details and the insight on the internals of MARF and some\nof the mentioned applications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/0905.3318v1",
            "date": "2009-05-20 14:12:42+00:00",
            "title": "An Object-Oriented and Fast Lexicon for Semantic Generation",
            "authors": [
                "Maarten Hijzelendoorn",
                "Crit Cremers"
            ],
            "abstract": "This paper is about the technical design of a large computational lexicon,\nits storage, and its access from a Prolog environment. Traditionally, efficient\naccess and storage of data structures is implemented by a relational database\nmanagement system. In Delilah, a lexicon-based NLP system, efficient access to\nthe lexicon by the semantic generator is vital. We show that our highly\ndetailed HPSG-style lexical specifications do not fit well in the Relational\nModel, and that they cannot be efficiently retrieved. We argue that they fit\nmore naturally in the Object-Oriented Model. Although storage of objects is\nredundant, we claim that efficient access is still possible by applying\nindexing, and compression techniques from the Relational Model to the\nObject-Oriented Model. We demonstrate that it is possible to implement\nobject-oriented storage and fast access in ISO Prolog.",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2010": [
        {
            "url": "http://arxiv.org/abs/1001.2267v1",
            "date": "2010-01-13 19:02:18+00:00",
            "title": "Speech Recognition by Machine, A Review",
            "authors": [
                "M. A. Anusuya",
                "S. K. Katti"
            ],
            "abstract": "This paper presents a brief survey on Automatic Speech Recognition and\ndiscusses the major themes and advances made in the past 60 years of research,\nso as to provide a technological perspective and an appreciation of the\nfundamental progress that has been accomplished in this important area of\nspeech communication. After years of research and development the accuracy of\nautomatic speech recognition remains one of the important research challenges\n(e.g., variations of the context, speakers, and environment).The design of\nSpeech Recognition system requires careful attentions to the following issues:\nDefinition of various types of speech classes, speech representation, feature\nextraction techniques, speech classifiers, database and performance evaluation.\nThe problems that are existing in ASR and the various techniques to solve these\nproblems constructed by various research workers have been presented in a\nchronological order. Hence authors hope that this work shall be a contribution\nin the area of speech recognition. The objective of this review paper is to\nsummarize and compare some of the well known methods used in various stages of\nspeech recognition system and identify research topic and applications which\nare at the forefront of this exciting and challenging field.",
            "journal ref": "International Journal of Computer Science and Information\n  Security, IJCSIS, Vol. 6, No. 3, pp. 181-205, December 2009, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1001.4273v1",
            "date": "2010-01-24 20:23:10+00:00",
            "title": "Sentence Simplification Aids Protein-Protein Interaction Extraction",
            "authors": [
                "Siddhartha Jonnalagadda",
                "Graciela Gonzalez"
            ],
            "abstract": "Accurate systems for extracting Protein-Protein Interactions (PPIs)\nautomatically from biomedical articles can help accelerate biomedical research.\nBiomedical Informatics researchers are collaborating to provide metaservices\nand advance the state-of-art in PPI extraction. One problem often neglected by\ncurrent Natural Language Processing systems is the characteristic complexity of\nthe sentences in biomedical literature. In this paper, we report on the impact\nthat automatic simplification of sentences has on the performance of a\nstate-of-art PPI extraction system, showing a substantial improvement in recall\n(8%) when the sentence simplification method is applied, without significant\nimpact to precision.",
            "journal ref": "The 3rd International Symposium on Languages in Biology and\n  Medicine, Jeju Island, South Korea, November 8-10, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1001.4277v1",
            "date": "2010-01-24 20:35:29+00:00",
            "title": "Towards Effective Sentence Simplification for Automatic Processing of Biomedical Text",
            "authors": [
                "Siddhartha Jonnalagadda",
                "Luis Tari",
                "Jorg Hakenberg",
                "Chitta Baral",
                "Graciela Gonzalez"
            ],
            "abstract": "The complexity of sentences characteristic to biomedical articles poses a\nchallenge to natural language parsers, which are typically trained on\nlarge-scale corpora of non-technical text. We propose a text simplification\nprocess, bioSimplify, that seeks to reduce the complexity of sentences in\nbiomedical abstracts in order to improve the performance of syntactic parsers\non the processed sentences. Syntactic parsing is typically one of the first\nsteps in a text mining pipeline. Thus, any improvement in performance would\nhave a ripple effect over all processing steps. We evaluated our method using a\ncorpus of biomedical sentences annotated with syntactic links. Our empirical\nresults show an improvement of 2.90% for the Charniak-McClosky parser and of\n4.23% for the Link Grammar parser when processing simplified sentences rather\nthan the original sentences in the corpus.",
            "journal ref": "Proc. of the NAACL-HLT 2009, Boulder, USA, June 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0478v1",
            "date": "2010-02-02 13:23:47+00:00",
            "title": "\u00c9tude et traitement automatique de l'anglais du XVIIe si\u00e8cle : outils morphosyntaxiques et dictionnaires",
            "authors": [
                "Odile Piton",
                "H\u00e9l\u00e8ne Pignot"
            ],
            "abstract": "In this article, we record the main linguistic differences or singularities\nof 17th century English, analyse them morphologically and syntactically and\npropose equivalent forms in contemporary English. We show how 17th century\ntexts may be transcribed into modern English, combining the use of electronic\ndictionaries with rules of transcription implemented as transducers. Apr\\`es\navoir expos\\'e la constitution du corpus, nous recensons les principales\ndiff\\'erences ou particularit\\'es linguistiques de la langue anglaise du XVIIe\nsi\\`ecle, les analysons du point de vue morphologique et syntaxique et\nproposons des \\'equivalents en anglais contemporain (AC). Nous montrons comment\nnous pouvons effectuer une transcription automatique de textes anglais du XVIIe\nsi\\`ecle en anglais moderne, en combinant l'utilisation de dictionnaires\n\\'electroniques avec des r\\`egles de transcriptions impl\\'ement\\'ees sous forme\nde transducteurs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0479v1",
            "date": "2010-02-02 13:24:20+00:00",
            "title": "\"Mind your p's and q's\": or the peregrinations of an apostrophe in 17th Century English",
            "authors": [
                "Odile Piton",
                "H\u00e9l\u00e8ne Pignot"
            ],
            "abstract": "If the use of the apostrophe in contemporary English often marks the Saxon\ngenitive, it may also indicate the omission of one or more let-ters. Some\nwriters (wrongly?) use it to mark the plural in symbols or abbreviations,\nvisual-ised thanks to the isolation of the morpheme \"s\". This punctuation mark\nwas imported from the Continent in the 16th century. During the 19th century\nits use was standardised. However the rules of its usage still seem problematic\nto many, including literate speakers of English. \"All too often, the apostrophe\nis misplaced\", or \"errant apostrophes are springing up every-where\" is a\ncomplaint that Internet users fre-quently come across when visiting grammar\nwebsites. Many of them detail its various uses and misuses, and attempt to\ncorrect the most common mistakes about it, especially its mis-use in the\nplural, called greengrocers' apostro-phes and humorously misspelled\n\"greengro-cers apostrophe's\". While studying English travel accounts published\nin the seventeenth century, we noticed that the different uses of this symbol\nmay accompany various models of metaplasms. We were able to highlight the\nlinguistic variations of some lexemes, and trace the origin of modern grammar\nrules gov-erning its usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0481v2",
            "date": "2010-02-02 13:24:38+00:00",
            "title": "Recognition and translation Arabic-French of Named Entities: case of the Sport places",
            "authors": [
                "Abdelmajid Ben Hamadou",
                "Odile Piton",
                "H\u00e9la Fehri"
            ],
            "abstract": "The recognition of Arabic Named Entities (NE) is a problem in different\ndomains of Natural Language Processing (NLP) like automatic translation.\nIndeed, NE translation allows the access to multilingual in-formation. This\ntranslation doesn't always lead to expected result especially when NE contains\na person name. For this reason and in order to ameliorate translation, we can\ntransliterate some part of NE. In this context, we propose a method that\nintegrates translation and transliteration together. We used the linguis-tic\nNooJ platform that is based on local grammars and transducers. In this paper,\nwe focus on sport domain. We will firstly suggest a refinement of the\ntypological model presented at the MUC Conferences we will describe the\nintegration of an Arabic transliteration module into translation system.\nFinally, we will detail our method and give the results of the evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0485v1",
            "date": "2010-02-02 13:35:02+00:00",
            "title": "Morphological study of Albanian words, and processing with NooJ",
            "authors": [
                "Odile Piton",
                "Klara Lagji"
            ],
            "abstract": "We are developing electronic dictionaries and transducers for the automatic\nprocessing of the Albanian Language. We will analyze the words inside a linear\nsegment of text. We will also study the relationship between units of sense and\nunits of form. The composition of words takes different forms in Albanian. We\nhave found that morphemes are frequently concatenated or simply juxtaposed or\ncontracted. The inflected grammar of NooJ allows constructing the dictionaries\nof flexed forms (declensions or conjugations). The diversity of word structures\nrequires tools to identify words created by simple concatenation, or to treat\ncontractions. The morphological tools of NooJ allow us to create grammatical\ntools to represent and treat these phenomena. But certain problems exceed the\nmorphological analysis and must be represented by syntactical grammars.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0773v1",
            "date": "2010-02-03 16:06:24+00:00",
            "title": "Approximations to the MMI criterion and their effect on lattice-based MMI",
            "authors": [
                "Steven Wegmann"
            ],
            "abstract": "Maximum mutual information (MMI) is a model selection criterion used for\nhidden Markov model (HMM) parameter estimation that was developed more than\ntwenty years ago as a discriminative alternative to the maximum likelihood\ncriterion for HMM-based speech recognition. It has been shown in the speech\nrecognition literature that parameter estimation using the current MMI\nparadigm, lattice-based MMI, consistently outperforms maximum likelihood\nestimation, but this is at the expense of undesirable convergence properties.\nIn particular, recognition performance is sensitive to the number of times that\nthe iterative MMI estimation algorithm, extended Baum-Welch, is performed. In\nfact, too many iterations of extended Baum-Welch will lead to degraded\nperformance, despite the fact that the MMI criterion improves at each\niteration. This phenomenon is at variance with the analogous behavior of\nmaximum likelihood estimation -- at least for the HMMs used in speech\nrecognition -- and it has previously been attributed to `over fitting'. In this\npaper, we present an analysis of lattice-based MMI that demonstrates, first of\nall, that the asymptotic behavior of lattice-based MMI is much worse than was\npreviously understood, i.e. it does not appear to converge at all, and, second\nof all, that this is not due to `over fitting'. Instead, we demonstrate that\nthe `over fitting' phenomenon is the result of standard methodology that\nexacerbates the poor behavior of two key approximations in the lattice-based\nMMI machinery. We also demonstrate that if we modify the standard methodology\nto improve the validity of these approximations, then the convergence\nproperties of lattice-based MMI become benign without sacrificing improvements\nto recognition accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.0904v1",
            "date": "2010-02-04 06:30:14+00:00",
            "title": "On Event Structure in the Torn Dress",
            "authors": [
                "Serguei A. Mokhov"
            ],
            "abstract": "Using Pustejovsky's \"The Syntax of Event Structure\" and Fong's \"On Mending a\nTorn Dress\" we give a glimpse of a Pustejovsky-like analysis to some example\nsentences in Fong. We attempt to give a framework for semantics to the noun\nphrases and adverbs as appropriate as well as the lexical entries for all words\nin the examples and critique both papers in light of our findings and\ndifficulties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.1095v1",
            "date": "2010-02-04 22:48:31+00:00",
            "title": "Towards a Heuristic Categorization of Prepositional Phrases in English with WordNet",
            "authors": [
                "Frank Rudzicz",
                "Serguei A. Mokhov"
            ],
            "abstract": "This document discusses an approach and its rudimentary realization towards\nautomatic classification of PPs; the topic, that has not received as much\nattention in NLP as NPs and VPs. The approach is a rule-based heuristics\noutlined in several levels of our research. There are 7 semantic categories of\nPPs considered in this document that we are able to classify from an annotated\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.1919v2",
            "date": "2010-02-09 20:01:06+00:00",
            "title": "Thai Rhetorical Structure Analysis",
            "authors": [
                "Somnuk Sinthupoun",
                "Ohm Sornil"
            ],
            "abstract": "Rhetorical structure analysis (RSA) explores discourse relations among\nelementary discourse units (EDUs) in a text. It is very useful in many text\nprocessing tasks employing relationships among EDUs such as text understanding,\nsummarization, and question-answering. Thai language with its distinctive\nlinguistic characteristics requires a unique technique. This article proposes\nan approach for Thai rhetorical structure analysis. First, EDUs are segmented\nby two hidden Markov models derived from syntactic rules. A rhetorical\nstructure tree is constructed from a clustering technique with its similarity\nmeasure derived from Thai semantic rules. Then, a decision tree whose features\nderived from the semantic rules is used to determine discourse relations.",
            "journal ref": "International Journal of Computer Science and Information\n  Security, IJCSIS, Vol. 7, No. 1, pp. 95-105, January 2010, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.3320v1",
            "date": "2010-02-17 17:30:22+00:00",
            "title": "Co-channel Interference Cancellation for Space-Time Coded OFDM Systems Using Adaptive Beamforming and Null Deepening",
            "authors": [
                "Raungrong Suleesathira"
            ],
            "abstract": "Combined with space-time coding, the orthogonal frequency division\nmultiplexing (OFDM) system explores space diversity. It is a potential scheme\nto offer spectral efficiency and robust high data rate transmissions over\nfrequency-selective fading channel. However, space-time coding impairs the\nsystem ability to suppress interferences as the signals transmitted from two\ntransmit antennas are superposed and interfered at the receiver antennas. In\nthis paper, we developed an adaptive beamforming based on least mean squared\nerror algorithm and null deepening to combat co-channel interference (CCI) for\nthe space-time coded OFDM (STC-OFDM) system. To illustrate the performance of\nthe presented approach, it is compared to the null steering beamformer which\nrequires a prior knowledge of directions of arrival (DOAs). The structure of\nspace-time decoders are preserved although there is the use of beamformers\nbefore decoding. By incorporating the proposed beamformer as a CCI canceller in\nthe STC-OFDM systems, the performance improvement is achieved as shown in the\nsimulation results.",
            "journal ref": "Raungrong Suleesathira, \"Co-channel Interference Cancellation for\n  Space-Time Coded OFDM Systems Using Adaptive Beamforming and Null Deepening\",\n  Journal of Telecommunications, Volume 1, Issue 1, pp6-13, February 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.4820v1",
            "date": "2010-02-25 16:27:36+00:00",
            "title": "SLAM : Solutions lexicales automatique pour m\u00e9taphores",
            "authors": [
                "Yann Desalle",
                "Bruno Gaume",
                "Karine Duvignau"
            ],
            "abstract": "This article presents SLAM, an Automatic Solver for Lexical Metaphors like\n?d\\'eshabiller* une pomme? (to undress* an apple). SLAM calculates a\nconventional solution for these productions. To carry on it, SLAM has to\nintersect the paradigmatic axis of the metaphorical verb ?d\\'eshabiller*?,\nwhere ?peler? (?to peel?) comes closer, with a syntagmatic axis that comes from\na corpus where ?peler une pomme? (to peel an apple) is semantically and\nsyntactically regular. We test this model on DicoSyn, which is a ?small world?\nnetwork of synonyms, to compute the paradigmatic axis and on Frantext.20, a\nFrench corpus, to compute the syntagmatic axis. Further, we evaluate the model\nwith a sample of an experimental corpus of the database of Flexsem",
            "journal ref": "Traitement Automatique des Langues 50, 1 (2009) 145--175",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.0206v1",
            "date": "2010-02-28 19:00:12+00:00",
            "title": "Why has (reasonably accurate) Automatic Speech Recognition been so hard to achieve?",
            "authors": [
                "Steven Wegmann",
                "Larry Gillick"
            ],
            "abstract": "Hidden Markov models (HMMs) have been successfully applied to automatic\nspeech recognition for more than 35 years in spite of the fact that a key HMM\nassumption -- the statistical independence of frames -- is obviously violated\nby speech data. In fact, this data/model mismatch has inspired many attempts to\nmodify or replace HMMs with alternative models that are better able to take\ninto account the statistical dependence of frames. However it is fair to say\nthat in 2010 the HMM is the consensus model of choice for speech recognition\nand that HMMs are at the heart of both commercially available products and\ncontemporary research systems. In this paper we present a preliminary\nexploration aimed at understanding how speech data depart from HMMs and what\neffect this departure has on the accuracy of HMM-based speech recognition. Our\nanalysis uses standard diagnostic tools from the field of statistics --\nhypothesis testing, simulation and resampling -- which are rarely used in the\nfield of speech recognition. Our main result, obtained by novel manipulations\nof real and resampled data, demonstrates that real data have statistical\ndependency and that this dependency is responsible for significant numbers of\nrecognition errors. We also demonstrate, using simulation and resampling, that\nif we `remove' the statistical dependency from data, then the resulting\nrecognition error rates become negligible. Taken together, these results\nsuggest that a better understanding of the structure of the statistical\ndependency in speech data is a crucial first step towards improving HMM-based\nspeech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.0337v1",
            "date": "2010-03-01 18:04:39+00:00",
            "title": "Change of word types to word tokens ratio in the course of translation (based on Russian translations of K. Vonnegut novels)",
            "authors": [
                "Andrey Kutuzov"
            ],
            "abstract": "The article provides lexical statistical analysis of K. Vonnegut's two novels\nand their Russian translations. It is found out that there happen some changes\nbetween the speed of word types and word tokens ratio change in the source and\ntarget texts. The author hypothesizes that these changes are typical for\nEnglish-Russian translations, and moreover, they represent an example of\nBaker's translation feature of levelling out.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.0628v1",
            "date": "2010-03-02 16:52:32+00:00",
            "title": "Linguistic Geometries for Unsupervised Dimensionality Reduction",
            "authors": [
                "Yi Mao",
                "Krishnakumar Balasubramanian",
                "Guy Lebanon"
            ],
            "abstract": "Text documents are complex high dimensional objects. To effectively visualize\nsuch data it is important to reduce its dimensionality and visualize the low\ndimensional embedding as a 2-D or 3-D scatter plot. In this paper we explore\ndimensionality reduction methods that draw upon domain knowledge in order to\nachieve a better low dimensional embedding and visualization of documents. We\nconsider the use of geometries specified manually by an expert, geometries\nderived automatically from corpus statistics, and geometries computed from\nlinguistic resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.1399v1",
            "date": "2010-03-06 16:34:40+00:00",
            "title": "Automatic derivation of domain terms and concept location based on the analysis of the identifiers",
            "authors": [
                "Peter Vaclavik",
                "Jaroslav Poruban",
                "Marek Mezei"
            ],
            "abstract": "Developers express the meaning of the domain ideas in specifically selected\nidentifiers and comments that form the target implemented code. Software\nmaintenance requires knowledge and understanding of the encoded ideas. This\npaper presents a way how to create automatically domain vocabulary. Knowledge\nof domain vocabulary supports the comprehension of a specific domain for later\ncode maintenance or evolution. We present experiments conducted in two selected\ndomains: application servers and web frameworks. Knowledge of domain terms\nenables easy localization of chunks of code that belong to a certain term. We\nconsider these chunks of code as \"concepts\" and their placement in the code as\n\"concept location\". Application developers may also benefit from the obtained\ndomain terms. These terms are parts of speech that characterize a certain\nconcept. Concepts are encoded in \"classes\" (OO paradigm) and the obtained\nvocabulary of terms supports the selection and the comprehension of the class'\nappropriate identifiers. We measured the following software products with our\ntool: JBoss, JOnAS, GlassFish, Tapestry, Google Web Toolkit and Echo2.",
            "journal ref": "Acta Univ. Sapientiae, Informatica, 2,1 (2010) 40-50",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.1455v1",
            "date": "2010-03-07 11:28:08+00:00",
            "title": "A Computational Algorithm based on Empirical Analysis, that Composes Sanskrit Poetry",
            "authors": [
                "Rama N.",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "Poetry-writing in Sanskrit is riddled with problems for even those who know\nthe language well. This is so because the rules that govern Sanskrit prosody\nare numerous and stringent. We propose a computational algorithm that converts\nprose given as E-text into poetry in accordance with the metrical rules of\nSanskrit prosody, simultaneously taking care to ensure that sandhi or euphonic\nconjunction, which is compulsory in verse, is handled. The algorithm is\nconsiderably speeded up by a novel method of reducing the target search\ndatabase. The algorithm further gives suggestions to the poet in case what\nhe/she has given as the input prose is impossible to fit into any allowed\nmetrical format. There is also an interactive component of the algorithm by\nwhich the algorithm interacts with the poet to resolve ambiguities. In\naddition, this unique work, which provides a solution to a problem that has\nnever been addressed before, provides a simple yet effective speech recognition\ninterface that would help the visually impaired dictate words in E-text, which\nis in turn versified by our Poetry Composer Engine.",
            "journal ref": "International Journal of Computer Science and Information\n  Security, IJCSIS, Vol. 7, No. 2, pp. 056-062, February 2010, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.4149v1",
            "date": "2010-03-22 13:09:57+00:00",
            "title": "Les Entit\u00e9s Nomm\u00e9es : usage et degr\u00e9s de pr\u00e9cision et de d\u00e9sambigu\u00efsation",
            "authors": [
                "Claude Martineau",
                "Elsa Tolone",
                "Stavroula Voyatzi"
            ],
            "abstract": "The recognition and classification of Named Entities (NER) are regarded as an\nimportant component for many Natural Language Processing (NLP) applications.\nThe classification is usually made by taking into account the immediate context\nin which the NE appears. In some cases, this immediate context does not allow\ngetting the right classification. We show in this paper that the use of an\nextended syntactic context and large-scale resources could be very useful in\nthe NER task.",
            "journal ref": "26\\`eme Colloque international sur le Lexique et la Grammaire\n  (LGC'07), Bonifacio : France (2007)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.4894v1",
            "date": "2010-03-25 14:03:51+00:00",
            "title": "La repr\u00e9sentation formelle des concepts spatiaux dans la langue",
            "authors": [
                "Michel Aurnague",
                "Laure Vieu",
                "Andr\u00e9e Borillo"
            ],
            "abstract": "In this chapter, we assume that systematically studying spatial markers\nsemantics in language provides a means to reveal fundamental properties and\nconcepts characterizing conceptual representations of space. We propose a\nformal system accounting for the properties highlighted by the linguistic\nanalysis, and we use these tools for representing the semantic content of\nseveral spatial relations of French. The first part presents a semantic\nanalysis of the expression of space in French aiming at describing the\nconstraints that formal representations have to take into account. In the\nsecond part, after presenting the structure of our formal system, we set out\nits components. A commonsense geometry is sketched out and several functional\nand pragmatic spatial concepts are formalized. We take a special attention in\nshowing that these concepts are well suited to representing the semantic\ncontent of several prepositions of French ('sur' (on), 'dans' (in), 'devant'\n(in front of), 'au-dessus' (above)), and in illustrating the inferential\nadequacy of these representations.",
            "journal ref": "Langage et cognition spatiale, Michel Denis (Ed.) (1997) 69-102",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.4898v1",
            "date": "2010-03-25 14:08:44+00:00",
            "title": "Les entit\u00e9s spatiales dans la langue : \u00e9tude descriptive, formelle et exp\u00e9rimentale de la cat\u00e9gorisation",
            "authors": [
                "Michel Aurnague",
                "Maya Hickmann",
                "Laure Vieu"
            ],
            "abstract": "While previous linguistic and psycholinguistic research on space has mainly\nanalyzed spatial relations, the studies reported in this paper focus on how\nlanguage distinguishes among spatial entities. Descriptive and experimental\nstudies first propose a classification of entities, which accounts for both\nstatic and dynamic space, has some cross-linguistic validity, and underlies\nadults' cognitive processing. Formal and computational analyses then introduce\ntheoretical elements aiming at modelling these categories, while fulfilling\nvarious properties of formal ontologies (generality, parsimony, coherence...).\nThis formal framework accounts, in particular, for functional dependences among\nentities underlying some part-whole descriptions. Finally, developmental\nresearch shows that language-specific properties have a clear impact on how\nchildren talk about space. The results suggest some cross-linguistic\nvariability in children's spatial representations from an early age onwards,\nbringing into question models in which general cognitive capacities are the\nonly determinants of spatial cognition during the course of development.",
            "journal ref": "Agir dans l'espace, Catherine Thinus-Blanc & Jean Bullier (Ed.)\n  (2005) 217-232",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.5372v1",
            "date": "2010-03-28 15:17:22+00:00",
            "title": "Learning Recursive Segments for Discourse Parsing",
            "authors": [
                "Stergos Afantenos",
                "Pascal Denis",
                "Philippe Muller",
                "Laurence Danlos"
            ],
            "abstract": "Automatically detecting discourse segments is an important preliminary step\ntowards full discourse parsing. Previous research on discourse segmentation\nhave relied on the assumption that elementary discourse units (EDUs) in a\ndocument always form a linear sequence (i.e., they can never be nested).\nUnfortunately, this assumption turns out to be too strong, for some theories of\ndiscourse like SDRT allows for nested discourse units. In this paper, we\npresent a simple approach to discourse segmentation that is able to produce\nnested EDUs. Our approach builds on standard multi-class classification\ntechniques combined with a simple repairing heuristic that enforces global\ncoherence. Our system was developed and evaluated on the first round of\nannotations provided by the French Annodis project (an ongoing effort to create\na discourse bank for French). Cross-validated on only 47 documents (1,445\nEDUs), our system achieves encouraging performance results with an F-score of\n73% for finding EDUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.3902v1",
            "date": "2010-05-21 08:12:12+00:00",
            "title": "Morphonette: a morphological network of French",
            "authors": [
                "Nabil Hathout"
            ],
            "abstract": "This paper describes in details the first version of Morphonette, a new\nFrench morphological resource and a new radically lexeme-based method of\nmorphological analysis. This research is grounded in a paradigmatic conception\nof derivational morphology where the morphological structure is a structure of\nthe entire lexicon and not one of the individual words it contains. The\ndiscovery of this structure relies on a measure of morphological similarity\nbetween words, on formal analogy and on the properties of two morphological\nparadigms:",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.4697v3",
            "date": "2010-05-25 20:36:09+00:00",
            "title": "The Lambek-Grishin calculus is NP-complete",
            "authors": [
                "Jeroen Bransen"
            ],
            "abstract": "The Lambek-Grishin calculus LG is the symmetric extension of the\nnon-associative Lambek calculus NL. In this paper we prove that the\nderivability problem for LG is NP-complete.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.5466v1",
            "date": "2010-05-29 16:37:02+00:00",
            "title": "Quantitative parametrization of texts written by Ivan Franko: An attempt of the project",
            "authors": [
                "Solomiya Buk"
            ],
            "abstract": "In the article, the project of quantitative parametrization of all texts by\nIvan Franko is manifested. It can be made only by using modern computer\ntechniques after the frequency dictionaries for all Franko's works are\ncompiled. The paper describes the application spheres, methodology, stages,\nprinciples and peculiarities in the compilation of the frequency dictionary of\nthe second half of the 19th century - the beginning of the 20th century. The\nrelation between the Ivan Franko frequency dictionary, explanatory dictionary\nof writer's language and text corpus is discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.5596v1",
            "date": "2010-05-31 06:37:40+00:00",
            "title": "A generic tool to generate a lexicon for NLP from Lexicon-Grammar tables",
            "authors": [
                "Matthieu Constant",
                "Elsa Tolone"
            ],
            "abstract": "Lexicon-Grammar tables constitute a large-coverage syntactic lexicon but they\ncannot be directly used in Natural Language Processing (NLP) applications\nbecause they sometimes rely on implicit information. In this paper, we\nintroduce LGExtract, a generic tool for generating a syntactic lexicon for NLP\nfrom the Lexicon-Grammar tables. It is based on a global table that contains\nundefined information and on a unique extraction script including all\noperations to be performed for all tables. We also present an experiment that\nhas been conducted to generate a new lexicon of French verbs and predicative\nnouns.",
            "journal ref": "Actes du 27e Colloque international sur le lexique et la grammaire\n  (L'Aquila, 10-13 septembre 2008). Seconde partie, Michele De Gioia (Ed.)\n  (2010) pages 79-93",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.0153v1",
            "date": "2010-06-01 15:20:59+00:00",
            "title": "Ivan Franko's novel Dlja domashnjoho ohnyshcha (For the Hearth) in the light of the frequency dictionary",
            "authors": [
                "Solomiya Buk"
            ],
            "abstract": "In the article, the methodology and the principles of the compilation of the\nFrequency dictionary for Ivan Franko's novel Dlja domashnjoho ohnyshcha (For\nthe Hearth) are described. The following statistical parameters of the novel\nvocabulary are obtained: variety, exclusiveness, concentration indexes,\ncorrelation between word rank and text coverage, etc. The main quantitative\ncharacteristics of Franko's novels Perekhresni stezhky (The Cross-Paths) and\nDlja domashnjoho ohnyshcha are compared on the basis of their frequency\ndictionaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.2809v1",
            "date": "2010-06-14 19:20:31+00:00",
            "title": "Offline Arabic Handwriting Recognition Using Artificial Neural Network",
            "authors": [
                "A. A Zaidan",
                "B. B Zaidan",
                "Hamid. A. Jalab",
                "Hamdan. O. Alanazi",
                "Rami Alnaqeib"
            ],
            "abstract": "The ambition of a character recognition system is to transform a text\ndocument typed on paper into a digital format that can be manipulated by word\nprocessor software Unlike other languages, Arabic has unique features, while\nother language doesn't have, from this language these are seven or eight\nlanguage such as ordo, jewie and Persian writing, Arabic has twenty eight\nletters, each of which can be linked in three different ways or separated\ndepending on the case. The difficulty of the Arabic handwriting recognition is\nthat, the accuracy of the character recognition which affects on the accuracy\nof the word recognition, in additional there is also two or three from for each\ncharacter, the suggested solution by using artificial neural network can solve\nthe problem and overcome the difficulty of Arabic handwriting recognition.",
            "journal ref": "Journal of Computer Science and Engineering, Volume 1, Issue 1,\n  p55-58, May 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.2835v1",
            "date": "2010-06-14 20:07:32+00:00",
            "title": "Fuzzy Modeling and Natural Language Processing for Panini's Sanskrit Grammar",
            "authors": [
                "P. Venkata Subba Reddy"
            ],
            "abstract": "Indian languages have long history in World Natural languages. Panini was the\nfirst to define Grammar for Sanskrit language with about 4000 rules in fifth\ncentury. These rules contain uncertainty information. It is not possible to\nComputer processing of Sanskrit language with uncertain information. In this\npaper, fuzzy logic and fuzzy reasoning are proposed to deal to eliminate\nuncertain information for reasoning with Sanskrit grammar. The Sanskrit\nlanguage processing is also discussed in this paper.",
            "journal ref": "Journal of Computer Science and Engineering, Volume 1, Issue 1,\n  p99-101, May 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.3787v7",
            "date": "2010-06-18 19:54:29+00:00",
            "title": "Complete Complementary Results Report of the MARF's NLP Approach to the DEFT 2010 Competition",
            "authors": [
                "Serguei A. Mokhov"
            ],
            "abstract": "This companion paper complements the main DEFT'10 article describing the MARF\napproach (arXiv:0905.1235) to the DEFT'10 NLP challenge (described at\nhttp://www.groupes.polymtl.ca/taln2010/deft.php in French). This paper is aimed\nto present the complete result sets of all the conducted experiments and their\nsettings in the resulting tables highlighting the approach and the best\nresults, but also showing the worse and the worst and their subsequent\nanalysis. This particular work focuses on application of the MARF's classical\nand NLP pipelines to identification tasks within various francophone corpora to\nidentify decades when certain articles were published for the first track\n(Piste 1) and place of origin of a publication (Piste 2), such as the journal\nand location (France vs. Quebec). This is the sixth iteration of the release of\nthe results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.5880v1",
            "date": "2010-06-30 15:10:33+00:00",
            "title": "Testing SDRT's Right Frontier",
            "authors": [
                "Stergos Afantenos",
                "Nicholas Asher"
            ],
            "abstract": "The Right Frontier Constraint (RFC), as a constraint on the attachment of new\nconstituents to an existing discourse structure, has important implications for\nthe interpretation of anaphoric elements in discourse and for Machine Learning\n(ML) approaches to learning discourse structures. In this paper we provide\nstrong empirical support for SDRT's version of RFC. The analysis of about 100\ndoubly annotated documents by five different naive annotators shows that SDRT's\nRFC is respected about 95% of the time. The qualitative analysis of presumed\nviolations that we have performed shows that they are either click-errors or\nstructural misconceptions.",
            "journal ref": "Proceedings of COLING 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1008.0170v1",
            "date": "2010-08-01 12:45:58+00:00",
            "title": "Symmetric categorial grammar: residuation and Galois connections",
            "authors": [
                "Michael Moortgat"
            ],
            "abstract": "The Lambek-Grishin calculus is a symmetric extension of the Lambek calculus:\nin addition to the residuated family of product, left and right division\noperations of Lambek's original calculus, one also considers a family of\ncoproduct, right and left difference operations, related to the former by an\narrow-reversing duality. Communication between the two families is implemented\nin terms of linear distributivity principles. The aim of this paper is to\ncomplement the symmetry between (dual) residuated type-forming operations with\nan orthogonal opposition that contrasts residuated and Galois connected\noperations. Whereas the (dual) residuated operations are monotone, the Galois\nconnected operations (and their duals) are antitone. We discuss the algebraic\nproperties of the (dual) Galois connected operations, and generalize the\n(co)product distributivity principles to include the negative operations. We\ngive a continuation-passing-style translation for the new type-forming\noperations, and discuss some linguistic applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1008.1986v1",
            "date": "2010-08-11 20:01:59+00:00",
            "title": "For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia",
            "authors": [
                "Mark Yatskar",
                "Bo Pang",
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee"
            ],
            "abstract": "We report on work in progress on extracting lexical simplifications (e.g.,\n\"collaborate\" -> \"work together\"), focusing on utilizing edit histories in\nSimple English Wikipedia for this task. We consider two main approaches: (1)\nderiving simplification probabilities via an edit model that accounts for a\nmixture of different operations, and (2) using metadata to focus on edits that\nare more likely to be simplification operations. We find our methods to\noutperform a reasonable baseline and yield many high-quality lexical\nsimplifications not included in an independently-created manually prepared\nlist.",
            "journal ref": "Proceedings of the NAACL, pp. 365-368, 2010. Short paper",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1008.3169v2",
            "date": "2010-08-18 20:08:22+00:00",
            "title": "Don't 'have a clue'? Unsupervised co-learning of downward-entailing operators",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee"
            ],
            "abstract": "Researchers in textual entailment have begun to consider inferences involving\n'downward-entailing operators', an interesting and important class of lexical\nitems that change the way inferences are made. Recent work proposed a method\nfor learning English downward-entailing operators that requires access to a\nhigh-quality collection of 'negative polarity items' (NPIs). However, English\nis one of the very few languages for which such a list exists. We propose the\nfirst approach that can be applied to the many languages for which there is no\npre-existing high-precision database of NPIs. As a case study, we apply our\nmethod to Romanian and show that our method yields good results. Also, we\nperform a cross-linguistic analysis that suggests interesting connections to\nsome findings in linguistic typology.",
            "journal ref": "Proceedings of the ACL Short Papers, pp. 247-252, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1009.1117v2",
            "date": "2010-09-06 18:35:08+00:00",
            "title": "Constructions d\u00e9finitoires des tables du Lexique-Grammaire",
            "authors": [
                "Elsa Tolone",
                "Stavroula Voyatzi",
                "Christian Lecl\u00e8re"
            ],
            "abstract": "Lexicon-Grammar tables are a very rich syntactic lexicon for the French\nlanguage. This linguistic database is nevertheless not directly suitable for\nuse by computer programs, as it is incomplete and lacks consistency. Tables are\ndefined on the basis of features which are not explicitly recorded in the\nlexicon. These features are only described in literature. Our aim is to define\nfor each tables these essential properties to make them usable in various\nNatural Language Processing (NLP) applications, such as parsing.",
            "journal ref": "29\\`eme Colloque international sur le Lexique et la Grammaire\n  (LGC'10), Belgrade : Serbie (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1009.3238v1",
            "date": "2010-09-16 18:19:57+00:00",
            "title": "Tableaux for the Lambek-Grishin calculus",
            "authors": [
                "Arno Bastenhof"
            ],
            "abstract": "Categorial type logics, pioneered by Lambek, seek a proof-theoretic\nunderstanding of natural language syntax by identifying categories with\nformulas and derivations with proofs. We typically observe an intuitionistic\nbias: a structural configuration of hypotheses (a constituent) derives a single\nconclusion (the category assigned to it). Acting upon suggestions of Grishin to\ndualize the logical vocabulary, Moortgat proposed the Lambek-Grishin calculus\n(LG) with the aim of restoring symmetry between hypotheses and conclusions. We\ndevelop a theory of labeled modal tableaux for LG, inspired by the\ninterpretation of its connectives as binary modal operators in the relational\nsemantics of Kurtonina and Moortgat. As a linguistic application of our method,\nwe show that grammars based on LG are context-free through use of an\ninterpolation lemma. This result complements that of Melissen, who proved that\nLG augmented by mixed associativity and -commutativity was exceeds LTAG in\nexpressive power.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.0519v1",
            "date": "2010-11-02 06:37:31+00:00",
            "title": "Stabilizing knowledge through standards - A perspective for the humanities",
            "authors": [
                "Laurent Romary"
            ],
            "abstract": "It is usual to consider that standards generate mixed feelings among\nscientists. They are often seen as not really reflecting the state of the art\nin a given domain and a hindrance to scientific creativity. Still, scientists\nshould theoretically be at the best place to bring their expertise into\nstandard developments, being even more neutral on issues that may typically be\nrelated to competing industrial interests. Even if it could be thought of as\neven more complex to think about developping standards in the humanities, we\nwill show how this can be made feasible through the experience gained both\nwithin the Text Encoding Initiative consortium and the International\nOrganisation for Standardisation. By taking the specific case of lexical\nresources, we will try to show how this brings about new ideas for designing\nfuture research infrastructures in the human and social sciences.",
            "journal ref": "Going Digital: Evolutionary and Revolutionary Aspects of\n  Digitization, Karl Grandin (Ed.) (2011)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.0835v1",
            "date": "2010-11-03 10:05:15+00:00",
            "title": "A PDTB-Styled End-to-End Discourse Parser",
            "authors": [
                "Ziheng Lin",
                "Hwee Tou Ng",
                "Min-Yen Kan"
            ],
            "abstract": "We have developed a full discourse parser in the Penn Discourse Treebank\n(PDTB) style. Our trained parser first identifies all discourse and\nnon-discourse relations, locates and labels their arguments, and then\nclassifies their relation types. When appropriate, the attribution spans to\nthese relations are also determined. We present a comprehensive evaluation from\nboth component-wise and error-cascading perspectives.",
            "journal ref": "Natural Language Engineering 20 (02), 151 - 184, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.2922v1",
            "date": "2010-11-12 14:43:04+00:00",
            "title": "Emoticonsciousness",
            "authors": [
                "Carl Vogel",
                "Jerom Janssen"
            ],
            "abstract": "A temporal analysis of emoticon use in Swedish, Italian, German and English\nasynchronous electronic communication is reported. Emoticons are classified as\npositive, negative and neutral. Postings to newsgroups over a 66 week period\nare considered. The aggregate analysis of emoticon use in newsgroups for\nscience and politics tend on the whole to be consistent over the entire time\nperiod. Where possible, events that coincide with divergences from trends in\nlanguage-subject pairs are noted. Political discourse in Italian over the\nperiod shows marked use of negative emoticons, and in Swedish, positive\nemoticons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.4155v1",
            "date": "2010-11-18 08:59:55+00:00",
            "title": "Motifs de graphe pour le calcul de d\u00e9pendances syntaxiques compl\u00e8tes",
            "authors": [
                "Jonathan Marchand",
                "Bruno Guillaume",
                "Guy Perrier"
            ],
            "abstract": "This article describes a method to build syntactical dependencies starting\nfrom the phrase structure parsing process. The goal is to obtain all the\ninformation needed for a detailled semantical analysis. Interaction Grammars\nare used for parsing; the saturation of polarities which is the core of this\nformalism can be mapped to dependency relation. Formally, graph patterns are\nused to express the set of constraints which control dependency creations.",
            "journal ref": "Conf\\'erence sur le Traitement Automatique des Langues Naturelles\n  - TALN'10, Montr\\'eal : Canada (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.4623v1",
            "date": "2010-11-21 00:15:27+00:00",
            "title": "Opinion Polarity Identification through Adjectives",
            "authors": [
                "Samaneh Moghaddam",
                "Fred Popowich"
            ],
            "abstract": "\"What other people think\" has always been an important piece of information\nduring various decision-making processes. Today people frequently make their\nopinions available via the Internet, and as a result, the Web has become an\nexcellent source for gathering consumer opinions. There are now numerous Web\nresources containing such opinions, e.g., product reviews forums, discussion\ngroups, and Blogs. But, due to the large amount of information and the wide\nrange of sources, it is essentially impossible for a customer to read all of\nthe reviews and make an informed decision on whether to purchase the product.\nIt is also difficult for the manufacturer or seller of a product to accurately\nmonitor customer opinions. For this reason, mining customer reviews, or opinion\nmining, has become an important issue for research in Web information\nextraction. One of the important topics in this research area is the\nidentification of opinion polarity. The opinion polarity of a review is usually\nexpressed with values 'positive', 'negative' or 'neutral'. We propose a\ntechnique for identifying polarity of reviews by identifying the polarity of\nthe adjectives that appear in them. Our evaluation shows the technique can\nprovide accuracy in the area of 73%, which is well above the 58%-64% provided\nby naive Bayesian classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.5188v2",
            "date": "2010-11-23 18:20:40+00:00",
            "title": "La r\u00e9duction de termes complexes dans les langues de sp\u00e9cialit\u00e9",
            "authors": [
                "Yannis Haralambous",
                "Elisa Lavagnino"
            ],
            "abstract": "Our study applies statistical methods to French and Italian corpora to\nexamine the phenomenon of multi-word term reduction in specialty languages.\nThere are two kinds of reduction: anaphoric and lexical. We show that anaphoric\nreduction depends on the discourse type (vulgarization, pedagogical,\nspecialized) but is independent of both domain and language; that lexical\nreduction depends on domain and is more frequent in technical, rapidly evolving\ndomains; and that anaphoric reductions tend to follow full terms rather than\nprecede them. We define the notion of the anaphoric tree of the term and study\nits properties. Concerning lexical reduction, we attempt to prove statistically\nthat there is a notion of term lifecycle, where the full form is progressively\nreplaced by a lexical reduction. ----- Nous \\'etudions par des m\\'ethodes\nstatistiques sur des corpus fran\\c{c}ais et italiens, le ph\\'enom\\`ene de\nr\\'eduction des termes complexes dans les langues de sp\\'ecialit\\'e. Il existe\ndeux types de r\\'eductions : anaphorique et lexicale. Nous montrons que la\nr\\'eduction anaphorique d\\'epend du type de discours (de vulgarisation,\np\\'edagogique, sp\\'ecialis\\'e) mais ne d\\'epend ni du domaine, ni de la langue,\nalors que la r\\'eduction lexicale d\\'epend du domaine et est plus fr\\'equente\ndans les domaines techniques \\`a \\'evolution rapide. D'autre part, nous\nmontrons que la r\\'eduction anaphorique a tendance \\`a suivre la forme pleine\ndu terme, nous d\\'efinissons une notion d'arbre anaphorique de terme et nous\n\\'etudions ses propri\\'et\\'es. Concernant la r\\'eduction lexicale, nous tentons\nde d\\'emontrer statistiquement qu'il existe une notion de cycle de vie de\nterme, o\\`u la forme pleine est progressivement remplac\\'ee par une r\\'eduction\nlexicale.",
            "journal ref": "Traitement Automatique de Langues 2011 (52/1)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1004.4181v1",
            "date": "2010-04-23 17:00:18+00:00",
            "title": "Displacement Calculus",
            "authors": [
                "Glyn Morrill",
                "Oriol Valent\u00edn"
            ],
            "abstract": "The Lambek calculus provides a foundation for categorial grammar in the form\nof a logic of concatenation. But natural language is characterized by\ndependencies which may also be discontinuous. In this paper we introduce the\ndisplacement calculus, a generalization of Lambek calculus, which preserves its\ngood proof-theoretic properties while embracing discontinuiity and subsuming\nit. We illustrate linguistic applications and prove Cut-elimination, the\nsubformula property, and decidability",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1010.1826v1",
            "date": "2010-10-09 10:09:18+00:00",
            "title": "A probabilistic top-down parser for minimalist grammars",
            "authors": [
                "Thomas Mainguy"
            ],
            "abstract": "This paper describes a probabilistic top-down parser for minimalist grammars.\nTop-down parsers have the great advantage of having a certain predictive power\nduring the parsing, which takes place in a left-to-right reading of the\nsentence. Such parsers have already been well-implemented and studied in the\ncase of Context-Free Grammars, which are already top-down, but these are\ndifficult to adapt to Minimalist Grammars, which generate sentences bottom-up.\nI propose here a way of rewriting Minimalist Grammars as Linear Context-Free\nRewriting Systems, allowing to easily create a top-down parser. This rewriting\nallows also to put a probabilistic field on these grammars, which can be used\nto accelerate the parser. Finally, I propose a method of refining the\nprobabilistic field by using algorithms used in data compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1010.2384v1",
            "date": "2010-10-12 13:20:30+00:00",
            "title": "Learning Taxonomy for Text Segmentation by Formal Concept Analysis",
            "authors": [
                "Mihaiela Lupea",
                "Doina Tatar",
                "Zsuzsana Marian"
            ],
            "abstract": "In this paper the problems of deriving a taxonomy from a text and\nconcept-oriented text segmentation are approached. Formal Concept Analysis\n(FCA) method is applied to solve both of these linguistic problems. The\nproposed segmentation method offers a conceptual view for text segmentation,\nusing a context-driven clustering of sentences. The Concept-oriented Clustering\nSegmentation algorithm (COCS) is based on k-means linear clustering of the\nsentences. Experimental results obtained using COCS algorithm are presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1012.5962v2",
            "date": "2010-12-29 15:40:09+00:00",
            "title": "Annotated English",
            "authors": [
                "Jose Hernandez-Orallo"
            ],
            "abstract": "This document presents Annotated English, a system of diacritical symbols\nwhich turns English pronunciation into a precise and unambiguous process. The\nannotations are defined and located in such a way that the original English\ntext is not altered (not even a letter), thus allowing for a consistent reading\nand learning of the English language with and without annotations. The\nannotations are based on a set of general rules that make the frequency of\nannotations not dramatically high. This makes the reader easily associate\nannotations with exceptions, and makes it possible to shape, internalise and\nconsolidate some rules for the English language which otherwise are weakened by\nthe enormous amount of exceptions in English pronunciation. The advantages of\nthis annotation system are manifold. Any existing text can be annotated without\na significant increase in size. This means that we can get an annotated version\nof any document or book with the same number of pages and fontsize. Since no\nletter is affected, the text can be perfectly read by a person who does not\nknow the annotation rules, since annotations can be simply ignored. The\nannotations are based on a set of rules which can be progressively learned and\nrecognised, even in cases where the reader has no access or time to read the\nrules. This means that a reader can understand most of the annotations after\nreading a few pages of Annotated English, and can take advantage from that\nknowledge for any other annotated document she may read in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1001.2263v1",
            "date": "2010-01-13 18:51:08+00:00",
            "title": "Syllable Analysis to Build a Dictation System in Telugu language",
            "authors": [
                "N. Kalyani",
                "Dr K. V. N. Sunitha"
            ],
            "abstract": "In recent decades, Speech interactive systems gained increasing importance.\nTo develop Dictation System like Dragon for Indian languages it is most\nimportant to adapt the system to a speaker with minimum training. In this paper\nwe focus on the importance of creating speech database at syllable units and\nidentifying minimum text to be considered while training any speech recognition\nsystem. There are systems developed for continuous speech recognition in\nEnglish and in few Indian languages like Hindi and Tamil. This paper gives the\nstatistical details of syllables in Telugu and its use in minimizing the search\nspace during recognition of speech. The minimum words that cover maximum\nsyllables are identified. This words list can be used for preparing a small\ntext which can be used for collecting speech sample while training the\ndictation system. The results are plotted for frequency of syllables and the\nnumber of syllables in each word. This approach is applied on the CIIL Mysore\ntext corpus which is of 3 million words.",
            "journal ref": "International Journal of Computer Science and Information\n  Security, IJCSIS, Vol. 6, No. 3, pp. 171-176, December 2009, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1001.4368v1",
            "date": "2010-01-25 19:44:03+00:00",
            "title": "Implicit media frames: Automated analysis of public debate on artificial sweeteners",
            "authors": [
                "Iina Hellsten",
                "James Dawson",
                "Loet Leydesdorff"
            ],
            "abstract": "The framing of issues in the mass media plays a crucial role in the public\nunderstanding of science and technology. This article contributes to research\nconcerned with diachronic analysis of media frames by making an analytical\ndistinction between implicit and explicit media frames, and by introducing an\nautomated method for analysing diachronic changes of implicit frames. In\nparticular, we apply a semantic maps method to a case study on the newspaper\ndebate about artificial sweeteners, published in The New York Times (NYT)\nbetween 1980 and 2006. Our results show that the analysis of semantic changes\nenables us to filter out the dynamics of implicit frames, and to detect\nemerging metaphors in public debates. Theoretically, we discuss the relation\nbetween implicit frames in public debates and codification of information in\nscientific discourses, and suggest further avenues for research interested in\nthe automated analysis of frame changes and trends in public debates.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1002.2034v1",
            "date": "2010-02-10 07:45:57+00:00",
            "title": "Dire n'est pas concevoir",
            "authors": [
                "Christophe Roche"
            ],
            "abstract": "The conceptual modelling built from text is rarely an ontology. As a matter\nof fact, such a conceptualization is corpus-dependent and does not offer the\nmain properties we expect from ontology. Furthermore, ontology extracted from\ntext in general does not match ontology defined by expert using a formal\nlanguage. It is not surprising since ontology is an extra-linguistic\nconceptualization whereas knowledge extracted from text is the concern of\ntextual linguistics. Incompleteness of text and using rhetorical figures, like\nellipsis, modify the perception of the conceptualization we may have.\nOntological knowledge, which is necessary for text understanding, is not in\ngeneral embedded into documents.",
            "journal ref": "Ing\\'enierie des Connaissances, Grenoble : France (2007)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1003.4065v1",
            "date": "2010-03-22 05:20:31+00:00",
            "title": "Plagiarism Detection using ROUGE and WordNet",
            "authors": [
                "Chien-Ying Chen",
                "Jen-Yuan Yeh",
                "Hao-Ren Ke"
            ],
            "abstract": "With the arrival of digital era and Internet, the lack of information control\nprovides an incentive for people to freely use any content available to them.\nPlagiarism occurs when users fail to credit the original owner for the content\nreferred to, and such behavior leads to violation of intellectual property. Two\nmain approaches to plagiarism detection are fingerprinting and term occurrence;\nhowever, one common weakness shared by both approaches, especially\nfingerprinting, is the incapability to detect modified text plagiarism. This\nstudy proposes adoption of ROUGE and WordNet to plagiarism detection. The\nformer includes ngram co-occurrence statistics, skip-bigram, and longest common\nsubsequence (LCS), while the latter acts as a thesaurus and provides semantic\ninformation. N-gram co-occurrence statistics can detect verbatim copy and\ncertain sentence modification, skip-bigram and LCS are immune from text\nmodification such as simple addition or deletion of words, and WordNet may\nhandle the problem of word substitution.",
            "journal ref": "Journal of Computing, Volume 2, Issue 3, March 2010",
            "category": "cs.OH"
        },
        {
            "url": "http://arxiv.org/abs/1003.5749v1",
            "date": "2010-03-30 07:04:46+00:00",
            "title": "Etiqueter un corpus oral par apprentissage automatique \u00e0 l'aide de connaissances linguistiques",
            "authors": [
                "Iris Eshkol",
                "Isabelle Tellier",
                "Taalab Samer",
                "Sylvie Billot"
            ],
            "abstract": "Thanks to the Eslo1 (\"Enqu\\^ete sociolinguistique d'Orl\\'eans\", i.e.\n\"Sociolinguistic Inquiery of Orl\\'eans\") campain, a large oral corpus has been\ngathered and transcribed in a textual format. The purpose of the work presented\nhere is to associate a morpho-syntactic label to each unit of this corpus. To\nthis aim, we have first studied the specificities of the necessary labels, and\ntheir various possible levels of description. This study has led to a new\noriginal hierarchical structuration of labels. Then, considering that our new\nset of labels was different from the one used in every available software, and\nthat these softwares usually do not fit for oral data, we have built a new\nlabeling tool by a Machine Learning approach, from data labeled by Cordial and\ncorrected by hand. We have applied linear CRF (Conditional Random Fields)\ntrying to take the best possible advantage of the linguistic knowledge that was\nused to define the set of labels. We obtain an accuracy between 85 and 90%,\ndepending of the parameters used.",
            "journal ref": "10\\`emes Journ\\'ees Internationales d'Analyse statistique des\n  Donn\\'ees Textuelles JADT'2010, Rome : Italie (2010)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1006.1343v1",
            "date": "2010-06-07 19:36:18+00:00",
            "title": "Segmentation and Nodal Points in Narrative: Study of Multiple Variations of a Ballad",
            "authors": [
                "Fionn Murtagh",
                "Adam Ganz"
            ],
            "abstract": "The Lady Maisry ballads afford us a framework within which to segment a\nstoryline into its major components. Segments and as a consequence nodal points\nare discussed for nine different variants of the Lady Maisry story of a (young)\nwoman being burnt to death by her family, on account of her becoming pregnant\nby a foreign personage. We motivate the importance of nodal points in textual\nand literary analysis. We show too how the openings of the nine variants can be\nanalyzed comparatively, and also the conclusions of the ballads.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.1786v1",
            "date": "2010-06-09 12:24:54+00:00",
            "title": "Measuring Meaning on the World-Wide Web",
            "authors": [
                "Diederik Aerts"
            ],
            "abstract": "We introduce the notion of the 'meaning bound' of a word with respect to\nanother word by making use of the World-Wide Web as a conceptual environment\nfor meaning. The meaning of a word with respect to another word is established\nby multiplying the product of the number of webpages containing both words by\nthe total number of webpages of the World-Wide Web, and dividing the result by\nthe product of the number of webpages for each of the single words. We\ncalculate the meaning bounds for several words and analyze different aspects of\nthese by looking at specific examples.",
            "journal ref": "In D. Aerts, J. Broekaert, B. D'Hooghe and N. Note (Eds.),\n  Worldviews, Science and Us: Bridging Knowledge and Its Implications for Our\n  Perspectives of the World. Singapore: World Scientific (2011)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1006.1930v1",
            "date": "2010-06-10 00:39:33+00:00",
            "title": "The Pet-Fish problem on the World-Wide Web",
            "authors": [
                "Diederik Aerts",
                "Marek Czachor",
                "Bart D'Hooghe",
                "Sandro Sozzo"
            ],
            "abstract": "We identify the presence of Pet-Fish problem situations and the corresponding\nGuppy effect of concept theory on the World-Wide Web. For this purpose, we\nintroduce absolute weights for words expressing concepts and relative weights\nbetween words expressing concepts, and the notion of 'meaning bound' between\ntwo words expressing concepts, making explicit use of the conceptual structure\nof the World-Wide Web. The Pet-Fish problem occurs whenever there are exemplars\n- in the case of Pet and Fish these can be Guppy or Goldfish - for which the\nmeaning bound with respect to the conjunction is stronger than the meaning\nbounds with respect to the individual concepts.",
            "journal ref": "Proceedings of the AAAI Fall Symposium (FS-10-08), Quantum\n  Informatics for Cognitive, Social, and Semantic Processes, pp. 17-21, (2010)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1006.5827v1",
            "date": "2010-06-30 10:28:56+00:00",
            "title": "Approximate Robotic Mapping from sonar data by modeling Perceptions with Antonyms",
            "authors": [
                "Sergio Guadarrama",
                "Antonio Ruiz-Mayor"
            ],
            "abstract": "This work, inspired by the idea of \"Computing with Words and Perceptions\"\nproposed by Zadeh in 2001, focuses on how to transform measurements into\nperceptions for the problem of map building by Autonomous Mobile Robots. We\npropose to model the perceptions obtained from sonar-sensors as two grid maps:\none for obstacles and another for empty spaces. The rules used to build and\nintegrate these maps are expressed by linguistic descriptions and modeled by\nfuzzy rules. The main difference of this approach from other studies reported\nin the literature is that the method presented here is based on the hypothesis\nthat the concepts \"occupied\" and \"empty\" are antonyms rather than complementary\n(as it happens in probabilistic approaches), or independent (as it happens in\nthe previous fuzzy models).\n  Controlled experimentation with a real robot in three representative indoor\nenvironments has been performed and the results presented. We offer a\nqualitative and quantitative comparison of the estimated maps obtained by the\nprobabilistic approach, the previous fuzzy method and the new antonyms-based\nfuzzy approach. It is shown that the maps obtained with the antonyms-based\napproach are better defined, capture better the shape of the walls and of the\nempty-spaces, and contain less errors due to rebounds and short-echoes.\nFurthermore, in spite of noise and low resolution inherent to the sonar-sensors\nused, the maps obtained are accurate and tolerant to imprecision.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1007.0936v1",
            "date": "2010-07-06 16:09:43+00:00",
            "title": "Linguistic complexity: English vs. Polish, text vs. corpus",
            "authors": [
                "Jaroslaw Kwapien",
                "Stanislaw Drozdz",
                "Adam Orczyk"
            ],
            "abstract": "We analyze the rank-frequency distributions of words in selected English and\nPolish texts. We show that for the lemmatized (basic) word forms the\nscale-invariant regime breaks after about two decades, while it might be\nconsistent for the whole range of ranks for the inflected word forms. We also\nfind that for a corpus consisting of texts written by different authors the\nbasic scale-invariant regime is broken more strongly than in the case of\ncomparable corpus consisting of texts written by the same author. Similarly,\nfor a corpus consisting of texts translated into Polish from other languages\nthe scale-invariant regime is broken more strongly than for a comparable corpus\nof native Polish texts. Moreover, we find that if the words are tagged with\ntheir proper part of speech, only verbs show rank-frequency distribution that\nis almost scale-invariant.",
            "journal ref": "Acta Phys. Pol. A 117, 716-720 (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1007.1025v1",
            "date": "2010-07-07 00:37:44+00:00",
            "title": "Inflection system of a language as a complex network",
            "authors": [
                "Henryk Fuk\u015b"
            ],
            "abstract": "We investigate inflection structure of a synthetic language using Latin as an\nexample. We construct a bipartite graph in which one group of vertices\ncorrespond to dictionary headwords and the other group to inflected forms\nencountered in a given text. Each inflected form is connected to its\ncorresponding headword, which in some cases in non-unique. The resulting sparse\ngraph decomposes into a large number of connected components, to be called word\ngroups. We then show how the concept of the word group can be used to construct\ncoverage curves of selected Latin texts. We also investigate a version of the\ninflection graph in which all theoretically possible inflected forms are\nincluded. Distribution of sizes of connected components of this graphs\nresembles cluster distribution in a lattice percolation near the critical\npoint.",
            "journal ref": "Proceedings of 2009 IEEE Toronto International Conference -\n  Science and Technology for Humanity, IEEE, Toronto 2009, pp. 491-496",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1007.4748v1",
            "date": "2010-07-27 15:16:36+00:00",
            "title": "Detecting influenza outbreaks by analyzing Twitter messages",
            "authors": [
                "Aron Culotta"
            ],
            "abstract": "We analyze over 500 million Twitter messages from an eight month period and\nfind that tracking a small number of flu-related keywords allows us to forecast\nfuture influenza rates with high accuracy, obtaining a 95% correlation with\nnational health statistics. We then analyze the robustness of this approach to\nspurious keyword matches, and we propose a document classification component to\nfilter these misleading messages. We find that this document classifier can\nreduce error rates by over half in simulated false alarm experiments, though\nmore research is needed to develop methods that are robust in cases of\nextremely high noise.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1008.0706v1",
            "date": "2010-08-04 06:00:21+00:00",
            "title": "Algorithmic Detection of Computer Generated Text",
            "authors": [
                "Allen Lavoie",
                "Mukkai Krishnamoorthy"
            ],
            "abstract": "Computer generated academic papers have been used to expose a lack of\nthorough human review at several computer science conferences. We assess the\nproblem of classifying such documents. After identifying and evaluating several\nquantifiable features of academic papers, we apply methods from machine\nlearning to build a binary classifier. In tests with two hundred papers, the\nresulting classifier correctly labeled papers either as human written or as\ncomputer generated with no false classifications of computer generated papers\nas human and a 2% false classification rate for human papers as computer\ngenerated. We believe generalizations of these features are applicable to\nsimilar classification problems. While most current text-based spam detection\ntechniques focus on the keyword-based classification of email messages, a new\ngeneration of unsolicited computer-generated advertisements masquerade as\nlegitimate postings in online groups, message boards and social news sites. Our\nresults show that taking the formatting and contextual clues offered by these\nenvironments into account may be of central importance when selecting features\nwith which to identify such unwanted postings.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1008.1673v2",
            "date": "2010-08-10 09:46:21+00:00",
            "title": "Space and the Synchronic A-Ram",
            "authors": [
                "Alex V Berka"
            ],
            "abstract": "Space is a circuit oriented, spatial programming language designed to exploit\nthe massive parallelism available in a novel formal model of computation called\nthe Synchronic A-Ram, and physically related FPGA and reconfigurable\narchitectures. Space expresses variable grained MIMD parallelism, is modular,\nstrictly typed, and deterministic. Barring operations associated with memory\nallocation and compilation, modules cannot access global variables, and are\nreferentially transparent. At a high level of abstraction, modules exhibit a\nsmall, sequential state transition system, aiding verification. Space deals\nwith communication, scheduling, and resource contention issues in parallel\ncomputing, by resolving them explicitly in an incremental manner, module by\nmodule, whilst ascending the ladder of abstraction. Whilst the Synchronic A-Ram\nmodel was inspired by linguistic considerations, it is also put forward as a\nformal model for reconfigurable digital circuits. A programming environment has\nbeen developed, that incorporates a simulator and compiler that transform Space\nprograms into Synchronic A-Ram machine code, consisting of only three bit-level\ninstructions, and a marking instruction. Space and the Synchronic A-Ram point\nto novel routes out of the parallel computing crisis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1008.5287v1",
            "date": "2010-08-31 11:37:32+00:00",
            "title": "Lexical Co-occurrence, Statistical Significance, and Word Association",
            "authors": [
                "Dipak Chaudhari",
                "Om P. Damani",
                "Srivatsan Laxman"
            ],
            "abstract": "Lexical co-occurrence is an important cue for detecting word associations. We\npresent a theoretical framework for discovering statistically significant\nlexical co-occurrences from a given corpus. In contrast with the prevalent\npractice of giving weightage to unigram frequencies, we focus only on the\ndocuments containing both the terms (of a candidate bigram). We detect biases\nin span distributions of associated words, while being agnostic to variations\nin global unigram frequencies. Our framework has the fidelity to distinguish\ndifferent classes of lexical co-occurrences, based on strengths of the document\nand corpuslevel cues of co-occurrence in the data. We perform extensive\nexperiments on benchmark data sets to study the performance of various\nco-occurrence measures that are currently known in literature. We find that a\nrelatively obscure measure called Ochiai, and a newly introduced measure CSA\ncapture the notion of lexical co-occurrence best, followed next by LLR, Dice,\nand TTest, while another popular measure, PMI, suprisingly, performs poorly in\nthe context of lexical co-occurrence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.2575v1",
            "date": "2010-11-11 06:46:10+00:00",
            "title": "Complex sequencing rules of birdsong can be explained by simple hidden Markov processes",
            "authors": [
                "Kentaro Katahira",
                "Kenta Suzuki",
                "Kazuo Okanoya",
                "Masato Okada"
            ],
            "abstract": "Complex sequencing rules observed in birdsongs provide an opportunity to\ninvestigate the neural mechanism for generating complex sequential behaviors.\nTo relate the findings from studying birdsongs to other sequential behaviors,\nit is crucial to characterize the statistical properties of the sequencing\nrules in birdsongs. However, the properties of the sequencing rules in\nbirdsongs have not yet been fully addressed. In this study, we investigate the\nstatistical propertiesof the complex birdsong of the Bengalese finch (Lonchura\nstriata var. domestica). Based on manual-annotated syllable sequences, we first\nshow that there are significant higher-order context dependencies in Bengalese\nfinch songs, that is, which syllable appears next depends on more than one\nprevious syllable. This property is shared with other complex sequential\nbehaviors. We then analyze acoustic features of the song and show that\nhigher-order context dependencies can be explained using first-order hidden\nstate transition dynamics with redundant hidden states. This model corresponds\nto hidden Markov models (HMMs), well known statistical models with a large\nrange of application for time series modeling. The song annotation with these\nmodels with first-order hidden state dynamics agreed well with manual\nannotation, the score was comparable to that of a second-order HMM, and\nsurpassed the zeroth-order model (the Gaussian mixture model (GMM)), which does\nnot use context information. Our results imply that the hierarchical\nrepresentation with hidden state dynamics may underlie the neural\nimplementation for generating complex sequences with higher-order dependencies.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1011.3258v1",
            "date": "2010-11-14 21:57:47+00:00",
            "title": "Integration of Agile Ontology Mapping towards NLP Search in I-SOAS",
            "authors": [
                "Zeeshan Ahmed",
                "Ina Tacheva"
            ],
            "abstract": "In this research paper we address the importance of Product Data Management\n(PDM) with respect to its contributions in industry. Moreover we also present\nsome currently available major challenges to PDM communities and targeting some\nof these challenges we present an approach i.e. I-SOAS, and briefly discuss how\nthis approach can be helpful in solving the PDM community's faced problems.\nFurthermore, limiting the scope of this research to one challenge, we focus on\nthe implementation of a semantic based search mechanism in PDM Systems. Going\ninto the details, at first we describe the respective field i.e. Language\nTechnology (LT), contributing towards natural language processing, to take\nadvantage in implementing a search engine capable of understanding the semantic\nout of natural language based search queries. Then we discuss how can we\npractically take advantage of LT by implementing its concepts in the form of\nsoftware application with the use of semantic web technology i.e. Ontology.\nLater, in the end of this research paper, we briefly present a prototype\napplication developed with the use of concepts of LT towards semantic based\nsearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1011.5076v1",
            "date": "2010-11-23 12:49:31+00:00",
            "title": "Application of a Quantum Ensemble Model to Linguistic Analysis",
            "authors": [
                "Andrij Rovenchak",
                "Solomija Buk"
            ],
            "abstract": "A new set of parameters to describe the word frequency behavior of texts is\nproposed. The analogy between the word frequency distribution and the\nBose-distribution is suggested and the notion of \"temperature\" is introduced\nfor this case. The calculations are made for English, Ukrainian, and the\nGuinean Maninka languages. The correlation between in-deep language structure\n(the level of analyticity) and the defined parameters is shown to exist.",
            "journal ref": "Physica A, Volume 390, Issue 7, Pages 1326-1331 (2011)",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1011.5209v2",
            "date": "2010-11-23 20:11:35+00:00",
            "title": "The semantic mapping of words and co-words in contexts",
            "authors": [
                "Loet Leydesdorff",
                "Kasper Welbers"
            ],
            "abstract": "Meaning can be generated when information is related at a systemic level.\nSuch a system can be an observer, but also a discourse, for example,\noperationalized as a set of documents. The measurement of semantics as\nsimilarity in patterns (correlations) and latent variables (factor analysis)\nhas been enhanced by computer techniques and the use of statistics; for\nexample, in \"Latent Semantic Analysis\". This communication provides an\nintroduction, an example, pointers to relevant software, and summarizes the\nchoices that can be made by the analyst. Visualization (\"semantic mapping\") is\nthus made more accessible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1004.4848v1",
            "date": "2010-04-27 16:13:19+00:00",
            "title": "Punctuation effects in English and Esperanto texts",
            "authors": [
                "M. Ausloos"
            ],
            "abstract": "A statistical physics study of punctuation effects on sentence lengths is\npresented for written texts: {\\it Alice in wonderland} and {\\it Through a\nlooking glass}. The translation of the first text into esperanto is also\nconsidered as a test for the role of punctuation in defining a style, and for\ncontrasting natural and artificial, but written, languages. Several log-log\nplots of the sentence length-rank relationship are presented for the major\npunctuation marks. Different power laws are observed with characteristic\nexponents. The exponent can take a value much less than unity ($ca.$ 0.50 or\n0.30) depending on how a sentence is defined. The texts are also mapped into\ntime series based on the word frequencies. The quantitative differences between\nthe original and translated texts are very minutes, at the exponent level. It\nis argued that sentences seem to be more reliable than word distributions in\ndiscussing an author style.",
            "journal ref": "Physica A389 (2010) 2835-2840",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1012.2042v1",
            "date": "2010-12-09 16:15:34+00:00",
            "title": "MUDOS-NG: Multi-document Summaries Using N-gram Graphs (Tech Report)",
            "authors": [
                "George Giannakopoulos",
                "George Vouros",
                "Vangelis Karkaletsis"
            ],
            "abstract": "This report describes the MUDOS-NG summarization system, which applies a set\nof language-independent and generic methods for generating extractive\nsummaries. The proposed methods are mostly combinations of simple operators on\na generic character n-gram graph representation of texts. This work defines the\nset of used operators upon n-gram graphs and proposes using these operators\nwithin the multi-document summarization process in such subtasks as document\nanalysis, salient sentence selection, query expansion and redundancy control.\nFurthermore, a novel chunking methodology is used, together with a novel way to\nassign concepts to sentences for query expansion. The experimental results of\nthe summarization system, performed upon widely used corpora from the Document\nUnderstanding and the Text Analysis Conferences, are promising and provide\nevidence for the potential of the generic methods introduced. This work aims to\ndesignate core methods exploiting the n-gram graph representation, providing\nthe basis for more advanced summarization systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1012.2661v1",
            "date": "2010-12-13 08:55:43+00:00",
            "title": "Categorial Minimalist Grammar",
            "authors": [
                "Maxime Amblard",
                "Alain Lecomte",
                "Christian Retor\u00e9"
            ],
            "abstract": "We first recall some basic notions on minimalist grammars and on categorial\ngrammars. Next we shortly introduce partially commutative linear logic, and our\nrepresentation of minimalist grammars within this categorial system, the\nso-called categorial minimalist grammars. Thereafter we briefly present\n\\lambda\\mu-DRT (Discourse Representation Theory) an extension of \\lambda-DRT\n(compositional DRT) in the framework of \\lambda\\mu calculus: it avoids type\nraising and derives different readings from a single semantic representation,\nin a setting which follows discourse structure. We run a complete example which\nillustrates the various structures and rules that are needed to derive a\nsemantic representation from the categorial view of a transformational\nsyntactic analysis.",
            "journal ref": "Linguistic Analysis 36, 1--4 (2010) 273--308",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1002.4665v1",
            "date": "2010-02-25 00:00:47+00:00",
            "title": "Syntactic Topic Models",
            "authors": [
                "Jordan Boyd-Graber",
                "David M. Blei"
            ],
            "abstract": "The syntactic topic model (STM) is a Bayesian nonparametric model of language\nthat discovers latent distributions of words (topics) that are both\nsemantically and syntactically coherent. The STM models dependency parsed\ncorpora where sentences are grouped into documents. It assumes that each word\nis drawn from a latent topic chosen by combining document-level features and\nthe local syntactic context. Each document has a distribution over latent\ntopics, as in topic models, which provides the semantic consistency. Each\nelement in the dependency parse tree also has a distribution over the topics of\nits children, as in latent-state syntax models, which provides the syntactic\nconsistency. These distributions are convolved so that the topic of each word\nis likely under both its document and syntactic context. We derive a fast\nposterior inference algorithm based on variational methods. We report\nqualitative and quantitative studies on both synthetic data and hand-parsed\ndocuments. We show that the STM is a more predictive model of language than\ncurrent models based only on syntax or only on topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.1141v1",
            "date": "2010-03-04 21:07:18+00:00",
            "title": "From Frequency to Meaning: Vector Space Models of Semantics",
            "authors": [
                "Peter D. Turney",
                "Patrick Pantel"
            ],
            "abstract": "Computers understand very little of the meaning of human language. This\nprofoundly limits our ability to give instructions to computers, the ability of\ncomputers to explain their actions to us, and the ability of computers to\nanalyse and process text. Vector space models (VSMs) of semantics are beginning\nto address these limits. This paper surveys the use of VSMs for semantic\nprocessing of text. We organize the literature on VSMs according to the\nstructure of the matrix in a VSM. There are currently three broad classes of\nVSMs, based on term-document, word-context, and pair-pattern matrices, yielding\nthree classes of applications. We survey a broad range of applications in these\nthree categories and we take a detailed look at a specific open source project\nin each category. Our goal in this survey is to show the breadth of\napplications of VSMs for semantics, to provide a new perspective on VSMs for\nthose who are already familiar with the area, and to provide pointers into the\nliterature for those who are less familiar with the field.",
            "journal ref": "Journal of Artificial Intelligence Research, (2010), 37, 141-188",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1003.1410v2",
            "date": "2010-03-06 18:08:12+00:00",
            "title": "Local Space-Time Smoothing for Version Controlled Documents",
            "authors": [
                "Seungyeon Kim",
                "Guy Lebanon"
            ],
            "abstract": "Unlike static documents, version controlled documents are continuously edited\nby one or more authors. Such collaborative revision process makes traditional\nmodeling and visualization techniques inappropriate. In this paper we propose a\nnew representation based on local space-time smoothing that captures important\nrevision patterns. We demonstrate the applicability of our framework using\nexperiments on synthetic and real-world data.",
            "journal ref": "Proceedings of the 23rd International Conference on Computational\n  Linguistics (Coling 2010); 2010 Aug 23-27; Beijing, CN",
            "category": "cs.GR"
        },
        {
            "url": "http://arxiv.org/abs/1003.4394v1",
            "date": "2010-03-23 12:32:01+00:00",
            "title": "Mathematical Foundations for a Compositional Distributional Model of Meaning",
            "authors": [
                "Bob Coecke",
                "Mehrnoosh Sadrzadeh",
                "Stephen Clark"
            ],
            "abstract": "We propose a mathematical framework for a unification of the distributional\ntheory of meaning in terms of vector space models, and a compositional theory\nfor grammatical types, for which we rely on the algebra of Pregroups,\nintroduced by Lambek. This mathematical framework enables us to compute the\nmeaning of a well-typed sentence from the meanings of its constituents.\nConcretely, the type reductions of Pregroups are `lifted' to morphisms in a\ncategory, a procedure that transforms meanings of constituents into a meaning\nof the (well-typed) whole. Importantly, meanings of whole sentences live in a\nsingle space, independent of the grammatical structure of the sentence. Hence\nthe inner-product can be used to compare meanings of arbitrary sentences, as it\nis for comparing the meanings of words in the distributional model. The\nmathematical structure we employ admits a purely diagrammatic calculus which\nexposes how the information flows between the words in a sentence in order to\nmake up the meaning of the whole sentence. A variation of our `categorical\nmodel' which involves constraining the scalars of the vector spaces to the\nsemiring of Booleans results in a Montague-style Boolean-valued semantics.",
            "journal ref": "Lambek Festschirft, special issue of Linguistic Analysis, 2010.",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.4997v1",
            "date": "2010-05-27 07:24:29+00:00",
            "title": "Network analysis of a corpus of undeciphered Indus civilization inscriptions indicates syntactic organization",
            "authors": [
                "Sitabhra Sinha",
                "Md Izhar Ashraf",
                "Raj Kumar Pan",
                "Bryan Kenneth Wells"
            ],
            "abstract": "Archaeological excavations in the sites of the Indus Valley civilization\n(2500-1900 BCE) in Pakistan and northwestern India have unearthed a large\nnumber of artifacts with inscriptions made up of hundreds of distinct signs. To\ndate there is no generally accepted decipherment of these sign sequences and\nthere have been suggestions that the signs could be non-linguistic. Here we\napply complex network analysis techniques to a database of available Indus\ninscriptions, with the aim of detecting patterns indicative of syntactic\norganization. Our results show the presence of patterns, e.g., recursive\nstructures in the segmentation trees of the sequences, that suggest the\nexistence of a grammar underlying these inscriptions.",
            "journal ref": "Computer Speech and Language, 25 (2011) 639-654",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1005.5253v1",
            "date": "2010-05-28 09:41:50+00:00",
            "title": "Using Soft Constraints To Learn Semantic Models Of Descriptions Of Shapes",
            "authors": [
                "Sergio Guadarrama",
                "David P. Pancho"
            ],
            "abstract": "The contribution of this paper is to provide a semantic model (using soft\nconstraints) of the words used by web-users to describe objects in a language\ngame; a game in which one user describes a selected object of those composing\nthe scene, and another user has to guess which object has been described. The\ngiven description needs to be non ambiguous and accurate enough to allow other\nusers to guess the described shape correctly.\n  To build these semantic models the descriptions need to be analyzed to\nextract the syntax and words' classes used. We have modeled the meaning of\nthese descriptions using soft constraints as a way for grounding the meaning.\n  The descriptions generated by the system took into account the context of the\nobject to avoid ambiguous descriptions, and allowed users to guess the\ndescribed object correctly 72% of the times.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1006.3271v1",
            "date": "2010-06-16 16:48:30+00:00",
            "title": "The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis",
            "authors": [
                "Anne S. Hsu",
                "Nick Chater",
                "Paul M. B. Vitanyi"
            ],
            "abstract": "There is much debate over the degree to which language learning is governed\nby innate language-specific biases, or acquired through cognition-general\nprinciples. Here we examine the probabilistic language acquisition hypothesis\non three levels: We outline a novel theoretical result showing that it is\npossible to learn the exact generative model underlying a wide class of\nlanguages, purely from observing samples of the language. We then describe a\nrecently proposed practical framework, which quantifies natural language\nlearnability, allowing specific learnability predictions to be made for the\nfirst time. In previous work, this framework was used to make learnability\npredictions for a wide variety of linguistic constructions, for which\nlearnability has been much debated. Here, we present a new experiment which\ntests these learnability predictions. We find that our experimental results\nsupport the possibility that these linguistic constructions are acquired\nprobabilistically from cognition-general principles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1007.3254v2",
            "date": "2010-07-15 23:30:11+00:00",
            "title": "Distinguishing Fact from Fiction: Pattern Recognition in Texts Using Complex Networks",
            "authors": [
                "J. T. Stevanak",
                "David M. Larue",
                "Lincoln D. Carr"
            ],
            "abstract": "We establish concrete mathematical criteria to distinguish between different\nkinds of written storytelling, fictional and non-fictional. Specifically, we\nconstructed a semantic network from both novels and news stories, with $N$\nindependent words as vertices or nodes, and edges or links allotted to words\noccurring within $m$ places of a given vertex; we call $m$ the word distance.\nWe then used measures from complex network theory to distinguish between news\nand fiction, studying the minimal text length needed as well as the optimized\nword distance $m$. The literature samples were found to be most effectively\nrepresented by their corresponding power laws over degree distribution $P(k)$\nand clustering coefficient $C(k)$; we also studied the mean geodesic distance,\nand found all our texts were small-world networks. We observed a natural\nbreak-point at $k=\\sqrt{N}$ where the power law in the degree distribution\nchanged, leading to separate power law fit for the bulk and the tail of $P(k)$.\nOur linear discriminant analysis yielded a $73.8 \\pm 5.15%$ accuracy for the\ncorrect classification of novels and $69.1 \\pm 1.22%$ for news stories. We\nfound an optimal word distance of $m=4$ and a minimum text length of 100 to 200\nwords $N$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1008.1394v1",
            "date": "2010-08-08 09:38:38+00:00",
            "title": "Towards Design and Implementation of a Language Technology based Information Processor for PDM Systems",
            "authors": [
                "Zeeshan Ahmed",
                "Saman Majeed",
                "Thomas Dandekar"
            ],
            "abstract": "Product Data Management (PDM) aims to provide 'Systems' contributing in\nindustries by electronically maintaining organizational data, improving data\nrepository system, facilitating with easy access to CAD and providing\nadditional information engineering and management modules to access, store,\nintegrate, secure, recover and manage information. Targeting one of the\nunresolved issues i.e., provision of natural language based processor for the\nimplementation of an intelligent record search mechanism, an approach is\nproposed and discussed in detail in this manuscript. Designing an intelligent\napplication capable of reading and analyzing user's structured and unstructured\nnatural language based text requests and then extracting desired concrete and\noptimized results from knowledge base is still a challenging task for the\ndesigners because it is still very difficult to completely extract Meta data\nout of raw data. Residing within the limited scope of current research and\ndevelopment; we present an approach capable of reading user's natural language\nbased input text, understanding the semantic and extracting results from\nrepositories. To evaluate the effectiveness of implemented prototyped version\nof proposed approach, it is compared with some existing PDM Systems, in the end\nthe discussion is concluded with an abstract presentation of resultant\ncomparison amongst implemented prototype and some existing PDM Systems.",
            "journal ref": "IST Transactions of Information Technology- Theory and\n  Applications, Vol. 1, No. 1 (2),ISSN 1913-8822, pp. 1-7, 2010",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1008.3667v1",
            "date": "2010-08-21 21:24:42+00:00",
            "title": "Pattern Classification In Symbolic Streams via Semantic Annihilation of Information",
            "authors": [
                "Ishanu Chattopadhyay",
                "Yicheng Wen",
                "Asok Ray"
            ],
            "abstract": "We propose a technique for pattern classification in symbolic streams via\nselective erasure of observed symbols, in cases where the patterns of interest\nare represented as Probabilistic Finite State Automata (PFSA). We define an\nadditive abelian group for a slightly restricted subset of probabilistic finite\nstate automata (PFSA), and the group sum is used to formulate pattern-specific\nsemantic annihilators. The annihilators attempt to identify pre-specified\npatterns via removal of essentially all inter-symbol correlations from observed\nsequences, thereby turning them into symbolic white noise. Thus a perfect\nannihilation corresponds to a perfect pattern match. This approach of\nclassification via information annihilation is shown to be strictly\nadvantageous, with theoretical guarantees, for a large class of PFSA models.\nThe results are supported by simulation experiments.",
            "journal ref": "None",
            "category": "cs.SC"
        },
        {
            "url": "http://arxiv.org/abs/1009.0108v1",
            "date": "2010-09-01 07:51:07+00:00",
            "title": "Emotional State Categorization from Speech: Machine vs. Human",
            "authors": [
                "Arslan Shaukat",
                "Ke Chen"
            ],
            "abstract": "This paper presents our investigations on emotional state categorization from\nspeech signals with a psychologically inspired computational model against\nhuman performance under the same experimental setup. Based on psychological\nstudies, we propose a multistage categorization strategy which allows\nestablishing an automatic categorization model flexibly for a given emotional\nspeech categorization task. We apply the strategy to the Serbian Emotional\nSpeech Corpus (GEES) and the Danish Emotional Speech Corpus (DES), where human\nperformance was reported in previous psychological studies. Our work is the\nfirst attempt to apply machine learning to the GEES corpus where the human\nrecognition rates were only available prior to our study. Unlike the previous\nwork on the DES corpus, our work focuses on a comparison to human performance\nunder the same experimental settings. Our studies suggest that\npsychology-inspired systems yield behaviours that, to a great extent, resemble\nwhat humans perceived and their performance is close to that of humans under\nthe same experimental setup. Furthermore, our work also uncovers some\ndifferences between machine and humans in terms of emotional state recognition\nfrom speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1009.2706v1",
            "date": "2010-09-14 16:06:02+00:00",
            "title": "Minimization Strategies for Maximally Parallel Multiset Rewriting Systems",
            "authors": [
                "Artiom Alhazov",
                "Sergey Verlan"
            ],
            "abstract": "Maximally parallel multiset rewriting systems (MPMRS) give a convenient way\nto express relations between unstructured objects. The functioning of various\ncomputational devices may be expressed in terms of MPMRS (e.g., register\nmachines and many variants of P systems). In particular, this means that MPMRS\nare computationally complete; however, a direct translation leads to quite a\nbig number of rules. Like for other classes of computationally complete\ndevices, there is a challenge to find a universal system having the smallest\nnumber of rules. In this article we present different rule minimization\nstrategies for MPMRS based on encodings and structural transformations. We\napply these strategies to the translation of a small universal register machine\n(Korec, 1996) and we show that there exists a universal MPMRS with 23 rules.\nSince MPMRS are identical to a restricted variant of P systems with antiport\nrules, the results we obtained improve previously known results on the number\nof rules for those systems.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1101.0309v1",
            "date": "2010-12-31 23:57:06+00:00",
            "title": "Concrete Sentence Spaces for Compositional Distributional Models of Meaning",
            "authors": [
                "Edward Grefenstette",
                "Mehrnoosh Sadrzadeh",
                "Stephen Clark",
                "Bob Coecke",
                "Stephen Pulman"
            ],
            "abstract": "Coecke, Sadrzadeh, and Clark (arXiv:1003.4394v1 [cs.CL]) developed a\ncompositional model of meaning for distributional semantics, in which each word\nin a sentence has a meaning vector and the distributional meaning of the\nsentence is a function of the tensor products of the word vectors. Abstractly\nspeaking, this function is the morphism corresponding to the grammatical\nstructure of the sentence in the category of finite dimensional vector spaces.\nIn this paper, we provide a concrete method for implementing this linear\nmeaning map, by constructing a corpus-based vector space for the type of\nsentence. Our construction method is based on structured vector spaces whereby\nmeaning vectors of all sentences, regardless of their grammatical structure,\nlive in the same vector space. Our proposed sentence space is the tensor\nproduct of two noun spaces, in which the basis vectors are pairs of words each\naugmented with a grammatical role. This enables us to compare meanings of\nsentences by simply taking the inner product of their vectors.",
            "journal ref": "Proceedings of the 9th International Conference on Computational\n  Semantics (2011)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1004.3183v2",
            "date": "2010-04-19 13:11:56+00:00",
            "title": "Statistical Physics for Natural Language Processing",
            "authors": [
                "Juan-Manuel Torres Moreno",
                "Silvia Fernandez",
                "Eric SanJuan"
            ],
            "abstract": "This paper has been withdrawn by the author.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1010.3003v1",
            "date": "2010-10-14 18:55:52+00:00",
            "title": "Twitter mood predicts the stock market",
            "authors": [
                "Johan Bollen",
                "Huina Mao",
                "Xiao-Jun Zeng"
            ],
            "abstract": "Behavioral economics tells us that emotions can profoundly affect individual\nbehavior and decision-making. Does this also apply to societies at large, i.e.,\ncan societies experience mood states that affect their collective decision\nmaking? By extension is the public mood correlated or even predictive of\neconomic indicators? Here we investigate whether measurements of collective\nmood states derived from large-scale Twitter feeds are correlated to the value\nof the Dow Jones Industrial Average (DJIA) over time. We analyze the text\ncontent of daily Twitter feeds by two mood tracking tools, namely OpinionFinder\nthat measures positive vs. negative mood and Google-Profile of Mood States\n(GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital,\nKind, and Happy). We cross-validate the resulting mood time series by comparing\ntheir ability to detect the public's response to the presidential election and\nThanksgiving day in 2008. A Granger causality analysis and a Self-Organizing\nFuzzy Neural Network are then used to investigate the hypothesis that public\nmood states, as measured by the OpinionFinder and GPOMS mood time series, are\npredictive of changes in DJIA closing values. Our results indicate that the\naccuracy of DJIA predictions can be significantly improved by the inclusion of\nspecific public mood dimensions but not others. We find an accuracy of 87.6% in\npredicting the daily up and down changes in the closing values of the DJIA and\na reduction of the Mean Average Percentage Error by more than 6%.",
            "journal ref": "Journal of Computational Science, 2(1), March 2011, Pages 1-8",
            "category": "cs.CE"
        },
        {
            "url": "http://arxiv.org/abs/1010.6091v5",
            "date": "2010-10-28 20:50:46+00:00",
            "title": "Network motifs in music sequences",
            "authors": [
                "Damian H. Zanette"
            ],
            "abstract": "This paper has been withdrawn by the author because it needs a deep\nmethodological revision.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1012.5248v1",
            "date": "2010-12-23 17:00:40+00:00",
            "title": "Matrix Insertion-Deletion Systems",
            "authors": [
                "Ion Petre",
                "Sergey Verlan"
            ],
            "abstract": "In this article, we consider for the first time the operations of insertion\nand deletion working in a matrix controlled manner. We show that, similarly as\nin the case of context-free productions, the computational power is strictly\nincreased when using a matrix control: computational completeness can be\nobtained by systems with insertion or deletion rules involving at most two\nsymbols in a contextual or in a context-free manner and using only binary\nmatrices.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1009.3321v2",
            "date": "2010-09-17 02:46:04+00:00",
            "title": "Niche as a determinant of word fate in online groups",
            "authors": [
                "Eduardo G. Altmann",
                "Janet B. Pierrehumbert",
                "Adilson E. Motter"
            ],
            "abstract": "Patterns of word use both reflect and influence a myriad of human activities\nand interactions. Like other entities that are reproduced and evolve, words\nrise or decline depending upon a complex interplay between {their intrinsic\nproperties and the environments in which they function}. Using Internet\ndiscussion communities as model systems, we define the concept of a word niche\nas the relationship between the word and the characteristic features of the\nenvironments in which it is used. We develop a method to quantify two important\naspects of the size of the word niche: the range of individuals using the word\nand the range of topics it is used to discuss. Controlling for word frequency,\nwe show that these aspects of the word niche are strong determinants of changes\nin word frequency. Previous studies have already indicated that word frequency\nitself is a correlate of word success at historical time scales. Our analysis\nof changes in word frequencies over time reveals that the relative sizes of\nword niches are far more important than word frequencies in the dynamics of the\nentire vocabulary at shorter time scales, as the language adapts to new\nconcepts and social groupings. We also distinguish endogenous versus exogenous\nfactors as additional contributors to the fates of words, and demonstrate the\nforce of this distinction in the rise of novel words. Our results indicate that\nshort-term nonstationarity in word statistics is strongly driven by individual\nproclivities, including inclinations to provide novel information and to\nproject a distinctive social identity.",
            "journal ref": "PLoS ONE 6(5), e19009 (2011)",
            "category": "cs.CL"
        }
    ],
    "2011": [
        {
            "url": "http://arxiv.org/abs/1101.5076v6",
            "date": "2011-01-26 14:56:20+00:00",
            "title": "Geometric representations for minimalist grammars",
            "authors": [
                "Peter beim Graben",
                "Sabrina Gerth"
            ],
            "abstract": "We reformulate minimalist grammars as partial functions on term algebras for\nstrings and trees. Using filler/role bindings and tensor product\nrepresentations, we construct homomorphisms for these data structures into\ngeometric vector spaces. We prove that the structure-building functions as well\nas simple processors for minimalist languages can be realized by piecewise\nlinear operators in representation space. We also propose harmony, i.e. the\ndistance of an intermediate processing step from the final well-formed state in\nrepresentation space, as a measure of processing complexity. Finally, we\nillustrate our findings by means of two particular arithmetic and fractal\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1101.5494v1",
            "date": "2011-01-28 09:58:39+00:00",
            "title": "Developing a New Approach for Arabic Morphological Analysis and Generation",
            "authors": [
                "Mourad Gridach",
                "Noureddine Chenfour"
            ],
            "abstract": "Arabic morphological analysis is one of the essential stages in Arabic\nNatural Language Processing. In this paper we present an approach for Arabic\nmorphological analysis. This approach is based on Arabic morphological\nautomaton (AMAUT). The proposed technique uses a morphological database\nrealized using XMODEL language. Arabic morphology represents a special type of\nmorphological systems because it is based on the concept of scheme to represent\nArabic words. We use this concept to develop the Arabic morphological automata.\nThe proposed approach has development standardization aspect. It can be\nexploited by NLP applications such as syntactic and semantic analysis,\ninformation retrieval, machine translation and orthographical correction. The\nproposed approach is compared with Xerox Arabic Analyzer and Smrz Arabic\nAnalyzer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1101.5757v1",
            "date": "2011-01-30 10:41:46+00:00",
            "title": "Polarized Montagovian Semantics for the Lambek-Grishin calculus",
            "authors": [
                "Arno Bastenhof"
            ],
            "abstract": "Grishin proposed enriching the Lambek calculus with multiplicative\ndisjunction (par) and coresiduals. Applications to linguistics were discussed\nby Moortgat, who spoke of the Lambek-Grishin calculus (LG). In this paper, we\nadapt Girard's polarity-sensitive double negation embedding for classical logic\nto extract a compositional Montagovian semantics from a display calculus for\nfocused proof search in LG. We seize the opportunity to illustrate our approach\nalongside an analysis of extraction, providing linguistic motivation for linear\ndistributivity of tensor over par, thus answering a question of\nKurtonina&Moortgat. We conclude by comparing our proposal to the continuation\nsemantics of Bernardi&Moortgat, corresponding to call-by- name and\ncall-by-value evaluation strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1103.1898v1",
            "date": "2011-03-09 21:43:46+00:00",
            "title": "Recognizing Uncertainty in Speech",
            "authors": [
                "Heather Pon-Barry",
                "Stuart M. Shieber"
            ],
            "abstract": "We address the problem of inferring a speaker's level of certainty based on\nprosodic information in the speech signal, which has application in\nspeech-based dialogue systems. We show that using phrase-level prosodic\nfeatures centered around the phrases causing uncertainty, in addition to\nutterance-level prosodic features, improves our model's level of certainty\nclassification. In addition, our models can be used to predict which phrase a\nperson is uncertain about. These results rely on a novel method for eliciting\nutterances of varying levels of certainty that allows us to compare the utility\nof contextually-based feature sets. We elicit level of certainty ratings from\nboth the speakers themselves and a panel of listeners, finding that there is\noften a mismatch between speakers' internal states and their perceived states,\nand highlighting the importance of this distinction.",
            "journal ref": "EURASIP Journal on Advances in Signal Processing, Volume 2011,\n  Article ID 251753, 11 pages",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1103.2950v1",
            "date": "2011-03-15 16:21:24+00:00",
            "title": "Fitting Ranked English and Spanish Letter Frequency Distribution in U.S. and Mexican Presidential Speeches",
            "authors": [
                "Wentian Li",
                "Pedro Miramontes"
            ],
            "abstract": "The limited range in its abscissa of ranked letter frequency distributions\ncauses multiple functions to fit the observed distribution reasonably well. In\norder to critically compare various functions, we apply the statistical model\nselections on ten functions, using the texts of U.S. and Mexican presidential\nspeeches in the last 1-2 centuries. Dispite minor switching of ranking order of\ncertain letters during the temporal evolution for both datasets, the letter\nusage is generally stable. The best fitting function, judged by either\nleast-square-error or by AIC/BIC model selection, is the Cocho/Beta function.\nWe also use a novel method to discover clusters of letters by their\nobserved-over-expected frequency ratios.",
            "journal ref": "Journal of Quantitative Linguistics, 18(4):359-380 (2011)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1103.5676v1",
            "date": "2011-03-29 15:05:11+00:00",
            "title": "Codeco: A Grammar Notation for Controlled Natural Language in Predictive Editors",
            "authors": [
                "Tobias Kuhn"
            ],
            "abstract": "Existing grammar frameworks do not work out particularly well for controlled\nnatural languages (CNL), especially if they are to be used in predictive\neditors. I introduce in this paper a new grammar notation, called Codeco, which\nis designed specifically for CNLs and predictive editors. Two different parsers\nhave been implemented and a large subset of Attempto Controlled English (ACE)\nhas been represented in Codeco. The results show that Codeco is practical,\nadequate and efficient.",
            "journal ref": "In Pre-Proceedings of the Second Workshop on Controlled Natural\n  Languages (CNL 2010), CEUR Workshop Proceedings, Volume 622, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1104.2034v1",
            "date": "2011-04-11 19:50:50+00:00",
            "title": "Materials to the Russian-Bulgarian Comparative Dictionary \"EAD\"",
            "authors": [
                "Yavor Angelov Parvanov"
            ],
            "abstract": "This article presents a fragment of a new comparative dictionary \"A\ncomparative dictionary of names of expansive action in Russian and Bulgarian\nlanguages\". Main features of the new web-based comparative dictionary are\nplaced, the principles of its formation are shown, primary links between the\nword-matches are classified. The principal difference between translation\ndictionaries and the model of double comparison is also shown. The\nclassification scheme of the pages is proposed. New concepts and keywords have\nbeen introduced. The real prototype of the dictionary with a few key pages is\npublished. The broad debate about the possibility of this prototype to become a\nversion of Russian-Bulgarian comparative dictionary of a new generation is\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1104.2086v1",
            "date": "2011-04-11 23:06:54+00:00",
            "title": "A Universal Part-of-Speech Tagset",
            "authors": [
                "Slav Petrov",
                "Dipanjan Das",
                "Ryan McDonald"
            ],
            "abstract": "To facilitate future research in unsupervised induction of syntactic\nstructure and to standardize best-practices, we propose a tagset that consists\nof twelve universal part-of-speech categories. In addition to the tagset, we\ndevelop a mapping from 25 different treebank tagsets to this universal set. As\na result, when combined with the original treebank data, this universal tagset\nand mapping produce a dataset consisting of common parts-of-speech for 22\ndifferent languages. We highlight the use of this resource via two experiments,\nincluding one that reports competitive accuracies for unsupervised grammar\ninduction without gold standard part-of-speech tags.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1104.4321v1",
            "date": "2011-04-21 17:56:50+00:00",
            "title": "Seeking Meaning in a Space Made out of Strokes, Radicals, Characters and Compounds",
            "authors": [
                "Yannis Haralambous"
            ],
            "abstract": "Chinese characters can be compared to a molecular structure: a character is\nanalogous to a molecule, radicals are like atoms, calligraphic strokes\ncorrespond to elementary particles, and when characters form compounds, they\nare like molecular structures. In chemistry the conjunction of all of these\nstructural levels produces what we perceive as matter. In language, the\nconjunction of strokes, radicals, characters, and compounds produces meaning.\nBut when does meaning arise? We all know that radicals are, in some sense, the\nbasic semantic components of Chinese script, but what about strokes?\nConsidering the fact that many characters are made by adding individual strokes\nto (combinations of) radicals, we can legitimately ask the question whether\nstrokes carry meaning, or not. In this talk I will present my project of\nextending traditional NLP techniques to radicals and strokes, aiming to obtain\na deeper understanding of the way ideographic languages model the world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1104.4681v1",
            "date": "2011-04-25 05:00:25+00:00",
            "title": "Performance Evaluation of Statistical Approaches for Text Independent Speaker Recognition Using Source Feature",
            "authors": [
                "R. Rajeswara Rao",
                "V. Kamakshi Prasad",
                "A. Nagesh"
            ],
            "abstract": "This paper introduces the performance evaluation of statistical approaches\nfor TextIndependent speaker recognition system using source feature. Linear\nprediction LP residual is used as a representation of excitation information in\nspeech. The speaker-specific information in the excitation of voiced speech is\ncaptured using statistical approaches such as Gaussian Mixture Models GMMs and\nHidden Markov Models HMMs. The decrease in the error during training and\nrecognizing speakers during testing phase close to 100 percent accuracy\ndemonstrates that the excitation component of speech contains speaker-specific\ninformation and is indeed being effectively captured by continuous Ergodic HMM\nthan GMM. The performance of the speaker recognition system is evaluated on GMM\nand 2 state ergodic HMM with different mixture components and test speech\nduration. We demonstrate the speaker recognition studies on TIMIT database for\nboth GMM and Ergodic HMM.",
            "journal ref": "InterJRI Computer Science and Networking, Volume 2, Issue 1 August\n  2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1105.1072v1",
            "date": "2011-05-05 13:51:46+00:00",
            "title": "English-Lithuanian-English Machine Translation lexicon and engine: current state and future work",
            "authors": [
                "G. Barisevi\u010dius",
                "B. Tamulynas"
            ],
            "abstract": "This article overviews the current state of the English-Lithuanian-English\nmachine translation system. The first part of the article describes the\nproblems that system poses today and what actions will be taken to solve them\nin the future. The second part of the article tackles the main issue of the\ntranslation process. Article briefly overviews the word sense disambiguation\nfor MT technique using Google.",
            "journal ref": "Informacin\\.{e}s technologijos 2006 : conference proceedings /\n  Kaunas University of Technology. T. 1. Kaunas : Technologija, 2006. p.\n  109-112",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1105.1226v1",
            "date": "2011-05-06 05:26:43+00:00",
            "title": "Multilingual lexicon design tool and database management system for MT",
            "authors": [
                "G. Barisevi\u010dius",
                "B. Tamulynas"
            ],
            "abstract": "The paper presents the design and development of English-Lithuanian-English\ndictionarylexicon tool and lexicon database management system for MT. The\nsystem is oriented to support two main requirements: to be open to the user and\nto describe much more attributes of speech parts as a regular dictionary that\nare required for the MT. Programming language Java and database management\nsystem MySql is used to implement the designing tool and lexicon database\nrespectively. This solution allows easily deploying this system in the\nInternet. The system is able to run on various OS such as: Windows, Linux, Mac\nand other OS where Java Virtual Machine is supported. Since the modern lexicon\ndatabase managing system is used, it is not a problem accessing the same\ndatabase for several users.",
            "journal ref": "Proceedings of the 2nd Baltic Conference on Human Language\n  Technologies, April 4-5, 2005, Tallin, Estonia. Tallin : Tallin University of\n  Technology, 2005. ISBN 9985894839. p. 219-224",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1105.6162v2",
            "date": "2011-05-31 05:03:06+00:00",
            "title": "A statistical learning algorithm for word segmentation",
            "authors": [
                "Jerry R. Van Aken"
            ],
            "abstract": "In natural speech, the speaker does not pause between words, yet a human\nlistener somehow perceives this continuous stream of phonemes as a series of\ndistinct words. The detection of boundaries between spoken words is an instance\nof a general capability of the human neocortex to remember and to recognize\nrecurring sequences. This paper describes a computer algorithm that is designed\nto solve the problem of locating word boundaries in blocks of English text from\nwhich the spaces have been removed. This problem avoids the complexities of\nspeech processing but requires similar capabilities for detecting recurring\nsequences. The algorithm relies entirely on statistical relationships between\nletters in the input stream to infer the locations of word boundaries. A\nViterbi trellis is used to simultaneously evaluate a set of hypothetical\nsegmentations of a block of adjacent words. This technique improves accuracy\nbut incurs a small latency between the arrival of letters in the input stream\nand the sending of words to the output stream. The source code for a C++\nversion of this algorithm is presented in an appendix.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.0193v3",
            "date": "2011-07-01 11:01:52+00:00",
            "title": "On the origin of ambiguity in efficient communication",
            "authors": [
                "Jordi Fortuny",
                "Bernat Corominas-Murtra"
            ],
            "abstract": "This article studies the emergence of ambiguity in communication through the\nconcept of logical irreversibility and within the framework of Shannon's\ninformation theory. This leads us to a precise and general expression of the\nintuition behind Zipf's vocabulary balance in terms of a symmetry equation\nbetween the complexities of the coding and the decoding processes that imposes\nan unavoidable amount of logical uncertainty in natural communication.\nAccordingly, the emergence of irreversible computations is required if the\ncomplexities of the coding and the decoding processes are balanced in a\nsymmetric scenario, which means that the emergence of ambiguous codes is a\nnecessary condition for natural communication to succeed.",
            "journal ref": "Journal of Logic, Language and Information. 0925-8531, pp 1--19\n  (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.1753v1",
            "date": "2011-07-09 00:40:06+00:00",
            "title": "Notes on Electronic Lexicography",
            "authors": [
                "Yavor Parvanov"
            ],
            "abstract": "These notes are a continuation of topics covered by V. Selegej in his article\n\"Electronic Dictionaries and Computational lexicography\". How can an electronic\ndictionary have as its object the description of closely related languages?\nObviously, such a question allows multiple answers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4687v2",
            "date": "2011-07-23 12:56:02+00:00",
            "title": "Fence - An Efficient Parser with Ambiguity Support for Model-Driven Language Specification",
            "authors": [
                "Luis Quesada",
                "Fernando Berzal",
                "Francisco J. Cortijo"
            ],
            "abstract": "Model-based language specification has applications in the implementation of\nlanguage processors, the design of domain-specific languages, model-driven\nsoftware development, data integration, text mining, natural language\nprocessing, and corpus-based induction of models. Model-based language\nspecification decouples language design from language processing and, unlike\ntraditional grammar-driven approaches, which constrain language designers to\nspecific kinds of grammars, it needs general parser generators able to deal\nwith ambiguities. In this paper, we propose Fence, an efficient bottom-up\nparsing algorithm with lexical and syntactic ambiguity support that enables the\nuse of model-based language specification in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4723v2",
            "date": "2011-07-24 06:27:56+00:00",
            "title": "A Semantic Relatedness Measure Based on Combined Encyclopedic, Ontological and Collocational Knowledge",
            "authors": [
                "Yannis Haralambous",
                "Vitaly Klyuev"
            ],
            "abstract": "We describe a new semantic relatedness measure combining the Wikipedia-based\nExplicit Semantic Analysis measure, the WordNet path measure and the mixed\ncollocation index. Our measure achieves the currently highest results on the\nWS-353 test: a Spearman rho coefficient of 0.79 (vs. 0.75 in (Gabrilovich and\nMarkovitch, 2007)) when applying the measure directly, and a value of 0.87 (vs.\n0.78 in (Agirre et al., 2009)) when using the prediction of a polynomial SVM\nclassifier trained on our measure.\n  In the appendix we discuss the adaptation of ESA to 2011 Wikipedia data, as\nwell as various unsuccessful attempts to enhance ESA by filtering at word,\nsentence, and section level.",
            "journal ref": "Proceedings of the 5th International Joint Conference on Natural\n  Language Processing, Chiang-Mai, Thailand, November 8-13, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4734v1",
            "date": "2011-07-24 08:35:12+00:00",
            "title": "Design of Arabic Diacritical Marks",
            "authors": [
                "Mohamed Hssini",
                "Azzeddine Lazrek"
            ],
            "abstract": "Diacritical marks play a crucial role in meeting the criteria of usability of\ntypographic text, such as: homogeneity, clarity and legibility. To change the\ndiacritic of a letter in a word could completely change its semantic. The\nsituation is very complicated with multilingual text. Indeed, the problem of\ndesign becomes more difficult by the presence of diacritics that come from\nvarious scripts; they are used for different purposes, and are controlled by\nvarious typographic rules. It is quite challenging to adapt rules from one\nscript to another. This paper aims to study the placement and sizing of\ndiacritical marks in Arabic script, with a comparison with the Latin's case.\nThe Arabic script is cursive and runs from right-to-left; its criteria and\nrules are quite distinct from those of the Latin script. In the beginning, we\ncompare the difficulty of processing diacritics in both scripts. After, we will\nstudy the limits of Latin resolution strategies when applied to Arabic. At the\nend, we propose an approach to resolve the problem for positioning and resizing\ndiacritics. This strategy includes creating an Arabic font, designed in\nOpenType format, along with suitable justification in TEX.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 8,\n  Issue 3, May 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4796v1",
            "date": "2011-07-24 20:37:57+00:00",
            "title": "Use Pronunciation by Analogy for text to speech system in Persian language",
            "authors": [
                "Ali Jowharpour",
                "Masha allah abbasi dezfuli",
                "Mohammad hosein Yektaee"
            ],
            "abstract": "The interest in text to speech synthesis increased in the world .text to\nspeech have been developed formany popular languages such as English, Spanish\nand French and many researches and developmentshave been applied to those\nlanguages. Persian on the other hand, has been given little attentioncompared\nto other languages of similar importance and the research in Persian is still\nin its infancy.Persian language possess many difficulty and exceptions that\nincrease complexity of text to speechsystems. For example: short vowels is\nabsent in written text or existence of homograph words. in thispaper we propose\na new method for persian text to phonetic that base on pronunciations by\nanalogy inwords, semantic relations and grammatical rules for finding proper\nphonetic. Keywords:PbA, text to speech, Persian language, FPbA",
            "journal ref": "IJCSI Volume 8, Issue 3, May 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.5743v1",
            "date": "2011-07-28 15:37:56+00:00",
            "title": "NEMO: Extraction and normalization of organization names from PubMed affiliation strings",
            "authors": [
                "Siddhartha Jonnalagadda",
                "Philip Topham"
            ],
            "abstract": "We propose NEMO, a system for extracting organization names in the\naffiliation and normalizing them to a canonical organization name. Our parsing\nprocess involves multi-layered rule matching with multiple dictionaries. The\nsystem achieves more than 98% f-score in extracting organization names. Our\nprocess of normalization that involves clustering based on local sequence\nalignment metrics and local learning based on finding connected components. A\nhigh precision was also observed in normalization. NEMO is the missing link in\nassociating each biomedical paper and its authors to an organization name in\nits canonical form and the Geopolitical location of the organization. This\nresearch could potentially help in analyzing large social networks of\norganizations for landscaping a particular topic, improving performance of\nauthor disambiguation, adding weak links in the co-author network of authors,\naugmenting NLM's MARS system for correcting errors in OCR output of affiliation\nfield, and automatically indexing the PubMed citations with the normalized\norganization name and country. Our system is available as a graphical user\ninterface available for download along with this paper.",
            "journal ref": "Siddhartha Jonnalagadda, Philip Topham. NEMO: Extraction and\n  normalization of organization names from PubMed affiliation strings. Journal\n  of Biomedical Discovery and Collaboration, 2010 Oct 4;5:50-75",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.5744v1",
            "date": "2011-07-28 15:40:31+00:00",
            "title": "BioSimplify: an open source sentence simplification engine to improve recall in automatic biomedical information extraction",
            "authors": [
                "Siddhartha Jonnalagadda",
                "Graciela Gonzalez"
            ],
            "abstract": "BioSimplify is an open source tool written in Java that introduces and\nfacilitates the use of a novel model for sentence simplification tuned for\nautomatic discourse analysis and information extraction (as opposed to sentence\nsimplification for improving human readability). The model is based on a\n\"shot-gun\" approach that produces many different (simpler) versions of the\noriginal sentence by combining variants of its constituent elements. This tool\nis optimized for processing biomedical scientific literature such as the\nabstracts indexed in PubMed. We tested our tool on its impact to the task of\nPPI extraction and it improved the f-score of the PPI tool by around 7%, with\nan improvement in recall of around 20%. The BioSimplify tool and test corpus\ncan be downloaded from https://biosimplify.sourceforge.net.",
            "journal ref": "Siddhartha Jonnalagadda, Graciela Gonzalez. BioSimplify: an open\n  source sentence simplification engine to improve recall in automatic\n  biomedical information extraction. In Annual Proceedings of AMIA 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.5752v2",
            "date": "2011-07-28 15:59:21+00:00",
            "title": "An Effective Approach to Biomedical Information Extraction with Limited Training Data",
            "authors": [
                "Siddhartha Jonnalagadda"
            ],
            "abstract": "Overall, the two main contributions of this work include the application of\nsentence simplification to association extraction as described above, and the\nuse of distributional semantics for concept extraction. The proposed work on\nconcept extraction amalgamates for the first time two diverse research areas\n-distributional semantics and information extraction. This approach renders all\nthe advantages offered in other semi-supervised machine learning systems, and,\nunlike other proposed semi-supervised approaches, it can be used on top of\ndifferent basic frameworks and algorithms.\nhttp://gradworks.umi.com/34/49/3449837.html",
            "journal ref": "Jonnalagadda S. An effective approach to biomedical information\n  extraction with limited training data (PhD Dissertation, Arizona State\n  University). 2011;",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.0353v2",
            "date": "2011-08-01 16:20:50+00:00",
            "title": "Cross-moments computation for stochastic context-free grammars",
            "authors": [
                "Velimir M. Ilic",
                "Miroslav D. Ciric",
                "Miomir S. Stankovic"
            ],
            "abstract": "In this paper we consider the problem of efficient computation of\ncross-moments of a vector random variable represented by a stochastic\ncontext-free grammar. Two types of cross-moments are discussed. The sample\nspace for the first one is the set of all derivations of the context-free\ngrammar, and the sample space for the second one is the set of all derivations\nwhich generate a string belonging to the language of the grammar. In the past,\nthis problem was widely studied, but mainly for the cross-moments of scalar\nvariables and up to the second order. This paper presents new algorithms for\ncomputing the cross-moments of an arbitrary order, and the previously developed\nones are derived as special cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.0631v3",
            "date": "2011-08-02 17:33:36+00:00",
            "title": "Serialising the ISO SynAF Syntactic Object Model",
            "authors": [
                "Laurent Romary",
                "Amir Zeldes",
                "Florian Zipser"
            ],
            "abstract": "This paper introduces, an XML format developed to serialise the object model\ndefined by the ISO Syntactic Annotation Framework SynAF. Based on widespread\nbest practices we adapt a popular XML format for syntactic annotation,\nTigerXML, with additional features to support a variety of syntactic phenomena\nincluding constituent and dependency structures, binding, and different node\ntypes such as compounds or empty elements. We also define interfaces to other\nformats and standards including the Morpho-syntactic Annotation Framework MAF\nand the ISOCat Data Category Registry. Finally a case study of the German\nTreebank TueBa-D/Z is presented, showcasing the handling of constituent\nstructures, topological fields and coreference annotation in tandem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.1966v1",
            "date": "2011-08-09 16:03:24+00:00",
            "title": "A Concise Query Language with Search and Transform Operations for Corpora with Multiple Levels of Annotation",
            "authors": [
                "Anil Kumar Singh"
            ],
            "abstract": "The usefulness of annotated corpora is greatly increased if there is an\nassociated tool that can allow various kinds of operations to be performed in a\nsimple way. Different kinds of annotation frameworks and many query languages\nfor them have been proposed, including some to deal with multiple layers of\nannotation. We present here an easy to learn query language for a particular\nkind of annotation framework based on 'threaded trees', which are somewhere\nbetween the complete order of a tree and the anarchy of a graph. Through\n'typed' threads, they can allow multiple levels of annotation in the same\ndocument. Our language has a simple, intuitive and concise syntax and high\nexpressive power. It allows not only to search for complicated patterns with\nshort queries but also allows data manipulation and specification of arbitrary\nreturn values. Many of the commonly used tasks that otherwise require writing\nprograms, can be performed with one or more queries. We compare the language\nwith some others and try to evaluate it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.3843v1",
            "date": "2011-08-18 20:04:28+00:00",
            "title": "Using Inverse lambda and Generalization to Translate English to Formal Languages",
            "authors": [
                "Chitta Baral",
                "Juraj Dzifcak",
                "Marcos Alvarez Gonzalez",
                "Jiayu Zhou"
            ],
            "abstract": "We present a system to translate natural language sentences to formulas in a\nformal or a knowledge representation language. Our system uses two inverse\nlambda-calculus operators and using them can take as input the semantic\nrepresentation of some words, phrases and sentences and from that derive the\nsemantic representation of other words and phrases. Our inverse lambda operator\nworks on many formal languages including first order logic, database query\nlanguages and answer set programming. Our system uses a syntactic combinatorial\ncategorial parser to parse natural language sentences and also to construct the\nsemantic meaning of the sentences as directed by their parsing. The same parser\nis used for both. In addition to the inverse lambda-calculus operators, our\nsystem uses a notion of generalization to learn semantic representation of\nwords from the semantic representation of other words that are of the same\ncategory. Together with this, we use an existing statistical learning approach\nto assign weights to deal with multiple meanings of words. Our system produces\nimproved results on standard corpora on natural language interfaces for robot\ncommand and control and database queries.",
            "journal ref": "Proceedings of International Conference on Computational Semantics\n  (IWCS) 2011, Oxford, pp:35-44",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.3848v1",
            "date": "2011-08-18 20:12:50+00:00",
            "title": "Language understanding as a step towards human level intelligence - automatizing the construction of the initial dictionary from example sentences",
            "authors": [
                "Chitta Baral",
                "Juraj Dzifcak"
            ],
            "abstract": "For a system to understand natural language, it needs to be able to take\nnatural language text and answer questions given in natural language with\nrespect to that text; it also needs to be able to follow instructions given in\nnatural language. To achieve this, a system must be able to process natural\nlanguage and be able to capture the knowledge within that text. Thus it needs\nto be able to translate natural language text into a formal language. We\ndiscuss our approach to do this, where the translation is achieved by composing\nthe meaning of words in a sentence. Our initial approach uses an inverse lambda\nmethod that we developed (and other methods) to learn meaning of words from\nmeaning of sentences and an initial lexicon. We then present an improved method\nwhere the initial lexicon is also learned by analyzing the training sentence\nand meaning pairs. We evaluate our methods and compare them with other existing\nmethods on a corpora of database querying and robot command and control.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.4052v1",
            "date": "2011-08-19 21:41:29+00:00",
            "title": "Query Expansion: Term Selection using the EWC Semantic Relatedness Measure",
            "authors": [
                "Vitaly Klyuev",
                "Yannis Haralambous"
            ],
            "abstract": "This paper investigates the efficiency of the EWC semantic relatedness\nmeasure in an ad-hoc retrieval task. This measure combines the Wikipedia-based\nExplicit Semantic Analysis measure, the WordNet path measure and the mixed\ncollocation index. In the experiments, the open source search engine Terrier\nwas utilised as a tool to index and retrieve data. The proposed technique was\ntested on the NTCIR data collection. The experiments demonstrated promising\nresults.",
            "journal ref": "Proceedings of 1st International Workshop on Advances in Semantic\n  Information Retrieval (ASIR'11), Szczecin, Poland, September 18-21, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5096v1",
            "date": "2011-08-25 14:15:46+00:00",
            "title": "Minimalist Grammars and Minimalist Categorial Grammars, definitions toward inclusion of generated languages",
            "authors": [
                "Maxime Amblard"
            ],
            "abstract": "Stabler proposes an implementation of the Chomskyan Minimalist Program,\nChomsky 95 with Minimalist Grammars - MG, Stabler 97. This framework inherits a\nlong linguistic tradition. But the semantic calculus is more easily added if\none uses the Curry-Howard isomorphism. Minimalist Categorial Grammars - MCG,\nbased on an extension of the Lambek calculus, the mixed logic, were introduced\nto provide a theoretically-motivated syntax-semantics interface, Amblard 07. In\nthis article, we give full definitions of MG with algebraic tree descriptions\nand of MCG, and take the first steps towards giving a proof of inclusion of\ntheir generated languages.",
            "journal ref": "Logic and Grammar (2011) 1-20",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.0069v2",
            "date": "2011-09-01 02:20:12+00:00",
            "title": "Inter-rater Agreement on Sentence Formality",
            "authors": [
                "Shibamouli Lahiri",
                "Xiaofei Lu"
            ],
            "abstract": "Formality is one of the most important dimensions of writing style variation.\nIn this study we conducted an inter-rater reliability experiment for assessing\nsentence formality on a five-point Likert scale, and obtained good agreement\nresults as well as different rating distributions for different sentence\ncategories. We also performed a difficulty analysis to identify the bottlenecks\nof our rating procedure. Our main objective is to design an automatic scoring\nmechanism for sentence-level formality, and this study is important for that\npurpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.0624v1",
            "date": "2011-09-03 14:30:44+00:00",
            "title": "Building Ontologies to Understand Spoken Tunisian Dialect",
            "authors": [
                "Marwa Graja",
                "Maher Jaoua",
                "Lamia Hadrich Belguith"
            ],
            "abstract": "This paper presents a method to understand spoken Tunisian dialect based on\nlexical semantic. This method takes into account the specificity of the\nTunisian dialect which has no linguistic processing tools. This method is\nontology-based which allows exploiting the ontological concepts for semantic\nannotation and ontological relations for speech interpretation. This\ncombination increases the rate of comprehension and limits the dependence on\nlinguistic resources. This paper also details the process of building the\nontology used for annotation and interpretation of Tunisian dialect in the\ncontext of speech understanding in dialogue systems for restricted domain.",
            "journal ref": "International Journal of Computer Science, Engineering and\n  Applications (IJCSEA) Vol.1, No.4, August 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.2128v2",
            "date": "2011-09-09 20:20:38+00:00",
            "title": "LexRank: Graph-based Lexical Centrality as Salience in Text Summarization",
            "authors": [
                "Gunes Erkan",
                "Dragomir R. Radev"
            ],
            "abstract": "We introduce a stochastic graph-based method for computing relative\nimportance of textual units for Natural Language Processing. We test the\ntechnique on the problem of Text Summarization (TS). Extractive TS relies on\nthe concept of sentence salience to identify the most important sentences in a\ndocument or set of documents. Salience is typically defined in terms of the\npresence of particular important words or in terms of similarity to a centroid\npseudo-sentence. We consider a new approach, LexRank, for computing sentence\nimportance based on the concept of eigenvector centrality in a graph\nrepresentation of sentences. In this model, a connectivity matrix based on\nintra-sentence cosine similarity is used as the adjacency matrix of the graph\nrepresentation of sentences. Our system, based on LexRank ranked in first place\nin more than one task in the recent DUC 2004 evaluation. In this paper we\npresent a detailed analysis of our approach and apply it to a larger data set\nincluding data from earlier DUC evaluations. We discuss several methods to\ncompute centrality using the similarity graph. The results show that\ndegree-based methods (including LexRank) outperform both centroid-based methods\nand other systems participating in DUC in most of the cases. Furthermore, the\nLexRank with threshold method outperforms the other degree-based techniques\nincluding continuous LexRank. We also show that our approach is quite\ninsensitive to the noise in the data that may result from an imperfect topical\nclustering of documents.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 22, pages\n  457-479, 2004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.2130v1",
            "date": "2011-09-09 20:22:04+00:00",
            "title": "Combining Knowledge- and Corpus-based Word-Sense-Disambiguation Methods",
            "authors": [
                "A. Montoyo",
                "M. Palomar",
                "G. Rigau",
                "A. Suarez"
            ],
            "abstract": "In this paper we concentrate on the resolution of the lexical ambiguity that\narises when a given word has several different meanings. This specific task is\ncommonly referred to as word sense disambiguation (WSD). The task of WSD\nconsists of assigning the correct sense to words using an electronic dictionary\nas the source of word definitions. We present two WSD methods based on two main\nmethodological approaches in this research area: a knowledge-based method and a\ncorpus-based method. Our hypothesis is that word-sense disambiguation requires\nseveral knowledge sources in order to solve the semantic ambiguity of the\nwords. These sources can be of different kinds--- for example, syntagmatic,\nparadigmatic or statistical information. Our approach combines various sources\nof knowledge, through combinations of the two WSD methods mentioned above.\nMainly, the paper concentrates on how to combine these methods and sources of\ninformation in order to achieve good results in the disambiguation. Finally,\nthis paper presents a comprehensive study and experimental work on evaluation\nof the methods and their combinations.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 23, pages\n  299-330, 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.2136v1",
            "date": "2011-09-09 20:24:57+00:00",
            "title": "Learning Content Selection Rules for Generating Object Descriptions in Dialogue",
            "authors": [
                "P. W. Jordan",
                "M. A. Walker"
            ],
            "abstract": "A fundamental requirement of any task-oriented dialogue system is the ability\nto generate object descriptions that refer to objects in the task domain. The\nsubproblem of content selection for object descriptions in task-oriented\ndialogue has been the focus of much previous work and a large number of models\nhave been proposed. In this paper, we use the annotated COCONUT corpus of\ntask-oriented design dialogues to develop feature sets based on Dale and\nReiters (1995) incremental model, Brennan and Clarks (1996) conceptual pact\nmodel, and Jordans (2000b) intentional influences model, and use these feature\nsets in a machine learning experiment to automatically learn a model of content\nselection for object descriptions. Since Dale and Reiters model requires a\nrepresentation of discourse structure, the corpus annotations are used to\nderive a representation based on Grosz and Sidners (1986) theory of the\nintentional structure of discourse, as well as two very simple representations\nof discourse structure based purely on recency. We then apply the\nrule-induction program RIPPER to train and test the content selection component\nof an object description generator on a set of 393 object descriptions from the\ncorpus. To our knowledge, this is the first reported experiment of a trainable\ncontent selection component for object description generation in dialogue.\nThree separate content selection models that are based on the three theoretical\nmodels, all independently achieve accuracies significantly above the majority\nclass baseline (17%) on unseen test data, with the intentional influences model\n(42.4%) performing significantly better than either the incremental model\n(30.4%) or the conceptual pact model (28.9%). But the best performing models\ncombine all the feature sets, achieving accuracies near 60%. Surprisingly, a\nsimple recency-based representation of discourse structure does as well as one\nbased on intentional structure. To our knowledge, this is also the first\nempirical comparison of a representation of Grosz and Sidners model of\ndiscourse structure with a simpler model for any generation task.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 24, pages\n  157-194, 2005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.4531v1",
            "date": "2011-09-21 13:57:49+00:00",
            "title": "A Probabilistic Approach to Pronunciation by Analogy",
            "authors": [
                "Janne V. Kujala",
                "Aleksi Keurulainen"
            ],
            "abstract": "The relationship between written and spoken words is convoluted in languages\nwith a deep orthography such as English and therefore it is difficult to devise\nexplicit rules for generating the pronunciations for unseen words.\nPronunciation by analogy (PbA) is a data-driven method of constructing\npronunciations for novel words from concatenated segments of known words and\ntheir pronunciations. PbA performs relatively well with English and outperforms\nseveral other proposed methods. However, the best published word accuracy of\n65.5% (for the 20,000 word NETtalk corpus) suggests there is much room for\nimprovement in it.\n  Previous PbA algorithms have used several different scoring strategies such\nas the product of the frequencies of the component pronunciations of the\nsegments, or the number of different segmentations that yield the same\npronunciation, and different combinations of these methods, to evaluate the\ncandidate pronunciations. In this article, we instead propose to use a\nprobabilistically justified scoring rule. We show that this principled approach\nalone yields better accuracy (66.21% for the NETtalk corpus) than any\npreviously published PbA algorithm. Furthermore, combined with certain ad hoc\nmodifications motivated by earlier algorithms, the performance climbs up to\n66.6%, and further improvements are possible by combining this method with\nother methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.4906v1",
            "date": "2011-09-22 18:37:17+00:00",
            "title": "Automatic transcription of 17th century English text in Contemporary English with NooJ: Method and Evaluation",
            "authors": [
                "Odile Piton",
                "Slim Mesfar",
                "H\u00e9l\u00e8ne Pignot"
            ],
            "abstract": "Since 2006 we have undertaken to describe the differences between 17th\ncentury English and contemporary English thanks to NLP software. Studying a\ncorpus spanning the whole century (tales of English travellers in the Ottoman\nEmpire in the 17th century, Mary Astell's essay A Serious Proposal to the\nLadies and other literary texts) has enabled us to highlight various lexical,\nmorphological or grammatical singularities. Thanks to the NooJ linguistic\nplatform, we created dictionaries indexing the lexical variants and their\ntranscription in CE. The latter is often the result of the validation of forms\nrecognized dynamically by morphological graphs. We also built syntactical\ngraphs aimed at transcribing certain archaic forms in contemporary English. Our\nprevious research implied a succession of elementary steps alternating textual\nanalysis and result validation. We managed to provide examples of\ntranscriptions, but we have not created a global tool for automatic\ntranscription. Therefore we need to focus on the results we have obtained so\nfar, study the conditions for creating such a tool, and analyze possible\ndifficulties. In this paper, we will be discussing the technical and linguistic\naspects we have not yet covered in our previous work. We are using the results\nof previous research and proposing a transcription method for words or\nsequences identified as archaic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.5798v1",
            "date": "2011-09-27 08:00:46+00:00",
            "title": "Object-oriented semantics of English in natural language understanding system",
            "authors": [
                "Yuriy Ostapov"
            ],
            "abstract": "A new approach to the problem of natural language understanding is proposed.\nThe knowledge domain under consideration is the social behavior of people.\nEnglish sentences are translated into set of predicates of a semantic database,\nwhich describe persons, occupations, organizations, projects, actions, events,\nmessages, machines, things, animals, location and time of actions, relations\nbetween objects, thoughts, cause-and-effect relations, abstract objects. There\nis a knowledge base containing the description of semantics of objects\n(functions and structure), actions (motives and causes), and operations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.1470v3",
            "date": "2011-10-07 09:53:41+00:00",
            "title": "A Constraint-Satisfaction Parser for Context-Free Grammars",
            "authors": [
                "Luis Quesada",
                "Fernando Berzal",
                "Francisco J. Cortijo"
            ],
            "abstract": "Traditional language processing tools constrain language designers to\nspecific kinds of grammars. In contrast, model-based language specification\ndecouples language design from language processing. As a consequence,\nmodel-based language specification tools need general parsers able to parse\nunrestricted context-free grammars. As languages specified following this\napproach may be ambiguous, parsers must deal with ambiguities. Model-based\nlanguage specification also allows the definition of associativity, precedence,\nand custom constraints. Therefore parsers generated by model-driven language\nspecification tools need to enforce constraints. In this paper, we propose\nFence, an efficient bottom-up chart parser with lexical and syntactic ambiguity\nsupport that allows the specification of constraints and, therefore, enables\nthe use of model-based language specification in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.1758v2",
            "date": "2011-10-08 19:15:12+00:00",
            "title": "Data formats for phonological corpora",
            "authors": [
                "Laurent Romary",
                "Andreas Witt"
            ],
            "abstract": "The goal of the present chapter is to explore the possibility of providing\nthe research (but also the industrial) community that commonly uses spoken\ncorpora with a stable portfolio of well-documented standardised formats that\nallow a high re-use rate of annotated spoken resources and, as a consequence,\nbetter interoperability across tools used to produce or exploit such resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.2215v1",
            "date": "2011-10-10 22:13:24+00:00",
            "title": "NP Animacy Identification for Anaphora Resolution",
            "authors": [
                "R. J. Evans",
                "C. Orasan"
            ],
            "abstract": "In anaphora resolution for English, animacy identification can play an\nintegral role in the application of agreement restrictions between pronouns and\ncandidates, and as a result, can improve the accuracy of anaphora resolution\nsystems. In this paper, two methods for animacy identification are proposed and\nevaluated using intrinsic and extrinsic measures. The first method is a\nrule-based one which uses information about the unique beginners in WordNet to\nclassify NPs on the basis of their animacy. The second method relies on a\nmachine learning algorithm which exploits a WordNet enriched with animacy\ninformation for each sense. The effect of word sense disambiguation on the two\nmethods is also assessed. The intrinsic evaluation reveals that the machine\nlearning method reaches human levels of performance. The extrinsic evaluation\ndemonstrates that animacy identification can be beneficial in anaphora\nresolution, especially in the cases where animate entities are identified with\nhigh precision.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 29, pages\n  79-103, 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.4248v1",
            "date": "2011-10-19 11:45:16+00:00",
            "title": "Ideogram Based Chinese Sentiment Word Orientation Computation",
            "authors": [
                "Luojie Xiang"
            ],
            "abstract": "This paper presents a novel algorithm to compute sentiment orientation of\nChinese sentiment word. The algorithm uses ideograms which are a distinguishing\nfeature of Chinese language. The proposed algorithm can be applied to any\nsentiment classification scheme. To compute a word's sentiment orientation\nusing the proposed algorithm, only the word itself and a precomputed character\nontology is required, rather than a corpus. The influence of three parameters\nover the algorithm performance is analyzed and verified by experiment.\nExperiment also shows that proposed algorithm achieves an F Measure of 85.02%\noutperforming existing ideogram based algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.0048v1",
            "date": "2011-10-31 21:53:32+00:00",
            "title": "Individual and Domain Adaptation in Sentence Planning for Dialogue",
            "authors": [
                "F. Mairesse",
                "R. Prasad",
                "A. Stent",
                "M. A. Walker"
            ],
            "abstract": "One of the biggest challenges in the development and deployment of spoken\ndialogue systems is the design of the spoken language generation module. This\nchallenge arises from the need for the generator to adapt to many features of\nthe dialogue domain, user population, and dialogue context. A promising\napproach is trainable generation, which uses general-purpose linguistic\nknowledge that is automatically adapted to the features of interest, such as\nthe application domain, individual user, or user group. In this paper we\npresent and evaluate a trainable sentence planner for providing restaurant\ninformation in the MATCH dialogue system. We show that trainable sentence\nplanning can produce complex information presentations whose quality is\ncomparable to the output of a template-based generator tuned to this domain. We\nalso show that our method easily supports adapting the sentence planner to\nindividuals, and that the individualized sentence planners generally perform\nbetter than models trained and tested on a population of individuals. Previous\nwork has documented and utilized individual preferences for content selection,\nbut to our knowledge, these results provide the first demonstration of\nindividual preferences for sentence planning operations, affecting the content\norder, discourse structure and sentence structure of system responses. Finally,\nwe evaluate the contribution of different feature sets, and show that, in our\napplication, n-gram features often do as well as features based on higher-level\nlinguistic representations.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 30, pages\n  413-456, 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.3122v1",
            "date": "2011-11-14 07:41:42+00:00",
            "title": "ESLO: from transcription to speakers' personal information annotation",
            "authors": [
                "Iris Eshkol",
                "D. Maurel",
                "Nathalie Friburger"
            ],
            "abstract": "This paper presents the preliminary works to put online a French oral corpus\nand its transcription. This corpus is the Socio-Linguistic Survey in Orleans,\nrealized in 1968. First, we numerized the corpus, then we handwritten\ntranscribed it with the Transcriber software adding different tags about\nspeakers, time, noise, etc. Each document (audio file and XML file of the\ntranscription) was described by a set of metadata stored in an XML format to\nallow an easy consultation. Second, we added different levels of annotations,\nrecognition of named entities and annotation of personal information about\nspeakers. This two annotation tasks used the CasSys system of transducer\ncascades. We used and modified a first cascade to recognize named entities.\nThen we built a second cascade to annote the designating entities, i.e.\ninformation about the speaker. These second cascade parsed the named entity\nannotated corpus. The objective is to locate information about the speaker and,\nalso, what kind of information can designate him/her. These two cascades was\nevaluated with precision and recall measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.3152v1",
            "date": "2011-11-14 09:34:34+00:00",
            "title": "\u00c9valuation de lexiques syntaxiques par leur int\u00e9gartion dans l'analyseur syntaxiques FRMG",
            "authors": [
                "Elsa Tolone",
                "\u00c9ric De La Clergerie",
                "Sagot Benoit"
            ],
            "abstract": "In this paper, we evaluate various French lexica with the parser FRMG: the\nLefff, LGLex, the lexicon built from the tables of the French Lexicon-Grammar,\nthe lexicon DICOVALENCE and a new version of the verbal entries of the Lefff,\nobtained by merging with DICOVALENCE and partial manual validation. For this,\nall these lexica have been converted to the format of the Lefff, Alexina\nformat. The evaluation was made on the part of the EASy corpus used in the\nfirst evaluation campaign Passage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.3153v1",
            "date": "2011-11-14 09:34:59+00:00",
            "title": "Construction du lexique LGLex \u00e0 partir des tables du Lexique-Grammaire des verbes du grec moderne",
            "authors": [
                "Kyriaki Ioannidou",
                "Elsa Tolone"
            ],
            "abstract": "In this paper, we summerize the work done on the resources of Modern Greek on\nthe Lexicon-Grammar of verbs. We detail the definitional features of each\ntable, and all changes made to the names of features to make them consistent.\nThrough the development of the table of classes, including all the features, we\nhave considered the conversion of tables in a syntactic lexicon: LGLex. The\nlexicon, in plain text format or XML, is generated by the LGExtract tool\n(Constant & Tolone, 2010). This format is directly usable in applications of\nNatural Language Processing (NLP).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.3462v1",
            "date": "2011-11-15 09:24:36+00:00",
            "title": "Extending the adverbial coverage of a NLP oriented resource for French",
            "authors": [
                "Elsa Tolone",
                "Voyatzi Stavroula"
            ],
            "abstract": "This paper presents a work on extending the adverbial entries of LGLex: a NLP\noriented syntactic resource for French. Adverbs were extracted from the\nLexicon-Grammar tables of both simple adverbs ending in -ment '-ly' (Molinier\nand Levrier, 2000) and compound adverbs (Gross, 1986; 1990). This work relies\non the exploitation of fine-grained linguistic information provided in existing\nresources. Various features are encoded in both LG tables and they haven't been\nexploited yet. They describe the relations of deleting, permuting, intensifying\nand paraphrasing that associate, on the one hand, the simple and compound\nadverbs and, on the other hand, different types of compound adverbs. The\nresulting syntactic resource is manually evaluated and freely available under\nthe LGPL-LR license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.4343v1",
            "date": "2011-11-18 12:31:49+00:00",
            "title": "Question Answering in a Natural Language Understanding System Based on Object-Oriented Semantics",
            "authors": [
                "Yuriy Ostapov"
            ],
            "abstract": "Algorithms of question answering in a computer system oriented on input and\nlogical processing of text information are presented. A knowledge domain under\nconsideration is social behavior of a person. A database of the system includes\nan internal representation of natural language sentences and supplemental\ninformation. The answer {\\it Yes} or {\\it No} is formed for a general question.\nA special question containing an interrogative word or group of interrogative\nwords permits to find a subject, object, place, time, cause, purpose and way of\naction or event. Answer generation is based on identification algorithms of\npersons, organizations, machines, things, places, and times. Proposed\nalgorithms of question answering can be realized in information systems closely\nconnected with text processing (criminology, operation of business, medicine,\ndocument systems).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.5293v1",
            "date": "2011-11-13 18:19:15+00:00",
            "title": "Rule based Part of speech Tagger for Homoeopathy Clinical realm",
            "authors": [
                "Sanjay K. Dwivedi",
                "Pramod P. Sukhadeve"
            ],
            "abstract": "A tagger is a mandatory segment of most text scrutiny systems, as it\nconsigned a s yntax class (e.g., noun, verb, adjective, and adverb) to every\nword in a sentence. In this paper, we present a simple part of speech tagger\nfor homoeopathy clinical language. This paper reports about the anticipated\npart of speech tagger for homoeopathy clinical language. It exploit standard\npattern for evaluating sentences, untagged clinical corpus of 20085 words is\nused, from which we had selected 125 sentences (2322 tokens). The problem of\ntagging in natural language processing is to find a way to tag every word in a\ntext as a meticulous part of speech. The basic idea is to apply a set of rules\non clinical sentences and on each word, Accuracy is the leading factor in\nevaluating any POS tagger so the accuracy of proposed tagger is also conversed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.6553v1",
            "date": "2011-11-28 19:17:57+00:00",
            "title": "Exploring Twitter Hashtags",
            "authors": [
                "Jan P\u00f6schko"
            ],
            "abstract": "Twitter messages often contain so-called hashtags to denote keywords related\nto them. Using a dataset of 29 million messages, I explore relations among\nthese hashtags with respect to co-occurrences. Furthermore, I present an\nattempt to classify hashtags into five intuitive classes, using a\nmachine-learning approach. The overall outcome is an interactive Web\napplication to explore Twitter hashtags.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1112.0168v1",
            "date": "2011-12-01 12:52:22+00:00",
            "title": "Statistical Sign Language Machine Translation: from English written text to American Sign Language Gloss",
            "authors": [
                "Achraf Othman",
                "Mohamed Jemni"
            ],
            "abstract": "This works aims to design a statistical machine translation from English text\nto American Sign Language (ASL). The system is based on Moses tool with some\nmodifications and the results are synthesized through a 3D avatar for\ninterpretation. First, we translate the input text to gloss, a written form of\nASL. Second, we pass the output to the WebSign Plug-in to play the sign.\nContributions of this work are the use of a new couple of language English/ASL\nand an improvement of statistical machine translation based on string matching\nthanks to Jaro-distance.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 8,\n  Issue 5, No 3, 2011, 65-73",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1112.0396v1",
            "date": "2011-12-02 07:15:49+00:00",
            "title": "Grammatical Relations of Myanmar Sentences Augmented by Transformation-Based Learning of Function Tagging",
            "authors": [
                "Win Win Thant",
                "Tin Myat Htwe",
                "Ni Lar Thein"
            ],
            "abstract": "In this paper we describe function tagging using Transformation Based\nLearning (TBL) for Myanmar that is a method of extensions to the previous\nstatistics-based function tagger. Contextual and lexical rules (developed using\nTBL) were critical in achieving good results. First, we describe a method for\nexpressing lexical relations in function tagging that statistical function\ntagging are currently unable to express. Function tagging is the preprocessing\nstep to show grammatical relations of the sentences. Then we use the context\nfree grammar technique to clarify the grammatical relations in Myanmar\nsentences or to output the parse trees. The grammatical relations are the\nfunctional structure of a language. They rely very much on the function tag of\nthe tokens. We augment the grammatical relations of Myanmar sentences with\ntransformation-based learning of function tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1112.2468v1",
            "date": "2011-12-12 08:07:49+00:00",
            "title": "Creating a Live, Public Short Message Service Corpus: The NUS SMS Corpus",
            "authors": [
                "Tao Chen",
                "Min-Yen Kan"
            ],
            "abstract": "Short Message Service (SMS) messages are largely sent directly from one\nperson to another from their mobile phones. They represent a means of personal\ncommunication that is an important communicative artifact in our current\ndigital era. As most existing studies have used private access to SMS corpora,\ncomparative studies using the same raw SMS data has not been possible up to\nnow. We describe our efforts to collect a public SMS corpus to address this\nproblem. We use a battery of methodologies to collect the corpus, paying\nparticular attention to privacy issues to address contributors' concerns. Our\nlive project collects new SMS message submissions, checks their quality and\nadds the valid messages, releasing the resultant corpus as XML and as SQL\ndumps, along with corpus statistics, every month. We opportunistically collect\nas much metadata about the messages and their sender as possible, so as to\nenable different types of analyses. To date, we have collected about 60,000\nmessages, focusing on English and Mandarin Chinese.",
            "journal ref": "Language Resources and Evaluation, Aug 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1112.6286v1",
            "date": "2011-12-29 11:47:05+00:00",
            "title": "Visualization and Analysis of Frames in Collections of Messages: Content Analysis and the Measurement of Meaning",
            "authors": [
                "Esther Vlieger",
                "Loet Leydesdorff"
            ],
            "abstract": "A step-to-step introduction is provided on how to generate a semantic map\nfrom a collection of messages (full texts, paragraphs or statements) using\nfreely available software and/or SPSS for the relevant statistics and the\nvisualization. The techniques are discussed in the various theoretical contexts\nof (i) linguistics (e.g., Latent Semantic Analysis), (ii) sociocybernetics and\nsocial systems theory (e.g., the communication of meaning), and (iii)\ncommunication studies (e.g., framing and agenda-setting). We distinguish\nbetween the communication of information in the network space (social network\nanalysis) and the communication of meaning in the vector space. The vector\nspace can be considered a generated as an architecture by the network of\nrelations in the network space; words are then not only related, but also\npositioned. These positions are expected rather than observed and therefore one\ncan communicate meaning. Knowledge can be generated when these meanings can\nrecursively be communicated and therefore also further codified.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1112.6384v1",
            "date": "2011-12-29 19:16:20+00:00",
            "title": "Proof nets for the Lambek-Grishin calculus",
            "authors": [
                "Michael Moortgat",
                "Richard Moot"
            ],
            "abstract": "Grishin's generalization of Lambek's Syntactic Calculus combines a\nnon-commutative multiplicative conjunction and its residuals (product, left and\nright division) with a dual family: multiplicative disjunction, right and left\ndifference. Interaction between these two families takes the form of linear\ndistributivity principles. We study proof nets for the Lambek-Grishin calculus\nand the correspondence between these nets and unfocused and focused versions of\nits sequent calculus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.0673v1",
            "date": "2011-06-03 14:54:46+00:00",
            "title": "Computational Approach to Anaphora Resolution in Spanish Dialogues",
            "authors": [
                "P. Martinez-Barco",
                "M. Palomar"
            ],
            "abstract": "This paper presents an algorithm for identifying noun-phrase antecedents of\npronouns and adjectival anaphors in Spanish dialogues. We believe that anaphora\nresolution requires numerous sources of information in order to find the\ncorrect antecedent of the anaphor. These sources can be of different kinds,\ne.g., linguistic information, discourse/dialogue structure information, or\ntopic information. For this reason, our algorithm uses various different kinds\nof information (hybrid information). The algorithm is based on linguistic\nconstraints and preferences and uses an anaphoric accessibility space within\nwhich the algorithm finds the noun phrase. We present some experiments related\nto this algorithm and this space using a corpus of 204 dialogues. The algorithm\nis implemented in Prolog. According to this study, 95.9% of antecedents were\nlocated in the proposed space, a precision of 81.3% was obtained for pronominal\nanaphora resolution, and 81.5% for adjectival anaphora.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 15, pages\n  263-287, 2001",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.5264v1",
            "date": "2011-06-26 21:05:14+00:00",
            "title": "Acquiring Correct Knowledge for Natural Language Generation",
            "authors": [
                "E. Reiter",
                "R. Robertson",
                "S. G. Sripada"
            ],
            "abstract": "Natural language generation (NLG) systems are computer software systems that\nproduce texts in English and other human languages, often from non-linguistic\ninput data. NLG systems, like most AI systems, need substantial amounts of\nknowledge. However, our experience in two NLG projects suggests that it is\ndifficult to acquire correct knowledge for NLG systems; indeed, every knowledge\nacquisition (KA) technique we tried had significant problems. In general terms,\nthese problems were due to the complexity, novelty, and poorly understood\nnature of the tasks our systems attempted, and were worsened by the fact that\npeople write so differently. This meant in particular that corpus-based KA\napproaches suffered because it was impossible to assemble a sizable corpus of\nhigh-quality consistent manually written texts in our domains; and structured\nexpert-oriented KA techniques suffered because experts disagreed and because we\ncould not get enough information about special and unusual cases to build\nrobust systems. We believe that such problems are likely to affect many other\nNLG systems as well. In the long term, we hope that new KA techniques may\nemerge to help NLG system builders. In the shorter term, we believe that\nunderstanding how individual KA techniques can fail, and using a mixture of\ndifferent KA techniques with different strengths and weaknesses, can help\ndevelopers acquire NLG knowledge that is mostly correct.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 18, pages\n  491-516, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.5973v1",
            "date": "2011-06-27 20:13:07+00:00",
            "title": "Entropy of Telugu",
            "authors": [
                "Venkata Ravinder Paruchuri"
            ],
            "abstract": "This paper presents an investigation of the entropy of the Telugu script.\nSince this script is syllabic, and not alphabetic, the computation of entropy\nis somewhat complicated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1101.4479v1",
            "date": "2011-01-24 09:52:39+00:00",
            "title": "A Context-theoretic Framework for Compositionality in Distributional Semantics",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "Techniques in which words are represented as vectors have proved useful in\nmany applications in computational linguistics, however there is currently no\ngeneral semantic formalism for representing meaning in terms of vectors. We\npresent a framework for natural language semantics in which words, phrases and\nsentences are all represented as vectors, based on a theoretical analysis which\nassumes that meaning is determined by context.\n  In the theoretical analysis, we define a corpus model as a mathematical\nabstraction of a text corpus. The meaning of a string of words is assumed to be\na vector representing the contexts in which it occurs in the corpus model.\nBased on this assumption, we can show that the vector representations of words\ncan be considered as elements of an algebra over a field. We note that in\napplications of vector spaces to representing meanings of words there is an\nunderlying lattice structure; we interpret the partial ordering of the lattice\nas describing entailment between meanings. We also define the context-theoretic\nprobability of a string, and, based on this and the lattice structure, a degree\nof entailment between strings.\n  We relate the framework to existing methods of composing vector-based\nrepresentations of meaning, and show that our approach generalises many of\nthese, including vector addition, component-wise multiplication, and the tensor\nproduct.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1102.2180v2",
            "date": "2011-02-10 17:16:50+00:00",
            "title": "Malagasy Dialects and the Peopling of Madagascar",
            "authors": [
                "M. Serva",
                "F. Petroni",
                "D. Volchenkov",
                "S. Wichmann"
            ],
            "abstract": "The origin of Malagasy DNA is half African and half Indonesian, nevertheless\nthe Malagasy language, spoken by the entire population, belongs to the\nAustronesian family. The language most closely related to Malagasy is Maanyan\n(Greater Barito East group of the Austronesian family), but related languages\nare also in Sulawesi, Malaysia and Sumatra. For this reason, and because\nMaanyan is spoken by a population which lives along the Barito river in\nKalimantan and which does not possess the necessary skill for long maritime\nnavigation, the ethnic composition of the Indonesian colonizers is still\nunclear.\n  There is a general consensus that Indonesian sailors reached Madagascar by a\nmaritime trek, but the time, the path and the landing area of the first\ncolonization are all disputed. In this research we try to answer these problems\ntogether with other ones, such as the historical configuration of Malagasy\ndialects, by types of analysis related to lexicostatistics and glottochronology\nwhich draw upon the automated method recently proposed by the authors\n\\cite{Serva:2008, Holman:2008, Petroni:2008, Bakker:2009}. The data were\ncollected by the first author at the beginning of 2010 with the invaluable help\nof Joselin\\`a Soafara N\\'er\\'e and consist of Swadesh lists of 200 items for 23\ndialects covering all areas of the Island.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1102.2831v2",
            "date": "2011-02-14 17:29:24+00:00",
            "title": "The effect of linguistic constraints on the large scale organization of language",
            "authors": [
                "Madhav Krishna",
                "Ahmed Hassan",
                "Yang Liu",
                "Dragomir Radev"
            ],
            "abstract": "This paper studies the effect of linguistic constraints on the large scale\norganization of language. It describes the properties of linguistic networks\nbuilt using texts of written language with the words randomized. These\nproperties are compared to those obtained for a network built over the text in\nnatural order. It is observed that the \"random\" networks too exhibit\nsmall-world and scale-free characteristics. They also show a high degree of\nclustering. This is indeed a surprising result - one that has not been\naddressed adequately in the literature. We hypothesize that many of the network\nstatistics reported here studied are in fact functions of the distribution of\nthe underlying data from which the network is built and may not be indicative\nof the nature of the concerned network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1102.5185v1",
            "date": "2011-02-25 08:13:10+00:00",
            "title": "Universal Higher Order Grammar",
            "authors": [
                "Victor Gluzberg"
            ],
            "abstract": "We examine the class of languages that can be defined entirely in terms of\nprovability in an extension of the sorted type theory (Ty_n) by embedding the\nlogic of phonologies, without introduction of special types for syntactic\nentities. This class is proven to precisely coincide with the class of\nlogically closed languages that may be thought of as functions from expressions\nto sets of logically equivalent Ty_n terms. For a specific sub-class of\nlogically closed languages that are described by finite sets of rules or rule\nschemata, we find effective procedures for building a compact Ty_n\nrepresentation, involving a finite number of axioms or axiom schemata. The\nproposed formalism is characterized by some useful features unavailable in a\ntwo-component architecture of a language model. A further specialization and\nextension of the formalism with a context type enable effective account of\nintensional and dynamic semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1103.0398v1",
            "date": "2011-03-02 11:34:50+00:00",
            "title": "Natural Language Processing (almost) from Scratch",
            "authors": [
                "Ronan Collobert",
                "Jason Weston",
                "Leon Bottou",
                "Michael Karlen",
                "Koray Kavukcuoglu",
                "Pavel Kuksa"
            ],
            "abstract": "We propose a unified neural network architecture and learning algorithm that\ncan be applied to various natural language processing tasks including:\npart-of-speech tagging, chunking, named entity recognition, and semantic role\nlabeling. This versatility is achieved by trying to avoid task-specific\nengineering and therefore disregarding a lot of prior knowledge. Instead of\nexploiting man-made input features carefully optimized for each task, our\nsystem learns internal representations on the basis of vast amounts of mostly\nunlabeled training data. This work is then used as a basis for building a\nfreely available tagging system with good performance and minimal computational\nrequirements.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1103.0890v2",
            "date": "2011-03-04 13:08:59+00:00",
            "title": "Efficient Multi-Template Learning for Structured Prediction",
            "authors": [
                "Qi Mao",
                "Ivor W. Tsang"
            ],
            "abstract": "Conditional random field (CRF) and Structural Support Vector Machine\n(Structural SVM) are two state-of-the-art methods for structured prediction\nwhich captures the interdependencies among output variables. The success of\nthese methods is attributed to the fact that their discriminative models are\nable to account for overlapping features on the whole input observations. These\nfeatures are usually generated by applying a given set of templates on labeled\ndata, but improper templates may lead to degraded performance. To alleviate\nthis issue, in this paper, we propose a novel multiple template learning\nparadigm to learn structured prediction and the importance of each template\nsimultaneously, so that hundreds of arbitrary templates could be added into the\nlearning model without caution. This paradigm can be formulated as a special\nmultiple kernel learning problem with exponential number of constraints. Then\nwe introduce an efficient cutting plane algorithm to solve this problem in the\nprimal, and its convergence is presented. We also evaluate the proposed\nlearning paradigm on two widely-studied structured prediction tasks,\n\\emph{i.e.} sequence labeling and dependency parsing. Extensive experimental\nresults show that the proposed method outperforms CRFs and Structural SVMs due\nto exploiting the importance of each template. Our complexity analysis and\nempirical results also show that our proposed method is more efficient than\nOnlineMKL on very sparse and high-dimensional data. We further extend this\nparadigm for structured prediction using generalized $p$-block norm\nregularization with $p>1$, and experiments show competitive performances when\n$p \\in [1,2)$.",
            "journal ref": "Efficient Multi-Template Learning for Structured Prediction. IEEE\n  Transactions on Neural Networks and Learning Systems, 24(2): 248 - 261, Feb\n  2013",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1103.2903v1",
            "date": "2011-03-15 13:39:20+00:00",
            "title": "A new ANEW: Evaluation of a word list for sentiment analysis in microblogs",
            "authors": [
                "Finn \u00c5rup Nielsen"
            ],
            "abstract": "Sentiment analysis of microblogs such as Twitter has recently gained a fair\namount of attention. One of the simplest sentiment analysis approaches compares\nthe words of a posting against a labeled word list, where each word has been\nscored for valence, -- a 'sentiment lexicon' or 'affective word lists'. There\nexist several affective word lists, e.g., ANEW (Affective Norms for English\nWords) developed before the advent of microblogging and sentiment analysis. I\nwanted to examine how well ANEW and other word lists performs for the detection\nof sentiment strength in microblog posts in comparison with a new word list\nspecifically constructed for microblogs. I used manually labeled postings from\nTwitter scored for sentiment. Using a simple word matching I show that the new\nword list may perform better than ANEW, though not as good as the more\nelaborate approach found in SentiStrength.",
            "journal ref": "Proceedings of the ESWC2011 Workshop on 'Making Sense of\n  Microposts': Big things come in small packages (2011) 93-98",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1104.4426v3",
            "date": "2011-04-22 12:20:56+00:00",
            "title": "Phylogeny and geometry of languages from normalized Levenshtein distance",
            "authors": [
                "Maurizio Serva"
            ],
            "abstract": "The idea that the distance among pairs of languages can be evaluated from\nlexical differences seems to have its roots in the work of the French explorer\nDumont D'Urville. He collected comparative words lists of various languages\nduring his voyages aboard the Astrolabe from 1826 to 1829 and, in his work\nabout the geographical division of the Pacific, he proposed a method to measure\nthe degree of relation between languages.\n  The method used by the modern lexicostatistics, developed by Morris Swadesh\nin the 1950s, measures distances from the percentage of shared cognates, which\nare words with a common historical origin. The weak point of this method is\nthat subjective judgment plays a relevant role.\n  Recently, we have proposed a new automated method which is motivated by the\nanalogy with genetics. The new approach avoids any subjectivity and results can\nbe easily replicated by other scholars. The distance between two languages is\ndefined by considering a renormalized Levenshtein distance between pair of\nwords with the same meaning and averaging on the words contained in a list. The\nrenormalization, which takes into account the length of the words, plays a\ncrucial role, and no sensible results can be found without it.\n  In this paper we give a short review of our automated method and we\nillustrate it by considering the cluster of Malagasy dialects. We show that it\nsheds new light on their kinship relation and also that it furnishes a lot of\nnew information concerning the modalities of the settlement of Madagascar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1104.4950v1",
            "date": "2011-04-26 15:36:47+00:00",
            "title": "A Machine Learning Based Analytical Framework for Semantic Annotation Requirements",
            "authors": [
                "Hamed Hassanzadeh",
                "MohammadReza Keyvanpour"
            ],
            "abstract": "The Semantic Web is an extension of the current web in which information is\ngiven well-defined meaning. The perspective of Semantic Web is to promote the\nquality and intelligence of the current web by changing its contents into\nmachine understandable form. Therefore, semantic level information is one of\nthe cornerstones of the Semantic Web. The process of adding semantic metadata\nto web resources is called Semantic Annotation. There are many obstacles\nagainst the Semantic Annotation, such as multilinguality, scalability, and\nissues which are related to diversity and inconsistency in content of different\nweb pages. Due to the wide range of domains and the dynamic environments that\nthe Semantic Annotation systems must be performed on, the problem of automating\nannotation process is one of the significant challenges in this domain. To\novercome this problem, different machine learning approaches such as supervised\nlearning, unsupervised learning and more recent ones like, semi-supervised\nlearning and active learning have been utilized. In this paper we present an\ninclusive layered classification of Semantic Annotation challenges and discuss\nthe most important issues in this field. Also, we review and analyze machine\nlearning applications for solving semantic annotation problems. For this goal,\nthe article tries to closely study and categorize related researches for better\nunderstanding and to reach a framework that can map machine learning techniques\ninto the Semantic Annotation challenges and requirements.",
            "journal ref": "International Journal of Web & Semantic Technology (IJWesT), Vol.\n  2, No. 2, pp. 27-38, Aprill 2011",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1104.5362v2",
            "date": "2011-04-28 11:44:33+00:00",
            "title": "Selected Operations, Algorithms, and Applications of n-Tape Weighted Finite-State Machines",
            "authors": [
                "Andr\u00e9 Kempe"
            ],
            "abstract": "A weighted finite-state machine with n tapes (n-WFSM) defines a rational\nrelation on n strings. It is a generalization of weighted acceptors (one tape)\nand transducers (two tapes).\n  After recalling some basic definitions about n-ary weighted rational\nrelations and n-WFSMs, we summarize some central operations on these relations\nand machines, such as join and auto-intersection. Unfortunately, due to Post's\nCorrespondence Problem, a fully general join or auto-intersection algorithm\ncannot exist. We recall a restricted algorithm for a class of n-WFSMs.\n  Through a series of practical applications, we finally investigate the\naugmented descriptive power of n-WFSMs and their join, compared to classical\ntransducers and their composition. Some applications are not feasible with the\nlatter. The series includes: the morphological analysis of Semitic languages,\nthe preservation of intermediate results in transducer cascades, the induction\nof morphological rules from corpora, the alignment of lexicon entries, the\nautomatic extraction of acronyms and their meaning from corpora, and the search\nfor cognates in a bilingual lexicon.\n  All described operations and applications have been implemented with Xerox's\nWFSC tool.",
            "journal ref": "Proc. FSMNLP 2009, Pretoria, South Africa. July 21-24. (invited\n  talk)",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1105.0673v1",
            "date": "2011-05-03 20:00:05+00:00",
            "title": "Mark My Words! Linguistic Style Accommodation in Social Media",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Michael Gamon",
                "Susan Dumais"
            ],
            "abstract": "The psycholinguistic theory of communication accommodation accounts for the\ngeneral observation that participants in conversations tend to converge to one\nanother's communicative behavior: they coordinate in a variety of dimensions\nincluding choice of words, syntax, utterance length, pitch and gestures. In its\nalmost forty years of existence, this theory has been empirically supported\nexclusively through small-scale or controlled laboratory studies. Here we\naddress this phenomenon in the context of Twitter conversations. Undoubtedly,\nthis setting is unlike any other in which accommodation was observed and, thus,\nchallenging to the theory. Its novelty comes not only from its size, but also\nfrom the non real-time nature of conversations, from the 140 character length\nrestriction, from the wide variety of social relation types, and from a design\nthat was initially not geared towards conversation at all. Given such\nconstraints, it is not clear a priori whether accommodation is robust enough to\noccur given the constraints of this new environment. To investigate this, we\ndevelop a probabilistic framework that can model accommodation and measure its\neffects. We apply it to a large Twitter conversational dataset specifically\ndeveloped for this task. This is the first time the hypothesis of linguistic\nstyle accommodation has been examined (and verified) in a large scale, real\nworld setting. Furthermore, when investigating concepts such as stylistic\ninfluence and symmetry of accommodation, we discover a complexity of the\nphenomenon which was never observed before. We also explore the potential\nrelation between stylistic influence and network features commonly associated\nwith social status.",
            "journal ref": "Proceedings of WWW, pp. 141--150, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1105.1702v2",
            "date": "2011-05-09 15:45:03+00:00",
            "title": "A Compositional Distributional Semantics, Two Concrete Constructions, and some Experimental Evaluations",
            "authors": [
                "Mehrnoosh Sadrzadeh",
                "Edward Grefenstette"
            ],
            "abstract": "We provide an overview of the hybrid compositional distributional model of\nmeaning, developed in Coecke et al. (arXiv:1003.4394v1 [cs.CL]), which is based\non the categorical methods also applied to the analysis of information flow in\nquantum protocols. The mathematical setting stipulates that the meaning of a\nsentence is a linear function of the tensor products of the meanings of its\nwords. We provide concrete constructions for this definition and present\ntechniques to build vector spaces for meaning vectors of words, as well as that\nof sentences. The applicability of these methods is demonstrated via a toy\nvector space as well as real data from the British National Corpus and two\ndisambiguation experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1105.4582v1",
            "date": "2011-05-23 18:49:02+00:00",
            "title": "Perception of Personality and Naturalness through Dialogues by Native Speakers of American English and Arabic",
            "authors": [
                "Maxim Makatchev",
                "Reid Simmons"
            ],
            "abstract": "Linguistic markers of personality traits have been studied extensively, but\nfew cross-cultural studies exist. In this paper, we evaluate how native\nspeakers of American English and Arabic perceive personality traits and\nnaturalness of English utterances that vary along the dimensions of verbosity,\nhedging, lexical and syntactic alignment, and formality. The utterances are the\nturns within dialogue fragments that are presented as text transcripts to the\nworkers of Amazon's Mechanical Turk. The results of the study suggest that all\nfour dimensions can be used as linguistic markers of all personality traits by\nboth language communities. A further comparative analysis shows cross-cultural\ndifferences for some combinations of measures of personality traits and\nnaturalness, the dimensions of linguistic variability and dialogue acts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.3119v2",
            "date": "2011-07-15 18:06:37+00:00",
            "title": "Experimenting with Transitive Verbs in a DisCoCat",
            "authors": [
                "Edward Grefenstette",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Formal and distributional semantic models offer complementary benefits in\nmodeling meaning. The categorical compositional distributional (DisCoCat) model\nof meaning of Coecke et al. (arXiv:1003.4394v1 [cs.CL]) combines aspected of\nboth to provide a general framework in which meanings of words, obtained\ndistributionally, are composed using methods from the logical setting to form\nsentence meaning. Concrete consequences of this general abstract setting and\napplications to empirical data are under active study (Grefenstette et al.,\narxiv:1101.0309; Grefenstette and Sadrzadeh, arXiv:1106.4058v1 [cs.CL]). . In\nthis paper, we extend this study by examining transitive verbs, represented as\nmatrices in a DisCoCat. We discuss three ways of constructing such matrices,\nand evaluate each method in a disambiguation task developed by Grefenstette and\nSadrzadeh (arXiv:1106.4058v1 [cs.CL]).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4218v1",
            "date": "2011-07-21 10:02:31+00:00",
            "title": "The settlement of Madagascar: what dialects and languages can tell",
            "authors": [
                "Maurizio Serva"
            ],
            "abstract": "The dialects of Madagascar belong to the Greater Barito East group of the\nAustronesian family and it is widely accepted that the Island was colonized by\nIndonesian sailors after a maritime trek which probably took place around 650\nCE. The language most closely related to Malagasy dialects is Maanyan but also\nMalay is strongly related especially for what concerns navigation terms. Since\nthe Maanyan Dayaks live along the Barito river in Kalimantan (Borneo) and they\ndo not possess the necessary skill for long maritime navigation, probably they\nwere brought as subordinates by Malay sailors.\n  In a recent paper we compared 23 different Malagasy dialects in order to\ndetermine the time and the landing area of the first colonization. In this\nresearch we use new data and new methods to confirm that the landing took place\non the south-east coast of the Island. Furthermore, we are able to state here\nthat it is unlikely that there were multiple settlements and, therefore,\ncolonization consisted in a single founding event.\n  To reach our goal we find out the internal kinship relations among all the 23\nMalagasy dialects and we also find out the different kinship degrees of the 23\ndialects versus Malay and Maanyan. The method used is an automated version of\nthe lexicostatistic approach. The data concerning Madagascar were collected by\nthe author at the beginning of 2010 and consist of Swadesh lists of 200 items\nfor 23 dialects covering all areas of the Island. The lists for Maanyan and\nMalay were obtained from published datasets integrated by author's interviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1107.4557v1",
            "date": "2011-07-22 16:02:06+00:00",
            "title": "Finding Deceptive Opinion Spam by Any Stretch of the Imagination",
            "authors": [
                "Myle Ott",
                "Yejin Choi",
                "Claire Cardie",
                "Jeffrey T. Hancock"
            ],
            "abstract": "Consumers increasingly rate, review and research products online.\nConsequently, websites containing consumer reviews are becoming targets of\nopinion spam. While recent work has focused primarily on manually identifiable\ninstances of opinion spam, in this work we study deceptive opinion\nspam---fictitious opinions that have been deliberately written to sound\nauthentic. Integrating work from psychology and computational linguistics, we\ndevelop and compare three approaches to detecting deceptive opinion spam, and\nultimately develop a classifier that is nearly 90% accurate on our\ngold-standard opinion spam dataset. Based on feature analysis of our learned\nmodels, we additionally make several theoretical contributions, including\nrevealing a relationship between deceptive opinions and imaginative writing.",
            "journal ref": "Proceedings of ACL 2011: HLT, pp. 309-319",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.4297v1",
            "date": "2011-08-22 12:55:20+00:00",
            "title": "Why is language well-designed for communication? (Commentary on Christiansen and Chater: 'Language as shaped by the brain')",
            "authors": [
                "Jean-Louis Dessalles"
            ],
            "abstract": "Selection through iterated learning explains no more than other\nnon-functional accounts, such as universal grammar, why language is so\nwell-designed for communicative efficiency. It does not predict several\ndistinctive features of language like central embedding, large lexicons or the\nlack of iconicity, that seem to serve communication purposes at the expense of\nlearnability.",
            "journal ref": "Behavioral and Brain Sciences 31, 5 (2008) 518-519",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5016v1",
            "date": "2011-08-25 06:07:06+00:00",
            "title": "Une analyse bas\u00e9e sur la S-DRT pour la mod\u00e9lisation de dialogues pathologiques",
            "authors": [
                "Maxime Amblard",
                "Musiol Michel",
                "Rebuschi Manuel"
            ],
            "abstract": "In this article, we present a corpus of dialogues between a schizophrenic\nspeaker and an interlocutor who drives the dialogue. We had identified specific\ndiscontinuities for paranoid schizophrenics. We propose a modeling of these\ndiscontinuities with S-DRT (its pragmatic part)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5192v3",
            "date": "2011-08-25 20:26:22+00:00",
            "title": "Positivity of the English language",
            "authors": [
                "Isabel M. Kloumann",
                "Christopher M. Danforth",
                "Kameron Decker Harris",
                "Catherine A. Bliss",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Over the last million years, human language has emerged and evolved as a\nfundamental instrument of social communication and semiotic representation.\nPeople use language in part to convey emotional information, leading to the\ncentral and contingent questions: (1) What is the emotional spectrum of natural\nlanguage? and (2) Are natural languages neutrally, positively, or negatively\nbiased? Here, we report that the human-perceived positivity of over 10,000 of\nthe most frequently used English words exhibits a clear positive bias. More\ndeeply, we characterize and quantify distributions of word positivity for four\nlarge and distinct corpora, demonstrating that their form is broadly invariant\nwith respect to frequency of word use.",
            "journal ref": "PLoS ONE, Vol 7, e29484, 2012",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1108.5567v1",
            "date": "2011-08-29 14:27:04+00:00",
            "title": "Parsing Combinatory Categorial Grammar with Answer Set Programming: Preliminary Report",
            "authors": [
                "Yuliya Lierler",
                "Peter Sch\u00fcller"
            ],
            "abstract": "Combinatory categorial grammar (CCG) is a grammar formalism used for natural\nlanguage parsing. CCG assigns structured lexical categories to words and uses a\nsmall set of combinatory rules to combine these categories to parse a sentence.\nIn this work we propose and implement a new approach to CCG parsing that relies\non a prominent knowledge representation formalism, answer set programming (ASP)\n- a declarative programming paradigm. We formulate the task of CCG parsing as a\nplanning problem and use an ASP computational tool to compute solutions that\ncorrespond to valid parses. Compared to other approaches, there is no need to\nimplement a specific parsing algorithm using such a declarative method. Our\napproach aims at producing all semantically distinct parse trees for a given\nsentence. From this goal, normalization and efficiency issues arise, and we\ndeal with them by combining and extending existing strategies. We have\nimplemented a CCG parsing tool kit - AspCcgTk - that uses ASP as its main\ncomputational means. The C&C supertagger can be used as a preprocessor within\nAspCcgTk, which allows us to achieve wide-coverage natural language parsing.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1109.6341v1",
            "date": "2011-09-28 20:18:30+00:00",
            "title": "Domain Adaptation for Statistical Classifiers",
            "authors": [
                "H. Daume III",
                "D. Marcu"
            ],
            "abstract": "The most basic assumption used in statistical learning theory is that\ntraining data and test data are drawn from the same underlying distribution.\nUnfortunately, in many applications, the \"in-domain\" test data is drawn from a\ndistribution that is related, but not identical, to the \"out-of-domain\"\ndistribution of the training data. We consider the common case in which labeled\nout-of-domain data is plentiful, but labeled in-domain data is scarce. We\nintroduce a statistical formulation of this problem in terms of a simple\nmixture model and present an instantiation of this framework to maximum entropy\nclassifiers and their linear chain counterparts. We present efficient inference\nalgorithms for this special case based on the technique of conditional\nexpectation maximization. Our experimental results show that our approach leads\nto improved performance on three real world tasks on four different data sets\nfrom the natural language processing domain.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 26, pages\n  101-126, 2006",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1110.1391v1",
            "date": "2011-10-06 20:48:58+00:00",
            "title": "A Comparison of Different Machine Transliteration Models",
            "authors": [
                "K. Choi",
                "H. Isahara",
                "J. Oh"
            ],
            "abstract": "Machine transliteration is a method for automatically converting words in one\nlanguage into phonetically equivalent ones in another language. Machine\ntransliteration plays an important role in natural language applications such\nas information retrieval and machine translation, especially for handling\nproper nouns and technical terms. Four machine transliteration models --\ngrapheme-based transliteration model, phoneme-based transliteration model,\nhybrid transliteration model, and correspondence-based transliteration model --\nhave been proposed by several researchers. To date, however, there has been\nlittle research on a framework in which multiple transliteration models can\noperate simultaneously. Furthermore, there has been no comparison of the four\nmodels within the same framework and using the same data. We addressed these\nproblems by 1) modeling the four models within the same framework, 2) comparing\nthem under the same conditions, and 3) developing a way to improve machine\ntransliteration through this comparison. Our comparison showed that the hybrid\nand correspondence-based models were the most effective and that the four\nmodels can be used in a complementary manner to improve machine transliteration\nperformance.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 27, pages\n  119-151, 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.1394v1",
            "date": "2011-10-06 20:55:54+00:00",
            "title": "Learning Sentence-internal Temporal Relations",
            "authors": [
                "M. Lapata",
                "A. Lascarides"
            ],
            "abstract": "In this paper we propose a data intensive approach for inferring\nsentence-internal temporal relations. Temporal inference is relevant for\npractical NLP applications which either extract or synthesize temporal\ninformation (e.g., summarisation, question answering). Our method bypasses the\nneed for manual coding by exploiting the presence of markers like after\", which\novertly signal a temporal relation. We first show that models trained on main\nand subordinate clauses connected with a temporal marker achieve good\nperformance on a pseudo-disambiguation task simulating temporal inference\n(during testing the temporal marker is treated as unseen and the models must\nselect the right marker from a set of possible candidates). Secondly, we assess\nwhether the proposed approach holds promise for the semi-automatic creation of\ntemporal annotations. Specifically, we use a model trained on noisy and\napproximate data (i.e., main and subordinate clauses) to predict\nintra-sentential relations present in TimeBank, a corpus annotated rich\ntemporal information. Our experiments compare and contrast several\nprobabilistic models differing in their feature space, linguistic assumptions\nand data requirements. We evaluate performance against gold standard corpora\nand also against human subjects.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 27, pages\n  85-117, 2006",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.1428v1",
            "date": "2011-10-07 04:56:43+00:00",
            "title": "Product Review Summarization based on Facet Identification and Sentence Clustering",
            "authors": [
                "Duy Khang Ly",
                "Kazunari Sugiyama",
                "Ziheng Lin",
                "Min-Yen Kan"
            ],
            "abstract": "Product review nowadays has become an important source of information, not\nonly for customers to find opinions about products easily and share their\nreviews with peers, but also for product manufacturers to get feedback on their\nproducts. As the number of product reviews grows, it becomes difficult for\nusers to search and utilize these resources in an efficient way. In this work,\nwe build a product review summarization system that can automatically process a\nlarge collection of reviews and aggregate them to generate a concise summary.\nMore importantly, the drawback of existing product summarization systems is\nthat they cannot provide the underlying reasons to justify users' opinions. In\nour method, we solve this problem by applying clustering, prior to selecting\nrepresentative candidates for summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.1648v1",
            "date": "2011-11-07 17:11:50+00:00",
            "title": "Sentiment Analysis of Document Based on Annotation",
            "authors": [
                "Archana Shukla"
            ],
            "abstract": "I present a tool which tells the quality of document or its usefulness based\non annotations. Annotation may include comments, notes, observation,\nhighlights, underline, explanation, question or help etc. comments are used for\nevaluative purpose while others are used for summarization or for expansion\nalso. Further these comments may be on another annotation. Such annotations are\nreferred as meta-annotation. All annotation may not get equal weightage. My\ntool considered highlights, underline as well as comments to infer the\ncollective sentiment of annotators. Collective sentiments of annotators are\nclassified as positive, negative, objectivity. My tool computes collective\nsentiment of annotations in two manners. It counts all the annotation present\non the documents as well as it also computes sentiment scores of all annotation\nwhich includes comments to obtain the collective sentiments about the document\nor to judge the quality of document. I demonstrate the use of tool on research\npaper.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1111.1673v1",
            "date": "2011-11-07 18:46:54+00:00",
            "title": "Algebras over a field and semantics for context based reasoning",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "This paper introduces context algebras and demonstrates their application to\ncombining logical and vector-based representations of meaning. Other approaches\nto this problem attempt to reproduce aspects of logical semantics within new\nframeworks. The approach we present here is different: We show how logical\nsemantics can be embedded within a vector space framework, and use this to\ncombine distributional semantics, in which the meanings of words are\nrepresented as vectors, with logical semantics, in which the meaning of a\nsentence is represented as a logical form.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.2399v1",
            "date": "2011-11-10 06:08:12+00:00",
            "title": "Genetic Algorithm (GA) in Feature Selection for CRF Based Manipuri Multiword Expression (MWE) Identification",
            "authors": [
                "Kishorjit Nongmeikapam",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "This paper deals with the identification of Multiword Expressions (MWEs) in\nManipuri, a highly agglutinative Indian Language. Manipuri is listed in the\nEight Schedule of Indian Constitution. MWE plays an important role in the\napplications of Natural Language Processing(NLP) like Machine Translation, Part\nof Speech tagging, Information Retrieval, Question Answering etc. Feature\nselection is an important factor in the recognition of Manipuri MWEs using\nConditional Random Field (CRF). The disadvantage of manual selection and\nchoosing of the appropriate features for running CRF motivates us to think of\nGenetic Algorithm (GA). Using GA we are able to find the optimal features to\nrun the CRF. We have tried with fifty generations in feature selection along\nwith three fold cross validation as fitness function. This model demonstrated\nthe Recall (R) of 64.08%, Precision (P) of 86.84% and F-measure (F) of 73.74%,\nshowing an improvement over the CRF based Manipuri MWE identification without\nGA application.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 3, No 5, Oct 2011, pp 53-66",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1111.4316v1",
            "date": "2011-11-18 10:13:54+00:00",
            "title": "Semantic Navigation on the Web of Data: Specification of Routes, Web Fragments and Actions",
            "authors": [
                "Valeria Fionda",
                "Claudio Gutierrez",
                "Giuseppe Pirr\u00f3"
            ],
            "abstract": "The massive semantic data sources linked in the Web of Data give new meaning\nto old features like navigation; introduce new challenges like semantic\nspecification of Web fragments; and make it possible to specify actions relying\non semantic data. In this paper we introduce a declarative language to face\nthese challenges. Based on navigational features, it is designed to specify\nfragments of the Web of Data and actions to be performed based on these data.\nWe implement it in a centralized fashion, and show its power and performance.\nFinally, we explore the same ideas in a distributed setting, showing their\nfeasibility, potentialities and challenges.",
            "journal ref": "None",
            "category": "cs.NI"
        },
        {
            "url": "http://arxiv.org/abs/1111.7190v3",
            "date": "2011-11-29 15:30:24+00:00",
            "title": "Developing Embodied Multisensory Dialogue Agents",
            "authors": [
                "Micha\u0142 B. Paradowski"
            ],
            "abstract": "A few decades of work in the AI field have focused efforts on developing a\nnew generation of systems which can acquire knowledge via interaction with the\nworld. Yet, until very recently, most such attempts were underpinned by\nresearch which predominantly regarded linguistic phenomena as separated from\nthe brain and body. This could lead one into believing that to emulate\nlinguistic behaviour, it suffices to develop 'software' operating on abstract\nrepresentations that will work on any computational machine. This picture is\ninaccurate for several reasons, which are elucidated in this paper and extend\nbeyond sensorimotor and semantic resonance. Beginning with a review of\nresearch, I list several heterogeneous arguments against disembodied language,\nin an attempt to draw conclusions for developing embodied multisensory agents\nwhich communicate verbally and non-verbally with their environment. Without\ntaking into account both the architecture of the human brain, and embodiment,\nit is unrealistic to replicate accurately the processes which take place during\nlanguage acquisition, comprehension, production, or during non-linguistic\nactions. While robots are far from isomorphic with humans, they could benefit\nfrom strengthened associative connections in the optimization of their\nprocesses and their reactivity and sensitivity to environmental stimuli, and in\nsituated human-machine interaction. The concept of multisensory integration\nshould be extended to cover linguistic input and the complementary information\ncombined from temporally coincident sensory impressions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1106.0107v1",
            "date": "2011-06-01 07:22:04+00:00",
            "title": "Handwritten Character Recognition of South Indian Scripts: A Review",
            "authors": [
                "John Jomy",
                "K. V. Pramod",
                "Balakrishnan Kannan"
            ],
            "abstract": "Handwritten character recognition is always a frontier area of research in\nthe field of pattern recognition and image processing and there is a large\ndemand for OCR on hand written documents. Even though, sufficient studies have\nperformed in foreign scripts like Chinese, Japanese and Arabic characters, only\na very few work can be traced for handwritten character recognition of Indian\nscripts especially for the South Indian scripts. This paper provides an\noverview of offline handwritten character recognition in South Indian Scripts,\nnamely Malayalam, Tamil, Kannada and Telungu.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1106.0411v1",
            "date": "2011-06-02 12:14:33+00:00",
            "title": "Quantum-Like Uncertain Conditionals for Text Analysis",
            "authors": [
                "Alvaro Francisco Huertas-Rosero",
                "C. J. van Rijsbergen"
            ],
            "abstract": "Simple representations of documents based on the occurrences of terms are\nubiquitous in areas like Information Retrieval, and also frequent in Natural\nLanguage Processing. In this work we propose a logical-probabilistic approach\nto the analysis of natural language text based in the concept of Uncertain\nConditional, on top of a formulation of lexical measurements inspired in the\ntheoretical concept of ideal quantum measurements. The proposed concept can be\nused for generating topic-specific representations of text, aiming to match in\na simple way the perception of a user with a pre-established idea of what the\nusage of terms in the text should be. A simple example is developed with two\nversions of a text in two languages, showing how regularities in the use of\nterms are detected and easily represented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.3077v1",
            "date": "2011-06-15 20:00:08+00:00",
            "title": "Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee"
            ],
            "abstract": "Conversational participants tend to immediately and unconsciously adapt to\neach other's language styles: a speaker will even adjust the number of articles\nand other function words in their next utterance in response to the number in\ntheir partner's immediately preceding utterance. This striking level of\ncoordination is thought to have arisen as a way to achieve social goals, such\nas gaining approval or emphasizing difference in status. But has the adaptation\nmechanism become so deeply embedded in the language-generation process as to\nbecome a reflex? We argue that fictional dialogs offer a way to study this\nquestion, since authors create the conversations but don't receive the social\nbenefits (rather, the imagined characters do). Indeed, we find significant\ncoordination across many families of function words in our large movie-script\ncorpus. We also report suggestive preliminary findings on the effects of gender\nand other features; e.g., surprisingly, for articles, on average, characters\nadapt more to females than to males.",
            "journal ref": "Proceedings of the ACL workshop on Cognitive Modeling and\n  Computational Linguistics, pp 76-87, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.4058v1",
            "date": "2011-06-20 23:23:11+00:00",
            "title": "Experimental Support for a Categorical Compositional Distributional Model of Meaning",
            "authors": [
                "Edward Grefenstette",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Modelling compositional meaning for sentences using empirical distributional\nmethods has been a challenge for computational linguists. We implement the\nabstract categorical model of Coecke et al. (arXiv:1003.4394v1 [cs.CL]) using\ndata from the BNC and evaluate it. The implementation is based on unsupervised\nlearning of matrices for relational words and applying them to the vectors of\ntheir arguments. The evaluation is based on the word disambiguation task\ndeveloped by Mitchell and Lapata (2008) for intransitive sentences, and on a\nsimilar new experiment designed for transitive sentences. Our model matches the\nresults of its competitors in the first experiment, and betters them in the\nsecond. The general improvement in results with increase in syntactic\ncomplexity showcases the compositional power of our model.",
            "journal ref": "Proceedings of the 2011 Conference on Empirical Methods in Natural\n  Language Processing (2011)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.4571v1",
            "date": "2011-06-22 20:57:18+00:00",
            "title": "Acquiring Word-Meaning Mappings for Natural Language Interfaces",
            "authors": [
                "C. Thompson"
            ],
            "abstract": "This paper focuses on a system, WOLFIE (WOrd Learning From Interpreted\nExamples), that acquires a semantic lexicon from a corpus of sentences paired\nwith semantic representations. The lexicon learned consists of phrases paired\nwith meaning representations. WOLFIE is part of an integrated system that\nlearns to transform sentences into representations such as logical database\nqueries. Experimental results are presented demonstrating WOLFIE's ability to\nlearn useful lexicons for a database interface in four different natural\nlanguages. The usefulness of the lexicons learned by WOLFIE are compared to\nthose acquired by a similar system, with results favorable to WOLFIE. A second\nset of experiments demonstrates WOLFIE's ability to scale to larger and more\ndifficult, albeit artificially generated, corpora. In natural language\nacquisition, it is difficult to gather the annotated data needed for supervised\nlearning; however, unannotated data is fairly plentiful. Active learning\nmethods attempt to select for annotation and training only the most informative\nexamples, and therefore are potentially very useful in natural language\napplications. However, most results to date for active learning have only\nconsidered standard classification tasks. To reduce annotation effort while\nmaintaining accuracy, we apply active learning to semantic lexicons. We show\nthat active learning can significantly reduce the number of annotated examples\nrequired to achieve a given level of performance.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 18, pages\n  1-44, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.4862v1",
            "date": "2011-06-24 00:55:35+00:00",
            "title": "Translation of Pronominal Anaphora between English and Spanish: Discrepancies and Evaluation",
            "authors": [
                "A. Ferrandez",
                "J. Peral"
            ],
            "abstract": "This paper evaluates the different tasks carried out in the translation of\npronominal anaphora in a machine translation (MT) system. The MT interlingua\napproach named AGIR (Anaphora Generation with an Interlingua Representation)\nimproves upon other proposals presented to date because it is able to translate\nintersentential anaphors, detect co-reference chains, and translate Spanish\nzero pronouns into English---issues hardly considered by other systems. The\npaper presents the resolution and evaluation of these anaphora problems in AGIR\nwith the use of different kinds of knowledge (lexical, morphological,\nsyntactic, and semantic). The translation of English and Spanish anaphoric\nthird-person personal pronouns (including Spanish zero pronouns) into the\ntarget language has been evaluated on unrestricted corpora. We have obtained a\nprecision of 80.4% and 84.8% in the translation of Spanish and English\npronouns, respectively. Although we have only studied the Spanish and English\nlanguages, our approach can be easily extended to other languages such as\nPortuguese, Italian, or Japanese.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 18, pages\n  117-147, 2003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1106.5308v1",
            "date": "2011-06-27 06:24:34+00:00",
            "title": "Clasificarea distribuita a mesajelor de e-mail",
            "authors": [
                "Florin Pop",
                "Diana Petrescu",
                "\u015etefan Trau\u015fan-Matu"
            ],
            "abstract": "A basic component in Internet applications is the electronic mail and its\nvarious implications. The paper proposes a mechanism for automatically\nclassifying emails and create dynamic groups that belong to these messages.\nProposed mechanisms will be based on natural language processing techniques and\nwill be designed to facilitate human-machine interaction in this direction.",
            "journal ref": "A Treia Conferin\\c{t}\\u{a} Na\\c{t}ional\\u{a} de Interac\\c{t}iune\n  Om-Calculator 2006, Informatica Economica, vol. X, Bucuresti, pp. 79-82, 2006",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1101.0510v1",
            "date": "2011-01-03 12:54:03+00:00",
            "title": "Good Friends, Bad News - Affect and Virality in Twitter",
            "authors": [
                "Lars Kai Hansen",
                "Adam Arvidsson",
                "Finn \u00c5rup Nielsen",
                "Elanor Colleoni",
                "Michael Etter"
            ],
            "abstract": "The link between affect, defined as the capacity for sentimental arousal on\nthe part of a message, and virality, defined as the probability that it be sent\nalong, is of significant theoretical and practical importance, e.g. for viral\nmarketing. A quantitative study of emailing of articles from the NY Times finds\na strong link between positive affect and virality, and, based on psychological\ntheories it is concluded that this relation is universally valid. The\nconclusion appears to be in contrast with classic theory of diffusion in news\nmedia emphasizing negative affect as promoting propagation. In this paper we\nexplore the apparent paradox in a quantitative analysis of information\ndiffusion on Twitter. Twitter is interesting in this context as it has been\nshown to present both the characteristics social and news media. The basic\nmeasure of virality in Twitter is the probability of retweet. Twitter is\ndifferent from email in that retweeting does not depend on pre-existing social\nrelations, but often occur among strangers, thus in this respect Twitter may be\nmore similar to traditional news media. We therefore hypothesize that negative\nnews content is more likely to be retweeted, while for non-news tweets positive\nsentiments support virality. To test the hypothesis we analyze three corpora: A\ncomplete sample of tweets about the COP15 climate summit, a random sample of\ntweets, and a general text corpus including news. The latter allows us to train\na classifier that can distinguish tweets that carry news and non-news\ninformation. We present evidence that negative sentiment enhances virality in\nthe news segment, but not in the non-news segment. We conclude that the\nrelation between affect and virality is more complex than expected based on the\nfindings of Berger and Milkman (2010), in short 'if you want to be cited: Sweet\ntalk your friends or serve bad news to the public'.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1101.2804v1",
            "date": "2011-01-14 13:45:28+00:00",
            "title": "Aging in language dynamics",
            "authors": [
                "Animesh Mukherjee",
                "Francesca Tria",
                "Andrea Baronchelli",
                "Andrea Puglisi",
                "Vittorio Loreto"
            ],
            "abstract": "Human languages evolve continuously, and a puzzling problem is how to\nreconcile the apparent robustness of most of the deep linguistic structures we\nuse with the evidence that they undergo possibly slow, yet ceaseless, changes.\nIs the state in which we observe languages today closer to what would be a\ndynamical attractor with statistically stationary properties or rather closer\nto a non-steady state slowly evolving in time? Here we address this question in\nthe framework of the emergence of shared linguistic categories in a population\nof individuals interacting through language games. The observed emerging\nasymptotic categorization, which has been previously tested - with success -\nagainst experimental data from human languages, corresponds to a metastable\nstate where global shifts are always possible but progressively more unlikely\nand the response properties depend on the age of the system. This aging\nmechanism exhibits striking quantitative analogies to what is observed in the\nstatistical mechanics of glassy systems. We argue that this can be a general\nscenario in language dynamics where shared linguistic conventions would not\nemerge as attractors, but rather as metastable states.",
            "journal ref": "PLoS ONE 6(2): e16677, 2011",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1101.3578v3",
            "date": "2011-01-18 22:44:46+00:00",
            "title": "Infinity in computable probability",
            "authors": [
                "Maarten McKubre-Jordens",
                "Phillip L. Wilson"
            ],
            "abstract": "Does combining a finite collection of objects infinitely many times guarantee\nthe construction of a particular object? Here we use recursive function theory\nto examine the popular scenario of an infinite collection of typing monkeys\nreproducing the works of Shakespeare. Our main result is to show that it is\npossible to assign typing probabilities in such a way that while it is\nimpossible that no monkey reproduces Shakespeare's works, the probability of\nany finite collection of monkeys doing so is arbitrarily small. We extend our\nresults to target-free writing, and end with a broad discussion and pointers to\nfuture work.",
            "journal ref": "Journal of Applied Logics - IfCoLog Journal of Logics and their\n  Applications 6(7) (2019) 1253-1261",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1103.0784v1",
            "date": "2011-03-03 21:05:17+00:00",
            "title": "Happiness is assortative in online social networks",
            "authors": [
                "Johan Bollen",
                "Bruno Goncalves",
                "Guangchen Ruan",
                "Huina Mao"
            ],
            "abstract": "Social networks tend to disproportionally favor connections between\nindividuals with either similar or dissimilar characteristics. This propensity,\nreferred to as assortative mixing or homophily, is expressed as the correlation\nbetween attribute values of nearest neighbour vertices in a graph. Recent\nresults indicate that beyond demographic features such as age, sex and race,\neven psychological states such as \"loneliness\" can be assortative in a social\nnetwork. In spite of the increasing societal importance of online social\nnetworks it is unknown whether assortative mixing of psychological states takes\nplace in situations where social ties are mediated solely by online networking\nservices in the absence of physical contact. Here, we show that general\nhappiness or Subjective Well-Being (SWB) of Twitter users, as measured from a 6\nmonth record of their individual tweets, is indeed assortative across the\nTwitter social network. To our knowledge this is the first result that shows\nassortative mixing in online networks at the level of SWB. Our results imply\nthat online social networks may be equally subject to the social mechanisms\nthat cause assortative mixing in real social networks and that such assortative\nmixing takes place at the level of SWB. Given the increasing prevalence of\nonline social networks, their propensity to connect users with similar levels\nof SWB may be an important instrument in better understanding how both positive\nand negative sentiments spread through online social ties. Future research may\nfocus on how event-specific mood states can propagate and influence user\nbehavior in \"real life\".",
            "journal ref": "Artificial Life 17(3), 237-251 (2011)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1103.2325v1",
            "date": "2011-03-11 17:57:16+00:00",
            "title": "Self reference in word definitions",
            "authors": [
                "David Levary",
                "Jean-Pierre Eckmann",
                "Elisha Moses",
                "Tsvi Tlusty"
            ],
            "abstract": "Dictionaries are inherently circular in nature. A given word is linked to a\nset of alternative words (the definition) which in turn point to further\ndescendants. Iterating through definitions in this way, one typically finds\nthat definitions loop back upon themselves. The graph formed by such\ndefinitional relations is our object of study. By eliminating those links which\nare not in loops, we arrive at a core subgraph of highly connected nodes.\n  We observe that definitional loops are conveniently classified by length,\nwith longer loops usually emerging from semantic misinterpretation. By breaking\nthe long loops in the graph of the dictionary, we arrive at a set of\ndisconnected clusters. We find that the words in these clusters constitute\nsemantic units, and moreover tend to have been introduced into the English\nlanguage at similar times, suggesting a possible mechanism for language\nevolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1103.3585v1",
            "date": "2011-03-18 10:07:13+00:00",
            "title": "Incremental dimension reduction of tensors with random index",
            "authors": [
                "Fredrik Sandin",
                "Blerim Emruli",
                "Magnus Sahlgren"
            ],
            "abstract": "We present an incremental, scalable and efficient dimension reduction\ntechnique for tensors that is based on sparse random linear coding. Data is\nstored in a compactified representation with fixed size, which makes memory\nrequirements low and predictable. Component encoding and decoding are performed\non-line without computationally expensive re-analysis of the data set. The\nrange of tensor indices can be extended dynamically without modifying the\ncomponent representation. This idea originates from a mathematical model of\nsemantic memory and a method known as random indexing in natural language\nprocessing. We generalize the random-indexing algorithm to tensors and present\nsignal-to-noise-ratio simulations for representations of vectors and matrices.\nWe present also a mathematical analysis of the approximate orthogonality of\nhigh-dimensional ternary vectors, which is a property that underpins this and\nother similar random-coding approaches to dimension reduction. To further\ndemonstrate the properties of random indexing we present results of a synonym\nidentification task. The method presented here has some similarities with\nrandom projection and Tucker decomposition, but it performs well at high\ndimensionality only (n>10^3). Random indexing is useful for a range of complex\npractical problems, e.g., in natural language processing, data mining, pattern\nrecognition, event detection, graph searching and search engines. Prototype\nsoftware is provided. It supports encoding and decoding of tensors of order >=\n1 in a unified framework, i.e., vectors, matrices and higher order tensors.",
            "journal ref": "Revised version published in Knowl. Inf. Syst. 2016 (Open Access)",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/1103.3952v2",
            "date": "2011-03-21 10:04:20+00:00",
            "title": "Mixing, Ergodic, and Nonergodic Processes with Rapidly Growing Information between Blocks",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "We construct mixing processes over an infinite alphabet and ergodic processes\nover a finite alphabet for which Shannon mutual information between adjacent\nblocks of length $n$ grows as $n^\\beta$, where $\\beta\\in(0,1)$. The processes\nare a modification of nonergodic Santa Fe processes, which were introduced in\nthe context of natural language modeling. The rates of mutual information for\nthe latter processes are alike and also established in this paper. As an\nauxiliary result, it is shown that infinite direct products of mixing processes\nare also mixing.",
            "journal ref": "IEEE Transactions on Information Theory 58:3392-3401, 2012",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1103.4012v1",
            "date": "2011-03-21 13:59:43+00:00",
            "title": "On the accuracy of language trees",
            "authors": [
                "Simone Pompei",
                "Vittorio Loreto",
                "Francesca Tria"
            ],
            "abstract": "Historical linguistics aims at inferring the most likely language\nphylogenetic tree starting from information concerning the evolutionary\nrelatedness of languages. The available information are typically lists of\nhomologous (lexical, phonological, syntactic) features or characters for many\ndifferent languages.\n  From this perspective the reconstruction of language trees is an example of\ninverse problems: starting from present, incomplete and often noisy,\ninformation, one aims at inferring the most likely past evolutionary history. A\nfundamental issue in inverse problems is the evaluation of the inference made.\nA standard way of dealing with this question is to generate data with\nartificial models in order to have full access to the evolutionary process one\nis going to infer. This procedure presents an intrinsic limitation: when\ndealing with real data sets, one typically does not know which model of\nevolution is the most suitable for them. A possible way out is to compare\nalgorithmic inference with expert classifications. This is the point of view we\ntake here by conducting a thorough survey of the accuracy of reconstruction\nmethods as compared with the Ethnologue expert classifications. We focus in\nparticular on state-of-the-art distance-based methods for phylogeny\nreconstruction using worldwide linguistic databases.\n  In order to assess the accuracy of the inferred trees we introduce and\ncharacterize two generalizations of standard definitions of distances between\ntrees. Based on these scores we quantify the relative performances of the\ndistance-based algorithms considered. Further we quantify how the completeness\nand the coverage of the available databases affect the accuracy of the\nreconstruction. Finally we draw some conclusions about where the accuracy of\nthe reconstructions in historical linguistics stands and about the leading\ndirections to improve it.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1103.4090v2",
            "date": "2011-03-21 17:33:32+00:00",
            "title": "A Linear Classifier Based on Entity Recognition Tools and a Statistical Approach to Method Extraction in the Protein-Protein Interaction Literature",
            "authors": [
                "An\u00e1lia Louren\u00e7o",
                "Michael Conover",
                "Andrew Wong",
                "Azadeh Nematzadeh",
                "Fengxia Pan",
                "Hagit Shatkay",
                "Luis M. Rocha"
            ],
            "abstract": "We participated, in the Article Classification and the Interaction Method\nsubtasks (ACT and IMT, respectively) of the Protein-Protein Interaction task of\nthe BioCreative III Challenge. For the ACT, we pursued an extensive testing of\navailable Named Entity Recognition and dictionary tools, and used the most\npromising ones to extend our Variable Trigonometric Threshold linear\nclassifier. For the IMT, we experimented with a primarily statistical approach,\nas opposed to employing a deeper natural language processing strategy. Finally,\nwe also studied the benefits of integrating the method extraction approach that\nwe have used for the IMT into the ACT pipeline. For the ACT, our linear article\nclassifier leads to a ranking and classification performance significantly\nhigher than all the reported submissions. For the IMT, our results are\ncomparable to those of other systems, which took very different approaches. For\nthe ACT, we show that the use of named entity recognition tools leads to a\nsubstantial improvement in the ranking and classification of articles relevant\nto protein-protein interaction. Thus, we show that our substantially expanded\nlinear classifier is a very competitive classifier in this domain. Moreover,\nthis classifier produces interpretable surfaces that can be understood as\n\"rules\" for human understanding of the classification. In terms of the IMT\ntask, in contrast to other participants, our approach focused on identifying\nsentences that are likely to bear evidence for the application of a PPI\ndetection method, rather than on classifying a document as relevant to a\nmethod. As BioCreative III did not perform an evaluation of the evidence\nprovided by the system, we have conducted a separate assessment; the evaluators\nagree that our tool is indeed effective in detecting relevant evidence for PPI\ndetection methods.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1105.1306v2",
            "date": "2011-05-06 15:35:24+00:00",
            "title": "Excess entropy in natural language: present state and perspectives",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "We review recent progress in understanding the meaning of mutual information\nin natural language. Let us define words in a text as strings that occur\nsufficiently often. In a few previous papers, we have shown that a power-law\ndistribution for so defined words (a.k.a. Herdan's law) is obeyed if there is a\nsimilar power-law growth of (algorithmic) mutual information between adjacent\nportions of texts of increasing length. Moreover, the power-law growth of\ninformation holds if texts describe a complicated infinite (algorithmically)\nrandom object in a highly repetitive way, according to an analogous power-law\ndistribution. The described object may be immutable (like a mathematical or\nphysical constant) or may evolve slowly in time (like cultural heritage). Here\nwe reflect on the respective mathematical results in a less technical way. We\nalso discuss feasibility of deciding to what extent these results apply to the\nactual human communication.",
            "journal ref": "Chaos 21:037105, 2011",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1107.3263v2",
            "date": "2011-07-16 22:58:41+00:00",
            "title": "Naming Game on Adaptive Weighted Networks",
            "authors": [
                "Dorota Lipowska",
                "Adam Lipowski"
            ],
            "abstract": "We examine a naming game on an adaptive weighted network. A weight of\nconnection for a given pair of agents depends on their communication success\nrate and determines the probability with which the agents communicate. In some\ncases, depending on the parameters of the model, the preference toward\nsuccessfully communicating agents is basically negligible and the model behaves\nsimilarly to the naming game on a complete graph. In particular, it quickly\nreaches a single-language state, albeit some details of the dynamics are\ndifferent from the complete-graph version. In some other cases, the preference\ntoward successfully communicating agents becomes much more relevant and the\nmodel gets trapped in a multi-language regime. In this case gradual coarsening\nand extinction of languages lead to the emergence of a dominant language,\nalbeit with some other languages still being present. A comparison of\ndistribution of languages in our model and in the human population is\ndiscussed.",
            "journal ref": "Artificial Life vol. 18, 311-323 (2012)",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/1107.4573v1",
            "date": "2011-07-22 16:54:11+00:00",
            "title": "Analogy perception applied to seven tests of word comprehension",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "It has been argued that analogy is the core of cognition. In AI research,\nalgorithms for analogy are often limited by the need for hand-coded high-level\nrepresentations as input. An alternative approach is to use high-level\nperception, in which high-level representations are automatically generated\nfrom raw data. Analogy perception is the process of recognizing analogies using\nhigh-level perception. We present PairClass, an algorithm for analogy\nperception that recognizes lexical proportional analogies using representations\nthat are automatically generated from a large corpus of raw textual data. A\nproportional analogy is an analogy of the form A:B::C:D, meaning \"A is to B as\nC is to D\". A lexical proportional analogy is a proportional analogy with\nwords, such as carpenter:wood::mason:stone. PairClass represents the semantic\nrelations between two words using a high-dimensional feature vector, in which\nthe elements are based on frequencies of patterns in the corpus. PairClass\nrecognizes analogies by applying standard supervised machine learning\ntechniques to the feature vectors. We show how seven different tests of word\ncomprehension can be framed as problems of analogy perception and we then apply\nPairClass to the seven resulting sets of analogy perception problems. We\nachieve competitive results on all seven tests. This is the first time a\nuniform approach has handled such a range of tests of word comprehension.",
            "journal ref": "Journal of Experimental & Theoretical Artificial Intelligence\n  (JETAI), 2011, Volume 23, Issue 3, pages 343-362",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1108.3850v1",
            "date": "2011-08-18 20:17:58+00:00",
            "title": "Solving puzzles described in English by automated translation to answer set programming and learning how to do that translation",
            "authors": [
                "Chitta Baral",
                "Juraj Dzifcak"
            ],
            "abstract": "We present a system capable of automatically solving combinatorial logic\npuzzles given in (simplified) English. It involves translating the English\ndescriptions of the puzzles into answer set programming(ASP) and using ASP\nsolvers to provide solutions of the puzzles. To translate the descriptions, we\nuse a lambda-calculus based approach using Probabilistic Combinatorial\nCategorial Grammars (PCCG) where the meanings of words are associated with\nparameters to be able to distinguish between multiple meanings of the same\nword. Meaning of many words and the parameters are learned. The puzzles are\nrepresented in ASP using an ontology which is applicable to a large set of\nlogic puzzles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5017v1",
            "date": "2011-08-25 06:08:16+00:00",
            "title": "Event in Compositional Dynamic Semantics",
            "authors": [
                "Sai Qian",
                "Maxime Amblard"
            ],
            "abstract": "We present a framework which constructs an event-style dis- course semantics.\nThe discourse dynamics are encoded in continuation semantics and various\nrhetorical relations are embedded in the resulting interpretation of the\nframework. We assume discourse and sentence are distinct semantic objects, that\nplay different roles in meaning evalua- tion. Moreover, two sets of composition\nfunctions, for handling different discourse relations, are introduced. The\npaper first gives the necessary background and motivation for event and dynamic\nsemantics, then the framework with detailed examples will be introduced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5027v1",
            "date": "2011-08-25 07:22:09+00:00",
            "title": "Encoding Phases using Commutativity and Non-commutativity in a Logical Framework",
            "authors": [
                "Maxime Amblard"
            ],
            "abstract": "This article presents an extension of Minimalist Categorial Gram- mars (MCG)\nto encode Chomsky's phases. These grammars are based on Par- tially Commutative\nLogic (PCL) and encode properties of Minimalist Grammars (MG) of Stabler. The\nfirst implementation of MCG were using both non- commutative properties (to\nrespect the linear word order in an utterance) and commutative ones (to model\nfeatures of different constituents). Here, we pro- pose to adding Chomsky's\nphases with the non-commutative tensor product of the logic. Then we could give\naccount of the PIC just by using logical prop- erties of the framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1108.5520v1",
            "date": "2011-08-29 10:27:19+00:00",
            "title": "A sentiment analysis of Singapore Presidential Election 2011 using Twitter data with census correction",
            "authors": [
                "Murphy Choy",
                "Michelle L. F. Cheong",
                "Ma Nang Laik",
                "Koo Ping Shung"
            ],
            "abstract": "Sentiment analysis is a new area in text analytics where it focuses on the\nanalysis and understanding of the emotions from the text patterns. This new\nform of analysis has been widely adopted in customer relation management\nespecially in the context of complaint management. With increasing level of\ninterest in this technology, more and more companies are adopting it and using\nit to champion their marketing efforts. However, sentiment analysis using\ntwitter has remained extremely difficult to manage due to the sampling bias. In\nthis paper, we will discuss about the application of using reweighting\ntechniques in conjunction with online sentiment divisions to predict the vote\npercentage that individual candidate will receive. There will be in depth\ndiscussion about the various aspects using sentiment analysis to predict\noutcomes as well as the potential pitfalls in the estimation due to the\nanonymous nature of the internet.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1108.5974v1",
            "date": "2011-08-30 14:45:41+00:00",
            "title": "Emotional Analysis of Blogs and Forums Data",
            "authors": [
                "Pawe\u0142 Wero\u0144ski",
                "Julian Sienkiewicz",
                "Georgios Paltoglou",
                "Kevan Buckley",
                "Mike Thelwall",
                "Janusz A. Ho\u0142yst"
            ],
            "abstract": "We perform a statistical analysis of emotionally annotated comments in two\nlarge online datasets, examining chains of consecutive posts in the\ndiscussions. Using comparisons with randomised data we show that there is a\nhigh level of correlation for the emotional content of messages.",
            "journal ref": "Acta Physica Polonica A 121, B-128 (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.1618v1",
            "date": "2011-09-08 03:13:12+00:00",
            "title": "An analysis of Twitter messages in the 2011 Tohoku Earthquake",
            "authors": [
                "Son Doan",
                "Bao-Khanh Ho Vo",
                "Nigel Collier"
            ],
            "abstract": "Social media such as Facebook and Twitter have proven to be a useful resource\nto understand public opinion towards real world events. In this paper, we\ninvestigate over 1.5 million Twitter messages (tweets) for the period 9th March\n2011 to 31st May 2011 in order to track awareness and anxiety levels in the\nTokyo metropolitan district to the 2011 Tohoku Earthquake and subsequent\ntsunami and nuclear emergencies. These three events were tracked using both\nEnglish and Japanese tweets. Preliminary results indicated: 1) close\ncorrespondence between Twitter data and earthquake events, 2) strong\ncorrelation between English and Japanese tweets on the same events, 3) tweets\nin the native language play an important roles in early warning, 4) tweets\nshowed how quickly Japanese people's anxiety returned to normal levels after\nthe earthquake event. Several distinctions between English and Japanese tweets\non earthquake events are also discussed. The results suggest that Twitter data\ncan be used as a useful resource for tracking the public mood of populations\naffected by natural disasters as well as an early warning system.",
            "journal ref": "Lecture Notes of the Institute for Computer Sciences, Social\n  Informatics and Telecommunications Engineering, 2012, Volume 91, Part 4,\n  58-66",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1109.2657v1",
            "date": "2011-09-13 01:25:43+00:00",
            "title": "From Contracts in Structured English to CL Specifications",
            "authors": [
                "Seyed M. Montazeri",
                "Nivir K. S. Roy",
                "Gerardo Schneider"
            ],
            "abstract": "In this paper we present a framework to analyze conflicts of contracts\nwritten in structured English. A contract that has manually been rewritten in a\nstructured English is automatically translated into a formal language using the\nGrammatical Framework (GF). In particular we use the contract language CL as a\ntarget formal language for this translation. In our framework CL specifications\ncould then be input into the tool CLAN to detect the presence of conflicts\n(whether there are contradictory obligations, permissions, and prohibitions. We\nalso use GF to get a version in (restricted) English of CL formulae. We discuss\nthe implementation of such a framework.",
            "journal ref": "EPTCS 68, 2011, pp. 55-69",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1109.6018v1",
            "date": "2011-09-27 20:00:47+00:00",
            "title": "User-level sentiment analysis incorporating social networks",
            "authors": [
                "Chenhao Tan",
                "Lillian Lee",
                "Jie Tang",
                "Long Jiang",
                "Ming Zhou",
                "Ping Li"
            ],
            "abstract": "We show that information about social relationships can be used to improve\nuser-level sentiment analysis. The main motivation behind our approach is that\nusers that are somehow \"connected\" may be more likely to hold similar opinions;\ntherefore, relationship information can complement what we can extract about a\nuser's viewpoints from their utterances. Employing Twitter as a source for our\nexperimental data, and working within a semi-supervised framework, we propose\nmodels that are induced either from the Twitter follower/followee network or\nfrom the network in Twitter formed by users referring to each other using \"@\"\nmentions. Our transductive learning results reveal that incorporating\nsocial-network information can indeed lead to statistically significant\nsentiment-classification improvements over the performance of an approach based\non Support Vector Machines having access only to textual features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.2162v2",
            "date": "2011-10-10 19:54:57+00:00",
            "title": "Large-Margin Learning of Submodular Summarization Methods",
            "authors": [
                "Ruben Sipos",
                "Pannaga Shivaswamy",
                "Thorsten Joachims"
            ],
            "abstract": "In this paper, we present a supervised learning approach to training\nsubmodular scoring functions for extractive multi-document summarization. By\ntaking a structured predicition approach, we provide a large-margin method that\ndirectly optimizes a convex relaxation of the desired performance measure. The\nlearning method applies to all submodular summarization methods, and we\ndemonstrate its effectiveness for both pairwise as well as coverage-based\nscoring functions on multiple datasets. Compared to state-of-the-art functions\nthat were tuned manually, our method significantly improves performance and\nenables high-fidelity models with numbers of parameters well beyond what could\nreasonbly be tuned by hand.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1110.3088v1",
            "date": "2011-10-13 23:05:48+00:00",
            "title": "Towards cross-lingual alerting for bursty epidemic events",
            "authors": [
                "Nigel Collier"
            ],
            "abstract": "Background: Online news reports are increasingly becoming a source for event\nbased early warning systems that detect natural disasters. Harnessing the\nmassive volume of information available from multilingual newswire presents as\nmany challenges as opportunities due to the patterns of reporting complex\nspatiotemporal events. Results: In this article we study the problem of\nutilising correlated event reports across languages. We track the evolution of\n16 disease outbreaks using 5 temporal aberration detection algorithms on\ntext-mined events classified according to disease and outbreak country. Using\nProMED reports as a silver standard, comparative analysis of news data for 13\nlanguages over a 129 day trial period showed improved sensitivity, F1 and\ntimeliness across most models using cross-lingual events. We report a detailed\ncase study analysis for Cholera in Angola 2010 which highlights the challenges\nfaced in correlating news events with the silver standard. Conclusions: The\nresults show that automated health surveillance using multilingual text mining\nhas the potential to turn low value news into high value alerts if informed\nchoices are used to govern the selection of models and data sources. An\nimplementation of the C2 alerting algorithm using multilingual news is\navailable at the BioCaster portal http://born.nii.ac.jp/?page=globalroundup.",
            "journal ref": "Journal of Biomedical Semantics 2011, 2(Suppl 5):S10",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.3089v1",
            "date": "2011-10-13 23:15:44+00:00",
            "title": "OMG U got flu? Analysis of shared health messages for bio-surveillance",
            "authors": [
                "Nigel Collier",
                "Nguyen Truong Son",
                "Ngoc Mai Nguyen"
            ],
            "abstract": "Background: Micro-blogging services such as Twitter offer the potential to\ncrowdsource epidemics in real-time. However, Twitter posts ('tweets') are often\nambiguous and reactive to media trends. In order to ground user messages in\nepidemic response we focused on tracking reports of self-protective behaviour\nsuch as avoiding public gatherings or increased sanitation as the basis for\nfurther risk analysis. Results: We created guidelines for tagging self\nprotective behaviour based on Jones and Salath\\'e (2009)'s behaviour response\nsurvey. Applying the guidelines to a corpus of 5283 Twitter messages related to\ninfluenza like illness showed a high level of inter-annotator agreement (kappa\n0.86). We employed supervised learning using unigrams, bigrams and regular\nexpressions as features with two supervised classifiers (SVM and Naive Bayes)\nto classify tweets into 4 self-reported protective behaviour categories plus a\nself-reported diagnosis. In addition to classification performance we report\nmoderately strong Spearman's Rho correlation by comparing classifier output\nagainst WHO/NREVSS laboratory data for A(H1N1) in the USA during the 2009-2010\ninfluenza season. Conclusions: The study adds to evidence supporting a high\ndegree of correlation between pre-diagnostic social media signals and\ndiagnostic influenza case data, pointing the way towards low cost sensor\nnetworks. We believe that the signals we have modelled may be applicable to a\nwide range of diseases.",
            "journal ref": "Journal of Biomedical Semantics 2011, 2(Suppl 5):S9",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.3091v1",
            "date": "2011-10-13 23:22:43+00:00",
            "title": "What's unusual in online disease outbreak news?",
            "authors": [
                "Nigel Collier"
            ],
            "abstract": "Background: Accurate and timely detection of public health events of\ninternational concern is necessary to help support risk assessment and response\nand save lives. Novel event-based methods that use the World Wide Web as a\nsignal source offer potential to extend health surveillance into areas where\ntraditional indicator networks are lacking. In this paper we address the issue\nof systematically evaluating online health news to support automatic alerting\nusing daily disease-country counts text mined from real world data using\nBioCaster. For 18 data sets produced by BioCaster, we compare 5 aberration\ndetection algorithms (EARS C2, C3, W2, F-statistic and EWMA) for performance\nagainst expert moderated ProMED-mail postings. Results: We report sensitivity,\nspecificity, positive predictive value (PPV), negative predictive value (NPV),\nmean alerts/100 days and F1, at 95% confidence interval (CI) for 287\nProMED-mail postings on 18 outbreaks across 14 countries over a 366 day period.\nResults indicate that W2 had the best F1 with a slight benefit for day of week\neffect over C2. In drill down analysis we indicate issues arising from the\ngranular choice of country-level modeling, sudden drops in reporting due to day\nof week effects and reporting bias. Automatic alerting has been implemented in\nBioCaster available from http://born.nii.ac.jp. Conclusions: Online health news\nalerts have the potential to enhance manual analytical methods by increasing\nthroughput, timeliness and detection rates. Systematic evaluation of health\nnews aberrations is necessary to push forward our understanding of the complex\nrelationship between news report volumes and case numbers and to select the\nbest performing features and algorithms.",
            "journal ref": "Journal of Biomedical Semantics 2010, 1:2",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.3094v1",
            "date": "2011-10-13 23:42:32+00:00",
            "title": "Syndromic classification of Twitter messages",
            "authors": [
                "Nigel Collier",
                "Son Doan"
            ],
            "abstract": "Recent studies have shown strong correlation between social networking data\nand national influenza rates. We expanded upon this success to develop an\nautomated text mining system that classifies Twitter messages in real time into\nsix syndromic categories based on key terms from a public health ontology.\n10-fold cross validation tests were used to compare Naive Bayes (NB) and\nSupport Vector Machine (SVM) models on a corpus of 7431 Twitter messages. SVM\nperformed better than NB on 4 out of 6 syndromes. The best performing\nclassifiers showed moderately strong F1 scores: respiratory = 86.2 (NB);\ngastrointestinal = 85.4 (SVM polynomial kernel degree 2); neurological = 88.6\n(SVM polynomial kernel degree 1); rash = 86.0 (SVM polynomial kernel degree 1);\nconstitutional = 89.3 (SVM polynomial kernel degree 1); hemorrhagic = 89.9\n(NB). The resulting classifiers were deployed together with an EARS C2\naberration detection algorithm in an experimental online system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.4123v4",
            "date": "2011-10-18 20:54:21+00:00",
            "title": "Positive words carry less information than negative words",
            "authors": [
                "David Garcia",
                "Antonios Garas",
                "Frank Schweitzer"
            ],
            "abstract": "We show that the frequency of word use is not only determined by the word\nlength \\cite{Zipf1935} and the average information content\n\\cite{Piantadosi2011}, but also by its emotional content. We have analyzed\nthree established lexica of affective word usage in English, German, and\nSpanish, to verify that these lexica have a neutral, unbiased, emotional\ncontent. Taking into account the frequency of word usage, we find that words\nwith a positive emotional content are more frequently used. This lends support\nto Pollyanna hypothesis \\cite{Boucher1969} that there should be a positive bias\nin human expression. We also find that negative words contain more information\nthan positive words, as the informativeness of a word increases uniformly with\nits valence decrease. Our findings support earlier conjectures about (i) the\nrelation between word frequency and information content, and (ii) the impact of\npositive emotions on communication and social links.",
            "journal ref": "EPJ Data Science 2012, 1:3",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1110.6200v2",
            "date": "2011-10-27 21:37:24+00:00",
            "title": "TopicViz: Semantic Navigation of Document Collections",
            "authors": [
                "Jacob Eisenstein",
                "Duen Horng \"Polo\" Chau",
                "Aniket Kittur",
                "Eric P. Xing"
            ],
            "abstract": "When people explore and manage information, they think in terms of topics and\nthemes. However, the software that supports information exploration sees text\nat only the surface level. In this paper we show how topic modeling -- a\ntechnique for identifying latent themes across large collections of documents\n-- can support semantic exploration. We present TopicViz, an interactive\nenvironment for information exploration. TopicViz combines traditional search\nand citation-graph functionality with a range of novel interactive\nvisualizations, centered around a force-directed layout that links documents to\nthe latent themes discovered by the topic model. We describe several use\nscenarios in which TopicViz supports rapid sensemaking on large document\ncollections.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1112.3670v3",
            "date": "2011-12-15 21:15:27+00:00",
            "title": "Echoes of power: Language effects and power differences in social interaction",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee",
                "Bo Pang",
                "Jon Kleinberg"
            ],
            "abstract": "Understanding social interaction within groups is key to analyzing online\ncommunities. Most current work focuses on structural properties: who talks to\nwhom, and how such interactions form larger network structures. The\ninteractions themselves, however, generally take place in the form of natural\nlanguage --- either spoken or written --- and one could reasonably suppose that\nsignals manifested in language might also provide information about roles,\nstatus, and other aspects of the group's dynamics. To date, however, finding\nsuch domain-independent language-based signals has been a challenge.\n  Here, we show that in group discussions power differentials between\nparticipants are subtly revealed by how much one individual immediately echoes\nthe linguistic style of the person they are responding to. Starting from this\nobservation, we propose an analysis framework based on linguistic coordination\nthat can be used to shed light on power relationships and that works\nconsistently across multiple types of power --- including a more \"static\" form\nof power based on status differences, and a more \"situational\" form of power in\nwhich one individual experiences a type of dependence on another. Using this\nframework, we study how conversational behavior can reveal power relationships\nin two very different settings: discussions among Wikipedians and arguments\nbefore the U.S. Supreme Court.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1112.5947v1",
            "date": "2011-12-27 10:16:35+00:00",
            "title": "Random Context and Semi-Conditional Insertion-Deletion Systems",
            "authors": [
                "Sergiu Ivanov",
                "Sergey Verlan"
            ],
            "abstract": "In this article we introduce the operations of insertion and deletion working\nin a random-context and semi-conditional manner. We show that the conditional\nuse of rules strictly increase the computational power. In the case of\nsemi-conditional insertion-deletion systems context-free insertion and deletion\nrules of one symbol are sufficient to get the computational completeness. In\nthe random context case our results expose an asymmetry between the\ncomputational power of insertion and deletion rules: systems of size $(2,0,0;\n1,1,0)$ are computationally complete, while systems of size $(1,1,0;2,0,0)$\n(and more generally of size $(1,1,0;p,1,1)$) are not. This is particularly\ninteresting because other control mechanisms like graph-control or matrix\ncontrol used together with insertion-deletion systems do not present such\nasymmetry.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1112.6045v1",
            "date": "2011-12-28 02:20:48+00:00",
            "title": "Comparing intermittency and network measurements of words and their dependency on authorship",
            "authors": [
                "Diego R. Amancio",
                "Eduardo G. Altmann",
                "Osvaldo N. Oliveira Jr.",
                "Luciano da F. Costa"
            ],
            "abstract": "Many features from texts and languages can now be inferred from statistical\nanalyses using concepts from complex networks and dynamical systems. In this\npaper we quantify how topological properties of word co-occurrence networks and\nintermittency (or burstiness) in word distribution depend on the style of\nauthors. Our database contains 40 books from 8 authors who lived in the 19th\nand 20th centuries, for which the following network measurements were obtained:\nclustering coefficient, average shortest path lengths, and betweenness. We\nfound that the two factors with stronger dependency on the authors were the\nskewness in the distribution of word intermittency and the average shortest\npaths. Other factors such as the betweeness and the Zipf's law exponent show\nonly weak dependency on authorship. Also assessed was the contribution from\neach measurement to authorship recognition using three machine learning\nmethods. The best performance was a ca. 65 % accuracy upon combining complex\nnetwork and intermittency features with the nearest neighbor algorithm. From a\ndetailed analysis of the interdependence of the various metrics it is concluded\nthat the methods used here are complementary for providing short- and\nlong-scale perspectives of texts, which are useful for applications such as\nidentification of topical words and information retrieval.",
            "journal ref": "New J. Phys. (2011) 13 123024",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1103.2681v1",
            "date": "2011-03-14 14:54:10+00:00",
            "title": "A Paradoxical Property of the Monkey Book",
            "authors": [
                "Sebastian Bernhardsson",
                "Seung Ki Baek",
                "Petter Minnhagen"
            ],
            "abstract": "A \"monkey book\" is a book consisting of a random distribution of letters and\nblanks, where a group of letters surrounded by two blanks is defined as a word.\nWe compare the statistics of the word distribution for a monkey book with the\ncorresponding distribution for the general class of random books, where the\nlatter are books for which the words are randomly distributed. It is shown that\nthe word distribution statistics for the monkey book is different and quite\ndistinct from a typical sampled book or real book. In particular the monkey\nbook obeys Heaps' power law to an extraordinary good approximation, in contrast\nto the word distributions for sampled and real books, which deviate from Heaps'\nlaw in a characteristics way. The somewhat counter-intuitive conclusion is that\na \"monkey book\" obeys Heaps' power law precisely because its word-frequency\ndistribution is not a smooth power law, contrary to the expectation based on\nsimple mathematical arguments that if one is a power law, so is the other.",
            "journal ref": "J. Stat. Mech. (2011) P07013",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1107.3707v2",
            "date": "2011-07-19 13:02:42+00:00",
            "title": "Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death",
            "authors": [
                "Alexander M. Petersen",
                "Joel Tenenbaum",
                "Shlomo Havlin",
                "H. Eugene Stanley"
            ],
            "abstract": "We analyze the dynamic properties of 10^7 words recorded in English, Spanish\nand Hebrew over the period 1800--2008 in order to gain insight into the\ncoevolution of language and culture. We report language independent patterns\nuseful as benchmarks for theoretical models of language evolution. A\nsignificantly decreasing (increasing) trend in the birth (death) rate of words\nindicates a recent shift in the selection laws governing word use. For new\nwords, we observe a peak in the growth-rate fluctuations around 40 years after\nintroduction, consistent with the typical entry time into standard dictionaries\nand the human generational timescale. Pronounced changes in the dynamics of\nlanguage during periods of war shows that word correlations, occurring across\ntime and between words, are largely influenced by coevolutionary social,\ntechnological, and political factors. We quantify cultural memory by analyzing\nthe long-term correlations in the use of individual words using detrended\nfluctuation analysis.",
            "journal ref": "Scientific Reports 2, 313 (2012)",
            "category": "physics.soc-ph"
        }
    ],
    "2012": [
        {
            "url": "http://arxiv.org/abs/1203.0145v2",
            "date": "2012-03-01 11:06:32+00:00",
            "title": "The Horse Raced Past: Gardenpath Processing in Dynamical Systems",
            "authors": [
                "Peter beim Graben"
            ],
            "abstract": "I pinpoint an interesting similarity between a recent account to rational\nparsing and the treatment of sequential decisions problems in a dynamical\nsystems approach. I argue that expectation-driven search heuristics aiming at\nfast computation resembles a high-risk decision strategy in favor of large\ntransition velocities. Hale's rational parser, combining generalized\nleft-corner parsing with informed $\\mathrm{A}^*$ search to resolve processing\nconflicts, explains gardenpath effects in natural sentence processing by\nmisleading estimates of future processing costs that are to be minimized. On\nthe other hand, minimizing the duration of cognitive computations in\ntime-continuous dynamical systems can be described by combining vector space\nrepresentations of cognitive states by means of filler/role decompositions and\nsubsequent tensor product representations with the paradigm of stable\nheteroclinic sequences. Maximizing transition velocities according to a\nhigh-risk decision strategy could account for a fast race even between states\nthat are apparently remote in representation space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.1685v1",
            "date": "2012-03-08 03:06:29+00:00",
            "title": "Statistical Function Tagging and Grammatical Relations of Myanmar Sentences",
            "authors": [
                "Win Win Thant",
                "Tin Myat Htwe",
                "Ni Lar Thein"
            ],
            "abstract": "This paper describes a context free grammar (CFG) based grammatical relations\nfor Myanmar sentences which combine corpus-based function tagging system. Part\nof the challenge of statistical function tagging for Myanmar sentences comes\nfrom the fact that Myanmar has free-phrase-order and a complex morphological\nsystem. Function tagging is a pre-processing step to show grammatical relations\nof Myanmar sentences. In the task of function tagging, which tags the function\nof Myanmar sentences with correct segmentation, POS (part-of-speech) tagging\nand chunking information, we use Naive Bayesian theory to disambiguate the\npossible function tags of a word. We apply context free grammar (CFG) to find\nout the grammatical relations of the function tags. We also create a functional\nannotated tagged corpus for Myanmar and propose the grammar rules for Myanmar\nsentences. Experiments show that our analysis achieves a good result with\nsimple sentences and complex sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.1858v1",
            "date": "2012-03-08 17:29:33+00:00",
            "title": "Distributional Measures of Semantic Distance: A Survey",
            "authors": [
                "Saif M. Mohammad",
                "Graeme Hirst"
            ],
            "abstract": "The ability to mimic human notions of semantic distance has widespread\napplications. Some measures rely only on raw text (distributional measures) and\nsome rely on knowledge sources such as WordNet. Although extensive studies have\nbeen performed to compare WordNet-based measures with human judgment, the use\nof distributional measures as proxies to estimate semantic distance has\nreceived little attention. Even though they have traditionally performed poorly\nwhen compared to WordNet-based measures, they lay claim to certain uniquely\nattractive features, such as their applicability in resource-poor languages and\ntheir ability to mimic both semantic similarity and semantic relatedness.\nTherefore, this paper presents a detailed study of distributional measures.\nParticular attention is paid to flesh out the strengths and limitations of both\nWordNet-based and distributional measures, and how distributional measures of\ndistance can be brought more in line with human notions of semantic distance.\nWe conclude with a brief discussion of recent work on hybrid measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.1889v1",
            "date": "2012-03-08 19:03:37+00:00",
            "title": "Distributional Measures as Proxies for Semantic Relatedness",
            "authors": [
                "Saif M Mohammad",
                "Graeme Hirst"
            ],
            "abstract": "The automatic ranking of word pairs as per their semantic relatedness and\nability to mimic human notions of semantic relatedness has widespread\napplications. Measures that rely on raw data (distributional measures) and\nthose that use knowledge-rich ontologies both exist. Although extensive studies\nhave been performed to compare ontological measures with human judgment, the\ndistributional measures have primarily been evaluated by indirect means. This\npaper is a detailed study of some of the major distributional measures; it\nlists their respective merits and limitations. New measures that overcome these\ndrawbacks, that are more in line with the human notions of semantic\nrelatedness, are suggested. The paper concludes with an exhaustive comparison\nof the distributional and ontology-based measures. Along the way, significant\nresearch problems are identified. Work on these problems may lead to a better\nunderstanding of how semantic relatedness is to be measured.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.2498v2",
            "date": "2012-03-08 13:28:03+00:00",
            "title": "Fault detection system for Arabic language",
            "authors": [
                "Riadh Bouslimi",
                "Houda Amraoui"
            ],
            "abstract": "The study of natural language, especially Arabic, and mechanisms for the\nimplementation of automatic processing is a fascinating field of study, with\nvarious potential applications. The importance of tools for natural language\nprocessing is materialized by the need to have applications that can\neffectively treat the vast mass of information available nowadays on electronic\nforms. Among these tools, mainly driven by the necessity of a fast writing in\nalignment to the actual daily life speed, our interest is on the writing\nauditors. The morphological and syntactic properties of Arabic make it a\ndifficult language to master, and explain the lack in the processing tools for\nthat language. Among these properties, we can mention: the complex structure of\nthe Arabic word, the agglutinative nature, lack of vocalization, the\nsegmentation of the text, the linguistic richness, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.3023v1",
            "date": "2012-03-14 08:51:14+00:00",
            "title": "Toward an example-based machine translation from written text to ASL using virtual agent animation",
            "authors": [
                "Mehrez Boulares",
                "Mohamed Jemni"
            ],
            "abstract": "Modern computational linguistic software cannot produce important aspects of\nsign language translation. Using some researches we deduce that the majority of\nautomatic sign language translation systems ignore many aspects when they\ngenerate animation; therefore the interpretation lost the truth information\nmeaning. Our goals are: to translate written text from any language to ASL\nanimation; to model maximum raw information using machine learning and\ncomputational techniques; and to produce a more adapted and expressive form to\nnatural looking and understandable ASL animations. Our methods include\nlinguistic annotation of initial text and semantic orientation to generate the\nfacial expression. We use the genetic algorithms coupled to learning/recognized\nsystems to produce the most natural form. To detect emotion we are based on\nfuzzy logic to produce the degree of interpolation between facial expressions.\nRoughly, we present a new expressive language Text Adapted Sign Modeling\nLanguage TASML that describes all maximum aspects related to a natural sign\nlanguage interpretation. This paper is organized as follow: the next section is\ndevoted to present the comprehension effect of using Space/Time/SVO form in ASL\nanimation based on experimentation. In section 3, we describe our technical\nconsiderations. We present the general approach we adopted to develop our tool\nin section 4. Finally, we give some perspectives and future works.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 9,\n  Issue 1, No 1, January 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.3584v1",
            "date": "2012-03-15 22:49:20+00:00",
            "title": "An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes",
            "authors": [
                "Tarek El-Shishtawy",
                "Fatma El-Ghannam"
            ],
            "abstract": "In spite of its robust syntax, semantic cohesion, and less ambiguity, lemma\nlevel analysis and generation does not yet focused in Arabic NLP literatures.\nIn the current research, we propose the first non-statistical accurate Arabic\nlemmatizer algorithm that is suitable for information retrieval (IR) systems.\nThe proposed lemmatizer makes use of different Arabic language knowledge\nresources to generate accurate lemma form and its relevant features that\nsupport IR purposes. As a POS tagger, the experimental results show that, the\nproposed algorithm achieves a maximum accuracy of 94.8%. For first seen\ndocuments, an accuracy of 89.15% is achieved, compared to 76.7% of up to date\nStanford accurate Arabic model, for the same, dataset.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 9,\n  Issue 1, No 3, January 2012 ISSN (Online): 1694-0814",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.4605v1",
            "date": "2012-03-20 21:52:35+00:00",
            "title": "Arabic Keyphrase Extraction using Linguistic knowledge and Machine Learning Techniques",
            "authors": [
                "Tarek El-shishtawy",
                "Abdulwahab Al-sammak"
            ],
            "abstract": "In this paper, a supervised learning technique for extracting keyphrases of\nArabic documents is presented. The extractor is supplied with linguistic\nknowledge to enhance its efficiency instead of relying only on statistical\ninformation such as term frequency and distance. During analysis, an annotated\nArabic corpus is used to extract the required lexical features of the document\nwords. The knowledge also includes syntactic rules based on part of speech tags\nand allowed word sequences to extract the candidate keyphrases. In this work,\nthe abstract form of Arabic words is used instead of its stem form to represent\nthe candidate terms. The Abstract form hides most of the inflections found in\nArabic words. The paper introduces new features of keyphrases based on\nlinguistic knowledge, to capture titles and subtitles of a document. A simple\nANOVA test is used to evaluate the validity of selected features. Then, the\nlearning model is built using the LDA - Linear Discriminant Analysis - and\ntraining documents. Although, the presented system is trained using documents\nin the IT domain, experiments carried out show that it has a significantly\nbetter performance than the existing Arabic extractor systems, where precision\nand recall values reach double their corresponding values in the other systems\nespecially for lengthy and non-scientific articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.4933v1",
            "date": "2012-03-22 09:50:51+00:00",
            "title": "Reduplicated MWE (RMWE) helps in improving the CRF based Manipuri POS Tagger",
            "authors": [
                "Kishorjit Nongmeikapam",
                "Lairenlakpam Nonglenjaoba",
                "Yumnam Nirmal",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "This paper gives a detail overview about the modified features selection in\nCRF (Conditional Random Field) based Manipuri POS (Part of Speech) tagging.\nSelection of features is so important in CRF that the better are the features\nthen the better are the outputs. This work is an attempt or an experiment to\nmake the previous work more efficient. Multiple new features are tried to run\nthe CRF and again tried with the Reduplicated Multiword Expression (RMWE) as\nanother feature. The CRF run with RMWE because Manipuri is rich of RMWE and\nidentification of RMWE becomes one of the necessities to bring up the result of\nPOS tagging. The new CRF system shows a Recall of 78.22%, Precision of 73.15%\nand F-measure of 75.60%. With the identification of RMWE and considering it as\na feature makes an improvement to a Recall of 80.20%, Precision of 74.31% and\nF-measure of 77.14%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5051v1",
            "date": "2012-03-22 17:45:39+00:00",
            "title": "Analysing Temporally Annotated Corpora with CAVaT",
            "authors": [
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "We present CAVaT, a tool that performs Corpus Analysis and Validation for\nTimeML. CAVaT is an open source, modular checking utility for statistical\nanalysis of features specific to temporally-annotated natural language corpora.\nIt provides reporting, highlights salient links between a variety of general\nand time-specific linguistic features, and also validates a temporal annotation\nto ensure that it is logically consistent and sufficiently annotated. Uniquely,\nCAVaT provides analysis specific to TimeML-annotated temporal information.\nTimeML is a standard for annotating temporal information in natural language\ntext. In this paper, we present the reporting part of CAVaT, and then its\nerror-checking ability, including the workings of several novel TimeML document\nverification methods. This is followed by the execution of some example tasks\nusing the tool to show relations between times, events, signals and links. We\nalso demonstrate inconsistencies in a TimeML corpus (TimeBank) that have been\ndetected with CAVaT.",
            "journal ref": "Proc. LREC (2010) 398-404",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5055v1",
            "date": "2012-03-22 17:50:08+00:00",
            "title": "Using Signals to Improve Automatic Classification of Temporal Relations",
            "authors": [
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "Temporal information conveyed by language describes how the world around us\nchanges through time. Events, durations and times are all temporal elements\nthat can be viewed as intervals. These intervals are sometimes temporally\nrelated in text. Automatically determining the nature of such relations is a\ncomplex and unsolved problem. Some words can act as \"signals\" which suggest a\ntemporal ordering between intervals. In this paper, we use these signal words\nto improve the accuracy of a recent approach to classification of temporal\nlinks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5060v1",
            "date": "2012-03-22 17:59:22+00:00",
            "title": "USFD2: Annotating Temporal Expresions and TLINKs for TempEval-2",
            "authors": [
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "We describe the University of Sheffield system used in the TempEval-2\nchallenge, USFD2. The challenge requires the automatic identification of\ntemporal entities and relations in text. USFD2 identifies and anchors temporal\nexpressions, and also attempts two of the four temporal relation assignment\ntasks. A rule-based system picks out and anchors temporal expressions, and a\nmaximum entropy classifier assigns temporal link labels, based on features that\ninclude descriptions of associated temporal signal words. USFD2 identified\ntemporal expressions successfully, and correctly classified their type in 90%\nof cases. Determining the relation between an event and time expression in the\nsame sentence was performed at 63% accuracy, the second highest score in this\npart of the challenge.",
            "journal ref": "Proc. 5th International Workshop on Semantic Evaluation (2010)\n  337-340",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5062v1",
            "date": "2012-03-22 18:05:26+00:00",
            "title": "An Annotation Scheme for Reichenbach's Verbal Tense Structure",
            "authors": [
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "In this paper we present RTMML, a markup language for the tenses of verbs and\ntemporal relations between verbs. There is a richness to tense in language that\nis not fully captured by existing temporal annotation schemata. Following\nReichenbach we present an analysis of tense in terms of abstract time points,\nwith the aim of supporting automated processing of tense and temporal relations\nin language. This allows for precise reasoning about tense in documents, and\nthe deduction of temporal relations between the times and verbal events in a\ndiscourse. We define the syntax of RTMML, and demonstrate the markup in a range\nof situations.",
            "journal ref": "Proc. 6th Joint ACL-ISO Workshop on Interoperable Semantic\n  Annotation (2011) 10-17",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5066v1",
            "date": "2012-03-22 18:08:47+00:00",
            "title": "A Corpus-based Study of Temporal Signals",
            "authors": [
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "Automatic temporal ordering of events described in discourse has been of\ngreat interest in recent years. Event orderings are conveyed in text via va\nrious linguistic mechanisms including the use of expressions such as \"before\",\n\"after\" or \"during\" that explicitly assert a temporal relation -- temporal\nsignals. In this paper, we investigate the role of temporal signals in temporal\nrelation extraction and provide a quantitative analysis of these expres sions\nin the TimeBank annotated corpus.",
            "journal ref": "Proceedings of the 6th Conference on Corpus Linguistics (2011),\n  No. 197, pp. 1--8",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5073v1",
            "date": "2012-03-22 18:34:19+00:00",
            "title": "USFD at KBP 2011: Entity Linking, Slot Filling and Temporal Bounding",
            "authors": [
                "Amev Burman",
                "Arun Jayapal",
                "Sathish Kannan",
                "Madhu Kavilikatta",
                "Ayman Alhelbawy",
                "Leon Derczynski",
                "Robert Gaizauskas"
            ],
            "abstract": "This paper describes the University of Sheffield's entry in the 2011 TAC KBP\nentity linking and slot filling tasks. We chose to participate in the\nmonolingual entity linking task, the monolingual slot filling task and the\ntemporal slot filling tasks. We set out to build a framework for\nexperimentation with knowledge base population. This framework was created, and\napplied to multiple KBP tasks. We demonstrated that our proposed framework is\neffective and suitable for collaborative development efforts, as well as useful\nin a teaching environment. Finally we present results that, while very modest,\nprovide improvements an order of magnitude greater than our 2010 attempt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5076v1",
            "date": "2012-03-22 18:45:07+00:00",
            "title": "Massively Increasing TIMEX3 Resources: A Transduction Approach",
            "authors": [
                "Leon Derczynski",
                "H\u00e9ctor Llorens",
                "Estela Saquete"
            ],
            "abstract": "Automatic annotation of temporal expressions is a research challenge of great\ninterest in the field of information extraction. Gold standard\ntemporally-annotated resources are limited in size, which makes research using\nthem difficult. Standards have also evolved over the past decade, so not all\ntemporally annotated data is in the same format. We vastly increase available\nhuman-annotated temporal expression resources by converting older format\nresources to TimeML/TIMEX3. This task is difficult due to differing annotation\nmethods. We present a robust conversion tool and a new, large temporal\nexpression resource. Using this, we evaluate our conversion process by using it\nas training data for an existing TimeML annotation tool, achieving a 0.87 F1\nmeasure -- better than any system in the TempEval-2 timex recognition exercise.",
            "journal ref": "Proceedings of the 8th international conference on Language\n  Resources and Evaluation (2012), pp. 3754-3761",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5255v1",
            "date": "2012-03-23 14:32:50+00:00",
            "title": "Post-Editing Error Correction Algorithm for Speech Recognition using Bing Spelling Suggestion",
            "authors": [
                "Youssef Bassil",
                "Mohammad Alwani"
            ],
            "abstract": "ASR short for Automatic Speech Recognition is the process of converting a\nspoken speech into text that can be manipulated by a computer. Although ASR has\nseveral applications, it is still erroneous and imprecise especially if used in\na harsh surrounding wherein the input speech is of low quality. This paper\nproposes a post-editing ASR error correction method and algorithm based on\nBing's online spelling suggestion. In this approach, the ASR recognized output\ntext is spell-checked using Bing's spelling suggestion technology to detect and\ncorrect misrecognized words. More specifically, the proposed algorithm breaks\ndown the ASR output text into several word-tokens that are submitted as search\nqueries to Bing search engine. A returned spelling suggestion implies that a\nquery is misspelled; and thus it is replaced by the suggested correction;\notherwise, no correction is performed and the algorithm continues with the next\ntoken until all tokens get validated. Experiments carried out on various\nspeeches in different languages indicated a successful decrease in the number\nof ASR errors and an improvement in the overall error correction rate. Future\nresearch can improve upon the proposed algorithm so much so that it can be\nparallelized to take advantage of multiprocessor computers.",
            "journal ref": "International Journal of Advanced Computer Science and\n  Applications, Vol.3, No.2, February 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5262v1",
            "date": "2012-03-23 14:51:05+00:00",
            "title": "ASR Context-Sensitive Error Correction Based on Microsoft N-Gram Dataset",
            "authors": [
                "Youssef Bassil",
                "Paul Semaan"
            ],
            "abstract": "At the present time, computers are employed to solve complex tasks and\nproblems ranging from simple calculations to intensive digital image processing\nand intricate algorithmic optimization problems to computationally-demanding\nweather forecasting problems. ASR short for Automatic Speech Recognition is yet\nanother type of computational problem whose purpose is to recognize human\nspoken speech and convert it into text that can be processed by a computer.\nDespite that ASR has many versatile and pervasive real-world applications,it is\nstill relatively erroneous and not perfectly solved as it is prone to produce\nspelling errors in the recognized text, especially if the ASR system is\noperating in a noisy environment, its vocabulary size is limited, and its input\nspeech is of bad or low quality. This paper proposes a post-editing ASR error\ncorrection method based on MicrosoftN-Gram dataset for detecting and correcting\nspelling errors generated by ASR systems. The proposed method comprises an\nerror detection algorithm for detecting word errors; a candidate corrections\ngeneration algorithm for generating correction suggestions for the detected\nword errors; and a context-sensitive error correction algorithm for selecting\nthe best candidate for correction. The virtue of using the Microsoft N-Gram\ndataset is that it contains real-world data and word sequences extracted from\nthe web which canmimica comprehensive dictionary of words having a large and\nall-inclusive vocabulary. Experiments conducted on numerous speeches, performed\nby different speakers, showed a remarkable reduction in ASR errors. Future\nresearch can improve upon the proposed algorithm so much so that it can be\nparallelized to take advantage of multiprocessor and distributed systems.",
            "journal ref": "Journal of Computing, Vol.4, No.1, January 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.6136v1",
            "date": "2012-03-28 02:13:39+00:00",
            "title": "Tree Transducers, Machine Translation, and Cross-Language Divergences",
            "authors": [
                "Alex Rudnick"
            ],
            "abstract": "Tree transducers are formal automata that transform trees into other trees.\nMany varieties of tree transducers have been explored in the automata theory\nliterature, and more recently, in the machine translation literature. In this\npaper I review T and xT transducers, situate them among related formalisms, and\nshow how they can be used to implement rules for machine translation systems\nthat cover all of the cross-language structural divergences described in Bonnie\nDorr's influential article on the topic. I also present an implementation of xT\ntransduction, suitable and convenient for experimenting with translation rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.0627v1",
            "date": "2012-05-03 06:49:59+00:00",
            "title": "Rule-weighted and terminal-weighted context-free grammars have identical expressivity",
            "authors": [
                "Yann Ponty"
            ],
            "abstract": "Two formalisms, both based on context-free grammars, have recently been\nproposed as a basis for a non-uniform random generation of combinatorial\nobjects. The former, introduced by Denise et al, associates weights with\nletters, while the latter, recently explored by Weinberg et al in the context\nof random generation, associates weights to transitions. In this short note, we\nuse a simple modification of the Greibach Normal Form transformation algorithm,\ndue to Blum and Koch, to show the equivalent expressivities, in term of their\ninduced distributions, of these two formalisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.1603v1",
            "date": "2012-05-08 07:01:40+00:00",
            "title": "Parsing of Myanmar sentences with function tagging",
            "authors": [
                "Win Win Thant",
                "Tin Myat Htwe",
                "Ni Lar Thein"
            ],
            "abstract": "This paper describes the use of Naive Bayes to address the task of assigning\nfunction tags and context free grammar (CFG) to parse Myanmar sentences. Part\nof the challenge of statistical function tagging for Myanmar sentences comes\nfrom the fact that Myanmar has free-phrase-order and a complex morphological\nsystem. Function tagging is a pre-processing step for parsing. In the task of\nfunction tagging, we use the functional annotated corpus and tag Myanmar\nsentences with correct segmentation, POS (part-of-speech) tagging and chunking\ninformation. We propose Myanmar grammar rules and apply context free grammar\n(CFG) to find out the parse tree of function tagged Myanmar sentences.\nExperiments show that our analysis achieves a good result with parsing of\nsimple sentences and three types of complex sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.3183v1",
            "date": "2012-05-14 20:12:06+00:00",
            "title": "A Model-Driven Probabilistic Parser Generator",
            "authors": [
                "Luis Quesada",
                "Fernando Berzal",
                "Francisco J. Cortijo"
            ],
            "abstract": "Existing probabilistic scanners and parsers impose hard constraints on the\nway lexical and syntactic ambiguities can be resolved. Furthermore, traditional\ngrammar-based parsing tools are limited in the mechanisms they allow for taking\ncontext into account. In this paper, we propose a model-driven tool that allows\nfor statistical language models with arbitrary probability estimators. Our work\non model-driven probabilistic parsing is built on top of ModelCC, a model-based\nparser generator, and enables the probabilistic interpretation and resolution\nof anaphoric, cataphoric, and recursive references in the disambiguation of\nabstract syntax graphs. In order to prove the expression power of ModelCC, we\ndescribe the design of a general-purpose natural language parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.3316v1",
            "date": "2012-05-15 10:34:05+00:00",
            "title": "Arabic Language Learning Assisted by Computer, based on Automatic Speech Recognition",
            "authors": [
                "Naim Terbeh",
                "Mounir Zrigui"
            ],
            "abstract": "This work consists of creating a system of the Computer Assisted Language\nLearning (CALL) based on a system of Automatic Speech Recognition (ASR) for the\nArabic language using the tool CMU Sphinx3 [1], based on the approach of HMM.\nTo this work, we have constructed a corpus of six hours of speech recordings\nwith a number of nine speakers. we find in the robustness to noise a grounds\nfor the choice of the HMM approach [2]. the results achieved are encouraging\nsince our corpus is made by only nine speakers, but they are always reasons\nthat open the door for other improvement works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.4298v1",
            "date": "2012-05-19 05:04:31+00:00",
            "title": "Task-specific Word-Clustering for Part-of-Speech Tagging",
            "authors": [
                "Yoav Goldberg"
            ],
            "abstract": "While the use of cluster features became ubiquitous in core NLP tasks, most\ncluster features in NLP are based on distributional similarity. We propose a\nnew type of clustering criteria, specific to the task of part-of-speech\ntagging. Instead of distributional similarity, these clusters are based on the\nbeha vior of a baseline tagger when applied to a large corpus. These cluster\nfeatures provide similar gains in accuracy to those achieved by\ndistributional-similarity derived clusters. Using both types of cluster\nfeatures together further improve tagging accuracies. We show that the method\nis effective for both the in-domain and out-of-domain scenarios for English,\nand for French, German and Italian. The effect is larger for out-of-domain\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.4387v1",
            "date": "2012-05-20 06:36:19+00:00",
            "title": "Precision-biased Parsing and High-Quality Parse Selection",
            "authors": [
                "Yoav Goldberg",
                "Michael Elhadad"
            ],
            "abstract": "We introduce precision-biased parsing: a parsing task which favors precision\nover recall by allowing the parser to abstain from decisions deemed uncertain.\nWe focus on dependency-parsing and present an ensemble method which is capable\nof assigning parents to 84% of the text tokens while being over 96% accurate on\nthese tokens. We use the precision-biased parsing task to solve the related\nhigh-quality parse-selection task: finding a subset of high-quality (accurate)\ntrees in a large collection of parsed text. We present a method for choosing\nover a third of the input trees while keeping unlabeled dependency parsing\naccuracy of 97% on these trees. We also present a method which is not based on\nan ensemble but rather on directly predicting the risk associated with\nindividual parser decisions. In addition to its efficiency, this method\ndemonstrates that a parsing system can provide reasonable estimates of\nconfidence in its predictions without relying on ensembles or aggregate corpus\ncounts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.5407v2",
            "date": "2012-05-24 11:53:41+00:00",
            "title": "FASTSUBS: An Efficient and Exact Procedure for Finding the Most Likely Lexical Substitutes Based on an N-gram Language Model",
            "authors": [
                "Deniz Yuret"
            ],
            "abstract": "Lexical substitutes have found use in areas such as paraphrasing, text\nsimplification, machine translation, word sense disambiguation, and part of\nspeech induction. However the computational complexity of accurately\nidentifying the most likely substitutes for a word has made large scale\nexperiments difficult. In this paper I introduce a new search algorithm,\nFASTSUBS, that is guaranteed to find the K most likely lexical substitutes for\na given word in a sentence based on an n-gram language model. The computation\nis sub-linear in both K and the vocabulary size V. An implementation of the\nalgorithm and a dataset with the top 100 substitutes of each token in the WSJ\nsection of the Penn Treebank are available at http://goo.gl/jzKH0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.6832v1",
            "date": "2012-01-20 17:33:47+00:00",
            "title": "Syst\u00e8me d'aide \u00e0 l'acc\u00e8s lexical : trouver le mot qu'on a sur le bout de la langue",
            "authors": [
                "Ga\u00eblle Lortal",
                "Brigitte Grau",
                "Michael Zock"
            ],
            "abstract": "The study of the Tip of the Tongue phenomenon (TOT) provides valuable clues\nand insights concerning the organisation of the mental lexicon (meaning, number\nof syllables, relation with other words, etc.). This paper describes a tool\nbased on psycho-linguistic observations concerning the TOT phenomenon. We've\nbuilt it to enable a speaker/writer to find the word he is looking for, word he\nmay know, but which he is unable to access in time. We try to simulate the TOT\nphenomenon by creating a situation where the system knows the target word, yet\nis unable to access it. In order to find the target word we make use of the\nparadigmatic and syntagmatic associations stored in the linguistic databases.\nOur experiment allows the following conclusion: a tool like SVETLAN, capable to\nstructure (automatically) a dictionary by domains can be used sucessfully to\nhelp the speaker/writer to find the word he is looking for, if it is combined\nwith a database rich in terms of paradigmatic links like EuroWordNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.0200v1",
            "date": "2012-08-01 13:06:54+00:00",
            "title": "Adaptation of pedagogical resources description standard (LOM) with the specificity of Arabic language",
            "authors": [
                "Asma Boudhief",
                "Mohsen Maraoui",
                "Mounir Zrigui"
            ],
            "abstract": "In this article we focus firstly on the principle of pedagogical indexing and\ncharacteristics of Arabic language and secondly on the possibility of adapting\nthe standard for describing learning resources used (the LOM and its\nApplication Profiles) with learning conditions such as the educational levels\nof students and their levels of understanding,... the educational context with\ntaking into account the representative elements of text, text length, ... in\nparticular, we put in relief the specificity of the Arabic language which is a\ncomplex language, characterized by its flexion, its voyellation and\nagglutination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.2777v1",
            "date": "2012-08-14 03:25:33+00:00",
            "title": "A Method for Selecting Noun Sense using Co-occurrence Relation in English-Korean Translation",
            "authors": [
                "Hyonil Kim",
                "Changil Choe"
            ],
            "abstract": "The sense analysis is still critical problem in machine translation system,\nespecially such as English-Korean translation which the syntactical different\nbetween source and target languages is very great. We suggest a method for\nselecting the noun sense using contextual feature in English-Korean\nTranslation.",
            "journal ref": "Serdica Journal of Computing, Vol.'6 No.4, 2012, pp. 401-408",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.3001v1",
            "date": "2012-08-15 00:53:39+00:00",
            "title": "More than Word Frequencies: Authorship Attribution via Natural Frequency Zoned Word Distribution Analysis",
            "authors": [
                "Zhili Chen",
                "Liusheng Huang",
                "Wei Yang",
                "Peng Meng",
                "Haibo Miao"
            ],
            "abstract": "With such increasing popularity and availability of digital text data,\nauthorships of digital texts can not be taken for granted due to the ease of\ncopying and parsing. This paper presents a new text style analysis called\nnatural frequency zoned word distribution analysis (NFZ-WDA), and then a basic\nauthorship attribution scheme and an open authorship attribution scheme for\ndigital texts based on the analysis. NFZ-WDA is based on the observation that\nall authors leave distinct intrinsic word usage traces on texts written by them\nand these intrinsic styles can be identified and employed to analyze the\nauthorship. The intrinsic word usage styles can be estimated through the\nanalysis of word distribution within a text, which is more than normal word\nfrequency analysis and can be expressed as: which groups of words are used in\nthe text; how frequently does each group of words occur; how are the\noccurrences of each group of words distributed in the text. Next, the basic\nauthorship attribution scheme and the open authorship attribution scheme\nprovide solutions for both closed and open authorship attribution problems.\nThrough analysis and extensive experimental studies, this paper demonstrates\nthe efficiency of the proposed method for authorship attribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.4079v1",
            "date": "2012-08-20 18:34:27+00:00",
            "title": "Recent Technological Advances in Natural Language Processing and Artificial Intelligence",
            "authors": [
                "Nishal Pradeepkumar Shah"
            ],
            "abstract": "A recent advance in computer technology has permitted scientists to implement\nand test algorithms that were known from quite some time (or not) but which\nwere computationally expensive. Two such projects are IBM's Jeopardy as a part\nof its DeepQA project [1] and Wolfram's Wolframalpha[2]. Both these methods\nimplement natural language processing (another goal of AI scientists) and try\nto answer questions as asked by the user. Though the goal of the two projects\nis similar, both of them have a different procedure at it's core. In the\nfollowing sections, the mechanism and history of IBM's Jeopardy and Wolfram\nalpha has been explained followed by the implications of these projects in\nrealizing Ray Kurzweil's [3] dream of passing the Turing test by 2029. A recipe\nof taking the above projects to a new level is also explained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.4503v1",
            "date": "2012-08-22 14:11:07+00:00",
            "title": "Introduction of the weight edition errors in the Levenshtein distance",
            "authors": [
                "Gueddah Hicham"
            ],
            "abstract": "In this paper, we present a new approach dedicated to correcting the spelling\nerrors of the Arabic language. This approach corrects typographical errors like\ninserting, deleting, and permutation. Our method is inspired from the\nLevenshtein algorithm, and allows a finer and better scheduling than\nLevenshtein. The results obtained are very satisfactory and encouraging, which\nshows the interest of our new approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1208.6109v1",
            "date": "2012-08-30 08:30:32+00:00",
            "title": "Average word length dynamics as indicator of cultural changes in society",
            "authors": [
                "Vladimir V. Bochkarev",
                "Anna V. Shevlyakova",
                "Valery D. Solovyev"
            ],
            "abstract": "Dynamics of average length of words in Russian and English is analysed in the\narticle. Words belonging to the diachronic text corpus Google Books Ngram and\ndated back to the last two centuries are studied. It was found out that average\nword length slightly increased in the 19th century, and then it was growing\nrapidly most of the 20th century and started decreasing over the period from\nthe end of the 20th - to the beginning of the 21th century. Words which\ncontributed mostly to increase or decrease of word average length were\nidentified. At that, content words and functional words are analysed\nseparately. Long content words contribute mostly to word average length of\nword. As it was shown, these words reflect the main tendencies of social\ndevelopment and thus, are used frequently. Change of frequency of personal\npronouns also contributes significantly to change of average word length. The\nother parameters connected with average length of word were also analysed.",
            "journal ref": "Social Evolution & History. Volume 14, number 2, p. 153-175 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.1300v1",
            "date": "2012-08-19 02:38:12+00:00",
            "title": "Input Scheme for Hindi Using Phonetic Mapping",
            "authors": [
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Written Communication on Computers requires knowledge of writing text for the\ndesired language using Computer. Mostly people do not use any other language\nbesides English. This creates a barrier. To resolve this issue we have\ndeveloped a scheme to input text in Hindi using phonetic mapping scheme. Using\nthis scheme we generate intermediate code strings and match them with\npronunciations of input text. Our system show significant success over other\ninput systems available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.1301v1",
            "date": "2012-08-19 02:31:29+00:00",
            "title": "Evaluation of Computational Grammar Formalisms for Indian Languages",
            "authors": [
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Natural Language Parsing has been the most prominent research area since the\ngenesis of Natural Language Processing. Probabilistic Parsers are being\ndeveloped to make the process of parser development much easier, accurate and\nfast. In Indian context, identification of which Computational Grammar\nFormalism is to be used is still a question which needs to be answered. In this\npaper we focus on this problem and try to analyze different formalisms for\nIndian languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.2400v1",
            "date": "2012-09-11 19:18:26+00:00",
            "title": "Identification of Fertile Translations in Medical Comparable Corpora: a Morpho-Compositional Approach",
            "authors": [
                "Estelle Delpech",
                "B\u00e9atrice Daille",
                "Emmanuel Morin",
                "Claire Lemaire"
            ],
            "abstract": "This paper defines a method for lexicon in the biomedical domain from\ncomparable corpora. The method is based on compositional translation and\nexploits morpheme-level translation equivalences. It can generate translations\nfor a large variety of morphologically constructed words and can also generate\n'fertile' translations. We show that fertile translations increase the overall\nquality of the extracted lexicon for English to French translation.",
            "journal ref": "AMTA, San Diego, CA : United States (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.6238v1",
            "date": "2012-09-25 21:05:08+00:00",
            "title": "Natural Language Processing - A Survey",
            "authors": [
                "Kevin Mote"
            ],
            "abstract": "The utility and power of Natural Language Processing (NLP) seems destined to\nchange our technological society in profound and fundamental ways. However\nthere are, to date, few accessible descriptions of the science of NLP that have\nbeen written for a popular audience, or even for an audience of intelligent,\nbut uninitiated scientists. This paper aims to provide just such an overview.\nIn short, the objective of this article is to describe the purpose, procedures\nand practical applications of NLP in a clear, balanced, and readable way. We\nwill examine the most recent literature describing the methods and processes of\nNLP, analyze some of the challenges that researchers are faced with, and\nbriefly survey some of the current and future applications of this science to\nIT research in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.0074v1",
            "date": "2012-11-01 02:10:06+00:00",
            "title": "Transition-Based Dependency Parsing With Pluggable Classifiers",
            "authors": [
                "Alex Rudnick"
            ],
            "abstract": "In principle, the design of transition-based dependency parsers makes it\npossible to experiment with any general-purpose classifier without other\nchanges to the parsing algorithm. In practice, however, it often takes\nsubstantial software engineering to bridge between the different\nrepresentations used by two software packages. Here we present extensions to\nMaltParser that allow the drop-in use of any classifier conforming to the\ninterface of the Weka machine learning package, a wrapper for the TiMBL\nmemory-based learner to this interface, and experiments on multilingual\ndependency parsing with a variety of classifiers. While earlier work had\nsuggested that memory-based learners might be a good choice for low-resource\nparsing scenarios, we cannot support that hypothesis in this work. We observed\nthat support-vector machines give better parsing performance than the\nmemory-based learner, regardless of the size of the training set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.0498v1",
            "date": "2012-11-02 17:37:06+00:00",
            "title": "Detecting English Writing Styles For Non-native Speakers",
            "authors": [
                "Rami Al-Rfou'"
            ],
            "abstract": "Analyzing writing styles of non-native speakers is a challenging task. In\nthis paper, we analyze the comments written in the discussion pages of the\nEnglish Wikipedia. Using learning algorithms, we are able to detect native\nspeakers' writing style with an accuracy of 74%. Given the diversity of the\nEnglish Wikipedia users and the large number of languages they speak, we\nmeasure the similarities among their native languages by comparing the\ninfluence they have on their English writing style. Our results show that\nlanguages known to have the same origin and development path have similar\nfootprint on their speakers' English writing style. To enable further studies,\nthe dataset we extracted from Wikipedia will be made available publicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.3643v1",
            "date": "2012-11-15 16:24:25+00:00",
            "title": "A Principled Approach to Grammars for Controlled Natural Languages and Predictive Editors",
            "authors": [
                "Tobias Kuhn"
            ],
            "abstract": "Controlled natural languages (CNL) with a direct mapping to formal logic have\nbeen proposed to improve the usability of knowledge representation systems,\nquery interfaces, and formal specifications. Predictive editors are a popular\napproach to solve the problem that CNLs are easy to read but hard to write.\nSuch predictive editors need to be able to \"look ahead\" in order to show all\npossible continuations of a given unfinished sentence. Such lookahead features,\nhowever, are difficult to implement in a satisfying way with existing grammar\nframeworks, especially if the CNL supports complex nonlocal structures such as\nanaphoric references. Here, methods and algorithms are presented for a new\ngrammar notation called Codeco, which is specifically designed for controlled\nnatural languages and predictive editors. A parsing approach for Codeco based\non an extended chart parsing algorithm is presented. A large subset of Attempto\nControlled English (ACE) has been represented in Codeco. Evaluation of this\ngrammar and the parser implementation shows that the approach is practical,\nadequate and efficient.",
            "journal ref": "Journal of Logic, Language and Information, 22(1), 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.4161v1",
            "date": "2012-11-17 20:27:06+00:00",
            "title": "Semantic Polarity of Adjectival Predicates in Online Reviews",
            "authors": [
                "Ae-Lim Ahn",
                "\u00c9ric Laporte",
                "Jee-Sun Nam"
            ],
            "abstract": "Web users produce more and more documents expressing opinions. Because these\nhave become important resources for customers and manufacturers, many have\nfocused on them. Opinions are often expressed through adjectives with positive\nor negative semantic values. In extracting information from users' opinion in\nonline reviews, exact recognition of the semantic polarity of adjectives is one\nof the most important requirements. Since adjectives have different semantic\norientations according to contexts, it is not satisfying to extract opinion\ninformation without considering the semantic and lexical relations between the\nadjectives and the feature nouns appropriate to a given domain. In this paper,\nwe present a classification of adjectives by polarity, and we analyze\nadjectives that are undetermined in the absence of contexts. Our research\nshould be useful for accurately predicting semantic orientations of opinion\nsentences, and should be taken into account before relying on an automatic\nmethods.",
            "journal ref": "Seoul International Conference on Linguistics (SICOL'10), Seoul :\n  Korea, Republic Of (2010)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0140v1",
            "date": "2012-03-31 21:53:56+00:00",
            "title": "Roget's Thesaurus as a Lexical Resource for Natural Language Processing",
            "authors": [
                "Mario Jarmasz"
            ],
            "abstract": "WordNet proved that it is possible to construct a large-scale electronic\nlexical database on the principles of lexical semantics. It has been accepted\nand used extensively by computational linguists ever since it was released.\nInspired by WordNet's success, we propose as an alternative a similar resource,\nbased on the 1987 Penguin edition of Roget's Thesaurus of English Words and\nPhrases.\n  Peter Mark Roget published his first Thesaurus over 150 years ago. Countless\nwriters, orators and students of the English language have used it.\nComputational linguists have employed Roget's for almost 50 years in Natural\nLanguage Processing, however hesitated in accepting Roget's Thesaurus because a\nproper machine tractable version was not available.\n  This dissertation presents an implementation of a machine-tractable version\nof the 1987 Penguin edition of Roget's Thesaurus - the first implementation of\nits kind to use an entire current edition. It explains the steps necessary for\ntaking a machine-readable file and transforming it into a tractable system.\nThis involves converting the lexical material into a format that can be more\neasily exploited, identifying data structures and designing classes to\ncomputerize the Thesaurus. Roget's organization is studied in detail and\ncontrasted with WordNet's.\n  We show two applications of the computerized Thesaurus: computing semantic\nsimilarity between words and phrases, and building lexical chains in a text.\nThe experiments are performed using well-known benchmarks and the results are\ncompared to those of other systems that use Roget's, WordNet and statistical\ntechniques. Roget's has turned out to be an excellent resource for measuring\nsemantic similarity; lexical chains are easily built but more difficult to\nevaluate. We also explain ways in which Roget's Thesaurus and WordNet can be\ncombined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0184v1",
            "date": "2012-04-01 09:28:20+00:00",
            "title": "Parallel Spell-Checking Algorithm Based on Yahoo! N-Grams Dataset",
            "authors": [
                "Youssef Bassil"
            ],
            "abstract": "Spell-checking is the process of detecting and sometimes providing\nsuggestions for incorrectly spelled words in a text. Basically, the larger the\ndictionary of a spell-checker is, the higher is the error detection rate;\notherwise, misspellings would pass undetected. Unfortunately, traditional\ndictionaries suffer from out-of-vocabulary and data sparseness problems as they\ndo not encompass large vocabulary of words indispensable to cover proper names,\ndomain-specific terms, technical jargons, special acronyms, and terminologies.\nAs a result, spell-checkers will incur low error detection and correction rate\nand will fail to flag all errors in the text. This paper proposes a new\nparallel shared-memory spell-checking algorithm that uses rich real-world word\nstatistics from Yahoo! N-Grams Dataset to correct non-word and real-word errors\nin computer text. Essentially, the proposed algorithm can be divided into three\nsub-algorithms that run in a parallel fashion: The error detection algorithm\nthat detects misspellings, the candidates generation algorithm that generates\ncorrection suggestions, and the error correction algorithm that performs\ncontextual error correction. Experiments conducted on a set of text articles\ncontaining misspellings, showed a remarkable spelling error correction rate\nthat resulted in a radical reduction of both non-word and real-word errors in\nelectronic text. In a further study, the proposed algorithm is to be optimized\nfor message-passing systems so as to become more flexible and less costly to\nscale over distributed machines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0191v1",
            "date": "2012-04-01 10:34:38+00:00",
            "title": "OCR Post-Processing Error Correction Algorithm using Google Online Spelling Suggestion",
            "authors": [
                "Youssef Bassil",
                "Mohammad Alwani"
            ],
            "abstract": "With the advent of digital optical scanners, a lot of paper-based books,\ntextbooks, magazines, articles, and documents are being transformed into an\nelectronic version that can be manipulated by a computer. For this purpose,\nOCR, short for Optical Character Recognition was developed to translate scanned\ngraphical text into editable computer text. Unfortunately, OCR is still\nimperfect as it occasionally mis-recognizes letters and falsely identifies\nscanned text, leading to misspellings and linguistics errors in the OCR output\ntext. This paper proposes a post-processing context-based error correction\nalgorithm for detecting and correcting OCR non-word and real-word errors. The\nproposed algorithm is based on Google's online spelling suggestion which\nharnesses an internal database containing a huge collection of terms and word\nsequences gathered from all over the web, convenient to suggest possible\nreplacements for words that have been misspelled during the OCR process.\nExperiments carried out revealed a significant improvement in OCR error\ncorrection rate. Future research can improve upon the proposed algorithm so\nmuch so that it can be parallelized and executed over multiprocessing\nplatforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0245v1",
            "date": "2012-04-01 17:04:13+00:00",
            "title": "Roget's Thesaurus and Semantic Similarity",
            "authors": [
                "Mario Jarmasz",
                "Stan Szpakowicz"
            ],
            "abstract": "We have implemented a system that measures semantic similarity using a\ncomputerized 1987 Roget's Thesaurus, and evaluated it by performing a few\ntypical tests. We compare the results of these tests with those produced by\nWordNet-based similarity measures. One of the benchmarks is Miller and Charles'\nlist of 30 noun pairs to which human judges had assigned similarity measures.\nWe correlate these measures with those computed by several NLP systems. The 30\npairs can be traced back to Rubenstein and Goodenough's 65 pairs, which we have\nalso studied. Our Roget's-based system gets correlations of .878 for the\nsmaller and .818 for the larger list of noun pairs; this is quite close to the\n.885 that Resnik obtained when he employed humans to replicate the Miller and\nCharles experiment. We further evaluate our measure by using Roget's and\nWordNet to answer 80 TOEFL, 50 ESL and 300 Reader's Digest questions: the\ncorrect synonym must be selected amongst a group of four words. Our system gets\n78.75%, 82.00% and 74.33% of the questions respectively.",
            "journal ref": "Proceedings of Conference on Recent Advances in Natural Language\n  Processing (RANLP 2003), Borovets, Bulgaria, September 2003, 212-219",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0257v1",
            "date": "2012-04-01 19:19:36+00:00",
            "title": "Not As Easy As It Seems: Automating the Construction of Lexical Chains Using Roget's Thesaurus",
            "authors": [
                "Mario Jarmasz",
                "Stan Szpakowicz"
            ],
            "abstract": "Morris and Hirst present a method of linking significant words that are about\nthe same topic. The resulting lexical chains are a means of identifying\ncohesive regions in a text, with applications in many natural language\nprocessing tasks, including text summarization. The first lexical chains were\nconstructed manually using Roget's International Thesaurus. Morris and Hirst\nwrote that automation would be straightforward given an electronic thesaurus.\nAll applications so far have used WordNet to produce lexical chains, perhaps\nbecause adequate electronic versions of Roget's were not available until\nrecently. We discuss the building of lexical chains using an electronic version\nof Roget's Thesaurus. We implement a variant of the original algorithm, and\nexplain the necessary design decisions. We include a comparison with other\nimplementations.",
            "journal ref": "Proceedings of the 16th Canadian Conference on Artificial\n  Intelligence (AI 2003), Halifax, Canada, June 2003. Lecture Notes in Computer\n  Science 2671, Springer-Verlag 2003, 544-549",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0258v1",
            "date": "2012-04-01 19:25:29+00:00",
            "title": "Roget's Thesaurus: a Lexical Resource to Treasure",
            "authors": [
                "Mario Jarmasz",
                "Stan Szpakowicz"
            ],
            "abstract": "This paper presents the steps involved in creating an electronic lexical\nknowledge base from the 1987 Penguin edition of Roget's Thesaurus. Semantic\nrelations are labelled with the help of WordNet. The two resources are compared\nin a qualitative and quantitative manner. Differences in the organization of\nthe lexical material are discussed, as well as the possibility of merging both\nresources.",
            "journal ref": "Proceedings of the NAACL WordNet and Other Lexical Resources\n  workshop. Pittsburgh, June 2001, 186 - 188",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.2847v2",
            "date": "2012-04-12 22:01:27+00:00",
            "title": "Segmentation Similarity and Agreement",
            "authors": [
                "Chris Fournier",
                "Diana Inkpen"
            ],
            "abstract": "We propose a new segmentation evaluation metric, called segmentation\nsimilarity (S), that quantifies the similarity between two segmentations as the\nproportion of boundaries that are not transformed when comparing them using\nedit distance, essentially using edit distance as a penalty function and\nscaling penalties by segmentation size. We propose several adapted\ninter-annotator agreement coefficients which use S that are suitable for\nsegmentation. We show that S is configurable enough to suit a wide variety of\nsegmentation evaluations, and is an improvement upon the state of the art. We\nalso propose using inter-annotator agreement coefficients to evaluate automatic\nsegmenters in terms of human performance.",
            "journal ref": "Proceedings of the 2012 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, pages 152-161, Montr\\'eal, Canada, June 3-8, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.3800v1",
            "date": "2012-04-17 14:14:26+00:00",
            "title": "Indus script corpora, archaeo-metallurgy and Meluhha (Mleccha)",
            "authors": [
                "Srinivasan Kalyanaraman"
            ],
            "abstract": "Jules Bloch's work on formation of the Marathi language has to be expanded\nfurther to provide for a study of evolution and formation of Indian languages\nin the Indian language union (sprachbund). The paper analyses the stages in the\nevolution of early writing systems which began with the evolution of counting\nin the ancient Near East. A stage anterior to the stage of syllabic\nrepresentation of sounds of a language, is identified. Unique geometric shapes\nrequired for tokens to categorize objects became too large to handle to\nabstract hundreds of categories of goods and metallurgical processes during the\nproduction of bronze-age goods. About 3500 BCE, Indus script as a writing\nsystem was developed to use hieroglyphs to represent the 'spoken words'\nidentifying each of the goods and processes. A rebus method of representing\nsimilar sounding words of the lingua franca of the artisans was used in Indus\nscript. This method is recognized and consistently applied for the lingua\nfranca of the Indian sprachbund. That the ancient languages of India,\nconstituted a sprachbund (or language union) is now recognized by many\nlinguists. The sprachbund area is proximate to the area where most of the Indus\nscript inscriptions were discovered, as documented in the corpora. That\nhundreds of Indian hieroglyphs continued to be used in metallurgy is evidenced\nby their use on early punch-marked coins. This explains the combined use of\nsyllabic scripts such as Brahmi and Kharoshti together with the hieroglyphs on\nRampurva copper bolt, and Sohgaura copper plate from about 6th century\nBCE.Indian hieroglyphs constitute a writing system for meluhha language and are\nrebus representations of archaeo-metallurgy lexemes. The rebus principle was\nemployed by the early scripts and can legitimately be used to decipher the\nIndus script, after secure pictorial identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.5852v1",
            "date": "2012-04-26 07:44:18+00:00",
            "title": "Context-sensitive Spelling Correction Using Google Web 1T 5-Gram Information",
            "authors": [
                "Youssef Bassil",
                "Mohammad Alwani"
            ],
            "abstract": "In computing, spell checking is the process of detecting and sometimes\nproviding spelling suggestions for incorrectly spelled words in a text.\nBasically, a spell checker is a computer program that uses a dictionary of\nwords to perform spell checking. The bigger the dictionary is, the higher is\nthe error detection rate. The fact that spell checkers are based on regular\ndictionaries, they suffer from data sparseness problem as they cannot capture\nlarge vocabulary of words including proper names, domain-specific terms,\ntechnical jargons, special acronyms, and terminologies. As a result, they\nexhibit low error detection rate and often fail to catch major errors in the\ntext. This paper proposes a new context-sensitive spelling correction method\nfor detecting and correcting non-word and real-word errors in digital text\ndocuments. The approach hinges around data statistics from Google Web 1T 5-gram\ndata set which consists of a big volume of n-gram word sequences, extracted\nfrom the World Wide Web. Fundamentally, the proposed method comprises an error\ndetector that detects misspellings, a candidate spellings generator based on a\ncharacter 2-gram model that generates correction suggestions, and an error\ncorrector that performs contextual error correction. Experiments conducted on a\nset of text documents from different domains and containing misspellings,\nshowed an outstanding spelling error correction rate and a drastic reduction of\nboth non-word and real-word errors. In a further study, the proposed algorithm\nis to be parallelized so as to lower the computational cost of the error\ndetection and correction processes.",
            "journal ref": "Computer and Information Science, Vol. 5, No. 3, May 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.6364v1",
            "date": "2012-04-28 03:52:21+00:00",
            "title": "A Corpus-based Evaluation of a Domain-specific Text to Knowledge Mapping Prototype",
            "authors": [
                "Rushdi Shams",
                "Adel Elsayed",
                "Quazi Mah-Zereen Akter"
            ],
            "abstract": "The aim of this paper is to evaluate a Text to Knowledge Mapping (TKM)\nPrototype. The prototype is domain-specific, the purpose of which is to map\ninstructional text onto a knowledge domain. The context of the knowledge domain\nis DC electrical circuit. During development, the prototype has been tested\nwith a limited data set from the domain. The prototype reached a stage where it\nneeds to be evaluated with a representative linguistic data set called corpus.\nA corpus is a collection of text drawn from typical sources which can be used\nas a test data set to evaluate NLP systems. As there is no available corpus for\nthe domain, we developed and annotated a representative corpus. The evaluation\nof the prototype considers two of its major components- lexical components and\nknowledge model. Evaluation on lexical components enriches the lexical\nresources of the prototype like vocabulary and grammar structures. This leads\nthe prototype to parse a reasonable amount of sentences in the corpus. While\ndealing with the lexicon was straight forward, the identification and\nextraction of appropriate semantic relations was much more involved. It was\nnecessary, therefore, to manually develop a conceptual structure for the domain\nto formulate a domain-specific framework of semantic relations. The framework\nof semantic relationsthat has resulted from this study consisted of 55\nrelations, out of which 42 have inverse relations. We also conducted rhetorical\nanalysis on the corpus to prove its representativeness in conveying semantic.\nFinally, we conducted a topical and discourse analysis on the corpus to analyze\nthe coverage of discourse by the prototype.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.0245v2",
            "date": "2012-07-01 21:13:05+00:00",
            "title": "Adversarial Evaluation for Models of Natural Language",
            "authors": [
                "Noah A. Smith"
            ],
            "abstract": "We now have a rich and growing set of modeling tools and algorithms for\ninducing linguistic structure from text that is less than fully annotated. In\nthis paper, we discuss some of the weaknesses of our current methodology. We\npresent a new abstract framework for evaluating natural language processing\n(NLP) models in general and unsupervised NLP models in particular. The central\nidea is to make explicit certain adversarial roles among researchers, so that\nthe different roles in an evaluation are more clearly defined and performers of\nall roles are offered ways to make measurable contributions to the larger goal.\nAdopting this approach may help to characterize model successes and failures by\nencouraging earlier consideration of error analysis. The framework can be\ninstantiated in a variety of ways, simulating some familiar intrinsic and\nextrinsic evaluations as well as some new evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.1420v1",
            "date": "2012-07-04 16:27:56+00:00",
            "title": "Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars",
            "authors": [
                "Luke S. Zettlemoyer",
                "Michael Collins"
            ],
            "abstract": "This paper addresses the problem of mapping natural language sentences to\nlambda-calculus encodings of their meaning. We describe a learning algorithm\nthat takes as input a training set of sentences labeled with expressions in the\nlambda calculus. The algorithm induces a grammar for the problem, along with a\nlog-linear model that represents a distribution over syntactic and semantic\nanalyses conditioned on the input sentence. We apply the method to the task of\nlearning natural language interfaces to databases and show that the learned\nparsers outperform previous methods in two benchmark database domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.2714v1",
            "date": "2012-07-11 17:31:11+00:00",
            "title": "Clustering based approach extracting collocations",
            "authors": [
                "Mohamed Achraf Ben Mohamed",
                "Mounir Zrigui",
                "Mohsen Maraoui"
            ],
            "abstract": "The following study presents a collocation extraction approach based on\nclustering technique. This study uses a combination of several classical\nmeasures which cover all aspects of a given corpus then it suggests separating\nbigrams found in the corpus in several disjoint groups according to the\nprobability of presence of collocations. This will allow excluding groups where\nthe presence of collocations is very unlikely and thus reducing in a meaningful\nway the search space.",
            "journal ref": "4th International Conference on Arabic Language Processing. CITALA\n  2012. May 2nd -- 3rd 2012, Rabat, Morocco",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.3932v1",
            "date": "2012-07-17 10:14:24+00:00",
            "title": "Automatic Segmentation of Manipuri (Meiteilon) Word into Syllabic Units",
            "authors": [
                "Kishorjit Nongmeikapam",
                "Vidya Raj RK",
                "Oinam Imocha Singh",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "The work of automatic segmentation of a Manipuri language (or Meiteilon) word\ninto syllabic units is demonstrated in this paper. This language is a scheduled\nIndian language of Tibeto-Burman origin, which is also a very highly\nagglutinative language. This language usages two script: a Bengali script and\nMeitei Mayek (Script). The present work is based on the second script. An\nalgorithm is designed so as to identify mainly the syllables of Manipuri origin\nword. The result of the algorithm shows a Recall of 74.77, Precision of 91.21\nand F-Score of 82.18 which is a reasonable score with the first attempt of such\nkind for this language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.4625v1",
            "date": "2012-07-19 12:04:26+00:00",
            "title": "Appropriate Nouns with Obligatory Modifiers",
            "authors": [
                "E. Laporte"
            ],
            "abstract": "The notion of appropriate sequence as introduced by Z. Harris provides a\npowerful syntactic way of analysing the detailed meaning of various sentences,\nincluding ambiguous ones. In an adjectival sentence like 'The leather was\nyellow', the introduction of an appropriate noun, here 'colour', specifies\nwhich quality the adjective describes. In some other adjectival sentences with\nan appropriate noun, that noun plays the same part as 'colour' and seems to be\nrelevant to the description of the adjective. These appropriate nouns can\nusually be used in elementary sentences like 'The leather had some colour', but\nin many cases they have a more or less obligatory modifier. For example, you\ncan hardly mention that an object has a colour without qualifying that colour\nat all. About 300 French nouns are appropriate in at least one adjectival\nsentence and have an obligatory modifier. They enter in a number of sentence\nstructures related by several syntactic transformations. The appropriateness of\nthe noun and the fact that the modifier is obligatory are reflected in these\ntransformations. The description of these syntactic phenomena provides a basis\nfor a classification of these nouns. It also concerns the lexical properties of\nthousands of predicative adjectives, and in particular the relations between\nthe sentence without the noun : 'The leather was yellow' and the adjectival\nsentence with the noun : 'The colour of the leather was yellow'.",
            "journal ref": "Language research, Seoul National University 31, 2 (1995) 251-289",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.5328v3",
            "date": "2012-07-23 09:02:18+00:00",
            "title": "A prototype for projecting HPSG syntactic lexica towards LMF",
            "authors": [
                "Kais Haddar",
                "H\u00e9la Fehri",
                "Laurent Romary"
            ],
            "abstract": "The comparative evaluation of Arabic HPSG grammar lexica requires a deep\nstudy of their linguistic coverage. The complexity of this task results mainly\nfrom the heterogeneity of the descriptive components within those lexica\n(underlying linguistic resources and different data categories, for example).\nIt is therefore essential to define more homogeneous representations, which in\nturn will enable us to compare them and eventually merge them. In this context,\nwe present a method for comparing HPSG lexica based on a rule system. This\nmethod is implemented within a prototype for the projection from Arabic HPSG to\na normalised pivot language compliant with LMF (ISO 24613 - Lexical Markup\nFramework) and serialised using a TEI (Text Encoding Initiative) based\nrepresentation. The design of this system is based on an initial study of the\nHPSG formalism looking at its adequacy for the representation of Arabic, and\nfrom this, we identify the appropriate feature structures corresponding to each\nArabic lexical category and their possible LMF counterparts.",
            "journal ref": "Journal of Language Technology and Computational Linguistics 27, 1\n  (2012) 21-46",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.4567v2",
            "date": "2012-10-16 20:22:56+00:00",
            "title": "Gender identity and lexical variation in social media",
            "authors": [
                "David Bamman",
                "Jacob Eisenstein",
                "Tyler Schnoebelen"
            ],
            "abstract": "We present a study of the relationship between gender, linguistic style, and\nsocial networks, using a novel corpus of 14,000 Twitter users. Prior\nquantitative work on gender often treats this social variable as a female/male\nbinary; we argue for a more nuanced approach. By clustering Twitter users, we\nfind a natural decomposition of the dataset into various styles and topical\ninterests. Many clusters have strong gender orientations, but their use of\nlinguistic resources sometimes directly conflicts with the population-level\nlanguage statistics. We view these clusters as a more accurate reflection of\nthe multifaceted nature of gendered language styles. Previous corpus-based work\nhas also had little to say about individuals whose linguistic styles defy\npopulation-level gender patterns. To identify such individuals, we train a\nstatistical classifier, and measure the classifier confidence for each\nindividual in the dataset. Examining individuals whose language does not match\nthe classifier's model for their gender, we find that they have social networks\nthat include significantly fewer same-gender social connections and that, in\ngeneral, social network homophily is correlated with the use of same-gender\nlanguage markers. Pairing computational methods and social theory thus offers a\nnew perspective on how gender emerges as individuals position themselves\nrelative to audiences, topics, and mainstream gender norms.",
            "journal ref": "Journal of Sociolinguistics 18 (2014) 135-160",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5486v2",
            "date": "2012-10-19 17:49:06+00:00",
            "title": "A Lightweight Stemmer for Gujarati",
            "authors": [
                "Juhi Ameta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Gujarati is a resource poor language with almost no language processing tools\nbeing available. In this paper we have shown an implementation of a rule based\nstemmer of Gujarati. We have shown the creation of rules for stemming and the\nrichness in morphology that Gujarati possesses. We have also evaluated our\nresults by verifying it with a human expert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5517v1",
            "date": "2012-10-19 17:59:56+00:00",
            "title": "Design of English-Hindi Translation Memory for Efficient Translation",
            "authors": [
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Developing parallel corpora is an important and a difficult activity for\nMachine Translation. This requires manual annotation by Human Translators.\nTranslating same text again is a useless activity. There are tools available to\nimplement this for European Languages, but no such tool is available for Indian\nLanguages. In this paper we present a tool for Indian Languages which not only\nprovides automatic translations of the previously available translation but\nalso provides multiple translations, in cases where a sentence has multiple\ntranslations, in ranked list of suggestive translations for a sentence.\nMoreover this tool also lets translators have global and local saving options\nof their work, so that they may share it with others, which further lightens\nthe task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5751v1",
            "date": "2012-10-21 19:06:11+00:00",
            "title": "Extraction of domain-specific bilingual lexicon from comparable corpora: compositional translation and ranking",
            "authors": [
                "Estelle Delpech",
                "B\u00e9atrice Daille",
                "Emmanuel Morin",
                "Claire Lemaire"
            ],
            "abstract": "This paper proposes a method for extracting translations of morphologically\nconstructed terms from comparable corpora. The method is based on compositional\ntranslation and exploits translation equivalences at the morpheme-level, which\nallows for the generation of \"fertile\" translations (translation pairs in which\nthe target term has more words than the source term). Ranking methods relying\non corpus-based and translation-based features are used to select the best\ncandidate translation. We obtain an average precision of 91% on the Top1\ncandidate translation. The method was tested on two language pairs\n(English-French and English-German) and with a small specialized comparable\ncorpora (400k words per language).",
            "journal ref": "COLING 2012, Mumbai : India (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5965v1",
            "date": "2012-10-22 16:40:35+00:00",
            "title": "Classification Analysis Of Authorship Fiction Texts in The Space Of Semantic Fields",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "The use of naive Bayesian classifier (NB) and the classifier by the k nearest\nneighbors (kNN) in classification semantic analysis of authors' texts of\nEnglish fiction has been analysed. The authors' works are considered in the\nvector space the basis of which is formed by the frequency characteristics of\nsemantic fields of nouns and verbs. Highly precise classification of authors'\ntexts in the vector space of semantic fields indicates about the presence of\nparticular spheres of author's idiolect in this space which characterizes the\nindividual author's style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.7282v1",
            "date": "2012-10-27 02:34:35+00:00",
            "title": "The Hangulphabet: A Descriptive Alphabet",
            "authors": [
                "Robert Bishop",
                "Ruggero Micheletto"
            ],
            "abstract": "This paper describes the Hangulphabet, a new writing system that should prove\nuseful in a number of contexts. Using the Hangulphabet, a user can instantly\nsee voicing, manner and place of articulation of any phoneme found in human\nlanguage. The Hangulphabet places consonant graphemes on a grid with the x-axis\nrepresenting the place of articulation and the y-axis representing manner of\narticulation. Each individual grapheme contains radicals from both axes where\nthe points intersect. The top radical represents manner of articulation where\nthe bottom represents place of articulation. A horizontal line running through\nthe middle of the bottom radical represents voicing. For vowels, place of\narticulation is located on a grid that represents the position of the tongue in\nthe mouth. This grid is similar to that of the IPA vowel chart (International\nPhonetic Association, 1999). The difference with the Hangulphabet being the\ntrapezoid representing the vocal apparatus is on a slight tilt. Place of\narticulation for a vowel is represented by a breakout figure from the grid.\nThis system can be used as an alternative to the International Phonetic\nAlphabet (IPA) or as a complement to it. Beginning students of linguistics may\nfind it particularly useful. A Hangulphabet font has been created to facilitate\nswitching between the Hangulphabet and the IPA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.8440v1",
            "date": "2012-10-31 18:57:14+00:00",
            "title": "Large Scale Language Modeling in Automatic Speech Recognition",
            "authors": [
                "Ciprian Chelba",
                "Dan Bikel",
                "Maria Shugrina",
                "Patrick Nguyen",
                "Shankar Kumar"
            ],
            "abstract": "Large language models have been proven quite beneficial for a variety of\nautomatic speech recognition tasks in Google. We summarize results on Voice\nSearch and a few YouTube speech transcription tasks to highlight the impact\nthat one can expect from increasing both the amount of training data, and the\nsize of the language model estimated from such data. Depending on the task,\navailability and amount of training data used, language model size and amount\nof work and care put into integrating them in the lattice rescoring step we\nobserve reductions in word error rate between 6% and 10% relative, for systems\non a wide range of operating points between 17% and 52% word error rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.1192v2",
            "date": "2012-12-05 22:10:04+00:00",
            "title": "Using external sources of bilingual information for on-the-fly word alignment",
            "authors": [
                "Miquel Espl\u00e0-Gomis",
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Mikel L. Forcada"
            ],
            "abstract": "In this paper we present a new and simple language-independent method for\nword-alignment based on the use of external sources of bilingual information\nsuch as machine translation systems. We show that the few parameters of the\naligner can be trained on a very small corpus, which leads to results\ncomparable to those obtained by the state-of-the-art tool GIZA++ in terms of\nprecision. Regarding other metrics, such as alignment error rate or F-measure,\nthe parametric aligner, when trained on a very small gold-standard (450 pairs\nof sentences), provides results comparable to those produced by GIZA++ when\ntrained on an in-domain corpus of around 10,000 pairs of sentences.\nFurthermore, the results obtained indicate that the training is\ndomain-independent, which enables the use of the trained aligner 'on the fly'\non any new pair of sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.3138v1",
            "date": "2012-12-13 11:47:09+00:00",
            "title": "Identifying Metaphor Hierarchies in a Corpus Analysis of Finance Articles",
            "authors": [
                "Aaron Georw",
                "Mark Keane"
            ],
            "abstract": "Using a corpus of over 17,000 financial news reports (involving over 10M\nwords), we perform an analysis of the argument-distributions of the UP- and\nDOWN-verbs used to describe movements of indices, stocks, and shares. Using\nmeasures of the overlap in the argument distributions of these verbs and\nk-means clustering of their distributions, we advance evidence for the proposal\nthat the metaphors referred to by these verbs are organised into hierarchical\nstructures of superordinate and subordinate groups.",
            "journal ref": "Proceedings of the 33rd Annual Meeting of the Cognitive Science\n  Society (CogSci '11), Boston, MA, USA, 20-23 July, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.3139v2",
            "date": "2012-12-13 11:53:25+00:00",
            "title": "Identifying Metaphoric Antonyms in a Corpus Analysis of Finance Articles",
            "authors": [
                "Aaron Gerow",
                "Mark Keane"
            ],
            "abstract": "Using a corpus of 17,000+ financial news reports (involving over 10M words),\nwe perform an analysis of the argument-distributions of the UP and DOWN verbs\nused to describe movements of indices, stocks and shares. In Study 1\nparticipants identified antonyms of these verbs in a free-response task and a\nmatching task from which the most commonly identified antonyms were compiled.\nIn Study 2, we determined whether the argument-distributions for the verbs in\nthese antonym-pairs were sufficiently similar to predict the most\nfrequently-identified antonym. Cosine similarity correlates moderately with the\nproportions of antonym-pairs identified by people (r = 0.31). More\nimpressively, 87% of the time the most frequently-identified antonym is either\nthe first- or second-most similar pair in the set of alternatives. The\nimplications of these results for distributional approaches to determining\nmetaphoric knowledge are discussed.",
            "journal ref": "Proceedings of the 33rd Annual Meeting of the Cognitive Science\n  Society (CogSci '11), Boston, MA, USA, 20-23 July, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.3162v1",
            "date": "2012-12-13 13:00:55+00:00",
            "title": "Diachronic Variation in Grammatical Relations",
            "authors": [
                "Aaron Gerow",
                "Khurshid Ahmad"
            ],
            "abstract": "We present a method of finding and analyzing shifts in grammatical relations\nfound in diachronic corpora. Inspired by the econometric technique of measuring\nreturn and volatility instead of relative frequencies, we propose them as a way\nto better characterize changes in grammatical patterns like nominalization,\nmodification and comparison. To exemplify the use of these techniques, we\nexamine a corpus of NIPS papers and report trends which manifest at the token,\npart-of-speech and grammatical levels. Building up from frequency observations\nto a second-order analysis, we show that shifts in frequencies overlook deeper\ntrends in language, even when part-of-speech information is included. Examining\ntoken, POS and grammatical levels of variation enables a summary view of\ndiachronic text as a whole. We conclude with a discussion about how these\nmethods can inform intuitions about specialist domains as well as changes in\nlanguage use as a whole.",
            "journal ref": "Proceedings of the 24th International Conference on Computational\n  Linguistics (COLING 2012), Mumbai, India",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.4315v1",
            "date": "2012-12-18 11:33:50+00:00",
            "title": "Assessing Sentiment Strength in Words Prior Polarities",
            "authors": [
                "Lorenzo Gatti",
                "Marco Guerini"
            ],
            "abstract": "Many approaches to sentiment analysis rely on lexica where words are tagged\nwith their prior polarity - i.e. if a word out of context evokes something\npositive or something negative. In particular, broad-coverage resources like\nSentiWordNet provide polarities for (almost) every word. Since words can have\nmultiple senses, we address the problem of how to compute the prior polarity of\na word starting from the polarity of each sense and returning its polarity\nstrength as an index between -1 and 1. We compare 14 such formulae that appear\nin the literature, and assess which one best approximates the human judgement\nof prior polarities, with both regression and classification models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.4674v1",
            "date": "2012-12-19 14:40:38+00:00",
            "title": "Natural Language Understanding Based on Semantic Relations between Sentences",
            "authors": [
                "Hyeok Kong"
            ],
            "abstract": "In this paper, we define event expression over sentences of natural language\nand semantic relations between events. Based on this definition, we formally\nconsider text understanding process having events as basic unit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1201.1192v1",
            "date": "2012-01-05 15:22:05+00:00",
            "title": "Formalization of semantic network of image constructions in electronic content",
            "authors": [
                "Oleg Bisikalo",
                "Irina Kravchuk"
            ],
            "abstract": "A formal theory based on a binary operator of directional associative\nrelation is constructed in the article and an understanding of an associative\nnormal form of image constructions is introduced. A model of a commutative\nsemigroup, which provides a presentation of a sentence as three components of\nan interrogative linguistic image construction, is considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1201.2010v1",
            "date": "2012-01-10 10:33:18+00:00",
            "title": "Recognizing Bangla Grammar using Predictive Parser",
            "authors": [
                "K. M. Azharul Hasan",
                "Al-Mahmud",
                "Amit Mondal",
                "Amit Saha"
            ],
            "abstract": "We describe a Context Free Grammar (CFG) for Bangla language and hence we\npropose a Bangla parser based on the grammar. Our approach is very much general\nto apply in Bangla Sentences and the method is well accepted for parsing a\nlanguage of a grammar. The proposed parser is a predictive parser and we\nconstruct the parse table for recognizing Bangla grammar. Using the parse table\nwe recognize syntactical mistakes of Bangla sentences when there is no entry\nfor a terminal in the parse table. If a natural language can be successfully\nparsed then grammar checking from this language becomes possible. The proposed\nscheme is based on Top down parsing method and we have avoided the left\nrecursion of the CFG using the idea of left factoring.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1201.6224v2",
            "date": "2012-01-30 14:26:31+00:00",
            "title": "Wikipedia Arborification and Stratified Explicit Semantic Analysis",
            "authors": [
                "Yannis Haralambous",
                "Vitaly Klyuev"
            ],
            "abstract": "[This is the translation of paper \"Arborification de Wikip\\'edia et analyse\ns\\'emantique explicite stratifi\\'ee\" submitted to TALN 2012.]\n  We present an extension of the Explicit Semantic Analysis method by\nGabrilovich and Markovitch. Using their semantic relatedness measure, we weight\nthe Wikipedia categories graph. Then, we extract a minimal spanning tree, using\nChu-Liu & Edmonds' algorithm. We define a notion of stratified tfidf where the\nstratas, for a given Wikipedia page and a given term, are the classical tfidf\nand categorical tfidfs of the term in the ancestor categories of the page\n(ancestors in the sense of the minimal spanning tree). Our method is based on\nthis stratified tfidf, which adds extra weight to terms that \"survive\" when\nclimbing up the category tree. We evaluate our method by a text classification\non the WikiNews corpus: it increases precision by 18%. Finally, we provide\nhints for future research",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.0042v1",
            "date": "2012-05-31 22:02:32+00:00",
            "title": "Language Acquisition in Computers",
            "authors": [
                "Megan Belzner",
                "Sean Colin-Ellerin",
                "Jorge H. Roman"
            ],
            "abstract": "This project explores the nature of language acquisition in computers, guided\nby techniques similar to those used in children. While existing natural\nlanguage processing methods are limited in scope and understanding, our system\naims to gain an understanding of language from first principles and hence\nminimal initial input. The first portion of our system was implemented in Java\nand is focused on understanding the morphology of language using bigrams. We\nuse frequency distributions and differences between them to define and\ndistinguish languages. English and French texts were analyzed to determine a\ndifference threshold of 55 before the texts are considered to be in different\nlanguages, and this threshold was verified using Spanish texts. The second\nportion of our system focuses on gaining an understanding of the syntax of a\nlanguage using a recursive method. The program uses one of two possible methods\nto analyze given sentences based on either sentence patterns or surrounding\nwords. Both methods have been implemented in C++. The program is able to\nunderstand the structure of simple sentences and learn new words. In addition,\nwe have provided some suggestions regarding future work and potential\nextensions of the existing program.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.0381v1",
            "date": "2012-06-02 13:23:18+00:00",
            "title": "UNL Based Bangla Natural Text Conversion - Predicate Preserving Parser Approach",
            "authors": [
                "Md. Nawab Yousuf Ali",
                "Shamim Ripon",
                "Shaikh Muhammad Allayear"
            ],
            "abstract": "Universal Networking Language (UNL) is a declarative formal language that is\nused to represent semantic data extracted from natural language texts. This\npaper presents a novel approach to converting Bangla natural language text into\nUNL using a method known as Predicate Preserving Parser (PPP) technique. PPP\nperforms morphological, syntactic and semantic, and lexical analysis of text\nsynchronously. This analysis produces a semantic-net like structure represented\nusing UNL. We demonstrate how Bangla texts are analyzed following the PPP\ntechnique to produce UNL documents which can then be translated into any other\nsuitable natural language facilitating the opportunity to develop a universal\nlanguage translation method via UNL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.1066v1",
            "date": "2012-06-05 20:15:35+00:00",
            "title": "Hedge detection as a lens on framing in the GMO debates: A position paper",
            "authors": [
                "Eunsol Choi",
                "Chenhao Tan",
                "Lillian Lee",
                "Cristian Danescu-Niculescu-Mizil",
                "Jennifer Spindel"
            ],
            "abstract": "Understanding the ways in which participants in public discussions frame\ntheir arguments is important in understanding how public opinion is formed. In\nthis paper, we adopt the position that it is time for more\ncomputationally-oriented research on problems involving framing. In the\ninterests of furthering that goal, we propose the following specific,\ninteresting and, we believe, relatively accessible question: In the controversy\nregarding the use of genetically-modified organisms (GMOs) in agriculture, do\npro- and anti-GMO articles differ in whether they choose to adopt a\n\"scientific\" tone?\n  Prior work on the rhetoric and sociology of science suggests that hedging may\ndistinguish popular-science text from text written by professional scientists\nfor their colleagues. We propose a detailed approach to studying whether hedge\ndetection can be used to understanding scientific framing in the GMO debates,\nand provide corpora to facilitate this study. Some of our preliminary analyses\nsuggest that hedges occur less frequently in scientific discourse than in\npopular text, a finding that contradicts prior assertions in the literature. We\nhope that our initial work and data will encourage others to pursue this\npromising line of inquiry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.2009v1",
            "date": "2012-06-10 09:27:51+00:00",
            "title": "Developing a model for a text database indexed pedagogically for teaching the Arabic language",
            "authors": [
                "Asma Boudhief",
                "Mohsen Maraoui",
                "Mounir Zrigui"
            ],
            "abstract": "In this memory we made the design of an indexing model for Arabic language\nand adapting standards for describing learning resources used (the LOM and\ntheir application profiles) with learning conditions such as levels education\nof students, their levels of understanding...the pedagogical context with\ntaking into account the repre-sentative elements of the text, text's\nlength,...in particular, we highlight the specificity of the Arabic language\nwhich is a complex language, characterized by its flexion, its voyellation and\nits agglutination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.4522v1",
            "date": "2012-06-20 15:06:48+00:00",
            "title": "BADREX: In situ expansion and coreference of biomedical abbreviations using dynamic regular expressions",
            "authors": [
                "Phil Gooch"
            ],
            "abstract": "BADREX uses dynamically generated regular expressions to annotate term\ndefinition-term abbreviation pairs, and corefers unpaired acronyms and\nabbreviations back to their initial definition in the text. Against the\nMedstract corpus BADREX achieves precision and recall of 98% and 97%, and\nagainst a much larger corpus, 90% and 85%, respectively. BADREX yields improved\nperformance over previous approaches, requires no training data and allows\nruntime customisation of its input parameters. BADREX is freely available from\nhttps://github.com/philgooch/BADREX-Biomedical-Abbreviation-Expander as a\nplugin for the General Architecture for Text Engineering (GATE) framework and\nis licensed under the GPLv3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.5333v2",
            "date": "2012-06-22 22:30:44+00:00",
            "title": "TempEval-3: Evaluating Events, Time Expressions, and Temporal Relations",
            "authors": [
                "Naushad UzZaman",
                "Hector Llorens",
                "James Allen",
                "Leon Derczynski",
                "Marc Verhagen",
                "James Pustejovsky"
            ],
            "abstract": "We describe the TempEval-3 task which is currently in preparation for the\nSemEval-2013 evaluation exercise. The aim of TempEval is to advance research on\ntemporal information processing. TempEval-3 follows on from previous TempEval\nevents, incorporating: a three-part task structure covering event, temporal\nexpression and temporal relation extraction; a larger dataset; and single\noverall task quality scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.0116v1",
            "date": "2012-02-01 08:36:50+00:00",
            "title": "Inference and Plausible Reasoning in a Natural Language Understanding System Based on Object-Oriented Semantics",
            "authors": [
                "Yuriy Ostapov"
            ],
            "abstract": "Algorithms of inference in a computer system oriented to input and semantic\nprocessing of text information are presented. Such inference is necessary for\nlogical questions when the direct comparison of objects from a question and\ndatabase can not give a result. The following classes of problems are\nconsidered: a check of hypotheses for persons and non-typical actions, the\ndetermination of persons and circumstances for non-typical actions, planning\nactions, the determination of event cause and state of persons. To form an\nanswer both deduction and plausible reasoning are used. As a knowledge domain\nunder consideration is social behavior of persons, plausible reasoning is based\non laws of social psychology. Proposed algorithms of inference and plausible\nreasoning can be realized in computer systems closely connected with text\nprocessing (criminology, operation of business, medicine, document systems).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.1054v1",
            "date": "2012-02-06 06:33:03+00:00",
            "title": "Considering a resource-light approach to learning verb valencies",
            "authors": [
                "Alex Rudnick"
            ],
            "abstract": "Here we describe work on learning the subcategories of verbs in a\nmorphologically rich language using only minimal linguistic resources. Our goal\nis to learn verb subcategorizations for Quechua, an under-resourced\nmorphologically rich language, from an unannotated corpus. We compare results\nfrom applying this approach to an unannotated Arabic corpus with those achieved\nby processing the same text in treebank form. The original plan was to use only\na morphological analyzer and an unannotated corpus, but experiments suggest\nthat this approach by itself will not be effective for learning the\ncombinatorial potential of Arabic verbs in general. The lower bound on\nresources for acquiring this information is somewhat higher, apparently\nrequiring a a part-of-speech tagger and chunker for most languages, and a\nmorphological disambiguater for Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.1568v2",
            "date": "2012-02-08 00:49:36+00:00",
            "title": "Beyond Sentiment: The Manifold of Human Emotions",
            "authors": [
                "Seungyeon Kim",
                "Fuxin Li",
                "Guy Lebanon",
                "Irfan Essa"
            ],
            "abstract": "Sentiment analysis predicts the presence of positive or negative emotions in\na text document. In this paper we consider higher dimensional extensions of the\nsentiment concept, which represent a richer set of human emotions. Our approach\ngoes beyond previous work in that our model contains a continuous manifold\nrather than a finite set of human emotions. We investigate the resulting model,\ncompare it to psychological observations, and explore its predictive\ncapabilities. Besides obtaining significant improvements over a baseline\nwithout manifold, we are also able to visualize different notions of positive\nsentiment in different domains.",
            "journal ref": "Proceedings of the 16 International Conference on Artificial\n  Intelligence and Statistics (AISTATS) 2013, Scottsdale, AZ, USA. Volume 31 of\n  JMLR: W&CP 31",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.6266v1",
            "date": "2012-02-28 16:04:36+00:00",
            "title": "Realisation d'un systeme de reconnaissance automatique de la parole arabe base sur CMU Sphinx",
            "authors": [
                "Ali Sadiqui",
                "Noureddine Chenfour"
            ],
            "abstract": "This paper presents the continuation of the work completed by Satori and all.\n[SCH07] by the realization of an automatic speech recognition system (ASR) for\nArabic language based SPHINX 4 system. The previous work was limited to the\nrecognition of the first ten digits, whereas the present work is a remarkable\nprojection consisting in continuous Arabic speech recognition with a rate of\nrecognition of surroundings 96%.",
            "journal ref": "Ann. Univ. Tibiscus Comp. Sci. Series VIII / 1 (2010), 27-40",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.2293v1",
            "date": "2012-03-11 00:04:57+00:00",
            "title": "Categories of Emotion names in Web retrieved texts",
            "authors": [
                "Sergey Petrov",
                "Jose F. Fontanari",
                "Leonid I. Perlovsky"
            ],
            "abstract": "The categorization of emotion names, i.e., the grouping of emotion words that\nhave similar emotional connotations together, is a key tool of Social\nPsychology used to explore people's knowledge about emotions. Without\nexception, the studies following that research line were based on the gauging\nof the perceived similarity between emotion names by the participants of the\nexperiments. Here we propose and examine a new approach to study the categories\nof emotion names - the similarities between target emotion names are obtained\nby comparing the contexts in which they appear in texts retrieved from the\nWorld Wide Web. This comparison does not account for any explicit semantic\ninformation; it simply counts the number of common words or lexical items used\nin the contexts. This procedure allows us to write the entries of the\nsimilarity matrix as dot products in a linear vector space of contexts. The\nproperties of this matrix were then explored using Multidimensional Scaling\nAnalysis and Hierarchical Clustering. Our main findings, namely, the underlying\ndimension of the emotion space and the categories of emotion names, were\nconsistent with those based on people's judgments of emotion names\nsimilarities.",
            "journal ref": "International Journal of Psychology and Behavioral Sciences 2\n  (2012) 173-184",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.2299v1",
            "date": "2012-03-11 00:45:01+00:00",
            "title": "A Cross-cultural Corpus of Annotated Verbal and Nonverbal Behaviors in Receptionist Encounters",
            "authors": [
                "Maxim Makatchev",
                "Reid Simmons",
                "Majd Sakr"
            ],
            "abstract": "We present the first annotated corpus of nonverbal behaviors in receptionist\ninteractions, and the first nonverbal corpus (excluding the original video and\naudio data) of service encounters freely available online. Native speakers of\nAmerican English and Arabic participated in a naturalistic role play at\nreception desks of university buildings in Doha, Qatar and Pittsburgh, USA.\nTheir manually annotated nonverbal behaviors include gaze direction, hand and\nhead gestures, torso positions, and facial expressions. We discuss possible\nuses of the corpus and envision it to become a useful tool for the human-robot\ninteraction community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.3227v1",
            "date": "2012-03-14 22:06:42+00:00",
            "title": "Generalisation of language and knowledge models for corpus analysis",
            "authors": [
                "Anton Loss"
            ],
            "abstract": "This paper takes new look on language and knowledge modelling for corpus\nlinguistics. Using ideas of Chaitin, a line of argument is made against\nlanguage/knowledge separation in Natural Language Processing. A simplistic\nmodel, that generalises approaches to language and knowledge, is proposed. One\nof hypothetical consequences of this model is Strong AI.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1203.3586v1",
            "date": "2012-03-15 22:56:29+00:00",
            "title": "Automated Text Summarization Base on Lexicales Chain and graph Using of WordNet and Wikipedia Knowledge Base",
            "authors": [
                "Mohsen Pourvali",
                "Mohammad Saniee Abadeh"
            ],
            "abstract": "The technology of automatic document summarization is maturing and may\nprovide a solution to the information overload problem. Nowadays, document\nsummarization plays an important role in information retrieval. With a large\nvolume of documents, presenting the user with a summary of each document\ngreatly facilitates the task of finding the desired documents. Document\nsummarization is a process of automatically creating a compressed version of a\ngiven document that provides useful information to users, and multi-document\nsummarization is to produce a summary delivering the majority of information\ncontent from a set of documents about an explicit or implicit main topic. The\nlexical cohesion structure of the text can be exploited to determine the\nimportance of a sentence/phrase. Lexical chains are useful tools to analyze the\nlexical cohesion structure in a text .In this paper we consider the effect of\nthe use of lexical cohesion features in Summarization, And presenting a\nalgorithm base on the knowledge base. Ours algorithm at first find the correct\nsense of any word, Then constructs the lexical chains, remove Lexical chains\nthat less score than other, detects topics roughly from lexical chains,\nsegments the text with respect to the topics and selects the most important\nsentences. The experimental results on an open benchmark datasets from DUC01\nand DUC02 show that our proposed approach can improve the performance compared\nto sate-of-the-art summarization approaches.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 9,\n  Issue 1, No 3, January 2012",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1203.4176v1",
            "date": "2012-03-10 10:16:10+00:00",
            "title": "SignsWorld; Deeping Into the Silence World and Hearing Its Signs (State of the Art)",
            "authors": [
                "A. M. Riad",
                "Hamdy K. Elmonier",
                "Samaa. M. Shohieb",
                "A. S. Asem"
            ],
            "abstract": "Automatic speech processing systems are employed more and more often in real\nenvironments. Although the underlying speech technology is mostly language\nindependent, differences between languages with respect to their structure and\ngrammar have substantial effect on the recognition systems performance. In this\npaper, we present a review of the latest developments in the sign language\nrecognition research in general and in the Arabic sign language (ArSL) in\nspecific. This paper also presents a general framework for improving the deaf\ncommunity communication with the hearing people that is called SignsWorld. The\noverall goal of the SignsWorld project is to develop a vision-based technology\nfor recognizing and translating continuous Arabic sign language ArSL.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 4, No 1, Feb 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.5084v1",
            "date": "2012-03-22 19:19:02+00:00",
            "title": "A Data Driven Approach to Query Expansion in Question Answering",
            "authors": [
                "Leon Derczynski",
                "Jun Wang",
                "Robert Gaizauskas",
                "Mark A. Greenwood"
            ],
            "abstract": "Automated answering of natural language questions is an interesting and\nuseful problem to solve. Question answering (QA) systems often perform\ninformation retrieval at an initial stage. Information retrieval (IR)\nperformance, provided by engines such as Lucene, places a bound on overall\nsystem performance. For example, no answer bearing documents are retrieved at\nlow ranks for almost 40% of questions.\n  In this paper, answer texts from previous QA evaluations held as part of the\nText REtrieval Conferences (TREC) are paired with queries and analysed in an\nattempt to identify performance-enhancing words. These words are then used to\nevaluate the performance of a query expansion method.\n  Data driven extension words were found to help in over 70% of difficult\nquestions. These words can be used to improve and evaluate query expansion\nmethods. Simple blind relevance feedback (RF) was correctly predicted as\nunlikely to help overall performance, and an possible explanation is provided\nfor its low value in IR for QA.",
            "journal ref": "Proc. IR4QA Workshop (2008) 34-41",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.6845v1",
            "date": "2012-03-30 16:16:43+00:00",
            "title": "Information Retrieval Systems Adapted to the Biomedical Domain",
            "authors": [
                "M\u00f3nica Marrero",
                "Sonia S\u00e1nchez-Cuadrado",
                "Juli\u00e1n Urbano",
                "Jorge Morato",
                "Jos\u00e9-Antonio Moreiro"
            ],
            "abstract": "The terminology used in Biomedicine shows lexical peculiarities that have\nrequired the elaboration of terminological resources and information retrieval\nsystems with specific functionalities. The main characteristics are the high\nrates of synonymy and homonymy, due to phenomena such as the proliferation of\npolysemic acronyms and their interaction with common language. Information\nretrieval systems in the biomedical domain use techniques oriented to the\ntreatment of these lexical peculiarities. In this paper we review some of the\ntechniques used in this domain, such as the application of Natural Language\nProcessing (BioNLP), the incorporation of lexical-semantic resources, and the\napplication of Named Entity Recognition (BioNER). Finally, we present the\nevaluation methods adopted to assess the suitability of these techniques for\nretrieving biomedical resources.",
            "journal ref": "El Profesional de la Informaci\\'on (2010), 19-3",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.1564v1",
            "date": "2012-05-08 00:29:48+00:00",
            "title": "Characterizing Ranked Chinese Syllable-to-Character Mapping Spectrum: A Bridge Between the Spoken and Written Chinese Language",
            "authors": [
                "Wentian Li"
            ],
            "abstract": "One important aspect of the relationship between spoken and written Chinese\nis the ranked syllable-to-character mapping spectrum, which is the ranked list\nof syllables by the number of characters that map to the syllable. Previously,\nthis spectrum is analyzed for more than 400 syllables without distinguishing\nthe four intonations. In the current study, the spectrum with 1280 toned\nsyllables is analyzed by logarithmic function, Beta rank function, and\npiecewise logarithmic function. Out of the three fitting functions, the\ntwo-piece logarithmic function fits the data the best, both by the smallest sum\nof squared errors (SSE) and by the lowest Akaike information criterion (AIC)\nvalue. The Beta rank function is the close second. By sampling from a Poisson\ndistribution whose parameter value is chosen from the observed data, we\nempirically estimate the $p$-value for testing the\ntwo-piece-logarithmic-function being better than the Beta rank function\nhypothesis, to be 0.16. For practical purposes, the piecewise logarithmic\nfunction and the Beta rank function can be considered a tie.",
            "journal ref": "Journal of Quantitative Linguistics, 20, 153-167 (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.1794v1",
            "date": "2012-05-08 19:54:13+00:00",
            "title": "A Novel Method For Speech Segmentation Based On Speakers' Characteristics",
            "authors": [
                "Behrouz Abdolali",
                "Hossein Sameti"
            ],
            "abstract": "Speech Segmentation is the process change point detection for partitioning an\ninput audio stream into regions each of which corresponds to only one audio\nsource or one speaker. One application of this system is in Speaker Diarization\nsystems. There are several methods for speaker segmentation; however, most of\nthe Speaker Diarization Systems use BIC-based Segmentation methods. The main\ngoal of this paper is to propose a new method for speaker segmentation with\nhigher speed than the current methods - e.g. BIC - and acceptable accuracy. Our\nproposed method is based on the pitch frequency of the speech. The accuracy of\nthis method is similar to the accuracy of common speaker segmentation methods.\nHowever, its computation cost is much less than theirs. We show that our method\nis about 2.4 times faster than the BIC-based method, while the average accuracy\nof pitch-based method is slightly higher than that of the BIC-based method.",
            "journal ref": "B. Abdolali, H. Sameti \"A Novel Method for Speech Segmentation\n  based on Speakers' Specifications\", Signal & Image Processing: An\n  International Journal (SIPIJ) Vol.3, No.2, pp. 65-78, April 2012",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1205.1975v1",
            "date": "2012-05-09 13:19:46+00:00",
            "title": "Expressivity of Time-Varying Graphs and the Power of Waiting in Dynamic Networks",
            "authors": [
                "Arnaud Casteigts",
                "Paola Flocchini",
                "Emmanuel Godard",
                "Nicola Santoro",
                "Masafumi Yamashita"
            ],
            "abstract": "In infrastructure-less highly dynamic networks, computing and performing even\nbasic tasks (such as routing and broadcasting) is a very challenging activity\ndue to the fact that connectivity does not necessarily hold, and the network\nmay actually be disconnected at every time instant. Clearly the task of\ndesigning protocols for these networks is less difficult if the environment\nallows waiting (i.e., it provides the nodes with store-carry-forward-like\nmechanisms such as local buffering) than if waiting is not feasible. No\nquantitative corroborations of this fact exist (e.g., no answer to the\nquestion: how much easier?). In this paper, we consider these qualitative\nquestions about dynamic networks, modeled as time-varying (or evolving) graphs,\nwhere edges exist only at some times.\n  We examine the difficulty of the environment in terms of the expressivity of\nthe corresponding time-varying graph; that is in terms of the language\ngenerated by the feasible journeys in the graph. We prove that the set of\nlanguages $L_{nowait}$ when no waiting is allowed contains all computable\nlanguages. On the other end, using algebraic properties of quasi-orders, we\nprove that $L_{wait}$ is just the family of regular languages. In other words,\nwe prove that, when waiting is no longer forbidden, the power of the accepting\nautomaton (difficulty of the environment) drops drastically from being as\npowerful as a Turing machine, to becoming that of a Finite-State machine. This\n(perhaps surprisingly large) gap is a measure of the computational power of\nwaiting.\n  We also study bounded waiting; that is when waiting is allowed at a node only\nfor at most $d$ time units. We prove the negative result that $L_{wait[d]} =\nL_{nowait}$; that is, the expressivity decreases only if the waiting is finite\nbut unpredictable (i.e., under the control of the protocol designer and not of\nthe environment).",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1205.6396v1",
            "date": "2012-05-29 15:37:46+00:00",
            "title": "Effective Listings of Function Stop words for Twitter",
            "authors": [
                "Murphy Choy"
            ],
            "abstract": "Many words in documents recur very frequently but are essentially meaningless\nas they are used to join words together in a sentence. It is commonly\nunderstood that stop words do not contribute to the context or content of\ntextual documents. Due to their high frequency of occurrence, their presence in\ntext mining presents an obstacle to the understanding of the content in the\ndocuments. To eliminate the bias effects, most text mining software or\napproaches make use of stop words list to identify and remove those words.\nHowever, the development of such top words list is difficult and inconsistent\nbetween textual sources. This problem is further aggravated by sources such as\nTwitter which are highly repetitive or similar in nature. In this paper, we\nwill be examining the original work using term frequency, inverse document\nfrequency and term adjacency for developing a stop words list for the Twitter\ndata source. We propose a new technique using combinatorial values as an\nalternative measure to effectively list out stop words.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1208.3047v1",
            "date": "2012-08-15 07:16:14+00:00",
            "title": "Parallelization of Maximum Entropy POS Tagging for Bahasa Indonesia with MapReduce",
            "authors": [
                "Arif Nurwidyantoro",
                "Edi Winarko"
            ],
            "abstract": "In this paper, MapReduce programming model is used to parallelize training\nand tagging proceess in Maximum Entropy part of speech tagging for Bahasa\nIndonesia. In training process, MapReduce model is implemented dictionary,\ntagtoken, and feature creation. In tagging process, MapReduce is implemented to\ntag lines of document in parallel. The training experiments showed that total\ntraining time using MapReduce is faster, but its result reading time inside the\nprocess slow down the total training time. The tagging experiments using\ndifferent number of map and reduce process showed that MapReduce implementation\ncould speedup the tagging process. The fastest tagging result is showed by\ntagging process using 1,000,000 word corpus and 30 map process.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1208.6268v4",
            "date": "2012-08-30 19:09:24+00:00",
            "title": "Authorship Identification in Bengali Literature: a Comparative Analysis",
            "authors": [
                "Tanmoy Chakraborty"
            ],
            "abstract": "Stylometry is the study of the unique linguistic styles and writing behaviors\nof individuals. It belongs to the core task of text categorization like\nauthorship identification, plagiarism detection etc. Though reasonable number\nof studies have been conducted in English language, no major work has been done\nso far in Bengali. In this work, We will present a demonstration of authorship\nidentification of the documents written in Bengali. We adopt a set of\nfine-grained stylistic features for the analysis of the text and use them to\ndevelop two different models: statistical similarity model consisting of three\nmeasures and their combination, and machine learning model with Decision Tree,\nNeural Network and SVM. Experimental results show that SVM outperforms other\nstate-of-the-art methods after 10-fold cross validations. We also validate the\nrelative importance of each stylistic feature to show that some of them remain\nconsistently significant in every model used in this experiment.",
            "journal ref": "Chakraborty, T., Authorship Identification in Bengali Literature:\n  a Comparative Analysis, Proceedings of COLING 2012: Demonstration Papers,\n  December, 2012, pp. 41-50",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.0249v1",
            "date": "2012-09-03 06:00:04+00:00",
            "title": "Robopinion: Opinion Mining Framework Inspired by Autonomous Robot Navigation",
            "authors": [
                "M. A. El-Dosuky",
                "M. Z. Rashad",
                "T. T. Hamza",
                "A. H. EL-Bassiouny"
            ],
            "abstract": "Data association methods are used by autonomous robots to find matches\nbetween the current landmarks and the new set of observed features. We seek a\nframework for opinion mining to benefit from advancements in autonomous robot\nnavigation in both research and development",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1209.2163v1",
            "date": "2012-09-10 22:03:11+00:00",
            "title": "Modeling controversies in the press: the case of the abnormal bees' death",
            "authors": [
                "Alexandre Delano\u00eb",
                "Serge Galam"
            ],
            "abstract": "The controversy about the cause(s) of abnormal death of bee colonies in\nFrance is investigated through an extensive analysis of the french speaking\npress. A statistical analysis of textual data is first performed on the lexicon\nused by journalists to describe the facts and to present associated\ninformations during the period 1998-2010. Three states are identified to\nexplain the phenomenon. The first state asserts a unique cause, the second one\nfocuses on multifactor causes and the third one states the absence of current\nproof. Assigning each article to one of the three states, we are able to follow\nthe associated opinion dynamics among the journalists over 13 years. Then, we\napply the Galam sequential probabilistic model of opinion dynamic to those\ndata. Assuming journalists are either open mind or inflexible about their\nrespective opinions, the results are reproduced precisely provided we account\nfor a series of annual changes in the proportions of respective inflexibles.\nThe results shed a new counter intuitive light on the various pressure supposed\nto apply on the journalists by either chemical industries or beekeepers and\nexperts or politicians. The obtained dynamics of respective inflexibles shows\nthe possible effect of lobbying, the inertia of the debate and the net\nadvantage gained by the first whistleblowers.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1209.2341v2",
            "date": "2012-09-11 15:02:20+00:00",
            "title": "Leveraging Sentiment to Compute Word Similarity",
            "authors": [
                "A. R. Balamurali",
                "Subhabrata Mukherjee",
                "Akshat Malu",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we introduce a new WordNet based similarity metric, SenSim,\nwhich incorporates sentiment content (i.e., degree of positive or negative\nsentiment) of the words being compared to measure the similarity between them.\nThe proposed metric is based on the hypothesis that knowing the sentiment is\nbeneficial in measuring the similarity. To verify this hypothesis, we measure\nand compare the annotator agreement for 2 annotation strategies: 1) sentiment\ninformation of a pair of words is considered while annotating and 2) sentiment\ninformation of a pair of words is not considered while annotating.\nInter-annotator correlation scores show that the agreement is better when the\ntwo annotators consider sentiment information while assigning a similarity\nscore to a pair of words. We use this hypothesis to measure the similarity\nbetween a pair of words. Specifically, we represent each word as a vector\ncontaining sentiment scores of all the content words in the WordNet gloss of\nthe sense of that word. These sentiment scores are derived from a sentiment\nlexicon. We then measure the cosine similarity between the two vectors. We\nperform both intrinsic and extrinsic evaluation of SenSim and compare the\nperformance with other widely usedWordNet similarity metrics.",
            "journal ref": "In Proceedings of The 6th International Global Wordnet Conference\n  (GWC 2012), Matsue, Japan, January, 9-13, 2012",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.2352v2",
            "date": "2012-09-11 15:39:18+00:00",
            "title": "Feature Specific Sentiment Analysis for Product Reviews",
            "authors": [
                "Subhabrata Mukherjee",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we present a novel approach to identify feature specific\nexpressions of opinion in product reviews with different features and mixed\nemotions. The objective is realized by identifying a set of potential features\nin the review and extracting opinion expressions about those features by\nexploiting their associations. Capitalizing on the view that more closely\nassociated words come together to express an opinion about a certain feature,\ndependency parsing is used to identify relations between the opinion\nexpressions. The system learns the set of significant relations to be used by\ndependency parsing and a threshold parameter which allows us to merge closely\nassociated opinion expressions. The data requirement is minimal as this is a\none time learning of the domain independent parameters. The associations are\nrepresented in the form of a graph which is partitioned to finally retrieve the\nopinion expression describing the user specified feature. We show that the\nsystem achieves a high accuracy across all domains and performs at par with\nstate-of-the-art systems despite its data limitations.",
            "journal ref": "COMPUTATIONAL LINGUISTICS AND INTELLIGENT TEXT PROCESSING, Lecture\n  Notes in Computer Science, 2012, Volume 7181/2012, 475-487",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.2493v2",
            "date": "2012-09-12 04:33:08+00:00",
            "title": "WikiSent : Weakly Supervised Sentiment Analysis Through Extractive Summarization With Wikipedia",
            "authors": [
                "Subhabrata Mukherjee",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "This paper describes a weakly supervised system for sentiment analysis in the\nmovie review domain. The objective is to classify a movie review into a\npolarity class, positive or negative, based on those sentences bearing opinion\non the movie alone. The irrelevant text, not directly related to the reviewer\nopinion on the movie, is left out of analysis. Wikipedia incorporates the world\nknowledge of movie-specific features in the system which is used to obtain an\nextractive summary of the review, consisting of the reviewer's opinions about\nthe specific aspects of the movie. This filters out the concepts which are\nirrelevant or objective with respect to the given movie. The proposed system,\nWikiSent, does not require any labeled data for training. The only weak\nsupervision arises out of the usage of resources like WordNet, Part-of-Speech\nTagger and Sentiment Lexicons by virtue of their construction. WikiSent\nachieves a considerable accuracy improvement over the baseline and has a better\nor comparable accuracy to the existing semi-supervised and unsupervised systems\nin the domain, on the same dataset. We also perform a general movie review\ntrend analysis using WikiSent to find the trend in movie-making and the public\nacceptance in terms of movie genre, year of release and polarity.",
            "journal ref": "Lecture Notes in Computer Science Volume 7523, 2012, pp 774-793",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.2495v2",
            "date": "2012-09-12 04:39:37+00:00",
            "title": "TwiSent: A Multistage System for Analyzing Sentiment in Twitter",
            "authors": [
                "Subhabrata Mukherjee",
                "Akshat Malu",
                "A. R. Balamurali",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we present TwiSent, a sentiment analysis system for Twitter.\nBased on the topic searched, TwiSent collects tweets pertaining to it and\ncategorizes them into the different polarity classes positive, negative and\nobjective. However, analyzing micro-blog posts have many inherent challenges\ncompared to the other text genres. Through TwiSent, we address the problems of\n1) Spams pertaining to sentiment analysis in Twitter, 2) Structural anomalies\nin the text in the form of incorrect spellings, nonstandard abbreviations,\nslangs etc., 3) Entity specificity in the context of the topic searched and 4)\nPragmatics embedded in text. The system performance is evaluated on manually\nannotated gold standard data and on an automatically annotated tweet set based\non hashtags. It is a common practise to show the efficacy of a supervised\nsystem on an automatically annotated dataset. However, we show that such a\nsystem achieves lesser classification accurcy when tested on generic twitter\ndataset. We also show that our system performs much better than an existing\nsystem.",
            "journal ref": "In Proceedings of The 21st ACM Conference on Information and\n  Knowledge Management (CIKM), 2012 as a poster",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.3126v1",
            "date": "2012-09-14 08:45:26+00:00",
            "title": "Beyond Stemming and Lemmatization: Ultra-stemming to Improve Automatic Text Summarization",
            "authors": [
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "In Automatic Text Summarization, preprocessing is an important phase to\nreduce the space of textual representation. Classically, stemming and\nlemmatization have been widely used for normalizing words. However, even using\nnormalization on large texts, the curse of dimensionality can disturb the\nperformance of summarizers. This paper describes a new method for normalization\nof words to further reduce the space of representation. We propose to reduce\neach word to its initial letters, as a form of Ultra-stemming. The results show\nthat Ultra-stemming not only preserve the content of summaries produced by this\nrepresentation, but often the performances of the systems can be dramatically\nimproved. Summaries on trilingual corpora were evaluated automatically with\nFresa. Results confirm an increase in the performance, regardless of summarizer\nsystem used.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.4471v1",
            "date": "2012-09-20 09:21:29+00:00",
            "title": "Stemmer for Serbian language",
            "authors": [
                "Nikola Milo\u0161evi\u0107"
            ],
            "abstract": "In linguistic morphology and information retrieval, stemming is the process\nfor reducing inflected (or sometimes derived) words to their stem, base or root\nform; generally a written word form. In this work is presented suffix stripping\nstemmer for Serbian language, one of the highly inflectional languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.0418v1",
            "date": "2012-11-02 11:09:44+00:00",
            "title": "Verbalizing Ontologies in Controlled Baltic Languages",
            "authors": [
                "Normunds Gr\u016bz\u012btis",
                "Gunta Ne\u0161pore",
                "Baiba Saul\u012bte"
            ],
            "abstract": "Controlled natural languages (mostly English-based) recently have emerged as\nseemingly informal supplementary means for OWL ontology authoring, if compared\nto the formal notations that are used by professional knowledge engineers. In\nthis paper we present by examples controlled Latvian language that has been\ndesigned to be compliant with the state of the art Attempto Controlled English.\nWe also discuss relation with controlled Lithuanian language that is being\ndesigned in parallel.",
            "journal ref": "Human Language Technologies - The Baltic Perspective, Frontiers in\n  Artificial Intelligence and Applications, Vol. 219, IOS Press, 2010, pp.\n  187-194",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.2290v1",
            "date": "2012-11-10 05:12:31+00:00",
            "title": "Dating Texts without Explicit Temporal Cues",
            "authors": [
                "Abhimanu Kumar",
                "Jason Baldridge",
                "Matthew Lease",
                "Joydeep Ghosh"
            ],
            "abstract": "This paper tackles temporal resolution of documents, such as determining when\na document is about or when it was written, based only on its text. We apply\ntechniques from information retrieval that predict dates via language models\nover a discretized timeline. Unlike most previous works, we rely {\\it solely}\non temporal cues implicit in the text. We consider both document-likelihood and\ndivergence based techniques and several smoothing methods for both of them. Our\nbest model predicts the mid-point of individuals' lives with a median of 22 and\nmean error of 36 years for Wikipedia biographies from 3800 B.C. to the present\nday. We also show that this approach works well when training on such\nbiographies and predicting dates both for non-biographical Wikipedia pages\nabout specific years (500 B.C. to 2010 A.D.) and for publication dates of short\nstories (1798 to 2008). Together, our work shows that, even in absence of\ntemporal extraction resources, it is possible to achieve remarkable temporal\nlocality across a diverse set of texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.3402v1",
            "date": "2012-11-14 20:04:51+00:00",
            "title": "Genetic Optimization of Keywords Subset in the Classification Analysis of Texts Authorship",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "The genetic selection of keywords set, the text frequencies of which are\nconsidered as attributes in text classification analysis, has been analyzed.\nThe genetic optimization was performed on a set of words, which is the fraction\nof the frequency dictionary with given frequency limits. The frequency\ndictionary was formed on the basis of analyzed text array of texts of English\nfiction. As the fitness function which is minimized by the genetic algorithm,\nthe error of nearest k neighbors classifier was used. The obtained results show\nhigh precision and recall of texts classification by authorship categories on\nthe basis of attributes of keywords set which were selected by the genetic\nalgorithm from the frequency dictionary.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1211.4488v1",
            "date": "2012-11-19 16:38:32+00:00",
            "title": "A Rule-Based Approach For Aligning Japanese-Spanish Sentences From A Comparable Corpora",
            "authors": [
                "Jessica C. Ram\u00edrez",
                "Yuji Matsumoto"
            ],
            "abstract": "The performance of a Statistical Machine Translation System (SMT) system is\nproportionally directed to the quality and length of the parallel corpus it\nuses. However for some pair of languages there is a considerable lack of them.\nThe long term goal is to construct a Japanese-Spanish parallel corpus to be\nused for SMT, whereas, there are a lack of useful Japanese-Spanish parallel\nCorpus. To address this problem, In this study we proposed a method for\nextracting Japanese-Spanish Parallel Sentences from Wikipedia using POS tagging\nand Rule-Based approach. The main focus of this approach is the syntactic\nfeatures of both languages. Human evaluation was performed over a sample and\nshows promising results, in comparison with the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.4929v1",
            "date": "2012-11-21 03:59:06+00:00",
            "title": "Summarizing Reviews with Variable-length Syntactic Patterns and Topic Models",
            "authors": [
                "Trung V. Nguyen",
                "Alice H. Oh"
            ],
            "abstract": "We present a novel summarization framework for reviews of products and\nservices by selecting informative and concise text segments from the reviews.\nOur method consists of two major steps. First, we identify five frequently\noccurring variable-length syntactic patterns and use them to extract candidate\nsegments. Then we use the output of a joint generative sentiment topic model to\nfilter out the non-informative segments. We verify the proposed method with\nquantitative and qualitative experiments. In a quantitative study, our approach\noutperforms previous methods in producing informative segments and summaries\nthat capture aspects of products and services as expressed in the\nuser-generated pros and cons lists. Our user study with ninety users resonates\nwith this result: individual segments extracted and filtered by our method are\nrated as more useful by users compared to previous approaches by users.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1211.6887v1",
            "date": "2012-11-29 11:35:25+00:00",
            "title": "Automating rule generation for grammar checkers",
            "authors": [
                "Marcin Mi\u0142kowski"
            ],
            "abstract": "In this paper, I describe several approaches to automatic or semi-automatic\ndevelopment of symbolic rules for grammar checkers from the information\ncontained in corpora. The rules obtained this way are an important addition to\nmanually-created rules that seem to dominate in rule-based checkers. However,\nthe manual process of creation of rules is costly, time-consuming and\nerror-prone. It seems therefore advisable to use machine-learning algorithms to\ncreate the rules automatically or semi-automatically. The results obtained seem\nto corroborate my initial hypothesis that symbolic machine learning algorithms\ncan be useful for acquiring new rules for grammar checking. It turns out,\nhowever, that for practical uses, error corpora cannot be the sole source of\ninformation used in grammar checking. I suggest therefore that only by using\ndifferent approaches, grammar-checkers, or more generally, computer-aided\nproofreading tools, will be able to cover most frequent and severe mistakes and\navoid false alarms that seem to distract users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0188v1",
            "date": "2012-04-01 10:06:55+00:00",
            "title": "OCR Context-Sensitive Error Correction Based on Google Web 1T 5-Gram Data Set",
            "authors": [
                "Youssef Bassil",
                "Mohammad Alwani"
            ],
            "abstract": "Since the dawn of the computing era, information has been represented\ndigitally so that it can be processed by electronic computers. Paper books and\ndocuments were abundant and widely being published at that time; and hence,\nthere was a need to convert them into digital format. OCR, short for Optical\nCharacter Recognition was conceived to translate paper-based books into digital\ne-books. Regrettably, OCR systems are still erroneous and inaccurate as they\nproduce misspellings in the recognized text, especially when the source\ndocument is of low printing quality. This paper proposes a post-processing OCR\ncontext-sensitive error correction method for detecting and correcting non-word\nand real-word OCR errors. The cornerstone of this proposed approach is the use\nof Google Web 1T 5-gram data set as a dictionary of words to spell-check OCR\ntext. The Google data set incorporates a very large vocabulary and word\nstatistics entirely reaped from the Internet, making it a reliable source to\nperform dictionary-based error correction. The core of the proposed solution is\na combination of three algorithms: The error detection, candidate spellings\ngenerator, and error correction algorithms, which all exploit information\nextracted from Google Web 1T 5-gram data set. Experiments conducted on scanned\nimages written in different languages showed a substantial improvement in the\nOCR error correction rate. As future developments, the proposed algorithm is to\nbe parallelised so as to support parallel and distributed computing\narchitectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.0255v1",
            "date": "2012-04-01 19:15:58+00:00",
            "title": "Keyphrase Extraction : Enhancing Lists",
            "authors": [
                "Mario Jarmasz",
                "Caroline Barri\u00e8re"
            ],
            "abstract": "This paper proposes some modest improvements to Extractor, a state-of-the-art\nkeyphrase extraction system, by using a terabyte-sized corpus to estimate the\ninformativeness and semantic similarity of keyphrases. We present two\ntechniques to improve the organization and remove outliers of lists of\nkeyphrases. The first is a simple ordering according to their occurrences in\nthe corpus; the second is clustering according to semantic similarity.\nEvaluation issues are discussed. We present a novel technique of comparing\nextracted keyphrases to a gold standard which relies on semantic similarity\nrather than string matching or an evaluation involving human judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.3198v2",
            "date": "2012-04-14 19:25:54+00:00",
            "title": "The failure of the law of brevity in two New World primates. Statistical caveats",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Antoni Hern\u00e1ndez-Fern\u00e1ndez"
            ],
            "abstract": "Parallels of Zipf's law of brevity, the tendency of more frequent words to be\nshorter, have been found in bottlenose dolphins and Formosan macaques. Although\nthese findings suggest that behavioral repertoires are shaped by a general\nprinciple of compression, common marmosets and golden-backed uakaris do not\nexhibit the law. However, we argue that the law may be impossible or difficult\nto detect statistically in a given species if the repertoire is too small, a\nproblem that could be affecting golden backed uakaris, and show that the law is\npresent in a subset of the repertoire of common marmosets. We suggest that the\nvisibility of the law will depend on the subset of the repertoire under\nconsideration or the repertoire size.",
            "journal ref": "Statistical caveats. Glottotheory 4 (1), 45-55 (2013)",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1204.5316v1",
            "date": "2012-04-24 09:13:59+00:00",
            "title": "ILexicOn: toward an ECD-compliant interlingual lexical ontology described with semantic web formalisms",
            "authors": [
                "Maxime Lefran\u00e7ois",
                "Fabien Gandon"
            ],
            "abstract": "We are interested in bridging the world of natural language and the world of\nthe semantic web in particular to support natural multilingual access to the\nweb of data. In this paper we introduce a new type of lexical ontology called\ninterlingual lexical ontology (ILexicOn), which uses semantic web formalisms to\nmake each interlingual lexical unit class (ILUc) support the projection of its\nsemantic decomposition on itself. After a short overview of existing lexical\nontologies, we briefly introduce the semantic web formalisms we use. We then\npresent the three layered architecture of our approach: i) the interlingual\nlexical meta-ontology (ILexiMOn); ii) the ILexicOn where ILUcs are formally\ndefined; iii) the data layer. We illustrate our approach with a standalone\nILexicOn, and introduce and explain a concise human-readable notation to\nrepresent ILexicOns. Finally, we show how semantic web formalisms enable the\nprojection of a semantic decomposition on the decomposed ILUc.",
            "journal ref": "MTT - 5th International Conference on Meaning-Text Theory - 2011\n  (2011) 155-164",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.5345v1",
            "date": "2012-04-24 11:46:53+00:00",
            "title": "Time-dependent wave selection for information processing in excitable media",
            "authors": [
                "William M. Stevens",
                "Andrew Adamatzky",
                "Ishrat Jahan",
                "Ben de Lacy Costello"
            ],
            "abstract": "We demonstrate an improved technique for implementing logic circuits in\nlight-sensitive chemical excitable media. The technique makes use of the\nconstant-speed propagation of waves along defined channels in an excitable\nmedium based on the Belousov-Zhabotinsky reaction, along with the mutual\nannihilation of colliding waves. What distinguishes this work from previous\nwork in this area is that regions where channels meet at a junction can\nperiodically alternate between permitting the propagation of waves and blocking\nthem. These valve-like areas are used to select waves based on the length of\ntime that it takes waves to propagate from one valve to another. In an\nexperimental implementation, the channels which make up the circuit layout are\nprojected by a digital projector connected to a computer. Excitable channels\nare projected as dark areas, unexcitable regions as light areas. Valves\nalternate between dark and light: every valve has the same period and phase,\nwith a 50% duty cycle. This scheme can be used to make logic gates based on\ncombinations of OR and AND-NOT operations, with few geometrical constraints.\nBecause there are few geometrical constraints, compact circuits can be\nimplemented. Experimental results from an implementation of a 4-bit input,\n2-bit output integer square root circuit are given. This is the most complex\nlogic circuit that has been implemented in BZ excitable media to date.",
            "journal ref": "Phys. Rev. E 85, 066129 (2012)",
            "category": "nlin.PS"
        },
        {
            "url": "http://arxiv.org/abs/1204.5369v1",
            "date": "2012-04-24 13:22:44+00:00",
            "title": "Ecological Evaluation of Persuasive Messages Using Google AdWords",
            "authors": [
                "Marco Guerini",
                "Carlo Strapparava",
                "Oliviero Stock"
            ],
            "abstract": "In recent years there has been a growing interest in crowdsourcing\nmethodologies to be used in experimental research for NLP tasks. In particular,\nevaluation of systems and theories about persuasion is difficult to accommodate\nwithin existing frameworks. In this paper we present a new cheap and fast\nmethodology that allows fast experiment building and evaluation with\nfully-automated analysis at a low cost. The central idea is exploiting existing\ncommercial tools for advertising on the web, such as Google AdWords, to measure\nmessage impact in an ecological setting. The paper includes a description of\nthe approach, tips for how to use AdWords for scientific research, and results\nof pilot experiments on the impact of affective text variations which confirm\nthe effectiveness of the approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.6362v1",
            "date": "2012-04-28 03:48:16+00:00",
            "title": "A Corpus-based Evaluation of Lexical Components of a Domainspecific Text to Knowledge Mapping Prototype",
            "authors": [
                "Rushdi Shams",
                "Adel Elsayed"
            ],
            "abstract": "The aim of this paper is to evaluate the lexical components of a Text to\nKnowledge Mapping (TKM) prototype. The prototype is domain-specific, the\npurpose of which is to map instructional text onto a knowledge domain. The\ncontext of the knowledge domain of the prototype is physics, specifically DC\nelectrical circuits. During development, the prototype has been tested with a\nlimited data set from the domain. The prototype now reached a stage where it\nneeds to be evaluated with a representative linguistic data set called corpus.\nA corpus is a collection of text drawn from typical sources which can be used\nas a test data set to evaluate NLP systems. As there is no available corpus for\nthe domain, we developed a representative corpus and annotated it with\nlinguistic information. The evaluation of the prototype considers one of its\ntwo main components- lexical knowledge base. With the corpus, the evaluation\nenriches the lexical knowledge resources like vocabulary and grammar structure.\nThis leads the prototype to parse a reasonable amount of sentences in the\ncorpus.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1207.0052v3",
            "date": "2012-06-30 06:36:04+00:00",
            "title": "The Complexity of Learning Principles and Parameters Grammars",
            "authors": [
                "Jacob Andreas"
            ],
            "abstract": "We investigate models for learning the class of context-free and\ncontext-sensitive languages (CFLs and CSLs). We begin with a brief discussion\nof some early hardness results which show that unrestricted language learning\nis impossible, and unrestricted CFL learning is computationally infeasible; we\nthen briefly survey the literature on algorithms for learning restricted\nsubclasses of the CFLs. Finally, we introduce a new family of subclasses, the\nprincipled parametric context-free grammars (and a corresponding family of\nprincipled parametric context-sensitive grammars), which roughly model the\n\"Principles and Parameters\" framework in psycholinguistics. We present three\nhardness results: first, that the PPCFGs are not efficiently learnable given\nequivalence and membership oracles, second, that the PPCFGs are not efficiently\nlearnable from positive presentations unless P = NP, and third, that the PPCSGs\nare not efficiently learnable from positive presentations unless integer\nfactorization is in P.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1207.0396v1",
            "date": "2012-07-02 14:19:21+00:00",
            "title": "Applying Deep Belief Networks to Word Sense Disambiguation",
            "authors": [
                "Peratham Wiriyathammabhum",
                "Boonserm Kijsirikul",
                "Hiroya Takamura",
                "Manabu Okumura"
            ],
            "abstract": "In this paper, we applied a novel learning algorithm, namely, Deep Belief\nNetworks (DBN) to word sense disambiguation (WSD). DBN is a probabilistic\ngenerative model composed of multiple layers of hidden units. DBN uses\nRestricted Boltzmann Machine (RBM) to greedily train layer by layer as a\npretraining. Then, a separate fine tuning step is employed to improve the\ndiscriminative power. We compared DBN with various state-of-the-art supervised\nlearning algorithms in WSD such as Support Vector Machine (SVM), Maximum\nEntropy model (MaxEnt), Naive Bayes classifier (NB) and Kernel Principal\nComponent Analysis (KPCA). We used all words in the given paragraph,\nsurrounding context words and part-of-speech of surrounding words as our\nknowledge sources. We conducted our experiment on the SENSEVAL-2 data set. We\nobserved that DBN outperformed all other learning algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.1847v1",
            "date": "2012-07-08 07:02:13+00:00",
            "title": "Finding Structure in Text, Genome and Other Symbolic Sequences",
            "authors": [
                "Ted Dunning"
            ],
            "abstract": "The statistical methods derived and described in this thesis provide new ways\nto elucidate the structural properties of text and other symbolic sequences.\nGenerically, these methods allow detection of a difference in the frequency of\na single feature, the detection of a difference between the frequencies of an\nensemble of features and the attribution of the source of a text. These three\nabstract tasks suffice to solve problems in a wide variety of settings.\nFurthermore, the techniques described in this thesis can be extended to provide\na wide range of additional tests beyond the ones described here.\n  A variety of applications for these methods are examined in detail. These\napplications are drawn from the area of text analysis and genetic sequence\nanalysis. The textually oriented tasks include finding interesting collocations\nand cooccurent phrases, language identification, and information retrieval. The\nbiologically oriented tasks include species identification and the discovery of\npreviously unreported long range structure in genes. In the applications\nreported here where direct comparison is possible, the performance of these new\nmethods substantially exceeds the state of the art.\n  Overall, the methods described here provide new and effective ways to analyse\ntext and other symbolic sequences. Their particular strength is that they deal\nwell with situations where relatively little data are available. Since these\nmethods are abstract in nature, they can be applied in novel situations with\nrelative ease.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.2265v1",
            "date": "2012-07-10 08:48:13+00:00",
            "title": "Challenges for Distributional Compositional Semantics",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "This paper summarises the current state-of-the art in the study of\ncompositionality in distributional semantics, and major challenges for this\narea. We single out generalised quantifiers and intensional semantics as areas\non which to focus attention for the development of the theory. Once suitable\ntheories have been developed, algorithms will be needed to apply the theory to\ntasks. Evaluation is a major problem; we single out application to recognising\ntextual entailment and machine translation for this purpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.2334v2",
            "date": "2012-07-10 13:00:19+00:00",
            "title": "Distinct word length frequencies: distributions and symbol entropies",
            "authors": [
                "Reginald D. Smith"
            ],
            "abstract": "The distribution of frequency counts of distinct words by length in a\nlanguage's vocabulary will be analyzed using two methods. The first, will look\nat the empirical distributions of several languages and derive a distribution\nthat reasonably explains the number of distinct words as a function of length.\nWe will be able to derive the frequency count, mean word length, and variance\nof word length based on the marginal probability of letters and spaces. The\nsecond, based on information theory, will demonstrate that the conditional\nentropies can also be used to estimate the frequency of distinct words of a\ngiven length in a language. In addition, it will be shown how these techniques\ncan also be applied to estimate higher order entropies using vocabulary word\nlength.",
            "journal ref": "Glottometrics 23, 2012, 7-22",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.4307v1",
            "date": "2012-07-18 09:09:35+00:00",
            "title": "Frame Interpretation and Validation in a Open Domain Dialogue System",
            "authors": [
                "Artur Ventura",
                "Nuno Diegues",
                "David Martins de Matos"
            ],
            "abstract": "Our goal in this paper is to establish a means for a dialogue platform to be\nable to cope with open domains considering the possible interaction between the\nembodied agent and humans. To this end we present an algorithm capable of\nprocessing natural language utterances and validate them against knowledge\nstructures of an intelligent agent's mind. Our algorithm leverages dialogue\ntechniques in order to solve ambiguities and acquire knowledge about unknown\nentities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1207.5409v1",
            "date": "2012-07-23 14:24:13+00:00",
            "title": "FST Based Morphological Analyzer for Hindi Language",
            "authors": [
                "Deepak Kumar",
                "Manjeet Singh",
                "Seema Shukla"
            ],
            "abstract": "Hindi being a highly inflectional language, FST (Finite State Transducer)\nbased approach is most efficient for developing a morphological analyzer for\nthis language. The work presented in this paper uses the SFST (Stuttgart Finite\nState Transducer) tool for generating the FST. A lexicon of root words is\ncreated. Rules are then added for generating inflectional and derivational\nwords from these root words. The Morph Analyzer developed was used in a Part Of\nSpeech (POS) Tagger based on Stanford POS Tagger. The system was first trained\nusing a manually tagged corpus and MAXENT (Maximum Entropy) approach of\nStanford POS tagger was then used for tagging input sentences. The\nmorphological analyzer gives approximately 97% correct results. POS tagger\ngives an accuracy of approximately 87% for the sentences that have the words\nknown to the trained model file, and 80% accuracy for the sentences that have\nthe words unknown to the trained model file.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.0252v2",
            "date": "2012-09-30 22:55:45+00:00",
            "title": "A Linguistic Model for Terminology Extraction based Conditional Random Fields",
            "authors": [
                "Fethi Fkih",
                "Mohamed Nazih Omri",
                "Imen Toumia"
            ],
            "abstract": "In this paper, we show the possibility of using a linear Conditional Random\nFields (CRF) for terminology extraction from a specialized text corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.0794v1",
            "date": "2012-10-02 14:56:07+00:00",
            "title": "A Semantic Approach for Automatic Structuring and Analysis of Software Process Patterns",
            "authors": [
                "Nahla Jlaiel",
                "Khouloud Madhbouh",
                "Mohamed Ben Ahmed"
            ],
            "abstract": "The main contribution of this paper, is to propose a novel semantic approach\nbased on a Natural Language Processing technique in order to ensure a semantic\nunification of unstructured process patterns which are expressed not only in\ndifferent formats but also, in different forms. This approach is implemented\nusing the GATE text engineering framework and then evaluated leading up to\nhigh-quality results motivating us to continue in this direction.",
            "journal ref": "Nahla Jlaiel, Khouloud Madhbouh and Mohamed Ben Ahmed. Article: A\n  Semantic Approach for Automatic Structuring and Analysis of Software Process\n  Patterns. International Journal of Computer Applications 54(15):24-31,\n  September 2012",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1210.0852v1",
            "date": "2012-10-02 17:41:58+00:00",
            "title": "Detecting multiword phrases in mathematical text corpora",
            "authors": [
                "Winfried G\u00f6dert"
            ],
            "abstract": "We present an approach for detecting multiword phrases in mathematical text\ncorpora. The method used is based on characteristic features of mathematical\nterminology. It makes use of a software tool named Lingo which allows to\nidentify words by means of previously defined dictionaries for specific word\nclasses as adjectives, personal names or nouns. The detection of multiword\ngroups is done algorithmically. Possible advantages of the method for indexing\nand information retrieval and conclusions for applying dictionary-based methods\nof automatic indexing instead of stemming procedures are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.3634v1",
            "date": "2012-10-12 20:55:39+00:00",
            "title": "Quick Summary",
            "authors": [
                "Robert Wahlstedt"
            ],
            "abstract": "Quick Summary is an innovate implementation of an automatic document\nsummarizer that inputs a document in the English language and evaluates each\nsentence. The scanner or evaluator determines criteria based on its grammatical\nstructure and place in the paragraph. The program then asks the user to specify\nthe number of sentences the person wishes to highlight. For example should the\nuser ask to have three of the most important sentences, it would highlight the\nfirst and most important sentence in green. Commonly this is the sentence\ncontaining the conclusion. Then Quick Summary finds the second most important\nsentence usually called a satellite and highlights it in yellow. This is\nusually the topic sentence. Then the program finds the third most important\nsentence and highlights it in red. The implementations of this technology are\nuseful in a society of information overload when a person typically receives 42\nemails a day (Microsoft). The paper also is a candid look at difficulty that\nmachine learning has in textural translating. However, it speaks on how to\novercome the obstacles that historically prevented progress. This paper\nproposes mathematical meta-data criteria that justify the place of importance\nof a sentence. Just as tools for the study of relational symmetry in\nbio-informatics, this tool seeks to classify words with greater clarity.\n\"Survey Finds Workers Average Only Three Productive Days per Week.\" Microsoft\nNews Center. Microsoft. Web. 31 Mar. 2012.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.3729v1",
            "date": "2012-10-13 18:02:26+00:00",
            "title": "Inference of Fine-grained Attributes of Bengali Corpus for Stylometry Detection",
            "authors": [
                "Tanmoy Chakraborty",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Stylometry, the science of inferring characteristics of the author from the\ncharacteristics of documents written by that author, is a problem with a long\nhistory and belongs to the core task of Text categorization that involves\nauthorship identification, plagiarism detection, forensic investigation,\ncomputer security, copyright and estate disputes etc. In this work, we present\na strategy for stylometry detection of documents written in Bengali. We adopt a\nset of fine-grained attribute features with a set of lexical markers for the\nanalysis of the text and use three semi-supervised measures for making\ndecisions. Finally, a majority voting approach has been taken for final\nclassification. The system is fully automatic and language-independent.\nEvaluation results of our attempt for Bengali author's stylometry detection\nshow reasonably promising accuracy in comparison to the baseline model.",
            "journal ref": "Polibits (44) 2011, pp. 79-83",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.4854v1",
            "date": "2012-10-16 17:37:21+00:00",
            "title": "Semantic Understanding of Professional Soccer Commentaries",
            "authors": [
                "Hannaneh Hajishirzi",
                "Mohammad Rastegari",
                "Ali Farhadi",
                "Jessica K. Hodgins"
            ],
            "abstract": "This paper presents a novel approach to the problem of semantic parsing via\nlearning the correspondences between complex sentences and rich sets of events.\nOur main intuition is that correct correspondences tend to occur more\nfrequently. Our model benefits from a discriminative notion of similarity to\nlearn the correspondence between sentence and an event and a ranking machinery\nthat scores the popularity of each correspondence. Our method can discover a\ngroup of events (called macro-events) that best describes a sentence. We\nevaluate our method on our novel dataset of professional soccer commentaries.\nThe empirical results show that our method significantly outperforms the\nstate-of-theart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5321v1",
            "date": "2012-10-19 05:50:58+00:00",
            "title": "The origin of Mayan languages from Formosan language group of Austronesian",
            "authors": [
                "Koji Ohnishi"
            ],
            "abstract": "Basic body-part names (BBPNs) were defined as body-part names in Swadesh\nbasic 200 words. Non-Mayan cognates of Mayan (MY) BBPNs were extensively\nsearched for, by comparing with non-MY vocabulary, including ca.1300 basic\nwords of 82 AN languages listed by Tryon (1985), etc. Thus found cognates (CGs)\nin non-MY are listed in Table 1, as classified by language groups to which most\nsimilar cognates (MSCs) of MY BBPNs belong. CGs of MY are classified to 23\nmutually unrelated CG-items, of which 17.5 CG-items have their MSCs in\nAustronesian (AN), giving its closest similarity score (CSS), CSS(AN) = 17.5,\nwhich consists of 10.33 MSCs in Formosan, 1.83 MSCs in Western\nMalayo-Polynesian (W.MP), 0.33 in Central MP, 0.0 in SHWNG, and 5.0 in Oceanic\n[i.e., CSS(FORM)= 10.33, CSS(W.MP) = 1.88, ..., CSS(OC)= 5.0]. These CSSs for\nlanguage (sub)groups are also listed in the underline portion of every section\nof (Section1 - Section 6) in Table 1. Chi-squar test (degree of freedom = 1)\nusing [Eq 1] and [Eqs.2] revealed that MSCs of MY BBPNs are distributed in\nFormosan in significantly higher frequency (P < 0.001) than in other subgroups\nof AN, as well as than in non-AN languages. MY is thus concluded to have been\nderived from Formosan of AN. Eskimo shows some BBPN similarities to FORM and\nMY.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.7137v1",
            "date": "2012-10-26 13:16:30+00:00",
            "title": "Alberti's letter counts",
            "authors": [
                "Bernard Ycart"
            ],
            "abstract": "Four centuries before modern statistical linguistics was born, Leon Battista\nAlberti (1404--1472) compared the frequency of vowels in Latin poems and\norations, making the first quantified observation of a stylistic difference\never. Using a corpus of 20 Latin texts (over 5 million letters), Alberti's\nobservations are statistically assessed. Letter counts prove that poets used\nsignificantly more a's, e's, and y's, whereas orators used more of the other\nvowels. The sample sizes needed to justify the assertions are studied, and\nproved to be within reach for Alberti's scholarship.",
            "journal ref": "Literary and Linguistic Computing (2013) 10.1093/llc/fqt034",
            "category": "math.HO"
        },
        {
            "url": "http://arxiv.org/abs/1210.7917v1",
            "date": "2012-10-30 07:25:46+00:00",
            "title": "The Model of Semantic Concepts Lattice For Data Mining Of Microblogs",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "The model of semantic concept lattice for data mining of microblogs has been\nproposed in this work. It is shown that the use of this model is effective for\nthe semantic relations analysis and for the detection of associative rules of\nkey words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.8436v1",
            "date": "2012-10-31 18:52:01+00:00",
            "title": "Optimal size, freshness and time-frame for voice search vocabulary",
            "authors": [
                "Maryam Kamvar",
                "Ciprian Chelba"
            ],
            "abstract": "In this paper, we investigate how to optimize the vocabulary for a voice\nsearch language model. The metric we optimize over is the out-of-vocabulary\n(OoV) rate since it is a strong indicator of user experience. In a departure\nfrom the usual way of measuring OoV rates, web search logs allow us to compute\nthe per-session OoV rate and thus estimate the percentage of users that\nexperience a given OoV rate. Under very conservative text normalization, we\nfind that a voice search vocabulary consisting of 2 to 2.5 million words\nextracted from 1 week of search query data will result in an aggregate OoV rate\nof 1%; at that size, the same OoV rate will also be experienced by 90% of\nusers. The number of words included in the vocabulary is a stable indicator of\nthe OoV rate. Altering the freshness of the vocabulary or the duration of the\ntime window over which the training data is gathered does not significantly\nchange the OoV rate. Surprisingly, a significantly larger vocabulary\n(approximately 10 million words) is required to guarantee OoV rates below 1%\nfor 95% of the users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.0570v1",
            "date": "2012-12-12 15:56:22+00:00",
            "title": "Reduction of Maximum Entropy Models to Hidden Markov Models",
            "authors": [
                "Joshua Goodman"
            ],
            "abstract": "We show that maximum entropy (maxent) models can be modeled with certain\nkinds of HMMs, allowing us to construct maxent models with hidden variables,\nhidden state sequences, or other characteristics. The models can be trained\nusing the forward-backward algorithm. While the results are primarily of\ntheoretical interest, unifying apparently unrelated concepts, we also give\nexperimental results for a maxent model with a hidden variable on a word\ndisambiguation task; the model outperforms standard techniques.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1212.0074v1",
            "date": "2012-12-01 07:01:27+00:00",
            "title": "Challenges in Kurdish Text Processing",
            "authors": [
                "Kyumars Sheykh Esmaili"
            ],
            "abstract": "Despite having a large number of speakers, the Kurdish language is among the\nless-resourced languages. In this work we highlight the challenges and problems\nin providing the required tools and techniques for processing texts written in\nKurdish. From a high-level perspective, the main challenges are: the inherent\ndiversity of the language, standardization and segmentation issues, and the\nlack of language resources.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.0229v1",
            "date": "2012-12-02 18:06:57+00:00",
            "title": "Simplification and integration in computing and cognition: the SP theory and the multiple alignment concept",
            "authors": [
                "James Gerard Wolff"
            ],
            "abstract": "The main purpose of this article is to describe potential benefits and\napplications of the SP theory, a unique attempt to simplify and integrate ideas\nacross artificial intelligence, mainstream computing and human cognition, with\ninformation compression as a unifying theme. The theory, including a concept of\nmultiple alignment, combines conceptual simplicity with descriptive and\nexplanatory power in several areas including representation of knowledge,\nnatural language processing, pattern recognition, several kinds of reasoning,\nthe storage and retrieval of information, planning and problem solving,\nunsupervised learning, information compression, and human perception and\ncognition. In the SP machine -- an expression of the SP theory which is\ncurrently realised in the form of computer models -- there is potential for an\noverall simplification of computing systems, including software. As a theory\nwith a broad base of support, the SP theory promises useful insights in many\nareas and the integration of structures and functions, both within a given area\nand amongst different areas. There are potential benefits in natural language\nprocessing (with potential for the understanding and translation of natural\nlanguages), the need for a versatile intelligence in autonomous robots,\ncomputer vision, intelligent databases, maintaining multiple versions of\ndocuments or web pages, software engineering, criminal investigations, the\nmanagement of big data and gaining benefits from it, the semantic web, medical\ndiagnosis, the detection of computer viruses, the economical transmission of\ndata, and data fusion. Further development of these ideas would be facilitated\nby the creation of a high-parallel, web-based, open-source version of the SP\nmachine, with a good user interface. This would provide a means for researchers\nto explore what can be done with the system and to refine it.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1212.1918v1",
            "date": "2012-12-09 20:55:52+00:00",
            "title": "Condens\u00e9s de textes par des m\u00e9thodes num\u00e9riques",
            "authors": [
                "Juan-Manuel Torres-Moreno",
                "Patricia Vel\u00e1zquez-Morales",
                "Jean-Guy Meunier"
            ],
            "abstract": "Since information in electronic form is already a standard, and that the\nvariety and the quantity of information become increasingly large, the methods\nof summarizing or automatic condensation of texts is a critical phase of the\nanalysis of texts. This article describes CORTEX a system based on numerical\nmethods, which allows obtaining a condensation of a text, which is independent\nof the topic and of the length of the text. The structure of the system enables\nit to find the abstracts in French or Spanish in very short times.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.2006v2",
            "date": "2012-12-10 09:41:12+00:00",
            "title": "A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization",
            "authors": [
                "Jiwei Li",
                "Sujian Li"
            ],
            "abstract": "Both supervised learning methods and LDA based topic model have been\nsuccessfully applied in the field of query focused multi-document\nsummarization. In this paper, we propose a novel supervised approach that can\nincorporate rich sentence features into Bayesian topic models in a principled\nway, thus taking advantages of both topic model and feature based supervised\nlearning methods. Experiments on TAC2008 and TAC2009 demonstrate the\neffectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.2036v3",
            "date": "2012-12-10 11:35:29+00:00",
            "title": "Query-focused Multi-document Summarization: Combining a Novel Topic Model with Graph-based Semi-supervised Learning",
            "authors": [
                "Jiwei Li",
                "Sujian Li"
            ],
            "abstract": "Graph-based semi-supervised learning has proven to be an effective approach\nfor query-focused multi-document summarization. The problem of previous\nsemi-supervised learning is that sentences are ranked without considering the\nhigher level information beyond sentence level. Researches on general\nsummarization illustrated that the addition of topic level can effectively\nimprove the summary quality. Inspired by previous researches, we propose a\ntwo-layer (i.e. sentence layer and topic layer) graph-based semi-supervised\nlearning approach. At the same time, we propose a novel topic model which makes\nfull use of the dependence between sentences and words. Experimental results on\nDUC and TAC data sets demonstrate the effectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.2145v1",
            "date": "2012-12-10 17:39:44+00:00",
            "title": "A Scale-Space Theory for Text",
            "authors": [
                "Shuang-Hong Yang"
            ],
            "abstract": "Scale-space theory has been established primarily by the computer vision and\nsignal processing communities as a well-founded and promising framework for\nmulti-scale processing of signals (e.g., images). By embedding an original\nsignal into a family of gradually coarsen signals parameterized with a\ncontinuous scale parameter, it provides a formal framework to capture the\nstructure of a signal at different scales in a consistent way. In this paper,\nwe present a scale space theory for text by integrating semantic and spatial\nfilters, and demonstrate how natural language documents can be understood,\nprocessed and analyzed at multiple resolutions, and how this scale-space\nrepresentation can be used to facilitate a variety of NLP and text analysis\ntasks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.2390v1",
            "date": "2012-12-11 11:35:30+00:00",
            "title": "On the complexity of learning a language: An improvement of Block's algorithm",
            "authors": [
                "Eric Werner"
            ],
            "abstract": "Language learning is thought to be a highly complex process. One of the\nhurdles in learning a language is to learn the rules of syntax of the language.\nRules of syntax are often ordered in that before one rule can applied one must\napply another. It has been thought that to learn the order of n rules one must\ngo through all n! permutations. Thus to learn the order of 27 rules would\nrequire 27! steps or 1.08889x10^{28} steps. This number is much greater than\nthe number of seconds since the beginning of the universe! In an insightful\nanalysis the linguist Block ([Block 86], pp. 62-63, p.238) showed that with the\nassumption of transitivity this vast number of learning steps reduces to a mere\n377 steps. We present a mathematical analysis of the complexity of Block's\nalgorithm. The algorithm has a complexity of order n^2 given n rules. In\naddition, we improve Block's results exponentially, by introducing an algorithm\nthat has complexity of order less than n log n.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.2453v1",
            "date": "2012-10-19 15:03:27+00:00",
            "title": "Web-Based Question Answering: A Decision-Making Perspective",
            "authors": [
                "David Azari",
                "Eric J. Horvitz",
                "Susan Dumais",
                "Eric Brill"
            ],
            "abstract": "We describe an investigation of the use of probabilistic models and\ncost-benefit analyses to guide resource-intensive procedures used by a\nWeb-based question answering system. We first provide an overview of research\non question-answering systems. Then, we present details on AskMSR, a prototype\nweb-based question answering system. We discuss Bayesian analyses of the\nquality of answers generated by the system and show how we can endow the system\nwith the ability to make decisions about the number of queries issued to a\nsearch engine, given the cost of queries and the expected value of query\nresults in refining an ultimate answer. Finally, we review the results of a set\nof experiments.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.2477v1",
            "date": "2012-10-19 15:06:15+00:00",
            "title": "1 Billion Pages = 1 Million Dollars? Mining the Web to Play \"Who Wants to be a Millionaire?\"",
            "authors": [
                "Shyong",
                "K. Lam",
                "David M Pennock",
                "Dan Cosley",
                "Steve Lawrence"
            ],
            "abstract": "We exploit the redundancy and volume of information on the web to build a\ncomputerized player for the ABC TV game show 'Who Wants To Be A Millionaire?'\nThe player consists of a question-answering module and a decision-making\nmodule. The question-answering module utilizes question transformation\ntechniques, natural language parsing, multiple information retrieval\nalgorithms, and multiple search engines; results are combined in the spirit of\nensemble learning using an adaptive weighting scheme. Empirically, the system\ncorrectly answers about 75% of questions from the Millionaire CD-ROM, 3rd\nedition - general-interest trivia questions often about popular culture and\ncommon knowledge. The decision-making module chooses from allowable actions in\nthe game in order to maximize expected risk-adjusted winnings, where the\nestimated probability of answering correctly is a function of past performance\nand confidence in in correctly answering the current question. When given a six\nquestion head start (i.e., when starting from the $2,000 level), we find that\nthe system performs about as well on average as humans starting at the\nbeginning. Our system demonstrates the potential of simple but well-chosen\ntechniques for mining answers from unstructured information such as the web.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.3023v1",
            "date": "2012-12-13 00:34:23+00:00",
            "title": "Keyword Extraction for Identifying Social Actors",
            "authors": [
                "Mahyuddin K. M. Nasution",
                "Shahrul Azman Mohd Noah"
            ],
            "abstract": "Identifying the social actor has become one of tasks in Artificial\nIntelligence, whereby extracting keyword from Web snippets depend on the use of\nweb is steadily gaining ground in this research. We develop therefore an\napproach based on overlap principle for utilizing a collection of features in\nweb snippets, where use of keyword will eliminate the un-relevant web pages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.3493v2",
            "date": "2012-12-14 14:51:22+00:00",
            "title": "Sentence Compression in Spanish driven by Discourse Segmentation and Language Models",
            "authors": [
                "Alejandro Molina",
                "Juan-Manuel Torres-Moreno",
                "Iria da Cunha",
                "Eric SanJuan",
                "Gerardo Sierra"
            ],
            "abstract": "Previous works demonstrated that Automatic Text Summarization (ATS) by\nsentences extraction may be improved using sentence compression. In this work\nwe present a sentence compressions approach guided by level-sentence discourse\nsegmentation and probabilistic language models (LM). The results presented here\nshow that the proposed solution is able to generate coherent summaries with\ngrammatical compressed sentences. The approach is simple enough to be\ntransposed into other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.3634v1",
            "date": "2012-12-14 23:34:07+00:00",
            "title": "A comparative study of root-based and stem-based approaches for measuring the similarity between arabic words for arabic text mining applications",
            "authors": [
                "Hanane Froud",
                "Abdelmonaim Lachkar",
                "Said Alaoui Ouatik"
            ],
            "abstract": "Representation of semantic information contained in the words is needed for\nany Arabic Text Mining applications. More precisely, the purpose is to better\ntake into account the semantic dependencies between words expressed by the\nco-occurrence frequencies of these words. There have been many proposals to\ncompute similarities between words based on their distributions in contexts. In\nthis paper, we compare and contrast the effect of two preprocessing techniques\napplied to Arabic corpus: Rootbased (Stemming), and Stem-based (Light Stemming)\napproaches for measuring the similarity between Arabic words with the well\nknown abstractive model -Latent Semantic Analysis (LSA)- with a wide variety of\ndistance functions and similarity measures, such as the Euclidean Distance,\nCosine Similarity, Jaccard Coefficient, and the Pearson Correlation\nCoefficient. The obtained results show that, on the one hand, the variety of\nthe corpus produces more accurate results; on the other hand, the Stem-based\napproach outperformed the Root-based one because this latter affects the words\nmeanings.",
            "journal ref": "Advanced Computing An International Journal (ACIJ), November 2012,\n  Volume 3, Number 6",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.6527v1",
            "date": "2012-12-28 15:39:46+00:00",
            "title": "Discovering Basic Emotion Sets via Semantic Clustering on a Twitter Corpus",
            "authors": [
                "Eugene Yuta Bann"
            ],
            "abstract": "A plethora of words are used to describe the spectrum of human emotions, but\nhow many emotions are there really, and how do they interact? Over the past few\ndecades, several theories of emotion have been proposed, each based around the\nexistence of a set of 'basic emotions', and each supported by an extensive\nvariety of research including studies in facial expression, ethology, neurology\nand physiology. Here we present research based on a theory that people transmit\ntheir understanding of emotions through the language they use surrounding\nemotion keywords. Using a labelled corpus of over 21,000 tweets, six of the\nbasic emotion sets proposed in existing literature were analysed using Latent\nSemantic Clustering (LSC), evaluating the distinctiveness of the semantic\nmeaning attached to the emotional label. We hypothesise that the more distinct\nthe language is used to express a certain emotion, then the more distinct the\nperception (including proprioception) of that emotion is, and thus more\n'basic'. This allows us to select the dimensions best representing the entire\nspectrum of emotion. We find that Ekman's set, arguably the most frequently\nused for classifying emotions, is in fact the most semantically distinct\noverall. Next, taking all analysed (that is, previously proposed) emotion terms\ninto account, we determine the optimal semantically irreducible basic emotion\nset using an iterative LSC algorithm. Our newly-derived set (Accepting,\nAshamed, Contempt, Interested, Joyful, Pleased, Sleepy, Stressed) generates a\n6.1% increase in distinctiveness over Ekman's set (Angry, Disgusted, Joyful,\nSad, Scared). We also demonstrate how using LSC data can help visualise\nemotions. We introduce the concept of an Emotion Profile and briefly analyse\ncompound emotions both visually and mathematically.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1201.1652v1",
            "date": "2012-01-08 19:24:08+00:00",
            "title": "Toward a Motor Theory of Sign Language Perception",
            "authors": [
                "Sylvie Gibet",
                "Pierre-Fran\u00e7ois Marteau",
                "Kyle Duarte"
            ],
            "abstract": "Researches on signed languages still strongly dissociate lin- guistic issues\nrelated on phonological and phonetic aspects, and gesture studies for\nrecognition and synthesis purposes. This paper focuses on the imbrication of\nmotion and meaning for the analysis, synthesis and evaluation of sign language\ngestures. We discuss the relevance and interest of a motor theory of perception\nin sign language communication. According to this theory, we consider that\nlinguistic knowledge is mapped on sensory-motor processes, and propose a\nmethodology based on the principle of a synthesis-by-analysis approach, guided\nby an evaluation process that aims to validate some hypothesis and concepts of\nthis theory. Examples from existing studies illustrate the di erent concepts\nand provide avenues for future work.",
            "journal ref": "Gesture and Sign Language in Human-Computer Interaction and\n  Embodied Communication (2012) Vol. 7206, 161-172",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1201.2240v1",
            "date": "2012-01-11 04:56:59+00:00",
            "title": "Bengali text summarization by sentence extraction",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "Text summarization is a process to produce an abstract or a summary by\nselecting significant portion of the information from one or more texts. In an\nautomatic text summarization process, a text is given to the computer and the\ncomputer returns a shorter less redundant extract or abstract of the original\ntext(s). Many techniques have been developed for summarizing English text(s).\nBut, a very few attempts have been made for Bengali text summarization. This\npaper presents a method for Bengali text summarization which extracts important\nsentences from a Bengali document to produce a summary.",
            "journal ref": "Proceedings of International Conference on Business and\n  Information Management(ICBIM-2012),NIT Durgapur, PP 233-245",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1201.2719v3",
            "date": "2012-01-13 01:00:41+00:00",
            "title": "Ultrametric Model of Mind, II: Application to Text Content Analysis",
            "authors": [
                "Fionn Murtagh"
            ],
            "abstract": "In a companion paper, Murtagh (2012), we discussed how Matte Blanco's work\nlinked the unrepressed unconscious (in the human) to symmetric logic and\nthought processes. We showed how ultrametric topology provides a most useful\nrepresentational and computational framework for this. Now we look at the\nextent to which we can find ultrametricity in text. We use coherent and\nmeaningful collections of nearly 1000 texts to show how we can measure inherent\nultrametricity. On the basis of our findings we hypothesize that inherent\nultrametricty is a basis for further exploring unconscious thought processes.",
            "journal ref": "p-Adic Numbers, Ultrametric Analysis and Applications, 4, 207-221,\n  2012",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1201.4733v1",
            "date": "2012-01-20 17:35:29+00:00",
            "title": "Du TAL au TIL",
            "authors": [
                "Michael Zock",
                "Guy Lapalme"
            ],
            "abstract": "Historically two types of NLP have been investigated: fully automated\nprocessing of language by machines (NLP) and autonomous processing of natural\nlanguage by people, i.e. the human brain (psycholinguistics). We believe that\nthere is room and need for another kind, INLP: interactive natural language\nprocessing. This intermediate approach starts from peoples' needs, trying to\nbridge the gap between their actual knowledge and a given goal. Given the fact\nthat peoples' knowledge is variable and often incomplete, the aim is to build\nbridges linking a given knowledge state to a given goal. We present some\nexamples, trying to show that this goal is worth pursuing, achievable and at a\nreasonable cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.0377v1",
            "date": "2012-06-02 13:11:17+00:00",
            "title": "Automated Word Puzzle Generation via Topic Dictionaries",
            "authors": [
                "Balazs Pinter",
                "Gyula Voros",
                "Zoltan Szabo",
                "Andras Lorincz"
            ],
            "abstract": "We propose a general method for automated word puzzle generation. Contrary to\nprevious approaches in this novel field, the presented method does not rely on\nhighly structured datasets obtained with serious human annotation effort: it\nonly needs an unstructured and unannotated corpus (i.e., document collection)\nas input. The method builds upon two additional pillars: (i) a topic model,\nwhich induces a topic dictionary from the input corpus (examples include e.g.,\nlatent semantic analysis, group-structured dictionaries or latent Dirichlet\nallocation), and (ii) a semantic similarity measure of word pairs. Our method\ncan (i) generate automatically a large number of proper word puzzles of\ndifferent types, including the odd one out, choose the related word and\nseparate the topics puzzle. (ii) It can easily create domain-specific puzzles\nby replacing the corpus component. (iii) It is also capable of automatically\ngenerating puzzles with parameterizable levels of difficulty suitable for,\ne.g., beginners or intermediate learners.",
            "journal ref": "International Conference on Machine Learning (ICML-2012) -\n  Sparsity, Dictionaries and Projections in Machine Learning and Signal\n  Processing Workshop, Edinburgh, Scotland, 30 June 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.2010v1",
            "date": "2012-06-10 09:32:07+00:00",
            "title": "Temporal expression normalisation in natural language texts",
            "authors": [
                "Michele Filannino"
            ],
            "abstract": "Automatic annotation of temporal expressions is a research challenge of great\ninterest in the field of information extraction. In this report, I describe a\nnovel rule-based architecture, built on top of a pre-existing system, which is\nable to normalise temporal expressions detected in English texts. Gold standard\ntemporally-annotated resources are limited in size and this makes research\ndifficult. The proposed system outperforms the state-of-the-art systems with\nrespect to TempEval-2 Shared Task (value attribute) and achieves substantially\nbetter results with respect to the pre-existing system on top of which it has\nbeen developed. I will also introduce a new free corpus consisting of 2822\nunique annotated temporal expressions. Both the corpus and the system are\nfreely available on-line.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.3293v1",
            "date": "2012-06-13 15:52:10+00:00",
            "title": "Propagation using Chain Event Graphs",
            "authors": [
                "Peter Thwaites",
                "Jim Q. Smith",
                "Robert G. Cowell"
            ],
            "abstract": "A Chain Event Graph (CEG) is a graphial model which designed to embody\nconditional independencies in problems whose state spaces are highly asymmetric\nand do not admit a natural product structure. In this paer we present a\nprobability propagation algorithm which uses the topology of the CEG to build a\ntransporter CEG. Intriungly,the transporter CEG is directly analogous to the\ntriangulated Bayesian Network (BN) in the more conventional junction tree\npropagation algorithms used with BNs. The propagation method uses factorization\nformulae also analogous to (but different from) the ones using potentials on\ncliques and separators of the BN. It appears that the methods will be typically\nmore efficient than the BN algorithms when applied to contexts where there is\nsignificant asymmetry present.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1206.5384v1",
            "date": "2012-06-23 12:19:38+00:00",
            "title": "Keyphrase Based Arabic Summarizer (KPAS)",
            "authors": [
                "Tarek El-Shishtawy",
                "Fatma El-Ghannam"
            ],
            "abstract": "This paper describes a computationally inexpensive and efficient generic\nsummarization algorithm for Arabic texts. The algorithm belongs to extractive\nsummarization family, which reduces the problem into representative sentences\nidentification and extraction sub-problems. Important keyphrases of the\ndocument to be summarized are identified employing combinations of statistical\nand linguistic features. The sentence extraction algorithm exploits keyphrases\nas the primary attributes to rank a sentence. The present experimental work,\ndemonstrates different techniques for achieving various summarization goals\nincluding: informative richness, coverage of both main and auxiliary topics,\nand keeping redundancy to a minimum. A scoring scheme is then adopted that\nbalances between these summarization goals. To evaluate the resulted Arabic\nsummaries with well-established systems, aligned English/Arabic texts are used\nthrough the experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.6403v1",
            "date": "2012-06-27 19:59:59+00:00",
            "title": "Two Step CCA: A new spectral method for estimating vector models of words",
            "authors": [
                "Paramveer Dhillon",
                "Jordan Rodu",
                "Dean Foster",
                "Lyle Ungar"
            ],
            "abstract": "Unlabeled data is often used to learn representations which can be used to\nsupplement baseline features in a supervised learner. For example, for text\napplications where the words lie in a very high dimensional space (the size of\nthe vocabulary), one can learn a low rank \"dictionary\" by an\neigen-decomposition of the word co-occurrence matrix (e.g. using PCA or CCA).\nIn this paper, we present a new spectral method based on CCA to learn an\neigenword dictionary. Our improved procedure computes two set of CCAs, the\nfirst one between the left and right contexts of the given word and the second\none between the projections resulting from this CCA and the word itself. We\nprove theoretically that this two-step procedure has lower sample complexity\nthan the simple single step procedure and also illustrate the empirical\nefficacy of our approach and the richness of representations learned by our Two\nStep CCA (TSCCA) procedure on the tasks of POS tagging and sentiment\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.6426v1",
            "date": "2012-06-27 19:59:59+00:00",
            "title": "A Fast and Simple Algorithm for Training Neural Probabilistic Language Models",
            "authors": [
                "Andriy Mnih",
                "Yee Whye Teh"
            ],
            "abstract": "In spite of their superior performance, neural probabilistic language models\n(NPLMs) remain far less widely used than n-gram models due to their notoriously\nlong training times, which are measured in weeks even for moderately-sized\ndatasets. Training NPLMs is computationally expensive because they are\nexplicitly normalized, which leads to having to consider all words in the\nvocabulary when computing the log-likelihood gradients.\n  We propose a fast and simple algorithm for training NPLMs based on\nnoise-contrastive estimation, a newly introduced procedure for estimating\nunnormalized continuous distributions. We investigate the behaviour of the\nalgorithm on the Penn Treebank corpus and show that it reduces the training\ntimes by more than an order of magnitude without affecting the quality of the\nresulting models. The algorithm is also more efficient and much more stable\nthan importance sampling because it requires far fewer noise samples to perform\nwell.\n  We demonstrate the scalability of the proposed approach by training several\nneural language models on a 47M-word corpus with a 80K-word vocabulary,\nobtaining state-of-the-art results on the Microsoft Research Sentence\nCompletion Challenge dataset.",
            "journal ref": "In Proceedings of the 29th International Conference on Machine\n  Learning, pages 1751-1758, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.6735v1",
            "date": "2012-06-28 15:43:57+00:00",
            "title": "Elimination of Spurious Ambiguity in Transition-Based Dependency Parsing",
            "authors": [
                "Shay B. Cohen",
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Giorgio Satta"
            ],
            "abstract": "We present a novel technique to remove spurious ambiguity from transition\nsystems for dependency parsing. Our technique chooses a canonical sequence of\ntransition operations (computation) for a given dependency tree. Our technique\ncan be applied to a large class of bottom-up transition systems, including for\ninstance Nivre (2004) and Attardi (2006).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.0617v2",
            "date": "2012-02-03 07:09:23+00:00",
            "title": "Classification of Flames in Computer Mediated Communications",
            "authors": [
                "Nitin",
                "Ankush Bansal",
                "Siddhartha Mahadev Sharma",
                "Kapil Kumar",
                "Anuj Aggarwal",
                "Sheenu Goyal",
                "Kanika Choudhary",
                "Kunal Chawla",
                "Kunal Jain",
                "Manav Bhasin"
            ],
            "abstract": "Computer Mediated Communication (CMC) has brought about a revolution in the\nway the world communicates with each other. With the increasing number of\npeople, interacting through the internet and the rise of new platforms and\ntechnologies has brought together the people from different social, cultural\nand geographical backgrounds to present their thoughts, ideas and opinions on\ntopics of their interest. CMC has, in some cases, gave users more freedom to\nexpress themselves as compared to Face-to-face communication. This has also led\nto rise in the use of hostile and aggressive language and terminologies\nuninhibitedly. Since such use of language is detrimental to the discussion\nprocess and affects the audience and individuals negatively, efforts are being\ntaken to control them. The research sees the need to understand the concept of\nflaming and hence attempts to classify them in order to give a better\nunderstanding of it. The classification is done on the basis of type of flame\ncontent being presented and the Style in which they are presented.",
            "journal ref": "International Journal of Computer Applications (0975-8887), Volume\n  14 - No.6, February 2011",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1202.2518v4",
            "date": "2012-02-12 11:23:38+00:00",
            "title": "Segmenting DNA sequence into `words'",
            "authors": [
                "Wang Liang"
            ],
            "abstract": "This paper presents a novel method to segment/decode DNA sequences based on\nn-grams statistical language model. Firstly, we find the length of most DNA\n'words' is 12 to 15 bps by analyzing the genomes of 12 model species. Then we\ndesign an unsupervised probability based approach to segment the DNA sequences.\nThe benchmark of segmenting method is also proposed.",
            "journal ref": "None",
            "category": "q-bio.GN"
        },
        {
            "url": "http://arxiv.org/abs/1202.4837v1",
            "date": "2012-02-22 06:43:11+00:00",
            "title": "The GF Mathematics Library",
            "authors": [
                "Jordi Saludes",
                "Sebastian Xamb\u00f3"
            ],
            "abstract": "This paper is devoted to present the Mathematics Grammar Library, a system\nfor multilingual mathematical text processing. We explain the context in which\nit originated, its current design and functionality and the current development\ngoals. We also present two prototype services and comment on possible future\napplications in the area of artificial mathematics assistants.",
            "journal ref": "EPTCS 79, 2012, pp. 102-110",
            "category": "cs.MS"
        },
        {
            "url": "http://arxiv.org/abs/1202.6583v1",
            "date": "2012-02-29 15:59:54+00:00",
            "title": "A Lexical Analysis Tool with Ambiguity Support",
            "authors": [
                "Luis Quesada",
                "Fernando Berzal",
                "Francisco J. Cortijo"
            ],
            "abstract": "Lexical ambiguities naturally arise in languages. We present Lamb, a lexical\nanalyzer that produces a lexical analysis graph describing all the possible\nsequences of tokens that can be found within the input string. Parsers can\nprocess such lexical analysis graphs and discard any sequence of tokens that\ndoes not produce a valid syntactic sentence, therefore performing, together\nwith Lamb, a context-sensitive lexical analysis in lexically-ambiguous language\nspecifications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.0504v1",
            "date": "2012-03-02 16:03:41+00:00",
            "title": "Modelling Social Structures and Hierarchies in Language Evolution",
            "authors": [
                "Martin Bachwerk",
                "Carl Vogel"
            ],
            "abstract": "Language evolution might have preferred certain prior social configurations\nover others. Experiments conducted with models of different social structures\n(varying subgroup interactions and the role of a dominant interlocutor) suggest\nthat having isolated agent groups rather than an interconnected agent is more\nadvantageous for the emergence of a social communication system. Distinctive\ngroups that are closely connected by communication yield systems less like\nnatural language than fully isolated groups inhabiting the same world.\nFurthermore, the addition of a dominant male who is asymmetrically favoured as\na hearer, and equally likely to be a speaker has no positive influence on the\ndisjoint groups.",
            "journal ref": "Research and Development in Intelligent Systems XXVII, 2011, pp.\n  49-62",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.0512v1",
            "date": "2012-03-02 16:25:13+00:00",
            "title": "Establishing linguistic conventions in task-oriented primeval dialogue",
            "authors": [
                "Martin Bachwerk",
                "Carl Vogel"
            ],
            "abstract": "In this paper, we claim that language is likely to have emerged as a\nmechanism for coordinating the solution of complex tasks. To confirm this\nthesis, computer simulations are performed based on the coordination task\npresented by Garrod & Anderson (1987). The role of success in task-oriented\ndialogue is analytically evaluated with the help of performance measurements\nand a thorough lexical analysis of the emergent communication system.\nSimulation results confirm a strong effect of success mattering on both\nreliability and dispersion of linguistic conventions.",
            "journal ref": "Analysis of Verbal and Nonverbal Communication and Enactment. The\n  Processing Issues. Lecture Notes in Computer Science, Volume 6800/2011, 2011,\n  pp. 48-55",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.1743v1",
            "date": "2012-03-08 10:47:07+00:00",
            "title": "Variable types for meaning assembly: a logical syntax for generic noun phrases introduced by most",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "This paper proposes a way to compute the meanings associated with sentences\nwith generic noun phrases corresponding to the generalized quantifier most. We\ncall these generics specimens and they resemble stereotypes or prototypes in\nlexical semantics. The meanings are viewed as logical formulae that can\nthereafter be interpreted in your favourite models. To do so, we depart\nsignificantly from the dominant Fregean view with a single untyped universe.\nIndeed, our proposal adopts type theory with some hints from Hilbert\n\\epsilon-calculus (Hilbert, 1922; Avigad and Zach, 2008) and from medieval\nphilosophy, see e.g. de Libera (1993, 1996). Our type theoretic analysis bears\nsome resemblance with ongoing work in lexical semantics (Asher 2011; Bassac et\nal. 2010; Moot, Pr\\'evot and Retor\\'e 2011). Our model also applies to\nclassical examples involving a class, or a generic element of this class, which\nis not uttered but provided by the context. An outcome of this study is that,\nin the minimalism-contextualism debate, see Conrad (2011), if one adopts a type\ntheoretical view, terms encode the purely semantic meaning component while\ntheir typing is pragmatically determined.",
            "journal ref": "Recherches linguistiques de Vincennes 41 (2012) 18",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1203.3511v1",
            "date": "2012-03-15 11:17:56+00:00",
            "title": "Inference by Minimizing Size, Divergence, or their Sum",
            "authors": [
                "Sebastian Riedel",
                "David A. Smith",
                "Andrew McCallum"
            ],
            "abstract": "We speed up marginal inference by ignoring factors that do not significantly\ncontribute to overall accuracy. In order to pick a suitable subset of factors\nto ignore, we propose three schemes: minimizing the number of model factors\nunder a bound on the KL divergence between pruned and full models; minimizing\nthe KL divergence under a bound on factor count; and minimizing the weighted\nsum of KL divergence and factor count. All three problems are solved using an\napproximation of the KL divergence than can be calculated in terms of marginals\ncomputed on a simple seed graph. Applied to synthetic image denoising and to\nthree different types of NLP parsing models, this technique performs marginal\ninference up to 11 times faster than loopy BP, with graph sizes reduced up to\n98%-at comparable error in marginals and parsing accuracy. We also show that\nminimizing the weighted sum of divergence and size is substantially faster than\nminimizing either of the other objectives based on the approximation to\ndivergence presented here.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1203.4238v1",
            "date": "2012-03-19 20:01:00+00:00",
            "title": "Do Linguistic Style and Readability of Scientific Abstracts affect their Virality?",
            "authors": [
                "Marco Guerini",
                "Alberto Pepe",
                "Bruno Lepri"
            ],
            "abstract": "Reactions to textual content posted in an online social network show\ndifferent dynamics depending on the linguistic style and readability of the\nsubmitted content. Do similar dynamics exist for responses to scientific\narticles? Our intuition, supported by previous research, suggests that the\nsuccess of a scientific article depends on its content, rather than on its\nlinguistic style. In this article, we examine a corpus of scientific abstracts\nand three forms of associated reactions: article downloads, citations, and\nbookmarks. Through a class-based psycholinguistic analysis and readability\nindices tests, we show that certain stylistic and readability features of\nabstracts clearly concur in determining the success and viral capability of a\nscientific article.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1203.5188v1",
            "date": "2012-03-23 07:13:06+00:00",
            "title": "Semi-Automatically Extracting FAQs to Improve Accessibility of Software Development Knowledge",
            "authors": [
                "Stefan Hen\u00df",
                "Martin Monperrus",
                "Mira Mezini"
            ],
            "abstract": "Frequently asked questions (FAQs) are a popular way to document software\ndevelopment knowledge. As creating such documents is expensive, this paper\npresents an approach for automatically extracting FAQs from sources of software\ndevelopment discussion, such as mailing lists and Internet forums, by combining\ntechniques of text mining and natural language processing. We apply the\napproach to popular mailing lists and carry out a survey among software\ndevelopers to show that it is able to extract high-quality FAQs that may be\nfurther improved by experts.",
            "journal ref": "ICSE - 34th International Conference on Software Engineering, 2012",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1203.5502v1",
            "date": "2012-03-25 14:56:04+00:00",
            "title": "Exploring Text Virality in Social Networks",
            "authors": [
                "Marco Guerini",
                "Carlo Strapparava",
                "Gozde Ozbal"
            ],
            "abstract": "This paper aims to shed some light on the concept of virality - especially in\nsocial networks - and to provide new insights on its structure. We argue that:\n(a) virality is a phenomenon strictly connected to the nature of the content\nbeing spread, rather than to the influencers who spread it, (b) virality is a\nphenomenon with many facets, i.e. under this generic term several different\neffects of persuasive communication are comprised and they only partially\noverlap. To give ground to our claims, we provide initial experiments in a\nmachine learning framework to show how various aspects of virality can be\nindependently predicted according to content features.",
            "journal ref": "Proceedings of the Fifth International AAAI Conference on Weblogs\n  and Social Media (ICWSM 2011), 17-21 July 2011, Barcelona, Spain",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1203.6339v1",
            "date": "2012-03-28 18:59:40+00:00",
            "title": "Intelligent Interface Architectures for Folksonomy Driven Structure Network",
            "authors": [
                "Massimiliano Dal Mas"
            ],
            "abstract": "The folksonomy is the result of free personal information or assignment of\ntags to an object (determined by the URI) in order to find them. The practice\nof tagging is done in a collective environment. Folksonomies are self\nconstructed, based on co-occurrence of definitions, rather than a hierarchical\nstructure of the data. The downside of this was that a few sites and\napplications are able to successfully exploit the sharing of bookmarks. The\nneed for tools that are able to resolve the ambiguity of the definitions is\nbecoming urgent as the need of simple instruments for their visualization,\nediting and exploitation in web applications still hinders their diffusion and\nwide adoption. An intelligent interactive interface design for folksonomies\nshould consider the contextual design and inquiry based on a concurrent\ninteraction for a perceptual user interfaces. To represent folksonomies a new\nconcept structure called \"Folksodriven\" is used in this paper. While it is\npresented the Folksodriven Structure Network (FSN) to resolve the ambiguity of\ndefinitions of folksonomy tags suggestions for the user. On this base a\nHuman-Computer Interactive (HCI) systems is developed for the visualization,\nnavigation, updating and maintenance of folksonomies Knowledge Bases - the FSN\n- through the web. System functionalities as well as its internal architecture\nwill be introduced.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1203.6360v2",
            "date": "2012-03-28 20:00:20+00:00",
            "title": "You had me at hello: How phrasing affects memorability",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Justin Cheng",
                "Jon Kleinberg",
                "Lillian Lee"
            ],
            "abstract": "Understanding the ways in which information achieves widespread public\nawareness is a research question of significant interest. We consider whether,\nand how, the way in which the information is phrased --- the choice of words\nand sentence structure --- can affect this process. To this end, we develop an\nanalysis framework and build a corpus of movie quotes, annotated with\nmemorability information, in which we are able to control for both the speaker\nand the setting of the quotes. We find that there are significant differences\nbetween memorable and non-memorable quotes in several key dimensions, even\nafter controlling for situational and contextual factors. One is lexical\ndistinctiveness: in aggregate, memorable quotes use less common word choices,\nbut at the same time are built upon a scaffolding of common syntactic patterns.\nAnother is that memorable quotes tend to be more general in ways that make them\neasy to apply in new contexts --- that is, more portable. We also show how the\nconcept of \"memorable language\" can be extended across domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.1639v1",
            "date": "2012-05-08 09:25:14+00:00",
            "title": "Spectral Analysis of Projection Histogram for Enhancing Close matching character Recognition in Malayalam",
            "authors": [
                "Sajilal Divakaran"
            ],
            "abstract": "The success rates of Optical Character Recognition (OCR) systems for printed\nMalayalam documents is quite impressive with the state of the art accuracy\nlevels in the range of 85-95% for various. However for real applications,\nfurther enhancement of this accuracy levels are required. One of the bottle\nnecks in further enhancement of the accuracy is identified as close-matching\ncharacters. In this paper, we delineate the close matching characters in\nMalayalam and report the development of a specialised classifier for these\nclose-matching characters. The output of a state of the art of OCR is taken and\ncharacters falling into the close-matching character set is further fed into\nthis specialised classifier for enhancing the accuracy. The classifier is based\non support vector machine algorithm and uses feature vectors derived out of\nspectral coefficients of projection histogram signals of close-matching\ncharacters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.2657v1",
            "date": "2012-05-09 14:53:11+00:00",
            "title": "Multilingual Topic Models for Unaligned Text",
            "authors": [
                "Jordan Boyd-Graber",
                "David Blei"
            ],
            "abstract": "We develop the multilingual topic model for unaligned text (MuTo), a\nprobabilistic model of text that is designed to analyze corpora composed of\ndocuments in two languages. From these documents, MuTo uses stochastic EM to\nsimultaneously discover both a matching between the languages and multilingual\nlatent topics. We demonstrate that MuTo is able to find shared topics on\nreal-world multilingual corpora, successfully pairing related documents across\nlanguages. MuTo provides a new framework for creating multilingual topic models\nwithout needing carefully curated parallel corpora and allows applications\nbuilt using the topic model formalism to be applied to a much wider class of\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1205.4324v2",
            "date": "2012-05-19 13:09:32+00:00",
            "title": "Universal Properties of Mythological Networks",
            "authors": [
                "P\u00e1draig Mac Carron",
                "Ralph Kenna"
            ],
            "abstract": "As in statistical physics, the concept of universality plays an important,\nalbeit qualitative, role in the field of comparative mythology. Here we apply\nstatistical mechanical tools to analyse the networks underlying three iconic\nmythological narratives with a view to identifying common and distinguishing\nquantitative features. Of the three narratives, an Anglo-Saxon and a Greek text\nare mostly believed by antiquarians to be partly historically based while the\nthird, an Irish epic, is often considered to be fictional. Here we show that\nnetwork analysis is able to discriminate real from imaginary social networks\nand place mythological narratives on the spectrum between them. Moreover, the\nperceived artificiality of the Irish narrative can be traced back to anomalous\nfeatures associated with six characters. Considering these as amalgams of\nseveral entities or proxies, renders the plausibility of the Irish text\ncomparable to the others from a network-theoretic point of view.",
            "journal ref": "EPL 99 (2012) 28002",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1208.3530v1",
            "date": "2012-08-17 04:48:58+00:00",
            "title": "Leveraging Subjective Human Annotation for Clustering Historic Newspaper Articles",
            "authors": [
                "Haimonti Dutta",
                "William Chan",
                "Deepak Shankargouda",
                "Manoj Pooleery",
                "Axinia Radeva",
                "Kyle Rego",
                "Boyi Xie",
                "Rebecca Passonneau",
                "Austin Lee",
                "Barbara Taranto"
            ],
            "abstract": "The New York Public Library is participating in the Chronicling America\ninitiative to develop an online searchable database of historically significant\nnewspaper articles. Microfilm copies of the newspapers are scanned and high\nresolution Optical Character Recognition (OCR) software is run on them. The\ntext from the OCR provides a wealth of data and opinion for researchers and\nhistorians. However, categorization of articles provided by the OCR engine is\nrudimentary and a large number of the articles are labeled editorial without\nfurther grouping. Manually sorting articles into fine-grained categories is\ntime consuming if not impossible given the size of the corpus. This paper\nstudies techniques for automatic categorization of newspaper articles so as to\nenhance search and retrieval on the archive. We explore unsupervised (e.g.\nKMeans) and semi-supervised (e.g. constrained clustering) learning algorithms\nto develop article categorization schemes geared towards the needs of\nend-users. A pilot study was designed to understand whether there was unanimous\nagreement amongst patrons regarding how articles can be categorized. It was\nfound that the task was very subjective and consequently automated algorithms\nthat could deal with subjective labels were used. While the small scale pilot\nstudy was extremely helpful in designing machine learning algorithms, a much\nlarger system needs to be developed to collect annotations from users of the\narchive. The \"BODHI\" system currently being developed is a step in that\ndirection, allowing users to correct wrongly scanned OCR and providing keywords\nand tags for newspaper articles used frequently. On successful implementation\nof the beta version of this system, we hope that it can be integrated with\nexisting software being developed for the Chronicling America project.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1209.1751v1",
            "date": "2012-09-08 20:19:56+00:00",
            "title": "Information content versus word length in random typing",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Ferm\u00edn Moscoso del Prado Mart\u00edn"
            ],
            "abstract": "Recently, it has been claimed that a linear relationship between a measure of\ninformation content and word length is expected from word length optimization\nand it has been shown that this linearity is supported by a strong correlation\nbetween information content and word length in many languages (Piantadosi et\nal. 2011, PNAS 108, 3825-3826). Here, we study in detail some connections\nbetween this measure and standard information theory. The relationship between\nthe measure and word length is studied for the popular random typing process\nwhere a text is constructed by pressing keys at random from a keyboard\ncontaining letters and a space behaving as a word delimiter. Although this\nrandom process does not optimize word lengths according to information content,\nit exhibits a linear relationship between information content and word length.\nThe exact slope and intercept are presented for three major variants of the\nrandom typing process. A strong correlation between information content and\nword length can simply arise from the units making a word (e.g., letters) and\nnot necessarily from the interplay between a word and its context as proposed\nby Piantadosi et al. In itself, the linear relation does not entail the results\nof any optimization process.",
            "journal ref": "Journal of Statistical Mechanics, L12002 (2011)",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1209.4277v2",
            "date": "2012-09-19 15:16:20+00:00",
            "title": "Multi-Level Modeling of Quotation Families Morphogenesis",
            "authors": [
                "Elisa Omodei",
                "Thierry Poibeau",
                "Jean-Philippe Cointet"
            ],
            "abstract": "This paper investigates cultural dynamics in social media by examining the\nproliferation and diversification of clearly-cut pieces of content: quoted\ntexts. In line with the pioneering work of Leskovec et al. and Simmons et al.\non memes dynamics we investigate in deep the transformations that quotations\npublished online undergo during their diffusion. We deliberately put aside the\nstructure of the social network as well as the dynamical patterns pertaining to\nthe diffusion process to focus on the way quotations are changed, how often\nthey are modified and how these changes shape more or less diverse families and\nsub-families of quotations. Following a biological metaphor, we try to\nunderstand in which way mutations can transform quotations at different scales\nand how mutation rates depend on various properties of the quotations.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1211.2741v1",
            "date": "2012-11-12 19:17:34+00:00",
            "title": "A Hindi Speech Actuated Computer Interface for Web Search",
            "authors": [
                "Kamlesh Sharma",
                "S. V. A. V. Prasad",
                "T. V. Prasad"
            ],
            "abstract": "Aiming at increasing system simplicity and flexibility, an audio evoked based\nsystem was developed by integrating simplified headphone and user-friendly\nsoftware design. This paper describes a Hindi Speech Actuated Computer\nInterface for Web search (HSACIWS), which accepts spoken queries in Hindi\nlanguage and provides the search result on the screen. This system recognizes\nspoken queries by large vocabulary continuous speech recognition (LVCSR),\nretrieves relevant document by text retrieval, and provides the search result\non the Web by the integration of the Web and the voice systems. The LVCSR in\nthis system showed enough performance levels for speech with acoustic and\nlanguage models derived from a query corpus with target contents.",
            "journal ref": "International Journal of Advanced Computer Science and\n  Applications 3(10), 2012, 147-152",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1211.6847v1",
            "date": "2012-11-29 09:00:59+00:00",
            "title": "Letter counting: a stem cell for Cryptology, Quantitative Linguistics, and Statistics",
            "authors": [
                "Bernard Ycart"
            ],
            "abstract": "Counting letters in written texts is a very ancient practice. It has\naccompanied the development of Cryptology, Quantitative Linguistics, and\nStatistics. In Cryptology, counting frequencies of the different characters in\nan encrypted message is the basis of the so called frequency analysis method.\nIn Quantitative Linguistics, the proportion of vowels to consonants in\ndifferent languages was studied long before authorship attribution. In\nStatistics, the alternation vowel-consonants was the only example that Markov\never gave of his theory of chained events. A short history of letter counting\nis presented. The three domains, Cryptology, Quantitative Linguistics, and\nStatistics, are then examined, focusing on the interactions with the other two\nfields through letter counting. As a conclusion, the eclectism of past\ncenturies scholars, their background in humanities, and their familiarity with\ncryptograms, are identified as contributing factors to the mutual enrichment\nprocess which is described here.",
            "journal ref": "Historiographia Linguistica 40, 3 (2013) 303-329",
            "category": "math.HO"
        },
        {
            "url": "http://arxiv.org/abs/1204.1615v1",
            "date": "2012-04-07 09:28:19+00:00",
            "title": "Discrimination between Arabic and Latin from bilingual documents",
            "authors": [
                "Sofiene Haboubi",
                "Samia Maddouri",
                "Hamid Amiri"
            ],
            "abstract": "2011 International Conference on Communications, Computing and Control\nApplications (CCCA)",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1204.2523v1",
            "date": "2012-04-11 18:53:58+00:00",
            "title": "Concept Modeling with Superwords",
            "authors": [
                "Khalid El-Arini",
                "Emily B. Fox",
                "Carlos Guestrin"
            ],
            "abstract": "In information retrieval, a fundamental goal is to transform a document into\nconcepts that are representative of its content. The term \"representative\" is\nin itself challenging to define, and various tasks require different\ngranularities of concepts. In this paper, we aim to model concepts that are\nsparse over the vocabulary, and that flexibly adapt their content based on\nother relevant semantic information such as textual structure or associated\nimage features. We explore a Bayesian nonparametric model based on nested beta\nprocesses that allows for inferring an unknown number of strictly sparse\nconcepts. The resulting model provides an inherently different representation\nof concepts than a standard LDA (or HDP) based topic model, and allows for\ndirect incorporation of semantic features. We demonstrate the utility of this\nrepresentation on multilingual blog data and the Congressional Record.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1204.2765v2",
            "date": "2012-04-12 16:21:10+00:00",
            "title": "A practical approach to language complexity: a Wikipedia case study",
            "authors": [
                "Taha Yasseri",
                "Andr\u00e1s Kornai",
                "J\u00e1nos Kert\u00e9sz"
            ],
            "abstract": "In this paper we present statistical analysis of English texts from\nWikipedia. We try to address the issue of language complexity empirically by\ncomparing the simple English Wikipedia (Simple) to comparable samples of the\nmain English Wikipedia (Main). Simple is supposed to use a more simplified\nlanguage with a limited vocabulary, and editors are explicitly requested to\nfollow this guideline, yet in practice the vocabulary richness of both samples\nare at the same level. Detailed analysis of longer units (n-grams of words and\npart of speech tags) shows that the language of Simple is less complex than\nthat of Main primarily due to the use of shorter sentences, as opposed to\ndrastically simplified syntax or vocabulary. Comparing the two language\nvarieties by the Gunning readability index supports this conclusion. We also\nreport on the topical dependence of language complexity, e.g. that the language\nis more advanced in conceptual articles compared to person-based (biographical)\nand object-based articles. Finally, we investigate the relation between\nconflict and language complexity by analyzing the content of the talk pages\nassociated to controversial and peacefully developing articles, concluding that\ncontroversy has the effect of reducing language complexity.",
            "journal ref": "PLoS ONE 7(11): e48386 (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1204.2804v1",
            "date": "2012-04-12 18:25:04+00:00",
            "title": "Estimating the Prevalence of Deception in Online Review Communities",
            "authors": [
                "Myle Ott",
                "Claire Cardie",
                "Jeff Hancock"
            ],
            "abstract": "Consumers' purchase decisions are increasingly influenced by user-generated\nonline reviews. Accordingly, there has been growing concern about the potential\nfor posting \"deceptive opinion spam\" -- fictitious reviews that have been\ndeliberately written to sound authentic, to deceive the reader. But while this\npractice has received considerable public attention and concern, relatively\nlittle is known about the actual prevalence, or rate, of deception in online\nreview communities, and less still about the factors that influence it.\n  We propose a generative model of deception which, in conjunction with a\ndeception classifier, we use to explore the prevalence of deception in six\npopular online review communities: Expedia, Hotels.com, Orbitz, Priceline,\nTripAdvisor, and Yelp. We additionally propose a theoretical model of online\nreviews based on economic signaling theory, in which consumer reviews diminish\nthe inherent information asymmetry between consumers and producers, by acting\nas a signal to a product's true, unknown quality. We find that deceptive\nopinion spam is a growing problem overall, but with different growth rates\nacross communities. These rates, we argue, are driven by the different\nsignaling costs associated with deception for each review community, e.g.,\nposting requirements. When measures are taken to increase signaling cost, e.g.,\nfiltering reviews written by first-time reviewers, deception prevalence is\neffectively reduced.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1204.3498v2",
            "date": "2012-04-16 14:27:39+00:00",
            "title": "A Computational Analysis of Collective Discourse",
            "authors": [
                "Vahed Qazvinian",
                "Dragomir R. Radev"
            ],
            "abstract": "This paper is focused on the computational analysis of collective discourse,\na collective behavior seen in non-expert content contributions in online social\nmedia. We collect and analyze a wide range of real-world collective discourse\ndatasets from movie user reviews to microblogs and news headlines to scientific\ncitations. We show that all these datasets exhibit diversity of perspective, a\nproperty seen in other collective systems and a criterion in wise crowds. Our\nexperiments also confirm that the network of different perspective\nco-occurrences exhibits the small-world property with high clustering of\ndifferent perspectives. Finally, we show that non-expert contributions in\ncollective discourse can be used to answer simple questions that are otherwise\nhard to answer.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1204.3731v1",
            "date": "2012-04-17 08:58:39+00:00",
            "title": "Towards Real-Time Summarization of Scheduled Events from Twitter Streams",
            "authors": [
                "Arkaitz Zubiaga",
                "Damiano Spina",
                "Enrique Amig\u00f3",
                "Julio Gonzalo"
            ],
            "abstract": "This paper explores the real-time summarization of scheduled events such as\nsoccer games from torrential flows of Twitter streams. We propose and evaluate\nan approach that substantially shrinks the stream of tweets in real-time, and\nconsists of two steps: (i) sub-event detection, which determines if something\nnew has occurred, and (ii) tweet selection, which picks a representative tweet\nto describe each sub-event. We compare the summaries generated in three\nlanguages for all the soccer games in \"Copa America 2011\" to reference live\nreports offered by Yahoo! Sports journalists. We show that simple text analysis\nmethods which do not involve external knowledge lead to summaries that cover\n84% of the sub-events on average, and 100% of key types of sub-events (such as\ngoals in soccer). Our approach should be straightforwardly applicable to other\nkinds of scheduled events such as other sports, award ceremonies, keynote\ntalks, TV shows, etc.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1204.4346v1",
            "date": "2012-04-19 13:37:34+00:00",
            "title": "Your Two Weeks of Fame and Your Grandmother's",
            "authors": [
                "James Cook",
                "Atish Das Sarma",
                "Alex Fabrikant",
                "Andrew Tomkins"
            ],
            "abstract": "Did celebrity last longer in 1929, 1992 or 2009? We investigate the\nphenomenon of fame by mining a collection of news articles that spans the\ntwentieth century, and also perform a side study on a collection of blog posts\nfrom the last 10 years. By analyzing mentions of personal names, we measure\neach person's time in the spotlight, using two simple metrics that evaluate,\nroughly, the duration of a single news story about a person, and the overall\nduration of public interest in a person. We watched the distribution evolve\nfrom 1895 to 2010, expecting to find significantly shortening fame durations,\nper the much popularly bemoaned shortening of society's attention spans and\nquickening of media's news cycles. Instead, we conclusively demonstrate that,\nthrough many decades of rapid technological and societal change, through the\nappearance of Twitter, communication satellites, and the Internet, fame\ndurations did not decrease, neither for the typical case nor for the extremely\nfamous, with the last statistically significant fame duration decreases coming\nin the early 20th century, perhaps from the spread of telegraphy and telephony.\nFurthermore, while median fame durations stayed persistently constant, for the\nmost famous of the famous, as measured by either volume or duration of media\nattention, fame durations have actually trended gently upward since the 1940s,\nwith statistically significant increases on 40-year timescales. Similar studies\nhave been done with much shorter timescales specifically in the context of\ninformation spreading on Twitter and similar social networking sites. To the\nbest of our knowledge, this is the first massive scale study of this nature\nthat spans over a century of archived data, thereby allowing us to track\nchanges across decades.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1204.4914v1",
            "date": "2012-04-22 17:17:56+00:00",
            "title": "Quantum Interference in Cognition: Structural Aspects of the Brain",
            "authors": [
                "Diederik Aerts",
                "Sandro Sozzo"
            ],
            "abstract": "We identify the presence of typically quantum effects, namely 'superposition'\nand 'interference', in what happens when human concepts are combined, and\nprovide a quantum model in complex Hilbert space that represents faithfully\nexperimental data measuring the situation of combining concepts. Our model\nshows how 'interference of concepts' explains the effects of underextension and\noverextension when two concepts combine to the disjunction of these two\nconcepts. This result supports our earlier hypothesis that human thought has a\nsuperposed two-layered structure, one layer consisting of 'classical logical\nthought' and a superposed layer consisting of 'quantum conceptual thought'.\nPossible connections with recent findings of a 'grid-structure' for the brain\nare analyzed, and influences on the mind/brain relation, and consequences on\napplied disciplines, such as artificial intelligence and quantum computation,\nare considered.",
            "journal ref": "In V. Ovchinnikov and P. Dini (Eds.), IARIA, Proceedings of the\n  Sixth International Conference on Quantum, Nano and Micro Technologies, pp.\n  33-41, 2012",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1204.6441v1",
            "date": "2012-04-28 22:18:06+00:00",
            "title": "\"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper\" -- A Balanced Survey on Election Prediction using Twitter Data",
            "authors": [
                "Daniel Gayo-Avello"
            ],
            "abstract": "Predicting X from Twitter is a popular fad within the Twitter research\nsubculture. It seems both appealing and relatively easy. Among such kind of\nstudies, electoral prediction is maybe the most attractive, and at this moment\nthere is a growing body of literature on such a topic. This is not only an\ninteresting research problem but, above all, it is extremely difficult.\nHowever, most of the authors seem to be more interested in claiming positive\nresults than in providing sound and reproducible methods. It is also especially\nworrisome that many recent papers seem to only acknowledge those studies\nsupporting the idea of Twitter predicting elections, instead of conducting a\nbalanced literature review showing both sides of the matter. After reading many\nof such papers I have decided to write such a survey myself. Hence, in this\npaper, every study relevant to the matter of electoral prediction using social\nmedia is commented. From this review it can be concluded that the predictive\npower of Twitter regarding elections has been greatly exaggerated, and that\nhard research problems still lie ahead.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1207.0658v1",
            "date": "2012-07-03 12:57:43+00:00",
            "title": "On the origin of long-range correlations in texts",
            "authors": [
                "Eduardo G. Altmann",
                "Giampaolo Cristadoro",
                "Mirko Degli Esposti"
            ],
            "abstract": "The complexity of human interactions with social and natural phenomena is\nmirrored in the way we describe our experiences through natural language. In\norder to retain and convey such a high dimensional information, the statistical\nproperties of our linguistic output has to be highly correlated in time. An\nexample are the robust observations, still largely not understood, of\ncorrelations on arbitrary long scales in literary texts. In this paper we\nexplain how long-range correlations flow from highly structured linguistic\nlevels down to the building blocks of a text (words, letters, etc..). By\ncombining calculations and data analysis we show that correlations take form of\na bursty sequence of events once we approach the semantically relevant topics\nof the text. The mechanisms we identify are fairly general and can be equally\napplied to other hierarchical settings.",
            "journal ref": "Proc. Natl. Acad. Sci. USA 109, 11582 (2012)",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1207.0742v1",
            "date": "2012-07-03 16:35:48+00:00",
            "title": "The OS* Algorithm: a Joint Approach to Exact Optimization and Sampling",
            "authors": [
                "Marc Dymetman",
                "Guillaume Bouchard",
                "Simon Carter"
            ],
            "abstract": "Most current sampling algorithms for high-dimensional distributions are based\non MCMC techniques and are approximate in the sense that they are valid only\nasymptotically. Rejection sampling, on the other hand, produces valid samples,\nbut is unrealistically slow in high-dimension spaces. The OS* algorithm that we\npropose is a unified approach to exact optimization and sampling, based on\nincremental refinements of a functional upper bound, which combines ideas of\nadaptive rejection sampling and of A* optimization search. We show that the\nchoice of the refinement can be done in a way that ensures tractability in\nhigh-dimension spaces, and we present first experiments in two different\nsettings: inference in high-order HMMs and in large discrete graphical models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1207.3169v2",
            "date": "2012-07-13 08:43:17+00:00",
            "title": "The law of brevity in macaque vocal communication is not an artifact of analyzing mean call durations",
            "authors": [
                "Stuart Semple",
                "Minna J. Hsu",
                "Govindasamy Agoramoorthy",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Words follow the law of brevity, i.e. more frequent words tend to be shorter.\nFrom a statistical point of view, this qualitative definition of the law states\nthat word length and word frequency are negatively correlated. Here the recent\nfinding of patterning consistent with the law of brevity in Formosan macaque\nvocal communication (Semple et al., 2010) is revisited. It is shown that the\nnegative correlation between mean duration and frequency of use in the\nvocalizations of Formosan macaques is not an artifact of the use of a mean\nduration for each call type instead of the customary 'word' length of studies\nof the law in human language. The key point demonstrated is that the total\nduration of calls of a particular type increases with the number of calls of\nthat type. The finding of the law of brevity in the vocalizations of these\nmacaques therefore defies a trivial explanation.",
            "journal ref": "Journal of Quantitative Linguistics, 20 (3), 209-217 (2013)",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1210.0848v1",
            "date": "2012-10-02 17:23:25+00:00",
            "title": "Enhancing Twitter Data Analysis with Simple Semantic Filtering: Example in Tracking Influenza-Like Illnesses",
            "authors": [
                "Son Doan",
                "Lucila Ohno-Machado",
                "Nigel Collier"
            ],
            "abstract": "Systems that exploit publicly available user generated content such as\nTwitter messages have been successful in tracking seasonal influenza. We\ndeveloped a novel filtering method for Influenza-Like-Illnesses (ILI)-related\nmessages using 587 million messages from Twitter micro-blogs. We first filtered\nmessages based on syndrome keywords from the BioCaster Ontology, an extant\nknowledge model of laymen's terms. We then filtered the messages according to\nsemantic features such as negation, hashtags, emoticons, humor and geography.\nThe data covered 36 weeks for the US 2009 influenza season from 30th August\n2009 to 8th May 2010. Results showed that our system achieved the highest\nPearson correlation coefficient of 98.46% (p-value<2.2e-16), an improvement of\n3.98% over the previous state-of-the-art method. The results indicate that\nsimple NLP-based enhancements to existing approaches to mine Twitter data can\nincrease the value of this inexpensive resource.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1210.3312v1",
            "date": "2012-10-11 18:21:01+00:00",
            "title": "Artex is AnotheR TEXt summarizer",
            "authors": [
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "This paper describes Artex, another algorithm for Automatic Text\nSummarization. In order to rank sentences, a simple inner product is calculated\nbetween each sentence, a document vector (text topic) and a lexical vector\n(vocabulary used by a sentence). Summaries are then generated by assembling the\nhighest ranked sentences. No ruled-based linguistic post-processing is\nnecessary in order to obtain summaries. Tests over several datasets (coming\nfrom Document Understanding Conferences (DUC), Text Analysis Conferences (TAC),\nevaluation campaigns, etc.) in French, English and Spanish have shown that\nsummarizer achieves interesting results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1210.3865v1",
            "date": "2012-10-15 00:58:11+00:00",
            "title": "Opinion Mining for Relating Subjective Expressions and Annual Earnings in US Financial Statements",
            "authors": [
                "Chien-Liang Chen",
                "Chao-Lin Liu",
                "Yuan-Chen Chang",
                "Hsiang-Ping Tsai"
            ],
            "abstract": "Financial statements contain quantitative information and manager's\nsubjective evaluation of firm's financial status. Using information released in\nU.S. 10-K filings. Both qualitative and quantitative appraisals are crucial for\nquality financial decisions. To extract such opinioned statements from the\nreports, we built tagging models based on the conditional random field (CRF)\ntechniques, considering a variety of combinations of linguistic factors\nincluding morphology, orthography, predicate-argument structure, syntax, and\nsimple semantics. Our results show that the CRF models are reasonably effective\nto find opinion holders in experiments when we adopted the popular MPQA corpus\nfor training and testing. The contribution of our paper is to identify opinion\npatterns in multiword expressions (MWEs) forms rather than in single word\nforms.\n  We find that the managers of corporations attempt to use more optimistic\nwords to obfuscate negative financial performance and to accentuate the\npositive financial performance. Our results also show that decreasing earnings\nwere often accompanied by ambiguous and mild statements in the reporting year\nand that increasing earnings were stated in assertive and positive way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.3926v2",
            "date": "2012-10-15 07:36:57+00:00",
            "title": "Learning Attitudes and Attributes from Multi-Aspect Reviews",
            "authors": [
                "Julian McAuley",
                "Jure Leskovec",
                "Dan Jurafsky"
            ],
            "abstract": "The majority of online reviews consist of plain-text feedback together with a\nsingle numeric score. However, there are multiple dimensions to products and\nopinions, and understanding the `aspects' that contribute to users' ratings may\nhelp us to better understand their individual preferences. For example, a\nuser's impression of an audiobook presumably depends on aspects such as the\nstory and the narrator, and knowing their opinions on these aspects may help us\nto recommend better products. In this paper, we build models for rating systems\nin which such dimensions are explicit, in the sense that users leave separate\nratings for each aspect of a product. By introducing new corpora consisting of\nfive million reviews, rated with between three and six aspects, we evaluate our\nmodels on three prediction tasks: First, we use our model to uncover which\nparts of a review discuss which of the rated aspects. Second, we use our model\nto summarize reviews, which for us means finding the sentences that best\nexplain a user's rating. Finally, since aspect ratings are optional in many of\nthe datasets we consider, we use our model to recover those ratings that are\nmissing from a user's evaluation. Our model matches state-of-the-art approaches\non existing small-scale datasets, while scaling to the real-world datasets we\nintroduce. Moreover, our model is able to `disentangle' content and sentiment\nwords: we automatically learn content words that are indicative of a particular\naspect as well as the aspect-specific sentiment words that are indicative of a\nparticular rating.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.4871v1",
            "date": "2012-10-16 17:41:30+00:00",
            "title": "Learning Mixtures of Submodular Shells with Application to Document Summarization",
            "authors": [
                "Hui Lin",
                "Jeff A. Bilmes"
            ],
            "abstract": "We introduce a method to learn a mixture of submodular \"shells\" in a\nlarge-margin setting. A submodular shell is an abstract submodular function\nthat can be instantiated with a ground set and a set of parameters to produce a\nsubmodular function. A mixture of such shells can then also be so instantiated\nto produce a more complex submodular function. What our algorithm learns are\nthe mixture weights over such shells. We provide a risk bound guarantee when\nlearning in a large-margin structured-prediction setting using a projected\nsubgradient method when only approximate submodular optimization is possible\n(such as with submodular function maximization). We apply this method to the\nproblem of multi-document summarization and produce the best results reported\nso far on the widely used NIST DUC-05 through DUC-07 document summarization\ncorpora.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1210.5268v4",
            "date": "2012-10-18 21:46:09+00:00",
            "title": "Diffusion of Lexical Change in Social Media",
            "authors": [
                "Jacob Eisenstein",
                "Brendan O'Connor",
                "Noah A. Smith",
                "Eric P. Xing"
            ],
            "abstract": "Computer-mediated communication is driving fundamental changes in the nature\nof written language. We investigate these changes by statistical analysis of a\ndataset comprising 107 million Twitter messages (authored by 2.7 million unique\nuser accounts). Using a latent vector autoregressive model to aggregate across\nthousands of words, we identify high-level patterns in diffusion of linguistic\nchange over the United States. Our model is robust to unpredictable changes in\nTwitter's sampling rate, and provides a probabilistic characterization of the\nrelationship of macro-scale linguistic influence to a set of demographic and\ngeographic predictors. The results of this analysis offer support for prior\narguments that focus on geographical proximity and population size. However,\ndemographic similarity -- especially with regard to race -- plays an even more\ncentral role, as cities with similar racial demographics are far more likely to\nshare linguistic influence. Rather than moving towards a single unified\n\"netspeak\" dialect, language evolution in computer-mediated communication\nreproduces existing fault lines in spoken American English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5581v1",
            "date": "2012-10-20 06:09:11+00:00",
            "title": "Hidden Trends in 90 Years of Harvard Business Review",
            "authors": [
                "Chia-Chi Tsai",
                "Chao-Lin Liu",
                "Wei-Jie Huang",
                "Man-Kwan Shan"
            ],
            "abstract": "In this paper, we demonstrate and discuss results of our mining the abstracts\nof the publications in Harvard Business Review between 1922 and 2012.\nTechniques for computing n-grams, collocations, basic sentiment analysis, and\nnamed-entity recognition were employed to uncover trends hidden in the\nabstracts. We present findings about international relationships, sentiment in\nHBR's abstracts, important international companies, influential technological\ninventions, renown researchers in management theories, US presidents via\nchronological analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.5898v1",
            "date": "2012-10-22 13:56:46+00:00",
            "title": "Some Chances and Challenges in Applying Language Technologies to Historical Studies in Chinese",
            "authors": [
                "Chao-Lin Liu",
                "Guantao Jin",
                "Qingfeng Liu",
                "Wei-Yun Chiu",
                "Yih-Soong Yu"
            ],
            "abstract": "We report applications of language technology to analyzing historical\ndocuments in the Database for the Study of Modern Chinese Thoughts and\nLiterature (DSMCTL). We studied two historical issues with the reported\ntechniques: the conceptualization of \"huaren\" (Chinese people) and the attempt\nto institute constitutional monarchy in the late Qing dynasty. We also discuss\nresearch challenges for supporting sophisticated issues using our experience\nwith DSMCTL, the Database of Government Officials of the Republic of China, and\nthe Dream of the Red Chamber. Advanced techniques and tools for lexical,\nsyntactic, semantic, and pragmatic processing of language information, along\nwith more thorough data collection, are needed to strengthen the collaboration\nbetween historians and computer scientists.",
            "journal ref": "International Journal of Computational Linguistics and Chinese\n  Language Processing, 16(1-2), 27-46, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1210.7599v2",
            "date": "2012-10-29 09:18:34+00:00",
            "title": "The automatic creation of concept maps from documents written using morphologically rich languages",
            "authors": [
                "Krunoslav Zubrinic",
                "Damir Kalpic",
                "Mario Milicevic"
            ],
            "abstract": "Concept map is a graphical tool for representing knowledge. They have been\nused in many different areas, including education, knowledge management,\nbusiness and intelligence. Constructing of concept maps manually can be a\ncomplex task; an unskilled person may encounter difficulties in determining and\npositioning concepts relevant to the problem area. An application that\nrecommends concept candidates and their position in a concept map can\nsignificantly help the user in that situation. This paper gives an overview of\ndifferent approaches to automatic and semi-automatic creation of concept maps\nfrom textual and non-textual sources. The concept map mining process is\ndefined, and one method suitable for the creation of concept maps from\nunstructured textual sources in highly inflected languages such as the Croatian\nlanguage is described in detail. Proposed method uses statistical and data\nmining techniques enriched with linguistic tools. With minor adjustments, that\nmethod can also be used for concept map mining from textual sources in other\nmorphologically rich languages.",
            "journal ref": "Expert Systems with Applications, Volume 39, Issue 16, 2012, Pages\n  12709-12718",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1212.0927v3",
            "date": "2012-12-05 03:50:46+00:00",
            "title": "Two Algorithms for Finding $k$ Shortest Paths of a Weighted Pushdown Automaton",
            "authors": [
                "Ke Wu",
                "Philip Resnik"
            ],
            "abstract": "We introduce efficient algorithms for finding the $k$ shortest paths of a\nweighted pushdown automaton (WPDA), a compact representation of a weighted set\nof strings with potential applications in parsing and machine translation. Both\nof our algorithms are derived from the same weighted deductive logic\ndescription of the execution of a WPDA using different search strategies.\nExperimental results show our Algorithm 2 adds very little overhead vs. the\nsingle shortest path algorithm, even with a large $k$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.1362v3",
            "date": "2012-12-06 16:06:21+00:00",
            "title": "Stochastic model for the vocabulary growth in natural languages",
            "authors": [
                "Martin Gerlach",
                "Eduardo G. Altmann"
            ],
            "abstract": "We propose a stochastic model for the number of different words in a given\ndatabase which incorporates the dependence on the database size and historical\nchanges. The main feature of our model is the existence of two different\nclasses of words: (i) a finite number of core-words which have higher frequency\nand do not affect the probability of a new word to be used; and (ii) the\nremaining virtually infinite number of noncore-words which have lower frequency\nand once used reduce the probability of a new word to be used in the future.\nOur model relies on a careful analysis of the google-ngram database of books\npublished in the last centuries and its main consequence is the generalization\nof Zipf's and Heaps' law to two scaling regimes. We confirm that these\ngeneralizations yield the best simple description of the data among generic\ndescriptive models and that the two free parameters depend only on the language\nbut not on the database. From the point of view of our model the main change on\nhistorical time scales is the composition of the specific words included in the\nfinite list of core-words, which we observe to decay exponentially in time with\na rate of approximately 30 words per year for English.",
            "journal ref": "Phys. Rev. X 3, 021006 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1212.1478v1",
            "date": "2012-12-06 21:28:19+00:00",
            "title": "The Clustering of Author's Texts of English Fiction in the Vector Space of Semantic Fields",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "The clustering of text documents in the vector space of semantic fields and\nin the semantic space with orthogonal basis has been analysed. It is shown that\nusing the vector space model with the basis of semantic fields is effective in\nthe cluster analysis algorithms of author's texts in English fiction. The\nanalysis of the author's texts distribution in cluster structure showed the\npresence of the areas of semantic space that represent the author's ideolects\nof individual authors. SVD factorization of the semantic fields matrix makes it\npossible to reduce significantly the dimension of the semantic space in the\ncluster analysis of author's texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.1709v1",
            "date": "2012-12-07 20:57:11+00:00",
            "title": "Evolution of the most common English words and phrases over the centuries",
            "authors": [
                "Matjaz Perc"
            ],
            "abstract": "By determining which were the most common English words and phrases since the\nbeginning of the 16th century, we obtain a unique large-scale view of the\nevolution of written text. We find that the most common words and phrases in\nany given year had a much shorter popularity lifespan in the 16th than they had\nin the 20th century. By measuring how their usage propagated across the years,\nwe show that for the past two centuries the process has been governed by linear\npreferential attachment. Along with the steady growth of the English lexicon,\nthis provides an empirical explanation for the ubiquity of the Zipf's law in\nlanguage statistics and confirms that writing, although undoubtedly an\nexpression of art and skill, is not immune to the same influences of\nself-organization that are known to regulate processes as diverse as the making\nof new friends and World Wide Web growth.",
            "journal ref": "J. R. Soc. Interface 9 (2012) 3323-3328",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1212.2616v1",
            "date": "2012-12-11 20:32:05+00:00",
            "title": "Languages cool as they expand: Allometric scaling and the decreasing need for new words",
            "authors": [
                "Alexander M. Petersen",
                "Joel N. Tenenbaum",
                "Shlomo Havlin",
                "H. Eugene Stanley",
                "Matjaz Perc"
            ],
            "abstract": "We analyze the occurrence frequencies of over 15 million words recorded in\nmillions of books published during the past two centuries in seven different\nlanguages. For all languages and chronological subsets of the data we confirm\nthat two scaling regimes characterize the word frequency distributions, with\nonly the more common words obeying the classic Zipf law. Using corpora of\nunprecedented size, we test the allometric scaling relation between the corpus\nsize and the vocabulary size of growing languages to demonstrate a decreasing\nmarginal need for new words, a feature that is likely related to the underlying\ncorrelations between words. We calculate the annual growth fluctuations of word\nuse which has a decreasing trend as the corpus size increases, indicating a\nslowdown in linguistic evolution following language expansion. This \"cooling\npattern\" forms the basis of a third statistical regularity, which unlike the\nZipf and the Heaps law, is dynamical in nature.",
            "journal ref": "Sci. Rep. 2 (2012) 943",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1212.2676v1",
            "date": "2012-12-11 23:47:56+00:00",
            "title": "Mining the Web for the Voice of the Herd to Track Stock Market Bubbles",
            "authors": [
                "Aaron Gerow",
                "Mark Keane"
            ],
            "abstract": "We show that power-law analyses of financial commentaries from newspaper\nweb-sites can be used to identify stock market bubbles, supplementing\ntraditional volatility analyses. Using a four-year corpus of 17,713 online,\nfinance-related articles (10M+ words) from the Financial Times, the New York\nTimes, and the BBC, we show that week-to-week changes in power-law\ndistributions reflect market movements of the Dow Jones Industrial Average\n(DJI), the FTSE-100, and the NIKKEI-225. Notably, the statistical regularities\nin language track the 2007 stock market bubble, showing emerging structure in\nthe language of commentators, as progressively greater agreement arose in their\npositive perceptions of the market. Furthermore, during the bubble period, a\nmarked divergence in positive language occurs as revealed by a Kullback-Leibler\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.3171v1",
            "date": "2012-12-13 13:41:03+00:00",
            "title": "Multifractal analysis of sentence lengths in English literary texts",
            "authors": [
                "Iwona Grabska-Gradzi\u0144ska",
                "Andrzej Kulig",
                "Jaros\u0142aw Kwapie\u0144",
                "Pawe\u0142 O\u015bwi\u0119cimka",
                "Stanis\u0142aw Dro\u017cd\u017c"
            ],
            "abstract": "This paper presents analysis of 30 literary texts written in English by\ndifferent authors. For each text, there were created time series representing\nlength of sentences in words and analyzed its fractal properties using two\nmethods of multifractal analysis: MFDFA and WTMM. Both methods showed that\nthere are texts which can be considered multifractal in this representation but\na majority of texts are not multifractal or even not fractal at all. Out of 30\nbooks, only a few have so-correlated lengths of consecutive sentences that the\nanalyzed signals can be interpreted as real multifractals. An interesting\ndirection for future investigations would be identifying what are the specific\nfeatures which cause certain texts to be multifractal and other to be\nmonofractal or even not fractal at all.",
            "journal ref": "AWERProcedia Information Technology & Computer Science 03, 1700,\n  2013",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1212.3228v1",
            "date": "2012-12-11 20:19:58+00:00",
            "title": "Language Without Words: A Pointillist Model for Natural Language Processing",
            "authors": [
                "Peiyou Song",
                "Anhei Shu",
                "David Phipps",
                "Dan Wallach",
                "Mohit Tiwari",
                "Jedidiah Crandall",
                "George Luger"
            ],
            "abstract": "This paper explores two separate questions: Can we perform natural language\nprocessing tasks without a lexicon?; and, Should we? Existing natural language\nprocessing techniques are either based on words as units or use units such as\ngrams only for basic classification tasks. How close can a machine come to\nreasoning about the meanings of words and phrases in a corpus without using any\nlexicon, based only on grams?\n  Our own motivation for posing this question is based on our efforts to find\npopular trends in words and phrases from online Chinese social media. This form\nof written Chinese uses so many neologisms, creative character placements, and\ncombinations of writing systems that it has been dubbed the \"Martian Language.\"\nReaders must often use visual queues, audible queues from reading out loud, and\ntheir knowledge and understanding of current events to understand a post. For\nanalysis of popular trends, the specific problem is that it is difficult to\nbuild a lexicon when the invention of new ways to refer to a word or concept is\neasy and common. For natural language processing in general, we argue in this\npaper that new uses of language in social media will challenge machines'\nabilities to operate with words as the basic unit of understanding, not only in\nChinese but potentially in other languages.",
            "journal ref": "The 6th International Conference on Soft Computing and Intelligent\n  Systems (SCIS-ISIS 2012) Kobe, Japan",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1212.5238v1",
            "date": "2012-12-20 20:43:12+00:00",
            "title": "The Twitter of Babel: Mapping World Languages through Microblogging Platforms",
            "authors": [
                "Delia Mocanu",
                "Andrea Baronchelli",
                "Bruno Gon\u00e7alves",
                "Nicola Perra",
                "Alessandro Vespignani"
            ],
            "abstract": "Large scale analysis and statistics of socio-technical systems that just a\nfew short years ago would have required the use of consistent economic and\nhuman resources can nowadays be conveniently performed by mining the enormous\namount of digital data produced by human activities. Although a\ncharacterization of several aspects of our societies is emerging from the data\nrevolution, a number of questions concerning the reliability and the biases\ninherent to the big data \"proxies\" of social life are still open. Here, we\nsurvey worldwide linguistic indicators and trends through the analysis of a\nlarge-scale dataset of microblogging posts. We show that available data allow\nfor the study of language geography at scales ranging from country-level\naggregation to specific city neighborhoods. The high resolution and coverage of\nthe data allows us to investigate different indicators such as the linguistic\nhomogeneity of different countries, the touristic seasonal patterns within\ncountries and the geographical distribution of different languages in\nmultilingual regions. This work highlights the potential of geolocalized\nstudies of open data sources to improve current analysis and develop indicators\nfor major social phenomena in specific communities.",
            "journal ref": "PLoS One 8, E61981 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1201.2073v1",
            "date": "2012-01-10 15:18:38+00:00",
            "title": "Pbm: A new dataset for blog mining",
            "authors": [
                "Mehwish Aziz",
                "Muhammad Rafi"
            ],
            "abstract": "Text mining is becoming vital as Web 2.0 offers collaborative content\ncreation and sharing. Now Researchers have growing interest in text mining\nmethods for discovering knowledge. Text mining researchers come from variety of\nareas like: Natural Language Processing, Computational Linguistic, Machine\nLearning, and Statistics. A typical text mining application involves\npreprocessing of text, stemming and lemmatization, tagging and annotation,\nderiving knowledge patterns, evaluating and interpreting the results. There are\nnumerous approaches for performing text mining tasks, like: clustering,\ncategorization, sentimental analysis, and summarization. There is a growing\nneed to standardize the evaluation of these tasks. One major component of\nestablishing standardization is to provide standard datasets for these tasks.\nAlthough there are various standard datasets available for traditional text\nmining tasks, but there are very few and expensive datasets for blog-mining\ntask. Blogs, a new genre in web 2.0 is a digital diary of web user, which has\nchronological entries and contains a lot of useful knowledge, thus offers a lot\nof challenges and opportunities for text mining. In this paper, we report a new\nindigenous dataset for Pakistani Political Blogosphere. The paper describes the\nprocess of data collection, organization, and standardization. We have used\nthis dataset for carrying out various text mining tasks for blogosphere, like:\nblog-search, political sentiments analysis and tracking, identification of\ninfluential blogger, and clustering of the blog-posts. We wish to offer this\ndataset free for others who aspire to pursue further in this domain.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1201.5477v1",
            "date": "2012-01-26 10:29:01+00:00",
            "title": "Entropy-growth-based model of emotionally charged online dialogues",
            "authors": [
                "Julian Sienkiewicz",
                "Marcin Skowron",
                "Georgios Paltoglou",
                "Janusz A. Holyst"
            ],
            "abstract": "We analyze emotionally annotated massive data from IRC (Internet Relay Chat)\nand model the dialogues between its participants by assuming that the driving\nforce for the discussion is the entropy growth of emotional probability\ndistribution. This process is claimed to be correlated to the emergence of the\npower-law distribution of the discussion lengths observed in the dialogues. We\nperform numerical simulations based on the noticed phenomenon obtaining a good\nagreement with the real data. Finally, we propose a method to artificially\nprolong the duration of the discussion that relies on the entropy of emotional\nprobability distribution.",
            "journal ref": "Advances in Complex Systems 16, 1350026 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1201.5484v2",
            "date": "2012-01-26 11:02:19+00:00",
            "title": "Statistical analysis of emotions and opinions at Digg website",
            "authors": [
                "Piotr Pohorecki",
                "Julian Sienkiewicz",
                "Marija Mitrovic",
                "Georgios Paltoglou",
                "Janusz A. Holyst"
            ],
            "abstract": "We performed statistical analysis on data from the Digg.com website, which\nenables its users to express their opinion on news stories by taking part in\nforum-like discussions as well as directly evaluate previous posts and stories\nby assigning so called \"diggs\". Owing to fact that the content of each post has\nbeen annotated with its emotional value, apart from the strictly structural\nproperties, the study also includes an analysis of the average emotional\nresponse of the posts commenting the main story. While analysing correlations\nat the story level, an interesting relationship between the number of diggs and\nthe number of comments received by a story was found. The correlation between\nthe two quantities is high for data where small threads dominate and\nconsistently decreases for longer threads. However, while the correlation of\nthe number of diggs and the average emotional response tends to grow for longer\nthreads, correlations between numbers of comments and the average emotional\nresponse are almost zero. We also show that the initial set of comments given\nto a story has a substantial impact on the further \"life\" of the discussion:\nhigh negative average emotions in the first 10 comments lead to longer threads\nwhile the opposite situation results in shorter discussions. We also suggest\npresence of two different mechanisms governing the evolution of the discussion\nand, consequently, its length.",
            "journal ref": "Acta Physica Polonica A 123, 604 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1206.1069v2",
            "date": "2012-06-05 20:24:34+00:00",
            "title": "Concepts and Their Dynamics: A Quantum-Theoretic Modeling of Human Thought",
            "authors": [
                "Diederik Aerts",
                "Liane Gabora",
                "Sandro Sozzo"
            ],
            "abstract": "We analyze different aspects of our quantum modeling approach of human\nconcepts, and more specifically focus on the quantum effects of contextuality,\ninterference, entanglement and emergence, illustrating how each of them makes\nits appearance in specific situations of the dynamics of human concepts and\ntheir combinations. We point out the relation of our approach, which is based\non an ontology of a concept as an entity in a state changing under influence of\na context, with the main traditional concept theories, i.e. prototype theory,\nexemplar theory and theory theory. We ponder about the question why quantum\ntheory performs so well in its modeling of human concepts, and shed light on\nthis question by analyzing the role of complex amplitudes, showing how they\nallow to describe interference in the statistics of measurement outcomes, while\nin the traditional theories statistics of outcomes originates in classical\nprobability weights, without the possibility of interference. The relevance of\ncomplex numbers, the appearance of entanglement, and the role of Fock space in\nexplaining contextual emergence, all as unique features of the quantum\nmodeling, are explicitly revealed in this paper by analyzing human concepts and\ntheir dynamics.",
            "journal ref": "Topics in Cognitive Science, 5, pp. 737-772, 2013",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1206.3254v1",
            "date": "2012-06-13 15:30:14+00:00",
            "title": "Latent Topic Models for Hypertext",
            "authors": [
                "Amit Gruber",
                "Michal Rosen-Zvi",
                "Yair Weiss"
            ],
            "abstract": "Latent topic models have been successfully applied as an unsupervised topic\ndiscovery technique in large document collections. With the proliferation of\nhypertext document collection such as the Internet, there has also been great\ninterest in extending these approaches to hypertext [6, 9]. These approaches\ntypically model links in an analogous fashion to how they model words - the\ndocument-link co-occurrence matrix is modeled in the same way that the\ndocument-word co-occurrence matrix is modeled in standard topic models. In this\npaper we present a probabilistic generative model for hypertext document\ncollections that explicitly models the generation of links. Specifically, links\nfrom a word w to a document d depend directly on how frequent the topic of w is\nin d, in addition to the in-degree of d. We show how to perform EM learning on\nthis model efficiently. By not modeling links as analogous to words, we end up\nusing far fewer free parameters and obtain better link prediction results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1206.4637v1",
            "date": "2012-06-18 15:15:28+00:00",
            "title": "Learning to Identify Regular Expressions that Describe Email Campaigns",
            "authors": [
                "Paul Prasse",
                "Christoph Sawade",
                "Niels Landwehr",
                "Tobias Scheffer"
            ],
            "abstract": "This paper addresses the problem of inferring a regular expression from a\ngiven set of strings that resembles, as closely as possible, the regular\nexpression that a human expert would have written to identify the language.\nThis is motivated by our goal of automating the task of postmasters of an email\nservice who use regular expressions to describe and blacklist email spam\ncampaigns. Training data contains batches of messages and corresponding regular\nexpressions that an expert postmaster feels confident to blacklist. We model\nthis task as a learning problem with structured output spaces and an\nappropriate loss function, derive a decoder and the resulting optimization\nproblem, and a report on a case study conducted with an email service.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1206.4958v1",
            "date": "2012-06-21 18:17:50+00:00",
            "title": "A Pointillism Approach for Natural Language Processing of Social Media",
            "authors": [
                "Peiyou Song",
                "Anhei Shu",
                "Anyu Zhou",
                "Dan Wallach",
                "Jedidiah R. Crandall"
            ],
            "abstract": "The Chinese language poses challenges for natural language processing based\non the unit of a word even for formal uses of the Chinese language, social\nmedia only makes word segmentation in Chinese even more difficult. In this\ndocument we propose a pointillism approach to natural language processing.\nRather than words that have individual meanings, the basic unit of a\npointillism approach is trigrams of characters. These grams take on meaning in\naggregate when they appear together in a way that is correlated over time.\n  Our results from three kinds of experiments show that when words and topics\ndo have a meme-like trend, they can be reconstructed from only trigrams. For\nexample, for 4-character idioms that appear at least 99 times in one day in our\ndata, the unconstrained precision (that is, precision that allows for deviation\nfrom a lexicon when the result is just as correct as the lexicon version of the\nword or phrase) is 0.93. For longer words and phrases collected from\nWiktionary, including neologisms, the unconstrained precision is 0.87. We\nconsider these results to be very promising, because they suggest that it is\nfeasible for a machine to reconstruct complex idioms, phrases, and neologisms\nwith good precision without any notion of words. Thus the colorful and baroque\nuses of language that typify social media in challenging languages such as\nChinese may in fact be accessible to machines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1206.5851v1",
            "date": "2012-06-25 21:59:58+00:00",
            "title": "A meta-analysis of state-of-the-art electoral prediction from Twitter data",
            "authors": [
                "Daniel Gayo-Avello"
            ],
            "abstract": "Electoral prediction from Twitter data is an appealing research topic. It\nseems relatively straightforward and the prevailing view is overly optimistic.\nThis is problematic because while simple approaches are assumed to be good\nenough, core problems are not addressed. Thus, this paper aims to (1) provide a\nbalanced and critical review of the state of the art; (2) cast light on the\npresume predictive power of Twitter data; and (3) depict a roadmap to push\nforward the field. Hence, a scheme to characterize Twitter prediction methods\nis proposed. It covers every aspect from data collection to performance\nevaluation, through data processing and vote inference. Using that scheme,\nprior research is analyzed and organized to explain the main approaches taken\nup to date but also their weaknesses. This is the first meta-analysis of the\nwhole body of research regarding electoral prediction from Twitter data. It\nreveals that its presumed predictive power regarding electoral prediction has\nbeen rather exaggerated: although social media may provide a glimpse on\nelectoral outcomes current research does not provide strong evidence to support\nit can replace traditional polls. Finally, future lines of research along with\na set of requirements they must fulfill are provided.",
            "journal ref": "Social Science Computer Review, August 23, 2013, 0894439313493979",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1206.6423v1",
            "date": "2012-06-27 19:59:59+00:00",
            "title": "A Joint Model of Language and Perception for Grounded Attribute Learning",
            "authors": [
                "Cynthia Matuszek",
                "Nicholas FitzGerald",
                "Luke Zettlemoyer",
                "Liefeng Bo",
                "Dieter Fox"
            ],
            "abstract": "As robots become more ubiquitous and capable, it becomes ever more important\nto enable untrained users to easily interact with them. Recently, this has led\nto study of the language grounding problem, where the goal is to extract\nrepresentations of the meanings of natural language tied to perception and\nactuation in the physical world. In this paper, we present an approach for\njoint learning of language and perception models for grounded attribute\ninduction. Our perception model includes attribute classifiers, for example to\ndetect object color and shape, and the language model is based on a\nprobabilistic categorial grammar that enables the construction of rich,\ncompositional meaning representations. The approach is evaluated on the task of\ninterpreting sentences that describe sets of objects in a physical workspace.\nWe demonstrate accurate task performance and effective latent-variable concept\ninduction in physical grounded scenes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1206.6481v1",
            "date": "2012-06-27 19:59:59+00:00",
            "title": "Cross Language Text Classification via Subspace Co-Regularized Multi-View Learning",
            "authors": [
                "Yuhong Guo",
                "Min Xiao"
            ],
            "abstract": "In many multilingual text classification problems, the documents in different\nlanguages often share the same set of categories. To reduce the labeling cost\nof training a classification model for each individual language, it is\nimportant to transfer the label knowledge gained from one language to another\nlanguage by conducting cross language classification. In this paper we develop\na novel subspace co-regularized multi-view learning method for cross language\ntext classification. This method is built on parallel corpora produced by\nmachine translation. It jointly minimizes the training error of each classifier\nin each language while penalizing the distance between the subspace\nrepresentations of parallel documents. Our empirical study on a large set of\ncross language text classification tasks shows the proposed method consistently\noutperforms a number of inductive methods, domain adaptation methods, and\nmulti-view learning methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1202.3752v1",
            "date": "2012-02-14 16:41:17+00:00",
            "title": "Multidimensional counting grids: Inferring word order from disordered bags of words",
            "authors": [
                "Nebojsa Jojic",
                "Alessandro Perina"
            ],
            "abstract": "Models of bags of words typically assume topic mixing so that the words in a\nsingle bag come from a limited number of topics. We show here that many sets of\nbag of words exhibit a very different pattern of variation than the patterns\nthat are efficiently captured by topic mixing. In many cases, from one bag of\nwords to the next, the words disappear and new ones appear as if the theme\nslowly and smoothly shifted across documents (providing that the documents are\nsomehow ordered). Examples of latent structure that describe such ordering are\neasily imagined. For example, the advancement of the date of the news stories\nis reflected in a smooth change over the theme of the day as certain evolving\nnews stories fall out of favor and new events create new stories. Overlaps\namong the stories of consecutive days can be modeled by using windows over\nlinearly arranged tight distributions over words. We show here that such\nstrategy can be extended to multiple dimensions and cases where the ordering of\ndata is not readily obvious. We demonstrate that this way of modeling\ncovariation in word occurrences outperforms standard topic models in\nclassification and prediction tasks in applications in biology, text modeling\nand computer vision.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1202.5913v1",
            "date": "2012-02-27 12:44:23+00:00",
            "title": "Fly out-smarts man",
            "authors": [
                "Ruedi Stoop",
                "Patrick N\u00fcesch",
                "Ralph Lukas Stoop",
                "Leonid Bunimovich"
            ],
            "abstract": "Precopulatory courtship is a high-cost, non-well understood animal world\nmystery. Drosophila's (=D.'s) precopulatory courtship not only shows marked\nstructural similarities with mammalian courtship, but also with human spoken\nlanguage. This suggests the study of purpose, modalities and in particular of\nthe power of this language and to compare it to human language. Following a\nmathematical symbolic dynamics approach, we translate courtship videos of D.'s\nbody language into a formal language. This approach made it possible to show\nthat D. may use its body language to express individual information -\ninformation that may be important for evolutionary optimization, on top of the\nsexual group membership. Here, we use Chomsky's hierarchical language\nclassification to characterize the power of D.'s body language, and then\ncompare it with the power of languages spoken by humans. We find that from a\nformal language point of view, D.'s body language is at least as powerful as\nthe languages spoken by humans. From this we conclude that human intellect\ncannot be the direct consequence of the formal grammar complexity of human\nlanguage.",
            "journal ref": "None",
            "category": "q-bio.PE"
        },
        {
            "url": "http://arxiv.org/abs/1204.3458v1",
            "date": "2012-04-16 12:22:13+00:00",
            "title": "The logic of quantum mechanics - Take II",
            "authors": [
                "Bob Coecke"
            ],
            "abstract": "We put forward a new take on the logic of quantum mechanics, following\nSchroedinger's point of view that it is composition which makes quantum theory\nwhat it is, rather than its particular propositional structure due to the\nexistence of superpositions, as proposed by Birkhoff and von Neumann. This\ngives rise to an intrinsically quantitative kind of logic, which truly deserves\nthe name `logic' in that it also models meaning in natural language, the latter\nbeing the origin of logic, that it supports automation, the most prominent\npractical use of logic, and that it supports probabilistic inference.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1206.4631v3",
            "date": "2012-06-18 15:11:38+00:00",
            "title": "A Poisson convolution model for characterizing topical content with word frequency and exclusivity",
            "authors": [
                "Edoardo M Airoldi",
                "Jonathan M Bischof"
            ],
            "abstract": "An ongoing challenge in the analysis of document collections is how to\nsummarize content in terms of a set of inferred themes that can be interpreted\nsubstantively in terms of topics. The current practice of parametrizing the\nthemes in terms of most frequent words limits interpretability by ignoring the\ndifferential use of words across topics. We argue that words that are both\ncommon and exclusive to a theme are more effective at characterizing topical\ncontent. We consider a setting where professional editors have annotated\ndocuments to a collection of topic categories, organized into a tree, in which\nleaf-nodes correspond to the most specific topics. Each document is annotated\nto multiple categories, at different levels of the tree. We introduce a\nhierarchical Poisson convolution model to analyze annotated documents in this\nsetting. The model leverages the structure among categories defined by\nprofessional editors to infer a clear semantic description for each topic in\nterms of words that are both frequent and exclusive. We carry out a large\nrandomized experiment on Amazon Turk to demonstrate that topic summaries based\non the FREX score are more interpretable than currently established frequency\nbased summaries, and that the proposed model produces more efficient estimates\nof exclusivity than with currently models. We also develop a parallelized\nHamiltonian Monte Carlo sampler that allows the inference to scale to millions\nof documents.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1208.2873v1",
            "date": "2012-08-13 18:59:54+00:00",
            "title": "Detecting Events and Patterns in Large-Scale User Generated Textual Streams with Statistical Learning Methods",
            "authors": [
                "Vasileios Lampos"
            ],
            "abstract": "A vast amount of textual web streams is influenced by events or phenomena\nemerging in the real world. The social web forms an excellent modern paradigm,\nwhere unstructured user generated content is published on a regular basis and\nin most occasions is freely distributed. The present Ph.D. Thesis deals with\nthe problem of inferring information - or patterns in general - about events\nemerging in real life based on the contents of this textual stream. We show\nthat it is possible to extract valuable information about social phenomena,\nsuch as an epidemic or even rainfall rates, by automatic analysis of the\ncontent published in Social Media, and in particular Twitter, using Statistical\nMachine Learning methods. An important intermediate task regards the formation\nand identification of features which characterise a target event; we select and\nuse those textual features in several linear, non-linear and hybrid inference\napproaches achieving a significantly good performance in terms of the applied\nloss function. By examining further this rich data set, we also propose methods\nfor extracting various types of mood signals revealing how affective norms - at\nleast within the social web's population - evolve during the day and how\nsignificant events emerging in the real world are influencing them. Lastly, we\npresent some preliminary findings showing several spatiotemporal\ncharacteristics of this textual information as well as the potential of using\nit to tackle tasks such as the prediction of voting intentions.",
            "journal ref": "None",
            "category": "cs.LG"
        }
    ],
    "2013": [
        {
            "url": "http://arxiv.org/abs/1302.1123v1",
            "date": "2013-02-05 17:09:49+00:00",
            "title": "Large Scale Distributed Acoustic Modeling With Back-off N-grams",
            "authors": [
                "Ciprian Chelba",
                "Peng Xu",
                "Fernando Pereira",
                "Thomas Richardson"
            ],
            "abstract": "The paper revives an older approach to acoustic modeling that borrows from\nn-gram language modeling in an attempt to scale up both the amount of training\ndata and model size (as measured by the number of parameters in the model), to\napproximately 100 times larger than current sizes used in automatic speech\nrecognition. In such a data-rich setting, we can expand the phonetic context\nsignificantly beyond triphones, as well as increase the number of Gaussian\nmixture components for the context-dependent states that allow it. We have\nexperimented with contexts that span seven or more context-independent phones,\nand up to 620 mixture components per state. Dealing with unseen phonetic\ncontexts is accomplished using the familiar back-off technique used in language\nmodeling due to implementation simplicity. The back-off acoustic model is\nestimated, stored and served using MapReduce distributed computing\ninfrastructure.\n  Speech recognition experiments are carried out in an N-best list rescoring\nframework for Google Voice Search. Training big models on large amounts of data\nproves to be an effective way to increase the accuracy of a state-of-the-art\nautomatic speech recognition system. We use 87,000 hours of training data\n(speech along with transcription) obtained by filtering utterances in Voice\nSearch logs on automatic speech recognition confidence. Models ranging in size\nbetween 20--40 million Gaussians are estimated using maximum likelihood\ntraining. They achieve relative reductions in word-error-rate of 11% and 6%\nwhen combined with first-pass models trained using maximum likelihood, and\nboosted maximum mutual information, respectively. Increasing the context size\nbeyond five phones (quinphones) does not help.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.1380v1",
            "date": "2013-02-06 14:17:55+00:00",
            "title": "Towards the Rapid Development of a Natural Language Understanding Module",
            "authors": [
                "Catarina Moreira",
                "Ana Cristina Mendes",
                "Lu\u00edsa Coheur",
                "Bruno Martins"
            ],
            "abstract": "When developing a conversational agent, there is often an urgent need to have\na prototype available in order to test the application with real users. A\nWizard of Oz is a possibility, but sometimes the agent should be simply\ndeployed in the environment where it will be used. Here, the agent should be\nable to capture as many interactions as possible and to understand how people\nreact to failure. In this paper, we focus on the rapid development of a natural\nlanguage understanding module by non experts. Our approach follows the learning\nparadigm and sees the process of understanding natural language as a\nclassification problem. We test our module with a conversational agent that\nanswers questions in the art domain. Moreover, we show how our approach can be\nused by a natural language interface to a cinema database.",
            "journal ref": "In Proceedings of the 11th International Conference on Intelligent\n  Virtual Agents, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.1422v2",
            "date": "2013-02-06 16:26:49+00:00",
            "title": "S\u00e9mantique des d\u00e9terminants dans un cadre richement typ\u00e9",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "The variation of word meaning according to the context leads us to enrich the\ntype system of our syntactical and semantic analyser of French based on\ncategorial grammars and Montague semantics (or lambda-DRT). The main advantage\nof a deep semantic analyse is too represent meaning by logical formulae that\ncan be easily used e.g. for inferences. Determiners and quantifiers play a\nfundamental role in the construction of those formulae. But in our rich type\nsystem the usual semantic terms do not work. We propose a solution ins- pired\nby the tau and epsilon operators of Hilbert, kinds of generic elements and\nchoice functions. This approach unifies the treatment of the different determi-\nners and quantifiers as well as the dynamic binding of pronouns. Above all,\nthis fully computational view fits in well within the wide coverage parser\nGrail, both from a theoretical and a practical viewpoint.",
            "journal ref": "Traitement Automatique du Langage Naturel 2013, Les Sables\n  d'Olonnes : France (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.1572v1",
            "date": "2013-02-06 15:59:24+00:00",
            "title": "Lexical Access for Speech Understanding using Minimum Message Length Encoding",
            "authors": [
                "Ian Thomas",
                "Ingrid Zukerman",
                "Jonathan Oliver",
                "David Albrecht",
                "Bhavani Raskutti"
            ],
            "abstract": "The Lexical Access Problem consists of determining the intended sequence of\nwords corresponding to an input sequence of phonemes (basic speech sounds) that\ncome from a low-level phoneme recognizer. In this paper we present an\ninformation-theoretic approach based on the Minimum Message Length Criterion\nfor solving the Lexical Access Problem. We model sentences using phoneme\nrealizations seen in training, and word and part-of-speech information obtained\nfrom text corpora. We show results on multiple-speaker, continuous, read speech\nand discuss a heuristic using equivalence classes of similar sounding words\nwhich speeds up the recognition process without significant deterioration in\nrecognition accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.3057v1",
            "date": "2013-02-13 11:54:56+00:00",
            "title": "Building a reordering system using tree-to-string hierarchical model",
            "authors": [
                "Jacob Dlougach",
                "Irina Galinskaya"
            ],
            "abstract": "This paper describes our submission to the First Workshop on Reordering for\nStatistical Machine Translation. We have decided to build a reordering system\nbased on tree-to-string model, using only publicly available tools to\naccomplish this task. With the provided training data we have built a\ntranslation model using Moses toolkit, and then we applied a chart decoder,\nimplemented in Moses, to reorder the sentences. Even though our submission only\ncovered English-Farsi language pair, we believe that the approach itself should\nwork regardless of the choice of the languages, so we have also carried out the\nexperiments for English-Italian and English-Urdu. For these language pairs we\nhave noticed a significant improvement over the baseline in BLEU, Kendall-Tau\nand Hamming metrics. A detailed description is given, so that everyone can\nreproduce our results. Also, some possible directions for further improvements\nare discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4489v1",
            "date": "2013-02-19 00:30:57+00:00",
            "title": "Termhood-based Comparability Metrics of Comparable Corpus in Special Domain",
            "authors": [
                "Sa Liu",
                "Chengzhi Zhang"
            ],
            "abstract": "Cross-Language Information Retrieval (CLIR) and machine translation (MT)\nresources, such as dictionaries and parallel corpora, are scarce and hard to\ncome by for special domains. Besides, these resources are just limited to a few\nlanguages, such as English, French, and Spanish and so on. So, obtaining\ncomparable corpora automatically for such domains could be an answer to this\nproblem effectively. Comparable corpora, that the subcorpora are not\ntranslations of each other, can be easily obtained from web. Therefore,\nbuilding and using comparable corpora is often a more feasible option in\nmultilingual information processing. Comparability metrics is one of key issues\nin the field of building and using comparable corpus. Currently, there is no\nwidely accepted definition or metrics method of corpus comparability. In fact,\nDifferent definitions or metrics methods of comparability might be given to\nsuit various tasks about natural language processing. A new comparability,\nnamely, termhood-based metrics, oriented to the task of bilingual terminology\nextraction, is proposed in this paper. In this method, words are ranked by\ntermhood not frequency, and then the cosine similarities, calculated based on\nthe ranking lists of word termhood, is used as comparability. Experiments\nresults show that termhood-based metrics performs better than traditional\nfrequency-based metrics.",
            "journal ref": "Lecture Notes in Computer Science Volume 7717, 2013, pp 134-144",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4492v1",
            "date": "2013-02-19 00:37:21+00:00",
            "title": "Bilingual Terminology Extraction Using Multi-level Termhood",
            "authors": [
                "Chengzhi Zhang",
                "Dan Wu"
            ],
            "abstract": "Purpose: Terminology is the set of technical words or expressions used in\nspecific contexts, which denotes the core concept in a formal discipline and is\nusually applied in the fields of machine translation, information retrieval,\ninformation extraction and text categorization, etc. Bilingual terminology\nextraction plays an important role in the application of bilingual dictionary\ncompilation, bilingual Ontology construction, machine translation and\ncross-language information retrieval etc. This paper addresses the issues of\nmonolingual terminology extraction and bilingual term alignment based on\nmulti-level termhood.\n  Design/methodology/approach: A method based on multi-level termhood is\nproposed. The new method computes the termhood of the terminology candidate as\nwell as the sentence that includes the terminology by the comparison of the\ncorpus. Since terminologies and general words usually have differently\ndistribution in the corpus, termhood can also be used to constrain and enhance\nthe performance of term alignment when aligning bilingual terms on the parallel\ncorpus. In this paper, bilingual term alignment based on termhood constraints\nis presented.\n  Findings: Experiment results show multi-level termhood can get better\nperformance than existing method for terminology extraction. If termhood is\nused as constrain factor, the performance of bilingual term alignment can be\nimproved.",
            "journal ref": "Electronic Library, The, Vol. 30 Iss: 2, 2012, pp.295 - 309",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4811v1",
            "date": "2013-02-20 05:46:53+00:00",
            "title": "Towards a Semantic-based Approach for Modeling Regulatory Documents in Building Industry",
            "authors": [
                "Khalil Riad Bouzidi",
                "Catherine Faron-Zucker",
                "Bruno Fies",
                "Olivier Corby",
                "Le-Thanh Nhan"
            ],
            "abstract": "Regulations in the Building Industry are becoming increasingly complex and\ninvolve more than one technical area. They cover products, components and\nproject implementation. They also play an important role to ensure the quality\nof a building, and to minimize its environmental impact. In this paper, we are\nparticularly interested in the modeling of the regulatory constraints derived\nfrom the Technical Guides issued by CSTB and used to validate Technical\nAssessments. We first describe our approach for modeling regulatory constraints\nin the SBVR language, and formalizing them in the SPARQL language. Second, we\ndescribe how we model the processes of compliance checking described in the\nCSTB Technical Guides. Third, we show how we implement these processes to\nassist industrials in drafting Technical Documents in order to acquire a\nTechnical Assessment; a compliance report is automatically generated to explain\nthe compliance or noncompliance of this Technical Documents.",
            "journal ref": "9th European Conference on Product \\& Process Modelling, ECPPM\n  2012 (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4813v1",
            "date": "2013-02-20 05:47:32+00:00",
            "title": "Probabilistic Frame Induction",
            "authors": [
                "Jackie Chi Kit Cheung",
                "Hoifung Poon",
                "Lucy Vanderwende"
            ],
            "abstract": "In natural-language discourse, related events tend to appear near each other\nto describe a larger scenario. Such structures can be formalized by the notion\nof a frame (a.k.a. template), which comprises a set of related events and\nprototypical participants and event transitions. Identifying frames is a\nprerequisite for information extraction and natural language generation, and is\nusually done manually. Methods for inducing frames have been proposed recently,\nbut they typically use ad hoc procedures and are difficult to diagnose or\nextend. In this paper, we propose the first probabilistic approach to frame\ninduction, which incorporates frames, events, participants as latent topics and\nlearns those frame and event transitions that best explain the text. The number\nof frames is inferred by a novel application of a split-merge method from\nsyntactic parsing. In end-to-end evaluations from text to induced frames and\nextracted facts, our method produced state-of-the-art results while\nsubstantially reducing engineering effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4814v1",
            "date": "2013-02-20 05:47:44+00:00",
            "title": "NLP and CALL: integration is working",
            "authors": [
                "Georges Antoniadis",
                "Sylviane Granger",
                "Olivier Kraif",
                "Claude Ponton",
                "Virginie Zampa"
            ],
            "abstract": "In the first part of this article, we explore the background of\ncomputer-assisted learning from its beginnings in the early XIXth century and\nthe first teaching machines, founded on theories of learning, at the start of\nthe XXth century. With the arrival of the computer, it became possible to offer\nlanguage learners different types of language activities such as comprehension\ntasks, simulations, etc. However, these have limits that cannot be overcome\nwithout some contribution from the field of natural language processing (NLP).\nIn what follows, we examine the challenges faced and the issues raised by\nintegrating NLP into CALL. We hope to demonstrate that the key to success in\nintegrating NLP into CALL is to be found in multidisciplinary work between\ncomputer experts, linguists, language teachers, didacticians and NLP\nspecialists.",
            "journal ref": "TaLC7, France (2006)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.5645v1",
            "date": "2013-02-18 15:28:51+00:00",
            "title": "Role of temporal inference in the recognition of textual inference",
            "authors": [
                "Djallel Bouneffouf"
            ],
            "abstract": "This project is a part of nature language processing and its aims to develop\na system of recognition inference text-appointed TIMINF. This type of system\ncan detect, given two portions of text, if a text is semantically deducted from\nthe other. We focused on making the inference time in this type of system. For\nthat we have built and analyzed a body built from questions collected through\nthe web. This study has enabled us to classify different types of times\ninferences and for designing the architecture of TIMINF which seeks to\nintegrate a module inference time in a detection system inference text. We also\nassess the performance of sorties TIMINF system on a test corpus with the same\nstrategy adopted in the challenge RTE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.6777v1",
            "date": "2013-02-27 14:13:10+00:00",
            "title": "Ending-based Strategies for Part-of-speech Tagging",
            "authors": [
                "Greg Adams",
                "Beth Millar",
                "Eric Neufeld",
                "Tim Philip"
            ],
            "abstract": "Probabilistic approaches to part-of-speech tagging rely primarily on\nwhole-word statistics about word/tag combinations as well as contextual\ninformation. But experience shows about 4 per cent of tokens encountered in\ntest sets are unknown even when the training set is as large as a million\nwords. Unseen words are tagged using secondary strategies that exploit word\nfeatures such as endings, capitalizations and punctuation marks. In this work,\nword-ending statistics are primary and whole-word statistics are secondary.\nFirst, a tagger was trained and tested on word endings only. Subsequent\nexperiments added back whole-word statistics for the words occurring most\nfrequently in the training set. As grew larger, performance was expected to\nimprove, in the limit performing the same as word-based taggers. Surprisingly,\nthe ending-based tagger initially performed nearly as well as the word-based\ntagger; in the best case, its performance significantly exceeded that of the\nword-based tagger. Lastly, and unexpectedly, an effect of negative returns was\nobserved - as grew larger, performance generally improved and then declined. By\nvarying factors such as ending length and tag-list strategy, we achieved a\nsuccess rate of 97.5 percent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.1004v3",
            "date": "2013-08-05 15:14:14+00:00",
            "title": "Boundary identification of events in clinical named entity recognition",
            "authors": [
                "Azad Dehghan"
            ],
            "abstract": "The problem of named entity recognition in the medical/clinical domain has\ngained increasing attention do to its vital role in a wide range of clinical\ndecision support applications. The identification of complete and correct term\nspan is vital for further knowledge synthesis (e.g., coding/mapping concepts\nthesauruses and classification standards). This paper investigates boundary\nadjustment by sequence labeling representations models and post-processing\ntechniques in the problem of clinical named entity recognition (recognition of\nclinical events). Using current state-of-the-art sequence labeling algorithm\n(conditional random fields), we show experimentally that sequence labeling\nrepresentation and post-processing can be significantly helpful in strict\nboundary identification of clinical events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.1507v1",
            "date": "2013-08-07 09:09:47+00:00",
            "title": "Logical analysis of natural language semantics to solve the problem of computer understanding",
            "authors": [
                "Yuriy Ostapov"
            ],
            "abstract": "An object--oriented approach to create a natural language understanding\nsystem is considered. The understanding program is a formal system built on the\nbase of predicative calculus. Horn's clauses are used as well--formed formulas.\nAn inference is based on the principle of resolution. Sentences of natural\nlanguage are represented in the view of typical predicate set. These predicates\ndescribe physical objects and processes, abstract objects, categories and\nsemantic relations between objects. Predicates for concrete assertions are\nsaved in a database. To describe the semantics of classes for physical objects,\nabstract concepts and processes, a knowledge base is applied. The proposed\nrepresentation of natural language sentences is a semantic net. Nodes of such\nnet are typical predicates. This approach is perspective as, firstly, such\ntypification of nodes facilitates essentially forming of processing algorithms\nand object descriptions, secondly, the effectiveness of algorithms is increased\n(particularly for the great number of nodes), thirdly, to describe the\nsemantics of words, encyclopedic knowledge is used, and this permits\nessentially to extend the class of solved problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.2428v2",
            "date": "2013-08-11 20:50:27+00:00",
            "title": "Hidden Structure and Function in the Lexicon",
            "authors": [
                "Olivier Picard",
                "M\u00e9lanie Lord",
                "Alexandre Blondin-Mass\u00e9",
                "Odile Marcotte",
                "Marcos Lopes",
                "Stevan Harnad"
            ],
            "abstract": "How many words are needed to define all the words in a dictionary?\nGraph-theoretic analysis reveals that about 10% of a dictionary is a unique\nKernel of words that define one another and all the rest, but this is not the\nsmallest such subset. The Kernel consists of one huge strongly connected\ncomponent (SCC), about half its size, the Core, surrounded by many small SCCs,\nthe Satellites. Core words can define one another but not the rest of the\ndictionary. The Kernel also contains many overlapping Minimal Grounding Sets\n(MGSs), each about the same size as the Core, each part-Core, part-Satellite.\nMGS words can define all the rest of the dictionary. They are learned earlier,\nmore concrete and more frequent than the rest of the dictionary. Satellite\nwords, not correlated with age or frequency, are less concrete (more abstract)\nwords that are also needed for full lexical power.",
            "journal ref": "NLPCS 2013: 10th International Workshop on Natural Language\n  Processing and Cognitive Science, Marseille, France 15-16 October 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.2696v1",
            "date": "2013-08-12 20:57:02+00:00",
            "title": "B(eo)W(u)LF: Facilitating recurrence analysis on multi-level language",
            "authors": [
                "A. Paxton",
                "R. Dale"
            ],
            "abstract": "Discourse analysis may seek to characterize not only the overall composition\nof a given text but also the dynamic patterns within the data. This technical\nreport introduces a data format intended to facilitate multi-level\ninvestigations, which we call the by-word long-form or B(eo)W(u)LF. Inspired by\nthe long-form data format required for mixed-effects modeling, B(eo)W(u)LF\nstructures linguistic data into an expanded matrix encoding any number of\nresearchers-specified markers, making it ideal for recurrence-based analyses.\nWhile we do not necessarily claim to be the first to use methods along these\nlines, we have created a series of tools utilizing Python and MATLAB to enable\nsuch discourse analyses and demonstrate them using 319 lines of the Old English\nepic poem, Beowulf, translated into modern English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.3839v2",
            "date": "2013-08-18 07:09:03+00:00",
            "title": "Consensus Sequence Segmentation",
            "authors": [
                "Tamal Chowdhury",
                "Rabindra Rakshit",
                "Arko Banerjee"
            ],
            "abstract": "In this paper we introduce a method to detect words or phrases in a given\nsequence of alphabets without knowing the lexicon. Our linear time unsupervised\nalgorithm relies entirely on statistical relationships among alphabets in the\ninput sequence to detect location of word boundaries. We compare our algorithm\nto previous approaches from unsupervised sequence segmentation literature and\nprovide superior segmentation over number of benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.4479v1",
            "date": "2013-08-21 03:44:04+00:00",
            "title": "An Investigation of the Sampling-Based Alignment Method and Its Contributions",
            "authors": [
                "Juan Luo",
                "Yves Lepage"
            ],
            "abstract": "By investigating the distribution of phrase pairs in phrase translation\ntables, the work in this paper describes an approach to increase the number of\nn-gram alignments in phrase translation tables output by a sampling-based\nalignment method. This approach consists in enforcing the alignment of n-grams\nin distinct translation subtables so as to increase the number of n-grams.\nStandard normal distribution is used to allot alignment time among translation\nsubtables, which results in adjustment of the distribution of n- grams. This\nleads to better evaluation results on statistical machine translation tasks\nthan the original sampling-based alignment approach. Furthermore, the\ntranslation quality obtained by merging phrase translation tables computed from\nthe sampling-based alignment method and from MGIZA++ is examined.",
            "journal ref": "International Journal of Artificial Intelligence & Applications\n  (IJAIA), Vol. 4, No. 4, July 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.5423v1",
            "date": "2013-08-25 16:41:06+00:00",
            "title": "A Literature Review: Stemming Algorithms for Indian Languages",
            "authors": [
                "M. Thangarasu",
                "R. Manavalan"
            ],
            "abstract": "Stemming is the process of extracting root word from the given inflection\nword. It also plays significant role in numerous application of Natural\nLanguage Processing (NLP). The stemming problem has addressed in many contexts\nand by researchers in many disciplines. This expository paper presents survey\nof some of the latest developments on stemming algorithms in data mining and\nalso presents with some of the solutions for various Indian language stemming\nalgorithms along with the results.",
            "journal ref": "International Journal of Computer Trends and Technology, Vol 4, No\n  8, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.5499v1",
            "date": "2013-08-26 07:18:17+00:00",
            "title": "Linear models and linear mixed effects models in R with linguistic applications",
            "authors": [
                "Bodo Winter"
            ],
            "abstract": "This text is a conceptual introduction to mixed effects modeling with\nlinguistic applications, using the R programming environment. The reader is\nintroduced to linear modeling and assumptions, as well as to mixed\neffects/multilevel modeling, including a discussion of random intercepts,\nrandom slopes and likelihood ratio tests. The example used throughout the text\nfocuses on the phonetic analysis of voice pitch data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.6242v1",
            "date": "2013-08-28 18:23:03+00:00",
            "title": "NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets",
            "authors": [
                "Saif M. Mohammad",
                "Svetlana Kiritchenko",
                "Xiaodan Zhu"
            ],
            "abstract": "In this paper, we describe how we created two state-of-the-art SVM\nclassifiers, one to detect the sentiment of messages such as tweets and SMS\n(message-level task) and one to detect the sentiment of a term within a\nsubmissions stood first in both tasks on tweets, obtaining an F-score of 69.02\nin the message-level task and 88.93 in the term-level task. We implemented a\nvariety of surface-form, semantic, and sentiment features. with sentiment-word\nhashtags, and one from tweets with emoticons. In the message-level task, the\nlexicon-based features provided a gain of 5 F-score points over all others.\nBoth of our systems can be replicated us available resources.",
            "journal ref": "In Proceedings of the seventh international workshop on Semantic\n  Evaluation Exercises (SemEval-2013), June 2013, Atlanta, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.6297v1",
            "date": "2013-08-28 20:13:32+00:00",
            "title": "Crowdsourcing a Word-Emotion Association Lexicon",
            "authors": [
                "Saif M. Mohammad",
                "Peter D. Turney"
            ],
            "abstract": "Even though considerable attention has been given to the polarity of words\n(positive and negative) and the creation of large polarity lexicons, research\nin emotion analysis has had to rely on limited and small emotion lexicons. In\nthis paper we show how the combined strength and wisdom of the crowds can be\nused to generate a large, high-quality, word-emotion and word-polarity\nassociation lexicon quickly and inexpensively. We enumerate the challenges in\nemotion annotation in a crowdsourcing scenario and propose solutions to address\nthem. Most notably, in addition to questions about emotions associated with\nterms, we show how the inclusion of a word choice question can discourage\nmalicious data entry, help identify instances where the annotator may not be\nfamiliar with the target term (allowing us to reject such annotations), and\nhelp obtain annotations at sense level (rather than at word level). We\nconducted experiments on how to formulate the emotion-annotation questions, and\nshow that asking if a term is associated with an emotion leads to markedly\nhigher inter-annotator agreement than that obtained by asking if a term evokes\nan emotion.",
            "journal ref": "Computational Intelligence, 29 (3), 436-465, Wiley Blackwell\n  Publishing Ltd, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.6300v1",
            "date": "2013-08-28 20:24:27+00:00",
            "title": "Computing Lexical Contrast",
            "authors": [
                "Saif M. Mohammad",
                "Bonnie J. Dorr",
                "Graeme Hirst",
                "Peter D. Turney"
            ],
            "abstract": "Knowing the degree of semantic contrast between words has widespread\napplication in natural language processing, including machine translation,\ninformation retrieval, and dialogue systems. Manually-created lexicons focus on\nopposites, such as {\\rm hot} and {\\rm cold}. Opposites are of many kinds such\nas antipodals, complementaries, and gradable. However, existing lexicons often\ndo not classify opposites into the different kinds. They also do not explicitly\nlist word pairs that are not opposites but yet have some degree of contrast in\nmeaning, such as {\\rm warm} and {\\rm cold} or {\\rm tropical} and {\\rm\nfreezing}. We propose an automatic method to identify contrasting word pairs\nthat is based on the hypothesis that if a pair of words, $A$ and $B$, are\ncontrasting, then there is a pair of opposites, $C$ and $D$, such that $A$ and\n$C$ are strongly related and $B$ and $D$ are strongly related. (For example,\nthere exists the pair of opposites {\\rm hot} and {\\rm cold} such that {\\rm\ntropical} is related to {\\rm hot,} and {\\rm freezing} is related to {\\rm\ncold}.) We will call this the contrast hypothesis. We begin with a large\ncrowdsourcing experiment to determine the amount of human agreement on the\nconcept of oppositeness and its different kinds. In the process, we flesh out\nkey features of different kinds of opposites. We then present an automatic and\nempirical measure of lexical contrast that relies on the contrast hypothesis,\ncorpus statistics, and the structure of a {\\it Roget}-like thesaurus. We show\nthat the proposed measure of lexical contrast obtains high precision and large\ncoverage, outperforming existing methods.",
            "journal ref": "Computational Linguistics, 39 (3), 555-590, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.1014v1",
            "date": "2013-09-04 12:56:37+00:00",
            "title": "Advances in the Logical Representation of Lexical Semantics",
            "authors": [
                "Bruno Mery",
                "Christian Retor\u00e9"
            ],
            "abstract": "The integration of lexical semantics and pragmatics in the analysis of the\nmeaning of natural lan- guage has prompted changes to the global framework\nderived from Montague. In those works, the original lexicon, in which words\nwere assigned an atomic type of a single-sorted logic, has been re- placed by a\nset of many-facetted lexical items that can compose their meaning with salient\ncontextual properties using a rich typing system as a guide. Having related our\nproposal for such an expanded framework \\LambdaTYn, we present some recent\nadvances in the logical formalisms associated, including constraints on lexical\ntransformations and polymorphic quantifiers, and ongoing discussions and\nresearch on the granularity of the type system and the limits of transitivity.",
            "journal ref": "NLCS'13 - Natural Language and Computer Science - 2013 (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.1125v1",
            "date": "2013-09-04 18:10:22+00:00",
            "title": "Learning to answer questions",
            "authors": [
                "Ana Cristina Mendes",
                "Lu\u00edsa Coheur",
                "S\u00e9rgio Curto"
            ],
            "abstract": "We present an open-domain Question-Answering system that learns to answer\nquestions based on successful past interactions. We follow a pattern-based\napproach to Answer-Extraction, where (lexico-syntactic) patterns that relate a\nquestion to its answer are automatically learned and used to answer future\nquestions. Results show that our approach contributes to the system's best\nperformance when it is conjugated with typical Answer-Extraction strategies.\nMoreover, it allows the system to learn with the answered questions and to\nrectify wrong or unsolved past questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.1129v1",
            "date": "2013-09-04 18:23:30+00:00",
            "title": "Analysing Quality of English-Hindi Machine Translation Engine Outputs Using Bayesian Classification",
            "authors": [
                "Rashmi Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "This paper considers the problem for estimating the quality of machine\ntranslation outputs which are independent of human intervention and are\ngenerally addressed using machine learning techniques.There are various\nmeasures through which a machine learns translations quality. Automatic\nEvaluation metrics produce good co-relation at corpus level but cannot produce\nthe same results at the same segment or sentence level. In this paper 16\nfeatures are extracted from the input sentences and their translations and a\nquality score is obtained based on Bayesian inference produced from training\ndata.",
            "journal ref": "International Journal of Artificial Intelligence & Applications\n  (IJAIA), Vol. 4, No. 4, July 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.1649v2",
            "date": "2013-09-06 14:28:02+00:00",
            "title": "Preparing Korean Data for the Shared Task on Parsing Morphologically Rich Languages",
            "authors": [
                "Jinho D. Choi"
            ],
            "abstract": "This document gives a brief description of Korean data prepared for the SPMRL\n2013 shared task. A total of 27,363 sentences with 350,090 tokens are used for\nthe shared task. All constituent trees are collected from the KAIST Treebank\nand transformed to the Penn Treebank style. All dependency trees are converted\nfrom the transformed constituent trees using heuristics and labeling rules de-\nsigned specifically for the KAIST Treebank. In addition to the gold-standard\nmorphological analysis provided by the KAIST Treebank, two sets of automatic\nmorphological analysis are provided for the shared task, one is generated by\nthe HanNanum morphological analyzer, and the other is generated by the Sejong\nmorphological analyzer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.2471v1",
            "date": "2013-09-10 12:03:32+00:00",
            "title": "Implementation of nlization framework for verbs, pronouns and determiners with eugene",
            "authors": [
                "Harinder Singh",
                "Parteek Kumar"
            ],
            "abstract": "UNL system is designed and implemented by a nonprofit organization, UNDL\nFoundation at Geneva in 1999. UNL applications are application softwares that\nallow end users to accomplish natural language tasks, such as translating,\nsummarizing, retrieving or extracting information, etc. Two major web based\napplication softwares are Interactive ANalyzer (IAN), which is a natural\nlanguage analysis system. It represents natural language sentences as semantic\nnetworks in the UNL format. Other application software is dEep-to-sUrface\nGENErator (EUGENE), which is an open-source interactive NLizer. It generates\nnatural language sentences out of semantic networks represented in the UNL\nformat. In this paper, NLization framework with EUGENE is focused, while using\nUNL system for accomplishing the task of machine translation. In whole\nNLization process, EUGENE takes a UNL input and delivers an output in natural\nlanguage without any human intervention. It is language-independent and has to\nbe parametrized to the natural language input through a dictionary and a\ngrammar, provided as separate interpretable files. In this paper, it is\nexplained that how UNL input is syntactically and semantically analyzed with\nthe UNL-NL T-Grammar for NLization of UNL sentences involving verbs, pronouns\nand determiners for Punjabi natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.2853v1",
            "date": "2013-09-11 15:16:26+00:00",
            "title": "General Purpose Textual Sentiment Analysis and Emotion Detection Tools",
            "authors": [
                "Alexandre Denis",
                "Samuel Cruz-Lara",
                "Nadia Bellalem"
            ],
            "abstract": "Textual sentiment analysis and emotion detection consists in retrieving the\nsentiment or emotion carried by a text or document. This task can be useful in\nmany domains: opinion mining, prediction, feedbacks, etc. However, building a\ngeneral purpose tool for doing sentiment analysis and emotion detection raises\na number of issues, theoretical issues like the dependence to the domain or to\nthe language but also pratical issues like the emotion representation for\ninteroperability. In this paper we present our sentiment/emotion analysis\ntools, the way we propose to circumvent the di culties and the applications\nthey are used for.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.4168v1",
            "date": "2013-09-17 03:23:13+00:00",
            "title": "Exploiting Similarities among Languages for Machine Translation",
            "authors": [
                "Tomas Mikolov",
                "Quoc V. Le",
                "Ilya Sutskever"
            ],
            "abstract": "Dictionaries and phrase tables are the basis of modern statistical machine\ntranslation systems. This paper develops a method that can automate the process\nof generating and extending dictionaries and phrase tables. Our method can\ntranslate missing word and phrase entries by learning language structures based\non large monolingual data and mapping between languages from small bilingual\ndata. It uses distributed representation of words and learns a linear mapping\nbetween vector spaces of languages. Despite its simplicity, our method is\nsurprisingly effective: we can achieve almost 90% precision@5 for translation\nof words between English and Spanish. This method makes little assumption about\nthe languages, so it can be used to extend and refine dictionaries and\ntranslation tables for any language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.4628v1",
            "date": "2013-09-18 12:38:34+00:00",
            "title": "Text segmentation with character-level text embeddings",
            "authors": [
                "Grzegorz Chrupa\u0142a"
            ],
            "abstract": "Learning word representations has recently seen much success in computational\nlinguistics. However, assuming sequences of word tokens as input to linguistic\nanalysis is often unjustified. For many languages word segmentation is a\nnon-trivial task and naturally occurring text is sometimes a mixture of natural\nlanguage strings and other character data. We propose to learn text\nrepresentations directly from raw character sequences by training a Simple\nrecurrent Network to predict the next character in text. The network uses its\nhidden layer to evolve abstract representations of the character sequences it\nsees. To demonstrate the usefulness of the learned text embeddings, we use them\nas features in a supervised character level text segmentation and labeling\ntask: recognizing spans of text containing programming language code. By using\nthe embeddings as features we are able to substantially improve over a baseline\nwhich uses only surface character n-grams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5223v1",
            "date": "2013-09-20 09:51:59+00:00",
            "title": "JRC EuroVoc Indexer JEX - A freely available multi-label categorisation tool",
            "authors": [
                "Ralf Steinberger",
                "Mohamed Ebrahim",
                "Marco Turchi"
            ],
            "abstract": "EuroVoc (2012) is a highly multilingual thesaurus consisting of over 6,700\nhierarchically organised subject domains used by European Institutions and many\nauthorities in Member States of the European Union (EU) for the classification\nand retrieval of official documents. JEX is JRC-developed multi-label\nclassification software that learns from manually labelled data to\nautomatically assign EuroVoc descriptors to new documents in a profile-based\ncategory-ranking task. The JEX release consists of trained classifiers for 22\nofficial EU languages, of parallel training data in the same languages, of an\ninterface that allows viewing and amending the assignment results, and of a\nmodule that allows users to re-train the tool on their own document\ncollections. JEX allows advanced users to change the document representation so\nas to possibly improve the categorisation result through linguistic\npre-processing. JEX can be used as a tool for interactive EuroVoc descriptor\nassignment to increase speed and consistency of the human categorisation\nprocess, or it can be used fully automatically. The output of JEX is a\nlanguage-independent EuroVoc feature vector lending itself also as input to\nvarious other Language Technology tasks, including cross-lingual clustering and\nclassification, cross-lingual plagiarism detection, sentence selection and\nranking, and more.",
            "journal ref": "Proceedings of the 8th international conference on Language\n  Resources and Evaluation (LREC'2012), pp. 798-805, Istanbul, 21-27 May 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5226v1",
            "date": "2013-09-20 10:02:58+00:00",
            "title": "DGT-TM: A freely Available Translation Memory in 22 Languages",
            "authors": [
                "Ralf Steinberger",
                "Andreas Eisele",
                "Szymon Klocek",
                "Spyridon Pilos",
                "Patrick Schl\u00fcter"
            ],
            "abstract": "The European Commission's (EC) Directorate General for Translation, together\nwith the EC's Joint Research Centre, is making available a large translation\nmemory (TM; i.e. sentences and their professionally produced translations)\ncovering twenty-two official European Union (EU) languages and their 231\nlanguage pairs. Such a resource is typically used by translation professionals\nin combination with TM software to improve speed and consistency of their\ntranslations. However, this resource has also many uses for translation studies\nand for language technology applications, including Statistical Machine\nTranslation (SMT), terminology extraction, Named Entity Recognition (NER),\nmultilingual classification and clustering, and many more. In this reference\npaper for DGT-TM, we introduce this new resource, provide statistics regarding\nits size, and explain how it was produced and how to use it.",
            "journal ref": "Proceedings of the 8th international conference on Language\n  Resources and Evaluation (LREC'2012), pp. 454-459, Istanbul, 21-27 May 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5290v1",
            "date": "2013-09-20 15:03:58+00:00",
            "title": "An introduction to the Europe Media Monitor family of applications",
            "authors": [
                "Ralf Steinberger",
                "Bruno Pouliquen",
                "Erik van der Goot"
            ],
            "abstract": "Most large organizations have dedicated departments that monitor the media to\nkeep up-to-date with relevant developments and to keep an eye on how they are\nrepresented in the news. Part of this media monitoring work can be automated.\nIn the European Union with its 23 official languages, it is particularly\nimportant to cover media reports in many languages in order to capture the\ncomplementary news content published in the different countries. It is also\nimportant to be able to access the news content across languages and to merge\nthe extracted information. We present here the four publicly accessible systems\nof the Europe Media Monitor (EMM) family of applications, which cover between\n19 and 50 languages (see http://press.jrc.it/overview.html). We give an\noverview of their functionality and discuss some of the implications of the\nfact that they cover quite so many languages. We discuss design issues\nnecessary to be able to achieve this high multilinguality, as well as the\nbenefits of this multilinguality.",
            "journal ref": "In: Fredric Gey, Noriko Kando & Jussi Karlgren (eds.): Information\n  Access in a Multilingual World - Proceedings of the SIGIR 2009 Workshop\n  (SIGIR-CLIR'2009), pp. 1-8. Boston, USA. 23 July 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5391v1",
            "date": "2013-09-20 21:06:46+00:00",
            "title": "Even the Abstract have Colour: Consensus in Word-Colour Associations",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "Colour is a key component in the successful dissemination of information.\nSince many real-world concepts are associated with colour, for example danger\nwith red, linguistic information is often complemented with the use of\nappropriate colours in information visualization and product marketing. Yet,\nthere is no comprehensive resource that captures concept-colour associations.\nWe present a method to create a large word-colour association lexicon by\ncrowdsourcing. A word-choice question was used to obtain sense-level\nannotations and to ensure data quality. We focus especially on abstract\nconcepts and emotions to show that even they tend to have strong colour\nassociations. Thus, using the right colours can not only improve semantic\ncoherence, but also inspire the desired emotional response.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5652v1",
            "date": "2013-09-22 21:09:07+00:00",
            "title": "LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual",
            "authors": [
                "Mona Diab",
                "Nizar Habash",
                "Owen Rambow",
                "Ryan Roth"
            ],
            "abstract": "The Linguistic Data Consortium (LDC) has developed hundreds of data corpora\nfor natural language processing (NLP) research. Among these are a number of\nannotated treebank corpora for Arabic. Typically, these corpora consist of a\nsingle collection of annotated documents. NLP research, however, usually\nrequires multiple data sets for the purposes of training models, developing\ntechniques, and final evaluation. Therefore it becomes necessary to divide the\ncorpora used into the required data sets (divisions). This document details a\nset of rules that have been defined to enable consistent divisions for old and\nnew Arabic treebanks (ATB) and related corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5657v1",
            "date": "2013-09-22 22:06:26+00:00",
            "title": "A Hybrid Algorithm for Matching Arabic Names",
            "authors": [
                "T. El-Shishtawy"
            ],
            "abstract": "In this paper, a new hybrid algorithm which combines both of token-based and\ncharacter-based approaches is presented. The basic Levenshtein approach has\nbeen extended to token-based distance metric. The distance metric is enhanced\nto set the proper granularity level behavior of the algorithm. It smoothly maps\na threshold of misspellings differences at the character level, and the\nimportance of token level errors in terms of token's position and frequency.\nUsing a large Arabic dataset, the experimental results show that the proposed\nalgorithm overcomes successfully many types of errors such as: typographical\nerrors, omission or insertion of middle name components, omission of\nnon-significant popular name components, and different writing styles character\nvariations. When compared the results with other classical algorithms, using\nthe same dataset, the proposed algorithm was found to increase the minimum\nsuccess level of best tested algorithms, while achieving higher upper limits .",
            "journal ref": "International Journal of Computational Linguistics Research Volume\n  4 Number 2 June 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5843v1",
            "date": "2013-09-23 15:26:09+00:00",
            "title": "Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet",
            "authors": [
                "Marco Guerini",
                "Lorenzo Gatti",
                "Marco Turchi"
            ],
            "abstract": "Assigning a positive or negative score to a word out of context (i.e. a\nword's prior polarity) is a challenging task for sentiment analysis. In the\nliterature, various approaches based on SentiWordNet have been proposed. In\nthis paper, we compare the most often used techniques together with newly\nproposed ones and incorporate all of them in a learning framework to see\nwhether blending them can further improve the estimation of prior polarity\nscores. Using two different versions of SentiWordNet and testing regression and\nclassification models across tasks and datasets, our learning approach\nconsistently outperforms the single metrics, providing a new state-of-the-art\napproach in computing words' prior polarity for sentiment analysis. We conclude\nour investigation showing interesting biases in calculated prior polarity\nscores when word Part of Speech and annotator gender are considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5909v1",
            "date": "2013-09-23 18:43:56+00:00",
            "title": "From Once Upon a Time to Happily Ever After: Tracking Emotions in Novels and Fairy Tales",
            "authors": [
                "Saif Mohammad"
            ],
            "abstract": "Today we have access to unprecedented amounts of literary texts. However,\nsearch still relies heavily on key words. In this paper, we show how sentiment\nanalysis can be used in tandem with effective visualizations to quantify and\ntrack emotions in both individual books and across very large collections. We\nintroduce the concept of emotion word density, and using the Brothers Grimm\nfairy tales as example, we show how collections of text can be organized for\nbetter search. Using the Google Books Corpus we show how to determine an\nentity's emotion associations from co-occurring words. Finally, we compare\nemotion words in fairy tales and novels, to show that fairy tales have a much\nwider range of emotion word densities than novels.",
            "journal ref": "In Proceedings of the ACL Workshop on Language Technology for\n  Cultural Heritage, Social Sciences, and Humanities (LaTeCH), 2011, Portland,\n  OR",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5942v1",
            "date": "2013-09-20 21:10:56+00:00",
            "title": "Colourful Language: Measuring Word-Colour Associations",
            "authors": [
                "Saif Mohammad"
            ],
            "abstract": "Since many real-world concepts are associated with colour, for example danger\nwith red, linguistic information is often complimented with the use of\nappropriate colours in information visualization and product marketing. Yet,\nthere is no comprehensive resource that captures concept-colour associations.\nWe present a method to create a large word-colour association lexicon by\ncrowdsourcing. We focus especially on abstract concepts and emotions to show\nthat even though they cannot be physically visualized, they too tend to have\nstrong colour associations. Finally, we show how word-colour associations\nmanifest themselves in language, and quantify usefulness of co-occurrence and\npolarity cues in automatically detecting colour associations.",
            "journal ref": "Colourful Language: Measuring Word-Colour Associations, Saif\n  Mohammad, In Proceedings of the ACL 2011 Workshop on Cognitive Modeling and\n  Computational Linguistics (CMCL), June 2011, Portland, OR",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6162v1",
            "date": "2013-09-24 14:09:53+00:00",
            "title": "JRC-Names: A freely available, highly multilingual named entity resource",
            "authors": [
                "Ralf Steinberger",
                "Bruno Pouliquen",
                "Mijail Kabadjov",
                "Erik van der Goot"
            ],
            "abstract": "This paper describes a new, freely available, highly multilingual named\nentity resource for person and organisation names that has been compiled over\nseven years of large-scale multilingual news analysis combined with Wikipedia\nmining, resulting in 205,000 per-son and organisation names plus about the same\nnumber of spelling variants written in over 20 different scripts and in many\nmore languages. This resource, produced as part of the Europe Media Monitor\nactivity (EMM, http://emm.newsbrief.eu/overview.html), can be used for a number\nof purposes. These include improving name search in databases or on the\ninternet, seeding machine learning systems to learn named entity recognition\nrules, improve machine translation results, and more. We describe here how this\nresource was created; we give statistics on its current size; we address the\nissue of morphological inflection; and we give details regarding its\nfunctionality. Updates to this resource will be made available daily.",
            "journal ref": "Proceedings of the 8th International Conference Recent Advances in\n  Natural Language Processing (RANLP'2011), pp. 104-110. Hissar, Bulgaria,\n  12-14 September 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6185v1",
            "date": "2013-09-24 14:41:33+00:00",
            "title": "Acronym recognition and processing in 22 languages",
            "authors": [
                "Maud Ehrmann",
                "Leonida della Rocca",
                "Ralf Steinberger",
                "Hristo Tanev"
            ],
            "abstract": "We are presenting work on recognising acronyms of the form Long-Form\n(Short-Form) such as \"International Monetary Fund (IMF)\" in millions of news\narticles in twenty-two languages, as part of our more general effort to\nrecognise entities and their variants in news text and to use them for the\nautomatic analysis of the news, including the linking of related news across\nlanguages. We show how the acronym recognition patterns, initially developed\nfor medical terms, needed to be adapted to the more general news domain and we\npresent evaluation results. We describe our effort to automatically merge the\nnumerous long-form variants referring to the same short-form, while keeping\nnon-related long-forms separate. Finally, we provide extensive statistics on\nthe frequency and the distribution of short-form/long-form pairs across\nlanguages.",
            "journal ref": "Proceedings of the 9th Conference 'Recent Advances in Natural\n  Language Processing' (RANLP), pp. 237-244. Hissar, Bulgaria, 7-13 September\n  2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6202v1",
            "date": "2013-09-24 15:11:43+00:00",
            "title": "Sentiment Analysis in the News",
            "authors": [
                "Alexandra Balahur",
                "Ralf Steinberger",
                "Mijail Kabadjov",
                "Vanni Zavarella",
                "Erik van der Goot",
                "Matina Halkia",
                "Bruno Pouliquen",
                "Jenya Belyaeva"
            ],
            "abstract": "Recent years have brought a significant growth in the volume of research in\nsentiment analysis, mostly on highly subjective text types (movie or product\nreviews). The main difference these texts have with news articles is that their\ntarget is clearly defined and unique across the text. Following different\nannotation efforts and the analysis of the issues encountered, we realised that\nnews opinion mining is different from that of other text types. We identified\nthree subtasks that need to be addressed: definition of the target; separation\nof the good and bad news content from the good and bad sentiment expressed on\nthe target; and analysis of clearly marked opinion that is expressed\nexplicitly, not needing interpretation or the use of world knowledge.\nFurthermore, we distinguish three different possible views on newspaper\narticles - author, reader and text, which have to be addressed differently at\nthe time of analysing sentiment. Given these definitions, we present work on\nmining opinions about entities in English language news, in which (a) we test\nthe relative suitability of various sentiment dictionaries and (b) we attempt\nto separate positive or negative opinion from good or bad news. In the\nexperiments described here, we tested whether or not subject domain-defining\nvocabulary should be ignored. Results showed that this idea is more appropriate\nin the context of news opinion mining and that the approaches taking this into\nconsideration produce a better performance.",
            "journal ref": "Proceedings of the 7th International Conference on Language\n  Resources and Evaluation (LREC'2010), pp. 2216-2220. Valletta, Malta, 19-21\n  May 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6347v1",
            "date": "2013-09-24 21:14:45+00:00",
            "title": "Tracking Sentiment in Mail: How Genders Differ on Emotional Axes",
            "authors": [
                "Saif M. Mohammad",
                "Tony",
                "Yang"
            ],
            "abstract": "With the widespread use of email, we now have access to unprecedented amounts\nof text that we ourselves have written. In this paper, we show how sentiment\nanalysis can be used in tandem with effective visualizations to quantify and\ntrack emotions in many types of mail. We create a large word--emotion\nassociation lexicon by crowdsourcing, and use it to compare emotions in love\nletters, hate mail, and suicide notes. We show that there are marked\ndifferences across genders in how they use emotion words in work-place email.\nFor example, women use many words from the joy--sadness axis, whereas men\nprefer terms from the fear--trust axis. Finally, we show visualizations that\ncan help people track emotions in their emails.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6352v1",
            "date": "2013-09-24 21:21:02+00:00",
            "title": "Using Nuances of Emotion to Identify Personality",
            "authors": [
                "Saif M. Mohammad",
                "Svetlana Kiritchenko"
            ],
            "abstract": "Past work on personality detection has shown that frequency of lexical\ncategories such as first person pronouns, past tense verbs, and sentiment words\nhave significant correlations with personality traits. In this paper, for the\nfirst time, we show that fine affect (emotion) categories such as that of\nexcitement, guilt, yearning, and admiration are significant indicators of\npersonality. Additionally, we perform experiments to show that the gains\nprovided by the fine affect categories are not obtained by using coarse affect\ncategories alone or with specificity features alone. We employ these features\nin five SVM classifiers for detecting five personality traits through essays.\nWe find that the use of fine emotion features leads to statistically\nsignificant improvement over a competitive baseline, whereas the use of coarse\naffect and specificity features does not.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6722v1",
            "date": "2013-09-26 05:18:12+00:00",
            "title": "Domain-Specific Sentiment Word Extraction by Seed Expansion and Pattern Generation",
            "authors": [
                "Tang Duyu",
                "Qin Bing",
                "Zhou LanJun",
                "Wong KamFai",
                "Zhao Yanyan",
                "Liu Ting"
            ],
            "abstract": "This paper focuses on the automatic extraction of domain-specific sentiment\nword (DSSW), which is a fundamental subtask of sentiment analysis. Most\nprevious work utilizes manual patterns for this task. However, the performance\nof those methods highly relies on the labelled patterns or selected seeds. In\norder to overcome the above problem, this paper presents an automatic framework\nto detect large-scale domain-specific patterns for DSSW extraction. To this\nend, sentiment seeds are extracted from massive dataset of user comments.\nSubsequently, these sentiment seeds are expanded by synonyms using a\nbootstrapping mechanism. Simultaneously, a synonymy graph is built and the\ngraph propagation algorithm is applied on the built synonymy graph. Afterwards,\nsyntactic and sequential relations between target words and high-ranked\nsentiment words are extracted automatically to construct large-scale patterns,\nwhich are further used to extracte DSSWs. The experimental results in three\ndomains reveal the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.7312v1",
            "date": "2013-09-27 17:54:14+00:00",
            "title": "Development and Transcription of Assamese Speech Corpus",
            "authors": [
                "Himangshu Sarma",
                "Navanath Saharia",
                "Utpal Sharma",
                "Smriti Kumar Sinha",
                "Mancha Jyoti Malakar"
            ],
            "abstract": "A balanced speech corpus is the basic need for any speech processing task. In\nthis report we describe our effort on development of Assamese speech corpus. We\nmainly focused on some issues and challenges faced during development of the\ncorpus. Being a less computationally aware language, this is the first effort\nto develop speech corpus for Assamese. As corpus development is an ongoing\nprocess, in this paper we report only the initial task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.0482v1",
            "date": "2013-11-28 04:58:59+00:00",
            "title": "Learning Semantic Representations for the Phrase Translation Model",
            "authors": [
                "Jianfeng Gao",
                "Xiaodong He",
                "Wen-tau Yih",
                "Li Deng"
            ],
            "abstract": "This paper presents a novel semantic-based phrase translation model. A pair\nof source and target phrases are projected into continuous-valued vector\nrepresentations in a low-dimensional latent semantic space, where their\ntranslation score is computed by the distance between the pair in this new\nspace. The projection is performed by a multi-layer neural network whose\nweights are learned on parallel training data. The learning is aimed to\ndirectly optimize the quality of end-to-end machine translation results.\nExperimental evaluation has been performed on two Europarl translation tasks,\nEnglish-French and German-English. The results show that the new semantic-based\nphrase translation model significantly improves the performance of a\nstate-of-the-art phrase-based statistical machine translation sys-tem, leading\nto a gain of 0.7-1.0 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.2087v1",
            "date": "2013-12-07 11:19:20+00:00",
            "title": "Towards Structural Natural Language Formalization: Mapping Discourse to Controlled Natural Language",
            "authors": [
                "Nicholas H. Kirk"
            ],
            "abstract": "The author describes a conceptual study towards mapping grounded natural\nlanguage discourse representation structures to instances of controlled\nlanguage statements. This can be achieved via a pipeline of preexisting state\nof the art technologies, namely natural language syntax to semantic discourse\nmapping, and a reduction of the latter to controlled language discourse, given\na set of previously learnt reduction rules. Concludingly a description on\nevaluation, potential and limitations for ontology-based reasoning is\npresented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.3005v3",
            "date": "2013-12-11 00:25:57+00:00",
            "title": "One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling",
            "authors": [
                "Ciprian Chelba",
                "Tomas Mikolov",
                "Mike Schuster",
                "Qi Ge",
                "Thorsten Brants",
                "Phillipp Koehn",
                "Tony Robinson"
            ],
            "abstract": "We propose a new benchmark corpus to be used for measuring progress in\nstatistical language modeling. With almost one billion words of training data,\nwe hope this benchmark will be useful to quickly evaluate novel language\nmodeling techniques, and to compare their contribution when combined with other\nadvanced techniques. We show performance of several well-known types of\nlanguage models, with the best results achieved with a recurrent neural network\nbased language model. The baseline unpruned Kneser-Ney 5-gram model achieves\nperplexity 67.6; a combination of techniques leads to 35% reduction in\nperplexity, or 10% reduction in cross-entropy (bits), over that baseline.\n  The benchmark is available as a code.google.com project; besides the scripts\nneeded to rebuild the training/held-out data, it also makes available\nlog-probability values for each word in each of ten held-out data sets, for\neach of the baseline n-gram models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.3168v1",
            "date": "2013-12-11 14:04:52+00:00",
            "title": "Semantic Types, Lexical Sorts and Classifiers",
            "authors": [
                "Bruno Mery",
                "Christian Retor\u00e9"
            ],
            "abstract": "We propose a cognitively and linguistically motivated set of sorts for\nlexical semantics in a compositional setting: the classifiers in languages that\ndo have such pronouns. These sorts are needed to include lexical considerations\nin a semantical analyser such as Boxer or Grail. Indeed, all proposed lexical\nextensions of usual Montague semantics to model restriction of selection,\nfelicitous and infelicitous copredication require a rich and refined type\nsystem whose base types are the lexical sorts, the basis of the many-sorted\nlogic in which semantical representations of sentences are stated. However,\nnone of those approaches define precisely the actual base types or sorts to be\nused in the lexicon. In this article, we shall discuss some of the options\ncommonly adopted by researchers in formal lexical semantics, and defend the\nview that classifiers in the languages which have such pronouns are an\nappealing solution, both linguistically and cognitively motivated.",
            "journal ref": "NLPCS '10- 10th International Workshop on Natural Language\n  Processing and Computer Science - 2013 (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.3251v1",
            "date": "2013-12-11 17:24:35+00:00",
            "title": "Towards The Development of a Bishnupriya Manipuri Corpus",
            "authors": [
                "Nayan Jyoti Kalita",
                "Navanath Saharia",
                "Smriti Kumar Sinha"
            ],
            "abstract": "For any deep computational processing of language we need evidences, and one\nsuch set of evidences is corpus. This paper describes the development of a\ntext-based corpus for the Bishnupriya Manipuri language. A Corpus is considered\nas a building block for any language processing tasks. Due to the lack of\nawareness like other Indian languages, it is also studied less frequently. As a\nresult the language still lacks a good corpus and basic language processing\ntools. As per our knowledge this is the first effort to develop a corpus for\nBishnupriya Manipuri language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.3258v1",
            "date": "2013-12-11 17:50:21+00:00",
            "title": "Implicit Sensitive Text Summarization based on Data Conveyed by Connectives",
            "authors": [
                "Henda Chorfi Ouertani"
            ],
            "abstract": "So far and trying to reach human capabilities, research in automatic\nsummarization has been based on hypothesis that are both enabling and limiting.\nSome of these limitations are: how to take into account and reflect (in the\ngenerated summary) the implicit information conveyed in the text, the author\nintention, the reader intention, the context influence, the general world\nknowledge. Thus, if we want machines to mimic human abilities, then they will\nneed access to this same large variety of knowledge. The implicit is affecting\nthe orientation and the argumentation of the text and consequently its summary.\nMost of Text Summarizers (TS) are processing as compressing the initial data\nand they necessarily suffer from information loss. TS are focusing on features\nof the text only, not on what the author intended or why the reader is reading\nthe text. In this paper, we address this problem and we present a system\nfocusing on acquiring knowledge that is implicit. We principally spotlight the\nimplicit information conveyed by the argumentative connectives such as: but,\neven, yet and their effect on the summary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.5129v2",
            "date": "2013-12-18 13:34:16+00:00",
            "title": "Deep Learning Embeddings for Discontinuous Linguistic Units",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Deep learning embeddings have been successfully used for many natural\nlanguage processing problems. Embeddings are mostly computed for word forms\nalthough a number of recent papers have extended this to other linguistic units\nlike morphemes and phrases. In this paper, we argue that learning embeddings\nfor discontinuous linguistic units should also be considered. In an\nexperimental evaluation on coreference resolution, we show that such embeddings\nperform better than word form embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.5559v3",
            "date": "2013-12-19 14:18:14+00:00",
            "title": "Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds",
            "authors": [
                "Irina Sergienya",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "There are two main approaches to the distributed representation of words:\nlow-dimensional deep learning embeddings and high-dimensional distributional\nmodels, in which each dimension corresponds to a context word. In this paper,\nwe combine these two approaches by learning embeddings based on\ndistributional-model vectors - as opposed to one-hot vectors as is standardly\ndone in deep learning. We show that the combined approach has better\nperformance on a word relatedness judgment task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.6173v4",
            "date": "2013-12-20 23:13:38+00:00",
            "title": "Multilingual Distributed Representations without Word Alignment",
            "authors": [
                "Karl Moritz Hermann",
                "Phil Blunsom"
            ],
            "abstract": "Distributed representations of meaning are a natural way to encode covariance\nrelationships between words and phrases in NLP. By overcoming data sparsity\nproblems, as well as providing information about semantic relatedness which is\nnot available in discrete representations, distributed representations have\nproven useful in many NLP tasks. Recent work has shown how compositional\nsemantic representations can successfully be applied to a number of monolingual\napplications such as sentiment analysis. At the same time, there has been some\ninitial success in work on learning shared word-level representations across\nlanguages. We combine these two approaches by proposing a method for learning\ndistributed representations in a multilingual setup. Our model learns to assign\nsimilar embeddings to aligned sentences and dissimilar ones to sentence which\nare not aligned while not requiring word alignments. We show that our\nrepresentations are semantically informative and apply them to a cross-lingual\ndocument classification task where we outperform the previous state of the art.\nFurther, by employing parallel corpora of multiple language pairs we find that\nour model learns representations that capture semantic relationships across\nlanguages for which no parallel data was used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.7223v1",
            "date": "2013-12-27 09:39:52+00:00",
            "title": "Quality Estimation of English-Hindi Outputs using Naive Bayes Classifier",
            "authors": [
                "Rashmi Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "In this paper we present an approach for estimating the quality of machine\ntranslation system. There are various methods for estimating the quality of\noutput sentences, but in this paper we focus on Na\\\"ive Bayes classifier to\nbuild model using features which are extracted from the input sentences. These\nfeatures are used for finding the likelihood of each of the sentences of the\ntraining data which are then further used for determining the scores of the\ntest data. On the basis of these scores we determine the class labels of the\ntest data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.2444v3",
            "date": "2013-01-11 10:38:09+00:00",
            "title": "TEI and LMF crosswalks",
            "authors": [
                "Laurent Romary"
            ],
            "abstract": "The present paper explores various arguments in favour of making the Text\nEncoding Initia-tive (TEI) guidelines an appropriate serialisation for ISO\nstandard 24613:2008 (LMF, Lexi-cal Mark-up Framework) . It also identifies the\nissues that would have to be resolved in order to reach an appropriate\nimplementation of these ideas, in particular in terms of infor-mational\ncoverage. We show how the customisation facilities offered by the TEI\nguidelines can provide an adequate background, not only to cover missing\ncomponents within the current Dictionary chapter of the TEI guidelines, but\nalso to allow specific lexical projects to deal with local constraints. We\nexpect this proposal to be a basis for a future ISO project in the context of\nthe on going revision of LMF.",
            "journal ref": "JLCL - Journal for Language Technology and Computational\n  Linguistics, 2015, 30 (1)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.2857v1",
            "date": "2013-01-14 04:01:25+00:00",
            "title": "SpeedRead: A Fast Named Entity Recognition Pipeline",
            "authors": [
                "Rami Al-Rfou'",
                "Steven Skiena"
            ],
            "abstract": "Online content analysis employs algorithmic methods to identify entities in\nunstructured text. Both machine learning and knowledge-base approaches lie at\nthe foundation of contemporary named entities extraction systems. However, the\nprogress in deploying these approaches on web-scale has been been hampered by\nthe computational cost of NLP over massive text corpora. We present SpeedRead\n(SR), a named entity recognition pipeline that runs at least 10 times faster\nthan Stanford NLP pipeline. This pipeline consists of a high performance Penn\nTreebank- compliant tokenizer, close to state-of-art part-of-speech (POS)\ntagger and knowledge-based named entity recognizer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3214v1",
            "date": "2013-01-15 03:45:27+00:00",
            "title": "The Manifold of Human Emotions",
            "authors": [
                "Seungyeon Kim",
                "Fuxin Li",
                "Guy Lebanon",
                "Irfan Essa"
            ],
            "abstract": "Sentiment analysis predicts the presence of positive or negative emotions in\na text document. In this paper, we consider higher dimensional extensions of\nthe sentiment concept, which represent a richer set of human emotions. Our\napproach goes beyond previous work in that our model contains a continuous\nmanifold rather than a finite set of human emotions. We investigate the\nresulting model, compare it to psychological observations, and explore its\npredictive capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3614v3",
            "date": "2013-01-16 07:56:20+00:00",
            "title": "Joint Space Neural Probabilistic Language Model for Statistical Machine Translation",
            "authors": [
                "Tsuyoshi Okita"
            ],
            "abstract": "A neural probabilistic language model (NPLM) provides an idea to achieve the\nbetter perplexity than n-gram language model and their smoothed language\nmodels. This paper investigates application area in bilingual NLP, specifically\nStatistical Machine Translation (SMT). We focus on the perspectives that NPLM\nhas potential to open the possibility to complement potentially `huge'\nmonolingual resources into the `resource-constraint' bilingual resources. We\nintroduce an ngram-HMM language model as NPLM using the non-parametric Bayesian\nconstruction. In order to facilitate the application to various tasks, we\npropose the joint space model of ngram-HMM language model. We show an\nexperiment of system combination in the area of SMT. One discovery was that our\ntreatment of noise improved the results 0.20 BLEU points if NPLM is trained in\nrelatively small corpus, in our case 500,000 sentence pairs, which is often the\ncase due to the long training time of NPLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3781v3",
            "date": "2013-01-16 18:24:43+00:00",
            "title": "Efficient Estimation of Word Representations in Vector Space",
            "authors": [
                "Tomas Mikolov",
                "Kai Chen",
                "Greg Corrado",
                "Jeffrey Dean"
            ],
            "abstract": "We propose two novel model architectures for computing continuous vector\nrepresentations of words from very large data sets. The quality of these\nrepresentations is measured in a word similarity task, and the results are\ncompared to the previously best performing techniques based on different types\nof neural networks. We observe large improvements in accuracy at much lower\ncomputational cost, i.e. it takes less than a day to learn high quality word\nvectors from a 1.6 billion words data set. Furthermore, we show that these\nvectors provide state-of-the-art performance on our test set for measuring\nsyntactic and semantic word similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.4432v1",
            "date": "2013-01-18 16:53:13+00:00",
            "title": "Language learning from positive evidence, reconsidered: A simplicity-based approach",
            "authors": [
                "Anne S. Hsu",
                "Nick Chater",
                "Paul M. B. Vit\u00e1nyi"
            ],
            "abstract": "Children learn their native language by exposure to their linguistic and\ncommunicative environment, but apparently without requiring that their mistakes\nare corrected. Such learning from positive evidence has been viewed as raising\nlogical problems for language acquisition. In particular, without correction,\nhow is the child to recover from conjecturing an over-general grammar, which\nwill be consistent with any sentence that the child hears? There have been many\nproposals concerning how this logical problem can be dissolved. Here, we review\nrecent formal results showing that the learner has sufficient data to learn\nsuccessfully from positive evidence, if it favours the simplest encoding of the\nlinguistic input. Results include the ability to learn a linguistic prediction,\ngrammaticality judgements, language production, and form-meaning mappings. The\nsimplicity approach can also be scaled-down to analyse the ability to learn a\nspecific linguistic constructions, and is amenable to empirical test as a\nframework for describing human language acquisition.",
            "journal ref": "A.S. Hsu, N. Chater, P.M.B. Vitanyi, Language learning from\n  positive evidence, reconsidered: A simplicity-based approach. Topics in\n  Cognitive Science, 5:1(2013), 35-55",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.3265v1",
            "date": "2013-04-11 11:56:39+00:00",
            "title": "Extension of hidden markov model for recognizing large vocabulary of sign language",
            "authors": [
                "Maher Jebali",
                "Patrice Dalle",
                "Mohamed Jemni"
            ],
            "abstract": "Computers still have a long way to go before they can interact with users in\na truly natural fashion. From a users perspective, the most natural way to\ninteract with a computer would be through a speech and gesture interface.\nAlthough speech recognition has made significant advances in the past ten\nyears, gesture recognition has been lagging behind. Sign Languages (SL) are the\nmost accomplished forms of gestural communication. Therefore, their automatic\nanalysis is a real challenge, which is interestingly implied to their lexical\nand syntactic organization levels. Statements dealing with sign language occupy\na significant interest in the Automatic Natural Language Processing (ANLP)\ndomain. In this work, we are dealing with sign language recognition, in\nparticular of French Sign Language (FSL). FSL has its own specificities, such\nas the simultaneity of several parameters, the important role of the facial\nexpression or movement and the use of space for the proper utterance\norganization. Unlike speech recognition, Frensh sign language (FSL) events\noccur both sequentially and simultaneously. Thus, the computational processing\nof FSL is too complex than the spoken languages. We present a novel approach\nbased on HMM to reduce the recognition complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.4520v1",
            "date": "2013-04-16 17:06:24+00:00",
            "title": "Sentiment Analysis : A Literature Survey",
            "authors": [
                "Subhabrata Mukherjee",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Our day-to-day life has always been influenced by what people think. Ideas\nand opinions of others have always affected our own opinions. The explosion of\nWeb 2.0 has led to increased activity in Podcasting, Blogging, Tagging,\nContributing to RSS, Social Bookmarking, and Social Networking. As a result\nthere has been an eruption of interest in people to mine these vast resources\nof data for opinions. Sentiment Analysis or Opinion Mining is the computational\ntreatment of opinions, sentiments and subjectivity of text. In this report, we\ntake a look at the various challenges and applications of Sentiment Analysis.\nWe will discuss in details various approaches to perform a computational\ntreatment of sentiments and opinions. Various supervised or data-driven\ntechniques to SA like Na\\\"ive Byes, Maximum Entropy, SVM, and Voted Perceptrons\nwill be discussed and their strengths and drawbacks will be touched upon. We\nwill also see a new dimension of analyzing sentiments by Cognitive Psychology\nmainly through the work of Janyce Wiebe, where we will see ways to detect\nsubjectivity, perspective in narrative and understanding the discourse\nstructure. We will also study some specific topics in Sentiment Analysis and\nthe contemporary works in those areas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.5880v1",
            "date": "2013-04-22 09:06:36+00:00",
            "title": "Dealing with natural language interfaces in a geolocation context",
            "authors": [
                "M. -A. Abchir",
                "Isis Truck",
                "Anna Pappa"
            ],
            "abstract": "In the geolocation field where high-level programs and low-level devices\ncoexist, it is often difficult to find a friendly user inter- face to configure\nall the parameters. The challenge addressed in this paper is to propose\nintuitive and simple, thus natural lan- guage interfaces to interact with\nlow-level devices. Such inter- faces contain natural language processing and\nfuzzy represen- tations of words that facilitate the elicitation of\nbusiness-level objectives in our context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.7282v1",
            "date": "2013-04-25 10:25:41+00:00",
            "title": "An Improved Approach for Word Ambiguity Removal",
            "authors": [
                "Priti Saktel",
                "Urmila Shrawankar"
            ],
            "abstract": "Word ambiguity removal is a task of removing ambiguity from a word, i.e.\ncorrect sense of word is identified from ambiguous sentences. This paper\ndescribes a model that uses Part of Speech tagger and three categories for word\nsense disambiguation (WSD). Human Computer Interaction is very needful to\nimprove interactions between users and computers. For this, the Supervised and\nUnsupervised methods are combined. The WSD algorithm is used to find the\nefficient and accurate sense of a word based on domain information. The\naccuracy of this work is evaluated with the aim of finding best suitable domain\nof word.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.7289v1",
            "date": "2013-04-26 21:31:08+00:00",
            "title": "TimeML-strict: clarifying temporal annotation",
            "authors": [
                "Leon Derczynski",
                "Hector Llorens",
                "Naushad UzZaman"
            ],
            "abstract": "TimeML is an XML-based schema for annotating temporal information over\ndiscourse. The standard has been used to annotate a variety of resources and is\nfollowed by a number of tools, the creation of which constitute hundreds of\nthousands of man-hours of research work. However, the current state of\nresources is such that many are not valid, or do not produce valid output, or\ncontain ambiguous or custom additions and removals. Difficulties arising from\nthese variances were highlighted in the TempEval-3 exercise, which included its\nown extra stipulations over conventional TimeML as a response.\n  To unify the state of current resources, and to make progress toward easy\nadoption of its current incarnation ISO-TimeML, this paper introduces\nTimeML-strict: a valid, unambiguous, and easy-to-process subset of TimeML. We\nalso introduce three resources -- a schema for TimeML-strict; a validator tool\nfor TimeML-strict, so that one may ensure documents are in the correct form;\nand a repair tool that corrects common invalidating errors and adds\ndisambiguating markup in order to convert documents from the laxer TimeML\nstandard to TimeML-strict.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.7942v1",
            "date": "2013-04-30 10:12:54+00:00",
            "title": "ManTIME: Temporal expression identification and normalization in the TempEval-3 challenge",
            "authors": [
                "Michele Filannino",
                "Gavin Brown",
                "Goran Nenadic"
            ],
            "abstract": "This paper describes a temporal expression identification and normalization\nsystem, ManTIME, developed for the TempEval-3 challenge. The identification\nphase combines the use of conditional random fields along with a\npost-processing identification pipeline, whereas the normalization phase is\ncarried out using NorMA, an open-source rule-based temporal normalizer. We\ninvestigate the performance variation with respect to different feature types.\nSpecifically, we show that the use of WordNet-based features in the\nidentification task negatively affects the overall performance, and that there\nis no statistically significant difference in using gazetteers, shallow parsing\nand propositional noun phrases labels on top of the morphological features. On\nthe test data, the best run achieved 0.95 (P), 0.85 (R) and 0.90 (F1) in the\nidentification phase. Normalization accuracies are 0.84 (type attribute) and\n0.77 (value attribute). Surprisingly, the use of the silver data (alone or in\naddition to the gold annotated ones) does not improve the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.1319v1",
            "date": "2013-05-06 20:27:55+00:00",
            "title": "New Alignment Methods for Discriminative Book Summarization",
            "authors": [
                "David Bamman",
                "Noah A. Smith"
            ],
            "abstract": "We consider the unsupervised alignment of the full text of a book with a\nhuman-written summary. This presents challenges not seen in other text\nalignment problems, including a disparity in length and, consequent to this, a\nviolation of the expectation that individual words and phrases should align,\nsince large passages and chapters can be distilled into a single summary\nphrase. We present two new methods, based on hidden Markov models, specifically\ntargeted to this problem, and demonstrate gains on an extractive book\nsummarization task. While there is still much room for improvement,\nunsupervised alignment holds intrinsic value in offering insight into what\nfeatures of a book are deemed worthy of summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.3882v2",
            "date": "2013-05-16 18:09:21+00:00",
            "title": "Rule-Based Semantic Tagging. An Application Undergoing Dictionary Glosses",
            "authors": [
                "Daniel Christen"
            ],
            "abstract": "The project presented in this article aims to formalize criteria and\nprocedures in order to extract semantic information from parsed dictionary\nglosses. The actual purpose of the project is the generation of a semantic\nnetwork (nearly an ontology) issued from a monolingual Italian dictionary,\nthrough unsupervised procedures. Since the project involves rule-based Parsing,\nSemantic Tagging and Word Sense Disambiguation techniques, its outcomes may\nfind an interest also beyond this immediate intent. The cooperation of both\nsyntactic and semantic features in meaning construction are investigated, and\nprocedures which allows a translation of syntactic dependencies in semantic\nrelations are discussed. The procedures that rise from this project can be\napplied also to other text types than dictionary glosses, as they convert the\noutput of a parsing process into a semantic representation. In addition some\nmechanism are sketched that may lead to a kind of procedural semantics, through\nwhich multiple paraphrases of an given expression can be generated. Which means\nthat these techniques may find an application also in 'query expansion'\nstrategies, interesting Information Retrieval, Search Engines and Question\nAnswering Systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.3981v1",
            "date": "2013-05-17 05:14:43+00:00",
            "title": "Binary Tree based Chinese Word Segmentation",
            "authors": [
                "Kaixu Zhang",
                "Can Wang",
                "Maosong Sun"
            ],
            "abstract": "Chinese word segmentation is a fundamental task for Chinese language\nprocessing. The granularity mismatch problem is the main cause of the errors.\nThis paper showed that the binary tree representation can store outputs with\ndifferent granularity. A binary tree based framework is also designed to\novercome the granularity mismatch problem. There are two steps in this\nframework, namely tree building and tree pruning. The tree pruning step is\nspecially designed to focus on the granularity problem. Previous work for\nChinese word segmentation such as the sequence tagging can be easily employed\nin this framework. This framework can also provide quantitative error analysis\nmethods. The experiments showed that after using a more sophisticated tree\npruning function for a state-of-the-art conditional random field based\nbaseline, the error reduction can be up to 20%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.5753v3",
            "date": "2013-05-23 03:14:50+00:00",
            "title": "A probabilistic framework for analysing the compositionality of conceptual combinations",
            "authors": [
                "Peter D. Bruza",
                "Kirsty Kitto",
                "Brentyn J. Ramm",
                "Laurianne Sitbon"
            ],
            "abstract": "Conceptual combination performs a fundamental role in creating the broad\nrange of compound phrases utilized in everyday language. This article provides\na novel probabilistic framework for assessing whether the semantics of\nconceptual combinations are compositional, and so can be considered as a\nfunction of the semantics of the constituent concepts, or not. While the\nsystematicity and productivity of language provide a strong argument in favor\nof assuming compositionality, this very assumption is still regularly\nquestioned in both cognitive science and philosophy. Additionally, the\nprinciple of semantic compositionality is underspecified, which means that\nnotions of both \"strong\" and \"weak\" compositionality appear in the literature.\nRather than adjudicating between different grades of compositionality, the\nframework presented here contributes formal methods for determining a clear\ndividing line between compositional and non-compositional semantics. In\naddition, we suggest that the distinction between these is contextually\nsensitive. Utilizing formal frameworks developed for analyzing composite\nsystems in quantum theory, we present two methods that allow the semantics of\nconceptual combinations to be classified as \"compositional\" or\n\"non-compositional\". Compositionality is first formalised by factorising the\njoint probability distribution modeling the combination, where the terms in the\nfactorisation correspond to individual concepts. This leads to the necessary\nand sufficient condition for the joint probability distribution to exist. A\nfailure to meet this condition implies that the underlying concepts cannot be\nmodeled in a single probability space when considering their combination, and\nthe combination is thus deemed \"non-compositional\". The formal analysis methods\nare demonstrated by applying them to an empirical study of twenty-four\nnon-lexicalised conceptual combinations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.5785v1",
            "date": "2013-05-24 16:34:22+00:00",
            "title": "An Inventory of Preposition Relations",
            "authors": [
                "Vivek Srikumar",
                "Dan Roth"
            ],
            "abstract": "We describe an inventory of semantic relations that are expressed by\nprepositions. We define these relations by building on the word sense\ndisambiguation task for prepositions and propose a mapping from preposition\nsenses to the relation labels by collapsing semantically related senses across\nprepositions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.5918v1",
            "date": "2013-05-25 13:20:31+00:00",
            "title": "Reduce Meaningless Words for Joint Chinese Word Segmentation and Part-of-speech Tagging",
            "authors": [
                "Kaixu Zhang",
                "Maosong Sun"
            ],
            "abstract": "Conventional statistics-based methods for joint Chinese word segmentation and\npart-of-speech tagging (S&T) have generalization ability to recognize new words\nthat do not appear in the training data. An undesirable side effect is that a\nnumber of meaningless words will be incorrectly created. We propose an\neffective and efficient framework for S&T that introduces features to\nsignificantly reduce meaningless words generation. A general lexicon, Wikepedia\nand a large-scale raw corpus of 200 billion characters are used to generate\nword-based features for the wordhood. The word-lattice based framework consists\nof a character-based model and a word-based model in order to employ our\nword-based features. Experiments on Penn Chinese treebank 5 show that this\nmethod has a 62.9% reduction of meaningless word generation in comparison with\nthe baseline. As a result, the F1 measure for segmentation is increased to\n0.984.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.6211v2",
            "date": "2013-05-24 18:01:34+00:00",
            "title": "Development of a Hindi Lemmatizer",
            "authors": [
                "Snigdha Paul",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "We live in a translingual society, in order to communicate with people from\ndifferent parts of the world we need to have an expertise in their respective\nlanguages. Learning all these languages is not at all possible; therefore we\nneed a mechanism which can do this task for us. Machine translators have\nemerged as a tool which can perform this task. In order to develop a machine\ntranslator we need to develop several different rules. The very first module\nthat comes in machine translation pipeline is morphological analysis. Stemming\nand lemmatization comes under morphological analysis. In this paper we have\ncreated a lemmatizer which generates rules for removing the affixes along with\nthe addition of rules for creating a proper root word.",
            "journal ref": "International Journal of Computational Linguistics and Natural\n  Language Processing, Vol 2, Issue 5, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.2091v2",
            "date": "2013-06-10 02:54:10+00:00",
            "title": "A framework for (under)specifying dependency syntax without overloading annotators",
            "authors": [
                "Nathan Schneider",
                "Brendan O'Connor",
                "Naomi Saphra",
                "David Bamman",
                "Manaal Faruqui",
                "Noah A. Smith",
                "Chris Dyer",
                "Jason Baldridge"
            ],
            "abstract": "We introduce a framework for lightweight dependency syntax annotation. Our\nformalism builds upon the typical representation for unlabeled dependencies,\npermitting a simple notation and annotation workflow. Moreover, the formalism\nencourages annotators to underspecify parts of the syntax if doing so would\nstreamline the annotation process. We demonstrate the efficacy of this\nannotation on three languages and develop algorithms to evaluate and compare\nunderspecified annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.2158v1",
            "date": "2013-06-10 10:29:09+00:00",
            "title": "\"Not not bad\" is not \"bad\": A distributional account of negation",
            "authors": [
                "Karl Moritz Hermann",
                "Edward Grefenstette",
                "Phil Blunsom"
            ],
            "abstract": "With the increasing empirical success of distributional models of\ncompositional semantics, it is timely to consider the types of textual logic\nthat such models are capable of capturing. In this paper, we address\nshortcomings in the ability of current models to capture logical operations\nsuch as negation. As a solution we propose a tripartite formulation for a\ncontinuous vector space representation of semantics and subsequently use this\nrepresentation to develop a formal compositional notion of negation within such\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.3584v1",
            "date": "2013-06-15 14:52:17+00:00",
            "title": "Recurrent Convolutional Neural Networks for Discourse Compositionality",
            "authors": [
                "Nal Kalchbrenner",
                "Phil Blunsom"
            ],
            "abstract": "The compositionality of meaning extends beyond the single sentence. Just as\nwords combine to form the meaning of sentences, so do sentences combine to form\nthe meaning of paragraphs, dialogues and general discourse. We introduce both a\nsentence model and a discourse model corresponding to the two levels of\ncompositionality. The sentence model adopts convolution as the central\noperation for composing semantic vectors and is based on a novel hierarchical\nconvolutional neural network. The discourse model extends the sentence model\nand is based on a recurrent neural network that is conditioned in a novel way\nboth on the current sentence and on the current speaker. The discourse model is\nable to capture both the sequentiality of sentences and the interaction between\ndifferent speakers. Without feature engineering or pretraining and with simple\ngreedy decoding, the discourse model coupled to the sentence model obtains\nstate of the art performance on a dialogue act classification experiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.4134v1",
            "date": "2013-06-18 10:32:43+00:00",
            "title": "Dialogue System: A Brief Review",
            "authors": [
                "Suket Arora",
                "Kamaljeet Batra",
                "Sarabjit Singh"
            ],
            "abstract": "A Dialogue System is a system which interacts with human in natural language.\nAt present many universities are developing the dialogue system in their\nregional language. This paper will discuss about dialogue system, its\ncomponents, challenges and its evaluation. This paper helps the researchers for\ngetting info regarding dialogues system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.6130v1",
            "date": "2013-06-26 05:02:26+00:00",
            "title": "Competency Tracking for English as a Second or Foreign Language Learners",
            "authors": [
                "Robert Bishop Jr"
            ],
            "abstract": "My system utilizes the outcomes feature found in Moodle and other learning\ncontent management systems (LCMSs) to keep track of where students are in terms\nof what language competencies they have mastered and the competencies they need\nto get where they want to go. These competencies are based on the Common\nEuropean Framework for (English) Language Learning. This data can be available\nfor everyone involved with a given student's progress (e.g. educators, parents,\nsupervisors and the students themselves). A given student's record of past\naccomplishments can also be meshed with those of his classmates. Not only are a\nstudent's competencies easily seen and tracked, educators can view competencies\nof a group of students that were achieved prior to enrollment in the class.\nThis should make curriculum decision making easier and more efficient for\neducators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.0833v1",
            "date": "2013-11-04 20:11:35+00:00",
            "title": "A Comparative Study on Linguistic Feature Selection in Sentiment Polarity Classification",
            "authors": [
                "Zitao Liu"
            ],
            "abstract": "Sentiment polarity classification is perhaps the most widely studied topic.\nIt classifies an opinionated document as expressing a positive or negative\nopinion. In this paper, using movie review dataset, we perform a comparative\nstudy with different single kind linguistic features and the combinations of\nthese features. We find that the classic topic-based classifier(Naive Bayes and\nSupport Vector Machine) do not perform as well on sentiment polarity\nclassification. And we find that with some combination of different linguistic\nfeatures, the classification accuracy can be boosted a lot. We give some\nreasonable explanations about these boosting outcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.1169v1",
            "date": "2013-11-05 19:31:33+00:00",
            "title": "Using Robust PCA to estimate regional characteristics of language use from geo-tagged Twitter messages",
            "authors": [
                "D\u00e1niel Kondor",
                "Istv\u00e1n Csabai",
                "L\u00e1szl\u00f3 Dobos",
                "J\u00e1nos Sz\u00fcle",
                "Norbert Barankai",
                "Tam\u00e1s Hanyecz",
                "Tam\u00e1s Seb\u0151k",
                "Zs\u00f3fia Kallus",
                "G\u00e1bor Vattay"
            ],
            "abstract": "Principal component analysis (PCA) and related techniques have been\nsuccessfully employed in natural language processing. Text mining applications\nin the age of the online social media (OSM) face new challenges due to\nproperties specific to these use cases (e.g. spelling issues specific to texts\nposted by users, the presence of spammers and bots, service announcements,\netc.). In this paper, we employ a Robust PCA technique to separate typical\noutliers and highly localized topics from the low-dimensional structure present\nin language use in online social networks. Our focus is on identifying\ngeospatial features among the messages posted by the users of the Twitter\nmicroblogging service. Using a dataset which consists of over 200 million\ngeolocated tweets collected over the course of a year, we investigate whether\nthe information present in word usage frequencies can be used to identify\nregional features of language use and topics of interest. Using the PCA pursuit\nmethod, we are able to identify important low-dimensional features, which\nconstitute smoothly varying functions of the geographic location.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.1194v1",
            "date": "2013-11-05 20:55:23+00:00",
            "title": "Identifying Purpose Behind Electoral Tweets",
            "authors": [
                "Saif M. Mohammad",
                "Svetlana Kiritchenko",
                "Joel Martin"
            ],
            "abstract": "Tweets pertaining to a single event, such as a national election, can number\nin the hundreds of millions. Automatically analyzing them is beneficial in many\ndownstream natural language applications such as question answering and\nsummarization. In this paper, we propose a new task: identifying the purpose\nbehind electoral tweets--why do people post election-oriented tweets? We show\nthat identifying purpose is correlated with the related phenomenon of sentiment\nand emotion detection, but yet significantly different. Detecting purpose has a\nnumber of applications including detecting the mood of the electorate,\nestimating the popularity of policies, identifying key issues of contention,\nand predicting the course of events. We create a large dataset of electoral\ntweets and annotate a few thousand tweets for purpose. We develop a system that\nautomatically classifies electoral tweets as per their purpose, obtaining an\naccuracy of 43.56% on an 11-class task and an accuracy of 73.91% on a 3-class\ntask (both accuracies well above the most-frequent-class baseline). Finally, we\nshow that resources developed for emotion detection are also helpful for\ndetecting purpose.",
            "journal ref": "In Proceedings of the KDD Workshop on Issues of Sentiment\n  Discovery and Opinion Mining (WISDOM-2013), August 2013, Chicago, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.2978v1",
            "date": "2013-11-12 23:11:40+00:00",
            "title": "Authorship Attribution Using Word Network Features",
            "authors": [
                "Shibamouli Lahiri",
                "Rada Mihalcea"
            ],
            "abstract": "In this paper, we explore a set of novel features for authorship attribution\nof documents. These features are derived from a word network representation of\nnatural language text. As has been noted in previous studies, natural language\ntends to show complex network structure at word level, with low degrees of\nseparation and scale-free (power law) degree distribution. There has also been\nwork on authorship attribution that incorporates ideas from complex networks.\nThe goal of our paper is to explore properties of these complex networks that\nare suitable as features for machine-learning-based authorship attribution of\ndocuments. We performed experiments on three different datasets, and obtained\npromising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.3011v2",
            "date": "2013-11-13 03:58:38+00:00",
            "title": "Cornell SPF: Cornell Semantic Parsing Framework",
            "authors": [
                "Yoav Artzi"
            ],
            "abstract": "The Cornell Semantic Parsing Framework (SPF) is a learning and inference\nframework for mapping natural language to formal representation of its meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.3961v1",
            "date": "2013-11-15 19:45:25+00:00",
            "title": "HEVAL: Yet Another Human Evaluation Metric",
            "authors": [
                "Nisheeth Joshi",
                "Iti Mathur",
                "Hemant Darbari",
                "Ajai Kumar"
            ],
            "abstract": "Machine translation evaluation is a very important activity in machine\ntranslation development. Automatic evaluation metrics proposed in literature\nare inadequate as they require one or more human reference translations to\ncompare them with output produced by machine translation. This does not always\ngive accurate results as a text can have several different translations. Human\nevaluation metrics, on the other hand, lacks inter-annotator agreement and\nrepeatability. In this paper we have proposed a new human evaluation metric\nwhich addresses these issues. Moreover this metric also provides solid grounds\nfor making sound assumptions on the quality of the text produced by a machine\ntranslation.",
            "journal ref": "International Journal on Natural Language Computing Vol. 2, No.5,\n  November 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.5836v1",
            "date": "2013-11-22 18:13:06+00:00",
            "title": "Automatic Ranking of MT Outputs using Approximations",
            "authors": [
                "Pooja Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Since long, research on machine translation has been ongoing. Still, we do\nnot get good translations from MT engines so developed. Manual ranking of these\noutputs tends to be very time consuming and expensive. Identifying which one is\nbetter or worse than the others is a very taxing task. In this paper, we show\nan approach which can provide automatic ranks to MT outputs (translations)\ntaken from different MT Engines and which is based on N-gram approximations. We\nprovide a solution where no human intervention is required for ranking systems.\nFurther we also show the evaluations of our results which show equivalent\nresults as that of human ranking.",
            "journal ref": "International Journal of Computer Applications 81(17):27-31,\n  November 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.6045v1",
            "date": "2013-11-23 20:10:24+00:00",
            "title": "Build Electronic Arabic Lexicon",
            "authors": [
                "Nidhal El-Abbadi",
                "Ahmed Nidhal Khdhair",
                "Adel Al-Nasrawi"
            ],
            "abstract": "There are many known Arabic lexicons organized on different ways, each of\nthem has a different number of Arabic words according to its organization way.\nThis paper has used mathematical relations to count a number of Arabic words,\nwhich proofs the number of Arabic words presented by Al Farahidy. The paper\nalso presents new way to build an electronic Arabic lexicon by using a hash\nfunction that converts each word (as input) to correspond a unique integer\nnumber (as output), these integer numbers will be used as an index to a lexicon\nentry.",
            "journal ref": "The International Arab Journal of Information Technology, Vol. 8,\n  No. 2, April 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.6063v5",
            "date": "2013-11-23 22:39:52+00:00",
            "title": "NILE: Fast Natural Language Processing for Electronic Health Records",
            "authors": [
                "Sheng Yu",
                "Tianrun Cai",
                "Tianxi Cai"
            ],
            "abstract": "Objective: Narrative text in Electronic health records (EHR) contain rich\ninformation for medical and data science studies. This paper introduces the\ndesign and performance of Narrative Information Linear Extraction (NILE), a\nnatural language processing (NLP) package for EHR analysis that we share with\nthe medical informatics community. Methods: NILE uses a modified prefix-tree\nsearch algorithm for named entity recognition, which can detect prefix and\nsuffix sharing. The semantic analyses are implemented as rule-based finite\nstate machines. Analyses include negation, location, modification, family\nhistory, and ignoring. Result: The processing speed of NILE is hundreds to\nthousands times faster than existing NLP software for medical text. The\naccuracy of presence analysis of NILE is on par with the best performing models\non the 2010 i2b2/VA NLP challenge data. Conclusion: The speed, accuracy, and\nbeing able to operate via API make NILE a valuable addition to the NLP software\nfor medical informatics and data science.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.0446v1",
            "date": "2013-03-03 01:38:03+00:00",
            "title": "Statistical sentiment analysis performance in Opinum",
            "authors": [
                "Boyan Bonev",
                "Gema Ram\u00edrez-S\u00e1nchez",
                "Sergio Ortiz Rojas"
            ],
            "abstract": "The classification of opinion texts in positive and negative is becoming a\nsubject of great interest in sentiment analysis. The existence of many labeled\nopinions motivates the use of statistical and machine-learning methods.\nFirst-order statistics have proven to be very limited in this field. The Opinum\napproach is based on the order of the words without using any syntactic and\nsemantic information. It consists of building one probabilistic model for the\npositive and another one for the negative opinions. Then the test opinions are\ncompared to both models and a decision and confidence measure are calculated.\nIn order to reduce the complexity of the training corpus we first lemmatize the\ntexts and we replace most named-entities with wildcards. Opinum presents an\naccuracy above 81% for Spanish opinions in the financial products domain. In\nthis work we discuss which are the most important factors that have impact on\nthe classification performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1929v1",
            "date": "2013-03-08 10:13:56+00:00",
            "title": "Towards the Fully Automatic Merging of Lexical Resources: A Step Forward",
            "authors": [
                "Muntsa Padr\u00f3",
                "N\u00faria Bel",
                "Silvia Necsulescu"
            ],
            "abstract": "This article reports on the results of the research done towards the fully\nautomatically merging of lexical resources. Our main goal is to show the\ngenerality of the proposed approach, which have been previously applied to\nmerge Spanish Subcategorization Frames lexica. In this work we extend and apply\nthe same technique to perform the merging of morphosyntactic lexica encoded in\nLMF. The experiments showed that the technique is general enough to obtain good\nresults in these two different tasks which is an important step towards\nperforming the merging of lexical resources fully automatically.",
            "journal ref": "LREC 2012 Workshop on Language Resource Merging; 2012 May 22;\n  Istanbul, Turkey. Paris: European Language Resources Association; 2012. p.\n  8-14",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1930v1",
            "date": "2013-03-08 10:14:04+00:00",
            "title": "Automatic lexical semantic classification of nouns",
            "authors": [
                "N\u00faria Bel",
                "Lauren Romeo",
                "Muntsa Padr\u00f3"
            ],
            "abstract": "The work we present here addresses cue-based noun classification in English\nand Spanish. Its main objective is to automatically acquire lexical semantic\ninformation by classifying nouns into previously known noun lexical classes.\nThis is achieved by using particular aspects of linguistic contexts as cues\nthat identify a specific lexical class. Here we concentrate on the task of\nidentifying such cues and the theoretical background that allows for an\nassessment of the complexity of the task. The results show that, despite of the\na-priori complexity of the task, cue-based classification is a useful tool in\nthe automatic acquisition of lexical semantic classes.",
            "journal ref": "Proceedings of the Eight International Conference on Language\n  Resources and Evaluation (LREC'12); 2012 May 23-25; Istanbul, Turkey. Paris:\n  European Language Resources Association; 2012. p. 1448-1455",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1931v1",
            "date": "2013-03-08 10:14:31+00:00",
            "title": "A Classification of Adjectives for Polarity Lexicons Enhancement",
            "authors": [
                "Silvia V\u00e1zquez",
                "N\u00faria Bel"
            ],
            "abstract": "Subjective language detection is one of the most important challenges in\nSentiment Analysis. Because of the weight and frequency in opinionated texts,\nadjectives are considered a key piece in the opinion extraction process. These\nsubjective units are more and more frequently collected in polarity lexicons in\nwhich they appear annotated with their prior polarity. However, at the moment,\nany polarity lexicon takes into account prior polarity variations across\ndomains. This paper proves that a majority of adjectives change their prior\npolarity value depending on the domain. We propose a distinction between domain\ndependent and domain independent adjectives. Moreover, our analysis led us to\npropose a further classification related to subjectivity degree: constant,\nmixed and highly subjective adjectives. Following this classification, polarity\nvalues will be a better support for Sentiment Analysis.",
            "journal ref": "Proceedings of the Eight International Conference on Language\n  Resources and Evaluation (LREC'12); 2012 May 23-25; Istanbul, Turkey. Paris:\n  European Language Resources Association; 2012. p. 3557-3561",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1932v1",
            "date": "2013-03-08 10:15:57+00:00",
            "title": "Mining and Exploiting Domain-Specific Corpora in the PANACEA Platform",
            "authors": [
                "N\u00faria Bel",
                "Vassilis Papavasiliou",
                "Prokopis Prokopidis",
                "Antonio Toral",
                "Victoria Arranz"
            ],
            "abstract": "The objective of the PANACEA ICT-2007.2.2 EU project is to build a platform\nthat automates the stages involved in the acquisition, production, updating and\nmaintenance of the large language resources required by, among others, MT\nsystems. The development of a Corpus Acquisition Component (CAC) for extracting\nmonolingual and bilingual data from the web is one of the most innovative\nbuilding blocks of PANACEA. The CAC, which is the first stage in the PANACEA\npipeline for building Language Resources, adopts an efficient and distributed\nmethodology to crawl for web documents with rich textual content in specific\nlanguages and predefined domains. The CAC includes modules that can acquire\nparallel data from sites with in-domain content available in more than one\nlanguage. In order to extrinsically evaluate the CAC methodology, we have\nconducted several experiments that used crawled parallel corpora for the\nidentification and extraction of parallel sentences using sentence alignment.\nThe corpora were then successfully used for domain adaptation of Machine\nTranslation Systems.",
            "journal ref": "Proceedings of the 5th Workshop on Building and Using Comparable\n  Corpora at the Eighth International Conference on Language Resources and\n  Evaluation (LREC-2012); 2012 May 23-25; Istanbul, Turkey. Paris: ELRA; 2012.\n  p. 24-26",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.2448v1",
            "date": "2013-03-11 08:21:17+00:00",
            "title": "Automatic Detection of Non-deverbal Event Nouns for Quick Lexicon Production",
            "authors": [
                "N\u00faria Bel",
                "Maria Coll",
                "Gabriela Resnik"
            ],
            "abstract": "In this work we present the results of our experimental work on the\ndevelop-ment of lexical class-based lexica by automatic means. The objective is\nto as-sess the use of linguistic lexical-class based information as a feature\nselection methodology for the use of classifiers in quick lexical development.\nThe results show that the approach can help in re-ducing the human effort\nrequired in the development of language resources sig-nificantly.",
            "journal ref": "Proceedings of the 23rd International Conference on Computational\n  Linguistics (Coling 2010); 2010 Aug 23-27; Beijing, CN. Stroudsburg: ACL;\n  2010. p. 46-52",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.2449v1",
            "date": "2013-03-11 08:21:48+00:00",
            "title": "Using qualia information to identify lexical semantic classes in an unsupervised clustering task",
            "authors": [
                "Lauren Romeo",
                "Sara Mendes",
                "N\u00faria Bel"
            ],
            "abstract": "Acquiring lexical information is a complex problem, typically approached by\nrelying on a number of contexts to contribute information for classification.\nOne of the first issues to address in this domain is the determination of such\ncontexts. The work presented here proposes the use of automatically obtained\nFORMAL role descriptors as features used to draw nouns from the same lexical\nsemantic class together in an unsupervised clustering task. We have dealt with\nthree lexical semantic classes (HUMAN, LOCATION and EVENT) in English. The\nresults obtained show that it is possible to discriminate between elements from\ndifferent lexical semantic classes using only FORMAL role information, hence\nvalidating our initial hypothesis. Also, iterating our method accurately\naccounts for fine-grained distinctions within lexical classes, namely\ndistinctions involving ambiguous expressions. Moreover, a filtering and\nbootstrapping strategy employed in extracting FORMAL role descriptors proved to\nminimize effects of sparse data and noise in our task.",
            "journal ref": "Proceedings of COLING 2012: Posters: 24th International Conference\n  on Computational Linguistics COLING 2012; 2012 December 8-15; Mumbai, India.\n  Mumbai: The COLING 2012 Organizing Committee; 2012. p. 1029-1038",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.2826v1",
            "date": "2013-03-12 10:20:50+00:00",
            "title": "Probabilistic Topic and Syntax Modeling with Part-of-Speech LDA",
            "authors": [
                "William M. Darling",
                "Fei Song"
            ],
            "abstract": "This article presents a probabilistic generative model for text based on\nsemantic topics and syntactic classes called Part-of-Speech LDA (POSLDA).\nPOSLDA simultaneously uncovers short-range syntactic patterns (syntax) and\nlong-range semantic patterns (topics) that exist in document collections. This\nresults in word distributions that are specific to both topics (sports,\neducation, ...) and parts-of-speech (nouns, verbs, ...). For example,\nmultinomial distributions over words are uncovered that can be understood as\n\"nouns about weather\" or \"verbs about law\". We describe the model and an\napproximate inference algorithm and then demonstrate the quality of the learned\ntopics both qualitatively and quantitatively. Then, we discuss an NLP\napplication where the output of POSLDA can lead to strong improvements in\nquality: unsupervised part-of-speech tagging. We describe algorithms for this\ntask that make use of POSLDA-learned distributions that result in improved\nperformance beyond the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.5960v3",
            "date": "2013-03-24 15:27:51+00:00",
            "title": "SYNTAGMA. A Linguistic Approach to Parsing",
            "authors": [
                "Daniel Christen"
            ],
            "abstract": "SYNTAGMA is a rule-based parsing system, structured on two levels: a general\nparsing engine and a language specific grammar. The parsing engine is a\nlanguage independent program, while grammar and language specific rules and\nresources are given as text files, consisting in a list of constituent\nstructuresand a lexical database with word sense related features and\nconstraints. Since its theoretical background is principally Tesniere's\nElements de syntaxe, SYNTAGMA's grammar emphasizes the role of argument\nstructure (valency) in constraint satisfaction, and allows also horizontal\nbounds, for instance treating coordination. Notions such as Pro, traces, empty\ncategories are derived from Generative Grammar and some solutions are close to\nGovernment&Binding Theory, although they are the result of an autonomous\nresearch. These properties allow SYNTAGMA to manage complex syntactic\nconfigurations and well known weak points in parsing engineering. An important\nresource is the semantic network, which is used in disambiguation tasks.\nParsing process follows a bottom-up, rule driven strategy. Its behavior can be\ncontrolled and fine-tuned.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.0596v1",
            "date": "2013-07-02 06:25:51+00:00",
            "title": "Improving Pointwise Mutual Information (PMI) by Incorporating Significant Co-occurrence",
            "authors": [
                "Om P. Damani"
            ],
            "abstract": "We design a new co-occurrence based word association measure by incorporating\nthe concept of significant cooccurrence in the popular word association measure\nPointwise Mutual Information (PMI). By extensive experiments with a large\nnumber of publicly available datasets we show that the newly introduced measure\nperforms better than other co-occurrence based measures and despite being\nresource-light, compares well with the best known resource-heavy distributional\nsimilarity and knowledge based word association measures. We investigate the\nsource of this performance improvement and find that of the two types of\nsignificant co-occurrence - corpus-level and document-level, the concept of\ncorpus level significance combined with the use of document counts in place of\nword counts is responsible for all the performance gains observed. The concept\nof document level significance is not helpful for PMI adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.1872v1",
            "date": "2013-07-07 15:04:11+00:00",
            "title": "Intelligent Hybrid Man-Machine Translation Quality Estimation",
            "authors": [
                "Ibrahim Sabek",
                "Noha A. Yousri",
                "Nagwa Elmakky",
                "Mona Habib"
            ],
            "abstract": "Inferring evaluation scores based on human judgments is invaluable compared\nto using current evaluation metrics which are not suitable for real-time\napplications e.g. post-editing. However, these judgments are much more\nexpensive to collect especially from expert translators, compared to evaluation\nbased on indicators contrasting source and translation texts. This work\nintroduces a novel approach for quality estimation by combining learnt\nconfidence scores from a probabilistic inference model based on human\njudgments, with selective linguistic features-based scores, where the proposed\ninference model infers the credibility of given human ranks to solve the\nscarcity and inconsistency issues of human judgments. Experimental results,\nusing challenging language-pairs, demonstrate improvement in correlation with\nhuman judgments over traditional evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.3310v1",
            "date": "2013-07-12 03:05:29+00:00",
            "title": "Improving the quality of Gujarati-Hindi Machine Translation through part-of-speech tagging and stemmer-assisted transliteration",
            "authors": [
                "Juhi Ameta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Machine Translation for Indian languages is an emerging research area.\nTransliteration is one such module that we design while designing a translation\nsystem. Transliteration means mapping of source language text into the target\nlanguage. Simple mapping decreases the efficiency of overall translation\nsystem. We propose the use of stemming and part-of-speech tagging for\ntransliteration. The effectiveness of translation can be improved if we use\npart-of-speech tagging and stemming assisted transliteration.We have shown that\nmuch of the content in Gujarati gets transliterated while being processed for\ntranslation to Hindi language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.4299v1",
            "date": "2013-07-15 15:59:12+00:00",
            "title": "Part of Speech Tagging of Marathi Text Using Trigram Method",
            "authors": [
                "Jyoti Singh",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "In this paper we present a Marathi part of speech tagger. It is a\nmorphologically rich language. It is spoken by the native people of\nMaharashtra. The general approach used for development of tagger is statistical\nusing trigram Method. The main concept of trigram is to explore the most likely\nPOS for a token based on given information of previous two tags by calculating\nprobabilities to determine which is the best sequence of a tag. In this paper\nwe show the development of the tagger. Moreover we have also shown the\nevaluation done.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.4300v1",
            "date": "2013-07-15 15:54:43+00:00",
            "title": "Rule Based Transliteration Scheme for English to Punjabi",
            "authors": [
                "Deepti Bhalla",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Machine Transliteration has come out to be an emerging and a very important\nresearch area in the field of machine translation. Transliteration basically\naims to preserve the phonological structure of words. Proper transliteration of\nname entities plays a very significant role in improving the quality of machine\ntranslation. In this paper we are doing machine transliteration for\nEnglish-Punjabi language pair using rule based approach. We have constructed\nsome rules for syllabification. Syllabification is the process to extract or\nseparate the syllable from the words. In this we are calculating the\nprobabilities for name entities (Proper names and location). For those words\nwhich do not come under the category of name entities, separate probabilities\nare being calculated by using relative frequency through a statistical machine\ntranslation toolkit known as MOSES. Using these probabilities we are\ntransliterating our input text from English to Punjabi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.5393v1",
            "date": "2013-07-20 09:05:15+00:00",
            "title": "Clustering Algorithm for Gujarati Language",
            "authors": [
                "Miral Patel",
                "Prem Balani"
            ],
            "abstract": "Natural language processing area is still under research. But now a day it is\non platform for worldwide researchers. Natural language processing includes\nanalyzing the language based on its structure and then tagging of each word\nappropriately with its grammar base. Here we have 50,000 tagged words set and\nwe try to cluster those Gujarati words based on proposed algorithm, we have\ndefined our own algorithm for processing. Many clustering techniques are\navailable Ex. Single linkage, complete, linkage,average linkage, Hear no of\nclusters to be formed are not known, so it is all depends on the type of data\nset provided . Clustering is preprocess for stemming . Stemming is the process\nwhere root is extracted from its word. Ex. cats= cat+S, meaning. Cat: Noun and\nplural form.",
            "journal ref": "International Journal for Scientific Research & Development Vol 1,\n  Issue 3, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.6163v2",
            "date": "2013-07-23 17:15:34+00:00",
            "title": "Human and Automatic Evaluation of English-Hindi Machine Translation",
            "authors": [
                "Nisheeth Joshi",
                "Hemant Darbari",
                "Iti Mathur"
            ],
            "abstract": "For the past 60 years, Research in machine translation is going on. For the\ndevelopment in this field, a lot of new techniques are being developed each\nday. As a result, we have witnessed development of many automatic machine\ntranslators. A manager of machine translation development project needs to know\nthe performance increase/decrease, after changes have been done in his system.\nDue to this reason, a need for evaluation of machine translation systems was\nfelt. In this article, we shall present the evaluation of some machine\ntranslators. This evaluation will be done by a human evaluator and by some\nautomatic evaluation metrics, which will be done at sentence, document and\nsystem level. In the end we shall also discuss the comparison between the\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.7382v1",
            "date": "2013-07-28 16:55:27+00:00",
            "title": "Learning Frames from Text with an Unsupervised Latent Variable Model",
            "authors": [
                "Brendan O'Connor"
            ],
            "abstract": "We develop a probabilistic latent-variable model to discover semantic\nframes---types of events and their participants---from corpora. We present a\nDirichlet-multinomial model in which frames are latent categories that explain\nthe linking of verb-subject-object triples, given document-level sparsity. We\nanalyze what the model learns, and compare it to FrameNet, noting it learns\nsome novel and interesting frames. This document also contains a discussion of\ninference issues, including concentration parameter learning; and a small-scale\nerror analysis of syntactic parsing accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.0573v1",
            "date": "2013-10-02 05:58:52+00:00",
            "title": "Improving the Quality of MT Output using Novel Name Entity Translation Scheme",
            "authors": [
                "Deepti Bhalla",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "This paper presents a novel approach to machine translation by combining the\nstate of art name entity translation scheme. Improper translation of name\nentities lapse the quality of machine translated output. In this work, name\nentities are transliterated by using statistical rule based approach. This\npaper describes the translation and transliteration of name entities from\nEnglish to Punjabi. We have experimented on four types of name entities which\nare: Proper names, Location names, Organization names and miscellaneous.\nVarious rules for the purpose of syllabification have been constructed.\nTransliteration of name entities is accomplished with the help of Probability\ncalculation. N-Gram probabilities for the extracted syllables have been\ncalculated using statistical machine translation toolkit MOSES.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.0575v2",
            "date": "2013-10-02 06:04:53+00:00",
            "title": "Development of Marathi Part of Speech Tagger Using Statistical Approach",
            "authors": [
                "Jyoti Singh",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Part-of-speech (POS) tagging is a process of assigning the words in a text\ncorresponding to a particular part of speech. A fundamental version of POS\ntagging is the identification of words as nouns, verbs, adjectives etc. For\nprocessing natural languages, Part of Speech tagging is a prominent tool. It is\none of the simplest as well as most constant and statistical model for many NLP\napplications. POS Tagging is an initial stage of linguistics, text analysis\nlike information retrieval, machine translator, text to speech synthesis,\ninformation extraction etc. In POS Tagging we assign a Part of Speech tag to\neach word in a sentence and literature. Various approaches have been proposed\nto implement POS taggers. In this paper we present a Marathi part of speech\ntagger. It is morphologically rich language. Marathi is spoken by the native\npeople of Maharashtra. The general approach used for development of tagger is\nstatistical using Unigram, Bigram, Trigram and HMM Methods. It presents a clear\nidea about all the algorithms with suitable examples. It also introduces a tag\nset for Marathi which can be used for tagging Marathi text. In this paper we\nhave shown the development of the tagger as well as compared to check the\naccuracy of taggers output. The three Marathi POS taggers viz. Unigram, Bigram,\nTrigram and HMM gives the accuracy of 77.38%, 90.30%, 91.46% and 93.82%\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.0578v1",
            "date": "2013-10-02 06:10:49+00:00",
            "title": "Subjective and Objective Evaluation of English to Urdu Machine Translation",
            "authors": [
                "Vaishali Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Machine translation is research based area where evaluation is very important\nphenomenon for checking the quality of MT output. The work is based on the\nevaluation of English to Urdu Machine translation. In this research work we\nhave evaluated the translation quality of Urdu language which has been\ntranslated by using different Machine Translation systems like Google, Babylon\nand Ijunoon. The evaluation process is done by using two approaches - Human\nevaluation and Automatic evaluation. We have worked for both the approaches\nwhere in human evaluation emphasis is given to scales and parameters while in\nautomatic evaluation emphasis is given to some automatic metric such as BLEU,\nGTM, METEOR and ATEC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.0581v1",
            "date": "2013-10-02 06:15:03+00:00",
            "title": "Rule Based Stemmer in Urdu",
            "authors": [
                "Vaishali Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Urdu is a combination of several languages like Arabic, Hindi, English,\nTurkish, Sanskrit etc. It has a complex and rich morphology. This is the reason\nwhy not much work has been done in Urdu language processing. Stemming is used\nto convert a word into its respective root form. In stemming, we separate the\nsuffix and prefix from the word. It is useful in search engines, natural\nlanguage processing and word processing, spell checkers, word parsing, word\nfrequency and count studies. This paper presents a rule based stemmer for Urdu.\nThe stemmer that we have discussed here is used in information retrieval. We\nhave also evaluated our results by verifying it with a human expert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.0754v1",
            "date": "2013-10-02 16:23:00+00:00",
            "title": "Stemmers for Tamil Language: Performance Analysis",
            "authors": [
                "M. Thangarasu",
                "R. Manavalan"
            ],
            "abstract": "Stemming is the process of extracting root word from the given inflection\nword and also plays significant role in numerous application of Natural\nLanguage Processing (NLP). Tamil Language raises several challenges to NLP,\nsince it has rich morphological patterns than other languages. The rule based\napproach light-stemmer is proposed in this paper, to find stem word for given\ninflection Tamil word. The performance of proposed approach is compared to a\nrule based suffix removal stemmer based on correctly and incorrectly predicted.\nThe experimental result clearly show that the proposed approach light stemmer\nfor Tamil language perform better than suffix removal stemmer and also more\neffective in Information Retrieval System (IRS).",
            "journal ref": "International Journal of Computer Science & Engineering\n  Technology, Vol. 4, No. 07, Jul 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1285v3",
            "date": "2013-10-04 14:21:42+00:00",
            "title": "Semantic Measures for the Comparison of Units of Language, Concepts or Instances from Text and Knowledge Base Analysis",
            "authors": [
                "S\u00e9bastien Harispe",
                "Sylvie Ranwez",
                "Stefan Janaqi",
                "Jacky Montmain"
            ],
            "abstract": "Semantic measures are widely used today to estimate the strength of the\nsemantic relationship between elements of various types: units of language\n(e.g., words, sentences, documents), concepts or even instances semantically\ncharacterized (e.g., diseases, genes, geographical locations). Semantic\nmeasures play an important role to compare such elements according to semantic\nproxies: texts and knowledge representations, which support their meaning or\ndescribe their nature. Semantic measures are therefore essential for designing\nintelligent agents which will for example take advantage of semantic analysis\nto mimic human ability to compare abstract or concrete objects. This paper\nproposes a comprehensive survey of the broad notion of semantic measure for the\ncomparison of units of language, concepts or instances based on semantic proxy\nanalyses. Semantic measures generalize the well-known notions of semantic\nsimilarity, semantic relatedness and semantic distance, which have been\nextensively studied by various communities over the last decades (e.g.,\nCognitive Sciences, Linguistics, and Artificial Intelligence to mention a few).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1425v1",
            "date": "2013-10-05 00:33:46+00:00",
            "title": "A State of the Art of Word Sense Induction: A Way Towards Word Sense Disambiguation for Under-Resourced Languages",
            "authors": [
                "Mohammad Nasiruddin"
            ],
            "abstract": "Word Sense Disambiguation (WSD), the process of automatically identifying the\nmeaning of a polysemous word in a sentence, is a fundamental task in Natural\nLanguage Processing (NLP). Progress in this approach to WSD opens up many\npromising developments in the field of NLP and its applications. Indeed,\nimprovement over current performance levels could allow us to take a first step\ntowards natural language understanding. Due to the lack of lexical resources it\nis sometimes difficult to perform WSD for under-resourced languages. This paper\nis an investigation on how to initiate research in WSD for under-resourced\nlanguages by applying Word Sense Induction (WSI) and suggests some interesting\ntopics to focus on.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1426v1",
            "date": "2013-10-05 00:39:02+00:00",
            "title": "Local Feature or Mel Frequency Cepstral Coefficients - Which One is Better for MLN-Based Bangla Speech Recognition?",
            "authors": [
                "Foyzul Hassan",
                "Mohammed Rokibul Alam Kotwal",
                "Md. Mostafizur Rahman",
                "Mohammad Nasiruddin",
                "Md. Abdul Latif",
                "Mohammad Nurul Huda"
            ],
            "abstract": "This paper discusses the dominancy of local features (LFs), as input to the\nmultilayer neural network (MLN), extracted from a Bangla input speech over mel\nfrequency cepstral coefficients (MFCCs). Here, LF-based method comprises three\nstages: (i) LF extraction from input speech, (ii) phoneme probabilities\nextraction using MLN from LF and (iii) the hidden Markov model (HMM) based\nclassifier to obtain more accurate phoneme strings. In the experiments on\nBangla speech corpus prepared by us, it is observed that the LFbased automatic\nspeech recognition (ASR) system provides higher phoneme correct rate than the\nMFCC-based system. Moreover, the proposed system requires fewer mixture\ncomponents in the HMMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1590v1",
            "date": "2013-10-06 14:37:05+00:00",
            "title": "Evolution of the Modern Phase of Written Bangla: A Statistical Study",
            "authors": [
                "Paheli Bhattacharya",
                "Arnab Bhattacharya"
            ],
            "abstract": "Active languages such as Bangla (or Bengali) evolve over time due to a\nvariety of social, cultural, economic, and political issues. In this paper, we\nanalyze the change in the written form of the modern phase of Bangla\nquantitatively in terms of character-level, syllable-level, morpheme-level and\nword-level features. We collect three different types of corpora---classical,\nnewspapers and blogs---and test whether the differences in their features are\nstatistically significant. Results suggest that there are significant changes\nin the length of a word when measured in terms of characters, but there is not\nmuch difference in usage of different characters, syllables and morphemes in a\nword or of different words in a sentence. To the best of our knowledge, this is\nthe first work on Bangla of this kind.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1964v1",
            "date": "2013-10-07 22:08:18+00:00",
            "title": "Named entity recognition using conditional random fields with non-local relational constraints",
            "authors": [
                "Flavio Massimiliano Cecchini",
                "Elisabetta Fersini"
            ],
            "abstract": "We begin by introducing the Computer Science branch of Natural Language\nProcessing, then narrowing the attention on its subbranch of Information\nExtraction and particularly on Named Entity Recognition, discussing briefly its\nmain methodological approaches. It follows an introduction to state-of-the-art\nConditional Random Fields under the form of linear chains. Subsequently, the\nidea of constrained inference as a way to model long-distance relationships in\na text is presented, based on an Integer Linear Programming representation of\nthe problem. Adding such relationships to the problem as automatically inferred\nlogical formulas, translatable into linear conditions, we propose to solve the\nresulting more complex problem with the aid of Lagrangian relaxation, of which\nsome technical details are explained. Lastly, we give some experimental\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1975v1",
            "date": "2013-10-08 00:30:51+00:00",
            "title": "ARKref: a rule-based coreference resolution system",
            "authors": [
                "Brendan O'Connor",
                "Michael Heilman"
            ],
            "abstract": "ARKref is a tool for noun phrase coreference. It is a deterministic,\nrule-based system that uses syntactic information from a constituent parser,\nand semantic information from an entity recognition component. Its architecture\nis based on the work of Haghighi and Klein (2009). ARKref was originally\nwritten in 2009. At the time of writing, the last released version was in March\n2011. This document describes that version, which is open-source and publicly\navailable at: http://www.ark.cs.cmu.edu/ARKref",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.8059v1",
            "date": "2013-10-30 08:08:43+00:00",
            "title": "Description and Evaluation of Semantic Similarity Measures Approaches",
            "authors": [
                "Thabet Slimani"
            ],
            "abstract": "In recent years, semantic similarity measure has a great interest in Semantic\nWeb and Natural Language Processing (NLP). Several similarity measures have\nbeen developed, being given the existence of a structured knowledge\nrepresentation offered by ontologies and corpus which enable semantic\ninterpretation of terms. Semantic similarity measures compute the similarity\nbetween concepts/terms included in knowledge sources in order to perform\nestimations. This paper discusses the existing semantic similarity methods\nbased on structure, information content and feature approaches. Additionally,\nwe present a critical evaluation of several categories of semantic similarity\napproaches based on two standard benchmarks. The aim of this paper is to give\nan efficient evaluation of all these measures which help researcher and\npractitioners to select the measure that best fit for their requirements.",
            "journal ref": "International Journal of Computer Applications 80(10):25-33,\n  October 2013. Published by Foundation of Computer Science, New York, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.1612v1",
            "date": "2013-02-06 23:24:37+00:00",
            "title": "Arabic text summarization based on latent semantic analysis to enhance arabic documents clustering",
            "authors": [
                "Hanane Froud",
                "Abdelmonaime Lachkar",
                "Said Alaoui Ouatik"
            ],
            "abstract": "Arabic Documents Clustering is an important task for obtaining good results\nwith the traditional Information Retrieval (IR) systems especially with the\nrapid growth of the number of online documents present in Arabic language.\nDocuments clustering aim to automatically group similar documents in one\ncluster using different similarity/distance measures. This task is often\naffected by the documents length, useful information on the documents is often\naccompanied by a large amount of noise, and therefore it is necessary to\neliminate this noise while keeping useful information to boost the performance\nof Documents clustering. In this paper, we propose to evaluate the impact of\ntext summarization using the Latent Semantic Analysis Model on Arabic Documents\nClustering in order to solve problems cited above, using five\nsimilarity/distance measures: Euclidean Distance, Cosine Similarity, Jaccard\nCoefficient, Pearson Correlation Coefficient and Averaged Kullback-Leibler\nDivergence, for two times: without and with stemming. Our experimental results\nindicate that our proposed approach effectively solves the problems of noisy\ninformation and documents length, and thus significantly improve the clustering\nperformance.",
            "journal ref": "International Journal of Data Mining & Knowledge Management\n  Process (IJDKP)- 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1302.4619v1",
            "date": "2013-02-19 14:32:17+00:00",
            "title": "Compactified Horizontal Visibility Graph for the Language Network",
            "authors": [
                "D. V. Lande",
                "A. A. Snarskii"
            ],
            "abstract": "A compactified horizontal visibility graph for the language network is\nproposed. It was found that the networks constructed in such way are scale\nfree, and have a property that among the nodes with largest degrees there are\nwords that determine not only a text structure communication, but also its\ninformational structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.4874v1",
            "date": "2013-02-20 11:06:25+00:00",
            "title": "A Labeled Graph Kernel for Relationship Extraction",
            "authors": [
                "Gon\u00e7alo Sim\u00f5es",
                "Helena Galhardas",
                "David Matos"
            ],
            "abstract": "In this paper, we propose an approach for Relationship Extraction (RE) based\non labeled graph kernels. The kernel we propose is a particularization of a\nrandom walk kernel that exploits two properties previously studied in the RE\nliterature: (i) the words between the candidate entities or connecting them in\na syntactic representation are particularly likely to carry information\nregarding the relationship; and (ii) combining information from distinct\nsources in a kernel may help the RE system make better decisions. We performed\nexperiments on a dataset of protein-protein interactions and the results show\nthat our approach obtains effectiveness values that are comparable with the\nstate-of-the art kernel methods. Moreover, our approach is able to outperform\nthe state-of-the-art kernels when combined with other kernel methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.5181v1",
            "date": "2013-02-21 04:58:48+00:00",
            "title": "Basic Classes of Grammars with Prohibition",
            "authors": [
                "Mark Burgin"
            ],
            "abstract": "A practical tool for natural language modeling and development of\nhuman-machine interaction is developed in the context of formal grammars and\nlanguages. A new type of formal grammars, called grammars with prohibition, is\nintroduced. Grammars with prohibition provide more powerful tools for natural\nlanguage generation and better describe processes of language learning than the\nconventional formal grammars. Here we study relations between languages\ngenerated by different grammars with prohibition based on conventional types of\nformal grammars such as context-free or context sensitive grammars. Besides, we\ncompare languages generated by different grammars with prohibition and\nlanguages generated by conventional formal grammars. In particular, it is\ndemonstrated that they have essentially higher computational power and\nexpressive possibilities in comparison with the conventional formal grammars.\nThus, while conventional formal grammars are recursive and subrecursive\nalgorithms, many classes of grammars with prohibition are superrecursive\nalgorithms. Results presented in this work are aimed at the development of\nhuman-machine interaction, modeling natural languages, empowerment of\nprogramming languages, computer simulation, better software systems, and theory\nof recursion.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1302.5675v1",
            "date": "2013-02-22 18:59:06+00:00",
            "title": "Development of Yes/No Arabic Question Answering System",
            "authors": [
                "Wafa N. Bdour",
                "Natheer K. Gharaibeh"
            ],
            "abstract": "Developing Question Answering systems has been one of the important research\nissues because it requires insights from a variety of\ndisciplines,including,Artificial Intelligence,Information Retrieval,\nInformation Extraction,Natural Language Processing, and Psychology.In this\npaper we realize a formal model for a lightweight semantic based open domain\nyes/no Arabic question answering system based on paragraph retrieval with\nvariable length. We propose a constrained semantic representation. Using an\nexplicit unification framework based on semantic similarities and query\nexpansion synonyms and antonyms.This frequently improves the precision of the\nsystem. Employing the passage retrieval system achieves a better precision by\nretrieving more paragraphs that contain relevant answers to the question; It\nsignificantly reduces the amount of text to be processed by the system.",
            "journal ref": "International Journal of Artificial Intelligence & Applications\n  (IJAIA), Vol.4, No.1, January 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.0658v1",
            "date": "2013-08-03 04:20:21+00:00",
            "title": "Exploring The Contribution of Unlabeled Data in Financial Sentiment Analysis",
            "authors": [
                "Jimmy SJ. Ren",
                "Wei Wang",
                "Jiawei Wang",
                "Stephen Shaoyi Liao"
            ],
            "abstract": "With the proliferation of its applications in various industries, sentiment\nanalysis by using publicly available web data has become an active research\narea in text classification during these years. It is argued by researchers\nthat semi-supervised learning is an effective approach to this problem since it\nis capable to mitigate the manual labeling effort which is usually expensive\nand time-consuming. However, there was a long-term debate on the effectiveness\nof unlabeled data in text classification. This was partially caused by the fact\nthat many assumptions in theoretic analysis often do not hold in practice. We\nargue that this problem may be further understood by adding an additional\ndimension in the experiment. This allows us to address this problem in the\nperspective of bias and variance in a broader view. We show that the well-known\nperformance degradation issue caused by unlabeled data can be reproduced as a\nsubset of the whole scenario. We argue that if the bias-variance trade-off is\nto be better balanced by a more effective feature selection method unlabeled\ndata is very likely to boost the classification performance. We then propose a\nfeature selection framework in which labeled and unlabeled training samples are\nboth considered. We discuss its potential in achieving such a balance. Besides,\nthe application in financial sentiment analysis is chosen because it not only\nexemplifies an important application, the data possesses better illustrative\npower as well. The implications of this study in text classification and\nfinancial sentiment analysis are both discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.0661v1",
            "date": "2013-08-03 05:57:48+00:00",
            "title": "A Comparison of Named Entity Recognition Tools Applied to Biographical Texts",
            "authors": [
                "Samet Atda\u011f",
                "Vincent Labatut"
            ],
            "abstract": "Named entity recognition (NER) is a popular domain of natural language\nprocessing. For this reason, many tools exist to perform this task. Amongst\nother points, they differ in the processing method they rely upon, the entity\ntypes they can detect, the nature of the text they can handle, and their\ninput/output formats. This makes it difficult for a user to select an\nappropriate NER tool for a specific situation. In this article, we try to\nanswer this question in the context of biographic texts. For this matter, we\nfirst constitute a new corpus by annotating Wikipedia articles. We then select\npublicly available, well known and free for research NER tools for comparison:\nStanford NER, Illinois NET, OpenCalais NER WS and Alias-i LingPipe. We apply\nthem to our corpus, assess their performances and compare them. When\nconsidering overall performances, a clear hierarchy emerges: Stanford has the\nbest results, followed by LingPipe, Illionois and OpenCalais. However, a more\ndetailed evaluation performed relatively to entity types and article categories\nhighlights the fact their performances are diversely influenced by those\nfactors. This complementarity opens an interesting perspective regarding the\ncombination of these individual tools in order to improve performance.",
            "journal ref": "2nd International Conference on Systems and Computer Science,\n  Villeneuve d'Ascq (FR), 228-233, 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1308.0701v1",
            "date": "2013-08-03 14:30:55+00:00",
            "title": "Ontology Enrichment by Extracting Hidden Assertional Knowledge from Text",
            "authors": [
                "Meisam Booshehri",
                "Abbas Malekpour",
                "Peter Luksch",
                "Kamran Zamanifar",
                "Shahdad Shariatmadari"
            ],
            "abstract": "In this position paper we present a new approach for discovering some special\nclasses of assertional knowledge in the text by using large RDF repositories,\nresulting in the extraction of new non-taxonomic ontological relations. Also we\nuse inductive reasoning beside our approach to make it outperform. Then, we\nprepare a case study by applying our approach on sample data and illustrate the\nsoundness of our proposed approach. Moreover in our point of view current LOD\ncloud is not a suitable base for our proposal in all informational domains.\nTherefore we figure out some directions based on prior works to enrich datasets\nof Linked Data by using web mining. The result of such enrichment can be reused\nfor further relation extraction and ontology enrichment from unstructured free\ntext documents.",
            "journal ref": "IJCSIS, 11(5), 64-72",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1308.0850v5",
            "date": "2013-08-04 21:04:36+00:00",
            "title": "Generating Sequences With Recurrent Neural Networks",
            "authors": [
                "Alex Graves"
            ],
            "abstract": "This paper shows how Long Short-term Memory recurrent neural networks can be\nused to generate complex sequences with long-range structure, simply by\npredicting one data point at a time. The approach is demonstrated for text\n(where the data are discrete) and online handwriting (where the data are\nreal-valued). It is then extended to handwriting synthesis by allowing the\nnetwork to condition its predictions on a text sequence. The resulting system\nis able to generate highly realistic cursive handwriting in a wide variety of\nstyles.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1308.0897v1",
            "date": "2013-08-05 07:22:07+00:00",
            "title": "Context Specific Event Model For News Articles",
            "authors": [
                "Kowcika A",
                "Uma Maheswari",
                "Geetha T V"
            ],
            "abstract": "We present a new context based event indexing and event ranking model for\nNews Articles. The context event clusters formed from the UNL Graphs uses the\nmodified scoring scheme for segmenting events which is followed by clustering\nof events. From the context clusters obtained three models are developed-\nIdentification of Main and Sub events; Event Indexing and Event Ranking. Based\non the properties considered from the UNL Graphs for the modified scoring main\nevents and sub events associated with main-events are identified. The temporal\ndetails obtained from the context cluster are stored using hashmap data\nstructure. The temporal details are place-where the event took; person-who\ninvolved in that event; time-when the event took place. Based on the\ninformation collected from the context clusters three indices are generated-\nTime index, Person index, and Place index. This index gives complete details\nabout every event obtained from context clusters. A new scoring scheme is\nintroduced for ranking the events. The scoring scheme for event ranking gives\nweight-age based on the priority level of the events. The priority level\nincludes the occurrence of the event in the title of the document, event\nfrequency, and inverse document frequency of the events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.3106v1",
            "date": "2013-08-14 12:54:31+00:00",
            "title": "System and Methods for Converting Speech to SQL",
            "authors": [
                "Sachin Kumar",
                "Ashish Kumar",
                "Pinaki Mitra",
                "Girish Sundaram"
            ],
            "abstract": "This paper concerns with the conversion of a Spoken English Language Query\ninto SQL for retrieving data from RDBMS. A User submits a query as speech\nsignal through the user interface and gets the result of the query in the text\nformat. We have developed the acoustic and language models using which a speech\nutterance can be converted into English text query and thus natural language\nprocessing techniques can be applied on this English text query to generate an\nequivalent SQL query. For conversion of speech into English text HTK and Julius\ntools have been used and for conversion of English text query into SQL query we\nhave implemented a System which uses rule based translation to translate\nEnglish Language Query into SQL Query. The translation uses lexical analyzer,\nparser and syntax directed translation techniques like in compilers. JFLex and\nBYACC tools have been used to build lexical analyzer and parser respectively.\nSystem is domain independent i.e. system can run on different database as it\ngenerates lex files from the underlying database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.3785v1",
            "date": "2013-08-17 14:04:00+00:00",
            "title": "Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech Recognition",
            "authors": [
                "Md. Ali Hossain",
                "Md. Mijanur Rahman",
                "Uzzal Kumar Prodhan",
                "Md. Farukuzzaman Khan"
            ],
            "abstract": "This paper is concerned with the development of Back-propagation Neural\nNetwork for Bangla Speech Recognition. In this paper, ten bangla digits were\nrecorded from ten speakers and have been recognized. The features of these\nspeech digits were extracted by the method of Mel Frequency Cepstral\nCoefficient (MFCC) analysis. The mfcc features of five speakers were used to\ntrain the network with Back propagation algorithm. The mfcc features of ten\nbangla digit speeches, from 0 to 9, of another five speakers were used to test\nthe system. All the methods and algorithms used in this research were\nimplemented using the features of Turbo C and C++ languages. From our\ninvestigation it is seen that the developed system can successfully encode and\nanalyze the mfcc features of the speech signal to recognition. The developed\nsystem achieved recognition rate about 96.332% for known speakers (i.e.,\nspeaker dependent) and 92% for unknown speakers (i.e., speaker independent).",
            "journal ref": "International Journal of Information Sciences and Techniques\n  (IJIST) Vol.3, No.4, July 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.4648v3",
            "date": "2013-08-21 18:01:42+00:00",
            "title": "PACE: Pattern Accurate Computationally Efficient Bootstrapping for Timely Discovery of Cyber-Security Concepts",
            "authors": [
                "Nikki McNeil",
                "Robert A. Bridges",
                "Michael D. Iannacone",
                "Bogdan Czejdo",
                "Nicolas Perez",
                "John R. Goodall"
            ],
            "abstract": "Public disclosure of important security information, such as knowledge of\nvulnerabilities or exploits, often occurs in blogs, tweets, mailing lists, and\nother online sources months before proper classification into structured\ndatabases. In order to facilitate timely discovery of such knowledge, we\npropose a novel semi-supervised learning algorithm, PACE, for identifying and\nclassifying relevant entities in text sources. The main contribution of this\npaper is an enhancement of the traditional bootstrapping method for entity\nextraction by employing a time-memory trade-off that simultaneously circumvents\na costly corpus search while strengthening pattern nomination, which should\nincrease accuracy. An implementation in the cyber-security domain is discussed\nas well as challenges to Natural Language Processing imposed by the security\ndomain.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1308.4941v3",
            "date": "2013-08-22 18:23:25+00:00",
            "title": "Automatic Labeling for Entity Extraction in Cyber Security",
            "authors": [
                "Robert A. Bridges",
                "Corinne L. Jones",
                "Michael D. Iannacone",
                "Kelly M. Testa",
                "John R. Goodall"
            ],
            "abstract": "Timely analysis of cyber-security information necessitates automated\ninformation extraction from unstructured text. While state-of-the-art\nextraction methods produce extremely accurate results, they require ample\ntraining data, which is generally unavailable for specialized applications,\nsuch as detecting security related entities; moreover, manual annotation of\ncorpora is very costly and often not a viable solution. In response, we develop\na very precise method to automatically label text from several data sources by\nleveraging related, domain-specific, structured data and provide public access\nto a corpus annotated with cyber-security entities. Next, we implement a\nMaximum Entropy Model trained with the average perceptron on a portion of our\ncorpus ($\\sim$750,000 words) and achieve near perfect precision, recall, and\naccuracy, with training times under 17 seconds.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1308.4965v1",
            "date": "2013-08-21 17:53:37+00:00",
            "title": "A proposal for a Chinese keyboard for cellphones, smartphones, ipads and tablets",
            "authors": [
                "Maurice Margenstern",
                "Lan Wu"
            ],
            "abstract": "In this paper, we investigate the possibility to use two tilings of the\nhyperbolic plane as basic frame for devising a way to input texts in Chinese\ncharacters into messages of cellphones, smartphones, ipads and tablets.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1309.0326v3",
            "date": "2013-09-02 09:09:27+00:00",
            "title": "Tagging Scientific Publications using Wikipedia and Natural Language Processing Tools. Comparison on the ArXiv Dataset",
            "authors": [
                "Micha\u0142 \u0141opuszy\u0144ski",
                "\u0141ukasz Bolikowski"
            ],
            "abstract": "In this work, we compare two simple methods of tagging scientific\npublications with labels reflecting their content. As a first source of labels\nWikipedia is employed, second label set is constructed from the noun phrases\noccurring in the analyzed corpus. We examine the statistical properties and the\neffectiveness of both approaches on the dataset consisting of abstracts from\n0.7 million of scientific documents deposited in the ArXiv preprint collection.\nWe believe that obtained tags can be later on applied as useful document\nfeatures in various machine learning tasks (document similarity, clustering,\ntopic modelling, etc.).",
            "journal ref": "Communications in Computer and Information Science Volume 416,\n  Springer 2014, pp 16-27",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.1536v2",
            "date": "2013-09-06 04:48:19+00:00",
            "title": "Rank-frequency relation for Chinese characters",
            "authors": [
                "W. B. Deng",
                "A. E. Allahverdyan",
                "B. Li",
                "Q. A. Wang"
            ],
            "abstract": "We show that the Zipf's law for Chinese characters perfectly holds for\nsufficiently short texts (few thousand different characters). The scenario of\nits validity is similar to the Zipf's law for words in short English texts. For\nlong Chinese texts (or for mixtures of short Chinese texts), rank-frequency\nrelations for Chinese characters display a two-layer, hierarchic structure that\ncombines a Zipfian power-law regime for frequent characters (first layer) with\nan exponential-like regime for less frequent characters (second layer). For\nthese two layers we provide different (though related) theoretical descriptions\nthat include the range of low-frequency characters (hapax legomena). The\ncomparative analysis of rank-frequency relations for Chinese characters versus\nEnglish words illustrates the extent to which the characters play for Chinese\nwriters the same role as the words for those writing within alphabetical\nsystems.",
            "journal ref": "Eur. Phys. J. B (2014) 87: 47",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.3323v2",
            "date": "2013-09-12 22:27:59+00:00",
            "title": "Mapping Mutable Genres in Structurally Complex Volumes",
            "authors": [
                "Ted Underwood",
                "Michael L. Black",
                "Loretta Auvil",
                "Boris Capitanu"
            ],
            "abstract": "To mine large digital libraries in humanistically meaningful ways, scholars\nneed to divide them by genre. This is a task that classification algorithms are\nwell suited to assist, but they need adjustment to address the specific\nchallenges of this domain. Digital libraries pose two problems of scale not\nusually found in the article datasets used to test these algorithms. 1) Because\nlibraries span several centuries, the genres being identified may change\ngradually across the time axis. 2) Because volumes are much longer than\narticles, they tend to be internally heterogeneous, and the classification task\nneeds to begin with segmentation. We describe a multi-layered solution that\ntrains hidden Markov models to segment volumes, and uses ensembles of\noverlapping classifiers to address historical change. We test this approach on\na collection of 469,200 volumes drawn from HathiTrust Digital Library. To\ndemonstrate the humanistic value of these methods, we extract 32,209 volumes of\nfiction from the digital library, and trace the changing proportions of first-\nand third-person narration in the corpus. We note that narrative points of view\nseem to have strong associations with particular themes and genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6047v1",
            "date": "2013-09-24 05:14:53+00:00",
            "title": "Non-negative Matrix Factorization with Linear Constraints for Single-Channel Speech Enhancement",
            "authors": [
                "Nikolay Lyubimov",
                "Mikhail Kotov"
            ],
            "abstract": "This paper investigates a non-negative matrix factorization (NMF)-based\napproach to the semi-supervised single-channel speech enhancement problem where\nonly non-stationary additive noise signals are given. The proposed method\nrelies on sinusoidal model of speech production which is integrated inside NMF\nframework using linear constraints on dictionary atoms. This method is further\ndeveloped to regularize harmonic amplitudes. Simple multiplicative algorithms\nare presented. The experimental evaluation was made on TIMIT corpus mixed with\nvarious types of noise. It has been shown that the proposed method outperforms\nsome of the state-of-the-art noise suppression techniques in terms of\nsignal-to-noise ratio.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1309.6650v1",
            "date": "2013-09-25 20:13:28+00:00",
            "title": "An Inter-lingual Reference Approach For Multi-Lingual Ontology Matching",
            "authors": [
                "Haytham Al-Feel",
                "Ralph Schafermeier",
                "Adrian Paschke"
            ],
            "abstract": "Ontologies are considered as the backbone of the Semantic Web. With the\nrising success of the Semantic Web, the number of participating communities\nfrom different countries is constantly increasing. The growing number of\nontologies available in different natural languages leads to an\ninteroperability problem. In this paper, we discuss several approaches for\nontology matching; examine similarities and differences, identify weaknesses,\nand compare the existing automated approaches with the manual approaches for\nintegrating multilingual ontologies. In addition to that, we propose a new\narchitecture for a multilingual ontology matching service. As a case study we\nused an example of two multilingual enterprise ontologies - the university\nontology of Freie Universitaet Berlin and the ontology for Fayoum University in\nEgypt.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 10,\n  Issue 2, No 1, March 2013 ISSN (Print): 1694-0814 | ISSN (Online): 1694-0784",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.7270v1",
            "date": "2013-09-27 15:14:34+00:00",
            "title": "Evaluating the Usefulness of Sentiment Information for Focused Crawlers",
            "authors": [
                "Tianjun Fu",
                "Ahmed Abbasi",
                "Daniel Zeng",
                "Hsinchun Chen"
            ],
            "abstract": "Despite the prevalence of sentiment-related content on the Web, there has\nbeen limited work on focused crawlers capable of effectively collecting such\ncontent. In this study, we evaluated the efficacy of using sentiment-related\ninformation for enhanced focused crawling of opinion-rich web content regarding\na particular topic. We also assessed the impact of using sentiment-labeled web\ngraphs to further improve collection accuracy. Experimental results on a large\ntest bed encompassing over half a million web pages revealed that focused\ncrawlers utilizing sentiment information as well as sentiment-labeled web\ngraphs are capable of gathering more holistic collections of opinion-related\ncontent regarding a particular topic. The results have important implications\nfor business and marketing intelligence gathering efforts in the Web 2.0 era.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1309.7340v3",
            "date": "2013-09-27 19:47:11+00:00",
            "title": "Early Stage Influenza Detection from Twitter",
            "authors": [
                "Jiwei Li",
                "Claire Cardie"
            ],
            "abstract": "Influenza is an acute respiratory illness that occurs virtually every year\nand results in substantial disease, death and expense. Detection of Influenza\nin its earliest stage would facilitate timely action that could reduce the\nspread of the illness. Existing systems such as CDC and EISS which try to\ncollect diagnosis data, are almost entirely manual, resulting in about two-week\ndelays for clinical data acquisition. Twitter, a popular microblogging service,\nprovides us with a perfect source for early-stage flu detection due to its\nreal- time nature. For example, when a flu breaks out, people that get the flu\nmay post related tweets which enables the detection of the flu breakout\npromptly. In this paper, we investigate the real-time flu detection problem on\nTwitter data by proposing Flu Markov Network (Flu-MN): a spatio-temporal\nunsupervised Bayesian algorithm based on a 4 phase Markov Network, trying to\nidentify the flu breakout at the earliest stage. We test our model on real\nTwitter datasets from the United States along with baselines in multiple\napplications, such as real-time flu breakout detection, future epidemic phase\nprediction, or Influenza-like illness (ILI) physician visits. Experimental\nresults show the robustness and effectiveness of our approach. We build up a\nreal time flu reporting system based on the proposed approach, and we are\nhopeful that it would help government or health organizations in identifying\nflu outbreaks and facilitating timely actions to decrease unnecessary\nmortality.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1312.2244v3",
            "date": "2013-12-08 19:15:15+00:00",
            "title": "Time-dependent Hierarchical Dirichlet Model for Timeline Generation",
            "authors": [
                "Rumeng Li",
                "Tao Wang",
                "Xun Wang"
            ],
            "abstract": "Timeline Generation aims at summarizing news from different epochs and\ntelling readers how an event evolves. It is a new challenge that combines\nsalience ranking with novelty detection. For long-term public events, the main\ntopic usually includes various aspects across different epochs and each aspect\nhas its own evolving pattern. Existing approaches neglect such hierarchical\ntopic structure involved in the news corpus in timeline generation. In this\npaper, we develop a novel time-dependent Hierarchical Dirichlet Model (HDM) for\ntimeline generation. Our model can aptly detect different levels of topic\ninformation across corpus and such structure is further used for sentence\nselection. Based on the topic mined fro HDM, sentences are selected by\nconsidering different aspects such as relevance, coherence and coverage. We\ndevelop experimental systems to evaluate 8 long-term events that public\nconcern. Performance comparison between different systems demonstrates the\neffectiveness of our model in terms of ROUGE metrics.",
            "journal ref": "SDM(2015)p550-558",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.4092v1",
            "date": "2013-12-14 21:48:49+00:00",
            "title": "Domain adaptation for sequence labeling using hidden Markov models",
            "authors": [
                "Edouard Grave",
                "Guillaume Obozinski",
                "Francis Bach"
            ],
            "abstract": "Most natural language processing systems based on machine learning are not\nrobust to domain shift. For example, a state-of-the-art syntactic dependency\nparser trained on Wall Street Journal sentences has an absolute drop in\nperformance of more than ten points when tested on textual data from the Web.\nAn efficient solution to make these methods more robust to domain shift is to\nfirst learn a word representation using large amounts of unlabeled data from\nboth domains, and then use this representation as features in a supervised\nlearning algorithm. In this paper, we propose to use hidden Markov models to\nlearn word representations for part-of-speech tagging. In particular, we study\nthe influence of using data from the source, the target or both domains to\nlearn the representation and the different ways to represent words using an\nHMM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.4617v2",
            "date": "2013-12-17 02:53:38+00:00",
            "title": "A Survey of Data Mining Techniques for Social Media Analysis",
            "authors": [
                "Mariam Adedoyin-Olowe",
                "Mohamed Medhat Gaber",
                "Frederic Stahl"
            ],
            "abstract": "Social network has gained remarkable attention in the last decade. Accessing\nsocial network sites such as Twitter, Facebook LinkedIn and Google+ through the\ninternet and the web 2.0 technologies has become more affordable. People are\nbecoming more interested in and relying on social network for information, news\nand opinion of other users on diverse subject matters. The heavy reliance on\nsocial network sites causes them to generate massive data characterised by\nthree computational issues namely; size, noise and dynamism. These issues often\nmake social network data very complex to analyse manually, resulting in the\npertinent use of computational means of analysing them. Data mining provides a\nwide range of techniques for detecting useful knowledge from massive datasets\nlike trends, patterns and rules [44]. Data mining techniques are used for\ninformation retrieval, statistical modelling and machine learning. These\ntechniques employ data pre-processing, data analysis, and data interpretation\nprocesses in the course of data analysis. This survey discusses different data\nmining techniques used in mining diverse aspects of the social network over\ndecades going from the historical techniques to the up-to-date models,\nincluding our novel technique named TRCM. All the techniques covered in this\nsurvey are listed in the Table.1 including the tools employed as well as names\nof their authors.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2014 (June 24, 2014)\n  jdmdh:18",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1312.4706v1",
            "date": "2013-12-17 10:18:44+00:00",
            "title": "Designing Spontaneous Speech Search Interface for Historical Archives",
            "authors": [
                "Donna Vakharia",
                "Rachel Gibbs"
            ],
            "abstract": "Spontaneous speech in the form of conversations, meetings, voice-mail,\ninterviews, oral history, etc. is one of the most ubiquitous forms of human\ncommunication. Search engines providing access to such speech collections have\nthe potential to better inform intelligence and make relevant data over vast\naudio/video archives available to users. This project presents a search user\ninterface design supporting search tasks over a speech collection consisting of\nan historical archive with nearly 52,000 audiovisual testimonies of survivors\nand witnesses of the Holocaust and other genocides. The design incorporates\nfaceted search, along with other UI elements like highlighted search items,\ntags, snippets, etc., to promote discovery and exploratory search. Two\ndifferent designs have been created to support both manual and automated\ntranscripts. Evaluation was performed using human subjects to measure accuracy\nin retrieving results, understanding user-perspective on the design elements,\nand ease of parsing information.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1312.4824v2",
            "date": "2013-12-17 15:32:56+00:00",
            "title": "Generation, Implementation and Appraisal of an N-gram based Stemming Algorithm",
            "authors": [
                "B. P. Pande",
                "Pawan Tamta",
                "H. S. Dhami"
            ],
            "abstract": "A language independent stemmer has always been looked for. Single N-gram\ntokenization technique works well, however, it often generates stems that start\nwith intermediate characters, rather than initial ones. We present a novel\ntechnique that takes the concept of N gram stemming one step ahead and compare\nour method with an established algorithm in the field, Porter's Stemmer.\nResults indicate that our N gram stemmer is not inferior to Porter's linguistic\nstemmer.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1312.5542v3",
            "date": "2013-12-19 13:31:11+00:00",
            "title": "Word Emdeddings through Hellinger PCA",
            "authors": [
                "R\u00e9mi Lebret",
                "Ronan Collobert"
            ],
            "abstract": "Word embeddings resulting from neural language models have been shown to be\nsuccessful for a large variety of NLP tasks. However, such architecture might\nbe difficult to train and time-consuming. Instead, we propose to drastically\nsimplify the word embeddings computation through a Hellinger PCA of the word\nco-occurence matrix. We compare those new word embeddings with some well-known\nembeddings on NER and movie review tasks and show that we can reach similar or\neven better performance. Although deep learning is not really necessary for\ngenerating good word embeddings, we show that it can provide an easy way to\nadapt embeddings to specific tasks.",
            "journal ref": "Conference of the European Chapter of the Association for\n  Computational Linguistics (EACL), 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.5985v2",
            "date": "2013-12-20 15:21:15+00:00",
            "title": "Learning Type-Driven Tensor-Based Meaning Representations",
            "authors": [
                "Tamara Polajnar",
                "Luana Fagarasan",
                "Stephen Clark"
            ],
            "abstract": "This paper investigates the learning of 3rd-order tensors representing the\nsemantics of transitive verbs. The meaning representations are part of a\ntype-driven tensor-based semantic framework, from the newly emerging field of\ncompositional distributional semantics. Standard techniques from the neural\nnetworks literature are used to learn the tensors, which are tested on a\nselectional preference-style task with a simple 2-dimensional sentence space.\nPromising results are obtained against a competitive corpus-based baseline. We\nargue that extending this work beyond transitive verbs, and to\nhigher-dimensional sentence spaces, is an interesting and challenging problem\nfor the machine learning community to consider.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.6168v3",
            "date": "2013-12-20 22:44:26+00:00",
            "title": "Factorial Hidden Markov Models for Learning Representations of Natural Language",
            "authors": [
                "Anjan Nepal",
                "Alexander Yates"
            ],
            "abstract": "Most representation learning algorithms for language and image processing are\nlocal, in that they identify features for a data point based on surrounding\npoints. Yet in language processing, the correct meaning of a word often depends\non its global context. As a step toward incorporating global context into\nrepresentation learning, we develop a representation learning algorithm that\nincorporates joint prediction into its technique for producing features for a\nword. We develop efficient variational methods for learning Factorial Hidden\nMarkov Models from large texts, and use variational distributions to produce\nfeatures for each word that are sensitive to the entire input sequence, not\njust to a local context window. Experiments on part-of-speech tagging and\nchunking indicate that the features are competitive with or better than\nexisting state-of-the-art representation learning methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.6192v4",
            "date": "2013-12-21 02:29:42+00:00",
            "title": "Can recursive neural tensor networks learn logical reasoning?",
            "authors": [
                "Samuel R. Bowman"
            ],
            "abstract": "Recursive neural network models and their accompanying vector representations\nfor words have seen success in an array of increasingly semantically\nsophisticated tasks, but almost nothing is known about their ability to\naccurately capture the aspects of linguistic meaning that are necessary for\ninterpretation or reasoning. To evaluate this, I train a recursive model on a\nnew corpus of constructed examples of logical reasoning in short sentences,\nlike the inference of \"some animal walks\" from \"some dog walks\" or \"some cat\nwalks,\" given that dogs and cats are animals. This model learns representations\nthat generalize well to new types of reasoning pattern in all but a few cases,\na result which is promising for the ability of learned representation models to\ncapture logical reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.6802v1",
            "date": "2013-12-24 12:06:48+00:00",
            "title": "Suffix Stripping Problem as an Optimization Problem",
            "authors": [
                "B. P. Pande",
                "Pawan Tamta",
                "H. S. Dhami"
            ],
            "abstract": "Stemming or suffix stripping, an important part of the modern Information\nRetrieval systems, is to find the root word (stem) out of a given cluster of\nwords. Existing algorithms targeting this problem have been developed in a\nhaphazard manner. In this work, we model this problem as an optimization\nproblem. An Integer Program is being developed to overcome the shortcomings of\nthe existing approaches. The sample results of the proposed method are also\nbeing compared with an established technique in the field for English language.\nAn AMPL code for the same IP has also been given.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1312.6947v2",
            "date": "2013-12-25 09:17:28+00:00",
            "title": "Formal Ontology Learning on Factual IS-A Corpus in English using Description Logics",
            "authors": [
                "Sourish Dasgupta",
                "Ankur Padia",
                "Kushal Shah",
                "Prasenjit Majumder"
            ],
            "abstract": "Ontology Learning (OL) is the computational task of generating a knowledge\nbase in the form of an ontology given an unstructured corpus whose content is\nin natural language (NL). Several works can be found in this area most of which\nare limited to statistical and lexico-syntactic pattern matching based\ntechniques Light-Weight OL. These techniques do not lead to very accurate\nlearning mostly because of several linguistic nuances in NL. Formal OL is an\nalternative (less explored) methodology were deep linguistics analysis is made\nusing theory and tools found in computational linguistics to generate formal\naxioms and definitions instead simply inducing a taxonomy. In this paper we\npropose \"Description Logic (DL)\" based formal OL framework for learning factual\nIS-A type sentences in English. We claim that semantic construction of IS-A\nsentences is non trivial. Hence, we also claim that such sentences requires\nspecial studies in the context of OL before any truly formal OL can be\nproposed. We introduce a learner tool, called DLOL_IS-A, that generated such\nontologies in the owl format. We have adopted \"Gold Standard\" based OL\nevaluation on IS-A rich WCL v.1.1 dataset and our own Community representative\nIS-A dataset. We observed significant improvement of DLOL_IS-A when compared to\nthe light-weight OL tool Text2Onto and formal OL tool FRED.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.6948v1",
            "date": "2013-12-25 09:23:49+00:00",
            "title": "Description Logics based Formalization of Wh-Queries",
            "authors": [
                "Sourish Dasgupta",
                "Rupali KaPatel",
                "Ankur Padia",
                "Kushal Shah"
            ],
            "abstract": "The problem of Natural Language Query Formalization (NLQF) is to translate a\ngiven user query in natural language (NL) into a formal language so that the\nsemantic interpretation has equivalence with the NL interpretation.\nFormalization of NL queries enables logic based reasoning during information\nretrieval, database query, question-answering, etc. Formalization also helps in\nWeb query normalization and indexing, query intent analysis, etc. In this paper\nwe are proposing a Description Logics based formal methodology for wh-query\nintent (also called desire) identification and corresponding formal\ntranslation. We evaluated the scalability of our proposed formalism using\nMicrosoft Encarta 98 query dataset and OWL-S TC v.4.0 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.0509v3",
            "date": "2013-12-20 17:08:26+00:00",
            "title": "Zero-Shot Learning for Semantic Utterance Classification",
            "authors": [
                "Yann N. Dauphin",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Larry Heck"
            ],
            "abstract": "We propose a novel zero-shot learning method for semantic utterance\nclassification (SUC). It learns a classifier $f: X \\to Y$ for problems where\nnone of the semantic categories $Y$ are present in the training set. The\nframework uncovers the link between categories and utterances using a semantic\nspace. We show that this semantic space can be learned by deep neural networks\ntrained on large amounts of search engine query log data. More precisely, we\npropose a novel method that can learn discriminative semantic features without\nsupervision. It uses the zero-shot learning framework to guide the learning of\nthe semantic features. We demonstrate the effectiveness of the zero-shot\nsemantic learning algorithm on the SUC dataset collected by (Tur, 2012).\nFurthermore, we achieve state-of-the-art results by combining the semantic\nfeatures with a supervised method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3230v1",
            "date": "2013-10-08 13:48:02+00:00",
            "title": "Optimization Of Cross Domain Sentiment Analysis Using Sentiwordnet",
            "authors": [
                "K Paramesha",
                "K C Ravishankar"
            ],
            "abstract": "The task of sentiment analysis of reviews is carried out using manually built\n/ automatically generated lexicon resources of their own with which terms are\nmatched with lexicon to compute the term count for positive and negative\npolarity. On the other hand the Sentiwordnet, which is quite different from\nother lexicon resources that gives scores (weights) of the positive and\nnegative polarity for each word. The polarity of a word namely positive,\nnegative and neutral have the score ranging between 0 to 1 indicates the\nstrength/weight of the word with that sentiment orientation. In this paper, we\nshow that using the Sentiwordnet, how we could enhance the performance of the\nclassification at both sentence and document level.",
            "journal ref": "International Journal in Foundations of Computer Science &\n  Technology (IJFCST), Vol. 3, No.5, September 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.0722v2",
            "date": "2013-01-04 13:45:35+00:00",
            "title": "Good parts first - a new algorithm for approximate search in lexica and string databases",
            "authors": [
                "Stefan Gerdjikov",
                "Stoyan Mihov",
                "Petar Mitankin",
                "Klaus U. Schulz"
            ],
            "abstract": "We present a new efficient method for approximate search in electronic\nlexica. Given an input string (the pattern) and a similarity threshold, the\nalgorithm retrieves all entries of the lexicon that are sufficiently similar to\nthe pattern. Search is organized in subsearches that always start with an exact\npartial match where a substring of the input pattern is aligned with a\nsubstring of a lexicon word. Afterwards this partial match is extended stepwise\nto larger substrings. For aligning further parts of the pattern with\ncorresponding parts of lexicon entries, more errors are tolerated at each\nsubsequent step. For supporting this alignment order, which may start at any\npart of the pattern, the lexicon is represented as a structure that enables\nimmediate access to any substring of a lexicon word and permits the extension\nof such substrings in both directions. Experimental evaluations of the\napproximate search procedure are given that show significant efficiency\nimprovements compared to existing techniques. Since the technique can be used\nfor large error bounds it offers interesting possibilities for approximate\nsearch in special collections of \"long\" strings, such as phrases, sentences, or\nbook ti",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.1950v1",
            "date": "2013-01-09 19:17:31+00:00",
            "title": "Syntactic Analysis Based on Morphological Characteristic Features of the Romanian Language",
            "authors": [
                "Bogdan Patrut"
            ],
            "abstract": "This paper refers to the syntactic analysis of phrases in Romanian, as an\nimportant process of natural language processing. We will suggest a real-time\nsolution, based on the idea of using some words or groups of words that\nindicate grammatical category; and some specific endings of some parts of\nsentence. Our idea is based on some characteristics of the Romanian language,\nwhere some prepositions, adverbs or some specific endings can provide a lot of\ninformation about the structure of a complex sentence. Such characteristics can\nbe found in other languages, too, such as French. Using a special grammar, we\ndeveloped a system (DIASEXP) that can perform a dialogue in natural language\nwith assertive and interogative sentences about a \"story\" (a set of sentences\ndescribing some events from the real life).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.2405v1",
            "date": "2013-01-11 07:46:27+00:00",
            "title": "Dating medieval English charters",
            "authors": [
                "Gelila Tilahun",
                "Andrey Feuerverger",
                "Michael Gervers"
            ],
            "abstract": "Deeds, or charters, dealing with property rights, provide a continuous\ndocumentation which can be used by historians to study the evolution of social,\neconomic and political changes. This study is concerned with charters (written\nin Latin) dating from the tenth through early fourteenth centuries in England.\nOf these, at least one million were left undated, largely due to administrative\nchanges introduced by William the Conqueror in 1066. Correctly dating such\ncharters is of vital importance in the study of English medieval history. This\npaper is concerned with computer-automated statistical methods for dating such\ndocument collections, with the goal of reducing the considerable efforts\nrequired to date them manually and of improving the accuracy of assigned dates.\nProposed methods are based on such data as the variation over time of word and\nphrase usage, and on measures of distance between documents. The extensive (and\ndated) Documents of Early England Data Set (DEEDS) maintained at the University\nof Toronto was used for this purpose.",
            "journal ref": "Annals of Applied Statistics 2012, Vol. 6, No. 4, 1615-1640",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1301.2466v1",
            "date": "2013-01-11 12:02:23+00:00",
            "title": "Determining token sequence mistakes in responses to questions with open text answer",
            "authors": [
                "Oleg Sychev",
                "Dmitry Mamontov"
            ],
            "abstract": "When learning grammar of the new language, a teacher should routinely check\nstudent's exercises for grammatical correctness. The paper describes a method\nof automatically detecting and reporting grammar mistakes, regarding an order\nof tokens in the response. It could report extra tokens, missing tokens and\nmisplaced tokens. The method is useful when teaching language, where order of\ntokens is important, which includes most formal languages and some natural ones\n(like English). The method was implemented in a question type plug-in\nCorrectWriting for the widely used learning manage system Moodle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.2811v3",
            "date": "2013-01-13 19:33:31+00:00",
            "title": "Cutting Recursive Autoencoder Trees",
            "authors": [
                "Christian Scheible",
                "Hinrich Schuetze"
            ],
            "abstract": "Deep Learning models enjoy considerable success in Natural Language\nProcessing. While deep architectures produce useful representations that lead\nto improvements in various tasks, they are often difficult to interpret. This\nmakes the analysis of learned structures particularly difficult. In this paper,\nwe rely on empirical tests to see whether a particular structure makes sense.\nWe present an analysis of the Semi-Supervised Recursive Autoencoder, a\nwell-known model that produces structural representations of text. We show that\nfor certain tasks, the structure of the autoencoder can be significantly\nreduced without loss of classification accuracy and we evaluate the produced\nstructures using human judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3547v1",
            "date": "2013-01-16 01:42:53+00:00",
            "title": "A Rhetorical Analysis Approach to Natural Language Processing",
            "authors": [
                "Benjamin Englard"
            ],
            "abstract": "The goal of this research was to find a way to extend the capabilities of\ncomputers through the processing of language in a more human way, and present\napplications which demonstrate the power of this method. This research presents\na novel approach, Rhetorical Analysis, to solving problems in Natural Language\nProcessing (NLP). The main benefit of Rhetorical Analysis, as opposed to\nprevious approaches, is that it does not require the accumulation of large sets\nof training data, but can be used to solve a multitude of problems within the\nfield of NLP. The NLP problems investigated with Rhetorical Analysis were the\nAuthor Identification problem - predicting the author of a piece of text based\non its rhetorical strategies, Election Prediction - predicting the winner of a\npresidential candidate's re-election campaign based on rhetorical strategies\nwithin that president's inaugural address, Natural Language Generation - having\na computer produce text containing rhetorical strategies, and Document\nSummarization. The results of this research indicate that an Author\nIdentification system based on Rhetorical Analysis could predict the correct\nauthor 100% of the time, that a re-election predictor based on Rhetorical\nAnalysis could predict the correct winner of a re-election campaign 55% of the\ntime, that a Natural Language Generation system based on Rhetorical Analysis\ncould output text with up to 87.3% similarity to Shakespeare in style, and that\na Document Summarization system based on Rhetorical Analysis could extract\nhighly relevant sentences. Overall, this study demonstrated that Rhetorical\nAnalysis could be a useful approach to solving problems in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3618v2",
            "date": "2013-01-16 08:05:35+00:00",
            "title": "Learning New Facts From Knowledge Bases With Neural Tensor Networks and Semantic Word Vectors",
            "authors": [
                "Danqi Chen",
                "Richard Socher",
                "Christopher D. Manning",
                "Andrew Y. Ng"
            ],
            "abstract": "Knowledge bases provide applications with the benefit of easily accessible,\nsystematic relational knowledge but often suffer in practice from their\nincompleteness and lack of knowledge of new entities and relations. Much work\nhas focused on building or extending them by finding patterns in large\nunannotated text corpora. In contrast, here we mainly aim to complete a\nknowledge base by predicting additional true relationships between entities,\nbased on generalizations that can be discerned in the given knowledgebase. We\nintroduce a neural tensor network (NTN) model which predicts new relationship\nentries that can be added to the database. This model can be improved by\ninitializing entity representations with word vectors learned in an\nunsupervised fashion from text, and when doing this, existing relations can\neven be queried for entities that were not present in the database. Our model\ngeneralizes and outperforms existing models for this problem, and can classify\nunseen relationships in WordNet with an accuracy of 75.8%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.3627v2",
            "date": "2013-01-16 08:37:39+00:00",
            "title": "Two SVDs produce more focal deep learning representations",
            "authors": [
                "Hinrich Schuetze",
                "Christian Scheible"
            ],
            "abstract": "A key characteristic of work on deep learning and neural networks in general\nis that it relies on representations of the input that support generalization,\nrobust inference, domain adaptation and other desirable functionalities. Much\nrecent progress in the field has focused on efficient and effective methods for\ncomputing representations. In this paper, we propose an alternative method that\nis more efficient than prior work and produces representations that have a\nproperty we call focality -- a property we hypothesize to be important for\nneural network representations. The method consists of a simple application of\ntwo consecutive SVDs and is inspired by Anandkumar (2012).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.4938v3",
            "date": "2013-01-21 17:42:19+00:00",
            "title": "A type theoretical framework for natural language semantics: the Montagovian generative lexicon",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "We present a framework, named the Montagovian generative lexicon, for\ncomputing the semantics of natural language sentences, expressed in many sorted\nhigher order logic. Word meaning is depicted by lambda terms of second order\nlambda calculus (Girard's system F) with base types including a type for\npropositions and many types for sorts of a many sorted logic. This framework is\nable to integrate a proper treatment of lexical phenomena into a Montagovian\ncompositional semantics, including the restriction of selection which imposes\nthe nature of the arguments of a predicate, and the possible adaptation of a\nword meaning to some contexts. Among these adaptations of a word's sense to the\ncontext, ontological inclusions are handled by an extension of system F with\ncoercive subtyping that is introduced in the present paper. The benefits of\nthis framework for lexical pragmatics are illustrated on meaning transfers and\ncoercions, on possible and impossible copredication over different senses, on\ndeverbal ambiguities, and on \"fictive motion\". Next we show that the\ncompositional treatment of determiners, quantifiers, plurals,... are finer\ngrained in our framework. We then conclude with the linguistic, logical and\ncomputational perspectives opened by the Montagovian generative lexicon.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1301.6939v2",
            "date": "2013-01-29 14:59:34+00:00",
            "title": "Multi-Step Regression Learning for Compositional Distributional Semantics",
            "authors": [
                "Edward Grefenstette",
                "Georgiana Dinu",
                "Yao-Zhong Zhang",
                "Mehrnoosh Sadrzadeh",
                "Marco Baroni"
            ],
            "abstract": "We present a model for compositional distributional semantics related to the\nframework of Coecke et al. (2010), and emulating formal semantics by\nrepresenting functions as tensors and arguments as vectors. We introduce a new\nlearning method for tensors, generalising the approach of Baroni and Zamparelli\n(2010). We evaluate it on two benchmark data sets, and find it to outperform\nexisting leading methods. We argue in our analysis that the nature of this\nlearning method also renders it suitable for solving more subtle problems\ncompositional distributional models might face.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.7738v2",
            "date": "2013-01-31 20:21:52+00:00",
            "title": "PyPLN: a Distributed Platform for Natural Language Processing",
            "authors": [
                "Fl\u00e1vio Code\u00e7o Coelho",
                "Renato Rocha Souza",
                "\u00c1lvaro Justen",
                "Fl\u00e1vio Amieiro",
                "Heliana Mello"
            ],
            "abstract": "This paper presents a distributed platform for Natural Language Processing\ncalled PyPLN. PyPLN leverages a vast array of NLP and text processing open\nsource tools, managing the distribution of the workload on a variety of\nconfigurations: from a single server to a cluster of linux servers. PyPLN is\ndeveloped using Python 2.7.3 but makes it very easy to incorporate other\nsoftwares for specific tasks as long as a linux version is available. PyPLN\nfacilitates analyses both at document and corpus level, simplifying management\nand publication of corpora and analytical results through an easy to use web\ninterface. In the current (beta) release, it supports English and Portuguese\nlanguages with support to other languages planned for future releases. To\nsupport the Portuguese language PyPLN uses the PALAVRAS parser\\citep{Bick2000}.\nCurrently PyPLN offers the following features: Text extraction with encoding\nnormalization (to UTF-8), part-of-speech tagging, token frequency, semantic\nannotation, n-gram extraction, word and sentence repertoire, and full-text\nsearch across corpora. The platform is licensed as GPL-v3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.0715v1",
            "date": "2013-03-31 22:17:19+00:00",
            "title": "A cookbook of translating English to Xapi",
            "authors": [
                "Ladislau B\u00f6l\u00f6ni"
            ],
            "abstract": "The Xapagy cognitive architecture had been designed to perform narrative\nreasoning: to model and mimic the activities performed by humans when\nwitnessing, reading, recalling, narrating and talking about stories. Xapagy\ncommunicates with the outside world using Xapi, a simplified, \"pidgin\" language\nwhich is strongly tied to the internal representation model (instances, scenes\nand verb instances) and reasoning techniques (shadows and headless shadows).\nWhile not fully a semantic equivalent of natural language, Xapi can represent a\nwide range of complex stories. We illustrate the representation technique used\nin Xapi through examples taken from folk physics, folk psychology as well as\nsome more unusual literary examples. We argue that while the Xapi model\nrepresents a conceptual shift from the English representation, the mapping is\nlogical and consistent, and a trained knowledge engineer can translate between\nEnglish and Xapi at near-native speed.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1304.2476v1",
            "date": "2013-04-09 07:30:20+00:00",
            "title": "Corpus-based Web Document Summarization using Statistical and Linguistic Approach",
            "authors": [
                "Rushdi Shams",
                "M. M. A. Hashem",
                "Afrina Hossain",
                "Suraiya Rumana Akter",
                "Monika Gope"
            ],
            "abstract": "Single document summarization generates summary by extracting the\nrepresentative sentences from the document. In this paper, we presented a novel\ntechnique for summarization of domain-specific text from a single web document\nthat uses statistical and linguistic analysis on the text in a reference corpus\nand the web document. The proposed summarizer uses the combinational function\nof Sentence Weight (SW) and Subject Weight (SuW) to determine the rank of a\nsentence, where SW is the function of number of terms (t_n) and number of words\n(w_n) in a sentence, and term frequency (t_f) in the corpus and SuW is the\nfunction of t_n and w_n in a subject, and t_f in the corpus. 30 percent of the\nranked sentences are considered to be the summary of the web document. We\ngenerated three web document summaries using our technique and compared each of\nthem with the summaries developed manually from 16 different human subjects.\nResults showed that 68 percent of the summaries produced by our approach\nsatisfy the manual summaries.",
            "journal ref": "Procs. of the IEEE International Conference on Computer and\n  Communication Engineering (ICCCE10), pp. 115-120, Kuala Lumpur, Malaysia, May\n  11-13, (2010)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1304.3092v1",
            "date": "2013-03-27 19:52:35+00:00",
            "title": "Imprecise Meanings as a Cause of Uncertainty in Medical Knowledge-Based Systems",
            "authors": [
                "Steven J. Henkind"
            ],
            "abstract": "There has been a considerable amount of work on uncertainty in\nknowledge-based systems. This work has generally been concerned with\nuncertainty arising from the strength of inferences and the weight of evidence.\nIn this paper we discuss another type of uncertainty: that which is due to\nimprecision in the underlying primitives used to represent the knowledge of the\nsystem. In particular, a given word may denote many similar but not identical\nentities. Such words are said to be lexically imprecise. Lexical imprecision\nhas caused widespread problems in many areas. Unless this phenomenon is\nrecognized and appropriately handled, it can degrade the performance of\nknowledge-based systems. In particular, it can lead to difficulties with the\nuser interface, and with the inferencing processes of these systems. Some\ntechniques are suggested for coping with this phenomenon.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1304.3841v2",
            "date": "2013-04-13 20:19:50+00:00",
            "title": "The risks of mixing dependency lengths from sequences of different length",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Haitao Liu"
            ],
            "abstract": "Mixing dependency lengths from sequences of different length is a common\npractice in language research. However, the empirical distribution of\ndependency lengths of sentences of the same length differs from that of\nsentences of varying length and the distribution of dependency lengths depends\non sentence length for real sentences and also under the null hypothesis that\ndependencies connect vertices located in random positions of the sequence. This\nsuggests that certain results, such as the distribution of syntactic dependency\nlengths mixing dependencies from sentences of varying length, could be a mere\nconsequence of that mixing. Furthermore, differences in the global averages of\ndependency length (mixing lengths from sentences of varying length) for two\ndifferent languages do not simply imply a priori that one language optimizes\ndependency lengths better than the other because those differences could be due\nto differences in the distribution of sentence lengths and other factors.",
            "journal ref": "Glottotheory 5 (2), 143-155 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.3879v1",
            "date": "2013-04-14 05:52:11+00:00",
            "title": "Automatic case acquisition from texts for process-oriented case-based reasoning",
            "authors": [
                "Valmi Dufour-Lussier",
                "Florence Le Ber",
                "Jean Lieber",
                "Emmanuel Nauer"
            ],
            "abstract": "This paper introduces a method for the automatic acquisition of a rich case\nrepresentation from free text for process-oriented case-based reasoning. Case\nengineering is among the most complicated and costly tasks in implementing a\ncase-based reasoning system. This is especially so for process-oriented\ncase-based reasoning, where more expressive case representations are generally\nused and, in our opinion, actually required for satisfactory case adaptation.\nIn this context, the ability to acquire cases automatically from procedural\ntexts is a major step forward in order to reason on processes. We therefore\ndetail a methodology that makes case acquisition from processes described as\nfree text possible, with special attention given to assembly instruction texts.\nThis methodology extends the techniques we used to extract actions from cooking\nrecipes. We argue that techniques taken from natural language processing are\nrequired for this task, and that they give satisfactory results. An evaluation\nbased on our implemented prototype extracting workflows from recipe texts is\nprovided.",
            "journal ref": "Information Systems (2012)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1304.7157v1",
            "date": "2013-04-26 13:27:19+00:00",
            "title": "Question Answering Against Very-Large Text Collections",
            "authors": [
                "Leon Derczynski",
                "Richard Shaw",
                "Ben Solway",
                "Jun Wang"
            ],
            "abstract": "Question answering involves developing methods to extract useful information\nfrom large collections of documents. This is done with specialised search\nengines such as Answer Finder. The aim of Answer Finder is to provide an answer\nto a question rather than a page listing related documents that may contain the\ncorrect answer. So, a question such as \"How tall is the Eiffel Tower\" would\nsimply return \"325m\" or \"1,063ft\". Our task was to build on the current version\nof Answer Finder by improving information retrieval, and also improving the\npre-processing involved in question series analysis.",
            "journal ref": "Master's theses, 2008, University of Sheffield",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.7507v1",
            "date": "2013-04-28 19:30:11+00:00",
            "title": "Measuring Cultural Relativity of Emotional Valence and Arousal using Semantic Clustering and Twitter",
            "authors": [
                "Eugene Yuta Bann",
                "Joanna J. Bryson"
            ],
            "abstract": "Researchers since at least Darwin have debated whether and to what extent\nemotions are universal or culture-dependent. However, previous studies have\nprimarily focused on facial expressions and on a limited set of emotions. Given\nthat emotions have a substantial impact on human lives, evidence for cultural\nemotional relativity might be derived by applying distributional semantics\ntechniques to a text corpus of self-reported behaviour. Here, we explore this\nidea by measuring the valence and arousal of the twelve most popular emotion\nkeywords expressed on the micro-blogging site Twitter. We do this in three\ngeographical regions: Europe, Asia and North America. We demonstrate that in\nour sample, the valence and arousal levels of the same emotion keywords differ\nsignificantly with respect to these geographical regions --- Europeans are, or\nat least present themselves as more positive and aroused, North Americans are\nmore negative and Asians appear to be more positive but less aroused when\ncompared to global valence and arousal levels of the same emotion keywords. Our\nwork is the first in kind to programatically map large text corpora to a\ndimensional model of affect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.7728v1",
            "date": "2013-04-29 18:04:26+00:00",
            "title": "Machine Translation Systems in India",
            "authors": [
                "Sugata Sanyal",
                "Rajdeep Borgohain"
            ],
            "abstract": "Machine Translation is the translation of one natural language into another\nusing automated and computerized means. For a multilingual country like India,\nwith the huge amount of information exchanged between various regions and in\ndifferent languages in digitized format, it has become necessary to find an\nautomated process from one language to another. In this paper, we take a look\nat the various Machine Translation System in India which is specifically built\nfor the purpose of translation between the Indian languages. We discuss the\nvarious approaches taken for building the machine translation system and then\ndiscuss some of the Machine Translation Systems in India along with their\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.8016v1",
            "date": "2013-04-23 22:14:18+00:00",
            "title": "On Semantic Word Cloud Representation",
            "authors": [
                "Lukas Barth",
                "Stephen Kobourov",
                "Sergey Pupyrev",
                "Torsten Ueckerdt"
            ],
            "abstract": "We study the problem of computing semantic-preserving word clouds in which\nsemantically related words are close to each other. While several heuristic\napproaches have been described in the literature, we formalize the underlying\ngeometric algorithm problem: Word Rectangle Adjacency Contact (WRAC). In this\nmodel each word is associated with rectangle with fixed dimensions, and the\ngoal is to represent semantically related words by ensuring that the two\ncorresponding rectangles touch. We design and analyze efficient polynomial-time\nalgorithms for some variants of the WRAC problem, show that several general\nvariants are NP-hard, and describe a number of approximation algorithms.\nFinally, we experimentally demonstrate that our theoretically-sound algorithms\noutperform the early heuristics.",
            "journal ref": "None",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/1305.0556v2",
            "date": "2013-05-02 18:49:01+00:00",
            "title": "A quantum teleportation inspired algorithm produces sentence meaning from word meaning and grammatical structure",
            "authors": [
                "Stephen Clark",
                "Bob Coecke",
                "Edward Grefenstette",
                "Stephen Pulman",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "We discuss an algorithm which produces the meaning of a sentence given\nmeanings of its words, and its resemblance to quantum teleportation. In fact,\nthis protocol was the main source of inspiration for this algorithm which has\nmany applications in the area of Natural Language Processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.0625v1",
            "date": "2013-05-03 06:25:18+00:00",
            "title": "CONATION: English Command Input/Output System for Computers",
            "authors": [
                "Kamlesh Sharma",
                "Dr. T. V. Prasad"
            ],
            "abstract": "In this information technology age, a convenient and user friendly interface\nis required to operate the computer system on very fast rate. In the human\nbeing, speech being a natural mode of communication has potential to being a\nfast and convenient mode of interaction with computer. Speech recognition will\nplay an important role in taking technology to them. It is the need of this era\nto access the information within seconds. This paper describes the design and\ndevelopment of speaker independent and English command interpreted system for\ncomputers. HMM model is used to represent the phoneme like speech commands.\nExperiments have been done on real world data and system has been trained in\nnormal condition for real world subject.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1305.1145v1",
            "date": "2013-05-06 10:42:34+00:00",
            "title": "Techniques for Feature Extraction In Speech Recognition System : A Comparative Study",
            "authors": [
                "Urmila Shrawankar",
                "V M Thakare"
            ],
            "abstract": "The time domain waveform of a speech signal carries all of the auditory\ninformation. From the phonological point of view, it little can be said on the\nbasis of the waveform itself. However, past research in mathematics, acoustics,\nand speech technology have provided many methods for converting data that can\nbe considered as information if interpreted correctly. In order to find some\nstatistically relevant information from incoming data, it is important to have\nmechanisms for reducing the information of each segment in the audio signal\ninto a relatively small number of parameters, or features. These features\nshould describe each segment in such a characteristic way that other similar\nsegments can be grouped together by comparing their features. There are\nenormous interesting and exceptional ways to describe the speech signal in\nterms of parameters. Though, they all have their strengths and weaknesses, we\nhave presented some of the most used methods with their importance.",
            "journal ref": "International Journal Of Computer Applications In Engineering,\n  Technology and Sciences (IJCAETS),ISSN 0974-3596,2010,pp 412-418",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1305.1426v1",
            "date": "2013-05-07 07:21:06+00:00",
            "title": "Speech Enhancement Modeling Towards Robust Speech Recognition System",
            "authors": [
                "Urmila Shrawankar",
                "V. M. Thakare"
            ],
            "abstract": "Form about four decades human beings have been dreaming of an intelligent\nmachine which can master the natural speech. In its simplest form, this machine\nshould consist of two subsystems, namely automatic speech recognition (ASR) and\nspeech understanding (SU). The goal of ASR is to transcribe natural speech\nwhile SU is to understand the meaning of the transcription. Recognizing and\nunderstanding a spoken sentence is obviously a knowledge-intensive process,\nwhich must take into account all variable information about the speech\ncommunication process, from acoustics to semantics and pragmatics. While\ndeveloping an Automatic Speech Recognition System, it is observed that some\nadverse conditions degrade the performance of the Speech Recognition System. In\nthis contribution, speech enhancement system is introduced for enhancing speech\nsignals corrupted by additive noise and improving the performance of Automatic\nSpeech Recognizers in noisy conditions. Automatic speech recognition\nexperiments show that replacing noisy speech signals by the corresponding\nenhanced speech signals leads to an improvement in the recognition accuracies.\nThe amount of improvement varies with the type of the corrupting noise.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1305.1925v1",
            "date": "2013-05-08 05:55:50+00:00",
            "title": "Speech: A Challenge to Digital Signal Processing Technology for Human-to-Computer Interaction",
            "authors": [
                "Urmila Shrawankar",
                "Anjali Mahajan"
            ],
            "abstract": "This software project based paper is for a vision of the near future in which\ncomputer interaction is characterized by natural face-to-face conversations\nwith lifelike characters that speak, emote, and gesture. The first step is\nspeech. The dream of a true virtual reality, a complete human-computer\ninteraction system will not come true unless we try to give some perception to\nmachine and make it perceive the outside world as humans communicate with each\nother. This software project is under development for listening and replying\nmachine (Computer) through speech. The Speech interface is developed to convert\nspeech input into some parametric form (Speech-to-Text) for further processing\nand the results, text output to speech synthesis (Text-to-Speech)",
            "journal ref": "Conference Proceedings National Conference on Recent Trends in\n  Electronics & Information Technology (RTEIT),2006,pp 206-212",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1305.2352v1",
            "date": "2013-05-09 08:39:11+00:00",
            "title": "Speech Enhancement Using Pitch Detection Approach For Noisy Environment",
            "authors": [
                "Rashmi Makhijani",
                "Urmila Shrawankar",
                "V M Thakare"
            ],
            "abstract": "Acoustical mismatch among training and testing phases degrades outstandingly\nspeech recognition results. This problem has limited the development of\nreal-world nonspecific applications, as testing conditions are highly variant\nor even unpredictable during the training process. Therefore the background\nnoise has to be removed from the noisy speech signal to increase the signal\nintelligibility and to reduce the listener fatigue. Enhancement techniques\napplied, as pre-processing stages; to the systems remarkably improve\nrecognition results. In this paper, a novel approach is used to enhance the\nperceived quality of the speech signal when the additive noise cannot be\ndirectly controlled. Instead of controlling the background noise, we propose to\nreinforce the speech signal so that it can be heard more clearly in noisy\nenvironments. The subjective evaluation shows that the proposed method improves\nperceptual quality of speech in various noisy environments. As in some cases\nspeaking may be more convenient than typing, even for rapid typists: many\nmathematical symbols are missing from the keyboard but can be easily spoken and\nrecognized. Therefore, the proposed system can be used in an application\ndesigned for mathematical symbol recognition (especially symbols not available\non the keyboard) in schools.",
            "journal ref": "International Journal of Engineering Science and Technology\n  (IJEST), 2011, ISSN : 0975-5462 Vol. 3 No. 2, pp 1764-1769",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1305.2680v1",
            "date": "2013-05-13 06:38:34+00:00",
            "title": "A study for the effect of the Emphaticness and language and dialect for Voice Onset Time (VOT) in Modern Standard Arabic (MSA)",
            "authors": [
                "Sulaiman S. AlDahri"
            ],
            "abstract": "The signal sound contains many different features, including Voice Onset Time\n(VOT), which is a very important feature of stop sounds in many languages. The\nonly application of VOT values is stopping phoneme subsets. This subset of\nconsonant sounds is stop phonemes exist in the Arabic language, and in fact,\nall languages. The pronunciation of these sounds is hard and unique especially\nfor less-educated Arabs and non-native Arabic speakers. VOT can be utilized by\nthe human auditory system to distinguish between voiced and unvoiced stops such\nas /p/ and /b/ in English.This search focuses on computing and analyzing VOT of\nModern Standard Arabic (MSA), within the Arabic language, for all pairs of\nnon-emphatic (namely, /d/ and /t/) and emphatic pairs (namely, /d?/ and /t?/)\ndepending on carrier words. This research uses a database built by ourselves,\nand uses the carrier words syllable structure: CV-CV-CV. One of the main\noutcomes always found is the emphatic sounds (/d?/, /t?/) are less than 50% of\nnon-emphatic (counter-part) sounds ( /d/, /t/).Also, VOT can be used to\nclassify or detect for a dialect ina language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.2846v1",
            "date": "2013-05-09 08:42:26+00:00",
            "title": "Opportunities & Challenges In Automatic Speech Recognition",
            "authors": [
                "Rashmi Makhijani",
                "Urmila Shrawankar",
                "V M Thakare"
            ],
            "abstract": "Automatic speech recognition enables a wide range of current and emerging\napplications such as automatic transcription, multimedia content analysis, and\nnatural human-computer interfaces. This paper provides a glimpse of the\nopportunities and challenges that parallelism provides for automatic speech\nrecognition and related application research from the point of view of speech\nresearchers. The increasing parallelism in computing platforms opens three\nmajor possibilities for speech recognition systems: improving recognition\naccuracy in non-ideal, everyday noisy environments; increasing recognition\nthroughput in batch processing of speech data; and reducing recognition latency\nin realtime usage scenarios. This paper describes technical challenges,\napproaches taken, and possible directions for future research to guide the\ndesign of efficient parallel software and hardware infrastructures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.2847v1",
            "date": "2013-05-09 08:44:58+00:00",
            "title": "An Overview of Hindi Speech Recognition",
            "authors": [
                "Neema Mishra",
                "Urmila Shrawankar",
                "V M Thakare"
            ],
            "abstract": "In this age of information technology, information access in a convenient\nmanner has gained importance. Since speech is a primary mode of communication\namong human beings, it is natural for people to expect to be able to carry out\nspoken dialogue with computer. Speech recognition system permits ordinary\npeople to speak to the computer to retrieve information. It is desirable to\nhave a human computer dialogue in local language. Hindi being the most widely\nspoken Language in India is the natural primary human language candidate for\nhuman machine interaction. There are five pairs of vowels in Hindi languages;\none member is longer than the other one. This paper describes an overview of\nspeech recognition system that includes how speech is produced and the\nproperties and characteristics of Hindi Phoneme.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.2959v1",
            "date": "2013-05-09 08:47:47+00:00",
            "title": "Automatic Speech Recognition Using Template Model for Man-Machine Interface",
            "authors": [
                "Neema Mishra",
                "Urmila Shrawankar",
                "V M Thakare"
            ],
            "abstract": "Speech is a natural form of communication for human beings, and computers\nwith the ability to understand speech and speak with a human voice are expected\nto contribute to the development of more natural man-machine interfaces.\nComputers with this kind of ability are gradually becoming a reality, through\nthe evolution of speech recognition technologies. Speech is being an important\nmode of interaction with computers. In this paper Feature extraction is\nimplemented using well-known Mel-Frequency Cepstral Coefficients (MFCC).Pattern\nmatching is done using Dynamic time warping (DTW) algorithm.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1305.3107v1",
            "date": "2013-05-14 10:49:16+00:00",
            "title": "I Wish I Didn't Say That! Analyzing and Predicting Deleted Messages in Twitter",
            "authors": [
                "Sasa Petrovic",
                "Miles Osborne",
                "Victor Lavrenko"
            ],
            "abstract": "Twitter has become a major source of data for social media researchers. One\nimportant aspect of Twitter not previously considered are {\\em deletions} --\nremoval of tweets from the stream. Deletions can be due to a multitude of\nreasons such as privacy concerns, rashness or attempts to undo public\nstatements. We show how deletions can be automatically predicted ahead of time\nand analyse which tweets are likely to be deleted and how.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1305.6238v1",
            "date": "2013-05-27 14:36:53+00:00",
            "title": "Extended Lambek calculi and first-order linear logic",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "First-order multiplicative intuitionistic linear logic (MILL1) can be seen as\nan extension of the Lambek calculus. In addition to the fragment of MILL1 which\ncorresponds to the Lambek calculus (of Moot & Piazza 2001), I will show\nfragments of MILL1 which generate the multiple context-free languages and which\ncorrespond to the Displacement calculus of Morrilll e.a.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.1343v1",
            "date": "2013-06-06 08:56:32+00:00",
            "title": "The User Feedback on SentiWordNet",
            "authors": [
                "Andrea Esuli"
            ],
            "abstract": "With the release of SentiWordNet 3.0 the related Web interface has been\nrestyled and improved in order to allow users to submit feedback on the\nSentiWordNet entries, in the form of the suggestion of alternative triplets of\nvalues for an entry. This paper reports on the release of the user feedback\ncollected so far and on the plans for the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.1927v1",
            "date": "2013-06-08 14:59:52+00:00",
            "title": "Learning About Meetings",
            "authors": [
                "Been Kim",
                "Cynthia Rudin"
            ],
            "abstract": "Most people participate in meetings almost every day, multiple times a day.\nThe study of meetings is important, but also challenging, as it requires an\nunderstanding of social signals and complex interpersonal dynamics. Our aim\nthis work is to use a data-driven approach to the science of meetings. We\nprovide tentative evidence that: i) it is possible to automatically detect when\nduring the meeting a key decision is taking place, from analyzing only the\nlocal dialogue acts, ii) there are common patterns in the way social dialogue\nacts are interspersed throughout a meeting, iii) at the time key decisions are\nmade, the amount of time left in the meeting can be predicted from the amount\nof time that has passed, iv) it is often possible to predict whether a proposal\nduring a meeting will be accepted or rejected based entirely on the language\n(the set of persuasive words) used by the speaker.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1306.2268v1",
            "date": "2013-06-07 04:42:02+00:00",
            "title": "Accomplishable Tasks in Knowledge Representation",
            "authors": [
                "Keehang Kwon",
                "Mi-Young Park"
            ],
            "abstract": "Knowledge Representation (KR) is traditionally based on the logic of facts,\nexpressed in boolean logic. However, facts about an agent can also be seen as a\nset of accomplished tasks by the agent. This paper proposes a new approach to\nKR: the notion of task logical KR based on Computability Logic. This notion\nallows the user to represent both accomplished tasks and accomplishable tasks\nby the agent. This notion allows us to build sophisticated KRs about many\ninteresting agents, which have not been supported by previous logical\nlanguages.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1306.2499v2",
            "date": "2013-06-11 12:24:55+00:00",
            "title": "Using Arabic Wordnet for semantic indexation in information retrieval system",
            "authors": [
                "Mohammed Alaeddine Abderrahim",
                "Mohammed El Amine Abderrahim",
                "Mohammed Amine Chikh"
            ],
            "abstract": "In the context of arabic Information Retrieval Systems (IRS) guided by arabic\nontology and to enable those systems to better respond to user requirements,\nthis paper aims to representing documents and queries by the best concepts\nextracted from Arabic Wordnet. Identified concepts belonging to Arabic WordNet\nsynsets are extracted from documents and queries, and those having a single\nsense are expanded. The expanded query is then used by the IRS to retrieve the\nrelevant documents searched. Our experiments are based primarily on a medium\nsize corpus of arabic text. The results obtained shown us that there are a\nglobal improvement in the performance of the arabic IRS.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 10,\n  Issue 1, No 2, January 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1306.2593v2",
            "date": "2013-06-11 17:48:01+00:00",
            "title": "A Perceptual Alphabet for the 10-dimensional Phonetic-prosodic Space",
            "authors": [
                "Elaine Y L Tsiang"
            ],
            "abstract": "We define an alphabet, the IHA, of the 10-D phonetic-prosodic space. The\ndimensions of this space are perceptual observables, rather than articulatory\nspecifications. Speech is defined as a random chain in time of the 4-D phonetic\nsubspace, that is, a symbolic sequence, augmented with diacritics of the\nremaining 6-D prosodic subspace. The definitions here are based on the model of\nspeech of oral billiards, and supersedes an earlier version. This paper only\nenumerates the IHA in detail as a supplement to the exposition of oral\nbilliards in a separate paper. The IHA has been implemented as the target\nrandom variable in a speech recognizer.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1306.3692v2",
            "date": "2013-06-16 18:19:15+00:00",
            "title": "An open diachronic corpus of historical Spanish: annotation criteria and automatic modernisation of spelling",
            "authors": [
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Isabel Mart\u00ednez-Sempere",
                "Xavier Ivars-Ribes",
                "Rafael C. Carrasco"
            ],
            "abstract": "The IMPACT-es diachronic corpus of historical Spanish compiles over one\nhundred books --containing approximately 8 million words-- in addition to a\ncomplementary lexicon which links more than 10 thousand lemmas with\nattestations of the different variants found in the documents. This textual\ncorpus and the accompanying lexicon have been released under an open license\n(Creative Commons by-nc-sa) in order to permit their intensive exploitation in\nlinguistic research. Approximately 7% of the words in the corpus (a selection\naimed at enhancing the coverage of the most frequent word forms) have been\nannotated with their lemma, part of speech, and modern equivalent. This paper\ndescribes the annotation criteria followed and the standards, based on the Text\nEncoding Initiative recommendations, used to the represent the texts in digital\nform. As an illustration of the possible synergies between diachronic textual\nresources and linguistic research, we describe the application of statistical\nmachine translation techniques to infer probabilistic context-sensitive rules\nfor the automatic modernisation of spelling. The automatic modernisation with\nthis type of statistical methods leads to very low character error rates when\nthe output is compared with the supervised modern version of the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.4139v1",
            "date": "2013-06-18 11:03:36+00:00",
            "title": "Punjabi Language Interface to Database: a brief review",
            "authors": [
                "Preeti Verma",
                "Suket Arora",
                "Kamaljit Batra"
            ],
            "abstract": "Unlike most user-computer interfaces, a natural language interface allows\nusers to communicate fluently with a computer system with very little\npreparation. Databases are often hard to use in cooperating with the users\nbecause of their rigid interface. A good NLIDB allows a user to enter commands\nand ask questions in native language and then after interpreting respond to the\nuser in native language. For a large number of applications requiring\ninteraction between humans and the computer systems, it would be convenient to\nprovide the end-user friendly interface. Punjabi language interface to database\nwould proof fruitful to native people of Punjab, as it provides ease to them to\nuse various e-governance applications like Punjab Sewa, Suwidha, Online Public\nUtility Forms, Online Grievance Cell, Land Records Management System,legacy\nmatters, e-District, agriculture, etc. Punjabi is the mother tongue of more\nthan 110 million people all around the world. According to available\ninformation, Punjabi ranks 10th from top out of a total of 6,900 languages\nrecognized internationally by the United Nations. This paper covers a brief\noverview of the Natural language interface to database, its different\ncomponents, its advantages, disadvantages, approaches and techniques used. The\npaper ends with the work done on Punjabi language interface to database and\nfuture enhancements that can be done.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.4886v1",
            "date": "2013-06-20 14:22:00+00:00",
            "title": "Supervised Topical Key Phrase Extraction of News Stories using Crowdsourcing, Light Filtering and Co-reference Normalization",
            "authors": [
                "Luis Marujo",
                "Anatole Gershman",
                "Jaime Carbonell",
                "Robert Frederking",
                "Jo\u00e3o P. Neto"
            ],
            "abstract": "Fast and effective automated indexing is critical for search and personalized\nservices. Key phrases that consist of one or more words and represent the main\nconcepts of the document are often used for the purpose of indexing. In this\npaper, we investigate the use of additional semantic features and\npre-processing steps to improve automatic key phrase extraction. These features\ninclude the use of signal words and freebase categories. Some of these features\nlead to significant improvements in the accuracy of the results. We also\nexperimented with 2 forms of document pre-processing that we call light\nfiltering and co-reference normalization. Light filtering removes sentences\nfrom the document, which are judged peripheral to its main content.\nCo-reference normalization unifies several written forms of the same named\nentity into a unique form. We also needed a \"Gold Standard\" - a set of labeled\ndocuments for training and evaluation. While the subjective nature of key\nphrase selection precludes a true \"Gold Standard\", we used Amazon's Mechanical\nTurk service to obtain a useful approximation. Our data indicates that the\nbiggest improvements in performance were due to shallow semantic features, news\ncategories, and rhetorical signals (nDCG 78.47% vs. 68.93%). The inclusion of\ndeeper semantic features such as Freebase sub-categories was not beneficial by\nitself, but in combination with pre-processing, did cause slight improvements\nin the nDCG scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.4890v1",
            "date": "2013-06-20 14:35:22+00:00",
            "title": "Key Phrase Extraction of Lightly Filtered Broadcast News",
            "authors": [
                "Luis Marujo",
                "Ricardo Ribeiro",
                "David Martins de Matos",
                "Jo\u00e3o P. Neto",
                "Anatole Gershman",
                "Jaime Carbonell"
            ],
            "abstract": "This paper explores the impact of light filtering on automatic key phrase\nextraction (AKE) applied to Broadcast News (BN). Key phrases are words and\nexpressions that best characterize the content of a document. Key phrases are\noften used to index the document or as features in further processing. This\nmakes improvements in AKE accuracy particularly important. We hypothesized that\nfiltering out marginally relevant sentences from a document would improve AKE\naccuracy. Our experiments confirmed this hypothesis. Elimination of as little\nas 10% of the document sentences lead to a 2% improvement in AKE precision and\nrecall. AKE is built over MAUI toolkit that follows a supervised learning\napproach. We trained and tested our AKE method on a gold standard made of 8 BN\nprograms containing 110 manually annotated news stories. The experiments were\nconducted within a Multimedia Monitoring Solution (MMS) system for TV and radio\nnews/programs, running daily, and monitoring 12 TV and 4 radio channels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.4908v1",
            "date": "2013-06-20 15:22:21+00:00",
            "title": "Recognition of Named-Event Passages in News Articles",
            "authors": [
                "Luis Marujo",
                "Wang Ling",
                "Anatole Gershman",
                "Jaime Carbonell",
                "Jo\u00e3o P. Neto",
                "David Matos"
            ],
            "abstract": "We extend the concept of Named Entities to Named Events - commonly occurring\nevents such as battles and earthquakes. We propose a method for finding\nspecific passages in news articles that contain information about such events\nand report our preliminary evaluation results. Collecting \"Gold Standard\" data\npresents many problems, both practical and conceptual. We present a method for\nobtaining such data using the Amazon Mechanical Turk service.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.5170v1",
            "date": "2013-06-21 15:30:09+00:00",
            "title": "Clinical Relationships Extraction Techniques from Patient Narratives",
            "authors": [
                "Wafaa Tawfik Abdel-moneim",
                "Mohamed Hashem Abdel-Aziz",
                "Mohamed Monier Hassan"
            ],
            "abstract": "The Clinical E-Science Framework (CLEF) project was used to extract important\ninformation from medical texts by building a system for the purpose of clinical\nresearch, evidence-based healthcare and genotype-meets-phenotype informatics.\nThe system is divided into two parts, one part concerns with the identification\nof relationships between clinically important entities in the text. The full\nparses and domain-specific grammars had been used to apply many approaches to\nextract the relationship. In the second part of the system, statistical machine\nlearning (ML) approaches are applied to extract relationship. A corpus of\noncology narratives that hand annotated with clinical relationships can be used\nto train and test a system that has been designed and implemented by supervised\nmachine learning (ML) approaches. Many features can be extracted from these\ntexts that are used to build a model by the classifier. Multiple supervised\nmachine learning algorithms can be applied for relationship extraction. Effects\nof adding the features, changing the size of the corpus, and changing the type\nof the algorithm on relationship extraction are examined. Keywords: Text\nmining; information extraction; NLP; entities; and relations.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol.10,\n  Issue 1, January 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1306.5263v1",
            "date": "2013-06-21 22:09:36+00:00",
            "title": "Discriminative Training: Learning to Describe Video with Sentences, from Video Described with Sentences",
            "authors": [
                "Haonan Yu",
                "Jeffrey Mark Siskind"
            ],
            "abstract": "We present a method for learning word meanings from complex and realistic\nvideo clips by discriminatively training (DT) positive sentential labels\nagainst negative ones, and then use the trained word models to generate\nsentential descriptions for new video. This new work is inspired by recent work\nwhich adopts a maximum likelihood (ML) framework to address the same problem\nusing only positive sentential labels. The new method, like the ML-based one,\nis able to automatically determine which words in the sentence correspond to\nwhich concepts in the video (i.e., ground words to meanings) in a weakly\nsupervised fashion. While both DT and ML yield comparable results with\nsufficient training data, DT outperforms ML significantly with smaller training\nsets because it can exploit negative training labels to better constrain the\nlearning problem.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1306.6755v1",
            "date": "2013-06-28 08:46:11+00:00",
            "title": "Arabizi Detection and Conversion to Arabic",
            "authors": [
                "Kareem Darwish"
            ],
            "abstract": "Arabizi is Arabic text that is written using Latin characters. Arabizi is\nused to present both Modern Standard Arabic (MSA) or Arabic dialects. It is\ncommonly used in informal settings such as social networking sites and is often\nwith mixed with English. In this paper we address the problems of: identifying\nArabizi in text and converting it to Arabic characters. We used word and\nsequence-level features to identify Arabizi that is mixed with English. We\nachieved an identification accuracy of 98.5%. As for conversion, we used\ntransliteration mining with language modeling to generate equivalent Arabic\ntext. We achieved 88.7% conversion accuracy, with roughly a third of errors\nbeing spelling and morphological variants of the forms in ground truth.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.1897v1",
            "date": "2013-11-08 08:24:08+00:00",
            "title": "Logique math\u00e9matique et linguistique formelle",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "As the etymology of the word shows, logic is intimately related to language,\nas exemplified by the work of philosophers from Antiquity and from the\nMiddle-Age. At the beginning of the XX century, the crisis of the foundations\nof mathematics invented mathematical logic and imposed logic as a\nlanguage-based foundation for mathematics. How did the relations between logic\nand language evolved in this newly defined mathematical framework? After a\nsurvey of the history of the relation between logic and linguistics,\ntraditionally focused on semantics, we focus on some present issues: 1) grammar\nas a deductive system 2) the transformation of the syntactic structure of a\nsentence to a logical formula representing its meaning 3) taking into account\nthe context when interpreting words. This lecture shows that type theory\nprovides a convenient framework both for natural language syntax and for the\ninterpretation of any of tis level (words, sentences, discourse).",
            "journal ref": "Le\\c{c}ons de math\\'ematiques d'aujourd'hui, G\\'eraud\n  S\\'enizergues (Ed.) (2013) 24",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1311.2252v1",
            "date": "2013-11-10 09:15:16+00:00",
            "title": "Semantic Sort: A Supervised Approach to Personalized Semantic Relatedness",
            "authors": [
                "Ran El-Yaniv",
                "David Yanay"
            ],
            "abstract": "We propose and study a novel supervised approach to learning statistical\nsemantic relatedness models from subjectively annotated training examples. The\nproposed semantic model consists of parameterized co-occurrence statistics\nassociated with textual units of a large background knowledge corpus. We\npresent an efficient algorithm for learning such semantic models from a\ntraining sample of relatedness preferences. Our method is corpus independent\nand can essentially rely on any sufficiently large (unstructured) collection of\ncoherent texts. Moreover, the approach facilitates the fitting of semantic\nmodels for specific users or groups of users. We present the results of\nextensive range of experiments from small to large scale, indicating that the\nproposed method is effective and competitive with the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.3175v1",
            "date": "2013-11-13 15:36:12+00:00",
            "title": "Architecture of an Ontology-Based Domain-Specific Natural Language Question Answering System",
            "authors": [
                "Athira P. M.",
                "Sreeja M.",
                "P. C. Reghu Raj"
            ],
            "abstract": "Question answering (QA) system aims at retrieving precise information from a\nlarge collection of documents against a query. This paper describes the\narchitecture of a Natural Language Question Answering (NLQA) system for a\nspecific domain based on the ontological information, a step towards semantic\nweb question answering. The proposed architecture defines four basic modules\nsuitable for enhancing current QA capabilities with the ability of processing\ncomplex questions. The first module was the question processing, which analyses\nand classifies the question and also reformulates the user query. The second\nmodule allows the process of retrieving the relevant documents. The next module\nprocesses the retrieved documents, and the last module performs the extraction\nand generation of a response. Natural language processing techniques are used\nfor processing the question and documents and also for answer extraction.\nOntology and domain knowledge are used for reformulating queries and\nidentifying the relations. The aim of the system is to generate short and\nspecific answer to the question that is asked in the natural language in a\nspecific domain. We have achieved 94 % accuracy of natural language question\nanswering in our implementation.",
            "journal ref": "International Journal of Web & Semantic Technology (IJWesT) Vol.4,\n  No.4, October 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.5401v2",
            "date": "2013-11-21 13:47:21+00:00",
            "title": "Clustering and Relational Ambiguity: from Text Data to Natural Data",
            "authors": [
                "Nicolas Turenne"
            ],
            "abstract": "Text data is often seen as \"take-away\" materials with little noise and easy\nto process information. Main questions are how to get data and transform them\ninto a good document format. But data can be sensitive to noise oftenly called\nambiguities. Ambiguities are aware from a long time, mainly because polysemy is\nobvious in language and context is required to remove uncertainty. I claim in\nthis paper that syntactic context is not suffisant to improve interpretation.\nIn this paper I try to explain that firstly noise can come from natural data\nthemselves, even involving high technology, secondly texts, seen as verified\nbut meaningless, can spoil content of a corpus; it may lead to contradictions\nand background noise.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2014 (June 24, 2014)\n  jdmdh:13",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.6421v1",
            "date": "2013-11-25 19:48:30+00:00",
            "title": "Synchronous Context-Free Grammars and Optimal Linear Parsing Strategies",
            "authors": [
                "Pierluigi Crescenzi",
                "Daniel Gildea",
                "Andrea Marino",
                "Gianluca Rossi",
                "Giorgio Satta"
            ],
            "abstract": "Synchronous Context-Free Grammars (SCFGs), also known as syntax-directed\ntranslation schemata, are unlike context-free grammars in that they do not have\na binary normal form. In general, parsing with SCFGs takes space and time\npolynomial in the length of the input strings, but with the degree of the\npolynomial depending on the permutations of the SCFG rules. We consider linear\nparsing strategies, which add one nonterminal at a time. We show that for a\ngiven input permutation, the problems of finding the linear parsing strategy\nwith the minimum space and time complexity are both NP-hard.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1303.0350v1",
            "date": "2013-03-02 04:26:02+00:00",
            "title": "Structure-semantics interplay in complex networks and its effects on the predictability of similarity in texts",
            "authors": [
                "Diego R. Amancio",
                "Osvaldo N. Oliveira Jr.",
                "Luciano da F. Costa"
            ],
            "abstract": "There are different ways to define similarity for grouping similar texts into\nclusters, as the concept of similarity may depend on the purpose of the task.\nFor instance, in topic extraction similar texts mean those within the same\nsemantic field, whereas in author recognition stylistic features should be\nconsidered. In this study, we introduce ways to classify texts employing\nconcepts of complex networks, which may be able to capture syntactic, semantic\nand even pragmatic features. The interplay between the various metrics of the\ncomplex networks is analyzed with three applications, namely identification of\nmachine translation (MT) systems, evaluation of quality of machine translated\ntexts and authorship recognition. We shall show that topological features of\nthe networks representing texts can enhance the ability to identify MT systems\nin particular cases. For evaluating the quality of MT texts, on the other hand,\nhigh correlation was obtained with methods capable of capturing the semantics.\nThis was expected because the golden standards used are themselves based on\nword co-occurrence. Notwithstanding, the Katz similarity, which involves\nsemantic and structure in the comparison of texts, achieved the highest\ncorrelation with the NIST measurement, indicating that in some cases the\ncombination of both approaches can improve the ability to quantify quality in\nMT. In authorship recognition, again the topological features were relevant in\nsome contexts, though for the books and authors analyzed good results were\nobtained with semantic features as well. Because hybrid approaches encompassing\nsemantic and topological features have not been extensively used, we believe\nthat the methodology proposed here may be useful to enhance text classification\nconsiderably, as it combines well-established strategies.",
            "journal ref": "Physica A 391 18 4406-4419, (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.0445v1",
            "date": "2013-03-03 01:21:58+00:00",
            "title": "Detecting and resolving spatial ambiguity in text using named entity extraction and self learning fuzzy logic techniques",
            "authors": [
                "Kanagavalli V R",
                "Raja. K"
            ],
            "abstract": "Information extraction identifies useful and relevant text in a document and\nconverts unstructured text into a form that can be loaded into a database\ntable. Named entity extraction is a main task in the process of information\nextraction and is a classification problem in which words are assigned to one\nor more semantic classes or to a default non-entity class. A word which can\nbelong to one or more classes and which has a level of uncertainty in it can be\nbest handled by a self learning Fuzzy Logic Technique. This paper proposes a\nmethod for detecting the presence of spatial uncertainty in the text and\ndealing with spatial ambiguity using named entity extraction techniques coupled\nwith self learning fuzzy logic techniques",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1303.0489v1",
            "date": "2013-03-03 12:19:18+00:00",
            "title": "A Semantic approach for effective document clustering using WordNet",
            "authors": [
                "Leena H. Patil",
                "Mohammed Atique"
            ],
            "abstract": "Now a days, the text document is spontaneously increasing over the internet,\ne-mail and web pages and they are stored in the electronic database format. To\narrange and browse the document it becomes difficult. To overcome such problem\nthe document preprocessing, term selection, attribute reduction and maintaining\nthe relationship between the important terms using background knowledge,\nWordNet, becomes an important parameters in data mining. In these paper the\ndifferent stages are formed, firstly the document preprocessing is done by\nremoving stop words, stemming is performed using porter stemmer algorithm, word\nnet thesaurus is applied for maintaining relationship between the important\nterms, global unique words, and frequent word sets get generated, Secondly,\ndata matrix is formed, and thirdly terms are extracted from the documents by\nusing term selection approaches tf-idf, tf-df, and tf2 based on their minimum\nthreshold value. Further each and every document terms gets preprocessed, where\nthe frequency of each term within the document is counted for representation.\nThe purpose of this approach is to reduce the attributes and find the effective\nterm selection method using WordNet for better clustering accuracy. Experiments\nare evaluated on Reuters Transcription Subsets, wheat, trade, money grain, and\nship, Reuters 21578, Classic 30, 20 News group (atheism), 20 News group\n(Hardware), 20 News group (Computer Graphics) etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1232v1",
            "date": "2013-03-06 01:30:58+00:00",
            "title": "Japanese-Spanish Thesaurus Construction Using English as a Pivot",
            "authors": [
                "Jessica Ram\u00edrez",
                "Masayuki Asahara",
                "Yuji Matsumoto"
            ],
            "abstract": "We present the results of research with the goal of automatically creating a\nmultilingual thesaurus based on the freely available resources of Wikipedia and\nWordNet. Our goal is to increase resources for natural language processing\ntasks such as machine translation targeting the Japanese-Spanish language pair.\nGiven the scarcity of resources, we use existing English resources as a pivot\nfor creating a trilingual Japanese-Spanish-English thesaurus. Our approach\nconsists of extracting the translation tuples from Wikipedia, disambiguating\nthem by mapping them to WordNet word senses. We present results comparing two\nmethods of disambiguation, the first using VSM on Wikipedia article texts and\nWordNet definitions, and the second using categorical information extracted\nfrom Wikipedia, We find that mixing the two methods produces favorable results.\nUsing the proposed method, we have constructed a multilingual\nSpanish-Japanese-English thesaurus consisting of 25,375 entries. The same\nmethod can be applied to any pair of languages that are linked to English in\nWikipedia.",
            "journal ref": "In Proceeding of The Third International Joint Conference on\n  Natural Language Processing (IJCNLP-08), Hyderabad, India. pages 473-480,\n  2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.1441v2",
            "date": "2013-03-06 20:09:05+00:00",
            "title": "A Hybrid Approach to Extract Keyphrases from Medical Documents",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "Keyphrases are the phrases, consisting of one or more words, representing the\nimportant concepts in the articles. Keyphrases are useful for a variety of\ntasks such as text summarization, automatic indexing,\nclustering/classification, text mining etc. This paper presents a hybrid\napproach to keyphrase extraction from medical documents. The keyphrase\nextraction approach presented in this paper is an amalgamation of two methods:\nthe first one assigns weights to candidate keyphrases based on an effective\ncombination of features such as position, term frequency, inverse document\nfrequency and the second one assign weights to candidate keyphrases using some\nknowledge about their similarities to the structure and characteristics of\nkeyphrases available in the memory (stored list of keyphrases). An efficient\ncandidate keyphrase identification method as the first component of the\nproposed keyphrase extraction system has also been introduced in this paper.\nThe experimental results show that the proposed hybrid approach performs better\nthan some state-of-the art keyphrase extraction approaches.",
            "journal ref": "International Journal of Computer Applications 63(18):14-19,\n  February 2013. Published by Foundation of Computer Science, New York, USA",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1303.1703v1",
            "date": "2013-03-07 14:46:17+00:00",
            "title": "Concept-based indexing in text information retrieval",
            "authors": [
                "Fatiha Boubekeur",
                "Wassila Azzoug"
            ],
            "abstract": "Traditional information retrieval systems rely on keywords to index documents\nand queries. In such systems, documents are retrieved based on the number of\nshared keywords with the query. This lexical-focused retrieval leads to\ninaccurate and incomplete results when different keywords are used to describe\nthe documents and queries. Semantic-focused retrieval approaches attempt to\novercome this problem by relying on concepts rather than on keywords to\nindexing and retrieval. The goal is to retrieve documents that are semantically\nrelevant to a given user query. This paper addresses this issue by proposing a\nsolution at the indexing level. More precisely, we propose a novel approach for\nsemantic indexing based on concepts identified from a linguistic resource. In\nparticular, our approach relies on the joint use of WordNet and WordNetDomains\nlexical databases for concept identification. Furthermore, we propose a\nsemantic-based concept weighting scheme that relies on a novel definition of\nconcept centrality. The resulting system is evaluated on the TIME test\ncollection. Experimental results show the effectiveness of our proposition over\ntraditional IR approaches.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 5, No 1, February 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1303.4293v1",
            "date": "2013-03-11 08:10:39+00:00",
            "title": "A Multilingual Semantic Wiki Based on Attempto Controlled English and Grammatical Framework",
            "authors": [
                "Kaarel Kaljurand",
                "Tobias Kuhn"
            ],
            "abstract": "We describe a semantic wiki system with an underlying controlled natural\nlanguage grammar implemented in Grammatical Framework (GF). The grammar\nrestricts the wiki content to a well-defined subset of Attempto Controlled\nEnglish (ACE), and facilitates a precise bidirectional automatic translation\nbetween ACE and language fragments of a number of other natural languages,\nmaking the wiki content accessible multilingually. Additionally, our approach\nallows for automatic translation into the Web Ontology Language (OWL), which\nenables automatic reasoning over the wiki content. The developed wiki\nenvironment thus allows users to build, query and view OWL knowledge bases via\na user-friendly multilingual natural language interface. As a further feature,\nthe underlying multilingual grammar is integrated into the wiki and can be\ncollaboratively edited to extend the vocabulary of the wiki or even customize\nits sentence structures. This work demonstrates the combination of the existing\ntechnologies of Attempto Controlled English and Grammatical Framework, and is\nimplemented as an extension of the existing semantic wiki engine AceWiki.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.4959v1",
            "date": "2013-03-20 15:12:01+00:00",
            "title": "Analytic solution of a model of language competition with bilingualism and interlinguistic similarity",
            "authors": [
                "Victoria Otero-Espinar",
                "Lu\u00eds F. Seoane",
                "Juan J. Nieto",
                "Jorge Mira"
            ],
            "abstract": "An in-depth analytic study of a model of language dynamics is presented: a\nmodel which tackles the problem of the coexistence of two languages within a\nclosed community of speakers taking into account bilingualism and incorporating\na parameter to measure the distance between languages. After previous numerical\nsimulations, the model yielded that coexistence might lead to survival of both\nlanguages within monolingual speakers along with a bilingual community or to\nextinction of the weakest tongue depending on different parameters. In this\npaper, such study is closed with thorough analytical calculations to settle the\nresults in a robust way and previous results are refined with some\nmodifications. From the present analysis it is possible to almost completely\nassay the number and nature of the equilibrium points of the model, which\ndepend on its parameters, as well as to build a phase space based on them.\nAlso, we obtain conclusions on the way the languages evolve with time. Our\nrigorous considerations also suggest ways to further improve the model and\nfacilitate the comparison of its consequences with those from other approaches\nor with real data.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1303.5148v1",
            "date": "2013-03-21 02:56:43+00:00",
            "title": "Estimating Confusions in the ASR Channel for Improved Topic-based Language Model Adaptation",
            "authors": [
                "Damianos Karakos",
                "Mark Dredze",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Human language is a combination of elemental languages/domains/styles that\nchange across and sometimes within discourses. Language models, which play a\ncrucial role in speech recognizers and machine translation systems, are\nparticularly sensitive to such changes, unless some form of adaptation takes\nplace. One approach to speech language model adaptation is self-training, in\nwhich a language model's parameters are tuned based on automatically\ntranscribed audio. However, transcription errors can misguide self-training,\nparticularly in challenging settings such as conversational speech. In this\nwork, we propose a model that considers the confusions (errors) of the ASR\nchannel. By modeling the likely confusions in the ASR output instead of using\njust the 1-best, we improve self-training efficacy by obtaining a more reliable\nreference transcription estimate. We demonstrate improved topic-based language\nmodeling adaptation results over both 1-best and lattice self-training using\nour ASR channel confusion estimates on telephone conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.5513v1",
            "date": "2013-03-22 04:20:16+00:00",
            "title": "Parameters Optimization for Improving ASR Performance in Adverse Real World Noisy Environmental Conditions",
            "authors": [
                "Urmila Shrawankar",
                "Vilas Thakare"
            ],
            "abstract": "From the existing research it has been observed that many techniques and\nmethodologies are available for performing every step of Automatic Speech\nRecognition (ASR) system, but the performance (Minimization of Word Error\nRecognition-WER and Maximization of Word Accuracy Rate- WAR) of the methodology\nis not dependent on the only technique applied in that method. The research\nwork indicates that, performance mainly depends on the category of the noise,\nthe level of the noise and the variable size of the window, frame, frame\noverlap etc is considered in the existing methods. The main aim of the work\npresented in this paper is to use variable size of parameters like window size,\nframe size and frame overlap percentage to observe the performance of\nalgorithms for various categories of noise with different levels and also train\nthe system for all size of parameters and category of real world noisy\nenvironment to improve the performance of the speech recognition system. This\npaper presents the results of Signal-to-Noise Ratio (SNR) and Accuracy test by\napplying variable size of parameters. It is observed that, it is really very\nhard to evaluate test results and decide parameter size for ASR performance\nimprovement for its resultant optimization. Hence, this study further suggests\nthe feasible and optimum parameter size using Fuzzy Inference System (FIS) for\nenhancing resultant accuracy in adverse real world noisy environmental\nconditions. This work will be helpful to give discriminative training of\nubiquitous ASR system for better Human Computer Interaction (HCI).",
            "journal ref": "International Journal of Human Computer Interaction (IJHCI) 3(3),\n  58-70, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.5515v1",
            "date": "2013-03-22 04:44:37+00:00",
            "title": "Adverse Conditions and ASR Techniques for Robust Speech User Interface",
            "authors": [
                "Urmila Shrawankar",
                "VM Thakare"
            ],
            "abstract": "The main motivation for Automatic Speech Recognition (ASR) is efficient\ninterfaces to computers, and for the interfaces to be natural and truly useful,\nit should provide coverage for a large group of users. The purpose of these\ntasks is to further improve man-machine communication. ASR systems exhibit\nunacceptable degradations in performance when the acoustical environments used\nfor training and testing the system are not the same. The goal of this research\nis to increase the robustness of the speech recognition systems with respect to\nchanges in the environment. A system can be labeled as environment-independent\nif the recognition accuracy for a new environment is the same or higher than\nthat obtained when the system is retrained for that environment. Attaining such\nperformance is the dream of the researchers. This paper elaborates some of the\ndifficulties with Automatic Speech Recognition (ASR). These difficulties are\nclassified into Speakers characteristics and environmental conditions, and\ntried to suggest some techniques to compensate variations in speech signal.\nThis paper focuses on the robustness with respect to speakers variations and\nchanges in the acoustical environment. We discussed several different external\nfactors that change the environment and physiological differences that affect\nthe performance of a speech recognition system followed by techniques that are\nhelpful to design a robust ASR system.",
            "journal ref": "International Journal of Computer Science Issues (IJCSI), 8(5),\n  440-449, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.5778v1",
            "date": "2013-03-22 20:55:48+00:00",
            "title": "Speech Recognition with Deep Recurrent Neural Networks",
            "authors": [
                "Alex Graves",
                "Abdel-rahman Mohamed",
                "Geoffrey Hinton"
            ],
            "abstract": "Recurrent neural networks (RNNs) are a powerful model for sequential data.\nEnd-to-end training methods such as Connectionist Temporal Classification make\nit possible to train RNNs for sequence labelling problems where the\ninput-output alignment is unknown. The combination of these methods with the\nLong Short-term Memory RNN architecture has proved particularly fruitful,\ndelivering state-of-the-art results in cursive handwriting recognition. However\nRNN performance in speech recognition has so far been disappointing, with\nbetter results returned by deep feedforward networks. This paper investigates\n\\emph{deep recurrent neural networks}, which combine the multiple levels of\nrepresentation that have proved so effective in deep networks with the flexible\nuse of long range context that empowers RNNs. When trained end-to-end with\nsuitable regularisation, we find that deep Long Short-term Memory RNNs achieve\na test set error of 17.7% on the TIMIT phoneme recognition benchmark, which to\nour knowledge is the best recorded score.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1303.7310v1",
            "date": "2013-03-29 06:31:40+00:00",
            "title": "Exploring the Role of Logically Related Non-Question Phrases for Answering Why-Questions",
            "authors": [
                "Niraj Kumar",
                "Rashmi Gangadharaiah",
                "Kannan Srinathan",
                "Vasudeva Varma"
            ],
            "abstract": "In this paper, we show that certain phrases although not present in a given\nquestion/query, play a very important role in answering the question. Exploring\nthe role of such phrases in answering questions not only reduces the dependency\non matching question phrases for extracting answers, but also improves the\nquality of the extracted answers. Here matching question phrases means phrases\nwhich co-occur in given question and candidate answers. To achieve the above\ndiscussed goal, we introduce a bigram-based word graph model populated with\nsemantic and topical relatedness of terms in the given document. Next, we apply\nan improved version of ranking with a prior-based approach, which ranks all\nwords in the candidate document with respect to a set of root words (i.e.\nnon-stopwords present in the question and in the candidate document). As a\nresult, terms logically related to the root words are scored higher than terms\nthat are not related to the root words. Experimental results show that our\ndevised system performs better than state-of-the-art for the task of answering\nWhy-questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.1662v2",
            "date": "2013-07-05 16:52:09+00:00",
            "title": "Polyglot: Distributed Word Representations for Multilingual NLP",
            "authors": [
                "Rami Al-Rfou",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "Distributed word representations (word embeddings) have recently contributed\nto competitive performance in language modeling and several NLP tasks. In this\nwork, we train word embeddings for more than 100 languages using their\ncorresponding Wikipedias. We quantitatively demonstrate the utility of our word\nembeddings by using them as the sole features for training a part of speech\ntagger for a subset of these languages. We find their performance to be\ncompetitive with near state-of-art methods in English, Danish and Swedish.\nMoreover, we investigate the semantic features captured by these embeddings\nthrough the proximity of word groupings. We will release these embeddings\npublicly to help researchers in the development and enhancement of multilingual\napplications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.3336v1",
            "date": "2013-07-12 06:20:36+00:00",
            "title": "Opinion Mining and Analysis: A survey",
            "authors": [
                "Arti Buche",
                "Dr. M. B. Chandak",
                "Akshay Zadgaonkar"
            ],
            "abstract": "The current research is focusing on the area of Opinion Mining also called as\nsentiment analysis due to sheer volume of opinion rich web resources such as\ndiscussion forums, review sites and blogs are available in digital form. One\nimportant problem in sentiment analysis of product reviews is to produce\nsummary of opinions based on product features. We have surveyed and analyzed in\nthis paper, various techniques that have been developed for the key tasks of\nopinion mining. We have provided an overall picture of what is involved in\ndeveloping a software system for opinion mining on the basis of our survey and\nanalysis.",
            "journal ref": "IJNLC Vol. 2, No.3, June 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.3489v1",
            "date": "2013-07-11 10:21:01+00:00",
            "title": "Genetic approach for arabic part of speech tagging",
            "authors": [
                "Bilel Ben Ali",
                "Fethi Jarray"
            ],
            "abstract": "With the growing number of textual resources available, the ability to\nunderstand them becomes critical. An essential first step in understanding\nthese sources is the ability to identify the part of speech in each sentence.\nArabic is a morphologically rich language, wich presents a challenge for part\nof speech tagging. In this paper, our goal is to propose, improve and implement\na part of speech tagger based on a genetic alorithm. The accuracy obtained with\nthis method is comparable to that of other probabilistic approaches.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  2, No.3, June 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.4879v2",
            "date": "2013-07-18 09:37:45+00:00",
            "title": "Says who? Automatic Text-Based Content Analysis of Television News",
            "authors": [
                "Carlos Castillo",
                "Gianmarco De Francisci Morales",
                "Marcelo Mendoza",
                "Nasir Khan"
            ],
            "abstract": "We perform an automatic analysis of television news programs, based on the\nclosed captions that accompany them. Specifically, we collect all the news\nbroadcasted in over 140 television channels in the US during a period of six\nmonths. We start by segmenting, processing, and annotating the closed captions\nautomatically. Next, we focus on the analysis of their linguistic style and on\nmentions of people using NLP methods. We present a series of key insights about\nnews providers, people in the news, and we discuss the biases that can be\nuncovered by automatic means. These insights are contrasted by looking at the\ndata from multiple points of view, including qualitative assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.6235v5",
            "date": "2013-07-18 11:03:14+00:00",
            "title": "Graphical law beneath each written natural language",
            "authors": [
                "Anindya Kumar Biswas"
            ],
            "abstract": "We study twenty four written natural languages. We draw in the log scale,\nnumber of words starting with a letter vs rank of the letter, both normalised.\nWe find that all the graphs are of the similar type. The graphs are\ntantalisingly closer to the curves of reduced magnetisation vs reduced\ntemperature for magnetic materials. We make a weak conjecture that a curve of\nmagnetisation underlies a written natural language.",
            "journal ref": "None",
            "category": "physics.gen-ph"
        },
        {
            "url": "http://arxiv.org/abs/1307.6937v1",
            "date": "2013-07-26 06:57:37+00:00",
            "title": "A Novel Architecture For Question Classification Based Indexing Scheme For Efficient Question Answering",
            "authors": [
                "Renu Mudgal",
                "Rosy Madaan",
                "A. K. Sharma",
                "Ashutosh Dixit"
            ],
            "abstract": "Question answering system can be seen as the next step in information\nretrieval, allowing users to pose question in natural language and receive\ncompact answers. For the Question answering system to be successful, research\nhas shown that the correct classification of question with respect to the\nexpected answer type is requisite. We propose a novel architecture for question\nclassification and searching in the index, maintained on the basis of expected\nanswer types, for efficient question answering. The system uses the criteria\nfor Answer Relevance Score for finding the relevance of each answer returned by\nthe system. On analysis of the proposed system, it has been found that the\nsystem has shown promising results than the existing systems based on question\nclassification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1307.8057v1",
            "date": "2013-07-30 17:27:29+00:00",
            "title": "Extracting Connected Concepts from Biomedical Texts using Fog Index",
            "authors": [
                "Rushdi Shams",
                "Robert E. Mercer"
            ],
            "abstract": "In this paper, we establish Fog Index (FI) as a text filter to locate the\nsentences in texts that contain connected biomedical concepts of interest. To\ndo so, we have used 24 random papers each containing four pairs of connected\nconcepts. For each pair, we categorize sentences based on whether they contain\nboth, any or none of the concepts. We then use FI to measure difficulty of the\nsentences of each category and find that sentences containing both of the\nconcepts have low readability. We rank sentences of a text according to their\nFI and select 30 percent of the most difficult sentences. We use an association\nmatrix to track the most frequent pairs of concepts in them. This matrix\nreports that the first filter produces some pairs that hold almost no\nconnections. To remove these unwanted pairs, we use the Equally Weighted\nHarmonic Mean of their Positive Predictive Value (PPV) and Sensitivity as a\nsecond filter. Experimental results demonstrate the effectiveness of our\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.8060v1",
            "date": "2013-07-30 17:36:53+00:00",
            "title": "Extracting Information-rich Part of Texts using Text Denoising",
            "authors": [
                "Rushdi Shams"
            ],
            "abstract": "The aim of this paper is to report on a novel text reduction technique,\ncalled Text Denoising, that highlights information-rich content when processing\na large volume of text data, especially from the biomedical domain. The core\nfeature of the technique, the text readability index, embodies the hypothesis\nthat complex text is more information-rich than the rest. When applied on tasks\nlike biomedical relation bearing text extraction, keyphrase indexing and\nextracting sentences describing protein interactions, it is evident that the\nreduced set of text produced by text denoising is more information-rich than\nthe rest.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1307.8225v1",
            "date": "2013-07-31 05:40:59+00:00",
            "title": "A Novel Architecture for Relevant Blog Page Identifcation",
            "authors": [
                "Deepti Kapri",
                "Rosy Madaan",
                "A. K Sharma",
                "Ashutosh Dixit"
            ],
            "abstract": "Blogs are undoubtedly the richest source of information available in\ncyberspace. Blogs can be of various natures i.e. personal blogs which contain\nposts on mixed issues or blogs can be domain specific which contains posts on\nparticular topics, this is the reason, they offer wide variety of relevant\ninformation which is often focused. A general search engine gives back a huge\ncollection of web pages which may or may not give correct answers, as web is\nthe repository of information of all kinds and a user has to go through various\ndocuments before he gets what he was originally looking for, which is a very\ntime consuming process. So, the search can be made more focused and accurate if\nit is limited to blogosphere instead of web pages. The reason being that the\nblogs are more focused in terms of information. So, User will only get related\nblogs in response to his query. These results will be then ranked according to\nour proposed method and are finally presented in front of user in descending\norder",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1310.0201v2",
            "date": "2013-10-01 09:20:21+00:00",
            "title": "Cross-Recurrence Quantification Analysis of Categorical and Continuous Time Series: an R package",
            "authors": [
                "Moreno I. Coco",
                "Rick Dale"
            ],
            "abstract": "This paper describes the R package crqa to perform cross-recurrence\nquantification analysis of two time series of either a categorical or\ncontinuous nature. Streams of behavioral information, from eye movements to\nlinguistic elements, unfold over time. When two people interact, such as in\nconversation, they often adapt to each other, leading these behavioral levels\nto exhibit recurrent states. In dialogue, for example, interlocutors adapt to\neach other by exchanging interactive cues: smiles, nods, gestures, choice of\nwords, and so on. In order for us to capture closely the goings-on of dynamic\ninteraction, and uncover the extent of coupling between two individuals, we\nneed to quantify how much recurrence is taking place at these levels. Methods\navailable in crqa would allow researchers in cognitive science to pose such\nquestions as how much are two people recurrent at some level of analysis, what\nis the characteristic lag time for one person to maximally match another, or\nwhether one person is leading another. First, we set the theoretical ground to\nunderstand the difference between 'correlation' and 'co-visitation' when\ncomparing two time series, using an aggregative or cross-recurrence approach.\nThen, we describe more formally the principles of cross-recurrence, and show\nwith the current package how to carry out analyses applying them. We end the\npaper by comparing computational efficiency, and results' consistency, of crqa\nR package, with the benchmark MATLAB toolbox crptoolbox. We show perfect\ncomparability between the two libraries on both levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1597v1",
            "date": "2013-10-06 16:34:30+00:00",
            "title": "Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning",
            "authors": [
                "Mengqiu Wang",
                "Christopher D. Manning"
            ],
            "abstract": "We consider a multilingual weakly supervised learning scenario where\nknowledge from annotated corpora in a resource-rich language is transferred via\nbitext to guide the learning in other languages. Past approaches project labels\nacross bitext and use them as features or gold labels for training. We propose\na new method that projects model expectations rather than labels, which\nfacilities transfer of model uncertainty across language boundaries. We encode\nexpectations as constraints and train a discriminative CRF model using\nGeneralized Expectation Criteria (Mann and McCallum, 2010). Evaluated on\nstandard Chinese-English and German-English NER datasets, our method\ndemonstrates F1 scores of 64% and 60% when no labeled data is used. Attaining\nthe same accuracy with supervised CRFs requires 12k and 1.5k labeled sentences.\nFurthermore, when combined with labeled examples, our method yields significant\nimprovements over state-of-the-art supervised methods, achieving best reported\nnumbers to date on Chinese OntoNotes and German CoNLL-03 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.2527v1",
            "date": "2013-10-08 17:19:06+00:00",
            "title": "Treating clitics with minimalist grammars",
            "authors": [
                "Maxime Amblard"
            ],
            "abstract": "We propose an extension of Stabler's version of clitics treatment for a wider\ncoverage of the French language. For this, we present the lexical entries\nneeded in the lexicon. Then, we show the recognition of complex syntactic\nphenomena as (left and right) dislo- cation, clitic climbing over modal and\nextraction from determiner phrase. The aim of this presentation is the\nsyntax-semantic interface for clitics analyses in which we will stress on\nclitic climbing over verb and raising verb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.5963v1",
            "date": "2013-10-22 15:35:46+00:00",
            "title": "Improving the methods of email classification based on words ontology",
            "authors": [
                "Foruzan Kiamarzpour",
                "Rouhollah Dianat",
                "Mohammad bahrani",
                "Mehdi Sadeghzadeh"
            ],
            "abstract": "The Internet has dramatically changed the relationship among people and their\nrelationships with others people and made the valuable information available\nfor the users. Email is the service, which the Internet provides today for its\nown users; this service has attracted most of the users' attention due to the\nlow cost. Along with the numerous benefits of Email, one of the weaknesses of\nthis service is that the number of received emails is continually being\nenhanced, thus the ways are needed to automatically filter these disturbing\nletters. Most of these filters utilize a combination of several techniques such\nas the Black or white List, using the keywords and so on in order to identify\nthe spam more accurately In this paper, we introduce a new method to classify\nthe spam. We are seeking to increase the accuracy of Email classification by\ncombining the output of several decision trees and the concept of ontology.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1302.0393v1",
            "date": "2013-02-02 15:57:26+00:00",
            "title": "Lambek vs. Lambek: Functorial Vector Space Semantics and String Diagrams for Lambek Calculus",
            "authors": [
                "Bob Coecke",
                "Edward Grefenstette",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "The Distributional Compositional Categorical (DisCoCat) model is a\nmathematical framework that provides compositional semantics for meanings of\nnatural language sentences. It consists of a computational procedure for\nconstructing meanings of sentences, given their grammatical structure in terms\nof compositional type-logic, and given the empirically derived meanings of\ntheir words. For the particular case that the meaning of words is modelled\nwithin a distributional vector space model, its experimental predictions,\nderived from real large scale data, have outperformed other empirically\nvalidated methods that could build vectors for a full sentence. This success\ncan be attributed to a conceptually motivated mathematical underpinning, by\nintegrating qualitative compositional type-logic and quantitative modelling of\nmeaning within a category-theoretic mathematical framework.\n  The type-logic used in the DisCoCat model is Lambek's pregroup grammar.\nPregroup types form a posetal compact closed category, which can be passed, in\na functorial manner, on to the compact closed structure of vector spaces,\nlinear maps and tensor product. The diagrammatic versions of the equational\nreasoning in compact closed categories can be interpreted as the flow of word\nmeanings within sentences. Pregroups simplify Lambek's previous type-logic, the\nLambek calculus, which has been extensively used to formalise and reason about\nvarious linguistic phenomena. The apparent reliance of the DisCoCat on\npregroups has been seen as a shortcoming. This paper addresses this concern, by\npointing out that one may as well realise a functorial passage from the\noriginal type-logic of Lambek, a monoidal bi-closed category, to vector spaces,\nor to any other model of meaning organised within a monoidal bi-closed\ncategory. The corresponding string diagram calculus, due to Baez and Stay, now\ndepicts the flow of word meanings.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1302.2131v1",
            "date": "2013-02-08 19:56:43+00:00",
            "title": "Data Mining of the Concept \"End of the World\" in Twitter Microblogs",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "This paper describes the analysis of quantitative characteristics of frequent\nsets and association rules in the posts of Twitter microblogs, related to the\ndiscussion of \"end of the world\", which was allegedly predicted on December 21,\n2012 due to the Mayan calendar. Discovered frequent sets and association rules\ncharacterize semantic relations between the concepts of analyzed subjects.The\nsupport for some fequent sets reaches the global maximum before the expected\nevent with some time delay. Such frequent sets may be considered as predictive\nmarkers that characterize the significance of expected events for blogosphere\nusers. It was shown that time dynamics of confidence of some revealed\nassociation rules can also have predictive characteristics. Exceeding a certain\nthreshold, it may be a signal for the corresponding reaction in the society\nduring the time interval between the maximum and probable coming of an event.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1302.2569v1",
            "date": "2013-02-11 18:51:03+00:00",
            "title": "Toric grammars: a new statistical approach to natural language modeling",
            "authors": [
                "Olivier Catoni",
                "Thomas Mainguy"
            ],
            "abstract": "We propose a new statistical model for computational linguistics. Rather than\ntrying to estimate directly the probability distribution of a random sentence\nof the language, we define a Markov chain on finite sets of sentences with many\nfinite recurrent communicating classes and define our language model as the\ninvariant probability measures of the chain on each recurrent communicating\nclass. This Markov chain, that we call a communication model, recombines at\neach step randomly the set of sentences forming its current state, using some\ngrammar rules. When the grammar rules are fixed and known in advance instead of\nbeing estimated on the fly, we can prove supplementary mathematical properties.\nIn particular, we can prove in this case that all states are recurrent states,\nso that the chain defines a partition of its state space into finite recurrent\ncommunicating classes. We show that our approach is a decisive departure from\nMarkov models at the sentence level and discuss its relationships with Context\nFree Grammars. Although the toric grammars we use are closely related to\nContext Free Grammars, the way we generate the language from the grammar is\nqualitatively different. Our communication model has two purposes. On the one\nhand, it is used to define indirectly the probability distribution of a random\nsentence of the language. On the other hand it can serve as a (crude) model of\nlanguage transmission from one speaker to another speaker through the\ncommunication of a (large) set of sentences.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1302.3831v2",
            "date": "2013-02-15 18:20:25+00:00",
            "title": "Quantum Entanglement in Concept Combinations",
            "authors": [
                "Diederik Aerts",
                "Sandro Sozzo"
            ],
            "abstract": "Research in the application of quantum structures to cognitive science\nconfirms that these structures quite systematically appear in the dynamics of\nconcepts and their combinations and quantum-based models faithfully represent\nexperimental data of situations where classical approaches are problematical.\nIn this paper, we analyze the data we collected in an experiment on a specific\nconceptual combination, showing that Bell's inequalities are violated in the\nexperiment. We present a new refined entanglement scheme to model these data\nwithin standard quantum theory rules, where 'entangled measurements and\nentangled evolutions' occur, in addition to the expected 'entangled states',\nand present a full quantum representation in complex Hilbert space of the data.\nThis stronger form of entanglement in measurements and evolutions might have\nrelevant applications in the foundations of quantum theory, as well as in the\ninterpretation of nonlocality tests. It could indeed explain some\nnon-negligible 'anomalies' identified in EPR-Bell experiments.",
            "journal ref": "International Journal of Theoretical Physics, 53, pp. 3587-3603,\n  2014",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1302.3892v1",
            "date": "2013-02-15 21:22:10+00:00",
            "title": "Identifying trends in word frequency dynamics",
            "authors": [
                "Eduardo G. Altmann",
                "Zakary L. Whichard",
                "Adilson E. Motter"
            ],
            "abstract": "The word-stock of a language is a complex dynamical system in which words can\nbe created, evolve, and become extinct. Even more dynamic are the short-term\nfluctuations in word usage by individuals in a population. Building on the\nrecent demonstration that word niche is a strong determinant of future rise or\nfall in word frequency, here we introduce a model that allows us to distinguish\npersistent from temporary increases in frequency. Our model is illustrated\nusing a 10^8-word database from an online discussion group and a 10^11-word\ncollection of digitized books. The model reveals a strong relation between\nchanges in word dissemination and changes in frequency. Aside from their\nimplications for short-term word frequency dynamics, these observations are\npotentially important for language evolution as new words must survive in the\nshort term in order to survive in the long term.",
            "journal ref": "J. Stat. Phys. 151, p. 277 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1302.4383v1",
            "date": "2013-02-18 18:38:05+00:00",
            "title": "Explaining Zipf's Law via Mental Lexicon",
            "authors": [
                "Armen E. Allahverdyan",
                "Weibing Deng",
                "Q. A. Wang"
            ],
            "abstract": "The Zipf's law is the major regularity of statistical linguistics that served\nas a prototype for rank-frequency relations and scaling laws in natural\nsciences. Here we show that the Zipf's law -- together with its applicability\nfor a single text and its generalizations to high and low frequencies including\nhapax legomena -- can be derived from assuming that the words are drawn into\nthe text with random probabilities. Their apriori density relates, via the\nBayesian statistics, to general features of the mental lexicon of the author\nwho produced the text.",
            "journal ref": "None",
            "category": "physics.data-an"
        },
        {
            "url": "http://arxiv.org/abs/1302.4465v1",
            "date": "2013-02-18 21:34:59+00:00",
            "title": "Unveiling the relationship between complex networks metrics and word senses",
            "authors": [
                "Diego R. Amancio",
                "Osvaldo N. Oliveira Jr.",
                "Luciano da F. Costa"
            ],
            "abstract": "The automatic disambiguation of word senses (i.e., the identification of\nwhich of the meanings is used in a given context for a word that has multiple\nmeanings) is essential for such applications as machine translation and\ninformation retrieval, and represents a key step for developing the so-called\nSemantic Web. Humans disambiguate words in a straightforward fashion, but this\ndoes not apply to computers. In this paper we address the problem of Word Sense\nDisambiguation (WSD) by treating texts as complex networks, and show that word\nsenses can be distinguished upon characterizing the local structure around\nambiguous words. Our goal was not to obtain the best possible disambiguation\nsystem, but we nevertheless found that in half of the cases our approach\noutperforms traditional shallow methods. We show that the hierarchical\nconnectivity and clustering of words are usually the most relevant features for\nWSD. The results reported here shine light on the relationship between semantic\nand structural parameters of complex networks. They also indicate that when\ncombined with traditional techniques the complex network approach may be useful\nto enhance the discrimination of senses in large texts",
            "journal ref": "Europhysics Letters (2012) 98 18002",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1302.4471v1",
            "date": "2013-02-18 22:06:52+00:00",
            "title": "Word sense disambiguation via high order of learning in complex networks",
            "authors": [
                "Thiago C. Silva",
                "Diego R. Amancio"
            ],
            "abstract": "Complex networks have been employed to model many real systems and as a\nmodeling tool in a myriad of applications. In this paper, we use the framework\nof complex networks to the problem of supervised classification in the word\ndisambiguation task, which consists in deriving a function from the supervised\n(or labeled) training data of ambiguous words. Traditional supervised data\nclassification takes into account only topological or physical features of the\ninput data. On the other hand, the human (animal) brain performs both low- and\nhigh-level orders of learning and it has facility to identify patterns\naccording to the semantic meaning of the input data. In this paper, we apply a\nhybrid technique which encompasses both types of learning in the field of word\nsense disambiguation and show that the high-level order of learning can really\nimprove the accuracy rate of the model. This evidence serves to demonstrate\nthat the internal structures formed by the words do present patterns that,\ngenerally, cannot be correctly unveiled by only traditional techniques.\nFinally, we exhibit the behavior of the model for different weights of the low-\nand high-level classifiers by plotting decision boundaries. This study helps\none to better understand the effectiveness of the model.",
            "journal ref": "Europhysics Letters (2012) 98 58001",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1302.4490v1",
            "date": "2013-02-19 00:32:22+00:00",
            "title": "Complex networks analysis of language complexity",
            "authors": [
                "Diego R. Amancio",
                "Sandra M. Aluisio",
                "Osvaldo N. Oliveira Jr.",
                "Luciano da F. Costa"
            ],
            "abstract": "Methods from statistical physics, such as those involving complex networks,\nhave been increasingly used in quantitative analysis of linguistic phenomena.\nIn this paper, we represented pieces of text with different levels of\nsimplification in co-occurrence networks and found that topological regularity\ncorrelated negatively with textual complexity. Furthermore, in less complex\ntexts the distance between concepts, represented as nodes, tended to decrease.\nThe complex networks metrics were treated with multivariate pattern recognition\ntechniques, which allowed us to distinguish between original texts and their\nsimplified versions. For each original text, two simplified versions were\ngenerated manually with increasing number of simplification operations. As\nexpected, distinction was easier for the strongly simplified versions, where\nthe most relevant metrics were node strength, shortest paths and diversity.\nAlso, the discrimination of complex texts was improved with higher hierarchical\nnetwork metrics, thus pointing to the usefulness of considering wider contexts\naround the concepts. Though the accuracy rate in the distinction was not as\nhigh as in methods using deep linguistic knowledge, the complex network\napproach is still useful for a rapid screening of texts whenever assessing\ncomplexity is essential to guarantee accessibility to readers with limited\nreading ability",
            "journal ref": "Europhysics Letters (2012) 100 58002",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1302.4726v1",
            "date": "2013-02-19 20:09:08+00:00",
            "title": "An Ontology for Modelling and Supporting the Process of Authoring Technical Assessments",
            "authors": [
                "Khalil Riad Bouzidi",
                "Bruno Fies",
                "Marc Bourdeau",
                "Catherine Faron-Zucker",
                "Nhan Le-Thanh"
            ],
            "abstract": "In this paper, we present a semantic web approach for modelling the process\nof creating new technical and regulatory documents related to the Building\nsector. This industry, among other industries, is currently experiencing a\nphenomenal growth in its technical and regulatory texts. Therefore, it is\nurgent and crucial to improve the process of creating regulations by automating\nit as much as possible. We focus on the creation of particular technical\ndocuments issued by the French Scientific and Technical Centre for Building\n(CSTB), called Technical Assessments, and we propose services based on Semantic\nWeb models and techniques for modelling the process of their creation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1302.5526v2",
            "date": "2013-02-22 09:23:50+00:00",
            "title": "Stochastic dynamics of lexicon learning in an uncertain and nonuniform world",
            "authors": [
                "Rainer Reisenauer",
                "Kenny Smith",
                "Richard A. Blythe"
            ],
            "abstract": "We study the time taken by a language learner to correctly identify the\nmeaning of all words in a lexicon under conditions where many plausible\nmeanings can be inferred whenever a word is uttered. We show that the most\nbasic form of cross-situational learning - whereby information from multiple\nepisodes is combined to eliminate incorrect meanings - can perform badly when\nwords are learned independently and meanings are drawn from a nonuniform\ndistribution. If learners further assume that no two words share a common\nmeaning, we find a phase transition between a maximally-efficient learning\nregime, where the learning time is reduced to the shortest it can possibly be,\nand a partially-efficient regime where incorrect candidate meanings for words\npersist at late times. We obtain exact results for the word-learning process\nthrough an equivalence to a statistical mechanical problem of enumerating loops\nin the space of word-meaning mappings.",
            "journal ref": "Phys Rev Lett (2013) 110 258701",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1302.6334v1",
            "date": "2013-02-26 06:50:08+00:00",
            "title": "Non-simplifying Graph Rewriting Termination",
            "authors": [
                "Guillaume Bonfante",
                "Bruno Guillaume"
            ],
            "abstract": "So far, a very large amount of work in Natural Language Processing (NLP) rely\non trees as the core mathematical structure to represent linguistic\ninformations (e.g. in Chomsky's work). However, some linguistic phenomena do\nnot cope properly with trees. In a former paper, we showed the benefit of\nencoding linguistic structures by graphs and of using graph rewriting rules to\ncompute on those structures. Justified by some linguistic considerations, graph\nrewriting is characterized by two features: first, there is no node creation\nalong computations and second, there are non-local edge modifications. Under\nthese hypotheses, we show that uniform termination is undecidable and that\nnon-uniform termination is decidable. We describe two termination techniques\nbased on weights and we give complexity bound on the derivation length for\nthese rewriting system.",
            "journal ref": "EPTCS 110, 2013, pp. 4-16",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1302.7056v1",
            "date": "2013-02-28 02:10:38+00:00",
            "title": "KSU KDD: Word Sense Induction by Clustering in Topic Space",
            "authors": [
                "Wesam Elshamy",
                "Doina Caragea",
                "William Hsu"
            ],
            "abstract": "We describe our language-independent unsupervised word sense induction\nsystem. This system only uses topic features to cluster different word senses\nin their global context topic space. Using unlabeled data, this system trains a\nlatent Dirichlet allocation (LDA) topic model then uses it to infer the topics\ndistribution of the test instances. By clustering these topics distributions in\ntheir topic space we cluster them into different senses. Our hypothesis is that\ncloseness in topic space reflects similarity between different word senses.\nThis system participated in SemEval-2 word sense induction and disambiguation\ntask and achieved the second highest V-measure score among all other systems.",
            "journal ref": "Proceedings of the 5th International Workshop on Semantic\n  Evaluation, pages 367-370, Uppsala, Sweden, July 2010. Association for\n  Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.1292v1",
            "date": "2013-08-06 14:58:25+00:00",
            "title": "Science Fiction as a Worldwide Phenomenon: A Study of International Creation, Consumption and Dissemination",
            "authors": [
                "Elysia Wells"
            ],
            "abstract": "This paper examines the international nature of science fiction. The focus of\nthis research is to determine whether science fiction is primarily English\nspeaking and Western or global; being created and consumed by people in\nnon-Western, non-English speaking countries? Science fiction's international\npresence was found in three ways, by network analysis, by examining a online\nretailer and with a survey. Condor, a program developed by GalaxyAdvisors was\nused to determine if science fiction is being talked about by non-English\nspeakers. An analysis of the international Amazon.com websites was done to\ndiscover if it was being consumed worldwide. A survey was also conducted to see\nif people had experience with science fiction. All three research methods\nrevealed similar results. Science fiction was found to be international, with\nscience fiction creators originating in different countries and writing in a\nhost of different languages. English and non-English science fiction was being\ncreated and consumed all over the world, not just in the English speaking West.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1308.1847v2",
            "date": "2013-08-08 13:31:15+00:00",
            "title": "The Royal Birth of 2013: Analysing and Visualising Public Sentiment in the UK Using Twitter",
            "authors": [
                "Vu Dung Nguyen",
                "Blesson Varghese",
                "Adam Barker"
            ],
            "abstract": "Analysis of information retrieved from microblogging services such as Twitter\ncan provide valuable insight into public sentiment in a geographic region. This\ninsight can be enriched by visualising information in its geographic context.\nTwo underlying approaches for sentiment analysis are dictionary-based and\nmachine learning. The former is popular for public sentiment analysis, and the\nlatter has found limited use for aggregating public sentiment from Twitter\ndata. The research presented in this paper aims to extend the machine learning\napproach for aggregating public sentiment. To this end, a framework for\nanalysing and visualising public sentiment from a Twitter corpus is developed.\nA dictionary-based approach and a machine learning approach are implemented\nwithin the framework and compared using one UK case study, namely the royal\nbirth of 2013. The case study validates the feasibility of the framework for\nanalysis and rapid visualisation. One observation is that there is good\ncorrelation between the results produced by the popular dictionary-based\napproach and the machine learning approach when large volumes of tweets are\nanalysed. However, for rapid analysis to be possible faster methods need to be\ndeveloped using big data techniques and parallel methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.2359v1",
            "date": "2013-08-11 02:28:11+00:00",
            "title": "Exploratory Analysis of Highly Heterogeneous Document Collections",
            "authors": [
                "Arun S. Maiya",
                "John P. Thompson",
                "Francisco Loaiza-Lemos",
                "Robert M. Rolfe"
            ],
            "abstract": "We present an effective multifaceted system for exploratory analysis of\nhighly heterogeneous document collections. Our system is based on intelligently\ntagging individual documents in a purely automated fashion and exploiting these\ntags in a powerful faceted browsing framework. Tagging strategies employed\ninclude both unsupervised and supervised approaches based on machine learning\nand natural language processing. As one of our key tagging strategies, we\nintroduce the KERA algorithm (Keyword Extraction for Reports and Articles).\nKERA extracts topic-representative terms from individual documents in a purely\nunsupervised fashion and is revealed to be significantly more effective than\nstate-of-the-art methods. Finally, we evaluate our system in its ability to\nhelp users locate documents pertaining to military critical technologies buried\ndeep in a large heterogeneous sea of information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.3243v1",
            "date": "2013-08-14 20:15:44+00:00",
            "title": "Arabic Text Recognition in Video Sequences",
            "authors": [
                "M. Ben Halima",
                "H. Karray",
                "A. M. Alimi"
            ],
            "abstract": "In this paper, we propose a robust approach for text extraction and\nrecognition from Arabic news video sequence. The text included in video\nsequences is an important needful for indexing and searching system. However,\nthis text is difficult to detect and recognize because of the variability of\nits size, their low resolution characters and the complexity of the\nbackgrounds. To solve these problems, we propose a system performing in two\nmain tasks: extraction and recognition of text. Our system is tested on a\nvaried database composed of different Arabic news programs and the obtained\nresults are encouraging and show the merits of our approach.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1308.3294v1",
            "date": "2013-08-15 02:44:12+00:00",
            "title": "A Secure and Comparable Text Encryption Algorithm",
            "authors": [
                "Nicholas Kersting"
            ],
            "abstract": "This paper discloses a simple algorithm for encrypting text messages, based\non the NP-completeness of the subset sum problem, such that the similarity\nbetween encryptions is roughly proportional to the semantic similarity between\ntheir generating messages. This allows parties to compare encrypted messages\nfor semantic overlap without trusting an intermediary and might be applied, for\nexample, as a means of finding scientific collaborators over the Internet.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1308.3830v1",
            "date": "2013-08-18 04:22:40+00:00",
            "title": "Natural Language Web Interface for Database (NLWIDB)",
            "authors": [
                "Rukshan Alexander",
                "Prashanthi Rukshan",
                "Sinnathamby Mahesan"
            ],
            "abstract": "It is a long term desire of the computer users to minimize the communication\ngap between the computer and a human. On the other hand, almost all ICT\napplications store information in to databases and retrieve from them.\nRetrieving information from the database requires knowledge of technical\nlanguages such as Structured Query Language. However majority of the computer\nusers who interact with the databases do not have a technical background and\nare intimidated by the idea of using languages such as SQL. For above reasons,\na Natural Language Web Interface for Database (NLWIDB) has been developed. The\nNLWIDB allows the user to query the database in a language more like English,\nthrough a convenient interface over the Internet.",
            "journal ref": "Proceedings of the Third International Symposium, SEUSL: 6-7 July\n  2013, Oluvil, Sri Lanka",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.4189v2",
            "date": "2013-08-19 23:28:47+00:00",
            "title": "Seeing What You're Told: Sentence-Guided Activity Recognition In Video",
            "authors": [
                "N. Siddharth",
                "Andrei Barbu",
                "Jeffrey Mark Siskind"
            ],
            "abstract": "We present a system that demonstrates how the compositional structure of\nevents, in concert with the compositional structure of language, can interplay\nwith the underlying focusing mechanisms in video action recognition, thereby\nproviding a medium, not only for top-down and bottom-up integration, but also\nfor multi-modal integration between vision and language. We show how the roles\nplayed by participants (nouns), their characteristics (adjectives), the actions\nperformed (verbs), the manner of such actions (adverbs), and changing spatial\nrelations between participants (prepositions) in the form of whole sentential\ndescriptions mediated by a grammar, guides the activity-recognition process.\nFurther, the utility and expressiveness of our framework is demonstrated by\nperforming three separate tasks in the domain of multi-activity videos:\nsentence-guided focus of attention, generation of sentential descriptions of\nvideo, and query-based video search, simply by leveraging the framework in\ndifferent manners.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1308.4618v1",
            "date": "2013-08-21 15:49:43+00:00",
            "title": "Can inferred provenance and its visualisation be used to detect erroneous annotation? A case study using UniProtKB",
            "authors": [
                "Michael J. Bell",
                "Matthew Collison",
                "Phillip Lord"
            ],
            "abstract": "A constant influx of new data poses a challenge in keeping the annotation in\nbiological databases current. Most biological databases contain significant\nquantities of textual annotation, which often contains the richest source of\nknowledge. Many databases reuse existing knowledge, during the curation process\nannotations are often propagated between entries. However, this is often not\nmade explicit. Therefore, it can be hard, potentially impossible, for a reader\nto identify where an annotation originated from. Within this work we attempt to\nidentify annotation provenance and track its subsequent propagation.\nSpecifically, we exploit annotation reuse within the UniProt Knowledgebase\n(UniProtKB), at the level of individual sentences. We describe a visualisation\napproach for the provenance and propagation of sentences in UniProtKB which\nenables a large-scale statistical analysis. Initially levels of sentence reuse\nwithin UniProtKB were analysed, showing that reuse is heavily prevalent, which\nenables the tracking of provenance and propagation. By analysing sentences\nthroughout UniProtKB, a number of interesting propagation patterns were\nidentified, covering over 100, 000 sentences. Over 8000 sentences remain in the\ndatabase after they have been removed from the entries where they originally\noccurred. Analysing a subset of these sentences suggest that approximately 30%\nare erroneous, whilst 35% appear to be inconsistent. These results suggest that\nbeing able to visualise sentence propagation and provenance can aid in the\ndetermination of the accuracy and quality of textual annotation. Source code\nand supplementary data are available from the authors website.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1308.5010v1",
            "date": "2013-08-22 22:14:25+00:00",
            "title": "Sentiment in New York City: A High Resolution Spatial and Temporal View",
            "authors": [
                "Karla Z. Bertrand",
                "Maya Bialik",
                "Kawandeep Virdee",
                "Andreas Gros",
                "Yaneer Bar-Yam"
            ],
            "abstract": "Measuring public sentiment is a key task for researchers and policymakers\nalike. The explosion of available social media data allows for a more\ntime-sensitive and geographically specific analysis than ever before. In this\npaper we analyze data from the micro-blogging site Twitter and generate a\nsentiment map of New York City. We develop a classifier specifically tuned for\n140-character Twitter messages, or tweets, using key words, phrases and\nemoticons to determine the mood of each tweet. This method, combined with\ngeotagging provided by users, enables us to gauge public sentiment on extremely\nfine-grained spatial and temporal scales. We find that public mood is generally\nhighest in public parks and lowest at transportation hubs, and locate other\nareas of strong sentiment such as cemeteries, medical centers, a jail, and a\nsewage facility. Sentiment progressively improves with proximity to Times\nSquare. Periodic patterns of sentiment fluctuate on both a daily and a weekly\nscale: more positive tweets are posted on weekends than on weekdays, with a\ndaily peak in sentiment around midnight and a nadir between 9:00 a.m. and noon.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1308.6628v2",
            "date": "2013-08-29 23:45:02+00:00",
            "title": "Joint Video and Text Parsing for Understanding Events and Answering Queries",
            "authors": [
                "Kewei Tu",
                "Meng Meng",
                "Mun Wai Lee",
                "Tae Eun Choe",
                "Song-Chun Zhu"
            ],
            "abstract": "We propose a framework for parsing video and text jointly for understanding\nevents and answering user queries. Our framework produces a parse graph that\nrepresents the compositional structures of spatial information (objects and\nscenes), temporal information (actions and events) and causal information\n(causalities between events and fluents) in the video and text. The knowledge\nrepresentation of our framework is based on a spatial-temporal-causal And-Or\ngraph (S/T/C-AOG), which jointly models possible hierarchical compositions of\nobjects, scenes and events as well as their interactions and mutual contexts,\nand specifies the prior probabilistic distribution of the parse graphs. We\npresent a probabilistic generative model for joint parsing that captures the\nrelations between the input video/text, their corresponding parse graphs and\nthe joint parse graph. Based on the probabilistic model, we propose a joint\nparsing system consisting of three modules: video parsing, text parsing and\njoint inference. Video parsing and text parsing produce two parse graphs from\nthe input video and text respectively. The joint inference module produces a\njoint parse graph by performing matching, deduction and revision on the video\nand text parse graphs. The proposed framework has the following objectives:\nFirstly, we aim at deep semantic parsing of video and text that goes beyond the\ntraditional bag-of-words approaches; Secondly, we perform parsing and reasoning\nacross the spatial, temporal and causal dimensions based on the joint S/T/C-AOG\nrepresentation; Thirdly, we show that deep joint parsing facilitates subsequent\napplications such as generating narrative text descriptions and answering\nqueries in the forms of who, what, when, where and why. We empirically\nevaluated our system based on comparison against ground-truth as well as\naccuracy of query answering and obtained satisfactory results.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1309.1939v2",
            "date": "2013-09-08 08:31:09+00:00",
            "title": "The placement of the head that minimizes online memory: a complex systems approach",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "It is well known that the length of a syntactic dependency determines its\nonline memory cost. Thus, the problem of the placement of a head and its\ndependents (complements or modifiers) that minimizes online memory is\nequivalent to the problem of the minimum linear arrangement of a star tree.\nHowever, how that length is translated into cognitive cost is not known. This\nstudy shows that the online memory cost is minimized when the head is placed at\nthe center, regardless of the function that transforms length into cost,\nprovided only that this function is strictly monotonically increasing. Online\nmemory defines a quasi-convex adaptive landscape with a single central minimum\nif the number of elements is odd and two central minima if that number is even.\nWe discuss various aspects of the dynamics of word order of subject (S), verb\n(V) and object (O) from a complex systems perspective and suggest that word\norders tend to evolve by swapping adjacent constituents from an initial or\nearly SOV configuration that is attracted towards a central word order by\nonline memory minimization. We also suggest that the stability of SVO is due to\nat least two factors, the quasi-convex shape of the adaptive landscape in the\nonline memory dimension and online memory adaptations that avoid regression to\nSOV. Although OVS is also optimal for placing the verb at the center, its low\nfrequency is explained by its long distance to the seminal SOV in the\npermutation space.",
            "journal ref": "Language Dynamics and Change 5 (1), 114-137 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.3946v1",
            "date": "2013-09-16 13:21:45+00:00",
            "title": "Using Self-Organizing Maps for Sentiment Analysis",
            "authors": [
                "Anuj Sharma",
                "Shubhamoy Dey"
            ],
            "abstract": "Web 2.0 services have enabled people to express their opinions, experience\nand feelings in the form of user-generated content. Sentiment analysis or\nopinion mining involves identifying, classifying and aggregating opinions as\nper their positive or negative polarity. This paper investigates the efficacy\nof different implementations of Self-Organizing Maps (SOM) for sentiment based\nvisualization and classification of online reviews. Specifically, this paper\nimplements the SOM algorithm for both supervised and unsupervised learning from\ntext documents. The unsupervised SOM algorithm is implemented for sentiment\nbased visualization and classification tasks. For supervised sentiment\nanalysis, a competitive learning algorithm known as Learning Vector\nQuantization is used. Both algorithms are also compared with their respective\nmulti-pass implementations where a quick rough ordering pass is followed by a\nfine tuning pass. The experimental results on the online movie review data set\nshow that SOMs are well suited for sentiment based classification and sentiment\npolarity visualization.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1309.3949v1",
            "date": "2013-09-16 13:27:04+00:00",
            "title": "Performance Investigation of Feature Selection Methods",
            "authors": [
                "Anuj sharma",
                "Shubhamoy Dey"
            ],
            "abstract": "Sentiment analysis or opinion mining has become an open research domain after\nproliferation of Internet and Web 2.0 social media. People express their\nattitudes and opinions on social media including blogs, discussion forums,\ntweets, etc. and, sentiment analysis concerns about detecting and extracting\nsentiment or opinion from online text. Sentiment based text classification is\ndifferent from topical text classification since it involves discrimination\nbased on expressed opinion on a topic. Feature selection is significant for\nsentiment analysis as the opinionated text may have high dimensions, which can\nadversely affect the performance of sentiment analysis classifier. This paper\nexplores applicability of feature selection methods for sentiment analysis and\ninvestigates their performance for classification in term of recall, precision\nand accuracy. Five feature selection methods (Document Frequency, Information\nGain, Gain Ratio, Chi Squared, and Relief-F) and three popular sentiment\nfeature lexicons (HM, GI and Opinion Lexicon) are investigated on movie reviews\ncorpus with a size of 2000 documents. The experimental results show that\nInformation Gain gave consistent results and Gain Ratio performs overall best\nfor sentimental feature selection while sentiment lexicons gave poor\nperformance. Furthermore, we found that performance of the classifier depends\non appropriate number of representative feature selected from text.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1309.4035v1",
            "date": "2013-09-16 16:51:02+00:00",
            "title": "Domain and Function: A Dual-Space Model of Semantic Relations and Compositions",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Given appropriate representations of the semantic relations between carpenter\nand wood and between mason and stone (for example, vectors in a vector space\nmodel), a suitable algorithm should be able to recognize that these relations\nare highly similar (carpenter is to wood as mason is to stone; the relations\nare analogous). Likewise, with representations of dog, house, and kennel, an\nalgorithm should be able to recognize that the semantic composition of dog and\nhouse, dog house, is highly similar to kennel (dog house and kennel are\nsynonymous). It seems that these two tasks, recognizing relations and\ncompositions, are closely connected. However, up to now, the best models for\nrelations are significantly different from the best models for compositions. In\nthis paper, we introduce a dual-space model that unifies these two tasks. This\nmodel matches the performance of the best previous models for relations and\ncompositions. The dual-space model consists of a space for measuring domain\nsimilarity and a space for measuring function similarity. Carpenter and wood\nshare the same domain, the domain of carpentry. Mason and stone share the same\ndomain, the domain of masonry. Carpenter and mason share the same function, the\nfunction of artisans. Wood and stone share the same function, the function of\nmaterials. In the composition dog house, kennel has some domain overlap with\nboth dog and house (the domains of pets and buildings). The function of kennel\nis similar to the function of house (the function of shelters). By combining\ndomain and function similarities in various ways, we can model relations,\ncompositions, and other aspects of semantics.",
            "journal ref": "Journal of Artificial Intelligence Research (JAIR), (2012), 44,\n  533-585",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.4058v1",
            "date": "2013-09-16 18:21:54+00:00",
            "title": "Why SOV might be initially preferred and then lost or recovered? A theoretical framework",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Little is known about why SOV order is initially preferred and then discarded\nor recovered. Here we present a framework for understanding these and many\nrelated word order phenomena: the diversity of dominant orders, the existence\nof free words orders, the need of alternative word orders and word order\nreversions and cycles in evolution. Under that framework, word order is\nregarded as a multiconstraint satisfaction problem in which at least two\nconstraints are in conflict: online memory minimization and maximum\npredictability.",
            "journal ref": "Proceedings of Evolang 2014, Cartmill, E. A., Roberts, S., Lyn, H.\n  & Cornish, H. (eds.), pp. 66-73 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.5174v1",
            "date": "2013-09-20 05:07:29+00:00",
            "title": "Saying What You're Looking For: Linguistics Meets Video Search",
            "authors": [
                "Andrei Barbu",
                "N. Siddharth",
                "Jeffrey Mark Siskind"
            ],
            "abstract": "We present an approach to searching large video corpora for video clips which\ndepict a natural-language query in the form of a sentence. This approach uses\ncompositional semantics to encode subtle meaning that is lost in other systems,\nsuch as the difference between two sentences which have identical words but\nentirely different meaning: \"The person rode the horse} vs. \\emph{The horse\nrode the person\". Given a video-sentence pair and a natural-language parser,\nalong with a grammar that describes the space of sentential queries, we produce\na score which indicates how well the video depicts the sentence. We produce\nsuch a score for each video clip in a corpus and return a ranked list of clips.\nFurthermore, this approach addresses two fundamental problems simultaneously:\ndetecting and tracking objects, and recognizing whether those tracks depict the\nquery. Because both tracking and object detection are unreliable, this uses\nknowledge about the intended sentential query to focus the tracker on the\nrelevant participants and ensures that the resulting tracks are described by\nthe sentential query. While earlier work was limited to single-word queries\nwhich correspond to either verbs or nouns, we show how one can search for\ncomplex queries which contain multiple phrases, such as prepositional phrases,\nand modifiers, such as adverbs. We demonstrate this approach by searching for\n141 queries involving people and horses interacting with each other in 10\nfull-length Hollywood movies.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1309.5319v1",
            "date": "2013-09-20 16:47:48+00:00",
            "title": "Recognizing Speech in a Novel Accent: The Motor Theory of Speech Perception Reframed",
            "authors": [
                "Cl\u00e9ment Moulin-Frier",
                "M. A. Arbib"
            ],
            "abstract": "The motor theory of speech perception holds that we perceive the speech of\nanother in terms of a motor representation of that speech. However, when we\nhave learned to recognize a foreign accent, it seems plausible that recognition\nof a word rarely involves reconstruction of the speech gestures of the speaker\nrather than the listener. To better assess the motor theory and this\nobservation, we proceed in three stages. Part 1 places the motor theory of\nspeech perception in a larger framework based on our earlier models of the\nadaptive formation of mirror neurons for grasping, and for viewing extensions\nof that mirror system as part of a larger system for neuro-linguistic\nprocessing, augmented by the present consideration of recognizing speech in a\nnovel accent. Part 2 then offers a novel computational model of how a listener\ncomes to understand the speech of someone speaking the listener's native\nlanguage with a foreign accent. The core tenet of the model is that the\nlistener uses hypotheses about the word the speaker is currently uttering to\nupdate probabilities linking the sound produced by the speaker to phonemes in\nthe native language repertoire of the listener. This, on average, improves the\nrecognition of later words. This model is neutral regarding the nature of the\nrepresentations it uses (motor vs. auditory). It serve as a reference point for\nthe discussion in Part 3, which proposes a dual-stream neuro-linguistic\narchitecture to revisits claims for and against the motor theory of speech\nperception and the relevance of mirror neurons, and extracts some implications\nfor the reframing of the motor theory.",
            "journal ref": "Biological Cybernetics 107, 4 (2013) 421-447",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6176v1",
            "date": "2013-09-23 13:51:28+00:00",
            "title": "Feature Learning with Gaussian Restricted Boltzmann Machine for Robust Speech Recognition",
            "authors": [
                "Xin Zheng",
                "Zhiyong Wu",
                "Helen Meng",
                "Weifeng Li",
                "Lianhong Cai"
            ],
            "abstract": "In this paper, we first present a new variant of Gaussian restricted\nBoltzmann machine (GRBM) called multivariate Gaussian restricted Boltzmann\nmachine (MGRBM), with its definition and learning algorithm. Then we propose\nusing a learned GRBM or MGRBM to extract better features for robust speech\nrecognition. Our experiments on Aurora2 show that both GRBM-extracted and\nMGRBM-extracted feature performs much better than Mel-frequency cepstral\ncoefficient (MFCC) with either HMM-GMM or hybrid HMM-deep neural network (DNN)\nacoustic model, and MGRBM-extracted feature is slightly better.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1309.6874v1",
            "date": "2013-09-26 12:54:02+00:00",
            "title": "Integrating Document Clustering and Topic Modeling",
            "authors": [
                "Pengtao Xie",
                "Eric P. Xing"
            ],
            "abstract": "Document clustering and topic modeling are two closely related tasks which\ncan mutually benefit each other. Topic modeling can project documents into a\ntopic space which facilitates effective document clustering. Cluster labels\ndiscovered by document clustering can be incorporated into topic models to\nextract local topics specific to each cluster and global topics shared by all\nclusters. In this paper, we propose a multi-grain clustering topic model\n(MGCTM) which integrates document clustering and topic modeling into a unified\nframework and jointly performs the two tasks to achieve the overall best\nperformance. Our model tightly couples two components: a mixture component used\nfor discovering latent groups in document collection and a topic model\ncomponent used for mining multi-grain topics including local topics specific to\neach cluster and global topics shared across clusters.We employ variational\ninference to approximate the posterior of hidden variables and learn model\nparameters. Experiments on two datasets demonstrate the effectiveness of our\nmodel.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.0493v1",
            "date": "2013-12-02 15:54:40+00:00",
            "title": "Bidirectional Recursive Neural Networks for Token-Level Labeling with Structure",
            "authors": [
                "Ozan \u0130rsoy",
                "Claire Cardie"
            ],
            "abstract": "Recently, deep architectures, such as recurrent and recursive neural networks\nhave been successfully applied to various natural language processing tasks.\nInspired by bidirectional recurrent neural networks which use representations\nthat summarize the past and future around an instance, we propose a novel\narchitecture that aims to capture the structural information around an input,\nand use it to label instances. We apply our method to the task of opinion\nexpression extraction, where we employ the binary parse tree of a sentence as\nthe structure, and word vector representations as the initial representation of\na single token. We conduct preliminary experiments to investigate its\nperformance and compare it to the sequential approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.2137v1",
            "date": "2013-12-07 19:55:02+00:00",
            "title": "End-to-end Phoneme Sequence Recognition using Convolutional Neural Networks",
            "authors": [
                "Dimitri Palaz",
                "Ronan Collobert",
                "Mathew Magimai. -Doss"
            ],
            "abstract": "Most phoneme recognition state-of-the-art systems rely on a classical neural\nnetwork classifiers, fed with highly tuned features, such as MFCC or PLP\nfeatures. Recent advances in ``deep learning'' approaches questioned such\nsystems, but while some attempts were made with simpler features such as\nspectrograms, state-of-the-art systems still rely on MFCCs. This might be\nviewed as a kind of failure from deep learning approaches, which are often\nclaimed to have the ability to train with raw signals, alleviating the need of\nhand-crafted features. In this paper, we investigate a convolutional neural\nnetwork approach for raw speech signals. While convolutional architectures got\ntremendous success in computer vision or text processing, they seem to have\nbeen let down in the past recent years in the speech processing field. We show\nthat it is possible to learn an end-to-end phoneme sequence classifier system\ndirectly from raw signal, with similar performance on the TIMIT and WSJ\ndatasets than existing systems based on MFCC, questioning the need of complex\nhand-crafted features on large datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.2844v1",
            "date": "2013-12-10 15:56:53+00:00",
            "title": "mARC: Memory by Association and Reinforcement of Contexts",
            "authors": [
                "Norbert Rimoux",
                "Patrice Descourt"
            ],
            "abstract": "This paper introduces the memory by Association and Reinforcement of Contexts\n(mARC). mARC is a novel data modeling technology rooted in the second\nquantization formulation of quantum mechanics. It is an all-purpose incremental\nand unsupervised data storage and retrieval system which can be applied to all\ntypes of signal or data, structured or unstructured, textual or not. mARC can\nbe applied to a wide range of information clas-sification and retrieval\nproblems like e-Discovery or contextual navigation. It can also for-mulated in\nthe artificial life framework a.k.a Conway \"Game Of Life\" Theory. In contrast\nto Conway approach, the objects evolve in a massively multidimensional space.\nIn order to start evaluating the potential of mARC we have built a mARC-based\nInternet search en-gine demonstrator with contextual functionality. We compare\nthe behavior of the mARC demonstrator with Google search both in terms of\nperformance and relevance. In the study we find that the mARC search engine\ndemonstrator outperforms Google search by an order of magnitude in response\ntime while providing more relevant results for some classes of queries.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1312.5198v4",
            "date": "2013-12-18 16:13:08+00:00",
            "title": "Learning Semantic Script Knowledge with Event Embeddings",
            "authors": [
                "Ashutosh Modi",
                "Ivan Titov"
            ],
            "abstract": "Induction of common sense knowledge about prototypical sequences of events\nhas recently received much attention. Instead of inducing this knowledge in the\nform of graphs, as in much of the previous work, in our method, distributed\nrepresentations of event realizations are computed based on distributed\nrepresentations of predicates and their arguments, and then these\nrepresentations are used to predict prototypical event orderings. The\nparameters of the compositional process for computing the event representations\nand the ranking component of the model are jointly estimated from texts. We\nshow that this approach results in a substantial boost in ordering performance\nwith respect to previous methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.6849v2",
            "date": "2013-12-24 16:36:16+00:00",
            "title": "Speech Recognition Front End Without Information Loss",
            "authors": [
                "Matthew Ager",
                "Zoran Cvetkovic",
                "Peter Sollich"
            ],
            "abstract": "Speech representation and modelling in high-dimensional spaces of acoustic\nwaveforms, or a linear transformation thereof, is investigated with the aim of\nimproving the robustness of automatic speech recognition to additive noise. The\nmotivation behind this approach is twofold: (i) the information in acoustic\nwaveforms that is usually removed in the process of extracting low-dimensional\nfeatures might aid robust recognition by virtue of structured redundancy\nanalogous to channel coding, (ii) linear feature domains allow for exact noise\nadaptation, as opposed to representations that involve non-linear processing\nwhich makes noise adaptation challenging. Thus, we develop a generative\nframework for phoneme modelling in high-dimensional linear feature domains, and\nuse it in phoneme classification and recognition tasks. Results show that\nclassification and recognition in this framework perform better than analogous\nPLP and MFCC classifiers below 18 dB SNR. A combination of the high-dimensional\nand MFCC features at the likelihood level performs uniformly better than either\nof the individual representations across all noise levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1312.6962v1",
            "date": "2013-12-25 12:38:17+00:00",
            "title": "Subjectivity Classification using Machine Learning Techniques for Mining Feature-Opinion Pairs from Web Opinion Sources",
            "authors": [
                "Ahmad Kamal"
            ],
            "abstract": "Due to flourish of the Web 2.0, web opinion sources are rapidly emerging\ncontaining precious information useful for both customers and manufactures.\nRecently, feature based opinion mining techniques are gaining momentum in which\ncustomer reviews are processed automatically for mining product features and\nuser opinions expressed over them. However, customer reviews may contain both\nopinionated and factual sentences. Distillations of factual contents improve\nmining performance by preventing noisy and irrelevant extraction. In this\npaper, combination of both supervised machine learning and rule-based\napproaches are proposed for mining feasible feature-opinion pairs from\nsubjective review sentences. In the first phase of the proposed approach, a\nsupervised machine learning technique is applied for classifying subjective and\nobjective sentences from customer reviews. In the next phase, a rule based\nmethod is implemented which applies linguistic and semantic analysis of texts\nto mine feasible feature-opinion pairs from subjective sentences retained after\nthe first phase. The effectiveness of the proposed methods is established\nthrough experimentation over customer reviews on different electronic products.",
            "journal ref": "International Journal of Computer Science Issues (IJCSI), Volume\n  10 Issue 5, 2013, pp 191-200",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1312.7077v2",
            "date": "2013-12-26 09:45:02+00:00",
            "title": "Language Modeling with Power Low Rank Ensembles",
            "authors": [
                "Ankur P. Parikh",
                "Avneesh Saluja",
                "Chris Dyer",
                "Eric P. Xing"
            ],
            "abstract": "We present power low rank ensembles (PLRE), a flexible framework for n-gram\nlanguage modeling where ensembles of low rank matrices and tensors are used to\nobtain smoothed probability estimates of words in context. Our method can be\nunderstood as a generalization of n-gram modeling to non-integer n, and\nincludes standard techniques such as absolute discounting and Kneser-Ney\nsmoothing as special cases. PLRE training is efficient and our approach\noutperforms state-of-the-art modified Kneser Ney baselines in terms of\nperplexity on large corpora as well as on BLEU score in a downstream machine\ntranslation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3322v1",
            "date": "2013-12-24 08:45:07+00:00",
            "title": "A Subband-Based SVM Front-End for Robust ASR",
            "authors": [
                "Jibran Yousafzai",
                "Zoran Cvetkovic",
                "Peter Sollich",
                "Matthew Ager"
            ],
            "abstract": "This work proposes a novel support vector machine (SVM) based robust\nautomatic speech recognition (ASR) front-end that operates on an ensemble of\nthe subband components of high-dimensional acoustic waveforms. The key issues\nof selecting the appropriate SVM kernels for classification in frequency\nsubbands and the combination of individual subband classifiers using ensemble\nmethods are addressed. The proposed front-end is compared with state-of-the-art\nASR front-ends in terms of robustness to additive noise and linear filtering.\nExperiments performed on the TIMIT phoneme classification task demonstrate the\nbenefits of the proposed subband based SVM front-end: it outperforms the\nstandard cepstral front-end in the presence of noise and linear filtering for\nsignal-to-noise ratio (SNR) below 12-dB. A combination of the proposed\nfront-end with a conventional front-end such as MFCC yields further\nimprovements over the individual front ends across the full range of noise\nlevels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.1429v1",
            "date": "2013-01-08 07:11:19+00:00",
            "title": "Adaptation of fictional and online conversations to communication media",
            "authors": [
                "Christian M. Alis",
                "May T. Lim"
            ],
            "abstract": "Conversations allow the quick transfer of short bits of information and it is\nreasonable to expect that changes in communication medium affect how we\nconverse. Using conversations in works of fiction and in an online social\nnetworking platform, we show that the utterance length of conversations is\nslowly shortening with time but adapts more strongly to the constraints of the\ncommunication medium. This indicates that the introduction of any new medium of\ncommunication can affect the way natural language evolves.",
            "journal ref": "Eur. Phys. J. B, vol. 85, no. 12, pp. 1-7, 2012",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1301.3226v4",
            "date": "2013-01-15 04:52:10+00:00",
            "title": "The Expressive Power of Word Embeddings",
            "authors": [
                "Yanqing Chen",
                "Bryan Perozzi",
                "Rami Al-Rfou",
                "Steven Skiena"
            ],
            "abstract": "We seek to better understand the difference in quality of the several\npublicly released embeddings. We propose several tasks that help to distinguish\nthe characteristics of different embeddings. Our evaluation of sentiment\npolarity and synonym/antonym relations shows that embeddings are able to\ncapture surprisingly nuanced semantics even in the absence of sentence\nstructure. Moreover, benchmarking the embeddings shows great variance in\nquality and characteristics of the semantics captured by the tested embeddings.\nFinally, we show the impact of varying the number of dimensions and the\nresolution of each dimension on the effective useful features captured by the\nembedding space. Our contributions highlight the importance of embeddings for\nNLP tasks and the effect of their quality on the final results.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1301.3605v3",
            "date": "2013-01-16 07:23:19+00:00",
            "title": "Feature Learning in Deep Neural Networks - Studies on Speech Recognition Tasks",
            "authors": [
                "Dong Yu",
                "Michael L. Seltzer",
                "Jinyu Li",
                "Jui-Ting Huang",
                "Frank Seide"
            ],
            "abstract": "Recent studies have shown that deep neural networks (DNNs) perform\nsignificantly better than shallow networks and Gaussian mixture models (GMMs)\non large vocabulary speech recognition tasks. In this paper, we argue that the\nimproved accuracy achieved by the DNNs is the result of their ability to\nextract discriminative internal representations that are robust to the many\nsources of variability in speech signals. We show that these representations\nbecome increasingly insensitive to small perturbations in the input with\nincreasing network depth, which leads to better speech recognition performance\nwith deeper networks. We also show that DNNs cannot extrapolate to test samples\nthat are substantially different from the training examples. If the training\ndata are sufficiently representative, however, internal features learned by the\nDNN are relatively stable with respect to speaker differences, bandwidth\ndifferences, and environment distortion. This enables DNN-based recognizers to\nperform as well or better than state-of-the-art systems based on GMMs or\nshallow networks without the need for explicit model adaptation or feature\nnormalization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1301.5686v2",
            "date": "2013-01-24 02:02:13+00:00",
            "title": "Transfer Topic Modeling with Ease and Scalability",
            "authors": [
                "Jeon-Hyung Kang",
                "Jun Ma",
                "Yan Liu"
            ],
            "abstract": "The increasing volume of short texts generated on social media sites, such as\nTwitter or Facebook, creates a great demand for effective and efficient topic\nmodeling approaches. While latent Dirichlet allocation (LDA) can be applied, it\nis not optimal due to its weakness in handling short texts with fast-changing\ntopics and scalability concerns. In this paper, we propose a transfer learning\napproach that utilizes abundant labeled documents from other domains (such as\nYahoo! News or Wikipedia) to improve topic modeling, with better model fitting\nand result interpretation. Specifically, we develop Transfer Hierarchical LDA\n(thLDA) model, which incorporates the label information from other domains via\ninformative priors. In addition, we develop a parallel implementation of our\nmodel for large-scale applications. We demonstrate the effectiveness of our\nthLDA model on both a microblogging dataset and standard text collections\nincluding AP and RCV1 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1301.7382v2",
            "date": "2013-01-30 15:04:21+00:00",
            "title": "Inferring Informational Goals from Free-Text Queries: A Bayesian Approach",
            "authors": [
                "David Heckerman",
                "Eric J. Horvitz"
            ],
            "abstract": "People using consumer software applications typically do not use technical\njargon when querying an online database of help topics. Rather, they attempt to\ncommunicate their goals with common words and phrases that describe software\nfunctionality in terms of structure and objects they understand. We describe a\nBayesian approach to modeling the relationship between words in a user's query\nfor assistance and the informational goals of the user. After reviewing the\ngeneral method, we describe several extensions that center on integrating\nadditional distinctions and structure about language usage and user goals into\nthe Bayesian models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1304.0104v1",
            "date": "2013-03-30 13:08:58+00:00",
            "title": "Meaning-focused and Quantum-inspired Information Retrieval",
            "authors": [
                "Diederik Aerts",
                "Jan Broekaert",
                "Sandro Sozzo",
                "Tomas Veloz"
            ],
            "abstract": "In recent years, quantum-based methods have promisingly integrated the\ntraditional procedures in information retrieval (IR) and natural language\nprocessing (NLP). Inspired by our research on the identification and\napplication of quantum structures in cognition, more specifically our work on\nthe representation of concepts and their combinations, we put forward a\n'quantum meaning based' framework for structured query retrieval in text\ncorpora and standardized testing corpora. This scheme for IR rests on\nconsidering as basic notions, (i) 'entities of meaning', e.g., concepts and\ntheir combinations and (ii) traces of such entities of meaning, which is how\ndocuments are considered in this approach. The meaning content of these\n'entities of meaning' is reconstructed by solving an 'inverse problem' in the\nquantum formalism, consisting of reconstructing the full states of the entities\nof meaning from their collapsed states identified as traces in relevant\ndocuments. The advantages with respect to traditional approaches, such as\nLatent Semantic Analysis (LSA), are discussed by means of concrete examples.",
            "journal ref": "Quantum Interaction. Lecture Notes in Computer Science, 8369, pp.\n  71-83, 2014",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1304.1018v2",
            "date": "2013-04-03 17:20:41+00:00",
            "title": "Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks",
            "authors": [
                "Dimitri Palaz",
                "Ronan Collobert",
                "Mathew Magimai. -Doss"
            ],
            "abstract": "In hybrid hidden Markov model/artificial neural networks (HMM/ANN) automatic\nspeech recognition (ASR) system, the phoneme class conditional probabilities\nare estimated by first extracting acoustic features from the speech signal\nbased on prior knowledge such as, speech perception or/and speech production\nknowledge, and, then modeling the acoustic features with an ANN. Recent\nadvances in machine learning techniques, more specifically in the field of\nimage processing and text processing, have shown that such divide and conquer\nstrategy (i.e., separating feature extraction and modeling steps) may not be\nnecessary. Motivated from these studies, in the framework of convolutional\nneural networks (CNNs), this paper investigates a novel approach, where the\ninput to the ANN is raw speech signal and the output is phoneme class\nconditional probability estimates. On TIMIT phoneme recognition task, we study\ndifferent ANN architectures to show the benefit of CNNs and compare the\nproposed approach against conventional approach where, spectral-based feature\nMFCC is extracted and modeled by a multilayer perceptron. Our studies show that\nthe proposed approach can yield comparable or better phoneme recognition\nperformance when compared to the conventional approach. It indicates that CNNs\ncan learn features relevant for phoneme classification automatically from the\nraw speech signal.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1304.3432v1",
            "date": "2013-03-27 19:56:55+00:00",
            "title": "Machine Learning, Clustering, and Polymorphy",
            "authors": [
                "Stephen Jose Hanson",
                "Malcolm Bauer"
            ],
            "abstract": "This paper describes a machine induction program (WITT) that attempts to\nmodel human categorization. Properties of categories to which human subjects\nare sensitive includes best or prototypical members, relative contrasts between\nputative categories, and polymorphy (neither necessary or sufficient features).\nThis approach represents an alternative to usual Artificial Intelligence\napproaches to generalization and conceptual clustering which tend to focus on\nnecessary and sufficient feature rules, equivalence classes, and simple search\nand match schemes. WITT is shown to be more consistent with human\ncategorization while potentially including results produced by more traditional\nclustering schemes. Applications of this approach in the domains of expert\nsystems and information retrieval are also discussed.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1304.4086v5",
            "date": "2013-04-15 13:10:22+00:00",
            "title": "Hubiness, length, crossings and their relationships in dependency trees",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Here tree dependency structures are studied from three different\nperspectives: their degree variance (hubiness), the mean dependency length and\nthe number of dependency crossings. Bounds that reveal pairwise dependencies\namong these three metrics are derived. Hubiness (the variance of degrees) plays\na central role: the mean dependency length is bounded below by hubiness while\nthe number of crossings is bounded above by hubiness. Our findings suggest that\nthe online memory cost of a sentence might be determined not just by the\nordering of words but also by the hubiness of the underlying structure. The 2nd\nmoment of degree plays a crucial role that is reminiscent of its role in large\ncomplex networks.",
            "journal ref": "Ferrer-i-Cancho, R. (2013). Hubiness, length, crossings and their\n  relationships in dependency trees. Glottometrics 25, 1-21",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1304.5823v2",
            "date": "2013-04-22 01:39:42+00:00",
            "title": "Towards a Formal Distributional Semantics: Simulating Logical Calculi with Tensors",
            "authors": [
                "Edward Grefenstette"
            ],
            "abstract": "The development of compositional distributional models of semantics\nreconciling the empirical aspects of distributional semantics with the\ncompositional aspects of formal semantics is a popular topic in the\ncontemporary literature. This paper seeks to bring this reconciliation one step\nfurther by showing how the mathematical constructs commonly used in\ncompositional distributional models, such as tensors and matrices, can be used\nto simulate different aspects of predicate logic.\n  This paper discusses how the canonical isomorphism between tensors and\nmultilinear maps can be exploited to simulate a full-blown quantifier-free\npredicate calculus using tensors. It provides tensor interpretations of the set\nof logical connectives required to model propositional calculi. It suggests a\nvariant of these tensor calculi capable of modelling quantifiers, using few\nnon-linear operations. It finally discusses the relation between these\nvariants, and how this relation should constitute the subject of future work.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1305.0194v1",
            "date": "2013-05-01 15:07:18+00:00",
            "title": "MATAWS: A Multimodal Approach for Automatic WS Semantic Annotation",
            "authors": [
                "Cihan Aksoy",
                "Vincent Labatut",
                "Chantal Cherifi",
                "Jean-Fran\u00e7ois Santucci"
            ],
            "abstract": "Many recent works aim at developing methods and tools for the processing of\nsemantic Web services. In order to be properly tested, these tools must be\napplied to an appropriate benchmark, taking the form of a collection of\nsemantic WS descriptions. However, all of the existing publicly available\ncollections are limited by their size or their realism (use of randomly\ngenerated or resampled descriptions). Larger and realistic syntactic (WSDL)\ncollections exist, but their semantic annotation requires a certain level of\nautomation, due to the number of operations to be processed. In this article,\nwe propose a fully automatic method to semantically annotate such large WS\ncollections. Our approach is multimodal, in the sense it takes advantage of the\nlatent semantics present not only in the parameter names, but also in the type\nnames and structures. Concept-to-word association is performed by using Sigma,\na mapping of WordNet to the SUMO ontology. After having described in details\nour annotation method, we apply it to the larger collection of real-world\nsyntactic WS descriptions we could find, and assess its efficiency.",
            "journal ref": "In International Conference on Networked Digital Technologies,\n  Springer CCIS 136),China (2011)",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1305.1343v1",
            "date": "2013-05-06 22:24:20+00:00",
            "title": "Towards an Author-Topic-Term-Model Visualization of 100 Years of German Sociological Society Proceedings",
            "authors": [
                "Arnim Bleier",
                "Andreas Strotmann"
            ],
            "abstract": "Author co-citation studies employ factor analysis to reduce high-dimensional\nco-citation matrices to low-dimensional and possibly interpretable factors, but\nthese studies do not use any information from the text bodies of publications.\nWe hypothesise that term frequencies may yield useful information for\nscientometric analysis. In our work we ask if word features in combination with\nBayesian analysis allow well-founded science mapping studies. This work goes\nback to the roots of Mosteller and Wallace's (1964) statistical text analysis\nusing word frequency features and a Bayesian inference approach, tough with\ndifferent goals. To answer our research question we (i) introduce a new data\nset on which the experiments are carried out, (ii) describe the Bayesian model\nemployed for inference and (iii) present first results of the analysis.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1305.4561v3",
            "date": "2013-05-20 15:44:54+00:00",
            "title": "Random crossings in dependency trees",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "It has been hypothesized that the rather small number of crossings in real\nsyntactic dependency trees is a side-effect of pressure for dependency length\nminimization. Here we answer a related important research question: what would\nbe the expected number of crossings if the natural order of a sentence was lost\nand replaced by a random ordering? We show that this number depends only on the\nnumber of vertices of the dependency tree (the sentence length) and the second\nmoment about zero of vertex degrees. The expected number of crossings is\nminimum for a star tree (crossings are impossible) and maximum for a linear\ntree (the number of crossings is of the order of the square of the sequence\nlength).",
            "journal ref": "Glottometrics 37, 1-12 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.6143v2",
            "date": "2013-05-27 08:37:26+00:00",
            "title": "Fast and accurate sentiment classification using an enhanced Naive Bayes model",
            "authors": [
                "Vivek Narayanan",
                "Ishan Arora",
                "Arjun Bhatia"
            ],
            "abstract": "We have explored different methods of improving the accuracy of a Naive Bayes\nclassifier for sentiment analysis. We observed that a combination of methods\nlike negation handling, word n-grams and feature selection by mutual\ninformation results in a significant improvement in accuracy. This implies that\na highly accurate and fast sentiment classifier can be built using a simple\nNaive Bayes model that has linear training and testing time complexities. We\nachieved an accuracy of 88.80% on the popular IMDB movie reviews dataset.",
            "journal ref": "Intelligent Data Engineering and Automated Learning IDEAL 2013\n  Lecture Notes in Computer Science Volume 8206, 2013, pp 194-201",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1305.7014v1",
            "date": "2013-05-30 06:35:52+00:00",
            "title": "Tweets Miner for Stock Market Analysis",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "In this paper, we present a software package for the data mining of Twitter\nmicroblogs for the purpose of using them for the stock market analysis. The\npackage is written in R langauge using apropriate R packages. The model of\ntweets has been considered. We have also compared stock market charts with\nfrequent sets of keywords in Twitter microblogs messages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1306.0963v1",
            "date": "2013-06-05 02:17:11+00:00",
            "title": "Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior",
            "authors": [
                "Been Kim",
                "Caleb M. Chacha",
                "Julie Shah"
            ],
            "abstract": "We aim to reduce the burden of programming and deploying autonomous systems\nto work in concert with people in time-critical domains, such as military field\noperations and disaster response. Deployment plans for these operations are\nfrequently negotiated on-the-fly by teams of human planners. A human operator\nthen translates the agreed upon plan into machine instructions for the robots.\nWe present an algorithm that reduces this translation burden by inferring the\nfinal plan from a processed form of the human team's planning conversation. Our\napproach combines probabilistic generative modeling with logical plan\nvalidation used to compute a highly structured prior over possible plans. This\nhybrid approach enables us to overcome the challenge of performing inference\nover the large solution space with only a small amount of noisy data from the\nteam planning session. We validate the algorithm through human subject\nexperimentation and show we are able to infer a human team's final plan with\n83% accuracy on average. We also describe a robot demonstration in which two\npeople plan and execute a first-response collaborative task with a PR2 robot.\nTo the best of our knowledge, this is the first work that integrates a logical\nplanning technique within a generative model to perform plan inference.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1306.2838v1",
            "date": "2013-06-12 14:35:11+00:00",
            "title": "The Quantum Challenge in Concept Theory and Natural Language Processing",
            "authors": [
                "Diederik Aerts",
                "Jan Broekaert",
                "Sandro Sozzo",
                "Tomas Veloz"
            ],
            "abstract": "The mathematical formalism of quantum theory has been successfully used in\nhuman cognition to model decision processes and to deliver representations of\nhuman knowledge. As such, quantum cognition inspired tools have improved\ntechnologies for Natural Language Processing and Information Retrieval. In this\npaper, we overview the quantum cognition approach developed in our Brussels\nteam during the last two decades, specifically our identification of quantum\nstructures in human concepts and language, and the modeling of data from\npsychological and corpus-text-based experiments. We discuss our\nquantum-theoretic framework for concepts and their conjunctions/disjunctions in\na Fock-Hilbert space structure, adequately modeling a large amount of data\ncollected on concept combinations. Inspired by this modeling, we put forward\nelements for a quantum contextual and meaning-based approach to information\ntechnologies in which 'entities of meaning' are inversely reconstructed from\ntexts, which are considered as traces of these entities' states.",
            "journal ref": "Proceedings of the 25th International Conference on System\n  Research, Informatics & Cybernetics (pp. 13-17). Ed.. E. G. Lasker, IIAS,\n  2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.3920v1",
            "date": "2013-06-17 16:25:51+00:00",
            "title": "Discriminating word senses with tourist walks in complex networks",
            "authors": [
                "Thiago C. Silva",
                "Diego R. Amancio"
            ],
            "abstract": "Patterns of topological arrangement are widely used for both animal and human\nbrains in the learning process. Nevertheless, automatic learning techniques\nfrequently overlook these patterns. In this paper, we apply a learning\ntechnique based on the structural organization of the data in the attribute\nspace to the problem of discriminating the senses of 10 polysemous words. Using\ntwo types of characterization of meanings, namely semantical and topological\napproaches, we have observed significative accuracy rates in identifying the\nsuitable meanings in both techniques. Most importantly, we have found that the\ncharacterization based on the deterministic tourist walk improves the\ndisambiguation process when one compares with the discrimination achieved with\ntraditional complex networks measurements such as assortativity and clustering\ncoefficient. To our knowledge, this is the first time that such deterministic\nwalk has been applied to such a kind of problem. Therefore, our finding\nsuggests that the tourist walk characterization may be useful in other related\napplications.",
            "journal ref": "Eur. Phys. J. B, 86 7 (2013) 297",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.6078v1",
            "date": "2013-06-25 20:00:09+00:00",
            "title": "A Computational Approach to Politeness with Application to Social Factors",
            "authors": [
                "Cristian Danescu-Niculescu-Mizil",
                "Moritz Sudhof",
                "Dan Jurafsky",
                "Jure Leskovec",
                "Christopher Potts"
            ],
            "abstract": "We propose a computational framework for identifying linguistic aspects of\npoliteness. Our starting point is a new corpus of requests annotated for\npoliteness, which we use to evaluate aspects of politeness theory and to\nuncover new interactions between politeness markers and context. These findings\nguide our construction of a classifier with domain-independent lexical and\nsyntactic features operationalizing key components of politeness theory, such\nas indirection, deference, impersonalization and modality. Our classifier\nachieves close to human performance and is effective across domains. We use our\nframework to study the relationship between politeness and social power,\nshowing that polite Wikipedia editors are more likely to achieve high status\nthrough elections, but, once elevated, they become less polite. We see a\nsimilar negative correlation between politeness and power on Stack Exchange,\nwhere users at the top of the reputation scale are less polite than those at\nthe bottom. Finally, we apply our classifier to a preliminary analysis of\npoliteness variation by gender and community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1306.6944v1",
            "date": "2013-06-07 15:48:06+00:00",
            "title": "The DeLiVerMATH project - Text analysis in mathematics",
            "authors": [
                "Ulf Sch\u00f6neberg",
                "Wolfram Sperber"
            ],
            "abstract": "A high-quality content analysis is essential for retrieval functionalities\nbut the manual extraction of key phrases and classification is expensive.\nNatural language processing provides a framework to automatize the process.\nHere, a machine-based approach for the content analysis of mathematical texts\nis described. A prototype for key phrase extraction and classification of\nmathematical texts is presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.1539v1",
            "date": "2013-11-06 22:06:15+00:00",
            "title": "Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics",
            "authors": [
                "Edward Grefenstette"
            ],
            "abstract": "This thesis is about the problem of compositionality in distributional\nsemantics. Distributional semantics presupposes that the meanings of words are\na function of their occurrences in textual contexts. It models words as\ndistributions over these contexts and represents them as vectors in high\ndimensional spaces. The problem of compositionality for such models concerns\nitself with how to produce representations for larger units of text by\ncomposing the representations of smaller units of text.\n  This thesis focuses on a particular approach to this compositionality\nproblem, namely using the categorical framework developed by Coecke, Sadrzadeh,\nand Clark, which combines syntactic analysis formalisms with distributional\nsemantic representations of meaning to produce syntactically motivated\ncomposition operations. This thesis shows how this approach can be\ntheoretically extended and practically implemented to produce concrete\ncompositional distributional models of natural language semantics. It\nfurthermore demonstrates that such models can perform on par with, or better\nthan, other competing approaches in the field of natural language processing.\n  There are three principal contributions to computational linguistics in this\nthesis. The first is to extend the DisCoCat framework on the syntactic front\nand semantic front, incorporating a number of syntactic analysis formalisms and\nproviding learning procedures allowing for the generation of concrete\ncompositional distributional models. The second contribution is to evaluate the\nmodels developed from the procedures presented here, showing that they\noutperform other compositional distributional models present in the literature.\nThe third contribution is to show how using category theory to solve linguistic\nproblems forms a sound basis for research, illustrated by examples of work on\nthis topic, that also suggest directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1311.3987v1",
            "date": "2013-11-14 06:10:15+00:00",
            "title": "Big Data and Cross-Document Coreference Resolution: Current State and Future Opportunities",
            "authors": [
                "Seyed-Mehdi-Reza Beheshti",
                "Srikumar Venugopal",
                "Seung Hwan Ryu",
                "Boualem Benatallah",
                "Wei Wang"
            ],
            "abstract": "Information Extraction (IE) is the task of automatically extracting\nstructured information from unstructured/semi-structured machine-readable\ndocuments. Among various IE tasks, extracting actionable intelligence from\never-increasing amount of data depends critically upon Cross-Document\nCoreference Resolution (CDCR) - the task of identifying entity mentions across\nmultiple documents that refer to the same underlying entity. Recently, document\ndatasets of the order of peta-/tera-bytes has raised many challenges for\nperforming effective CDCR such as scaling to large numbers of mentions and\nlimited representational power. The problem of analysing such datasets is\ncalled \"big data\". The aim of this paper is to provide readers with an\nunderstanding of the central concepts, subtasks, and the current\nstate-of-the-art in CDCR process. We provide assessment of existing\ntools/techniques for CDCR subtasks and highlight big data challenges in each of\nthem to help readers identify important and outstanding issues for further\ninvestigation. Finally, we provide concluding remarks and discuss possible\ndirections for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.0347v1",
            "date": "2013-03-02 03:57:15+00:00",
            "title": "Probing the statistical properties of unknown texts: application to the Voynich Manuscript",
            "authors": [
                "Diego R. Amancio",
                "Eduardo G. Altmann",
                "Diego Rybski",
                "Osvaldo N. Oliveira Jr.",
                "Luciano da F. Costa"
            ],
            "abstract": "While the use of statistical physics methods to analyze large corpora has\nbeen useful to unveil many patterns in texts, no comprehensive investigation\nhas been performed investigating the properties of statistical measurements\nacross different languages and texts. In this study we propose a framework that\naims at determining if a text is compatible with a natural language and which\nlanguages are closest to it, without any knowledge of the meaning of the words.\nThe approach is based on three types of statistical measurements, i.e. obtained\nfrom first-order statistics of word properties in a text, from the topology of\ncomplex networks representing text, and from intermittency concepts where text\nis treated as a time series. Comparative experiments were performed with the\nNew Testament in 15 different languages and with distinct books in English and\nPortuguese in order to quantify the dependency of the different measurements on\nthe language and on the story being told in the book. The metrics found to be\ninformative in distinguishing real texts from their shuffled versions include\nassortativity, degree and selectivity of words. As an illustration, we analyze\nan undeciphered medieval manuscript known as the Voynich Manuscript. We show\nthat it is mostly compatible with natural languages and incompatible with\nrandom texts. We also obtain candidates for key-words of the Voynich Manuscript\nwhich could be helpful in the effort of deciphering it. Because we were able to\nidentify statistical measurements that are more dependent on the syntax than on\nthe semantics, the framework may also serve for text analysis in\nlanguage-dependent applications.",
            "journal ref": "PLoS ONE 8(7): e67310 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1303.1599v1",
            "date": "2013-03-07 03:19:34+00:00",
            "title": "Efficient learning strategy of Chinese characters based on network approach",
            "authors": [
                "Xiao-Yong Yan",
                "Ying Fan",
                "Zengru Di",
                "Shlomo Havlin",
                "Jinshan Wu"
            ],
            "abstract": "Based on network analysis of hierarchical structural relations among Chinese\ncharacters, we develop an efficient learning strategy of Chinese characters. We\nregard a more efficient learning method if one learns the same number of useful\nChinese characters in less effort or time. We construct a node-weighted network\nof Chinese characters, where character usage frequencies are used as node\nweights. Using this hierarchical node-weighted network, we propose a new\nlearning method, the distributed node weight (DNW) strategy, which is based on\na new measure of nodes' importance that takes into account both the weight of\nthe nodes and the hierarchical structure of the network. Chinese character\nlearning strategies, particularly their learning order, are analyzed as\ndynamical processes over the network. We compare the efficiency of three\ntheoretical learning methods and two commonly used methods from mainstream\nChinese textbooks, one for Chinese elementary school students and the other for\nstudents learning Chinese as a second language. We find that the DNW method\nsignificantly outperforms the others, implying that the efficiency of current\nlearning methods of major textbooks can be greatly improved.",
            "journal ref": "PLoS ONE 8(8): e69745 (2013)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1303.2430v2",
            "date": "2013-03-11 05:36:34+00:00",
            "title": "Quantum and Concept Combination, Entangled Measurements and Prototype Theory",
            "authors": [
                "Diederik Aerts"
            ],
            "abstract": "We analyze the meaning of the violation of the marginal probability law for\nsituations of correlation measurements where entanglement is identified. We\nshow that for quantum theory applied to the cognitive realm such a violation\ndoes not lead to the type of problems commonly believed to occur in situations\nof quantum theory applied to the physical realm. We briefly situate our quantum\napproach for modeling concepts and their combinations with respect to the\nnotions of 'extension' and 'intension' in theories of meaning, and in existing\nconcept theories.",
            "journal ref": "Topics in Cognitive Science, 6, pp. 129-137, 2014",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1303.3036v1",
            "date": "2013-03-12 21:06:22+00:00",
            "title": "Type-theoretical natural language semantics: on the system F for meaning assembly",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "This paper presents and extends our type theoretical framework for a\ncompositional treatment of natural language semantics with some lexical\nfeatures like coercions (e.g. of a town into a football club) and copredication\n(e.g. on a town as a set of people and as a location). The second order typed\nlambda calculus was shown to be a good framework, and here we discuss how to\nintroduced predefined types and coercive subtyping which are much more natural\nthan internally coded similar constructs. Linguistic applications of these new\nfeatures are also exemplified.",
            "journal ref": "TYPES 2013, Toulouse : France (2013)",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1303.3170v1",
            "date": "2013-03-13 14:32:03+00:00",
            "title": "Types and forgetfulness in categorical linguistics and quantum mechanics",
            "authors": [
                "Peter Hines"
            ],
            "abstract": "The role of types in categorical models of meaning is investigated. A general\nscheme for how typed models of meaning may be used to compare sentences,\nregardless of their grammatical structure is described, and a toy example is\nused as an illustration. Taking as a starting point the question of whether the\nevaluation of such a type system 'loses information', we consider the\nparametrized typing associated with connectives from this viewpoint.\n  The answer to this question implies that, within full categorical models of\nmeaning, the objects associated with types must exhibit a simple but subtle\ncategorical property known as self-similarity. We investigate the category\ntheory behind this, with explicit reference to typed systems, and their\nmonoidal closed structure. We then demonstrate close connections between such\nself-similar structures and dagger Frobenius algebras. In particular, we\ndemonstrate that the categorical structures implied by the polymorphically\ntyped connectives give rise to a (lax unitless) form of the special forms of\nFrobenius algebras known as classical structures, used heavily in abstract\ncategorical approaches to quantum mechanics.",
            "journal ref": "in C. Heunen, M. Sadrzadeh, E. Grefenstette (ed.s), Quantum\n  Physics and Linguistics: a compositional diagrammatic discourse, Oxford\n  University Press (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.3592v1",
            "date": "2013-03-14 20:07:48+00:00",
            "title": "Expressing Ethnicity through Behaviors of a Robot Character",
            "authors": [
                "Maxim Makatchev",
                "Reid Simmons",
                "Majd Sakr",
                "Micheline Ziadee"
            ],
            "abstract": "Achieving homophily, or association based on similarity, between a human user\nand a robot holds a promise of improved perception and task performance.\nHowever, no previous studies that address homophily via ethnic similarity with\nrobots exist. In this paper, we discuss the difficulties of evoking ethnic cues\nin a robot, as opposed to a virtual agent, and an approach to overcome those\ndifficulties based on using ethnically salient behaviors. We outline our\nmethodology for selecting and evaluating such behaviors, and culminate with a\nstudy that evaluates our hypotheses of the possibility of ethnic attribution of\na robot character through verbal and nonverbal behaviors and of achieving the\nhomophily effect.",
            "journal ref": "Proceedings of the 8th ACM/IEEE international conference on\n  Human-robot interaction (HRI), Tokyo, Japan, 2013, pages 357-364",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1303.3948v1",
            "date": "2013-03-16 05:35:39+00:00",
            "title": "An Adaptive Methodology for Ubiquitous ASR System",
            "authors": [
                "Urmila Shrawankar",
                "Vilas Thakare"
            ],
            "abstract": "Achieving and maintaining the performance of ubiquitous (Automatic Speech\nRecognition) ASR system is a real challenge. The main objective of this work is\nto develop a method that will improve and show the consistency in performance\nof ubiquitous ASR system for real world noisy environment. An adaptive\nmethodology has been developed to achieve an objective with the help of\nimplementing followings, -Cleaning speech signal as much as possible while\npreserving originality / intangibility using various modified filters and\nenhancement techniques. -Extracting features from speech signals using various\nsizes of parameter. -Train the system for ubiquitous environment using\nmulti-environmental adaptation training methods. -Optimize the word recognition\nrate with appropriate variable size of parameters using fuzzy technique. The\nconsistency in performance is tested using standard noise databases as well as\nin real world environment. A good improvement is noticed. This work will be\nhelpful to give discriminative training of ubiquitous ASR system for better\nHuman Computer Interaction (HCI) using Speech User Interface (SUI).",
            "journal ref": "Computer and Information Science;Vol.6,No.1;2013 ISSN 1913-8989\n  E-ISSN 1913-8997 Computer and Information Science; Vol. 6, No. 1; 2013, ISSN\n  1913-8989 E-ISSN 1913-8997, Published by Canadian Center of Science and\n  Education",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.0087v1",
            "date": "2013-06-29 10:40:59+00:00",
            "title": "Semantics and pragmatics in actual software applications and in web search engines: exploring innovations",
            "authors": [
                "Fabrizio M. A. Lolli"
            ],
            "abstract": "While new ways to use the Semantic Web are developed every week, which allow\nthe user to find information on web more accurately - for example in search\nengines - some sophisticated pragmatic tools are becoming more important - for\nexample in web interfaces known as Social Intelligence, or in the most famous\nSiri by Apple. The work aims to analyze whether and where we can identify the\nboundary between semantics and pragmatics in the software used by analyzed\nsystems. examining how the linguistic disciplines are fundamental in their\nprogress. Is it possible to assume that the tools of social intelligence have a\npragmatic approach to the questions of the user, or it is just a use of a very\nrich vocabulary, with the use of semantic tools?",
            "journal ref": "International Symposium on Language and Communication: Exploring\n  Novelties, vol.3, 955-962. IICS, 2013, Turkey",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1307.0261v1",
            "date": "2013-07-01 02:49:08+00:00",
            "title": "WebSets: Extracting Sets of Entities from the Web Using Unsupervised Information Extraction",
            "authors": [
                "Bhavana Dalvi",
                "William W. Cohen",
                "Jamie Callan"
            ],
            "abstract": "We describe a open-domain information extraction method for extracting\nconcept-instance pairs from an HTML corpus. Most earlier approaches to this\nproblem rely on combining clusters of distributionally similar terms and\nconcept-instance pairs obtained with Hearst patterns. In contrast, our method\nrelies on a novel approach for clustering terms found in HTML tables, and then\nassigning concept names to these clusters using Hearst patterns. The method can\nbe efficiently applied to a large corpus, and experimental results on several\ndatasets show that our method can accurately extract large numbers of\nconcept-instance pairs.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1307.4038v1",
            "date": "2013-07-15 18:21:51+00:00",
            "title": "An alternative Gospel of structure: order, composition, processes",
            "authors": [
                "Bob Coecke"
            ],
            "abstract": "We survey some basic mathematical structures, which arguably are more\nprimitive than the structures taught at school. These structures are orders,\nwith or without composition, and (symmetric) monoidal categories. We list\nseveral `real life' incarnations of each of these. This paper also serves as an\nintroduction to these structures and their current and potentially future uses\nin linguistics, physics and knowledge representation.",
            "journal ref": "None",
            "category": "math.CT"
        },
        {
            "url": "http://arxiv.org/abs/1307.4986v2",
            "date": "2013-07-18 15:44:31+00:00",
            "title": "On the Necessity of Mixed Models: Dynamical Frustrations in the Mind",
            "authors": [
                "Diego Gabriel Krivochen"
            ],
            "abstract": "In the present work we will present and analyze some basic processes at the\nlocal and global level in linguistic derivations that seem to go beyond the\nlimits of Markovian or Turing-like computation, and require, in our opinion, a\nquantum processor. We will first present briefly the working hypothesis and\nthen focus on the empirical domain. At the same time, we will argue that a\nmodel appealing to only one kind of computation (be it quantum or not) is\nnecessarily insufficient, and thus both linear and non-linear formal models are\nto be invoked in order to pursue a fuller understanding of mental computations\nwithin a unified framework.",
            "journal ref": "None",
            "category": "nlin.CD"
        },
        {
            "url": "http://arxiv.org/abs/1307.5336v2",
            "date": "2013-07-19 20:49:06+00:00",
            "title": "Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts",
            "authors": [
                "Pekka Malo",
                "Ankur Sinha",
                "Pyry Takala",
                "Pekka Korhonen",
                "Jyrki Wallenius"
            ],
            "abstract": "The use of robo-readers to analyze news texts is an emerging technology trend\nin computational finance. In recent research, a substantial effort has been\ninvested to develop sophisticated financial polarity-lexicons that can be used\nto investigate how financial sentiments relate to future company performance.\nHowever, based on experience from other fields, where sentiment analysis is\ncommonly applied, it is well-known that the overall semantic orientation of a\nsentence may differ from the prior polarity of individual words. The objective\nof this article is to investigate how semantic orientations can be better\ndetected in financial and economic news by accommodating the overall\nphrase-structure information and domain-specific use of language. Our three\nmain contributions are: (1) establishment of a human-annotated finance\nphrase-bank, which can be used as benchmark for training and evaluating\nalternative models; (2) presentation of a technique to enhance financial\nlexicons with attributes that help to identify expected direction of events\nthat affect overall sentiment; (3) development of a linearized phrase-structure\nmodel for detecting contextual semantic orientations in financial and economic\nnews texts. The relevance of the newly added lexicon features and the benefit\nof using the proposed learning-algorithm are demonstrated in a comparative\nstudy against previously used general sentiment models as well as the popular\nword frequency models used in recent financial studies. The proposed framework\nis parsimonious and avoids the explosion in feature-space caused by the use of\nconventional n-gram features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.5736v1",
            "date": "2013-07-19 05:27:46+00:00",
            "title": "Speaker Independent Continuous Speech to Text Converter for Mobile Application",
            "authors": [
                "R. Sandanalakshmi",
                "P. Abinaya Viji",
                "M. Kiruthiga",
                "M. Manjari",
                "M. Sharina"
            ],
            "abstract": "An efficient speech to text converter for mobile application is presented in\nthis work. The prime motive is to formulate a system which would give optimum\nperformance in terms of complexity, accuracy, delay and memory requirements for\nmobile environment. The speech to text converter consists of two stages namely\nfront-end analysis and pattern recognition. The front end analysis involves\npreprocessing and feature extraction. The traditional voice activity detection\nalgorithms which track only energy cannot successfully identify potential\nspeech from input because the unwanted part of the speech also has some energy\nand appears to be speech. In the proposed system, VAD that calculates energy of\nhigh frequency part separately as zero crossing rate to differentiate noise\nfrom speech is used. Mel Frequency Cepstral Coefficient (MFCC) is used as\nfeature extraction method and Generalized Regression Neural Network is used as\nrecognizer. MFCC provides low word error rate and better feature extraction.\nNeural Network improves the accuracy. Thus a small database containing all\npossible syllable pronunciation of the user is sufficient to give recognition\naccuracy closer to 100%. Thus the proposed technique entertains realization of\nreal time speaker independent applications like mobile phones, PDAs etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.6726v1",
            "date": "2013-07-25 12:53:54+00:00",
            "title": "Information content versus word length in natural language: A reply to Ferrer-i-Cancho and Moscoso del Prado Martin [arXiv:1209.1751]",
            "authors": [
                "Steven T. Piantadosi",
                "Harry Tily",
                "Edward Gibson"
            ],
            "abstract": "Recently, Ferrer i Cancho and Moscoso del Prado Martin [arXiv:1209.1751]\nargued that an observed linear relationship between word length and average\nsurprisal (Piantadosi, Tily, & Gibson, 2011) is not evidence for communicative\nefficiency in human language. We discuss several shortcomings of their approach\nand critique: their model critically rests on inaccurate assumptions, is\nincapable of explaining key surprisal patterns in language, and is incompatible\nwith recent behavioral results. More generally, we argue that statistical\nmodels must not critically rely on assumptions that are incompatible with the\nreal system under study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.7973v1",
            "date": "2013-07-30 13:37:09+00:00",
            "title": "Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction",
            "authors": [
                "Jason Weston",
                "Antoine Bordes",
                "Oksana Yakhnenko",
                "Nicolas Usunier"
            ],
            "abstract": "This paper proposes a novel approach for relation extraction from free text\nwhich is trained to jointly use information from the text and from existing\nknowledge. Our model is based on two scoring functions that operate by learning\nlow-dimensional embeddings of words and of entities and relationships from a\nknowledge base. We empirically show on New York Times articles aligned with\nFreebase relations that our approach is able to efficiently use the extra\ninformation provided by a large subset of Freebase data (4M entities, 23k\nrelationships) to improve over existing methods that rely on text features\nalone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.1249v1",
            "date": "2013-10-04 13:04:45+00:00",
            "title": "Reading Stockholm Riots 2013 in social media by text-mining",
            "authors": [
                "Andrzej Jarynowski",
                "Amir Rostami"
            ],
            "abstract": "The riots in Stockholm in May 2013 were an event that reverberated in the\nworld media for its dimension of violence that had spread through the Swedish\ncapital. In this study we have investigated the role of social media in\ncreating media phenomena via text mining and natural language processing. We\nhave focused on two channels of communication for our analysis: Twitter and\nPoloniainfo.se (Forum of Polish community in Sweden). Our preliminary results\nshow some hot topics driving discussion related mostly to Swedish Police and\nSwedish Politics by counting word usage. Typical features for media\nintervention are presented. We have built networks of most popular phrases,\nclustered by categories (geography, media institution, etc.). Sentiment\nanalysis shows negative connotation with Police. The aim of this preliminary\nexploratory quantitative study was to generate questions and hypotheses, which\nwe could carefully follow by deeper more qualitative methods.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1310.2408v1",
            "date": "2013-10-09 09:23:10+00:00",
            "title": "Improved Bayesian Logistic Supervised Topic Models with Data Augmentation",
            "authors": [
                "Jun Zhu",
                "Xun Zheng",
                "Bo Zhang"
            ],
            "abstract": "Supervised topic models with a logistic likelihood have two issues that\npotentially limit their practical use: 1) response variables are usually\nover-weighted by document word counts; and 2) existing variational inference\nmethods make strict mean-field assumptions. We address these issues by: 1)\nintroducing a regularization constant to better balance the two parts based on\nan optimization formulation of Bayesian inference; and 2) developing a simple\nGibbs sampling algorithm by introducing auxiliary Polya-Gamma variables and\ncollapsing out Dirichlet variables. Our augment-and-collapse sampling algorithm\nhas analytical forms of each conditional distribution without making any\nrestricting assumptions and can be easily parallelized. Empirical results\ndemonstrate significant improvements on prediction performance and time\nefficiency.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1310.2479v3",
            "date": "2013-10-09 13:38:01+00:00",
            "title": "Spatio-temporal variation of conversational utterances on Twitter",
            "authors": [
                "Christian M. Alis",
                "May T. Lim"
            ],
            "abstract": "Conversations reflect the existing norms of a language. Previously, we found\nthat utterance lengths in English fictional conversations in books and movies\nhave shortened over a period of 200 years. In this work, we show that this\nshortening occurs even for a brief period of 3 years (September 2009-December\n2012) using 229 million utterances from Twitter. Furthermore, the subset of\ngeographically-tagged tweets from the United States show an inverse proportion\nbetween utterance lengths and the state-level percentage of the Black\npopulation. We argue that shortening of utterances can be explained by the\nincreasing usage of jargon including coined words.",
            "journal ref": "PLoS ONE 8: e77793",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1310.3099v2",
            "date": "2013-10-11 12:07:57+00:00",
            "title": "A Bayesian Network View on Acoustic Model-Based Techniques for Robust Speech Recognition",
            "authors": [
                "Roland Maas",
                "Christian Huemmer",
                "Armin Sehr",
                "Walter Kellermann"
            ],
            "abstract": "This article provides a unifying Bayesian network view on various approaches\nfor acoustic model adaptation, missing feature, and uncertainty decoding that\nare well-known in the literature of robust automatic speech recognition. The\nrepresentatives of these classes can often be deduced from a Bayesian network\nthat extends the conventional hidden Markov models used in speech recognition.\nThese extensions, in turn, can in many cases be motivated from an underlying\nobservation model that relates clean and distorted feature vectors. By\nconverting the observation models into a Bayesian network representation, we\nformulate the corresponding compensation rules leading to a unified view on\nknown derivations as well as to new formulations for certain approaches. The\ngeneric Bayesian perspective provided in this contribution thus highlights\nstructural differences and similarities between the analyzed approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1310.3333v1",
            "date": "2013-10-12 03:48:38+00:00",
            "title": "Visualizing Bags of Vectors",
            "authors": [
                "Sriramkumar Balasubramanian",
                "Raghuram Reddy Nagireddy"
            ],
            "abstract": "The motivation of this work is two-fold - a) to compare between two different\nmodes of visualizing data that exists in a bag of vectors format b) to propose\na theoretical model that supports a new mode of visualizing data. Visualizing\nhigh dimensional data can be achieved using Minimum Volume Embedding, but the\ndata has to exist in a format suitable for computing similarities while\npreserving local distances. This paper compares the visualization between two\nmethods of representing data and also proposes a new method providing sample\nvisualizations for that method.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1310.3499v1",
            "date": "2013-10-13 18:17:54+00:00",
            "title": "Forecasting of Events by Tweet Data Mining",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "This paper describes the analysis of quantitative characteristics of frequent\nsets and association rules in the posts of Twitter microblogs related to\ndifferent event discussions. For the analysis, we used a theory of frequent\nsets, association rules and a theory of formal concept analysis. We revealed\nthe frequent sets and association rules which characterize the semantic\nrelations between the concepts of analyzed subjects. The support of some\nfrequent sets reaches its global maximum before the expected event but with\nsome time delay. Such frequent sets may be considered as predictive markers\nthat characterize the significance of expected events for blogosphere users. We\nshowed that the time dynamics of confidence in some revealed association rules\ncan also have predictive characteristics. Exceeding a certain threshold may be\na signal for corresponding reaction in the society within the time interval\nbetween the maximum and the probable coming of an event. In this paper, we\nconsidered two types of events: the Olympic tennis tournament final in London,\n2012 and the prediction of Eurovision 2013 winner.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1310.3500v1",
            "date": "2013-10-13 18:22:44+00:00",
            "title": "Can Twitter Predict Royal Baby's Name ?",
            "authors": [
                "Bohdan Pavlyshenko"
            ],
            "abstract": "In this paper, we analyze the existence of possible correlation between\npublic opinion of twitter users and the decision-making of persons who are\ninfluential in the society. We carry out this analysis on the example of the\ndiscussion of probable name of the British crown baby, born in July, 2013. In\nour study, we use the methods of quantitative processing of natural language,\nthe theory of frequent sets, the algorithms of visual displaying of users'\ncommunities. We also analyzed the time dynamics of keyword frequencies. The\nanalysis showed that the main predictable name was dominating in the spectrum\nof names before the official announcement. Using the theories of frequent sets,\nwe showed that the full name consisting of three component names was the part\nof top 5 by the value of support. It was revealed that the structure of\ndynamically formed users' communities participating in the discussion is\ndetermined by only a few leaders who influence significantly the viewpoints of\nother users.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1310.4546v1",
            "date": "2013-10-16 23:28:53+00:00",
            "title": "Distributed Representations of Words and Phrases and their Compositionality",
            "authors": [
                "Tomas Mikolov",
                "Ilya Sutskever",
                "Kai Chen",
                "Greg Corrado",
                "Jeffrey Dean"
            ],
            "abstract": "The recently introduced continuous Skip-gram model is an efficient method for\nlearning high-quality distributed vector representations that capture a large\nnumber of precise syntactic and semantic word relationships. In this paper we\npresent several extensions that improve both the quality of the vectors and the\ntraining speed. By subsampling of the frequent words we obtain significant\nspeedup and also learn more regular word representations. We also describe a\nsimple alternative to the hierarchical softmax called negative sampling. An\ninherent limitation of word representations is their indifference to word order\nand their inability to represent idiomatic phrases. For example, the meanings\nof \"Canada\" and \"Air\" cannot be easily combined to obtain \"Air Canada\".\nMotivated by this example, we present a simple method for finding phrases in\ntext, and show that learning good vector representations for millions of\nphrases is possible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.4909v1",
            "date": "2013-10-18 04:18:09+00:00",
            "title": "Text Classification For Authorship Attribution Analysis",
            "authors": [
                "M. Sudheep Elayidom",
                "Chinchu Jose",
                "Anitta Puthussery",
                "Neenu K Sasi"
            ],
            "abstract": "Authorship attribution mainly deals with undecided authorship of literary\ntexts. Authorship attribution is useful in resolving issues like uncertain\nauthorship, recognize authorship of unknown texts, spot plagiarism so on.\nStatistical methods can be used to set apart the approach of an author\nnumerically. The basic methodologies that are made use in computational\nstylometry are word length, sentence length, vocabulary affluence, frequencies\netc. Each author has an inborn style of writing, which is particular to\nhimself. Statistical quantitative techniques can be used to differentiate the\napproach of an author in a numerical way. The problem can be broken down into\nthree sub problems as author identification, author characterization and\nsimilarity detection. The steps involved are pre-processing, extracting\nfeatures, classification and author identification. For this different\nclassifiers can be used. Here fuzzy learning classifier and SVM are used. After\nauthor identification the SVM was found to have more accuracy than Fuzzy\nclassifier. Later combined the classifiers to obtain a better accuracy when\ncompared to individual SVM and fuzzy classifier.",
            "journal ref": "Advanced Computing: An International Journal (ACIJ), Vol.4, No.5,\n  September 2013",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1310.4938v1",
            "date": "2013-10-18 08:10:32+00:00",
            "title": "A Logic-based Approach for Recognizing Textual Entailment Supported by Ontological Background Knowledge",
            "authors": [
                "Andreas Wotzlaw",
                "Ravi Coote"
            ],
            "abstract": "We present the architecture and the evaluation of a new system for\nrecognizing textual entailment (RTE). In RTE we want to identify automatically\nthe type of a logical relation between two input texts. In particular, we are\ninterested in proving the existence of an entailment between them. We conceive\nour system as a modular environment allowing for a high-coverage syntactic and\nsemantic text analysis combined with logical inference. For the syntactic and\nsemantic analysis we combine a deep semantic analysis with a shallow one\nsupported by statistical models in order to increase the quality and the\naccuracy of results. For RTE we use logical inference of first-order employing\nmodel-theoretic techniques and automated reasoning tools. The inference is\nsupported with problem-relevant background knowledge extracted automatically\nand on demand from external sources like, e.g., WordNet, YAGO, and OpenCyc, or\nother, more experimental sources with, e.g., manually defined presupposition\nresolutions, or with axiomatized general and common sense knowledge. The\nresults show that fine-grained and consistent knowledge coming from diverse\nsources is a necessary condition determining the correctness and traceability\nof results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.5042v1",
            "date": "2013-10-18 14:50:39+00:00",
            "title": "Distributional semantics beyond words: Supervised learning of analogy and paraphrase",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "There have been several efforts to extend distributional semantics beyond\nindividual words, to measure the similarity of word pairs, phrases, and\nsentences (briefly, tuples; ordered sets of words, contiguous or\nnoncontiguous). One way to extend beyond words is to compare two tuples using a\nfunction that combines pairwise similarities between the component words in the\ntuples. A strength of this approach is that it works with both relational\nsimilarity (analogy) and compositional similarity (paraphrase). However, past\nwork required hand-coding the combination function for different tasks. The\nmain contribution of this paper is that combination functions are generated by\nsupervised learning. We achieve state-of-the-art results in measuring\nrelational similarity between word pairs (SAT analogies and SemEval~2012 Task\n2) and measuring compositional similarity between noun-modifier phrases and\nunigrams (multiple-choice paraphrase questions).",
            "journal ref": "Transactions of the Association for Computational Linguistics\n  (TACL), (2013), 1, 353-366",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1310.5884v3",
            "date": "2013-10-22 11:37:36+00:00",
            "title": "The optimality of attaching unlinked labels to unlinked meanings",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Vocabulary learning by children can be characterized by many biases. When\nencountering a new word, children as well as adults, are biased towards\nassuming that it means something totally different from the words that they\nalready know. To the best of our knowledge, the 1st mathematical proof of the\noptimality of this bias is presented here. First, it is shown that this bias is\na particular case of the maximization of mutual information between words and\nmeanings. Second, the optimality is proven within a more general information\ntheoretic framework where mutual information maximization competes with other\ninformation theoretic principles. The bias is a prediction from modern\ninformation theory. The relationship between information theoretic principles\nand the principles of contrast and mutual exclusivity is also shown.",
            "journal ref": "Glottometrics 36, 1-16 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.6772v1",
            "date": "2013-10-24 20:59:27+00:00",
            "title": "Sockpuppet Detection in Wikipedia: A Corpus of Real-World Deceptive Writing for Linking Identities",
            "authors": [
                "Thamar Solorio",
                "Ragib Hasan",
                "Mainul Mizan"
            ],
            "abstract": "This paper describes the corpus of sockpuppet cases we gathered from\nWikipedia. A sockpuppet is an online user account created with a fake identity\nfor the purpose of covering abusive behavior and/or subverting the editing\nregulation process. We used a semi-automated method for crawling and curating a\ndataset of real sockpuppet investigation cases. To the best of our knowledge,\nthis is the first corpus available on real-world deceptive writing. We describe\nthe process for crawling the data and some preliminary results that can be used\nas baseline for benchmarking research. The dataset will be released under a\nCreative Commons license from our project website: http://docsig.cis.uab.edu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1310.6775v1",
            "date": "2013-10-24 21:10:53+00:00",
            "title": "Durkheim Project Data Analysis Report",
            "authors": [
                "Linas Vepstas"
            ],
            "abstract": "This report describes the suicidality prediction models created under the\nDARPA DCAPS program in association with the Durkheim Project\n[http://durkheimproject.org/]. The models were built primarily from\nunstructured text (free-format clinician notes) for several hundred patient\nrecords obtained from the Veterans Health Administration (VHA). The models were\nconstructed using a genetic programming algorithm applied to bag-of-words and\nbag-of-phrases datasets. The influence of additional structured data was\nexplored but was found to be minor. Given the small dataset size,\nclassification between cohorts was high fidelity (98%). Cross-validation\nsuggests these models are reasonably predictive, with an accuracy of 50% to 69%\non five rotating folds, with ensemble averages of 58% to 67%. One particularly\nnoteworthy result is that word-pairs can dramatically improve classification\naccuracy; but this is the case only when one of the words in the pair is\nalready known to have a high predictive value. By contrast, the set of all\npossible word-pairs does not improve on a simple bag-of-words model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1310.7782v2",
            "date": "2013-10-29 12:29:12+00:00",
            "title": "Individual Biases, Cultural Evolution, and the Statistical Nature of Language Universals: The Case of Colour Naming Systems",
            "authors": [
                "Andrea Baronchelli",
                "Vittorio Loreto",
                "Andrea Puglisi"
            ],
            "abstract": "Language universals have long been attributed to an innate Universal Grammar.\nAn alternative explanation states that linguistic universals emerged\nindependently in every language in response to shared cognitive or perceptual\nbiases. A computational model has recently shown how this could be the case,\nfocusing on the paradigmatic example of the universal properties of colour\nnaming patterns, and producing results in quantitative agreement with the\nexperimental data. Here we investigate the role of an individual perceptual\nbias in the framework of the model. We study how, and to what extent, the\nstructure of the bias influences the corresponding linguistic universal\npatterns. We show that the cultural history of a group of speakers introduces\npopulation-specific constraints that act against the pressure for uniformity\narising from the individual bias, and we clarify the interplay between these\ntwo forces.",
            "journal ref": "PLoS ONE 10(5): e0125019 (2015)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1310.8511v2",
            "date": "2013-10-31 14:12:30+00:00",
            "title": "A Preadapted Universal Switch Distribution for Testing Hilberg's Conjecture",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "Hilberg's conjecture about natural language states that the mutual\ninformation between two adjacent long blocks of text grows like a power of the\nblock length. The exponent in this statement can be upper bounded using the\npointwise mutual information estimate computed for a carefully chosen code. The\nbound is the better, the lower the compression rate is but there is a\nrequirement that the code be universal. So as to improve a received upper bound\nfor Hilberg's exponent, in this paper, we introduce two novel universal codes,\ncalled the plain switch distribution and the preadapted switch distribution.\nGenerally speaking, switch distributions are certain mixtures of adaptive\nMarkov chains of varying orders with some additional communication to avoid so\ncalled catch-up phenomenon. The advantage of these distributions is that they\nboth achieve a low compression rate and are guaranteed to be universal. Using\nthe switch distributions we obtain that a sample of a text in English is\nnon-Markovian with Hilberg's exponent being $\\le 0.83$, which improves over the\nprevious bound $\\le 0.94$ obtained using the Lempel-Ziv code.",
            "journal ref": "IEEE Transactions on Information Theory 61(10):5708-5715, 2015",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1309.1501v3",
            "date": "2013-09-05 22:06:58+00:00",
            "title": "Improvements to deep convolutional neural networks for LVCSR",
            "authors": [
                "Tara N. Sainath",
                "Brian Kingsbury",
                "Abdel-rahman Mohamed",
                "George E. Dahl",
                "George Saon",
                "Hagen Soltau",
                "Tomas Beran",
                "Aleksandr Y. Aravkin",
                "Bhuvana Ramabhadran"
            ],
            "abstract": "Deep Convolutional Neural Networks (CNNs) are more powerful than Deep Neural\nNetworks (DNN), as they are able to better reduce spectral variation in the\ninput signal. This has also been confirmed experimentally, with CNNs showing\nimprovements in word error rate (WER) between 4-12% relative compared to DNNs\nacross a variety of LVCSR tasks. In this paper, we describe different methods\nto further improve CNN performance. First, we conduct a deep analysis comparing\nlimited weight sharing and full weight sharing with state-of-the-art features.\nSecond, we apply various pooling strategies that have shown improvements in\ncomputer vision to an LVCSR speech task. Third, we introduce a method to\neffectively incorporate speaker adaptation, namely fMLLR, into log-mel\nfeatures. Fourth, we introduce an effective strategy to use dropout during\nHessian-free sequence training. We find that with these improvements,\nparticularly with fMLLR and dropout, we are able to achieve an additional 2-3%\nrelative improvement in WER on a 50-hour Broadcast News task over our previous\nbest CNN baseline. On a larger 400-hour BN task, we find an additional 4-5%\nrelative improvement over our previous best CNN baseline.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1309.1508v3",
            "date": "2013-09-05 23:21:02+00:00",
            "title": "Accelerating Hessian-free optimization for deep neural networks by implicit preconditioning and sampling",
            "authors": [
                "Tara N. Sainath",
                "Lior Horesh",
                "Brian Kingsbury",
                "Aleksandr Y. Aravkin",
                "Bhuvana Ramabhadran"
            ],
            "abstract": "Hessian-free training has become a popular parallel second or- der\noptimization technique for Deep Neural Network training. This study aims at\nspeeding up Hessian-free training, both by means of decreasing the amount of\ndata used for training, as well as through reduction of the number of Krylov\nsubspace solver iterations used for implicit estimation of the Hessian. In this\npaper, we develop an L-BFGS based preconditioning scheme that avoids the need\nto access the Hessian explicitly. Since L-BFGS cannot be regarded as a\nfixed-point iteration, we further propose the employment of flexible Krylov\nsubspace solvers that retain the desired theoretical convergence guarantees of\ntheir conventional counterparts. Second, we propose a new sampling algorithm,\nwhich geometrically increases the amount of data utilized for gradient and\nKrylov subspace iteration calculations. On a 50-hr English Broadcast News task,\nwe find that these methodologies provide roughly a 1.5x speed-up, whereas, on a\n300-hr Switchboard task, these techniques provide over a 2.3x speedup, with no\nloss in WER. These results suggest that even further speed-up is expected, as\nproblems scale and complexity grows.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1312.0976v2",
            "date": "2013-12-03 23:04:36+00:00",
            "title": "Multilinguals and Wikipedia Editing",
            "authors": [
                "Scott A. Hale"
            ],
            "abstract": "This article analyzes one month of edits to Wikipedia in order to examine the\nrole of users editing multiple language editions (referred to as multilingual\nusers). Such multilingual users may serve an important function in diffusing\ninformation across different language editions of the encyclopedia, and prior\nwork has suggested this could reduce the level of self-focus bias in each\nedition. This study finds multilingual users are much more active than their\nsingle-edition (monolingual) counterparts. They are found in all language\neditions, but smaller-sized editions with fewer users have a higher percentage\nof multilingual users than larger-sized editions. About a quarter of\nmultilingual users always edit the same articles in multiple languages, while\njust over 40% of multilingual users edit different articles in different\nlanguages. When non-English users do edit a second language edition, that\nedition is most frequently English. Nonetheless, several regional and\nlinguistic cross-editing patterns are also present.",
            "journal ref": "Proceedings of the 6th Annual ACM Web Science Conference, WebSci\n  2014, ACM",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1304.7359v2",
            "date": "2013-04-27 11:47:40+00:00",
            "title": "Constant conditional entropy and related hypotheses",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "\u0141ukasz D\u0119bowski",
                "Ferm\u00edn Moscoso del Prado Mart\u00edn"
            ],
            "abstract": "Constant entropy rate (conditional entropies must remain constant as the\nsequence length increases) and uniform information density (conditional\nprobabilities must remain constant as the sequence length increases) are two\ninformation theoretic principles that are argued to underlie a wide range of\nlinguistic phenomena. Here we revise the predictions of these principles to the\nlight of Hilberg's law on the scaling of conditional entropy in language and\nrelated laws. We show that constant entropy rate (CER) and two interpretations\nfor uniform information density (UID), full UID and strong UID, are\ninconsistent with these laws. Strong UID implies CER but the reverse is not\ntrue. Full UID, a particular case of UID, leads to costly uncorrelated\nsequences that are totally unrealistic. We conclude that CER and its particular\ncases are incomplete hypotheses about the scaling of conditional entropies.",
            "journal ref": "Journal of Statistical Mechanics, L07001 (2013)",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/1305.5566v2",
            "date": "2013-05-23 21:09:51+00:00",
            "title": "The most controversial topics in Wikipedia: A multilingual and geographical analysis",
            "authors": [
                "Taha Yasseri",
                "Anselm Spoerri",
                "Mark Graham",
                "J\u00e1nos Kert\u00e9sz"
            ],
            "abstract": "We present, visualize and analyse the similarities and differences between\nthe controversial topics related to \"edit wars\" identified in 10 different\nlanguage versions of Wikipedia. After a brief review of the related work we\ndescribe the methods developed to locate, measure, and categorize the\ncontroversial topics in the different languages. Visualizations of the degree\nof overlap between the top 100 lists of most controversial articles in\ndifferent languages and the content related to geographical locations will be\npresented. We discuss what the presented analysis and visualizations can tell\nus about the multicultural aspects of Wikipedia and practices of\npeer-production. Our results indicate that Wikipedia is more than just an\nencyclopaedia; it is also a window into convergent and divergent social-spatial\npriorities, interests and preferences.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1311.2702v1",
            "date": "2013-11-12 07:44:10+00:00",
            "title": "Verifiable Source Code Documentation in Controlled Natural Language",
            "authors": [
                "Tobias Kuhn",
                "Alexandre Bergel"
            ],
            "abstract": "Writing documentation about software internals is rarely considered a\nrewarding activity. It is highly time-consuming and the resulting documentation\nis fragile when the software is continuously evolving in a multi-developer\nsetting. Unfortunately, traditional programming environments poorly support the\nwriting and maintenance of documentation. Consequences are severe as the lack\nof documentation on software structure negatively impacts the overall quality\nof the software product. We show that using a controlled natural language with\na reasoner and a query engine is a viable technique for verifying the\nconsistency and accuracy of documentation and source code. Using ACE, a\nstate-of-the-art controlled natural language, we present positive results on\nthe comprehensibility and the general feasibility of creating and verifying\ndocumentation. As a case study, we used automatic documentation verification to\nidentify and fix severe flaws in the architecture of a non-trivial piece of\nsoftware. Moreover, a user experiment shows that our language is faster and\neasier to learn and understand than other formal languages for software\ndocumentation.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1311.5427v2",
            "date": "2013-11-20 02:43:22+00:00",
            "title": "Complexity measurement of natural and artificial languages",
            "authors": [
                "Gerardo Febres",
                "Klaus Jaffe",
                "Carlos Gershenson"
            ],
            "abstract": "We compared entropy for texts written in natural languages (English, Spanish)\nand artificial languages (computer software) based on a simple expression for\nthe entropy as a function of message length and specific word diversity. Code\ntext written in artificial languages showed higher entropy than text of similar\nlength expressed in natural languages. Spanish texts exhibit more symbolic\ndiversity than English ones. Results showed that algorithms based on complexity\nmeasures differentiate artificial from natural languages, and that text\nanalysis based on complexity measures allows the unveiling of important aspects\nof their nature. We propose specific expressions to examine entropy related\naspects of tests and estimate the values of entropy, emergence,\nself-organization and complexity based on specific diversity and message\nlength.",
            "journal ref": "Complexity 20 6 429- (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1307.3040v2",
            "date": "2013-07-11 10:01:29+00:00",
            "title": "Between Sense and Sensibility: Declarative narrativisation of mental models as a basis and benchmark for visuo-spatial cognition and computation focussed collaborative cognitive systems",
            "authors": [
                "Mehul Bhatt"
            ],
            "abstract": "What lies between `\\emph{sensing}' and `\\emph{sensibility}'? In other words,\nwhat kind of cognitive processes mediate sensing capability, and the formation\nof sensible impressions ---e.g., abstractions, analogies, hypotheses and theory\nformation, beliefs and their revision, argument formation--- in domain-specific\nproblem solving, or in regular activities of everyday living, working and\nsimply going around in the environment? How can knowledge and reasoning about\nsuch capabilities, as exhibited by humans in particular problem contexts, be\nused as a model and benchmark for the development of collaborative cognitive\n(interaction) systems concerned with human assistance, assurance, and\nempowerment?\n  We pose these questions in the context of a range of assistive technologies\nconcerned with \\emph{visuo-spatial perception and cognition} tasks encompassing\naspects such as commonsense, creativity, and the application of specialist\ndomain knowledge and problem-solving thought processes. Assistive technologies\nbeing considered include: (a) human activity interpretation; (b) high-level\ncognitive rovotics; (c) people-centred creative design in domains such as\narchitecture & digital media creation, and (d) qualitative analyses geographic\ninformation systems. Computational narratives not only provide a rich cognitive\nbasis, but they also serve as a benchmark of functional performance in our\ndevelopment of computational cognitive assistance systems. We posit that\ncomputational narrativisation pertaining to space, actions, and change provides\na useful model of \\emph{visual} and \\emph{spatio-temporal thinking} within a\nwide-range of problem-solving tasks and application areas where collaborative\ncognitive systems could serve an assistive and empowering function.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1303.6175v1",
            "date": "2013-03-25 15:43:48+00:00",
            "title": "Compression as a universal principle of animal behavior",
            "authors": [
                "R. Ferrer-i-Cancho",
                "A. Hern\u00e1ndez-Fern\u00e1ndez",
                "D. Lusseau",
                "G. Agoramoorthy",
                "M. J. Hsu",
                "S. Semple"
            ],
            "abstract": "A key aim in biology and psychology is to identify fundamental principles\nunderpinning the behavior of animals, including humans. Analyses of human\nlanguage and the behavior of a range of non-human animal species have provided\nevidence for a common pattern underlying diverse behavioral phenomena: words\nfollow Zipf's law of brevity (the tendency of more frequently used words to be\nshorter), and conformity to this general pattern has been seen in the behavior\nof a number of other animals. It has been argued that the presence of this law\nis a sign of efficient coding in the information theoretic sense. However, no\nstrong direct connection has been demonstrated between the law and compression,\nthe information theoretic principle of minimizing the expected length of a\ncode. Here we show that minimizing the expected code length implies that the\nlength of a word cannot increase as its frequency increases. Furthermore, we\nshow that the mean code length or duration is significantly small in human\nlanguage, and also in the behavior of other species in all cases where\nagreement with the law of brevity has been found. We argue that compression is\na general principle of animal behavior, that reflects selection for efficiency\nof coding.",
            "journal ref": "None",
            "category": "q-bio.NC"
        }
    ],
    "2014": [
        {
            "url": "http://arxiv.org/abs/1401.0569v2",
            "date": "2014-01-03 00:57:13+00:00",
            "title": "Natural Language Processing in Biomedicine: A Unified System Architecture Overview",
            "authors": [
                "Son Doan",
                "Mike Conway",
                "Tu Minh Phuong",
                "Lucila Ohno-Machado"
            ],
            "abstract": "In modern electronic medical records (EMR) much of the clinically important\ndata - signs and symptoms, symptom severity, disease status, etc. - are not\nprovided in structured data fields, but rather are encoded in clinician\ngenerated narrative text. Natural language processing (NLP) provides a means of\n\"unlocking\" this important data source for applications in clinical decision\nsupport, quality assurance, and public health. This chapter provides an\noverview of representative NLP systems in biomedicine based on a unified\narchitectural view. A general architecture in an NLP system consists of two\nmain components: background knowledge that includes biomedical knowledge\nresources and a framework that integrates NLP tools to process text. Systems\ndiffer in both components, which we will review briefly. Additionally,\nchallenges facing current research efforts in biomedical NLP include the\npaucity of large, publicly available annotated corpora, although initiatives\nthat facilitate data sharing, system evaluation, and collaborative work between\nresearchers in clinical NLP are starting to emerge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.0640v1",
            "date": "2014-01-03 13:07:29+00:00",
            "title": "Multi-Topic Multi-Document Summarizer",
            "authors": [
                "Fatma El-Ghannam",
                "Tarek El-Shishtawy"
            ],
            "abstract": "Current multi-document summarization systems can successfully extract summary\nsentences, however with many limitations including: low coverage, inaccurate\nextraction to important sentences, redundancy and poor coherence among the\nselected sentences. The present study introduces a new concept of centroid\napproach and reports new techniques for extracting summary sentences for\nmulti-document. In both techniques keyphrases are used to weigh sentences and\ndocuments. The first summarization technique (Sen-Rich) prefers maximum\nrichness sentences. While the second (Doc-Rich), prefers sentences from\ncentroid document. To demonstrate the new summarization system application to\nextract summaries of Arabic documents we performed two experiments. First, we\napplied Rouge measure to compare the new techniques among systems presented at\nTAC2011. The results show that Sen-Rich outperformed all systems in ROUGE-S.\nSecond, the system was applied to summarize multi-topic documents. Using human\nevaluators, the results show that Doc-Rich is the superior, where summary\nsentences characterized by extra coverage and more cohesion.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 5, No 6, December 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.0660v1",
            "date": "2014-01-03 15:37:19+00:00",
            "title": "Plurals: individuals and sets in a richly typed semantics",
            "authors": [
                "Bruno Mery",
                "Richard Moot",
                "Christian Retor\u00e9"
            ],
            "abstract": "We developed a type-theoretical framework for natural lan- guage semantics\nthat, in addition to the usual Montagovian treatment of compositional\nsemantics, includes a treatment of some phenomena of lex- ical semantic:\ncoercions, meaning, transfers, (in)felicitous co-predication. In this setting\nwe see how the various readings of plurals (collective, dis- tributive,\ncoverings,...) can be modelled.",
            "journal ref": "LENSL'10 - 10th Workshop on Logic and Engineering of Natural\n  Semantics of Language, Japanese Symposium for Artifitial Intelligence,\n  International Society for AI - 2013 (2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.1158v1",
            "date": "2014-01-06 18:03:11+00:00",
            "title": "Effective Slot Filling Based on Shallow Distant Supervision Methods",
            "authors": [
                "Benjamin Roth",
                "Tassilo Barth",
                "Michael Wiegand",
                "Mittul Singh",
                "Dietrich Klakow"
            ],
            "abstract": "Spoken Language Systems at Saarland University (LSV) participated this year\nwith 5 runs at the TAC KBP English slot filling track. Effective algorithms for\nall parts of the pipeline, from document retrieval to relation prediction and\nresponse post-processing, are bundled in a modular end-to-end relation\nextraction system called RelationFactory. The main run solely focuses on\nshallow techniques and achieved significant improvements over LSV's last year's\nsystem, while using the same training data and patterns. Improvements mainly\nhave been obtained by a feature representation focusing on surface skip n-grams\nand improved scoring for extracted distant supervision patterns. Important\nfactors for effective extraction are the training and tuning scheme for distant\nsupervision classifiers, and the query expansion by a translation model based\non Wikipedia links. In the TAC KBP 2013 English Slotfilling evaluation, the\nsubmitted main run of the LSV RelationFactory system achieved the top-ranked\nF1-score of 37.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.2517v1",
            "date": "2014-01-11 10:35:37+00:00",
            "title": "The semantic similarity ensemble",
            "authors": [
                "Andrea Ballatore",
                "Michela Bertolotto",
                "David C. Wilson"
            ],
            "abstract": "Computational measures of semantic similarity between geographic terms\nprovide valuable support across geographic information retrieval, data mining,\nand information integration. To date, a wide variety of approaches to\ngeo-semantic similarity have been devised. A judgment of similarity is not\nintrinsically right or wrong, but obtains a certain degree of cognitive\nplausibility, depending on how closely it mimics human behavior. Thus selecting\nthe most appropriate measure for a specific task is a significant challenge. To\naddress this issue, we make an analogy between computational similarity\nmeasures and soliciting domain expert opinions, which incorporate a subjective\nset of beliefs, perceptions, hypotheses, and epistemic biases. Following this\nanalogy, we define the semantic similarity ensemble (SSE) as a composition of\ndifferent similarity measures, acting as a panel of experts having to reach a\ndecision on the semantic similarity of a set of geographic terms. The approach\nis evaluated in comparison to human judgments, and results indicate that an SSE\nperforms better than the average of its parts. Although the best member tends\nto outperform the ensemble, all ensembles outperform the average performance of\neach ensemble's member. Hence, in contexts where the best measure is unknown,\nthe ensemble provides a more cognitively plausible approach.",
            "journal ref": "Journal of Spatial Information Science (JOSIS), Number 7 (2013),\n  pp. 27-44",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.2641v1",
            "date": "2014-01-12 16:49:53+00:00",
            "title": "Towards a Generic Framework for the Development of Unicode Based Digital Sindhi Dictionaries",
            "authors": [
                "Imdad Ali Ismaili",
                "Zeeshan Bhatti",
                "Azhar Ali Shah"
            ],
            "abstract": "Dictionaries are essence of any language providing vital linguistic recourse\nfor the language learners, researchers and scholars. This paper focuses on the\nmethodology and techniques used in developing software architecture for a\nUBSESD (Unicode Based Sindhi to English and English to Sindhi Dictionary). The\nproposed system provides an accurate solution for construction and\nrepresentation of Unicode based Sindhi characters in a dictionary implementing\nHash Structure algorithm and a custom java Object as its internal data\nstructure saved in a file. The System provides facilities for Insertion,\nDeletion and Editing of new records of Sindhi. Through this framework any type\nof Sindhi to English and English to Sindhi Dictionary (belonging to different\ndomains of knowledge, e.g. engineering, medicine, computer, biology etc.) could\nbe developed easily with accurate representation of Unicode Characters in font\nindependent manner.",
            "journal ref": "Mehran University Research Journal of Engineering & Technology\n  Volume 31, No. 1, January 2012 [ISSN 0254-7821]",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.2663v1",
            "date": "2014-01-12 19:52:49+00:00",
            "title": "Dictionary-Based Concept Mining: An Application for Turkish",
            "authors": [
                "Cem R\u0131fk\u0131 Ayd\u0131n",
                "Ali Erkan",
                "Tunga G\u00fcng\u00f6r",
                "Hidayet Tak\u00e7\u0131"
            ],
            "abstract": "In this study, a dictionary-based method is used to extract expressive\nconcepts from documents. So far, there have been many studies concerning\nconcept mining in English, but this area of study for Turkish, an agglutinative\nlanguage, is still immature. We used dictionary instead of WordNet, a lexical\ndatabase grouping words into synsets that is widely used for concept\nextraction. The dictionaries are rarely used in the domain of concept mining,\nbut taking into account that dictionary entries have synonyms, hypernyms,\nhyponyms and other relationships in their meaning texts, the success rate has\nbeen high for determining concepts. This concept extraction method is\nimplemented on documents, that are collected from different corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.2937v1",
            "date": "2014-01-13 18:05:28+00:00",
            "title": "A survey of methods to ease the development of highly multilingual text mining applications",
            "authors": [
                "Ralf Steinberger"
            ],
            "abstract": "Multilingual text processing is useful because the information content found\nin different languages is complementary, both regarding facts and opinions.\nWhile Information Extraction and other text mining software can, in principle,\nbe developed for many languages, most text analysis tools have only been\napplied to small sets of languages because the development effort per language\nis large. Self-training tools obviously alleviate the problem, but even the\neffort of providing training data and of manually tuning the results is usually\nconsiderable. In this paper, we gather insights by various multilingual system\ndevelopers on how to minimise the effort of developing natural language\nprocessing applications for many languages. We also explain the main guidelines\nunderlying our own effort to develop complex text mining software for tens of\nlanguages. While these guidelines - most of all: extreme simplicity - can be\nvery restrictive and limiting, we believe to have shown the feasibility of the\napproach through the development of the Europe Media Monitor (EMM) family of\napplications (http://emm.newsbrief.eu/overview.html). EMM is a set of complex\nmedia monitoring tools that process and analyse up to 100,000 online news\narticles per day in between twenty and fifty languages. We will also touch upon\nthe kind of language resources that would make it easier for all to develop\nhighly multilingual text mining applications. We will argue that - to achieve\nthis - the most needed resources would be freely available, simple, parallel\nand uniform multilingual dictionaries, corpora and software tools.",
            "journal ref": "Language Resources and Evaluation, Volume 46, Issue 2, pp 155-176,\n  June 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.2943v1",
            "date": "2014-01-13 18:25:10+00:00",
            "title": "ONTS: \"Optima\" News Translation System",
            "authors": [
                "Marco Turchi",
                "Martin Atkinson",
                "Alastair Wilcox",
                "Brett Crawley",
                "Stefano Bucci",
                "Ralf Steinberger",
                "Erik Van der Goot"
            ],
            "abstract": "We propose a real-time machine translation system that allows users to select\na news category and to translate the related live news articles from Arabic,\nCzech, Danish, Farsi, French, German, Italian, Polish, Portuguese, Spanish and\nTurkish into English. The Moses-based system was optimised for the news domain\nand differs from other available systems in four ways: (1) News items are\nautomatically categorised on the source side, before translation; (2) Named\nentity translation is optimised by recognising and extracting them on the\nsource side and by re-inserting their translation in the target language,\nmaking use of a separate entity repository; (3) News titles are translated with\na separate translation system which is optimised for the specific style of news\ntitles; (4) The system was optimised for speed in order to cope with the large\nvolume of daily news articles.",
            "journal ref": "Proceedings of the 13th Conference of the European Chapter of the\n  Association for Computational Linguistics, pages 25-30, Avignon, France,\n  April 23 - 27 2012. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3669v1",
            "date": "2014-01-15 17:01:36+00:00",
            "title": "Hrebs and Cohesion Chains as similar tools for semantic text properties research",
            "authors": [
                "D. Tatar",
                "M. Lupea",
                "E. Kapetanios"
            ],
            "abstract": "In this study it is proven that the Hrebs used in Denotation analysis of\ntexts and Cohesion Chains (defined as a fusion between Lexical Chains and\nCoreference Chains) represent similar linguistic tools. This result gives us\nthe possibility to extend to Cohesion Chains (CCs) some important indicators\nas, for example the Kernel of CCs, the topicality of a CC, text concentration,\nCC-diffuseness and mean diffuseness of the text. Let us mention that nowhere in\nthe Lexical Chains or Coreference Chains literature these kinds of indicators\nare introduced and used since now. Similarly, some applications of CCs in the\nstudy of a text (as for example segmentation or summarization of a text) could\nbe realized starting from hrebs. As an illustration of the similarity between\nHrebs and CCs a detailed analyze of the poem \"Lacul\" by Mihai Eminescu is\ngiven.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5674v1",
            "date": "2014-01-18 21:11:30+00:00",
            "title": "Generalized Biwords for Bitext Compression and Translation Spotting",
            "authors": [
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Rafael C. Carrasco",
                "Miguel A. Mart\u00ednez-Prieto",
                "Joaquin Adiego"
            ],
            "abstract": "Large bilingual parallel texts (also known as bitexts) are usually stored in\na compressed form, and previous work has shown that they can be more\nefficiently compressed if the fact that the two texts are mutual translations\nis exploited. For example, a bitext can be seen as a sequence of biwords\n---pairs of parallel words with a high probability of co-occurrence--- that can\nbe used as an intermediate representation in the compression process. However,\nthe simple biword approach described in the literature can only exploit\none-to-one word alignments and cannot tackle the reordering of words. We\ntherefore introduce a generalization of biwords which can describe multi-word\nexpressions and reorderings. We also describe some methods for the binary\ncompression of generalized biword sequences, and compare their performance when\ndifferent schemes are applied to the extraction of the biword sequence. In\naddition, we show that this generalization of biwords allows for the\nimplementation of an efficient algorithm to look on the compressed bitext for\nwords or text segments in one of the texts and retrieve their counterpart\ntranslations in the other text ---an application usually referred to as\ntranslation spotting--- with only some minor modifications in the compression\nalgorithm.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 43, pages\n  389-418, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5693v1",
            "date": "2014-01-15 05:19:15+00:00",
            "title": "Sentence Compression as Tree Transduction",
            "authors": [
                "Trevor Anthony Cohn",
                "Mirella Lapata"
            ],
            "abstract": "This paper presents a tree-to-tree transduction method for sentence\ncompression. Our model is based on synchronous tree substitution grammar, a\nformalism that allows local distortion of the tree topology and can thus\nnaturally capture structural mismatches. We describe an algorithm for decoding\nin this framework and show how the model can be trained discriminatively within\na large margin framework. Experimental results on sentence compression bring\nsignificant improvements over a state-of-the-art model.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  637-674, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5694v1",
            "date": "2014-01-15 05:40:37+00:00",
            "title": "Cross-lingual Annotation Projection for Semantic Roles",
            "authors": [
                "Sebastian Pado",
                "Mirella Lapata"
            ],
            "abstract": "This article considers the task of automatically inducing role-semantic\nannotations in the FrameNet paradigm for new languages. We propose a general\nframework that is based on annotation projection, phrased as a graph\noptimization problem. It is relatively inexpensive and has the potential to\nreduce the human effort involved in creating role-semantic resources. Within\nthis framework, we present projection models that exploit lexical and syntactic\ninformation. We provide an experimental evaluation on an English-German\nparallel corpus which demonstrates the feasibility of inducing high-precision\nGerman semantic role annotation both for manually and automatically annotated\nEnglish data.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 36, pages\n  307-340, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5695v1",
            "date": "2014-01-15 05:39:01+00:00",
            "title": "Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches",
            "authors": [
                "Tahira Naseem",
                "Benjamin Snyder",
                "Jacob Eisenstein",
                "Regina Barzilay"
            ],
            "abstract": "We demonstrate the effectiveness of multilingual learning for unsupervised\npart-of-speech tagging. The central assumption of our work is that by combining\ncues from multiple languages, the structure of each becomes more apparent. We\nconsider two ways of applying this intuition to the problem of unsupervised\npart-of-speech tagging: a model that directly merges tag structures for a pair\nof languages into a single sequence and a second model which instead\nincorporates multilingual context using latent variables. Both approaches are\nformulated as hierarchical Bayesian models, using Markov Chain Monte Carlo\nsampling techniques for inference. Our results demonstrate that by\nincorporating multilingual evidence we can achieve impressive performance gains\nacross a range of scenarios. We also found that performance improves steadily\nas the number of available languages increases.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 36, pages\n  341-385, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5696v1",
            "date": "2014-01-15 05:33:07+00:00",
            "title": "Unsupervised Methods for Determining Object and Relation Synonyms on the Web",
            "authors": [
                "Alexander Pieter Yates",
                "Oren Etzioni"
            ],
            "abstract": "The task of identifying synonymous relations and objects, or synonym\nresolution, is critical for high-quality information extraction. This paper\ninvestigates synonym resolution in the context of unsupervised information\nextraction, where neither hand-tagged training examples nor domain knowledge is\navailable. The paper presents a scalable, fully-implemented system that runs in\nO(KN log N) time in the number of extractions, N, and the maximum number of\nsynonyms per word, K. The system, called Resolver, introduces a probabilistic\nrelational model for predicting whether two strings are co-referential based on\nthe similarity of the assertions containing them. On a set of two million\nassertions extracted from the Web, Resolver resolves objects with 78% precision\nand 68% recall, and resolves relations with 90% precision and 35% recall.\nSeveral variations of resolvers probabilistic model are explored, and\nexperiments demonstrate that under appropriate conditions these variations can\nimprove F1 by 5%. An extension to the basic Resolver system allows it to handle\npolysemous names with 97% precision and 95% recall on a data set from the TREC\ncorpus.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  255-296, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5697v1",
            "date": "2014-01-15 05:21:01+00:00",
            "title": "Wikipedia-based Semantic Interpretation for Natural Language Processing",
            "authors": [
                "Evgeniy Gabrilovich",
                "Shaul Markovitch"
            ],
            "abstract": "Adequate representation of natural language semantics requires access to vast\namounts of common sense and domain-specific world knowledge. Prior work in the\nfield was based on purely statistical techniques that did not make use of\nbackground knowledge, on limited lexicographic knowledge bases such as WordNet,\nor on huge manual efforts such as the CYC project. Here we propose a novel\nmethod, called Explicit Semantic Analysis (ESA), for fine-grained semantic\ninterpretation of unrestricted natural language texts. Our method represents\nmeaning in a high-dimensional space of concepts derived from Wikipedia, the\nlargest encyclopedia in existence. We explicitly represent the meaning of any\ntext in terms of Wikipedia-based concepts. We evaluate the effectiveness of our\nmethod on text categorization and on computing the degree of semantic\nrelatedness between fragments of natural language text. Using ESA results in\nsignificant improvements over the previous state of the art in both tasks.\nImportantly, due to the use of natural concepts, the ESA model is easy to\nexplain to human users.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  443-498, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5698v1",
            "date": "2014-01-15 05:11:43+00:00",
            "title": "Identification of Pleonastic It Using the Web",
            "authors": [
                "Yifan Li",
                "Petr Musilek",
                "Marek Reformat",
                "Loren Wyard-Scott"
            ],
            "abstract": "In a significant minority of cases, certain pronouns, especially the pronoun\nit, can be used without referring to any specific entity. This phenomenon of\npleonastic pronoun usage poses serious problems for systems aiming at even a\nshallow understanding of natural language texts. In this paper, a novel\napproach is proposed to identify such uses of it: the extrapositional cases are\nidentified using a series of queries against the web, and the cleft cases are\nidentified using a simple set of syntactic rules. The system is evaluated with\nfour sets of news articles containing 679 extrapositional cases as well as 78\ncleft constructs. The identification results are comparable to those obtained\nby human efforts.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  339-389, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5699v1",
            "date": "2014-01-15 05:41:08+00:00",
            "title": "Text Relatedness Based on a Word Thesaurus",
            "authors": [
                "George Tsatsaronis",
                "Iraklis Varlamis",
                "Michalis Vazirgiannis"
            ],
            "abstract": "The computation of relatedness between two fragments of text in an automated\nmanner requires taking into account a wide range of factors pertaining to the\nmeaning the two fragments convey, and the pairwise relations between their\nwords. Without doubt, a measure of relatedness between text segments must take\ninto account both the lexical and the semantic relatedness between words. Such\na measure that captures well both aspects of text relatedness may help in many\ntasks, such as text retrieval, classification and clustering. In this paper we\npresent a new approach for measuring the semantic relatedness between words\nbased on their implicit semantic links. The approach exploits only a word\nthesaurus in order to devise implicit semantic links between words. Based on\nthis approach, we introduce Omiotis, a new measure of semantic relatedness\nbetween texts which capitalizes on the word-to-word semantic relatedness\nmeasure (SR) and extends it to measure the relatedness between texts. We\ngradually validate our method: we first evaluate the performance of the\nsemantic relatedness measure between individual words, covering word-to-word\nsimilarity and relatedness, synonym identification and word analogy; then, we\nproceed with evaluating the performance of our method in measuring text-to-text\nsemantic relatedness in two tasks, namely sentence-to-sentence similarity and\nparaphrase recognition. Experimental evaluation shows that the proposed method\noutperforms every lexicon-based method of semantic relatedness in the selected\ntasks and the used data sets, and competes well against corpus-based and hybrid\napproaches.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 37, pages\n  1-39, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5700v1",
            "date": "2014-01-15 05:28:26+00:00",
            "title": "Inferring Shallow-Transfer Machine Translation Rules from Small Parallel Corpora",
            "authors": [
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Mikel L. Forcada"
            ],
            "abstract": "This paper describes a method for the automatic inference of structural\ntransfer rules to be used in a shallow-transfer machine translation (MT) system\nfrom small parallel corpora. The structural transfer rules are based on\nalignment templates, like those used in statistical MT. Alignment templates are\nextracted from sentence-aligned parallel corpora and extended with a set of\nrestrictions which are derived from the bilingual dictionary of the MT system\nand control their application as transfer rules. The experiments conducted\nusing three different language pairs in the free/open-source MT platform\nApertium show that translation quality is improved as compared to word-for-word\ntranslation (when no transfer rules are used), and that the resulting\ntranslation quality is close to that obtained using hand-coded transfer rules.\nThe method we present is entirely unsupervised and benefits from information in\nthe rest of modules of the MT system in which the inferred rules are applied.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  605-635, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6050v1",
            "date": "2014-01-23 16:45:39+00:00",
            "title": "Integrative Semantic Dependency Parsing via Efficient Large-scale Feature Selection",
            "authors": [
                "Hai Zhao",
                "Xiaotian Zhang",
                "Chunyu Kit"
            ],
            "abstract": "Semantic parsing, i.e., the automatic derivation of meaning representation\nsuch as an instantiated predicate-argument structure for a sentence, plays a\ncritical role in deep processing of natural language. Unlike all other top\nsystems of semantic dependency parsing that have to rely on a pipeline\nframework to chain up a series of submodels each specialized for a specific\nsubtask, the one presented in this article integrates everything into one\nmodel, in hopes of achieving desirable integrity and practicality for real\napplications while maintaining a competitive performance. This integrative\napproach tackles semantic parsing as a word pair classification problem using a\nmaximum entropy classifier. We leverage adaptive pruning of argument candidates\nand large-scale feature selection engineering to allow the largest feature\nspace ever in use so far in this field, it achieves a state-of-the-art\nperformance on the evaluation data set for CoNLL-2008 shared task, on top of\nall but one top pipeline system, confirming its feasibility and effectiveness.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 46, pages\n  203-233, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6122v1",
            "date": "2014-01-23 19:03:18+00:00",
            "title": "Identifying Bengali Multiword Expressions using Semantic Clustering",
            "authors": [
                "Tanmoy Chakraborty",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "One of the key issues in both natural language understanding and generation\nis the appropriate processing of Multiword Expressions (MWEs). MWEs pose a huge\nproblem to the precise language processing due to their idiosyncratic nature\nand diversity in lexical, syntactical and semantic properties. The semantics of\na MWE cannot be expressed after combining the semantics of its constituents.\nTherefore, the formalism of semantic clustering is often viewed as an\ninstrument for extracting MWEs especially for resource constraint languages\nlike Bengali. The present semantic clustering approach contributes to locate\nclusters of the synonymous noun tokens present in the document. These clusters\nin turn help measure the similarity between the constituent words of a\npotentially candidate phrase using a vector space model and judge the\nsuitability of this phrase to be a MWE. In this experiment, we apply the\nsemantic clustering approach for noun-noun bigram MWEs, though it can be\nextended to any types of MWEs. In parallel, the well known statistical models,\nnamely Point-wise Mutual Information (PMI), Log Likelihood Ratio (LLR),\nSignificance function are also employed to extract MWEs from the Bengali\ncorpus. The comparative evaluation shows that the semantic clustering approach\noutperforms all other competing statistical models. As a by-product of this\nexperiment, we have started developing a standard lexicon in Bengali that\nserves as a productive Bengali linguistic thesaurus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6330v2",
            "date": "2014-01-24 12:56:36+00:00",
            "title": "A Statistical Parsing Framework for Sentiment Classification",
            "authors": [
                "Li Dong",
                "Furu Wei",
                "Shujie Liu",
                "Ming Zhou",
                "Ke Xu"
            ],
            "abstract": "We present a statistical parsing framework for sentence-level sentiment\nclassification in this article. Unlike previous works that employ syntactic\nparsing results for sentiment analysis, we develop a statistical parser to\ndirectly analyze the sentiment structure of a sentence. We show that\ncomplicated phenomena in sentiment analysis (e.g., negation, intensification,\nand contrast) can be handled the same as simple and straightforward sentiment\nexpressions in a unified and probabilistic way. We formulate the sentiment\ngrammar upon Context-Free Grammars (CFGs), and provide a formal description of\nthe sentiment parsing framework. We develop the parsing model to obtain\npossible sentiment parse trees for a sentence, from which the polarity model is\nproposed to derive the sentiment strength and polarity, and the ranking model\nis dedicated to selecting the best sentiment tree. We train the parser directly\nfrom examples of sentences annotated only with sentiment polarity labels but\nwithout any syntactic annotations or polarity annotations of constituents\nwithin sentences. Therefore we can obtain training data easily. In particular,\nwe train a sentiment parser, s.parser, from a large amount of review sentences\nwith users' ratings as rough sentiment polarity labels. Extensive experiments\non existing benchmark datasets show significant improvements over baseline\nsentiment classification approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6422v1",
            "date": "2014-01-23 02:48:07+00:00",
            "title": "Automatic Aggregation by Joint Modeling of Aspects and Values",
            "authors": [
                "Christina Sauper",
                "Regina Barzilay"
            ],
            "abstract": "We present a model for aggregation of product review snippets by joint aspect\nidentification and sentiment analysis. Our model simultaneously identifies an\nunderlying set of ratable aspects presented in the reviews of a product (e.g.,\nsushi and miso for a Japanese restaurant) and determines the corresponding\nsentiment of each aspect. This approach directly enables discovery of\nhighly-rated or inconsistent aspects of a product. Our generative model admits\nan efficient variational mean-field inference algorithm. It is also easily\nextensible, and we describe several modifications and their effects on model\nstructure and inference. We test our model on two tasks, joint aspect\nidentification and sentiment analysis on a set of Yelp reviews and aspect\nidentification alone on a set of medical summaries. We evaluate the performance\nof the model on aspect identification, sentiment analysis, and per-word\nlabeling accuracy. We demonstrate that our model outperforms applicable\nbaselines by a considerable margin, yielding up to 32% relative error reduction\non aspect identification and up to 20% relative error reduction on sentiment\nanalysis.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 46, pages\n  89-127, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6875v1",
            "date": "2014-01-16 04:48:43+00:00",
            "title": "Context-based Word Acquisition for Situated Dialogue in a Virtual World",
            "authors": [
                "Shaolin Qu",
                "Joyce Y. Chai"
            ],
            "abstract": "To tackle the vocabulary problem in conversational systems, previous work has\napplied unsupervised learning approaches on co-occurring speech and eye gaze\nduring interaction to automatically acquire new words. Although these\napproaches have shown promise, several issues related to human language\nbehavior and human-machine conversation have not been addressed. First,\npsycholinguistic studies have shown certain temporal regularities between human\neye movement and language production. While these regularities can potentially\nguide the acquisition process, they have not been incorporated in the previous\nunsupervised approaches. Second, conversational systems generally have an\nexisting knowledge base about the domain and vocabulary. While the existing\nknowledge can potentially help bootstrap and constrain the acquired new words,\nit has not been incorporated in the previous models. Third, eye gaze could\nserve different functions in human-machine conversation. Some gaze streams may\nnot be closely coupled with speech stream, and thus are potentially detrimental\nto word acquisition. Automated recognition of closely-coupled speech-gaze\nstreams based on conversation context is important. To address these issues, we\ndeveloped new approaches that incorporate user language behavior, domain\nknowledge, and conversation context in word acquisition. We evaluated these\napproaches in the context of situated dialogue in a virtual world. Our\nexperimental results have shown that incorporating the above three types of\ncontextual information significantly improves word acquisition performance.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 37, pages\n  247-277, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6876v1",
            "date": "2014-01-23 02:42:12+00:00",
            "title": "Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages",
            "authors": [
                "Preslav Ivanov Nakov",
                "Hwee Tou Ng"
            ],
            "abstract": "We propose a novel language-independent approach for improving machine\ntranslation for resource-poor languages by exploiting their similarity to\nresource-rich ones. More precisely, we improve the translation from a\nresource-poor source language X_1 into a resource-rich language Y given a\nbi-text containing a limited number of parallel sentences for X_1-Y and a\nlarger bi-text for X_2-Y for some resource-rich language X_2 that is closely\nrelated to X_1. This is achieved by taking advantage of the opportunities that\nvocabulary overlap and similarities between the languages X_1 and X_2 in\nspelling, word order, and syntax offer: (1) we improve the word alignments for\nthe resource-poor language, (2) we further augment it with additional\ntranslation options, and (3) we take care of potential spelling differences\nthrough appropriate transliteration. The evaluation for Indonesian- >English\nusing Malay and for Spanish -> English using Portuguese and pretending Spanish\nis resource-poor shows an absolute gain of up to 1.35 and 3.37 BLEU points,\nrespectively, which is an improvement over the best rivaling approaches, while\nusing much less additional data. Overall, our method cuts the amount of\nnecessary \"real training data by a factor of 2--5.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 44, pages\n  179-222, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.7077v4",
            "date": "2014-01-28 03:48:01+00:00",
            "title": "Quantifying literature quality using complexity criteria",
            "authors": [
                "Gerardo Febres",
                "Klaus Jaffe"
            ],
            "abstract": "We measured entropy and symbolic diversity for English and Spanish texts\nincluding literature Nobel laureates and other famous authors. Entropy, symbol\ndiversity and symbol frequency profiles were compared for these four groups. We\nalso built a scale sensitive to the quality of writing and evaluated its\nrelationship with the Flesch's readability index for English and the\nSzigriszt's perspicuity index for Spanish. Results suggest a correlation\nbetween entropy and word diversity with quality of writing. Text genre also\ninfluences the resulting entropy and diversity of the text. Results suggest the\nplausibility of automated quality assessment of texts.",
            "journal ref": "Journal of Quantitative Linguistics, 2017, Vol 24, Iss 1",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.0314v2",
            "date": "2014-09-01 08:01:54+00:00",
            "title": "Empirical Evaluation of Tree distances for Parser Evaluation",
            "authors": [
                "Taraka Rama"
            ],
            "abstract": "In this empirical study, I compare various tree distance measures --\noriginally developed in computational biology for the purpose of tree\ncomparison -- for the purpose of parser evaluation. I will control for the\nparser setting by comparing the automatically generated parse trees from the\nstate-of-the-art parser Charniak, 2000) with the gold-standard parse trees. The\narticle describes two different tree distance measures (RF and QD) along with\nits variants (GRF and GQD) for the purpose of parser evaluation. The article\nwill argue that RF measure captures similar information as the standard EvalB\nmetric (Sekine and Collins, 1997) and the tree edit distance (Zhang and Shasha,\n1989) applied by Tsarfaty et al. (2011). Finally, the article also provides\nempirical evidence by reporting high correlations between the different tree\ndistances and EvalB metric's scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.2073v1",
            "date": "2014-09-07 02:31:03+00:00",
            "title": "An NLP Assistant for Clide",
            "authors": [
                "Tobias Kortkamp"
            ],
            "abstract": "This report describes an NLP assistant for the collaborative development\nenvironment Clide, that supports the development of NLP applications by\nproviding easy access to some common NLP data structures. The assistant\nvisualizes text fragments and their dependencies by displaying the semantic\ngraph of a sentence, the coreference chain of a paragraph and mined triples\nthat are extracted from a paragraph's semantic graphs and linked using its\ncoreference chain. Using this information and a logic programming library, we\ncreate an NLP database which is used by a series of queries to mine the\ntriples. The algorithm is tested by translating a natural language text\ndescribing a graph to an actual graph that is shown as an annotation in the\ntext editor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3005v1",
            "date": "2014-09-10 09:52:41+00:00",
            "title": "A Study of Association Measures and their Combination for Arabic MWT Extraction",
            "authors": [
                "Abdelkader El Mahdaouy",
                "Sa\u00efd EL Alaoui Ouatik",
                "Eric Gaussier"
            ],
            "abstract": "Automatic Multi-Word Term (MWT) extraction is a very important issue to many\napplications, such as information retrieval, question answering, and text\ncategorization. Although many methods have been used for MWT extraction in\nEnglish and other European languages, few studies have been applied to Arabic.\nIn this paper, we propose a novel, hybrid method which combines linguistic and\nstatistical approaches for Arabic Multi-Word Term extraction. The main\ncontribution of our method is to consider contextual information and both\ntermhood and unithood for association measures at the statistical filtering\nstep. In addition, our technique takes into account the problem of MWT\nvariation in the linguistic filtering step. The performance of the proposed\nstatistical measure (NLC-value) is evaluated using an Arabic environment corpus\nby comparing it with some existing competitors. Experimental results show that\nour NLC-value measure outperforms the other ones in term of precision for both\nbi-grams and tri-grams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3512v1",
            "date": "2014-09-10 19:01:18+00:00",
            "title": "Word Sense Disambiguation using WSD specific Wordnet of Polysemy Words",
            "authors": [
                "Udaya Raj Dhungana",
                "Subarna Shakya",
                "Kabita Baral",
                "Bharat Sharma"
            ],
            "abstract": "This paper presents a new model of WordNet that is used to disambiguate the\ncorrect sense of polysemy word based on the clue words. The related words for\neach sense of a polysemy word as well as single sense word are referred to as\nthe clue words. The conventional WordNet organizes nouns, verbs, adjectives and\nadverbs together into sets of synonyms called synsets each expressing a\ndifferent concept. In contrast to the structure of WordNet, we developed a new\nmodel of WordNet that organizes the different senses of polysemy words as well\nas the single sense words based on the clue words. These clue words for each\nsense of a polysemy word as well as for single sense word are used to\ndisambiguate the correct meaning of the polysemy word in the given context\nusing knowledge based Word Sense Disambiguation (WSD) algorithms. The clue word\ncan be a noun, verb, adjective or adverb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3813v1",
            "date": "2014-09-12 18:37:35+00:00",
            "title": "Incorporating Semi-supervised Features into Discontinuous Easy-First Constituent Parsing",
            "authors": [
                "Yannick Versley"
            ],
            "abstract": "This paper describes adaptations for EaFi, a parser for easy-first parsing of\ndiscontinuous constituents, to adapt it to multiple languages as well as make\nuse of the unlabeled data that was provided as part of the SPMRL shared task\n2014.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4169v1",
            "date": "2014-09-15 08:05:36+00:00",
            "title": "An Algorithm Based on Empirical Methods, for Text-to-Tuneful-Speech Synthesis of Sanskrit Verse",
            "authors": [
                "Rama N.",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "The rendering of Sanskrit poetry from text to speech is a problem that has\nnot been solved before. One reason may be the complications in the language\nitself. We present unique algorithms based on extensive empirical analysis, to\nsynthesize speech from a given text input of Sanskrit verses. Using a\npre-recorded audio units database which is itself tremendously reduced in size\ncompared to the colossal size that would otherwise be required, the algorithms\nwork on producing the best possible, tunefully rendered chanting of the given\nverse. His would enable the visually impaired and those with reading\ndisabilities to easily access the contents of Sanskrit verses otherwise\navailable only in writing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4354v1",
            "date": "2014-09-15 18:08:33+00:00",
            "title": "A Binary Schema and Computational Algorithms to Process Vowel-based Euphonic Conjunctions for Word Searches",
            "authors": [
                "S. V. Kasmir Raja",
                "V. Rajitha",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "Comprehensively searching for words in Sanskrit E-text is a non-trivial\nproblem because words could change their forms in different contexts. One such\ncontext is sandhi or euphonic conjunctions, which cause a word to change owing\nto the presence of adjacent letters or words. The change wrought by these\npossible conjunctions can be so significant in Sanskrit that a simple search\nfor the word in its given form alone can significantly reduce the success level\nof the search. This work presents a representational schema that represents\nletters in a binary format and reduces Paninian rules of euphonic conjunctions\nto simple bit set-unset operations. The work presents an efficient algorithm to\nprocess vowel-based sandhis using this schema. It further presents another\nalgorithm that uses the sandhi processor to generate the possible transformed\nword forms of a given word to use in a comprehensive word search.",
            "journal ref": "International Journal of Applied Engineering Research, ISSN\n  0973-4562, Vol. 9, No. 20, 2014, pp 7127-7142",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4364v1",
            "date": "2014-09-15 18:19:06+00:00",
            "title": "Computational Algorithms Based on the Paninian System to Process Euphonic Conjunctions for Word Searches",
            "authors": [
                "S. V. Kasmir Raja",
                "V. Rajitha",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "Searching for words in Sanskrit E-text is a problem that is accompanied by\ncomplexities introduced by features of Sanskrit such as euphonic conjunctions\nor sandhis. A word could occur in an E-text in a transformed form owing to the\noperation of rules of sandhi. Simple word search would not yield these\ntransformed forms of the word. Further, there is no search engine in the\nliterature that can comprehensively search for words in Sanskrit E-texts taking\neuphonic conjunctions into account. This work presents an optimal binary\nrepresentational schema for letters of the Sanskrit alphabet along with\nalgorithms to efficiently process the sandhi rules of Sanskrit grammar. The\nwork further presents an algorithm that uses the sandhi processing algorithm to\nperform a comprehensive word search on E-text.",
            "journal ref": "International Journal of Computer Science and Information\n  Security, Vol. 12, No. 8, 2014, pp. 64-76",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4614v4",
            "date": "2014-09-16 12:59:07+00:00",
            "title": "Lexical Normalisation of Twitter Data",
            "authors": [
                "Bilal Ahmed"
            ],
            "abstract": "Twitter with over 500 million users globally, generates over 100,000 tweets\nper minute . The 140 character limit per tweet, perhaps unintentionally,\nencourages users to use shorthand notations and to strip spellings to their\nbare minimum \"syllables\" or elisions e.g. \"srsly\". The analysis of twitter\nmessages which typically contain misspellings, elisions, and grammatical\nerrors, poses a challenge to established Natural Language Processing (NLP)\ntools which are generally designed with the assumption that the data conforms\nto the basic grammatical structure commonly used in English language. In order\nto make sense of Twitter messages it is necessary to first transform them into\na canonical form, consistent with the dictionary or grammar. This process,\nperformed at the level of individual tokens (\"words\"), is called lexical\nnormalisation. This paper investigates various techniques for lexical\nnormalisation of Twitter data and presents the findings as the techniques are\napplied to process raw data from Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.5502v1",
            "date": "2014-09-19 02:50:04+00:00",
            "title": "Using crowdsourcing system for creating site-specific statistical machine translation engine",
            "authors": [
                "Alexander Kalinin",
                "George Savchenko"
            ],
            "abstract": "A crowdsourcing translation approach is an effective tool for globalization\nof site content, but it is also an important source of parallel linguistic\ndata. For the given site, processed with a crowdsourcing system, a\nsentence-aligned corpus can be fetched, which covers a very narrow domain of\nterminology and language patterns - a site-specific domain. These data can be\nused for training and estimation of site-specific statistical machine\ntranslation engine",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.7386v1",
            "date": "2014-09-25 15:35:27+00:00",
            "title": "Performance of Stanford and Minipar Parser on Biomedical Texts",
            "authors": [
                "Rushdi Shams"
            ],
            "abstract": "In this paper, the performance of two dependency parsers, namely Stanford and\nMinipar, on biomedical texts has been reported. The performance of te parsers\nto assignm dependencies between two biomedical concepts that are already proved\nto be connected is not satisfying. Both Stanford and Minipar, being statistical\nparsers, fail to assign dependency relation between two connected concepts if\nthey are distant by at least one clause. Minipar's performance, in terms of\nprecision, recall and the F-score of the attachment score (e.g., correctly\nidentified head in a dependency), to parse biomedical text is also measured\ntaking the Stanford's as a gold standard. The results suggest that Minipar is\nnot suitable yet to parse biomedical texts. In addition, a qualitative\ninvestigation reveals that the difference between working principles of the\nparsers also play a vital role for Minipar's degraded performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.7619v1",
            "date": "2014-09-25 13:54:37+00:00",
            "title": "Generating Conceptual Metaphors from Proposition Stores",
            "authors": [
                "Ekaterina Ovchinnikova",
                "Vladimir Zaytsev",
                "Suzanne Wertheim",
                "Ross Israel"
            ],
            "abstract": "Contemporary research on computational processing of linguistic metaphors is\ndivided into two main branches: metaphor recognition and metaphor\ninterpretation. We take a different line of research and present an automated\nmethod for generating conceptual metaphors from linguistic data. Given the\ngenerated conceptual metaphors, we find corresponding linguistic metaphors in\ncorpora. In this paper, we describe our approach and its evaluation using\nEnglish and Russian data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.8008v1",
            "date": "2014-09-29 07:11:30+00:00",
            "title": "CRF-based Named Entity Recognition @ICON 2013",
            "authors": [
                "Arjun Das",
                "Utpal Garain"
            ],
            "abstract": "This paper describes performance of CRF based systems for Named Entity\nRecognition (NER) in Indian language as a part of ICON 2013 shared task. In\nthis task we have considered a set of language independent features for all the\nlanguages. Only for English a language specific feature, i.e. capitalization,\nhas been added. Next the use of gazetteer is explored for Bengali, Hindi and\nEnglish. The gazetteers are built from Wikipedia and other sources. Test\nresults show that the system achieves the highest F measure of 88% for English\nand the lowest F measure of 69% for both Tamil and Telugu. Note that for the\nleast performing two languages no gazetteer was used. NER in Bengali and Hindi\nfinds accuracy (F measure) of 87% and 79%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.8581v1",
            "date": "2014-09-29 04:54:03+00:00",
            "title": "Improving the Performance of English-Tamil Statistical Machine Translation System using Source-Side Pre-Processing",
            "authors": [
                "M. Anand Kumar",
                "V. Dhanalakshmi",
                "K. P. Soman",
                "V. Sharmiladevi"
            ],
            "abstract": "Machine Translation is one of the major oldest and the most active research\narea in Natural Language Processing. Currently, Statistical Machine Translation\n(SMT) dominates the Machine Translation research. Statistical Machine\nTranslation is an approach to Machine Translation which uses models to learn\ntranslation patterns directly from data, and generalize them to translate a new\nunseen text. The SMT approach is largely language independent, i.e. the models\ncan be applied to any language pair. Statistical Machine Translation (SMT)\nattempts to generate translations using statistical methods based on bilingual\ntext corpora. Where such corpora are available, excellent results can be\nattained translating similar texts, but such corpora are still not available\nfor many language pairs. Statistical Machine Translation systems, in general,\nhave difficulty in handling the morphology on the source or the target side\nespecially for morphologically rich languages. Errors in morphology or syntax\nin the target language can have severe consequences on meaning of the sentence.\nThey change the grammatical function of words or the understanding of the\nsentence through the incorrect tense information in verb. Baseline SMT also\nknown as Phrase Based Statistical Machine Translation (PBSMT) system does not\nuse any linguistic information and it only operates on surface word form.\nRecent researches shown that adding linguistic information helps to improve the\naccuracy of the translation with less amount of bilingual corpora. Adding\nlinguistic information can be done using the Factored Statistical Machine\nTranslation system through pre-processing steps. This paper investigates about\nhow English side pre-processing is used to improve the accuracy of\nEnglish-Tamil SMT system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.0751v1",
            "date": "2014-12-02 00:53:35+00:00",
            "title": "Tiered Clustering to Improve Lexical Entailment",
            "authors": [
                "John Wieting"
            ],
            "abstract": "Many tasks in Natural Language Processing involve recognizing lexical\nentailment. Two different approaches to this problem have been proposed\nrecently that are quite different from each other. The first is an asymmetric\nsimilarity measure designed to give high scores when the contexts of the\nnarrower term in the entailment are a subset of those of the broader term. The\nsecond is a supervised approach where a classifier is learned to predict\nentailment given a concatenated latent vector representation of the word. Both\nof these approaches are vector space models that use a single context vector as\na representation of the word. In this work, I study the effects of clustering\nwords into senses and using these multiple context vectors to infer entailment\nusing extensions of these two algorithms. I find that this approach offers some\nimprovement to these entailment algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1215v1",
            "date": "2014-12-03 07:16:04+00:00",
            "title": "Mary Astell's words in A Serious Proposal to the Ladies (part I), a lexicographic inquiry with NooJ",
            "authors": [
                "H\u00e9l\u00e8ne Pignot",
                "Odile Piton"
            ],
            "abstract": "In the following article we elected to study with NooJ the lexis of a 17 th\ncentury text, Mary Astell's seminal essay, A Serious Proposal to the Ladies,\npart I, published in 1694. We first focused on the semantics to see how Astell\nbuilds her vindication of the female sex, which words she uses to sensitise\nwomen to their alienated condition and promote their education. Then we studied\nthe morphology of the lexemes (which is different from contemporary English)\nused by the author, thanks to the NooJ tools we have devised for this purpose.\nNooJ has great functionalities for lexicographic work. Its commands and graphs\nprove to be most efficient in the spotting of archaic words or variants in\nspelling. Introduction In our previous articles, we have studied the\nsingularities of 17 th century English within the framework of a diachronic\nanalysis thanks to syntactical and morphological graphs and thanks to the\ndictionaries we have compiled from a corpus that may be expanded overtime. Our\nearly work was based on a limited corpus of English travel literature to Greece\nin the 17 th century. This article deals with a late seventeenth century text\nwritten by a woman philosopher and essayist, Mary Astell (1666--1731),\nconsidered as one of the first English feminists. Astell wrote her essay at a\ntime in English history when women were \"the weaker vessel\" and their main\nbusiness in life was to charm and please men by their looks and submissiveness.\nIn this essay we will see how NooJ can help us analyse Astell's rhetoric (what\npoint of view does she adopt, does she speak in her own name, in the name of\nall women, what is her representation of men and women and their relationships\nin the text, what are the goals of education?). Then we will turn our attention\nto the morphology of words in the text and use NooJ commands and graphs to\ncarry out a lexicographic inquiry into Astell's lexemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1342v1",
            "date": "2014-12-03 14:37:36+00:00",
            "title": "A perspective on the advancement of natural language processing tasks via topological analysis of complex networks",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "Comment on \"Approaching human language with complex networks\" by Cong and Liu\n(Physics of Life Reviews, Volume 11, Issue 4, December 2014, Pages 598-618).",
            "journal ref": "Physics of Life Reviews, v. 11, p. 641-643, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1632v1",
            "date": "2014-12-04 11:53:02+00:00",
            "title": "Deep Learning for Answer Sentence Selection",
            "authors": [
                "Lei Yu",
                "Karl Moritz Hermann",
                "Phil Blunsom",
                "Stephen Pulman"
            ],
            "abstract": "Answer sentence selection is the task of identifying sentences that contain\nthe answer to a given question. This is an important problem in its own right\nas well as in the larger context of open domain question answering. We propose\na novel approach to solving this task via means of distributed representations,\nand learn to match questions with answers by considering their semantic\nencoding. This contrasts prior work on this task, which typically relies on\nclassifiers with large numbers of hand-crafted syntactic and semantic features\nand various external resources. Our approach does not require any feature\nengineering nor does it involve specialist linguistic data, making this model\neasily applicable to a wide range of domains and languages. Experimental\nresults on a standard benchmark dataset from TREC demonstrate that---despite\nits simplicity---our model matches state of the art performance on the answer\nsentence selection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1820v2",
            "date": "2014-12-03 23:26:33+00:00",
            "title": "Context-Dependent Fine-Grained Entity Type Tagging",
            "authors": [
                "Dan Gillick",
                "Nevena Lazic",
                "Kuzman Ganchev",
                "Jesse Kirchner",
                "David Huynh"
            ],
            "abstract": "Entity type tagging is the task of assigning category labels to each mention\nof an entity in a document. While standard systems focus on a small set of\ntypes, recent work (Ling and Weld, 2012) suggests that using a large\nfine-grained label set can lead to dramatic improvements in downstream tasks.\nIn the absence of labeled training data, existing fine-grained tagging systems\nobtain examples automatically, using resolved entities and their types\nextracted from a knowledge base. However, since the appropriate type often\ndepends on context (e.g. Washington could be tagged either as city or\ngovernment), this procedure can result in spurious labels, leading to poorer\ngeneralization. We propose the task of context-dependent fine type tagging,\nwhere the set of acceptable labels for a mention is restricted to only those\ndeducible from the local context (e.g. sentence or document). We introduce new\nresources for this task: 12,017 mentions annotated with their context-dependent\nfine types, and we provide baseline experimental results on this data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2007v2",
            "date": "2014-12-05 14:26:27+00:00",
            "title": "On Using Very Large Target Vocabulary for Neural Machine Translation",
            "authors": [
                "S\u00e9bastien Jean",
                "Kyunghyun Cho",
                "Roland Memisevic",
                "Yoshua Bengio"
            ],
            "abstract": "Neural machine translation, a recently proposed approach to machine\ntranslation based purely on neural networks, has shown promising results\ncompared to the existing approaches such as phrase-based statistical machine\ntranslation. Despite its recent success, neural machine translation has its\nlimitation in handling a larger vocabulary, as training complexity as well as\ndecoding complexity increase proportionally to the number of target words. In\nthis paper, we propose a method that allows us to use a very large target\nvocabulary without increasing training complexity, based on importance\nsampling. We show that decoding can be efficiently done even with the model\nhaving a very large target vocabulary by selecting only a small subset of the\nwhole target vocabulary. The models trained by the proposed approach are\nempirically found to outperform the baseline models with a small vocabulary as\nwell as the LSTM-based neural machine translation models. Furthermore, when we\nuse the ensemble of a few models with very large target vocabularies, we\nachieve the state-of-the-art translation performance (measured by BLEU) on the\nEnglish->German translation and almost as high performance as state-of-the-art\nEnglish->French translation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2197v3",
            "date": "2014-12-06 04:40:18+00:00",
            "title": "Practice in Synonym Extraction at Large Scale",
            "authors": [
                "Liangliang Cao",
                "Chang Wang"
            ],
            "abstract": "Synonym extraction is an important task in natural language processing and\noften used as a submodule in query expansion, question answering and other\napplications. Automatic synonym extractor is highly preferred for large scale\napplications. Previous studies in synonym extraction are most limited to small\nscale datasets. In this paper, we build a large dataset with 3.4 million\nsynonym/non-synonym pairs to capture the challenges in real world scenarios. We\nproposed (1) a new cost function to accommodate the unbalanced learning\nproblem, and (2) a feature learning based deep neural network to model the\ncomplicated relationships in synonym pairs. We compare several different\napproaches based on SVMs and neural networks, and find out a novel feature\nlearning based neural network outperforms the methods with hand-assigned\nfeatures. Specifically, the best performance of our model surpasses the SVM\nbaseline with a significant 97\\% relative improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2378v1",
            "date": "2014-12-07 17:49:53+00:00",
            "title": "Learning Word Representations from Relational Graphs",
            "authors": [
                "Danushka Bollegala",
                "Takanori Maehara",
                "Yuichi Yoshida",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Attributes of words and relations between two words are central to numerous\ntasks in Artificial Intelligence such as knowledge representation, similarity\nmeasurement, and analogy detection. Often when two words share one or more\nattributes in common, they are connected by some semantic relations. On the\nother hand, if there are numerous semantic relations between two words, we can\nexpect some of the attributes of one of the words to be inherited by the other.\nMotivated by this close connection between attributes and relations, given a\nrelational graph in which words are inter- connected via numerous semantic\nrelations, we propose a method to learn a latent representation for the\nindividual words. The proposed method considers not only the co-occurrences of\nwords as done by existing approaches for word representation learning, but also\nthe semantic relations in which two words co-occur. To evaluate the accuracy of\nthe word representations learnt using the proposed method, we use the learnt\nword representations to solve semantic word analogy problems. Our experimental\nresults show that it is possible to learn better word representations by using\nsemantic semantics between words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2442v1",
            "date": "2014-12-08 04:14:05+00:00",
            "title": "Rediscovering the Alphabet - On the Innate Universal Grammar",
            "authors": [
                "M. Yahia Kaadan",
                "Asaad Kaadan"
            ],
            "abstract": "Universal Grammar (UG) theory has been one of the most important research\ntopics in linguistics since introduced five decades ago. UG specifies the\nrestricted set of languages learnable by human brain, and thus, many\nresearchers believe in its biological roots. Numerous empirical studies of\nneurobiological and cognitive functions of the human brain, and of many natural\nlanguages, have been conducted to unveil some aspects of UG. This, however,\nresulted in different and sometimes contradicting theories that do not indicate\na universally unique grammar. In this research, we tackle the UG problem from\nan entirely different perspective. We search for the Unique Universal Grammar\n(UUG) that facilitates communication and knowledge transfer, the sole purpose\nof a language. We formulate this UG and show that it is unique, intrinsic, and\ncosmic, rather than humanistic. Initial analysis on a widespread natural\nlanguage already showed some positive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.3336v2",
            "date": "2014-12-10 15:22:08+00:00",
            "title": "Statistical Patterns in Written Language",
            "authors": [
                "Dami\u00e1n H. Zanette"
            ],
            "abstract": "Quantitative linguistics has been allowed, in the last few decades, within\nthe admittedly blurry boundaries of the field of complex systems. A growing\nhost of applied mathematicians and statistical physicists devote their efforts\nto disclose regularities, correlations, patterns, and structural properties of\nlanguage streams, using techniques borrowed from statistics and information\ntheory. Overall, results can still be categorized as modest, but the prospects\nare promising: medium- and long-range features in the organization of human\nlanguage -which are beyond the scope of traditional linguistics- have already\nemerged from this kind of analysis and continue to be reported, contributing a\nnew perspective to our understanding of this most complex communication system.\nThis short book is intended to review some of these recent contributions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4021v5",
            "date": "2014-12-12 15:26:43+00:00",
            "title": "A Robust Transformation-Based Learning Approach Using Ripple Down Rules for Part-of-Speech Tagging",
            "authors": [
                "Dat Quoc Nguyen",
                "Dai Quoc Nguyen",
                "Dang Duc Pham",
                "Son Bao Pham"
            ],
            "abstract": "In this paper, we propose a new approach to construct a system of\ntransformation rules for the Part-of-Speech (POS) tagging task. Our approach is\nbased on an incremental knowledge acquisition method where rules are stored in\nan exception structure and new rules are only added to correct the errors of\nexisting rules; thus allowing systematic control of the interaction between the\nrules. Experimental results on 13 languages show that our approach is fast in\nterms of training time and tagging speed. Furthermore, our approach obtains\nvery competitive accuracy in comparison to state-of-the-art POS and\nmorphological taggers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4369v3",
            "date": "2014-12-14 15:18:18+00:00",
            "title": "Incorporating Both Distributional and Relational Semantics in Word Representations",
            "authors": [
                "Daniel Fried",
                "Kevin Duh"
            ],
            "abstract": "We investigate the hypothesis that word representations ought to incorporate\nboth distributional and relational semantics. To this end, we employ the\nAlternating Direction Method of Multipliers (ADMM), which flexibly optimizes a\ndistributional objective on raw text and a relational objective on WordNet.\nPreliminary results on knowledge base completion, analogy tests, and parsing\nshow that word representations trained on both objectives can give improvements\nin some cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4682v1",
            "date": "2014-12-15 17:20:47+00:00",
            "title": "Rule-based Emotion Detection on Social Media: Putting Tweets on Plutchik's Wheel",
            "authors": [
                "Erik Tromp",
                "Mykola Pechenizkiy"
            ],
            "abstract": "We study sentiment analysis beyond the typical granularity of polarity and\ninstead use Plutchik's wheel of emotions model. We introduce RBEM-Emo as an\nextension to the Rule-Based Emission Model algorithm to deduce such emotions\nfrom human-written messages. We evaluate our approach on two different datasets\nand compare its performance with the current state-of-the-art techniques for\nemotion detection, including a recursive auto-encoder. The results of the\nexperimental study suggest that RBEM-Emo is a promising approach advancing the\ncurrent state-of-the-art in emotion detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4930v2",
            "date": "2014-12-16 09:43:56+00:00",
            "title": "Rehabilitation of Count-based Models for Word Vector Representations",
            "authors": [
                "R\u00e9mi Lebret",
                "Ronan Collobert"
            ],
            "abstract": "Recent works on word representations mostly rely on predictive models.\nDistributed word representations (aka word embeddings) are trained to optimally\npredict the contexts in which the corresponding words tend to appear. Such\nmodels have succeeded in capturing word similarties as well as semantic and\nsyntactic regularities. Instead, we aim at reviving interest in a model based\non counts. We present a systematic study of the use of the Hellinger distance\nto extract semantic representations from the word co-occurence statistics of\nlarge text corpora. We show that this distance gives good performance on word\nsimilarity and analogy tasks, with a proper type and size of context, and a\ndimensionality reduction based on a stochastic low-rank approximation. Besides\nbeing both simple and intuitive, this method also provides an encoding function\nwhich can be used to infer unseen words or phrases. This becomes a clear\nadvantage compared to predictive models which must train these new words.",
            "journal ref": "CICLing 2015, Part I, LNCS 9041, pp. 417-429, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5212v1",
            "date": "2014-12-16 22:00:52+00:00",
            "title": "Application of Topic Models to Judgments from Public Procurement Domain",
            "authors": [
                "Micha\u0142 \u0141opuszy\u0144ski"
            ],
            "abstract": "In this work, automatic analysis of themes contained in a large corpora of\njudgments from public procurement domain is performed. The employed technique\nis unsupervised latent Dirichlet allocation (LDA). In addition, it is proposed,\nto use LDA in conjunction with recently developed method of unsupervised\nkeyword extraction. Such an approach improves the interpretability of the\nautomatically obtained topics and allows for better computational performance.\nThe described analysis illustrates a potential of the method in detecting\nrecurring themes and discovering temporal trends in lodged contract appeals.\nThese results may be in future applied to improve information retrieval from\nrepositories of legal texts or as auxiliary material for legal analyses carried\nout by human experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5477v1",
            "date": "2014-12-17 16:56:43+00:00",
            "title": "Computational Model to Generate Case-Inflected Forms of Masculine Nouns for Word Search in Sanskrit E-Text",
            "authors": [
                "S V Kasmir Raja",
                "V Rajitha",
                "Lakshmanan Meenakshi"
            ],
            "abstract": "The problem of word search in Sanskrit is inseparable from complexities that\ninclude those caused by euphonic conjunctions and case-inflections. The\ncase-inflectional forms of a noun normally number 24 owing to the fact that in\nSanskrit there are eight cases and three numbers-singular, dual and plural. The\ntraditional method of generating these inflectional forms is rather elaborate\nowing to the fact that there are differences in the forms generated between\neven very similar words and there are subtle nuances involved. Further, it\nwould be a cumbersome exercise to generate and search for 24 forms of a word\nduring a word search in a large text, using the currently available\ncase-inflectional form generators. This study presents a new approach to\ngenerating case-inflectional forms that is simpler to compute. Further, an\noptimized model that is sufficient for generating only those word forms that\nare required in a word search and is more than 80% efficient compared to the\ncomplete case-inflectional forms generator, is presented in this study for the\nfirst time.",
            "journal ref": "Journal of Computer Science (ISSN Print: 1549-3636, ISSN Online:\n  1552-6607), December 2014, Volume 10, Issue 11, Pages 2269-2283",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5836v3",
            "date": "2014-12-18 12:30:55+00:00",
            "title": "Incorporating Both Distributional and Relational Semantics in Word Representations",
            "authors": [
                "Daniel Fried",
                "Kevin Duh"
            ],
            "abstract": "We investigate the hypothesis that word representations ought to incorporate\nboth distributional and relational semantics. To this end, we employ the\nAlternating Direction Method of Multipliers (ADMM), which flexibly optimizes a\ndistributional objective on raw text and a relational objective on WordNet.\nPreliminary results on knowledge base completion, analogy tests, and parsing\nshow that word representations trained on both objectives can give improvements\nin some cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6045v2",
            "date": "2014-12-18 20:14:10+00:00",
            "title": "A Simple and Efficient Method To Generate Word Sense Representations",
            "authors": [
                "Luis Nieto Pi\u00f1a",
                "Richard Johansson"
            ],
            "abstract": "Distributed representations of words have boosted the performance of many\nNatural Language Processing tasks. However, usually only one representation per\nword is obtained, not acknowledging the fact that some words have multiple\nmeanings. This has a negative effect on the individual word representations and\nthe language model as a whole. In this paper we present a simple model that\nenables recent techniques for building word vectors to represent distinct\nsenses of polysemic words. In our assessment of this model we show that it is\nable to effectively discriminate between words' senses and to do so in a\ncomputationally efficient manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6264v1",
            "date": "2014-12-19 09:53:57+00:00",
            "title": "Supertagging: Introduction, learning, and application",
            "authors": [
                "Taraka Rama K"
            ],
            "abstract": "Supertagging is an approach originally developed by Bangalore and Joshi\n(1999) to improve the parsing efficiency. In the beginning, the scholars used\nsmall training datasets and somewhat na\\\"ive smoothing techniques to learn the\nprobability distributions of supertags. Since its inception, the applicability\nof Supertags has been explored for TAG (tree-adjoining grammar) formalism as\nwell as other related yet, different formalisms such as CCG. This article will\ntry to summarize the various chapters, relevant to statistical parsing, from\nthe most recent edited book volume (Bangalore and Joshi, 2010). The chapters\nwere selected so as to blend the learning of supertags, its integration into\nfull-scale parsing, and in semantic parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6277v2",
            "date": "2014-12-19 10:29:33+00:00",
            "title": "N-gram-Based Low-Dimensional Representation for Document Classification",
            "authors": [
                "R\u00e9mi Lebret",
                "Ronan Collobert"
            ],
            "abstract": "The bag-of-words (BOW) model is the common approach for classifying\ndocuments, where words are used as feature for training a classifier. This\ngenerally involves a huge number of features. Some techniques, such as Latent\nSemantic Analysis (LSA) or Latent Dirichlet Allocation (LDA), have been\ndesigned to summarize documents in a lower dimension with the least semantic\ninformation loss. Some semantic information is nevertheless always lost, since\nonly words are considered. Instead, we aim at using information coming from\nn-grams to overcome this limitation, while remaining in a low-dimension space.\nMany approaches, such as the Skip-gram model, provide good word vector\nrepresentations very quickly. We propose to average these representations to\nobtain representations of n-grams. All n-grams are thus embedded in a same\nsemantic space. A K-means clustering can then group them into semantic\nconcepts. The number of features is therefore dramatically reduced and\ndocuments can be represented as bag of semantic concepts. We show that this\nmodel outperforms LSA and LDA on a sentiment classification task, and yields\nsimilar results than a traditional BOW-model with far less features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6334v4",
            "date": "2014-12-19 13:23:35+00:00",
            "title": "Leveraging Monolingual Data for Crosslingual Compositional Word Representations",
            "authors": [
                "Hubert Soyer",
                "Pontus Stenetorp",
                "Akiko Aizawa"
            ],
            "abstract": "In this work, we present a novel neural network based architecture for\ninducing compositional crosslingual word representations. Unlike previously\nproposed methods, our method fulfills the following three criteria; it\nconstrains the word-level representations to be compositional, it is capable of\nleveraging both bilingual and monolingual data, and it is scalable to large\nvocabularies and large quantities of data. The key component of our approach is\nwhat we refer to as a monolingual inclusion criterion, that exploits the\nobservation that phrases are more closely semantically related to their\nsub-phrases than to other randomly sampled phrases. We evaluate our method on a\nwell-established crosslingual document classification task and achieve results\nthat are either comparable, or greatly improve upon previous state-of-the-art\nmethods. Concretely, our method reaches a level of 92.7% and 84.4% accuracy for\nthe English to German and German to English sub-tasks respectively. The former\nadvances the state of the art by 0.9% points of accuracy, the latter is an\nabsolute improvement upon the previous state of the art by 7.7% points of\naccuracy and an improvement of 33.0% in error reduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6448v4",
            "date": "2014-12-19 17:22:03+00:00",
            "title": "Embedding Word Similarity with Neural Machine Translation",
            "authors": [
                "Felix Hill",
                "Kyunghyun Cho",
                "Sebastien Jean",
                "Coline Devin",
                "Yoshua Bengio"
            ],
            "abstract": "Neural language models learn word representations, or embeddings, that\ncapture rich linguistic and conceptual information. Here we investigate the\nembeddings learned by neural machine translation models, a recently-developed\nclass of neural language model. We show that embeddings from translation models\noutperform those learned by monolingual models at tasks that require knowledge\nof both conceptual similarity and lexical-syntactic role. We further show that\nthese effects hold when translating from both English to French and English to\nGerman, and argue that the desirable properties of translation embeddings\nshould emerge largely independently of the source and target languages.\nFinally, we apply a new method for training neural translation models with very\nlarge vocabularies, and show that this vocabulary expansion algorithm results\nin minimal degradation of embedding quality. Our embedding spaces can be\nqueried in an online demo and downloaded from our web page. Overall, our\nanalyses indicate that translation-based embeddings should be used in\napplications that require concepts to be organised according to similarity\nand/or lexical function, while monolingual embeddings are better suited to\nmodelling (nonspecific) inter-word relatedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6575v4",
            "date": "2014-12-20 01:37:16+00:00",
            "title": "Embedding Entities and Relations for Learning and Inference in Knowledge Bases",
            "authors": [
                "Bishan Yang",
                "Wen-tau Yih",
                "Xiaodong He",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "We consider learning representations of entities and relations in KBs using\nthe neural-embedding approach. We show that most existing models, including NTN\n(Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized\nunder a unified learning framework, where entities are low-dimensional vectors\nlearned from a neural network and relations are bilinear and/or linear mapping\nfunctions. Under this framework, we compare a variety of embedding models on\nthe link prediction task. We show that a simple bilinear formulation achieves\nnew state-of-the-art results for the task (achieving a top-10 accuracy of 73.2%\nvs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach\nthat utilizes the learned relation embeddings to mine logical rules such as\n\"BornInCity(a,b) and CityInCountry(b,c) => Nationality(a,c)\". We find that\nembeddings learned from the bilinear objective are particularly good at\ncapturing relational semantics and that the composition of relations is\ncharacterized by matrix multiplication. More interestingly, we demonstrate that\nour embedding-based rule extraction approach successfully outperforms a\nstate-of-the-art confidence-based rule mining approach in mining Horn rules\nthat involve compositional reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7119v3",
            "date": "2014-12-22 20:08:06+00:00",
            "title": "Pragmatic Neural Language Modelling in Machine Translation",
            "authors": [
                "Paul Baltescu",
                "Phil Blunsom"
            ],
            "abstract": "This paper presents an in-depth investigation on integrating neural language\nmodels in translation systems. Scaling neural language models is a difficult\ntask, but crucial for real-world applications. This paper evaluates the impact\non end-to-end MT quality of both new and existing scaling techniques. We show\nwhen explicitly normalising neural models is necessary and what optimisation\ntricks one should use in such scenarios. We also focus on scalable training\nalgorithms and investigate noise contrastive estimation and diagonal contexts\nas sources for further speed improvements. We explore the trade-offs between\nneural models and back-off n-gram models and find that neural models make\nstrong candidates for natural language applications in memory constrained\nenvironments, yet still lag behind traditional models in raw translation\nquality. We conclude with a set of recommendations one should follow to build a\nscalable neural language model for MT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7415v2",
            "date": "2014-12-23 15:51:41+00:00",
            "title": "A prototype Malayalam to Sign Language Automatic Translator",
            "authors": [
                "Jestin Joy",
                "Kannan Balakrishnan"
            ],
            "abstract": "Sign language, which is a medium of communication for deaf people, uses\nmanual communication and body language to convey meaning, as opposed to using\nsound. This paper presents a prototype Malayalam text to sign language\ntranslation system. The proposed system takes Malayalam text as input and\ngenerates corresponding Sign Language. Output animation is rendered using a\ncomputer generated model. This system will help to disseminate information to\nthe deaf people in public utility places like railways, banks, hospitals etc.\nThis will also act as an educational tool in learning Sign Language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.8010v1",
            "date": "2014-12-27 01:54:15+00:00",
            "title": "Construction of Vietnamese SentiWordNet by using Vietnamese Dictionary",
            "authors": [
                "Xuan-Son Vu",
                "Seong-Bae Park"
            ],
            "abstract": "SentiWordNet is an important lexical resource supporting sentiment analysis\nin opinion mining applications. In this paper, we propose a novel approach to\nconstruct a Vietnamese SentiWordNet (VSWN). SentiWordNet is typically generated\nfrom WordNet in which each synset has numerical scores to indicate its opinion\npolarities. Many previous studies obtained these scores by applying a machine\nlearning method to WordNet. However, Vietnamese WordNet is not available\nunfortunately by the time of this paper. Therefore, we propose a method to\nconstruct VSWN from a Vietnamese dictionary, not from WordNet. We show the\neffectiveness of the proposed method by generating a VSWN with 39,561 synsets\nautomatically. The method is experimentally tested with 266 synsets with aspect\nof positivity and negativity. It attains a competitive result compared with\nEnglish SentiWordNet that is 0.066 and 0.052 differences for positivity and\nnegativity sets respectively.",
            "journal ref": "The 40th Conference of the Korea Information Processing Society,\n  pp. 745-748, April 2014, South Korea",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.0563v1",
            "date": "2014-02-04 01:34:56+00:00",
            "title": "Evaluating Indirect Strategies for Chinese-Spanish Statistical Machine Translation",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Carlos A. Henr\u00edquez",
                "Rafael E. Banchs"
            ],
            "abstract": "Although, Chinese and Spanish are two of the most spoken languages in the\nworld, not much research has been done in machine translation for this language\npair. This paper focuses on investigating the state-of-the-art of\nChinese-to-Spanish statistical machine translation (SMT), which nowadays is one\nof the most popular approaches to machine translation. For this purpose, we\nreport details of the available parallel corpus which are Basic Traveller\nExpressions Corpus (BTEC), Holy Bible and United Nations (UN). Additionally, we\nconduct experimental work with the largest of these three corpora to explore\nalternative SMT strategies by means of using a pivot language. Three\nalternatives are considered for pivoting: cascading, pseudo-corpus and\ntriangulation. As pivot language, we use either English, Arabic or French.\nResults show that, for a phrase-based SMT system, English is the best pivot\nlanguage between Chinese and Spanish. We propose a system output combination\nusing the pivot strategies which is capable of outperforming the direct\ntranslation strategy. The main objective of this work is motivating and\ninvolving the research community to work in this important pair of languages\ngiven their demographic impact.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 45, pages\n  761-780, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.0578v1",
            "date": "2014-02-04 01:40:42+00:00",
            "title": "Natural Language Inference for Arabic Using Extended Tree Edit Distance with Subtrees",
            "authors": [
                "Maytham Alabbas",
                "Allan Ramsay"
            ],
            "abstract": "Many natural language processing (NLP) applications require the computation\nof similarities between pairs of syntactic or semantic trees. Many researchers\nhave used tree edit distance for this task, but this technique suffers from the\ndrawback that it deals with single node operations only. We have extended the\nstandard tree edit distance algorithm to deal with subtree transformation\noperations as well as single nodes. The extended algorithm with subtree\noperations, TED+ST, is more effective and flexible than the standard algorithm,\nespecially for applications that pay attention to relations among nodes (e.g.\nin linguistic trees, deleting a modifier subtree should be cheaper than the sum\nof deleting its components individually). We describe the use of TED+ST for\nchecking entailment between two Arabic text snippets. The preliminary results\nof using TED+ST were encouraging when compared with two string-based approaches\nand with the standard algorithm.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 48, pages\n  1-22, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.0586v1",
            "date": "2014-02-04 01:43:35+00:00",
            "title": "Topic Segmentation and Labeling in Asynchronous Conversations",
            "authors": [
                "Shafiq Rayhan Joty",
                "Giuseppe Carenini",
                "Raymond T Ng"
            ],
            "abstract": "Topic segmentation and labeling is often considered a prerequisite for\nhigher-level conversation analysis and has been shown to be useful in many\nNatural Language Processing (NLP) applications. We present two new corpora of\nemail and blog conversations annotated with topics, and evaluate annotator\nreliability for the segmentation and labeling tasks in these asynchronous\nconversations. We propose a complete computational framework for topic\nsegmentation and labeling in asynchronous conversations. Our approach extends\nstate-of-the-art methods by considering a fine-grained structure of an\nasynchronous conversation, along with other conversational features by applying\nrecent graph-based methods for NLP. For topic segmentation, we propose two\nnovel unsupervised models that exploit the fine-grained conversational\nstructure, and a novel graph-theoretic supervised model that combines lexical,\nconversational and topic features. For topic labeling, we propose two novel\n(unsupervised) random walk models that respectively capture conversation\nspecific clues from two different sources: the leading sentences and the\nfine-grained conversational structure. Empirical evaluation shows that the\nsegmentation and the labeling performed by our best models beat the\nstate-of-the-art, and are highly correlated with human annotations.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 47, pages\n  521-573, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.2561v1",
            "date": "2014-01-18 21:08:40+00:00",
            "title": "The CQC Algorithm: Cycling in Graphs to Semantically Enrich and Enhance a Bilingual Dictionary",
            "authors": [
                "Tiziano Flati",
                "Roberto Navigli"
            ],
            "abstract": "Bilingual machine-readable dictionaries are knowledge resources useful in\nmany automatic tasks. However, compared to monolingual computational lexicons\nlike WordNet, bilingual dictionaries typically provide a lower amount of\nstructured information, such as lexical and semantic relations, and often do\nnot cover the entire range of possible translations for a word of interest. In\nthis paper we present Cycles and Quasi-Cycles (CQC), a novel algorithm for the\nautomated disambiguation of ambiguous translations in the lexical entries of a\nbilingual machine-readable dictionary. The dictionary is represented as a\ngraph, and cyclic patterns are sought in the graph to assign an appropriate\nsense tag to each translation in a lexical entry. Further, we use the\nalgorithms output to improve the quality of the dictionary itself, by\nsuggesting accurate solutions to structural problems such as misalignments,\npartial alignments and missing entries. Finally, we successfully apply CQC to\nthe task of synonym extraction.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 43, pages\n  135-171, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.2796v1",
            "date": "2014-02-12 11:55:31+00:00",
            "title": "PR2: A Language Independent Unsupervised Tool for Personality Recognition from Text",
            "authors": [
                "Fabio Celli",
                "Massimo Poesio"
            ],
            "abstract": "We present PR2, a personality recognition system available online, that\nperforms instance-based classification of Big5 personality types from\nunstructured text, using language-independent features. It has been tested on\nEnglish and Italian, achieving performances up to f=.68.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3040v1",
            "date": "2014-02-13 06:44:24+00:00",
            "title": "Event Structure of Transitive Verb: A MARVS perspective",
            "authors": [
                "Jia-Fei Hong",
                "Kathleen Ahrens",
                "Chu-Ren Huang"
            ],
            "abstract": "Module-Attribute Representation of Verbal Semantics (MARVS) is a theory of\nthe representation of verbal semantics that is based on Mandarin Chinese data\n(Huang et al. 2000). In the MARVS theory, there are two different types of\nmodules: Event Structure Modules and Role Modules. There are also two sets of\nattributes: Event-Internal Attributes and Role-Internal Attributes, which are\nlinked to the Event Structure Module and the Role Module, respectively. In this\nstudy, we focus on four transitive verbs as chi1(eat), wan2(play),\nhuan4(change) and shao1(burn) and explore their event structures by the MARVS\ntheory.",
            "journal ref": "International Journal of Computer Processing of Languages (Vol.\n  24, No. 01, March 2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3371v1",
            "date": "2014-02-14 06:06:47+00:00",
            "title": "An evaluative baseline for geo-semantic relatedness and similarity",
            "authors": [
                "Andrea Ballatore",
                "Michela Bertolotto",
                "David C. Wilson"
            ],
            "abstract": "In geographic information science and semantics, the computation of semantic\nsimilarity is widely recognised as key to supporting a vast number of tasks in\ninformation integration and retrieval. By contrast, the role of geo-semantic\nrelatedness has been largely ignored. In natural language processing, semantic\nrelatedness is often confused with the more specific semantic similarity. In\nthis article, we discuss a notion of geo-semantic relatedness based on Lehrer's\nsemantic fields, and we compare it with geo-semantic similarity. We then\ndescribe and validate the Geo Relatedness and Similarity Dataset (GeReSiD), a\nnew open dataset designed to evaluate computational measures of geo-semantic\nrelatedness and similarity. This dataset is larger than existing datasets of\nthis kind, and includes 97 geographic terms combined into 50 term pairs rated\nby 203 human subjects. GeReSiD is available online and can be used as an\nevaluation baseline to determine empirically to what degree a given\ncomputational model approximates geo-semantic relatedness and similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3382v1",
            "date": "2014-02-14 06:46:44+00:00",
            "title": "Machine Learning of Phonologically Conditioned Noun Declensions For Tamil Morphological Generators",
            "authors": [
                "K. Rajan",
                "Dr. V. Ramalingam",
                "Dr. M. Ganesan"
            ],
            "abstract": "This paper presents machine learning solutions to a practical problem of\nNatural Language Generation (NLG), particularly the word formation in\nagglutinative languages like Tamil, in a supervised manner. The morphological\ngenerator is an important component of Natural Language Processing in\nArtificial Intelligence. It generates word forms given a root and affixes. The\nmorphophonemic changes like addition, deletion, alternation etc., occur when\ntwo or more morphemes or words joined together. The Sandhi rules should be\nexplicitly specified in the rule based morphological analyzers and generators.\nIn machine learning framework, these rules can be learned automatically by the\nsystem from the training samples and subsequently be applied for new inputs. In\nthis paper we proposed the machine learning models which learn the\nmorphophonemic rules for noun declensions from the given training data. These\nmodels are trained to learn sandhi rules using various learning algorithms and\nthe performance of those algorithms are presented. From this we conclude that\nmachine learning of morphological processing such as word form generation can\nbe successfully learned in a supervised manner, without explicit description of\nrules. The performance of Decision trees and Bayesian machine learning\nalgorithms on noun declensions are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.4380v1",
            "date": "2014-02-18 16:02:05+00:00",
            "title": "A Comparative Study of Machine Learning Methods for Verbal Autopsy Text Classification",
            "authors": [
                "Samuel Danso",
                "Eric Atwell",
                "Owen Johnson"
            ],
            "abstract": "A Verbal Autopsy is the record of an interview about the circumstances of an\nuncertified death. In developing countries, if a death occurs away from health\nfacilities, a field-worker interviews a relative of the deceased about the\ncircumstances of the death; this Verbal Autopsy can be reviewed off-site. We\nreport on a comparative study of the processes involved in Text Classification\napplied to classifying Cause of Death: feature value representation; machine\nlearning classification algorithms; and feature reduction strategies in order\nto identify the suitable approaches applicable to the classification of Verbal\nAutopsy text. We demonstrate that normalised term frequency and the standard\nTFiDF achieve comparable performance across a number of classifiers. The\nresults also show Support Vector Machine is superior to other classification\nalgorithms employed in this research. Finally, we demonstrate the effectiveness\nof employing a \"locally-semi-supervised\" feature reduction strategy in order to\nincrease performance accuracy.",
            "journal ref": "International Journal of Computer Science Issues, Volume 10, Issue\n  6, No 2, November 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.4678v1",
            "date": "2014-02-18 02:18:10+00:00",
            "title": "When Learners Surpass their Sources: Mathematical Modeling of Learning from an Inconsistent Source",
            "authors": [
                "Yelena Mandelshtam",
                "Natalia Komarova"
            ],
            "abstract": "We present a new algorithm to model and investigate the learning process of a\nlearner mastering a set of grammatical rules from an inconsistent source. The\ncompelling interest of human language acquisition is that the learning succeeds\nin virtually every case, despite the fact that the input data are formally\ninadequate to explain the success of learning. Our model explains how a learner\ncan successfully learn from or even surpass its imperfect source without\npossessing any additional biases or constraints about the types of patterns\nthat exist in the language. We use the data collected by Singleton and Newport\n(2004) on the performance of a 7-year boy Simon, who mastered the American Sign\nLanguage (ASL) by learning it from his parents, both of whom were imperfect\nspeakers of ASL. We show that the algorithm possesses a frequency-boosting\nproperty, whereby the frequency of the most common form of the source is\nincreased by the learner. We also explain several key features of Simon's ASL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.6516v1",
            "date": "2014-02-26 12:37:04+00:00",
            "title": "Modelling the Lexicon in Unsupervised Part of Speech Induction",
            "authors": [
                "Greg Dubbin",
                "Phil Blunsom"
            ],
            "abstract": "Automatically inducing the syntactic part-of-speech categories for words in\ntext is a fundamental task in Computational Linguistics. While the performance\nof unsupervised tagging models has been slowly improving, current\nstate-of-the-art systems make the obviously incorrect assumption that all\ntokens of a given word type must share a single part-of-speech tag. This\none-tag-per-type heuristic counters the tendency of Hidden Markov Model based\ntaggers to over generate tags for a given word type. However, it is clearly\nincompatible with basic syntactic theory. In this paper we extend a\nstate-of-the-art Pitman-Yor Hidden Markov Model tagger with an explicit model\nof the lexicon. In doing so we are able to incorporate a soft bias towards\ninducing few tags per type. We develop a particle filter for drawing samples\nfrom the posterior of our model and present empirical results that show that\nour model is competitive with and faster than the state-of-the-art without\nmaking any unrealistic restrictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.6880v1",
            "date": "2014-02-27 12:12:11+00:00",
            "title": "It's distributions all the way down!: Second order changes in statistical distributions also occur",
            "authors": [
                "M. T. Keane",
                "A. Gerow"
            ],
            "abstract": "The textual, big-data literature misses Bentley, OBrien, & Brocks (Bentley et\nals) message on distributions; it largely examines the first-order effects of\nhow a single, signature distribution can predict population behaviour,\nneglecting second-order effects involving distributional shifts, either between\nsignature distributions or within a given signature distribution. Indeed,\nBentley et al. themselves under-emphasise the potential richness of the latter,\nwithin-distribution effects.",
            "journal ref": "Behavioral & Brain Sciences, 2014, 37(1), 87",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.7265v1",
            "date": "2014-02-28 14:49:31+00:00",
            "title": "Semantics, Modelling, and the Problem of Representation of Meaning -- a Brief Survey of Recent Literature",
            "authors": [
                "Yarin Gal"
            ],
            "abstract": "Over the past 50 years many have debated what representation should be used\nto capture the meaning of natural language utterances. Recently new needs of\nsuch representations have been raised in research. Here I survey some of the\ninteresting representations suggested to answer for these new needs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.0052v1",
            "date": "2014-03-01 06:46:11+00:00",
            "title": "TBX goes TEI -- Implementing a TBX basic extension for the Text Encoding Initiative guidelines",
            "authors": [
                "Laurent Romary"
            ],
            "abstract": "This paper presents an attempt to customise the TEI (Text Encoding\nInitiative) guidelines in order to offer the possibility to incorporate TBX\n(TermBase eXchange) based terminological entries within any kind of TEI\ndocuments. After presenting the general historical, conceptual and technical\ncontexts, we describe the various design choices we had to take while creating\nthis customisation, which in turn have led to make various changes in the\nactual TBX serialisation. Keeping in mind the objective to provide the TEI\nguidelines with, again, an onomasiological model, we try to identify the best\ncomprise in maintaining both the isomorphism with the existing TBX Basic\nstandard and the characteristics of the TEI framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.0531v1",
            "date": "2014-03-03 19:27:23+00:00",
            "title": "We Tweet Like We Talk and Other Interesting Observations: An Analysis of English Communication Modalities",
            "authors": [
                "Josiah P. Zayner"
            ],
            "abstract": "Modalities of communication for human beings are gradually increasing in\nnumber with the advent of new forms of technology. Many human beings can\nreadily transition between these different forms of communication with little\nor no effort, which brings about the question: How similar are these different\ncommunication modalities? To understand technology$\\text{'}$s influence on\nEnglish communication, four different corpora were analyzed and compared:\nWriting from Books using the 1-grams database from the Google Books project,\nTwitter, IRC Chat, and transcribed Talking. Multi-word confusion matrices\nrevealed that Talking has the most similarity when compared to the other modes\nof communication, while 1-grams were the least similar form of communication\nanalyzed. Based on the analysis of word usage, word usage frequency\ndistributions, and word class usage, among other things, Talking is also the\nmost similar to Twitter and IRC Chat. This suggests that communicating using\nTwitter and IRC Chat evolved from Talking rather than Writing. When we\ncommunicate online, even though we are writing, we do not Tweet or Chat how we\nwrite books; we Tweet and Chat how we Speak. Nonfiction and Fiction writing\nwere clearly differentiable from our analysis with Twitter and Chat being much\nmore similar to Fiction than Nonfiction writing. These hypotheses were then\ntested using author and journalists Cory Doctorow. Mr. Doctorow$\\text{'}$s\nWriting, Twitter usage, and Talking were all found to have very similar\nvocabulary usage patterns as the amalgamized populations, as long as the\nwriting was Fiction. However, Mr. Doctorow$\\text{'}$s Nonfiction writing is\ndifferent from 1-grams and other collected Nonfiction writings. This data could\nperhaps be used to create more entertaining works of Nonfiction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.0801v2",
            "date": "2014-03-04 14:45:56+00:00",
            "title": "Is getting the right answer just about choosing the right words? The role of syntactically-informed features in short answer scoring",
            "authors": [
                "Derrick Higgins",
                "Chris Brew",
                "Michael Heilman",
                "Ramon Ziai",
                "Lei Chen",
                "Aoife Cahill",
                "Michael Flor",
                "Nitin Madnani",
                "Joel Tetreault",
                "Daniel Blanchard",
                "Diane Napolitano",
                "Chong Min Lee",
                "John Blackmore"
            ],
            "abstract": "Developments in the educational landscape have spurred greater interest in\nthe problem of automatically scoring short answer questions. A recent shared\ntask on this topic revealed a fundamental divide in the modeling approaches\nthat have been applied to this problem, with the best-performing systems split\nbetween those that employ a knowledge engineering approach and those that\nalmost solely leverage lexical information (as opposed to higher-level\nsyntactic information) in assigning a score to a given response. This paper\naims to introduce the NLP community to the largest corpus currently available\nfor short-answer scoring, provide an overview of methods used in the shared\ntask using this data, and explore the extent to which more\nsyntactically-informed features can contribute to the short answer scoring task\nin a way that avoids the question-specific manual effort of the knowledge\nengineering approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.2004v1",
            "date": "2014-03-08 20:10:37+00:00",
            "title": "Natural Language Feature Selection via Cooccurrence",
            "authors": [
                "Michael Stewart"
            ],
            "abstract": "Specificity is important for extracting collocations, keyphrases, multi-word\nand index terms [Newman et al. 2012]. It is also useful for tagging, ontology\nconstruction [Ryu and Choi 2006], and automatic summarization of documents\n[Louis and Nenkova 2011, Chali and Hassan 2012]. Term frequency and\ninverse-document frequency (TF-IDF) are typically used to do this, but fail to\ntake advantage of the semantic relationships between terms [Church and Gale\n1995]. The result is that general idiomatic terms are mistaken for specific\nterms. We demonstrate use of relational data for estimation of term\nspecificity. The specificity of a term can be learned from its distribution of\nrelations with other terms. This technique is useful for identifying relevant\nwords or terms for other natural language processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.2124v1",
            "date": "2014-03-10 01:41:09+00:00",
            "title": "Generating Music from Literature",
            "authors": [
                "Hannah Davis",
                "Saif M. Mohammad"
            ],
            "abstract": "We present a system, TransProse, that automatically generates musical pieces\nfrom text. TransProse uses known relations between elements of music such as\ntempo and scale, and the emotions they evoke. Further, it uses a novel\nmechanism to determine sequences of notes that capture the emotional activity\nin the text. The work has applications in information visualization, in\ncreating audio-visual e-books, and in developing music apps.",
            "journal ref": "In Proceedings of the EACL Workshop on Computational Linguistics\n  for Literature, April 2014, Gothenburg, Sweden",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.2837v1",
            "date": "2014-03-12 08:08:49+00:00",
            "title": "HPS: a hierarchical Persian stemming method",
            "authors": [
                "Ayshe Rashidi",
                "Mina Zolfy Lighvan"
            ],
            "abstract": "In this paper, a novel hierarchical Persian stemming approach based on the\nPart-Of-Speech of the word in a sentence is presented. The implemented stemmer\nincludes hash tables and several deterministic finite automata in its different\nlevels of hierarchy for removing the prefixes and suffixes of the words. We had\ntwo intentions in using hash tables in our method. The first one is that the\nDFA don't support some special words, so hash table can partly solve the\naddressed problem. the second goal is to speed up the implemented stemmer with\nomitting the time that deterministic finite automata need. Because of the\nhierarchical organization, this method is fast and flexible enough. Our\nexperiments on test sets from Hamshahri collection and security news (istna.ir)\nshow that our method has the average accuracy of 95.37% which is even improved\nin using the method on a test set with common topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.3351v1",
            "date": "2014-03-13 18:20:10+00:00",
            "title": "Semantic Unification A sheaf theoretic approach to natural language",
            "authors": [
                "Samson Abramsky",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Language is contextual and sheaf theory provides a high level mathematical\nframework to model contextuality. We show how sheaf theory can model the\ncontextual nature of natural language and how gluing can be used to provide a\nglobal semantics for a discourse by putting together the local logical\nsemantics of each sentence within the discourse. We introduce a presheaf\nstructure corresponding to a basic form of Discourse Representation Structures.\nWithin this setting, we formulate a notion of semantic unification --- gluing\nmeanings of parts of a discourse into a coherent whole --- as a form of\nsheaf-theoretic gluing. We illustrate this idea with a number of examples where\nit can used to represent resolutions of anaphoric references. We also discuss\nmultivalued gluing, described using a distributions functor, which can be used\nto represent situations where multiple gluings are possible, and where we may\nneed to rank them using quantitative measures.\n  Dedicated to Jim Lambek on the occasion of his 90th birthday.",
            "journal ref": "Categories and Types in Logic, Language, and Physics, A Festshrift\n  for Jim Lambek. Casadio, Coecke, Moortgat, Scott (eds.), Lecture Notes in\n  Computer Science, Volume 8222, pp. 1-13, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.3668v2",
            "date": "2014-03-14 18:53:51+00:00",
            "title": "Language Heedless of Logic - Philosophy Mindful of What? Failures of Distributive and Absorption Laws",
            "authors": [
                "Arthur Merin"
            ],
            "abstract": "Much of philosophical logic and all of philosophy of language make empirical\nclaims about the vernacular natural language. They presume semantics under\nwhich `and' and `or' are related by the dually paired distributive and\nabsorption laws. However, at least one of each pair of laws fails in the\nvernacular. `Implicature'-based auxiliary theories associated with the\nprogramme of H.P. Grice do not prove remedial. Conceivable alternatives that\nmight replace the familiar logics as descriptive instruments are briefly noted:\n(i) substructural logics and (ii) meaning composition in linear algebras over\nthe reals, occasionally constrained by norms of classical logic. Alternative\n(ii) locates the problem in violations of one of the idempotent laws. Reasons\nfor a lack of curiosity about elementary and easily testable implications of\nthe received theory are considered. The concept of `reflective equilibrium' is\ncritically examined for its role in reconciling normative desiderata and\ndescriptive commitments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4024v3",
            "date": "2014-03-17 08:22:54+00:00",
            "title": "Measuring Global Similarity between Texts",
            "authors": [
                "Uli Fahrenberg",
                "Fabrizio Biondi",
                "Kevin Corre",
                "Cyrille Jegourel",
                "Simon Kongsh\u00f8j",
                "Axel Legay"
            ],
            "abstract": "We propose a new similarity measure between texts which, contrary to the\ncurrent state-of-the-art approaches, takes a global view of the texts to be\ncompared. We have implemented a tool to compute our textual distance and\nconducted experiments on several corpuses of texts. The experiments show that\nour methods can reliably identify different global types of texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4467v2",
            "date": "2014-03-18 14:16:45+00:00",
            "title": "A hybrid formalism to parse Sign Languages",
            "authors": [
                "R\u00e9mi Dubot",
                "Christophe Collet"
            ],
            "abstract": "Sign Language (SL) linguistic is dependent on the expensive task of\nannotating. Some automation is already available for low-level information (eg.\nbody part tracking) and the lexical level has shown significant progresses. The\nsyntactic level lacks annotated corpora as well as complete and consistent\nmodels. This article presents a solution for the automatic annotation of SL\nsyntactic elements. It exposes a formalism able to represent both\nconstituency-based and dependency-based models. The first enable the\nrepresentation the structures one may want to annotate, the second aims at\nfulfilling the holes of the first. A parser is presented and used to conduct\ntwo experiments on the solution. One experiment is on a real corpus, the other\nis on a synthetic corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4473v1",
            "date": "2014-03-18 14:31:51+00:00",
            "title": "Sign Language Gibberish for syntactic parsing evaluation",
            "authors": [
                "R\u00e9mi Dubot",
                "Christophe Collet"
            ],
            "abstract": "Sign Language (SL) automatic processing slowly progresses bottom-up. The\nfield has seen proposition to handle the video signal, to recognize and\nsynthesize sublexical and lexical units. It starts to see the development of\nsupra-lexical processing. But the recognition, at this level, lacks data. The\nsyntax of SL appears very specific as it uses massively the multiplicity of\narticulators and its access to the spatial dimensions. Therefore new parsing\ntechniques are developed. However these need to be evaluated. The shortage on\nreal data restrains the corpus-based models to small sizes. We propose here a\nsolution to produce data-sets for the evaluation of parsers on the specific\nproperties of SL. The article first describes the general model used to\ngenerates dependency grammars and the phrase generation from these lasts. It\nthen discusses the limits of approach. The solution shows to be of particular\ninterest to evaluate the scalability of the techniques on big models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4759v1",
            "date": "2014-03-19 10:45:29+00:00",
            "title": "Spelling Error Trends and Patterns in Sindhi",
            "authors": [
                "Zeeshan Bhatti",
                "Imdad Ali Ismaili",
                "Asad Ali Shaikh",
                "Waseem Javaid"
            ],
            "abstract": "Statistical error Correction technique is the most accurate and widely used\napproach today, but for a language like Sindhi which is a low resourced\nlanguage the trained corpora's are not available, so the statistical techniques\nare not possible at all. Instead a useful alternative would be to exploit\nvarious spelling error trends in Sindhi by using a Rule based approach. For\ndesigning such technique an essential prerequisite would be to study the\nvarious error patterns in a language. This pa per presents various studies of\nspelling error trends and their types in Sindhi Language. The research shows\nthat the error trends common to all languages are also encountered in Sindhi\nbut their do exist some error patters that are catered specifically to a Sindhi\nlanguage.",
            "journal ref": "Journal of Emerging Trends in Computing and Information Sciences,\n  VOL. 3, NO.10 Oct, 2012, ISSN 2079-8407",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4887v2",
            "date": "2014-03-19 17:29:24+00:00",
            "title": "Using Entropy Estimates for DAG-Based Ontologies",
            "authors": [
                "Andrew Warren",
                "Joao Setubal"
            ],
            "abstract": "Motivation: Entropy measurements on hierarchical structures have been used in\nmethods for information retrieval and natural language modeling. Here we\nexplore its application to semantic similarity. By finding shared ontology\nterms, semantic similarity can be established between annotated genes. A common\nprocedure for establishing semantic similarity is to calculate the\ndescriptiveness (information content) of ontology terms and use these values to\ndetermine the similarity of annotations. Most often information content is\ncalculated for an ontology term by analyzing its frequency in an annotation\ncorpus. The inherent problems in using these values to model functional\nsimilarity motivates our work. Summary: We present a novel calculation for\nestablishing the entropy of a DAG-based ontology, which can be used in an\nalternative method for establishing the information content of its terms. We\nalso compare our IC metric to two others using semantic and sequence\nsimilarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.4928v1",
            "date": "2014-03-19 19:59:49+00:00",
            "title": "Clinical TempEval",
            "authors": [
                "Steven Bethard",
                "Leon Derczynski",
                "James Pustejovsky",
                "Marc Verhagen"
            ],
            "abstract": "We describe the Clinical TempEval task which is currently in preparation for\nthe SemEval-2015 evaluation exercise. This task involves identifying and\ndescribing events, times and the relations between them in clinical text. Six\ndiscrete subtasks are included, focusing on recognising mentions of times and\nevents, describing those mentions for both entity types, identifying the\nrelation between an event and the document creation time, and identifying\nnarrative container relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.6381v1",
            "date": "2014-03-21 04:54:28+00:00",
            "title": "An efficiency dependency parser using hybrid approach for tamil language",
            "authors": [
                "K. Sureka",
                "K. G. Srinivasagan",
                "S. Suganthi"
            ],
            "abstract": "Natural language processing is a prompt research area across the country.\nParsing is one of the very crucial tool in language analysis system which aims\nto forecast the structural relationship among the words in a given sentence.\nMany researchers have already developed so many language tools but the accuracy\nis not meet out the human expectation level, thus the research is still exists.\nMachine translation is one of the major application area under Natural Language\nProcessing. While translation between one language to another language, the\nstructure identification of a sentence play a key role. This paper introduces\nthe hybrid way to solve the identification of relationship among the given\nwords in a sentence. In existing system is implemented using rule based\napproach, which is not suited in huge amount of data. The machine learning\napproaches is suitable for handle larger amount of data and also to get better\naccuracy via learning and training the system. The proposed approach takes a\nTamil sentence as an input and produce the result of a dependency relation as a\ntree like structure using hybrid approach. This proposed tool is very helpful\nfor researchers and act as an odd-on improve the quality of existing\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.6392v2",
            "date": "2014-03-25 15:36:36+00:00",
            "title": "Implementation of an Automatic Sign Language Lexical Annotation Framework based on Propositional Dynamic Logic",
            "authors": [
                "Arturo Curiel",
                "Christophe Collet"
            ],
            "abstract": "In this paper, we present the implementation of an automatic Sign Language\n(SL) sign annotation framework based on a formal logic, the Propositional\nDynamic Logic (PDL). Our system relies heavily on the use of a specific variant\nof PDL, the Propositional Dynamic Logic for Sign Language (PDLSL), which lets\nus describe SL signs as formulae and corpora videos as labeled transition\nsystems (LTSs). Here, we intend to show how a generic annotation system can be\nconstructed upon these underlying theoretical principles, regardless of the\ntracking technologies available or the input format of corpora. With this in\nmind, we generated a development framework that adapts the system to specific\nuse cases. Furthermore, we present some results obtained by our application\nwhen adapted to one distinct case, 2D corpora analysis with pre-processed\ntracking information. We also present some insights on how such a technology\ncan be used to analyze 3D real-time data, captured with a depth device.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.6636v1",
            "date": "2014-03-26 11:47:37+00:00",
            "title": "Sign Language Lexical Recognition With Propositional Dynamic Logic",
            "authors": [
                "Arturo Curiel",
                "Christophe Collet"
            ],
            "abstract": "This paper explores the use of Propositional Dynamic Logic (PDL) as a\nsuitable formal framework for describing Sign Language (SL), the language of\ndeaf people, in the context of natural language processing. SLs are visual,\ncomplete, standalone languages which are just as expressive as oral languages.\nSigns in SL usually correspond to sequences of highly specific body postures\ninterleaved with movements, which make reference to real world objects,\ncharacters or situations. Here we propose a formal representation of SL signs,\nthat will help us with the analysis of automatically-collected hand tracking\ndata from French Sign Language (FSL) video corpora. We further show how such a\nrepresentation could help us with the design of computer aided SL verification\ntools, which in turn would bring us closer to the development of an automatic\nrecognition system for these languages.",
            "journal ref": "In Proceedings of the 51st Annual Meeting of the Association for\n  Computational Linguistics, vol. 2, pp. 328-333. 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.7455v1",
            "date": "2014-03-28 17:30:32+00:00",
            "title": "Hybrid Approach to English-Hindi Name Entity Transliteration",
            "authors": [
                "Shruti Mathur",
                "Varun Prakash Saxena"
            ],
            "abstract": "Machine translation (MT) research in Indian languages is still in its\ninfancy. Not much work has been done in proper transliteration of name entities\nin this domain. In this paper we address this issue. We have used English-Hindi\nlanguage pair for our experiments and have used a hybrid approach. At first we\nhave processed English words using a rule based approach which extracts\nindividual phonemes from the words and then we have applied statistical\napproach which converts the English into its equivalent Hindi phoneme and in\nturn the corresponding Hindi word. Through this approach we have attained\n83.40% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0145v1",
            "date": "2014-05-01 12:29:30+00:00",
            "title": "Contextual Semantic Parsing using Crowdsourced Spatial Descriptions",
            "authors": [
                "Kais Dukes"
            ],
            "abstract": "We describe a contextual parser for the Robot Commands Treebank, a new\ncrowdsourced resource. In contrast to previous semantic parsers that select the\nmost-probable parse, we consider the different problem of parsing using\nadditional situational context to disambiguate between different readings of a\nsentence. We show that multiple semantic analyses can be searched using dynamic\nprogramming via interaction with a spatial planner, to guide the parsing\nprocess. We are able to parse sentences in near linear-time by ruling out\nanalyses early on that are incompatible with spatial context. We report a 34%\nupper bound on accuracy, as our planner correctly processes spatial context for\n3,394 out of 10,000 sentences. However, our parser achieves a 96.53%\nexact-match score for parsing within the subset of sentences recognized by the\nplanner, compared to 82.14% for a non-contextual parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0603v1",
            "date": "2014-05-03 16:07:19+00:00",
            "title": "Extracting Family Relationship Networks from Novels",
            "authors": [
                "Aibek Makazhanov",
                "Denilson Barbosa",
                "Grzegorz Kondrak"
            ],
            "abstract": "We present an approach to the extraction of family relations from literary\nnarrative, which incorporates a technique for utterance attribution proposed\nrecently by Elson and McKeown (2010). In our work this technique is used in\ncombination with the detection of vocatives - the explicit forms of address\nused by the characters in a novel. We take advantage of the fact that certain\nvocatives indicate family relations between speakers. The extracted relations\nare then propagated using a set of rules. We report the results of the\napplication of our method to Jane Austen's Pride and Prejudice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0701v1",
            "date": "2014-05-04 14:23:53+00:00",
            "title": "\"Translation can't change a name\": Using Multilingual Data for Named Entity Recognition",
            "authors": [
                "Manaal Faruqui"
            ],
            "abstract": "Named Entities (NEs) are often written with no orthographic changes across\ndifferent languages that share a common alphabet. We show that this can be\nleveraged so as to improve named entity recognition (NER) by using unsupervised\nword clusters from secondary languages as features in state-of-the-art\ndiscriminative NER systems. We observe significant increases in performance,\nfinding that person and location identification is particularly improved, and\nthat phylogenetically close languages provide more valuable features than more\ndistant languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0947v1",
            "date": "2014-05-05 16:24:09+00:00",
            "title": "Learning Bilingual Word Representations by Marginalizing Alignments",
            "authors": [
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Karl Moritz Hermann",
                "Phil Blunsom"
            ],
            "abstract": "We present a probabilistic model that simultaneously learns alignments and\ndistributed representations for bilingual data. By marginalizing over word\nalignments the model captures a larger semantic context than prior work relying\non hard alignments. The advantage of this approach is demonstrated in a\ncross-lingual classification task, where we outperform the prior published\nstate of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1346v1",
            "date": "2014-05-06 16:28:16+00:00",
            "title": "Automatic Method Of Domain Ontology Construction based on Characteristics of Corpora POS-Analysis",
            "authors": [
                "Olena Orobinska"
            ],
            "abstract": "It is now widely recognized that ontologies, are one of the fundamental\ncornerstones of knowledge-based systems. What is lacking, however, is a\ncurrently accepted strategy of how to build ontology; what kinds of the\nresources and techniques are indispensables to optimize the expenses and the\ntime on the one hand and the amplitude, the completeness, the robustness of en\nontology on the other hand. The paper offers a semi-automatic ontology\nconstruction method from text corpora in the domain of radiological protection.\nThis method is composed from next steps: 1) text annotation with part-of-speech\ntags; 2) revelation of the significant linguistic structures and forming the\ntemplates; 3) search of text fragments corresponding to these templates; 4)\nbasic ontology instantiation process",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1359v3",
            "date": "2014-05-06 16:52:34+00:00",
            "title": "Latent semantics of action verbs reflect phonetic parameters of intensity and emotional content",
            "authors": [
                "Michael Kai Petersen"
            ],
            "abstract": "Conjuring up our thoughts, language reflects statistical patterns of word\nco-occurrences which in turn come to describe how we perceive the world.\nWhether counting how frequently nouns and verbs combine in Google search\nqueries, or extracting eigenvectors from term document matrices made up of\nWikipedia lines and Shakespeare plots, the resulting latent semantics capture\nnot only the associative links which form concepts, but also spatial dimensions\nembedded within the surface structure of language. As both the shape and\nmovements of objects have been found to be associated with phonetic contrasts\nalready in toddlers, this study explores whether articulatory and acoustic\nparameters may likewise differentiate the latent semantics of action verbs.\nSelecting 3 x 20 emotion, face, and hand related verbs known to activate\npremotor areas in the brain, their mutual cosine similarities were computed\nusing latent semantic analysis LSA, and the resulting adjacency matrices were\ncompared based on two different large scale text corpora; HAWIK and TASA.\nApplying hierarchical clustering to identify common structures across the two\ntext corpora, the verbs largely divide into combined mouth and hand movements\nversus emotional expressions. Transforming the verbs into their constituent\nphonemes, the clustered small and large size movements appear differentiated by\nfront versus back vowels corresponding to increasing levels of arousal. Whereas\nthe clustered emotional verbs seem characterized by sequences of close versus\nopen jaw produced phonemes, generating up- or downwards shifts in formant\nfrequencies that may influence their perceived valence. Suggesting, that the\nlatent semantics of action verbs reflect parameters of intensity and emotional\npolarity that appear correlated with the articulatory contrasts and acoustic\ncharacteristics of phonemes",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1406v1",
            "date": "2014-05-06 19:26:35+00:00",
            "title": "D-Bees: A Novel Method Inspired by Bee Colony Optimization for Solving Word Sense Disambiguation",
            "authors": [
                "Sallam Abualhaija",
                "Karl-Heinz Zimmermann"
            ],
            "abstract": "Word sense disambiguation (WSD) is a problem in the field of computational\nlinguistics given as finding the intended sense of a word (or a set of words)\nwhen it is activated within a certain context. WSD was recently addressed as a\ncombinatorial optimization problem in which the goal is to find a sequence of\nsenses that maximize the semantic relatedness among the target words. In this\narticle, a novel algorithm for solving the WSD problem called D-Bees is\nproposed which is inspired by bee colony optimization (BCO)where artificial bee\nagents collaborate to solve the problem. The D-Bees algorithm is evaluated on a\nstandard dataset (SemEval 2007 coarse-grained English all-words task corpus)and\nis compared to simulated annealing, genetic algorithms, and two ant colony\noptimization techniques (ACO). It will be observed that the BCO and ACO\napproaches are on par.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1439v2",
            "date": "2014-05-06 20:05:23+00:00",
            "title": "A Corpus of Sentence-level Revisions in Academic Writing: A Step towards Understanding Statement Strength in Communication",
            "authors": [
                "Chenhao Tan",
                "Lillian Lee"
            ],
            "abstract": "The strength with which a statement is made can have a significant impact on\nthe audience. For example, international relations can be strained by how the\nmedia in one country describes an event in another; and papers can be rejected\nbecause they overstate or understate their findings. It is thus important to\nunderstand the effects of statement strength. A first step is to be able to\ndistinguish between strong and weak statements. However, even this problem is\nunderstudied, partly due to a lack of data. Since strength is inherently\nrelative, revisions of texts that make claims are a natural source of data on\nstrength differences. In this paper, we introduce a corpus of sentence-level\nrevisions from academic writing. We also describe insights gained from our\nannotation efforts for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1924v1",
            "date": "2014-05-08 13:47:36+00:00",
            "title": "An Expert System for Automatic Reading of A Text Written in Standard Arabic",
            "authors": [
                "Tebbi Hanane",
                "Azzoune Hamid"
            ],
            "abstract": "In this work we present our expert system of Automatic reading or speech\nsynthesis based on a text written in Standard Arabic, our work is carried out\nin two great stages: the creation of the sound data base, and the\ntransformation of the written text into speech (Text To Speech TTS). This\ntransformation is done firstly by a Phonetic Orthographical Transcription (POT)\nof any written Standard Arabic text with the aim of transforming it into his\ncorresponding phonetics sequence, and secondly by the generation of the voice\nsignal which corresponds to the chain transcribed. We spread out the different\nof conception of the system, as well as the results obtained compared to others\nworks studied to realize TTS based on Standard Arabic.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  3, No.2, April 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.2434v1",
            "date": "2014-05-10 13:55:03+00:00",
            "title": "Coordinate System Selection for Minimum Error Rate Training in Statistical Machine Translation",
            "authors": [
                "Chen Lijiang"
            ],
            "abstract": "Minimum error rate training (MERT) is a widely used training procedure for\nstatistical machine translation. A general problem of this approach is that the\nsearch space is easy to converge to a local optimum and the acquired weight set\nis not in accord with the real distribution of feature functions. This paper\nintroduces coordinate system selection (RSS) into the search algorithm for\nMERT. Contrary to previous approaches in which every dimension only corresponds\nto one independent feature function, we create several coordinate systems by\nmoving one of the dimensions to a new direction. The basic idea is quite simple\nbut critical that the training procedure of MERT should be based on a\ncoordinate system formed by search directions but not directly on feature\nfunctions. Experiments show that by selecting coordinate systems with tuning\nset results, better results can be obtained without any other language\nknowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3033v1",
            "date": "2014-05-13 04:33:04+00:00",
            "title": "Phonetic based SoundEx & ShapeEx algorithm for Sindhi Spell Checker System",
            "authors": [
                "Zeeshan Bhatti",
                "Ahmad Waqas",
                "Imdad Ali Ismaili",
                "Dil Nawaz Hakro",
                "Waseem Javaid Soomro"
            ],
            "abstract": "This paper presents a novel combinational phonetic algorithm for Sindhi\nLanguage, to be used in developing Sindhi Spell Checker which has yet not been\ndeveloped prior to this work. The compound textual forms and glyphs of Sindhi\nlanguage presents a substantial challenge for developing Sindhi spell checker\nsystem and generating similar suggestion list for misspelled words. In order to\nimplement such a system, phonetic based Sindhi language rules and patterns must\nbe considered into account for increasing the accuracy and efficiency. The\nproposed system is developed with a blend between Phonetic based SoundEx\nalgorithm and ShapeEx algorithm for pattern or glyph matching, generating\naccurate and efficient suggestion list for incorrect or misspelled Sindhi\nwords. A table of phonetically similar sounding Sindhi characters for SoundEx\nalgorithm is also generated along with another table containing similar glyph\nor shape based character groups for ShapeEx algorithm. Both these are first\never attempt of any such type of categorization and representation for Sindhi\nLanguage.",
            "journal ref": "Adv. Environ. Biol., 8(4), 1147-1155, AENSI Publisher, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3515v1",
            "date": "2014-05-14 14:47:22+00:00",
            "title": "Temporal Analysis of Language through Neural Language Models",
            "authors": [
                "Yoon Kim",
                "Yi-I Chiu",
                "Kentaro Hanaki",
                "Darshan Hegde",
                "Slav Petrov"
            ],
            "abstract": "We provide a method for automatically detecting change in language across\ntime through a chronologically trained neural language model. We train the\nmodel on the Google Books Ngram corpus to obtain word vector representations\nspecific to each year, and identify words that have changed significantly from\n1900 to 2009. The model identifies words such as \"cell\" and \"gay\" as having\nchanged during that time period. The model simultaneously identifies the\nspecific years during which such words underwent change.",
            "journal ref": "Proceedings of the ACL 2014 Workshop on Language Technologies and\n  Computational Social Science. June, 2014. 61--65",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3772v1",
            "date": "2014-05-15 09:11:53+00:00",
            "title": "INAUT, a Controlled Language for the French Coast Pilot Books Instructions nautiques",
            "authors": [
                "Yannis Haralambous",
                "Julie Sauvage-Vincent",
                "John Puentes"
            ],
            "abstract": "We describe INAUT, a controlled natural language dedicated to collaborative\nupdate of a knowledge base on maritime navigation and to automatic generation\nof coast pilot books (Instructions nautiques) of the French National\nHydrographic and Oceanographic Service SHOM. INAUT is based on French language\nand abundantly uses georeferenced entities. After describing the structure of\nthe overall system, giving details on the language and on its generation, and\ndiscussing the three major applications of INAUT (document production,\ninteraction with ENCs and collaborative updates of the knowledge base), we\nconclude with future extensions and open problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3925v1",
            "date": "2014-05-15 18:10:36+00:00",
            "title": "M\u00e9thodes pour la repr\u00e9sentation informatis\u00e9e de donn\u00e9es lexicales / Methoden der Speicherung lexikalischer Daten",
            "authors": [
                "Laurent Romary",
                "Andreas Witt"
            ],
            "abstract": "In recent years, new developments in the area of lexicography have altered\nnot only the management, processing and publishing of lexicographical data, but\nalso created new types of products such as electronic dictionaries and\nthesauri. These expand the range of possible uses of lexical data and support\nusers with more flexibility, for instance in assisting human translation. In\nthis article, we give a short and easy-to-understand introduction to the\nproblematic nature of the storage, display and interpretation of lexical data.\nWe then describe the main methods and specifications used to build and\nrepresent lexical data. This paper is targeted for the following groups of\npeople: linguists, lexicographers, IT specialists, computer linguists and all\nothers who wish to learn more about the modelling, representation and\nvisualization of lexical knowledge. This paper is written in two languages:\nFrench and German.",
            "journal ref": "Lexicographica 30 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4097v2",
            "date": "2014-05-16 08:57:40+00:00",
            "title": "A preliminary study of Croatian Language Syllable Networks",
            "authors": [
                "Kristina Ban",
                "Ivan Ivaki\u0107",
                "Ana Me\u0161trovi\u0107"
            ],
            "abstract": "This paper presents preliminary results of Croatian syllable networks\nanalysis. Syllable network is a network in which nodes are syllables and links\nbetween them are constructed according to their connections within words. In\nthis paper we analyze networks of syllables generated from texts collected from\nthe Croatian Wikipedia and Blogs. As a main tool we use complex network\nanalysis methods which provide mechanisms that can reveal new patterns in a\nlanguage structure. We aim to show that syllable networks have much higher\nclustering coefficient in comparison to Erd\\\"os-Renyi random networks. The\nresults indicate that Croatian syllable networks exhibit certain properties of\na small world networks. Furthermore, we compared Croatian syllable networks\nwith Portuguese and Chinese syllable networks and we showed that they have\nsimilar properties.",
            "journal ref": "IEEE 36th International Convention on Information and\n  Communication Technology, Electronics and Microelectronics (MIPRO 2013), pp.\n  1004-1008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4248v1",
            "date": "2014-05-16 17:17:19+00:00",
            "title": "Les math\u00e9matiques de la langue : l'approche formelle de Montague",
            "authors": [
                "Yannis Haralambous"
            ],
            "abstract": "We present a natural language modelization method which is strongely relying\non mathematics. This method, called \"Formal Semantics,\" has been initiated by\nthe American linguist Richard M. Montague in the 1970's. It uses mathematical\ntools such as formal languages and grammars, first-order logic, type theory and\n$\\lambda$-calculus. Our goal is to have the reader discover both Montagovian\nformal semantics and the mathematical tools that he used in his method.\n  -----\n  Nous pr\\'esentons une m\\'ethode de mod\\'elisation de la langue naturelle qui\nest fortement bas\\'ee sur les math\\'ematiques. Cette m\\'ethode, appel\\'ee\n{\\guillemotleft}s\\'emantique formelle{\\guillemotright}, a \\'et\\'e initi\\'ee par\nle linguiste am\\'ericain Richard M. Montague dans les ann\\'ees 1970. Elle\nutilise des outils math\\'ematiques tels que les langages et grammaires formels,\nla logique du 1er ordre, la th\\'eorie de types et le $\\lambda$-calcul. Nous\nnous proposons de faire d\\'ecouvrir au lecteur tant la s\\'emantique formelle de\nMontague que les outils math\\'ematiques dont il s'est servi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4273v1",
            "date": "2014-05-16 19:08:14+00:00",
            "title": "Compositional Morphology for Word Representations and Language Modelling",
            "authors": [
                "Jan A. Botha",
                "Phil Blunsom"
            ],
            "abstract": "This paper presents a scalable method for integrating compositional\nmorphological representations into a vector-based probabilistic language model.\nOur approach is evaluated in the context of log-bilinear language models,\nrendered suitably efficient for implementation inside a machine translation\ndecoder by factoring the vocabulary. We perform both intrinsic and extrinsic\nevaluations, presenting results on a range of languages which demonstrate that\nour model learns morphological representations that both perform well on word\nsimilarity tasks and lead to substantial reductions in perplexity. When used\nfor translation into morphologically rich languages with large vocabularies,\nour models obtain improvements of up to 1.2 BLEU points relative to a baseline\nsystem using back-off n-gram models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4364v1",
            "date": "2014-05-17 07:58:58+00:00",
            "title": "Thematically Reinforced Explicit Semantic Analysis",
            "authors": [
                "Yannis Haralambous",
                "Vitaly Klyuev"
            ],
            "abstract": "We present an extended, thematically reinforced version of Gabrilovich and\nMarkovitch's Explicit Semantic Analysis (ESA), where we obtain thematic\ninformation through the category structure of Wikipedia. For this we first\ndefine a notion of categorical tfidf which measures the relevance of terms in\ncategories. Using this measure as a weight we calculate a maximal spanning tree\nof the Wikipedia corpus considered as a directed graph of pages and categories.\nThis tree provides us with a unique path of \"most related categories\" between\neach page and the top of the hierarchy. We reinforce tfidf of words in a page\nby aggregating it with categorical tfidfs of the nodes of these paths, and\ndefine a thematically reinforced ESA semantic relatedness measure which is more\nrobust than standard ESA and less sensitive to noise caused by out-of-context\nwords. We apply our method to the French Wikipedia corpus, evaluate it through\na text classification on a 37.5 MB corpus of 20 French newsgroups and obtain a\nprecision increase of 9-10% compared with standard ESA.",
            "journal ref": "IJCLA vol. 4, no. 1, Jan.-Jun. 2013, pp. 79--94",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5202v1",
            "date": "2014-01-16 05:06:09+00:00",
            "title": "Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution",
            "authors": [
                "Altaf Rahman",
                "Vincent Ng"
            ],
            "abstract": "Traditional learning-based coreference resolvers operate by training the\nmention-pair model for determining whether two mentions are coreferent or not.\nThough conceptually simple and easy to understand, the mention-pair model is\nlinguistically rather unappealing and lags far behind the heuristic-based\ncoreference models proposed in the pre-statistical NLP era in terms of\nsophistication. Two independent lines of recent research have attempted to\nimprove the mention-pair model, one by acquiring the mention-ranking model to\nrank preceding mentions for a given anaphor, and the other by training the\nentity-mention model to determine whether a preceding cluster is coreferent\nwith a given mention. We propose a cluster-ranking approach to coreference\nresolution, which combines the strengths of the mention-ranking model and the\nentity-mention model, and is therefore theoretically more appealing than both\nof these models. In addition, we seek to improve cluster rankers via two\nextensions: (1) lexicalization and (2) incorporating knowledge of anaphoricity\nby jointly modeling anaphoricity determination and coreference resolution.\nExperimental results on the ACE data sets demonstrate the superior performance\nof cluster rankers to competing approaches as well as the effectiveness of our\ntwo extensions.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 40, pages\n  469-521, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5474v1",
            "date": "2014-05-21 16:49:50+00:00",
            "title": "New Perspectives in Sinographic Language Processing Through the Use of Character Structure",
            "authors": [
                "Yannis Haralambous"
            ],
            "abstract": "Chinese characters have a complex and hierarchical graphical structure\ncarrying both semantic and phonetic information. We use this structure to\nenhance the text model and obtain better results in standard NLP operations.\nFirst of all, to tackle the problem of graphical variation we define\nallographic classes of characters. Next, the relation of inclusion of a\nsubcharacter in a characters, provides us with a directed graph of allographic\nclasses. We provide this graph with two weights: semanticity (semantic relation\nbetween subcharacter and character) and phoneticity (phonetic relation) and\ncalculate \"most semantic subcharacter paths\" for each character. Finally,\nadding the information contained in these paths to unigrams we claim to\nincrease the efficiency of text mining methods. We evaluate our method on a\ntext classification task on two corpora (Chinese and Japanese) of a total of 18\nmillion characters and get an improvement of 3% on an already high baseline of\n89.6% precision, obtained by a linear SVM classifier. Other possible\napplications and perspectives of the system are discussed.",
            "journal ref": "Lecture Notes in Computer Science 7816 (2013), pp. 201--217",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5654v1",
            "date": "2014-05-22 07:58:56+00:00",
            "title": "Machine Translation Model based on Non-parallel Corpus and Semi-supervised Transductive Learning",
            "authors": [
                "Lijiang Chen"
            ],
            "abstract": "Although the parallel corpus has an irreplaceable role in machine\ntranslation, its scale and coverage is still beyond the actual needs.\nNon-parallel corpus resources on the web have an inestimable potential value in\nmachine translation and other natural language processing tasks. This article\nproposes a semi-supervised transductive learning method for expanding the\ntraining corpus in statistical machine translation system by extracting\nparallel sentences from the non-parallel corpus. This method only requires a\nsmall amount of labeled corpus and a large unlabeled corpus to build a\nhigh-performance classifier, especially for when there is short of labeled\ncorpus. The experimental results show that by combining the non-parallel corpus\nalignment and the semi-supervised transductive learning method, we can more\neffectively use their respective strengths to improve the performance of\nmachine translation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5674v1",
            "date": "2014-05-22 08:57:54+00:00",
            "title": "Mot\u00e0Mot project: conversion of a French-Khmer published dictionary for building a multilingual lexical system",
            "authors": [
                "Mathieu Mangeot"
            ],
            "abstract": "Economic issues related to the information processing techniques are very\nimportant. The development of such technologies is a major asset for developing\ncountries like Cambodia and Laos, and emerging ones like Vietnam, Malaysia and\nThailand. The MotAMot project aims to computerize an under-resourced language:\nKhmer, spoken mainly in Cambodia. The main goal of the project is the\ndevelopment of a multilingual lexical system targeted for Khmer. The\nmacrostructure is a pivot one with each word sense of each language linked to a\npivot axi. The microstructure comes from a simplification of the explanatory\nand combinatory dictionary. The lexical system has been initialized with data\ncoming mainly from the conversion of the French-Khmer bilingual dictionary of\nDenis Richer from Word to XML format. The French part was completed with\npronunciation and parts-of-speech coming from the FeM French-english-Malay\ndictionary. The Khmer headwords noted in IPA in the Richer dictionary were\nconverted to Khmer writing with OpenFST, a finite state transducer tool. The\nresulting resource is available online for lookup, editing, download and remote\nprogramming via a REST API on a Jibiki platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5893v1",
            "date": "2014-05-22 20:15:57+00:00",
            "title": "Computerization of African languages-French dictionaries",
            "authors": [
                "Chantal Enguehard",
                "Mathieu Mangeot"
            ],
            "abstract": "This paper relates work done during the DiLAF project. It consists in\nconverting 5 bilingual African language-French dictionaries originally in Word\nformat into XML following the LMF model. The languages processed are Bambara,\nHausa, Kanuri, Tamajaq and Songhai-zarma, still considered as under-resourced\nlanguages concerning Natural Language Processing tools. Once converted, the\ndictionaries are available online on the Jibiki platform for lookup and\nmodification. The DiLAF project is first presented. A description of each\ndictionary follows. Then, the conversion methodology from .doc format to XML\nfiles is presented. A specific point on the usage of Unicode follows. Then,\neach step of the conversion into XML and LMF is detailed. The last part\npresents the Jibiki lexical resources management platform used for the project.",
            "journal ref": "Iceland (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.6068v1",
            "date": "2014-05-23 14:05:45+00:00",
            "title": "Building of Networks of Natural Hierarchies of Terms Based on Analysis of Texts Corpora",
            "authors": [
                "Dmitry Lande"
            ],
            "abstract": "The technique of building of networks of hierarchies of terms based on the\nanalysis of chosen text corpora is offered. The technique is based on the\nmethodology of horizontal visibility graphs. Constructed and investigated\nlanguage network, formed on the basis of electronic preprints arXiv on topics\nof information retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.6103v1",
            "date": "2014-05-23 15:51:10+00:00",
            "title": "Evaluating the fully automatic multi-language translation of the Swiss avalanche bulletin",
            "authors": [
                "Kurt Winkler",
                "Tobias Kuhn",
                "Martin Volk"
            ],
            "abstract": "The Swiss avalanche bulletin is produced twice a day in four languages. Due\nto the lack of time available for manual translation, a fully automated\ntranslation system is employed, based on a catalogue of predefined phrases and\npredetermined rules of how these phrases can be combined to produce sentences.\nThe system is able to automatically translate such sentences from German into\nthe target languages French, Italian and English without subsequent\nproofreading or correction. Our catalogue of phrases is limited to a small\nsublanguage. The reduction of daily translation costs is expected to offset the\ninitial development costs within a few years. After being operational for two\nwinter seasons, we assess here the quality of the produced texts based on an\nevaluation where participants rate real danger descriptions from both origins,\nthe catalogue of phrases versus the manually written and translated texts. With\na mean recognition rate of 55%, users can hardly distinguish between the two\ntypes of texts, and give similar ratings with respect to their language\nquality. Overall, the output from the catalogue system can be considered\nvirtually equivalent to a text written by avalanche forecasters and then\nmanually translated by professional translators. Furthermore, forecasters\ndeclared that all relevant situations were captured by the system with\nsufficient accuracy and within the limited time available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.6293v1",
            "date": "2014-05-24 11:39:34+00:00",
            "title": "Cross-Language Personal Name Mapping",
            "authors": [
                "Ahmed H. Yousef"
            ],
            "abstract": "Name matching between multiple natural languages is an important step in\ncross-enterprise integration applications and data mining. It is difficult to\ndecide whether or not two syntactic values (names) from two heterogeneous data\nsources are alternative designation of the same semantic entity (person), this\nprocess becomes more difficult with Arabic language due to several factors\nincluding spelling and pronunciation variation, dialects and special vowel and\nconsonant distinction and other linguistic characteristics. This paper proposes\na new framework for name matching between the Arabic language and other\nlanguages. The framework uses a dictionary based on a new proposed version of\nthe Soundex algorithm to encapsulate the recognition of special features of\nArabic names. The framework proposes a new proximity matching algorithm to suit\nthe high importance of order sensitivity in Arabic name matching. New\nperformance evaluation metrics are proposed as well. The framework is\nimplemented and verified empirically in several case studies demonstrating\nsubstantial improvements compared to other well-known techniques found in\nliterature.",
            "journal ref": "International Journal of Computational Linguistics Research, vol\n  4, issue 4, December 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.6682v1",
            "date": "2014-05-26 18:51:06+00:00",
            "title": "Optimality Theory as a Framework for Lexical Acquisition",
            "authors": [
                "Thierry Poibeau"
            ],
            "abstract": "This paper re-investigates a lexical acquisition system initially developed\nfor French.We show that, interestingly, the architecture of the system\nreproduces and implements the main components of Optimality Theory. However, we\nformulate the hypothesis that some of its limitations are mainly due to a poor\nrepresentation of the constraints used. Finally, we show how a better\nrepresentation of the constraints used would yield better results.",
            "journal ref": "15th International Conference on Intelligent Text Processing and\n  Computational Linguistics, Nepal (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.7397v1",
            "date": "2014-05-28 21:05:00+00:00",
            "title": "An HMM Based Named Entity Recognition System for Indian Languages: The JU System at ICON 2013",
            "authors": [
                "Vivekananda Gayen",
                "Kamal Sarkar"
            ],
            "abstract": "This paper reports about our work in the ICON 2013 NLP TOOLS CONTEST on Named\nEntity Recognition. We submitted runs for Bengali, English, Hindi, Marathi,\nPunjabi, Tamil and Telugu. A statistical HMM (Hidden Markov Models) based model\nhas been used to implement our system. The system has been trained and tested\non the NLP TOOLS CONTEST: ICON 2013 datasets. Our system obtains F-measures of\n0.8599, 0.7704, 0.7520, 0.4289, 0.5455, 0.4466, and 0.4003 for Bengali,\nEnglish, Hindi, Marathi, Punjabi, Tamil and Telugu respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.7711v1",
            "date": "2014-01-16 04:29:26+00:00",
            "title": "Training a Multilingual Sportscaster: Using Perceptual Context to Learn Language",
            "authors": [
                "David L. Chen",
                "Joohyun Kim",
                "Raymond J. Mooney"
            ],
            "abstract": "We present a novel framework for learning to interpret and generate language\nusing only perceptual context as supervision. We demonstrate its capabilities\nby developing a system that learns to sportscast simulated robot soccer games\nin both English and Korean without any language-specific prior knowledge.\nTraining employs only ambiguous supervision consisting of a stream of\ndescriptive textual comments and a sequence of events extracted from the\nsimulation trace. The system simultaneously establishes correspondences between\nindividual comments and the events that they describe while building a\ntranslation model that supports both parsing and generation. We also present a\nnovel algorithm for learning which events are worth describing. Human\nevaluations of the generated commentaries indicate they are of reasonable\nquality and in some cases even on par with those produced by humans for our\nlimited domain.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 37, pages\n  397-435, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.1605v1",
            "date": "2014-07-07 07:08:07+00:00",
            "title": "Les noms propres se traduisent-ils ? \u00c9tude d'un corpus multilingue",
            "authors": [
                "\u00c9meline Lecuit",
                "Denis Maurel",
                "Dusko Vitas"
            ],
            "abstract": "In this paper, we tackle the problem of the translation of proper names. We\nintroduce our hypothesis according to which proper names can be translated more\noften than most people seem to think. Then, we describe the construction of a\nparallel multilingual corpus used to illustrate our point. We eventually\nevaluate both the advantages and limits of this corpus in our study.",
            "journal ref": "Corpus 10 (2011) 201-218",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.1976v1",
            "date": "2014-07-08 07:35:16+00:00",
            "title": "Inter-Rater Agreement Study on Readability Assessment in Bengali",
            "authors": [
                "Shanta Phani",
                "Shibamouli Lahiri",
                "Arindam Biswas"
            ],
            "abstract": "An inter-rater agreement study is performed for readability assessment in\nBengali. A 1-7 rating scale was used to indicate different levels of\nreadability. We obtained moderate to fair agreement among seven independent\nannotators on 30 text passages written by four eminent Bengali authors. As a by\nproduct of our study, we obtained a readability-annotated ground truth dataset\nin Bengali. .",
            "journal ref": "International Journal on Natural Language Computing (IJNLC), 3(3),\n  2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.2019v1",
            "date": "2014-07-08 10:04:07+00:00",
            "title": "Assamese-English Bilingual Machine Translation",
            "authors": [
                "Kalyanee Kanchan Baruah",
                "Pranjal Das",
                "Abdul Hannan",
                "Shikhar Kr. Sarma"
            ],
            "abstract": "Machine translation is the process of translating text from one language to\nanother. In this paper, Statistical Machine Translation is done on Assamese and\nEnglish language by taking their respective parallel corpus. A statistical\nphrase based translation toolkit Moses is used here. To develop the language\nmodel and to align the words we used two another tools IRSTLM, GIZA\nrespectively. BLEU score is used to check our translation system performance,\nhow good it is. A difference in BLEU scores is obtained while translating\nsentences from Assamese to English and vice-versa. Since Indian languages are\nmorphologically very rich hence translation is relatively harder from English\nto Assamese resulting in a low BLEU score. A statistical transliteration system\nis also introduced with our translation system to deal basically with proper\nnouns, OOV (out of vocabulary) words which are not present in our corpus.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  3, No.3, June 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.2694v1",
            "date": "2014-07-10 05:26:14+00:00",
            "title": "Quality Estimation Of Machine Translation Outputs Through Stemming",
            "authors": [
                "Pooja Gupta",
                "Nisheeth Joshi",
                "Iti Mathur"
            ],
            "abstract": "Machine Translation is the challenging problem for Indian languages. Every\nday we can see some machine translators being developed, but getting a high\nquality automatic translation is still a very distant dream . The correct\ntranslated sentence for Hindi language is rarely found. In this paper, we are\nemphasizing on English-Hindi language pair, so in order to preserve the correct\nMT output we present a ranking system, which employs some machine learning\ntechniques and morphological features. In ranking no human intervention is\nrequired. We have also validated our results by comparing it with human\nranking.",
            "journal ref": "International Journal on Computational Sciences & Applications\n  (IJCSA) Vol.4, No.3, June 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.2918v1",
            "date": "2014-07-09 13:59:27+00:00",
            "title": "A Survey of Named Entity Recognition in Assamese and other Indian Languages",
            "authors": [
                "Gitimoni Talukdar",
                "Pranjal Protim Borah",
                "Arup Baruah"
            ],
            "abstract": "Named Entity Recognition is always important when dealing with major Natural\nLanguage Processing tasks such as information extraction, question-answering,\nmachine translation, document summarization etc so in this paper we put forward\na survey of Named Entities in Indian Languages with particular reference to\nAssamese. There are various rule-based and machine learning approaches\navailable for Named Entity Recognition. At the very first of the paper we give\nan idea of the available approaches for Named Entity Recognition and then we\ndiscuss about the related research in this field. Assamese like other Indian\nlanguages is agglutinative and suffers from lack of appropriate resources as\nNamed Entity Recognition requires large data sets, gazetteer list, dictionary\netc and some useful feature like capitalization as found in English cannot be\nfound in Assamese. Apart from this we also describe some of the issues faced in\nAssamese while doing Named Entity Recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.2989v1",
            "date": "2014-07-10 23:57:54+00:00",
            "title": "Hidden Markov Model Based Part of Speech Tagger for Sinhala Language",
            "authors": [
                "A. J. P. M. P. Jayaweera",
                "N. G. J. Dias"
            ],
            "abstract": "In this paper we present a fundamental lexical semantics of Sinhala language\nand a Hidden Markov Model (HMM) based Part of Speech (POS) Tagger for Sinhala\nlanguage. In any Natural Language processing task, Part of Speech is a very\nvital topic, which involves analysing of the construction, behaviour and the\ndynamics of the language, which the knowledge could utilized in computational\nlinguistics analysis and automation applications. Though Sinhala is a\nmorphologically rich and agglutinative language, in which words are inflected\nwith various grammatical features, tagging is very essential for further\nanalysis of the language. Our research is based on statistical based approach,\nin which the tagging process is done by computing the tag sequence probability\nand the word-likelihood probability from the given corpus, where the linguistic\nknowledge is automatically extracted from the annotated corpus. The current\ntagger could reach more than 90% of accuracy for known words.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  3, No.3, June 2014. Pages 9-23",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.6853v1",
            "date": "2014-07-25 11:17:28+00:00",
            "title": "Substitute Based SCODE Word Embeddings in Supervised NLP Tasks",
            "authors": [
                "Volkan Cirik",
                "Deniz Yuret"
            ],
            "abstract": "We analyze a word embedding method in supervised tasks. It maps words on a\nsphere such that words co-occurring in similar contexts lie closely. The\nsimilarity of contexts is measured by the distribution of substitutes that can\nfill them. We compared word embeddings, including more recent representations,\nin Named Entity Recognition (NER), Chunking, and Dependency Parsing. We examine\nour framework in multilingual dependency parsing as well. The results show that\nthe proposed method achieves as good as or better results compared to the other\nword embeddings in the tasks we investigate. It achieves state-of-the-art\nresults in multilingual dependency parsing. Word embeddings in 7 languages are\navailable for public use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.8215v1",
            "date": "2014-07-30 21:00:25+00:00",
            "title": "Two-pass Discourse Segmentation with Pairing and Global Features",
            "authors": [
                "Vanessa Wei Feng",
                "Graeme Hirst"
            ],
            "abstract": "Previous attempts at RST-style discourse segmentation typically adopt\nfeatures centered on a single token to predict whether to insert a boundary\nbefore that token. In contrast, we develop a discourse segmenter utilizing a\nset of pairing features, which are centered on a pair of adjacent tokens in the\nsentence, by equally taking into account the information from both tokens.\nMoreover, we propose a novel set of global features, which encode\ncharacteristics of the segmentation as a whole, once we have an initial\nsegmentation. We show that both the pairing and global features are useful on\ntheir own, and their combination achieved an $F_1$ of 92.6% of identifying\nin-sentence discourse boundaries, which is a 17.8% error-rate reduction over\nthe state-of-the-art performance, approaching 95% of human performance. In\naddition, similar improvement is observed across different classification\nframeworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.0782v1",
            "date": "2014-08-04 19:31:32+00:00",
            "title": "Targetable Named Entity Recognition in Social Media",
            "authors": [
                "Sandeep Ashwini",
                "Jinho D. Choi"
            ],
            "abstract": "We present a novel approach for recognizing what we call targetable named\nentities; that is, named entities in a targeted set (e.g, movies, books, TV\nshows). Unlike many other NER systems that need to retrain their statistical\nmodels as new entities arrive, our approach does not require such retraining,\nwhich makes it more adaptable for types of entities that are frequently\nupdated. For this preliminary study, we focus on one entity type, movie title,\nusing data collected from Twitter. Our system is tested on two evaluation sets,\none including only entities corresponding to movies in our training set, and\nthe other excluding any of those entities. Our final model shows F1-scores of\n76.19% and 78.70% on these evaluation sets, which gives strong evidence that\nour approach is completely unbiased to any par- ticular set of entities found\nduring training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.1928v1",
            "date": "2014-08-08 17:49:02+00:00",
            "title": "Microtask crowdsourcing for disease mention annotation in PubMed abstracts",
            "authors": [
                "Benjamin M Good",
                "Max Nanis",
                "Andrew I. Su"
            ],
            "abstract": "Identifying concepts and relationships in biomedical text enables knowledge\nto be applied in computational analyses. Many biological natural language\nprocess (BioNLP) projects attempt to address this challenge, but the state of\nthe art in BioNLP still leaves much room for improvement. Progress in BioNLP\nresearch depends on large, annotated corpora for evaluating information\nextraction systems and training machine learning models. Traditionally, such\ncorpora are created by small numbers of expert annotators often working over\nextended periods of time. Recent studies have shown that workers on microtask\ncrowdsourcing platforms such as Amazon's Mechanical Turk (AMT) can, in\naggregate, generate high-quality annotations of biomedical text. Here, we\ninvestigated the use of the AMT in capturing disease mentions in PubMed\nabstracts. We used the NCBI Disease corpus as a gold standard for refining and\nbenchmarking our crowdsourcing protocol. After several iterations, we arrived\nat a protocol that reproduced the annotations of the 593 documents in the\ntraining set of this gold standard with an overall F measure of 0.872\n(precision 0.862, recall 0.883). The output can also be tuned to optimize for\nprecision (max = 0.984 when recall = 0.269) or recall (max = 0.980 when\nprecision = 0.436). Each document was examined by 15 workers, and their\nannotations were merged based on a simple voting method. In total 145 workers\ncombined to complete all 593 documents in the span of 1 week at a cost of $.06\nper abstract per worker. The quality of the annotations, as judged with the F\nmeasure, increases with the number of workers assigned to each task such that\nthe system can be tuned to balance cost against quality. These results\ndemonstrate that microtask crowdsourcing can be a valuable tool for generating\nwell-annotated corpora in BioNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.2359v2",
            "date": "2014-08-11 09:32:39+00:00",
            "title": "Gap-weighted subsequences for automatic cognate identification and phylogenetic inference",
            "authors": [
                "Taraka Rama"
            ],
            "abstract": "In this paper, we describe the problem of cognate identification and its\nrelation to phylogenetic inference. We introduce subsequence based features for\ndiscriminating cognates from non-cognates. We show that subsequence based\nfeatures perform better than the state-of-the-art string similarity measures\nfor the purpose of cognate identification. We use the cognate judgments for the\npurpose of phylogenetic inference and observe that these classifiers infer a\ntree which is close to the gold standard tree. The contribution of this paper\nis the use of subsequence features for cognate identification and to employ the\ncognate judgments for phylogenetic inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.3153v2",
            "date": "2014-08-13 22:09:23+00:00",
            "title": "Detection is the central problem in real-word spelling correction",
            "authors": [
                "L. Amber Wilcox-O'Hearn"
            ],
            "abstract": "Real-word spelling correction differs from non-word spelling correction in\nits aims and its challenges. Here we show that the central problem in real-word\nspelling correction is detection. Methods from non-word spelling correction,\nwhich focus instead on selection among candidate corrections, do not address\ndetection adequately, because detection is either assumed in advance or heavily\nconstrained. As we demonstrate in this paper, merely discriminating between the\nintended word and a random close variation of it within the context of a\nsentence is a task that can be performed with high accuracy using\nstraightforward models. Trigram models are sufficient in almost all cases. The\ndifficulty comes when every word in the sentence is a potential error, with a\nlarge set of possible candidate corrections. Despite their strengths, trigram\nmodels cannot reliably find true errors without introducing many more, at least\nnot when used in the obvious sequential way without added structure. The\ndetection task exposes weakness not visible in the selection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.3456v1",
            "date": "2014-08-15 01:59:29+00:00",
            "title": "SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation",
            "authors": [
                "Felix Hill",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "We present SimLex-999, a gold standard resource for evaluating distributional\nsemantic models that improves on existing resources in several important ways.\nFirst, in contrast to gold standards such as WordSim-353 and MEN, it explicitly\nquantifies similarity rather than association or relatedness, so that pairs of\nentities that are associated but not actually similar [Freud, psychology] have\na low rating. We show that, via this focus on similarity, SimLex-999\nincentivizes the development of models with a different, and arguably wider\nrange of applications than those which reflect conceptual association. Second,\nSimLex-999 contains a range of concrete and abstract adjective, noun and verb\npairs, together with an independent rating of concreteness and (free)\nassociation strength for each pair. This diversity enables fine-grained\nanalyses of the performance of models on concepts of different types, and\nconsequently greater insight into how architectures can be improved. Further,\nunlike existing gold standard evaluations, for which automatic approaches have\nreached or surpassed the inter-annotator agreement ceiling, state-of-the-art\nmodels perform well below this ceiling on SimLex-999. There is therefore plenty\nof scope for SimLex-999 to quantify future improvements to distributional\nsemantic models, guiding the development of the next generation of\nrepresentation-learning architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.3731v2",
            "date": "2014-08-16 10:09:28+00:00",
            "title": "Unsupervised Keyword Extraction from Polish Legal Texts",
            "authors": [
                "Micha\u0142 Jungiewicz",
                "Micha\u0142 \u0141opuszy\u0144ski"
            ],
            "abstract": "In this work, we present an application of the recently proposed unsupervised\nkeyword extraction algorithm RAKE to a corpus of Polish legal texts from the\nfield of public procurement. RAKE is essentially a language and domain\nindependent method. Its only language-specific input is a stoplist containing a\nset of non-content words. The performance of the method heavily depends on the\nchoice of such a stoplist, which should be domain adopted. Therefore, we\ncomplement RAKE algorithm with an automatic approach to selecting non-content\nwords, which is based on the statistical properties of term distribution.",
            "journal ref": "Lecture Notes in Computer Science, Volume 8686, Springer 2014, pp\n  65-70",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.4753v1",
            "date": "2014-08-20 18:40:13+00:00",
            "title": "Be Careful When Assuming the Obvious: Commentary on \"The placement of the head that minimizes online memory: a complex systems approach\"",
            "authors": [
                "Phillip M. Alday"
            ],
            "abstract": "Ferrer-i-Cancho (2015) presents a mathematical model of both the synchronic\nand diachronic nature of word order based on the assumption that memory costs\nare a never decreasing function of distance and a few very general linguistic\nassumptions. However, even these minimal and seemingly obvious assumptions are\nnot as safe as they appear in light of recent typological and psycholinguistic\nevidence. The interaction of word order and memory has further depths to be\nexplored.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.6179v1",
            "date": "2014-08-26 16:28:21+00:00",
            "title": "Evaluating Neural Word Representations in Tensor-Based Compositional Settings",
            "authors": [
                "Dmitrijs Milajevs",
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh",
                "Matthew Purver"
            ],
            "abstract": "We provide a comparative study between neural word representations and\ntraditional vector spaces based on co-occurrence counts, in a number of\ncompositional tasks. We use three different semantic spaces and implement seven\ntensor-based compositional models, which we then test (together with simpler\nadditive and multiplicative approaches) in tasks involving verb disambiguation\nand sentence similarity. To check their scalability, we additionally evaluate\nthe spaces using simple compositional methods on larger-scale tasks with less\nconstrained language: paraphrase detection and dialogue act tagging. In the\nmore constrained tasks, co-occurrence vectors are competitive, although choice\nof compositional method is important; on the larger-scale tasks, they are\noutperformed by neural word embeddings, which show robust, stable performance\nacross the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.6181v1",
            "date": "2014-08-26 16:43:30+00:00",
            "title": "Resolving Lexical Ambiguity in Tensor Regression Models of Meaning",
            "authors": [
                "Dimitri Kartsaklis",
                "Nal Kalchbrenner",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "This paper provides a method for improving tensor-based compositional\ndistributional models of meaning by the addition of an explicit disambiguation\nstep prior to composition. In contrast with previous research where this\nhypothesis has been successfully tested against relatively simple compositional\nmodels, in our work we use a robust model trained with linear regression. The\nresults we get in two experiments show the superiority of the prior\ndisambiguation method and suggest that the effectiveness of this approach is\nmodel-independent.",
            "journal ref": "Proceedings of ACL 2014, Vol. 2:Short Papers, pp:212-217",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.6788v2",
            "date": "2014-08-28 17:29:55+00:00",
            "title": "Strongly Incremental Repair Detection",
            "authors": [
                "Julian Hough",
                "Matthew Purver"
            ],
            "abstract": "We present STIR (STrongly Incremental Repair detection), a system that\ndetects speech repairs and edit terms on transcripts incrementally with minimal\nlatency. STIR uses information-theoretic measures from n-gram models as its\nprincipal decision features in a pipeline of classifiers detecting the\ndifferent stages of repairs. Results on the Switchboard disfluency tagged\ncorpus show utterance-final accuracy on a par with state-of-the-art incremental\nrepair detection methods, but with better incremental accuracy, faster\ntime-to-detection and less computational overhead. We evaluate its performance\nusing incremental metrics and propose new repair processing evaluation\nstandards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.0588v1",
            "date": "2014-11-03 18:05:54+00:00",
            "title": "On Detecting Noun-Adjective Agreement Errors in Bulgarian Language Using GATE",
            "authors": [
                "Nadezhda Borisova",
                "Grigor Iliev",
                "Elena Karashtranova"
            ],
            "abstract": "In this article, we describe an approach for automatic detection of\nnoun-adjective agreement errors in Bulgarian texts by explaining the necessary\nsteps required to develop a simple Java-based language processing application.\nFor this purpose, we use the GATE language processing framework, which is\ncapable of analyzing texts in Bulgarian language and can be embedded in\nsoftware applications, accessed through a set of Java APIs. In our example\napplication we also demonstrate how to use the functionality of GATE to perform\nregular expressions over annotations for detecting agreement errors in simple\nnoun phrases formed by two words - attributive adjective and a noun, where the\nattributive adjective precedes the noun. The provided code samples can also be\nused as a starting point for implementing natural language processing\nfunctionalities in software applications related to language processing tasks\nlike detection, annotation and retrieval of word groups meeting a specific set\nof criteria.",
            "journal ref": "FMNS2013 (2013) 180-187",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.0778v1",
            "date": "2014-11-04 03:48:20+00:00",
            "title": "Detecting Suicidal Ideation in Chinese Microblogs with Psychological Lexicons",
            "authors": [
                "Xiaolei Huang",
                "Lei Zhang",
                "Tianli Liu",
                "David Chiu",
                "Tingshao Zhu",
                "Xin Li"
            ],
            "abstract": "Suicide is among the leading causes of death in China. However, technical\napproaches toward preventing suicide are challenging and remaining under\ndevelopment. Recently, several actual suicidal cases were preceded by users who\nposted microblogs with suicidal ideation to Sina Weibo, a Chinese social media\nnetwork akin to Twitter. It would therefore be desirable to detect suicidal\nideations from microblogs in real-time, and immediately alert appropriate\nsupport groups, which may lead to successful prevention. In this paper, we\npropose a real-time suicidal ideation detection system deployed over Weibo,\nusing machine learning and known psychological techniques. Currently, we have\nidentified 53 known suicidal cases who posted suicide notes on Weibo prior to\ntheir deaths.We explore linguistic features of these known cases using a\npsychological lexicon dictionary, and train an effective suicidal Weibo post\ndetection model. 6714 tagged posts and several classifiers are used to verify\nthe model. By combining both machine learning and psychological knowledge, SVM\nclassifier has the best performance of different classifiers, yielding an\nF-measure of 68:3%, a Precision of 78:9%, and a Recall of 60:3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.2738v4",
            "date": "2014-11-11 09:24:00+00:00",
            "title": "word2vec Parameter Learning Explained",
            "authors": [
                "Xin Rong"
            ],
            "abstract": "The word2vec model and application by Mikolov et al. have attracted a great\namount of attention in recent two years. The vector representations of words\nlearned by word2vec models have been shown to carry semantic meanings and are\nuseful in various NLP tasks. As an increasing number of researchers would like\nto experiment with word2vec or similar techniques, I notice that there lacks a\nmaterial that comprehensively explains the parameter learning process of word\nembedding models in details, thus preventing researchers that are non-experts\nin neural networks from understanding the working mechanism of such models.\n  This note provides detailed derivations and explanations of the parameter\nupdate equations of the word2vec models, including the original continuous\nbag-of-word (CBOW) and skip-gram (SG) models, as well as advanced optimization\ntechniques, including hierarchical softmax and negative sampling. Intuitive\ninterpretations of the gradient equations are also provided alongside\nmathematical derivations.\n  In the appendix, a review on the basics of neuron networks and\nbackpropagation is provided. I also created an interactive demo, wevi, to\nfacilitate the intuitive understanding of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.3146v1",
            "date": "2014-11-12 11:26:51+00:00",
            "title": "Distributed Representations for Compositional Semantics",
            "authors": [
                "Karl Moritz Hermann"
            ],
            "abstract": "The mathematical representation of semantics is a key issue for Natural\nLanguage Processing (NLP). A lot of research has been devoted to finding ways\nof representing the semantics of individual words in vector spaces.\nDistributional approaches --- meaning distributed representations that exploit\nco-occurrence statistics of large corpora --- have proved popular and\nsuccessful across a number of tasks. However, natural language usually comes in\nstructures beyond the word level, with meaning arising not only from the\nindividual words but also the structure they are contained in at the phrasal or\nsentential level. Modelling the compositional process by which the meaning of\nan utterance arises from the meaning of its parts is an equally fundamental\ntask of NLP.\n  This dissertation explores methods for learning distributed semantic\nrepresentations and models for composing these into representations for larger\nlinguistic units. Our underlying hypothesis is that neural models are a\nsuitable vehicle for learning semantically rich representations and that such\nrepresentations in turn are suitable vehicles for solving important tasks in\nnatural language processing. The contribution of this thesis is a thorough\nevaluation of our hypothesis, as part of which we introduce several new\napproaches to representation learning and compositional semantics, as well as\nmultiple state-of-the-art models which apply distributed semantic\nrepresentations to various tasks in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.3561v1",
            "date": "2014-11-13 14:44:00+00:00",
            "title": "A Text to Speech (TTS) System with English to Punjabi Conversion",
            "authors": [
                "Prabhsimran Singh",
                "Amritpal Singh"
            ],
            "abstract": "The paper aims to show how an application can be developed that converts the\nEnglish language into the Punjabi Language, and the same application can\nconvert the Text to Speech(TTS) i.e. pronounce the text. This application can\nbe really beneficial for those with special needs.",
            "journal ref": "International Journal of Computer and Communication System\n  Engineering, Volume 1, Issue 04, December 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4166v4",
            "date": "2014-11-15 17:34:20+00:00",
            "title": "Retrofitting Word Vectors to Semantic Lexicons",
            "authors": [
                "Manaal Faruqui",
                "Jesse Dodge",
                "Sujay K. Jauhar",
                "Chris Dyer",
                "Eduard Hovy",
                "Noah A. Smith"
            ],
            "abstract": "Vector space word representations are learned from distributional information\nof words in large corpora. Although such statistics are semantically\ninformative, they disregard the valuable information that is contained in\nsemantic lexicons such as WordNet, FrameNet, and the Paraphrase Database. This\npaper proposes a method for refining vector space representations using\nrelational information from semantic lexicons by encouraging linked words to\nhave similar vector representations, and it makes no assumptions about how the\ninput vectors were constructed. Evaluated on a battery of standard lexical\nsemantic evaluation tasks in several languages, we obtain substantial\nimprovements starting with a variety of word vector models. Our refinement\nmethod outperforms prior techniques for incorporating semantic lexicons into\nthe word vector training algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4472v1",
            "date": "2014-11-17 13:36:49+00:00",
            "title": "Opinion mining of text documents written in Macedonian language",
            "authors": [
                "Andrej Gajduk",
                "Ljupco Kocarev"
            ],
            "abstract": "The ability to extract public opinion from web portals such as review sites,\nsocial networks and blogs will enable companies and individuals to form a view,\nan attitude and make decisions without having to do lengthy and costly\nresearches and surveys. In this paper machine learning techniques are used for\ndetermining the polarity of forum posts on kajgana which are written in\nMacedonian language. The posts are classified as being positive, negative or\nneutral. We test different feature metrics and classifiers and provide detailed\nevaluation of their participation in improving the overall performance on a\nmanually generated dataset. By achieving 92% accuracy, we show that the\nperformance of systems for automated opinion mining is comparable to a human\nevaluator, thus making it a viable option for text data analysis. Finally, we\npresent a few statistics derived from the forum posts using the developed\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4614v1",
            "date": "2014-09-26 05:09:40+00:00",
            "title": "Using graph transformation algorithms to generate natural language equivalents of icons expressing medical concepts",
            "authors": [
                "Pascal Vaillant",
                "Jean-Baptiste Lamy"
            ],
            "abstract": "A graphical language addresses the need to communicate medical information in\na synthetic way. Medical concepts are expressed by icons conveying fast visual\ninformation about patients' current state or about the known effects of drugs.\nIn order to increase the visual language's acceptance and usability, a natural\nlanguage generation interface is currently developed. In this context, this\npaper describes the use of an informatics method ---graph transformation--- to\nprepare data consisting of concepts in an OWL-DL ontology for use in a natural\nlanguage generation component. The OWL concept may be considered as a\nstar-shaped graph with a central node. The method transforms it into a graph\nrepresenting the deep semantic structure of a natural language phrase. This\nwork may be of future use in other contexts where ontology concepts have to be\nmapped to half-formalized natural language expressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4960v1",
            "date": "2014-11-18 18:46:36+00:00",
            "title": "Network Motifs Analysis of Croatian Literature",
            "authors": [
                "Hana Rizvi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107",
                "Ana Me\u0161trovi\u0107"
            ],
            "abstract": "In this paper we analyse network motifs in the co-occurrence directed\nnetworks constructed from five different texts (four books and one portal) in\nthe Croatian language. After preparing the data and network construction, we\nperform the network motif analysis. We analyse the motif frequencies and\nZ-scores in the five networks. We present the triad significance profile for\nfive datasets. Furthermore, we compare our results with the existing results\nfor the linguistic networks. Firstly, we show that the triad significance\nprofile for the Croatian language is very similar with the other languages and\nall the networks belong to the same family of networks. However, there are\ncertain differences between the Croatian language and other analysed languages.\nWe conclude that this is due to the free word-order of the Croatian language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.5379v3",
            "date": "2014-11-19 21:06:15+00:00",
            "title": "Type-Driven Incremental Semantic Parsing with Polymorphism",
            "authors": [
                "Kai Zhao",
                "Liang Huang"
            ],
            "abstract": "Semantic parsing has made significant progress, but most current semantic\nparsers are extremely slow (CKY-based) and rather primitive in representation.\nWe introduce three new techniques to tackle these problems. First, we design\nthe first linear-time incremental shift-reduce-style semantic parsing algorithm\nwhich is more efficient than conventional cubic-time bottom-up semantic\nparsers. Second, our parser, being type-driven instead of syntax-driven, uses\ntype-checking to decide the direction of reduction, which eliminates the need\nfor a syntactic grammar such as CCG. Third, to fully exploit the power of\ntype-driven semantic parsing beyond simple types (such as entities and truth\nvalues), we borrow from programming language theory the concepts of subtype\npolymorphism and parametric polymorphism to enrich the type system in order to\nbetter guide the parsing. Our system learns very accurate parses in GeoQuery,\nJobs and Atis domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.5796v1",
            "date": "2014-11-21 08:50:30+00:00",
            "title": "Pre-processing of Domain Ontology Graph Generation System in Punjabi",
            "authors": [
                "Rajveer Kaur",
                "Saurabh Sharma"
            ],
            "abstract": "This paper describes pre-processing phase of ontology graph generation system\nfrom Punjabi text documents of different domains. This research paper focuses\non pre-processing of Punjabi text documents. Pre-processing is structured\nrepresentation of the input text. Pre-processing of ontology graph generation\nincludes allowing input restrictions to the text, removal of special symbols\nand punctuation marks, removal of duplicate terms, removal of stop words,\nextract terms by matching input terms with dictionary and gazetteer lists\nterms.",
            "journal ref": "International Journal of Engineering Trends and Technology\n  (IJETT), V17(3),141-146, Nov 2014. Published by Seventh Sense Research Group",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.7820v1",
            "date": "2014-11-28 11:33:02+00:00",
            "title": "Coarse-grained Cross-lingual Alignment of Comparable Texts with Topic Models and Encyclopedic Knowledge",
            "authors": [
                "Vivi Nastase",
                "Angela Fahrni"
            ],
            "abstract": "We present a method for coarse-grained cross-lingual alignment of comparable\ntexts: segments consisting of contiguous paragraphs that discuss the same theme\n(e.g. history, economy) are aligned based on induced multilingual topics. The\nmethod combines three ideas: a two-level LDA model that filters out words that\ndo not convey themes, an HMM that models the ordering of themes in the\ncollection of documents, and language-independent concept annotations to serve\nas a cross-language bridge and to strengthen the connection between paragraphs\nin the same segment through concept relations. The method is evaluated on\nEnglish and French data previously used for monolingual alignment. The results\nshow state-of-the-art performance in both monolingual and cross-lingual\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.7942v2",
            "date": "2014-11-28 16:57:34+00:00",
            "title": "Using Sentence Plausibility to Learn the Semantics of Transitive Verbs",
            "authors": [
                "Tamara Polajnar",
                "Laura Rimell",
                "Stephen Clark"
            ],
            "abstract": "The functional approach to compositional distributional semantics considers\ntransitive verbs to be linear maps that transform the distributional vectors\nrepresenting nouns into a vector representing a sentence. We conduct an initial\ninvestigation that uses a matrix consisting of the parameters of a logistic\nregression classifier trained on a plausibility task as a transitive verb\nfunction. We compare our method to a commonly used corpus-based method for\nconstructing a verb matrix and find that the plausibility training may be more\neffective for disambiguation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.1847v1",
            "date": "2014-04-07 16:45:42+00:00",
            "title": "Evaluation and Ranking of Machine Translated Output in Hindi Language using Precision and Recall Oriented Metrics",
            "authors": [
                "Aditi Kalyani",
                "Hemant Kumud",
                "Shashi Pal Singh",
                "Ajai Kumar",
                "Hemant Darbari"
            ],
            "abstract": "Evaluation plays a crucial role in development of Machine translation\nsystems. In order to judge the quality of an existing MT system i.e. if the\ntranslated output is of human translation quality or not, various automatic\nmetrics exist. We here present the implementation results of different metrics\nwhen used on Hindi language along with their comparisons, illustrating how\neffective are these metrics on languages like Hindi (free word order language).",
            "journal ref": "International Journal of Advanced Computer Research, Volume-4\n  Number-1 Issue-14 March 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.1872v1",
            "date": "2014-04-07 18:12:08+00:00",
            "title": "Int\u00e9gration des donn\u00e9es d'un lexique syntaxique dans un analyseur syntaxique probabiliste",
            "authors": [
                "Anthony Sigogne",
                "Matthieu Constant",
                "Eric Laporte"
            ],
            "abstract": "This article reports the evaluation of the integration of data from a\nsyntactic-semantic lexicon, the Lexicon-Grammar of French, into a syntactic\nparser. We show that by changing the set of labels for verbs and predicational\nnouns, we can improve the performance on French of a non-lexicalized\nprobabilistic parser.",
            "journal ref": "Penser le Lexique-Grammaire. Perspectives actuelles, Fryni\n  Kakoyianni-Doa (Ed.) (2014) 505-516",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.2071v1",
            "date": "2014-04-08 10:08:22+00:00",
            "title": "Extracting a bilingual semantic grammar from FrameNet-annotated corpora",
            "authors": [
                "Dana Dann\u00e9lls",
                "Normunds Gr\u016bz\u012btis"
            ],
            "abstract": "We present the creation of an English-Swedish FrameNet-based grammar in\nGrammatical Framework. The aim of this research is to make existing framenets\ncomputationally accessible for multilingual natural language applications via a\ncommon semantic grammar API, and to facilitate the porting of such grammar to\nother languages. In this paper, we describe the abstract syntax of the semantic\ngrammar while focusing on its automatic extraction possibilities. We have\nextracted a shared abstract syntax from ~58,500 annotated sentences in Berkeley\nFrameNet (BFN) and ~3,500 annotated sentences in Swedish FrameNet (SweFN). The\nabstract syntax defines 769 frame-specific valence patterns that cover 77.8%\nexamples in BFN and 74.9% in SweFN belonging to the shared set of 471 frames.\nAs a side result, we provide a unified method for comparing semantic and\nsyntactic valence patterns across framenets.",
            "journal ref": "Proceedings of the 9th International Conference on Language\n  Resources and Evaluation (LREC), 2014, pp. 2466-2473",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.2188v1",
            "date": "2014-04-08 15:46:44+00:00",
            "title": "A Convolutional Neural Network for Modelling Sentences",
            "authors": [
                "Nal Kalchbrenner",
                "Edward Grefenstette",
                "Phil Blunsom"
            ],
            "abstract": "The ability to accurately represent sentences is central to language\nunderstanding. We describe a convolutional architecture dubbed the Dynamic\nConvolutional Neural Network (DCNN) that we adopt for the semantic modelling of\nsentences. The network uses Dynamic k-Max Pooling, a global pooling operation\nover linear sequences. The network handles input sentences of varying length\nand induces a feature graph over the sentence that is capable of explicitly\ncapturing short and long-range relations. The network does not rely on a parse\ntree and is easily applicable to any language. We test the DCNN in four\nexperiments: small scale binary and multi-class sentiment prediction, six-way\nquestion classification and Twitter sentiment prediction by distant\nsupervision. The network achieves excellent performance in the first three\ntasks and a greater than 25% error reduction in the last task with respect to\nthe strongest baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.2878v1",
            "date": "2014-04-10 17:16:01+00:00",
            "title": "Overview of Stemming Algorithms for Indian and Non-Indian Languages",
            "authors": [
                "Dalwadi Bijal",
                "Suthar Sanket"
            ],
            "abstract": "Stemming is a pre-processing step in Text Mining applications as well as a\nvery common requirement of Natural Language processing functions. Stemming is\nthe process for reducing inflected words to their stem. The main purpose of\nstemming is to reduce different grammatical forms / word forms of a word like\nits noun, adjective, verb, adverb etc. to its root form. Stemming is widely\nuses in Information Retrieval system and reduces the size of index files. We\ncan say that the goal of stemming is to reduce inflectional forms and sometimes\nderivationally related forms of a word to a common base form. In this paper we\nhave discussed different stemming algorithm for non-Indian and Indian language,\nmethods of stemming, accuracy and errors.",
            "journal ref": "International Journal of Computer Science and Information\n  Technologies, Vol. 5 (2) , 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3377v1",
            "date": "2014-04-13 12:39:41+00:00",
            "title": "A Generalized Language Model as the Combination of Skipped n-grams and Modified Kneser-Ney Smoothing",
            "authors": [
                "Rene Pickhardt",
                "Thomas Gottron",
                "Martin K\u00f6rner",
                "Paul Georg Wagner",
                "Till Speicher",
                "Steffen Staab"
            ],
            "abstract": "We introduce a novel approach for building language models based on a\nsystematic, recursive exploration of skip n-gram models which are interpolated\nusing modified Kneser-Ney smoothing. Our approach generalizes language models\nas it contains the classical interpolation with lower order models as a special\ncase. In this paper we motivate, formalize and present our approach. In an\nextensive empirical experiment over English text corpora we demonstrate that\nour generalized language models lead to a substantial reduction of perplexity\nbetween 3.1% and 12.7% in comparison to traditional language models using\nmodified Kneser-Ney smoothing. Furthermore, we investigate the behaviour over\nthree other languages and a domain specific corpus where we observed consistent\nimprovements. Finally, we also show that the strength of our approach lies in\nits ability to cope in particular with sparse training data. Using a very small\ntraining data set of only 736 KB text we yield improvements of even 25.7%\nreduction of perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3759v1",
            "date": "2014-04-14 21:33:42+00:00",
            "title": "Meta-evaluation of comparability metrics using parallel corpora",
            "authors": [
                "Bogdan Babych",
                "Anthony Hartley"
            ],
            "abstract": "Metrics for measuring the comparability of corpora or texts need to be\ndeveloped and evaluated systematically. Applications based on a corpus, such as\ntraining Statistical MT systems in specialised narrow domains, require finding\na reasonable balance between the size of the corpus and its consistency, with\ncontrolled and benchmarked levels of comparability for any newly added\nsections. In this article we propose a method that can meta-evaluate\ncomparability metrics by calculating monolingual comparability scores\nseparately on the 'source' and 'target' sides of parallel corpora. The range of\nscores on the source side is then correlated (using Pearson's r coefficient)\nwith the range of 'target' scores; the higher the correlation - the more\nreliable is the metric. The intuition is that a good metric should yield the\nsame distance between different domains in different languages. Our method\ngives consistent results for the same metrics on different data sets, which\nindicates that it is reliable and can be used for metric comparison or for\noptimising settings of parametrised metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3992v1",
            "date": "2014-04-15 17:13:26+00:00",
            "title": "Assessing the Quality of MT Systems for Hindi to English Translation",
            "authors": [
                "Aditi Kalyani",
                "Hemant Kumud",
                "Shashi Pal Singh",
                "Ajai Kumar"
            ],
            "abstract": "Evaluation plays a vital role in checking the quality of MT output. It is\ndone either manually or automatically. Manual evaluation is very time consuming\nand subjective, hence use of automatic metrics is done most of the times. This\npaper evaluates the translation quality of different MT Engines for\nHindi-English (Hindi data is provided as input and English is obtained as\noutput) using various automatic metrics like BLEU, METEOR etc. Further the\ncomparison automatic evaluation results with Human ranking have also been\ngiven.",
            "journal ref": "International Journal of Computer Applications, Volume 89, No 15,\n  March 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4314v1",
            "date": "2014-04-16 17:06:35+00:00",
            "title": "An Empirical Comparison of Parsing Methods for Stanford Dependencies",
            "authors": [
                "Lingpeng Kong",
                "Noah A. Smith"
            ],
            "abstract": "Stanford typed dependencies are a widely desired representation of natural\nlanguage sentences, but parsing is one of the major computational bottlenecks\nin text analysis systems. In light of the evolving definition of the Stanford\ndependencies and developments in statistical dependency parsing algorithms,\nthis paper revisits the question of Cer et al. (2010): what is the tradeoff\nbetween accuracy and speed in obtaining Stanford dependencies in particular? We\nalso explore the effects of input representations on this tradeoff:\npart-of-speech tags, the novel use of an alternative dependency representation\nas input, and distributional representaions of words. We find that direct\ndependency parsing is a more viable solution than it was found to be in the\npast. An accompanying software release can be found at:\nhttp://www.ark.cs.cmu.edu/TBSD",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4572v1",
            "date": "2014-04-17 16:22:40+00:00",
            "title": "The First Parallel Multilingual Corpus of Persian: Toward a Persian BLARK",
            "authors": [
                "Behrang Qasemizadeh",
                "Saeed Rahimi",
                "Behrooz Mahmoodi Bakhtiari"
            ],
            "abstract": "In this article, we have introduced the first parallel corpus of Persian with\nmore than 10 other European languages. This article describes primary steps\ntoward preparing a Basic Language Resources Kit (BLARK) for Persian. Up to now,\nwe have proposed morphosyntactic specification of Persian based on\nEAGLE/MULTEXT guidelines and specific resources of MULTEXT-East. The article\nintroduces Persian Language, with emphasis on its orthography and\nmorphosyntactic features, then a new Part-of-Speech categorization and\northography for Persian in digital environments is proposed. Finally, the\ncorpus and related statistic will be analyzed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4641v1",
            "date": "2014-04-17 20:18:03+00:00",
            "title": "Multilingual Models for Compositional Distributed Semantics",
            "authors": [
                "Karl Moritz Hermann",
                "Phil Blunsom"
            ],
            "abstract": "We present a novel technique for learning semantic representations, which\nextends the distributional hypothesis to multilingual data and joint-space\nembeddings. Our models leverage parallel data and learn to strongly align the\nembeddings of semantically equivalent sentences, while maintaining sufficient\ndistance between those of dissimilar sentences. The models do not rely on word\nalignments or any syntactic information and are successfully applied to a\nnumber of diverse languages. We extend our approach to learn semantic\nrepresentations at the document level, too. We evaluate these models on two\ncross-lingual document classification tasks, outperforming the prior state of\nthe art. Through qualitative analysis and the study of pivoting effects we\ndemonstrate that our representations are semantically plausible and can capture\nsemantic relationships across languages without parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4714v1",
            "date": "2014-04-18 07:48:02+00:00",
            "title": "Radical-Enhanced Chinese Character Embedding",
            "authors": [
                "Yaming Sun",
                "Lei Lin",
                "Duyu Tang",
                "Nan Yang",
                "Zhenzhou Ji",
                "Xiaolong Wang"
            ],
            "abstract": "We present a method to leverage radical for learning Chinese character\nembedding. Radical is a semantic and phonetic component of Chinese character.\nIt plays an important role as characters with the same radical usually have\nsimilar semantic meaning and grammatical usage. However, existing Chinese\nprocessing algorithms typically regard word or character as the basic unit but\nignore the crucial radical information. In this paper, we fill this gap by\nleveraging radical for learning continuous representation of Chinese character.\nWe develop a dedicated neural architecture to effectively learn character\nembedding and apply it on Chinese character similarity judgement and Chinese\nword segmentation. Experiment results show that our radical-enhanced method\noutperforms existing embedding learning algorithms on both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4740v1",
            "date": "2014-04-18 10:30:47+00:00",
            "title": "Challenges in Persian Electronic Text Analysis",
            "authors": [
                "Behrang QasemiZadeh",
                "Saeed Rahimi",
                "Mehdi Safaee Ghalati"
            ],
            "abstract": "Farsi, also known as Persian, is the official language of Iran and Tajikistan\nand one of the two main languages spoken in Afghanistan. Farsi enjoys a unified\nArabic script as its writing system. In this paper we briefly introduce the\nwriting standards of Farsi and highlight problems one would face when analyzing\nFarsi electronic texts, especially during development of Farsi corpora\nregarding to transcription and encoding of Farsi e-texts. The pointes mentioned\nmay sounds easy but they are crucial when developing and processing written\ncorpora of Farsi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.5278v1",
            "date": "2014-04-21 19:31:48+00:00",
            "title": "The Frobenius anatomy of word meanings I: subject and object relative pronouns",
            "authors": [
                "Mehrnoosh Sadrzadeh",
                "Stephen Clark",
                "Bob Coecke"
            ],
            "abstract": "This paper develops a compositional vector-based semantics of subject and\nobject relative pronouns within a categorical framework. Frobenius algebras are\nused to formalise the operations required to model the semantics of relative\npronouns, including passing information between the relative clause and the\nmodified noun phrase, as well as copying, combining, and discarding parts of\nthe relative clause. We develop two instantiations of the abstract semantics,\none based on a truth-theoretic approach and one based on corpus statistics.",
            "journal ref": "Journal of Logic and Computation, Special Issue: The Incomputable,\n  an Isaac Newton Institute Workshop, 23(6), pp.1293-1317, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.5357v1",
            "date": "2014-04-22 00:17:26+00:00",
            "title": "Morphological Analysis of the Bishnupriya Manipuri Language using Finite State Transducers",
            "authors": [
                "Nayan Jyoti Kalita",
                "Navanath Saharia",
                "Smriti Kumar Sinha"
            ],
            "abstract": "In this work we present a morphological analysis of Bishnupriya Manipuri\nlanguage, an Indo-Aryan language spoken in the north eastern India. As of now,\nthere is no computational work available for the language. Finite state\nmorphology is one of the successful approaches applied in a wide variety of\nlanguages over the year. Therefore we adapted the finite state approach to\nanalyse morphology of the Bishnupriya Manipuri language.",
            "journal ref": "Computational Linguistics and Intelligent Text Processing,\n  vol.8403, series: Lecture Notes in Computer Science, pp 206-213, (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.5367v1",
            "date": "2014-04-22 02:12:06+00:00",
            "title": "Lexicon Infused Phrase Embeddings for Named Entity Resolution",
            "authors": [
                "Alexandre Passos",
                "Vineet Kumar",
                "Andrew McCallum"
            ],
            "abstract": "Most state-of-the-art approaches for named-entity recognition (NER) use semi\nsupervised information in the form of word clusters and lexicons. Recently\nneural network-based language models have been explored, as they as a byproduct\ngenerate highly informative vector representations for words, known as word\nembeddings. In this paper we present two contributions: a new form of learning\nword embeddings that can leverage information from relevant lexicons to improve\nthe representations, and the first system to use neural word embeddings to\nachieve state-of-the-art results on named-entity recognition in both CoNLL and\nOntonotes NER. Our system achieves an F1 score of 90.90 on the test set for\nCoNLL 2003---significantly better than any previous system trained on public\ndata, and matching a system employing massive private industrial query-log\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.6312v2",
            "date": "2014-04-25 04:10:57+00:00",
            "title": "Reconstructing Native Language Typology from Foreign Language Usage",
            "authors": [
                "Yevgeni Berzak",
                "Roi Reichart",
                "Boris Katz"
            ],
            "abstract": "Linguists and psychologists have long been studying cross-linguistic\ntransfer, the influence of native language properties on linguistic performance\nin a foreign language. In this work we provide empirical evidence for this\nprocess in the form of a strong correlation between language similarities\nderived from structural features in English as Second Language (ESL) texts and\nequivalent similarities obtained from the typological features of the native\nlanguages. We leverage this finding to recover native language typological\nsimilarity structure directly from ESL text, and perform prediction of\ntypological features in an unsupervised fashion with respect to the target\nlanguages. Our method achieves 72.2% accuracy on the typology prediction task,\na result that is highly competitive with equivalent methods that rely on\ntypological resources.",
            "journal ref": "Proceedings of the Eighteenth Conference on Computational Language\n  Learning , pages 21-29, Baltimore, Maryland USA, June 26-27 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.7296v1",
            "date": "2014-04-29 10:10:13+00:00",
            "title": "A Deep Architecture for Semantic Parsing",
            "authors": [
                "Edward Grefenstette",
                "Phil Blunsom",
                "Nando de Freitas",
                "Karl Moritz Hermann"
            ],
            "abstract": "Many successful approaches to semantic parsing build on top of the syntactic\nanalysis of text, and make use of distributional representations or statistical\nmodels to match parses to ontology-specific queries. This paper presents a\nnovel deep learning architecture which provides a semantic parsing system\nthrough the union of two neural models of language semantics. It allows for the\ngeneration of ontology-specific queries from natural language statements and\nquestions without the need for parsing, which makes it especially suitable to\ngrammatically malformed or syntactically atypical text, such as tweets, as well\nas permitting the development of semantic parsers for resource-poor languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.0032v1",
            "date": "2014-05-30 22:47:49+00:00",
            "title": "Comparing and Combining Sentiment Analysis Methods",
            "authors": [
                "Pollyanna Gon\u00e7alves",
                "Matheus Ara\u00fajo",
                "Fabr\u00edcio Benevenuto",
                "Meeyoung Cha"
            ],
            "abstract": "Several messages express opinions about events, products, and services,\npolitical views or even their author's emotional state and mood. Sentiment\nanalysis has been used in several applications including analysis of the\nrepercussions of events in social networks, analysis of opinions about products\nand services, and simply to better understand aspects of social communication\nin Online Social Networks (OSNs). There are multiple methods for measuring\nsentiments, including lexical-based approaches and supervised machine learning\nmethods. Despite the wide use and popularity of some methods, it is unclear\nwhich method is better for identifying the polarity (i.e., positive or\nnegative) of a message as the current literature does not provide a method of\ncomparison among existing methods. Such a comparison is crucial for\nunderstanding the potential limitations, advantages, and disadvantages of\npopular methods in analyzing the content of OSNs messages. Our study aims at\nfilling this gap by presenting comparisons of eight popular sentiment analysis\nmethods in terms of coverage (i.e., the fraction of messages whose sentiment is\nidentified) and agreement (i.e., the fraction of identified sentiments that are\nin tune with ground truth). We develop a new method that combines existing\napproaches, providing the best coverage results and competitive agreement. We\nalso present a free Web service called iFeel, which provides an open API for\naccessing and comparing results across different sentiment methods for a given\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1203v1",
            "date": "2014-06-04 20:22:30+00:00",
            "title": "A Semantic Approach to Summarization",
            "authors": [
                "Divyanshu Bhartiya",
                "Ashudeep Singh"
            ],
            "abstract": "Sentence extraction based summarization methods has some limitations as it\ndoesn't go into the semantics of the document. Also, it lacks the capability of\nsentence generation which is intuitive to humans. Here we present a novel\nmethod to summarize text documents taking the process to semantic levels with\nthe use of WordNet and other resources, and using a technique for sentence\ngeneration. We involve semantic role labeling to get the semantic\nrepresentation of text and use of segmentation to form clusters of the related\npieces of text. Picking out the centroids and sentence generation completes the\ntask. We evaluate our system against human composed summaries and also present\nan evaluation done by humans to measure the quality attributes of our\nsummaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1241v1",
            "date": "2014-06-04 23:56:08+00:00",
            "title": "The Best Templates Match Technique For Example Based Machine Translation",
            "authors": [
                "T. El-Shishtawy",
                "A. El-Sammak"
            ],
            "abstract": "It has been proved that large scale realistic Knowledge Based Machine\nTranslation applications require acquisition of huge knowledge about language\nand about the world. This knowledge is encoded in computational grammars,\nlexicons and domain models. Another approach which avoids the need for\ncollecting and analyzing massive knowledge, is the Example Based approach,\nwhich is the topic of this paper. We show through the paper that using Example\nBased in its native form is not suitable for translating into Arabic. Therefore\na modification to the basic approach is presented to improve the accuracy of\nthe translation process. The basic idea of the new approach is to improve the\ntechnique by which template-based approaches select the appropriate templates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1280v1",
            "date": "2014-06-05 07:07:44+00:00",
            "title": "Basis Identification for Automatic Creation of Pronunciation Lexicon for Proper Names",
            "authors": [
                "Sunil Kumar Kopparapu",
                "M Laxminarayana"
            ],
            "abstract": "Development of a proper names pronunciation lexicon is usually a manual\neffort which can not be avoided. Grapheme to phoneme (G2P) conversion modules,\nin literature, are usually rule based and work best for non-proper names in a\nparticular language. Proper names are foreign to a G2P module. We follow an\noptimization approach to enable automatic construction of proper names\npronunciation lexicon. The idea is to construct a small orthogonal set of words\n(basis) which can span the set of names in a given database. We propose two\nalgorithms for the construction of this basis. The transcription lexicon of all\nthe proper names in a database can be produced by the manual transcription of\nonly the small set of basis words. We first construct a cost function and show\nthat the minimization of the cost function results in a basis. We derive\nconditions for convergence of this cost function and validate them\nexperimentally on a very large proper name database. Experiments show the\ntranscription can be achieved by transcribing a set of small number of basis\nwords. The algorithms proposed are generic and independent of language; however\nperformance is better if the proper names have same origin, namely, same\nlanguage or geographical region.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1870v1",
            "date": "2014-06-07 07:41:54+00:00",
            "title": "Toward verbalizing ontologies in isiZulu",
            "authors": [
                "C. Maria Keet",
                "Langa Khumalo"
            ],
            "abstract": "IsiZulu is one of the eleven official languages of South Africa and roughly\nhalf the population can speak it. It is the first (home) language for over 10\nmillion people in South Africa. Only a few computational resources exist for\nisiZulu and its related Nguni languages, yet the imperative for tool\ndevelopment exists. We focus on natural language generation, and the grammar\noptions and preferences in particular, which will inform verbalization of\nknowledge representation languages and could contribute to machine translation.\nThe verbalization pattern specification shows that the grammar rules are\nelaborate and there are several options of which one may have preference. We\ndevised verbalization patterns for subsumption, basic disjointness, existential\nand universal quantification, and conjunction. This was evaluated in a survey\namong linguists and non-linguists. Some differences between linguists and\nnon-linguists can be observed, with the former much more in agreement, and\npreferences depend on the overall structure of the sentence, such as singular\nfor subsumption and plural in other cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2204v1",
            "date": "2014-06-09 14:54:22+00:00",
            "title": "How Easy is it to Learn a Controlled Natural Language for Building a Knowledge Base?",
            "authors": [
                "Sandra Williams",
                "Richard Power",
                "Allan Third"
            ],
            "abstract": "Recent developments in controlled natural language editors for knowledge\nengineering (KE) have given rise to expectations that they will make KE tasks\nmore accessible and perhaps even enable non-engineers to build knowledge bases.\nThis exploratory research focussed on novices and experts in knowledge\nengineering during their attempts to learn a controlled natural language (CNL)\nknown as OWL Simplified English and use it to build a small knowledge base.\nParticipants' behaviours during the task were observed through eye-tracking and\nscreen recordings. This was an attempt at a more ambitious user study than in\nprevious research because we used a naturally occurring text as the source of\ndomain knowledge, and left them without guidance on which information to\nselect, or how to encode it. We have identified a number of skills\n(competencies) required for this difficult task and key problems that authors\nface.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2400v1",
            "date": "2014-06-10 01:01:48+00:00",
            "title": "Controlled Natural Language Generation from a Multilingual FrameNet-based Grammar",
            "authors": [
                "Dana Dann\u00e9lls",
                "Normunds Gr\u016bz\u012btis"
            ],
            "abstract": "This paper presents a currently bilingual but potentially multilingual\nFrameNet-based grammar library implemented in Grammatical Framework. The\ncontribution of this paper is two-fold. First, it offers a methodological\napproach to automatically generate the grammar based on semantico-syntactic\nvalence patterns extracted from FrameNet-annotated corpora. Second, it provides\na proof of concept for two use cases illustrating how the acquired multilingual\ngrammar can be exploited in different CNL applications in the domains of arts\nand tourism.",
            "journal ref": "Controlled Natural Language, Lecture Notes in Computer Science,\n  Vol. 8625, Springer, 2014, pp. 155-166",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2903v2",
            "date": "2014-06-11 13:47:22+00:00",
            "title": "A Brief State of the Art for Ontology Authoring",
            "authors": [
                "Hazem Safwat",
                "Brian Davis"
            ],
            "abstract": "One of the main challenges for building the Semantic web is Ontology\nAuthoring. Controlled Natural Languages CNLs offer a user friendly means for\nnon-experts to author ontologies. This paper provides a snapshot of the\nstate-of-the-art for the core CNLs for ontology authoring and reviews their\nrespective evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3460v1",
            "date": "2014-06-13 09:23:53+00:00",
            "title": "Are Style Guides Controlled Languages? The Case of Koenig & Bauer AG",
            "authors": [
                "Karolina Suchowolec"
            ],
            "abstract": "Controlled natural languages for industrial application are often regarded as\na response to the challenges of translation and multilingual communication.\nThis paper presents a quite different approach taken by Koenig & Bauer AG,\nwhere the main goal was the improvement of the authoring process for technical\ndocumentation. Most importantly, this paper explores the notion of a controlled\nlanguage and demonstrates how style guides can emerge from non-linguistic\nconsiderations. Moreover, it shows the transition from loose language\nrecommendations into precise and prescriptive rules and investigates whether\nsuch rules can be regarded as a full-fledged controlled language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3676v3",
            "date": "2014-06-14 03:00:23+00:00",
            "title": "Question Answering with Subgraph Embeddings",
            "authors": [
                "Antoine Bordes",
                "Sumit Chopra",
                "Jason Weston"
            ],
            "abstract": "This paper presents a system which learns to answer questions on a broad\nrange of topics from a knowledge base using few hand-crafted features. Our\nmodel learns low-dimensional embeddings of words and knowledge base\nconstituents; these representations are used to score natural language\nquestions against candidate answers. Training our system using pairs of\nquestions and structured representations of their answers, and pairs of\nquestion paraphrases, yields competitive results on a competitive benchmark of\nthe literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3969v1",
            "date": "2014-06-16 10:59:03+00:00",
            "title": "Translation Of Telugu-Marathi and Vice-Versa using Rule Based Machine Translation",
            "authors": [
                "Siddhartha Ghosh",
                "Sujata Thamke",
                "Kalyani U. R. S"
            ],
            "abstract": "In todays digital world automated Machine Translation of one language to\nanother has covered a long way to achieve different kinds of success stories.\nWhereas Babel Fish supports a good number of foreign languages and only Hindi\nfrom Indian languages, the Google Translator takes care of about 10 Indian\nlanguages. Though most of the Automated Machine Translation Systems are doing\nwell but handling Indian languages needs a major care while handling the local\nproverbs/ idioms. Most of the Machine Translation system follows the direct\ntranslation approach while translating one Indian language to other. Our\nresearch at KMIT R&D Lab found that handling the local proverbs/idioms is not\ngiven enough attention by the earlier research work. This paper focuses on two\nof the majorly spoken Indian languages Marathi and Telugu, and translation\nbetween them. Handling proverbs and idioms of both the languages have been\ngiven a special care, and the research outcome shows a significant achievement\nin this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3976v1",
            "date": "2014-06-16 11:30:51+00:00",
            "title": "Handling non-compositionality in multilingual CNLs",
            "authors": [
                "Ramona Enache",
                "Inari Listenmaa",
                "Prasanth Kolachina"
            ],
            "abstract": "In this paper, we describe methods for handling multilingual\nnon-compositional constructions in the framework of GF. We specifically look at\nmethods to detect and extract non-compositional phrases from parallel texts and\npropose methods to handle such constructions in GF grammars. We expect that the\nmethods to handle non-compositional constructions will enrich CNLs by providing\nmore flexibility in the design of controlled languages. We look at two specific\nuse cases of non-compositional constructions: a general-purpose method to\ndetect and extract multilingual multiword expressions and a procedure to\nidentify nominal compounds in German. We evaluate our procedure for multiword\nexpressions by performing a qualitative analysis of the results. For the\nexperiments on nominal compounds, we incorporate the detected compounds in a\nfull SMT pipeline and evaluate the impact of our method in machine translation\nprocess.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3987v1",
            "date": "2014-06-16 12:03:49+00:00",
            "title": "Towards an Error Correction Memory to Enhance Technical Texts Authoring in LELIE",
            "authors": [
                "Juyeon Kang",
                "Patrick Saint Dizier"
            ],
            "abstract": "In this paper, we investigate and experiment the notion of error correction\nmemory applied to error correction in technical texts. The main purpose is to\ninduce relatively generic correction patterns associated with more contextual\ncorrection recommendations, based on previously memorized and analyzed\ncorrections. The notion of error correction memory is developed within the\nframework of the LELIE project and illustrated on the case of fuzzy lexical\nitems, which is a major problem in technical texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.4057v1",
            "date": "2014-06-16 16:11:32+00:00",
            "title": "Embedded Controlled Languages",
            "authors": [
                "Aarne Ranta"
            ],
            "abstract": "Inspired by embedded programming languages, an embedded CNL (controlled\nnatural language) is a proper fragment of an entire natural language (its host\nlanguage), but it has a parser that recognizes the entire host language. This\nmakes it possible to process out-of-CNL input and give useful feedback to\nusers, instead of just reporting syntax errors. This extended abstract explains\nthe main concepts of embedded CNL implementation in GF (Grammatical Framework),\nwith examples from machine translation and some other ongoing work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.4211v1",
            "date": "2014-06-17 01:34:22+00:00",
            "title": "Mapping the Economic Crisis: Some Preliminary Investigations",
            "authors": [
                "Pierre Bourreau",
                "Thierry Poibeau"
            ],
            "abstract": "In this paper we describe our contribution to the PoliInformatics 2014\nChallenge on the 2007-2008 financial crisis. We propose a state of the art\ntechnique to extract information from texts and provide different\nrepresentations, giving first a static overview of the domain and then a\ndynamic representation of its main evolutions. We show that this strategy\nprovides a practical solution to some recent theories in social sciences that\nare facing a lack of methods and tools to automatically extract information\nfrom natural language texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.5598v1",
            "date": "2014-06-21 11:30:05+00:00",
            "title": "A survey on phrase structure learning methods for text classification",
            "authors": [
                "Reshma Prasad",
                "Mary Priya Sebastian"
            ],
            "abstract": "Text classification is a task of automatic classification of text into one of\nthe predefined categories. The problem of text classification has been widely\nstudied in different communities like natural language processing, data mining\nand information retrieval. Text classification is an important constituent in\nmany information management tasks like topic identification, spam filtering,\nemail routing, language identification, genre classification, readability\nassessment etc. The performance of text classification improves notably when\nphrase patterns are used. The use of phrase patterns helps in capturing\nnon-local behaviours and thus helps in the improvement of text classification\ntask. Phrase structure extraction is the first step to continue with the phrase\npattern identification. In this survey, detailed study of phrase structure\nlearning methods have been carried out. This will enable future work in several\nNLP tasks, which uses syntactic information from phrase structure like grammar\ncheckers, question answering, information extraction, machine translation, text\nclassification. The paper also provides different levels of classification and\ndetailed comparison of the phrase structure learning methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.6844v1",
            "date": "2014-06-26 11:14:44+00:00",
            "title": "FrameNet Resource Grammar Library for GF",
            "authors": [
                "Normunds Gruzitis",
                "Peteris Paikens",
                "Guntis Barzdins"
            ],
            "abstract": "In this paper we present an ongoing research investigating the possibility\nand potential of integrating frame semantics, particularly FrameNet, in the\nGrammatical Framework (GF) application grammar development. An important\ncomponent of GF is its Resource Grammar Library (RGL) that encapsulates the\nlow-level linguistic knowledge about morphology and syntax of currently more\nthan 20 languages facilitating rapid development of multilingual applications.\nIn the ideal case, porting a GF application grammar to a new language would\nonly require introducing the domain lexicon - translation equivalents that are\ninterlinked via common abstract terms. While it is possible for a highly\nrestricted CNL, developing and porting a less restricted CNL requires above\naverage linguistic knowledge about the particular language, and above average\nGF experience. Specifying a lexicon is mostly straightforward in the case of\nnouns (incl. multi-word units), however, verbs are the most complex category\n(in terms of both inflectional paradigms and argument structure), and adding\nthem to a GF application grammar is not a straightforward task. In this paper\nwe are focusing on verbs, investigating the possibility of creating a\nmultilingual FrameNet-based GF library. We propose an extension to the current\nRGL, allowing GF application developers to define clauses on the semantic\nlevel, thus leaving the language-specific syntactic mapping to this extension.\nWe demonstrate our approach by reengineering the MOLTO Phrasebook application\ngrammar.",
            "journal ref": "Controlled Natural Language, Lecture Notes in Computer Science,\n  Vol. 7427, Springer, 2012, pp. 121-137",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.7483v1",
            "date": "2014-06-29 10:08:54+00:00",
            "title": "Jabalin: a Comprehensive Computational Model of Modern Standard Arabic Verbal Morphology Based on Traditional Arabic Prosody",
            "authors": [
                "Alicia Gonzalez Martinez",
                "Susana Lopez Hervas",
                "Doaa Samy",
                "Carlos G. Arques",
                "Antonio Moreno Sandoval"
            ],
            "abstract": "The computational handling of Modern Standard Arabic is a challenge in the\nfield of natural language processing due to its highly rich morphology.\nHowever, several authors have pointed out that the Arabic morphological system\nis in fact extremely regular. The existing Arabic morphological analyzers have\nexploited this regularity to variable extent, yet we believe there is still\nsome scope for improvement. Taking inspiration in traditional Arabic prosody,\nwe have designed and implemented a compact and simple morphological system\nwhich in our opinion takes further advantage of the regularities encountered in\nthe Arabic morphological system. The output of the system is a large-scale\nlexicon of inflected forms that has subsequently been used to create an Online\nInterface for a morphological analyzer of Arabic verbs. The Jabalin Online\nInterface is available at http://elvira.lllf.uam.es/jabalin/, hosted at the\nLLI-UAM lab. The generation system is also available under a GNU GPL 3 license.",
            "journal ref": "Systems and Frameworks for Computational Morphology, Springer\n  Berlin Heidelberg, (2013) pp. 35-52",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.0286v1",
            "date": "2014-10-01 16:53:35+00:00",
            "title": "LAF-Fabric: a data analysis tool for Linguistic Annotation Framework with an application to the Hebrew Bible",
            "authors": [
                "Dirk Roorda",
                "Gino Kalkman",
                "Martijn Naaijer",
                "Andreas van Cranenburgh"
            ],
            "abstract": "The Linguistic Annotation Framework (LAF) provides a general, extensible\nstand-off markup system for corpora. This paper discusses LAF-Fabric, a new\ntool to analyse LAF resources in general with an extension to process the\nHebrew Bible in particular. We first walk through the history of the Hebrew\nBible as text database in decennium-wide steps. Then we describe how LAF-Fabric\nmay serve as an analysis tool for this corpus. Finally, we describe three\nanalytic projects/workflows that benefit from the new LAF representation:\n  1) the study of linguistic variation: extract cooccurrence data of common\nnouns between the books of the Bible (Martijn Naaijer); 2) the study of the\ngrammar of Hebrew poetry in the Psalms: extract clause typology (Gino Kalkman);\n3) construction of a parser of classical Hebrew by Data Oriented Parsing:\ngenerate tree structures from the database (Andreas van Cranenburgh).",
            "journal ref": "Computational Linguistics in the Netherlands Journal, Volume 4,\n  December 2014, pp. 105-109",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.0291v2",
            "date": "2014-10-01 17:03:18+00:00",
            "title": "A Morphological Analyzer for Japanese Nouns, Verbs and Adjectives",
            "authors": [
                "Yanchuan Sim"
            ],
            "abstract": "We present an open source morphological analyzer for Japanese nouns, verbs\nand adjectives. The system builds upon the morphological analyzing capabilities\nof MeCab to incorporate finer details of classification such as politeness,\ntense, mood and voice attributes. We implemented our analyzer in the form of a\nfinite state transducer using the open source finite state compiler FOMA\ntoolkit. The source code and tool is available at\nhttps://bitbucket.org/skylander/yc-nlplab/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.0718v2",
            "date": "2014-10-02 21:35:35+00:00",
            "title": "Not All Neural Embeddings are Born Equal",
            "authors": [
                "Felix Hill",
                "KyungHyun Cho",
                "Sebastien Jean",
                "Coline Devin",
                "Yoshua Bengio"
            ],
            "abstract": "Neural language models learn word representations that capture rich\nlinguistic and conceptual information. Here we investigate the embeddings\nlearned by neural machine translation models. We show that translation-based\nembeddings outperform those learned by cutting-edge monolingual models at\nsingle-language tasks requiring knowledge of conceptual similarity and/or\nsyntactic role. The findings suggest that, while monolingual models learn\ninformation about how concepts are related, neural-translation models better\ncapture their true ontological status.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.1135v1",
            "date": "2014-10-05 09:02:31+00:00",
            "title": "Corpora Preparation and Stopword List Generation for Arabic data in Social Network",
            "authors": [
                "Walaa Medhat",
                "Ahmed H. Yousef",
                "Hoda Korashy"
            ],
            "abstract": "This paper proposes a methodology to prepare corpora in Arabic language from\nonline social network (OSN) and review site for Sentiment Analysis (SA) task.\nThe paper also proposes a methodology for generating a stopword list from the\nprepared corpora. The aim of the paper is to investigate the effect of removing\nstopwords on the SA task. The problem is that the stopwords lists generated\nbefore were on Modern Standard Arabic (MSA) which is not the common language\nused in OSN. We have generated a stopword list of Egyptian dialect and a\ncorpus-based list to be used with the OSN corpora. We compare the efficiency of\ntext classification when using the generated lists along with previously\ngenerated lists of MSA and combining the Egyptian dialect list with the MSA\nlist. The text classification was performed using Na\\\"ive Bayes and Decision\nTree classifiers and two feature selection approaches, unigrams and bigram. The\nexperiments show that the general lists containing the Egyptian dialects words\ngive better performance than using lists of MSA stopwords only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2082v2",
            "date": "2014-10-08 12:24:38+00:00",
            "title": "Contrastive Unsupervised Word Alignment with Non-Local Features",
            "authors": [
                "Yang Liu",
                "Maosong Sun"
            ],
            "abstract": "Word alignment is an important natural language processing task that\nindicates the correspondence between natural languages. Recently, unsupervised\nlearning of log-linear models for word alignment has received considerable\nattention as it combines the merits of generative and discriminative\napproaches. However, a major challenge still remains: it is intractable to\ncalculate the expectations of non-local features that are critical for\ncapturing the divergence between natural languages. We propose a contrastive\napproach that aims to differentiate observed training examples from noises. It\nnot only introduces prior knowledge to guide unsupervised learning but also\ncancels out partition functions. Based on the observation that the probability\nmass of log-linear models for word alignment is usually highly concentrated, we\npropose to use top-n alignments to approximate the expectations with respect to\nposterior distributions. This allows for efficient and accurate calculation of\nexpectations of non-local features. Experiments show that our approach achieves\nsignificant improvements over state-of-the-art unsupervised word alignment\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2149v1",
            "date": "2014-10-05 01:36:30+00:00",
            "title": "Language-based Examples in the Statistics Classroom",
            "authors": [
                "Roger Bilisoly"
            ],
            "abstract": "Statistics pedagogy values using a variety of examples. Thanks to text\nresources on the Web, and since statistical packages have the ability to\nanalyze string data, it is now easy to use language-based examples in a\nstatistics class. Three such examples are discussed here. First, many types of\nwordplay (e.g., crosswords and hangman) involve finding words with letters that\nsatisfy a certain pattern. Second, linguistics has shown that idiomatic pairs\nof words often appear together more frequently than chance. For example, in the\nBrown Corpus, this is true of the phrasal verb to throw up (p-value=7.92E-10.)\nThird, a pangram contains all the letters of the alphabet at least once. These\nare searched for in Charles Dickens' A Christmas Carol, and their lengths are\ncompared to the expected value given by the unequal probability coupon\ncollector's problem as well as simulations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2646v1",
            "date": "2014-10-09 22:56:44+00:00",
            "title": "Hybrid approaches for automatic vowelization of Arabic texts",
            "authors": [
                "Mohamed Bebah",
                "Chennoufi Amine",
                "Mazroui Azzeddine",
                "Lakhouaja Abdelhak"
            ],
            "abstract": "Hybrid approaches for automatic vowelization of Arabic texts are presented in\nthis article. The process is made up of two modules. In the first one, a\nmorphological analysis of the text words is performed using the open source\nmorphological Analyzer AlKhalil Morpho Sys. Outputs for each word analyzed out\nof context, are its different possible vowelizations. The integration of this\nAnalyzer in our vowelization system required the addition of a lexical database\ncontaining the most frequent words in Arabic language. Using a statistical\napproach based on two hidden Markov models (HMM), the second module aims to\neliminate the ambiguities. Indeed, for the first HMM, the unvowelized Arabic\nwords are the observed states and the vowelized words are the hidden states.\nThe observed states of the second HMM are identical to those of the first, but\nthe hidden states are the lists of possible diacritics of the word without its\nArabic letters. Our system uses Viterbi algorithm to select the optimal path\namong the solutions proposed by Al Khalil Morpho Sys. Our approach opens an\nimportant way to improve the performance of automatic vowelization of Arabic\ntexts for other uses in automatic natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2871v2",
            "date": "2014-10-10 19:19:31+00:00",
            "title": "An Ontology for Comprehensive Tutoring of Euphonic Conjunctions of Sanskrit Grammar",
            "authors": [
                "S. V. Kasmir Raja",
                "V. Rajitha",
                "Meenakshi Lakshmanan"
            ],
            "abstract": "Euphonic conjunctions (sandhis) form a very important aspect of Sanskrit\nmorphology and phonology. The traditional and modern methods of studying about\neuphonic conjunctions in Sanskrit follow different methodologies. The former\ninvolves a rigorous study of the Paninian system embodied in Panini's\nAshtadhyayi, while the latter usually involves the study of a few important\nsandhi rules with the use of examples. The former is not suitable for\nbeginners, and the latter, not sufficient to gain a comprehensive understanding\nof the operation of sandhi rules. This is so since there are not only numerous\nsandhi rules and exceptions, but also complex precedence rules involved. The\nneed for a new ontology for sandhi-tutoring was hence felt. This work presents\na comprehensive ontology designed to enable a student-user to learn in stages\nall about euphonic conjunctions and the relevant aphorisms of Sanskrit grammar\nand to test and evaluate the progress of the student-user. The ontology forms\nthe basis of a multimedia sandhi tutor that was given to different categories\nof users including Sanskrit scholars for extensive and rigorous testing.",
            "journal ref": "European Journal of Scientific Research, ISSN 1450-216X /\n  1450-202X, Vol. 124, No. 4, September 2014, pp 460-467",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4176v1",
            "date": "2014-10-15 19:27:10+00:00",
            "title": "Learning Distributed Word Representations for Natural Logic Reasoning",
            "authors": [
                "Samuel R. Bowman",
                "Christopher Potts",
                "Christopher D. Manning"
            ],
            "abstract": "Natural logic offers a powerful relational conception of meaning that is a\nnatural counterpart to distributed semantic representations, which have proven\nvaluable in a wide range of sophisticated language tasks. However, it remains\nan open question whether it is possible to train distributed representations to\nsupport the rich, diverse logical reasoning captured by natural logic. We\naddress this question using two neural network-based models for learning\nembeddings: plain neural networks and neural tensor networks. Our experiments\nevaluate the models' ability to learn the basic algebra of natural logic\nrelations from simulated data and from the WordNet noun graph. The overall\npositive results are promising for the future of learned distributed\nrepresentations in the applied modeling of logical semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4639v3",
            "date": "2014-10-17 05:19:12+00:00",
            "title": "Dependent Types for Pragmatics",
            "authors": [
                "Darryl McAdams",
                "Jonathan Sterling"
            ],
            "abstract": "This paper proposes the use of dependent types for pragmatic phenomena such\nas pronoun binding and presupposition resolution as a type-theoretic\nalternative to formalisms such as Discourse Representation Theory and Dynamic\nSemantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4863v1",
            "date": "2014-10-17 21:05:01+00:00",
            "title": "Arabic Language Text Classification Using Dependency Syntax-Based Feature Selection",
            "authors": [
                "Yannis Haralambous",
                "Yassir Elidrissi",
                "Philippe Lenca"
            ],
            "abstract": "We study the performance of Arabic text classification combining various\ntechniques: (a) tfidf vs. dependency syntax, for feature selection and\nweighting; (b) class association rules vs. support vector machines, for\nclassification. The Arabic text is used in two forms: rootified and lightly\nstemmed. The results we obtain show that lightly stemmed text leads to better\nperformance than rootified text; that class association rules are better suited\nfor small feature sets obtained by dependency syntax constraints; and, finally,\nthat support vector machines are better suited for large feature sets based on\nmorphological feature selection criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4868v1",
            "date": "2014-10-17 21:19:15+00:00",
            "title": "A Modality Lexicon and its use in Automatic Tagging",
            "authors": [
                "Kathryn Baker",
                "Michael Bloodgood",
                "Bonnie J. Dorr",
                "Nathaniel W. Filardo",
                "Lori Levin",
                "Christine Piatko"
            ],
            "abstract": "This paper describes our resource-building results for an eight-week JHU\nHuman Language Technology Center of Excellence Summer Camp for Applied Language\nExploration (SCALE-2009) on Semantically-Informed Machine Translation.\nSpecifically, we describe the construction of a modality annotation scheme, a\nmodality lexicon, and two automated modality taggers that were built using the\nlexicon and annotation scheme. Our annotation scheme is based on identifying\nthree components of modality: a trigger, a target and a holder. We describe how\nour modality lexicon was produced semi-automatically, expanding from an initial\nhand-selected list of modality trigger words and phrases. The resulting\nexpanded modality lexicon is being made publicly available. We demonstrate that\none tagger---a structure-based tagger---results in precision around 86%\n(depending on genre) for tagging of a standard LDC data set. In a machine\ntranslation application, using the structure-based tagger to annotate English\nmodalities on an English-Urdu training corpus improved the translation quality\nscore for Urdu by 0.3 Bleu points in the face of sparse training data.",
            "journal ref": "In Proceedings of the Seventh International Conference on Language\n  Resources and Evaluation (LREC'10), pages 1402-1407, Valletta, Malta, May\n  2010. European Language Resources Association",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4966v1",
            "date": "2014-10-18 14:53:19+00:00",
            "title": "The Visualization of Change in Word Meaning over Time using Temporal Word Embeddings",
            "authors": [
                "Chiraag Lala",
                "Shay B. Cohen"
            ],
            "abstract": "We describe a visualization tool that can be used to view the change in\nmeaning of words over time. The tool makes use of existing (static) word\nembedding datasets together with a timestamped $n$-gram corpus to create {\\em\ntemporal} word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.7182v1",
            "date": "2014-10-27 11:09:36+00:00",
            "title": "Analysis of Named Entity Recognition and Linking for Tweets",
            "authors": [
                "Leon Derczynski",
                "Diana Maynard",
                "Giuseppe Rizzo",
                "Marieke van Erp",
                "Genevieve Gorrell",
                "Rapha\u00ebl Troncy",
                "Johann Petrak",
                "Kalina Bontcheva"
            ],
            "abstract": "Applying natural language processing for mining and intelligent information\naccess to tweets (a form of microblog) is a challenging, emerging research\narea. Unlike carefully authored news text and other longer content, tweets pose\na number of new challenges, due to their short, noisy, context-dependent, and\ndynamic nature. Information extraction from tweets is typically performed in a\npipeline, comprising consecutive stages of language identification,\ntokenisation, part-of-speech tagging, named entity recognition and entity\ndisambiguation (e.g. with respect to DBpedia). In this work, we describe a new\nTwitter entity disambiguation dataset, and conduct an empirical analysis of\nnamed entity recognition and disambiguation, investigating how robust a number\nof state-of-the-art systems are on such noisy texts, what the main sources of\nerror are, and which problems should be further investigated to improve the\nstate of the art.",
            "journal ref": "Information Processing & Management 51 (2), 32-49, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.7787v1",
            "date": "2014-10-28 20:12:50+00:00",
            "title": "Correcting Errors in Digital Lexicographic Resources Using a Dictionary Manipulation Language",
            "authors": [
                "David Zajic",
                "Michael Maxwell",
                "David Doermann",
                "Paul Rodrigues",
                "Michael Bloodgood"
            ],
            "abstract": "We describe a paradigm for combining manual and automatic error correction of\nnoisy structured lexicographic data. Modifications to the structure and\nunderlying text of the lexicographic data are expressed in a simple,\ninterpreted programming language. Dictionary Manipulation Language (DML)\ncommands identify nodes by unique identifiers, and manipulations are performed\nusing simple commands such as create, move, set text, etc. Corrected lexicons\nare produced by applying sequences of DML commands to the source version of the\nlexicon. DML commands can be written manually to repair one-off errors or\ngenerated automatically to correct recurring problems. We discuss advantages of\nthe paradigm for the task of editing digital bilingual dictionaries.",
            "journal ref": "In Proceedings of Electronic Lexicography in the 21st Century\n  (eLex), pages 297-301, Bled, Slovenia, November 2011. Trojina Institute for\n  Applied Slovene Studies",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8668v1",
            "date": "2014-10-31 08:35:55+00:00",
            "title": "Experiments to Improve Named Entity Recognition on Turkish Tweets",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Ralf Steinberger"
            ],
            "abstract": "Social media texts are significant information sources for several\napplication areas including trend analysis, event monitoring, and opinion\nmining. Unfortunately, existing solutions for tasks such as named entity\nrecognition that perform well on formal texts usually perform poorly when\napplied to social media texts. In this paper, we report on experiments that\nhave the purpose of improving named entity recognition on Turkish tweets, using\ntwo different annotated data sets. In these experiments, starting with a\nbaseline named entity recognition system, we adapt its recognition rules and\nresources to better fit Twitter language by relaxing its capitalization\nconstraint and by diacritics-based expansion of its lexical resources, and we\nemploy a simplistic normalization scheme on tweets to observe the effects of\nthese on the overall named entity recognition performance on Turkish tweets.\nThe evaluation results of the system with these different settings are provided\nwith discussions of these results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.0708v1",
            "date": "2014-01-03 20:17:47+00:00",
            "title": "Quantitative methods for Phylogenetic Inference in Historical Linguistics: An experimental case study of South Central Dravidian",
            "authors": [
                "Taraka Rama",
                "Sudheer Kolachina",
                "Lakshmi Bai B"
            ],
            "abstract": "In this paper we examine the usefulness of two classes of algorithms Distance\nMethods, Discrete Character Methods (Felsenstein and Felsenstein 2003) widely\nused in genetics, for predicting the family relationships among a set of\nrelated languages and therefore, diachronic language change. Applying these\nalgorithms to the data on the numbers of shared cognates- with-change and\nchanged as well as unchanged cognates for a group of six languages belonging to\na Dravidian language sub-family given in Krishnamurti et al. (1983), we\nobserved that the resultant phylogenetic trees are largely in agreement with\nthe linguistic family tree constructed using the comparative method of\nreconstruction with only a few minor differences. Furthermore, we studied these\nminor differences and found that they were cases of genuine ambiguity even for\na well-trained historical linguist. We evaluated the trees obtained through our\nexperiments using a well-defined criterion and report the results here. We\nfinally conclude that quantitative methods like the ones we examined are quite\nuseful in predicting family relationships among languages. In addition, we\nconclude that a modest degree of confidence attached to the intuition that\nthere could indeed exist a parallelism between the processes of linguistic and\ngenetic change is not totally misplaced.",
            "journal ref": "Indian Linguistics, Volume 70, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.0794v1",
            "date": "2014-01-04 09:50:55+00:00",
            "title": "Properties of phoneme N -grams across the world's language families",
            "authors": [
                "Taraka Rama",
                "Lars Borin"
            ],
            "abstract": "In this article, we investigate the properties of phoneme N-grams across half\nof the world's languages. We investigate if the sizes of three different N-gram\ndistributions of the world's language families obey a power law. Further, the\nN-gram distributions of language families parallel the sizes of the families,\nwhich seem to obey a power law distribution. The correlation between N-gram\ndistributions and language family sizes improves with increasing values of N.\nWe applied statistical tests, originally given by physicists, to test the\nhypothesis of power law fit to twelve different datasets. The study also raises\nsome new questions about the use of N-gram distributions in linguistic\nresearch, which we answer by running a statistical test.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.1486v1",
            "date": "2014-01-07 20:07:51+00:00",
            "title": "Design & Development of the Graphical User Interface for Sindhi Language",
            "authors": [
                "Imdad Ali Ismaili",
                "Zeeshan Bhatti",
                "Azhar Ali Shah"
            ],
            "abstract": "This paper describes the design and implementation of a Unicode-based GUISL\n(Graphical User Interface for Sindhi Language). The idea is to provide a\nsoftware platform to the people of Sindh as well as Sindhi diasporas living\nacross the globe to make use of computing for basic tasks such as editing,\ncomposition, formatting, and printing of documents in Sindhi by using GUISL.\nThe implementation of the GUISL has been done in the Java technology to make\nthe system platform independent. The paper describes several design issues of\nSindhi GUI in the context of existing software tools and technologies and\nexplains how mapping and concatenation techniques have been employed to achieve\nthe cursive shape of Sindhi script.",
            "journal ref": "Mehran University Research Journal of Engineering & Technology,\n  Volume 30, No. 4, October 2011 [ISSN 0254-7821]",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1401.2258v1",
            "date": "2014-01-10 08:50:54+00:00",
            "title": "Assessing Wikipedia-Based Cross-Language Retrieval Models",
            "authors": [
                "Benjamin Roth"
            ],
            "abstract": "This work compares concept models for cross-language retrieval: First, we\nadapt probabilistic Latent Semantic Analysis (pLSA) for multilingual documents.\nExperiments with different weighting schemes show that a weighting method\nfavoring documents of similar length in both language sides gives best results.\nConsidering that both monolingual and multilingual Latent Dirichlet Allocation\n(LDA) behave alike when applied for such documents, we use a training corpus\nbuilt on Wikipedia where all documents are length-normalized and obtain\nimprovements over previously reported scores for LDA. Another focus of our work\nis on model combination. For this end we include Explicit Semantic Analysis\n(ESA) in the experiments. We observe that ESA is not competitive with LDA in a\nquery based retrieval task on CLEF 2000 data. The combination of machine\ntranslation with concept models increased performance by 21.1% map in\ncomparison to machine translation alone. Machine translation relies on parallel\ncorpora, which may not be available for many language pairs. We further explore\nhow much cross-lingual information can be carried over by a specific\ninformation source in Wikipedia, namely linked text. The best results are\nobtained using a language modeling approach, entirely without information from\nparallel corpora. The need for smoothing raises interesting questions on\nsoundness and efficiency. Link models capture only a certain kind of\ninformation and suggest weighting schemes to emphasize particular words. For a\ncombined model, another interesting question is therefore how to integrate\ndifferent weighting schemes. Using a very simple combination scheme, we obtain\nresults that compare favorably to previously reported results on the CLEF 2000\ndataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.2618v1",
            "date": "2014-01-12 12:35:57+00:00",
            "title": "Sentiment Analysis Using Collaborated Opinion Mining",
            "authors": [
                "Deepali Virmani",
                "Vikrant Malhotra",
                "Ridhi Tyagi"
            ],
            "abstract": "Opinion mining and Sentiment analysis have emerged as a field of study since\nthe widespread of World Wide Web and internet. Opinion refers to extraction of\nthose lines or phrase in the raw and huge data which express an opinion.\nSentiment analysis on the other hand identifies the polarity of the opinion\nbeing extracted. In this paper we propose the sentiment analysis in\ncollaboration with opinion extraction, summarization, and tracking the records\nof the students. The paper modifies the existing algorithm in order to obtain\nthe collaborated opinion about the students. The resultant opinion is\nrepresented as very high, high, moderate, low and very low. The paper is based\non a case study where teachers give their remarks about the students and by\napplying the proposed sentiment analysis algorithm the opinion is extracted and\nrepresented.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.2851v1",
            "date": "2014-01-09 18:05:15+00:00",
            "title": "Statistical Analysis based Hypothesis Testing Method in Biological Knowledge Discovery",
            "authors": [
                "Md. Naseef-Ur-Rahman Chowdhury",
                "Suvankar Paul",
                "Kazi Zakia Sultana"
            ],
            "abstract": "The correlation and interactions among different biological entities comprise\nthe biological system. Although already revealed interactions contribute to the\nunderstanding of different existing systems, researchers face many questions\neveryday regarding inter-relationships among entities. Their queries have\npotential role in exploring new relations which may open up a new area of\ninvestigation. In this paper, we introduce a text mining based method for\nanswering the biological queries in terms of statistical computation such that\nresearchers can come up with new knowledge discovery. It facilitates user to\nsubmit their query in natural linguistic form which can be treated as\nhypothesis. Our proposed approach analyzes the hypothesis and measures the\np-value of the hypothesis with respect to the existing literature. Based on the\nmeasured value, the system either accepts or rejects the hypothesis from\nstatistical point of view. Moreover, even it does not find any direct\nrelationship among the entities of the hypothesis, it presents a network to\ngive an integral overview of all the entities through which the entities might\nbe related. This is also congenial for the researchers to widen their view and\nthus think of new hypothesis for further investigation. It assists researcher\nto get a quantitative evaluation of their assumptions such that they can reach\na logical conclusion and thus aids in relevant re-searches of biological\nknowledge discovery. The system also provides the researchers a graphical\ninteractive interface to submit their hypothesis for assessment in a more\nconvenient way.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.3372v1",
            "date": "2014-01-14 22:10:30+00:00",
            "title": "Learning Language from a Large (Unannotated) Corpus",
            "authors": [
                "Linas Vepstas",
                "Ben Goertzel"
            ],
            "abstract": "A novel approach to the fully automated, unsupervised extraction of\ndependency grammars and associated syntax-to-semantic-relationship mappings\nfrom large text corpora is described. The suggested approach builds on the\nauthors' prior work with the Link Grammar, RelEx and OpenCog systems, as well\nas on a number of prior papers and approaches from the statistical language\nlearning literature. If successful, this approach would enable the mining of\nall the information needed to power a natural language comprehension and\ngeneration system, directly from a large, unannotated corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3457v1",
            "date": "2014-01-15 05:14:31+00:00",
            "title": "Learning Document-Level Semantic Properties from Free-Text Annotations",
            "authors": [
                "S. R. K. Branavan",
                "Harr Chen",
                "Jacob Eisenstein",
                "Regina Barzilay"
            ],
            "abstract": "This paper presents a new method for inferring the semantic properties of\ndocuments by leveraging free-text keyphrase annotations. Such annotations are\nbecoming increasingly abundant due to the recent dramatic growth in\nsemi-structured, user-generated online content. One especially relevant domain\nis product reviews, which are often annotated by their authors with pros/cons\nkeyphrases such as a real bargain or good value. These annotations are\nrepresentative of the underlying semantic properties; however, unlike expert\nannotations, they are noisy: lay authors may use different labels to denote the\nsame property, and some labels may be missing. To learn using such noisy\nannotations, we find a hidden paraphrase structure which clusters the\nkeyphrases. The paraphrase structure is linked with a latent topic model of the\nreview texts, enabling the system to predict the properties of unannotated\ndocuments and to effectively aggregate the semantic properties of multiple\nreviews. Our approach is implemented as a hierarchical Bayesian model with\njoint inference. We find that joint inference increases the robustness of the\nkeyphrase clustering and encourages the latent topics to correlate with\nsemantically meaningful properties. Multiple evaluations demonstrate that our\nmodel substantially outperforms alternative approaches for summarizing single\nand multiple documents into a set of semantically salient keyphrases.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 34, pages\n  569-603, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3510v1",
            "date": "2014-01-15 08:07:08+00:00",
            "title": "Improving Performance Of English-Hindi Cross Language Information Retrieval Using Transliteration Of Query Terms",
            "authors": [
                "Saurabh Varshney",
                "Jyoti Bajpai"
            ],
            "abstract": "The main issue in Cross Language Information Retrieval (CLIR) is the poor\nperformance of retrieval in terms of average precision when compared to\nmonolingual retrieval performance. The main reasons behind poor performance of\nCLIR are mismatching of query terms, lexical ambiguity and un-translated query\nterms. The existing problems of CLIR are needed to be addressed in order to\nincrease the performance of the CLIR system. In this paper, we are putting our\neffort to solve the given problem by proposed an algorithm for improving the\nperformance of English-Hindi CLIR system. We used all possible combination of\nHindi translated query using transliteration of English query terms and\nchoosing the best query among them for retrieval of documents. The experiment\nis performed on FIRE 2010 (Forum of Information Retrieval Evaluation) datasets.\nThe experimental result show that the proposed approach gives better\nperformance of English-Hindi CLIR system and also helps in overcoming existing\nproblems and outperforms the existing English-Hindi CLIR system in terms of\naverage precision.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.3832v1",
            "date": "2014-01-16 04:49:45+00:00",
            "title": "Constructing Reference Sets from Unstructured, Ungrammatical Text",
            "authors": [
                "Matthew Michelson",
                "Craig A. Knoblock"
            ],
            "abstract": "Vast amounts of text on the Web are unstructured and ungrammatical, such as\nclassified ads, auction listings, forum postings, etc. We call such text\n\"posts.\" Despite their inconsistent structure and lack of grammar, posts are\nfull of useful information. This paper presents work on semi-automatically\nbuilding tables of relational information, called \"reference sets,\" by\nanalyzing such posts directly. Reference sets can be applied to a number of\ntasks such as ontology maintenance and information extraction. Our\nreference-set construction method starts with just a small amount of background\nknowledge, and constructs tuples representing the entities in the posts to form\na reference set. We also describe an extension to this approach for the special\ncase where even this small amount of background knowledge is impossible to\ndiscover and use. To evaluate the utility of the machine-constructed reference\nsets, we compare them to manually constructed reference sets in the context of\nreference-set-based information extraction. Our results show the reference sets\nconstructed by our method outperform manually constructed reference sets. We\nalso compare the reference-set-based extraction approach using the\nmachine-constructed reference set to supervised extraction approaches using\ngeneric features. These results demonstrate that using machine-constructed\nreference sets outperforms the supervised methods, even though the supervised\nmethods require training data.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 38, pages\n  189-221, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3865v1",
            "date": "2014-01-16 05:05:45+00:00",
            "title": "Evaluating Temporal Graphs Built from Texts via Transitive Reduction",
            "authors": [
                "Xavier Tannier",
                "Philippe Muller"
            ],
            "abstract": "Temporal information has been the focus of recent attention in information\nextraction, leading to some standardization effort, in particular for the task\nof relating events in a text. This task raises the problem of comparing two\nannotations of a given text, because relations between events in a story are\nintrinsically interdependent and cannot be evaluated separately. A proper\nevaluation measure is also crucial in the context of a machine learning\napproach to the problem. Finding a common comparison referent at the text level\nis not obvious, and we argue here in favor of a shift from event-based measures\nto measures on a unique textual object, a minimal underlying temporal graph, or\nmore formally the transitive reduction of the graph of relations between event\nboundaries. We support it by an investigation of its properties on synthetic\ndata and on a well-know temporal corpus.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 40, pages\n  375-413, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3908v1",
            "date": "2014-01-16 05:23:22+00:00",
            "title": "Centrality-as-Relevance: Support Sets and Similarity as Geometric Proximity",
            "authors": [
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "In automatic summarization, centrality-as-relevance means that the most\nimportant content of an information source, or a collection of information\nsources, corresponds to the most central passages, considering a representation\nwhere such notion makes sense (graph, spatial, etc.). We assess the main\nparadigms, and introduce a new centrality-based relevance model for automatic\nsummarization that relies on the use of support sets to better estimate the\nrelevant content. Geometric proximity is used to compute semantic relatedness.\nCentrality (relevance) is determined by considering the whole input source (and\nnot only local information), and by taking into account the existence of minor\ntopics or lateral subjects in the information sources to be summarized. The\nmethod consists in creating, for each passage of the input source, a support\nset consisting only of the most semantically related passages. Then, the\ndetermination of the most relevant content is achieved by selecting the\npassages that occur in the largest number of support sets. This model produces\nextractive summaries that are generic, and language- and domain-independent.\nThorough automatic evaluation shows that the method achieves state-of-the-art\nperformance, both in written text, and automatically transcribed speech\nsummarization, including when compared to considerably more complex approaches.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 42, pages\n  275-308, 2011",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.4205v1",
            "date": "2014-01-17 00:00:32+00:00",
            "title": "Entropy analysis of word-length series of natural language texts: Effects of text language and genre",
            "authors": [
                "Maria Kalimeri",
                "Vassilios Constantoudis",
                "Constantinos Papadimitriou",
                "Kostantinos Karamanos",
                "Fotis K. Diakonos",
                "Haris Papageorgiou"
            ],
            "abstract": "We estimate the $n$-gram entropies of natural language texts in word-length\nrepresentation and find that these are sensitive to text language and genre. We\nattribute this sensitivity to changes in the probability distribution of the\nlengths of single words and emphasize the crucial role of the uniformity of\nprobabilities of having words with length between five and ten. Furthermore,\ncomparison with the entropies of shuffled data reveals the impact of word\nlength correlations on the estimated $n$-gram entropies.",
            "journal ref": "International Journal of Bifurcation and Chaos, 22, 1250223,\n  (2012)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.4436v1",
            "date": "2014-01-16 04:53:44+00:00",
            "title": "Cause Identification from Aviation Safety Incident Reports via Weakly Supervised Semantic Lexicon Construction",
            "authors": [
                "Muhammad Arshad Ul Abedin",
                "Vincent Ng",
                "Latifur Khan"
            ],
            "abstract": "The Aviation Safety Reporting System collects voluntarily submitted reports\non aviation safety incidents to facilitate research work aiming to reduce such\nincidents. To effectively reduce these incidents, it is vital to accurately\nidentify why these incidents occurred. More precisely, given a set of possible\ncauses, or shaping factors, this task of cause identification involves\nidentifying all and only those shaping factors that are responsible for the\nincidents described in a report. We investigate two approaches to cause\nidentification. Both approaches exploit information provided by a semantic\nlexicon, which is automatically constructed via Thelen and Riloffs Basilisk\nframework augmented with our linguistic and algorithmic modifications. The\nfirst approach labels a report using a simple heuristic, which looks for the\nwords and phrases acquired during the semantic lexicon learning process in the\nreport. The second approach recasts cause identification as a text\nclassification problem, employing supervised and transductive text\nclassification algorithms to learn models from incident reports labeled with\nshaping factors and using the models to label unseen reports. Our experiments\nshow that both the heuristic-based approach and the learning-based approach\n(when given sufficient training data) outperform the baseline system\nsignificantly.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 38, pages\n  569-631, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.4603v1",
            "date": "2014-01-18 21:09:42+00:00",
            "title": "Semantic Similarity Measures Applied to an Ontology for Human-Like Interaction",
            "authors": [
                "Esperanza Albacete",
                "Javier Calle",
                "Elena Castro",
                "Dolores Cuadra"
            ],
            "abstract": "The focus of this paper is the calculation of similarity between two concepts\nfrom an ontology for a Human-Like Interaction system. In order to facilitate\nthis calculation, a similarity function is proposed based on five dimensions\n(sort, compositional, essential, restrictive and descriptive) constituting the\nstructure of ontological knowledge. The paper includes a proposal for computing\na similarity function for each dimension of knowledge. Later on, the similarity\nvalues obtained are weighted and aggregated to obtain a global similarity\nmeasure. In order to calculate those weights associated to each dimension, four\ntraining methods have been proposed. The training methods differ in the element\nto fit: the user, concepts or pairs of concepts, and a hybrid approach. For\nevaluating the proposal, the knowledge base was fed from WordNet and extended\nby using a knowledge editing toolkit (Cognos). The evaluation of the proposal\nis carried out through the comparison of system responses with those given by\nhuman test subjects, both providing a measure of the soundness of the procedure\nand revealing ways in which the proposal may be improved.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 44, pages\n  397-421, 2012",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1401.4869v1",
            "date": "2014-01-20 11:49:11+00:00",
            "title": "Does Syntactic Knowledge help English-Hindi SMT?",
            "authors": [
                "Taraka Rama",
                "Karthik Gali",
                "Avinesh PVS"
            ],
            "abstract": "In this paper we explore various parameter settings of the state-of-art\nStatistical Machine Translation system to improve the quality of the\ntranslation for a `distant' language pair like English-Hindi. We proposed new\ntechniques for efficient reordering. A slight improvement over the baseline is\nreported using these techniques. We also show that a simple pre-processing step\ncan improve the quality of the translation significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.4994v1",
            "date": "2014-01-20 17:31:05+00:00",
            "title": "A Review of Verbal and Non-Verbal Human-Robot Interactive Communication",
            "authors": [
                "Nikolaos Mavridis"
            ],
            "abstract": "In this paper, an overview of human-robot interactive communication is\npresented, covering verbal as well as non-verbal aspects of human-robot\ninteraction. Following a historical introduction, and motivation towards fluid\nhuman-robot communication, ten desiderata are proposed, which provide an\norganizational axis both of recent as well as of future research on human-robot\ncommunication. Then, the ten desiderata are examined in detail, culminating to\na unifying discussion, and a forward-looking conclusion.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1401.5644v1",
            "date": "2014-01-22 12:36:38+00:00",
            "title": "A new keyphrases extraction method based on suffix tree data structure for arabic documents clustering",
            "authors": [
                "Issam Sahmoudi",
                "Hanane Froud",
                "Abdelmonaime Lachkar"
            ],
            "abstract": "Document Clustering is a branch of a larger area of scientific study known as\ndata mining .which is an unsupervised classification using to find a structure\nin a collection of unlabeled data. The useful information in the documents can\nbe accompanied by a large amount of noise words when using Full Text\nRepresentation, and therefore will affect negatively the result of the\nclustering process. So it is with great need to eliminate the noise words and\nkeeping just the useful information in order to enhance the quality of the\nclustering results. This problem occurs with different degree for any language\nsuch as English, European, Hindi, Chinese, and Arabic Language. To overcome\nthis problem, in this paper, we propose a new and efficient Keyphrases\nextraction method based on the Suffix Tree data structure (KpST), the extracted\nKeyphrases are then used in the clustering process instead of Full Text\nRepresentation. The proposed method for Keyphrases extraction is language\nindependent and therefore it may be applied to any language. In this\ninvestigation, we are interested to deal with the Arabic language which is one\nof the most complex languages. To evaluate our method, we conduct an\nexperimental study on Arabic Documents using the most popular Clustering\napproach of Hierarchical algorithms: Agglomerative Hierarchical algorithm with\nseven linkage techniques and a variety of distance functions and similarity\nmeasures to perform Arabic Document Clustering task. The obtained results show\nthat our method for extracting Keyphrases increases the quality of the\nclustering results. We propose also to study the effect of using the stemming\nfor the testing dataset to cluster it with the same documents clustering\ntechniques and similarity/distance measures.",
            "journal ref": "International Journal of Database Management Systems ( IJDMS )\n  Vol.5, No.6, December 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6131v1",
            "date": "2014-01-16 05:20:08+00:00",
            "title": "Controlling Complexity in Part-of-Speech Induction",
            "authors": [
                "Jo\u00e3o V. Gra\u00e7a",
                "Kuzman Ganchev",
                "Luisa Coheur",
                "Fernando Pereira",
                "Ben Taskar"
            ],
            "abstract": "We consider the problem of fully unsupervised learning of grammatical\n(part-of-speech) categories from unlabeled text. The standard\nmaximum-likelihood hidden Markov model for this task performs poorly, because\nof its weak inductive bias and large model capacity. We address this problem by\nrefining the model and modifying the learning objective to control its capacity\nvia para- metric and non-parametric constraints. Our approach enforces\nword-category association sparsity, adds morphological and orthographic\nfeatures, and eliminates hard-to-estimate parameters for rare words. We develop\nan efficient learning algorithm that is not much more computationally intensive\nthan standard training. We also provide an open-source implementation of the\nalgorithm. Our experiments on five diverse languages (Bulgarian, Danish,\nEnglish, Portuguese, Spanish) achieve significant improvements compared with\nprevious methods for the same task.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 41, pages\n  527-551, 2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6224v1",
            "date": "2014-01-24 00:26:58+00:00",
            "title": "Word-length entropies and correlations of natural language written texts",
            "authors": [
                "Maria Kalimeri",
                "Vassilios Constantoudis",
                "Constantinos Papadimitriou",
                "Konstantinos Karamanos",
                "Fotis K. Diakonos",
                "Harris Papageorgiou"
            ],
            "abstract": "We study the frequency distributions and correlations of the word lengths of\nten European languages. Our findings indicate that a) the word-length\ndistribution of short words quantified by the mean value and the entropy\ndistinguishes the Uralic (Finnish) corpus from the others, b) the tails at long\nwords, manifested in the high-order moments of the distributions, differentiate\nthe Germanic languages (except for English) from the Romanic languages and\nGreek and c) the correlations between nearby word lengths measured by the\ncomparison of the real entropies with those of the shuffled texts are found to\nbe smaller in the case of Germanic and Finnish languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6427v1",
            "date": "2014-01-23 02:50:50+00:00",
            "title": "Towards Unsupervised Learning of Temporal Relations between Events",
            "authors": [
                "Seyed Abolghasem Mirroshandel",
                "Gholamreza Ghassem-Sani"
            ],
            "abstract": "Automatic extraction of temporal relations between event pairs is an\nimportant task for several natural language processing applications such as\nQuestion Answering, Information Extraction, and Summarization. Since most\nexisting methods are supervised and require large corpora, which for many\nlanguages do not exist, we have concentrated our efforts to reduce the need for\nannotated data as much as possible. This paper presents two different\nalgorithms towards this goal. The first algorithm is a weakly supervised\nmachine learning approach for classification of temporal relations between\nevents. In the first stage, the algorithm learns a general classifier from an\nannotated corpus. Then, inspired by the hypothesis of \"one type of temporal\nrelation per discourse, it extracts useful information from a cluster of\ntopically related documents. We show that by combining the global information\nof such a cluster with local decisions of a general classifier, a bootstrapping\ncross-document classifier can be built to extract temporal relations between\nevents. Our experiments show that without any additional annotated data, the\naccuracy of the proposed algorithm is higher than that of several previous\nsuccessful systems. The second proposed method for temporal relation extraction\nis based on the expectation maximization (EM) algorithm. Within EM, we used\ndifferent techniques such as a greedy best-first search and integer linear\nprogramming for temporal inconsistency removal. We think that the experimental\nresults of our EM based algorithm, as a first step toward a fully unsupervised\ntemporal relation extraction method, is encouraging.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 45, pages\n  125-163, 2012",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1401.6567v1",
            "date": "2014-01-25 18:45:29+00:00",
            "title": "A Machine Learning Approach for the Identification of Bengali Noun-Noun Compound Multiword Expressions",
            "authors": [
                "Vivekananda Gayen",
                "Kamal Sarkar"
            ],
            "abstract": "This paper presents a machine learning approach for identification of Bengali\nmultiword expressions (MWE) which are bigram nominal compounds. Our proposed\napproach has two steps: (1) candidate extraction using chunk information and\nvarious heuristic rules and (2) training the machine learning algorithm called\nRandom Forest to classify the candidates into two groups: bigram nominal\ncompound MWE or not bigram nominal compound MWE. A variety of association\nmeasures, syntactic and linguistic clues and a set of WordNet-based similarity\nfeatures have been used for our MWE identification task. The approach presented\nin this paper can be used to identify bigram nominal compound MWE in Bengali\nrunning text.",
            "journal ref": "In Proceedings of ICON-2013: 10th International Conference on\n  Natural Language Processing, pp 290-296",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6571v1",
            "date": "2014-01-25 19:05:45+00:00",
            "title": "Keyword and Keyphrase Extraction Using Centrality Measures on Collocation Networks",
            "authors": [
                "Shibamouli Lahiri",
                "Sagnik Ray Choudhury",
                "Cornelia Caragea"
            ],
            "abstract": "Keyword and keyphrase extraction is an important problem in natural language\nprocessing, with applications ranging from summarization to semantic search to\ndocument clustering. Graph-based approaches to keyword and keyphrase extraction\navoid the problem of acquiring a large in-domain training corpus by applying\nvariants of PageRank algorithm on a network of words. Although graph-based\napproaches are knowledge-lean and easily adoptable in online systems, it\nremains largely open whether they can benefit from centrality measures other\nthan PageRank. In this paper, we experiment with an array of centrality\nmeasures on word and noun phrase collocation networks, and analyze their\nperformance on four benchmark datasets. Not only are there centrality measures\nthat perform as well as or better than PageRank, but they are much simpler\n(e.g., degree, strength, and neighborhood size). Furthermore, centrality-based\nmethods give results that are competitive with and, in some cases, better than\ntwo strong unsupervised baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6573v1",
            "date": "2014-01-25 19:50:51+00:00",
            "title": "Deverbal semantics and the Montagovian generative lexicon",
            "authors": [
                "Livy-Maria Real-Coelho",
                "Christian Retor\u00e9"
            ],
            "abstract": "We propose a lexical account of action nominals, in particular of deverbal\nnominalisations, whose meaning is related to the event expressed by their base\nverb. The literature about nominalisations often assumes that the semantics of\nthe base verb completely defines the structure of action nominals. We argue\nthat the information in the base verb is not sufficient to completely determine\nthe semantics of action nominals. We exhibit some data from different\nlanguages, especially from Romance language, which show that nominalisations\nfocus on some aspects of the verb semantics. The selected aspects, however,\nseem to be idiosyncratic and do not automatically result from the internal\nstructure of the verb nor from its interaction with the morphological suffix.\nWe therefore propose a partially lexicalist approach view of deverbal nouns. It\nis made precise and computable by using the Montagovian Generative Lexicon, a\ntype theoretical framework introduced by Bassac, Mery and Retor\\'e in this\njournal in 2010. This extension of Montague semantics with a richer type system\neasily incorporates lexical phenomena like the semantics of action nominals in\nparticular deverbals, including their polysemy and (in)felicitous\ncopredications.",
            "journal ref": "Journal of Logic, Language and Information (2014) 21",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6984v1",
            "date": "2014-01-27 19:55:34+00:00",
            "title": "Kaldi+PDNN: Building DNN-based ASR Systems with Kaldi and PDNN",
            "authors": [
                "Yajie Miao"
            ],
            "abstract": "The Kaldi toolkit is becoming popular for constructing automated speech\nrecognition (ASR) systems. Meanwhile, in recent years, deep neural networks\n(DNNs) have shown state-of-the-art performance on various ASR tasks. This\ndocument describes our open-source recipes to implement fully-fledged DNN\nacoustic modeling using Kaldi and PDNN. PDNN is a lightweight deep learning\ntoolkit developed under the Theano environment. Using these recipes, we can\nbuild up multiple systems including DNN hybrid systems, convolutional neural\nnetwork (CNN) systems and bottleneck feature systems. These recipes are\ndirectly based on the Kaldi Switchboard 110-hour setup. However, adapting them\nto new datasets is easy to achieve.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1409.0915v1",
            "date": "2014-09-02 22:59:52+00:00",
            "title": "An Approach for Text Steganography Based on Markov Chains",
            "authors": [
                "H. Hernan Moraldo"
            ],
            "abstract": "A text steganography method based on Markov chains is introduced, together\nwith a reference implementation. This method allows for information hiding in\ntexts that are automatically generated following a given Markov model. Other\nMarkov - based systems of this kind rely on big simplifications of the language\nmodel to work, which produces less natural looking and more easily detectable\ntexts. The method described here is designed to generate texts within a good\napproximation of the original language model provided.",
            "journal ref": "41 JAIIO - WSegI 2012, ISSN: 2313-9110, pages 21 - 35",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1409.1259v2",
            "date": "2014-09-03 21:03:41+00:00",
            "title": "On the Properties of Neural Machine Translation: Encoder-Decoder Approaches",
            "authors": [
                "Kyunghyun Cho",
                "Bart van Merrienboer",
                "Dzmitry Bahdanau",
                "Yoshua Bengio"
            ],
            "abstract": "Neural machine translation is a relatively new approach to statistical\nmachine translation based purely on neural networks. The neural machine\ntranslation models often consist of an encoder and a decoder. The encoder\nextracts a fixed-length representation from a variable-length input sentence,\nand the decoder generates a correct translation from this representation. In\nthis paper, we focus on analyzing the properties of the neural machine\ntranslation using two models; RNN Encoder--Decoder and a newly proposed gated\nrecursive convolutional neural network. We show that the neural machine\ntranslation performs relatively well on short sentences without unknown words,\nbut its performance degrades rapidly as the length of the sentence and the\nnumber of unknown words increase. Furthermore, we find that the proposed gated\nrecursive convolutional network learns a grammatical structure of a sentence\nautomatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.1612v2",
            "date": "2014-09-04 21:09:26+00:00",
            "title": "Semantic clustering of Russian web search results: possibilities and problems",
            "authors": [
                "Andrey Kutuzov"
            ],
            "abstract": "The paper deals with word sense induction from lexical co-occurrence graphs.\nWe construct such graphs on large Russian corpora and then apply this data to\ncluster Mail.ru Search results according to meanings of the query. We compare\ndifferent methods of performing such clustering and different source corpora.\nModels of applying distributional semantics to big linguistic data are\ndescribed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3215v3",
            "date": "2014-09-10 19:55:35+00:00",
            "title": "Sequence to Sequence Learning with Neural Networks",
            "authors": [
                "Ilya Sutskever",
                "Oriol Vinyals",
                "Quoc V. Le"
            ],
            "abstract": "Deep Neural Networks (DNNs) are powerful models that have achieved excellent\nperformance on difficult learning tasks. Although DNNs work well whenever large\nlabeled training sets are available, they cannot be used to map sequences to\nsequences. In this paper, we present a general end-to-end approach to sequence\nlearning that makes minimal assumptions on the sequence structure. Our method\nuses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to\na vector of a fixed dimensionality, and then another deep LSTM to decode the\ntarget sequence from the vector. Our main result is that on an English to\nFrench translation task from the WMT'14 dataset, the translations produced by\nthe LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's\nBLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did\nnot have difficulty on long sentences. For comparison, a phrase-based SMT\nsystem achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM\nto rerank the 1000 hypotheses produced by the aforementioned SMT system, its\nBLEU score increases to 36.5, which is close to the previous best result on\nthis task. The LSTM also learned sensible phrase and sentence representations\nthat are sensitive to word order and are relatively invariant to the active and\nthe passive voice. Finally, we found that reversing the order of the words in\nall source sentences (but not target sentences) improved the LSTM's performance\nmarkedly, because doing so introduced many short term dependencies between the\nsource and the target sentence which made the optimization problem easier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3870v3",
            "date": "2014-09-12 21:34:28+00:00",
            "title": "Text mixing shapes the anatomy of rank-frequency distributions: A modern Zipfian mechanics for natural language",
            "authors": [
                "Jake Ryland Williams",
                "James P. Bagrow",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Natural languages are full of rules and exceptions. One of the most famous\nquantitative rules is Zipf's law which states that the frequency of occurrence\nof a word is approximately inversely proportional to its rank. Though this\n`law' of ranks has been found to hold across disparate texts and forms of data,\nanalyses of increasingly large corpora over the last 15 years have revealed the\nexistence of two scaling regimes. These regimes have thus far been explained by\na hypothesis suggesting a separability of languages into core and non-core\nlexica. Here, we present and defend an alternative hypothesis, that the two\nscaling regimes result from the act of aggregating texts. We observe that text\nmixing leads to an effective decay of word introduction, which we show provides\naccurate predictions of the location and severity of breaks in scaling. Upon\nexamining large corpora from 10 languages in the Project Gutenberg eBooks\ncollection (eBooks), we find emphatic empirical support for the universality of\nour claim.",
            "journal ref": "Phys. Rev. E 91, 052811 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.3942v1",
            "date": "2014-09-13 12:36:56+00:00",
            "title": "Polarity detection movie reviews in hindi language",
            "authors": [
                "Richa Sharma",
                "Shweta Nigam",
                "Rekha Jain"
            ],
            "abstract": "Nowadays peoples are actively involved in giving comments and reviews on\nsocial networking websites and other websites like shopping websites, news\nwebsites etc. large number of people everyday share their opinion on the web,\nresults is a large number of user data is collected .users also find it trivial\ntask to read all the reviews and then reached into the decision. It would be\nbetter if these reviews are classified into some category so that the user\nfinds it easier to read. Opinion Mining or Sentiment Analysis is a natural\nlanguage processing task that mines information from various text forms such as\nreviews, news, and blogs and classify them on the basis of their polarity as\npositive, negative or neutral. But, from the last few years, user content in\nHindi language is also increasing at a rapid rate on the Web. So it is very\nimportant to perform opinion mining in Hindi language as well. In this paper a\nHindi language opinion mining system is proposed. The system classifies the\nreviews as positive, negative and neutral for Hindi language. Negation is also\nhandled in the proposed system. Experimental results using reviews of movies\nshow the effectiveness of the system",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4504v1",
            "date": "2014-09-16 05:12:50+00:00",
            "title": "Voting for Deceptive Opinion Spam Detection",
            "authors": [
                "Tao Wang",
                "Hua Zhu"
            ],
            "abstract": "Consumers' purchase decisions are increasingly influenced by user-generated\nonline reviews. Accordingly, there has been growing concern about the potential\nfor posting deceptive opinion spam fictitious reviews that have been\ndeliberately written to sound authentic, to deceive the readers. Existing\napproaches mainly focus on developing automatic supervised learning based\nmethods to help users identify deceptive opinion spams.\n  This work, we used the LSI and Sprinkled LSI technique to reduce the\ndimension for deception detection. We make our contribution to demonstrate what\nLSI is capturing in latent semantic space and reveal how deceptive opinions can\nbe recognized automatically from truthful opinions. Finally, we proposed a\nvoting scheme which integrates different approaches to further improve the\nclassification performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4714v2",
            "date": "2014-09-16 17:49:31+00:00",
            "title": "Modeling the average shortest path length in growth of word-adjacency networks",
            "authors": [
                "Andrzej Kulig",
                "Stanislaw Drozdz",
                "Jaroslaw Kwapien",
                "Pawel Oswiecimka"
            ],
            "abstract": "We investigate properties of evolving linguistic networks defined by the\nword-adjacency relation. Such networks belong to the category of networks with\naccelerated growth but their shortest path length appears to reveal the network\nsize dependence of different functional form than the ones known so far. We\nthus compare the networks created from literary texts with their artificial\nsubstitutes based on different variants of the Dorogovtsev-Mendes model and\nobserve that none of them is able to properly simulate the novel asymptotics of\nthe shortest path length. Then, we identify the local chain-like linear growth\ninduced by grammar and style as a missing element in this model and extend it\nby incorporating such effects. It is in this way that a satisfactory agreement\nwith the empirical result is obtained.",
            "journal ref": "Phys. Rev. E. 91, 032810 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.5623v2",
            "date": "2014-09-19 12:26:39+00:00",
            "title": "Interactive Visual Exploration of Topic Models using Graphs",
            "authors": [
                "Samuel R\u00f6nnqvist",
                "Xiaolu Wang",
                "Peter Sarlin"
            ],
            "abstract": "Probabilistic topic modeling is a popular and powerful family of tools for\nuncovering thematic structure in large sets of unstructured text documents.\nWhile much attention has been directed towards the modeling algorithms and\ntheir various extensions, comparatively few studies have concerned how to\npresent or visualize topic models in meaningful ways. In this paper, we present\na novel design that uses graphs to visually communicate topic structure and\nmeaning. By connecting topic nodes via descriptive keyterms, the graph\nrepresentation reveals topic similarities, topic meaning and shared, ambiguous\nkeyterms. At the same time, the graph can be used for information retrieval\npurposes, to find documents by topic or topic subsets. To exemplify the utility\nof the design, we illustrate its use for organizing and exploring corpora of\nfinancial patents.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1409.7612v1",
            "date": "2014-09-25 15:18:44+00:00",
            "title": "Semi-supervised Classification for Natural Language Processing",
            "authors": [
                "Rushdi Shams"
            ],
            "abstract": "Semi-supervised classification is an interesting idea where classification\nmodels are learned from both labeled and unlabeled data. It has several\nadvantages over supervised classification in natural language processing\ndomain. For instance, supervised classification exploits only labeled data that\nare expensive, often difficult to get, inadequate in quantity, and require\nhuman experts for annotation. On the other hand, unlabeled data are inexpensive\nand abundant. Despite the fact that many factors limit the wide-spread use of\nsemi-supervised classification, it has become popular since its level of\nperformance is empirically as good as supervised classification. This study\nexplores the possibilities and achievements as well as complexity and\nlimitations of semi-supervised classification for several natural langue\nprocessing tasks like parsing, biomedical information processing, text\nclassification, and summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.8152v1",
            "date": "2014-09-29 15:23:50+00:00",
            "title": "Controversy and Sentiment in Online News",
            "authors": [
                "Yelena Mejova",
                "Amy X. Zhang",
                "Nicholas Diakopoulos",
                "Carlos Castillo"
            ],
            "abstract": "How do news sources tackle controversial issues? In this work, we take a\ndata-driven approach to understand how controversy interplays with emotional\nexpression and biased language in the news. We begin by introducing a new\ndataset of controversial and non-controversial terms collected using\ncrowdsourcing. Then, focusing on 15 major U.S. news outlets, we compare\nmillions of articles discussing controversial and non-controversial issues over\na span of 7 months. We find that in general, when it comes to controversial\nissues, the use of negative affect and biased language is prevalent, while the\nuse of strong emotion is tempered. We also observe many differences across news\nsources. Using these findings, we show that we can indicate to what extent an\nissue is controversial, by comparing it with other issues in terms of how they\nare portrayed across different media.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1409.8309v1",
            "date": "2014-09-29 20:18:12+00:00",
            "title": "Arabic Spelling Correction using Supervised Learning",
            "authors": [
                "Youssef Hassan",
                "Mohamed Aly",
                "Amir Atiya"
            ],
            "abstract": "In this work, we address the problem of spelling correction in the Arabic\nlanguage utilizing the new corpus provided by QALB (Qatar Arabic Language Bank)\nproject which is an annotated corpus of sentences with errors and their\ncorrections. The corpus contains edit, add before, split, merge, add after,\nmove and other error types. We are concerned with the first four error types as\nthey contribute more than 90% of the spelling errors in the corpus. The\nproposed system has many models to address each error type on its own and then\nintegrating all the models to provide an efficient and robust system that\nachieves an overall recall of 0.59, precision of 0.58 and F1 score of 0.58\nincluding all the error types on the development set. Our system participated\nin the QALB 2014 shared task \"Automatic Arabic Error Correction\" and achieved\nan F1 score of 0.6, earning the sixth place out of nine participants.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.0879v1",
            "date": "2014-12-02 12:15:18+00:00",
            "title": "Watsonsim: Overview of a Question Answering Engine",
            "authors": [
                "Sean Gallagher",
                "Wlodek Zadrozny",
                "Walid Shalaby",
                "Adarsh Avadhani"
            ],
            "abstract": "The objective of the project is to design and run a system similar to Watson,\ndesigned to answer Jeopardy questions. In the course of a semester, we\ndeveloped an open source question answering system using the Indri, Lucene,\nBing and Google search engines, Apache UIMA, Open- and CoreNLP, and Weka among\nadditional modules. By the end of the semester, we achieved 18% accuracy on\nJeopardy questions, and work has not stopped since then.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1454v2",
            "date": "2014-12-03 19:42:12+00:00",
            "title": "Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation",
            "authors": [
                "Noam Shazeer",
                "Joris Pelemans",
                "Ciprian Chelba"
            ],
            "abstract": "We present a novel family of language model (LM) estimation techniques named\nSparse Non-negative Matrix (SNM) estimation. A first set of experiments\nempirically evaluating it on the One Billion Word Benchmark shows that SNM\n$n$-gram LMs perform almost as well as the well-established Kneser-Ney (KN)\nmodels. When using skip-gram features the models are able to match the\nstate-of-the-art recurrent neural network (RNN) LMs; combining the two modeling\ntechniques yields the best known result on the benchmark. The computational\nadvantages of SNM over both maximum entropy and RNN LM estimation are probably\nits main strength, promising an approach that has the same flexibility in\ncombining arbitrary features effectively and yet should scale to very large\namounts of data as gracefully as $n$-gram LMs do.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.2487v2",
            "date": "2014-12-08 09:08:32+00:00",
            "title": "Word learning under infinite uncertainty",
            "authors": [
                "Richard A. Blythe",
                "Andrew D. M. Smith",
                "Kenny Smith"
            ],
            "abstract": "Language learners must learn the meanings of many thousands of words, despite\nthose words occurring in complex environments in which infinitely many meanings\nmight be inferred by the learner as a word's true meaning. This problem of\ninfinite referential uncertainty is often attributed to Willard Van Orman\nQuine. We provide a mathematical formalisation of an ideal cross-situational\nlearner attempting to learn under infinite referential uncertainty, and\nidentify conditions under which word learning is possible. As Quine's\nintuitions suggest, learning under infinite uncertainty is in fact possible,\nprovided that learners have some means of ranking candidate word meanings in\nterms of their plausibility; furthermore, our analysis shows that this ranking\ncould in fact be exceedingly weak, implying that constraints which allow\nlearners to infer the plausibility of candidate word meanings could themselves\nbe weak. This approach lifts the burden of explanation from `smart' word\nlearning constraints in learners, and suggests a programme of research into\nweak, unreliable, probabilistic constraints on the inference of word meaning in\nreal word learners.",
            "journal ref": "Cognition (2016) v151 pp18-27",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1412.4160v4",
            "date": "2014-12-12 23:30:06+00:00",
            "title": "Ripple Down Rules for Question Answering",
            "authors": [
                "Dat Quoc Nguyen",
                "Dai Quoc Nguyen",
                "Son Bao Pham"
            ],
            "abstract": "Recent years have witnessed a new trend of building ontology-based question\nanswering systems. These systems use semantic web information to produce more\nprecise answers to users' queries. However, these systems are mostly designed\nfor English. In this paper, we introduce an ontology-based question answering\nsystem named KbQAS which, to the best of our knowledge, is the first one made\nfor Vietnamese. KbQAS employs our question analysis approach that\nsystematically constructs a knowledge base of grammar rules to convert each\ninput question into an intermediate representation element. KbQAS then takes\nthe intermediate representation element with respect to a target ontology and\napplies concept-matching techniques to return an answer. On a wide range of\nVietnamese questions, experimental results show that the performance of KbQAS\nis promising with accuracies of 84.1% and 82.4% for analyzing input questions\nand retrieving output answers, respectively. Furthermore, our question analysis\napproach can easily be applied to new domains and new languages, thus saving\ntime and human effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4314v2",
            "date": "2014-12-14 05:34:25+00:00",
            "title": "Recurrent-Neural-Network for Language Detection on Twitter Code-Switching Corpus",
            "authors": [
                "Joseph Chee Chang",
                "Chu-Cheng Lin"
            ],
            "abstract": "Mixed language data is one of the difficult yet less explored domains of\nnatural language processing. Most research in fields like machine translation\nor sentiment analysis assume monolingual input. However, people who are capable\nof using more than one language often communicate using multiple languages at\nthe same time. Sociolinguists believe this \"code-switching\" phenomenon to be\nsocially motivated. For example, to express solidarity or to establish\nauthority. Most past work depend on external tools or resources, such as\npart-of-speech tagging, dictionary look-up, or named-entity recognizers to\nextract rich features for training machine learning models. In this paper, we\ntrain recurrent neural networks with only raw features, and use word embedding\nto automatically learn meaningful representations. Using the same\nmixed-language Twitter corpus, our system is able to outperform the best\nSVM-based systems reported in the EMNLP'14 Code-Switching Workshop by 1% in\naccuracy, or by 17% in error rate reduction.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1412.4385v3",
            "date": "2014-12-14 17:44:58+00:00",
            "title": "Unsupervised Domain Adaptation with Feature Embeddings",
            "authors": [
                "Yi Yang",
                "Jacob Eisenstein"
            ],
            "abstract": "Representation learning is the dominant technique for unsupervised domain\nadaptation, but existing approaches often require the specification of \"pivot\nfeatures\" that generalize across domains, which are selected by task-specific\nheuristics. We show that a novel but simple feature embedding approach provides\nbetter performance, by exploiting the feature template structure common in NLP\nproblems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4401v1",
            "date": "2014-12-14 20:03:36+00:00",
            "title": "Tools for Terminology Processing",
            "authors": [
                "C. Enguehard",
                "B. Daille",
                "E. Morin"
            ],
            "abstract": "Automatic terminology processing appeared 10 years ago when electronic\ncorpora became widely available. Such processing may be statistically or\nlinguistically based and produces terminology resources that can be used in a\nnumber of applications : indexing, information retrieval, technology watch,\netc. We present the tools that have been developed in the IRIN Institute. They\nall take as input texts (or collection of texts) and reflect different states\nof terminology processing: term acquisition, term recognition and term\nstructuring.",
            "journal ref": "R. K. Arora, M. Kulkarni, H. Darbari. The Indo-European Conference\n  on Multilingual Communications Technologies (IEMCT), Jun 2002, Pune, India.\n  Tata McGraw-Hill, pp.218 - 229",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1412.4616v1",
            "date": "2014-12-15 14:34:38+00:00",
            "title": "A Broadcast News Corpus for Evaluation and Tuning of German LVCSR Systems",
            "authors": [
                "Felix Weninger",
                "Bj\u00f6rn Schuller",
                "Florian Eyben",
                "Martin W\u00f6llmer",
                "Gerhard Rigoll"
            ],
            "abstract": "Transcription of broadcast news is an interesting and challenging application\nfor large-vocabulary continuous speech recognition (LVCSR). We present in\ndetail the structure of a manually segmented and annotated corpus including\nover 160 hours of German broadcast news, and propose it as an evaluation\nframework of LVCSR systems. We show our own experimental results on the corpus,\nachieved with a state-of-the-art LVCSR decoder, measuring the effect of\ndifferent feature sets and decoding parameters, and thereby demonstrate that\nreal-time decoding of our test set is feasible on a desktop PC at 9.2% word\nerror rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.4729v3",
            "date": "2014-12-15 19:21:50+00:00",
            "title": "Translating Videos to Natural Language Using Deep Recurrent Neural Networks",
            "authors": [
                "Subhashini Venugopalan",
                "Huijuan Xu",
                "Jeff Donahue",
                "Marcus Rohrbach",
                "Raymond Mooney",
                "Kate Saenko"
            ],
            "abstract": "Solving the visual symbol grounding problem has long been a goal of\nartificial intelligence. The field appears to be advancing closer to this goal\nwith recent breakthroughs in deep learning for natural language grounding in\nstatic images. In this paper, we propose to translate videos directly to\nsentences using a unified deep neural network with both convolutional and\nrecurrent structure. Described video datasets are scarce, and most existing\nmethods have been applied to toy domains with a small vocabulary of possible\nwords. By transferring knowledge from 1.2M+ images with category labels and\n100,000+ images with captions, our method is able to create sentence\ndescriptions of open-domain videos with large vocabularies. We compare our\napproach with recent work using language generation metrics, subject, verb, and\nobject prediction accuracy, and a human evaluation.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1412.4846v2",
            "date": "2014-12-16 00:34:37+00:00",
            "title": "Scaling laws in human speech, decreasing emergence of new words and a generalized model",
            "authors": [
                "Ruokuang Lin",
                "Qianli D. Y. Ma",
                "Chunhua Bian"
            ],
            "abstract": "Human language, as a typical complex system, its organization and evolution\nis an attractive topic for both physical and cultural researchers. In this\npaper, we present the first exhaustive analysis of the text organization of\nhuman speech. Two important results are that: (i) the construction and\norganization of spoken language can be characterized as Zipf's law and Heaps'\nlaw, as observed in written texts; (ii) word frequency vs. rank distribution\nand the growth of distinct words with the increase of text length shows\nsignificant differences between book and speech. In speech word frequency\ndistribution are more concentrated on higher frequency words, and the emergence\nof new words decreases much rapidly when the content length grows. Based on\nthese observations, a new generalized model is proposed to explain these\ncomplex dynamical behaviors and the differences between speech and book.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5404v1",
            "date": "2014-12-17 14:18:52+00:00",
            "title": "Word Network Topic Model: A Simple but General Solution for Short and Imbalanced Texts",
            "authors": [
                "Yuan Zuo",
                "Jichang Zhao",
                "Ke Xu"
            ],
            "abstract": "The short text has been the prevalent format for information of Internet in\nrecent decades, especially with the development of online social media, whose\nmillions of users generate a vast number of short messages everyday. Although\nsophisticated signals delivered by the short text make it a promising source\nfor topic modeling, its extreme sparsity and imbalance brings unprecedented\nchallenges to conventional topic models like LDA and its variants. Aiming at\npresenting a simple but general solution for topic modeling in short texts, we\npresent a word co-occurrence network based model named WNTM to tackle the\nsparsity and imbalance simultaneously. Different from previous approaches, WNTM\nmodels the distribution over topics for each word instead of learning topics\nfor each document, which successfully enhance the semantic density of data\nspace without importing too much time or space complexity. Meanwhile, the rich\ncontextual information preserved in the word-word space also guarantees its\nsensitivity in identifying rare topics with convincing quality. Furthermore,\nemploying the same Gibbs sampling with LDA makes WNTM easily to be extended to\nvarious application scenarios. Extensive validations on both short and normal\ntexts testify the outperformance of WNTM as compared to baseline methods. And\nfinally we also demonstrate its potential in precisely discovering newly\nemerging topics or unexpected events in Weibo at pretty early stages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5448v1",
            "date": "2014-12-17 15:46:28+00:00",
            "title": "Extended Recommendation Framework: Generating the Text of a User Review as a Personalized Summary",
            "authors": [
                "Micka\u00ebl Poussevin",
                "Vincent Guigue",
                "Patrick Gallinari"
            ],
            "abstract": "We propose to augment rating based recommender systems by providing the user\nwith additional information which might help him in his choice or in the\nunderstanding of the recommendation. We consider here as a new task, the\ngeneration of personalized reviews associated to items. We use an extractive\nsummary formulation for generating these reviews. We also show that the two\ninformation sources, ratings and items could be used both for estimating\nratings and for generating summaries, leading to improved performance for each\nsystem compared to the use of a single source. Besides these two contributions,\nwe show how a personalized polarity classifier can integrate the rating and\ntextual aspects. Overall, the proposed system offers the user three\npersonalized hints for a recommendation: rating, text and polarity. We evaluate\nthese three components on two datasets using appropriate measures for each\ntask.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1412.5659v1",
            "date": "2014-12-17 22:41:14+00:00",
            "title": "Effective sampling for large-scale automated writing evaluation systems",
            "authors": [
                "Nicholas Dronen",
                "Peter W. Foltz",
                "Kyle Habermehl"
            ],
            "abstract": "Automated writing evaluation (AWE) has been shown to be an effective\nmechanism for quickly providing feedback to students. It has already seen wide\nadoption in enterprise-scale applications and is starting to be adopted in\nlarge-scale contexts. Training an AWE model has historically required a single\nbatch of several hundred writing examples and human scores for each of them.\nThis requirement limits large-scale adoption of AWE since human-scoring essays\nis costly. Here we evaluate algorithms for ensuring that AWE models are\nconsistently trained using the most informative essays. Our results show how to\nminimize training set sizes while maximizing predictive performance, thereby\nreducing cost without unduly sacrificing accuracy. We conclude with a\ndiscussion of how to integrate this approach into large-scale AWE systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5673v3",
            "date": "2014-12-17 23:26:48+00:00",
            "title": "Entity-Augmented Distributional Semantics for Discourse Relations",
            "authors": [
                "Yangfeng Ji",
                "Jacob Eisenstein"
            ],
            "abstract": "Discourse relations bind smaller linguistic elements into coherent texts.\nHowever, automatically identifying discourse relations is difficult, because it\nrequires understanding the semantics of the linked sentences. A more subtle\nchallenge is that it is not enough to represent the meaning of each sentence of\na discourse relation, because the relation may depend on links between\nlower-level elements, such as entity mentions. Our solution computes\ndistributional meaning representations by composition up the syntactic parse\ntree. A key difference from previous work on compositional distributional\nsemantics is that we also compute representations for entity mentions, using a\nnovel downward compositional pass. Discourse relations are predicted not only\nfrom the distributional representations of the sentences, but also of their\ncoreferent entity mentions. The resulting system obtains substantial\nimprovements over the previous state-of-the-art in predicting implicit\ndiscourse relations in the Penn Discourse Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6069v1",
            "date": "2014-10-07 08:17:03+00:00",
            "title": "Annotation as a New Paradigm in Research Archiving",
            "authors": [
                "Dirk Roorda",
                "Charles van den Heuvel"
            ],
            "abstract": "We outline a paradigm to preserve results of digital scholarship, whether\nthey are query results, feature values, or topic assignments. This paradigm is\ncharacterized by using annotations as multifunctional carriers and making them\nportable. The testing grounds we have chosen are two significant enterprises,\none in the history of science, and one in Hebrew scholarship. The first one\n(CKCC) focuses on the results of a project where a Dutch consortium of\nuniversities, research institutes, and cultural heritage institutions\nexperimented for 4 years with language techniques and topic modeling methods\nwith the aim to analyze the emergence of scholarly debates. The data: a complex\nset of about 20.000 letters. The second one (DTHB) is a multi-year effort to\nexpress the linguistic features of the Hebrew bible in a text database, which\nis still growing in detail and sophistication. Versions of this database are\npackaged in commercial bible study software. We state that the results of these\nforms of scholarship require new knowledge management and archive practices.\nOnly when researchers can build efficiently on each other's (intermediate)\nresults, they can achieve the aggregations of quality data by which new\nquestions can be answered, and hidden patterns visualized. Archives are\nrequired to find a balance between preserving authoritative versions of sources\nand supporting collaborative efforts in digital scholarship. Annotations are\npromising vehicles for preserving and reusing research results. Keywords\nannotation, portability, archiving, queries, features, topics, keywords,\nRepublic of Letters, Hebrew text databases.",
            "journal ref": "Proceedings of the American Society for Information Science and\n  Technology; Volume 49, Issue 1, pages 1-10, 2012",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6211v1",
            "date": "2014-12-19 04:31:11+00:00",
            "title": "Multiple Authors Detection: A Quantitative Analysis of Dream of the Red Chamber",
            "authors": [
                "Xianfeng Hu",
                "Yang Wang",
                "Qiang Wu"
            ],
            "abstract": "Inspired by the authorship controversy of Dream of the Red Chamber and the\napplication of machine learning in the study of literary stylometry, we develop\na rigorous new method for the mathematical analysis of authorship by testing\nfor a so-called chrono-divide in writing styles. Our method incorporates some\nof the latest advances in the study of authorship attribution, particularly\ntechniques from support vector machines. By introducing the notion of relative\nfrequency as a feature ranking metric our method proves to be highly effective\nand robust.\n  Applying our method to the Cheng-Gao version of Dream of the Red Chamber has\nled to convincing if not irrefutable evidence that the first $80$ chapters and\nthe last $40$ chapters of the book were written by two different authors.\nFurthermore, our analysis has unexpectedly provided strong support to the\nhypothesis that Chapter 67 was not the work of Cao Xueqin either.\n  We have also tested our method to the other three Great Classical Novels in\nChinese. As expected no chrono-divides have been found. This provides further\nevidence of the robustness of our method.",
            "journal ref": "Advances in Adaptive Data Analysis, Article ID 1450012 (18 pages),\n  2014",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.6568v3",
            "date": "2014-12-20 01:03:46+00:00",
            "title": "Improving zero-shot learning by mitigating the hubness problem",
            "authors": [
                "Georgiana Dinu",
                "Angeliki Lazaridou",
                "Marco Baroni"
            ],
            "abstract": "The zero-shot paradigm exploits vector-based word representations extracted\nfrom text corpora with unsupervised methods to learn general mapping functions\nfrom other feature spaces onto word space, where the words associated to the\nnearest neighbours of the mapped vectors are used as their linguistic labels.\nWe show that the neighbourhoods of the mapped elements are strongly polluted by\nhubs, vectors that tend to be near a high proportion of items, pushing their\ncorrect labels down the neighbour list. After illustrating the problem\nempirically, we propose a simple method to correct it by taking the proximity\ndistribution of potential neighbours across many mapped vectors into account.\nWe show that this correction leads to consistent improvements in realistic\nzero-shot experiments in the cross-lingual, image labeling and image retrieval\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6616v2",
            "date": "2014-12-20 07:07:29+00:00",
            "title": "Outperforming Word2Vec on Analogy Tasks with Random Projections",
            "authors": [
                "Abram Demski",
                "Volkan Ustun",
                "Paul Rosenbloom",
                "Cody Kommers"
            ],
            "abstract": "We present a distributed vector representation based on a simplification of\nthe BEAGLE system, designed in the context of the Sigma cognitive architecture.\nOur method does not require gradient-based training of neural networks, matrix\ndecompositions as with LSA, or convolutions as with BEAGLE. All that is\ninvolved is a sum of random vectors and their pointwise products. Despite the\nsimplicity of this technique, it gives state-of-the-art results on analogy\nproblems, in most cases better than Word2Vec. To explain this success, we\ninterpret it as a dimension reduction via random projection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6623v4",
            "date": "2014-12-20 07:42:40+00:00",
            "title": "Word Representations via Gaussian Embedding",
            "authors": [
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "Current work in lexical distributed representations maps each word to a point\nvector in low-dimensional space. Mapping instead to a density provides many\ninteresting advantages, including better capturing uncertainty about a\nrepresentation and its relationships, expressing asymmetries more naturally\nthan dot product or cosine similarity, and enabling more expressive\nparameterization of decision boundaries. This paper advocates for density-based\ndistributed embeddings and presents a method for learning representations in\nthe space of Gaussian distributions. We compare performance on various word\nembedding benchmarks, investigate the ability of these embeddings to model\nentailment and other asymmetric relationships, and explore novel properties of\nthe representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7004v2",
            "date": "2014-12-22 14:49:19+00:00",
            "title": "Tailoring Word Embeddings for Bilexical Predictions: An Experimental Comparison",
            "authors": [
                "Pranava Swaroop Madhyastha",
                "Xavier Carreras",
                "Ariadna Quattoni"
            ],
            "abstract": "We investigate the problem of inducing word embeddings that are tailored for\na particular bilexical relation. Our learning algorithm takes an existing\nlexical vector space and compresses it such that the resulting word embeddings\nare good predictors for a target bilexical relation. In experiments we show\nthat task-specific embeddings can benefit both the quality and efficiency in\nlexical prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7026v2",
            "date": "2014-12-22 15:34:43+00:00",
            "title": "Language Recognition using Random Indexing",
            "authors": [
                "Aditya Joshi",
                "Johan Halseth",
                "Pentti Kanerva"
            ],
            "abstract": "Random Indexing is a simple implementation of Random Projections with a wide\nrange of applications. It can solve a variety of problems with good accuracy\nwithout introducing much complexity. Here we use it for identifying the\nlanguage of text samples. We present a novel method of generating language\nrepresentation vectors using letter blocks. Further, we show that the method is\neasily implemented and requires little computational power and space.\nExperiments on a number of model parameters illustrate certain properties about\nhigh dimensional sparse vector representations of data. Proof of statistically\nrelevant language vectors are shown through the extremely high success of\nvarious language recognition tasks. On a difficult data set of 21,000 short\nsentences from 21 different languages, our model performs a language\nrecognition task and achieves 97.8% accuracy, comparable to state-of-the-art\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7180v1",
            "date": "2014-12-22 21:44:00+00:00",
            "title": "Bayesian Optimisation for Machine Translation",
            "authors": [
                "Yishu Miao",
                "Ziyu Wang",
                "Phil Blunsom"
            ],
            "abstract": "This paper presents novel Bayesian optimisation algorithms for minimum error\nrate training of statistical machine translation systems. We explore two\nclasses of algorithms for efficiently exploring the translation space, with the\nfirst based on N-best lists and the second based on a hypergraph representation\nthat compactly represents an exponential number of translation options. Our\nalgorithms exhibit faster convergence and are capable of obtaining lower error\nrates than the existing translation model specific approaches, all within a\ngeneric Bayesian optimisation framework. Further more, we also introduce a\nrandom embedding algorithm to scale our approach to sparse high dimensional\nfeature sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7782v1",
            "date": "2014-12-25 03:54:01+00:00",
            "title": "Plagiarism Detection on Electronic Text based Assignments using Vector Space Model (ICIAfS14)",
            "authors": [
                "MAC Jiffriya",
                "MAC Akmal Jahan",
                "Roshan G. Ragel"
            ],
            "abstract": "Plagiarism is known as illegal use of others' part of work or whole work as\none's own in any field such as art, poetry, literature, cinema, research and\nother creative forms of study. Plagiarism is one of the important issues in\nacademic and research fields and giving more concern in academic systems. The\nsituation is even worse with the availability of ample resources on the web.\nThis paper focuses on an effective plagiarism detection tool on identifying\nsuitable intra-corpal plagiarism detection for text based assignments by\ncomparing unigram, bigram, trigram of vector space model with cosine similarity\nmeasure. Manually evaluated, labelled dataset was tested using unigram, bigram\nand trigram vector. Even though trigram vector consumes comparatively more\ntime, it shows better results with the labelled data. In addition, the selected\ntrigram vector space model with cosine similarity measure is compared with\ntri-gram sequence matching technique with Jaccard measure. In the results,\ncosine similarity score shows slightly higher values than the other. Because,\nit focuses on giving more weight for terms that do not frequently exist in the\ndataset and cosine similarity measure using trigram technique is more\npreferable than the other. Therefore, we present our new tool and it could be\nused as an effective tool to evaluate text based electronic assignments and\nminimize the plagiarism among students.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1412.8079v1",
            "date": "2014-12-27 21:00:24+00:00",
            "title": "Persian Sentiment Analyzer: A Framework based on a Novel Feature Selection Method",
            "authors": [
                "Ayoub Bagheri",
                "Mohamad Saraee"
            ],
            "abstract": "In the recent decade, with the enormous growth of digital content in internet\nand databases, sentiment analysis has received more and more attention between\ninformation retrieval and natural language processing researchers. Sentiment\nanalysis aims to use automated tools to detect subjective information from\nreviews. One of the main challenges in sentiment analysis is feature selection.\nFeature selection is widely used as the first stage of analysis and\nclassification tasks to reduce the dimension of problem, and improve speed by\nthe elimination of irrelevant and redundant features. Up to now as there are\nfew researches conducted on feature selection in sentiment analysis, there are\nvery rare works for Persian sentiment analysis. This paper considers the\nproblem of sentiment classification using different feature selection methods\nfor online customer reviews in Persian language. Three of the challenges of\nPersian text are using of a wide variety of declensional suffixes, different\nword spacing and many informal or colloquial words. In this paper we study\nthese challenges by proposing a model for sentiment classification of Persian\nreview documents. The proposed model is based on lemmatization and feature\nselection and is employed Naive Bayes algorithm for classification. We evaluate\nthe performance of the model on a manually gathered collection of cellphone\nreviews, where the results show the effectiveness of the proposed approaches.",
            "journal ref": "International Journal of Artificial Intelligence 12.2 (2014):\n  115-129",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.8319v2",
            "date": "2014-12-29 12:00:25+00:00",
            "title": "Quantifying origin and character of long-range correlations in narrative texts",
            "authors": [
                "Stanis\u0142aw Dro\u017cd\u017c",
                "Pawe\u0142 O\u015bwi\u0119cimka",
                "Andrzej Kulig",
                "Jaros\u0142aw Kwapie\u0144",
                "Katarzyna Bazarnik",
                "Iwona Grabska-Gradzi\u0144ska",
                "Jan Rybicki",
                "Marek Stanuszek"
            ],
            "abstract": "In natural language using short sentences is considered efficient for\ncommunication. However, a text composed exclusively of such sentences looks\ntechnical and reads boring. A text composed of long ones, on the other hand,\ndemands significantly more effort for comprehension. Studying characteristics\nof the sentence length variability (SLV) in a large corpus of world-famous\nliterary texts shows that an appealing and aesthetic optimum appears somewhere\nin between and involves selfsimilar, cascade-like alternation of various\nlengths sentences. A related quantitative observation is that the power spectra\nS(f) of thus characterized SLV universally develop a convincing `1/f^beta'\nscaling with the average exponent beta =~ 1/2, close to what has been\nidentified before in musical compositions or in the brain waves. An\noverwhelming majority of the studied texts simply obeys such fractal attributes\nbut especially spectacular in this respect are hypertext-like, \"stream of\nconsciousness\" novels. In addition, they appear to develop structures\ncharacteristic of irreducibly interwoven sets of fractals called multifractals.\nScaling of S(f) in the present context implies existence of the long-range\ncorrelations in texts and appearance of multifractality indicates that they\ncarry even a nonlinear component. A distinct role of the full stops in inducing\nthe long-range correlations in texts is evidenced by the fact that the above\nquantitative characteristics on the long-range correlations manifest themselves\nin variation of the full stops recurrence times along texts, thus in SLV, but\nto a much lesser degree in the recurrence times of the most frequent words. In\nthis latter case the nonlinear correlations, thus multifractality, disappear\neven completely for all the texts considered. Treated as one extra word, the\nfull stops at the same time appear to obey the Zipfian rank-frequency\ndistribution, however.",
            "journal ref": "Information Sciences 331 (2016) 32-44",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.8504v1",
            "date": "2014-12-29 23:09:13+00:00",
            "title": "Probing the topological properties of complex networks modeling short written texts",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "In recent years, graph theory has been widely employed to probe several\nlanguage properties. More specifically, the so-called word adjacency model has\nbeen proven useful for tackling several practical problems, especially those\nrelying on textual stylistic analysis. The most common approach to treat texts\nas networks has simply considered either large pieces of texts or entire books.\nThis approach has certainly worked well -- many informative discoveries have\nbeen made this way -- but it raises an uncomfortable question: could there be\nimportant topological patterns in small pieces of texts? To address this\nproblem, the topological properties of subtexts sampled from entire books was\nprobed. Statistical analyzes performed on a dataset comprising 50 novels\nrevealed that most of the traditional topological measurements are stable for\nshort subtexts. When the performance of the authorship recognition task was\nanalyzed, it was found that a proper sampling yields a discriminability similar\nto the one found with full texts. Surprisingly, the support vector machine\nclassification based on the characterization of short texts outperformed the\none performed with entire books. These findings suggest that a local\ntopological analysis of large documents might improve its global\ncharacterization. Most importantly, it was verified, as a proof of principle,\nthat short texts can be analyzed with the methods and concepts of complex\nnetworks. As a consequence, the techniques described here can be extended in a\nstraightforward fashion to analyze texts as time-varying complex networks.",
            "journal ref": "PLoS ONE 10(2): e0118394, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.0543v1",
            "date": "2014-02-03 23:09:28+00:00",
            "title": "How Does Latent Semantic Analysis Work? A Visualisation Approach",
            "authors": [
                "Jan Koeman",
                "William Rea"
            ],
            "abstract": "By using a small example, an analogy to photographic compression, and a\nsimple visualization using heatmaps, we show that latent semantic analysis\n(LSA) is able to extract what appears to be semantic meaning of words from a\nset of documents by blurring the distinctions between the words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.0556v1",
            "date": "2014-02-04 01:33:10+00:00",
            "title": "Generating Extractive Summaries of Scientific Paradigms",
            "authors": [
                "Vahed Qazvinian",
                "Dragomir R. Radev",
                "Saif M. Mohammad",
                "Bonnie Dorr",
                "David Zajic",
                "Michael Whidby",
                "Taesun Moon"
            ],
            "abstract": "Researchers and scientists increasingly find themselves in the position of\nhaving to quickly understand large amounts of technical material. Our goal is\nto effectively serve this need by using bibliometric text mining and\nsummarization techniques to generate summaries of scientific literature. We\nshow how we can use citations to produce automatically generated, readily\nconsumable, technical extractive summaries. We first propose C-LexRank, a model\nfor summarizing single scientific articles based on citations, which employs\ncommunity detection and extracts salient information-rich sentences. Next, we\nfurther extend our experiments to summarize a set of papers, which cover the\nsame scientific topic. We generate extractive summaries of a set of Question\nAnswering (QA) and Dependency Parsing (DP) papers, their abstracts, and their\ncitation sentences and show that citations have unique information amenable to\ncreating a summary.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 46, pages\n  165-201, 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1402.1668v1",
            "date": "2014-02-07 15:33:44+00:00",
            "title": "Evaluation of YTEX and MetaMap for clinical concept recognition",
            "authors": [
                "John David Osborne",
                "Binod Gyawali",
                "Thamar Solorio"
            ],
            "abstract": "We used MetaMap and YTEX as a basis for the construc- tion of two separate\nsystems to participate in the 2013 ShARe/CLEF eHealth Task 1[9], the\nrecognition of clinical concepts. No modifications were directly made to these\nsystems, but output concepts were filtered using stop concepts, stop concept\ntext and UMLS semantic type. Con- cept boundaries were also adjusted using a\nsmall collection of rules to increase precision on the strict task. Overall\nMetaMap had better per- formance than YTEX on the strict task, primarily due to\na 20% perfor- mance improvement in precision. In the relaxed task YTEX had\nbetter performance in both precision and recall giving it an overall F-Score\n4.6% higher than MetaMap on the test data. Our results also indicated a 1.3%\nhigher accuracy for YTEX in UMLS CUI mapping.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1402.1939v2",
            "date": "2014-02-09 11:47:15+00:00",
            "title": "Maximum Entropy, Word-Frequency, Chinese Characters, and Multiple Meanings",
            "authors": [
                "Xiao-Yong Yan",
                "Petter Minnhagen"
            ],
            "abstract": "The word-frequency distribution of a text written by an author is well\naccounted for by a maximum entropy distribution, the RGF (random group\nformation)-prediction. The RGF-distribution is completely determined by the a\npriori values of the total number of words in the text (M), the number of\ndistinct words (N) and the number of repetitions of the most common word\n(k_max). It is here shown that this maximum entropy prediction also describes a\ntext written in Chinese characters. In particular it is shown that although the\nsame Chinese text written in words and Chinese characters have quite\ndifferently shaped distributions, they are nevertheless both well predicted by\ntheir respective three a priori characteristic values. It is pointed out that\nthis is analogous to the change in the shape of the distribution when\ntranslating a given text to another language. Another consequence of the\nRGF-prediction is that taking a part of a long text will change the input\nparameters (M, N, k_max) and consequently also the shape of the frequency\ndistribution. This is explicitly confirmed for texts written in Chinese\ncharacters. Since the RGF-prediction has no system-specific information beyond\nthe three a priori values (M, N, k_max), any specific language characteristic\nhas to be sought in systematic deviations from the RGF-prediction and the\nmeasured frequencies. One such systematic deviation is identified and, through\na statistical information theoretical argument and an extended RGF-model, it is\nproposed that this deviation is caused by multiple meanings of Chinese\ncharacters. The effect is stronger for Chinese characters than for Chinese\nwords. The relation between Zipf's law, the Simon-model for texts and the\npresent results are discussed.",
            "journal ref": "PLoS ONE 10(5): e0125592 (2015)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1402.2562v1",
            "date": "2014-02-10 20:46:24+00:00",
            "title": "\u00c9tude cognitive des processus de construction d'une requ\u00eate dans un syst\u00e8me de gestion de connaissances m\u00e9dicales",
            "authors": [
                "Nathalie Chaignaud",
                "Val\u00e9rie Delavigne",
                "Maryvonne Holzem",
                "Jean-Philippe Kotowicz",
                "Alain Loisel"
            ],
            "abstract": "This article presents the Cogni-CISMeF project, which aims at improving\nmedical information search in the CISMeF system (Catalog and Index of\nFrench-language health resources) by including a conversational agent to\ninteract with the user in natural language. To study the cognitive processes\ninvolved during the information search, a bottom-up methodology was adopted.\nExperimentation has been set up to obtain human dialogs between a user (playing\nthe role of patient) dealing with medical information search and a CISMeF\nexpert refining the request. The analysis of these dialogs underlined the use\nof discursive evidence: vocabulary, reformulation, implicit or explicit\nexpression of user intentions, conversational sequences, etc. A model of\nartificial agent is proposed. It leads the user in its information search by\nproposing to him examples, assistance and choices. This model was implemented\nand integrated in the CISMeF system. ---- Cet article d\\'ecrit le projet\nCogni-CISMeF qui propose un module de dialogue Homme-Machine \\`a int\\'egrer\ndans le syst\\`eme d'indexation de connaissances m\\'edicales CISMeF (Catalogue\net Index des Sites M\\'edicaux Francophones). Nous avons adopt\\'e une d\\'emarche\nde mod\\'elisation cognitive en proc\\'edant \\`a un recueil de corpus de\ndialogues entre un utilisateur (jouant le r\\^ole d'un patient) d\\'esirant une\ninformation m\\'edicale et un expert CISMeF af inant cette demande pour\nconstruire la requ\\^ete. Nous avons analys\\'e la structure des dialogues ainsi\nobtenus et avons \\'etudi\\'e un certain nombre d'indices discursifs :\nvocabulaire employ\\'e, marques de reformulation, commentaires m\\'eta et\n\\'epilinguistiques, expression implicite ou explicite des intentions de\nl'utilisateur, encha\\^inement conversationnel, etc. De cette analyse, nous\navons construit un mod\\`ele d'agent artificiel dot\\'e de capacit\\'es cognitives\ncapables d'aider l'utilisateur dans sa t\\^ache de recherche d'information. Ce\nmod\\`ele a \\'et\\'e impl\\'ement\\'e et int\\'egr\\'e dans le syst\\`eme CISMeF.",
            "journal ref": "Revue Revue Technique et Science Informatiques 29 (2010) 991-1021",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1402.3080v1",
            "date": "2014-02-13 10:28:43+00:00",
            "title": "Software Requirement Specification Using Reverse Speech Technology",
            "authors": [
                "Santhy Viswam",
                "Sajeer Karattil"
            ],
            "abstract": "Speech analysis had been taken to a new level with the discovery of Reverse\nSpeech (RS). RS is the discovery of hidden messages, referred as reversals, in\nnormal speech. Works are in progress for exploiting the relevance of RS in\ndifferent real world applications such as investigation, medical field etc. In\nthis paper we represent an innovative method for preparing a reliable Software\nRequirement Specification (SRS) document with the help of reverse speech. As\nSRS act as the backbone for the successful completion of any project, a\nreliable method is needed to overcome the inconsistencies. Using RS such a\nreliable method for SRS documentation was developed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3648v1",
            "date": "2014-02-15 05:11:35+00:00",
            "title": "Auto Spell Suggestion for High Quality Speech Synthesis in Hindi",
            "authors": [
                "Shikha Kabra",
                "Ritika Agarwal"
            ],
            "abstract": "The goal of Text-to-Speech (TTS) synthesis in a particular language is to\nconvert arbitrary input text to intelligible and natural sounding speech.\nHowever, for a particular language like Hindi, which is a highly confusing\nlanguage (due to very close spellings), it is not an easy task to identify\nerrors/mistakes in input text and an incorrect text degrade the quality of\noutput speech hence this paper is a contribution to the development of high\nquality speech synthesis with the involvement of Spellchecker which generates\nspell suggestions for misspelled words automatically. Involvement of\nspellchecker would increase the efficiency of speech synthesis by providing\nspell suggestions for incorrect input text. Furthermore, we have provided the\ncomparative study for evaluating the resultant effect on to phonetic text by\nadding spellchecker on to input text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.4259v1",
            "date": "2014-02-18 09:35:19+00:00",
            "title": "Extracting Networks of Characters and Places from Written Works with CHAPLIN",
            "authors": [
                "Roberto Marazzato",
                "Amelia Carolina Sparavigna"
            ],
            "abstract": "We are proposing a tool able to gather information on social networks from\nnarrative texts. Its name is CHAPLIN, CHAracters and PLaces Interaction\nNetwork, implemented in VB.NET. Characters and places of the narrative works\nare extracted in a list of raw words. Aided by the interface, the user selects\nnames out of them. After this choice, the tool allows the user to enter some\nparameters, and, according to them, creates a network where the nodes are the\ncharacters and places, and the edges their interactions. Edges are labelled by\nperformances. The output is a GV file, written in the DOT graph scripting\nlanguage, which is rendered by means of the free open source software Graphviz.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1402.5123v1",
            "date": "2014-02-20 20:15:18+00:00",
            "title": "Detecting Opinions in Tweets",
            "authors": [
                "Abdelmalek Amine",
                "Reda Mohamed Hamou",
                "Michel Simonet"
            ],
            "abstract": "Given the incessant growth of documents describing the opinions of different\npeople circulating on the web, including Web 2.0 has made it possible to give\nan opinion on any product in the net. In this paper, we examine the various\nopinions expressed in the tweets and classify them positive, negative or\nneutral by using the emoticons for the Bayesian method and adjectives and\nadverbs for the Turney's method",
            "journal ref": "International Journal Of Data Mining And Emerging Technologies,\n  Year : 2013, Volume : 3, Issue : 1 First page : ( 23) Last page : ( 32) Print\n  ISSN : 2249-3212. Online ISSN : 2249-3220",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.6764v1",
            "date": "2014-02-27 01:47:46+00:00",
            "title": "A method to identify potential ambiguous Malay words through Ambiguity Attributes mapping: An exploratory Study",
            "authors": [
                "Hazlina Haron",
                "Abdul Azim Abd. Ghani"
            ],
            "abstract": "We describe here a methodology to identify a list of ambiguous Malay words\nthat are commonly being used in Malay documentations such as Requirement\nSpecification. We compiled several relevant and appropriate requirement quality\nattributes and sentence rules from previous literatures and adopt it to come\nout with a set of ambiguity attributes that most suit Malay words. The\nextracted Malay ambiguous words (potential) are then being mapped onto the\nconstructed ambiguity attributes to confirm their vagueness. The list is then\nverified by Malay linguist experts. This paper aims to identify a list of\npotential ambiguous words in Malay as an attempt to assist writers to avoid\nusing the vague words while documenting Malay Requirement Specification as well\nas to any other related Malay documentation. The result of this study is a list\nof 120 potential ambiguous Malay words that could act as guidelines in writing\nMalay sentences",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1403.1194v1",
            "date": "2014-03-05 17:20:01+00:00",
            "title": "Latent Semantic Word Sense Disambiguation Using Global Co-occurrence Information",
            "authors": [
                "Minoru Sasaki"
            ],
            "abstract": "In this paper, I propose a novel word sense disambiguation method based on\nthe global co-occurrence information using NMF. When I calculate the dependency\nrelation matrix, the existing method tends to produce very sparse co-occurrence\nmatrix from a small training set. Therefore, the NMF algorithm sometimes does\nnot converge to desired solutions. To obtain a large number of co-occurrence\nrelations, I propose to use co-occurrence frequencies of dependency relations\nbetween word features in the whole training set. This enables us to solve data\nsparseness problem and induce more effective latent features. To evaluate the\nefficiency of the method of word sense disambiguation, I make some experiments\nto compare with the result of the two baseline methods. The results of the\nexperiments show this method is effective for word sense disambiguation in\ncomparison with the all baseline methods. Moreover, the proposed method is\neffective for obtaining a stable effect by analyzing the global co-occurrence\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.1314v1",
            "date": "2014-03-06 01:33:55+00:00",
            "title": "Authorship detection of SMS messages using unigrams",
            "authors": [
                "R. G. Ragel",
                "P. Herath",
                "U. Senanayake"
            ],
            "abstract": "SMS messaging is a popular media of communication. Because of its popularity\nand privacy, it could be used for many illegal purposes. Additionally, since\nthey are part of the day to day life, SMSes can be used as evidence for many\nlegal disputes. Since a cellular phone might be accessible to people close to\nthe owner, it is important to establish the fact that the sender of the message\nis indeed the owner of the phone. For this purpose, the straight forward\nsolutions seem to be the use of popular stylometric methods. However, in\ncomparison with the data used for stylometry in the literature, SMSes have\nunusual characteristics making it hard or impossible to apply these methods in\na conventional way. Our target is to come up with a method of authorship\ndetection of SMS messages that could still give a usable accuracy. We argue\nthat, considering the methods of author attribution, the best method that could\nbe applied to SMS messages is an n-gram method. To prove our point, we checked\ntwo different methods of distribution comparison with varying number of\ntraining and testing data. We specifically try to compare how well our\nalgorithms work under less amount of testing data and large number of candidate\nauthors (which we believe to be the real world scenario) against controlled\ntests with less number of authors and selected SMSes with large number of\nwords. To counter the lack of information in an SMS message, we propose the\nmethod of stacking together few SMSes.",
            "journal ref": "Industrial and Information Systems (ICIIS), 2013 8th IEEE\n  International Conference on, pp. 387-392 , 17-20 Dec. 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.1618v1",
            "date": "2014-03-06 22:57:29+00:00",
            "title": "Design a Persian Automated Plagiarism Detector (AMZPPD)",
            "authors": [
                "Maryam Mahmoodi",
                "Mohammad Mahmoodi Varnamkhasti"
            ],
            "abstract": "Currently there are lots of plagiarism detection approaches. But few of them\nimplemented and adapted for Persian languages. In this paper, our work on\ndesigning and implementation of a plagiarism detection system based on\npre-processing and NLP technics will be described. And the results of testing\non a corpus will be presented.",
            "journal ref": "International Journal of Engineering Trends and Technology(IJETT),\n  V8(8),465-467 February 2014",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1403.1773v1",
            "date": "2014-03-07 15:13:13+00:00",
            "title": "Finding Eyewitness Tweets During Crises",
            "authors": [
                "Fred Morstatter",
                "Nichola Lubold",
                "Heather Pon-Barry",
                "J\u00fcrgen Pfeffer",
                "Huan Liu"
            ],
            "abstract": "Disaster response agencies have started to incorporate social media as a\nsource of fast-breaking information to understand the needs of people affected\nby the many crises that occur around the world. These agencies look for tweets\nfrom within the region affected by the crisis to get the latest updates of the\nstatus of the affected region. However only 1% of all tweets are geotagged with\nexplicit location information. First responders lose valuable information\nbecause they cannot assess the origin of many of the tweets they collect. In\nthis work we seek to identify non-geotagged tweets that originate from within\nthe crisis region. Towards this, we address three questions: (1) is there a\ndifference between the language of tweets originating within a crisis region\nand tweets originating outside the region, (2) what are the linguistic patterns\nthat can be used to differentiate within-region and outside-region tweets, and\n(3) for non-geotagged tweets, can we automatically identify those originating\nwithin the crisis region in real-time?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.2152v1",
            "date": "2014-03-10 07:26:50+00:00",
            "title": "Parsing using a grammar of word association vectors",
            "authors": [
                "Robert John Freeman"
            ],
            "abstract": "This paper was was first drafted in 2001 as a formalization of the system\ndescribed in U.S. patent U.S. 7,392,174. It describes a system for implementing\na parser based on a kind of cross-product over vectors of contextually similar\nwords. It is being published now in response to nascent interest in vector\ncombination models of syntax and semantics. The method used aggressive\nsubstitution of contextually similar words and word groups to enable product\nvectors to stay in the same space as their operands and make entire sentences\ncomparable syntactically, and potentially semantically. The vectors generated\nhad sufficient representational strength to generate parse trees at least\ncomparable with contemporary symbolic parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.3142v3",
            "date": "2014-03-13 01:07:32+00:00",
            "title": "ARSENAL: Automatic Requirements Specification Extraction from Natural Language",
            "authors": [
                "Shalini Ghosh",
                "Daniel Elenius",
                "Wenchao Li",
                "Patrick Lincoln",
                "Natarajan Shankar",
                "Wilfried Steiner"
            ],
            "abstract": "Requirements are informal and semi-formal descriptions of the expected\nbehavior of a complex system from the viewpoints of its stakeholders\n(customers, users, operators, designers, and engineers). However, for the\npurpose of design, testing, and verification for critical systems, we can\ntransform requirements into formal models that can be analyzed automatically.\nARSENAL is a framework and methodology for systematically transforming natural\nlanguage (NL) requirements into analyzable formal models and logic\nspecifications. These models can be analyzed for consistency and\nimplementability. The ARSENAL methodology is specialized to individual domains,\nbut the approach is general enough to be adapted to new domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.3185v1",
            "date": "2014-03-13 07:48:15+00:00",
            "title": "Sentiment Analysis by Using Fuzzy Logic",
            "authors": [
                "Md. Ansarul Haque"
            ],
            "abstract": "How could a product or service is reasonably evaluated by anyone in the\nshortest time? A million dollar question but it is having a simple answer:\nSentiment analysis. Sentiment analysis is consumers review on products and\nservices which helps both the producers and consumers (stakeholders) to take\neffective and efficient decision within a shortest period of time. Producers\ncan have better knowledge of their products and services through the sentiment\nanalysis (ex. positive and negative comments or consumers likes and dislikes)\nwhich will help them to know their products status (ex. product limitations or\nmarket status). Consumers can have better knowledge of their interested\nproducts and services through the sentiment analysis (ex. positive and negative\ncomments or consumers likes and dislikes) which will help them to know their\ndeserving products status (ex. product limitations or market status). For more\nspecification of the sentiment values, fuzzy logic could be introduced.\nTherefore, sentiment analysis with the help of fuzzy logic (deals with\nreasoning and gives closer views to the exact sentiment values) will help the\nproducers or consumers or any interested person for taking the effective\ndecision according to their product or service interest.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1403.4362v1",
            "date": "2014-03-18 07:18:39+00:00",
            "title": "Concept Based vs. Pseudo Relevance Feedback Performance Evaluation for Information Retrieval System",
            "authors": [
                "Mohammed El Amine Abderrahim"
            ],
            "abstract": "This article evaluates the performance of two techniques for query\nreformulation in a system for information retrieval, namely, the concept based\nand the pseudo relevance feedback reformulation. The experiments performed on a\ncorpus of Arabic text have allowed us to compare the contribution of these two\nreformulation techniques in improving the performance of an information\nretrieval system for Arabic texts.",
            "journal ref": "International Journal of Computational Linguistics Research, ISSN:\n  0976-416X, Volume 4, Issue 4, December, 2013, Pages 149-158",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1403.5596v1",
            "date": "2014-03-22 00:13:03+00:00",
            "title": "A Lemma Based Evaluator for Semitic Language Text Summarization Systems",
            "authors": [
                "Tarek El-Shishtawy",
                "Fatma El-Ghannam"
            ],
            "abstract": "Matching texts in highly inflected languages such as Arabic by simple\nstemming strategy is unlikely to perform well. In this paper, we present a\nstrategy for automatic text matching technique for for inflectional languages,\nusing Arabic as the test case. The system is an extension of ROUGE test in\nwhich texts are matched on token's lemma level. The experimental results show\nan enhancement of detecting similarities between different sentences having\nsame semantics but written in different lexical forms..",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.6023v1",
            "date": "2014-03-24 16:21:04+00:00",
            "title": "Ensemble Detection of Single & Multiple Events at Sentence-Level",
            "authors": [
                "Lu\u00eds Marujo",
                "Anatole Gershman",
                "Jaime Carbonell",
                "Jo\u00e3o P. Neto",
                "David Martins de Matos"
            ],
            "abstract": "Event classification at sentence level is an important Information Extraction\ntask with applications in several NLP, IR, and personalization systems.\nMulti-label binary relevance (BR) are the state-of-art methods. In this work,\nwe explored new multi-label methods known for capturing relations between event\ntypes. These new methods, such as the ensemble Chain of Classifiers, improve\nthe F1 on average across the 6 labels by 2.8% over the Binary Relevance. The\nlow occurrence of multi-label sentences motivated the reduction of the hard\nimbalanced multi-label classification problem with low number of occurrences of\nmultiple labels per instance to an more tractable imbalanced multiclass problem\nwith better results (+ 4.6%). We report the results of adding new features,\nsuch as sentiment strength, rhetorical signals, domain-id (source-id and date),\nand key-phrases in both single-label and multi-label event classification\nscenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.6173v1",
            "date": "2014-03-24 22:28:38+00:00",
            "title": "Coherent Multi-Sentence Video Description with Variable Level of Detail",
            "authors": [
                "Anna Senina",
                "Marcus Rohrbach",
                "Wei Qiu",
                "Annemarie Friedrich",
                "Sikandar Amin",
                "Mykhaylo Andriluka",
                "Manfred Pinkal",
                "Bernt Schiele"
            ],
            "abstract": "Humans can easily describe what they see in a coherent way and at varying\nlevel of detail. However, existing approaches for automatic video description\nare mainly focused on single sentence generation and produce descriptions at a\nfixed level of detail. In this paper, we address both of these limitations: for\na variable level of detail we produce coherent multi-sentence descriptions of\ncomplex videos. We follow a two-step approach where we first learn to predict a\nsemantic representation (SR) from video and then generate natural language\ndescriptions from the SR. To produce consistent multi-sentence descriptions, we\nmodel across-sentence consistency at the level of the SR by enforcing a\nconsistent topic. We also contribute both to the visual recognition of objects\nproposing a hand-centric approach as well as to the robust generation of\nsentences using a word lattice. Human judges rate our multi-sentence\ndescriptions as more readable, correct, and relevant than related work. To\nunderstand the difference between more detailed and shorter descriptions, we\ncollect and analyze a video description corpus of three levels of detail.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1405.0049v2",
            "date": "2014-04-30 22:44:59+00:00",
            "title": "Exemplar Dynamics Models of the Stability of Phonological Categories",
            "authors": [
                "P. F. Tupper"
            ],
            "abstract": "We develop a model for the stability and maintenance of phonological\ncategories. Examples of phonological categories are vowel sounds such as \"i\"\nand \"e\". We model such categories as consisting of collections of labeled\nexemplars that language users store in their memory. Each exemplar is a\ndetailed memory of an instance of the linguistic entity in question. Starting\nfrom an exemplar-level model we derive integro-differential equations for the\nlong-term evolution of the density of exemplars in different portions of\nphonetic space. Using these latter equations we investigate under what\nconditions two phonological categories merge or not. Our main conclusion is\nthat for the preservation of distinct phonological categories, it is necessary\nthat anomalous speech tokens of a given category are discarded, and not merely\nstored in memory as an exemplar of another category.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0941v1",
            "date": "2014-05-05 16:03:04+00:00",
            "title": "Towards a Benchmark of Natural Language Arguments",
            "authors": [
                "Elena Cabrio",
                "Serena Villata"
            ],
            "abstract": "The connections among natural language processing and argumentation theory\nare becoming stronger in the latest years, with a growing amount of works going\nin this direction, in different scenarios and applying heterogeneous\ntechniques. In this paper, we present two datasets we built to cope with the\ncombination of the Textual Entailment framework and bipolar abstract\nargumentation. In our approach, such datasets are used to automatically\nidentify through a Textual Entailment system the relations among the arguments\n(i.e., attack, support), and then the resulting bipolar argumentation graphs\nare analyzed to compute the accepted arguments.",
            "journal ref": "Proceedings of the 15th International Workshop on Non-Monotonic\n  Reasoning (NMR 2014)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1405.1379v1",
            "date": "2014-05-05 14:37:47+00:00",
            "title": "Design and Optimization of a Speech Recognition Front-End for Distant-Talking Control of a Music Playback Device",
            "authors": [
                "Ramin Pichevar",
                "Jason Wung",
                "Daniele Giacobello",
                "Joshua Atkins"
            ],
            "abstract": "This paper addresses the challenging scenario for the distant-talking control\nof a music playback device, a common portable speaker with four small\nloudspeakers in close proximity to one microphone. The user controls the device\nthrough voice, where the speech-to-music ratio can be as low as -30 dB during\nmusic playback. We propose a speech enhancement front-end that relies on known\nrobust methods for echo cancellation, double-talk detection, and noise\nsuppression, as well as a novel adaptive quasi-binary mask that is well suited\nfor speech recognition. The optimization of the system is then formulated as a\nlarge scale nonlinear programming problem where the recognition rate is\nmaximized and the optimal values for the system parameters are found through a\ngenetic algorithm. We validate our methodology by testing over the TIMIT\ndatabase for different music playback levels and noise types. Finally, we show\nthat the proposed front-end allows a natural interaction with the device for\nlimited-vocabulary voice commands.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1405.1605v1",
            "date": "2014-05-07 13:40:47+00:00",
            "title": "DepecheMood: a Lexicon for Emotion Analysis from Crowd-Annotated News",
            "authors": [
                "Jacopo Staiano",
                "Marco Guerini"
            ],
            "abstract": "While many lexica annotated with words polarity are available for sentiment\nanalysis, very few tackle the harder task of emotion analysis and are usually\nquite limited in coverage. In this paper, we present a novel approach for\nextracting - in a totally automated way - a high-coverage and high-precision\nlexicon of roughly 37 thousand terms annotated with emotion scores, called\nDepecheMood. Our approach exploits in an original way 'crowd-sourced' affective\nannotation implicitly provided by readers of news articles from rappler.com. By\nproviding new state-of-the-art performances in unsupervised settings for\nregression and classification tasks, even using a na\\\"{\\i}ve approach, our\nexperiments show the beneficial impact of harvesting social media data for\naffective lexicon building.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.2048v1",
            "date": "2014-05-07 19:47:51+00:00",
            "title": "Learning Alternative Name Spellings",
            "authors": [
                "Jeffrey Sukharev",
                "Leonid Zhukov",
                "Alexandrin Popescul"
            ],
            "abstract": "Name matching is a key component of systems for entity resolution or record\nlinkage. Alternative spellings of the same names are a com- mon occurrence in\nmany applications. We use the largest collection of genealogy person records in\nthe world together with user search query logs to build name matching models.\nThe procedure for building a crowd-sourced training set is outlined together\nwith the presentation of our method. We cast the problem of learning\nalternative spellings as a machine translation problem at the character level.\nWe use in- formation retrieval evaluation methodology to show that this method\nsubstantially outperforms on our data a number of standard well known phonetic\nand string similarity methods in terms of precision and re- call. Additionally,\nwe rigorously compare the performance of standard methods when compared with\neach other. Our result can lead to a significant practical impact in entity\nresolution applications.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.2584v1",
            "date": "2014-05-11 21:05:28+00:00",
            "title": "Sentiment Analysis: A Survey",
            "authors": [
                "Rahul Tejwani"
            ],
            "abstract": "Sentiment analysis (also known as opinion mining) refers to the use of\nnatural language processing, text analysis and computational linguistics to\nidentify and extract subjective information in source materials. Mining\nopinions expressed in the user generated content is a challenging yet\npractically very useful problem. This survey would cover various approaches and\nmethodology used in Sentiment Analysis and Opinion Mining in general. The focus\nwould be on Internet text like, Product review, tweets and other social media.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.3272v2",
            "date": "2014-05-13 19:38:58+00:00",
            "title": "Fast and Fuzzy Private Set Intersection",
            "authors": [
                "Nicholas Kersting"
            ],
            "abstract": "Private Set Intersection (PSI) is usually implemented as a sequence of\nencryption rounds between pairs of users, whereas the present work implements\nPSI in a simpler fashion: each set only needs to be encrypted once, after which\neach pair of users need only one ordinary set comparison. This is typically\norders of magnitude faster than ordinary PSI at the cost of some ``fuzziness\"\nin the matching, which may nonetheless be tolerable or even desirable. This is\ndemonstrated in the case where the sets consist of English words processed with\nWordNet.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1405.3518v2",
            "date": "2014-05-14 14:50:59+00:00",
            "title": "Credibility Adjusted Term Frequency: A Supervised Term Weighting Scheme for Sentiment Analysis and Text Classification",
            "authors": [
                "Yoon Kim",
                "Owen Zhang"
            ],
            "abstract": "We provide a simple but novel supervised weighting scheme for adjusting term\nfrequency in tf-idf for sentiment analysis and text classification. We compare\nour method to baseline weighting schemes and find that it outperforms them on\nmultiple benchmarks. The method is robust and works well on both snippets and\nlonger documents.",
            "journal ref": "Proceedings of the 5th Workshop on Computational Approaches to\n  Subjectivity, Sentiment and Social Media Analysis. June, 2014. 79--83",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3539v3",
            "date": "2014-05-14 15:29:48+00:00",
            "title": "Pattern Recognition in Narrative: Tracking Emotional Expression in Context",
            "authors": [
                "Fionn Murtagh",
                "Adam Ganz"
            ],
            "abstract": "Using geometric data analysis, our objective is the analysis of narrative,\nwith narrative of emotion being the focus in this work. The following two\nprinciples for analysis of emotion inform our work. Firstly, emotion is\nrevealed not as a quality in its own right but rather through interaction. We\nstudy the 2-way relationship of Ilsa and Rick in the movie Casablanca, and the\n3-way relationship of Emma, Charles and Rodolphe in the novel {\\em Madame\nBovary}. Secondly, emotion, that is expression of states of mind of subjects,\nis formed and evolves within the narrative that expresses external events and\n(personal, social, physical) context. In addition to the analysis methodology\nwith key aspects that are innovative, the input data used is crucial. We use,\nfirstly, dialogue, and secondly, broad and general description that\nincorporates dialogue. In a follow-on study, we apply our unsupervised\nnarrative mapping to data streams with very low emotional expression. We map\nthe narrative of Twitter streams. Thus we demonstrate map analysis of general\nnarratives.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2015 (May 26, 2015)\n  jdmdh:647",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1405.3786v1",
            "date": "2014-05-15 09:51:11+00:00",
            "title": "Complex Networks Measures for Differentiation between Normal and Shuffled Croatian Texts",
            "authors": [
                "Domagoj Margan",
                "Ana Me\u0161trovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "This paper studies the properties of the Croatian texts via complex networks.\nWe present network properties of normal and shuffled Croatian texts for\ndifferent shuffling principles: on the sentence level and on the text level. In\nboth experiments we preserved the vocabulary size, word and sentence frequency\ndistributions. Additionally, in the first shuffling approach we preserved the\nsentence structure of the text and the number of words per sentence. Obtained\nresults showed that degree rank distributions exhibit no substantial deviation\nin shuffled networks, and strength rank distributions are preserved due to the\nsame word frequencies. Therefore, standard approach to study the structure of\nlinguistic co-occurrence networks showed no clear difference among the\ntopologies of normal and shuffled texts. Finally, we showed that the in- and\nout- selectivity values from shuffled texts are constantly below selectivity\nvalues calculated from normal texts. Our results corroborate that the node\nselectivity measure can capture structural differences between original and\nshuffled Croatian texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4392v1",
            "date": "2014-05-17 13:29:13+00:00",
            "title": "That's sick dude!: Automatic identification of word sense change across different timescales",
            "authors": [
                "Sunny Mitra",
                "Ritwik Mitra",
                "Martin Riedl",
                "Chris Biemann",
                "Animesh Mukherjee",
                "Pawan Goyal"
            ],
            "abstract": "In this paper, we propose an unsupervised method to identify noun sense\nchanges based on rigorous analysis of time-varying text data available in the\nform of millions of digitized books. We construct distributional thesauri based\nnetworks from data at different time points and cluster each of them separately\nto obtain word-centric sense clusters corresponding to the different time\npoints. Subsequently, we compare these sense clusters of two different time\npoints to find if (i) there is birth of a new sense or (ii) if an older sense\nhas got split into more than one sense or (iii) if a newer sense has been\nformed from the joining of older senses or (iv) if a particular sense has died.\nWe conduct a thorough evaluation of the proposed methodology both manually as\nwell as through comparison with WordNet. Manual evaluation indicates that the\nalgorithm could correctly identify 60.4% birth cases from a set of 48 randomly\npicked samples and 57% split/join cases from a set of 21 randomly picked\nsamples. Remarkably, in 44% cases the birth of a novel sense is attested by\nWordNet, while in 46% cases and 43% cases split and join are respectively\nconfirmed by WordNet. Our approach can be applied for lexicography, as well as\nfor applications like word sense disambiguation or semantic search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4918v1",
            "date": "2014-05-19 23:26:44+00:00",
            "title": "Fighting Authorship Linkability with Crowdsourcing",
            "authors": [
                "Mishari Almishari",
                "Ekin Oguz",
                "Gene Tsudik"
            ],
            "abstract": "Massive amounts of contributed content -- including traditional literature,\nblogs, music, videos, reviews and tweets -- are available on the Internet\ntoday, with authors numbering in many millions. Textual information, such as\nproduct or service reviews, is an important and increasingly popular type of\ncontent that is being used as a foundation of many trendy community-based\nreviewing sites, such as TripAdvisor and Yelp. Some recent results have shown\nthat, due partly to their specialized/topical nature, sets of reviews authored\nby the same person are readily linkable based on simple stylometric features.\nIn practice, this means that individuals who author more than a few reviews\nunder different accounts (whether within one site or across multiple sites) can\nbe linked, which represents a significant loss of privacy.\n  In this paper, we start by showing that the problem is actually worse than\npreviously believed. We then explore ways to mitigate authorship linkability in\ncommunity-based reviewing. We first attempt to harness the global power of\ncrowdsourcing by engaging random strangers into the process of re-writing\nreviews. As our empirical results (obtained from Amazon Mechanical Turk)\nclearly demonstrate, crowdsourcing yields impressively sensible reviews that\nreflect sufficiently different stylometric characteristics such that prior\nstylometric linkability techniques become largely ineffective. We also consider\nusing machine translation to automatically re-write reviews. Contrary to what\nwas previously believed, our results show that translation decreases authorship\nlinkability as the number of intermediate languages grows. Finally, we explore\nthe combination of crowdsourcing and machine translation and report on the\nresults.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5208v1",
            "date": "2014-01-23 02:50:15+00:00",
            "title": "A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing",
            "authors": [
                "Alexander M. Rush",
                "Michael Collins"
            ],
            "abstract": "Dual decomposition, and more generally Lagrangian relaxation, is a classical\nmethod for combinatorial optimization; it has recently been applied to several\ninference problems in natural language processing (NLP). This tutorial gives an\noverview of the technique. We describe example algorithms, describe formal\nguarantees for the method, and describe practical issues in implementing the\nalgorithms. While our examples are predominantly drawn from the NLP literature,\nthe material should be of general relevance to inference problems in machine\nlearning. A central theme of this tutorial is that Lagrangian relaxation is\nnaturally applied in conjunction with a broad class of combinatorial\nalgorithms, allowing inference in models that go significantly beyond previous\nwork on Lagrangian relaxation for inference in graphical models.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 45, pages\n  305-362, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.5447v1",
            "date": "2014-05-20 11:45:06+00:00",
            "title": "Learning to Exploit Different Translation Resources for Cross Language Information Retrieval",
            "authors": [
                "Hosein Azarbonyad",
                "Azadeh Shakery",
                "Heshaam Faili"
            ],
            "abstract": "One of the important factors that affects the performance of Cross Language\nInformation Retrieval(CLIR)is the quality of translations being employed in\nCLIR. In order to improve the quality of translations, it is important to\nexploit available resources efficiently. Employing different translation\nresources with different characteristics has many challenges. In this paper, we\npropose a method for exploiting available translation resources simultaneously.\nThis method employs Learning to Rank(LTR) for exploiting different translation\nresources. To apply LTR methods for query translation, we define different\ntranslation relation based features in addition to context based features. We\nuse the contextual information contained in translation resources for\nextracting context based features.The proposed method uses LTR to construct a\ntranslation ranking model based on defined features. The constructed model is\nused for ranking translation candidates of query words. To evaluate the\nproposed method we do English-Persian CLIR, in which we employ the translation\nranking model to find translations of English queries and employ the\ntranslations to retrieve Persian documents. Experimental results show that our\napproach significantly outperforms single resource based CLIR methods.",
            "journal ref": "International Journal of Information and Communication Technology\n  Research, Volume 6, Issue 1, pp. 55-68, 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.6164v1",
            "date": "2014-04-24 02:47:37+00:00",
            "title": "Generating Natural Language Descriptions from OWL Ontologies: the NaturalOWL System",
            "authors": [
                "Ion Androutsopoulos",
                "Gerasimos Lampouras",
                "Dimitrios Galanis"
            ],
            "abstract": "We present NaturalOWL, a natural language generation system that produces\ntexts describing individuals or classes of OWL ontologies. Unlike simpler OWL\nverbalizers, which typically express a single axiom at a time in controlled,\noften not entirely fluent natural language primarily for the benefit of domain\nexperts, we aim to generate fluent and coherent multi-sentence texts for\nend-users. With a system like NaturalOWL, one can publish information in OWL on\nthe Web, along with automatically produced corresponding texts in multiple\nlanguages, making the information accessible not only to computer programs and\ndomain experts, but also end-users. We discuss the processing stages of\nNaturalOWL, the optional domain-dependent linguistic resources that the system\ncan use at each stage, and why they are useful. We also present trials showing\nthat when the domain-dependent llinguistic resources are available, NaturalOWL\nproduces significantly better texts compared to a simpler verbalizer, and that\nthe resources can be created with relatively light effort.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 48, pages\n  671-715, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.6667v1",
            "date": "2014-05-26 18:25:35+00:00",
            "title": "Inferring gender of a Twitter user using celebrities it follows",
            "authors": [
                "Puneet Singh Ludu"
            ],
            "abstract": "This paper addresses the task of user gender classification in social media,\nwith an application to Twitter. The approach automatically predicts gender by\nleveraging observable information such as the tweet behavior, linguistic\ncontent of the user's Twitter feed and the celebrities followed by the user.\nThis paper first evaluates linguistic content based features using LIWC\ndictionary and popular neighborhood features using Wikipedia and Freebase. Then\naugments both features which yielded a significant increase in the accuracy for\ngender prediction. Results show that rich linguistic features combined with\npopular neighborhood prove valuables and promising for additional user\nclassification needs.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.6678v1",
            "date": "2014-05-26 18:48:15+00:00",
            "title": "Hybrid Type-Logical Grammars, First-Order Linear Logic and the Descriptive Inadequacy of Lambda Grammars",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "In this article we show that hybrid type-logical grammars are a fragment of\nfirst-order linear logic. This embedding result has several important\nconsequences: it not only provides a simple new proof theory for the calculus,\nthereby clarifying the proof-theoretic foundations of hybrid type-logical\ngrammars, but, since the translation is simple and direct, it also provides\nseveral new parsing strategies for hybrid type-logical grammars. Second,\nNP-completeness of hybrid type-logical grammars follows immediately. The main\nembedding result also sheds new light on problems with lambda grammars/abstract\ncategorial grammars and shows lambda grammars/abstract categorial grammars\nsuffer from problems of over-generation and from problems at the\nsyntax-semantics interface unlike any other categorial grammar.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1405.7519v1",
            "date": "2014-05-29 11:05:29+00:00",
            "title": "Aspect Based Sentiment Analysis to Extract Meticulous Opinion Value",
            "authors": [
                "Deepali Virmani",
                "Vikrant Malhotra",
                "Ridhi Tyagi"
            ],
            "abstract": "Opinion Mining and Sentiment Analysis is a process of identifying opinions in\nlarge unstructured/structured data and then analysing polarity of those\nopinions. Opinion mining and sentiment analysis have found vast application in\nanalysing online ratings, analysing product based reviews, e-governance, and\nmanaging hostile content over the internet. This paper proposes an algorithm to\nimplement aspect level sentiment analysis. The algorithm takes input from the\nremarks submitted by various teachers of a student. An aspect tree is formed\nwhich has various levels and weights are assigned to each branch to identify\nlevel of aspect. Aspect value is calculated by the algorithm by means of the\nproposed aspect tree. Dictionary based method is implemented to evaluate the\npolarity of the remark. The algorithm returns the aspect value clubbed with\nopinion value and sentiment value which helps in concluding the summarized\nvalue of remark.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.7975v1",
            "date": "2014-05-17 22:21:00+00:00",
            "title": "Multi-layered graph-based multi-document summarization model",
            "authors": [
                "Ercan Canhasi"
            ],
            "abstract": "Multi-document summarization is a process of automatic generation of a\ncompressed version of the given collection of documents. Recently, the\ngraph-based models and ranking algorithms have been actively investigated by\nthe extractive document summarization community. While most work to date\nfocuses on homogeneous connecteness of sentences and heterogeneous connecteness\nof documents and sentences (e.g. sentence similarity weighted by document\nimportance), in this paper we present a novel 3-layered graph model that\nemphasizes not only sentence and document level relations but also the\ninfluence of under sentence level relations (e.g. a part of sentence\nsimilarity).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1407.1165v1",
            "date": "2014-07-04 09:32:10+00:00",
            "title": "Recognition of Isolated Words using Zernike and MFCC features for Audio Visual Speech Recognition",
            "authors": [
                "Prashant Bordea",
                "Amarsinh Varpeb",
                "Ramesh Manzac",
                "Pravin Yannawara"
            ],
            "abstract": "Automatic Speech Recognition (ASR) by machine is an attractive research topic\nin signal processing domain and has attracted many researchers to contribute in\nthis area. In recent year, there have been many advances in automatic speech\nreading system with the inclusion of audio and visual speech features to\nrecognize words under noisy conditions. The objective of audio-visual speech\nrecognition system is to improve recognition accuracy. In this paper we\ncomputed visual features using Zernike moments and audio feature using Mel\nFrequency Cepstral Coefficients (MFCC) on vVISWa (Visual Vocabulary of\nIndependent Standard Words) dataset which contains collection of isolated set\nof city names of 10 speakers. The visual features were normalized and dimension\nof features set was reduced by Principal Component Analysis (PCA) in order to\nrecognize the isolated word utterance on PCA space.The performance of\nrecognition of isolated words based on visual only and audio only features\nresults in 63.88 and 100 respectively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1407.1640v1",
            "date": "2014-07-07 09:31:21+00:00",
            "title": "WordRep: A Benchmark for Research on Learning Word Representations",
            "authors": [
                "Bin Gao",
                "Jiang Bian",
                "Tie-Yan Liu"
            ],
            "abstract": "WordRep is a benchmark collection for the research on learning distributed\nword representations (or word embeddings), released by Microsoft Research. In\nthis paper, we describe the details of the WordRep collection and show how to\nuse it in different types of machine learning research related to word\nembedding. Specifically, we describe how the evaluation tasks in WordRep are\nselected, how the data are sampled, and how the evaluation tool is built. We\nthen compare several state-of-the-art word representations on WordRep, report\ntheir evaluation performance, and make discussions on the results. After that,\nwe discuss new potential research topics that can be supported by WordRep, in\naddition to algorithm comparison. We hope that this paper can help people gain\ndeeper understanding of WordRep, and enable more interesting research on\nlearning distributed word representations and related topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.1687v3",
            "date": "2014-07-07 12:45:10+00:00",
            "title": "KNET: A General Framework for Learning Word Embedding using Morphological Knowledge",
            "authors": [
                "Qing Cui",
                "Bin Gao",
                "Jiang Bian",
                "Siyu Qiu",
                "Tie-Yan Liu"
            ],
            "abstract": "Neural network techniques are widely applied to obtain high-quality\ndistributed representations of words, i.e., word embeddings, to address text\nmining, information retrieval, and natural language processing tasks. Recently,\nefficient methods have been proposed to learn word embeddings from context that\ncaptures both semantic and syntactic relationships between words. However, it\nis challenging to handle unseen words or rare words with insufficient context.\nIn this paper, inspired by the study on word recognition process in cognitive\npsychology, we propose to take advantage of seemingly less obvious but\nessentially important morphological knowledge to address these challenges. In\nparticular, we introduce a novel neural network architecture called KNET that\nleverages both contextual information and morphological word similarity built\nbased on morphological knowledge to learn word embeddings. Meanwhile, the\nlearning architecture is also able to refine the pre-defined morphological\nknowledge and obtain more accurate word similarity. Experiments on an\nanalogical reasoning task and a word similarity task both demonstrate that the\nproposed KNET framework can greatly enhance the effectiveness of word\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.1933v1",
            "date": "2014-07-08 02:53:29+00:00",
            "title": "Lexpresso: a Controlled Natural Language",
            "authors": [
                "Adam Saulwick"
            ],
            "abstract": "This paper presents an overview of `Lexpresso', a Controlled Natural Language\ndeveloped at the Defence Science & Technology Organisation as a bidirectional\nnatural language interface to a high-level information fusion system. The paper\ndescribes Lexpresso's main features including lexical coverage, expressiveness\nand range of linguistic syntactic and semantic structures. It also touches on\nits tight integration with a formal semantic formalism and tentatively\nclassifies it against the PENS system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.3636v1",
            "date": "2014-07-14 13:22:36+00:00",
            "title": "Toward Network-based Keyword Extraction from Multitopic Web Documents",
            "authors": [
                "Sabina \u0160i\u0161ovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107",
                "Ana Me\u0161trovi\u0107"
            ],
            "abstract": "In this paper we analyse the selectivity measure calculated from the complex\nnetwork in the task of the automatic keyword extraction. Texts, collected from\ndifferent web sources (portals, forums), are represented as directed and\nweighted co-occurrence complex networks of words. Words are nodes and links are\nestablished between two nodes if they are directly co-occurring within the\nsentence. We test different centrality measures for ranking nodes - keyword\ncandidates. The promising results are achieved using the selectivity measure.\nThen we propose an approach which enables extracting word pairs according to\nthe values of the in/out selectivity and weight measures combined with\nfiltering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.3751v1",
            "date": "2014-07-14 18:01:08+00:00",
            "title": "Benchmarking Named Entity Disambiguation approaches for Streaming Graphs",
            "authors": [
                "Sutanay Choudhury",
                "Chase Dowling"
            ],
            "abstract": "Named Entity Disambiaguation (NED) is a central task for applications dealing\nwith natural language text. Assume that we have a graph based knowledge base\n(subsequently referred as Knowledge Graph) where nodes represent various real\nworld entities such as people, location, organization and concepts. Given data\nsources such as social media streams and web pages Entity Linking is the task\nof mapping named entities that are extracted from the data to those present in\nthe Knowledge Graph. This is an inherently difficult task due to several\nreasons. Almost all these data sources are generated without any formal\nontology; the unstructured nature of the input, limited context and the\nambiguity involved when multiple entities are mapped to the same name make this\na hard task. This report looks at two state of the art systems employing two\ndistinctive approaches: graph based Accurate Online Disambiguation of Entities\n(AIDA) and Mined Evidence Named Entity Disambiguation (MENED), which employs a\nstatistical inference approach. We compare both approaches using the data set\nand queries provided by the Knowledge Base Population (KBP) track at 2011 NIST\nText Analytics Conference (TAC). This report begins with an overview of the\nrespective approaches, followed by detailed description of the experimental\nsetup. It concludes with our findings from the benchmarking exercise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.4610v2",
            "date": "2014-07-17 09:38:07+00:00",
            "title": "Understanding Zipf's law of word frequencies through sample-space collapse in sentence formation",
            "authors": [
                "Stefan Thurner",
                "Rudolf Hanel",
                "Bo Liu",
                "Bernat Corominas-Murtra"
            ],
            "abstract": "The formation of sentences is a highly structured and history-dependent\nprocess. The probability of using a specific word in a sentence strongly\ndepends on the 'history' of word-usage earlier in that sentence. We study a\nsimple history-dependent model of text generation assuming that the\nsample-space of word usage reduces along sentence formation, on average. We\nfirst show that the model explains the approximate Zipf law found in word\nfrequencies as a direct consequence of sample-space reduction. We then\nempirically quantify the amount of sample-space reduction in the sentences of\nten famous English books, by analysis of corresponding word-transition tables\nthat capture which words can follow any given word in a text. We find a highly\nnested structure in these transition tables and show that this `nestedness' is\ntightly related to the power law exponents of the observed word frequency\ndistributions. With the proposed model it is possible to understand that the\nnestedness of a text can be the origin of the actual scaling exponent, and that\ndeviations from the exact Zipf law can be understood by variations of the\ndegree of nestedness on a book-by-book basis. On a theoretical level we are\nable to show that in case of weak nesting, Zipf's law breaks down in a fast\ntransition. Unlike previous attempts to understand Zipf's law in language the\nsample-space reducing model is not based on assumptions of multiplicative,\npreferential, or self-organised critical mechanisms behind language formation,\nbut simply used the empirically quantifiable parameter 'nestedness' to\nunderstand the statistics of word frequencies.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1407.6027v1",
            "date": "2014-07-22 20:25:40+00:00",
            "title": "Modeling languages from graph networks",
            "authors": [
                "Alberto Besana",
                "Cristina Mart\u00ednez"
            ],
            "abstract": "We model and compute the probability distribution of the letters in random\ngenerated words in a language by using the theory of set partitions, Young\ntableaux and graph theoretical representation methods. This has been of\ninterest for several application areas such as network systems, bioinformatics,\ninternet search, data mining and computacional linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.6099v1",
            "date": "2014-07-23 03:29:44+00:00",
            "title": "Autonomous requirements specification processing using natural language processing",
            "authors": [
                "S. G. Macdonell",
                "K. Min",
                "A. M. Connor"
            ],
            "abstract": "We describe our ongoing research that centres on the application of natural\nlanguage processing (NLP) to software engineering and systems development\nactivities. In particular, this paper addresses the use of NLP in the\nrequirements analysis and systems design processes. We have developed a\nprototype toolset that can assist the systems analyst or software engineer to\nselect and verify terms relevant to a project. In this paper we describe the\nprocesses employed by the system to extract and classify objects of interest\nfrom requirements documents. These processes are illustrated using a small\nexample.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.6639v3",
            "date": "2014-07-24 16:26:45+00:00",
            "title": "How the Voynich Manuscript was created",
            "authors": [
                "Torsten Timm"
            ],
            "abstract": "The Voynich manuscript is a medieval book written in an unknown script. This\npaper studies the relation between similarly spelled words in the Voynich\nmanuscript. By means of a detailed analysis of similar spelled words it was\npossible to reveal the text generation method used for the Voynich manuscript.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1407.7357v1",
            "date": "2014-07-28 09:00:57+00:00",
            "title": "Text Classification Using Association Rules, Dependency Pruning and Hyperonymization",
            "authors": [
                "Yannis Haralambous",
                "Philippe Lenca"
            ],
            "abstract": "We present new methods for pruning and enhancing item- sets for text\nclassification via association rule mining. Pruning methods are based on\ndependency syntax and enhancing methods are based on replacing words by their\nhyperonyms of various orders. We discuss the impact of these methods, compared\nto pruning based on tfidf rank of words.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1408.0016v1",
            "date": "2014-06-27 01:00:59+00:00",
            "title": "Architecture of a Web-based Predictive Editor for Controlled Natural Language Processing",
            "authors": [
                "Stephen Guy",
                "Rolf Schwitter"
            ],
            "abstract": "In this paper, we describe the architecture of a web-based predictive text\neditor being developed for the controlled natural language PENG$^{ASP)$. This\ncontrolled language can be used to write non-monotonic specifications that have\nthe same expressive power as Answer Set Programs. In order to support the\nwriting process of these specifications, the predictive text editor\ncommunicates asynchronously with the controlled natural language processor that\ngenerates lookahead categories and additional auxiliary information for the\nauthor of a specification text. The text editor can display multiple sets of\nlookahead categories simultaneously for different possible sentence\ncompletions, anaphoric expressions, and supports the addition of new content\nwords to the lexicon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.1031v2",
            "date": "2014-08-01 03:18:56+00:00",
            "title": "Text to Multi-level MindMaps: A Novel Method for Hierarchical Visual Abstraction of Natural Language Text",
            "authors": [
                "Mohamed Elhoseiny",
                "Ahmed Elgammal"
            ],
            "abstract": "MindMapping is a well-known technique used in note taking, which encourages\nlearning and studying. MindMapping has been manually adopted to help present\nknowledge and concepts in a visual form. Unfortunately, there is no reliable\nautomated approach to generate MindMaps from Natural Language text. This work\nfirstly introduces MindMap Multilevel Visualization concept which is to jointly\nvisualize and summarize textual information. The visualization is achieved\npictorially across multiple levels using semantic information (i.e. ontology),\nwhile the summarization is achieved by the information in the highest levels as\nthey represent abstract information in the text. This work also presents the\nfirst automated approach that takes a text input and generates a MindMap\nvisualization out of it. The approach could visualize text documents in\nmultilevel MindMaps, in which a high-level MindMap node could be expanded into\nchild MindMaps. \\ignore{ As far as we know, this is the first work that view\nMindMapping as a new approach to jointly summarize and visualize textual\ninformation.} The proposed method involves understanding of the input text and\nconverting it into intermediate Detailed Meaning Representation (DMR). The DMR\nis then visualized with two modes; Single level or Multiple levels, which is\nconvenient for larger text. The generated MindMaps from both approaches were\nevaluated based on Human Subject experiments performed on Amazon Mechanical\nTurk with various parameter settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.2430v1",
            "date": "2014-08-11 14:58:30+00:00",
            "title": "Optimizing Component Combination in a Multi-Indexing Paragraph Retrieval System",
            "authors": [
                "Boris Iolis",
                "Gianluca Bontempi"
            ],
            "abstract": "We demonstrate a method to optimize the combination of distinct components in\na paragraph retrieval system. Our system makes use of several indices, query\ngenerators and filters, each of them potentially contributing to the quality of\nthe returned list of results. The components are combined with a weighed sum,\nand we optimize the weights using a heuristic optimization algorithm. This\nallows us to maximize the quality of our results, but also to determine which\ncomponents are most valuable in our system. We evaluate our approach on the\nparagraph selection task of a Question Answering dataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1408.2466v1",
            "date": "2014-07-15 02:20:23+00:00",
            "title": "Controlled Natural Language Processing as Answer Set Programming: an Experiment",
            "authors": [
                "Rolf Schwitter"
            ],
            "abstract": "Most controlled natural languages (CNLs) are processed with the help of a\npipeline architecture that relies on different software components. We\ninvestigate in this paper in an experimental way how well answer set\nprogramming (ASP) is suited as a unifying framework for parsing a CNL, deriving\na formal representation for the resulting syntax trees, and for reasoning with\nthat representation. We start from a list of input tokens in ASP notation and\nshow how this input can be transformed into a syntax tree using an ASP grammar\nand then into reified ASP rules in form of a set of facts. These facts are then\nprocessed by an ASP meta-interpreter that allows us to infer new knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.2699v1",
            "date": "2014-08-12 12:07:27+00:00",
            "title": "Internal and external dynamics in language: Evidence from verb regularity in a historical corpus of English",
            "authors": [
                "Christine F. Cuskley",
                "Martina Pugliese",
                "Claudio Castellano",
                "Francesca Colaiori",
                "Vittorio Loreto",
                "Francesca Tria"
            ],
            "abstract": "Human languages are rule governed, but almost invariably these rules have\nexceptions in the form of irregularities. Since rules in language are efficient\nand productive, the persistence of irregularity is an anomaly. How does\nirregularity linger in the face of internal (endogenous) and external\n(exogenous) pressures to conform to a rule? Here we address this problem by\ntaking a detailed look at simple past tense verbs in the Corpus of Historical\nAmerican English. The data show that the language is open, with many new verbs\nentering. At the same time, existing verbs might tend to regularize or\nirregularize as a consequence of internal dynamics, but overall, the amount of\nirregularity sustained by the language stays roughly constant over time.\nDespite continuous vocabulary growth, and presumably, an attendant increase in\nexpressive power, there is no corresponding growth in irregularity. We analyze\nthe set of irregulars, showing they may adhere to a set of minority rules,\nallowing for increased stability of irregularity over time. These findings\ncontribute to the debate on how language systems become rule governed, and how\nand why they sustain exceptions to rules, providing insight into the interplay\nbetween the emergence and maintenance of rules and exceptions in language.",
            "journal ref": "PLoS ONE 9(8): e102882 (2014)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1408.3829v1",
            "date": "2014-08-17 14:56:36+00:00",
            "title": "Opinion mining of movie reviews at document level",
            "authors": [
                "Richa Sharma",
                "Shweta Nigam",
                "Rekha Jain"
            ],
            "abstract": "The whole world is changed rapidly and using the current technologies\nInternet becomes an essential need for everyone. Web is used in every field.\nMost of the people use web for a common purpose like online shopping, chatting\netc. During an online shopping large number of reviews/opinions are given by\nthe users that reflect whether the product is good or bad. These reviews need\nto be explored, analyse and organized for better decision making. Opinion\nMining is a natural language processing task that deals with finding\norientation of opinion in a piece of text with respect to a topic. In this\npaper a document based opinion mining system is proposed that classify the\ndocuments as positive, negative and neutral. Negation is also handled in the\nproposed system. Experimental results using reviews of movies show the\neffectiveness of the system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1408.4245v2",
            "date": "2014-08-19 08:32:49+00:00",
            "title": "Towards crowdsourcing and cooperation in linguistic resources",
            "authors": [
                "Dmitry Ustalov"
            ],
            "abstract": "Linguistic resources can be populated with data through the use of such\napproaches as crowdsourcing and gamification when motivated people are\ninvolved. However, current crowdsourcing genre taxonomies lack the concept of\ncooperation, which is the principal element of modern video games and may\npotentially drive the annotators' interest. This survey on crowdsourcing\ntaxonomies and cooperation in linguistic resources provides recommendations on\nusing cooperation in existent genres of crowdsourcing and an evidence of the\nefficiency of cooperation using a popular Russian linguistic resource created\nthrough crowdsourcing as an example.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1408.5882v2",
            "date": "2014-08-25 19:48:04+00:00",
            "title": "Convolutional Neural Networks for Sentence Classification",
            "authors": [
                "Yoon Kim"
            ],
            "abstract": "We report on a series of experiments with convolutional neural networks (CNN)\ntrained on top of pre-trained word vectors for sentence-level classification\ntasks. We show that a simple CNN with little hyperparameter tuning and static\nvectors achieves excellent results on multiple benchmarks. Learning\ntask-specific vectors through fine-tuning offers further gains in performance.\nWe additionally propose a simple modification to the architecture to allow for\nthe use of both task-specific and static vectors. The CNN models discussed\nherein improve upon the state of the art on 4 out of 7 tasks, which include\nsentiment analysis and question classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.6746v2",
            "date": "2014-08-28 15:06:50+00:00",
            "title": "Non-Standard Words as Features for Text Categorization",
            "authors": [
                "Slobodan Beliga",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "This paper presents categorization of Croatian texts using Non-Standard Words\n(NSW) as features. Non-Standard Words are: numbers, dates, acronyms,\nabbreviations, currency, etc. NSWs in Croatian language are determined\naccording to Croatian NSW taxonomy. For the purpose of this research, 390 text\ndocuments were collected and formed the SKIPEZ collection with 6 classes:\nofficial, literary, informative, popular, educational and scientific. Text\ncategorization experiment was conducted on three different representations of\nthe SKIPEZ collection: in the first representation, the frequencies of NSWs are\nused as features; in the second representation, the statistic measures of NSWs\n(variance, coefficient of variation, standard deviation, etc.) are used as\nfeatures; while the third representation combines the first two feature sets.\nNaive Bayes, CN2, C4.5, kNN, Classification Trees and Random Forest algorithms\nwere used in text categorization experiments. The best categorization results\nare achieved using the first feature set (NSW frequencies) with the\ncategorization accuracy of 87%. This suggests that the NSWs should be\nconsidered as features in highly inflectional languages, such as Croatian. NSW\nbased features reduce the dimensionality of the feature space without standard\nlemmatization procedures, and therefore the bag-of-NSWs should be considered\nfor further Croatian texts categorization experiments.",
            "journal ref": "IEEE 37th International Convention on Information and\n  Communication Technology, Electronics and Microelectronics (MIPRO 2014), pp.\n  1415-1419, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.6762v1",
            "date": "2014-08-28 16:01:10+00:00",
            "title": "Chatbot for admissions",
            "authors": [
                "Nikolaos Polatidis"
            ],
            "abstract": "The communication of potential students with a university department is\nperformed manually and it is a very time consuming procedure. The opportunity\nto communicate with on a one-to-one basis is highly valued. However with many\nhundreds of applications each year, one-to-one conversations are not feasible\nin most cases. The communication will require a member of academic staff to\nexpend several hours to find suitable answers and contact each student. It\nwould be useful to reduce his costs and time.\n  The project aims to reduce the burden on the head of admissions, and\npotentially other users, by developing a convincing chatbot. A suitable\nalgorithm must be devised to search through the set of data and find a\npotential answer. The program then replies to the user and provides a relevant\nweb link if the user is not satisfied by the answer. Furthermore a web\ninterface is provided for both users and an administrator.\n  The achievements of the project can be summarised as follows. To prepare the\nbackground of the project a literature review was undertaken, together with an\ninvestigation of existing tools, and consultation with the head of admissions.\nThe requirements of the system were established and a range of algorithms and\ntools were investigated, including keyword and template matching. An algorithm\nthat combines keyword matching with string similarity has been developed. A\nusable system using the proposed algorithm has been implemented. The system was\nevaluated by keeping logs of questions and answers and by feedback received by\npotential students that used it.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1408.6988v1",
            "date": "2014-08-29 12:04:15+00:00",
            "title": "An Information Retrieval Approach to Short Text Conversation",
            "authors": [
                "Zongcheng Ji",
                "Zhengdong Lu",
                "Hang Li"
            ],
            "abstract": "Human computer conversation is regarded as one of the most difficult problems\nin artificial intelligence. In this paper, we address one of its key\nsub-problems, referred to as short text conversation, in which given a message\nfrom human, the computer returns a reasonable response to the message. We\nleverage the vast amount of short conversation data available on social media\nto study the issue. We propose formalizing short text conversation as a search\nproblem at the first step, and employing state-of-the-art information retrieval\n(IR) techniques to carry out the task. We investigate the significance as well\nas the limitation of the IR approach. Our experiments demonstrate that the\nretrieval-based model can make the system behave rather \"intelligently\", when\ncombined with a huge repository of conversation data from social media.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1411.0129v2",
            "date": "2014-11-01 15:52:05+00:00",
            "title": "The Latent Structure of Dictionaries",
            "authors": [
                "Philippe Vincent-Lamarre",
                "Alexandre Blondin Mass\u00e9",
                "Marcos Lopes",
                "M\u00e9lanie Lord",
                "Odile Marcotte",
                "Stevan Harnad"
            ],
            "abstract": "How many words (and which ones) are sufficient to define all other words?\nWhen dictionaries are analyzed as directed graphs with links from defining\nwords to defined words, they reveal a latent structure. Recursively removing\nall words that are reachable by definition but that do not define any further\nwords reduces the dictionary to a Kernel of about 10%. This is still not the\nsmallest number of words that can define all the rest. About 75% of the Kernel\nturns out to be its Core, a Strongly Connected Subset of words with a\ndefinitional path to and from any pair of its words and no word's definition\ndepending on a word outside the set. But the Core cannot define all the rest of\nthe dictionary. The 25% of the Kernel surrounding the Core consists of small\nstrongly connected subsets of words: the Satellites. The size of the smallest\nset of words that can define all the rest (the graph's Minimum Feedback Vertex\nSet or MinSet) is about 1% of the dictionary, 15% of the Kernel, and half-Core,\nhalf-Satellite. But every dictionary has a huge number of MinSets. The Core\nwords are learned earlier, more frequent, and less concrete than the\nSatellites, which in turn are learned earlier and more frequent but more\nconcrete than the rest of the Dictionary. In principle, only one MinSet's words\nwould need to be grounded through the sensorimotor capacity to recognize and\ncategorize their referents. In a dual-code sensorimotor-symbolic model of the\nmental lexicon, the symbolic code could do all the rest via re-combinatory\ndefinition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.0861v1",
            "date": "2014-11-04 11:06:01+00:00",
            "title": "Using Linguistic Features to Estimate Suicide Probability of Chinese Microblog Users",
            "authors": [
                "Lei Zhang",
                "Xiaolei Huang",
                "Tianli Liu",
                "Zhenxiang Chen",
                "Tingshao Zhu"
            ],
            "abstract": "If people with high risk of suicide can be identified through social media\nlike microblog, it is possible to implement an active intervention system to\nsave their lives. Based on this motivation, the current study administered the\nSuicide Probability Scale(SPS) to 1041 weibo users at Sina Weibo, which is a\nleading microblog service provider in China. Two NLP (Natural Language\nProcessing) methods, the Chinese edition of Linguistic Inquiry and Word Count\n(LIWC) lexicon and Latent Dirichlet Allocation (LDA), are used to extract\nlinguistic features from the Sina Weibo data. We trained predicting models by\nmachine learning algorithm based on these two types of features, to estimate\nsuicide probability based on linguistic features. The experiment results\nindicate that LDA can find topics that relate to suicide probability, and\nimprove the performance of prediction. Our study adds value in prediction of\nsuicidal probability of social network users with their behaviors.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1411.0895v1",
            "date": "2014-11-04 13:11:06+00:00",
            "title": "Tied Probabilistic Linear Discriminant Analysis for Speech Recognition",
            "authors": [
                "Liang Lu",
                "Steve Renals"
            ],
            "abstract": "Acoustic models using probabilistic linear discriminant analysis (PLDA)\ncapture the correlations within feature vectors using subspaces which do not\nvastly expand the model. This allows high dimensional and correlated feature\nspaces to be used, without requiring the estimation of multiple high dimension\ncovariance matrices. In this letter we extend the recently presented PLDA\nmixture model for speech recognition through a tied PLDA approach, which is\nbetter able to control the model size to avoid overfitting. We carried out\nexperiments using the Switchboard corpus, with both mel frequency cepstral\ncoefficient features and bottleneck feature derived from a deep neural network.\nReductions in word error rate were obtained by using tied PLDA, compared with\nthe PLDA mixture model, subspace Gaussian mixture models, and deep neural\nnetworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.1006v2",
            "date": "2014-11-04 19:15:59+00:00",
            "title": "A Probabilistic Translation Method for Dictionary-based Cross-lingual Information Retrieval in Agglutinative Languages",
            "authors": [
                "Javid Dadashkarimi",
                "Azadeh Shakery",
                "Heshaam Faili"
            ],
            "abstract": "Translation ambiguity, out of vocabulary words and missing some translations\nin bilingual dictionaries make dictionary-based Cross-language Information\nRetrieval (CLIR) a challenging task. Moreover, in agglutinative languages which\ndo not have reliable stemmers, missing various lexical formations in bilingual\ndictionaries degrades CLIR performance. This paper aims to introduce a\nprobabilistic translation model to solve the ambiguity problem, and also to\nprovide most likely formations of a dictionary candidate. We propose Minimum\nEdit Support Candidates (MESC) method that exploits a monolingual corpus and a\nbilingual dictionary to translate users' native language queries to documents'\nlanguage. Our experiments show that the proposed method outperforms\nstate-of-the-art dictionary-based English-Persian CLIR.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1411.1147v2",
            "date": "2014-11-05 04:49:38+00:00",
            "title": "Conditional Random Field Autoencoders for Unsupervised Structured Prediction",
            "authors": [
                "Waleed Ammar",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We introduce a framework for unsupervised learning of structured predictors\nwith overlapping, global features. Each input's latent representation is\npredicted conditional on the observable data using a feature-rich conditional\nrandom field. Then a reconstruction of the input is (re)generated, conditional\non the latent structure, using models for which maximum likelihood estimation\nhas a closed-form. Our autoencoder formulation enables efficient learning\nwithout making unrealistic independence assumptions or restricting the kinds of\nfeatures that can be used. We illustrate insightful connections to traditional\nautoencoders, posterior regularization and multi-view learning. We show\ncompetitive results with instantiations of the model for two canonical NLP\ntasks: part-of-speech induction and bitext word alignment, and show that\ntraining our model can be substantially more efficient than comparable\nfeature-rich baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1411.1999v1",
            "date": "2014-11-07 18:23:00+00:00",
            "title": "Azhary: An Arabic Lexical Ontology",
            "authors": [
                "Hossam Ishkewy",
                "Hany Harb",
                "Hassan Farahat"
            ],
            "abstract": "Arabic language is the most spoken languages in the Semitic languages group,\nand one of the most common languages in the world spoken by more than 422\nmillion. It is also of paramount importance to Muslims, it is a sacred language\nof the Islamic Holly Book (Quran) and prayer (and other acts of worship) in\nIslam is performed only by mastering some of Arabic words. Arabic is also a\nmajor ritual language of a number of Christian churches in the Arab world and\nit is also used in writing several intellectual and religious Jewish books in\nthe Middle Ages. Despite this, there is no semantic Arabic lexicon which\nresearchers can depend on. In this paper we introduce Azhary as a lexical\nontology for the Arabic language. It groups Arabic words into sets of synonyms\ncalled synsets, and records a number of relationships between words such as\nsynonym, antonym, hypernym, hyponym, meronym, holonym and association\nrelations. The ontology contains 26,195 words organized in 13,328 synsets. It\nhas been developed and contrasted against AWN which is the most common\navailable Arabic lexical ontology.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1411.2328v1",
            "date": "2014-11-10 05:24:41+00:00",
            "title": "Modeling Word Relatedness in Latent Dirichlet Allocation",
            "authors": [
                "Xun Wang"
            ],
            "abstract": "Standard LDA model suffers the problem that the topic assignment of each word\nis independent and word correlation hence is neglected. To address this\nproblem, in this paper, we propose a model called Word Related Latent Dirichlet\nAllocation (WR-LDA) by incorporating word correlation into LDA topic models.\nThis leads to new capabilities that standard LDA model does not have such as\nestimating infrequently occurring words or multi-language topic modeling.\nExperimental results demonstrate the effectiveness of our model compared with\nstandard LDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.3827v4",
            "date": "2014-11-14 08:44:22+00:00",
            "title": "Autonomization of Monoidal Categories",
            "authors": [
                "Antonin Delpeuch"
            ],
            "abstract": "We show that contrary to common belief in the DisCoCat community, a monoidal\ncategory is all that is needed to define a categorical compositional model of\nnatural language. This relies on a construction which freely adds adjoints to a\nmonoidal category. In the case of distributional semantics, this broadens the\nrange of available models, to include non-linear maps and cartesian products\nfor instance. We illustrate the applications of this principle to various\ndistributional models of meaning.",
            "journal ref": "EPTCS 323, 2020, pp. 24-43",
            "category": "math.CT"
        },
        {
            "url": "http://arxiv.org/abs/1411.4109v1",
            "date": "2014-11-15 03:43:01+00:00",
            "title": "Resolution of Difficult Pronouns Using the ROSS Method",
            "authors": [
                "Glenn R. Hofford"
            ],
            "abstract": "A new natural language understanding method for disambiguation of difficult\npronouns is described. Difficult pronouns are those pronouns for which a level\nof world or domain knowledge is needed in order to perform anaphoral or other\ntypes of resolution. Resolution of difficult pronouns may in some cases require\na prior step involving the application of inference to a situation that is\nrepresented by the natural language text. A general method is described: it\nperforms entity resolution and pronoun resolution. An extension to the general\npronoun resolution method performs inference as an embedded commonsense\nreasoning method. The general method and the embedded method utilize features\nof the ROSS representational scheme; in particular the methods use ROSS\nontology classes and the ROSS situation model. The overall method is a working\nsolution that solves the following Winograd schemas: a) trophy and suitcase, b)\nperson lifts person, c) person pays detective, and d) councilmen and\ndemonstrators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4194v1",
            "date": "2014-11-15 22:47:35+00:00",
            "title": "ROSS User's Guide and Reference Manual (Version 1.0)",
            "authors": [
                "Glenn R. Hofford"
            ],
            "abstract": "The ROSS method is a new approach in the area of knowledge representation\nthat is useful for many artificial intelligence and natural language\nunderstanding representation and reasoning tasks. (ROSS stands for\n\"Representation\", \"Ontology\", \"Structure\", \"Star\" language). ROSS is a physical\nsymbol-based representational scheme. ROSS provides a complex model for the\ndeclarative representation of physical structure and for the representation of\nprocesses and causality. From the metaphysical perspective, the ROSS view of\nexternal reality involves a 4D model, wherein discrete single-time-point\nunit-sized locations with states are the basis for all objects, processes and\naspects that can be modeled. ROSS includes a language called \"Star\" for the\nspecification of ontology classes. The ROSS method also includes a formal\nscheme called the \"instance model\". Instance models are used in the area of\nnatural language meaning representation to represent situations. This document\nis an in-depth specification of the ROSS method.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1411.4455v1",
            "date": "2014-11-17 12:43:30+00:00",
            "title": "Errata: Distant Supervision for Relation Extraction with Matrix Completion",
            "authors": [
                "Miao Fan",
                "Deli Zhao",
                "Qiang Zhou",
                "Zhiyuan Liu",
                "Thomas Fang Zheng",
                "Edward Y. Chang"
            ],
            "abstract": "The essence of distantly supervised relation extraction is that it is an\nincomplete multi-label classification problem with sparse and noisy features.\nTo tackle the sparsity and noise challenges, we propose solving the\nclassification problem using matrix completion on factorized matrix of\nminimized rank. We formulate relation classification as completing the unknown\nlabels of testing items (entity pairs) in a sparse matrix that concatenates\ntraining and testing textual features with training labels. Our algorithmic\nframework is based on the assumption that the rank of item-by-feature and\nitem-by-label joint matrix is low. We apply two optimization models to recover\nthe underlying low-rank matrix leveraging the sparsity of feature-label matrix.\nThe matrix completion problem is then solved by the fixed point continuation\n(FPC) algorithm, which can find the global optimum. Experiments on two widely\nused datasets with different dimensions of textual features demonstrate that\nour low-rank matrix completion approach significantly outperforms the baseline\nand the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4618v1",
            "date": "2014-11-17 20:15:00+00:00",
            "title": "Relations World: A Possibilistic Graphical Model",
            "authors": [
                "Christopher J. C. Burges",
                "Erin Renshaw",
                "Andrzej Pastusiak"
            ],
            "abstract": "We explore the idea of using a \"possibilistic graphical model\" as the basis\nfor a world model that drives a dialog system. As a first step we have\ndeveloped a system that uses text-based dialog to derive a model of the user's\nfamily relations. The system leverages its world model to infer relational\ntriples, to learn to recover from upstream coreference resolution errors and\nambiguities, and to learn context-dependent paraphrase models. We also explore\nsome theoretical aspects of the underlying graphical model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4825v1",
            "date": "2014-11-18 12:37:00+00:00",
            "title": "Cognitive Systems and Question Answering",
            "authors": [
                "Ulrich Furbach",
                "Claudia Schon",
                "Frieder Stolzenburg"
            ],
            "abstract": "This paper briefly characterizes the field of cognitive computing. As an\nexemplification, the field of natural language question answering is introduced\ntogether with its specific challenges. A possibility to master these challenges\nis illustrated by a detailed presentation of the LogAnswer system, which is a\nsuccessful representative of the field of natural language question answering.",
            "journal ref": "Industrie 4.0 Management, 31(1):29-32, 2015",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1411.4925v1",
            "date": "2014-11-18 17:35:59+00:00",
            "title": "Linguistic Descriptions for Automatic Generation of Textual Short-Term Weather Forecasts on Real Prediction Data",
            "authors": [
                "A. Ramos-Soto",
                "A. Bugar\u00edn",
                "S. Barro",
                "J. Taboada"
            ],
            "abstract": "We present in this paper an application which automatically generates textual\nshort-term weather forecasts for every municipality in Galicia (NW Spain),\nusing the real data provided by the Galician Meteorology Agency (MeteoGalicia).\nThis solution combines in an innovative way computing with perceptions\ntechniques and strategies for linguistic description of data together with a\nnatural language generation (NLG) system. The application, named GALiWeather,\nextracts relevant information from weather forecast input data and encodes it\ninto intermediate descriptions using linguistic variables and temporal\nreferences. These descriptions are later translated into natural language texts\nby the natural language generation system. The obtained forecast results have\nbeen thoroughly validated by an expert meteorologist from MeteoGalicia using a\nquality assessment methodology which covers two key dimensions of a text: the\naccuracy of its content and the correctness of its form. Following this\nvalidation GALiWeather will be released as a real service offering custom\nforecasts for a wide public.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1411.4952v3",
            "date": "2014-11-18 18:23:45+00:00",
            "title": "From Captions to Visual Concepts and Back",
            "authors": [
                "Hao Fang",
                "Saurabh Gupta",
                "Forrest Iandola",
                "Rupesh Srivastava",
                "Li Deng",
                "Piotr Doll\u00e1r",
                "Jianfeng Gao",
                "Xiaodong He",
                "Margaret Mitchell",
                "John C. Platt",
                "C. Lawrence Zitnick",
                "Geoffrey Zweig"
            ],
            "abstract": "This paper presents a novel approach for automatically generating image\ndescriptions: visual detectors, language models, and multimodal similarity\nmodels learnt directly from a dataset of image captions. We use multiple\ninstance learning to train visual detectors for words that commonly occur in\ncaptions, including many different parts of speech such as nouns, verbs, and\nadjectives. The word detector outputs serve as conditional inputs to a\nmaximum-entropy language model. The language model learns from a set of over\n400,000 image descriptions to capture the statistics of word usage. We capture\nglobal semantics by re-ranking caption candidates using sentence-level features\nand a deep multimodal similarity model. Our system is state-of-the-art on the\nofficial Microsoft COCO benchmark, producing a BLEU-4 score of 29.1%. When\nhuman judges compare the system captions to ones written by other people on our\nheld-out test set, the system captions have equal or better quality 34% of the\ntime.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1411.6699v1",
            "date": "2014-11-25 01:25:56+00:00",
            "title": "One Vector is Not Enough: Entity-Augmented Distributional Semantics for Discourse Relations",
            "authors": [
                "Yangfeng Ji",
                "Jacob Eisenstein"
            ],
            "abstract": "Discourse relations bind smaller linguistic units into coherent texts.\nHowever, automatically identifying discourse relations is difficult, because it\nrequires understanding the semantics of the linked arguments. A more subtle\nchallenge is that it is not enough to represent the meaning of each argument of\na discourse relation, because the relation may depend on links between\nlower-level components, such as entity mentions. Our solution computes\ndistributional meaning representations by composition up the syntactic parse\ntree. A key difference from previous work on compositional distributional\nsemantics is that we also compute representations for entity mentions, using a\nnovel downward compositional pass. Discourse relations are predicted from the\ndistributional representations of the arguments, and also of their coreferent\nentity mentions. The resulting system obtains substantial improvements over the\nprevious state-of-the-art in predicting implicit discourse relations in the\nPenn Discourse Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.6718v2",
            "date": "2014-11-25 03:48:56+00:00",
            "title": "LABR: A Large Scale Arabic Sentiment Analysis Benchmark",
            "authors": [
                "Mahmoud Nabil",
                "Mohamed Aly",
                "Amir Atiya"
            ],
            "abstract": "We introduce LABR, the largest sentiment analysis dataset to-date for the\nArabic language. It consists of over 63,000 book reviews, each rated on a scale\nof 1 to 5 stars. We investigate the properties of the dataset, and present its\nstatistics. We explore using the dataset for two tasks: (1) sentiment polarity\nclassification; and (2) ratings classification. Moreover, we provide standard\nsplits of the dataset into training, validation and testing, for both polarity\nand ratings classification, in both balanced and unbalanced settings. We extend\nour previous work by performing a comprehensive analysis on the dataset. In\nparticular, we perform an extended survey of the different classifiers\ntypically used for the sentiment polarity classification problem. We also\nconstruct a sentiment lexicon from the dataset that contains both single and\ncompound sentiment words and we explore its effectiveness. We make the dataset\nand experimental details publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.1521v3",
            "date": "2014-04-05 21:25:54+00:00",
            "title": "Exploring the power of GPU's for training Polyglot language models",
            "authors": [
                "Vivek Kulkarni",
                "Rami Al-Rfou'",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "One of the major research trends currently is the evolution of heterogeneous\nparallel computing. GP-GPU computing is being widely used and several\napplications have been designed to exploit the massive parallelism that\nGP-GPU's have to offer. While GPU's have always been widely used in areas of\ncomputer vision for image processing, little has been done to investigate\nwhether the massive parallelism provided by GP-GPU's can be utilized\neffectively for Natural Language Processing(NLP) tasks. In this work, we\ninvestigate and explore the power of GP-GPU's in the task of learning language\nmodels. More specifically, we investigate the performance of training Polyglot\nlanguage models using deep belief neural networks. We evaluate the performance\nof training the model on the GPU and present optimizations that boost the\nperformance on the GPU.One of the key optimizations, we propose increases the\nperformance of a function involved in calculating and updating the gradient by\napproximately 50 times on the GPU for sufficiently large batch sizes. We show\nthat with the above optimizations, the GP-GPU's performance on the task\nincreases by factor of approximately 3-4. The optimizations we made are generic\nTheano optimizations and hence potentially boost the performance of other\nmodels which rely on these operations.We also show that these optimizations\nresult in the GPU's performance at this task being now comparable to that on\nthe CPU. We conclude by presenting a thorough evaluation of the applicability\nof GP-GPU's for this task and highlight the factors limiting the performance of\ntraining a Polyglot model on the GPU.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1404.1890v1",
            "date": "2014-04-07 19:23:29+00:00",
            "title": "Polish and English wordnets -- statistical analysis of interconnected networks",
            "authors": [
                "Maksymilian Bujok",
                "Piotr Fronczak",
                "Agata Fronczak"
            ],
            "abstract": "Wordnets are semantic networks containing nouns, verbs, adjectives, and\nadverbs organized according to linguistic principles, by means of semantic\nrelations. In this work, we adopt a complex network perspective to perform a\ncomparative analysis of the English and Polish wordnets. We determine their\nsimilarities and show that the networks exhibit some of the typical\ncharacteristics observed in other real-world networks. We analyse interlingual\nrelations between both wordnets and deliberate over the problem of mapping the\nPolish lexicon onto the English one.",
            "journal ref": "Acta Phys. Pol. B Proc. Suppl. Vol. 7 (2014) 245-256",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.1982v1",
            "date": "2014-04-08 01:19:14+00:00",
            "title": "Aspect-Based Opinion Extraction from Customer reviews",
            "authors": [
                "Amani K Samha",
                "Yuefeng Li",
                "Jinglan Zhang"
            ],
            "abstract": "Text is the main method of communicating information in the digital age.\nMessages, blogs, news articles, reviews, and opinionated information abound on\nthe Internet. People commonly purchase products online and post their opinions\nabout purchased items. This feedback is displayed publicly to assist others\nwith their purchasing decisions, creating the need for a mechanism with which\nto extract and summarize useful information for enhancing the decision-making\nprocess. Our contribution is to improve the accuracy of extraction by combining\ndifferent techniques from three major areas, named Data Mining, Natural\nLanguage Processing techniques and Ontologies. The proposed framework\nsequentially mines products aspects and users opinions, groups representative\naspects by similarity, and generates an output summary. This paper focuses on\nthe task of extracting product aspects and users opinions by extracting all\npossible aspects and opinions from reviews using natural language, ontology,\nand frequent (tag) sets. The proposed framework, when compared with an existing\nbaseline model, yielded promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.2997v1",
            "date": "2014-04-11 04:36:05+00:00",
            "title": "Automatic Detection of Reuses and Citations in Literary Texts",
            "authors": [
                "Jean-Gabriel Ganascia",
                "Pierre Glaudes",
                "Andrea Del Lungo"
            ],
            "abstract": "For more than forty years now, modern theories of literature (Compagnon,\n1979) insist on the role of paraphrases, rewritings, citations, reciprocal\nborrowings and mutual contributions of any kinds. The notions of\nintertextuality, transtextuality, hypertextuality/hypotextuality, were\nintroduced in the seventies and eighties to approach these phenomena. The\ncareful analysis of these references is of particular interest in evaluating\nthe distance that the creator voluntarily introduces with his/her masters.\nPhoebus is collaborative project that makes computer scientists from the\nUniversity Pierre and Marie Curie (LIP6-UPMC) collaborate with the literary\nteams of Paris-Sorbonne University with the aim to develop efficient tools for\nliterary studies that take advantage of modern computer science techniques. In\nthis context, we have developed a piece of software that automatically detects\nand explores networks of textual reuses in classical literature. This paper\ndescribes the principles on which is based this program, the significant\nresults that have already been obtained and the perspectives for the near\nfuture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3233v1",
            "date": "2014-04-11 21:50:02+00:00",
            "title": "Pagination: It's what you say, not how long it takes to say it",
            "authors": [
                "Joshua Hailpern",
                "Niranjan Damera Venkata",
                "Marina Danilevsky"
            ],
            "abstract": "Pagination - the process of determining where to break an article across\npages in a multi-article layout is a common layout challenge for most\ncommercially printed newspapers and magazines. To date, no one has created an\nalgorithm that determines a minimal pagination break point based on the content\nof the article. Existing approaches for automatic multi-article layout focus\nexclusively on maximizing content (number of articles) and optimizing aesthetic\npresentation (e.g., spacing between articles). However, disregarding the\nsemantic information within the article can lead to overly aggressive cutting,\nthereby eliminating key content and potentially confusing the reader, or\nsetting too generous of a break point, thereby leaving in superfluous content\nand making automatic layout more difficult. This is one of the remaining\nchallenges on the path from manual layouts to fully automated processes that\nstill ensure article content quality. In this work, we present a new approach\nto calculating a document minimal break point for the task of pagination. Our\napproach uses a statistical language model to predict minimal break points\nbased on the semantic content of an article. We then compare 4 novel candidate\napproaches, and 4 baselines (currently in use by layout algorithms). Results\nfrom this experiment show that one of our approaches strongly outperforms the\nbaselines and alternatives. Results from a second study suggest that humans are\nnot able to agree on a single \"best\" break point. Therefore, this work shows\nthat a semantic-based lower bound break point prediction is necessary for ideal\nautomated document synthesis within a real-world context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3925v2",
            "date": "2014-04-13 14:19:27+00:00",
            "title": "Complexity of Grammar Induction for Quantum Types",
            "authors": [
                "Antonin Delpeuch"
            ],
            "abstract": "Most categorical models of meaning use a functor from the syntactic category\nto the semantic category. When semantic information is available, the problem\nof grammar induction can therefore be defined as finding preimages of the\nsemantic types under this forgetful functor, lifting the information flow from\nthe semantic level to a valid reduction at the syntactic level. We study the\ncomplexity of grammar induction, and show that for a variety of type systems,\nincluding pivotal and compact closed categories, the grammar induction problem\nis NP-complete. Our approach could be extended to linguistic type systems such\nas autonomous or bi-closed categories.",
            "journal ref": "EPTCS 172, 2014, pp. 236-248",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.3959v1",
            "date": "2014-04-15 15:41:34+00:00",
            "title": "Is it morally acceptable for a system to lie to persuade me?",
            "authors": [
                "Marco Guerini",
                "Fabio Pianesi",
                "Oliviero Stock"
            ],
            "abstract": "Given the fast rise of increasingly autonomous artificial agents and robots,\na key acceptability criterion will be the possible moral implications of their\nactions. In particular, intelligent persuasive systems (systems designed to\ninfluence humans via communication) constitute a highly sensitive topic because\nof their intrinsically social nature. Still, ethical studies in this area are\nrare and tend to focus on the output of the required action. Instead, this work\nfocuses on the persuasive acts themselves (e.g. \"is it morally acceptable that\na machine lies or appeals to the emotions of a person to persuade her, even if\nfor a good end?\"). Exploiting a behavioral approach, based on human assessment\nof moral dilemmas -- i.e. without any prior assumption of underlying ethical\ntheories -- this paper reports on a set of experiments. These experiments\naddress the type of persuader (human or machine), the strategies adopted\n(purely argumentative, appeal to positive emotions, appeal to negative\nemotions, lie) and the circumstances. Findings display no differences due to\nthe agent, mild acceptability for persuasion and reveal that truth-conditional\nreasoning (i.e. argument validity) is a significant dimension affecting\nsubjects' judgment. Some implications for the design of intelligent persuasive\nsystems are discussed.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1404.4326v1",
            "date": "2014-04-16 17:57:01+00:00",
            "title": "Open Question Answering with Weakly Supervised Embedding Models",
            "authors": [
                "Antoine Bordes",
                "Jason Weston",
                "Nicolas Usunier"
            ],
            "abstract": "Building computers able to answer questions on any subject is a long standing\ngoal of artificial intelligence. Promising progress has recently been achieved\nby methods that learn to map questions to logical forms or database queries.\nSuch approaches can be effective but at the cost of either large amounts of\nhuman-labeled data or by defining lexicons and grammars tailored by\npractitioners. In this paper, we instead take the radical approach of learning\nto map questions to vectorial feature representations. By mapping answers into\nthe same space one can query any knowledge base independent of its schema,\nwithout requiring any grammar or lexicon. Our method is trained with a new\noptimization procedure combining stochastic gradient descent followed by a\nfine-tuning step using the weak supervision provided by blending automatically\nand collaboratively generated resources. We empirically demonstrate that our\nmodel can capture meaningful signals from its noisy supervision leading to\nmajor improvements over paralex, the only existing method able to be trained on\nsimilar weakly labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.4935v1",
            "date": "2014-04-19 08:14:39+00:00",
            "title": "Opinion Mining In Hindi Language: A Survey",
            "authors": [
                "Richa Sharma",
                "Shweta Nigam",
                "Rekha Jain"
            ],
            "abstract": "Opinions are very important in the life of human beings. These Opinions\nhelped the humans to carry out the decisions. As the impact of the Web is\nincreasing day by day, Web documents can be seen as a new source of opinion for\nhuman beings. Web contains a huge amount of information generated by the users\nthrough blogs, forum entries, and social networking websites and so on To\nanalyze this large amount of information it is required to develop a method\nthat automatically classifies the information available on the Web. This domain\nis called Sentiment Analysis and Opinion Mining. Opinion Mining or Sentiment\nAnalysis is a natural language processing task that mine information from\nvarious text forms such as reviews, news, and blogs and classify them on the\nbasis of their polarity as positive, negative or neutral. But, from the last\nfew years, enormous increase has been seen in Hindi language on the Web.\nResearch in opinion mining mostly carried out in English language but it is\nvery important to perform the opinion mining in Hindi language also as large\namount of information in Hindi is also available on the Web. This paper gives\nan overview of the work that has been done Hindi language.",
            "journal ref": "International Journal in Foundations of Computer Science &\n  Technology (IJFCST) International Journal in Foundations of Computer Science\n  & Technology (IJFCST), Vol.4, No.2, March 2014",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1404.5372v1",
            "date": "2014-04-22 03:28:07+00:00",
            "title": "Linking Geographic Vocabularies through WordNet",
            "authors": [
                "Andrea Ballatore",
                "Michela Bertolotto",
                "David C. Wilson"
            ],
            "abstract": "The linked open data (LOD) paradigm has emerged as a promising approach to\nstructuring and sharing geospatial information. One of the major obstacles to\nthis vision lies in the difficulties found in the automatic integration between\nheterogeneous vocabularies and ontologies that provides the semantic backbone\nof the growing constellation of open geo-knowledge bases. In this article, we\nshow how to utilize WordNet as a semantic hub to increase the integration of\nLOD. With this purpose in mind, we devise Voc2WordNet, an unsupervised mapping\ntechnique between a given vocabulary and WordNet, combining intensional and\nextensional aspects of the geographic terms. Voc2WordNet is evaluated against a\nsample of human-generated alignments with the OpenStreetMap (OSM) Semantic\nNetwork, a crowdsourced geospatial resource, and the GeoNames ontology, the\nvocabulary of a large digital gazetteer. These empirical results indicate that\nthe approach can obtain high precision and recall.",
            "journal ref": "Annals of GIS, 20 (2), 2014, 73-84",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1404.5585v1",
            "date": "2014-04-22 18:26:09+00:00",
            "title": "A Structural Query System for Han Characters",
            "authors": [
                "Matthew Skala"
            ],
            "abstract": "The IDSgrep structural query system for Han character dictionaries is\npresented. This system includes a data model and syntax for describing the\nspatial structure of Han characters using Extended Ideographic Description\nSequences (EIDSes) based on the Unicode IDS syntax; a language for querying\nEIDS databases, designed to suit the needs of font developers and foreign\nlanguage learners; a bit vector index inspired by Bloom filters for faster\nquery operations; a freely available implementation; and format translation\nfrom popular third-party IDS and XML character databases. Experimental results\nare included, with a comparison to other software used for similar\napplications.",
            "journal ref": "International Journal of Asian Language Processing 23(2) (2015)\n  127-159",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.6491v1",
            "date": "2014-04-23 19:28:55+00:00",
            "title": "An Account of Opinion Implicatures",
            "authors": [
                "Janyce Wiebe",
                "Lingjia Deng"
            ],
            "abstract": "While previous sentiment analysis research has concentrated on the\ninterpretation of explicitly stated opinions and attitudes, this work initiates\nthe computational study of a type of opinion implicature (i.e.,\nopinion-oriented inference) in text. This paper described a rule-based\nframework for representing and analyzing opinion implicatures which we hope\nwill contribute to deeper automatic interpretation of subjective language. In\nthe course of understanding implicatures, the system recognizes implicit\nsentiments (and beliefs) toward various events and entities in the sentence,\noften attributed to different sources (holders) and of mixed polarities; thus,\nit produces a richer interpretation than is typical in opinion analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.7362v1",
            "date": "2014-04-29 13:53:38+00:00",
            "title": "Concise comparative summaries (CCS) of large text corpora with a human experiment",
            "authors": [
                "Jinzhu Jia",
                "Luke Miratrix",
                "Bin Yu",
                "Brian Gawalt",
                "Laurent El Ghaoui",
                "Luke Barnesmoore",
                "Sophie Clavier"
            ],
            "abstract": "In this paper we propose a general framework for topic-specific summarization\nof large text corpora and illustrate how it can be used for the analysis of\nnews databases. Our framework, concise comparative summarization (CCS), is\nbuilt on sparse classification methods. CCS is a lightweight and flexible tool\nthat offers a compromise between simple word frequency based methods currently\nin wide use and more heavyweight, model-intensive methods such as latent\nDirichlet allocation (LDA). We argue that sparse methods have much to offer for\ntext analysis and hope CCS opens the door for a new branch of research in this\nimportant field. For a particular topic of interest (e.g., China or energy),\nCSS automatically labels documents as being either on- or off-topic (usually\nvia keyword search), and then uses sparse classification methods to predict\nthese labels with the high-dimensional counts of all the other words and\nphrases in the documents. The resulting small set of phrases found as\npredictive are then harvested as the summary. To validate our tool, we, using\nnews articles from the New York Times international section, designed and\nconducted a human survey to compare the different summarizers with human\nunderstanding. We demonstrate our approach with two case studies, a media\nanalysis of the framing of \"Egypt\" in the New York Times throughout the Arab\nSpring and an informal comparison of the New York Times' and Wall Street\nJournal's coverage of \"energy.\" Overall, we find that the Lasso with $L^2$\nnormalization can be effectively and usefully used to summarize large corpora,\nregardless of document size.",
            "journal ref": "Annals of Applied Statistics 2014, Vol. 8, No. 1, 499-529",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.0079v1",
            "date": "2014-05-31 14:16:30+00:00",
            "title": "Bridging the gap between Legal Practitioners and Knowledge Engineers using semi-formal KR",
            "authors": [
                "Shashishekar Ramakrishna",
                "Adrian Paschke"
            ],
            "abstract": "The use of Structured English as a computation independent knowledge\nrepresentation format for non-technical users in business rules representation\nhas been proposed in OMGs Semantics and Business Vocabulary Representation\n(SBVR). In the legal domain we face a similar problem. Formal representation\nlanguages, such as OASIS LegalRuleML and legal ontologies (LKIF, legal OWL2\nontologies etc.) support the technical knowledge engineer and the automated\nreasoning. But, they can be hardly used directly by the legal domain experts\nwho do not have a computer science background. In this paper we adapt the SBVR\nStructured English approach for the legal domain and implement a\nproof-of-concept, called KR4IPLaw, which enables legal domain experts to\nrepresent their knowledge in Structured English in a computational independent\nand hence, for them, more usable way. The benefit of this approach is that the\nunderlying pre-defined semantics of the Structured English approach makes\ntransformations into formal languages such as OASIS LegalRuleML and OWL2\nontologies possible. We exemplify our approach in the domain of patent law.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1765v1",
            "date": "2014-06-06 18:09:59+00:00",
            "title": "Linguistic Analysis of Requirements of a Space Project and their Conformity with the Recommendations Proposed by a Controlled Natural Language",
            "authors": [
                "Anne Condamines",
                "Maxime Warnier"
            ],
            "abstract": "The long term aim of the project carried out by the French National Space\nAgency (CNES) is to design a writing guide based on the real and regular\nwriting of requirements. As a first step in the project, this paper proposes a\nlin-guistic analysis of requirements written in French by CNES engineers. The\naim is to determine to what extent they conform to two rules laid down in\nINCOSE, a recent guide for writing requirements. Although CNES engineers are\nnot obliged to follow any Controlled Natural Language in their writing of\nrequirements, we believe that language regularities are likely to emerge from\nthis task, mainly due to the writers' experience. The issue is approached using\nnatural language processing tools to identify sentences that do not comply with\nINCOSE rules. We further review these sentences to understand why the\nrecommendations cannot (or should not) always be applied when specifying\nlarge-scale projects.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1406.1953v2",
            "date": "2014-06-08 07:41:07+00:00",
            "title": "Automatic Extraction of Protein Interaction in Literature",
            "authors": [
                "Peilei Liu",
                "Ting Wang"
            ],
            "abstract": "Protein-protein interaction extraction is the key precondition of the\nconstruction of protein knowledge network, and it is very important for the\nresearch in the biomedicine. This paper extracted directional protein-protein\ninteraction from the biological text, using the SVM-based method. Experiments\nwere evaluated on the LLL05 corpus with good results. The results show that\ndependency features are import for the protein-protein interaction extraction\nand features related to the interaction word are effective for the interaction\ndirection judgment. At last, we analyzed the effects of different features and\nplaned for the next step.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2022v1",
            "date": "2014-06-08 20:05:36+00:00",
            "title": "Two-dimensional Sentiment Analysis of text",
            "authors": [
                "Rahul Tejwani"
            ],
            "abstract": "Sentiment Analysis aims to get the underlying viewpoint of the text, which\ncould be anything that holds a subjective opinion, such as an online review,\nMovie rating, Comments on Blog posts etc. This paper presents a novel approach\nthat classify text in two-dimensional Emotional space, based on the sentiments\nof the author. The approach uses existing lexical resources to extract feature\nset, which is trained using Supervised Learning techniques.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1406.2096v1",
            "date": "2014-06-09 07:19:53+00:00",
            "title": "RuleCNL: A Controlled Natural Language for Business Rule Specifications",
            "authors": [
                "Paul Brillant Feuto Njonko",
                "Sylviane Cardey",
                "Peter Greenfield",
                "Walid El Abed"
            ],
            "abstract": "Business rules represent the primary means by which companies define their\nbusiness, perform their actions in order to reach their objectives. Thus, they\nneed to be expressed unambiguously to avoid inconsistencies between business\nstakeholders and formally in order to be machine-processed. A promising\nsolution is the use of a controlled natural language (CNL) which is a good\nmediator between natural and formal languages. This paper presents RuleCNL,\nwhich is a CNL for defining business rules. Its core feature is the alignment\nof the business rule definition with the business vocabulary which ensures\ntraceability and consistency with the business domain. The RuleCNL tool\nprovides editors that assist end-users in the writing process and automatic\nmappings into the Semantics of Business Vocabulary and Business Rules (SBVR)\nstandard. SBVR is grounded in first order logic and includes constructs called\nsemantic formulations that structure the meaning of rules.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1406.2298v1",
            "date": "2014-06-09 19:51:10+00:00",
            "title": "Explaining Violation Traces with Finite State Natural Language Generation Models",
            "authors": [
                "Gordon J. Pace",
                "Michael Rosner"
            ],
            "abstract": "An essential element of any verification technique is that of identifying and\ncommunicating to the user, system behaviour which leads to a deviation from the\nexpected behaviour. Such behaviours are typically made available as long traces\nof system actions which would benefit from a natural language explanation of\nthe trace and especially in the context of business logic level specifications.\nIn this paper we present a natural language generation model which can be used\nto explain such traces. A key idea is that the explanation language is a CNL\nthat is, formally speaking, regular language susceptible transformations that\ncan be expressed with finite state machinery. At the same time it admits\nvarious forms of abstraction and simplification which contribute to the\nnaturalness of explanations that are communicated to the user.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1406.2710v1",
            "date": "2014-06-10 20:29:10+00:00",
            "title": "A Multiplicative Model for Learning Distributed Text-Based Attribute Representations",
            "authors": [
                "Ryan Kiros",
                "Richard S. Zemel",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "In this paper we propose a general framework for learning distributed\nrepresentations of attributes: characteristics of text whose representations\ncan be jointly learned with word embeddings. Attributes can correspond to\ndocument indicators (to learn sentence vectors), language indicators (to learn\ndistributed language representations), meta-data and side information (such as\nthe age, gender and industry of a blogger) or representations of authors. We\ndescribe a third-order model where word context and attribute vectors interact\nmultiplicatively to predict the next word in a sequence. This leads to the\nnotion of conditional word similarity: how meanings of words change when\nconditioned on different attributes. We perform several experimental tasks\nincluding sentiment classification, cross-lingual document classification, and\nblog authorship attribution. We also qualitatively evaluate conditional word\nneighbours and attribute-conditioned text generation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1406.3714v1",
            "date": "2014-06-14 10:39:57+00:00",
            "title": "Mining of product reviews at aspect level",
            "authors": [
                "Richa Sharma",
                "Shweta Nigam",
                "Rekha Jain"
            ],
            "abstract": "Todays world is a world of Internet, almost all work can be done with the\nhelp of it, from simple mobile phone recharge to biggest business deals can be\ndone with the help of this technology. People spent their most of the times on\nsurfing on the Web it becomes a new source of entertainment, education,\ncommunication, shopping etc. Users not only use these websites but also give\ntheir feedback and suggestions that will be useful for other users. In this way\na large amount of reviews of users are collected on the Web that needs to be\nexplored, analyse and organized for better decision making. Opinion Mining or\nSentiment Analysis is a Natural Language Processing and Information Extraction\ntask that identifies the users views or opinions explained in the form of\npositive, negative or neutral comments and quotes underlying the text. Aspect\nbased opinion mining is one of the level of Opinion mining that determines the\naspect of the given reviews and classify the review for each feature. In this\npaper an aspect based opinion mining system is proposed to classify the reviews\nas positive, negative and neutral for each feature. Negation is also handled in\nthe proposed system. Experimental results using reviews of products show the\neffectiveness of the system.",
            "journal ref": "International Journal in Foundations of Computer Science &\n  Technology (IJFCST), Vol.4, No.3, May 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.4690v1",
            "date": "2014-06-18 11:54:13+00:00",
            "title": "The Frobenius anatomy of word meanings II: possessive relative pronouns",
            "authors": [
                "Mehrnoosh Sadrzadeh",
                "Stephen Clark",
                "Bob Coecke"
            ],
            "abstract": "Within the categorical compositional distributional model of meaning, we\nprovide semantic interpretations for the subject and object roles of the\npossessive relative pronoun `whose'. This is done in terms of Frobenius\nalgebras over compact closed categories. These algebras and their diagrammatic\nlanguage expose how meanings of words in relative clauses interact with each\nother. We show how our interpretation is related to Montague-style semantics\nand provide a truth-theoretic interpretation. We also show how vector spaces\nprovide a concrete interpretation and provide preliminary corpus-based\nexperimental evidence. In a prequel to this paper, we used similar methods and\ndealt with the case of subject and object relative pronouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.4824v2",
            "date": "2014-06-12 21:20:46+00:00",
            "title": "What is India speaking: The \"Hinglish\" invasion",
            "authors": [
                "Rana D. Parshad",
                "Vineeta Chand",
                "Neha Sinha",
                "Nitu Kumari"
            ],
            "abstract": "While language competition models of diachronic language shift are\nincreasingly sophisticated, drawing on sociolinguistic components like variable\nlanguage prestige, distance from language centers and intermediate bilingual\ntransitionary populations, in one significant way they fall short. They fail to\nconsider contact-based outcomes resulting in mixed language practices, e.g.\noutcome scenarios such as creoles or unmarked code switching as an emergent\ncommunicative norm. On these lines something very interesting is uncovered in\nIndia, where traditionally there have been monolingual Hindi speakers and\nHindi/English bilinguals, but virtually no monolingual English speakers. While\nthe Indian census data reports a sharp increase in the proportion of\nHindi/English bilinguals, we argue that the number of Hindi/English bilinguals\nin India is inaccurate, given a new class of urban individuals speaking a mixed\nlect of Hindi and English, popularly known as \"Hinglish\". Based on\npredator-prey, sociolinguistic theories, salient local ecological factors and\nthe rural-urban divide in India, we propose a new mathematical model of\ninteracting monolingual Hindi speakers, Hindi/English bilinguals and Hinglish\nspeakers. The model yields globally asymptotic stable states of coexistence, as\nwell as bilingual extinction. To validate our model, sociolinguistic data from\ndifferent Indian classes are contrasted with census reports: We see that\npurported urban Hindi/English bilinguals are unable to maintain fluent Hindi\nspeech and instead produce Hinglish, whereas rural speakers evidence\nmonolingual Hindi. Thus we present evidence for the first time where an\nunrecognized mixed lect involving English but not \"English\", has possibly taken\nover a sizeable faction of a large global population.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.5181v2",
            "date": "2014-06-19 20:00:05+00:00",
            "title": "Zipf's law holds for phrases, not words",
            "authors": [
                "Jake Ryland Williams",
                "Paul R. Lessard",
                "Suma Desu",
                "Eric Clark",
                "James P. Bagrow",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "With Zipf's law being originally and most famously observed for word\nfrequency, it is surprisingly limited in its applicability to human language,\nholding over no more than three to four orders of magnitude before hitting a\nclear break in scaling. Here, building on the simple observation that phrases\nof one or more words comprise the most coherent units of meaning in language,\nwe show empirically that Zipf's law for phrases extends over as many as nine\norders of rank magnitude. In doing so, we develop a principled and scalable\nstatistical mechanical method of random text partitioning, which opens up a\nrich frontier of rigorous text analysis via a rank ordering of mixed length\nphrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.5691v1",
            "date": "2014-06-22 09:41:24+00:00",
            "title": "A CNL for Contract-Oriented Diagrams",
            "authors": [
                "John J. Camilleri",
                "Gabriele Paganelli",
                "Gerardo Schneider"
            ],
            "abstract": "We present a first step towards a framework for defining and manipulating\nnormative documents or contracts described as Contract-Oriented (C-O) Diagrams.\nThese diagrams provide a visual representation for such texts, giving the\npossibility to express a signatory's obligations, permissions and prohibitions,\nwith or without timing constraints, as well as the penalties resulting from the\nnon-fulfilment of a contract. This work presents a CNL for verbalising C-O\nDiagrams, a web-based tool allowing editing in this CNL, and another for\nvisualising and manipulating the diagrams interactively. We then show how these\nproof-of-concept tools can be used by applying them to a small example.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.6101v1",
            "date": "2014-06-23 22:21:17+00:00",
            "title": "Improved Frame Level Features and SVM Supervectors Approach for the Recogniton of Emotional States from Speech: Application to categorical and dimensional states",
            "authors": [
                "Imen Trabelsi",
                "Dorra Ben Ayed",
                "Noureddine Ellouze"
            ],
            "abstract": "The purpose of speech emotion recognition system is to classify speakers\nutterances into different emotional states such as disgust, boredom, sadness,\nneutral and happiness. Speech features that are commonly used in speech emotion\nrecognition rely on global utterance level prosodic features. In our work, we\nevaluate the impact of frame level feature extraction. The speech samples are\nfrom Berlin emotional database and the features extracted from these utterances\nare energy, different variant of mel frequency cepstrum coefficients, velocity\nand acceleration features.",
            "journal ref": "I.J. Image, Graphics and Signal Processing, 2013, 9, 8-13",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.7314v1",
            "date": "2014-06-27 20:56:00+00:00",
            "title": "On the Use of Different Feature Extraction Methods for Linear and Non Linear kernels",
            "authors": [
                "Imen Trabelsi",
                "Dorra Ben Ayed"
            ],
            "abstract": "The speech feature extraction has been a key focus in robust speech\nrecognition research; it significantly affects the recognition performance. In\nthis paper, we first study a set of different features extraction methods such\nas linear predictive coding (LPC), mel frequency cepstral coefficient (MFCC)\nand perceptual linear prediction (PLP) with several features normalization\ntechniques like rasta filtering and cepstral mean subtraction (CMS). Based on\nthis, a comparative evaluation of these features is performed on the task of\ntext independent speaker identification using a combination between gaussian\nmixture models (GMM) and linear and non-linear kernels based on support vector\nmachine (SVM).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.1080v1",
            "date": "2014-10-04 19:00:54+00:00",
            "title": "Generating abbreviations using Google Books library",
            "authors": [
                "Valery D. Solovyev",
                "Vladimir V. Bochkarev"
            ],
            "abstract": "The article describes the original method of creating a dictionary of\nabbreviations based on the Google Books Ngram Corpus. The dictionary of\nabbreviations is designed for Russian, yet as its methodology is universal it\ncan be applied to any language. The dictionary can be used to define the\nfunction of the period during text segmentation in various applied systems of\ntext processing. The article describes difficulties encountered in the process\nof its construction as well as the ways to overcome them. A model of evaluating\na probability of first and second type errors (extraction accuracy and\nfullness) is constructed. Certain statistical data for the use of abbreviations\nare provided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2045v1",
            "date": "2014-10-08 10:01:47+00:00",
            "title": "Supervised learning Methods for Bangla Web Document Categorization",
            "authors": [
                "Ashis Kumar Mandal",
                "Rikta Sen"
            ],
            "abstract": "This paper explores the use of machine learning approaches, or more\nspecifically, four supervised learning Methods, namely Decision Tree(C 4.5),\nK-Nearest Neighbour (KNN), Na\\\"ive Bays (NB), and Support Vector Machine (SVM)\nfor categorization of Bangla web documents. This is a task of automatically\nsorting a set of documents into categories from a predefined set. Whereas a\nwide range of methods have been applied to English text categorization,\nrelatively few studies have been conducted on Bangla language text\ncategorization. Hence, we attempt to analyze the efficiency of those four\nmethods for categorization of Bangla documents. In order to validate, Bangla\ncorpus from various websites has been developed and used as examples for the\nexperiment. For Bangla, empirical results support that all four methods produce\nsatisfactory performance with SVM attaining good result in terms of high\ndimensional and relatively noisy document feature vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.2265v1",
            "date": "2014-10-08 20:29:39+00:00",
            "title": "A Scalable, Lexicon Based Technique for Sentiment Analysis",
            "authors": [
                "Chetan Kaushik",
                "Atul Mishra"
            ],
            "abstract": "Rapid increase in the volume of sentiment rich social media on the web has\nresulted in an increased interest among researchers regarding Sentimental\nAnalysis and opinion mining. However, with so much social media available on\nthe web, sentiment analysis is now considered as a big data task. Hence the\nconventional sentiment analysis approaches fails to efficiently handle the vast\namount of sentiment data available now a days. The main focus of the research\nwas to find such a technique that can efficiently perform sentiment analysis on\nbig data sets. A technique that can categorize the text as positive, negative\nand neutral in a fast and accurate manner. In the research, sentiment analysis\nwas performed on a large data set of tweets using Hadoop and the performance of\nthe technique was measured in form of speed and accuracy. The experimental\nresults shows that the technique exhibits very good efficiency in handling big\nsentiment data sets.",
            "journal ref": "International Journal in Foundations of Computer Science &\n  Technology (IJFCST), Vol.4, No.5, September 2014",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1410.2686v2",
            "date": "2014-10-10 06:42:25+00:00",
            "title": "Polarization Measurement of High Dimensional Social Media Messages With Support Vector Machine Algorithm Using Mapreduce",
            "authors": [
                "Ferhat \u00d6zg\u00fcr \u00c7atak"
            ],
            "abstract": "In this article, we propose a new Support Vector Machine (SVM) training\nalgorithm based on distributed MapReduce technique. In literature, there are a\nlots of research that shows us SVM has highest generalization property among\nclassification algorithms used in machine learning area. Also, SVM classifier\nmodel is not affected by correlations of the features. But SVM uses quadratic\noptimization techniques in its training phase. The SVM algorithm is formulated\nas quadratic optimization problem. Quadratic optimization problem has $O(m^3)$\ntime and $O(m^2)$ space complexity, where m is the training set size. The\ncomputation time of SVM training is quadratic in the number of training\ninstances. In this reason, SVM is not a suitable classification algorithm for\nlarge scale dataset classification. To solve this training problem we developed\na new distributed MapReduce method developed. Accordingly, (i) SVM algorithm is\ntrained in distributed dataset individually; (ii) then merge all support\nvectors of classifier model in every trained node; and (iii) iterate these two\nsteps until the classifier model converges to the optimal classifier function.\nIn the implementation phase, large scale social media dataset is presented in\nTFxIDF matrix. The matrix is used for sentiment analysis to get polarization\nvalue. Two and three class models are created for classification method.\nConfusion matrices of each classification model are presented in tables. Social\nmedia messages corpus consists of 108 public and 66 private universities\nmessages in Turkey. Twitter is used for source of corpus. Twitter user messages\nare collected using Twitter Streaming API. Results are shown in graphics and\ntables.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1410.2910v1",
            "date": "2014-10-10 21:14:02+00:00",
            "title": "Riesz Logic",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "We introduce Riesz Logic, whose models are abelian lattice ordered groups,\nwhich generalise Riesz spaces (vector lattices), and show soundness and\ncompleteness. Our motivation is to provide a logic for distributional semantics\nof natural language, where words are typically represented as elements of a\nvector space whose dimensions correspond to contexts in which words may occur.\nThis basis provides a lattice ordering on the space, and this ordering may be\ninterpreted as \"distributional entailment\". Several axioms of Riesz Logic are\nfamiliar from Basic Fuzzy Logic, and we show how the models of these two logics\nmay be related; Riesz Logic may thus be considered a new fuzzy logic. In\naddition to applications in natural language processing, there is potential for\napplying the theory to neuro-fuzzy systems.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1410.3460v1",
            "date": "2014-10-13 11:37:37+00:00",
            "title": "Sentiment Analysis based on User Tag for Traditional Chinese Medicine in Weibo",
            "authors": [
                "Junhui Shen",
                "Peiyan Zhu",
                "Rui Fan",
                "Wei Tan"
            ],
            "abstract": "With the acceptance of Western culture and science, Traditional Chinese\nMedicine (TCM) has become a controversial issue in China. So, it's important to\nstudy the public's sentiment and opinion on TCM. The rapid development of\nonline social network, such as twitter, make it convenient and efficient to\nsample hundreds of millions of people for the aforementioned sentiment study.\nTo the best of our knowledge, the present work is the first attempt that\napplies sentiment analysis to the domain of TCM on Sina Weibo (a twitter-like\nmicroblogging service in China). In our work, firstly we collect tweets topic\nabout TCM from Sina Weibo, and label the tweets as supporting TCM and opposing\nTCM automatically based on user tag. Then, a support vector machine classifier\nhas been built to predict the sentiment of TCM tweets without labels. Finally,\nwe present a method to adjust the classifier result. The performance of\nF-measure attained with our method is 97%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.3791v1",
            "date": "2014-10-14 18:37:32+00:00",
            "title": "POLYGLOT-NER: Massive Multilingual Named Entity Recognition",
            "authors": [
                "Rami Al-Rfou",
                "Vivek Kulkarni",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "The increasing diversity of languages used on the web introduces a new level\nof complexity to Information Retrieval (IR) systems. We can no longer assume\nthat textual content is written in one language or even the same language\nfamily. In this paper, we demonstrate how to build massive multilingual\nannotators with minimal human expertise and intervention. We describe a system\nthat builds Named Entity Recognition (NER) annotators for 40 major languages\nusing Wikipedia and Freebase. Our approach does not require NER human annotated\ndatasets or language specific resources like treebanks, parallel corpora, and\northographic rules. The novelty of approach lies therein - using only language\nagnostic techniques, while achieving competitive performance.\n  Our method learns distributed word representations (word embeddings) which\nencode semantic and syntactic features of words in each language. Then, we\nautomatically generate datasets from Wikipedia link structure and Freebase\nattributes. Finally, we apply two preprocessing stages (oversampling and exact\nsurface form matching) which do not require any linguistic expertise.\n  Our evaluation is two fold: First, we demonstrate the system performance on\nhuman annotated datasets. Second, for languages where no gold-standard\nbenchmarks are available, we propose a new method, distant evaluation, based on\nstatistical machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4281v2",
            "date": "2014-10-16 02:44:41+00:00",
            "title": "Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition",
            "authors": [
                "Xiangang Li",
                "Xihong Wu"
            ],
            "abstract": "Long short-term memory (LSTM) based acoustic modeling methods have recently\nbeen shown to give state-of-the-art performance on some speech recognition\ntasks. To achieve a further performance improvement, in this research, deep\nextensions on LSTM are investigated considering that deep hierarchical model\nhas turned out to be more efficient than a shallow one. Motivated by previous\nresearch on constructing deep recurrent neural networks (RNNs), alternative\ndeep LSTM architectures are proposed and empirically evaluated on a large\nvocabulary conversational telephone speech recognition task. Meanwhile,\nregarding to multi-GPU devices, the training process for LSTM networks is\nintroduced and discussed. Experimental results demonstrate that the deep LSTM\nnetworks benefit from the depth and yield the state-of-the-art performance on\nthis task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.4445v3",
            "date": "2014-10-16 14:25:01+00:00",
            "title": "Patterns in the English Language: Phonological Networks, Percolation and Assembly Models",
            "authors": [
                "Massimo Stella",
                "Markus Brede"
            ],
            "abstract": "In this paper we provide a quantitative framework for the study of\nphonological networks (PNs) for the English language by carrying out principled\ncomparisons to null models, either based on site percolation, randomization\ntechniques, or network growth models. In contrast to previous work, we mainly\nfocus on null models that reproduce lower order characteristics of the\nempirical data. We find that artificial networks matching connectivity\nproperties of the English PN are exceedingly rare: this leads to the hypothesis\nthat the word repertoire might have been assembled over time by preferentially\nintroducing new words which are small modifications of old words. Our null\nmodels are able to explain the \"power-law-like\" part of the degree\ndistributions and generally retrieve qualitative features of the PN such as\nhigh clustering, high assortativity coefficient, and small-world\ncharacteristics. However, the detailed comparison to expectations from null\nmodels also points out significant differences, suggesting the presence of\nadditional constraints in word assembly. Key constraints we identify are the\navoidance of large degrees, the avoidance of triadic closure, and the avoidance\nof large non-percolating clusters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.5078v1",
            "date": "2014-10-19 14:34:23+00:00",
            "title": "Learning Vague Concepts for the Semantic Web",
            "authors": [
                "Paolo Pareti",
                "Ewan Klein"
            ],
            "abstract": "Ontologies can be a powerful tool for structuring knowledge, and they are\ncurrently the subject of extensive research. Updating the contents of an\nontology or improving its interoperability with other ontologies is an\nimportant but difficult process. In this paper, we focus on the presence of\nvague concepts, which are pervasive in natural language, within the framework\nof formal ontologies. We will adopt a framework in which vagueness is captured\nvia numerical restrictions that can be automatically adjusted. Since updating\nvague concepts, either through ontology alignment or ontology evolution, can\nlead to inconsistent sets of axioms, we define and implement a method to\ndetecting and repairing such inconsistencies in a local fashion.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1410.6830v1",
            "date": "2014-10-24 20:34:01+00:00",
            "title": "Clustering Words by Projection Entropy",
            "authors": [
                "I\u015f\u0131k Bar\u0131\u015f Fidaner",
                "Ali Taylan Cemgil"
            ],
            "abstract": "We apply entropy agglomeration (EA), a recently introduced algorithm, to\ncluster the words of a literary text. EA is a greedy agglomerative procedure\nthat minimizes projection entropy (PE), a function that can quantify the\nsegmentedness of an element set. To apply it, the text is reduced to a feature\nallocation, a combinatorial object to represent the word occurences in the\ntext's paragraphs. The experiment results demonstrate that EA, despite its\nreduction and simplicity, is useful in capturing significant relationships\namong the words in the text. This procedure was implemented in Python and\npublished as a free software: REBUS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.6903v1",
            "date": "2014-10-25 09:40:46+00:00",
            "title": "Choice of Mel Filter Bank in Computing MFCC of a Resampled Speech",
            "authors": [
                "Laxmi Narayana M.",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Mel Frequency Cepstral Coefficients (MFCCs) are the most popularly used\nspeech features in most speech and speaker recognition applications. In this\npaper, we study the effect of resampling a speech signal on these speech\nfeatures. We first derive a relationship between the MFCC param- eters of the\nresampled speech and the MFCC parameters of the original speech. We propose six\nmethods of calculating the MFCC parameters of downsampled speech by\ntransforming the Mel filter bank used to com- pute MFCC of the original speech.\nWe then experimentally compute the MFCC parameters of the down sampled speech\nusing the proposed meth- ods and compute the Pearson coefficient between the\nMFCC parameters of the downsampled speech and that of the original speech to\nidentify the most effective choice of Mel-filter band that enables the computed\nMFCC of the resampled speech to be as close as possible to the original speech\nsample MFCC.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1410.7382v1",
            "date": "2014-10-25 10:00:14+00:00",
            "title": "Modified Mel Filter Bank to Compute MFCC of Subsampled Speech",
            "authors": [
                "Kiran Kumar Bhuvanagiri",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Mel Frequency Cepstral Coefficients (MFCCs) are the most popularly used\nspeech features in most speech and speaker recognition applications. In this\nwork, we propose a modified Mel filter bank to extract MFCCs from subsampled\nspeech. We also propose a stronger metric which effectively captures the\ncorrelation between MFCCs of original speech and MFCC of resampled speech. It\nis found that the proposed method of filter bank construction performs\ndistinguishably well and gives recognition performance on resampled speech\nclose to recognition accuracies on original speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8149v1",
            "date": "2014-10-29 20:07:21+00:00",
            "title": "Detecting Structural Irregularity in Electronic Dictionaries Using Language Modeling",
            "authors": [
                "Paul Rodrigues",
                "David Zajic",
                "David Doermann",
                "Michael Bloodgood",
                "Peng Ye"
            ],
            "abstract": "Dictionaries are often developed using tools that save to Extensible Markup\nLanguage (XML)-based standards. These standards often allow high-level\nrepeating elements to represent lexical entries, and utilize descendants of\nthese repeating elements to represent the structure within each lexical entry,\nin the form of an XML tree. In many cases, dictionaries are published that have\nerrors and inconsistencies that are expensive to find manually. This paper\ndiscusses a method for dictionary writers to quickly audit structural\nregularity across entries in a dictionary by using statistical language\nmodeling. The approach learns the patterns of XML nodes that could occur within\nan XML tree, and then calculates the probability of each XML tree in the\ndictionary against these patterns to look for entries that diverge from the\nnorm.",
            "journal ref": "In Proceedings of Electronic Lexicography in the 21st Century\n  (eLex), pages 227-232, Bled, Slovenia, November 2011. Trojina Institute for\n  Applied Slovene Studies",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8498v2",
            "date": "2014-10-30 19:02:48+00:00",
            "title": "Training for Fast Sequential Prediction Using Dynamic Feature Selection",
            "authors": [
                "Emma Strubell",
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "We present paired learning and inference algorithms for significantly\nreducing computation and increasing speed of the vector dot products in the\nclassifiers that are at the heart of many NLP components. This is accomplished\nby partitioning the features into a sequence of templates which are ordered\nsuch that high confidence can often be reached using only a small fraction of\nall features. Parameter estimation is arranged to maximize accuracy and early\nconfidence in this sequence. We present experiments in left-to-right\npart-of-speech tagging on WSJ, demonstrating that we can preserve accuracy\nabove 97% with over a five-fold reduction in run-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8581v1",
            "date": "2014-10-30 22:38:11+00:00",
            "title": "Semi-Automatic Construction of a Domain Ontology for Wind Energy Using Wikipedia Articles",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Yusuf Arslan"
            ],
            "abstract": "Domain ontologies are important information sources for knowledge-based\nsystems. Yet, building domain ontologies from scratch is known to be a very\nlabor-intensive process. In this study, we present our semi-automatic approach\nto building an ontology for the domain of wind energy which is an important\ntype of renewable energy with a growing share in electricity generation all\nover the world. Related Wikipedia articles are first processed in an automated\nmanner to determine the basic concepts of the domain together with their\nproperties and next the concepts, properties, and relationships are organized\nto arrive at the ultimate ontology. We also provide pointers to other\nengineering ontologies which could be utilized together with the proposed wind\nenergy ontology in addition to its prospective application areas. The current\nstudy is significant as, to the best of our knowledge, it proposes the first\nconsiderably wide-coverage ontology for the wind energy domain and the ontology\nis built through a semi-automatic process which makes use of the related Web\nresources, thereby reducing the overall cost of the ontology building process.",
            "journal ref": "Renewable Energy, Volume 62, pp. 484-489, February 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8749v1",
            "date": "2014-10-30 18:51:55+00:00",
            "title": "What a Nasty day: Exploring Mood-Weather Relationship from Twitter",
            "authors": [
                "Jiwei Li",
                "Xun Wang",
                "Eduard Hovy"
            ],
            "abstract": "While it has long been believed in psychology that weather somehow influences\nhuman's mood, the debates have been going on for decades about how they are\ncorrelated. In this paper, we try to study this long-lasting topic by\nharnessing a new source of data compared from traditional psychological\nresearches: Twitter. We analyze 2 years' twitter data collected by twitter API\nwhich amounts to $10\\%$ of all postings and try to reveal the correlations\nbetween multiple dimensional structure of human mood with meteorological\neffects. Some of our findings confirm existing hypotheses, while others\ncontradict them. We are hopeful that our approach, along with the new data\nsource, can shed on the long-going debates on weather-mood correlation.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1410.8783v1",
            "date": "2014-10-31 15:53:49+00:00",
            "title": "Supervised learning model for parsing Arabic language",
            "authors": [
                "Nabil Khoufi",
                "Chafik Aloulou",
                "Lamia Hadrich Belguith"
            ],
            "abstract": "Parsing the Arabic language is a difficult task given the specificities of\nthis language and given the scarcity of digital resources (grammars and\nannotated corpora). In this paper, we suggest a method for Arabic parsing based\non supervised machine learning. We used the SVMs algorithm to select the\nsyntactic labels of the sentence. Furthermore, we evaluated our parser\nfollowing the cross validation method by using the Penn Arabic Treebank. The\nobtained results are very encouraging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8808v1",
            "date": "2014-10-29 15:48:40+00:00",
            "title": "A Semantic Web of Know-How: Linked Data for Community-Centric Tasks",
            "authors": [
                "Paolo Pareti",
                "Ewan Klein",
                "Adam Barker"
            ],
            "abstract": "This paper proposes a novel framework for representing community know-how on\nthe Semantic Web. Procedural knowledge generated by web communities typically\ntakes the form of natural language instructions or videos and is largely\nunstructured. The absence of semantic structure impedes the deployment of many\nuseful applications, in particular the ability to discover and integrate\nknow-how automatically. We discuss the characteristics of community know-how\nand argue that existing knowledge representation frameworks fail to represent\nit adequately. We present a novel framework for representing the semantic\nstructure of community know-how and demonstrate the feasibility of our approach\nby providing a concrete implementation which includes a method for\nautomatically acquiring procedural knowledge for real-world tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1401.1803v1",
            "date": "2014-01-08 20:36:57+00:00",
            "title": "Learning Multilingual Word Representations using a Bag-of-Words Autoencoder",
            "authors": [
                "Stanislas Lauly",
                "Alex Boulanger",
                "Hugo Larochelle"
            ],
            "abstract": "Recent work on learning multilingual word representations usually relies on\nthe use of word-level alignements (e.g. infered with the help of GIZA++)\nbetween translated sentences, in order to align the word embeddings in\ndifferent languages. In this workshop paper, we investigate an autoencoder\nmodel for learning multilingual word representations that does without such\nword-level alignements. The autoencoder is trained to reconstruct the\nbag-of-word representation of given sentence from an encoded representation\nextracted from its translation. We evaluate our approach on a multilingual\ndocument classification task, where labeled data is available only for one\nlanguage (e.g. English) while classification must be performed in a different\nlanguage (e.g. French). In our experiments, we observe that our method compares\nfavorably with a previously proposed method that exploits word-level alignments\nto learn word representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3479v1",
            "date": "2014-01-15 05:33:57+00:00",
            "title": "Complex Question Answering: Unsupervised Learning Approaches and Experiments",
            "authors": [
                "Yllias Chali",
                "Shafiq Rayhan Joty",
                "Sadid A. Hasan"
            ],
            "abstract": "Complex questions that require inferencing and synthesizing information from\nmultiple documents can be seen as a kind of topic-oriented, informative\nmulti-document summarization where the goal is to produce a single text as a\ncompressed version of a set of documents with a minimum loss of relevant\ninformation. In this paper, we experiment with one empirical method and two\nunsupervised statistical machine learning techniques: K-means and Expectation\nMaximization (EM), for computing relative importance of the sentences. We\ncompare the results of these approaches. Our experiments show that the\nempirical approach outperforms the other two techniques and EM performs better\nthan K-means. However, the performance of these approaches depends entirely on\nthe feature set used and the weighting of these features. In order to measure\nthe importance and relevance to the user query we extract different kinds of\nfeatures (i.e. lexical, lexical semantic, cosine similarity, basic element,\ntree kernel based syntactic and shallow-semantic) for each of the document\nsentences. We use a local search technique to learn the weights of the\nfeatures. To the best of our knowledge, no study has used tree kernel functions\nto encode syntactic/semantic information for more complex tasks such as\ncomputing the relatedness between the query sentences and the document\nsentences in order to generate query-focused summaries (or answers to complex\nquestions). For each of our methods of generating summaries (i.e. empirical,\nK-means and EM) we show the effects of syntactic and shallow-semantic features\nover the bag-of-words (BOW) features.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 35, pages\n  1-47, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3482v1",
            "date": "2014-01-15 05:35:49+00:00",
            "title": "Enhancing QA Systems with Complex Temporal Question Processing Capabilities",
            "authors": [
                "Estela Saquete",
                "Jose Luis Vicedo",
                "Patricio Mart\u00ednez-Barco",
                "Rafael Mu\u00f1oz",
                "Hector Llorens"
            ],
            "abstract": "This paper presents a multilayered architecture that enhances the\ncapabilities of current QA systems and allows different types of complex\nquestions or queries to be processed. The answers to these questions need to be\ngathered from factual information scattered throughout different documents.\nSpecifically, we designed a specialized layer to process the different types of\ntemporal questions. Complex temporal questions are first decomposed into simple\nquestions, according to the temporal relations expressed in the original\nquestion. In the same way, the answers to the resulting simple questions are\nrecomposed, fulfilling the temporal restrictions of the original complex\nquestion. A novel aspect of this approach resides in the decomposition which\nuses a minimal quantity of resources, with the final aim of obtaining a\nportable platform that is easily extensible to other languages. In this paper\nwe also present a methodology for evaluation of the decomposition of the\nquestions as well as the ability of the implemented temporal layer to perform\nat a multilingual level. The temporal layer was first performed for English,\nthen evaluated and compared with: a) a general purpose QA system (F-measure\n65.47% for QA plus English temporal layer vs. 38.01% for the general QA\nsystem), and b) a well-known QA system. Much better results were obtained for\ntemporal questions with the multilayered system. This system was therefore\nextended to Spanish and very good results were again obtained in the evaluation\n(F-measure 40.36% for QA plus Spanish temporal layer vs. 22.94% for the general\nQA system).",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 35, pages\n  775-811, 2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.3488v1",
            "date": "2014-01-15 05:38:17+00:00",
            "title": "Content Modeling Using Latent Permutations",
            "authors": [
                "Harr Chen",
                "S. R. K. Branavan",
                "Regina Barzilay",
                "David R. Karger"
            ],
            "abstract": "We present a novel Bayesian topic model for learning discourse-level document\nstructure. Our model leverages insights from discourse theory to constrain\nlatent topic assignments in a way that reflects the underlying organization of\ndocument topics. We propose a global model in which both topic selection and\nordering are biased to be similar across a collection of related documents. We\nshow that this space of orderings can be effectively represented using a\ndistribution over permutations called the Generalized Mallows Model. We apply\nour method to three complementary discourse-level tasks: cross-document\nalignment, document segmentation, and information ordering. Our experiments\nshow that incorporating our permutation-based model in these applications\nyields substantial improvements in performance over previously proposed\nmethods.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 36, pages\n  129-163, 2009",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.4634v1",
            "date": "2014-01-19 03:57:39+00:00",
            "title": "The Capacity of String-Replication Systems",
            "authors": [
                "Farzad Farnoud",
                "Moshe Schwartz",
                "Jehoshua Bruck"
            ],
            "abstract": "It is known that the majority of the human genome consists of repeated\nsequences. Furthermore, it is believed that a significant part of the rest of\nthe genome also originated from repeated sequences and has mutated to its\ncurrent form. In this paper, we investigate the possibility of constructing an\nexponentially large number of sequences from a short initial sequence and\nsimple replication rules, including those resembling genomic replication\nprocesses. In other words, our goal is to find out the capacity, or the\nexpressive power, of these string-replication systems. Our results include\nexact capacities, and bounds on the capacities, of four fundamental\nstring-replication systems.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1401.5327v1",
            "date": "2014-01-21 14:28:01+00:00",
            "title": "Compositional Operators in Distributional Semantics",
            "authors": [
                "Dimitri Kartsaklis"
            ],
            "abstract": "This survey presents in some detail the main advances that have been recently\ntaking place in Computational Linguistics towards the unification of the two\nprominent semantic paradigms: the compositional formal semantics view and the\ndistributional models of meaning based on vector spaces. After an introduction\nto these two approaches, I review the most important models that aim to provide\ncompositionality in distributional semantics. Then I proceed and present in\nmore detail a particular framework by Coecke, Sadrzadeh and Clark (2010) based\non the abstract mathematical setting of category theory, as a more complete\nexample capable to demonstrate the diversity of techniques and scientific\ndisciplines that this kind of research can draw from. This paper concludes with\na discussion about important open issues that need to be addressed by the\nresearchers in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5389v1",
            "date": "2014-01-16 04:56:03+00:00",
            "title": "Which Clustering Do You Want? Inducing Your Ideal Clustering with Minimal Feedback",
            "authors": [
                "Sajib Dasgupta",
                "Vincent Ng"
            ],
            "abstract": "While traditional research on text clustering has largely focused on grouping\ndocuments by topic, it is conceivable that a user may want to cluster documents\nalong other dimensions, such as the authors mood, gender, age, or sentiment.\nWithout knowing the users intention, a clustering algorithm will only group\ndocuments along the most prominent dimension, which may not be the one the user\ndesires. To address the problem of clustering documents along the user-desired\ndimension, previous work has focused on learning a similarity metric from data\nmanually annotated with the users intention or having a human construct a\nfeature space in an interactive manner during the clustering process. With the\ngoal of reducing reliance on human knowledge for fine-tuning the similarity\nfunction or selecting the relevant features required by these approaches, we\npropose a novel active clustering algorithm, which allows a user to easily\nselect the dimension along which she wants to cluster the documents by\ninspecting only a small number of words. We demonstrate the viability of our\nalgorithm on a variety of commonly-used sentiment datasets.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 39, pages\n  581-632, 2010",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1401.5390v1",
            "date": "2014-01-18 21:10:57+00:00",
            "title": "Learning to Win by Reading Manuals in a Monte-Carlo Framework",
            "authors": [
                "S. R. K. Branavan",
                "David Silver",
                "Regina Barzilay"
            ],
            "abstract": "Domain knowledge is crucial for effective performance in autonomous control\nsystems. Typically, human effort is required to encode this knowledge into a\ncontrol algorithm. In this paper, we present an approach to language grounding\nwhich automatically interprets text in the context of a complex control\napplication, such as a game, and uses domain knowledge extracted from the text\nto improve control performance. Both text analysis and control strategies are\nlearned jointly using only a feedback signal inherent to the application. To\neffectively leverage textual information, our method automatically extracts the\ntext segment most relevant to the current game state, and labels it with a\ntask-centric predicate structure. This labeled text is then used to bias an\naction selection policy for the game, guiding it towards promising regions of\nthe action space. We encode our model for text analysis and game playing in a\nmulti-layer neural network, representing linguistic decisions via latent\nvariables in the hidden layers, and game action quality via the output layer.\nOperating within the Monte-Carlo Search framework, we estimate model parameters\nusing feedback from simulated games. We apply our approach to the complex\nstrategy game Civilization II using the official game manual as the text guide.\nOur results show that a linguistically-informed game-playing agent\nsignificantly outperforms its language-unaware counterpart, yielding a 34%\nabsolute improvement and winning over 65% of games when playing against the\nbuilt-in AI of Civilization.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 43, pages\n  661-704, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.5980v1",
            "date": "2014-01-23 14:20:58+00:00",
            "title": "Reasoning about Meaning in Natural Language with Compact Closed Categories and Frobenius Algebras",
            "authors": [
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh",
                "Stephen Pulman",
                "Bob Coecke"
            ],
            "abstract": "Compact closed categories have found applications in modeling quantum\ninformation protocols by Abramsky-Coecke. They also provide semantics for\nLambek's pregroup algebras, applied to formalizing the grammatical structure of\nnatural language, and are implicit in a distributional model of word meaning\nbased on vector spaces. Specifically, in previous work Coecke-Clark-Sadrzadeh\nused the product category of pregroups with vector spaces and provided a\ndistributional model of meaning for sentences. We recast this theory in terms\nof strongly monoidal functors and advance it via Frobenius algebras over vector\nspaces. The former are used to formalize topological quantum field theories by\nAtiyah and Baez-Dolan, and the latter are used to model classical data in\nquantum protocols by Coecke-Pavlovic-Vicary. The Frobenius algebras enable us\nto work in a single space in which meanings of words, phrases, and sentences of\nany structure live. Hence we can compare meanings of different language\nconstructs and enhance the applicability of the theory. We report on\nexperimental results on a number of language tasks and verify the theoretical\npredictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1401.6169v2",
            "date": "2014-01-22 21:47:48+00:00",
            "title": "Parsimonious Topic Models with Salient Word Discovery",
            "authors": [
                "Hossein Soleimani",
                "David J. Miller"
            ],
            "abstract": "We propose a parsimonious topic model for text corpora. In related models\nsuch as Latent Dirichlet Allocation (LDA), all words are modeled\ntopic-specifically, even though many words occur with similar frequencies\nacross different topics. Our modeling determines salient words for each topic,\nwhich have topic-specific probabilities, with the rest explained by a universal\nshared model. Further, in LDA all topics are in principle present in every\ndocument. By contrast our model gives sparse topic representation, determining\nthe (small) subset of relevant topics for each document. We derive a Bayesian\nInformation Criterion (BIC), balancing model complexity and goodness of fit.\nHere, interestingly, we identify an effective sample size and corresponding\npenalty specific to each parameter type in our model. We minimize BIC to\njointly determine our entire model -- the topic-specific words,\ndocument-specific topics, all model parameter values, {\\it and} the total\nnumber of topics -- in a wholly unsupervised fashion. Results on three text\ncorpora and an image dataset show that our model achieves higher test set\nlikelihood and better agreement with ground-truth class labels, compared to LDA\nand to a model designed to incorporate sparsity.",
            "journal ref": "IEEE Transaction on Knowledge and Data Engineering, 27 (2015)\n  824-837",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1401.6574v1",
            "date": "2014-01-25 19:54:18+00:00",
            "title": "Category theory, logic and formal linguistics: some connections, old and new",
            "authors": [
                "Jean Gillibert",
                "Christian Retor\u00e9"
            ],
            "abstract": "We seize the opportunity of the publication of selected papers from the\n\\emph{Logic, categories, semantics} workshop in the \\emph{Journal of Applied\nLogic} to survey some current trends in logic, namely intuitionistic and linear\ntype theories, that interweave categorical, geometrical and computational\nconsiderations. We thereafter present how these rich logical frameworks can\nmodel the way language conveys meaning.",
            "journal ref": "Journal of Applied Logic 12, 1 (2014) 1--13",
            "category": "math.CT"
        },
        {
            "url": "http://arxiv.org/abs/1401.8269v1",
            "date": "2014-01-31 19:42:19+00:00",
            "title": "Experiments with Three Approaches to Recognizing Lexical Entailment",
            "authors": [
                "Peter D. Turney",
                "Saif M. Mohammad"
            ],
            "abstract": "Inference in natural language often involves recognizing lexical entailment\n(RLE); that is, identifying whether one word entails another. For example,\n\"buy\" entails \"own\". Two general strategies for RLE have been proposed: One\nstrategy is to manually construct an asymmetric similarity measure for context\nvectors (directional similarity) and another is to treat RLE as a problem of\nlearning to recognize semantic relations using supervised machine learning\ntechniques (relation classification). In this paper, we experiment with two\nrecent state-of-the-art representatives of the two general strategies. The\nfirst approach is an asymmetric similarity measure (an instance of the\ndirectional similarity strategy), designed to capture the degree to which the\ncontexts of a word, a, form a subset of the contexts of another word, b. The\nsecond approach (an instance of the relation classification strategy)\nrepresents a word pair, a:b, with a feature vector that is the concatenation of\nthe context vectors of a and b, and then applies supervised learning to a\ntraining set of labeled feature vectors. Additionally, we introduce a third\napproach that is a new instance of the relation classification strategy. The\nthird approach represents a word pair, a:b, with a feature vector in which the\nfeatures are the differences in the similarities of a and b to a set of\nreference words. All three approaches use vector space models (VSMs) of\nsemantics, based on word-context matrices. We perform an extensive evaluation\nof the three approaches using three different datasets. The proposed new\napproach (similarity differences) performs significantly better than the other\ntwo approaches on some datasets and there is no dataset for which it is\nsignificantly worse. Our results suggest it is beneficial to make connections\nbetween the research in lexical entailment and the research in semantic\nrelation classification.",
            "journal ref": "Natural Language Engineering, 21 (3), (2015), 437-476",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.0473v7",
            "date": "2014-09-01 16:33:02+00:00",
            "title": "Neural Machine Translation by Jointly Learning to Align and Translate",
            "authors": [
                "Dzmitry Bahdanau",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "Neural machine translation is a recently proposed approach to machine\ntranslation. Unlike the traditional statistical machine translation, the neural\nmachine translation aims at building a single neural network that can be\njointly tuned to maximize the translation performance. The models proposed\nrecently for neural machine translation often belong to a family of\nencoder-decoders and consists of an encoder that encodes a source sentence into\na fixed-length vector from which a decoder generates a translation. In this\npaper, we conjecture that the use of a fixed-length vector is a bottleneck in\nimproving the performance of this basic encoder-decoder architecture, and\npropose to extend this by allowing a model to automatically (soft-)search for\nparts of a source sentence that are relevant to predicting a target word,\nwithout having to form these parts as a hard segment explicitly. With this new\napproach, we achieve a translation performance comparable to the existing\nstate-of-the-art phrase-based system on the task of English-to-French\ntranslation. Furthermore, qualitative analysis reveals that the\n(soft-)alignments found by the model agree well with our intuition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.1257v2",
            "date": "2014-09-03 21:00:49+00:00",
            "title": "Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation",
            "authors": [
                "Jean Pouget-Abadie",
                "Dzmitry Bahdanau",
                "Bart van Merrienboer",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "The authors of (Cho et al., 2014a) have shown that the recently introduced\nneural network translation systems suffer from a significant drop in\ntranslation quality when translating long sentences, unlike existing\nphrase-based translation systems. In this paper, we propose a way to address\nthis issue by automatically segmenting an input sentence into phrases that can\nbe easily translated by the neural network translation model. Once each segment\nhas been independently translated by the neural machine translation model, the\ntranslated clauses are concatenated to form a final translation. Empirical\nresults show a significant improvement in translation quality for long\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.1744v2",
            "date": "2014-09-05 11:42:26+00:00",
            "title": "Structure of a media co-occurrence network",
            "authors": [
                "V. A. Traag",
                "R. Reinanda",
                "G. van Klinken"
            ],
            "abstract": "Social networks have been of much interest in recent years. We here focus on\na network structure derived from co-occurrences of people in traditional\nnewspaper media. We find three clear deviations from what can be expected in a\nrandom graph. First, the average degree in the empirical network is much lower\nthan expected, and the average weight of a link much higher than expected.\nSecondly, high degree nodes attract disproportionately much weight. Thirdly,\nrelatively much of the weight seems to concentrate between high degree nodes.\nWe believe this can be explained by the fact that most people tend to co-occur\nrepeatedly with the same people. We create a model that replicates these\nobservations qualitatively based on two self-reinforcing processes: (1) more\nfrequently occurring persons are more likely to occur again; and (2) if two\npeople co-occur frequently, they are more likely to co-occur again. This\nsuggest that the media tends to focus on people that are already in the news,\nand that they reinforce existing co-occurrences.",
            "journal ref": "Proceedings of ECCS 2014, 2016, pp. 81-91",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1409.2195v2",
            "date": "2014-09-08 03:07:54+00:00",
            "title": "Analyzing the Language of Food on Social Media",
            "authors": [
                "Daniel Fried",
                "Mihai Surdeanu",
                "Stephen Kobourov",
                "Melanie Hingle",
                "Dane Bell"
            ],
            "abstract": "We investigate the predictive power behind the language of food on social\nmedia. We collect a corpus of over three million food-related posts from\nTwitter and demonstrate that many latent population characteristics can be\ndirectly predicted from this data: overweight rate, diabetes rate, political\nleaning, and home geographical location of authors. For all tasks, our\nlanguage-based models significantly outperform the majority-class baselines.\nPerformance is further improved with more complex natural language processing,\nsuch as topic modeling. We analyze which textual features have most predictive\npower for these datasets, providing insight into the connections between the\nlanguage of food, geographic locale, and community characteristics. Lastly, we\ndesign and implement an online system for real-time query and visualization of\nthe dataset. Visualization tools, such as geo-referenced heatmaps,\nsemantics-preserving wordclouds and temporal histograms, allow us to discover\nmore complex, global patterns mirrored in the language of food.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.2433v1",
            "date": "2014-09-08 17:19:11+00:00",
            "title": "Approximating solution structure of the Weighted Sentence Alignment problem",
            "authors": [
                "Antonina Kolokolova",
                "Renesa Nizamee"
            ],
            "abstract": "We study the complexity of approximating solution structure of the bijective\nweighted sentence alignment problem of DeNero and Klein (2008). In particular,\nwe consider the complexity of finding an alignment that has a significant\noverlap with an optimal alignment. We discuss ways of representing the solution\nfor the general weighted sentence alignment as well as phrases-to-words\nalignment problem, and show that computing a string which agrees with the\noptimal sentence partition on more than half (plus an arbitrarily small\npolynomial fraction) positions for the phrases-to-words alignment is NP-hard.\nFor the general weighted sentence alignment we obtain such bound from the\nagreement on a little over 2/3 of the bits. Additionally, we generalize the\nHamming distance approximation of a solution structure to approximating it with\nrespect to the edit distance metric, obtaining similar lower bounds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.2450v1",
            "date": "2014-09-08 18:14:16+00:00",
            "title": "Exploiting Social Network Structure for Person-to-Person Sentiment Analysis",
            "authors": [
                "Robert West",
                "Hristo S. Paskov",
                "Jure Leskovec",
                "Christopher Potts"
            ],
            "abstract": "Person-to-person evaluations are prevalent in all kinds of discourse and\nimportant for establishing reputations, building social bonds, and shaping\npublic opinion. Such evaluations can be analyzed separately using signed social\nnetworks and textual sentiment analysis, but this misses the rich interactions\nbetween language and social context. To capture such interactions, we develop a\nmodel that predicts individual A's opinion of individual B by synthesizing\ninformation from the signed social network in which A and B are embedded with\nsentiment analysis of the evaluative texts relating A to B. We prove that this\nproblem is NP-hard but can be relaxed to an efficiently solvable hinge-loss\nMarkov random field, and we show that this implementation outperforms text-only\nand network-only versions in two very different datasets involving\ncommunity-level decision-making: the Wikipedia Requests for Adminship corpus\nand the Convote U.S. Congressional speech corpus.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1409.2993v1",
            "date": "2014-09-10 08:41:35+00:00",
            "title": "\"Look Ma, No Hands!\" A Parameter-Free Topic Model",
            "authors": [
                "Jian Tang",
                "Ming Zhang",
                "Qiaozhu Mei"
            ],
            "abstract": "It has always been a burden to the users of statistical topic models to\npredetermine the right number of topics, which is a key parameter of most topic\nmodels. Conventionally, automatic selection of this parameter is done through\neither statistical model selection (e.g., cross-validation, AIC, or BIC) or\nBayesian nonparametric models (e.g., hierarchical Dirichlet process). These\nmethods either rely on repeated runs of the inference algorithm to search\nthrough a large range of parameter values which does not suit the mining of big\ndata, or replace this parameter with alternative parameters that are less\nintuitive and still hard to be determined. In this paper, we explore to\n\"eliminate\" this parameter from a new perspective. We first present a\nnonparametric treatment of the PLSA model named nonparametric probabilistic\nlatent semantic analysis (nPLSA). The inference procedure of nPLSA allows for\nthe exploration and comparison of different numbers of topics within a single\nexecution, yet remains as simple as that of PLSA. This is achieved by\nsubstituting the parameter of the number of topics with an alternative\nparameter that is the minimal goodness of fit of a document. We show that the\nnew parameter can be further eliminated by two parameter-free treatments:\neither by monitoring the diversity among the discovered topics or by a weak\nsupervision from users in the form of an exemplar topic. The parameter-free\ntopic model finds the appropriate number of topics when the diversity among the\ndiscovered topics is maximized, or when the granularity of the discovered\ntopics matches the exemplar topic. Experiments on both synthetic and real data\nprove that the parameter-free topic model extracts topics with a comparable\nquality comparing to classical topic models with \"manual transmission\". The\nquality of the topics outperforms those extracted through classical Bayesian\nnonparametric models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1409.3881v1",
            "date": "2014-09-12 22:40:38+00:00",
            "title": "An Approach to Reducing Annotation Costs for BioNLP",
            "authors": [
                "Michael Bloodgood",
                "K. Vijay-Shanker"
            ],
            "abstract": "There is a broad range of BioNLP tasks for which active learning (AL) can\nsignificantly reduce annotation costs and a specific AL algorithm we have\ndeveloped is particularly effective in reducing annotation costs for these\ntasks. We have previously developed an AL algorithm called ClosestInitPA that\nworks best with tasks that have the following characteristics: redundancy in\ntraining material, burdensome annotation costs, Support Vector Machines (SVMs)\nwork well for the task, and imbalanced datasets (i.e. when set up as a binary\nclassification problem, one class is substantially rarer than the other). Many\nBioNLP tasks have these characteristics and thus our AL algorithm is a natural\napproach to apply to BioNLP tasks.",
            "journal ref": "In Proceedings of the Workshop on Current Trends in Biomedical\n  Natural Language Processing, pages 104-105, Columbus, Ohio, June 2008.\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.4617v1",
            "date": "2014-09-16 13:03:51+00:00",
            "title": "The Role of Emotions in Propagating Brands in Social Networks",
            "authors": [
                "Ronald Hochreiter",
                "Christoph Waldhauser"
            ],
            "abstract": "A key aspect of word of mouth marketing are emotions. Emotions in texts help\npropagating messages in conventional advertising. In word of mouth scenarios,\nemotions help to engage consumers and incite to propagate the message further.\nWhile the function of emotions in offline marketing in general and word of\nmouth marketing in particular is rather well understood, online marketing can\nonly offer a limited view on the function of emotions. In this contribution we\nseek to close this gap. We therefore investigate how emotions function in\nsocial media. To do so, we collected more than 30,000 brand marketing messages\nfrom the Google+ social networking site. Using state of the art computational\nlinguistics classifiers, we compute the sentiment of these messages. Starting\nout with Poisson regression-based baseline models, we seek to replicate earlier\nfindings using this large data set. We extend upon earlier research by\ncomputing multi-level mixed effects models that compare the function of\nemotions across different industries. We find that while the well known notion\nof activating emotions propagating messages holds in general for our data as\nwell. But there are significant differences between the observed industries.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1409.4835v1",
            "date": "2014-09-17 00:00:11+00:00",
            "title": "Taking into Account the Differences between Actively and Passively Acquired Data: The Case of Active Learning with Support Vector Machines for Imbalanced Datasets",
            "authors": [
                "Michael Bloodgood",
                "K. Vijay-Shanker"
            ],
            "abstract": "Actively sampled data can have very different characteristics than passively\nsampled data. Therefore, it's promising to investigate using different\ninference procedures during AL than are used during passive learning (PL). This\ngeneral idea is explored in detail for the focused case of AL with\ncost-weighted SVMs for imbalanced data, a situation that arises for many HLT\ntasks. The key idea behind the proposed InitPA method for addressing imbalance\nis to base cost models during AL on an estimate of overall corpus imbalance\ncomputed via a small unbiased sample rather than the imbalance in the labeled\ntraining data, which is the leading method used during PL.",
            "journal ref": "Proceedings of HLT: The 2009 Annual Conference of the North\n  American Chapter of the Association for Computational Linguistics, Short\n  Papers, pages 137-140, Boulder, Colorado, June 2009. Association for\n  Computational Linguistics",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1409.5165v1",
            "date": "2014-09-17 23:28:59+00:00",
            "title": "A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping",
            "authors": [
                "Michael Bloodgood",
                "K. Vijay-Shanker"
            ],
            "abstract": "A survey of existing methods for stopping active learning (AL) reveals the\nneeds for methods that are: more widely applicable; more aggressive in saving\nannotations; and more stable across changing datasets. A new method for\nstopping AL based on stabilizing predictions is presented that addresses these\nneeds. Furthermore, stopping methods are required to handle a broad range of\ndifferent annotation/performance tradeoff valuations. Despite this, the\nexisting body of work is dominated by conservative methods with little (if any)\nattention paid to providing users with control over the behavior of stopping\nmethods. The proposed method is shown to fill a gap in the level of\naggressiveness available for stopping AL and supports providing users with\ncontrol over stopping behavior.",
            "journal ref": "In Proceedings of the Thirteenth Conference on Computational\n  Natural Language Learning (CoNLL-2009), pages 39-47, Boulder, Colorado, June\n  2009. Association for Computational Linguistics",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1409.7085v1",
            "date": "2014-09-24 20:16:49+00:00",
            "title": "Semantically-Informed Syntactic Machine Translation: A Tree-Grafting Approach",
            "authors": [
                "Kathryn Baker",
                "Michael Bloodgood",
                "Chris Callison-Burch",
                "Bonnie J. Dorr",
                "Nathaniel W. Filardo",
                "Lori Levin",
                "Scott Miller",
                "Christine Piatko"
            ],
            "abstract": "We describe a unified and coherent syntactic framework for supporting a\nsemantically-informed syntactic approach to statistical machine translation.\nSemantically enriched syntactic tags assigned to the target-language training\ntexts improved translation quality. The resulting system significantly\noutperformed a linguistically naive baseline model (Hiero), and reached the\nhighest scores yet reported on the NIST 2009 Urdu-English translation task.\nThis finding supports the hypothesis (posed by many researchers in the MT\ncommunity, e.g., in DARPA GALE) that both syntactic and semantic information\nare critical for improving translation quality---and further demonstrates that\nlarge gains can be achieved for low-resource languages with different word\norder than English.",
            "journal ref": "In Proceedings of the Ninth Conference of the Association for\n  Machine Translation in the Americas (AMTA), Denver, Colorado, October 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.7275v2",
            "date": "2014-09-25 14:42:02+00:00",
            "title": "The meaning-frequency law in Zipfian optimization models of communication",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "According to Zipf's meaning-frequency law, words that are more frequent tend\nto have more meanings. Here it is shown that a linear dependency between the\nfrequency of a form and its number of meanings is found in a family of models\nof Zipf's law for word frequencies. This is evidence for a weak version of the\nmeaning-frequency law. Interestingly, that weak law (a) is not an inevitable of\nproperty of the assumptions of the family and (b) is found at least in the\nnarrow regime where those models exhibit Zipf's law for word frequencies.",
            "journal ref": "Glottometrics 35, 28-37 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.7336v2",
            "date": "2014-09-23 16:36:21+00:00",
            "title": "Does network complexity help organize Babel's library?",
            "authors": [
                "Juan Pablo C\u00e1rdenas",
                "Iv\u00e1n Gonz\u00e1lez",
                "Gerardo Vidal",
                "Miguel Fuentes"
            ],
            "abstract": "In this work, we study properties of texts from the perspective of complex\nnetwork theory. Words in given texts are linked by co-occurrence and\ntransformed into networks, and we observe that these display topological\nproperties common to other complex systems. However, there are some properties\nthat seem to be exclusive to texts; many of these properties depend on the\nfrequency of words in the text, while others seem to be strictly determined by\nthe grammar. Precisely, these properties allow for a categorization of texts as\neither with a sense and others encoded or senseless.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1409.7985v5",
            "date": "2014-09-29 03:04:26+00:00",
            "title": "The Utility of Text: The Case of Amicus Briefs and the Supreme Court",
            "authors": [
                "Yanchuan Sim",
                "Bryan Routledge",
                "Noah A. Smith"
            ],
            "abstract": "We explore the idea that authoring a piece of text is an act of maximizing\none's expected utility. To make this idea concrete, we consider the societally\nimportant decisions of the Supreme Court of the United States. Extensive past\nwork in quantitative political science provides a framework for empirically\nmodeling the decisions of justices and how they relate to text. We incorporate\ninto such a model texts authored by amici curiae (\"friends of the court\"\nseparate from the litigants) who seek to weigh in on the decision, then\nexplicitly model their goals in a random utility model. We demonstrate the\nbenefits of this approach in improved vote prediction and the ability to\nperform counterfactual analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.8558v1",
            "date": "2014-09-30 14:20:29+00:00",
            "title": "A Deep Learning Approach to Data-driven Parameterizations for Statistical Parametric Speech Synthesis",
            "authors": [
                "Prasanna Kumar Muthukumar",
                "Alan W. Black"
            ],
            "abstract": "Nearly all Statistical Parametric Speech Synthesizers today use Mel Cepstral\ncoefficients as the vocal tract parameterization of the speech signal. Mel\nCepstral coefficients were never intended to work in a parametric speech\nsynthesis framework, but as yet, there has been little success in creating a\nbetter parameterization that is more suited to synthesis. In this paper, we use\ndeep learning algorithms to investigate a data-driven parameterization\ntechnique that is designed for the specific requirements of synthesis. We\ncreate an invertible, low-dimensional, noise-robust encoding of the Mel Log\nSpectrum by training a tapered Stacked Denoising Autoencoder (SDA). This SDA is\nthen unwrapped and used as the initialization for a Multi-Layer Perceptron\n(MLP). The MLP is fine-tuned by training it to reconstruct the input at the\noutput layer. This MLP is then split down the middle to form encoding and\ndecoding networks. These networks produce a parameterization of the Mel Log\nSpectrum that is intended to better fulfill the requirements of synthesis.\nResults are reported for experiments conducted using this resulting\nparameterization with the ClusterGen speech synthesizer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1058v2",
            "date": "2014-12-01 16:19:51+00:00",
            "title": "Effective Use of Word Order for Text Categorization with Convolutional Neural Networks",
            "authors": [
                "Rie Johnson",
                "Tong Zhang"
            ],
            "abstract": "Convolutional neural network (CNN) is a neural network that can make use of\nthe internal structure of data such as the 2D structure of image data. This\npaper studies CNN on text categorization to exploit the 1D structure (namely,\nword order) of text data for accurate prediction. Instead of using\nlow-dimensional word vectors as input as is often done, we directly apply CNN\nto high-dimensional text data, which leads to directly learning embedding of\nsmall text regions for use in classification. In addition to a straightforward\nadaptation of CNN from image to text, a simple but new variation which employs\nbag-of-word conversion in the convolution layer is proposed. An extension to\ncombine multiple convolution layers is also explored for higher accuracy. The\nexperiments demonstrate the effectiveness of our approach in comparison with\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1841v2",
            "date": "2014-12-02 22:30:53+00:00",
            "title": "Exemplar Dynamics and Sound Merger in Language",
            "authors": [
                "P. F. Tupper"
            ],
            "abstract": "We develop a model of phonological contrast in natural language.\nSpecifically, the model describes the maintenance of contrast between different\nwords in a language, and the elimination of such contrast when sounds in the\nwords merge. An example of such a contrast is that provided by the two vowel\nsounds 'i' and 'e', which distinguish pairs of words such as 'pin' and 'pen' in\nmost dialects of English. We model language users' knowledge of the\npronunciation of a word as consisting of collections of labeled exemplars\nstored in memory. Each exemplar is a detailed memory of a particular utterance\nof the word in question. In our model an exemplar is represented by one or two\nphonetic variables along with a weight indicating how strong the memory of the\nutterance is. Starting from an exemplar-level model we derive\nintegro-differential equations for the evolution of exemplar density fields in\nphonetic space. Using these latter equations we investigate under what\nconditions two sounds merge, thus eliminating the contrast. Our main conclusion\nis that for the preservation of phonological contrast, it is necessary that\nanomalous utterances of a given word are discarded, and not merely stored in\nmemory as an exemplar of another word.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.1866v4",
            "date": "2014-12-05 00:30:09+00:00",
            "title": "Integer-Programming Ensemble of Temporal-Relations Classifiers",
            "authors": [
                "Catherine Kerr",
                "Terri Hoare",
                "Paula Carroll",
                "Jakub Marecek"
            ],
            "abstract": "The extraction and understanding of temporal events and their relations are\nmajor challenges in natural language processing. Processing text on a\nsentence-by-sentence or expression-by-expression basis often fails, in part due\nto the challenge of capturing the global consistency of the text. We present an\nensemble method, which reconciles the outputs of multiple classifiers of\ntemporal expressions across the text using integer programming. Computational\nexperiments show that the ensemble improves upon the best individual results\nfrom two recent challenges, SemEval-2013 TempEval-3 (Temporal Annotation) and\nSemEval-2016 Task 12 (Clinical TempEval).",
            "journal ref": "Data Mining and Knowledge Discovery, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2486v2",
            "date": "2014-12-08 09:05:40+00:00",
            "title": "Optimization models of natural communication",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "A family of information theoretic models of communication was introduced more\nthan a decade ago to explain the origins of Zipf's law for word frequencies.\nThe family is a based on a combination of two information theoretic principles:\nmaximization of mutual information between forms and meanings and minimization\nof form entropy. The family also sheds light on the origins of three other\npatterns: the principle of contrast, a related vocabulary learning bias and the\nmeaning-frequency law. Here two important components of the family, namely the\ninformation theoretic principles and the energy function that combines them\nlinearly, are reviewed from the perspective of psycholinguistics, language\nlearning, information theory and synergetic linguistics. The minimization of\nthis linear function is linked to the problem of compression of standard\ninformation theory and might be tuned by self-organization.",
            "journal ref": "Journal of Quantitative Linguistics 25 (3), 207-237 (2018)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1412.2812v1",
            "date": "2014-12-08 23:40:41+00:00",
            "title": "Unsupervised Induction of Semantic Roles within a Reconstruction-Error Minimization Framework",
            "authors": [
                "Ivan Titov",
                "Ehsan Khoddam"
            ],
            "abstract": "We introduce a new approach to unsupervised estimation of feature-rich\nsemantic role labeling models. Our model consists of two components: (1) an\nencoding component: a semantic role labeling model which predicts roles given a\nrich set of syntactic and lexical features; (2) a reconstruction component: a\ntensor factorization model which relies on roles to predict argument fillers.\nWhen the components are estimated jointly to minimize errors in argument\nreconstruction, the induced roles largely correspond to roles defined in\nannotated resources. Our method performs on par with most accurate role\ninduction methods on English and German, even though, unlike these previous\napproaches, we do not incorporate any prior linguistic knowledge about the\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.2821v1",
            "date": "2014-12-09 00:39:16+00:00",
            "title": "Zipf's Law and the Frequency of Characters or Words of Oracles",
            "authors": [
                "Xiuli Wang"
            ],
            "abstract": "The article discusses the frequency of characters of Oracle,concluding that\nthe frequency and the rank of a word or character is fit to Zipf-Mandelboit Law\nor Zipf's law with three parameters,and figuring out the parameters based on\nthe frequency,and pointing out that what some researchers of Oracle call the\nassembling on the two ends is just a description by their impression about the\nOracle data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.3714v2",
            "date": "2014-12-11 16:35:27+00:00",
            "title": "Feature Weight Tuning for Recursive Neural Networks",
            "authors": [
                "Jiwei Li"
            ],
            "abstract": "This paper addresses how a recursive neural network model can automatically\nleave out useless information and emphasize important evidence, in other words,\nto perform \"weight tuning\" for higher-level representation acquisition. We\npropose two models, Weighted Neural Network (WNN) and Binary-Expectation Neural\nNetwork (BENN), which automatically control how much one specific unit\ncontributes to the higher-level representation. The proposed model can be\nviewed as incorporating a more powerful compositional function for embedding\nacquisition in recursive neural networks. Experimental results demonstrate the\nsignificant improvement over standard neural models.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1412.5335v7",
            "date": "2014-12-17 11:02:04+00:00",
            "title": "Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews",
            "authors": [
                "Gr\u00e9goire Mesnil",
                "Tomas Mikolov",
                "Marc'Aurelio Ranzato",
                "Yoshua Bengio"
            ],
            "abstract": "Sentiment analysis is a common task in natural language processing that aims\nto detect polarity of a text document (typically a consumer review). In the\nsimplest settings, we discriminate only between positive and negative\nsentiment, turning the task into a standard binary classification problem. We\ncompare several ma- chine learning approaches to this problem, and combine them\nto achieve the best possible results. We show how to use for this task the\nstandard generative lan- guage models, which are slightly complementary to the\nstate of the art techniques. We achieve strong results on a well-known dataset\nof IMDB movie reviews. Our results are easily reproducible, as we publish also\nthe code needed to repeat the experiments. This should simplify further advance\nof the state of the art, as other researchers can combine their techniques with\nours with little effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.5567v2",
            "date": "2014-12-17 20:39:45+00:00",
            "title": "Deep Speech: Scaling up end-to-end speech recognition",
            "authors": [
                "Awni Hannun",
                "Carl Case",
                "Jared Casper",
                "Bryan Catanzaro",
                "Greg Diamos",
                "Erich Elsen",
                "Ryan Prenger",
                "Sanjeev Satheesh",
                "Shubho Sengupta",
                "Adam Coates",
                "Andrew Y. Ng"
            ],
            "abstract": "We present a state-of-the-art speech recognition system developed using\nend-to-end deep learning. Our architecture is significantly simpler than\ntraditional speech systems, which rely on laboriously engineered processing\npipelines; these traditional systems also tend to perform poorly when used in\nnoisy environments. In contrast, our system does not need hand-designed\ncomponents to model background noise, reverberation, or speaker variation, but\ninstead directly learns a function that is robust to such effects. We do not\nneed a phoneme dictionary, nor even the concept of a \"phoneme.\" Key to our\napproach is a well-optimized RNN training system that uses multiple GPUs, as\nwell as a set of novel data synthesis techniques that allow us to efficiently\nobtain a large amount of varied data for training. Our system, called Deep\nSpeech, outperforms previously published results on the widely studied\nSwitchboard Hub5'00, achieving 16.0% error on the full test set. Deep Speech\nalso handles challenging noisy environments better than widely used,\nstate-of-the-art commercial speech systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6418v3",
            "date": "2014-12-19 16:30:33+00:00",
            "title": "Inducing Semantic Representation from Text by Jointly Predicting and Factorizing Relations",
            "authors": [
                "Ivan Titov",
                "Ehsan Khoddam"
            ],
            "abstract": "In this work, we propose a new method to integrate two recent lines of work:\nunsupervised induction of shallow semantics (e.g., semantic roles) and\nfactorization of relations in text and knowledge bases. Our model consists of\ntwo components: (1) an encoding component: a semantic role labeling model which\npredicts roles given a rich set of syntactic and lexical features; (2) a\nreconstruction component: a tensor factorization model which relies on roles to\npredict argument fillers. When the components are estimated jointly to minimize\nerrors in argument reconstruction, the induced roles largely correspond to\nroles defined in annotated resources. Our method performs on par with most\naccurate role induction methods on English, even though, unlike these previous\napproaches, we do not incorporate any prior linguistic knowledge about the\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6577v3",
            "date": "2014-12-20 01:53:22+00:00",
            "title": "Modeling Compositionality with Multiplicative Recurrent Neural Networks",
            "authors": [
                "Ozan \u0130rsoy",
                "Claire Cardie"
            ],
            "abstract": "We present the multiplicative recurrent neural network as a general model for\ncompositional meaning in language, and evaluate it on the task of fine-grained\nsentiment analysis. We establish a connection to the previously investigated\nmatrix-space models for compositionality, and show they are special cases of\nthe multiplicative recurrent net. Our experiments show that these models\nperform comparably or better than Elman-type additive recurrent neural networks\nand outperform matrix-space models on a standard fine-grained sentiment\nanalysis corpus. Furthermore, they yield comparable results to structural deep\nmodels on the recently published Stanford Sentiment Treebank without the need\nfor generating parse trees.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.6632v5",
            "date": "2014-12-20 08:10:04+00:00",
            "title": "Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)",
            "authors": [
                "Junhua Mao",
                "Wei Xu",
                "Yi Yang",
                "Jiang Wang",
                "Zhiheng Huang",
                "Alan Yuille"
            ],
            "abstract": "In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model\nfor generating novel image captions. It directly models the probability\ndistribution of generating a word given previous words and an image. Image\ncaptions are generated by sampling from this distribution. The model consists\nof two sub-networks: a deep recurrent neural network for sentences and a deep\nconvolutional network for images. These two sub-networks interact with each\nother in a multimodal layer to form the whole m-RNN model. The effectiveness of\nour model is validated on four benchmark datasets (IAPR TC-12, Flickr 8K,\nFlickr 30K and MS COCO). Our model outperforms the state-of-the-art methods. In\naddition, we apply the m-RNN model to retrieval tasks for retrieving images or\nsentences, and achieves significant performance improvement over the\nstate-of-the-art methods which directly optimize the ranking objective function\nfor retrieval. The project page of this work is:\nwww.stat.ucla.edu/~junhua.mao/m-RNN.html .",
            "journal ref": "ICLR 2015",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1412.6645v3",
            "date": "2014-12-20 11:54:41+00:00",
            "title": "Weakly Supervised Multi-Embeddings Learning of Acoustic Models",
            "authors": [
                "Gabriel Synnaeve",
                "Emmanuel Dupoux"
            ],
            "abstract": "We trained a Siamese network with multi-task same/different information on a\nspeech dataset, and found that it was possible to share a network for both\ntasks without a loss in performance. The first task was to discriminate between\ntwo same or different words, and the second was to discriminate between two\nsame or different talkers.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1412.6650v4",
            "date": "2014-12-20 13:06:05+00:00",
            "title": "Incremental Adaptation Strategies for Neural Network Language Models",
            "authors": [
                "Aram Ter-Sarkisov",
                "Holger Schwenk",
                "Loic Barrault",
                "Fethi Bougares"
            ],
            "abstract": "It is today acknowledged that neural network language models outperform\nbackoff language models in applications like speech recognition or statistical\nmachine translation. However, training these models on large amounts of data\ncan take several days. We present efficient techniques to adapt a neural\nnetwork language model to new data. Instead of training a completely new model\nor relying on mixture approaches, we propose two new methods: continued\ntraining on resampled data or insertion of adaptation layers. We present\nexperimental results in an CAT environment where the post-edits of professional\ntranslators are used to improve an SMT system. Both methods are very fast and\nachieve significant improvements without overfitting the small adaptation data.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1412.6815v2",
            "date": "2014-12-21 17:38:19+00:00",
            "title": "Extraction of Salient Sentences from Labelled Documents",
            "authors": [
                "Misha Denil",
                "Alban Demiraj",
                "Nando de Freitas"
            ],
            "abstract": "We present a hierarchical convolutional document model with an architecture\ndesigned to support introspection of the document structure. Using this model,\nwe show how to use visualisation techniques from the computer vision literature\nto identify and extract topic-relevant sentences.\n  We also introduce a new scalable evaluation technique for automatic sentence\nextraction systems that avoids the need for time consuming human annotation of\nvalidation data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.6881v3",
            "date": "2014-12-22 06:12:06+00:00",
            "title": "On Learning Vector Representations in Hierarchical Label Spaces",
            "authors": [
                "Jinseok Nam",
                "Johannes F\u00fcrnkranz"
            ],
            "abstract": "An important problem in multi-label classification is to capture label\npatterns or underlying structures that have an impact on such patterns. This\npaper addresses one such problem, namely how to exploit hierarchical structures\nover labels. We present a novel method to learn vector representations of a\nlabel space given a hierarchy of labels and label co-occurrence patterns. Our\nexperimental results demonstrate qualitatively that the proposed method is able\nto learn regularities among labels by exploiting a label hierarchy as well as\nlabel co-occurrences. It highlights the importance of the hierarchical\ninformation in order to obtain regularities which facilitate analogical\nreasoning over a label space. We also experimentally illustrate the dependency\nof the learned representations on the label hierarchy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.7028v4",
            "date": "2014-12-22 15:40:31+00:00",
            "title": "Joint RNN-Based Greedy Parsing and Word Composition",
            "authors": [
                "Jo\u00ebl Legrand",
                "Ronan Collobert"
            ],
            "abstract": "This paper introduces a greedy parser based on neural networks, which\nleverages a new compositional sub-tree representation. The greedy parser and\nthe compositional procedure are jointly trained, and tightly depends on\neach-other. The composition procedure outputs a vector representation which\nsummarizes syntactically (parsing tags) and semantically (words) sub-trees.\nComposition and tagging is achieved over continuous (word or tag)\nrepresentations, and recurrent neural networks. We reach F1 performance on par\nwith well-known existing parsers, while having the advantage of speed, thanks\nto the greedy nature of the parser. We provide a fully functional\nimplementation of the method described in this paper.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.7063v5",
            "date": "2014-12-22 17:19:56+00:00",
            "title": "Diverse Embedding Neural Network Language Models",
            "authors": [
                "Kartik Audhkhasi",
                "Abhinav Sethy",
                "Bhuvana Ramabhadran"
            ],
            "abstract": "We propose Diverse Embedding Neural Network (DENN), a novel architecture for\nlanguage models (LMs). A DENNLM projects the input word history vector onto\nmultiple diverse low-dimensional sub-spaces instead of a single\nhigher-dimensional sub-space as in conventional feed-forward neural network\nLMs. We encourage these sub-spaces to be diverse during network training\nthrough an augmented loss function. Our language modeling experiments on the\nPenn Treebank data set show the performance benefit of using a DENNLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7091v3",
            "date": "2014-12-22 18:51:08+00:00",
            "title": "Efficient Exact Gradient Update for training Deep Networks with Very Large Sparse Targets",
            "authors": [
                "Pascal Vincent",
                "Alexandre de Br\u00e9bisson",
                "Xavier Bouthillier"
            ],
            "abstract": "An important class of problems involves training deep neural networks with\nsparse prediction targets of very high dimension D. These occur naturally in\ne.g. neural language models or the learning of word-embeddings, often posed as\npredicting the probability of next words among a vocabulary of size D (e.g. 200\n000). Computing the equally large, but typically non-sparse D-dimensional\noutput vector from a last hidden layer of reasonable dimension d (e.g. 500)\nincurs a prohibitive O(Dd) computational cost for each example, as does\nupdating the D x d output weight matrix and computing the gradient needed for\nbackpropagation to previous layers. While efficient handling of large sparse\nnetwork inputs is trivial, the case of large sparse targets is not, and has\nthus so far been sidestepped with approximate alternatives such as hierarchical\nsoftmax or sampling-based approximations during training. In this work we\ndevelop an original algorithmic approach which, for a family of loss functions\nthat includes squared error and spherical softmax, can compute the exact loss,\ngradient update for the output weights, and gradient for backpropagation, all\nin O(d^2) per example instead of O(Dd), remarkably without ever computing the\nD-dimensional output. The proposed algorithm yields a speedup of D/4d , i.e.\ntwo orders of magnitude for typical sizes, for that critical part of the\ncomputations that often dominates the training time in this kind of network\narchitecture.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1412.7110v6",
            "date": "2014-12-22 19:46:01+00:00",
            "title": "Learning linearly separable features for speech recognition using convolutional neural networks",
            "authors": [
                "Dimitri Palaz",
                "Mathew Magimai Doss",
                "Ronan Collobert"
            ],
            "abstract": "Automatic speech recognition systems usually rely on spectral-based features,\nsuch as MFCC of PLP. These features are extracted based on prior knowledge such\nas, speech perception or/and speech production. Recently, convolutional neural\nnetworks have been shown to be able to estimate phoneme conditional\nprobabilities in a completely data-driven manner, i.e. using directly temporal\nraw speech signal as input. This system was shown to yield similar or better\nperformance than HMM/ANN based system on phoneme recognition task and on large\nscale continuous speech recognition task, using less parameters. Motivated by\nthese studies, we investigate the use of simple linear classifier in the\nCNN-based framework. Thus, the network learns linearly separable features from\nraw speech. We show that such system yields similar or better performance than\nMLP based system using cepstral-based features as input.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1412.7186v2",
            "date": "2014-12-22 22:05:06+00:00",
            "title": "Reply to the commentary \"Be careful when assuming the obvious\", by P. Alday",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Here we respond to some comments by Alday concerning headedness in linguistic\ntheory and the validity of the assumptions of a mathematical model for word\norder. For brevity, we focus only on two assumptions: the unit of measurement\nof dependency length and the monotonicity of the cost of a dependency as a\nfunction of its length. We also revise the implicit psychological bias in\nAlday's comments. Notwithstanding, Alday is indicating the path for linguistic\nresearch with his unusual concerns about parsimony from multiple dimensions.",
            "journal ref": "Language Dynamics and Change 5 (1), 147-155 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.7449v3",
            "date": "2014-12-23 17:16:24+00:00",
            "title": "Grammar as a Foreign Language",
            "authors": [
                "Oriol Vinyals",
                "Lukasz Kaiser",
                "Terry Koo",
                "Slav Petrov",
                "Ilya Sutskever",
                "Geoffrey Hinton"
            ],
            "abstract": "Syntactic constituency parsing is a fundamental problem in natural language\nprocessing and has been the subject of intensive research and engineering for\ndecades. As a result, the most accurate parsers are domain specific, complex,\nand inefficient. In this paper we show that the domain agnostic\nattention-enhanced sequence-to-sequence model achieves state-of-the-art results\non the most widely used syntactic constituency parsing dataset, when trained on\na large synthetic corpus that was annotated using existing parsers. It also\nmatches the performance of standard parsers when trained only on a small\nhuman-annotated dataset, which shows that this model is highly data-efficient,\nin contrast to sequence-to-sequence models without the attention mechanism. Our\nparser is also fast, processing over a hundred sentences per second with an\nunoptimized CPU implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.8102v1",
            "date": "2014-12-28 03:54:16+00:00",
            "title": "Proceedings of the 11th workshop on Quantum Physics and Logic",
            "authors": [
                "Bob Coecke",
                "Ichiro Hasuo",
                "Prakash Panangaden"
            ],
            "abstract": "This volume contains the proceedings of the 11th International Workshop on\nQuantum Physics and Logic (QPL 2014), which was held from the 4th to the 6th of\nJune, 2014, at Kyoto University, Japan.\n  The goal of the QPL workshop series is to bring together researchers working\non mathematical foundations of quantum physics, quantum computing and\nspatio-temporal causal structures, and in particular those that use logical\ntools, ordered algebraic and category-theoretic structures, formal languages,\nsemantic methods and other computer science methods for the study of physical\nbehavior in general. Over the past few years, there has been growing activity\nin these foundational approaches, together with a renewed interest in the\nfoundations of quantum theory, which complement the more mainstream research in\nquantum computation. Earlier workshops in this series, with the same acronym\nunder the name \"Quantum Programming Languages\", were held in Ottawa (2003),\nTurku (2004), Chicago (2005), and Oxford (2006). The first QPL under the new\nname Quantum Physics and Logic was held in Reykjavik (2008), followed by Oxford\n(2009 and 2010), Nijmegen (2011), Brussels (2012) and Barcelona (2013).",
            "journal ref": "EPTCS 172, 2014",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1412.8419v3",
            "date": "2014-12-29 18:43:10+00:00",
            "title": "Simple Image Description Generator via a Linear Phrase-Based Approach",
            "authors": [
                "Remi Lebret",
                "Pedro O. Pinheiro",
                "Ronan Collobert"
            ],
            "abstract": "Generating a novel textual description of an image is an interesting problem\nthat connects computer vision and natural language processing. In this paper,\nwe present a simple model that is able to generate descriptive sentences given\na sample image. This model has a strong focus on the syntax of the\ndescriptions. We train a purely bilinear model that learns a metric between an\nimage representation (generated from a previously trained Convolutional Neural\nNetwork) and phrases that are used to described them. The system is then able\nto infer phrases from a given image sample. Based on caption syntax statistics,\nwe propose a simple language model that can produce relevant descriptions for a\ngiven test image using the phrases inferred. Our approach, which is\nconsiderably simpler than state-of-the-art models, achieves comparable results\non the recently release Microsoft COCO dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1412.8527v1",
            "date": "2014-12-30 01:43:39+00:00",
            "title": "From Logical to Distributional Models",
            "authors": [
                "Anne Preller"
            ],
            "abstract": "The paper relates two variants of semantic models for natural language,\nlogical functional models and compositional distributional vector space models,\nby transferring the logic and reasoning from the logical to the distributional\nmodels.\n  The geometrical operations of quantum logic are reformulated as algebraic\noperations on vectors. A map from functional models to vector space models\nmakes it possible to compare the meaning of sentences word by word.",
            "journal ref": "EPTCS 171, 2014, pp. 113-131",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1402.0574v1",
            "date": "2014-02-04 01:39:12+00:00",
            "title": "Learning to Predict from Textual Data",
            "authors": [
                "Kira Radinsky",
                "Sagie Davidovich",
                "Shaul Markovitch"
            ],
            "abstract": "Given a current news event, we tackle the problem of generating plausible\npredictions of future events it might cause. We present a new methodology for\nmodeling and predicting such future news events using machine learning and data\nmining techniques. Our Pundit algorithm generalizes examples of causality pairs\nto infer a causality predictor. To obtain precisely labeled causality examples,\nwe mine 150 years of news articles and apply semantic natural language modeling\ntechniques to headlines containing certain predefined causality patterns. For\ngeneralization, the model uses a vast number of world knowledge ontologies.\nEmpirical evaluation on real news articles shows that our Pundit algorithm\nperforms as well as non-expert humans.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 45, pages\n  641-684, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.1128v1",
            "date": "2014-02-05 19:01:51+00:00",
            "title": "Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition",
            "authors": [
                "Ha\u015fim Sak",
                "Andrew Senior",
                "Fran\u00e7oise Beaufays"
            ],
            "abstract": "Long Short-Term Memory (LSTM) is a recurrent neural network (RNN)\narchitecture that has been designed to address the vanishing and exploding\ngradient problems of conventional RNNs. Unlike feedforward neural networks,\nRNNs have cyclic connections making them powerful for modeling sequences. They\nhave been successfully used for sequence labeling and sequence prediction\ntasks, such as handwriting recognition, language modeling, phonetic labeling of\nacoustic frames. However, in contrast to the deep neural networks, the use of\nRNNs in speech recognition has been limited to phone recognition in small scale\ntasks. In this paper, we present novel LSTM based RNN architectures which make\nmore effective use of model parameters to train acoustic models for large\nvocabulary speech recognition. We train and compare LSTM, RNN and DNN models at\nvarious numbers of parameters and configurations. We show that LSTM models\nconverge quickly and give state of the art speech recognition performance for\nrelatively small sized models.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1402.1454v1",
            "date": "2014-02-06 18:53:30+00:00",
            "title": "An Autoencoder Approach to Learning Bilingual Word Representations",
            "authors": [
                "Sarath Chandar A P",
                "Stanislas Lauly",
                "Hugo Larochelle",
                "Mitesh M. Khapra",
                "Balaraman Ravindran",
                "Vikas Raykar",
                "Amrita Saha"
            ],
            "abstract": "Cross-language learning allows us to use training data from one language to\nbuild models for a different language. Many approaches to bilingual learning\nrequire that we have word-level alignment of sentences from parallel corpora.\nIn this work we explore the use of autoencoder-based methods for cross-language\nlearning of vectorial word representations that are aligned between two\nlanguages, while not relying on word-level alignments. We show that by simply\nlearning to reconstruct the bag-of-words representations of aligned sentences,\nwithin and between languages, we can in fact learn high-quality representations\nand do without word alignments. Since training autoencoders on word\nobservations presents certain computational issues, we propose and compare\ndifferent variations adapted to this setting. We also propose an explicit\ncorrelation maximizing regularizer that leads to significant improvement in the\nperformance. We empirically investigate the success of our approach on the\nproblem of cross-language test classification, where a classifier trained on a\ngiven language (e.g., English) must learn to generalize to a different language\n(e.g., German). These experiments demonstrate that our approaches are\ncompetitive with the state-of-the-art, achieving up to 10-14 percentage point\nimprovements over the best reported results on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.2427v1",
            "date": "2014-02-11 10:27:43+00:00",
            "title": "An evaluation of keyword extraction from online communication for the characterisation of social relations",
            "authors": [
                "Jan Hauffa",
                "Tobias Lichtenberg",
                "Georg Groh"
            ],
            "abstract": "The set of interpersonal relationships on a social network service or a\nsimilar online community is usually highly heterogenous. The concept of tie\nstrength captures only one aspect of this heterogeneity. Since the unstructured\ntext content of online communication artefacts is a salient source of\ninformation about a social relationship, we investigate the utility of keywords\nextracted from the message body as a representation of the relationship's\ncharacteristics as reflected by the conversation topics. Keyword extraction is\nperformed using standard natural language processing methods. Communication\ndata and human assessments of the extracted keywords are obtained from Facebook\nusers via a custom application. The overall positive quality assessment\nprovides evidence that the keywords indeed convey relevant information about\nthe relationship.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1402.3405v1",
            "date": "2014-02-14 09:25:59+00:00",
            "title": "Authorship Analysis based on Data Compression",
            "authors": [
                "Daniele Cerra",
                "Mihai Datcu",
                "Peter Reinartz"
            ],
            "abstract": "This paper proposes to perform authorship analysis using the Fast Compression\nDistance (FCD), a similarity measure based on compression with dictionaries\ndirectly extracted from the written texts. The FCD computes a similarity\nbetween two documents through an effective binary search on the intersection\nset between the two related dictionaries. In the reported experiments the\nproposed method is applied to documents which are heterogeneous in style,\nwritten in five different languages and coming from different historical\nperiods. Results are comparable to the state of the art and outperform\ntraditional compression-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3722v1",
            "date": "2014-02-15 21:03:02+00:00",
            "title": "word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method",
            "authors": [
                "Yoav Goldberg",
                "Omer Levy"
            ],
            "abstract": "The word2vec software of Tomas Mikolov and colleagues\n(https://code.google.com/p/word2vec/ ) has gained a lot of traction lately, and\nprovides state-of-the-art word embeddings. The learning models behind the\nsoftware are described in two research papers. We found the description of the\nmodels in these papers to be somewhat cryptic and hard to follow. While the\nmotivations and presentation may be obvious to the neural-networks\nlanguage-modeling crowd, we had to struggle quite a bit to figure out the\nrationale behind the equations.\n  This note is an attempt to explain equation (4) (negative sampling) in\n\"Distributed Representations of Words and Phrases and their Compositionality\"\nby Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1402.3891v1",
            "date": "2014-02-17 05:24:42+00:00",
            "title": "Performance Evaluation of Machine Learning Classifiers in Sentiment Mining",
            "authors": [
                "Vinodhini G Chandrasekaran RM"
            ],
            "abstract": "In recent years, the use of machine learning classifiers is of great value in\nsolving a variety of problems in text classification. Sentiment mining is a\nkind of text classification in which, messages are classified according to\nsentiment orientation such as positive or negative. This paper extends the idea\nof evaluating the performance of various classifiers to show their\neffectiveness in sentiment mining of online product reviews. The product\nreviews are collected from Amazon reviews. To evaluate the performance of\nclassifiers various evaluation methods like random sampling, linear sampling\nand bootstrap sampling are used. Our results shows that support vector machine\nwith bootstrap sampling method outperforms others classifiers and sampling\nmethods in terms of misclassification rate.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1402.4802v2",
            "date": "2014-02-18 09:26:17+00:00",
            "title": "Ambiguity in language networks",
            "authors": [
                "Ricard V. Sol\u00e9",
                "Lu\u00eds F. Seoane"
            ],
            "abstract": "Human language defines the most complex outcomes of evolution. The emergence\nof such an elaborated form of communication allowed humans to create extremely\nstructured societies and manage symbols at different levels including, among\nothers, semantics. All linguistic levels have to deal with an astronomic\ncombinatorial potential that stems from the recursive nature of languages. This\nrecursiveness is indeed a key defining trait. However, not all words are\nequally combined nor frequent. In breaking the symmetry between less and more\noften used and between less and more meaning-bearing units, universal scaling\nlaws arise. Such laws, common to all human languages, appear on different\nstages from word inventories to networks of interacting words. Among these\nseemingly universal traits exhibited by language networks, ambiguity appears to\nbe a specially relevant component. Ambiguity is avoided in most computational\napproaches to language processing, and yet it seems to be a crucial element of\nlanguage architecture. Here we review the evidence both from language network\narchitecture and from theoretical reasonings based on a least effort argument.\nAmbiguity is shown to play an essential role in providing a source of language\nefficiency, and is likely to be an inevitable byproduct of network growth.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1402.6010v3",
            "date": "2014-02-24 22:58:28+00:00",
            "title": "Tripartite Graph Clustering for Dynamic Sentiment Analysis on Social Media",
            "authors": [
                "Linhong Zhu",
                "Aram Galstyan",
                "James Cheng",
                "Kristina Lerman"
            ],
            "abstract": "The growing popularity of social media (e.g, Twitter) allows users to easily\nshare information with each other and influence others by expressing their own\nsentiments on various subjects. In this work, we propose an unsupervised\n\\emph{tri-clustering} framework, which analyzes both user-level and tweet-level\nsentiments through co-clustering of a tripartite graph. A compelling feature of\nthe proposed framework is that the quality of sentiment clustering of tweets,\nusers, and features can be mutually improved by joint clustering. We further\ninvestigate the evolution of user-level sentiments and latent feature vectors\nin an online framework and devise an efficient online algorithm to sequentially\nupdate the clustering of tweets, users and features with newly arrived data.\nThe online framework not only provides better quality of both dynamic\nuser-level and tweet-level sentiment analysis, but also improves the\ncomputational and storage efficiency. We verified the effectiveness and\nefficiency of the proposed approaches on the November 2012 California ballot\nTwitter data.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1402.6238v1",
            "date": "2014-02-25 16:52:05+00:00",
            "title": "Improving Collaborative Filtering based Recommenders using Topic Modelling",
            "authors": [
                "Jobin Wilson",
                "Santanu Chaudhury",
                "Brejesh Lall",
                "Prateek Kapadia"
            ],
            "abstract": "Standard Collaborative Filtering (CF) algorithms make use of interactions\nbetween users and items in the form of implicit or explicit ratings alone for\ngenerating recommendations. Similarity among users or items is calculated\npurely based on rating overlap in this case,without considering explicit\nproperties of users or items involved, limiting their applicability in domains\nwith very sparse rating spaces. In many domains such as movies, news or\nelectronic commerce recommenders, considerable contextual data in text form\ndescribing item properties is available along with the rating data, which could\nbe utilized to improve recommendation quality.In this paper, we propose a novel\napproach to improve standard CF based recommenders by utilizing latent\nDirichlet allocation (LDA) to learn latent properties of items, expressed in\nterms of topic proportions, derived from their textual description. We infer\nuser's topic preferences or persona in the same latent space,based on her\nhistorical ratings. While computing similarity between users, we make use of a\ncombined similarity measure involving rating overlap as well as similarity in\nthe latent topic space. This approach alleviates sparsity problem as it allows\ncalculation of similarity between users even if they have not rated any items\nin common. Our experiments on multiple public datasets indicate that the\nproposed hybrid approach significantly outperforms standard user Based and item\nBased CF recommenders in terms of classification accuracy metrics such as\nprecision, recall and f-measure.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1402.6690v1",
            "date": "2014-02-26 20:58:00+00:00",
            "title": "Why Are You More Engaged? Predicting Social Engagement from Word Use",
            "authors": [
                "Jalal Mahmud",
                "Jilin Chen",
                "Jeffrey Nichols"
            ],
            "abstract": "We present a study to analyze how word use can predict social engagement\nbehaviors such as replies and retweets in Twitter. We compute psycholinguistic\ncategory scores from word usage, and investigate how people with different\nscores exhibited different reply and retweet behaviors on Twitter. We also\nfound psycholinguistic categories that show significant correlations with such\nsocial engagement behaviors. In addition, we have built predictive models of\nreplies and retweets from such psycholinguistic category based features. Our\nexperiments using a real world dataset collected from Twitter validates that\nsuch predictions can be done with reasonable accuracy.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1402.6792v1",
            "date": "2014-02-27 05:16:27+00:00",
            "title": "Information Evolution in Social Networks",
            "authors": [
                "Lada A. Adamic",
                "Thomas M. Lento",
                "Eytan Adar",
                "Pauline C. Ng"
            ],
            "abstract": "Social networks readily transmit information, albeit with less than perfect\nfidelity. We present a large-scale measurement of this imperfect information\ncopying mechanism by examining the dissemination and evolution of thousands of\nmemes, collectively replicated hundreds of millions of times in the online\nsocial network Facebook. The information undergoes an evolutionary process that\nexhibits several regularities. A meme's mutation rate characterizes the\npopulation distribution of its variants, in accordance with the Yule process.\nVariants further apart in the diffusion cascade have greater edit distance, as\nwould be expected in an iterative, imperfect replication process. Some text\nsequences can confer a replicative advantage; these sequences are abundant and\ntransfer \"laterally\" between different memes. Subpopulations of the social\nnetwork can preferentially transmit a specific variant of a meme if the variant\nmatches their beliefs or culture. Understanding the mechanism driving change in\ndiffusing information has important implications for how we interpret and\nharness the information that reaches us through our social networks.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1403.0541v1",
            "date": "2014-03-03 19:45:41+00:00",
            "title": "Representing, reasoning and answering questions about biological pathways - various applications",
            "authors": [
                "Saadat Anwar"
            ],
            "abstract": "Biological organisms are composed of numerous interconnected biochemical\nprocesses. Diseases occur when normal functionality of these processes is\ndisrupted. Thus, understanding these biochemical processes and their\ninterrelationships is a primary task in biomedical research and a prerequisite\nfor diagnosing diseases, and drug development. Scientists studying these\nprocesses have identified various pathways responsible for drug metabolism, and\nsignal transduction, etc.\n  Newer techniques and speed improvements have resulted in deeper knowledge\nabout these pathways, resulting in refined models that tend to be large and\ncomplex, making it difficult for a person to remember all aspects of it. Thus,\ncomputer models are needed to analyze them. We want to build such a system that\nallows modeling of biological systems and pathways in such a way that we can\nanswer questions about them.\n  Many existing models focus on structural and/or factoid questions, using\nsurface-level knowledge that does not require understanding the underlying\nmodel. We believe these are not the kind of questions that a biologist may ask\nsomeone to test their understanding of the biological processes. We want our\nsystem to answer the kind of questions a biologist may ask. Such questions\nappear in early college level text books.\n  Thus the main goal of our thesis is to develop a system that allows us to\nencode knowledge about biological pathways and answer such questions about them\ndemonstrating understanding of the pathway. To that end, we develop a language\nthat will allow posing such questions and illustrate the utility of our\nframework with various applications in the biological domain. We use some\nexisting tools with modifications to accomplish our goal.\n  Finally, we apply our system to real world applications by extracting pathway\nknowledge from text and answering questions related to drug development.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1403.1252v2",
            "date": "2014-03-06 01:36:53+00:00",
            "title": "Inducing Language Networks from Continuous Space Word Representations",
            "authors": [
                "Bryan Perozzi",
                "Rami Al-Rfou",
                "Vivek Kulkarni",
                "Steven Skiena"
            ],
            "abstract": "Recent advancements in unsupervised feature learning have developed powerful\nlatent representations of words. However, it is still not clear what makes one\nrepresentation better than another and how we can learn the ideal\nrepresentation. Understanding the structure of latent spaces attained is key to\nany future advancement in unsupervised learning. In this work, we introduce a\nnew view of continuous space word representations as language networks. We\nexplore two techniques to create language networks from learned features by\ninducing them for two popular word representation methods and examining the\nproperties of their resulting networks. We find that the induced networks\ndiffer from other methods of creating language networks, and that they contain\nmeaningful community structure.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1403.1310v1",
            "date": "2014-03-06 01:16:01+00:00",
            "title": "AntiPlag: Plagiarism Detection on Electronic Submissions of Text Based Assignments",
            "authors": [
                "M. A. C. Jiffriya",
                "M. A. C. Akmal Jahan",
                "R. G. Ragel",
                "S. Deegalla"
            ],
            "abstract": "Plagiarism is one of the growing issues in academia and is always a concern\nin Universities and other academic institutions. The situation is becoming even\nworse with the availability of ample resources on the web. This paper focuses\non creating an effective and fast tool for plagiarism detection for text based\nelectronic assignments. Our plagiarism detection tool named AntiPlag is\ndeveloped using the tri-gram sequence matching technique. Three sets of text\nbased assignments were tested by AntiPlag and the results were compared against\nan existing commercial plagiarism detection tool. AntiPlag showed better\nresults in terms of false positives compared to the commercial tool due to the\npre-processing steps performed in AntiPlag. In addition, to improve the\ndetection latency, AntiPlag applies a data clustering technique making it four\ntimes faster than the commercial tool considered. AntiPlag could be used to\nisolate plagiarized text based assignments from non-plagiarised assignments\neasily. Therefore, we present AntiPlag, a fast and effective tool for\nplagiarism detection on text based electronic assignments.",
            "journal ref": "Industrial and Information Systems (ICIIS), 2013 8th IEEE\n  International Conference on, pp. 376 - 380, 17-20 Dec. 2013",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1403.1349v2",
            "date": "2014-03-06 05:24:02+00:00",
            "title": "Learning Soft Linear Constraints with Application to Citation Field Extraction",
            "authors": [
                "Sam Anzaroot",
                "Alexandre Passos",
                "David Belanger",
                "Andrew McCallum"
            ],
            "abstract": "Accurately segmenting a citation string into fields for authors, titles, etc.\nis a challenging task because the output typically obeys various global\nconstraints. Previous work has shown that modeling soft constraints, where the\nmodel is encouraged, but not require to obey the constraints, can substantially\nimprove segmentation performance. On the other hand, for imposing hard\nconstraints, dual decomposition is a popular technique for efficient prediction\ngiven existing algorithms for unconstrained inference. We extend the technique\nto perform prediction subject to soft constraints. Moreover, with a technique\nfor performing inference given soft constraints, it is easy to automatically\ngenerate large families of constraints and learn their costs with a simple\nconvex optimization problem during training. This allows us to obtain\nsubstantial gains in accuracy on a new, challenging citation extraction\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1403.1451v1",
            "date": "2014-03-06 14:23:53+00:00",
            "title": "Real-Time Classification of Twitter Trends",
            "authors": [
                "Arkaitz Zubiaga",
                "Damiano Spina",
                "Raquel Mart\u00ednez",
                "V\u00edctor Fresno"
            ],
            "abstract": "Social media users give rise to social trends as they share about common\ninterests, which can be triggered by different reasons. In this work, we\nexplore the types of triggers that spark trends on Twitter, introducing a\ntypology with following four types: 'news', 'ongoing events', 'memes', and\n'commemoratives'. While previous research has analyzed trending topics in a\nlong term, we look at the earliest tweets that produce a trend, with the aim of\ncategorizing trends early on. This would allow to provide a filtered subset of\ntrends to end users. We analyze and experiment with a set of straightforward\nlanguage-independent features based on the social spread of trends to\ncategorize them into the introduced typology. Our method provides an efficient\nway to accurately categorize trending topics without need of external data,\nenabling news organizations to discover breaking news in real-time, or to\nquickly identify viral memes that might enrich marketing decisions, among\nothers. The analysis of social features also reveals patterns associated with\neach type of trend, such as tweets about ongoing events being shorter as many\nwere likely sent from mobile devices, or memes having more retweets originating\nfrom a few trend-setters.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1403.2345v1",
            "date": "2014-03-07 05:29:42+00:00",
            "title": "Home Location Identification of Twitter Users",
            "authors": [
                "Jalal Mahmud",
                "Jeffrey Nichols",
                "Clemens Drews"
            ],
            "abstract": "We present a new algorithm for inferring the home location of Twitter users\nat different granularities, including city, state, time zone or geographic\nregion, using the content of users tweets and their tweeting behavior. Unlike\nexisting approaches, our algorithm uses an ensemble of statistical and\nheuristic classifiers to predict locations and makes use of a geographic\ngazetteer dictionary to identify place-name entities. We find that a\nhierarchical classification approach, where time zone, state or geographic\nregion is predicted first and city is predicted next, can improve prediction\naccuracy. We have also analyzed movement variations of Twitter users, built a\nclassifier to predict whether a user was travelling in a certain period of time\nand use that to further improve the location detection accuracy. Experimental\nevidence suggests that our algorithm works well in practice and outperforms the\nbest existing algorithms for predicting the home location of Twitter users.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1403.3460v1",
            "date": "2014-03-13 23:22:21+00:00",
            "title": "Scalable and Robust Construction of Topical Hierarchies",
            "authors": [
                "Chi Wang",
                "Xueqing Liu",
                "Yanglei Song",
                "Jiawei Han"
            ],
            "abstract": "Automated generation of high-quality topical hierarchies for a text\ncollection is a dream problem in knowledge engineering with many valuable\napplications. In this paper a scalable and robust algorithm is proposed for\nconstructing a hierarchy of topics from a text collection. We divide and\nconquer the problem using a top-down recursive framework, based on a tensor\northogonal decomposition technique. We solve a critical challenge to perform\nscalable inference for our newly designed hierarchical topic model. Experiments\nwith various real-world datasets illustrate its ability to generate robust,\nhigh-quality hierarchies efficiently. Our method reduces the time of\nconstruction by several orders of magnitude, and its robust feature renders it\npossible for users to interactively revise the hierarchy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1403.6397v1",
            "date": "2014-03-25 15:44:14+00:00",
            "title": "Evaluating topic coherence measures",
            "authors": [
                "Frank Rosner",
                "Alexander Hinneburg",
                "Michael R\u00f6der",
                "Martin Nettling",
                "Andreas Both"
            ],
            "abstract": "Topic models extract representative word sets - called topics - from word\ncounts in documents without requiring any semantic annotations. Topics are not\nguaranteed to be well interpretable, therefore, coherence measures have been\nproposed to distinguish between good and bad topics. Studies of topic coherence\nso far are limited to measures that score pairs of individual words. For the\nfirst time, we include coherence measures from scientific philosophy that score\npairs of more complex word subsets and apply them to topic scoring.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1403.7335v1",
            "date": "2014-03-28 10:45:31+00:00",
            "title": "Emotion Analysis Platform on Chinese Microblog",
            "authors": [
                "Duyu Tang",
                "Bing Qin",
                "Ting Liu",
                "Qiuhui Shi"
            ],
            "abstract": "Weibo, as the largest social media service in China, has billions of messages\ngenerated every day. The huge number of messages contain rich sentimental\ninformation. In order to analyze the emotional changes in accordance with time\nand space, this paper presents an Emotion Analysis Platform (EAP), which\nexplores the emotional distribution of each province, so that can monitor the\nglobal pulse of each province in China. The massive data of Weibo and the\nreal-time requirements make the building of EAP challenging. In order to solve\nthe above problems, emoticons, emotion lexicon and emotion-shifting rules are\nadopted in EAP to analyze the emotion of each tweet. In order to verify the\neffectiveness of the platform, case study on the Sichuan earthquake is done,\nand the analysis result of the platform accords with the fact. In order to\nanalyze from quantity, we manually annotate a test set and conduct experiment\non it. The experimental results show that the macro-Precision of EAP reaches\n80% and the EAP works effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.0546v2",
            "date": "2014-05-03 01:41:27+00:00",
            "title": "Kaggle LSHTC4 Winning Solution",
            "authors": [
                "Antti Puurula",
                "Jesse Read",
                "Albert Bifet"
            ],
            "abstract": "Our winning submission to the 2014 Kaggle competition for Large Scale\nHierarchical Text Classification (LSHTC) consists mostly of an ensemble of\nsparse generative models extending Multinomial Naive Bayes. The\nbase-classifiers consist of hierarchically smoothed models combining document,\nlabel, and hierarchy level Multinomials, with feature pre-processing using\nvariants of TF-IDF and BM25. Additional diversification is introduced by\ndifferent types of folds and random search optimization for different measures.\nThe ensemble algorithm optimizes macroFscore by predicting the documents for\neach label, instead of the usual prediction of labels per document. Scores for\ndocuments are predicted by weighted voting of base-classifier outputs with a\nvariant of Feature-Weighted Linear Stacking. The number of documents per label\nis chosen using label priors and thresholding of vote scores. This document\ndescribes the models and software used to build our solution. Reproducing the\nresults for our solution can be done by running the scripts included in the\nKaggle package. A package omitting precomputed result files is also\ndistributed. All code is open source, released under GNU GPL 2.0, and GPL 3.0\nfor Weka and Meka dependencies.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1405.0616v1",
            "date": "2014-05-03 19:02:44+00:00",
            "title": "Automated Attribution and Intertextual Analysis",
            "authors": [
                "James Brofos",
                "Ajay Kannan",
                "Rui Shu"
            ],
            "abstract": "In this work, we employ quantitative methods from the realm of statistics and\nmachine learning to develop novel methodologies for author attribution and\ntextual analysis. In particular, we develop techniques and software suitable\nfor applications to Classical study, and we illustrate the efficacy of our\napproach in several interesting open questions in the field. We apply our\nnumerical analysis techniques to questions of authorship attribution in the\ncase of the Greek tragedian Euripides, to instances of intertextuality and\ninfluence in the poetry of the Roman statesman Seneca the Younger, and to cases\nof \"interpolated\" text with respect to the histories of Livy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.1438v1",
            "date": "2014-05-06 20:04:44+00:00",
            "title": "The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter",
            "authors": [
                "Chenhao Tan",
                "Lillian Lee",
                "Bo Pang"
            ],
            "abstract": "Consider a person trying to spread an important message on a social network.\nHe/she can spend hours trying to craft the message. Does it actually matter?\nWhile there has been extensive prior work looking into predicting popularity of\nsocial-media content, the effect of wording per se has rarely been studied\nsince it is often confounded with the popularity of the author and the topic.\nTo control for these confounding factors, we take advantage of the surprising\nfact that there are many pairs of tweets containing the same url and written by\nthe same user but employing different wording. Given such pairs, we ask: which\nversion attracts more retweets? This turns out to be a more difficult task than\npredicting popular topics. Still, humans can answer this question better than\nchance (but far from perfectly), and the computational methods we develop can\ndo better than both an average human and a strong competing method trained on\nnon-controlled data.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1405.1893v2",
            "date": "2014-05-08 11:47:03+00:00",
            "title": "Initial Comparison of Linguistic Networks Measures for Parallel Texts",
            "authors": [
                "Kristina Ban",
                "Ana Me\u0161trovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "This paper presents preliminary results of Croatian syllable networks\nanalysis. Syllable network is a network in which nodes are syllables and links\nbetween them are constructed according to their connections within words. In\nthis paper we analyze networks of syllables generated from texts collected from\nthe Croatian Wikipedia and Blogs. As a main tool we use complex network\nanalysis methods which provide mechanisms that can reveal new patterns in a\nlanguage structure. We aim to show that syllable networks have much higher\nclustering coefficient in comparison to Erd\\\"os-Renyi random networks. The\nresults indicate that Croatian syllable networks exhibit certain properties of\na small world networks. Furthermore, we compared Croatian syllable networks\nwith Portuguese and Chinese syllable networks and we showed that they have\nsimilar properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.2386v1",
            "date": "2014-05-10 03:43:14+00:00",
            "title": "Predicting Central Topics in a Blog Corpus from a Networks Perspective",
            "authors": [
                "Srayan Datta"
            ],
            "abstract": "In today's content-centric Internet, blogs are becoming increasingly popular\nand important from a data analysis perspective. According to Wikipedia, there\nwere over 156 million public blogs on the Internet as of February 2011. Blogs\nare a reflection of our contemporary society. The contents of different blog\nposts are important from social, psychological, economical and political\nperspectives. Discovery of important topics in the blogosphere is an area which\nstill needs much exploring. We try to come up with a procedure using\nprobabilistic topic modeling and network centrality measures which identifies\nthe central topics in a blog corpus.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1405.2702v2",
            "date": "2014-05-12 10:46:06+00:00",
            "title": "Comparison of the language networks from literature and blogs",
            "authors": [
                "Sabina \u0160i\u0161ovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107",
                "Ana Me\u0161trovi\u0107"
            ],
            "abstract": "In this paper we present the comparison of the linguistic networks from\nliterature and blog texts. The linguistic networks are constructed from texts\nas directed and weighted co-occurrence networks of words. Words are nodes and\nlinks are established between two nodes if they are directly co-occurring\nwithin the sentence. The comparison of the networks structure is performed at\nglobal level (network) in terms of: average node degree, average shortest path\nlength, diameter, clustering coefficient, density and number of components.\nFurthermore, we perform analysis on the local level (node) by comparing the\nrank plots of in and out degree, strength and selectivity. The\nselectivity-based results point out that there are differences between the\nstructure of the networks constructed from literature and blogs.",
            "journal ref": "37th IEEE International Convention on Information and\n  Communication Technology, Electronics and Microelectronics (MIPRO 2014),\n  pp.1824--1829, (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.2874v2",
            "date": "2014-05-12 18:48:54+00:00",
            "title": "A Study of Entanglement in a Categorical Framework of Natural Language",
            "authors": [
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "In both quantum mechanics and corpus linguistics based on vector spaces, the\nnotion of entanglement provides a means for the various subsystems to\ncommunicate with each other. In this paper we examine a number of\nimplementations of the categorical framework of Coecke, Sadrzadeh and Clark\n(2010) for natural language, from an entanglement perspective. Specifically,\nour goal is to better understand in what way the level of entanglement of the\nrelational tensors (or the lack of it) affects the compositional structures in\npractical situations. Our findings reveal that a number of proposals for verb\nconstruction lead to almost separable tensors, a fact that considerably\nsimplifies the interactions between the words. We examine the ramifications of\nthis fact, and we show that the use of Frobenius algebras mitigates the\npotential problems to a great extent. Finally, we briefly examine a machine\nlearning method that creates verb tensors exhibiting a sufficient level of\nentanglement.",
            "journal ref": "EPTCS 172, 2014, pp. 249-261",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.3282v1",
            "date": "2014-05-13 20:00:07+00:00",
            "title": "How to Ask for a Favor: A Case Study on the Success of Altruistic Requests",
            "authors": [
                "Tim Althoff",
                "Cristian Danescu-Niculescu-Mizil",
                "Dan Jurafsky"
            ],
            "abstract": "Requests are at the core of many social media systems such as question &\nanswer sites and online philanthropy communities. While the success of such\nrequests is critical to the success of the community, the factors that lead\ncommunity members to satisfy a request are largely unknown. Success of a\nrequest depends on factors like who is asking, how they are asking, when are\nthey asking, and most critically what is being requested, ranging from small\nfavors to substantial monetary donations. We present a case study of altruistic\nrequests in an online community where all requests ask for the very same\ncontribution and do not offer anything tangible in return, allowing us to\ndisentangle what is requested from textual and social factors. Drawing from\nsocial psychology literature, we extract high-level social features from text\nthat operationalize social relations between recipient and donor and\ndemonstrate that these extracted relations are predictive of success. More\nspecifically, we find that clearly communicating need through the narrative is\nessential and that that linguistic indications of gratitude, evidentiality, and\ngeneralized reciprocity, as well as high status of the asker further increase\nthe likelihood of success. Building on this understanding, we develop a model\nthat can predict the success of unseen requests, significantly improving over\nseveral baselines. We link these findings to research in psychology on helping\nbehavior, providing a basis for further analysis of success in social media\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4053v2",
            "date": "2014-05-16 07:12:16+00:00",
            "title": "Distributed Representations of Sentences and Documents",
            "authors": [
                "Quoc V. Le",
                "Tomas Mikolov"
            ],
            "abstract": "Many machine learning algorithms require the input to be represented as a\nfixed-length feature vector. When it comes to texts, one of the most common\nfixed-length features is bag-of-words. Despite their popularity, bag-of-words\nfeatures have two major weaknesses: they lose the ordering of the words and\nthey also ignore semantics of the words. For example, \"powerful,\" \"strong\" and\n\"Paris\" are equally distant. In this paper, we propose Paragraph Vector, an\nunsupervised algorithm that learns fixed-length feature representations from\nvariable-length pieces of texts, such as sentences, paragraphs, and documents.\nOur algorithm represents each document by a dense vector which is trained to\npredict words in the document. Its construction gives our algorithm the\npotential to overcome the weaknesses of bag-of-words models. Empirical results\nshow that Paragraph Vectors outperform bag-of-words models as well as other\ntechniques for text representations. Finally, we achieve new state-of-the-art\nresults on several text classification and sentiment analysis tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4433v1",
            "date": "2014-05-17 20:43:00+00:00",
            "title": "Preliminary Report on the Structure of Croatian Linguistic Co-occurrence Networks",
            "authors": [
                "Domagoj Margan",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107",
                "Ana Me\u0161trovi\u0107"
            ],
            "abstract": "In this article, we investigate the structure of Croatian linguistic\nco-occurrence networks. We examine the change of network structure properties\nby systematically varying the co-occurrence window sizes, the corpus sizes and\nremoving stopwords. In a co-occurrence window of size $n$ we establish a link\nbetween the current word and $n-1$ subsequent words. The results point out that\nthe increase of the co-occurrence window size is followed by a decrease in\ndiameter, average path shortening and expectedly condensing the average\nclustering coefficient. The same can be noticed for the removal of the\nstopwords. Finally, since the size of texts is reflected in the network\nproperties, our results suggest that the corpus influence can be reduced by\nincreasing the co-occurrence window size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.4599v1",
            "date": "2014-05-19 04:36:38+00:00",
            "title": "Modelling Data Dispersion Degree in Automatic Robust Estimation for Multivariate Gaussian Mixture Models with an Application to Noisy Speech Processing",
            "authors": [
                "Dalei Wu",
                "Haiqing Wu"
            ],
            "abstract": "The trimming scheme with a prefixed cutoff portion is known as a method of\nimproving the robustness of statistical models such as multivariate Gaussian\nmixture models (MG- MMs) in small scale tests by alleviating the impacts of\noutliers. However, when this method is applied to real- world data, such as\nnoisy speech processing, it is hard to know the optimal cut-off portion to\nremove the outliers and sometimes removes useful data samples as well. In this\npaper, we propose a new method based on measuring the dispersion degree (DD) of\nthe training data to avoid this problem, so as to realise automatic robust\nestimation for MGMMs. The DD model is studied by using two different measures.\nFor each one, we theoretically prove that the DD of the data samples in a\ncontext of MGMMs approximately obeys a specific (chi or chi-square)\ndistribution. The proposed method is evaluated on a real-world application with\na moderately-sized speaker recognition task. Experiments show that the proposed\nmethod can significantly improve the robustness of the conventional training\nmethod of GMMs for speaker recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.7713v1",
            "date": "2014-01-16 04:51:47+00:00",
            "title": "Using Local Alignments for Relation Recognition",
            "authors": [
                "Sophia Katrenko",
                "Pieter Adriaans",
                "Maarten van Someren"
            ],
            "abstract": "This paper discusses the problem of marrying structural similarity with\nsemantic relatedness for Information Extraction from text. Aiming at accurate\nrecognition of relations, we introduce local alignment kernels and explore\nvarious possibilities of using them for this task. We give a definition of a\nlocal alignment (LA) kernel based on the Smith-Waterman score as a sequence\nsimilarity measure and proceed with a range of possibilities for computing\nsimilarity between elements of sequences. We show how distributional similarity\nmeasures obtained from unlabeled data can be incorporated into the learning\ntask as semantic knowledge. Our experiments suggest that the LA kernel yields\npromising results on various biomedical corpora outperforming two baselines by\na large margin. Additional series of experiments have been conducted on the\ndata sets of seven general relation types, where the performance of the LA\nkernel is comparable to the current state-of-the-art results.",
            "journal ref": "Journal Of Artificial Intelligence Research, Volume 38, pages\n  1-48, 2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1405.7908v1",
            "date": "2014-05-30 16:36:07+00:00",
            "title": "Semantic Composition and Decomposition: From Recognition to Generation",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "Semantic composition is the task of understanding the meaning of text by\ncomposing the meanings of the individual words in the text. Semantic\ndecomposition is the task of understanding the meaning of an individual word by\ndecomposing it into various aspects (factors, constituents, components) that\nare latent in the meaning of the word. We take a distributional approach to\nsemantics, in which a word is represented by a context vector. Much recent work\nhas considered the problem of recognizing compositions and decompositions, but\nwe tackle the more difficult generation problem. For simplicity, we focus on\nnoun-modifier bigrams and noun unigrams. A test for semantic composition is,\ngiven context vectors for the noun and modifier in a noun-modifier bigram (\"red\nsalmon\"), generate a noun unigram that is synonymous with the given bigram\n(\"sockeye\"). A test for semantic decomposition is, given a context vector for a\nnoun unigram (\"snifter\"), generate a noun-modifier bigram that is synonymous\nwith the given unigram (\"brandy glass\"). With a vocabulary of about 73,000\nunigrams from WordNet, there are 73,000 candidate unigram compositions for a\nbigram and 5,300,000,000 (73,000 squared) candidate bigram decompositions for a\nunigram. We generate ranked lists of potential solutions in two passes. A fast\nunsupervised learning algorithm generates an initial list of candidates and\nthen a slower supervised learning algorithm refines the list. We evaluate the\ncandidate solutions by comparing them to WordNet synonym sets. For\ndecomposition (unigram to bigram), the top 100 most highly ranked bigrams\ninclude a WordNet synonym of the given unigram 50.7% of the time. For\ncomposition (bigram to unigram), the top 100 most highly ranked unigrams\ninclude a WordNet synonym of the given bigram 77.8% of the time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.0167v1",
            "date": "2014-07-01 09:56:42+00:00",
            "title": "Mathematical Language Processing Project",
            "authors": [
                "Robert Pagael",
                "Moritz Schubotz"
            ],
            "abstract": "In natural language, words and phrases themselves imply the semantics. In\ncontrast, the meaning of identifiers in mathematical formulae is undefined.\nThus scientists must study the context to decode the meaning. The Mathematical\nLanguage Processing (MLP) project aims to support that process. In this paper,\nwe compare two approaches to discover identifier-definition tuples. At first we\nuse a simple pattern matching approach. Second, we present the MLP approach\nthat uses part-of-speech tag based distances as well as sentence positions to\ncalculate identifier-definition probabilities. The evaluation of our\nprototypical system, applied on the Wikipedia text corpus, shows that our\napproach augments the user experience substantially. While hovering the\nidentifiers in the formula, tool-tips with the most probable definitions occur.\nTests with random samples show that the displayed definitions provide a good\nmatch with the actual meaning of the identifiers.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1407.4723v1",
            "date": "2014-07-17 16:12:04+00:00",
            "title": "Toward Selectivity Based Keyword Extraction for Croatian News",
            "authors": [
                "Slobodan Beliga",
                "Ana Me\u0161trovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "Preliminary report on network based keyword extraction for Croatian is an\nunsupervised method for keyword extraction from the complex network. We build\nour approach with a new network measure the node selectivity, motivated by the\nresearch of the graph based centrality approaches. The node selectivity is\ndefined as the average weight distribution on the links of the single node. We\nextract nodes (keyword candidates) based on the selectivity value. Furthermore,\nwe expand extracted nodes to word-tuples ranked with the highest in/out\nselectivity values. Selectivity based extraction does not require linguistic\nknowledge while it is purely derived from statistical and structural\ninformation en-compassed in the source text which is reflected into the\nstructure of the network. Obtained sets are evaluated on a manually annotated\nkeywords: for the set of extracted keyword candidates average F1 score is\n24,63%, and average F2 score is 21,19%; for the exacted words-tuples candidates\naverage F1 score is 25,9% and average F2 score is 24,47%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.6439v3",
            "date": "2014-07-24 03:34:41+00:00",
            "title": "Feature Engineering for Knowledge Base Construction",
            "authors": [
                "Christopher R\u00e9",
                "Amir Abbas Sadeghian",
                "Zifei Shan",
                "Jaeho Shin",
                "Feiran Wang",
                "Sen Wu",
                "Ce Zhang"
            ],
            "abstract": "Knowledge base construction (KBC) is the process of populating a knowledge\nbase, i.e., a relational database together with inference rules, with\ninformation extracted from documents and structured sources. KBC blurs the\ndistinction between two traditional database problems, information extraction\nand information integration. For the last several years, our group has been\nbuilding knowledge bases with scientific collaborators. Using our approach, we\nhave built knowledge bases that have comparable and sometimes better quality\nthan those constructed by human volunteers. In contrast to these knowledge\nbases, which took experts a decade or more human years to construct, many of\nour projects are constructed by a single graduate student.\n  Our approach to KBC is based on joint probabilistic inference and learning,\nbut we do not see inference as either a panacea or a magic bullet: inference is\na tool that allows us to be systematic in how we construct, debug, and improve\nthe quality of such systems. In addition, inference allows us to construct\nthese systems in a more loosely coupled way than traditional approaches. To\nsupport this idea, we have built the DeepDive system, which has the design goal\nof letting the user \"think about features---not algorithms.\" We think of\nDeepDive as declarative in that one specifies what they want but not how to get\nit. We describe our approach with a focus on feature engineering, which we\nargue is an understudied problem relative to its importance to end-to-end\nquality.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1407.6872v1",
            "date": "2014-07-25 12:46:18+00:00",
            "title": "Interpretable Low-Rank Document Representations with Label-Dependent Sparsity Patterns",
            "authors": [
                "Ivan Ivek"
            ],
            "abstract": "In context of document classification, where in a corpus of documents their\nlabel tags are readily known, an opportunity lies in utilizing label\ninformation to learn document representation spaces with better discriminative\nproperties. To this end, in this paper application of a Variational Bayesian\nSupervised Nonnegative Matrix Factorization (supervised vbNMF) with\nlabel-driven sparsity structure of coefficients is proposed for learning of\ndiscriminative nonsubtractive latent semantic components occuring in TF-IDF\ndocument representations. Constraints are such that the components pursued are\nmade to be frequently occuring in a small set of labels only, making it\npossible to yield document representations with distinctive label-specific\nsparse activation patterns. A simple measure of quality of this kind of\nsparsity structure, dubbed inter-label sparsity, is introduced and\nexperimentally brought into tight connection with classification performance.\nRepresenting a great practical convenience, inter-label sparsity is shown to be\neasily controlled in supervised vbNMF by a single parameter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.7094v1",
            "date": "2014-07-26 04:16:31+00:00",
            "title": "Crowdsourcing Dialect Characterization through Twitter",
            "authors": [
                "Bruno Gon\u00e7alves",
                "David S\u00e1nchez"
            ],
            "abstract": "We perform a large-scale analysis of language diatopic variation using\ngeotagged microblogging datasets. By collecting all Twitter messages written in\nSpanish over more than two years, we build a corpus from which a carefully\nselected list of concepts allows us to characterize Spanish varieties on a\nglobal scale. A cluster analysis proves the existence of well defined\nmacroregions sharing common lexical properties. Remarkably enough, we find that\nSpanish language is split into two superdialects, namely, an urban speech used\nacross major American and Spanish citites and a diverse form that encompasses\nrural areas and small towns. The latter can be further clustered into smaller\nvarieties with a stronger regional character.",
            "journal ref": "PLoS One 9, E112074 (2014)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1407.7169v1",
            "date": "2014-07-26 23:48:32+00:00",
            "title": "Principles and Parameters: a coding theory perspective",
            "authors": [
                "Matilde Marcolli"
            ],
            "abstract": "We propose an approach to Longobardi's parametric comparison method (PCM) via\nthe theory of error-correcting codes. One associates to a collection of\nlanguages to be analyzed with the PCM a binary (or ternary) code with one code\nwords for each language in the family and each word consisting of the binary\nvalues of the syntactic parameters of the language, with the ternary case\nallowing for an additional parameter state that takes into account phenomena of\nentailment of parameters. The code parameters of the resulting code can be\ncompared with some classical bounds in coding theory: the asymptotic bound, the\nGilbert-Varshamov bound, etc. The position of the code parameters with respect\nto some of these bounds provides quantitative information on the variability of\nsyntactic parameters within and across historical-linguistic families. While\ncomputations carried out for languages belonging to the same family yield codes\nbelow the GV curve, comparisons across different historical families can give\nexamples of isolated codes lying above the asymptotic bound.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1407.7736v1",
            "date": "2014-07-29 14:25:41+00:00",
            "title": "A Latent Space Analysis of Editor Lifecycles in Wikipedia",
            "authors": [
                "Xiangju Qin",
                "Derek Greene",
                "P\u00e1draig Cunningham"
            ],
            "abstract": "Collaborations such as Wikipedia are a key part of the value of the modern\nInternet. At the same time there is concern that these collaborations are\nthreatened by high levels of member turnover. In this paper we borrow ideas\nfrom topic analysis to editor activity on Wikipedia over time into a latent\nspace that offers an insight into the evolving patterns of editor behavior.\nThis latent space representation reveals a number of different categories of\neditor (e.g. content experts, social networkers) and we show that it does\nprovide a signal that predicts an editor's departure from the community. We\nalso show that long term editors gradually diversify their participation by\nshifting edit preference from one or two namespaces to multiple namespaces and\nexperience relatively soft evolution in their editor profiles, while short term\neditors generally distribute their contribution randomly among the namespaces\nand experience considerably fluctuated evolution in their editor profiles.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1407.8322v2",
            "date": "2014-07-31 09:02:15+00:00",
            "title": "Zipf's law for word frequencies: word forms versus lemmas in long texts",
            "authors": [
                "Alvaro Corral",
                "Gemma Boleda",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Zipf's law is a fundamental paradigm in the statistics of written and spoken\nnatural language as well as in other communication systems. We raise the\nquestion of the elementary units for which Zipf's law should hold in the most\nnatural way, studying its validity for plain word forms and for the\ncorresponding lemma forms. In order to have as homogeneous sources as possible,\nwe analyze some of the longest literary texts ever written, comprising four\ndifferent languages, with different levels of morphological complexity. In all\ncases Zipf's law is fulfilled, in the sense that a power-law distribution of\nword or lemma frequencies is valid for several orders of magnitude. We\ninvestigate the extent to which the word-lemma transformation preserves two\nparameters of Zipf's law: the exponent and the low-frequency cut-off. We are\nnot able to demonstrate a strict invariance of the tail, as for a few texts\nboth exponents deviate significantly, but we conclude that the exponents are\nvery similar, despite the remarkable transformation that going from words to\nlemmas represents, considerably affecting all ranges of frequencies. In\ncontrast, the low-frequency cut-offs are less stable.",
            "journal ref": "PLoS ONE 10 (7), e0129031",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1408.0985v1",
            "date": "2014-08-05 14:34:20+00:00",
            "title": "Speech earthquakes: scaling and universality in human voice",
            "authors": [
                "Jordi Luque",
                "Bartolo Luque",
                "Lucas Lacasa"
            ],
            "abstract": "Speech is a distinctive complex feature of human capabilities. In order to\nunderstand the physics underlying speech production, in this work we\nempirically analyse the statistics of large human speech datasets ranging\nseveral languages. We first show that during speech the energy is unevenly\nreleased and power-law distributed, reporting a universal robust\nGutenberg-Richter-like law in speech. We further show that such earthquakes in\nspeech show temporal correlations, as the interevent statistics are again\npower-law distributed. Since this feature takes place in the intra-phoneme\nrange, we conjecture that the responsible for this complex phenomenon is not\ncognitive, but it resides on the physiological speech production mechanism.\nMoreover, we show that these waiting time distributions are scale invariant\nunder a renormalisation group transformation, suggesting that the process of\nspeech generation is indeed operating close to a critical point. These results\nare put in contrast with current paradigms in speech processing, which point\ntowards low dimensional deterministic chaos as the origin of nonlinear traits\nin speech fluctuations. As these latter fluctuations are indeed the aspects\nthat humanize synthetic speech, these findings may have an impact in future\nspeech synthesis technologies. Results are robust and independent of the\ncommunication language or the number of speakers, pointing towards an universal\npattern and yet another hint of complexity in human speech.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1408.1774v1",
            "date": "2014-08-08 07:39:13+00:00",
            "title": "Beyond description. Comment on \"Approaching human language with complex networks\" by Cong & Liu",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Comment on \"Approaching human language with complex networks\" by Cong & Liu",
            "journal ref": "Physics of Life Reviews 11 (4), 621-623 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.1985v1",
            "date": "2014-08-08 21:23:55+00:00",
            "title": "A model of grassroots changes in linguistic systems",
            "authors": [
                "Janet B. Pierrehumbert",
                "Forrest Stonedahl",
                "Robert Daland"
            ],
            "abstract": "Linguistic norms emerge in human communities because people imitate each\nother. A shared linguistic system provides people with the benefits of shared\nknowledge and coordinated planning. Once norms are in place, why would they\never change? This question, echoing broad questions in the theory of social\ndynamics, has particular force in relation to language. By definition, an\ninnovator is in the minority when the innovation first occurs. In some areas of\nsocial dynamics, important minorities can strongly influence the majority\nthrough their power, fame, or use of broadcast media. But most linguistic\nchanges are grassroots developments that originate with ordinary people. Here,\nwe develop a novel model of communicative behavior in communities, and identify\na mechanism for arbitrary innovations by ordinary people to have a good chance\nof being widely adopted.\n  To imitate each other, people must form a mental representation of what other\npeople do. Each time they speak, they must also decide which form to produce\nthemselves. We introduce a new decision function that enables us to smoothly\nexplore the space between two types of behavior: probability matching (matching\nthe probabilities of incoming experience) and regularization (producing some\nforms disproportionately often). Using Monte Carlo methods, we explore the\ninteractions amongst the degree of regularization, the distribution of biases\nin a network, and the network position of the innovator. We identify two\nregimes for the widespread adoption of arbritrary innovations, viewed as\ninformational cascades in the network. With moderate regularization of\nexperienced input, average people (not well-connected people) are the most\nlikely source of successful innovations. Our results shed light on a major\noutstanding puzzle in the theory of language change. The framework also holds\npromise for understanding the dynamics of other social norms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.2873v2",
            "date": "2014-08-12 22:40:21+00:00",
            "title": "First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs",
            "authors": [
                "Awni Y. Hannun",
                "Andrew L. Maas",
                "Daniel Jurafsky",
                "Andrew Y. Ng"
            ],
            "abstract": "We present a method to perform first-pass large vocabulary continuous speech\nrecognition using only a neural network and language model. Deep neural network\nacoustic models are now commonplace in HMM-based speech recognition systems,\nbut building such systems is a complex, domain-specific task. Recent work\ndemonstrated the feasibility of discarding the HMM sequence modeling framework\nby directly predicting transcript text from audio. This paper extends this\napproach in two ways. First, we demonstrate that a straightforward recurrent\nneural network architecture can achieve a high level of accuracy. Second, we\npropose and evaluate a modified prefix-search decoding algorithm. This approach\nto decoding enables first-pass speech recognition with a language model,\ncompletely unaided by the cumbersome infrastructure of HMM-based systems.\nExperiments on the Wall Street Journal corpus demonstrate fairly competitive\nword error rates, and the importance of bi-directional network recurrence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.3934v1",
            "date": "2014-08-18 08:47:55+00:00",
            "title": "On Detecting Messaging Abuse in Short Text Messages using Linguistic and Behavioral patterns",
            "authors": [
                "Alejandro Mosquera",
                "Lamine Aouad",
                "Slawomir Grzonkowski",
                "Dylan Morss"
            ],
            "abstract": "The use of short text messages in social media and instant messaging has\nbecome a popular communication channel during the last years. This rising\npopularity has caused an increment in messaging threats such as spam, phishing\nor malware as well as other threats. The processing of these short text message\nthreats could pose additional challenges such as the presence of lexical\nvariants, SMS-like contractions or advanced obfuscations which can degrade the\nperformance of traditional filtering solutions. By using a real-world SMS data\nset from a large telecommunications operator from the US and a social media\ncorpus, in this paper we analyze the effectiveness of machine learning filters\nbased on linguistic and behavioral patterns in order to detect short text spam\nand abusive users in the network. We have also explored different ways to deal\nwith short text message challenges such as tokenization and entity detection by\nusing text normalization and substring clustering techniques. The obtained\nresults show the validity of the proposed solution by enhancing baseline\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1408.5403v1",
            "date": "2014-08-22 12:15:54+00:00",
            "title": "Neural Mechanism of Language",
            "authors": [
                "Peilei Liu",
                "Ting Wang"
            ],
            "abstract": "This paper is based on our previous work on neural coding. It is a\nself-organized model supported by existing evidences. Firstly, we briefly\nintroduce this model in this paper, and then we explain the neural mechanism of\nlanguage and reasoning with it. Moreover, we find that the position of an area\ndetermines its importance. Specifically, language relevant areas are in the\ncapital position of the cortical kingdom. Therefore they are closely related\nwith autonomous consciousness and working memories. In essence, language is a\nminiature of the real world. Briefly, this paper would like to bridge the gap\nbetween molecule mechanism of neurons and advanced functions such as language\nand reasoning.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1408.5427v1",
            "date": "2014-08-21 17:58:33+00:00",
            "title": "A Case Study in Text Mining: Interpreting Twitter Data From World Cup Tweets",
            "authors": [
                "Daniel Godfrey",
                "Caley Johns",
                "Carl Meyer",
                "Shaina Race",
                "Carol Sadek"
            ],
            "abstract": "Cluster analysis is a field of data analysis that extracts underlying\npatterns in data. One application of cluster analysis is in text-mining, the\nanalysis of large collections of text to find similarities between documents.\nWe used a collection of about 30,000 tweets extracted from Twitter just before\nthe World Cup started. A common problem with real world text data is the\npresence of linguistic noise. In our case it would be extraneous tweets that\nare unrelated to dominant themes. To combat this problem, we created an\nalgorithm that combined the DBSCAN algorithm and a consensus matrix. This way\nwe are left with the tweets that are related to those dominant themes. We then\nused cluster analysis to find those topics that the tweets describe. We\nclustered the tweets using k-means, a commonly used clustering algorithm, and\nNon-Negative Matrix Factorization (NMF) and compared the results. The two\nalgorithms gave similar results, but NMF proved to be faster and provided more\neasily interpreted results. We explored our results using two visualization\ntools, Gephi and Wordle.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1408.6418v1",
            "date": "2014-08-09 05:43:12+00:00",
            "title": "Video In Sentences Out",
            "authors": [
                "Andrei Barbu",
                "Alexander Bridge",
                "Zachary Burchill",
                "Dan Coroian",
                "Sven Dickinson",
                "Sanja Fidler",
                "Aaron Michaux",
                "Sam Mussman",
                "Siddharth Narayanaswamy",
                "Dhaval Salvi",
                "Lara Schmidt",
                "Jiangnan Shangguan",
                "Jeffrey Mark Siskind",
                "Jarrell Waggoner",
                "Song Wang",
                "Jinlian Wei",
                "Yifan Yin",
                "Zhiqi Zhang"
            ],
            "abstract": "We present a system that produces sentential descriptions of video: who did\nwhat to whom, and where and how they did it. Action class is rendered as a\nverb, participant objects as noun phrases, properties of those objects as\nadjectival modifiers in those noun phrases, spatial relations between those\nparticipants as prepositional phrases, and characteristics of the event as\nprepositional-phrase adjuncts and adverbial modifiers. Extracting the\ninformation needed to render these linguistic entities requires an approach to\nevent recognition that recovers object tracks, the trackto-role assignments,\nand changing body posture.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1411.0007v1",
            "date": "2014-10-31 20:04:09+00:00",
            "title": "Rapid Adaptation of POS Tagging for Domain Specific Uses",
            "authors": [
                "John E. Miller",
                "Michael Bloodgood",
                "Manabu Torii",
                "K. Vijay-Shanker"
            ],
            "abstract": "Part-of-speech (POS) tagging is a fundamental component for performing\nnatural language tasks such as parsing, information extraction, and question\nanswering. When POS taggers are trained in one domain and applied in\nsignificantly different domains, their performance can degrade dramatically. We\npresent a methodology for rapid adaptation of POS taggers to new domains. Our\ntechnique is unsupervised in that a manually annotated corpus for the new\ndomain is not necessary. We use suffix information gathered from large amounts\nof raw text as well as orthographic information to increase the lexical\ncoverage. We present an experiment in the Biological domain where our POS\ntagger achieves results comparable to POS taggers specifically trained to this\ndomain.",
            "journal ref": "In Proceedings of the HLT-NAACL BioNLP Workshop on Linking Natural\n  Language and Biology, pages 118-119, New York, New York, June 2006.\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.1243v1",
            "date": "2014-11-05 11:50:15+00:00",
            "title": "Using Twitter to predict football outcomes",
            "authors": [
                "Stylianos Kampakis",
                "Andreas Adamides"
            ],
            "abstract": "Twitter has been proven to be a notable source for predictive modelling on\nvarious domains such as the stock market, the dissemination of diseases or\nsports outcomes. However, such a study has not been conducted in football\n(soccer) so far. The purpose of this research was to study whether data mined\nfrom Twitter can be used for this purpose. We built a set of predictive models\nfor the outcome of football games of the English Premier League for a 3 month\nperiod based on tweets and we studied whether these models can overcome\npredictive models which use only historical data and simple football\nstatistics. Moreover, combined models are constructed using both Twitter and\nhistorical data. The final results indicate that data mined from Twitter can\nindeed be a useful source for predicting games in the Premier League. The final\nTwitter-based model performs significantly better than chance when measured by\nCohen's kappa and is comparable to the model that uses simple statistics and\nhistorical data. Combining both models raises the performance higher than it\nwas achieved by each individual model. Thereby, this study provides evidence\nthat Twitter derived features can indeed provide useful information for the\nprediction of football (soccer) outcomes.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1411.2539v1",
            "date": "2014-11-10 19:09:41+00:00",
            "title": "Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models",
            "authors": [
                "Ryan Kiros",
                "Ruslan Salakhutdinov",
                "Richard S. Zemel"
            ],
            "abstract": "Inspired by recent advances in multimodal learning and machine translation,\nwe introduce an encoder-decoder pipeline that learns (a): a multimodal joint\nembedding space with images and text and (b): a novel language model for\ndecoding distributed representations from our space. Our pipeline effectively\nunifies joint image-text embedding models with multimodal neural language\nmodels. We introduce the structure-content neural language model that\ndisentangles the structure of a sentence to its content, conditioned on\nrepresentations produced by the encoder. The encoder allows one to rank images\nand sentences while the decoder can generate novel descriptions from scratch.\nUsing LSTM to encode sentences, we match the state-of-the-art performance on\nFlickr8K and Flickr30K without using object detections. We also set new best\nresults when using the 19-layer Oxford convolutional network. Furthermore we\nshow that with linear encoders, the learned embedding space captures multimodal\nregularities in terms of vector space arithmetic e.g. *image of a blue car* -\n\"blue\" + \"red\" is near images of red cars. Sample captions generated for 800\nimages are made available for comparison.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1411.2645v1",
            "date": "2014-11-10 22:12:56+00:00",
            "title": "Non-crossing dependencies: least effort, not grammar",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "The use of null hypotheses (in a statistical sense) is common in hard\nsciences but not in theoretical linguistics. Here the null hypothesis that the\nlow frequency of syntactic dependency crossings is expected by an arbitrary\nordering of words is rejected. It is shown that this would require star\ndependency structures, which are both unrealistic and too restrictive. The\nhypothesis of the limited resources of the human brain is revisited. Stronger\nnull hypotheses taking into account actual dependency lengths for the\nlikelihood of crossings are presented. Those hypotheses suggests that crossings\nare likely to reduce when dependencies are shortened. A hypothesis based on\npressure to reduce dependency lengths is more parsimonious than a principle of\nminimization of crossings or a grammatical ban that is totally dissociated from\nthe general and non-linguistic principle of economy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.2674v3",
            "date": "2014-11-11 01:02:20+00:00",
            "title": "The Bayesian Echo Chamber: Modeling Social Influence via Linguistic Accommodation",
            "authors": [
                "Fangjian Guo",
                "Charles Blundell",
                "Hanna Wallach",
                "Katherine Heller"
            ],
            "abstract": "We present the Bayesian Echo Chamber, a new Bayesian generative model for\nsocial interaction data. By modeling the evolution of people's language usage\nover time, this model discovers latent influence relationships between them.\nUnlike previous work on inferring influence, which has primarily focused on\nsimple temporal dynamics evidenced via turn-taking behavior, our model captures\nmore nuanced influence relationships, evidenced via linguistic accommodation\npatterns in interaction content. The model, which is based on a discrete analog\nof the multivariate Hawkes process, permits a fully Bayesian inference\nalgorithm. We validate our model's ability to discover latent influence\npatterns using transcripts of arguments heard by the US Supreme Court and the\nmovie \"12 Angry Men.\" We showcase our model's capabilities by using it to infer\nlatent influence patterns from Federal Open Market Committee meeting\ntranscripts, demonstrating state-of-the-art performance at uncovering social\ndynamics in group discussions.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1411.2679v1",
            "date": "2014-11-11 01:53:21+00:00",
            "title": "Inferring User Preferences by Probabilistic Logical Reasoning over Social Networks",
            "authors": [
                "Jiwei Li",
                "Alan Ritter",
                "Dan Jurafsky"
            ],
            "abstract": "We propose a framework for inferring the latent attitudes or preferences of\nusers by performing probabilistic first-order logical reasoning over the social\nnetwork graph. Our method answers questions about Twitter users like {\\em Does\nthis user like sushi?} or {\\em Is this user a New York Knicks fan?} by building\na probabilistic model that reasons over user attributes (the user's location or\ngender) and the social network (the user's friends and spouse), via inferences\nlike homophily (I am more likely to like sushi if spouse or friends like sushi,\nI am more likely to like the Knicks if I live in New York). The algorithm uses\ndistant supervision, semi-supervised data harvesting and vector space models to\nextract user attributes (e.g. spouse, education, location) and preferences\n(likes and dislikes) from text. The extracted propositions are then fed into a\nprobabilistic reasoner (we investigate both Markov Logic and Probabilistic Soft\nLogic). Our experiments show that probabilistic logical reasoning significantly\nimproves the performance on attribute and relation extraction, and also\nachieves an F-score of 0.791 at predicting a users likes or dislikes,\nsignificantly better than two strong baselines.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1411.3315v1",
            "date": "2014-11-12 20:37:08+00:00",
            "title": "Statistically Significant Detection of Linguistic Change",
            "authors": [
                "Vivek Kulkarni",
                "Rami Al-Rfou",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "We propose a new computational approach for tracking and detecting\nstatistically significant linguistic shifts in the meaning and usage of words.\nSuch linguistic shifts are especially prevalent on the Internet, where the\nrapid exchange of ideas can quickly change a word's meaning. Our meta-analysis\napproach constructs property time series of word usage, and then uses\nstatistically sound change point detection algorithms to identify significant\nlinguistic shifts.\n  We consider and analyze three approaches of increasing complexity to generate\nsuch linguistic property time series, the culmination of which uses\ndistributional characteristics inferred from word co-occurrences. Using\nrecently proposed deep neural language models, we first train vector\nrepresentations of words for each time period. Second, we warp the vector\nspaces into one unified coordinate system. Finally, we construct a\ndistance-based distributional time series for each word to track it's\nlinguistic displacement over time.\n  We demonstrate that our approach is scalable by tracking linguistic change\nacross years of micro-blogging using Twitter, a decade of product reviews using\na corpus of movie reviews from Amazon, and a century of written books using the\nGoogle Book-ngrams. Our analysis reveals interesting patterns of language usage\nchange commensurate with each medium.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4072v1",
            "date": "2014-11-14 22:08:01+00:00",
            "title": "Learning Multi-Relational Semantics Using Neural-Embedding Models",
            "authors": [
                "Bishan Yang",
                "Wen-tau Yih",
                "Xiaodong He",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "In this paper we present a unified framework for modeling multi-relational\nrepresentations, scoring, and learning, and conduct an empirical study of\nseveral recent multi-relational embedding models under the framework. We\ninvestigate the different choices of relation operators based on linear and\nbilinear transformations, and also the effects of entity representations by\nincorporating unsupervised vectors pre-trained on extra textual resources. Our\nresults show several interesting findings, enabling the design of a simple\nembedding model that achieves the new state-of-the-art performance on a popular\nknowledge base completion task evaluated on Freebase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4114v1",
            "date": "2014-11-15 05:44:10+00:00",
            "title": "Definition of Visual Speech Element and Research on a Method of Extracting Feature Vector for Korean Lip-Reading",
            "authors": [
                "Ha Jong Won",
                "Li Gwang Chol",
                "Kim Hyok Chol",
                "Li Kum Song"
            ],
            "abstract": "In this paper, we defined the viseme (visual speech element) and described\nabout the method of extracting visual feature vector. We defined the 10 visemes\nbased on vowel by analyzing of Korean utterance and proposed the method of\nextracting the 20-dimensional visual feature vector, combination of static\nfeatures and dynamic features. Lastly, we took an experiment in recognizing\nwords based on 3-viseme HMM and evaluated the efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.4116v1",
            "date": "2014-11-15 06:32:49+00:00",
            "title": "Investigating the Role of Prior Disambiguation in Deep-learning Compositional Models of Meaning",
            "authors": [
                "Jianpeng Cheng",
                "Dimitri Kartsaklis",
                "Edward Grefenstette"
            ],
            "abstract": "This paper aims to explore the effect of prior disambiguation on neural\nnetwork- based compositional models, with the hope that better semantic\nrepresentations for text compounds can be produced. We disambiguate the input\nword vectors before they are fed into a compositional deep net. A series of\nevaluations shows the positive effect of prior disambiguation for such deep\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.5595v2",
            "date": "2014-11-20 16:39:28+00:00",
            "title": "Linking GloVe with word2vec",
            "authors": [
                "Tianze Shi",
                "Zhiyuan Liu"
            ],
            "abstract": "The Global Vectors for word representation (GloVe), introduced by Jeffrey\nPennington et al. is reported to be an efficient and effective method for\nlearning vector representations of words. State-of-the-art performance is also\nprovided by skip-gram with negative-sampling (SGNS) implemented in the word2vec\ntool. In this note, we explain the similarities between the training objectives\nof the two models, and show that the objective of SGNS is similar to the\nobjective of a specialized form of GloVe, though their cost functions are\ndefined differently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1411.5654v1",
            "date": "2014-11-20 19:50:27+00:00",
            "title": "Learning a Recurrent Visual Representation for Image Caption Generation",
            "authors": [
                "Xinlei Chen",
                "C. Lawrence Zitnick"
            ],
            "abstract": "In this paper we explore the bi-directional mapping between images and their\nsentence-based descriptions. We propose learning this mapping using a recurrent\nneural network. Unlike previous approaches that map both sentences and images\nto a common embedding, we enable the generation of novel sentences given an\nimage. Using the same model, we can also reconstruct the visual features\nassociated with an image given its visual description. We use a novel recurrent\nvisual memory that automatically learns to remember long-term visual concepts\nto aid in both sentence generation and visual feature reconstruction. We\nevaluate our approach on several tasks. These include sentence generation,\nsentence retrieval and image retrieval. State-of-the-art results are shown for\nthe task of generating novel image descriptions. When compared to human\ngenerated captions, our automatically generated captions are preferred by\nhumans over $19.8\\%$ of the time. Results are better than or comparable to\nstate-of-the-art results on the image and sentence retrieval tasks for methods\nusing similar visual features.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1411.5726v2",
            "date": "2014-11-20 23:54:35+00:00",
            "title": "CIDEr: Consensus-based Image Description Evaluation",
            "authors": [
                "Ramakrishna Vedantam",
                "C. Lawrence Zitnick",
                "Devi Parikh"
            ],
            "abstract": "Automatically describing an image with a sentence is a long-standing\nchallenge in computer vision and natural language processing. Due to recent\nprogress in object detection, attribute classification, action recognition,\netc., there is renewed interest in this area. However, evaluating the quality\nof descriptions has proven to be challenging. We propose a novel paradigm for\nevaluating image descriptions that uses human consensus. This paradigm consists\nof three main parts: a new triplet-based method of collecting human annotations\nto measure consensus, a new automated metric (CIDEr) that captures consensus,\nand two new datasets: PASCAL-50S and ABSTRACT-50S that contain 50 sentences\ndescribing each image. Our simple metric captures human judgment of consensus\nbetter than existing metrics across sentences generated by various sources. We\nalso evaluate five state-of-the-art image description approaches using this new\nprotocol and provide a benchmark for future comparisons. A version of CIDEr\nnamed CIDEr-D is available as a part of MS COCO evaluation server to enable\nsystematic evaluation and benchmarking.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1411.5732v1",
            "date": "2014-11-21 00:53:02+00:00",
            "title": "A Joint Probabilistic Classification Model of Relevant and Irrelevant Sentences in Mathematical Word Problems",
            "authors": [
                "Suleyman Cetintas",
                "Luo Si",
                "Yan Ping Xin",
                "Dake Zhang",
                "Joo Young Park",
                "Ron Tzur"
            ],
            "abstract": "Estimating the difficulty level of math word problems is an important task\nfor many educational applications. Identification of relevant and irrelevant\nsentences in math word problems is an important step for calculating the\ndifficulty levels of such problems. This paper addresses a novel application of\ntext categorization to identify two types of sentences in mathematical word\nproblems, namely relevant and irrelevant sentences. A novel joint probabilistic\nclassification model is proposed to estimate the joint probability of\nclassification decisions for all sentences of a math word problem by utilizing\nthe correlation among all sentences along with the correlation between the\nquestion sentence and other sentences, and sentence text. The proposed model is\ncompared with i) a SVM classifier which makes independent classification\ndecisions for individual sentences by only using the sentence text and ii) a\nnovel SVM classifier that considers the correlation between the question\nsentence and other sentences along with the sentence text. An extensive set of\nexperiments demonstrates the effectiveness of the joint probabilistic\nclassification model for identifying relevant and irrelevant sentences as well\nas the novel SVM classifier that utilizes the correlation between the question\nsentence and other sentences. Furthermore, empirical results and analysis show\nthat i) it is highly beneficial not to remove stopwords and ii) utilizing part\nof speech tagging does not make a significant improvement although it has been\nshown to be effective for the related task of math word problem type\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1404.0850v1",
            "date": "2014-04-03 10:44:20+00:00",
            "title": "Application of Ontologies in Identifying Requirements Patterns in Use Cases",
            "authors": [
                "Rui Couto",
                "Ant\u00f3nio Nestor Ribeiro",
                "Jos\u00e9 Creissac Campos"
            ],
            "abstract": "Use case specifications have successfully been used for requirements\ndescription. They allow joining, in the same modeling space, the expectations\nof the stakeholders as well as the needs of the software engineer and analyst\ninvolved in the process. While use cases are not meant to describe a system's\nimplementation, by formalizing their description we are able to extract\nimplementation relevant information from them. More specifically, we are\ninterested in identifying requirements patterns (common requirements with\ntypical implementation solutions) in support for a requirements based software\ndevelopment approach. In the paper we propose the transformation of Use Case\ndescriptions expressed in a Controlled Natural Language into an ontology\nexpressed in the Web Ontology Language (OWL). OWL's query engines can then be\nused to identify requirements patterns expressed as queries over the ontology.\nWe describe a tool that we have developed to support the approach and provide\nan example of usage.",
            "journal ref": "EPTCS 147, 2014, pp. 62-76",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1404.3026v1",
            "date": "2014-04-11 07:55:51+00:00",
            "title": "On the Ground Validation of Online Diagnosis with Twitter and Medical Records",
            "authors": [
                "Todd Bodnar",
                "Victoria C Barclay",
                "Nilam Ram",
                "Conrad S Tucker",
                "Marcel Salath\u00e9"
            ],
            "abstract": "Social media has been considered as a data source for tracking disease.\nHowever, most analyses are based on models that prioritize strong correlation\nwith population-level disease rates over determining whether or not specific\nindividual users are actually sick. Taking a different approach, we develop a\nnovel system for social-media based disease detection at the individual level\nusing a sample of professionally diagnosed individuals. Specifically, we\ndevelop a system for making an accurate influenza diagnosis based on an\nindividual's publicly available Twitter data. We find that about half (17/35 =\n48.57%) of the users in our sample that were sick explicitly discuss their\ndisease on Twitter. By developing a meta classifier that combines text\nanalysis, anomaly detection, and social network analysis, we are able to\ndiagnose an individual with greater than 99% accuracy even if she does not\ndiscuss her health.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1404.3610v1",
            "date": "2014-04-11 02:33:17+00:00",
            "title": "Targeting HIV-related Medication Side Effects and Sentiment Using Twitter Data",
            "authors": [
                "Cosme Adrover",
                "Todd Bodnar",
                "Marcel Salathe"
            ],
            "abstract": "We present a descriptive analysis of Twitter data. Our study focuses on\nextracting the main side effects associated with HIV treatments. The crux of\nour work was the identification of personal tweets referring to HIV. We\nsummarize our results in an infographic aimed at the general public. In\naddition, we present a measure of user sentiment based on hand-rated tweets.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1404.4606v3",
            "date": "2014-04-16 12:59:29+00:00",
            "title": "How Many Topics? Stability Analysis for Topic Models",
            "authors": [
                "Derek Greene",
                "Derek O'Callaghan",
                "P\u00e1draig Cunningham"
            ],
            "abstract": "Topic modeling refers to the task of discovering the underlying thematic\nstructure in a text corpus, where the output is commonly presented as a report\nof the top terms appearing in each topic. Despite the diversity of topic\nmodeling algorithms that have been proposed, a common challenge in successfully\napplying these techniques is the selection of an appropriate number of topics\nfor a given corpus. Choosing too few topics will produce results that are\noverly broad, while choosing too many will result in the \"over-clustering\" of a\ncorpus into many small, highly-similar topics. In this paper, we propose a\nterm-centric stability analysis strategy to address this issue, the idea being\nthat a model with an appropriate number of topics will be more robust to\nperturbations in the data. Using a topic modeling approach based on matrix\nfactorization, evaluations performed on a range of corpora show that this\nstrategy can successfully guide the model selection process.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1406.1078v3",
            "date": "2014-06-03 17:47:08+00:00",
            "title": "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation",
            "authors": [
                "Kyunghyun Cho",
                "Bart van Merrienboer",
                "Caglar Gulcehre",
                "Dzmitry Bahdanau",
                "Fethi Bougares",
                "Holger Schwenk",
                "Yoshua Bengio"
            ],
            "abstract": "In this paper, we propose a novel neural network model called RNN\nEncoder-Decoder that consists of two recurrent neural networks (RNN). One RNN\nencodes a sequence of symbols into a fixed-length vector representation, and\nthe other decodes the representation into another sequence of symbols. The\nencoder and decoder of the proposed model are jointly trained to maximize the\nconditional probability of a target sequence given a source sequence. The\nperformance of a statistical machine translation system is empirically found to\nimprove by using the conditional probabilities of phrase pairs computed by the\nRNN Encoder-Decoder as an additional feature in the existing log-linear model.\nQualitatively, we show that the proposed model learns a semantically and\nsyntactically meaningful representation of linguistic phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.1143v1",
            "date": "2014-06-04 18:59:00+00:00",
            "title": "Identifying Duplicate and Contradictory Information in Wikipedia",
            "authors": [
                "Sarah Weissman",
                "Samet Ayhan",
                "Joshua Bradley",
                "Jimmy Lin"
            ],
            "abstract": "Our study identifies sentences in Wikipedia articles that are either\nidentical or highly similar by applying techniques for near-duplicate detection\nof web pages. This is accomplished with a MapReduce implementation of minhash\nto identify clusters of sentences with high Jaccard similarity. We show that\nthese clusters can be categorized into six different types, two of which are\nparticularly interesting: identical sentences quantify the extent to which\ncontent in Wikipedia is copied and pasted, and near-duplicate sentences that\nstate contradictory facts point to quality issues in Wikipedia.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1406.1827v4",
            "date": "2014-06-06 22:09:27+00:00",
            "title": "Recursive Neural Networks Can Learn Logical Semantics",
            "authors": [
                "Samuel R. Bowman",
                "Christopher Potts",
                "Christopher D. Manning"
            ],
            "abstract": "Tree-structured recursive neural networks (TreeRNNs) for sentence meaning\nhave been successful for many applications, but it remains an open question\nwhether the fixed-length representations that they learn can support tasks as\ndemanding as logical deduction. We pursue this question by evaluating whether\ntwo such models---plain TreeRNNs and tree-structured neural tensor networks\n(TreeRNTNs)---can correctly learn to identify logical relationships such as\nentailment and contradiction using these representations. In our first set of\nexperiments, we generate artificial data from a logical grammar and use it to\nevaluate the models' ability to learn to handle basic relational reasoning,\nrecursive structures, and quantification. We then evaluate the models on the\nmore natural SICK challenge data. Both models perform competitively on the SICK\ndata and generalize well in all three experiments on simulated data, suggesting\nthat they can learn suitable representations for logical inference in natural\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2035v2",
            "date": "2014-06-08 22:35:09+00:00",
            "title": "Learning Word Representations with Hierarchical Sparse Coding",
            "authors": [
                "Dani Yogatama",
                "Manaal Faruqui",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We propose a new method for learning word representations using hierarchical\nregularization in sparse coding inspired by the linguistic study of word\nmeanings. We show an efficient learning algorithm based on stochastic proximal\nmethods that is significantly faster than previous approaches, making it\npossible to perform hierarchical sparse coding on a corpus of billions of word\ntokens. Experiments on various benchmark tasks---word similarity ranking,\nanalogies, sentence completion, and sentiment analysis---demonstrate that the\nmethod outperforms or is competitive with state-of-the-art methods. Our word\nrepresentations are available at\n\\url{http://www.ark.cs.cmu.edu/dyogatam/wordvecs/}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2538v1",
            "date": "2014-06-10 13:16:36+00:00",
            "title": "FrameNet CNL: a Knowledge Representation and Information Extraction Language",
            "authors": [
                "Guntis Barzdins"
            ],
            "abstract": "The paper presents a FrameNet-based information extraction and knowledge\nrepresentation framework, called FrameNet-CNL. The framework is used on natural\nlanguage documents and represents the extracted knowledge in a tailor-made\nFrame-ontology from which unambiguous FrameNet-CNL paraphrase text can be\ngenerated automatically in multiple languages. This approach brings together\nthe fields of information extraction and CNL, because a source text can be\nconsidered belonging to FrameNet-CNL, if information extraction parser produces\nthe correct knowledge representation as a result. We describe a\nstate-of-the-art information extraction parser used by a national news agency\nand speculate that FrameNet-CNL eventually could shape the natural language\nsubset used for writing the newswire articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2880v1",
            "date": "2014-06-11 12:25:26+00:00",
            "title": "POS Tagging and its Applications for Mathematics",
            "authors": [
                "Ulf Sch\u00f6neberg",
                "Wolfram Sperber"
            ],
            "abstract": "Content analysis of scientific publications is a nontrivial task, but a\nuseful and important one for scientific information services. In the Gutenberg\nera it was a domain of human experts; in the digital age many machine-based\nmethods, e.g., graph analysis tools and machine-learning techniques, have been\ndeveloped for it. Natural Language Processing (NLP) is a powerful\nmachine-learning approach to semiautomatic speech and language processing,\nwhich is also applicable to mathematics. The well established methods of NLP\nhave to be adjusted for the special needs of mathematics, in particular for\nhandling mathematical formulae. We demonstrate a mathematics-aware part of\nspeech tagger and give a short overview about our adaptation of NLP methods for\nmathematical publications. We show the use of the tools developed for key\nphrase extraction and classification in the database zbMATH.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1406.2963v2",
            "date": "2014-06-11 17:02:14+00:00",
            "title": "A machine-compiled macroevolutionary history of Phanerozoic life",
            "authors": [
                "Shanan E. Peters",
                "Ce Zhang",
                "Miron Livny",
                "Christopher R\u00e9"
            ],
            "abstract": "Many aspects of macroevolutionary theory and our understanding of biotic\nresponses to global environmental change derive from literature-based\ncompilations of palaeontological data. Existing manually assembled databases\nare, however, incomplete and difficult to assess and enhance. Here, we develop\nand validate the quality of a machine reading system, PaleoDeepDive, that\nautomatically locates and extracts data from heterogeneous text, tables, and\nfigures in publications. PaleoDeepDive performs comparably to humans in complex\ndata extraction and inference tasks and generates congruent synthetic\nmacroevolutionary results. Unlike traditional databases, PaleoDeepDive produces\na probabilistic database that systematically improves as information is added.\nWe also show that the system can readily accommodate sophisticated data types,\nsuch as morphological data in biological illustrations and associated textual\ndescriptions. Our machine reading approach to scientific data integration and\nsynthesis brings within reach many questions that are currently underdetermined\nand does so in ways that may stimulate entirely new modes of inquiry.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1406.3287v3",
            "date": "2014-06-12 17:01:10+00:00",
            "title": "A Clustering Analysis of Tweet Length and its Relation to Sentiment",
            "authors": [
                "Matthew Mayo"
            ],
            "abstract": "Sentiment analysis of Twitter data is performed. The researcher has made the\nfollowing contributions via this paper: (1) an innovative method for deriving\nsentiment score dictionaries using an existing sentiment dictionary as seed\nwords is explored, and (2) an analysis of clustered tweet sentiment scores\nbased on tweet length is performed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3830v1",
            "date": "2014-06-15 17:15:32+00:00",
            "title": "Modelling, Visualising and Summarising Documents with a Single Convolutional Neural Network",
            "authors": [
                "Misha Denil",
                "Alban Demiraj",
                "Nal Kalchbrenner",
                "Phil Blunsom",
                "Nando de Freitas"
            ],
            "abstract": "Capturing the compositional process which maps the meaning of words to that\nof documents is a central challenge for researchers in Natural Language\nProcessing and Information Retrieval. We introduce a model that is able to\nrepresent the meaning of documents by embedding them in a low dimensional\nvector space, while preserving distinctions of word and sentence order crucial\nfor capturing nuanced semantics. Our model is based on an extended Dynamic\nConvolution Neural Network, which learns convolution filters at both the\nsentence and document level, hierarchically learning to capture and compose low\nlevel lexical features into high level semantic concepts. We demonstrate the\neffectiveness of this model on a range of document modelling tasks, achieving\nstrong results with no feature engineering and with a more compact model.\nInspired by recent advances in visualising deep convolution networks for\ncomputer vision, we present a novel visualisation technique for our document\nnetworks which not only provides insight into their learning process, but also\ncan be interpreted to produce a compelling automatic summarisation system for\ntexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.3855v1",
            "date": "2014-06-15 19:38:25+00:00",
            "title": "Human language reveals a universal positivity bias",
            "authors": [
                "Peter Sheridan Dodds",
                "Eric M. Clark",
                "Suma Desu",
                "Morgan R. Frank",
                "Andrew J. Reagan",
                "Jake Ryland Williams",
                "Lewis Mitchell",
                "Kameron Decker Harris",
                "Isabel M. Kloumann",
                "James P. Bagrow",
                "Karine Megerdoomian",
                "Matthew T. McMahon",
                "Brian F. Tivnan",
                "Christopher M. Danforth"
            ],
            "abstract": "Using human evaluation of 100,000 words spread across 24 corpora in 10\nlanguages diverse in origin and culture, we present evidence of a deep imprint\nof human sociality in language, observing that (1) the words of natural human\nlanguage possess a universal positivity bias; (2) the estimated emotional\ncontent of words is consistent between languages under translation; and (3)\nthis positivity bias is strongly independent of frequency of word usage.\nAlongside these general regularities, we describe inter-language variations in\nthe emotional spectrum of languages which allow us to rank corpora. We also\nshow how our word evaluations can be used to construct physical-like\ninstruments for both real-time and offline measurement of the emotional content\nof large-scale texts.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1406.3915v1",
            "date": "2014-06-16 06:41:54+00:00",
            "title": "A Bengali HMM Based Speech Synthesis System",
            "authors": [
                "Sankar Mukherjee",
                "Shyamal Kumar Das Mandal"
            ],
            "abstract": "The paper presents the capability of an HMM-based TTS system to produce\nBengali speech. In this synthesis method, trajectories of speech parameters are\ngenerated from the trained Hidden Markov Models. A final speech waveform is\nsynthesized from those speech parameters. In our experiments, spectral\nproperties were represented by Mel Cepstrum Coefficients. Both the training and\nsynthesis issues are investigated in this paper using annotated Bengali speech\ndatabase. Experimental evaluation depicts that the developed text-to-speech\nsystem is capable of producing adequately natural speech in terms of\nintelligibility and intonation for Bengali.",
            "journal ref": "Oriental COCOSDA 2012, pp.225 259",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1406.4441v2",
            "date": "2014-06-17 17:29:39+00:00",
            "title": "Scaling laws and fluctuations in the statistics of word frequencies",
            "authors": [
                "Martin Gerlach",
                "Eduardo G. Altmann"
            ],
            "abstract": "In this paper we combine statistical analysis of large text databases and\nsimple stochastic models to explain the appearance of scaling laws in the\nstatistics of word frequencies. Besides the sublinear scaling of the vocabulary\nsize with database size (Heaps' law), here we report a new scaling of the\nfluctuations around this average (fluctuation scaling analysis). We explain\nboth scaling laws by modeling the usage of words by simple stochastic processes\nin which the overall distribution of word-frequencies is fat tailed (Zipf's\nlaw) and the frequency of a single word is subject to fluctuations across\ndocuments (as in topic models). In this framework, the mean and the variance of\nthe vocabulary size can be expressed as quenched averages, implying that: i)\nthe inhomogeneous dissemination of words cause a reduction of the average\nvocabulary size in comparison to the homogeneous case, and ii) correlations in\nthe co-occurrence of words lead to an increase in the variance and the\nvocabulary size becomes a non-self-averaging quantity. We address the\nimplications of these observations to the measurement of lexical richness. We\ntest our results in three large text databases (Google-ngram, Enlgish\nWikipedia, and a collection of scientific articles).",
            "journal ref": "New Journal of Physics 16 (2014), 113010",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1406.4469v1",
            "date": "2014-06-17 18:32:18+00:00",
            "title": "Authorship Attribution through Function Word Adjacency Networks",
            "authors": [
                "Santiago Segarra",
                "Mark Eisen",
                "Alejandro Ribeiro"
            ],
            "abstract": "A method for authorship attribution based on function word adjacency networks\n(WANs) is introduced. Function words are parts of speech that express\ngrammatical relationships between other words but do not carry lexical meaning\non their own. In the WANs in this paper, nodes are function words and directed\nedges stand in for the likelihood of finding the sink word in the ordered\nvicinity of the source word. WANs of different authors can be interpreted as\ntransition probabilities of a Markov chain and are therefore compared in terms\nof their relative entropies. Optimal selection of WAN parameters is studied and\nattribution accuracy is benchmarked across a diverse pool of authors and\nvarying text lengths. This analysis shows that, since function words are\nindependent of content, their use tends to be specific to an author and that\nthe relational data captured by function WANs is a good summary of stylometric\nfingerprints. Attribution accuracy is observed to exceed the one achieved by\nmethods that rely on word frequencies alone. Further combining WANs with\nmethods that rely on word frequencies alone, results in larger attribution\naccuracy, indicating that both sources of information encode different aspects\nof authorial styles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.4498v2",
            "date": "2014-06-17 19:49:03+00:00",
            "title": "Extracting information from S-curves of language change",
            "authors": [
                "Fakhteh Ghanbarnejad",
                "Martin Gerlach",
                "Jose M. Miotto",
                "Eduardo G. Altmann"
            ],
            "abstract": "It is well accepted that adoption of innovations are described by S-curves\n(slow start, accelerating period, and slow end). In this paper, we analyze how\nmuch information on the dynamics of innovation spreading can be obtained from a\nquantitative description of S-curves. We focus on the adoption of linguistic\ninnovations for which detailed databases of written texts from the last 200\nyears allow for an unprecedented statistical precision. Combining data analysis\nwith simulations of simple models (e.g., the Bass dynamics on complex networks)\nwe identify signatures of endogenous and exogenous factors in the S-curves of\nadoption. We propose a measure to quantify the strength of these factors and\nthree different methods to estimate it from S-curves. We obtain cases in which\nthe exogenous factors are dominant (in the adoption of German orthographic\nreforms and of one irregular verb) and cases in which endogenous factors are\ndominant (in the adoption of conventions for romanization of Russian names and\nin the regularization of most studied verbs). These results show that the shape\nof S-curve is not universal and contains information on the adoption mechanism.\n(published at \"J. R. Soc. Interface, vol. 11, no. 101, (2014) 1044\"; DOI:\nhttp://dx.doi.org/10.1098/rsif.2014.1044)",
            "journal ref": "J. R. Soc. Interface 6 December 2014 vol. 11 no. 101 20141044",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1406.4710v1",
            "date": "2014-06-18 13:32:55+00:00",
            "title": "Typed Hilbert Epsilon Operators and the Semantics of Determiner Phrases (Invited Lecture)",
            "authors": [
                "Christian Retor\u00e9"
            ],
            "abstract": "The semantics of determiner phrases, be they definite de- scriptions,\nindefinite descriptions or quantified noun phrases, is often as- sumed to be a\nfully solved question: common nouns are properties, and determiners are\ngeneralised quantifiers that apply to two predicates: the property\ncorresponding to the common noun and the one corresponding to the verb phrase.\nWe first present a criticism of this standard view. Firstly, the semantics of\ndeterminers does not follow the syntactical structure of the sentence. Secondly\nthe standard interpretation of the indefinite article cannot ac- count for\nnominal sentences. Thirdly, the standard view misses the linguis- tic asymmetry\nbetween the two properties of a generalised quantifier. In the sequel, we\npropose a treatment of determiners and quantifiers as Hilbert terms in a richly\ntyped system that we initially developed for lexical semantics, using a many\nsorted logic for semantical representations. We present this semantical\nframework called the Montagovian generative lexicon and show how these terms\nbetter match the syntactical structure and avoid the aforementioned problems of\nthe standard approach. Hilbert terms rather differ from choice functions in\nthat there is one polymorphic operator and not one operator per formula. They\nalso open an intriguing connection between the logic for meaning assembly, the\ntyped lambda calculus handling compositionality and the many-sorted logic for\nsemantical representations. Furthermore epsilon terms naturally introduce\ntype-judgements and confirm the claim that type judgment are a form of\npresupposition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.5679v1",
            "date": "2014-06-22 06:22:50+00:00",
            "title": "Deep Fragment Embeddings for Bidirectional Image Sentence Mapping",
            "authors": [
                "Andrej Karpathy",
                "Armand Joulin",
                "Li Fei-Fei"
            ],
            "abstract": "We introduce a model for bidirectional retrieval of images and sentences\nthrough a multi-modal embedding of visual and natural language data. Unlike\nprevious models that directly map images or sentences into a common embedding\nspace, our model works on a finer level and embeds fragments of images\n(objects) and fragments of sentences (typed dependency tree relations) into a\ncommon space. In addition to a ranking objective seen in previous work, this\nallows us to add a new fragment alignment objective that learns to directly\nassociate these fragments across modalities. Extensive experimental evaluation\nshows that reasoning on both the global level of images and sentences and the\nfiner level of their respective fragments significantly improves performance on\nimage-sentence retrieval tasks. Additionally, our model provides interpretable\npredictions since the inferred inter-modal fragment alignment is explicit.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1406.5824v1",
            "date": "2014-06-23 07:56:23+00:00",
            "title": "VideoSET: Video Summary Evaluation through Text",
            "authors": [
                "Serena Yeung",
                "Alireza Fathi",
                "Li Fei-Fei"
            ],
            "abstract": "In this paper we present VideoSET, a method for Video Summary Evaluation\nthrough Text that can evaluate how well a video summary is able to retain the\nsemantic information contained in its original video. We observe that semantics\nis most easily expressed in words, and develop a text-based approach for the\nevaluation. Given a video summary, a text representation of the video summary\nis first generated, and an NLP-based metric is then used to measure its\nsemantic distance to ground-truth text summaries written by humans. We show\nthat our technique has higher agreement with human judgment than pixel-based\ndistance metrics. We also release text annotations and ground-truth text\nsummaries for a number of publicly available video datasets, for use by the\ncomputer vision community.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1406.6312v2",
            "date": "2014-06-24 17:10:29+00:00",
            "title": "Scalable Topical Phrase Mining from Text Corpora",
            "authors": [
                "Ahmed El-Kishky",
                "Yanglei Song",
                "Chi Wang",
                "Clare Voss",
                "Jiawei Han"
            ],
            "abstract": "While most topic modeling algorithms model text corpora with unigrams, human\ninterpretation often relies on inherent grouping of terms into phrases. As\nsuch, we consider the problem of discovering topical phrases of mixed lengths.\nExisting work either performs post processing to the inference results of\nunigram-based topic models, or utilizes complex n-gram-discovery topic models.\nThese methods generally produce low-quality topical phrases or suffer from poor\nscalability on even moderately-sized datasets. We propose a different approach\nthat is both computationally efficient and effective. Our solution combines a\nnovel phrase mining framework to segment a document into single and multi-word\nphrases, and a new topic model that operates on the induced document partition.\nOur approach discovers high quality topical phrases with negligible extra cost\nto the bag-of-words topic model in a variety of datasets including research\npublication titles, abstracts, reviews, and news articles.",
            "journal ref": "Proceedings of the VLDB Endowment, Vol. 8(3), pp. 305 - 316, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1406.7558v1",
            "date": "2014-06-29 20:16:55+00:00",
            "title": "Human Communication Systems Evolve by Cultural Selection",
            "authors": [
                "Nicolas Fay",
                "Monica Tamariz",
                "T Mark Ellison",
                "Dale Barr"
            ],
            "abstract": "Human communication systems, such as language, evolve culturally; their\ncomponents undergo reproduction and variation. However, a role for selection in\ncultural evolutionary dynamics is less clear. Often neutral evolution (also\nknown as 'drift') models, are used to explain the evolution of human\ncommunication systems, and cultural evolution more generally. Under this\naccount, cultural change is unbiased: for instance, vocabulary, baby names and\npottery designs have been found to spread through random copying.\n  While drift is the null hypothesis for models of cultural evolution it does\nnot always adequately explain empirical results. Alternative models include\ncultural selection, which assumes variant adoption is biased. Theoretical\nmodels of human communication argue that during conversation interlocutors are\nbiased to adopt the same labels and other aspects of linguistic representation\n(including prosody and syntax). This basic alignment mechanism has been\nextended by computer simulation to account for the emergence of linguistic\nconventions. When agents are biased to match the linguistic behavior of their\ninterlocutor, a single variant can propagate across an entire population of\ninteracting computer agents. This behavior-matching account operates at the\nlevel of the individual. We call it the Conformity-biased model. Under a\ndifferent selection account, called content-biased selection, functional\nselection or replicator selection, variant adoption depends upon the intrinsic\nvalue of the particular variant (e.g., ease of learning or use). This second\nalternative account operates at the level of the cultural variant. Following\nBoyd and Richerson we call it the Content-biased model. The present paper tests\nthe drift model and the two biased selection models' ability to explain the\nspread of communicative signal variants in an experimental micro-society.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1406.7806v2",
            "date": "2014-06-30 16:42:25+00:00",
            "title": "Building DNN Acoustic Models for Large Vocabulary Speech Recognition",
            "authors": [
                "Andrew L. Maas",
                "Peng Qi",
                "Ziang Xie",
                "Awni Y. Hannun",
                "Christopher T. Lengerich",
                "Daniel Jurafsky",
                "Andrew Y. Ng"
            ],
            "abstract": "Deep neural networks (DNNs) are now a central component of nearly all\nstate-of-the-art speech recognition systems. Building neural network acoustic\nmodels requires several design decisions including network architecture, size,\nand training loss function. This paper offers an empirical investigation on\nwhich aspects of DNN acoustic model design are most important for speech\nrecognition system performance. We report DNN classifier performance and final\nspeech recognizer word error rates, and compare DNNs using several metrics to\nquantify factors influencing differences in task performance. Our first set of\nexperiments use the standard Switchboard benchmark corpus, which contains\napproximately 300 hours of conversational telephone speech. We compare standard\nDNNs to convolutional networks, and present the first experiments using\nlocally-connected, untied neural networks for acoustic modeling. We\nadditionally build systems on a corpus of 2,100 hours of training data by\ncombining the Switchboard and Fisher corpora. This larger corpus allows us to\nmore thoroughly examine performance of large DNN models -- with up to ten times\nmore parameters than those typically used in speech recognition systems. Our\nresults suggest that a relatively simple DNN architecture and optimization\ntechnique produces strong results. These findings, along with previous work,\nhelp establish a set of best practices for building DNN hybrid speech\nrecognition systems with maximum likelihood training. Our experiments in DNN\noptimization additionally serve as a case study for training DNNs with\ndiscriminative loss functions for speech tasks, as well as DNN classifiers more\ngenerally.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.0210v4",
            "date": "2014-10-01 12:59:16+00:00",
            "title": "A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input",
            "authors": [
                "Mateusz Malinowski",
                "Mario Fritz"
            ],
            "abstract": "We propose a method for automatically answering questions about images by\nbringing together recent advances from natural language processing and computer\nvision. We combine discrete reasoning with uncertain predictions by a\nmulti-world approach that represents uncertainty about the perceived world in a\nbayesian framework. Our approach can handle human questions of high complexity\nabout realistic scenes and replies with range of answer like counts, object\nclasses, instances and lists of them. The system is directly trained from\nquestion-answer pairs. We establish a first benchmark for this task that can be\nseen as a modern attempt at a visual turing test.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1410.0316v1",
            "date": "2014-10-01 18:15:50+00:00",
            "title": "Using social network graph analysis for interest detection",
            "authors": [
                "Brian Lee Yung Rowe"
            ],
            "abstract": "A person's interests exist as an internal state and are difficult to define.\nSince only external actions are observable, a proxy must be used that\nrepresents someone's interests. Techniques like collaborative filtering,\nbehavioral targeting, and hashtag analysis implicitly model an individual's\ninterests. I argue that these models are limited to shallow, temporary\ninterests, which do not reflect people's deeper interests or passions. I\npropose an alternative model of interests that takes advantage of a user's\nsocial graph. The basic principle is that people only follow those that\ninterest them, so the social graph is an effective and robust proxy for\npeople's interests.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1410.1090v1",
            "date": "2014-10-04 20:24:34+00:00",
            "title": "Explain Images with Multimodal Recurrent Neural Networks",
            "authors": [
                "Junhua Mao",
                "Wei Xu",
                "Yi Yang",
                "Jiang Wang",
                "Alan L. Yuille"
            ],
            "abstract": "In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model\nfor generating novel sentence descriptions to explain the content of images. It\ndirectly models the probability distribution of generating a word given\nprevious words and the image. Image descriptions are generated by sampling from\nthis distribution. The model consists of two sub-networks: a deep recurrent\nneural network for sentences and a deep convolutional network for images. These\ntwo sub-networks interact with each other in a multimodal layer to form the\nwhole m-RNN model. The effectiveness of our model is validated on three\nbenchmark datasets (IAPR TC-12, Flickr 8K, and Flickr 30K). Our model\noutperforms the state-of-the-art generative method. In addition, the m-RNN\nmodel can be applied to retrieval tasks for retrieving images or sentences, and\nachieves significant performance improvement over the state-of-the-art methods\nwhich directly optimize the ranking objective function for retrieval.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1410.2455v3",
            "date": "2014-10-09 13:41:18+00:00",
            "title": "BilBOWA: Fast Bilingual Distributed Representations without Word Alignments",
            "authors": [
                "Stephan Gouws",
                "Yoshua Bengio",
                "Greg Corrado"
            ],
            "abstract": "We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple\nand computationally-efficient model for learning bilingual distributed\nrepresentations of words which can scale to large monolingual datasets and does\nnot require word-aligned parallel training data. Instead it trains directly on\nmonolingual data and extracts a bilingual signal from a smaller set of raw-text\nsentence-aligned data. This is achieved using a novel sampled bag-of-words\ncross-lingual objective, which is used to regularize two noise-contrastive\nlanguage models for efficient cross-lingual feature learning. We show that\nbilingual embeddings learned using the proposed model outperform\nstate-of-the-art methods on a cross-lingual document classification task as\nwell as a lexical translation task on WMT11 data.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1410.2479v2",
            "date": "2014-10-09 14:15:42+00:00",
            "title": "Spatial Diffuseness Features for DNN-Based Speech Recognition in Noisy and Reverberant Environments",
            "authors": [
                "Andreas Schwarz",
                "Christian Huemmer",
                "Roland Maas",
                "Walter Kellermann"
            ],
            "abstract": "We propose a spatial diffuseness feature for deep neural network (DNN)-based\nautomatic speech recognition to improve recognition accuracy in reverberant and\nnoisy environments. The feature is computed in real-time from multiple\nmicrophone signals without requiring knowledge or estimation of the direction\nof arrival, and represents the relative amount of diffuse noise in each time\nand frequency bin. It is shown that using the diffuseness feature as an\nadditional input to a DNN-based acoustic model leads to a reduced word error\nrate for the REVERB challenge corpus, both compared to logmelspec features\nextracted from noisy signals, and features enhanced by spectral subtraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.3916v11",
            "date": "2014-10-15 03:13:18+00:00",
            "title": "Memory Networks",
            "authors": [
                "Jason Weston",
                "Sumit Chopra",
                "Antoine Bordes"
            ],
            "abstract": "We describe a new class of learning models called memory networks. Memory\nnetworks reason with inference components combined with a long-term memory\ncomponent; they learn how to use these jointly. The long-term memory can be\nread and written to, with the goal of using it for prediction. We investigate\nthese models in the context of question answering (QA) where the long-term\nmemory effectively acts as a (dynamic) knowledge base, and the output is a\ntextual response. We evaluate them on a large-scale QA task, and a smaller, but\nmore complex, toy task generated from a simulated world. In the latter, we show\nthe reasoning power of such models by chaining multiple supporting sentences to\nanswer questions that require understanding the intension of verbs.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1410.4510v2",
            "date": "2014-10-16 17:35:31+00:00",
            "title": "Graph-Sparse LDA: A Topic Model with Structured Sparsity",
            "authors": [
                "Finale Doshi-Velez",
                "Byron Wallace",
                "Ryan Adams"
            ],
            "abstract": "Originally designed to model text, topic modeling has become a powerful tool\nfor uncovering latent structure in domains including medicine, finance, and\nvision. The goals for the model vary depending on the application: in some\ncases, the discovered topics may be used for prediction or some other\ndownstream task. In other cases, the content of the topic itself may be of\nintrinsic scientific interest.\n  Unfortunately, even using modern sparse techniques, the discovered topics are\noften difficult to interpret due to the high dimensionality of the underlying\nspace. To improve topic interpretability, we introduce Graph-Sparse LDA, a\nhierarchical topic model that leverages knowledge of relationships between\nwords (e.g., as encoded by an ontology). In our model, topics are summarized by\na few latent concept-words from the underlying graph that explain the observed\nwords. Graph-Sparse LDA recovers sparse, interpretable summaries on two\nreal-world biomedical datasets while matching state-of-the-art prediction\nperformance.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1410.5485v2",
            "date": "2014-10-20 22:04:08+00:00",
            "title": "A stronger null hypothesis for crossing dependencies",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "The syntactic structure of a sentence can be modeled as a tree where vertices\nare words and edges indicate syntactic dependencies between words. It is\nwell-known that those edges normally do not cross when drawn over the sentence.\nHere a new null hypothesis for the number of edge crossings of a sentence is\npresented. That null hypothesis takes into account the length of the pair of\nedges that may cross and predicts the relative number of crossings in random\ntrees with a small error, suggesting that a ban of crossings or a principle of\nminimization of crossings are not needed in general to explain the origins of\nnon-crossing dependencies. Our work paves the way for more powerful null\nhypotheses to investigate the origins of non-crossing dependencies in nature.",
            "journal ref": "(2014). Europhysics Letters 108 (5), 58003",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.5491v1",
            "date": "2014-10-20 22:28:55+00:00",
            "title": "Using Mechanical Turk to Build Machine Translation Evaluation Sets",
            "authors": [
                "Michael Bloodgood",
                "Chris Callison-Burch"
            ],
            "abstract": "Building machine translation (MT) test sets is a relatively expensive task.\nAs MT becomes increasingly desired for more and more language pairs and more\nand more domains, it becomes necessary to build test sets for each case. In\nthis paper, we investigate using Amazon's Mechanical Turk (MTurk) to make MT\ntest sets cheaply. We find that MTurk can be used to make test sets much\ncheaper than professionally-produced test sets. More importantly, in\nexperiments with multiple MT systems, we find that the MTurk-produced test sets\nyield essentially the same conclusions regarding system performance as the\nprofessionally-produced test sets yield.",
            "journal ref": "In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech\n  and Language Data with Amazon's Mechanical Turk, pages 208-211, Los Angeles,\n  California, June 2010. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.5877v1",
            "date": "2014-10-21 22:55:48+00:00",
            "title": "Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation",
            "authors": [
                "Michael Bloodgood",
                "Chris Callison-Burch"
            ],
            "abstract": "We explore how to improve machine translation systems by adding more\ntranslation data in situations where we already have substantial resources. The\nmain challenge is how to buck the trend of diminishing returns that is commonly\nencountered. We present an active learning-style data solicitation algorithm to\nmeet this challenge. We test it, gathering annotations via Amazon Mechanical\nTurk, and find that we get an order of magnitude increase in performance rates\nof improvement.",
            "journal ref": "In Proceedings of the 48th Annual Meeting of the Association for\n  Computational Linguistics, pages 854-864, Uppsala, Sweden, July 2010.\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8206v4",
            "date": "2014-10-30 00:20:31+00:00",
            "title": "Addressing the Rare Word Problem in Neural Machine Translation",
            "authors": [
                "Minh-Thang Luong",
                "Ilya Sutskever",
                "Quoc V. Le",
                "Oriol Vinyals",
                "Wojciech Zaremba"
            ],
            "abstract": "Neural Machine Translation (NMT) is a new approach to machine translation\nthat has shown promising results that are comparable to traditional approaches.\nA significant weakness in conventional NMT systems is their inability to\ncorrectly translate very rare words: end-to-end NMTs tend to have relatively\nsmall vocabularies with a single unk symbol that represents every possible\nout-of-vocabulary (OOV) word. In this paper, we propose and implement an\neffective technique to address this problem. We train an NMT system on data\nthat is augmented by the output of a word alignment algorithm, allowing the NMT\nsystem to emit, for each OOV word in the target sentence, the position of its\ncorresponding word in the source sentence. This information is later utilized\nin a post-processing step that translates every OOV word using a dictionary.\nOur experiments on the WMT14 English to French translation task show that this\nmethod provides a substantial improvement of up to 2.8 BLEU points over an\nequivalent NMT system that does not use this technique. With 37.5 BLEU points,\nour NMT system is the first to surpass the best result achieved on a WMT14\ncontest task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8326v1",
            "date": "2014-10-30 11:02:39+00:00",
            "title": "Towards Learning Object Affordance Priors from Technical Texts",
            "authors": [
                "Nicholas H. Kirk"
            ],
            "abstract": "Everyday activities performed by artificial assistants can potentially be\nexecuted naively and dangerously given their lack of common sense knowledge.\nThis paper presents conceptual work towards obtaining prior knowledge on the\nusual modality (passive or active) of any given entity, and their affordance\nestimates, by extracting high-confidence ability modality semantic relations (X\ncan Y relationship) from non-figurative texts, by analyzing co-occurrence of\ngrammatical instances of subjects and verbs, and verbs and objects. The\ndiscussion includes an outline of the concept, potential and limitations, and\npossible feature and learning framework adoption.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1410.8553v1",
            "date": "2014-10-30 20:52:48+00:00",
            "title": "A random forest system combination approach for error detection in digital dictionaries",
            "authors": [
                "Michael Bloodgood",
                "Peng Ye",
                "Paul Rodrigues",
                "David Zajic",
                "David Doermann"
            ],
            "abstract": "When digitizing a print bilingual dictionary, whether via optical character\nrecognition or manual entry, it is inevitable that errors are introduced into\nthe electronic version that is created. We investigate automating the process\nof detecting errors in an XML representation of a digitized print dictionary\nusing a hybrid approach that combines rule-based, feature-based, and language\nmodel-based methods. We investigate combining methods and show that using\nrandom forests is a promising approach. We find that in isolation, unsupervised\nmethods rival the performance of supervised methods. Random forests typically\nrequire training data so we investigate how we can apply random forests to\ncombine individual base methods that are themselves unsupervised without\nrequiring large amounts of training data. Experiments reveal empirically that a\nrelatively small amount of data is sufficient and can potentially be further\nreduced through specific selection criteria.",
            "journal ref": "In Proceedings of the Workshop on Innovative Hybrid Approaches to\n  the Processing of Textual Data, pages 78-86, Avignon, France, April 2012.\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.2944v2",
            "date": "2014-09-10 03:05:22+00:00",
            "title": "Collaborative Deep Learning for Recommender Systems",
            "authors": [
                "Hao Wang",
                "Naiyan Wang",
                "Dit-Yan Yeung"
            ],
            "abstract": "Collaborative filtering (CF) is a successful approach commonly used by many\nrecommender systems. Conventional CF-based methods use the ratings given to\nitems by users as the sole source of information for learning to make\nrecommendation. However, the ratings are often very sparse in many\napplications, causing CF-based methods to degrade significantly in their\nrecommendation performance. To address this sparsity problem, auxiliary\ninformation such as item content information may be utilized. Collaborative\ntopic regression (CTR) is an appealing recent method taking this approach which\ntightly couples the two components that learn from two different sources of\ninformation. Nevertheless, the latent representation learned by CTR may not be\nvery effective when the auxiliary information is very sparse. To address this\nproblem, we generalize recent advances in deep learning from i.i.d. input to\nnon-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian\nmodel called collaborative deep learning (CDL), which jointly performs deep\nrepresentation learning for the content information and collaborative filtering\nfor the ratings (feedback) matrix. Extensive experiments on three real-world\ndatasets from different domains show that CDL can significantly advance the\nstate of the art.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1409.7591v1",
            "date": "2014-09-26 15:11:57+00:00",
            "title": "Topic Similarity Networks: Visual Analytics for Large Document Sets",
            "authors": [
                "Arun S. Maiya",
                "Robert M. Rolfe"
            ],
            "abstract": "We investigate ways in which to improve the interpretability of LDA topic\nmodels by better analyzing and visualizing their outputs. We focus on examining\nwhat we refer to as topic similarity networks: graphs in which nodes represent\nlatent topics in text collections and links represent similarity among topics.\nWe describe efficient and effective approaches to both building and labeling\nsuch networks. Visualizations of topic models based on these networks are shown\nto be a powerful means of exploring, characterizing, and summarizing large\ncollections of unstructured text documents. They help to \"tease out\"\nnon-obvious connections among different sets of documents and provide insights\ninto how topics form larger themes. We demonstrate the efficacy and\npracticality of these approaches through two case studies: 1) NSF grants for\nbasic research spanning a 14 year period and 2) the entire English portion of\nWikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1409.8484v1",
            "date": "2014-09-30 11:10:23+00:00",
            "title": "An agent-driven semantical identifier using radial basis neural networks and reinforcement learning",
            "authors": [
                "Christian Napoli",
                "Giuseppe Pappalardo",
                "Emiliano Tramontana"
            ],
            "abstract": "Due to the huge availability of documents in digital form, and the deception\npossibility raise bound to the essence of digital documents and the way they\nare spread, the authorship attribution problem has constantly increased its\nrelevance. Nowadays, authorship attribution,for both information retrieval and\nanalysis, has gained great importance in the context of security, trust and\ncopyright preservation. This work proposes an innovative multi-agent driven\nmachine learning technique that has been developed for authorship attribution.\nBy means of a preprocessing for word-grouping and time-period related analysis\nof the common lexicon, we determine a bias reference level for the recurrence\nfrequency of the words within analysed texts, and then train a Radial Basis\nNeural Networks (RBPNN)-based classifier to identify the correct author. The\nmain advantage of the proposed approach lies in the generality of the semantic\nanalysis, which can be applied to different contexts and lexical domains,\nwithout requiring any modification. Moreover, the proposed system is able to\nincorporate an external input, meant to tune the classifier, and then\nself-adjust by means of continuous learning reinforcement.",
            "journal ref": "Proceedings of the XV Workshop \"Dagli Oggetti agli Agenti\" (WOA\n  2014), on CEUR-WS, volume 1260, ISSN: 1613-073, Catania, Italy, Sepember.\n  25-26, 2014. http://ceur-ws.org/Vol-1260/",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1412.0696v2",
            "date": "2014-12-01 21:27:18+00:00",
            "title": "Understanding confounding effects in linguistic coordination: an information-theoretic approach",
            "authors": [
                "Shuyang Gao",
                "Greg Ver Steeg",
                "Aram Galstyan"
            ],
            "abstract": "We suggest an information-theoretic approach for measuring stylistic\ncoordination in dialogues. The proposed measure has a simple predictive\ninterpretation and can account for various confounding factors through proper\nconditioning. We revisit some of the previous studies that reported strong\nsignatures of stylistic accommodation, and find that a significant part of the\nobserved coordination can be attributed to a simple confounding effect - length\ncoordination. Specifically, longer utterances tend to be followed by longer\nresponses, which gives rise to spurious correlations in the other stylistic\nfeatures. We propose a test to distinguish correlations in length due to\ncontextual factors (topic of conversation, user verbosity, etc.) and\nturn-by-turn coordination. We also suggest a test to identify whether stylistic\ncoordination persists even after accounting for length coordination and\ncontextual factors.",
            "journal ref": "PLoS ONE 10(6): e0130167, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1410.8027v3",
            "date": "2014-10-29 15:38:29+00:00",
            "title": "Towards a Visual Turing Challenge",
            "authors": [
                "Mateusz Malinowski",
                "Mario Fritz"
            ],
            "abstract": "As language and visual understanding by machines progresses rapidly, we are\nobserving an increasing interest in holistic architectures that tightly\ninterlink both modalities in a joint learning and inference process. This trend\nhas allowed the community to progress towards more challenging and open tasks\nand refueled the hope at achieving the old AI dream of building machines that\ncould pass a turing test in open domains. In order to steadily make progress\ntowards this goal, we realize that quantifying performance becomes increasingly\ndifficult. Therefore we ask how we can precisely define such challenges and how\nwe can evaluate different algorithms on this open tasks? In this paper, we\nsummarize and discuss such challenges as well as try to give answers where\nappropriate options are available in the literature. We exemplify some of the\nsolutions on a recently presented dataset of question-answering task based on\nreal-world indoor images that establishes a visual turing challenge. Finally,\nwe argue despite the success of unique ground-truth annotation, we likely have\nto step away from carefully curated dataset and rather rely on 'social\nconsensus' as the main driving force to create suitable benchmarks. Providing\ncoverage in this inherently ambiguous output space is an emerging challenge\nthat we face in order to make quantifiable progress in this area.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1406.1234v1",
            "date": "2014-06-04 23:28:51+00:00",
            "title": "A Geometric Method to Obtain the Generation Probability of a Sentence",
            "authors": [
                "Chen Lijiang"
            ],
            "abstract": "\"How to generate a sentence\" is the most critical and difficult problem in\nall the natural language processing technologies. In this paper, we present a\nnew approach to explain the generation process of a sentence from the\nperspective of mathematics. Our method is based on the premise that in our\nbrain a sentence is a part of a word network which is formed by many word\nnodes. Experiments show that the probability of the entire sentence can be\nobtained by the probabilities of single words and the probabilities of the\nco-occurrence of word pairs, which indicate that human use the synthesis method\nto generate a sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2015": [
        {
            "url": "http://arxiv.org/abs/1501.00841v1",
            "date": "2015-01-05 12:55:03+00:00",
            "title": "Chasing the Ghosts of Ibsen: A computational stylistic analysis of drama in translation",
            "authors": [
                "Gerard Lynch",
                "Carl Vogel"
            ],
            "abstract": "Research into the stylistic properties of translations is an issue which has\nreceived some attention in computational stylistics. Previous work by Rybicki\n(2006) on the distinguishing of character idiolects in the work of Polish\nauthor Henryk Sienkiewicz and two corresponding English translations using\nBurrow's Delta method concluded that idiolectal differences could be observed\nin the source texts and this variation was preserved to a large degree in both\ntranslations. This study also found that the two translations were also highly\ndistinguishable from one another. Burrows (2002) examined English translations\nof Juvenal also using the Delta method, results of this work suggest that some\ntranslators are more adept at concealing their own style when translating the\nworks of another author whereas other authors tend to imprint their own style\nto a greater extent on the work they translate. Our work examines the writing\nof a single author, Norwegian playwright Henrik Ibsen, and these writings\ntranslated into both German and English from Norwegian, in an attempt to\ninvestigate the preservation of characterization, defined here as the\ndistinctiveness of textual contributions of characters.",
            "journal ref": "Digital Humanities 2009 Proceedings, University of Maryland,\n  College Park, MD, USA, pages 192-195",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.01243v1",
            "date": "2015-01-06 17:27:40+00:00",
            "title": "Un r\u00e9sumeur \u00e0 base de graphes, ind\u00e9p\u00e9ndant de la langue",
            "authors": [
                "Juan-Manuel Torres-Moreno",
                "Javier Ramirez",
                "Iria da Cunha"
            ],
            "abstract": "In this paper we present REG, a graph-based approach for study a fundamental\nproblem of Natural Language Processing (NLP): the automatic text summarization.\nThe algorithm maps a document as a graph, then it computes the weight of their\nsentences. We have applied this approach to summarize documents in three\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.01254v1",
            "date": "2015-01-06 18:03:13+00:00",
            "title": "Unknown Words Analysis in POS tagging of Sinhala Language",
            "authors": [
                "A. J. P. M. P. Jayaweera",
                "N. G. J. Dias"
            ],
            "abstract": "Part of Speech (POS) is a very vital topic in Natural Language Processing\n(NLP) task in any language, which involves analysing the construction of the\nlanguage, behaviours and the dynamics of the language, the knowledge that could\nbe utilized in computational linguistics analysis and automation applications.\nIn this context, dealing with unknown words (words do not appear in the lexicon\nreferred as unknown words) is also an important task, since growing NLP systems\nare used in more and more new applications. One aid of predicting lexical\ncategories of unknown words is the use of syntactical knowledge of the\nlanguage. The distinction between open class words and closed class words\ntogether with syntactical features of the language used in this research to\npredict lexical categories of unknown words in the tagging process. An\nexperiment is performed to investigate the ability of the approach to parse\nunknown words using syntactical knowledge without human intervention. This\nexperiment shows that the performance of the tagging process is enhanced when\nword class distinction is used together with syntactic rules to parse sentences\ncontaining unknown words in Sinhala language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.01894v1",
            "date": "2015-01-08 16:12:34+00:00",
            "title": "Quantifying Scripts: Defining metrics of characters for quantitative and descriptive analysis",
            "authors": [
                "Vinodh Rajan"
            ],
            "abstract": "Analysis of scripts plays an important role in paleography and in\nquantitative linguistics. Especially in the field of digital paleography\nquantitative features are much needed to differentiate glyphs. We describe an\nelaborate set of metrics that quantify qualitative information contained in\ncharacters and hence indirectly also quantify the scribal features. We broadly\ndivide the metrics into several categories and describe each individual metric\nwith its underlying qualitative significance. The metrics are largely derived\nfrom the related area of gesture design and recognition. We also propose\nseveral novel metrics. The proposed metrics are soundly grounded on the\nprinciples of handwriting production and handwriting analysis. These computed\nmetrics could serve as descriptors for scripts and also be used for comparing\nand analyzing scripts. We illustrate some quantitative analysis based on the\nproposed metrics by applying it to the paleographic evolution of the medieval\nTamil script from Brahmi. We also outline future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.02670v1",
            "date": "2015-01-12 14:48:54+00:00",
            "title": "Navigating the Semantic Horizon using Relative Neighborhood Graphs",
            "authors": [
                "Amaru Cuba Gyllensten",
                "Magnus Sahlgren"
            ],
            "abstract": "This paper is concerned with nearest neighbor search in distributional\nsemantic models. A normal nearest neighbor search only returns a ranked list of\nneighbors, with no information about the structure or topology of the local\nneighborhood. This is a potentially serious shortcoming of the mode of querying\na distributional semantic model, since a ranked list of neighbors may conflate\nseveral different senses. We argue that the topology of neighborhoods in\nsemantic space provides important information about the different senses of\nterms, and that such topological structures can be used for word-sense\ninduction. We also argue that the topology of the neighborhoods in semantic\nspace can be used to determine the semantic horizon of a point, which we define\nas the set of neighbors that have a direct connection to the point. We\nintroduce relative neighborhood graphs as method to uncover the topological\nproperties of neighborhoods in semantic models. We also provide examples of\nrelative neighborhood graphs for three well-known semantic models; the PMI\nmodel, the GloVe model, and the skipgram model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.03191v1",
            "date": "2015-01-13 22:14:57+00:00",
            "title": "Annotating Cognates and Etymological Origin in Turkic Languages",
            "authors": [
                "Benjamin S. Mericli",
                "Michael Bloodgood"
            ],
            "abstract": "Turkic languages exhibit extensive and diverse etymological relationships\namong lexical items. These relationships make the Turkic languages promising\nfor exploring automated translation lexicon induction by leveraging cognate and\nother etymological information. However, due to the extent and diversity of the\ntypes of relationships between words, it is not clear how to annotate such\ninformation. In this paper, we present a methodology for annotating cognates\nand etymological origin in Turkic languages. Our method strives to balance the\namount of research effort the annotator expends with the utility of the\nannotations for supporting research on improving automated translation lexicon\ninduction.",
            "journal ref": "In Proceedings of the First Workshop on Language Resources and\n  Technologies for Turkic Languages at LREC'12, pages 47-51, Istanbul, Turkey,\n  May 2012. European Language Resources Association",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.04324v1",
            "date": "2015-01-18 16:37:53+00:00",
            "title": "Phrase Based Language Model For Statistical Machine Translation",
            "authors": [
                "Jia Xu",
                "Geliang Chen"
            ],
            "abstract": "We consider phrase based Language Models (LM), which generalize the commonly\nused word level models. Similar concept on phrase based LMs appears in speech\nrecognition, which is rather specialized and thus less suitable for machine\ntranslation (MT). In contrast to the dependency LM, we first introduce the\nexhaustive phrase-based LMs tailored for MT use. Preliminary experimental\nresults show that our approach outperform word based LMs with the respect to\nperplexity and translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.05203v3",
            "date": "2015-01-21 15:48:28+00:00",
            "title": "Phrase Based Language Model for Statistical Machine Translation: Empirical Study",
            "authors": [
                "Geliang Chen"
            ],
            "abstract": "Reordering is a challenge to machine translation (MT) systems. In MT, the\nwidely used approach is to apply word based language model (LM) which considers\nthe constituent units of a sentence as words. In speech recognition (SR), some\nphrase based LM have been proposed. However, those LMs are not necessarily\nsuitable or optimal for reordering. We propose two phrase based LMs which\nconsiders the constituent units of a sentence as phrases. Experiments show that\nour phrase based LMs outperform the word based LM with the respect of\nperplexity and n-best list re-ranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.07005v1",
            "date": "2015-01-28 07:04:18+00:00",
            "title": "Survey:Natural Language Parsing For Indian Languages",
            "authors": [
                "Monika T. Makwana",
                "Deepak C. Vegda"
            ],
            "abstract": "Syntactic parsing is a necessary task which is required for NLP applications\nincluding machine translation. It is a challenging task to develop a\nqualitative parser for morphological rich and agglutinative languages.\nSyntactic analysis is used to understand the grammatical structure of a natural\nlanguage sentence. It outputs all the grammatical information of each word and\nits constituent. Also issues related to it help us to understand the language\nin a more detailed way. This literature survey is groundwork to understand the\ndifferent parser development for Indian languages and various approaches that\nare used to develop such tools and techniques. This paper provides a survey of\nresearch papers from well known journals and conferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00030v1",
            "date": "2015-02-27 22:52:37+00:00",
            "title": "Parsing as Reduction",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "We reduce phrase-representation parsing to dependency parsing. Our reduction\nis grounded on a new intermediate representation, \"head-ordered dependency\ntrees\", shown to be isomorphic to constituent trees. By encoding order\ninformation in the dependency labels, we show that any off-the-shelf, trainable\ndependency parser can be used to produce constituents. When this parser is\nnon-projective, we can perform discontinuous parsing in a very natural manner.\nDespite the simplicity of our approach, experiments show that the resulting\nparsers are on par with strong baselines, such as the Berkeley parser for\nEnglish and the best single system in the SPMRL-2014 shared task. Results are\nparticularly striking for discontinuous parsing of German, where we surpass the\ncurrent state of the art by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00095v3",
            "date": "2015-02-28 07:59:59+00:00",
            "title": "Task-Oriented Learning of Word Embeddings for Semantic Relation Classification",
            "authors": [
                "Kazuma Hashimoto",
                "Pontus Stenetorp",
                "Makoto Miwa",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "We present a novel learning method for word embeddings designed for relation\nclassification. Our word embeddings are trained by predicting words between\nnoun pairs using lexical relation-specific features on a large unlabeled\ncorpus. This allows us to explicitly incorporate relation-specific information\ninto the word embeddings. The learned word embeddings are then used to\nconstruct feature vectors for a relation classification model. On a\nwell-established semantic relation classification task, our method\nsignificantly outperforms a baseline based on a previously introduced word\nembedding method, and compares favorably to previous state-of-the-art models\nthat use syntactic information or manually constructed external resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00168v1",
            "date": "2015-02-28 19:46:50+00:00",
            "title": "The NLP Engine: A Universal Turing Machine for NLP",
            "authors": [
                "Jiwei Li",
                "Eduard Hovy"
            ],
            "abstract": "It is commonly accepted that machine translation is a more complex task than\npart of speech tagging. But how much more complex? In this paper we make an\nattempt to develop a general framework and methodology for computing the\ninformational and/or processing complexity of NLP applications and tasks. We\ndefine a universal framework akin to a Turning Machine that attempts to fit\n(most) NLP tasks into one paradigm. We calculate the complexities of various\nNLP tasks using measures of Shannon Entropy, and compare `simple' ones such as\npart of speech tagging to `complex' ones such as machine translation. This\npaper provides a first, though far from perfect, attempt to quantify NLP tasks\nunder a uniform paradigm. We point out current deficiencies and suggest some\navenues for fruitful research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.01655v2",
            "date": "2015-03-05 15:08:21+00:00",
            "title": "Studying the Wikipedia Hyperlink Graph for Relatedness and Disambiguation",
            "authors": [
                "Eneko Agirre",
                "Ander Barrena",
                "Aitor Soroa"
            ],
            "abstract": "Hyperlinks and other relations in Wikipedia are a extraordinary resource\nwhich is still not fully understood. In this paper we study the different types\nof links in Wikipedia, and contrast the use of the full graph with respect to\njust direct links. We apply a well-known random walk algorithm on two tasks,\nword relatedness and named-entity disambiguation. We show that using the full\ngraph is more effective than just direct links by a large margin, that\nnon-reciprocal links harm performance, and that there is no benefit from\ncategories and infoboxes, with coherent results on both tasks. We set new\nstate-of-the-art figures for systems based on Wikipedia links, comparable to\nsystems exploiting several information sources and/or supervised machine\nlearning. Our approach is open source, with instruction to reproduce results,\nand amenable to be integrated with complementary text-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02335v1",
            "date": "2015-03-08 22:18:30+00:00",
            "title": "An Unsupervised Method for Uncovering Morphological Chains",
            "authors": [
                "Karthik Narasimhan",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "Most state-of-the-art systems today produce morphological analysis based only\non orthographic patterns. In contrast, we propose a model for unsupervised\nmorphological analysis that integrates orthographic and semantic views of\nwords. We model word formation in terms of morphological chains, from base\nwords to the observed words, breaking the chains into parent-child relations.\nWe use log-linear models with morpheme and word-level features to predict\npossible parents, including their modifications, for each word. The limited set\nof candidate parents for each word render contrastive estimation feasible. Our\nmodel consistently matches or outperforms five state-of-the-art systems on\nArabic, English and Turkish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.03535v2",
            "date": "2015-03-11 23:50:04+00:00",
            "title": "On Using Monolingual Corpora in Neural Machine Translation",
            "authors": [
                "Caglar Gulcehre",
                "Orhan Firat",
                "Kelvin Xu",
                "Kyunghyun Cho",
                "Loic Barrault",
                "Huei-Chi Lin",
                "Fethi Bougares",
                "Holger Schwenk",
                "Yoshua Bengio"
            ],
            "abstract": "Recent work on end-to-end neural network-based architectures for machine\ntranslation has shown promising results for En-Fr and En-De translation.\nArguably, one of the major factors behind this success has been the\navailability of high quality parallel corpora. In this work, we investigate how\nto leverage abundant monolingual corpora for neural machine translation.\nCompared to a phrase-based and hierarchical baseline, we obtain up to $1.96$\nBLEU improvement on the low-resource language pair Turkish-English, and $1.59$\nBLEU on the focused domain task of Chinese-English chat messages. While our\nmethod was initially targeted toward such tasks with less parallel data, we\nshow that it also extends to high resource languages such as Cs-En and De-En\nwhere we obtain an improvement of $0.39$ and $0.47$ BLEU scores over the neural\nmachine translation baselines, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.03989v1",
            "date": "2015-03-13 09:03:21+00:00",
            "title": "An implementation of Apertium based Assamese morphological analyzer",
            "authors": [
                "Mirzanur Rahman",
                "Shikhar Kumar Sarma"
            ],
            "abstract": "Morphological Analysis is an important branch of linguistics for any Natural\nLanguage Processing Technology. Morphology studies the word structure and\nformation of word of a language. In current scenario of NLP research,\nmorphological analysis techniques have become more popular day by day. For\nprocessing any language, morphology of the word should be first analyzed.\nAssamese language contains very complex morphological structure. In our work we\nhave used Apertium based Finite-State-Transducers for developing morphological\nanalyzer for Assamese Language with some limited domain and we get 72.7%\naccuracy",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.05034v2",
            "date": "2015-03-17 13:26:08+00:00",
            "title": "$gen$CNN: A Convolutional Architecture for Word Sequence Prediction",
            "authors": [
                "Mingxuan Wang",
                "Zhengdong Lu",
                "Hang Li",
                "Wenbin Jiang",
                "Qun Liu"
            ],
            "abstract": "We propose a novel convolutional architecture, named $gen$CNN, for word\nsequence prediction. Different from previous work on neural network-based\nlanguage modeling and generation (e.g., RNN or LSTM), we choose not to greedily\nsummarize the history of words as a fixed length vector. Instead, we use a\nconvolutional neural network to predict the next word with the history of words\nof variable length. Also different from the existing feedforward networks for\nlanguage modeling, our model can effectively fuse the local correlation and\nglobal correlation in the word sequence, with a convolution-gating strategy\nspecifically designed for the task. We argue that our model can give adequate\nrepresentation of the history, and therefore can naturally exploit both the\nshort and long range dependencies. Our model is fast, easy to train, and\nreadily parallelized. Our extensive experiments on text generation and $n$-best\nre-ranking in machine translation show that $gen$CNN outperforms the\nstate-of-the-arts with big margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.05123v1",
            "date": "2015-03-17 17:04:27+00:00",
            "title": "Prediction Using Note Text: Synthetic Feature Creation with word2vec",
            "authors": [
                "Manuel Amunategui",
                "Tristan Markwell",
                "Yelena Rozenfeld"
            ],
            "abstract": "word2vec affords a simple yet powerful approach of extracting quantitative\nvariables from unstructured textual data. Over half of healthcare data is\nunstructured and therefore hard to model without involved expertise in data\nengineering and natural language processing. word2vec can serve as a bridge to\nquickly gather intelligence from such data sources.\n  In this study, we ran 650 megabytes of unstructured, medical chart notes from\nthe Providence Health & Services electronic medical record through word2vec. We\nused two different approaches in creating predictive variables and tested them\non the risk of readmission for patients with COPD (Chronic Obstructive Lung\nDisease). As a comparative benchmark, we ran the same test using the LACE risk\nmodel (a single score based on length of stay, acuity, comorbid conditions, and\nemergency department visits).\n  Using only free text and mathematical might, we found word2vec comparable to\nLACE in predicting the risk of readmission of COPD patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.05626v1",
            "date": "2015-03-19 01:37:40+00:00",
            "title": "Phrase database Approach to structural and semantic disambiguation in English-Korean Machine Translation",
            "authors": [
                "Myong-Chol Pak"
            ],
            "abstract": "In machine translation it is common phenomenon that machine-readable\ndictionaries and standard parsing rules are not enough to ensure accuracy in\nparsing and translating English phrases into Korean language, which is revealed\nin misleading translation results due to consequent structural and semantic\nambiguities. This paper aims to suggest a solution to structural and semantic\nambiguities due to the idiomaticity and non-grammaticalness of phrases commonly\nused in English language by applying bilingual phrase database in\nEnglish-Korean Machine Translation (EKMT). This paper firstly clarifies what\nthe phrase unit in EKMT is based on the definition of the English phrase,\nsecondly clarifies what kind of language unit can be the target of the phrase\ndatabase for EKMT, thirdly suggests a way to build the phrase database by\npresenting the format of the phrase database with examples, and finally\ndiscusses briefly the method to apply this bilingual phrase database to the\nEKMT for structural and semantic disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.05907v1",
            "date": "2015-03-19 19:45:24+00:00",
            "title": "Syntagma Lexical Database",
            "authors": [
                "Daniel Christen"
            ],
            "abstract": "This paper discusses the structure of Syntagma's Lexical Database (focused on\nItalian). The basic database consists in four tables. Table Forms contains word\ninflections, used by the POS-tagger for the identification of input-words.\nForms is related to Lemma. Table Lemma stores all kinds of grammatical features\nof words, word-level semantic data and restrictions. In the table Meanings\nmeaning-related data are stored: definition, examples, domain, and semantic\ninformation. Table Valency contains the argument structure of each meaning,\nwith syntactic and semantic features for each argument. The extended version of\nSLD contains the links to Syntagma's Semantic Net and to the WordNet synsets of\nother languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.06151v1",
            "date": "2015-03-20 16:41:05+00:00",
            "title": "On measuring linguistic intelligence",
            "authors": [
                "Maxim Litvak"
            ],
            "abstract": "This work addresses the problem of measuring how many languages a person\n\"effectively\" speaks given that some of the languages are close to each other.\nIn other words, to assign a meaningful number to her language portfolio.\nIntuition says that someone who speaks fluently Spanish and Portuguese is\nlinguistically less proficient compared to someone who speaks fluently Spanish\nand Chinese since it takes more effort for a native Spanish speaker to learn\nChinese than Portuguese. As the number of languages grows and their proficiency\nlevels vary, it gets even more complicated to assign a score to a language\nportfolio. In this article we propose such a measure (\"linguistic quotient\" -\nLQ) that can account for these effects.\n  We define the properties that such a measure should have. They are based on\nthe idea of coherent risk measures from the mathematical finance. Having laid\ndown the foundation, we propose one such a measure together with the algorithm\nthat works on languages classification tree as input.\n  The algorithm together with the input is available online at lingvometer.com",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.06450v3",
            "date": "2015-03-22 18:05:08+00:00",
            "title": "Multilingual Open Relation Extraction Using Cross-lingual Projection",
            "authors": [
                "Manaal Faruqui",
                "Shankar Kumar"
            ],
            "abstract": "Open domain relation extraction systems identify relation and argument\nphrases in a sentence without relying on any underlying schema. However,\ncurrent state-of-the-art relation extraction systems are available only for\nEnglish because of their heavy reliance on linguistic tools such as\npart-of-speech taggers and dependency parsers. We present a cross-lingual\nannotation projection method for language independent relation extraction. We\nevaluate our method on a manually annotated test set and present results on\nthree typologically different languages. We release these manual annotations\nand extracted relations in 61 languages from Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.06733v2",
            "date": "2015-03-23 17:20:54+00:00",
            "title": "Yara Parser: A Fast and Accurate Dependency Parser",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Joel Tetreault"
            ],
            "abstract": "Dependency parsers are among the most crucial tools in natural language\nprocessing as they have many important applications in downstream tasks such as\ninformation retrieval, machine translation and knowledge acquisition. We\nintroduce the Yara Parser, a fast and accurate open-source dependency parser\nbased on the arc-eager algorithm and beam search. It achieves an unlabeled\naccuracy of 93.32 on the standard WSJ test set which ranks it among the top\ndependency parsers. At its fastest, Yara can parse about 4000 sentences per\nsecond when in greedy mode (1 beam). When optimizing for accuracy (using 64\nbeams and Brown cluster features), Yara can parse 45 sentences per second. The\nparser can be trained on any syntactic dependency treebank and different\noptions are provided in order to make it more flexible and tunable for specific\ntasks. It is released with the Apache version 2.0 license and can be used for\nboth commercial and academic purposes. The parser can be found at\nhttps://github.com/yahoo/YaraParser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.06760v1",
            "date": "2015-03-23 18:32:56+00:00",
            "title": "Unsupervised POS Induction with Word Embeddings",
            "authors": [
                "Chu-Cheng Lin",
                "Waleed Ammar",
                "Chris Dyer",
                "Lori Levin"
            ],
            "abstract": "Unsupervised word embeddings have been shown to be valuable as features in\nsupervised learning problems; however, their role in unsupervised problems has\nbeen less thoroughly explored. In this paper, we show that embeddings can\nlikewise add value to the problem of unsupervised POS induction. In two\nrepresentative models of POS induction, we replace multinomial distributions\nover the vocabulary with multivariate Gaussian distributions over word\nembeddings and observe consistent improvements in eight languages. We also\nanalyze the effect of various choices while inducing word embeddings on\n\"downstream\" POS induction results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.07283v1",
            "date": "2015-03-25 05:28:50+00:00",
            "title": "Morphological Analyzer and Generator for Russian and Ukrainian Languages",
            "authors": [
                "Mikhail Korobov"
            ],
            "abstract": "pymorphy2 is a morphological analyzer and generator for Russian and Ukrainian\nlanguages. It uses large efficiently encoded lexi- cons built from OpenCorpora\nand LanguageTool data. A set of linguistically motivated rules is developed to\nenable morphological analysis and generation of out-of-vocabulary words\nobserved in real-world documents. For Russian pymorphy2 provides\nstate-of-the-arts morphological analysis quality. The analyzer is implemented\nin Python programming language with optional C++ extensions. Emphasis is put on\nease of use, documentation and extensibility. The package is distributed under\na permissive open-source license, encouraging its use in both academic and\ncommercial setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.07613v1",
            "date": "2015-03-26 04:02:26+00:00",
            "title": "Unsupervised authorship attribution",
            "authors": [
                "David Fifield",
                "Torbj\u00f8rn Follan",
                "Emil Lunde"
            ],
            "abstract": "We describe a technique for attributing parts of a written text to a set of\nunknown authors. Nothing is assumed to be known a priori about the writing\nstyles of potential authors. We use multiple independent clusterings of an\ninput text to identify parts that are similar and dissimilar to one another. We\ndescribe algorithms necessary to combine the multiple clusterings into a\nmeaningful output. We show results of the application of the technique on texts\nhaving multiple writing styles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.08167v2",
            "date": "2015-03-27 17:57:00+00:00",
            "title": "Normalization of Non-Standard Words in Croatian Texts",
            "authors": [
                "Slobodan Beliga",
                "Miran Pobar",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "This paper presents text normalization which is an integral part of any\ntext-to-speech synthesis system. Text normalization is a set of methods with a\ntask to write non-standard words, like numbers, dates, times, abbreviations,\nacronyms and the most common symbols, in their full expanded form are\npresented. The whole taxonomy for classification of non-standard words in\nCroatian language together with rule-based normalization methods combined with\na lookup dictionary are proposed. Achieved token rate for normalization of\nCroatian texts is 95%, where 80% of expanded words are in correct morphological\nform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.09144v1",
            "date": "2015-03-31 17:56:07+00:00",
            "title": "Towards Using Machine Translation Techniques to Induce Multilingual Lexica of Discourse Markers",
            "authors": [
                "Ant\u00f3nio Lopes",
                "David Martins de Matos",
                "Vera Cabarr\u00e3o",
                "Ricardo Ribeiro",
                "Helena Moniz",
                "Isabel Trancoso",
                "Ana Isabel Mata"
            ],
            "abstract": "Discourse markers are universal linguistic events subject to language\nvariation. Although an extensive literature has already reported language\nspecific traits of these events, little has been said on their cross-language\nbehavior and on building an inventory of multilingual lexica of discourse\nmarkers. This work describes new methods and approaches for the description,\nclassification, and annotation of discourse markers in the specific domain of\nthe Europarl corpus. The study of discourse markers in the context of\ntranslation is crucial due to the idiomatic nature of these structures.\nMultilingual lexica together with the functional analysis of such structures\nare useful tools for the hard task of translating discourse markers into\npossible equivalents from one language to another. Using Daniel Marcu's\nvalidated discourse markers for English, extracted from the Brown Corpus, our\npurpose is to build multilingual lexica of discourse markers for other\nlanguages, based on machine translation techniques. The major assumption in\nthis study is that the usage of a discourse marker is independent of the\nlanguage, i.e., the rhetorical function of a discourse marker in a sentence in\none language is equivalent to the rhetorical function of the same discourse\nmarker in another language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.00133v1",
            "date": "2015-07-01 07:29:12+00:00",
            "title": "Prior Polarity Lexical Resources for the Italian Language",
            "authors": [
                "Valeria Borz\u00ec",
                "Simone Faro",
                "Arianna Pavone",
                "Sabrina Sansone"
            ],
            "abstract": "In this paper we present SABRINA (Sentiment Analysis: a Broad Resource for\nItalian Natural language Applications) a manually annotated prior polarity\nlexical resource for Italian natural language applications in the field of\nopinion mining and sentiment induction. The resource consists in two different\nsets, an Italian dictionary of more than 277.000 words tagged with their prior\npolarity value, and a set of polarity modifiers, containing more than 200\nwords, which can be used in combination with non neutral terms of the\ndictionary in order to induce the sentiment of Italian compound terms. To the\nbest of our knowledge this is the first prior polarity manually annotated\nresource which has been developed for the Italian natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.00639v1",
            "date": "2015-07-02 15:58:25+00:00",
            "title": "Simple, Fast Semantic Parsing with a Tensor Kernel",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "We describe a simple approach to semantic parsing based on a tensor product\nkernel. We extract two feature vectors: one for the query and one for each\ncandidate logical form. We then train a classifier using the tensor product of\nthe two vectors. Using very simple features for both, our system achieves an\naverage F1 score of 40.1% on the WebQuestions dataset. This is comparable to\nmore complex systems but is simpler to implement and runs faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.01127v1",
            "date": "2015-07-04 16:59:30+00:00",
            "title": "AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes",
            "authors": [
                "Sascha Rothe",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present \\textit{AutoExtend}, a system to learn embeddings for synsets and\nlexemes. It is flexible in that it can take any word embeddings as input and\ndoes not need an additional training corpus. The synset/lexeme embeddings\nobtained live in the same vector space as the word embeddings. A sparse tensor\nformalization guarantees efficiency and parallelizability. We use WordNet as a\nlexical resource, but AutoExtend can be easily applied to other resources like\nFreebase. AutoExtend achieves state-of-the-art performance on word similarity\nand word sense disambiguation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.01529v1",
            "date": "2015-07-06 16:32:52+00:00",
            "title": "Correspondence Factor Analysis of Big Data Sets: A Case Study of 30 Million Words; and Contrasting Analytics using Apache Solr and Correspondence Analysis in R",
            "authors": [
                "Fionn Murtagh"
            ],
            "abstract": "We consider a large number of text data sets. These are cooking recipes. Term\ndistribution and other distributional properties of the data are investigated.\nOur aim is to look at various analytical approaches which allow for mining of\ninformation on both high and low detail scales. Metric space embedding is\nfundamental to our interest in the semantic properties of this data. We\nconsider the projection of all data into analyses of aggregated versions of the\ndata. We contrast that with projection of aggregated versions of the data into\nanalyses of all the data. Analogously for the term set, we look at analysis of\nselected terms. We also look at inherent term associations such as between\nsingular and plural. In addition to our use of Correspondence Analysis in R,\nfor latent semantic space mapping, we also use Apache Solr. Setting up the Solr\nserver and carrying out querying is described. A further novelty is that\nquerying is supported in Solr based on the principal factor plane mapping of\nall the data. This uses a bounding box query, based on factor projections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.01636v1",
            "date": "2015-07-06 22:25:55+00:00",
            "title": "Reflections on Sentiment/Opinion Analysis",
            "authors": [
                "Jiwei Li",
                "Eduard Hovy"
            ],
            "abstract": "In this paper, we described possible directions for deeper understanding,\nhelping bridge the gap between psychology / cognitive science and computational\napproaches in sentiment/opinion analysis literature. We focus on the opinion\nholder's underlying needs and their resultant goals, which, in a utilitarian\nmodel of sentiment, provides the basis for explaining the reason a sentiment\nvalence is held. While these thoughts are still immature, scattered,\nunstructured, and even imaginary, we believe that these perspectives might\nsuggest fruitful avenues for various kinds of future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.01701v1",
            "date": "2015-07-07 08:23:31+00:00",
            "title": "A Survey and Classification of Controlled Natural Languages",
            "authors": [
                "Tobias Kuhn"
            ],
            "abstract": "What is here called controlled natural language (CNL) has traditionally been\ngiven many different names. Especially during the last four decades, a wide\nvariety of such languages have been designed. They are applied to improve\ncommunication among humans, to improve translation, or to provide natural and\nintuitive representations for formal notations. Despite the apparent\ndifferences, it seems sensible to put all these languages under the same\numbrella. To bring order to the variety of languages, a general classification\nscheme is presented here. A comprehensive survey of existing English-based CNLs\nis given, listing and describing 100 languages from 1930 until today.\nClassification of these languages reveals that they form a single scattered\ncloud filling the conceptual space between natural languages such as English on\nthe one end and formal languages such as propositional logic on the other. The\ngoal of this article is to provide a common terminology and a common model for\nCNL, to contribute to the understanding of their general nature, to provide a\nstarting point for researchers interested in the area, and to help developers\nto make design decisions.",
            "journal ref": "Computational Linguistics, March 2014, Vol. 40, No. 1, pages\n  121-170",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02012v1",
            "date": "2015-07-08 03:50:47+00:00",
            "title": "Hindi to English Transfer Based Machine Translation System",
            "authors": [
                "Akanksha Gehlot",
                "Vaishali Sharma",
                "Shashi Pal Singh",
                "Ajai Kumar"
            ],
            "abstract": "In large societies like India there is a huge demand to convert one human\nlanguage into another. Lots of work has been done in this area. Many transfer\nbased MTS have developed for English to other languages, as MANTRA CDAC Pune,\nMATRA CDAC Pune, SHAKTI IISc Bangalore and IIIT Hyderabad. Still there is a\nlittle work done for Hindi to other languages. Currently we are working on it.\nIn this paper we focus on designing a system, that translate the document from\nHindi to English by using transfer based approach. This system takes an input\ntext check its structure through parsing. Reordering rules are used to generate\nthe text in target language. It is better than Corpus Based MTS because Corpus\nBased MTS require large amount of word aligned data for translation that is not\navailable for many languages while Transfer Based MTS requires only knowledge\nof both the languages(source language and target language) to make transfer\nrules. We get correct translation for simple assertive sentences and almost\ncorrect for complex and compound sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02045v2",
            "date": "2015-07-08 06:52:50+00:00",
            "title": "What Your Username Says About You",
            "authors": [
                "Aaron Jaech",
                "Mari Ostendorf"
            ],
            "abstract": "Usernames are ubiquitous on the Internet, and they are often suggestive of\nuser demographics. This work looks at the degree to which gender and language\ncan be inferred from a username alone by making use of unsupervised morphology\ninduction to decompose usernames into sub-units. Experimental results on the\ntwo tasks demonstrate the effectiveness of the proposed morphological features\ncompared to a character n-gram baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02062v1",
            "date": "2015-07-08 08:26:23+00:00",
            "title": "Multi-Document Summarization via Discriminative Summary Reranking",
            "authors": [
                "Xiaojun Wan",
                "Ziqiang Cao",
                "Furu Wei",
                "Sujian Li",
                "Ming Zhou"
            ],
            "abstract": "Existing multi-document summarization systems usually rely on a specific\nsummarization model (i.e., a summarization method with a specific parameter\nsetting) to extract summaries for different document sets with different\ntopics. However, according to our quantitative analysis, none of the existing\nsummarization models can always produce high-quality summaries for different\ndocument sets, and even a summarization model with good overall performance may\nproduce low-quality summaries for some document sets. On the contrary, a\nbaseline summarization model may produce high-quality summaries for some\ndocument sets. Based on the above observations, we treat the summaries produced\nby different summarization models as candidate summaries, and then explore\ndiscriminative reranking techniques to identify high-quality summaries from the\ncandidates for difference document sets. We propose to extract a set of\ncandidate summaries for each document set based on an ILP framework, and then\nleverage Ranking SVM for summary reranking. Various useful features have been\ndeveloped for the reranking process, including word-level features,\nsentence-level features and summary-level features. Evaluation results on the\nbenchmark DUC datasets validate the efficacy and robustness of our proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02145v2",
            "date": "2015-07-08 13:27:43+00:00",
            "title": "Learning to Mine Chinese Coordinate Terms Using the Web",
            "authors": [
                "Xiaojiang Huang",
                "Xiaojun Wan",
                "Jianguo Xiao"
            ],
            "abstract": "Coordinate relation refers to the relation between instances of a concept and\nthe relation between the directly hyponyms of a concept. In this paper, we\nfocus on the task of extracting terms which are coordinate with a user given\nseed term in Chinese, and grouping the terms which belong to different concepts\nif the seed term has several meanings. We propose a semi-supervised method that\nintegrates manually defined linguistic patterns and automatically learned\nsemi-structural patterns to extract coordinate terms in Chinese from web search\nresults. In addition, terms are grouped into different concepts based on their\nco-occurring terms and contexts. We further calculate the saliency scores of\nextracted terms and rank them accordingly. Experimental results demonstrate\nthat our proposed method generates results with high quality and wide coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02628v1",
            "date": "2015-07-09 18:11:03+00:00",
            "title": "FAQ-based Question Answering via Word Alignment",
            "authors": [
                "Zhiguo Wang",
                "Abraham Ittycheriah"
            ],
            "abstract": "In this paper, we propose a novel word-alignment-based method to solve the\nFAQ-based question answering task. First, we employ a neural network model to\ncalculate question similarity, where the word alignment between two questions\nis used for extracting features. Second, we design a bootstrap-based feature\nextraction method to extract a small set of effective lexical features. Third,\nwe propose a learning-to-rank algorithm to train parameters more suitable for\nthe ranking tasks. Experimental results, conducted on three languages (English,\nSpanish and Japanese), demonstrate that the question similarity model is more\neffective than baseline systems, the sparse features bring 5% improvements on\ntop-1 accuracy, and the learning-to-rank algorithm works significantly better\nthan the traditional method. We further evaluate our method on the answer\nsentence selection task. Our method outperforms all the previous systems on the\nstandard TREC data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.03223v1",
            "date": "2015-07-12 12:14:19+00:00",
            "title": "Classifier-Based Text Simplification for Improved Machine Translation",
            "authors": [
                "Shruti Tyagi",
                "Deepti Chopra",
                "Iti Mathur",
                "Nisheeth Joshi"
            ],
            "abstract": "Machine Translation is one of the research fields of Computational\nLinguistics. The objective of many MT Researchers is to develop an MT System\nthat produce good quality and high accuracy output translations and which also\ncovers maximum language pairs. As internet and Globalization is increasing day\nby day, we need a way that improves the quality of translation. For this\nreason, we have developed a Classifier based Text Simplification Model for\nEnglish-Hindi Machine Translation Systems. We have used support vector machines\nand Na\\\"ive Bayes Classifier to develop this model. We have also evaluated the\nperformance of these classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.03462v1",
            "date": "2015-07-13 14:00:22+00:00",
            "title": "Supervised Hierarchical Classification for Student Answer Scoring",
            "authors": [
                "Itziar Aldabe",
                "Oier Lopez de Lacalle",
                "I\u00f1igo Lopez-Gazpio",
                "Montse Maritxalar"
            ],
            "abstract": "This paper describes a hierarchical system that predicts one label at a time\nfor automated student response analysis. For the task, we build a\nclassification binary tree that delays more easily confused labels to later\nstages using hierarchical processes. In particular, the paper describes how the\nhierarchical classifier has been built and how the classification task has been\nbroken down into binary subtasks. It finally discusses the motivations and\nfundamentals of such an approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.03471v1",
            "date": "2015-07-13 14:27:16+00:00",
            "title": "Incremental LSTM-based Dialog State Tracker",
            "authors": [
                "Lukas Zilka",
                "Filip Jurcicek"
            ],
            "abstract": "A dialog state tracker is an important component in modern spoken dialog\nsystems. We present an incremental dialog state tracker, based on LSTM\nnetworks. It directly uses automatic speech recognition hypotheses to track the\nstate. We also present the key non-standard aspects of the model that bring its\nperformance close to the state-of-the-art and experimentally analyze their\ncontribution: including the ASR confidence scores, abstracting scarcely\nrepresented values, including transcriptions in the training data, and model\naveraging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.03934v2",
            "date": "2015-07-14 17:18:30+00:00",
            "title": "Recurrent Polynomial Network for Dialogue State Tracking",
            "authors": [
                "Kai Sun",
                "Qizhe Xie",
                "Kai Yu"
            ],
            "abstract": "Dialogue state tracking (DST) is a process to estimate the distribution of\nthe dialogue states as a dialogue progresses. Recent studies on constrained\nMarkov Bayesian polynomial (CMBP) framework take the first step towards\nbridging the gap between rule-based and statistical approaches for DST. In this\npaper, the gap is further bridged by a novel framework -- recurrent polynomial\nnetwork (RPN). RPN's unique structure enables the framework to have all the\nadvantages of CMBP including efficiency, portability and interpretability.\nAdditionally, RPN achieves more properties of statistical approaches than CMBP.\nRPN was evaluated on the data corpora of the second and the third Dialog State\nTracking Challenge (DSTC-2/3). Experiments showed that RPN can significantly\noutperform both traditional rule-based approaches and statistical approaches\nwith similar feature set. Compared with the state-of-the-art statistical DST\napproaches with a lot richer features, RPN is also competitive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04214v1",
            "date": "2015-07-15 13:40:28+00:00",
            "title": "Associative Measures and Multi-word Unit Extraction in Turkish",
            "authors": [
                "Umit Mersinli"
            ],
            "abstract": "Associative measures are \"mathematical formulas determining the strength of\nassociation between two or more words based on their occurrences and\ncooccurrences in a text corpus\" (Pecina, 2010, p. 138). The purpose of this\npaper is to test the 12 associative measures that Text-NSP (Banerjee &\nPedersen, 2003) contains on a 10-million-word subcorpus of Turkish National\nCorpus (TNC) (Aksan et.al., 2012). A statistical comparison of those measures\nis out of the scope of the study, and the measures will be evaluated according\nto the linguistic relevance of the rankings they provide. The focus of the\nstudy is basically on optimizing the corpus data, before applying the measures\nand then, evaluating the rankings produced by these measures as a whole, not on\nthe linguistic relevance of individual n-grams. The findings include\nintra-linguistically relevant associative measures for a comma delimited,\nsentence splitted, lower-cased, well-balanced, representative, 10-million-word\ncorpus of Turkish.",
            "journal ref": "Associative Measures and Multi-word Unit Extraction in Turkish.\n  Dil ve Edebiyat Dergisi. 12(1). 43-61",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.05523v1",
            "date": "2015-07-20 15:07:53+00:00",
            "title": "How to Generate a Good Word Embedding?",
            "authors": [
                "Siwei Lai",
                "Kang Liu",
                "Liheng Xu",
                "Jun Zhao"
            ],
            "abstract": "We analyze three critical components of word embedding training: the model,\nthe corpus, and the training parameters. We systematize existing\nneural-network-based word embedding algorithms and compare them using the same\ncorpus. We evaluate each word embedding in three ways: analyzing its semantic\nproperties, using it as a feature for supervised tasks and using it to\ninitialize neural networks. We also provide several simple guidelines for\ntraining word embeddings. First, we discover that corpus domain is more\nimportant than corpus size. We recommend choosing a corpus in a suitable domain\nfor the desired task, after that, using a larger corpus yields better results.\nSecond, we find that faster models provide sufficient performance in most\ncases, and more complex models can be used if the training corpus is\nsufficiently large. Third, the early stopping metric for iterating should rely\non the development set of the desired task rather than the validation loss of\ntraining embedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.05630v1",
            "date": "2015-07-20 20:01:44+00:00",
            "title": "Notes About a More Aware Dependency Parser",
            "authors": [
                "Matteo Grella"
            ],
            "abstract": "In this paper I explain the reasons that led me to research and conceive a\nnovel technology for dependency parsing, mixing together the strengths of\ndata-driven transition-based and constraint-based approaches. In particular I\nhighlight the problem to infer the reliability of the results of a data-driven\ntransition-based parser, which is extremely important for high-level processes\nthat expect to use correct parsing results. I then briefly introduce a number\nof notes about a new parser model I'm working on, capable to proceed with the\nanalysis in a \"more aware\" way, with a more \"robust\" concept of robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06073v2",
            "date": "2015-07-22 06:54:09+00:00",
            "title": "Discriminative Segmental Cascades for Feature-Rich Phone Recognition",
            "authors": [
                "Hao Tang",
                "Weiran Wang",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "Discriminative segmental models, such as segmental conditional random fields\n(SCRFs) and segmental structured support vector machines (SSVMs), have had\nsuccess in speech recognition via both lattice rescoring and first-pass\ndecoding. However, such models suffer from slow decoding, hampering the use of\ncomputationally expensive features, such as segment neural networks or other\nhigh-order features. A typical solution is to use approximate decoding, either\nby beam pruning in a single pass or by beam pruning to generate a lattice\nfollowed by a second pass. In this work, we study discriminative segmental\nmodels trained with a hinge loss (i.e., segmental structured SVMs). We show\nthat beam search is not suitable for learning rescoring models in this\napproach, though it gives good approximate decoding performance when the model\nis already well-trained. Instead, we consider an approach inspired by\nstructured prediction cascades, which use max-marginal pruning to generate\nlattices. We obtain a high-accuracy phonetic recognition system with several\nexpensive feature types: a segment neural network, a second-order language\nmodel, and second-order phone boundary features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.07636v1",
            "date": "2015-07-28 03:51:43+00:00",
            "title": "Reasoning about Linguistic Regularities in Word Embeddings using Matrix Manifolds",
            "authors": [
                "Sridhar Mahadevan",
                "Sarath Chandar"
            ],
            "abstract": "Recent work has explored methods for learning continuous vector space word\nrepresentations reflecting the underlying semantics of words. Simple vector\nspace arithmetic using cosine distances has been shown to capture certain types\nof analogies, such as reasoning about plurals from singulars, past tense from\npresent tense, etc. In this paper, we introduce a new approach to capture\nanalogies in continuous word representations, based on modeling not just\nindividual word vectors, but rather the subspaces spanned by groups of words.\nWe exploit the property that the set of subspaces in n-dimensional Euclidean\nspace form a curved manifold space called the Grassmannian, a quotient subgroup\nof the Lie group of rotations in n- dimensions. Based on this mathematical\nmodel, we develop a modified cosine distance model based on geodesic kernels\nthat captures relation-specific distances across word categories. Our\nexperiments on analogy tasks show that our approach performs significantly\nbetter than the previous approaches for the given task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.07826v1",
            "date": "2015-07-28 15:59:39+00:00",
            "title": "Classifying informative and imaginative prose using complex networks",
            "authors": [
                "Henrique F. de Arruda",
                "Luciano da F. Costa",
                "Diego R. Amancio"
            ],
            "abstract": "Statistical methods have been widely employed in recent years to grasp many\nlanguage properties. The application of such techniques have allowed an\nimprovement of several linguistic applications, which encompasses machine\ntranslation, automatic summarization and document classification. In the\nlatter, many approaches have emphasized the semantical content of texts, as it\nis the case of bag-of-word language models. This approach has certainly yielded\nreasonable performance. However, some potential features such as the structural\norganization of texts have been used only on a few studies. In this context, we\nprobe how features derived from textual structure analysis can be effectively\nemployed in a classification task. More specifically, we performed a supervised\nclassification aiming at discriminating informative from imaginative documents.\nUsing a networked model that describes the local topological/dynamical\nproperties of function words, we achieved an accuracy rate of up to 95%, which\nis much higher than similar networked approaches. A systematic analysis of\nfeature relevance revealed that symmetry and accessibility measurements are\namong the most prominent network measurements. Our results suggest that these\nmeasurements could be used in related language applications, as they play a\ncomplementary role in characterizing texts.",
            "journal ref": "Europhysics Letters (EPL) 113 (2016) 28007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.08449v2",
            "date": "2015-07-30 10:53:11+00:00",
            "title": "One model, two languages: training bilingual parsers with harmonized treebanks",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Miguel A. Alonso"
            ],
            "abstract": "We introduce an approach to train lexicalized parsers using bilingual corpora\nobtained by merging harmonized treebanks of different languages, producing\nparsers that can analyze sentences in either of the learned languages, or even\nsentences that mix both. We test the approach on the Universal Dependency\nTreebanks, training with MaltParser and MaltOptimizer. The results show that\nthese bilingual parsers are more than competitive, as most combinations not\nonly preserve accuracy, but some even achieve significant improvements over the\ncorresponding monolingual parsers. Preliminary experiments also show the\napproach to be promising on texts with code-switching and when more languages\nare added.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.08452v3",
            "date": "2015-07-30 11:05:01+00:00",
            "title": "Unsupervised Sentence Simplification Using Deep Semantics",
            "authors": [
                "Shashi Narayan",
                "Claire Gardent"
            ],
            "abstract": "We present a novel approach to sentence simplification which departs from\nprevious work in two main ways. First, it requires neither hand written rules\nnor a training corpus of aligned standard and simplified sentences. Second,\nsentence splitting operates on deep semantic structure. We show (i) that the\nunsupervised framework we propose is competitive with four state-of-the-art\nsupervised systems and (ii) that our semantic based approach allows for a\nprincipled and effective handling of sentence splitting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.08539v1",
            "date": "2015-07-30 15:11:00+00:00",
            "title": "Multilayer Network of Language: a Unified Framework for Structural Analysis of Linguistic Subsystems",
            "authors": [
                "Domagoj Margan",
                "Ana Me\u0161trovi\u0107",
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107"
            ],
            "abstract": "Recently, the focus of complex networks research has shifted from the\nanalysis of isolated properties of a system toward a more realistic modeling of\nmultiple phenomena - multilayer networks. Motivated by the prosperity of\nmultilayer approach in social, transport or trade systems, we propose the\nintroduction of multilayer networks for language. The multilayer network of\nlanguage is a unified framework for modeling linguistic subsystems and their\nstructural properties enabling the exploration of their mutual interactions.\nVarious aspects of natural language systems can be represented as complex\nnetworks, whose vertices depict linguistic units, while links model their\nrelations. The multilayer network of language is defined by three aspects: the\nnetwork construction principle, the linguistic subsystem and the language of\ninterest. More precisely, we construct a word-level (syntax, co-occurrence and\nits shuffled counterpart) and a subword level (syllables and graphemes) network\nlayers, from five variations of original text (in the modeled language). The\nobtained results suggest that there are substantial differences between the\nnetworks structures of different language subsystems, which are hidden during\nthe exploration of an isolated layer. The word-level layers share structural\nproperties regardless of the language (e.g. Croatian or English), while the\nsyllabic subword level expresses more language dependent structural properties.\nThe preserved weighted overlap quantifies the similarity of word-level layers\nin weighted and directed networks. Moreover, the analysis of motifs reveals a\nclose topological structure of the syntactic and syllabic layers for both\nlanguages. The findings corroborate that the multilayer network framework is a\npowerful, consistent and systematic approach to model several linguistic\nsubsystems simultaneously and hence to provide a more unified view on language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00103v2",
            "date": "2015-12-01 00:23:44+00:00",
            "title": "Multilingual Language Processing From Bytes",
            "authors": [
                "Dan Gillick",
                "Cliff Brunk",
                "Oriol Vinyals",
                "Amarnag Subramanya"
            ],
            "abstract": "We describe an LSTM-based model which we call Byte-to-Span (BTS) that reads\ntext as bytes and outputs span annotations of the form [start, length, label]\nwhere start positions, lengths, and labels are separate entries in our\nvocabulary. Because we operate directly on unicode bytes rather than\nlanguage-specific words or characters, we can analyze text in many languages\nwith a single model. Due to the small vocabulary size, these multilingual\nmodels are very compact, but produce results similar to or better than the\nstate-of- the-art in Part-of-Speech tagging and Named Entity Recognition that\nuse only the provided training datasets (no external data sources). Our models\nare learning \"from scratch\" in that they do not rely on any elements of the\nstandard pipeline in Natural Language Processing (including tokenization), and\nthus can run in standalone fashion on raw text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00170v1",
            "date": "2015-12-01 08:10:49+00:00",
            "title": "Augmenting Phrase Table by Employing Lexicons for Pivot-based SMT",
            "authors": [
                "Yiming Cui",
                "Conghui Zhu",
                "Xiaoning Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Pivot language is employed as a way to solve the data sparseness problem in\nmachine translation, especially when the data for a particular language pair\ndoes not exist. The combination of source-to-pivot and pivot-to-target\ntranslation models can induce a new translation model through the pivot\nlanguage. However, the errors in two models may compound as noise, and still,\nthe combined model may suffer from a serious phrase sparsity problem. In this\npaper, we directly employ the word lexical model in IBM models as an additional\nresource to augment pivot phrase table. In addition, we also propose a phrase\ntable pruning method which takes into account both of the source and target\nphrasal coverage. Experimental result shows that our pruning method\nsignificantly outperforms the conventional one, which only considers source\nside phrasal coverage. Furthermore, by including the entries in the lexicon\nmodel, the phrase coverage increased, and we achieved improved results in\nChinese-to-Japanese translation using English as pivot language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00531v5",
            "date": "2015-12-02 00:34:51+00:00",
            "title": "Benchmarking sentiment analysis methods for large-scale texts: A case for using continuum-scored words and word shift graphs",
            "authors": [
                "Andrew J. Reagan",
                "Brian Tivnan",
                "Jake Ryland Williams",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "The emergence and global adoption of social media has rendered possible the\nreal-time estimation of population-scale sentiment, bearing profound\nimplications for our understanding of human behavior. Given the growing\nassortment of sentiment measuring instruments, comparisons between them are\nevidently required. Here, we perform detailed tests of 6 dictionary-based\nmethods applied to 4 different corpora, and briefly examine a further 20\nmethods. We show that a dictionary-based method will only perform both reliably\nand meaningfully if (1) the dictionary covers a sufficiently large enough\nportion of a given text's lexicon when weighted by word usage frequency; and\n(2) words are scored on a continuous scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00728v1",
            "date": "2015-12-02 15:09:31+00:00",
            "title": "Annotating Character Relationships in Literary Texts",
            "authors": [
                "Philip Massey",
                "Patrick Xia",
                "David Bamman",
                "Noah A. Smith"
            ],
            "abstract": "We present a dataset of manually annotated relationships between characters\nin literary texts, in order to support the training and evaluation of automatic\nmethods for relation type prediction in this domain (Makazhanov et al., 2014;\nKokkinakis, 2013) and the broader computational analysis of literary character\n(Elson et al., 2010; Bamman et al., 2014; Vala et al., 2015; Flekova and\nGurevych, 2015). In this work, we solicit annotations from workers on Amazon\nMechanical Turk for 109 texts ranging from Homer's _Iliad_ to Joyce's _Ulysses_\non four dimensions of interest: for a given pair of characters, we collect\njudgments as to the coarse-grained category (professional, social, familial),\nfine-grained category (friend, lover, parent, rival, employer), and affinity\n(positive, negative, neutral) that describes their primary relationship in a\ntext. We do not assume that this relationship is static; we also collect\njudgments as to whether it changes at any point in the course of the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01100v2",
            "date": "2015-12-03 14:54:39+00:00",
            "title": "Effective LSTMs for Target-Dependent Sentiment Classification",
            "authors": [
                "Duyu Tang",
                "Bing Qin",
                "Xiaocheng Feng",
                "Ting Liu"
            ],
            "abstract": "Target-dependent sentiment classification remains a challenge: modeling the\nsemantic relatedness of a target with its context words in a sentence.\nDifferent context words have different influences on determining the sentiment\npolarity of a sentence towards the target. Therefore, it is desirable to\nintegrate the connections between target word and context words when building a\nlearning system. In this paper, we develop two target dependent long short-term\nmemory (LSTM) models, where target information is automatically taken into\naccount. We evaluate our methods on a benchmark dataset from Twitter. Empirical\nresults show that modeling sentence representation with standard LSTM does not\nperform well. Incorporating target information into LSTM can significantly\nboost the classification accuracy. The target-dependent LSTM models achieve\nstate-of-the-art performances without using syntactic parser or external\nsentiment lexicons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01337v4",
            "date": "2015-12-04 08:31:02+00:00",
            "title": "Neural Generative Question Answering",
            "authors": [
                "Jun Yin",
                "Xin Jiang",
                "Zhengdong Lu",
                "Lifeng Shang",
                "Hang Li",
                "Xiaoming Li"
            ],
            "abstract": "This paper presents an end-to-end neural network model, named Neural\nGenerative Question Answering (GENQA), that can generate answers to simple\nfactoid questions, based on the facts in a knowledge-base. More specifically,\nthe model is built on the encoder-decoder framework for sequence-to-sequence\nlearning, while equipped with the ability to enquire the knowledge-base, and is\ntrained on a corpus of question-answer pairs, with their associated triples in\nthe knowledge-base. Empirical study shows the proposed model can effectively\ndeal with the variations of questions and answers, and generate right and\nnatural answers by referring to the facts in the knowledge-base. The experiment\non question answering demonstrates that the proposed model can outperform an\nembedding-based QA model as well as a neural dialogue model trained on the same\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01409v1",
            "date": "2015-12-04 14:01:32+00:00",
            "title": "What Makes it Difficult to Understand a Scientific Literature?",
            "authors": [
                "Mengyun Cao",
                "Jiao Tian",
                "Dezhi Cheng",
                "Jin Liu",
                "Xiaoping Sun"
            ],
            "abstract": "In the artificial intelligence area, one of the ultimate goals is to make\ncomputers understand human language and offer assistance. In order to achieve\nthis ideal, researchers of computer science have put forward a lot of models\nand algorithms attempting at enabling the machine to analyze and process human\nnatural language on different levels of semantics. Although recent progress in\nthis field offers much hope, we still have to ask whether current research can\nprovide assistance that people really desire in reading and comprehension. To\nthis end, we conducted a reading comprehension test on two scientific papers\nwhich are written in different styles. We use the semantic link models to\nanalyze the understanding obstacles that people will face in the process of\nreading and figure out what makes it difficult for human to understand a\nscientific literature. Through such analysis, we summarized some\ncharacteristics and problems which are reflected by people with different\nlevels of knowledge on the comprehension of difficult science and technology\nliterature, which can be modeled in semantic link network. We believe that\nthese characteristics and problems will help us re-examine the existing machine\nmodels and are helpful in the designing of new one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01768v1",
            "date": "2015-12-06 10:31:12+00:00",
            "title": "Want Answers? A Reddit Inspired Study on How to Pose Questions",
            "authors": [
                "Danish",
                "Yogesh Dahiya",
                "Partha Talukdar"
            ],
            "abstract": "Questions form an integral part of our everyday communication, both offline\nand online. Getting responses to our questions from others is fundamental to\nsatisfying our information need and in extending our knowledge boundaries. A\nquestion may be represented using various factors such as social, syntactic,\nsemantic, etc. We hypothesize that these factors contribute with varying\ndegrees towards getting responses from others for a given question. We perform\na thorough empirical study to measure effects of these factors using a novel\nquestion and answer dataset from the website Reddit.com. To the best of our\nknowledge, this is the first such analysis of its kind on this important topic.\nWe also use a sparse nonnegative matrix factorization technique to\nautomatically induce interpretable semantic factors from the question dataset.\nWe also document various patterns on response prediction we observe during our\nanalysis in the data. For instance, we found that preference-probing questions\nare scantily answered. Our method is robust to capture such latent response\nfactors. We hope to make our code and datasets publicly available upon\npublication of the paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.02433v3",
            "date": "2015-12-08 12:42:00+00:00",
            "title": "Minimum Risk Training for Neural Machine Translation",
            "authors": [
                "Shiqi Shen",
                "Yong Cheng",
                "Zhongjun He",
                "Wei He",
                "Hua Wu",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "We propose minimum risk training for end-to-end neural machine translation.\nUnlike conventional maximum likelihood estimation, minimum risk training is\ncapable of optimizing model parameters directly with respect to arbitrary\nevaluation metrics, which are not necessarily differentiable. Experiments show\nthat our approach achieves significant improvements over maximum likelihood\nestimation on a state-of-the-art neural machine translation system across\nvarious languages pairs. Transparent to architectures, our approach can be\napplied to more neural networks and potentially benefit more NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.02595v1",
            "date": "2015-12-08 19:13:50+00:00",
            "title": "Deep Speech 2: End-to-End Speech Recognition in English and Mandarin",
            "authors": [
                "Dario Amodei",
                "Rishita Anubhai",
                "Eric Battenberg",
                "Carl Case",
                "Jared Casper",
                "Bryan Catanzaro",
                "Jingdong Chen",
                "Mike Chrzanowski",
                "Adam Coates",
                "Greg Diamos",
                "Erich Elsen",
                "Jesse Engel",
                "Linxi Fan",
                "Christopher Fougner",
                "Tony Han",
                "Awni Hannun",
                "Billy Jun",
                "Patrick LeGresley",
                "Libby Lin",
                "Sharan Narang",
                "Andrew Ng",
                "Sherjil Ozair",
                "Ryan Prenger",
                "Jonathan Raiman",
                "Sanjeev Satheesh",
                "David Seetapun",
                "Shubho Sengupta",
                "Yi Wang",
                "Zhiqian Wang",
                "Chong Wang",
                "Bo Xiao",
                "Dani Yogatama",
                "Jun Zhan",
                "Zhenyao Zhu"
            ],
            "abstract": "We show that an end-to-end deep learning approach can be used to recognize\neither English or Mandarin Chinese speech--two vastly different languages.\nBecause it replaces entire pipelines of hand-engineered components with neural\nnetworks, end-to-end learning allows us to handle a diverse variety of speech\nincluding noisy environments, accents and different languages. Key to our\napproach is our application of HPC techniques, resulting in a 7x speedup over\nour previous system. Because of this efficiency, experiments that previously\ntook weeks now run in days. This enables us to iterate more quickly to identify\nsuperior architectures and algorithms. As a result, in several cases, our\nsystem is competitive with the transcription of human workers when benchmarked\non standard datasets. Finally, using a technique called Batch Dispatch with\nGPUs in the data center, we show that our system can be inexpensively deployed\nin an online setting, delivering low latency when serving users at scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.03465v3",
            "date": "2015-12-10 22:15:10+00:00",
            "title": "Mined Semantic Analysis: A New Concept Space Model for Semantic Representation of Textual Data",
            "authors": [
                "Walid Shalaby",
                "Wlodek Zadrozny"
            ],
            "abstract": "Mined Semantic Analysis (MSA) is a novel concept space model which employs\nunsupervised learning to generate semantic representations of text. MSA\nrepresents textual structures (terms, phrases, documents) as a Bag of Concepts\n(BoC) where concepts are derived from concept rich encyclopedic corpora.\nTraditional concept space models exploit only target corpus content to\nconstruct the concept space. MSA, alternatively, uncovers implicit relations\nbetween concepts by mining for their associations (e.g., mining Wikipedia's\n\"See also\" link graph). We evaluate MSA's performance on benchmark datasets for\nmeasuring semantic relatedness of words and sentences. Empirical results show\ncompetitive performance of MSA compared to prior state-of-the-art methods.\nAdditionally, we introduce the first analytical study to examine statistical\nsignificance of results reported by different semantic relatedness methods. Our\nstudy shows that, the nuances of results across top performing methods could be\nstatistically insignificant. The study positions MSA as one of state-of-the-art\nmethods for measuring semantic relatedness, besides the inherent\ninterpretability and simplicity of the generated semantic representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.03950v1",
            "date": "2015-12-12 18:57:11+00:00",
            "title": "A Hidden Markov Model Based System for Entity Extraction from Social Media English Text at FIRE 2015",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "This paper presents the experiments carried out by us at Jadavpur University\nas part of the participation in FIRE 2015 task: Entity Extraction from Social\nMedia Text - Indian Languages (ESM-IL). The tool that we have developed for the\ntask is based on Trigram Hidden Markov Model that utilizes information like\ngazetteer list, POS tag and some other word level features to enhance the\nobservation probabilities of the known tokens as well as unknown tokens. We\nsubmitted runs for English only. A statistical HMM (Hidden Markov Models) based\nmodel has been used to implement our system. The system has been trained and\ntested on the datasets released for FIRE 2015 task: Entity Extraction from\nSocial Media Text - Indian Languages (ESM-IL). Our system is the best performer\nfor English language and it obtains precision, recall and F-measures of 61.96,\n39.46 and 48.21 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04650v2",
            "date": "2015-12-15 04:55:06+00:00",
            "title": "Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation",
            "authors": [
                "Yong Cheng",
                "Shiqi Shen",
                "Zhongjun He",
                "Wei He",
                "Hua Wu",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "The attentional mechanism has proven to be effective in improving end-to-end\nneural machine translation. However, due to the intricate structural divergence\nbetween natural languages, unidirectional attention-based models might only\ncapture partial aspects of attentional regularities. We propose agreement-based\njoint training for bidirectional attention-based end-to-end neural machine\ntranslation. Instead of training source-to-target and target-to-source\ntranslation models independently,our approach encourages the two complementary\nmodels to agree on word alignment matrices on the same training data.\nExperiments on Chinese-English and English-French translation tasks show that\nagreement-based joint training significantly improves both alignment and\ntranslation quality over independent training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05030v3",
            "date": "2015-12-16 02:27:14+00:00",
            "title": "Morpho-syntactic Lexicon Generation Using Graph-based Semi-supervised Learning",
            "authors": [
                "Manaal Faruqui",
                "Ryan McDonald",
                "Radu Soricut"
            ],
            "abstract": "Morpho-syntactic lexicons provide information about the morphological and\nsyntactic roles of words in a language. Such lexicons are not available for all\nlanguages and even when available, their coverage can be limited. We present a\ngraph-based semi-supervised learning method that uses the morphological,\nsyntactic and semantic relations between words to automatically construct wide\ncoverage lexicons from small seed sets. Our method is language-independent, and\nwe show that we can expand a 1000 word seed lexicon to more than 100 times its\nsize with high quality for 11 languages. In addition, the automatically created\nlexicons provide features that improve performance in two downstream tasks:\nmorphological tagging and dependency parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05193v4",
            "date": "2015-12-16 14:55:17+00:00",
            "title": "ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "How to model a pair of sentences is a critical issue in many NLP tasks such\nas answer selection (AS), paraphrase identification (PI) and textual entailment\n(TE). Most prior work (i) deals with one individual task by fine-tuning a\nspecific system; (ii) models each sentence's representation separately, rarely\nconsidering the impact of the other sentence; or (iii) relies fully on manually\ndesigned, task-specific linguistic features. This work presents a general\nAttention Based Convolutional Neural Network (ABCNN) for modeling a pair of\nsentences. We make three contributions. (i) ABCNN can be applied to a wide\nvariety of tasks that require modeling of sentence pairs. (ii) We propose three\nattention schemes that integrate mutual influence between sentences into CNN;\nthus, the representation of each sentence takes into consideration its\ncounterpart. These interdependent sentence pair representations are more\npowerful than isolated sentence representations. (iii) ABCNN achieves\nstate-of-the-art performance on AS, PI and TE tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05670v2",
            "date": "2015-12-17 16:55:57+00:00",
            "title": "Towards automating the generation of derivative nouns in Sanskrit by simulating Panini",
            "authors": [
                "Amrith Krishna",
                "Pawan Goyal"
            ],
            "abstract": "About 1115 rules in Astadhyayi from A.4.1.76 to A.5.4.160 deal with\ngeneration of derivative nouns, making it one of the largest topical sections\nin Astadhyayi, called as the Taddhita section owing to the head rule A.4.1.76.\nThis section is a systematic arrangement of rules that enumerates various\naffixes that are used in the derivation under specific semantic relations. We\npropose a system that automates the process of generation of derivative nouns\nas per the rules in Astadhyayi. The proposed system follows a completely object\noriented approach, that models each rule as a class of its own and then groups\nthem as rule groups. The rule groups are decided on the basis of selective\ngrouping of rules by virtue of anuvrtti. The grouping of rules results in an\ninheritance network of rules which is a directed acyclic graph. Every rule\ngroup has a head rule and the head rule notifies all the direct member rules of\nthe group about the environment which contains all the details about data\nentities, participating in the derivation process. The system implements this\nmechanism using multilevel inheritance and observer design patterns. The system\nfocuses not only on generation of the desired final form, but also on the\ncorrectness of sequence of rules applied to make sure that the derivation has\ntaken place in strict adherence to Astadhyayi. The proposed system's design\nallows to incorporate various conflict resolution methods mentioned in\nauthentic texts and hence the effectiveness of those rules can be validated\nwith the results from the system. We also present cases where we have checked\nthe applicability of the system with the rules which are not specifically\napplicable to derivation of derivative nouns, in order to see the effectiveness\nof the proposed schema as a generic system for modeling Astadhyayi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05919v2",
            "date": "2015-12-18 12:10:42+00:00",
            "title": "A Planning based Framework for Essay Generation",
            "authors": [
                "Bing Qin",
                "Duyu Tang",
                "Xinwei Geng",
                "Dandan Ning",
                "Jiahao Liu",
                "Ting Liu"
            ],
            "abstract": "Generating an article automatically with computer program is a challenging\ntask in artificial intelligence and natural language processing. In this paper,\nwe target at essay generation, which takes as input a topic word in mind and\ngenerates an organized article under the theme of the topic. We follow the idea\nof text planning \\cite{Reiter1997} and develop an essay generation framework.\nThe framework consists of three components, including topic understanding,\nsentence extraction and sentence reordering. For each component, we studied\nseveral statistical algorithms and empirically compared between them in terms\nof qualitative or quantitative analysis. Although we run experiments on Chinese\ncorpus, the method is language independent and can be easily adapted to other\nlanguage. We lay out the remaining challenges and suggest avenues for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.06110v3",
            "date": "2015-12-18 20:48:26+00:00",
            "title": "Morphological Inflection Generation Using Character Sequence to Sequence Learning",
            "authors": [
                "Manaal Faruqui",
                "Yulia Tsvetkov",
                "Graham Neubig",
                "Chris Dyer"
            ],
            "abstract": "Morphological inflection generation is the task of generating the inflected\nform of a given lemma corresponding to a particular linguistic transformation.\nWe model the problem of inflection generation as a character sequence to\nsequence learning problem and present a variant of the neural encoder-decoder\nmodel for solving it. Our model is language independent and can be trained in\nboth supervised and semi-supervised settings. We evaluate our system on seven\ndatasets of morphologically rich languages and achieve either better or\ncomparable results to existing state-of-the-art models of inflection\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.06643v1",
            "date": "2015-12-21 14:31:31+00:00",
            "title": "The 2015 Sheffield System for Transcription of Multi-Genre Broadcast Media",
            "authors": [
                "Oscar Saz",
                "Mortaza Doulaty",
                "Salil Deena",
                "Rosanna Milner",
                "Raymond W. M. Ng",
                "Madina Hasan",
                "Yulan Liu",
                "Thomas Hain"
            ],
            "abstract": "We describe the University of Sheffield system for participation in the 2015\nMulti-Genre Broadcast (MGB) challenge task of transcribing multi-genre\nbroadcast shows. Transcription was one of four tasks proposed in the MGB\nchallenge, with the aim of advancing the state of the art of automatic speech\nrecognition, speaker diarisation and automatic alignment of subtitles for\nbroadcast media. Four topics are investigated in this work: Data selection\ntechniques for training with unreliable data, automatic speech segmentation of\nbroadcast media shows, acoustic modelling and adaptation in highly variable\nenvironments, and language modelling of multi-genre shows. The final system\noperates in multiple passes, using an initial unadapted decoding stage to\nrefine segmentation, followed by three adapted passes: a hybrid DNN pass with\ninput features normalised by speaker-based cepstral normalisation, another\nhybrid stage with input features normalised by speaker feature-MLLR\ntransformations, and finally a bottleneck-based tandem stage with noise and\nspeaker factorisation. The combination of these three system outputs provides a\nfinal error rate of 27.5% on the official development set, consisting of 47\nmulti-genre shows.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.08066v1",
            "date": "2015-12-26 01:52:03+00:00",
            "title": "The Improvement of Negative Sentences Translation in English-to-Korean Machine Translation",
            "authors": [
                "Chung-Hyok Jang",
                "Kwang-Hyok Kim"
            ],
            "abstract": "This paper describes the algorithm for translating English negative sentences\ninto Korean in English-Korean Machine Translation (EKMT). The proposed\nalgorithm is based on the comparative study of English and Korean negative\nsentences. The earlier translation software cannot translate English negative\nsentences into accurate Korean equivalents. We established a new algorithm for\nthe negative sentence translation and evaluated it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.08183v5",
            "date": "2015-12-27 08:12:53+00:00",
            "title": "Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews",
            "authors": [
                "Bofang Li",
                "Tao Liu",
                "Xiaoyong Du",
                "Deyuan Zhang",
                "Zhe Zhao"
            ],
            "abstract": "Despite the loss of semantic information, bag-of-ngram based methods still\nachieve state-of-the-art results for tasks such as sentiment classification of\nlong movie reviews. Many document embeddings methods have been proposed to\ncapture semantics, but they still can't outperform bag-of-ngram based methods\non this task. In this paper, we modify the architecture of the recently\nproposed Paragraph Vector, allowing it to learn document vectors by predicting\nnot only words, but n-gram features as well. Our model is able to capture both\nsemantics and word order in documents while keeping the expressive power of\nlearned vectors. Experimental results on IMDB movie review dataset shows that\nour model outperforms previous deep learning models and bag-of-ngram based\nmodels due to the above advantages. More robust results are also obtained when\nour model is combined with other models. The source code of our model will be\nalso published together with this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04241v1",
            "date": "2015-11-19 11:38:42+00:00",
            "title": "Detection of Slang Words in e-Data using semi-Supervised Learning",
            "authors": [
                "Alok Ranjan Pal",
                "Diganta Saha"
            ],
            "abstract": "The proposed algorithmic approach deals with finding the sense of a word in\nan electronic data. Now a day,in different communication mediums like internet,\nmobile services etc. people use few words, which are slang in nature. This\napproach detects those abusive words using supervised learning procedure. But\nin the real life scenario, the slang words are not used in complete word forms\nalways. Most of the times, those words are used in different abbreviated forms\nlike sounds alike forms, taboo morphemes etc. This proposed approach can detect\nthose abbreviated forms also using semi supervised learning procedure. Using\nthe synset and concept analysis of the text, the probability of a suspicious\nword to be a slang word is also evaluated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00240v1",
            "date": "2015-10-01 14:01:32+00:00",
            "title": "Determination of the Internet Anonymity Influence on the Level of Aggression and Usage of Obscene Lexis",
            "authors": [
                "Rodmonga Potapova",
                "Denis Gordeev"
            ],
            "abstract": "This article deals with the analysis of the semantic content of the anonymous\nRussian-speaking forum 2ch.hk, different verbal means of expressing of the\nemotional state of aggression are revealed for this site, and aggression is\nclassified by its directions. The lexis of different Russian-and English-\nspeaking anonymous forums (2ch.hk and iichan.hk, 4chan.org) and public\ncommunity \"MDK\" of the Russian-speaking social network VK is analyzed and\ncompared with the Open Corpus of the Russian language (Opencorpora.org and\nBrown corpus). The analysis shows that anonymity has no influence on the amount\nof invective items usage. The effectiveness of moderation was shown for\nanonymous forums. It was established that Russian obscene lexis was used to\nexpress the emotional state of aggression only in 60.4% of cases for 2ch.hk.\nThese preliminary results show that the Russian obscene lexis on the Internet\ndoes not have direct dependence on the emotional state of aggression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00436v1",
            "date": "2015-10-01 22:09:34+00:00",
            "title": "Response to Liu, Xu, and Liang (2015) and Ferrer-i-Cancho and G\u00f3mez-Rodr\u00edguez (2015) on Dependency Length Minimization",
            "authors": [
                "Richard Futrell",
                "Kyle Mahowald",
                "Edward Gibson"
            ],
            "abstract": "We address recent criticisms (Liu et al., 2015; Ferrer-i-Cancho and\nG\\'omez-Rodr\\'iguez, 2015) of our work on empirical evidence of dependency\nlength minimization across languages (Futrell et al., 2015). First, we\nacknowledge error in failing to acknowledge Liu (2008)'s previous work on\ncorpora of 20 languages with similar aims. A correction will appear in PNAS.\nNevertheless, we argue that our work provides novel, strong evidence for\ndependency length minimization as a universal quantitative property of\nlanguages, beyond this previous work, because it provides baselines which focus\non word order preferences. Second, we argue that our choices of baselines were\nappropriate because they control for alternative theories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00618v2",
            "date": "2015-10-02 15:02:38+00:00",
            "title": "Automatic Taxonomy Extraction from Query Logs with no Additional Sources of Information",
            "authors": [
                "Miguel Fernandez-Fernandez",
                "Daniel Gayo-Avello"
            ],
            "abstract": "Search engine logs store detailed information on Web users interactions.\nThus, as more and more people use search engines on a daily basis, important\ntrails of users common knowledge are being recorded in those files. Previous\nresearch has shown that it is possible to extract concept taxonomies from full\ntext documents, while other scholars have proposed methods to obtain similar\nqueries from query logs. We propose a mixture of both lines of research, that\nis, mining query logs not to find related queries nor query hierarchies, but\nactual term taxonomies that could be used to improve search engine\neffectiveness and efficiency. As a result, in this study we have developed a\nmethod that combines lexical heuristics with a supervised classification model\nto successfully extract hyponymy relations from specialization search patterns\nrevealed from log missions, with no additional sources of information, and in a\nlanguage independent way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00726v1",
            "date": "2015-10-02 20:17:33+00:00",
            "title": "A Primer on Neural Network Models for Natural Language Processing",
            "authors": [
                "Yoav Goldberg"
            ],
            "abstract": "Over the past few years, neural networks have re-emerged as powerful\nmachine-learning models, yielding state-of-the-art results in fields such as\nimage recognition and speech processing. More recently, neural network models\nstarted to be applied also to textual natural language signals, again with very\npromising results. This tutorial surveys neural network models from the\nperspective of natural language processing research, in an attempt to bring\nnatural-language researchers up to speed with the neural techniques. The\ntutorial covers input encoding for natural language tasks, feed-forward\nnetworks, convolutional networks, recurrent networks and recursive networks, as\nwell as the computation graph abstraction for automatic gradient computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00759v1",
            "date": "2015-10-03 00:08:18+00:00",
            "title": "It is not all downhill from here: Syllable Contact Law in Persian",
            "authors": [
                "Afshin Rahimi",
                "Moharram Eslami",
                "Bahram Vazirnezhad"
            ],
            "abstract": "Syllable contact pairs crosslinguistically tend to have a falling sonority\nslope a constraint which is called the Syllable Contact Law SCL In this study\nthe phonotactics of syllable contacts in 4202 CVCCVC words of Persian lexicon\nis investigated The consonants of Persian were divided into five sonority\ncategories and the frequency of all possible sonority slopes is computed both\nin lexicon type frequency and in corpus token frequency Since an unmarked\nphonological structure has been shown to diachronically become more frequent we\nexpect to see the same pattern for syllable contact pairs with falling sonority\nslope The correlation of sonority categories of the two consonants in a\nsyllable contact pair is measured using Pointwise Mutual Information",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00760v1",
            "date": "2015-10-03 00:13:29+00:00",
            "title": "P-trac Procedure: The Dispersion and Neutralization of Contrasts in Lexicon",
            "authors": [
                "Afshin Rahimi",
                "Bahram Vazirnezhad",
                "Moharram Eslami"
            ],
            "abstract": "Cognitive acoustic cues have an important role in shaping the phonological\nstructure of language as a means to optimal communication. In this paper we\nintroduced P-trac procedure in order to track dispersion of contrasts in\ndifferent contexts in lexicon. The results of applying P-trac procedure to the\ncase of dispersion of contrasts in pre- consonantal contexts and in consonantal\npositions of CVCC sequences in Persian provide Evidence in favor of phonetic\nbasis of dispersion argued by Licensing by Cue hypothesis and the Dispersion\nTheory of Contrast. The P- trac procedure is proved to be very effective in\nrevealing the dispersion of contrasts in lexicon especially when comparing the\ndispersion of contrasts in different contexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.01032v2",
            "date": "2015-10-05 05:25:32+00:00",
            "title": "Deep convolutional acoustic word embeddings using word-pair side information",
            "authors": [
                "Herman Kamper",
                "Weiran Wang",
                "Karen Livescu"
            ],
            "abstract": "Recent studies have been revisiting whole words as the basic modelling unit\nin speech recognition and query applications, instead of phonetic units. Such\nwhole-word segmental systems rely on a function that maps a variable-length\nspeech segment to a vector in a fixed-dimensional space; the resulting acoustic\nword embeddings need to allow for accurate discrimination between different\nword types, directly in the embedding space. We compare several old and new\napproaches in a word discrimination task. Our best approach uses side\ninformation in the form of known word pairs to train a Siamese convolutional\nneural network (CNN): a pair of tied networks that take two speech segments as\ninput and produce their embeddings, trained with a hinge loss that separates\nsame-word pairs and different-word pairs by some margin. A word classifier CNN\nperforms similarly, but requires much stronger supervision. Both types of CNNs\nyield large improvements over the best previously published results on the word\ndiscrimination task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.01570v1",
            "date": "2015-10-06 13:33:46+00:00",
            "title": "Analyzer and generator for Pali",
            "authors": [
                "David Alfter"
            ],
            "abstract": "This work describes a system that performs morphological analysis and\ngeneration of Pali words. The system works with regular inflectional paradigms\nand a lexical database. The generator is used to build a collection of\ninflected and derived words, which in turn is used by the analyzer. Generating\nand storing morphological forms along with the corresponding morphological\ninformation allows for efficient and simple look up by the analyzer. Indeed, by\nlooking up a word and extracting the attached morphological information, the\nanalyzer does not have to compute this information. As we must, however, assume\nthe lexical database to be incomplete, the system can also work without the\ndictionary component, using a rule-based approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.01717v1",
            "date": "2015-10-06 19:35:23+00:00",
            "title": "Language Segmentation",
            "authors": [
                "David Alfter"
            ],
            "abstract": "Language segmentation consists in finding the boundaries where one language\nends and another language begins in a text written in more than one language.\nThis is important for all natural language processing tasks. The problem can be\nsolved by training language models on language data. However, in the case of\nlow- or no-resource languages, this is problematic. I therefore investigate\nwhether unsupervised methods perform better than supervised methods when it is\ndifficult or impossible to train supervised approaches. A special focus is\ngiven to difficult texts, i.e. texts that are rather short (one sentence),\ncontaining abbreviations, low-resource languages and non-standard language. I\ncompare three approaches: supervised n-gram language models, unsupervised\nclustering and weakly supervised n-gram language model induction. I devised the\nweakly supervised approach in order to deal with difficult text specifically.\nIn order to test the approach, I compiled a small corpus of different text\ntypes, ranging from one-sentence texts to texts of about 300 words. The weakly\nsupervised language model induction approach works well on short and difficult\ntexts, outperforming the clustering algorithm and reaching scores in the\nvicinity of the supervised approach. The results look promising, but there is\nroom for improvement and a more thorough investigation should be undertaken.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.01886v1",
            "date": "2015-10-07 10:50:31+00:00",
            "title": "Using Ontology-Based Context in the Portuguese-English Translation of Homographs in Textual Dialogues",
            "authors": [
                "Diego Moussallem",
                "Ricardo Choren"
            ],
            "abstract": "This paper introduces a novel approach to tackle the existing gap on message\ntranslations in dialogue systems. Currently, submitted messages to the dialogue\nsystems are considered as isolated sentences. Thus, missing context information\nimpede the disambiguation of homographs words in ambiguous sentences. Our\napproach solves this disambiguation problem by using concepts over existing\nontologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02049v1",
            "date": "2015-10-07 18:08:45+00:00",
            "title": "Assisting Composition of Email Responses: a Topic Prediction Approach",
            "authors": [
                "Spandana Gella",
                "Marc Dymetman",
                "Jean Michel Renders",
                "Sriram Venkatapathy"
            ],
            "abstract": "We propose an approach for helping agents compose email replies to customer\nrequests. To enable that, we use LDA to extract latent topics from a collection\nof email exchanges. We then use these latent topics to label our data,\nobtaining a so-called \"silver standard\" topic labelling. We exploit this\nlabelled set to train a classifier to: (i) predict the topic distribution of\nthe entire agent's email response, based on features of the customer's email;\nand (ii) predict the topic distribution of the next sentence in the agent's\nreply, based on the customer's email features and on features of the agent's\ncurrent sentence. The experimental results on a large email collection from a\ncontact center in the tele- com domain show that the proposed ap- proach is\neffective in predicting the best topic of the agent's next sentence. In 80% of\nthe cases, the correct topic is present among the top five recommended topics\n(out of fifty possible ones). This shows the potential of this method to be\napplied in an interactive setting, where the agent is presented a small list of\nlikely topics to choose from for the next sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02125v3",
            "date": "2015-10-07 20:52:22+00:00",
            "title": "Resolving References to Objects in Photographs using the Words-As-Classifiers Model",
            "authors": [
                "David Schlangen",
                "Sina Zarriess",
                "Casey Kennington"
            ],
            "abstract": "A common use of language is to refer to visually present objects. Modelling\nit in computers requires modelling the link between language and perception.\nThe \"words as classifiers\" model of grounded semantics views words as\nclassifiers of perceptual contexts, and composes the meaning of a phrase\nthrough composition of the denotations of its component words. It was recently\nshown to perform well in a game-playing scenario with a small number of object\ntypes. We apply it to two large sets of real-world photographs that contain a\nmuch larger variety of types and for which referring expressions are available.\nUsing a pre-trained convolutional neural network to extract image features, and\naugmenting these with in-picture positional information, we show that the model\nachieves performance competitive with the state of the art in a reference\nresolution task (given expression, find bounding box of its referent), while,\nas we argue, being conceptually simpler and more flexible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02358v2",
            "date": "2015-10-08 15:15:09+00:00",
            "title": "Automata networks for multi-party communication in the Naming Game",
            "authors": [
                "Javier Vera",
                "Pedro Montealegre",
                "Eric Goles"
            ],
            "abstract": "The Naming Game has been studied to explore the role of self-organization in\nthe development and negotiation of linguistic conventions. In this paper, we\ndefine an automata networks approach to the Naming Game. Two problems are\nfaced: (1) the definition of an automata networks for multi-party communicative\ninteractions; and (2) the proof of convergence for three different orders in\nwhich the individuals are updated (updating schemes). Finally, computer\nsimulations are explored in two-dimensional lattices with the purpose to\nrecover the main features of the Naming Game and to describe the dynamics under\ndifferent updating schemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02675v2",
            "date": "2015-10-09 14:03:33+00:00",
            "title": "Controlled Experiments for Word Embeddings",
            "authors": [
                "Benjamin J. Wilson",
                "Adriaan M. J. Schakel"
            ],
            "abstract": "An experimental approach to studying the properties of word embeddings is\nproposed. Controlled experiments, achieved through modifications of the\ntraining corpus, permit the demonstration of direct relations between word\nproperties and word vector direction and length. The approach is demonstrated\nusing the word2vec CBOW model with experiments that independently vary word\nfrequency and word co-occurrence noise. The experiments reveal that word vector\nlength depends more or less linearly on both word frequency and the level of\nnoise in the co-occurrence distribution of the word. The coefficients of\nlinearity depend upon the word. The special point in feature space, defined by\nthe (artificial) word with pure noise in its co-occurrence distribution, is\nfound to be small but non-zero.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02823v1",
            "date": "2015-10-09 21:05:02+00:00",
            "title": "Human languages order information efficiently",
            "authors": [
                "Daniel Gildea",
                "T. Florian Jaeger"
            ],
            "abstract": "Most languages use the relative order between words to encode meaning\nrelations. Languages differ, however, in what orders they use and how these\norders are mapped onto different meanings. We test the hypothesis that, despite\nthese differences, human languages might constitute different `solutions' to\ncommon pressures of language use. Using Monte Carlo simulations over data from\nfive languages, we find that their word orders are efficient for processing in\nterms of both dependency length and local lexical probability. This suggests\nthat biases originating in how the brain understands language strongly\nconstrain how human languages change over generations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03055v3",
            "date": "2015-10-11 14:04:57+00:00",
            "title": "A Diversity-Promoting Objective Function for Neural Conversation Models",
            "authors": [
                "Jiwei Li",
                "Michel Galley",
                "Chris Brockett",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "Sequence-to-sequence neural network models for generation of conversational\nresponses tend to generate safe, commonplace responses (e.g., \"I don't know\")\nregardless of the input. We suggest that the traditional objective function,\ni.e., the likelihood of output (response) given input (message) is unsuited to\nresponse generation tasks. Instead we propose using Maximum Mutual Information\n(MMI) as the objective function in neural models. Experimental results\ndemonstrate that the proposed MMI models produce more diverse, interesting, and\nappropriate responses, yielding substantive gains in BLEU scores on two\nconversational datasets and in human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03421v2",
            "date": "2015-10-12 13:52:40+00:00",
            "title": "Towards Meaningful Maps of Polish Case Law",
            "authors": [
                "Michal Jungiewicz",
                "Micha\u0142 \u0141opuszy\u0144ski"
            ],
            "abstract": "In this work, we analyze the utility of two dimensional document maps for\nexploratory analysis of Polish case law. We start by comparing two methods of\ngenerating such visualizations. First is based on linear principal component\nanalysis (PCA). Second makes use of the modern nonlinear t-Distributed\nStochastic Neighbor Embedding method (t-SNE). We apply both PCA and t-SNE to a\ncorpus of judgments from different courts in Poland. It emerges that t-SNE\nprovides better, more interpretable results than PCA. As a next test, we apply\nt-SNE to randomly selected sample of common court judgments corresponding to\ndifferent keywords. We show that t-SNE, in this case, reveals hidden topical\nstructure of the documents related to keyword,,pension\". In conclusion, we find\nthat the t-SNE method could be a promising tool to facilitate the exploitative\nanalysis of legal texts, e.g., by complementing search or browse functionality\nin legal databases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03519v3",
            "date": "2015-10-13 03:25:18+00:00",
            "title": "Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning",
            "authors": [
                "Janarthanan Rajendran",
                "Mitesh M. Khapra",
                "Sarath Chandar",
                "Balaraman Ravindran"
            ],
            "abstract": "Recently there has been a lot of interest in learning common representations\nfor multiple views of data. Typically, such common representations are learned\nusing a parallel corpus between the two views (say, 1M images and their English\ncaptions). In this work, we address a real-world scenario where no direct\nparallel data is available between two views of interest (say, $V_1$ and $V_2$)\nbut parallel data is available between each of these views and a pivot view\n($V_3$). We propose a model for learning a common representation for $V_1$,\n$V_2$ and $V_3$ using only the parallel data available between $V_1V_3$ and\n$V_2V_3$. The proposed model is generic and even works when there are $n$ views\nof interest and only one pivot view which acts as a bridge between them. There\nare two specific downstream applications that we focus on (i) transfer learning\nbetween languages $L_1$,$L_2$,...,$L_n$ using a pivot language $L$ and (ii)\ncross modal access between images and a language $L_1$ using a pivot language\n$L_2$. Our model achieves state-of-the-art performance in multilingual document\nclassification on the publicly available multilingual TED corpus and promising\nresults in multilingual multimodal retrieval on a new dataset created and\nreleased as a part of this work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03710v3",
            "date": "2015-10-13 14:44:01+00:00",
            "title": "Hybrid Dialog State Tracker",
            "authors": [
                "Miroslav Vodol\u00e1n",
                "Rudolf Kadlec",
                "Jan Kleindienst"
            ],
            "abstract": "This paper presents a hybrid dialog state tracker that combines a rule based\nand a machine learning based approach to belief state tracking. Therefore, we\ncall it a hybrid tracker. The machine learning in our tracker is realized by a\nLong Short Term Memory (LSTM) network. To our knowledge, our hybrid tracker\nsets a new state-of-the-art result for the Dialog State Tracking Challenge\n(DSTC) 2 dataset when the system uses only live SLU as its input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03753v2",
            "date": "2015-10-13 15:56:26+00:00",
            "title": "Improved Deep Learning Baselines for Ubuntu Corpus Dialogs",
            "authors": [
                "Rudolf Kadlec",
                "Martin Schmid",
                "Jan Kleindienst"
            ],
            "abstract": "This paper presents results of our experiments for the next utterance ranking\non the Ubuntu Dialog Corpus -- the largest publicly available multi-turn dialog\ncorpus. First, we use an in-house implementation of previously reported models\nto do an independent evaluation using the same data. Second, we evaluate the\nperformances of various LSTMs, Bi-LSTMs and CNNs on the dataset. Third, we\ncreate an ensemble by averaging predictions of multiple models. The ensemble\nfurther improves the performance and it achieves a state-of-the-art result for\nthe next utterance ranking on this dataset. Finally, we discuss our future\nplans using this corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04500v1",
            "date": "2015-10-15 12:29:14+00:00",
            "title": "Noisy-parallel and comparable corpora filtering methodology for the extraction of bi-lingual equivalent data at sentence level",
            "authors": [
                "Krzysztof Wo\u0142k"
            ],
            "abstract": "Text alignment and text quality are critical to the accuracy of Machine\nTranslation (MT) systems, some NLP tools, and any other text processing tasks\nrequiring bilingual data. This research proposes a language independent\nbi-sentence filtering approach based on Polish (not a position-sensitive\nlanguage) to English experiments. This cleaning approach was developed on the\nTED Talks corpus and also initially tested on the Wikipedia comparable corpus,\nbut it can be used for any text domain or language pair. The proposed approach\nimplements various heuristics for sentence comparison. Some of them leverage\nsynonyms and semantic and structural analysis of text as additional\ninformation. Minimization of data loss was ensured. An improvement in MT system\nscore with text processed using the tool is discussed.",
            "journal ref": "Computer Science 16.2 (2015): 169-184",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04600v1",
            "date": "2015-10-15 16:02:50+00:00",
            "title": "Telemedicine as a special case of Machine Translation",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek",
                "Wojciech Glinkowski"
            ],
            "abstract": "Machine translation is evolving quite rapidly in terms of quality. Nowadays,\nwe have several machine translation systems available in the web, which provide\nreasonable translations. However, these systems are not perfect, and their\nquality may decrease in some specific domains. This paper examines the effects\nof different training methods when it comes to Polish - English Statistical\nMachine Translation system used for the medical data. Numerous elements of the\nEMEA parallel text corpora and not related OPUS Open Subtitles project were\nused as the ground for creation of phrase tables and different language models\nincluding the development, tuning and testing of these translation systems. The\nBLEU, NIST, METEOR, and TER metrics have been used in order to evaluate the\nresults of various systems. Our experiments deal with the systems that include\nPOS tagging, factored phrase models, hierarchical models, syntactic taggers,\nand other alignment methods. We also executed a deep analysis of Polish data as\npreparatory work before automatized data processing such as true casing or\npunctuation normalization phase. Normalized metrics was used to compare\nresults. Scores lower than 15% mean that Machine Translation engine is unable\nto provide satisfying quality, scores greater than 30% mean that translations\nshould be understandable without problems and scores over 50 reflect adequate\ntranslations. The average results of Polish to English translations scores for\nBLEU, NIST, METEOR, and TER were relatively high and ranged from 70,58 to\n82,72. The lowest score was 64,38. The average results ranges for English to\nPolish translations were little lower (67,58 - 78,97). The real-life\nimplementations of presented high quality Machine Translation Systems are\nanticipated in general medical practice and telemedicine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04734v1",
            "date": "2015-10-15 22:36:08+00:00",
            "title": "A Method for Modeling Co-Occurrence Propensity of Clinical Codes with Application to ICD-10-PCS Auto-Coding",
            "authors": [
                "Michael Subotin",
                "Anthony R. Davis"
            ],
            "abstract": "Objective. Natural language processing methods for medical auto-coding, or\nautomatic generation of medical billing codes from electronic health records,\ngenerally assign each code independently of the others. They may thus assign\ncodes for closely related procedures or diagnoses to the same document, even\nwhen they do not tend to occur together in practice, simply because the right\nchoice can be difficult to infer from the clinical narrative.\n  Materials and Methods. We propose a method that injects awareness of the\npropensities for code co-occurrence into this process. First, a model is\ntrained to estimate the conditional probability that one code is assigned by a\nhuman coder, given than another code is known to have been assigned to the same\ndocument. Then, at runtime, an iterative algorithm is used to apply this model\nto the output of an existing statistical auto-coder to modify the confidence\nscores of the codes.\n  Results. We tested this method in combination with a primary auto-coder for\nICD-10 procedure codes, achieving a 12% relative improvement in F-score over\nthe primary auto-coder baseline.\n  Discussion. The proposed method can be used, with appropriate features, in\ncombination with any auto-coder that generates codes with different levels of\nconfidence.\n  Conclusion. The promising results obtained for ICD-10 procedure codes suggest\nthat the proposed method may have wider applications in auto-coding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.05203v1",
            "date": "2015-10-18 06:26:42+00:00",
            "title": "Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015",
            "authors": [
                "Graham Neubig",
                "Makoto Morishita",
                "Satoshi Nakamura"
            ],
            "abstract": "This year, the Nara Institute of Science and Technology (NAIST)'s submission\nto the 2015 Workshop on Asian Translation was based on syntax-based statistical\nmachine translation, with the addition of a reranking component using neural\nattentional machine translation models. Experiments re-confirmed results from\nprevious work stating that neural MT reranking provides a large gain in\nobjective evaluation measures such as BLEU, and also confirmed for the first\ntime that these results also carry over to manual evaluation. We further\nperform a detailed analysis of reasons for this increase, finding that the main\ncontributions of the neural models lie in improvement of the grammatical\ncorrectness of the output, as opposed to improvements in lexical choice of\ncontent words.",
            "journal ref": "Proceedings of the 2nd Workshop on Asian Translation (WAT), pp.\n  35-41, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.06168v1",
            "date": "2015-10-21 08:57:30+00:00",
            "title": "Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network",
            "authors": [
                "Peilu Wang",
                "Yao Qian",
                "Frank K. Soong",
                "Lei He",
                "Hai Zhao"
            ],
            "abstract": "Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has\nbeen shown to be very effective for tagging sequential data, e.g. speech\nutterances or handwritten documents. While word embedding has been demoed as a\npowerful representation for characterizing the statistical properties of\nnatural language. In this study, we propose to use BLSTM-RNN with word\nembedding for part-of-speech (POS) tagging task. When tested on Penn Treebank\nWSJ test set, a state-of-the-art performance of 97.40 tagging accuracy is\nachieved. Without using morphological features, this approach can also achieve\na good performance comparable with the Stanford POS tagger.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.06807v1",
            "date": "2015-10-23 02:24:23+00:00",
            "title": "Learning in the Rational Speech Acts Model",
            "authors": [
                "Will Monroe",
                "Christopher Potts"
            ],
            "abstract": "The Rational Speech Acts (RSA) model treats language use as a recursive\nprocess in which probabilistic speaker and listener agents reason about each\nother's intentions to enrich the literal semantics of their language along\nbroadly Gricean lines. RSA has been shown to capture many kinds of\nconversational implicature, but it has been criticized as an unrealistic model\nof speakers, and it has so far required the manual specification of a semantic\nlexicon, preventing its use in natural language processing applications that\nlearn lexical knowledge from data. We address these concerns by showing how to\ndefine and optimize a trained statistical classifier that uses the intermediate\nagents of RSA as hidden layers of representation forming a non-linear\nactivation function. This treatment opens up new application domains and new\npossibilities for learning effectively from data. We validate the model on a\nreferential expression generation task, showing that the best performance is\nachieved by incorporating features approximating well-established insights\nabout natural language generation into RSA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07099v1",
            "date": "2015-10-24 02:24:54+00:00",
            "title": "Combine CRF and MMSEG to Boost Chinese Word Segmentation in Social Media",
            "authors": [
                "Yao Yushi",
                "Huang Zheng"
            ],
            "abstract": "In this paper, we propose a joint algorithm for the word segmentation on\nChinese social media. Previous work mainly focus on word segmentation for plain\nChinese text, in order to develop a Chinese social media processing tool, we\nneed to take the main features of social media into account, whose grammatical\nstructure is not rigorous, and the tendency of using colloquial and Internet\nterms makes the existing Chinese-processing tools inefficient to obtain good\nperformance on social media.\n  In our approach, we combine CRF and MMSEG algorithm and extend features of\ntraditional CRF algorithm to train the model for word segmentation, We use\nInternet lexicon in order to improve the performance of our model on Chinese\nsocial media. Our experimental result on Sina Weibo shows that our approach\noutperforms the state-of-the-art model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07193v1",
            "date": "2015-10-25 00:53:03+00:00",
            "title": "Statistical Parsing by Machine Learning from a Classical Arabic Treebank",
            "authors": [
                "Kais Dukes"
            ],
            "abstract": "Research into statistical parsing for English has enjoyed over a decade of\nsuccessful results. However, adapting these models to other languages has met\nwith difficulties. Previous comparative work has shown that Modern Arabic is\none of the most difficult languages to parse due to rich morphology and free\nword order. Classical Arabic is the ancient form of Arabic, and is understudied\nin computational linguistics, relative to its worldwide reach as the language\nof the Quran. The thesis is based on seven publications that make significant\ncontributions to knowledge relating to annotating and parsing Classical Arabic.\n  A central argument of this thesis is that using a hybrid representation\nclosely aligned to traditional grammar leads to improved parsing for Arabic. To\ntest this hypothesis, two approaches are compared. As a reference, a pure\ndependency parser is adapted using graph transformations, resulting in an\n87.47% F1-score. This is compared to an integrated parsing model with an\nF1-score of 89.03%, demonstrating that joint dependency-constituency parsing is\nbetter suited to Classical Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07482v4",
            "date": "2015-10-26 13:56:25+00:00",
            "title": "Edge-Linear First-Order Dependency Parsing with Undirected Minimum Spanning Tree Inference",
            "authors": [
                "Effi Levi",
                "Roi Reichart",
                "Ari Rappoport"
            ],
            "abstract": "The run time complexity of state-of-the-art inference algorithms in\ngraph-based dependency parsing is super-linear in the number of input words\n(n). Recently, pruning algorithms for these models have shown to cut a large\nportion of the graph edges, with minimal damage to the resulting parse trees.\nSolving the inference problem in run time complexity determined solely by the\nnumber of edges (m) is hence of obvious importance.\n  We propose such an inference algorithm for first-order models, which encodes\nthe problem as a minimum spanning tree (MST) problem in an undirected graph.\nThis allows us to utilize state-of-the-art undirected MST algorithms whose run\ntime is O(m) at expectation and with a very high probability. A directed parse\ntree is then inferred from the undirected MST and is subsequently improved with\nrespect to the directed parsing model through local greedy updates, both steps\nrunning in O(n) time. In experiments with 18 languages, a variant of the\nfirst-order MSTParser (McDonald et al., 2005b) that employs our algorithm\nperforms very similarly to the original parser that runs an O(n^2) directed MST\ninference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07586v1",
            "date": "2015-10-26 18:34:16+00:00",
            "title": "Parser for Abstract Meaning Representation using Learning to Search",
            "authors": [
                "Sudha Rao",
                "Yogarshi Vyas",
                "Hal Daume III",
                "Philip Resnik"
            ],
            "abstract": "We develop a novel technique to parse English sentences into Abstract Meaning\nRepresentation (AMR) using SEARN, a Learning to Search approach, by modeling\nthe concept and the relation learning in a unified framework. We evaluate our\nparser on multiple datasets from varied domains and show an absolute\nimprovement of 2% to 6% over the state-of-the-art. Additionally we show that\nusing the most frequent concept gives us a baseline that is stronger than the\nstate-of-the-art for concept prediction. We plan to release our parser for\npublic use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07851v1",
            "date": "2015-10-27 10:43:50+00:00",
            "title": "Standards for language resources in ISO -- Looking back at 13 fruitful years",
            "authors": [
                "Laurent Romary"
            ],
            "abstract": "This paper provides an overview of the various projects carried out within\nISO committee TC 37/SC 4 dealing with the management of language (digital)\nresources. On the basis of the technical experience gained in the committee and\nthe wider standardization landscape the paper identifies some possible trends\nfor the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.08418v1",
            "date": "2015-10-28 19:04:30+00:00",
            "title": "Fast k-best Sentence Compression",
            "authors": [
                "Katja Filippova",
                "Enrique Alfonseca"
            ],
            "abstract": "A popular approach to sentence compression is to formulate the task as a\nconstrained optimization problem and solve it with integer linear programming\n(ILP) tools. Unfortunately, dependence on ILP may make the compressor\nprohibitively slow, and thus approximation techniques have been proposed which\nare often complex and offer a moderate gain in speed. As an alternative\nsolution, we introduce a novel compression algorithm which generates k-best\ncompressions relying on local deletion decisions. Our algorithm is two orders\nof magnitude faster than a recent ILP-based method while producing better\ncompressions. Moreover, an extensive evaluation demonstrates that the quality\nof compressions does not degrade much as we move from single best to top-five\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.08480v1",
            "date": "2015-10-28 20:41:30+00:00",
            "title": "Emoticons vs. Emojis on Twitter: A Causal Inference Approach",
            "authors": [
                "Umashanthi Pavalanathan",
                "Jacob Eisenstein"
            ],
            "abstract": "Online writing lacks the non-verbal cues present in face-to-face\ncommunication, which provide additional contextual information about the\nutterance, such as the speaker's intention or affective state. To fill this\nvoid, a number of orthographic features, such as emoticons, expressive\nlengthening, and non-standard punctuation, have become popular in social media\nservices including Twitter and Instagram. Recently, emojis have been introduced\nto social media, and are increasingly popular. This raises the question of\nwhether these predefined pictographic characters will come to replace earlier\northographic methods of paralinguistic communication. In this abstract, we\nattempt to shed light on this question, using a matching approach from causal\ninference to test whether the adoption of emojis causes individual users to\nemploy fewer emoticons in their text on Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.09079v1",
            "date": "2015-10-30 13:19:47+00:00",
            "title": "SentiWords: Deriving a High Precision and High Coverage Lexicon for Sentiment Analysis",
            "authors": [
                "Lorenzo Gatti",
                "Marco Guerini",
                "Marco Turchi"
            ],
            "abstract": "Deriving prior polarity lexica for sentiment analysis - where positive or\nnegative scores are associated with words out of context - is a challenging\ntask. Usually, a trade-off between precision and coverage is hard to find, and\nit depends on the methodology used to build the lexicon. Manually annotated\nlexica provide a high precision but lack in coverage, whereas automatic\nderivation from pre-existing knowledge guarantees high coverage at the cost of\na lower precision. Since the automatic derivation of prior polarities is less\ntime consuming than manual annotation, there has been a great bloom of these\napproaches, in particular based on the SentiWordNet resource. In this paper, we\ncompare the most frequently used techniques based on SentiWordNet with newer\nones and blend them in a learning framework (a so called 'ensemble method'). By\ntaking advantage of manually built prior polarity lexica, our ensemble method\nis better able to predict the prior value of unseen words and to outperform all\nthe other SentiWordNet approaches. Using this technique we have built\nSentiWords, a prior polarity lexicon of approximately 155,000 words, that has\nboth a high precision and a high coverage. We finally show that in sentiment\nanalysis tasks, using our lexicon allows us to outperform both the single\nmetrics derived from SentiWordNet and popular manually annotated sentiment\nlexica.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.00161v1",
            "date": "2015-05-01 11:43:34+00:00",
            "title": "Embedding Semantic Relations into Word Representations",
            "authors": [
                "Danushka Bollegala",
                "Takanori Maehara",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Learning representations for semantic relations is important for various\ntasks such as analogy detection, relational search, and relation\nclassification. Although there have been several proposals for learning\nrepresentations for individual words, learning word representations that\nexplicitly capture the semantic relations between words remains under\ndeveloped. We propose an unsupervised method for learning vector\nrepresentations for words such that the learnt representations are sensitive to\nthe semantic relations that exist between two words. First, we extract lexical\npatterns from the co-occurrence contexts of two words in a corpus to represent\nthe semantic relations that exist between those two words. Second, we represent\na lexical pattern as the weighted sum of the representations of the words that\nco-occur with that lexical pattern. Third, we train a binary classifier to\ndetect relationally similar vs. non-similar lexical pattern pairs. The proposed\nmethod is unsupervised in the sense that the lexical pattern pairs we use as\ntrain data are automatically sampled from a corpus, without requiring any\nmanual intervention. Our proposed method statistically significantly\noutperforms the current state-of-the-art word representations on three\nbenchmark datasets for proportional analogy detection, demonstrating its\nability to accurately capture the semantic relations among words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.02425v1",
            "date": "2015-05-10 19:26:31+00:00",
            "title": "Fast Rhetorical Structure Theory Discourse Parsing",
            "authors": [
                "Michael Heilman",
                "Kenji Sagae"
            ],
            "abstract": "In recent years, There has been a variety of research on discourse parsing,\nparticularly RST discourse parsing. Most of the recent work on RST parsing has\nfocused on implementing new types of features or learning algorithms in order\nto improve accuracy, with relatively little focus on efficiency, robustness, or\npractical use. Also, most implementations are not widely available. Here, we\ndescribe an RST segmentation and parsing system that adapts models and feature\nsets from various previous work, as described below. Its accuracy is near\nstate-of-the-art, and it was developed to be fast, robust, and practical. For\nexample, it can process short documents such as news articles or essays in less\nthan a second.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03081v1",
            "date": "2015-05-12 16:33:03+00:00",
            "title": "Turn Segmentation into Utterances for Arabic Spontaneous Dialogues and Instance Messages",
            "authors": [
                "AbdelRahim A. Elmadany",
                "Sherif M. Abdou",
                "Mervat Gheith"
            ],
            "abstract": "Text segmentation task is an essential processing task for many of Natural\nLanguage Processing (NLP) such as text summarization, text translation,\ndialogue language understanding, among others. Turns segmentation considered\nthe key player in dialogue understanding task for building automatic\nHuman-Computer systems. In this paper, we introduce a novel approach to turn\nsegmentation into utterances for Egyptian spontaneous dialogues and Instance\nMessages (IM) using Machine Learning (ML) approach as a part of automatic\nunderstanding Egyptian spontaneous dialogues and IM task. Due to the lack of\nEgyptian dialect dialogue corpus the system evaluated by our corpus includes\n3001 turns, which are collected, segmented, and annotated manually from\nEgyptian call-centers. The system achieves F1 scores of 90.74% and accuracy of\n95.98%.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  4, No.2,April 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03084v1",
            "date": "2015-05-12 16:38:39+00:00",
            "title": "A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message",
            "authors": [
                "AbdelRahim A. Elmadany",
                "Sherif M. Abdou",
                "Mervat Gheith"
            ],
            "abstract": "Building dialogues systems interaction has recently gained considerable\nattention, but most of the resources and systems built so far are tailored to\nEnglish and other Indo-European languages. The need for designing systems for\nother languages is increasing such as Arabic language. For this reasons, there\nare more interest for Arabic dialogue acts classification task because it a key\nplayer in Arabic language understanding to building this systems. This paper\nsurveys different techniques for dialogue acts classification for Arabic. We\ndescribe the main existing techniques for utterances segmentations and\nclassification, annotation schemas, and test corpora for Arabic dialogues\nunderstanding that have introduced in the literature",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  4, No.2,April 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03085v1",
            "date": "2015-05-12 16:45:52+00:00",
            "title": "Indonesian Social Media Sentiment Analysis With Sarcasm Detection",
            "authors": [
                "Edwin Lunando",
                "Ayu Purwarianti"
            ],
            "abstract": "Sarcasm is considered one of the most difficult problem in sentiment\nanalysis. In our ob-servation on Indonesian social media, for cer-tain topics,\npeople tend to criticize something using sarcasm. Here, we proposed two\nadditional features to detect sarcasm after a common sentiment analysis is\nconducted. The features are the negativity information and the number of\ninterjection words. We also employed translated SentiWordNet in the sentiment\nclassification. All the classifications were conducted with machine learning\nalgorithms. The experimental results showed that the additional features are\nquite effective in the sarcasm detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03105v1",
            "date": "2015-05-12 18:10:53+00:00",
            "title": "Sentiment Analysis For Modern Standard Arabic And Colloquial",
            "authors": [
                "Hossam S. Ibrahim",
                "Sherif M. Abdou",
                "Mervat Gheith"
            ],
            "abstract": "The rise of social media such as blogs and social networks has fueled\ninterest in sentiment analysis. With the proliferation of reviews, ratings,\nrecommendations and other forms of online expression, online opinion has turned\ninto a kind of virtual currency for businesses looking to market their\nproducts, identify new opportunities and manage their reputations, therefore\nmany are now looking to the field of sentiment analysis. In this paper, we\npresent a feature-based sentence level approach for Arabic sentiment analysis.\nOur approach is using Arabic idioms/saying phrases lexicon as a key importance\nfor improving the detection of the sentiment polarity in Arabic sentences as\nwell as a number of novels and rich set of linguistically motivated features\ncontextual Intensifiers, contextual Shifter and negation handling), syntactic\nfeatures for conflicting phrases which enhance the sentiment classification\naccuracy. Furthermore, we introduce an automatic expandable wide coverage\npolarity lexicon of Arabic sentiment words. The lexicon is built with\ngold-standard sentiment words as a seed which is manually collected and\nannotated and it expands and detects the sentiment orientation automatically of\nnew sentiment words using synset aggregation technique and free online Arabic\nlexicons and thesauruses. Our data focus on modern standard Arabic (MSA) and\nEgyptian dialectal Arabic tweets and microblogs (hotel reservation, product\nreviews, etc.). The experimental results using our resources and techniques\nwith SVM classifier indicate high performance levels, with accuracies of over\n95%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03239v1",
            "date": "2015-05-13 04:50:27+00:00",
            "title": "Feature selection using Fisher's ratio technique for automatic speech recognition",
            "authors": [
                "Sarika Hegde",
                "K. K. Achary",
                "Surendra Shetty"
            ],
            "abstract": "Automatic Speech Recognition involves mainly two steps; feature extraction\nand classification . Mel Frequency Cepstral Coefficient is used as one of the\nprominent feature extraction techniques in ASR. Usually, the set of all 12 MFCC\ncoefficients is used as the feature vector in the classification step. But the\nquestion is whether the same or improved classification accuracy can be\nachieved by using a subset of 12 MFCC as feature vector. In this paper,\nFisher's ratio technique is used for selecting a subset of 12 MFCC coefficients\nthat contribute more in discriminating a pattern. The selected coefficients are\nused in classification with Hidden Markov Model algorithm. The classification\naccuracies that we get by using 12 coefficients and by using the selected\ncoefficients are compared.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03783v1",
            "date": "2015-05-14 16:21:02+00:00",
            "title": "Rank diversity of languages: Generic behavior in computational linguistics",
            "authors": [
                "Germinal Cocho",
                "Jorge Flores",
                "Carlos Gershenson",
                "Carlos Pineda",
                "Sergio S\u00e1nchez"
            ],
            "abstract": "Statistical studies of languages have focused on the rank-frequency\ndistribution of words. Instead, we introduce here a measure of how word ranks\nchange in time and call this distribution \\emph{rank diversity}. We calculate\nthis diversity for books published in six European languages since 1800, and\nfind that it follows a universal lognormal distribution. Based on the mean and\nstandard deviation associated with the lognormal distribution, we define three\ndifferent word regimes of languages: \"heads\" consist of words which almost do\nnot change their rank in time, \"bodies\" are words of general use, while \"tails\"\nare comprised by context-specific words and vary their rank considerably in\ntime. The heads and bodies reflect the size of language cores identified by\nlinguists for basic communication. We propose a Gaussian random walk model\nwhich reproduces the rank variation of words in time and thus the diversity.\nRank diversity of words can be understood as the result of random variations in\nrank, where the size of the variation depends on the rank itself. We find that\nthe core size is similar for all languages studied.",
            "journal ref": "PLoS ONE 10(4): e0121898 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04197v1",
            "date": "2015-05-15 20:13:16+00:00",
            "title": "Arabic Inquiry-Answer Dialogue Acts Annotation Schema",
            "authors": [
                "AbdelRahim A. Elmadany",
                "Sherif M. Abdou",
                "Mervat Gheith"
            ],
            "abstract": "We present an annotation schema as part of an effort to create a manually\nannotated corpus for Arabic dialogue language understanding including spoken\ndialogue and written \"chat\" dialogue for inquiry-answer domain. The proposed\nschema handles mainly the request and response acts that occurs frequently in\ninquiry-answer debate conversations expressing request services, suggests, and\noffers. We applied the proposed schema on 83 Arabic inquiry-answer dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04342v6",
            "date": "2015-05-17 01:22:00+00:00",
            "title": "Sifting Robotic from Organic Text: A Natural Language Approach for Detecting Automation on Twitter",
            "authors": [
                "Eric M. Clark",
                "Jake Ryland Williams",
                "Chris A. Jones",
                "Richard A. Galbraith",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Twitter, a popular social media outlet, has evolved into a vast source of\nlinguistic data, rich with opinion, sentiment, and discussion. Due to the\nincreasing popularity of Twitter, its perceived potential for exerting social\ninfluence has led to the rise of a diverse community of automatons, commonly\nreferred to as bots. These inorganic and semi-organic Twitter entities can\nrange from the benevolent (e.g., weather-update bots, help-wanted-alert bots)\nto the malevolent (e.g., spamming messages, advertisements, or radical\nopinions). Existing detection algorithms typically leverage meta-data (time\nbetween tweets, number of followers, etc.) to identify robotic accounts. Here,\nwe present a powerful classification scheme that exclusively uses the natural\nlanguage text from organic users to provide a criterion for identifying\naccounts posting automated messages. Since the classifier operates on text\nalone, it is flexible and may be applied to any textual data beyond the\nTwitter-sphere.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04420v1",
            "date": "2015-05-17 17:26:36+00:00",
            "title": "CCG Parsing and Multiword Expressions",
            "authors": [
                "Miryam de Lhoneux"
            ],
            "abstract": "This thesis presents a study about the integration of information about\nMultiword Expressions (MWEs) into parsing with Combinatory Categorial Grammar\n(CCG). We build on previous work which has shown the benefit of adding\ninformation about MWEs to syntactic parsing by implementing a similar pipeline\nwith CCG parsing. More specifically, we collapse MWEs to one token in training\nand test data in CCGbank, a corpus which contains sentences annotated with CCG\nderivations. Our collapsing algorithm however can only deal with MWEs when they\nform a constituent in the data which is one of the limitations of our approach.\n  We study the effect of collapsing training and test data. A parsing effect\ncan be obtained if collapsed data help the parser in its decisions and a\ntraining effect can be obtained if training on the collapsed data improves\nresults. We also collapse the gold standard and show that our model\nsignificantly outperforms the baseline model on our gold standard, which\nindicates that there is a training effect. We show that the baseline model\nperforms significantly better on our gold standard when the data are collapsed\nbefore parsing than when the data are collapsed after parsing which indicates\nthat there is a parsing effect. We show that these results can lead to improved\nperformance on the non-collapsed standard benchmark although we fail to show\nthat it does so significantly. We conclude that despite the limited settings,\nthere are noticeable improvements from using MWEs in parsing. We discuss ways\nin which the incorporation of MWEs into parsing can be improved and hypothesize\nthat this will lead to more substantial results.\n  We finally show that turning the MWE recognition part of the pipeline into an\nexperimental part is a useful thing to do as we obtain different results with\ndifferent recognizers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04891v2",
            "date": "2015-05-19 07:08:10+00:00",
            "title": "Learning Better Word Embedding by Asymmetric Low-Rank Projection of Knowledge Graph",
            "authors": [
                "Fei Tian",
                "Bin Gao",
                "Enhong Chen",
                "Tie-Yan Liu"
            ],
            "abstract": "Word embedding, which refers to low-dimensional dense vector representations\nof natural words, has demonstrated its power in many natural language\nprocessing tasks. However, it may suffer from the inaccurate and incomplete\ninformation contained in the free text corpus as training data. To tackle this\nchallenge, there have been quite a few works that leverage knowledge graphs as\nan additional information source to improve the quality of word embedding.\nAlthough these works have achieved certain success, they have neglected some\nimportant facts about knowledge graphs: (i) many relationships in knowledge\ngraphs are \\emph{many-to-one}, \\emph{one-to-many} or even \\emph{many-to-many},\nrather than simply \\emph{one-to-one}; (ii) most head entities and tail entities\nin knowledge graphs come from very different semantic spaces. To address these\nissues, in this paper, we propose a new algorithm named ProjectNet. ProjecNet\nmodels the relationships between head and tail entities after transforming them\nwith different low-rank projection matrices. The low-rank projection can allow\nnon \\emph{one-to-one} relationships between entities, while different\nprojection matrices for head and tail entities allow them to originate in\ndifferent semantic spaces. The experimental results demonstrate that ProjectNet\nyields more accurate word embedding than previous works, thus leads to clear\nimprovements in various natural language processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.05008v2",
            "date": "2015-05-19 14:21:37+00:00",
            "title": "Boosting Named Entity Recognition with Neural Character Embeddings",
            "authors": [
                "Cicero Nogueira dos Santos",
                "Victor Guimar\u00e3es"
            ],
            "abstract": "Most state-of-the-art named entity recognition (NER) systems rely on\nhandcrafted features and on the output of other NLP tasks such as\npart-of-speech (POS) tagging and text chunking. In this work we propose a\nlanguage-independent NER system that uses automatically learned features only.\nOur approach is based on the CharWNN deep neural network, which uses word-level\nand character-level representations (embeddings) to perform sequential\nclassification. We perform an extensive number of experiments using two\nannotated corpora in two different languages: HAREM I corpus, which contains\ntexts in Portuguese; and the SPA CoNLL-2002 corpus, which contains texts in\nSpanish. Our experimental results shade light on the contribution of neural\ncharacter embeddings for NER. Moreover, we demonstrate that the same neural\nnetwork which has been successfully applied to POS tagging can also achieve\nstate-of-the-art results for language-independet NER, using the same\nhyperparameters, and without any handcrafted features. For the HAREM I corpus,\nCharWNN outperforms the state-of-the-art system by 7.9 points in the F1-score\nfor the total scenario (ten NE classes), and by 7.2 points in the F1 for the\nselective scenario (five NE classes).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.05253v2",
            "date": "2015-05-20 05:57:32+00:00",
            "title": "Knowlege Graph Embedding by Flexible Translation",
            "authors": [
                "Jun Feng",
                "Mantong Zhou",
                "Yu Hao",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Knowledge graph embedding refers to projecting entities and relations in\nknowledge graph into continuous vector spaces. State-of-the-art methods, such\nas TransE, TransH, and TransR build embeddings by treating relation as\ntranslation from head entity to tail entity. However, previous models can not\ndeal with reflexive/one-to-many/many-to-one/many-to-many relations properly, or\nlack of scalability and efficiency. Thus, we propose a novel method, flexible\ntranslation, named TransF, to address the above issues. TransF regards relation\nas translation between head entity vector and tail entity vector with flexible\nmagnitude. To evaluate the proposed model, we conduct link prediction and\ntriple classification on benchmark datasets. Experimental results show that our\nmethod remarkably improve the performance compared with several\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.05841v1",
            "date": "2015-05-21 18:57:34+00:00",
            "title": "Translation Memory Retrieval Methods",
            "authors": [
                "Michael Bloodgood",
                "Benjamin Strauss"
            ],
            "abstract": "Translation Memory (TM) systems are one of the most widely used translation\ntechnologies. An important part of TM systems is the matching algorithm that\ndetermines what translations get retrieved from the bank of available\ntranslations to assist the human translator. Although detailed accounts of the\nmatching algorithms used in commercial systems can't be found in the\nliterature, it is widely believed that edit distance algorithms are used. This\npaper investigates and evaluates the use of several matching algorithms,\nincluding the edit distance algorithm that is believed to be at the heart of\nmost modern commercial TM systems. This paper presents results showing how well\nvarious matching algorithms correlate with human judgments of helpfulness\n(collected via crowdsourcing with Amazon's Mechanical Turk). A new algorithm\nbased on weighted n-gram precision that can be adjusted for translator length\npreferences consistently returns translations judged to be most helpful by\ntranslators for multiple domains and language pairs.",
            "journal ref": "In Proceedings of the 14th Conference of the European Chapter of\n  the Association for Computational Linguistics, pages 202-210, Gothenburg,\n  Sweden, April 2014. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.05899v1",
            "date": "2015-05-21 20:49:32+00:00",
            "title": "The IBM 2015 English Conversational Telephone Speech Recognition System",
            "authors": [
                "George Saon",
                "Hong-Kwang J. Kuo",
                "Steven Rennie",
                "Michael Picheny"
            ],
            "abstract": "We describe the latest improvements to the IBM English conversational\ntelephone speech recognition system. Some of the techniques that were found\nbeneficial are: maxout networks with annealed dropout rates; networks with a\nvery large number of outputs trained on 2000 hours of data; joint modeling of\npartially unfolded recurrent neural networks and convolutional nets by\ncombining the bottleneck and output layers and retraining the resulting model;\nand lastly, sophisticated language model rescoring with exponential and neural\nnetwork LMs. These techniques result in an 8.0% word error rate on the\nSwitchboard part of the Hub5-2000 evaluation test set which is 23% relative\nbetter than our previous best published result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06228v1",
            "date": "2015-05-22 21:12:35+00:00",
            "title": "Keyphrase Based Evaluation of Automatic Text Summarization",
            "authors": [
                "Fatma Elghannam",
                "Tarek El-Shishtawy"
            ],
            "abstract": "The development of methods to deal with the informative contents of the text\nunits in the matching process is a major challenge in automatic summary\nevaluation systems that use fixed n-gram matching. The limitation causes\ninaccurate matching between units in a peer and reference summaries. The\npresent study introduces a new Keyphrase based Summary Evaluator KpEval for\nevaluating automatic summaries. The KpEval relies on the keyphrases since they\nconvey the most important concepts of a text. In the evaluation process, the\nkeyphrases are used in their lemma form as the matching text unit. The system\nwas applied to evaluate different summaries of Arabic multi-document data set\npresented at TAC2011. The results showed that the new evaluation technique\ncorrelates well with the known evaluation systems: Rouge1, Rouge2, RougeSU4,\nand AutoSummENG MeMoG. KpEval has the strongest correlation with AutoSummENG\nMeMoG, Pearson and spearman correlation coefficient measures are 0.8840, 0.9667\nrespectively.",
            "journal ref": "International Journal of Computer Applications 117(7):5-8, May\n  2015. ISBN : 973-93-80886-51-2",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06816v5",
            "date": "2015-05-26 06:19:18+00:00",
            "title": "Representing Meaning with a Combination of Logical and Distributional Models",
            "authors": [
                "I. Beltagy",
                "Stephen Roller",
                "Pengxiang Cheng",
                "Katrin Erk",
                "Raymond J. Mooney"
            ],
            "abstract": "NLP tasks differ in the semantic information they require, and at this time\nno single se- mantic representation fulfills all requirements. Logic-based\nrepresentations characterize sentence structure, but do not capture the graded\naspect of meaning. Distributional models give graded similarity ratings for\nwords and phrases, but do not capture sentence structure in the same detail as\nlogic-based approaches. So it has been argued that the two are complementary.\nWe adopt a hybrid approach that combines logic-based and distributional\nsemantics through probabilistic logic inference in Markov Logic Networks\n(MLNs). In this paper, we focus on the three components of a practical system\nintegrating logical and distributional models: 1) Parsing and task\nrepresentation is the logic-based part where input problems are represented in\nprobabilistic logic. This is quite different from representing them in standard\nfirst-order logic. 2) For knowledge base construction we form weighted\ninference rules. We integrate and compare distributional information with other\nsources, notably WordNet and an existing paraphrase collection. In particular,\nwe use our system to evaluate distributional lexical entailment approaches. We\nuse a variant of Robinson resolution to determine the necessary inference\nrules. More sources can easily be added by mapping them to logical rules; our\nsystem learns a resource-specific weight that corrects for scaling differences\nbetween resources. 3) In discussing probabilistic inference, we show how to\nsolve the inference problems efficiently. To evaluate our approach, we use the\ntask of textual entailment (RTE), which can utilize the strengths of both\nlogic-based and distributional representations. In particular we focus on the\nSICK dataset, where we achieve state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.07184v1",
            "date": "2015-05-27 04:02:56+00:00",
            "title": "Unsupervised Cross-Domain Word Representation Learning",
            "authors": [
                "Danushka Bollegala",
                "Takanori Maehara",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Meaning of a word varies from one domain to another. Despite this important\ndomain dependence in word semantics, existing word representation learning\nmethods are bound to a single domain. Given a pair of\n\\emph{source}-\\emph{target} domains, we propose an unsupervised method for\nlearning domain-specific word representations that accurately capture the\ndomain-specific aspects of word semantics. First, we select a subset of\nfrequent words that occur in both domains as \\emph{pivots}. Next, we optimize\nan objective function that enforces two constraints: (a) for both source and\ntarget domain documents, pivots that appear in a document must accurately\npredict the co-occurring non-pivots, and (b) word representations learnt for\npivots must be similar in the two domains. Moreover, we propose a method to\nperform domain adaptation using the learnt word representations. Our proposed\nmethod significantly outperforms competitive baselines including the\nstate-of-the-art domain-insensitive word representations, and reports best\nsentiment classification accuracies for all domain-pairs in a benchmark\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.07599v3",
            "date": "2015-05-28 08:54:13+00:00",
            "title": "Overview of the NLPCC 2015 Shared Task: Chinese Word Segmentation and POS Tagging for Micro-blog Texts",
            "authors": [
                "Xipeng Qiu",
                "Peng Qian",
                "Liusong Yin",
                "Shiyu Wu",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we give an overview for the shared task at the 4th CCF\nConference on Natural Language Processing \\& Chinese Computing (NLPCC 2015):\nChinese word segmentation and part-of-speech (POS) tagging for micro-blog\ntexts. Different with the popular used newswire datasets, the dataset of this\nshared task consists of the relatively informal micro-texts. The shared task\nhas two sub-tasks: (1) individual Chinese word segmentation and (2) joint\nChinese word segmentation and POS Tagging. Each subtask has three tracks to\ndistinguish the systems with different resources. We first introduce the\ndataset and task, then we characterize the different approaches of the\nparticipating systems, report the test results, and provide a overview analysis\nof these results. An online system is available for open registration and\nevaluation at http://nlp.fudan.edu.cn/nlpcc2015.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.07931v1",
            "date": "2015-05-29 06:11:00+00:00",
            "title": "Supervised Fine Tuning for Word Embedding with Integrated Knowledge",
            "authors": [
                "Xuefeng Yang",
                "Kezhi Mao"
            ],
            "abstract": "Learning vector representation for words is an important research field which\nmay benefit many natural language processing tasks. Two limitations exist in\nnearly all available models, which are the bias caused by the context\ndefinition and the lack of knowledge utilization. They are difficult to tackle\nbecause these algorithms are essentially unsupervised learning approaches.\nInspired by deep learning, the authors propose a supervised framework for\nlearning vector representation of words to provide additional supervised fine\ntuning after unsupervised learning. The framework is knowledge rich approacher\nand compatible with any numerical vectors word representation. The authors\nperform both intrinsic evaluation like attributional and relational similarity\nprediction and extrinsic evaluations like the sentence completion and sentiment\nanalysis. Experiments results on 6 embeddings and 4 tasks with 10 datasets show\nthat the proposed fine tuning framework may significantly improve the quality\nof the vector representation of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.08149v3",
            "date": "2015-05-29 19:06:42+00:00",
            "title": "Modeling meaning: computational interpreting and understanding of natural language fragments",
            "authors": [
                "Michael Kapustin",
                "Pavlo Kapustin"
            ],
            "abstract": "In this introductory article we present the basics of an approach to\nimplementing computational interpreting of natural language aiming to model the\nmeanings of words and phrases. Unlike other approaches, we attempt to define\nthe meanings of text fragments in a composable and computer interpretable way.\nWe discuss models and ideas for detecting different types of semantic\nincomprehension and choosing the interpretation that makes most sense in a\ngiven context. Knowledge representation is designed for handling\ncontext-sensitive and uncertain / imprecise knowledge, and for easy\naccommodation of new information. It stores quantitative information capturing\nthe essence of the concepts, because it is crucial for working with natural\nlanguage understanding and reasoning. Still, the representation is general\nenough to allow for new knowledge to be learned, and even generated by the\nsystem. The article concludes by discussing some reasoning-related topics:\npossible approaches to generation of new abstract concepts, and describing\nsituations and concepts in words (e.g. for specifying interpretation\ndifficulties).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.00705v1",
            "date": "2015-09-02 13:54:57+00:00",
            "title": "Analysis of Communication Pattern with Scammers in Enron Corpus",
            "authors": [
                "Dinesh Balaji Sashikanth"
            ],
            "abstract": "This paper is an exploratory analysis into fraud detection taking Enron email\ncorpus as the case study. The paper posits conclusions like strict servitude\nand unquestionable faith among employees as breeding grounds for sham among\nhigher executives. We also try to infer on the nature of communication between\nfraudulent employees and between non- fraudulent-fraudulent employees",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.00963v1",
            "date": "2015-09-03 07:23:10+00:00",
            "title": "On TimeML-Compliant Temporal Expression Extraction in Turkish",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Do\u011fan K\u00fc\u00e7\u00fck"
            ],
            "abstract": "It is commonly acknowledged that temporal expression extractors are important\ncomponents of larger natural language processing systems like information\nretrieval and question answering systems. Extraction and normalization of\ntemporal expressions in Turkish has not been given attention so far except the\nextraction of some date and time expressions within the course of named entity\nrecognition. As TimeML is the current standard of temporal expression and event\nannotation in natural language texts, in this paper, we present an analysis of\ntemporal expressions in Turkish based on the related TimeML classification\n(i.e., date, time, duration, and set expressions). We have created a lexicon\nfor Turkish temporal expressions and devised considerably wide-coverage\npatterns using the lexical classes as the building blocks. We believe that the\nproposed patterns, together with convenient normalization rules, can be readily\nused by prospective temporal expression extraction tools for Turkish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01007v3",
            "date": "2015-09-03 09:39:36+00:00",
            "title": "Encoding Prior Knowledge with Eigenword Embeddings",
            "authors": [
                "Dominique Osborne",
                "Shashi Narayan",
                "Shay B. Cohen"
            ],
            "abstract": "Canonical correlation analysis (CCA) is a method for reducing the dimension\nof data represented using two views. It has been previously used to derive word\nembeddings, where one view indicates a word, and the other view indicates its\ncontext. We describe a way to incorporate prior knowledge into CCA, give a\ntheoretical justification for it, and test it by deriving word embeddings and\nevaluating them on a myriad of datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01310v1",
            "date": "2015-09-03 23:57:46+00:00",
            "title": "The influence of Chunking on Dependency Crossing and Distance",
            "authors": [
                "Qian Lu",
                "Chunshan Xu",
                "Haitao Liu"
            ],
            "abstract": "This paper hypothesizes that chunking plays important role in reducing\ndependency distance and dependency crossings. Computer simulations, when\ncompared with natural languages,show that chunking reduces mean dependency\ndistance (MDD) of a linear sequence of nodes (constrained by continuity or\nprojectivity) to that of natural languages. More interestingly, chunking alone\nbrings about less dependency crossings as well, though having failed to reduce\nthem, to such rarity as found in human languages. These results suggest that\nchunking may play a vital role in the minimization of dependency distance, and\na somewhat contributing role in the rarity of dependency crossing. In addition,\nthe results point to a possibility that the rarity of dependency crossings is\nnot a mere side-effect of minimization of dependency distance, but a linguistic\nphenomenon with its own motivations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01692v4",
            "date": "2015-09-05 11:23:44+00:00",
            "title": "Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning",
            "authors": [
                "Ekaterina Vylomova",
                "Laura Rimell",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Recent work on word embeddings has shown that simple vector subtraction over\npre-trained embeddings is surprisingly effective at capturing different lexical\nrelations, despite lacking explicit supervision. Prior work has evaluated this\nintriguing result using a word analogy prediction formulation and hand-selected\nrelations, but the generality of the finding over a broader range of lexical\nrelation types and different learning settings has not been evaluated. In this\npaper, we carry out such an evaluation in two learning settings: (1) spectral\nclustering to induce word relations, and (2) supervised learning to classify\nvector differences into relation types. We find that word embeddings capture a\nsurprising amount of information, and that, under suitable supervised training,\nvector subtraction generalises well to a broad range of relations, including\nover unseen lexical items.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01722v2",
            "date": "2015-09-05 17:52:16+00:00",
            "title": "A commentary on \"The now-or-never bottleneck: a fundamental constraint on language\", by Christiansen and Chater (2016)",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "In a recent article, Christiansen and Chater (2016) present a fundamental\nconstraint on language, i.e. a now-or-never bottleneck that arises from our\nfleeting memory, and explore its implications, e.g., chunk-and-pass processing,\noutlining a framework that promises to unify different areas of research. Here\nwe explore additional support for this constraint and suggest further\nconnections from quantitative linguistics and information theory.",
            "journal ref": "Glottometrics 38, 107-111 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01899v2",
            "date": "2015-09-07 04:40:14+00:00",
            "title": "Integrate Document Ranking Information into Confidence Measure Calculation for Spoken Term Detection",
            "authors": [
                "Quan Liu",
                "Wu Guo",
                "Zhen-Hua Ling"
            ],
            "abstract": "This paper proposes an algorithm to improve the calculation of confidence\nmeasure for spoken term detection (STD). Given an input query term, the\nalgorithm first calculates a measurement named document ranking weight for each\ndocument in the speech database to reflect its relevance with the query term by\nsumming all the confidence measures of the hypothesized term occurrences in\nthis document. The confidence measure of each term occurrence is then\nre-estimated through linear interpolation with the calculated document ranking\nweight to improve its reliability by integrating document-level information.\nExperiments are conducted on three standard STD tasks for Tamil, Vietnamese and\nEnglish respectively. The experimental results all demonstrate that the\nproposed algorithm achieves consistent improvements over the state-of-the-art\nmethod for confidence measure calculation. Furthermore, this algorithm is still\neffective even if a high accuracy speech recognizer is not available, which\nmakes it applicable for the languages with limited speech resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01938v1",
            "date": "2015-09-07 07:54:17+00:00",
            "title": "Exploiting Out-of-Domain Data Sources for Dialectal Arabic Statistical Machine Translation",
            "authors": [
                "Katrin Kirchhoff",
                "Bing Zhao",
                "Wen Wang"
            ],
            "abstract": "Statistical machine translation for dialectal Arabic is characterized by a\nlack of data since data acquisition involves the transcription and translation\nof spoken language. In this study we develop techniques for extracting parallel\ndata for one particular dialect of Arabic (Iraqi Arabic) from out-of-domain\ncorpora in different dialects of Arabic or in Modern Standard Arabic. We\ncompare two different data selection strategies (cross-entropy based and\nsubmodular selection) and demonstrate that a very small but highly targeted\namount of found data can improve the performance of a baseline machine\ntranslation system. We furthermore report on preliminary experiments on using\nautomatically translated speech data as additional training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.02208v1",
            "date": "2015-09-07 22:23:01+00:00",
            "title": "Unsupervised Discovery of Linguistic Structure Including Two-level Acoustic Patterns Using Three Cascaded Stages of Iterative Optimization",
            "authors": [
                "Cheng-Tao Chung",
                "Chun-an Chan",
                "Lin-shan Lee"
            ],
            "abstract": "Techniques for unsupervised discovery of acoustic patterns are getting\nincreasingly attractive, because huge quantities of speech data are becoming\navailable but manual annotations remain hard to acquire. In this paper, we\npropose an approach for unsupervised discovery of linguistic structure for the\ntarget spoken language given raw speech data. This linguistic structure\nincludes two-level (subword-like and word-like) acoustic patterns, the lexicon\nof word-like patterns in terms of subword-like patterns and the N-gram language\nmodel based on word-like patterns. All patterns, models, and parameters can be\nautomatically learned from the unlabelled speech corpus. This is achieved by an\ninitialization step followed by three cascaded stages for acoustic, linguistic,\nand lexical iterative optimization. The lexicon of word-like patterns defines\nallowed consecutive sequence of HMMs for subword-like patterns. In each\niteration, model training and decoding produces updated labels from which the\nlexicon and HMMs can be further updated. In this way, model parameters and\ndecoded labels are respectively optimized in each iteration, and the knowledge\nabout the linguistic structure is learned gradually layer after layer. The\nproposed approach was tested in preliminary experiments on a corpus of Mandarin\nbroadcast news, including a task of spoken term detection with performance\ncompared to a parallel test using models trained in a supervised way. Results\nshow that the proposed system not only yields reasonable performance on its\nown, but is also complimentary to existing large vocabulary ASR systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.02213v1",
            "date": "2015-09-07 22:40:31+00:00",
            "title": "Unsupervised Spoken Term Detection with Spoken Queries by Multi-level Acoustic Patterns with Varying Model Granularity",
            "authors": [
                "Cheng-Tao Chung",
                "Chun-an Chan",
                "Lin-shan Lee"
            ],
            "abstract": "This paper presents a new approach for unsupervised Spoken Term Detection\nwith spoken queries using multiple sets of acoustic patterns automatically\ndiscovered from the target corpus. The different pattern HMM\nconfigurations(number of states per model, number of distinct models, number of\nGaussians per state)form a three-dimensional model granularity space. Different\nsets of acoustic patterns automatically discovered on different points properly\ndistributed over this three-dimensional space are complementary to one another,\nthus can jointly capture the characteristics of the spoken terms. By\nrepresenting the spoken content and spoken query as sequences of acoustic\npatterns, a series of approaches for matching the pattern index sequences while\nconsidering the signal variations are developed. In this way, not only the\non-line computation load can be reduced, but the signal distributions caused by\ndifferent speakers and acoustic conditions can be reasonably taken care of. The\nresults indicate that this approach significantly outperformed the unsupervised\nfeature-based DTW baseline by 16.16\\% in mean average precision on the TIMIT\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.02217v1",
            "date": "2015-09-07 22:56:49+00:00",
            "title": "Enhancing Automatically Discovered Multi-level Acoustic Patterns Considering Context Consistency With Applications in Spoken Term Detection",
            "authors": [
                "Cheng-Tao Chung",
                "Wei-Ning Hsu",
                "Cheng-Yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "This paper presents a novel approach for enhancing the multiple sets of\nacoustic patterns automatically discovered from a given corpus. In a previous\nwork it was proposed that different HMM configurations (number of states per\nmodel, number of distinct models) for the acoustic patterns form a\ntwo-dimensional space. Multiple sets of acoustic patterns automatically\ndiscovered with the HMM configurations properly located on different points\nover this two-dimensional space were shown to be complementary to one another,\njointly capturing the characteristics of the given corpus. By representing the\ngiven corpus as sequences of acoustic patterns on different HMM sets, the\npattern indices in these sequences can be relabeled considering the context\nconsistency across the different sequences. Good improvements were observed in\npreliminary experiments of pattern spoken term detection (STD) performed on\nboth TIMIT and Mandarin Broadcast News with such enhanced patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.02301v3",
            "date": "2015-09-08 09:43:13+00:00",
            "title": "Probabilistic Bag-Of-Hyperlinks Model for Entity Linking",
            "authors": [
                "Octavian-Eugen Ganea",
                "Marina Ganea",
                "Aurelien Lucchi",
                "Carsten Eickhoff",
                "Thomas Hofmann"
            ],
            "abstract": "Many fundamental problems in natural language processing rely on determining\nwhat entities appear in a given text. Commonly referenced as entity linking,\nthis step is a fundamental component of many NLP tasks such as text\nunderstanding, automatic summarization, semantic search or machine translation.\nName ambiguity, word polysemy, context dependencies and a heavy-tailed\ndistribution of entities contribute to the complexity of this problem.\n  We here propose a probabilistic approach that makes use of an effective\ngraphical model to perform collective entity disambiguation. Input mentions\n(i.e.,~linkable token spans) are disambiguated jointly across an entire\ndocument by combining a document-level prior of entity co-occurrences with\nlocal information captured from mentions and their surrounding context. The\nmodel is based on simple sufficient statistics extracted from data, thus\nrelying on few parameters to be learned.\n  Our method does not require extensive feature engineering, nor an expensive\ntraining procedure. We use loopy belief propagation to perform approximate\ninference. The low complexity of our model makes this step sufficiently fast\nfor real-time usage. We demonstrate the accuracy of our approach on a wide\nrange of benchmark datasets, showing that it matches, and in many cases\noutperforms, existing state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.02412v1",
            "date": "2015-09-08 15:29:23+00:00",
            "title": "Unsupervised Domain Discovery using Latent Dirichlet Allocation for Acoustic Modelling in Speech Recognition",
            "authors": [
                "Mortaza Doulaty",
                "Oscar Saz",
                "Thomas Hain"
            ],
            "abstract": "Speech recognition systems are often highly domain dependent, a fact widely\nreported in the literature. However the concept of domain is complex and not\nbound to clear criteria. Hence it is often not evident if data should be\nconsidered to be out-of-domain. While both acoustic and language models can be\ndomain specific, work in this paper concentrates on acoustic modelling. We\npresent a novel method to perform unsupervised discovery of domains using\nLatent Dirichlet Allocation (LDA) modelling. Here a set of hidden domains is\nassumed to exist in the data, whereby each audio segment can be considered to\nbe a weighted mixture of domain properties. The classification of audio\nsegments into domains allows the creation of domain specific acoustic models\nfor automatic speech recognition. Experiments are conducted on a dataset of\ndiverse speech data covering speech from radio and TV broadcasts, telephone\nconversations, meetings, lectures and read speech, with a joint training set of\n60 hours and a test set of 6 hours. Maximum A Posteriori (MAP) adaptation to\nLDA based domains was shown to yield relative Word Error Rate (WER)\nimprovements of up to 16% relative, compared to pooled training, and up to 10%,\ncompared with models adapted with human-labelled prior domain knowledge.",
            "journal ref": "16th Interspeech.Proc. (2015) 3640-3644, Dresden, Germany",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.03208v1",
            "date": "2015-07-14 02:47:40+00:00",
            "title": "Towards Understanding Egyptian Arabic Dialogues",
            "authors": [
                "Abdelrahim A Elmadany",
                "Sherif M Abdou",
                "Mervat Gheith"
            ],
            "abstract": "Labelling of user's utterances to understanding his attends which called\nDialogue Act (DA) classification, it is considered the key player for dialogue\nlanguage understanding layer in automatic dialogue systems. In this paper, we\nproposed a novel approach to user's utterances labeling for Egyptian\nspontaneous dialogues and Instant Messages using Machine Learning (ML) approach\nwithout relying on any special lexicons, cues, or rules. Due to the lack of\nEgyptian dialect dialogue corpus, the system evaluated by multi-genre corpus\nincludes 4725 utterances for three domains, which are collected and annotated\nmanually from Egyptian call-centers. The system achieves F1 scores of 70. 36%\noverall domains.",
            "journal ref": "International Journal of Computer Applications 120(220, PP 7-12,\n  June 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.03488v2",
            "date": "2015-09-11 13:05:15+00:00",
            "title": "Verbs Taking Clausal and Non-Finite Arguments as Signals of Modality - Revisiting the Issue of Meaning Grounded in Syntax",
            "authors": [
                "Judith Eckle-Kohler"
            ],
            "abstract": "We revisit Levin's theory about the correspondence of verb meaning and syntax\nand infer semantic classes from a large syntactic classification of more than\n600 German verbs taking clausal and non-finite arguments. Grasping the meaning\ncomponents of Levin-classes is known to be hard. We address this challenge by\nsetting up a multi-perspective semantic characterization of the inferred\nclasses. To this end, we link the inferred classes and their English\ntranslation to independently constructed semantic classes in three different\nlexicons - the German wordnet GermaNet, VerbNet and FrameNet - and perform a\ndetailed analysis and evaluation of the resulting German-English classification\n(available at www.ukp.tu-darmstadt.de/modality-verbclasses/).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.03611v2",
            "date": "2015-09-11 19:07:49+00:00",
            "title": "A Parallel Corpus of Translationese",
            "authors": [
                "Ella Rabinovich",
                "Shuly Wintner",
                "Ofek Luis Lewinsohn"
            ],
            "abstract": "We describe a set of bilingual English--French and English--German parallel\ncorpora in which the direction of translation is accurately and reliably\nannotated. The corpora are diverse, consisting of parliamentary proceedings,\nliterary works, transcriptions of TED talks and political commentary. They will\nbe instrumental for research of translationese and its applications to (human\nand machine) translation; specifically, they can be used for the task of\ntranslationese identification, a research direction that enjoys a growing\ninterest in recent years. To validate the quality and reliability of the\ncorpora, we replicated previous results of supervised and unsupervised\nidentification of translationese, and further extended the experiments to\nadditional datasets and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.03739v1",
            "date": "2015-09-12 12:59:05+00:00",
            "title": "Improving distant supervision using inference learning",
            "authors": [
                "Roland Roller",
                "Eneko Agirre",
                "Aitor Soroa",
                "Mark Stevenson"
            ],
            "abstract": "Distant supervision is a widely applied approach to automatic training of\nrelation extraction systems and has the advantage that it can generate large\namounts of labelled data with minimal effort. However, this data may contain\nerrors and consequently systems trained using distant supervision tend not to\nperform as well as those based on manually labelled data. This work proposes a\nnovel method for detecting potential false negative training examples using a\nknowledge inference method. Results show that our approach improves the\nperformance of relation extraction systems trained using distantly supervised\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.03870v1",
            "date": "2015-09-13 16:58:41+00:00",
            "title": "The USFD Spoken Language Translation System for IWSLT 2014",
            "authors": [
                "Raymond W. M. Ng",
                "Mortaza Doulaty",
                "Rama Doddipatla",
                "Wilker Aziz",
                "Kashif Shah",
                "Oscar Saz",
                "Madina Hasan",
                "Ghada AlHarbi",
                "Lucia Specia",
                "Thomas Hain"
            ],
            "abstract": "The University of Sheffield (USFD) participated in the International Workshop\nfor Spoken Language Translation (IWSLT) in 2014. In this paper, we will\nintroduce the USFD SLT system for IWSLT. Automatic speech recognition (ASR) is\nachieved by two multi-pass deep neural network systems with adaptation and\nrescoring techniques. Machine translation (MT) is achieved by a phrase-based\nsystem. The USFD primary system incorporates state-of-the-art ASR and MT\ntechniques and gives a BLEU score of 23.45 and 14.75 on the English-to-French\nand English-to-German speech-to-text translation task with the IWSLT 2014 data.\nThe USFD contrastive systems explore the integration of ASR and MT by using a\nquality estimation system to rescore the ASR outputs, optimising towards better\ntranslation. This gives a further 0.54 and 0.26 BLEU improvement respectively\non the IWSLT 2012 and 2014 evaluation data.",
            "journal ref": "Proc. of 11th International Workshop on Spoken Language\n  Translation (SLT 2014) 86-91, Lake Tahoe, USA, December 4th and 5th, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.04385v1",
            "date": "2015-09-12 06:07:16+00:00",
            "title": "Kannada named entity recognition and classification (nerc) based on multinomial na\u00efve bayes (mnb) classifier",
            "authors": [
                "S. Amarappa",
                "S. V. Sathyanarayana"
            ],
            "abstract": "Named Entity Recognition and Classification (NERC) is a process of\nidentification of proper nouns in the text and classification of those nouns\ninto certain predefined categories like person name, location, organization,\ndate, and time etc. NERC in Kannada is an essential and challenging task. The\naim of this work is to develop a novel model for NERC, based on Multinomial\nNa\\\"ive Bayes (MNB) Classifier. The Methodology adopted in this paper is based\non feature extraction of training corpus, by using term frequency, inverse\ndocument frequency and fitting them to a tf-idf-vectorizer. The paper discusses\nthe various issues in developing the proposed model. The details of\nimplementation and performance evaluation are discussed. The experiments are\nconducted on a training corpus of size 95,170 tokens and test corpus of 5,000\ntokens. It is observed that the model works with Precision, Recall and\nF1-measure of 83%, 79% and 81% respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.04393v1",
            "date": "2015-09-15 04:29:50+00:00",
            "title": "Dependency length minimization: Puzzles and Promises",
            "authors": [
                "Haitao Liu",
                "Chunshan Xu",
                "Junying Liang"
            ],
            "abstract": "In the recent issue of PNAS, Futrell et al. claims that their study of 37\nlanguages gives the first large scale cross-language evidence for Dependency\nLength Minimization, which is an overstatement that ignores similar previous\nresearches. In addition,this study seems to pay no attention to factors like\nthe uniformity of genres,which weakens the validity of the argument that DLM is\nuniversal. Another problem is that this study sets the baseline random language\nas projective, which fails to truly uncover the difference between natural\nlanguage and random language, since projectivity is an important feature of\nmany natural languages. Finally, the paper contends an \"apparent relationship\nbetween head finality and dependency length\" despite the lack of an explicit\nstatistical comparison, which renders this conclusion rather hasty and\nimproper.",
            "journal ref": "Glottometrics 2016, 33: 35 - 38",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.04473v1",
            "date": "2015-09-15 10:03:35+00:00",
            "title": "Splitting Compounds by Semantic Analogy",
            "authors": [
                "Joachim Daiber",
                "Lautaro Quiroz",
                "Roger Wechsler",
                "Stella Frank"
            ],
            "abstract": "Compounding is a highly productive word-formation process in some languages\nthat is often problematic for natural language processing applications. In this\npaper, we investigate whether distributional semantics in the form of word\nembeddings can enable a deeper, i.e., more knowledge-rich, processing of\ncompounds than the standard string-based methods. We present an unsupervised\napproach that exploits regularities in the semantic vector space (based on\nanalogies such as \"bookshop is to shop as bookshelf is to shelf\") to produce\ncompound analyses of high quality. A subsequent compound splitting algorithm\nbased on these analyses is highly effective, particularly for ambiguous\ncompounds. German to English machine translation experiments show that this\nsemantic analogy-based compound splitter leads to better translations than a\ncommonly used frequency-based method.",
            "journal ref": "Proceedings of the 1st Deep Machine Translation Workshop. Prague,\n  Czech Republic. 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05488v7",
            "date": "2015-09-18 02:30:17+00:00",
            "title": "TransG : A Generative Mixture Model for Knowledge Graph Embedding",
            "authors": [
                "Han Xiao",
                "Minlie Huang",
                "Yu Hao",
                "Xiaoyan Zhu"
            ],
            "abstract": "Recently, knowledge graph embedding, which projects symbolic entities and\nrelations into continuous vector space, has become a new, hot topic in\nartificial intelligence. This paper addresses a new issue of multiple relation\nsemantics that a relation may have multiple meanings revealed by the entity\npairs associated with the corresponding triples, and proposes a novel Gaussian\nmixture model for embedding, TransG. The new model can discover latent\nsemantics for a relation and leverage a mixture of relation component vectors\nfor embedding a fact triple. To the best of our knowledge, this is the first\ngenerative model for knowledge graph embedding, which is able to deal with\nmultiple relation semantics. Extensive experiments show that the proposed model\nachieves substantial improvements against the state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05490v2",
            "date": "2015-09-18 02:40:07+00:00",
            "title": "TransA: An Adaptive Approach for Knowledge Graph Embedding",
            "authors": [
                "Han Xiao",
                "Minlie Huang",
                "Yu Hao",
                "Xiaoyan Zhu"
            ],
            "abstract": "Knowledge representation is a major topic in AI, and many studies attempt to\nrepresent entities and relations of knowledge base in a continuous vector\nspace. Among these attempts, translation-based methods build entity and\nrelation vectors by minimizing the translation loss from a head entity to a\ntail one. In spite of the success of these methods, translation-based methods\nalso suffer from the oversimplified loss metric, and are not competitive enough\nto model various and complex entities/relations in knowledge bases. To address\nthis issue, we propose \\textbf{TransA}, an adaptive metric approach for\nembedding, utilizing the metric learning ideas to provide a more flexible\nembedding method. Experiments are conducted on the benchmark datasets and our\nproposed method makes significant and consistent improvements over the\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05517v1",
            "date": "2015-09-18 06:56:38+00:00",
            "title": "A Light Sliding-Window Part-of-Speech Tagger for the Apertium Free/Open-Source Machine Translation Platform",
            "authors": [
                "Gang Chen",
                "Mikel L. Forcada"
            ],
            "abstract": "This paper describes a free/open-source implementation of the light\nsliding-window (LSW) part-of-speech tagger for the Apertium free/open-source\nmachine translation platform. Firstly, the mechanism and training process of\nthe tagger are reviewed, and a new method for incorporating linguistic rules is\nproposed. Secondly, experiments are conducted to compare the performances of\nthe tagger under different window settings, with or without Apertium-style\n\"forbid\" rules, with or without Constraint Grammar, and also with respect to\nthe traditional HMM tagger in Apertium.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.06053v1",
            "date": "2015-09-20 21:01:51+00:00",
            "title": "Early text classification: a Naive solution",
            "authors": [
                "Hugo Jair Escalante",
                "Manuel Montes-y-G\u00f3mez",
                "Luis Villase\u00f1or-Pineda",
                "Marcelo Luis Errecalde"
            ],
            "abstract": "Text classification is a widely studied problem, and it can be considered\nsolved for some domains and under certain circumstances. There are scenarios,\nhowever, that have received little or no attention at all, despite its\nrelevance and applicability. One of such scenarios is early text\nclassification, where one needs to know the category of a document by using\npartial information only. A document is processed as a sequence of terms, and\nthe goal is to devise a method that can make predictions as fast as possible.\nThe importance of this variant of the text classification problem is evident in\ndomains like sexual predator detection, where one wants to identify an offender\nas early as possible. This paper analyzes the suitability of the standard naive\nBayes classifier for approaching this problem. Specifically, we assess its\nperformance when classifying documents after seeing an increasingly number of\nterms. A simple modification to the standard naive Bayes implementation allows\nus to make predictions with partial information. To the best of our knowledge\nnaive Bayes has not been used for this purpose before. Throughout an extensive\nexperimental evaluation we show the effectiveness of the classifier for early\ntext classification. What is more, we show that this simple solution is very\ncompetitive when compared with state of the art methodologies that are more\nelaborated. We foresee our work will pave the way for the development of more\neffective early text classification techniques based in the naive Bayes\nformulation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.06928v2",
            "date": "2015-09-23 11:41:10+00:00",
            "title": "Automatic Dialect Detection in Arabic Broadcast Speech",
            "authors": [
                "Ahmed Ali",
                "Najim Dehak",
                "Patrick Cardinal",
                "Sameer Khurana",
                "Sree Harsha Yella",
                "James Glass",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "We investigate different approaches for dialect identification in Arabic\nbroadcast speech, using phonetic, lexical features obtained from a speech\nrecognition system, and acoustic features using the i-vector framework. We\nstudied both generative and discriminate classifiers, and we combined these\nfeatures using a multi-class Support Vector Machine (SVM). We validated our\nresults on an Arabic/English language identification task, with an accuracy of\n100%. We used these features in a binary classifier to discriminate between\nModern Standard Arabic (MSA) and Dialectal Arabic, with an accuracy of 100%. We\nfurther report results using the proposed method to discriminate between the\nfive most widely used dialects of Arabic: namely Egyptian, Gulf, Levantine,\nNorth African, and MSA, with an accuracy of 52%. We discuss dialect\nidentification errors in the context of dialect code-switching between\nDialectal Arabic and MSA, and compare the error pattern between manually\nlabeled data, and the output from our classifier. We also release the train and\ntest data as standard corpus for dialect identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.06937v1",
            "date": "2015-09-23 12:09:07+00:00",
            "title": "Fully automatic multi-language translation with a catalogue of phrases - successful employment for the Swiss avalanche bulletin",
            "authors": [
                "Kurt Winkler",
                "Tobias Kuhn"
            ],
            "abstract": "The Swiss avalanche bulletin is produced twice a day in four languages. Due\nto the lack of time available for manual translation, a fully automated\ntranslation system is employed, based on a catalogue of predefined phrases and\npredetermined rules of how these phrases can be combined to produce sentences.\nBecause this catalogue of phrases is limited to a small sublanguage, the system\nis able to automatically translate such sentences from German into the target\nlanguages French, Italian and English without subsequent proofreading or\ncorrection. Having been operational for two winter seasons, we assess here the\nquality of the produced texts based on two different surveys where participants\nrated texts from real avalanche bulletins from both origins, the catalogue of\nphrases versus manually written and translated texts. With a mean recognition\nrate of 55%, users can hardly distinguish between thetwo types of texts, and\ngive very similar ratings with respect to their language quality. Overall, the\noutput from the catalogue system can be considered virtually equivalent to a\ntext written by avalanche forecasters and then manually translated by\nprofessional translators. Furthermore, forecasters declared that all relevant\nsituations were captured by the system with sufficient accuracy. Forecaster's\nworking load did not change with the introduction of the catalogue: the extra\ntime to find matching sentences is compensated by the fact that they no longer\nneed to double-check manually translated texts. The reduction of daily\ntranslation costs is expected to offset the initial development costs within a\nfew years.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07308v2",
            "date": "2015-09-24 11:00:04+00:00",
            "title": "Bilingual Distributed Word Representations from Document-Aligned Comparable Data",
            "authors": [
                "Ivan Vuli\u0107",
                "Marie-Francine Moens"
            ],
            "abstract": "We propose a new model for learning bilingual word representations from\nnon-parallel document-aligned data. Following the recent advances in word\nrepresentation learning, our model learns dense real-valued word vectors, that\nis, bilingual word embeddings (BWEs). Unlike prior work on inducing BWEs which\nheavily relied on parallel sentence-aligned corpora and/or readily available\ntranslation resources such as dictionaries, the article reveals that BWEs may\nbe learned solely on the basis of document-aligned comparable data without any\nadditional lexical resources nor syntactic information. We present a comparison\nof our approach with previous state-of-the-art models for learning bilingual\nword representations from comparable data that rely on the framework of\nmultilingual probabilistic topic modeling (MuPTM), as well as with\ndistributional local context-counting models. We demonstrate the utility of the\ninduced BWEs in two semantic tasks: (1) bilingual lexicon extraction, (2)\nsuggesting word translations in context for polysemous words. Our simple yet\neffective BWE-based models significantly outperform the MuPTM-based and\ncontext-counting representation models from comparable data as well as prior\nBWE-based models, and acquire the best reported results on both tasks for all\nthree tested language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07513v1",
            "date": "2015-09-24 20:10:27+00:00",
            "title": "Description of the Odin Event Extraction Framework and Rule Language",
            "authors": [
                "Marco A. Valenzuela-Esc\u00e1rcega",
                "Gus Hahn-Powell",
                "Mihai Surdeanu"
            ],
            "abstract": "This document describes the Odin framework, which is a domain-independent\nplatform for developing rule-based event extraction models. Odin aims to be\npowerful (the rule language allows the modeling of complex syntactic\nstructures) and robust (to recover from syntactic parsing errors, syntactic\npatterns can be freely mixed with surface, token-based patterns), while\nremaining simple (some domain grammars can be up and running in minutes), and\nfast (Odin processes over 100 sentences/second in a real-world domain with over\n200 rules). Here we include a thorough definition of the Odin rule language,\ntogether with a description of the Odin API in the Scala language, which allows\none to apply these rules to arbitrary texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07612v1",
            "date": "2015-09-25 07:55:26+00:00",
            "title": "Sentiment Uncertainty and Spam in Twitter Streams and Its Implications for General Purpose Realtime Sentiment Analysis",
            "authors": [
                "Nils Haldenwang",
                "Oliver Vornberger"
            ],
            "abstract": "State of the art benchmarks for Twitter Sentiment Analysis do not consider\nthe fact that for more than half of the tweets from the public stream a\ndistinct sentiment cannot be chosen. This paper provides a new perspective on\nTwitter Sentiment Analysis by highlighting the necessity of explicitly\nincorporating uncertainty. Moreover, a dataset of high quality to evaluate\nsolutions for this new problem is introduced and made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07761v2",
            "date": "2015-09-25 15:41:13+00:00",
            "title": "Sentiment of Emojis",
            "authors": [
                "Petra Kralj Novak",
                "Jasmina Smailovi\u0107",
                "Borut Sluban",
                "Igor Mozeti\u010d"
            ],
            "abstract": "There is a new generation of emoticons, called emojis, that is increasingly\nbeing used in mobile communications and social media. In the past two years,\nover ten billion emojis were used on Twitter. Emojis are Unicode graphic\nsymbols, used as a shorthand to express concepts and ideas. In contrast to the\nsmall number of well-known emoticons that carry clear emotional contents, there\nare hundreds of emojis. But what are their emotional contents? We provide the\nfirst emoji sentiment lexicon, called the Emoji Sentiment Ranking, and draw a\nsentiment map of the 751 most frequently used emojis. The sentiment of the\nemojis is computed from the sentiment of the tweets in which they occur. We\nengaged 83 human annotators to label over 1.6 million tweets in 13 European\nlanguages by the sentiment polarity (negative, neutral, or positive). About 4%\nof the annotated tweets contain emojis. The sentiment analysis of the emojis\nallows us to draw several interesting conclusions. It turns out that most of\nthe emojis are positive, especially the most popular ones. The sentiment\ndistribution of the tweets with and without emojis is significantly different.\nThe inter-annotator agreement on the tweets with emojis is higher. Emojis tend\nto occur at the end of the tweets, and their sentiment polarity increases with\nthe distance. We observe no significant differences in the emoji rankings\nbetween the 13 languages and the Emoji Sentiment Ranking. Consequently, we\npropose our Emoji Sentiment Ranking as a European language-independent resource\nfor automated sentiment analysis. Finally, the paper provides a formalization\nof sentiment and a novel visualization in the form of a sentiment bar.",
            "journal ref": "PLoS ONE 10(12): e0144296, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08842v1",
            "date": "2015-09-29 16:46:52+00:00",
            "title": "Automatically Segmenting Oral History Transcripts",
            "authors": [
                "Ryan Shaw"
            ],
            "abstract": "Dividing oral histories into topically coherent segments can make them more\naccessible online. People regularly make judgments about where coherent\nsegments can be extracted from oral histories. But making these judgments can\nbe taxing, so automated assistance is potentially attractive to speed the task\nof extracting segments from open-ended interviews. When different people are\nasked to extract coherent segments from the same oral histories, they often do\nnot agree about precisely where such segments begin and end. This low agreement\nmakes the evaluation of algorithmic segmenters challenging, but there is reason\nto believe that for segmenting oral history transcripts, some approaches are\nmore promising than others. The BayesSeg algorithm performs slightly better\nthan TextTiling, while TextTiling does not perform significantly better than a\nuniform segmentation. BayesSeg might be used to suggest boundaries to someone\nsegmenting oral histories, but this segmentation task needs to be better\ndefined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08874v1",
            "date": "2015-09-29 18:17:22+00:00",
            "title": "Polish - English Speech Statistical Machine Translation Systems for the IWSLT 2014",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "This research explores effects of various training settings between Polish\nand English Statistical Machine Translation systems for spoken language.\nVarious elements of the TED parallel text corpora for the IWSLT 2014 evaluation\ncampaign were used as the basis for training of language models, and for\ndevelopment, tuning and testing of the translation system as well as Wikipedia\nbased comparable corpora prepared by us. The BLEU, NIST, METEOR and TER metrics\nwere used to evaluate the effects of data preparations on translation results.\nOur experiments included systems, which use lemma and morphological information\non Polish words. We also conducted a deep analysis of provided Polish data as\npreparatory work for the automatic data correction and cleaning phase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.09121v2",
            "date": "2015-09-30 10:59:24+00:00",
            "title": "The \"handedness\" of language: Directional symmetry breaking of sign usage in words",
            "authors": [
                "Md Izhar Ashraf",
                "Sitabhra Sinha"
            ],
            "abstract": "Language, which allows complex ideas to be communicated through symbolic\nsequences, is a characteristic feature of our species and manifested in a\nmultitude of forms. Using large written corpora for many different languages\nand scripts, we show that the occurrence probability distributions of signs at\nthe left and right ends of words have a distinct heterogeneous nature.\nCharacterizing this asymmetry using quantitative inequality measures, viz.\ninformation entropy and the Gini index, we show that the beginning of a word is\nless restrictive in sign usage than the end. This property is not simply\nattributable to the use of common affixes as it is seen even when only word\nroots are considered. We use the existence of this asymmetry to infer the\ndirection of writing in undeciphered inscriptions that agrees with the\narchaeological evidence. Unlike traditional investigations of phonotactic\nconstraints which focus on language-specific patterns, our study reveals a\nproperty valid across languages and writing systems. As both language and\nwriting are unique aspects of our species, this universal signature may reflect\nan innate feature of the human cognitive phenomenon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01083v1",
            "date": "2015-11-19 10:36:52+00:00",
            "title": "A Hybrid Approach to Word Sense Disambiguation Combining Supervised and Unsupervised Learning",
            "authors": [
                "Alok Ranjan Pal",
                "Anirban Kundu",
                "Abhay Singh",
                "Raj Shekhar",
                "Kunal Sinha"
            ],
            "abstract": "In this paper, we are going to find meaning of words based on distinct\nsituations. Word Sense Disambiguation is used to find meaning of words based on\nlive contexts using supervised and unsupervised approaches. Unsupervised\napproaches use online dictionary for learning, and supervised approaches use\nmanual learning sets. Hand tagged data are populated which might not be\neffective and sufficient for learning procedure. This limitation of information\nis main flaw of the supervised approach. Our proposed approach focuses to\novercome the limitation using learning set which is enriched in dynamic way\nmaintaining new data. Trivial filtering method is utilized to achieve\nappropriate training data. We introduce a mixed methodology having Modified\nLesk approach and Bag-of-Words having enriched bags using learning methods. Our\napproach establishes the superiority over individual Modified Lesk and\nBag-of-Words approaches based on experimentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.00548v4",
            "date": "2015-04-02 13:30:27+00:00",
            "title": "Learning to Understand Phrases by Embedding the Dictionary",
            "authors": [
                "Felix Hill",
                "Kyunghyun Cho",
                "Anna Korhonen",
                "Yoshua Bengio"
            ],
            "abstract": "Distributional models that learn rich semantic word representations are a\nsuccess story of recent NLP research. However, developing models that learn\nuseful representations of phrases and sentences has proved far harder. We\npropose using the definitions found in everyday dictionaries as a means of\nbridging this gap between lexical and phrasal semantics. Neural language\nembedding models can be effectively trained to map dictionary definitions\n(phrases) to (lexical) representations of the words defined by those\ndefinitions. We present two applications of these architectures: \"reverse\ndictionaries\" that return the name of a concept given a definition or\ndescription and general-knowledge crossword question answerers. On both tasks,\nneural language embedding models trained on definitions from a handful of\nfreely-available lexical resources perform as well or better than existing\ncommercial systems that rely on significant task-specific engineering. The\nresults highlight the effectiveness of both neural embedding architectures and\ndefinition-based training for developing models that understand phrases and\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01182v1",
            "date": "2015-04-06 01:18:24+00:00",
            "title": "Bengali to Assamese Statistical Machine Translation using Moses (Corpus Based)",
            "authors": [
                "Nayan Jyoti Kalita",
                "Baharul Islam"
            ],
            "abstract": "Machine dialect interpretation assumes a real part in encouraging man-machine\ncorrespondence and in addition men-men correspondence in Natural Language\nProcessing (NLP). Machine Translation (MT) alludes to utilizing machine to\nchange one dialect to an alternate. Statistical Machine Translation is a type\nof MT consisting of Language Model (LM), Translation Model (TM) and decoder. In\nthis paper, Bengali to Assamese Statistical Machine Translation Model has been\ncreated by utilizing Moses. Other translation tools like IRSTLM for Language\nModel and GIZA-PP-V1.0.7 for Translation model are utilized within this\nframework which is accessible in Linux situations. The purpose of the LM is to\nencourage fluent output and the purpose of TM is to encourage similarity\nbetween input and output, the decoder increases the probability of translated\ntext in target language. A parallel corpus of 17100 sentences in Bengali and\nAssamese has been utilized for preparing within this framework. Measurable MT\nprocedures have not so far been generally investigated for Indian dialects. It\nmight be intriguing to discover to what degree these models can help the\nimmense continuous MT deliberations in the nation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01427v1",
            "date": "2015-04-06 22:00:12+00:00",
            "title": "A Metric to Classify Style of Spoken Speech",
            "authors": [
                "Sunil Kopparapu",
                "Saurabh Bhatnagar",
                "K. Sahana",
                "Sathyanarayana",
                "Akhilesh Srivastava",
                "P. V. S. Rao"
            ],
            "abstract": "The ability to classify spoken speech based on the style of speaking is an\nimportant problem. With the advent of BPO's in recent times, specifically those\nthat cater to a population other than the local population, it has become\nnecessary for BPO's to identify people with certain style of speaking\n(American, British etc). Today BPO's employ accent analysts to identify people\nhaving the required style of speaking. This process while involving human bias,\nit is becoming increasingly infeasible because of the high attrition rate in\nthe BPO industry. In this paper, we propose a new metric, which robustly and\naccurately helps classify spoken speech based on the style of speaking. The\nrole of the proposed metric is substantiated by using it to classify real\nspeech data collected from over seventy different people working in a BPO. We\ncompare the performance of the metric against human experts who independently\ncarried out the classification process. Experimental results show that the\nperformance of the system using the novel metric performs better than two\ndifferent human expert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01496v1",
            "date": "2015-04-07 07:02:38+00:00",
            "title": "Voice based self help System: User Experience Vs Accuracy",
            "authors": [
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "In general, self help systems are being increasingly deployed by service\nbased industries because they are capable of delivering better customer service\nand increasingly the switch is to voice based self help systems because they\nprovide a natural interface for a human to interact with a machine. A speech\nbased self help system ideally needs a speech recognition engine to convert\nspoken speech to text and in addition a language processing engine to take care\nof any misrecognitions by the speech recognition engine. Any off-the-shelf\nspeech recognition engine is generally a combination of acoustic processing and\nspeech grammar. While this is the norm, we believe that ideally a speech\nrecognition application should have in addition to a speech recognition engine\na separate language processing engine to give the system better performance. In\nthis paper, we discuss ways in which the speech recognition engine and the\nlanguage processing engine can be combined to give a better user experience.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01683v4",
            "date": "2015-04-07 17:44:30+00:00",
            "title": "Jointly Embedding Relations and Mentions for Knowledge Population",
            "authors": [
                "Miao Fan",
                "Kai Cao",
                "Yifan He",
                "Ralph Grishman"
            ],
            "abstract": "This paper contributes a joint embedding model for predicting relations\nbetween a pair of entities in the scenario of relation inference. It differs\nfrom most stand-alone approaches which separately operate on either knowledge\nbases or free texts. The proposed model simultaneously learns low-dimensional\nvector representations for both triplets in knowledge repositories and the\nmentions of relations in free texts, so that we can leverage the evidence both\nresources to make more accurate predictions. We use NELL to evaluate the\nperformance of our approach, compared with cutting-edge methods. Results of\nextensive experiments show that our model achieves significant improvement on\nrelation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.02162v2",
            "date": "2015-04-09 00:49:36+00:00",
            "title": "Concentric network symmetry grasps authors' styles in word adjacency networks",
            "authors": [
                "Diego R. Amancio",
                "Filipi N. Silva",
                "Luciano da F. Costa"
            ],
            "abstract": "Several characteristics of written texts have been inferred from statistical\nanalysis derived from networked models. Even though many network measurements\nhave been adapted to study textual properties at several levels of complexity,\nsome textual aspects have been disregarded. In this paper, we study the\nsymmetry of word adjacency networks, a well-known representation of text as a\ngraph. A statistical analysis of the symmetry distribution performed in several\nnovels showed that most of the words do not display symmetric patterns of\nconnectivity. More specifically, the merged symmetry displayed a distribution\nsimilar to the ubiquitous power-law distribution. Our experiments also revealed\nthat the studied metrics do not correlate with other traditional network\nmeasurements, such as the degree or betweenness centrality. The effectiveness\nof the symmetry measurements was verified in the authorship attribution task.\nInterestingly, we found that specific authors prefer particular types of\nsymmetric motifs. As a consequence, the authorship of books could be accurately\nidentified in 82.5% of the cases, in a dataset comprising books written by 8\nauthors. Because the proposed measurements for text analysis are complementary\nto the traditional approach, they can be used to improve the characterization\nof text networks, which might be useful for related applications, such as those\nrelying on the identification of topical words and information retrieval.",
            "journal ref": "Europhys. Lett. 110 68001 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.02490v1",
            "date": "2015-04-09 20:21:32+00:00",
            "title": "Leveraging Twitter for Low-Resource Conversational Speech Language Modeling",
            "authors": [
                "Aaron Jaech",
                "Mari Ostendorf"
            ],
            "abstract": "In applications involving conversational speech, data sparsity is a limiting\nfactor in building a better language model. We propose a simple,\nlanguage-independent method to quickly harvest large amounts of data from\nTwitter to supplement a smaller training set that is more closely matched to\nthe domain. The techniques lead to a significant reduction in perplexity on\nfour low-resource languages even though the presence on Twitter of these\nlanguages is relatively small. We also find that the Twitter text is more\nuseful for learning word classes than the in-domain text and that use of these\nword classes leads to further reductions in perplexity. Additionally, we\nintroduce a method of using social and textual information to prioritize the\ndownload queue during the Twitter crawling. This maximizes the amount of useful\ndata that can be collected, impacting both perplexity and vocabulary coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.04751v1",
            "date": "2015-04-18 18:34:19+00:00",
            "title": "A Knowledge-poor Pronoun Resolution System for Turkish",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Meltem Turhan Y\u00f6ndem"
            ],
            "abstract": "A pronoun resolution system which requires limited syntactic knowledge to\nidentify the antecedents of personal and reflexive pronouns in Turkish is\npresented. As in its counterparts for languages like English, Spanish and\nFrench, the core of the system is the constraints and preferences determined\nempirically. In the evaluation phase, it performed considerably better than the\nbaseline algorithm used for comparison. The system is significant for its being\nthe first fully specified knowledge-poor computational framework for pronoun\nresolution in Turkish where Turkish possesses different structural properties\nfrom the languages for which knowledge-poor systems had been developed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.05319v2",
            "date": "2015-04-21 06:58:26+00:00",
            "title": "Big Data Small Data, In Domain Out-of Domain, Known Word Unknown Word: The Impact of Word Representation on Sequence Labelling Tasks",
            "authors": [
                "Lizhen Qu",
                "Gabriela Ferraro",
                "Liyuan Zhou",
                "Weiwei Hou",
                "Nathan Schneider",
                "Timothy Baldwin"
            ],
            "abstract": "Word embeddings -- distributed word representations that can be learned from\nunlabelled data -- have been shown to have high utility in many natural\nlanguage processing applications. In this paper, we perform an extrinsic\nevaluation of five popular word embedding methods in the context of four\nsequence labelling tasks: POS-tagging, syntactic chunking, NER and MWE\nidentification. A particular focus of the paper is analysing the effects of\ntask-based updating of word representations. We show that when using word\nembeddings as features, as few as several hundred training instances are\nsufficient to achieve competitive results, and that word embeddings lead to\nimprovements over OOV words and out of domain. Perhaps more surprisingly, our\nresults indicate there is little difference between the different word\nembedding methods, and that simple Brown clusters are often competitive with\nword embeddings across all tasks we consider.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06391v1",
            "date": "2015-04-24 04:45:55+00:00",
            "title": "On the Stability of Online Language Features: How Much Text do you Need to know a Person?",
            "authors": [
                "Eben M. Haber"
            ],
            "abstract": "In recent years, numerous studies have inferred personality and other traits\nfrom people's online writing. While these studies are encouraging, more\ninformation is needed in order to use these techniques with confidence. How do\nlinguistic features vary across different online media, and how much text is\nrequired to have a representative sample for a person? In this paper, we\nexamine several large sets of online, user-generated text, drawn from Twitter,\nemail, blogs, and online discussion forums. We examine and compare\npopulation-wide results for the linguistic measure LIWC, and the inferred\ntraits of Big5 Personality and Basic Human Values. We also empirically measure\nthe stability of these traits across different sized samples for each\nindividual. Our results highlight the importance of tuning models to each\nonline medium, and include guidelines for the minimum amount of text required\nfor a representative result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07678v1",
            "date": "2015-04-28 22:47:25+00:00",
            "title": "Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation",
            "authors": [
                "Hongzhao Huang",
                "Larry Heck",
                "Heng Ji"
            ],
            "abstract": "Entity Disambiguation aims to link mentions of ambiguous entities to a\nknowledge base (e.g., Wikipedia). Modeling topical coherence is crucial for\nthis task based on the assumption that information from the same semantic\ncontext tends to belong to the same topic. This paper presents a novel deep\nsemantic relatedness model (DSRM) based on deep neural networks (DNN) and\nsemantic knowledge graphs (KGs) to measure entity semantic relatedness for\ntopical coherence modeling. The DSRM is directly trained on large-scale KGs and\nit maps heterogeneous types of knowledge of an entity from KGs to numerical\nfeature vectors in a latent space such that the distance between two\nsemantically-related entities is minimized. Compared with the state-of-the-art\nrelatedness approach proposed by (Milne and Witten, 2008a), the DSRM obtains\n19.4% and 24.5% reductions in entity disambiguation errors on two publicly\navailable datasets respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.08102v1",
            "date": "2015-04-30 07:27:56+00:00",
            "title": "Detecting and ordering adjectival scalemates",
            "authors": [
                "Emiel van Miltenburg"
            ],
            "abstract": "This paper presents a pattern-based method that can be used to infer\nadjectival scales, such as <lukewarm, warm, hot>, from a corpus. Specifically,\nthe proposed method uses lexical patterns to automatically identify and order\npairs of scalemates, followed by a filtering phase in which unrelated pairs are\ndiscarded. For the filtering phase, several different similarity measures are\nimplemented and compared. The model presented in this paper is evaluated using\nthe current standard, along with a novel evaluation set, and shown to be at\nleast as good as the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.08183v1",
            "date": "2015-04-30 12:03:10+00:00",
            "title": "Texts in, meaning out: neural language models in semantic similarity task for Russian",
            "authors": [
                "Andrey Kutuzov",
                "Igor Andreev"
            ],
            "abstract": "Distributed vector representations for natural language vocabulary get a lot\nof attention in contemporary computational linguistics. This paper summarizes\nthe experience of applying neural network language models to the task of\ncalculating semantic similarity for Russian. The experiments were performed in\nthe course of Russian Semantic Similarity Evaluation track, where our models\ntook from the 2nd to the 5th position, depending on the task.\n  We introduce the tools and corpora used, comment on the nature of the shared\ntask and describe the achieved results. It was found out that Continuous\nSkip-gram and Continuous Bag-of-words models, previously successfully applied\nto English material, can be used for semantic modeling of Russian as well.\nMoreover, we show that texts in Russian National Corpus (RNC) provide an\nexcellent training material for such models, outperforming other, much larger\ncorpora. It is especially true for semantic relatedness tasks (although\nstacking models trained on larger corpora on top of RNC models improves\nperformance even more).\n  High-quality semantic vectors learned in such a way can be used in a variety\nof linguistic tasks and promise an exciting field for further study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00037v1",
            "date": "2015-05-29 21:51:04+00:00",
            "title": "Using Syntactic Features for Phishing Detection",
            "authors": [
                "Gilchan Park",
                "Julia M. Taylor"
            ],
            "abstract": "This paper reports on the comparison of the subject and object of verbs in\ntheir usage between phishing emails and legitimate emails. The purpose of this\nresearch is to explore whether the syntactic structures and subjects and\nobjects of verbs can be distinguishable features for phishing detection. To\nachieve the objective, we have conducted two series of experiments: the\nsyntactic similarity for sentences, and the subject and object of verb\ncomparison. The results of the experiments indicated that both features can be\nused for some verbs, but more work has to be done for others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00196v3",
            "date": "2015-05-31 05:14:06+00:00",
            "title": "Sequence-to-Sequence Neural Net Models for Grapheme-to-Phoneme Conversion",
            "authors": [
                "Kaisheng Yao",
                "Geoffrey Zweig"
            ],
            "abstract": "Sequence-to-sequence translation methods based on generation with a\nside-conditioned language model have recently shown promising results in\nseveral tasks. In machine translation, models conditioned on source side words\nhave been used to produce target-language text, and in image captioning, models\nconditioned images have been used to generate caption text. Past work with this\napproach has focused on large vocabulary tasks, and measured quality in terms\nof BLEU. In this paper, we explore the applicability of such models to the\nqualitatively different grapheme-to-phoneme task. Here, the input and output\nside vocabularies are small, plain n-gram models do well, and credit is only\ngiven when the output is exactly correct. We find that the simple\nside-conditioned generation approach is able to rival the state-of-the-art, and\nwe are able to significantly advance the stat-of-the-art with bi-directional\nlong short-term memory (LSTM) neural networks that use the same alignment\ninformation that is used in conventional approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00275v2",
            "date": "2015-05-31 19:21:26+00:00",
            "title": "Diversity in Spectral Learning for Natural Language Parsing",
            "authors": [
                "Shashi Narayan",
                "Shay B. Cohen"
            ],
            "abstract": "We describe an approach to create a diverse set of predictions with spectral\nlearning of latent-variable PCFGs (L-PCFGs). Our approach works by creating\nmultiple spectral models where noise is added to the underlying features in the\ntraining set before the estimation of each model. We describe three ways to\ndecode with multiple models. In addition, we describe a simple variant of the\nspectral algorithm for L-PCFGs that is fast and leads to compact models. Our\nexperiments for natural language parsing, for English and German, show that we\nget a significant improvement over baselines comparable to state of the art.\nFor English, we achieve the $F_1$ score of 90.18, and for German we achieve the\n$F_1$ score of 83.38.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00379v2",
            "date": "2015-06-01 08:22:49+00:00",
            "title": "Modeling Relation Paths for Representation Learning of Knowledge Bases",
            "authors": [
                "Yankai Lin",
                "Zhiyuan Liu",
                "Huanbo Luan",
                "Maosong Sun",
                "Siwei Rao",
                "Song Liu"
            ],
            "abstract": "Representation learning of knowledge bases (KBs) aims to embed both entities\nand relations into a low-dimensional space. Most existing methods only consider\ndirect relations in representation learning. We argue that multiple-step\nrelation paths also contain rich inference patterns between entities, and\npropose a path-based representation learning model. This model considers\nrelation paths as translations between entities for representation learning,\nand addresses two key challenges: (1) Since not all relation paths are\nreliable, we design a path-constraint resource allocation algorithm to measure\nthe reliability of relation paths. (2) We represent relation paths via semantic\ncomposition of relation embeddings. Experimental results on real-world datasets\nshow that, as compared with baselines, our model achieves significant and\nconsistent improvements on knowledge base completion and relation extraction\nfrom text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00406v3",
            "date": "2015-06-01 09:36:23+00:00",
            "title": "Monolingually Derived Phrase Scores for Phrase Based SMT Using Neural Networks Vector Representations",
            "authors": [
                "Amir Pouya Aghasadeghi",
                "Mohadeseh Bastan"
            ],
            "abstract": "In this paper, we propose two new features for estimating phrase-based\nmachine translation parameters from mainly monolingual data. Our method is\nbased on two recently introduced neural network vector representation models\nfor words and sentences. It is the first time that these models have been used\nin an end to end phrase-based machine translation system. Scores obtained from\nour method can recover more than 80% of BLEU loss caused by removing phrase\ntable probabilities. We also show that our features combined with the phrase\ntable probabilities improve the BLEU score by absolute 0.74 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00528v1",
            "date": "2015-06-01 15:21:00+00:00",
            "title": "Medical Synonym Extraction with Concept Space Models",
            "authors": [
                "Chang Wang",
                "Liangliang Cao",
                "Bowen Zhou"
            ],
            "abstract": "In this paper, we present a novel approach for medical synonym extraction. We\naim to integrate the term embedding with the medical domain knowledge for\nhealthcare applications. One advantage of our method is that it is very\nscalable. Experiments on a dataset with more than 1M term pairs show that the\nproposed approach outperforms the baseline approaches by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00698v1",
            "date": "2015-06-01 22:52:36+00:00",
            "title": "Statistical Machine Translation Features with Multitask Tensor Networks",
            "authors": [
                "Hendra Setiawan",
                "Zhongqiang Huang",
                "Jacob Devlin",
                "Thomas Lamar",
                "Rabih Zbib",
                "Richard Schwartz",
                "John Makhoul"
            ],
            "abstract": "We present a three-pronged approach to improving Statistical Machine\nTranslation (SMT), building on recent success in the application of neural\nnetworks to SMT. First, we propose new features based on neural networks to\nmodel various non-local translation phenomena. Second, we augment the\narchitecture of the neural network with tensor layers that capture important\nhigher-order interaction among the network units. Third, we apply multitask\nlearning to estimate the neural network parameters jointly. Each of our\nproposed methods results in significant improvements that are complementary.\nThe overall improvement is +2.7 and +1.8 BLEU points for Arabic-English and\nChinese-English translation over a state-of-the-art system that already\nincludes neural network features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00839v2",
            "date": "2015-06-02 11:12:19+00:00",
            "title": "The Influence of Context on Dialogue Act Recognition",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "This article presents an analysis of the influence of context information on\ndialog act recognition. We performed experiments on the widely explored\nSwitchboard corpus, as well as on data annotated according to the recent ISO\n24617-2 standard. The latter was obtained from the Tilburg DialogBank and\nthrough the mapping of the annotations of a subset of the Let's Go corpus. We\nused a classification approach based on SVMs, which had proved successful in\nprevious work and allowed us to limit the amount of context information\nprovided. This way, we were able to observe the influence patterns as the\namount of context information increased. Our base features consisted of\nn-grams, punctuation, and wh-words. Context information was obtained from one\nto five preceding segments and provided either as n-grams or dialog act\nclassifications, with the latter typically leading to better results and more\nstable influence patterns. In addition to the conclusions about the importance\nand influence of context information, our experiments on the Switchboard corpus\nalso led to results that advanced the state-of-the-art on the dialog act\nrecognition task on that corpus. Furthermore, the results obtained on data\nannotated according to the ISO 24617-2 standard define a baseline for future\nwork and contribute for the standardization of experiments in the area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01057v2",
            "date": "2015-06-02 20:53:53+00:00",
            "title": "A Hierarchical Neural Autoencoder for Paragraphs and Documents",
            "authors": [
                "Jiwei Li",
                "Minh-Thang Luong",
                "Dan Jurafsky"
            ],
            "abstract": "Natural language generation of coherent long texts like paragraphs or longer\ndocuments is a challenging problem for recurrent networks models. In this\npaper, we explore an important step toward this generation task: training an\nLSTM (Long-short term memory) auto-encoder to preserve and reconstruct\nmulti-sentence paragraphs. We introduce an LSTM model that hierarchically\nbuilds an embedding for a paragraph from embeddings for sentences and words,\nthen decodes this embedding to reconstruct the original paragraph. We evaluate\nthe reconstructed paragraph using standard metrics like ROUGE and Entity Grid,\nshowing that neural models are able to encode texts in a way that preserve\nsyntactic, semantic, and discourse coherence. While only a first step toward\ngenerating coherent text units from neural models, our work has the potential\nto significantly impact natural language generation and\nsummarization\\footnote{Code for the three models described in this paper can be\nfound at www.stanford.edu/~jiweil/ .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01066v2",
            "date": "2015-06-02 21:17:31+00:00",
            "title": "Visualizing and Understanding Neural Models in NLP",
            "authors": [
                "Jiwei Li",
                "Xinlei Chen",
                "Eduard Hovy",
                "Dan Jurafsky"
            ],
            "abstract": "While neural networks have been successfully applied to many NLP tasks the\nresulting vector-based models are very difficult to interpret. For example it's\nnot clear how they achieve {\\em compositionality}, building sentence meaning\nfrom the meanings of words and phrases. In this paper we describe four\nstrategies for visualizing compositionality in neural models for NLP, inspired\nby similar work in computer vision. We first plot unit values to visualize\ncompositionality of negation, intensification, and concessive clauses, allow us\nto see well-known markedness asymmetries in negation. We then introduce three\nsimple and straightforward methods for visualizing a unit's {\\em salience}, the\namount it contributes to the final composed meaning: (1) gradient\nback-propagation, (2) the variance of a token from the average word node, (3)\nLSTM-style gates that measure information flow. We test our methods on\nsentiment using simple recurrent nets and LSTMs. Our general-purpose methods\nmay have wide applications for understanding compositionality and other\nsemantic properties of deep networks , and also shed light on why LSTMs\noutperform simple recurrent nets,",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01070v3",
            "date": "2015-06-02 21:30:21+00:00",
            "title": "Do Multi-Sense Embeddings Improve Natural Language Understanding?",
            "authors": [
                "Jiwei Li",
                "Dan Jurafsky"
            ],
            "abstract": "Learning a distinct representation for each sense of an ambiguous word could\nlead to more powerful and fine-grained models of vector-space representations.\nYet while `multi-sense' methods have been proposed and tested on artificial\nword-similarity tasks, we don't know if they improve real natural language\nunderstanding tasks. In this paper we introduce a multi-sense embedding model\nbased on Chinese Restaurant Processes that achieves state of the art\nperformance on matching human word similarity judgments, and propose a\npipelined architecture for incorporating multi-sense embeddings into language\nunderstanding.\n  We then test the performance of our model on part-of-speech tagging, named\nentity recognition, sentiment analysis, semantic relation identification and\nsemantic relatedness, controlling for embedding dimensionality. We find that\nmulti-sense embeddings do improve performance on some tasks (part-of-speech\ntagging, semantic relation identification, semantic relatedness) but not on\nothers (named entity recognition, various forms of sentiment analysis). We\ndiscuss how these differences may be caused by the different role of word sense\ninformation in each of the tasks. The results highlight the importance of\ntesting embedding models in real applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01171v1",
            "date": "2015-06-03 09:18:05+00:00",
            "title": "A Hybrid Model for Enhancing Lexical Statistical Machine Translation (SMT)",
            "authors": [
                "Ahmed G. M. ElSayed",
                "Ahmed S. Salama",
                "Alaa El-Din M. El-Ghazali"
            ],
            "abstract": "The interest in statistical machine translation systems increases currently\ndue to political and social events in the world. A proposed Statistical Machine\nTranslation (SMT) based model that can be used to translate a sentence from the\nsource Language (English) to the target language (Arabic) automatically through\nefficiently incorporating different statistical and Natural Language Processing\n(NLP) models such as language model, alignment model, phrase based model,\nreordering model, and translation model. These models are combined to enhance\nthe performance of statistical machine translation (SMT). Many implementation\ntools have been used in this work such as Moses, Gizaa++, IRSTLM, KenLM, and\nBLEU. Based on the implementation, evaluation of this model, and comparing the\ngenerated translation with other implemented machine translation systems like\nGoogle Translate, it was proved that this proposed model has enhanced the\nresults of the statistical machine translation, and forms a reliable and\nefficient model in this field of research.",
            "journal ref": "International Journal of Computer Science Issues Volume 12, Issue\n  2, March 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01906v1",
            "date": "2015-06-05 13:29:33+00:00",
            "title": "Idioms-Proverbs Lexicon for Modern Standard Arabic and Colloquial Sentiment Analysis",
            "authors": [
                "Hossam S. Ibrahim",
                "Sherif M. Abdou",
                "Mervat Gheith"
            ],
            "abstract": "Although, the fair amount of works in sentiment analysis (SA) and opinion\nmining (OM) systems in the last decade and with respect to the performance of\nthese systems, but it still not desired performance, especially for\nmorphologically-Rich Language (MRL) such as Arabic, due to the complexities and\nchallenges exist in the nature of the languages itself. One of these challenges\nis the detection of idioms or proverbs phrases within the writer text or\ncomment. An idiom or proverb is a form of speech or an expression that is\npeculiar to itself. Grammatically, it cannot be understood from the individual\nmeanings of its elements and can yield different sentiment when treats as\nseparate words. Consequently, In order to facilitate the task of detection and\nclassification of lexical phrases for automated SA systems, this paper presents\nAIPSeLEX a novel idioms/ proverbs sentiment lexicon for modern standard Arabic\n(MSA) and colloquial. AIPSeLEX is manually collected and annotated at sentence\nlevel with semantic orientation (positive or negative). The efforts of manually\nbuilding and annotating the lexicon are reported. Moreover, we build a\nclassifier that extracts idioms and proverbs, phrases from text using n-gram\nand similarity measure methods. Finally, several experiments were carried out\non various data, including Arabic tweets and Arabic microblogs (hotel\nreservation, product reviews, and TV program comments) from publicly available\nArabic online reviews websites (social media, blogs, forums, e-commerce web\nsites) to evaluate the coverage and accuracy of AIPSeLEX.",
            "journal ref": "International Journal of Computer Applications 118(11):26-31, May\n  2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01914v1",
            "date": "2015-06-05 13:48:56+00:00",
            "title": "Content Translation: Computer-assisted translation tool for Wikipedia articles",
            "authors": [
                "Niklas Laxstr\u00f6m",
                "Pau Giner",
                "Santhosh Thottingal"
            ],
            "abstract": "The quality and quantity of articles in each Wikipedia language varies\ngreatly. Translating from another Wikipedia is a natural way to add more\ncontent, but the translation process is not properly supported in the software\nused by Wikipedia. Past computer-assisted translation tools built for Wikipedia\nare not commonly used. We created a tool that adapts to the specific needs of\nan open community and to the kind of content in Wikipedia. Qualitative and\nquantitative data indicates that the new tool helps users translate articles\neasier and faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02004v1",
            "date": "2015-06-05 18:20:43+00:00",
            "title": "Sparse Overcomplete Word Vector Representations",
            "authors": [
                "Manaal Faruqui",
                "Yulia Tsvetkov",
                "Dani Yogatama",
                "Chris Dyer",
                "Noah Smith"
            ],
            "abstract": "Current distributed representations of words show little resemblance to\ntheories of lexical semantics. The former are dense and uninterpretable, the\nlatter largely based on familiar, discrete classes (e.g., supersenses) and\nrelations (e.g., synonymy and hypernymy). We propose methods that transform\nword vectors into sparse (and optionally binary) vectors. The resulting\nrepresentations are more similar to the interpretable features typically used\nin NLP, though they are discovered automatically from raw corpora. Because the\nvectors are highly sparse, they are computationally easy to work with. Most\nimportantly, we find that they outperform the original vectors on benchmark\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02275v2",
            "date": "2015-06-07 15:29:26+00:00",
            "title": "Confounds and Consequences in Geotagged Twitter Data",
            "authors": [
                "Umashanthi Pavalanathan",
                "Jacob Eisenstein"
            ],
            "abstract": "Twitter is often used in quantitative studies that identify\ngeographically-preferred topics, writing styles, and entities. These studies\nrely on either GPS coordinates attached to individual messages, or on the\nuser-supplied location field in each profile. In this paper, we compare these\ndata acquisition techniques and quantify the biases that they introduce; we\nalso measure their effects on linguistic analysis and text-based geolocation.\nGPS-tagging and self-reported locations yield measurably different corpora, and\nthese linguistic differences are partially attributable to differences in\ndataset composition by age and gender. Using a latent variable model to induce\nage and gender, we show how these demographic variables interact with geography\nto affect language use. We also show that the accuracy of text-based\ngeolocation varies with population demographics, giving the best results for\nmen above the age of 40.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02306v2",
            "date": "2015-06-07 19:54:00+00:00",
            "title": "SQUINKY! A Corpus of Sentence-level Formality, Informativeness, and Implicature",
            "authors": [
                "Shibamouli Lahiri"
            ],
            "abstract": "We introduce a corpus of 7,032 sentences rated by human annotators for\nformality, informativeness, and implicature on a 1-7 scale. The corpus was\nannotated using Amazon Mechanical Turk. Reliability in the obtained judgments\nwas examined by comparing mean ratings across two MTurk experiments, and\ncorrelation with pilot annotations (on sentence formality) conducted in a more\ncontrolled setting. Despite the subjectivity and inherent difficulty of the\nannotation task, correlations between mean ratings were quite encouraging,\nespecially on formality and informativeness. We further explored correlation\nbetween the three linguistic variables, genre-wise variation of ratings and\ncorrelations within genres, compatibility with automatic stylistic scoring, and\nsentential make-up of a document in terms of style. To date, our corpus is the\nlargest sentence-level annotated corpus released for formality,\ninformativeness, and implicature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02338v3",
            "date": "2015-06-08 02:21:46+00:00",
            "title": "Modeling Order in Neural Word Embeddings at Scale",
            "authors": [
                "Andrew Trask",
                "David Gilmore",
                "Matthew Russell"
            ],
            "abstract": "Natural Language Processing (NLP) systems commonly leverage bag-of-words\nco-occurrence techniques to capture semantic and syntactic word relationships.\nThe resulting word-level distributed representations often ignore morphological\ninformation, though character-level embeddings have proven valuable to NLP\ntasks. We propose a new neural language model incorporating both word order and\ncharacter order in its embedding. The model produces several vector spaces with\nmeaningful substructure, as evidenced by its performance of 85.8% on a recent\nword-analogy task, exceeding best published syntactic word-analogy scores by a\n58% error margin. Furthermore, the model includes several parallel training\nmethods, most notably allowing a skip-gram network with 160 billion parameters\nto be trained overnight on 3 multi-core CPUs, 14x larger than the previous\nlargest neural network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02739v3",
            "date": "2015-06-09 00:58:51+00:00",
            "title": "Connotation Frames: A Data-Driven Investigation",
            "authors": [
                "Hannah Rashkin",
                "Sameer Singh",
                "Yejin Choi"
            ],
            "abstract": "Through a particular choice of a predicate (e.g., \"x violated y\"), a writer\ncan subtly connote a range of implied sentiments and presupposed facts about\nthe entities x and y: (1) writer's perspective: projecting x as an\n\"antagonist\"and y as a \"victim\", (2) entities' perspective: y probably dislikes\nx, (3) effect: something bad happened to y, (4) value: y is something valuable,\nand (5) mental state: y is distressed by the event. We introduce connotation\nframes as a representation formalism to organize these rich dimensions of\nconnotation using typed relations. First, we investigate the feasibility of\nobtaining connotative labels through crowdsourcing experiments. We then present\nmodels for predicting the connotation frames of verb predicates based on their\ndistributional word representations and the interplay between different types\nof connotative relations. Empirical results confirm that connotation frames can\nbe induced from various data sources that reflect how people use language and\ngive rise to the connotative meanings. We conclude with analytical results that\nshow the potential use of connotation frames for analyzing subtle biases in\nonline news media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03139v1",
            "date": "2015-06-10 00:40:12+00:00",
            "title": "Robust Subgraph Generation Improves Abstract Meaning Representation Parsing",
            "authors": [
                "Keenon Werling",
                "Gabor Angeli",
                "Christopher Manning"
            ],
            "abstract": "The Abstract Meaning Representation (AMR) is a representation for open-domain\nrich semantics, with potential use in fields like event extraction and machine\ntranslation. Node generation, typically done using a simple dictionary lookup,\nis currently an important limiting factor in AMR parsing. We propose a small\nset of actions that derive AMR subgraphs by transformations on spans of text,\nwhich allows for more robust learning of this stage. Our set of construction\nactions generalize better than the previous approach, and can be learned with a\nsimple classifier. We improve on the previous state-of-the-art result for AMR\nparsing, boosting end-to-end performance by 3 F$_1$ on both the LDC2013E117 and\nLDC2014T12 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03229v3",
            "date": "2015-06-10 09:25:59+00:00",
            "title": "A cognitive neural architecture able to learn and communicate through natural language",
            "authors": [
                "Bruno Golosio",
                "Angelo Cangelosi",
                "Olesya Gamotina",
                "Giovanni Luca Masala"
            ],
            "abstract": "Communicative interactions involve a kind of procedural knowledge that is\nused by the human brain for processing verbal and nonverbal inputs and for\nlanguage production. Although considerable work has been done on modeling human\nlanguage abilities, it has been difficult to bring them together to a\ncomprehensive tabula rasa system compatible with current knowledge of how\nverbal information is processed in the brain. This work presents a cognitive\nsystem, entirely based on a large-scale neural architecture, which was\ndeveloped to shed light on the procedural knowledge involved in language\nelaboration. The main component of this system is the central executive, which\nis a supervising system that coordinates the other components of the working\nmemory. In our model, the central executive is a neural network that takes as\ninput the neural activation states of the short-term memory and yields as\noutput mental actions, which control the flow of information among the working\nmemory components through neural gating mechanisms. The proposed system is\ncapable of learning to communicate through natural language starting from\ntabula rasa, without any a priori knowledge of the structure of phrases,\nmeaning of words, role of the different classes of words, only by interacting\nwith a human through a text-based interface, using an open-ended incremental\nlearning process. It is able to learn nouns, verbs, adjectives, pronouns and\nother word classes, and to use them in expressive language. The model was\nvalidated on a corpus of 1587 input sentences, based on literature on early\nlanguage assessment, at the level of about 4-years old child, and produced 521\noutput sentences, expressing a broad range of language processing\nfunctionalities.",
            "journal ref": "PLoS ONE 10 (2015) e0140866",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03257v1",
            "date": "2015-06-10 11:28:31+00:00",
            "title": "Combining Temporal Information and Topic Modeling for Cross-Document Event Ordering",
            "authors": [
                "Borja Navarro-Colorado",
                "Estela Saquete"
            ],
            "abstract": "Building unified timelines from a collection of written news articles\nrequires cross-document event coreference resolution and temporal relation\nextraction. In this paper we present an approach event coreference resolution\naccording to: a) similar temporal information, and b) similar semantic\narguments. Temporal information is detected using an automatic temporal\ninformation system (TIPSem), while semantic information is represented by means\nof LDA Topic Modeling. The evaluation of our approach shows that it obtains the\nhighest Micro-average F-score results in the SemEval2015 Task 4: TimeLine:\nCross-Document Event Ordering (25.36\\% for TrackB, 23.15\\% for SubtrackB), with\nan improvement of up to 6\\% in comparison to the other systems. However, our\nexperiment also showed some draw-backs in the Topic Modeling approach that\ndegrades performance of the system.",
            "journal ref": "Proceedings of the 9th International Workshop on Semantic\n  Evaluation (SemEval 2015) http://www.aclweb.org/anthology/S/S15/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03487v2",
            "date": "2015-06-10 21:29:28+00:00",
            "title": "From Paraphrase Database to Compositional Paraphrase Model and Back",
            "authors": [
                "John Wieting",
                "Mohit Bansal",
                "Kevin Gimpel",
                "Karen Livescu",
                "Dan Roth"
            ],
            "abstract": "The Paraphrase Database (PPDB; Ganitkevitch et al., 2013) is an extensive\nsemantic resource, consisting of a list of phrase pairs with (heuristic)\nconfidence estimates. However, it is still unclear how it can best be used, due\nto the heuristic nature of the confidences and its necessarily incomplete\ncoverage. We propose models to leverage the phrase pairs from the PPDB to build\nparametric paraphrase models that score paraphrase pairs more accurately than\nthe PPDB's internal scores while simultaneously improving its coverage. They\nallow for learning phrase embeddings as well as improved word embeddings.\nMoreover, we introduce two new, manually annotated datasets to evaluate\nshort-phrase paraphrasing models. Using our paraphrase model trained using\nPPDB, we achieve state-of-the-art results on standard word and bigram\nsimilarity tasks and beat strong baselines on our new short phrase paraphrase\ntasks.",
            "journal ref": "TACL Vol 3 (2015) pg 345-358",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03694v2",
            "date": "2015-06-11 14:45:49+00:00",
            "title": "Learning language through pictures",
            "authors": [
                "Grzegorz Chrupa\u0142a",
                "\u00c1kos K\u00e1d\u00e1r",
                "Afra Alishahi"
            ],
            "abstract": "We propose Imaginet, a model of learning visually grounded representations of\nlanguage from coupled textual and visual input. The model consists of two Gated\nRecurrent Unit networks with shared word embeddings, and uses a multi-task\nobjective by receiving a textual description of a scene and trying to\nconcurrently predict its visual representation and the next word in the\nsentence. Mimicking an important aspect of human language learning, it acquires\nmeaning representations for individual words from descriptions of visual\nscenes. Moreover, it learns to effectively use sequential structure in semantic\ninterpretation of multi-word phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04228v1",
            "date": "2015-06-13 05:47:50+00:00",
            "title": "A Publicly Available Cross-Platform Lemmatizer for Bulgarian",
            "authors": [
                "Grigor Iliev",
                "Nadezhda Borisova",
                "Elena Karashtranova",
                "Dafina Kostadinova"
            ],
            "abstract": "Our dictionary-based lemmatizer for the Bulgarian language presented here is\ndistributed as free software, publicly available to download and use under the\nGPL v3 license. The presented software is written entirely in Java and is\ndistributed as a GATE plugin. To our best knowledge, at the time of writing\nthis article, there are not any other free lemmatization tools specifically\ntargeting the Bulgarian language. The presented lemmatizer is a work in\nprogress and currently yields an accuracy of about 95% in comparison to the\nmanually annotated corpus BulTreeBank-Morph, which contains 273933 tokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04229v1",
            "date": "2015-06-13 05:53:57+00:00",
            "title": "Evaluation of the Accuracy of the BGLemmatizer",
            "authors": [
                "Elena Karashtranova",
                "Grigor Iliev",
                "Nadezhda Borisova",
                "Yana Chankova",
                "Irena Atanasova"
            ],
            "abstract": "This paper reveals the results of an analysis of the accuracy of developed\nsoftware for automatic lemmatization for the Bulgarian language. This\nlemmatization software is written entirely in Java and is distributed as a GATE\nplugin. Certain statistical methods are used to define the accuracy of this\nsoftware. The results of the analysis show 95% lemmatization accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04334v2",
            "date": "2015-06-13 23:39:09+00:00",
            "title": "A Bayesian Model for Generative Transition-based Dependency Parsing",
            "authors": [
                "Jan Buys",
                "Phil Blunsom"
            ],
            "abstract": "We propose a simple, scalable, fully generative model for transition-based\ndependency parsing with high accuracy. The model, parameterized by Hierarchical\nPitman-Yor Processes, overcomes the limitations of previous generative models\nby allowing fast and accurate inference. We propose an efficient decoding\nalgorithm based on particle filtering that can adapt the beam size to the\nuncertainty in the model while jointly predicting POS tags and parse trees. The\nUAS of the parser is on par with that of a greedy discriminative baseline. As a\nlanguage model, it obtains better perplexity than a n-gram model by performing\nsemi-supervised learning over a large unlabelled corpus. We show that the model\nis able to generate locally and syntactically coherent sentences, opening the\ndoor to further applications in language generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04897v1",
            "date": "2015-06-16 09:54:27+00:00",
            "title": "Parsing Natural Language Sentences by Semi-supervised Methods",
            "authors": [
                "Rudolf Rosa"
            ],
            "abstract": "We present our work on semi-supervised parsing of natural language sentences,\nfocusing on multi-source crosslingual transfer of delexicalized dependency\nparsers. We first evaluate the influence of treebank annotation styles on\nparsing performance, focusing on adposition attachment style. Then, we present\nKLcpos3, an empirical language similarity measure, designed and tuned for\nsource parser weighting in multi-source delexicalized parser transfer. And\nfinally, we introduce a novel resource combination method, based on\ninterpolation of trained parser models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04940v1",
            "date": "2015-06-16 12:31:43+00:00",
            "title": "Recognize Foreign Low-Frequency Words with Similar Pairs",
            "authors": [
                "Xi Ma",
                "Xiaoxi Wang",
                "Dong Wang",
                "Zhiyong Zhang"
            ],
            "abstract": "Low-frequency words place a major challenge for automatic speech recognition\n(ASR). The probabilities of these words, which are often important name\nentities, are generally under-estimated by the language model (LM) due to their\nlimited occurrences in the training data. Recently, we proposed a word-pair\napproach to deal with the problem, which borrows information of frequent words\nto enhance the probabilities of low-frequency words. This paper presents an\nextension to the word-pair method by involving multiple `predicting words' to\nproduce better estimation for low-frequency words. We also employ this approach\nto deal with out-of-language words in the task of multi-lingual speech\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05230v1",
            "date": "2015-06-17 07:40:14+00:00",
            "title": "Non-distributional Word Vector Representations",
            "authors": [
                "Manaal Faruqui",
                "Chris Dyer"
            ],
            "abstract": "Data-driven representation learning for words is a technique of central\nimportance in NLP. While indisputably useful as a source of features in\ndownstream tasks, such vectors tend to consist of uninterpretable components\nwhose relationship to the categories of traditional lexical semantic theories\nis tenuous at best. We present a method for constructing interpretable word\nvectors from hand-crafted linguistic resources like WordNet, FrameNet etc.\nThese vectors are binary (i.e, contain only 0 and 1) and are 99.9% sparse. We\nanalyze their performance on state-of-the-art evaluation methods for\ndistributional models of word vectors and find they are competitive to standard\ndistributional approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05676v1",
            "date": "2015-06-17 19:55:13+00:00",
            "title": "Pragmatic Side Effects",
            "authors": [
                "Jiri Marsik",
                "Maxime Amblard"
            ],
            "abstract": "In the quest to give a formal compositional semantics to natural languages,\nsemanticists have started turning their attention to phenomena that have been\nalso considered as parts of pragmatics (e.g., discourse anaphora and\npresupposition projection). To account for these phenomena, the very kinds of\nmeanings assigned to words and phrases are often revisited. To be more\nspecific, in the prevalent paradigm of modeling natural language denotations\nusing the simply-typed lambda calculus (higher-order logic) this means\nrevisiting the types of denotations assigned to individual parts of speech.\nHowever, the lambda calculus also serves as a fundamental theory of\ncomputation, and in the study of computation, similar type shifts have been\nemployed to give a meaning to side effects. Side effects in programming\nlanguages correspond to actions that go beyond the lexical scope of an\nexpression (a thrown exception might propagate throughout a program, a variable\nmodified at one point might later be read at an another) or even beyond the\nscope of the program itself (a program might interact with the outside world by\ne.g., printing documents, making sounds, operating robotic limbs...).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05702v2",
            "date": "2015-06-18 14:46:15+00:00",
            "title": "Comparing the writing style of real and artificial papers",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "Recent years have witnessed the increase of competition in science. While\npromoting the quality of research in many cases, an intense competition among\nscientists can also trigger unethical scientific behaviors. To increase the\ntotal number of published papers, some authors even resort to software tools\nthat are able to produce grammatical, but meaningless scientific manuscripts.\nBecause automatically generated papers can be misunderstood as real papers, it\nbecomes of paramount importance to develop means to identify these scientific\nfrauds. In this paper, I devise a methodology to distinguish real manuscripts\nfrom those generated with SCIGen, an automatic paper generator. Upon modeling\ntexts as complex networks (CN), it was possible to discriminate real from fake\npapers with at least 89\\% of accuracy. A systematic analysis of features\nrelevance revealed that the accessibility and betweenness were useful in\nparticular cases, even though the relevance depended upon the dataset. The\nsuccessful application of the methods described here show, as a proof of\nprinciple, that network features can be used to identify scientific gibberish\npapers. In addition, the CN-based approach can be combined in a straightforward\nfashion with traditional statistical language processing methods to improve the\nperformance in identifying artificially generated papers.",
            "journal ref": "Scientometrics 105 (3), (2015) pp. 1763-1779",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05703v1",
            "date": "2015-06-18 14:46:44+00:00",
            "title": "\"The Sum of Its Parts\": Joint Learning of Word and Phrase Representations with Autoencoders",
            "authors": [
                "R\u00e9mi Lebret",
                "Ronan Collobert"
            ],
            "abstract": "Recently, there has been a lot of effort to represent words in continuous\nvector spaces. Those representations have been shown to capture both semantic\nand syntactic information about words. However, distributed representations of\nphrases remain a challenge. We introduce a novel model that jointly learns word\nvector representations and their summation. Word representations are learnt\nusing the word co-occurrence statistical information. To embed sequences of\nwords (i.e. phrases) with different sizes into a common semantic space, we\npropose to average word vector representations. In contrast with previous\nmethods which reported a posteriori some compositionality aspects by simple\nsummation, we simultaneously train words to sum, while keeping the maximum\ninformation from the original vectors. We evaluate the quality of the word\nrepresentations on several classical word evaluation tasks, and we introduce a\nnovel task to evaluate the quality of the phrase representations. While our\ndistributed representations compete with other methods of learning word\nrepresentations on word evaluations, we show that they give better performance\non the phrase evaluation. Such representations of phrases could be interesting\nfor many tasks in natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05869v3",
            "date": "2015-06-19 02:52:23+00:00",
            "title": "A Neural Conversational Model",
            "authors": [
                "Oriol Vinyals",
                "Quoc Le"
            ],
            "abstract": "Conversational modeling is an important task in natural language\nunderstanding and machine intelligence. Although previous approaches exist,\nthey are often restricted to specific domains (e.g., booking an airline ticket)\nand require hand-crafted rules. In this paper, we present a simple approach for\nthis task which uses the recently proposed sequence to sequence framework. Our\nmodel converses by predicting the next sentence given the previous sentence or\nsentences in a conversation. The strength of our model is that it can be\ntrained end-to-end and thus requires much fewer hand-crafted rules. We find\nthat this straightforward model can generate simple conversations given a large\nconversational training dataset. Our preliminary results suggest that, despite\noptimizing the wrong objective function, the model is able to converse well. It\nis able extract knowledge from both a domain specific dataset, and from a\nlarge, noisy, and general domain dataset of movie subtitles. On a\ndomain-specific IT helpdesk dataset, the model can find a solution to a\ntechnical problem via conversations. On a noisy open-domain movie transcript\ndataset, the model can perform simple forms of common sense reasoning. As\nexpected, we also find that the lack of consistency is a common failure mode of\nour model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06158v1",
            "date": "2015-06-19 21:05:01+00:00",
            "title": "Structured Training for Neural Network Transition-Based Parsing",
            "authors": [
                "David Weiss",
                "Chris Alberti",
                "Michael Collins",
                "Slav Petrov"
            ],
            "abstract": "We present structured perceptron training for neural network transition-based\ndependency parsing. We learn the neural network representation using a gold\ncorpus augmented by a large number of automatically parsed sentences. Given\nthis fixed network representation, we learn a final layer using the structured\nperceptron with beam-search decoding. On the Penn Treebank, our parser reaches\n94.26% unlabeled and 92.41% labeled attachment accuracy, which to our knowledge\nis the best accuracy on Stanford Dependencies to date. We also provide in-depth\nablative analysis to determine which aspects of our model provide the largest\ngains in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06863v2",
            "date": "2015-06-23 05:24:53+00:00",
            "title": "deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets",
            "authors": [
                "Michel Galley",
                "Chris Brockett",
                "Alessandro Sordoni",
                "Yangfeng Ji",
                "Michael Auli",
                "Chris Quirk",
                "Margaret Mitchell",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic\nevaluation of generated text in tasks that admit a diverse range of possible\noutputs. Reference strings are scored for quality by human raters on a scale of\n[-1, +1] to weight multi-reference BLEU. In tasks involving generation of\nconversational responses, deltaBLEU correlates reasonably with human judgments\nand outperforms sentence-level and IBM BLEU in terms of both Spearman's rho and\nKendall's tau.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06904v1",
            "date": "2015-06-23 08:26:50+00:00",
            "title": "New Approach to translation of Isolated Units in English-Korean Machine Translation",
            "authors": [
                "Kim Song Jon",
                "An Hae Gum"
            ],
            "abstract": "It is the most effective way for quick translation of tremendous amount of\nexplosively increasing science and technique information material to develop a\npracticable machine translation system and introduce it into translation\npractice. This essay treats problems arising from translation of isolated units\non the basis of the practical materials and experiments obtained in the\ndevelopment and introduction of English-Korean machine translation system. In\nother words, this essay considers establishment of information for isolated\nunits and their Korean equivalents and word order.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08052v3",
            "date": "2015-06-26 13:17:31+00:00",
            "title": "Automagically encoding Adverse Drug Reactions in MedDRA",
            "authors": [
                "Carlo Combi",
                "Riccardo Lora",
                "Ugo Moretti",
                "Marco Pagliarini",
                "Margherita Zorzi"
            ],
            "abstract": "Pharmacovigilance is the field of science devoted to the collection, analysis\nand prevention of Adverse Drug Reactions (ADRs). Efficient strategies for the\nextraction of information about ADRs from free text resources are essential to\nsupport the work of experts, employed in the crucial task of detecting and\nclassifying unexpected pathologies possibly related to drug assumptions.\nNarrative ADR descriptions may be collected in several way, e.g. by monitoring\nsocial networks or through the so called spontaneous reporting, the main method\npharmacovigilance adopts in order to identify ADRs. The encoding of free-text\nADR descriptions according to MedDRA standard terminology is central for report\nanalysis. It is a complex work, which has to be manually implemented by the\npharmacovigilance experts. The manual encoding is expensive (in terms of time).\nMoreover, a problem about the accuracy of the encoding may occur, since the\nnumber of reports is growing up day by day. In this paper, we propose\nMagiCoder, an efficient Natural Language Processing algorithm able to\nautomatically derive MedDRA terminologies from free-text ADR descriptions.\nMagiCoder is part of VigiWork, a web application for online ADR reporting and\nanalysis. From a practical view-point, MagiCoder radically reduces the revision\ntime of ADR reports: the pharmacologist has simply to revise and validate the\nautomatic solution versus the hard task of choosing solutions in the 70k terms\nof MedDRA. This improvement of the expert work efficiency has a meaningful\nimpact on the quality of data analysis. Moreover, our procedure is general\npurpose. We developed MagiCoder for the Italian pharmacovigilance language, but\npreliminarily analyses show that it is robust to language and dictionary\nchanges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.09107v2",
            "date": "2015-06-30 14:32:30+00:00",
            "title": "A complex network approach to stylometry",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "Statistical methods have been widely employed to study the fundamental\nproperties of language. In recent years, methods from complex and dynamical\nsystems proved useful to create several language models. Despite the large\namount of studies devoted to represent texts with physical models, only a\nlimited number of studies have shown how the properties of the underlying\nphysical systems can be employed to improve the performance of natural language\nprocessing tasks. In this paper, I address this problem by devising complex\nnetworks methods that are able to improve the performance of current\nstatistical methods. Using a fuzzy classification strategy, I show that the\ntopological properties extracted from texts complement the traditional textual\ndescription. In several cases, the performance obtained with hybrid approaches\noutperformed the results obtained when only traditional or networked methods\nwere used. Because the proposed model is generic, the framework devised here\ncould be straightforwardly used to study similar textual applications where the\ntopology plays a pivotal role in the description of the interacting agents.",
            "journal ref": "PLoS ONE 10(8): e0136076, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00106v5",
            "date": "2015-08-01 10:24:27+00:00",
            "title": "Separated by an Un-common Language: Towards Judgment Language Informed Vector Space Modeling",
            "authors": [
                "Ira Leviant",
                "Roi Reichart"
            ],
            "abstract": "A common evaluation practice in the vector space models (VSMs) literature is\nto measure the models' ability to predict human judgments about lexical\nsemantic relations between word pairs. Most existing evaluation sets, however,\nconsist of scores collected for English word pairs only, ignoring the potential\nimpact of the judgment language in which word pairs are presented on the human\nscores. In this paper we translate two prominent evaluation sets, wordsim353\n(association) and SimLex999 (similarity), from English to Italian, German and\nRussian and collect scores for each dataset from crowdworkers fluent in its\nlanguage. Our analysis reveals that human judgments are strongly impacted by\nthe judgment language. Moreover, we show that the predictions of monolingual\nVSMs do not necessarily best correlate with human judgments made with the\nlanguage used for model training, suggesting that models and humans are\naffected differently by the language they use when making semantic judgments.\nFinally, we show that in a large number of setups, multilingual VSM combination\nresults in improved correlations with human judgments, suggesting that\nmultilingualism may partially compensate for the judgment language effect on\nhuman judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00305v1",
            "date": "2015-08-03 02:53:01+00:00",
            "title": "Compositional Semantic Parsing on Semi-Structured Tables",
            "authors": [
                "Panupong Pasupat",
                "Percy Liang"
            ],
            "abstract": "Two important aspects of semantic parsing for question answering are the\nbreadth of the knowledge source and the depth of logical compositionality.\nWhile existing work trades off one aspect for another, this paper\nsimultaneously makes progress on both fronts through a new task: answering\ncomplex questions on semi-structured tables using question-answer pairs as\nsupervision. The central challenge arises from two compounding factors: the\nbroader domain results in an open-ended set of relations, and the deeper\ncompositionality results in a combinatorial explosion in the space of logical\nforms. We propose a logical-form driven parsing algorithm guided by strong\ntyping constraints and show that it obtains significant improvements over\nnatural baselines. For evaluation, we created a new dataset of 22,033 complex\nquestions on Wikipedia tables, which is made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00657v2",
            "date": "2015-08-04 04:36:36+00:00",
            "title": "Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs",
            "authors": [
                "Miguel Ballesteros",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We present extensions to a continuous-state dependency parsing method that\nmakes it applicable to morphologically rich languages. Starting with a\nhigh-performance transition-based parser that uses long short-term memory\n(LSTM) recurrent neural networks to learn representations of the parser state,\nwe replace lookup-based word representations with representations constructed\nfrom the orthographic representations of the words, also using LSTMs. This\nallows statistical sharing across word forms that are similar on the surface.\nExperiments for morphologically rich languages show that the parsing model\nbenefits from incorporating the character-based encodings of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01346v1",
            "date": "2015-08-06 10:15:51+00:00",
            "title": "Word sense disambiguation: a survey",
            "authors": [
                "Alok Ranjan Pal",
                "Diganta Saha"
            ],
            "abstract": "In this paper, we made a survey on Word Sense Disambiguation (WSD). Near\nabout in all major languages around the world, research in WSD has been\nconducted upto different extents. In this paper, we have gone through a survey\nregarding the different approaches adopted in different research works, the\nState of the Art in the performance in this domain, recent works in different\nIndian languages and finally a survey in Bengali language. We have made a\nsurvey on different competitions in this field and the bench mark results,\nobtained from those competitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01349v1",
            "date": "2015-08-06 10:26:40+00:00",
            "title": "Automatic classification of bengali sentences based on sense definitions present in bengali wordnet",
            "authors": [
                "Alok Ranjan Pal",
                "Diganta Saha",
                "Niladri Sekhar Dash"
            ],
            "abstract": "Based on the sense definition of words available in the Bengali WordNet, an\nattempt is made to classify the Bengali sentences automatically into different\ngroups in accordance with their underlying senses. The input sentences are\ncollected from 50 different categories of the Bengali text corpus developed in\nthe TDIL project of the Govt. of India, while information about the different\nsenses of particular ambiguous lexical item is collected from Bengali WordNet.\nIn an experimental basis we have used Naive Bayes probabilistic model as a\nuseful classifier of sentences. We have applied the algorithm over 1747\nsentences that contain a particular Bengali lexical item which, because of its\nambiguous nature, is able to trigger different senses that render sentences in\ndifferent meanings. In our experiment we have achieved around 84% accurate\nresult on the sense classification over the total input sentences. We have\nanalyzed those residual sentences that did not comply with our experiment and\ndid affect the results to note that in many cases, wrong syntactic structures\nand less semantic information are the main hurdles in semantic classification\nof sentences. The applicational relevance of this study is attested in\nautomatic text classification, machine learning, information extraction, and\nword sense disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01476v1",
            "date": "2015-08-06 18:02:54+00:00",
            "title": "Hyponymy extraction of domain ontology concept based on ccrfs and hierarchy clustering",
            "authors": [
                "Qiang Zhan",
                "Chunhong Wang"
            ],
            "abstract": "Concept hierarchy is the backbone of ontology, and the concept hierarchy\nacquisition has been a hot topic in the field of ontology learning. this paper\nproposes a hyponymy extraction method of domain ontology concept based on\ncascaded conditional random field(CCRFs) and hierarchy clustering. It takes\nfree text as extracting object, adopts CCRFs identifying the domain concepts.\nFirst the low layer of CCRFs is used to identify simple domain concept, then\nthe results are sent to the high layer, in which the nesting concepts are\nrecognized. Next we adopt hierarchy clustering to identify the hyponymy\nrelation between domain ontology concepts. The experimental results demonstrate\nthe proposed method is efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01718v1",
            "date": "2015-08-07 15:06:13+00:00",
            "title": "Study of Phonemes Confusions in Hierarchical Automatic Phoneme Recognition System",
            "authors": [
                "Rimah Amami",
                "Noureddine Ellouze"
            ],
            "abstract": "In this paper, we have analyzed the impact of confusions on the robustness of\nphoneme recognitions system. The confusions are detected at the pronunciation\nand the confusions matrices of the phoneme recognizer. The confusions show that\nsome similarities between phonemes at the pronunciation affect significantly\nthe recognition rates. This paper proposes to understand those confusions in\norder to improve the performance of the phoneme recognition system by isolating\nthe problematic phonemes. Confusion analysis leads to build a new hierarchical\nrecognizer using new phoneme distribution and the information from the\nconfusion matrices. This new hierarchical phoneme recognition system shows\nsignificant improvements of the recognition rates on TIMIT database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01745v2",
            "date": "2015-08-07 16:16:44+00:00",
            "title": "Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems",
            "authors": [
                "Tsung-Hsien Wen",
                "Milica Gasic",
                "Nikola Mrksic",
                "Pei-Hao Su",
                "David Vandyke",
                "Steve Young"
            ],
            "abstract": "Natural language generation (NLG) is a critical component of spoken dialogue\nand it has a significant impact both on usability and perceived quality. Most\nNLG systems in common use employ rules and heuristics and tend to generate\nrigid and stylised responses without the natural variation of human language.\nThey are also not easily scaled to systems covering multiple domains and\nlanguages. This paper presents a statistical language generator based on a\nsemantically controlled Long Short-term Memory (LSTM) structure. The LSTM\ngenerator can learn from unaligned data by jointly optimising sentence planning\nand surface realisation using a simple cross entropy training criterion, and\nlanguage variation can be easily achieved by sampling from output candidates.\nWith fewer heuristics, an objective evaluation in two differing test domains\nshowed the proposed method improved performance compared to previous methods.\nHuman judges scored the LSTM system higher on informativeness and naturalness\nand overall preferred it to the other systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01755v1",
            "date": "2015-08-07 16:34:11+00:00",
            "title": "Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking",
            "authors": [
                "Tsung-Hsien Wen",
                "Milica Gasic",
                "Dongho Kim",
                "Nikola Mrksic",
                "Pei-Hao Su",
                "David Vandyke",
                "Steve Young"
            ],
            "abstract": "The natural language generation (NLG) component of a spoken dialogue system\n(SDS) usually needs a substantial amount of handcrafting or a well-labeled\ndataset to be trained on. These limitations add significantly to development\ncosts and make cross-domain, multi-lingual dialogue systems intractable.\nMoreover, human languages are context-aware. The most natural response should\nbe directly learned from data rather than depending on predefined syntaxes or\nrules. This paper presents a statistical language generator based on a joint\nrecurrent and convolutional neural network structure which can be trained on\ndialogue act-utterance pairs without any semantic alignments or predefined\ngrammar trees. Objective metrics suggest that this new model outperforms\nprevious methods under the same experimental conditions. Results of an\nevaluation by human judges indicate that it produces not only high quality but\nlinguistically varied utterances which are preferred compared to n-gram and\nrule-based systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01991v1",
            "date": "2015-08-09 06:32:47+00:00",
            "title": "Bidirectional LSTM-CRF Models for Sequence Tagging",
            "authors": [
                "Zhiheng Huang",
                "Wei Xu",
                "Kai Yu"
            ],
            "abstract": "In this paper, we propose a variety of Long Short-Term Memory (LSTM) based\nmodels for sequence tagging. These models include LSTM networks, bidirectional\nLSTM (BI-LSTM) networks, LSTM with a Conditional Random Field (CRF) layer\n(LSTM-CRF) and bidirectional LSTM with a CRF layer (BI-LSTM-CRF). Our work is\nthe first to apply a bidirectional LSTM CRF (denoted as BI-LSTM-CRF) model to\nNLP benchmark sequence tagging data sets. We show that the BI-LSTM-CRF model\ncan efficiently use both past and future input features thanks to a\nbidirectional LSTM component. It can also use sentence level tag information\nthanks to a CRF layer. The BI-LSTM-CRF model can produce state of the art (or\nclose to) accuracy on POS, chunking and NER data sets. In addition, it is\nrobust and has less dependence on word embedding as compared to previous\nobservations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01996v2",
            "date": "2015-08-09 07:55:51+00:00",
            "title": "An Automatic Machine Translation Evaluation Metric Based on Dependency Parsing Model",
            "authors": [
                "Hui Yu",
                "Xiaofeng Wu",
                "Wenbin Jiang",
                "Qun Liu",
                "ShouXun Lin"
            ],
            "abstract": "Most of the syntax-based metrics obtain the similarity by comparing the\nsub-structures extracted from the trees of hypothesis and reference. These\nsub-structures are defined by human and can't express all the information in\nthe trees because of the limited length of sub-structures. In addition, the\noverlapped parts between these sub-structures are computed repeatedly. To avoid\nthese problems, we propose a novel automatic evaluation metric based on\ndependency parsing model, with no need to define sub-structures by human.\nFirst, we train a dependency parsing model by the reference dependency tree.\nThen we generate the hypothesis dependency tree and the corresponding\nprobability by the dependency parsing model. The quality of the hypothesis can\nbe judged by this probability. In order to obtain the lexicon similarity, we\nalso introduce the unigram F-score to the new metric. Experiment results show\nthat the new metric gets the state-of-the-art performance on system level, and\nis comparable with METEOR on sentence level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02060v1",
            "date": "2015-04-13 15:56:00+00:00",
            "title": "Egyptian Dialect Stopword List Generation from Social Network Data",
            "authors": [
                "Walaa Medhat",
                "Ahmed H. Yousef",
                "Hoda Korashy"
            ],
            "abstract": "This paper proposes a methodology for generating a stopword list from online\nsocial network (OSN) corpora in Egyptian Dialect(ED). The aim of the paper is\nto investigate the effect of removingED stopwords on the Sentiment Analysis\n(SA) task. The stopwords lists generated before were on Modern Standard Arabic\n(MSA) which is not the common language used in OSN. We have generated a\nstopword list of Egyptian dialect to be used with the OSN corpora. We compare\nthe efficiency of text classification when using the generated list along with\npreviously generated lists of MSA and combining the Egyptian dialect list with\nthe MSA list. The text classification was performed using Na\\\"ive Bayes and\nDecision Tree classifiers and two feature selection approaches, unigram and\nbigram. The experiments show that removing ED stopwords give better performance\nthan using lists of MSA stopwords only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02096v2",
            "date": "2015-08-09 23:41:38+00:00",
            "title": "Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation",
            "authors": [
                "Wang Ling",
                "Tiago Lu\u00eds",
                "Lu\u00eds Marujo",
                "Ram\u00f3n Fernandez Astudillo",
                "Silvio Amir",
                "Chris Dyer",
                "Alan W. Black",
                "Isabel Trancoso"
            ],
            "abstract": "We introduce a model for constructing vector representations of words by\ncomposing characters using bidirectional LSTMs. Relative to traditional word\nrepresentation models that have independent vectors for each word type, our\nmodel requires only a single vector per character type and a fixed set of\nparameters for the compositional model. Despite the compactness of this model\nand, more importantly, the arbitrary nature of the form-function relationship\nin language, our \"composed\" word representations yield state-of-the-art results\nin language modeling and part-of-speech tagging. Benefits over traditional\nbaselines are particularly pronounced in morphologically rich languages (e.g.,\nTurkish).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02142v1",
            "date": "2015-08-10 07:02:49+00:00",
            "title": "Feature-based Decipherment for Large Vocabulary Machine Translation",
            "authors": [
                "Iftekhar Naim",
                "Daniel Gildea"
            ],
            "abstract": "Orthographic similarities across languages provide a strong signal for\nprobabilistic decipherment, especially for closely related language pairs. The\nexisting decipherment models, however, are not well-suited for exploiting these\northographic similarities. We propose a log-linear model with latent variables\nthat incorporates orthographic similarity features. Maximum likelihood training\nis computationally expensive for the proposed log-linear model. To address this\nchallenge, we perform approximate inference via MCMC sampling and contrastive\ndivergence. Our results show that the proposed log-linear model with\ncontrastive divergence scales to large vocabularies and outperforms the\nexisting generative decipherment models by exploiting the orthographic\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02225v2",
            "date": "2015-08-10 12:46:52+00:00",
            "title": "Improve the Evaluation of Fluency Using Entropy for Machine Translation Evaluation Metrics",
            "authors": [
                "Hui Yu",
                "Xiaofeng Wu",
                "Wenbin Jiang",
                "Qun Liu",
                "Shouxun Lin"
            ],
            "abstract": "The widely-used automatic evaluation metrics cannot adequately reflect the\nfluency of the translations. The n-gram-based metrics, like BLEU, limit the\nmaximum length of matched fragments to n and cannot catch the matched fragments\nlonger than n, so they can only reflect the fluency indirectly. METEOR, which\nis not limited by n-gram, uses the number of matched chunks but it does not\nconsider the length of each chunk. In this paper, we propose an entropy-based\nmethod, which can sufficiently reflect the fluency of translations through the\ndistribution of matched words. This method can easily combine with the\nwidely-used automatic evaluation metrics to improve the evaluation of fluency.\nExperiments show that the correlations of BLEU and METEOR are improved on\nsentence level after combining with the entropy-based method on WMT 2010 and\nWMT 2012.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02285v1",
            "date": "2015-08-10 15:29:22+00:00",
            "title": "Adapting Phrase-based Machine Translation to Normalise Medical Terms in Social Media Messages",
            "authors": [
                "Nut Limsopatham",
                "Nigel Collier"
            ],
            "abstract": "Previous studies have shown that health reports in social media, such as\nDailyStrength and Twitter, have potential for monitoring health conditions\n(e.g. adverse drug reactions, infectious diseases) in particular communities.\nHowever, in order for a machine to understand and make inferences on these\nhealth conditions, the ability to recognise when laymen's terms refer to a\nparticular medical concept (i.e.\\ text normalisation) is required. To achieve\nthis, we propose to adapt an existing phrase-based machine translation (MT)\ntechnique and a vector representation of words to map between a social media\nphrase and a medical concept. We evaluate our proposed approach using a\ncollection of phrases from tweets related to adverse drug reactions. Our\nexperimental results show that the combination of a phrase-based MT technique\nand the similarity between word vector representations outperforms the\nbaselines that apply only either of them by up to 55%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02297v1",
            "date": "2015-08-10 15:52:49+00:00",
            "title": "Measuring Word Significance using Distributed Representations of Words",
            "authors": [
                "Adriaan M. J. Schakel",
                "Benjamin J. Wilson"
            ],
            "abstract": "Distributed representations of words as real-valued vectors in a relatively\nlow-dimensional space aim at extracting syntactic and semantic features from\nlarge text corpora. A recently introduced neural network, named word2vec\n(Mikolov et al., 2013a; Mikolov et al., 2013b), was shown to encode semantic\ninformation in the direction of the word vectors. In this brief report, it is\nproposed to use the length of the vectors, together with the term frequency, as\nmeasure of word significance in a corpus. Experimental evidence using a\ndomain-specific corpus of abstracts is presented to support this proposal. A\nuseful visualization technique for text corpora emerges, where words are mapped\nonto a two-dimensional plane and automatically ranked by significance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02445v1",
            "date": "2015-08-10 22:24:36+00:00",
            "title": "Removing Biases from Trainable MT Metrics by Using Self-Training",
            "authors": [
                "Milo\u0161 Stanojevi\u0107"
            ],
            "abstract": "Most trainable machine translation (MT) metrics train their weights on human\njudgments of state-of-the-art MT systems outputs. This makes trainable metrics\nbiases in many ways. One of them is preferring longer translations. These\nbiased metrics when used for tuning are evaluating different types of\ntranslations -- n-best lists of translations with very diverse quality. Systems\ntuned with these metrics tend to produce overly long translations that are\npreferred by the metric but not by humans. This is usually solved by manually\ntweaking metric's weights to equally value recall and precision. Our solution\nis more general: (1) it does not address only the recall bias but also all\nother biases that might be present in the data and (2) it does not require any\nknowledge of the types of features used which is useful in cases when manual\ntuning of metric's weights is not possible. This is accomplished by\nself-training on unlabeled n-best lists by using metric that was initially\ntrained on standard human judgments. One way of looking at this is as domain\nadaptation from the domain of state-of-the-art MT translations to diverse\nn-best list translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03040v7",
            "date": "2015-08-12 09:04:01+00:00",
            "title": "Syntax Evolution: Problems and Recursion",
            "authors": [
                "Ram\u00f3n Casares"
            ],
            "abstract": "To investigate the evolution of syntax, we need to ascertain the evolutionary\nr\\^ole of syntax and, before that, the very nature of syntax. Here, we will\nassume that syntax is computing. And then, since we are computationally Turing\ncomplete, we meet an evolutionary anomaly, the anomaly of sytax: we are\nsyntactically too competent for syntax. Assuming that problem solving is\ncomputing, and realizing that the evolutionary advantage of Turing completeness\nis full problem solving and not syntactic proficiency, we explain the anomaly\nof syntax by postulating that syntax and problem solving co-evolved in humans\ntowards Turing completeness. Examining the requirements that full problem\nsolving impose on language, we find firstly that semantics is not sufficient\nand that syntax is necessary to represent problems. Our final conclusion is\nthat full problem solving requires a functional semantics on an infinite\ntree-structured syntax. Besides these results, the introduction of Turing\ncompleteness and problem solving to explain the evolution of syntax should help\nus to fit the evolution of language within the evolution of cognition, giving\nus some new clues to understand the elusive relation between language and\nthinking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04025v5",
            "date": "2015-08-17 13:43:19+00:00",
            "title": "Effective Approaches to Attention-based Neural Machine Translation",
            "authors": [
                "Minh-Thang Luong",
                "Hieu Pham",
                "Christopher D. Manning"
            ],
            "abstract": "An attentional mechanism has lately been used to improve neural machine\ntranslation (NMT) by selectively focusing on parts of the source sentence\nduring translation. However, there has been little work exploring useful\narchitectures for attention-based NMT. This paper examines two simple and\neffective classes of attentional mechanism: a global approach which always\nattends to all source words and a local one that only looks at a subset of\nsource words at a time. We demonstrate the effectiveness of both approaches\nover the WMT translation tasks between English and German in both directions.\nWith local attention, we achieve a significant gain of 5.0 BLEU points over\nnon-attentional systems which already incorporate known techniques such as\ndropout. Our ensemble model using different attention architectures has\nestablished a new state-of-the-art result in the WMT'15 English to German\ntranslation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over\nthe existing best system backed by NMT and an n-gram reranker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04257v2",
            "date": "2015-08-18 09:29:22+00:00",
            "title": "Learning Meta-Embeddings by Using Ensembles of Embedding Sets",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word embeddings -- distributed representations of words -- in deep learning\nare beneficial for many tasks in natural language processing (NLP). However,\ndifferent embedding sets vary greatly in quality and characteristics of the\ncaptured semantics. Instead of relying on a more advanced algorithm for\nembedding learning, this paper proposes an ensemble approach of combining\ndifferent public embedding sets with the aim of learning meta-embeddings.\nExperiments on word similarity and analogy tasks and on part-of-speech tagging\nshow better performance of meta-embeddings compared to individual embedding\nsets. One advantage of meta-embeddings is the increased vocabulary coverage. We\nwill release our meta-embeddings publicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04271v1",
            "date": "2015-08-18 10:29:10+00:00",
            "title": "Probabilistic Modelling of Morphologically Rich Languages",
            "authors": [
                "Jan A. Botha"
            ],
            "abstract": "This thesis investigates how the sub-structure of words can be accounted for\nin probabilistic models of language. Such models play an important role in\nnatural language processing tasks such as translation or speech recognition,\nbut often rely on the simplistic assumption that words are opaque symbols. This\nassumption does not fit morphologically complex language well, where words can\nhave rich internal structure and sub-word elements are shared across distinct\nword forms.\n  Our approach is to encode basic notions of morphology into the assumptions of\nthree different types of language models, with the intention that leveraging\nshared sub-word structure can improve model performance and help overcome data\nsparsity that arises from morphological processes.\n  In the context of n-gram language modelling, we formulate a new Bayesian\nmodel that relies on the decomposition of compound words to attain better\nsmoothing, and we develop a new distributed language model that learns vector\nrepresentations of morphemes and leverages them to link together\nmorphologically related words. In both cases, we show that accounting for word\nsub-structure improves the models' intrinsic performance and provides benefits\nwhen applied to other tasks, including machine translation.\n  We then shift the focus beyond the modelling of word sequences and consider\nmodels that automatically learn what the sub-word elements of a given language\nare, given an unannotated list of words. We formulate a novel model that can\nlearn discontiguous morphemes in addition to the more conventional contiguous\nmorphemes that most previous models are limited to. This approach is\ndemonstrated on Semitic languages, and we find that modelling discontiguous\nsub-word structures leads to improvements in the task of segmenting words into\ntheir contiguous morphemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04515v1",
            "date": "2015-08-19 03:26:05+00:00",
            "title": "Exploring Metaphorical Senses and Word Representations for Identifying Metonyms",
            "authors": [
                "Wei Zhang",
                "Judith Gelernter"
            ],
            "abstract": "A metonym is a word with a figurative meaning, similar to a metaphor. Because\nmetonyms are closely related to metaphors, we apply features that are used\nsuccessfully for metaphor recognition to the task of detecting metonyms. On the\nACL SemEval 2007 Task 8 data with gold standard metonym annotations, our system\nachieved 86.45% accuracy on the location metonyms. Our code can be found on\nGitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05051v1",
            "date": "2015-08-20 17:21:50+00:00",
            "title": "Auto-Sizing Neural Networks: With Applications to n-gram Language Models",
            "authors": [
                "Kenton Murray",
                "David Chiang"
            ],
            "abstract": "Neural networks have been shown to improve performance across a range of\nnatural-language tasks. However, designing and training them can be\ncomplicated. Frequently, researchers resort to repeated experimentation to pick\noptimal settings. In this paper, we address the issue of choosing the correct\nnumber of units in hidden layers. We introduce a method for automatically\nadjusting network size by pruning out hidden units through $\\ell_{\\infty,1}$\nand $\\ell_{2,1}$ regularization. We apply this method to language modeling and\ndemonstrate its ability to correctly choose the number of hidden units while\nmaintaining perplexity. We also include these models in a machine translation\ndecoder and show that these smaller neural models maintain the significant\nimprovements of their unpruned versions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05154v2",
            "date": "2015-08-21 00:25:51+00:00",
            "title": "Posterior calibration and exploratory analysis for natural language processing models",
            "authors": [
                "Khanh Nguyen",
                "Brendan O'Connor"
            ],
            "abstract": "Many models in natural language processing define probabilistic distributions\nover linguistic structures. We argue that (1) the quality of a model' s\nposterior distribution can and should be directly evaluated, as to whether\nprobabilities correspond to empirical frequencies, and (2) NLP uncertainty can\nbe projected not only to pipeline components, but also to exploratory data\nanalysis, telling a user when to trust and not trust the NLP analysis. We\npresent a method to analyze calibration, and apply it to compare the\nmiscalibration of several commonly used models. We also contribute a\ncoreference sampling algorithm that can create confidence intervals for a\npolitical event extraction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05326v1",
            "date": "2015-08-21 16:17:01+00:00",
            "title": "A large annotated corpus for learning natural language inference",
            "authors": [
                "Samuel R. Bowman",
                "Gabor Angeli",
                "Christopher Potts",
                "Christopher D. Manning"
            ],
            "abstract": "Understanding entailment and contradiction is fundamental to understanding\nnatural language, and inference about entailment and contradiction is a\nvaluable testing ground for the development of semantic representations.\nHowever, machine learning research in this area has been dramatically limited\nby the lack of large-scale resources. To address this, we introduce the\nStanford Natural Language Inference corpus, a new, freely available collection\nof labeled sentence pairs, written by humans doing a novel grounded task based\non image captioning. At 570K pairs, it is two orders of magnitude larger than\nall other resources of its type. This increase in scale allows lexicalized\nclassifiers to outperform some sophisticated existing entailment models, and it\nallows a neural network-based model to perform competitively on natural\nlanguage inference benchmarks for the first time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06044v1",
            "date": "2015-08-25 06:34:00+00:00",
            "title": "Visualizing NLP annotations for Crowdsourcing",
            "authors": [
                "Hanchuan Li",
                "Haichen Shen",
                "Shengliang Xu",
                "Congle Zhang"
            ],
            "abstract": "Visualizing NLP annotation is useful for the collection of training data for\nthe statistical NLP approaches. Existing toolkits either provide limited visual\naid, or introduce comprehensive operators to realize sophisticated linguistic\nrules. Workers must be well trained to use them. Their audience thus can hardly\nbe scaled to large amounts of non-expert crowdsourced workers. In this paper,\nwe present CROWDANNO, a visualization toolkit to allow crowd-sourced workers to\nannotate two general categories of NLP problems: clustering and parsing.\nWorkers can finish the tasks with simplified operators in an interactive\ninterface, and fix errors conveniently. User studies show our toolkit is very\nfriendly to NLP non-experts, and allow them to produce high quality labels for\nseveral sophisticated problems. We release our source code and toolkit to spur\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06491v2",
            "date": "2015-08-26 13:44:54+00:00",
            "title": "Alignment-based compositional semantics for instruction following",
            "authors": [
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "This paper describes an alignment-based model for interpreting natural\nlanguage instructions in context. We approach instruction following as a search\nover plans, scoring sequences of actions conditioned on structured observations\nof text and the environment. By explicitly modeling both the low-level\ncompositional structure of individual actions and the high-level structure of\nfull plans, we are able to learn both grounded representations of sentence\nmeaning and pragmatic constraints on interpretation. To demonstrate the model's\nflexibility, we apply it to a diverse set of benchmark tasks. On every task, we\noutperform strong task-specific baselines, and achieve several new\nstate-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06669v1",
            "date": "2015-08-26 21:25:25+00:00",
            "title": "Component-Enhanced Chinese Character Embeddings",
            "authors": [
                "Yanran Li",
                "Wenjie Li",
                "Fei Sun",
                "Sujian Li"
            ],
            "abstract": "Distributed word representations are very useful for capturing semantic\ninformation and have been successfully applied in a variety of NLP tasks,\nespecially on English. In this work, we innovatively develop two\ncomponent-enhanced Chinese character embedding models and their bigram\nextensions. Distinguished from English word embeddings, our models explore the\ncompositions of Chinese characters, which often serve as semantic indictors\ninherently. The evaluations on both word similarity and text classification\ndemonstrate the effectiveness of our models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.07544v2",
            "date": "2015-08-30 08:39:02+00:00",
            "title": "Computational Sociolinguistics: A Survey",
            "authors": [
                "Dong Nguyen",
                "A. Seza Do\u011fru\u00f6z",
                "Carolyn P. Ros\u00e9",
                "Franciska de Jong"
            ],
            "abstract": "Language is a social phenomenon and variation is inherent to its social\nnature. Recently, there has been a surge of interest within the computational\nlinguistics (CL) community in the social dimension of language. In this article\nwe present a survey of the emerging field of \"Computational Sociolinguistics\"\nthat reflects this increased interest. We aim to provide a comprehensive\noverview of CL research on sociolinguistic themes, featuring topics such as the\nrelation between language and social identity, language use in social\ninteraction and multilingual communication. Moreover, we demonstrate the\npotential for synergy between the research communities involved, by showing how\nthe large-scale data-driven methods that are widely used in CL can complement\nexisting sociolinguistic studies, and how sociolinguistics can inform and\nchallenge the methods and assumptions employed in CL studies. We hope to convey\nthe possible benefits of a closer collaboration between the two communities and\nconclude with a discussion of open challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.07555v1",
            "date": "2015-08-30 11:22:38+00:00",
            "title": "An Event Network for Exploring Open Information",
            "authors": [
                "Yanping Chen"
            ],
            "abstract": "In this paper, an event network is presented for exploring open information,\nwhere linguistic units about an event are organized for analysing. The process\nis divided into three steps: document event detection, event network\nconstruction and event network analysis. First, by implementing event detection\nor tracking, documents are retrospectively (or on-line) organized into document\nevents. Secondly, for each of the document event, linguistic units are\nextracted and combined into event networks. Thirdly, various analytic methods\nare proposed for event network analysis. In our application methodologies are\npresented for exploring open information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.07909v5",
            "date": "2015-08-31 16:37:31+00:00",
            "title": "Neural Machine Translation of Rare Words with Subword Units",
            "authors": [
                "Rico Sennrich",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "Neural machine translation (NMT) models typically operate with a fixed\nvocabulary, but translation is an open-vocabulary problem. Previous work\naddresses the translation of out-of-vocabulary words by backing off to a\ndictionary. In this paper, we introduce a simpler and more effective approach,\nmaking the NMT model capable of open-vocabulary translation by encoding rare\nand unknown words as sequences of subword units. This is based on the intuition\nthat various word classes are translatable via smaller units than words, for\ninstance names (via character copying or transliteration), compounds (via\ncompositional translation), and cognates and loanwords (via phonological and\nmorphological transformations). We discuss the suitability of different word\nsegmentation techniques, including simple character n-gram models and a\nsegmentation based on the byte pair encoding compression algorithm, and\nempirically show that subword models improve over a back-off dictionary\nbaseline for the WMT 15 translation tasks English-German and English-Russian by\n1.1 and 1.3 BLEU, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.00215v1",
            "date": "2015-11-01 07:59:48+00:00",
            "title": "A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding",
            "authors": [
                "Peilu Wang",
                "Yao Qian",
                "Frank K. Soong",
                "Lei He",
                "Hai Zhao"
            ],
            "abstract": "Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN) has\nbeen shown to be very effective for modeling and predicting sequential data,\ne.g. speech utterances or handwritten documents. In this study, we propose to\nuse BLSTM-RNN for a unified tagging solution that can be applied to various\ntagging tasks including part-of-speech tagging, chunking and named entity\nrecognition. Instead of exploiting specific features carefully optimized for\neach task, our solution only uses one set of task-independent features and\ninternal representations learnt from unlabeled text for all tasks.Requiring no\ntask specific knowledge or sophisticated feature engineering, our approach gets\nnearly state-of-the-art performance in all these three tagging tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01574v2",
            "date": "2015-11-05 01:45:29+00:00",
            "title": "Multinomial Loss on Held-out Data for the Sparse Non-negative Matrix Language Model",
            "authors": [
                "Ciprian Chelba",
                "Fernando Pereira"
            ],
            "abstract": "We describe Sparse Non-negative Matrix (SNM) language model estimation using\nmultinomial loss on held-out data.\n  Being able to train on held-out data is important in practical situations\nwhere the training data is usually mismatched from the held-out/test data. It\nis also less constrained than the previous training algorithm using\nleave-one-out on training data: it allows the use of richer meta-features in\nthe adjustment model, e.g. the diversity counts used by Kneser-Ney smoothing\nwhich would be difficult to deal with correctly in leave-one-out training.\n  In experiments on the one billion words language modeling benchmark, we are\nable to slightly improve on our previous results which use a different loss\nfunction, and employ leave-one-out training on a subset of the main training\nset. Surprisingly, an adjustment model with meta-features that discard all\nlexical information can perform as well as lexicalized meta-features. We find\nthat fairly small amounts of held-out data (on the order of 30-70 thousand\nwords) are sufficient for training the adjustment model.\n  In a real-life scenario where the training data is a mix of data sources that\nare imbalanced in size, and of different degrees of relevance to the held-out\nand test data, taking into account the data source for a given skip-/n-gram\nfeature and combining them for best performance on held-out/test data improves\nover skip-/n-gram SNM models trained on pooled data by about 8% in the SMT\nsetup, or as much as 15% in the ASR/IME setup.\n  The ability to mix various data sources based on how relevant they are to a\nmismatched held-out set is probably the most attractive feature of the new\nestimation method for SNM LM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01665v1",
            "date": "2015-11-05 09:25:21+00:00",
            "title": "An Empirical Study on Sentiment Classification of Chinese Review using Word Embedding",
            "authors": [
                "Yiou Lin",
                "Hang Lei",
                "Jia Wu",
                "Xiaoyu Li"
            ],
            "abstract": "In this article, how word embeddings can be used as features in Chinese\nsentiment classification is presented. Firstly, a Chinese opinion corpus is\nbuilt with a million comments from hotel review websites. Then the word\nembeddings which represent each comment are used as input in different machine\nlearning methods for sentiment classification, including SVM, Logistic\nRegression, Convolutional Neural Network (CNN) and ensemble methods. These\nmethods get better performance compared with N-gram models using Naive Bayes\n(NB) and Maximum Entropy (ME). Finally, a combination of machine learning\nmethods is proposed which presents an outstanding performance in precision,\nrecall and F1 score. After selecting the most useful methods to construct the\ncombinational model and testing over the corpus, the final F1 score is 0.920.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01666v1",
            "date": "2015-11-05 09:25:41+00:00",
            "title": "Comparing Writing Styles using Word Embedding and Dynamic Time Warping",
            "authors": [
                "Abhinav Tushar",
                "Abhinav Dahiya"
            ],
            "abstract": "The development of plot or story in novels is reflected in the content and\nthe words used. The flow of sentiments, which is one aspect of writing style,\ncan be quantified by analyzing the flow of words. This study explores literary\nworks as signals in word embedding space and tries to compare writing styles of\npopular classic novels using dynamic time warping.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01756v2",
            "date": "2015-11-05 14:20:01+00:00",
            "title": "\"Pale as death\" or \"p\u00e2le comme la mort\" : Frozen similes used as literary clich\u00e9s",
            "authors": [
                "Suzanne Mpouli",
                "Jean-Gabriel Ganascia"
            ],
            "abstract": "The present study is focused on the automatic identification and description\nof frozen similes in British and French novels written between the 19 th\ncentury and the beginning of the 20 th century. Two main patterns of frozen\nsimiles were considered: adjectival ground + simile marker + nominal vehicle\n(e.g. happy as a lark) and eventuality + simile marker + nominal vehicle (e.g.\nsleep like a top). All potential similes and their components were first\nextracted using a rule-based algorithm. Then, frozen similes were identified\nbased on reference lists of existing similes and semantic distance between the\ntenor and the vehicle. The results obtained tend to confirm the fact that\nfrozen similes are not used haphazardly in literary texts. In addition,\ncontrary to how they are often presented, frozen similes often go beyond the\nground or the eventuality and the vehicle to also include the tenor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02014v2",
            "date": "2015-11-06 09:39:24+00:00",
            "title": "Population size predicts lexical diversity, but so does the mean sea level - why it is important to correctly account for the structure of temporal data",
            "authors": [
                "Alexander Koplenig",
                "Carolin Mueller-Spitzer"
            ],
            "abstract": "In order to demonstrate why it is important to correctly account for the\n(serial dependent) structure of temporal data, we document an apparently\nspectacular relationship between population size and lexical diversity: for\nfive out of seven investigated languages, there is a strong relationship\nbetween population size and lexical diversity of the primary language in this\ncountry. We show that this relationship is the result of a misspecified model\nthat does not consider the temporal aspect of the data by presenting a similar\nbut nonsensical relationship between the global annual mean sea level and\nlexical diversity. Given the fact that in the recent past, several studies were\npublished that present surprising links between different economic, cultural,\npolitical and (socio-)demographical variables on the one hand and cultural or\nlinguistic characteristics on the other hand, but seem to suffer from exactly\nthis problem, we explain the cause of the misspecification and show that it has\nprofound consequences. We demonstrate how simple transformation of the time\nseries can often solve problems of this type and argue that the evaluation of\nthe plausibility of a relationship is important in this context. We hope that\nour paper will help both researchers and reviewers to understand why it is\nimportant to use special models for the analysis of data with a natural\ntemporal ordering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02117v1",
            "date": "2015-11-06 15:36:54+00:00",
            "title": "Introducing SKYSET - a Quintuple Approach for Improving Instructions",
            "authors": [
                "Kerry Fultz",
                "Seth Filip"
            ],
            "abstract": "A new approach called SKYSET (Synthetic Knowledge Yield Social Entities\nTranslation) is proposed to validate completeness and to reduce ambiguity from\nwritten instructional documentation. SKYSET utilizes a quintuple set of\nstandardized categories, which differs from traditional approaches that\ntypically use triples. The SKYSET System defines the categories required to\nform a standard template for representing information that is portable across\ndifferent domains. It provides a standardized framework that enables sentences\nfrom written instructions to be translated into sets of category typed entities\non a table or database. The SKYSET entities contain conceptual units or phrases\nthat represent information from the original source documentation. SKYSET\nenables information concatenation where multiple documents from different\ndomains can be translated and combined into a single common filterable and\nsearchable table of entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02301v4",
            "date": "2015-11-07 04:36:20+00:00",
            "title": "The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations",
            "authors": [
                "Felix Hill",
                "Antoine Bordes",
                "Sumit Chopra",
                "Jason Weston"
            ],
            "abstract": "We introduce a new test of how well language models capture meaning in\nchildren's books. Unlike standard language modelling benchmarks, it\ndistinguishes the task of predicting syntactic function words from that of\npredicting lower-frequency words, which carry greater semantic content. We\ncompare a range of state-of-the-art models, each with a different way of\nencoding what has been previously read. We show that models which store\nexplicit representations of long-term contexts outperform state-of-the-art\nneural language models at predicting semantic content words, although this\nadvantage is not observed for syntactic function words. Interestingly, we find\nthat the amount of text encoded in a single memory representation is highly\ninfluential to the performance: there is a sweet-spot, not too big and not too\nsmall, between single words and full sentences that allows the most meaningful\ninformation in a text to be effectively retained and recalled. Further, the\nattention over such window-based memories can be trained effectively through\nself-supervision. We then assess the generality of this principle by applying\nit to the CNN QA benchmark, which involves identifying named entities in\nparaphrased summaries of news articles, and achieve state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02435v1",
            "date": "2015-11-08 03:44:26+00:00",
            "title": "A Chinese POS Decision Method Using Korean Translation Information",
            "authors": [
                "Son-Il Kwak",
                "O-Chol Kown",
                "Chang-Sin Kim",
                "Yong-Il Pak",
                "Gum-Chol Son",
                "Chol-Jun Hwang",
                "Hyon-Chol Kim",
                "Hyok-Chol Sin",
                "Gyong-Il Hyon",
                "Sok-Min Han"
            ],
            "abstract": "In this paper we propose a method that imitates a translation expert using\nthe Korean translation information and analyse the performance. Korean is good\nat tagging than Chinese, so we can use this property in Chinese POS tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.03053v1",
            "date": "2015-11-10 10:33:47+00:00",
            "title": "Investigating the stylistic relevance of adjective and verb simile markers",
            "authors": [
                "Suzanne Mpouli",
                "Jean-Gabriel Ganascia"
            ],
            "abstract": "Similes play an important role in literary texts not only as rhetorical\ndevices and as figures of speech but also because of their evocative power,\ntheir aptness for description and the relative ease with which they can be\ncombined with other figures of speech (Israel et al. 2004). Detecting all types\nof simile constructions in a particular text therefore seems crucial when\nanalysing the style of an author. Few research studies however have been\ndedicated to the study of less prominent simile markers in fictional prose and\ntheir relevance for stylistic studies. The present paper studies the frequency\nof adjective and verb simile markers in a corpus of British and French novels\nin order to determine which ones are really informative and worth including in\na stylistic analysis. Furthermore, are those adjectives and verb simile markers\nused differently in both languages?",
            "journal ref": "Corpus Linguistics 2015, Jul 2015, Lancaster, United Kingdom",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.03088v1",
            "date": "2015-11-10 12:34:47+00:00",
            "title": "USFD: Twitter NER with Drift Compensation and Linked Data",
            "authors": [
                "Leon Derczynski",
                "Isabelle Augenstein",
                "Kalina Bontcheva"
            ],
            "abstract": "This paper describes a pilot NER system for Twitter, comprising the USFD\nsystem entry to the W-NUT 2015 NER shared task. The goal is to correctly label\nentities in a tweet dataset, using an inventory of ten types. We employ\nstructured learning, drawing on gazetteers taken from Linked Data, and on\nunsupervised clustering features, and attempting to compensate for stylistic\nand topic drift - a key challenge in social media text. Our result is\ncompetitive; we provide an analysis of the components of our methodology, and\nan examination of the target dataset in the context of this task.",
            "journal ref": "Proceedings of the ACL Workshop on Noisy User-generated Text\n  (2015), pp. 48--53",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.03729v2",
            "date": "2015-11-11 23:24:29+00:00",
            "title": "Larger-Context Language Modelling",
            "authors": [
                "Tian Wang",
                "Kyunghyun Cho"
            ],
            "abstract": "In this work, we propose a novel method to incorporate corpus-level discourse\ninformation into language modelling. We call this larger-context language\nmodel. We introduce a late fusion approach to a recurrent language model based\non long short-term memory units (LSTM), which helps the LSTM unit keep\nintra-sentence dependencies and inter-sentence dependencies separate from each\nother. Through the evaluation on three corpora (IMDB, BBC, and PennTree Bank),\nwe demon- strate that the proposed model improves perplexity significantly. In\nthe experi- ments, we evaluate the proposed approach while varying the number\nof context sentences and observe that the proposed late fusion is superior to\nthe usual way of incorporating additional inputs to the LSTM. By analyzing the\ntrained larger- context language model, we discover that content words,\nincluding nouns, adjec- tives and verbs, benefit most from an increasing number\nof context sentences. This analysis suggests that larger-context language model\nimproves the unconditional language model by capturing the theme of a document\nbetter and more easily.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.03924v1",
            "date": "2015-11-12 15:23:37+00:00",
            "title": "A Multilingual FrameNet-based Grammar and Lexicon for Controlled Natural Language",
            "authors": [
                "Normunds Gruzitis",
                "Dana Dann\u00e9lls"
            ],
            "abstract": "Berkeley FrameNet is a lexico-semantic resource for English based on the\ntheory of frame semantics. It has been exploited in a range of natural language\nprocessing applications and has inspired the development of framenets for many\nlanguages. We present a methodological approach to the extraction and\ngeneration of a computational multilingual FrameNet-based grammar and lexicon.\nThe approach leverages FrameNet-annotated corpora to automatically extract a\nset of cross-lingual semantico-syntactic valence patterns. Based on data from\nBerkeley FrameNet and Swedish FrameNet, the proposed approach has been\nimplemented in Grammatical Framework (GF), a categorial grammar formalism\nspecialized for multilingual grammars. The implementation of the grammar and\nlexicon is supported by the design of FrameNet, providing a frame semantic\nabstraction layer, an interlingual semantic API (application programming\ninterface), over the interlingual syntactic API already provided by GF Resource\nGrammar Library. The evaluation of the acquired grammar and lexicon shows the\nfeasibility of the approach. Additionally, we illustrate how the FrameNet-based\ngrammar and lexicon are exploited in two distinct multilingual controlled\nnatural language applications. The produced resources are available under an\nopen source license.",
            "journal ref": "Language Resources and Evaluation 51(1), 37-66, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04586v1",
            "date": "2015-11-14 17:36:43+00:00",
            "title": "Character-based Neural Machine Translation",
            "authors": [
                "Wang Ling",
                "Isabel Trancoso",
                "Chris Dyer",
                "Alan W Black"
            ],
            "abstract": "We introduce a neural machine translation model that views the input and\noutput sentences as sequences of characters rather than words. Since word-level\ninformation provides a crucial source of bias, our input model composes\nrepresentations of character sequences into representations of words (as\ndetermined by whitespace boundaries), and then these are translated using a\njoint attention/translation model. In the target language, the translation is\nmodeled as a sequence of word vectors, but each word is generated one character\nat a time, conditional on the previous character generations in each word. As\nthe representation and generation of words is performed at the character level,\nour model is capable of interpreting and generating unseen word forms. A\nsecondary benefit of this approach is that it alleviates much of the challenges\nassociated with preprocessing/tokenization of the source and target languages.\nWe show that our model can achieve translation results that are on par with\nconventional word-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04623v2",
            "date": "2015-11-14 21:36:38+00:00",
            "title": "Learning to Represent Words in Context with Multilingual Supervision",
            "authors": [
                "Kazuya Kawakami",
                "Chris Dyer"
            ],
            "abstract": "We present a neural network architecture based on bidirectional LSTMs to\ncompute representations of words in the sentential contexts. These\ncontext-sensitive word representations are suitable for, e.g., distinguishing\ndifferent word senses and other context-modulated variations in meaning. To\nlearn the parameters of our model, we use cross-lingual supervision,\nhypothesizing that a good representation of a word in context will be one that\nis sufficient for selecting the correct translation into a second language. We\nevaluate the quality of our representations as features in three downstream\ntasks: prediction of semantic supersenses (which assign nouns and verbs into a\nfew dozen semantic classes), low resource machine translation, and a lexical\nsubstitution task, and obtain state-of-the-art results on all of these.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04661v1",
            "date": "2015-11-15 05:53:13+00:00",
            "title": "A System for Extracting Sentiment from Large-Scale Arabic Social Data",
            "authors": [
                "Hao Wang",
                "Vijay R. Bommireddipalli",
                "Ayman Hanafy",
                "Mohamed Bahgat",
                "Sara Noeman",
                "Ossama S. Emam"
            ],
            "abstract": "Social media data in Arabic language is becoming more and more abundant. It\nis a consensus that valuable information lies in social media data. Mining this\ndata and making the process easier are gaining momentum in the industries. This\npaper describes an enterprise system we developed for extracting sentiment from\nlarge volumes of social data in Arabic dialects. First, we give an overview of\nthe Big Data system for information extraction from multilingual social data\nfrom a variety of sources. Then, we focus on the Arabic sentiment analysis\ncapability that was built on top of the system including normalizing written\nArabic dialects, building sentiment lexicons, sentiment classification, and\nperformance evaluation. Lastly, we demonstrate the value of enriching sentiment\nresults with user profiles in understanding sentiments of a specific user\ngroup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.05076v1",
            "date": "2015-11-16 18:25:33+00:00",
            "title": "Latent Dirichlet Allocation Based Organisation of Broadcast Media Archives for Deep Neural Network Adaptation",
            "authors": [
                "Mortaza Doulaty",
                "Oscar Saz",
                "Raymond W. M. Ng",
                "Thomas Hain"
            ],
            "abstract": "This paper presents a new method for the discovery of latent domains in\ndiverse speech data, for the use of adaptation of Deep Neural Networks (DNNs)\nfor Automatic Speech Recognition. Our work focuses on transcription of\nmulti-genre broadcast media, which is often only categorised broadly in terms\nof high level genres such as sports, news, documentary, etc. However, in terms\nof acoustic modelling these categories are coarse. Instead, it is expected that\na mixture of latent domains can better represent the complex and diverse\nbehaviours within a TV show, and therefore lead to better and more robust\nperformance. We propose a new method, whereby these latent domains are\ndiscovered with Latent Dirichlet Allocation, in an unsupervised manner. These\nare used to adapt DNNs using the Unique Binary Code (UBIC) representation for\nthe LDA domains. Experiments conducted on a set of BBC TV broadcasts, with more\nthan 2,000 shows for training and 47 shows for testing, show that the use of\nLDA-UBIC DNNs reduces the error up to 13% relative compared to the baseline\nhybrid DNN models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.05389v4",
            "date": "2015-11-17 13:18:07+00:00",
            "title": "Learning to retrieve out-of-vocabulary words in speech recognition",
            "authors": [
                "Imran Sheikh",
                "Irina Illina",
                "Dominique Fohr",
                "Georges Linar\u00e8s"
            ],
            "abstract": "Many Proper Names (PNs) are Out-Of-Vocabulary (OOV) words for speech\nrecognition systems used to process diachronic audio data. To help recovery of\nthe PNs missed by the system, relevant OOV PNs can be retrieved out of the many\nOOVs by exploiting semantic context of the spoken content. In this paper, we\npropose two neural network models targeted to retrieve OOV PNs relevant to an\naudio document: (a) Document level Continuous Bag of Words (D-CBOW), (b)\nDocument level Continuous Bag of Weighted Words (D-CBOW2). Both these models\ntake document words as input and learn with an objective to maximise the\nretrieval of co-occurring OOV PNs. With the D-CBOW2 model we propose a new\napproach in which the input embedding layer is augmented with a context anchor\nlayer. This layer learns to assign importance to input words and has the\nability to capture (task specific) key-words in a bag-of-word neural network\nmodel. With experiments on French broadcast news videos we show that these two\nmodels outperform the baseline methods based on raw embeddings from LDA,\nSkip-gram and Paragraph Vectors. Combining the D-CBOW and D-CBOW2 models gives\nfaster convergence during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06246v1",
            "date": "2015-11-19 16:46:49+00:00",
            "title": "Gaussian Mixture Embeddings for Multiple Word Prototypes",
            "authors": [
                "Xinchi Chen",
                "Xipeng Qiu",
                "Jingxiang Jiang",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, word representation has been increasingly focused on for its\nexcellent properties in representing the word semantics. Previous works mainly\nsuffer from the problem of polysemy phenomenon. To address this problem, most\nof previous models represent words as multiple distributed vectors. However, it\ncannot reflect the rich relations between words by representing words as points\nin the embedded space. In this paper, we propose the Gaussian mixture skip-gram\n(GMSG) model to learn the Gaussian mixture embeddings for words based on\nskip-gram framework. Each word can be regarded as a gaussian mixture\ndistribution in the embedded space, and each gaussian component represents a\nword sense. Since the number of senses varies from word to word, we further\npropose the Dynamic GMSG (D-GMSG) model by adaptively increasing the sense\nnumber of words during training. Experiments on four benchmarks show the\neffectiveness of our proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06312v1",
            "date": "2015-11-19 19:13:58+00:00",
            "title": "Good, Better, Best: Choosing Word Embedding Context",
            "authors": [
                "James Cross",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "We propose two methods of learning vector representations of words and\nphrases that each combine sentence context with structural features extracted\nfrom dependency trees. Using several variations of neural network classifier,\nwe show that these combined methods lead to improved performance when used as\ninput features for supervised term-matching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06426v4",
            "date": "2015-11-19 22:30:10+00:00",
            "title": "Reasoning in Vector Space: An Exploratory Study of Question Answering",
            "authors": [
                "Moontae Lee",
                "Xiaodong He",
                "Wen-tau Yih",
                "Jianfeng Gao",
                "Li Deng",
                "Paul Smolensky"
            ],
            "abstract": "Question answering tasks have shown remarkable progress with distributed\nvector representation. In this paper, we investigate the recently proposed\nFacebook bAbI tasks which consist of twenty different categories of questions\nthat require complex reasoning. Because the previous work on bAbI are all\nend-to-end models, errors could come from either an imperfect understanding of\nsemantics or in certain steps of the reasoning. For clearer analysis, we\npropose two vector space models inspired by Tensor Product Representation (TPR)\nto perform knowledge encoding and logical reasoning based on common-sense\ninference. They together achieve near-perfect accuracy on all categories\nincluding positional reasoning and path finding that have proved difficult for\nmost of the previous approaches. We hypothesize that the difficulties in these\ncategories are due to the multi-relations in contrast to uni-relational\ncharacteristic of other categories. Our exploration sheds light on designing\nmore sophisticated dataset and moving one step toward integrating transparent\nand interpretable formalism of TPR into existing learning paradigms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06591v1",
            "date": "2015-11-20 13:41:31+00:00",
            "title": "Polysemy in Controlled Natural Language Texts",
            "authors": [
                "Normunds Gruzitis",
                "Guntis Barzdins"
            ],
            "abstract": "Computational semantics and logic-based controlled natural languages (CNL) do\nnot address systematically the word sense disambiguation problem of content\nwords, i.e., they tend to interpret only some functional words that are crucial\nfor construction of discourse representation structures. We show that\nmicro-ontologies and multi-word units allow integration of the rich and\npolysemous multi-domain background knowledge into CNL thus providing\ninterpretation for the content words. The proposed approach is demonstrated by\nextending the Attempto Controlled English (ACE) with polysemous and procedural\nconstructs resulting in a more natural CNL named PAO covering narrative\nmulti-domain texts.",
            "journal ref": "Controlled Natural Language, Lecture Notes in Computer Science,\n  Vol. 5972, Springer, 2010, pp. 102-120",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06709v4",
            "date": "2015-11-20 17:58:37+00:00",
            "title": "Improving Neural Machine Translation Models with Monolingual Data",
            "authors": [
                "Rico Sennrich",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "Neural Machine Translation (NMT) has obtained state-of-the art performance\nfor several language pairs, while only using parallel data for training.\nTarget-side monolingual data plays an important role in boosting fluency for\nphrase-based statistical machine translation, and we investigate the use of\nmonolingual data for NMT. In contrast to previous work, which combines NMT\nmodels with separately trained language models, we note that encoder-decoder\nNMT architectures already have the capacity to learn the same information as a\nlanguage model, and we explore strategies to train with monolingual data\nwithout changing the neural network architecture. By pairing monolingual\ntraining data with an automatic back-translation, we can treat it as additional\nparallel training data, and we obtain substantial improvements on the WMT 15\ntask English<->German (+2.8-3.7 BLEU), and for the low-resourced IWSLT 14 task\nTurkish->English (+2.1-3.4 BLEU), obtaining new state-of-the-art results. We\nalso show that fine-tuning on in-domain monolingual and parallel data gives\nsubstantial improvements for the IWSLT 15 task English->German.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.07788v1",
            "date": "2015-11-24 16:28:16+00:00",
            "title": "Spoken Language Translation for Polish",
            "authors": [
                "Krzysztof Marasek",
                "\u0141ukasz Brocki",
                "Danijel Korzinek",
                "Krzysztof Wo\u0142k",
                "Ryszard Gubrynowicz"
            ],
            "abstract": "Spoken language translation (SLT) is becoming more important in the\nincreasingly globalized world, both from a social and economic point of view.\nIt is one of the major challenges for automatic speech recognition (ASR) and\nmachine translation (MT), driving intense research activities in these areas.\nWhile past research in SLT, due to technology limitations, dealt mostly with\nspeech recorded under controlled conditions, today's major challenge is the\ntranslation of spoken language as it can be found in real life. Considered\napplication scenarios range from portable translators for tourists, lectures\nand presentations translation, to broadcast news and shows with live\ncaptioning. We would like to present PJIIT's experiences in the SLT gained from\nthe Eu-Bridge 7th framework project and the U-Star consortium activities for\nthe Polish/English language pair. Presented research concentrates on ASR\nadaptation for Polish (state-of-the-art acoustic models: DBN-BLSTM training,\nKaldi: LDA+MLLT+SAT+MMI), language modeling for ASR & MT (text normalization,\nRNN-based LMs, n-gram model domain interpolation) and statistical translation\ntechniques (hierarchical models, factored translation models, automatic casing\nand punctuation, comparable and bilingual corpora preparation). While results\nfor the well-defined domains (phrases for travelers, parliament speeches,\nmedical documentation, movie subtitling) are very encouraging, less defined\ndomains (presentation, lectures) still form a challenge. Our progress in the\nIWSLT TED task (MT only) will be presented, as well as current progress in the\nPolish ASR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08411v1",
            "date": "2015-11-26 15:10:18+00:00",
            "title": "OntoSeg: a Novel Approach to Text Segmentation using Ontological Similarity",
            "authors": [
                "Mostafa Bayomi",
                "Killian Levacher",
                "M. Rami Ghorab",
                "S\u00e9amus Lawless"
            ],
            "abstract": "Text segmentation (TS) aims at dividing long text into coherent segments\nwhich reflect the subtopic structure of the text. It is beneficial to many\nnatural language processing tasks, such as Information Retrieval (IR) and\ndocument summarisation. Current approaches to text segmentation are similar in\nthat they all use word-frequency metrics to measure the similarity between two\nregions of text, so that a document is segmented based on the lexical cohesion\nbetween its words. Various NLP tasks are now moving towards the semantic web\nand ontologies, such as ontology-based IR systems, to capture the\nconceptualizations associated with user needs and contents. Text segmentation\nbased on lexical cohesion between words is hence not sufficient anymore for\nsuch tasks. This paper proposes OntoSeg, a novel approach to text segmentation\nbased on the ontological similarity between text blocks. The proposed method\nuses ontological similarity to explore conceptual relations between text\nsegments and a Hierarchical Agglomerative Clustering (HAC) algorithm to\nrepresent the text as a tree-like hierarchy that is conceptually structured.\nThe rich structure of the created tree further allows the segmentation of text\nin a linear fashion at various levels of granularity. The proposed method was\nevaluated on a wellknown dataset, and the results show that using ontological\nsimilarity in text segmentation is very promising. Also we enhance the proposed\nmethod by combining ontological similarity with lexical similarity and the\nresults show an enhancement of the segmentation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08629v2",
            "date": "2015-11-27 11:38:57+00:00",
            "title": "Category Enhanced Word Embedding",
            "authors": [
                "Chunting Zhou",
                "Chonglin Sun",
                "Zhiyuan Liu",
                "Francis C. M. Lau"
            ],
            "abstract": "Distributed word representations have been demonstrated to be effective in\ncapturing semantic and syntactic regularities. Unsupervised representation\nlearning from large unlabeled corpora can learn similar representations for\nthose words that present similar co-occurrence statistics. Besides local\noccurrence statistics, global topical information is also important knowledge\nthat may help discriminate a word from another. In this paper, we incorporate\ncategory information of documents in the learning of word representations and\nto learn the proposed models in a document-wise manner. Our models outperform\nseveral state-of-the-art models in word analogy and word similarity tasks.\nMoreover, we evaluate the learned word vectors on sentiment analysis and text\nclassification tasks, which shows the superiority of our learned word vectors.\nWe also learn high-quality category embeddings that reflect topical meanings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08630v2",
            "date": "2015-11-27 11:44:17+00:00",
            "title": "A C-LSTM Neural Network for Text Classification",
            "authors": [
                "Chunting Zhou",
                "Chonglin Sun",
                "Zhiyuan Liu",
                "Francis C. M. Lau"
            ],
            "abstract": "Neural network models have been demonstrated to be capable of achieving\nremarkable performance in sentence and document modeling. Convolutional neural\nnetwork (CNN) and recurrent neural network (RNN) are two mainstream\narchitectures for such modeling tasks, which adopt totally different ways of\nunderstanding natural languages. In this work, we combine the strengths of both\narchitectures and propose a novel and unified model called C-LSTM for sentence\nrepresentation and text classification. C-LSTM utilizes CNN to extract a\nsequence of higher-level phrase representations, and are fed into a long\nshort-term memory recurrent neural network (LSTM) to obtain the sentence\nrepresentation. C-LSTM is able to capture both local features of phrases as\nwell as global and temporal sentence semantics. We evaluate the proposed\narchitecture on sentiment classification and question classification tasks. The\nexperimental results show that the C-LSTM outperforms both CNN and LSTM and can\nachieve excellent performance on these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06601v1",
            "date": "2015-11-19 11:29:10+00:00",
            "title": "An Approach to Speed-up the Word Sense Disambiguation Procedure through Sense Filtering",
            "authors": [
                "Alok Ranjan Pal",
                "Anupam Munshi",
                "Diganta Saha"
            ],
            "abstract": "In this paper, we are going to focus on speed up of the Word Sense\nDisambiguation procedure by filtering the relevant senses of an ambiguous word\nthrough Part-of-Speech Tagging. First, this proposed approach performs the\nPart-of-Speech Tagging operation before the disambiguation procedure using\nBigram approximation. As a result, the exact Part-of-Speech of the ambiguous\nword at a particular text instance is derived. In the next stage, only those\ndictionary definitions (glosses) are retrieved from an online dictionary, which\nare associated with that particular Part-of-Speech to disambiguate the exact\nsense of the ambiguous word. In the training phase, we have used Brown Corpus\nfor Part-of-Speech Tagging and WordNet as an online dictionary. The proposed\napproach reduces the execution time upto half (approximately) of the normal\nexecution time for a text, containing around 200 sentences. Not only that, we\nhave found several instances, where the correct sense of an ambiguous word is\nfound for using the Part-of-Speech Tagging before the Disambiguation procedure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01245v1",
            "date": "2015-02-04 16:12:45+00:00",
            "title": "Authorship recognition via fluctuation analysis of network topology and word intermittency",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "Statistical methods have been widely employed in many practical natural\nlanguage processing applications. More specifically, complex networks concepts\nand methods from dynamical systems theory have been successfully applied to\nrecognize stylistic patterns in written texts. Despite the large amount of\nstudies devoted to represent texts with physical models, only a few studies\nhave assessed the relevance of attributes derived from the analysis of\nstylistic fluctuations. Because fluctuations represent a pivotal factor for\ncharacterizing a myriad of real systems, this study focused on the analysis of\nthe properties of stylistic fluctuations in texts via topological analysis of\ncomplex networks and intermittency measurements. The results showed that\ndifferent authors display distinct fluctuation patterns. In particular, it was\nfound that it is possible to identify the authorship of books using the\nintermittency of specific words. Taken together, the results described here\nsuggest that the patterns found in stylistic fluctuations could be used to\nanalyze other related complex systems. Furthermore, the discovery of novel\npatterns related to textual stylistic fluctuations indicates that these\npatterns could be useful to improve the state of the art of many\nstylistic-based natural language processing tasks.",
            "journal ref": "J. Stat. Mech. (2015) P03005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01271v2",
            "date": "2015-02-04 17:53:01+00:00",
            "title": "INRIASAC: Simple Hypernym Extraction Methods",
            "authors": [
                "Gregory Grefenstette"
            ],
            "abstract": "Given a set of terms from a given domain, how can we structure them into a\ntaxonomy without manual intervention? This is the task 17 of SemEval 2015. Here\nwe present our simple taxonomy structuring techniques which, despite their\nsimplicity, ranked first in this 2015 benchmark. We use large quantities of\ntext (English Wikipedia) and simple heuristics such as term overlap and\ndocument and sentence co-occurrence to produce hypernym lists. We describe\nthese techniques and pre-sent an initial evaluation of results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01446v1",
            "date": "2015-02-05 07:42:18+00:00",
            "title": "Beyond Word-based Language Model in Statistical Machine Translation",
            "authors": [
                "Jiajun Zhang",
                "Shujie Liu",
                "Mu Li",
                "Ming Zhou",
                "Chengqing Zong"
            ],
            "abstract": "Language model is one of the most important modules in statistical machine\ntranslation and currently the word-based language model dominants this\ncommunity. However, many translation models (e.g. phrase-based models) generate\nthe target language sentences by rendering and compositing the phrases rather\nthan the words. Thus, it is much more reasonable to model dependency between\nphrases, but few research work succeed in solving this problem. In this paper,\nwe tackle this problem by designing a novel phrase-based language model which\nattempts to solve three key sub-problems: 1, how to define a phrase in language\nmodel; 2, how to determine the phrase boundary in the large-scale monolingual\ndata in order to enlarge the training set; 3, how to alleviate the data\nsparsity problem due to the huge vocabulary size of phrases. By carefully\nhandling these issues, the extensive experiments on Chinese-to-English\ntranslation show that our phrase-based language model can significantly improve\nthe translation quality by up to +1.47 absolute BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.02655v1",
            "date": "2015-02-07 21:59:21+00:00",
            "title": "An investigation into language complexity of World-of-Warcraft game-external texts",
            "authors": [
                "Simon \u0160uster"
            ],
            "abstract": "We present a language complexity analysis of World of Warcraft (WoW)\ncommunity texts, which we compare to texts from a general corpus of web\nEnglish. Results from several complexity types are presented, including lexical\ndiversity, density, readability and syntactic complexity. The language of WoW\ntexts is found to be comparable to the general corpus on some complexity\nmeasures, yet more specialized on other measures. Our findings can be used by\neducators willing to include game-related activities into school curricula.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.03671v2",
            "date": "2015-02-12 14:17:15+00:00",
            "title": "Phrase-based Image Captioning",
            "authors": [
                "R\u00e9mi Lebret",
                "Pedro O. Pinheiro",
                "Ronan Collobert"
            ],
            "abstract": "Generating a novel textual description of an image is an interesting problem\nthat connects computer vision and natural language processing. In this paper,\nwe present a simple model that is able to generate descriptive sentences given\na sample image. This model has a strong focus on the syntax of the\ndescriptions. We train a purely bilinear model that learns a metric between an\nimage representation (generated from a previously trained Convolutional Neural\nNetwork) and phrases that are used to described them. The system is then able\nto infer phrases from a given image sample. Based on caption syntax statistics,\nwe propose a simple language model that can produce relevant descriptions for a\ngiven test image using the phrases inferred. Our approach, which is\nconsiderably simpler than state-of-the-art models, achieves comparable results\nin two popular datasets for the task: Flickr30k and the recently proposed\nMicrosoft COCO.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.03752v1",
            "date": "2015-02-12 17:49:45+00:00",
            "title": "A new hybrid metric for verifying parallel corpora of Arabic-English",
            "authors": [
                "Saad Alkahtani",
                "Wei Liu",
                "William J. Teahan"
            ],
            "abstract": "This paper discusses a new metric that has been applied to verify the quality\nin translation between sentence pairs in parallel corpora of Arabic-English.\nThis metric combines two techniques, one based on sentence length and the other\nbased on compression code length. Experiments on sample test parallel\nArabic-English corpora indicate the combination of these two techniques\nimproves accuracy of the identification of satisfactory and unsatisfactory\nsentence pairs compared to sentence length and compression code length alone.\nThe new method proposed in this research is effective at filtering noise and\nreducing mis-translations resulting in greatly improved quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.04174v1",
            "date": "2015-02-14 06:47:34+00:00",
            "title": "Probabilistic Models for High-Order Projective Dependency Parsing",
            "authors": [
                "Xuezhe Ma",
                "Hai Zhao"
            ],
            "abstract": "This paper presents generalized probabilistic models for high-order\nprojective dependency parsing and an algorithmic framework for learning these\nstatistical models involving dependency trees. Partition functions and\nmarginals for high-order dependency trees can be computed efficiently, by\nadapting our algorithms which extend the inside-outside algorithm to\nhigher-order cases. To show the effectiveness of our algorithms, we perform\nexperiments on three languages---English, Chinese and Czech, using maximum\nconditional likelihood estimation for model training and L-BFGS for parameter\nestimation. Our methods achieve competitive performance for English, and\noutperform all previously reported dependency parsers for Chinese and Czech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.04938v2",
            "date": "2015-02-17 15:59:09+00:00",
            "title": "A Survey of Word Reordering in Statistical Machine Translation: Computational Models and Language Phenomena",
            "authors": [
                "Arianna Bisazza",
                "Marcello Federico"
            ],
            "abstract": "Word reordering is one of the most difficult aspects of statistical machine\ntranslation (SMT), and an important factor of its quality and efficiency.\nDespite the vast amount of research published to date, the interest of the\ncommunity in this problem has not decreased, and no single method appears to be\nstrongly dominant across language pairs. Instead, the choice of the optimal\napproach for a new translation task still seems to be mostly driven by\nempirical trials. To orientate the reader in this vast and complex research\narea, we present a comprehensive survey of word reordering viewed as a\nstatistical modeling challenge and as a natural language phenomenon. The survey\ndescribes in detail how word reordering is modeled within different\nstring-based and tree-based SMT frameworks and as a stand-alone task, including\nsystematic overviews of the literature in advanced reordering modeling. We then\nquestion why some approaches are more successful than others in different\nlanguage pairs. We argue that, besides measuring the amount of reordering, it\nis important to understand which kinds of reordering occur in a given language\npair. To this end, we conduct a qualitative analysis of word reordering\nphenomena in a diverse sample of language pairs, based on a large collection of\nlinguistic knowledge. Empirical results in the SMT literature are shown to\nsupport the hypothesis that a few linguistic facts can be very useful to\nanticipate the reordering characteristics of a language pair and to select the\nSMT framework that best suits them.",
            "journal ref": "Computational Linguistics, Vol. 42, No. 2: 163-205, MIT Press\n  (June 2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.07038v1",
            "date": "2015-02-25 03:27:38+00:00",
            "title": "Web-scale Surface and Syntactic n-gram Features for Dependency Parsing",
            "authors": [
                "Dominick Ng",
                "Mohit Bansal",
                "James R. Curran"
            ],
            "abstract": "We develop novel first- and second-order features for dependency parsing\nbased on the Google Syntactic Ngrams corpus, a collection of subtree counts of\nparsed sentences from scanned books. We also extend previous work on surface\n$n$-gram features from Web1T to the Google Books corpus and from first-order to\nsecond-order, comparing and analysing performance over newswire and web\ntreebanks.\n  Surface and syntactic $n$-grams both produce substantial and complementary\ngains in parsing accuracy across domains. Our best system combines the two\nfeature sets, achieving up to 0.8% absolute UAS improvements on newswire and\n1.4% on web text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.07257v2",
            "date": "2015-02-25 17:15:56+00:00",
            "title": "Breaking Sticks and Ambiguities with Adaptive Skip-gram",
            "authors": [
                "Sergey Bartunov",
                "Dmitry Kondrashkin",
                "Anton Osokin",
                "Dmitry Vetrov"
            ],
            "abstract": "Recently proposed Skip-gram model is a powerful method for learning\nhigh-dimensional word representations that capture rich semantic relationships\nbetween words. However, Skip-gram as well as most prior work on learning word\nrepresentations does not take into account word ambiguity and maintain only\nsingle representation per word. Although a number of Skip-gram modifications\nwere proposed to overcome this limitation and learn multi-prototype word\nrepresentations, they either require a known number of word meanings or learn\nthem using greedy heuristic approaches. In this paper we propose the Adaptive\nSkip-gram model which is a nonparametric Bayesian extension of Skip-gram\ncapable to automatically learn the required number of representations for all\nwords at desired semantic resolution. We derive efficient online variational\nlearning algorithm for the model and empirically demonstrate its efficiency on\nword-sense induction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.07504v1",
            "date": "2015-02-26 11:09:59+00:00",
            "title": "Rational Kernels for Arabic Stemming and Text Classification",
            "authors": [
                "Attia Nehar",
                "Djelloul Ziadi",
                "Hadda Cherroun"
            ],
            "abstract": "In this paper, we address the problems of Arabic Text Classification and\nstemming using Transducers and Rational Kernels. We introduce a new stemming\ntechnique based on the use of Arabic patterns (Pattern Based Stemmer). Patterns\nare modelled using transducers and stemming is done without depending on any\ndictionary. Using transducers for stemming, documents are transformed into\nfinite state transducers. This document representation allows us to use and\nexplore rational kernels as a framework for Arabic Text Classification.\nStemming experiments are conducted on three word collections and classification\nexperiments are done on the Saudi Press Agency dataset. Results show that our\napproach, when compared with other approaches, is promising specially in terms\nof Accuracy, Recall and F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.07920v1",
            "date": "2015-02-27 14:55:15+00:00",
            "title": "Local Translation Prediction with Global Sentence Representation",
            "authors": [
                "Jiajun Zhang"
            ],
            "abstract": "Statistical machine translation models have made great progress in improving\nthe translation quality. However, the existing models predict the target\ntranslation with only the source- and target-side local context information. In\npractice, distinguishing good translations from bad ones does not only depend\non the local features, but also rely on the global sentence-level information.\nIn this paper, we explore the source-side global sentence-level features for\ntarget-side local translation prediction. We propose a novel\nbilingually-constrained chunk-based convolutional neural network to learn\nsentence semantic representations. With the sentence-level feature\nrepresentation, we further design a feed-forward neural network to better\npredict translations using both local and global information. The large-scale\nexperiments show that our method can obtain substantial improvements in\ntranslation quality over the strong baseline: the hierarchical phrase-based\ntranslation model augmented with the neural network joint model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.00311v1",
            "date": "2015-01-01 20:51:25+00:00",
            "title": "QANUS: An Open-source Question-Answering Platform",
            "authors": [
                "Jun-Ping Ng",
                "Min-Yen Kan"
            ],
            "abstract": "In this paper, we motivate the need for a publicly available, generic\nsoftware framework for question-answering (QA) systems. We present an\nopen-source QA framework QANUS which researchers can leverage on to build new\nQA systems easily and rapidly. The framework implements much of the code that\nwill otherwise have been repeated across different QA systems. To demonstrate\nthe utility and practicality of the framework, we further present a fully\nfunctioning factoid QA system QA-SYS built on top of QANUS.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1501.01252v1",
            "date": "2015-01-06 17:58:43+00:00",
            "title": "Optimisation using Natural Language Processing: Personalized Tour Recommendation for Museums",
            "authors": [
                "Mayeul Mathias",
                "Assema Moussa",
                "Fen Zhou",
                "Juan-Manuel Torres-Moreno",
                "Marie-Sylvie Poli",
                "Didier Josselin",
                "Marc El-B\u00e8ze",
                "Andr\u00e9a Carneiro Linhares",
                "Francoise Rigat"
            ],
            "abstract": "This paper proposes a new method to provide personalized tour recommendation\nfor museum visits. It combines an optimization of preference criteria of\nvisitors with an automatic extraction of artwork importance from museum\ninformation based on Natural Language Processing using textual energy. This\nproject includes researchers from computer and social sciences. Some results\nare obtained with numerical experiments. They show that our model clearly\nimproves the satisfaction of the visitor who follows the proposed tour. This\nwork foreshadows some interesting outcomes and applications about on-demand\npersonalized visit of museums in a very near future.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1501.01318v1",
            "date": "2015-01-06 21:10:26+00:00",
            "title": "Arabic Text Categorization Algorithm using Vector Evaluation Method",
            "authors": [
                "Ashraf Odeh",
                "Aymen Abu-Errub",
                "Qusai Shambour",
                "Nidal Turab"
            ],
            "abstract": "Text categorization is the process of grouping documents into categories\nbased on their contents. This process is important to make information\nretrieval easier, and it became more important due to the huge textual\ninformation available online. The main problem in text categorization is how to\nimprove the classification accuracy. Although Arabic text categorization is a\nnew promising field, there are a few researches in this field. This paper\nproposes a new method for Arabic text categorization using vector evaluation.\nThe proposed method uses a categorized Arabic documents corpus, and then the\nweights of the tested document's words are calculated to determine the document\nkeywords which will be compared with the keywords of the corpus categorizes to\ndetermine the tested document's best category.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 6, No 6, December 2014",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1501.01386v1",
            "date": "2015-01-07 08:04:32+00:00",
            "title": "Roman Urdu Opinion Mining System (RUOMiS)",
            "authors": [
                "Misbah Daud",
                "Rafiullah Khan",
                "Mohibullah",
                "Aitazaz Daud"
            ],
            "abstract": "Convincing a customer is always considered as a challenging task in every\nbusiness. But when it comes to online business, this task becomes even more\ndifficult. Online retailers try everything possible to gain the trust of the\ncustomer. One of the solutions is to provide an area for existing users to\nleave their comments. This service can effectively develop the trust of the\ncustomer however normally the customer comments about the product in their\nnative language using Roman script. If there are hundreds of comments this\nmakes difficulty even for the native customers to make a buying decision. This\nresearch proposes a system which extracts the comments posted in Roman Urdu,\ntranslate them, find their polarity and then gives us the rating of the\nproduct. This rating will help the native and non-native customers to make\nbuying decision efficiently from the comments posted in Roman Urdu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.01866v1",
            "date": "2015-01-08 14:21:39+00:00",
            "title": "The Hebrew Bible as Data: Laboratory - Sharing - Experiences",
            "authors": [
                "Dirk Roorda"
            ],
            "abstract": "The systematic study of ancient texts including their production,\ntransmission and interpretation is greatly aided by the digital methods that\nstarted taking off in the 1970s. But how is that research in turn transmitted\nto new generations of researchers? We tell a story of Bible and computer across\nthe decades and then point out the current challenges: (1) finding a stable\ndata representation for changing methods of computation; (2) sharing results in\ninter- and intra-disciplinary ways, for reproducibility and\ncross-fertilization. We report recent developments in meeting these challenges.\nThe scene is the text database of the Hebrew Bible, constructed by the Eep\nTalstra Centre for Bible and Computer (ETCBC), which is still growing in detail\nand sophistication. We show how a subtle mix of computational ingredients\nenable scholars to research the transmission and interpretation of the Hebrew\nBible in new ways: (1) a standard data format, Linguistic Annotation Framework\n(LAF); (2) the methods of scientific computing, made accessible by\n(interactive) Python and its associated ecosystem. Additionally, we show how\nthese efforts have culminated in the construction of a new, publicly accessible\nsearch engine SHEBANQ, where the text of the Hebrew Bible and its underlying\ndata can be queried in a simple, yet powerful query language MQL, and where\nthose queries can be saved and shared.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.02714v2",
            "date": "2015-01-12 16:48:19+00:00",
            "title": "From Visual Attributes to Adjectives through Decompositional Distributional Semantics",
            "authors": [
                "Angeliki Lazaridou",
                "Georgiana Dinu",
                "Adam Liska",
                "Marco Baroni"
            ],
            "abstract": "As automated image analysis progresses, there is increasing interest in\nricher linguistic annotation of pictures, with attributes of objects (e.g.,\nfurry, brown...) attracting most attention. By building on the recent\n\"zero-shot learning\" approach, and paying attention to the linguistic nature of\nattributes as noun modifiers, and specifically adjectives, we show that it is\npossible to tag images with attribute-denoting adjectives even when no training\ndata containing the relevant annotation are available. Our approach relies on\ntwo key observations. First, objects can be seen as bundles of attributes,\ntypically expressed as adjectival modifiers (a dog is something furry, brown,\netc.), and thus a function trained to map visual representations of objects to\nnominal labels can implicitly learn to map attributes to adjectives. Second,\nobjects and attributes come together in pictures (the same thing is a dog and\nit is brown). We can thus achieve better attribute (and object) label retrieval\nby treating images as \"visual phrases\", and decomposing their linguistic\nrepresentation into an attribute-denoting adjective and an object-denoting\nnoun. Our approach performs comparably to a method exploiting manual attribute\nannotation, it outperforms various competitive alternatives in both attribute\nand object annotation, and it automatically constructs attribute-centric\nrepresentations that significantly improve performance in supervised object\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.03210v1",
            "date": "2015-01-13 23:51:29+00:00",
            "title": "Towards Deep Semantic Analysis Of Hashtags",
            "authors": [
                "Piyush Bansal",
                "Romil Bansal",
                "Vasudeva Varma"
            ],
            "abstract": "Hashtags are semantico-syntactic constructs used across various social\nnetworking and microblogging platforms to enable users to start a topic\nspecific discussion or classify a post into a desired category. Segmenting and\nlinking the entities present within the hashtags could therefore help in better\nunderstanding and extraction of information shared across the social media.\nHowever, due to lack of space delimiters in the hashtags (e.g #nsavssnowden),\nthe segmentation of hashtags into constituent entities (\"NSA\" and \"Edward\nSnowden\" in this case) is not a trivial task. Most of the current\nstate-of-the-art social media analytics systems like Sentiment Analysis and\nEntity Linking tend to either ignore hashtags, or treat them as a single word.\nIn this paper, we present a context aware approach to segment and link entities\nin the hashtags to a knowledge base (KB) entry, based on the context within the\ntweet. Our approach segments and links the entities in hashtags such that the\ncoherence between hashtag semantics and the tweet is maximized. To the best of\nour knowledge, no existing study addresses the issue of linking entities in\nhashtags for extracting semantic information. We evaluate our method on two\ndifferent datasets, and demonstrate the effectiveness of our technique in\nimproving the overall entity linking in tweets via additional semantic\ninformation provided by segmenting and linking entities in a hashtag.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1501.03214v1",
            "date": "2015-01-14 00:06:15+00:00",
            "title": "Quantifying Prosodic Variability in Middle English Alliterative Poetry",
            "authors": [
                "Roger Bilisoly"
            ],
            "abstract": "Interest in the mathematical structure of poetry dates back to at least the\n19th century: after retiring from his mathematics position, J. J. Sylvester\nwrote a book on prosody called $\\textit{The Laws of Verse}$. Today there is\ninterest in the computer analysis of poems, and this paper discusses how a\nstatistical approach can be applied to this task. Starting with the definition\nof what Middle English alliteration is, $\\textit{Sir Gawain and the Green\nKnight}$ and William Langland's $\\textit{Piers Plowman}$ are used to illustrate\nthe methodology. Theory first developed for analyzing data from a Riemannian\nmanifold turns out to be applicable to strings allowing one to compute a\ngeneralized mean and variance for textual data, which is applied to the poems\nabove. The ratio of these two variances produces the analogue of the F test,\nand resampling allows p-values to be estimated. Consequently, this methodology\nprovides a way to compare prosodic variability between two texts.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1501.04920v1",
            "date": "2015-01-20 19:01:20+00:00",
            "title": "Regroupement s\u00e9mantique de d\u00e9finitions en espagnol",
            "authors": [
                "Gerardo Sierra",
                "Juan-Manuel Torres-Moreno",
                "Alejandro Molina"
            ],
            "abstract": "This article focuses on the description and evaluation of a new unsupervised\nlearning method of clustering of definitions in Spanish according to their\nsemantic. Textual Energy was used as a clustering measure, and we study an\nadaptation of the Precision and Recall to evaluate our method.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1501.05396v1",
            "date": "2015-01-22 05:25:33+00:00",
            "title": "Deep Multimodal Learning for Audio-Visual Speech Recognition",
            "authors": [
                "Youssef Mroueh",
                "Etienne Marcheret",
                "Vaibhava Goel"
            ],
            "abstract": "In this paper, we present methods in deep multimodal learning for fusing\nspeech and visual modalities for Audio-Visual Automatic Speech Recognition\n(AV-ASR). First, we study an approach where uni-modal deep networks are trained\nseparately and their final hidden layers fused to obtain a joint feature space\nin which another deep network is built. While the audio network alone achieves\na phone error rate (PER) of $41\\%$ under clean condition on the IBM large\nvocabulary audio-visual studio dataset, this fusion model achieves a PER of\n$35.83\\%$ demonstrating the tremendous value of the visual channel in phone\nclassification even in audio with high signal to noise ratio. Second, we\npresent a new deep network architecture that uses a bilinear softmax layer to\naccount for class specific correlations between modalities. We show that\ncombining the posteriors from the bilinear networks with those from the fused\nmodel mentioned above results in a further significant phone error rate\nreduction, yielding a final PER of $34.03\\%$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.07676v1",
            "date": "2015-01-30 06:51:42+00:00",
            "title": "Towards Resolving Software Quality-in-Use Measurement Challenges",
            "authors": [
                "Issa Atoum",
                "Chih How Bong",
                "Narayanan Kulathuramaiyer"
            ],
            "abstract": "Software quality-in-use comprehends the quality from user's perspectives. It\nhas gained its importance in e-learning applications, mobile service based\napplications and project management tools. User's decisions on software\nacquisitions are often ad hoc or based on preference due to difficulty in\nquantitatively measure software quality-in-use. However, why quality-in-use\nmeasurement is difficult? Although there are many software quality models to\nour knowledge, no works surveys the challenges related to software\nquality-in-use measurement. This paper has two main contributions; 1) presents\nmajor issues and challenges in measuring software quality-in-use in the context\nof the ISO SQuaRE series and related software quality models, 2) Presents a\nnovel framework that can be used to predict software quality-in-use, and 3)\npresents preliminary results of quality-in-use topic prediction. Concisely, the\nissues are related to the complexity of the current standard models and the\nlimitations and incompleteness of the customized software quality models. The\nproposed framework employs sentiment analysis techniques to predict software\nquality-in-use.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1503.00064v1",
            "date": "2015-02-28 04:26:21+00:00",
            "title": "Generating Multi-Sentence Lingual Descriptions of Indoor Scenes",
            "authors": [
                "Dahua Lin",
                "Chen Kong",
                "Sanja Fidler",
                "Raquel Urtasun"
            ],
            "abstract": "This paper proposes a novel framework for generating lingual descriptions of\nindoor scenes. Whereas substantial efforts have been made to tackle this\nproblem, previous approaches focusing primarily on generating a single sentence\nfor each image, which is not sufficient for describing complex scenes. We\nattempt to go beyond this, by generating coherent descriptions with multiple\nsentences. Our approach is distinguished from conventional ones in several\naspects: (1) a 3D visual parsing system that jointly infers objects,\nattributes, and relations; (2) a generative grammar learned automatically from\ntraining text; and (3) a text generation algorithm that takes into account the\ncoherence among sentences. Experiments on the augmented NYU-v2 dataset show\nthat our framework can generate natural descriptions with substantially higher\nROGUE scores compared to those produced by the baseline.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1503.00107v1",
            "date": "2015-02-28 09:53:32+00:00",
            "title": "Non-linear Learning for Statistical Machine Translation",
            "authors": [
                "Shujian Huang",
                "Huadong Chen",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Modern statistical machine translation (SMT) systems usually use a linear\ncombination of features to model the quality of each translation hypothesis.\nThe linear combination assumes that all the features are in a linear\nrelationship and constrains that each feature interacts with the rest features\nin an linear manner, which might limit the expressive power of the model and\nlead to a under-fit model on the current data. In this paper, we propose a\nnon-linear modeling for the quality of translation hypotheses based on neural\nnetworks, which allows more complex interaction between features. A learning\nframework is presented for training the non-linear models. We also discuss\npossible heuristics in designing the network structure which may improve the\nnon-linear learning performance. Experimental results show that with the basic\nfeatures of a hierarchical phrase-based machine translation system, our method\nproduce translations that are better than a linear model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00185v5",
            "date": "2015-02-28 21:39:31+00:00",
            "title": "When Are Tree Structures Necessary for Deep Learning of Representations?",
            "authors": [
                "Jiwei Li",
                "Minh-Thang Luong",
                "Dan Jurafsky",
                "Eudard Hovy"
            ],
            "abstract": "Recursive neural models, which use syntactic parse trees to recursively\ngenerate representations bottom-up, are a popular architecture. But there have\nnot been rigorous evaluations showing for exactly which tasks this syntax-based\nmethod is appropriate. In this paper we benchmark {\\bf recursive} neural models\nagainst sequential {\\bf recurrent} neural models (simple recurrent and LSTM\nmodels), enforcing apples-to-apples comparison as much as possible. We\ninvestigate 4 tasks: (1) sentiment classification at the sentence level and\nphrase level; (2) matching questions to answer-phrases; (3) discourse parsing;\n(4) semantic relation extraction (e.g., {\\em component-whole} between nouns).\n  Our goal is to understand better when, and why, recursive models can\noutperform simpler models. We find that recursive models help mainly on tasks\n(like semantic relation extraction) that require associating headwords across a\nlong distance, particularly on very long sequences. We then introduce a method\nfor allowing recurrent models to achieve similar performance: breaking long\nsentences into clause-like units at punctuation and processing them separately\nbefore combining. Our results thus help understand the limitations of both\nclasses of models, and suggest directions for improving recurrent models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1503.01258v1",
            "date": "2015-03-04 09:27:13+00:00",
            "title": "The concept \"altruism\" for sociological research: from conceptualization to operationalization",
            "authors": [
                "Oleg V. Pavenkov",
                "Vladimir G. Pavenkov",
                "Mariia V. Rubtcova"
            ],
            "abstract": "This article addresses the question of the relevant conceptualization of\n{\\guillemotleft}altruism{\\guillemotright} in Russian from the perspective\nsociological research operationalization. It investigates the spheres of social\napplication of the word {\\guillemotleft}altruism{\\guillemotright}, include\nRussian equivalent {\\guillemotleft}vzaimopomoshh`{\\guillemotright} (mutual\nhelp). The data for the study comes from Russian National Corpus (Russian). The\ntheoretical framework consists of Paul F. Lazarsfeld`s Theory of Sociological\nResearch Methodology and the Natural Semantic Metalanguage (NSM). Quantitative\nanalysis shows features in the representation of altruism in Russian that\nsociologists need to know in the preparation of questionnaires, interview\nguides and analysis of transcripts.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1503.01549v1",
            "date": "2015-03-05 06:22:15+00:00",
            "title": "Visualization of Clandestine Labs from Seizure Reports: Thematic Mapping and Data Mining Research Directions",
            "authors": [
                "William Hsu",
                "Mohammed Abduljabbar",
                "Ryuichi Osuga",
                "Max Lu",
                "Wesam Elshamy"
            ],
            "abstract": "The problem of spatiotemporal event visualization based on reports entails\nsubtasks ranging from named entity recognition to relationship extraction and\nmapping of events. We present an approach to event extraction that is driven by\ndata mining and visualization goals, particularly thematic mapping and trend\nanalysis. This paper focuses on bridging the information extraction and\nvisualization tasks and investigates topic modeling approaches. We develop a\nstatic, finite topic model and examine the potential benefits and feasibility\nof extending this to dynamic topic modeling with a large number of topics and\ncontinuous time. We describe an experimental test bed for event mapping that\nuses this end-to-end information retrieval system, and report preliminary\nresults on a geoinformatics problem: tracking of methamphetamine lab seizure\nevents across time and space.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1503.02417v1",
            "date": "2015-03-09 10:35:10+00:00",
            "title": "Structured Prediction of Sequences and Trees using Infinite Contexts",
            "authors": [
                "Ehsan Shareghi",
                "Gholamreza Haffari",
                "Trevor Cohn",
                "Ann Nicholson"
            ],
            "abstract": "Linguistic structures exhibit a rich array of global phenomena, however\ncommonly used Markov models are unable to adequately describe these phenomena\ndue to their strong locality assumptions. We propose a novel hierarchical model\nfor structured prediction over sequences and trees which exploits global\ncontext by conditioning each generation decision on an unbounded context of\nprior decisions. This builds on the success of Markov models but without\nimposing a fixed bound in order to better represent global phenomena. To\nfacilitate learning of this large and unbounded model, we use a hierarchical\nPitman-Yor process prior which provides a recursive form of smoothing. We\npropose prediction algorithms based on A* and Markov Chain Monte Carlo\nsampling. Empirical results demonstrate the potential of our model compared to\nbaseline finite-context Markov models on part-of-speech tagging and syntactic\nparsing.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1503.02801v1",
            "date": "2015-03-10 07:51:59+00:00",
            "title": "Short Text Hashing Improved by Integrating Multi-Granularity Topics and Tags",
            "authors": [
                "Jiaming Xu",
                "Bo Xu",
                "Guanhua Tian",
                "Jun Zhao",
                "Fangyuan Wang",
                "Hongwei Hao"
            ],
            "abstract": "Due to computational and storage efficiencies of compact binary codes,\nhashing has been widely used for large-scale similarity search. Unfortunately,\nmany existing hashing methods based on observed keyword features are not\neffective for short texts due to the sparseness and shortness. Recently, some\nresearchers try to utilize latent topics of certain granularity to preserve\nsemantic similarity in hash codes beyond keyword matching. However, topics of\ncertain granularity are not adequate to represent the intrinsic semantic\ninformation. In this paper, we present a novel unified approach for short text\nHashing using Multi-granularity Topics and Tags, dubbed HMTT. In particular, we\npropose a selection method to choose the optimal multi-granularity topics\ndepending on the type of dataset, and design two distinct hashing strategies to\nincorporate multi-granularity topics. We also propose a simple and effective\nmethod to exploit tags to enhance the similarity of related texts. We carry out\nextensive experiments on one short text dataset as well as on one normal text\ndataset. The results demonstrate that our approach is effective and\nsignificantly outperforms baselines on several evaluation metrics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1503.04250v1",
            "date": "2015-03-13 23:36:42+00:00",
            "title": "The YLI-MED Corpus: Characteristics, Procedures, and Plans",
            "authors": [
                "Julia Bernd",
                "Damian Borth",
                "Benjamin Elizalde",
                "Gerald Friedland",
                "Heather Gallagher",
                "Luke Gottlieb",
                "Adam Janin",
                "Sara Karabashlieva",
                "Jocelyn Takahashi",
                "Jennifer Won"
            ],
            "abstract": "The YLI Multimedia Event Detection corpus is a public-domain index of videos\nwith annotations and computed features, specialized for research in multimedia\nevent detection (MED), i.e., automatically identifying what's happening in a\nvideo by analyzing the audio and visual content. The videos indexed in the\nYLI-MED corpus are a subset of the larger YLI feature corpus, which is being\ndeveloped by the International Computer Science Institute and Lawrence\nLivermore National Laboratory based on the Yahoo Flickr Creative Commons 100\nMillion (YFCC100M) dataset. The videos in YLI-MED are categorized as depicting\none of ten target events, or no target event, and are annotated for additional\nattributes like language spoken and whether the video has a musical score. The\nannotations also include degree of annotator agreement and average annotator\nconfidence scores for the event categorization of each video. Version 1.0 of\nYLI-MED includes 1823 \"positive\" videos that depict the target events and\n48,138 \"negative\" videos, as well as 177 supplementary videos that are similar\nto event videos but are not positive examples. Our goal in producing YLI-MED is\nto be as open about our data and procedures as possible. This report describes\nthe procedures used to collect the corpus; gives detailed descriptive\nstatistics about the corpus makeup (and how video attributes affected\nannotators' judgments); discusses possible biases in the corpus introduced by\nour procedural choices and compares it with the most similar existing dataset,\nTRECVID MED's HAVIC corpus; and gives an overview of our future plans for\nexpanding the annotation effort.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1503.05543v1",
            "date": "2015-03-18 19:44:06+00:00",
            "title": "Text Segmentation based on Semantic Word Embeddings",
            "authors": [
                "Alexander A Alemi",
                "Paul Ginsparg"
            ],
            "abstract": "We explore the use of semantic word embeddings in text segmentation\nalgorithms, including the C99 segmentation algorithm and new algorithms\ninspired by the distributed word vector representation. By developing a general\nframework for discussing a class of segmentation objectives, we study the\neffectiveness of greedy versus exact optimization approaches and suggest a new\niterative refinement technique for improving the performance of greedy\nstrategies. We compare our results to known benchmarks, using known metrics. We\ndemonstrate state-of-the-art performance for an untrained method with our\nContent Vector Segmentation (CVS) on the Choi test set. Finally, we apply the\nsegmentation procedure to an in-the-wild dataset consisting of text extracted\nfrom scholarly articles in the arXiv.org database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.05615v2",
            "date": "2015-03-18 23:33:17+00:00",
            "title": "Learning to Search for Dependencies",
            "authors": [
                "Kai-Wei Chang",
                "He He",
                "Hal Daum\u00e9 III",
                "John Langford"
            ],
            "abstract": "We demonstrate that a dependency parser can be built using a credit\nassignment compiler which removes the burden of worrying about low-level\nmachine learning details from the parser implementation. The result is a simple\nparser which robustly applies to many languages that provides similar\nstatistical and computational performance with best-to-date transition-based\nparsing approaches, while avoiding various downsides including randomization,\nextra feature requirements, and custom learning algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.06934v1",
            "date": "2015-03-24 07:35:16+00:00",
            "title": "Measuring Software Quality in Use: State-of-the-Art and Research Challenges",
            "authors": [
                "Issa Atoum",
                "Chih How Bong"
            ],
            "abstract": "Software quality in use comprises quality from the user's perspective. It has\ngained its importance in e-government applications, mobile-based applications,\nembedded systems, and even business process development. User's decisions on\nsoftware acquisitions are often ad hoc or based on preference due to difficulty\nin quantitatively measuring software quality in use. But, why is quality-in-use\nmeasurement difficult? Although there are many software quality models, to the\nauthors' knowledge no works survey the challenges related to software\nquality-in-use measurement. This article has two main contributions: 1) it\nidentifies and explains major issues and challenges in measuring software\nquality in use in the context of the ISO SQuaRE series and related software\nquality models and highlights open research areas; and 2) it sheds light on a\nresearch direction that can be used to predict software quality in use. In\nshort, the quality-in-use measurement issues are related to the complexity of\nthe current standard models and the limitations and incompleteness of the\ncustomized software quality models. A sentiment analysis of software reviews is\nproposed to deal with these issues.",
            "journal ref": "ASQ.Software Quality Professional, 17(2), 2015",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1503.07921v2",
            "date": "2015-03-26 22:20:40+00:00",
            "title": "Breaking the News: First Impressions Matter on Online News",
            "authors": [
                "Julio Reis",
                "Fabr\u0131cio Benevenuto",
                "Pedro O. S. Vaz de Melo",
                "Raquel Prates",
                "Haewoon Kwak",
                "Jisun An"
            ],
            "abstract": "A growing number of people are changing the way they consume news, replacing\nthe traditional physical newspapers and magazines by their virtual online\nversions or/and weblogs. The interactivity and immediacy present in online news\nare changing the way news are being produced and exposed by media corporations.\nNews websites have to create effective strategies to catch people's attention\nand attract their clicks. In this paper we investigate possible strategies used\nby online news corporations in the design of their news headlines. We analyze\nthe content of 69,907 headlines produced by four major global media\ncorporations during a minimum of eight consecutive months in 2014. In order to\ndiscover strategies that could be used to attract clicks, we extracted features\nfrom the text of the news headlines related to the sentiment polarity of the\nheadline. We discovered that the sentiment of the headline is strongly related\nto the popularity of the news and also with the dynamics of the posted comments\non that particular news.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1503.08155v1",
            "date": "2015-03-27 17:13:03+00:00",
            "title": "Learning Embedding Representations for Knowledge Inference on Imperfect and Incomplete Repositories",
            "authors": [
                "Miao Fan",
                "Qiang Zhou",
                "Thomas Fang Zheng"
            ],
            "abstract": "This paper considers the problem of knowledge inference on large-scale\nimperfect repositories with incomplete coverage by means of embedding entities\nand relations at the first attempt. We propose IIKE (Imperfect and Incomplete\nKnowledge Embedding), a probabilistic model which measures the probability of\neach belief, i.e. $\\langle h,r,t\\rangle$, in large-scale knowledge bases such\nas NELL and Freebase, and our objective is to learn a better low-dimensional\nvector representation for each entity ($h$ and $t$) and relation ($r$) in the\nprocess of minimizing the loss of fitting the corresponding confidence given by\nmachine learning (NELL) or crowdsouring (Freebase), so that we can use $||{\\bf\nh} + {\\bf r} - {\\bf t}||$ to assess the plausibility of a belief when\nconducting inference. We use subsets of those inexact knowledge bases to train\nour model and test the performances of link prediction and triplet\nclassification on ground truth beliefs, respectively. The results of extensive\nexperiments show that IIKE achieves significant improvement compared with the\nbaseline and state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1503.08895v5",
            "date": "2015-03-31 03:05:37+00:00",
            "title": "End-To-End Memory Networks",
            "authors": [
                "Sainbayar Sukhbaatar",
                "Arthur Szlam",
                "Jason Weston",
                "Rob Fergus"
            ],
            "abstract": "We introduce a neural network with a recurrent attention model over a\npossibly large external memory. The architecture is a form of Memory Network\n(Weston et al., 2015) but unlike the model in that work, it is trained\nend-to-end, and hence requires significantly less supervision during training,\nmaking it more generally applicable in realistic settings. It can also be seen\nas an extension of RNNsearch to the case where multiple computational steps\n(hops) are performed per output symbol. The flexibility of the model allows us\nto apply it to tasks as diverse as (synthetic) question answering and to\nlanguage modeling. For the former our approach is competitive with Memory\nNetworks, but with less supervision. For the latter, on the Penn TreeBank and\nText8 datasets our approach demonstrates comparable performance to RNNs and\nLSTMs. In both cases we show that the key concept of multiple computational\nhops yields improved results.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1507.00209v1",
            "date": "2015-07-01 12:39:50+00:00",
            "title": "Dimensionality on Summarization",
            "authors": [
                "Hai Zhuge"
            ],
            "abstract": "Summarization is one of the key features of human intelligence. It plays an\nimportant role in understanding and representation. With rapid and continual\nexpansion of texts, pictures and videos in cyberspace, automatic summarization\nbecomes more and more desirable. Text summarization has been studied for over\nhalf century, but it is still hard to automatically generate a satisfied\nsummary. Traditional methods process texts empirically and neglect the\nfundamental characteristics and principles of language use and understanding.\nThis paper summarizes previous text summarization approaches in a\nmulti-dimensional classification space, introduces a multi-dimensional\nmethodology for research and development, unveils the basic characteristics and\nprinciples of language use and understanding, investigates some fundamental\nmechanisms of summarization, studies the dimensions and forms of\nrepresentations, and proposes a multi-dimensional evaluation mechanisms.\nInvestigation extends to the incorporation of pictures into summary and to the\nsummarization of videos, graphs and pictures, and then reaches a general\nsummarization framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02086v1",
            "date": "2015-07-08 10:04:14+00:00",
            "title": "The Role of Pragmatics in Legal Norm Representation",
            "authors": [
                "Shashishekar Ramakrishna",
                "Lukasz Gorski",
                "Adrian Paschke"
            ],
            "abstract": "Despite the 'apparent clarity' of a given legal provision, its application\nmay result in an outcome that does not exactly conform to the semantic level of\na statute. The vagueness within a legal text is induced intentionally to\naccommodate all possible scenarios under which such norms should be applied,\nthus making the role of pragmatics an important aspect also in the\nrepresentation of a legal norm and reasoning on top of it. The notion of\npragmatics considered in this paper does not focus on the aspects associated\nwith judicial decision making. The paper aims to shed light on the aspects of\npragmatics in legal linguistics, mainly focusing on the domain of patent law,\nonly from a knowledge representation perspective. The philosophical discussions\npresented in this paper are grounded based on the legal theories from Grice and\nMarmor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02205v2",
            "date": "2015-07-08 15:55:18+00:00",
            "title": "Talking to the crowd: What do people react to in online discussions?",
            "authors": [
                "Aaron Jaech",
                "Victoria Zayats",
                "Hao Fang",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "This paper addresses the question of how language use affects community\nreaction to comments in online discussion forums, and the relative importance\nof the message vs. the messenger. A new comment ranking task is proposed based\non community annotated karma in Reddit discussions, which controls for topic\nand timing of comments. Experimental work with discussion threads from six\nsubreddits shows that the importance of different types of language features\nvaries with the community of interest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02447v1",
            "date": "2015-07-09 10:20:52+00:00",
            "title": "Data Mining of Causal Relations from Text: Analysing Maritime Accident Investigation Reports",
            "authors": [
                "Santosh Tirunagari"
            ],
            "abstract": "Text mining is a process of extracting information of interest from text.\nSuch a method includes techniques from various areas such as Information\nRetrieval (IR), Natural Language Processing (NLP), and Information Extraction\n(IE). In this study, text mining methods are applied to extract causal\nrelations from maritime accident investigation reports collected from the\nMarine Accident Investigation Branch (MAIB). These causal relations provide\ninformation on various mechanisms behind accidents, including human and\norganizational factors relating to the accident. The objective of this study is\nto facilitate the analysis of the maritime accident investigation reports, by\nmeans of extracting contributory causes with more feasibility. A careful\ninvestigation of contributory causes from the reports provide opportunity to\nimprove safety in future.\n  Two methods have been employed in this study to extract the causal relations.\nThey are 1) Pattern classification method and 2) Connectives method. The\nearlier one uses naive Bayes and Support Vector Machines (SVM) as classifiers.\nThe latter simply searches for the words connecting cause and effect in\nsentences.\n  The causal patterns extracted using these two methods are compared to the\nmanual (human expert) extraction. The pattern classification method showed a\nfair and sensible performance with F-measure(average) = 65% when compared to\nconnectives method with F-measure(average) = 58%. This study is an evidence,\nthat text mining methods could be employed in extracting causal relations from\nmarine accident investigation reports.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1507.02907v1",
            "date": "2015-07-10 13:59:00+00:00",
            "title": "Extending a Single-Document Summarizer to Multi-Document: a Hierarchical Approach",
            "authors": [
                "Lu\u00eds Marujo",
                "Ricardo Ribeiro",
                "David Martins de Matos",
                "Jo\u00e3o P. Neto",
                "Anatole Gershman",
                "Jaime Carbonell"
            ],
            "abstract": "The increasing amount of online content motivated the development of\nmulti-document summarization methods. In this work, we explore straightforward\napproaches to extend single-document summarization methods to multi-document\nsummarization. The proposed methods are based on the hierarchical combination\nof single-document summaries, and achieves state of the art results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1507.03045v1",
            "date": "2015-07-10 23:17:53+00:00",
            "title": "Markov Logic Networks for Natural Language Question Answering",
            "authors": [
                "Tushar Khot",
                "Niranjan Balasubramanian",
                "Eric Gribkoff",
                "Ashish Sabharwal",
                "Peter Clark",
                "Oren Etzioni"
            ],
            "abstract": "Our goal is to answer elementary-level science questions using knowledge\nextracted automatically from science textbooks, expressed in a subset of\nfirst-order logic. Given the incomplete and noisy nature of these automatically\nextracted rules, Markov Logic Networks (MLNs) seem a natural model to use, but\nthe exact way of leveraging MLNs is by no means obvious. We investigate three\nways of applying MLNs to our task. In the first, we simply use the extracted\nscience rules directly as MLN clauses. Unlike typical MLN applications, our\ndomain has long and complex rules, leading to an unmanageable number of\ngroundings. We exploit the structure present in hard constraints to improve\ntractability, but the formulation remains ineffective. In the second approach,\nwe instead interpret science rules as describing prototypical entities, thus\nmapping rules directly to grounded MLN assertions, whose constants are then\nclustered using existing entity resolution methods. This drastically simplifies\nthe network, but still suffers from brittleness. Finally, our third approach,\ncalled Praline, uses MLNs to align the lexical elements as well as define and\ncontrol how inference should be performed in this task. Our experiments,\ndemonstrating a 15\\% accuracy boost and a 10x reduction in runtime, suggest\nthat the flexibility and different inference semantics of Praline are a better\nfit for the natural language question answering task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1507.03077v2",
            "date": "2015-07-11 08:54:45+00:00",
            "title": "A new hybrid stemming algorithm for Persian",
            "authors": [
                "Adel Rahimi"
            ],
            "abstract": "Stemming has been an influential part in Information retrieval and search\nengines. There have been tremendous endeavours in making stemmer that are both\nefficient and accurate. Stemmers can have three method in stemming, Dictionary\nbased stemmer, statistical-based stemmers, and rule-based stemmers. This paper\naims at building a hybrid stemmer that uses both Dictionary based method and\nrule-based method for stemming. This ultimately helps the efficacy and\naccurateness of the stemmer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.03641v1",
            "date": "2015-07-13 22:23:51+00:00",
            "title": "Neural CRF Parsing",
            "authors": [
                "Greg Durrett",
                "Dan Klein"
            ],
            "abstract": "This paper describes a parsing model that combines the exact dynamic\nprogramming of CRF parsing with the rich nonlinear featurization of neural net\napproaches. Our model is structurally a CRF that factors over anchored rule\nproductions, but instead of linear potential functions based on sparse\nfeatures, we use nonlinear potentials computed via a feedforward neural\nnetwork. Because potentials are still local to anchored rules, structured\ninference (CKY) is unchanged from the sparse case. Computing gradients during\nlearning involves backpropagating an error signal formed from standard CRF\nsufficient statistics (expected rule counts). Using only dense features, our\nneural CRF already exceeds a strong baseline CRF model (Hall et al., 2014). In\ncombination with sparse features, our system achieves 91.1 F1 on section 23 of\nthe Penn Treebank, and more generally outperforms the best prior single parser\nresults on a range of languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04019v1",
            "date": "2015-07-14 20:34:16+00:00",
            "title": "Feature Normalisation for Robust Speech Recognition",
            "authors": [
                "D. S. Pavan Kumar"
            ],
            "abstract": "Speech recognition system performance degrades in noisy environments. If the\nacoustic models are built using features of clean utterances, the features of a\nnoisy test utterance would be acoustically mismatched with the trained model.\nThis gives poor likelihoods and poor recognition accuracy. Model adaptation and\nfeature normalisation are two broad areas that address this problem. While the\nformer often gives better performance, the latter involves estimation of lesser\nnumber of parameters, making the system feasible for practical implementations.\n  This research focuses on the efficacies of various subspace, statistical and\nstereo based feature normalisation techniques. A subspace projection based\nmethod has been investigated as a standalone and adjunct technique involving\nreconstruction of noisy speech features from a precomputed set of clean speech\nbuilding-blocks. The building blocks are learned using non-negative matrix\nfactorisation (NMF) on log-Mel filter bank coefficients, which form a basis for\nthe clean speech subspace. The work provides a detailed study on how the method\ncan be incorporated into the extraction process of Mel-frequency cepstral\ncoefficients. Experimental results show that the new features are robust to\nnoise, and achieve better results when combined with the existing techniques.\n  The work also proposes a modification to the training process of SPLICE\nalgorithm for noise robust speech recognition. It is based on feature\ncorrelations, and enables this stereo-based algorithm to improve the\nperformance in all noise conditions, especially in unseen cases. Further, the\nmodified framework is extended to work for non-stereo datasets where clean and\nnoisy training utterances, but not stereo counterparts, are required. An\nMLLR-based computationally efficient run-time noise adaptation method in SPLICE\nframework has been proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04420v1",
            "date": "2015-07-16 00:27:02+00:00",
            "title": "Bias and population structure in the actuation of sound change",
            "authors": [
                "James Kirby",
                "Morgan Sonderegger"
            ],
            "abstract": "Why do human languages change at some times, and not others? We address this\nlongstanding question from a computational perspective, focusing on the case of\nsound change. Sound change arises from the pronunciation variability ubiquitous\nin every speech community, but most such variability does not lead to change.\nHence, an adequate model must allow for stability as well as change. Existing\ntheories of sound change tend to emphasize factors at the level of individual\nlearners promoting one outcome or the other, such as channel bias (which favors\nchange) or inductive bias (which favors stability). Here, we consider how the\ninteraction of these biases can lead to both stability and change in a\npopulation setting. We find that population structure itself can act as a\nsource of stability, but that both stability and change are possible only when\nboth types of bias are active, suggesting that it is possible to understand why\nsound change occurs at some times and not others as the population-level result\nof the interplay between forces promoting each outcome in individual speakers.\nIn addition, if it is assumed that learners learn from two or more teachers,\nthe transition from stability to change is marked by a phase transition,\nconsistent with the abrupt transitions seen in many empirical cases of sound\nchange. The predictions of multiple-teacher models thus match empirical cases\nof sound change better than the predictions of single-teacher models,\nunderscoring the importance of modeling language change in a population\nsetting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04908v1",
            "date": "2015-07-17 10:34:23+00:00",
            "title": "Analysis of the South Slavic Scripts by Run-Length Features of the Image Texture",
            "authors": [
                "Darko Brodic",
                "Zoran N. Milivojevic",
                "Alessia Amelio"
            ],
            "abstract": "The paper proposes an algorithm for the script recognition based on the\ntexture characteristics. The image texture is achieved by coding each letter\nwith the equivalent script type (number code) according to its position in the\ntext line. Each code is transformed into equivalent gray level pixel creating\nan 1-D image. Then, the image texture is subjected to the run-length analysis.\nThis analysis extracts the run-length features, which are classified to make a\ndistinction between the scripts under consideration. In the experiment, a\ncustom oriented database is subject to the proposed algorithm. The database\nconsists of some text documents written in Cyrillic, Latin and Glagolitic\nscripts. Furthermore, it is divided into training and test parts. The results\nof the experiment show that 3 out of 5 run-length features can be used for\neffective differentiation between the analyzed South Slavic scripts.",
            "journal ref": "Elektronika Ir Elektrotechnika, ISSN 1392-1215, VOL. 21, NO. 4,\n  2015",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1507.05134v1",
            "date": "2015-07-18 00:07:52+00:00",
            "title": "Persistent Topology of Syntax",
            "authors": [
                "Alexander Port",
                "Iulia Gheorghita",
                "Daniel Guth",
                "John M. Clark",
                "Crystal Liang",
                "Shival Dasu",
                "Matilde Marcolli"
            ],
            "abstract": "We study the persistent homology of the data set of syntactic parameters of\nthe world languages. We show that, while homology generators behave erratically\nover the whole data set, non-trivial persistent homology appears when one\nrestricts to specific language families. Different families exhibit different\npersistent homology. We focus on the cases of the Indo-European and the\nNiger-Congo families, for which we compare persistent homology over different\ncluster filtering values. We investigate the possible significance, in\nhistorical linguistic terms, of the presence of persistent generators of the\nfirst homology. In particular, we show that the persistent first homology\ngenerator we find in the Indo-European family is not due (as one might guess)\nto the Anglo-Norman bridge in the Indo-European phylogenetic network, but is\nrelated to the position of Ancient Greek and the Hellenic branch within the\nnetwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06020v1",
            "date": "2015-07-22 00:27:11+00:00",
            "title": "Practical Selection of SVM Supervised Parameters with Different Feature Representations for Vowel Recognition",
            "authors": [
                "Rimah Amami",
                "Dorra Ben Ayed",
                "Noureddine Ellouze"
            ],
            "abstract": "It is known that the classification performance of Support Vector Machine\n(SVM) can be conveniently affected by the different parameters of the kernel\ntricks and the regularization parameter, C. Thus, in this article, we propose a\nstudy in order to find the suitable kernel with which SVM may achieve good\ngeneralization performance as well as the parameters to use. We need to analyze\nthe behavior of the SVM classifier when these parameters take very small or\nvery large values. The study is conducted for a multi-class vowel recognition\nusing the TIMIT corpus. Furthermore, for the experiments, we used different\nfeature representations such as MFCC and PLP. Finally, a comparative study was\ndone to point out the impact of the choice of the parameters, kernel trick and\nfeature representations on the performance of the SVM classifier",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06021v1",
            "date": "2015-07-22 00:34:15+00:00",
            "title": "An Empirical Comparison of SVM and Some Supervised Learning Algorithms for Vowel recognition",
            "authors": [
                "Rimah Amami",
                "Dorra Ben Ayed",
                "Noureddine Ellouze"
            ],
            "abstract": "In this article, we conduct a study on the performance of some supervised\nlearning algorithms for vowel recognition. This study aims to compare the\naccuracy of each algorithm. Thus, we present an empirical comparison between\nfive supervised learning classifiers and two combined classifiers: SVM, KNN,\nNaive Bayes, Quadratic Bayes Normal (QDC) and Nearst Mean. Those algorithms\nwere tested for vowel recognition using TIMIT Corpus and Mel-frequency cepstral\ncoefficients (MFCCs).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06023v1",
            "date": "2015-07-22 00:49:48+00:00",
            "title": "Robust speech recognition using consensus function based on multi-layer networks",
            "authors": [
                "Rimah Amami",
                "Ghaith Manita",
                "Abir Smiti"
            ],
            "abstract": "The clustering ensembles mingle numerous partitions of a specified data into\na single clustering solution. Clustering ensemble has emerged as a potent\napproach for ameliorating both the forcefulness and the stability of\nunsupervised classification results. One of the major problems in clustering\nensembles is to find the best consensus function. Finding final partition from\ndifferent clustering results requires skillfulness and robustness of the\nclassification algorithm. In addition, the major problem with the consensus\nfunction is its sensitivity to the used data sets quality. This limitation is\ndue to the existence of noisy, silence or redundant data. This paper proposes a\nnovel consensus function of cluster ensembles based on Multilayer networks\ntechnique and a maintenance database method. This maintenance database approach\nis used in order to handle any given noisy speech and, thus, to guarantee the\nquality of databases. This can generates good results and efficient data\npartitions. To show its effectiveness, we support our strategy with empirical\nevaluation using distorted speech from Aurora speech databases.",
            "journal ref": "9th Iberian Conference on Information Systems and Technologies\n  (CISTI), Barcelona 18-21 June, 2014, pgs 1-6",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06025v1",
            "date": "2015-07-22 01:03:28+00:00",
            "title": "Incorporating Belief Function in SVM for Phoneme Recognition",
            "authors": [
                "Rimah Amami",
                "Dorra Ben Ayed",
                "Nouerddine Ellouze"
            ],
            "abstract": "The Support Vector Machine (SVM) method has been widely used in numerous\nclassification tasks. The main idea of this algorithm is based on the principle\nof the margin maximization to find an hyperplane which separates the data into\ntwo different classes.In this paper, SVM is applied to phoneme recognition\ntask. However, in many real-world problems, each phoneme in the data set for\nrecognition problems may differ in the degree of significance due to noise,\ninaccuracies, or abnormal characteristics; All those problems can lead to the\ninaccuracies in the prediction phase. Unfortunately, the standard formulation\nof SVM does not take into account all those problems and, in particular, the\nvariation in the speech input. This paper presents a new formulation of SVM\n(B-SVM) that attributes to each phoneme a confidence degree computed based on\nits geometric position in the space. Then, this degree is used in order to\nstrengthen the class membership of the tested phoneme. Hence, we introduce a\nreformulation of the standard SVM that incorporates the degree of belief.\nExperimental performance on TIMIT database shows the effectiveness of the\nproposed method B-SVM on a phoneme recognition problem.",
            "journal ref": "Lecture Notes in Computer Science Volume 8480, 2014, pp 191-199",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06028v1",
            "date": "2015-07-22 01:27:05+00:00",
            "title": "The challenges of SVM optimization using Adaboost on a phoneme recognition problem",
            "authors": [
                "Rimah Amami",
                "Dorra Ben Ayed",
                "Noureddine Ellouze"
            ],
            "abstract": "The use of digital technology is growing at a very fast pace which led to the\nemergence of systems based on the cognitive infocommunications. The expansion\nof this sector impose the use of combining methods in order to ensure the\nrobustness in cognitive systems.",
            "journal ref": "IEEE 4th International Conference on Cognitive Infocommunications\n  (CogInfoCom), Budapest 2-5 Dec. 2013, pgs 463-468",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06711v2",
            "date": "2015-07-24 00:36:01+00:00",
            "title": "The SYSU System for the Interspeech 2015 Automatic Speaker Verification Spoofing and Countermeasures Challenge",
            "authors": [
                "Shitao Weng",
                "Shushan Chen",
                "Lei Yu",
                "Xuewei Wu",
                "Weicheng Cai",
                "Zhi Liu",
                "Ming Li"
            ],
            "abstract": "Many existing speaker verification systems are reported to be vulnerable\nagainst different spoofing attacks, for example speaker-adapted speech\nsynthesis, voice conversion, play back, etc. In order to detect these spoofed\nspeech signals as a countermeasure, we propose a score level fusion approach\nwith several different i-vector subsystems. We show that the acoustic level\nMel-frequency cepstral coefficients (MFCC) features, the phase level modified\ngroup delay cepstral coefficients (MGDCC) and the phonetic level phoneme\nposterior probability (PPP) tandem features are effective for the\ncountermeasure. Furthermore, feature level fusion of these features before\ni-vector modeling also enhance the performance. A polynomial kernel support\nvector machine is adopted as the supervised classifier. In order to enhance the\ngeneralizability of the countermeasure, we also adopted the cosine similarity\nand PLDA scoring as one-class classifications methods. By combining the\nproposed i-vector subsystems with the OpenSMILE baseline which covers the\nacoustic and prosodic information further improves the final performance. The\nproposed fusion system achieves 0.29% and 3.26% EER on the development and test\nset of the database provided by the INTERSPEECH 2015 automatic speaker\nverification spoofing and countermeasures challenge.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1507.06837v1",
            "date": "2015-07-24 13:21:59+00:00",
            "title": "YARBUS : Yet Another Rule Based belief Update System",
            "authors": [
                "Jeremy Fix",
                "Herve Frezza-buet"
            ],
            "abstract": "We introduce a new rule based system for belief tracking in dialog systems.\nDespite the simplicity of the rules being considered, the proposed belief\ntracker ranks favourably compared to the previous submissions on the second and\nthird Dialog State Tracking challenges. The results of this simple tracker\nallows to reconsider the performances of previous submissions using more\nelaborate techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.08240v3",
            "date": "2015-07-29 17:53:50+00:00",
            "title": "EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding",
            "authors": [
                "Yajie Miao",
                "Mohammad Gowayyed",
                "Florian Metze"
            ],
            "abstract": "The performance of automatic speech recognition (ASR) has improved\ntremendously due to the application of deep neural networks (DNNs). Despite\nthis progress, building a new ASR system remains a challenging task, requiring\nvarious resources, multiple training stages and significant expertise. This\npaper presents our Eesen framework which drastically simplifies the existing\npipeline to build state-of-the-art ASR systems. Acoustic modeling in Eesen\ninvolves learning a single recurrent neural network (RNN) predicting\ncontext-independent targets (phonemes or characters). To remove the need for\npre-generated frame labels, we adopt the connectionist temporal classification\n(CTC) objective function to infer the alignments between speech and label\nsequences. A distinctive feature of Eesen is a generalized decoding approach\nbased on weighted finite-state transducers (WFSTs), which enables the efficient\nincorporation of lexicons and language models into CTC decoding. Experiments\nshow that compared with the standard hybrid DNN systems, Eesen achieves\ncomparable word error rates (WERs), while at the same time speeding up decoding\nsignificantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00576v1",
            "date": "2015-12-02 04:41:58+00:00",
            "title": "Probabilistic Latent Semantic Analysis (PLSA) untuk Klasifikasi Dokumen Teks Berbahasa Indonesia",
            "authors": [
                "Derwin Suhartono"
            ],
            "abstract": "One task that is included in managing documents is how to find substantial\ninformation inside. Topic modeling is a technique that has been developed to\nproduce document representation in form of keywords. The keywords will be used\nin the indexing process and document retrieval as needed by users. In this\nresearch, we will discuss specifically about Probabilistic Latent Semantic\nAnalysis (PLSA). It will cover PLSA mechanism which involves Expectation\nMaximization (EM) as the training algorithm, how to conduct testing, and obtain\nthe accuracy result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00578v1",
            "date": "2015-12-02 04:58:38+00:00",
            "title": "Klasifikasi Komponen Argumen Secara Otomatis pada Dokumen Teks berbentuk Esai Argumentatif",
            "authors": [
                "Derwin Suhartono"
            ],
            "abstract": "By automatically recognize argument component, essay writers can do some\ninspections to texts that they have written. It will assist essay scoring\nprocess objectively and precisely because essay grader is able to see how well\nthe argument components are constructed. Some reseachers have tried to do\nargument detection and classification along with its implementation in some\ndomains. The common approach is by doing feature extraction to the text.\nGenerally, the features are structural, lexical, syntactic, indicator, and\ncontextual. In this research, we add new feature to the existing features. It\nadopts keywords list by Knott and Dale (1993). The experiment result shows the\nargument classification achieves 72.45% accuracy. Moreover, we still get the\nsame accuracy without the keyword lists. This concludes that the keyword lists\ndo not affect significantly to the features. All features are still weak to\nclassify major claim and claim, so we need other features which are useful to\ndifferentiate those two kind of argument components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00765v1",
            "date": "2015-12-02 16:31:20+00:00",
            "title": "Learning Semantic Similarity for Very Short Texts",
            "authors": [
                "Cedric De Boom",
                "Steven Van Canneyt",
                "Steven Bohez",
                "Thomas Demeester",
                "Bart Dhoedt"
            ],
            "abstract": "Levering data on social media, such as Twitter and Facebook, requires\ninformation retrieval algorithms to become able to relate very short text\nfragments to each other. Traditional text similarity methods such as tf-idf\ncosine-similarity, based on word overlap, mostly fail to produce good results\nin this case, since word overlap is little or non-existent. Recently,\ndistributed word representations, or word embeddings, have been shown to\nsuccessfully allow words to match on the semantic level. In order to pair short\ntext fragments - as a concatenation of separate words - an adequate distributed\nsentence representation is needed, in existing literature often obtained by\nnaively combining the individual word representations. We therefore\ninvestigated several text representations as a combination of word embeddings\nin the context of semantic pair matching. This paper investigates the\neffectiveness of several such naive techniques, as well as traditional tf-idf\nsimilarity, for fragments of different lengths. Our main contribution is a\nfirst step towards a hybrid method that combines the strength of dense\ndistributed representations - as opposed to sparse term matching - with the\nstrength of tf-idf based methods to automatically reduce the impact of less\ninformative terms. Our new approach outperforms the existing techniques in a\ntoy experimental set-up, leading to the conclusion that the combination of word\nembeddings and tf-idf information might lead to a better model for semantic\ncontent within very short text fragments.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1512.01283v1",
            "date": "2015-12-03 23:42:10+00:00",
            "title": "Predicting the top and bottom ranks of billboard songs using Machine Learning",
            "authors": [
                "Vivek Datla",
                "Abhinav Vishnu"
            ],
            "abstract": "The music industry is a $130 billion industry. Predicting whether a song\ncatches the pulse of the audience impacts the industry. In this paper we\nanalyze language inside the lyrics of the songs using several computational\nlinguistic algorithms and predict whether a song would make to the top or\nbottom of the billboard rankings based on the language features. We trained and\ntested an SVM classifier with a radial kernel function on the linguistic\nfeatures. Results indicate that we can classify whether a song belongs to top\nand bottom of the billboard charts with a precision of 0.76.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01370v1",
            "date": "2015-12-04 11:09:55+00:00",
            "title": "Locally Adaptive Translation for Knowledge Graph Embedding",
            "authors": [
                "Yantao Jia",
                "Yuanzhuo Wang",
                "Hailun Lin",
                "Xiaolong Jin",
                "Xueqi Cheng"
            ],
            "abstract": "Knowledge graph embedding aims to represent entities and relations in a\nlarge-scale knowledge graph as elements in a continuous vector space. Existing\nmethods, e.g., TransE and TransH, learn embedding representation by defining a\nglobal margin-based loss function over the data. However, the optimal loss\nfunction is determined during experiments whose parameters are examined among a\nclosed set of candidates. Moreover, embeddings over two knowledge graphs with\ndifferent entities and relations share the same set of candidate loss\nfunctions, ignoring the locality of both graphs. This leads to the limited\nperformance of embedding related applications. In this paper, we propose a\nlocally adaptive translation method for knowledge graph embedding, called\nTransA, to find the optimal loss function by adaptively determining its margin\nover different knowledge graphs. Experiments on two benchmark data sets\ndemonstrate the superiority of the proposed method, as compared to\nthe-state-of-the-art ones.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1512.01384v1",
            "date": "2015-12-04 12:22:26+00:00",
            "title": "Topic segmentation via community detection in complex networks",
            "authors": [
                "Henrique F. de Arruda",
                "Luciano da F. Costa",
                "Diego R. Amancio"
            ],
            "abstract": "Many real systems have been modelled in terms of network concepts, and\nwritten texts are a particular example of information networks. In recent\nyears, the use of network methods to analyze language has allowed the discovery\nof several interesting findings, including the proposition of novel models to\nexplain the emergence of fundamental universal patterns. While syntactical\nnetworks, one of the most prevalent networked models of written texts, display\nboth scale-free and small-world properties, such representation fails in\ncapturing other textual features, such as the organization in topics or\nsubjects. In this context, we propose a novel network representation whose main\npurpose is to capture the semantical relationships of words in a simple way. To\ndo so, we link all words co-occurring in the same semantic context, which is\ndefined in a threefold way. We show that the proposed representations favours\nthe emergence of communities of semantically related words, and this feature\nmay be used to identify relevant topics. The proposed methodology to detect\ntopics was applied to segment selected Wikipedia articles. We have found that,\nin general, our methods outperform traditional bag-of-words representations,\nwhich suggests that a high-level textual representation may be useful to study\nsemantical features of texts.",
            "journal ref": "Chaos 26, 063120 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01639v1",
            "date": "2015-12-05 08:55:31+00:00",
            "title": "PJAIT Systems for the IWSLT 2015 Evaluation Campaign Enhanced by Comparable Corpora",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "In this paper, we attempt to improve Statistical Machine Translation (SMT)\nsystems on a very diverse set of language pairs (in both directions): Czech -\nEnglish, Vietnamese - English, French - English and German - English. To\naccomplish this, we performed translation model training, created adaptations\nof training settings for each language pair, and obtained comparable corpora\nfor our SMT systems. Innovative tools and data adaptation techniques were\nemployed. The TED parallel text corpora for the IWSLT 2015 evaluation campaign\nwere used to train language models, and to develop, tune, and test the system.\nIn addition, we prepared Wikipedia-based comparable corpora for use with our\nSMT system. This data was specified as permissible for the IWSLT 2015\nevaluation. We explored the use of domain adaptation techniques, symmetrized\nword alignment models, the unsupervised transliteration models and the KenLM\nlanguage modeling tool. To evaluate the effects of different preparations on\ntranslation results, we conducted experiments and used the BLEU, NIST and TER\nmetrics. Our results indicate that our approach produced a positive impact on\nSMT quality.",
            "journal ref": "Proceedings of the 12th International Workshop on Spoken Language\n  Translation, Da Nang, Vietnam, December 3-4, 2015, p.101-104",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01641v1",
            "date": "2015-12-05 08:59:28+00:00",
            "title": "Unsupervised comparable corpora preparation and exploration for bi-lingual translation equivalents",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "The multilingual nature of the world makes translation a crucial requirement\ntoday. Parallel dictionaries constructed by humans are a widely-available\nresource, but they are limited and do not provide enough coverage for good\nquality translation purposes, due to out-of-vocabulary words and neologisms.\nThis motivates the use of statistical translation systems, which are\nunfortunately dependent on the quantity and quality of training data. Such\nsystems have a very limited availability especially for some languages and very\nnarrow text domains. In this research we present our improvements to current\ncomparable corpora mining methodologies by re- implementation of the comparison\nalgorithms (using Needleman-Wunch algorithm), introduction of a tuning script\nand computation time improvement by GPU acceleration. Experiments are carried\nout on bilingual data extracted from the Wikipedia, on various domains. For the\nWikipedia itself, additional cross-lingual comparison heuristics were\nintroduced. The modifications made a positive impact on the quality and\nquantity of mined data and on the translation quality.",
            "journal ref": "Proceedings of the 12th IWSLT, Da Nang, Vietnam, December 3-4,\n  2015, p.118-125",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01818v5",
            "date": "2015-12-06 18:52:51+00:00",
            "title": "SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods",
            "authors": [
                "Filipe Nunes Ribeiro",
                "Matheus Ara\u00fajo",
                "Pollyanna Gon\u00e7alves",
                "Fabr\u00edcio Benevenuto",
                "Marcos Andr\u00e9 Gon\u00e7alves"
            ],
            "abstract": "In the last few years thousands of scientific papers have investigated\nsentiment analysis, several startups that measure opinions on real data have\nemerged and a number of innovative products related to this theme have been\ndeveloped. There are multiple methods for measuring sentiments, including\nlexical-based and supervised machine learning methods. Despite the vast\ninterest on the theme and wide popularity of some methods, it is unclear which\none is better for identifying the polarity (i.e., positive or negative) of a\nmessage. Accordingly, there is a strong need to conduct a thorough\napple-to-apple comparison of sentiment analysis methods, \\textit{as they are\nused in practice}, across multiple datasets originated from different data\nsources. Such a comparison is key for understanding the potential limitations,\nadvantages, and disadvantages of popular methods. This article aims at filling\nthis gap by presenting a benchmark comparison of twenty-four popular sentiment\nanalysis methods (which we call the state-of-the-practice methods). Our\nevaluation is based on a benchmark of eighteen labeled datasets, covering\nmessages posted on social networks, movie and product reviews, as well as\nopinions and comments in news articles. Our results highlight the extent to\nwhich the prediction performance of these methods varies considerably across\ndatasets. Aiming at boosting the development of this research area, we open the\nmethods' codes and datasets used in this article, deploying them in a benchmark\nsystem, which provides an open API for accessing and comparing sentence-level\nsentiment analysis methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01882v2",
            "date": "2015-12-07 02:07:21+00:00",
            "title": "THCHS-30 : A Free Chinese Speech Corpus",
            "authors": [
                "Dong Wang",
                "Xuewei Zhang"
            ],
            "abstract": "Speech data is crucially important for speech recognition research. There are\nquite some speech databases that can be purchased at prices that are reasonable\nfor most research institutes. However, for young people who just start research\nactivities or those who just gain initial interest in this direction, the cost\nfor data is still an annoying barrier. We support the `free data' movement in\nspeech recognition: research institutes (particularly supported by public\nfunds) publish their data freely so that new researchers can obtain sufficient\ndata to kick of their career. In this paper, we follow this trend and release a\nfree Chinese speech database THCHS-30 that can be used to build a full- edged\nChinese speech recognition system. We report the baseline system established\nwith this database, including the performance under highly noisy conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.02167v2",
            "date": "2015-12-07 19:00:54+00:00",
            "title": "Simple Baseline for Visual Question Answering",
            "authors": [
                "Bolei Zhou",
                "Yuandong Tian",
                "Sainbayar Sukhbaatar",
                "Arthur Szlam",
                "Rob Fergus"
            ],
            "abstract": "We describe a very simple bag-of-words baseline for visual question\nanswering. This baseline concatenates the word features from the question and\nCNN features from the image to predict the answer. When evaluated on the\nchallenging VQA dataset [2], it shows comparable performance to many recent\napproaches using recurrent neural networks. To explore the strength and\nweakness of the trained model, we also provide an interactive web demo and\nopen-source code. .",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1512.02902v2",
            "date": "2015-12-09 15:34:31+00:00",
            "title": "MovieQA: Understanding Stories in Movies through Question-Answering",
            "authors": [
                "Makarand Tapaswi",
                "Yukun Zhu",
                "Rainer Stiefelhagen",
                "Antonio Torralba",
                "Raquel Urtasun",
                "Sanja Fidler"
            ],
            "abstract": "We introduce the MovieQA dataset which aims to evaluate automatic story\ncomprehension from both video and text. The dataset consists of 14,944\nquestions about 408 movies with high semantic diversity. The questions range\nfrom simpler \"Who\" did \"What\" to \"Whom\", to \"Why\" and \"How\" certain events\noccurred. Each question comes with a set of five possible answers; a correct\none and four deceiving answers provided by human annotators. Our dataset is\nunique in that it contains multiple sources of information -- video clips,\nplots, subtitles, scripts, and DVS. We analyze our data through various\nstatistics and methods. We further extend existing QA techniques to show that\nquestion-answering with such open-ended semantics is hard. We make this data\nset public along with an evaluation benchmark to encourage inspiring work in\nthis challenging domain.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1512.04092v1",
            "date": "2015-12-13 17:52:44+00:00",
            "title": "Stack Exchange Tagger",
            "authors": [
                "Sanket Mehta",
                "Shagun Sodhani"
            ],
            "abstract": "The goal of our project is to develop an accurate tagger for questions posted\non Stack Exchange. Our problem is an instance of the more general problem of\ndeveloping accurate classifiers for large scale text datasets. We are tackling\nthe multilabel classification problem where each item (in this case, question)\ncan belong to multiple classes (in this case, tags). We are predicting the tags\n(or keywords) for a particular Stack Exchange post given only the question text\nand the title of the post. In the process, we compare the performance of\nSupport Vector Classification (SVC) for different kernel functions, loss\nfunction, etc. We found linear SVC with Crammer Singer technique produces best\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04906v1",
            "date": "2015-12-15 19:29:01+00:00",
            "title": "Strategies for Training Large Vocabulary Neural Language Models",
            "authors": [
                "Welin Chen",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "Training neural network language models over large vocabularies is still\ncomputationally very costly compared to count-based models such as Kneser-Ney.\nAt the same time, neural language models are gaining popularity for many\napplications such as speech recognition and machine translation whose success\ndepends on scalability. We present a systematic comparison of strategies to\nrepresent and train large vocabularies, including softmax, hierarchical\nsoftmax, target sampling, noise contrastive estimation and self normalization.\nWe further extend self normalization to be a proper estimator of likelihood and\nintroduce an efficient variant of softmax. We evaluate each method on three\npopular benchmarks, examining performance on rare words, the speed/accuracy\ntrade-off and complementarity to Kneser-Ney.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04973v1",
            "date": "2015-12-15 21:23:20+00:00",
            "title": "An Operator for Entity Extraction in MapReduce",
            "authors": [
                "Ndapandula Nakashole"
            ],
            "abstract": "Dictionary-based entity extraction involves finding mentions of dictionary\nentities in text. Text mentions are often noisy, containing spurious or missing\nwords. Efficient algorithms for detecting approximate entity mentions follow\none of two general techniques. The first approach is to build an index on the\nentities and perform index lookups of document substrings. The second approach\nrecognizes that the number of substrings generated from documents can explode\nto large numbers, to get around this, they use a filter to prune many such\nsubstrings which do not match any dictionary entity and then only verify the\nremaining substrings if they are entity mentions of dictionary entities, by\nmeans of a text join. The choice between the index-based approach and the\nfilter & verification-based approach is a case-to-case decision as the best\napproach depends on the characteristics of the input entity dictionary, for\nexample frequency of entity mentions. Choosing the right approach for the\nsetting can make a substantial difference in execution time. Making this choice\nis however non-trivial as there are parameters within each of the approaches\nthat make the space of possible approaches very large. In this paper, we\npresent a cost-based operator for making the choice among execution plans for\nentity extraction. Since we need to deal with large dictionaries and even\nlarger large datasets, our operator is developed for implementations of\nMapReduce distributed algorithms.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1512.05726v2",
            "date": "2015-12-17 19:14:20+00:00",
            "title": "Semi-supervised Question Retrieval with Gated Convolutions",
            "authors": [
                "Tao Lei",
                "Hrishikesh Joshi",
                "Regina Barzilay",
                "Tommi Jaakkola",
                "Katerina Tymoshenko",
                "Alessandro Moschitti",
                "Lluis Marquez"
            ],
            "abstract": "Question answering forums are rapidly growing in size with no effective\nautomated ability to refer to and reuse answers already available for previous\nposted questions. In this paper, we develop a methodology for finding\nsemantically related questions. The task is difficult since 1) key pieces of\ninformation are often buried in extraneous details in the question body and 2)\navailable annotations on similar questions are scarce and fragmented. We design\na recurrent and convolutional model (gated convolution) to effectively map\nquestions to their semantic representations. The models are pre-trained within\nan encoder-decoder framework (from body to title) on the basis of the entire\nraw corpus, and fine-tuned discriminatively from limited annotations. Our\nevaluation demonstrates that our model yields substantial gains over a standard\nIR baseline and various neural network architectures (including CNNs, LSTMs and\nGRUs).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.07046v1",
            "date": "2015-12-22 12:11:32+00:00",
            "title": "News Across Languages - Cross-Lingual Document Similarity and Event Tracking",
            "authors": [
                "Jan Rupnik",
                "Andrej Muhic",
                "Gregor Leban",
                "Primoz Skraba",
                "Blaz Fortuna",
                "Marko Grobelnik"
            ],
            "abstract": "In today's world, we follow news which is distributed globally. Significant\nevents are reported by different sources and in different languages. In this\nwork, we address the problem of tracking of events in a large multilingual\nstream. Within a recently developed system Event Registry we examine two\naspects of this problem: how to compare articles in different languages and how\nto link collections of articles in different languages which refer to the same\nevent. Taking a multilingual stream and clusters of articles from each\nlanguage, we compare different cross-lingual document similarity measures based\non Wikipedia. This allows us to compute the similarity of any two articles\nregardless of language. Building on previous work, we show there are methods\nwhich scale well and can compute a meaningful similarity between articles from\nlanguages with little or no direct overlap in the training data. Using this\ncapability, we then propose an approach to link clusters of articles across\nlanguages which represent the same event. We provide an extensive evaluation of\nthe system as a whole, as well as an evaluation of the quality and robustness\nof the similarity measure and the linking algorithm.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1512.07685v1",
            "date": "2015-12-24 01:40:20+00:00",
            "title": "Service Choreography, SBVR, and Time",
            "authors": [
                "Nurulhuda A. Manaf",
                "Sotiris Moschoyiannis",
                "Paul Krause"
            ],
            "abstract": "We propose the use of structured natural language (English) in specifying\nservice choreographies, focusing on the what rather than the how of the\nrequired coordination of participant services in realising a business\napplication scenario. The declarative approach we propose uses the OMG standard\nSemantics of Business Vocabulary and Rules (SBVR) as a modelling language. The\nservice choreography approach has been proposed for describing the global\norderings of the invocations on interfaces of participant services. We\ntherefore extend SBVR with a notion of time which can capture the coordination\nof the participant services, in terms of the observable message exchanges\nbetween them. The extension is done using existing modelling constructs in\nSBVR, and hence respects the standard specification. The idea is that users -\ndomain specialists rather than implementation specialists - can verify the\nrequested service composition by directly reading the structured English used\nby SBVR. At the same time, the SBVR model can be represented in formal logic so\nit can be parsed and executed by a machine.",
            "journal ref": "EPTCS 201, 2015, pp. 63-77",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1512.08347v4",
            "date": "2015-12-28 08:50:50+00:00",
            "title": "Communicating with sentences: A multi-word naming game model",
            "authors": [
                "Yang Lou",
                "Guanrong Chen",
                "Jianwei Hu"
            ],
            "abstract": "Naming game simulates the process of naming an object by a single word, in\nwhich a population of communicating agents can reach global consensus\nasymptotically through iteratively pair-wise conversations. We propose an\nextension of the single-word model to a multi-word naming game (MWNG),\nsimulating the case of describing a complex object by a sentence (multiple\nwords). Words are defined in categories, and then organized as sentences by\ncombining them from different categories. We refer to a formatted combination\nof several words as a pattern. In such an MWNG, through a pair-wise\nconversation, it requires the hearer to achieve consensus with the speaker with\nrespect to both every single word in the sentence as well as the sentence\npattern, so as to guarantee the correct meaning of the saying, otherwise, they\nfail reaching consensus in the interaction. We validate the model in three\ntypical topologies as the underlying communication network, and employ both\nconventional and man-designed patterns in performing the MWNG.",
            "journal ref": "Physica A 490 (2018) 857-868",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.08422v3",
            "date": "2015-12-28 14:28:21+00:00",
            "title": "Natural Language Inference by Tree-Based Convolution and Heuristic Matching",
            "authors": [
                "Lili Mou",
                "Rui Men",
                "Ge Li",
                "Yan Xu",
                "Lu Zhang",
                "Rui Yan",
                "Zhi Jin"
            ],
            "abstract": "In this paper, we propose the TBCNN-pair model to recognize entailment and\ncontradiction between two sentences. In our model, a tree-based convolutional\nneural network (TBCNN) captures sentence-level semantics; then heuristic\nmatching layers like concatenation, element-wise product/difference combine the\ninformation in individual sentences. Experimental results show that our model\noutperforms existing sentence encoding-based approaches by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.08569v1",
            "date": "2015-12-29 00:39:52+00:00",
            "title": "Analyzing Walter Skeat's Forty-Five Parallel Extracts of William Langland's Piers Plowman",
            "authors": [
                "Roger Bilisoly"
            ],
            "abstract": "Walter Skeat published his critical edition of William Langland's 14th\ncentury alliterative poem, Piers Plowman, in 1886. In preparation for this he\nlocated forty-five manuscripts, and to compare dialects, he published excerpts\nfrom each of these. This paper does three statistical analyses using these\nexcerpts, each of which mimics a task he did in writing his critical edition.\nFirst, he combined multiple versions of a poetic line to create a best line,\nwhich is compared to the mean string that is computed by a generalization of\nthe arithmetic mean that uses edit distance. Second, he claims that a certain\nsubset of manuscripts varies little. This is quantified by computing a string\nvariance, which is closely related to the above generalization of the mean.\nThird, he claims that the manuscripts fall into three groups, which is a\nclustering problem that is addressed by using edit distance. The overall goal\nis to develop methodology that would be of use to a literary critic.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1512.08982v1",
            "date": "2015-12-30 15:46:30+00:00",
            "title": "Technical Report: a tool for measuring Prosodic Accommodation",
            "authors": [
                "Sucheta Ghosh"
            ],
            "abstract": "This article has been withdrawn by arXiv administrators because the submitter\ndid not have the legal authority to grant the license applied to the work.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1510.00001v1",
            "date": "2015-09-30 09:41:00+00:00",
            "title": "Polish to English Statistical Machine Translation",
            "authors": [
                "Krzysztof Wo\u0142k"
            ],
            "abstract": "This research explores the effects of various training settings on a Polish\nto English Statistical Machine Translation system for spoken language. Various\nelements of the TED, Europarl, and OPUS parallel text corpora were used as the\nbasis for training of language models, for development, tuning and testing of\nthe translation system. The BLEU, NIST, METEOR and TER metrics were used to\nevaluate the effects of the data preparations on the translation results.",
            "journal ref": "Polish to English Statistical Machine Translation., XV\n  International Phd Workshop OWD 2013, Wis{\\l}a, p.108-115, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00244v1",
            "date": "2015-10-01 14:10:25+00:00",
            "title": "RDF Knowledge Graph Visualization From a Knowledge Extraction System",
            "authors": [
                "Fadhela Kerdjoudj",
                "Olivier Cur\u00e9"
            ],
            "abstract": "In this paper, we present a system to visualize RDF knowledge graphs. These\ngraphs are obtained from a knowledge extraction system designed by\nGEOLSemantics. This extraction is performed using natural language processing\nand trigger detection. The user can visualize subgraphs by selecting some\nontology features like concepts or individuals. The system is also\nmultilingual, with the use of the annotated ontology in English, French, Arabic\nand Chinese.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1510.01315v2",
            "date": "2015-10-05 09:43:25+00:00",
            "title": "Stochastic model for phonemes uncovers an author-dependency of their usage",
            "authors": [
                "Weibing Deng",
                "Armen E. Allahverdyan"
            ],
            "abstract": "We study rank-frequency relations for phonemes, the minimal units that still\nrelate to linguistic meaning. We show that these relations can be described by\nthe Dirichlet distribution, a direct analogue of the ideal-gas model in\nstatistical mechanics. This description allows us to demonstrate that the\nrank-frequency relations for phonemes of a text do depend on its author. The\nauthor-dependency effect is not caused by the author's vocabulary (common words\nused in different texts), and is confirmed by several alternative means. This\nsuggests that it can be directly related to phonemes. These features contrast\nto rank-frequency relations for words, which are both author and text\nindependent and are governed by the Zipf's law.",
            "journal ref": "PLoS ONE 11(4): e0152561 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.01431v2",
            "date": "2015-10-06 04:57:47+00:00",
            "title": "SentiCap: Generating Image Descriptions with Sentiments",
            "authors": [
                "Alexander Mathews",
                "Lexing Xie",
                "Xuming He"
            ],
            "abstract": "The recent progress on image recognition and language modeling is making\nautomatic description of image content a reality. However, stylized,\nnon-factual aspects of the written description are missing from the current\nsystems. One such style is descriptions with emotions, which is commonplace in\neveryday communication, and influences decision-making and interpersonal\nrelationships. We design a system to describe an image with emotions, and\npresent a model that automatically generates captions with positive or negative\nsentiments. We propose a novel switching recurrent neural network with\nword-level regularization, which is able to produce emotional image captions\nusing only 2000+ training sentences containing sentiments. We evaluate the\ncaptions with different automatic and crowd-sourcing metrics. Our model\ncompares favourably in common quality metrics for image captioning. In 84.6% of\ncases the generated positive captions were judged as being at least as\ndescriptive as the factual captions. Of these positive captions 88% were\nconfirmed by the crowd-sourced workers as having the appropriate sentiment.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1510.01562v1",
            "date": "2015-10-06 13:07:31+00:00",
            "title": "Parameterized Neural Network Language Models for Information Retrieval",
            "authors": [
                "Benjamin Piwowarski",
                "Sylvain Lamprier",
                "Nicolas Despres"
            ],
            "abstract": "Information Retrieval (IR) models need to deal with two difficult issues,\nvocabulary mismatch and term dependencies. Vocabulary mismatch corresponds to\nthe difficulty of retrieving relevant documents that do not contain exact query\nterms but semantically related terms. Term dependencies refers to the need of\nconsidering the relationship between the words of the query when estimating the\nrelevance of a document. A multitude of solutions has been proposed to solve\neach of these two problems, but no principled model solve both. In parallel, in\nthe last few years, language models based on neural networks have been used to\ncope with complex natural language processing tasks like emotion and paraphrase\ndetection. Although they present good abilities to cope with both term\ndependencies and vocabulary mismatch problems, thanks to the distributed\nrepresentation of words they are based upon, such models could not be used\nreadily in IR, where the estimation of one language model per document (or\nquery) is required. This is both computationally unfeasible and prone to\nover-fitting. Based on a recent work that proposed to learn a generic language\nmodel that can be modified through a set of document-specific parameters, we\nexplore use of new neural network models that are adapted to ad-hoc IR tasks.\nWithin the language model IR framework, we propose and study the use of a\ngeneric language model as well as a document-specific language model. Both can\nbe used as a smoothing component, but the latter is more adapted to the\ndocument at hand and has the potential of being used as a full document\nlanguage model. We experiment with such models and analyze their results on\nTREC-1 to 8 datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1510.01942v1",
            "date": "2015-10-07 13:47:12+00:00",
            "title": "Helping Domain Experts Build Speech Translation Systems",
            "authors": [
                "Manny Rayner",
                "Alejandro Armando",
                "Pierrette Bouillon",
                "Sarah Ebling",
                "Johanna Gerlach",
                "Sonia Halimi",
                "Irene Strasly",
                "Nikos Tsourakis"
            ],
            "abstract": "We present a new platform, \"Regulus Lite\", which supports rapid development\nand web deployment of several types of phrasal speech translation systems using\na minimal formalism. A distinguishing feature is that most development work can\nbe performed directly by domain experts. We motivate the need for platforms of\nthis type and discuss three specific cases: medical speech translation,\nspeech-to-sign-language translation and voice questionnaires. We briefly\ndescribe initial experiences in developing practical systems.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1510.01949v1",
            "date": "2015-10-07 14:08:13+00:00",
            "title": "Hierarchical Representation of Prosody for Statistical Speech Synthesis",
            "authors": [
                "Antti Suni",
                "Daniel Aalto",
                "Martti Vainio"
            ],
            "abstract": "Prominences and boundaries are the essential constituents of prosodic\nstructure in speech. They provide for means to chunk the speech stream into\nlinguistically relevant units by providing them with relative saliences and\ndemarcating them within coherent utterance structures. Prominences and\nboundaries have both been widely used in both basic research on prosody as well\nas in text-to-speech synthesis. However, there are no representation schemes\nthat would provide for both estimating and modelling them in a unified fashion.\nHere we present an unsupervised unified account for estimating and representing\nprosodic prominences and boundaries using a scale-space analysis based on\ncontinuous wavelet transform. The methods are evaluated and compared to earlier\nwork using the Boston University Radio News corpus. The results show that the\nproposed method is comparable with the best published supervised annotation\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02387v2",
            "date": "2015-10-08 16:17:47+00:00",
            "title": "Mapping Unseen Words to Task-Trained Embedding Spaces",
            "authors": [
                "Pranava Swaroop Madhyastha",
                "Mohit Bansal",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We consider the supervised training setting in which we learn task-specific\nword embeddings. We assume that we start with initial embeddings learned from\nunlabelled data and update them to learn task-specific embeddings for words in\nthe supervised training data. However, for new words in the test set, we must\nuse either their initial embeddings or a single unknown embedding, which often\nleads to errors. We address this by learning a neural network to map from\ninitial embeddings to the task-specific embedding space, via a multi-loss\nobjective function. The technique is general, but here we demonstrate its use\nfor improved dependency parsing (especially for sentences with\nout-of-vocabulary words), as well as for downstream improvements on sentiment\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.02755v2",
            "date": "2015-10-04 09:24:27+00:00",
            "title": "A Novel Approach to Document Classification using WordNet",
            "authors": [
                "Koushiki Sarkar",
                "Ritwika Law"
            ],
            "abstract": "Content based Document Classification is one of the biggest challenges in the\ncontext of free text mining. Current algorithms on document classifications\nmostly rely on cluster analysis based on bag-of-words approach. However that\nmethod is still being applied to many modern scientific dilemmas. It has\nestablished a strong presence in fields like economics and social science to\nmerit serious attention from the researchers. In this paper we would like to\npropose and explore an alternative grounded more securely on the dictionary\nclassification and correlatedness of words and phrases. It is expected that\napplication of our existing knowledge about the underlying classification\nstructure may lead to improvement of the classifier's performance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1510.02983v1",
            "date": "2015-10-10 21:22:00+00:00",
            "title": "OmniGraph: Rich Representation and Graph Kernel Learning",
            "authors": [
                "Boyi Xie",
                "Rebecca J. Passonneau"
            ],
            "abstract": "OmniGraph, a novel representation to support a range of NLP classification\ntasks, integrates lexical items, syntactic dependencies and frame semantic\nparses into graphs. Feature engineering is folded into the learning through\nconvolution graph kernel learning to explore different extents of the graph. A\nhigh-dimensional space of features includes individual nodes as well as complex\nsubgraphs. In experiments on a text-forecasting problem that predicts stock\nprice change from news for company mentions, OmniGraph beats several benchmarks\nbased on bag-of-words, syntactic dependencies, and semantic trees. The highly\nexpressive features OmniGraph discovers provide insights into the semantics\nacross distinct market sectors. To demonstrate the method's generality, we also\nreport its high performance results on a fine-grained sentiment corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03021v1",
            "date": "2015-10-11 07:45:01+00:00",
            "title": "Textual Analysis for Studying Chinese Historical Documents and Literary Novels",
            "authors": [
                "Chao-Lin Liu",
                "Guan-Tao Jin",
                "Hongsu Wang",
                "Qing-Feng Liu",
                "Wen-Huei Cheng",
                "Wei-Yun Chiu",
                "Richard Tzong-Han Tsai",
                "Yu-Chun Wang"
            ],
            "abstract": "We analyzed historical and literary documents in Chinese to gain insights\ninto research issues, and overview our studies which utilized four different\nsources of text materials in this paper. We investigated the history of\nconcepts and transliterated words in China with the Database for the Study of\nModern China Thought and Literature, which contains historical documents about\nChina between 1830 and 1930. We also attempted to disambiguate names that were\nshared by multiple government officers who served between 618 and 1912 and were\nrecorded in Chinese local gazetteers. To showcase the potentials and challenges\nof computer-assisted analysis of Chinese literatures, we explored some\ninteresting yet non-trivial questions about two of the Four Great Classical\nNovels of China: (1) Which monsters attempted to consume the Buddhist monk\nXuanzang in the Journey to the West (JTTW), which was published in the 16th\ncentury, (2) Which was the most powerful monster in JTTW, and (3) Which major\nrole smiled the most in the Dream of the Red Chamber, which was published in\nthe 18th century. Similar approaches can be applied to the analysis and study\nof modern documents, such as the newspaper articles published about the 228\nincident that occurred in 1947 in Taiwan.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.03602v1",
            "date": "2015-10-13 09:51:23+00:00",
            "title": "A language model based approach towards large scale and lightweight language identification systems",
            "authors": [
                "Brij Mohan Lal Srivastava",
                "Hari Krishna Vydana",
                "Anil Kumar Vuppala",
                "Manish Shrivastava"
            ],
            "abstract": "Multilingual spoken dialogue systems have gained prominence in the recent\npast necessitating the requirement for a front-end Language Identification\n(LID) system. Most of the existing LID systems rely on modeling the language\ndiscriminative information from low-level acoustic features. Due to the\nvariabilities of speech (speaker and emotional variabilities, etc.),\nlarge-scale LID systems developed using low-level acoustic features suffer from\na degradation in the performance. In this approach, we have attempted to model\nthe higher level language discriminative phonotactic information for developing\nan LID system. In this paper, the input speech signal is tokenized to phone\nsequences by using a language independent phone recognizer. The language\ndiscriminative phonotactic information in the obtained phone sequences are\nmodeled using statistical and recurrent neural network based language modeling\napproaches. As this approach, relies on higher level phonotactical information\nit is more robust to variabilities of speech. Proposed approach is\ncomputationally light weight, highly scalable and it can be used in complement\nwith the existing LID systems.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1510.04104v1",
            "date": "2015-09-28 15:21:00+00:00",
            "title": "A Preliminary Study on the Learning Informativeness of Data Subsets",
            "authors": [
                "Simon Kaltenbacher",
                "Nicholas H. Kirk",
                "Dongheui Lee"
            ],
            "abstract": "Estimating the internal state of a robotic system is complex: this is\nperformed from multiple heterogeneous sensor inputs and knowledge sources.\nDiscretization of such inputs is done to capture saliences, represented as\nsymbolic information, which often presents structure and recurrence. As these\nsequences are used to reason over complex scenarios, a more compact\nrepresentation would aid exactness of technical cognitive reasoning\ncapabilities, which are today constrained by computational complexity issues\nand fallback to representational heuristics or human intervention. Such\nproblems need to be addressed to ensure timely and meaningful human-robot\ninteraction. Our work is towards understanding the variability of learning\ninformativeness when training on subsets of a given input dataset. This is in\nview of reducing the training size while retaining the majority of the symbolic\nlearning potential. We prove the concept on human-written texts, and conjecture\nthis work will reduce training data size of sequential instructions, while\npreserving semantic relations, when gathering information from large remote\nsources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04780v2",
            "date": "2015-10-16 05:35:59+00:00",
            "title": "A Graph Traversal Based Approach to Answer Non-Aggregation Questions Over DBpedia",
            "authors": [
                "Chenhao Zhu",
                "Kan Ren",
                "Xuan Liu",
                "Haofen Wang",
                "Yiding Tian",
                "Yong Yu"
            ],
            "abstract": "We present a question answering system over DBpedia, filling the gap between\nuser information needs expressed in natural language and a structured query\ninterface expressed in SPARQL over the underlying knowledge base (KB). Given\nthe KB, our goal is to comprehend a natural language query and provide\ncorresponding accurate answers. Focusing on solving the non-aggregation\nquestions, in this paper, we construct a subgraph of the knowledge base from\nthe detected entities and propose a graph traversal method to solve both the\nsemantic item mapping problem and the disambiguation problem in a joint way.\nCompared with existing work, we simplify the process of query intention\nunderstanding and pay more attention to the answer path ranking. We evaluate\nour method on a non-aggregation question dataset and further on a complete\ndataset. Experimental results show that our method achieves best performance\ncompared with several state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.05198v4",
            "date": "2015-10-18 04:26:08+00:00",
            "title": "Learning multi-faceted representations of individuals from heterogeneous evidence using neural networks",
            "authors": [
                "Jiwei Li",
                "Alan Ritter",
                "Dan Jurafsky"
            ],
            "abstract": "Inferring latent attributes of people online is an important social computing\ntask, but requires integrating the many heterogeneous sources of information\navailable on the web. We propose learning individual representations of people\nusing neural nets to integrate rich linguistic and network evidence gathered\nfrom social media. The algorithm is able to combine diverse cues, such as the\ntext a person writes, their attributes (e.g. gender, employer, education,\nlocation) and social relations to other people. We show that by integrating\nboth textual and network evidence, these representations offer improved\nperformance at four important tasks in social media inference on Twitter:\npredicting (1) gender, (2) occupation, (3) location, and (4) friendships for\nusers. Our approach scales to large datasets and the learned representations\ncan be used as general features in and have the potential to benefit a large\nnumber of downstream tasks including link prediction, community detection, or\nprobabilistic reasoning over social networks.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1510.07385v1",
            "date": "2015-10-26 07:17:36+00:00",
            "title": "How to merge three different methods for information filtering ?",
            "authors": [
                "Jean-Val\u00e8re Cossu",
                "Ludovic Bonnefoy",
                "Xavier Bost",
                "Marc El B\u00e8ze"
            ],
            "abstract": "Twitter is now a gold marketing tool for entities concerned with online\nreputation. To automatically monitor online reputation of entities , systems\nhave to deal with ambiguous entity names, polarity detection and topic\ndetection. We propose three approaches to tackle the first issue: monitoring\nTwitter in order to find relevant tweets about a given entity. Evaluated within\nthe framework of the RepLab-2013 Filtering task, each of them has been shown\ncompetitive with state-of-the-art approaches. Mainly we investigate on how much\nmerging strategies may impact performances on a filtering task according to the\nevaluation measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07439v1",
            "date": "2015-10-26 11:12:59+00:00",
            "title": "Object Oriented Analysis using Natural Language Processing concepts: A Review",
            "authors": [
                "Abinash Tripathy",
                "Santanu Kumar Rath"
            ],
            "abstract": "The Software Development Life Cycle (SDLC) starts with eliciting requirements\nof the customers in the form of Software Requirement Specification (SRS). SRS\ndocument needed for software development is mostly written in Natural\nLanguage(NL) convenient for the client. From the SRS document only, the class\nname, its attributes and the functions incorporated in the body of the class\nare traced based on pre-knowledge of analyst. The paper intends to present a\nreview on Object Oriented (OO) analysis using Natural Language Processing (NLP)\ntechniques. This analysis can be manual where domain expert helps to generate\nthe required diagram or automated system, where the system generates the\nrequired diagram, from the input in the form of SRS.",
            "journal ref": "International Journal of Information Processing, vol. 9, no. 3,\n  pp. 38-50, 2015",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1505.00122v2",
            "date": "2015-05-01 08:44:45+00:00",
            "title": "Hierarchy of Scales in Language Dynamics",
            "authors": [
                "Richard A. Blythe"
            ],
            "abstract": "Methods and insights from statistical physics are finding an increasing\nvariety of applications where one seeks to understand the emergent properties\nof a complex interacting system. One such area concerns the dynamics of\nlanguage at a variety of levels of description, from the behaviour of\nindividual agents learning simple artificial languages from each other, up to\nchanges in the structure of languages shared by large groups of speakers over\nhistorical timescales. In this Colloquium, we survey a hierarchy of scales at\nwhich language and linguistic behaviour can be described, along with the main\nprogress in understanding that has been made at each of them---much of which\nhas come from the statistical physics community. We argue that future\ndevelopments may arise by linking the different levels of the hierarchy\ntogether in a more coherent fashion, in particular where this allows more\neffective use of rich empirical data sets.",
            "journal ref": "EPJB (2015) v88 295",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1505.00468v7",
            "date": "2015-05-03 20:07:39+00:00",
            "title": "VQA: Visual Question Answering",
            "authors": [
                "Aishwarya Agrawal",
                "Jiasen Lu",
                "Stanislaw Antol",
                "Margaret Mitchell",
                "C. Lawrence Zitnick",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "We propose the task of free-form and open-ended Visual Question Answering\n(VQA). Given an image and a natural language question about the image, the task\nis to provide an accurate natural language answer. Mirroring real-world\nscenarios, such as helping the visually impaired, both the questions and\nanswers are open-ended. Visual questions selectively target different areas of\nan image, including background details and underlying context. As a result, a\nsystem that succeeds at VQA typically needs a more detailed understanding of\nthe image and complex reasoning than a system producing generic image captions.\nMoreover, VQA is amenable to automatic evaluation, since many open-ended\nanswers contain only a few words or a closed set of answers that can be\nprovided in a multiple-choice format. We provide a dataset containing ~0.25M\nimages, ~0.76M questions, and ~10M answers (www.visualqa.org), and discuss the\ninformation it provides. Numerous baselines and methods for VQA are provided\nand compared with human performance. Our VQA demo is available on CloudCV\n(http://cloudcv.org/vqa).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.01072v1",
            "date": "2015-05-05 16:36:27+00:00",
            "title": "Mining Measured Information from Text",
            "authors": [
                "Arun S. Maiya",
                "Dale Visser",
                "Andrew Wan"
            ],
            "abstract": "We present an approach to extract measured information from text (e.g., a\n1370 degrees C melting point, a BMI greater than 29.9 kg/m^2 ). Such\nextractions are critically important across a wide range of domains -\nespecially those involving search and exploration of scientific and technical\ndocuments. We first propose a rule-based entity extractor to mine measured\nquantities (i.e., a numeric value paired with a measurement unit), which\nsupports a vast and comprehensive set of both common and obscure measurement\nunits. Our method is highly robust and can correctly recover valid measured\nquantities even when significant errors are introduced through the process of\nconverting document formats like PDF to plain text. Next, we describe an\napproach to extracting the properties being measured (e.g., the property \"pixel\npitch\" in the phrase \"a pixel pitch as high as 352 {\\mu}m\"). Finally, we\npresent MQSearch: the realization of a search engine with full support for\nmeasured information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.01393v1",
            "date": "2015-05-06 15:18:04+00:00",
            "title": "Mining Scientific Papers for Bibliometrics: a (very) Brief Survey of Methods and Tools",
            "authors": [
                "Iana Atanassova",
                "Marc Bertin",
                "Philipp Mayr"
            ],
            "abstract": "The Open Access movement in scientific publishing and search engines like\nGoogle Scholar have made scientific articles more broadly accessible. During\nthe last decade, the availability of scientific papers in full text has become\nmore and more widespread thanks to the growing number of publications on online\nplatforms such as ArXiv and CiteSeer. The efforts to provide articles in\nmachine-readable formats and the rise of Open Access publishing have resulted\nin a number of standardized formats for scientific papers (such as NLM-JATS,\nTEI, DocBook). Our aim is to stimulate research at the intersection of\nBibliometrics and Computational Linguistics in order to study the ways\nBibliometrics can benefit from large-scale text analytics and sense mining of\nscientific papers, thus exploring the interdisciplinarity of Bibliometrics and\nNatural Language Processing.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1505.01757v1",
            "date": "2015-05-07 16:03:02+00:00",
            "title": "Contextual Analysis for Middle Eastern Languages with Hidden Markov Models",
            "authors": [
                "Kazem Taghva"
            ],
            "abstract": "Displaying a document in Middle Eastern languages requires contextual\nanalysis due to different presentational forms for each character of the\nalphabet. The words of the document will be formed by the joining of the\ncorrect positional glyphs representing corresponding presentational forms of\nthe characters. A set of rules defines the joining of the glyphs. As usual,\nthese rules vary from language to language and are subject to interpretation by\nthe software developers.\n  In this paper, we propose a machine learning approach for contextual analysis\nbased on the first order Hidden Markov Model. We will design and build a model\nfor the Farsi language to exhibit this technology. The Farsi model achieves 94\n\\% accuracy with the training based on a short list of 89 Farsi vocabularies\nconsisting of 2780 Farsi characters.\n  The experiment can be easily extended to many languages including Arabic,\nUrdu, and Sindhi. Furthermore, the advantage of this approach is that the same\nsoftware can be used to perform contextual analysis without coding complex\nrules for each specific language. Of particular interest is that the languages\nwith fewer speakers can have greater representation on the web, since they are\ntypically ignored by software developers due to lack of financial incentives.",
            "journal ref": "International Journal on Natural Language Computing, vol. 4, No.\n  4, August 2015, pp. 1-11",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.03823v3",
            "date": "2015-05-14 18:15:49+00:00",
            "title": "Distant Supervision for Entity Linking",
            "authors": [
                "Miao Fan",
                "Qiang Zhou",
                "Thomas Fang Zheng"
            ],
            "abstract": "Entity linking is an indispensable operation of populating knowledge\nrepositories for information extraction. It studies on aligning a textual\nentity mention to its corresponding disambiguated entry in a knowledge\nrepository. In this paper, we propose a new paradigm named distantly supervised\nentity linking (DSEL), in the sense that the disambiguated entities that belong\nto a huge knowledge repository (Freebase) are automatically aligned to the\ncorresponding descriptive webpages (Wiki pages). In this way, a large scale of\nweakly labeled data can be generated without manual annotation and fed to a\nclassifier for linking more newly discovered entities. Compared with\ntraditional paradigms based on solo knowledge base, DSEL benefits more via\njointly leveraging the respective advantages of Freebase and Wikipedia.\nSpecifically, the proposed paradigm facilitates bridging the disambiguated\nlabels (Freebase) of entities and their textual descriptions (Wikipedia) for\nWeb-scale entities. Experiments conducted on a dataset of 140,000 items and\n60,000 features achieve a baseline F1-measure of 0.517. Furthermore, we analyze\nthe feature performance and improve the F1-measure to 0.545.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04313v1",
            "date": "2015-05-16 19:28:49+00:00",
            "title": "A type-theoretical approach to Universal Grammar",
            "authors": [
                "Erkki Luuk"
            ],
            "abstract": "The idea of Universal Grammar (UG) as the hypothetical linguistic structure\nshared by all human languages harkens back at least to the 13th century. The\nbest known modern elaborations of the idea are due to Chomsky. Following a\ndevastating critique from theoretical, typological and field linguistics, these\nelaborations, the idea of UG itself and the more general idea of language\nuniversals stand untenable and are largely abandoned. The proposal tackles the\nhypothetical contents of UG using dependent and polymorphic type theory in a\nframework very different from the Chomskyan ones. We introduce a type logic for\na precise, universal and parsimonious representation of natural language\nmorphosyntax and compositional semantics. The logic handles grammatical\nambiguity (with polymorphic types), selectional restrictions and diverse kinds\nof anaphora (with dependent types), and features a partly universal set of\nmorphosyntactic types (by the Curry-Howard isomorphism).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04657v2",
            "date": "2015-05-18 14:25:03+00:00",
            "title": "Mining User Opinions in Mobile App Reviews: A Keyword-based Approach",
            "authors": [
                "Phong Minh Vu",
                "Tam The Nguyen",
                "Hung Viet Pham",
                "Tung Thanh Nguyen"
            ],
            "abstract": "User reviews of mobile apps often contain complaints or suggestions which are\nvaluable for app developers to improve user experience and satisfaction.\nHowever, due to the large volume and noisy-nature of those reviews, manually\nanalyzing them for useful opinions is inherently challenging. To address this\nproblem, we propose MARK, a keyword-based framework for semi-automated review\nanalysis. MARK allows an analyst describing his interests in one or some mobile\napps by a set of keywords. It then finds and lists the reviews most relevant to\nthose keywords for further analysis. It can also draw the trends over time of\nthose keywords and detect their sudden changes, which might indicate the\noccurrences of serious issues. To help analysts describe their interests more\neffectively, MARK can automatically extract keywords from raw reviews and rank\nthem by their associations with negative reviews. In addition, based on a\nvector-based semantic representation of keywords, MARK can divide a large set\nof keywords into more cohesive subsets, or suggest keywords similar to the\nselected ones.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1505.04870v4",
            "date": "2015-05-19 04:46:03+00:00",
            "title": "Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models",
            "authors": [
                "Bryan A. Plummer",
                "Liwei Wang",
                "Chris M. Cervantes",
                "Juan C. Caicedo",
                "Julia Hockenmaier",
                "Svetlana Lazebnik"
            ],
            "abstract": "The Flickr30k dataset has become a standard benchmark for sentence-based\nimage description. This paper presents Flickr30k Entities, which augments the\n158k captions from Flickr30k with 244k coreference chains, linking mentions of\nthe same entities across different captions for the same image, and associating\nthem with 276k manually annotated bounding boxes. Such annotations are\nessential for continued progress in automatic image description and grounded\nlanguage understanding. They enable us to define a new benchmark for\nlocalization of textual entity mentions in an image. We present a strong\nbaseline for this task that combines an image-text embedding, detectors for\ncommon objects, a color classifier, and a bias towards selecting larger\nobjects. While our baseline rivals in accuracy more complex state-of-the-art\nmodels, we show that its gains cannot be easily parlayed into improvements on\nsuch tasks as image-sentence retrieval, thus underlining the limitations of\ncurrent methods and the need for further research.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1505.06027v2",
            "date": "2015-05-22 11:08:39+00:00",
            "title": "Weakly-Supervised Alignment of Video With Text",
            "authors": [
                "Piotr Bojanowski",
                "R\u00e9mi Lajugie",
                "Edouard Grave",
                "Francis Bach",
                "Ivan Laptev",
                "Jean Ponce",
                "Cordelia Schmid"
            ],
            "abstract": "Suppose that we are given a set of videos, along with natural language\ndescriptions in the form of multiple sentences (e.g., manual annotations, movie\nscripts, sport summaries etc.), and that these sentences appear in the same\ntemporal order as their visual counterparts. We propose in this paper a method\nfor aligning the two modalities, i.e., automatically providing a time stamp for\nevery sentence. Given vectorial features for both video and text, we propose to\ncast this task as a temporal assignment problem, with an implicit linear\nmapping between the two feature modalities. We formulate this problem as an\ninteger quadratic program, and solve its continuous convex relaxation using an\nefficient conditional gradient algorithm. Several rounding procedures are\nproposed to construct the final integer solution. After demonstrating\nsignificant improvements over the state of the art on the related task of\naligning video with symbolic labels [7], we evaluate our method on a\nchallenging dataset of videos with associated textual descriptions [36], using\nboth bag-of-words and continuous representations for text.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1505.06169v1",
            "date": "2015-05-22 18:28:21+00:00",
            "title": "Learning Dynamic Feature Selection for Fast Sequential Prediction",
            "authors": [
                "Emma Strubell",
                "Luke Vilnis",
                "Kate Silverstein",
                "Andrew McCallum"
            ],
            "abstract": "We present paired learning and inference algorithms for significantly\nreducing computation and increasing speed of the vector dot products in the\nclassifiers that are at the heart of many NLP components. This is accomplished\nby partitioning the features into a sequence of templates which are ordered\nsuch that high confidence can often be reached using only a small fraction of\nall features. Parameter estimation is arranged to maximize accuracy and early\nconfidence in this sequence. Our approach is simpler and better suited to NLP\nthan other related cascade methods. We present experiments in left-to-right\npart-of-speech tagging, named entity recognition, and transition-based\ndependency parsing. On the typical benchmarking datasets we can preserve POS\ntagging accuracy above 97% and parsing LAS above 88.5% both with over a\nfive-fold reduction in run-time, and NER F1 above 88 with more than 2x increase\nin speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06256v1",
            "date": "2015-05-23 00:32:33+00:00",
            "title": "Exposing ambiguities in a relation-extraction gold standard with crowdsourcing",
            "authors": [
                "Tong Shu Li",
                "Benjamin M. Good",
                "Andrew I. Su"
            ],
            "abstract": "Semantic relation extraction is one of the frontiers of biomedical natural\nlanguage processing research. Gold standards are key tools for advancing this\nresearch. It is challenging to generate these standards because of the high\ncost of expert time and the difficulty in establishing agreement between\nannotators. We implemented and evaluated a microtask crowdsourcing approach\nthat can produce a gold standard for extracting drug-disease relations. The\naggregated crowd judgment agreed with expert annotations from a pre-existing\ncorpus on 43 of 60 sentences tested. The levels of crowd agreement varied in a\nsimilar manner to the levels of agreement among the original expert annotators.\nThis work rein-forces the power of crowdsourcing in the process of assembling\ngold standards for relation extraction. Further, it high-lights the importance\nof exposing the levels of agreement between human annotators, expert or crowd,\nin gold standard corpora as these are reproducible signals indicating\nambiguities in the data or in the annotation guidelines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06289v2",
            "date": "2015-05-23 08:32:11+00:00",
            "title": "Text to 3D Scene Generation with Rich Lexical Grounding",
            "authors": [
                "Angel Chang",
                "Will Monroe",
                "Manolis Savva",
                "Christopher Potts",
                "Christopher D. Manning"
            ],
            "abstract": "The ability to map descriptions of scenes to 3D geometric representations has\nmany applications in areas such as art, education, and robotics. However, prior\nwork on the text to 3D scene generation task has used manually specified object\ncategories and language that identifies them. We introduce a dataset of 3D\nscenes annotated with natural language descriptions and learn from this data\nhow to ground textual descriptions to physical objects. Our method successfully\ngrounds a variety of lexical terms to concrete referents, and we show\nquantitatively that our method improves 3D scene generation over previous work\nusing purely rule-based methods. We evaluate the fidelity and plausibility of\n3D scenes generated with our grounding approach through human judgments. To\nease evaluation on this task, we also introduce an automated metric that\nstrongly correlates with human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06750v2",
            "date": "2015-05-25 20:33:13+00:00",
            "title": "Reply to Garcia et al.: Common mistakes in measuring frequency dependent word characteristics",
            "authors": [
                "P. S. Dodds",
                "E. M. Clark",
                "S. Desu",
                "M. R. Frank",
                "A. J. Reagan",
                "J. R. Williams",
                "L. Mitchell",
                "K. D. Harris",
                "I. M. Kloumann",
                "J. P. Bagrow",
                "K. Megerdoomian",
                "M. T. McMahon",
                "B. F. Tivnan",
                "C. M. Danforth"
            ],
            "abstract": "We demonstrate that the concerns expressed by Garcia et al. are misplaced,\ndue to (1) a misreading of our findings in [1]; (2) a widespread failure to\nexamine and present words in support of asserted summary quantities based on\nword usage frequencies; and (3) a range of misconceptions about word usage\nfrequency, word rank, and expert-constructed word lists. In particular, we show\nthat the English component of our study compares well statistically with two\nrelated surveys, that no survey design influence is apparent, and that\nestimates of measurement error do not explain the positivity biases reported in\nour work and that of others. We further demonstrate that for the frequency\ndependence of positivity---of which we explored the nuances in great detail in\n[1]---Garcia et al. did not perform a reanalysis of our data---they instead\ncarried out an analysis of a different, statistically improper data set and\nintroduced a nonlinearity before performing linear regression.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1505.07302v4",
            "date": "2015-05-27 13:17:35+00:00",
            "title": "Unveiling the Political Agenda of the European Parliament Plenary: A Topical Analysis",
            "authors": [
                "Derek Greene",
                "James P. Cross"
            ],
            "abstract": "This study analyzes political interactions in the European Parliament (EP) by\nconsidering how the political agenda of the plenary sessions has evolved over\ntime and the manner in which Members of the European Parliament (MEPs) have\nreacted to external and internal stimuli when making Parliamentary speeches. It\ndoes so by considering the context in which speeches are made, and the content\nof those speeches. To detect latent themes in legislative speeches over time,\nspeech content is analyzed using a new dynamic topic modeling method, based on\ntwo layers of matrix factorization. This method is applied to a new corpus of\nall English language legislative speeches in the EP plenary from the period\n1999-2014. Our findings suggest that the political agenda of the EP has evolved\nsignificantly over time, is impacted upon by the committee structure of the\nParliament, and reacts to exogenous events such as EU Treaty referenda and the\nemergence of the Euro-crisis have a significant impact on what is being\ndiscussed in Parliament.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.00685v2",
            "date": "2015-09-02 13:20:40+00:00",
            "title": "A Neural Attention Model for Abstractive Sentence Summarization",
            "authors": [
                "Alexander M. Rush",
                "Sumit Chopra",
                "Jason Weston"
            ],
            "abstract": "Summarization based on text extraction is inherently limited, but\ngeneration-style abstractive methods have proven challenging to build. In this\nwork, we propose a fully data-driven approach to abstractive sentence\nsummarization. Our method utilizes a local attention-based model that generates\neach word of the summary conditioned on the input sentence. While the model is\nstructurally simple, it can easily be trained end-to-end and scales to a large\namount of training data. The model shows significant performance gains on the\nDUC-2004 shared task compared with several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01023v1",
            "date": "2015-09-03 10:21:16+00:00",
            "title": "Generating Weather Forecast Texts with Case Based Reasoning",
            "authors": [
                "Ibrahim Adeyanju"
            ],
            "abstract": "Several techniques have been used to generate weather forecast texts. In this\npaper, case based reasoning (CBR) is proposed for weather forecast text\ngeneration because similar weather conditions occur over time and should have\nsimilar forecast texts. CBR-METEO, a system for generating weather forecast\ntexts was developed using a generic framework (jCOLIBRI) which provides modules\nfor the standard components of the CBR architecture. The advantage in a CBR\napproach is that systems can be built in minimal time with far less human\neffort after initial consultation with experts. The approach depends heavily on\nthe goodness of the retrieval and revision components of the CBR process. We\nevaluated CBRMETEO with NIST, an automated metric which has been shown to\ncorrelate well with human judgements for this domain. The system shows\ncomparable performance with other NLG systems that perform the same task.",
            "journal ref": "International Journal of Computer Applications 45(10) (2012) 35-40",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1509.01599v2",
            "date": "2015-09-04 20:28:12+00:00",
            "title": "Better Document-level Sentiment Analysis from RST Discourse Parsing",
            "authors": [
                "Parminder Bhatia",
                "Yangfeng Ji",
                "Jacob Eisenstein"
            ],
            "abstract": "Discourse structure is the hidden link between surface features and\ndocument-level properties, such as sentiment polarity. We show that the\ndiscourse analyses produced by Rhetorical Structure Theory (RST) parsers can\nimprove document-level sentiment analysis, via composition of local information\nup the discourse tree. First, we show that reweighting discourse units\naccording to their position in a dependency representation of the rhetorical\nstructure can yield substantial improvements on lexicon-based sentiment\nanalysis. Next, we present a recursive neural network over the RST structure,\nwhich offers significant improvements over classification-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01626v3",
            "date": "2015-09-04 22:31:53+00:00",
            "title": "Character-level Convolutional Networks for Text Classification",
            "authors": [
                "Xiang Zhang",
                "Junbo Zhao",
                "Yann LeCun"
            ],
            "abstract": "This article offers an empirical exploration on the use of character-level\nconvolutional networks (ConvNets) for text classification. We constructed\nseveral large-scale datasets to show that character-level convolutional\nnetworks could achieve state-of-the-art or competitive results. Comparisons are\noffered against traditional models such as bag of words, n-grams and their\nTFIDF variants, and deep learning models such as word-based ConvNets and\nrecurrent neural networks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1509.01865v1",
            "date": "2015-09-06 23:16:45+00:00",
            "title": "A Hybrid Approach to Domain-Specific Entity Linking",
            "authors": [
                "Alex Olieman",
                "Jaap Kamps",
                "Maarten Marx",
                "Arjan Nusselder"
            ],
            "abstract": "The current state-of-the-art Entity Linking (EL) systems are geared towards\ncorpora that are as heterogeneous as the Web, and therefore perform\nsub-optimally on domain-specific corpora. A key open problem is how to\nconstruct effective EL systems for specific domains, as knowledge of the local\ncontext should in principle increase, rather than decrease, effectiveness. In\nthis paper we propose the hybrid use of simple specialist linkers in\ncombination with an existing generalist system to address this problem. Our\nmain findings are the following. First, we construct a new reusable benchmark\nfor EL on a corpus of domain-specific conversations. Second, we test the\nperformance of a range of approaches under the same conditions, and show that\nspecialist linkers obtain high precision in isolation, and high recall when\ncombined with generalist linkers. Hence, we can effectively exploit local\ncontext and get the best of both worlds.",
            "journal ref": "Proc. Posters and Demos track of 11th Int. Conf. on Semantic\n  Systems (2015) 55-58",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1509.04556v4",
            "date": "2015-09-10 18:08:13+00:00",
            "title": "On the evolution of word usage of classical Chinese poetry",
            "authors": [
                "Liang Liu",
                "Lili Yu"
            ],
            "abstract": "The hierarchy of classical Chinese poetry has been broadly acknowledged by a\nnumber of studies in Chinese literature. However, quantitative investigations\nabout the evolutionary linkages of classical Chinese poetry are limited. The\nprimary goal of this study is to provide quantitative evidence of the\nevolutionary linkages, with emphasis on character usage, among different period\ngenres of classical Chinese poetry. Specifically, various statistical analyses\nare performed to find and compare the patterns of character usage in the poems\nof nine period genres, including shi jing, chu ci, Han shi , Jin shi, Tang shi,\nSong shi, Yuan shi, Ming shi, and Qing shi. The result of analysis indicates\nthat each of nine period genres has unique patterns of character usage, with\nsome Chinese characters that are preferably used in the poems of a particular\nperiod genre. The analysis on the general pattern of character preference\nimplies a decreasing trend in the use of Chinese characters that rarely occur\nin modern Chinese literature along the timeline of dynastic types of classical\nChinese poetry. The phylogenetic analysis based on the distance matrix suggests\nthat the evolutionary linkages of different types of classical Chinese poetry\nare congruent with their chronological order, suggesting that character\nfrequencies contain phylogenetic information that is useful for inferring\nevolutionary linkages among various types of classical Chinese poetry. The\nestimated phylogenetic tree identifies four groups (shi jing, chu ci), (Han\nshi, Jin shi), (Tang shi, Song shi, Yuan shi), and (Ming shi, Qing shi). The\nstatistical analyses conducted in this study can be generalized to analyze the\ndata sets of general Chinese literature. Such analyses can provide quantitative\ninsights about the evolutionary linkages of general Chinese literature.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1509.04811v1",
            "date": "2015-09-16 05:00:59+00:00",
            "title": "amLite: Amharic Transliteration Using Key Map Dictionary",
            "authors": [
                "Tadele Tedla"
            ],
            "abstract": "amLite is a framework developed to map ASCII transliterated Amharic texts\nback to the original Amharic letter texts. The aim of such a framework is to\nmake existing Amharic linguistic data consistent and interoperable among\nresearchers. For achieving the objective, a key map dictionary is constructed\nusing the possible ASCII combinations actively in use for transliterating\nAmharic letters; and a mapping of the combinations to the corresponding Amharic\nletters is done. The mapping is then used to replace the Amharic linguistic\ntext back to form the original Amharic letters text. The framework indicated\n97.7, 99.7 and 98.4 percentage accuracy on converting the three sample random\ntest data. It is; however, possible to improve the accuracy of the framework by\nadding an exception to the implementation of the algorithm, or by preprocessing\nthe input text prior to conversion. This paper outlined the rationales behind\nthe need for developing the framework and the processes undertaken in the\ndevelopment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05209v1",
            "date": "2015-09-17 11:20:35+00:00",
            "title": "Extraction of evidence tables from abstracts of randomized clinical trials using a maximum entropy classifier and global constraints",
            "authors": [
                "Antonio Trenta",
                "Anthony Hunter",
                "Sebastian Riedel"
            ],
            "abstract": "Systematic use of the published results of randomized clinical trials is\nincreasingly important in evidence-based medicine. In order to collate and\nanalyze the results from potentially numerous trials, evidence tables are used\nto represent trials concerning a set of interventions of interest. An evidence\ntable has columns for the patient group, for each of the interventions being\ncompared, for the criterion for the comparison (e.g. proportion who survived\nafter 5 years from treatment), and for each of the results. Currently, it is a\nlabour-intensive activity to read each published paper and extract the\ninformation for each field in an evidence table. There have been some NLP\nstudies investigating how some of the features from papers can be extracted, or\nat least the relevant sentences identified. However, there is a lack of an NLP\nsystem for the systematic extraction of each item of information required for\nan evidence table. We address this need by a combination of a maximum entropy\nclassifier, and integer linear programming. We use the later to handle\nconstraints on what is an acceptable classification of the features to be\nextracted. With experimental results, we demonstrate substantial advantages in\nusing global constraints (such as the features describing the patient group,\nand the interventions, must occur before the features describing the results of\nthe comparison).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05736v1",
            "date": "2015-09-18 18:19:48+00:00",
            "title": "Building a Pilot Software Quality-in-Use Benchmark Dataset",
            "authors": [
                "Issa Atoum",
                "Chih How Bong",
                "Narayanan Kulathuramaiyer"
            ],
            "abstract": "Prepared domain specific datasets plays an important role to supervised\nlearning approaches. In this article a new sentence dataset for software\nquality-in-use is proposed. Three experts were chosen to annotate the data\nusing a proposed annotation scheme. Then the data were reconciled in a (no\nmatch eliminate) process to reduce bias. The Kappa, k statistics revealed an\nacceptable level of agreement; moderate to substantial agreement between the\nexperts. The built data can be used to evaluate software quality-in-use models\nin sentiment analysis models. Moreover, the annotation scheme can be used to\nextend the current dataset.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1509.06103v1",
            "date": "2015-09-21 03:37:11+00:00",
            "title": "Noise Robust IOA/CAS Speech Separation and Recognition System For The Third 'CHIME' Challenge",
            "authors": [
                "Xiaofei Wang",
                "Chao Wu",
                "Pengyuan Zhang",
                "Ziteng Wang",
                "Yong Liu",
                "Xu Li",
                "Qiang Fu",
                "Yonghong Yan"
            ],
            "abstract": "This paper presents the contribution to the third 'CHiME' speech separation\nand recognition challenge including both front-end signal processing and\nback-end speech recognition. In the front-end, Multi-channel Wiener filter\n(MWF) is designed to achieve background noise reduction. Different from\ntraditional MWF, optimized parameter for the tradeoff between noise reduction\nand target signal distortion is built according to the desired noise reduction\nlevel. In the back-end, several techniques are taken advantage to improve the\nnoisy Automatic Speech Recognition (ASR) performance including Deep Neural\nNetwork (DNN), Convolutional Neural Network (CNN) and Long short-term memory\n(LSTM) using medium vocabulary, Lattice rescoring with a big vocabulary\nlanguage model finite state transducer, and ROVER scheme. Experimental results\nshow the proposed system combining front-end and back-end is effective to\nimprove the ASR performance.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1509.06585v3",
            "date": "2015-09-22 13:12:34+00:00",
            "title": "A Review of Features for the Discrimination of Twitter Users: Application to the Prediction of Offline Influence",
            "authors": [
                "Jean-Val\u00e8re Cossu",
                "Vincent Labatut",
                "Nicolas Dugu\u00e9"
            ],
            "abstract": "Many works related to Twitter aim at characterizing its users in some way:\nrole on the service (spammers, bots, organizations, etc.), nature of the user\n(socio-professional category, age, etc.), topics of interest , and others.\nHowever, for a given user classification problem, it is very difficult to\nselect a set of appropriate features, because the many features described in\nthe literature are very heterogeneous, with name overlaps and collisions, and\nnumerous very close variants. In this article, we review a wide range of such\nfeatures. In order to present a clear state-of-the-art description, we unify\ntheir names, definitions and relationships, and we propose a new, neutral,\ntypology. We then illustrate the interest of our review by applying a selection\nof these features to the offline influence detection problem. This task\nconsists in identifying users which are influential in real-life, based on\ntheir Twitter account and related data. We show that most features deemed\nefficient to predict online influence, such as the numbers of retweets and\nfollowers, are not relevant to this problem. However, We propose several\ncontent-based approaches to label Twitter users as Influencers or not. We also\nrank them according to a predicted influence level. Our proposals are evaluated\nover the CLEF RepLab 2014 dataset, and outmatch state-of-the-art methods.",
            "journal ref": "Social Network Analysis and Mining, Springer, 2016, 6 (1), pp.25",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07211v1",
            "date": "2015-09-24 02:16:11+00:00",
            "title": "Noise-Robust ASR for the third 'CHiME' Challenge Exploiting Time-Frequency Masking based Multi-Channel Speech Enhancement and Recurrent Neural Network",
            "authors": [
                "Zaihu Pang",
                "Fengyun Zhu"
            ],
            "abstract": "In this paper, the Lingban entry to the third 'CHiME' speech separation and\nrecognition challenge is presented. A time-frequency masking based speech\nenhancement front-end is proposed to suppress the environmental noise utilizing\nmulti-channel coherence and spatial cues. The state-of-the-art speech\nrecognition techniques, namely recurrent neural network based acoustic and\nlanguage modeling, state space minimum Bayes risk based discriminative acoustic\nmodeling, and i-vector based acoustic condition modeling, are carefully\nintegrated into the speech recognition back-end. To further improve the system\nperformance by fully exploiting the advantages of different technologies, the\nfinal recognition results are obtained by lattice combination and rescoring.\nEvaluations carried out on the official dataset prove the effectiveness of the\nproposed systems. Comparing with the best baseline result, the proposed system\nobtains consistent improvements with over 57% relative word error rate\nreduction on the real-data test set.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1509.08967v2",
            "date": "2015-09-29 22:28:11+00:00",
            "title": "Very Deep Multilingual Convolutional Neural Networks for LVCSR",
            "authors": [
                "Tom Sercu",
                "Christian Puhrsch",
                "Brian Kingsbury",
                "Yann LeCun"
            ],
            "abstract": "Convolutional neural networks (CNNs) are a standard component of many current\nstate-of-the-art Large Vocabulary Continuous Speech Recognition (LVCSR)\nsystems. However, CNNs in LVCSR have not kept pace with recent advances in\nother domains where deeper neural networks provide superior performance. In\nthis paper we propose a number of architectural advances in CNNs for LVCSR.\nFirst, we introduce a very deep convolutional network architecture with up to\n14 weight layers. There are multiple convolutional layers before each pooling\nlayer, with small 3x3 kernels, inspired by the VGG Imagenet 2014 architecture.\nThen, we introduce multilingual CNNs with multiple untied layers. Finally, we\nintroduce multi-scale input features aimed at exploiting more context at\nnegligible computational cost. We evaluate the improvements first on a Babel\ntask for low resource speech recognition, obtaining an absolute 5.77% WER\nimprovement over the baseline PLP DNN by training our CNN on the combined data\nof six different languages. We then evaluate the very deep CNNs on the Hub5'00\nbenchmark (using the 262 hours of SWB-1 training data) achieving a word error\nrate of 11.8% after cross-entropy training, a 1.4% WER improvement (10.6%\nrelative) over the best published CNN result so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.09088v1",
            "date": "2015-09-30 09:13:00+00:00",
            "title": "Enhanced Bilingual Evaluation Understudy",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation\ntechnique for statistical machine translation to make it more adjustable and\nrobust. We intend to adapt it to resemble human evaluation more. We perform\nexperiments to evaluate the performance of our technique against the primary\nexisting evaluation methods. We describe and show the improvements it makes\nover existing methods as well as correlation to them. When human translators\ntranslate a text, they often use synonyms, different word orders or style, and\nother similar variations. We propose an SMT evaluation technique that enhances\nthe BLEU metric to consider variations such as those.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.09090v1",
            "date": "2015-09-30 09:20:27+00:00",
            "title": "Real-Time Statistical Speech Translation",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "This research investigates the Statistical Machine Translation approaches to\ntranslate speech in real time automatically. Such systems can be used in a\npipeline with speech recognition and synthesis software in order to produce a\nreal-time voice communication system between foreigners. We obtained three main\ndata sets from spoken proceedings that represent three different types of human\nspeech. TED, Europarl, and OPUS parallel text corpora were used as the basis\nfor training of language models, for developmental tuning and testing of the\ntranslation system. We also conducted experiments involving part of speech\ntagging, compound splitting, linear language model interpolation, TrueCasing\nand morphosyntactic analysis. We evaluated the effects of variety of data\npreparations on the translation results using the BLEU, NIST, METEOR and TER\nmetrics and tried to give answer which metric is most suitable for PL-EN\nlanguage pair.",
            "journal ref": "Advances in Intelligent Systems and Computing volume 275,\n  p.107-114, Publisher: Springer, ISSN 2194-5357, ISBN 978-3-319-05950-1, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.09093v1",
            "date": "2015-09-30 09:29:51+00:00",
            "title": "A Sentence Meaning Based Alignment Method for Parallel Text Corpora Preparation",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "Text alignment is crucial to the accuracy of Machine Translation (MT)\nsystems, some NLP tools or any other text processing tasks requiring bilingual\ndata. This research proposes a language independent sentence alignment approach\nbased on Polish (not position-sensitive language) to English experiments. This\nalignment approach was developed on the TED Talks corpus, but can be used for\nany text domain or language pair. The proposed approach implements various\nheuristics for sentence recognition. Some of them value synonyms and semantic\ntext structure analysis as a part of additional information. Minimization of\ndata loss was ensured. The solution is compared to other sentence alignment\nimplementations. Also an improvement in MT system score with text processed\nwith described tool is shown.",
            "journal ref": "Advances in Intelligent Systems and Computing volume 275,\n  p.107-114, Publisher: Springer, ISSN 2194-5357, ISBN 978-3-319-05950-1, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.09097v1",
            "date": "2015-09-30 09:35:38+00:00",
            "title": "Polish - English Speech Statistical Machine Translation Systems for the IWSLT 2013",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "This research explores the effects of various training settings from Polish\nto English Statistical Machine Translation system for spoken language. Various\nelements of the TED parallel text corpora for the IWSLT 2013 evaluation\ncampaign were used as the basis for training of language models, and for\ndevelopment, tuning and testing of the translation system. The BLEU, NIST,\nMETEOR and TER metrics were used to evaluate the effects of data preparations\non translation results. Our experiments included systems, which use stems and\nmorphological information on Polish words. We also conducted a deep analysis of\nprovided Polish data as preparatory work for the automatic data correction and\ncleaning phase.",
            "journal ref": "Proceedings of the 10th International Workshop on Spoken Language\n  Translation, Heidelberg, Germany, p. 113-119, 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00027v1",
            "date": "2015-10-28 10:10:51+00:00",
            "title": "CBAS: context based arabic stemmer",
            "authors": [
                "Mahmoud El-Defrawy",
                "Yasser El-Sonbaty",
                "Nahla A. Belal"
            ],
            "abstract": "Arabic morphology encapsulates many valuable features such as word root.\nArabic roots are being utilized for many tasks; the process of extracting a\nword root is referred to as stemming. Stemming is an essential part of most\nNatural Language Processing tasks, especially for derivative languages such as\nArabic. However, stemming is faced with the problem of ambiguity, where two or\nmore roots could be extracted from the same word. On the other hand,\ndistributional semantics is a powerful co-occurrence model. It captures the\nmeaning of a word based on its context. In this paper, a distributional\nsemantics model utilizing Smoothed Pointwise Mutual Information (SPMI) is\nconstructed to investigate its effectiveness on the stemming analysis task. It\nshowed an accuracy of 81.5%, with a at least 9.4% improvement over other\nstemmers.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  4, No.3, June 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.00325v2",
            "date": "2015-04-01 18:13:43+00:00",
            "title": "Microsoft COCO Captions: Data Collection and Evaluation Server",
            "authors": [
                "Xinlei Chen",
                "Hao Fang",
                "Tsung-Yi Lin",
                "Ramakrishna Vedantam",
                "Saurabh Gupta",
                "Piotr Dollar",
                "C. Lawrence Zitnick"
            ],
            "abstract": "In this paper we describe the Microsoft COCO Caption dataset and evaluation\nserver. When completed, the dataset will contain over one and a half million\ncaptions describing over 330,000 images. For the training and validation\nimages, five independent human generated captions will be provided. To ensure\nconsistency in evaluation of automatic caption generation algorithms, an\nevaluation server is used. The evaluation server receives candidate captions\nand scores them using several popular metrics, including BLEU, METEOR, ROUGE\nand CIDEr. Instructions for using the evaluation server are provided.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1504.01684v1",
            "date": "2015-04-07 17:50:31+00:00",
            "title": "Large Margin Nearest Neighbor Embedding for Knowledge Representation",
            "authors": [
                "Miao Fan",
                "Qiang Zhou",
                "Thomas Fang Zheng",
                "Ralph Grishman"
            ],
            "abstract": "Traditional way of storing facts in triplets ({\\it head\\_entity, relation,\ntail\\_entity}), abbreviated as ({\\it h, r, t}), makes the knowledge intuitively\ndisplayed and easily acquired by mankind, but hardly computed or even reasoned\nby AI machines. Inspired by the success in applying {\\it Distributed\nRepresentations} to AI-related fields, recent studies expect to represent each\nentity and relation with a unique low-dimensional embedding, which is different\nfrom the symbolic and atomic framework of displaying knowledge in triplets. In\nthis way, the knowledge computing and reasoning can be essentially facilitated\nby means of a simple {\\it vector calculation}, i.e. ${\\bf h} + {\\bf r} \\approx\n{\\bf t}$. We thus contribute an effective model to learn better embeddings\nsatisfying the formula by pulling the positive tail entities ${\\bf t^{+}}$ to\nget together and close to {\\bf h} + {\\bf r} ({\\it Nearest Neighbor}), and\nsimultaneously pushing the negatives ${\\bf t^{-}}$ away from the positives\n${\\bf t^{+}}$ via keeping a {\\it Large Margin}. We also design a corresponding\nlearning algorithm to efficiently find the optimal solution based on {\\it\nStochastic Gradient Descent} in iterative fashion. Quantitative experiments\nillustrate that our approach can achieve the state-of-the-art performance,\ncompared with several latest methods on some benchmark datasets for two\nclassical applications, i.e. {\\it Link prediction} and {\\it Triplet\nclassification}. Moreover, we analyze the parameter complexities among all the\nevaluated models, and analytical results indicate that our model needs fewer\ncomputational resources on outperforming the other methods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.02059v1",
            "date": "2015-04-08 18:12:07+00:00",
            "title": "Supporting Language Learners with the Meanings Of Closed Class Items",
            "authors": [
                "Hayat Alrefaie",
                "Allan Ramsay"
            ],
            "abstract": "The process of language learning involves the mastery of countless tasks:\nmaking the constituent sounds of the language being learned, learning the\ngrammatical patterns, and acquiring the requisite vocabulary for reception and\nproduction. While a plethora of computational tools exist to facilitate the\nfirst and second of these tasks, a number of challenges arise with respect to\nenabling the third. This paper describes a tool that has been designed to\nsupport language learners with the challenge of understanding the use of\nclosed-class lexical items. The process of learning the Arabic for office is\n(mktb) is relatively simple and should be possible by means of simple\nrepetition of the word. However, it is much more difficult to learn and\ncorrectly use the Arabic equivalent of the word on. The current paper describes\na mechanism for the delivery of diagnostic information regarding specific\nlexical examples, with the aim of clearly demonstrating why a particular\ntranslation of a given closed-class item may be appropriate in certain\nsituations but not others, thereby helping learners to understand and use the\nterm correctly.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.03068v2",
            "date": "2015-04-13 05:53:59+00:00",
            "title": "Review Mining for Feature Based Opinion Summarization and Visualization",
            "authors": [
                "Ahmad Kamal"
            ],
            "abstract": "The application and usage of opinion mining, especially for business\nintelligence, product recommendation, targeted marketing etc. have fascinated\nmany research attentions around the globe. Various research efforts attempted\nto mine opinions from customer reviews at different levels of granularity,\nincluding word-, sentence-, and document-level. However, development of a fully\nautomatic opinion mining and sentiment analysis system is still elusive. Though\nthe development of opinion mining and sentiment analysis systems are getting\nmomentum, most of them attempt to perform document-level sentiment analysis,\nclassifying a review document as positive, negative, or neutral. Such\ndocument-level opinion mining approaches fail to provide insight about users\nsentiment on individual features of a product or service. Therefore, it seems\nto be a great help for both customers and manufacturers, if the reviews could\nbe processed at a finer-grained level and presented in a summarized form\nthrough some visual means, highlighting individual features of a product and\nusers sentiment expressed over them. In this paper, the design of a unified\nopinion mining and sentiment analysis framework is presented at the\nintersection of both machine learning and natural language processing\napproaches. Also, design of a novel feature-level review summarization scheme\nis proposed to visualize mined features, opinions and their polarity values in\na comprehendible way.",
            "journal ref": "International Journal of Computer Applications, 119(17), 2015, pp.\n  6-13",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1504.03608v1",
            "date": "2015-04-14 16:23:30+00:00",
            "title": "A data-based classification of Slavic languages: Indices of qualitative variation applied to grapheme frequencies",
            "authors": [
                "Michaela Koscov\u00e1",
                "J\u00e1n Macutek",
                "Emmerich Kelih"
            ],
            "abstract": "The Ord's graph is a simple graphical method for displaying frequency\ndistributions of data or theoretical distributions in the two-dimensional\nplane. Its coordinates are proportions of the first three moments, either\nempirical or theoretical ones. A modification of the Ord's graph based on\nproportions of indices of qualitative variation is presented. Such a\nmodification makes the graph applicable also to data of categorical character.\nIn addition, the indices are normalized with values between 0 and 1, which\nenables comparing data files divided into different numbers of categories. Both\nthe original and the new graph are used to display grapheme frequencies in\neleven Slavic languages. As the original Ord's graph requires an assignment of\nnumbers to the categories, graphemes were ordered decreasingly according to\ntheir frequencies. Data were taken from parallel corpora, i.e., we work with\ngrapheme frequencies from a Russian novel and its translations to ten other\nSlavic languages. Then, cluster analysis is applied to the graph coordinates.\nWhile the original graph yields results which are not linguistically\ninterpretable, the modification reveals meaningful relations among the\nlanguages.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1504.03659v1",
            "date": "2015-04-14 18:48:58+00:00",
            "title": "Temporal ordering of clinical events",
            "authors": [
                "Azad Dehghan"
            ],
            "abstract": "This report describes a minimalistic set of methods engineered to anchor\nclinical events onto a temporal space. Specifically, we describe methods to\nextract clinical events (e.g., Problems, Treatments and Tests), temporal\nexpressions (i.e., time, date, duration, and frequency), and temporal links\n(e.g., Before, After, Overlap) between events and temporal entities. These\nmethods are developed and validated using high quality datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.04666v1",
            "date": "2015-04-18 00:23:16+00:00",
            "title": "Unsupervised Dependency Parsing: Let's Use Supervised Parsers",
            "authors": [
                "Phong Le",
                "Willem Zuidema"
            ],
            "abstract": "We present a self-training approach to unsupervised dependency parsing that\nreuses existing supervised and unsupervised parsing algorithms. Our approach,\ncalled `iterated reranking' (IR), starts with dependency trees generated by an\nunsupervised parser, and iteratively improves these trees using the richer\nprobability models used in supervised parsing that are in turn trained on these\ntrees. Our system achieves 1.8% accuracy higher than the state-of-the-part\nparser of Spitkovsky et al. (2013) on the WSJ corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.04716v1",
            "date": "2015-04-18 13:28:59+00:00",
            "title": "Gap Analysis of Natural Language Processing Systems with respect to Linguistic Modality",
            "authors": [
                "Vishal Shukla"
            ],
            "abstract": "Modality is one of the important components of grammar in linguistics. It\nlets speaker to express attitude towards, or give assessment or potentiality of\nstate of affairs. It implies different senses and thus has different\nperceptions as per the context. This paper presents an account showing the gap\nin the functionality of the current state of art Natural Language Processing\n(NLP) systems. The contextual nature of linguistic modality is studied. In this\npaper, the works and logical approaches employed by Natural Language Processing\nsystems dealing with modality are reviewed. It sees human cognition and\nintelligence as multi-layered approach that can be implemented by intelligent\nsystems for learning. Lastly, current flow of research going on within this\nfield is talked providing futurology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.04770v1",
            "date": "2015-04-18 22:08:50+00:00",
            "title": "Online Inference for Relation Extraction with a Reduced Feature Set",
            "authors": [
                "Maxim Rabinovich",
                "C\u00e9dric Archambeau"
            ],
            "abstract": "Access to web-scale corpora is gradually bringing robust automatic knowledge\nbase creation and extension within reach. To exploit these large\nunannotated---and extremely difficult to annotate---corpora, unsupervised\nmachine learning methods are required. Probabilistic models of text have\nrecently found some success as such a tool, but scalability remains an obstacle\nin their application, with standard approaches relying on sampling schemes that\nare known to be difficult to scale. In this report, we therefore present an\nempirical assessment of the sublinear time sparse stochastic variational\ninference (SSVI) scheme applied to RelLDA. We demonstrate that online inference\nleads to relatively strong qualitative results but also identify some of its\npathologies---and those of the model---which will need to be overcome if SSVI\nis to be used for large-scale relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.05929v2",
            "date": "2015-04-22 19:13:49+00:00",
            "title": "A Hierarchical Distance-dependent Bayesian Model for Event Coreference Resolution",
            "authors": [
                "Bishan Yang",
                "Claire Cardie",
                "Peter Frazier"
            ],
            "abstract": "We present a novel hierarchical distance-dependent Bayesian model for event\ncoreference resolution. While existing generative models for event coreference\nresolution are completely unsupervised, our model allows for the incorporation\nof pairwise distances between event mentions -- information that is widely used\nin supervised coreference models to guide the generative clustering processing\nfor better event clustering both within and across documents. We model the\ndistances between event mentions using a feature-rich learnable distance\nfunction and encode them as Bayesian priors for nonparametric clustering.\nExperiments on the ECB+ corpus show that our model outperforms state-of-the-art\nmethods for both within- and cross-document event coreference resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06077v1",
            "date": "2015-04-23 08:27:29+00:00",
            "title": "Open Data Platform for Knowledge Access in Plant Health Domain : VESPA Mining",
            "authors": [
                "Nicolas Turenne",
                "Mathieu Andro",
                "Roselyne Corbi\u00e8re",
                "Tien T. Phan"
            ],
            "abstract": "Important data are locked in ancient literature. It would be uneconomic to\nproduce these data again and today or to extract them without the help of text\nmining technologies. Vespa is a text mining project whose aim is to extract\ndata on pest and crops interactions, to model and predict attacks on crops, and\nto reduce the use of pesticides. A few attempts proposed an agricultural\ninformation access. Another originality of our work is to parse documents with\na dependency of the document architecture.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1504.06078v1",
            "date": "2015-04-23 08:28:01+00:00",
            "title": "x.ent: R Package for Entities and Relations Extraction based on Unsupervised Learning and Document Structure",
            "authors": [
                "Nicolas Turenne",
                "Tien Phan"
            ],
            "abstract": "Relation extraction with accurate precision is still a challenge when\nprocessing full text databases. We propose an approach based on cooccurrence\nanalysis in each document for which we used document organization to improve\naccuracy of relation extraction. This approach is implemented in a R package\ncalled \\emph{x.ent}. Another facet of extraction relies on use of extracted\nrelation into a querying system for expert end-users. Two datasets had been\nused. One of them gets interest from specialists of epidemiology in plant\nhealth. For this dataset usage is dedicated to plant-disease exploration\nthrough agricultural information news. An open-data platform exploits exports\nfrom \\emph{x.ent} and is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06080v1",
            "date": "2015-04-23 08:29:11+00:00",
            "title": "svcR: An R Package for Support Vector Clustering improved with Geometric Hashing applied to Lexical Pattern Discovery",
            "authors": [
                "Nicolas Turenne"
            ],
            "abstract": "We present a new R package which takes a numerical matrix format as data\ninput, and computes clusters using a support vector clustering method (SVC). We\nhave implemented an original 2D-grid labeling approach to speed up cluster\nextraction. In this sense, SVC can be seen as an efficient cluster extraction\nif clusters are separable in a 2-D map. Secondly we showed that this SVC\napproach using a Jaccard-Radial base kernel can help to classify well enough a\nset of terms into ontological classes and help to define regular expression\nrules for information extraction in documents; our case study concerns a set of\nterms and documents about developmental and molecular biology.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1504.06650v1",
            "date": "2015-04-24 21:43:55+00:00",
            "title": "Learning Dictionaries for Named Entity Recognition using Minimal Supervision",
            "authors": [
                "Arvind Neelakantan",
                "Michael Collins"
            ],
            "abstract": "This paper describes an approach for automatic construction of dictionaries\nfor Named Entity Recognition (NER) using large amounts of unlabeled data and a\nfew seed examples. We use Canonical Correlation Analysis (CCA) to obtain lower\ndimensional embeddings (representations) for candidate phrases and classify\nthese phrases using a small number of labeled examples. Our method achieves\n16.5% and 11.3% F-1 score improvement over co-training on disease and virus NER\nrespectively. We also show that by adding candidate phrase embeddings as\nfeatures in a sequence tagger gives better performance compared to using word\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06654v1",
            "date": "2015-04-24 22:12:14+00:00",
            "title": "Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space",
            "authors": [
                "Arvind Neelakantan",
                "Jeevan Shankar",
                "Alexandre Passos",
                "Andrew McCallum"
            ],
            "abstract": "There is rising interest in vector-space word embeddings and their use in\nNLP, especially given recent methods for their fast estimation at very large\nscale. Nearly all this work, however, assumes a single vector per word type\nignoring polysemy and thus jeopardizing their usefulness for downstream tasks.\nWe present an extension to the Skip-gram model that efficiently learns multiple\nembeddings per word type. It differs from recent related work by jointly\nperforming word sense discrimination and embedding learning, by\nnon-parametrically estimating the number of senses per word type, and by its\nefficiency and scalability. We present new state-of-the-art results in the word\nsimilarity in context task and demonstrate its scalability by training with one\nmachine on a corpus of nearly 1 billion tokens in less than 6 hours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06658v1",
            "date": "2015-04-24 22:32:40+00:00",
            "title": "Inferring Missing Entity Type Instances for Knowledge Base Completion: New Dataset and Methods",
            "authors": [
                "Arvind Neelakantan",
                "Ming-Wei Chang"
            ],
            "abstract": "Most of previous work in knowledge base (KB) completion has focused on the\nproblem of relation extraction. In this work, we focus on the task of inferring\nmissing entity type instances in a KB, a fundamental task for KB competition\nyet receives little attention. Due to the novelty of this task, we construct a\nlarge-scale dataset and design an automatic evaluation methodology. Our\nknowledge base completion method uses information within the existing KB and\nexternal information from Wikipedia. We show that individual methods trained\nwith a global objective that considers unobserved cells from both the entity\nand the type side gives consistently higher quality predictions compared to\nbaseline methods. We also perform manual evaluation on a small subset of the\ndata to verify the effectiveness of our knowledge base completion methods and\nthe correctness of our proposed automatic evaluation method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06662v2",
            "date": "2015-04-24 23:06:10+00:00",
            "title": "Compositional Vector Space Models for Knowledge Base Completion",
            "authors": [
                "Arvind Neelakantan",
                "Benjamin Roth",
                "Andrew McCallum"
            ],
            "abstract": "Knowledge base (KB) completion adds new facts to a KB by making inferences\nfrom existing facts, for example by inferring with high likelihood\nnationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop\nrelational synonyms like this, or use as evidence a multi-hop relational path\ntreated as an atomic feature, like bornIn(X,Z) -> containedIn(Z,Y). This paper\npresents an approach that reasons about conjunctions of multi-hop relations\nnon-atomically, composing the implications of a path using a recursive neural\nnetwork (RNN) that takes as inputs vector embeddings of the binary relation in\nthe path. Not only does this allow us to generalize to paths unseen at training\ntime, but also, with a single high-capacity RNN, to predict new relation types\nnot seen when the compositional model was trained (zero-shot learning). We\nassemble a new dataset of over 52M relational triples, and show that our method\nimproves over a traditional classifier by 11%, and a method leveraging\npre-trained embeddings by 7%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06665v2",
            "date": "2015-04-24 23:24:10+00:00",
            "title": "Using Syntax-Based Machine Translation to Parse English into Abstract Meaning Representation",
            "authors": [
                "Michael Pust",
                "Ulf Hermjakob",
                "Kevin Knight",
                "Daniel Marcu",
                "Jonathan May"
            ],
            "abstract": "We present a parser for Abstract Meaning Representation (AMR). We treat\nEnglish-to-AMR conversion within the framework of string-to-tree, syntax-based\nmachine translation (SBMT). To make this work, we transform the AMR structure\ninto a form suitable for the mechanics of SBMT and useful for modeling. We\nintroduce an AMR-specific language model and add data and features drawn from\nsemantic resources. Our resulting AMR parser improves upon state-of-the-art\nresults by 7 Smatch points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07071v1",
            "date": "2015-04-27 13:08:36+00:00",
            "title": "Exploring semantically-related concepts from Wikipedia: the case of SeRE",
            "authors": [
                "Daniel Hienert",
                "Dennis Wegener",
                "Siegfried Schomisch"
            ],
            "abstract": "In this paper we present our web application SeRE designed to explore\nsemantically related concepts. Wikipedia and DBpedia are rich data sources to\nextract related entities for a given topic, like in- and out-links, broader and\nnarrower terms, categorisation information etc. We use the Wikipedia full text\nbody to compute the semantic relatedness for extracted terms, which results in\na list of entities that are most relevant for a topic. For any given query, the\nuser interface of SeRE visualizes these related concepts, ordered by semantic\nrelatedness; with snippets from Wikipedia articles that explain the connection\nbetween those two entities. In a user study we examine how SeRE can be used to\nfind important entities and their relationships for a given topic and to answer\nthe question of how the classification system can be used for filtering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07324v1",
            "date": "2015-04-28 01:34:33+00:00",
            "title": "Reader-Aware Multi-Document Summarization via Sparse Coding",
            "authors": [
                "Piji Li",
                "Lidong Bing",
                "Wai Lam",
                "Hang Li",
                "Yi Liao"
            ],
            "abstract": "We propose a new MDS paradigm called reader-aware multi-document\nsummarization (RA-MDS). Specifically, a set of reader comments associated with\nthe news reports are also collected. The generated summaries from the reports\nfor the event should be salient according to not only the reports but also the\nreader comments. To tackle this RA-MDS problem, we propose a\nsparse-coding-based method that is able to calculate the salience of the text\nunits by jointly considering news reports and reader comments. Another\nreader-aware characteristic of our framework is to improve linguistic quality\nvia entity rewriting. The rewriting consideration is jointly assessed together\nwith other summarization requirements under a unified optimization model. To\nsupport the generation of compressive summaries via optimization, we explore a\nfiner syntactic unit, namely, noun/verb phrase. In this work, we also generate\na data set for conducting RA-MDS. Extensive experiments on this data set and\nsome classical data sets demonstrate the effectiveness of our proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07459v1",
            "date": "2015-04-28 13:18:00+00:00",
            "title": "CommentWatcher: An Open Source Web-based platform for analyzing discussions on web forums",
            "authors": [
                "Marian-Andrei Rizoiu",
                "Adrien Guille",
                "Julien Velcin"
            ],
            "abstract": "We present CommentWatcher, an open source tool aimed at analyzing discussions\non web forums. Constructed as a web platform, CommentWatcher features automatic\nmass fetching of user posts from forum on multiple sites, extracting topics,\nvisualizing the topics as an expression cloud and exploring their temporal\nevolution. The underlying social network of users is simultaneously constructed\nusing the citation relations between users and visualized as a graph structure.\nOur platform addresses the issues of the diversity and dynamics of structures\nof webpages hosting the forums by implementing a parser architecture that is\nindependent of the HTML structure of webpages. This allows easy on-the-fly\nadding of new websites. Two types of users are targeted: end users who seek to\nstudy the discussed topics and their temporal evolution, and researchers in\nneed of establishing a forum benchmark dataset and comparing the performances\nof analysis tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07843v1",
            "date": "2015-04-29 13:06:20+00:00",
            "title": "On the universal structure of human lexical semantics",
            "authors": [
                "Hyejin Youn",
                "Logan Sutton",
                "Eric Smith",
                "Cristopher Moore",
                "Jon F. Wilkins",
                "Ian Maddieson",
                "William Croft",
                "Tanmoy Bhattacharya"
            ],
            "abstract": "How universal is human conceptual structure? The way concepts are organized\nin the human brain may reflect distinct features of cultural, historical, and\nenvironmental background in addition to properties universal to human\ncognition. Semantics, or meaning expressed through language, provides direct\naccess to the underlying conceptual structure, but meaning is notoriously\ndifficult to measure, let alone parameterize. Here we provide an empirical\nmeasure of semantic proximity between concepts using cross-linguistic\ndictionaries. Across languages carefully selected from a phylogenetically and\ngeographically stratified sample of genera, translations of words reveal cases\nwhere a particular language uses a single polysemous word to express concepts\nrepresented by distinct words in another. We use the frequency of polysemies\nlinking two concepts as a measure of their semantic proximity, and represent\nthe pattern of such linkages by a weighted network. This network is highly\nuneven and fragmented: certain concepts are far more prone to polysemy than\nothers, and there emerge naturally interpretable clusters loosely connected to\neach other. Statistical analysis shows such structural properties are\nconsistent across different language groups, largely independent of geography,\nenvironment, and literacy. It is therefore possible to conclude the conceptual\nstructure connecting basic vocabulary studied is primarily due to universal\nfeatures of human cognition and language use.",
            "journal ref": "PNAS 113 7 1766-1771 (2016)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1504.08050v1",
            "date": "2015-04-30 00:35:32+00:00",
            "title": "Detecting Concept-level Emotion Cause in Microblogging",
            "authors": [
                "Shuangyong Song",
                "Yao Meng"
            ],
            "abstract": "In this paper, we propose a Concept-level Emotion Cause Model (CECM), instead\nof the mere word-level models, to discover causes of microblogging users'\ndiversified emotions on specific hot event. A modified topic-supervised biterm\ntopic model is utilized in CECM to detect emotion topics' in event-related\ntweets, and then context-sensitive topical PageRank is utilized to detect\nmeaningful multiword expressions as emotion causes. Experimental results on a\ndataset from Sina Weibo, one of the largest microblogging websites in China,\nshow CECM can better detect emotion causes than baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.08342v3",
            "date": "2015-04-30 18:53:06+00:00",
            "title": "Parsing Linear Context-Free Rewriting Systems with Fast Matrix Multiplication",
            "authors": [
                "Shay B. Cohen",
                "Daniel Gildea"
            ],
            "abstract": "We describe a matrix multiplication recognition algorithm for a subset of\nbinary linear context-free rewriting systems (LCFRS) with running time\n$O(n^{\\omega d})$ where $M(m) = O(m^{\\omega})$ is the running time for $m\n\\times m$ matrix multiplication and $d$ is the \"contact rank\" of the LCFRS --\nthe maximal number of combination and non-combination points that appear in the\ngrammar rules. We also show that this algorithm can be used as a subroutine to\nget a recognition algorithm for general binary LCFRS with running time\n$O(n^{\\omega d + 1})$. The currently best known $\\omega$ is smaller than\n$2.38$. Our result provides another proof for the best known result for parsing\nmildly context sensitive formalisms such as combinatory categorial grammars,\nhead grammars, linear indexed grammars, and tree adjoining grammars, which can\nbe parsed in time $O(n^{4.76})$. It also shows that inversion transduction\ngrammars can be parsed in time $O(n^{5.76})$. In addition, binary LCFRS\nsubsumes many other formalisms and types of grammars, for some of which we also\nimprove the asymptotic complexity of parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00278v1",
            "date": "2015-05-31 19:39:44+00:00",
            "title": "Visual Madlibs: Fill in the blank Image Generation and Question Answering",
            "authors": [
                "Licheng Yu",
                "Eunbyung Park",
                "Alexander C. Berg",
                "Tamara L. Berg"
            ],
            "abstract": "In this paper, we introduce a new dataset consisting of 360,001 focused\nnatural language descriptions for 10,738 images. This dataset, the Visual\nMadlibs dataset, is collected using automatically produced fill-in-the-blank\ntemplates designed to gather targeted descriptions about: people and objects,\ntheir appearances, activities, and interactions, as well as inferences about\nthe general scene or its broader context. We provide several analyses of the\nVisual Madlibs dataset and demonstrate its applicability to two new description\ngeneration tasks: focused description generation, and multiple-choice\nquestion-answering for images. Experiments using joint-embedding and deep\nlearning methods show promising results on these tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1506.00301v1",
            "date": "2015-05-31 22:56:43+00:00",
            "title": "Interactive Knowledge Base Population",
            "authors": [
                "Travis Wolfe",
                "Mark Dredze",
                "James Mayfield",
                "Paul McNamee",
                "Craig Harman",
                "Tim Finin",
                "Benjamin Van Durme"
            ],
            "abstract": "Most work on building knowledge bases has focused on collecting entities and\nfacts from as large a collection of documents as possible. We argue for and\ndescribe a new paradigm where the focus is on a high-recall extraction over a\nsmall collection of documents under the supervision of a human expert, that we\ncall Interactive Knowledge Base Population (IKBP).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1506.00799v1",
            "date": "2015-06-02 08:59:47+00:00",
            "title": "Learning Speech Rate in Speech Recognition",
            "authors": [
                "Xiangyu Zeng",
                "Shi Yin",
                "Dong Wang"
            ],
            "abstract": "A significant performance reduction is often observed in speech recognition\nwhen the rate of speech (ROS) is too low or too high. Most of present\napproaches to addressing the ROS variation focus on the change of speech\nsignals in dynamic properties caused by ROS, and accordingly modify the dynamic\nmodel, e.g., the transition probabilities of the hidden Markov model (HMM).\nHowever, an abnormal ROS changes not only the dynamic but also the static\nproperty of speech signals, and thus can not be compensated for purely by\nmodifying the dynamic model. This paper proposes an ROS learning approach based\non deep neural networks (DNN), which involves an ROS feature as the input of\nthe DNN model and so the spectrum distortion caused by ROS can be learned and\ncompensated for. The experimental results show that this approach can deliver\nbetter performance for too slow and too fast utterances, demonstrating our\nconjecture that ROS impacts both the dynamic and the static property of speech.\nIn addition, the proposed approach can be combined with the conventional HMM\ntransition adaptation method, offering additional performance gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01192v3",
            "date": "2015-06-03 10:14:21+00:00",
            "title": "Personalizing Universal Recurrent Neural Network Language Model with User Characteristic Features by Social Network Crowdsouring",
            "authors": [
                "Bo-Hsiang Tseng",
                "Hung-Yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "With the popularity of mobile devices, personalized speech recognizer becomes\nmore realizable today and highly attractive. Each mobile device is primarily\nused by a single user, so it's possible to have a personalized recognizer well\nmatching to the characteristics of individual user. Although acoustic model\npersonalization has been investigated for decades, much less work have been\nreported on personalizing language model, probably because of the difficulties\nin collecting enough personalized corpora. Previous work used the corpora\ncollected from social networks to solve the problem, but constructing a\npersonalized model for each user is troublesome. In this paper, we propose a\nuniversal recurrent neural network language model with user characteristic\nfeatures, so all users share the same model, except each with different user\ncharacteristic features. These user characteristic features can be obtained by\ncrowdsouring over social networks, which include huge quantity of texts posted\nby users with known friend relationships, who may share some subject topics and\nwording patterns. The preliminary experiments on Facebook corpus showed that\nthis proposed approach not only drastically reduced the model perplexity, but\noffered very good improvement in recognition accuracy in n-best rescoring\ntests. This approach also mitigated the data sparseness problem for\npersonalized language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01597v2",
            "date": "2015-06-04 14:04:10+00:00",
            "title": "Abstractive Multi-Document Summarization via Phrase Selection and Merging",
            "authors": [
                "Lidong Bing",
                "Piji Li",
                "Yi Liao",
                "Wai Lam",
                "Weiwei Guo",
                "Rebecca J. Passonneau"
            ],
            "abstract": "We propose an abstraction-based multi-document summarization framework that\ncan construct new sentences by exploring more fine-grained syntactic units than\nsentences, namely, noun/verb phrases. Different from existing abstraction-based\napproaches, our method first constructs a pool of concepts and facts\nrepresented by phrases from the input documents. Then new sentences are\ngenerated by selecting and merging informative phrases to maximize the salience\nof phrases and meanwhile satisfy the sentence construction constraints. We\nemploy integer linear optimization for conducting phrase selection and merging\nsimultaneously in order to achieve the global optimal solution for a summary.\nExperimental results on the benchmark data set TAC 2011 show that our framework\noutperforms the state-of-the-art models under automated pyramid evaluation\nmetric, and achieves reasonably well results on manual linguistic quality\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01698v1",
            "date": "2015-06-04 19:45:36+00:00",
            "title": "The Long-Short Story of Movie Description",
            "authors": [
                "Anna Rohrbach",
                "Marcus Rohrbach",
                "Bernt Schiele"
            ],
            "abstract": "Generating descriptions for videos has many applications including assisting\nblind people and human-robot interaction. The recent advances in image\ncaptioning as well as the release of large-scale movie description datasets\nsuch as MPII Movie Description allow to study this task in more depth. Many of\nthe proposed methods for image captioning rely on pre-trained object classifier\nCNNs and Long-Short Term Memory recurrent networks (LSTMs) for generating\ndescriptions. While image description focuses on objects, we argue that it is\nimportant to distinguish verbs, objects, and places in the challenging setting\nof movie description. In this work we show how to learn robust visual\nclassifiers from the weak annotations of the sentence descriptions. Based on\nthese visual classifiers we learn how to generate a description using an LSTM.\nWe explore different design choices to build and train the LSTM and achieve the\nbest performance to date on the challenging MPII-MD dataset. We compare and\nanalyze our approach and prior work along various dimensions to better\nunderstand the key challenges of the movie description task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1506.02075v1",
            "date": "2015-06-05 21:48:39+00:00",
            "title": "Large-scale Simple Question Answering with Memory Networks",
            "authors": [
                "Antoine Bordes",
                "Nicolas Usunier",
                "Sumit Chopra",
                "Jason Weston"
            ],
            "abstract": "Training large-scale question answering systems is complicated because\ntraining sources usually cover a small portion of the range of possible\nquestions. This paper studies the impact of multitask and transfer learning for\nsimple question answering; a setting for which the reasoning required to answer\nis quite easy, as long as one can retrieve the correct evidence given a\nquestion, which can be difficult in large-scale conditions. To this end, we\nintroduce a new dataset of 100k questions that we use in conjunction with\nexisting benchmarks. We conduct our study within the framework of Memory\nNetworks (Weston et al., 2015) because this perspective allows us to eventually\nscale up to more complex reasoning, and show that Memory Networks can be\nsuccessfully trained to achieve excellent performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1506.02170v1",
            "date": "2015-06-06 17:05:00+00:00",
            "title": "Hybridized Feature Extraction and Acoustic Modelling Approach for Dysarthric Speech Recognition",
            "authors": [
                "Megha Rughani",
                "D. Shivakrishna"
            ],
            "abstract": "Dysarthria is malfunctioning of motor speech caused by faintness in the human\nnervous system. It is characterized by the slurred speech along with physical\nimpairment which restricts their communication and creates the lack of\nconfidence and affects the lifestyle. This paper attempt to increase the\nefficiency of Automatic Speech Recognition (ASR) system for unimpaired speech\nsignal. It describes state of art of research into improving ASR for speakers\nwith dysarthria by means of incorporated knowledge of their speech production.\nHybridized approach for feature extraction and acoustic modelling technique\nalong with evolutionary algorithm is proposed for increasing the efficiency of\nthe overall system. Here number of feature vectors are varied and tested the\nsystem performance. It is observed that system performance is boosted by\ngenetic algorithm. System with 16 acoustic features optimized with genetic\nalgorithm has obtained highest recognition rate of 98.28% with training time of\n5:30:17.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1506.02816v2",
            "date": "2015-06-09 08:09:34+00:00",
            "title": "Leveraging Textual Features for Best Answer Prediction in Community-based Question Answering",
            "authors": [
                "George Gkotsis",
                "Maria Liakata",
                "Carlos Pedrinaci",
                "John Domingue"
            ],
            "abstract": "This paper addresses the problem of determining the best answer in\nCommunity-based Question Answering (CQA) websites by focussing on the content.\nIn particular, we present a system, ACQUA [http://acqua.kmi.open.ac.uk], that\ncan be installed onto the majority of browsers as a plugin. The service offers\na seamless and accurate prediction of the answer to be accepted. Previous\nresearch on this topic relies on the exploitation of community feedback on the\nanswers, which involves rating of either users (e.g., reputation) or answers\n(e.g. scores manually assigned to answers). We propose a new technique that\nleverages the content/textual features of answers in a novel way. Our approach\ndelivers better results than related linguistics-based solutions and manages to\nmatch rating-based approaches. More specifically, the gain in performance is\nachieved by rendering the values of these features into a discretised form. We\nalso show how our technique manages to deliver equally good results in\nreal-time settings, as opposed to having to rely on information not always\nreadily available, such as user ratings and answer scores. We ran an evaluation\non 21 StackExchange websites covering around 4 million questions and more than\n8 million answers. We obtain 84% average precision and 70% recall, which shows\nthat our technique is robust, effective, and widely applicable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02922v1",
            "date": "2015-06-09 14:17:06+00:00",
            "title": "An Ensemble method for Content Selection for Data-to-text Systems",
            "authors": [
                "Dimitra Gkatzia",
                "Helen Hastie"
            ],
            "abstract": "We present a novel approach for automatic report generation from time-series\ndata, in the context of student feedback generation. Our proposed methodology\ntreats content selection as a multi-label classification (MLC) problem, which\ntakes as input time-series data (students' learning data) and outputs a summary\nof these data (feedback). Unlike previous work, this method considers all data\nsimultaneously using ensembles of classifiers, and therefore, it achieves\nhigher accuracy and F- score compared to meaningful baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03500v2",
            "date": "2015-06-10 22:57:20+00:00",
            "title": "Unveiling the Dreams of Word Embeddings: Towards Language-Driven Image Generation",
            "authors": [
                "Angeliki Lazaridou",
                "Dat Tien Nguyen",
                "Raffaella Bernardi",
                "Marco Baroni"
            ],
            "abstract": "We introduce language-driven image generation, the task of generating an\nimage visualizing the semantic contents of a word embedding, e.g., given the\nword embedding of grasshopper, we generate a natural image of a grasshopper. We\nimplement a simple method based on two mapping functions. The first takes as\ninput a word embedding (as produced, e.g., by the word2vec toolkit) and maps it\nonto a high-level visual space (e.g., the space defined by one of the top\nlayers of a Convolutional Neural Network). The second function maps this\nabstract visual representation to pixel space, in order to generate the target\nimage. Several user studies suggest that the current system produces images\nthat capture general visual properties of the concepts encoded in the word\nembedding, such as color or typical environment, and are sufficient to\ndiscriminate between general categories of objects.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1506.04365v1",
            "date": "2015-06-14 09:18:36+00:00",
            "title": "Leveraging Word Embeddings for Spoken Document Summarization",
            "authors": [
                "Kuan-Yu Chen",
                "Shih-Hung Liu",
                "Hsin-Min Wang",
                "Berlin Chen",
                "Hsin-Hsi Chen"
            ],
            "abstract": "Owing to the rapidly growing multimedia content available on the Internet,\nextractive spoken document summarization, with the purpose of automatically\nselecting a set of representative sentences from a spoken document to concisely\nexpress the most important theme of the document, has been an active area of\nresearch and experimentation. On the other hand, word embedding has emerged as\na newly favorite research subject because of its excellent performance in many\nnatural language processing (NLP)-related tasks. However, as far as we are\naware, there are relatively few studies investigating its use in extractive\ntext or speech summarization. A common thread of leveraging word embeddings in\nthe summarization process is to represent the document (or sentence) by\naveraging the word embeddings of the words occurring in the document (or\nsentence). Then, intuitively, the cosine similarity measure can be employed to\ndetermine the relevance degree between a pair of representations. Beyond the\ncontinued efforts made to improve the representation of words, this paper\nfocuses on building novel and efficient ranking models based on the general\nword embedding methods for extractive speech summarization. Experimental\nresults demonstrate the effectiveness of our proposed methods, compared to\nexisting state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04488v2",
            "date": "2015-06-15 06:30:36+00:00",
            "title": "Distilling Word Embeddings: An Encoding Approach",
            "authors": [
                "Lili Mou",
                "Ran Jia",
                "Yan Xu",
                "Ge Li",
                "Lu Zhang",
                "Zhi Jin"
            ],
            "abstract": "Distilling knowledge from a well-trained cumbersome network to a small one\nhas recently become a new research topic, as lightweight neural networks with\nhigh performance are particularly in need in various resource-restricted\nsystems. This paper addresses the problem of distilling word embeddings for NLP\ntasks. We propose an encoding approach to distill task-specific knowledge from\na set of high-dimensional embeddings, which can reduce model complexity by a\nlarge margin as well as retain high accuracy, showing a good compromise between\nefficiency and performance. Experiments in two tasks reveal the phenomenon that\ndistilling knowledge from cumbersome embeddings is better than directly\ntraining neural networks with small embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04803v1",
            "date": "2015-06-16 00:32:33+00:00",
            "title": "Exploiting Text and Network Context for Geolocation of Social Media Users",
            "authors": [
                "Afshin Rahimi",
                "Duy Vu",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Research on automatically geolocating social media users has conventionally\nbeen based on the text content of posts from a given user or the social network\nof the user, with very little crossover between the two, and no bench-marking\nof the two approaches over compara- ble datasets. We bring the two threads of\nresearch together in first proposing a text-based method based on adaptive\ngrids, followed by a hybrid network- and text-based method. Evaluating over\nthree Twitter datasets, we show that the empirical difference between text- and\nnetwork-based methods is not great, and that hybridisation of the two is\nsuperior to the component methods, especially in contexts where the user graph\nis not well connected. We achieve state-of-the-art results on all three\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04828v2",
            "date": "2015-06-16 04:03:06+00:00",
            "title": "Significance of the levels of spectral valleys with application to front/back distinction of vowel sounds",
            "authors": [
                "T. V. Ananthapadmanabha",
                "A. G. Ramakrishnan",
                "Shubham Sharma"
            ],
            "abstract": "An objective critical distance (OCD) has been defined as that spacing between\nadjacent formants, when the level of the valley between them reaches the mean\nspectral level. The measured OCD lies in the same range (viz., 3-3.5 bark) as\nthe critical distance determined by subjective experiments for similar\nexperimental conditions. The level of spectral valley serves a purpose similar\nto that of the spacing between the formants with an added advantage that it can\nbe measured from the spectral envelope without an explicit knowledge of formant\nfrequencies. Based on the relative spacing of formant frequencies, the level of\nthe spectral valley, VI (between F1 and F2) is much higher than the level of\nVII (spectral valley between F2 and F3) for back vowels and vice-versa for\nfront vowels. Classification of vowels into front/back distinction with the\ndifference (VI-VII) as an acoustic feature, tested using TIMIT, NTIMIT, Tamil\nand Kannada language databases gives, on the average, an accuracy of about 95%,\nwhich is comparable to the accuracy (90.6%) obtained using a neural network\nclassifier trained and tested using MFCC as the feature vector for TIMIT\ndatabase. The acoustic feature (VI-VII) has also been tested for its robustness\non the TIMIT database for additive white and babble noise and an accuracy of\nabout 95% has been obtained for SNRs down to 25 dB for both types of noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04834v3",
            "date": "2015-06-16 05:12:52+00:00",
            "title": "Tree-structured composition in neural networks without tree-structured architectures",
            "authors": [
                "Samuel R. Bowman",
                "Christopher D. Manning",
                "Christopher Potts"
            ],
            "abstract": "Tree-structured neural networks encode a particular tree geometry for a\nsentence in the network design. However, these models have at best only\nslightly outperformed simpler sequence-based models. We hypothesize that neural\nsequence models like LSTMs are in fact able to discover and implicitly use\nrecursive compositional structure, at least for tasks with clear cues to that\nstructure in the data. We demonstrate this possibility using an artificial data\ntask for which recursive compositional structure is crucial, and find an\nLSTM-based sequence model can indeed learn to exploit the underlying tree\nstructure. However, its performance consistently lags behind that of tree\nmodels, even on large training sets, suggesting that tree-structured models are\nmore effective at exploiting recursive structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05561v1",
            "date": "2015-06-18 07:10:26+00:00",
            "title": "Comparing and evaluating extended Lambek calculi",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "Lambeks Syntactic Calculus, commonly referred to as the Lambek calculus, was\ninnovative in many ways, notably as a precursor of linear logic. But it also\nshowed that we could treat our grammatical framework as a logic (as opposed to\na logical theory). However, though it was successful in giving at least a basic\ntreatment of many linguistic phenomena, it was also clear that a slightly more\nexpressive logical calculus was needed for many other cases. Therefore, many\nextensions and variants of the Lambek calculus have been proposed, since the\neighties and up until the present day. As a result, there is now a large class\nof calculi, each with its own empirical successes and theoretical results, but\nalso each with its own logical primitives. This raises the question: how do we\ncompare and evaluate these different logical formalisms? To answer this\nquestion, I present two unifying frameworks for these extended Lambek calculi.\nBoth are proof net calculi with graph contraction criteria. The first calculus\nis a very general system: you specify the structure of your sequents and it\ngives you the connectives and contractions which correspond to it. The calculus\ncan be extended with structural rules, which translate directly into graph\nrewrite rules. The second calculus is first-order (multiplicative\nintuitionistic) linear logic, which turns out to have several other,\nindependently proposed extensions of the Lambek calculus as fragments. I will\nillustrate the use of each calculus in building bridges between analyses\nproposed in different frameworks, in highlighting differences and in helping to\nidentify problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06724v1",
            "date": "2015-06-22 19:26:56+00:00",
            "title": "Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books",
            "authors": [
                "Yukun Zhu",
                "Ryan Kiros",
                "Richard Zemel",
                "Ruslan Salakhutdinov",
                "Raquel Urtasun",
                "Antonio Torralba",
                "Sanja Fidler"
            ],
            "abstract": "Books are a rich source of both fine-grained information, how a character, an\nobject or a scene looks like, as well as high-level semantics, what someone is\nthinking, feeling and how these states evolve through a story. This paper aims\nto align books to their movie releases in order to provide rich descriptive\nexplanations for visual content that go semantically far beyond the captions\navailable in current datasets. To align movies and books we exploit a neural\nsentence embedding that is trained in an unsupervised way from a large corpus\nof books, as well as a video-text neural embedding for computing similarities\nbetween movie clips and sentences in the book. We propose a context-aware CNN\nto combine information from multiple sources. We demonstrate good quantitative\nperformance for movie/book alignment and show several qualitative examples that\nshowcase the diversity of tasks our model can be used for.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1506.06726v1",
            "date": "2015-06-22 19:33:40+00:00",
            "title": "Skip-Thought Vectors",
            "authors": [
                "Ryan Kiros",
                "Yukun Zhu",
                "Ruslan Salakhutdinov",
                "Richard S. Zemel",
                "Antonio Torralba",
                "Raquel Urtasun",
                "Sanja Fidler"
            ],
            "abstract": "We describe an approach for unsupervised learning of a generic, distributed\nsentence encoder. Using the continuity of text from books, we train an\nencoder-decoder model that tries to reconstruct the surrounding sentences of an\nencoded passage. Sentences that share semantic and syntactic properties are\nthus mapped to similar vector representations. We next introduce a simple\nvocabulary expansion method to encode words that were not seen as part of\ntraining, allowing us to expand our vocabulary to a million words. After\ntraining our model, we extract and evaluate our vectors with linear models on 8\ntasks: semantic relatedness, paraphrase detection, image-sentence ranking,\nquestion-type classification and 4 benchmark sentiment and subjectivity\ndatasets. The end result is an off-the-shelf encoder that can produce highly\ngeneric sentence representations that are robust and perform well in practice.\nWe will make our encoder publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.07190v1",
            "date": "2015-06-23 20:16:06+00:00",
            "title": "Multi-domain Dialog State Tracking using Recurrent Neural Networks",
            "authors": [
                "Nikola Mrk\u0161i\u0107",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Blaise Thomson",
                "Milica Ga\u0161i\u0107",
                "Pei-Hao Su",
                "David Vandyke",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "Dialog state tracking is a key component of many modern dialog systems, most\nof which are designed with a single, well-defined domain in mind. This paper\nshows that dialog data drawn from different dialog domains can be used to train\na general belief tracking model which can operate across all of these domains,\nexhibiting superior performance to each of the domain-specific models. We\npropose a training procedure which uses out-of-domain data to initialise belief\ntracking models for entirely new domains. This procedure leads to improvements\nin belief tracking performance regardless of the amount of in-domain data\navailable for training the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.07650v1",
            "date": "2015-06-25 07:51:55+00:00",
            "title": "Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling",
            "authors": [
                "Kun Xu",
                "Yansong Feng",
                "Songfang Huang",
                "Dongyan Zhao"
            ],
            "abstract": "Syntactic features play an essential role in identifying relationship in a\nsentence. Previous neural network models often suffer from irrelevant\ninformation introduced when subjects and objects are in a long distance. In\nthis paper, we propose to learn more robust relation representations from the\nshortest dependency path through a convolution neural network. We further\npropose a straightforward negative sampling strategy to improve the assignment\nof subjects and objects. Experimental results show that our method outperforms\nthe state-of-the-art methods on the SemEval-2010 Task 8 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08259v3",
            "date": "2015-06-27 04:51:18+00:00",
            "title": "Twitter User Geolocation Using a Unified Text and Network Prediction Model",
            "authors": [
                "Afshin Rahimi",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "We propose a label propagation approach to geolocation prediction based on\nModified Adsorption, with two enhancements:(1) the removal of \"celebrity\" nodes\nto increase location homophily and boost tractability, and (2) he incorporation\nof text-based geolocation priors for test users. Experiments over three Twitter\nbenchmark datasets achieve state-of-the-art results, and demonstrate the\neffectiveness of the enhancements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08422v1",
            "date": "2015-06-28 16:17:40+00:00",
            "title": "Topic2Vec: Learning Distributed Representations of Topics",
            "authors": [
                "Li-Qiang Niu",
                "Xin-Yu Dai"
            ],
            "abstract": "Latent Dirichlet Allocation (LDA) mining thematic structure of documents\nplays an important role in nature language processing and machine learning\nareas. However, the probability distribution from LDA only describes the\nstatistical relationship of occurrences in the corpus and usually in practice,\nprobability is not the best choice for feature representations. Recently,\nembedding methods have been proposed to represent words and documents by\nlearning essential concepts and representations, such as Word2Vec and Doc2Vec.\nThe embedded representations have shown more effectiveness than LDA-style\nrepresentations in many tasks. In this paper, we propose the Topic2Vec approach\nwhich can learn topic representations in the same semantic vector space with\nwords, as an alternative to probability. The experimental results show that\nTopic2Vec achieves interesting and meaningful results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08663v1",
            "date": "2015-06-14 12:09:01+00:00",
            "title": "Linguistics and some aspects of its underlying dynamics",
            "authors": [
                "Massimo Piattelli-Palmarini",
                "Giuseppe Vitiello"
            ],
            "abstract": "In recent years, central components of a new approach to linguistics, the\nMinimalist Program (MP) have come closer to physics. Features of the Minimalist\nProgram, such as the unconstrained nature of recursive Merge, the operation of\nthe Labeling Algorithm that only operates at the interface of Narrow Syntax\nwith the Conceptual-Intentional and the Sensory-Motor interfaces, the\ndifference between pronounced and un-pronounced copies of elements in a\nsentence and the build-up of the Fibonacci sequence in the syntactic derivation\nof sentence structures, are directly accessible to representation in terms of\nalgebraic formalism. Although in our scheme linguistic structures are classical\nones, we find that an interesting and productive isomorphism can be established\nbetween the MP structure, algebraic structures and many-body field theory\nopening new avenues of inquiry on the dynamics underlying some central aspects\nof linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08941v2",
            "date": "2015-06-30 05:51:11+00:00",
            "title": "Language Understanding for Text-based Games Using Deep Reinforcement Learning",
            "authors": [
                "Karthik Narasimhan",
                "Tejas Kulkarni",
                "Regina Barzilay"
            ],
            "abstract": "In this paper, we consider the task of learning control policies for\ntext-based games. In these games, all interactions in the virtual world are\nthrough text and the underlying state is not observed. The resulting language\nbarrier makes such environments challenging for automatic game players. We\nemploy a deep reinforcement learning framework to jointly learn state\nrepresentations and action policies using game rewards as feedback. This\nframework enables us to map text descriptions into vector representations that\ncapture the semantics of the game states. We evaluate our approach on two game\nworlds, comparing against baselines using bag-of-words and bag-of-bigrams for\nstate representations. Our algorithm outperforms the baselines on both worlds\ndemonstrating the importance of learning expressive representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00189v1",
            "date": "2015-08-02 04:17:40+00:00",
            "title": "Class Vectors: Embedding representation of Document Classes",
            "authors": [
                "Devendra Singh Sachan",
                "Shailesh Kumar"
            ],
            "abstract": "Distributed representations of words and paragraphs as semantic embeddings in\nhigh dimensional data are used across a number of Natural Language\nUnderstanding tasks such as retrieval, translation, and classification. In this\nwork, we propose \"Class Vectors\" - a framework for learning a vector per class\nin the same embedding space as the word and paragraph embeddings. Similarity\nbetween these class vectors and word vectors are used as features to classify a\ndocument to a class. In experiment on several sentiment analysis tasks such as\nYelp reviews and Amazon electronic product reviews, class vectors have shown\nbetter or comparable results in classification while learning very meaningful\nclass embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00354v1",
            "date": "2015-08-03 09:28:22+00:00",
            "title": "Significance of Maximum Spectral Amplitude in Sub-bands for Spectral Envelope Estimation and Its Application to Statistical Parametric Speech Synthesis",
            "authors": [
                "Sivanand Achanta",
                "Anandaswarup Vadapalli",
                "Sai Krishna R.",
                "Suryakanth V. Gangashetty"
            ],
            "abstract": "In this paper we propose a technique for spectral envelope estimation using\nmaximum values in the sub-bands of Fourier magnitude spectrum (MSASB). Most\nother methods in the literature parametrize spectral envelope in cepstral\ndomain such as Mel-generalized cepstrum etc. Such cepstral domain\nrepresentations, although compact, are not readily interpretable. This\ndifficulty is overcome by our method which parametrizes in the spectral domain\nitself. In our experiments, spectral envelope estimated using MSASB method was\nincorporated in the STRAIGHT vocoder. Both objective and subjective results of\nanalysis-by-synthesis indicate that the proposed method is comparable to\nSTRAIGHT. We also evaluate the effectiveness of the proposed parametrization in\na statistical parametric speech synthesis framework using deep neural networks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1508.00715v2",
            "date": "2015-08-04 09:34:22+00:00",
            "title": "Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs",
            "authors": [
                "Zhilin Yang",
                "Jie Tang",
                "William Cohen"
            ],
            "abstract": "We study the extent to which online social networks can be connected to open\nknowledge bases. The problem is referred to as learning social knowledge\ngraphs. We propose a multi-modal Bayesian embedding model, GenVector, to learn\nlatent topics that generate word and network embeddings. GenVector leverages\nlarge-scale unlabeled data with embeddings and represents data of two\nmodalities---i.e., social network users and knowledge concepts---in a shared\nlatent topic space. Experiments on three datasets show that the proposed method\nclearly outperforms state-of-the-art methods. We then deploy the method on\nAMiner, a large-scale online academic search system with a network of\n38,049,189 researchers with a knowledge base with 35,415,011 concepts. Our\nmethod significantly decreases the error rate in an online A/B test with live\nusers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01067v1",
            "date": "2015-08-05 13:18:51+00:00",
            "title": "Topic Stability over Noisy Sources",
            "authors": [
                "Jing Su",
                "Ois\u00edn Boydell",
                "Derek Greene",
                "Gerard Lynch"
            ],
            "abstract": "Topic modelling techniques such as LDA have recently been applied to speech\ntranscripts and OCR output. These corpora may contain noisy or erroneous texts\nwhich may undermine topic stability. Therefore, it is important to know how\nwell a topic modelling algorithm will perform when applied to noisy data. In\nthis paper we show that different types of textual noise will have diverse\neffects on the stability of different topic models. From these observations, we\npropose guidelines for text corpus generation, with a focus on automatic speech\ntranscription. We also suggest topic model selection methods for noisy corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01321v1",
            "date": "2015-08-06 08:39:20+00:00",
            "title": "On Gobbledygook and Mood of the Philippine Senate: An Exploratory Study on the Readability and Sentiment of Selected Philippine Senators' Microposts",
            "authors": [
                "Fatima M. Moncada",
                "Jaderick P. Pabico"
            ],
            "abstract": "This paper presents the findings of a readability assessment and sentiment\nanalysis of selected six Philippine senators' microposts over the popular\nTwitter microblog. Using the Simple Measure of Gobbledygook (SMOG), tweets of\nSenators Cayetano, Defensor-Santiago, Pangilinan, Marcos, Guingona, and\nEscudero were assessed. A sentiment analysis was also done to determine the\npolarity of the senators' respective microposts. Results showed that on the\naverage, the six senators are tweeting at an eight to ten SMOG level. This\nmeans that, at least a sixth grader will be able to understand the senators'\ntweets. Moreover, their tweets are mostly neutral and their sentiments vary in\nunison at some period of time. This could mean that a senator's tweet sentiment\nis affected by specific Philippine-based events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01571v1",
            "date": "2015-08-06 23:53:30+00:00",
            "title": "A Mood-based Genre Classification of Television Content",
            "authors": [
                "Humberto Corona",
                "Michael P. O'Mahony"
            ],
            "abstract": "The classification of television content helps users organise and navigate\nthrough the large list of channels and programs now available. In this paper,\nwe address the problem of television content classification by exploiting text\ninformation extracted from program transcriptions. We present an analysis which\nadapts a model for sentiment that has been widely and successfully applied in\nother fields such as music or blog posts. We use a real-world dataset obtained\nfrom the Boxfish API to compare the performance of classifiers trained on a\nnumber of different feature sets. Our experiments show that, over a large\ncollection of television content, program genres can be represented in a\nthree-dimensional space of valence, arousal and dominance, and that promising\nclassification results can be achieved using features based on this\nrepresentation. This finding supports the use of the proposed representation of\ntelevision content as a feature space for similarity computation and\nrecommendation generation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1508.01577v2",
            "date": "2015-08-07 00:57:18+00:00",
            "title": "Automata networks model for alignment and least effort on vocabulary formation",
            "authors": [
                "Javier Vera",
                "Felipe Urbina",
                "Eric Goles"
            ],
            "abstract": "Can artificial communities of agents develop language with scaling relations\nclose to the Zipf law? As a preliminary answer to this question, we propose an\nAutomata Networks model of the formation of a vocabulary on a population of\nindividuals, under two in principle opposite strategies: the alignment and the\nleast effort principle. Within the previous account to the emergence of\nlinguistic conventions (specially, the Naming Game), we focus on modeling\nspeaker and hearer efforts as actions over their vocabularies and we study the\nimpact of these actions on the formation of a shared language. The numerical\nsimulations are essentially based on an energy function, that measures the\namount of local agreement between the vocabularies. The results suggests that\non one dimensional lattices the best strategy to the formation of shared\nlanguages is the one that minimizes the efforts of speakers on communicative\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01580v2",
            "date": "2015-08-07 01:15:23+00:00",
            "title": "Automata networks for memory loss effects in the formation of linguistic conventions",
            "authors": [
                "Javier Vera",
                "Eric Goles"
            ],
            "abstract": "This work attempts to give new theoretical insights to the absence of\nintermediate stages in the evolution of language. In particular, it is\ndeveloped an automata networks approach to a crucial question: how a population\nof language users can reach agreement on a linguistic convention? To describe\nthe appearance of sharp transitions in the self-organization of language, it is\nadopted an extremely simple model of (working) memory. At each time step,\nlanguage users simply loss part of their word-memories. Through computer\nsimulations of low-dimensional lattices, it appear sharp transitions at\ncritical values that depend on the size of the vicinities of the individuals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01585v2",
            "date": "2015-08-07 01:54:04+00:00",
            "title": "Applying Deep Learning to Answer Selection: A Study and An Open Task",
            "authors": [
                "Minwei Feng",
                "Bing Xiang",
                "Michael R. Glass",
                "Lidan Wang",
                "Bowen Zhou"
            ],
            "abstract": "We apply a general deep learning framework to address the non-factoid\nquestion answering task. Our approach does not rely on any linguistic tools and\ncan be applied to different languages or domains. Various architectures are\npresented and compared. We create and release a QA corpus and setup a new QA\ntask in the insurance domain. Experimental results demonstrate superior\nperformance compared to the baseline methods and various technologies give\nfurther improvements. For this highly challenging task, the top-1 accuracy can\nreach up to 65.3% on a test set, which indicates a great potential for\npractical use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01786v1",
            "date": "2015-08-07 19:35:52+00:00",
            "title": "Mimicry Is Presidential: Linguistic Style Matching in Presidential Debates and Improved Polling Numbers",
            "authors": [
                "Daniel M. Romero",
                "Roderick I. Swaab",
                "Brian Uzzi",
                "Adam D. Galinsky"
            ],
            "abstract": "The current research used the contexts of U.S. presidential debates and\nnegotiations to examine whether matching the linguistic style of an opponent in\na two-party exchange affects the reactions of third-party observers. Building\noff communication accommodation theory (CAT), interaction alignment theory\n(IAT), and processing fluency, we propose that language style matching (LSM)\nwill improve subsequent third-party evaluations because matching an opponent's\nlinguistic style reflects greater perspective taking and will make one's\narguments easier to process. In contrast, research on status inferences\npredicts that LSM will negatively impact third-party evaluations because LSM\nimplies followership. We conduct two studies to test these competing\nhypotheses. Study 1 analyzed transcripts of U.S. presidential debates between\n1976 and 2012 and found that candidates who matched their opponent's linguistic\nstyle increased their standing in the polls. Study 2 demonstrated a causal\nrelationship between LSM and third-party observer evaluations using negotiation\ntranscripts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02091v1",
            "date": "2015-08-09 22:52:10+00:00",
            "title": "Image Representations and New Domains in Neural Image Captioning",
            "authors": [
                "Jack Hessel",
                "Nicolas Savva",
                "Michael J. Wilber"
            ],
            "abstract": "We examine the possibility that recent promising results in automatic caption\ngeneration are due primarily to language models. By varying image\nrepresentation quality produced by a convolutional neural network, we find that\na state-of-the-art neural captioning algorithm is able to produce quality\ncaptions even when provided with surprisingly poor image representations. We\nreplicate this result in a new, fine-grained, transfer learned captioning\ndomain, consisting of 66K recipe image/title pairs. We also provide some\nexperiments regarding the appropriateness of datasets for automatic captioning,\nand find that having multiple captions per image is beneficial, but not an\nabsolute requirement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02131v1",
            "date": "2015-08-10 05:57:14+00:00",
            "title": "Learning Structural Kernels for Natural Language Processing",
            "authors": [
                "Daniel Beck",
                "Trevor Cohn",
                "Christian Hardmeier",
                "Lucia Specia"
            ],
            "abstract": "Structural kernels are a flexible learning paradigm that has been widely used\nin Natural Language Processing. However, the problem of model selection in\nkernel-based methods is usually overlooked. Previous approaches mostly rely on\nsetting default values for kernel hyperparameters or using grid search, which\nis slow and coarse-grained. In contrast, Bayesian methods allow efficient model\nselection by maximizing the evidence on the training data through\ngradient-based methods. In this paper we show how to perform this in the\ncontext of structural kernels by using Gaussian Processes. Experimental results\non tree kernels show that this procedure results in better prediction\nperformance compared to hyperparameter optimization via grid search. The\nframework proposed in this paper can be adapted to other structures besides\ntrees, e.g., strings and graphs, thereby extending the utility of kernel-based\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.02375v1",
            "date": "2015-08-10 19:48:33+00:00",
            "title": "Approximation-Aware Dependency Parsing by Belief Propagation",
            "authors": [
                "Matthew R. Gormley",
                "Mark Dredze",
                "Jason Eisner"
            ],
            "abstract": "We show how to train the fast dependency parser of Smith and Eisner (2008)\nfor improved accuracy. This parser can consider higher-order interactions among\nedges while retaining O(n^3) runtime. It outputs the parse with maximum\nexpected recall -- but for speed, this expectation is taken under a posterior\ndistribution that is constructed only approximately, using loopy belief\npropagation through structured factors. We show how to adjust the model\nparameters to compensate for the errors introduced by this approximation, by\nfollowing the gradient of the actual loss on training data. We find this\ngradient by back-propagation. That is, we treat the entire parser\n(approximations and all) as a differentiable circuit, as Stoyanov et al. (2011)\nand Domke (2010) did for loopy CRFs. The resulting trained parser obtains\nhigher accuracy with fewer iterations of belief propagation than one trained by\nconditional log-likelihood.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03386v1",
            "date": "2015-08-13 23:44:03+00:00",
            "title": "Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems",
            "authors": [
                "Pei-Hao Su",
                "David Vandyke",
                "Milica Gasic",
                "Dongho Kim",
                "Nikola Mrksic",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "To train a statistical spoken dialogue system (SDS) it is essential that an\naccurate method for measuring task success is available. To date training has\nrelied on presenting a task to either simulated or paid users and inferring the\ndialogue's success by observing whether this presented task was achieved or\nnot. Our aim however is to be able to learn from real users acting under their\nown volition, in which case it is non-trivial to rate the success as any prior\nknowledge of the task is simply unavailable. User feedback may be utilised but\nhas been found to be inconsistent. Hence, here we present two neural network\nmodels that evaluate a sequence of turn-level features to rate the success of a\ndialogue. Importantly these models make no use of any prior knowledge of the\nuser's task. The models are trained on dialogues generated by a simulated user\nand the best model is then used to train a policy on-line which is shown to\nperform at least as well as a baseline system using prior knowledge of the\nuser's task. We note that the models should also be of interest for evaluating\nSDS and for monitoring a dialogue in rule-based SDS.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1508.03391v2",
            "date": "2015-08-14 00:41:12+00:00",
            "title": "Reward Shaping with Recurrent Neural Networks for Speeding up On-Line Policy Learning in Spoken Dialogue Systems",
            "authors": [
                "Pei-Hao Su",
                "David Vandyke",
                "Milica Gasic",
                "Nikola Mrksic",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "Statistical spoken dialogue systems have the attractive property of being\nable to be optimised from data via interactions with real users. However in the\nreinforcement learning paradigm the dialogue manager (agent) often requires\nsignificant time to explore the state-action space to learn to behave in a\ndesirable manner. This is a critical issue when the system is trained on-line\nwith real users where learning costs are expensive. Reward shaping is one\npromising technique for addressing these concerns. Here we examine three\nrecurrent neural network (RNN) approaches for providing reward shaping\ninformation in addition to the primary (task-orientated) environmental\nfeedback. These RNNs are trained on returns from dialogues generated by a\nsimulated user and attempt to diffuse the overall evaluation of the dialogue\nback down to the turn level to guide the agent towards good behaviour faster.\nIn both simulated and real user scenarios these RNNs are shown to increase\npolicy learning speed. Importantly, they do not require prior knowledge of the\nuser's goal.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1508.03601v1",
            "date": "2015-08-14 18:39:18+00:00",
            "title": "Is Stack Overflow Overflowing With Questions and Tags",
            "authors": [
                "Ranjitha R. K.",
                "Sanjay Singh"
            ],
            "abstract": "Programming question and answer (Q & A) websites, such as Quora, Stack\nOverflow, and Yahoo! Answer etc. helps us to understand the programming\nconcepts easily and quickly in a way that has been tested and applied by many\nsoftware developers. Stack Overflow is one of the most frequently used\nprogramming Q\\&A website where the questions and answers posted are presently\nanalyzed manually, which requires a huge amount of time and resource. To save\nthe effort, we present a topic modeling based technique to analyze the words of\nthe original texts to discover the themes that run through them. We also\npropose a method to automate the process of reviewing the quality of questions\non Stack Overflow dataset in order to avoid ballooning the stack overflow with\ninsignificant questions. The proposed method also recommends the appropriate\ntags for the new post, which averts the creation of unnecessary tags on Stack\nOverflow.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1508.03720v1",
            "date": "2015-08-15 11:15:32+00:00",
            "title": "Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path",
            "authors": [
                "Xu Yan",
                "Lili Mou",
                "Ge Li",
                "Yunchuan Chen",
                "Hao Peng",
                "Zhi Jin"
            ],
            "abstract": "Relation classification is an important research arena in the field of\nnatural language processing (NLP). In this paper, we present SDP-LSTM, a novel\nneural network to classify the relation of two entities in a sentence. Our\nneural architecture leverages the shortest dependency path (SDP) between two\nentities; multichannel recurrent neural networks, with long short term memory\n(LSTM) units, pick up heterogeneous information along the SDP. Our proposed\nmodel has several distinct features: (1) The shortest dependency paths retain\nmost relevant information (to relation classification), while eliminating\nirrelevant words in the sentence. (2) The multichannel LSTM networks allow\neffective information integration from heterogeneous sources over the\ndependency paths. (3) A customized dropout strategy regularizes the neural\nnetwork to alleviate overfitting. We test our model on the SemEval 2010\nrelation classification task, and achieve an $F_1$-score of 83.7\\%, higher than\ncompeting methods in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03721v1",
            "date": "2015-08-15 11:16:39+00:00",
            "title": "A Comparative Study on Regularization Strategies for Embedding-based Neural Networks",
            "authors": [
                "Hao Peng",
                "Lili Mou",
                "Ge Li",
                "Yunchuan Chen",
                "Yangyang Lu",
                "Zhi Jin"
            ],
            "abstract": "This paper aims to compare different regularization strategies to address a\ncommon phenomenon, severe overfitting, in embedding-based neural networks for\nNLP. We chose two widely studied neural models and tasks as our testbed. We\ntried several frequently applied or newly proposed regularization strategies,\nincluding penalizing weights (embeddings excluded), penalizing embeddings,\nre-embedding words, and dropout. We also emphasized on incremental\nhyperparameter tuning, and combining different regularizations. The results\nprovide a picture on tuning hyperparameters for neural NLP models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03790v4",
            "date": "2015-08-16 04:31:37+00:00",
            "title": "Depth-Gated LSTM",
            "authors": [
                "Kaisheng Yao",
                "Trevor Cohn",
                "Katerina Vylomova",
                "Kevin Duh",
                "Chris Dyer"
            ],
            "abstract": "In this short note, we present an extension of long short-term memory (LSTM)\nneural networks to using a depth gate to connect memory cells of adjacent\nlayers. Doing so introduces a linear dependence between lower and upper layer\nrecurrent units. Importantly, the linear dependence is gated through a gating\nfunction, which we call depth gate. This gate is a function of the lower layer\nmemory cell, the input to and the past memory cell of this layer. We conducted\nexperiments and verified that this new architecture of LSTMs was able to\nimprove machine translation and language modeling performances.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1508.04112v2",
            "date": "2015-08-17 19:02:45+00:00",
            "title": "Molding CNNs for text: non-linear, non-consecutive convolutions",
            "authors": [
                "Tao Lei",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "The success of deep learning often derives from well-chosen operational\nbuilding blocks. In this work, we revise the temporal convolution operation in\nCNNs to better adapt it to text processing. Instead of concatenating word\nrepresentations, we appeal to tensor algebra and use low-rank n-gram tensors to\ndirectly exploit interactions between words already at the convolution stage.\nMoreover, we extend the n-gram convolution to non-consecutive words to\nrecognize patterns with intervening words. Through a combination of low-rank\ntensors, and pattern weighting, we can efficiently evaluate the resulting\nconvolution operation via dynamic programming. We test the resulting\narchitecture on standard sentiment classification and news categorization\ntasks. Our model achieves state-of-the-art performance both in terms of\naccuracy and training speed. For instance, we obtain 51.2% accuracy on the\nfine-grained sentiment classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04525v1",
            "date": "2015-08-19 04:17:47+00:00",
            "title": "Recognizing Extended Spatiotemporal Expressions by Actively Trained Average Perceptron Ensembles",
            "authors": [
                "Wei Zhang",
                "Yang Yu",
                "Osho Gupta",
                "Judith Gelernter"
            ],
            "abstract": "Precise geocoding and time normalization for text requires that location and\ntime phrases be identified. Many state-of-the-art geoparsers and temporal\nparsers suffer from low recall. Categories commonly missed by parsers are:\nnouns used in a non- spatiotemporal sense, adjectival and adverbial phrases,\nprepositional phrases, and numerical phrases. We collected and annotated data\nset by querying commercial web searches API with such spatiotemporal\nexpressions as were missed by state-of-the- art parsers. Due to the high cost\nof sentence annotation, active learning was used to label training data, and a\nnew strategy was designed to better select training examples to reduce labeling\ncost. For the learning algorithm, we applied an average perceptron trained\nFeaturized Hidden Markov Model (FHMM). Five FHMM instances were used to create\nan ensemble, with the output phrase selected by voting. Our ensemble model was\ntested on a range of sequential labeling tasks, and has shown competitive\nperformance. Our contributions include (1) an new dataset annotated with named\nentities and expanded spatiotemporal expressions; (2) a comparison of inference\nalgorithms for ensemble models showing the superior accuracy of Belief\nPropagation over Viterbi Decoding; (3) a new example re-weighting method for\nactive ensemble learning that 'memorizes' the latest examples trained; (4) a\nspatiotemporal parser that jointly recognizes expanded spatiotemporal\nexpressions as well as named entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.04562v1",
            "date": "2015-08-19 08:30:37+00:00",
            "title": "Fast, Flexible Models for Discovering Topic Correlation across Weakly-Related Collections",
            "authors": [
                "Jingwei Zhang",
                "Aaron Gerow",
                "Jaan Altosaar",
                "James Evans",
                "Richard Jean So"
            ],
            "abstract": "Weak topic correlation across document collections with different numbers of\ntopics in individual collections presents challenges for existing\ncross-collection topic models. This paper introduces two probabilistic topic\nmodels, Correlated LDA (C-LDA) and Correlated HDP (C-HDP). These address\nproblems that can arise when analyzing large, asymmetric, and potentially\nweakly-related collections. Topic correlations in weakly-related collections\ntypically lie in the tail of the topic distribution, where they would be\noverlooked by models unable to fit large numbers of topics. To efficiently\nmodel this long tail for large-scale analysis, our models implement a parallel\nsampling algorithm based on the Metropolis-Hastings and alias methods (Yuan et\nal., 2015). The models are first evaluated on synthetic data, generated to\nsimulate various collection-level asymmetries. We then present a case study of\nmodeling over 300k documents in collections of sciences and humanities research\nfrom JSTOR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05163v1",
            "date": "2015-08-21 01:40:54+00:00",
            "title": "Simple Text Mining for Sentiment Analysis of Political Figure Using Naive Bayes Classifier Method",
            "authors": [
                "Yustinus Eko Soelistio",
                "Martinus Raditia Sigit Surendra"
            ],
            "abstract": "Text mining can be applied to many fields. One of the application is using\ntext mining in digital newspaper to do politic sentiment analysis. In this\npaper sentiment analysis is applied to get information from digital news\narticles about its positive or negative sentiment regarding particular\npolitician. This paper suggests a simple model to analyze digital newspaper\nsentiment polarity using naive Bayes classifier method. The model uses a set of\ninitial data to begin with which will be updated when new information appears.\nThe model showed promising result when tested and can be implemented to some\nother sentiment analysis problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05902v1",
            "date": "2015-08-24 18:14:34+00:00",
            "title": "A Framework for Comparing Groups of Documents",
            "authors": [
                "Arun S. Maiya"
            ],
            "abstract": "We present a general framework for comparing multiple groups of documents. A\nbipartite graph model is proposed where document groups are represented as one\nnode set and the comparison criteria are represented as the other node set.\nUsing this model, we present basic algorithms to extract insights into\nsimilarities and differences among the document groups. Finally, we demonstrate\nthe versatility of our framework through an analysis of NSF funding programs\nfor basic research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06034v1",
            "date": "2015-08-25 05:04:53+00:00",
            "title": "Better Summarization Evaluation with Word Embeddings for ROUGE",
            "authors": [
                "Jun-Ping Ng",
                "Viktoria Abrecht"
            ],
            "abstract": "ROUGE is a widely adopted, automatic evaluation measure for text\nsummarization. While it has been shown to correlate well with human judgements,\nit is biased towards surface lexical similarities. This makes it unsuitable for\nthe evaluation of abstractive summarization, or summaries with substantial\nparaphrasing. We study the effectiveness of word embeddings to overcome this\ndisadvantage of ROUGE. Specifically, instead of measuring lexical overlaps,\nword embeddings are used to compute the semantic similarity of the words used\nin summaries instead. Our experimental results show that our proposal is able\nto achieve better correlations with human judgements when measured with the\nSpearman and Kendall rank coefficients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.00060v3",
            "date": "2015-10-31 02:05:28+00:00",
            "title": "Top-down Tree Long Short-Term Memory Networks",
            "authors": [
                "Xingxing Zhang",
                "Liang Lu",
                "Mirella Lapata"
            ],
            "abstract": "Long Short-Term Memory (LSTM) networks, a type of recurrent neural network\nwith a more complex computational unit, have been successfully applied to a\nvariety of sequence modeling tasks. In this paper we develop Tree Long\nShort-Term Memory (TreeLSTM), a neural network model based on LSTM, which is\ndesigned to predict a tree rather than a linear sequence. TreeLSTM defines the\nprobability of a sentence by estimating the generation probability of its\ndependency tree. At each time step, a node is generated based on the\nrepresentation of the generated sub-tree. We further enhance the modeling power\nof TreeLSTM by explicitly representing the correlations between left and right\ndependents. Application of our model to the MSR sentence completion challenge\nachieves results beyond the current state of the art. We also report results on\ndependency parsing reranking achieving competitive performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.00360v1",
            "date": "2015-11-02 02:34:12+00:00",
            "title": "Automatic Prosody Prediction for Chinese Speech Synthesis using BLSTM-RNN and Embedding Features",
            "authors": [
                "Chuang Ding",
                "Lei Xie",
                "Jie Yan",
                "Weini Zhang",
                "Yang Liu"
            ],
            "abstract": "Prosody affects the naturalness and intelligibility of speech. However,\nautomatic prosody prediction from text for Chinese speech synthesis is still a\ngreat challenge and the traditional conditional random fields (CRF) based\nmethod always heavily relies on feature engineering. In this paper, we propose\nto use neural networks to predict prosodic boundary labels directly from\nChinese characters without any feature engineering. Experimental results show\nthat stacking feed-forward and bidirectional long short-term memory (BLSTM)\nrecurrent network layers achieves superior performance over the CRF-based\nmethod. The embedding features learned from raw text further enhance the\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01259v2",
            "date": "2015-11-04 09:41:31+00:00",
            "title": "Transforming Wikipedia into an Ontology-based Information Retrieval Search Engine for Local Experts using a Third-Party Taxonomy",
            "authors": [
                "Gregory Grefenstette",
                "Karima Rafes"
            ],
            "abstract": "Wikipedia is widely used for finding general information about a wide variety\nof topics. Its vocation is not to provide local information. For example, it\nprovides plot, cast, and production information about a given movie, but not\nshowing times in your local movie theatre. Here we describe how we can connect\nlocal information to Wikipedia, without altering its content. The case study we\npresent involves finding local scientific experts. Using a third-party\ntaxonomy, independent from Wikipedia's category hierarchy, we index information\nconnected to our local experts, present in their activity reports, and we\nre-index Wikipedia content using the same taxonomy. The connections between\nWikipedia pages and local expert reports are stored in a relational database,\naccessible through as public SPARQL endpoint. A Wikipedia gadget (or plugin)\nactivated by the interested user, accesses the endpoint as each Wikipedia page\nis accessed. An additional tab on the Wikipedia page allows the user to open up\na list of teams of local experts associated with the subject matter in the\nWikipedia page. The technique, though presented here as a way to identify local\nexperts, is generic, in that any third party taxonomy, can be used in this to\nconnect Wikipedia to any non-Wikipedia data source.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1511.01432v1",
            "date": "2015-11-04 18:48:36+00:00",
            "title": "Semi-supervised Sequence Learning",
            "authors": [
                "Andrew M. Dai",
                "Quoc V. Le"
            ],
            "abstract": "We present two approaches that use unlabeled data to improve sequence\nlearning with recurrent networks. The first approach is to predict what comes\nnext in a sequence, which is a conventional language model in natural language\nprocessing. The second approach is to use a sequence autoencoder, which reads\nthe input sequence into a vector and predicts the input sequence again. These\ntwo algorithms can be used as a \"pretraining\" step for a later supervised\nsequence learning algorithm. In other words, the parameters obtained from the\nunsupervised step can be used as a starting point for other supervised training\nmodels. In our experiments, we find that long short term memory recurrent\nnetworks after being pretrained with the two approaches are more stable and\ngeneralize better. With pretraining, we are able to train long short term\nmemory recurrent networks up to a few hundred timesteps, thereby achieving\nstrong performance in many text classification tasks, such as IMDB, DBpedia and\n20 Newsgroups.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.01974v1",
            "date": "2015-11-06 03:07:20+00:00",
            "title": "Multi-lingual Geoparsing based on Machine Translation",
            "authors": [
                "Xu Chen",
                "Han Zhang",
                "Judith Gelernter"
            ],
            "abstract": "Our method for multi-lingual geoparsing uses monolingual tools and resources\nalong with machine translation and alignment to return location words in many\nlanguages. Not only does our method save the time and cost of developing\ngeoparsers for each language separately, but also it allows the possibility of\na wide range of language capabilities within a single interface. We evaluated\nour method in our LanguageBridge prototype on location named entities using\nnewswire, broadcast news and telephone conversations in English, Arabic and\nChinese data from the Linguistic Data Consortium (LDC). Our results for\ngeoparsing Chinese and Arabic text using our multi-lingual geoparsing method\nare comparable to our results for geoparsing English text with our English\ntools. Furthermore, experiments using our machine translation approach results\nin accuracy comparable to results from the same data that was translated\nmanually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02024v1",
            "date": "2015-11-06 10:29:26+00:00",
            "title": "Towards a Better Understanding of Predict and Count Models",
            "authors": [
                "S. Sathiya Keerthi",
                "Tobias Schnabel",
                "Rajiv Khanna"
            ],
            "abstract": "In a recent paper, Levy and Goldberg pointed out an interesting connection\nbetween prediction-based word embedding models and count models based on\npointwise mutual information. Under certain conditions, they showed that both\nmodels end up optimizing equivalent objective functions. This paper explores\nthis connection in more detail and lays out the factors leading to differences\nbetween these models. We find that the most relevant differences from an\noptimization perspective are (i) predict models work in a low dimensional space\nwhere embedding vectors can interact heavily; (ii) since predict models have\nfewer parameters, they are less prone to overfitting.\n  Motivated by the insight of our analysis, we show how count models can be\nregularized in a principled manner and provide closed-form solutions for L1 and\nL2 regularization. Finally, we propose a new embedding model with a convex\nobjective and the additional benefit of being intelligible.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.02436v2",
            "date": "2015-11-08 03:45:49+00:00",
            "title": "Learning Linguistic Biomarkers for Predicting Mild Cognitive Impairment using Compound Skip-grams",
            "authors": [
                "Sylvester Olubolu Orimaye",
                "Kah Yee Tai",
                "Jojo Sze-Meng Wong",
                "Chee Piau Wong"
            ],
            "abstract": "Predicting Mild Cognitive Impairment (MCI) is currently a challenge as\nexisting diagnostic criteria rely on neuropsychological examinations. Automated\nMachine Learning (ML) models that are trained on verbal utterances of MCI\npatients can aid diagnosis. Using a combination of skip-gram features, our\nmodel learned several linguistic biomarkers to distinguish between 19 patients\nwith MCI and 19 healthy control individuals from the DementiaBank language\ntranscript clinical dataset. Results show that a model with compound of\nskip-grams has better AUC and could help ML prediction on small MCI data\nsample.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02556v1",
            "date": "2015-11-09 02:31:31+00:00",
            "title": "Sentiment Expression via Emoticons on Social Media",
            "authors": [
                "Hao Wang",
                "Jorge A. Castanon"
            ],
            "abstract": "Emoticons (e.g., :) and :( ) have been widely used in sentiment analysis and\nother NLP tasks as features to ma- chine learning algorithms or as entries of\nsentiment lexicons. In this paper, we argue that while emoticons are strong and\ncommon signals of sentiment expression on social media the relationship between\nemoticons and sentiment polarity are not always clear. Thus, any algorithm that\ndeals with sentiment polarity should take emoticons into account but extreme\ncau- tion should be exercised in which emoticons to depend on. First, to\ndemonstrate the prevalence of emoticons on social media, we analyzed the\nfrequency of emoticons in a large re- cent Twitter data set. Then we carried\nout four analyses to examine the relationship between emoticons and sentiment\npolarity as well as the contexts in which emoticons are used. The first\nanalysis surveyed a group of participants for their perceived sentiment\npolarity of the most frequent emoticons. The second analysis examined\nclustering of words and emoti- cons to better understand the meaning conveyed\nby the emoti- cons. The third analysis compared the sentiment polarity of\nmicroblog posts before and after emoticons were removed from the text. The last\nanalysis tested the hypothesis that removing emoticons from text hurts\nsentiment classification by training two machine learning models with and\nwithout emoticons in the text respectively. The results confirms the arguments\nthat: 1) a few emoticons are strong and reliable signals of sentiment polarity\nand one should take advantage of them in any senti- ment analysis; 2) a large\ngroup of the emoticons conveys com- plicated sentiment hence they should be\ntreated with extreme caution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02570v2",
            "date": "2015-11-09 05:25:57+00:00",
            "title": "Explicit Knowledge-based Reasoning for Visual Question Answering",
            "authors": [
                "Peng Wang",
                "Qi Wu",
                "Chunhua Shen",
                "Anton van den Hengel",
                "Anthony Dick"
            ],
            "abstract": "We describe a method for visual question answering which is capable of\nreasoning about contents of an image on the basis of information extracted from\na large-scale knowledge base. The method not only answers natural language\nquestions using concepts not contained in the image, but can provide an\nexplanation of the reasoning by which it developed its answer. The method is\ncapable of answering far more complex questions than the predominant long\nshort-term memory-based approach, and outperforms it significantly in the\ntesting. We also provide a dataset and a protocol by which to evaluate such\nmethods, thus addressing one of the key issues in general visual ques- tion\nanswering.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.02669v1",
            "date": "2015-11-09 13:21:02+00:00",
            "title": "Enacting textual entailment and ontologies for automated essay grading in chemical domain",
            "authors": [
                "Adrian Groza",
                "Roxana Szabo"
            ],
            "abstract": "We propose a system for automated essay grading using ontologies and textual\nentailment. The process of textual entailment is guided by hypotheses, which\nare extracted from a domain ontology. Textual entailment checks if the truth of\nthe hypothesis follows from a given text. We enact textual entailment to\ncompare students answer to a model answer obtained from ontology. We validated\nthe solution against various essays written by students in the chemistry\ndomain.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.03683v5",
            "date": "2015-11-11 21:16:59+00:00",
            "title": "Generative Concatenative Nets Jointly Learn to Write and Classify Reviews",
            "authors": [
                "Zachary C. Lipton",
                "Sharad Vikram",
                "Julian McAuley"
            ],
            "abstract": "A recommender system's basic task is to estimate how users will respond to\nunseen items. This is typically modeled in terms of how a user might rate a\nproduct, but here we aim to extend such approaches to model how a user would\nwrite about the product. To do so, we design a character-level Recurrent Neural\nNetwork (RNN) that generates personalized product reviews. The network\nconvincingly learns styles and opinions of nearly 1000 distinct authors, using\na large corpus of reviews from BeerAdvocate.com. It also tailors reviews to\ndescribe specific items, categories, and star ratings. Using a simple input\nreplication strategy, the Generative Concatenative Network (GCN) preserves the\nsignal of static auxiliary inputs across wide sequence intervals. Without any\nadditional training, the generative model can classify reviews, identifying the\nauthor of the review, the product category, and the sentiment (rating), with\nremarkable accuracy. Our evaluation shows the GCN captures complex dynamics in\ntext, such as the effect of negation, misspellings, slang, and large\nvocabularies gracefully absent any machinery explicitly dedicated to the\npurpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04024v2",
            "date": "2015-11-12 19:32:08+00:00",
            "title": "Multimodal Skip-gram Using Convolutional Pseudowords",
            "authors": [
                "Zachary Seymour",
                "Yingming Li",
                "Zhongfei Zhang"
            ],
            "abstract": "This work studies the representational mapping across multimodal data such\nthat given a piece of the raw data in one modality the corresponding semantic\ndescription in terms of the raw data in another modality is immediately\nobtained. Such a representational mapping can be found in a wide spectrum of\nreal-world applications including image/video retrieval, object recognition,\naction/behavior recognition, and event understanding and prediction. To that\nend, we introduce a simplified training objective for learning multimodal\nembeddings using the skip-gram architecture by introducing convolutional\n\"pseudowords:\" embeddings composed of the additive combination of distributed\nword representations and image features from convolutional neural networks\nprojected into the multimodal space. We present extensive results of the\nrepresentational properties of these embeddings on various word similarity\nbenchmarks to show the promise of this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04108v4",
            "date": "2015-11-12 22:01:54+00:00",
            "title": "LSTM-based Deep Learning Models for Non-factoid Answer Selection",
            "authors": [
                "Ming Tan",
                "Cicero dos Santos",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "In this paper, we apply a general deep learning (DL) framework for the answer\nselection task, which does not depend on manually defined features or\nlinguistic tools. The basic framework is to build the embeddings of questions\nand answers based on bidirectional long short-term memory (biLSTM) models, and\nmeasure their closeness by cosine similarity. We further extend this basic\nmodel in two directions. One direction is to define a more composite\nrepresentation for questions and answers by combining convolutional neural\nnetwork with the basic framework. The other direction is to utilize a simple\nbut efficient attention mechanism in order to generate the answer\nrepresentation according to the question context. Several variations of models\nare provided. The models are examined by two datasets, including TREC-QA and\nInsuranceQA. Experimental results demonstrate that the proposed models\nsubstantially outperform several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04164v3",
            "date": "2015-11-13 05:53:37+00:00",
            "title": "Natural Language Object Retrieval",
            "authors": [
                "Ronghang Hu",
                "Huazhe Xu",
                "Marcus Rohrbach",
                "Jiashi Feng",
                "Kate Saenko",
                "Trevor Darrell"
            ],
            "abstract": "In this paper, we address the task of natural language object retrieval, to\nlocalize a target object within a given image based on a natural language query\nof the object. Natural language object retrieval differs from text-based image\nretrieval task as it involves spatial information about objects within the\nscene and global scene context. To address this issue, we propose a novel\nSpatial Context Recurrent ConvNet (SCRC) model as scoring function on candidate\nboxes for object retrieval, integrating spatial configurations and global\nscene-level contextual information into the network. Our model processes query\ntext, local image descriptors, spatial configurations and global context\nfeatures through a recurrent network, outputs the probability of the query text\nconditioned on each candidate box as a score for the box, and can transfer\nvisual-linguistic knowledge from image captioning domain to our task.\nExperimental results demonstrate that our method effectively utilizes both\nlocal and global information, outperforming previous baseline methods\nsignificantly on different datasets and scenarios, and can exploit large scale\nvision and language datasets for knowledge transfer.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.04646v2",
            "date": "2015-11-15 02:59:22+00:00",
            "title": "Word Embedding based Correlation Model for Question/Answer Matching",
            "authors": [
                "Yikang Shen",
                "Wenge Rong",
                "Nan Jiang",
                "Baolin Peng",
                "Jie Tang",
                "Zhang Xiong"
            ],
            "abstract": "With the development of community based question answering (Q&A) services, a\nlarge scale of Q&A archives have been accumulated and are an important\ninformation and knowledge resource on the web. Question and answer matching has\nbeen attached much importance to for its ability to reuse knowledge stored in\nthese systems: it can be useful in enhancing user experience with recurrent\nquestions. In this paper, we try to improve the matching accuracy by overcoming\nthe lexical gap between question and answer pairs. A Word Embedding based\nCorrelation (WEC) model is proposed by integrating advantages of both the\ntranslation model and word embedding, given a random pair of words, WEC can\nscore their co-occurrence probability in Q&A pairs and it can also leverage the\ncontinuity and smoothness of continuous space word representation to deal with\nnew pairs of words that are rare in the training parallel text. An experimental\nstudy on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new\nmethod's promising potential.",
            "journal ref": "AAAI (2017) 3511--3517",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04747v6",
            "date": "2015-11-15 18:16:20+00:00",
            "title": "Learning Representations of Affect from Speech",
            "authors": [
                "Sayan Ghosh",
                "Eugene Laksana",
                "Louis-Philippe Morency",
                "Stefan Scherer"
            ],
            "abstract": "There has been a lot of prior work on representation learning for speech\nrecognition applications, but not much emphasis has been given to an\ninvestigation of effective representations of affect from speech, where the\nparalinguistic elements of speech are separated out from the verbal content. In\nthis paper, we explore denoising autoencoders for learning paralinguistic\nattributes i.e. categorical and dimensional affective traits from speech. We\nshow that the representations learnt by the bottleneck layer of the autoencoder\nare highly discriminative of activation intensity and at separating out\nnegative valence (sadness and anger) from positive valence (happiness). We\nexperiment with different input speech features (such as FFT and log-mel\nspectrograms with temporal context windows), and different autoencoder\narchitectures (such as stacked and deep autoencoders). We also learn utterance\nspecific representations by a combination of denoising autoencoders and BLSTM\nbased recurrent autoencoders. Emotion classification is performed with the\nlearnt temporal/dynamic representations to evaluate the quality of the\nrepresentations. Experiments on a well-established real-life speech dataset\n(IEMOCAP) show that the learnt representations are comparable to state of the\nart feature extractors (such as voice quality features and MFCCs) and are\ncompetitive with state-of-the-art approaches at emotion and dimensional affect\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.05284v2",
            "date": "2015-11-17 06:44:48+00:00",
            "title": "Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data",
            "authors": [
                "Lisa Anne Hendricks",
                "Subhashini Venugopalan",
                "Marcus Rohrbach",
                "Raymond Mooney",
                "Kate Saenko",
                "Trevor Darrell"
            ],
            "abstract": "While recent deep neural network models have achieved promising results on\nthe image captioning task, they rely largely on the availability of corpora\nwith paired image and sentence captions to describe objects in context. In this\nwork, we propose the Deep Compositional Captioner (DCC) to address the task of\ngenerating descriptions of novel objects which are not present in paired\nimage-sentence datasets. Our method achieves this by leveraging large object\nrecognition datasets and external text corpora and by transferring knowledge\nbetween semantically similar concepts. Current deep caption models can only\ndescribe objects contained in paired image-sentence corpora, despite the fact\nthat they are pre-trained with large object recognition datasets, namely\nImageNet. In contrast, our model can compose sentences that describe novel\nobjects and their interactions with other objects. We demonstrate our model's\nability to describe novel concepts by empirically evaluating its performance on\nMSCOCO and show qualitative results on ImageNet images of objects for which no\npaired image-caption data exist. Further, we extend our approach to generate\ndescriptions of objects in video clips. Our results show that DCC has distinct\nadvantages over existing image and video captioning approaches for generating\ndescriptions of new objects in context.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.05926v1",
            "date": "2015-11-18 20:17:39+00:00",
            "title": "Combining Neural Networks and Log-linear Models to Improve Relation Extraction",
            "authors": [
                "Thien Huu Nguyen",
                "Ralph Grishman"
            ],
            "abstract": "The last decade has witnessed the success of the traditional feature-based\nmethod on exploiting the discrete structures such as words or lexical patterns\nto extract relations from text. Recently, convolutional and recurrent neural\nnetworks has provided very effective mechanisms to capture the hidden\nstructures within sentences via continuous representations, thereby\nsignificantly advancing the performance of relation extraction. The advantage\nof convolutional neural networks is their capacity to generalize the\nconsecutive k-grams in the sentences while recurrent neural networks are\neffective to encode long ranges of sentence context. This paper proposes to\ncombine the traditional feature-based method, the convolutional and recurrent\nneural networks to simultaneously benefit from their advantages. Our systematic\nevaluation of different network architectures and combination methods\ndemonstrates the effectiveness of this approach and results in the\nstate-of-the-art performance on the ACE 2005 and SemEval dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06018v2",
            "date": "2015-11-18 23:02:45+00:00",
            "title": "Segmental Recurrent Neural Networks",
            "authors": [
                "Lingpeng Kong",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We introduce segmental recurrent neural networks (SRNNs) which define, given\nan input sequence, a joint probability distribution over segmentations of the\ninput and labelings of the segments. Representations of the input segments\n(i.e., contiguous subsequences of the input) are computed by encoding their\nconstituent tokens using bidirectional recurrent neural nets, and these\n\"segment embeddings\" are used to define compatibility scores with output\nlabels. These local compatibility scores are integrated using a global\nsemi-Markov conditional random field. Both fully supervised training -- in\nwhich segment boundaries and labels are observed -- as well as partially\nsupervised training -- in which segment boundaries are latent -- are\nstraightforward. Experiments on handwriting recognition and joint Chinese word\nsegmentation/POS tagging show that, compared to models that do not explicitly\nrepresent segments such as BIO tagging schemes and connectionist temporal\nclassification (CTC), SRNNs obtain substantially higher accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06066v1",
            "date": "2015-11-19 05:54:45+00:00",
            "title": "Transfer Learning for Speech and Language Processing",
            "authors": [
                "Dong Wang",
                "Thomas Fang Zheng"
            ],
            "abstract": "Transfer learning is a vital technique that generalizes models trained for\none setting or task to other settings or tasks. For example in speech\nrecognition, an acoustic model trained for one language can be used to\nrecognize speech in another language, with little or no re-training data.\nTransfer learning is closely related to multi-task learning (cross-lingual vs.\nmultilingual), and is traditionally studied in the name of `model adaptation'.\nRecent advance in deep learning shows that transfer learning becomes much\neasier and more effective with high-level abstract features learned by deep\nmodels, and the `transfer' can be conducted not only between data distributions\nand data types, but also between model structures (e.g., shallow nets and deep\nnets) or even model types (e.g., Bayesian models and neural models). This\nreview paper summarizes some recent prominent research towards this direction,\nparticularly for speech and language processing. We also report some results\nfrom our group and highlight the potential of this very interesting research\nfield.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06219v2",
            "date": "2015-11-19 15:51:31+00:00",
            "title": "Knowledge Base Population using Semantic Label Propagation",
            "authors": [
                "Lucas Sterckx",
                "Thomas Demeester",
                "Johannes Deleu",
                "Chris Develder"
            ],
            "abstract": "A crucial aspect of a knowledge base population system that extracts new\nfacts from text corpora, is the generation of training data for its relation\nextractors. In this paper, we present a method that maximizes the effectiveness\nof newly trained relation extractors at a minimal annotation cost. Manual\nlabeling can be significantly reduced by Distant Supervision, which is a method\nto construct training data automatically by aligning a large text corpus with\nan existing knowledge base of known facts. For example, all sentences\nmentioning both 'Barack Obama' and 'US' may serve as positive training\ninstances for the relation born_in(subject,object). However, distant\nsupervision typically results in a highly noisy training set: many training\nsentences do not really express the intended relation. We propose to combine\ndistant supervision with minimal manual supervision in a technique called\nfeature labeling, to eliminate noise from the large and noisy initial training\nset, resulting in a significant increase of precision. We further improve on\nthis approach by introducing the Semantic Label Propagation method, which uses\nthe similarity between low-dimensional representations of candidate training\ninstances, to extend the training set in order to increase recall while\nmaintaining high precision. Our proposed strategy for generating training data\nis studied and evaluated on an established test collection designed for\nknowledge base population tasks. The experimental results show that the\nSemantic Label Propagation strategy leads to substantial performance gains when\ncompared to existing approaches, while requiring an almost negligible manual\nannotation effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06285v1",
            "date": "2015-11-18 15:26:06+00:00",
            "title": "Harvesting comparable corpora and mining them for equivalent bilingual sentences using statistical classification and analogy- based heuristics",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Emilia Rejmund",
                "Krzysztof Marasek"
            ],
            "abstract": "Parallel sentences are a relatively scarce but extremely useful resource for\nmany applications including cross-lingual retrieval and statistical machine\ntranslation. This research explores our new methodologies for mining such data\nfrom previously obtained comparable corpora. The task is highly practical since\nnon-parallel multilingual data exist in far greater quantities than parallel\ncorpora, but parallel sentences are a much more useful resource. Here we\npropose a web crawling method for building subject-aligned comparable corpora\nfrom e.g. Wikipedia dumps and Euronews web page. The improvements in machine\ntranslation are shown on Polish-English language pair for various text domains.\nWe also tested another method of building parallel corpora based on comparable\ncorpora data. It lets automatically broad existing corpus of sentences from\nsubject of corpora based on analogies between them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06303v2",
            "date": "2015-11-19 18:46:21+00:00",
            "title": "Alternative structures for character-level RNNs",
            "authors": [
                "Piotr Bojanowski",
                "Armand Joulin",
                "Tomas Mikolov"
            ],
            "abstract": "Recurrent neural networks are convenient and efficient models for language\nmodeling. However, when applied on the level of characters instead of words,\nthey suffer from several problems. In order to successfully model long-term\ndependencies, the hidden representation needs to be large. This in turn implies\nhigher computational costs, which can become prohibitive in practice. We\npropose two alternative structural modifications to the classical RNN model.\nThe first one consists on conditioning the character level representation on\nthe previous word representation. The other one uses the character history to\ncondition the output probability. We evaluate the performance of the two\nproposed modifications on challenging, multi-lingual real world data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06341v4",
            "date": "2015-11-19 20:14:43+00:00",
            "title": "Communicating Semantics: Reference by Description",
            "authors": [
                "Ramanathan V Guha",
                "Vineet Gupta"
            ],
            "abstract": "Messages often refer to entities such as people, places and events. Correct\nidentification of the intended reference is an essential part of communication.\nLack of shared unique names often complicates entity reference. Shared\nknowledge can be used to construct uniquely identifying descriptive references\nfor entities with ambiguous names. We introduce a mathematical model for\n`Reference by Description', derive results on the conditions under which, with\nhigh probability, programs can construct unambiguous references to most\nentities in the domain of discourse and provide empirical validation of these\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06349v4",
            "date": "2015-11-19 20:38:45+00:00",
            "title": "Generating Sentences from a Continuous Space",
            "authors": [
                "Samuel R. Bowman",
                "Luke Vilnis",
                "Oriol Vinyals",
                "Andrew M. Dai",
                "Rafal Jozefowicz",
                "Samy Bengio"
            ],
            "abstract": "The standard recurrent neural network language model (RNNLM) generates\nsentences one word at a time and does not work from an explicit global sentence\nrepresentation. In this work, we introduce and study an RNN-based variational\nautoencoder generative model that incorporates distributed latent\nrepresentations of entire sentences. This factorization allows it to explicitly\nmodel holistic properties of sentences such as style, topic, and high-level\nsyntactic features. Samples from the prior over these sentence representations\nremarkably produce diverse and well-formed sentences through simple\ndeterministic decoding. By examining paths through this latent space, we are\nable to generate coherent novel sentences that interpolate between known\nsentences. We present techniques for solving the difficult learning problem\npresented by this model, demonstrate its effectiveness in imputing missing\nwords, explore many interesting properties of the model's latent sentence\nspace, and present negative results on the use of the model in language\nmodeling.",
            "journal ref": "SIGNLL Conference on Computational Natural Language Learning\n  (CONLL), 2016",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06379v1",
            "date": "2015-11-19 21:07:27+00:00",
            "title": "Dynamic Adaptive Network Intelligence",
            "authors": [
                "Richard Searle",
                "Megan Bingham-Walker"
            ],
            "abstract": "Accurate representational learning of both the explicit and implicit\nrelationships within data is critical to the ability of machines to perform\nmore complex and abstract reasoning tasks. We describe the efficient weakly\nsupervised learning of such inferences by our Dynamic Adaptive Network\nIntelligence (DANI) model. We report state-of-the-art results for DANI over\nquestion answering tasks in the bAbI dataset that have proved difficult for\ncontemporary approaches to learning representation (Weston et al., 2015).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06388v1",
            "date": "2015-11-19 21:22:42+00:00",
            "title": "sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings",
            "authors": [
                "Andrew Trask",
                "Phil Michalak",
                "John Liu"
            ],
            "abstract": "Neural word representations have proven useful in Natural Language Processing\n(NLP) tasks due to their ability to efficiently model complex semantic and\nsyntactic word relationships. However, most techniques model only one\nrepresentation per word, despite the fact that a single word can have multiple\nmeanings or \"senses\". Some techniques model words by using multiple vectors\nthat are clustered based on context. However, recent neural approaches rarely\nfocus on the application to a consuming NLP algorithm. Furthermore, the\ntraining process of recent word-sense models is expensive relative to\nsingle-sense embedding processes. This paper presents a novel approach which\naddresses these concerns by modeling multiple embeddings for each word based on\nsupervised disambiguation, which provides a fast and accurate way for a\nconsuming NLP model to select a sense-disambiguated embedding. We demonstrate\nthat these embeddings can disambiguate both contrastive senses such as nominal\nand verbal senses as well as nuanced senses such as sarcasm. We further\nevaluate Part-of-Speech disambiguated embeddings on neural dependency parsing,\nyielding a greater than 8% average error reduction in unlabeled attachment\nscores across 6 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06396v2",
            "date": "2015-11-19 21:42:23+00:00",
            "title": "Multilingual Relation Extraction using Compositional Universal Schema",
            "authors": [
                "Patrick Verga",
                "David Belanger",
                "Emma Strubell",
                "Benjamin Roth",
                "Andrew McCallum"
            ],
            "abstract": "Universal schema builds a knowledge base (KB) of entities and relations by\njointly embedding all relation types from input KBs as well as textual patterns\nexpressing relations from raw text. In most previous applications of universal\nschema, each textual pattern is represented as a single embedding, preventing\ngeneralization to unseen patterns. Recent work employs a neural network to\ncapture patterns' compositional semantics, providing generalization to all\npossible input text. In response, this paper introduces significant further\nimprovements to the coverage and flexibility of universal schema relation\nextraction: predictions for entities unseen in training and multilingual\ntransfer learning to domains with no annotation. We evaluate our model through\nextensive experiments on the English and Spanish TAC KBP benchmark,\noutperforming the top system from TAC 2013 slot-filling using no handwritten\npatterns or additional annotation. We also consider a multilingual setting in\nwhich English training data entities overlap with the seed KB, but Spanish text\ndoes not. Despite having no annotation for Spanish data, we train an accurate\npredictor, with additional improvements obtained by tying word embeddings\nacross languages. Furthermore, we find that multilingual training improves\nEnglish relation extraction accuracy. Our approach is thus suited to\nbroad-coverage automated knowledge base construction in a variety of languages\nand domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06397v2",
            "date": "2015-11-19 21:42:47+00:00",
            "title": "Compressing Word Embeddings",
            "authors": [
                "Martin Andrews"
            ],
            "abstract": "Recent methods for learning vector space representations of words have\nsucceeded in capturing fine-grained semantic and syntactic regularities using\nvector arithmetic. However, these vector space representations (created through\nlarge-scale text analysis) are typically stored verbatim, since their internal\nstructure is opaque. Using word-analogy tests to monitor the level of detail\nstored in compressed re-representations of the same vector space, the\ntrade-offs between the reduction in memory usage and expressiveness are\ninvestigated. A simple scheme is outlined that can reduce the memory footprint\nof a state-of-the-art embedding by a factor of 10, with only minimal impact on\nperformance. Then, using the same `bit budget', a binary (approximate)\nfactorisation of the same space is also explored, with the aim of creating an\nequivalent representation with better interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06407v2",
            "date": "2015-11-19 21:56:53+00:00",
            "title": "Recurrent Models for Auditory Attention in Multi-Microphone Distance Speech Recognition",
            "authors": [
                "Suyoun Kim",
                "Ian Lane"
            ],
            "abstract": "Integration of multiple microphone data is one of the key ways to achieve\nrobust speech recognition in noisy environments or when the speaker is located\nat some distance from the input device. Signal processing techniques such as\nbeamforming are widely used to extract a speech signal of interest from\nbackground noise. These techniques, however, are highly dependent on prior\nspatial information about the microphones and the environment in which the\nsystem is being used. In this work, we present a neural attention network that\ndirectly combines multi-channel audio to generate phonetic states without\nrequiring any prior knowledge of the microphone layout or any explicit signal\npreprocessing for speech enhancement. We embed an attention mechanism within a\nRecurrent Neural Network (RNN) based acoustic model to automatically tune its\nattention to a more reliable input source. Unlike traditional multi-channel\npreprocessing, our system can be optimized towards the desired output in one\nstep. Although attention-based models have recently achieved impressive results\non sequence-to-sequence learning, no attention mechanisms have previously been\napplied to learn potentially asynchronous and non-stationary multiple inputs.\nWe evaluate our neural attention model on the CHiME-3 challenge task, and show\nthat the model achieves comparable performance to beamforming using a purely\ndata-driven method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06438v1",
            "date": "2015-11-19 22:58:10+00:00",
            "title": "Joint Word Representation Learning using a Corpus and a Semantic Lexicon",
            "authors": [
                "Danushka Bollegala",
                "Alsuhaibani Mohammed",
                "Takanori Maehara",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Methods for learning word representations using large text corpora have\nreceived much attention lately due to their impressive performance in numerous\nnatural language processing (NLP) tasks such as, semantic similarity\nmeasurement, and word analogy detection. Despite their success, these\ndata-driven word representation learning methods do not consider the rich\nsemantic relational structure between words in a co-occurring context. On the\nother hand, already much manual effort has gone into the construction of\nsemantic lexicons such as the WordNet that represent the meanings of words by\ndefining the various relationships that exist among the words in a language. We\nconsider the question, can we improve the word representations learnt using a\ncorpora by integrating the knowledge from semantic lexicons?. For this purpose,\nwe propose a joint word representation learning method that simultaneously\npredicts the co-occurrences of two words in a sentence subject to the\nrelational constrains given by the semantic lexicon. We use relations that\nexist between words in the lexicon to regularize the word representations\nlearnt from the corpus. Our proposed method statistically significantly\noutperforms previously proposed methods for incorporating semantic lexicons\ninto word representations on several benchmark datasets for semantic similarity\nand word analogy.",
            "journal ref": "Proceedings of the AAAI 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06674v1",
            "date": "2015-11-20 16:33:13+00:00",
            "title": "Stories in the Eye: Contextual Visual Interactions for Efficient Video to Language Translation",
            "authors": [
                "Anirudh Goyal",
                "Marius Leordeanu"
            ],
            "abstract": "Integrating higher level visual and linguistic interpretations is at the\nheart of human intelligence. As automatic visual category recognition in images\nis approaching human performance, the high level understanding in the dynamic\nspatiotemporal domain of videos and its translation into natural language is\nstill far from being solved. While most works on vision-to-text translations\nuse pre-learned or pre-established computational linguistic models, in this\npaper we present an approach that uses vision alone to efficiently learn how to\ntranslate into language the video content. We discover, in simple form, the\nstory played by main actors, while using only visual cues for representing\nobjects and their interactions. Our method learns in a hierarchical manner\nhigher level representations for recognizing subjects, actions and objects\ninvolved, their relevant contextual background and their interaction to one\nanother over time. We have a three stage approach: first we take in\nconsideration features of the individual entities at the local level of\nappearance, then we consider the relationship between these objects and actions\nand their video background, and third, we consider their spatiotemporal\nrelations as inputs to classifiers at the highest level of interpretation.\nThus, our approach finds a coherent linguistic description of videos in the\nform of a subject, verb and object based on their role played in the overall\nvisual story learned directly from training data, without using a known\nlanguage model. We test the efficiency of our approach on a large scale dataset\ncontaining YouTube clips taken in the wild and demonstrate state-of-the-art\nperformance, often superior to current approaches that use more complex,\npre-learned linguistic knowledge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.06732v7",
            "date": "2015-11-20 19:25:54+00:00",
            "title": "Sequence Level Training with Recurrent Neural Networks",
            "authors": [
                "Marc'Aurelio Ranzato",
                "Sumit Chopra",
                "Michael Auli",
                "Wojciech Zaremba"
            ],
            "abstract": "Many natural language processing applications use language models to generate\ntext. These models are typically trained to predict the next word in a\nsequence, given the previous words and some context such as an image. However,\nat test time the model is expected to generate the entire sequence from\nscratch. This discrepancy makes generation brittle, as errors may accumulate\nalong the way. We address this issue by proposing a novel sequence level\ntraining algorithm that directly optimizes the metric used at test time, such\nas BLEU or ROUGE. On three different tasks, our approach outperforms several\nstrong baselines for greedy generation. The method is also competitive when\nthese baselines employ beam search, while being several times faster.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06833v1",
            "date": "2015-11-21 04:11:44+00:00",
            "title": "Semi-supervised Bootstrapping approach for Named Entity Recognition",
            "authors": [
                "S. Thenmalar",
                "J. Balaji",
                "T. V. Geetha"
            ],
            "abstract": "The aim of Named Entity Recognition (NER) is to identify references of named\nentities in unstructured documents, and to classify them into pre-defined\nsemantic categories. NER often aids from added background knowledge in the form\nof gazetteers. However using such a collection does not deal with name variants\nand cannot resolve ambiguities associated in identifying the entities in\ncontext and associating them with predefined categories. We present a\nsemi-supervised NER approach that starts with identifying named entities with a\nsmall set of training data. Using the identified named entities, the word and\nthe context features are used to define the pattern. This pattern of each named\nentity category is used as a seed pattern to identify the named entities in the\ntest set. Pattern scoring and tuple value score enables the generation of the\nnew patterns to identify the named entity categories. We have evaluated the\nproposed system for English language with the dataset of tagged (IEER) and\nuntagged (CoNLL 2003) named entity corpus and for Tamil language with the\ndocuments from the FIRE corpus and yield an average f-measure of 75% for both\nthe languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06838v1",
            "date": "2015-11-21 04:58:46+00:00",
            "title": "Mapping Images to Sentiment Adjective Noun Pairs with Factorized Neural Nets",
            "authors": [
                "Takuya Narihira",
                "Damian Borth",
                "Stella X. Yu",
                "Karl Ni",
                "Trevor Darrell"
            ],
            "abstract": "We consider the visual sentiment task of mapping an image to an adjective\nnoun pair (ANP) such as \"cute baby\". To capture the two-factor structure of our\nANP semantics as well as to overcome annotation noise and ambiguity, we propose\na novel factorized CNN model which learns separate representations for\nadjectives and nouns but optimizes the classification performance over their\nproduct. Our experiments on the publicly available SentiBank dataset show that\nour model significantly outperforms not only independent ANP classifiers on\nunseen ANPs and on retrieving images of novel ANPs, but also image captioning\nmodels which capture word semantics from co-occurrence of natural text; the\nlatter turn out to be surprisingly poor at capturing the sentiment evoked by\npure visual experience. That is, our factorized ANP CNN not only trains better\nfrom noisy labels, generalizes better to new images, but can also expands the\nANP vocabulary on its own.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.06931v6",
            "date": "2015-11-21 22:26:49+00:00",
            "title": "Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems",
            "authors": [
                "Jesse Dodge",
                "Andreea Gane",
                "Xiang Zhang",
                "Antoine Bordes",
                "Sumit Chopra",
                "Alexander Miller",
                "Arthur Szlam",
                "Jason Weston"
            ],
            "abstract": "A long-term goal of machine learning is to build intelligent conversational\nagents. One recent popular approach is to train end-to-end models on a large\namount of real dialog transcripts between humans (Sordoni et al., 2015; Vinyals\n& Le, 2015; Shang et al., 2015). However, this approach leaves many questions\nunanswered as an understanding of the precise successes and shortcomings of\neach model is hard to assess. A contrasting recent proposal are the bAbI tasks\n(Weston et al., 2015b) which are synthetic data that measure the ability of\nlearning machines at various reasoning tasks over toy language. Unfortunately,\nthose tests are very small and hence may encourage methods that do not scale.\nIn this work, we propose a suite of new tasks of a much larger scale that\nattempt to bridge the gap between the two regimes. Choosing the domain of\nmovies, we provide tasks that test the ability of models to answer factual\nquestions (utilizing OMDB), provide personalization (utilizing MovieLens),\ncarry short conversations about the two, and finally to perform on natural\ndialogs from Reddit. We provide a dataset covering 75k movie entities and with\n3.5M training examples. We present results of various models on these tasks,\nand evaluate their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06961v1",
            "date": "2015-11-22 04:28:39+00:00",
            "title": "On the Linear Algebraic Structure of Distributed Word Representations",
            "authors": [
                "Lisa Seung-Yeon Lee"
            ],
            "abstract": "In this work, we leverage the linear algebraic structure of distributed word\nrepresentations to automatically extend knowledge bases and allow a machine to\nlearn new facts about the world. Our goal is to extract structured facts from\ncorpora in a simpler manner, without applying classifiers or patterns, and\nusing only the co-occurrence statistics of words. We demonstrate that the\nlinear algebraic structure of word embeddings can be used to reduce data\nrequirements for methods of learning facts. In particular, we demonstrate that\nwords belonging to a common category, or pairs of words satisfying a certain\nrelation, form a low-rank subspace in the projected space. We compute a basis\nfor this low-rank subspace using singular value decomposition (SVD), then use\nthis basis to discover new facts and to fit vectors for less frequent words\nwhich we do not yet have vectors for.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06995v1",
            "date": "2015-11-22 11:28:26+00:00",
            "title": "Non-Sentential Utterances in Dialogue: Experiments in Classification and Interpretation",
            "authors": [
                "Paolo Dragone"
            ],
            "abstract": "Non-sentential utterances (NSUs) are utterances that lack a complete\nsentential form but whose meaning can be inferred from the dialogue context,\nsuch as \"OK\", \"where?\", \"probably at his apartment\". The interpretation of\nnon-sentential utterances is an important problem in computational linguistics\nsince they constitute a frequent phenomena in dialogue and they are\nintrinsically context-dependent. The interpretation of NSUs is the task of\nretrieving their full semantic content from their form and the dialogue\ncontext. The first half of this thesis is devoted to the NSU classification\ntask. Our work builds upon Fern\\'andez et al. (2007) which present a series of\nmachine-learning experiments on the classification of NSUs. We extended their\napproach with a combination of new features and semi-supervised learning\ntechniques. The empirical results presented in this thesis show a modest but\nsignificant improvement over the state-of-the-art classification performance.\nThe consecutive, yet independent, problem is how to infer an appropriate\nsemantic representation of such NSUs on the basis of the dialogue context.\nFern\\'andez (2006) formalizes this task in terms of \"resolution rules\" built on\ntop of the Type Theory with Records (TTR). Our work is focused on the\nreimplementation of the resolution rules from Fern\\'andez (2006) with a\nprobabilistic account of the dialogue state. The probabilistic rules formalism\nLison (2014) is particularly suited for this task because, similarly to the\nframework developed by Ginzburg (2012) and Fern\\'andez (2006), it involves the\nspecification of update rules on the variables of the dialogue state to capture\nthe dynamics of the conversation. However, the probabilistic rules can also\nencode probabilistic knowledge, thereby providing a principled account of\nambiguities in the NSU resolution process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.07067v2",
            "date": "2015-11-22 20:46:42+00:00",
            "title": "Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes",
            "authors": [
                "Satwik Kottur",
                "Ramakrishna Vedantam",
                "Jos\u00e9 M. F. Moura",
                "Devi Parikh"
            ],
            "abstract": "We propose a model to learn visually grounded word embeddings (vis-w2v) to\ncapture visual notions of semantic relatedness. While word embeddings trained\nusing text have been extremely successful, they cannot uncover notions of\nsemantic relatedness implicit in our visual world. For instance, although\n\"eats\" and \"stares at\" seem unrelated in text, they share semantics visually.\nWhen people are eating something, they also tend to stare at the food.\nGrounding diverse relations like \"eats\" and \"stares at\" into vision remains\nchallenging, despite recent progress in vision. We note that the visual\ngrounding of words depends on semantics, and not the literal pixels. We thus\nuse abstract scenes created from clipart to provide the visual grounding. We\nfind that the embeddings we learn capture fine-grained, visually grounded\nnotions of semantic relatedness. We show improvements over text-only word\nembeddings (word2vec) on three tasks: common-sense assertion classification,\nvisual paraphrasing and text-based image retrieval. Our code and datasets are\navailable online.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.07916v1",
            "date": "2015-11-24 23:23:13+00:00",
            "title": "Natural Language Understanding with Distributed Representation",
            "authors": [
                "Kyunghyun Cho"
            ],
            "abstract": "This is a lecture note for the course DS-GA 3001 <Natural Language\nUnderstanding with Distributed Representation> at the Center for Data Science ,\nNew York University in Fall, 2015. As the name of the course suggests, this\nlecture note introduces readers to a neural network based approach to natural\nlanguage understanding/processing. In order to make it as self-contained as\npossible, I spend much time on describing basics of machine learning and neural\nnetworks, only after which how they are used for natural languages is\nintroduced. On the language front, I almost solely focus on language modelling\nand machine translation, two of which I personally find most fascinating and\nmost fundamental to natural language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08130v2",
            "date": "2015-11-25 17:32:18+00:00",
            "title": "A Roadmap towards Machine Intelligence",
            "authors": [
                "Tomas Mikolov",
                "Armand Joulin",
                "Marco Baroni"
            ],
            "abstract": "The development of intelligent machines is one of the biggest unsolved\nchallenges in computer science. In this paper, we propose some fundamental\nproperties these machines should have, focusing in particular on communication\nand learning. We discuss a simple environment that could be used to\nincrementally teach a machine the basics of natural-language-based\ncommunication, as a prerequisite to more complex interaction with human users.\nWe also present some conjectures on the sort of algorithms the machine should\nsupport in order to profitably learn from the environment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.08198v3",
            "date": "2015-11-25 20:52:15+00:00",
            "title": "Towards Universal Paraphrastic Sentence Embeddings",
            "authors": [
                "John Wieting",
                "Mohit Bansal",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We consider the problem of learning general-purpose, paraphrastic sentence\nembeddings based on supervision from the Paraphrase Database (Ganitkevitch et\nal., 2013). We compare six compositional architectures, evaluating them on\nannotated textual similarity datasets drawn both from the same distribution as\nthe training data and from a wide range of other domains. We find that the most\ncomplex architectures, such as long short-term memory (LSTM) recurrent neural\nnetworks, perform best on the in-domain data. However, in out-of-domain\nscenarios, simple architectures such as word averaging vastly outperform LSTMs.\nOur simplest averaging model is even competitive with systems tuned for the\nparticular tasks while also being extremely efficient and easy to use.\n  In order to better understand how these architectures compare, we conduct\nfurther experiments on three supervised NLP tasks: sentence similarity,\nentailment, and sentiment classification. We again find that the word averaging\nmodels perform well for sentence similarity and entailment, outperforming\nLSTMs. However, on sentiment classification, we find that the LSTM performs\nvery strongly-even recording new state-of-the-art performance on the Stanford\nSentiment Treebank.\n  We then demonstrate how to combine our pretrained sentence embeddings with\nthese supervised tasks, using them both as a prior and as a black box feature\nextractor. This leads to performance rivaling the state of the art on the SICK\nsimilarity and entailment tasks. We release all of our resources to the\nresearch community with the hope that they can serve as the new baseline for\nfurther work on universal sentence embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08407v4",
            "date": "2015-11-26 14:58:17+00:00",
            "title": "The Mechanism of Additive Composition",
            "authors": [
                "Ran Tian",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "Additive composition (Foltz et al, 1998; Landauer and Dumais, 1997; Mitchell\nand Lapata, 2010) is a widely used method for computing meanings of phrases,\nwhich takes the average of vector representations of the constituent words. In\nthis article, we prove an upper bound for the bias of additive composition,\nwhich is the first theoretical analysis on compositional frameworks from a\nmachine learning point of view. The bound is written in terms of collocation\nstrength; we prove that the more exclusively two successive words tend to occur\ntogether, the more accurate one can guarantee their additive composition as an\napproximation to the natural phrase vector. Our proof relies on properties of\nnatural language data that are empirically verified, and can be theoretically\nderived from an assumption that the data is generated from a Hierarchical\nPitman-Yor Process. The theory endorses additive composition as a reasonable\noperation for calculating meanings of phrases, and suggests ways to improve\nadditive compositionality, including: transforming entries of distributional\nword vectors by a function that meets a specific condition, constructing a\nnovel type of vector representations to make additive composition sensitive to\nword order, and utilizing singular value decomposition to train word vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08417v1",
            "date": "2015-11-26 15:22:54+00:00",
            "title": "TGSum: Build Tweet Guided Multi-Document Summarization Dataset",
            "authors": [
                "Ziqiang Cao",
                "Chengyao Chen",
                "Wenjie Li",
                "Sujian Li",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "The development of summarization research has been significantly hampered by\nthe costly acquisition of reference summaries. This paper proposes an effective\nway to automatically collect large scales of news-related multi-document\nsummaries with reference to social media's reactions. We utilize two types of\nsocial labels in tweets, i.e., hashtags and hyper-links. Hashtags are used to\ncluster documents into different topic sets. Also, a tweet with a hyper-link\noften highlights certain key points of the corresponding document. We\nsynthesize a linked document cluster to form a reference summary which can\ncover most key points. To this aim, we adopt the ROUGE metrics to measure the\ncoverage ratio, and develop an Integer Linear Programming solution to discover\nthe sentence set reaching the upper bound of ROUGE. Since we allow summary\nsentences to be selected from both documents and high-quality tweets, the\ngenerated reference summaries could be abstractive. Both informativeness and\nreadability of the collected summaries are verified by manual judgment. In\naddition, we train a Support Vector Regression summarizer on DUC generic\nmulti-document summarization benchmarks. With the collected data as extra\ntraining resource, the performance of the summarizer improves a lot on all the\ntest sets. We release this dataset for further research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1511.08952v2",
            "date": "2015-11-29 00:49:13+00:00",
            "title": "Bootstrapping Ternary Relation Extractors",
            "authors": [
                "Ndapandula Nakashole"
            ],
            "abstract": "Binary relation extraction methods have been widely studied in recent years.\nHowever, few methods have been developed for higher n-ary relation extraction.\nOne limiting factor is the effort required to generate training data. For\nbinary relations, one only has to provide a few dozen pairs of entities per\nrelation, as training data. For ternary relations (n=3), each training instance\nis a triplet of entities, placing a greater cognitive load on people. For\nexample, many people know that Google acquired Youtube but not the dollar\namount or the date of the acquisition and many people know that Hillary Clinton\nis married to Bill Clinton by not the location or date of their wedding. This\nmakes higher n-nary training data generation a time consuming exercise in\nsearching the Web. We present a resource for training ternary relation\nextractors. This was generated using a minimally supervised yet effective\napproach. We present statistics on the size and the quality of the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.09173v1",
            "date": "2015-11-30 06:15:31+00:00",
            "title": "Recognizing Temporal Linguistic Expression Pattern of Individual with Suicide Risk on Social Media",
            "authors": [
                "Aiqi Zhang",
                "Ang Li",
                "Tingshao Zhu"
            ],
            "abstract": "Suicide is a global public health problem. Early detection of individual\nsuicide risk plays a key role in suicide prevention. In this paper, we propose\nto look into individual suicide risk through time series analysis of personal\nlinguistic expression on social media (Weibo). We examined temporal patterns of\nthe linguistic expression of individuals on Chinese social media (Weibo). Then,\nwe used such temporal patterns as predictor variables to build classification\nmodels for estimating levels of individual suicide risk. Characteristics of\ntime sequence curves to linguistic features including parentheses, auxiliary\nverbs, personal pronouns and body words are reported to affect performance of\nsuicide most, and the predicting model has a accuracy higher than 0.60, shown\nby the results. This paper confirms the efficiency of the social media data in\ndetecting individual suicide risk. Results of this study may be insightful for\nimproving the performance of suicide prevention programs.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1511.09376v1",
            "date": "2015-11-30 16:32:58+00:00",
            "title": "Modeling Dynamic Relationships Between Characters in Literary Novels",
            "authors": [
                "Snigdha Chaturvedi",
                "Shashank Srivastava",
                "Hal Daume III",
                "Chris Dyer"
            ],
            "abstract": "Studying characters plays a vital role in computationally representing and\ninterpreting narratives. Unlike previous work, which has focused on inferring\ncharacter roles, we focus on the problem of modeling their relationships.\nRather than assuming a fixed relationship for a character pair, we hypothesize\nthat relationships are dynamic and temporally evolve with the progress of the\nnarrative, and formulate the problem of relationship modeling as a structured\nprediction problem. We propose a semi-supervised framework to learn\nrelationship sequences from fully as well as partially labeled data. We present\na Markovian model capable of accumulating historical beliefs about the\nrelationship and status changes. We use a set of rich linguistic and\nsemantically motivated features that incorporate world knowledge to investigate\nthe textual content of narrative. We empirically demonstrate that such a\nframework outperforms competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.09392v1",
            "date": "2015-11-18 15:34:21+00:00",
            "title": "Enhancements in statistical spoken language translation by de-normalization of ASR results",
            "authors": [
                "Agnieszka Wo\u0142k",
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "Spoken language translation (SLT) has become very important in an\nincreasingly globalized world. Machine translation (MT) for automatic speech\nrecognition (ASR) systems is a major challenge of great interest. This research\ninvestigates that automatic sentence segmentation of speech that is important\nfor enriching speech recognition output and for aiding downstream language\nprocessing. This article focuses on the automatic sentence segmentation of\nspeech and improving MT results. We explore the problem of identifying sentence\nboundaries in the transcriptions produced by automatic speech recognition\nsystems in the Polish language. We also experiment with reverse normalization\nof the recognized speech samples.",
            "journal ref": "Journal of Computers, 2016 VOL 11, ISSN: 1796-203X, p. 33-40, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.09460v1",
            "date": "2015-11-30 20:37:03+00:00",
            "title": "Ask, and shall you receive?: Understanding Desire Fulfillment in Natural Language Text",
            "authors": [
                "Snigdha Chaturvedi",
                "Dan Goldwasser",
                "Hal Daume III"
            ],
            "abstract": "The ability to comprehend wishes or desires and their fulfillment is\nimportant to Natural Language Understanding. This paper introduces the task of\nidentifying if a desire expressed by a subject in a given short piece of text\nwas fulfilled. We propose various unstructured and structured models that\ncapture fulfillment cues such as the subject's emotional state and actions. Our\nexperiments with two different datasets demonstrate the importance of\nunderstanding the narrative and discourse structure to address this task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1502.00512v1",
            "date": "2015-02-02 15:27:37+00:00",
            "title": "Scaling Recurrent Neural Network Language Models",
            "authors": [
                "Will Williams",
                "Niranjani Prasad",
                "David Mrva",
                "Tom Ash",
                "Tony Robinson"
            ],
            "abstract": "This paper investigates the scaling properties of Recurrent Neural Network\nLanguage Models (RNNLMs). We discuss how to train very large RNNs on GPUs and\naddress the questions of how RNNLMs scale with respect to model size,\ntraining-set size, computational costs and memory. Our analysis shows that\ndespite being more costly to train, RNNLMs obtain much lower perplexities on\nstandard benchmarks than n-gram models. We train the largest known RNNs and\npresent relative word error rates gains of 18% on an ASR task. We also present\nthe new lowest perplexities on the recently released billion word language\nmodelling benchmark, 1 BLEU point gain on machine translation and a 17%\nrelative hit rate gain in word prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01710v5",
            "date": "2015-02-05 20:45:19+00:00",
            "title": "Text Understanding from Scratch",
            "authors": [
                "Xiang Zhang",
                "Yann LeCun"
            ],
            "abstract": "This article demontrates that we can apply deep learning to text\nunderstanding from character-level inputs all the way up to abstract text\nconcepts, using temporal convolutional networks (ConvNets). We apply ConvNets\nto various large-scale datasets, including ontology classification, sentiment\nanalysis, and text categorization. We show that temporal ConvNets can achieve\nastonishing performance without the knowledge of words, phrases, sentences and\nany other syntactic or semantic structures with regards to a human language.\nEvidence shows that our models can work for both English and Chinese.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1502.02233v1",
            "date": "2015-02-08 10:25:20+00:00",
            "title": "Hierarchical Dirichlet process for tracking complex topical structure evolution and its application to autism research literature",
            "authors": [
                "Adham Beykikhoshk",
                "Ognjen Arandjelovic",
                "Dinh Phung",
                "Svetha Venkatesh"
            ],
            "abstract": "In this paper we describe a novel framework for the discovery of the topical\ncontent of a data corpus, and the tracking of its complex structural changes\nacross the temporal dimension. In contrast to previous work our model does not\nimpose a prior on the rate at which documents are added to the corpus nor does\nit adopt the Markovian assumption which overly restricts the type of changes\nthat the model can capture. Our key technical contribution is a framework based\non (i) discretization of time into epochs, (ii) epoch-wise topic discovery\nusing a hierarchical Dirichlet process-based model, and (iii) a temporal\nsimilarity graph which allows for the modelling of complex topic changes:\nemergence and disappearance, evolution, and splitting and merging. The power of\nthe proposed framework is demonstrated on the medical literature corpus\nconcerned with the autism spectrum disorder (ASD) - an increasingly important\nresearch subject of significant social and healthcare importance. In addition\nto the collected ASD literature corpus which we will make freely available, our\ncontributions also include two free online tools we built as aids to ASD\nresearchers. These can be used for semantically meaningful navigation and\nsearching, as well as knowledge discovery from this large and rapidly growing\ncorpus of literature.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1502.02277v1",
            "date": "2015-02-08 17:32:44+00:00",
            "title": "Improving Term Frequency Normalization for Multi-topical Documents, and Application to Language Modeling Approaches",
            "authors": [
                "Seung-Hoon Na",
                "In-Su Kang",
                "Jong-Hyeok Lee"
            ],
            "abstract": "Term frequency normalization is a serious issue since lengths of documents\nare various. Generally, documents become long due to two different reasons -\nverbosity and multi-topicality. First, verbosity means that the same topic is\nrepeatedly mentioned by terms related to the topic, so that term frequency is\nmore increased than the well-summarized one. Second, multi-topicality indicates\nthat a document has a broad discussion of multi-topics, rather than single\ntopic. Although these document characteristics should be differently handled,\nall previous methods of term frequency normalization have ignored these\ndifferences and have used a simplified length-driven approach which decreases\nthe term frequency by only the length of a document, causing an unreasonable\npenalization. To attack this problem, we propose a novel TF normalization\nmethod which is a type of partially-axiomatic approach. We first formulate two\nformal constraints that the retrieval model should satisfy for documents having\nverbose and multi-topicality characteristic, respectively. Then, we modify\nlanguage modeling approaches to better satisfy these two constraints, and\nderive novel smoothing methods. Experimental results show that the proposed\nmethod increases significantly the precision for keyword queries, and\nsubstantially improves MAP (Mean Average Precision) for verbose queries.",
            "journal ref": "Advances in Information Retrieval Lecture Notes in Computer\n  Science Volume 4956, 2008, pp 382-393",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1502.03322v1",
            "date": "2015-02-11 14:45:41+00:00",
            "title": "Boost Phrase-level Polarity Labelling with Review-level Sentiment Classification",
            "authors": [
                "Yongfeng Zhang",
                "Min Zhang",
                "Yiqun Liu",
                "Shaoping Ma"
            ],
            "abstract": "Sentiment analysis on user reviews helps to keep track of user reactions\ntowards products, and make advices to users about what to buy. State-of-the-art\nreview-level sentiment classification techniques could give pretty good\nprecisions of above 90%. However, current phrase-level sentiment analysis\napproaches might only give sentiment polarity labelling precisions of around\n70%~80%, which is far from satisfaction and restricts its application in many\npractical tasks. In this paper, we focus on the problem of phrase-level\nsentiment polarity labelling and attempt to bridge the gap between phrase-level\nand review-level sentiment analysis. We investigate the inconsistency between\nthe numerical star ratings and the sentiment orientation of textual user\nreviews. Although they have long been treated as identical, which serves as a\nbasic assumption in previous work, we find that this assumption is not\nnecessarily true. We further propose to leverage the results of review-level\nsentiment classification to boost the performance of phrase-level polarity\nlabelling using a novel constrained convex optimization framework. Besides, the\nframework is capable of integrating various kinds of information sources and\nheuristics, while giving the global optimal solution due to its convexity.\nExperimental results on both English and Chinese reviews show that our\nframework achieves high labelling precisions of up to 89%, which is a\nsignificant improvement from current approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.07157v2",
            "date": "2015-02-25 13:07:41+00:00",
            "title": "Exploiting a comparability mapping to improve bi-lingual data categorization: a three-mode data analysis perspective",
            "authors": [
                "Pierre-Fran\u00e7ois Marteau",
                "Guiyao Ke"
            ],
            "abstract": "We address in this paper the co-clustering and co-classification of bilingual\ndata laying in two linguistic similarity spaces when a comparability measure\ndefining a mapping between these two spaces is available. A new approach that\nwe can characterized as a three-mode analysis scheme, is proposed to mix the\ncomparability measure with the two similarity measures. Our aim is to improve\njointly the accuracy of classification and clustering tasks performed in each\nof the two linguistic spaces, as well as the quality of the final alignment of\ncomparable clusters that can be obtained. We used first some purely synthetic\nrandom data sets to assess our formal similarity-comparability mixing model. We\nthen propose two variants of the comparability measure that has been defined by\n(Li and Gaussier 2010) in the context of bilingual lexicon extraction to adapt\nit to clustering or categorizing tasks. These two variant measures are\nsubsequently used to evaluate our similarity-comparability mixing model in the\ncontext of the co-classification and co-clustering of comparable textual data\nsets collected from Wikipedia categories for the English and French languages.\nOur experiments show clear improvements in clustering and classification\naccuracies when mixing comparability with similarity measures, with, as\nexpected, a higher robustness obtained when the two comparability variant\nmeasures that we propose are used. We believe that this approach is\nparticularly well suited for the construction of thematic comparable corpora of\ncontrollable quality.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1501.00657v2",
            "date": "2015-01-04 10:10:32+00:00",
            "title": "Cross-language Wikipedia Editing of Okinawa, Japan",
            "authors": [
                "Scott A. Hale"
            ],
            "abstract": "This article analyzes users who edit Wikipedia articles about Okinawa, Japan,\nin English and Japanese. It finds these users are among the most active and\ndedicated users in their primary languages, where they make many large,\nhigh-quality edits. However, when these users edit in their non-primary\nlanguages, they tend to make edits of a different type that are overall smaller\nin size and more often restricted to the narrow set of articles that exist in\nboth languages. Design changes to motivate wider contributions from users in\ntheir non-primary languages and to encourage multilingual users to transfer\nmore information across language divides are presented.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1501.00960v4",
            "date": "2015-01-05 20:09:59+00:00",
            "title": "Characterizing the Google Books corpus: Strong limits to inferences of socio-cultural and linguistic evolution",
            "authors": [
                "Eitan Adam Pechenick",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "It is tempting to treat frequency trends from the Google Books data sets as\nindicators of the \"true\" popularity of various words and phrases. Doing so\nallows us to draw quantitatively strong conclusions about the evolution of\ncultural perception of a given topic, such as time or gender. However, the\nGoogle Books corpus suffers from a number of limitations which make it an\nobscure mask of cultural popularity. A primary issue is that the corpus is in\neffect a library, containing one of each book. A single, prolific author is\nthereby able to noticeably insert new phrases into the Google Books lexicon,\nwhether the author is widely read or not. With this understood, the Google\nBooks corpus remains an important data set to be considered more lexicon-like\nthan text-like. Here, we show that a distinct problematic feature arises from\nthe inclusion of scientific texts, which have become an increasingly\nsubstantive portion of the corpus throughout the 1900s. The result is a surge\nof phrases typical to academic articles but less common in general, such as\nreferences to time in the form of citations. We highlight these dynamics by\nexamining and comparing major contributions to the statistical divergence of\nEnglish data sets between decades in the period 1800--2000. We find that only\nthe English Fiction data set from the second version of the corpus is not\nheavily affected by professional texts, in clear contrast to the first version\nof the fiction data set and both unfiltered English data sets. Our findings\nemphasize the need to fully characterize the dynamics of the Google Books\ncorpus before using these data sets to draw broad conclusions about cultural\nand linguistic evolution.",
            "journal ref": "PLoS ONE, 10, e0137041, 2015",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1501.02527v1",
            "date": "2015-01-12 03:10:01+00:00",
            "title": "Autodetection and Classification of Hidden Cultural City Districts from Yelp Reviews",
            "authors": [
                "Harini Suresh",
                "Nicholas Locascio"
            ],
            "abstract": "Topic models are a way to discover underlying themes in an otherwise\nunstructured collection of documents. In this study, we specifically used the\nLatent Dirichlet Allocation (LDA) topic model on a dataset of Yelp reviews to\nclassify restaurants based off of their reviews. Furthermore, we hypothesize\nthat within a city, restaurants can be grouped into similar \"clusters\" based on\nboth location and similarity. We used several different clustering methods,\nincluding K-means Clustering and a Probabilistic Mixture Model, in order to\nuncover and classify districts, both well-known and hidden (i.e. cultural areas\nlike Chinatown or hearsay like \"the best street for Italian restaurants\")\nwithin a city. We use these models to display and label different clusters on a\nmap. We also introduce a topic similarity heatmap that displays the similarity\ndistribution in a city to a new restaurant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.02530v1",
            "date": "2015-01-12 03:31:33+00:00",
            "title": "A Dataset for Movie Description",
            "authors": [
                "Anna Rohrbach",
                "Marcus Rohrbach",
                "Niket Tandon",
                "Bernt Schiele"
            ],
            "abstract": "Descriptive video service (DVS) provides linguistic descriptions of movies\nand allows visually impaired people to follow a movie along with their peers.\nSuch descriptions are by design mainly visual and thus naturally form an\ninteresting data source for computer vision and computational linguistics. In\nthis work we propose a novel dataset which contains transcribed DVS, which is\ntemporally aligned to full length HD movies. In addition we also collected the\naligned movie scripts which have been used in prior work and compare the two\ndifferent sources of descriptions. In total the Movie Description dataset\ncontains a parallel corpus of over 54,000 sentences and video snippets from 72\nHD movies. We characterize the dataset by benchmarking different approaches for\ngenerating video descriptions. Comparing DVS to scripts, we find that DVS is\nfar more visual and describes precisely what is shown rather than what should\nhappen according to the scripts created prior to movie production.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1501.02598v3",
            "date": "2015-01-12 10:48:32+00:00",
            "title": "Combining Language and Vision with a Multimodal Skip-gram Model",
            "authors": [
                "Angeliki Lazaridou",
                "Nghia The Pham",
                "Marco Baroni"
            ],
            "abstract": "We extend the SKIP-GRAM model of Mikolov et al. (2013a) by taking visual\ninformation into account. Like SKIP-GRAM, our multimodal models (MMSKIP-GRAM)\nbuild vector-based word representations by learning to predict linguistic\ncontexts in text corpora. However, for a restricted set of words, the models\nare also exposed to visual representations of the objects they denote\n(extracted from natural images), and must predict linguistic and visual\nfeatures jointly. The MMSKIP-GRAM models achieve good performance on a variety\nof semantic benchmarks. Moreover, since they propagate visual information to\nall words, we use them to improve image labeling and retrieval in the zero-shot\nsetup, where the test concepts are never seen during model training. Finally,\nthe MMSKIP-GRAM models discover intriguing visual properties of abstract words,\npaving the way to realistic implementations of embodied theories of meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.03302v2",
            "date": "2015-01-14 10:38:43+00:00",
            "title": "Hard to Cheat: A Turing Test based on Answering Questions about Images",
            "authors": [
                "Mateusz Malinowski",
                "Mario Fritz"
            ],
            "abstract": "Progress in language and image understanding by machines has sparkled the\ninterest of the research community in more open-ended, holistic tasks, and\nrefueled an old AI dream of building intelligent machines. We discuss a few\nprominent challenges that characterize such holistic tasks and argue for\n\"question answering about images\" as a particular appealing instance of such a\nholistic task. In particular, we point out that it is a version of a Turing\nTest that is likely to be more robust to over-interpretations and contrast it\nwith tasks like grounding and generation of descriptions. Finally, we discuss\ntools to measure progress in this field.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1501.04325v1",
            "date": "2015-01-18 17:12:59+00:00",
            "title": "Deep Belief Nets for Topic Modeling",
            "authors": [
                "Lars Maaloe",
                "Morten Arngren",
                "Ole Winther"
            ],
            "abstract": "Applying traditional collaborative filtering to digital publishing is\nchallenging because user data is very sparse due to the high volume of\ndocuments relative to the number of users. Content based approaches, on the\nother hand, is attractive because textual content is often very informative. In\nthis paper we describe large-scale content based collaborative filtering for\ndigital publishing. To solve the digital publishing recommender problem we\ncompare two approaches: latent Dirichlet allocation (LDA) and deep belief nets\n(DBN) that both find low-dimensional latent representations for documents.\nEfficient retrieval can be carried out in the latent representation. We work\nboth on public benchmarks and digital media content provided by Issuu, an\nonline publishing platform. This article also comes with a newly developed deep\nbelief nets toolbox for topic modeling tailored towards performance evaluation\nof the DBN model and comparisons to the LDA model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1501.04346v1",
            "date": "2015-01-18 20:50:39+00:00",
            "title": "Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions",
            "authors": [
                "Andrew S. Lan",
                "Divyanshu Vats",
                "Andrew E. Waters",
                "Richard G. Baraniuk"
            ],
            "abstract": "While computer and communication technologies have provided effective means\nto scale up many aspects of education, the submission and grading of\nassessments such as homework assignments and tests remains a weak link. In this\npaper, we study the problem of automatically grading the kinds of open response\nmathematical questions that figure prominently in STEM (science, technology,\nengineering, and mathematics) courses. Our data-driven framework for\nmathematical language processing (MLP) leverages solution data from a large\nnumber of learners to evaluate the correctness of their solutions, assign\npartial-credit scores, and provide feedback to each learner on the likely\nlocations of any errors. MLP takes inspiration from the success of natural\nlanguage processing for text data and comprises three main steps. First, we\nconvert each solution to an open response mathematical question into a series\nof numerical features. Second, we cluster the features from several solutions\nto uncover the structures of correct, partially correct, and incorrect\nsolutions. We develop two different clustering approaches, one that leverages\ngeneric clustering algorithms and one based on Bayesian nonparametrics. Third,\nwe automatically grade the remaining (potentially large number of) solutions\nbased on their assigned cluster and one instructor-provided grade per cluster.\nAs a bonus, we can track the cluster assignment of each step of a multistep\nsolution and determine when it departs from a cluster of correct solutions,\nwhich enables us to indicate the likely locations of errors to learners. We\ntest and validate MLP on real-world MOOC data to demonstrate how it can\nsubstantially reduce the human effort required in large-scale educational\nplatforms.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1501.05940v1",
            "date": "2015-01-22 22:18:45+00:00",
            "title": "A New Efficient Method for Calculating Similarity Between Web Services",
            "authors": [
                "T. Rachad",
                "J. Boutahar",
                "S. El ghazi"
            ],
            "abstract": "Web services allow communication between heterogeneous systems in a\ndistributed environment. Their enormous success and their increased use led to\nthe fact that thousands of Web services are present on the Internet. This\nsignificant number of Web services which not cease to increase has led to\nproblems of the difficulty in locating and classifying web services, these\nproblems are encountered mainly during the operations of web services discovery\nand substitution. Traditional ways of search based on keywords are not\nsuccessful in this context, their results do not support the structure of Web\nservices and they consider in their search only the identifiers of the web\nservice description language (WSDL) interface elements. The methods based on\nsemantics (WSDLS, OWLS, SAWSDL...) which increase the WSDL description of a Web\nservice with a semantic description allow raising partially this problem, but\ntheir complexity and difficulty delays their adoption in real cases. Measuring\nthe similarity between the web services interfaces is the most suitable\nsolution for this kind of problems, it will classify available web services so\nas to know those that best match the searched profile and those that do not\nmatch. Thus, the main goal of this work is to study the degree of similarity\nbetween any two web services by offering a new method that is more effective\nthan existing works.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1501.06587v1",
            "date": "2015-01-26 21:06:02+00:00",
            "title": "Measuring academic influence: Not all citations are equal",
            "authors": [
                "Xiaodan Zhu",
                "Peter Turney",
                "Daniel Lemire",
                "Andr\u00e9 Vellino"
            ],
            "abstract": "The importance of a research article is routinely measured by counting how\nmany times it has been cited. However, treating all citations with equal weight\nignores the wide variety of functions that citations perform. We want to\nautomatically identify the subset of references in a bibliography that have a\ncentral academic influence on the citing paper. For this purpose, we examine\nthe effectiveness of a variety of features for determining the academic\ninfluence of a citation. By asking authors to identify the key references in\ntheir own work, we created a data set in which citations were labeled according\nto their academic influence. Using automatic feature selection with supervised\nmachine learning, we found a model for predicting academic influence that\nachieves good performance on this data set using only four features. The best\nfeatures, among those we evaluated, were those based on the number of times a\nreference is mentioned in the body of a citing paper. The performance of these\nfeatures inspired us to design an influence-primed h-index (the hip-index).\nUnlike the conventional h-index, it weights citations by how many times a\nreference is mentioned. According to our experiments, the hip-index is a better\nindicator of researcher performance than the conventional h-index.",
            "journal ref": "Journal of the Association for Information Science and Technology,\n  66: 408-427",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1501.07496v1",
            "date": "2015-01-29 16:09:17+00:00",
            "title": "Implementation of an Automatic Syllabic Division Algorithm from Speech Files in Portuguese Language",
            "authors": [
                "E. L. F. Da Silva",
                "H. M. de Oliveira"
            ],
            "abstract": "A new algorithm for voice automatic syllabic splitting in the Portuguese\nlanguage is proposed, which is based on the envelope of the speech signal of\nthe input audio file. A computational implementation in MatlabTM is presented\nand made available at the URL\nhttp://www2.ee.ufpe.br/codec/divisao_silabica.html. Due to its\nstraightforwardness, the proposed method is very attractive for embedded\nsystems (e.g. i-phones). It can also be used as a screen to assist more\nsophisticated methods. Voice excerpts containing more than one syllable and\nidentified by the same envelope are named as super-syllables and they are\nsubsequently separated. The results indicate which samples corresponds to the\nbeginning and end of each detected syllable. Preliminary tests were performed\nto fifty words at an identification rate circa 70% (further improvements may be\nincorporated to treat particular phonemes). This algorithm is also useful in\nvoice command systems, as a tool in the teaching of Portuguese language or even\nfor patients with speech pathology.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1503.00075v3",
            "date": "2015-02-28 06:31:50+00:00",
            "title": "Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks",
            "authors": [
                "Kai Sheng Tai",
                "Richard Socher",
                "Christopher D. Manning"
            ],
            "abstract": "Because of their superior ability to preserve sequence information over time,\nLong Short-Term Memory (LSTM) networks, a type of recurrent neural network with\na more complex computational unit, have obtained strong results on a variety of\nsequence modeling tasks. The only underlying LSTM structure that has been\nexplored so far is a linear chain. However, natural language exhibits syntactic\nproperties that would naturally combine words to phrases. We introduce the\nTree-LSTM, a generalization of LSTMs to tree-structured network topologies.\nTree-LSTMs outperform all existing systems and strong LSTM baselines on two\ntasks: predicting the semantic relatedness of two sentences (SemEval 2014, Task\n1) and sentiment classification (Stanford Sentiment Treebank).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00339v2",
            "date": "2015-03-01 19:37:27+00:00",
            "title": "Variation of word frequencies in Russian literary texts",
            "authors": [
                "Vladislav Kargin"
            ],
            "abstract": "We study the variation of word frequencies in Russian literary texts. Our\nfindings indicate that the standard deviation of a word's frequency across\ntexts depends on its average frequency according to a power law with exponent\n$0.62,$ showing that the rarer words have a relatively larger degree of\nfrequency volatility (i.e., \"burstiness\").\n  Several latent factors models have been estimated to investigate the\nstructure of the word frequency distribution. The dependence of a word's\nfrequency volatility on its average frequency can be explained by the asymmetry\nin the distribution of latent factors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00693v1",
            "date": "2015-03-02 20:23:18+00:00",
            "title": "Bayesian Optimization of Text Representations",
            "authors": [
                "Dani Yogatama",
                "Noah A. Smith"
            ],
            "abstract": "When applying machine learning to problems in NLP, there are many choices to\nmake about how to represent input texts. These choices can have a big effect on\nperformance, but they are often uninteresting to researchers or practitioners\nwho simply need a module that performs well. We propose an approach to\noptimizing over this space of choices, formulating the problem as global\noptimization. We apply a sequential model-based optimization technique and show\nthat our method makes standard linear models competitive with more\nsophisticated, expensive state-of-the-art methods based on latent variable\nmodels or neural networks on various topic classification and sentiment\nanalysis problems. Our approach is a first step towards black-box NLP systems\nthat work with raw text and do not require manual tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.00841v1",
            "date": "2015-03-03 06:59:28+00:00",
            "title": "Robustly Leveraging Prior Knowledge in Text Classification",
            "authors": [
                "Biao Liu",
                "Minlie Huang"
            ],
            "abstract": "Prior knowledge has been shown very useful to address many natural language\nprocessing tasks. Many approaches have been proposed to formalise a variety of\nknowledge, however, whether the proposed approach is robust or sensitive to the\nknowledge supplied to the model has rarely been discussed. In this paper, we\npropose three regularization terms on top of generalized expectation criteria,\nand conduct extensive experiments to justify the robustness of the proposed\nmethods. Experimental results demonstrate that our proposed methods obtain\nremarkable improvements and are much more robust than baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.01129v1",
            "date": "2015-03-03 21:18:05+00:00",
            "title": "Complexity and universality in the long-range order of words",
            "authors": [
                "Marcelo A Montemurro",
                "Dami\u00e1n H Zanette"
            ],
            "abstract": "As is the case of many signals produced by complex systems, language presents\na statistical structure that is balanced between order and disorder. Here we\nreview and extend recent results from quantitative characterisations of the\ndegree of order in linguistic sequences that give insights into two relevant\naspects of language: the presence of statistical universals in word ordering,\nand the link between semantic information and the statistical linguistic\nstructure. We first analyse a measure of relative entropy that assesses how\nmuch the ordering of words contributes to the overall statistical structure of\nlanguage. This measure presents an almost constant value close to 3.5 bits/word\nacross several linguistic families. Then, we show that a direct application of\ninformation theory leads to an entropy measure that can quantify and extract\nsemantic structures from linguistic samples, even without prior knowledge of\nthe underlying language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.01180v2",
            "date": "2015-03-04 01:05:41+00:00",
            "title": "All Who Wander: On the Prevalence and Characteristics of Multi-community Engagement",
            "authors": [
                "Chenhao Tan",
                "Lillian Lee"
            ],
            "abstract": "Although analyzing user behavior within individual communities is an active\nand rich research domain, people usually interact with multiple communities\nboth on- and off-line. How do users act in such multi-community environments?\nAlthough there are a host of intriguing aspects to this question, it has\nreceived much less attention in the research community in comparison to the\nintra-community case. In this paper, we examine three aspects of\nmulti-community engagement: the sequence of communities that users post to, the\nlanguage that users employ in those communities, and the feedback that users\nreceive, using longitudinal posting behavior on Reddit as our main data source,\nand DBLP for auxiliary experiments. We also demonstrate the effectiveness of\nfeatures drawn from these aspects in predicting users' future level of\nactivity.\n  One might expect that a user's trajectory mimics the \"settling-down\" process\nin real life: an initial exploration of sub-communities before settling down\ninto a few niches. However, we find that the users in our data continually post\nin new communities; moreover, as time goes on, they post increasingly evenly\namong a more diverse set of smaller communities. Interestingly, it seems that\nusers that eventually leave the community are \"destined\" to do so from the very\nbeginning, in the sense of showing significantly different \"wandering\" patterns\nvery early on in their trajectories; this finding has potentially important\ndesign implications for community maintainers. Our multi-community perspective\nalso allows us to investigate the \"situation vs. personality\" debate from\nlanguage usage across different communities.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1503.01190v1",
            "date": "2015-03-04 01:34:36+00:00",
            "title": "Statistical modality tagging from rule-based annotations and crowdsourcing",
            "authors": [
                "Vinodkumar Prabhakaran",
                "Michael Bloodgood",
                "Mona Diab",
                "Bonnie Dorr",
                "Lori Levin",
                "Christine D. Piatko",
                "Owen Rambow",
                "Benjamin Van Durme"
            ],
            "abstract": "We explore training an automatic modality tagger. Modality is the attitude\nthat a speaker might have toward an event or state. One of the main hurdles for\ntraining a linguistic tagger is gathering training data. This is particularly\nproblematic for training a tagger for modality because modality triggers are\nsparse for the overwhelming majority of sentences. We investigate an approach\nto automatically training a modality tagger where we first gathered sentences\nbased on a high-recall simple rule-based modality tagger and then provided\nthese sentences to Mechanical Turk annotators for further annotation. We used\nthe resulting set of training data to train a precise modality tagger using a\nmulti-class SVM that delivers good performance.",
            "journal ref": "In Proceedings of the Workshop on Extra-Propositional Aspects of\n  Meaning in Computational Linguistics, pages 57-64, Jeju, Republic of Korea,\n  July 2012. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.01397v3",
            "date": "2015-03-04 17:36:49+00:00",
            "title": "Bethe Projections for Non-Local Inference",
            "authors": [
                "Luke Vilnis",
                "David Belanger",
                "Daniel Sheldon",
                "Andrew McCallum"
            ],
            "abstract": "Many inference problems in structured prediction are naturally solved by\naugmenting a tractable dependency structure with complex, non-local auxiliary\nobjectives. This includes the mean field family of variational inference\nalgorithms, soft- or hard-constrained inference using Lagrangian relaxation or\nlinear programming, collective graphical models, and forms of semi-supervised\nlearning such as posterior regularization. We present a method to\ndiscriminatively learn broad families of inference objectives, capturing\npowerful non-local statistics of the latent variables, while maintaining\ntractable and provably fast inference using non-Euclidean projected gradient\ndescent with a distance-generating function given by the Bethe entropy. We\ndemonstrate the performance and flexibility of our method by (1) extracting\nstructured citations from research papers by learning soft global constraints,\n(2) achieving state-of-the-art results on a widely-used handwriting recognition\ntask using a novel learned non-convex inference procedure, and (3) providing a\nfast and highly scalable algorithm for the challenging problem of inference in\na collective graphical model applied to bird migration.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1503.01558v3",
            "date": "2015-03-05 07:07:48+00:00",
            "title": "What's Cookin'? Interpreting Cooking Videos using Text, Speech and Vision",
            "authors": [
                "Jonathan Malmaud",
                "Jonathan Huang",
                "Vivek Rathod",
                "Nick Johnston",
                "Andrew Rabinovich",
                "Kevin Murphy"
            ],
            "abstract": "We present a novel method for aligning a sequence of instructions to a video\nof someone carrying out a task. In particular, we focus on the cooking domain,\nwhere the instructions correspond to the recipe. Our technique relies on an HMM\nto align the recipe steps to the (automatically generated) speech transcript.\nWe then refine this alignment using a state-of-the-art visual food detector,\nbased on a deep convolutional neural network. We show that our technique\noutperforms simpler techniques based on keyword spotting. It also enables\ninteresting applications, such as automatically illustrating recipes with\nkeyframes, and searching within a video for events of interest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.01838v5",
            "date": "2015-03-06 03:04:54+00:00",
            "title": "Encoding Source Language with Convolutional Neural Network for Machine Translation",
            "authors": [
                "Fandong Meng",
                "Zhengdong Lu",
                "Mingxuan Wang",
                "Hang Li",
                "Wenbin Jiang",
                "Qun Liu"
            ],
            "abstract": "The recently proposed neural network joint model (NNJM) (Devlin et al., 2014)\naugments the n-gram target language model with a heuristically chosen source\ncontext window, achieving state-of-the-art performance in SMT. In this paper,\nwe give a more systematic treatment by summarizing the relevant source\ninformation through a convolutional architecture guided by the target\ninformation. With different guiding signals during decoding, our specifically\ndesigned convolution+gating architectures can pinpoint the parts of a source\nsentence that are relevant to predicting a target word, and fuse them with the\ncontext of entire source sentence to form a unified representation. This\nrepresentation, together with target language words, are fed to a deep neural\nnetwork (DNN) to form a stronger NNJM. Experiments on two NIST Chinese-English\ntranslation tasks show that the proposed model can achieve significant\nimprovements over the previous NNJM by up to +1.08 BLEU points on average",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02108v2",
            "date": "2015-03-06 22:48:29+00:00",
            "title": "Maximum a Posteriori Adaptation of Network Parameters in Deep Models",
            "authors": [
                "Zhen Huang",
                "Sabato Marco Siniscalchi",
                "I-Fan Chen",
                "Jiadong Wu",
                "Chin-Hui Lee"
            ],
            "abstract": "We present a Bayesian approach to adapting parameters of a well-trained\ncontext-dependent, deep-neural-network, hidden Markov model (CD-DNN-HMM) to\nimprove automatic speech recognition performance. Given an abundance of DNN\nparameters but with only a limited amount of data, the effectiveness of the\nadapted DNN model can often be compromised. We formulate maximum a posteriori\n(MAP) adaptation of parameters of a specially designed CD-DNN-HMM with an\naugmented linear hidden networks connected to the output tied states, or\nsenones, and compare it to feature space MAP linear regression previously\nproposed. Experimental evidences on the 20,000-word open vocabulary Wall Street\nJournal task demonstrate the feasibility of the proposed framework. In\nsupervised adaptation, the proposed MAP adaptation approach provides more than\n10% relative error reduction and consistently outperforms the conventional\ntransformation based methods. Furthermore, we present an initial attempt to\ngenerate hierarchical priors to improve adaptation efficiency and effectiveness\nwith limited adaptation data by exploiting similarities among senones.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1503.02120v3",
            "date": "2015-03-07 02:45:06+00:00",
            "title": "Identifying missing dictionary entries with frequency-conserving context models",
            "authors": [
                "Jake Ryland Williams",
                "Eric M. Clark",
                "James P. Bagrow",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "In an effort to better understand meaning from natural language texts, we\nexplore methods aimed at organizing lexical objects into contexts. A number of\nthese methods for organization fall into a family defined by word ordering.\nUnlike demographic or spatial partitions of data, these collocation models are\nof special importance for their universal applicability. While we are\ninterested here in text and have framed our treatment appropriately, our work\nis potentially applicable to other areas of research (e.g., speech, genomics,\nand mobility patterns) where one has ordered categorical data, (e.g., sounds,\ngenes, and locations). Our approach focuses on the phrase (whether word or\nlarger) as the primary meaning-bearing lexical unit and object of study. To do\nso, we employ our previously developed framework for generating word-conserving\nphrase-frequency data. Upon training our model with the Wiktionary---an\nextensive, online, collaborative, and open-source dictionary that contains over\n100,000 phrasal-definitions---we develop highly effective filters for the\nidentification of meaningful, missing phrase-entries. With our predictions we\nthen engage the editorial community of the Wiktionary and propose short lists\nof potential missing entries for definition, developing a breakthrough, lexical\nextraction technique, and expanding our knowledge of the defined English\nlexicon of phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02357v2",
            "date": "2015-03-09 02:16:19+00:00",
            "title": "Context-Dependent Translation Selection Using Convolutional Neural Network",
            "authors": [
                "Zhaopeng Tu",
                "Baotian Hu",
                "Zhengdong Lu",
                "Hang Li"
            ],
            "abstract": "We propose a novel method for translation selection in statistical machine\ntranslation, in which a convolutional neural network is employed to judge the\nsimilarity between a phrase pair in two languages. The specifically designed\nconvolutional architecture encodes not only the semantic similarity of the\ntranslation pair, but also the context containing the phrase in the source\nlanguage. Therefore, our approach is able to capture context-dependent semantic\nsimilarities of translation pairs. We adopt a curriculum learning strategy to\ntrain the model: we classify the training examples into easy, medium, and\ndifficult categories, and gradually build the ability of representing phrase\nand sentence level context by using training examples from easy to difficult.\nExperimental results show that our approach significantly outperforms the\nbaseline system by up to 1.4 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02364v2",
            "date": "2015-03-09 02:54:29+00:00",
            "title": "Neural Responding Machine for Short-Text Conversation",
            "authors": [
                "Lifeng Shang",
                "Zhengdong Lu",
                "Hang Li"
            ],
            "abstract": "We propose Neural Responding Machine (NRM), a neural network-based response\ngenerator for Short-Text Conversation. NRM takes the general encoder-decoder\nframework: it formalizes the generation of response as a decoding process based\non the latent representation of the input text, while both encoding and\ndecoding are realized with recurrent neural networks (RNN). The NRM is trained\nwith a large amount of one-round conversation data collected from a\nmicroblogging service. Empirical study shows that NRM can generate\ngrammatically correct and content-wise appropriate responses to over 75% of the\ninput text, outperforming state-of-the-arts in the same setting, including\nretrieval-based and SMT-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02427v6",
            "date": "2015-03-09 11:11:15+00:00",
            "title": "Syntax-based Deep Matching of Short Texts",
            "authors": [
                "Mingxuan Wang",
                "Zhengdong Lu",
                "Hang Li",
                "Qun Liu"
            ],
            "abstract": "Many tasks in natural language processing, ranging from machine translation\nto question answering, can be reduced to the problem of matching two sentences\nor more generally two short texts. We propose a new approach to the problem,\ncalled Deep Match Tree (DeepMatch$_{tree}$), under a general setting. The\napproach consists of two components, 1) a mining algorithm to discover patterns\nfor matching two short-texts, defined in the product space of dependency trees,\nand 2) a deep neural network for matching short texts using the mined patterns,\nas well as a learning algorithm to build the network having a sparse structure.\nWe test our algorithm on the problem of matching a tweet and a response in\nsocial media, a hard matching problem proposed in [Wang et al., 2013], and show\nthat DeepMatch$_{tree}$ can outperform a number of competitor models including\none without using dependency trees and one based on word-embedding, all with\nlarge margins",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.02510v2",
            "date": "2015-03-09 15:13:38+00:00",
            "title": "Compositional Distributional Semantics with Long Short Term Memory",
            "authors": [
                "Phong Le",
                "Willem Zuidema"
            ],
            "abstract": "We are proposing an extension of the recursive neural network that makes use\nof a variant of the long short-term memory architecture. The extension allows\ninformation low in parse trees to be stored in a memory register (the `memory\ncell') and used much later higher up in the parse tree. This provides a\nsolution to the vanishing gradient problem and allows the network to capture\nlong range dependencies. Experimental results show that our composition\noutperformed the traditional neural-network composition on the Stanford\nSentiment Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.03244v1",
            "date": "2015-03-11 09:46:36+00:00",
            "title": "Convolutional Neural Network Architectures for Matching Natural Language Sentences",
            "authors": [
                "Baotian Hu",
                "Zhengdong Lu",
                "Hang Li",
                "Qingcai Chen"
            ],
            "abstract": "Semantic matching is of central importance to many natural language tasks\n\\cite{bordes2014semantic,RetrievalQA}. A successful matching algorithm needs to\nadequately model the internal structures of language objects and the\ninteraction between them. As a step toward this goal, we propose convolutional\nneural network models for matching two sentences, by adapting the convolutional\nstrategy in vision and speech. The proposed models not only nicely represent\nthe hierarchical structures of sentences with their layer-by-layer composition\nand pooling, but also capture the rich matching patterns at different levels.\nOur models are rather generic, requiring no prior knowledge on language, and\ncan hence be applied to matching tasks of different nature and in different\nlanguages. The empirical study on a variety of matching tasks demonstrates the\nefficacy of the proposed model on a variety of matching tasks and its\nsuperiority to competitor models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.04723v1",
            "date": "2015-03-16 16:43:58+00:00",
            "title": "Deep Feelings: A Massive Cross-Lingual Study on the Relation between Emotions and Virality",
            "authors": [
                "Marco Guerini",
                "Jacopo Staiano"
            ],
            "abstract": "This article provides a comprehensive investigation on the relations between\nvirality of news articles and the emotions they are found to evoke. Virality,\nin our view, is a phenomenon with many facets, i.e. under this generic term\nseveral different effects of persuasive communication are comprised. By\nexploiting a high-coverage and bilingual corpus of documents containing metrics\nof their spread on social networks as well as a massive affective annotation\nprovided by readers, we present a thorough analysis of the interplay between\nevoked emotions and viral facets. We highlight and discuss our findings in\nlight of a cross-lingual approach: while we discover differences in evoked\nemotions and corresponding viral effects, we provide preliminary evidence of a\ngeneralized explanatory model rooted in the deep structure of emotions: the\nValence-Arousal-Dominance (VAD) circumplex. We find that viral facets appear to\nbe consistently affected by particular VAD configurations, and these\nconfigurations indicate a clear connection with distinct phenomena underlying\npersuasive communication.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1503.04881v1",
            "date": "2015-03-16 23:59:02+00:00",
            "title": "Long Short-Term Memory Over Tree Structures",
            "authors": [
                "Xiaodan Zhu",
                "Parinaz Sobhani",
                "Hongyu Guo"
            ],
            "abstract": "The chain-structured long short-term memory (LSTM) has showed to be effective\nin a wide range of problems such as speech recognition and machine translation.\nIn this paper, we propose to extend it to tree structures, in which a memory\ncell can reflect the history memories of multiple child cells or multiple\ndescendant cells in a recursive process. We call the model S-LSTM, which\nprovides a principled way of considering long-distance interaction over\nhierarchies, e.g., language or image parse structures. We leverage the models\nfor semantic composition to understand the meaning of text, a fundamental\nproblem in natural language understanding, and show that it outperforms a\nstate-of-the-art recursive model by replacing its composition layers with the\nS-LSTM memory blocks. We also show that utilizing the given structures is\nhelpful in achieving a performance better than that without considering the\nstructures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.07294v1",
            "date": "2015-03-25 06:42:05+00:00",
            "title": "Using Latent Semantic Analysis to Identify Quality in Use (QU) Indicators from User Reviews",
            "authors": [
                "Wendy Tan Wei Syn",
                "Bong Chih How",
                "Issa Atoum"
            ],
            "abstract": "The paper describes a novel approach to categorize users' reviews according\nto the three Quality in Use (QU) indicators defined in ISO: effectiveness,\nefficiency and freedom from risk. With the tremendous amount of reviews\npublished each day, there is a need to automatically summarize user reviews to\ninform us if any of the software able to meet requirement of a company\naccording to the quality requirements. We implemented the method of Latent\nSemantic Analysis (LSA) and its subspace to predict QU indicators. We build a\nreduced dimensionality universal semantic space from Information System\njournals and Amazon reviews. Next, we projected set of indicators' measurement\nscales into the universal semantic space and represent them as subspace. In the\nsubspace, we can map similar measurement scales to the unseen reviews and\npredict the QU indicators. Our preliminary study able to obtain the average of\nF-measure, 0.3627.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1503.08542v1",
            "date": "2015-03-30 05:40:41+00:00",
            "title": "Nonparametric Relational Topic Models through Dependent Gamma Processes",
            "authors": [
                "Junyu Xuan",
                "Jie Lu",
                "Guangquan Zhang",
                "Richard Yi Da Xu",
                "Xiangfeng Luo"
            ],
            "abstract": "Traditional Relational Topic Models provide a way to discover the hidden\ntopics from a document network. Many theoretical and practical tasks, such as\ndimensional reduction, document clustering, link prediction, benefit from this\nrevealed knowledge. However, existing relational topic models are based on an\nassumption that the number of hidden topics is known in advance, and this is\nimpractical in many real-world applications. Therefore, in order to relax this\nassumption, we propose a nonparametric relational topic model in this paper.\nInstead of using fixed-dimensional probability distributions in its generative\nmodel, we use stochastic processes. Specifically, a gamma process is assigned\nto each document, which represents the topic interest of this document.\nAlthough this method provides an elegant solution, it brings additional\nchallenges when mathematically modeling the inherent network structure of\ntypical document network, i.e., two spatially closer documents tend to have\nmore similar topics. Furthermore, we require that the topics are shared by all\nthe documents. In order to resolve these challenges, we use a subsampling\nstrategy to assign each document a different gamma process from the global\ngamma process, and the subsampling probabilities of documents are assigned with\na Markov Random Field constraint that inherits the document network structure.\nThrough the designed posterior inference algorithm, we can discover the hidden\ntopics and its number simultaneously. Experimental results on both synthetic\nand real-world network datasets demonstrate the capabilities of learning the\nhidden topics and, more importantly, the number of topics.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1503.08581v1",
            "date": "2015-03-30 08:03:47+00:00",
            "title": "LSHTC: A Benchmark for Large-Scale Text Classification",
            "authors": [
                "Ioannis Partalas",
                "Aris Kosmopoulos",
                "Nicolas Baskiotis",
                "Thierry Artieres",
                "George Paliouras",
                "Eric Gaussier",
                "Ion Androutsopoulos",
                "Massih-Reza Amini",
                "Patrick Galinari"
            ],
            "abstract": "LSHTC is a series of challenges which aims to assess the performance of\nclassification systems in large-scale classification in a a large number of\nclasses (up to hundreds of thousands). This paper describes the dataset that\nhave been released along the LSHTC series. The paper details the construction\nof the datsets and the design of the tracks as well as the evaluation measures\nthat we implemented and a quick overview of the results. All of these datasets\nare available online and runs may still be submitted on the online server of\nthe challenges.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1507.01053v1",
            "date": "2015-07-04 01:06:16+00:00",
            "title": "Describing Multimedia Content using Attention-based Encoder--Decoder Networks",
            "authors": [
                "Kyunghyun Cho",
                "Aaron Courville",
                "Yoshua Bengio"
            ],
            "abstract": "Whereas deep neural networks were first mostly used for classification tasks,\nthey are rapidly expanding in the realm of structured output problems, where\nthe observed target is composed of multiple random variables that have a rich\njoint distribution, given the input. We focus in this paper on the case where\nthe input also has a rich structure and the input and output structures are\nsomehow related. We describe systems that learn to attend to different places\nin the input, for each element of the output, for a variety of tasks: machine\ntranslation, image caption generation, video clip description and speech\nrecognition. All these systems are based on a shared set of building blocks:\ngated recurrent neural networks and convolutional neural networks, along with\ntrained attention mechanisms. We report on experimental results with these\nsystems, showing impressively good performance and the advantage of the\nattention mechanism.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1507.01193v1",
            "date": "2015-07-05 11:10:24+00:00",
            "title": "Dependency Recurrent Neural Language Models for Sentence Completion",
            "authors": [
                "Piotr Mirowski",
                "Andreas Vlachos"
            ],
            "abstract": "Recent work on language modelling has shifted focus from count-based models\nto neural models. In these works, the words in each sentence are always\nconsidered in a left-to-right order. In this paper we show how we can improve\nthe performance of the recurrent neural network (RNN) language model by\nincorporating the syntactic dependencies of a sentence, which have the effect\nof bringing relevant contexts closer to the word being predicted. We evaluate\nour approach on the Microsoft Research Sentence Completion Challenge and show\nthat the dependency RNN proposed improves over the RNN by about 10 points in\naccuracy. Furthermore, we achieve results comparable with the state-of-the-art\nmodels on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.01526v3",
            "date": "2015-07-06 16:30:05+00:00",
            "title": "Grid Long Short-Term Memory",
            "authors": [
                "Nal Kalchbrenner",
                "Ivo Danihelka",
                "Alex Graves"
            ],
            "abstract": "This paper introduces Grid Long Short-Term Memory, a network of LSTM cells\narranged in a multidimensional grid that can be applied to vectors, sequences\nor higher dimensional data such as images. The network differs from existing\ndeep LSTM architectures in that the cells are connected between network layers\nas well as along the spatiotemporal dimensions of the data. The network\nprovides a unified way of using LSTM for both deep and sequential computation.\nWe apply the model to algorithmic tasks such as 15-digit integer addition and\nsequence memorization, where it is able to significantly outperform the\nstandard LSTM. We then give results for two empirical tasks. We find that 2D\nGrid LSTM achieves 1.47 bits per character on the Wikipedia character\nprediction benchmark, which is state-of-the-art among neural approaches. In\naddition, we use the Grid LSTM to define a novel two-dimensional translation\nmodel, the Reencoder, and show that it outperforms a phrase-based reference\nsystem on a Chinese-to-English translation task.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1507.01839v2",
            "date": "2015-07-07 15:20:36+00:00",
            "title": "Dependency-based Convolutional Neural Networks for Sentence Embedding",
            "authors": [
                "Mingbo Ma",
                "Liang Huang",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "In sentence modeling and classification, convolutional neural network\napproaches have recently achieved state-of-the-art results, but all such\nefforts process word vectors sequentially and neglect long-distance\ndependencies. To exploit both deep learning and linguistic structures, we\npropose a tree-based convolutional neural network model which exploit various\nlong-distance relationships between words. Our model improves the sequential\nbaselines on all three sentiment and question classification tasks, and\nachieves the highest published accuracy on TREC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02020v1",
            "date": "2015-07-08 04:27:48+00:00",
            "title": "Generating Navigable Semantic Maps from Social Sciences Corpora",
            "authors": [
                "Thierry Poibeau",
                "Pablo Ruiz"
            ],
            "abstract": "It is now commonplace to observe that we are facing a deluge of online\ninformation. Researchers have of course long acknowledged the potential value\nof this information since digital traces make it possible to directly observe,\ndescribe and analyze social facts, and above all the co-evolution of ideas and\ncommunities over time. However, most online information is expressed through\ntext, which means it is not directly usable by machines, since computers\nrequire structured, organized and typed information in order to be able to\nmanipulate it. Our goal is thus twofold: 1. Provide new natural language\nprocessing techniques aiming at automatically extracting relevant information\nfrom texts, especially in the context of social sciences, and connect these\npieces of information so as to obtain relevant socio-semantic networks; 2.\nProvide new ways of exploring these socio-semantic networks, thanks to tools\nallowing one to dynamically navigate these networks, de-construct and\nre-construct them interactively, from different points of view following the\nneeds expressed by domain experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.02140v1",
            "date": "2015-07-08 13:14:38+00:00",
            "title": "Mining and Analyzing the Future Works in Scientific Articles",
            "authors": [
                "Yue Hu",
                "Xiaojun Wan"
            ],
            "abstract": "Future works in scientific articles are valuable for researchers and they can\nguide researchers to new research directions or ideas. In this paper, we mine\nthe future works in scientific articles in order to 1) provide an insight for\nfuture work analysis and 2) facilitate researchers to search and browse future\nworks in a research area. First, we study the problem of future work extraction\nand propose a regular expression based method to address the problem. Second,\nwe define four different categories for the future works by observing the data\nand investigate the multi-class future work classification problem. Third, we\napply the extraction method and the classification model to a paper dataset in\nthe computer science field and conduct a further analysis of the future works.\nFinally, we design a prototype system to search and demonstrate the future\nworks mined from the scientific papers. Our evaluation results show that our\nextraction method can get high precision and recall values and our\nclassification model can also get good results and it outperforms several\nbaseline models. Further analysis of the future work sentences also indicates\ninteresting results.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04116v1",
            "date": "2015-07-15 08:19:14+00:00",
            "title": "Language discrimination and clustering via a neural network approach",
            "authors": [
                "Angelo Mariano",
                "Giorgio Parisi",
                "Saverio Pascazio"
            ],
            "abstract": "We classify twenty-one Indo-European languages starting from written text. We\nuse neural networks in order to define a distance among different languages,\nconstruct a dendrogram and analyze the ultrametric structure that emerges. Four\nor five subgroups of languages are identified, according to the \"cut\" of the\ndendrogram, drawn with an entropic criterion. The results and the method are\ndiscussed.",
            "journal ref": "None",
            "category": "cond-mat.dis-nn"
        },
        {
            "url": "http://arxiv.org/abs/1507.04646v1",
            "date": "2015-07-16 16:43:55+00:00",
            "title": "A Dependency-Based Neural Network for Relation Classification",
            "authors": [
                "Yang Liu",
                "Furu Wei",
                "Sujian Li",
                "Heng Ji",
                "Ming Zhou",
                "Houfeng Wang"
            ],
            "abstract": "Previous research on relation classification has verified the effectiveness\nof using dependency shortest paths or subtrees. In this paper, we further\nexplore how to make full use of the combination of these dependency\ninformation. We first propose a new structure, termed augmented dependency path\n(ADP), which is composed of the shortest dependency path between two entities\nand the subtrees attached to the shortest path. To exploit the semantic\nrepresentation behind the ADP structure, we develop dependency-based neural\nnetworks (DepNN): a recursive neural network designed to model the subtrees,\nand a convolutional neural network to capture the most important features on\nthe shortest path. Experiments on the SemEval-2010 dataset show that our\nproposed method achieves state-of-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.04798v1",
            "date": "2015-07-16 23:11:45+00:00",
            "title": "Exploratory topic modeling with distributional semantics",
            "authors": [
                "Samuel R\u00f6nnqvist"
            ],
            "abstract": "As we continue to collect and store textual data in a multitude of domains,\nwe are regularly confronted with material whose largely unknown thematic\nstructure we want to uncover. With unsupervised, exploratory analysis, no prior\nknowledge about the content is required and highly open-ended tasks can be\nsupported. In the past few years, probabilistic topic modeling has emerged as a\npopular approach to this problem. Nevertheless, the representation of the\nlatent topics as aggregations of semi-coherent terms limits their\ninterpretability and level of detail.\n  This paper presents an alternative approach to topic modeling that maps\ntopics as a network for exploration, based on distributional semantics using\nlearned word vectors. From the granular level of terms and their semantic\nsimilarity relations global topic structures emerge as clustered regions and\ngradients of concepts. Moreover, the paper discusses the visual interactive\nrepresentation of the topic map, which plays an important role in supporting\nits exploration.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1507.04808v3",
            "date": "2015-07-17 00:21:39+00:00",
            "title": "Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models",
            "authors": [
                "Iulian V. Serban",
                "Alessandro Sordoni",
                "Yoshua Bengio",
                "Aaron Courville",
                "Joelle Pineau"
            ],
            "abstract": "We investigate the task of building open domain, conversational dialogue\nsystems based on large dialogue corpora using generative models. Generative\nmodels produce system responses that are autonomously generated word-by-word,\nopening up the possibility for realistic, flexible interactions. In support of\nthis goal, we extend the recently proposed hierarchical recurrent\nencoder-decoder neural network to the dialogue domain, and demonstrate that\nthis model is competitive with state-of-the-art neural language models and\nback-off n-gram models. We investigate the limitations of this and similar\napproaches, and show how its performance can be improved by bootstrapping the\nlearning from a larger question-answer pair corpus and from pretrained word\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.05910v3",
            "date": "2015-07-21 16:53:12+00:00",
            "title": "Clustering is Efficient for Approximate Maximum Inner Product Search",
            "authors": [
                "Alex Auvolat",
                "Sarath Chandar",
                "Pascal Vincent",
                "Hugo Larochelle",
                "Yoshua Bengio"
            ],
            "abstract": "Efficient Maximum Inner Product Search (MIPS) is an important task that has a\nwide applicability in recommendation systems and classification with a large\nnumber of classes. Solutions based on locality-sensitive hashing (LSH) as well\nas tree-based solutions have been investigated in the recent literature, to\nperform approximate MIPS in sublinear time. In this paper, we compare these to\nanother extremely simple approach for solving approximate MIPS, based on\nvariants of the k-means clustering algorithm. Specifically, we propose to train\na spherical k-means, after having reduced the MIPS problem to a Maximum Cosine\nSimilarity Search (MCSS). Experiments on two standard recommendation system\nbenchmarks as well as on large vocabulary word embeddings, show that this\nsimple approach yields much higher speedups, for the same retrieval precision,\nthan current state-of-the-art hashing-based and tree-based methods. This simple\nmethod also yields more robust retrievals when the query is corrupted by noise.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1507.06829v1",
            "date": "2015-07-24 13:01:20+00:00",
            "title": "The Polylingual Labeled Topic Model",
            "authors": [
                "Lisa Posch",
                "Arnim Bleier",
                "Philipp Schaer",
                "Markus Strohmaier"
            ],
            "abstract": "In this paper, we present the Polylingual Labeled Topic Model, a model which\ncombines the characteristics of the existing Polylingual Topic Model and\nLabeled LDA. The model accounts for multiple languages with separate topic\ndistributions for each language while restricting the permitted topics of a\ndocument to a set of predefined labels. We explore the properties of the model\nin a two-language setting on a dataset from the social science domain. Our\nexperiments show that our model outperforms LDA and Labeled LDA in terms of\ntheir held-out perplexity and that it produces semantically coherent topics\nwhich are well interpretable by human subjects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.06947v1",
            "date": "2015-07-24 18:28:32+00:00",
            "title": "Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition",
            "authors": [
                "Ha\u015fim Sak",
                "Andrew Senior",
                "Kanishka Rao",
                "Fran\u00e7oise Beaufays"
            ],
            "abstract": "We have recently shown that deep Long Short-Term Memory (LSTM) recurrent\nneural networks (RNNs) outperform feed forward deep neural networks (DNNs) as\nacoustic models for speech recognition. More recently, we have shown that the\nperformance of sequence trained context dependent (CD) hidden Markov model\n(HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained\nphone models initialized with connectionist temporal classification (CTC). In\nthis paper, we present techniques that further improve performance of LSTM RNN\nacoustic models for large vocabulary speech recognition. We show that frame\nstacking and reduced frame rate lead to more accurate models and faster\ndecoding. CD phone modeling leads to further improvements. We also present\ninitial results for LSTM RNN models outputting words directly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.07998v1",
            "date": "2015-07-29 01:04:28+00:00",
            "title": "Document Embedding with Paragraph Vectors",
            "authors": [
                "Andrew M. Dai",
                "Christopher Olah",
                "Quoc V. Le"
            ],
            "abstract": "Paragraph Vectors has been recently proposed as an unsupervised method for\nlearning distributed representations for pieces of texts. In their work, the\nauthors showed that the method can learn an embedding of movie review texts\nwhich can be leveraged for sentiment analysis. That proof of concept, while\nencouraging, was rather narrow. Here we consider tasks other than sentiment\nanalysis, provide a more thorough comparison of Paragraph Vectors to other\ndocument modelling algorithms such as Latent Dirichlet Allocation, and evaluate\nperformance of the method as we vary the dimensionality of the learned\nrepresentation. We benchmarked the models on two document similarity data sets,\none from Wikipedia, one from arXiv. We observe that the Paragraph Vector method\nperforms significantly better than other methods, and propose a simple\nimprovement to enhance embedding quality. Somewhat surprisingly, we also show\nthat much like word embeddings, vector operations on Paragraph Vectors can\nperform useful semantic results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.08396v1",
            "date": "2015-07-30 06:44:37+00:00",
            "title": "Tag-Weighted Topic Model For Large-scale Semi-Structured Documents",
            "authors": [
                "Shuangyin Li",
                "Jiefei Li",
                "Guan Huang",
                "Ruiyang Tan",
                "Rong Pan"
            ],
            "abstract": "To date, there have been massive Semi-Structured Documents (SSDs) during the\nevolution of the Internet. These SSDs contain both unstructured features (e.g.,\nplain text) and metadata (e.g., tags). Most previous works focused on modeling\nthe unstructured text, and recently, some other methods have been proposed to\nmodel the unstructured text with specific tags. To build a general model for\nSSDs remains an important problem in terms of both model fitness and\nefficiency. We propose a novel method to model the SSDs by a so-called\nTag-Weighted Topic Model (TWTM). TWTM is a framework that leverages both the\ntags and words information, not only to learn the document-topic and topic-word\ndistributions, but also to infer the tag-topic distributions for text mining\ntasks. We present an efficient variational inference method with an EM\nalgorithm for estimating the model parameters. Meanwhile, we propose three\nlarge-scale solutions for our model under the MapReduce distributed computing\nplatform for modeling large-scale SSDs. The experimental results show the\neffectiveness, efficiency and the robustness by comparing our model with the\nstate-of-the-art methods in document modeling, tags prediction and text\nclassification. We also show the performance of the three distributed solutions\nin terms of time and accuracy on document modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00112v1",
            "date": "2015-12-01 01:11:46+00:00",
            "title": "Inferring Interpersonal Relations in Narrative Summaries",
            "authors": [
                "Shashank Srivastava",
                "Snigdha Chaturvedi",
                "Tom Mitchell"
            ],
            "abstract": "Characterizing relationships between people is fundamental for the\nunderstanding of narratives. In this work, we address the problem of inferring\nthe polarity of relationships between people in narrative summaries. We\nformulate the problem as a joint structured prediction for each narrative, and\npresent a model that combines evidence from linguistic and semantic features,\nas well as features based on the structure of the social community in the text.\nWe also provide a clustering-based approach that can exploit regularities in\nnarrative types. e.g., learn an affinity for love-triangles in romantic\nstories. On a dataset of movie summaries from Wikipedia, our structured models\nprovide more than a 30% error-reduction over a competitive baseline that\nconsiders pairs of characters in isolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00177v3",
            "date": "2015-12-01 08:43:19+00:00",
            "title": "LSTM Neural Reordering Feature for Statistical Machine Translation",
            "authors": [
                "Yiming Cui",
                "Shijin Wang",
                "Jianfeng Li"
            ],
            "abstract": "Artificial neural networks are powerful models, which have been widely\napplied into many aspects of machine translation, such as language modeling and\ntranslation modeling. Though notable improvements have been made in these\nareas, the reordering problem still remains a challenge in statistical machine\ntranslations. In this paper, we present a novel neural reordering model that\ndirectly models word pairs and alignment. By utilizing LSTM recurrent neural\nnetworks, much longer context could be learned for reordering prediction.\nExperimental results on NIST OpenMT12 Arabic-English and Chinese-English\n1000-best rescoring task show that our LSTM neural reordering feature is robust\nand achieves significant improvements over various baseline systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.00818v2",
            "date": "2015-12-02 19:34:00+00:00",
            "title": "Zero-Shot Event Detection by Multimodal Distributional Semantic Embedding of Videos",
            "authors": [
                "Mohamed Elhoseiny",
                "Jingen Liu",
                "Hui Cheng",
                "Harpreet Sawhney",
                "Ahmed Elgammal"
            ],
            "abstract": "We propose a new zero-shot Event Detection method by Multi-modal\nDistributional Semantic embedding of videos. Our model embeds object and action\nconcepts as well as other available modalities from videos into a\ndistributional semantic space. To our knowledge, this is the first Zero-Shot\nevent detection model that is built on top of distributional semantics and\nextends it in the following directions: (a) semantic embedding of multimodal\ninformation in videos (with focus on the visual modalities), (b) automatically\ndetermining relevance of concepts/attributes to a free text query, which could\nbe useful for other applications, and (c) retrieving videos by free text event\nquery (e.g., \"changing a vehicle tire\") based on their content. We embed videos\ninto a distributional semantic space and then measure the similarity between\nvideos and the event query in a free text form. We validated our method on the\nlarge TRECVID MED (Multimedia Event Detection) challenge. Using only the event\ntitle as a query, our method outperformed the state-of-the-art that uses big\ndescriptions from 12.6% to 13.5% with MAP metric and 0.73 to 0.83 with ROC-AUC\nmetric. It is also an order of magnitude faster.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1512.00965v2",
            "date": "2015-12-03 06:46:27+00:00",
            "title": "Neural Enquirer: Learning to Query Tables with Natural Language",
            "authors": [
                "Pengcheng Yin",
                "Zhengdong Lu",
                "Hang Li",
                "Ben Kao"
            ],
            "abstract": "We proposed Neural Enquirer as a neural network architecture to execute a\nnatural language (NL) query on a knowledge-base (KB) for answers. Basically,\nNeural Enquirer finds the distributed representation of a query and then\nexecutes it on knowledge-base tables to obtain the answer as one of the values\nin the tables. Unlike similar efforts in end-to-end training of semantic\nparsers, Neural Enquirer is fully \"neuralized\": it not only gives\ndistributional representation of the query and the knowledge-base, but also\nrealizes the execution of compositional queries as a series of differentiable\noperations, with intermediate results (consisting of annotations of the tables\nat different levels) saved on multiple layers of memory. Neural Enquirer can be\ntrained with gradient descent, with which not only the parameters of the\ncontrolling components and semantic parsing component, but also the embeddings\nof the tables and query words can be learned from scratch. The training can be\ndone in an end-to-end fashion, but it can take stronger guidance, e.g., the\nstep-by-step supervision for complicated queries, and benefit from it. Neural\nEnquirer is one step towards building neural network systems which seek to\nunderstand language by executing it on real-world. Our experiments show that\nNeural Enquirer can learn to execute fairly complicated NL queries on tables\nwith rich structures.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1512.01043v1",
            "date": "2015-12-03 11:29:36+00:00",
            "title": "Approaches for Sentiment Analysis on Twitter: A State-of-Art study",
            "authors": [
                "Harsh Thakkar",
                "Dhiren Patel"
            ],
            "abstract": "Microbloging is an extremely prevalent broadcast medium amidst the Internet\nfraternity these days. People share their opinions and sentiments about variety\nof subjects like products, news, institutions, etc., every day on microbloging\nwebsites. Sentiment analysis plays a key role in prediction systems, opinion\nmining systems, etc. Twitter, one of the microbloging platforms allows a limit\nof 140 characters to its users. This restriction stimulates users to be very\nconcise about their opinion and twitter an ocean of sentiments to analyze.\nTwitter also provides developer friendly streaming API for data retrieval\npurpose allowing the analyst to search real time tweets from various users. In\nthis paper, we discuss the state-of-art of the works which are focused on\nTwitter, the online social network platform, for sentiment analysis. We survey\nvarious lexical, machine learning and hybrid approaches for sentiment analysis\non Twitter.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1512.01173v1",
            "date": "2015-12-03 17:52:50+00:00",
            "title": "Building Memory with Concept Learning Capabilities from Large-scale Knowledge Base",
            "authors": [
                "Jiaxin Shi",
                "Jun Zhu"
            ],
            "abstract": "We present a new perspective on neural knowledge base (KB) embeddings, from\nwhich we build a framework that can model symbolic knowledge in the KB together\nwith its learning process. We show that this framework well regularizes\nprevious neural KB embedding model for superior performance in reasoning tasks,\nwhile having the capabilities of dealing with unseen entities, that is, to\nlearn their embeddings from natural language descriptions, which is very like\nhuman's behavior of learning semantic concepts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01525v1",
            "date": "2015-12-04 20:00:08+00:00",
            "title": "Learning the Semantics of Manipulation Action",
            "authors": [
                "Yezhou Yang",
                "Yiannis Aloimonos",
                "Cornelia Fermuller",
                "Eren Erdal Aksoy"
            ],
            "abstract": "In this paper we present a formal computational framework for modeling\nmanipulation actions. The introduced formalism leads to semantics of\nmanipulation action and has applications to both observing and understanding\nhuman manipulation actions as well as executing them with a robotic mechanism\n(e.g. a humanoid robot). It is based on a Combinatory Categorial Grammar. The\ngoal of the introduced framework is to: (1) represent manipulation actions with\nboth syntax and semantic parts, where the semantic part employs\n$\\lambda$-calculus; (2) enable a probabilistic semantic parsing schema to learn\nthe $\\lambda$-calculus representation of manipulation action from an annotated\naction corpus of videos; (3) use (1) and (2) to develop a system that visually\nobserves manipulation actions and understands their meaning while it can reason\nbeyond observations using propositional logic and axiom schemata. The\nexperiments conducted on a public available large manipulation action dataset\nvalidate the theoretical framework and our implementation.",
            "journal ref": "The 53rd Annual Meeting of the Association for Computational\n  Linguistics (ACL) 1 (2015) 676-686",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1512.01712v1",
            "date": "2015-12-05 23:41:22+00:00",
            "title": "Generating News Headlines with Recurrent Neural Networks",
            "authors": [
                "Konstantin Lopyrev"
            ],
            "abstract": "We describe an application of an encoder-decoder recurrent neural network\nwith LSTM units and attention to generating headlines from the text of news\narticles. We find that the model is quite effective at concisely paraphrasing\nnews articles. Furthermore, we study how the neural network decides which input\nwords to pay attention to, and specifically we identify the function of the\ndifferent neurons in a simplified attention mechanism. Interestingly, our\nsimplified attention mechanism performs better that the more complex attention\nmechanism on a held out set of articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.01926v1",
            "date": "2015-12-07 06:37:49+00:00",
            "title": "Thinking Required",
            "authors": [
                "Kamil Rocki"
            ],
            "abstract": "There exists a theory of a single general-purpose learning algorithm which\ncould explain the principles its operation. It assumes the initial rough\narchitecture, a small library of simple innate circuits which are prewired at\nbirth. and proposes that all significant mental algorithms are learned. Given\ncurrent understanding and observations, this paper reviews and lists the\ningredients of such an algorithm from architectural and functional\nperspectives.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1512.02009v1",
            "date": "2015-12-07 12:16:58+00:00",
            "title": "Jointly Modeling Topics and Intents with Global Order Structure",
            "authors": [
                "Bei Chen",
                "Jun Zhu",
                "Nan Yang",
                "Tian Tian",
                "Ming Zhou",
                "Bo Zhang"
            ],
            "abstract": "Modeling document structure is of great importance for discourse analysis and\nrelated applications. The goal of this research is to capture the document\nintent structure by modeling documents as a mixture of topic words and\nrhetorical words. While the topics are relatively unchanged through one\ndocument, the rhetorical functions of sentences usually change following\ncertain orders in discourse. We propose GMM-LDA, a topic modeling based\nBayesian unsupervised model, to analyze the document intent structure\ncooperated with order information. Our model is flexible that has the ability\nto combine the annotations and do supervised learning. Additionally, entropic\nregularization can be introduced to model the significant divergence between\ntopics and intents. We perform experiments in both unsupervised and supervised\nsettings, results show the superiority of our model over several\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.02567v1",
            "date": "2015-12-08 17:54:17+00:00",
            "title": "Distributed Adaptive LMF Algorithm for Sparse Parameter Estimation in Gaussian Mixture Noise",
            "authors": [
                "Mojtaba Hajiabadi"
            ],
            "abstract": "A distributed adaptive algorithm for estimation of sparse unknown parameters\nin the presence of nonGaussian noise is proposed in this paper based on\nnormalized least mean fourth (NLMF) criterion. At the first step, local\nadaptive NLMF algorithm is modified by zero norm in order to speed up the\nconvergence rate and also to reduce the steady state error power in sparse\nconditions. Then, the proposed algorithm is extended for distributed scenario\nin which more improvement in estimation performance is achieved due to\ncooperation of local adaptive filters. Simulation results show the superiority\nof the proposed algorithm in comparison with conventional NLMF algorithms.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1512.03460v1",
            "date": "2015-12-10 21:58:46+00:00",
            "title": "Neural Self Talk: Image Understanding via Continuous Questioning and Answering",
            "authors": [
                "Yezhou Yang",
                "Yi Li",
                "Cornelia Fermuller",
                "Yiannis Aloimonos"
            ],
            "abstract": "In this paper we consider the problem of continuously discovering image\ncontents by actively asking image based questions and subsequently answering\nthe questions being asked. The key components include a Visual Question\nGeneration (VQG) module and a Visual Question Answering module, in which\nRecurrent Neural Networks (RNN) and Convolutional Neural Network (CNN) are\nused. Given a dataset that contains images, questions and their answers, both\nmodules are trained at the same time, with the difference being VQG uses the\nimages as input and the corresponding questions as output, while VQA uses\nimages and questions as input and the corresponding answers as output. We\nevaluate the self talk process subjectively using Amazon Mechanical Turk, which\nshow effectiveness of the proposed method.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1512.03549v1",
            "date": "2015-12-11 08:44:45+00:00",
            "title": "Words are not Equal: Graded Weighting Model for building Composite Document Vectors",
            "authors": [
                "Pranjal Singh",
                "Amitabha Mukerjee"
            ],
            "abstract": "Despite the success of distributional semantics, composing phrases from word\nvectors remains an important challenge. Several methods have been tried for\nbenchmark tasks such as sentiment classification, including word vector\naveraging, matrix-vector approaches based on parsing, and on-the-fly learning\nof paragraph vectors. Most models usually omit stop words from the composition.\nInstead of such an yes-no decision, we consider several graded schemes where\nwords are weighted according to their discriminatory relevance with respect to\nits use in the document (e.g., idf). Some of these methods (particularly\ntf-idf) are seen to result in a significant improvement in performance over\nprior state of the art. Further, combining such approaches into an ensemble\nbased on alternate classifiers such as the RNN model, results in an 1.6%\nperformance improvement on the standard IMDB movie review dataset, and a 7.01%\nimprovement on Amazon product reviews. Since these are language free models and\ncan be obtained in an unsupervised manner, they are of interest also for\nunder-resourced languages such as Hindi as well and many more languages. We\ndemonstrate the language free aspects by showing a gain of 12% for two review\ndatasets over earlier results, and also release a new larger dataset for future\ntesting (Singh,2015).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04280v4",
            "date": "2015-12-14 12:29:32+00:00",
            "title": "Small-footprint Deep Neural Networks with Highway Connections for Speech Recognition",
            "authors": [
                "Liang Lu",
                "Steve Renals"
            ],
            "abstract": "For speech recognition, deep neural networks (DNNs) have significantly\nimproved the recognition accuracy in most of benchmark datasets and application\ndomains. However, compared to the conventional Gaussian mixture models,\nDNN-based acoustic models usually have much larger number of model parameters,\nmaking it challenging for their applications in resource constrained platforms,\ne.g., mobile devices. In this paper, we study the application of the recently\nproposed highway network to train small-footprint DNNs, which are {\\it thinner}\nand {\\it deeper}, and have significantly smaller number of model parameters\ncompared to conventional DNNs. We investigated this approach on the AMI meeting\nspeech transcription corpus which has around 70 hours of audio data. The\nhighway neural networks constantly outperformed their plain DNN counterparts,\nand the number of model parameters can be reduced significantly without\nsacrificing the recognition accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04407v4",
            "date": "2015-12-14 16:59:35+00:00",
            "title": "We Are Humor Beings: Understanding and Predicting Visual Humor",
            "authors": [
                "Arjun Chandrasekaran",
                "Ashwin K. Vijayakumar",
                "Stanislaw Antol",
                "Mohit Bansal",
                "Dhruv Batra",
                "C. Lawrence Zitnick",
                "Devi Parikh"
            ],
            "abstract": "Humor is an integral part of human lives. Despite being tremendously\nimpactful, it is perhaps surprising that we do not have a detailed\nunderstanding of humor yet. As interactions between humans and AI systems\nincrease, it is imperative that these systems are taught to understand\nsubtleties of human expressions such as humor. In this work, we are interested\nin the question - what content in a scene causes it to be funny? As a first\nstep towards understanding visual humor, we analyze the humor manifested in\nabstract scenes and design computational models for them. We collect two\ndatasets of abstract scenes that facilitate the study of humor at both the\nscene-level and the object-level. We analyze the funny scenes and explore the\ndifferent types of humor depicted in them via human studies. We model two tasks\nthat we believe demonstrate an understanding of some aspects of visual humor.\nThe tasks involve predicting the funniness of a scene and altering the\nfunniness of a scene. We show that our models perform well quantitatively, and\nqualitatively through human studies. Our datasets are publicly available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1512.04419v2",
            "date": "2015-12-14 17:36:35+00:00",
            "title": "Sentence Entailment in Compositional Distributional Semantics",
            "authors": [
                "Esma Balkir",
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Distributional semantic models provide vector representations for words by\ngathering co-occurrence frequencies from corpora of text. Compositional\ndistributional models extend these from words to phrases and sentences. In\ncategorical compositional distributional semantics, phrase and sentence\nrepresentations are functions of their grammatical structure and\nrepresentations of the words therein. In this setting, grammatical structures\nare formalised by morphisms of a compact closed category and meanings of words\nare formalised by objects of the same category. These can be instantiated in\nthe form of vectors or density matrices. This paper concerns the applications\nof this model to phrase and sentence level entailment. We argue that\nentropy-based distances of vectors and density matrices provide a good\ncandidate to measure word-level entailment, show the advantage of density\nmatrices over vectors for word level entailments, and prove that these\ndistances extend compositionally from words to phrases and sentences. We\nexemplify our theoretical constructions on real data and a toy entailment\ndataset and provide preliminary experimental evidence.",
            "journal ref": "Ann Math Artif Intell (2018) 82: 189.\n  https://doi.org/10.1007/s10472-017-9570-x",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.04701v1",
            "date": "2015-12-15 10:01:37+00:00",
            "title": "Joint Image-Text News Topic Detection and Tracking with And-Or Graph Representation",
            "authors": [
                "Weixin Li",
                "Jungseock Joo",
                "Hang Qi",
                "Song-Chun Zhu"
            ],
            "abstract": "In this paper, we aim to develop a method for automatically detecting and\ntracking topics in broadcast news. We present a hierarchical And-Or graph (AOG)\nto jointly represent the latent structure of both texts and visuals. The AOG\nembeds a context sensitive grammar that can describe the hierarchical\ncomposition of news topics by semantic elements about people involved, related\nplaces and what happened, and model contextual relationships between elements\nin the hierarchy. We detect news topics through a cluster sampling process\nwhich groups stories about closely related events. Swendsen-Wang Cuts (SWC), an\neffective cluster sampling algorithm, is adopted for traversing the solution\nspace and obtaining optimal clustering solutions by maximizing a Bayesian\nposterior probability. Topics are tracked to deal with the continuously updated\nnews streams. We generate topic trajectories to show how topics emerge, evolve\nand disappear over time. The experimental results show that our method can\nexplicitly describe the textual and visual data in news videos and produce\nmeaningful topic trajectories. Our method achieves superior performance\ncompared to state-of-the-art methods on both a public dataset Reuters-21578 and\na self-collected dataset named UCLA Broadcast News Dataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1512.05004v3",
            "date": "2015-12-15 23:07:58+00:00",
            "title": "Towards Evaluation of Cultural-scale Claims in Light of Topic Model Sampling Effects",
            "authors": [
                "Jaimie Murdock",
                "Jiaan Zeng",
                "Colin Allen"
            ],
            "abstract": "Cultural-scale models of full text documents are prone to over-interpretation\nby researchers making unintentionally strong socio-linguistic claims (Pechenick\net al., 2015) without recognizing that even large digital libraries are merely\nsamples of all the books ever produced. In this study, we test the sensitivity\nof the topic models to the sampling process by taking random samples of books\nin the Hathi Trust Digital Library from different areas of the Library of\nCongress Classification Outline. For each classification area, we train several\ntopic models over the entire class with different random seeds, generating a\nset of spanning models. Then, we train topic models on random samples of books\nfrom the classification area, generating a set of sample models. Finally, we\nperform a topic alignment between each pair of models by computing the\nJensen-Shannon distance (JSD) between the word probability distributions for\neach topic. We take two measures on each model alignment: alignment distance\nand topic overlap. We find that sample models with a large sample size\ntypically have an alignment distance that falls in the range of the alignment\ndistance between spanning models. Unsurprisingly, as sample size increases,\nalignment distance decreases. We also find that the topic overlap increases as\nsample size increases. However, the decomposition of these measures by sample\nsize differs by number of topics and by classification area. We speculate that\nthese measures could be used to find classes which have a common \"canon\"\ndiscussed among all books in the area, as shown by high topic overlap and low\nalignment distance even in small sample sizes.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1512.06612v2",
            "date": "2015-12-21 13:07:31+00:00",
            "title": "Backward and Forward Language Modeling for Constrained Sentence Generation",
            "authors": [
                "Lili Mou",
                "Rui Yan",
                "Ge Li",
                "Lu Zhang",
                "Zhi Jin"
            ],
            "abstract": "Recent language models, especially those based on recurrent neural networks\n(RNNs), make it possible to generate natural language from a learned\nprobability. Language generation has wide applications including machine\ntranslation, summarization, question answering, conversation systems, etc.\nExisting methods typically learn a joint probability of words conditioned on\nadditional information, which is (either statically or dynamically) fed to\nRNN's hidden layer. In many applications, we are likely to impose hard\nconstraints on the generated texts, i.e., a particular word must appear in the\nsentence. Unfortunately, existing approaches could not solve this problem. In\nthis paper, we propose a novel backward and forward language model. Provided a\nspecific word, we use RNNs to generate previous words and future words, either\nsimultaneously or asynchronously, resulting in two model variants. In this way,\nthe given word could appear at any position in the sentence. Experimental\nresults show that the generated texts are comparable to sequential LMs in\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.07281v1",
            "date": "2015-12-22 22:13:31+00:00",
            "title": "Topical differences between Chinese language Twitter and Sina Weibo",
            "authors": [
                "Qian Zhang",
                "Bruno Gon\u00e7alves"
            ],
            "abstract": "Sina Weibo, China's most popular microblogging platform, is currently used by\nover $500M$ users and is considered to be a proxy of Chinese social life. In\nthis study, we contrast the discussions occurring on Sina Weibo and on Chinese\nlanguage Twitter in order to observe two different strands of Chinese culture:\npeople within China who use Sina Weibo with its government imposed restrictions\nand those outside that are free to speak completely anonymously. We first\npropose a simple ad-hoc algorithm to identify topics of Tweets and Weibo.\nDifferent from previous works on micro-message topic detection, our algorithm\nconsiders topics of the same contents but with different \\#tags. Our algorithm\ncan also detect topics for Tweets and Weibos without any \\#tags. Using a large\ncorpus of Weibo and Chinese language tweets, covering the period from January\n$1$ to December $31$, $2012$, we obtain a list of topics using clustered \\#tags\nthat we can then use to compare the two platforms. Surprisingly, we find that\nthere are no common entries among the Top $100$ most popular topics.\nFurthermore, only $9.2\\%$ of tweets correspond to the Top $1000$ topics on Sina\nWeibo platform, and conversely only $4.4\\%$ of weibos were found to discuss the\nmost popular Twitter topics. Our results reveal significant differences in\nsocial attention on the two platforms, with most popular topics on Sina Weibo\nrelating to entertainment while most tweets corresponded to cultural or\npolitical contents that is practically non existent in Sina Weibo.",
            "journal ref": "2nd Natural Language Processing for Informal Text (NLPIT), 625\n  (2016)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1512.08849v2",
            "date": "2015-12-30 05:02:53+00:00",
            "title": "Learning Natural Language Inference with LSTM",
            "authors": [
                "Shuohang Wang",
                "Jing Jiang"
            ],
            "abstract": "Natural language inference (NLI) is a fundamentally important task in natural\nlanguage processing that has many applications. The recently released Stanford\nNatural Language Inference (SNLI) corpus has made it possible to develop and\nevaluate learning-centered methods such as deep neural networks for natural\nlanguage inference (NLI). In this paper, we propose a special long short-term\nmemory (LSTM) architecture for NLI. Our model builds on top of a recently\nproposed neural attention model for NLI but is based on a significantly\ndifferent idea. Instead of deriving sentence embeddings for the premise and the\nhypothesis to be used for classification, our solution uses a match-LSTM to\nperform word-by-word matching of the hypothesis with the premise. This LSTM is\nable to place more emphasis on important word-level matching results. In\nparticular, we observe that this LSTM remembers important mismatches that are\ncritical for predicting the contradiction or the neutral relationship label. On\nthe SNLI corpus, our model achieves an accuracy of 86.1%, outperforming the\nstate of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.08903v1",
            "date": "2015-12-30 10:32:12+00:00",
            "title": "Online Keyword Spotting with a Character-Level Recurrent Neural Network",
            "authors": [
                "Kyuyeon Hwang",
                "Minjae Lee",
                "Wonyong Sung"
            ],
            "abstract": "In this paper, we propose a context-aware keyword spotting model employing a\ncharacter-level recurrent neural network (RNN) for spoken term detection in\ncontinuous speech. The RNN is end-to-end trained with connectionist temporal\nclassification (CTC) to generate the probabilities of character and\nword-boundary labels. There is no need for the phonetic transcription, senone\nmodeling, or system dictionary in training and testing. Also, keywords can\neasily be added and modified by editing the text based keyword list without\nretraining the RNN. Moreover, the unidirectional RNN processes an infinitely\nlong input audio streams without pre-segmentation and keywords are detected\nwith low-latency before the utterance is finished. Experimental results show\nthat the proposed keyword spotter significantly outperforms the deep neural\nnetwork (DNN) and hidden Markov model (HMM) based keyword-filler model even\nwith less computations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00414v1",
            "date": "2015-04-27 11:24:50+00:00",
            "title": "On a Possible Similarity between Gene and Semantic Networks",
            "authors": [
                "Nicolas Turenne"
            ],
            "abstract": "In several domains such as linguistics, molecular biology or social sciences,\nholistic effects are hardly well-defined by modeling with single units, but\nmore and more studies tend to understand macro structures with the help of\nmeaningful and useful associations in fields such as social networks, systems\nbiology or semantic web. A stochastic multi-agent system offers both accurate\ntheoretical framework and operational computing implementations to model\nlarge-scale associations, their dynamics and patterns extraction. We show that\nclustering around a target object in a set of associations of object prove some\nsimilarity in specific data and two case studies about gene-gene and term-term\nrelationships leading to an idea of a common organizing principle of cognition\nwith random and deterministic effects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00259v2",
            "date": "2015-10-01 14:42:19+00:00",
            "title": "A Generative Model of Words and Relationships from Multiple Sources",
            "authors": [
                "Stephanie L. Hyland",
                "Theofanis Karaletsos",
                "Gunnar R\u00e4tsch"
            ],
            "abstract": "Neural language models are a powerful tool to embed words into semantic\nvector spaces. However, learning such models generally relies on the\navailability of abundant and diverse training examples. In highly specialised\ndomains this requirement may not be met due to difficulties in obtaining a\nlarge corpus, or the limited range of expression in average use. Such domains\nmay encode prior knowledge about entities in a knowledge base or ontology. We\npropose a generative model which integrates evidence from diverse data sources,\nenabling the sharing of semantic information. We achieve this by generalising\nthe concept of co-occurrence from distributional semantics to include other\nrelationships between entities or words, which we model as affine\ntransformations on the embedding space. We demonstrate the effectiveness of\nthis approach by outperforming recent models on a link prediction task and\ndemonstrating its ability to profit from partially or fully unobserved data\ntraining labels. We further demonstrate the usefulness of learning from\ndifferent data sources with overlapping vocabularies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.00277v2",
            "date": "2015-10-01 15:10:09+00:00",
            "title": "Similarity of symbol frequency distributions with heavy tails",
            "authors": [
                "Martin Gerlach",
                "Francesc Font-Clos",
                "Eduardo G. Altmann"
            ],
            "abstract": "Quantifying the similarity between symbolic sequences is a traditional\nproblem in Information Theory which requires comparing the frequencies of\nsymbols in different sequences. In numerous modern applications, ranging from\nDNA over music to texts, the distribution of symbol frequencies is\ncharacterized by heavy-tailed distributions (e.g., Zipf's law). The large\nnumber of low-frequency symbols in these distributions poses major difficulties\nto the estimation of the similarity between sequences, e.g., they hinder an\naccurate finite-size estimation of entropies. Here we show analytically how the\nsystematic (bias) and statistical (fluctuations) errors in these estimations\ndepend on the sample size~$N$ and on the exponent~$\\gamma$ of the heavy-tailed\ndistribution. Our results are valid for the Shannon entropy $(\\alpha=1)$, its\ncorresponding similarity measures (e.g., the Jensen-Shanon divergence), and\nalso for measures based on the generalized entropy of order $\\alpha$. For small\n$\\alpha$'s, including $\\alpha=1$, the errors decay slower than the $1/N$-decay\nobserved in short-tailed distributions. For $\\alpha$ larger than a critical\nvalue $\\alpha^* = 1+1/\\gamma \\leq 2$, the $1/N$-decay is recovered. We show the\npractical significance of our results by quantifying the evolution of the\nEnglish language over the last two centuries using a complete $\\alpha$-spectrum\nof measures. We find that frequent words change more slowly than less frequent\nwords and that $\\alpha=2$ provides the most robust measure to quantify language\nchange.",
            "journal ref": "Phys. Rev. X 6, 021009 (2016)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1510.01026v1",
            "date": "2015-10-05 04:16:34+00:00",
            "title": "Calculating entropy at different scales among diverse communication systems",
            "authors": [
                "Gerardo Febres",
                "Klaus Jaffe"
            ],
            "abstract": "We evaluated the impact of changing the observation scale over the entropy\nmeasures for text descriptions. MIDI coded Music, computer code and two human\nnatural languages were studied at the scale of characters, words, and at the\nFundamental Scale resulting from adjusting the symbols length used to interpret\neach text-description until it produced minimum entropy. The results show that\nthe Fundamental Scale method is comparable with the use of words when measuring\nentropy levels in written texts. However, this method can also be used in\ncommunication systems lacking words such as music. Measuring symbolic entropy\nat the fundamental scale allows to calculate quantitatively, relative levels of\ncomplexity for different communication systems. The results open novel vision\non differences among the structure of the communication systems studied.",
            "journal ref": "Complexity, Vol 21, Iss 51 (2016)",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1510.02693v1",
            "date": "2015-10-09 15:04:11+00:00",
            "title": "Feedforward Sequential Memory Neural Networks without Recurrent Feedback",
            "authors": [
                "ShiLiang Zhang",
                "Hui Jiang",
                "Si Wei",
                "LiRong Dai"
            ],
            "abstract": "We introduce a new structure for memory neural networks, called feedforward\nsequential memory networks (FSMN), which can learn long-term dependency without\nusing recurrent feedback. The proposed FSMN is a standard feedforward neural\nnetworks equipped with learnable sequential memory blocks in the hidden layers.\nIn this work, we have applied FSMN to several language modeling (LM) tasks.\nExperimental results have shown that the memory blocks in FSMN can learn\neffective representations of long history. Experiments have shown that FSMN\nbased language models can significantly outperform not only feedforward neural\nnetwork (FNN) based LMs but also the popular recurrent neural network (RNN)\nLMs.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1510.03797v1",
            "date": "2015-10-13 17:49:09+00:00",
            "title": "Complex Politics: A Quantitative Semantic and Topological Analysis of UK House of Commons Debates",
            "authors": [
                "Stefano Gurciullo",
                "Michael Smallegan",
                "Mar\u00eda Pereda",
                "Federico Battiston",
                "Alice Patania",
                "Sebastian Poledna",
                "Daniel Hedblom",
                "Bahattin Tolga Oztan",
                "Alexander Herzog",
                "Peter John",
                "Slava Mikhaylov"
            ],
            "abstract": "This study is a first, exploratory attempt to use quantitative semantics\ntechniques and topological analysis to analyze systemic patterns arising in a\ncomplex political system. In particular, we use a rich data set covering all\nspeeches and debates in the UK House of Commons between 1975 and 2014. By the\nuse of dynamic topic modeling (DTM) and topological data analysis (TDA) we show\nthat both members and parties feature specific roles within the system,\nconsistent over time, and extract global patterns indicating levels of\npolitical cohesion. Our results provide a wide array of novel hypotheses about\nthe complex dynamics of political systems, with valuable policy applications.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1510.03820v4",
            "date": "2015-10-13 19:00:57+00:00",
            "title": "A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification",
            "authors": [
                "Ye Zhang",
                "Byron Wallace"
            ],
            "abstract": "Convolutional Neural Networks (CNNs) have recently achieved remarkably strong\nperformance on the practically important task of sentence classification (kim\n2014, kalchbrenner 2014, johnson 2014). However, these models require\npractitioners to specify an exact model architecture and set accompanying\nhyperparameters, including the filter region size, regularization parameters,\nand so on. It is currently unknown how sensitive model performance is to\nchanges in these configurations for the task of sentence classification. We\nthus conduct a sensitivity analysis of one-layer CNNs to explore the effect of\narchitecture components on model performance; our aim is to distinguish between\nimportant and comparatively inconsequential design decisions for sentence\nclassification. We focus on one-layer CNNs (to the exclusion of more complex\nmodels) due to their comparative simplicity and strong empirical performance,\nwhich makes it a modern standard baseline method akin to Support Vector Machine\n(SVMs) and logistic regression. We derive practical advice from our extensive\nempirical results for those interested in getting the most out of CNNs for\nsentence classification in real world settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04709v2",
            "date": "2015-10-15 20:29:21+00:00",
            "title": "Multilingual Image Description with Neural Sequence Models",
            "authors": [
                "Desmond Elliott",
                "Stella Frank",
                "Eva Hasler"
            ],
            "abstract": "In this paper we present an approach to multi-language image description\nbringing together insights from neural machine translation and neural image\ndescription. To create a description of an image for a given target language,\nour sequence generation models condition on feature vectors from the image, the\ndescription from the source language, and/or a multimodal vector computed over\nthe image and a description in the source language. In image description\nexperiments on the IAPR-TC12 dataset of images aligned with English and German\nsentences, we find significant and substantial improvements in BLEU4 and Meteor\nscores for models trained over multiple languages, compared to a monolingual\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.04972v1",
            "date": "2015-10-16 18:35:13+00:00",
            "title": "Normalization of Relative and Incomplete Temporal Expressions in Clinical Narratives",
            "authors": [
                "Weiyi Sun",
                "Anna Rumshisky",
                "Ozlem Uzuner"
            ],
            "abstract": "We analyze the RI-TIMEXes in temporally annotated corpora and propose two\nhypotheses regarding the normalization of RI-TIMEXes in the clinical narrative\ndomain: the anchor point hypothesis and the anchor relation hypothesis. We\nannotate the RI-TIMEXes in three corpora to study the characteristics of\nRI-TMEXes in different domains. This informed the design of our RI-TIMEX\nnormalization system for the clinical domain, which consists of an anchor point\nclassifier, an anchor relation classifier and a rule-based RI-TIMEX text span\nparser. We experiment with different feature sets and perform error analysis\nfor each system component. The annotation confirmed the hypotheses that we can\nsimplify the RI-TIMEXes normalization task using two multi-label classifiers.\nOur system achieves anchor point classification, anchor relation classification\nand rule-based parsing accuracy of 74.68%, 87.71% and 57.2% (82.09% under\nrelaxed matching criteria) respectively on the held-out test set of the 2012\ni2b2 temporal relation challenge. Experiments with feature sets reveals some\ninteresting findings such as the verbal tense feature does not inform the\nanchor relation classification in clinical narratives as much as the tokens\nnear the RI-TIMEX. Error analysis shows that underrepresented anchor point and\nanchor relation classes are difficult to detect. We formulate the RI-TIMEX\nnormalization problem as a pair of multi-label classification problems.\nConsidering only the RI-TIMEX extraction and normalization, the system achieves\nstatistically significant improvement over the RI-TIMEX results of the best\nsystems in the 2012 i2b2 challenge.",
            "journal ref": "Journal of the American Medical Informatics Association (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.06342v1",
            "date": "2015-10-21 17:11:13+00:00",
            "title": "Prevalence and recoverability of syntactic parameters in sparse distributed memories",
            "authors": [
                "Jeong Joon Park",
                "Ronnel Boettcher",
                "Andrew Zhao",
                "Alex Mun",
                "Kevin Yuh",
                "Vibhor Kumar",
                "Matilde Marcolli"
            ],
            "abstract": "We propose a new method, based on Sparse Distributed Memory (Kanerva\nNetworks), for studying dependency relations between different syntactic\nparameters in the Principles and Parameters model of Syntax. We store data of\nsyntactic parameters of world languages in a Kanerva Network and we check the\nrecoverability of corrupted parameter data from the network. We find that\ndifferent syntactic parameters have different degrees of recoverability. We\nidentify two different effects: an overall underlying relation between the\nprevalence of parameters across languages and their degree of recoverability,\nand a finer effect that makes some parameters more easily recoverable beyond\nwhat their prevalence would indicate. We interpret a higher recoverability for\na syntactic parameter as an indication of the existence of a dependency\nrelation, through which the given parameter can be determined using the\nremaining uncorrupted data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.06549v1",
            "date": "2015-10-22 09:40:54+00:00",
            "title": "Multi-GPU Distributed Parallel Bayesian Differential Topic Modelling",
            "authors": [
                "Aaron Q Li"
            ],
            "abstract": "There is an explosion of data, documents, and other content, and people\nrequire tools to analyze and interpret these, tools to turn the content into\ninformation and knowledge. Topic modeling have been developed to solve these\nproblems. Topic models such as LDA [Blei et. al. 2003] allow salient patterns\nin data to be extracted automatically. When analyzing texts, these patterns are\ncalled topics. Among numerous extensions of LDA, few of them can reliably\nanalyze multiple groups of documents and extract topic similarities. Recently,\nthe introduction of differential topic modeling (SPDP) [Chen et. al. 2012]\nperforms uniformly better than many topic models in a discriminative setting.\n  There is also a need to improve the sampling speed for topic models. While\nsome effort has been made for distributed algorithms, there is no work\ncurrently done using graphical processing units (GPU). Note the GPU framework\nhas already become the most cost-efficient platform for many problems.\n  In this thesis, I propose and implement a scalable multi-GPU distributed\nparallel framework which approximates SPDP. Through experiments, I have shown\nmy algorithms have a gain in speed of about 50 times while being almost as\naccurate, with only one single cheap laptop GPU. Furthermore, I have shown the\nspeed improvement is sublinearly scalable when multiple GPUs are used, while\nfairly maintaining the accuracy. Therefore on a medium-sized GPU cluster, the\nspeed improvement could potentially reach a factor of a thousand.\n  Note SPDP is just a representative of other extensions of LDA. Although my\nalgorithm is implemented to work with SPDP, it is designed to be a general\nenough to work with other topic models. The speed-up on smaller collections\n(i.e., 1000s of documents), means that these more complex LDA extensions could\nnow be done in real-time, thus opening up a new way of using these LDA models\nin industry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.06646v2",
            "date": "2015-10-22 14:39:58+00:00",
            "title": "A 'Gibbs-Newton' Technique for Enhanced Inference of Multivariate Polya Parameters and Topic Models",
            "authors": [
                "Osama Khalifa",
                "David Wolfe Corne",
                "Mike Chantler"
            ],
            "abstract": "Hyper-parameters play a major role in the learning and inference process of\nlatent Dirichlet allocation (LDA). In order to begin the LDA latent variables\nlearning process, these hyper-parameters values need to be pre-determined. We\npropose an extension for LDA that we call 'Latent Dirichlet allocation Gibbs\nNewton' (LDA-GN), which places non-informative priors over these\nhyper-parameters and uses Gibbs sampling to learn appropriate values for them.\nAt the heart of LDA-GN is our proposed 'Gibbs-Newton' algorithm, which is a new\ntechnique for learning the parameters of multivariate Polya distributions. We\nreport Gibbs-Newton performance results compared with two prominent existing\napproaches to the latter task: Minka's fixed-point iteration method and the\nMoments method. We then evaluate LDA-GN in two ways: (i) by comparing it with\nstandard LDA in terms of the ability of the resulting topic models to\ngeneralize to unseen documents; (ii) by comparing it with standard LDA in its\nperformance on a binary classification task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1510.06786v2",
            "date": "2015-10-22 22:53:10+00:00",
            "title": "Freshman or Fresher? Quantifying the Geographic Variation of Internet Language",
            "authors": [
                "Vivek Kulkarni",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "We present a new computational technique to detect and analyze statistically\nsignificant geographic variation in language. Our meta-analysis approach\ncaptures statistical properties of word usage across geographical regions and\nuses statistical methods to identify significant changes specific to regions.\nWhile previous approaches have primarily focused on lexical variation between\nregions, our method identifies words that demonstrate semantic and syntactic\nvariation as well.\n  We extend recently developed techniques for neural language models to learn\nword representations which capture differing semantics across geographical\nregions. In order to quantify this variation and ensure robust detection of\ntrue regional differences, we formulate a null model to determine whether\nobserved changes are statistically significant. Our method is the first such\napproach to explicitly account for random variation due to chance while\ndetecting regional variation in word meaning.\n  To validate our model, we study and analyze two different massive online data\nsets: millions of tweets from Twitter spanning not only four different\ncountries but also fifty states, as well as millions of phrases contained in\nthe Google Book Ngrams. Our analysis reveals interesting facets of language\nchange at multiple scales of geographic resolution -- from neighboring states\nto distant continents.\n  Finally, using our model, we propose a measure of semantic distance between\nlanguages. Our analysis of British and American English over a period of 100\nyears reveals that semantic variation between these dialects is shrinking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.07035v1",
            "date": "2015-10-23 05:26:09+00:00",
            "title": "Fast Latent Variable Models for Inference and Visualization on Mobile Devices",
            "authors": [
                "Joseph W Robinson",
                "Aaron Q Li"
            ],
            "abstract": "In this project we outline Vedalia, a high performance distributed network\nfor performing inference on latent variable models in the context of Amazon\nreview visualization. We introduce a new model, RLDA, which extends Latent\nDirichlet Allocation (LDA) [Blei et al., 2003] for the review space by\nincorporating auxiliary data available in online reviews to improve modeling\nwhile simultaneously remaining compatible with pre-existing fast sampling\ntechniques such as [Yao et al., 2009; Li et al., 2014a] to achieve high\nperformance. The network is designed such that computation is efficiently\noffloaded to the client devices using the Chital system [Robinson & Li, 2015],\nimproving response times and reducing server costs. The resulting system is\nable to rapidly compute a large number of specialized latent variable models\nwhile requiring minimal server resources.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1510.07526v3",
            "date": "2015-10-26 16:03:27+00:00",
            "title": "Empirical Study on Deep Learning Models for Question Answering",
            "authors": [
                "Yang Yu",
                "Wei Zhang",
                "Chung-Wei Hang",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "In this paper we explore deep learning models with memory component or\nattention mechanism for question answering task. We combine and compare three\nmodels, Neural Machine Translation, Neural Turing Machine, and Memory Networks\nfor a simulated QA data set. This paper is the first one that uses Neural\nMachine Translation and Neural Turing Machines for solving QA tasks. Our\nresults suggest that the combination of attention and memory have potential to\nsolve certain QA problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.08985v1",
            "date": "2015-10-30 06:42:03+00:00",
            "title": "Prediction-Adaptation-Correction Recurrent Neural Networks for Low-Resource Language Speech Recognition",
            "authors": [
                "Yu Zhang",
                "Ekapol Chuangsuwanich",
                "James Glass",
                "Dong Yu"
            ],
            "abstract": "In this paper, we investigate the use of prediction-adaptation-correction\nrecurrent neural networks (PAC-RNNs) for low-resource speech recognition. A\nPAC-RNN is comprised of a pair of neural networks in which a {\\it correction}\nnetwork uses auxiliary information given by a {\\it prediction} network to help\nestimate the state probability. The information from the correction network is\nalso used by the prediction network in a recurrent loop. Our model outperforms\nother state-of-the-art neural networks (DNNs, LSTMs) on IARPA-Babel tasks.\nMoreover, transfer learning from a language that is similar to the target\nlanguage can help improve performance further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.09202v1",
            "date": "2015-10-30 19:02:53+00:00",
            "title": "Generating Text with Deep Reinforcement Learning",
            "authors": [
                "Hongyu Guo"
            ],
            "abstract": "We introduce a novel schema for sequence to sequence learning with a Deep\nQ-Network (DQN), which decodes the output sequence iteratively. The aim here is\nto enable the decoder to first tackle easier portions of the sequences, and\nthen turn to cope with difficult parts. Specifically, in each iteration, an\nencoder-decoder Long Short-Term Memory (LSTM) network is employed to, from the\ninput sequence, automatically create features to represent the internal states\nof and formulate a list of potential actions for the DQN. Take rephrasing a\nnatural sentence as an example. This list can contain ranked potential words.\nNext, the DQN learns to make decision on which action (e.g., word) will be\nselected from the list to modify the current decoded sequence. The newly\nmodified output sequence is subsequently used as the input to the DQN for the\nnext decoding iteration. In each iteration, we also bias the reinforcement\nlearning's attention to explore sequence portions which are previously\ndifficult to be decoded. For evaluation, the proposed strategy was trained to\ndecode ten thousands natural sentences. Our experiments indicate that, when\ncompared to a left-to-right greedy beam search LSTM decoder, the proposed\nmethod performed competitively well when decoding sentences from the training\nset, but significantly outperformed the baseline when decoding unseen\nsentences, in terms of BLEU score obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.00277v1",
            "date": "2015-05-01 20:40:00+00:00",
            "title": "Grounded Discovery of Coordinate Term Relationships between Software Entities",
            "authors": [
                "Dana Movshovitz-Attias",
                "William W. Cohen"
            ],
            "abstract": "We present an approach for the detection of coordinate-term relationships\nbetween entities from the software domain, that refer to Java classes. Usually,\nrelations are found by examining corpus statistics associated with text\nentities. In some technical domains, however, we have access to additional\ninformation about the real-world objects named by the entities, suggesting that\ncoupling information about the \"grounded\" entities with corpus statistics might\nlead to improved methods for relation discovery. To this end, we develop a\nsimilarity measure for Java classes using distributional information about how\nthey are used in software, which we combine with corpus statistics on the\ndistribution of contexts in which the classes appear in text. Using our\napproach, cross-validation accuracy on this dataset can be improved\ndramatically, from around 60% to 88%. Human labeling results show that our\nclassifier has an F1 score of 86% over the top 1000 predicted pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.00863v1",
            "date": "2015-05-05 02:06:23+00:00",
            "title": "A Feature-based Classification Technique for Answering Multi-choice World History Questions",
            "authors": [
                "Shuangyong Song",
                "Yao Meng",
                "Zhongguang Zheng",
                "Jun Sun"
            ],
            "abstract": "Our FRDC_QA team participated in the QA-Lab English subtask of the NTCIR-11.\nIn this paper, we describe our system for solving real-world university\nentrance exam questions, which are related to world history. Wikipedia is used\nas the main external resource for our system. Since problems with choosing\nright/wrong sentence from multiple sentence choices account for about\ntwo-thirds of the total, we individually design a classification based model\nfor solving this type of questions. For other types of questions, we also\ndesign some simple methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1505.01121v3",
            "date": "2015-05-05 18:39:29+00:00",
            "title": "Ask Your Neurons: A Neural-based Approach to Answering Questions about Images",
            "authors": [
                "Mateusz Malinowski",
                "Marcus Rohrbach",
                "Mario Fritz"
            ],
            "abstract": "We address a question answering task on real-world images that is set up as a\nVisual Turing Test. By combining latest advances in image representation and\nnatural language processing, we propose Neural-Image-QA, an end-to-end\nformulation to this problem for which all parts are trained jointly. In\ncontrast to previous efforts, we are facing a multi-modal problem where the\nlanguage output (answer) is conditioned on visual and natural language input\n(image and question). Our approach Neural-Image-QA doubles the performance of\nthe previous best approach on this problem. We provide additional insights into\nthe problem by analyzing how much information is contained only in the language\npart for which we provide a new human baseline. To study human consensus, which\nis related to the ambiguities inherent in this challenging task, we propose two\nnovel metrics and collect additional answers which extends the original DAQUAR\ndataset to DAQUAR-Consensus.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1505.01504v2",
            "date": "2015-05-06 20:14:25+00:00",
            "title": "A Fixed-Size Encoding Method for Variable-Length Sequences with its Application to Neural Network Language Models",
            "authors": [
                "Shiliang Zhang",
                "Hui Jiang",
                "Mingbin Xu",
                "Junfeng Hou",
                "Lirong Dai"
            ],
            "abstract": "In this paper, we propose the new fixed-size ordinally-forgetting encoding\n(FOFE) method, which can almost uniquely encode any variable-length sequence of\nwords into a fixed-size representation. FOFE can model the word order in a\nsequence using a simple ordinally-forgetting mechanism according to the\npositions of words. In this work, we have applied FOFE to feedforward neural\nnetwork language models (FNN-LMs). Experimental results have shown that without\nusing any recurrent feedbacks, FOFE based FNN-LMs can significantly outperform\nnot only the standard fixed-input FNN-LMs but also the popular RNN-LMs.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1505.01809v3",
            "date": "2015-05-07 18:36:14+00:00",
            "title": "Language Models for Image Captioning: The Quirks and What Works",
            "authors": [
                "Jacob Devlin",
                "Hao Cheng",
                "Hao Fang",
                "Saurabh Gupta",
                "Li Deng",
                "Xiaodong He",
                "Geoffrey Zweig",
                "Margaret Mitchell"
            ],
            "abstract": "Two recent approaches have achieved state-of-the-art results in image\ncaptioning. The first uses a pipelined process where a set of candidate words\nis generated by a convolutional neural network (CNN) trained on images, and\nthen a maximum entropy (ME) language model is used to arrange these words into\na coherent sentence. The second uses the penultimate activation layer of the\nCNN as input to a recurrent neural network (RNN) that then generates the\ncaption sequence. In this paper, we compare the merits of these different\nlanguage modeling approaches for the first time by using the same\nstate-of-the-art CNN as input. We examine issues in the different approaches,\nincluding linguistic irregularities, caption repetition, and data set overlap.\nBy combining key aspects of the ME and RNN methods, we achieve a new record\nperformance over previously published results on the benchmark COCO dataset.\nHowever, the gains we see in BLEU do not translate to human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.02074v4",
            "date": "2015-05-08 15:59:44+00:00",
            "title": "Exploring Models and Data for Image Question Answering",
            "authors": [
                "Mengye Ren",
                "Ryan Kiros",
                "Richard Zemel"
            ],
            "abstract": "This work aims to address the problem of image-based question-answering (QA)\nwith new models and datasets. In our work, we propose to use neural networks\nand visual semantic embeddings, without intermediate stages such as object\ndetection and image segmentation, to predict answers to simple questions about\nimages. Our model performs 1.8 times better than the only published results on\nan existing image QA dataset. We also present a question generation algorithm\nthat converts image descriptions, which are widely available, into QA form. We\nused this algorithm to produce an order-of-magnitude larger dataset, with more\nevenly distributed answers. A suite of baseline results on this new dataset are\nalso presented.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1505.02251v1",
            "date": "2015-05-09 09:39:04+00:00",
            "title": "Probabilistic Cascading for Large Scale Hierarchical Classification",
            "authors": [
                "Aris Kosmopoulos",
                "Georgios Paliouras",
                "Ion Androutsopoulos"
            ],
            "abstract": "Hierarchies are frequently used for the organization of objects. Given a\nhierarchy of classes, two main approaches are used, to automatically classify\nnew instances: flat classification and cascade classification. Flat\nclassification ignores the hierarchy, while cascade classification greedily\ntraverses the hierarchy from the root to the predicted leaf. In this paper we\npropose a new approach, which extends cascade classification to predict the\nright leaf by estimating the probability of each root-to-leaf path. We provide\nexperimental results which indicate that, using the same classification\nalgorithm, one can achieve better results with our approach, compared to the\ntraditional flat and cascade classifications.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1505.02419v3",
            "date": "2015-05-10 18:47:06+00:00",
            "title": "Improved Relation Extraction with Feature-Rich Compositional Embedding Models",
            "authors": [
                "Matthew R. Gormley",
                "Mo Yu",
                "Mark Dredze"
            ],
            "abstract": "Compositional embedding models build a representation (or embedding) for a\nlinguistic structure based on its component word embeddings. We propose a\nFeature-rich Compositional Embedding Model (FCM) for relation extraction that\nis expressive, generalizes to new domains, and is easy-to-implement. The key\nidea is to combine both (unlexicalized) hand-crafted features with learned word\nembeddings. The model is able to directly tackle the difficulties met by\ntraditional compositional embeddings models, such as handling arbitrary types\nof sentence annotations and utilizing global information for composition. We\ntest the proposed model on two relation extraction tasks, and demonstrate that\nour model outperforms both previous compositional models and traditional\nfeature rich models on the ACE 2005 relation extraction task, and the SemEval\n2010 relation classification task. The combination of our model and a\nlog-linear classifier with hand-crafted features gives state-of-the-art\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.02973v1",
            "date": "2015-05-12 12:05:19+00:00",
            "title": "Comparing methods for Twitter Sentiment Analysis",
            "authors": [
                "Evangelos Psomakelis",
                "Konstantinos Tserpes",
                "Dimosthenis Anagnostopoulos",
                "Theodora Varvarigou"
            ],
            "abstract": "This work extends the set of works which deal with the popular problem of\nsentiment analysis in Twitter. It investigates the most popular document\n(\"tweet\") representation methods which feed sentiment evaluation mechanisms. In\nparticular, we study the bag-of-words, n-grams and n-gram graphs approaches and\nfor each of them we evaluate the performance of a lexicon-based and 7\nlearning-based classification algorithms (namely SVM, Na\\\"ive Bayesian\nNetworks, Logistic Regression, Multilayer Perceptrons, Best-First Trees,\nFunctional Trees and C4.5) as well as their combinations, using a set of 4451\nmanually annotated tweets. The results demonstrate the superiority of\nlearning-based methods and in particular of n-gram graphs approaches for\npredicting the sentiment of tweets. They also show that the combinatory\napproach has impressive effects on n-grams, raising the confidence up to 83.15%\non the 5-Grams, using majority vote and a balanced dataset (equal number of\npositive, negative and neutral tweets for training). In the n-gram graph cases\nthe improvement was small to none, reaching 94.52% on the 4-gram graphs, using\nOrthodromic distance and a threshold of 0.001.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.04630v5",
            "date": "2015-05-18 13:26:02+00:00",
            "title": "Recurrent Neural Network Training with Dark Knowledge Transfer",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Zhiyong Zhang"
            ],
            "abstract": "Recurrent neural networks (RNNs), particularly long short-term memory (LSTM),\nhave gained much attention in automatic speech recognition (ASR). Although some\nsuccessful stories have been reported, training RNNs remains highly\nchallenging, especially with limited training data. Recent research found that\na well-trained model can be used as a teacher to train other child models, by\nusing the predictions generated by the teacher model as supervision. This\nknowledge transfer learning has been employed to train simple neural nets with\na complex one, so that the final performance can reach a level that is\ninfeasible to obtain by regular training. In this paper, we employ the\nknowledge transfer learning approach to train RNNs (precisely LSTM) using a\ndeep neural network (DNN) model as the teacher. This is different from most of\nthe existing research on knowledge transfer learning, since the teacher (DNN)\nis assumed to be weaker than the child (RNN); however, our experiments on an\nASR task showed that it works fairly well: without applying any tricks on the\nlearning scheme, this approach can train RNNs successfully even with limited\ntraining data.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1505.04771v2",
            "date": "2015-05-18 19:35:21+00:00",
            "title": "DopeLearning: A Computational Approach to Rap Lyrics Generation",
            "authors": [
                "Eric Malmi",
                "Pyry Takala",
                "Hannu Toivonen",
                "Tapani Raiko",
                "Aristides Gionis"
            ],
            "abstract": "Writing rap lyrics requires both creativity to construct a meaningful,\ninteresting story and lyrical skills to produce complex rhyme patterns, which\nform the cornerstone of good flow. We present a rap lyrics generation method\nthat captures both of these aspects. First, we develop a prediction model to\nidentify the next line of existing lyrics from a set of candidate next lines.\nThis model is based on two machine-learning techniques: the RankSVM algorithm\nand a deep neural network model with a novel structure. Results show that the\nprediction model can identify the true next line among 299 randomly selected\nlines with an accuracy of 17%, i.e., over 50 times more likely than by random.\nSecond, we employ the prediction model to combine lines from existing songs,\nproducing lyrics with rhyme and a meaning. An evaluation of the produced lyrics\nshows that in terms of quantitative rhyme density, the method outperforms the\nbest human rappers by 21%. The rap lyrics generator has been deployed as an\nonline tool called DeepBeat, and the performance of the tool has been assessed\nby analyzing its usage logs. This analysis shows that machine-learned rankings\ncorrelate with user preferences.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1505.05612v3",
            "date": "2015-05-21 06:09:36+00:00",
            "title": "Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering",
            "authors": [
                "Haoyuan Gao",
                "Junhua Mao",
                "Jie Zhou",
                "Zhiheng Huang",
                "Lei Wang",
                "Wei Xu"
            ],
            "abstract": "In this paper, we present the mQA model, which is able to answer questions\nabout the content of an image. The answer can be a sentence, a phrase or a\nsingle word. Our model contains four components: a Long Short-Term Memory\n(LSTM) to extract the question representation, a Convolutional Neural Network\n(CNN) to extract the visual representation, an LSTM for storing the linguistic\ncontext in an answer, and a fusing component to combine the information from\nthe first three components and generate the answer. We construct a Freestyle\nMultilingual Image Question Answering (FM-IQA) dataset to train and evaluate\nour mQA model. It contains over 150,000 images and 310,000 freestyle Chinese\nquestion-answer pairs and their English translations. The quality of the\ngenerated answers of our mQA model on this dataset is evaluated by human judges\nthrough a Turing Test. Specifically, we mix the answers provided by humans and\nour model. The human judges need to distinguish our model from the human. They\nwill also provide a score (i.e. 0, 1, 2, the larger the better) indicating the\nquality of the answer. We propose strategies to monitor the quality of this\nevaluation process. The experiments show that in 64.7% of cases, the human\njudges cannot distinguish our model from humans. The average score is 1.454\n(1.918 for human). The details of this work, including the FM-IQA dataset, can\nbe found on the project page: http://idl.baidu.com/FM-IQA.html",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1505.05667v1",
            "date": "2015-05-21 10:23:10+00:00",
            "title": "A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network",
            "authors": [
                "Chenxi Zhu",
                "Xipeng Qiu",
                "Xinchi Chen",
                "Xuanjing Huang"
            ],
            "abstract": "In this work, we address the problem to model all the nodes (words or\nphrases) in a dependency tree with the dense representations. We propose a\nrecursive convolutional neural network (RCNN) architecture to capture syntactic\nand compositional-semantic representations of phrases and words in a dependency\ntree. Different with the original recursive neural network, we introduce the\nconvolution and pooling layers, which can model a variety of compositions by\nthe feature maps and choose the most informative compositions by the pooling\nlayers. Based on RCNN, we use a discriminative model to re-rank a $k$-best list\nof candidate dependency parsing trees. The experiments show that RCNN is very\neffective to improve the state-of-the-art dependency parsing on both English\nand Chinese datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06294v1",
            "date": "2015-05-23 09:08:30+00:00",
            "title": "A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics",
            "authors": [
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "The categorical compositional distributional model of Coecke, Sadrzadeh and\nClark provides a linguistically motivated procedure for computing the meaning\nof a sentence as a function of the distributional meaning of the words therein.\nThe theoretical framework allows for reasoning about compositional aspects of\nlanguage and offers structural ways of studying the underlying relationships.\nWhile the model so far has been applied on the level of syntactic structures, a\nsentence can bring extra information conveyed in utterances via intonational\nmeans. In the current paper we extend the framework in order to accommodate\nthis additional information, using Frobenius algebraic structures canonically\ninduced over the basis of finite-dimensional vector spaces. We detail the\ntheory, provide truth-theoretic and distributional semantics for meanings of\nintonationally-marked utterances, and present justifications and extensive\nexamples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.06427v1",
            "date": "2015-05-24 11:22:40+00:00",
            "title": "Deep Speaker Vectors for Semi Text-independent Speaker Verification",
            "authors": [
                "Lantian Li",
                "Dong Wang",
                "Zhiyong Zhang",
                "Thomas Fang Zheng"
            ],
            "abstract": "Recent research shows that deep neural networks (DNNs) can be used to extract\ndeep speaker vectors (d-vectors) that preserve speaker characteristics and can\nbe used in speaker verification. This new method has been tested on\ntext-dependent speaker verification tasks, and improvement was reported when\ncombined with the conventional i-vector method.\n  This paper extends the d-vector approach to semi text-independent speaker\nverification tasks, i.e., the text of the speech is in a limited set of short\nphrases. We explore various settings of the DNN structure used for d-vector\nextraction, and present a phone-dependent training which employs the posterior\nfeatures obtained from an ASR system. The experimental results show that it is\npossible to apply d-vectors on semi text-independent speaker recognition, and\nthe phone-dependent training improves system performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.07712v1",
            "date": "2015-05-28 15:00:27+00:00",
            "title": "A Category Theory of Communication Theory",
            "authors": [
                "Eric Werner"
            ],
            "abstract": "A theory of how agents can come to understand a language is presented. If\nunderstanding a sentence $\\alpha$ is to associate an operator with $\\alpha$\nthat transforms the representational state of the agent as intended by the\nsender, then coming to know a language involves coming to know the operators\nthat correspond to the meaning of any sentence. This involves a higher order\noperator that operates on the possible transformations that operate on the\nrepresentational capacity of the agent. We formalize these constructs using\nconcepts and diagrams analogous to category theory.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1505.07909v4",
            "date": "2015-05-29 02:46:44+00:00",
            "title": "Solving Verbal Comprehension Questions in IQ Test by Knowledge-Powered Word Embedding",
            "authors": [
                "Huazheng Wang",
                "Fei Tian",
                "Bin Gao",
                "Jiang Bian",
                "Tie-Yan Liu"
            ],
            "abstract": "Intelligence Quotient (IQ) Test is a set of standardized questions designed\nto evaluate human intelligence. Verbal comprehension questions appear very\nfrequently in IQ tests, which measure human's verbal ability including the\nunderstanding of the words with multiple senses, the synonyms and antonyms, and\nthe analogies among words. In this work, we explore whether such tests can be\nsolved automatically by artificial intelligence technologies, especially the\ndeep learning technologies that are recently developed and successfully applied\nin a number of fields. However, we found that the task was quite challenging,\nand simply applying existing technologies (e.g., word embedding) could not\nachieve a good performance, mainly due to the multiple senses of words and the\ncomplex relations among words. To tackle these challenges, we propose a novel\nframework consisting of three components. First, we build a classifier to\nrecognize the specific type of a verbal question (e.g., analogy,\nclassification, synonym, or antonym). Second, we obtain distributed\nrepresentations of words and relations by leveraging a novel word embedding\nmethod that considers the multi-sense nature of words and the relational\nknowledge among words (or their senses) contained in dictionaries. Third, for\neach type of questions, we propose a specific solver based on the obtained\ndistributed word representations and relation representations. Experimental\nresults have shown that the proposed framework can not only outperform existing\nmethods for solving verbal comprehension questions but also exceed the average\nperformance of the Amazon Mechanical Turk workers involved in the study. The\nresults indicate that with appropriate uses of the deep learning technologies\nwe might be a further step closer to the human intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1505.08075v1",
            "date": "2015-05-29 14:58:12+00:00",
            "title": "Transition-Based Dependency Parsing with Stack Long Short-Term Memory",
            "authors": [
                "Chris Dyer",
                "Miguel Ballesteros",
                "Wang Ling",
                "Austin Matthews",
                "Noah A. Smith"
            ],
            "abstract": "We propose a technique for learning representations of parser states in\ntransition-based dependency parsers. Our primary innovation is a new control\nstructure for sequence-to-sequence neural networks---the stack LSTM. Like the\nconventional stack data structures used in transition-based parsing, elements\ncan be pushed to or popped from the top of the stack in constant time, but, in\naddition, an LSTM maintains a continuous space embedding of the stack contents.\nThis lets us formulate an efficient parsing model that captures three facets of\na parser's state: (i) unbounded look-ahead into the buffer of incoming words,\n(ii) the complete history of actions taken by the parser, and (iii) the\ncomplete contents of the stack of partially built tree fragments, including\ntheir internal structures. Standard backpropagation techniques are used for\ntraining and yield state-of-the-art parsing performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.00533v1",
            "date": "2015-09-02 00:31:40+00:00",
            "title": "Enhancement and Recognition of Reverberant and Noisy Speech by Extending Its Coherence",
            "authors": [
                "Scott Wisdom",
                "Thomas Powers",
                "Les Atlas",
                "James Pitton"
            ],
            "abstract": "Most speech enhancement algorithms make use of the short-time Fourier\ntransform (STFT), which is a simple and flexible time-frequency decomposition\nthat estimates the short-time spectrum of a signal. However, the duration of\nshort STFT frames are inherently limited by the nonstationarity of speech\nsignals. The main contribution of this paper is a demonstration of speech\nenhancement and automatic speech recognition in the presence of reverberation\nand noise by extending the length of analysis windows. We accomplish this\nextension by performing enhancement in the short-time fan-chirp transform\n(STFChT) domain, an overcomplete time-frequency representation that is coherent\nwith speech signals over longer analysis window durations than the STFT. This\nextended coherence is gained by using a linear model of fundamental frequency\nvariation of voiced speech signals. Our approach centers around using a\nsingle-channel minimum mean-square error log-spectral amplitude (MMSE-LSA)\nestimator proposed by Habets, which scales coefficients in a time-frequency\ndomain to suppress noise and reverberation. In the case of multiple\nmicrophones, we preprocess the data with either a minimum variance\ndistortionless response (MVDR) beamformer, or a delay-and-sum beamformer (DSB).\nWe evaluate our algorithm on both speech enhancement and recognition tasks for\nthe REVERB challenge dataset. Compared to the same processing done in the STFT\ndomain, our approach achieves significant improvement in terms of objective\nenhancement metrics (including PESQ---the ITU-T standard measurement for speech\nquality). In terms of automatic speech recognition (ASR) performance as\nmeasured by word error rate (WER), our experiments indicate that the STFT with\na long window is more effective for ASR.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1509.00838v2",
            "date": "2015-09-02 19:52:56+00:00",
            "title": "What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment",
            "authors": [
                "Hongyuan Mei",
                "Mohit Bansal",
                "Matthew R. Walter"
            ],
            "abstract": "We propose an end-to-end, domain-independent neural encoder-aligner-decoder\nmodel for selective generation, i.e., the joint task of content selection and\nsurface realization. Our model first encodes a full set of over-determined\ndatabase event records via an LSTM-based recurrent neural network, then\nutilizes a novel coarse-to-fine aligner to identify the small subset of salient\nrecords to talk about, and finally employs a decoder to generate free-form\ndescriptions of the aligned, selected records. Our model achieves the best\nselection and generation results reported to-date (with 59% relative\nimprovement in generation) on the benchmark WeatherGov dataset, despite using\nno specialized features or linguistic resources. Using an improved k-nearest\nneighbor beam filter helps further. We also perform a series of ablations and\nvisualizations to elucidate the contributions of our key model components.\nLastly, we evaluate the generalizability of our model on the RoboCup dataset,\nand get results that are competitive with or better than the state-of-the-art,\ndespite being severely data-starved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.01288v1",
            "date": "2015-09-03 22:11:10+00:00",
            "title": "Incremental Active Opinion Learning Over a Stream of Opinionated Documents",
            "authors": [
                "Max Zimmermann",
                "Eirini Ntoutsi",
                "Myra Spiliopoulou"
            ],
            "abstract": "Applications that learn from opinionated documents, like tweets or product\nreviews, face two challenges. First, the opinionated documents constitute an\nevolving stream, where both the author's attitude and the vocabulary itself may\nchange. Second, labels of documents are scarce and labels of words are\nunreliable, because the sentiment of a word depends on the (unknown) context in\nthe author's mind. Most of the research on mining over opinionated streams\nfocuses on the first aspect of the problem, whereas for the second a continuous\nsupply of labels from the stream is assumed. Such an assumption though is\nutopian as the stream is infinite and the labeling cost is prohibitive. To this\nend, we investigate the potential of active stream learning algorithms that ask\nfor labels on demand. Our proposed ACOSTREAM 1 approach works with limited\nlabels: it uses an initial seed of labeled documents, occasionally requests\nadditional labels for documents from the human expert and incrementally adapts\nto the underlying stream while exploiting the available labeled documents. In\nits core, ACOSTREAM consists of a MNB classifier coupled with \"sampling\"\nstrategies for requesting class labels for new unlabeled documents. In the\nexperiments, we evaluate the classifier performance over time by varying: (a)\nthe class distribution of the opinionated stream, while assuming that the set\nof the words in the vocabulary is fixed but their polarities may change with\nthe class distribution; and (b) the number of unknown words arriving at each\nmoment, while the class polarity may also change. Our results show that active\nlearning on a stream of opinionated documents, delivers good performance while\nrequiring a small selection of labels",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1509.01771v2",
            "date": "2015-09-06 06:09:28+00:00",
            "title": "Sampled Weighted Min-Hashing for Large-Scale Topic Mining",
            "authors": [
                "Gibran Fuentes-Pineda",
                "Ivan Vladimir Meza-Ruiz"
            ],
            "abstract": "We present Sampled Weighted Min-Hashing (SWMH), a randomized approach to\nautomatically mine topics from large-scale corpora. SWMH generates multiple\nrandom partitions of the corpus vocabulary based on term co-occurrence and\nagglomerates highly overlapping inter-partition cells to produce the mined\ntopics. While other approaches define a topic as a probabilistic distribution\nover a vocabulary, SWMH topics are ordered subsets of such vocabulary.\nInterestingly, the topics mined by SWMH underlie themes from the corpus at\ndifferent levels of granularity. We extensively evaluate the meaningfulness of\nthe mined topics both qualitatively and quantitatively on the NIPS (1.7 K\ndocuments), 20 Newsgroups (20 K), Reuters (800 K) and Wikipedia (4 M) corpora.\nAdditionally, we compare the quality of SWMH with Online LDA topics for\ndocument representation in classification.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1509.01978v1",
            "date": "2015-09-07 10:39:20+00:00",
            "title": "An Approach to the Analysis of the South Slavic Medieval Labels Using Image Texture",
            "authors": [
                "Darko Brodic",
                "Alessia Amelio",
                "Zoran N. Milivojevic"
            ],
            "abstract": "The paper presents a new script classification method for the discrimination\nof the South Slavic medieval labels. It consists in the textural analysis of\nthe script types. In the first step, each letter is coded by the equivalent\nscript type, which is defined by its typographical features. Obtained coded\ntext is subjected to the run-length statistical analysis and to the adjacent\nlocal binary pattern analysis in order to extract the features. The result\nshows a diversity between the extracted features of the scripts, which makes\nthe feature classification more effective. It is the basis for the\nclassification process of the script identification by using an extension of a\nstate-of-the-art approach for document clustering. The proposed method is\nevaluated on an example of hand-engraved in stone and hand-printed in paper\nlabels in old Cyrillic, angular and round Glagolitic. Experiments demonstrate\nvery positive results, which prove the effectiveness of the proposed method.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1509.02409v1",
            "date": "2015-09-08 15:20:12+00:00",
            "title": "Data-selective Transfer Learning for Multi-Domain Speech Recognition",
            "authors": [
                "Mortaza Doulaty",
                "Oscar Saz",
                "Thomas Hain"
            ],
            "abstract": "Negative transfer in training of acoustic models for automatic speech\nrecognition has been reported in several contexts such as domain change or\nspeaker characteristics. This paper proposes a novel technique to overcome\nnegative transfer by efficient selection of speech data for acoustic model\ntraining. Here data is chosen on relevance for a specific target. A submodular\nfunction based on likelihood ratios is used to determine how acoustically\nsimilar each training utterance is to a target test set. The approach is\nevaluated on a wide-domain data set, covering speech from radio and TV\nbroadcasts, telephone conversations, meetings, lectures and read speech.\nExperiments demonstrate that the proposed technique both finds relevant data\nand limits negative transfer. Results on a 6--hour test set show a relative\nimprovement of 4% with data selection over using all data in PLP based models,\nand 2% with DNN features.",
            "journal ref": "16th Interspeech.Proc. (2015) 2897-2901",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1509.02437v1",
            "date": "2015-09-08 16:36:04+00:00",
            "title": "Improved Twitter Sentiment Prediction through Cluster-then-Predict Model",
            "authors": [
                "Rishabh Soni",
                "K. James Mathai"
            ],
            "abstract": "Over the past decade humans have experienced exponential growth in the use of\nonline resources, in particular social media and microblogging websites such as\nFacebook, Twitter, YouTube and also mobile applications such as WhatsApp, Line,\netc. Many companies have identified these resources as a rich mine of marketing\nknowledge. This knowledge provides valuable feedback which allows them to\nfurther develop the next generation of their product. In this paper, sentiment\nanalysis of a product is performed by extracting tweets about that product and\nclassifying the tweets showing it as positive and negative sentiment. The\nauthors propose a hybrid approach which combines unsupervised learning in the\nform of K-means clustering to cluster the tweets and then performing supervised\nlearning methods such as Decision Trees and Support Vector Machines for\nclassification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1509.03295v3",
            "date": "2015-09-09 11:27:03+00:00",
            "title": "Liberating language research from dogmas of the 20th century",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "A commentary on the article \"Large-scale evidence of dependency length\nminimization in 37 languages\" by Futrell, Mahowald & Gibson (PNAS 2015 112 (33)\n10336-10341).",
            "journal ref": "Liberating language research from dogmas of the 20th century.\n  Glottometrics 33, 33-34 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.04219v1",
            "date": "2015-09-14 17:39:37+00:00",
            "title": "Twitter Sentiment Analysis",
            "authors": [
                "Afroze Ibrahim Baqapuri"
            ],
            "abstract": "This project addresses the problem of sentiment analysis in twitter; that is\nclassifying tweets according to the sentiment expressed in them: positive,\nnegative or neutral. Twitter is an online micro-blogging and social-networking\nplatform which allows users to write short status updates of maximum length 140\ncharacters. It is a rapidly expanding service with over 200 million registered\nusers - out of which 100 million are active users and half of them log on\ntwitter on a daily basis - generating nearly 250 million tweets per day. Due to\nthis large amount of usage we hope to achieve a reflection of public sentiment\nby analysing the sentiments expressed in the tweets. Analysing the public\nsentiment is important for many applications such as firms trying to find out\nthe response of their products in the market, predicting political elections\nand predicting socioeconomic phenomena like stock exchange. The aim of this\nproject is to develop a functional classifier for accurate and automatic\nsentiment classification of an unknown tweet stream.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05281v2",
            "date": "2015-09-17 15:08:36+00:00",
            "title": "Network analysis of named entity co-occurrences in written texts",
            "authors": [
                "Diego R. Amancio"
            ],
            "abstract": "The use of methods borrowed from statistics and physics to analyze written\ntexts has allowed the discovery of unprecedent patterns of human behavior and\ncognition by establishing links between models features and language structure.\nWhile current models have been useful to unveil patterns via analysis of\nsyntactical and semantical networks, only a few works have probed the relevance\nof investigating the structure arising from the relationship between relevant\nentities such as characters, locations and organizations. In this study, we\nrepresent entities appearing in the same context as a co-occurrence network,\nwhere links are established according to a null model based on random, shuffled\ntexts. Computational simulations performed in novels revealed that the proposed\nmodel displays interesting topological features, such as the small world\nfeature, characterized by high values of clustering coefficient. The\neffectiveness of our model was verified in a practical pattern recognition task\nin real networks. When compared with traditional word adjacency networks, our\nmodel displayed optimized results in identifying unknown references in texts.\nBecause the proposed representation plays a complementary role in\ncharacterizing unstructured documents via topological analysis of named\nentities, we believe that it could be useful to improve the characterization of\nwritten texts (and related systems), specially if combined with traditional\napproaches based on statistical and deeper paradigms.",
            "journal ref": "EPL 114 (2016) 58005",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.05808v1",
            "date": "2015-09-18 21:50:38+00:00",
            "title": "Word, graph and manifold embedding from Markov processes",
            "authors": [
                "Tatsunori B. Hashimoto",
                "David Alvarez-Melis",
                "Tommi S. Jaakkola"
            ],
            "abstract": "Continuous vector representations of words and objects appear to carry\nsurprisingly rich semantic content. In this paper, we advance both the\nconceptual and theoretical understanding of word embeddings in three ways.\nFirst, we ground embeddings in semantic spaces studied in\ncognitive-psychometric literature and introduce new evaluation tasks. Second,\nin contrast to prior work, we take metric recovery as the key object of study,\nunify existing algorithms as consistent metric recovery methods based on\nco-occurrence counts from simple Markov random walks, and propose a new\nrecovery algorithm. Third, we generalize metric recovery to graphs and\nmanifolds, relating co-occurence counts on random walks in graphs and random\nprocesses on manifolds to the underlying metric to be recovered, thereby\nreconciling manifold estimation and embedding algorithms. We compare embedding\nalgorithms across a range of tasks, from nonlinear dimensionality reduction to\nthree semantic language tasks, including analogies, sequence completion, and\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.06594v1",
            "date": "2015-09-22 13:33:34+00:00",
            "title": "A Compositional Explanation of the Pet Fish Phenomenon",
            "authors": [
                "Bob Coecke",
                "Martha Lewis"
            ],
            "abstract": "The `pet fish' phenomenon is often cited as a paradigm example of the\n`non-compositionality' of human concept use. We show here how this phenomenon\nis naturally accommodated within a compositional distributional model of\nmeaning. This model describes the meaning of a composite concept by accounting\nfor interaction between its constituents via their grammatical roles. We give\ntwo illustrative examples to show how the qualitative phenomena are exhibited.\nWe go on to apply the model to experimental data, and finally discuss\nextensions of the formalism.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1509.06664v4",
            "date": "2015-09-22 16:08:24+00:00",
            "title": "Reasoning about Entailment with Neural Attention",
            "authors": [
                "Tim Rockt\u00e4schel",
                "Edward Grefenstette",
                "Karl Moritz Hermann",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Phil Blunsom"
            ],
            "abstract": "While most approaches to automatically recognizing entailment relations have\nused classifiers employing hand engineered features derived from complex\nnatural language processing pipelines, in practice their performance has been\nonly slightly better than bag-of-word pair classifiers using only lexical\nsimilarity. The only attempt so far to build an end-to-end differentiable\nneural network for entailment failed to outperform such a simple similarity\nclassifier. In this paper, we propose a neural model that reads two sentences\nto determine entailment using long short-term memory units. We extend this\nmodel with a word-by-word neural attention mechanism that encourages reasoning\nover entailments of pairs of words and phrases. Furthermore, we present a\nqualitative analysis of attention weights produced by this model, demonstrating\nsuch reasoning capabilities. On a large entailment dataset this model\noutperforms the previous best neural model and a classifier with engineered\nfeatures by a substantial margin. It is the first generic end-to-end\ndifferentiable system that achieves state-of-the-art accuracy on a textual\nentailment dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07179v1",
            "date": "2015-09-23 23:22:38+00:00",
            "title": "IllinoisSL: A JAVA Library for Structured Prediction",
            "authors": [
                "Kai-Wei Chang",
                "Shyam Upadhyay",
                "Ming-Wei Chang",
                "Vivek Srikumar",
                "Dan Roth"
            ],
            "abstract": "IllinoisSL is a Java library for learning structured prediction models. It\nsupports structured Support Vector Machines and structured Perceptron. The\nlibrary consists of a core learning module and several applications, which can\nbe executed from command-lines. Documentation is provided to guide users. In\nComparison to other structured learning libraries, IllinoisSL is efficient,\ngeneral, and easy to use.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1509.07845v1",
            "date": "2015-09-25 19:27:54+00:00",
            "title": "Selecting Relevant Web Trained Concepts for Automated Event Retrieval",
            "authors": [
                "Bharat Singh",
                "Xintong Han",
                "Zhe Wu",
                "Vlad I. Morariu",
                "Larry S. Davis"
            ],
            "abstract": "Complex event retrieval is a challenging research problem, especially when no\ntraining videos are available. An alternative to collecting training videos is\nto train a large semantic concept bank a priori. Given a text description of an\nevent, event retrieval is performed by selecting concepts linguistically\nrelated to the event description and fusing the concept responses on unseen\nvideos. However, defining an exhaustive concept lexicon and pre-training it\nrequires vast computational resources. Therefore, recent approaches automate\nconcept discovery and training by leveraging large amounts of weakly annotated\nweb data. Compact visually salient concepts are automatically obtained by the\nuse of concept pairs or, more generally, n-grams. However, not all visually\nsalient n-grams are necessarily useful for an event query--some combinations of\nconcepts may be visually compact but irrelevant--and this drastically affects\nperformance. We propose an event retrieval algorithm that constructs pairs of\nautomatically discovered concepts and then prunes those concepts that are\nunlikely to be helpful for retrieval. Pruning depends both on the query and on\nthe specific video instance being evaluated. Our approach also addresses\ncalibration and domain adaptation issues that arise when applying concept\ndetectors to unseen videos. We demonstrate large improvements over other vision\nbased systems on the TRECVID MED 13 dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1509.08639v1",
            "date": "2015-09-29 08:44:14+00:00",
            "title": "Tuned and GPU-accelerated parallel data mining from comparable corpora",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "The multilingual nature of the world makes translation a crucial requirement\ntoday. Parallel dictionaries constructed by humans are a widely-available\nresource, but they are limited and do not provide enough coverage for good\nquality translation purposes, due to out-of-vocabulary words and neologisms.\nThis motivates the use of statistical translation systems, which are\nunfortunately dependent on the quantity and quality of training data. Such has\na very limited availability especially for some languages and very narrow text\ndomains. Is this research we present our improvements to Yalign mining\nmethodology by reimplementing the comparison algorithm, introducing a tuning\nscripts and by improving performance using GPU computing acceleration. The\nexperiments are conducted on various text domains and bi-data is extracted from\nthe Wikipedia dumps.",
            "journal ref": "Lecture Notes in Artificial Intelligence, p. 32-40, ISBN:\n  978-3-319-24032-9, Springer, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08644v1",
            "date": "2015-09-29 08:54:48+00:00",
            "title": "Neural-based machine translation for medical text domain. Based on European Medicines Agency leaflet texts",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "The quality of machine translation is rapidly evolving. Today one can find\nseveral machine translation systems on the web that provide reasonable\ntranslations, although the systems are not perfect. In some specific domains,\nthe quality may decrease. A recently proposed approach to this domain is neural\nmachine translation. It aims at building a jointly-tuned single neural network\nthat maximizes translation performance, a very different approach from\ntraditional statistical machine translation. Recently proposed neural machine\ntranslation models often belong to the encoder-decoder family in which a source\nsentence is encoded into a fixed length vector that is, in turn, decoded to\ngenerate a translation. The present research examines the effects of different\ntraining methods on a Polish-English Machine Translation system used for\nmedical data. The European Medicines Agency parallel text corpus was used as\nthe basis for training of neural and statistical network-based translation\nsystems. The main machine translation evaluation metrics have also been used in\nanalysis of the systems. A comparison and implementation of a real-time medical\ntranslator is the main focus of our experiments.",
            "journal ref": "Procedia Computer Science, 2015, 64: 2-9",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08881v1",
            "date": "2015-09-29 18:35:49+00:00",
            "title": "Building Subject-aligned Comparable Corpora and Mining it for Truly Parallel Sentence Pairs",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "Parallel sentences are a relatively scarce but extremely useful resource for\nmany applications including cross-lingual retrieval and statistical machine\ntranslation. This research explores our methodology for mining such data from\npreviously obtained comparable corpora. The task is highly practical since\nnon-parallel multilingual data exist in far greater quantities than parallel\ncorpora, but parallel sentences are a much more useful resource. Here we\npropose a web crawling method for building subject-aligned comparable corpora\nfrom Wikipedia articles. We also introduce a method for extracting truly\nparallel sentences that are filtered out from noisy or just comparable sentence\npairs. We describe our implementation of a specialized tool for this task as\nwell as training and adaption of a machine translation system that supplies our\nfilter with additional information about the similarity of comparable sentence\npairs.",
            "journal ref": "Procedia Technology, 18, Elsevier, p.126-132, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08909v1",
            "date": "2015-09-29 19:57:24+00:00",
            "title": "Polish -English Statistical Machine Translation of Medical Texts",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Krzysztof Marasek"
            ],
            "abstract": "This new research explores the effects of various training methods on a\nPolish to English Statistical Machine Translation system for medical texts.\nVarious elements of the EMEA parallel text corpora from the OPUS project were\nused as the basis for training of phrase tables and language models and for\ndevelopment, tuning and testing of the translation system. The BLEU, NIST,\nMETEOR, RIBES and TER metrics have been used to evaluate the effects of various\nsystem and data preparations on translation results. Our experiments included\nsystems that used POS tagging, factored phrase models, hierarchical models,\nsyntactic taggers, and many different alignment methods. We also conducted a\ndeep analysis of Polish data as preparatory work for automatic data correction\nsuch as true casing and punctuation normalization phase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.08973v1",
            "date": "2015-09-29 23:16:48+00:00",
            "title": "Symbol Emergence in Robotics: A Survey",
            "authors": [
                "Tadahiro Taniguchi",
                "Takayuki Nagai",
                "Tomoaki Nakamura",
                "Naoto Iwahashi",
                "Tetsuya Ogata",
                "Hideki Asoh"
            ],
            "abstract": "Humans can learn the use of language through physical interaction with their\nenvironment and semiotic communication with other people. It is very important\nto obtain a computational understanding of how humans can form a symbol system\nand obtain semiotic skills through their autonomous mental development.\nRecently, many studies have been conducted on the construction of robotic\nsystems and machine-learning methods that can learn the use of language through\nembodied multimodal interaction with their environment and other systems.\nUnderstanding human social interactions and developing a robot that can\nsmoothly communicate with human users in the long term, requires an\nunderstanding of the dynamics of symbol systems and is crucially important. The\nembodied cognition and social interaction of participants gradually change a\nsymbol system in a constructive manner. In this paper, we introduce a field of\nresearch called symbol emergence in robotics (SER). SER is a constructive\napproach towards an emergent symbol system. The emergent symbol system is\nsocially self-organized through both semiotic communications and physical\ninteractions with autonomous cognitive developmental agents, i.e., humans and\ndevelopmental robots. Specifically, we describe some state-of-art research\ntopics concerning SER, e.g., multimodal categorization, word discovery, and a\ndouble articulation analysis, that enable a robot to obtain words and their\nembodied meanings from raw sensory--motor information, including visual\ninformation, haptic information, auditory information, and acoustic speech\nsignals, in a totally unsupervised manner. Finally, we suggest future\ndirections of research in SER.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.00657v4",
            "date": "2015-04-02 19:34:01+00:00",
            "title": "Eliciting Disease Data from Wikipedia Articles",
            "authors": [
                "Geoffrey Fairchild",
                "Lalindra De Silva",
                "Sara Y. Del Valle",
                "Alberto M. Segre"
            ],
            "abstract": "Traditional disease surveillance systems suffer from several disadvantages,\nincluding reporting lags and antiquated technology, that have caused a movement\ntowards internet-based disease surveillance systems. Internet systems are\nparticularly attractive for disease outbreaks because they can provide data in\nnear real-time and can be verified by individuals around the globe. However,\nmost existing systems have focused on disease monitoring and do not provide a\ndata repository for policy makers or researchers. In order to fill this gap, we\nanalyzed Wikipedia article content.\n  We demonstrate how a named-entity recognizer can be trained to tag case\ncounts, death counts, and hospitalization counts in the article narrative that\nachieves an F1 score of 0.753. We also show, using the 2014 West African Ebola\nvirus disease epidemic article as a case study, that there are detailed time\nseries data that are consistently updated that closely align with ground truth\ndata.\n  We argue that Wikipedia can be used to create the first community-driven\nopen-source emerging disease detection, monitoring, and repository system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1504.00854v1",
            "date": "2015-04-03 14:46:29+00:00",
            "title": "Evaluation Evaluation a Monte Carlo study",
            "authors": [
                "David M. W. Powers"
            ],
            "abstract": "Over the last decade there has been increasing concern about the biases\nembodied in traditional evaluation methods for Natural Language\nProcessing/Learning, particularly methods borrowed from Information Retrieval.\nWithout knowledge of the Bias and Prevalence of the contingency being tested,\nor equivalently the expectation due to chance, the simple conditional\nprobabilities Recall, Precision and Accuracy are not meaningful as evaluation\nmeasures, either individually or in combinations such as F-factor. The\nexistence of bias in NLP measures leads to the 'improvement' of systems by\nincreasing their bias, such as the practice of improving tagging and parsing\nscores by using most common value (e.g. water is always a Noun) rather than the\nattempting to discover the correct one. The measures Cohen Kappa and Powers\nInformedness are discussed as unbiased alternative to Recall and related to the\npsychologically significant measure DeltaP. In this paper we will analyze both\nbiased and unbiased measures theoretically, characterizing the precise\nrelationship between all these measures as well as evaluating the evaluation\nmeasures themselves empirically using a Monte Carlo simulation.",
            "journal ref": "ECAI 2008, pp.843-844",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.01106v5",
            "date": "2015-04-05 10:18:32+00:00",
            "title": "Discriminative Neural Sentence Modeling by Tree-Based Convolution",
            "authors": [
                "Lili Mou",
                "Hao Peng",
                "Ge Li",
                "Yan Xu",
                "Lu Zhang",
                "Zhi Jin"
            ],
            "abstract": "This paper proposes a tree-based convolutional neural network (TBCNN) for\ndiscriminative sentence modeling. Our models leverage either constituency trees\nor dependency trees of sentences. The tree-based convolution process extracts\nsentences' structural features, and these features are aggregated by max\npooling. Such architecture allows short propagation paths between the output\nlayer and underlying feature detectors, which enables effective structural\nfeature learning and extraction. We evaluate our models on two tasks: sentiment\nanalysis and question classification. In both experiments, TBCNN outperforms\nprevious state-of-the-art results, including existing neural networks and\ndedicated feature/rule engineering. We also make efforts to visualize the\ntree-based convolution process, shedding light on how our models work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01255v3",
            "date": "2015-04-06 10:42:07+00:00",
            "title": "Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding",
            "authors": [
                "Rie Johnson",
                "Tong Zhang"
            ],
            "abstract": "This paper presents a new semi-supervised framework with convolutional neural\nnetworks (CNNs) for text categorization. Unlike the previous approaches that\nrely on word embeddings, our method learns embeddings of small text regions\nfrom unlabeled data for integration into a supervised CNN. The proposed scheme\nfor embedding learning is based on the idea of two-view semi-supervised\nlearning, which is intended to be useful for the task of interest even though\nthe training is done on unlabeled data. Our models achieve better results than\nprevious approaches on sentiment classification and topic classification tasks.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1504.01383v1",
            "date": "2015-04-06 20:00:28+00:00",
            "title": "QUOTUS: The Structure of Political Media Coverage as Revealed by Quoting Patterns",
            "authors": [
                "Vlad Niculae",
                "Caroline Suen",
                "Justine Zhang",
                "Cristian Danescu-Niculescu-Mizil",
                "Jure Leskovec"
            ],
            "abstract": "Given the extremely large pool of events and stories available, media outlets\nneed to focus on a subset of issues and aspects to convey to their audience.\nOutlets are often accused of exhibiting a systematic bias in this selection\nprocess, with different outlets portraying different versions of reality.\nHowever, in the absence of objective measures and empirical evidence, the\ndirection and extent of systematicity remains widely disputed.\n  In this paper we propose a framework based on quoting patterns for\nquantifying and characterizing the degree to which media outlets exhibit\nsystematic bias. We apply this framework to a massive dataset of news articles\nspanning the six years of Obama's presidency and all of his speeches, and\nreveal that a systematic pattern does indeed emerge from the outlet's quoting\nbehavior. Moreover, we show that this pattern can be successfully exploited in\nan unsupervised prediction setting, to determine which new quotes an outlet\nwill select to broadcast. By encoding bias patterns in a low-rank space we\nprovide an analysis of the structure of political media coverage. This reveals\na latent media bias space that aligns surprisingly well with political ideology\nand outlet type. A linguistic analysis exposes striking differences across\nthese latent dimensions, showing how the different types of media outlets\nportray different realities even when reporting on the same events. For\nexample, outlets mapped to the mainstream conservative side of the latent space\nfocus on quotes that portray a presidential persona disproportionately\ncharacterized by negativity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.01482v1",
            "date": "2015-04-07 06:12:14+00:00",
            "title": "Deep Recurrent Neural Networks for Acoustic Modelling",
            "authors": [
                "William Chan",
                "Ian Lane"
            ],
            "abstract": "We present a novel deep Recurrent Neural Network (RNN) model for acoustic\nmodelling in Automatic Speech Recognition (ASR). We term our contribution as a\nTC-DNN-BLSTM-DNN model, the model combines a Deep Neural Network (DNN) with\nTime Convolution (TC), followed by a Bidirectional Long Short-Term Memory\n(BLSTM), and a final DNN. The first DNN acts as a feature processor to our\nmodel, the BLSTM then generates a context from the sequence acoustic signal,\nand the final DNN takes the context and models the posterior probabilities of\nthe acoustic states. We achieve a 3.47 WER on the Wall Street Journal (WSJ)\neval92 task or more than 8% relative improvement over the baseline DNN models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1504.01483v1",
            "date": "2015-04-07 06:15:44+00:00",
            "title": "Transferring Knowledge from a RNN to a DNN",
            "authors": [
                "William Chan",
                "Nan Rosemary Ke",
                "Ian Lane"
            ],
            "abstract": "Deep Neural Network (DNN) acoustic models have yielded many state-of-the-art\nresults in Automatic Speech Recognition (ASR) tasks. More recently, Recurrent\nNeural Network (RNN) models have been shown to outperform DNNs counterparts.\nHowever, state-of-the-art DNN and RNN models tend to be impractical to deploy\non embedded systems with limited computational capacity. Traditionally, the\napproach for embedded platforms is to either train a small DNN directly, or to\ntrain a small DNN that learns the output distribution of a large DNN. In this\npaper, we utilize a state-of-the-art RNN to transfer knowledge to small DNN. We\nuse the RNN model to generate soft alignments and minimize the Kullback-Leibler\ndivergence against the small DNN. The small DNN trained on the soft RNN\nalignments achieved a 3.93 WER on the Wall Street Journal (WSJ) eval92 task\ncompared to a baseline 4.54 WER or more than 13% relative improvement.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1504.02148v1",
            "date": "2015-04-08 22:38:35+00:00",
            "title": "Mining and discovering biographical information in Difangzhi with a language-model-based approach",
            "authors": [
                "Peter K. Bol",
                "Chao-Lin Liu",
                "Hongsu Wang"
            ],
            "abstract": "We present results of expanding the contents of the China Biographical\nDatabase by text mining historical local gazetteers, difangzhi. The goal of the\ndatabase is to see how people are connected together, through kinship, social\nconnections, and the places and offices in which they served. The gazetteers\nare the single most important collection of names and offices covering the Song\nthrough Qing periods. Although we begin with local officials we shall\neventually include lists of local examination candidates, people from the\nlocality who served in government, and notable local figures with biographies.\nThe more data we collect the more connections emerge. The value of doing\nsystematic text mining work is that we can identify relevant connections that\nare either directly informative or can become useful without deep historical\nresearch. Academia Sinica is developing a name database for officials in the\ncentral governments of the Ming and Qing dynasties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.02150v1",
            "date": "2015-04-08 22:47:59+00:00",
            "title": "Exploring Lexical, Syntactic, and Semantic Features for Chinese Textual Entailment in NTCIR RITE Evaluation Tasks",
            "authors": [
                "Wei-Jie Huang",
                "Chao-Lin Liu"
            ],
            "abstract": "We computed linguistic information at the lexical, syntactic, and semantic\nlevels for Recognizing Inference in Text (RITE) tasks for both traditional and\nsimplified Chinese in NTCIR-9 and NTCIR-10. Techniques for syntactic parsing,\nnamed-entity recognition, and near synonym recognition were employed, and\nfeatures like counts of common words, statement lengths, negation words, and\nantonyms were considered to judge the entailment relationships of two\nstatements, while we explored both heuristics-based functions and\nmachine-learning approaches. The reported systems showed robustness by\nsimultaneously achieving second positions in the binary-classification subtasks\nfor both simplified and traditional Chinese in NTCIR-10 RITE-2. We conducted\nmore experiments with the test data of NTCIR-9 RITE, with good results. We also\nextended our work to search for better configurations of our classifiers and\ninvestigated contributions of individual features. This extended work showed\ninteresting results and should encourage further discussion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.03425v1",
            "date": "2015-04-14 05:49:26+00:00",
            "title": "Automated Analysis and Prediction of Job Interview Performance",
            "authors": [
                "Iftekhar Naim",
                "M. Iftekhar Tanveer",
                "Daniel Gildea",
                "Mohammed",
                "Hoque"
            ],
            "abstract": "We present a computational framework for automatically quantifying verbal and\nnonverbal behaviors in the context of job interviews. The proposed framework is\ntrained by analyzing the videos of 138 interview sessions with 69\ninternship-seeking undergraduates at the Massachusetts Institute of Technology\n(MIT). Our automated analysis includes facial expressions (e.g., smiles, head\ngestures, facial tracking points), language (e.g., word counts, topic\nmodeling), and prosodic information (e.g., pitch, intonation, and pauses) of\nthe interviewees. The ground truth labels are derived by taking a weighted\naverage over the ratings of 9 independent judges. Our framework can\nautomatically predict the ratings for interview traits such as excitement,\nfriendliness, and engagement with correlation coefficients of 0.75 or higher,\nand can quantify the relative importance of prosody, language, and facial\nexpressions. By analyzing the relative feature weights learned by the\nregression models, our framework recommends to speak more fluently, use less\nfiller words, speak as \"we\" (vs. \"I\"), use more unique words, and smile more.\nWe also find that the students who were rated highly while answering the first\ninterview question were also rated highly overall (i.e., first impression\nmatters). Finally, our MIT Interview dataset will be made available to other\nresearchers to further validate and expand our findings.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1504.04317v1",
            "date": "2015-04-16 17:26:24+00:00",
            "title": "Towards a relation extraction framework for cyber-security concepts",
            "authors": [
                "Corinne L. Jones",
                "Robert A. Bridges",
                "Kelly Huffer",
                "John Goodall"
            ],
            "abstract": "In order to assist security analysts in obtaining information pertaining to\ntheir network, such as novel vulnerabilities, exploits, or patches, information\nretrieval methods tailored to the security domain are needed. As labeled text\ndata is scarce and expensive, we follow developments in semi-supervised Natural\nLanguage Processing and implement a bootstrapping algorithm for extracting\nsecurity entities and their relationships from text. The algorithm requires\nlittle input data, specifically, a few relations or patterns (heuristics for\nidentifying relations), and incorporates an active learning component which\nqueries the user on the most important decisions to prevent drifting from the\ndesired relations. Preliminary testing on a small corpus shows promising\nresults, obtaining precision of .82.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1504.04802v1",
            "date": "2015-04-19 07:03:17+00:00",
            "title": "Gradual Classical Logic for Attributed Objects - Extended in Re-Presentation",
            "authors": [
                "Ryuta Arisaka"
            ],
            "abstract": "Our understanding about things is conceptual. By stating that we reason about\nobjects, it is in fact not the objects but concepts referring to them that we\nmanipulate. Now, so long just as we acknowledge infinitely extending notions\nsuch as space, time, size, colour, etc, - in short, any reasonable quality -\ninto which an object is subjected, it becomes infeasible to affirm atomicity in\nthe concept referring to the object. However, formal/symbolic logics typically\npresume atomic entities upon which other expressions are built. Can we reflect\nour intuition about the concept onto formal/symbolic logics at all? I assure\nthat we can, but the usual perspective about the atomicity needs inspected. In\nthis work, I present gradual logic which materialises the observation that we\ncannot tell apart whether a so-regarded atomic entity is atomic or is just\natomic enough not to be considered non-atomic. The motivation is to capture\ncertain phenomena that naturally occur around concepts with attributes,\nincluding presupposition and contraries. I present logical particulars of the\nlogic, which is then mapped onto formal semantics. Two linguistically\ninteresting semantics will be considered. Decidability is shown.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.05070v2",
            "date": "2015-04-20 14:26:41+00:00",
            "title": "Self-Adaptive Hierarchical Sentence Model",
            "authors": [
                "Han Zhao",
                "Zhengdong Lu",
                "Pascal Poupart"
            ],
            "abstract": "The ability to accurately model a sentence at varying stages (e.g.,\nword-phrase-sentence) plays a central role in natural language processing. As\nan effort towards this goal we propose a self-adaptive hierarchical sentence\nmodel (AdaSent). AdaSent effectively forms a hierarchy of representations from\nwords to phrases and then to sentences through recursive gated local\ncomposition of adjacent segments. We design a competitive mechanism (through\ngating networks) to allow the representations of the same sentence to be\nengaged in a particular learning task (e.g., classification), therefore\neffectively mitigating the gradient vanishing problem persistent in other\nrecursive models. Both qualitative and quantitative analysis shows that AdaSent\ncan automatically form and select the representations suitable for the task at\nhand during training, yielding superior classification performance over\ncompetitor models on 5 benchmark data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06063v5",
            "date": "2015-04-23 07:10:13+00:00",
            "title": "Multimodal Convolutional Neural Networks for Matching Image and Sentence",
            "authors": [
                "Lin Ma",
                "Zhengdong Lu",
                "Lifeng Shang",
                "Hang Li"
            ],
            "abstract": "In this paper, we propose multimodal convolutional neural networks (m-CNNs)\nfor matching image and sentence. Our m-CNN provides an end-to-end framework\nwith convolutional architectures to exploit image representation, word\ncomposition, and the matching relations between the two modalities. More\nspecifically, it consists of one image CNN encoding the image content, and one\nmatching CNN learning the joint representation of image and sentence. The\nmatching CNN composes words to different semantic fragments and learns the\ninter-modal relations between image and the composed fragments at different\nlevels, thus fully exploit the matching relations between image and sentence.\nExperimental results on benchmark databases of bidirectional image and sentence\nretrieval demonstrate that the proposed m-CNNs can effectively capture the\ninformation necessary for image and sentence matching. Specifically, our\nproposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and\nMicrosoft COCO databases achieve the state-of-the-art performances.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1504.06329v1",
            "date": "2015-04-23 20:07:01+00:00",
            "title": "Analysis of Stopping Active Learning based on Stabilizing Predictions",
            "authors": [
                "Michael Bloodgood",
                "John Grothendieck"
            ],
            "abstract": "Within the natural language processing (NLP) community, active learning has\nbeen widely investigated and applied in order to alleviate the annotation\nbottleneck faced by developers of new NLP systems and technologies. This paper\npresents the first theoretical analysis of stopping active learning based on\nstabilizing predictions (SP). The analysis has revealed three elements that are\ncentral to the success of the SP method: (1) bounds on Cohen's Kappa agreement\nbetween successively trained models impose bounds on differences in F-measure\nperformance of the models; (2) since the stop set does not have to be labeled,\nit can be made large in practice, helping to guarantee that the results\ntransfer to previously unseen streams of examples at test/application time; and\n(3) good (low variance) sample estimates of Kappa between successive models can\nbe obtained. Proofs of relationships between the level of Kappa agreement and\nthe difference in performance between consecutive models are presented.\nSpecifically, if the Kappa agreement between two models exceeds a threshold T\n(where $T>0$), then the difference in F-measure performance between those\nmodels is bounded above by $\\frac{4(1-T)}{T}$ in all cases. If precision of the\npositive conjunction of the models is assumed to be $p$, then the bound can be\ntightened to $\\frac{4(1-T)}{(p+1)T}$.",
            "journal ref": "In Proceedings of the Seventeenth Conference on Computational\n  Natural Language Learning, pages 10-19, Sofia, Bulgaria, August 2013.\n  Association for Computational Linguistics",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1504.06580v2",
            "date": "2015-04-24 17:50:33+00:00",
            "title": "Classifying Relations by Ranking with Convolutional Neural Networks",
            "authors": [
                "Cicero Nogueira dos Santos",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "Relation classification is an important semantic processing task for which\nstate-ofthe-art systems still rely on costly handcrafted features. In this work\nwe tackle the relation classification task using a convolutional neural network\nthat performs classification by ranking (CR-CNN). We propose a new pairwise\nranking loss function that makes it easy to reduce the impact of artificial\nclasses. We perform experiments using the the SemEval-2010 Task 8 dataset,\nwhich is designed for the task of classifying the relationship between two\nnominals marked in a sentence. Using CRCNN, we outperform the state-of-the-art\nfor this dataset and achieve a F1 of 84.1 without using any costly handcrafted\nfeatures. Additionally, our experimental results show that: (1) our approach is\nmore effective than CNN followed by a softmax classifier; (2) omitting the\nrepresentation of the artificial class Other improves both precision and\nrecall; and (3) using only word embeddings as input features is enough to\nachieve state-of-the-art results if we consider only the text between the two\ntarget nominals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.06692v2",
            "date": "2015-04-25 06:45:35+00:00",
            "title": "Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images",
            "authors": [
                "Junhua Mao",
                "Wei Xu",
                "Yi Yang",
                "Jiang Wang",
                "Zhiheng Huang",
                "Alan Yuille"
            ],
            "abstract": "In this paper, we address the task of learning novel visual concepts, and\ntheir interactions with other concepts, from a few images with sentence\ndescriptions. Using linguistic context and visual features, our method is able\nto efficiently hypothesize the semantic meaning of new words and add them to\nits word dictionary so that they can be used to describe images which contain\nthese novel concepts. Our method has an image captioning module based on m-RNN\nwith several improvements. In particular, we propose a transposed weight\nsharing scheme, which not only improves performance on image captioning, but\nalso makes the model more suitable for the novel concept learning task. We\npropose methods to prevent overfitting the new concepts. In addition, three\nnovel concept datasets are constructed for this new task. In the experiments,\nwe show that our method effectively learns novel visual concepts from a few\nexamples without disturbing the previously learned concepts. The project page\nis http://www.stat.ucla.edu/~junhua.mao/projects/child_learning.html",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1504.06936v1",
            "date": "2015-04-27 05:56:13+00:00",
            "title": "Concept Extraction to Identify Adverse Drug Reactions in Medical Forums: A Comparison of Algorithms",
            "authors": [
                "Alejandro Metke-Jimenez",
                "Sarvnaz Karimi"
            ],
            "abstract": "Social media is becoming an increasingly important source of information to\ncomplement traditional pharmacovigilance methods. In order to identify signals\nof potential adverse drug reactions, it is necessary to first identify medical\nconcepts in the social media text. Most of the existing studies use\ndictionary-based methods which are not evaluated independently from the overall\nsignal detection task.\n  We compare different approaches to automatically identify and normalise\nmedical concepts in consumer reviews in medical forums. Specifically, we\nimplement several dictionary-based methods popular in the relevant literature,\nas well as a method we suggest based on a state-of-the-art machine learning\nmethod for entity recognition. MetaMap, a popular biomedical concept extraction\ntool, is used as a baseline. Our evaluations were performed in a controlled\nsetting on a common corpus which is a collection of medical forum posts\nannotated with concepts and linked to controlled vocabularies such as MedDRA\nand SNOMED CT.\n  To our knowledge, our study is the first to systematically examine the effect\nof popular concept extraction methods in the area of signal detection for\nadverse reactions. We show that the choice of algorithm or controlled\nvocabulary has a significant impact on concept extraction, which will impact\nthe overall signal detection process. We also show that our proposed machine\nlearning approach significantly outperforms all the other methods in\nidentification of both adverse reactions and drugs, even when trained with a\nrelatively small set of annotated text.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1504.07225v3",
            "date": "2015-04-27 19:51:34+00:00",
            "title": "Correlational Neural Networks",
            "authors": [
                "Sarath Chandar",
                "Mitesh M. Khapra",
                "Hugo Larochelle",
                "Balaraman Ravindran"
            ],
            "abstract": "Common Representation Learning (CRL), wherein different descriptions (or\nviews) of the data are embedded in a common subspace, is receiving a lot of\nattention recently. Two popular paradigms here are Canonical Correlation\nAnalysis (CCA) based approaches and Autoencoder (AE) based approaches. CCA\nbased approaches learn a joint representation by maximizing correlation of the\nviews when projected to the common subspace. AE based methods learn a common\nrepresentation by minimizing the error of reconstructing the two views. Each of\nthese approaches has its own advantages and disadvantages. For example, while\nCCA based approaches outperform AE based approaches for the task of transfer\nlearning, they are not as scalable as the latter. In this work we propose an AE\nbased approach called Correlational Neural Network (CorrNet), that explicitly\nmaximizes correlation among the views when projected to the common subspace.\nThrough a series of experiments, we demonstrate that the proposed CorrNet is\nbetter than the above mentioned approaches with respect to its ability to learn\ncorrelated common representations. Further, we employ CorrNet for several cross\nlanguage tasks and show that the representations learned using CorrNet perform\nbetter than the ones learned using other state of the art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07295v3",
            "date": "2015-04-27 22:32:40+00:00",
            "title": "Document Classification by Inversion of Distributed Language Representations",
            "authors": [
                "Matt Taddy"
            ],
            "abstract": "There have been many recent advances in the structure and measurement of\ndistributed language models: those that map from words to a vector-space that\nis rich in information about word choice and composition. This vector-space is\nthe distributed language representation. The goal of this note is to point out\nthat any distributed representation can be turned into a classifier through\ninversion via Bayes rule. The approach is simple and modular, in that it will\nwork with any language representation whose training can be formulated as\noptimizing a probability model. In our application to 2 million sentences from\nYelp reviews, we also find that it performs as well as or better than complex\npurpose-built algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.07395v1",
            "date": "2015-04-28 09:43:40+00:00",
            "title": "Lexical Translation Model Using a Deep Neural Network Architecture",
            "authors": [
                "Thanh-Le Ha",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "In this paper we combine the advantages of a model using global source\nsentence contexts, the Discriminative Word Lexicon, and neural networks. By\nusing deep neural networks instead of the linear maximum entropy model in the\nDiscriminative Word Lexicon models, we are able to leverage dependencies\nbetween different source words due to the non-linearity. Furthermore, the\nmodels for different target words can share parameters and therefore data\nsparsity problems are effectively reduced.\n  By using this approach in a state-of-the-art translation system, we can\nimprove the performance by up to 0.5 BLEU points for three different language\npairs on the TED translation task.",
            "journal ref": "Proceedings of the 11th International Workshop on Spoken Language\n  Translation (IWSLT 2014), page 223-229, Lake Tahoe - US, December 4th and\n  5th, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00195v1",
            "date": "2015-05-31 05:10:03+00:00",
            "title": "Recurrent Neural Networks with External Memory for Language Understanding",
            "authors": [
                "Baolin Peng",
                "Kaisheng Yao"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) have become increasingly popular for the\ntask of language understanding. In this task, a semantic tagger is deployed to\nassociate a semantic label to each word in an input sequence. The success of\nRNN may be attributed to its ability to memorize long-term dependence that\nrelates the current-time semantic label prediction to the observations many\ntime instances away. However, the memory capacity of simple RNNs is limited\nbecause of the gradient vanishing and exploding problem. We propose to use an\nexternal memory to improve memorization capability of RNNs. We conducted\nexperiments on the ATIS dataset, and observed that the proposed model was able\nto achieve the state-of-the-art results. We compare our proposed model with\nalternative models and report analysis results that may provide insights for\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00333v2",
            "date": "2015-06-01 03:09:49+00:00",
            "title": "Learning to Answer Questions From Image Using Convolutional Neural Network",
            "authors": [
                "Lin Ma",
                "Zhengdong Lu",
                "Hang Li"
            ],
            "abstract": "In this paper, we propose to employ the convolutional neural network (CNN)\nfor the image question answering (QA). Our proposed CNN provides an end-to-end\nframework with convolutional architectures for learning not only the image and\nquestion representations, but also their inter-modal interactions to produce\nthe answer. More specifically, our model consists of three CNNs: one image CNN\nto encode the image content, one sentence CNN to compose the words of the\nquestion, and one multimodal convolution layer to learn their joint\nrepresentation for the classification in the space of candidate answer words.\nWe demonstrate the efficacy of our proposed model on the DAQUAR and COCO-QA\ndatasets, which are two benchmark datasets for the image QA, with the\nperformances significantly outperforming the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.00468v2",
            "date": "2015-06-01 12:20:21+00:00",
            "title": "Classifying Tweet Level Judgements of Rumours in Social Media",
            "authors": [
                "Michal Lukasik",
                "Trevor Cohn",
                "Kalina Bontcheva"
            ],
            "abstract": "Social media is a rich source of rumours and corresponding community\nreactions. Rumours reflect different characteristics, some shared and some\nindividual. We formulate the problem of classifying tweet level judgements of\nrumours as a supervised learning task. Both supervised and unsupervised domain\nadaptation are considered, in which tweets from a rumour are classified on the\nbasis of other annotated rumours. We demonstrate how multi-task learning helps\nachieve good results on rumours from the 2011 England riots.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1506.00572v2",
            "date": "2015-06-01 17:06:00+00:00",
            "title": "How much is said in a microblog? A multilingual inquiry based on Weibo and Twitter",
            "authors": [
                "Han-Teng Liao",
                "King-wa Fu",
                "Scott A. Hale"
            ],
            "abstract": "This paper presents a multilingual study on, per single post of microblog\ntext, (a) how much can be said, (b) how much is written in terms of characters\nand bytes, and (c) how much is said in terms of information content in posts by\ndifferent organizations in different languages. Focusing on three different\nlanguages (English, Chinese, and Japanese), this research analyses Weibo and\nTwitter accounts of major embassies and news agencies. We first establish our\ncriterion for quantifying \"how much can be said\" in a digital text based on the\nopenly available Universal Declaration of Human Rights and the translated\nsubtitles from TED talks. These parallel corpora allow us to determine the\nnumber of characters and bits needed to represent the same content in different\nlanguages and character encodings. We then derive the amount of information\nthat is actually contained in microblog posts authored by selected accounts on\nWeibo and Twitter. Our results confirm that languages with larger character\nsets such as Chinese and Japanese contain more information per character than\nEnglish, but the actual information content contained within a microblog text\nvaries depending on both the type of organization and the language of the post.\nWe conclude with a discussion on the design implications of microblog text\nlimits for different languages.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1506.00578v1",
            "date": "2015-06-01 17:23:38+00:00",
            "title": "On Quantum Generalizations of Information-Theoretic Measures and their Contribution to Distributional Semantics",
            "authors": [
                "William Blacoe"
            ],
            "abstract": "Information-theoretic measures such as relative entropy and correlation are\nextremely useful when modeling or analyzing the interaction of probabilistic\nsystems. We survey the quantum generalization of 5 such measures and point out\nsome of their commonalities and interpretations. In particular we find the\napplication of information theory to distributional semantics useful. By\nmodeling the distributional meaning of words as density operators rather than\nvectors, more of their semantic structure may be exploited. Furthermore,\nproperties of and interactions between words such as ambiguity, similarity and\nentailment can be simulated more richly and intuitively when using methods from\nquantum information theory.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1506.00765v1",
            "date": "2015-06-02 06:31:57+00:00",
            "title": "Video (GIF) Sentiment Analysis using Large-Scale Mid-Level Ontology",
            "authors": [
                "Zheng Cai",
                "Donglin Cao",
                "Rongrong Ji"
            ],
            "abstract": "With faster connection speed, Internet users are now making social network a\nhuge reservoir of texts, images and video clips (GIF). Sentiment analysis for\nsuch online platform can be used to predict political elections, evaluates\neconomic indicators and so on. However, GIF sentiment analysis is quite\nchallenging, not only because it hinges on spatio-temporal visual\ncontentabstraction, but also for the relationship between such abstraction and\nfinal sentiment remains unknown.In this paper, we dedicated to find out such\nrelationship.We proposed a SentiPairSequence basedspatiotemporal visual\nsentiment ontology, which forms the midlevel representations for GIFsentiment.\nThe establishment process of SentiPair contains two steps. First, we construct\nthe Synset Forest to define the semantic tree structure of visual sentiment\nlabel elements. Then, through theSynset Forest, we organically select and\ncombine sentiment label elements to form a mid-level visual sentiment\nrepresentation. Our experiments indicate that SentiPair outperforms other\ncompeting mid-level attributes. Using SentiPair, our analysis frameworkcan\nachieve satisfying prediction accuracy (72.6%). We also opened ourdataset\n(GSO-2015) to the research community. GSO-2015 contains more than 6,000\nmanually annotated GIFs out of more than 40,000 candidates. Each is labeled\nwith both sentiment and SentiPair Sequence.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1506.00999v1",
            "date": "2015-06-02 19:34:19+00:00",
            "title": "Combining Two And Three-Way Embeddings Models for Link Prediction in Knowledge Bases",
            "authors": [
                "Alberto Garcia-Duran",
                "Antoine Bordes",
                "Nicolas Usunier",
                "Yves Grandvalet"
            ],
            "abstract": "This paper tackles the problem of endogenous link prediction for Knowledge\nBase completion. Knowledge Bases can be represented as directed graphs whose\nnodes correspond to entities and edges to relationships. Previous attempts\neither consist of powerful systems with high capacity to model complex\nconnectivity patterns, which unfortunately usually end up overfitting on rare\nrelationships, or in approaches that trade capacity for simplicity in order to\nfairly model all relationships, frequent or not. In this paper, we propose\nTatec a happy medium obtained by complementing a high-capacity model with a\nsimpler one, both pre-trained separately and then combined. We present several\nvariants of this model with different kinds of regularization and combination\nstrategies and show that this approach outperforms existing methods on\ndifferent types of relationships by achieving state-of-the-art results on four\nbenchmarks of the literature.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1506.01094v2",
            "date": "2015-06-03 00:38:25+00:00",
            "title": "Traversing Knowledge Graphs in Vector Space",
            "authors": [
                "Kelvin Guu",
                "John Miller",
                "Percy Liang"
            ],
            "abstract": "Path queries on a knowledge graph can be used to answer compositional\nquestions such as \"What languages are spoken by people living in Lisbon?\".\nHowever, knowledge graphs often have missing facts (edges) which disrupts path\nqueries. Recent models for knowledge base completion impute missing facts by\nembedding knowledge graphs in vector spaces. We show that these models can be\nrecursively applied to answer path queries, but that they suffer from cascading\nerrors. This motivates a new \"compositional\" training objective, which\ndramatically improves all models' ability to answer path queries, in some cases\nmore than doubling accuracy. On a standard knowledge base completion task, we\nalso demonstrate that compositional training acts as a novel form of structural\nregularization, reliably improving performance across all base models (reducing\nerrors by up to 43%) and achieving new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.01273v3",
            "date": "2015-06-03 15:07:14+00:00",
            "title": "Summarization of Films and Documentaries Based on Subtitles and Scripts",
            "authors": [
                "Marta Apar\u00edcio",
                "Paulo Figueiredo",
                "Francisco Raposo",
                "David Martins de Matos",
                "Ricardo Ribeiro",
                "Lu\u00eds Marujo"
            ],
            "abstract": "We assess the performance of generic text summarization algorithms applied to\nfilms and documentaries, using the well-known behavior of summarization of news\narticles as reference. We use three datasets: (i) news articles, (ii) film\nscripts and subtitles, and (iii) documentary subtitles. Standard ROUGE metrics\nare used for comparing generated summaries against news abstracts, plot\nsummaries, and synopses. We show that the best performing algorithms are LSA,\nfor news articles and documentaries, and LexRank and Support Sets, for films.\nDespite the different nature of films and documentaries, their relative\nbehavior is in accordance with that obtained for news articles.",
            "journal ref": "Pattern Recognition Letters, Volume 73, 1 April 2016, Pages 7-12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02078v2",
            "date": "2015-06-05 22:33:04+00:00",
            "title": "Visualizing and Understanding Recurrent Networks",
            "authors": [
                "Andrej Karpathy",
                "Justin Johnson",
                "Li Fei-Fei"
            ],
            "abstract": "Recurrent Neural Networks (RNNs), and specifically a variant with Long\nShort-Term Memory (LSTM), are enjoying renewed interest as a result of\nsuccessful applications in a wide range of machine learning problems that\ninvolve sequential data. However, while LSTMs provide exceptional results in\npractice, the source of their performance and their limitations remain rather\npoorly understood. Using character-level language models as an interpretable\ntestbed, we aim to bridge this gap by providing an analysis of their\nrepresentations, predictions and error types. In particular, our experiments\nreveal the existence of interpretable cells that keep track of long-range\ndependencies such as line lengths, quotes and brackets. Moreover, our\ncomparative analysis with finite horizon n-gram models traces the source of the\nLSTM improvements to long-range structural dependencies. Finally, we provide\nanalysis of the remaining errors and suggests areas for further study.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1506.02327v1",
            "date": "2015-06-07 23:52:54+00:00",
            "title": "A Multi-layered Acoustic Tokenizing Deep Neural Network (MAT-DNN) for Unsupervised Discovery of Linguistic Units and Generation of High Quality Features",
            "authors": [
                "Cheng-Tao Chung",
                "Cheng-Yu Tsai",
                "Hsiang-Hung Lu",
                "Yuan-ming Liou",
                "Yen-Chen Wu",
                "Yen-Ju Lu",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "This paper summarizes the work done by the authors for the Zero Resource\nSpeech Challenge organized in the technical program of Interspeech 2015. The\ngoal of the challenge is to discover linguistic units directly from unlabeled\nspeech data. The Multi-layered Acoustic Tokenizer (MAT) proposed in this work\nautomatically discovers multiple sets of acoustic tokens from the given corpus.\nEach acoustic token set is specified by a set of hyperparameters that describe\nthe model configuration. These sets of acoustic tokens carry different\ncharacteristics of the given corpus and the language behind thus can be\nmutually reinforced. The multiple sets of token labels are then used as the\ntargets of a Multi-target DNN (MDNN) trained on low-level acoustic features.\nBottleneck features extracted from the MDNN are used as feedback for the MAT\nand the MDNN itself. We call this iterative system the Multi-layered Acoustic\nTokenizing Deep Neural Network (MAT-DNN) which generates high quality features\nfor track 1 of the challenge and acoustic tokens for track 2 of the challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.02516v3",
            "date": "2015-06-08 14:23:30+00:00",
            "title": "Learning to Transduce with Unbounded Memory",
            "authors": [
                "Edward Grefenstette",
                "Karl Moritz Hermann",
                "Mustafa Suleyman",
                "Phil Blunsom"
            ],
            "abstract": "Recently, strong results have been demonstrated by Deep Recurrent Neural\nNetworks on natural language transduction problems. In this paper we explore\nthe representational power of these models using synthetic grammars designed to\nexhibit phenomena similar to those found in real transduction problems such as\nmachine translation. These experiments lead us to propose new memory-based\nrecurrent networks that implement continuously differentiable analogues of\ntraditional data structures such as Stacks, Queues, and DeQues. We show that\nthese architectures exhibit superior generalisation performance to Deep RNNs\nand are often able to learn the underlying generating algorithms in our\ntransduction experiments.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1506.02761v4",
            "date": "2015-06-09 03:08:06+00:00",
            "title": "WordRank: Learning Word Embeddings via Robust Ranking",
            "authors": [
                "Shihao Ji",
                "Hyokun Yun",
                "Pinar Yanardag",
                "Shin Matsushima",
                "S. V. N. Vishwanathan"
            ],
            "abstract": "Embedding words in a vector space has gained a lot of attention in recent\nyears. While state-of-the-art methods provide efficient computation of word\nsimilarities via a low-dimensional matrix embedding, their motivation is often\nleft unclear. In this paper, we argue that word embedding can be naturally\nviewed as a ranking problem due to the ranking nature of the evaluation\nmetrics. Then, based on this insight, we propose a novel framework WordRank\nthat efficiently estimates word representations via robust ranking, in which\nthe attention mechanism and robustness to noise are readily achieved via the\nDCG-like ranking losses. The performance of WordRank is measured in word\nsimilarity and word analogy benchmarks, and the results are compared to the\nstate-of-the-art word embedding techniques. Our algorithm is very competitive\nto the state-of-the- arts on large corpora, while outperforms them by a\nsignificant margin when the training set is limited (i.e., sparse and noisy).\nWith 17 million tokens, WordRank performs almost as well as existing methods\nusing 7.2 billion tokens on a popular word similarity benchmark. Our multi-node\ndistributed implementation of WordRank is publicly available for general usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03099v3",
            "date": "2015-06-09 20:33:47+00:00",
            "title": "Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks",
            "authors": [
                "Samy Bengio",
                "Oriol Vinyals",
                "Navdeep Jaitly",
                "Noam Shazeer"
            ],
            "abstract": "Recurrent Neural Networks can be trained to produce sequences of tokens given\nsome input, as exemplified by recent results in machine translation and image\ncaptioning. The current approach to training them consists of maximizing the\nlikelihood of each token in the sequence given the current (recurrent) state\nand the previous token. At inference, the unknown previous token is then\nreplaced by a token generated by the model itself. This discrepancy between\ntraining and inference can yield errors that can accumulate quickly along the\ngenerated sequence. We propose a curriculum learning strategy to gently change\nthe training process from a fully guided scheme using the true previous token,\ntowards a less guided scheme which mostly uses the generated token instead.\nExperiments on several sequence prediction tasks show that this approach yields\nsignificant improvements. Moreover, it was used successfully in our winning\nentry to the MSCOCO image captioning challenge, 2015.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1506.03340v3",
            "date": "2015-06-10 14:54:39+00:00",
            "title": "Teaching Machines to Read and Comprehend",
            "authors": [
                "Karl Moritz Hermann",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Edward Grefenstette",
                "Lasse Espeholt",
                "Will Kay",
                "Mustafa Suleyman",
                "Phil Blunsom"
            ],
            "abstract": "Teaching machines to read natural language documents remains an elusive\nchallenge. Machine reading systems can be tested on their ability to answer\nquestions posed on the contents of documents that they have seen, but until now\nlarge scale training and test datasets have been missing for this type of\nevaluation. In this work we define a new methodology that resolves this\nbottleneck and provides large scale supervised reading comprehension data. This\nallows us to develop a class of attention based deep neural networks that learn\nto read real documents and answer complex questions with minimal prior\nknowledge of language structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.03775v1",
            "date": "2015-06-11 18:53:56+00:00",
            "title": "Entity-Specific Sentiment Classification of Yahoo News Comments",
            "authors": [
                "Prakhar Biyani",
                "Cornelia Caragea",
                "Narayan Bhamidipati"
            ],
            "abstract": "Sentiment classification is widely used for product reviews and in online\nsocial media such as forums, Twitter, and blogs. However, the problem of\nclassifying the sentiment of user comments on news sites has not been addressed\nyet. News sites cover a wide range of domains including politics, sports,\ntechnology, and entertainment, in contrast to other online social sites such as\nforums and review sites, which are specific to a particular domain. A user\nassociated with a news site is likely to post comments on diverse topics (e.g.,\npolitics, smartphones, and sports) or diverse entities (e.g., Obama, iPhone, or\nGoogle). Classifying the sentiment of users tied to various entities may help\nobtain a holistic view of their personality, which could be useful in\napplications such as online advertising, content personalization, and political\ncampaign planning. In this paper, we formulate the problem of entity-specific\nsentiment classification of comments posted on news articles in Yahoo News and\npropose novel features that are specific to news comments. Experimental results\nshow that our models outperform state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04147v2",
            "date": "2015-06-12 20:00:29+00:00",
            "title": "On the accuracy of self-normalized log-linear models",
            "authors": [
                "Jacob Andreas",
                "Maxim Rabinovich",
                "Dan Klein",
                "Michael I. Jordan"
            ],
            "abstract": "Calculation of the log-normalizer is a major computational obstacle in\napplications of log-linear models with large output spaces. The problem of fast\nnormalizer computation has therefore attracted significant attention in the\ntheoretical and applied machine learning literature. In this paper, we analyze\na recently proposed technique known as \"self-normalization\", which introduces a\nregularization term in training to penalize log normalizers for deviating from\nzero. This makes it possible to use unnormalized model scores as approximate\nprobabilities. Empirical evidence suggests that self-normalization is extremely\neffective, but a theoretical understanding of why it should work, and how\ngenerally it can be applied, is largely lacking. We prove generalization bounds\non the estimated variance of normalizers and upper bounds on the loss in\naccuracy due to self-normalization, describe classes of input distributions\nthat self-normalize easily, and construct explicit examples of high-variance\ninput distributions. Our theoretical results make predictions about the\ndifficulty of fitting self-normalized models to several classes of\ndistributions, and we conclude with empirical validation of these predictions.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1506.04891v2",
            "date": "2015-06-16 09:41:55+00:00",
            "title": "Author Identification using Multi-headed Recurrent Neural Networks",
            "authors": [
                "Douglas Bagnall"
            ],
            "abstract": "Recurrent neural networks (RNNs) are very good at modelling the flow of text,\nbut typically need to be trained on a far larger corpus than is available for\nthe PAN 2015 Author Identification task. This paper describes a novel approach\nwhere the output layer of a character-level RNN language model is split into\nseveral independent predictive sub-models, each representing an author, while\nthe recurrent layer is shared by all. This allows the recurrent layer to model\nthe language as a whole without over-fitting, while the outputs select aspects\nof the underlying model that reflect their author's style. The method proves\ncompetitive, ranking first in two of the four languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05012v1",
            "date": "2015-06-16 16:04:08+00:00",
            "title": "Emotion Analysis of Songs Based on Lyrical and Audio Features",
            "authors": [
                "Adit Jamdar",
                "Jessica Abraham",
                "Karishma Khanna",
                "Rahul Dubey"
            ],
            "abstract": "In this paper, a method is proposed to detect the emotion of a song based on\nits lyrical and audio features. Lyrical features are generated by segmentation\nof lyrics during the process of data extraction. ANEW and WordNet knowledge is\nthen incorporated to compute Valence and Arousal values. In addition to this,\nlinguistic association rules are applied to ensure that the issue of ambiguity\nis properly addressed. Audio features are used to supplement the lyrical ones\nand include attributes like energy, tempo, and danceability. These features are\nextracted from The Echo Nest, a widely used music intelligence platform.\nConstruction of training and test sets is done on the basis of social tags\nextracted from the last.fm website. The classification is done by applying\nfeature weighting and stepwise threshold reduction on the k-Nearest Neighbors\nalgorithm to provide fuzziness in the classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05402v1",
            "date": "2015-06-17 18:03:25+00:00",
            "title": "Editorial for the First Workshop on Mining Scientific Papers: Computational Linguistics and Bibliometrics",
            "authors": [
                "Iana Atanassova",
                "Marc Bertin",
                "Philipp Mayr"
            ],
            "abstract": "The workshop \"Mining Scientific Papers: Computational Linguistics and\nBibliometrics\" (CLBib 2015), co-located with the 15th International Society of\nScientometrics and Informetrics Conference (ISSI 2015), brought together\nresearchers in Bibliometrics and Computational Linguistics in order to study\nthe ways Bibliometrics can benefit from large-scale text analytics and sense\nmining of scientific papers, thus exploring the interdisciplinarity of\nBibliometrics and Natural Language Processing (NLP). The goals of the workshop\nwere to answer questions like: How can we enhance author network analysis and\nBibliometrics using data obtained by text analytics? What insights can NLP\nprovide on the structure of scientific writing, on citation networks, and on\nin-text citation analysis? This workshop is the first step to foster the\nreflection on the interdisciplinarity and the benefits that the two disciplines\nBibliometrics and Natural Language Processing can drive from it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.05514v1",
            "date": "2015-06-17 23:03:43+00:00",
            "title": "Learning Contextualized Semantics from Co-occurring Terms via a Siamese Architecture",
            "authors": [
                "Ubai Sandouk",
                "Ke Chen"
            ],
            "abstract": "One of the biggest challenges in Multimedia information retrieval and\nunderstanding is to bridge the semantic gap by properly modeling concept\nsemantics in context. The presence of out of vocabulary (OOV) concepts\nexacerbates this difficulty. To address the semantic gap issues, we formulate a\nproblem on learning contextualized semantics from descriptive terms and propose\na novel Siamese architecture to model the contextualized semantics from\ndescriptive terms. By means of pattern aggregation and probabilistic topic\nmodels, our Siamese architecture captures contextualized semantics from the\nco-occurring descriptive terms via unsupervised learning, which leads to a\nconcept embedding space of the terms in context. Furthermore, the co-occurring\nOOV concepts can be easily represented in the learnt concept embedding space.\nThe main properties of the concept embedding space are demonstrated via\nvisualization. Using various settings in semantic priming, we have carried out\na thorough evaluation by comparing our approach to a number of state-of-the-art\nmethods on six annotation corpora in different domains, i.e., MagTag5K, CAL500\nand Million Song Dataset in the music domain as well as Corel5K, LabelMe and\nSUNDatabase in the image domain. Experimental results on semantic priming\nsuggest that our approach outperforms those state-of-the-art methods\nconsiderably in various aspects.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1506.05865v4",
            "date": "2015-06-19 02:40:42+00:00",
            "title": "LCSTS: A Large Scale Chinese Short Text Summarization Dataset",
            "authors": [
                "Baotian Hu",
                "Qingcai Chen",
                "Fangze Zhu"
            ],
            "abstract": "Automatic text summarization is widely regarded as the highly difficult\nproblem, partially because of the lack of large text summarization data set.\nDue to the great challenge of constructing the large scale summaries for full\ntext, in this paper, we introduce a large corpus of Chinese short text\nsummarization dataset constructed from the Chinese microblogging website Sina\nWeibo, which is released to the public\n{http://icrc.hitsz.edu.cn/Article/show/139.html}. This corpus consists of over\n2 million real Chinese short texts with short summaries given by the author of\neach text. We also manually tagged the relevance of 10,666 short summaries with\ntheir corresponding short texts. Based on the corpus, we introduce recurrent\nneural network for the summary generation and achieve promising results, which\nnot only shows the usefulness of the proposed corpus for short text\nsummarization research, but also provides a baseline for further research on\nthis topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06418v1",
            "date": "2015-06-21 22:04:39+00:00",
            "title": "Extreme Extraction: Only One Hour per Relation",
            "authors": [
                "Raphael Hoffmann",
                "Luke Zettlemoyer",
                "Daniel S. Weld"
            ],
            "abstract": "Information Extraction (IE) aims to automatically generate a large knowledge\nbase from natural language text, but progress remains slow. Supervised learning\nrequires copious human annotation, while unsupervised and weakly supervised\napproaches do not deliver competitive accuracy. As a result, most fielded\napplications of IE, as well as the leading TAC-KBP systems, rely on significant\namounts of manual engineering. Even \"Extreme\" methods, such as those reported\nin Freedman et al. 2011, require about 10 hours of expert labor per relation.\n  This paper shows how to reduce that effort by an order of magnitude. We\npresent a novel system, InstaRead, that streamlines authoring with an ensemble\nof methods: 1) encoding extraction rules in an expressive and compositional\nrepresentation, 2) guiding the user to promising rules based on corpus\nstatistics and mined resources, and 3) introducing a new interactive\ndevelopment cycle that provides immediate feedback --- even on large datasets.\nExperiments show that experts can create quality extractors in under an hour\nand even NLP novices can author good extractors. These extractors equal or\noutperform ones obtained by comparably supervised and state-of-the-art\ndistantly supervised approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06442v4",
            "date": "2015-06-22 02:12:54+00:00",
            "title": "A Deep Memory-based Architecture for Sequence-to-Sequence Learning",
            "authors": [
                "Fandong Meng",
                "Zhengdong Lu",
                "Zhaopeng Tu",
                "Hang Li",
                "Qun Liu"
            ],
            "abstract": "We propose DEEPMEMORY, a novel deep architecture for sequence-to-sequence\nlearning, which performs the task through a series of nonlinear transformations\nfrom the representation of the input sequence (e.g., a Chinese sentence) to the\nfinal output sequence (e.g., translation to English). Inspired by the recently\nproposed Neural Turing Machine (Graves et al., 2014), we store the intermediate\nrepresentations in stacked layers of memories, and use read-write operations on\nthe memories to realize the nonlinear transformations between the\nrepresentations. The types of transformations are designed in advance but the\nparameters are learned from data. Through layer-by-layer transformations,\nDEEPMEMORY can model complicated relations between sequences necessary for\napplications such as machine translation between distant languages. The\narchitecture can be trained with normal back-propagation on sequenceto-sequence\ndata, and the learning can be easily scaled up to a large corpus. DEEPMEMORY is\nbroad enough to subsume the state-of-the-art neural translation model in\n(Bahdanau et al., 2015) as its special case, while significantly improving upon\nthe model with its deeper architecture. Remarkably, DEEPMEMORY, being purely\nneural network-based, can achieve performance comparable to the traditional\nphrase-based machine translation system Moses with a small vocabulary and a\nmodest parameter size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06490v1",
            "date": "2015-06-22 07:26:51+00:00",
            "title": "Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering",
            "authors": [
                "Xiaoqiang Zhou",
                "Baotian Hu",
                "Qingcai Chen",
                "Buzhou Tang",
                "Xiaolong Wang"
            ],
            "abstract": "In this paper, the answer selection problem in community question answering\n(CQA) is regarded as an answer sequence labeling task, and a novel approach is\nproposed based on the recurrent architecture for this problem. Our approach\napplies convolution neural networks (CNNs) to learning the joint representation\nof question-answer pair firstly, and then uses the joint representation as\ninput of the long short-term memory (LSTM) to learn the answer sequence of a\nquestion for labeling the matching quality of each answer. Experiments\nconducted on the SemEval 2015 CQA dataset shows the effectiveness of our\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06646v2",
            "date": "2015-06-22 15:21:57+00:00",
            "title": "Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals",
            "authors": [
                "Tadahiro Taniguchi",
                "Ryo Nakashima",
                "Shogo Nagasaka"
            ],
            "abstract": "Human infants can discover words directly from unsegmented speech signals\nwithout any explicitly labeled data. In this paper, we develop a novel machine\nlearning method called nonparametric Bayesian double articulation analyzer\n(NPB-DAA) that can directly acquire language and acoustic models from observed\ncontinuous speech signals. For this purpose, we propose an integrative\ngenerative model that combines a language model and an acoustic model into a\nsingle generative model called the \"hierarchical Dirichlet process hidden\nlanguage model\" (HDP-HLM). The HDP-HLM is obtained by extending the\nhierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM) proposed by\nJohnson et al. An inference procedure for the HDP-HLM is derived using the\nblocked Gibbs sampler originally proposed for the HDP-HSMM. This procedure\nenables the simultaneous and direct inference of language and acoustic models\nfrom continuous speech signals. Based on the HDP-HLM and its inference\nprocedure, we developed a novel double articulation analyzer. By assuming\nHDP-HLM as a generative model of observed time series data, and by inferring\nlatent variables of the model, the method can analyze latent double\narticulation structure, i.e., hierarchically organized latent words and\nphonemes, of the data in an unsupervised manner. The novel unsupervised double\narticulation analyzer is called NPB-DAA.\n  The NPB-DAA can automatically estimate double articulation structure embedded\nin speech signals. We also carried out two evaluation experiments using\nsynthetic data and actual human continuous speech signals representing Japanese\nvowel sequences. In the word acquisition and phoneme categorization tasks, the\nNPB-DAA outperformed a conventional double articulation analyzer (DAA) and\nbaseline automatic speech recognition system whose acoustic model was trained\nin a supervised manner.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1506.06714v1",
            "date": "2015-06-22 18:29:03+00:00",
            "title": "A Neural Network Approach to Context-Sensitive Generation of Conversational Responses",
            "authors": [
                "Alessandro Sordoni",
                "Michel Galley",
                "Michael Auli",
                "Chris Brockett",
                "Yangfeng Ji",
                "Margaret Mitchell",
                "Jian-Yun Nie",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "We present a novel response generation system that can be trained end to end\non large quantities of unstructured Twitter conversations. A neural network\narchitecture is used to address sparsity issues that arise when integrating\ncontextual information into classic statistical models, allowing the system to\ntake into account previous dialog utterances. Our dynamic-context generative\nmodels show consistent gains over both context-sensitive and\nnon-context-sensitive Machine Translation and Information Retrieval baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06832v1",
            "date": "2015-06-23 00:28:08+00:00",
            "title": "Detection and Analysis of Emotion From Speech Signals",
            "authors": [
                "Assel Davletcharova",
                "Sherin Sugathan",
                "Bibia Abraham",
                "Alex Pappachen James"
            ],
            "abstract": "Recognizing emotion from speech has become one the active research themes in\nspeech processing and in applications based on human-computer interaction. This\npaper conducts an experimental study on recognizing emotions from human speech.\nThe emotions considered for the experiments include neutral, anger, joy and\nsadness. The distinuishability of emotional features in speech were studied\nfirst followed by emotion classification performed on a custom dataset. The\nclassification was performed for different classifiers. One of the main feature\nattribute considered in the prepared dataset was the peak-to-peak distance\nobtained from the graphical representation of the speech signals. After\nperforming the classification tests on a dataset formed from 30 different\nsubjects, it was found that for getting better accuracy, one should consider\nthe data collected from one person rather than considering the data from a\ngroup of people.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1506.06833v2",
            "date": "2015-06-23 00:59:27+00:00",
            "title": "A Survey of Current Datasets for Vision and Language Research",
            "authors": [
                "Francis Ferraro",
                "Nasrin Mostafazadeh",
                "Ting-Hao",
                "Huang",
                "Lucy Vanderwende",
                "Jacob Devlin",
                "Michel Galley",
                "Margaret Mitchell"
            ],
            "abstract": "Integrating vision and language has long been a dream in work on artificial\nintelligence (AI). In the past two years, we have witnessed an explosion of\nwork that brings together vision and language from images to videos and beyond.\nThe available corpora have played a crucial role in advancing this area of\nresearch. In this paper, we propose a set of quality metrics for evaluating and\nanalyzing the vision & language datasets and categorize them accordingly. Our\nanalyses show that the most recent datasets have been using more complex\nlanguage and more abstract concepts, however, there are different strengths and\nweaknesses in each.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.07220v1",
            "date": "2015-06-24 01:43:11+00:00",
            "title": "Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks",
            "authors": [
                "Yangtuo Peng",
                "Hui Jiang"
            ],
            "abstract": "Financial news contains useful information on public companies and the\nmarket. In this paper we apply the popular word embedding methods and deep\nneural networks to leverage financial news to predict stock price movements in\nthe market. Experimental results have shown that our proposed methods are\nsimple but very effective, which can significantly improve the stock prediction\naccuracy on a standard financial database over the baseline system using only\nthe historical price information.",
            "journal ref": "None",
            "category": "cs.CE"
        },
        {
            "url": "http://arxiv.org/abs/1506.07285v5",
            "date": "2015-06-24 08:27:02+00:00",
            "title": "Ask Me Anything: Dynamic Memory Networks for Natural Language Processing",
            "authors": [
                "Ankit Kumar",
                "Ozan Irsoy",
                "Peter Ondruska",
                "Mohit Iyyer",
                "James Bradbury",
                "Ishaan Gulrajani",
                "Victor Zhong",
                "Romain Paulus",
                "Richard Socher"
            ],
            "abstract": "Most tasks in natural language processing can be cast into question answering\n(QA) problems over language input. We introduce the dynamic memory network\n(DMN), a neural network architecture which processes input sequences and\nquestions, forms episodic memories, and generates relevant answers. Questions\ntrigger an iterative attention process which allows the model to condition its\nattention on the inputs and the result of previous iterations. These results\nare then reasoned over in a hierarchical recurrent sequence model to generate\nanswers. The DMN can be trained end-to-end and obtains state-of-the-art results\non several types of tasks and datasets: question answering (Facebook's bAbI\ndataset), text classification for sentiment analysis (Stanford Sentiment\nTreebank) and sequence modeling for part-of-speech tagging (WSJ-PTB). The\ntraining for these different tasks relies exclusively on trained word vector\nrepresentations and input-question-answer triplets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.07477v1",
            "date": "2015-06-24 17:27:28+00:00",
            "title": "Efficient Learning for Undirected Topic Models",
            "authors": [
                "Jiatao Gu",
                "Victor O. K. Li"
            ],
            "abstract": "Replicated Softmax model, a well-known undirected topic model, is powerful in\nextracting semantic representations of documents. Traditional learning\nstrategies such as Contrastive Divergence are very inefficient. This paper\nprovides a novel estimator to speed up the learning based on Noise Contrastive\nEstimate, extended for documents of variant lengths and weighted inputs.\nExperiments on two benchmarks show that the new estimator achieves great\nlearning efficiency and high accuracy on document retrieval and classification.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1506.07503v1",
            "date": "2015-06-24 19:10:33+00:00",
            "title": "Attention-Based Models for Speech Recognition",
            "authors": [
                "Jan Chorowski",
                "Dzmitry Bahdanau",
                "Dmitriy Serdyuk",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "Recurrent sequence generators conditioned on input data through an attention\nmechanism have recently shown very good performance on a range of tasks in-\ncluding machine translation, handwriting synthesis and image caption gen-\neration. We extend the attention-mechanism with features needed for speech\nrecognition. We show that while an adaptation of the model used for machine\ntranslation in reaches a competitive 18.7% phoneme error rate (PER) on the\nTIMIT phoneme recognition task, it can only be applied to utterances which are\nroughly as long as the ones it was trained on. We offer a qualitative\nexplanation of this failure and propose a novel and generic method of adding\nlocation-awareness to the attention mechanism to alleviate this issue. The new\nmethod yields a model that is robust to long inputs and achieves 18% PER in\nsingle utterances and 20% in 10-times longer (repeated) utterances. Finally, we\npropose a change to the at- tention mechanism that prevents it from\nconcentrating too much on single frames, which further reduces PER to 17.6%\nlevel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.07732v1",
            "date": "2015-06-25 12:56:23+00:00",
            "title": "How to improve robustness in Kohonen maps and display additional information in Factorial Analysis: application to text mining",
            "authors": [
                "Nicolas Bourgeois",
                "Marie Cottrell",
                "Benjamin D\u00e9ruelle",
                "St\u00e9phane Lamass\u00e9",
                "Patrick Letr\u00e9my"
            ],
            "abstract": "This article is an extended version of a paper presented in the WSOM'2012\nconference [1]. We display a combination of factorial projections, SOM\nalgorithm and graph techniques applied to a text mining problem. The corpus\ncontains 8 medieval manuscripts which were used to teach arithmetic techniques\nto merchants. Among the techniques for Data Analysis, those used for\nLexicometry (such as Factorial Analysis) highlight the discrepancies between\nmanuscripts. The reason for this is that they focus on the deviation from the\nindependence between words and manuscripts. Still, we also want to discover and\ncharacterize the common vocabulary among the whole corpus. Using the properties\nof stochastic Kohonen maps, which define neighborhood between inputs in a\nnon-deterministic way, we highlight the words which seem to play a special role\nin the vocabulary. We call them fickle and use them to improve both Kohonen map\nrobustness and significance of FCA visualization. Finally we use graph\nalgorithmic to exploit this fickleness for classification of words.",
            "journal ref": "Neurocomputing, Elsevier, 2014, 147, pp.120-135",
            "category": "math.ST"
        },
        {
            "url": "http://arxiv.org/abs/1506.08126v1",
            "date": "2015-06-26 15:48:10+00:00",
            "title": "Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest",
            "authors": [
                "Dragomir Radev",
                "Amanda Stent",
                "Joel Tetreault",
                "Aasish Pappu",
                "Aikaterini Iliakopoulou",
                "Agustin Chanfreau",
                "Paloma de Juan",
                "Jordi Vallmitjana",
                "Alejandro Jaimes",
                "Rahul Jha",
                "Bob Mankoff"
            ],
            "abstract": "The New Yorker publishes a weekly captionless cartoon. More than 5,000\nreaders submit captions for it. The editors select three of them and ask the\nreaders to pick the funniest one. We describe an experiment that compares a\ndozen automatic methods for selecting the funniest caption. We show that\nnegative sentiment, human-centeredness, and lexical centrality most strongly\nmatch the funniest captions, followed by positive sentiment. These results are\nuseful for understanding humor and also in the design of more engaging\nconversational agents in text and multimodal (vision+text) systems. As part of\nthis work, a large set of cartoons and captions is being made available to the\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08349v1",
            "date": "2015-06-28 03:32:02+00:00",
            "title": "Improved Deep Speaker Feature Learning for Text-Dependent Speaker Recognition",
            "authors": [
                "Lantian Li",
                "Yiye Lin",
                "Zhiyong Zhang",
                "Dong Wang"
            ],
            "abstract": "A deep learning approach has been proposed recently to derive speaker\nidentifies (d-vector) by a deep neural network (DNN). This approach has been\napplied to text-dependent speaker recognition tasks and shows reasonable\nperformance gains when combined with the conventional i-vector approach.\nAlthough promising, the existing d-vector implementation still can not compete\nwith the i-vector baseline. This paper presents two improvements for the deep\nlearning approach: a phonedependent DNN structure to normalize phone variation,\nand a new scoring approach based on dynamic time warping (DTW). Experiments on\na text-dependent speaker recognition task demonstrated that the proposed\nmethods can provide considerable performance improvement over the existing\nd-vector implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08454v2",
            "date": "2015-06-28 21:17:26+00:00",
            "title": "WYSIWYE: An Algebra for Expressing Spatial and Textual Rules for Visual Information Extraction",
            "authors": [
                "Vijil Chenthamarakshan",
                "Prasad M Desphande",
                "Raghu Krishnapuram",
                "Ramakrishna Varadarajan",
                "Knut Stolze"
            ],
            "abstract": "The visual layout of a webpage can provide valuable clues for certain types\nof Information Extraction (IE) tasks. In traditional rule based IE frameworks,\nthese layout cues are mapped to rules that operate on the HTML source of the\nwebpages. In contrast, we have developed a framework in which the rules can be\nspecified directly at the layout level. This has many advantages, since the\nhigher level of abstraction leads to simpler extraction rules that are largely\nindependent of the source code of the page, and, therefore, more robust. It can\nalso enable specification of new types of rules that are not otherwise\npossible. To the best of our knowledge, there is no general framework that\nallows declarative specification of information extraction rules based on\nspatial layout. Our framework is complementary to traditional text based rules\nframework and allows a seamless combination of spatial layout based rules with\ntraditional text based rules. We describe the algebra that enables such a\nsystem and its efficient implementation using standard relational and text\nindexing features of a relational database. We demonstrate the simplicity and\nefficiency of this system for a task involving the extraction of software\nsystem requirements from software product pages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.08789v1",
            "date": "2015-06-29 19:21:04+00:00",
            "title": "Requirement Tracing using Term Extraction",
            "authors": [
                "Najla Al-Saati",
                "Raghda Abdul-Jaleel"
            ],
            "abstract": "Requirements traceability is an essential step in ensuring the quality of\nsoftware during the early stages of its development life cycle. Requirements\ntracing usually consists of document parsing, candidate link generation and\nevaluation and traceability analysis. This paper demonstrates the applicability\nof Statistical Term Extraction metrics to generate candidate links. It is\napplied and validated using two data sets and four types of filters two for\neach data set, 0.2 and 0.25 for MODIS, 0 and 0.05 for CM1. This method\ngenerates requirements traceability matrices between textual requirements\nartifacts (such as high-level requirements traced to low-level requirements).\nThe proposed method includes ten word frequency metrics divided into three main\ngroups for calculating the frequency of terms. The results show that the\nproposed method gives better result when compared with the traditional TF-IDF\nmethod.",
            "journal ref": "IJCSIS Vol. 13, No. 5, 2015",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1506.08909v3",
            "date": "2015-06-30 00:37:09+00:00",
            "title": "The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems",
            "authors": [
                "Ryan Lowe",
                "Nissan Pow",
                "Iulian Serban",
                "Joelle Pineau"
            ],
            "abstract": "This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost\n1 million multi-turn dialogues, with a total of over 7 million utterances and\n100 million words. This provides a unique resource for research into building\ndialogue managers based on neural language models that can make use of large\namounts of unlabeled data. The dataset has both the multi-turn property of\nconversations in the Dialog State Tracking Challenge datasets, and the\nunstructured nature of interactions from microblog services such as Twitter. We\nalso describe two neural learning architectures suitable for analyzing this\ndataset, and provide benchmark performance on the task of selecting the best\nnext response.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00200v1",
            "date": "2015-08-02 06:18:10+00:00",
            "title": "PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks",
            "authors": [
                "Jian Tang",
                "Meng Qu",
                "Qiaozhu Mei"
            ],
            "abstract": "Unsupervised text embedding methods, such as Skip-gram and Paragraph Vector,\nhave been attracting increasing attention due to their simplicity, scalability,\nand effectiveness. However, comparing to sophisticated deep learning\narchitectures such as convolutional neural networks, these methods usually\nyield inferior results when applied to particular machine learning tasks. One\npossible reason is that these text embedding methods learn the representation\nof text in a fully unsupervised way, without leveraging the labeled information\navailable for the task. Although the low dimensional representations learned\nare applicable to many different tasks, they are not particularly tuned for any\ntask. In this paper, we fill this gap by proposing a semi-supervised\nrepresentation learning method for text data, which we call the\n\\textit{predictive text embedding} (PTE). Predictive text embedding utilizes\nboth labeled and unlabeled data to learn the embedding of text. The labeled\ninformation and different levels of word co-occurrence information are first\nrepresented as a large-scale heterogeneous text network, which is then embedded\ninto a low dimensional space through a principled and efficient algorithm. This\nlow dimensional embedding not only preserves the semantic closeness of words\nand documents, but also has a strong predictive power for the particular task.\nCompared to recent supervised approaches based on convolutional neural\nnetworks, predictive text embedding is comparable or more effective, much more\nefficient, and has fewer parameters to tune.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00504v1",
            "date": "2015-07-31 16:21:17+00:00",
            "title": "Spin Glass Models of Syntax and Language Evolution",
            "authors": [
                "Karthik Siva",
                "Jim Tao",
                "Matilde Marcolli"
            ],
            "abstract": "Using the SSWL database of syntactic parameters of world languages, and the\nMIT Media Lab data on language interactions, we construct a spin glass model of\nlanguage evolution. We treat binary syntactic parameters as spin states, with\nlanguages as vertices of a graph, and assigned interaction energies along the\nedges. We study a rough model of syntax evolution, under the assumption that a\nstrong interaction energy tends to cause parameters to align, as in the case of\nferromagnetic materials. We also study how the spin glass model needs to be\nmodified to account for entailment relations between syntactic parameters. This\nmodification leads naturally to a generalization of Potts models with external\nmagnetic field, which consists of a coupling at the vertices of an Ising model\nand a Potts model with q=3, that have the same edge interactions. We describe\nthe results of simulations of the dynamics of these models, in different\ntemperature and energy regimes. We discuss the linguistic interpretation of the\nparameters of the physical model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.00973v1",
            "date": "2015-08-05 05:00:32+00:00",
            "title": "Progressive EM for Latent Tree Models and Hierarchical Topic Detection",
            "authors": [
                "Peixian Chen",
                "Nevin L. Zhang",
                "Leonard K. M. Poon",
                "Zhourong Chen"
            ],
            "abstract": "Hierarchical latent tree analysis (HLTA) is recently proposed as a new method\nfor topic detection. It differs fundamentally from the LDA-based methods in\nterms of topic definition, topic-document relationship, and learning method. It\nhas been shown to discover significantly more coherent topics and better topic\nhierarchies. However, HLTA relies on the Expectation-Maximization (EM)\nalgorithm for parameter estimation and hence is not efficient enough to deal\nwith large datasets. In this paper, we propose a method to drastically speed up\nHLTA using a technique inspired by recent advances in the moments method.\nEmpirical experiments show that our method greatly improves the efficiency of\nHLTA. It is as efficient as the state-of-the-art LDA-based method for\nhierarchical topic detection and finds substantially better topics and topic\nhierarchies.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1508.01006v2",
            "date": "2015-08-05 09:03:46+00:00",
            "title": "Relation Classification via Recurrent Neural Network",
            "authors": [
                "Dongxu Zhang",
                "Dong Wang"
            ],
            "abstract": "Deep learning has gained much success in sentence-level relation\nclassification. For example, convolutional neural networks (CNN) have delivered\ncompetitive performance without much effort on feature engineering as the\nconventional pattern-based methods. Thus a lot of works have been produced\nbased on CNN structures. However, a key issue that has not been well addressed\nby the CNN-based method is the lack of capability to learn temporal features,\nespecially long-distance dependency between nominal pairs. In this paper, we\npropose a simple framework based on recurrent neural networks (RNN) and compare\nit with CNN-based model. To show the limitation of popular used SemEval-2010\nTask 8 dataset, we introduce another dataset refined from MIMLRE(Angeli et al.,\n2014). Experiments on two different datasets strongly indicates that the\nRNN-based model can deliver better performance on relation classification, and\nit is particularly capable of learning long-distance relation patterns. This\nmakes it suitable for real-world applications where complicated expressions are\noften involved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01011v1",
            "date": "2015-08-05 09:22:25+00:00",
            "title": "Learning from LDA using Deep Neural Networks",
            "authors": [
                "Dongxu Zhang",
                "Tianyi Luo",
                "Dong Wang",
                "Rong Liu"
            ],
            "abstract": "Latent Dirichlet Allocation (LDA) is a three-level hierarchical Bayesian\nmodel for topic inference. In spite of its great success, inferring the latent\ntopic distribution with LDA is time-consuming. Motivated by the transfer\nlearning approach proposed by~\\newcite{hinton2015distilling}, we present a\nnovel method that uses LDA to supervise the training of a deep neural network\n(DNN), so that the DNN can approximate the costly LDA inference with less\ncomputation. Our experiments on a document classification task show that a\nsimple DNN can learn the LDA behavior pretty well, while the inference is\nspeeded up tens or hundreds of times.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1508.01211v2",
            "date": "2015-08-05 20:17:58+00:00",
            "title": "Listen, Attend and Spell",
            "authors": [
                "William Chan",
                "Navdeep Jaitly",
                "Quoc V. Le",
                "Oriol Vinyals"
            ],
            "abstract": "We present Listen, Attend and Spell (LAS), a neural network that learns to\ntranscribe speech utterances to characters. Unlike traditional DNN-HMM models,\nthis model learns all the components of a speech recognizer jointly. Our system\nhas two components: a listener and a speller. The listener is a pyramidal\nrecurrent network encoder that accepts filter bank spectra as inputs. The\nspeller is an attention-based recurrent network decoder that emits characters\nas outputs. The network produces character sequences without making any\nindependence assumptions between the characters. This is the key improvement of\nLAS over previous end-to-end CTC models. On a subset of the Google voice search\ntask, LAS achieves a word error rate (WER) of 14.1% without a dictionary or a\nlanguage model, and 10.3% with language model rescoring over the top 32 beams.\nBy comparison, the state-of-the-art CLDNN-HMM model achieves a WER of 8.0%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01306v2",
            "date": "2015-08-06 07:56:59+00:00",
            "title": "Replication and Generalization of PRECISE",
            "authors": [
                "Michael Minock",
                "Nils Everling"
            ],
            "abstract": "This report describes an initial replication study of the PRECISE system and\ndevelops a clearer, more formal description of the approach. Based on our\nevaluation, we conclude that the PRECISE results do not fully replicate.\nHowever the formalization developed here suggests a road map to further enhance\nand extend the approach pioneered by PRECISE.\n  After a long, productive discussion with Ana-Maria Popescu (one of the\nauthors of PRECISE) we got more clarity on the PRECISE approach and how the\nlexicon was authored for the GEO evaluation. Based on this we built a more\ndirect implementation over a repaired formalism. Although our new evaluation is\nnot yet complete, it is clear that the system is performing much better now. We\nwill continue developing our ideas and implementation and generate a future\nreport/publication that more accurately evaluates PRECISE like approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01420v1",
            "date": "2015-08-06 14:30:47+00:00",
            "title": "Privacy-Preserving Multi-Document Summarization",
            "authors": [
                "Lu\u00eds Marujo",
                "Jos\u00e9 Port\u00ealo",
                "Wang Ling",
                "David Martins de Matos",
                "Jo\u00e3o P. Neto",
                "Anatole Gershman",
                "Jaime Carbonell",
                "Isabel Trancoso",
                "Bhiksha Raj"
            ],
            "abstract": "State-of-the-art extractive multi-document summarization systems are usually\ndesigned without any concern about privacy issues, meaning that all documents\nare open to third parties. In this paper we propose a privacy-preserving\napproach to multi-document summarization. Our approach enables other parties to\nobtain summaries without learning anything else about the original documents'\ncontent. We use a hashing scheme known as Secure Binary Embeddings to convert\ndocuments representation containing key phrases and bag-of-words into bit\nstrings, allowing the computation of approximate distances, instead of exact\nones. Our experiments indicate that our system yields similar results to its\nnon-private counterpart on standard multi-document evaluation datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1508.01447v1",
            "date": "2015-08-06 16:10:21+00:00",
            "title": "Using Linguistic Analysis to Translate Arabic Natural Language Queries to SPARQL",
            "authors": [
                "Iyad AlAgha"
            ],
            "abstract": "The logic-based machine-understandable framework of the Semantic Web often\nchallenges naive users when they try to query ontology-based knowledge bases.\nExisting research efforts have approached this problem by introducing Natural\nLanguage (NL) interfaces to ontologies. These NL interfaces have the ability to\nconstruct SPARQL queries based on NL user queries. However, most efforts were\nrestricted to queries expressed in English, and they often benefited from the\nadvancement of English NLP tools. However, little research has been done to\nsupport querying the Arabic content on the Semantic Web by using NL queries.\nThis paper presents a domain-independent approach to translate Arabic NL\nqueries to SPARQL by leveraging linguistic analysis. Based on a special\nconsideration on Noun Phrases (NPs), our approach uses a language parser to\nextract NPs and the relations from Arabic parse trees and match them to the\nunderlying ontology. It then utilizes knowledge in the ontology to group NPs\ninto triple-based representations. A SPARQL query is finally generated by\nextracting targets and modifiers, and interpreting them into SPARQL. The\ninterpretation of advanced semantic features including negation, conjunctive\nand disjunctive modifiers is also supported. The approach was evaluated by\nusing two datasets consisting of OWL test data and queries, and the obtained\nresults have confirmed its feasibility to translate Arabic NL queries to\nSPARQL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01993v2",
            "date": "2015-08-09 07:39:24+00:00",
            "title": "Improving Decision Analytics with Deep Learning: The Case of Financial Disclosures",
            "authors": [
                "Stefan Feuerriegel",
                "Ralph Fehrer"
            ],
            "abstract": "Decision analytics commonly focuses on the text mining of financial news\nsources in order to provide managerial decision support and to predict stock\nmarket movements. Existing predictive frameworks almost exclusively apply\ntraditional machine learning methods, whereas recent research indicates that\ntraditional machine learning methods are not sufficiently capable of extracting\nsuitable features and capturing the non-linear nature of complex tasks. As a\nremedy, novel deep learning models aim to overcome this issue by extending\ntraditional neural network models with additional hidden layers. Indeed, deep\nlearning has been shown to outperform traditional methods in terms of\npredictive performance. In this paper, we adapt the novel deep learning\ntechnique to financial decision support. In this instance, we aim to predict\nthe direction of stock movements following financial disclosures. As a result,\nwe show how deep learning can outperform the accuracy of random forests as a\nbenchmark for machine learning by 5.66%.",
            "journal ref": "Twenty-Fourth European Conference on Information Systems (ECIS\n  2016), Istanbul, Turkey, 2016",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1508.02354v2",
            "date": "2015-08-10 19:04:18+00:00",
            "title": "Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning",
            "authors": [
                "Jianpeng Cheng",
                "Dimitri Kartsaklis"
            ],
            "abstract": "Deep compositional models of meaning acting on distributional representations\nof words in order to produce vectors of larger text constituents are evolving\nto a popular area of NLP research. We detail a compositional distributional\nframework based on a rich form of word embeddings that aims at facilitating the\ninteractions between words in the context of a sentence. Embeddings and\ncomposition layers are jointly learned against a generic objective that\nenhances the vectors with syntactic information from the surrounding context.\nFurthermore, each word is associated with a number of senses, the most\nplausible of which is selected dynamically during the composition process. We\nevaluate the produced vectors qualitatively and quantitatively with positive\nresults. At the sentence level, the effectiveness of the framework is\ndemonstrated on the MSRPar task, for which we report results within the\nstate-of-the-art range.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03170v1",
            "date": "2015-08-13 10:56:42+00:00",
            "title": "Generation of Multimedia Artifacts: An Extractive Summarization-based Approach",
            "authors": [
                "Paulo Figueiredo",
                "Marta Apar\u00edcio",
                "David Martins de Matos",
                "Ricardo Ribeiro"
            ],
            "abstract": "We explore methods for content selection and address the issue of coherence\nin the context of the generation of multimedia artifacts. We use audio and\nvideo to present two case studies: generation of film tributes, and\nlecture-driven science talks. For content selection, we use centrality-based\nand diversity-based summarization, along with topic analysis. To establish\ncoherence, we use the emotional content of music, for film tributes, and ensure\ntopic similarity between lectures and documentaries, for science talks.\nComposition techniques for the production of multimedia artifacts are addressed\nas a means of organizing content, in order to improve coherence. We discuss our\nresults considering the above aspects.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1508.03276v1",
            "date": "2015-08-13 17:34:07+00:00",
            "title": "Talking about the Moving Image: A Declarative Model for Image Schema Based Embodied Perception Grounding and Language Generation",
            "authors": [
                "Jakob Suchan",
                "Mehul Bhatt",
                "Harshita Jhavar"
            ],
            "abstract": "We present a general theory and corresponding declarative model for the\nembodied grounding and natural language based analytical summarisation of\ndynamic visuo-spatial imagery. The declarative model ---ecompassing\nspatio-linguistic abstractions, image schemas, and a spatio-temporal feature\nbased language generator--- is modularly implemented within Constraint Logic\nProgramming (CLP). The implemented model is such that primitives of the theory,\ne.g., pertaining to space and motion, image schemata, are available as\nfirst-class objects with `deep semantics' suited for inference and query. We\ndemonstrate the model with select examples broadly motivated by areas such as\nfilm, design, geography, smart environments where analytical natural language\nbased externalisations of the moving image are central from the viewpoint of\nhuman interaction, evidence-based qualitative analysis, and sensemaking.\n  Keywords: moving image, visual semantics and embodiment, visuo-spatial\ncognition and computation, cognitive vision, computational models of narrative,\ndeclarative spatial reasoning",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1508.03530v1",
            "date": "2015-07-30 03:10:18+00:00",
            "title": "Information-theoretical analysis of the statistical dependencies among three variables: Applications to written language",
            "authors": [
                "Dami\u00e1n G. Hern\u00e1ndez",
                "Dami\u00e1n H. Zanette",
                "In\u00e9s Samengo"
            ],
            "abstract": "We develop the information-theoretical concepts required to study the\nstatistical dependencies among three variables. Some of such dependencies are\npure triple interactions, in the sense that they cannot be explained in terms\nof a combination of pairwise correlations. We derive bounds for triple\ndependencies, and characterize the shape of the joint probability distribution\nof three binary variables with high triple interaction. The analysis also\nallows us to quantify the amount of redundancy in the mutual information\nbetween pairs of variables, and to assess whether the information between two\nvariables is or is not mediated by a third variable. These concepts are applied\nto the analysis of written texts. We find that the probability that a given\nword is found in a particular location within the text is not only modulated by\nthe presence or absence of other nearby words, but also, on the presence or\nabsence of nearby pairs of words. We identify the words enclosing the key\nsemantic concepts of the text, the triplets of words with high pairwise and\ntriple interactions, and the words that mediate the pairwise interactions\nbetween other words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03826v1",
            "date": "2015-08-16 14:12:17+00:00",
            "title": "A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution",
            "authors": [
                "Shaohua Li",
                "Jun Zhu",
                "Chunyan Miao"
            ],
            "abstract": "Most existing word embedding methods can be categorized into Neural Embedding\nModels and Matrix Factorization (MF)-based methods. However some models are\nopaque to probabilistic interpretation, and MF-based methods, typically solved\nusing Singular Value Decomposition (SVD), may incur loss of corpus information.\nIn addition, it is desirable to incorporate global latent factors, such as\ntopics, sentiments or writing styles, into the word embedding model. Since\ngenerative models provide a principled way to incorporate latent factors, we\npropose a generative word embedding model, which is easy to interpret, and can\nserve as a basis of more sophisticated latent factor models. The model\ninference reduces to a low rank weighted positive semidefinite approximation\nproblem. Its optimization is approached by eigendecomposition on a submatrix,\nfollowed by online blockwise regression, which is scalable and avoids the\ninformation loss in SVD. In experiments on 7 common benchmark datasets, our\nvectors are competitive to word2vec, and better than other MF-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03854v1",
            "date": "2015-08-16 18:27:25+00:00",
            "title": "Online Representation Learning in Recurrent Neural Language Models",
            "authors": [
                "Marek Rei"
            ],
            "abstract": "We investigate an extension of continuous online learning in recurrent neural\nnetwork language models. The model keeps a separate vector representation of\nthe current unit of text being processed and adaptively adjusts it after each\nprediction. The initial experiments give promising results, indicating that the\nmethod is able to increase language modelling accuracy, while also decreasing\nthe parameters needed to store the model along with the computation required at\neach step.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.03868v3",
            "date": "2015-08-16 21:43:59+00:00",
            "title": "Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology",
            "authors": [
                "Brendan Jou",
                "Tao Chen",
                "Nikolaos Pappas",
                "Miriam Redi",
                "Mercan Topkara",
                "Shih-Fu Chang"
            ],
            "abstract": "Every culture and language is unique. Our work expressly focuses on the\nuniqueness of culture and language in relation to human affect, specifically\nsentiment and emotion semantics, and how they manifest in social multimedia. We\ndevelop sets of sentiment- and emotion-polarized visual concepts by adapting\nsemantic structures called adjective-noun pairs, originally introduced by Borth\net al. (2013), but in a multilingual context. We propose a new\nlanguage-dependent method for automatic discovery of these adjective-noun\nconstructs. We show how this pipeline can be applied on a social multimedia\nplatform for the creation of a large-scale multilingual visual sentiment\nconcept ontology (MVSO). Unlike the flat structure in Borth et al. (2013), our\nunified ontology is organized hierarchically by multilingual clusters of\nvisually detectable nouns and subclusters of emotionally biased versions of\nthese nouns. In addition, we present an image-based prediction task to show how\ngeneralizable language-specific models are in a multilingual context. A new,\npublicly available dataset of >15.6K sentiment-biased visual concepts across 12\nlanguages with language-specific detector banks, >7.36M images and their\nmetadata is also released.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1508.04395v2",
            "date": "2015-08-18 17:40:00+00:00",
            "title": "End-to-End Attention-based Large Vocabulary Speech Recognition",
            "authors": [
                "Dzmitry Bahdanau",
                "Jan Chorowski",
                "Dmitriy Serdyuk",
                "Philemon Brakel",
                "Yoshua Bengio"
            ],
            "abstract": "Many of the current state-of-the-art Large Vocabulary Continuous Speech\nRecognition Systems (LVCSR) are hybrids of neural networks and Hidden Markov\nModels (HMMs). Most of these systems contain separate components that deal with\nthe acoustic modelling, language modelling and sequence decoding. We\ninvestigate a more direct approach in which the HMM is replaced with a\nRecurrent Neural Network (RNN) that performs sequence prediction directly at\nthe character level. Alignment between the input features and the desired\ncharacter sequence is learned automatically by an attention mechanism built\ninto the RNN. For each predicted character, the attention mechanism scans the\ninput sequence and chooses relevant frames. We propose two methods to speed up\nthis operation: limiting the scan to a subset of most promising frames and\npooling over time the information contained in neighboring frames, thereby\nreducing source sequence length. Integrating an n-gram language model into the\ndecoding process yields recognition accuracies similar to other HMM-free\nRNN-based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.05508v1",
            "date": "2015-08-22 13:15:09+00:00",
            "title": "Towards Neural Network-based Reasoning",
            "authors": [
                "Baolin Peng",
                "Zhengdong Lu",
                "Hang Li",
                "Kam-Fai Wong"
            ],
            "abstract": "We propose Neural Reasoner, a framework for neural network-based reasoning\nover natural language sentences. Given a question, Neural Reasoner can infer\nover multiple supporting facts and find an answer to the question in specific\nforms. Neural Reasoner has 1) a specific interaction-pooling mechanism,\nallowing it to examine multiple facts, and 2) a deep architecture, allowing it\nto model the complicated logical relations in reasoning tasks. Assuming no\nparticular structure exists in the question and facts, Neural Reasoner is able\nto accommodate different types of reasoning and different forms of language\nexpressions. Despite the model complexity, Neural Reasoner can still be trained\neffectively in an end-to-end manner. Our empirical studies show that Neural\nReasoner can outperform existing neural reasoning systems with remarkable\nmargins on two difficult artificial tasks (Positional Reasoning and Path\nFinding) proposed in [8]. For example, it improves the accuracy on Path\nFinding(10K) from 33.4% [6] to over 98%.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1508.05565v2",
            "date": "2015-08-23 03:44:26+00:00",
            "title": "Necessary and Sufficient Conditions and a Provably Efficient Algorithm for Separable Topic Discovery",
            "authors": [
                "Weicong Ding",
                "Prakash Ishwar",
                "Venkatesh Saligrama"
            ],
            "abstract": "We develop necessary and sufficient conditions and a novel provably\nconsistent and efficient algorithm for discovering topics (latent factors) from\nobservations (documents) that are realized from a probabilistic mixture of\nshared latent factors that have certain properties. Our focus is on the class\nof topic models in which each shared latent factor contains a novel word that\nis unique to that factor, a property that has come to be known as separability.\nOur algorithm is based on the key insight that the novel words correspond to\nthe extreme points of the convex hull formed by the row-vectors of a suitably\nnormalized word co-occurrence matrix. We leverage this geometric insight to\nestablish polynomial computation and sample complexity bounds based on a few\nisotropic random projections of the rows of the normalized word co-occurrence\nmatrix. Our proposed random-projections-based algorithm is naturally amenable\nto an efficient distributed implementation and is attractive for modern\nweb-scale distributed data mining applications.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1508.05817v1",
            "date": "2015-08-24 14:15:39+00:00",
            "title": "Echoes of Persuasion: The Effect of Euphony in Persuasive Communication",
            "authors": [
                "Marco Guerini",
                "G\u00f6zde \u00d6zbal",
                "Carlo Strapparava"
            ],
            "abstract": "While the effect of various lexical, syntactic, semantic and stylistic\nfeatures have been addressed in persuasive language from a computational point\nof view, the persuasive effect of phonetics has received little attention. By\nmodeling a notion of euphony and analyzing four datasets comprising persuasive\nand non-persuasive sentences in different domains (political speeches, movie\nquotes, slogans and tweets), we explore the impact of sounds on different forms\nof persuasiveness. We conduct a series of analyses and prediction experiments\nwithin and across datasets. Our results highlight the positive role of phonetic\ndevices on persuasion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06374v2",
            "date": "2015-08-26 06:18:51+00:00",
            "title": "A fully data-driven method to identify (correlated) changes in diachronic corpora",
            "authors": [
                "Alexander Koplenig"
            ],
            "abstract": "In this paper, a method for measuring synchronic corpus (dis-)similarity put\nforward by Kilgarriff (2001) is adapted and extended to identify trends and\ncorrelated changes in diachronic text data, using the Corpus of Historical\nAmerican English (Davies 2010a) and the Google Ngram Corpora (Michel et al.\n2010a). This paper shows that this fully data-driven method, which extracts\nword types that have undergone the most pronounced change in frequency in a\ngiven period of time, is computationally very cheap and that it allows\ninterpretations of diachronic trends that are both intuitively plausible and\nmotivated from the perspective of information theory. Furthermore, it\ndemonstrates that the method is able to identify correlated linguistic changes\nand diachronic shifts that can be linked to historical events. Finally, it can\nhelp to improve diachronic POS tagging and complement existing NLP approaches.\nThis indicates that the approach can facilitate an improved understanding of\ndiachronic processes in language change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06451v2",
            "date": "2015-08-26 11:39:18+00:00",
            "title": "Crossings as a side effect of dependency lengths",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "The syntactic structure of sentences exhibits a striking regularity:\ndependencies tend to not cross when drawn above the sentence. We investigate\ntwo competing explanations. The traditional hypothesis is that this trend\narises from an independent principle of syntax that reduces crossings\npractically to zero. An alternative to this view is the hypothesis that\ncrossings are a side effect of dependency lengths, i.e. sentences with shorter\ndependency lengths should tend to have fewer crossings. We are able to reject\nthe traditional view in the majority of languages considered. The alternative\nhypothesis can lead to a more parsimonious theory of language.",
            "journal ref": "Complexity, 21(S2):320-328, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.06615v4",
            "date": "2015-08-26 19:25:34+00:00",
            "title": "Character-Aware Neural Language Models",
            "authors": [
                "Yoon Kim",
                "Yacine Jernite",
                "David Sontag",
                "Alexander M. Rush"
            ],
            "abstract": "We describe a simple neural language model that relies only on\ncharacter-level inputs. Predictions are still made at the word-level. Our model\nemploys a convolutional neural network (CNN) and a highway network over\ncharacters, whose output is given to a long short-term memory (LSTM) recurrent\nneural network language model (RNN-LM). On the English Penn Treebank the model\nis on par with the existing state-of-the-art despite having 60% fewer\nparameters. On languages with rich morphology (Arabic, Czech, French, German,\nSpanish, Russian), the model outperforms word-level/morpheme-level LSTM\nbaselines, again with fewer parameters. The results suggest that on many\nlanguages, character inputs are sufficient for language modeling. Analysis of\nword representations obtained from the character composition part of the model\nreveals that the model is able to encode, from characters only, both semantic\nand orthographic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.07266v1",
            "date": "2015-08-28 16:21:03+00:00",
            "title": "Understanding Editing Behaviors in Multilingual Wikipedia",
            "authors": [
                "Suin Kim",
                "Sungjoon Park",
                "Scott A. Hale",
                "Sooyoung Kim",
                "Jeongmin Byun",
                "Alice Oh"
            ],
            "abstract": "Multilingualism is common offline, but we have a more limited understanding\nof the ways multilingualism is displayed online and the roles that\nmultilinguals play in the spread of content between speakers of different\nlanguages. We take a computational approach to studying multilingualism using\none of the largest user-generated content platforms, Wikipedia. We study\nmultilingualism by collecting and analyzing a large dataset of the content\nwritten by multilingual editors of the English, German, and Spanish editions of\nWikipedia. This dataset contains over two million paragraphs edited by over\n15,000 multilingual users from July 8 to August 9, 2013. We analyze these\nmultilingual editors in terms of their engagement, interests, and language\nproficiency in their primary and non-primary (secondary) languages and find\nthat the English edition of Wikipedia displays different dynamics from the\nSpanish and German editions. Users primarily editing the Spanish and German\neditions make more complex edits than users who edit these editions as a second\nlanguage. In contrast, users editing the English edition as a second language\nmake edits that are just as complex as the edits by users who primarily edit\nthe English edition. In this way, English serves a special role bringing\ntogether content written by multilinguals from many language editions.\nNonetheless, language remains a formidable hurdle to the spread of content: we\nfind evidence for a complexity barrier whereby editors are less likely to edit\ncomplex content in a second language. In addition, we find that multilinguals\nare less engaged and show lower levels of language proficiency in their second\nlanguages. We also examine the topical interests of multilingual editors and\nfind that there is no significant difference between primary and non-primary\neditors in each language.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1508.07709v2",
            "date": "2015-08-31 07:52:50+00:00",
            "title": "Word Representations, Tree Models and Syntactic Functions",
            "authors": [
                "Simon \u0160uster",
                "Gertjan van Noord",
                "Ivan Titov"
            ],
            "abstract": "Word representations induced from models with discrete latent variables\n(e.g.\\ HMMs) have been shown to be beneficial in many NLP applications. In this\nwork, we exploit labeled syntactic dependency trees and formalize the induction\nproblem as unsupervised learning of tree-structured hidden Markov models.\nSyntactic functions are used as additional observed variables in the model,\ninfluencing both transition and emission components. Such syntactic information\ncan potentially lead to capturing more fine-grain and functional distinctions\nbetween words, which, in turn, may be desirable in many NLP applications. We\nevaluate the word representations on two tasks -- named entity recognition and\nsemantic frame identification. We observe improvements from exploiting\nsyntactic function information in both cases, and the results rivaling those of\nstate-of-the-art representation learning methods. Additionally, we revisit the\nrelationship between sequential and unlabeled-tree models and find that the\nadvantage of the latter is not self-evident.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.00040v2",
            "date": "2015-10-30 22:21:45+00:00",
            "title": "Quantifying the Cognitive Extent of Science",
            "authors": [
                "Sta\u0161a Milojevi\u0107"
            ],
            "abstract": "While the modern science is characterized by an exponential growth in\nscientific literature, the increase in publication volume clearly does not\nreflect the expansion of the cognitive boundaries of science. Nevertheless,\nmost of the metrics for assessing the vitality of science or for making funding\nand policy decisions are based on productivity. Similarly, the increasing level\nof knowledge production by large science teams, whose results often enjoy\ngreater visibility, does not necessarily mean that \"big science\" leads to\ncognitive expansion. Here we present a novel, big-data method to quantify the\nextents of cognitive domains of different bodies of scientific literature\nindependently from publication volume, and apply it to 20 million articles\npublished over 60-130 years in physics, astronomy, and biomedicine. The method\nis based on the lexical diversity of titles of fixed quotas of research\narticles. Owing to large size of quotas, the method overcomes the inherent\nstochasticity of article titles to achieve <1% precision. We show that the\nperiods of cognitive growth do not necessarily coincide with the trends in\npublication volume. Furthermore, we show that the articles produced by larger\nteams cover significantly smaller cognitive territory than (the same quota of)\narticles from smaller teams. Our findings provide a new perspective on the role\nof small teams and individual researchers in expanding the cognitive boundaries\nof science. The proposed method of quantifying the extent of the cognitive\nterritory can also be applied to study many other aspects of \"science of\nscience.\"",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1511.00352v3",
            "date": "2015-11-02 01:45:41+00:00",
            "title": "Spatial Semantic Scan: Jointly Detecting Subtle Events and their Spatial Footprint",
            "authors": [
                "Abhinav Maurya"
            ],
            "abstract": "Many methods have been proposed for detecting emerging events in text streams\nusing topic modeling. However, these methods have shortcomings that make them\nunsuitable for rapid detection of locally emerging events on massive text\nstreams. We describe Spatially Compact Semantic Scan (SCSS) that has been\ndeveloped specifically to overcome the shortcomings of current methods in\ndetecting new spatially compact events in text streams. SCSS employs\nalternating optimization between using semantic scan to estimate contrastive\nforeground topics in documents, and discovering spatial neighborhoods with high\noccurrence of documents containing the foreground topics. We evaluate our\nmethod on Emergency Department chief complaints dataset (ED dataset) to verify\nthe effectiveness of our method in detecting real-world disease outbreaks from\nfree-text ED chief complaint data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.00622v2",
            "date": "2015-11-02 18:17:26+00:00",
            "title": "On the Number of Many-to-Many Alignments of Multiple Sequences",
            "authors": [
                "Steffen Eger"
            ],
            "abstract": "We count the number of alignments of $N \\ge 1$ sequences when match-up types\nare from a specified set $S\\subseteq \\mathbb{N}^N$. Equivalently, we count the\nnumber of nonnegative integer matrices whose rows sum to a given fixed vector\nand each of whose columns lie in $S$. We provide a new asymptotic formula for\nthe case $S=\\{(s_1,\\ldots,s_N) \\:|\\: 1\\le s_i\\le 2\\}$.",
            "journal ref": "Journal of Automata, Languages and Combinatorics 20 (2015) 1,\n  53-65",
            "category": "math.CO"
        },
        {
            "url": "http://arxiv.org/abs/1511.01042v2",
            "date": "2015-11-03 19:26:16+00:00",
            "title": "Detecting Interrogative Utterances with Recurrent Neural Networks",
            "authors": [
                "Junyoung Chung",
                "Jacob Devlin",
                "Hany Hassan Awadalla"
            ],
            "abstract": "In this paper, we explore different neural network architectures that can\npredict if a speaker of a given utterance is asking a question or making a\nstatement. We com- pare the outcomes of regularization methods that are\npopularly used to train deep neural networks and study how different context\nfunctions can affect the classification performance. We also compare the\nefficacy of gated activation functions that are favorably used in recurrent\nneural networks and study how to combine multimodal inputs. We evaluate our\nmodels on two multimodal datasets: MSR-Skype and CALLHOME.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01158v3",
            "date": "2015-11-03 23:18:35+00:00",
            "title": "Distributed Deep Learning for Question Answering",
            "authors": [
                "Minwei Feng",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "This paper is an empirical study of the distributed deep learning for\nquestion answering subtasks: answer selection and question classification.\nComparison studies of SGD, MSGD, ADADELTA, ADAGRAD, ADAM/ADAMAX, RMSPROP,\nDOWNPOUR and EASGD/EAMSGD algorithms have been presented. Experimental results\nshow that the distributed framework based on the message passing interface can\naccelerate the convergence speed at a sublinear scale. This paper demonstrates\nthe importance of distributed training. For example, with 48 workers, a 24x\nspeedup is achievable for the answer selection task and running time is\ndecreased from 138.2 hours to 5.81 hours, which will increase the productivity\nsignificantly.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.01480v1",
            "date": "2015-11-04 19:30:27+00:00",
            "title": "Approximation of the truncated Zeta distribution and Zipf's law",
            "authors": [
                "Maurizio Naldi"
            ],
            "abstract": "Zipf's law appears in many application areas but does not have a closed form\nexpression, which may make its use cumbersome. Since it coincides with the\ntruncated version of the Zeta distribution, in this paper we propose three\napproximate closed form expressions for the truncated Zeta distribution, which\nmay be employed for Zipf's law as well. The three approximations are based on\nthe replacement of the sum occurring in Zipf's law with an integral, and are\nnamed respectively the integral approximation, the average integral\napproximation, and the trapezoidal approximation. While the first one is shown\nto be of little use, the trapezoidal approximation exhibits an error which is\ntypically lower than 1\\%, but is as low as 0.1\\% for the range of values of the\nZipf parameter below 1.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1511.01556v1",
            "date": "2015-11-04 23:39:46+00:00",
            "title": "Mining Local Gazetteers of Literary Chinese with CRF and Pattern based Methods for Biographical Information in Chinese History",
            "authors": [
                "Chao-Lin Liu",
                "Chih-Kai Huang",
                "Hongsu Wang",
                "Peter K. Bol"
            ],
            "abstract": "Person names and location names are essential building blocks for identifying\nevents and social networks in historical documents that were written in\nliterary Chinese. We take the lead to explore the research on algorithmically\nrecognizing named entities in literary Chinese for historical studies with\nlanguage-model based and conditional-random-field based methods, and extend our\nwork to mining the document structures in historical documents. Practical\nevaluations were conducted with texts that were extracted from more than 220\nvolumes of local gazetteers (Difangzhi). Difangzhi is a huge and the single\nmost important collection that contains information about officers who served\nin local government in Chinese history. Our methods performed very well on\nthese realistic tests. Thousands of names and addresses were identified from\nthe texts. A good portion of the extracted names match the biographical\ninformation currently recorded in the China Biographical Database (CBDB) of\nHarvard University, and many others can be verified by historians and will\nbecome as new additions to CBDB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.01559v1",
            "date": "2015-11-05 00:21:40+00:00",
            "title": "Color Aesthetics and Social Networks in Complete Tang Poems: Explorations and Discoveries",
            "authors": [
                "Chao-Lin Liu",
                "Hongsu Wang",
                "Wen-Huei Cheng",
                "Chu-Ting Hsu",
                "Wei-Yun Chiu"
            ],
            "abstract": "The Complete Tang Poems (CTP) is the most important source to study Tang\npoems. We look into CTP with computational tools from specific linguistic\nperspectives, including distributional semantics and collocational analysis.\nFrom such quantitative viewpoints, we compare the usage of \"wind\" and \"moon\" in\nthe poems of Li Bai and Du Fu. Colors in poems function like sounds in movies,\nand play a crucial role in the imageries of poems. Thus, words for colors are\nstudied, and \"white\" is the main focus because it is the most frequent color in\nCTP. We also explore some cases of using colored words in antithesis pairs that\nwere central for fostering the imageries of the poems. CTP also contains useful\nhistorical information, and we extract person names in CTP to study the social\nnetworks of the Tang poets. Such information can then be integrated with the\nChina Biographical Database of Harvard University.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02274v2",
            "date": "2015-11-07 00:43:32+00:00",
            "title": "Stacked Attention Networks for Image Question Answering",
            "authors": [
                "Zichao Yang",
                "Xiaodong He",
                "Jianfeng Gao",
                "Li Deng",
                "Alex Smola"
            ],
            "abstract": "This paper presents stacked attention networks (SANs) that learn to answer\nnatural language questions from images. SANs use semantic representation of a\nquestion as query to search for the regions in an image that are related to the\nanswer. We argue that image question answering (QA) often requires multiple\nsteps of reasoning. Thus, we develop a multiple-layer SAN in which we query an\nimage multiple times to infer the answer progressively. Experiments conducted\non four image QA data sets demonstrate that the proposed SANs significantly\noutperform previous state-of-the-art approaches. The visualization of the\nattention layers illustrates the progress that the SAN locates the relevant\nvisual clues that lead to the answer of the question layer-by-layer.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.02283v3",
            "date": "2015-11-07 02:17:36+00:00",
            "title": "Generation and Comprehension of Unambiguous Object Descriptions",
            "authors": [
                "Junhua Mao",
                "Jonathan Huang",
                "Alexander Toshev",
                "Oana Camburu",
                "Alan Yuille",
                "Kevin Murphy"
            ],
            "abstract": "We propose a method that can generate an unambiguous description (known as a\nreferring expression) of a specific object or region in an image, and which can\nalso comprehend or interpret such an expression to infer which object is being\ndescribed. We show that our method outperforms previous methods that generate\ndescriptions of objects without taking into account other potentially ambiguous\nobjects in the scene. Our model is inspired by recent successes of deep\nlearning methods for image captioning, but while image captioning is difficult\nto evaluate, our task allows for easy objective evaluation. We also present a\nnew large-scale dataset for referring expressions, based on MS-COCO. We have\nreleased the dataset and a toolbox for visualization and evaluation, see\nhttps://github.com/mjhucla/Google_Refexp_toolbox",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.02385v1",
            "date": "2015-11-07 18:38:22+00:00",
            "title": "Review-Level Sentiment Classification with Sentence-Level Polarity Correction",
            "authors": [
                "Sylvester Olubolu Orimaye",
                "Saadat M. Alhashmi",
                "Eu-Gene Siew",
                "Sang Jung Kang"
            ],
            "abstract": "We propose an effective technique to solving review-level sentiment\nclassification problem by using sentence-level polarity correction. Our\npolarity correction technique takes into account the consistency of the\npolarities (positive and negative) of sentences within each product review\nbefore performing the actual machine learning task. While sentences with\ninconsistent polarities are removed, sentences with consistent polarities are\nused to learn state-of-the-art classifiers. The technique achieved better\nresults on different types of products reviews and outperforms baseline models\nwithout the correction technique. Experimental results show an average of 82%\nF-measure on four different product review domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02506v1",
            "date": "2015-11-08 17:08:54+00:00",
            "title": "Towards Structured Deep Neural Network for Automatic Speech Recognition",
            "authors": [
                "Yi-Hsiu Liao",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "In this paper we propose the Structured Deep Neural Network (structured DNN)\nas a structured and deep learning framework. This approach can learn to find\nthe best structured object (such as a label sequence) given a structured input\n(such as a vector sequence) by globally considering the mapping relationships\nbetween the structures rather than item by item.\n  When automatic speech recognition is viewed as a special case of such a\nstructured learning problem, where we have the acoustic vector sequence as the\ninput and the phoneme label sequence as the output, it becomes possible to\ncomprehensively learn utterance by utterance as a whole, rather than frame by\nframe.\n  Structured Support Vector Machine (structured SVM) was proposed to perform\nASR with structured learning previously, but limited by the linear nature of\nSVM. Here we propose structured DNN to use nonlinear transformations in\nmulti-layers as a structured and deep learning approach. This approach was\nshown to beat structured SVM in preliminary experiments on TIMIT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.02799v4",
            "date": "2015-11-09 18:48:39+00:00",
            "title": "Neural Module Networks",
            "authors": [
                "Jacob Andreas",
                "Marcus Rohrbach",
                "Trevor Darrell",
                "Dan Klein"
            ],
            "abstract": "Visual question answering is fundamentally compositional in nature---a\nquestion like \"where is the dog?\" shares substructure with questions like \"what\ncolor is the dog?\" and \"where is the cat?\" This paper seeks to simultaneously\nexploit the representational capacity of deep networks and the compositional\nlinguistic structure of questions. We describe a procedure for constructing and\nlearning *neural module networks*, which compose collections of jointly-trained\nneural \"modules\" into deep networks for question answering. Our approach\ndecomposes questions into their linguistic substructures, and uses these\nstructures to dynamically instantiate modular networks (with reusable\ncomponents for recognizing dogs, classifying colors, etc.). The resulting\ncompound networks are jointly trained. We evaluate our approach on two\nchallenging datasets for visual question answering, achieving state-of-the-art\nresults on both the VQA natural image dataset and a new dataset of complex\nquestions about abstract shapes.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.03012v1",
            "date": "2015-11-10 08:13:49+00:00",
            "title": "Information retrieval in folktales using natural language processing",
            "authors": [
                "Adrian Groza",
                "Lidia Corde"
            ],
            "abstract": "Our aim is to extract information about literary characters in unstructured\ntexts. We employ natural language processing and reasoning on domain\nontologies. The first task is to identify the main characters and the parts of\nthe story where these characters are described or act. We illustrate the system\nin a scenario in the folktale domain. The system relies on a folktale ontology\nthat we have developed based on Propp's model for folktales morphology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.03292v1",
            "date": "2015-11-10 21:14:51+00:00",
            "title": "From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge",
            "authors": [
                "Somak Aditya",
                "Yezhou Yang",
                "Chitta Baral",
                "Cornelia Fermuller",
                "Yiannis Aloimonos"
            ],
            "abstract": "In this paper we propose the construction of linguistic descriptions of\nimages. This is achieved through the extraction of scene description graphs\n(SDGs) from visual scenes using an automatically constructed knowledge base.\nSDGs are constructed using both vision and reasoning. Specifically, commonsense\nreasoning is applied on (a) detections obtained from existing perception\nmethods on given images, (b) a \"commonsense\" knowledge base constructed using\nnatural language processing of image annotations and (c) lexical ontological\nknowledge from resources such as WordNet. Amazon Mechanical Turk(AMT)-based\nevaluations on Flickr8k, Flickr30k and MS-COCO datasets show that in most\ncases, sentences auto-constructed from SDGs obtained by our method give a more\nrelevant and thorough description of an image than a recent state-of-the-art\nimage caption based approach. Our Image-Sentence Alignment Evaluation results\nare also comparable to that of the recent state-of-the art approaches.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.03546v4",
            "date": "2015-11-11 15:58:30+00:00",
            "title": "Hierarchical Latent Semantic Mapping for Automated Topic Generation",
            "authors": [
                "Guorui Zhou",
                "Guang Chen"
            ],
            "abstract": "Much of information sits in an unprecedented amount of text data. Managing\nallocation of these large scale text data is an important problem for many\nareas. Topic modeling performs well in this problem. The traditional generative\nmodels (PLSA,LDA) are the state-of-the-art approaches in topic modeling and\nmost recent research on topic generation has been focusing on improving or\nextending these models. However, results of traditional generative models are\nsensitive to the number of topics K, which must be specified manually. The\nproblem of generating topics from corpus resembles community detection in\nnetworks. Many effective algorithms can automatically detect communities from\nnetworks without a manually specified number of the communities. Inspired by\nthese algorithms, in this paper, we propose a novel method named Hierarchical\nLatent Semantic Mapping (HLSM), which automatically generates topics from\ncorpus. HLSM calculates the association between each pair of words in the\nlatent topic space, then constructs a unipartite network of words with this\nassociation and hierarchically generates topics from this network. We apply\nHLSM to several document collections and the experimental comparisons against\nseveral state-of-the-art approaches demonstrate the promising performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.03690v1",
            "date": "2015-11-11 21:30:10+00:00",
            "title": "Deep Multimodal Semantic Embeddings for Speech and Images",
            "authors": [
                "David Harwath",
                "James Glass"
            ],
            "abstract": "In this paper, we present a model which takes as input a corpus of images\nwith relevant spoken captions and finds a correspondence between the two\nmodalities. We employ a pair of convolutional neural networks to model visual\nobjects and speech signals at the word level, and tie the networks together\nwith an embedding and alignment model which learns a joint semantic space over\nboth modalities. We evaluate our model using image search and annotation tasks\non the Flickr8k dataset, which we augmented by collecting a corpus of 40,000\nspoken captions using Amazon Mechanical Turk.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.03745v4",
            "date": "2015-11-12 01:13:47+00:00",
            "title": "Grounding of Textual Phrases in Images by Reconstruction",
            "authors": [
                "Anna Rohrbach",
                "Marcus Rohrbach",
                "Ronghang Hu",
                "Trevor Darrell",
                "Bernt Schiele"
            ],
            "abstract": "Grounding (i.e. localizing) arbitrary, free-form textual phrases in visual\ncontent is a challenging problem with many applications for human-computer\ninteraction and image-text reference resolution. Few datasets provide the\nground truth spatial localization of phrases, thus it is desirable to learn\nfrom data with no or little grounding supervision. We propose a novel approach\nwhich learns grounding by reconstructing a given phrase using an attention\nmechanism, which can be either latent or optimized directly. During training\nour approach encodes the phrase using a recurrent network language model and\nthen learns to attend to the relevant image region in order to reconstruct the\ninput phrase. At test time, the correct attention, i.e., the grounding, is\nevaluated. If grounding supervision is available it can be directly applied via\na loss over the attention mechanism. We demonstrate the effectiveness of our\napproach on the Flickr 30k Entities and ReferItGame datasets with different\nlevels of supervision, ranging from no supervision over partial supervision to\nfull supervision. Our supervised variant improves by a large margin over the\nstate-of-the-art on both datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.03962v4",
            "date": "2015-11-12 16:53:50+00:00",
            "title": "Document Context Language Models",
            "authors": [
                "Yangfeng Ji",
                "Trevor Cohn",
                "Lingpeng Kong",
                "Chris Dyer",
                "Jacob Eisenstein"
            ],
            "abstract": "Text documents are structured on multiple levels of detail: individual words\nare related by syntax, but larger units of text are related by discourse\nstructure. Existing language models generally fail to account for discourse\nstructure, but it is crucial if we are to have language models that reward\ncoherence and generate coherent texts. We present and empirically evaluate a\nset of multi-level recurrent neural network language models, called\nDocument-Context Language Models (DCLM), which incorporate contextual\ninformation both within and beyond the sentence. In comparison with word-level\nrecurrent neural network language models, the DCLM models obtain slightly\nbetter predictive likelihoods, and considerably better assessments of document\ncoherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.04401v5",
            "date": "2015-11-13 18:59:36+00:00",
            "title": "Symbol Grounding Association in Multimodal Sequences with Missing Elements",
            "authors": [
                "Federico Raue",
                "Andreas Dengel",
                "Thomas M. Breuel",
                "Marcus Liwicki"
            ],
            "abstract": "In this paper, we extend a symbolic association framework for being able to\nhandle missing elements in multimodal sequences. The general scope of the work\nis the symbolic associations of object-word mappings as it happens in language\ndevelopment in infants. In other words, two different representations of the\nsame abstract concepts can associate in both directions. This scenario has been\nlong interested in Artificial Intelligence, Psychology, and Neuroscience. In\nthis work, we extend a recent approach for multimodal sequences (visual and\naudio) to also cope with missing elements in one or both modalities. Our method\nuses two parallel Long Short-Term Memories (LSTMs) with a learning rule based\non EM-algorithm. It aligns both LSTM outputs via Dynamic Time Warping (DTW). We\npropose to include an extra step for the combination with the max operation for\nexploiting the common elements between both sequences. The motivation behind is\nthat the combination acts as a condition selector for choosing the best\nrepresentation from both LSTMs. We evaluated the proposed extension in the\nfollowing scenarios: missing elements in one modality (visual or audio) and\nmissing elements in both modalities (visual and sound). The performance of our\nextension reaches better results than the original model and similar results to\nindividual LSTM trained in each modality.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.04590v5",
            "date": "2015-11-14 18:02:39+00:00",
            "title": "Oracle performance for visual captioning",
            "authors": [
                "Li Yao",
                "Nicolas Ballas",
                "Kyunghyun Cho",
                "John R. Smith",
                "Yoshua Bengio"
            ],
            "abstract": "The task of associating images and videos with a natural language description\nhas attracted a great amount of attention recently. Rapid progress has been\nmade in terms of both developing novel algorithms and releasing new datasets.\nIndeed, the state-of-the-art results on some of the standard datasets have been\npushed into the regime where it has become more and more difficult to make\nsignificant improvements. Instead of proposing new models, this work\ninvestigates the possibility of empirically establishing performance upper\nbounds on various visual captioning datasets without extra data labelling\neffort or human evaluation. In particular, it is assumed that visual captioning\nis decomposed into two steps: from visual inputs to visual concepts, and from\nvisual concepts to natural language descriptions. One would be able to obtain\nan upper bound when assuming the first step is perfect and only requiring\ntraining a conditional language model for the second step. We demonstrate the\nconstruction of such bounds on MS-COCO, YouTube2Text and LSMDC (a combination\nof M-VAD and MPII-MD). Surprisingly, despite of the imperfect process we used\nfor visual concept extraction in the first step and the simplicity of the\nlanguage model for the second step, we show that current state-of-the-art\nmodels fall short when being compared with the learned upper bounds.\nFurthermore, with such a bound, we quantify several important factors\nconcerning image and video captioning: the number of visual concepts captured\nby different models, the trade-off between the amount of visual elements\ncaptured and their accuracy, and the intrinsic difficulty and blessing of\ndifferent datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.04636v5",
            "date": "2015-11-14 23:30:39+00:00",
            "title": "Deep Reinforcement Learning with a Natural Language Action Space",
            "authors": [
                "Ji He",
                "Jianshu Chen",
                "Xiaodong He",
                "Jianfeng Gao",
                "Lihong Li",
                "Li Deng",
                "Mari Ostendorf"
            ],
            "abstract": "This paper introduces a novel architecture for reinforcement learning with\ndeep neural networks designed to handle state and action spaces characterized\nby natural language, as found in text-based games. Termed a deep reinforcement\nrelevance network (DRRN), the architecture represents action and state spaces\nwith separate embedding vectors, which are combined with an interaction\nfunction to approximate the Q-function in reinforcement learning. We evaluate\nthe DRRN on two popular text games, showing superior performance over other\ndeep Q-learning architectures. Experiments with paraphrased action descriptions\nshow that the model is extracting meaning rather than simply memorizing strings\nof text.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.04834v3",
            "date": "2015-11-16 06:03:58+00:00",
            "title": "Neural Programmer: Inducing Latent Programs with Gradient Descent",
            "authors": [
                "Arvind Neelakantan",
                "Quoc V. Le",
                "Ilya Sutskever"
            ],
            "abstract": "Deep neural networks have achieved impressive supervised classification\nperformance in many tasks including image recognition, speech recognition, and\nsequence to sequence learning. However, this success has not been translated to\napplications like question answering that may involve complex arithmetic and\nlogic reasoning. A major limitation of these models is in their inability to\nlearn even simple arithmetic and logic operations. For example, it has been\nshown that neural networks fail to learn to add two binary numbers reliably. In\nthis work, we propose Neural Programmer, an end-to-end differentiable neural\nnetwork augmented with a small set of basic arithmetic and logic operations.\nNeural Programmer can call these augmented operations over several steps,\nthereby inducing compositional programs that are more complex than the built-in\noperations. The model learns from a weak supervision signal which is the result\nof execution of the correct program, hence it does not require expensive\nannotation of the correct program itself. The decisions of what operations to\ncall, and what data segments to apply to are inferred by Neural Programmer.\nSuch decisions, during training, are done in a differentiable fashion so that\nthe entire network can be trained jointly by gradient descent. We find that\ntraining the model is difficult, but it can be greatly improved by adding\nrandom noise to the gradient. On a fairly complex synthetic table-comprehension\ndataset, traditional recurrent networks and attentional models perform poorly\nwhile Neural Programmer typically obtains nearly perfect accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.04868v4",
            "date": "2015-11-16 08:53:44+00:00",
            "title": "A Neural Transducer",
            "authors": [
                "Navdeep Jaitly",
                "David Sussillo",
                "Quoc V. Le",
                "Oriol Vinyals",
                "Ilya Sutskever",
                "Samy Bengio"
            ],
            "abstract": "Sequence-to-sequence models have achieved impressive results on various\ntasks. However, they are unsuitable for tasks that require incremental\npredictions to be made as more data arrives or tasks that have long input\nsequences and output sequences. This is because they generate an output\nsequence conditioned on an entire input sequence. In this paper, we present a\nNeural Transducer that can make incremental predictions as more input arrives,\nwithout redoing the entire computation. Unlike sequence-to-sequence models, the\nNeural Transducer computes the next-step distribution conditioned on the\npartially observed input sequence and the partially generated sequence. At each\ntime step, the transducer can decide to emit zero to many output symbols. The\ndata can be processed using an encoder and presented as input to the\ntransducer. The discrete decision to emit a symbol at every time step makes it\ndifficult to learn with conventional backpropagation. It is however possible to\ntrain the transducer by using a dynamic programming algorithm to generate\ntarget discrete decisions. Our experiments show that the Neural Transducer\nworks well in settings where it is required to produce output predictions as\ndata come in. We also find that the Neural Transducer performs well for long\nsequences even when attention mechanisms are not used.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.04891v4",
            "date": "2015-11-16 09:56:04+00:00",
            "title": "Sherlock: Scalable Fact Learning in Images",
            "authors": [
                "Mohamed Elhoseiny",
                "Scott Cohen",
                "Walter Chang",
                "Brian Price",
                "Ahmed Elgammal"
            ],
            "abstract": "We study scalable and uniform understanding of facts in images. Existing\nvisual recognition systems are typically modeled differently for each fact type\nsuch as objects, actions, and interactions. We propose a setting where all\nthese facts can be modeled simultaneously with a capacity to understand\nunbounded number of facts in a structured way. The training data comes as\nstructured facts in images, including (1) objects (e.g., $<$boy$>$), (2)\nattributes (e.g., $<$boy, tall$>$), (3) actions (e.g., $<$boy, playing$>$), and\n(4) interactions (e.g., $<$boy, riding, a horse $>$). Each fact has a semantic\nlanguage view (e.g., $<$ boy, playing$>$) and a visual view (an image with this\nfact). We show that learning visual facts in a structured way enables not only\na uniform but also generalizable visual understanding. We propose and\ninvestigate recent and strong approaches from the multiview learning literature\nand also introduce two learning representation models as potential baselines.\nWe applied the investigated methods on several datasets that we augmented with\nstructured facts and a large scale dataset of more than 202,000 facts and\n814,000 images. Our experiments show the advantage of relating facts by the\nstructure by the proposed models compared to the designed baselines on\nbidirectional fact retrieval.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.05099v5",
            "date": "2015-11-16 19:38:14+00:00",
            "title": "Yin and Yang: Balancing and Answering Binary Visual Questions",
            "authors": [
                "Peng Zhang",
                "Yash Goyal",
                "Douglas Summers-Stay",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "The complex compositional structure of language makes problems at the\nintersection of vision and language challenging. But language also provides a\nstrong prior that can result in good superficial performance, without the\nunderlying models truly understanding the visual content. This can hinder\nprogress in pushing state of art in the computer vision aspects of multi-modal\nAI. In this paper, we address binary Visual Question Answering (VQA) on\nabstract scenes. We formulate this problem as visual verification of concepts\ninquired in the questions. Specifically, we convert the question to a tuple\nthat concisely summarizes the visual concept to be detected in the image. If\nthe concept can be found in the image, the answer to the question is \"yes\", and\notherwise \"no\". Abstract scenes play two roles (1) They allow us to focus on\nthe high-level semantics of the VQA task as opposed to the low-level\nrecognition problems, and perhaps more importantly, (2) They provide us the\nmodality to balance the dataset such that language priors are controlled, and\nthe role of vision is essential. In particular, we collect fine-grained pairs\nof scenes for every question, such that the answer to the question is \"yes\" for\none scene, and \"no\" for the other for the exact same question. Indeed, language\npriors alone do not perform better than chance on our balanced dataset.\nMoreover, our proposed approach matches the performance of a state-of-the-art\nVQA approach on the unbalanced dataset, and outperforms it on the balanced\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.05234v2",
            "date": "2015-11-17 01:00:04+00:00",
            "title": "Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering",
            "authors": [
                "Huijuan Xu",
                "Kate Saenko"
            ],
            "abstract": "We address the problem of Visual Question Answering (VQA), which requires\njoint image and language understanding to answer a question about a given\nphotograph. Recent approaches have applied deep image captioning methods based\non convolutional-recurrent networks to this problem, but have failed to model\nspatial inference. To remedy this, we propose a model we call the Spatial\nMemory Network and apply it to the VQA task. Memory networks are recurrent\nneural networks with an explicit attention mechanism that selects certain parts\nof the information stored in memory. Our Spatial Memory Network stores neuron\nactivations from different spatial regions of the image in its memory, and uses\nthe question to choose relevant regions for computing the answer, a process of\nwhich constitutes a single \"hop\" in the network. We propose a novel spatial\nattention architecture that aligns words with image patches in the first hop,\nand obtain improved results by adding a second attention hop which considers\nthe whole question to choose visual evidence based on the results of the first\nhop. To better understand the inference process learned by the network, we\ndesign synthetic questions that specifically require spatial inference and\nvisualize the attention weights. We evaluate our model on two published visual\nquestion answering datasets, DAQUAR [1] and VQA [2], and obtain improved\nresults compared to a strong deep baseline model (iBOWIMG) which concatenates\nimage and question features to predict the answer [3].",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.05392v3",
            "date": "2015-11-17 13:28:55+00:00",
            "title": "Learning the Dimensionality of Word Embeddings",
            "authors": [
                "Eric Nalisnick",
                "Sachin Ravi"
            ],
            "abstract": "We describe a method for learning word embeddings with data-dependent\ndimensionality. Our Stochastic Dimensionality Skip-Gram (SD-SG) and Stochastic\nDimensionality Continuous Bag-of-Words (SD-CBOW) are nonparametric analogs of\nMikolov et al.'s (2013) well-known 'word2vec' models. Vector dimensionality is\nmade dynamic by employing techniques used by Cote & Larochelle (2016) to define\nan RBM with an infinite number of hidden units. We show qualitatively and\nquantitatively that SD-SG and SD-CBOW are competitive with their\nfixed-dimension counterparts while providing a distribution over embedding\ndimensionalities, which offers a window into how semantics distribute across\ndimensions.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1511.05526v2",
            "date": "2015-11-17 19:55:34+00:00",
            "title": "Learning Articulated Motion Models from Visual and Lingual Signals",
            "authors": [
                "Zhengyang Wu",
                "Mohit Bansal",
                "Matthew R. Walter"
            ],
            "abstract": "In order for robots to operate effectively in homes and workplaces, they must\nbe able to manipulate the articulated objects common within environments built\nfor and by humans. Previous work learns kinematic models that prescribe this\nmanipulation from visual demonstrations. Lingual signals, such as natural\nlanguage descriptions and instructions, offer a complementary means of\nconveying knowledge of such manipulation models and are suitable to a wide\nrange of interactions (e.g., remote manipulation). In this paper, we present a\nmultimodal learning framework that incorporates both visual and lingual\ninformation to estimate the structure and parameters that define kinematic\nmodels of articulated objects. The visual signal takes the form of an RGB-D\nimage stream that opportunistically captures object motion in an unprepared\nscene. Accompanying natural language descriptions of the motion constitute the\nlingual signal. We present a probabilistic language model that uses word\nembeddings to associate lingual verbs with their corresponding kinematic\nstructures. By exploiting the complementary nature of the visual and lingual\ninput, our method infers correct kinematic structures for various multiple-part\nobjects on which the previous state-of-the-art, visual-only system fails. We\nevaluate our multimodal learning framework on a dataset comprised of a variety\nof household objects, and demonstrate a 36% improvement in model accuracy over\nthe vision-only baseline.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1511.05756v1",
            "date": "2015-11-18 12:30:57+00:00",
            "title": "Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction",
            "authors": [
                "Hyeonwoo Noh",
                "Paul Hongsuck Seo",
                "Bohyung Han"
            ],
            "abstract": "We tackle image question answering (ImageQA) problem by learning a\nconvolutional neural network (CNN) with a dynamic parameter layer whose weights\nare determined adaptively based on questions. For the adaptive parameter\nprediction, we employ a separate parameter prediction network, which consists\nof gated recurrent unit (GRU) taking a question as its input and a\nfully-connected layer generating a set of candidate weights as its output.\nHowever, it is challenging to construct a parameter prediction network for a\nlarge number of parameters in the fully-connected dynamic parameter layer of\nthe CNN. We reduce the complexity of this problem by incorporating a hashing\ntechnique, where the candidate weights given by the parameter prediction\nnetwork are selected using a predefined hash function to determine individual\nweights in the dynamic parameter layer. The proposed network---joint network\nwith the CNN for ImageQA and the parameter prediction network---is trained\nend-to-end through back-propagation, where its weights are initialized using a\npre-trained CNN and GRU. The proposed algorithm illustrates the\nstate-of-the-art performance on all available public ImageQA benchmarks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.06038v4",
            "date": "2015-11-19 01:23:28+00:00",
            "title": "Neural Variational Inference for Text Processing",
            "authors": [
                "Yishu Miao",
                "Lei Yu",
                "Phil Blunsom"
            ],
            "abstract": "Recent advances in neural variational inference have spawned a renaissance in\ndeep latent variable models. In this paper we introduce a generic variational\ninference framework for generative and conditional models of text. While\ntraditional variational methods derive an analytic approximation for the\nintractable distributions over latent variables, here we construct an inference\nnetwork conditioned on the discrete text input to provide the variational\ndistribution. We validate this framework on two very different text modelling\napplications, generative document modelling and supervised question answering.\nOur neural variational document model combines a continuous stochastic document\nrepresentation with a bag-of-words generative model and achieves the lowest\nreported perplexities on two standard test corpora. The neural answer selection\nmodel employs a stochastic representation layer within an attention mechanism\nto extract the semantics between a question and answer pair. On two question\nanswering benchmarks this model exceeds all previous published benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06052v4",
            "date": "2015-11-19 03:54:15+00:00",
            "title": "Overcoming Language Variation in Sentiment Analysis with Social Attention",
            "authors": [
                "Yi Yang",
                "Jacob Eisenstein"
            ],
            "abstract": "Variation in language is ubiquitous, particularly in newer forms of writing\nsuch as social media. Fortunately, variation is not random, it is often linked\nto social properties of the author. In this paper, we show how to exploit\nsocial networks to make sentiment analysis more robust to social language\nvariation. The key idea is linguistic homophily: the tendency of socially\nlinked individuals to use language in similar ways. We formalize this idea in a\nnovel attention-based neural network architecture, in which attention is\ndivided among several basis models, depending on the author's position in the\nsocial network. This has the effect of smoothing the classification function\nacross the social network, and makes it possible to induce personalized\nclassifiers even for authors for whom there is no labeled data or demographic\nmetadata. This model significantly improves the accuracies of sentiment\nanalysis on Twitter and on review data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06078v2",
            "date": "2015-11-19 07:17:49+00:00",
            "title": "Learning Deep Structure-Preserving Image-Text Embeddings",
            "authors": [
                "Liwei Wang",
                "Yin Li",
                "Svetlana Lazebnik"
            ],
            "abstract": "This paper proposes a method for learning joint embeddings of images and text\nusing a two-branch neural network with multiple layers of linear projections\nfollowed by nonlinearities. The network is trained using a large margin\nobjective that combines cross-view ranking constraints with within-view\nneighborhood structure preservation constraints inspired by metric learning\nliterature. Extensive experiments show that our approach gains significant\nimprovements in accuracy for image-to-text and text-to-image retrieval. Our\nmethod achieves new state-of-the-art results on the Flickr30K and MSCOCO\nimage-sentence datasets and shows promise on the new task of phrase\nlocalization on the Flickr30K Entities dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1511.06114v4",
            "date": "2015-11-19 10:24:14+00:00",
            "title": "Multi-task Sequence to Sequence Learning",
            "authors": [
                "Minh-Thang Luong",
                "Quoc V. Le",
                "Ilya Sutskever",
                "Oriol Vinyals",
                "Lukasz Kaiser"
            ],
            "abstract": "Sequence to sequence learning has recently emerged as a new paradigm in\nsupervised learning. To date, most of its applications focused on only one task\nand not much work explored this framework for multiple tasks. This paper\nexamines three multi-task learning (MTL) settings for sequence to sequence\nmodels: (a) the oneto-many setting - where the encoder is shared between\nseveral tasks such as machine translation and syntactic parsing, (b) the\nmany-to-one setting - useful when only the decoder can be shared, as in the\ncase of translation and image caption generation, and (c) the many-to-many\nsetting - where multiple encoders and decoders are shared, which is the case\nwith unsupervised objectives and translation. Our results show that training on\na small amount of parsing and image caption data can improve the translation\nquality between English and German by up to 1.5 BLEU points over strong\nsingle-task baselines on the WMT benchmarks. Furthermore, we have established a\nnew state-of-the-art result in constituent parsing with 93.0 F1. Lastly, we\nreveal interesting properties of the two unsupervised learning objectives,\nautoencoder and skip-thought, in the MTL context: autoencoder helps less in\nterms of perplexities but more on BLEU scores compared to skip-thought.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06361v6",
            "date": "2015-11-19 20:56:14+00:00",
            "title": "Order-Embeddings of Images and Language",
            "authors": [
                "Ivan Vendrov",
                "Ryan Kiros",
                "Sanja Fidler",
                "Raquel Urtasun"
            ],
            "abstract": "Hypernymy, textual entailment, and image captioning can be seen as special\ncases of a single visual-semantic hierarchy over words, sentences, and images.\nIn this paper we advocate for explicitly modeling the partial order structure\nof this hierarchy. Towards this goal, we introduce a general method for\nlearning ordered representations, and show how it can be applied to a variety\nof tasks involving images and language. We show that the resulting\nrepresentations improve performance over current approaches for hypernym\nprediction and image-caption retrieval.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.06391v4",
            "date": "2015-11-19 21:31:26+00:00",
            "title": "Order Matters: Sequence to sequence for sets",
            "authors": [
                "Oriol Vinyals",
                "Samy Bengio",
                "Manjunath Kudlur"
            ],
            "abstract": "Sequences have become first class citizens in supervised learning thanks to\nthe resurgence of recurrent neural networks. Many complex tasks that require\nmapping from or to a sequence of observations can now be formulated with the\nsequence-to-sequence (seq2seq) framework which employs the chain rule to\nefficiently represent the joint probability of sequences. In many cases,\nhowever, variable sized inputs and/or outputs might not be naturally expressed\nas sequences. For instance, it is not clear how to input a set of numbers into\na model where the task is to sort them; similarly, we do not know how to\norganize outputs when they correspond to random variables and the task is to\nmodel their unknown joint probability. In this paper, we first show using\nvarious examples that the order in which we organize input and/or output data\nmatters significantly when learning an underlying model. We then discuss an\nextension of the seq2seq framework that goes beyond sequences and handles input\nsets in a principled way. In addition, we propose a loss which, by searching\nover possible orders during training, deals with the lack of structure of\noutput sets. We show empirical evidence of our claims regarding ordering, and\non the modifications to the seq2seq framework on benchmark language modeling\nand parsing tasks, as well as two artificial tasks -- sorting numbers and\nestimating the joint probability of unknown graphical models.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1511.06420v2",
            "date": "2015-11-19 22:15:46+00:00",
            "title": "Skip-Thought Memory Networks",
            "authors": [
                "Ethan Caballero"
            ],
            "abstract": "Question Answering (QA) is fundamental to natural language processing in that\nmost nlp problems can be phrased as QA (Kumar et al., 2015). Current weakly\nsupervised memory network models that have been proposed so far struggle at\nanswering questions that involve relations among multiple entities (such as\nfacebook's bAbi qa5-three-arg-relations in (Weston et al., 2015)). To address\nthis problem of learning multi-argument multi-hop semantic relations for the\npurpose of QA, we propose a method that combines the jointly learned long-term\nread-write memory and attentive inference components of end-to-end memory\nnetworks (MemN2N) (Sukhbaatar et al., 2015) with distributed sentence vector\nrepresentations encoded by a Skip-Thought model (Kiros et al., 2015). This\nchoice to append Skip-Thought Vectors to the existing MemN2N framework is\nmotivated by the fact that Skip-Thought Vectors have been shown to accurately\nmodel multi-argument semantic relations (Kiros et al., 2015).",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1511.06798v2",
            "date": "2015-11-20 23:39:48+00:00",
            "title": "Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability",
            "authors": [
                "Luke Miratrix",
                "Robin Ackerman"
            ],
            "abstract": "We propose a general framework for topic-specific summarization of large text\ncorpora, and illustrate how it can be used for analysis in two quite different\ncontexts: an OSHA database of fatality and catastrophe reports (to facilitate\nsurveillance for patterns in circumstances leading to injury or death) and\nlegal decisions on workers' compensation claims (to explore relevant case law).\nOur summarization framework, built on sparse classification methods, is a\ncompromise between simple word frequency based methods currently in wide use,\nand more heavyweight, model-intensive methods such as Latent Dirichlet\nAllocation (LDA). For a particular topic of interest (e.g., mental health\ndisability, or chemical reactions), we regress a labeling of documents onto the\nhigh-dimensional counts of all the other words and phrases in the documents.\nThe resulting small set of phrases found as predictive are then harvested as\nthe summary. Using a branch-and-bound approach, this method can be extended to\nallow for phrases of arbitrary length, which allows for potentially rich\nsummarization. We discuss how focus on the purpose of the summaries can inform\nchoices of regularization parameters and model constraints. We evaluate this\ntool by comparing computational time and summary statistics of the resulting\nword lists to three other methods in the literature. We also present a new R\npackage, textreg. Overall, we argue that sparse methods have much to offer text\nanalysis, and is a branch of research that should be considered further in this\ncontext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.06909v7",
            "date": "2015-11-21 17:49:30+00:00",
            "title": "BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies",
            "authors": [
                "Shihao Ji",
                "S. V. N. Vishwanathan",
                "Nadathur Satish",
                "Michael J. Anderson",
                "Pradeep Dubey"
            ],
            "abstract": "We propose BlackOut, an approximation algorithm to efficiently train massive\nrecurrent neural network language models (RNNLMs) with million word\nvocabularies. BlackOut is motivated by using a discriminative loss, and we\ndescribe a new sampling strategy which significantly reduces computation while\nimproving stability, sample efficiency, and rate of convergence. One way to\nunderstand BlackOut is to view it as an extension of the DropOut strategy to\nthe output layer, wherein we use a discriminative training loss and a weighted\nsampling scheme. We also establish close connections between BlackOut,\nimportance sampling, and noise contrastive estimation (NCE). Our experiments,\non the recently released one billion word language modeling benchmark,\ndemonstrate scalability and accuracy of BlackOut; we outperform the\nstate-of-the art, and achieve the lowest perplexity scores on this dataset.\nMoreover, unlike other established methods which typically require GPUs or CPU\nclusters, we show that a carefully implemented version of BlackOut requires\nonly 1-10 days on a single machine to train a RNNLM with a million word\nvocabulary and billions of parameters on one billion words. Although we\ndescribe BlackOut in the context of RNNLM training, it can be used to any\nnetworks with large softmax output layers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1511.07001v1",
            "date": "2015-11-22 12:06:34+00:00",
            "title": "Analysis of a Play by Means of CHAPLIN, the Characters and Places Interaction Network Software",
            "authors": [
                "A. C. Sparavigna",
                "R. Marazzato"
            ],
            "abstract": "Recently, we have developed a software able of gathering information on\nsocial networks from written texts. This software, the CHAracters and PLaces\nInteraction Network (CHAPLIN) tool, is implemented in Visual Basic. By means of\nit, characters and places of a literary work can be extracted from a list of\nraw words. The software interface helps users to select their names out of this\nlist. Setting some parameters, CHAPLIN creates a network where nodes represent\ncharacters/places and edges give their interactions. Nodes and edges are\nlabelled by performances. In this paper, we propose to use CHAPLIN for the\nanalysis a William Shakespeare's play, the famous 'Tragedy of Hamlet, Prince of\nDenmark'. Performances of characters in the play as a whole and in each act of\nit are given by graphs.",
            "journal ref": "International Journal of Sciences, 2015, 4(3):60-68",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1511.07607v2",
            "date": "2015-11-24 08:34:20+00:00",
            "title": "Fine-Grain Annotation of Cricket Videos",
            "authors": [
                "Rahul Anand Sharma",
                "Pramod Sankar K",
                "CV Jawahar"
            ],
            "abstract": "The recognition of human activities is one of the key problems in video\nunderstanding. Action recognition is challenging even for specific categories\nof videos, such as sports, that contain only a small set of actions.\nInterestingly, sports videos are accompanied by detailed commentaries available\nonline, which could be used to perform action annotation in a weakly-supervised\nsetting. For the specific case of Cricket videos, we address the challenge of\ntemporal segmentation and annotation of ctions with semantic descriptions. Our\nsolution consists of two stages. In the first stage, the video is segmented\ninto \"scenes\", by utilizing the scene category information extracted from\ntext-commentary. The second stage consists of classifying video-shots as well\nas the phrases in the textual description into various categories. The relevant\nphrases are then suitably mapped to the video-shots. The novel aspect of this\nwork is the fine temporal scale at which semantic information is assigned to\nthe video. As a result of our approach, we enable retrieval of specific actions\nthat last only a few seconds, from several hours of video. This solution yields\na large number of labeled exemplars, with no manual effort, that could be used\nby machine learning algorithms to learn complex actions.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1511.07972v9",
            "date": "2015-11-25 07:06:09+00:00",
            "title": "Learning with Memory Embeddings",
            "authors": [
                "Volker Tresp",
                "Crist\u00f3bal Esteban",
                "Yinchong Yang",
                "Stephan Baier",
                "Denis Krompa\u00df"
            ],
            "abstract": "Embedding learning, a.k.a. representation learning, has been shown to be able\nto model large-scale semantic knowledge graphs. A key concept is a mapping of\nthe knowledge graph to a tensor representation whose entries are predicted by\nmodels using latent representations of generalized entities. Latent variable\nmodels are well suited to deal with the high dimensionality and sparsity of\ntypical knowledge graphs. In recent publications the embedding models were\nextended to also consider time evolutions, time patterns and subsymbolic\nrepresentations. In this paper we map embedding models, which were developed\npurely as solutions to technical problems for modelling temporal knowledge\ngraphs, to various cognitive memory functions, in particular to semantic and\nconcept memory, episodic memory, sensory memory, short-term memory, and working\nmemory. We discuss learning, query answering, the path from sensory input to\nsemantic decoding, and the relationship between episodic memory and semantic\nmemory. We introduce a number of hypotheses on human memory that can be derived\nfrom the developed mathematical models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.08277v1",
            "date": "2015-11-26 02:57:54+00:00",
            "title": "A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations",
            "authors": [
                "Shengxian Wan",
                "Yanyan Lan",
                "Jiafeng Guo",
                "Jun Xu",
                "Liang Pang",
                "Xueqi Cheng"
            ],
            "abstract": "Matching natural language sentences is central for many applications such as\ninformation retrieval and question answering. Existing deep models rely on a\nsingle sentence representation or multiple granularity representations for\nmatching. However, such methods cannot well capture the contextualized local\ninformation in the matching process. To tackle this problem, we present a new\ndeep architecture to match two sentences with multiple positional sentence\nrepresentations. Specifically, each positional sentence representation is a\nsentence representation at this position, generated by a bidirectional long\nshort term memory (Bi-LSTM). The matching score is finally produced by\naggregating interactions between these different positional sentence\nrepresentations, through $k$-Max pooling and a multi-layer perceptron. Our\nmodel has several advantages: (1) By using Bi-LSTM, rich context of the whole\nsentence is leveraged to capture the contextualized local information in each\npositional sentence representation; (2) By matching with multiple positional\nsentence representations, it is flexible to aggregate different important\ncontextualized local information in a sentence to support the matching; (3)\nExperiments on different tasks such as question answering and sentence\ncompletion demonstrate the superiority of our model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.08299v1",
            "date": "2015-11-26 06:57:06+00:00",
            "title": "Hierarchical classification of e-commerce related social media",
            "authors": [
                "Matthew Long",
                "Aditya Jami",
                "Ashutosh Saxena"
            ],
            "abstract": "In this paper, we attempt to classify tweets into root categories of the\nAmazon browse node hierarchy using a set of tweets with browse node ID labels,\na much larger set of tweets without labels, and a set of Amazon reviews.\nExamining twitter data presents unique challenges in that the samples are short\n(under 140 characters) and often contain misspellings or abbreviations that are\ntrivial for a human to decipher but difficult for a computer to parse. A\nvariety of query and document expansion techniques are implemented in an effort\nto improve information retrieval to modest success.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1511.08308v5",
            "date": "2015-11-26 07:40:33+00:00",
            "title": "Named Entity Recognition with Bidirectional LSTM-CNNs",
            "authors": [
                "Jason P. C. Chiu",
                "Eric Nichols"
            ],
            "abstract": "Named entity recognition is a challenging task that has traditionally\nrequired large amounts of knowledge in the form of feature engineering and\nlexicons to achieve high performance. In this paper, we present a novel neural\nnetwork architecture that automatically detects word- and character-level\nfeatures using a hybrid bidirectional LSTM and CNN architecture, eliminating\nthe need for most feature engineering. We also propose a novel method of\nencoding partial lexicon matches in neural networks and compare it to existing\napproaches. Extensive evaluation shows that, given only tokenized text and\npublicly available word embeddings, our system is competitive on the CoNLL-2003\ndataset and surpasses the previously reported state of the art performance on\nthe OntoNotes 5.0 dataset by 2.13 F1 points. By using two lexicons constructed\nfrom publicly-available sources, we establish new state of the art performance\nwith an F1 score of 91.62 on CoNLL-2003 and 86.28 on OntoNotes, surpassing\nsystems that employ heavy feature engineering, proprietary lexicons, and rich\nentity linking information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.08400v7",
            "date": "2015-11-26 14:35:27+00:00",
            "title": "Regularizing RNNs by Stabilizing Activations",
            "authors": [
                "David Krueger",
                "Roland Memisevic"
            ],
            "abstract": "We stabilize the activations of Recurrent Neural Networks (RNNs) by\npenalizing the squared distance between successive hidden states' norms.\n  This penalty term is an effective regularizer for RNNs including LSTMs and\nIRNNs, improving performance on character-level language modeling and phoneme\nrecognition, and outperforming weight noise and dropout.\n  We achieve competitive performance (18.6\\% PER) on the TIMIT phoneme\nrecognition task for RNNs evaluated without beam search or an RNN transducer.\n  With this penalty term, IRNN can achieve similar performance to LSTM on\nlanguage modeling, although adding the penalty term to the LSTM results in\nsuperior performance.\n  Our penalty term also prevents the exponential growth of IRNN's activations\noutside of their training horizon, allowing them to generalize to much longer\nsequences.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1511.08855v2",
            "date": "2015-11-28 00:13:09+00:00",
            "title": "Semantic Folding Theory And its Application in Semantic Fingerprinting",
            "authors": [
                "Francisco De Sousa Webber"
            ],
            "abstract": "Human language is recognized as a very complex domain since decades. No\ncomputer system has been able to reach human levels of performance so far. The\nonly known computational system capable of proper language processing is the\nhuman brain. While we gather more and more data about the brain, its\nfundamental computational processes still remain obscure. The lack of a sound\ncomputational brain theory also prevents the fundamental understanding of\nNatural Language Processing. As always when science lacks a theoretical\nfoundation, statistical modeling is applied to accommodate as many sampled\nreal-world data as possible. An unsolved fundamental issue is the actual\nrepresentation of language (data) within the brain, denoted as the\nRepresentational Problem. Starting with Jeff Hawkins' Hierarchical Temporal\nMemory (HTM) theory, a consistent computational theory of the human cortex, we\nhave developed a corresponding theory of language data representation: The\nSemantic Folding Theory. The process of encoding words, by using a topographic\nsemantic space as distributional reference frame into a sparse binary\nrepresentational vector is called Semantic Folding and is the central topic of\nthis document. Semantic Folding describes a method of converting language from\nits symbolic representation (text) into an explicit, semantically grounded\nrepresentation that can be generically processed by Hawkins' HTM networks. As\nit turned out, this change in representation, by itself, can solve many complex\nNLP problems by applying Boolean operators and a generic similarity function\nlike the Euclidian Distance. Many practical problems of statistical NLP\nsystems, like the high cost of computation, the fundamental incongruity of\nprecision and recall , the complex tuning procedures etc., can be elegantly\novercome by applying Semantic Folding.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1511.09107v1",
            "date": "2015-11-29 22:41:43+00:00",
            "title": "Machine Learning Sentiment Prediction based on Hybrid Document Representation",
            "authors": [
                "Panagiotis Stalidis",
                "Maria Giatsoglou",
                "Konstantinos Diamantaras",
                "George Sarigiannidis",
                "Konstantinos Ch. Chatzisavvas"
            ],
            "abstract": "Automated sentiment analysis and opinion mining is a complex process\nconcerning the extraction of useful subjective information from text. The\nexplosion of user generated content on the Web, especially the fact that\nmillions of users, on a daily basis, express their opinions on products and\nservices to blogs, wikis, social networks, message boards, etc., render the\nreliable, automated export of sentiments and opinions from unstructured text\ncrucial for several commercial applications. In this paper, we present a novel\nhybrid vectorization approach for textual resources that combines a weighted\nvariant of the popular Word2Vec representation (based on Term Frequency-Inverse\nDocument Frequency) representation and with a Bag- of-Words representation and\na vector of lexicon-based sentiment values. The proposed text representation\napproach is assessed through the application of several machine learning\nclassification algorithms on a dataset that is used extensively in literature\nfor sentiment detection. The classification accuracy derived through the\nproposed hybrid vectorization approach is higher than when its individual\ncomponents are used for text represenation, and comparable with\nstate-of-the-art sentiment detection methodologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1511.09128v1",
            "date": "2015-11-30 01:46:15+00:00",
            "title": "Aspect-based Opinion Summarization with Convolutional Neural Networks",
            "authors": [
                "Haibing Wu",
                "Yiwei Gu",
                "Shangdi Sun",
                "Xiaodong Gu"
            ],
            "abstract": "This paper considers Aspect-based Opinion Summarization (AOS) of reviews on\nparticular products. To enable real applications, an AOS system needs to\naddress two core subtasks, aspect extraction and sentiment classification. Most\nexisting approaches to aspect extraction, which use linguistic analysis or\ntopic modeling, are general across different products but not precise enough or\nsuitable for particular products. Instead we take a less general but more\nprecise scheme, directly mapping each review sentence into pre-defined aspects.\nTo tackle aspect mapping and sentiment classification, we propose two\nConvolutional Neural Network (CNN) based methods, cascaded CNN and multitask\nCNN. Cascaded CNN contains two levels of convolutional networks. Multiple CNNs\nat level 1 deal with aspect mapping task, and a single CNN at level 2 deals\nwith sentiment classification. Multitask CNN also contains multiple aspect CNNs\nand a sentiment CNN, but different networks share the same word embeddings.\nExperimental results indicate that both cascaded and multitask CNNs outperform\nSVM-based methods by large margins. Multitask CNN generally performs better\nthan cascaded CNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00025v2",
            "date": "2015-12-31 22:23:34+00:00",
            "title": "Write a Classifier: Predicting Visual Classifiers from Unstructured Text",
            "authors": [
                "Mohamed Elhoseiny",
                "Ahmed Elgammal",
                "Babak Saleh"
            ],
            "abstract": "People typically learn through exposure to visual concepts associated with\nlinguistic descriptions. For instance, teaching visual object categories to\nchildren is often accompanied by descriptions in text or speech. In a machine\nlearning context, these observations motivates us to ask whether this learning\nprocess could be computationally modeled to learn visual classifiers. More\nspecifically, the main question of this work is how to utilize purely textual\ndescription of visual classes with no training images, to learn explicit visual\nclassifiers for them. We propose and investigate two baseline formulations,\nbased on regression and domain transfer, that predict a linear classifier.\nThen, we propose a new constrained optimization formulation that combines a\nregression function and a knowledge transfer function with additional\nconstraints to predict the parameters of a linear classifier. We also propose a\ngeneric kernelized models where a kernel classifier is predicted in the form\ndefined by the representer theorem. The kernelized models allow defining and\nutilizing any two RKHS (Reproducing Kernel Hilbert Space) kernel functions in\nthe visual space and text space, respectively. We finally propose a kernel\nfunction between unstructured text descriptions that builds on distributional\nsemantics, which shows an advantage in our setting and could be useful for\nother applications. We applied all the studied models to predict visual\nclassifiers on two fine-grained and challenging categorization datasets (CU\nBirds and Flower Datasets), and the results indicate successful predictions of\nour final model over several baselines that we designed.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1601.03478v1",
            "date": "2015-09-14 17:19:33+00:00",
            "title": "Deep Learning Applied to Image and Text Matching",
            "authors": [
                "Afroze Ibrahim Baqapuri"
            ],
            "abstract": "The ability to describe images with natural language sentences is the\nhallmark for image and language understanding. Such a system has wide ranging\napplications such as annotating images and using natural sentences to search\nfor images.In this project we focus on the task of bidirectional image\nretrieval: such asystem is capable of retrieving an image based on a sentence\n(image search) andretrieve sentence based on an image query (image annotation).\nWe present asystem based on a global ranking objective function which uses a\ncombinationof convolutional neural networks (CNN) and multi layer perceptrons\n(MLP).It takes a pair of image and sentence and processes them in different\nchannels,finally embedding it into a common multimodal vector space. These\nembeddingsencode abstract semantic information about the two inputs and can be\ncomparedusing traditional information retrieval approaches. For each such pair,\nthe modelreturns a score which is interpretted as a similarity metric. If this\nscore is high,the image and sentence are likely to convey similar meaning, and\nif the score is low then they are likely not to.\n  The visual input is modeled via deep convolutional neural network. On\ntheother hand we explore three models for the textual module. The first one\nisbag of words with an MLP. The second one uses n-grams (bigram, trigrams,and a\ncombination of trigram & skip-grams) with an MLP. The third is morespecialized\ndeep network specific for modeling variable length sequences (SSE).We report\ncomparable performance to recent work in the field, even though ouroverall\nmodel is simpler. We also show that the training time choice of how wecan\ngenerate our negative samples has a significant impact on performance, and can\nbe used to specialize the bi-directional system in one particular task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1502.00731v4",
            "date": "2015-02-03 04:16:24+00:00",
            "title": "Incremental Knowledge Base Construction Using DeepDive",
            "authors": [
                "Jaeho Shin",
                "Sen Wu",
                "Feiran Wang",
                "Christopher De Sa",
                "Ce Zhang",
                "Christopher R\u00e9"
            ],
            "abstract": "Populating a database with unstructured information is a long-standing\nproblem in industry and research that encompasses problems of extraction,\ncleaning, and integration. Recent names used for this problem include dealing\nwith dark data and knowledge base construction (KBC). In this work, we describe\nDeepDive, a system that combines database and machine learning ideas to help\ndevelop KBC systems, and we present techniques to make the KBC process more\nefficient. We observe that the KBC process is iterative, and we develop\ntechniques to incrementally produce inference results for KBC systems. We\npropose two methods for incremental inference, based respectively on sampling\nand variational techniques. We also study the tradeoff space of these methods\nand develop a simple rule-based optimizer. DeepDive includes all of these\ncontributions, and we evaluate DeepDive on five KBC systems, showing that it\ncan speed up KBC inference tasks by up to two orders of magnitude with\nnegligible impact on quality.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1502.00831v2",
            "date": "2015-02-03 12:16:19+00:00",
            "title": "Open System Categorical Quantum Semantics in Natural Language Processing",
            "authors": [
                "Robin Piedeleu",
                "Dimitri Kartsaklis",
                "Bob Coecke",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Originally inspired by categorical quantum mechanics (Abramsky and Coecke,\nLiCS'04), the categorical compositional distributional model of natural\nlanguage meaning of Coecke, Sadrzadeh and Clark provides a conceptually\nmotivated procedure to compute the meaning of a sentence, given its grammatical\nstructure within a Lambek pregroup and a vectorial representation of the\nmeaning of its parts. The predictions of this first model have outperformed\nthat of other models in mainstream empirical language processing tasks on large\nscale data. Moreover, just like CQM allows for varying the model in which we\ninterpret quantum axioms, one can also vary the model in which we interpret\nword meaning.\n  In this paper we show that further developments in categorical quantum\nmechanics are relevant to natural language processing too. Firstly, Selinger's\nCPM-construction allows for explicitly taking into account lexical ambiguity\nand distinguishing between the two inherently different notions of homonymy and\npolysemy. In terms of the model in which we interpret word meaning, this means\na passage from the vector space model to density matrices. Despite this change\nof model, standard empirical methods for comparing meanings can be easily\nadopted, which we demonstrate by a small-scale experiment on real-world data.\nThis experiment moreover provides preliminary evidence of the validity of our\nproposed new model for word meaning.\n  Secondly, commutative classical structures as well as their non-commutative\ncounterparts that arise in the image of the CPM-construction allow for encoding\nrelative pronouns, verbs and adjectives, and finally, iteration of the\nCPM-construction, something that has no counterpart in the quantum realm,\nenables one to accommodate both entailment and ambiguity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01682v1",
            "date": "2015-02-05 19:10:26+00:00",
            "title": "Use of Modality and Negation in Semantically-Informed Syntactic MT",
            "authors": [
                "Kathryn Baker",
                "Michael Bloodgood",
                "Bonnie J. Dorr",
                "Chris Callison-Burch",
                "Nathaniel W. Filardo",
                "Christine Piatko",
                "Lori Levin",
                "Scott Miller"
            ],
            "abstract": "This paper describes the resource- and system-building efforts of an\neight-week Johns Hopkins University Human Language Technology Center of\nExcellence Summer Camp for Applied Language Exploration (SCALE-2009) on\nSemantically-Informed Machine Translation (SIMT). We describe a new\nmodality/negation (MN) annotation scheme, the creation of a (publicly\navailable) MN lexicon, and two automated MN taggers that we built using the\nannotation scheme and lexicon. Our annotation scheme isolates three components\nof modality and negation: a trigger (a word that conveys modality or negation),\na target (an action associated with modality or negation) and a holder (an\nexperiencer of modality). We describe how our MN lexicon was semi-automatically\nproduced and we demonstrate that a structure-based MN tagger results in\nprecision around 86% (depending on genre) for tagging of a standard LDC data\nset.\n  We apply our MN annotation scheme to statistical machine translation using a\nsyntactic framework that supports the inclusion of semantic annotations.\nSyntactic tags enriched with semantic annotations are assigned to parse trees\nin the target-language training texts through a process of tree grafting. While\nthe focus of our work is modality and negation, the tree grafting procedure is\ngeneral and supports other types of semantic information. We exploit this\ncapability by including named entities, produced by a pre-existing tagger, in\naddition to the MN elements produced by the taggers described in this paper.\nThe resulting system significantly outperformed a linguistically naive baseline\nmodel (Hiero), and reached the highest scores yet reported on the NIST 2009\nUrdu-English test set. This finding supports the hypothesis that both syntactic\nand semantic information can improve translation quality.",
            "journal ref": "Computational Linguistics, 38(2):411-438, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.01753v1",
            "date": "2015-02-05 22:51:45+00:00",
            "title": "Monitoring Term Drift Based on Semantic Consistency in an Evolving Vector Field",
            "authors": [
                "Peter Wittek",
                "S\u00e1ndor Dar\u00e1nyi",
                "Efstratios Kontopoulos",
                "Theodoros Moysiadis",
                "Ioannis Kompatsiaris"
            ],
            "abstract": "Based on the Aristotelian concept of potentiality vs. actuality allowing for\nthe study of energy and dynamics in language, we propose a field approach to\nlexical analysis. Falling back on the distributional hypothesis to\nstatistically model word meaning, we used evolving fields as a metaphor to\nexpress time-dependent changes in a vector space model by a combination of\nrandom indexing and evolving self-organizing maps (ESOM). To monitor semantic\ndrifts within the observation period, an experiment was carried out on the term\nspace of a collection of 12.8 million Amazon book reviews. For evaluation, the\nsemantic consistency of ESOM term clusters was compared with their respective\nneighbourhoods in WordNet, and contrasted with distances among term vectors by\nrandom indexing. We found that at 0.05 level of significance, the terms in the\nclusters showed a high level of semantic consistency. Tracking the drift of\ndistributional patterns in the term space across time periods, we found that\nconsistency decreased, but not at a statistically significant level. Our method\nis highly scalable, with interpretations in philosophy.",
            "journal ref": "Proceedings of IJCNN-15, International Joint Conference on Neural\n  Networks, pages 1--8, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.03520v8",
            "date": "2015-02-12 02:50:08+00:00",
            "title": "A Latent Variable Model Approach to PMI-based Word Embeddings",
            "authors": [
                "Sanjeev Arora",
                "Yuanzhi Li",
                "Yingyu Liang",
                "Tengyu Ma",
                "Andrej Risteski"
            ],
            "abstract": "Semantic word embeddings represent the meaning of a word via a vector, and\nare created by diverse methods. Many use nonlinear operations on co-occurrence\nstatistics, and have hand-tuned hyperparameters and reweighting methods.\n  This paper proposes a new generative model, a dynamic version of the\nlog-linear topic model of~\\citet{mnih2007three}. The methodological novelty is\nto use the prior to compute closed form expressions for word statistics. This\nprovides a theoretical justification for nonlinear models like PMI, word2vec,\nand GloVe, as well as some hyperparameter choices. It also helps explain why\nlow-dimensional semantic embeddings contain linear algebraic structure that\nallows solution of word analogies, as shown by~\\citet{mikolov2013efficient} and\nmany subsequent papers.\n  Experimental support is provided for the generative model assumptions, the\nmost important of which is that latent word vectors are fairly uniformly\ndispersed in space.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1502.03630v1",
            "date": "2015-02-12 12:32:39+00:00",
            "title": "Ordering-sensitive and Semantic-aware Topic Modeling",
            "authors": [
                "Min Yang",
                "Tianyi Cui",
                "Wenting Tu"
            ],
            "abstract": "Topic modeling of textual corpora is an important and challenging problem. In\nmost previous work, the \"bag-of-words\" assumption is usually made which ignores\nthe ordering of words. This assumption simplifies the computation, but it\nunrealistically loses the ordering information and the semantic of words in the\ncontext. In this paper, we present a Gaussian Mixture Neural Topic Model\n(GMNTM) which incorporates both the ordering of words and the semantic meaning\nof sentences into topic modeling. Specifically, we represent each topic as a\ncluster of multi-dimensional vectors and embed the corpus into a collection of\nvectors generated by the Gaussian mixture model. Each word is affected not only\nby its topic, but also by the embedding vector of its surrounding words and the\ncontext. The Gaussian mixture components and the topic of documents, sentences\nand words can be learnt jointly. Extensive experiments show that our model can\nlearn better topics and more accurate word distributions for each topic.\nQuantitatively, comparing to state-of-the-art topic modeling approaches, GMNTM\nobtains significantly better performance in terms of perplexity, retrieval\naccuracy and classification accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1502.03682v1",
            "date": "2015-02-12 14:44:15+00:00",
            "title": "Applying deep learning techniques on medical corpora from the World Wide Web: a prototypical system and evaluation",
            "authors": [
                "Jose Antonio Mi\u00f1arro-Gim\u00e9nez",
                "Oscar Mar\u00edn-Alonso",
                "Matthias Samwald"
            ],
            "abstract": "BACKGROUND: The amount of biomedical literature is rapidly growing and it is\nbecoming increasingly difficult to keep manually curated knowledge bases and\nontologies up-to-date. In this study we applied the word2vec deep learning\ntoolkit to medical corpora to test its potential for identifying relationships\nfrom unstructured text. We evaluated the efficiency of word2vec in identifying\nproperties of pharmaceuticals based on mid-sized, unstructured medical text\ncorpora available on the web. Properties included relationships to diseases\n('may treat') or physiological processes ('has physiological effect'). We\ncompared the relationships identified by word2vec with manually curated\ninformation from the National Drug File - Reference Terminology (NDF-RT)\nontology as a gold standard. RESULTS: Our results revealed a maximum accuracy\nof 49.28% which suggests a limited ability of word2vec to capture linguistic\nregularities on the collected medical corpora compared with other published\nresults. We were able to document the influence of different parameter settings\non result accuracy and found and unexpected trade-off between ranking quality\nand accuracy. Pre-processing corpora to reduce syntactic variability proved to\nbe a good strategy for increasing the utility of the trained vector models.\nCONCLUSIONS: Word2vec is a very efficient implementation for computing vector\nrepresentations and for its ability to identify relationships in textual data\nwithout any prior domain knowledge. We found that the ranking and retrieved\nresults generated by word2vec were not of sufficient quality for automatic\npopulation of knowledge bases and ontologies, but could serve as a starting\npoint for further manual curation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.04049v1",
            "date": "2015-02-13 16:28:40+00:00",
            "title": "How essential are unstructured clinical narratives and information fusion to clinical trial recruitment?",
            "authors": [
                "Preethi Raghavan",
                "James L. Chen",
                "Eric Fosler-Lussier",
                "Albert M. Lai"
            ],
            "abstract": "Electronic health records capture patient information using structured\ncontrolled vocabularies and unstructured narrative text. While structured data\ntypically encodes lab values, encounters and medication lists, unstructured\ndata captures the physician's interpretation of the patient's condition,\nprognosis, and response to therapeutic intervention. In this paper, we\ndemonstrate that information extraction from unstructured clinical narratives\nis essential to most clinical applications. We perform an empirical study to\nvalidate the argument and show that structured data alone is insufficient in\nresolving eligibility criteria for recruiting patients onto clinical trials for\nchronic lymphocytic leukemia (CLL) and prostate cancer. Unstructured data is\nessential to solving 59% of the CLL trial criteria and 77% of the prostate\ncancer trial criteria. More specifically, for resolving eligibility criteria\nwith temporal constraints, we show the need for temporal reasoning and\ninformation integration with medical events within and across unstructured\nclinical narratives and structured data.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1502.04081v2",
            "date": "2015-02-13 18:39:29+00:00",
            "title": "A Linear Dynamical System Model for Text",
            "authors": [
                "David Belanger",
                "Sham Kakade"
            ],
            "abstract": "Low dimensional representations of words allow accurate NLP models to be\ntrained on limited annotated data. While most representations ignore words'\nlocal context, a natural way to induce context-dependent representations is to\nperform inference in a probabilistic latent-variable sequence model. Given the\nrecent success of continuous vector space word representations, we provide such\nan inference procedure for continuous states, where words' representations are\ngiven by the posterior mean of a linear dynamical system. Here, efficient\ninference can be performed using Kalman filtering. Our learning algorithm is\nextremely scalable, operating on simple cooccurrence counts for both parameter\ninitialization using the method of moments and subsequent iterations of EM. In\nour experiments, we employ our inferred word embeddings as features in standard\ntagging tasks, obtaining significant accuracy improvements. Finally, the Kalman\nfilter updates can be seen as a linear recurrent neural network. We demonstrate\nthat using the parameters of our model to initialize a non-linear recurrent\nneural network language model reduces its training time by a day and yields\nlower perplexity.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1502.05441v1",
            "date": "2015-02-18 23:16:00+00:00",
            "title": "Rule-and Dictionary-based Solution for Variations in Written Arabic Names in Social Networks, Big Data, Accounting Systems and Large Databases",
            "authors": [
                "Ahmad B. A. Hassanat",
                "Ghada Awad Altarawneh"
            ],
            "abstract": "This paper investigates the problem that some Arabic names can be written in\nmultiple ways. When someone searches for only one form of a name, neither exact\nnor approximate matching is appropriate for returning the multiple variants of\nthe name. Exact matching requires the user to enter all forms of the name for\nthe search, and approximate matching yields names not among the variations of\nthe one being sought. In this paper, we attempt to solve the problem with a\ndictionary of all Arabic names mapped to their different (alternative) writing\nforms. We generated alternatives based on rules we derived from reviewing the\nfirst names of 9.9 million citizens and former citizens of Jordan. This\ndictionary can be used for both standardizing the written form when inserting a\nnew name into a database and for searching for the name and all its alternative\nwritten forms. Creating the dictionary automatically based on rules resulted in\nat least 7% erroneous acceptance errors and 7.9% erroneous rejection errors. We\naddressed the errors by manually editing the dictionary. The dictionary can be\nof help to real world-databases, with the qualification that manual editing\ndoes not guarantee 100% correctness.",
            "journal ref": "Research Journal of Applied Sciences, Engineering and Technology,\n  2014, 8(14): 1630-1638",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1502.05472v2",
            "date": "2015-02-19 06:04:40+00:00",
            "title": "On the Effects of Low-Quality Training Data on Information Extraction from Clinical Reports",
            "authors": [
                "Diego Marcheggiani",
                "Fabrizio Sebastiani"
            ],
            "abstract": "In the last five years there has been a flurry of work on information\nextraction from clinical documents, i.e., on algorithms capable of extracting,\nfrom the informal and unstructured texts that are generated during everyday\nclinical practice, mentions of concepts relevant to such practice. Most of this\nliterature is about methods based on supervised learning, i.e., methods for\ntraining an information extraction system from manually annotated examples.\nWhile a lot of work has been devoted to devising learning methods that generate\nmore and more accurate information extractors, no work has been devoted to\ninvestigating the effect of the quality of training data on the learning\nprocess. Low quality in training data often derives from the fact that the\nperson who has annotated the data is different from the one against whose\njudgment the automatically annotated data must be evaluated. In this paper we\ntest the impact of such data quality issues on the accuracy of information\nextraction systems as applied to the clinical domain. We do this by comparing\nthe accuracy deriving from training data annotated by the authoritative coder\n(i.e., the one who has also annotated the test data, and by whose judgment we\nmust abide), with the accuracy deriving from training data annotated by a\ndifferent coder. The results indicate that, although the disagreement between\nthe two coders (as measured on the training set) is substantial, the difference\nis (surprisingly enough) not always statistically significant.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1502.05698v10",
            "date": "2015-02-19 20:46:10+00:00",
            "title": "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks",
            "authors": [
                "Jason Weston",
                "Antoine Bordes",
                "Sumit Chopra",
                "Alexander M. Rush",
                "Bart van Merri\u00ebnboer",
                "Armand Joulin",
                "Tomas Mikolov"
            ],
            "abstract": "One long-term goal of machine learning research is to produce methods that\nare applicable to reasoning and natural language, in particular building an\nintelligent dialogue agent. To measure progress towards that goal, we argue for\nthe usefulness of a set of proxy tasks that evaluate reading comprehension via\nquestion answering. Our tasks measure understanding in several ways: whether a\nsystem is able to answer questions via chaining facts, simple induction,\ndeduction and many more. The tasks are designed to be prerequisites for any\nsystem that aims to be capable of conversing with a human. We believe many\nexisting learning systems can currently not solve them, and hence our aim is to\nclassify these tasks into skill sets, so that researchers can identify (and\nthen rectify) the failings of their systems. We also extend and improve the\nrecently introduced Memory Networks model, and show it is able to solve some,\nbut not all, of the tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1502.05957v2",
            "date": "2015-02-20 17:55:58+00:00",
            "title": "Web Similarity in Sets of Search Terms using Database Queries",
            "authors": [
                "Andrew R. Cohen",
                "Paul M. B. Vitanyi"
            ],
            "abstract": "Normalized web distance (NWD) is a similarity or normalized semantic distance\nbased on the World Wide Web or another large electronic database, for instance\nWikipedia, and a search engine that returns reliable aggregate page counts. For\nsets of search terms the NWD gives a common similarity (common semantics) on a\nscale from 0 (identical) to 1 (completely different). The NWD approximates the\nsimilarity of members of a set according to all (upper semi)computable\nproperties. We develop the theory and give applications of classifying using\nAmazon, Wikipedia, and the NCBI website from the National Institutes of Health.\nThe last gives new correlations between health hazards. A restriction of the\nNWD to a set of two yields the earlier normalized google distance (NGD) but no\ncombination of the NGD's of pairs in a set can extract the information the NWD\nextracts from the set. The NWD enables a new contextual (different databases)\nlearning approachbased on Kolmogorov complexity theory that incorporates\nknowledge from these databases.",
            "journal ref": "SN COMPUT. SCI. 1, 161(2020)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1502.06161v1",
            "date": "2015-02-22 01:30:32+00:00",
            "title": "Using NLP to measure democracy",
            "authors": [
                "Thiago Marzag\u00e3o"
            ],
            "abstract": "This paper uses natural language processing to create the first machine-coded\ndemocracy index, which I call Automated Democracy Scores (ADS). The ADS are\nbased on 42 million news articles from 6,043 different sources and cover all\nindependent countries in the 1993-2012 period. Unlike the democracy indices we\nhave today the ADS are replicable and have standard errors small enough to\nactually distinguish between cases.\n  The ADS are produced with supervised learning. Three approaches are tried: a)\na combination of Latent Semantic Analysis and tree-based regression methods; b)\na combination of Latent Dirichlet Allocation and tree-based regression methods;\nand c) the Wordscores algorithm. The Wordscores algorithm outperforms the\nalternatives, so it is the one on which the ADS are based.\n  There is a web application where anyone can change the training set and see\nhow the results change: democracy-scores.org",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.06922v3",
            "date": "2015-02-24 19:39:27+00:00",
            "title": "Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval",
            "authors": [
                "Hamid Palangi",
                "Li Deng",
                "Yelong Shen",
                "Jianfeng Gao",
                "Xiaodong He",
                "Jianshu Chen",
                "Xinying Song",
                "Rabab Ward"
            ],
            "abstract": "This paper develops a model that addresses sentence embedding, a hot topic in\ncurrent natural language processing research, using recurrent neural networks\nwith Long Short-Term Memory (LSTM) cells. Due to its ability to capture long\nterm memory, the LSTM-RNN accumulates increasingly richer information as it\ngoes through the sentence, and when it reaches the last word, the hidden layer\nof the network provides a semantic representation of the whole sentence. In\nthis paper, the LSTM-RNN is trained in a weakly supervised manner on user\nclick-through data logged by a commercial web search engine. Visualization and\nanalysis are performed to understand how the embedding process works. The model\nis found to automatically attenuate the unimportant words and detects the\nsalient keywords in the sentence. Furthermore, these detected keywords are\nfound to automatically activate different cells of the LSTM-RNN, where words\nbelonging to a similar topic activate the same cell. As a semantic\nrepresentation of the sentence, the embedding vector can be used in many\ndifferent applications. These automatic keyword detection and topic allocation\nabilities enabled by the LSTM-RNN allow the network to perform document\nretrieval, a difficult language processing task, where the similarity between\nthe query and documents can be measured by the distance between their\ncorresponding sentence embedding vectors computed by the LSTM-RNN. On a web\nsearch task, the LSTM-RNN embedding is shown to significantly outperform\nseveral existing state of the art methods. We emphasize that the proposed model\ngenerates sentence embedding vectors that are specially useful for web document\nretrieval tasks. A comparison with a well known general sentence embedding\nmethod, the Paragraph Vector, is performed. The results show that the proposed\nmethod in this paper significantly outperforms it for web document retrieval\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1502.08030v2",
            "date": "2015-02-27 19:34:42+00:00",
            "title": "Author Name Disambiguation by Using Deep Neural Network",
            "authors": [
                "Hung Nghiep Tran",
                "Tin Huynh",
                "Tien Do"
            ],
            "abstract": "Author name ambiguity decreases the quality and reliability of information\nretrieved from digital libraries. Existing methods have tried to solve this\nproblem by predefining a feature set based on expert's knowledge for a specific\ndataset. In this paper, we propose a new approach which uses deep neural\nnetwork to learn features automatically from data. Additionally, we propose the\ngeneral system architecture for author name disambiguation on any dataset. In\nthis research, we evaluate the proposed method on a dataset containing\nVietnamese author names. The results show that this method significantly\noutperforms other methods that use predefined feature set. The proposed method\nachieves 99.31% in terms of accuracy. Prediction error rate decreases from\n1.83% to 0.69%, i.e., it decreases by 1.14%, or 62.3% relatively compared with\nother methods that use predefined feature set (Table 3).",
            "journal ref": "Asian Conference on Intelligent Information and Database Systems\n  (ACIIDS 2014)",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1502.08033v1",
            "date": "2015-02-27 19:35:24+00:00",
            "title": "SciRecSys: A Recommendation System for Scientific Publication by Discovering Keyword Relationships",
            "authors": [
                "Vu Le Anh",
                "Vo Hoang Hai",
                "Hung Nghiep Tran",
                "Jason J. Jung"
            ],
            "abstract": "In this work, we propose a new approach for discovering various relationships\namong keywords over the scientific publications based on a Markov Chain model.\nIt is an important problem since keywords are the basic elements for\nrepresenting abstract objects such as documents, user profiles, topics and many\nthings else. Our model is very effective since it combines four important\nfactors in scientific publications: content, publicity, impact and randomness.\nParticularly, a recommendation system (called SciRecSys) has been presented to\nsupport users to efficiently find out relevant articles.",
            "journal ref": "International Conference on Computational Collective Intelligence\n  (ICCCI 2014)",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1503.03512v4",
            "date": "2015-03-11 21:32:01+00:00",
            "title": "Is language evolution grinding to a halt? The scaling of lexical turbulence in English fiction suggests it is not",
            "authors": [
                "Eitan Adam Pechenick",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Of basic interest is the quantification of the long term growth of a\nlanguage's lexicon as it develops to more completely cover both a culture's\ncommunication requirements and knowledge space. Here, we explore the usage\ndynamics of words in the English language as reflected by the Google Books 2012\nEnglish Fiction corpus. We critique an earlier method that found decreasing\nbirth and increasing death rates of words over the second half of the 20th\nCentury, showing death rates to be strongly affected by the imposed time cutoff\nof the arbitrary present and not increasing dramatically. We provide a robust,\nprincipled approach to examining lexical evolution by tracking the volume of\nword flux across various relative frequency thresholds. We show that while the\noverall statistical structure of the English language remains stable over time\nin terms of its raw Zipf distribution, we find evidence of an enduring `lexical\nturbulence': The flux of words across frequency thresholds from decade to\ndecade scales superlinearly with word rank and exhibits a scaling break we\nconnect to that of Zipf's law. To better understand the changing lexicon, we\nexamine the contributions to the Jensen-Shannon divergence of individual words\ncrossing frequency thresholds. We also find indications that scholarly works\nabout fiction are strongly represented in the 2012 English Fiction corpus, and\nsuggest that a future revision of the corpus should attempt to separate\ncritical works from fiction itself.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1507.00955v3",
            "date": "2015-07-03 15:46:55+00:00",
            "title": "Twitter Sentiment Analysis: Lexicon Method, Machine Learning Method and Their Combination",
            "authors": [
                "Olga Kolchyna",
                "Tharsis T. P. Souza",
                "Philip Treleaven",
                "Tomaso Aste"
            ],
            "abstract": "This paper covers the two approaches for sentiment analysis: i) lexicon based\nmethod; ii) machine learning method. We describe several techniques to\nimplement these approaches and discuss how they can be adopted for sentiment\nclassification of Twitter messages. We present a comparative study of different\nlexicon combinations and show that enhancing sentiment lexicons with emoticons,\nabbreviations and social-media slang expressions increases the accuracy of\nlexicon-based classification for Twitter. We discuss the importance of feature\ngeneration and feature selection processes for machine learning sentiment\nclassification. To quantify the performance of the main sentiment analysis\nmethods over Twitter we run these algorithms on a benchmark Twitter dataset\nfrom the SemEval-2013 competition, task 2-B. The results show that machine\nlearning method based on SVM and Naive Bayes classifiers outperforms the\nlexicon method. We present a new ensemble method that uses a lexicon based\nsentiment score as input feature for the machine learning approach. The\ncombined method proved to produce more precise classifications. We also show\nthat employing a cost-sensitive classifier for highly unbalanced datasets\nyields an improvement of sentiment classification performance up to 7%.",
            "journal ref": "Handbook of Sentiment Analysis in Finance. Mitra, G. and Yu, X.\n  (Eds.). (2016). ISBN 1910571571",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05582v2",
            "date": "2015-12-17 14:01:14+00:00",
            "title": "Kauffman's adjacent possible in word order evolution",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Word order evolution has been hypothesized to be constrained by a word order\npermutation ring: transitions involving orders that are closer in the\npermutation ring are more likely. The hypothesis can be seen as a particular\ncase of Kauffman's adjacent possible in word order evolution. Here we consider\nthe problem of the association of the six possible orders of S, V and O to\nyield a couple of primary alternating orders as a window to word order\nevolution. We evaluate the suitability of various competing hypotheses to\npredict one member of the couple from the other with the help of information\ntheoretic model selection. Our ensemble of models includes a six-way model that\nis based on the word order permutation ring (Kauffman's adjacent possible) and\nanother model based on the dual two-way of standard typology, that reduces word\norder to basic orders preferences (e.g., a preference for SV over VS and\nanother for SO over OS). Our analysis indicates that the permutation ring\nyields the best model when favoring parsimony strongly, providing support for\nKauffman's general view and a six-way typology.",
            "journal ref": "In S.G. Roberts et al (eds.). The Evolution of Language:\n  Proceedings of the 11th International Conference (EVOLANG11). New Orleans,\n  USA, March 21-24 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1512.05742v3",
            "date": "2015-12-17 19:52:39+00:00",
            "title": "A Survey of Available Corpora for Building Data-Driven Dialogue Systems",
            "authors": [
                "Iulian Vlad Serban",
                "Ryan Lowe",
                "Peter Henderson",
                "Laurent Charlin",
                "Joelle Pineau"
            ],
            "abstract": "During the past decade, several areas of speech and language understanding\nhave witnessed substantial breakthroughs from the use of data-driven models. In\nthe area of dialogue systems, the trend is less obvious, and most practical\nsystems are still built through significant engineering and expert knowledge.\nNevertheless, several recent results suggest that data-driven approaches are\nfeasible and quite promising. To facilitate research in this area, we have\ncarried out a wide survey of publicly available datasets suitable for\ndata-driven learning of dialogue systems. We discuss important characteristics\nof these datasets, how they can be used to learn diverse dialogue strategies,\nand their other potential uses. We also examine methods for transfer learning\nbetween datasets and the use of external knowledge. Finally, we discuss\nappropriate choice of evaluation metrics for the learning objective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1510.08983v2",
            "date": "2015-10-30 06:40:14+00:00",
            "title": "Highway Long Short-Term Memory RNNs for Distant Speech Recognition",
            "authors": [
                "Yu Zhang",
                "Guoguo Chen",
                "Dong Yu",
                "Kaisheng Yao",
                "Sanjeev Khudanpur",
                "James Glass"
            ],
            "abstract": "In this paper, we extend the deep long short-term memory (DLSTM) recurrent\nneural networks by introducing gated direct connections between memory cells in\nadjacent layers. These direct links, called highway connections, enable\nunimpeded information flow across different layers and thus alleviate the\ngradient vanishing problem when building deeper LSTMs. We further introduce the\nlatency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole\nhistory while keeping the latency under control. Efficient algorithms are\nproposed to train these novel networks using both frame and sequence\ndiscriminative criteria. Experiments on the AMI distant speech recognition\n(DSR) task indicate that we can train deeper LSTMs and achieve better\nimprovement from sequence training with highway LSTMs (HLSTMs). Our novel model\nobtains $43.9/47.7\\%$ WER on AMI (SDM) dev and eval sets, outperforming all\nprevious works. It beats the strong DNN and DLSTM baselines with $15.7\\%$ and\n$5.3\\%$ relative improvement respectively.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1505.00138v1",
            "date": "2015-05-01 10:00:33+00:00",
            "title": "Compositional Distributional Semantics with Compact Closed Categories and Frobenius Algebras",
            "authors": [
                "Dimitri Kartsaklis"
            ],
            "abstract": "This thesis contributes to ongoing research related to the categorical\ncompositional model for natural language of Coecke, Sadrzadeh and Clark in\nthree ways: Firstly, I propose a concrete instantiation of the abstract\nframework based on Frobenius algebras (joint work with Sadrzadeh). The theory\nimproves shortcomings of previous proposals, extends the coverage of the\nlanguage, and is supported by experimental work that improves existing results.\nThe proposed framework describes a new class of compositional models that find\nintuitive interpretations for a number of linguistic phenomena. Secondly, I\npropose and evaluate in practice a new compositional methodology which\nexplicitly deals with the different levels of lexical ambiguity (joint work\nwith Pulman). A concrete algorithm is presented, based on the separation of\nvector disambiguation from composition in an explicit prior step. Extensive\nexperimental work shows that the proposed methodology indeed results in more\naccurate composite representations for the framework of Coecke et al. in\nparticular and every other class of compositional models in general. As a last\ncontribution, I formalize the explicit treatment of lexical ambiguity in the\ncontext of the categorical framework by resorting to categorical quantum\nmechanics (joint work with Coecke). In the proposed extension, the concept of a\ndistributional vector is replaced with that of a density matrix, which\ncompactly represents a probability distribution over the potential different\nmeanings of the specific word. Composition takes the form of quantum\nmeasurements, leading to interesting analogies between quantum physics and\nlinguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1509.07175v5",
            "date": "2015-09-23 22:41:46+00:00",
            "title": "Exploration and Exploitation of Victorian Science in Darwin's Reading Notebooks",
            "authors": [
                "Jaimie Murdock",
                "Colin Allen",
                "Simon DeDeo"
            ],
            "abstract": "Search in an environment with an uncertain distribution of resources involves\na trade-off between exploitation of past discoveries and further exploration.\nThis extends to information foraging, where a knowledge-seeker shifts between\nreading in depth and studying new domains. To study this decision-making\nprocess, we examine the reading choices made by one of the most celebrated\nscientists of the modern era: Charles Darwin. From the full-text of books\nlisted in his chronologically-organized reading journals, we generate topic\nmodels to quantify his local (text-to-text) and global (text-to-past) reading\ndecisions using Kullback-Liebler Divergence, a cognitively-validated,\ninformation-theoretic measure of relative surprise. Rather than a pattern of\nsurprise-minimization, corresponding to a pure exploitation strategy, Darwin's\nbehavior shifts from early exploitation to later exploration, seeking unusually\nhigh levels of cognitive surprise relative to previous eras. These shifts,\ndetected by an unsupervised Bayesian model, correlate with major intellectual\nepochs of his career as identified both by qualitative scholarship and Darwin's\nown self-commentary. Our methods allow us to compare his consumption of texts\nwith their publication order. We find Darwin's consumption more exploratory\nthan the culture's production, suggesting that underneath gradual societal\nchanges are the explorations of individual synthesis and discovery. Our\nquantitative methods advance the study of cognitive search through a framework\nfor testing interactions between individual and collective behavior and between\nshort- and long-term consumption choices. This novel application of topic\nmodeling to characterize individual reading complements widespread studies of\ncollective scientific behavior.",
            "journal ref": "Cognition 159 (2017) 117-126",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.00923v1",
            "date": "2015-04-03 19:57:06+00:00",
            "title": "A Unified Deep Neural Network for Speaker and Language Recognition",
            "authors": [
                "Fred Richardson",
                "Douglas Reynolds",
                "Najim Dehak"
            ],
            "abstract": "Learned feature representations and sub-phoneme posteriors from Deep Neural\nNetworks (DNNs) have been used separately to produce significant performance\ngains for speaker and language recognition tasks. In this work we show how\nthese gains are possible using a single DNN for both speaker and language\nrecognition. The unified DNN approach is shown to yield substantial performance\nimprovements on the the 2013 Domain Adaptation Challenge speaker recognition\ntask (55% reduction in EER for the out-of-domain condition) and on the NIST\n2011 Language Recognition Evaluation (48% reduction in EER for the 30s test\ncondition).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1504.04884v3",
            "date": "2015-04-19 20:57:11+00:00",
            "title": "Compression and the origins of Zipf's law of abbreviation",
            "authors": [
                "R. Ferrer-i-Cancho",
                "C. Bentz",
                "C. Seguin"
            ],
            "abstract": "Languages across the world exhibit Zipf's law of abbreviation, namely more\nfrequent words tend to be shorter. The generalized version of the law - an\ninverse relationship between the frequency of a unit and its magnitude - holds\nalso for the behaviours of other species and the genetic code. The apparent\nuniversality of this pattern in human language and its ubiquity in other\ndomains calls for a theoretical understanding of its origins. To this end, we\ngeneralize the information theoretic concept of mean code length as a mean\nenergetic cost function over the probability and the magnitude of the types of\nthe repertoire. We show that the minimization of that cost function and a\nnegative correlation between probability and the magnitude of types are\nintimately related.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1506.04089v4",
            "date": "2015-06-12 18:05:00+00:00",
            "title": "Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences",
            "authors": [
                "Hongyuan Mei",
                "Mohit Bansal",
                "Matthew R. Walter"
            ],
            "abstract": "We propose a neural sequence-to-sequence model for direction following, a\ntask that is essential to realizing effective autonomous agents. Our\nalignment-based encoder-decoder model with long short-term memory recurrent\nneural networks (LSTM-RNN) translates natural language instructions to action\nsequences based upon a representation of the observable world state. We\nintroduce a multi-level aligner that empowers our model to focus on sentence\n\"regions\" salient to the current world state by using multiple abstractions of\nthe input sentence. In contrast to existing methods, our model uses no\nspecialized linguistic resources (e.g., parsers) or task-specific annotations\n(e.g., seed lexicons). It is therefore generalizable, yet still achieves the\nbest results reported to-date on a benchmark single-sentence dataset and\ncompetitive results for the limited-training multi-sentence setting. We analyze\nour model through a series of ablations that elucidate the contributions of the\nprimary components of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.04744v1",
            "date": "2015-06-15 20:00:29+00:00",
            "title": "Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game",
            "authors": [
                "Vlad Niculae",
                "Srijan Kumar",
                "Jordan Boyd-Graber",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Interpersonal relations are fickle, with close friendships often dissolving\ninto enmity. In this work, we explore linguistic cues that presage such\ntransitions by studying dyadic interactions in an online strategy game where\nplayers form alliances and break those alliances through betrayal. We\ncharacterize friendships that are unlikely to last and examine temporal\npatterns that foretell betrayal.\n  We reveal that subtle signs of imminent betrayal are encoded in the\nconversational patterns of the dyad, even if the victim is not aware of the\nrelationship's fate. In particular, we find that lasting friendships exhibit a\nform of balance that manifests itself through language. In contrast, sudden\nchanges in the balance of certain conversational attributes---such as positive\nsentiment, politeness, or focus on future planning---signal impending betrayal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1506.06534v2",
            "date": "2015-06-22 10:14:47+00:00",
            "title": "Distributional Sentence Entailment Using Density Matrices",
            "authors": [
                "Esma Balkir",
                "Mehrnoosh Sadrzadeh",
                "Bob Coecke"
            ],
            "abstract": "Categorical compositional distributional model of Coecke et al. (2010)\nsuggests a way to combine grammatical composition of the formal, type logical\nmodels with the corpus based, empirical word representations of distributional\nsemantics. This paper contributes to the project by expanding the model to also\ncapture entailment relations. This is achieved by extending the representations\nof words from points in meaning space to density operators, which are\nprobability distributions on the subspaces of the space. A symmetric measure of\nsimilarity and an asymmetric measure of entailment is defined, where lexical\nentailment is measured using von Neumann entropy, the quantum variant of\nKullback-Leibler divergence. Lexical entailment, combined with the composition\nmap on word representations, provides a method to obtain entailment relations\non the level of sentences. Truth theoretic and corpus-based examples are\nprovided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1508.01746v2",
            "date": "2015-08-07 16:20:52+00:00",
            "title": "Using Deep Learning for Detecting Spoofing Attacks on Speech Signals",
            "authors": [
                "Alan Godoy",
                "Fl\u00e1vio Sim\u00f5es",
                "Jos\u00e9 Augusto Stuchi",
                "Marcus de Assis Angeloni",
                "M\u00e1rio Uliani",
                "Ricardo Violato"
            ],
            "abstract": "It is well known that speaker verification systems are subject to spoofing\nattacks. The Automatic Speaker Verification Spoofing and Countermeasures\nChallenge -- ASVSpoof2015 -- provides a standard spoofing database, containing\nattacks based on synthetic speech, along with a protocol for experiments. This\npaper describes CPqD's systems submitted to the ASVSpoof2015 Challenge, based\non deep neural networks, working both as a classifier and as a feature\nextraction module for a GMM and a SVM classifier. Results show the validity of\nthis approach, achieving less than 0.5\\% EER for known attacks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1508.06161v1",
            "date": "2015-08-25 14:10:21+00:00",
            "title": "Robot Language Learning, Generation, and Comprehension",
            "authors": [
                "Daniel Paul Barrett",
                "Scott Alan Bronikowski",
                "Haonan Yu",
                "Jeffrey Mark Siskind"
            ],
            "abstract": "We present a unified framework which supports grounding natural-language\nsemantics in robotic driving. This framework supports acquisition (learning\ngrounded meanings of nouns and prepositions from human annotation of robotic\ndriving paths), generation (using such acquired meanings to generate sentential\ndescription of new robotic driving paths), and comprehension (using such\nacquired meanings to support automated driving to accomplish navigational goals\nspecified in natural language). We evaluate the performance of these three\ntasks by having independent human judges rate the semantic fidelity of the\nsentences associated with paths, achieving overall average correctness of 94.6%\nand overall average completeness of 85.6%.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1511.04970v2",
            "date": "2015-11-16 14:29:38+00:00",
            "title": "Learning about Spanish dialects through Twitter",
            "authors": [
                "Bruno Gon\u00e7alves",
                "David S\u00e1nchez"
            ],
            "abstract": "This paper maps the large-scale variation of the Spanish language by\nemploying a corpus based on geographically tagged Twitter messages. Lexical\ndialects are extracted from an analysis of variants of tens of concepts. The\nresulting maps show linguistic variation on an unprecedented scale across the\nglobe. We discuss the properties of the main dialects within a machine learning\napproach and find that varieties spoken in urban areas have an international\ncharacter in contrast to country areas where dialects show a more regional\nuniformity.",
            "journal ref": "RILI, XVI 2 (28), 65-75 (2016)",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1502.08029v5",
            "date": "2015-02-27 19:30:40+00:00",
            "title": "Describing Videos by Exploiting Temporal Structure",
            "authors": [
                "Li Yao",
                "Atousa Torabi",
                "Kyunghyun Cho",
                "Nicolas Ballas",
                "Christopher Pal",
                "Hugo Larochelle",
                "Aaron Courville"
            ],
            "abstract": "Recent progress in using recurrent neural networks (RNNs) for image\ndescription has motivated the exploration of their application for video\ndescription. However, while images are static, working with videos requires\nmodeling their dynamic temporal structure and then properly integrating that\ninformation into a natural language description. In this context, we propose an\napproach that successfully takes into account both the local and global\ntemporal structure of videos to produce descriptions. First, our approach\nincorporates a spatial temporal 3-D convolutional neural network (3-D CNN)\nrepresentation of the short temporal dynamics. The 3-D CNN representation is\ntrained on video action recognition tasks, so as to produce a representation\nthat is tuned to human motion and behavior. Second we propose a temporal\nattention mechanism that allows to go beyond local temporal modeling and learns\nto automatically select the most relevant temporal segments given the\ntext-generating RNN. Our approach exceeds the current state-of-art for both\nBLEU and METEOR metrics on the Youtube2Text dataset. We also present results on\na new, larger and more challenging dataset of paired video and natural language\ndescriptions.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1503.06410v2",
            "date": "2015-03-22 11:32:34+00:00",
            "title": "What the F-measure doesn't measure: Features, Flaws, Fallacies and Fixes",
            "authors": [
                "David M. W. Powers"
            ],
            "abstract": "The F-measure or F-score is one of the most commonly used single number\nmeasures in Information Retrieval, Natural Language Processing and Machine\nLearning, but it is based on a mistake, and the flawed assumptions render it\nunsuitable for use in most contexts! Fortunately, there are better\nalternatives.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1512.01587v1",
            "date": "2015-12-04 22:58:29+00:00",
            "title": "Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text",
            "authors": [
                "Sahil Garg",
                "Aram Galstyan",
                "Ulf Hermjakob",
                "Daniel Marcu"
            ],
            "abstract": "We advance the state of the art in biomolecular interaction extraction with\nthree contributions: (i) We show that deep, Abstract Meaning Representations\n(AMR) significantly improve the accuracy of a biomolecular interaction\nextraction system when compared to a baseline that relies solely on surface-\nand syntax-based features; (ii) In contrast with previous approaches that infer\nrelations on a sentence-by-sentence basis, we expand our framework to enable\nconsistent predictions over sets of sentences (documents); (iii) We further\nmodify and expand a graph kernel learning framework to enable concurrent\nexploitation of automatically induced AMR (semantic) and dependency structure\n(syntactic) representations. Our experiments show that our approach yields\ninteraction extraction systems that are more robust in environments where there\nis a significant mismatch between training and test conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2016": [
        {
            "url": "http://arxiv.org/abs/1606.00025v5",
            "date": "2016-05-31 20:09:59+00:00",
            "title": "Implementing a Reverse Dictionary, based on word definitions, using a Node-Graph Architecture",
            "authors": [
                "Sushrut Thorat",
                "Varad Choudhari"
            ],
            "abstract": "In this paper, we outline an approach to build graph-based reverse\ndictionaries using word definitions. A reverse dictionary takes a phrase as an\ninput and outputs a list of words semantically similar to that phrase. It is a\nsolution to the Tip-of-the-Tongue problem. We use a distance-based similarity\nmeasure, computed on a graph, to assess the similarity between a word and the\ninput phrase. We compare the performance of our approach with the Onelook\nReverse Dictionary and a distributional semantics method based on word2vec, and\nshow that our approach is much better than the distributional semantics method,\nand as good as Onelook, on a 3k lexicon. This simple approach sets a new\nperformance baseline for reverse dictionaries.",
            "journal ref": "Proceedings of COLING 2016, the 26th International Conference on\n  Computational Linguistics: Technical Papers, pages 2797-2806, Osaka, Japan,\n  December 11-17 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00189v1",
            "date": "2016-06-01 09:31:00+00:00",
            "title": "Neural Network Translation Models for Grammatical Error Correction",
            "authors": [
                "Shamil Chollampatt",
                "Kaveh Taghipour",
                "Hwee Tou Ng"
            ],
            "abstract": "Phrase-based statistical machine translation (SMT) systems have previously\nbeen used for the task of grammatical error correction (GEC) to achieve\nstate-of-the-art accuracy. The superiority of SMT systems comes from their\nability to learn text transformations from erroneous to corrected text, without\nexplicitly modeling error types. However, phrase-based SMT systems suffer from\nlimitations of discrete word representation, linear mapping, and lack of global\ncontext. In this paper, we address these limitations by using two different yet\ncomplementary neural network models, namely a neural network global lexicon\nmodel and a neural network joint model. These neural networks can generalize\nbetter by using continuous space representation of words and learn non-linear\nmappings. Moreover, they can leverage contextual information from the source\nsentence more effectively. By adding these two components, we achieve\nstatistically significant improvement in accuracy for grammatical error\ncorrection over a state-of-the-art GEC system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00210v1",
            "date": "2016-06-01 10:32:28+00:00",
            "title": "Exploiting N-Best Hypotheses to Improve an SMT Approach to Grammatical Error Correction",
            "authors": [
                "Duc Tam Hoang",
                "Shamil Chollampatt",
                "Hwee Tou Ng"
            ],
            "abstract": "Grammatical error correction (GEC) is the task of detecting and correcting\ngrammatical errors in texts written by second language learners. The\nstatistical machine translation (SMT) approach to GEC, in which sentences\nwritten by second language learners are translated to grammatically correct\nsentences, has achieved state-of-the-art accuracy. However, the SMT approach is\nunable to utilize global context. In this paper, we propose a novel approach to\nimprove the accuracy of GEC, by exploiting the n-best hypotheses generated by\nan SMT approach. Specifically, we build a classifier to score the edits in the\nn-best hypotheses. The classifier can be used to select appropriate edits or\nre-rank the n-best hypotheses. We apply these methods to a state-of-the-art GEC\nsystem that uses the SMT approach. Our experiments show that our methods\nachieve statistically significant improvements in accuracy over the best\npublished results on a benchmark test dataset on GEC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00294v1",
            "date": "2016-06-01 14:06:41+00:00",
            "title": "Improved Parsing for Argument-Clusters Coordination",
            "authors": [
                "Jessica Ficler",
                "Yoav Goldberg"
            ],
            "abstract": "Syntactic parsers perform poorly in prediction of Argument-Cluster\nCoordination (ACC). We change the PTB representation of ACC to be more suitable\nfor learning by a statistical PCFG parser, affecting 125 trees in the training\nset. Training on the modified trees yields a slight improvement in EVALB scores\non sections 22 and 23. The main evaluation is on a corpus of 4th grade science\nexams, in which ACC structures are prevalent. On this corpus, we obtain an\nimpressive x2.7 improvement in recovering ACC structures compared to a parser\ntrained on the original PTB trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00499v2",
            "date": "2016-06-01 23:26:20+00:00",
            "title": "Generalizing and Hybridizing Count-based and Neural Language Models",
            "authors": [
                "Graham Neubig",
                "Chris Dyer"
            ],
            "abstract": "Language models (LMs) are statistical models that calculate probabilities\nover sequences of words or other discrete symbols. Currently two major\nparadigms for language modeling exist: count-based n-gram models, which have\nadvantages of scalability and test-time speed, and neural LMs, which often\nachieve superior modeling performance. We demonstrate how both varieties of\nmodels can be unified in a single modeling framework that defines a set of\nprobability distributions over the vocabulary of words, and then dynamically\ncalculates mixture weights over these distributions. This formulation allows us\nto create novel hybrid models that combine the desirable features of\ncount-based and neural LMs, and experiments demonstrate the advantages of these\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00589v1",
            "date": "2016-06-02 08:57:14+00:00",
            "title": "Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection",
            "authors": [
                "Katharina Kann",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Morphological reinflection is the task of generating a target form given a\nsource form, a source tag and a target tag. We propose a new way of modeling\nthis task with neural encoder-decoder models. Our approach reduces the amount\nof required training data for this architecture and achieves state-of-the-art\nresults, making encoder-decoder models applicable to morphological reinflection\neven for low-resource languages. We further present a new automatic correction\nmethod for the outputs based on edit trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00819v2",
            "date": "2016-06-02 19:35:46+00:00",
            "title": "Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations",
            "authors": [
                "Alexandre Salle",
                "Marco Idiart",
                "Aline Villavicencio"
            ],
            "abstract": "In this paper, we propose LexVec, a new method for generating distributed\nword representations that uses low-rank, weighted factorization of the Positive\nPoint-wise Mutual Information matrix via stochastic gradient descent, employing\na weighting scheme that assigns heavier penalties for errors on frequent\nco-occurrences while still accounting for negative co-occurrence. Evaluation on\nword similarity and analogy tasks shows that LexVec matches and often\noutperforms state-of-the-art methods on many of these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01151v4",
            "date": "2016-06-03 15:43:15+00:00",
            "title": "Using Neural Generative Models to Release Synthetic Twitter Corpora with Reduced Stylometric Identifiability of Users",
            "authors": [
                "Alexander G. Ororbia II",
                "Fridolin Linder",
                "Joshua Snoke"
            ],
            "abstract": "We present a method for generating synthetic versions of Twitter data using\nneural generative models. The goal is protecting individuals in the source data\nfrom stylometric re-identification attacks while still releasing data that\ncarries research value. Specifically, we generate tweet corpora that maintain\nuser-level word distributions by augmenting the neural language models with\nuser-specific components. We compare our approach to two standard text data\nprotection methods: redaction and iterative translation. We evaluate the three\nmethods on measures of risk and utility. We define risk following the\nstylometric models of re-identification, and we define utility based on two\ngeneral word distribution measures and two common text analysis research tasks.\nWe find that neural models are able to significantly lower risk over previous\nmethods with little cost to utility. We also demonstrate that the neural models\nallow data providers to actively control the risk-utility trade-off through\nmodel tuning parameters. This work presents promising results for a new tool\naddressing the problem of privacy for free text and sharing social media data\nin a way that respects privacy and is ethically responsible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01161v1",
            "date": "2016-06-03 16:09:52+00:00",
            "title": "Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning",
            "authors": [
                "Jiang Guo",
                "Wanxiang Che",
                "Haifeng Wang",
                "Ting Liu"
            ],
            "abstract": "Various treebanks have been released for dependency parsing. Despite that\ntreebanks may belong to different languages or have different annotation\nschemes, they contain syntactic knowledge that is potential to benefit each\nother. This paper presents an universal framework for exploiting these\nmulti-typed treebanks to improve parsing with deep multi-task learning. We\nconsider two kinds of treebanks as source: the multilingual universal treebanks\nand the monolingual heterogeneous treebanks. Multiple treebanks are trained\njointly and interacted with multi-level parameter sharing. Experiments on\nseveral benchmark datasets in various languages demonstrate that our approach\ncan make effective use of arbitrary source treebanks to improve target parsing\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01283v1",
            "date": "2016-06-03 21:39:42+00:00",
            "title": "Enhancing the LexVec Distributed Word Representation Model Using Positional Contexts and External Memory",
            "authors": [
                "Alexandre Salle",
                "Marco Idiart",
                "Aline Villavicencio"
            ],
            "abstract": "In this paper we take a state-of-the-art model for distributed word\nrepresentation that explicitly factorizes the positive pointwise mutual\ninformation (PPMI) matrix using window sampling and negative sampling and\naddress two of its shortcomings. We improve syntactic performance by using\npositional contexts, and solve the need to store the PPMI matrix in memory by\nworking on aggregate data in external memory. The effectiveness of both\nmodifications is shown using word similarity and analogy tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01323v2",
            "date": "2016-06-04 04:08:45+00:00",
            "title": "Improving Coreference Resolution by Learning Entity-Level Distributed Representations",
            "authors": [
                "Kevin Clark",
                "Christopher D. Manning"
            ],
            "abstract": "A long-standing challenge in coreference resolution has been the\nincorporation of entity-level information - features defined over clusters of\nmentions instead of mention pairs. We present a neural network based\ncoreference system that produces high-dimensional vector representations for\npairs of coreference clusters. Using these representations, our system learns\nwhen combining clusters is desirable. We train the system with a\nlearning-to-search algorithm that teaches it which local decisions (cluster\nmerges) will lead to a high-scoring final coreference partition. The system\nsubstantially outperforms the current state-of-the-art on the English and\nChinese portions of the CoNLL 2012 Shared Task dataset despite using few\nhand-engineered features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01341v2",
            "date": "2016-06-04 07:52:22+00:00",
            "title": "Neural Architectures for Fine-grained Entity Type Classification",
            "authors": [
                "Sonse Shimaoka",
                "Pontus Stenetorp",
                "Kentaro Inui",
                "Sebastian Riedel"
            ],
            "abstract": "In this work, we investigate several neural network architectures for\nfine-grained entity type classification. Particularly, we consider extensions\nto a recently proposed attentive neural architecture and make three key\ncontributions. Previous work on attentive neural architectures do not consider\nhand-crafted features, we combine learnt and hand-crafted features and observe\nthat they complement each other. Additionally, through quantitative analysis we\nestablish that the attention mechanism is capable of learning to attend over\nsyntactic heads and the phrase containing the mention, where both are known\nstrong hand-crafted features for our task. We enable parameter sharing through\na hierarchical label encoding method, that in low-dimensional projections show\nclear clusters for each type hierarchy. Lastly, despite using the same\nevaluation dataset, the literature frequently compare models trained using\ndifferent data. We establish that the choice of training data has a drastic\nimpact on performance, with decreases by as much as 9.85% loose micro F1 score\nfor a previously proposed method. Despite this, our best model achieves\nstate-of-the-art results with 75.36% loose micro F1 score on the well-\nestablished FIGER (GOLD) dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01433v1",
            "date": "2016-06-04 23:22:41+00:00",
            "title": "Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction",
            "authors": [
                "Jason Alan Fries"
            ],
            "abstract": "We submitted two systems to the SemEval-2016 Task 12: Clinical TempEval\nchallenge, participating in Phase 1, where we identified text spans of time and\nevent expressions in clinical notes and Phase 2, where we predicted a relation\nbetween an event and its parent document creation time.\n  For temporal entity extraction, we find that a joint inference-based approach\nusing structured prediction outperforms a vanilla recurrent neural network that\nincorporates word embeddings trained on a variety of large clinical document\nsets. For document creation time relations, we find that a combination of date\ncanonicalization and distant supervision rules for predicting relations on both\nevents and time expressions improves classification, though gains are limited,\nlikely due to the small scale of training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01541v4",
            "date": "2016-06-05 17:59:23+00:00",
            "title": "Deep Reinforcement Learning for Dialogue Generation",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Alan Ritter",
                "Michel Galley",
                "Jianfeng Gao",
                "Dan Jurafsky"
            ],
            "abstract": "Recent neural models of dialogue generation offer great promise for\ngenerating responses for conversational agents, but tend to be shortsighted,\npredicting utterances one at a time while ignoring their influence on future\noutcomes. Modeling the future direction of a dialogue is crucial to generating\ncoherent, interesting dialogues, a need which led traditional NLP models of\ndialogue to draw on reinforcement learning. In this paper, we show how to\nintegrate these goals, applying deep reinforcement learning to model future\nreward in chatbot dialogue. The model simulates dialogues between two virtual\nagents, using policy gradient methods to reward sequences that display three\nuseful conversational properties: informativity (non-repetitive turns),\ncoherence, and ease of answering (related to forward-looking function). We\nevaluate our model on diversity, length as well as with human judges, showing\nthat the proposed algorithm generates more interactive responses and manages to\nfoster a more sustained conversation in dialogue simulation. This work marks a\nfirst step towards learning a neural conversational model based on the\nlong-term success of dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01545v3",
            "date": "2016-06-05 18:29:45+00:00",
            "title": "Neural Net Models for Open-Domain Discourse Coherence",
            "authors": [
                "Jiwei Li",
                "Dan Jurafsky"
            ],
            "abstract": "Discourse coherence is strongly associated with text quality, making it\nimportant to natural language generation and understanding. Yet existing models\nof coherence focus on measuring individual aspects of coherence (lexical\noverlap, rhetorical structure, entity centering) in narrow domains.\n  In this paper, we describe domain-independent neural models of discourse\ncoherence that are capable of measuring multiple aspects of coherence in\nexisting sentences and can maintain coherence while generating new sentences.\nWe study both discriminative models that learn to distinguish coherent from\nincoherent discourse, and generative models that produce coherent text,\nincluding a novel neural latent-variable Markovian generative model that\ncaptures the latent discourse dependencies between sentences in a text.\n  Our work achieves state-of-the-art performance on multiple coherence\nevaluations, and marks an initial step in generating coherent texts given\ndiscourse contexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01603v3",
            "date": "2016-06-06 02:45:47+00:00",
            "title": "Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution",
            "authors": [
                "Ting Liu",
                "Yiming Cui",
                "Qingyu Yin",
                "Weinan Zhang",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Most existing approaches for zero pronoun resolution are heavily relying on\nannotated data, which is often released by shared task organizers. Therefore,\nthe lack of annotated data becomes a major obstacle in the progress of zero\npronoun resolution task. Also, it is expensive to spend manpower on labeling\nthe data for better performance. To alleviate the problem above, in this paper,\nwe propose a simple but novel approach to automatically generate large-scale\npseudo training data for zero pronoun resolution. Furthermore, we successfully\ntransfer the cloze-style reading comprehension neural network model into zero\npronoun resolution task and propose a two-step training mechanism to overcome\nthe gap between the pseudo training data and the real one. Experimental results\nshow that the proposed approach significantly outperforms the state-of-the-art\nsystems with an absolute improvements of 3.1% F-score on OntoNotes 5.0 data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01614v5",
            "date": "2016-06-06 05:04:23+00:00",
            "title": "Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification",
            "authors": [
                "Xilun Chen",
                "Yu Sun",
                "Ben Athiwaratkun",
                "Claire Cardie",
                "Kilian Weinberger"
            ],
            "abstract": "In recent years great success has been achieved in sentiment classification\nfor English, thanks in part to the availability of copious annotated resources.\nUnfortunately, most languages do not enjoy such an abundance of labeled data.\nTo tackle the sentiment classification problem in low-resource languages\nwithout adequate annotated data, we propose an Adversarial Deep Averaging\nNetwork (ADAN) to transfer the knowledge learned from labeled data on a\nresource-rich source language to low-resource languages where only unlabeled\ndata exists. ADAN has two discriminative branches: a sentiment classifier and\nan adversarial language discriminator. Both branches take input from a shared\nfeature extractor to learn hidden representations that are simultaneously\nindicative for the classification task and invariant across languages.\nExperiments on Chinese and Arabic sentiment classification demonstrate that\nADAN significantly outperforms state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01700v2",
            "date": "2016-06-06 11:43:28+00:00",
            "title": "Gated Word-Character Recurrent Language Model",
            "authors": [
                "Yasumasa Miyamoto",
                "Kyunghyun Cho"
            ],
            "abstract": "We introduce a recurrent neural network language model (RNN-LM) with long\nshort-term memory (LSTM) units that utilizes both character-level and\nword-level inputs. Our model has a gate that adaptively finds the optimal\nmixture of the character-level and word-level inputs. The gate creates the\nfinal vector representation of a word by combining two distinct representations\nof the word. The character-level inputs are converted into vector\nrepresentations of words using a bidirectional LSTM. The word-level inputs are\nprojected into another high-dimensional space by a word lookup table. The final\nvector representations of words are used in the LSTM language model which\npredicts the next word given all the preceding words. Our model with the gating\nmechanism effectively utilizes the character-level inputs for rare and\nout-of-vocabulary words and outperforms word-level language models on several\nEnglish corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01792v1",
            "date": "2016-06-06 15:45:41+00:00",
            "title": "Neural Machine Translation with External Phrase Memory",
            "authors": [
                "Yaohua Tang",
                "Fandong Meng",
                "Zhengdong Lu",
                "Hang Li",
                "Philip L. H. Yu"
            ],
            "abstract": "In this paper, we propose phraseNet, a neural machine translator with a\nphrase memory which stores phrase pairs in symbolic form, mined from corpus or\nspecified by human experts. For any given source sentence, phraseNet scans the\nphrase memory to determine the candidate phrase pairs and integrates tagging\ninformation in the representation of source sentence accordingly. The decoder\nutilizes a mixture of word-generating component and phrase-generating\ncomponent, with a specifically designed strategy to generate a sequence of\nmultiple words all at once. The phraseNet not only approaches one step towards\nincorporating external knowledge into neural machine translation, but also\nmakes an effort to extend the word-by-word generation mechanism of recurrent\nneural network. Our empirical study on Chinese-to-English translation shows\nthat, with carefully-chosen phrase table in memory, phraseNet yields 3.45 BLEU\nimprovement over the generic neural machine translator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01933v2",
            "date": "2016-06-06 20:30:57+00:00",
            "title": "A Decomposable Attention Model for Natural Language Inference",
            "authors": [
                "Ankur P. Parikh",
                "Oscar T\u00e4ckstr\u00f6m",
                "Dipanjan Das",
                "Jakob Uszkoreit"
            ],
            "abstract": "We propose a simple neural architecture for natural language inference. Our\napproach uses attention to decompose the problem into subproblems that can be\nsolved separately, thus making it trivially parallelizable. On the Stanford\nNatural Language Inference (SNLI) dataset, we obtain state-of-the-art results\nwith almost an order of magnitude fewer parameters than previous work and\nwithout relying on any word-order information. Adding intra-sentence attention\nthat takes a minimum amount of order into account yields further improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01990v1",
            "date": "2016-06-07 01:17:00+00:00",
            "title": "Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features",
            "authors": [
                "Attapol T. Rutherford",
                "Vera Demberg",
                "Nianwen Xue"
            ],
            "abstract": "Inferring implicit discourse relations in natural language text is the most\ndifficult subtask in discourse parsing. Surface features achieve good\nperformance, but they are not readily applicable to other languages without\nsemantic lexicons. Previous neural models require parses, surface features, or\na small label set to work well. Here, we propose neural network models that are\nbased on feedforward and long-short term memory architecture without any\nsurface features. To our surprise, our best configured feedforward architecture\noutperforms LSTM-based model in most cases despite thorough tuning. Under\nvarious fine-grained label sets and a cross-linguistic setting, our feedforward\nmodels perform consistently better or at least just as well as systems that\nrequire hand-crafted surface features. Our models present the first neural\nChinese discourse parser in the style of Chinese Discourse Treebank, showing\nthat our results hold cross-linguistically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01994v2",
            "date": "2016-06-07 01:36:07+00:00",
            "title": "CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases",
            "authors": [
                "Zihang Dai",
                "Lei Li",
                "Wei Xu"
            ],
            "abstract": "How can we enable computers to automatically answer questions like \"Who\ncreated the character Harry Potter\"? Carefully built knowledge bases provide\nrich sources of facts. However, it remains a challenge to answer factoid\nquestions raised in natural language due to numerous expressions of one\nquestion. In particular, we focus on the most common questions --- ones that\ncan be answered with a single fact in the knowledge base. We propose CFO, a\nConditional Focused neural-network-based approach to answering factoid\nquestions with knowledge bases. Our approach first zooms in a question to find\nmore probable candidate subject mentions, and infers the final answers with a\nunified conditional probabilistic framework. Powered by deep recurrent neural\nnetworks and neural embeddings, our proposed CFO achieves an accuracy of 75.7%\non a dataset of 108k questions - the largest public one to date. It outperforms\nthe current state of the art by an absolute margin of 11.8%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02003v1",
            "date": "2016-06-07 02:28:19+00:00",
            "title": "Memory-enhanced Decoder for Neural Machine Translation",
            "authors": [
                "Mingxuan Wang",
                "Zhengdong Lu",
                "Hang Li",
                "Qun Liu"
            ],
            "abstract": "We propose to enhance the RNN decoder in a neural machine translator (NMT)\nwith external memory, as a natural but powerful extension to the state in the\ndecoding RNN. This memory-enhanced RNN decoder is called \\textsc{MemDec}. At\neach time during decoding, \\textsc{MemDec} will read from this memory and write\nto this memory once, both with content-based addressing. Unlike the unbounded\nmemory in previous work\\cite{RNNsearch} to store the representation of source\nsentence, the memory in \\textsc{MemDec} is a matrix with pre-determined size\ndesigned to better capture the information important for the decoding process\nat each time step. Our empirical study on Chinese-English translation shows\nthat it can improve by $4.8$ BLEU upon Groundhog and $5.3$ BLEU upon on Moses,\nyielding the best performance achieved with the same training set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02006v2",
            "date": "2016-06-07 02:40:42+00:00",
            "title": "Incorporating Discrete Translation Lexicons into Neural Machine Translation",
            "authors": [
                "Philip Arthur",
                "Graham Neubig",
                "Satoshi Nakamura"
            ],
            "abstract": "Neural machine translation (NMT) often makes mistakes in translating\nlow-frequency content words that are essential to understanding the meaning of\nthe sentence. We propose a method to alleviate this problem by augmenting NMT\nsystems with discrete translation lexicons that efficiently encode translations\nof these low-frequency words. We describe a method to calculate the lexicon\nprobability of the next word in the translation candidate by using the\nattention vector of the NMT model to select which source word lexical\nprobabilities the model should focus on. We test two methods to combine this\nprobability with the standard NMT probability: (1) using it as a bias, and (2)\nlinear interpolation. Experiments on two corpora show an improvement of 2.0-2.3\nBLEU and 0.13-0.44 NIST score, and faster convergence time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02012v1",
            "date": "2016-06-07 03:38:46+00:00",
            "title": "Can neural machine translation do simultaneous translation?",
            "authors": [
                "Kyunghyun Cho",
                "Masha Esipova"
            ],
            "abstract": "We investigate the potential of attention-based neural machine translation in\nsimultaneous translation. We introduce a novel decoding algorithm, called\nsimultaneous greedy decoding, that allows an existing neural machine\ntranslation model to begin translating before a full source sentence is\nreceived. This approach is unique from previous works on simultaneous\ntranslation in that segmentation and translation are done jointly to maximize\nthe translation quality and that translating each segment is strongly\nconditioned on all the previous segments. This paper presents a first step\ntoward building a full simultaneous translation system based on neural machine\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02126v4",
            "date": "2016-06-07 13:00:48+00:00",
            "title": "Supervised Syntax-based Alignment between English Sentences and Abstract Meaning Representation Graphs",
            "authors": [
                "Chenhui Chu",
                "Sadao Kurohashi"
            ],
            "abstract": "As alignment links are not given between English sentences and Abstract\nMeaning Representation (AMR) graphs in the AMR annotation, automatic alignment\nbecomes indispensable for training an AMR parser. Previous studies formalize it\nas a string-to-string problem and solve it in an unsupervised way, which\nsuffers from data sparseness due to the small size of training data for\nEnglish-AMR alignment. In this paper, we formalize it as a syntax-based\nalignment problem and solve it in a supervised manner based on syntax trees,\nwhich can address the data sparseness problem by generalizing English-AMR\ntokens to syntax tags. Experiments verify the effectiveness of the proposed\nmethod not only for English-AMR alignment, but also for AMR parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02270v2",
            "date": "2016-06-07 19:27:04+00:00",
            "title": "Natural Language Comprehension with the EpiReader",
            "authors": [
                "Adam Trischler",
                "Zheng Ye",
                "Xingdi Yuan",
                "Kaheer Suleman"
            ],
            "abstract": "We present the EpiReader, a novel model for machine comprehension of text.\nMachine comprehension of unstructured, real-world text is a major research goal\nfor natural language processing. Current tests of machine comprehension pose\nquestions whose answers can be inferred from some supporting text, and evaluate\na model's response to the questions. The EpiReader is an end-to-end neural\nmodel comprising two components: the first component proposes a small set of\ncandidate answers after comparing a question to its supporting text, and the\nsecond component formulates hypotheses using the proposed candidates and the\nquestion, then reranks the hypotheses based on their estimated concordance with\nthe supporting text. We present experiments demonstrating that the EpiReader\nsets a new state-of-the-art on the CNN and Children's Book Test machine\ncomprehension benchmarks, outperforming previous neural models by a significant\nmargin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02342v3",
            "date": "2016-06-07 21:58:41+00:00",
            "title": "Optimizing Spectral Learning for Parsing",
            "authors": [
                "Shashi Narayan",
                "Shay B. Cohen"
            ],
            "abstract": "We describe a search algorithm for optimizing the number of latent states\nwhen estimating latent-variable PCFGs with spectral methods. Our results show\nthat contrary to the common belief that the number of latent states for each\nnonterminal in an L-PCFG can be decided in isolation with spectral methods,\nparsing results significantly improve if the number of latent states for each\nnonterminal is globally optimized, while taking into account interactions\nbetween the different nonterminals. In addition, we contribute an empirical\nanalysis of spectral algorithms on eight morphologically rich languages:\nBasque, French, German, Hebrew, Hungarian, Korean, Polish and Swedish. Our\nresults show that our estimation consistently performs better or close to\ncoarse-to-fine expectation-maximization techniques for these languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02461v1",
            "date": "2016-06-08 09:12:17+00:00",
            "title": "Learning Semantically and Additively Compositional Distributional Representations",
            "authors": [
                "Ran Tian",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "This paper connects a vector-based composition model to a formal semantics,\nthe Dependency-based Compositional Semantics (DCS). We show theoretical\nevidence that the vector compositions in our model conform to the logic of DCS.\nExperimentally, we show that vector-based composition brings a strong ability\nto calculate similar phrases as similar vectors, achieving near\nstate-of-the-art on a wide range of phrase similarity tasks and relation\nclassification; meanwhile, DCS can guide building vectors for structured\nqueries that can be directly executed. We evaluate this utility on sentence\ncompletion task and report a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02514v1",
            "date": "2016-06-08 11:22:12+00:00",
            "title": "DefExt: A Semi Supervised Definition Extraction Tool",
            "authors": [
                "Luis Espinosa-Anke",
                "Roberto Carlini",
                "Horacio Saggion",
                "Francesco Ronzano"
            ],
            "abstract": "We present DefExt, an easy to use semi supervised Definition Extraction Tool.\nDefExt is designed to extract from a target corpus those textual fragments\nwhere a term is explicitly mentioned together with its core features, i.e. its\ndefinition. It works on the back of a Conditional Random Fields based\nsequential labeling algorithm and a bootstrapping approach. Bootstrapping\nenables the model to gradually become more aware of the idiosyncrasies of the\ntarget corpus. In this paper we describe the main components of the toolkit as\nwell as experimental results stemming from both automatic and manual\nevaluation. We release DefExt as open source along with the necessary files to\nrun it in any Unix machine. We also provide access to training and test data\nfor immediate use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02529v1",
            "date": "2016-06-08 12:44:51+00:00",
            "title": "Coordination Annotation Extension in the Penn Tree Bank",
            "authors": [
                "Jessica Ficler",
                "Yoav Goldberg"
            ],
            "abstract": "Coordination is an important and common syntactic construction which is not\nhandled well by state of the art parsers. Coordinations in the Penn Treebank\nare missing internal structure in many cases, do not include explicit marking\nof the conjuncts and contain various errors and inconsistencies. In this work,\nwe initiated manual annotation process for solving these issues. We identify\nthe different elements in a coordination phrase and label each element with its\nfunction. We add phrase boundaries when these are missing, unify\ninconsistencies, and fix errors. The outcome is an extension of the PTB that\nincludes consistent and detailed structures for coordinations. We make the\ncoordination annotation publicly available, in hope that they will facilitate\nfurther research into coordination disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02601v1",
            "date": "2016-06-08 15:24:22+00:00",
            "title": "A Joint Model for Word Embedding and Word Morphology",
            "authors": [
                "Kris Cao",
                "Marek Rei"
            ],
            "abstract": "This paper presents a joint model for performing unsupervised morphological\nanalysis on words, and learning a character-level composition function from\nmorphemes to word embeddings. Our model splits individual words into segments,\nand weights each segment according to its ability to predict context words. Our\nmorphological analysis is comparable to dedicated morphological analyzers at\nthe task of morpheme boundary recovery, and also performs better than\nword-based embedding models at the task of syntactic analogy answering.\nFinally, we show that incorporating morphology explicitly into character-level\nmodels help them produce embeddings for unseen words which correlate better\nwith human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02638v1",
            "date": "2016-06-08 16:56:19+00:00",
            "title": "Addressing Limited Data for Textual Entailment Across Domains",
            "authors": [
                "Chaitanya Shivade",
                "Preethi Raghavan",
                "Siddharth Patwardhan"
            ],
            "abstract": "We seek to address the lack of labeled data (and high cost of annotation) for\ntextual entailment in some domains. To that end, we first create (for\nexperimental purposes) an entailment dataset for the clinical domain, and a\nhighly competitive supervised entailment system, ENT, that is effective (out of\nthe box) on two domains. We then explore self-training and active learning\nstrategies to address the lack of labeled data. With self-training, we\nsuccessfully exploit unlabeled data to improve over ENT by 15% F-score on the\nnewswire domain, and 13% F-score on clinical data. On the other hand, our\nactive learning experiments demonstrate that we can match (and even beat) ENT\nusing only 6.6% of the training data in the clinical domain, and only 5.8% of\nthe training data in the newswire domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02680v1",
            "date": "2016-06-08 18:36:09+00:00",
            "title": "First Result on Arabic Neural Machine Translation",
            "authors": [
                "Amjad Almahairi",
                "Kyunghyun Cho",
                "Nizar Habash",
                "Aaron Courville"
            ],
            "abstract": "Neural machine translation has become a major alternative to widely used\nphrase-based statistical machine translation. We notice however that much of\nresearch on neural machine translation has focused on European languages\ndespite its language agnostic nature. In this paper, we apply neural machine\ntranslation to the task of Arabic translation (Ar<->En) and compare it against\na standard phrase-based translation system. We run extensive comparison using\nvarious configurations in preprocessing Arabic script and show that the\nphrase-based and neural translation systems perform comparably to each other\nand that proper preprocessing of Arabic script has a similar effect on both of\nthe systems. We however observe that the neural machine translation\nsignificantly outperform the phrase-based system on an out-of-domain test set,\nmaking it attractive for real-world deployment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02785v1",
            "date": "2016-06-09 00:15:23+00:00",
            "title": "Neural Network-Based Abstract Generation for Opinions and Arguments",
            "authors": [
                "Lu Wang",
                "Wang Ling"
            ],
            "abstract": "We study the problem of generating abstractive summaries for opinionated\ntext. We propose an attention-based neural network model that is able to absorb\ninformation from multiple text units to construct informative, concise, and\nfluent summaries. An importance-based sampling method is designed to allow the\nencoder to integrate information from an important subset of input. Automatic\nevaluation indicates that our system outperforms state-of-the-art abstractive\nand extractive summarization systems on two newly collected datasets of movie\nreviews and arguments. Our system summaries are also rated as more informative\nand grammatical in human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02820v2",
            "date": "2016-06-09 04:28:10+00:00",
            "title": "Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora",
            "authors": [
                "William L. Hamilton",
                "Kevin Clark",
                "Jure Leskovec",
                "Dan Jurafsky"
            ],
            "abstract": "A word's sentiment depends on the domain in which it is used. Computational\nsocial science research thus requires sentiment lexicons that are specific to\nthe domains being studied. We combine domain-specific word embeddings with a\nlabel propagation framework to induce accurate domain-specific sentiment\nlexicons using small sets of seed words, achieving state-of-the-art performance\ncompetitive with approaches that rely on hand-curated resources. Using our\nframework we perform two large-scale empirical studies to quantify the extent\nto which sentiment varies across time and between communities. We induce and\nrelease historical sentiment lexicons for 150 years of English and\ncommunity-specific sentiment lexicons for 250 online communities from the\nsocial media forum Reddit. The historical lexicons show that more than 5% of\nsentiment-bearing (non-neutral) English words completely switched polarity\nduring the last 150 years, and the community-specific lexicons highlight how\nsentiment varies drastically between different communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02821v2",
            "date": "2016-06-09 04:42:12+00:00",
            "title": "Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change",
            "authors": [
                "William L. Hamilton",
                "Jure Leskovec",
                "Dan Jurafsky"
            ],
            "abstract": "Words shift in meaning for many reasons, including cultural factors like new\ntechnologies and regular linguistic processes like subjectification.\nUnderstanding the evolution of language and culture requires disentangling\nthese underlying causes. Here we show how two different distributional measures\ncan be used to detect two different types of semantic change. The first\nmeasure, which has been used in many previous works, analyzes global shifts in\na word's distributional semantics, it is sensitive to changes due to regular\nprocesses of linguistic drift, such as the semantic generalization of promise\n(\"I promise.\" -> \"It promised to be exciting.\"). The second measure, which we\ndevelop here, focuses on local changes to a word's nearest semantic neighbors;\nit is more sensitive to cultural shifts, such as the change in the meaning of\ncell (\"prison cell\" -> \"cell phone\"). Comparing measurements made by these two\nmethods allows researchers to determine whether changes are more cultural or\nlinguistic in nature, a distinction that is essential for work in the digital\nhumanities and historical linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02891v2",
            "date": "2016-06-09 10:06:28+00:00",
            "title": "Edinburgh Neural Machine Translation Systems for WMT 16",
            "authors": [
                "Rico Sennrich",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "We participated in the WMT 2016 shared news translation task by building\nneural translation systems for four language pairs, each trained in both\ndirections: English<->Czech, English<->German, English<->Romanian and\nEnglish<->Russian. Our systems are based on an attentional encoder-decoder,\nusing BPE subword segmentation for open-vocabulary translation with a fixed\nvocabulary. We experimented with using automatic back-translations of the\nmonolingual News corpus as additional training data, pervasive dropout, and\ntarget-bidirectional models. All reported methods give substantial\nimprovements, and we see improvements of 4.3--11.2 BLEU over our baseline\nsystems. In the human evaluation, our systems were the (tied) best constrained\nsystem for 7 out of 8 translation directions in which we participated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02892v2",
            "date": "2016-06-09 10:12:36+00:00",
            "title": "Linguistic Input Features Improve Neural Machine Translation",
            "authors": [
                "Rico Sennrich",
                "Barry Haddow"
            ],
            "abstract": "Neural machine translation has recently achieved impressive results, while\nusing little in the way of external linguistic information. In this paper we\nshow that the strong learning capability of neural MT models does not make\nlinguistic features redundant; they can be easily incorporated to provide\nfurther improvements in performance. We generalize the embedding layer of the\nencoder in the attentional encoder--decoder architecture to support the\ninclusion of arbitrary features, in addition to the baseline word feature. We\nadd morphological features, part-of-speech tags, and syntactic dependency\nlabels as input features to English<->German, and English->Romanian neural\nmachine translation systems. In experiments on WMT16 training and test sets, we\nfind that linguistic input features improve model quality according to three\nmetrics: perplexity, BLEU and CHRF3. An open-source implementation of our\nneural MT system is available, as are sample files and configurations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03126v2",
            "date": "2016-06-09 21:33:55+00:00",
            "title": "Key-Value Memory Networks for Directly Reading Documents",
            "authors": [
                "Alexander Miller",
                "Adam Fisch",
                "Jesse Dodge",
                "Amir-Hossein Karimi",
                "Antoine Bordes",
                "Jason Weston"
            ],
            "abstract": "Directly reading documents and being able to answer questions from them is an\nunsolved challenge. To avoid its inherent difficulty, question answering (QA)\nhas been directed towards using Knowledge Bases (KBs) instead, which has proven\neffective. Unfortunately KBs often suffer from being too restrictive, as the\nschema cannot support certain types of answers, and too sparse, e.g. Wikipedia\ncontains much more information than Freebase. In this work we introduce a new\nmethod, Key-Value Memory Networks, that makes reading documents more viable by\nutilizing different encodings in the addressing and output stages of the memory\nread operation. To compare using KBs, information extraction or Wikipedia\ndocuments directly in a single framework we construct an analysis tool,\nWikiMovies, a QA dataset that contains raw text alongside a preprocessed KB, in\nthe domain of movies. Our method reduces the gap between all three settings. It\nalso achieves state-of-the-art results on the existing WikiQA benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03143v1",
            "date": "2016-06-09 23:32:41+00:00",
            "title": "PerSum: Novel Systems for Document Summarization in Persian",
            "authors": [
                "Saeid Parvandeh",
                "Shibamouli Lahiri",
                "Fahimeh Boroumand"
            ],
            "abstract": "In this paper we explore the problem of document summarization in Persian\nlanguage from two distinct angles. In our first approach, we modify a popular\nand widely cited Persian document summarization framework to see how it works\non a realistic corpus of news articles. Human evaluation on generated summaries\nshows that graph-based methods perform better than the modified systems. We\ncarry this intuition forward in our second approach, and probe deeper into the\nnature of graph-based systems by designing several summarizers based on\ncentrality measures. Ad hoc evaluation using ROUGE score on these summarizers\nsuggests that there is a small class of centrality measures that perform better\nthan three strong unsupervised baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03192v1",
            "date": "2016-06-10 05:55:58+00:00",
            "title": "PSDVec: a Toolbox for Incremental and Scalable Word Embedding",
            "authors": [
                "Shaohua Li",
                "Jun Zhu",
                "Chunyan Miao"
            ],
            "abstract": "PSDVec is a Python/Perl toolbox that learns word embeddings, i.e. the mapping\nof words in a natural language to continuous vectors which encode the\nsemantic/syntactic regularities between the words. PSDVec implements a word\nembedding learning method based on a weighted low-rank positive semidefinite\napproximation. To scale up the learning process, we implement a blockwise\nonline learning algorithm to learn the embeddings incrementally. This strategy\ngreatly reduces the learning time of word embeddings on a large vocabulary, and\ncan learn the embeddings of new words without re-learning the whole vocabulary.\nOn 9 word similarity/analogy benchmark sets and 2 Natural Language Processing\n(NLP) tasks, PSDVec produces embeddings that has the best average performance\namong popular word embedding tools. PSDVec provides a new option for NLP\npractitioners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03391v2",
            "date": "2016-06-10 16:54:51+00:00",
            "title": "Simple Question Answering by Attentive Convolutional Neural Network",
            "authors": [
                "Wenpeng Yin",
                "Mo Yu",
                "Bing Xiang",
                "Bowen Zhou",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This work focuses on answering single-relation factoid questions over\nFreebase. Each question can acquire the answer from a single fact of form\n(subject, predicate, object) in Freebase. This task, simple question answering\n(SimpleQA), can be addressed via a two-step pipeline: entity linking and fact\nselection. In fact selection, we match the subject entity in a fact candidate\nwith the entity mention in the question by a character-level convolutional\nneural network (char-CNN), and match the predicate in that fact with the\nquestion by a word-level CNN (word-CNN). This work makes two main\ncontributions. (i) A simple and effective entity linker over Freebase is\nproposed. Our entity linker outperforms the state-of-the-art entity linker over\nSimpleQA task. (ii) A novel attentive maxpooling is stacked over word-CNN, so\nthat the predicate representation can be matched with the predicate-focused\nquestion representation more effectively. Experiments show that our system sets\nnew state-of-the-art in this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03398v2",
            "date": "2016-06-10 17:14:11+00:00",
            "title": "Bootstrapping Distantly Supervised IE using Joint Learning and Small Well-structured Corpora",
            "authors": [
                "Lidong Bing",
                "Bhuwan Dhingra",
                "Kathryn Mazaitis",
                "Jong Hyuk Park",
                "William W. Cohen"
            ],
            "abstract": "We propose a framework to improve performance of distantly-supervised\nrelation extraction, by jointly learning to solve two related tasks:\nconcept-instance extraction and relation extraction. We combine this with a\nnovel use of document structure: in some small, well-structured corpora,\nsections can be identified that correspond to relation arguments, and\ndistantly-labeled examples from such sections tend to have good precision.\nUsing these as seeds we extract additional relation examples by applying label\npropagation on a graph composed of noisy examples extracted from a large\nunstructured testing corpus. Combined with the soft constraint that concept\nexamples should have the same type as the second argument of the relation, we\nget significant improvements over several state-of-the-art approaches to\ndistantly-supervised relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03622v1",
            "date": "2016-06-11 20:34:09+00:00",
            "title": "Data Recombination for Neural Semantic Parsing",
            "authors": [
                "Robin Jia",
                "Percy Liang"
            ],
            "abstract": "Modeling crisp logical regularities is crucial in semantic parsing, making it\ndifficult for neural models with no task-specific prior knowledge to achieve\ngood results. In this paper, we introduce data recombination, a novel framework\nfor injecting such prior knowledge into a model. From the training data, we\ninduce a high-precision synchronous context-free grammar, which captures\nimportant conditional independence properties commonly found in semantic\nparsing. We then train a sequence-to-sequence recurrent network (RNN) model\nwith a novel attention-based copying mechanism on datapoints sampled from this\ngrammar, thereby teaching the model about these structural properties. Data\nrecombination improves the accuracy of our RNN model on three semantic parsing\ndatasets, leading to new state-of-the-art performance on the standard GeoQuery\ndataset for models with comparable supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03632v3",
            "date": "2016-06-11 21:24:43+00:00",
            "title": "Natural Language Generation in Dialogue using Lexicalized and Delexicalized Data",
            "authors": [
                "Shikhar Sharma",
                "Jing He",
                "Kaheer Suleman",
                "Hannes Schulz",
                "Philip Bachman"
            ],
            "abstract": "Natural language generation plays a critical role in spoken dialogue systems.\nWe present a new approach to natural language generation for task-oriented\ndialogue using recurrent neural networks in an encoder-decoder framework. In\ncontrast to previous work, our model uses both lexicalized and delexicalized\ncomponents i.e. slot-value pairs for dialogue acts, with slots and\ncorresponding values aligned together. This allows our model to learn from all\navailable data including the slot-value pairing, rather than being restricted\nto delexicalized slots. We show that this helps our model generate more natural\nsentences with better grammar. We further improve our model's performance by\ntransferring weights learnt from a pretrained sentence auto-encoder. Human\nevaluation of our best-performing model indicates that it generates sentences\nwhich users find more appealing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03676v2",
            "date": "2016-06-12 08:06:55+00:00",
            "title": "External Lexical Information for Multilingual Part-of-Speech Tagging",
            "authors": [
                "Beno\u00eet Sagot"
            ],
            "abstract": "Morphosyntactic lexicons and word vector representations have both proven\nuseful for improving the accuracy of statistical part-of-speech taggers. Here\nwe compare the performances of four systems on datasets covering 16 languages,\ntwo of these systems being feature-based (MEMMs and CRFs) and two of them being\nneural-based (bi-LSTMs). We show that, on average, all four approaches perform\nsimilarly and reach state-of-the-art results. Yet better performances are\nobtained with our feature-based models on lexically richer datasets (e.g. for\nmorphologically rich languages), whereas neural-based results are higher on\ndatasets with less lexical variability (e.g. for English). These conclusions\nhold in particular for the MEMM models relying on our system MElt, which\nbenefited from newly designed features. This shows that, under certain\nconditions, feature-based approaches enriched with morphosyntactic lexicons are\ncompetitive with respect to neural methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03821v2",
            "date": "2016-06-13 06:17:32+00:00",
            "title": "Learning to Generate Compositional Color Descriptions",
            "authors": [
                "Will Monroe",
                "Noah D. Goodman",
                "Christopher Potts"
            ],
            "abstract": "The production of color language is essential for grounded language\ngeneration. Color descriptions have many challenging properties: they can be\nvague, compositionally complex, and denotationally rich. We present an\neffective approach to generating color descriptions using recurrent neural\nnetworks and a Fourier-transformed color representation. Our model outperforms\nprevious work on a conditional language modeling task over a large corpus of\nnaturalistic color descriptions. In addition, probing the model's output\nreveals that it can accurately produce not only basic color terms but also\ndescriptors with non-convex denotations (\"greenish\"), bare modifiers (\"bright\",\n\"dull\"), and compositional phrases (\"faded teal\") not seen in training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04164v1",
            "date": "2016-06-13 22:40:33+00:00",
            "title": "Zero-Resource Translation with Multi-Lingual Neural Machine Translation",
            "authors": [
                "Orhan Firat",
                "Baskaran Sankaran",
                "Yaser Al-Onaizan",
                "Fatos T. Yarman Vural",
                "Kyunghyun Cho"
            ],
            "abstract": "In this paper, we propose a novel finetuning algorithm for the recently\nintroduced multi-way, mulitlingual neural machine translate that enables\nzero-resource machine translation. When used together with novel many-to-one\ntranslation strategies, we empirically show that this finetuning algorithm\nallows the multi-way, multilingual model to translate a zero-resource language\npair (1) as well as a single-pair neural translation model trained with up to\n1M direct parallel sentences of the same language pair and (2) better than\npivot-based translation strategy, while keeping only one additional copy of\nattention-related parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04212v4",
            "date": "2016-06-14 06:49:40+00:00",
            "title": "Active Discriminative Text Representation Learning",
            "authors": [
                "Ye Zhang",
                "Matthew Lease",
                "Byron C. Wallace"
            ],
            "abstract": "We propose a new active learning (AL) method for text classification with\nconvolutional neural networks (CNNs). In AL, one selects the instances to be\nmanually labeled with the aim of maximizing model performance with minimal\neffort. Neural models capitalize on word embeddings as representations\n(features), tuning these to the task at hand. We argue that AL strategies for\nmulti-layered neural models should focus on selecting instances that most\naffect the embedding space (i.e., induce discriminative word representations).\nThis is in contrast to traditional AL approaches (e.g., entropy-based\nuncertainty sampling), which specify higher level objectives. We propose a\nsimple approach for sentence classification that selects instances containing\nwords whose embeddings are likely to be updated with the greatest magnitude,\nthereby rapidly learning discriminative, task-specific embeddings. We extend\nthis approach to document classification by jointly considering: (1) the\nexpected changes to the constituent word representations; and (2) the model's\ncurrent overall uncertainty regarding the instance. The relative emphasis\nplaced on these criteria is governed by a stochastic process that favors\nselecting instances likely to improve representations at the outset of\nlearning, and then shifts toward general uncertainty sampling as AL progresses.\nEmpirical results show that our method outperforms baseline AL approaches on\nboth sentence and document classification tasks. We also show that, as\nexpected, the method quickly learns discriminative word embeddings. To the best\nof our knowledge, this is the first work on AL addressing neural models for\ntext classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04279v1",
            "date": "2016-06-14 09:43:36+00:00",
            "title": "Cross-Lingual Morphological Tagging for Low-Resource Languages",
            "authors": [
                "Jan Buys",
                "Jan A. Botha"
            ],
            "abstract": "Morphologically rich languages often lack the annotated linguistic resources\nrequired to develop accurate natural language processing tools. We propose\nmodels suitable for training morphological taggers with rich tagsets for\nlow-resource languages without using direct supervision. Our approach extends\nexisting approaches of projecting part-of-speech tags across languages, using\nbitext to infer constraints on the possible tags for a given word type or\ntoken. We propose a tagging model using Wsabie, a discriminative\nembedding-based model with rank-based learning. In our evaluation on 11\nlanguages, on average this model performs on par with a baseline\nweakly-supervised HMM, while being more scalable. Multilingual experiments show\nthat the method performs best when projecting between related language pairs.\nDespite the inherently lossy projection, we show that the morphological tags\npredicted by our models improve the downstream performance of a parser by +0.6\nLAS on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04300v2",
            "date": "2016-06-14 10:52:21+00:00",
            "title": "Neural Word Segmentation Learning for Chinese",
            "authors": [
                "Deng Cai",
                "Hai Zhao"
            ],
            "abstract": "Most previous approaches to Chinese word segmentation formalize this problem\nas a character-based sequence labeling task where only contextual information\nwithin fixed sized local windows and simple interactions between adjacent tags\ncan be captured. In this paper, we propose a novel neural framework which\nthoroughly eliminates context windows and can utilize complete segmentation\nhistory. Our model employs a gated combination neural network over characters\nto produce distributed representations of word candidates, which are then given\nto a long short-term memory (LSTM) language scoring model. Experiments on the\nbenchmark datasets show that without the help of feature engineering as most\nexisting approaches, our models achieve competitive or better performances with\nprevious state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04503v1",
            "date": "2016-06-14 19:00:59+00:00",
            "title": "Shallow Discourse Parsing Using Distributed Argument Representations and Bayesian Optimization",
            "authors": [
                "Akanksha",
                "Jacob Eisenstein"
            ],
            "abstract": "This paper describes the Georgia Tech team's approach to the CoNLL-2016\nsupplementary evaluation on discourse relation sense classification. We use\nlong short-term memories (LSTM) to induce distributed representations of each\nargument, and then combine these representations with surface features in a\nneural network. The architecture of the neural network is determined by\nBayesian hyperparameter search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04596v3",
            "date": "2016-06-15 00:22:27+00:00",
            "title": "Semi-Supervised Learning for Neural Machine Translation",
            "authors": [
                "Yong Cheng",
                "Wei Xu",
                "Zhongjun He",
                "Wei He",
                "Hua Wu",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "While end-to-end neural machine translation (NMT) has made remarkable\nprogress recently, NMT systems only rely on parallel corpora for parameter\nestimation. Since parallel corpora are usually limited in quantity, quality,\nand coverage, especially for low-resource languages, it is appealing to exploit\nmonolingual corpora to improve NMT. We propose a semi-supervised approach for\ntraining NMT models on the concatenation of labeled (parallel corpora) and\nunlabeled (monolingual corpora) data. The central idea is to reconstruct the\nmonolingual corpora using an autoencoder, in which the source-to-target and\ntarget-to-source translation models serve as the encoder and decoder,\nrespectively. Our approach can not only exploit the monolingual corpora of the\ntarget language, but also of the source language. Experiments on the\nChinese-English dataset show that our approach achieves significant\nimprovements over state-of-the-art SMT and NMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04597v1",
            "date": "2016-06-15 00:28:51+00:00",
            "title": "Agreement-based Learning of Parallel Lexicons and Phrases from Non-Parallel Corpora",
            "authors": [
                "Chunyang Liu",
                "Yang Liu",
                "Huanbo Luan",
                "Maosong Sun",
                "Heng Yu"
            ],
            "abstract": "We introduce an agreement-based approach to learning parallel lexicons and\nphrases from non-parallel corpora. The basic idea is to encourage two\nasymmetric latent-variable translation models (i.e., source-to-target and\ntarget-to-source) to agree on identifying latent phrase and word alignments.\nThe agreement is defined at both word and phrase levels. We develop a Viterbi\nEM algorithm for jointly training the two unidirectional models efficiently.\nExperiments on the Chinese-English dataset show that agreement-based learning\nsignificantly improves both alignment and translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04640v1",
            "date": "2016-06-15 04:47:43+00:00",
            "title": "Siamese CBOW: Optimizing Word Embeddings for Sentence Representations",
            "authors": [
                "Tom Kenter",
                "Alexey Borisov",
                "Maarten de Rijke"
            ],
            "abstract": "We present the Siamese Continuous Bag of Words (Siamese CBOW) model, a neural\nnetwork for efficient estimation of high-quality sentence embeddings. Averaging\nthe embeddings of words in a sentence has proven to be a surprisingly\nsuccessful and efficient way of obtaining sentence embeddings. However, word\nembeddings trained with the methods currently available are not optimized for\nthe task of sentence representation, and, thus, likely to be suboptimal.\nSiamese CBOW handles this problem by training word embeddings directly for the\npurpose of being averaged. The underlying neural network learns word embeddings\nby predicting, from a sentence representation, its surrounding sentences. We\nshow the robustness of the Siamese CBOW model by evaluating it on 20 datasets\nstemming from a wide variety of sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04754v1",
            "date": "2016-06-15 13:27:16+00:00",
            "title": "A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation",
            "authors": [
                "Amrita Saha",
                "Mitesh M. Khapra",
                "Sarath Chandar",
                "Janarthanan Rajendran",
                "Kyunghyun Cho"
            ],
            "abstract": "Interlingua based Machine Translation (MT) aims to encode multiple languages\ninto a common linguistic representation and then decode sentences in multiple\ntarget languages from this representation. In this work we explore this idea in\nthe context of neural encoder decoder architectures, albeit on a smaller scale\nand without MT as the end goal. Specifically, we consider the case of three\nlanguages or modalities X, Z and Y wherein we are interested in generating\nsequences in Y starting from information available in X. However, there is no\nparallel training data available between X and Y but, training data is\navailable between X & Z and Z & Y (as is often the case in many real world\napplications). Z thus acts as a pivot/bridge. An obvious solution, which is\nperhaps less elegant but works very well in practice is to train a two stage\nmodel which first converts from X to Z and then from Z to Y. Instead we explore\nan interlingua inspired solution which jointly learns to do the following (i)\nencode X and Z to a common representation and (ii) decode Y from this common\nrepresentation. We evaluate our model on two tasks: (i) bridge transliteration\nand (ii) bridge captioning. We report promising results in both these\napplications and believe that this is a right step towards truly interlingua\ninspired encoder decoder architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04835v4",
            "date": "2016-06-15 16:14:09+00:00",
            "title": "Learning Word Sense Embeddings from Word Sense Definitions",
            "authors": [
                "Qi Li",
                "Tianshi Li",
                "Baobao Chang"
            ],
            "abstract": "Word embeddings play a significant role in many modern NLP systems. Since\nlearning one representation per word is problematic for polysemous words and\nhomonymous words, researchers propose to use one embedding per word sense.\nTheir approaches mainly train word sense embeddings on a corpus. In this paper,\nwe propose to use word sense definitions to learn one embedding per word sense.\nExperimental results on word similarity tasks and a word sense disambiguation\ntask show that word sense embeddings produced by our approach are of high\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04870v1",
            "date": "2016-06-15 17:23:12+00:00",
            "title": "Smart Reply: Automated Response Suggestion for Email",
            "authors": [
                "Anjuli Kannan",
                "Karol Kurach",
                "Sujith Ravi",
                "Tobias Kaufmann",
                "Andrew Tomkins",
                "Balint Miklos",
                "Greg Corrado",
                "Laszlo Lukacs",
                "Marina Ganea",
                "Peter Young",
                "Vivek Ramavajjala"
            ],
            "abstract": "In this paper we propose and investigate a novel end-to-end method for\nautomatically generating short email responses, called Smart Reply. It\ngenerates semantically diverse suggestions that can be used as complete email\nresponses with just one tap on mobile. The system is currently used in Inbox by\nGmail and is responsible for assisting with 10% of all mobile responses. It is\ndesigned to work at very high throughput and process hundreds of millions of\nmessages daily. The system exploits state-of-the-art, large-scale deep\nlearning.\n  We describe the architecture of the system as well as the challenges that we\nfaced while building it, like response diversity and scalability. We also\nintroduce a new method for semantic clustering of user-generated content that\nrequires only a modest amount of explicitly labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04963v1",
            "date": "2016-06-15 20:08:01+00:00",
            "title": "The Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16",
            "authors": [
                "Felix Stahlberg",
                "Eva Hasler",
                "Bill Byrne"
            ],
            "abstract": "This paper presents the University of Cambridge submission to WMT16.\nMotivated by the complementary nature of syntactical machine translation and\nneural machine translation (NMT), we exploit the synergies of Hiero and NMT in\ndifferent combination schemes. Starting out with a simple neural lattice\nrescoring approach, we show that the Hiero lattices are often too narrow for\nNMT ensembles. Therefore, instead of a hard restriction of the NMT search space\nto the lattice, we propose to loosely couple NMT and Hiero by composition with\na modified version of the edit distance transducer. The loose combination\noutperforms lattice rescoring, especially when using multiple NMT systems in an\nensemble.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05029v2",
            "date": "2016-06-16 02:20:04+00:00",
            "title": "No Need to Pay Attention: Simple Recurrent Neural Networks Work! (for Answering \"Simple\" Questions)",
            "authors": [
                "Ferhan Ture",
                "Oliver Jojic"
            ],
            "abstract": "First-order factoid question answering assumes that the question can be\nanswered by a single fact in a knowledge base (KB). While this does not seem\nlike a challenging task, many recent attempts that apply either complex\nlinguistic reasoning or deep neural networks achieve 65%-76% accuracy on\nbenchmark sets. Our approach formulates the task as two machine learning\nproblems: detecting the entities in the question, and classifying the question\nas one of the relation types in the KB. We train a recurrent neural network to\nsolve each problem. On the SimpleQuestions dataset, our approach yields\nsubstantial improvements over previously published results --- even neural\nnetworks based on much more complex architectures. The simplicity of our\napproach also has practical advantages, such as efficiency and modularity, that\nare valuable especially in an industry setting. In fact, we present a\npreliminary analysis of the performance of our model on real queries from\nComcast's X1 entertainment platform with millions of users every day.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05250v3",
            "date": "2016-06-16 16:36:00+00:00",
            "title": "SQuAD: 100,000+ Questions for Machine Comprehension of Text",
            "authors": [
                "Pranav Rajpurkar",
                "Jian Zhang",
                "Konstantin Lopyrev",
                "Percy Liang"
            ],
            "abstract": "We present the Stanford Question Answering Dataset (SQuAD), a new reading\ncomprehension dataset consisting of 100,000+ questions posed by crowdworkers on\na set of Wikipedia articles, where the answer to each question is a segment of\ntext from the corresponding reading passage. We analyze the dataset to\nunderstand the types of reasoning required to answer the questions, leaning\nheavily on dependency and constituency trees. We build a strong logistic\nregression model, which achieves an F1 score of 51.0%, a significant\nimprovement over a simple baseline (20%). However, human performance (86.8%) is\nmuch higher, indicating that the dataset presents a good challenge problem for\nfuture research.\n  The dataset is freely available at https://stanford-qa.com",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05378v1",
            "date": "2016-06-16 21:57:11+00:00",
            "title": "Simpler Context-Dependent Logical Forms via Model Projections",
            "authors": [
                "Reginald Long",
                "Panupong Pasupat",
                "Percy Liang"
            ],
            "abstract": "We consider the task of learning a context-dependent mapping from utterances\nto denotations. With only denotations at training time, we must search over a\ncombinatorially large space of logical forms, which is even larger with\ncontext-dependent utterances. To cope with this challenge, we perform\nsuccessive projections of the full model onto simpler models that operate over\nequivalence classes of logical forms. Though less expressive, we find that\nthese simpler models are much faster and can be surprisingly effective.\nMoreover, they can be used to bootstrap the full model. Finally, we collected\nthree new context-dependent semantic parsing datasets, and develop a new\nleft-to-right parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05409v2",
            "date": "2016-06-17 02:49:52+00:00",
            "title": "Sense Embedding Learning for Word Sense Induction",
            "authors": [
                "Linfeng Song",
                "Zhiguo Wang",
                "Haitao Mi",
                "Daniel Gildea"
            ],
            "abstract": "Conventional word sense induction (WSI) methods usually represent each\ninstance with discrete linguistic features or cooccurrence features, and train\na model for each polysemous word individually. In this work, we propose to\nlearn sense embeddings for the WSI task. In the training stage, our method\ninduces several sense centroids (embedding) for each polysemous word. In the\ntesting stage, our method represents each instance as a contextual vector, and\ninduces its sense by finding the nearest sense centroid in the embedding space.\nThe advantages of our method are (1) distributed sense vectors are taken as the\nknowledge representations which are trained discriminatively, and usually have\nbetter performance than traditional count-based distributional models, and (2)\na general model for the whole vocabulary is jointly trained to induce sense\ncentroids under the mutlitask learning framework. Evaluated on SemEval-2010 WSI\ndataset, our method outperforms all participants and most of the recent\nstate-of-the-art methods. We further verify the two advantages by comparing\nwith carefully designed baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05491v1",
            "date": "2016-06-17 11:51:25+00:00",
            "title": "Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "We present a natural language generator based on the sequence-to-sequence\napproach that can be trained to produce natural language strings as well as\ndeep syntax dependency trees from input dialogue acts, and we use it to\ndirectly compare two-step generation with separate sentence planning and\nsurface realization stages to a joint, one-step approach. We were able to train\nboth setups successfully using very little training data. The joint setup\noffers better performance, surpassing state-of-the-art with regards to\nn-gram-based scores while providing more relevant outputs.",
            "journal ref": "Proceedings of the 54th Annual Meeting of the Association for\n  Computational Linguistics, pages 45-51, Berlin, Germany, August 7-12, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05545v2",
            "date": "2016-06-17 14:53:02+00:00",
            "title": "Universal, Unsupervised (Rule-Based), Uncovered Sentiment Analysis",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Miguel A. Alonso"
            ],
            "abstract": "We present a novel unsupervised approach for multilingual sentiment analysis\ndriven by compositional syntax-based rules. On the one hand, we exploit some of\nthe main advantages of unsupervised algorithms: (1) the interpretability of\ntheir output, in contrast with most supervised models, which behave as a black\nbox and (2) their robustness across different corpora and domains. On the other\nhand, by introducing the concept of compositional operations and exploiting\nsyntactic information in the form of universal dependencies, we tackle one of\ntheir main drawbacks: their rigidity on data that are structured differently\ndepending on the language concerned. Experiments show an improvement both over\nexisting unsupervised methods, and over state-of-the-art supervised models when\nevaluating outside their corpus of origin. Experiments also show how the same\ncompositional operations can be shared across languages. The system is\navailable at http://www.grupolys.org/software/UUUSA/",
            "journal ref": "Knowledge-Based Systems, 118:45-55, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05679v2",
            "date": "2016-06-17 21:19:35+00:00",
            "title": "Two Discourse Driven Language Models for Semantics",
            "authors": [
                "Haoruo Peng",
                "Dan Roth"
            ],
            "abstract": "Natural language understanding often requires deep semantic knowledge.\nExpanding on previous proposals, we suggest that some important aspects of\nsemantic knowledge can be modeled as a language model if done at an appropriate\nlevel of abstraction. We develop two distinct models that capture semantic\nframe chains and discourse information while abstracting over the specific\nmentions of predicates and entities. For each model, we investigate four\nimplementations: a \"standard\" N-gram language model and three discriminatively\ntrained \"neural\" language models that generate embeddings for semantic frames.\nThe quality of the semantic language models (SemLM) is evaluated both\nintrinsically, using perplexity and a narrative cloze test and extrinsically -\nwe show that our SemLM helps improve performance on semantic natural language\nprocessing tasks such as co-reference resolution and discourse parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05699v1",
            "date": "2016-06-17 22:52:09+00:00",
            "title": "Socially-Informed Timeline Generation for Complex Events",
            "authors": [
                "Lu Wang",
                "Claire Cardie",
                "Galen Marchetti"
            ],
            "abstract": "Existing timeline generation systems for complex events consider only\ninformation from traditional media, ignoring the rich social context provided\nby user-generated content that reveals representative public interests or\ninsightful opinions. We instead aim to generate socially-informed timelines\nthat contain both news article summaries and selected user comments. We present\nan optimization framework designed to balance topical cohesion between the\narticle and comment summaries along with their informativeness and coverage of\nthe event. Automatic evaluations on real-world datasets that cover four complex\nevents show that our system produces more informative timelines than\nstate-of-the-art systems. In human evaluation, the associated comment summaries\nare furthermore rated more insightful than editor's picks and comments ranked\nhighly by users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05702v1",
            "date": "2016-06-17 23:05:41+00:00",
            "title": "Query-Focused Opinion Summarization for User-Generated Content",
            "authors": [
                "Lu Wang",
                "Hema Raghavan",
                "Claire Cardie",
                "Vittorio Castelli"
            ],
            "abstract": "We present a submodular function-based framework for query-focused opinion\nsummarization. Within our framework, relevance ordering produced by a\nstatistical ranker, and information coverage with respect to topic distribution\nand diverse viewpoints are both encoded as submodular functions. Dispersion\nfunctions are utilized to minimize the redundancy. We are the first to evaluate\ndifferent metrics of text similarity for submodularity-based summarization\nmethods. By experimenting on community QA and blog summarization, we show that\nour system outperforms state-of-the-art approaches in both automatic evaluation\nand human evaluation. A human evaluation task is conducted on Amazon Mechanical\nTurk with scale, and shows that our systems are able to generate summaries of\nhigh overall quality and information diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05704v1",
            "date": "2016-06-17 23:22:39+00:00",
            "title": "A Piece of My Mind: A Sentiment Analysis Approach for Online Dispute Detection",
            "authors": [
                "Lu Wang",
                "Claire Cardie"
            ],
            "abstract": "We investigate the novel task of online dispute detection and propose a\nsentiment analysis solution to the problem: we aim to identify the sequence of\nsentence-level sentiments expressed during a discussion and to use them as\nfeatures in a classifier that predicts the DISPUTE/NON-DISPUTE label for the\ndiscussion as a whole. We evaluate dispute detection approaches on a newly\ncreated corpus of Wikipedia Talk page disputes and find that classifiers that\nrely on our sentiment tagging features outperform those that do not. The best\nmodel achieves a very promising F1 score of 0.78 and an accuracy of 0.80.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05706v1",
            "date": "2016-06-17 23:29:11+00:00",
            "title": "Improving Agreement and Disagreement Identification in Online Discussions with A Socially-Tuned Sentiment Lexicon",
            "authors": [
                "Lu Wang",
                "Claire Cardie"
            ],
            "abstract": "We study the problem of agreement and disagreement detection in online\ndiscussions. An isotonic Conditional Random Fields (isotonic CRF) based\nsequential model is proposed to make predictions on sentence- or segment-level.\nWe automatically construct a socially-tuned lexicon that is bootstrapped from\nexisting general-purpose sentiment lexicons to further improve the performance.\nWe evaluate our agreement and disagreement tagging model on two disparate\nonline discussion corpora -- Wikipedia Talk pages and online debates. Our model\nis shown to outperform the state-of-the-art approaches in both datasets. For\nexample, the isotonic CRF model achieves F1 scores of 0.74 and 0.67 for\nagreement and disagreement detection, when a linear chain CRF obtains 0.58 and\n0.56 for the discussions on Wikipedia Talk pages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05759v1",
            "date": "2016-06-18 14:34:07+00:00",
            "title": "Egyptian Arabic to English Statistical Machine Translation System for NIST OpenMT'2015",
            "authors": [
                "Hassan Sajjad",
                "Nadir Durrani",
                "Francisco Guzman",
                "Preslav Nakov",
                "Ahmed Abdelali",
                "Stephan Vogel",
                "Wael Salloum",
                "Ahmed El Kholy",
                "Nizar Habash"
            ],
            "abstract": "The paper describes the Egyptian Arabic-to-English statistical machine\ntranslation (SMT) system that the QCRI-Columbia-NYUAD (QCN) group submitted to\nthe NIST OpenMT'2015 competition. The competition focused on informal dialectal\nArabic, as used in SMS, chat, and speech. Thus, our efforts focused on\nprocessing and standardizing Arabic, e.g., using tools such as 3arrib and\nMADAMIRA. We further trained a phrase-based SMT system using state-of-the-art\nfeatures and components such as operation sequence model, class-based language\nmodel, sparse features, neural network joint model, genre-based\nhierarchically-interpolated language model, unsupervised transliteration\nmining, phrase-table merging, and hypothesis combination. Our system ranked\nsecond on all three genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05804v2",
            "date": "2016-06-18 20:38:42+00:00",
            "title": "Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema",
            "authors": [
                "Patrick Verga",
                "Arvind Neelakantan",
                "Andrew McCallum"
            ],
            "abstract": "Universal schema predicts the types of entities and relations in a knowledge\nbase (KB) by jointly embedding the union of all available schema types---not\nonly types from multiple structured databases (such as Freebase or Wikipedia\ninfoboxes), but also types expressed as textual patterns from raw text. This\nprediction is typically modeled as a matrix completion problem, with one type\nper column, and either one or two entities per row (in the case of entity types\nor binary relation types, respectively). Factorizing this sparsely observed\nmatrix yields a learned vector embedding for each row and each column. In this\npaper we explore the problem of making predictions for entities or entity-pairs\nunseen at training time (and hence without a pre-learned row embedding). We\npropose an approach having no per-row parameters at all; rather we produce a\nrow vector on the fly using a learned aggregation function of the vectors of\nthe observed columns for that row. We experiment with various aggregation\nfunctions, including neural network attention models. Our approach can be\nunderstood as a natural language database, in that questions about KB entities\nare answered by attending to textual or database evidence. In experiments\npredicting both relations and entity types, we demonstrate that despite having\nan order of magnitude fewer parameters than traditional universal schema, we\ncan match the accuracy of the traditional model, and more importantly, we can\nnow make predictions about unseen rows with nearly the same accuracy as rows\navailable at training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05829v1",
            "date": "2016-06-19 03:17:29+00:00",
            "title": "Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test",
            "authors": [
                "Qixin Wang",
                "Tianyi Luo",
                "Dong Wang"
            ],
            "abstract": "Recent progress in neural learning demonstrated that machines can do well in\nregularized tasks, e.g., the game of Go. However, artistic activities such as\npoem generation are still widely regarded as human's special capability. In\nthis paper, we demonstrate that a simple neural model can imitate human in some\ntasks of art generation. We particularly focus on traditional Chinese poetry,\nand show that machines can do as well as many contemporary poets and weakly\npass the Feigenbaum Test, a variant of Turing test in professional domains. Our\nmethod is based on an attention-based recurrent neural network, which accepts a\nset of keywords as the theme and generates poems by looking at each keyword\nduring the generation. A number of techniques are proposed to improve the\nmodel, including character vector initialization, attention to input and\nhybrid-style training. Compared to existing poetry generation methods, our\nmodel can generate much more theme-consistent and semantic-rich poems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05854v2",
            "date": "2016-06-19 11:03:47+00:00",
            "title": "Full-Time Supervision based Bidirectional RNN for Factoid Question Answering",
            "authors": [
                "Dong Xu",
                "Wu-Jun Li"
            ],
            "abstract": "Recently, bidirectional recurrent neural network (BRNN) has been widely used\nfor question answering (QA) tasks with promising performance. However, most\nexisting BRNN models extract the information of questions and answers by\ndirectly using a pooling operation to generate the representation for loss or\nsimilarity calculation. Hence, these existing models don't put supervision\n(loss or similarity calculation) at every time step, which will lose some\nuseful information. In this paper, we propose a novel BRNN model called\nfull-time supervision based BRNN (FTS-BRNN), which can put supervision at every\ntime step. Experiments on the factoid QA task show that our FTS-BRNN can\noutperform other baselines to achieve the state-of-the-art accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05967v1",
            "date": "2016-06-20 04:06:23+00:00",
            "title": "A Nonparametric Bayesian Approach for Spoken Term detection by Example Query",
            "authors": [
                "Amir Hossein Harati Nejad Torbati",
                "Joseph Picone"
            ],
            "abstract": "State of the art speech recognition systems use data-intensive\ncontext-dependent phonemes as acoustic units. However, these approaches do not\ntranslate well to low resourced languages where large amounts of training data\nis not available. For such languages, automatic discovery of acoustic units is\ncritical. In this paper, we demonstrate the application of nonparametric\nBayesian models to acoustic unit discovery. We show that the discovered units\nare correlated with phonemes and therefore are linguistically meaningful. We\nalso present a spoken term detection (STD) by example query algorithm based on\nthese automatically learned units. We show that our proposed system produces a\nP@N of 61.2% and an EER of 13.95% on the TIMIT dataset. The improvement in the\nEER is 5% while P@N is only slightly lower than the best reported system in the\nliterature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05994v1",
            "date": "2016-06-20 07:15:55+00:00",
            "title": "The Role of CNL and AMR in Scalable Abstractive Summarization for Multilingual Media Monitoring",
            "authors": [
                "Normunds Gruzitis",
                "Guntis Barzdins"
            ],
            "abstract": "In the era of Big Data and Deep Learning, there is a common view that machine\nlearning approaches are the only way to cope with the robust and scalable\ninformation extraction and summarization. It has been recently proposed that\nthe CNL approach could be scaled up, building on the concept of embedded CNL\nand, thus, allowing for CNL-based information extraction from e.g. normative or\nmedical texts that are rather controlled by nature but still infringe the\nboundaries of CNL. Although it is arguable if CNL can be exploited to approach\nthe robust wide-coverage semantic parsing for use cases like media monitoring,\nits potential becomes much more obvious in the opposite direction: generation\nof story highlights from the summarized AMR graphs, which is in the focus of\nthis position paper.",
            "journal ref": "Controlled Natural Language, Lecture Notes in Computer Science,\n  Vol. 9767, Springer, 2016, pp. 127-130",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06274v1",
            "date": "2016-06-20 19:53:53+00:00",
            "title": "A Data-Driven Approach for Semantic Role Labeling from Induced Grammar Structures in Language",
            "authors": [
                "Vivek Datla",
                "David Lin",
                "Max Louwerse",
                "Abhinav Vishnu"
            ],
            "abstract": "Semantic roles play an important role in extracting knowledge from text.\nCurrent unsupervised approaches utilize features from grammar structures, to\ninduce semantic roles. The dependence on these grammars, however, makes it\ndifficult to adapt to noisy and new languages. In this paper we develop a\ndata-driven approach to identifying semantic roles, the approach is entirely\nunsupervised up to the point where rules need to be learned to identify the\nposition the semantic role occurs. Specifically we develop a modified-ADIOS\nalgorithm based on ADIOS Solan et al. (2005) to learn grammar structures, and\nuse these grammar structures to learn the rules for identifying the semantic\nroles based on the context in which the grammar structures appeared. The\nresults obtained are comparable with the current state-of-art models that are\ninherently dependent on human annotated data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06406v1",
            "date": "2016-06-21 03:20:59+00:00",
            "title": "Incremental Parsing with Minimal Features Using Bi-Directional LSTM",
            "authors": [
                "James Cross",
                "Liang Huang"
            ],
            "abstract": "Recently, neural network approaches for parsing have largely automated the\ncombination of individual features, but still rely on (often a larger number\nof) atomic features created from human linguistic intuition, and potentially\nomitting important global context. To further reduce feature engineering to the\nbare minimum, we use bi-directional LSTM sentence representations to model a\nparser state with only three sentence positions, which automatically identifies\nimportant aspects of the entire sentence. This model achieves state-of-the-art\nresults among greedy dependency parsers for English. We also introduce a novel\ntransition system for constituency parsing which does not require binarization,\nand together with the above architecture, achieves state-of-the-art results\namong greedy parsers for both English and Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06640v1",
            "date": "2016-06-21 16:25:31+00:00",
            "title": "Neural Morphological Tagging from Characters for Morphologically Rich Languages",
            "authors": [
                "Georg Heigold",
                "Guenter Neumann",
                "Josef van Genabith"
            ],
            "abstract": "This paper investigates neural character-based morphological tagging for\nlanguages with complex morphology and large tag sets. We systematically explore\na variety of neural architectures (DNN, CNN, CNNHighway, LSTM, BLSTM) to obtain\ncharacter-based word vectors combined with bidirectional LSTMs to model\nacross-word context in an end-to-end setting. We explore supplementary use of\nword-based vectors trained on large amounts of unlabeled data. Our experiments\nfor morphological tagging suggest that for \"simple\" model configurations, the\nchoice of the network architecture (CNN vs. CNNHighway vs. LSTM vs. BLSTM) or\nthe augmentation with pre-trained word embeddings can be important and clearly\nimpact the accuracy. Increasing the model capacity by adding depth, for\nexample, and carefully optimizing the neural networks can lead to substantial\nimprovements, and the differences in accuracy (but not training time) become\nmuch smaller or even negligible. Overall, our best morphological taggers for\nGerman and Czech outperform the best results reported in the literature by a\nlarge margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06710v1",
            "date": "2016-06-21 19:12:01+00:00",
            "title": "Correlation-based Intrinsic Evaluation of Word Vector Representations",
            "authors": [
                "Yulia Tsvetkov",
                "Manaal Faruqui",
                "Chris Dyer"
            ],
            "abstract": "We introduce QVEC-CCA--an intrinsic evaluation metric for word vector\nrepresentations based on correlations of learned vectors with features\nextracted from linguistic resources. We show that QVEC-CCA scores are an\neffective proxy for a range of extrinsic semantic and syntactic tasks. We also\nshow that the proposed evaluation obtains higher and more consistent\ncorrelations with downstream tasks, compared to existing approaches to\nintrinsic evaluation of word vectors that are based on word similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06996v1",
            "date": "2016-06-22 16:00:52+00:00",
            "title": "The word entropy of natural languages",
            "authors": [
                "Christian Bentz",
                "Dimitrios Alikaniotis"
            ],
            "abstract": "The average uncertainty associated with words is an information-theoretic\nconcept at the heart of quantitative and computational linguistics. The entropy\nhas been established as a measure of this average uncertainty - also called\naverage information content. We here use parallel texts of 21 languages to\nestablish the number of tokens at which word entropies converge to stable\nvalues. These convergence points are then used to select texts from a massively\nparallel corpus, and to estimate word entropies across more than 1000\nlanguages. Our results help to establish quantitative language comparisons, to\nunderstand the performance of multilingual translation systems, and to\nnormalize semantic similarity measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07046v2",
            "date": "2016-06-22 19:19:29+00:00",
            "title": "Semantic Parsing to Probabilistic Programs for Situated Question Answering",
            "authors": [
                "Jayant Krishnamurthy",
                "Oyvind Tafjord",
                "Aniruddha Kembhavi"
            ],
            "abstract": "Situated question answering is the problem of answering questions about an\nenvironment such as an image or diagram. This problem requires jointly\ninterpreting a question and an environment using background knowledge to select\nthe correct answer. We present Parsing to Probabilistic Programs (P3), a novel\nsituated question answering model that can use background knowledge and global\nfeatures of the question/environment interpretation while retaining efficient\napproximate inference. Our key insight is to treat semantic parses as\nprobabilistic programs that execute nondeterministically and whose possible\nexecutions represent environmental uncertainty. We evaluate our approach on a\nnew, publicly-released data set of 5000 science diagram questions,\noutperforming several competitive classical and neural baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07481v1",
            "date": "2016-06-23 21:23:29+00:00",
            "title": "CUNI System for WMT16 Automatic Post-Editing and Multimodal Translation Tasks",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Jind\u0159ich Helcl",
                "Marek Tlust\u00fd",
                "Pavel Pecina",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Neural sequence to sequence learning recently became a very promising\nparadigm in machine translation, achieving competitive results with statistical\nphrase-based systems. In this system description paper, we attempt to utilize\nseveral recently published methods used for neural sequential learning in order\nto build systems for WMT 2016 shared tasks of Automatic Post-Editing and\nMultimodal Machine Translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07548v1",
            "date": "2016-06-24 02:57:04+00:00",
            "title": "A Sentence Compression Based Framework to Query-Focused Multi-Document Summarization",
            "authors": [
                "Lu Wang",
                "Hema Raghavan",
                "Vittorio Castelli",
                "Radu Florian",
                "Claire Cardie"
            ],
            "abstract": "We consider the problem of using sentence compression techniques to\nfacilitate query-focused multi-document summarization. We present a\nsentence-compression-based framework for the task, and design a series of\nlearning-based compression models built on parse trees. An innovative beam\nsearch decoder is proposed to efficiently find highly probable compressions.\nUnder this framework, we show how to integrate various indicative metrics such\nas linguistic motivation and query relevance into the compression process by\nderiving a novel formulation of a compression scoring function. Our best model\nachieves statistically significant improvement over the state-of-the-art\nsystems on several metrics (e.g. 8.0% and 5.4% improvements in ROUGE-2\nrespectively) for the DUC 2006 and 2007 summarization task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07601v1",
            "date": "2016-06-24 08:35:01+00:00",
            "title": "Evaluation method of word embedding by roots and affixes",
            "authors": [
                "KeBin Peng"
            ],
            "abstract": "Word embedding has been shown to be remarkably effective in a lot of Natural\nLanguage Processing tasks. However, existing models still have a couple of\nlimitations in interpreting the dimensions of word vector. In this paper, we\nprovide a new approach---roots and affixes model(RAAM)---to interpret it from\nthe intrinsic structures of natural language. Also it can be used as an\nevaluation measure of the quality of word embedding. We introduce the\ninformation entropy into our model and divide the dimensions into two\ncategories, just like roots and affixes in lexical semantics. Then considering\neach category as a whole rather than individually. We experimented with English\nWikipedia corpus. Our result show that there is a negative linear relation\nbetween the two attributes and a high positive correlation between our model\nand downstream semantic evaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07736v1",
            "date": "2016-06-24 15:59:14+00:00",
            "title": "Issues in evaluating semantic spaces using word analogies",
            "authors": [
                "Tal Linzen"
            ],
            "abstract": "The offset method for solving word analogies has become a standard evaluation\ntool for vector-space semantic models: it is considered desirable for a space\nto represent semantic relations as consistent vector offsets. We show that the\nmethod's reliance on cosine similarity conflates offset consistency with\nlargely irrelevant neighborhood structure, and propose simple baselines that\nshould be used to improve the utility of the method in vector space evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07772v3",
            "date": "2016-06-24 18:00:42+00:00",
            "title": "The emotional arcs of stories are dominated by six basic shapes",
            "authors": [
                "Andrew J. Reagan",
                "Lewis Mitchell",
                "Dilan Kiley",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Advances in computing power, natural language processing, and digitization of\ntext now make it possible to study a culture's evolution through its texts\nusing a \"big data\" lens. Our ability to communicate relies in part upon a\nshared emotional experience, with stories often following distinct emotional\ntrajectories and forming patterns that are meaningful to us. Here, by\nclassifying the emotional arcs for a filtered subset of 1,327 stories from\nProject Gutenberg's fiction collection, we find a set of six core emotional\narcs which form the essential building blocks of complex emotional\ntrajectories. We strengthen our findings by separately applying Matrix\ndecomposition, supervised learning, and unsupervised learning. For each of\nthese six core emotional arcs, we examine the closest characteristic stories in\npublication today and find that particular emotional arcs enjoy greater\nsuccess, as measured by downloads.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07783v1",
            "date": "2016-06-24 18:35:56+00:00",
            "title": "Sequential Convolutional Neural Networks for Slot Filling in Spoken Language Understanding",
            "authors": [
                "Ngoc Thang Vu"
            ],
            "abstract": "We investigate the usage of convolutional neural networks (CNNs) for the slot\nfilling task in spoken language understanding. We propose a novel CNN\narchitecture for sequence labeling which takes into account the previous\ncontext words with preserved order information and pays special attention to\nthe current word with its surrounding context. Moreover, it combines the\ninformation from the past and the future words for classification. Our proposed\nCNN architecture outperforms even the previously best ensembling recurrent\nneural network model and achieves state-of-the-art results with an F1-score of\n95.61% on the ATIS benchmark dataset without using any additional linguistic\nknowledge and resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07829v1",
            "date": "2016-06-24 20:17:44+00:00",
            "title": "Unsupervised Topic Modeling Approaches to Decision Summarization in Spoken Meetings",
            "authors": [
                "Lu Wang",
                "Claire Cardie"
            ],
            "abstract": "We present a token-level decision summarization framework that utilizes the\nlatent topic structures of utterances to identify \"summary-worthy\" words.\nConcretely, a series of unsupervised topic models is explored and experimental\nresults show that fine-grained topic models, which discover topics at the\nutterance-level rather than the document-level, can better identify the gist of\nthe decision-making process. Moreover, our proposed token-level summarization\napproach, which is able to remove redundancies within utterances, outperforms\nexisting utterance ranking based summarization methods. Finally, context\ninformation is also investigated to add additional relevant information to the\nsummary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07849v1",
            "date": "2016-06-24 22:49:56+00:00",
            "title": "Focused Meeting Summarization via Unsupervised Relation Extraction",
            "authors": [
                "Lu Wang",
                "Claire Cardie"
            ],
            "abstract": "We present a novel unsupervised framework for focused meeting summarization\nthat views the problem as an instance of relation extraction. We adapt an\nexisting in-domain relation learner (Chen et al., 2011) by exploiting a set of\ntask-specific constraints and features. We evaluate the approach on a decision\nsummarization task and show that it outperforms unsupervised utterance-level\nextractive summarization baselines as well as an existing generic\nrelation-extraction-based summarization method. Moreover, our approach produces\nsummaries competitive with those generated by supervised methods in terms of\nthe standard ROUGE score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07901v1",
            "date": "2016-06-25 12:22:05+00:00",
            "title": "Corpus-level Fine-grained Entity Typing Using Contextual Information",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper addresses the problem of corpus-level entity typing, i.e.,\ninferring from a large corpus that an entity is a member of a class such as\n\"food\" or \"artist\". The application of entity typing we are interested in is\nknowledge base completion, specifically, to learn which classes an entity is a\nmember of. We propose FIGMENT to tackle this problem. FIGMENT is\nembedding-based and combines (i) a global model that scores based on aggregated\ncontextual information of an entity and (ii) a context model that first scores\nthe individual occurrences of an entity and then aggregates the scores. In our\nevaluation, FIGMENT strongly outperforms an approach to entity typing that\nrelies on relations obtained by an open information extraction system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07902v1",
            "date": "2016-06-25 12:27:17+00:00",
            "title": "Intrinsic Subspace Evaluation of Word Embedding Representations",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We introduce a new methodology for intrinsic evaluation of word\nrepresentations. Specifically, we identify four fundamental criteria based on\nthe characteristics of natural language that pose difficulties to NLP systems;\nand develop tests that directly show whether or not representations contain the\nsubspaces necessary to satisfy these criteria. Current intrinsic evaluations\nare mostly based on the overall similarity or full-space similarity of words\nand thus view vector representations as points. We show the limits of these\npoint-based intrinsic evaluations. We apply our evaluation methodology to the\ncomparison of a count vector model and several neural network models and\ndemonstrate important properties of these models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07950v2",
            "date": "2016-06-25 19:08:19+00:00",
            "title": "Word sense disambiguation: a complex network approach",
            "authors": [
                "Edilson A. Correa Jr.",
                "Alneu de Andrade Lopes",
                "Diego R. Amancio"
            ],
            "abstract": "In recent years, concepts and methods of complex networks have been employed\nto tackle the word sense disambiguation (WSD) task by representing words as\nnodes, which are connected if they are semantically similar. Despite the\nincreasingly number of studies carried out with such models, most of them use\nnetworks just to represent the data, while the pattern recognition performed on\nthe attribute space is performed using traditional learning techniques. In\nother words, the structural relationship between words have not been explicitly\nused in the pattern recognition process. In addition, only a few investigations\nhave probed the suitability of representations based on bipartite networks and\ngraphs (bigraphs) for the problem, as many approaches consider all possible\nlinks between words. In this context, we assess the relevance of a bipartite\nnetwork model representing both feature words (i.e. the words characterizing\nthe context) and target (ambiguous) words to solve ambiguities in written\ntexts. Here, we focus on the semantical relationships between these two type of\nwords, disregarding the relationships between feature words. In special, the\nproposed method not only serves to represent texts as graphs, but also\nconstructs a structure on which the discrimination of senses is accomplished.\nOur results revealed that the proposed learning algorithm in such bipartite\nnetworks provides excellent results mostly when topical features are employed\nto characterize the context. Surprisingly, our method even outperformed the\nsupport vector machine algorithm in particular cases, with the advantage of\nbeing robust even if a small training dataset is available. Taken together, the\nresults obtained here show that the proposed representation/classification\nmethod might be useful to improve the semantical characterization of written\ntexts.",
            "journal ref": "Information Sciences 442-443, pages 103-113 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07965v1",
            "date": "2016-06-25 20:45:14+00:00",
            "title": "Summarizing Decisions in Spoken Meetings",
            "authors": [
                "Lu Wang",
                "Claire Cardie"
            ],
            "abstract": "This paper addresses the problem of summarizing decisions in spoken meetings:\nour goal is to produce a concise {\\it decision abstract} for each meeting\ndecision. We explore and compare token-level and dialogue act-level automatic\nsummarization methods using both unsupervised and supervised learning\nframeworks. In the supervised summarization setting, and given true clusterings\nof decision-related utterances, we find that token-level summaries that employ\ndiscourse context can approach an upper bound for decision abstracts derived\ndirectly from dialogue acts. In the unsupervised summarization setting,we find\nthat summaries based on unsupervised partitioning of decision-related\nutterances perform comparably to those based on partitions generated using\nsupervised techniques (0.22 ROUGE-F1 using LDA-based topic models vs. 0.23\nusing SVMs).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07967v1",
            "date": "2016-06-25 20:54:49+00:00",
            "title": "Leveraging Semantic Web Search and Browse Sessions for Multi-Turn Spoken Dialog Systems",
            "authors": [
                "Lu Wang",
                "Larry Heck",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Training statistical dialog models in spoken dialog systems (SDS) requires\nlarge amounts of annotated data. The lack of scalable methods for data mining\nand annotation poses a significant hurdle for state-of-the-art statistical\ndialog managers. This paper presents an approach that directly leverage\nbillions of web search and browse sessions to overcome this hurdle. The key\ninsight is that task completion through web search and browse sessions is (a)\npredictable and (b) generalizes to spoken dialog task completion. The new\nmethod automatically mines behavioral search and browse patterns from web logs\nand translates them into spoken dialog models. We experiment with naturally\noccurring spoken dialogs and large scale web logs. Our session-based models\noutperform the state-of-the-art method for entity extraction task in SDS. We\nalso achieve better performance for both entity and relation extraction on web\nsearch queries when compared with nontrivial baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07993v1",
            "date": "2016-06-26 04:39:39+00:00",
            "title": "Learning for Biomedical Information Extraction: Methodological Review of Recent Advances",
            "authors": [
                "Feifan Liu",
                "Jinying Chen",
                "Abhyuday Jagannatha",
                "Hong Yu"
            ],
            "abstract": "Biomedical information extraction (BioIE) is important to many applications,\nincluding clinical decision support, integrative biology, and\npharmacovigilance, and therefore it has been an active research. Unlike\nexisting reviews covering a holistic view on BioIE, this review focuses on\nmainly recent advances in learning based approaches, by systematically\nsummarizing them into different aspects of methodological development. In\naddition, we dive into open information extraction and deep learning, two\nemerging and influential techniques and envision next generation of BioIE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08003v1",
            "date": "2016-06-26 07:44:08+00:00",
            "title": "Functional Distributional Semantics",
            "authors": [
                "Guy Emerson",
                "Ann Copestake"
            ],
            "abstract": "Vector space models have become popular in distributional semantics, despite\nthe challenges they face in capturing various semantic phenomena. We propose a\nnovel probabilistic framework which draws on both formal semantics and recent\nadvances in machine learning. In particular, we separate predicates from the\nentities they refer to, allowing us to perform Bayesian inference based on\nlogical forms. We describe an implementation of this framework using a\ncombination of Restricted Boltzmann Machines and feedforward neural networks.\nFinally, we demonstrate the feasibility of this approach by training it on a\nparsed corpus and evaluating it on established similarity datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08089v1",
            "date": "2016-06-26 21:55:40+00:00",
            "title": "This before That: Causal Precedence in the Biomedical Domain",
            "authors": [
                "Gus Hahn-Powell",
                "Dane Bell",
                "Marco A. Valenzuela-Esc\u00e1rcega",
                "Mihai Surdeanu"
            ],
            "abstract": "Causal precedence between biochemical interactions is crucial in the\nbiomedical domain, because it transforms collections of individual\ninteractions, e.g., bindings and phosphorylations, into the causal mechanisms\nneeded to inform meaningful search and inference. Here, we analyze causal\nprecedence in the biomedical domain as distinct from open-domain, temporal\nprecedence. First, we describe a novel, hand-annotated text corpus of causal\nprecedence in the biomedical domain. Second, we use this corpus to investigate\na battery of models of precedence, covering rule-based, feature-based, and\nlatent representation models. The highest-performing individual model achieved\na micro F1 of 43 points, approaching the best performers on the simpler\ntemporal-only precedence tasks. Feature-based and latent representation models\neach outperform the rule-based models, but their performance is complementary\nto one another. We apply a sieve-based architecture to capitalize on this lack\nof overlap, achieving a micro F1 score of 46 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08270v1",
            "date": "2016-06-27 13:39:54+00:00",
            "title": "Evaluating Informal-Domain Word Representations With UrbanDictionary",
            "authors": [
                "Naomi Saphra",
                "Adam Lopez"
            ],
            "abstract": "Existing corpora for intrinsic evaluation are not targeted towards tasks in\ninformal domains such as Twitter or news comment forums. We want to test\nwhether a representation of informal words fulfills the promise of eliding\nexplicit text normalization as a preprocessing step. One possible evaluation\nmetric for such domains is the proximity of spelling variants. We propose how\nsuch a metric might be computed and how a spelling variant dataset can be\ncollected using UrbanDictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08340v2",
            "date": "2016-06-21 05:47:59+00:00",
            "title": "Topic Aware Neural Response Generation",
            "authors": [
                "Chen Xing",
                "Wei Wu",
                "Yu Wu",
                "Jie Liu",
                "Yalou Huang",
                "Ming Zhou",
                "Wei-Ying Ma"
            ],
            "abstract": "We consider incorporating topic information into the sequence-to-sequence\nframework to generate informative and interesting responses for chatbots. To\nthis end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. The\nmodel utilizes topics to simulate prior knowledge of human that guides them to\nform informative and interesting responses in conversation, and leverages the\ntopic information in generation by a joint attention mechanism and a biased\ngeneration probability. The joint attention mechanism summarizes the hidden\nvectors of an input message as context vectors by message attention,\nsynthesizes topic vectors by topic attention from the topic words of the\nmessage obtained from a pre-trained LDA model, and let these vectors jointly\naffect the generation of words in decoding. To increase the possibility of\ntopic words appearing in responses, the model modifies the generation\nprobability of topic words by adding an extra probability item to bias the\noverall distribution. Empirical study on both automatic evaluation metrics and\nhuman annotations shows that TA-Seq2Seq can generate more informative and\ninteresting responses, and significantly outperform the-state-of-the-art\nresponse generation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08425v3",
            "date": "2016-06-27 19:48:40+00:00",
            "title": "Predicting the Relative Difficulty of Single Sentences With and Without Surrounding Context",
            "authors": [
                "Elliot Schumacher",
                "Maxine Eskenazi",
                "Gwen Frishkoff",
                "Kevyn Collins-Thompson"
            ],
            "abstract": "The problem of accurately predicting relative reading difficulty across a set\nof sentences arises in a number of important natural language applications,\nsuch as finding and curating effective usage examples for intelligent language\ntutoring systems. Yet while significant research has explored document- and\npassage-level reading difficulty, the special challenges involved in assessing\naspects of readability for single sentences have received much less attention,\nparticularly when considering the role of surrounding passages. We introduce\nand evaluate a novel approach for estimating the relative reading difficulty of\na set of sentences, with and without surrounding context. Using different sets\nof lexical and grammatical features, we explore models for predicting pairwise\nrelative difficulty using logistic regression, and examine rankings generated\nby aggregating pairwise difficulty labels using a Bayesian rating system to\nform a final ranking. We also compare rankings derived for sentences assessed\nwith and without context, and find that contextual features can help predict\ndifferences in relative difficulty judgments across these two conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08495v1",
            "date": "2016-06-27 22:00:21+00:00",
            "title": "Network-Efficient Distributed Word2vec Training System for Large Vocabularies",
            "authors": [
                "Erik Ordentlich",
                "Lee Yang",
                "Andy Feng",
                "Peter Cnudde",
                "Mihajlo Grbovic",
                "Nemanja Djuric",
                "Vladan Radosavljevic",
                "Gavin Owens"
            ],
            "abstract": "Word2vec is a popular family of algorithms for unsupervised training of dense\nvector representations of words on large text corpuses. The resulting vectors\nhave been shown to capture semantic relationships among their corresponding\nwords, and have shown promise in reducing a number of natural language\nprocessing (NLP) tasks to mathematical operations on these vectors. While\nheretofore applications of word2vec have centered around vocabularies with a\nfew million words, wherein the vocabulary is the set of words for which vectors\nare simultaneously trained, novel applications are emerging in areas outside of\nNLP with vocabularies comprising several 100 million words. Existing word2vec\ntraining systems are impractical for training such large vocabularies as they\neither require that the vectors of all vocabulary words be stored in the memory\nof a single server or suffer unacceptable training latency due to massive\nnetwork data transfer. In this paper, we present a novel distributed, parallel\ntraining system that enables unprecedented practical training of vectors for\nvocabularies with several 100 million words on a shared cluster of commodity\nservers, using far less network traffic than the existing solutions. We\nevaluate the proposed system on a benchmark dataset, showing that the quality\nof vectors does not degrade relative to non-distributed training. Finally, for\nseveral quarters, the system has been deployed for the purpose of matching\nqueries to ads in Gemini, the sponsored search advertising platform at Yahoo,\nresulting in significant improvement of business metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08513v3",
            "date": "2016-06-27 23:48:16+00:00",
            "title": "SelQA: A New Benchmark for Selection-based Question Answering",
            "authors": [
                "Tomasz Jurczyk",
                "Michael Zhai",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents a new selection-based question answering dataset, SelQA.\nThe dataset consists of questions generated through crowdsourcing and sentence\nlength answers that are drawn from the ten most prevalent topics in the English\nWikipedia. We introduce a corpus annotation scheme that enhances the generation\nof large, diverse, and challenging datasets by explicitly aiming to reduce word\nco-occurrences between the question and answers. Our annotation scheme is\ncomposed of a series of crowdsourcing tasks with a view to more effectively\nutilize crowdsourcing in the creation of question answering datasets in various\ndomains. Several systems are compared on the tasks of answer sentence selection\nand answer triggering, providing strong baseline results for future work to\nimprove upon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08733v2",
            "date": "2016-06-28 14:33:29+00:00",
            "title": "Recurrent Neural Networks for Dialogue State Tracking",
            "authors": [
                "Ond\u0159ej Pl\u00e1tek",
                "Petr B\u011blohl\u00e1vek",
                "Vojt\u011bch Hude\u010dek",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "This paper discusses models for dialogue state tracking using recurrent\nneural networks (RNN). We present experiments on the standard dialogue state\ntracking (DST) dataset, DSTC2. On the one hand, RNN models became the state of\nthe art models in DST, on the other hand, most state-of-the-art models are only\nturn-based and require dataset-specific preprocessing (e.g. DSTC2-specific) in\norder to achieve such results. We implemented two architectures which can be\nused in incremental settings and require almost no preprocessing. We compare\ntheir performance to the benchmarks on DSTC2 and discuss their properties. With\nonly trivial preprocessing, the performance of our models is close to the\nstate-of- the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08821v1",
            "date": "2016-06-28 18:44:38+00:00",
            "title": "Generation and Pruning of Pronunciation Variants to Improve ASR Accuracy",
            "authors": [
                "Zhenhao Ge",
                "Aravind Ganapathiraju",
                "Ananth N. Iyer",
                "Scott A. Randal",
                "Felix I. Wyss"
            ],
            "abstract": "Speech recognition, especially name recognition, is widely used in phone\nservices such as company directory dialers, stock quote providers or location\nfinders. It is usually challenging due to pronunciation variations. This paper\nproposes an efficient and robust data-driven technique which automatically\nlearns acceptable word pronunciations and updates the pronunciation dictionary\nto build a better lexicon without affecting recognition of other words similar\nto the target word. It generalizes well on datasets with various sizes, and\nreduces the error rate on a database with 13000+ human names by 42%, compared\nto a baseline with regular dictionaries already covering canonical\npronunciations of 97%+ words in names, plus a well-trained\nspelling-to-pronunciation (STP) engine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09370v1",
            "date": "2016-06-30 07:10:07+00:00",
            "title": "Relation extraction from clinical texts using domain invariant convolutional neural network",
            "authors": [
                "Sunil Kumar Sahu",
                "Ashish Anand",
                "Krishnadev Oruganty",
                "Mahanandeeshwar Gattu"
            ],
            "abstract": "In recent years extracting relevant information from biomedical and clinical\ntexts such as research articles, discharge summaries, or electronic health\nrecords have been a subject of many research efforts and shared challenges.\nRelation extraction is the process of detecting and classifying the semantic\nrelation among entities in a given piece of texts. Existing models for this\ntask in biomedical domain use either manually engineered features or kernel\nmethods to create feature vector. These features are then fed to classifier for\nthe prediction of the correct class. It turns out that the results of these\nmethods are highly dependent on quality of user designed features and also\nsuffer from curse of dimensionality. In this work we focus on extracting\nrelations from clinical discharge summaries. Our main objective is to exploit\nthe power of convolution neural network (CNN) to learn features automatically\nand thus reduce the dependency on manual feature engineering. We evaluate\nperformance of the proposed model on i2b2-2010 clinical relation extraction\nchallenge dataset. Our results indicate that convolution neural network can be\na good model for relation exaction in clinical text without being dependent on\nexpert's knowledge on defining quality features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09371v1",
            "date": "2016-06-30 07:15:56+00:00",
            "title": "Recurrent neural network models for disease name recognition using domain invariant features",
            "authors": [
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Hand-crafted features based on linguistic and domain-knowledge play crucial\nrole in determining the performance of disease name recognition systems. Such\nmethods are further limited by the scope of these features or in other words,\ntheir ability to cover the contexts or word dependencies within a sentence. In\nthis work, we focus on reducing such dependencies and propose a\ndomain-invariant framework for the disease name recognition task. In\nparticular, we propose various end-to-end recurrent neural network (RNN) models\nfor the tasks of disease name recognition and their classification into four\npre-defined categories. We also utilize convolution neural network (CNN) in\ncascade of RNN to get character-based embedded features and employ it with\nword-embedded features in our model. We compare our models with the\nstate-of-the-art results for the two tasks on NCBI disease dataset. Our results\nfor the disease mention recognition task indicate that state-of-the-art\nperformance can be obtained without relying on feature engineering. Further the\nproposed models obtained improved performance on the classification task of\ndisease names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09560v1",
            "date": "2016-06-30 16:32:00+00:00",
            "title": "Neural Network-based Word Alignment through Score Aggregation",
            "authors": [
                "Joel Legrand",
                "Michael Auli",
                "Ronan Collobert"
            ],
            "abstract": "We present a simple neural network for word alignment that builds source and\ntarget word window representations to compute alignment scores for sentence\npairs. To enable unsupervised training, we use an aggregation operation that\nsummarizes the alignment scores for a given target word. A soft-margin\nobjective increases scores for true target words while decreasing scores for\ntarget words that are not present. Compared to the popular Fast Align model,\nour approach improves alignment accuracy by 7 AER on English-Czech, by 6 AER on\nRomanian-English and by 1.7 AER on English-French alignment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09600v1",
            "date": "2016-06-30 18:10:46+00:00",
            "title": "Exploring Prediction Uncertainty in Machine Translation Quality Estimation",
            "authors": [
                "Daniel Beck",
                "Lucia Specia",
                "Trevor Cohn"
            ],
            "abstract": "Machine Translation Quality Estimation is a notoriously difficult task, which\nlessens its usefulness in real-world translation environments. Such scenarios\ncan be improved if quality predictions are accompanied by a measure of\nuncertainty. However, models in this task are traditionally evaluated only in\nterms of point estimate metrics, which do not take prediction uncertainty into\naccount. We investigate probabilistic methods for Quality Estimation that can\nprovide well-calibrated uncertainty estimates and evaluate them in terms of\ntheir full posterior predictive distributions. We also show how this posterior\ninformation can be useful in an asymmetric risk scenario, which aims to capture\ntypical situations in translation workflows.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09604v1",
            "date": "2016-06-30 18:23:47+00:00",
            "title": "SnapToGrid: From Statistical to Interpretable Models for Biomedical Information Extraction",
            "authors": [
                "Marco A. Valenzuela-Escarcega",
                "Gus Hahn-Powell",
                "Dane Bell",
                "Mihai Surdeanu"
            ],
            "abstract": "We propose an approach for biomedical information extraction that marries the\nadvantages of machine learning models, e.g., learning directly from data, with\nthe benefits of rule-based approaches, e.g., interpretability. Our approach\nstarts by training a feature-based statistical model, then converts this model\nto a rule-based variant by converting its features to rules, and \"snapping to\ngrid\" the feature weights to discrete votes. In doing so, our proposal takes\nadvantage of the large body of work in machine learning, but it produces an\ninterpretable model, which can be directly edited by experts. We evaluate our\napproach on the BioNLP 2009 event extraction task. Our results show that there\nis a small performance penalty when converting the statistical model to rules,\nbut the gain in interpretability compensates for that: with minimal effort,\nhuman experts improve this model to have similar performance to the statistical\nmodel that served as starting point.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09636v2",
            "date": "2016-06-30 19:47:17+00:00",
            "title": "Representation of texts as complex networks: a mesoscopic approach",
            "authors": [
                "Henrique F. de Arruda",
                "Filipi N. Silva",
                "Vanessa Q. Marinho",
                "Diego R. Amancio",
                "Luciano da F. Costa"
            ],
            "abstract": "Statistical techniques that analyze texts, referred to as text analytics,\nhave departed from the use of simple word count statistics towards a new\nparadigm. Text mining now hinges on a more sophisticated set of methods,\nincluding the representations in terms of complex networks. While\nwell-established word-adjacency (co-occurrence) methods successfully grasp\nsyntactical features of written texts, they are unable to represent important\naspects of textual data, such as its topical structure, i.e. the sequence of\nsubjects developing at a mesoscopic level along the text. Such aspects are\noften overlooked by current methodologies. In order to grasp the mesoscopic\ncharacteristics of semantical content in written texts, we devised a network\nmodel which is able to analyze documents in a multi-scale fashion. In the\nproposed model, a limited amount of adjacent paragraphs are represented as\nnodes, which are connected whenever they share a minimum semantical content. To\nillustrate the capabilities of our model, we present, as a case example, a\nqualitative analysis of \"Alice's Adventures in Wonderland\". We show that the\nmesoscopic structure of a document, modeled as a network, reveals many semantic\ntraits of texts. Such an approach paves the way to a myriad of semantic-based\napplications. In addition, our approach is illustrated in a machine learning\ncontext, in which texts are classified among real texts and randomized\ninstances.",
            "journal ref": "Journal of Complex Networks 6(1), 125-144, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00112v1",
            "date": "2016-07-30 12:39:19+00:00",
            "title": "Supervised Attentions for Neural Machine Translation",
            "authors": [
                "Haitao Mi",
                "Zhiguo Wang",
                "Abe Ittycheriah"
            ],
            "abstract": "In this paper, we improve the attention or alignment accuracy of neural\nmachine translation by utilizing the alignments of training sentence pairs. We\nsimply compute the distance between the machine attentions and the \"true\"\nalignments, and minimize this cost in the training procedure. Our experiments\non large-scale Chinese-to-English task show that our model improves both\ntranslation and alignment qualities significantly over the large-vocabulary\nneural machine translation system, and even beats a state-of-the-art\ntraditional syntax-based system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00293v1",
            "date": "2016-08-01 01:18:53+00:00",
            "title": "Left-corner Methods for Syntactic Modeling with Universal Structural Constraints",
            "authors": [
                "Hiroshi Noji"
            ],
            "abstract": "The primary goal in this thesis is to identify better syntactic constraint or\nbias, that is language independent but also efficiently exploitable during\nsentence processing. We focus on a particular syntactic construction called\ncenter-embedding, which is well studied in psycholinguistics and noted to cause\nparticular difficulty for comprehension. Since people use language as a tool\nfor communication, one expects such complex constructions to be avoided for\ncommunication efficiency. From a computational perspective, center-embedding is\nclosely relevant to a left-corner parsing algorithm, which can capture the\ndegree of center-embedding of a parse tree being constructed. This connection\nsuggests left-corner methods can be a tool to exploit the universal syntactic\nconstraint that people avoid generating center-embedded structures. We explore\nsuch utilities of center-embedding as well as left-corner methods extensively\nthrough several theoretical and empirical examinations.\n  Our primary task is unsupervised grammar induction. In this task, the input\nto the algorithm is a collection of sentences, from which the model tries to\nextract the salient patterns on them as a grammar. This is a particularly hard\nproblem although we expect the universal constraint may help in improving the\nperformance since it can effectively restrict the possible search space for the\nmodel. We build the model by extending the left-corner parsing algorithm for\nefficiently tabulating the search space except those involving center-embedding\nup to a specific degree. We examine the effectiveness of our approach on many\ntreebanks, and demonstrate that often our constraint leads to better parsing\nperformance. We thus conclude that left-corner methods are particularly useful\nfor syntax-oriented systems, as it can exploit efficiently the inherent\nuniversal constraints in languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00339v1",
            "date": "2016-08-01 07:30:38+00:00",
            "title": "Crowd-sourcing NLG Data: Pictures Elicit Better Data",
            "authors": [
                "Jekaterina Novikova",
                "Oliver Lemon",
                "Verena Rieser"
            ],
            "abstract": "Recent advances in corpus-based Natural Language Generation (NLG) hold the\npromise of being easily portable across domains, but require costly training\ndata, consisting of meaning representations (MRs) paired with Natural Language\n(NL) utterances. In this work, we propose a novel framework for crowdsourcing\nhigh quality NLG training data, using automatic quality control measures and\nevaluating different MRs with which to elicit data. We show that pictorial MRs\nresult in better NL data being collected than logic-based MRs: utterances\nelicited by pictorial MRs are judged as significantly more natural, more\ninformative, and better phrased, with a significant increase in average quality\nratings (around 0.5 points on a 6-point scale), compared to using the logical\nMRs. As the MR becomes more complex, the benefits of pictorial stimuli\nincrease. The collected data will be released as part of this submission.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00508v2",
            "date": "2016-08-01 17:51:03+00:00",
            "title": "Blind phoneme segmentation with temporal prediction errors",
            "authors": [
                "Paul Michel",
                "Okko R\u00e4s\u00e4nen",
                "Roland Thiolli\u00e8re",
                "Emmanuel Dupoux"
            ],
            "abstract": "Phonemic segmentation of speech is a critical step of speech recognition\nsystems. We propose a novel unsupervised algorithm based on sequence prediction\nmodels such as Markov chains and recurrent neural network. Our approach\nconsists in analyzing the error profile of a model trained to predict speech\nfeatures frame-by-frame. Specifically, we try to learn the dynamics of speech\nin the MFCC space and hypothesize boundaries from local maxima in the\nprediction error. We evaluate our system on the TIMIT dataset, with\nimprovements over similar methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00612v1",
            "date": "2016-08-01 20:54:22+00:00",
            "title": "Structured prediction models for RNN based sequence labeling in clinical text",
            "authors": [
                "Abhyuday Jagannatha",
                "Hong Yu"
            ],
            "abstract": "Sequence labeling is a widely used method for named entity recognition and\ninformation extraction from unstructured natural language data. In clinical\ndomain one major application of sequence labeling involves extraction of\nmedical entities such as medication, indication, and side-effects from\nElectronic Health Record narratives. Sequence labeling in this domain, presents\nits own set of challenges and objectives. In this work we experimented with\nvarious CRF based structured learning models with Recurrent Neural Networks. We\nextend the previously studied LSTM-CRF models with explicit modeling of\npairwise potentials. We also propose an approximate version of skip-chain CRF\ninference with RNN potentials. We use these methodologies for structured\nprediction in order to improve the exact phrase detection of various medical\nentities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00789v1",
            "date": "2016-08-02 12:31:06+00:00",
            "title": "New word analogy corpus for exploring embeddings of Czech words",
            "authors": [
                "Luk\u00e1\u0161 Svoboda",
                "Tom\u00e1\u0161 Brychc\u00edn"
            ],
            "abstract": "The word embedding methods have been proven to be very useful in many tasks\nof NLP (Natural Language Processing). Much has been investigated about word\nembeddings of English words and phrases, but only little attention has been\ndedicated to other languages.\n  Our goal in this paper is to explore the behavior of state-of-the-art word\nembedding methods on Czech, the language that is characterized by very rich\nmorphology. We introduce new corpus for word analogy task that inspects\nsyntactic, morphosyntactic and semantic properties of Czech words and phrases.\nWe experiment with Word2Vec and GloVe algorithms and discuss the results on\nthis corpus. The corpus is available for the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00841v1",
            "date": "2016-08-02 14:35:31+00:00",
            "title": "Semantic Representations of Word Senses and Concepts",
            "authors": [
                "Jos\u00e9 Camacho-Collados",
                "Ignacio Iacobacci",
                "Roberto Navigli",
                "Mohammad Taher Pilehvar"
            ],
            "abstract": "Representing the semantics of linguistic items in a machine-interpretable\nform has been a major goal of Natural Language Processing since its earliest\ndays. Among the range of different linguistic items, words have attracted the\nmost research attention. However, word representations have an important\nlimitation: they conflate different meanings of a word into a single vector.\nRepresentations of word senses have the potential to overcome this inherent\nlimitation. Indeed, the representation of individual word senses and concepts\nhas recently gained in popularity with several experimental results showing\nthat a considerable performance improvement can be achieved across different\nNLP applications upon moving from word level to the deeper sense and concept\nlevels. Another interesting point regarding the representation of concepts and\nword senses is that these models can be seamlessly applied to other linguistic\nitems, such as words, phrases and sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00869v4",
            "date": "2016-08-02 15:35:12+00:00",
            "title": "SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity",
            "authors": [
                "Daniela Gerz",
                "Ivan Vuli\u0107",
                "Felix Hill",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "Verbs play a critical role in the meaning of sentences, but these ubiquitous\nwords have received little attention in recent distributional semantics\nresearch. We introduce SimVerb-3500, an evaluation resource that provides human\nratings for the similarity of 3,500 verb pairs. SimVerb-3500 covers all normed\nverb types from the USF free-association database, providing at least three\nexamples for every VerbNet class. This broad coverage facilitates detailed\nanalyses of how syntactic and semantic phenomena together influence human\nunderstanding of verb meaning. Further, with significantly larger development\nand test sets than existing benchmarks, SimVerb-3500 enables more robust\nevaluation of representation learning architectures and promotes the\ndevelopment of methods tailored to verbs. We hope that SimVerb-3500 will enable\na richer understanding of the diversity and complexity of verb semantics and\nguide the development of systems that can effectively represent and interpret\nthis meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00892v3",
            "date": "2016-08-02 16:39:10+00:00",
            "title": "Knowledge Distillation for Small-footprint Highway Networks",
            "authors": [
                "Liang Lu",
                "Michelle Guo",
                "Steve Renals"
            ],
            "abstract": "Deep learning has significantly advanced state-of-the-art of speech\nrecognition in the past few years. However, compared to conventional Gaussian\nmixture acoustic models, neural network models are usually much larger, and are\ntherefore not very deployable in embedded devices. Previously, we investigated\na compact highway deep neural network (HDNN) for acoustic modelling, which is a\ntype of depth-gated feedforward neural network. We have shown that HDNN-based\nacoustic models can achieve comparable recognition accuracy with much smaller\nnumber of model parameters compared to plain deep neural network (DNN) acoustic\nmodels. In this paper, we push the boundary further by leveraging on the\nknowledge distillation technique that is also known as {\\it teacher-student}\ntraining, i.e., we train the compact HDNN model with the supervision of a high\naccuracy cumbersome model. Furthermore, we also investigate sequence training\nand adaptation in the context of teacher-student training. Our experiments were\nperformed on the AMI meeting speech recognition corpus. With this technique, we\nsignificantly improved the recognition accuracy of the HDNN acoustic model with\nless than 0.8 million parameters, and narrowed the gap between this model and\nthe plain DNN with 30 million parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00929v1",
            "date": "2016-08-02 18:45:53+00:00",
            "title": "Efficient Segmental Cascades for Speech Recognition",
            "authors": [
                "Hao Tang",
                "Weiran Wang",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "Discriminative segmental models offer a way to incorporate flexible feature\nfunctions into speech recognition. However, their appeal has been limited by\ntheir computational requirements, due to the large number of possible segments\nto consider. Multi-pass cascades of segmental models introduce features of\nincreasing complexity in different passes, where in each pass a segmental model\nrescores lattices produced by a previous (simpler) segmental model. In this\npaper, we explore several ways of making segmental cascades efficient and\npractical: reducing the feature set in the first pass, frame subsampling, and\nvarious pruning approaches. In experiments on phonetic recognition, we find\nthat with a combination of such techniques, it is possible to maintain\ncompetitive performance while greatly reducing decoding, pruning, and training\ntime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01056v2",
            "date": "2016-08-03 02:21:16+00:00",
            "title": "Morphological Priors for Probabilistic Neural Word Embeddings",
            "authors": [
                "Parminder Bhatia",
                "Robert Guthrie",
                "Jacob Eisenstein"
            ],
            "abstract": "Word embeddings allow natural language processing systems to share\nstatistical information across related words. These embeddings are typically\nbased on distributional statistics, making it difficult for them to generalize\nto rare or unseen words. We propose to improve word embeddings by incorporating\nmorphological information, capturing shared sub-word features. Unlike previous\nwork that constructs word embeddings directly from morphemes, we combine\nmorphological and distributional information in a unified probabilistic\nframework, in which the word embedding is a latent variable. The morphological\ninformation provides a prior distribution on the latent word embeddings, which\nin turn condition a likelihood function over an observed corpus. This approach\nyields improvements on intrinsic word similarity evaluations, and also in the\ndownstream task of part-of-speech tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01084v1",
            "date": "2016-08-03 06:24:01+00:00",
            "title": "To Swap or Not to Swap? Exploiting Dependency Word Pairs for Reordering in Statistical Machine Translation",
            "authors": [
                "Christian Hadiwinoto",
                "Yang Liu",
                "Hwee Tou Ng"
            ],
            "abstract": "Reordering poses a major challenge in machine translation (MT) between two\nlanguages with significant differences in word order. In this paper, we present\na novel reordering approach utilizing sparse features based on dependency word\npairs. Each instance of these features captures whether two words, which are\nrelated by a dependency link in the source sentence dependency parse tree,\nfollow the same order or are swapped in the translation output. Experiments on\nChinese-to-English translation show a statistically significant improvement of\n1.21 BLEU point using our approach, compared to a state-of-the-art statistical\nMT system that incorporates prior reordering approaches.",
            "journal ref": "Proceedings of AAAI-16. (pp. 2943--2949) (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01403v1",
            "date": "2016-08-04 00:36:48+00:00",
            "title": "Words, Concepts, and the Geometry of Analogy",
            "authors": [
                "Stephen McGregor",
                "Matthew Purver",
                "Geraint Wiggins"
            ],
            "abstract": "This paper presents a geometric approach to the problem of modelling the\nrelationship between words and concepts, focusing in particular on analogical\nphenomena in language and cognition. Grounded in recent theories regarding\ngeometric conceptual spaces, we begin with an analysis of existing static\ndistributional semantic models and move on to an exploration of a dynamic\napproach to using high dimensional spaces of word meaning to project subspaces\nwhere analogies can potentially be solved in an online, contextualised way. The\ncrucial element of this analysis is the positioning of statistics in a\ngeometric environment replete with opportunities for interpretation.",
            "journal ref": "EPTCS 221, 2016, pp. 39-48",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01405v1",
            "date": "2016-08-04 00:37:07+00:00",
            "title": "Entailment Relations on Distributions",
            "authors": [
                "John van de Wetering"
            ],
            "abstract": "In this paper we give an overview of partial orders on the space of\nprobability distributions that carry a notion of information content and serve\nas a generalisation of the Bayesian order given in (Coecke and Martin, 2011).\nWe investigate what constraints are necessary in order to get a unique notion\nof information content. These partial orders can be used to give an ordering on\nwords in vector space models of natural language meaning relating to the\ncontexts in which words are used, which is useful for a notion of entailment\nand word disambiguation. The construction used also points towards a way to\ncreate orderings on the space of density operators which allow a more\nfine-grained study of entailment. The partial orders in this paper are directed\ncomplete and form domains in the sense of domain theory.",
            "journal ref": "EPTCS 221, 2016, pp. 58-66",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01413v2",
            "date": "2016-08-04 01:47:23+00:00",
            "title": "Solving General Arithmetic Word Problems",
            "authors": [
                "Subhro Roy",
                "Dan Roth"
            ],
            "abstract": "This paper presents a novel approach to automatically solving arithmetic word\nproblems. This is the first algorithmic approach that can handle arithmetic\nproblems with multiple steps and operations, without depending on additional\nannotations or predefined templates. We develop a theory for expression trees\nthat can be used to represent and evaluate the target arithmetic expressions;\nwe use it to uniquely decompose the target arithmetic problem to multiple\nclassification problems; we then compose an expression tree, combining these\nwith world knowledge through a constrained inference framework. Our classifiers\ngain from the use of {\\em quantity schemas} that supports better extraction of\nfeatures. Experimental results show that our method outperforms existing\nsystems, achieving state of the art performance on benchmark datasets of\narithmetic word problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01448v2",
            "date": "2016-08-04 07:37:54+00:00",
            "title": "Word Segmentation on Micro-blog Texts with External Lexicon and Heterogeneous Data",
            "authors": [
                "Qingrong Xia",
                "Zhenghua Li",
                "Jiayuan Chao",
                "Min Zhang"
            ],
            "abstract": "This paper describes our system designed for the NLPCC 2016 shared task on\nword segmentation on micro-blog texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01561v1",
            "date": "2016-08-04 14:44:52+00:00",
            "title": "UsingWord Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval",
            "authors": [
                "Paheli Bhattacharya",
                "Pawan Goyal",
                "Sudeshna Sarkar"
            ],
            "abstract": "Cross-Language Information Retrieval (CLIR) has become an important problem\nto solve in the recent years due to the growth of content in multiple languages\nin the Web. One of the standard methods is to use query translation from source\nto target language. In this paper, we propose an approach based on word\nembeddings, a method that captures contextual clues for a particular word in\nthe source language and gives those words as translations that occur in a\nsimilar context in the target language. Once we obtain the word embeddings of\nthe source and target language pairs, we learn a projection from source to\ntarget word embeddings, making use of a dictionary with word translation\npairs.We then propose various methods of query translation and aggregation. The\nadvantage of this approach is that it does not require the corpora to be\naligned (which is difficult to obtain for resource-scarce languages), a\ndictionary with word translation pairs is enough to train the word vectors for\ntranslation. We experiment with Forum for Information Retrieval and Evaluation\n(FIRE) 2008 and 2012 datasets for Hindi to English CLIR. The proposed word\nembedding based approach outperforms the basic dictionary based approach by 70%\nand when the word embeddings are combined with the dictionary, the hybrid\napproach beats the baseline dictionary based method by 77%. It outperforms the\nEnglish monolingual baseline by 15%, when combined with the translations\nobtained from Google Translate and Dictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01910v1",
            "date": "2016-08-05 15:11:58+00:00",
            "title": "Resolving Out-of-Vocabulary Words with Bilingual Embeddings in Machine Translation",
            "authors": [
                "Pranava Swaroop Madhyastha",
                "Cristina Espa\u00f1a-Bonet"
            ],
            "abstract": "Out-of-vocabulary words account for a large proportion of errors in machine\ntranslation systems, especially when the system is used on a different domain\nthan the one where it was trained. In order to alleviate the problem, we\npropose to use a log-bilinear softmax-based model for vocabulary expansion,\nsuch that given an out-of-vocabulary source word, the model generates a\nprobabilistic list of possible translations in the target language. Our model\nuses only word embeddings trained on significantly large unlabelled monolingual\ncorpora and trains over a fairly small, word-to-word bilingual dictionary. We\ninput this probabilistic list into a standard phrase-based statistical machine\ntranslation system and obtain consistent improvements in translation quality on\nthe English-Spanish language pair. Especially, we get an improvement of 3.9\nBLEU points when tested over an out-of-domain test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01965v1",
            "date": "2016-07-29 20:33:22+00:00",
            "title": "Text authorship identified using the dynamics of word co-occurrence networks",
            "authors": [
                "Camilo Akimushkin",
                "Diego R. Amancio",
                "Osvaldo N. Oliveira Jr"
            ],
            "abstract": "The identification of authorship in disputed documents still requires human\nexpertise, which is now unfeasible for many tasks owing to the large volumes of\ntext and authors in practical applications. In this study, we introduce a\nmethodology based on the dynamics of word co-occurrence networks representing\nwritten texts to classify a corpus of 80 texts by 8 authors. The texts were\ndivided into sections with equal number of linguistic tokens, from which time\nseries were created for 12 topological metrics. The series were proven to be\nstationary (p-value>0.05), which permits to use distribution moments as\nlearning attributes. With an optimized supervised learning procedure using a\nRadial Basis Function Network, 68 out of 80 texts were correctly classified,\ni.e. a remarkable 85% author matching success rate. Therefore, fluctuations in\npurely dynamic network metrics were found to characterize authorship, thus\nopening the way for the description of texts in terms of small evolving\nnetworks. Moreover, the approach introduced allows for comparison of texts with\ndiverse characteristics in a simple, fast fashion.",
            "journal ref": "PLoS ONE 12(1): e0170527, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02025v3",
            "date": "2016-08-05 21:27:00+00:00",
            "title": "Boundary-based MWE segmentation with text partitioning",
            "authors": [
                "Jake Ryland Williams"
            ],
            "abstract": "This work presents a fine-grained, text-chunking algorithm designed for the\ntask of multiword expressions (MWEs) segmentation. As a lexical class, MWEs\ninclude a wide variety of idioms, whose automatic identification are a\nnecessity for the handling of colloquial language. This algorithm's core\nnovelty is its use of non-word tokens, i.e., boundaries, in a bottom-up\nstrategy. Leveraging boundaries refines token-level information, forging\nhigh-level performance from relatively basic data. The generality of this\nmodel's feature space allows for its application across languages and domains.\nExperiments spanning 19 different languages exhibit a broadly-applicable,\nstate-of-the-art model. Evaluation against recent shared-task data places text\npartitioning as the overall, best performing MWE segmentation algorithm,\ncovering all MWE classes and multiple English domains (including user-generated\ntext). This performance, coupled with a non-combinatorial, fast-running design,\nproduces an ideal combination for implementations at scale, which are\nfacilitated through the release of open-source software.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02094v1",
            "date": "2016-08-06 10:47:05+00:00",
            "title": "Desiderata for Vector-Space Word Representations",
            "authors": [
                "Leon Derczynski"
            ],
            "abstract": "A plethora of vector-space representations for words is currently available,\nwhich is growing. These consist of fixed-length vectors containing real values,\nwhich represent a word. The result is a representation upon which the power of\nmany conventional information processing and data mining techniques can be\nbrought to bear, as long as the representations are designed with some\nforethought and fit certain constraints. This paper details desiderata for the\ndesign of vector space representations of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02097v2",
            "date": "2016-08-06 11:41:05+00:00",
            "title": "Encoder-decoder with Focus-mechanism for Sequence Labelling Based Spoken Language Understanding",
            "authors": [
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "This paper investigates the framework of encoder-decoder with attention for\nsequence labelling based spoken language understanding. We introduce\nBidirectional Long Short Term Memory - Long Short Term Memory networks\n(BLSTM-LSTM) as the encoder-decoder model to fully utilize the power of deep\nlearning. In the sequence labelling task, the input and output sequences are\naligned word by word, while the attention mechanism cannot provide the exact\nalignment. To address this limitation, we propose a novel focus mechanism for\nencoder-decoder framework. Experiments on the standard ATIS dataset showed that\nBLSTM-LSTM with focus mechanism defined the new state-of-the-art by\noutperforming standard BLSTM and attention based encoder-decoder. Further\nexperiments also show that the proposed model is more robust to speech\nrecognition errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02117v2",
            "date": "2016-08-06 15:29:34+00:00",
            "title": "HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment",
            "authors": [
                "Ivan Vuli\u0107",
                "Daniela Gerz",
                "Douwe Kiela",
                "Felix Hill",
                "Anna Korhonen"
            ],
            "abstract": "We introduce HyperLex - a dataset and evaluation resource that quantifies the\nextent of of the semantic category membership, that is, type-of relation also\nknown as hyponymy-hypernymy or lexical entailment (LE) relation between 2,616\nconcept pairs. Cognitive psychology research has established that typicality\nand category/class membership are computed in human semantic memory as a\ngradual rather than binary relation. Nevertheless, most NLP research, and\nexisting large-scale invetories of concept category membership (WordNet,\nDBPedia, etc.) treat category membership and LE as binary. To address this, we\nasked hundreds of native English speakers to indicate typicality and strength\nof category membership between a diverse range of concept pairs on a\ncrowdsourcing platform. Our results confirm that category membership and LE are\nindeed more gradual than binary. We then compare these human judgements with\nthe predictions of automatic systems, which reveals a huge gap between human\nperformance and state-of-the-art LE, distributional and representation learning\nmodels, and substantial differences between the models themselves. We discuss a\npathway for improving semantic models to overcome this discrepancy, and\nindicate future application areas for improved graded LE systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02214v2",
            "date": "2016-08-07 13:28:46+00:00",
            "title": "Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network",
            "authors": [
                "Keisuke Sakaguchi",
                "Kevin Duh",
                "Matt Post",
                "Benjamin Van Durme"
            ],
            "abstract": "Language processing mechanism by humans is generally more robust than\ncomputers. The Cmabrigde Uinervtisy (Cambridge University) effect from the\npsycholinguistics literature has demonstrated such a robust word processing\nmechanism, where jumbled words (e.g. Cmabrigde / Cambridge) are recognized with\nlittle cost. On the other hand, computational models for word recognition (e.g.\nspelling checkers) perform poorly on data with such noise. Inspired by the\nfindings from the Cmabrigde Uinervtisy effect, we propose a word recognition\nmodel based on a semi-character level recurrent neural network (scRNN). In our\nexperiments, we demonstrate that scRNN has significantly more robust\nperformance in word spelling correction (i.e. word recognition) compared to\nexisting spelling checkers and character-based convolutional neural network.\nFurthermore, we demonstrate that the model is cognitively plausible by\nreplicating a psycholinguistics experiment about human reading difficulty using\nour model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02784v2",
            "date": "2016-08-09 12:26:19+00:00",
            "title": "Canonical Correlation Inference for Mapping Abstract Scenes to Text",
            "authors": [
                "Nikos Papasarantopoulos",
                "Helen Jiang",
                "Shay B. Cohen"
            ],
            "abstract": "We describe a technique for structured prediction, based on canonical\ncorrelation analysis. Our learning algorithm finds two projections for the\ninput and the output spaces that aim at projecting a given input and its\ncorrect output into points close to each other. We demonstrate our technique on\na language-vision problem, namely the problem of giving a textual description\nto an \"abstract scene\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02926v4",
            "date": "2016-08-09 19:40:21+00:00",
            "title": "The Language of Generalization",
            "authors": [
                "Michael Henry Tessler",
                "Noah D. Goodman"
            ],
            "abstract": "Language provides simple ways of communicating generalizable knowledge to\neach other (e.g., \"Birds fly\", \"John hikes\", \"Fire makes smoke\"). Though found\nin every language and emerging early in development, the language of\ngeneralization is philosophically puzzling and has resisted precise\nformalization. Here, we propose the first formal account of generalizations\nconveyed with language that makes quantitative predictions about human\nunderstanding. We test our model in three diverse domains: generalizations\nabout categories (generic language), events (habitual language), and causes\n(causal language). The model explains the gradience in human endorsement\nthrough the interplay between a simple truth-conditional semantic theory and\ndiverse beliefs about properties, formalized in a probabilistic model of\nlanguage understanding. This work opens the door to understanding precisely how\nabstract knowledge is learned from language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02927v1",
            "date": "2016-08-09 19:42:14+00:00",
            "title": "Temporal Attention Model for Neural Machine Translation",
            "authors": [
                "Baskaran Sankaran",
                "Haitao Mi",
                "Yaser Al-Onaizan",
                "Abe Ittycheriah"
            ],
            "abstract": "Attention-based Neural Machine Translation (NMT) models suffer from attention\ndeficiency issues as has been observed in recent research. We propose a novel\nmechanism to address some of these limitations and improve the NMT attention.\nSpecifically, our approach memorizes the alignments temporally (within each\nsentence) and modulates the attention with the accumulated temporal memory, as\nthe decoder generates the candidate translation. We compare our approach\nagainst the baseline NMT model and two other related approaches that address\nthis issue either explicitly or implicitly. Large-scale experiments on two\nlanguage pairs show that our approach achieves better and robust gains over the\nbaseline and related NMT approaches. Our model further outperforms strong SMT\nbaselines in some settings even without using ensembles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03030v1",
            "date": "2016-08-10 03:05:26+00:00",
            "title": "Hierarchical Character-Word Models for Language Identification",
            "authors": [
                "Aaron Jaech",
                "George Mulcaire",
                "Shobhit Hathi",
                "Mari Ostendorf",
                "Noah A. Smith"
            ],
            "abstract": "Social media messages' brevity and unconventional spelling pose a challenge\nto language identification. We introduce a hierarchical model that learns\ncharacter and contextualized word-level representations for language\nidentification. Our method performs well against strong base- lines, and can\nalso reveal code-switching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03065v1",
            "date": "2016-08-10 07:45:46+00:00",
            "title": "An assessment of orthographic similarity measures for several African languages",
            "authors": [
                "C. Maria Keet"
            ],
            "abstract": "Natural Language Interfaces and tools such as spellcheckers and Web search in\none's own language are known to be useful in ICT-mediated communication. Most\nlanguages in Southern Africa are under-resourced, however. Therefore, it would\nbe very useful if both the generic and the few language-specific NLP tools\ncould be reused or easily adapted across languages. This depends on the notion,\nand extent, of similarity between the languages. We assess this from the angle\nof orthography and corpora. Twelve versions of the Universal Declaration of\nHuman Rights (UDHR) are examined, showing clusters of languages, and which are\nthus more or less amenable to cross-language adaptation of NLP tools, which do\nnot match with Guthrie zones. To examine the generalisability of these results,\nwe zoom in on isiZulu both quantitatively and qualitatively with four other\ncorpora and texts in different genres. The results show that the UDHR is a\ntypical text document orthographically. The results also provide insight into\nusability of typical measures such as lexical diversity and genre, and that the\nsame statistic may mean different things in different documents. While NLTK for\nPython could be used for basic analyses of text, it, and similar NLP tools,\nwill need considerable customization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03448v1",
            "date": "2016-08-11 13:10:02+00:00",
            "title": "Sex, drugs, and violence",
            "authors": [
                "Stefania Raimondo",
                "Frank Rudzicz"
            ],
            "abstract": "Automatically detecting inappropriate content can be a difficult NLP task,\nrequiring understanding context and innuendo, not just identifying specific\nkeywords. Due to the large quantity of online user-generated content, automatic\ndetection is becoming increasingly necessary. We take a largely unsupervised\napproach using a large corpus of narratives from a community-based\nself-publishing website and a small segment of crowd-sourced annotations. We\nexplore topic modelling using latent Dirichlet allocation (and a variation),\nand use these to regress appropriateness ratings, effectively automating rating\nfor suitability. The results suggest that certain topics inferred may be useful\nin detecting latent inappropriateness -- yielding recall up to 96% and low\nregression errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03542v2",
            "date": "2016-08-11 17:34:12+00:00",
            "title": "WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia",
            "authors": [
                "Daniel Hewlett",
                "Alexandre Lacoste",
                "Llion Jones",
                "Illia Polosukhin",
                "Andrew Fandrianto",
                "Jay Han",
                "Matthew Kelcey",
                "David Berthelot"
            ],
            "abstract": "We present WikiReading, a large-scale natural language understanding task and\npublicly-available dataset with 18 million instances. The task is to predict\ntextual values from the structured knowledge base Wikidata by reading the text\nof the corresponding Wikipedia articles. The task contains a rich variety of\nchallenging classification and extraction sub-tasks, making it well-suited for\nend-to-end models such as deep neural networks (DNNs). We compare various\nstate-of-the-art DNN-based architectures for document classification,\ninformation extraction, and question answering. We find that models supporting\na rich answer space, such as word or character sequences, perform best. Our\nbest-performing model, a word-level sequence to sequence model with a mechanism\nto copy out-of-vocabulary words, obtains an accuracy of 71.8%.",
            "journal ref": "Proceedings of the 54th Annual Meeting of the Association for\n  Computational Linguistics (Volume 1: Long Papers), 2016, pp. 1535-1545",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03587v2",
            "date": "2016-08-11 09:01:04+00:00",
            "title": "The statistical trade-off between word order and word structure - large-scale evidence for the principle of least effort",
            "authors": [
                "Alexander Koplenig",
                "Peter Meyer",
                "Sascha Wolfer",
                "Carolin Mueller-Spitzer"
            ],
            "abstract": "Languages employ different strategies to transmit structural and grammatical\ninformation. While, for example, grammatical dependency relationships in\nsentences are mainly conveyed by the ordering of the words for languages like\nMandarin Chinese, or Vietnamese, the word ordering is much less restricted for\nlanguages such as Inupiatun or Quechua, as those languages (also) use the\ninternal structure of words (e.g. inflectional morphology) to mark grammatical\nrelationships in a sentence. Based on a quantitative analysis of more than\n1,500 unique translations of different books of the Bible in more than 1,100\ndifferent languages that are spoken as a native language by approximately 6\nbillion people (more than 80% of the world population), we present large-scale\nevidence for a statistical trade-off between the amount of information conveyed\nby the ordering of words and the amount of information conveyed by internal\nword structure: languages that rely more strongly on word order information\ntend to rely less on word structure information and vice versa. In addition, we\nfind that - despite differences in the way information is expressed - there is\nalso evidence for a trade-off between different books of the biblical canon\nthat recurs with little variation across languages: the more informative the\nword order of the book, the less informative its word structure and vice versa.\nWe argue that this might suggest that, on the one hand, languages encode\ninformation in very different (but efficient) ways. On the other hand,\ncontent-related and stylistic features are statistically encoded in very\nsimilar ways.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03803v1",
            "date": "2016-08-12 14:15:13+00:00",
            "title": "Redefining part-of-speech classes with distributional semantic models",
            "authors": [
                "Andrey Kutuzov",
                "Erik Velldal",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "This paper studies how word embeddings trained on the British National Corpus\ninteract with part of speech boundaries. Our work targets the Universal PoS tag\nset, which is currently actively being used for annotation of a range of\nlanguages. We experiment with training classifiers for predicting PoS tags for\nwords based on their embeddings. The results show that the information about\nPoS affiliation contained in the distributional vectors allows us to discover\ngroups of words with distributional patterns that differ from other words of\nthe same part of speech.\n  This data often reveals hidden inconsistencies of the annotation process or\nguidelines. At the same time, it supports the notion of `soft' or `graded' part\nof speech affiliations. Finally, we show that information about PoS is\ndistributed among dozens of vector components, not limited to only one or two\nfeatures.",
            "journal ref": "Proceedings of The 20th SIGNLL Conference on Computational Natural\n  Language Learning, 2016, pp. 115-125",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03995v2",
            "date": "2016-08-13 15:54:10+00:00",
            "title": "An Analysis of Lemmatization on Topic Models of Morphologically Rich Language",
            "authors": [
                "Chandler May",
                "Ryan Cotterell",
                "Benjamin Van Durme"
            ],
            "abstract": "Topic models are typically represented by top-$m$ word lists for human\ninterpretation. The corpus is often pre-processed with lemmatization (or\nstemming) so that those representations are not undermined by a proliferation\nof words with similar meanings, but there is little public work on the effects\nof that pre-processing. Recent work studied the effect of stemming on topic\nmodels of English texts and found no supporting evidence for the practice. We\nstudy the effect of lemmatization on topic models of Russian Wikipedia\narticles, finding in one configuration that it significantly improves\ninterpretability according to a word intrusion metric. We conclude that\nlemmatization may benefit topic models on morphologically rich languages, but\nthat further investigation is needed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04207v3",
            "date": "2016-08-15 08:51:38+00:00",
            "title": "Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks",
            "authors": [
                "Yossi Adi",
                "Einat Kermany",
                "Yonatan Belinkov",
                "Ofer Lavi",
                "Yoav Goldberg"
            ],
            "abstract": "There is a lot of research interest in encoding variable length sentences\ninto fixed length vectors, in a way that preserves the sentence meanings. Two\ncommon methods include representations based on averaging word vectors, and\nrepresentations based on the hidden states of recurrent neural networks such as\nLSTMs. The sentence vectors are used as features for subsequent machine\nlearning tasks or for pre-training in the context of deep learning. However,\nnot much is known about the properties that are encoded in these sentence\nrepresentations and about the language information they capture. We propose a\nframework that facilitates better understanding of the encoded representations.\nWe define prediction tasks around isolated aspects of sentence structure\n(namely sentence length, word content, and word order), and score\nrepresentations by the ability to train a classifier to solve each prediction\ntask when using the representation as input. We demonstrate the potential\ncontribution of the approach by analyzing different sentence representation\nmechanisms. The analysis sheds light on the relative strengths of different\nsentence embedding methods with respect to these low level prediction tasks,\nand on the effect of the encoded vector's dimensionality on the resulting\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04434v1",
            "date": "2016-08-15 23:09:21+00:00",
            "title": "Natural Language Processing using Hadoop and KOSHIK",
            "authors": [
                "Emre Erturk",
                "Hong Shi"
            ],
            "abstract": "Natural language processing, as a data analytics related technology, is used\nwidely in many research areas such as artificial intelligence, human language\nprocessing, and translation. At present, due to explosive growth of data, there\nare many challenges for natural language processing. Hadoop is one of the\nplatforms that can process the large amount of data required for natural\nlanguage processing. KOSHIK is one of the natural language processing\narchitectures, and utilizes Hadoop and contains language processing components\nsuch as Stanford CoreNLP and OpenNLP. This study describes how to build a\nKOSHIK platform with the relevant tools, and provides the steps to analyze wiki\ndata. Finally, it evaluates and discusses the advantages and disadvantages of\nthe KOSHIK architecture, and gives recommendations on improving the processing\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04465v1",
            "date": "2016-08-16 02:33:21+00:00",
            "title": "Fast, Small and Exact: Infinite-order Language Modelling with Compressed Suffix Trees",
            "authors": [
                "Ehsan Shareghi",
                "Matthias Petri",
                "Gholamreza Haffari",
                "Trevor Cohn"
            ],
            "abstract": "Efficient methods for storing and querying are critical for scaling\nhigh-order n-gram language models to large corpora. We propose a language model\nbased on compressed suffix trees, a representation that is highly compact and\ncan be easily held in memory, while supporting queries needed in computing\nlanguage model probabilities on-the-fly. We present several optimisations which\nimprove query runtimes up to 2500x, despite only incurring a modest increase in\nconstruction time and memory usage. For large corpora and high Markov orders,\nour method is highly competitive with the state-of-the-art KenLM package. It\nimposes much lower memory requirements, often by orders of magnitude, and has\nruntimes that are either similar (for training) or comparable (for querying).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04485v1",
            "date": "2016-08-16 05:04:46+00:00",
            "title": "Authorship clustering using multi-headed recurrent neural networks",
            "authors": [
                "Douglas Bagnall"
            ],
            "abstract": "A recurrent neural network that has been trained to separately model the\nlanguage of several documents by unknown authors is used to measure similarity\nbetween the documents. It is able to find clues of common authorship even when\nthe documents are very short and about disparate topics. While it is easy to\nmake statistically significant predictions regarding authorship, it is\ndifficult to group documents into definite clusters with high accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04631v2",
            "date": "2016-08-16 15:04:18+00:00",
            "title": "Neural versus Phrase-Based Machine Translation Quality: a Case Study",
            "authors": [
                "Luisa Bentivogli",
                "Arianna Bisazza",
                "Mauro Cettolo",
                "Marcello Federico"
            ],
            "abstract": "Within the field of Statistical Machine Translation (SMT), the neural\napproach (NMT) has recently emerged as the first technology able to challenge\nthe long-standing dominance of phrase-based approaches (PBMT). In particular,\nat the IWSLT 2015 evaluation campaign, NMT outperformed well established\nstate-of-the-art PBMT systems on English-German, a language pair known to be\nparticularly hard because of morphology and syntactic differences. To\nunderstand in what respects NMT provides better translation quality than PBMT,\nwe perform a detailed analysis of neural versus phrase-based SMT outputs,\nleveraging high quality post-edits performed by professional translators on the\nIWSLT data. For the first time, our analysis provides useful insights on what\nlinguistic phenomena are best modeled by neural models -- such as the\nreordering of verbs -- while pointing out other aspects that remain to be\nimproved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04767v1",
            "date": "2016-08-14 16:23:55+00:00",
            "title": "Proceedings of the LexSem+Logics Workshop 2016",
            "authors": [
                "Steven Neale",
                "Valeria de Paiva",
                "Arantxa Otegi",
                "Alexandre Rademaker"
            ],
            "abstract": "Lexical semantics continues to play an important role in driving research\ndirections in NLP, with the recognition and understanding of context becoming\nincreasingly important in delivering successful outcomes in NLP tasks. Besides\ntraditional processing areas such as word sense and named entity\ndisambiguation, the creation and maintenance of dictionaries, annotated corpora\nand resources have become cornerstones of lexical semantics research and\nproduced a wealth of contextual information that NLP processes can exploit. New\nefforts both to link and construct from scratch such information - as Linked\nOpen Data or by way of formal tools coming from logic, ontologies and automated\nreasoning - have increased the interoperability and accessibility of resources\nfor lexical and computational semantics, even in those languages for which they\nhave previously been limited.\n  LexSem+Logics 2016 combines the 1st Workshop on Lexical Semantics for\nLesser-Resources Languages and the 3rd Workshop on Logics and Ontologies. The\naccepted papers in our program covered topics across these two areas,\nincluding: the encoding of plurals in Wordnets, the creation of a thesaurus\nfrom multiple sources based on semantic similarity metrics, and the use of\ncross-lingual treebanks and annotations for universal part-of-speech tagging.\nWe also welcomed talks from two distinguished speakers: on Portuguese lexical\nknowledge bases (different approaches, results and their application in NLP\ntasks) and on new strategies for open information extraction (the capture of\nverb-based propositions from massive text corpora).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04983v1",
            "date": "2016-08-17 14:43:17+00:00",
            "title": "Ensemble of Jointly Trained Deep Neural Network-Based Acoustic Models for Reverberant Speech Recognition",
            "authors": [
                "Jeehye Lee",
                "Myungin Lee",
                "Joon-Hyuk Chang"
            ],
            "abstract": "Distant speech recognition is a challenge, particularly due to the corruption\nof speech signals by reverberation caused by large distances between the\nspeaker and microphone. In order to cope with a wide range of reverberations in\nreal-world situations, we present novel approaches for acoustic modeling\nincluding an ensemble of deep neural networks (DNNs) and an ensemble of jointly\ntrained DNNs. First, multiple DNNs are established, each of which corresponds\nto a different reverberation time 60 (RT60) in a setup step. Also, each model\nin the ensemble of DNN acoustic models is further jointly trained, including\nboth feature mapping and acoustic modeling, where the feature mapping is\ndesigned for the dereverberation as a front-end. In a testing phase, the two\nmost likely DNNs are chosen from the DNN ensemble using maximum a posteriori\n(MAP) probabilities, computed in an online fashion by using maximum likelihood\n(ML)-based blind RT60 estimation and then the posterior probability outputs\nfrom two DNNs are combined using the ML-based weights as a simple average.\nExtensive experiments demonstrate that the proposed approach leads to\nsubstantial improvements in speech recognition accuracy over the conventional\nDNN baseline systems under diverse reverberant conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05014v4",
            "date": "2016-08-17 16:27:49+00:00",
            "title": "Path-based vs. Distributional Information in Recognizing Lexical Semantic Relations",
            "authors": [
                "Vered Shwartz",
                "Ido Dagan"
            ],
            "abstract": "Recognizing various semantic relations between terms is beneficial for many\nNLP tasks. While path-based and distributional information sources are\nconsidered complementary for this task, the superior results the latter showed\nrecently suggested that the former's contribution might have become obsolete.\nWe follow the recent success of an integrated neural method for hypernymy\ndetection (Shwartz et al., 2016) and extend it to recognize multiple relations.\nThe empirical results show that this method is effective in the multiclass\nsetting as well. We further show that the path-based information source always\ncontributes to the classification, and analyze the cases in which it mostly\ncomplements the distributional information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05129v1",
            "date": "2016-08-17 23:32:57+00:00",
            "title": "SlangSD: Building and Using a Sentiment Dictionary of Slang Words for Short-Text Sentiment Classification",
            "authors": [
                "Liang Wu",
                "Fred Morstatter",
                "Huan Liu"
            ],
            "abstract": "Sentiment in social media is increasingly considered as an important resource\nfor customer segmentation, market understanding, and tackling other\nsocio-economic issues. However, sentiment in social media is difficult to\nmeasure since user-generated content is usually short and informal. Although\nmany traditional sentiment analysis methods have been proposed, identifying\nslang sentiment words remains untackled. One of the reasons is that slang\nsentiment words are not available in existing dictionaries or sentiment\nlexicons. To this end, we propose to build the first sentiment dictionary of\nslang words to aid sentiment analysis of social media content. It is laborious\nand time-consuming to collect and label the sentiment polarity of a\ncomprehensive list of slang words. We present an approach to leverage web\nresources to construct an extensive Slang Sentiment word Dictionary (SlangSD)\nthat is easy to maintain and extend. SlangSD is publicly available for research\npurposes. We empirically show the advantages of using SlangSD, the newly-built\nslang sentiment word dictionary for sentiment classification, and provide\nexamples demonstrating its ease of use with an existing sentiment system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05243v1",
            "date": "2016-08-18 11:41:45+00:00",
            "title": "Multilingual Modal Sense Classification using a Convolutional Neural Network",
            "authors": [
                "Ana Marasovi\u0107",
                "Anette Frank"
            ],
            "abstract": "Modal sense classification (MSC) is a special WSD task that depends on the\nmeaning of the proposition in the modal's scope. We explore a CNN architecture\nfor classifying modal sense in English and German. We show that CNNs are\nsuperior to manually designed feature-based classifiers and a standard NN\nclassifier. We analyze the feature maps learned by the CNN and identify known\nand previously unattested linguistic features. We benchmark the CNN on a\nstandard WSD task, where it compares favorably to models using\nsense-disambiguated target vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05374v1",
            "date": "2016-08-18 18:58:39+00:00",
            "title": "DNN-based Speech Synthesis for Indian Languages from ASCII text",
            "authors": [
                "Srikanth Ronanki",
                "Siva Reddy",
                "Bajibabu Bollepalli",
                "Simon King"
            ],
            "abstract": "Text-to-Speech synthesis in Indian languages has a seen lot of progress over\nthe decade partly due to the annual Blizzard challenges. These systems assume\nthe text to be written in Devanagari or Dravidian scripts which are nearly\nphonemic orthography scripts. However, the most common form of computer\ninteraction among Indians is ASCII written transliterated text. Such text is\ngenerally noisy with many variations in spelling for the same word. In this\npaper we evaluate three approaches to synthesize speech from such noisy ASCII\ntext: a naive Uni-Grapheme approach, a Multi-Grapheme approach, and a\nsupervised Grapheme-to-Phoneme (G2P) approach. These methods first convert the\nASCII text to a phonetic script, and then learn a Deep Neural Network to\nsynthesize speech from that. We train and test our models on Blizzard Challenge\ndatasets that were transliterated to ASCII using crowdsourcing. Our experiments\non Hindi, Tamil and Telugu demonstrate that our models generate speech of\ncompetetive quality from ASCII text compared to the speech synthesized from the\nnative scripts. All the accompanying transliterated datasets are released for\npublic access.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05426v2",
            "date": "2016-08-18 20:27:46+00:00",
            "title": "A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments",
            "authors": [
                "Omer Levy",
                "Anders S\u00f8gaard",
                "Yoav Goldberg"
            ],
            "abstract": "While cross-lingual word embeddings have been studied extensively in recent\nyears, the qualitative differences between the different algorithms remain\nvague. We observe that whether or not an algorithm uses a particular feature\nset (sentence IDs) accounts for a significant performance gap among these\nalgorithms. This feature set is also used by traditional alignment algorithms,\nsuch as IBM Model-1, which demonstrate similar performance to state-of-the-art\nembedding algorithms on a variety of benchmarks. Overall, we observe that\ndifferent algorithmic approaches for utilizing the sentence ID feature space\nresult in similar performance. This paper draws both empirical and theoretical\nparallels between the embedding and alignment literature, and suggests that\nadding additional sources of information, which go beyond the traditional\nsignal of bilingual sentence-aligned corpora, may substantially improve\ncross-lingual word embeddings, and that future baselines should at least take\nsuch features into account.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05457v1",
            "date": "2016-08-19 00:13:10+00:00",
            "title": "Who did What: A Large-Scale Person-Centered Cloze Dataset",
            "authors": [
                "Takeshi Onishi",
                "Hai Wang",
                "Mohit Bansal",
                "Kevin Gimpel",
                "David McAllester"
            ],
            "abstract": "We have constructed a new \"Who-did-What\" dataset of over 200,000\nfill-in-the-gap (cloze) multiple choice reading comprehension problems\nconstructed from the LDC English Gigaword newswire corpus. The WDW dataset has\na variety of novel features. First, in contrast with the CNN and Daily Mail\ndatasets (Hermann et al., 2015) we avoid using article summaries for question\nformation. Instead, each problem is formed from two independent articles --- an\narticle given as the passage to be read and a separate article on the same\nevents used to form the question. Second, we avoid anonymization --- each\nchoice is a person named entity. Third, the problems have been filtered to\nremove a fraction that are easily solved by simple baselines, while remaining\n84% solvable by humans. We report performance benchmarks of standard systems\nand propose the WDW dataset as a challenge task for the community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05528v3",
            "date": "2016-08-19 08:30:35+00:00",
            "title": "Automatic Selection of Context Configurations for Improved Class-Specific Word Representations",
            "authors": [
                "Ivan Vuli\u0107",
                "Roy Schwartz",
                "Ari Rappoport",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "This paper is concerned with identifying contexts useful for training word\nrepresentation models for different word classes such as adjectives (A), verbs\n(V), and nouns (N). We introduce a simple yet effective framework for an\nautomatic selection of class-specific context configurations. We construct a\ncontext configuration space based on universal dependency relations between\nwords, and efficiently search this space with an adapted beam search algorithm.\nIn word similarity tasks for each word class, we show that our framework is\nboth effective and efficient. Particularly, it improves the Spearman's rho\ncorrelation with human scores on SimLex-999 over the best previously proposed\nclass-specific contexts by 6 (A), 6 (V) and 5 (N) rho points. With our selected\ncontext configurations, we train on only 14% (A), 26.2% (V), and 33.6% (N) of\nall dependency-based contexts, resulting in a reduced training time. Our\nresults generalise: we show that the configurations our algorithm learns for\none English training setup outperform previously proposed context types in\nanother training setup for English. Moreover, basing the configuration space on\nuniversal dependencies, it is possible to transfer the learned configurations\nto German and Italian. We also demonstrate improved per-class results over\nother context types in these two languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05554v1",
            "date": "2016-08-19 09:48:13+00:00",
            "title": "Learning to Start for Sequence to Sequence Architecture",
            "authors": [
                "Qingfu Zhu",
                "Weinan Zhang",
                "Lianqiang Zhou",
                "Ting Liu"
            ],
            "abstract": "The sequence to sequence architecture is widely used in the response\ngeneration and neural machine translation to model the potential relationship\nbetween two sentences. It typically consists of two parts: an encoder that\nreads from the source sentence and a decoder that generates the target sentence\nword by word according to the encoder's output and the last generated word.\nHowever, it faces to the cold start problem when generating the first word as\nthere is no previous word to refer. Existing work mainly use a special start\nsymbol </s>to generate the first word. An obvious drawback of these work is\nthat there is not a learnable relationship between words and the start symbol.\nFurthermore, it may lead to the error accumulation for decoding when the first\nword is incorrectly generated. In this paper, we proposed a novel approach to\nlearning to generate the first word in the sequence to sequence architecture\nrather than using the start symbol. Experimental results on the task of\nresponse generation of short text conversation show that the proposed approach\noutperforms the state-of-the-art approach in both of the automatic and manual\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05604v2",
            "date": "2016-08-19 14:03:46+00:00",
            "title": "Modeling Human Reading with Neural Attention",
            "authors": [
                "Michael Hahn",
                "Frank Keller"
            ],
            "abstract": "When humans read text, they fixate some words and skip others. However, there\nhave been few attempts to explain skipping behavior with computational models,\nas most existing work has focused on predicting reading times (e.g.,~using\nsurprisal). In this paper, we propose a novel approach that models both\nskipping and reading, using an unsupervised architecture that combines a neural\nattention with autoencoding, trained on raw text using reinforcement learning.\nOur model explains human reading behavior as a tradeoff between precision of\nlanguage understanding (encoding the input accurately) and economy of attention\n(fixating as few words as possible). We evaluate the model on the Dundee\neye-tracking corpus, showing that it accurately predicts skipping behavior and\nreading times, is competitive with surprisal, and captures known qualitative\nfeatures of human reading.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05605v1",
            "date": "2016-08-19 14:05:03+00:00",
            "title": "Using Distributed Representations to Disambiguate Biomedical and Clinical Concepts",
            "authors": [
                "St\u00e9phan Tulkens",
                "Simon \u0160uster",
                "Walter Daelemans"
            ],
            "abstract": "In this paper, we report a knowledge-based method for Word Sense\nDisambiguation in the domains of biomedical and clinical text. We combine word\nrepresentations created on large corpora with a small number of definitions\nfrom the UMLS to create concept representations, which we then compare to\nrepresentations of the context of ambiguous terms. Using no relational\ninformation, we obtain comparable performance to previous approaches on the\nMSH-WSD dataset, which is a well-known dataset in the biomedical domain.\nAdditionally, our method is fast and easy to set up and extend to other\ndomains. Supplementary materials, including source code, can be found at https:\n//github.com/clips/yarn",
            "journal ref": "Proceedings of the 15th Workshop on Biomedical Natural Language\n  Processing, Berlin, Germany, 2016, pages 77-82. Association for Computational\n  Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05777v1",
            "date": "2016-08-20 03:43:29+00:00",
            "title": "Topic Sensitive Neural Headline Generation",
            "authors": [
                "Lei Xu",
                "Ziyun Wang",
                "Ayana",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Neural models have recently been used in text summarization including\nheadline generation. The model can be trained using a set of document-headline\npairs. However, the model does not explicitly consider topical similarities and\ndifferences of documents. We suggest to categorizing documents into various\ntopics so that documents within the same topic are similar in content and share\nsimilar summarization patterns. Taking advantage of topic information of\ndocuments, we propose topic sensitive neural headline generation model. Our\nmodel can generate more accurate summaries guided by document topics. We test\nour model on LCSTS dataset, and experiments show that our method outperforms\nother baselines on each topic and achieves the state-of-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05859v3",
            "date": "2016-08-20 18:32:05+00:00",
            "title": "Using the Output Embedding to Improve Language Models",
            "authors": [
                "Ofir Press",
                "Lior Wolf"
            ],
            "abstract": "We study the topmost weight matrix of neural network language models. We show\nthat this matrix constitutes a valid word embedding. When training language\nmodels, we recommend tying the input embedding and this output embedding. We\nanalyze the resulting update rules and show that the tied embedding evolves in\na more similar way to the output embedding than to the input embedding in the\nuntied model. We also offer a new method of regularizing the output embedding.\nOur methods lead to a significant reduction in perplexity, as we are able to\nshow on a variety of neural network language models. Finally, we show that\nweight tying can reduce the size of neural translation models to less than half\nof their original size without harming their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06043v3",
            "date": "2016-08-22 03:19:27+00:00",
            "title": "Context Gates for Neural Machine Translation",
            "authors": [
                "Zhaopeng Tu",
                "Yang Liu",
                "Zhengdong Lu",
                "Xiaohua Liu",
                "Hang Li"
            ],
            "abstract": "In neural machine translation (NMT), generation of a target word depends on\nboth source and target contexts. We find that source contexts have a direct\nimpact on the adequacy of a translation while target contexts affect the\nfluency. Intuitively, generation of a content word should rely more on the\nsource context and generation of a functional word should rely more on the\ntarget context. Due to the lack of effective control over the influence from\nsource and target contexts, conventional NMT tends to yield fluent but\ninadequate translations. To address this problem, we propose context gates\nwhich dynamically control the ratios at which source and target contexts\ncontribute to the generation of target words. In this way, we can enhance both\nthe adequacy and fluency of NMT with more careful control of the information\nflow from contexts. Experiments show that our approach significantly improves\nupon a standard attention-based NMT system by +2.3 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06111v5",
            "date": "2016-08-22 10:30:18+00:00",
            "title": "An Incremental Parser for Abstract Meaning Representation",
            "authors": [
                "Marco Damonte",
                "Shay B. Cohen",
                "Giorgio Satta"
            ],
            "abstract": "Meaning Representation (AMR) is a semantic representation for natural\nlanguage that embeds annotations related to traditional tasks such as named\nentity recognition, semantic role labeling, word sense disambiguation and\nco-reference resolution. We describe a transition-based parser for AMR that\nparses sentences left-to-right, in linear time. We further propose a test-suite\nthat assesses specific subtasks that are helpful in comparing AMR parsers, and\nshow that our parser is competitive with the state of the art on the LDC2015E86\ndataset and that it outperforms state-of-the-art parsers for recovering named\nentities and handling polarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06134v2",
            "date": "2016-08-22 11:52:55+00:00",
            "title": "Median-Based Generation of Synthetic Speech Durations using a Non-Parametric Approach",
            "authors": [
                "Srikanth Ronanki",
                "Oliver Watts",
                "Simon King",
                "Gustav Eje Henter"
            ],
            "abstract": "This paper proposes a new approach to duration modelling for statistical\nparametric speech synthesis in which a recurrent statistical model is trained\nto output a phone transition probability at each timestep (acoustic frame).\nUnlike conventional approaches to duration modelling -- which assume that\nduration distributions have a particular form (e.g., a Gaussian) and use the\nmean of that distribution for synthesis -- our approach can in principle model\nany distribution supported on the non-negative integers. Generation from this\nmodel can be performed in many ways; here we consider output generation based\non the median predicted duration. The median is more typical (more probable)\nthan the conventional mean duration, is robust to training-data irregularities,\nand enables incremental generation. Furthermore, a frame-level approach to\nduration prediction is consistent with a longer-term goal of modelling\ndurations and acoustic features together. Results indicate that the proposed\nmethod is competitive with baseline approaches in approximating the median\nduration of held-out natural speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06378v1",
            "date": "2016-08-23 04:27:41+00:00",
            "title": "Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine",
            "authors": [
                "Bo-Hsiang Tseng",
                "Sheng-Syun Shen",
                "Hung-Yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "Multimedia or spoken content presents more attractive information than plain\ntext content, but it's more difficult to display on a screen and be selected by\na user. As a result, accessing large collections of the former is much more\ndifficult and time-consuming than the latter for humans. It's highly attractive\nto develop a machine which can automatically understand spoken content and\nsummarize the key information for humans to browse over. In this endeavor, we\npropose a new task of machine comprehension of spoken content. We define the\ninitial goal as the listening comprehension test of TOEFL, a challenging\nacademic English examination for English learners whose native language is not\nEnglish. We further propose an Attention-based Multi-hop Recurrent Neural\nNetwork (AMRNN) architecture for this task, achieving encouraging results in\nthe initial tests. Initial results also have shown that word-level attention is\nprobably more robust than sentence-level attention for this task with ASR\nerrors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06386v1",
            "date": "2016-08-23 05:27:46+00:00",
            "title": "Which techniques does your application use?: An information extraction framework for scientific articles",
            "authors": [
                "Soham Dan",
                "Sanyam Agarwal",
                "Mayank Singh",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "Every field of research consists of multiple application areas with various\ntechniques routinely used to solve problems in these wide range of application\nareas. With the exponential growth in research volumes, it has become difficult\nto keep track of the ever-growing number of application areas as well as the\ncorresponding problem solving techniques. In this paper, we consider the\ncomputational linguistics domain and present a novel information extraction\nsystem that automatically constructs a pool of all application areas in this\ndomain and appropriately links them with corresponding problem solving\ntechniques. Further, we categorize individual research articles based on their\napplication area and the techniques proposed/used in the article. k-gram based\ndiscounting method along with handwritten rules and bootstrapped pattern\nlearning is employed to extract application areas. Subsequently, a language\nmodeling approach is proposed to characterize each article based on its\napplication area. Similarly, regular expressions and high-scoring noun phrases\nare used for the extraction of the problem solving techniques. We propose a\ngreedy approach to characterize each article based on the techniques. Towards\nthe end, we present a table representing the most frequent techniques adopted\nfor a particular application area. Finally, we propose three use cases\npresenting an extensive temporal analysis of the usage of techniques and\napplication areas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06697v1",
            "date": "2016-08-24 03:36:04+00:00",
            "title": "Semantic descriptions of 24 evaluational adjectives, for application in sentiment analysis",
            "authors": [
                "Cliff Goddard",
                "Maite Taboada",
                "Radoslava Trnavac"
            ],
            "abstract": "We apply the Natural Semantic Metalanguage (NSM) approach (Goddard and\nWierzbicka 2014) to the lexical-semantic analysis of English evaluational\nadjectives and compare the results with the picture developed in the Appraisal\nFramework (Martin and White 2005). The analysis is corpus-assisted, with\nexamples mainly drawn from film and book reviews, and supported by\ncollocational and statistical information from WordBanks Online. We propose NSM\nexplications for 24 evaluational adjectives, arguing that they fall into five\ngroups, each of which corresponds to a distinct semantic template. The groups\ncan be sketched as follows: \"First-person thought-plus-affect\", e.g. wonderful;\n\"Experiential\", e.g. entertaining; \"Experiential with bodily reaction\", e.g.\ngripping; \"Lasting impact\", e.g. memorable; \"Cognitive evaluation\", e.g.\ncomplex, excellent. These groupings and semantic templates are compared with\nthe classifications in the Appraisal Framework's system of Appreciation. In\naddition, we are particularly interested in sentiment analysis, the automatic\nidentification of evaluation and subjectivity in text. We discuss the relevance\nof the two frameworks for sentiment analysis and other language technology\napplications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06718v1",
            "date": "2016-08-24 05:30:45+00:00",
            "title": "A Large-Scale Multilingual Disambiguation of Glosses",
            "authors": [
                "Jos\u00e9 Camacho Collados",
                "Claudio Delli Bovi",
                "Alessandro Raganato",
                "Roberto Navigli"
            ],
            "abstract": "Linking concepts and named entities to knowledge bases has become a crucial\nNatural Language Understanding task. In this respect, recent works have shown\nthe key advantage of exploiting textual definitions in various Natural Language\nProcessing applications. However, to date there are no reliable large-scale\ncorpora of sense-annotated textual definitions available to the research\ncommunity. In this paper we present a large-scale high-quality corpus of\ndisambiguated glosses in multiple languages, comprising sense annotations of\nboth concepts and named entities from a unified sense inventory. Our approach\nfor the construction and disambiguation of the corpus builds upon the structure\nof a large multilingual semantic network and a state-of-the-art disambiguation\nsystem; first, we gather complementary information of equivalent definitions\nacross different languages to provide context for disambiguation, and then we\ncombine it with a semantic similarity-based refinement. As a result we obtain a\nmultilingual corpus of textual definitions featuring over 38 million\ndefinitions in 263 languages, and we make it freely available at\nhttp://lcl.uniroma1.it/disambiguated-glosses. Experiments on Open Information\nExtraction and Sense Clustering show how two state-of-the-art approaches\nimprove their performance by integrating our disambiguated corpus into their\npipeline.",
            "journal ref": "Proceedings of the Tenth International Conference on Language\n  Resources and Evaluation (LREC), 2016, pages 1701-1708, Portoroz, Slovenia",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06757v1",
            "date": "2016-08-24 09:06:14+00:00",
            "title": "Robust Named Entity Recognition in Idiosyncratic Domains",
            "authors": [
                "Sebastian Arnold",
                "Felix A. Gers",
                "Torsten Kilias",
                "Alexander L\u00f6ser"
            ],
            "abstract": "Named entity recognition often fails in idiosyncratic domains. That causes a\nproblem for depending tasks, such as entity linking and relation extraction. We\npropose a generic and robust approach for high-recall named entity recognition.\nOur approach is easy to train and offers strong generalization over diverse\ndomain-specific language, such as news documents (e.g. Reuters) or biomedical\ntext (e.g. Medline). Our approach is based on deep contextual sequence learning\nand utilizes stacked bidirectional LSTM networks. Our model is trained with\nonly few hundred labeled sentences and does not rely on further external\nknowledge. We report from our results F1 scores in the range of 84-94% on\nstandard datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06794v1",
            "date": "2016-08-24 12:38:45+00:00",
            "title": "Improving Sparse Word Representations with Distributional Inference for Semantic Composition",
            "authors": [
                "Thomas Kober",
                "Julie Weeds",
                "Jeremy Reffin",
                "David Weir"
            ],
            "abstract": "Distributional models are derived from co-occurrences in a corpus, where only\na small proportion of all possible plausible co-occurrences will be observed.\nThis results in a very sparse vector space, requiring a mechanism for inferring\nmissing knowledge. Most methods face this challenge in ways that render the\nresulting word representations uninterpretable, with the consequence that\nsemantic composition becomes hard to model. In this paper we explore an\nalternative which involves explicitly inferring unobserved co-occurrences using\nthe distributional neighbourhood. We show that distributional inference\nimproves sparse word representations on several word similarity benchmarks and\ndemonstrate that our model is competitive with the state-of-the-art for\nadjective-noun, noun-noun and verb-object compositions while being fully\ninterpretable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07076v1",
            "date": "2016-08-25 10:43:56+00:00",
            "title": "A Context-aware Natural Language Generator for Dialogue Systems",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "We present a novel natural language generation system for spoken dialogue\nsystems capable of entraining (adapting) to users' way of speaking, providing\ncontextually appropriate responses. The generator is based on recurrent neural\nnetworks and the sequence-to-sequence approach. It is fully trainable from data\nwhich include preceding context along with responses to be generated. We show\nthat the context-aware generator yields significant improvements over the\nbaseline in both automatic metrics and a human pairwise preference test.",
            "journal ref": "Proceedings of the SIGDIAL 2016 Conference, pages 185-190, Los\n  Angeles, USA, 13-15 September 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07115v1",
            "date": "2016-08-25 12:44:05+00:00",
            "title": "Aligning Packed Dependency Trees: a theory of composition for distributional semantics",
            "authors": [
                "David Weir",
                "Julie Weeds",
                "Jeremy Reffin",
                "Thomas Kober"
            ],
            "abstract": "We present a new framework for compositional distributional semantics in\nwhich the distributional contexts of lexemes are expressed in terms of anchored\npacked dependency trees. We show that these structures have the potential to\ncapture the full sentential contexts of a lexeme and provide a uniform basis\nfor the composition of distributional knowledge in a way that captures both\nmutual disambiguation and generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07720v1",
            "date": "2016-08-27 15:41:22+00:00",
            "title": "A Bi-LSTM-RNN Model for Relation Classification Using Low-Cost Sequence Features",
            "authors": [
                "Fei Li",
                "Meishan Zhang",
                "Guohong Fu",
                "Tao Qian",
                "Donghong Ji"
            ],
            "abstract": "Relation classification is associated with many potential applications in the\nartificial intelligence area. Recent approaches usually leverage neural\nnetworks based on structure features such as syntactic or dependency features\nto solve this problem. However, high-cost structure features make such\napproaches inconvenient to be directly used. In addition, structure features\nare probably domain-dependent. Therefore, this paper proposes a bi-directional\nlong-short-term-memory recurrent-neural-network (Bi-LSTM-RNN) model based on\nlow-cost sequence features to address relation classification. This model\ndivides a sentence or text segment into five parts, namely two target entities\nand their three contexts. It learns the representations of entities and their\ncontexts, and uses them to classify relations. We evaluate our model on two\nstandard benchmark datasets in different domains, namely SemEval-2010 Task 8\nand BioNLP-ST 2016 Task BB3. In the former dataset, our model achieves\ncomparable performance compared with other models using sequence features. In\nthe latter dataset, our model obtains the third best results compared with\nother models in the official evaluation. Moreover, we find that the context\nbetween two target entities plays the most important role in relation\nclassification. Furthermore, statistic experiments show that the context\nbetween two target entities can be used as an approximate replacement of the\nshortest dependency path when dependency parsing is not used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07738v2",
            "date": "2016-08-27 19:57:55+00:00",
            "title": "Testing APSyn against Vector Cosine on Similarity Estimation",
            "authors": [
                "Enrico Santus",
                "Emmanuele Chersoni",
                "Alessandro Lenci",
                "Chu-Ren Huang",
                "Philippe Blache"
            ],
            "abstract": "In Distributional Semantic Models (DSMs), Vector Cosine is widely used to\nestimate similarity between word vectors, although this measure was noticed to\nsuffer from several shortcomings. The recent literature has proposed other\nmethods which attempt to mitigate such biases. In this paper, we intend to\ninvestigate APSyn, a measure that computes the extent of the intersection\nbetween the most associated contexts of two target words, weighting it by\ncontext relevance. We evaluated this metric in a similarity estimation task on\nseveral popular test sets, and our results show that APSyn is in fact highly\ncompetitive, even with respect to the results reported in the literature for\nword embeddings. On top of it, APSyn addresses some of the weaknesses of Vector\nCosine, performing well also on genuine similarity estimation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07775v3",
            "date": "2016-08-28 06:48:14+00:00",
            "title": "Hierarchical Attention Model for Improved Machine Comprehension of Spoken Content",
            "authors": [
                "Wei Fang",
                "Jui-Yang Hsu",
                "Hung-yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "Multimedia or spoken content presents more attractive information than plain\ntext content, but the former is more difficult to display on a screen and be\nselected by a user. As a result, accessing large collections of the former is\nmuch more difficult and time-consuming than the latter for humans. It's\ntherefore highly attractive to develop machines which can automatically\nunderstand spoken content and summarize the key information for humans to\nbrowse over. In this endeavor, a new task of machine comprehension of spoken\ncontent was proposed recently. The initial goal was defined as the listening\ncomprehension test of TOEFL, a challenging academic English examination for\nEnglish learners whose native languages are not English. An Attention-based\nMulti-hop Recurrent Neural Network (AMRNN) architecture was also proposed for\nthis task, which considered only the sequential relationship within the speech\nutterances. In this paper, we propose a new Hierarchical Attention Model (HAM),\nwhich constructs multi-hopped attention mechanism over tree-structured rather\nthan sequential representations for the utterances. Improved comprehension\nperformance robust with respect to ASR errors were obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.07836v1",
            "date": "2016-08-28 17:51:41+00:00",
            "title": "What to do about non-standard (or non-canonical) language in NLP",
            "authors": [
                "Barbara Plank"
            ],
            "abstract": "Real world data differs radically from the benchmark corpora we use in\nnatural language processing (NLP). As soon as we apply our technologies to the\nreal world, performance drops. The reason for this problem is obvious: NLP\nmodels are trained on samples from a limited set of canonical varieties that\nare considered standard, most prominently English newswire. However, there are\nmany dimensions, e.g., socio-demographics, language, genre, sentence type, etc.\non which texts can differ from the standard. The solution is not obvious: we\ncannot control for all factors, and it is not clear how to best go beyond the\ncurrent practice of training on homogeneous data from a single domain and\nlanguage.\n  In this paper, I review the notion of canonicity, and how it shapes our\ncommunity's approach to language. I argue for leveraging what I call fortuitous\ndata, i.e., non-obvious data that is hitherto neglected, hidden in plain sight,\nor raw data that needs to be refined. If we embrace the variety of this\nheterogeneous data by combining it with proper algorithms, we will not only\nproduce more robust models, but will also enable adaptive language technology\ncapable of addressing natural language variation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08738v1",
            "date": "2016-08-31 06:28:28+00:00",
            "title": "A Dictionary-based Approach to Racism Detection in Dutch Social Media",
            "authors": [
                "St\u00e9phan Tulkens",
                "Lisa Hilte",
                "Elise Lodewyckx",
                "Ben Verhoeven",
                "Walter Daelemans"
            ],
            "abstract": "We present a dictionary-based approach to racism detection in Dutch social\nmedia comments, which were retrieved from two public Belgian social media sites\nlikely to attract racist reactions. These comments were labeled as racist or\nnon-racist by multiple annotators. For our approach, three discourse\ndictionaries were created: first, we created a dictionary by retrieving\npossibly racist and more neutral terms from the training data, and then\naugmenting these with more general words to remove some bias. A second\ndictionary was created through automatic expansion using a \\texttt{word2vec}\nmodel trained on a large corpus of general Dutch text. Finally, a third\ndictionary was created by manually filtering out incorrect expansions. We\ntrained multiple Support Vector Machines, using the distribution of words over\nthe different categories in the dictionaries as features. The best-performing\nmodel used the manually cleaned dictionary and obtained an F-score of 0.46 for\nthe racist class on a test set consisting of unseen Dutch comments, retrieved\nfrom the same sites used for the training set. The automated expansion of the\ndictionary only slightly boosted the model's performance, and this increase in\nperformance was not statistically significant. The fact that the coverage of\nthe expanded dictionaries did increase indicates that the words that were\nautomatically added did occur in the corpus, but were not able to meaningfully\nimpact performance. The dictionaries, code, and the procedure for requesting\nthe corpus are available at: https://github.com/clips/hades",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08868v1",
            "date": "2016-08-31 14:12:01+00:00",
            "title": "Demographic Dialectal Variation in Social Media: A Case Study of African-American English",
            "authors": [
                "Su Lin Blodgett",
                "Lisa Green",
                "Brendan O'Connor"
            ],
            "abstract": "Though dialectal language is increasingly abundant on social media, few\nresources exist for developing NLP tools to handle such language. We conduct a\ncase study of dialectal language in online conversational text by investigating\nAfrican-American English (AAE) on Twitter. We propose a distantly supervised\nmodel to identify AAE-like language from demographics associated with\ngeo-located messages, and we verify that this language follows well-known AAE\nlinguistic phenomena. In addition, we analyze the quality of existing language\nidentification and dependency parsing tools on AAE-like text, demonstrating\nthat they perform poorly on such text compared to text associated with white\nspeakers. We also provide an ensemble classifier for language identification\nwhich eliminates this disparity and release a new corpus of tweets containing\nAAE-like language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00068v2",
            "date": "2016-10-31 22:42:02+00:00",
            "title": "RNN Approaches to Text Normalization: A Challenge",
            "authors": [
                "Richard Sproat",
                "Navdeep Jaitly"
            ],
            "abstract": "This paper presents a challenge to the community: given a large corpus of\nwritten text aligned to its normalized spoken form, train an RNN to learn the\ncorrect normalization function. We present a data set of general text where the\nnormalizations were generated using an existing text normalization component of\na text-to-speech system. This data set will be released open-source in the near\nfuture.\n  We also present our own experiments with this data set with a variety of\ndifferent RNN architectures. While some of the architectures do in fact produce\nvery good results when measured in terms of overall accuracy, the errors that\nare produced are problematic, since they would convey completely the wrong\nmessage if such a system were deployed in a speech application. On the other\nhand, we show that a simple FST-based filter can mitigate those errors, and\nachieve a level of accuracy not achievable by the RNN alone.\n  Though our conclusions are largely negative on this point, we are actually\nnot arguing that the text normalization problem is intractable using an pure\nRNN approach, merely that it is not going to be something that can be solved\nmerely by having huge amounts of annotated text data and feeding that to a\ngeneral RNN model. And when we open-source our data, we will be providing a\nnovel data set for sequence-to-sequence modeling in the hopes that the the\ncommunity can find better solutions.\n  The data used in this work have been released and are available at:\nhttps://github.com/rwsproat/text-normalization-data",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00126v1",
            "date": "2016-11-01 04:48:09+00:00",
            "title": "Improving Twitter Sentiment Classification via Multi-Level Sentiment-Enriched Word Embeddings",
            "authors": [
                "Shufeng Xiong"
            ],
            "abstract": "Most of existing work learn sentiment-specific word representation for\nimproving Twitter sentiment classification, which encoded both n-gram and\ndistant supervised tweet sentiment information in learning process. They assume\nall words within a tweet have the same sentiment polarity as the whole tweet,\nwhich ignores the word its own sentiment polarity. To address this problem, we\npropose to learn sentiment-specific word embedding by exploiting both lexicon\nresource and distant supervised information. We develop a multi-level\nsentiment-enriched word embedding learning method, which uses parallel\nasymmetric neural network to model n-gram, word level sentiment and tweet level\nsentiment in learning process. Experiments on standard benchmarks show our\napproach outperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00179v1",
            "date": "2016-11-01 10:38:29+00:00",
            "title": "Dual Learning for Machine Translation",
            "authors": [
                "Yingce Xia",
                "Di He",
                "Tao Qin",
                "Liwei Wang",
                "Nenghai Yu",
                "Tie-Yan Liu",
                "Wei-Ying Ma"
            ],
            "abstract": "While neural machine translation (NMT) is making good progress in the past\ntwo years, tens of millions of bilingual sentence pairs are needed for its\ntraining. However, human labeling is very costly. To tackle this training data\nbottleneck, we develop a dual-learning mechanism, which can enable an NMT\nsystem to automatically learn from unlabeled data through a dual-learning game.\nThis mechanism is inspired by the following observation: any machine\ntranslation task has a dual task, e.g., English-to-French translation (primal)\nversus French-to-English translation (dual); the primal and dual tasks can form\na closed loop, and generate informative feedback signals to train the\ntranslation models, even if without the involvement of a human labeler. In the\ndual-learning mechanism, we use one agent to represent the model for the primal\ntask and the other agent to represent the model for the dual task, then ask\nthem to teach each other through a reinforcement learning process. Based on the\nfeedback signals generated during this process (e.g., the language-model\nlikelihood of the output of a model, and the reconstruction error of the\noriginal sentence after the primal and dual translations), we can iteratively\nupdate the two models until convergence (e.g., using the policy gradient\nmethods). We call the corresponding approach to neural machine translation\n\\emph{dual-NMT}. Experiments show that dual-NMT works very well on\nEnglish$\\leftrightarrow$French translation; especially, by learning from\nmonolingual data (with 10% bilingual data for warm start), it achieves a\ncomparable accuracy to NMT trained from the full bilingual data for the\nFrench-to-English translation task.",
            "journal ref": "NIPS 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00196v1",
            "date": "2016-11-01 12:14:02+00:00",
            "title": "Recurrent Neural Network Language Model Adaptation Derived Document Vector",
            "authors": [
                "Wei Li",
                "Brian Kan Wing Mak"
            ],
            "abstract": "In many natural language processing (NLP) tasks, a document is commonly\nmodeled as a bag of words using the term frequency-inverse document frequency\n(TF-IDF) vector. One major shortcoming of the frequency-based TF-IDF feature\nvector is that it ignores word orders that carry syntactic and semantic\nrelationships among the words in a document, and they can be important in some\nNLP tasks such as genre classification. This paper proposes a novel distributed\nvector representation of a document: a simple recurrent-neural-network language\nmodel (RNN-LM) or a long short-term memory RNN language model (LSTM-LM) is\nfirst created from all documents in a task; some of the LM parameters are then\nadapted by each document, and the adapted parameters are vectorized to\nrepresent the document. The new document vectors are labeled as DV-RNN and\nDV-LSTM respectively. We believe that our new document vectors can capture some\nhigh-level sequential information in the documents, which other current\ndocument representations fail to capture. The new document vectors were\nevaluated in the genre classification of documents in three corpora: the Brown\nCorpus, the BNC Baby Corpus and an artificially created Penn Treebank dataset.\nTheir classification performances are compared with the performance of TF-IDF\nvector and the state-of-the-art distributed memory model of paragraph vector\n(PV-DM). The results show that DV-LSTM significantly outperforms TF-IDF and\nPV-DM in most cases, and combinations of the proposed document vectors with\nTF-IDF or PV-DM may further improve performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00354v1",
            "date": "2016-11-01 19:56:36+00:00",
            "title": "Faster decoding for subword level Phrase-based SMT between related languages",
            "authors": [
                "Anoop Kunchukuttan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "A common and effective way to train translation systems between related\nlanguages is to consider sub-word level basic units. However, this increases\nthe length of the sentences resulting in increased decoding time. The increase\nin length is also impacted by the specific choice of data format for\nrepresenting the sentences as subwords. In a phrase-based SMT framework, we\ninvestigate different choices of decoder parameters as well as data format and\ntheir impact on decoding time and translation accuracy. We suggest best options\nfor these settings that significantly improve decoding time with little impact\non the translation accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00472v1",
            "date": "2016-11-02 05:23:53+00:00",
            "title": "Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text",
            "authors": [
                "Ameya Prabhu",
                "Aditya Joshi",
                "Manish Shrivastava",
                "Vasudeva Varma"
            ],
            "abstract": "Sentiment analysis (SA) using code-mixed data from social media has several\napplications in opinion mining ranging from customer satisfaction to social\ncampaign analysis in multilingual societies. Advances in this area are impeded\nby the lack of a suitable annotated dataset. We introduce a Hindi-English\n(Hi-En) code-mixed dataset for sentiment analysis and perform empirical\nanalysis comparing the suitability and performance of various state-of-the-art\nSA methods in social media.\n  In this paper, we introduce learning sub-word level representations in LSTM\n(Subword-LSTM) architecture instead of character-level or word-level\nrepresentations. This linguistic prior in our architecture enables us to learn\nthe information about sentiment value of important morphemes. This also seems\nto work well in highly noisy text containing misspellings as shown in our\nexperiments which is demonstrated in morpheme-level feature maps learned by our\nmodel. Also, we hypothesize that encoding this linguistic prior in the\nSubword-LSTM architecture leads to the superior performance. Our system attains\naccuracy 4-5% greater than traditional approaches on our dataset, and also\noutperforms the available system for sentiment analysis in Hi-En code-mixed\ntext by 18%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00483v2",
            "date": "2016-11-02 07:01:25+00:00",
            "title": "Detecting Context Dependent Messages in a Conversational Environment",
            "authors": [
                "Chaozhuo Li",
                "Yu Wu",
                "Wei Wu",
                "Chen Xing",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "While automatic response generation for building chatbot systems has drawn a\nlot of attention recently, there is limited understanding on when we need to\nconsider the linguistic context of an input text in the generation process. The\ntask is challenging, as messages in a conversational environment are short and\ninformal, and evidence that can indicate a message is context dependent is\nscarce. After a study of social conversation data crawled from the web, we\nobserved that some characteristics estimated from the responses of messages are\ndiscriminative for identifying context dependent messages. With the\ncharacteristics as weak supervision, we propose using a Long Short Term Memory\n(LSTM) network to learn a classifier. Our method carries out text\nrepresentation and classifier learning in a unified framework. Experimental\nresults show that the proposed method can significantly outperform baseline\nmethods on accuracy of classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00601v3",
            "date": "2016-11-02 13:38:32+00:00",
            "title": "Ordinal Common-sense Inference",
            "authors": [
                "Sheng Zhang",
                "Rachel Rudinger",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "Humans have the capacity to draw common-sense inferences from natural\nlanguage: various things that are likely but not certain to hold based on\nestablished discourse, and are rarely stated explicitly. We propose an\nevaluation of automated common-sense inference based on an extension of\nrecognizing textual entailment: predicting ordinal human responses on the\nsubjective likelihood of an inference holding in a given context. We describe a\nframework for extracting common-sense knowledge from corpora, which is then\nused to construct a dataset for this ordinal entailment task. We train a neural\nsequence-to-sequence model on this dataset, which we use to score and generate\npossible inferences. Further, we annotate subsets of previously established\ndatasets via our ordinal annotation protocol in order to then analyze the\ndistinctions between these and what we have constructed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00674v9",
            "date": "2016-11-02 16:38:18+00:00",
            "title": "Fuzzy paraphrases in learning word representations with a lexicon",
            "authors": [
                "Yuanzhi Ke",
                "Masafumi Hagiwara"
            ],
            "abstract": "A synonym of a polysemous word is usually only the paraphrase of one sense\namong many. When lexicons are used to improve vector-space word\nrepresentations, such paraphrases are unreliable and bring noise to the\nvector-space. The prior works use a coefficient to adjust the overall learning\nof the lexicons. They regard the paraphrases equally. In this paper, we propose\na novel approach that regards the paraphrases diversely to alleviate the\nadverse effects of polysemy. We annotate each paraphrase with a degree of\nreliability. The paraphrases are randomly eliminated according to the degrees\nwhen our model learns word representations. In this way, our approach drops the\nunreliable paraphrases, keeping more reliable paraphrases at the same time. The\nexperimental results show that the proposed method improves the word vectors.\nOur approach is an attempt to address the polysemy problem keeping one vector\nper word. It makes the approach easier to use than the conventional methods\nthat estimate multiple vectors for a word. Our approach also outperforms the\nprior works in the experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00801v1",
            "date": "2016-11-02 20:52:46+00:00",
            "title": "A FOFE-based Local Detection Approach for Named Entity Recognition and Mention Detection",
            "authors": [
                "Mingbin Xu",
                "Hui Jiang"
            ],
            "abstract": "In this paper, we study a novel approach for named entity recognition (NER)\nand mention detection in natural language processing. Instead of treating NER\nas a sequence labelling problem, we propose a new local detection approach,\nwhich rely on the recent fixed-size ordinally forgetting encoding (FOFE) method\nto fully encode each sentence fragment and its left/right contexts into a\nfixed-size representation. Afterwards, a simple feedforward neural network is\nused to reject or predict entity label for each individual fragment. The\nproposed method has been evaluated in several popular NER and mention detection\ntasks, including the CoNLL 2003 NER task and TAC-KBP2015 and TAC-KBP2016\nTri-lingual Entity Discovery and Linking (EDL) tasks. Our methods have yielded\npretty strong performance in all of these examined tasks. This local detection\napproach has shown many advantages over the traditional sequence labelling\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00995v1",
            "date": "2016-11-03 13:14:52+00:00",
            "title": "An empirical study for Vietnamese dependency parsing",
            "authors": [
                "Dat Quoc Nguyen",
                "Mark Dras",
                "Mark Johnson"
            ],
            "abstract": "This paper presents an empirical comparison of different dependency parsers\nfor Vietnamese, which has some unusual characteristics such as copula drop and\nverb serialization. Experimental results show that the neural network-based\nparsers perform significantly better than the traditional parsers. We report\nthe highest parsing scores published to date for Vietnamese with the labeled\nattachment score (LAS) at 73.53% and the unlabeled attachment score (UAS) at\n80.66%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01101v1",
            "date": "2016-11-03 17:41:47+00:00",
            "title": "CogALex-V Shared Task: ROOT18",
            "authors": [
                "Emmanuele Chersoni",
                "Giulia Rambelli",
                "Enrico Santus"
            ],
            "abstract": "In this paper, we describe ROOT 18, a classifier using the scores of several\nunsupervised distributional measures as features to discriminate between\nsemantically related and unrelated words, and then to classify the related\npairs according to their semantic relation (i.e. synonymy, antonymy, hypernymy,\npart-whole meronymy). Our classifier participated in the CogALex-V Shared Task,\nshowing a solid performance on the first subtask, but a poor performance on the\nsecond subtask. The low scores reported on the second subtask suggest that\ndistributional measures are not sufficient to discriminate between multiple\nsemantic relations at once.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01116v3",
            "date": "2016-11-03 18:10:35+00:00",
            "title": "Binary Paragraph Vectors",
            "authors": [
                "Karol Grzegorczyk",
                "Marcin Kurdziel"
            ],
            "abstract": "Recently Le & Mikolov described two log-linear models, called Paragraph\nVector, that can be used to learn state-of-the-art distributed representations\nof documents. Inspired by this work, we present Binary Paragraph Vector models:\nsimple neural networks that learn short binary codes for fast information\nretrieval. We show that binary paragraph vectors outperform autoencoder-based\nbinary codes, despite using fewer bits. We also evaluate their precision in\ntransfer learning settings, where binary codes are inferred for documents\nunrelated to the training corpus. Results from these experiments indicate that\nbinary paragraph vectors can capture semantics relevant for various\ndomain-specific documents. Finally, we present a model that simultaneously\nlearns short binary codes and longer, real-valued representations. This model\ncan be used to rapidly retrieve a short list of highly relevant documents from\na large document collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01242v1",
            "date": "2016-11-04 01:54:03+00:00",
            "title": "Answering Complicated Question Intents Expressed in Decomposed Question Sequences",
            "authors": [
                "Mohit Iyyer",
                "Wen-tau Yih",
                "Ming-Wei Chang"
            ],
            "abstract": "Recent work in semantic parsing for question answering has focused on long\nand complicated questions, many of which would seem unnatural if asked in a\nnormal conversation between two humans. In an effort to explore a\nconversational QA setting, we present a more realistic task: answering\nsequences of simple but inter-related questions. We collect a dataset of 6,066\nquestion sequences that inquire about semi-structured tables from Wikipedia,\nwith 17,553 question-answer pairs in total. Existing QA systems face two major\nproblems when evaluated on our dataset: (1) handling questions that contain\ncoreferences to previous questions or answers, and (2) matching words or\nphrases in a question to corresponding entries in the associated table. We\nconclude by proposing strategies to handle both of these issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01368v1",
            "date": "2016-11-04 13:36:32+00:00",
            "title": "Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies",
            "authors": [
                "Tal Linzen",
                "Emmanuel Dupoux",
                "Yoav Goldberg"
            ],
            "abstract": "The success of long short-term memory (LSTM) neural networks in language\nprocessing is typically attributed to their ability to capture long-distance\nstatistical regularities. Linguistic regularities are often sensitive to\nsyntactic structure; can such dependencies be captured by LSTMs, which do not\nhave explicit structural representations? We begin addressing this question\nusing number agreement in English subject-verb dependencies. We probe the\narchitecture's grammatical competence both using training objectives with an\nexplicit grammatical target (number prediction, grammaticality judgments) and\nusing language models. In the strongly supervised settings, the LSTM achieved\nvery high overall accuracy (less than 1% errors), but errors increased when\nsequential and structural information conflicted. The frequency of such errors\nrose sharply in the language-modeling setting. We conclude that LSTMs can\ncapture a non-trivial amount of grammatical structure given targeted\nsupervision, but stronger architectures may be required to further reduce\nerrors; furthermore, the language modeling signal is insufficient for capturing\nsyntax-sensitive dependencies, and should be supplemented with more direct\nsupervision if such dependencies need to be captured.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01436v2",
            "date": "2016-11-04 16:12:46+00:00",
            "title": "Learning Recurrent Span Representations for Extractive Question Answering",
            "authors": [
                "Kenton Lee",
                "Shimi Salant",
                "Tom Kwiatkowski",
                "Ankur Parikh",
                "Dipanjan Das",
                "Jonathan Berant"
            ],
            "abstract": "The reading comprehension task, that asks questions about a given evidence\ndocument, is a central problem in natural language understanding. Recent\nformulations of this task have typically focused on answer selection from a set\nof candidates pre-defined manually or through the use of an external NLP\npipeline. However, Rajpurkar et al. (2016) recently released the SQuAD dataset\nin which the answers can be arbitrary strings from the supplied text. In this\npaper, we focus on this answer extraction task, presenting a novel model\narchitecture that efficiently builds fixed length representations of all spans\nin the evidence document with a recurrent network. We show that scoring\nexplicit span representations significantly improves performance over other\napproaches that factor the prediction into separate predictions about words or\nstart and end markers. Our approach improves upon the best published results of\nWang & Jiang (2016) by 5% and decreases the error of Rajpurkar et al.'s\nbaseline by > 50%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01487v3",
            "date": "2016-11-04 18:42:47+00:00",
            "title": "Morphological Inflection Generation with Hard Monotonic Attention",
            "authors": [
                "Roee Aharoni",
                "Yoav Goldberg"
            ],
            "abstract": "We present a neural model for morphological inflection generation which\nemploys a hard attention mechanism, inspired by the nearly-monotonic alignment\ncommonly found between the characters in a word and the characters in its\ninflection. We evaluate the model on three previously studied morphological\ninflection generation datasets and show that it provides state of the art\nresults in various setups compared to previous neural and non-neural\napproaches. Finally we present an analysis of the continuous representations\nlearned by both the hard and soft attention \\cite{bahdanauCB14} models for the\ntask, shedding some light on the features such models extract.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01603v6",
            "date": "2016-11-05 04:49:00+00:00",
            "title": "Bidirectional Attention Flow for Machine Comprehension",
            "authors": [
                "Minjoon Seo",
                "Aniruddha Kembhavi",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Machine comprehension (MC), answering a query about a given context\nparagraph, requires modeling complex interactions between the context and the\nquery. Recently, attention mechanisms have been successfully extended to MC.\nTypically these methods use attention to focus on a small portion of the\ncontext and summarize it with a fixed-size vector, couple attentions\ntemporally, and/or often form a uni-directional attention. In this paper we\nintroduce the Bi-Directional Attention Flow (BIDAF) network, a multi-stage\nhierarchical process that represents the context at different levels of\ngranularity and uses bi-directional attention flow mechanism to obtain a\nquery-aware context representation without early summarization. Our\nexperimental evaluations show that our model achieves the state-of-the-art\nresults in Stanford Question Answering Dataset (SQuAD) and CNN/DailyMail cloze\ntest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01628v5",
            "date": "2016-11-05 10:55:37+00:00",
            "title": "Reference-Aware Language Models",
            "authors": [
                "Zichao Yang",
                "Phil Blunsom",
                "Chris Dyer",
                "Wang Ling"
            ],
            "abstract": "We propose a general class of language models that treat reference as an\nexplicit stochastic latent variable. This architecture allows models to create\nmentions of entities and their attributes by accessing external databases\n(required by, e.g., dialogue generation and recipe generation) and internal\nstate (required by, e.g. language models which are aware of coreference). This\nfacilitates the incorporation of information that can be accessed in\npredictable locations in databases or discourse context, even when the targets\nof the reference may be rare words. Experiments on three tasks shows our model\nvariants based on deterministic attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01839v2",
            "date": "2016-11-06 20:24:40+00:00",
            "title": "Hierarchical Question Answering for Long Documents",
            "authors": [
                "Eunsol Choi",
                "Daniel Hewlett",
                "Alexandre Lacoste",
                "Illia Polosukhin",
                "Jakob Uszkoreit",
                "Jonathan Berant"
            ],
            "abstract": "We present a framework for question answering that can efficiently scale to\nlonger documents while maintaining or even improving performance of\nstate-of-the-art models. While most successful approaches for reading\ncomprehension rely on recurrent neural networks (RNNs), running them over long\ndocuments is prohibitively slow because it is difficult to parallelize over\nsequences. Inspired by how people first skim the document, identify relevant\nparts, and carefully read these parts to produce an answer, we combine a\ncoarse, fast model for selecting relevant sentences and a more expensive RNN\nfor producing the answer from those sentences. We treat sentence selection as a\nlatent variable trained jointly from the answer only using reinforcement\nlearning. Experiments demonstrate the state of the art performance on a\nchallenging subset of the Wikireading and on a new dataset, while speeding up\nthe model by 3.5x-6.7x.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01867v1",
            "date": "2016-11-07 00:56:19+00:00",
            "title": "Latent Attention For If-Then Program Synthesis",
            "authors": [
                "Xinyun Chen",
                "Chang Liu",
                "Richard Shin",
                "Dawn Song",
                "Mingcheng Chen"
            ],
            "abstract": "Automatic translation from natural language descriptions into programs is a\nlongstanding challenging problem. In this work, we consider a simple yet\nimportant sub-problem: translation from textual descriptions to If-Then\nprograms. We devise a novel neural network architecture for this task which we\ntrain end-to-end. Specifically, we introduce Latent Attention, which computes\nmultiplicative weights for the words in the description in a two-stage process\nwith the goal of better leveraging the natural language structures that\nindicate the relevant parts for predicting program elements. Our architecture\nreduces the error rate by 28.57% compared to prior art. We also propose a\none-shot learning scenario of If-Then program synthesis and simulate it with\nour existing dataset. We demonstrate a variation on the training procedure for\nthis scenario that outperforms the original procedure, significantly closing\nthe gap to the model trained with all data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01874v2",
            "date": "2016-11-07 02:03:55+00:00",
            "title": "Neural Machine Translation with Reconstruction",
            "authors": [
                "Zhaopeng Tu",
                "Yang Liu",
                "Lifeng Shang",
                "Xiaohua Liu",
                "Hang Li"
            ],
            "abstract": "Although end-to-end Neural Machine Translation (NMT) has achieved remarkable\nprogress in the past two years, it suffers from a major drawback: translations\ngenerated by NMT systems often lack of adequacy. It has been widely observed\nthat NMT tends to repeatedly translate some source words while mistakenly\nignoring other words. To alleviate this problem, we propose a novel\nencoder-decoder-reconstructor framework for NMT. The reconstructor,\nincorporated into the NMT model, manages to reconstruct the input source\nsentence from the hidden layer of the output target sentence, to ensure that\nthe information in the source side is transformed to the target side as much as\npossible. Experiments show that the proposed framework significantly improves\nthe adequacy of NMT output and achieves superior translation result over\nstate-of-the-art NMT and statistical MT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01884v3",
            "date": "2016-11-07 03:39:52+00:00",
            "title": "AC-BLSTM: Asymmetric Convolutional Bidirectional LSTM Networks for Text Classification",
            "authors": [
                "Depeng Liang",
                "Yongdong Zhang"
            ],
            "abstract": "Recently deeplearning models have been shown to be capable of making\nremarkable performance in sentences and documents classification tasks. In this\nwork, we propose a novel framework called AC-BLSTM for modeling sentences and\ndocuments, which combines the asymmetric convolution neural network (ACNN) with\nthe Bidirectional Long Short-Term Memory network (BLSTM). Experiment results\ndemonstrate that our model achieves state-of-the-art results on five tasks,\nincluding sentiment analysis, question type classification, and subjectivity\nclassification. In order to further improve the performance of AC-BLSTM, we\npropose a semi-supervised learning framework called G-AC-BLSTM for text\nclassification by combining the generative model with AC-BLSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02007v1",
            "date": "2016-11-07 12:08:13+00:00",
            "title": "Keyphrase Annotation with Graph Co-Ranking",
            "authors": [
                "Adrien Bougouin",
                "Florian Boudin",
                "B\u00e9atrice Daille"
            ],
            "abstract": "Keyphrase annotation is the task of identifying textual units that represent\nthe main content of a document. Keyphrase annotation is either carried out by\nextracting the most important phrases from a document, keyphrase extraction, or\nby assigning entries from a controlled domain-specific vocabulary, keyphrase\nassignment. Assignment methods are generally more reliable. They provide\nbetter-formed keyphrases, as well as keyphrases that do not occur in the\ndocument. But they are often silent on the contrary of extraction methods that\ndo not depend on manually built resources. This paper proposes a new method to\nperform both keyphrase extraction and keyphrase assignment in an integrated and\nmutual reinforcing manner. Experiments have been carried out on datasets\ncovering different domains of humanities and social sciences. They show\nstatistically significant improvements compared to both keyphrase extraction\nand keyphrase assignment state-of-the art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02025v1",
            "date": "2016-11-07 12:47:25+00:00",
            "title": "Presenting a New Dataset for the Timeline Generation Problem",
            "authors": [
                "Xavier Holt",
                "Will Radford",
                "Ben Hachey"
            ],
            "abstract": "The timeline generation task summarises an entity's biography by selecting\nstories representing key events from a large pool of relevant documents. This\npaper addresses the lack of a standard dataset and evaluative methodology for\nthe problem. We present and make publicly available a new dataset of 18,793\nnews articles covering 39 entities. For each entity, we provide a gold standard\ntimeline and a set of entity-related articles. We propose ROUGE as an\nevaluation metric and validate our dataset by showing that top Google results\noutperform straw-man baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02027v1",
            "date": "2016-11-07 12:51:22+00:00",
            "title": ":telephone::person::sailboat::whale::okhand:; or \"Call me Ishmael\" - How do you translate emoji?",
            "authors": [
                "Will Radford",
                "Andrew Chisholm",
                "Ben Hachey",
                "Bo Han"
            ],
            "abstract": "We report on an exploratory analysis of Emoji Dick, a project that leverages\ncrowdsourcing to translate Melville's Moby Dick into emoji. This distinctive\nuse of emoji removes textual context, and leads to a varying translation\nquality. In this paper, we use statistical word alignment and part-of-speech\ntagging to explore how people use emoji. Despite these simple methods, we\nobserved differences in token and part-of-speech distributions. Experiments\nalso suggest that semantics are preserved in the translation, and repetition is\nmore common in emoji.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02091v2",
            "date": "2016-11-07 15:05:06+00:00",
            "title": "Building a comprehensive syntactic and semantic corpus of Chinese clinical texts",
            "authors": [
                "Bin He",
                "Bin Dong",
                "Yi Guan",
                "Jinfeng Yang",
                "Zhipeng Jiang",
                "Qiubin Yu",
                "Jianyi Cheng",
                "Chunyan Qu"
            ],
            "abstract": "Objective: To build a comprehensive corpus covering syntactic and semantic\nannotations of Chinese clinical texts with corresponding annotation guidelines\nand methods as well as to develop tools trained on the annotated corpus, which\nsupplies baselines for research on Chinese texts in the clinical domain.\n  Materials and methods: An iterative annotation method was proposed to train\nannotators and to develop annotation guidelines. Then, by using annotation\nquality assurance measures, a comprehensive corpus was built, containing\nannotations of part-of-speech (POS) tags, syntactic tags, entities, assertions,\nand relations. Inter-annotator agreement (IAA) was calculated to evaluate the\nannotation quality and a Chinese clinical text processing and information\nextraction system (CCTPIES) was developed based on our annotated corpus.\n  Results: The syntactic corpus consists of 138 Chinese clinical documents with\n47,424 tokens and 2553 full parsing trees, while the semantic corpus includes\n992 documents that annotated 39,511 entities with their assertions and 7695\nrelations. IAA evaluation shows that this comprehensive corpus is of good\nquality, and the system modules are effective.\n  Discussion: The annotated corpus makes a considerable contribution to natural\nlanguage processing (NLP) research into Chinese texts in the clinical domain.\nHowever, this corpus has a number of limitations. Some additional types of\nclinical text should be introduced to improve corpus coverage and active\nlearning methods should be utilized to promote annotation efficiency.\n  Conclusions: In this study, several annotation guidelines and an annotation\nmethod for Chinese clinical texts were proposed, and a comprehensive corpus\nwith its NLP modules were constructed, providing a foundation for further study\nof applying NLP techniques to Chinese texts in the clinical domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02344v3",
            "date": "2016-11-07 23:46:45+00:00",
            "title": "A Convolutional Encoder Model for Neural Machine Translation",
            "authors": [
                "Jonas Gehring",
                "Michael Auli",
                "David Grangier",
                "Yann N. Dauphin"
            ],
            "abstract": "The prevalent approach to neural machine translation relies on bi-directional\nLSTMs to encode the source sentence. In this paper we present a faster and\nsimpler architecture based on a succession of convolutional layers. This allows\nto encode the entire source sentence simultaneously compared to recurrent\nnetworks for which computation is constrained by temporal dependencies. On\nWMT'16 English-Romanian translation we achieve competitive accuracy to the\nstate-of-the-art and we outperform several recently published results on the\nWMT'15 English-German task. Our models obtain almost the same accuracy as a\nvery deep LSTM setup on WMT'14 English-French translation. Our convolutional\nencoder speeds up CPU decoding by more than two times at the same or higher\naccuracy as a strong bi-directional LSTM baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02360v2",
            "date": "2016-11-08 01:47:41+00:00",
            "title": "Cruciform: Solving Crosswords with Natural Language Processing",
            "authors": [
                "Dragomir Radev",
                "Rui Zhang",
                "Steve Wilson",
                "Derek Van Assche",
                "Henrique Spyra Gubert",
                "Alisa Krivokapic",
                "MeiXing Dong",
                "Chongruo Wu",
                "Spruce Bondera",
                "Luke Brandl",
                "Jeremy Dohmann"
            ],
            "abstract": "Crossword puzzles are popular word games that require not only a large\nvocabulary, but also a broad knowledge of topics. Answering each clue is a\nnatural language task on its own as many clues contain nuances, puns, or\ncounter-intuitive word definitions. Additionally, it can be extremely difficult\nto ascertain definitive answers without the constraints of the crossword grid\nitself. This task is challenging for both humans and computers. We describe\nhere a new crossword solving system, Cruciform. We employ a group of natural\nlanguage components, each of which returns a list of candidate words with\nscores when given a clue. These lists are used in conjunction with the fill\nintersections in the puzzle grid to formulate a constraint satisfaction\nproblem, in a manner similar to the one used in the Dr. Fill system. We\ndescribe the results of several of our experiments with the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02361v1",
            "date": "2016-11-08 01:48:15+00:00",
            "title": "Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents",
            "authors": [
                "Rui Zhang",
                "Honglak Lee",
                "Dragomir Radev"
            ],
            "abstract": "The goal of sentence and document modeling is to accurately represent the\nmeaning of sentences and documents for various Natural Language Processing\ntasks. In this work, we present Dependency Sensitive Convolutional Neural\nNetworks (DSCNN) as a general-purpose classification system for both sentences\nand documents. DSCNN hierarchically builds textual representations by\nprocessing pretrained word embeddings via Long Short-Term Memory networks and\nsubsequently extracting features with convolution operators. Compared with\nexisting recursive neural models with tree structures, DSCNN does not rely on\nparsers and expensive phrase labeling, and thus is not restricted to\nsentence-level tasks. Moreover, unlike other CNN-based models that analyze\nsentences locally by sliding windows, our system captures both the dependency\ninformation within each sentence and relationships across sentences in the same\ndocument. Experiment results demonstrate that our approach is achieving\nstate-of-the-art performance on several tasks, including sentiment analysis,\nquestion type classification, and subjectivity classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02378v2",
            "date": "2016-11-08 03:33:46+00:00",
            "title": "A Surrogate-based Generic Classifier for Chinese TV Series Reviews",
            "authors": [
                "Yufeng Ma",
                "Long Xia",
                "Wenqi Shen",
                "Mi Zhou",
                "Weiguo Fan"
            ],
            "abstract": "With the emerging of various online video platforms like Youtube, Youku and\nLeTV, online TV series' reviews become more and more important both for viewers\nand producers. Customers rely heavily on these reviews before selecting TV\nseries, while producers use them to improve the quality. As a result,\nautomatically classifying reviews according to different requirements evolves\nas a popular research topic and is essential in our daily life. In this paper,\nwe focused on reviews of hot TV series in China and successfully trained\ngeneric classifiers based on eight predefined categories. The experimental\nresults showed promising performance and effectiveness of its generalization to\ndifferent TV series.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02550v1",
            "date": "2016-11-08 15:13:19+00:00",
            "title": "Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches",
            "authors": [
                "Shane Settle",
                "Karen Livescu"
            ],
            "abstract": "Acoustic word embeddings --- fixed-dimensional vector representations of\nvariable-length spoken word segments --- have begun to be considered for tasks\nsuch as speech recognition and query-by-example search. Such embeddings can be\nlearned discriminatively so that they are similar for speech segments\ncorresponding to the same word, while being dissimilar for segments\ncorresponding to different words. Recent work has found that acoustic word\nembeddings can outperform dynamic time warping on query-by-example search and\nrelated word discrimination tasks. However, the space of embedding models and\ntraining approaches is still relatively unexplored. In this paper we present\nnew discriminative embedding models based on recurrent neural networks (RNNs).\nWe consider training losses that have been successful in prior work, in\nparticular a cross entropy loss for word classification and a contrastive loss\nthat explicitly aims to separate same-word and different-word pairs in a\n\"Siamese network\" training setting. We find that both classifier-based and\nSiamese RNN embeddings improve over previously reported results on a word\ndiscrimination task, with Siamese RNNs outperforming classification models. In\naddition, we present analyses of the learned embeddings and the effects of\nvariables such as dimensionality and network structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02588v2",
            "date": "2016-11-08 16:19:17+00:00",
            "title": "Contradiction Detection for Rumorous Claims",
            "authors": [
                "Piroska Lendvai",
                "Uwe D. Reichel"
            ],
            "abstract": "The utilization of social media material in journalistic workflows is\nincreasing, demanding automated methods for the identification of mis- and\ndisinformation. Since textual contradiction across social media posts can be a\nsignal of rumorousness, we seek to model how claims in Twitter posts are being\ntextually contradicted. We identify two different contexts in which\ncontradiction emerges: its broader form can be observed across independently\nposted tweets and its more specific form in threaded conversations. We define\nhow the two scenarios differ in terms of central elements of argumentation:\nclaims and conversation structure. We design and evaluate models for the two\nscenarios uniformly as 3-way Recognizing Textual Entailment tasks in order to\nrepresent claims and conversation structure implicitly in a generic inference\nmodel, while previous studies used explicit or no representation of these\nproperties. To address noisy text, our classifiers use simple similarity\nfeatures derived from the string and part-of-speech level. Corpus statistics\nreveal distribution differences for these features in contradictory as opposed\nto non-contradictory tweet relations, and the classifiers yield state of the\nart performance.",
            "journal ref": "Proc. Extra-Propositional Aspects of Meaning (ExProM) in\n  Computational Linguistics, Osaka, Japan, 2016, pp 31-40",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02590v2",
            "date": "2016-11-08 16:21:16+00:00",
            "title": "Veracity Computing from Lexical Cues and Perceived Certainty Trends",
            "authors": [
                "Uwe D. Reichel",
                "Piroska Lendvai"
            ],
            "abstract": "We present a data-driven method for determining the veracity of a set of\nrumorous claims on social media data. Tweets from different sources pertaining\nto a rumor are processed on three levels: first, factuality values are assigned\nto each tweet based on four textual cue categories relevant for our journalism\nuse case; these amalgamate speaker support in terms of polarity and commitment\nin terms of certainty and speculation. Next, the proportions of these lexical\ncues are utilized as predictors for tweet certainty in a generalized linear\nregression model. Subsequently, lexical cue proportions, predicted certainty,\nas well as their time course characteristics are used to compute veracity for\neach rumor in terms of the identity of the rumor-resolving tweet and its binary\nresolution value judgment. The system operates without access to\nextralinguistic resources. Evaluated on the data portion for which hand-labeled\nexamples were available, it achieves .74 F1-score on identifying rumor\nresolving tweets and .76 F1-score on predicting if a rumor is resolved as true\nor false.",
            "journal ref": "Proc. 2nd Workshop on Noisy User-generated Text, Osaka, Japan,\n  2016, pp 33--42",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02839v1",
            "date": "2016-11-09 07:37:28+00:00",
            "title": "Old Content and Modern Tools - Searching Named Entities in a Finnish OCRed Historical Newspaper Collection 1771-1910",
            "authors": [
                "Kimmo Kettunen",
                "Eetu M\u00e4kel\u00e4",
                "Teemu Ruokolainen",
                "Juha Kuokkala",
                "Laura L\u00f6fberg"
            ],
            "abstract": "Named Entity Recognition (NER), search, classification and tagging of names\nand name like frequent informational elements in texts, has become a standard\ninformation extraction procedure for textual data. NER has been applied to many\ntypes of texts and different types of entities: newspapers, fiction, historical\nrecords, persons, locations, chemical compounds, protein families, animals etc.\nIn general a NER system's performance is genre and domain dependent and also\nused entity categories vary (Nadeau and Sekine, 2007). The most general set of\nnamed entities is usually some version of three partite categorization of\nlocations, persons and organizations. In this paper we report first large scale\ntrials and evaluation of NER with data out of a digitized Finnish historical\nnewspaper collection Digi. Experiments, results and discussion of this research\nserve development of the Web collection of historical Finnish newspapers.\n  Digi collection contains 1,960,921 pages of newspaper material from years\n1771-1910 both in Finnish and Swedish. We use only material of Finnish\ndocuments in our evaluation. The OCRed newspaper collection has lots of OCR\nerrors; its estimated word level correctness is about 70-75 % (Kettunen and\nP\\\"a\\\"akk\\\"onen, 2016). Our principal NER tagger is a rule-based tagger of\nFinnish, FiNER, provided by the FIN-CLARIN consortium. We show also results of\nlimited category semantic tagging with tools of the Semantic Computing Research\nGroup (SeCo) of the Aalto University. Three other tools are also evaluated\nbriefly.\n  This research reports first published large scale results of NER in a\nhistorical Finnish OCRed newspaper collection. Results of the research\nsupplement NER results of other languages with similar noisy data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02956v3",
            "date": "2016-11-09 14:50:01+00:00",
            "title": "A Comparison of Word Embeddings for English and Cross-Lingual Chinese Word Sense Disambiguation",
            "authors": [
                "Hong Jin Kang",
                "Tao Chen",
                "Muthu Kumar Chandrasekaran",
                "Min-Yen Kan"
            ],
            "abstract": "Word embeddings are now ubiquitous forms of word representation in natural\nlanguage processing. There have been applications of word embeddings for\nmonolingual word sense disambiguation (WSD) in English, but few comparisons\nhave been done. This paper attempts to bridge that gap by examining popular\nembeddings for the task of monolingual English WSD. Our simplified method leads\nto comparable state-of-the-art performance without expensive retraining.\nCross-Lingual WSD - where the word senses of a word in a source language e come\nfrom a separate target translation language f - can also assist in language\nlearning; for example, when providing translations of target vocabulary for\nlearners. Thus we have also applied word embeddings to the novel task of\ncross-lingual WSD for Chinese and provide a public dataset for further\nbenchmarking. We have also experimented with using word embeddings for LSTM\nnetworks and found surprisingly that a basic LSTM network does not work well.\nWe discuss the ramifications of this outcome.",
            "journal ref": "Proceedings of the 3rd Workshop on Natural Language Processing\n  Techniques for Educational Applications, pages 30 to 39, Osaka, Japan,\n  December 12 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02988v1",
            "date": "2016-11-09 15:49:31+00:00",
            "title": "Distant supervision for emotion detection using Facebook reactions",
            "authors": [
                "Chris Pool",
                "Malvina Nissim"
            ],
            "abstract": "We exploit the Facebook reaction feature in a distant supervised fashion to\ntrain a support vector machine classifier for emotion detection, using several\nfeature combinations and combining different Facebook pages. We test our models\non existing benchmarks for emotion detection and show that employing only\ninformation that is derived completely automatically, thus without relying on\nany handcrafted lexicon as it's usually done, we can achieve competitive\nresults. The results also show that there is large room for improvement,\nespecially by gearing the collection of Facebook pages, with a view to the\ntarget domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03057v1",
            "date": "2016-11-09 19:39:15+00:00",
            "title": "When silver glitters more than gold: Bootstrapping an Italian part-of-speech tagger for Twitter",
            "authors": [
                "Barbara Plank",
                "Malvina Nissim"
            ],
            "abstract": "We bootstrap a state-of-the-art part-of-speech tagger to tag Italian Twitter\ndata, in the context of the Evalita 2016 PoSTWITA shared task. We show that\ntraining the tagger on native Twitter data enriched with little amounts of\nspecifically selected gold data and additional silver-labelled data scraped\nfrom Facebook, yields better results than using large amounts of manually\nannotated data from a mix of genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03279v1",
            "date": "2016-11-10 12:22:43+00:00",
            "title": "Tracing metaphors in time through self-distance in vector spaces",
            "authors": [
                "Marco Del Tredici",
                "Malvina Nissim",
                "Andrea Zaninello"
            ],
            "abstract": "From a diachronic corpus of Italian, we build consecutive vector spaces in\ntime and use them to compare a term's cosine similarity to itself in different\ntime spans. We assume that a drop in similarity might be related to the\nemergence of a metaphorical sense at a given time. Similarity-based\nobservations are matched to the actual year when a figurative meaning was\ndocumented in a reference dictionary and through manual inspection of corpus\noccurrences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03382v1",
            "date": "2016-11-10 16:23:04+00:00",
            "title": "Efficient Summarization with Read-Again and Copy Mechanism",
            "authors": [
                "Wenyuan Zeng",
                "Wenjie Luo",
                "Sanja Fidler",
                "Raquel Urtasun"
            ],
            "abstract": "Encoder-decoder models have been widely used to solve sequence to sequence\nprediction tasks. However current approaches suffer from two shortcomings.\nFirst, the encoders compute a representation of each word taking into account\nonly the history of the words it has read so far, yielding suboptimal\nrepresentations. Second, current decoders utilize large vocabularies in order\nto minimize the problem of unknown words, resulting in slow decoding times. In\nthis paper we address both shortcomings. Towards this goal, we first introduce\na simple mechanism that first reads the input sequence before committing to a\nrepresentation of each word. Furthermore, we propose a simple copy mechanism\nthat is able to exploit very small vocabularies and handle out-of-vocabulary\nwords. We demonstrate the effectiveness of our approach on the Gigaword dataset\nand DUC competition outperforming the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03466v3",
            "date": "2016-11-10 20:13:36+00:00",
            "title": "Syntactic Enhancement to VSIMM for Roadmap Based Anomalous Trajectory Detection: A Natural Language Processing Approach",
            "authors": [
                "Vikram Krishnamurthy",
                "Sijia Gao"
            ],
            "abstract": "The aim of syntactic tracking is to classify spatio-temporal patterns of a\ntarget's motion using natural language processing models. In this paper, we\ngeneralize earlier work by considering a constrained stochastic context free\ngrammar (CSCFG) for modeling patterns confined to a roadmap. The constrained\ngrammar facilitates modeling specific directions and road names in a roadmap.\nWe present a novel particle filtering algorithm that exploits the CSCFG model\nfor estimating the target's patterns. This meta-level algorithm operates in\nconjunction with a base-level tracking algorithm. Extensive numerical results\nusing simulated ground moving target indicator (GMTI) radar measurements show\nsubstantial improvement in target tracking accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03641v2",
            "date": "2016-11-11 10:06:29+00:00",
            "title": "Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure",
            "authors": [
                "Oded Avraham",
                "Yoav Goldberg"
            ],
            "abstract": "We suggest a new method for creating and using gold-standard datasets for\nword similarity evaluation. Our goal is to improve the reliability of the\nevaluation, and we do this by redesigning the annotation task to achieve higher\ninter-rater agreement, and by defining a performance measure which takes the\nreliability of each annotation decision in the dataset into account.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03932v1",
            "date": "2016-11-12 01:49:48+00:00",
            "title": "Training IBM Watson using Automatically Generated Question-Answer Pairs",
            "authors": [
                "Jangho Lee",
                "Gyuwan Kim",
                "Jaeyoon Yoo",
                "Changwoo Jung",
                "Minseok Kim",
                "Sungroh Yoon"
            ],
            "abstract": "IBM Watson is a cognitive computing system capable of question answering in\nnatural languages. It is believed that IBM Watson can understand large corpora\nand answer relevant questions more effectively than any other\nquestion-answering system currently available. To unleash the full power of\nWatson, however, we need to train its instance with a large number of\nwell-prepared question-answer pairs. Obviously, manually generating such pairs\nin a large quantity is prohibitively time consuming and significantly limits\nthe efficiency of Watson's training. Recently, a large-scale dataset of over 30\nmillion question-answer pairs was reported. Under the assumption that using\nsuch an automatically generated dataset could relieve the burden of manual\nquestion-answer generation, we tried to use this dataset to train an instance\nof Watson and checked the training efficiency and accuracy. According to our\nexperiments, using this auto-generated dataset was effective for training\nWatson, complementing manually crafted question-answer pairs. To the best of\nthe authors' knowledge, this work is the first attempt to use a large-scale\ndataset of automatically generated question-answer pairs for training IBM\nWatson. We anticipate that the insights and lessons obtained from our\nexperiments will be useful for researchers who want to expedite Watson training\nleveraged by automatically generated question-answer pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03949v2",
            "date": "2016-11-12 03:55:10+00:00",
            "title": "Linguistically Regularized LSTMs for Sentiment Classification",
            "authors": [
                "Qiao Qian",
                "Minlie Huang",
                "Jinhao Lei",
                "Xiaoyan Zhu"
            ],
            "abstract": "Sentiment understanding has been a long-term goal of AI in the past decades.\nThis paper deals with sentence-level sentiment classification. Though a variety\nof neural network models have been proposed very recently, however, previous\nmodels either depend on expensive phrase-level annotation, whose performance\ndrops substantially when trained with only sentence-level annotation; or do not\nfully employ linguistic resources (e.g., sentiment lexicons, negation words,\nintensity words), thus not being able to produce linguistically coherent\nrepresentations. In this paper, we propose simple models trained with\nsentence-level annotation, but also attempt to generating linguistically\ncoherent representations by employing regularizers that model the linguistic\nrole of sentiment lexicons, negation words, and intensity words. Results show\nthat our models are effective to capture the sentiment shifting effect of\nsentiment, negation, and intensity words, while still obtain competitive\nresults without sacrificing the models' simplicity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04010v2",
            "date": "2016-11-12 15:59:22+00:00",
            "title": "Multi-Language Identification Using Convolutional Recurrent Neural Network",
            "authors": [
                "Vrishabh Ajay Lakhani",
                "Rohan Mahadev"
            ],
            "abstract": "Language Identification, being an important aspect of Automatic Speaker\nRecognition has had many changes and new approaches to ameliorate performance\nover the last decade. We compare the performance of using audio spectrum in the\nlog scale and using Polyphonic sound sequences from raw audio samples to train\nthe neural network and to classify speech as either English or Spanish. To\nachieve this, we use the novel approach of using a Convolutional Recurrent\nNeural Network using Long Short Term Memory (LSTM) or a Gated Recurrent Unit\n(GRU) for forward propagation of the neural network. Our hypothesis is that the\nperformance of using polyphonic sound sequence as features and both LSTM and\nGRU as the gating mechanisms for the neural network outperform the traditional\nMFCC features using a unidirectional Deep Neural Network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04052v1",
            "date": "2016-11-12 23:24:00+00:00",
            "title": "Semi-automatic Simultaneous Interpreting Quality Evaluation",
            "authors": [
                "Xiaojun Zhang"
            ],
            "abstract": "Increasing interpreting needs a more objective and automatic measurement. We\nhold a basic idea that 'translating means translating meaning' in that we can\nassessment interpretation quality by comparing the meaning of the interpreting\noutput with the source input. That is, a translation unit of a 'chunk' named\nFrame which comes from frame semantics and its components named Frame Elements\n(FEs) which comes from Frame Net are proposed to explore their matching rate\nbetween target and source texts. A case study in this paper verifies the\nusability of semi-automatic graded semantic-scoring measurement for human\nsimultaneous interpreting and shows how to use frame and FE matches to score.\nExperiments results show that the semantic-scoring metrics have a significantly\ncorrelation coefficient with human judgment.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  5, No.5, October 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04122v1",
            "date": "2016-11-13 12:20:33+00:00",
            "title": "Cross-lingual Dataless Classification for Languages with Small Wikipedia Presence",
            "authors": [
                "Yangqiu Song",
                "Stephen Mayhew",
                "Dan Roth"
            ],
            "abstract": "This paper presents an approach to classify documents in any language into an\nEnglish topical label space, without any text categorization training data. The\napproach, Cross-Lingual Dataless Document Classification (CLDDC) relies on\nmapping the English labels or short category description into a Wikipedia-based\nsemantic representation, and on the use of the target language Wikipedia.\nConsequently, performance could suffer when Wikipedia in the target language is\nsmall. In this paper, we focus on languages with small Wikipedias,\n(Small-Wikipedia languages, SWLs). We use a word-level dictionary to convert\ndocuments in a SWL to a large-Wikipedia language (LWLs), and then perform CLDDC\nbased on the LWL's Wikipedia. This approach can be applied to thousands of\nlanguages, which can be contrasted with machine translation, which is a\nsupervision heavy approach and can be done for about 100 languages. We also\ndevelop a ranking algorithm that makes use of language similarity metrics to\nautomatically select a good LWL, and show that this significantly improves\nclassification of SWLs' documents, performing comparably to the best bridge\npossible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04125v1",
            "date": "2016-11-13 12:32:20+00:00",
            "title": "Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion",
            "authors": [
                "Xu Han",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Joint representation learning of text and knowledge within a unified semantic\nspace enables us to perform knowledge graph completion more accurately. In this\nwork, we propose a novel framework to embed words, entities and relations into\nthe same continuous vector space. In this model, both entity and relation\nembeddings are learned by taking knowledge graph and plain text into\nconsideration. In experiments, we evaluate the joint learning model on three\ntasks including entity prediction, relation prediction and relation\nclassification from text. The experiment results show that our model can\nsignificantly and consistently improve the performance on the three tasks as\ncompared with other baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04230v1",
            "date": "2016-11-14 02:44:14+00:00",
            "title": "SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents",
            "authors": [
                "Ramesh Nallapati",
                "Feifei Zhai",
                "Bowen Zhou"
            ],
            "abstract": "We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model\nfor extractive summarization of documents and show that it achieves performance\nbetter than or comparable to state-of-the-art. Our model has the additional\nadvantage of being very interpretable, since it allows visualization of its\npredictions broken up by abstract features such as information content,\nsalience and novelty. Another novel contribution of our work is abstractive\ntraining of our extractive model that can train on human generated reference\nsummaries alone, eliminating the need for sentence-level extractive labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04233v1",
            "date": "2016-11-14 02:48:46+00:00",
            "title": "A New Recurrent Neural CRF for Learning Non-linear Edge Features",
            "authors": [
                "Shuming Ma",
                "Xu Sun"
            ],
            "abstract": "Conditional Random Field (CRF) and recurrent neural models have achieved\nsuccess in structured prediction. More recently, there is a marriage of CRF and\nrecurrent neural models, so that we can gain from both non-linear dense\nfeatures and globally normalized CRF objective. These recurrent neural CRF\nmodels mainly focus on encode node features in CRF undirected graphs. However,\nedge features prove important to CRF in structured prediction. In this work, we\nintroduce a new recurrent neural CRF model, which learns non-linear edge\nfeatures, and thus makes non-linear features encoded completely. We compare our\nmodel with different neural models in well-known structured prediction tasks.\nExperiments show that our model outperforms state-of-the-art methods in NP\nchunking, shallow parsing, Chinese word segmentation and POS tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04234v2",
            "date": "2016-11-14 02:50:33+00:00",
            "title": "F-Score Driven Max Margin Neural Network for Named Entity Recognition in Chinese Social Media",
            "authors": [
                "Hangfeng He",
                "Xu Sun"
            ],
            "abstract": "We focus on named entity recognition (NER) for Chinese social media. With\nmassive unlabeled text and quite limited labelled corpus, we propose a\nsemi-supervised learning model based on B-LSTM neural network. To take\nadvantage of traditional methods in NER such as CRF, we combine transition\nprobability with deep learning in our model. To bridge the gap between label\naccuracy and F-score of NER, we construct a model which can be directly trained\non F-score. When considering the instability of F-score driven method and\nmeaningful information provided by label accuracy, we propose an integrated\nmethod to train on both F-score and label accuracy. Our integrated model yields\n7.44\\% improvement over previous state-of-the-art result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04244v1",
            "date": "2016-11-14 03:54:10+00:00",
            "title": "Classify or Select: Neural Architectures for Extractive Document Summarization",
            "authors": [
                "Ramesh Nallapati",
                "Bowen Zhou",
                "Mingbo Ma"
            ],
            "abstract": "We present two novel and contrasting Recurrent Neural Network (RNN) based\narchitectures for extractive summarization of documents. The Classifier based\narchitecture sequentially accepts or rejects each sentence in the original\ndocument order for its membership in the final summary. The Selector\narchitecture, on the other hand, is free to pick one sentence at a time in any\narbitrary order to piece together the summary. Our models under both\narchitectures jointly capture the notions of salience and redundancy of\nsentences. In addition, these models have the advantage of being very\ninterpretable, since they allow visualization of their predictions broken up by\nabstract features such as information content, salience and redundancy. We show\nthat our models reach or outperform state-of-the-art supervised models on two\ndifferent corpora. We also recommend the conditions under which one\narchitecture is superior to the other based on experimental evidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04326v2",
            "date": "2016-11-14 10:40:44+00:00",
            "title": "`Who would have thought of that!': A Hierarchical Topic Model for Extraction of Sarcasm-prevalent Topics and Sarcasm Detection",
            "authors": [
                "Aditya Joshi",
                "Prayas Jain",
                "Pushpak Bhattacharyya",
                "Mark Carman"
            ],
            "abstract": "Topic Models have been reported to be beneficial for aspect-based sentiment\nanalysis. This paper reports a simple topic model for sarcasm detection, a\nfirst, to the best of our knowledge. Designed on the basis of the intuition\nthat sarcastic tweets are likely to have a mixture of words of both sentiments\nas against tweets with literal sentiment (either positive or negative), our\nhierarchical topic model discovers sarcasm-prevalent topics and topic-level\nsentiment. Using a dataset of tweets labeled using hashtags, the model\nestimates topic-level, and sentiment-level distributions. Our evaluation shows\nthat topics such as `work', `gun laws', `weather' are sarcasm-prevalent topics.\nOur model is also able to discover the mixture of sentiment-bearing words that\nexist in a text of a given sentiment-related label. Finally, we apply our model\nto predict sarcasm in tweets. We outperform two prior work based on statistical\nclassifiers with specific features, by around 25\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04358v2",
            "date": "2016-11-14 12:24:27+00:00",
            "title": "Character-level Convolutional Network for Text Classification Applied to Chinese Corpus",
            "authors": [
                "Weijie Huang",
                "Jun Wang"
            ],
            "abstract": "This article provides an interesting exploration of character-level\nconvolutional neural network solving Chinese corpus text classification\nproblem. We constructed a large-scale Chinese language dataset, and the result\nshows that character-level convolutional neural network works better on Chinese\ncorpus than its corresponding pinyin format dataset. This is the first time\nthat character-level convolutional neural network applied to text\nclassification problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04491v1",
            "date": "2016-11-14 17:36:05+00:00",
            "title": "Ranking medical jargon in electronic health record notes by adapted distant supervision",
            "authors": [
                "Jinying Chen",
                "Abhyuday N. Jagannatha",
                "Samah J. Jarad",
                "Hong Yu"
            ],
            "abstract": "Objective: Allowing patients to access their own electronic health record\n(EHR) notes through online patient portals has the potential to improve\npatient-centered care. However, medical jargon, which abounds in EHR notes, has\nbeen shown to be a barrier for patient EHR comprehension. Existing knowledge\nbases that link medical jargon to lay terms or definitions play an important\nrole in alleviating this problem but have low coverage of medical jargon in\nEHRs. We developed a data-driven approach that mines EHRs to identify and rank\nmedical jargon based on its importance to patients, to support the building of\nEHR-centric lay language resources.\n  Methods: We developed an innovative adapted distant supervision (ADS) model\nbased on support vector machines to rank medical jargon from EHRs. For distant\nsupervision, we utilized the open-access, collaborative consumer health\nvocabulary, a large, publicly available resource that links lay terms to\nmedical jargon. We explored both knowledge-based features from the Unified\nMedical Language System and distributed word representations learned from\nunlabeled large corpora. We evaluated the ADS model using physician-identified\nimportant medical terms.\n  Results: Our ADS model significantly surpassed two state-of-the-art automatic\nterm recognition methods, TF*IDF and C-Value, yielding 0.810 ROC-AUC versus\n0.710 and 0.667, respectively. Our model identified 10K important medical\njargon terms after ranking over 100K candidate terms mined from over 7,500 EHR\nnarratives.\n  Conclusion: Our work is an important step towards enriching lexical resources\nthat link medical jargon to lay terms/definitions to support patient EHR\ncomprehension. The identified medical jargon terms and their rankings are\navailable upon request.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04496v2",
            "date": "2016-11-14 17:47:03+00:00",
            "title": "Multi-view Recurrent Neural Acoustic Word Embeddings",
            "authors": [
                "Wanjia He",
                "Weiran Wang",
                "Karen Livescu"
            ],
            "abstract": "Recent work has begun exploring neural acoustic word\nembeddings---fixed-dimensional vector representations of arbitrary-length\nspeech segments corresponding to words. Such embeddings are applicable to\nspeech retrieval and recognition tasks, where reasoning about whole words may\nmake it possible to avoid ambiguous sub-word representations. The main idea is\nto map acoustic sequences to fixed-dimensional vectors such that examples of\nthe same word are mapped to similar vectors, while different-word examples are\nmapped to very different vectors. In this work we take a multi-view approach to\nlearning acoustic word embeddings, in which we jointly learn to embed acoustic\nsequences and their corresponding character sequences. We use deep\nbidirectional LSTM embedding models and multi-view contrastive losses. We study\nthe effect of different loss variants, including fixed-margin and\ncost-sensitive losses. Our acoustic word embeddings improve over previous\napproaches for the task of word discrimination. We also present results on\nother tasks that are enabled by the multi-view approach, including cross-view\nword discrimination and word similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04684v1",
            "date": "2016-11-15 03:11:59+00:00",
            "title": "Knowledge Enhanced Hybrid Neural Network for Text Matching",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "Long text brings a big challenge to semantic matching due to their\ncomplicated semantic and syntactic structures. To tackle the challenge, we\nconsider using prior knowledge to help identify useful information and filter\nout noise to matching in long text. To this end, we propose a knowledge\nenhanced hybrid neural network (KEHNN). The model fuses prior knowledge into\nword representations by knowledge gates and establishes three matching channels\nwith words, sequential structures of sentences given by Gated Recurrent Units\n(GRU), and knowledge enhanced representations. The three channels are processed\nby a convolutional neural network to generate high level features for matching,\nand the features are synthesized as a matching score by a multilayer\nperceptron. The model extends the existing methods by conducting matching on\nwords, local structures of sentences, and global context of sentences.\nEvaluation results from extensive experiments on public data sets for question\nanswering and conversation show that KEHNN can significantly outperform\nthe-state-of-the-art matching models and particularly improve the performance\non pairs with long text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04741v2",
            "date": "2016-11-15 08:48:22+00:00",
            "title": "A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference",
            "authors": [
                "Biswajit Paria",
                "K. M. Annervaz",
                "Ambedkar Dukkipati",
                "Ankush Chatterjee",
                "Sanjay Podder"
            ],
            "abstract": "In this work we use the recent advances in representation learning to propose\na neural architecture for the problem of natural language inference. Our\napproach is aligned to mimic how a human does the natural language inference\nprocess given two statements. The model uses variants of Long Short Term Memory\n(LSTM), attention mechanism and composable neural networks, to carry out the\ntask. Each part of our model can be mapped to a clear functionality humans do\nfor carrying out the overall task of natural language inference. The model is\nend-to-end differentiable enabling training by stochastic gradient descent. On\nStanford Natural Language Inference(SNLI) dataset, the proposed model achieves\nbetter accuracy numbers than all published models in literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04798v1",
            "date": "2016-11-15 11:47:42+00:00",
            "title": "Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder",
            "authors": [
                "Thanh-Le Ha",
                "Jan Niehues",
                "Alexander Waibel"
            ],
            "abstract": "In this paper, we present our first attempts in building a multilingual\nNeural Machine Translation framework under a unified approach. We are then able\nto employ attention-based NMT for many-to-many multilingual translation tasks.\nOur approach does not require any special treatment on the network architecture\nand it allows us to learn minimal number of free parameters in a standard way\nof training. Our approach has shown its effectiveness in an under-resourced\ntranslation scenario with considerable improvements up to 2.6 BLEU points. In\naddition, the approach has achieved interesting and promising results when\napplied in the translation task that there is no direct parallel corpus between\nsource and target languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04822v2",
            "date": "2016-11-15 13:31:28+00:00",
            "title": "SimDoc: Topic Sequence Alignment based Document Similarity Framework",
            "authors": [
                "Gaurav Maheshwari",
                "Priyansh Trivedi",
                "Harshita Sahijwani",
                "Kunal Jha",
                "Sourish Dasgupta",
                "Jens Lehmann"
            ],
            "abstract": "Document similarity is the problem of estimating the degree to which a given\npair of documents has similar semantic content. An accurate document similarity\nmeasure can improve several enterprise relevant tasks such as document\nclustering, text mining, and question-answering. In this paper, we show that a\ndocument's thematic flow, which is often disregarded by bag-of-word techniques,\nis pivotal in estimating their similarity. To this end, we propose a novel\nsemantic document similarity framework, called SimDoc. We model documents as\ntopic-sequences, where topics represent latent generative clusters of related\nwords. Then, we use a sequence alignment algorithm to estimate their semantic\nsimilarity. We further conceptualize a novel mechanism to compute topic-topic\nsimilarity to fine tune our system. In our experiments, we show that SimDoc\noutperforms many contemporary bag-of-words techniques in accurately computing\ndocument similarity, and on practical applications such as document clustering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04837v1",
            "date": "2016-11-14 20:50:03+00:00",
            "title": "Lost in Space: Geolocation in Event Data",
            "authors": [
                "Sophie J. Lee",
                "Howard Liu",
                "Michael D. Ward"
            ],
            "abstract": "Extracting the \"correct\" location information from text data, i.e.,\ndetermining the place of event, has long been a goal for automated text\nprocessing. To approximate human-like coding schema, we introduce a supervised\nmachine learning algorithm that classifies each location word to be either\ncorrect or incorrect. We use news articles collected from around the world\n(Integrated Crisis Early Warning System [ICEWS] data and Open Event Data\nAlliance [OEDA] data) to test our algorithm that consists of two stages. In the\nfeature selection stage, we extract contextual information from texts, namely,\nthe N-gram patterns for location words, the frequency of mention, and the\ncontext of the sentences containing location words. In the classification\nstage, we use three classifiers to estimate the model parameters in the\ntraining set and then to predict whether a location word in the test set news\narticles is the place of the event. The validation results show that our\nalgorithm improves the accuracy rate of the current geolocation methods of\ndictionary approach by as much as 25%.",
            "journal ref": "PSRM 7 (2019) 871-888",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04928v2",
            "date": "2016-11-15 16:44:54+00:00",
            "title": "Neural Machine Translation with Pivot Languages",
            "authors": [
                "Yong Cheng",
                "Yang Liu",
                "Qian Yang",
                "Maosong Sun",
                "Wei Xu"
            ],
            "abstract": "While recent neural machine translation approaches have delivered\nstate-of-the-art performance for resource-rich language pairs, they suffer from\nthe data scarcity problem for resource-scarce language pairs. Although this\nproblem can be alleviated by exploiting a pivot language to bridge the source\nand target languages, the source-to-pivot and pivot-to-target translation\nmodels are usually independently trained. In this work, we introduce a joint\ntraining algorithm for pivot-based neural machine translation. We propose three\nmethods to connect the two models and enable them to interact with each other\nduring training. Experiments on Europarl and WMT corpora show that joint\ntraining of source-to-pivot and pivot-to-target models leads to significant\nimprovements over independent training across various languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04953v2",
            "date": "2016-11-15 17:38:10+00:00",
            "title": "End-to-End Neural Sentence Ordering Using Pointer Network",
            "authors": [
                "Jingjing Gong",
                "Xinchi Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Sentence ordering is one of important tasks in NLP. Previous works mainly\nfocused on improving its performance by using pair-wise strategy. However, it\nis nontrivial for pair-wise models to incorporate the contextual sentence\ninformation. In addition, error prorogation could be introduced by using the\npipeline strategy in pair-wise models. In this paper, we propose an end-to-end\nneural approach to address the sentence ordering problem, which uses the\npointer network (Ptr-Net) to alleviate the error propagation problem and\nutilize the whole contextual information. Experimental results show the\neffectiveness of the proposed model. Source codes and dataset of this paper are\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04989v2",
            "date": "2016-11-15 19:02:35+00:00",
            "title": "Recurrent Neural Network based Part-of-Speech Tagger for Code-Mixed Social Media Text",
            "authors": [
                "Raj Nath Patel",
                "Prakash B. Pimpale",
                "M Sasikumar"
            ],
            "abstract": "This paper describes Centre for Development of Advanced Computing's (CDACM)\nsubmission to the shared task-'Tool Contest on POS tagging for Code-Mixed\nIndian Social Media (Facebook, Twitter, and Whatsapp) Text', collocated with\nICON-2016. The shared task was to predict Part of Speech (POS) tag at word\nlevel for a given text. The code-mixed text is generated mostly on social media\nby multilingual users. The presence of the multilingual words,\ntransliterations, and spelling variations make such content linguistically\ncomplex. In this paper, we propose an approach to POS tag code-mixed social\nmedia text using Recurrent Neural Network Language Model (RNN-LM) architecture.\nWe submitted the results for Hindi-English (hi-en), Bengali-English (bn-en),\nand Telugu-English (te-en) code-mixed data.",
            "journal ref": "In Proceedings of the Tool Contest on POS tagging for Indian\n  Social Media Text, ICON 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05239v2",
            "date": "2016-11-16 12:04:19+00:00",
            "title": "How to do lexical quality estimation of a large OCRed historical Finnish newspaper collection with scarce resources",
            "authors": [
                "Kimmo Kettunen"
            ],
            "abstract": "The National Library of Finland has digitized the historical newspapers\npublished in Finland between 1771 and 1910. This collection contains\napproximately 1.95 million pages in Finnish and Swedish. Finnish part of the\ncollection consists of about 2.40 billion words. The National Library's Digital\nCollections are offered via the digi.kansalliskirjasto.fi web service, also\nknown as Digi. Part of the newspaper material (from 1771 to 1874) is also\navailable freely downloadable in The Language Bank of Finland provided by the\nFINCLARIN consortium. The collection can also be accessed through the Korp\nenvironment that has been developed by Spr{\\aa}kbanken at the University of\nGothenburg and extended by FINCLARIN team at the University of Helsinki to\nprovide concordances of text resources. A Cranfield style information retrieval\ntest collection has also been produced out of a small part of the Digi\nnewspaper material at the University of Tampere.\n  Quality of OCRed collections is an important topic in digital humanities, as\nit affects general usability and searchability of collections. There is no\nsingle available method to assess quality of large collections, but different\nmethods can be used to approximate quality. This paper discusses different\ncorpus analysis style methods to approximate overall lexical quality of the\nFinnish part of the Digi collection. Methods include usage of parallel samples\nand word error rates, usage of morphological analyzers, frequency analysis of\nwords and comparisons to comparable edited lexical data. Our aim in the quality\nanalysis is twofold: firstly to analyze the present state of the lexical data\nand secondly, to establish a set of assessment methods that build up a compact\nprocedure for quality assessment after e.g. new OCRing or post correction of\nthe material. In the discussion part of the paper we shall synthesize results\nof our different analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05360v1",
            "date": "2016-11-16 16:55:42+00:00",
            "title": "The Life of Lazarillo de Tormes and of His Machine Learning Adversities",
            "authors": [
                "Javier de la Rosa",
                "Juan-Luis Su\u00e1rez"
            ],
            "abstract": "Summit work of the Spanish Golden Age and forefather of the so-called\npicaresque novel, The Life of Lazarillo de Tormes and of His Fortunes and\nAdversities still remains an anonymous text. Although distinguished scholars\nhave tried to attribute it to different authors based on a variety of criteria,\na consensus has yet to be reached. The list of candidates is long and not all\nof them enjoy the same support within the scholarly community. Analyzing their\nworks from a data-driven perspective and applying machine learning techniques\nfor style and text fingerprinting, we shed light on the authorship of the\nLazarillo. As in a state-of-the-art survey, we discuss the methods used and how\nthey perform in our specific case. According to our methodology, the most\nlikely author seems to be Juan Arce de Ot\\'alora, closely followed by Alfonso\nde Vald\\'es. The method states that not certain attribution can be made with\nthe given corpus.",
            "journal ref": "Lemir: Revista de Literatura Espa\\~nola Medieval y del\n  Renacimiento, 20 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05384v2",
            "date": "2016-11-16 17:47:57+00:00",
            "title": "A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging",
            "authors": [
                "Xinchi Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, neural network models for natural language processing tasks have\nbeen increasingly focused on for their ability of alleviating the burden of\nmanual feature engineering. However, the previous neural models cannot extract\nthe complicated feature compositions as the traditional methods with discrete\nfeatures. In this work, we propose a feature-enriched neural model for joint\nChinese word segmentation and part-of-speech tagging task. Specifically, to\nsimulate the feature templates of traditional discrete feature based models, we\nuse different filters to model the complex compositional features with\nconvolutional and pooling layer, and then utilize long distance dependency\ninformation with recurrent layer. Experimental results on five different\ndatasets show the effectiveness of our proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05774v2",
            "date": "2016-11-17 16:41:41+00:00",
            "title": "What Do Recurrent Neural Network Grammars Learn About Syntax?",
            "authors": [
                "Adhiguna Kuncoro",
                "Miguel Ballesteros",
                "Lingpeng Kong",
                "Chris Dyer",
                "Graham Neubig",
                "Noah A. Smith"
            ],
            "abstract": "Recurrent neural network grammars (RNNG) are a recently proposed\nprobabilistic generative modeling family for natural language. They show\nstate-of-the-art language modeling and parsing performance. We investigate what\ninformation they learn, from a linguistic perspective, through various\nablations to the model and the data, and by augmenting the model with an\nattention mechanism (GA-RNNG) to enable closer inspection. We find that\nexplicit modeling of composition is crucial for achieving the best performance.\nThrough the attention mechanism, we find that headedness plays a central role\nin phrasal representation (with the model's latent attention largely agreeing\nwith predictions made by hand-crafted head rules, albeit with some important\ndifferences). By training grammars without nonterminal labels, we find that\nphrasal representations depend minimally on nonterminals, providing support for\nthe endocentricity hypothesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05962v1",
            "date": "2016-11-18 03:21:28+00:00",
            "title": "Word and Document Embeddings based on Neural Network Approaches",
            "authors": [
                "Siwei Lai"
            ],
            "abstract": "Data representation is a fundamental task in machine learning. The\nrepresentation of data affects the performance of the whole machine learning\nsystem. In a long history, the representation of data is done by feature\nengineering, and researchers aim at designing better features for specific\ntasks. Recently, the rapid development of deep learning and representation\nlearning has brought new inspiration to various domains.\n  In natural language processing, the most widely used feature representation\nis the Bag-of-Words model. This model has the data sparsity problem and cannot\nkeep the word order information. Other features such as part-of-speech tagging\nor more complex syntax features can only fit for specific tasks in most cases.\nThis thesis focuses on word representation and document representation. We\ncompare the existing systems and present our new model.\n  First, for generating word embeddings, we make comprehensive comparisons\namong existing word embedding models. In terms of theory, we figure out the\nrelationship between the two most important models, i.e., Skip-gram and GloVe.\nIn our experiments, we analyze three key points in generating word embeddings,\nincluding the model construction, the training corpus and parameter design. We\nevaluate word embeddings with three types of tasks, and we argue that they\ncover the existing use of word embeddings. Through theory and practical\nexperiments, we present some guidelines for how to generate a good word\nembedding.\n  Second, in Chinese character or word representation. We introduce the joint\ntraining of Chinese character and word. ...\n  Third, for document representation, we analyze the existing document\nrepresentation models, including recursive NNs, recurrent NNs and convolutional\nNNs. We point out the drawbacks of these models and present our new model, the\nrecurrent convolutional neural networks. ...",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06320v2",
            "date": "2016-11-19 07:14:59+00:00",
            "title": "Tracking Words in Chinese Poetry of Tang and Song Dynasties with the China Biographical Database",
            "authors": [
                "Chao-Lin Liu",
                "Kuo-Feng Luo"
            ],
            "abstract": "Large-scale comparisons between the poetry of Tang and Song dynasties shed\nlight on how words, collocations, and expressions were used and shared among\nthe poets. That some words were used only in the Tang poetry and some only in\nthe Song poetry could lead to interesting research in linguistics. That the\nmost frequent colors are different in the Tang and Song poetry provides a trace\nof the changing social circumstances in the dynasties. Results of the current\nwork link to research topics of lexicography, semantics, and social\ntransitions. We discuss our findings and present our algorithms for efficient\ncomparisons among the poems, which are crucial for completing billion times of\ncomparisons within acceptable time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06423v2",
            "date": "2016-11-19 20:12:19+00:00",
            "title": "Incorporating Pass-Phrase Dependent Background Models for Text-Dependent Speaker Verification",
            "authors": [
                "A. K. Sarkar",
                "Zheng-Hua Tan"
            ],
            "abstract": "In this paper, we propose pass-phrase dependent background models (PBMs) for\ntext-dependent (TD) speaker verification (SV) to integrate the pass-phrase\nidentification process into the conventional TD-SV system, where a PBM is\nderived from a text-independent background model through adaptation using the\nutterances of a particular pass-phrase. During training, pass-phrase specific\ntarget speaker models are derived from the particular PBM using the training\ndata for the respective target model. While testing, the best PBM is first\nselected for the test utterance in the maximum likelihood (ML) sense and the\nselected PBM is then used for the log likelihood ratio (LLR) calculation with\nrespect to the claimant model. The proposed method incorporates the pass-phrase\nidentification step in the LLR calculation, which is not considered in\nconventional standalone TD-SV systems. The performance of the proposed method\nis compared to conventional text-independent background model based TD-SV\nsystems using either Gaussian mixture model (GMM)-universal background model\n(UBM) or Hidden Markov model (HMM)-UBM or i-vector paradigms. In addition, we\nconsider two approaches to build PBMs: speaker-independent and\nspeaker-dependent. We show that the proposed method significantly reduces the\nerror rates of text-dependent speaker verification for the non-target types:\ntarget-wrong and imposter-wrong while it maintains comparable TD-SV performance\nwhen imposters speak a correct utterance with respect to the conventional\nsystem. Experiments are conducted on the RedDots challenge and the RSR2015\ndatabases that consist of short utterances.",
            "journal ref": "Computer speech Language: Volume 47,January 2018, pp. 259-271",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06639v1",
            "date": "2016-11-21 03:26:29+00:00",
            "title": "Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling",
            "authors": [
                "Peng Zhou",
                "Zhenyu Qi",
                "Suncong Zheng",
                "Jiaming Xu",
                "Hongyun Bao",
                "Bo Xu"
            ],
            "abstract": "Recurrent Neural Network (RNN) is one of the most popular architectures used\nin Natural Language Processsing (NLP) tasks because its recurrent structure is\nvery suitable to process variable-length text. RNN can utilize distributed\nrepresentations of words by first converting the tokens comprising each text\ninto vectors, which form a matrix. And this matrix includes two dimensions: the\ntime-step dimension and the feature vector dimension. Then most existing models\nusually utilize one-dimensional (1D) max pooling operation or attention-based\noperation only on the time-step dimension to obtain a fixed-length vector.\nHowever, the features on the feature vector dimension are not mutually\nindependent, and simply applying 1D pooling operation over the time-step\ndimension independently may destroy the structure of the feature\nrepresentation. On the other hand, applying two-dimensional (2D) pooling\noperation over the two dimensions may sample more meaningful features for\nsequence modeling tasks. To integrate the features on both dimensions of the\nmatrix, this paper explores applying 2D max pooling operation to obtain a\nfixed-length representation of the text. This paper also utilizes 2D\nconvolution to sample more meaningful information of the matrix. Experiments\nare conducted on six text classification tasks, including sentiment analysis,\nquestion classification, subjectivity classification and newsgroup\nclassification. Compared with the state-of-the-art models, the proposed models\nachieve excellent performance on 4 out of 6 tasks. Specifically, one of the\nproposed models achieves highest accuracy on Stanford Sentiment Treebank binary\nclassification and fine-grained classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06722v1",
            "date": "2016-11-21 11:07:11+00:00",
            "title": "False-Friend Detection and Entity Matching via Unsupervised Transliteration",
            "authors": [
                "Yanqing Chen",
                "Steven Skiena"
            ],
            "abstract": "Transliterations play an important role in multilingual entity reference\nresolution, because proper names increasingly travel between languages in news\nand social media. Previous work associated with machine translation targets\ntransliteration only single between language pairs, focuses on specific classes\nof entities (such as cities and celebrities) and relies on manual curation,\nwhich limits the expression power of transliteration in multilingual\nenvironment.\n  By contrast, we present an unsupervised transliteration model covering 69\nmajor languages that can generate good transliterations for arbitrary strings\nbetween any language pair. Our model yields top-(1, 20, 100) averages of\n(32.85%, 60.44%, 83.20%) in matching gold standard transliteration compared to\nresults from a recently-published system of (26.71%, 50.27%, 72.79%). We also\nshow the quality of our model in detecting true and false friends from\nWikipedia high frequency lexicons. Our method indicates a strong signal of\npronunciation similarity and boosts the probability of finding true friends in\n68 out of 69 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06788v1",
            "date": "2016-11-21 14:01:53+00:00",
            "title": "Bidirectional Tree-Structured LSTM with Head Lexicalization",
            "authors": [
                "Zhiyang Teng",
                "Yue Zhang"
            ],
            "abstract": "Sequential LSTM has been extended to model tree structures, giving\ncompetitive results for a number of tasks. Existing methods model constituent\ntrees by bottom-up combinations of constituent nodes, making direct use of\ninput word information only for leaf nodes. This is different from sequential\nLSTMs, which contain reference to input words for each node. In this paper, we\npropose a method for automatic head-lexicalization for tree-structure LSTMs,\npropagating head words from leaf nodes to every constituent node. In addition,\nenabled by head lexicalization, we build a tree LSTM in the top-down direction,\nwhich corresponds to bidirectional sequential LSTM structurally. Experiments\nshow that both extensions give better representations of tree structures. Our\nfinal model gives the best results on the Standford Sentiment Treebank and\nhighly competitive results on the TREC question type classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07206v1",
            "date": "2016-11-22 09:11:42+00:00",
            "title": "Learning to Distill: The Essence Vector Modeling Framework",
            "authors": [
                "Kuan-Yu Chen",
                "Shih-Hung Liu",
                "Berlin Chen",
                "Hsin-Min Wang"
            ],
            "abstract": "In the context of natural language processing, representation learning has\nemerged as a newly active research subject because of its excellent performance\nin many applications. Learning representations of words is a pioneering study\nin this school of research. However, paragraph (or sentence and document)\nembedding learning is more suitable/reasonable for some tasks, such as\nsentiment classification and document summarization. Nevertheless, as far as we\nare aware, there is relatively less work focusing on the development of\nunsupervised paragraph embedding methods. Classic paragraph embedding methods\ninfer the representation of a given paragraph by considering all of the words\noccurring in the paragraph. Consequently, those stop or function words that\noccur frequently may mislead the embedding learning process to produce a misty\nparagraph representation. Motivated by these observations, our major\ncontributions in this paper are twofold. First, we propose a novel unsupervised\nparagraph embedding method, named the essence vector (EV) model, which aims at\nnot only distilling the most representative information from a paragraph but\nalso excluding the general background information to produce a more informative\nlow-dimensional vector representation for the paragraph. Second, in view of the\nincreasing importance of spoken content processing, an extension of the EV\nmodel, named the denoising essence vector (D-EV) model, is proposed. The D-EV\nmodel not only inherits the advantages of the EV model but also can infer a\nmore robust representation for a given spoken paragraph against imperfect\nspeech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07232v4",
            "date": "2016-11-22 10:11:56+00:00",
            "title": "Compositional Learning of Relation Path Embedding for Knowledge Base Completion",
            "authors": [
                "Xixun Lin",
                "Yanchun Liang",
                "Fausto Giunchiglia",
                "Xiaoyue Feng",
                "Renchu Guan"
            ],
            "abstract": "Large-scale knowledge bases have currently reached impressive sizes; however,\nthese knowledge bases are still far from complete. In addition, most of the\nexisting methods for knowledge base completion only consider the direct links\nbetween entities, ignoring the vital impact of the consistent semantics of\nrelation paths. In this paper, we study the problem of how to better embed\nentities and relations of knowledge bases into different low-dimensional spaces\nby taking full advantage of the additional semantics of relation paths, and we\npropose a compositional learning model of relation path embedding (RPE).\nSpecifically, with the corresponding relation and path projections, RPE can\nsimultaneously embed each entity into two types of latent spaces. It is also\nproposed that type constraints could be extended from traditional\nrelation-specific constraints to the new proposed path-specific constraints.\nThe results of experiments show that the proposed model achieves significant\nand consistent improvements compared with the state-of-the-art algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07804v1",
            "date": "2016-11-23 14:14:52+00:00",
            "title": "ATR4S: Toolkit with State-of-the-art Automatic Terms Recognition Methods in Scala",
            "authors": [
                "N. Astrakhantsev"
            ],
            "abstract": "Automatically recognized terminology is widely used for various\ndomain-specific texts processing tasks, such as machine translation,\ninformation retrieval or sentiment analysis. However, there is still no\nagreement on which methods are best suited for particular settings and,\nmoreover, there is no reliable comparison of already developed methods. We\nbelieve that one of the main reasons is the lack of state-of-the-art methods\nimplementations, which are usually non-trivial to recreate. In order to address\nthese issues, we present ATR4S, an open-source software written in Scala that\ncomprises more than 15 methods for automatic terminology recognition (ATR) and\nimplements the whole pipeline from text document preprocessing, to term\ncandidates collection, term candidates scoring, and finally, term candidates\nranking. It is highly scalable, modular and configurable tool with support of\nautomatic caching. We also compare 10 state-of-the-art methods on 7 open\ndatasets by average precision and processing time. Experimental comparison\nreveals that no single method demonstrates best average precision for all\ndatasets and that other available tools for ATR do not contain the best\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07954v2",
            "date": "2016-11-23 19:51:34+00:00",
            "title": "Emergent Predication Structure in Hidden State Vectors of Neural Readers",
            "authors": [
                "Hai Wang",
                "Takeshi Onishi",
                "Kevin Gimpel",
                "David McAllester"
            ],
            "abstract": "A significant number of neural architectures for reading comprehension have\nrecently been developed and evaluated on large cloze-style datasets. We present\nexperiments supporting the emergence of \"predication structure\" in the hidden\nstate vectors of these readers. More specifically, we provide evidence that the\nhidden state vectors represent atomic formulas $\\Phi[c]$ where $\\Phi$ is a\nsemantic property (predicate) and $c$ is a constant symbol entity identifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08358v1",
            "date": "2016-11-25 06:18:29+00:00",
            "title": "Kannada Spell Checker with Sandhi Splitter",
            "authors": [
                "A N Akshatha",
                "Chandana G Upadhyaya",
                "Rajashekara S Murthy"
            ],
            "abstract": "Spelling errors are introduced in text either during typing, or when the user\ndoes not know the correct phoneme or grapheme. If a language contains complex\nwords like sandhi where two or more morphemes join based on some rules, spell\nchecking becomes very tedious. In such situations, having a spell checker with\nsandhi splitter which alerts the user by flagging the errors and providing\nsuggestions is very useful. A novel algorithm of sandhi splitting is proposed\nin this paper. The sandhi splitter can split about 7000 most common sandhi\nwords in Kannada language used as test samples. The sandhi splitter was\nintegrated with a Kannada spell checker and a mechanism for generating\nsuggestions was added. A comprehensive, platform independent, standalone spell\nchecker with sandhi splitter application software was thus developed and tested\nextensively for its efficiency and correctness. A comparative analysis of this\nspell checker with sandhi splitter was made and results concluded that the\nKannada spell checker with sandhi splitter has an improved performance. It is\ntwice as fast, 200 times more space efficient, and it is 90% accurate in case\nof complex nouns and 50% accurate for complex verbs. Such a spell checker with\nsandhi splitter will be of foremost significance in machine translation\nsystems, voice processing, etc. This is the first sandhi splitter in Kannada\nand the advantage of the novel algorithm is that, it can be extended to all\nIndian languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08459v1",
            "date": "2016-11-25 14:10:39+00:00",
            "title": "Neural Machine Translation with Latent Semantic of Image and Text",
            "authors": [
                "Joji Toyama",
                "Masanori Misono",
                "Masahiro Suzuki",
                "Kotaro Nakayama",
                "Yutaka Matsuo"
            ],
            "abstract": "Although attention-based Neural Machine Translation have achieved great\nsuccess, attention-mechanism cannot capture the entire meaning of the source\nsentence because the attention mechanism generates a target word depending\nheavily on the relevant parts of the source sentence. The report of earlier\nstudies has introduced a latent variable to capture the entire meaning of\nsentence and achieved improvement on attention-based Neural Machine\nTranslation. We follow this approach and we believe that the capturing meaning\nof sentence benefits from image information because human beings understand the\nmeaning of language not only from textual information but also from perceptual\ninformation such as that gained from vision. As described herein, we propose a\nneural machine translation model that introduces a continuous latent variable\ncontaining an underlying semantic extracted from texts and images. Our model,\nwhich can be trained end-to-end, requires image information only when training.\nExperiments conducted with an English--German translation task show that our\nmodel outperforms over the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08562v2",
            "date": "2016-11-25 19:18:27+00:00",
            "title": "A Simple, Fast Diverse Decoding Algorithm for Neural Generation",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Dan Jurafsky"
            ],
            "abstract": "In this paper, we propose a simple, fast decoding algorithm that fosters\ndiversity in neural generation. The algorithm modifies the standard beam search\nalgorithm by adding an inter-sibling ranking penalty, favoring choosing\nhypotheses from diverse parents. We evaluate the proposed model on the tasks of\ndialogue response generation, abstractive summarization and machine\ntranslation. We find that diverse decoding helps across all tasks, especially\nthose for which reranking is needed.\n  We further propose a variation that is capable of automatically adjusting its\ndiversity decoding rates for different inputs using reinforcement learning\n(RL). We observe a further performance boost from this RL technique. This paper\nincludes material from the unpublished script \"Mutual Information and Diverse\nDecoding Improve Neural Machine Translation\" (Li and Jurafsky, 2016).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08656v1",
            "date": "2016-11-26 04:25:00+00:00",
            "title": "Attention-based Memory Selection Recurrent Network for Language Modeling",
            "authors": [
                "Da-Rong Liu",
                "Shun-Po Chuang",
                "Hung-yi Lee"
            ],
            "abstract": "Recurrent neural networks (RNNs) have achieved great success in language\nmodeling. However, since the RNNs have fixed size of memory, their memory\ncannot store all the information about the words it have seen before in the\nsentence, and thus the useful long-term information may be ignored when\npredicting the next words. In this paper, we propose Attention-based Memory\nSelection Recurrent Network (AMSRN), in which the model can review the\ninformation stored in the memory at each previous time step and select the\nrelevant information to help generate the outputs. In AMSRN, the attention\nmechanism finds the time steps storing the relevant information in the memory,\nand memory selection determines which dimensions of the memory are involved in\ncomputing the attention weights and from which the information is extracted.In\nthe experiments, AMSRN outperformed long short-term memory (LSTM) based\nlanguage models on both English and Chinese corpora. Moreover, we investigate\nusing entropy as a regularizer for attention weights and visualize how the\nattention mechanism helps language modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08661v2",
            "date": "2016-11-26 05:42:20+00:00",
            "title": "Knowledge Graph Representation with Jointly Structural and Textual Encoding",
            "authors": [
                "Jiacheng Xu",
                "Kan Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "The objective of knowledge graph embedding is to encode both entities and\nrelations of knowledge graphs into continuous low-dimensional vector spaces.\nPreviously, most works focused on symbolic representation of knowledge graph\nwith structure information, which can not handle new entities or entities with\nfew facts well. In this paper, we propose a novel deep architecture to utilize\nboth structural and textual information of entities. Specifically, we introduce\nthree neural models to encode the valuable information from text description of\nentity, among which an attentive model can select related information as\nneeded. Then, a gating mechanism is applied to integrate representations of\nstructure and text into a unified architecture. Experiments show that our\nmodels outperform baseline by margin on link prediction and triplet\nclassification tasks. Source codes of this paper will be available on Github.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08765v1",
            "date": "2016-11-26 23:39:41+00:00",
            "title": "Fill it up: Exploiting partial dependency annotations in a minimum spanning tree parser",
            "authors": [
                "Liang Sun",
                "Jason Mielens",
                "Jason Baldridge"
            ],
            "abstract": "Unsupervised models of dependency parsing typically require large amounts of\nclean, unlabeled data plus gold-standard part-of-speech tags. Adding indirect\nsupervision (e.g. language universals and rules) can help, but we show that\nobtaining small amounts of direct supervision - here, partial dependency\nannotations - provides a strong balance between zero and full supervision. We\nadapt the unsupervised ConvexMST dependency parser to learn from partial\ndependencies expressed in the Graph Fragment Language. With less than 24 hours\nof total annotation, we obtain 7% and 17% absolute improvement in unlabeled\ndependency scores for English and Spanish, respectively, compared to the same\nparser using only universal grammar constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08813v1",
            "date": "2016-11-27 09:53:36+00:00",
            "title": "Semi Supervised Preposition-Sense Disambiguation using Multilingual Data",
            "authors": [
                "Hila Gonen",
                "Yoav Goldberg"
            ],
            "abstract": "Prepositions are very common and very ambiguous, and understanding their\nsense is critical for understanding the meaning of the sentence. Supervised\ncorpora for the preposition-sense disambiguation task are small, suggesting a\nsemi-supervised approach to the task. We show that signals from unannotated\nmultilingual data can be used to improve supervised preposition-sense\ndisambiguation. Our approach pre-trains an LSTM encoder for predicting the\ntranslation of a preposition, and then incorporates the pre-trained encoder as\na component in a supervised classification system, and fine-tunes it for the\ntask. The multilingual signals consistently improve results on two\npreposition-sense datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08987v2",
            "date": "2016-11-28 05:32:35+00:00",
            "title": "Exploiting Unlabeled Data for Neural Grammatical Error Detection",
            "authors": [
                "Zhuoran Liu",
                "Yang Liu"
            ],
            "abstract": "Identifying and correcting grammatical errors in the text written by\nnon-native writers has received increasing attention in recent years. Although\na number of annotated corpora have been established to facilitate data-driven\ngrammatical error detection and correction approaches, they are still limited\nin terms of quantity and coverage because human annotation is labor-intensive,\ntime-consuming, and expensive. In this work, we propose to utilize unlabeled\ndata to train neural network based grammatical error detection models. The\nbasic idea is to cast error detection as a binary classification problem and\nderive positive and negative training examples from unlabeled data. We\nintroduce an attention-based neural network to capture long-distance\ndependencies that influence the word being detected. Experiments show that the\nproposed approach significantly outperforms SVMs and convolutional networks\nwith fixed-size context window.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09020v2",
            "date": "2016-11-28 08:20:54+00:00",
            "title": "Developing a cardiovascular disease risk factor annotated corpus of Chinese electronic medical records",
            "authors": [
                "Jia Su",
                "Bin He",
                "Yi Guan",
                "Jingchi Jiang",
                "Jinfeng Yang"
            ],
            "abstract": "Cardiovascular disease (CVD) has become the leading cause of death in China,\nand most of the cases can be prevented by controlling risk factors. The goal of\nthis study was to build a corpus of CVD risk factor annotations based on\nChinese electronic medical records (CEMRs). This corpus is intended to be used\nto develop a risk factor information extraction system that, in turn, can be\napplied as a foundation for the further study of the progress of risk factors\nand CVD. We designed a light annotation task to capture CVD risk factors with\nindicators, temporal attributes and assertions that were explicitly or\nimplicitly displayed in the records. The task included: 1) preparing data; 2)\ncreating guidelines for capturing annotations (these were created with the help\nof clinicians); 3) proposing an annotation method including building the\nguidelines draft, training the annotators and updating the guidelines, and\ncorpus construction. Then, a risk factor annotated corpus based on\nde-identified discharge summaries and progress notes from 600 patients was\ndeveloped. Built with the help of clinicians, this corpus has an\ninter-annotator agreement (IAA) F1-measure of 0.968, indicating a high\nreliability. To the best of our knowledge, this is the first annotated corpus\nconcerning CVD risk factors in CEMRs and the guidelines for capturing CVD risk\nfactor annotations from CEMRs were proposed. The obtained document-level\nannotations can be applied in future studies to monitor risk factors and CVD\nover the long term.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09100v1",
            "date": "2016-11-28 12:57:07+00:00",
            "title": "Learning to Compose Words into Sentences with Reinforcement Learning",
            "authors": [
                "Dani Yogatama",
                "Phil Blunsom",
                "Chris Dyer",
                "Edward Grefenstette",
                "Wang Ling"
            ],
            "abstract": "We use reinforcement learning to learn tree-structured neural networks for\ncomputing representations of natural language sentences. In contrast with prior\nwork on tree-structured models in which the trees are either provided as input\nor predicted using supervision from explicit treebank annotations, the tree\nstructures in this work are optimized to improve performance on a downstream\ntask. Experiments demonstrate the benefit of learning task-specific composition\norders, outperforming both sequential encoders and recursive encoders based on\ntreebank annotations. We analyze the induced trees and show that while they\ndiscover some linguistically intuitive structures (e.g., noun phrases, simple\nverb phrases), they are different than conventional English syntactic\nstructures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09405v1",
            "date": "2016-11-28 22:03:22+00:00",
            "title": "An End-to-End Architecture for Keyword Spotting and Voice Activity Detection",
            "authors": [
                "Chris Lengerich",
                "Awni Hannun"
            ],
            "abstract": "We propose a single neural network architecture for two tasks: on-line\nkeyword spotting and voice activity detection. We develop novel inference\nalgorithms for an end-to-end Recurrent Neural Network trained with the\nConnectionist Temporal Classification loss function which allow our model to\nachieve high accuracy on both keyword spotting and voice activity detection\nwithout retraining. In contrast to prior voice activity detection models, our\narchitecture does not require aligned training data and uses the same\nparameters as the keyword spotting model. This allows us to deploy a high\nquality voice activity detector with no additional memory or maintenance\nrequirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09799v1",
            "date": "2016-11-29 19:23:41+00:00",
            "title": "Geometry of Compositionality",
            "authors": [
                "Hongyu Gong",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "This paper proposes a simple test for compositionality (i.e., literal usage)\nof a word or phrase in a context-specific way. The test is computationally\nsimple, relying on no external resources and only uses a set of trained word\nvectors. Experiments show that the proposed method is competitive with state of\nthe art and displays high accuracy in context-specific compositionality\ndetection of a variety of natural language phenomena (idiomaticity, sarcasm,\nmetaphor) for different datasets in multiple languages. The key insight is to\nconnect compositionality to a curious geometric property of word embeddings,\nwhich is of independent interest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09900v1",
            "date": "2016-11-29 21:45:42+00:00",
            "title": "Context-aware Natural Language Generation with Recurrent Neural Networks",
            "authors": [
                "Jian Tang",
                "Yifan Yang",
                "Sam Carton",
                "Ming Zhang",
                "Qiaozhu Mei"
            ],
            "abstract": "This paper studied generating natural languages at particular contexts or\nsituations. We proposed two novel approaches which encode the contexts into a\ncontinuous semantic representation and then decode the semantic representation\ninto text sequences with recurrent neural networks. During decoding, the\ncontext information are attended through a gating mechanism, addressing the\nproblem of long-range dependency caused by lengthy sequences. We evaluate the\neffectiveness of the proposed approaches on user review data, in which rich\ncontexts are available and two informative contexts, sentiments and products,\nare selected for evaluation. Experiments show that the fake reviews generated\nby our approaches are very natural. Results of fake review detection with human\njudges show that more than 50\\% of the fake reviews are misclassified as the\nreal reviews, and more than 90\\% are misclassified by existing state-of-the-art\nfake review detection algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.10038v1",
            "date": "2016-11-30 07:53:34+00:00",
            "title": "Towards Accurate Word Segmentation for Chinese Patents",
            "authors": [
                "Si Li",
                "Nianwen Xue"
            ],
            "abstract": "A patent is a property right for an invention granted by the government to\nthe inventor. An invention is a solution to a specific technological problem.\nSo patents often have a high concentration of scientific and technical terms\nthat are rare in everyday language. The Chinese word segmentation model trained\non currently available everyday language data sets performs poorly because it\ncannot effectively recognize these scientific and technical terms. In this\npaper we describe a pragmatic approach to Chinese word segmentation on patents\nwhere we train a character-based semi-supervised sequence labeling model by\nextracting features from a manually segmented corpus of 142 patents, enhanced\nwith information extracted from the Chinese TreeBank. Experiments show that the\naccuracy of our model reached 95.08% (F1 score) on a held-out test set and\n96.59% on development set, compared with an F1 score of 91.48% on development\nset if the model is trained on the Chinese TreeBank. We also experimented with\nsome existing domain adaptation techniques, the results show that the amount of\ntarget domain data and the selected features impact the performance of the\ndomain adaptation techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.10122v1",
            "date": "2016-11-30 12:30:11+00:00",
            "title": "Deep encoding of etymological information in TEI",
            "authors": [
                "Jack Bowers",
                "Laurent Romary"
            ],
            "abstract": "This paper aims to provide a comprehensive modeling and representation of\netymological data in digital dictionaries. The purpose is to integrate in one\ncoherent framework both digital representations of legacy dictionaries, and\nalso born-digital lexical databases that are constructed manually or\nsemi-automatically. We want to propose a systematic and coherent set of\nmodeling principles for a variety of etymological phenomena that may contribute\nto the creation of a continuum between existing and future lexical constructs,\nwhere anyone interested in tracing the history of words and their meanings will\nbe able to seamlessly query lexical resources.Instead of designing an ad hoc\nmodel and representation language for digital etymological data, we will focus\non identifying all the possibilities offered by the TEI guidelines for the\nrepresentation of lexical information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00030v2",
            "date": "2016-06-30 20:35:47+00:00",
            "title": "HUME: Human UCCA-Based Evaluation of Machine Translation",
            "authors": [
                "Alexandra Birch",
                "Omri Abend",
                "Ondrej Bojar",
                "Barry Haddow"
            ],
            "abstract": "Human evaluation of machine translation normally uses sentence-level measures\nsuch as relative ranking or adequacy scales. However, these provide no insight\ninto possible errors, and do not scale well with sentence length. We argue for\na semantics-based evaluation, which captures what meaning components are\nretained in the MT output, thus providing a more fine-grained analysis of\ntranslation quality, and enabling the construction and tuning of\nsemantics-based MT. We present a novel human semantic evaluation measure, Human\nUCCA-based MT Evaluation (HUME), building on the UCCA semantic representation\nscheme. HUME covers a wider range of semantic phenomena than previous methods\nand does not rely on semantic annotation of the potentially garbled MT output.\nWe experiment with four language pairs, demonstrating HUME's broad\napplicability, and report good inter-annotator agreement rates and correlation\nwith human adequacy scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00070v1",
            "date": "2016-06-30 22:51:00+00:00",
            "title": "A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems",
            "authors": [
                "Layla El Asri",
                "Jing He",
                "Kaheer Suleman"
            ],
            "abstract": "User simulation is essential for generating enough data to train a\nstatistical spoken dialogue system. Previous models for user simulation suffer\nfrom several drawbacks, such as the inability to take dialogue history into\naccount, the need of rigid structure to ensure coherent user behaviour, heavy\ndependence on a specific domain, the inability to output several user\nintentions during one dialogue turn, or the requirement of a summarized action\nspace for tractability. This paper introduces a data-driven user simulator\nbased on an encoder-decoder recurrent neural network. The model takes as input\na sequence of dialogue contexts and outputs a sequence of dialogue acts\ncorresponding to user intentions. The dialogue contexts include information\nabout the machine acts and the status of the user goal. We show on the Dialogue\nState Tracking Challenge 2 (DSTC2) dataset that the sequence-to-sequence model\noutperforms an agenda-based simulator and an n-gram simulator, according to\nF-score. Furthermore, we show how this model can be used on the original action\nspace and thereby models user behaviour with finer granularity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00139v2",
            "date": "2016-07-01 07:50:02+00:00",
            "title": "TensiStrength: Stress and relaxation magnitude detection for social media texts",
            "authors": [
                "Mike Thelwall"
            ],
            "abstract": "Computer systems need to be able to react to stress in order to perform\noptimally on some tasks. This article describes TensiStrength, a system to\ndetect the strength of stress and relaxation expressed in social media text\nmessages. TensiStrength uses a lexical approach and a set of rules to detect\ndirect and indirect expressions of stress or relaxation, particularly in the\ncontext of transportation. It is slightly more effective than a comparable\nsentiment analysis program, although their similar performances occur despite\ndifferences on almost half of the tweets gathered. The effectiveness of\nTensiStrength depends on the nature of the tweets classified, with tweets that\nare rich in stress-related terms being particularly problematic. Although\ngeneric machine learning methods can give better performance than TensiStrength\noverall, they exploit topic-related terms in a way that may be undesirable in\npractical applications and that may not work as well in more focused contexts.\nIn conclusion, TensiStrength and generic machine learning approaches work well\nenough to be practical choices for intelligent applications that need to take\nadvantage of stress information, and the decision about which to use depends on\nthe nature of the texts analysed and the purpose of the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00198v1",
            "date": "2016-07-01 10:35:59+00:00",
            "title": "Sharing Network Parameters for Crosslingual Named Entity Recognition",
            "authors": [
                "Rudra Murthy V",
                "Mitesh Khapra",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Most state of the art approaches for Named Entity Recognition rely on hand\ncrafted features and annotated corpora. Recently Neural network based models\nhave been proposed which do not require handcrafted features but still require\nannotated corpora. However, such annotated corpora may not be available for\nmany languages. In this paper, we propose a neural network based model which\nallows sharing the decoder as well as word and character level parameters\nbetween two languages thereby allowing a resource fortunate language to aid a\nresource deprived language. Specifically, we focus on the case when limited\nannotated corpora is available in one language ($L_1$) and abundant annotated\ncorpora is available in another language ($L_2$). Sharing the network\narchitecture and parameters between $L_1$ and $L_2$ leads to improved\nperformance in $L_1$. Further, our approach does not require any hand crafted\nfeatures but instead directly learns meaningful feature representations from\nthe training data itself. We experiment with 4 language pairs and show that\nindeed in a resource constrained setup (lesser annotated corpora), a model\njointly trained with data from another language performs better than a model\ntrained only on the limited corpora in one language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00225v1",
            "date": "2016-07-01 12:48:35+00:00",
            "title": "Evaluating Unsupervised Dutch Word Embeddings as a Linguistic Resource",
            "authors": [
                "St\u00e9phan Tulkens",
                "Chris Emmery",
                "Walter Daelemans"
            ],
            "abstract": "Word embeddings have recently seen a strong increase in interest as a result\nof strong performance gains on a variety of tasks. However, most of this\nresearch also underlined the importance of benchmark datasets, and the\ndifficulty of constructing these for a variety of language-specific tasks.\nStill, many of the datasets used in these tasks could prove to be fruitful\nlinguistic resources, allowing for unique observations into language use and\nvariability. In this paper we demonstrate the performance of multiple types of\nembeddings, created with both count and prediction-based architectures on a\nvariety of corpora, in two language-specific tasks: relation evaluation, and\ndialect identification. For the latter, we compare unsupervised methods with a\ntraditional, hand-crafted dictionary. With this research, we provide the\nembeddings themselves, the relation evaluation task benchmark for use in\nfurther research, and demonstrate how the benchmarked embeddings prove a useful\nunsupervised linguistic resource, effectively used in a downstream task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00359v1",
            "date": "2016-07-01 19:20:50+00:00",
            "title": "Moving Toward High Precision Dynamical Modelling in Hidden Markov Models",
            "authors": [
                "S\u00e9bastien Gagnon",
                "Jean Rouat"
            ],
            "abstract": "Hidden Markov Model (HMM) is often regarded as the dynamical model of choice\nin many fields and applications. It is also at the heart of most\nstate-of-the-art speech recognition systems since the 70's. However, from\nGaussian mixture models HMMs (GMM-HMM) to deep neural network HMMs (DNN-HMM),\nthe underlying Markovian chain of state-of-the-art models did not changed much.\nThe \"left-to-right\" topology is mostly always employed because very few other\nalternatives exist. In this paper, we propose that finely-tuned HMM topologies\nare essential for precise temporal modelling and that this approach should be\ninvestigated in state-of-the-art HMM system. As such, we propose a\nproof-of-concept framework for learning efficient topologies by pruning down\ncomplex generic models. Speech recognition experiments that were conducted\nindicate that complex time dependencies can be better learned by this approach\nthan with classical \"left-to-right\" models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00534v1",
            "date": "2016-07-02 17:17:22+00:00",
            "title": "Text comparison using word vector representations and dimensionality reduction",
            "authors": [
                "Hendrik Heuer"
            ],
            "abstract": "This paper describes a technique to compare large text sources using word\nvector representations (word2vec) and dimensionality reduction (t-SNE) and how\nit can be implemented using Python. The technique provides a bird's-eye view of\ntext sources, e.g. text summaries and their source material, and enables users\nto explore text sources like a geographical map. Word vector representations\ncapture many linguistic properties such as gender, tense, plurality and even\nsemantic concepts like \"capital city of\". Using dimensionality reduction, a 2D\nmap can be computed where semantically similar words are close to each other.\nThe technique uses the word2vec model from the gensim Python library and t-SNE\nfrom scikit-learn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00578v1",
            "date": "2016-07-03 02:18:16+00:00",
            "title": "Context-Dependent Word Representation for Neural Machine Translation",
            "authors": [
                "Heeyoul Choi",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "We first observe a potential weakness of continuous vector representations of\nsymbols in neural machine translation. That is, the continuous vector\nrepresentation, or a word embedding vector, of a symbol encodes multiple\ndimensions of similarity, equivalent to encoding more than one meaning of the\nword. This has the consequence that the encoder and decoder recurrent networks\nin neural machine translation need to spend substantial amount of their\ncapacity in disambiguating source and target words based on the context which\nis defined by a source sentence. Based on this observation, in this paper we\npropose to contextualize the word embedding vectors using a nonlinear\nbag-of-words representation of the source sentence. Additionally, we propose to\nrepresent special tokens (such as numbers, proper nouns and acronyms) with\ntyped symbols to facilitate translating those words that are not well-suited to\nbe translated via continuous vectors. The experiments on En-Fr and En-De reveal\nthat the proposed approaches of contextualization and symbolization improves\nthe translation quality of neural machine translation systems significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00718v1",
            "date": "2016-07-04 01:55:17+00:00",
            "title": "Towards Abstraction from Extraction: Multiple Timescale Gated Recurrent Unit for Summarization",
            "authors": [
                "Minsoo Kim",
                "Moirangthem Dennis Singh",
                "Minho Lee"
            ],
            "abstract": "In this work, we introduce temporal hierarchies to the sequence to sequence\n(seq2seq) model to tackle the problem of abstractive summarization of\nscientific articles. The proposed Multiple Timescale model of the Gated\nRecurrent Unit (MTGRU) is implemented in the encoder-decoder setting to better\ndeal with the presence of multiple compositionalities in larger texts. The\nproposed model is compared to the conventional RNN encoder-decoder, and the\nresults demonstrate that our model trains faster and shows significant\nperformance gains. The results also show that the temporal hierarchies help\nimprove the ability of seq2seq models to capture compositionalities better\nwithout the presence of highly complex architectural hierarchies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01133v1",
            "date": "2016-07-05 07:31:22+00:00",
            "title": "Learning when to trust distant supervision: An application to low-resource POS tagging using cross-lingual projection",
            "authors": [
                "Meng Fang",
                "Trevor Cohn"
            ],
            "abstract": "Cross lingual projection of linguistic annotation suffers from many sources\nof bias and noise, leading to unreliable annotations that cannot be used\ndirectly. In this paper, we introduce a novel approach to sequence tagging that\nlearns to correct the errors from cross-lingual projection using an explicit\ndebiasing layer. This is framed as joint learning over two corpora, one tagged\nwith gold standard and the other with projected tags. We evaluated with only\n1,000 tokens tagged with gold standard tags, along with more plentiful parallel\ndata. Our system equals or exceeds the state-of-the-art on eight simulated\nlow-resource settings, as well as two real low-resource languages, Malagasy and\nKinyarwanda.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01149v1",
            "date": "2016-07-05 08:51:21+00:00",
            "title": "Target-Side Context for Discriminative Models in Statistical Machine Translation",
            "authors": [
                "Ale\u0161 Tamchyna",
                "Alexander Fraser",
                "Ond\u0159ej Bojar",
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "Discriminative translation models utilizing source context have been shown to\nhelp statistical machine translation performance. We propose a novel extension\nof this work using target context information. Surprisingly, we show that this\nmodel can be efficiently integrated directly in the decoding process. Our\napproach scales to large training data sizes and results in consistent\nimprovements in translation quality on four language pairs. We also provide an\nanalysis comparing the strengths of the baseline source-context model with our\nextended source-context and target-context model and we show that our extension\nallows us to better capture morphological coherence. Our work is freely\navailable as part of Moses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01426v3",
            "date": "2016-07-05 21:59:04+00:00",
            "title": "Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks",
            "authors": [
                "Rajarshi Das",
                "Arvind Neelakantan",
                "David Belanger",
                "Andrew McCallum"
            ],
            "abstract": "Our goal is to combine the rich multistep inference of symbolic logical\nreasoning with the generalization capabilities of neural networks. We are\nparticularly interested in complex reasoning about entities and relations in\ntext and large-scale knowledge bases (KBs). Neelakantan et al. (2015) use RNNs\nto compose the distributed semantics of multi-hop paths in KBs; however for\nmultiple reasons, the approach lacks accuracy and practicality. This paper\nproposes three significant modeling advances: (1) we learn to jointly reason\nabout relations, entities, and entity-types; (2) we use neural attention\nmodeling to incorporate multiple paths; (3) we learn to share strength in a\nsingle RNN that represents logical composition across all relations. On a\nlargescale Freebase+ClueWeb prediction task, we achieve 25% error reduction,\nand a 53% error reduction on sparse relations due to shared strength. On chains\nof reasoning in WordNet we reduce error in mean quantile by 84% versus previous\nstate-of-the-art. The code and data are available at\nhttps://rajarshd.github.io/ChainsofReasoning",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01432v2",
            "date": "2016-07-05 22:25:10+00:00",
            "title": "Global Neural CCG Parsing with Optimality Guarantees",
            "authors": [
                "Kenton Lee",
                "Mike Lewis",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce the first global recursive neural parsing model with optimality\nguarantees during decoding. To support global features, we give up dynamic\nprograms and instead search directly in the space of all possible subtrees.\nAlthough this space is exponentially large in the sentence length, we show it\nis possible to learn an efficient A* parser. We augment existing parsing\nmodels, which have informative bounds on the outside score, with a global model\nthat has loose bounds but only needs to model non-local phenomena. The global\nmodel is trained with a new objective that encourages the parser to explore a\ntiny fraction of the search space. The approach is applied to CCG parsing,\nimproving state-of-the-art accuracy by 0.4 F1. The parser finds the optimal\nparse for 99.9% of held-out sentences, exploring on average only 190 subtrees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01485v1",
            "date": "2016-07-06 06:25:07+00:00",
            "title": "Extracting Formal Models from Normative Texts",
            "authors": [
                "John J. Camilleri",
                "Normunds Gruzitis",
                "Gerardo Schneider"
            ],
            "abstract": "Normative texts are documents based on the deontic notions of obligation,\npermission, and prohibition. Our goal is to model such texts using the C-O\nDiagram formalism, making them amenable to formal analysis, in particular\nverifying that a text satisfies properties concerning causality of actions and\ntiming constraints. We present an experimental, semi-automatic aid to bridge\nthe gap between a normative text and its formal representation. Our approach\nuses dependency trees combined with our own rules and heuristics for extracting\nthe relevant components. The resulting tabular data can then be converted into\na C-O Diagram.",
            "journal ref": "Natural Language Processing and Information Systems, Lecture Notes\n  in Computer Science, Vol. 9612, Springer, 2016, pp. 403-408",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01759v3",
            "date": "2016-07-06 19:40:15+00:00",
            "title": "Bag of Tricks for Efficient Text Classification",
            "authors": [
                "Armand Joulin",
                "Edouard Grave",
                "Piotr Bojanowski",
                "Tomas Mikolov"
            ],
            "abstract": "This paper explores a simple and efficient baseline for text classification.\nOur experiments show that our fast text classifier fastText is often on par\nwith deep learning classifiers in terms of accuracy, and many orders of\nmagnitude faster for training and evaluation. We can train fastText on more\nthan one billion words in less than ten minutes using a standard multicore~CPU,\nand classify half a million sentences among~312K classes in less than a minute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01856v1",
            "date": "2016-07-07 02:25:57+00:00",
            "title": "Neural Name Translation Improves Neural Machine Translation",
            "authors": [
                "Xiaoqing Li",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "In order to control computational complexity, neural machine translation\n(NMT) systems convert all rare words outside the vocabulary into a single unk\nsymbol. Previous solution (Luong et al., 2015) resorts to use multiple numbered\nunks to learn the correspondence between source and target rare words. However,\ntesting words unseen in the training corpus cannot be handled by this method.\nAnd it also suffers from the noisy word alignment. In this paper, we focus on a\nmajor type of rare words -- named entity (NE), and propose to translate them\nwith character level sequence to sequence model. The NE translation model is\nfurther used to derive high quality NE alignment in the bilingual training\ncorpus. With the integration of NE translation and alignment modules, our NMT\nsystem is able to surpass the baseline system by 2.9 BLEU points on the Chinese\nto English task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02789v1",
            "date": "2016-07-10 21:59:19+00:00",
            "title": "Charagram: Embedding Words and Sentences via Character n-grams",
            "authors": [
                "John Wieting",
                "Mohit Bansal",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We present Charagram embeddings, a simple approach for learning\ncharacter-based compositional models to embed textual sequences. A word or\nsentence is represented using a character n-gram count vector, followed by a\nsingle nonlinear transformation to yield a low-dimensional embedding. We use\nthree tasks for evaluation: word similarity, sentence similarity, and\npart-of-speech tagging. We demonstrate that Charagram embeddings outperform\nmore complex architectures based on character-level recurrent and convolutional\nneural networks, achieving new state-of-the-art performance on several\nsimilarity tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02791v1",
            "date": "2016-07-10 22:25:43+00:00",
            "title": "Syntactic Phylogenetic Trees",
            "authors": [
                "Kevin Shu",
                "Sharjeel Aziz",
                "Vy-Luan Huynh",
                "David Warrick",
                "Matilde Marcolli"
            ],
            "abstract": "In this paper we identify several serious problems that arise in the use of\nsyntactic data from the SSWL database for the purpose of computational\nphylogenetic reconstruction. We show that the most naive approach fails to\nproduce reliable linguistic phylogenetic trees. We identify some of the sources\nof the observed problems and we discuss how they may be, at least partly,\ncorrected by using additional information, such as prior subdivision into\nlanguage families and subfamilies, and a better use of the information about\nancient languages. We also describe how the use of phylogenetic algebraic\ngeometry can help in estimating to what extent the probability distribution at\nthe leaves of the phylogenetic tree obtained from the SSWL data can be\nconsidered reliable, by testing it on phylogenetic trees established by other\nforms of linguistic analysis. In simple examples, we find that, after\nrestricting to smaller language subfamilies and considering only those SSWL\nparameters that are fully mapped for the whole subfamily, the SSWL data match\nextremely well reliable phylogenetic trees, according to the evaluation of\nphylogenetic invariants. This is a promising sign for the use of SSWL data for\nlinguistic phylogenetics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02810v4",
            "date": "2016-07-11 02:46:48+00:00",
            "title": "The Benefits of Word Embeddings Features for Active Learning in Clinical Information Extraction",
            "authors": [
                "Mahnoosh Kholghi",
                "Lance De Vine",
                "Laurianne Sitbon",
                "Guido Zuccon",
                "Anthony Nguyen"
            ],
            "abstract": "This study investigates the use of unsupervised word embeddings and sequence\nfeatures for sample representation in an active learning framework built to\nextract clinical concepts from clinical free text. The objective is to further\nreduce the manual annotation effort while achieving higher effectiveness\ncompared to a set of baseline features. Unsupervised features are derived from\nskip-gram word embeddings and a sequence representation approach. The\ncomparative performance of unsupervised features and baseline hand-crafted\nfeatures in an active learning framework are investigated using a wide range of\nselection criteria including least confidence, information diversity,\ninformation density and diversity, and domain knowledge informativeness. Two\nclinical datasets are used for evaluation: the i2b2/VA 2010 NLP challenge and\nthe ShARe/CLEF 2013 eHealth Evaluation Lab. Our results demonstrate significant\nimprovements in terms of effectiveness as well as annotation effort savings\nacross both datasets. Using unsupervised features along with baseline features\nfor sample representation lead to further savings of up to 9% and 10% of the\ntoken and concept annotation rates, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03542v2",
            "date": "2016-07-12 23:13:26+00:00",
            "title": "Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge",
            "authors": [
                "Matt Gardner",
                "Jayant Krishnamurthy"
            ],
            "abstract": "Traditional semantic parsers map language onto compositional, executable\nqueries in a fixed schema. This mapping allows them to effectively leverage the\ninformation contained in large, formal knowledge bases (KBs, e.g., Freebase) to\nanswer questions, but it is also fundamentally limiting---these semantic\nparsers can only assign meaning to language that falls within the KB's\nmanually-produced schema. Recently proposed methods for open vocabulary\nsemantic parsing overcome this limitation by learning execution models for\narbitrary language, essentially using a text corpus as a kind of knowledge\nbase. However, all prior approaches to open vocabulary semantic parsing replace\na formal KB with textual information, making no use of the KB in their models.\nWe show how to combine the disparate representations used by these two\napproaches, presenting for the first time a semantic parser that (1) produces\ncompositional, executable representations of language, (2) can successfully\nleverage the information contained in both a formal KB and a large corpus, and\n(3) is not limited to the schema of the underlying KB. We demonstrate\nsignificantly improved performance over state-of-the-art baselines on an\nopen-domain natural language question answering task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04853v2",
            "date": "2016-07-17 11:15:38+00:00",
            "title": "An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks",
            "authors": [
                "Anirban Laha",
                "Vikas Raykar"
            ],
            "abstract": "Several tasks in argumentation mining and debating, question-answering, and\nnatural language inference involve classifying a sequence in the context of\nanother sequence (referred as bi-sequence classification). For several single\nsequence classification tasks, the current state-of-the-art approaches are\nbased on recurrent and convolutional neural networks. On the other hand, for\nbi-sequence classification problems, there is not much understanding as to the\nbest deep learning architecture. In this paper, we attempt to get an\nunderstanding of this category of problems by extensive empirical evaluation of\n19 different deep learning architectures (specifically on different ways of\nhandling context) for various problems originating in natural language\nprocessing like debating, textual entailment and question-answering. Following\nthe empirical evaluation, we offer our insights and conclusions regarding the\narchitectures we have considered. We also establish the first deep learning\nbaselines for three argumentation mining tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04982v2",
            "date": "2016-07-18 09:04:31+00:00",
            "title": "Dependency Language Models for Transition-based Dependency Parsing",
            "authors": [
                "Juntao Yu",
                "Bernd Bohnet"
            ],
            "abstract": "In this paper, we present an approach to improve the accuracy of a strong\ntransition-based dependency parser by exploiting dependency language models\nthat are extracted from a large parsed corpus. We integrated a small number of\nfeatures based on the dependency language models into the parser. To\ndemonstrate the effectiveness of the proposed approach, we evaluate our parser\non standard English and Chinese data where the base parser could achieve\ncompetitive accuracy scores. Our enhanced parser achieved state-of-the-art\naccuracy on Chinese data and competitive results on English data. We gained a\nlarge absolute improvement of one point (UAS) on Chinese and 0.5 points for\nEnglish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05014v2",
            "date": "2016-07-18 10:59:39+00:00",
            "title": "Language classification from bilingual word embedding graphs",
            "authors": [
                "Steffen Eger",
                "Armin Hoenen",
                "Alexander Mehler"
            ],
            "abstract": "We study the role of the second language in bilingual word embeddings in\nmonolingual semantic evaluation tasks. We find strongly and weakly positive\ncorrelations between down-stream task performance and second language\nsimilarity to the target language. Additionally, we show how bilingual word\nembeddings can be employed for the task of semantic language classification and\nthat joint semantic spaces vary in meaningful ways across second languages. Our\nresults support the hypothesis that semantic language similarity is influenced\nby both structural similarity as well as geography/contact.",
            "journal ref": "COLING 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05142v1",
            "date": "2016-07-18 15:51:11+00:00",
            "title": "Joint Event Detection and Entity Resolution: a Virtuous Cycle",
            "authors": [
                "Matthias Galle",
                "Jean-Michel Renders",
                "Guillaume Jacquet"
            ],
            "abstract": "Clustering web documents has numerous applications, such as aggregating news\narticles into meaningful events, detecting trends and hot topics on the Web,\npreserving diversity in search results, etc. At the same time, the importance\nof named entities and, in particular, the ability to recognize them and to\nsolve the associated co-reference resolution problem are widely recognized as\nkey enabling factors when mining, aggregating and comparing content on the Web.\n  Instead of considering these two problems separately, we propose in this\npaper a method that tackles jointly the problem of clustering news articles\ninto events and cross-document co-reference resolution of named entities. The\nco-occurrence of named entities in the same clusters is used as an additional\nsignal to decide whether two referents should be merged into one entity. These\nrefined entities can in turn be used as enhanced features to re-cluster the\ndocuments and then be refined again, entering into a virtuous cycle that\nimproves simultaneously the performances of both tasks. We implemented a\nprototype system and report results using the TDT5 collection of news articles,\ndemonstrating the potential of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05368v1",
            "date": "2016-07-19 01:55:55+00:00",
            "title": "An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation",
            "authors": [
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "Recently, Le and Mikolov (2014) proposed doc2vec as an extension to word2vec\n(Mikolov et al., 2013a) to learn document-level embeddings. Despite promising\nresults in the original paper, others have struggled to reproduce those\nresults. This paper presents a rigorous empirical evaluation of doc2vec over\ntwo tasks. We compare doc2vec to two baselines and two state-of-the-art\ndocument embedding methodologies. We found that doc2vec performs robustly when\nusing models trained on large external corpora, and can be further improved by\nusing pre-trained word embeddings. We also provide recommendations on\nhyper-parameter settings for general purpose applications, and release source\ncode to induce document embeddings using our trained doc2vec models.",
            "journal ref": "Proceedings of the 1st Workshop on Representation Learning for\n  NLP, Berlin, Germany, pp. 78--86",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05408v1",
            "date": "2016-07-19 05:38:58+00:00",
            "title": "Discriminating between similar languages in Twitter using label propagation",
            "authors": [
                "Will Radford",
                "Matthias Galle"
            ],
            "abstract": "Identifying the language of social media messages is an important first step\nin linguistic processing. Existing models for Twitter focus on content\nanalysis, which is successful for dissimilar language pairs. We propose a label\npropagation approach that takes the social graph of tweet authors into account\nas well as content to better tease apart similar languages. This results in\nstate-of-the-art shared task performance of $76.63\\%$, $1.4\\%$ higher than the\ntop system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05755v4",
            "date": "2016-07-10 19:14:00+00:00",
            "title": "A New Bengali Readability Score",
            "authors": [
                "Shanta Phani",
                "Shibamouli Lahiri",
                "Arindam Biswas"
            ],
            "abstract": "In this paper we have proposed methods to analyze the readability of Bengali\nlanguage texts. We have got some exceptionally good results out of the\nexperiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05818v1",
            "date": "2016-07-20 04:22:50+00:00",
            "title": "An Adaptation of Topic Modeling to Sentences",
            "authors": [
                "Ruey-Cheng Chen",
                "Reid Swanson",
                "Andrew S. Gordon"
            ],
            "abstract": "Advances in topic modeling have yielded effective methods for characterizing\nthe latent semantics of textual data. However, applying standard topic modeling\napproaches to sentence-level tasks introduces a number of challenges. In this\npaper, we adapt the approach of latent-Dirichlet allocation to include an\nadditional layer for incorporating information about the sentence boundaries in\ndocuments. We show that the addition of this minimal information of document\nstructure improves the perplexity results of a trained model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05822v2",
            "date": "2016-07-20 04:38:01+00:00",
            "title": "Incremental Learning for Fully Unsupervised Word Segmentation Using Penalized Likelihood and Model Selection",
            "authors": [
                "Ruey-Cheng Chen"
            ],
            "abstract": "We present a novel incremental learning approach for unsupervised word\nsegmentation that combines features from probabilistic modeling and model\nselection. This includes super-additive penalties for addressing the cognitive\nburden imposed by long word formation, and new model selection criteria based\non higher-order generative assumptions. Our approach is fully unsupervised; it\nrelies on a small number of parameters that permits flexible modeling and a\nmechanism that automatically learns parameters from the data. Through\nexperimentation, we show that this intricate design has led to top-tier\nperformance in both phonemic and orthographic word segmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06208v1",
            "date": "2016-07-21 06:49:02+00:00",
            "title": "Exploring phrase-compositionality in skip-gram models",
            "authors": [
                "Xiaochang Peng",
                "Daniel Gildea"
            ],
            "abstract": "In this paper, we introduce a variation of the skip-gram model which jointly\nlearns distributed word vector representations and their way of composing to\nform phrase embeddings. In particular, we propose a learning procedure that\nincorporates a phrase-compositionality function which can capture how we want\nto compose phrases vectors from their component word vectors. Our experiments\nshow improvement in word and phrase similarity tasks as well as syntactic tasks\nlike dependency parsing using the proposed joint models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06221v2",
            "date": "2016-07-21 07:48:08+00:00",
            "title": "A Perspective on Sentiment Analysis",
            "authors": [
                "K Paramesha",
                "K C Ravishankar"
            ],
            "abstract": "Sentiment Analysis (SA) is indeed a fascinating area of research which has\nstolen the attention of researchers as it has many facets and more importantly\nit promises economic stakes in the corporate and governance sector. SA has been\nstemmed out of text analytics and established itself as a separate identity and\na domain of research. The wide ranging results of SA have proved to influence\nthe way some critical decisions are taken. Hence, it has become relevant in\nthorough understanding of the different dimensions of the input, output and the\nprocesses and approaches of SA.",
            "journal ref": "Proceedings of ERCICA 2014 - Emerging Research in Computing\n  Information Communication and Applications, Vol. 1, Elsevier, NMIT,\n  Bengaluru,India, 2014, pp. 412-418",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06299v1",
            "date": "2016-07-21 12:43:21+00:00",
            "title": "Opinion Mining in Online Reviews About Distance Education Programs",
            "authors": [
                "Janik Jaskolski",
                "Fabian Siegberg",
                "Thomas Tibroni",
                "Philipp Cimiano",
                "Roman Klinger"
            ],
            "abstract": "The popularity of distance education programs is increasing at a fast pace.\nEn par with this development, online communication in fora, social media and\nreviewing platforms between students is increasing as well. Exploiting this\ninformation to support fellow students or institutions requires to extract the\nrelevant opinions in order to automatically generate reports providing an\noverview of pros and cons of different distance education programs. We report\non an experiment involving distance education experts with the goal to develop\na dataset of reviews annotated with relevant categories and aspects in each\ncategory discussed in the specific review together with an indication of the\nsentiment.\n  Based on this experiment, we present an approach to extract general\ncategories and specific aspects under discussion in a review together with\ntheir sentiment. We frame this task as a multi-label hierarchical text\nclassification problem and empirically investigate the performance of different\nclassification architectures to couple the prediction of a category with the\nprediction of particular aspects in this category. We evaluate different\narchitectures and show that a hierarchical approach leads to superior results\nin comparison to a flat model which makes decisions independently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06330v1",
            "date": "2016-06-01 09:39:12+00:00",
            "title": "La representaci\u00f3n de la variaci\u00f3n contextual mediante definiciones terminol\u00f3gicas flexibles",
            "authors": [
                "Antonio San Mart\u00edn"
            ],
            "abstract": "In this doctoral thesis, we apply premises of cognitive linguistics to\nterminological definitions and present a proposal called the flexible\nterminological definition. This consists of a set of definitions of the same\nconcept made up of a general definition (in this case, one encompassing the\nentire environmental domain) along with additional definitions describing the\nconcept from the perspective of the subdomains in which it is relevant. Since\ncontext is a determining factor in the construction of the meaning of lexical\nunits (including terms), we assume that terminological definitions can, and\nshould, reflect the effects of context, even though definitions have\ntraditionally been treated as the expression of meaning void of any contextual\neffect. The main objective of this thesis is to analyze the effects of\ncontextual variation on specialized environmental concepts with a view to their\nrepresentation in terminological definitions. Specifically, we focused on\ncontextual variation based on thematic restrictions. To accomplish the\nobjectives of this doctoral thesis, we conducted an empirical study consisting\nof the analysis of a set of contextually variable concepts and the creation of\na flexible definition for two of them. As a result of the first part of our\nempirical study, we divided our notion of domain-dependent contextual variation\ninto three different phenomena: modulation, perspectivization and\nsubconceptualization. These phenomena are additive in that all concepts\nexperience modulation, some concepts also undergo perspectivization, and\nfinally, a small number of concepts are additionally subjected to\nsubconceptualization. In the second part, we applied these notions to\nterminological definitions and we presented we presented guidelines on how to\nbuild flexible definitions, from the extraction of knowledge to the actual\nwriting of the definition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06556v1",
            "date": "2016-07-22 04:21:54+00:00",
            "title": "Syntax-based Attention Model for Natural Language Inference",
            "authors": [
                "PengFei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Introducing attentional mechanism in neural network is a powerful concept,\nand has achieved impressive results in many natural language processing tasks.\nHowever, most of the existing models impose attentional distribution on a flat\ntopology, namely the entire input representation sequence. Clearly, any\nwell-formed sentence has its accompanying syntactic tree structure, which is a\nmuch rich topology. Applying attention to such topology not only exploits the\nunderlying syntax, but also makes attention more interpretable. In this paper,\nwe explore this direction in the context of natural language inference. The\nresults demonstrate its efficacy. We also perform extensive qualitative\nanalysis, deriving insights and intuitions of why and how our model works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06852v1",
            "date": "2016-07-22 22:05:20+00:00",
            "title": "CFGs-2-NLU: Sequence-to-Sequence Learning for Mapping Utterances to Semantics and Pragmatics",
            "authors": [
                "Adam James Summerville",
                "James Ryan",
                "Michael Mateas",
                "Noah Wardrip-Fruin"
            ],
            "abstract": "In this paper, we present a novel approach to natural language understanding\nthat utilizes context-free grammars (CFGs) in conjunction with\nsequence-to-sequence (seq2seq) deep learning. Specifically, we take a CFG\nauthored to generate dialogue for our target application for NLU, a videogame,\nand train a long short-term memory (LSTM) recurrent neural network (RNN) to map\nthe surface utterances that it produces to traces of the grammatical expansions\nthat yielded them. Critically, this CFG was authored using a tool we have\ndeveloped that supports arbitrary annotation of the nonterminal symbols in the\ngrammar. Because we already annotated the symbols in this grammar for the\nsemantic and pragmatic considerations that our game's dialogue manager operates\nover, we can use the grammatical trace associated with any surface utterance to\ninfer such information. During gameplay, we translate player utterances into\ngrammatical traces (using our RNN), collect the mark-up attributed to the\nsymbols included in that trace, and pass this information to the dialogue\nmanager, which updates the conversation state accordingly. From an offline\nevaluation task, we demonstrate that our trained RNN translates surface\nutterances to grammatical traces with great accuracy. To our knowledge, this is\nthe first usage of seq2seq learning for conversational agents (our game's\ncharacters) who explicitly reason over semantic and pragmatic considerations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06952v1",
            "date": "2016-07-23 16:22:23+00:00",
            "title": "Neural Sentence Ordering",
            "authors": [
                "Xinchi Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Sentence ordering is a general and critical task for natural language\ngeneration applications. Previous works have focused on improving its\nperformance in an external, downstream task, such as multi-document\nsummarization. Given its importance, we propose to study it as an isolated\ntask. We collect a large corpus of academic texts, and derive a data driven\napproach to learn pairwise ordering of sentences, and validate the efficacy\nwith extensive experiments. Source codes and dataset of this paper will be made\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06961v1",
            "date": "2016-07-23 19:07:53+00:00",
            "title": "Authorship attribution via network motifs identification",
            "authors": [
                "Vanessa Queiroz Marinho",
                "Graeme Hirst",
                "Diego Raphael Amancio"
            ],
            "abstract": "Concepts and methods of complex networks can be used to analyse texts at\ntheir different complexity levels. Examples of natural language processing\n(NLP) tasks studied via topological analysis of networks are keyword\nidentification, automatic extractive summarization and authorship attribution.\nEven though a myriad of network measurements have been applied to study the\nauthorship attribution problem, the use of motifs for text analysis has been\nrestricted to a few works. The goal of this paper is to apply the concept of\nmotifs, recurrent interconnection patterns, in the authorship attribution task.\nThe absolute frequencies of all thirteen directed motifs with three nodes were\nextracted from the co-occurrence networks and used as classification features.\nThe effectiveness of these features was verified with four machine learning\nmethods. The results show that motifs are able to distinguish the writing style\nof different authors. In our best scenario, 57.5% of the books were correctly\nclassified. The chance baseline for this problem is 12.5%. In addition, we have\nfound that function words play an important role in these recurrent patterns.\nTaken together, our findings suggest that motifs should be further explored in\nother related linguistic tasks.",
            "journal ref": "2016 5th Brazilian Conference on Intelligent Systems (BRACIS),\n  Recife, Brazil, 2016, pp. 355-360",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07057v3",
            "date": "2016-07-24 15:40:36+00:00",
            "title": "Latent Tree Language Model",
            "authors": [
                "Tomas Brychcin"
            ],
            "abstract": "In this paper we introduce Latent Tree Language Model (LTLM), a novel\napproach to language modeling that encodes syntax and semantics of a given\nsentence as a tree of word roles.\n  The learning phase iteratively updates the trees by moving nodes according to\nGibbs sampling. We introduce two algorithms to infer a tree for a given\nsentence. The first one is based on Gibbs sampling. It is fast, but does not\nguarantee to find the most probable tree. The second one is based on dynamic\nprogramming. It is slower, but guarantees to find the most probable tree. We\nprovide comparison of both algorithms.\n  We combine LTLM with 4-gram Modified Kneser-Ney language model via linear\ninterpolation. Our experiments with English and Czech corpora show significant\nperplexity reductions (up to 46% for English and 49% for Czech) compared with\nstandalone 4-gram Modified Kneser-Ney language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07565v1",
            "date": "2016-07-26 07:22:52+00:00",
            "title": "Grounding Dynamic Spatial Relations for Embodied (Robot) Interaction",
            "authors": [
                "Michael Spranger",
                "Jakob Suchan",
                "Mehul Bhatt",
                "Manfred Eppe"
            ],
            "abstract": "This paper presents a computational model of the processing of dynamic\nspatial relations occurring in an embodied robotic interaction setup. A\ncomplete system is introduced that allows autonomous robots to produce and\ninterpret dynamic spatial phrases (in English) given an environment of moving\nobjects. The model unites two separate research strands: computational\ncognitive semantics and on commonsense spatial representation and reasoning.\nThe model for the first time demonstrates an integration of these different\nstrands.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07630v1",
            "date": "2016-07-26 10:37:24+00:00",
            "title": "Grounded Lexicon Acquisition - Case Studies in Spatial Language",
            "authors": [
                "Michael Spranger"
            ],
            "abstract": "This paper discusses grounded acquisition experiments of increasing\ncomplexity. Humanoid robots acquire English spatial lexicons from robot tutors.\nWe identify how various spatial language systems, such as projective, absolute\nand proximal can be learned. The proposed learning mechanisms do not rely on\ndirect meaning transfer or direct access to world models of interlocutors.\nFinally, we show how multiple systems can be acquired at the same time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07657v1",
            "date": "2016-07-26 12:04:31+00:00",
            "title": "Machine Learned Resume-Job Matching Solution",
            "authors": [
                "Yiou Lin",
                "Hang Lei",
                "Prince Clement Addo",
                "Xiaoyu Li"
            ],
            "abstract": "Job search through online matching engines nowadays are very prominent and\nbeneficial to both job seekers and employers. But the solutions of traditional\nengines without understanding the semantic meanings of different resumes have\nnot kept pace with the incredible changes in machine learning techniques and\ncomputing capability. These solutions are usually driven by manual rules and\npredefined weights of keywords which lead to an inefficient and frustrating\nsearch experience. To this end, we present a machine learned solution with rich\nfeatures and deep learning methods. Our solution includes three configurable\nmodules that can be plugged with little restrictions. Namely, unsupervised\nfeature extraction, base classifiers training and ensemble method learning. In\nour solution, rather than using manual rules, machine learned methods to\nautomatically detect the semantic similarity of positions are proposed. Then\nfour competitive \"shallow\" estimators and \"deep\" estimators are selected.\nFinally, ensemble methods to bag these estimators and aggregate their\nindividual predictions to form a final prediction are verified. Experimental\nresults of over 47 thousand resumes show that our solution can significantly\nimprove the predication precision current position, salary, educational\nbackground and company scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07931v1",
            "date": "2016-07-27 01:44:54+00:00",
            "title": "Synthetic Language Generation and Model Validation in BEAST2",
            "authors": [
                "Stuart Bradley"
            ],
            "abstract": "Generating synthetic languages aids in the testing and validation of future\ncomputational linguistic models and methods. This thesis extends the BEAST2\nphylogenetic framework to add linguistic sequence generation under multiple\nmodels. The new plugin is then used to test the effects of the phenomena of\nword borrowing on the inference process under two widely used phylolinguistic\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08692v2",
            "date": "2016-07-29 06:28:32+00:00",
            "title": "A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique",
            "authors": [
                "Rui Wang",
                "Hai Zhao",
                "Sabine Ploux",
                "Bao-Liang Lu",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "Most of the existing methods for bilingual word embedding only consider\nshallow context or simple co-occurrence information. In this paper, we propose\na latent bilingual sense unit (Bilingual Sense Clique, BSC), which is derived\nfrom a maximum complete sub-graph of pointwise mutual information based graph\nover bilingual corpus. In this way, we treat source and target words equally\nand a separated bilingual projection processing that have to be used in most\nexisting works is not necessary any more. Several dimension reduction methods\nare evaluated to summarize the BSC-word relationship. The proposed method is\nevaluated on bilingual lexicon translation tasks and empirical results show\nthat bilingual sense embedding methods outperform existing bilingual word\nembedding methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08693v1",
            "date": "2016-07-29 06:29:37+00:00",
            "title": "Connecting Phrase based Statistical Machine Translation Adaptation",
            "authors": [
                "Rui Wang",
                "Hai Zhao",
                "Bao-Liang Lu",
                "Masao Utiyama",
                "Eiichro Sumita"
            ],
            "abstract": "Although more additional corpora are now available for Statistical Machine\nTranslation (SMT), only the ones which belong to the same or similar domains\nwith the original corpus can indeed enhance SMT performance directly. Most of\nthe existing adaptation methods focus on sentence selection. In comparison,\nphrase is a smaller and more fine grained unit for data selection, therefore we\npropose a straightforward and efficient connecting phrase based adaptation\nmethod, which is applied to both bilingual phrase pair and monolingual n-gram\nadaptation. The proposed method is evaluated on IWSLT/NIST data sets, and the\nresults show that phrase based SMT performance are significantly improved (up\nto +1.6 in comparison with phrase based SMT baseline system and +0.9 in\ncomparison with existing methods).",
            "journal ref": "It is published in COLING 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08725v2",
            "date": "2016-07-29 08:35:10+00:00",
            "title": "Cseq2seq: Cyclic Sequence-to-Sequence Learning",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su"
            ],
            "abstract": "The vanilla sequence-to-sequence learning (seq2seq) reads and encodes a\nsource sequence into a fixed-length vector only once, suffering from its\ninsufficiency in modeling structural correspondence between the source and\ntarget sequence. Instead of handling this insufficiency with a linearly\nweighted attention mechanism, in this paper, we propose to use a recurrent\nneural network (RNN) as an alternative (Cseq2seq-I). During decoding,\nCseq2seq-I cyclically feeds the previous decoding state back to the encoder as\nthe initial state of the RNN, and reencodes source representations to produce\ncontext vectors. We surprisingly find that the introduced RNN succeeds in\ndynamically detecting translationrelated source tokens according to the partial\ntarget sequence. Based on this finding, we further hypothesize that the partial\ntarget sequence can act as a feedback to improve the understanding of the\nsource sequence. To test this hypothesis, we propose cyclic\nsequence-to-sequence learning (Cseq2seq-II) which differs from the seq2seq only\nin the reintroduction of previous decoding state into the same encoder. We\nfurther perform parameter sharing on Cseq2seq-II to reduce parameter redundancy\nand enhance regularization. In particular, we share the weights of the encoder\nand decoder, and two targetside word embeddings, making Cseq2seq-II equivalent\nto a single conditional RNN model, with 31% parameters pruned but even better\nperformance. Cseq2seq-II not only preserves the simplicity of seq2seq but also\nyields comparable and promising results on machine translation tasks.\nExperiments on Chinese- English and English-German translation show that\nCseq2seq achieves significant and consistent improvements over seq2seq and is\nas competitive as the attention-based seq2seq model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08885v1",
            "date": "2016-07-29 18:22:02+00:00",
            "title": "Authorship Verification - An Approach based on Random Forest",
            "authors": [
                "Promita Maitra",
                "Souvick Ghosh",
                "Dipankar Das"
            ],
            "abstract": "Authorship attribution, being an important problem in many areas in-cluding\ninformation retrieval, computational linguistics, law and journalism etc., has\nbeen identified as a subject of increasingly research interest in the re-cent\nyears. In case of Author Identification task in PAN at CLEF 2015, the main\nfocus was given on cross-genre and cross-topic author verification tasks. We\nhave used several word-based and style-based features to identify the\ndif-ferences between the known and unknown problems of one given set and label\nthe unknown ones accordingly using a Random Forest based classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00087v3",
            "date": "2016-01-01 15:42:27+00:00",
            "title": "Sentiment/Subjectivity Analysis Survey for Languages other than English",
            "authors": [
                "Mohammed Korayem",
                "Khalifeh Aljadda",
                "David Crandall"
            ],
            "abstract": "Subjective and sentiment analysis have gained considerable attention\nrecently. Most of the resources and systems built so far are done for English.\nThe need for designing systems for other languages is increasing. This paper\nsurveys different ways used for building systems for subjective and sentiment\nanalysis for languages other than English. There are three different types of\nsystems used for building these systems. The first (and the best) one is the\nlanguage specific systems. The second type of systems involves reusing or\ntransferring sentiment resources from English to the target language. The third\ntype of methods is based on using language independent methods. The paper\npresents a separate section devoted to Arabic sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00248v2",
            "date": "2016-01-03 05:47:42+00:00",
            "title": "Contrastive Entropy: A new evaluation metric for unnormalized language models",
            "authors": [
                "Kushal Arora",
                "Anand Rangarajan"
            ],
            "abstract": "Perplexity (per word) is the most widely used metric for evaluating language\nmodels. Despite this, there has been no dearth of criticism for this metric.\nMost of these criticisms center around lack of correlation with extrinsic\nmetrics like word error rate (WER), dependence upon shared vocabulary for model\ncomparison and unsuitability for unnormalized language model evaluation. In\nthis paper, we address the last problem and propose a new discriminative\nentropy based intrinsic metric that works for both traditional word level\nmodels and unnormalized language models like sentence level models. We also\npropose a discriminatively trained sentence level interpretation of recurrent\nneural network based language model (RNN) as an example of unnormalized\nsentence level model. We demonstrate that for word level models, contrastive\nentropy shows a strong correlation with perplexity. We also observe that when\ntrained at lower distortion levels, sentence level RNN considerably outperforms\ntraditional RNNs on this new metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00620v1",
            "date": "2016-01-04 19:46:00+00:00",
            "title": "Distant IE by Bootstrapping Using Lists and Document Structure",
            "authors": [
                "Lidong Bing",
                "Mingyang Ling",
                "Richard C. Wang",
                "William W. Cohen"
            ],
            "abstract": "Distant labeling for information extraction (IE) suffers from noisy training\ndata. We describe a way of reducing the noise associated with distant IE by\nidentifying coupling constraints between potential instance labels. As one\nexample of coupling, items in a list are likely to have the same label. A\nsecond example of coupling comes from analysis of document structure: in some\ncorpora, sections can be identified such that items in the same section are\nlikely to have the same label. Such sections do not exist in all corpora, but\nwe show that augmenting a large corpus with coupling constraints from even a\nsmall, well-structured corpus can improve performance substantially, doubling\nF1 on one task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00710v1",
            "date": "2016-01-05 00:49:22+00:00",
            "title": "Multi-Source Neural Translation",
            "authors": [
                "Barret Zoph",
                "Kevin Knight"
            ],
            "abstract": "We build a multi-source machine translation model and train it to maximize\nthe probability of a target English string given French and German sources.\nUsing the neural encoder-decoder framework, we explore several combination\nmethods and report up to +4.8 Bleu increases on top of a very strong\nattention-based neural translation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00893v2",
            "date": "2016-01-05 16:28:42+00:00",
            "title": "The Role of Context Types and Dimensionality in Learning Word Embeddings",
            "authors": [
                "Oren Melamud",
                "David McClosky",
                "Siddharth Patwardhan",
                "Mohit Bansal"
            ],
            "abstract": "We provide the first extensive evaluation of how using different types of\ncontext to learn skip-gram word embeddings affects performance on a wide range\nof intrinsic and extrinsic NLP tasks. Our results suggest that while intrinsic\ntasks tend to exhibit a clear preference to particular types of contexts and\nhigher dimensionality, more careful tuning is required for finding the optimal\nsettings for most of the extrinsic tasks that we considered. Furthermore, for\nthese extrinsic tasks, we find that once the benefit from increasing the\nembedding dimensionality is mostly exhausted, simple concatenation of word\nembeddings, learned with different context types, can yield further performance\ngains. As an additional contribution, we propose a new variant of the skip-gram\nmodel that learns word embeddings from weighted contexts of substitute words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01085v1",
            "date": "2016-01-06 06:03:17+00:00",
            "title": "Incorporating Structural Alignment Biases into an Attentional Neural Translation Model",
            "authors": [
                "Trevor Cohn",
                "Cong Duy Vu Hoang",
                "Ekaterina Vymolova",
                "Kaisheng Yao",
                "Chris Dyer",
                "Gholamreza Haffari"
            ],
            "abstract": "Neural encoder-decoder models of machine translation have achieved impressive\nresults, rivalling traditional translation models. However their modelling\nformulation is overly simplistic, and omits several key inductive biases built\ninto traditional models. In this paper we extend the attentional neural\ntranslation model to include structural biases from word based alignment\nmodels, including positional bias, Markov conditioning, fertility and agreement\nover translation directions. We show improvements over a baseline attentional\nmodel and standard phrase-based model over several language pairs, evaluating\non difficult languages in a low resource setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01195v1",
            "date": "2016-01-06 14:40:38+00:00",
            "title": "Part-of-Speech Tagging for Code-mixed Indian Social Media Text at ICON 2015",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "This paper discusses the experiments carried out by us at Jadavpur University\nas part of the participation in ICON 2015 task: POS Tagging for Code-mixed\nIndian Social Media Text. The tool that we have developed for the task is based\non Trigram Hidden Markov Model that utilizes information from dictionary as\nwell as some other word level features to enhance the observation probabilities\nof the known tokens as well as unknown tokens. We submitted runs for\nBengali-English, Hindi-English and Tamil-English Language pairs. Our system has\nbeen trained and tested on the datasets released for ICON 2015 shared task: POS\nTagging For Code-mixed Indian Social Media Text. In constrained mode, our\nsystem obtains average overall accuracy (averaged over all three language\npairs) of 75.60% which is very close to other participating two systems (76.79%\nfor IIITH and 75.79% for AMRITA_CEN) ranked higher than our system. In\nunconstrained mode, our system obtains average overall accuracy of 70.65% which\nis also close to the system (72.85% for AMRITA_CEN) which obtains the highest\naverage overall accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01272v2",
            "date": "2016-01-06 18:44:07+00:00",
            "title": "Recurrent Memory Networks for Language Modeling",
            "authors": [
                "Ke Tran",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "Recurrent Neural Networks (RNN) have obtained excellent result in many\nnatural language processing (NLP) tasks. However, understanding and\ninterpreting the source of this success remains a challenge. In this paper, we\npropose Recurrent Memory Network (RMN), a novel RNN architecture, that not only\namplifies the power of RNN but also facilitates our understanding of its\ninternal functioning and allows us to discover underlying patterns in data. We\ndemonstrate the power of RMN on language modeling and sentence completion\ntasks. On language modeling, RMN outperforms Long Short-Term Memory (LSTM)\nnetwork on three large German, Italian, and English dataset. Additionally we\nperform in-depth analysis of various linguistic dimensions that RMN captures.\nOn Sentence Completion Challenge, for which it is essential to capture sentence\ncoherence, our RMN obtains 69.2% accuracy, surpassing the previous\nstate-of-the-art by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01280v2",
            "date": "2016-01-06 19:13:12+00:00",
            "title": "Language to Logical Form with Neural Attention",
            "authors": [
                "Li Dong",
                "Mirella Lapata"
            ],
            "abstract": "Semantic parsing aims at mapping natural language to machine interpretable\nmeaning representations. Traditional approaches rely on high-quality lexicons,\nmanually-built templates, and linguistic features which are either domain- or\nrepresentation-specific. In this paper we present a general method based on an\nattention-enhanced encoder-decoder model. We encode input utterances into\nvector representations, and generate their logical forms by conditioning the\noutput sequences or trees on the encoding vectors. Experimental results on four\ndatasets show that our approach performs competitively without using\nhand-engineered features and is easy to adapt across domains and meaning\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01343v4",
            "date": "2016-01-06 22:19:20+00:00",
            "title": "Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation",
            "authors": [
                "Ikuya Yamada",
                "Hiroyuki Shindo",
                "Hideaki Takeda",
                "Yoshiyasu Takefuji"
            ],
            "abstract": "Named Entity Disambiguation (NED) refers to the task of resolving multiple\nnamed entity mentions in a document to their correct references in a knowledge\nbase (KB) (e.g., Wikipedia). In this paper, we propose a novel embedding method\nspecifically designed for NED. The proposed method jointly maps words and\nentities into the same continuous vector space. We extend the skip-gram model\nby using two models. The KB graph model learns the relatedness of entities\nusing the link structure of the KB, whereas the anchor context model aims to\nalign vectors such that similar words and entities occur close to one another\nin the vector space by leveraging KB anchors and their context words. By\ncombining contexts based on the proposed embedding with standard NED features,\nwe achieved state-of-the-art accuracy of 93.1% on the standard CoNLL dataset\nand 85.2% on the TAC 2010 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01530v4",
            "date": "2016-01-07 13:32:31+00:00",
            "title": "Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling",
            "authors": [
                "Gakuto Kurata",
                "Bing Xiang",
                "Bowen Zhou",
                "Mo Yu"
            ],
            "abstract": "Recurrent Neural Network (RNN) and one of its specific architectures, Long\nShort-Term Memory (LSTM), have been widely used for sequence labeling. In this\npaper, we first enhance LSTM-based sequence labeling to explicitly model label\ndependencies. Then we propose another enhancement to incorporate the global\ninformation spanning over the whole input sequence. The latter proposed method,\nencoder-labeler LSTM, first encodes the whole input sequence into a fixed\nlength vector with the encoder LSTM, and then uses this encoded vector as the\ninitial state of another LSTM for sequence labeling. Combining these methods,\nwe can predict the label sequence with considering label dependencies and\ninformation of whole input sequence. In the experiments of a slot filling task,\nwhich is an essential component of natural language understanding, with using\nthe standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02166v1",
            "date": "2016-01-09 23:34:05+00:00",
            "title": "Empirical Gaussian priors for cross-lingual transfer learning",
            "authors": [
                "Anders S\u00f8gaard"
            ],
            "abstract": "Sequence model learning algorithms typically maximize log-likelihood minus\nthe norm of the model (or minimize Hamming loss + norm). In cross-lingual\npart-of-speech (POS) tagging, our target language training data consists of\nsequences of sentences with word-by-word labels projected from translations in\n$k$ languages for which we have labeled data, via word alignments. Our training\ndata is therefore very noisy, and if Rademacher complexity is high, learning\nalgorithms are prone to overfit. Norm-based regularization assumes a constant\nwidth and zero mean prior. We instead propose to use the $k$ source language\nmodels to estimate the parameters of a Gaussian prior for learning new POS\ntaggers. This leads to significantly better performance in multi-source\ntransfer set-ups. We also present a drop-out version that injects (empirical)\nGaussian noise during online learning. Finally, we note that using empirical\nGaussian priors leads to much lower Rademacher complexity, and is superior to\noptimally weighted model interpolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02403v5",
            "date": "2016-01-11 11:28:49+00:00",
            "title": "Argumentation Mining in User-Generated Web Discourse",
            "authors": [
                "Ivan Habernal",
                "Iryna Gurevych"
            ],
            "abstract": "The goal of argumentation mining, an evolving research field in computational\nlinguistics, is to design methods capable of analyzing people's argumentation.\nIn this article, we go beyond the state of the art in several ways. (i) We deal\nwith actual Web data and take up the challenges given by the variety of\nregisters, multiple domains, and unrestricted noisy user-generated Web\ndiscourse. (ii) We bridge the gap between normative argumentation theories and\nargumentation phenomena encountered in actual data by adapting an argumentation\nmodel tested in an extensive annotation study. (iii) We create a new gold\nstandard corpus (90k tokens in 340 documents) and experiment with several\nmachine learning methods to identify argument components. We offer the data,\nsource codes, and annotation guidelines to the community under free licenses.\nOur findings show that argumentation mining in user-generated Web discourse is\na feasible but challenging task.",
            "journal ref": "Computational Linguistics 43(1), 2017, pp. 125-179",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02431v1",
            "date": "2016-01-11 13:02:59+00:00",
            "title": "The Effects of Age, Gender and Region on Non-standard Linguistic Variation in Online Social Networks",
            "authors": [
                "Claudia Peersman",
                "Walter Daelemans",
                "Reinhild Vandekerckhove",
                "Bram Vandekerckhove",
                "Leona Van Vaerenbergh"
            ],
            "abstract": "We present a corpus-based analysis of the effects of age, gender and region\nof origin on the production of both \"netspeak\" or \"chatspeak\" features and\nregional speech features in Flemish Dutch posts that were collected from a\nBelgian online social network platform. The present study shows that combining\nquantitative and qualitative approaches is essential for understanding\nnon-standard linguistic variation in a CMC corpus. It also presents a\nmethodology that enables the systematic study of this variation by including\nall non-standard words in the corpus. The analyses resulted in a convincing\nillustration of the Adolescent Peak Principle. In addition, our approach\nrevealed an intriguing correlation between the use of regional speech features\nand chatspeak features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02502v1",
            "date": "2016-01-11 16:12:32+00:00",
            "title": "Trans-gram, Fast Cross-lingual Word-embeddings",
            "authors": [
                "Jocelyn Coulmance",
                "Jean-Marc Marty",
                "Guillaume Wenzek",
                "Amine Benhalloum"
            ],
            "abstract": "We introduce Trans-gram, a simple and computationally-efficient method to\nsimultaneously learn and align wordembeddings for a variety of languages, using\nonly monolingual data and a smaller set of sentence-aligned data. We use our\nnew method to compute aligned wordembeddings for twenty-one languages using\nEnglish as a pivot language. We show that some linguistic features are aligned\nacross languages for which we do not have aligned data, even though those\nproperties do not exist in the pivot language. We also achieve state of the art\nresults on standard cross-lingual text classification and word translation\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02553v2",
            "date": "2016-01-11 18:38:18+00:00",
            "title": "Environmental Noise Embeddings for Robust Speech Recognition",
            "authors": [
                "Suyoun Kim",
                "Bhiksha Raj",
                "Ian Lane"
            ],
            "abstract": "We propose a novel deep neural network architecture for speech recognition\nthat explicitly employs knowledge of the background environmental noise within\na deep neural network acoustic model. A deep neural network is used to predict\nthe acoustic environment in which the system in being used. The discriminative\nembedding generated at the bottleneck layer of this network is then\nconcatenated with traditional acoustic features as input to a deep neural\nnetwork acoustic model. Through a series of experiments on Resource Management,\nCHiME-3 task, and Aurora4, we show that the proposed approach significantly\nimproves speech recognition accuracy in noisy and highly reverberant\nenvironments, outperforming multi-condition training, noise-aware training,\ni-vector framework, and multi-task learning on both in-domain noise and unseen\nnoise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03288v1",
            "date": "2016-01-13 15:55:36+00:00",
            "title": "Predicting the Effectiveness of Self-Training: Application to Sentiment Classification",
            "authors": [
                "Vincent Van Asch",
                "Walter Daelemans"
            ],
            "abstract": "The goal of this paper is to investigate the connection between the\nperformance gain that can be obtained by selftraining and the similarity\nbetween the corpora used in this approach. Self-training is a semi-supervised\ntechnique designed to increase the performance of machine learning algorithms\nby automatically classifying instances of a task and adding these as additional\ntraining material to the same classifier. In the context of language processing\ntasks, this training material is mostly an (annotated) corpus. Unfortunately\nself-training does not always lead to a performance increase and whether it\nwill is largely unpredictable. We show that the similarity between corpora can\nbe used to identify those setups for which self-training can be beneficial. We\nconsider this research as a step in the process of developing a classifier that\nis able to adapt itself to each new test corpus that it is presented with.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03313v2",
            "date": "2016-01-13 16:58:05+00:00",
            "title": "Political Speech Generation",
            "authors": [
                "Valentin Kassarnig"
            ],
            "abstract": "In this report we present a system that can generate political speeches for a\ndesired political party. Furthermore, the system allows to specify whether a\nspeech should hold a supportive or opposing opinion. The system relies on a\ncombination of several state-of-the-art NLP methods which are discussed in this\nreport. These include n-grams, Justeson & Katz POS tag filter, recurrent neural\nnetworks, and latent Dirichlet allocation. Sequences of words are generated\nbased on probabilities obtained from two underlying models: A language model\ntakes care of the grammatical correctness while a topic model aims for textual\nconsistency. Both models were trained on the Convote dataset which contains\ntranscripts from US congressional floor debates. Furthermore, we present a\nmanual and an automated approach to evaluate the quality of generated speeches.\nIn an experimental evaluation generated speeches have shown very high quality\nin terms of grammatical correctness and sentence transitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03317v3",
            "date": "2016-01-13 17:14:01+00:00",
            "title": "Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model",
            "authors": [
                "Shi Feng",
                "Shujie Liu",
                "Mu Li",
                "Ming Zhou"
            ],
            "abstract": "Neural machine translation has shown very promising results lately. Most NMT\nmodels follow the encoder-decoder framework. To make encoder-decoder models\nmore flexible, attention mechanism was introduced to machine translation and\nalso other tasks like speech recognition and image captioning. We observe that\nthe quality of translation by attention-based encoder-decoder can be\nsignificantly damaged when the alignment is incorrect. We attribute these\nproblems to the lack of distortion and fertility models. Aiming to resolve\nthese problems, we propose new variations of attention-based encoder-decoder\nand compare them with other models on machine translation. Our proposed method\nachieved an improvement of 2 BLEU points over the original attention-based\nencoder-decoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03348v1",
            "date": "2016-01-13 18:59:06+00:00",
            "title": "EvoGrader: an online formative assessment tool for automatically evaluating written evolutionary explanations",
            "authors": [
                "Kayhan Moharreri",
                "Minsu Ha",
                "Ross H Nehm"
            ],
            "abstract": "EvoGrader is a free, online, on-demand formative assessment service designed\nfor use in undergraduate biology classrooms. EvoGrader's web portal is powered\nby Amazon's Elastic Cloud and run with LightSIDE Lab's open-source\nmachine-learning tools. The EvoGrader web portal allows biology instructors to\nupload a response file (.csv) containing unlimited numbers of evolutionary\nexplanations written in response to 86 different ACORNS (Assessing COntextual\nReasoning about Natural Selection) instrument items. The system automatically\nanalyzes the responses and provides detailed information about the scientific\nand naive concepts contained within each student's response, as well as overall\nstudent (and sample) reasoning model types. Graphs and visual models provided\nby EvoGrader summarize class-level responses; downloadable files of raw scores\n(in .csv format) are also provided for more detailed analyses. Although the\ncomputational machinery that EvoGrader employs is complex, using the system is\neasy. Users only need to know how to use spreadsheets to organize student\nresponses, upload files to the web, and use a web browser. A series of\nexperiments using new samples of 2,200 written evolutionary explanations\ndemonstrate that EvoGrader scores are comparable to those of trained human\nraters, although EvoGrader scoring takes 99% less time and is free. EvoGrader\nwill be of interest to biology instructors teaching large classes who seek to\nemphasize scientific practices such as generating scientific explanations, and\nto teach crosscutting ideas such as evolution and natural selection. The\nsoftware architecture of EvoGrader is described as it may serve as a template\nfor developing machine-learning portals for other core concepts within biology\nand across other disciplines.",
            "journal ref": "Evolution: Education and Outreach, vol. 7, pp. 1-14, 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03650v4",
            "date": "2016-01-14 16:30:09+00:00",
            "title": "Smoothing parameter estimation framework for IBM word alignment models",
            "authors": [
                "Vuong Van Bui",
                "Cuong Anh Le"
            ],
            "abstract": "IBM models are very important word alignment models in Machine Translation.\nFollowing the Maximum Likelihood Estimation principle to estimate their\nparameters, the models will easily overfit the training data when the data are\nsparse. While smoothing is a very popular solution in Language Model, there\nstill lacks studies on smoothing for word alignment. In this paper, we propose\na framework which generalizes the notable work Moore [2004] of applying\nadditive smoothing to word alignment models. The framework allows developers to\ncustomize the smoothing amount for each pair of word. The added amount will be\nscaled appropriately by a common factor which reflects how much the framework\ntrusts the adding strategy according to the performance on data. We also\ncarefully examine various performance criteria and propose a smoothened version\nof the error count, which generally gives the best result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03783v1",
            "date": "2016-01-15 00:09:52+00:00",
            "title": "Towards Turkish ASR: Anatomy of a rule-based Turkish g2p",
            "authors": [
                "Duygu Altinok"
            ],
            "abstract": "This paper describes the architecture and implementation of a rule-based\ngrapheme to phoneme converter for Turkish. The system accepts surface form as\ninput, outputs SAMPA mapping of the all parallel pronounciations according to\nthe morphological analysis together with stress positions. The system has been\nimplemented in Python",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03916v3",
            "date": "2016-01-15 13:42:04+00:00",
            "title": "Multimodal Pivots for Image Caption Translation",
            "authors": [
                "Julian Hitschler",
                "Shigehiko Schamoni",
                "Stefan Riezler"
            ],
            "abstract": "We present an approach to improve statistical machine translation of image\ndescriptions by multimodal pivots defined in visual space. The key idea is to\nperform image retrieval over a database of images that are captioned in the\ntarget language, and use the captions of the most similar images for\ncrosslingual reranking of translation outputs. Our approach does not depend on\nthe availability of large amounts of in-domain parallel data, but only relies\non available large datasets of monolingually captioned images, and on\nstate-of-the-art convolutional neural networks to compute image similarities.\nOur experimental evaluation shows improvements of 1 BLEU point over strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04012v1",
            "date": "2016-01-15 17:33:39+00:00",
            "title": "Detecting and Extracting Events from Text Documents",
            "authors": [
                "Jugal Kalita"
            ],
            "abstract": "Events of various kinds are mentioned and discussed in text documents,\nwhether they are books, news articles, blogs or microblog feeds. The paper\nstarts by giving an overview of how events are treated in linguistics and\nphilosophy. We follow this discussion by surveying how events and associated\ninformation are handled in computationally. In particular, we look at how\ntextual documents can be mined to extract events and ancillary information.\nThese days, it is mostly through the application of various machine learning\ntechniques. We also discuss applications of event detection and extraction\nsystems, particularly in summarization, in the medical domain and in the\ncontext of Twitter posts. We end the paper with a discussion of challenges and\nfuture directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04811v6",
            "date": "2016-01-19 07:09:38+00:00",
            "title": "Modeling Coverage for Neural Machine Translation",
            "authors": [
                "Zhaopeng Tu",
                "Zhengdong Lu",
                "Yang Liu",
                "Xiaohua Liu",
                "Hang Li"
            ],
            "abstract": "Attention mechanism has enhanced state-of-the-art Neural Machine Translation\n(NMT) by jointly learning to align and translate. It tends to ignore past\nalignment information, however, which often leads to over-translation and\nunder-translation. To address this problem, we propose coverage-based NMT in\nthis paper. We maintain a coverage vector to keep track of the attention\nhistory. The coverage vector is fed to the attention model to help adjust\nfuture attention, which lets NMT system to consider more about untranslated\nsource words. Experiments show that the proposed approach significantly\nimproves both translation quality and alignment quality over standard\nattention-based NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05472v1",
            "date": "2016-01-20 23:31:58+00:00",
            "title": "Hierarchical Latent Word Clustering",
            "authors": [
                "Halid Ziya Yerebakan",
                "Fitsum Reda",
                "Yiqiang Zhan",
                "Yoshihisa Shinagawa"
            ],
            "abstract": "This paper presents a new Bayesian non-parametric model by extending the\nusage of Hierarchical Dirichlet Allocation to extract tree structured word\nclusters from text data. The inference algorithm of the model collects words in\na cluster if they share similar distribution over documents. In our\nexperiments, we observed meaningful hierarchical structures on NIPS corpus and\nradiology reports collected from public repositories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05647v3",
            "date": "2016-01-21 14:15:41+00:00",
            "title": "On Structured Sparsity of Phonological Posteriors for Linguistic Parsing",
            "authors": [
                "Milos Cernak",
                "Afsaneh Asaei",
                "Herv\u00e9 Bourlard"
            ],
            "abstract": "The speech signal conveys information on different time scales from short\ntime scale or segmental, associated to phonological and phonetic information to\nlong time scale or supra segmental, associated to syllabic and prosodic\ninformation. Linguistic and neurocognitive studies recognize the phonological\nclasses at segmental level as the essential and invariant representations used\nin speech temporal organization. In the context of speech processing, a deep\nneural network (DNN) is an effective computational method to infer the\nprobability of individual phonological classes from a short segment of speech\nsignal. A vector of all phonological class probabilities is referred to as\nphonological posterior. There are only very few classes comprising a short term\nspeech signal; hence, the phonological posterior is a sparse vector. Although\nthe phonological posteriors are estimated at segmental level, we claim that\nthey convey supra-segmental information. Specifically, we demonstrate that\nphonological posteriors are indicative of syllabic and prosodic events.\nBuilding on findings from converging linguistic evidence on the gestural model\nof Articulatory Phonology as well as the neural basis of speech perception, we\nhypothesize that phonological posteriors convey properties of linguistic\nclasses at multiple time scales, and this information is embedded in their\nsupport (index) of active coefficients. To verify this hypothesis, we obtain a\nbinary representation of phonological posteriors at the segmental level which\nis referred to as first-order sparsity structure; the high-order structures are\nobtained by the concatenation of first-order binary vectors. It is then\nconfirmed that the classification of supra-segmental linguistic events, the\nproblem known as linguistic parsing, can be achieved with high accuracy using\nasimple binary pattern matching of first-order or high-order structures.",
            "journal ref": "Speech Communication, Volume 84, November 2016, Pages 36-45",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05768v1",
            "date": "2016-01-21 20:19:31+00:00",
            "title": "Syntax-Semantics Interaction Parsing Strategies. Inside SYNTAGMA",
            "authors": [
                "Daniel Christen"
            ],
            "abstract": "This paper discusses SYNTAGMA, a rule based NLP system addressing the tricky\nissues of syntactic ambiguity reduction and word sense disambiguation as well\nas providing innovative and original solutions for constituent generation and\nconstraints management. To provide an insight into how it operates, the\nsystem's general architecture and components, as well as its lexical, syntactic\nand semantic resources are described. After that, the paper addresses the\nmechanism that performs selective parsing through an interaction between\nsyntactic and semantic information, leading the parser to a coherent and\naccurate interpretation of the input text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06068v2",
            "date": "2016-01-22 16:50:22+00:00",
            "title": "Paraphrase Generation from Latent-Variable PCFGs for Semantic Parsing",
            "authors": [
                "Shashi Narayan",
                "Siva Reddy",
                "Shay B. Cohen"
            ],
            "abstract": "One of the limitations of semantic parsing approaches to open-domain question\nanswering is the lexicosyntactic gap between natural language questions and\nknowledge base entries -- there are many ways to ask a question, all with the\nsame answer. In this paper we propose to bridge this gap by generating\nparaphrases of the input question with the goal that at least one of them will\nbe correctly mapped to a knowledge-base query. We introduce a novel grammar\nmodel for paraphrase generation that does not require any sentence-aligned\nparaphrase corpus. Our key idea is to leverage the flexibility and scalability\nof latent-variable probabilistic context-free grammars to sample paraphrases.\nWe do an extrinsic evaluation of our paraphrases by plugging them into a\nsemantic parser for Freebase. Our evaluation experiments on the WebQuestions\nbenchmark dataset show that the performance of the semantic parser\nsignificantly improves over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06579v2",
            "date": "2016-01-25 12:45:59+00:00",
            "title": "A Kernel Independence Test for Geographical Language Variation",
            "authors": [
                "Dong Nguyen",
                "Jacob Eisenstein"
            ],
            "abstract": "Quantifying the degree of spatial dependence for linguistic variables is a\nkey task for analyzing dialectal variation. However, existing approaches have\nimportant drawbacks. First, they are based on parametric models of dependence,\nwhich limits their power in cases where the underlying parametric assumptions\nare violated. Second, they are not applicable to all types of linguistic data:\nsome approaches apply only to frequencies, others to boolean indicators of\nwhether a linguistic variable is present. We present a new method for measuring\ngeographical language variation, which solves both of these problems. Our\napproach builds on Reproducing Kernel Hilbert space (RKHS) representations for\nnonparametric statistics, and takes the form of a test statistic that is\ncomputed from pairs of individual geotagged observations without aggregation\ninto predefined geographical bins. We compare this test with prior work using\nsynthetic data as well as a diverse set of real datasets: a corpus of Dutch\ntweets, a Dutch syntactic atlas, and a dataset of letters to the editor in\nNorth American newspapers. Our proposed test is shown to support robust\ninferences across a broad range of scenarios and types of data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06971v3",
            "date": "2016-01-26 10:44:30+00:00",
            "title": "Sentiment Analysis of Twitter Data: A Survey of Techniques",
            "authors": [
                "Vishal. A. Kharde",
                "Prof. Sheetal. Sonawane"
            ],
            "abstract": "With the advancement of web technology and its growth, there is a huge volume\nof data present in the web for internet users and a lot of data is generated\ntoo. Internet has become a platform for online learning, exchanging ideas and\nsharing opinions. Social networking sites like Twitter, Facebook, Google+ are\nrapidly gaining popularity as they allow people to share and express their\nviews about topics,have discussion with different communities, or post messages\nacross the world. There has been lot of work in the field of sentiment analysis\nof twitter data. This survey focuses mainly on sentiment analysis of twitter\ndata which is helpful to analyze the information in the tweets where opinions\nare highly unstructured, heterogeneous and are either positive or negative, or\nneutral in some cases. In this paper, we provide a survey and a comparative\nanalyses of existing techniques for opinion mining like machine learning and\nlexicon-based approaches, together with evaluation metrics. Using various\nmachine learning algorithms like Naive Bayes, Max Entropy, and Support Vector\nMachine, we provide a research on twitter data streams.General challenges and\napplications of Sentiment Analysis on Twitter are also discussed in this paper.",
            "journal ref": "International Journal of Computer Applications 139(11): 5-15,\n  April 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.07215v1",
            "date": "2016-01-26 22:53:45+00:00",
            "title": "Recurrent Neural Network Postfilters for Statistical Parametric Speech Synthesis",
            "authors": [
                "Prasanna Kumar Muthukumar",
                "Alan W Black"
            ],
            "abstract": "In the last two years, there have been numerous papers that have looked into\nusing Deep Neural Networks to replace the acoustic model in traditional\nstatistical parametric speech synthesis. However, far less attention has been\npaid to approaches like DNN-based postfiltering where DNNs work in conjunction\nwith traditional acoustic models. In this paper, we investigate the use of\nRecurrent Neural Networks as a potential postfilter for synthesis. We explore\nthe possibility of replacing existing postfilters, as well as highlight the\nease with which arbitrary new features can be added as input to the postfilter.\nWe also tried a novel approach of jointly training the Classification And\nRegression Tree and the postfilter, rather than the traditional approach of\ntraining them independently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.07969v2",
            "date": "2016-01-29 02:39:56+00:00",
            "title": "Zipf's law is a consequence of coherent language production",
            "authors": [
                "Jake Ryland Williams",
                "James P. Bagrow",
                "Andrew J. Reagan",
                "Sharon E. Alajajian",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "The task of text segmentation may be undertaken at many levels in text\nanalysis---paragraphs, sentences, words, or even letters. Here, we focus on a\nrelatively fine scale of segmentation, hypothesizing it to be in accord with a\nstochastic model of language generation, as the smallest scale where\nindependent units of meaning are produced. Our goals in this letter include the\ndevelopment of methods for the segmentation of these minimal independent units,\nwhich produce feature-representations of texts that align with the independence\nassumption of the bag-of-terms model, commonly used for prediction and\nclassification in computational text analysis. We also propose the measurement\nof texts' association (with respect to realized segmentations) to the model of\nlanguage generation. We find (1) that our segmentations of phrases exhibit much\nbetter associations to the generation model than words and (2), that texts\nwhich are well fit are generally topically homogeneous. Because our generative\nmodel produces Zipf's law, our study further suggests that Zipf's law may be a\nconsequence of homogeneity in language production.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.00367v1",
            "date": "2016-02-01 02:53:41+00:00",
            "title": "Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers",
            "authors": [
                "Yijun Xiao",
                "Kyunghyun Cho"
            ],
            "abstract": "Document classification tasks were primarily tackled at word level. Recent\nresearch that works with character-level inputs shows several benefits over\nword-level approaches such as natural incorporation of morphemes and better\nhandling of rare words. We propose a neural network architecture that utilizes\nboth convolution and recurrent layers to efficiently encode character inputs.\nWe validate the proposed model on eight large scale document classification\ntasks and compare with character-level convolution-only models. It achieves\ncomparable performances with much less parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.00812v2",
            "date": "2016-02-02 07:35:02+00:00",
            "title": "The Grail theorem prover: Type theory for syntax and semantics",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "As the name suggests, type-logical grammars are a grammar formalism based on\nlogic and type theory. From the prespective of grammar design, type-logical\ngrammars develop the syntactic and semantic aspects of linguistic phenomena\nhand-in-hand, letting the desired semantics of an expression inform the\nsyntactic type and vice versa. Prototypical examples of the successful\napplication of type-logical grammars to the syntax-semantics interface include\ncoordination, quantifier scope and extraction.This chapter describes the Grail\ntheorem prover, a series of tools for designing and testing grammars in various\nmodern type-logical grammars which functions as a tool . All tools described in\nthis chapter are freely available.",
            "journal ref": "Modern Perspectives in Type Theoretical Semantics, Springer, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.01595v4",
            "date": "2016-02-04 08:51:18+00:00",
            "title": "Many Languages, One Parser",
            "authors": [
                "Waleed Ammar",
                "George Mulcaire",
                "Miguel Ballesteros",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We train one multilingual model for dependency parsing and use it to parse\nsentences in several languages. The parsing model uses (i) multilingual word\nclusters and embeddings; (ii) token-level language information; and (iii)\nlanguage-specific features (fine-grained POS tags). This input representation\nenables the parser not only to parse effectively in multiple languages, but\nalso to generalize across languages based on linguistic universals and\ntypological similarities, making it more effective to learn from limited\nannotations. Our parser's performance compares favorably to strong baselines in\na range of data scenarios, including when the target language has a large\ntreebank, a small treebank, or no treebank for training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.01925v2",
            "date": "2016-02-05 04:26:38+00:00",
            "title": "Massively Multilingual Word Embeddings",
            "authors": [
                "Waleed Ammar",
                "George Mulcaire",
                "Yulia Tsvetkov",
                "Guillaume Lample",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We introduce new methods for estimating and evaluating embeddings of words in\nmore than fifty languages in a single shared embedding space. Our estimation\nmethods, multiCluster and multiCCA, use dictionaries and monolingual data; they\ndo not require parallel data. Our new evaluation method, multiQVEC-CCA, is\nshown to correlate better than previous ones with two downstream tasks (text\ncategorization and parsing). We also describe a web portal for evaluation that\nwill facilitate further research in this area, along with open-source releases\nof all our methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.01929v1",
            "date": "2016-02-05 06:02:51+00:00",
            "title": "Fantastic 4 system for NIST 2015 Language Recognition Evaluation",
            "authors": [
                "Kong Aik Lee",
                "Ville Hautam\u00e4ki",
                "Anthony Larcher",
                "Wei Rao",
                "Hanwu Sun",
                "Trung Hieu Nguyen",
                "Guangsen Wang",
                "Aleksandr Sizov",
                "Ivan Kukanov",
                "Amir Poorjam",
                "Trung Ngo Trong",
                "Xiong Xiao",
                "Cheng-Lin Xu",
                "Hai-Hua Xu",
                "Bin Ma",
                "Haizhou Li",
                "Sylvain Meignier"
            ],
            "abstract": "This article describes the systems jointly submitted by Institute for\nInfocomm (I$^2$R), the Laboratoire d'Informatique de l'Universit\\'e du Maine\n(LIUM), Nanyang Technology University (NTU) and the University of Eastern\nFinland (UEF) for 2015 NIST Language Recognition Evaluation (LRE). The\nsubmitted system is a fusion of nine sub-systems based on i-vectors extracted\nfrom different types of features. Given the i-vectors, several classifiers are\nadopted for the language detection task including support vector machines\n(SVM), multi-class logistic regression (MCLR), Probabilistic Linear\nDiscriminant Analysis (PLDA) and Deep Neural Networks (DNN).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02215v1",
            "date": "2016-02-06 04:39:41+00:00",
            "title": "Swivel: Improving Embeddings by Noticing What's Missing",
            "authors": [
                "Noam Shazeer",
                "Ryan Doherty",
                "Colin Evans",
                "Chris Waterson"
            ],
            "abstract": "We present Submatrix-wise Vector Embedding Learner (Swivel), a method for\ngenerating low-dimensional feature embeddings from a feature co-occurrence\nmatrix. Swivel performs approximate factorization of the point-wise mutual\ninformation matrix via stochastic gradient descent. It uses a piecewise loss\nwith special handling for unobserved co-occurrences, and thus makes use of all\nthe information in the matrix. While this requires computation proportional to\nthe size of the entire matrix, we make use of vectorized multiplication to\nprocess thousands of rows and columns at once to compute millions of predicted\nvalues. Furthermore, we partition the matrix into shards in order to\nparallelize the computation across many nodes. This approach results in more\naccurate embeddings than can be achieved with methods that consider only\nobserved co-occurrences, and can scale to much larger corpora than can be\nhandled with sampling methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02410v2",
            "date": "2016-02-07 19:11:17+00:00",
            "title": "Exploring the Limits of Language Modeling",
            "authors": [
                "Rafal Jozefowicz",
                "Oriol Vinyals",
                "Mike Schuster",
                "Noam Shazeer",
                "Yonghui Wu"
            ],
            "abstract": "In this work we explore recent advances in Recurrent Neural Networks for\nlarge scale Language Modeling, a task central to language understanding. We\nextend current models to deal with two key challenges present in this task:\ncorpora and vocabulary sizes, and complex, long term structure of language. We\nperform an exhaustive study on techniques such as character Convolutional\nNeural Networks or Long-Short Term Memory, on the One Billion Word Benchmark.\nOur best single model significantly improves state-of-the-art perplexity from\n51.3 down to 30.0 (whilst reducing the number of parameters by a factor of 20),\nwhile an ensemble of models sets a new record by improving perplexity from 41.0\ndown to 23.7. We also release these models for the NLP and ML community to\nstudy and improve upon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03265v2",
            "date": "2016-02-10 04:54:15+00:00",
            "title": "Simple Search Algorithms on Semantic Networks Learned from Language Use",
            "authors": [
                "Aida Nematzadeh",
                "Filip Miscevic",
                "Suzanne Stevenson"
            ],
            "abstract": "Recent empirical and modeling research has focused on the semantic fluency\ntask because it is informative about semantic memory. An interesting interplay\narises between the richness of representations in semantic memory and the\ncomplexity of algorithms required to process it. It has remained an open\nquestion whether representations of words and their relations learned from\nlanguage use can enable a simple search algorithm to mimic the observed\nbehavior in the fluency task. Here we show that it is plausible to learn rich\nrepresentations from naturalistic data for which a very simple search algorithm\n(a random walk) can replicate the human patterns. We suggest that explicitly\nstructuring knowledge about words into a semantic network plays a crucial role\nin modeling human behavior in memory search and retrieval; moreover, this is\nthe case across a range of semantic information sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03426v2",
            "date": "2016-02-10 16:02:46+00:00",
            "title": "Automatic Sarcasm Detection: A Survey",
            "authors": [
                "Aditya Joshi",
                "Pushpak Bhattacharyya",
                "Mark James Carman"
            ],
            "abstract": "Automatic sarcasm detection is the task of predicting sarcasm in text. This\nis a crucial step to sentiment analysis, considering prevalence and challenges\nof sarcasm in sentiment-bearing text. Beginning with an approach that used\nspeech-based features, sarcasm detection has witnessed great interest from the\nsentiment analysis community. This paper is the first known compilation of past\nwork in automatic sarcasm detection. We observe three milestones in the\nresearch so far: semi-supervised pattern extraction to identify implicit\nsentiment, use of hashtag-based supervision, and use of context beyond target\ntext. In this paper, we describe datasets, approaches, trends and issues in\nsarcasm detection. We also discuss representative performance values, shared\ntasks and pointers to future work, as given in prior works. In terms of\nresources that could be useful for understanding state-of-the-art, the survey\npresents several useful illustrations - most prominently, a table that\nsummarizes past papers along different dimensions such as features, annotation\ntechniques, data forms, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03960v1",
            "date": "2016-02-12 03:54:43+00:00",
            "title": "TabMCQ: A Dataset of General Knowledge Tables and Multiple-choice Questions",
            "authors": [
                "Sujay Kumar Jauhar",
                "Peter Turney",
                "Eduard Hovy"
            ],
            "abstract": "We describe two new related resources that facilitate modelling of general\nknowledge reasoning in 4th grade science exams. The first is a collection of\ncurated facts in the form of tables, and the second is a large set of\ncrowd-sourced multiple-choice questions covering the facts in the tables.\nThrough the setup of the crowd-sourced annotation task we obtain implicit\nalignment information between questions and tables. We envisage that the\nresources will be useful not only to researchers working on question answering,\nbut also to people investigating a diverse range of other applications such as\ninformation extraction, question parsing, answer type identification, and\nlexical semantic modelling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.04341v1",
            "date": "2016-02-13 14:38:47+00:00",
            "title": "Attention-Based Convolutional Neural Network for Machine Comprehension",
            "authors": [
                "Wenpeng Yin",
                "Sebastian Ebert",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Understanding open-domain text is one of the primary challenges in natural\nlanguage processing (NLP). Machine comprehension benchmarks evaluate the\nsystem's ability to understand text based on the text content only. In this\nwork, we investigate machine comprehension on MCTest, a question answering (QA)\nbenchmark. Prior work is mainly based on feature engineering approaches. We\ncome up with a neural network framework, named hierarchical attention-based\nconvolutional neural network (HABCNN), to address this task without any\nmanually designed features. Specifically, we explore HABCNN for this task by\ntwo routes, one is through traditional joint modeling of passage, question and\nanswer, one is through textual entailment. HABCNN employs an attention\nmechanism to detect key phrases, key sentences and key snippets that are\nrelevant to answering the question. Experiments show that HABCNN outperforms\nprior deep learning approaches by a big margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.04427v2",
            "date": "2016-02-14 07:31:28+00:00",
            "title": "Exploiting Lists of Names for Named Entity Identification of Financial Institutions from Unstructured Documents",
            "authors": [
                "Zheng Xu",
                "Douglas Burdick",
                "Louiqa Raschid"
            ],
            "abstract": "There is a wealth of information about financial systems that is embedded in\ndocument collections. In this paper, we focus on a specialized text extraction\ntask for this domain. The objective is to extract mentions of names of\nfinancial institutions, or FI names, from financial prospectus documents, and\nto identify the corresponding real world entities, e.g., by matching against a\ncorpus of such entities. The tasks are Named Entity Recognition (NER) and\nEntity Resolution (ER); both are well studied in the literature. Our\ncontribution is to develop a rule-based approach that will exploit lists of FI\nnames for both tasks; our solution is labeled Dict-based NER and Rank-based ER.\nSince the FI names are typically represented by a root, and a suffix that\nmodifies the root, we use these lists of FI names to create specialized root\nand suffix dictionaries. To evaluate the effectiveness of our specialized\nsolution for extracting FI names, we compare Dict-based NER with a general\npurpose rule-based NER solution, ORG NER. Our evaluation highlights the\nbenefits and limitations of specialized versus general purpose approaches, and\npresents additional suggestions for tuning and customization for FI name\nextraction. To our knowledge, our proposed solutions, Dict-based NER and\nRank-based ER, and the root and suffix dictionaries, are the first attempt to\nexploit specialized knowledge, i.e., lists of FI names, for rule-based NER and\nER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05388v2",
            "date": "2016-02-17 12:29:56+00:00",
            "title": "Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages",
            "authors": [
                "Muhammad Imran",
                "Prasenjit Mitra",
                "Jaideep Srivastava"
            ],
            "abstract": "Rapid crisis response requires real-time analysis of messages. After a\ndisaster happens, volunteers attempt to classify tweets to determine needs,\ne.g., supplies, infrastructure damage, etc. Given labeled data, supervised\nmachine learning can help classify these messages. Scarcity of labeled data\ncauses poor performance in machine training. Can we reuse old tweets to train\nclassifiers? How can we choose labeled tweets for training? Specifically, we\nstudy the usefulness of labeled data of past events. Do labeled tweets in\ndifferent language help? We observe the performance of our classifiers trained\nusing different combinations of training sets obtained from past disasters. We\nperform extensive experimentation on real crisis datasets and show that the\npast labels are useful when both source and target events are of the same type\n(e.g. both earthquakes). For similar languages (e.g., Italian and Spanish),\ncross-language domain adaptation was useful, however, when for different\nlanguages (e.g., Italian and English), the performance decreased.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05772v1",
            "date": "2016-02-18 12:08:05+00:00",
            "title": "Corpus analysis without prior linguistic knowledge - unsupervised mining of phrases and subphrase structure",
            "authors": [
                "Stefan Gerdjikov",
                "Klaus U. Schulz"
            ],
            "abstract": "When looking at the structure of natural language, \"phrases\" and \"words\" are\ncentral notions. We consider the problem of identifying such \"meaningful\nsubparts\" of language of any length and underlying composition principles in a\ncompletely corpus-based and language-independent way without using any kind of\nprior linguistic knowledge. Unsupervised methods for identifying \"phrases\",\nmining subphrase structure and finding words in a fully automated way are\ndescribed. This can be considered as a step towards automatically computing a\n\"general dictionary and grammar of the corpus\". We hope that in the long run\nvariants of our approach turn out to be useful for other kind of sequence data\nas well, such as, e.g., speech, genom sequences, or music annotation. Even if\nwe are not primarily interested in immediate applications, results obtained for\na variety of languages show that our methods are interesting for many practical\ntasks in text mining, terminology extraction and lexicography, search engine\ntechnology, and related fields.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05944v1",
            "date": "2016-02-18 20:53:26+00:00",
            "title": "The Interaction of Memory and Attention in Novel Word Generalization: A Computational Investigation",
            "authors": [
                "Erin Grant",
                "Aida Nematzadeh",
                "Suzanne Stevenson"
            ],
            "abstract": "People exhibit a tendency to generalize a novel noun to the basic-level in a\nhierarchical taxonomy -- a cognitively salient category such as \"dog\" -- with\nthe degree of generalization depending on the number and type of exemplars.\nRecently, a change in the presentation timing of exemplars has also been shown\nto have an effect, surprisingly reversing the prior observed pattern of\nbasic-level generalization. We explore the precise mechanisms that could lead\nto such behavior by extending a computational model of word learning and word\ngeneralization to integrate cognitive processes of memory and attention. Our\nresults show that the interaction of forgetting and attention to novelty, as\nwell as sensitivity to both type and token frequencies of exemplars, enables\nthe model to replicate the empirical results from different presentation\ntimings. Our results reinforce the need to incorporate general cognitive\nprocesses within word learning models to better understand the range of\nobserved behaviors in vocabulary acquisition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06023v5",
            "date": "2016-02-19 02:04:18+00:00",
            "title": "Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond",
            "authors": [
                "Ramesh Nallapati",
                "Bowen Zhou",
                "Cicero Nogueira dos santos",
                "Caglar Gulcehre",
                "Bing Xiang"
            ],
            "abstract": "In this work, we model abstractive text summarization using Attentional\nEncoder-Decoder Recurrent Neural Networks, and show that they achieve\nstate-of-the-art performance on two different corpora. We propose several novel\nmodels that address critical problems in summarization that are not adequately\nmodeled by the basic architecture, such as modeling key-words, capturing the\nhierarchy of sentence-to-word structure, and emitting words that are rare or\nunseen at training time. Our work shows that many of our proposed models\ncontribute to further improvement in performance. We also propose a new dataset\nconsisting of multi-sentence summaries, and establish performance benchmarks\nfor further research.",
            "journal ref": "The SIGNLL Conference on Computational Natural Language Learning\n  (CoNLL), 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06064v3",
            "date": "2016-02-19 07:27:49+00:00",
            "title": "On Training Bi-directional Neural Network Language Model with Noise Contrastive Estimation",
            "authors": [
                "Tianxing He",
                "Yu Zhang",
                "Jasha Droppo",
                "Kai Yu"
            ],
            "abstract": "We propose to train bi-directional neural network language model(NNLM) with\nnoise contrastive estimation(NCE). Experiments are conducted on a rescore task\non the PTB data set. It is shown that NCE-trained bi-directional NNLM\noutperformed the one trained by conventional maximum likelihood training. But\nstill(regretfully), it did not out-perform the baseline uni-directional NNLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06289v2",
            "date": "2016-02-19 20:48:19+00:00",
            "title": "Learning to SMILE(S)",
            "authors": [
                "Stanis\u0142aw Jastrz\u0119bski",
                "Damian Le\u015bniak",
                "Wojciech Marian Czarnecki"
            ],
            "abstract": "This paper shows how one can directly apply natural language processing (NLP)\nmethods to classification problems in cheminformatics. Connection between these\nseemingly separate fields is shown by considering standard textual\nrepresentation of compound, SMILES. The problem of activity prediction against\na target protein is considered, which is a crucial part of computer aided drug\ndesign process. Conducted experiments show that this way one can not only\noutrank state of the art results of hand crafted representations but also gets\ndirect structural insights into the way decisions are made.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06291v2",
            "date": "2016-02-19 20:52:08+00:00",
            "title": "Contextual LSTM (CLSTM) models for Large scale NLP tasks",
            "authors": [
                "Shalini Ghosh",
                "Oriol Vinyals",
                "Brian Strope",
                "Scott Roy",
                "Tom Dean",
                "Larry Heck"
            ],
            "abstract": "Documents exhibit sequential structure at multiple levels of abstraction\n(e.g., sentences, paragraphs, sections). These abstractions constitute a\nnatural hierarchy for representing the context in which to infer the meaning of\nwords and larger fragments of text. In this paper, we present CLSTM (Contextual\nLSTM), an extension of the recurrent neural network LSTM (Long-Short Term\nMemory) model, where we incorporate contextual features (e.g., topics) into the\nmodel. We evaluate CLSTM on three specific NLP tasks: word prediction, next\nsentence selection, and sentence topic prediction. Results from experiments run\non two corpora, English documents in Wikipedia and a subset of articles from a\nrecent snapshot of English Google News, indicate that using both words and\ntopics as features improves performance of the CLSTM models over baseline LSTM\nmodels for these tasks. For example on the next sentence selection task, we get\nrelative accuracy improvements of 21% for the Wikipedia dataset and 18% for the\nGoogle News dataset. This clearly demonstrates the significant benefit of using\ncontext appropriately in natural language (NL) tasks. This has implications for\na wide variety of NL applications like question answering, sentence completion,\nparaphrase generation, and next utterance prediction in dialog systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06797v2",
            "date": "2016-02-22 14:55:26+00:00",
            "title": "Semi-supervised Clustering for Short Text via Deep Representation Learning",
            "authors": [
                "Zhiguo Wang",
                "Haitao Mi",
                "Abraham Ittycheriah"
            ],
            "abstract": "In this work, we propose a semi-supervised method for short text clustering,\nwhere we represent texts as distributed vectors with neural networks, and use a\nsmall amount of labeled data to specify our intention for clustering. We design\na novel objective to combine the representation learning process and the\nk-means clustering process together, and optimize the objective with both\nlabeled data and unlabeled data iteratively until convergence through three\nsteps: (1) assign each short text to its nearest centroid based on its\nrepresentation from the current neural networks; (2) re-estimate the cluster\ncentroids based on cluster assignments from step (1); (3) update neural\nnetworks according to the objective by keeping centroids and cluster\nassignments fixed. Experimental results on four datasets show that our method\nworks significantly better than several other text clustering methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07019v2",
            "date": "2016-02-23 03:08:50+00:00",
            "title": "Sentence Similarity Learning by Lexical Decomposition and Composition",
            "authors": [
                "Zhiguo Wang",
                "Haitao Mi",
                "Abraham Ittycheriah"
            ],
            "abstract": "Most conventional sentence similarity methods only focus on similar parts of\ntwo input sentences, and simply ignore the dissimilar parts, which usually give\nus some clues and semantic meanings about the sentences. In this work, we\npropose a model to take into account both the similarities and dissimilarities\nby decomposing and composing lexical semantics over sentences. The model\nrepresents each word as a vector, and calculates a semantic matching vector for\neach word based on all words in the other sentence. Then, each word vector is\ndecomposed into a similar component and a dissimilar component based on the\nsemantic matching vector. After this, a two-channel CNN model is employed to\ncapture features by composing the similar and dissimilar components. Finally, a\nsimilarity score is estimated over the composed feature vectors. Experimental\nresults show that our model gets the state-of-the-art performance on the answer\nsentence selection task, and achieves a comparable result on the paraphrase\nidentification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07236v1",
            "date": "2016-02-23 17:05:06+00:00",
            "title": "Petrarch 2 : Petrarcher",
            "authors": [
                "Clayton Norris"
            ],
            "abstract": "PETRARCH 2 is the fourth generation of a series of Event-Data coders stemming\nfrom research by Phillip Schrodt. Each iteration has brought new functionality\nand usability, and this is no exception.Petrarch 2 takes much of the power of\nthe original Petrarch's dictionaries and redirects it into a faster and smarter\ncore logic. Earlier iterations handled sentences largely as a list of words,\nincorporating some syntactic information here and there. Petrarch 2 now views\nthe sentence entirely on the syntactic level. It receives the syntactic parse\nof a sentence from the Stanford CoreNLP software, and stores this data as a\ntree structure of linked nodes, where each node is a Phrase object.\nPrepositional, noun, and verb phrases each have their own version of this\nPhrase class, which deals with the logic particular to those kinds of phrases.\nSince this is an event coder, the core of the logic focuses around the verbs:\nwho is acting, who is being acted on, and what is happening. The theory behind\nthis new structure and its logic is founded in Generative Grammar, Information\nTheory, and Lambda-Calculus Semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07572v2",
            "date": "2016-02-24 16:06:25+00:00",
            "title": "Ultradense Word Embeddings by Orthogonal Transformation",
            "authors": [
                "Sascha Rothe",
                "Sebastian Ebert",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Embeddings are generic representations that are useful for many NLP tasks. In\nthis paper, we introduce DENSIFIER, a method that learns an orthogonal\ntransformation of the embedding space that focuses the information relevant for\na task in an ultradense subspace of a dimensionality that is smaller by a\nfactor of 100 than the original space. We show that ultradense embeddings\ngenerated by DENSIFIER reach state of the art on a lexicon creation task in\nwhich words are annotated with three types of lexical information - sentiment,\nconcreteness and frequency. On the SemEval2015 10B sentiment analysis task we\nshow that no information is lost when the ultradense subspace is used, but\ntraining is an order of magnitude more efficient due to the compactness of the\nultradense space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07749v1",
            "date": "2016-02-24 23:14:01+00:00",
            "title": "Toward Mention Detection Robustness with Recurrent Neural Networks",
            "authors": [
                "Thien Huu Nguyen",
                "Avirup Sil",
                "Georgiana Dinu",
                "Radu Florian"
            ],
            "abstract": "One of the key challenges in natural language processing (NLP) is to yield\ngood performance across application domains and languages. In this work, we\ninvestigate the robustness of the mention detection systems, one of the\nfundamental tasks in information extraction, via recurrent neural networks\n(RNNs). The advantage of RNNs over the traditional approaches is their capacity\nto capture long ranges of context and implicitly adapt the word embeddings,\ntrained on a large corpus, into a task-specific word representation, but still\npreserve the original semantic generalization to be helpful across domains. Our\nsystematic evaluation for RNN architectures demonstrates that RNNs not only\noutperform the best reported systems (up to 9\\% relative error reduction) in\nthe general setting but also achieve the state-of-the-art performance in the\ncross-domain setting for English. Regarding other languages, RNNs are\nsignificantly better than the traditional methods on the similar task of named\nentity recognition for Dutch (up to 22\\% relative error reduction).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07803v1",
            "date": "2016-02-25 05:35:16+00:00",
            "title": "Automated Word Prediction in Bangla Language Using Stochastic Language Models",
            "authors": [
                "Md. Masudul Haque",
                "Md. Tarek Habib",
                "Md. Mokhlesur Rahman"
            ],
            "abstract": "Word completion and word prediction are two important phenomena in typing\nthat benefit users who type using keyboard or other similar devices. They can\nhave profound impact on the typing of disable people. Our work is based on word\nprediction on Bangla sentence by using stochastic, i.e. N-gram language model\nsuch as unigram, bigram, trigram, deleted Interpolation and backoff models for\nauto completing a sentence by predicting a correct word in a sentence which\nsaves time and keystrokes of typing and also reduces misspelling. We use large\ndata corpus of Bangla language of different word types to predict correct word\nwith the accuracy as much as possible. We have found promising results. We hope\nthat our work will impact on the baseline for automated Bangla typing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08657v2",
            "date": "2016-02-28 02:25:22+00:00",
            "title": "QuotationFinder - Searching for Quotations and Allusions in Greek and Latin Texts and Establishing the Degree to Which a Quotation or Allusion Matches Its Source",
            "authors": [
                "Luc Herren"
            ],
            "abstract": "The software programs generally used with the TLG (Thesaurus Linguae Graecae)\nand the CLCLT (CETEDOC Library of Christian Latin Texts) CD-ROMs are not well\nsuited for finding quotations and allusions. QuotationFinder uses more\nsophisticated criteria as it ranks search results based on how closely they\nmatch the source text, listing search results with literal quotations first and\nloose verbal parallels last.",
            "journal ref": "Journal of Data Mining & Digital Humanities, Special Issue on\n  Computer-Aided Processing of Intertextuality in Ancient Languages, Project\n  presentations (August 2, 2017) jdmdh:3825",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08715v2",
            "date": "2016-02-28 13:43:33+00:00",
            "title": "Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus",
            "authors": [
                "Avi Shmidman",
                "Moshe Koppel",
                "Ely Porat"
            ],
            "abstract": "We propose a method for efficiently finding all parallel passages in a large\ncorpus, even if the passages are not quite identical due to rephrasing and\northographic variation. The key ideas are the representation of each word in\nthe corpus by its two most infrequent letters, finding matched pairs of strings\nof four or five words that differ by at most one word and then identifying\nclusters of such matched pairs. Using this method, over 4600 parallel pairs of\npassages were identified in the Babylonian Talmud, a Hebrew-Aramaic corpus of\nover 1.8 million words, in just over 30 seconds. Empirical comparisons on\nsample data indicate that the coverage obtained by our method is essentially\nthe same as that obtained using slow exhaustive methods.",
            "journal ref": "Journal of Data Mining & Digital Humanities, Special Issue on\n  Computer-Aided Processing of Intertextuality in Ancient Languages, Towards a\n  Digital Ecosystem: NLP. Corpus infrastructure. Methods for Retrieving Texts\n  and Computing Text Similarities (March 11, 2018) jdmdh:4175",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08741v1",
            "date": "2016-02-28 16:58:01+00:00",
            "title": "Gibberish Semantics: How Good is Russian Twitter in Word Semantic Similarity Task?",
            "authors": [
                "Nikolay N. Vasiliev"
            ],
            "abstract": "The most studied and most successful language models were developed and\nevaluated mainly for English and other close European languages, such as\nFrench, German, etc. It is important to study applicability of these models to\nother languages. The use of vector space models for Russian was recently\nstudied for multiple corpora, such as Wikipedia, RuWac, lib.ru. These models\nwere evaluated against word semantic similarity task. For our knowledge Twitter\nwas not considered as a corpus for this task, with this work we fill the gap.\nResults for vectors trained on Twitter corpus are comparable in accuracy with\nother single-corpus trained models, although the best performance is currently\nachieved by combination of multiple corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08844v2",
            "date": "2016-02-29 07:20:59+00:00",
            "title": "Bioinformatics and Classical Literary Study",
            "authors": [
                "Pramit Chaudhuri",
                "Joseph P. Dexter"
            ],
            "abstract": "This paper describes the Quantitative Criticism Lab, a collaborative\ninitiative between classicists, quantitative biologists, and computer\nscientists to apply ideas and methods drawn from the sciences to the study of\nliterature. A core goal of the project is the use of computational biology,\nnatural language processing, and machine learning techniques to investigate\nauthorial style, intertextuality, and related phenomena of literary\nsignificance. As a case study in our approach, here we review the use of\nsequence alignment, a common technique in genomics and computational\nlinguistics, to detect intertextuality in Latin literature. Sequence alignment\nis distinguished by its ability to find inexact verbal similarities, which\nmakes it ideal for identifying phonetic echoes in large corpora of Latin texts.\nAlthough especially suited to Latin, sequence alignment in principle can be\nextended to many other languages.",
            "journal ref": "Journal of Data Mining & Digital Humanities, Special Issue on\n  Computer-Aided Processing of Intertextuality in Ancient Languages, Project\n  presentations (August 19, 2017) jdmdh:3807",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00070v1",
            "date": "2016-09-01 00:20:41+00:00",
            "title": "How Much is 131 Million Dollars? Putting Numbers in Perspective with Compositional Descriptions",
            "authors": [
                "Arun Tejasvi Chaganty",
                "Percy Liang"
            ],
            "abstract": "How much is 131 million US dollars? To help readers put such numbers in\ncontext, we propose a new task of automatically generating short descriptions\nknown as perspectives, e.g. \"$131 million is about the cost to employ everyone\nin Texas over a lunch period\". First, we collect a dataset of numeric mentions\nin news articles, where each mention is labeled with a set of rated\nperspectives. We then propose a system to generate these descriptions\nconsisting of two steps: formula construction and description generation. In\nconstruction, we compose formulae from numeric facts in a knowledge base and\nrank the resulting formulas based on familiarity, numeric proximity and\nsemantic compatibility. In generation, we convert a formula into natural\nlanguage using a sequence-to-sequence recurrent neural network. Our system\nobtains a 15.2% F1 improvement over a non-compositional baseline at formula\nconstruction and a 12.5 BLEU point improvement over a baseline description\ngeneration.",
            "journal ref": "ACL (2016), 578-587",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00559v2",
            "date": "2016-09-02 11:44:17+00:00",
            "title": "Improving Correlation with Human Judgments by Integrating Semantic Similarity with Second--Order Vectors",
            "authors": [
                "Bridget T. McInnes",
                "Ted Pedersen"
            ],
            "abstract": "Vector space methods that measure semantic similarity and relatedness often\nrely on distributional information such as co--occurrence frequencies or\nstatistical measures of association to weight the importance of particular\nco--occurrences. In this paper, we extend these methods by incorporating a\nmeasure of semantic similarity based on a human curated taxonomy into a\nsecond--order vector representation. This results in a measure of semantic\nrelatedness that combines both the contextual information available in a\ncorpus--based vector space representation with the semantic knowledge found in\na biomedical ontology. Our results show that incorporating semantic similarity\ninto a second order co--occurrence matrices improves correlation with human\njudgments for both similarity and relatedness, and that our method compares\nfavorably to various different word embedding methods that have recently been\nevaluated on the same reference standards we have used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00565v1",
            "date": "2016-09-02 11:57:46+00:00",
            "title": "Skipping Word: A Character-Sequential Representation based Framework for Question Answering",
            "authors": [
                "Lingxun Meng",
                "Yan Li",
                "Mengyi Liu",
                "Peng Shu"
            ],
            "abstract": "Recent works using artificial neural networks based on word distributed\nrepresentation greatly boost the performance of various natural language\nlearning tasks, especially question answering. Though, they also carry along\nwith some attendant problems, such as corpus selection for embedding learning,\ndictionary transformation for different learning tasks, etc. In this paper, we\npropose to straightforwardly model sentences by means of character sequences,\nand then utilize convolutional neural networks to integrate character embedding\nlearning together with point-wise answer selection training. Compared with deep\nmodels pre-trained on word embedding (WE) strategy, our character-sequential\nrepresentation (CSR) based method shows a much simpler procedure and more\nstable performance across different benchmarks. Extensive experiments on two\nbenchmark answer selection datasets exhibit the competitive performance\ncompared with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01188v1",
            "date": "2016-09-05 14:55:31+00:00",
            "title": "Bi-Text Alignment of Movie Subtitles for Spoken English-Arabic Statistical Machine Translation",
            "authors": [
                "Fahad Al-Obaidli",
                "Stephen Cox",
                "Preslav Nakov"
            ],
            "abstract": "We describe efforts towards getting better resources for English-Arabic\nmachine translation of spoken text. In particular, we look at movie subtitles\nas a unique, rich resource, as subtitles in one language often get translated\ninto other languages. Movie subtitles are not new as a resource and have been\nexplored in previous research; however, here we create a much larger bi-text\n(the biggest to date), and we further generate better quality alignment for it.\nGiven the subtitles for the same movie in different languages, a key problem is\nhow to align them at the fragment level. Typically, this is done using\nlength-based alignment, but for movie subtitles, there is also time\ninformation. Here we exploit this information to develop an original algorithm\nthat outperforms the current best subtitle alignment tool, subalign. The\nevaluation results show that adding our bi-text to the IWSLT training bi-text\nyields an improvement of over two BLEU points absolute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01235v1",
            "date": "2016-09-05 17:47:49+00:00",
            "title": "PMI Matrix Approximations with Applications to Neural Language Modeling",
            "authors": [
                "Oren Melamud",
                "Ido Dagan",
                "Jacob Goldberger"
            ],
            "abstract": "The negative sampling (NEG) objective function, used in word2vec, is a\nsimplification of the Noise Contrastive Estimation (NCE) method. NEG was found\nto be highly effective in learning continuous word representations. However,\nunlike NCE, it was considered inapplicable for the purpose of learning the\nparameters of a language model. In this study, we refute this assertion by\nproviding a principled derivation for NEG-based language modeling, founded on a\nnovel analysis of a low-dimensional approximation of the matrix of pointwise\nmutual information between the contexts and the predicted words. The obtained\nlanguage modeling is closely related to NCE language models but is based on a\nsimplified objective function. We thus provide a unified formulation for two\nmain language processing tasks, namely word embedding and language modeling,\nbased on the NEG objective function. Experimental results on two popular\nlanguage modeling benchmarks show comparable perplexity results, with a small\nadvantage to NEG over NCE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01454v1",
            "date": "2016-09-06 09:29:12+00:00",
            "title": "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "Attention-based encoder-decoder neural network models have recently shown\npromising results in machine translation and speech recognition. In this work,\nwe propose an attention-based neural network model for joint intent detection\nand slot filling, both of which are critical steps for many speech\nunderstanding and dialog systems. Unlike in machine translation and speech\nrecognition, alignment is explicit in slot filling. We explore different\nstrategies in incorporating this alignment information to the encoder-decoder\nframework. Learning from the attention mechanism in encoder-decoder model, we\nfurther propose introducing attention to the alignment-based RNN models. Such\nattentions provide additional information to the intent classification and slot\nlabel prediction. Our independent task models achieve state-of-the-art intent\ndetection error rate and slot filling F1 score on the benchmark ATIS task. Our\njoint training model further obtains 0.56% absolute (23.8% relative) error\nreduction on intent detection and 0.23% absolute gain on slot filling over the\nindependent task models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01462v1",
            "date": "2016-09-06 09:45:51+00:00",
            "title": "Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networks",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "Speaker intent detection and semantic slot filling are two critical tasks in\nspoken language understanding (SLU) for dialogue systems. In this paper, we\ndescribe a recurrent neural network (RNN) model that jointly performs intent\ndetection, slot filling, and language modeling. The neural network model keeps\nupdating the intent estimation as word in the transcribed utterance arrives and\nuses it as contextual features in the joint model. Evaluation of the language\nmodel and online SLU model is made on the ATIS benchmarking data set. On\nlanguage modeling task, our joint model achieves 11.8% relative reduction on\nperplexity comparing to the independent training language model. On SLU tasks,\nour joint model outperforms the independent task training model by 22.3% on\nintent detection error rate, with slight degradation on slot filling F1 score.\nThe joint model also shows advantageous performance in the realistic ASR\nsettings with noisy speech input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01933v1",
            "date": "2016-09-07 10:59:58+00:00",
            "title": "Sentiment Classification of Food Reviews",
            "authors": [
                "Hua Feng",
                "Ruixi Lin"
            ],
            "abstract": "Sentiment analysis of reviews is a popular task in natural language\nprocessing. In this work, the goal is to predict the score of food reviews on a\nscale of 1 to 5 with two recurrent neural networks that are carefully tuned. As\nfor baseline, we train a simple RNN for classification. Then we extend the\nbaseline to GRU. In addition, we present two different methods to deal with\nhighly skewed data, which is a common problem for reviews. Models are evaluated\nusing accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02549v3",
            "date": "2016-09-08 19:51:47+00:00",
            "title": "Learning Lexical Entries for Robotic Commands using Crowdsourcing",
            "authors": [
                "Junjie Hu",
                "Jean Oh",
                "Anatole Gershman"
            ],
            "abstract": "Robotic commands in natural language usually contain various spatial\ndescriptions that are semantically similar but syntactically different. Mapping\nsuch syntactic variants into semantic concepts that can be understood by robots\nis challenging due to the high flexibility of natural language expressions. To\ntackle this problem, we collect robotic commands for navigation and\nmanipulation tasks using crowdsourcing. We further define a robot language and\nuse a generative machine translation model to translate robotic commands from\nnatural language to robot language. The main purpose of this paper is to\nsimulate the interaction process between human and robots using crowdsourcing\nplatforms, and investigate the possibility of translating natural language to\nrobot language with paraphrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02809v1",
            "date": "2016-09-09 14:23:02+00:00",
            "title": "Harassment detection: a benchmark on the #HackHarassment dataset",
            "authors": [
                "Alexei Bastidas",
                "Edward Dixon",
                "Chris Loo",
                "John Ryan"
            ],
            "abstract": "Online harassment has been a problem to a greater or lesser extent since the\nearly days of the internet. Previous work has applied anti-spam techniques like\nmachine-learning based text classification (Reynolds, 2011) to detecting\nharassing messages. However, existing public datasets are limited in size, with\nlabels of varying quality. The #HackHarassment initiative (an alliance of 1\ntech companies and NGOs devoted to fighting bullying on the internet) has begun\nto address this issue by creating a new dataset superior to its predecssors in\nterms of both size and quality. As we (#HackHarassment) complete further rounds\nof labelling, later iterations of this dataset will increase the available\nsamples by at least an order of magnitude, enabling corresponding improvements\nin the quality of machine learning models for harassment detection. In this\npaper, we introduce the first models built on the #HackHarassment dataset v1.0\n(a new open dataset, which we are delighted to share with any interested\nresearcherss) as a benchmark for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02846v1",
            "date": "2016-09-09 16:02:57+00:00",
            "title": "Dialogue manager domain adaptation using Gaussian process reinforcement learning",
            "authors": [
                "Milica Gasic",
                "Nikola Mrksic",
                "Lina M. Rojas-Barahona",
                "Pei-Hao Su",
                "Stefan Ultes",
                "David Vandyke",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "Spoken dialogue systems allow humans to interact with machines using natural\nspeech. As such, they have many benefits. By using speech as the primary\ncommunication medium, a computer interface can facilitate swift, human-like\nacquisition of information. In recent years, speech interfaces have become ever\nmore popular, as is evident from the rise of personal assistants such as Siri,\nGoogle Now, Cortana and Amazon Alexa. Recently, data-driven machine learning\nmethods have been applied to dialogue modelling and the results achieved for\nlimited-domain applications are comparable to or outperform traditional\napproaches. Methods based on Gaussian processes are particularly effective as\nthey enable good models to be estimated from limited training data.\nFurthermore, they provide an explicit estimate of the uncertainty which is\nparticularly useful for reinforcement learning. This article explores the\nadditional steps that are necessary to extend these methods to model multiple\ndialogue domains. We show that Gaussian process reinforcement learning is an\nelegant framework that naturally supports a range of methods, including prior\nknowledge, Bayesian committee machines and multi-agent learning, for\nfacilitating extensible and adaptable dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02960v1",
            "date": "2016-09-09 22:22:53+00:00",
            "title": "A Large Scale Corpus of Gulf Arabic",
            "authors": [
                "Salam Khalifa",
                "Nizar Habash",
                "Dana Abdulrahim",
                "Sara Hassan"
            ],
            "abstract": "Most Arabic natural language processing tools and resources are developed to\nserve Modern Standard Arabic (MSA), which is the official written language in\nthe Arab World. Some Dialectal Arabic varieties, notably Egyptian Arabic, have\nreceived some attention lately and have a growing collection of resources that\ninclude annotated corpora and morphological analyzers and taggers. Gulf Arabic,\nhowever, lags behind in that respect. In this paper, we present the Gumar\nCorpus, a large-scale corpus of Gulf Arabic consisting of 110 million words\nfrom 1,200 forum novels. We annotate the corpus for sub-dialect information at\nthe document level. We also present results of a preliminary study in the\nmorphological annotation of Gulf Arabic which includes developing guidelines\nfor a conventional orthography. The text of the corpus is publicly browsable\nthrough a web interface we developed for it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03204v1",
            "date": "2016-09-11 19:51:46+00:00",
            "title": "On the Similarities Between Native, Non-native and Translated Texts",
            "authors": [
                "Ella Rabinovich",
                "Sergiu Nisioi",
                "Noam Ordan",
                "Shuly Wintner"
            ],
            "abstract": "We present a computational analysis of three language varieties: native,\nadvanced non-native, and translation. Our goal is to investigate the\nsimilarities and differences between non-native language productions and\ntranslations, contrasting both with native language. Using a collection of\ncomputational methods we establish three main results: (1) the three types of\ntexts are easily distinguishable; (2) non-native language and translations are\ncloser to each other than each of them is to native language; and (3) some of\nthese characteristics depend on the source or native language, while others do\nnot, reflecting, perhaps, unified principles that similarly affect translations\nand non-native language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03205v1",
            "date": "2016-09-11 19:52:28+00:00",
            "title": "Unsupervised Identification of Translationese",
            "authors": [
                "Ella Rabinovich",
                "Shuly Wintner"
            ],
            "abstract": "Translated texts are distinctively different from original ones, to the\nextent that supervised text classification methods can distinguish between them\nwith high accuracy. These differences were proven useful for statistical\nmachine translation. However, it has been suggested that the accuracy of\ntranslation detection deteriorates when the classifier is evaluated outside the\ndomain it was trained on. We show that this is indeed the case, in a variety of\nevaluation scenarios. We then show that unsupervised classification is highly\naccurate on this task. We suggest a method for determining the correct labels\nof the clustering outcomes, and then use the labels for voting, improving the\naccuracy even further. Moreover, we suggest a simple method for clustering in\nthe challenging case of mixed-domain datasets, in spite of the dominance of\ndomain-related features over translation-related ones. The result is an\neffective, fully-unsupervised method for distinguishing between original and\ntranslated texts that can be applied to new domains with reasonable accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03376v1",
            "date": "2016-09-12 12:52:37+00:00",
            "title": "Morphological Constraints for Phrase Pivot Statistical Machine Translation",
            "authors": [
                "Ahmed El Kholy",
                "Nizar Habash"
            ],
            "abstract": "The lack of parallel data for many language pairs is an important challenge\nto statistical machine translation (SMT). One common solution is to pivot\nthrough a third language for which there exist parallel corpora with the source\nand target languages. Although pivoting is a robust technique, it introduces\nsome low quality translations especially when a poor morphology language is\nused as the pivot between rich morphology languages. In this paper, we examine\nthe use of synchronous morphology constraint features to improve the quality of\nphrase pivot SMT. We compare hand-crafted constraints to those learned from\nlimited parallel data between source and target languages. The learned\nmorphology constraints are based on projected align- ments between the source\nand target phrases in the pivot phrase table. We show positive results on\nHebrew-Arabic SMT (pivoting on English). We get 1.5 BLEU points over a phrase\npivot baseline and 0.8 BLEU points over a system combination baseline with a\ndirect model built from parallel data.",
            "journal ref": "Proceedings of MT Summit XV. Miami. pp 104-116 (2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03441v2",
            "date": "2016-09-12 15:16:43+00:00",
            "title": "Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing",
            "authors": [
                "Jan Chorowski",
                "Micha\u0142 Zapotoczny",
                "Pawe\u0142 Rychlikowski"
            ],
            "abstract": "We present a dependency parser implemented as a single deep neural network\nthat reads orthographic representations of words and directly generates\ndependencies and their labels. Unlike typical approaches to parsing, the model\ndoesn't require part-of-speech (POS) tagging of the sentences. With proper\nregularization and additional supervision achieved with multitask learning we\nreach state-of-the-art performance on Slavic languages from the Universal\nDependencies treebank: with no linguistic features other than characters, our\nparser is as accurate as a transition- based system trained on perfect POS\ntags.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04186v1",
            "date": "2016-09-14 09:31:40+00:00",
            "title": "Neural Machine Translation with Supervised Attention",
            "authors": [
                "Lemao Liu",
                "Masao Utiyama",
                "Andrew Finch",
                "Eiichiro Sumita"
            ],
            "abstract": "The attention mechanisim is appealing for neural machine translation, since\nit is able to dynam- ically encode a source sentence by generating a alignment\nbetween a target word and source words. Unfortunately, it has been proved to be\nworse than conventional alignment models in aligment accuracy. In this paper,\nwe analyze and explain this issue from the point view of re- ordering, and\npropose a supervised attention which is learned with guidance from conventional\nalignment models. Experiments on two Chinese-to-English translation tasks show\nthat the super- vised attention mechanism yields better alignments leading to\nsubstantial gains over the standard attention based NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04253v1",
            "date": "2016-09-14 13:12:12+00:00",
            "title": "Neural Machine Transliteration: Preliminary Results",
            "authors": [
                "Amir H. Jadidinejad"
            ],
            "abstract": "Machine transliteration is the process of automatically transforming the\nscript of a word from a source language to a target language, while preserving\npronunciation. Sequence to sequence learning has recently emerged as a new\nparadigm in supervised learning. In this paper a character-based\nencoder-decoder model has been proposed that consists of two Recurrent Neural\nNetworks. The encoder is a Bidirectional recurrent neural network that encodes\na sequence of symbols into a fixed-length vector representation, and the\ndecoder generates the target sequence using an attention-based recurrent neural\nnetwork. The encoder, the decoder and the attention mechanism are jointly\ntrained to maximize the conditional probability of a target sequence given a\nsource sequence. Our experiments on different datasets show that the proposed\nencoder-decoder model is able to achieve significantly higher transliteration\nquality over traditional statistical models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04325v1",
            "date": "2016-09-14 15:58:55+00:00",
            "title": "Transliteration in Any Language with Surrogate Languages",
            "authors": [
                "Stephen Mayhew",
                "Christos Christodoulopoulos",
                "Dan Roth"
            ],
            "abstract": "We introduce a method for transliteration generation that can produce\ntransliterations in every language. Where previous results are only as\nmultilingual as Wikipedia, we show how to use training data from Wikipedia as\nsurrogate training for any language. Thus, the problem becomes one of ranking\nWikipedia languages in order of suitability with respect to a target language.\nWe introduce several task-specific methods for ranking languages, and show that\nour approach is comparable to the oracle ceiling, and even outperforms it in\nsome cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04621v1",
            "date": "2016-09-15 13:15:01+00:00",
            "title": "Factored Neural Machine Translation",
            "authors": [
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Lo\u00efc Barrault",
                "Fethi Bougares"
            ],
            "abstract": "We present a new approach for neural machine translation (NMT) using the\nmorphological and grammatical decomposition of the words (factors) in the\noutput side of the neural network. This architecture addresses two main\nproblems occurring in MT, namely dealing with a large target language\nvocabulary and the out of vocabulary (OOV) words. By the means of factors, we\nare able to handle larger vocabulary and reduce the training time (for systems\nwith equivalent target language vocabulary size). In addition, we can produce\nnew words that are not in the vocabulary. We use a morphological analyser to\nget a factored representation of each word (lemmas, Part of Speech tag, tense,\nperson, gender and number). We have extended the NMT approach with attention\nmechanism in order to have two different outputs, one for the lemmas and the\nother for the rest of the factors. The final translation is built using some\n\\textit{a priori} linguistic information. We compare our extension with a\nword-based NMT system. The experiments, performed on the IWSLT'15 dataset\ntranslating from English to French, show that while the performance do not\nalways increase, the system can manage a much larger vocabulary and\nconsistently reduce the OOV rate. We observe up to 2% BLEU point improvement in\na simulated out of domain translation setup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04779v1",
            "date": "2016-09-15 19:07:17+00:00",
            "title": "Characterizing the Language of Online Communities and its Relation to Community Reception",
            "authors": [
                "Trang Tran",
                "Mari Ostendorf"
            ],
            "abstract": "This work investigates style and topic aspects of language in online\ncommunities: looking at both utility as an identifier of the community and\ncorrelation with community reception of content. Style is characterized using a\nhybrid word and part-of-speech tag n-gram language model, while topic is\nrepresented using Latent Dirichlet Allocation. Experiments with several Reddit\nforums show that style is a better indicator of community identity than topic,\neven for communities organized around specific topics. Further, there is a\npositive correlation between the community reception to a contribution and the\nstyle similarity to that community, but not so for topic similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04873v3",
            "date": "2016-09-15 22:01:26+00:00",
            "title": "Distant Supervision for Relation Extraction beyond the Sentence Boundary",
            "authors": [
                "Chris Quirk",
                "Hoifung Poon"
            ],
            "abstract": "The growing demand for structured knowledge has led to great interest in\nrelation extraction, especially in cases with limited supervision. However,\nexisting distance supervision approaches only extract relations expressed in\nsingle sentences. In general, cross-sentence relation extraction is\nunder-explored, even in the supervised-learning setting. In this paper, we\npropose the first approach for applying distant supervision to cross- sentence\nrelation extraction. At the core of our approach is a graph representation that\ncan incorporate both standard dependencies and discourse relations, thus\nproviding a unifying way to model relations within and across sentences. We\nextract features from multiple paths in this graph, increasing accuracy and\nrobustness when confronted with linguistic variation and analysis error.\nExperiments on an important extraction task for precision medicine show that\nour approach can learn an accurate cross-sentence extractor, using only a small\nexisting knowledge base and unlabeled text from biomedical research articles.\nCompared to the existing distant supervision paradigm, our approach extracted\ntwice as many relations at similar precision, thus demonstrating the prevalence\nof cross-sentence relations and the promise of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04909v3",
            "date": "2016-09-16 04:58:54+00:00",
            "title": "An Iterative Transfer Learning Based Ensemble Technique for Automatic Short Answer Grading",
            "authors": [
                "Shourya Roy",
                "Himanshu S. Bhatt",
                "Y. Narahari"
            ],
            "abstract": "Automatic short answer grading (ASAG) techniques are designed to\nautomatically assess short answers to questions in natural language, having a\nlength of a few words to a few sentences. Supervised ASAG techniques have been\ndemonstrated to be effective but suffer from a couple of key practical\nlimitations. They are greatly reliant on instructor provided model answers and\nneed labeled training data in the form of graded student answers for every\nassessment task. To overcome these, in this paper, we introduce an ASAG\ntechnique with two novel features. We propose an iterative technique on an\nensemble of (a) a text classifier of student answers and (b) a classifier using\nnumeric features derived from various similarity measures with respect to model\nanswers. Second, we employ canonical correlation analysis based transfer\nlearning on a common feature representation to build the classifier ensemble\nfor questions having no labelled data. The proposed technique handsomely beats\nall winning supervised entries on the SCIENTSBANK dataset from the Student\nResponse Analysis task of SemEval 2013. Additionally, we demonstrate\ngeneralizability and benefits of the proposed technique through evaluation on\nmultiple ASAG datasets from different subject topics and standards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05511v5",
            "date": "2016-09-18 16:29:20+00:00",
            "title": "Multilinear Grammar: Ranks and Interpretations",
            "authors": [
                "Dafydd Gibbon",
                "Sascha Griffiths"
            ],
            "abstract": "Multilinear Grammar provides a framework for integrating the many different\nsyntagmatic structures of language into a coherent semiotically based Rank\nInterpretation Architecture, with default linear grammars at each rank. The\narchitecture defines a Sui Generis Condition on ranks, from discourse through\nutterance and phrasal structures to the word, with its sub-ranks of morphology\nand phonology. Each rank has unique structures and its own semantic-pragmatic\nand prosodic-phonetic interpretation models. Default computational models for\neach rank are proposed, based on a Procedural Plausibility Condition:\nincremental processing in linear time with finite working memory. We suggest\nthat the Rank Interpretation Architecture and its multilinear properties\nprovide systematic design features of human languages, contrasting with\nunordered lists of key properties or single structural properties at one rank,\nsuch as recursion, which have previously been been put forward as language\ndesign features. The framework provides a realistic background for the gradual\ndevelopment of complexity in the phylogeny and ontogeny of language, and\nclarifies a range of challenges for the evaluation of realistic linguistic\ntheories and applications. The empirical objective of the paper is to\ndemonstrate unique multilinear properties at each rank and thereby motivate the\nMultilinear Grammar and Rank Interpretation Architecture framework as a\ncoherent approach to capturing the complexity of human languages in the\nsimplest possible way.",
            "journal ref": "Open Linguistics 2017, 3(1): 265-307",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05625v3",
            "date": "2016-09-19 07:57:35+00:00",
            "title": "The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition",
            "authors": [
                "Ahmed Ali",
                "Peter Bell",
                "James Glass",
                "Yacine Messaoui",
                "Hamdy Mubarak",
                "Steve Renals",
                "Yifan Zhang"
            ],
            "abstract": "This paper describes the Arabic Multi-Genre Broadcast (MGB-2) Challenge for\nSLT-2016. Unlike last year's English MGB Challenge, which focused on\nrecognition of diverse TV genres, this year, the challenge has an emphasis on\nhandling the diversity in dialect in Arabic speech. Audio data comes from 19\ndistinct programmes from the Aljazeera Arabic TV channel between March 2005 and\nDecember 2015. Programmes are split into three groups: conversations,\ninterviews, and reports. A total of 1,200 hours have been released with lightly\nsupervised transcriptions for the acoustic modelling. For language modelling,\nwe made available over 110M words crawled from Aljazeera Arabic website\nAljazeera.net for a 10 year duration 2000-2011. Two lexicons have been\nprovided, one phoneme based and one grapheme based. Finally, two tasks were\nproposed for this year's challenge: standard speech transcription, and word\nalignment. This paper describes the task data and evaluation process used in\nthe MGB challenge, and summarises the results obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05650v1",
            "date": "2016-09-19 09:44:54+00:00",
            "title": "Multi-view Dimensionality Reduction for Dialect Identification of Arabic Broadcast Speech",
            "authors": [
                "Sameer Khurana",
                "Ahmed Ali",
                "Steve Renals"
            ],
            "abstract": "In this work, we present a new Vector Space Model (VSM) of speech utterances\nfor the task of spoken dialect identification. Generally, DID systems are built\nusing two sets of features that are extracted from speech utterances; acoustic\nand phonetic. The acoustic and phonetic features are used to form vector\nrepresentations of speech utterances in an attempt to encode information about\nthe spoken dialects. The Phonotactic and Acoustic VSMs, thus formed, are used\nfor the task of DID. The aim of this paper is to construct a single VSM that\nencodes information about spoken dialects from both the Phonotactic and\nAcoustic VSMs. Given the two views of the data, we make use of a well known\nmulti-view dimensionality reduction technique known as Canonical Correlation\nAnalysis (CCA), to form a single vector representation for each speech\nutterance that encodes dialect specific discriminative information from both\nthe phonetic and acoustic representations. We refer to this approach as feature\nspace combination approach and show that our CCA based feature vector\nrepresentation performs better on the Arabic DID task than the phonetic and\nacoustic feature representations used alone. We also present the feature space\ncombination approach as a viable alternative to the model based combination\napproach, where two DID systems are built using the two VSMs (Phonotactic and\nAcoustic) and the final prediction score is the output score combination from\nthe two systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05935v2",
            "date": "2016-09-19 20:52:44+00:00",
            "title": "Advances in All-Neural Speech Recognition",
            "authors": [
                "G. Zweig",
                "C. Yu",
                "J. Droppo",
                "A. Stolcke"
            ],
            "abstract": "This paper advances the design of CTC-based all-neural (or end-to-end) speech\nrecognizers. We propose a novel symbol inventory, and a novel iterated-CTC\nmethod in which a second system is used to transform a noisy initial output\ninto a cleaner version. We present a number of stabilization and initialization\nmethods we have found useful in training these networks. We evaluate our system\non the commonly used NIST 2000 conversational telephony test set, and\nsignificantly exceed the previously published performance of similar systems,\nboth with and without the use of an external language model and decoding\ntechnology.",
            "journal ref": "Proc. IEEE ICASSP, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06038v3",
            "date": "2016-09-20 06:59:31+00:00",
            "title": "Enhanced LSTM for Natural Language Inference",
            "authors": [
                "Qian Chen",
                "Xiaodan Zhu",
                "Zhenhua Ling",
                "Si Wei",
                "Hui Jiang",
                "Diana Inkpen"
            ],
            "abstract": "Reasoning and inference are central to human and artificial intelligence.\nModeling inference in human language is very challenging. With the availability\nof large annotated data (Bowman et al., 2015), it has recently become feasible\nto train neural network based inference models, which have shown to be very\neffective. In this paper, we present a new state-of-the-art result, achieving\nthe accuracy of 88.6% on the Stanford Natural Language Inference Dataset.\nUnlike the previous top models that use very complicated network architectures,\nwe first demonstrate that carefully designing sequential inference models based\non chain LSTMs can outperform all previous models. Based on this, we further\nshow that by explicitly considering recursive architectures in both local\ninference modeling and inference composition, we achieve additional\nimprovement. Particularly, incorporating syntactic parsing information\ncontributes to our best result---it further improves the performance even when\nadded to the already very strong model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06049v1",
            "date": "2016-09-20 08:04:33+00:00",
            "title": "Automatic Quality Assessment for Speech Translation Using Joint ASR and MT Features",
            "authors": [
                "Ngoc-Tien Le",
                "Benjamin Lecouteux",
                "Laurent Besacier"
            ],
            "abstract": "This paper addresses automatic quality assessment of spoken language\ntranslation (SLT). This relatively new task is defined and formalized as a\nsequence labeling problem where each word in the SLT hypothesis is tagged as\ngood or bad according to a large feature set. We propose several word\nconfidence estimators (WCE) based on our automatic evaluation of transcription\n(ASR) quality, translation (MT) quality, or both (combined ASR+MT). This\nresearch work is possible because we built a specific corpus which contains\n6.7k utterances for which a quintuplet containing: ASR output, verbatim\ntranscript, text translation, speech translation and post-edition of\ntranslation is built. The conclusion of our multiple experiments using joint\nASR and MT features for WCE is that MT features remain the most influent while\nASR feature can bring interesting complementary information. Our robust quality\nestimators for SLT can be used for re-scoring speech translation graphs or for\nproviding feedback to the user in interactive speech translation or\ncomputer-assisted speech-to-text scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06082v1",
            "date": "2016-09-20 10:23:47+00:00",
            "title": "Learning Robust Representations of Text",
            "authors": [
                "Yitong Li",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Deep neural networks have achieved remarkable results across many language\nprocessing tasks, however these methods are highly sensitive to noise and\nadversarial attacks. We present a regularization based method for limiting\nnetwork sensitivity to its inputs, inspired by ideas from computer vision, thus\nlearning models that are more robust. Empirical evaluation over a range of\nsentiment datasets with a convolutional neural network shows that, compared to\na baseline model and the dropout method, our method achieves superior\nperformance over noisy inputs and out-of-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06204v2",
            "date": "2016-09-20 14:53:05+00:00",
            "title": "Italy goes to Stanford: a collection of CoreNLP modules for Italian",
            "authors": [
                "Alessio Palmero Aprosio",
                "Giovanni Moretti"
            ],
            "abstract": "In this we paper present Tint, an easy-to-use set of fast, accurate and\nextendable Natural Language Processing modules for Italian. It is based on\nStanford CoreNLP and is freely available as a standalone software or a library\nthat can be integrated in an existing project.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06239v1",
            "date": "2016-09-20 16:19:54+00:00",
            "title": "Generating Politically-Relevant Event Data",
            "authors": [
                "John Beieler"
            ],
            "abstract": "Automatically generated political event data is an important part of the\nsocial science data ecosystem. The approaches for generating this data, though,\nhave remained largely the same for two decades. During this time, the field of\ncomputational linguistics has progressed tremendously. This paper presents an\noverview of political event data, including methods and ontologies, and a set\nof experiments to determine the applicability of deep neural networks to the\nextraction of political events from news text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06490v1",
            "date": "2016-09-21 10:28:57+00:00",
            "title": "One Sentence One Model for Neural Machine Translation",
            "authors": [
                "Xiaoqing Li",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Neural machine translation (NMT) becomes a new state-of-the-art and achieves\npromising translation results using a simple encoder-decoder neural network.\nThis neural network is trained once on the parallel corpus and the fixed\nnetwork is used to translate all the test sentences. We argue that the general\nfixed network cannot best fit the specific test sentences. In this paper, we\npropose the dynamic NMT which learns a general network as usual, and then\nfine-tunes the network for each test sentence. The fine-tune work is done on a\nsmall set of the bilingual training data that is obtained through similarity\nsearch according to the test sentence. Extensive experiments demonstrate that\nthis method can significantly improve the translation performance, especially\nwhen highly similar sentences are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06530v1",
            "date": "2016-09-21 12:43:53+00:00",
            "title": "Weakly supervised spoken term discovery using cross-lingual side information",
            "authors": [
                "Sameer Bansal",
                "Herman Kamper",
                "Sharon Goldwater",
                "Adam Lopez"
            ],
            "abstract": "Recent work on unsupervised term discovery (UTD) aims to identify and cluster\nrepeated word-like units from audio alone. These systems are promising for some\nvery low-resource languages where transcribed audio is unavailable, or where no\nwritten form of the language exists. However, in some cases it may still be\nfeasible (e.g., through crowdsourcing) to obtain (possibly noisy) text\ntranslations of the audio. If so, this information could be used as a source of\nside information to improve UTD. Here, we present a simple method for rescoring\nthe output of a UTD system using text translations, and test it on a corpus of\nSpanish audio with English translations. We show that it greatly improves the\naverage precision of the results over a wide range of system configurations and\ndata preprocessing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06577v1",
            "date": "2016-09-21 14:24:44+00:00",
            "title": "Semi-supervised knowledge extraction for detection of drugs and their effects",
            "authors": [
                "Fabio Del Vigna",
                "Marinella Petrocchi",
                "Alessandro Tommasi",
                "Cesare Zavattari",
                "Maurizio Tesconi"
            ],
            "abstract": "New Psychoactive Substances (NPS) are drugs that lay in a grey area of\nlegislation, since they are not internationally and officially banned, possibly\nleading to their not prosecutable trade. The exacerbation of the phenomenon is\nthat NPS can be easily sold and bought online. Here, we consider large corpora\nof textual posts, published on online forums specialized on drug discussions,\nplus a small set of known substances and associated effects, which we call\nseeds. We propose a semi-supervised approach to knowledge extraction, applied\nto the detection of drugs (comprising NPS) and effects from the corpora under\ninvestigation. Based on the very small set of initial seeds, the work\nhighlights how a contrastive approach and context deduction are effective in\ndetecting substances and effects from the corpora. Our promising results, which\nfeature a F1 score close to 0.9, pave the way for shortening the detection time\nof new psychoactive substances, once these are discussed and advertised on the\nInternet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06649v1",
            "date": "2016-09-21 17:51:11+00:00",
            "title": "Minimally Supervised Written-to-Spoken Text Normalization",
            "authors": [
                "Ke Wu",
                "Kyle Gorman",
                "Richard Sproat"
            ],
            "abstract": "In speech-applications such as text-to-speech (TTS) or automatic speech\nrecognition (ASR), \\emph{text normalization} refers to the task of converting\nfrom a \\emph{written} representation into a representation of how the text is\nto be \\emph{spoken}. In all real-world speech applications, the text\nnormalization engine is developed---in large part---by hand. For example, a\nhand-built grammar may be used to enumerate the possible ways of saying a given\ntoken in a given language, and a statistical model used to select the most\nappropriate pronunciation in context. In this study we examine the tradeoffs\nassociated with using more or less language-specific domain knowledge in a text\nnormalization engine. In the most data-rich scenario, we have access to a\ncarefully constructed hand-built normalization grammar that for any given token\nwill produce a set of all possible verbalizations for that token. We also\nassume a corpus of aligned written-spoken utterances, from which we can train a\nranking model that selects the appropriate verbalization for the given context.\nAs a substitute for the carefully constructed grammar, we also consider a\nscenario with a language-universal normalization \\emph{covering grammar}, where\nthe developer merely needs to provide a set of lexical items particular to the\nlanguage. As a substitute for the aligned corpus, we also consider a scenario\nwhere one only has the spoken side, and the corresponding written side is\n\"hallucinated\" by composing the spoken side with the inverted normalization\ngrammar. We investigate the accuracy of a text normalization engine under each\nof these scenarios. We report the results of experiments on English and\nRussian.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06773v2",
            "date": "2016-09-21 22:48:53+00:00",
            "title": "Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning",
            "authors": [
                "Suyoun Kim",
                "Takaaki Hori",
                "Shinji Watanabe"
            ],
            "abstract": "Recently, there has been an increasing interest in end-to-end speech\nrecognition that directly transcribes speech to text without any predefined\nalignments. One approach is the attention-based encoder-decoder framework that\nlearns a mapping between variable-length input and output sequences in one step\nusing a purely data-driven method. The attention model has often been shown to\nimprove the performance over another end-to-end approach, the Connectionist\nTemporal Classification (CTC), mainly because it explicitly uses the history of\nthe target character without any conditional independence assumptions. However,\nwe observed that the performance of the attention has shown poor results in\nnoisy condition and is hard to learn in the initial training stage with long\ninput sequences. This is because the attention model is too flexible to predict\nproper alignments in such cases due to the lack of left-to-right constraints as\nused in CTC. This paper presents a novel method for end-to-end speech\nrecognition to improve robustness and achieve fast convergence by using a joint\nCTC-attention model within the multi-task learning framework, thereby\nmitigating the alignment issue. An experiment on the WSJ and CHiME-4 tasks\ndemonstrates its advantages over both the CTC and attention-based\nencoder-decoder baselines, showing 5.4-14.6% relative improvements in Character\nError Rate (CER).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07033v1",
            "date": "2016-09-22 15:50:50+00:00",
            "title": "Generating Abstractive Summaries from Meeting Transcripts",
            "authors": [
                "Siddhartha Banerjee",
                "Prasenjit Mitra",
                "Kazunari Sugiyama"
            ],
            "abstract": "Summaries of meetings are very important as they convey the essential content\nof discussions in a concise form. Generally, it is time consuming to read and\nunderstand the whole documents. Therefore, summaries play an important role as\nthe readers are interested in only the important context of discussions. In\nthis work, we address the task of meeting document summarization. Automatic\nsummarization systems on meeting conversations developed so far have been\nprimarily extractive, resulting in unacceptable summaries that are hard to\nread. The extracted utterances contain disfluencies that affect the quality of\nthe extractive summaries. To make summaries much more readable, we propose an\napproach to generating abstractive summaries by fusing important content from\nseveral utterances. We first separate meeting transcripts into various topic\nsegments, and then identify the important utterances in each segment using a\nsupervised learning approach. The important utterances are then combined\ntogether to generate a one-sentence summary. In the text generation step, the\ndependency parses of the utterances in each segment are combined together to\ncreate a directed graph. The most informative and well-formed sub-graph\nobtained by integer linear programming (ILP) is selected to generate a\none-sentence summary for each topic segment. The ILP formulation reduces\ndisfluencies by leveraging grammatical relations that are more prominent in\nnon-conversational style of text, and therefore generates summaries that is\ncomparable to human-written abstractive summaries. Experimental results show\nthat our method can generate more informative summaries than the baselines. In\naddition, readability assessments by human judges as well as log-likelihood\nestimates obtained from the dependency parser show that our generated summaries\nare significantly readable and well-formed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07034v1",
            "date": "2016-09-22 15:51:43+00:00",
            "title": "Multi-document abstractive summarization using ILP based multi-sentence compression",
            "authors": [
                "Siddhartha Banerjee",
                "Prasenjit Mitra",
                "Kazunari Sugiyama"
            ],
            "abstract": "Abstractive summarization is an ideal form of summarization since it can\nsynthesize information from multiple documents to create concise informative\nsummaries. In this work, we aim at developing an abstractive summarizer. First,\nour proposed approach identifies the most important document in the\nmulti-document set. The sentences in the most important document are aligned to\nsentences in other documents to generate clusters of similar sentences. Second,\nwe generate K-shortest paths from the sentences in each cluster using a\nword-graph structure. Finally, we select sentences from the set of shortest\npaths generated from all the clusters employing a novel integer linear\nprogramming (ILP) model with the objective of maximizing information content\nand readability of the final summary. Our ILP model represents the shortest\npaths as binary variables and considers the length of the path, information\nscore and linguistic quality score in the objective function. Experimental\nresults on the DUC 2004 and 2005 multi-document summarization datasets show\nthat our proposed approach outperforms all the baselines and state-of-the-art\nextractive summarizers as measured by the ROUGE scores. Our method also\noutperforms a recent abstractive summarization technique. In manual evaluation,\nour approach also achieves promising results on informativeness and\nreadability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07035v1",
            "date": "2016-09-22 15:53:04+00:00",
            "title": "Abstractive Meeting Summarization UsingDependency Graph Fusion",
            "authors": [
                "Siddhartha Banerjee",
                "Prasenjit Mitra",
                "Kazunari Sugiyama"
            ],
            "abstract": "Automatic summarization techniques on meeting conversations developed so far\nhave been primarily extractive, resulting in poor summaries. To improve this,\nwe propose an approach to generate abstractive summaries by fusing important\ncontent from several utterances. Any meeting is generally comprised of several\ndiscussion topic segments. For each topic segment within a meeting\nconversation, we aim to generate a one sentence summary from the most important\nutterances using an integer linear programming-based sentence fusion approach.\nExperimental results show that our method can generate more informative\nsummaries than the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07053v2",
            "date": "2016-09-22 16:34:00+00:00",
            "title": "Semantic Tagging with Deep Residual Networks",
            "authors": [
                "Johannes Bjerva",
                "Barbara Plank",
                "Johan Bos"
            ],
            "abstract": "We propose a novel semantic tagging task, sem-tagging, tailored for the\npurpose of multilingual semantic parsing, and present the first tagger using\ndeep residual networks (ResNets). Our tagger uses both word and character\nrepresentations and includes a novel residual bypass architecture. We evaluate\nthe tagset both intrinsically on the new task of semantic tagging, as well as\non Part-of-Speech (POS) tagging. Our system, consisting of a ResNet and an\nauxiliary loss function predicting our semantic tags, significantly outperforms\nprior results on English Universal Dependencies POS tagging (95.71% accuracy on\nUD v1.2 and 95.67% accuracy on UD v1.3).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07075v1",
            "date": "2016-09-22 17:16:43+00:00",
            "title": "Knowledge Representation via Joint Learning of Sequential Text and Knowledge Graphs",
            "authors": [
                "Jiawei Wu",
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Textual information is considered as significant supplement to knowledge\nrepresentation learning (KRL). There are two main challenges for constructing\nknowledge representations from plain texts: (1) How to take full advantages of\nsequential contexts of entities in plain texts for KRL. (2) How to dynamically\nselect those informative sentences of the corresponding entities for KRL. In\nthis paper, we propose the Sequential Text-embodied Knowledge Representation\nLearning to build knowledge representations from multiple sentences. Given each\nreference sentence of an entity, we first utilize recurrent neural network with\npooling or long short-term memory network to encode the semantic information of\nthe sentence with respect to the entity. Then we further design an attention\nmodel to measure the informativeness of each sentence, and build text-based\nrepresentations of entities. We evaluate our method on two tasks, including\ntriple classification and link prediction. Experimental results demonstrate\nthat our method outperforms other baselines on both tasks, which indicates that\nour method is capable of selecting informative sentences and encoding the\ntextual information well into knowledge representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07197v2",
            "date": "2016-09-23 00:38:59+00:00",
            "title": "Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problems",
            "authors": [
                "Shyam Upadhyay",
                "Ming-Wei Chang"
            ],
            "abstract": "We propose a new evaluation for automatic solvers for algebra word problems,\nwhich can identify mistakes that existing evaluations overlook. Our proposal is\nto evaluate such solvers using derivations, which reflect how an equation\nsystem was constructed from the word problem. To accomplish this, we develop an\nalgorithm for checking the equivalence between two derivations, and show how\nderivation an- notations can be semi-automatically added to existing datasets.\nTo make our experiments more comprehensive, we include the derivation\nannotation for DRAW-1K, a new dataset containing 1000 general algebra word\nproblems. In our experiments, we found that the annotated derivations enable a\nmore accurate evaluation of automatic solvers than previously used metrics. We\nrelease derivation annotations for over 2300 algebra word problems for future\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07222v1",
            "date": "2016-09-23 03:35:27+00:00",
            "title": "Deep Multi-Task Learning with Shared Memory",
            "authors": [
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural network based models have achieved impressive results on various\nspecific tasks. However, in previous works, most models are learned separately\nbased on single-task supervised objectives, which often suffer from\ninsufficient training data. In this paper, we propose two deep architectures\nwhich can be trained jointly on multiple related tasks. More specifically, we\naugment neural model with an external memory, which is shared by several tasks.\nExperiments on two groups of text classification tasks show that our proposed\narchitectures can improve the performance of a task with the help of other\nrelated tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07451v1",
            "date": "2016-09-23 18:12:12+00:00",
            "title": "AMR-to-text generation as a Traveling Salesman Problem",
            "authors": [
                "Linfeng Song",
                "Yue Zhang",
                "Xiaochang Peng",
                "Zhiguo Wang",
                "Daniel Gildea"
            ],
            "abstract": "The task of AMR-to-text generation is to generate grammatical text that\nsustains the semantic meaning for a given AMR graph. We at- tack the task by\nfirst partitioning the AMR graph into smaller fragments, and then generating\nthe translation for each fragment, before finally deciding the order by solving\nan asymmetric generalized traveling salesman problem (AGTSP). A Maximum Entropy\nclassifier is trained to estimate the traveling costs, and a TSP solver is used\nto find the optimized solution. The final model reports a BLEU score of 22.44\non the SemEval-2016 Task8 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07479v3",
            "date": "2016-09-23 19:59:51+00:00",
            "title": "Incorporating Relation Paths in Neural Relation Extraction",
            "authors": [
                "Wenyuan Zeng",
                "Yankai Lin",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Distantly supervised relation extraction has been widely used to find novel\nrelational facts from plain text. To predict the relation between a pair of two\ntarget entities, existing methods solely rely on those direct sentences\ncontaining both entities. In fact, there are also many sentences containing\nonly one of the target entities, which provide rich and useful information for\nrelation extraction. To address this issue, we build inference chains between\ntwo target entities via intermediate entities, and propose a path-based neural\nrelation extraction model to encode the relational semantics from both direct\nsentences and inference chains. Experimental results on real-world datasets\nshow that, our model can make full use of those sentences containing only one\ntarget entity, and achieves significant and consistent improvements on relation\nextraction as compared with baselines. The source code of this paper can be\nobtained from https: //github.com/thunlp/PathNRE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07561v1",
            "date": "2016-09-24 02:58:26+00:00",
            "title": "Distilling an Ensemble of Greedy Dependency Parsers into One MST Parser",
            "authors": [
                "Adhiguna Kuncoro",
                "Miguel Ballesteros",
                "Lingpeng Kong",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We introduce two first-order graph-based dependency parsers achieving a new\nstate of the art. The first is a consensus parser built from an ensemble of\nindependently trained greedy LSTM transition-based parsers with different\nrandom initializations. We cast this approach as minimum Bayes risk decoding\n(under the Hamming cost) and argue that weaker consensus within the ensemble is\na useful signal of difficulty or ambiguity. The second parser is a\n\"distillation\" of the ensemble into a single model. We train the distillation\nparser using a structured hinge loss objective with a novel cost that\nincorporates ensemble uncertainty estimates for each possible attachment,\nthereby avoiding the intractable cross-entropy computations required by\napplying standard distillation objectives to problems with structured outputs.\nThe first-order distillation parser matches or surpasses the state of the art\non English, Chinese, and German.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07568v1",
            "date": "2016-09-24 04:02:13+00:00",
            "title": "A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects",
            "authors": [
                "Yonatan Belinkov",
                "James Glass"
            ],
            "abstract": "Discriminating between closely-related language varieties is considered a\nchallenging and important task. This paper describes our submission to the DSL\n2016 shared-task, which included two sub-tasks: one on discriminating similar\nlanguages and one on identifying Arabic dialects. We developed a\ncharacter-level neural network for this task. Given a sequence of characters,\nour model embeds each character in vector space, runs the sequence through\nmultiple convolutions with different filter widths, and pools the convolutional\nrepresentations to obtain a hidden vector representation of the text that is\nused for predicting the language or dialect. We primarily focused on the Arabic\ndialect identification task and obtained an F1 score of 0.4834, ranking 6th out\nof 18 participants. We also analyze errors made by our system on the Arabic\ndata in some detail, and point to challenges such an approach is faced with.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07585v1",
            "date": "2016-09-24 08:45:17+00:00",
            "title": "An Investigation of Recurrent Neural Architectures for Drug Name Recognition",
            "authors": [
                "Raghavendra Chalapathy",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "Drug name recognition (DNR) is an essential step in the Pharmacovigilance\n(PV) pipeline. DNR aims to find drug name mentions in unstructured biomedical\ntexts and classify them into predefined categories. State-of-the-art DNR\napproaches heavily rely on hand crafted features and domain specific resources\nwhich are difficult to collect and tune. For this reason, this paper\ninvestigates the effectiveness of contemporary recurrent neural architectures -\nthe Elman and Jordan networks and the bidirectional LSTM with CRF decoding - at\nperforming DNR straight from the text. The experimental results achieved on the\nauthoritative SemEval-2013 Task 9.1 benchmarks show that the bidirectional\nLSTM-CRF ranks closely to highly-dedicated, hand-crafted systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07681v1",
            "date": "2016-09-24 23:36:31+00:00",
            "title": "The distribution of information content in English sentences",
            "authors": [
                "Shuiyuan Yu",
                "Jin Cong",
                "Junying Liang",
                "Haitao Liu"
            ],
            "abstract": "Sentence is a basic linguistic unit, however, little is known about how\ninformation content is distributed across different positions of a sentence.\nBased on authentic language data of English, the present study calculated the\nentropy and other entropy-related statistics for different sentence positions.\nThe statistics indicate a three-step staircase-shaped distribution pattern,\nwith entropy in the initial position lower than the medial positions (positions\nother than the initial and final), the medial positions lower than the final\nposition and the medial positions showing no significant difference. The\nresults suggest that: (1) the hypotheses of Constant Entropy Rate and Uniform\nInformation Density do not hold for the sentence-medial positions; (2) the\ncontext of a word in a sentence should not be simply defined as all the words\npreceding it in the same sentence; and (3) the contextual information content\nin a sentence does not accumulate incrementally but follows a pattern of \"the\nwhole is greater than the sum of parts\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07701v1",
            "date": "2016-09-25 05:07:55+00:00",
            "title": "Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results",
            "authors": [
                "Yonatan Belinkov",
                "James Glass"
            ],
            "abstract": "Machine translation between Arabic and Hebrew has so far been limited by a\nlack of parallel corpora, despite the political and cultural importance of this\nlanguage pair. Previous work relied on manually-crafted grammars or pivoting\nvia English, both of which are unsatisfactory for building a scalable and\naccurate MT system. In this work, we compare standard phrase-based and neural\nsystems on Arabic-Hebrew translation. We experiment with tokenization by\nexternal tools and sub-word modeling by character-level neural models, and show\nthat both methods lead to improved translation performance, with a small\nadvantage to the neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07730v2",
            "date": "2016-09-25 10:59:01+00:00",
            "title": "Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation",
            "authors": [
                "Jinsong Su",
                "Zhixing Tan",
                "Deyi Xiong",
                "Rongrong Ji",
                "Xiaodong Shi",
                "Yang Liu"
            ],
            "abstract": "Neural machine translation (NMT) heavily relies on word-level modelling to\nlearn semantic representations of input sentences. However, for languages\nwithout natural word delimiters (e.g., Chinese) where input sentences have to\nbe tokenized first, conventional NMT is confronted with two issues: 1) it is\ndifficult to find an optimal tokenization granularity for source sentence\nmodelling, and 2) errors in 1-best tokenizations may propagate to the encoder\nof NMT. To handle these issues, we propose word-lattice based Recurrent Neural\nNetwork (RNN) encoders for NMT, which generalize the standard RNN to word\nlattice topology. The proposed encoders take as input a word lattice that\ncompactly encodes multiple tokenizations, and learn to generate new hidden\nstates from arbitrarily many inputs and hidden states in preceding time steps.\nAs such, the word-lattice based encoders not only alleviate the negative impact\nof tokenization errors but also are more expressive and flexible to embed input\nsentences. Experiment results on Chinese-English translation demonstrate the\nsuperiorities of the proposed encoders over the conventional encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07756v1",
            "date": "2016-09-25 15:10:16+00:00",
            "title": "A Factorized Model for Transitive Verbs in Compositional Distributional Semantics",
            "authors": [
                "Lilach Edelstein",
                "Roi Reichart"
            ],
            "abstract": "We present a factorized compositional distributional semantics model for the\nrepresentation of transitive verb constructions. Our model first produces\n(subject, verb) and (verb, object) vector representations based on the\nsimilarity of the nouns in the construction to each of the nouns in the\nvocabulary and the tendency of these nouns to take the subject and object roles\nof the verb. These vectors are then combined into a final (subject,verb,object)\nrepresentation through simple vector operations. On two established tasks for\nthe transitive verb construction our model outperforms recent previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08075v1",
            "date": "2016-09-26 17:01:03+00:00",
            "title": "S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking",
            "authors": [
                "Yi Yang",
                "Ming-Wei Chang"
            ],
            "abstract": "Non-linear models recently receive a lot of attention as people are starting\nto discover the power of statistical and embedding features. However,\ntree-based models are seldom studied in the context of structured learning\ndespite their recent success on various classification and ranking tasks. In\nthis paper, we propose S-MART, a tree-based structured learning framework based\non multiple additive regression trees. S-MART is especially suitable for\nhandling tasks with dense features, and can be used to learn many different\nstructures under various loss functions.\n  We apply S-MART to the task of tweet entity linking --- a core component of\ntweet information extraction, which aims to identify and link name mentions to\nentities in a knowledge base. A novel inference algorithm is proposed to handle\nthe special structure of the task. The experimental results show that S-MART\nsignificantly outperforms state-of-the-art tweet entity linking systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08084v1",
            "date": "2016-09-26 17:19:07+00:00",
            "title": "Toward Socially-Infused Information Extraction: Embedding Authors, Mentions, and Entities",
            "authors": [
                "Yi Yang",
                "Ming-Wei Chang",
                "Jacob Eisenstein"
            ],
            "abstract": "Entity linking is the task of identifying mentions of entities in text, and\nlinking them to entries in a knowledge base. This task is especially difficult\nin microblogs, as there is little additional text to provide disambiguating\ncontext; rather, authors rely on an implicit common ground of shared knowledge\nwith their readers. In this paper, we attempt to capture some of this implicit\ncontext by exploiting the social network structure in microblogs. We build on\nthe theory of homophily, which implies that socially linked individuals share\ninterests, and are therefore likely to mention the same sorts of entities. We\nimplement this idea by encoding authors, mentions, and entities in a continuous\nvector space, which is constructed so that socially-connected authors have\nsimilar vector representations. These vectors are incorporated into a neural\nstructured prediction model, which captures structural constraints that are\ninherent in the entity linking task. Together, these design decisions yield F1\nimprovements of 1%-5% on benchmark datasets, as compared to the previous\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08097v1",
            "date": "2016-09-26 17:50:15+00:00",
            "title": "Creating Causal Embeddings for Question Answering with Minimal Supervision",
            "authors": [
                "Rebecca Sharp",
                "Mihai Surdeanu",
                "Peter Jansen",
                "Peter Clark",
                "Michael Hammond"
            ],
            "abstract": "A common model for question answering (QA) is that a good answer is one that\nis closely related to the question, where relatedness is often determined using\ngeneral-purpose lexical models such as word embeddings. We argue that a better\napproach is to look for answers that are related to the question in a relevant\nway, according to the information need of the question, which may be determined\nthrough task-specific embeddings. With causality as a use case, we implement\nthis insight in three steps. First, we generate causal embeddings\ncost-effectively by bootstrapping cause-effect pairs extracted from free text\nusing a small set of seed patterns. Second, we train dedicated embeddings over\nthis data, by using task-specific contexts, i.e., the context of a cause is its\neffect. Finally, we extend a state-of-the-art reranking approach for QA to\nincorporate these causal embeddings. We evaluate the causal embedding models\nboth directly with a casual implication task, and indirectly, in a downstream\ncausal QA task using data from Yahoo! Answers. We show that explicitly modeling\ncausality improves performance in both tasks. In the QA task our best model\nachieves 37.3% P@1, significantly outperforming a strong baseline by 7.7%\n(relative).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08139v1",
            "date": "2016-09-26 19:50:59+00:00",
            "title": "An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languages",
            "authors": [
                "Antonios Anastasopoulos",
                "David Chiang",
                "Long Duong"
            ],
            "abstract": "For many low-resource languages, spoken language resources are more likely to\nbe annotated with translations than with transcriptions. Translated speech data\nis potentially valuable for documenting endangered languages or for training\nspeech translation systems. A first step towards making use of such data would\nbe to automatically align spoken words with their translations. We present a\nmodel that combines Dyer et al.'s reparameterization of IBM Model 2\n(fast-align) and k-means clustering using Dynamic Time Warping as a distance\nmetric. The two components are trained jointly using expectation-maximization.\nIn an extremely low-resource scenario, our model performs significantly better\nthan both a neural model and a strong baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08237v1",
            "date": "2016-09-27 01:19:41+00:00",
            "title": "Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment",
            "authors": [
                "Tao Ge",
                "Qing Dou",
                "Xiaoman Pan",
                "Heng Ji",
                "Lei Cui",
                "Baobao Chang",
                "Zhifang Sui",
                "Ming Zhou"
            ],
            "abstract": "Aligning coordinated text streams from multiple sources and multiple\nlanguages has opened many new research venues on cross-lingual knowledge\ndiscovery. In this paper we aim to advance state-of-the-art by: (1). extending\ncoarse-grained topic-level knowledge mining to fine-grained information units\nsuch as entities and events; (2). following a novel\nData-to-Network-to-Knowledge (D2N2K) paradigm to construct and utilize network\nstructures to capture and propagate reliable evidence. We introduce a novel\nBurst Information Network (BINet) representation that can display the most\nimportant information and illustrate the connections among bursty entities,\nevents and keywords in the corpus. We propose an effective approach to\nconstruct and decipher BINets, incorporating novel criteria based on\nmulti-dimensional clues from pronunciation, translation, burst, neighbor and\ngraph topological structure. The experimental results on Chinese and English\ncoordinated text streams show that our approach can accurately decipher the\nnodes with high confidence in the BINets and that the algorithm can be\nefficiently run in parallel, which makes it possible to apply it to huge\namounts of streaming data for never-ending language and information\ndecipherment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08293v1",
            "date": "2016-09-27 07:38:29+00:00",
            "title": "The Effects of Data Size and Frequency Range on Distributional Semantic Models",
            "authors": [
                "Magnus Sahlgren",
                "Alessandro Lenci"
            ],
            "abstract": "This paper investigates the effects of data size and frequency range on\ndistributional semantic models. We compare the performance of a number of\nrepresentative models for several test settings over data of varying sizes, and\nover test items of various frequency. Our results show that neural\nnetwork-based models underperform when the data is small, and that the most\nreliable model over data of varying sizes and frequency ranges is the inverted\nfactorized model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08359v2",
            "date": "2016-09-27 11:32:25+00:00",
            "title": "emoji2vec: Learning Emoji Representations from their Description",
            "authors": [
                "Ben Eisner",
                "Tim Rockt\u00e4schel",
                "Isabelle Augenstein",
                "Matko Bo\u0161njak",
                "Sebastian Riedel"
            ],
            "abstract": "Many current natural language processing applications for social media rely\non representation learning and utilize pre-trained word embeddings. There\ncurrently exist several publicly-available, pre-trained sets of word\nembeddings, but they contain few or no emoji representations even as emoji\nusage in social media has increased. In this paper we release emoji2vec,\npre-trained embeddings for all Unicode emoji which are learned from their\ndescription in the Unicode emoji standard. The resulting emoji embeddings can\nbe readily used in downstream social natural language processing applications\nalongside word2vec. We demonstrate, for the downstream task of sentiment\nanalysis, that emoji embeddings learned from short descriptions outperforms a\nskip-gram model trained on a large collection of tweets, while avoiding the\nneed for contexts in which emoji need to appear frequently in order to estimate\na representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08412v1",
            "date": "2016-09-27 13:25:51+00:00",
            "title": "OC16-CE80: A Chinese-English Mixlingual Database and A Speech Recognition Baseline",
            "authors": [
                "Dong Wang",
                "Zhiyuan Tang",
                "Difei Tang",
                "Qing Chen"
            ],
            "abstract": "We present the OC16-CE80 Chinese-English mixlingual speech database which was\nreleased as a main resource for training, development and test for the\nChinese-English mixlingual speech recognition (MixASR-CHEN) challenge on\nO-COCOSDA 2016. This database consists of 80 hours of speech signals recorded\nfrom more than 1,400 speakers, where the utterances are in Chinese but each\ninvolves one or several English words. Based on the database and another two\nfree data resources (THCHS30 and the CMU dictionary), a speech recognition\n(ASR) baseline was constructed with the deep neural network-hidden Markov model\n(DNN-HMM) hybrid system. We then report the baseline results following the\nMixASR-CHEN evaluation rules and demonstrate that OC16-CE80 is a reasonable\ndata resource for mixlingual research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08667v3",
            "date": "2016-09-27 21:00:26+00:00",
            "title": "Deep Reinforcement Learning for Mention-Ranking Coreference Models",
            "authors": [
                "Kevin Clark",
                "Christopher D. Manning"
            ],
            "abstract": "Coreference resolution systems are typically trained with heuristic loss\nfunctions that require careful tuning. In this paper we instead apply\nreinforcement learning to directly optimize a neural mention-ranking model for\ncoreference evaluation metrics. We experiment with two approaches: the\nREINFORCE policy gradient algorithm and a reward-rescaled max-margin objective.\nWe find the latter to be more effective, resulting in significant improvements\nover the current state-of-the-art on the English and Chinese portions of the\nCoNLL 2012 Shared Task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08777v2",
            "date": "2016-09-28 05:41:18+00:00",
            "title": "Character Sequence Models for ColorfulWords",
            "authors": [
                "Kazuya Kawakami\u007f",
                "Chris Dyer\u007f",
                "Bryan R. Routledge",
                "Noah A. Smith"
            ],
            "abstract": "We present a neural network architecture to predict a point in color space\nfrom the sequence of characters in the color's name. Using large scale\ncolor--name pairs obtained from an online color design forum, we evaluate our\nmodel on a \"color Turing test\" and find that, given a name, the colors\npredicted by our model are preferred by annotators to color names created by\nhumans. Our datasets and demo system are available online at colorlab.us.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08810v2",
            "date": "2016-09-28 08:11:28+00:00",
            "title": "Effective Combination of Language and Vision Through Model Composition and the R-CCA Method",
            "authors": [
                "Hagar Loeub",
                "Roi Reichart"
            ],
            "abstract": "We address the problem of integrating textual and visual information in\nvector space models for word meaning representation. We first present the\nResidual CCA (R-CCA) method, that complements the standard CCA method by\nrepresenting, for each modality, the difference between the original signal and\nthe signal projected to the shared, max correlation, space. We then show that\nconstructing visual and textual representations and then post-processing them\nthrough composition of common modeling motifs such as PCA, CCA, R-CCA and\nlinear interpolation (a.k.a sequential modeling) yields high quality models. On\nfive standard semantic benchmarks our sequential models outperform recent\nmultimodal representation learning alternatives, including ones that rely on\njoint representation learning. For two of these benchmarks our R-CCA method is\npart of the Best configuration our algorithm yields.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08824v1",
            "date": "2016-09-28 08:54:05+00:00",
            "title": "Equation Parsing: Mapping Sentences to Grounded Equations",
            "authors": [
                "Subhro Roy",
                "Shyam Upadhyay",
                "Dan Roth"
            ],
            "abstract": "Identifying mathematical relations expressed in text is essential to\nunderstanding a broad range of natural language text from election reports, to\nfinancial news, to sport commentaries to mathematical word problems. This paper\nfocuses on identifying and understanding mathematical relations described\nwithin a single sentence. We introduce the problem of Equation Parsing -- given\na sentence, identify noun phrases which represent variables, and generate the\nmathematical equation expressing the relation described in the sentence. We\nintroduce the notion of projective equation parsing and provide an efficient\nalgorithm to parse text to projective equations. Our system makes use of a high\nprecision lexicon of mathematical expressions and a pipeline of structured\npredictors, and generates correct equations in $70\\%$ of the cases. In $60\\%$\nof the time, it also identifies the correct noun phrase $\\rightarrow$ variables\nmapping, significantly outperforming baselines. We also release a new annotated\ndataset for task evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09004v2",
            "date": "2016-09-28 16:51:56+00:00",
            "title": "Byte-based Language Identification with Deep Convolutional Networks",
            "authors": [
                "Johannes Bjerva"
            ],
            "abstract": "We report on our system for the shared task on discriminating between similar\nlanguages (DSL 2016). The system uses only byte representations in a deep\nresidual network (ResNet). The system, named ResIdent, is trained only on the\ndata released with the task (closed training). We obtain 84.88% accuracy on\nsubtask A, 68.80% accuracy on subtask B1, and 69.80% accuracy on subtask B2. A\nlarge difference in accuracy on development data can be observed with\nrelatively minor changes in our network's architecture and hyperparameters. We\ntherefore expect fine-tuning of these parameters to yield higher accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09019v1",
            "date": "2016-09-28 17:58:23+00:00",
            "title": "Psychologically Motivated Text Mining",
            "authors": [
                "Ekaterina Shutova",
                "Patricia Lichtenstein"
            ],
            "abstract": "Natural language processing techniques are increasingly applied to identify\nsocial trends and predict behavior based on large text collections. Existing\nmethods typically rely on surface lexical and syntactic information. Yet,\nresearch in psychology shows that patterns of human conceptualisation, such as\nmetaphorical framing, are reliable predictors of human expectations and\ndecisions. In this paper, we present a method to learn patterns of metaphorical\nframing from large text collections, using statistical techniques. We apply the\nmethod to data in three different languages and evaluate the identified\npatterns, demonstrating their psychological validity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09171v2",
            "date": "2016-09-29 01:53:08+00:00",
            "title": "Empirical Evaluation of RNN Architectures on Sentence Classification Task",
            "authors": [
                "Lei Shen",
                "Junlin Zhang"
            ],
            "abstract": "Recurrent Neural Networks have achieved state-of-the-art results for many\nproblems in NLP and two most popular RNN architectures are Tail Model and\nPooling Model. In this paper, a hybrid architecture is proposed and we present\nthe first empirical study using LSTMs to compare performance of the three RNN\nstructures on sentence classification task. Experimental results show that the\nMax Pooling Model or Hybrid Max Pooling Model achieves the best performance on\nmost datasets, while Tail Model does not outperform other models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09189v2",
            "date": "2016-09-29 03:26:53+00:00",
            "title": "Learning Sentence Representation with Guidance of Human Attention",
            "authors": [
                "Shaonan Wang",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Recently, much progress has been made in learning general-purpose sentence\nrepresentations that can be used across domains. However, most of the existing\nmodels typically treat each word in a sentence equally. In contrast, extensive\nstudies have proven that human read sentences efficiently by making a sequence\nof fixation and saccades. This motivates us to improve sentence representations\nby assigning different weights to the vectors of the component words, which can\nbe treated as an attention mechanism on single sentences. To that end, we\npropose two novel attention models, in which the attention weights are derived\nusing significant predictors of human reading time, i.e., Surprisal, POS tags\nand CCG supertags. The extensive experiments demonstrate that the proposed\nmethods significantly improve upon the state-of-the-art sentence representation\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09382v1",
            "date": "2016-09-29 15:19:13+00:00",
            "title": "Inducing Multilingual Text Analysis Tools Using Bidirectional Recurrent Neural Networks",
            "authors": [
                "Othman Zennaki",
                "Nasredine Semmar",
                "Laurent Besacier"
            ],
            "abstract": "This work focuses on the rapid development of linguistic annotation tools for\nresource-poor languages. We experiment several cross-lingual annotation\nprojection methods using Recurrent Neural Networks (RNN) models. The\ndistinctive feature of our approach is that our multilingual word\nrepresentation requires only a parallel corpus between the source and target\nlanguage. More precisely, our method has the following characteristics: (a) it\ndoes not use word alignment information, (b) it does not assume any knowledge\nabout foreign languages, which makes it applicable to a wide range of\nresource-poor languages, (c) it provides truly multilingual taggers. We\ninvestigate both uni- and bi-directional RNN models and propose a method to\ninclude external information (for instance low level information from POS) in\nthe RNN to train higher level taggers (for instance, super sense taggers). We\ndemonstrate the validity and genericity of our model by using parallel corpora\n(obtained by manual or automatic translation). Our experiments are conducted to\ninduce cross-lingual POS and super sense taggers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09552v1",
            "date": "2016-09-30 00:01:27+00:00",
            "title": "Controlling Output Length in Neural Encoder-Decoders",
            "authors": [
                "Yuta Kikuchi",
                "Graham Neubig",
                "Ryohei Sasano",
                "Hiroya Takamura",
                "Manabu Okumura"
            ],
            "abstract": "Neural encoder-decoder models have shown great success in many sequence\ngeneration tasks. However, previous work has not investigated situations in\nwhich we would like to control the length of encoder-decoder outputs. This\ncapability is crucial for applications such as text summarization, in which we\nhave to generate concise summaries with a desired length. In this paper, we\npropose methods for controlling the output sequence length for neural\nencoder-decoder models: two decoding-based methods and two learning-based\nmethods. Results show that our learning-based methods have the capability to\ncontrol length without degrading summary quality in a summarization task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09580v1",
            "date": "2016-09-30 03:20:52+00:00",
            "title": "Referential Uncertainty and Word Learning in High-dimensional, Continuous Meaning Spaces",
            "authors": [
                "Michael Spranger",
                "Katrien Beuls"
            ],
            "abstract": "This paper discusses lexicon word learning in high-dimensional meaning spaces\nfrom the viewpoint of referential uncertainty. We investigate various\nstate-of-the-art Machine Learning algorithms and discuss the impact of scaling,\nrepresentation and meaning space structure. We demonstrate that current Machine\nLearning techniques successfully deal with high-dimensional meaning spaces. In\nparticular, we show that exponentially increasing dimensions linearly impact\nlearner performance and that referential uncertainty from word sensitivity has\nno impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00030v1",
            "date": "2016-09-30 20:57:01+00:00",
            "title": "Modeling Language Change in Historical Corpora: The Case of Portuguese",
            "authors": [
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Mark Dras"
            ],
            "abstract": "This paper presents a number of experiments to model changes in a historical\nPortuguese corpus composed of literary texts for the purpose of temporal text\nclassification. Algorithms were trained to classify texts with respect to their\npublication date taking into account lexical variation represented as word\nn-grams, and morphosyntactic variation represented by part-of-speech (POS)\ndistribution. We report results of 99.8% accuracy using word unigram features\nwith a Support Vector Machines classifier to predict the publication date of\ndocuments in time intervals of both one century and half a century. A feature\nanalysis is performed to investigate the most informative features for this\ntask and how they are linked to language change.",
            "journal ref": "Proceedings of Language Resources and Evaluation (LREC). Portoroz,\n  Slovenia. pp. 4098-4104 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00031v1",
            "date": "2016-09-30 20:57:52+00:00",
            "title": "Discriminating Similar Languages: Evaluations and Explorations",
            "authors": [
                "Cyril Goutte",
                "Serge L\u00e9ger",
                "Shervin Malmasi",
                "Marcos Zampieri"
            ],
            "abstract": "We present an analysis of the performance of machine learning classifiers on\ndiscriminating between similar languages and language varieties. We carried out\na number of experiments using the results of the two editions of the\nDiscriminating between Similar Languages (DSL) shared task. We investigate the\nprogress made between the two tasks, estimate an upper bound on possible\nperformance using ensemble and oracle combination, and provide learning curves\nto help us understand which languages are more challenging. A number of\ndifficult sentences are identified and investigated further with human\nannotation.",
            "journal ref": "Proceedings of Language Resources and Evaluation (LREC). Portoroz,\n  Slovenia. pp 1800-1807 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00072v1",
            "date": "2016-10-01 02:23:03+00:00",
            "title": "Vocabulary Selection Strategies for Neural Machine Translation",
            "authors": [
                "Gurvan L'Hostis",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "Classical translation models constrain the space of possible outputs by\nselecting a subset of translation rules based on the input sentence. Recent\nwork on improving the efficiency of neural translation models adopted a similar\nstrategy by restricting the output vocabulary to a subset of likely candidates\ngiven the source. In this paper we experiment with context and embedding-based\nselection methods and extend previous work by examining speed and accuracy\ntrade-offs in more detail. We show that decoding time on CPUs can be reduced by\nup to 90% and training time by 25% on the WMT15 English-German and WMT16\nEnglish-Romanian tasks at the same or only negligible change in accuracy. This\nbrings the time to decode with a state of the art neural translation system to\njust over 140 msec per sentence on a single CPU core for English-German.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00211v2",
            "date": "2016-10-02 00:49:15+00:00",
            "title": "Sentence Segmentation in Narrative Transcripts from Neuropsychological Tests using Recurrent Convolutional Neural Networks",
            "authors": [
                "Marcos Vin\u00edcius Treviso",
                "Christopher Shulby",
                "Sandra Maria Alu\u00edsio"
            ],
            "abstract": "Automated discourse analysis tools based on Natural Language Processing (NLP)\naiming at the diagnosis of language-impairing dementias generally extract\nseveral textual metrics of narrative transcripts. However, the absence of\nsentence boundary segmentation in the transcripts prevents the direct\napplication of NLP methods which rely on these marks to function properly, such\nas taggers and parsers. We present the first steps taken towards automatic\nneuropsychological evaluation based on narrative discourse analysis, presenting\na new automatic sentence segmentation method for impaired speech. Our model\nuses recurrent convolutional neural networks with prosodic, Part of Speech\n(PoS) features, and word embeddings. It was evaluated intrinsically on\nimpaired, spontaneous speech, as well as, normal, prepared speech, and presents\nbetter results for healthy elderly (CTL) (F1 = 0.74) and Mild Cognitive\nImpairment (MCI) patients (F1 = 0.70) than the Conditional Random Fields method\n(F1 = 0.55 and 0.53, respectively) used in the same context of our study. The\nresults suggest that our model is robust for impaired speech and can be used in\nautomated discourse analysis tools to differentiate narratives produced by MCI\nand CTL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00277v1",
            "date": "2016-10-02 13:29:14+00:00",
            "title": "Very Deep Convolutional Neural Networks for Robust Speech Recognition",
            "authors": [
                "Yanmin Qian",
                "Philip C Woodland"
            ],
            "abstract": "This paper describes the extension and optimization of our previous work on\nvery deep convolutional neural networks (CNNs) for effective recognition of\nnoisy speech in the Aurora 4 task. The appropriate number of convolutional\nlayers, the sizes of the filters, pooling operations and input feature maps are\nall modified: the filter and pooling sizes are reduced and dimensions of input\nfeature maps are extended to allow adding more convolutional layers.\nFurthermore appropriate input padding and input feature map selection\nstrategies are developed. In addition, an adaptation framework using joint\ntraining of very deep CNN with auxiliary features i-vector and fMLLR features\nis developed. These modifications give substantial word error rate reductions\nover the standard CNN used as baseline. Finally the very deep CNN is combined\nwith an LSTM-RNN acoustic model and it is shown that state-level weighted log\nlikelihood score combination in a joint acoustic model decoding scheme is very\neffective. On the Aurora 4 task, the very deep CNN achieves a WER of 8.81%,\nfurther 7.99% with auxiliary feature joint training, and 7.09% with LSTM-RNN\njoint decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00311v1",
            "date": "2016-10-02 16:54:41+00:00",
            "title": "Syntactic Structures and Code Parameters",
            "authors": [
                "Kevin Shu",
                "Matilde Marcolli"
            ],
            "abstract": "We assign binary and ternary error-correcting codes to the data of syntactic\nstructures of world languages and we study the distribution of code points in\nthe space of code parameters. We show that, while most codes populate the lower\nregion approximating a superposition of Thomae functions, there is a\nsubstantial presence of codes above the Gilbert-Varshamov bound and even above\nthe asymptotic bound and the Plotkin bound. We investigate the dynamics induced\non the space of code parameters by spin glass models of language change, and\nshow that, in the presence of entailment relations between syntactic parameters\nthe dynamics can sometimes improve the code. For large sets of languages and\nsyntactic data, one can gain information on the spin glass dynamics from the\ninduced dynamics in the space of code parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00479v3",
            "date": "2016-10-03 10:30:13+00:00",
            "title": "Nonsymbolic Text Representation",
            "authors": [
                "Hinrich Schuetze",
                "Heike Adel",
                "Ehsaneddin Asgari"
            ],
            "abstract": "We introduce the first generic text representation model that is completely\nnonsymbolic, i.e., it does not require the availability of a segmentation or\ntokenization method that attempts to identify words or other symbolic units in\ntext. This applies to training the parameters of the model on a training corpus\nas well as to applying it when computing the representation of a new text. We\nshow that our model performs better than prior work on an information\nextraction and a text denoising task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00602v1",
            "date": "2016-10-03 15:40:08+00:00",
            "title": "Multimodal Semantic Simulations of Linguistically Underspecified Motion Events",
            "authors": [
                "Nikhil Krishnaswamy",
                "James Pustejovsky"
            ],
            "abstract": "In this paper, we describe a system for generating three-dimensional visual\nsimulations of natural language motion expressions. We use a rich formal model\nof events and their participants to generate simulations that satisfy the\nminimal constraints entailed by the associated utterance, relying on semantic\nknowledge of physical objects and motion events. This paper outlines technical\nconsiderations and discusses implementing the aforementioned semantic models\ninto such a system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00634v1",
            "date": "2016-10-03 16:53:10+00:00",
            "title": "Orthographic Syllable as basic unit for SMT between Related Languages",
            "authors": [
                "Anoop Kunchukuttan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "We explore the use of the orthographic syllable, a variable-length\nconsonant-vowel sequence, as a basic unit of translation between related\nlanguages which use abugida or alphabetic scripts. We show that orthographic\nsyllable level translation significantly outperforms models trained over other\nbasic units (word, morpheme and character) when training over small parallel\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00765v2",
            "date": "2016-10-03 21:50:27+00:00",
            "title": "Distributed Representations of Lexical Sets and Prototypes in Causal Alternation Verbs",
            "authors": [
                "Edoardo Maria Ponti",
                "Elisabetta Jezek",
                "Bernardo Magnini"
            ],
            "abstract": "Lexical sets contain the words filling an argument slot of a verb, and are in\npart determined by selectional preferences. The purpose of this paper is to\nunravel the properties of lexical sets through distributional semantics. We\ninvestigate 1) whether lexical set behave as prototypical categories with a\ncentre and a periphery; 2) whether they are polymorphic, i.e. composed by\nsubcategories; 3) whether the distance between lexical sets of different\narguments is explanatory of verb properties. In particular, our case study are\nlexical sets of causative-inchoative verbs in Italian. Having studied several\nvector models, we find that 1) based on spatial distance from the centroid,\nobject fillers are scattered uniformly across the category, whereas\nintransitive subject fillers lie on its edge; 2) a correlation exists between\nthe amount of verb senses and that of clusters discovered automatically,\nespecially for intransitive subjects; 3) the distance between the centroids of\nobject and intransitive subject is correlated with other properties of verbs,\nsuch as their cross-lingual tendency to appear in the intransitive pattern\nrather than transitive one. This paper is noncommittal with respect to the\nhypothesis that this connection is underpinned by a semantic reason, namely the\nspontaneity of the event denoted by the verb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00842v1",
            "date": "2016-10-04 04:56:23+00:00",
            "title": "Chinese Event Extraction Using DeepNeural Network with Word Embedding",
            "authors": [
                "Yandi Xia",
                "Yang Liu"
            ],
            "abstract": "A lot of prior work on event extraction has exploited a variety of features\nto represent events. Such methods have several drawbacks: 1) the features are\noften specific for a particular domain and do not generalize well; 2) the\nfeatures are derived from various linguistic analyses and are error-prone; and\n3) some features may be expensive and require domain expert. In this paper, we\ndevelop a Chinese event extraction system that uses word embedding vectors to\nrepresent language, and deep neural networks to learn the abstract feature\nrepresentation in order to greatly reduce the effort of feature engineering. In\naddition, in this framework, we leverage large amount of unlabeled data, which\ncan address the problem of limited labeled corpus for this task. Our\nexperiments show that our proposed method performs better compared to the\nsystem using rich language features, and using unlabeled data benefits the word\nembeddings. This study suggests the potential of DNN and word embedding for the\nevent extraction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00879v1",
            "date": "2016-10-04 07:34:24+00:00",
            "title": "A Computational Approach to Automatic Prediction of Drunk Texting",
            "authors": [
                "Aditya Joshi",
                "Abhijit Mishra",
                "Balamurali AR",
                "Pushpak Bhattacharyya",
                "Mark Carman"
            ],
            "abstract": "Alcohol abuse may lead to unsociable behavior such as crime, drunk driving,\nor privacy leaks. We introduce automatic drunk-texting prediction as the task\nof identifying whether a text was written when under the influence of alcohol.\nWe experiment with tweets labeled using hashtags as distant supervision. Our\nclassifiers use a set of N-gram and stylistic features to detect drunk tweets.\nOur observations present the first quantitative evidence that text contains\nsignals that can be exploited to detect drunk-texting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00883v1",
            "date": "2016-10-04 07:50:06+00:00",
            "title": "Are Word Embedding-based Features Useful for Sarcasm Detection?",
            "authors": [
                "Aditya Joshi",
                "Vaibhav Tripathi",
                "Kevin Patel",
                "Pushpak Bhattacharyya",
                "Mark Carman"
            ],
            "abstract": "This paper makes a simple increment to state-of-the-art in sarcasm detection\nresearch. Existing approaches are unable to capture subtle forms of context\nincongruity which lies at the heart of sarcasm. We explore if prior work can be\nenhanced using semantic similarity/discordance between word embeddings. We\naugment word embedding-based features to four feature sets reported in the\npast. We also experiment with four types of word embeddings. We observe an\nimprovement in sarcasm detection, irrespective of the word embedding used or\nthe original feature set to which our features are augmented. For example, this\naugmentation results in an improvement in F-score of around 4\\% for three out\nof these four feature sets, and a minor degradation in case of the fourth, when\nWord2Vec embeddings are used. Finally, a comparison of the four embeddings\nshows that Word2Vec and dependency weight-based features outperform LSA and\nGloVe, in terms of their benefit to sarcasm detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01108v3",
            "date": "2016-10-04 17:52:42+00:00",
            "title": "Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Tomasz Dwojak",
                "Hieu Hoang"
            ],
            "abstract": "In this paper we provide the largest published comparison of translation\nquality for phrase-based SMT and neural machine translation across 30\ntranslation directions. For ten directions we also include hierarchical\nphrase-based MT. Experiments are performed for the recently published United\nNations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus.\nIn the second part of the paper we investigate aspects of translation speed,\nintroducing AmuNMT, our efficient neural machine translation decoder. We\ndemonstrate that current neural machine translation could already be used for\nin-production systems when comparing words-per-second ratios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01291v1",
            "date": "2016-10-05 07:18:42+00:00",
            "title": "Word2Vec vs DBnary: Augmenting METEOR using Vector Representations or Lexical Resources?",
            "authors": [
                "Christophe Servan",
                "Alexandre Berard",
                "Zied Elloumi",
                "Herv\u00e9 Blanchon",
                "Laurent Besacier"
            ],
            "abstract": "This paper presents an approach combining lexico-semantic resources and\ndistributed representations of words applied to the evaluation in machine\ntranslation (MT). This study is made through the enrichment of a well-known MT\nevaluation metric: METEOR. This metric enables an approximate match (synonymy\nor morphological similarity) between an automatic and a reference translation.\nOur experiments are made in the framework of the Metrics task of WMT 2014. We\nshow that distributed representations are a good alternative to lexico-semantic\nresources for MT evaluation and they can even bring interesting additional\ninformation. The augmented versions of METEOR, using vector representations,\nare made available on our Github page.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01486v2",
            "date": "2016-10-05 15:48:08+00:00",
            "title": "A tentative model for dimensionless phoneme distance from binary distinctive features",
            "authors": [
                "Tiago Tresoldi"
            ],
            "abstract": "This work proposes a tentative model for the calculation of dimensionless\ndistances between phonemes; sounds are described with binary distinctive\nfeatures and distances show linear consistency in terms of such features. The\nmodel can be used as a scoring function for local and global pairwise alignment\nof phoneme sequences, and the distances can be used as prior probabilities for\nBayesian analyses on the phylogenetic relationship between languages,\nparticularly for cognate identification in cases where no empirical prior\nprobability is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01508v1",
            "date": "2016-10-05 16:27:48+00:00",
            "title": "VoxML: A Visualization Modeling Language",
            "authors": [
                "James Pustejovsky",
                "Nikhil Krishnaswamy"
            ],
            "abstract": "We present the specification for a modeling language, VoxML, which encodes\nsemantic knowledge of real-world objects represented as three-dimensional\nmodels, and of events and attributes related to and enacted over these objects.\nVoxML is intended to overcome the limitations of existing 3D visual markup\nlanguages by allowing for the encoding of a broad range of semantic knowledge\nthat can be exploited by a variety of systems and platforms, leading to\nmultimodal simulations of real-world scenarios using conceptual objects that\nrepresent their semantic values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01588v3",
            "date": "2016-10-05 19:57:21+00:00",
            "title": "Neural Structural Correspondence Learning for Domain Adaptation",
            "authors": [
                "Yftah Ziser",
                "Roi Reichart"
            ],
            "abstract": "Domain adaptation, adapting models from domains rich in labeled training data\nto domains poor in such data, is a fundamental NLP challenge. We introduce a\nneural network model that marries together ideas from two prominent strands of\nresearch on domain adaptation through representation learning: structural\ncorrespondence learning (SCL, (Blitzer et al., 2006)) and autoencoder neural\nnetworks. Particularly, our model is a three-layer neural network that learns\nto encode the nonpivot features of an input example into a low-dimensional\nrepresentation, so that the existence of pivot features (features that are\nprominent in both domains and convey useful information for the NLP task) in\nthe example can be decoded from that representation. The low-dimensional\nrepresentation is then employed in a learning algorithm for the task. Moreover,\nwe show how to inject pre-trained word embeddings into our model in order to\nimprove generalization across examples with similar pivot features. On the task\nof cross-domain product sentiment classification (Blitzer et al., 2007),\nconsisting of 12 domain pairs, our model outperforms both the SCL and the\nmarginalized stacked denoising autoencoder (MSDA, (Chen et al., 2012)) methods\nby 3.77% and 2.17% respectively, on average across domain pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01713v1",
            "date": "2016-10-06 01:56:23+00:00",
            "title": "Generating Simulations of Motion Events from Verbal Descriptions",
            "authors": [
                "James Pustejovsky",
                "Nikhil Krishnaswamy"
            ],
            "abstract": "In this paper, we describe a computational model for motion events in natural\nlanguage that maps from linguistic expressions, through a dynamic event\ninterpretation, into three-dimensional temporal simulations in a model.\nStarting with the model from (Pustejovsky and Moszkowicz, 2011), we analyze\nmotion events using temporally-traced Labelled Transition Systems. We model the\ndistinction between path- and manner-motion in an operational semantics, and\nfurther distinguish different types of manner-of-motion verbs in terms of the\nmereo-topological relations that hold throughout the process of movement. From\nthese representations, we generate minimal models, which are realized as\nthree-dimensional simulations in software developed with the game engine,\nUnity. The generated simulations act as a conceptual \"debugger\" for the\nsemantics of different motion verbs: that is, by testing for consistency and\ninformativeness in the model, simulations expose the presuppositions associated\nwith linguistic expressions and their compositions. Because the model\ngeneration component is still incomplete, this paper focuses on an\nimplementation which maps directly from linguistic interpretations into the\nUnity code snippets that create the simulations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01874v1",
            "date": "2016-10-06 13:52:52+00:00",
            "title": "Neural-based Noise Filtering from Word Embeddings",
            "authors": [
                "Kim Anh Nguyen",
                "Sabine Schulte im Walde",
                "Ngoc Thang Vu"
            ],
            "abstract": "Word embeddings have been demonstrated to benefit NLP tasks impressively.\nYet, there is room for improvement in the vector representations, because\ncurrent word embeddings typically contain unnecessary information, i.e., noise.\nWe propose two novel models to improve word embeddings by unsupervised\nlearning, in order to yield word denoising embeddings. The word denoising\nembeddings are obtained by strengthening salient information and weakening\nnoise in the original word embeddings, based on a deep feed-forward neural\nnetwork filter. Results from benchmark tasks show that the filtered word\ndenoising embeddings outperform the original word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01910v1",
            "date": "2016-10-06 15:23:31+00:00",
            "title": "Toward Automatic Understanding of the Function of Affective Language in Support Groups",
            "authors": [
                "Amit Navindgi",
                "Caroline Brun",
                "C\u00e9cile Boulard Masson",
                "Scott Nowson"
            ],
            "abstract": "Understanding expressions of emotions in support forums has considerable\nvalue and NLP methods are key to automating this. Many approaches\nunderstandably use subjective categories which are more fine-grained than a\nstraightforward polarity-based spectrum. However, the definition of such\ncategories is non-trivial and, in fact, we argue for a need to incorporate\ncommunicative elements even beyond subjectivity. To support our position, we\nreport experiments on a sentiment-labelled corpus of posts taken from a medical\nsupport forum. We argue that not only is a more fine-grained approach to text\nanalysis important, but simultaneously recognising the social function behind\naffective expressions enable a more accurate and valuable level of\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02003v1",
            "date": "2016-10-06 19:22:49+00:00",
            "title": "Scalable Machine Translation in Memory Constrained Environments",
            "authors": [
                "Paul Baltescu"
            ],
            "abstract": "Machine translation is the discipline concerned with developing automated\ntools for translating from one human language to another. Statistical machine\ntranslation (SMT) is the dominant paradigm in this field. In SMT, translations\nare generated by means of statistical models whose parameters are learned from\nbilingual data. Scalability is a key concern in SMT, as one would like to make\nuse of as much data as possible to train better translation systems.\n  In recent years, mobile devices with adequate computing power have become\nwidely available. Despite being very successful, mobile applications relying on\nNLP systems continue to follow a client-server architecture, which is of\nlimited use because access to internet is often limited and expensive. The goal\nof this dissertation is to show how to construct a scalable machine translation\nsystem that can operate with the limited resources available on a mobile\ndevice.\n  The main challenge for porting translation systems on mobile devices is\nmemory usage. The amount of memory available on a mobile device is far less\nthan what is typically available on the server side of a client-server\napplication. In this thesis, we investigate alternatives for the two components\nwhich prevent standard translation systems from working on mobile devices due\nto high memory usage. We show that once these standard components are replaced\nwith our proposed alternatives, we obtain a scalable translation system that\ncan work on a device with limited memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02124v1",
            "date": "2016-10-07 02:17:17+00:00",
            "title": "There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction",
            "authors": [
                "Courtney Napoles",
                "Keisuke Sakaguchi",
                "Joel Tetreault"
            ],
            "abstract": "Current methods for automatically evaluating grammatical error correction\n(GEC) systems rely on gold-standard references. However, these methods suffer\nfrom penalizing grammatical edits that are correct but not in the gold\nstandard. We show that reference-less grammaticality metrics correlate very\nstrongly with human judgments and are competitive with the leading\nreference-based evaluation metrics. By interpolating both methods, we achieve\nstate-of-the-art correlation with human judgments. Finally, we show that GEC\nmetrics are much more reliable when they are calculated at the sentence level\ninstead of the corpus level. We have set up a CodaLab site for benchmarking GEC\noutput using a common dataset and different evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02213v1",
            "date": "2016-10-07 10:22:26+00:00",
            "title": "Challenges of Computational Processing of Code-Switching",
            "authors": [
                "\u00d6zlem \u00c7etino\u011flu",
                "Sarah Schulz",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper addresses challenges of Natural Language Processing (NLP) on\nnon-canonical multilingual data in which two or more languages are mixed. It\nrefers to code-switching which has become more popular in our daily life and\ntherefore obtains an increasing amount of attention from the research\ncommunity. We report our experience that cov- ers not only core NLP tasks such\nas normalisation, language identification, language modelling, part-of-speech\ntagging and dependency parsing but also more downstream ones such as machine\ntranslation and automatic speech recognition. We highlight and discuss the key\nproblems for each of the tasks with supporting examples from different language\npairs and relevant previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02493v1",
            "date": "2016-10-08 06:33:35+00:00",
            "title": "A Semantic Analyzer for the Comprehension of the Spontaneous Arabic Speech",
            "authors": [
                "Mourad Mars",
                "Mounir Zrigui",
                "Mohamed Belgacem",
                "Anis Zouaghi"
            ],
            "abstract": "This work is part of a large research project entitled \"Or\\'eodule\" aimed at\ndeveloping tools for automatic speech recognition, translation, and synthesis\nfor Arabic language. Our attention has mainly been focused on an attempt to\nimprove the probabilistic model on which our semantic decoder is based. To\nachieve this goal, we have decided to test the influence of the pertinent\ncontext use, and of the contextual data integration of different types, on the\neffectiveness of the semantic decoder. The findings are quite satisfactory.",
            "journal ref": "Journal: Research in Computing Science Vol34, 2008, pp. 129-140,\n  ISSN: 1870-4069",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02544v1",
            "date": "2016-10-08 15:24:50+00:00",
            "title": "Computational linking theory",
            "authors": [
                "Aaron Steven White",
                "Drew Reisinger",
                "Rachel Rudinger",
                "Kyle Rawlins",
                "Benjamin Van Durme"
            ],
            "abstract": "A linking theory explains how verbs' semantic arguments are mapped to their\nsyntactic arguments---the inverse of the Semantic Role Labeling task from the\nshallow semantic parsing literature. In this paper, we develop the\nComputational Linking Theory framework as a method for implementing and testing\nlinking theories proposed in the theoretical literature. We deploy this\nframework to assess two cross-cutting types of linking theory: local v. global\nmodels and categorical v. featural models. To further investigate the behavior\nof these models, we develop a measurement model in the spirit of previous work\nin semantic role induction: the Semantic Proto-Role Linking Model. We use this\nmodel, which implements a generalization of Dowty's seminal Proto-Role Theory,\nto induce semantic proto-roles, which we compare to those Dowty proposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02633v1",
            "date": "2016-10-09 06:42:02+00:00",
            "title": "Enabling Medical Translation for Low-Resource Languages",
            "authors": [
                "Ahmad Musleh",
                "Nadir Durrani",
                "Irina Temnikova",
                "Preslav Nakov",
                "Stephan Vogel",
                "Osama Alsaad"
            ],
            "abstract": "We present research towards bridging the language gap between migrant workers\nin Qatar and medical staff. In particular, we present the first steps towards\nthe development of a real-world Hindi-English machine translation system for\ndoctor-patient communication. As this is a low-resource language pair,\nespecially for speech and for the medical domain, our initial focus has been on\ngathering suitable training data from various sources. We applied a variety of\nmethods ranging from fully automatic extraction from the Web to manual\nannotation of test data. Moreover, we developed a method for automatically\naugmenting the training data with synthetically generated variants, which\nyielded a very sizable improvement of more than 3 BLEU points absolute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02749v1",
            "date": "2016-10-10 01:47:50+00:00",
            "title": "A Dynamic Window Neural Network for CCG Supertagging",
            "authors": [
                "Huijia Wu",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Combinatory Category Grammar (CCG) supertagging is a task to assign lexical\ncategories to each word in a sentence. Almost all previous methods use fixed\ncontext window sizes as input features. However, it is obvious that different\ntags usually rely on different context window sizes. These motivate us to build\na supertagger with a dynamic window approach, which can be treated as an\nattention mechanism on the local contexts. Applying dropout on the dynamic\nfilters can be seen as drop on words directly, which is superior to the regular\ndropout on word embeddings. We use this approach to demonstrate the\nstate-of-the-art CCG supertagging performance on the standard test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02806v1",
            "date": "2016-10-10 08:52:36+00:00",
            "title": "Modelling Sentence Pairs with Tree-structured Attentive Encoder",
            "authors": [
                "Yao Zhou",
                "Cong Liu",
                "Yan Pan"
            ],
            "abstract": "We describe an attentive encoder that combines tree-structured recursive\nneural networks and sequential recurrent neural networks for modelling sentence\npairs. Since existing attentive models exert attention on the sequential\nstructure, we propose a way to incorporate attention into the tree topology.\nSpecially, given a pair of sentences, our attentive encoder uses the\nrepresentation of one sentence, which generated via an RNN, to guide the\nstructural encoding of the other sentence on the dependency parse tree. We\nevaluate the proposed attentive encoder on three tasks: semantic similarity,\nparaphrase identification and true-false question selection. Experimental\nresults show that our encoder outperforms all baselines and achieves\nstate-of-the-art results on two tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03022v1",
            "date": "2016-10-10 18:43:58+00:00",
            "title": "Very Deep Convolutional Networks for End-to-End Speech Recognition",
            "authors": [
                "Yu Zhang",
                "William Chan",
                "Navdeep Jaitly"
            ],
            "abstract": "Sequence-to-sequence models have shown success in end-to-end speech\nrecognition. However these models have only used shallow acoustic encoder\nnetworks. In our work, we successively train very deep convolutional networks\nto add more expressive power and better generalization for end-to-end ASR\nmodels. We apply network-in-network principles, batch normalization, residual\nconnections and convolutional LSTMs to build very deep recurrent and\nconvolutional structures. Our models exploit the spectral structure in the\nfeature space and add computational depth without overfitting issues. We\nexperiment with the WSJ ASR task and achieve 10.5\\% word error rate without any\ndictionary or language using a 15 layer deep network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03098v3",
            "date": "2016-10-10 21:01:00+00:00",
            "title": "Neural Paraphrase Generation with Stacked Residual LSTM Networks",
            "authors": [
                "Aaditya Prakash",
                "Sadid A. Hasan",
                "Kathy Lee",
                "Vivek Datla",
                "Ashequl Qadir",
                "Joey Liu",
                "Oladimeji Farri"
            ],
            "abstract": "In this paper, we propose a novel neural approach for paraphrase generation.\nConventional para- phrase generation methods either leverage hand-written rules\nand thesauri-based alignments, or use statistical machine learning principles.\nTo the best of our knowledge, this work is the first to explore deep learning\nmodels for paraphrase generation. Our primary contribution is a stacked\nresidual LSTM network, where we add residual connections between LSTM layers.\nThis allows for efficient training of deep LSTMs. We evaluate our model and\nother state-of-the-art deep learning models on three different datasets: PPDB,\nWikiAnswers and MSCOCO. Evaluation results demonstrate that our model\noutperforms sequence to sequence, attention-based and bi- directional LSTM\nmodels on BLEU, METEOR, TER and an embedding-based sentence similarity metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03112v1",
            "date": "2016-10-10 22:08:13+00:00",
            "title": "Leveraging Recurrent Neural Networks for Multimodal Recognition of Social Norm Violation in Dialog",
            "authors": [
                "Tiancheng Zhao",
                "Ran Zhao",
                "Zhao Meng",
                "Justine Cassell"
            ],
            "abstract": "Social norms are shared rules that govern and facilitate social interaction.\nViolating such social norms via teasing and insults may serve to upend power\nimbalances or, on the contrary reinforce solidarity and rapport in\nconversation, rapport which is highly situated and context-dependent. In this\nwork, we investigate the task of automatically identifying the phenomena of\nsocial norm violation in discourse. Towards this goal, we leverage the power of\nrecurrent neural networks and multimodal information present in the\ninteraction, and propose a predictive model to recognize social norm violation.\nUsing long-term temporal and contextual information, our model achieves an F1\nscore of 0.705. Implications of our work regarding developing a social-aware\nagent are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03167v1",
            "date": "2016-10-11 03:02:38+00:00",
            "title": "An Empirical Exploration of Skip Connections for Sequential Tagging",
            "authors": [
                "Huijia Wu",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "In this paper, we empirically explore the effects of various kinds of skip\nconnections in stacked bidirectional LSTMs for sequential tagging. We\ninvestigate three kinds of skip connections connecting to LSTM cells: (a) skip\nconnections to the gates, (b) skip connections to the internal states and (c)\nskip connections to the cell outputs. We present comprehensive experiments\nshowing that skip connections to cell outputs outperform the remaining two.\nFurthermore, we observe that using gated identity functions as skip mappings\nworks pretty well. Based on this novel skip connections, we successfully train\ndeep stacked bidirectional LSTM models and obtain state-of-the-art results on\nCCG supertagging and comparable results on POS tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03246v1",
            "date": "2016-10-11 09:19:06+00:00",
            "title": "Toward a new instances of NELL",
            "authors": [
                "Maisa C. Duarte",
                "Pierre Maret"
            ],
            "abstract": "We are developing the method to start new instances of NELL in various\nlanguages and develop then NELL multilingualism. We base our method on our\nexperience on NELL Portuguese and NELL French. This reports explain our method\nand develops some research perspectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03256v1",
            "date": "2016-10-11 09:52:57+00:00",
            "title": "GMM-Free Flat Start Sequence-Discriminative DNN Training",
            "authors": [
                "G\u00e1bor Gosztolya",
                "Tam\u00e1s Gr\u00f3sz",
                "L\u00e1szl\u00f3 T\u00f3th"
            ],
            "abstract": "Recently, attempts have been made to remove Gaussian mixture models (GMM)\nfrom the training process of deep neural network-based hidden Markov models\n(HMM/DNN). For the GMM-free training of a HMM/DNN hybrid we have to solve two\nproblems, namely the initial alignment of the frame-level state labels and the\ncreation of context-dependent states. Although flat-start training via\niteratively realigning and retraining the DNN using a frame-level error\nfunction is viable, it is quite cumbersome. Here, we propose to use a\nsequence-discriminative training criterion for flat start. While\nsequence-discriminative training is routinely applied only in the final phase\nof model training, we show that with proper caution it is also suitable for\ngetting an alignment of context-independent DNN models. For the construction of\ntied states we apply a recently proposed KL-divergence-based state clustering\nmethod, hence our whole training process is GMM-free. In the experimental\nevaluation we found that the sequence-discriminative flat start training method\nis not only significantly faster than the straightforward approach of iterative\nretraining and realignment, but the word error rates attained are slightly\nbetter as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03321v1",
            "date": "2016-10-11 13:20:52+00:00",
            "title": "Keystroke dynamics as signal for shallow syntactic parsing",
            "authors": [
                "Barbara Plank"
            ],
            "abstract": "Keystroke dynamics have been extensively used in psycholinguistic and writing\nresearch to gain insights into cognitive processing. But do keystroke logs\ncontain actual signal that can be used to learn better natural language\nprocessing models?\n  We postulate that keystroke dynamics contain information about syntactic\nstructure that can inform shallow syntactic parsing. To test this hypothesis,\nwe explore labels derived from keystroke logs as auxiliary task in a multi-task\nbidirectional Long Short-Term Memory (bi-LSTM). Our results show promising\nresults on two shallow syntactic parsing tasks, chunking and CCG supertagging.\nOur model is simple, has the advantage that data can come from distinct\nsources, and produces models that are significantly better than models trained\non the text annotations alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03349v1",
            "date": "2016-10-11 14:11:55+00:00",
            "title": "Survey on the Use of Typological Information in Natural Language Processing",
            "authors": [
                "Helen O'Horan",
                "Yevgeni Berzak",
                "Ivan Vuli\u0107",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "In recent years linguistic typology, which classifies the world's languages\naccording to their functional and structural properties, has been widely used\nto support multilingual NLP. While the growing importance of typological\ninformation in supporting multilingual tasks has been recognised, no systematic\nsurvey of existing typological resources and their use in NLP has been\npublished. This paper provides such a survey as well as discussion which we\nhope will both inform and inspire future work in the area.",
            "journal ref": "COLING 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03585v1",
            "date": "2016-10-12 02:45:45+00:00",
            "title": "A Paradigm for Situated and Goal-Driven Language Learning",
            "authors": [
                "Jon Gauthier",
                "Igor Mordatch"
            ],
            "abstract": "A distinguishing property of human intelligence is the ability to flexibly\nuse language in order to communicate complex ideas with other humans in a\nvariety of contexts. Research in natural language dialogue should focus on\ndesigning communicative agents which can integrate themselves into these\ncontexts and productively collaborate with humans. In this abstract, we propose\na general situated language learning paradigm which is designed to bring about\nrobust language agents able to cooperate productively with humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03759v2",
            "date": "2016-10-12 15:53:02+00:00",
            "title": "Language Models with Pre-Trained (GloVe) Word Embeddings",
            "authors": [
                "Victor Makarenkov",
                "Bracha Shapira",
                "Lior Rokach"
            ],
            "abstract": "In this work we implement a training of a Language Model (LM), using\nRecurrent Neural Network (RNN) and GloVe word embeddings, introduced by\nPennigton et al. in [1]. The implementation is following the general idea of\ntraining RNNs for LM tasks presented in [2], but is rather using Gated\nRecurrent Unit (GRU) [3] for a memory cell, and not the more commonly used LSTM\n[4].",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03771v1",
            "date": "2016-10-12 16:23:11+00:00",
            "title": "SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods",
            "authors": [
                "Marzieh Saeidi",
                "Guillaume Bouchard",
                "Maria Liakata",
                "Sebastian Riedel"
            ],
            "abstract": "In this paper, we introduce the task of targeted aspect-based sentiment\nanalysis. The goal is to extract fine-grained information with respect to\nentities mentioned in user comments. This work extends both aspect-based\nsentiment analysis that assumes a single entity per document and targeted\nsentiment analysis that assumes a single sentiment towards a target entity. In\nparticular, we identify the sentiment towards each aspect of one or more\nentities. As a testbed for this task, we introduce the SentiHood dataset,\nextracted from a question answering (QA) platform where urban neighbourhoods\nare discussed by users. In this context units of text often mention several\naspects of one or more neighbourhoods. This is the first time that a generic\nsocial media platform in this case a QA platform, is used for fine-grained\nopinion mining. Text coming from QA platforms is far less constrained compared\nto text from review specific platforms which current datasets are based on. We\ndevelop several strong baselines, relying on logistic regression and\nstate-of-the-art recurrent neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03807v2",
            "date": "2016-10-12 17:55:56+00:00",
            "title": "Question Generation from a Knowledge Base with Web Exploration",
            "authors": [
                "Linfeng Song",
                "Lin Zhao"
            ],
            "abstract": "Question generation from a knowledge base (KB) is the task of generating\nquestions related to the domain of the input KB. We propose a system for\ngenerating fluent and natural questions from a KB, which significantly reduces\nthe human effort by leveraging massive web resources. In more detail, a seed\nquestion set is first generated by applying a small number of hand-crafted\ntemplates on the input KB, then more questions are retrieved by iteratively\nforming already obtained questions as search queries into a standard search\nengine, before finally questions are selected by estimating their fluency and\ndomain relevance. Evaluated by human graders on 500 random-selected triples\nfrom Freebase, questions generated by our system are judged to be more fluent\nthan those of \\newcite{serban-EtAl:2016:P16-1} by human graders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03946v1",
            "date": "2016-10-13 06:42:51+00:00",
            "title": "A Neural Network for Coordination Boundary Prediction",
            "authors": [
                "Jessica Ficler",
                "Yoav Goldberg"
            ],
            "abstract": "We propose a neural-network based model for coordination boundary prediction.\nThe network is designed to incorporate two signals: the similarity between\nconjuncts and the observation that replacing the whole coordination phrase with\na conjunct tends to produce a coherent sentences. The modeling makes use of\nseveral LSTM networks. The model is trained solely on conjunction annotations\nin a Treebank, without using external resources. We show improvements on\npredicting coordination boundaries on the PTB compared to two state-of-the-art\nparsers; as well as improvement over previous coordination boundary prediction\nsystems on the Genia corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04265v2",
            "date": "2016-10-13 21:25:34+00:00",
            "title": "Fast, Scalable Phrase-Based SMT Decoding",
            "authors": [
                "Hieu Hoang",
                "Nikolay Bogoychev",
                "Lane Schwartz",
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "The utilization of statistical machine translation (SMT) has grown enormously\nover the last decade, many using open-source software developed by the NLP\ncommunity. As commercial use has increased, there is need for software that is\noptimized for commercial requirements, in particular, fast phrase-based\ndecoding and more efficient utilization of modern multicore servers.\n  In this paper we re-examine the major components of phrase-based decoding and\ndecoder implementation with particular emphasis on speed and scalability on\nmulticore machines. The result is a drop-in replacement for the Moses decoder\nwhich is up to fifteen times faster and scales monotonically with the number of\ncores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04841v2",
            "date": "2016-10-16 10:54:23+00:00",
            "title": "Translation Quality Estimation using Recurrent Neural Network",
            "authors": [
                "Raj Nath Patel",
                "Sasikumar M"
            ],
            "abstract": "This paper describes our submission to the shared task on word/phrase level\nQuality Estimation (QE) in the First Conference on Statistical Machine\nTranslation (WMT16). The objective of the shared task was to predict if the\ngiven word/phrase is a correct/incorrect (OK/BAD) translation in the given\nsentence. In this paper, we propose a novel approach for word level Quality\nEstimation using Recurrent Neural Network Language Model (RNN-LM) architecture.\nRNN-LMs have been found very effective in different Natural Language Processing\n(NLP) applications. RNN-LM is mainly used for vector space language modeling\nfor different NLP problems. For this task, we modify the architecture of\nRNN-LM. The modified system predicts a label (OK/BAD) in the slot rather than\npredicting the word. The input to the system is a word sequence, similar to the\nstandard RNN-LM. The approach is language independent and requires only the\ntranslated text for QE. To estimate the phrase level quality, we use the output\nof the word level QE system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05011v1",
            "date": "2016-10-17 08:33:20+00:00",
            "title": "Interactive Attention for Neural Machine Translation",
            "authors": [
                "Fandong Meng",
                "Zhengdong Lu",
                "Hang Li",
                "Qun Liu"
            ],
            "abstract": "Conventional attention-based Neural Machine Translation (NMT) conducts\ndynamic alignment in generating the target sentence. By repeatedly reading the\nrepresentation of source sentence, which keeps fixed after generated by the\nencoder (Bahdanau et al., 2015), the attention mechanism has greatly enhanced\nstate-of-the-art NMT. In this paper, we propose a new attention mechanism,\ncalled INTERACTIVE ATTENTION, which models the interaction between the decoder\nand the representation of source sentence during translation by both reading\nand writing operations. INTERACTIVE ATTENTION can keep track of the interaction\nhistory and therefore improve the translation performance. Experiments on NIST\nChinese-English translation task show that INTERACTIVE ATTENTION can achieve\nsignificant improvements over both the previous attention-based NMT baseline\nand some state-of-the-art variants of attention-based NMT (i.e., coverage\nmodels (Tu et al., 2016)). And neural machine translator with our INTERACTIVE\nATTENTION can outperform the open source attention-based NMT system Groundhog\nby 4.22 BLEU points and the open source phrase-based system Moses by 3.94 BLEU\npoints averagely on multiple test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05150v2",
            "date": "2016-10-17 14:50:58+00:00",
            "title": "Neural Machine Translation Advised by Statistical Machine Translation",
            "authors": [
                "Xing Wang",
                "Zhengdong Lu",
                "Zhaopeng Tu",
                "Hang Li",
                "Deyi Xiong",
                "Min Zhang"
            ],
            "abstract": "Neural Machine Translation (NMT) is a new approach to machine translation\nthat has made great progress in recent years. However, recent studies show that\nNMT generally produces fluent but inadequate translations (Tu et al. 2016b; Tu\net al. 2016a; He et al. 2016; Tu et al. 2017). This is in contrast to\nconventional Statistical Machine Translation (SMT), which usually yields\nadequate but non-fluent translations. It is natural, therefore, to leverage the\nadvantages of both models for better translations, and in this work we propose\nto incorporate SMT model into NMT framework. More specifically, at each\ndecoding step, SMT offers additional recommendations of generated words based\non the decoding information from NMT (e.g., the generated partial translation\nand attention history). Then we employ an auxiliary classifier to score the SMT\nrecommendations and a gating function to combine the SMT recommendations with\nNMT generations, both of which are jointly trained within the NMT architecture\nin an end-to-end manner. Experimental results on Chinese-English translation\nshow that the proposed approach achieves significant and consistent\nimprovements over state-of-the-art NMT and SMT systems on multiple NIST test\nsets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05243v1",
            "date": "2016-10-17 18:14:24+00:00",
            "title": "Pre-Translation for Neural Machine Translation",
            "authors": [
                "Jan Niehues",
                "Eunah Cho",
                "Thanh-Le Ha",
                "Alex Waibel"
            ],
            "abstract": "Recently, the development of neural machine translation (NMT) has\nsignificantly improved the translation quality of automatic machine\ntranslation. While most sentences are more accurate and fluent than\ntranslations by statistical machine translation (SMT)-based systems, in some\ncases, the NMT system produces translations that have a completely different\nmeaning. This is especially the case when rare words occur.\n  When using statistical machine translation, it has already been shown that\nsignificant gains can be achieved by simplifying the input in a preprocessing\nstep. A commonly used example is the pre-reordering approach.\n  In this work, we used phrase-based machine translation to pre-translate the\ninput into the target language. Then a neural machine translation system\ngenerates the final hypothesis using the pre-translation. Thereby, we use\neither only the output of the phrase-based machine translation (PBMT) system or\na combination of the PBMT output and the source sentence.\n  We evaluate the technique on the English to German translation task. Using\nthis approach we are able to outperform the PBMT system as well as the baseline\nneural MT system by up to 2 BLEU points. We analyzed the influence of the\nquality of the initial system on the final result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05361v1",
            "date": "2016-10-17 21:16:56+00:00",
            "title": "End-to-end attention-based distant speech recognition with Highway LSTM",
            "authors": [
                "Hassan Taherian"
            ],
            "abstract": "End-to-end attention-based models have been shown to be competitive\nalternatives to conventional DNN-HMM models in the Speech Recognition Systems.\nIn this paper, we extend existing end-to-end attention-based models that can be\napplied for Distant Speech Recognition (DSR) task. Specifically, we propose an\nend-to-end attention-based speech recognizer with multichannel input that\nperforms sequence prediction directly at the character level. To gain a better\nperformance, we also incorporate Highway long short-term memory (HLSTM) which\noutperforms previous models on AMI distant speech recognition task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05461v2",
            "date": "2016-10-18 07:39:40+00:00",
            "title": "Personalized Machine Translation: Preserving Original Author Traits",
            "authors": [
                "Ella Rabinovich",
                "Shachar Mirkin",
                "Raj Nath Patel",
                "Lucia Specia",
                "Shuly Wintner"
            ],
            "abstract": "The language that we produce reflects our personality, and various personal\nand demographic characteristics can be detected in natural language texts. We\nfocus on one particular personal trait of the author, gender, and study how it\nis manifested in original texts and in translations. We show that author's\ngender has a powerful, clear signal in originals texts, but this signal is\nobfuscated in human and machine translation. We then propose simple\ndomain-adaptation techniques that help retain the original gender traits in the\ntranslation, without harming the quality of the translation, thereby creating\nmore personalized machine translation systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05522v1",
            "date": "2016-10-18 10:22:46+00:00",
            "title": "Addressing Community Question Answering in English and Arabic",
            "authors": [
                "Giovanni Da San Martino",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Salvatore Romeo",
                "Alessandro Moschitti",
                "Shafiq Joty",
                "Fahad A. Al Obaidli",
                "Kateryna Tymoshenko",
                "Antonio Uva"
            ],
            "abstract": "This paper studies the impact of different types of features applied to\nlearning to re-rank questions in community Question Answering. We tested our\nmodels on two datasets released in SemEval-2016 Task 3 on \"Community Question\nAnswering\". Task 3 targeted real-life Web fora both in English and Arabic. Our\nmodels include bag-of-words features (BoW), syntactic tree kernels (TKs), rank\nfeatures, embeddings, and machine translation evaluation features. To the best\nof our knowledge, structural kernels have barely been applied to the question\nreranking task, where they have to model paraphrase relations. In the case of\nthe English question re-ranking task, we compare our learning to rank (L2R)\nalgorithms against a strong baseline given by the Google-generated ranking\n(GR). The results show that i) the shallow structures used in our TKs are\nrobust enough to noisy data and ii) improving GR is possible, but effective BoW\nfeatures and TKs along with an accurate model of GR features in the used L2R\nalgorithm are required. In the case of the Arabic question re-ranking task, for\nthe first time we applied tree kernels on syntactic trees of Arabic sentences.\nOur approaches to both tasks obtained the second best results on SemEval-2016\nsubtasks B on English and D on Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05540v1",
            "date": "2016-10-18 11:32:42+00:00",
            "title": "SYSTRAN's Pure Neural Machine Translation Systems",
            "authors": [
                "Josep Crego",
                "Jungi Kim",
                "Guillaume Klein",
                "Anabel Rebollo",
                "Kathy Yang",
                "Jean Senellart",
                "Egor Akhanov",
                "Patrice Brunelle",
                "Aurelien Coquard",
                "Yongchao Deng",
                "Satoshi Enoue",
                "Chiyo Geiss",
                "Joshua Johanson",
                "Ardas Khalsa",
                "Raoum Khiari",
                "Byeongil Ko",
                "Catherine Kobus",
                "Jean Lorieux",
                "Leidiana Martins",
                "Dang-Chuan Nguyen",
                "Alexandra Priori",
                "Thomas Riccardi",
                "Natalia Segal",
                "Christophe Servan",
                "Cyril Tiquet",
                "Bo Wang",
                "Jin Yang",
                "Dakun Zhang",
                "Jing Zhou",
                "Peter Zoldan"
            ],
            "abstract": "Since the first online demonstration of Neural Machine Translation (NMT) by\nLISA, NMT development has recently moved from laboratory to production systems\nas demonstrated by several entities announcing roll-out of NMT engines to\nreplace their existing technologies. NMT systems have a large number of\ntraining configurations and the training process of such systems is usually\nvery long, often a few weeks, so role of experimentation is critical and\nimportant to share. In this work, we present our approach to production-ready\nsystems simultaneously with release of online demonstrators covering a large\nvariety of languages (12 languages, for 32 language pairs). We explore\ndifferent practical choices: an efficient and evolutive open-source framework;\ndata preparation; network architecture; additional implemented features; tuning\nfor production; etc. We discuss about evaluation methodology, present our first\nfindings and we finally outline further work.\n  Our ultimate goal is to share our expertise to build competitive production\nsystems for \"generic\" translation. We aim at contributing to set up a\ncollaborative framework to speed-up adoption of the technology, foster further\nresearch efforts and enable the delivery and adoption to/by industry of\nuse-case specific engines integrated in real production workflows. Mastering of\nthe technology would allow us to build translation engines suited for\nparticular needs, outperforming current simplest/uniform systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05652v2",
            "date": "2016-10-18 14:44:01+00:00",
            "title": "Vietnamese Named Entity Recognition using Token Regular Expressions and Bidirectional Inference",
            "authors": [
                "Phuong Le-Hong"
            ],
            "abstract": "This paper describes an efficient approach to improve the accuracy of a named\nentity recognition system for Vietnamese. The approach combines regular\nexpressions over tokens and a bidirectional inference method in a sequence\nlabelling model. The proposed method achieves an overall $F_1$ score of 89.66%\non a test set of an evaluation campaign, organized in late 2016 by the\nVietnamese Language and Speech Processing (VLSP) community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05858v1",
            "date": "2016-10-19 03:59:30+00:00",
            "title": "Bidirectional LSTM-CRF for Clinical Concept Extraction",
            "authors": [
                "Raghavendra Chalapathy",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "Extraction of concepts present in patient clinical records is an essential\nstep in clinical research. The 2010 i2b2/VA Workshop on Natural Language\nProcessing Challenges for clinical records presented concept extraction (CE)\ntask, with aim to identify concepts (such as treatments, tests, problems) and\nclassify them into predefined categories. State-of-the-art CE approaches\nheavily rely on hand crafted features and domain specific resources which are\nhard to collect and tune. For this reason, this paper employs bidirectional\nLSTM with CRF decoding initialized with general purpose off-the-shelf word\nembeddings for CE. The experimental results achieved on 2010 i2b2/VA reference\nstandard corpora using bidirectional LSTM CRF ranks closely with top ranked\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06053v1",
            "date": "2016-10-19 15:09:21+00:00",
            "title": "Chinese Restaurant Process for cognate clustering: A threshold free approach",
            "authors": [
                "Taraka Rama"
            ],
            "abstract": "In this paper, we introduce a threshold free approach, motivated from Chinese\nRestaurant Process, for the purpose of cognate clustering. We show that our\napproach yields similar results to a linguistically motivated cognate\nclustering system known as LexStat. Our Chinese Restaurant Process system is\nfast and does not require any threshold and can be applied to any language\nfamily of the world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06210v1",
            "date": "2016-10-19 20:49:23+00:00",
            "title": "A Theme-Rewriting Approach for Generating Algebra Word Problems",
            "authors": [
                "Rik Koncel-Kedziorski",
                "Ioannis Konstas",
                "Luke Zettlemoyer",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Texts present coherent stories that have a particular theme or overall\nsetting, for example science fiction or western. In this paper, we present a\ntext generation method called {\\it rewriting} that edits existing\nhuman-authored narratives to change their theme without changing the underlying\nstory. We apply the approach to math word problems, where it might help\nstudents stay more engaged by quickly transforming all of their homework\nassignments to the theme of their favorite movie without changing the math\nconcepts that are being taught. Our rewriting method uses a two-stage decoding\nprocess, which proposes new words from the target theme and scores the\nresulting stories according to a number of factors defining aspects of\nsyntactic, semantic, and thematic coherence. Experiments demonstrate that the\nfinal stories typically represent the new theme well while still testing the\noriginal math concepts, outperforming a number of baselines. We also release a\nnew dataset of human-authored rewrites of math word problems in several themes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06227v2",
            "date": "2016-10-19 21:25:39+00:00",
            "title": "Cross-Lingual Syntactic Transfer with Limited Resources",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Michael Collins"
            ],
            "abstract": "We describe a simple but effective method for cross-lingual syntactic\ntransfer of dependency parsers, in the scenario where a large amount of\ntranslation data is not available. The method makes use of three steps: 1) a\nmethod for deriving cross-lingual word clusters, which can then be used in a\nmultilingual parser; 2) a method for transferring lexical information from a\ntarget language to source language treebanks; 3) a method for integrating these\nsteps with the density-driven annotation projection method of Rasooli and\nCollins (2015). Experiments show improvements over the state-of-the-art in\nseveral languages used in previous work, in a setting where the only source of\ntranslation data is the Bible, a considerably smaller corpus than the Europarl\ncorpus used in previous work. Results using the Europarl corpus as a source of\ntranslation data show additional improvements over the results of Rasooli and\nCollins (2015). We conclude with results on 38 datasets from the Universal\nDependencies corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06272v2",
            "date": "2016-10-20 03:10:57+00:00",
            "title": "Lexicon Integrated CNN Models with Attention for Sentiment Analysis",
            "authors": [
                "Bonggun Shin",
                "Timothy Lee",
                "Jinho D. Choi"
            ],
            "abstract": "With the advent of word embeddings, lexicons are no longer fully utilized for\nsentiment analysis although they still provide important features in the\ntraditional setting. This paper introduces a novel approach to sentiment\nanalysis that integrates lexicon embeddings and an attention mechanism into\nConvolutional Neural Networks. Our approach performs separate convolutions for\nword and lexicon embeddings and provides a global view of the document using\nattention. Our models are experimented on both the SemEval'16 Task 4 dataset\nand the Stanford Sentiment Treebank, and show comparative or better results\nagainst the existing state-of-the-art systems. Our analysis shows that lexicon\nembeddings allow to build high-performing models with much smaller word\nembeddings, and the attention mechanism effectively dims out noisy words for\nsentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06498v1",
            "date": "2016-10-20 17:00:42+00:00",
            "title": "Authorship Attribution Based on Life-Like Network Automata",
            "authors": [
                "Jeaneth Machicao",
                "Edilson A. Corr\u00eaa Jr.",
                "Gisele H. B. Miranda",
                "Diego R. Amancio",
                "Odemir M. Bruno"
            ],
            "abstract": "The authorship attribution is a problem of considerable practical and\ntechnical interest. Several methods have been designed to infer the authorship\nof disputed documents in multiple contexts. While traditional statistical\nmethods based solely on word counts and related measurements have provided a\nsimple, yet effective solution in particular cases; they are prone to\nmanipulation. Recently, texts have been successfully modeled as networks, where\nwords are represented by nodes linked according to textual similarity\nmeasurements. Such models are useful to identify informative topological\npatterns for the authorship recognition task. However, there is no consensus on\nwhich measurements should be used. Thus, we proposed a novel method to\ncharacterize text networks, by considering both topological and dynamical\naspects of networks. Using concepts and methods from cellular automata theory,\nwe devised a strategy to grasp informative spatio-temporal patterns from this\nmodel. Our experiments revealed an outperformance over traditional analysis\nrelying only on topological measurements. Remarkably, we have found a\ndependence of pre-processing steps (such as the lemmatization) on the obtained\nresults, a feature that has mostly been disregarded in related works. The\noptimized results obtained here pave the way for a better characterization of\ntextual networks.",
            "journal ref": "PLoS ONE 13(3): e0193703, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06510v3",
            "date": "2016-10-20 17:32:32+00:00",
            "title": "Learning variable length units for SMT between related languages via Byte Pair Encoding",
            "authors": [
                "Anoop Kunchukuttan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "We explore the use of segments learnt using Byte Pair Encoding (referred to\nas BPE units) as basic units for statistical machine translation between\nrelated languages and compare it with orthographic syllables, which are\ncurrently the best performing basic units for this translation task. BPE\nidentifies the most frequent character sequences as basic units, while\northographic syllables are linguistically motivated pseudo-syllables. We show\nthat BPE units modestly outperform orthographic syllables as units of\ntranslation, showing up to 11% increase in BLEU score. While orthographic\nsyllables can be used only for languages whose writing systems use vowel\nrepresentations, BPE is writing system independent and we show that BPE\noutperforms other units for non-vowel writing systems too. Our results are\nsupported by extensive experimentation spanning multiple language families and\nwriting systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06542v1",
            "date": "2016-10-20 19:10:09+00:00",
            "title": "Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016",
            "authors": [
                "Graham Neubig"
            ],
            "abstract": "This year, the Nara Institute of Science and Technology (NAIST)/Carnegie\nMellon University (CMU) submission to the Japanese-English translation track of\nthe 2016 Workshop on Asian Translation was based on attentional neural machine\ntranslation (NMT) models. In addition to the standard NMT model, we make a\nnumber of improvements, most notably the use of discrete translation lexicons\nto improve probability estimates, and the use of minimum risk training to\noptimize the MT system for BLEU score. As a result, our system achieved the\nhighest translation evaluation scores for the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06550v1",
            "date": "2016-10-20 19:33:02+00:00",
            "title": "Neural Machine Translation with Characters and Hierarchical Encoding",
            "authors": [
                "Alexander Rosenberg Johansen",
                "Jonas Meinertz Hansen",
                "Elias Khazen Obeid",
                "Casper Kaae S\u00f8nderby",
                "Ole Winther"
            ],
            "abstract": "Most existing Neural Machine Translation models use groups of characters or\nwhole words as their unit of input and output. We propose a model with a\nhierarchical char2word encoder, that takes individual characters both as input\nand output. We first argue that this hierarchical representation of the\ncharacter encoder reduces computational complexity, and show that it improves\ntranslation performance. Secondly, by qualitatively studying attention plots\nfrom the decoder we find that the model learns to compress common words into a\nsingle embedding whereas rare words, such as names and places, are represented\ncharacter by character.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06602v3",
            "date": "2016-10-20 20:54:07+00:00",
            "title": "Iterative Refinement for Machine Translation",
            "authors": [
                "Roman Novak",
                "Michael Auli",
                "David Grangier"
            ],
            "abstract": "Existing machine translation decoding algorithms generate translations in a\nstrictly monotonic fashion and never revisit previous decisions. As a result,\nearlier mistakes cannot be corrected at a later stage. In this paper, we\npresent a translation scheme that starts from an initial guess and then makes\niterative improvements that may revisit previous decisions. We parameterize our\nmodel as a convolutional neural network that predicts discrete substitutions to\nan existing translation based on an attention mechanism over both the source\nsentence as well as the current translation output. By making less than one\nmodification per sentence, we improve the output of a phrase-based translation\nsystem by up to 0.4 BLEU on WMT15 German-English translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07091v2",
            "date": "2016-10-22 19:50:01+00:00",
            "title": "Automatic Identification of Sarcasm Target: An Introductory Approach",
            "authors": [
                "Aditya Joshi",
                "Pranav Goel",
                "Pushpak Bhattacharyya",
                "Mark Carman"
            ],
            "abstract": "Past work in computational sarcasm deals primarily with sarcasm detection. In\nthis paper, we introduce a novel, related problem: sarcasm target\nidentification i.e., extracting the target of ridicule in a sarcastic\nsentence). We present an introductory approach for sarcasm target\nidentification. Our approach employs two types of extractors: one based on\nrules, and another consisting of a statistical classifier. To compare our\napproach, we use two baselines: a na\\\"ive baseline and another baseline based\non work in sentiment target identification. We perform our experiments on book\nsnippets and tweets, and show that our hybrid approach performs better than the\ntwo baselines and also, in comparison with using the two extractors\nindividually. Our introductory approach establishes the viability of sarcasm\ntarget identification, and will serve as a baseline for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07149v1",
            "date": "2016-10-23 11:22:40+00:00",
            "title": "Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems",
            "authors": [
                "Yiping Song",
                "Rui Yan",
                "Xiang Li",
                "Dongyan Zhao",
                "Ming Zhang"
            ],
            "abstract": "Open-domain human-computer conversation has attracted much attention in the\nfield of NLP. Contrary to rule- or template-based domain-specific dialog\nsystems, open-domain conversation usually requires data-driven approaches,\nwhich can be roughly divided into two categories: retrieval-based and\ngeneration-based systems. Retrieval systems search a user-issued utterance\n(called a query) in a large database, and return a reply that best matches the\nquery. Generative approaches, typically based on recurrent neural networks\n(RNNs), can synthesize new replies, but they suffer from the problem of\ngenerating short, meaningless utterances. In this paper, we propose a novel\nensemble of retrieval-based and generation-based dialog systems in the open\ndomain. In our approach, the retrieved candidate, in addition to the original\nquery, is fed to an RNN-based reply generator, so that the neural model is\naware of more information. The generated reply is then fed back as a new\ncandidate for post-reranking. Experimental results show that such ensemble\noutperforms each single part of it by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07272v1",
            "date": "2016-10-24 03:39:22+00:00",
            "title": "Bridging Neural Machine Translation and Bilingual Dictionaries",
            "authors": [
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Neural Machine Translation (NMT) has become the new state-of-the-art in\nseveral language pairs. However, it remains a challenging problem how to\nintegrate NMT with a bilingual dictionary which mainly contains words rarely or\nnever seen in the bilingual training data. In this paper, we propose two\nmethods to bridge NMT and the bilingual dictionaries. The core idea behind is\nto design novel models that transform the bilingual dictionaries into adequate\nsentence pairs, so that NMT can distil latent bilingual mappings from the ample\nand repetitive phenomena. One method leverages a mixed word/character model and\nthe other attempts at synthesizing parallel sentences guaranteeing massive\noccurrence of the translation lexicon. Extensive experiments demonstrate that\nthe proposed methods can remarkably improve the translation quality, and most\nof the rare words in the test sentences can obtain correct translations if they\nare covered by the dictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07418v1",
            "date": "2016-10-24 14:06:31+00:00",
            "title": "Statistical Machine Translation for Indian Languages: Mission Hindi",
            "authors": [
                "Raj Nath Patel",
                "Prakash B. Pimpale",
                "Sasikumar M"
            ],
            "abstract": "This paper discusses Centre for Development of Advanced Computing Mumbai's\n(CDACM) submission to the NLP Tools Contest on Statistical Machine Translation\nin Indian Languages (ILSMT) 2014 (collocated with ICON 2014). The objective of\nthe contest was to explore the effectiveness of Statistical Machine Translation\n(SMT) for Indian language to Indian language and English-Hindi machine\ntranslation. In this paper, we have proposed that suffix separation and word\nsplitting for SMT from agglutinative languages to Hindi significantly improves\nover the baseline (BL). We have also shown that the factored model with\nreordering outperforms the phrase-based SMT for English-Hindi (\\enhi). We\nreport our work on all five pairs of languages, namely Bengali-Hindi (\\bnhi),\nMarathi-Hindi (\\mrhi), Tamil-Hindi (\\tahi), Telugu-Hindi (\\tehi), and \\enhi for\nHealth, Tourism, and General domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07420v1",
            "date": "2016-10-24 14:08:59+00:00",
            "title": "Reordering rules for English-Hindi SMT",
            "authors": [
                "Raj Nath Patel",
                "Rohit Gupta",
                "Prakash B. Pimpale",
                "Sasikumar M"
            ],
            "abstract": "Reordering is a preprocessing stage for Statistical Machine Translation (SMT)\nsystem where the words of the source sentence are reordered as per the syntax\nof the target language. We are proposing a rich set of rules for better\nreordering. The idea is to facilitate the training process by better alignments\nand parallel phrase extraction for a phrase-based SMT system. Reordering also\nhelps the decoding process and hence improving the machine translation quality.\nWe have observed significant improvements in the translation quality by using\nour approach over the baseline SMT. We have used BLEU, NIST, multi-reference\nword error rate, multi-reference position independent error rate for judging\nthe improvements. We have exploited open source SMT toolkit MOSES to develop\nthe system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07651v1",
            "date": "2016-10-24 21:05:05+00:00",
            "title": "UTD-CRSS Systems for 2016 NIST Speaker Recognition Evaluation",
            "authors": [
                "Chunlei Zhang",
                "Fahimeh Bahmaninezhad",
                "Shivesh Ranjan",
                "Chengzhu Yu",
                "Navid Shokouhi",
                "John H. L. Hansen"
            ],
            "abstract": "This document briefly describes the systems submitted by the Center for\nRobust Speech Systems (CRSS) from The University of Texas at Dallas (UTD) to\nthe 2016 National Institute of Standards and Technology (NIST) Speaker\nRecognition Evaluation (SRE). We developed several UBM and DNN i-Vector based\nspeaker recognition systems with different data sets and feature\nrepresentations. Given that the emphasis of the NIST SRE 2016 is on language\nmismatch between training and enrollment/test data, so-called domain mismatch,\nin our system development we focused on: (1) using unlabeled in-domain data for\ncentralizing data to alleviate the domain mismatch problem, (2) finding the\nbest data set for training LDA/PLDA, (3) using newly proposed dimension\nreduction technique incorporating unlabeled in-domain data before PLDA\ntraining, (4) unsupervised speaker clustering of unlabeled data and using them\nalone or with previous SREs for PLDA training, (5) score calibration using only\nunlabeled data and combination of unlabeled and development (Dev) data as\nseparate experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07710v1",
            "date": "2016-10-25 02:36:52+00:00",
            "title": "EmojiNet: Building a Machine Readable Sense Inventory for Emoji",
            "authors": [
                "Sanjaya Wijeratne",
                "Lakshika Balasuriya",
                "Amit Sheth",
                "Derek Doran"
            ],
            "abstract": "Emoji are a contemporary and extremely popular way to enhance electronic\ncommunication. Without rigid semantics attached to them, emoji symbols take on\ndifferent meanings based on the context of a message. Thus, like the word sense\ndisambiguation task in natural language processing, machines also need to\ndisambiguate the meaning or sense of an emoji. In a first step toward achieving\nthis goal, this paper presents EmojiNet, the first machine readable sense\ninventory for emoji. EmojiNet is a resource enabling systems to link emoji with\ntheir context-specific meaning. It is automatically constructed by integrating\nmultiple emoji resources with BabelNet, which is the most comprehensive\nmultilingual sense inventory available to date. The paper discusses its\nconstruction, evaluates the automatic resource creation process, and presents a\nuse case where EmojiNet disambiguates emoji usage in tweets. EmojiNet is\navailable online for use at http://emojinet.knoesis.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07796v2",
            "date": "2016-10-25 09:14:05+00:00",
            "title": "Still not there? Comparing Traditional Sequence-to-Sequence Models to Encoder-Decoder Neural Networks on Monotone String Translation Tasks",
            "authors": [
                "Carsten Schnober",
                "Steffen Eger",
                "Erik-L\u00e2n Do Dinh",
                "Iryna Gurevych"
            ],
            "abstract": "We analyze the performance of encoder-decoder neural models and compare them\nwith well-known established methods. The latter represent different classes of\ntraditional approaches that are applied to the monotone sequence-to-sequence\ntasks OCR post-correction, spelling correction, grapheme-to-phoneme conversion,\nand lemmatization. Such tasks are of practical relevance for various\nhigher-level research fields including digital humanities, automatic text\ncorrection, and speech recognition. We investigate how well generic\ndeep-learning approaches adapt to these tasks, and how they perform in\ncomparison with established and more specialized methods, including our own\nadaptation of pruned CRFs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07809v1",
            "date": "2016-10-25 09:59:13+00:00",
            "title": "How Document Pre-processing affects Keyphrase Extraction Performance",
            "authors": [
                "Florian Boudin",
                "Hugo Mougard",
                "Damien Cram"
            ],
            "abstract": "The SemEval-2010 benchmark dataset has brought renewed attention to the task\nof automatic keyphrase extraction. This dataset is made up of scientific\narticles that were automatically converted from PDF format to plain text and\nthus require careful preprocessing so that irrevelant spans of text do not\nnegatively affect keyphrase extraction performance. In previous work, a wide\nrange of document preprocessing techniques were described but their impact on\nthe overall performance of keyphrase extraction models is still unexplored.\nHere, we re-assess the performance of several keyphrase extraction models and\nmeasure their robustness against increasingly sophisticated levels of document\npreprocessing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07844v1",
            "date": "2016-10-25 12:30:26+00:00",
            "title": "Improving historical spelling normalization with bi-directional LSTMs and multi-task learning",
            "authors": [
                "Marcel Bollmann",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Natural-language processing of historical documents is complicated by the\nabundance of variant spellings and lack of annotated data. A common approach is\nto normalize the spelling of historical words to modern forms. We explore the\nsuitability of a deep neural network architecture for this task, particularly a\ndeep bi-LSTM network applied on a character level. Our model compares well to\npreviously established normalization algorithms when evaluated on a diverse set\nof texts from Early New High German. We show that multi-task learning with\nadditional normalization data can improve our model's performance further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07918v1",
            "date": "2016-10-25 15:21:25+00:00",
            "title": "Sequence Segmentation Using Joint RNN and Structured Prediction Models",
            "authors": [
                "Yossi Adi",
                "Joseph Keshet",
                "Emily Cibelli",
                "Matthew Goldrick"
            ],
            "abstract": "We describe and analyze a simple and effective algorithm for sequence\nsegmentation applied to speech processing tasks. We propose a neural\narchitecture that is composed of two modules trained jointly: a recurrent\nneural network (RNN) module and a structured prediction model. The RNN outputs\nare considered as feature functions to the structured model. The overall model\nis trained with a structured loss function which can be designed to the given\nsegmentation task. We demonstrate the effectiveness of our method by applying\nit to two simple tasks commonly used in phonetic studies: word segmentation and\nvoice onset time segmentation. Results sug- gest the proposed model is superior\nto previous methods, ob- taining state-of-the-art results on the tested\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08000v1",
            "date": "2016-10-25 18:20:08+00:00",
            "title": "Statistical Machine Translation for Indian Languages: Mission Hindi 2",
            "authors": [
                "Raj Nath Patel",
                "Prakash B. Pimpale"
            ],
            "abstract": "This paper presents Centre for Development of Advanced Computing Mumbai's\n(CDACM) submission to NLP Tools Contest on Statistical Machine Translation in\nIndian Languages (ILSMT) 2015 (collocated with ICON 2015). The aim of the\ncontest was to collectively explore the effectiveness of Statistical Machine\nTranslation (SMT) while translating within Indian languages and between English\nand Indian languages. In this paper, we report our work on all five language\npairs, namely Bengali-Hindi (\\bnhi), Marathi-Hindi (\\mrhi), Tamil-Hindi\n(\\tahi), Telugu-Hindi (\\tehi), and English-Hindi (\\enhi) for Health, Tourism,\nand General domains. We have used suffix separation, compound splitting and\npreordering prior to SMT training and testing.",
            "journal ref": "In the Proceedings of the 12th International Conference on Natural\n  Language Processing (ICON 2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08375v1",
            "date": "2016-10-26 15:20:47+00:00",
            "title": "Content Selection in Data-to-Text Systems: A Survey",
            "authors": [
                "Dimitra Gkatzia"
            ],
            "abstract": "Data-to-text systems are powerful in generating reports from data\nautomatically and thus they simplify the presentation of complex data. Rather\nthan presenting data using visualisation techniques, data-to-text systems use\nnatural (human) language, which is the most common way for human-human\ncommunication. In addition, data-to-text systems can adapt their output content\nto users' preferences, background or interests and therefore they can be\npleasant for users to interact with. Content selection is an important part of\nevery data-to-text system, because it is the module that determines which from\nthe available information should be conveyed to the user. This survey initially\nintroduces the field of data-to-text generation, describes the general\ndata-to-text system architecture and then it reviews the state-of-the-art\ncontent selection methods. Finally, it provides recommendations for choosing an\napproach and discusses opportunities for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08431v3",
            "date": "2016-10-26 17:25:38+00:00",
            "title": "Broad Context Language Modeling as Reading Comprehension",
            "authors": [
                "Zewei Chu",
                "Hai Wang",
                "Kevin Gimpel",
                "David McAllester"
            ],
            "abstract": "Progress in text understanding has been driven by large datasets that test\nparticular capabilities, like recent datasets for reading comprehension\n(Hermann et al., 2015). We focus here on the LAMBADA dataset (Paperno et al.,\n2016), a word prediction task requiring broader context than the immediate\nsentence. We view LAMBADA as a reading comprehension problem and apply\ncomprehension models based on neural networks. Though these models are\nconstrained to choose a word from the context, they improve the state of the\nart on LAMBADA from 7.3% to 49%. We analyze 100 instances, finding that neural\nnetwork readers perform well in cases that involve selecting a name from the\ncontext based on dialogue or discourse cues but struggle when coreference\nresolution or external knowledge is needed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08462v1",
            "date": "2016-10-26 18:57:00+00:00",
            "title": "Distraction-Based Neural Networks for Document Summarization",
            "authors": [
                "Qian Chen",
                "Xiaodan Zhu",
                "Zhenhua Ling",
                "Si Wei",
                "Hui Jiang"
            ],
            "abstract": "Distributed representation learned with neural networks has recently shown to\nbe effective in modeling natural languages at fine granularities such as words,\nphrases, and even sentences. Whether and how such an approach can be extended\nto help model larger spans of text, e.g., documents, is intriguing, and further\ninvestigation would still be desirable. This paper aims to enhance neural\nnetwork models for such a purpose. A typical problem of document-level modeling\nis automatic summarization, which aims to model documents in order to generate\nsummaries. In this paper, we propose neural models to train computers not just\nto pay attention to specific regions and content of input documents with\nattention models, but also distract them to traverse between different content\nof a document so as to better grasp the overall meaning for summarization.\nWithout engineering any features, we train the models on two large datasets.\nThe models achieve the state-of-the-art performance, and they significantly\nbenefit from the distraction modeling, particularly when input documents are\nlong.",
            "journal ref": "IJCAI, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08557v5",
            "date": "2016-10-26 21:49:15+00:00",
            "title": "Knowledge-Based Biomedical Word Sense Disambiguation with Neural Concept Embeddings",
            "authors": [
                "A. K. M. Sabbir",
                "Antonio Jimeno Yepes",
                "Ramakanth Kavuluru"
            ],
            "abstract": "Biomedical word sense disambiguation (WSD) is an important intermediate task\nin many natural language processing applications such as named entity\nrecognition, syntactic parsing, and relation extraction. In this paper, we\nemploy knowledge-based approaches that also exploit recent advances in neural\nword/concept embeddings to improve over the state-of-the-art in biomedical WSD\nusing the MSH WSD dataset as the test set. Our methods involve weak supervision\n- we do not use any hand-labeled examples for WSD to build our prediction\nmodels; however, we employ an existing well known named entity recognition and\nconcept mapping program, MetaMap, to obtain our concept vectors. Over the MSH\nWSD dataset, our linear time (in terms of numbers of senses and words in the\ntest instance) method achieves an accuracy of 92.24% which is an absolute 3%\nimprovement over the best known results obtained via unsupervised or\nknowledge-based means. A more expensive approach that we developed relies on a\nnearest neighbor framework and achieves an accuracy of 94.34%. Employing dense\nvector representations learned from unlabeled free text has been shown to\nbenefit many language processing tasks recently and our efforts show that\nbiomedical WSD is no exception to this trend. For a complex and rapidly\nevolving domain such as biomedicine, building labeled datasets for larger sets\nof ambiguous terms may be impractical. Here, we show that weak supervision that\nleverages recent advances in representation learning can rival supervised\napproaches in biomedical WSD. However, external knowledge bases (here sense\ninventories) play a key role in the improvements achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08694v3",
            "date": "2016-10-27 10:49:00+00:00",
            "title": "CogALex-V Shared Task: LexNET - Integrated Path-based and Distributional Method for the Identification of Semantic Relations",
            "authors": [
                "Vered Shwartz",
                "Ido Dagan"
            ],
            "abstract": "We present a submission to the CogALex 2016 shared task on the corpus-based\nidentification of semantic relations, using LexNET (Shwartz and Dagan, 2016),\nan integrated path-based and distributional method for semantic relation\nclassification. The reported results in the shared task bring this submission\nto the third place on subtask 1 (word relatedness), and the first place on\nsubtask 2 (semantic relation classification), demonstrating the utility of\nintegrating the complementary path-based and distributional information sources\nin recognizing concrete semantic relations. Combined with a common similarity\nmeasure, LexNET performs fairly good on the word relatedness task (subtask 1).\nThe relatively low performance of LexNET and all other systems on subtask 2,\nhowever, confirms the difficulty of the semantic relation classification task,\nand stresses the need to develop additional methods for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08815v2",
            "date": "2016-10-27 14:50:43+00:00",
            "title": "A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks",
            "authors": [
                "Soujanya Poria",
                "Erik Cambria",
                "Devamanyu Hazarika",
                "Prateek Vij"
            ],
            "abstract": "Sarcasm detection is a key task for many natural language processing tasks.\nIn sentiment analysis, for example, sarcasm can flip the polarity of an\n\"apparently positive\" sentence and, hence, negatively affect polarity detection\nperformance. To date, most approaches to sarcasm detection have treated the\ntask primarily as a text categorization problem. Sarcasm, however, can be\nexpressed in very subtle ways and requires a deeper understanding of natural\nlanguage that standard text categorization techniques cannot grasp. In this\nwork, we develop models based on a pre-trained convolutional neural network for\nextracting sentiment, emotion and personality features for sarcasm detection.\nSuch features, along with the network's baseline features, allow the proposed\nmodels to outperform the state of the art on benchmark datasets. We also\naddress the often ignored generalizability issue of classifying data that have\nnot been seen by the models at learning phase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08914v2",
            "date": "2016-10-27 18:18:18+00:00",
            "title": "Ex Machina: Personal Attacks Seen at Scale",
            "authors": [
                "Ellery Wulczyn",
                "Nithum Thain",
                "Lucas Dixon"
            ],
            "abstract": "The damage personal attacks cause to online discourse motivates many\nplatforms to try to curb the phenomenon. However, understanding the prevalence\nand impact of personal attacks in online platforms at scale remains\nsurprisingly difficult. The contribution of this paper is to develop and\nillustrate a method that combines crowdsourcing and machine learning to analyze\npersonal attacks at scale. We show an evaluation method for a classifier in\nterms of the aggregated number of crowd-workers it can approximate. We apply\nour methodology to English Wikipedia, generating a corpus of over 100k high\nquality human-labeled comments and 63M machine-labeled ones from a classifier\nthat is as good as the aggregate of 3 crowd-workers, as measured by the area\nunder the ROC curve and Spearman correlation. Using this corpus of\nmachine-labeled scores, our methodology allows us to explore some of the open\nquestions about the nature of online personal attacks. This reveals that the\nmajority of personal attacks on Wikipedia are not the result of a few malicious\nusers, nor primarily the consequence of allowing anonymous contributions from\nunregistered users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09091v3",
            "date": "2016-10-28 06:33:33+00:00",
            "title": "Representation Learning Models for Entity Search",
            "authors": [
                "Shijia E",
                "Yang Xiang",
                "Mohan Zhang"
            ],
            "abstract": "We focus on the problem of learning distributed representations for entity\nsearch queries, named entities, and their short descriptions. With our\nrepresentation learning models, the entity search query, named entity and\ndescription can be represented as low-dimensional vectors. Our goal is to\ndevelop a simple but effective model that can make the distributed\nrepresentations of query related entities similar to the query in the vector\nspace. Hence, we propose three kinds of learning strategies, and the difference\nbetween them mainly lies in how to deal with the relationship between an entity\nand its description. We analyze the strengths and weaknesses of each learning\nstrategy and validate our methods on public datasets which contain four kinds\nof named entities, i.e., movies, TV shows, restaurants and celebrities. The\nexperimental results indicate that our proposed methods can adapt to different\ntypes of entity search queries, and outperform the current state-of-the-art\nmethods based on keyword matching and vanilla word2vec models. Besides, the\nproposed methods can be trained fast and be easily extended to other similar\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09333v1",
            "date": "2016-10-28 18:15:08+00:00",
            "title": "Word Embeddings for the Construction Domain",
            "authors": [
                "Antoine J. -P. Tixier",
                "Michalis Vazirgiannis",
                "Matthew R. Hallowell"
            ],
            "abstract": "We introduce word vectors for the construction domain. Our vectors were\nobtained by running word2vec on an 11M-word corpus that we created from scratch\nby leveraging freely-accessible online sources of construction-related text. We\nfirst explore the embedding space and show that our vectors capture meaningful\nconstruction-specific concepts. We then evaluate the performance of our vectors\nagainst that of ones trained on a 100B-word corpus (Google News) within the\nframework of an injury report classification task. Without any parameter\ntuning, our embeddings give competitive results, and outperform the Google News\nvectors in many cases. Using a keyword-based compression of the reports also\nleads to a significant speed-up with only a limited loss in performance. We\nrelease our corpus and the data set we created for the classification task as\npublicly available, in the hope that they will be used by future studies for\nbenchmarking and building on our work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09565v1",
            "date": "2016-10-29 19:21:19+00:00",
            "title": "Sequence-to-sequence neural network models for transliteration",
            "authors": [
                "Mihaela Rosca",
                "Thomas Breuel"
            ],
            "abstract": "Transliteration is a key component of machine translation systems and\nsoftware internationalization. This paper demonstrates that neural\nsequence-to-sequence models obtain state of the art or close to state of the\nart results on existing datasets. In an effort to make machine transliteration\naccessible, we open source a new Arabic to English transliteration dataset and\nour trained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09722v1",
            "date": "2016-10-30 22:33:47+00:00",
            "title": "Represent, Aggregate, and Constrain: A Novel Architecture for Machine Reading from Noisy Sources",
            "authors": [
                "Jason Naradowsky",
                "Sebastian Riedel"
            ],
            "abstract": "In order to extract event information from text, a machine reading model must\nlearn to accurately read and interpret the ways in which that information is\nexpressed. But it must also, as the human reader must, aggregate numerous\nindividual value hypotheses into a single coherent global analysis, applying\nglobal constraints which reflect prior knowledge of the domain.\n  In this work we focus on the task of extracting plane crash event information\nfrom clusters of related news articles whose labels are derived via distant\nsupervision. Unlike previous machine reading work, we assume that while most\ntarget values will occur frequently in most clusters, they may also be missing\nor incorrect.\n  We introduce a novel neural architecture to explicitly model the noisy nature\nof the data and to deal with these aforementioned learning issues. Our models\nare trained end-to-end and achieve an improvement of more than 12.1 F$_1$ over\nprevious work, despite using far less linguistic annotation. We apply factor\ngraph constraints to promote more coherent event analyses, with belief\npropagation inference formulated within the transitions of a recurrent neural\nnetwork. We show this technique additionally improves maximum F$_1$ by up to\n2.8 points, resulting in a relative improvement of $50\\%$ over the previous\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09799v1",
            "date": "2016-10-31 06:13:31+00:00",
            "title": "Experiments with POS Tagging Code-mixed Indian Social Media Text",
            "authors": [
                "Prakash B. Pimpale",
                "Raj Nath Patel"
            ],
            "abstract": "This paper presents Centre for Development of Advanced Computing Mumbai's\n(CDACM) submission to the NLP Tools Contest on Part-Of-Speech (POS) Tagging For\nCode-mixed Indian Social Media Text (POSCMISMT) 2015 (collocated with ICON\n2015). We submitted results for Hindi (hi), Bengali (bn), and Telugu (te)\nlanguages mixed with English (en). In this paper, we have described our\napproaches to the POS tagging techniques, we exploited for this task. Machine\nlearning has been used to POS tag the mixed language text. For POS tagging,\ndistributed representations of words in vector space (word2vec) for feature\nextraction and Log-linear models have been tried. We report our work on all\nthree languages hi, bn, and te mixed with en.",
            "journal ref": "In the Proceedings of the 12th International Conference on Natural\n  Language Processing (ICON 2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09914v1",
            "date": "2016-10-31 13:36:35+00:00",
            "title": "Named Entity Recognition for Novel Types by Transfer Learning",
            "authors": [
                "Lizhen Qu",
                "Gabriela Ferraro",
                "Liyuan Zhou",
                "Weiwei Hou",
                "Timothy Baldwin"
            ],
            "abstract": "In named entity recognition, we often don't have a large in-domain training\ncorpus or a knowledge base with adequate coverage to train a model directly. In\nthis paper, we propose a method where, given training data in a related domain\nwith similar (but not identical) named entity (NE) types and a small amount of\nin-domain training data, we use transfer learning to learn a domain-specific NE\nmodel. That is, the novelty in the task setup is that we assume not just domain\nmismatch, but also label mismatch.",
            "journal ref": "EMNLP 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09935v2",
            "date": "2016-10-31 14:27:07+00:00",
            "title": "Knowledge Questions from Knowledge Graphs",
            "authors": [
                "Dominic Seyler",
                "Mohamed Yahya",
                "Klaus Berberich"
            ],
            "abstract": "We address the novel problem of automatically generating quiz-style knowledge\nquestions from a knowledge graph such as DBpedia. Questions of this kind have\nample applications, for instance, to educate users about or to evaluate their\nknowledge in a specific domain. To solve the problem, we propose an end-to-end\napproach. The approach first selects a named entity from the knowledge graph as\nan answer. It then generates a structured triple-pattern query, which yields\nthe answer as its sole result. If a multiple-choice question is desired, the\napproach selects alternative answer options. Finally, our approach uses a\ntemplate-based method to verbalize the structured query and yield a natural\nlanguage question. A key challenge is estimating how difficult the generated\nquestion is to human users. To do this, we make use of historical data from the\nJeopardy! quiz show and a semantically annotated Web-scale document collection,\nengineer suitable features, and train a logistic regression classifier to\npredict question difficulty. Experiments demonstrate the viability of our\noverall approach.",
            "journal ref": "ICTIR (2017) 11-18",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09995v1",
            "date": "2016-10-31 16:12:16+00:00",
            "title": "Generating Sentiment Lexicons for German Twitter",
            "authors": [
                "Uladzimir Sidarenka",
                "Manfred Stede"
            ],
            "abstract": "Despite a substantial progress made in developing new sentiment lexicon\ngeneration (SLG) methods for English, the task of transferring these approaches\nto other languages and domains in a sound way still remains open. In this\npaper, we contribute to the solution of this problem by systematically\ncomparing semi-automatic translations of common English polarity lists with the\nresults of the original automatic SLG algorithms, which were applied directly\nto German data. We evaluate these lexicons on a corpus of 7,992 manually\nannotated tweets. In addition to that, we also collate the results of\ndictionary- and corpus-based SLG methods in order to find out which of these\nparadigms is better suited for the inherently noisy domain of social media. Our\nexperiments show that semi-automatic translations notably outperform automatic\nsystems (reaching a macro-averaged F1-score of 0.589), and that\ndictionary-based techniques produce much better polarity lists as compared to\ncorpus-based approaches (whose best F1-scores run up to 0.479 and 0.419\nrespectively) even for the non-standard Twitter genre.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09996v2",
            "date": "2016-10-31 16:14:08+00:00",
            "title": "End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension",
            "authors": [
                "Yang Yu",
                "Wei Zhang",
                "Kazi Hasan",
                "Mo Yu",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "This paper proposes dynamic chunk reader (DCR), an end-to-end neural reading\ncomprehension (RC) model that is able to extract and rank a set of answer\ncandidates from a given document to answer questions. DCR is able to predict\nanswers of variable lengths, whereas previous neural RC models primarily\nfocused on predicting single tokens or entities. DCR encodes a document and an\ninput question with recurrent neural networks, and then applies a word-by-word\nattention mechanism to acquire question-aware representations for the document,\nfollowed by the generation of chunk representations and a ranking module to\npropose the top-ranked chunk as the answer. Experimental results show that DCR\nachieves state-of-the-art exact match and F1 scores on the SQuAD dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00246v1",
            "date": "2016-12-01 14:01:00+00:00",
            "title": "Multilingual Multiword Expressions",
            "authors": [
                "Lahari Poddar"
            ],
            "abstract": "The project aims to provide a semi-supervised approach to identify Multiword\nExpressions in a multilingual context consisting of English and most of the\nmajor Indian languages. Multiword expressions are a group of words which refers\nto some conventional or regional way of saying things. If they are literally\ntranslated from one language to another the expression will lose its inherent\nmeaning.\n  To automatically extract multiword expressions from a corpus, an extraction\npipeline have been constructed which consist of a combination of rule based and\nstatistical approaches. There are several types of multiword expressions which\ndiffer from each other widely by construction. We employ different methods to\ndetect different types of multiword expressions. Given a POS tagged corpus in\nEnglish or any Indian language the system initially applies some regular\nexpression filters to narrow down the search space to certain patterns (like,\nreduplication, partial reduplication, compound nouns, compound verbs, conjunct\nverbs etc.). The word sequences matching the required pattern are subjected to\na series of linguistic tests which include verb filtering, named entity\nfiltering and hyphenation filtering test to exclude false positives. The\ncandidates are then checked for semantic relationships among themselves (using\nWordnet). In order to detect partial reduplication we make use of Wordnet as a\nlexical database as well as a tool for lemmatising. We detect complex\npredicates by investigating the features of the constituent words. Statistical\nmethods are applied to detect collocations. Finally, lexicographers examine the\nlist of automatically extracted candidates to validate whether they are true\nmultiword expressions or not and add them to the multiword dictionary\naccordingly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00394v1",
            "date": "2016-12-01 19:42:37+00:00",
            "title": "Definition Modeling: Learning to define word embeddings in natural language",
            "authors": [
                "Thanapon Noraset",
                "Chen Liang",
                "Larry Birnbaum",
                "Doug Downey"
            ],
            "abstract": "Distributed representations of words have been shown to capture lexical\nsemantics, as demonstrated by their effectiveness in word similarity and\nanalogical relation tasks. But, these tasks only evaluate lexical semantics\nindirectly. In this paper, we study whether it is possible to utilize\ndistributed representations to generate dictionary definitions of words, as a\nmore direct and transparent representation of the embeddings' semantics. We\nintroduce definition modeling, the task of generating a definition for a given\nword and its embedding. We present several definition model architectures based\non recurrent neural networks, and experiment with the models over multiple data\nsets. Our results show that a model that controls dependencies between the word\nbeing defined and the definition words performs significantly better, and that\na character-level convolution layer designed to leverage morphology can\ncomplement word-level embeddings. Finally, an error analysis suggests that the\nerrors made by a definition model may provide insight into the shortcomings of\nword embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00467v1",
            "date": "2016-12-01 21:06:54+00:00",
            "title": "Neural Document Embeddings for Intensive Care Patient Mortality Prediction",
            "authors": [
                "Paulina Grnarova",
                "Florian Schmidt",
                "Stephanie L. Hyland",
                "Carsten Eickhoff"
            ],
            "abstract": "We present an automatic mortality prediction scheme based on the unstructured\ntextual content of clinical notes. Proposing a convolutional document embedding\napproach, our empirical investigation using the MIMIC-III intensive care\ndatabase shows significant performance gains compared to previously employed\nmethods such as latent topic distributions or generic doc2vec embeddings. These\nimprovements are especially pronounced for the difficult problem of\npost-discharge mortality prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00567v1",
            "date": "2016-12-02 04:55:24+00:00",
            "title": "Shift-Reduce Constituent Parsing with Neural Lookahead Features",
            "authors": [
                "Jiangming Liu",
                "Yue Zhang"
            ],
            "abstract": "Transition-based models can be fast and accurate for constituent parsing.\nCompared with chart-based models, they leverage richer features by extracting\nhistory information from a parser stack, which spans over non-local\nconstituents. On the other hand, during incremental parsing, constituent\ninformation on the right hand side of the current word is not utilized, which\nis a relative weakness of shift-reduce parsing. To address this limitation, we\nleverage a fast neural model to extract lookahead features. In particular, we\nbuild a bidirectional LSTM model, which leverages the full sentence information\nto predict the hierarchy of constituents that each word starts and ends. The\nresults are then passed to a strong transition-based constituent parser as\nlookahead features. The resulting parser gives 1.3% absolute improvement in WSJ\nand 2.3% in CTB compared to the baseline, given the highest reported accuracies\nfor fully-supervised parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00584v2",
            "date": "2016-12-02 07:45:40+00:00",
            "title": "Alleviating Overfitting for Polysemous Words for Word Representation Estimation Using Lexicons",
            "authors": [
                "Yuanzhi Ke",
                "Masafumi Hagiwara"
            ],
            "abstract": "Though there are some works on improving distributed word representations\nusing lexicons, the improper overfitting of the words that have multiple\nmeanings is a remaining issue deteriorating the learning when lexicons are\nused, which needs to be solved. An alternative method is to allocate a vector\nper sense instead of a vector per word. However, the word representations\nestimated in the former way are not as easy to use as the latter one. Our\nprevious work uses a probabilistic method to alleviate the overfitting, but it\nis not robust with a small corpus. In this paper, we propose a new neural\nnetwork to estimate distributed word representations using a lexicon and a\ncorpus. We add a lexicon layer in the continuous bag-of-words model and a\nthreshold node after the output of the lexicon layer. The threshold rejects the\nunreliable outputs of the lexicon layer that are less likely to be the same\nwith their inputs. In this way, it alleviates the overfitting of the polysemous\nwords. The proposed neural network can be trained using negative sampling,\nwhich maximizing the log probabilities of target words given the context words,\nby distinguishing the target words from random noises. We compare the proposed\nneural network with the continuous bag-of-words model, the other works\nimproving it, and the previous works estimating distributed word\nrepresentations using both a lexicon and a corpus. The experimental results\nshow that the proposed neural network is more efficient and balanced for both\nsemantic tasks and syntactic tasks than the previous works, and robust to the\nsize of the corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00694v2",
            "date": "2016-12-01 13:16:00+00:00",
            "title": "ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA",
            "authors": [
                "Song Han",
                "Junlong Kang",
                "Huizi Mao",
                "Yiming Hu",
                "Xin Li",
                "Yubin Li",
                "Dongliang Xie",
                "Hong Luo",
                "Song Yao",
                "Yu Wang",
                "Huazhong Yang",
                "William J. Dally"
            ],
            "abstract": "Long Short-Term Memory (LSTM) is widely used in speech recognition. In order\nto achieve higher prediction accuracy, machine learning scientists have built\nlarger and larger models. Such large model is both computation intensive and\nmemory intensive. Deploying such bulky model results in high power consumption\nand leads to high total cost of ownership (TCO) of a data center. In order to\nspeedup the prediction and make it energy efficient, we first propose a\nload-balance-aware pruning method that can compress the LSTM model size by 20x\n(10x from pruning and 2x from quantization) with negligible loss of the\nprediction accuracy. The pruned model is friendly for parallel processing.\nNext, we propose scheduler that encodes and partitions the compressed model to\neach PE for parallelism, and schedule the complicated LSTM data flow. Finally,\nwe design the hardware architecture, named Efficient Speech Recognition Engine\n(ESE) that works directly on the compressed model. Implemented on Xilinx\nXCKU060 FPGA running at 200MHz, ESE has a performance of 282 GOPS working\ndirectly on the compressed LSTM network, corresponding to 2.52 TOPS on the\nuncompressed one, and processes a full LSTM for speech recognition with a power\ndissipation of 41 Watts. Evaluated on the LSTM for speech recognition\nbenchmark, ESE is 43x and 3x faster than Core i7 5930k CPU and Pascal Titan X\nGPU implementations. It achieves 40x and 11.5x higher energy efficiency\ncompared with the CPU and GPU respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00729v1",
            "date": "2016-12-02 16:22:49+00:00",
            "title": "Automated assessment of non-native learner essays: Investigating the role of linguistic features",
            "authors": [
                "Sowmya Vajjala"
            ],
            "abstract": "Automatic essay scoring (AES) refers to the process of scoring free text\nresponses to given prompts, considering human grader scores as the gold\nstandard. Writing such essays is an essential component of many language and\naptitude exams. Hence, AES became an active and established area of research,\nand there are many proprietary systems used in real life applications today.\nHowever, not much is known about which specific linguistic features are useful\nfor prediction and how much of this is consistent across datasets. This article\naddresses that by exploring the role of various linguistic features in\nautomatic essay scoring using two publicly available datasets of non-native\nEnglish essays written in test taking scenarios. The linguistic properties are\nmodeled by encoding lexical, syntactic, discourse and error types of learner\nlanguage in the feature set. Predictive models are then developed using these\nfeatures on both datasets and the most predictive features are compared. While\nthe results show that the feature set used results in good predictive models\nwith both datasets, the question \"what are the most predictive features?\" has a\ndifferent answer for each dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00866v1",
            "date": "2016-12-02 21:28:00+00:00",
            "title": "Creating a Real-Time, Reproducible Event Dataset",
            "authors": [
                "John Beieler"
            ],
            "abstract": "The generation of political event data has remained much the same since the\nmid-1990s, both in terms of data acquisition and the process of coding text\ninto data. Since the 1990s, however, there have been significant improvements\nin open-source natural language processing software and in the availability of\ndigitized news content. This paper presents a new, next-generation event\ndataset, named Phoenix, that builds from these and other advances. This dataset\nincludes improvements in the underlying news collection process and event\ncoding software, along with the creation of a general processing pipeline\nnecessary to produce daily-updated data. This paper provides a face validity\nchecks by briefly examining the data for the conflict in Syria, and a\ncomparison between Phoenix and the Integrated Crisis Early Warning System data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00969v1",
            "date": "2016-12-03 14:14:11+00:00",
            "title": "Unit Dependency Graph and its Application to Arithmetic Word Problem Solving",
            "authors": [
                "Subhro Roy",
                "Dan Roth"
            ],
            "abstract": "Math word problems provide a natural abstraction to a range of natural\nlanguage understanding problems that involve reasoning about quantities, such\nas interpreting election results, news about casualties, and the financial\nsection of a newspaper. Units associated with the quantities often provide\ninformation that is essential to support this reasoning. This paper proposes a\nprincipled way to capture and reason about units and shows how it can benefit\nan arithmetic word problem solver. This paper presents the concept of Unit\nDependency Graphs (UDGs), which provides a compact representation of the\ndependencies between units of numbers mentioned in a given problem. Inducing\nthe UDG alleviates the brittleness of the unit extraction system and allows for\na natural way to leverage domain knowledge about unit compatibility, for word\nproblem solving. We introduce a decomposed model for inducing UDGs with minimal\nadditional annotations, and use it to augment the expressions used in the\narithmetic word problem solver of (Roy and Roth 2015) via a constrained\ninference framework. We show that introduction of UDGs reduces the error of the\nsolver by over 10 %, surpassing all existing systems for solving arithmetic\nword problems. In addition, it also makes the system more robust to adaptation\nto new vocabulary and equation forms .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01039v1",
            "date": "2016-12-04 00:22:44+00:00",
            "title": "CER: Complementary Entity Recognition via Knowledge Expansion on Large Unlabeled Product Reviews",
            "authors": [
                "Hu Xu",
                "Sihong Xie",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Product reviews contain a lot of useful information about product features\nand customer opinions. One important product feature is the complementary\nentity (products) that may potentially work together with the reviewed product.\nKnowing complementary entities of the reviewed product is very important\nbecause customers want to buy compatible products and avoid incompatible ones.\nIn this paper, we address the problem of Complementary Entity Recognition\n(CER). Since no existing method can solve this problem, we first propose a\nnovel unsupervised method to utilize syntactic dependency paths to recognize\ncomplementary entities. Then we expand category-level domain knowledge about\ncomplementary entities using only a few general seed verbs on a large amount of\nunlabeled reviews. The domain knowledge helps the unsupervised method to adapt\nto different products and greatly improves the precision of the CER task. The\nadvantage of the proposed method is that it does not require any labeled data\nfor training. We conducted experiments on 7 popular products with about 1200\nreviews in total to demonstrate that the proposed approach is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01404v1",
            "date": "2016-12-05 15:48:06+00:00",
            "title": "Mapping the Dialog Act Annotations of the LEGO Corpus into the Communicative Functions of ISO 24617-2",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "In this paper we present strategies for mapping the dialog act annotations of\nthe LEGO corpus into the communicative functions of the ISO 24617-2 standard.\nUsing these strategies, we obtained an additional 347 dialogs annotated\naccording to the standard. This is particularly important given the reduced\namount of existing data in those conditions due to the recency of the standard.\nFurthermore, these are dialogs from a widely explored corpus for dialog related\ntasks. However, its dialog annotations have been neglected due to their high\ndomain-dependency, which renders them unuseful outside the context of the\ncorpus. Thus, through our mapping process, we both obtain more data annotated\naccording to a recent standard and provide useful dialog act annotations for a\nwidely explored corpus in the context of dialog research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01627v2",
            "date": "2016-12-06 01:57:39+00:00",
            "title": "Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Chen Xing",
                "Ming Zhou",
                "Zhoujun Li"
            ],
            "abstract": "We study response selection for multi-turn conversation in retrieval-based\nchatbots. Existing work either concatenates utterances in context or matches a\nresponse with a highly abstract context vector finally, which may lose\nrelationships among utterances or important contextual information. We propose\na sequential matching network (SMN) to address both problems. SMN first matches\na response with each utterance in the context on multiple levels of\ngranularity, and distills important matching information from each pair as a\nvector with convolution and pooling operations. The vectors are then\naccumulated in a chronological order through a recurrent neural network (RNN)\nwhich models relationships among utterances. The final matching score is\ncalculated with the hidden states of the RNN. An empirical study on two public\ndata sets shows that SMN can significantly outperform state-of-the-art methods\nfor response selection in multi-turn conversation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01744v1",
            "date": "2016-12-06 10:48:56+00:00",
            "title": "Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation",
            "authors": [
                "Alexandre Berard",
                "Olivier Pietquin",
                "Christophe Servan",
                "Laurent Besacier"
            ],
            "abstract": "This paper proposes a first attempt to build an end-to-end speech-to-text\ntranslation system, which does not use source language transcription during\nlearning or decoding. We propose a model for direct speech-to-text translation,\nwhich gives promising results on a small French-English synthetic corpus.\nRelaxing the need for source language transcription would drastically change\nthe data collection methodology in speech translation, especially in\nunder-resourced scenarios. For instance, in the former project DARPA TRANSTAC\n(speech translation from spoken Arabic dialects), a large effort was devoted to\nthe collection of speech transcripts (and a prerequisite to obtain transcripts\nwas often a detailed transcription guide for languages with little standardized\nspelling). Now, if end-to-end approaches for speech-to-text translation are\nsuccessful, one might consider collecting data by asking bilingual speakers to\ndirectly utter speech in the source language from target language text\nutterances. Such an approach has the advantage to be applicable to any\nunwritten (source) language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01848v2",
            "date": "2016-12-06 15:15:27+00:00",
            "title": "Condensed Memory Networks for Clinical Diagnostic Inferencing",
            "authors": [
                "Aaditya Prakash",
                "Siyuan Zhao",
                "Sadid A. Hasan",
                "Vivek Datla",
                "Kathy Lee",
                "Ashequl Qadir",
                "Joey Liu",
                "Oladimeji Farri"
            ],
            "abstract": "Diagnosis of a clinical condition is a challenging task, which often requires\nsignificant medical investigation. Previous work related to diagnostic\ninferencing problems mostly consider multivariate observational data (e.g.\nphysiological signals, lab tests etc.). In contrast, we explore the problem\nusing free-text medical notes recorded in an electronic health record (EHR).\nComplex tasks like these can benefit from structured knowledge bases, but those\nare not scalable. We instead exploit raw text from Wikipedia as a knowledge\nsource. Memory networks have been demonstrated to be effective in tasks which\nrequire comprehension of free-form text. They use the final iteration of the\nlearned representation to predict probable classes. We introduce condensed\nmemory neural networks (C-MemNNs), a novel model with iterative condensation of\nmemory representations that preserves the hierarchy of features in the memory.\nExperiments on the MIMIC-III dataset show that the proposed model outperforms\nother variants of memory networks to predict the most probable diagnoses given\na complex clinical scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02251v2",
            "date": "2016-12-07 14:03:15+00:00",
            "title": "When is multitask learning effective? Semantic sequence prediction under varying data conditions",
            "authors": [
                "H\u00e9ctor Mart\u00ednez Alonso",
                "Barbara Plank"
            ],
            "abstract": "Multitask learning has been applied successfully to a range of tasks, mostly\nmorphosyntactic. However, little is known on when MTL works and whether there\nare data characteristics that help to determine its success. In this paper we\nevaluate a range of semantic sequence labeling tasks in a MTL setup. We examine\ndifferent auxiliary tasks, amongst which a novel setup, and correlate their\nimpact to data-dependent conditions. Our results show that MTL is not always\neffective, significant improvements are obtained only for 1 out of 5 tasks.\nWhen successful, auxiliary tasks with compact and more uniform label\ndistributions are preferable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02703v2",
            "date": "2016-12-08 15:54:00+00:00",
            "title": "Embedding Words and Senses Together via Joint Knowledge-Enhanced Training",
            "authors": [
                "Massimiliano Mancini",
                "Jose Camacho-Collados",
                "Ignacio Iacobacci",
                "Roberto Navigli"
            ],
            "abstract": "Word embeddings are widely used in Natural Language Processing, mainly due to\ntheir success in capturing semantic information from massive corpora. However,\ntheir creation process does not allow the different meanings of a word to be\nautomatically separated, as it conflates them into a single vector. We address\nthis issue by proposing a new model which learns word and sense embeddings\njointly. Our model exploits large corpora and knowledge from semantic networks\nin order to produce a unified vector space of word and sense embeddings. We\nevaluate the main features of our approach both qualitatively and\nquantitatively in a variety of tasks, highlighting the advantages of the\nproposed method in comparison to state-of-the-art word- and sense-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02706v2",
            "date": "2016-12-08 16:05:03+00:00",
            "title": "Entity Identification as Multitasking",
            "authors": [
                "Karl Stratos"
            ],
            "abstract": "Standard approaches in entity identification hard-code boundary detection and\ntype prediction into labels (e.g., John/B-PER Smith/I-PER) and then perform\nViterbi. This has two disadvantages: 1. the runtime complexity grows\nquadratically in the number of types, and 2. there is no natural segment-level\nrepresentation. In this paper, we propose a novel neural architecture that\naddresses these disadvantages. We frame the problem as multitasking, separating\nboundary detection and type prediction but optimizing them jointly. Despite its\nsimplicity, this architecture performs competitively with fully structured\nmodels such as BiLSTM-CRFs while scaling linearly in the number of types.\nFurthermore, by construction, the model induces type-disambiguating embeddings\nof predicted mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02801v1",
            "date": "2016-12-08 20:33:17+00:00",
            "title": "Discovering Conversational Dependencies between Messages in Dialogs",
            "authors": [
                "Wenchao Du",
                "Pascal Poupart",
                "Wei Xu"
            ],
            "abstract": "We investigate the task of inferring conversational dependencies between\nmessages in one-on-one online chat, which has become one of the most popular\nforms of customer service. We propose a novel probabilistic classifier that\nleverages conversational, lexical and semantic information. The approach is\nevaluated empirically on a set of customer service chat logs from a Chinese\ne-commerce website. It outperforms heuristic baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03205v1",
            "date": "2016-12-09 22:36:33+00:00",
            "title": "Evaluating Creative Language Generation: The Case of Rap Lyric Ghostwriting",
            "authors": [
                "Peter Potash",
                "Alexey Romanov",
                "Anna Rumshisky"
            ],
            "abstract": "Language generation tasks that seek to mimic human ability to use language\ncreatively are difficult to evaluate, since one must consider creativity,\nstyle, and other non-trivial aspects of the generated text. The goal of this\npaper is to develop evaluation methods for one such task, ghostwriting of rap\nlyrics, and to provide an explicit, quantifiable foundation for the goals and\nfuture directions of this task. Ghostwriting must produce text that is similar\nin style to the emulated artist, yet distinct in content. We develop a novel\nevaluation methodology that addresses several complementary aspects of this\ntask, and illustrate how such evaluation can be used to meaningfully analyze\nsystem performance. We provide a corpus of lyrics for 13 rap artists, annotated\nfor stylistic similarity, which allows us to assess the feasibility of manual\nevaluation for generated verse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03216v2",
            "date": "2016-12-09 23:28:16+00:00",
            "title": "#HashtagWars: Learning a Sense of Humor",
            "authors": [
                "Peter Potash",
                "Alexey Romanov",
                "Anna Rumshisky"
            ],
            "abstract": "In this work, we present a new dataset for computational humor, specifically\ncomparative humor ranking, which attempts to eschew the ubiquitous binary\napproach to humor detection. The dataset consists of tweets that are humorous\nresponses to a given hashtag. We describe the motivation for this new dataset,\nas well as the collection process, which includes a description of our\nsemi-automated system for data collection. We also present initial experiments\nfor this dataset using both unsupervised and supervised approaches. Our best\nsupervised system achieved 63.7% accuracy, suggesting that this task is much\nmore difficult than comparable humor detection tasks. Initial experiments\nindicate that a character-level model is more suitable for this task than a\ntoken-level model, likely due to a large amount of puns that can be captured by\na character-level model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03266v1",
            "date": "2016-12-10 08:05:38+00:00",
            "title": "A Character-Word Compositional Neural Language Model for Finnish",
            "authors": [
                "Matti Lankinen",
                "Hannes Heikinheimo",
                "Pyry Takala",
                "Tapani Raiko",
                "Juha Karhunen"
            ],
            "abstract": "Inspired by recent research, we explore ways to model the highly\nmorphological Finnish language at the level of characters while maintaining the\nperformance of word-level models. We propose a new\nCharacter-to-Word-to-Character (C2W2C) compositional language model that uses\ncharacters as input and output while still internally processing word level\nembeddings. Our preliminary experiments, using the Finnish Europarl V7 corpus,\nindicate that C2W2C can respond well to the challenges of morphologically rich\nlanguages such as high out of vocabulary rates, the prediction of novel words,\nand growing vocabulary size. Notably, the model is able to correctly score\ninflectional forms that are not present in the training data and sample\ngrammatically and semantically correct Finnish sentences character by\ncharacter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03659v1",
            "date": "2016-12-12 13:08:55+00:00",
            "title": "Unraveling reported dreams with text analytics",
            "authors": [
                "Iris Hendrickx",
                "Louis Onrust",
                "Florian Kunneman",
                "Ali H\u00fcrriyeto\u011flu",
                "Antal van den Bosch",
                "Wessel Stoop"
            ],
            "abstract": "We investigate what distinguishes reported dreams from other personal\nnarratives. The continuity hypothesis, stemming from psychological dream\nanalysis work, states that most dreams refer to a person's daily life and\npersonal concerns, similar to other personal narratives such as diary entries.\nDifferences between the two texts may reveal the linguistic markers of dream\ntext, which could be the basis for new dream analysis work and for the\nautomatic detection of dream descriptions. We used three text analytics\nmethods: text classification, topic modeling, and text coherence analysis, and\napplied these methods to a balanced set of texts representing dreams, diary\nentries, and other personal stories. We observed that dream texts could be\ndistinguished from other personal narratives nearly perfectly, mostly based on\nthe presence of uncertainty markers and descriptions of scenes. Important\nmarkers for non-dream narratives are specific time expressions and\nconversational expressions. Dream texts also exhibit a lower discourse\ncoherence than other personal narratives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03762v1",
            "date": "2016-12-12 16:14:02+00:00",
            "title": "From narrative descriptions to MedDRA: automagically encoding adverse drug reactions",
            "authors": [
                "Carlo Combi",
                "Margherita Zorzi",
                "Gabriele Pozzani",
                "Ugo Moretti"
            ],
            "abstract": "The collection of narrative spontaneous reports is an irreplaceable source\nfor the prompt detection of suspected adverse drug reactions (ADRs): qualified\ndomain experts manually revise a huge amount of narrative descriptions and then\nencode texts according to MedDRA standard terminology. The manual annotation of\nnarrative documents with medical terminology is a subtle and expensive task,\nsince the number of reports is growing up day-by-day. MagiCoder, a Natural\nLanguage Processing algorithm, is proposed for the automatic encoding of\nfree-text descriptions into MedDRA terms. MagiCoder procedure is efficient in\nterms of computational complexity (in particular, it is linear in the size of\nthe narrative input and the terminology). We tested it on a large dataset of\nabout 4500 manually revised reports, by performing an automated comparison\nbetween human and MagiCoder revisions. For the current base version of\nMagiCoder, we measured: on short descriptions, an average recall of $86\\%$ and\nan average precision of $88\\%$; on medium-long descriptions (up to 255\ncharacters), an average recall of $64\\%$ and an average precision of $63\\%$.\nFrom a practical point of view, MagiCoder reduces the time required for\nencoding ADR reports. Pharmacologists have simply to review and validate the\nMagiCoder terms proposed by the application, instead of choosing the right\nterms among the 70K low level terms of MedDRA. Such improvement in the\nefficiency of pharmacologists' work has a relevant impact also on the quality\nof the subsequent data analysis. We developed MagiCoder for the Italian\npharmacovigilance language. However, our proposal is based on a general\napproach, not depending on the considered language nor the term dictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03791v2",
            "date": "2016-12-12 17:12:44+00:00",
            "title": "Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices",
            "authors": [
                "Felix Stahlberg",
                "Adri\u00e0 de Gispert",
                "Eva Hasler",
                "Bill Byrne"
            ],
            "abstract": "We present a novel scheme to combine neural machine translation (NMT) with\ntraditional statistical machine translation (SMT). Our approach borrows ideas\nfrom linearised lattice minimum Bayes-risk decoding for SMT. The NMT score is\ncombined with the Bayes-risk of the translation according the SMT lattice. This\nmakes our approach much more flexible than $n$-best list or lattice rescoring\nas the neural decoder is not restricted to the SMT search space. We show an\nefficient and simple way to integrate risk estimation into the NMT decoder\nwhich is suitable for word-level as well as subword-unit-level NMT. We test our\nmethod on English-German and Japanese-English and report significant gains over\nlattice rescoring on several data sets for both single and ensembled NMT. The\nMBR decoder produces entirely new hypotheses far beyond simply rescoring the\nSMT search space or fixing UNKs in the NMT output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03969v3",
            "date": "2016-12-12 23:29:40+00:00",
            "title": "Tracking the World State with Recurrent Entity Networks",
            "authors": [
                "Mikael Henaff",
                "Jason Weston",
                "Arthur Szlam",
                "Antoine Bordes",
                "Yann LeCun"
            ],
            "abstract": "We introduce a new model, the Recurrent Entity Network (EntNet). It is\nequipped with a dynamic long-term memory which allows it to maintain and update\na representation of the state of the world as it receives new data. For\nlanguage understanding tasks, it can reason on-the-fly as it reads text, not\njust when it is required to answer a question or respond as is the case for a\nMemory Network (Sukhbaatar et al., 2015). Like a Neural Turing Machine or\nDifferentiable Neural Computer (Graves et al., 2014; 2016) it maintains a fixed\nsize memory and can learn to perform location and content-based read and write\noperations. However, unlike those models it has a simple parallel architecture\nin which several memory locations can be updated simultaneously. The EntNet\nsets a new state-of-the-art on the bAbI tasks, and is the first method to solve\nall the tasks in the 10k training examples setting. We also demonstrate that it\ncan solve a reasoning task which requires a large number of supporting facts,\nwhich other methods are not able to solve, and can generalize past its training\nhorizon. It can also be practically used on large scale datasets such as\nChildren's Book Test, where it obtains competitive performance, reading the\nstory in a single pass.",
            "journal ref": "ICLR 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03975v2",
            "date": "2016-12-12 23:54:52+00:00",
            "title": "ConceptNet 5.5: An Open Multilingual Graph of General Knowledge",
            "authors": [
                "Robyn Speer",
                "Joshua Chin",
                "Catherine Havasi"
            ],
            "abstract": "Machine learning about language can be improved by supplying it with specific\nknowledge and sources of external information. We present here a new version of\nthe linked open data resource ConceptNet that is particularly well suited to be\nused with modern NLP techniques such as word embeddings.\n  ConceptNet is a knowledge graph that connects words and phrases of natural\nlanguage with labeled edges. Its knowledge is collected from many sources that\ninclude expert-created resources, crowd-sourcing, and games with a purpose. It\nis designed to represent the general knowledge involved in understanding\nlanguage, improving natural language applications by allowing the application\nto better understand the meanings behind the words people use.\n  When ConceptNet is combined with word embeddings acquired from distributional\nsemantics (such as word2vec), it provides applications with understanding that\nthey would not acquire from distributional semantics alone, nor from narrower\nresources such as WordNet or DBPedia. We demonstrate this with state-of-the-art\nresults on intrinsic evaluations of word relatedness that translate into\nimprovements on applications of word vectors, including solving SAT-style\nanalogies.",
            "journal ref": "AAAI 31 (2017) 4444-4451",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03990v1",
            "date": "2016-12-13 01:18:39+00:00",
            "title": "Evaluating Automatic Speech Recognition Systems in Comparison With Human Perception Results Using Distinctive Feature Measures",
            "authors": [
                "Xiang Kong",
                "Jeung-Yoon Choi",
                "Stefanie Shattuck-Hufnagel"
            ],
            "abstract": "This paper describes methods for evaluating automatic speech recognition\n(ASR) systems in comparison with human perception results, using measures\nderived from linguistic distinctive features. Error patterns in terms of\nmanner, place and voicing are presented, along with an examination of confusion\nmatrices via a distinctive-feature-distance metric. These evaluation methods\ncontrast with conventional performance criteria that focus on the phone or word\nlevel, and are intended to provide a more detailed profile of ASR system\nperformance,as well as a means for direct comparison with human perception\nresults at the sub-phonemic level.",
            "journal ref": "ICASSP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03991v1",
            "date": "2016-12-13 01:25:14+00:00",
            "title": "Performance Improvements of Probabilistic Transcript-adapted ASR with Recurrent Neural Network and Language-specific Constraints",
            "authors": [
                "Xiang Kong",
                "Preethi Jyothi",
                "Mark Hasegawa-Johnson"
            ],
            "abstract": "Mismatched transcriptions have been proposed as a mean to acquire\nprobabilistic transcriptions from non-native speakers of a language.Prior work\nhas demonstrated the value of these transcriptions by successfully adapting\ncross-lingual ASR systems for different tar-get languages. In this work, we\ndescribe two techniques to refine these probabilistic transcriptions: a\nnoisy-channel model of non-native phone misperception is trained using a\nrecurrent neural net-work, and decoded using minimally-resourced\nlanguage-dependent pronunciation constraints. Both innovations improve quality\nof the transcript, and both innovations reduce phone error rate of a\ntrainedASR, by 7% and 9% respectively",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04113v1",
            "date": "2016-12-13 12:03:39+00:00",
            "title": "Vicinity-Driven Paragraph and Sentence Alignment for Comparable Corpora",
            "authors": [
                "Gustavo Henrique Paetzold",
                "Lucia Specia"
            ],
            "abstract": "Parallel corpora have driven great progress in the field of Text\nSimplification. However, most sentence alignment algorithms either offer a\nlimited range of alignment types supported, or simply ignore valuable clues\npresent in comparable documents. We address this problem by introducing a new\nset of flexible vicinity-driven paragraph and sentence alignment algorithms\nthat 1-N, N-1, N-N and long distance null alignments without the need for\nhard-to-replicate supervised models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04211v1",
            "date": "2016-12-13 14:49:47+00:00",
            "title": "Multi-Perspective Context Matching for Machine Comprehension",
            "authors": [
                "Zhiguo Wang",
                "Haitao Mi",
                "Wael Hamza",
                "Radu Florian"
            ],
            "abstract": "Previous machine comprehension (MC) datasets are either too small to train\nend-to-end deep learning models, or not difficult enough to evaluate the\nability of current MC techniques. The newly released SQuAD dataset alleviates\nthese limitations, and gives us a chance to develop more realistic MC models.\nBased on this dataset, we propose a Multi-Perspective Context Matching (MPCM)\nmodel, which is an end-to-end system that directly predicts the answer\nbeginning and ending points in a passage. Our model first adjusts each\nword-embedding vector in the passage by multiplying a relevancy weight computed\nagainst the question. Then, we encode the question and weighted passage by\nusing bi-directional LSTMs. For each point in the passage, our model matches\nthe context of this point against the encoded question from multiple\nperspectives and produces a matching vector. Given those matched vectors, we\nemploy another bi-directional LSTM to aggregate all the information and predict\nthe beginning and ending points. Experimental result on the test set of SQuAD\nshows that our model achieves a competitive result on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04342v1",
            "date": "2016-12-13 20:22:36+00:00",
            "title": "Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors",
            "authors": [
                "Radu Soricut",
                "Nan Ding"
            ],
            "abstract": "We present a dual contribution to the task of machine reading-comprehension:\na technique for creating large-sized machine-comprehension (MC) datasets using\nparagraph-vector models; and a novel, hybrid neural-network architecture that\ncombines the representation power of recurrent neural networks with the\ndiscriminative power of fully-connected multi-layered networks. We use the\nMC-dataset generation technique to build a dataset of around 2 million\nexamples, for which we empirically determine the high-ceiling of human\nperformance (around 91% accuracy), as well as the performance of a variety of\ncomputer models. Among all the models we have experimented with, our hybrid\nneural-network architecture achieves the highest performance (83.2% accuracy).\nThe remaining gap to the human-performance ceiling provides enough room for\nfuture model improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04460v2",
            "date": "2016-12-14 02:28:29+00:00",
            "title": "Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection",
            "authors": [
                "Vered Shwartz",
                "Enrico Santus",
                "Dominik Schlechtweg"
            ],
            "abstract": "The fundamental role of hypernymy in NLP has motivated the development of\nmany methods for the automatic identification of this relation, most of which\nrely on word distribution. We investigate an extensive number of such\nunsupervised measures, using several distributional semantic models that differ\nby context type and feature weighting. We analyze the performance of the\ndifferent methods based on their linguistic motivation. Comparison to the\nstate-of-the-art supervised methods shows that while supervised methods\ngenerally outperform the unsupervised ones, the former are sensitive to the\ndistribution of training instances, hurting their reliability. Being based on\ngeneral linguistic hypotheses and independent from training data, unsupervised\nmeasures are more robust, and therefore are still useful artillery for\nhypernymy detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04499v1",
            "date": "2016-12-14 06:05:39+00:00",
            "title": "Mining Compatible/Incompatible Entities from Question and Answering via Yes/No Answer Classification using Distant Label Expansion",
            "authors": [
                "Hu Xu",
                "Lei Shu",
                "Jingyuan Zhang",
                "Philip S. Yu"
            ],
            "abstract": "Product Community Question Answering (PCQA) provides useful information about\nproducts and their features (aspects) that may not be well addressed by product\ndescriptions and reviews. We observe that a product's compatibility issues with\nother products are frequently discussed in PCQA and such issues are more\nfrequently addressed in accessories, i.e., via a yes/no question \"Does this\nmouse work with windows 10?\". In this paper, we address the problem of\nextracting compatible and incompatible products from yes/no questions in PCQA.\nThis problem can naturally have a two-stage framework: first, we perform\nComplementary Entity (product) Recognition (CER) on yes/no questions; second,\nwe identify the polarities of yes/no answers to assign the complementary\nentities a compatibility label (compatible, incompatible or unknown). We\nleverage an existing unsupervised method for the first stage and a 3-class\nclassifier by combining a distant PU-learning method (learning from positive\nand unlabeled examples) together with a binary classifier for the second stage.\nThe benefit of using distant PU-learning is that it can help to expand more\nimplicit yes/no answers without using any human annotated data. We conduct\nexperiments on 4 products to show that the proposed method is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04538v1",
            "date": "2016-12-14 09:00:21+00:00",
            "title": "Grammatical Constraints on Intra-sentential Code-Switching: From Theories to Working Models",
            "authors": [
                "Gayatri Bhat",
                "Monojit Choudhury",
                "Kalika Bali"
            ],
            "abstract": "We make one of the first attempts to build working models for\nintra-sentential code-switching based on the Equivalence-Constraint (Poplack\n1980) and Matrix-Language (Myers-Scotton 1993) theories. We conduct a detailed\ntheoretical analysis, and a small-scale empirical study of the two models for\nHindi-English CS. Our analyses show that the models are neither sound nor\ncomplete. Taking insights from the errors made by the models, we propose a new\nmodel that combines features of both the theories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04609v1",
            "date": "2016-12-14 12:46:18+00:00",
            "title": "Neural Emoji Recommendation in Dialogue Systems",
            "authors": [
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Rui Yan",
                "Maosong Sun"
            ],
            "abstract": "Emoji is an essential component in dialogues which has been broadly utilized\non almost all social platforms. It could express more delicate feelings beyond\nplain texts and thus smooth the communications between users, making dialogue\nsystems more anthropomorphic and vivid. In this paper, we focus on\nautomatically recommending appropriate emojis given the contextual information\nin multi-turn dialogue systems, where the challenges locate in understanding\nthe whole conversations. More specifically, we propose the hierarchical long\nshort-term memory model (H-LSTM) to construct dialogue representations,\nfollowed by a softmax classifier for emoji classification. We evaluate our\nmodels on the task of emoji classification in a real-world dataset, with some\nfurther explorations on parameter sensitivity and case study. Experimental\nresults demonstrate that our method achieves the best performances on all\nevaluation metrics. It indicates that our method could well capture the\ncontextual information and emotion flow in dialogues, which is significant for\nemoji recommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04629v3",
            "date": "2016-12-14 13:45:35+00:00",
            "title": "How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs",
            "authors": [
                "Rico Sennrich"
            ],
            "abstract": "Analysing translation quality in regards to specific linguistic phenomena has\nhistorically been difficult and time-consuming. Neural machine translation has\nthe attractive property that it can produce scores for arbitrary translations,\nand we propose a novel method to assess how well NMT systems model specific\nlinguistic phenomena such as agreement over long distances, the production of\nnovel words, and the faithful translation of polarity. The core idea is that we\nmeasure whether a reference translation is more probable under a NMT model than\na contrastive translation which introduces a specific type of error. We present\nLingEval97, a large-scale data set of 97000 contrastive translation pairs based\non the WMT English->German translation task, with errors automatically created\nwith simple rules. We report results for a number of systems, and find that\nrecently introduced character-level NMT systems perform better at\ntransliteration than models with byte-pair encoding (BPE) segmentation, but\nperform more poorly at morphosyntactic agreement, and translating discontiguous\nunits of meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04683v1",
            "date": "2016-12-14 15:21:48+00:00",
            "title": "Unsupervised Clustering of Commercial Domains for Adaptive Machine Translation",
            "authors": [
                "Mauro Cettolo",
                "Mara Chinea Rios",
                "Roldano Cattoni"
            ],
            "abstract": "In this paper, we report on domain clustering in the ambit of an adaptive MT\narchitecture. A standard bottom-up hierarchical clustering algorithm has been\ninstantiated with five different distances, which have been compared, on an MT\nbenchmark built on 40 commercial domains, in terms of dendrograms, intrinsic\nand extrinsic evaluations. The main outcome is that the most expensive distance\nis also the only one able to allow the MT engine to guarantee good performance\neven with few, but highly populated clusters of domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04732v1",
            "date": "2016-12-14 17:13:01+00:00",
            "title": "Multilingual Word Embeddings using Multigraphs",
            "authors": [
                "Radu Soricut",
                "Nan Ding"
            ],
            "abstract": "We present a family of neural-network--inspired models for computing\ncontinuous word representations, specifically designed to exploit both\nmonolingual and multilingual text. This framework allows us to perform\nunsupervised training of embeddings that exhibit higher accuracy on syntactic\nand semantic compositionality, as well as multilingual semantic similarity,\ncompared to previous models trained in an unsupervised fashion. We also show\nthat such multilingual embeddings, optimized for semantic similarity, can\nimprove the performance of statistical machine translation with respect to how\nit handles words not present in the parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04765v10",
            "date": "2016-12-14 18:37:30+00:00",
            "title": "CoPaSul Manual -- Contour-based parametric and superpositional intonation stylization",
            "authors": [
                "Uwe D. Reichel"
            ],
            "abstract": "The purposes of the CoPaSul toolkit are (1) automatic prosodic annotation and\n(2) prosodic feature extraction from syllable to utterance level. CoPaSul\nstands for contour-based, parametric, superpositional intonation stylization.\nIn this framework intonation is represented as a superposition of global and\nlocal contours that are described parametrically in terms of polynomial\ncoefficients. On the global level (usually associated but not necessarily\nrestricted to intonation phrases) the stylization serves to represent register\nin terms of time-varying F0 level and range. On the local level (e.g. accent\ngroups), local contour shapes are described. From this parameterization several\nfeatures related to prosodic boundaries and prominence can be derived.\nFurthermore, by coefficient clustering prosodic contour classes can be obtained\nin a bottom-up way. Next to the stylization-based feature extraction also\nstandard F0 and energy measures (e.g. mean and variance) as well as rhythmic\naspects can be calculated. At the current state automatic annotation comprises:\nsegmentation into interpausal chunks, syllable nucleus extraction, and\nunsupervised localization of prosodic phrase boundaries and prominent\nsyllables. F0 and partly also energy feature sets can be derived for: standard\nmeasurements (as median and IQR), register in terms of F0 level and range,\nprosodic boundaries, local contour shapes, bottom-up derived contour classes,\nGestalt of accent groups in terms of their deviation from higher level prosodic\nunits, as well as for rhythmic aspects quantifying the relation between F0 and\nenergy contours and prosodic event rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05131v1",
            "date": "2016-12-15 16:30:11+00:00",
            "title": "Transition-based Parsing with Context Enhancement and Future Reward Reranking",
            "authors": [
                "Fugen Zhou",
                "Fuxiang Wu",
                "Zhengchen Zhang",
                "Minghui Dong"
            ],
            "abstract": "This paper presents a novel reranking model, future reward reranking, to\nre-score the actions in a transition-based parser by using a global scorer.\nDifferent to conventional reranking parsing, the model searches for the best\ndependency tree in all feasible trees constraining by a sequence of actions to\nget the future reward of the sequence. The scorer is based on a first-order\ngraph-based parser with bidirectional LSTM, which catches different parsing\nview compared with the transition-based parser. Besides, since context\nenhancement has shown substantial improvement in the arc-stand transition-based\nparsing over the parsing accuracy, we implement context enhancement on an\narc-eager transition-base parser with stack LSTMs, the dynamic oracle and\ndropout supporting and achieve further improvement. With the global scorer and\ncontext enhancement, the results show that UAS of the parser increases as much\nas 1.20% for English and 1.66% for Chinese, and LAS increases as much as 1.32%\nfor English and 1.63% for Chinese. Moreover, we get state-of-the-art LASs,\nachieving 87.58% for Chinese and 93.37% for English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05202v1",
            "date": "2016-12-15 19:28:17+00:00",
            "title": "Building a robust sentiment lexicon with (almost) no resource",
            "authors": [
                "Mickael Rouvier",
                "Benoit Favre"
            ],
            "abstract": "Creating sentiment polarity lexicons is labor intensive. Automatically\ntranslating them from resourceful languages requires in-domain machine\ntranslation systems, which rely on large quantities of bi-texts. In this paper,\nwe propose to replace machine translation by transferring words from the\nlexicon through word embeddings aligned across languages with a simple linear\ntransform. The approach leads to no degradation, compared to machine\ntranslation, when tested on sentiment polarity classification on tweets from\nfour languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05310v2",
            "date": "2016-12-15 23:41:13+00:00",
            "title": "Modeling Trolling in Social Media Conversations",
            "authors": [
                "Luis Gerardo Mojica"
            ],
            "abstract": "Social media websites, electronic newspapers and Internet forums allow\nvisitors to leave comments for others to read and interact. This exchange is\nnot free from participants with malicious intentions, who troll others by\npositing messages that are intended to be provocative, offensive, or menacing.\nWith the goal of facilitating the computational modeling of trolling, we\npropose a trolling categorization that is novel in the sense that it allows\ncomment-based analysis from both the trolls' and the responders' perspectives,\ncharacterizing these two perspectives using four aspects, namely, the troll's\nintention and his intention disclosure, as well as the responder's\ninterpretation of the troll's intention and her response strategy. Using this\ncategorization, we annotate and release a dataset containing excerpts of Reddit\nconversations involving suspected trolls and their interactions with other\nusers. Finally, we identify the difficult-to-classify cases in our corpus and\nsuggest potential solutions for them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05340v2",
            "date": "2016-12-16 02:49:53+00:00",
            "title": "Automatic Labelling of Topics with Neural Embeddings",
            "authors": [
                "Shraey Bhatia",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "Topics generated by topic models are typically represented as list of terms.\nTo reduce the cognitive overhead of interpreting these topics for end-users, we\npropose labelling a topic with a succinct phrase that summarises its theme or\nidea. Using Wikipedia document titles as label candidates, we compute neural\nembeddings for documents and words to select the most relevant labels for\ntopics. Compared to a state-of-the-art topic labelling system, our methodology\nis simpler, more efficient, and finds better topic labels.",
            "journal ref": "Proceedings of the 26th International Conference on Computational\n  Linguistics (COLING 2016), pp 953--963",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05420v1",
            "date": "2016-12-16 10:39:53+00:00",
            "title": "A Two-Phase Approach Towards Identifying Argument Structure in Natural Language",
            "authors": [
                "Arkanath Pathak",
                "Pawan Goyal",
                "Plaban Bhowmick"
            ],
            "abstract": "We propose a new approach for extracting argument structure from natural\nlanguage texts that contain an underlying argument. Our approach comprises of\ntwo phases: Score Assignment and Structure Prediction. The Score Assignment\nphase trains models to classify relations between argument units (Support,\nAttack or Neutral). To that end, different training strategies have been\nexplored. We identify different linguistic and lexical features for training\nthe classifiers. Through ablation study, we observe that our novel use of\nword-embedding features is most effective for this task. The Structure\nPrediction phase makes use of the scores from the Score Assignment phase to\narrive at the optimal structure. We perform experiments on three argumentation\ndatasets, namely, AraucariaDB, Debatepedia and Wikipedia. We also propose two\nbaselines and observe that the proposed approach outperforms baseline systems\nfor the final task of Structure Prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05555v2",
            "date": "2016-12-16 17:00:37+00:00",
            "title": "Neural Networks Classifier for Data Selection in Statistical Machine Translation",
            "authors": [
                "\u00c1lvaro Peris",
                "Mara Chinea-Rios",
                "Francisco Casacuberta"
            ],
            "abstract": "We address the data selection problem in statistical machine translation\n(SMT) as a classification task. The new data selection method is based on a\nneural network classifier. We present a new method description and empirical\nresults proving that our data selection method provides better translation\nquality, compared to a state-of-the-art method (i.e., Cross entropy). Moreover,\nthe empirical results reported are coherent across different language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06027v3",
            "date": "2016-12-19 02:21:24+00:00",
            "title": "Neural Multi-Source Morphological Reinflection",
            "authors": [
                "Katharina Kann",
                "Ryan Cotterell",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We explore the task of multi-source morphological reinflection, which\ngeneralizes the standard, single-source version. The input consists of (i) a\ntarget tag and (ii) multiple pairs of source form and source tag for a lemma.\nThe motivation is that it is beneficial to have access to more than one source\nform since different source forms can provide complementary information, e.g.,\ndifferent stems. We further present a novel extension to the encoder- decoder\nrecurrent neural architecture, consisting of multiple encoders, to better solve\nthe task. We show that our new architecture outperforms single-source\nreinflection models and publish our dataset for multi-source morphological\nreinflection to facilitate future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06138v2",
            "date": "2016-12-19 11:49:49+00:00",
            "title": "Boosting Neural Machine Translation",
            "authors": [
                "Dakun Zhang",
                "Jungi Kim",
                "Josep Crego",
                "Jean Senellart"
            ],
            "abstract": "Training efficiency is one of the main problems for Neural Machine\nTranslation (NMT). Deep networks need for very large data as well as many\ntraining iterations to achieve state-of-the-art performance. This results in\nvery high computation cost, slowing down research and industrialisation. In\nthis paper, we propose to alleviate this problem with several training methods\nbased on data boosting and bootstrap with no modifications to the neural\nnetwork. It imitates the learning process of humans, which typically spend more\ntime when learning \"difficult\" concepts than easier ones. We experiment on an\nEnglish-French translation task showing accuracy improvements of up to 1.63\nBLEU while saving 20% of training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06139v1",
            "date": "2016-12-19 11:50:58+00:00",
            "title": "Neural Machine Translation from Simplified Translations",
            "authors": [
                "Josep Crego",
                "Jean Senellart"
            ],
            "abstract": "Text simplification aims at reducing the lexical, grammatical and structural\ncomplexity of a text while keeping the same meaning. In the context of machine\ntranslation, we introduce the idea of simplified translations in order to boost\nthe learning ability of deep neural translation models. We conduct preliminary\nexperiments showing that translation complexity is actually reduced in a\ntranslation of a source bi-text compared to the target reference of the bi-text\nwhile using a neural machine translation (NMT) system learned on the exact same\nbi-text. Based on knowledge distillation idea, we then train an NMT system\nusing the simplified bi-text, and show that it outperforms the initial system\nthat was built over the reference data set. Performance is further boosted when\nboth reference and automatic translations are used to learn the network. We\nperform an elementary analysis of the translated corpus and report accuracy\nresults of the proposed approach on English-to-French and English-to-German\ntranslation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06140v2",
            "date": "2016-12-19 11:51:35+00:00",
            "title": "Domain Control for Neural Machine Translation",
            "authors": [
                "Catherine Kobus",
                "Josep Crego",
                "Jean Senellart"
            ],
            "abstract": "Machine translation systems are very sensitive to the domains they were\ntrained on. Several domain adaptation techniques have been deeply studied. We\npropose a new technique for neural machine translation (NMT) that we call\ndomain control which is performed at runtime using a unique neural network\ncovering multiple domains. The presented approach shows quality improvements\nwhen compared to dedicated domains translating on any of the covered domains\nand even on out-of-domain data. In addition, model parameters do not need to be\nre-estimated for each domain, making this effective to real use cases.\nEvaluation is carried out on English-to-French translation for two different\ntesting scenarios. We first consider the case where an end-user performs\ntranslations on a known domain. Secondly, we consider the scenario where the\ndomain is not known and predicted at the sentence level before translating.\nResults show consistent accuracy improvements for both conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06141v1",
            "date": "2016-12-19 11:52:08+00:00",
            "title": "Domain specialization: a post-training domain adaptation for Neural Machine Translation",
            "authors": [
                "Christophe Servan",
                "Josep Crego",
                "Jean Senellart"
            ],
            "abstract": "Domain adaptation is a key feature in Machine Translation. It generally\nencompasses terminology, domain and style adaptation, especially for human\npost-editing workflows in Computer Assisted Translation (CAT). With Neural\nMachine Translation (NMT), we introduce a new notion of domain adaptation that\nwe call \"specialization\" and which is showing promising results both in the\nlearning speed and in adaptation accuracy. In this paper, we propose to explore\nthis approach under several perspectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06475v1",
            "date": "2016-12-20 01:23:00+00:00",
            "title": "Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles",
            "authors": [
                "James Cross",
                "Liang Huang"
            ],
            "abstract": "Parsing accuracy using efficient greedy transition systems has improved\ndramatically in recent years thanks to neural networks. Despite striking\nresults in dependency parsing, however, neural models have not surpassed\nstate-of-the-art approaches in constituency parsing. To remedy this, we\nintroduce a new shift-reduce system whose stack contains merely sentence spans,\nrepresented by a bare minimum of LSTM features. We also design the first\nprovably optimal dynamic oracle for constituency parsing, which runs in\namortized O(1) time, compared to O(n^3) oracles for standard dependency\nparsing. Training with this oracle, we achieve the best F1 scores on both\nEnglish and French of any parser that does not use reranking or external data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06549v2",
            "date": "2016-12-20 08:49:59+00:00",
            "title": "Exploring Different Dimensions of Attention for Uncertainty Detection",
            "authors": [
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Neural networks with attention have proven effective for many natural\nlanguage processing tasks. In this paper, we develop attention mechanisms for\nuncertainty detection. In particular, we generalize standardly used attention\nmechanisms by introducing external attention and sequence-preserving attention.\nThese novel architectures differ from standard approaches in that they use\nexternal resources to compute attention weights and preserve sequence\ninformation. We compare them to other configurations along different dimensions\nof attention. Our novel architectures set the new state of the art on a\nWikipedia benchmark dataset and perform similar to the state-of-the-art model\non a biomedical benchmark which uses a large set of linguistic features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06572v2",
            "date": "2016-12-20 09:52:36+00:00",
            "title": "Unsupervised Dialogue Act Induction using Gaussian Mixtures",
            "authors": [
                "Tom\u00e1\u0161 Brychc\u00edn",
                "Pavel Kr\u00e1l"
            ],
            "abstract": "This paper introduces a new unsupervised approach for dialogue act induction.\nGiven the sequence of dialogue utterances, the task is to assign them the\nlabels representing their function in the dialogue.\n  Utterances are represented as real-valued vectors encoding their meaning. We\nmodel the dialogue as Hidden Markov model with emission probabilities estimated\nby Gaussian mixtures. We use Gibbs sampling for posterior inference.\n  We present the results on the standard Switchboard-DAMSL corpus. Our\nalgorithm achieves promising results compared with strong supervised baselines\nand outperforms other unsupervised algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06581v1",
            "date": "2016-12-20 10:10:34+00:00",
            "title": "Grammar rules for the isiZulu complex verb",
            "authors": [
                "C. Maria Keet",
                "Langa Khumalo"
            ],
            "abstract": "The isiZulu verb is known for its morphological complexity, which is a\nsubject for on-going linguistics research, as well as for prospects of\ncomputational use, such as controlled natural language interfaces, machine\ntranslation, and spellcheckers. To this end, we seek to answer the question as\nto what the precise grammar rules for the isiZulu complex verb are (and, by\nextension, the Bantu verb morphology). To this end, we iteratively specify the\ngrammar as a Context Free Grammar, and evaluate it computationally. The grammar\npresented in this paper covers the subject and object concords, negation,\npresent tense, aspect, mood, and the causative, applicative, stative, and the\nreciprocal verbal extensions, politeness, the wh-question modifiers, and aspect\ndoubling, ensuring their correct order as they appear in verbs. The grammar\nconforms to specification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06671v1",
            "date": "2016-12-20 14:13:38+00:00",
            "title": "Inferring the location of authors from words in their texts",
            "authors": [
                "Max Berggren",
                "Jussi Karlgren",
                "Robert \u00d6stling",
                "Mikael Parkvall"
            ],
            "abstract": "For the purposes of computational dialectology or other geographically bound\ntext analysis tasks, texts must be annotated with their or their authors'\nlocation. Many texts are locatable through explicit labels but most have no\nexplicit annotation of place. This paper describes a series of experiments to\ndetermine how positionally annotated microblog posts can be used to learn\nlocation-indicating words which then can be used to locate blog texts and their\nauthors. A Gaussian distribution is used to model the locational qualities of\nwords. We introduce the notion of placeness to describe how locational words\nare.\n  We find that modelling word distributions to account for several locations\nand thus several Gaussian distributions per word, defining a filter which picks\nout words with high placeness based on their local distributional context, and\naggregating locational information in a centroid for each text gives the most\nuseful results. The results are applied to data in the Swedish language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06685v1",
            "date": "2016-12-20 14:44:19+00:00",
            "title": "Stateology: State-Level Interactive Charting of Language, Feelings, and Values",
            "authors": [
                "Konstantinos Pappas",
                "Steven Wilson",
                "Rada Mihalcea"
            ],
            "abstract": "People's personality and motivations are manifest in their everyday language\nusage. With the emergence of social media, ample examples of such usage are\nprocurable. In this paper, we aim to analyze the vocabulary used by close to\n200,000 Blogger users in the U.S. with the purpose of geographically portraying\nvarious demographic, linguistic, and psychological dimensions at the state\nlevel. We give a description of a web-based tool for viewing maps that depict\nvarious characteristics of the social media users as derived from this large\nblog dataset of over two billion words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06778v3",
            "date": "2016-12-20 17:38:57+00:00",
            "title": "SCDV : Sparse Composite Document Vectors using soft clustering over distributional representations",
            "authors": [
                "Dheeraj Mekala",
                "Vivek Gupta",
                "Bhargavi Paranjape",
                "Harish Karnick"
            ],
            "abstract": "We present a feature vector formation technique for documents - Sparse\nComposite Document Vector (SCDV) - which overcomes several shortcomings of the\ncurrent distributional paragraph vector representations that are widely used\nfor text representation. In SCDV, word embedding's are clustered to capture\nmultiple semantic contexts in which words occur. They are then chained together\nto form document topic-vectors that can express complex, multi-topic documents.\nThrough extensive experiments on multi-class and multi-label classification\ntasks, we outperform the previous state-of-the-art method, NTSG (Liu et al.,\n2015a). We also show that SCDV embedding's perform well on heterogeneous tasks\nlike Topic Coherence, context-sensitive Learning and Information Retrieval.\nMoreover, we achieve significant reduction in training and prediction times\ncompared to other representation methods. SCDV achieves best of both worlds -\nbetter performance with lower time and space complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06821v2",
            "date": "2016-12-20 19:39:59+00:00",
            "title": "User Bias Removal in Review Score Prediction",
            "authors": [
                "Rahul Wadbude",
                "Vivek Gupta",
                "Dheeraj Mekala",
                "Harish Karnick"
            ],
            "abstract": "Review score prediction of text reviews has recently gained a lot of\nattention in recommendation systems. A major problem in models for review score\nprediction is the presence of noise due to user-bias in review scores. We\npropose two simple statistical methods to remove such noise and improve review\nscore prediction. Compared to other methods that use multiple classifiers, one\nfor each user, our model uses a single global classifier to predict review\nscores. We empirically evaluate our methods on two major categories\n(\\textit{Electronics} and \\textit{Movies and TV}) of the SNAP published Amazon\ne-Commerce Reviews data-set and Amazon \\textit{Fine Food} reviews data-set. We\nobtain improved review score prediction for three commonly used text feature\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06897v1",
            "date": "2016-12-20 22:07:51+00:00",
            "title": "Fast Domain Adaptation for Neural Machine Translation",
            "authors": [
                "Markus Freitag",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Neural Machine Translation (NMT) is a new approach for automatic translation\nof text from one human language into another. The basic concept in NMT is to\ntrain a large Neural Network that maximizes the translation performance on a\ngiven parallel corpus. NMT is gaining popularity in the research community\nbecause it outperformed traditional SMT approaches in several translation tasks\nat WMT and other evaluation tasks/benchmarks at least for some language pairs.\nHowever, many of the enhancements in SMT over the years have not been\nincorporated into the NMT framework. In this paper, we focus on one such\nenhancement namely domain adaptation. We propose an approach for adapting a NMT\nsystem to a new domain. The main idea behind domain adaptation is that the\navailability of large out-of-domain training data and a small in-domain\ntraining data. We report significant gains with our proposed method in both\nautomatic metrics and a human subjective evaluation metric on two language\npairs. With our adaptation method, we show large improvement on the new domain\nwhile the performance of our general domain only degrades slightly. In\naddition, our approach is fast enough to adapt an already trained system to a\nnew domain within few hours without the need to retrain the NMT model on the\ncombined data which usually takes several days/weeks depending on the volume of\nthe data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07130v1",
            "date": "2016-12-21 14:17:53+00:00",
            "title": "Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling",
            "authors": [
                "G\u00e1bor Berend"
            ],
            "abstract": "In this paper we propose and carefully evaluate a sequence labeling framework\nwhich solely utilizes sparse indicator features derived from dense distributed\nword representations. The proposed model obtains (near) state-of-the art\nperformance for both part-of-speech tagging and named entity recognition for a\nvariety of languages. Our model relies only on a few thousand sparse\ncoding-derived features, without applying any modification of the word\nrepresentations employed for the different tasks. The proposed model has\nfavorable generalization properties as it retains over 89.8% of its average POS\ntagging accuracy when trained at 1.2% of the total available training data,\ni.e.~150 sentences per language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07215v2",
            "date": "2016-12-21 16:12:45+00:00",
            "title": "Inverted Bilingual Topic Models for Lexicon Extraction from Non-parallel Data",
            "authors": [
                "Tengfei Ma",
                "Tetsuya Nasukawa"
            ],
            "abstract": "Topic models have been successfully applied in lexicon extraction. However,\nmost previous methods are limited to document-aligned data. In this paper, we\ntry to address two challenges of applying topic models to lexicon extraction in\nnon-parallel data: 1) hard to model the word relationship and 2) noisy seed\ndictionary. To solve these two challenges, we propose two new bilingual topic\nmodels to better capture the semantic information of each word while\ndiscriminating the multiple translations in a noisy seed dictionary. We extend\nthe scope of topic models by inverting the roles of \"word\" and \"document\". In\naddition, to solve the problem of noise in seed dictionary, we incorporate the\nprobability of translation selection in our models. Moreover, we also propose\nan effective measure to evaluate the similarity of words in different languages\nand select the optimal translation pairs. Experimental results using real world\ndata demonstrate the utility and efficacy of the proposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07486v2",
            "date": "2016-12-22 08:29:25+00:00",
            "title": "Continuous multilinguality with language vectors",
            "authors": [
                "Robert \u00d6stling",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Most existing models for multilingual natural language processing (NLP) treat\nlanguage as a discrete category, and make predictions for either one language\nor the other. In contrast, we propose using continuous vector representations\nof language. We show that these can be learned efficiently with a\ncharacter-based neural language model, and used to improve inference about\nlanguage varieties not seen during training. In experiments with 1303 Bible\ntranslations into 990 different languages, we empirically explore the capacity\nof multilingual language models, and also show that the language vectors\ncapture genetic relationships between languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07495v2",
            "date": "2016-12-22 09:05:35+00:00",
            "title": "Noise Mitigation for Neural Entity Typing and Relation Extraction",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In this paper, we address two different types of noise in information\nextraction models: noise from distant supervision and noise from pipeline input\nfeatures. Our target tasks are entity typing and relation extraction. For the\nfirst noise type, we introduce multi-instance multi-label learning algorithms\nusing neural network models, and apply them to fine-grained entity typing for\nthe first time. This gives our models comparable performance with the\nstate-of-the-art supervised approach which uses global embeddings of entities.\nFor the second noise type, we propose ways to improve the integration of noisy\nentity type predictions into relation extraction. Our experiments show that\nprobabilistic predictions are more robust than discrete predictions and that\njoint training of the two tasks performs best.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07956v1",
            "date": "2016-12-23 12:58:58+00:00",
            "title": "A CRF Based POS Tagger for Code-mixed Indian Social Media Text",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "In this work, we describe a conditional random fields (CRF) based system for\nPart-Of- Speech (POS) tagging of code-mixed Indian social media text as part of\nour participation in the tool contest on POS tagging for codemixed Indian\nsocial media text, held in conjunction with the 2016 International Conference\non Natural Language Processing, IIT(BHU), India. We participated only in\nconstrained mode contest for all three language pairs, Bengali-English,\nHindi-English and Telegu-English. Our system achieves the overall average F1\nscore of 79.99, which is the highest overall average F1 score among all 16\nsystems participated in constrained mode contest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08083v3",
            "date": "2016-12-23 20:32:33+00:00",
            "title": "Language Modeling with Gated Convolutional Networks",
            "authors": [
                "Yann N. Dauphin",
                "Angela Fan",
                "Michael Auli",
                "David Grangier"
            ],
            "abstract": "The pre-dominant approach to language modeling to date is based on recurrent\nneural networks. Their success on this task is often linked to their ability to\ncapture unbounded context. In this paper we develop a finite context approach\nthrough stacked convolutions, which can be more efficient since they allow\nparallelization over sequential tokens. We propose a novel simplified gating\nmechanism that outperforms Oord et al (2016) and investigate the impact of key\narchitectural decisions. The proposed approach achieves state-of-the-art on the\nWikiText-103 benchmark, even though it features long-term dependencies, as well\nas competitive results on the Google Billion Words benchmark. Our model reduces\nthe latency to score a sentence by an order of magnitude compared to a\nrecurrent baseline. To our knowledge, this is the first time a non-recurrent\napproach is competitive with strong recurrent models on these large scale\nlanguage tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08171v1",
            "date": "2016-12-24 12:37:19+00:00",
            "title": "KS_JU@DPIL-FIRE2016:Detecting Paraphrases in Indian Languages Using Multinomial Logistic Regression Model",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "In this work, we describe a system that detects paraphrases in Indian\nLanguages as part of our participation in the shared Task on detecting\nparaphrases in Indian Languages (DPIL) organized by Forum for Information\nRetrieval Evaluation (FIRE) in 2016. Our paraphrase detection method uses a\nmultinomial logistic regression model trained with a variety of features which\nare basically lexical and semantic level similarities between two sentences in\na pair. The performance of the system has been evaluated against the test set\nreleased for the FIRE 2016 shared task on DPIL. Our system achieves the highest\nf-measure of 0.95 on task1 in Punjabi language.The performance of our system on\ntask1 in Hindi language is f-measure of 0.90. Out of 11 teams participated in\nthe shared task, only four teams participated in all four languages, Hindi,\nPunjabi, Malayalam and Tamil, but the remaining 7 teams participated in one of\nthe four languages. We also participated in task1 and task2 both for all four\nIndian Languages. The overall average performance of our system including task1\nand task2 overall four languages is F1-score of 0.81 which is the second\nhighest score among the four systems that participated in all four languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08220v3",
            "date": "2016-12-24 21:36:07+00:00",
            "title": "Understanding Neural Networks through Representation Erasure",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Dan Jurafsky"
            ],
            "abstract": "While neural networks have been successfully applied to many natural language\nprocessing tasks, they come at the cost of interpretability. In this paper, we\npropose a general methodology to analyze and interpret decisions from a neural\nmodel by observing the effects on the model of erasing various parts of the\nrepresentation, such as input word-vector dimensions, intermediate hidden\nunits, or input words. We present several approaches to analyzing the effects\nof such erasure, from computing the relative difference in evaluation metrics,\nto using reinforcement learning to erase the minimum set of input words in\norder to flip a neural model's decision. In a comprehensive analysis of\nmultiple NLP tasks, including linguistic feature classification, sentence-level\nsentiment analysis, and document level sentiment aspect prediction, we show\nthat the proposed methodology not only offers clear explanations about neural\nmodel decisions, but also provides a way to conduct error analysis on neural\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08333v4",
            "date": "2016-12-26 07:17:30+00:00",
            "title": "Text Summarization using Deep Learning and Ridge Regression",
            "authors": [
                "Karthik Bangalore Mani"
            ],
            "abstract": "We develop models and extract relevant features for automatic text\nsummarization and investigate the performance of different models on the DUC\n2001 dataset. Two different models were developed, one being a ridge regressor\nand the other one was a multi-layer perceptron. The hyperparameters were varied\nand their performance were noted. We segregated the summarization task into 2\nmain steps, the first being sentence ranking and the second step being sentence\nselection. In the first step, given a document, we sort the sentences based on\ntheir Importance, and in the second step, in order to obtain non-redundant\nsentences, we weed out the sentences that are have high similarity with the\npreviously selected sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08375v1",
            "date": "2016-12-26 13:13:38+00:00",
            "title": "Abstractive Headline Generation for Spoken Content by Attentive Recurrent Neural Networks with ASR Error Modeling",
            "authors": [
                "Lang-Chi Yu",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Headline generation for spoken content is important since spoken content is\ndifficult to be shown on the screen and browsed by the user. It is a special\ntype of abstractive summarization, for which the summaries are generated word\nby word from scratch without using any part of the original content. Many deep\nlearning approaches for headline generation from text document have been\nproposed recently, all requiring huge quantities of training data, which is\ndifficult for spoken document summarization. In this paper, we propose an ASR\nerror modeling approach to learn the underlying structure of ASR error patterns\nand incorporate this model in an Attentive Recurrent Neural Network (ARNN)\narchitecture. In this way, the model for abstractive headline generation for\nspoken content can be learned from abundant text data and the ASR data for some\nrecognizers. Experiments showed very encouraging results and verified that the\nproposed ASR error model works well even when the input spoken content is\nrecognized by a recognizer very different from the one the model learned from.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08989v1",
            "date": "2016-12-28 21:10:28+00:00",
            "title": "Shamela: A Large-Scale Historical Arabic Corpus",
            "authors": [
                "Yonatan Belinkov",
                "Alexander Magidow",
                "Maxim Romanov",
                "Avi Shmidman",
                "Moshe Koppel"
            ],
            "abstract": "Arabic is a widely-spoken language with a rich and long history spanning more\nthan fourteen centuries. Yet existing Arabic corpora largely focus on the\nmodern period or lack sufficient diachronic information. We develop a\nlarge-scale, historical corpus of Arabic of about 1 billion words from diverse\nperiods of time. We clean this corpus, process it with a morphological\nanalyzer, and enhance it by detecting parallel passages and automatically\ndating undated texts. We demonstrate its utility with selected case-studies in\nwhich we show its application to the digital humanities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08994v2",
            "date": "2016-12-28 21:36:19+00:00",
            "title": "Here's My Point: Joint Pointer Architecture for Argument Mining",
            "authors": [
                "Peter Potash",
                "Alexey Romanov",
                "Anna Rumshisky"
            ],
            "abstract": "One of the major goals in automated argumentation mining is to uncover the\nargument structure present in argumentative text. In order to determine this\nstructure, one must understand how different individual components of the\noverall argument are linked. General consensus in this field dictates that the\nargument components form a hierarchy of persuasion, which manifests itself in a\ntree structure. This work provides the first neural network-based approach to\nargumentation mining, focusing on the two tasks of extracting links between\nargument components, and classifying types of argument components. In order to\nsolve this problem, we propose to use a joint model that is based on a Pointer\nNetwork architecture. A Pointer Network is appealing for this task for the\nfollowing reasons: 1) It takes into account the sequential nature of argument\ncomponents; 2) By construction, it enforces certain properties of the tree\nstructure present in argument relations; 3) The hidden representations can be\napplied to auxiliary tasks. In order to extend the contribution of the original\nPointer Network model, we construct a joint model that simultaneously attempts\nto learn the type of argument component, as well as continuing to predict links\nbetween argument components. The proposed joint model achieves state-of-the-art\nresults on two separate evaluation corpora, achieving far superior performance\nthan a regular Pointer Network model. Our results show that optimizing for both\ntasks, and adding a fully-connected layer prior to recurrent neural network\ninput, is crucial for high performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.09113v1",
            "date": "2016-12-29 12:01:22+00:00",
            "title": "Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies",
            "authors": [
                "Jonathan Godwin",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "In this paper we present a novel Neural Network algorithm for conducting\nsemi-supervised learning for sequence labeling tasks arranged in a\nlinguistically motivated hierarchy. This relationship is exploited to\nregularise the representations of supervised tasks by backpropagating the error\nof the unsupervised task through the supervised tasks. We introduce a neural\nnetwork where lower layers are supervised by junior downstream tasks and the\nfinal layer task is an auxiliary unsupervised task. The architecture shows\nimprovements of up to two percentage points F1 for Chunking compared to a\nplausible baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.00375v2",
            "date": "2016-03-01 17:43:37+00:00",
            "title": "Easy-First Dependency Parsing with Hierarchical Tree LSTMs",
            "authors": [
                "Eliyahu Kiperwasser",
                "Yoav Goldberg"
            ],
            "abstract": "We suggest a compositional vector representation of parse trees that relies\non a recursive combination of recurrent-neural network encoders. To demonstrate\nits effectiveness, we use the representation as the backbone of a greedy,\nbottom-up dependency parser, achieving state-of-the-art accuracies for English\nand Chinese, without relying on external word embeddings. The parser's\nimplementation is available for download at the first author's webpage.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  4:445-461 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.00786v2",
            "date": "2016-03-02 16:41:40+00:00",
            "title": "Improving Named Entity Recognition for Chinese Social Media with Word Segmentation Representation Learning",
            "authors": [
                "Nanyun Peng",
                "Mark Dredze"
            ],
            "abstract": "Named entity recognition, and other information extraction tasks, frequently\nuse linguistic features such as part of speech tags or chunkings. For languages\nwhere word boundaries are not readily identified in text, word segmentation is\na key first step to generating features for an NER system. While using word\nboundary tags as features are helpful, the signals that aid in identifying\nthese boundaries may provide richer information for an NER system. New\nstate-of-the-art word segmentation systems use neural models to learn\nrepresentations for predicting word boundaries. We show that these same\nrepresentations, jointly trained with an NER system, yield significant\nimprovements in NER for Chinese social media. In our experiments, jointly\ntraining NER and word segmentation with an LSTM-CRF model yields nearly 5%\nabsolute improvement over previously published results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.00957v3",
            "date": "2016-03-03 03:22:01+00:00",
            "title": "Question Answering on Freebase via Relation Extraction and Textual Evidence",
            "authors": [
                "Kun Xu",
                "Siva Reddy",
                "Yansong Feng",
                "Songfang Huang",
                "Dongyan Zhao"
            ],
            "abstract": "Existing knowledge-based question answering systems often rely on small\nannotated training data. While shallow methods like relation extraction are\nrobust to data scarcity, they are less expressive than the deep meaning\nrepresentation methods like semantic parsing, thereby failing at answering\nquestions involving multiple constraints. Here we alleviate this problem by\nempowering a relation extraction method with additional evidence from\nWikipedia. We first present a neural network based relation extractor to\nretrieve the candidate answers from Freebase, and then infer over Wikipedia to\nvalidate these answers. Experiments on the WebQuestions question answering\ndataset show that our method achieves an F_1 of 53.3%, a substantial\nimprovement over the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.00968v2",
            "date": "2016-03-03 04:12:02+00:00",
            "title": "MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification",
            "authors": [
                "Ye Zhang",
                "Stephen Roller",
                "Byron Wallace"
            ],
            "abstract": "We introduce a novel, simple convolution neural network (CNN) architecture -\nmulti-group norm constraint CNN (MGNC-CNN) that capitalizes on multiple sets of\nword embeddings for sentence classification. MGNC-CNN extracts features from\ninput embedding sets independently and then joins these at the penultimate\nlayer in the network to form a final feature vector. We then adopt a group\nregularization strategy that differentially penalizes weights associated with\nthe subcomponents generated from the respective embedding sets. This model is\nmuch simpler than comparable alternative architectures and requires\nsubstantially less training time. Furthermore, it is flexible in that it does\nnot require input word embeddings to be of the same dimensionality. We show\nthat MGNC-CNN consistently outperforms baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01032v2",
            "date": "2016-03-03 09:33:09+00:00",
            "title": "Right Ideals of a Ring and Sublanguages of Science",
            "authors": [
                "Javier Arias Navarro"
            ],
            "abstract": "Among Zellig Harris's numerous contributions to linguistics his theory of the\nsublanguages of science probably ranks among the most underrated. However, not\nonly has this theory led to some exhaustive and meaningful applications in the\nstudy of the grammar of immunology language and its changes over time, but it\nalso illustrates the nature of mathematical relations between chunks or subsets\nof a grammar and the language as a whole. This becomes most clear when dealing\nwith the connection between metalanguage and language, as well as when\nreflecting on operators.\n  This paper tries to justify the claim that the sublanguages of science stand\nin a particular algebraic relation to the rest of the language they are\nembedded in, namely, that of right ideals in a ring.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01232v1",
            "date": "2016-03-03 19:49:32+00:00",
            "title": "Multi-domain Neural Network Language Generation for Spoken Dialogue Systems",
            "authors": [
                "Tsung-Hsien Wen",
                "Milica Gasic",
                "Nikola Mrksic",
                "Lina M. Rojas-Barahona",
                "Pei-Hao Su",
                "David Vandyke",
                "Steve Young"
            ],
            "abstract": "Moving from limited-domain natural language generation (NLG) to open domain\nis difficult because the number of semantic input combinations grows\nexponentially with the number of domains. Therefore, it is important to\nleverage existing resources and exploit similarities between domains to\nfacilitate domain adaptation. In this paper, we propose a procedure to train\nmulti-domain, Recurrent Neural Network-based (RNN) language generators via\nmultiple adaptation steps. In this procedure, a model is first trained on\ncounterfeited data synthesised from an out-of-domain dataset, and then fine\ntuned on a small set of in-domain utterances with a discriminative objective\nfunction. Corpus-based evaluation results show that the proposed procedure can\nachieve competitive performance in terms of BLEU score and slot error rate\nwhile significantly reducing the data needed to train generators in new, unseen\ndomains. In subjective testing, human judges confirm that the procedure greatly\nimproves generator performance when only a small amount of data is available in\nthe domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01333v1",
            "date": "2016-03-04 02:15:11+00:00",
            "title": "Joint Learning Templates and Slots for Event Schema Induction",
            "authors": [
                "Lei Sha",
                "Sujian Li",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "Automatic event schema induction (AESI) means to extract meta-event from raw\ntext, in other words, to find out what types (templates) of event may exist in\nthe raw text and what roles (slots) may exist in each event type. In this\npaper, we propose a joint entity-driven model to learn templates and slots\nsimultaneously based on the constraints of templates and slots in the same\nsentence. In addition, the entities' semantic information is also considered\nfor the inner connectivity of the entities. We borrow the normalized cut\ncriteria in image segmentation to divide the entities into more accurate\ntemplate clusters and slot clusters. The experiment shows that our model gains\na relatively higher result than previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01360v3",
            "date": "2016-03-04 06:36:29+00:00",
            "title": "Neural Architectures for Named Entity Recognition",
            "authors": [
                "Guillaume Lample",
                "Miguel Ballesteros",
                "Sandeep Subramanian",
                "Kazuya Kawakami",
                "Chris Dyer"
            ],
            "abstract": "State-of-the-art named entity recognition systems rely heavily on\nhand-crafted features and domain-specific knowledge in order to learn\neffectively from the small, supervised training corpora that are available. In\nthis paper, we introduce two new neural architectures---one based on\nbidirectional LSTMs and conditional random fields, and the other that\nconstructs and labels segments using a transition-based approach inspired by\nshift-reduce parsers. Our models rely on two sources of information about\nwords: character-based word representations learned from the supervised corpus\nand unsupervised word representations learned from unannotated corpora. Our\nmodels obtain state-of-the-art performance in NER in four languages without\nresorting to any language-specific knowledge or resources such as gazetteers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01514v1",
            "date": "2016-03-04 16:03:53+00:00",
            "title": "A Bayesian Model of Multilingual Unsupervised Semantic Role Induction",
            "authors": [
                "Nikhil Garg",
                "James Henderson"
            ],
            "abstract": "We propose a Bayesian model of unsupervised semantic role induction in\nmultiple languages, and use it to explore the usefulness of parallel corpora\nfor this task. Our joint Bayesian model consists of individual models for each\nlanguage plus additional latent variables that capture alignments between roles\nacross languages. Because it is a generative Bayesian model, we can do\nevaluations in a variety of scenarios just by varying the inference procedure,\nwithout changing the model, thereby comparing the scenarios directly. We\ncompare using only monolingual data, using a parallel corpus, using a parallel\ncorpus with annotations in the other language, and using small amounts of\nannotation in the target language. We find that the biggest impact of adding a\nparallel corpus to training is actually the increase in mono-lingual data, with\nthe alignments to another language resulting in small improvements, even with\nlabeled data for the other language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01541v1",
            "date": "2016-03-04 17:11:51+00:00",
            "title": "Parallel Texts in the Hebrew Bible, New Methods and Visualizations",
            "authors": [
                "Martijn Naaijer",
                "Dirk Roorda"
            ],
            "abstract": "In this article we develop an algorithm to detect parallel texts in the\nMasoretic Text of the Hebrew Bible. The results are presented online and\nchapters in the Hebrew Bible containing parallel passages can be inspected\nsynoptically. Differences between parallel passages are highlighted. In a\nsimilar way the MT of Isaiah is presented synoptically with 1QIsaa. We also\ninvestigate how one can investigate the degree of similarity between parallel\npassages with the help of a case study of 2 Kings 19-25 and its parallels in\nIsaiah, Jeremiah and 2 Chronicles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01547v2",
            "date": "2016-03-04 17:32:42+00:00",
            "title": "Text Understanding with the Attention Sum Reader Network",
            "authors": [
                "Rudolf Kadlec",
                "Martin Schmid",
                "Ondrej Bajgar",
                "Jan Kleindienst"
            ],
            "abstract": "Several large cloze-style context-question-answer datasets have been\nintroduced recently: the CNN and Daily Mail news data and the Children's Book\nTest. Thanks to the size of these datasets, the associated text comprehension\ntask is well suited for deep-learning techniques that currently seem to\noutperform all alternative approaches. We present a new, simple model that uses\nattention to directly pick the answer from the context as opposed to computing\nthe answer using a blended representation of words in the document as is usual\nin similar models. This makes the model particularly suitable for\nquestion-answering problems where the answer is a single word from the\ndocument. Ensemble of our models sets new state of the art on all evaluated\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01648v1",
            "date": "2016-03-04 22:47:46+00:00",
            "title": "Getting More Out Of Syntax with PropS",
            "authors": [
                "Gabriel Stanovsky",
                "Jessica Ficler",
                "Ido Dagan",
                "Yoav Goldberg"
            ],
            "abstract": "Semantic NLP applications often rely on dependency trees to recognize major\nelements of the proposition structure of sentences. Yet, while much semantic\nstructure is indeed expressed by syntax, many phenomena are not easily read out\nof dependency trees, often leading to further ad-hoc heuristic post-processing\nor to information loss. To directly address the needs of semantic applications,\nwe present PropS -- an output representation designed to explicitly and\nuniformly express much of the proposition structure which is implied from\nsyntax, and an associated tool for extracting it from dependency trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02488v1",
            "date": "2016-03-08 11:47:35+00:00",
            "title": "Extracting Arabic Relations from the Web",
            "authors": [
                "Shimaa M. Abd El-salam",
                "Enas M. F. El Houby",
                "A. K. Al Sammak",
                "T. A. El-Shishtawy"
            ],
            "abstract": "The goal of this research is to extract a large list or table from named\nentities and relations in a specific domain. A small set of a handful of\ninstance relations is required as input from the user. The system exploits\nsummaries from Google search engine as a source text. These instances are used\nto extract patterns. The output is a set of new entities and their relations.\nThe results from four experiments show that precision and recall varies\naccording to relation type. Precision ranges from 0.61 to 0.75 while recall\nranges from 0.71 to 0.83. The best result is obtained for (player, club)\nrelationship, 0.72 and 0.83 for precision and recall respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02604v1",
            "date": "2016-03-08 17:43:48+00:00",
            "title": "Observing Trends in Automated Multilingual Media Analysis",
            "authors": [
                "Ralf Steinberger",
                "Aldo Podavini",
                "Alexandra Balahur",
                "Guillaume Jacquet",
                "Hristo Tanev",
                "Jens Linge",
                "Martin Atkinson",
                "Michele Chinosi",
                "Vanni Zavarella",
                "Yaniv Steiner",
                "Erik van der Goot"
            ],
            "abstract": "Any large organisation, be it public or private, monitors the media for\ninformation to keep abreast of developments in their field of interest, and\nusually also to become aware of positive or negative opinions expressed towards\nthem. At least for the written media, computer programs have become very\nefficient at helping the human analysts significantly in their monitoring task\nby gathering media reports, analysing them, detecting trends and - in some\ncases - even to issue early warnings or to make predictions of likely future\ndevelopments. We present here trend recognition-related functionality of the\nEurope Media Monitor (EMM) system, which was developed by the European\nCommission's Joint Research Centre (JRC) for public administrations in the\nEuropean Union (EU) and beyond. EMM performs large-scale media analysis in up\nto seventy languages and recognises various types of trends, some of them\ncombining information from news articles written in different languages and\nfrom social media posts. EMM also lets users explore the huge amount of\nmultilingual media data through interactive maps and graphs, allowing them to\nexamine the data from various view points and according to multiple criteria. A\nlot of EMM's functionality is accessibly freely over the internet or via apps\nfor hand-held devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02845v1",
            "date": "2016-03-09 11:14:23+00:00",
            "title": "Unsupervised word segmentation and lexicon discovery using acoustic word embeddings",
            "authors": [
                "Herman Kamper",
                "Aren Jansen",
                "Sharon Goldwater"
            ],
            "abstract": "In settings where only unlabelled speech data is available, speech technology\nneeds to be developed without transcriptions, pronunciation dictionaries, or\nlanguage modelling text. A similar problem is faced when modelling infant\nlanguage acquisition. In these cases, categorical linguistic structure needs to\nbe discovered directly from speech audio. We present a novel unsupervised\nBayesian model that segments unlabelled speech and clusters the segments into\nhypothesized word groupings. The result is a complete unsupervised tokenization\nof the input speech in terms of discovered word types. In our approach, a\npotential word segment (of arbitrary length) is embedded in a fixed-dimensional\nacoustic vector space. The model, implemented as a Gibbs sampler, then builds a\nwhole-word acoustic model in this space while jointly performing segmentation.\nWe report word error rates in a small-vocabulary connected digit recognition\ntask by mapping the unsupervised decoded output to ground truth transcriptions.\nThe model achieves around 20% error rate, outperforming a previous HMM-based\nsystem by about 10% absolute. Moreover, in contrast to the baseline, our model\ndoes not require a pre-specified vocabulary size.",
            "journal ref": "IEEE/ACM Trans. Audio, Speech, Language Process. 24 (2016) 669-679",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02905v2",
            "date": "2016-03-09 14:56:44+00:00",
            "title": "Lexical bundles in computational linguistics academic literature",
            "authors": [
                "Adel Rahimi"
            ],
            "abstract": "In this study we analyzed a corpus of 8 million words academic literature\nfrom Computational lingustics' academic literature. the lexical bundles from\nthis corpus are categorized based on structures and functions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03758v2",
            "date": "2016-03-11 20:48:49+00:00",
            "title": "Sieve-based Coreference Resolution in the Biomedical Domain",
            "authors": [
                "Dane Bell",
                "Gus Hahn-Powell",
                "Marco A. Valenzuela-Esc\u00e1rcega",
                "Mihai Surdeanu"
            ],
            "abstract": "We describe challenges and advantages unique to coreference resolution in the\nbiomedical domain, and a sieve-based architecture that leverages domain\nknowledge for both entity and event coreference resolution. Domain-general\ncoreference resolution algorithms perform poorly on biomedical documents,\nbecause the cues they rely on such as gender are largely absent in this domain,\nand because they do not encode domain-specific knowledge such as the number and\ntype of participants required in chemical reactions. Moreover, it is difficult\nto directly encode this knowledge into most coreference resolution algorithms\nbecause they are not rule-based. Our rule-based architecture uses sequentially\napplied hand-designed \"sieves\", with the output of each sieve informing and\nconstraining subsequent sieves. This architecture provides a 3.2% increase in\nthroughput to our Reach event extraction system with precision parallel to that\nof the stricter system that relies solely on syntactic patterns for extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03793v2",
            "date": "2016-03-11 21:34:20+00:00",
            "title": "Training with Exploration Improves a Greedy Stack-LSTM Parser",
            "authors": [
                "Miguel Ballesteros",
                "Yoav Goldberg",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We adapt the greedy Stack-LSTM dependency parser of Dyer et al. (2015) to\nsupport a training-with-exploration procedure using dynamic oracles(Goldberg\nand Nivre, 2013) instead of cross-entropy minimization. This form of training,\nwhich accounts for model predictions at training time rather than assuming an\nerror-free action history, improves parsing accuracies for both English and\nChinese, obtaining very strong results for both languages. We discuss some\nmodifications needed in order to get training with exploration to work well for\na probabilistic neural-network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03873v1",
            "date": "2016-03-12 08:54:16+00:00",
            "title": "Neural Discourse Relation Recognition with Semantic Memory",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su"
            ],
            "abstract": "Humans comprehend the meanings and relations of discourses heavily relying on\ntheir semantic memory that encodes general knowledge about concepts and facts.\nInspired by this, we propose a neural recognizer for implicit discourse\nrelation analysis, which builds upon a semantic memory that stores knowledge in\na distributed fashion. We refer to this recognizer as SeMDER. Starting from\nword embeddings of discourse arguments, SeMDER employs a shallow encoder to\ngenerate a distributed surface representation for a discourse. A semantic\nencoder with attention to the semantic memory matrix is further established\nover surface representations. It is able to retrieve a deep semantic meaning\nrepresentation for the discourse from the memory. Using the surface and\nsemantic representations as input, SeMDER finally predicts implicit discourse\nrelations via a neural recognizer. Experiments on the benchmark data set show\nthat SeMDER benefits from the semantic memory and achieves substantial\nimprovements of 2.56\\% on average over current state-of-the-art baselines in\nterms of F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03876v2",
            "date": "2016-03-12 09:11:30+00:00",
            "title": "Variational Neural Discourse Relation Recognizer",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su",
                "Qun Liu",
                "Rongrong Ji",
                "Hong Duan",
                "Min Zhang"
            ],
            "abstract": "Implicit discourse relation recognition is a crucial component for automatic\ndiscourselevel analysis and nature language understanding. Previous studies\nexploit discriminative models that are built on either powerful manual features\nor deep discourse representations. In this paper, instead, we explore\ngenerative models and propose a variational neural discourse relation\nrecognizer. We refer to this model as VarNDRR. VarNDRR establishes a directed\nprobabilistic model with a latent continuous variable that generates both a\ndiscourse and the relation between the two arguments of the discourse. In order\nto perform efficient inference and learning, we introduce neural discourse\nrelation models to approximate the prior and posterior distributions of the\nlatent variable, and employ these approximated distributions to optimize a\nreparameterized variational lower bound. This allows VarNDRR to be trained with\nstandard stochastic gradient methods. Experiments on the benchmark data set\nshow that VarNDRR can achieve comparable results against stateof- the-art\nbaselines without using any manual features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04236v5",
            "date": "2016-03-14 12:33:00+00:00",
            "title": "Interactive Tools and Tasks for the Hebrew Bible",
            "authors": [
                "Nicolai Winther-Nielsen"
            ],
            "abstract": "This contribution to a special issue on \"Computer-aided processing of\nintertextuality\" in ancient texts will illustrate how using digital tools to\ninteract with the Hebrew Bible offers new promising perspectives for\nvisualizing the texts and for performing tasks in education and research. This\ncontribution explores how the corpus of the Hebrew Bible created and maintained\nby the Eep Talstra Centre for Bible and Computer can support new methods for\nmodern knowledge workers within the field of digital humanities and theology be\napplied to ancient texts, and how this can be envisioned as a new field of\ndigital intertextuality. The article first describes how the corpus was used to\ndevelop the Bible Online Learner as a persuasive technology to enhance language\nlearning with, in, and around a database that acts as the engine driving\ninteractive tasks for learners. Intertextuality in this case is a matter of\nactive exploration and ongoing practice. Furthermore, interactive\ncorpus-technology has an important bearing on the task of textual criticism as\na specialized area of research that depends increasingly on the availability of\ndigital resources. Commercial solutions developed by software companies like\nLogos and Accordance offer a market-based intertextuality defined by the\nproduction of advanced digital resources for scholars and students as useful\nalternatives to often inaccessible and expensive printed versions. It is\nreasonable to expect that in the future interactive corpus technology will\nallow scholars to do innovative academic tasks in textual criticism and\ninterpretation. We have already seen the emergence of promising tools for text\ncategorization, analysis of translation shifts, and interpretation. Broadly\nspeaking, interactive tools and tasks within the three areas of language\nlearning, textual criticism, and Biblical studies illustrate a new kind of\nintertextuality emerging within digital humanities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04351v3",
            "date": "2016-03-14 17:18:27+00:00",
            "title": "Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations",
            "authors": [
                "Eliyahu Kiperwasser",
                "Yoav Goldberg"
            ],
            "abstract": "We present a simple and effective scheme for dependency parsing which is\nbased on bidirectional-LSTMs (BiLSTMs). Each sentence token is associated with\na BiLSTM vector representing the token in its sentential context, and feature\nvectors are constructed by concatenating a few BiLSTM vectors. The BiLSTM is\ntrained jointly with the parser objective, resulting in very effective feature\nextractors for parsing. We demonstrate the effectiveness of the approach by\napplying it to a greedy transition-based parser as well as to a globally\noptimized graph-based parser. The resulting parsers have very simple\narchitectures, and match or surpass the state-of-the-art accuracies on English\nand Chinese.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  4:313--327 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04513v1",
            "date": "2016-03-15 00:25:02+00:00",
            "title": "Multichannel Variable-Size Convolution for Sentence Classification",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We propose MVCNN, a convolution neural network (CNN) architecture for\nsentence classification. It (i) combines diverse versions of pretrained word\nembeddings and (ii) extracts features of multigranular phrases with\nvariable-size convolution filters. We also show that pretraining MVCNN is\ncritical for good performance. MVCNN achieves state-of-the-art performance on\nfour tasks: on small-scale binary, small-scale multi-class and largescale\nTwitter sentiment prediction and on subjectivity classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04747v1",
            "date": "2016-03-15 16:21:58+00:00",
            "title": "Topic Modeling Using Distributed Word Embeddings",
            "authors": [
                "Ramandeep S Randhawa",
                "Parag Jain",
                "Gagan Madan"
            ],
            "abstract": "We propose a new algorithm for topic modeling, Vec2Topic, that identifies the\nmain topics in a corpus using semantic information captured via\nhigh-dimensional distributed word embeddings. Our technique is unsupervised and\ngenerates a list of topics ranked with respect to importance. We find that it\nworks better than existing topic modeling techniques such as Latent Dirichlet\nAllocation for identifying key topics in user-generated content, such as\nemails, chats, etc., where topics are diffused across the corpus. We also find\nthat Vec2Topic works equally well for non-user generated content, such as\npapers, reports, etc., and for small corpora such as a single-document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04767v1",
            "date": "2016-03-15 17:16:02+00:00",
            "title": "Evaluating the word-expert approach for Named-Entity Disambiguation",
            "authors": [
                "Angel X. Chang",
                "Valentin I. Spitkovsky",
                "Christopher D. Manning",
                "Eneko Agirre"
            ],
            "abstract": "Named Entity Disambiguation (NED) is the task of linking a named-entity\nmention to an instance in a knowledge-base, typically Wikipedia. This task is\nclosely related to word-sense disambiguation (WSD), where the supervised\nword-expert approach has prevailed. In this work we present the results of the\nword-expert approach to NED, where one classifier is built for each target\nentity mention string. The resources necessary to build the system, a\ndictionary and a set of training instances, have been automatically derived\nfrom Wikipedia. We provide empirical evidence of the value of this approach, as\nwell as a study of the differences between WSD and NED, including ambiguity and\nsynonymy statistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05118v2",
            "date": "2016-03-16 14:33:47+00:00",
            "title": "Recurrent Dropout without Memory Loss",
            "authors": [
                "Stanislau Semeniuta",
                "Aliaksei Severyn",
                "Erhardt Barth"
            ],
            "abstract": "This paper presents a novel approach to recurrent neural network (RNN)\nregularization. Differently from the widely adopted dropout method, which is\napplied to \\textit{forward} connections of feed-forward architectures or RNNs,\nwe propose to drop neurons directly in \\textit{recurrent} connections in a way\nthat does not cause loss of long-term memory. Our approach is as easy to\nimplement and apply as the regular feed-forward dropout and we demonstrate its\neffectiveness for Long Short-Term Memory network, the most popular type of RNN\ncells. Our experiments on NLP benchmarks show consistent improvements even when\ncombined with conventional feed-forward dropout.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05157v2",
            "date": "2016-03-16 16:02:03+00:00",
            "title": "Comparing Convolutional Neural Networks to Traditional Models for Slot Filling",
            "authors": [
                "Heike Adel",
                "Benjamin Roth",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We address relation classification in the context of slot filling, the task\nof finding and evaluating fillers like \"Steve Jobs\" for the slot X in \"X\nfounded Apple\". We propose a convolutional neural network which splits the\ninput sentence into three parts according to the relation arguments and compare\nit to state-of-the-art and traditional approaches of relation classification.\nFinally, we combine different methods and show that the combination is better\nthan individual approaches. We also analyze the effect of genre differences on\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05570v1",
            "date": "2016-03-17 16:43:37+00:00",
            "title": "Predicate Gradual Logic and Linguistics",
            "authors": [
                "Ryuta Arisaka"
            ],
            "abstract": "There are several major proposals for treating donkey anaphora such as\ndiscourse representation theory and the likes, or E-Type theories and the\nlikes. Every one of them works well for a set of specific examples that they\nuse to demonstrate validity of their approaches. As I show in this paper,\nhowever, they are not very generalisable and do not account for essentially the\nsame problem that they remedy when it manifests in other examples. I propose\nanother logical approach. I develoop logic that extends a recent, propositional\ngradual logic, and show that it can treat donkey anaphora generally. I also\nidentify and address a problem around the modern convention on existential\nimport. Furthermore, I show that Aristotle's syllogisms and conversion are\nrealisable in this logic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05739v1",
            "date": "2016-03-18 00:55:52+00:00",
            "title": "A Readability Analysis of Campaign Speeches from the 2016 US Presidential Campaign",
            "authors": [
                "Elliot Schumacher",
                "Maxine Eskenazi"
            ],
            "abstract": "Readability is defined as the reading level of the speech from grade 1 to\ngrade 12. It results from the use of the REAP readability analysis (vocabulary\n- Collins-Thompson and Callan, 2004; syntax - Heilman et al ,2006, 2007), which\nuse the lexical contents and grammatical structure of the sentences in a\ndocument to predict the reading level. After analysis, results were grouped\ninto the average readability of each candidate, the evolution of the\ncandidate's speeches' readability over time and the standard deviation, or how\nmuch each candidate varied their speech from one venue to another. For\ncomparison, one speech from four past presidents and the Gettysburg Address\nwere also analyzed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06009v1",
            "date": "2016-03-18 22:24:54+00:00",
            "title": "Readability-based Sentence Ranking for Evaluating Text Simplification",
            "authors": [
                "Sowmya Vajjala",
                "Detmar Meurers"
            ],
            "abstract": "We propose a new method for evaluating the readability of simplified\nsentences through pair-wise ranking. The validity of the method is established\nthrough in-corpus and cross-corpus evaluation experiments. The approach\ncorrectly identifies the ranking of simplified and unsimplified sentences in\nterms of their reading level with an accuracy of over 80%, significantly\noutperforming previous results. To gain qualitative insights into the nature of\nsimplification at the sentence level, we studied the impact of specific\nlinguistic features. We empirically confirm that both word-level and syntactic\nfeatures play a role in comparing the degree of simplification of authentic\ndata. To carry out this research, we created a new sentence-aligned corpus from\nprofessionally simplified news articles. The new corpus resource enriches the\nempirical basis of sentence-level simplification research, which so far relied\non a single resource. Most importantly, it facilitates cross-corpus evaluation\nfor simplification, a key step towards generalizable results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06021v3",
            "date": "2016-03-19 00:22:20+00:00",
            "title": "A Fast Unified Model for Parsing and Sentence Understanding",
            "authors": [
                "Samuel R. Bowman",
                "Jon Gauthier",
                "Abhinav Rastogi",
                "Raghav Gupta",
                "Christopher D. Manning",
                "Christopher Potts"
            ],
            "abstract": "Tree-structured neural networks exploit valuable syntactic parse information\nas they interpret the meanings of sentences. However, they suffer from two key\ntechnical problems that make them slow and unwieldy for large-scale NLP tasks:\nthey usually operate on parsed sentences and they do not directly support\nbatched computation. We address these issues by introducing the Stack-augmented\nParser-Interpreter Neural Network (SPINN), which combines parsing and\ninterpretation within a single tree-sequence hybrid model by integrating\ntree-structured sentence interpretation into the linear sequential structure of\na shift-reduce parser. Our model supports batched computation for a speedup of\nup to 25 times over other tree-structured models, and its integrated parser can\noperate on unparsed data with little loss in accuracy. We evaluate it on the\nStanford NLI entailment task and show that it significantly outperforms other\nsentence-encoding models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06067v3",
            "date": "2016-03-19 08:53:29+00:00",
            "title": "Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings",
            "authors": [
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "We present a novel method for jointly learning compositional and\nnon-compositional phrase embeddings by adaptively weighting both types of\nembeddings using a compositionality scoring function. The scoring function is\nused to quantify the level of compositionality of each phrase, and the\nparameters of the function are jointly optimized with the objective for\nlearning phrase embeddings. In experiments, we apply the adaptive joint\nlearning method to the task of learning embeddings of transitive verb phrases,\nand show that the compositionality scores have strong correlation with human\nratings for verb-object compositionality, substantially outperforming the\nprevious state of the art. Moreover, our embeddings improve upon the previous\nbest model on a transitive verb disambiguation task. We also show that a simple\nensemble technique further improves the results for both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06075v3",
            "date": "2016-03-19 10:08:40+00:00",
            "title": "Tree-to-Sequence Attentional Neural Machine Translation",
            "authors": [
                "Akiko Eriguchi",
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "Most of the existing Neural Machine Translation (NMT) models focus on the\nconversion of sequential data and do not directly use syntactic information. We\npropose a novel end-to-end syntactic NMT model, extending a\nsequence-to-sequence model with the source-side phrase structure. Our model has\nan attention mechanism that enables the decoder to generate a translated word\nwhile softly aligning it with phrases as well as words of the source sentence.\nExperimental results on the WAT'15 English-to-Japanese dataset demonstrate that\nour proposed model considerably outperforms sequence-to-sequence attentional\nNMT models and compares favorably with the state-of-the-art tree-to-string SMT\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06076v3",
            "date": "2016-03-19 10:09:53+00:00",
            "title": "Improving Hypernymy Detection with an Integrated Path-based and Distributional Method",
            "authors": [
                "Vered Shwartz",
                "Yoav Goldberg",
                "Ido Dagan"
            ],
            "abstract": "Detecting hypernymy relations is a key task in NLP, which is addressed in the\nliterature using two complementary approaches. Distributional methods, whose\nsupervised variants are the current best performers, and path-based methods,\nwhich received less research attention. We suggest an improved path-based\nalgorithm, in which the dependency paths are encoded using a recurrent neural\nnetwork, that achieves results comparable to distributional methods. We then\nextend the approach to integrate both path-based and distributional signals,\nsignificantly improving upon the state-of-the-art on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06155v2",
            "date": "2016-03-19 23:15:18+00:00",
            "title": "A Persona-Based Neural Conversation Model",
            "authors": [
                "Jiwei Li",
                "Michel Galley",
                "Chris Brockett",
                "Georgios P. Spithourakis",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "We present persona-based models for handling the issue of speaker consistency\nin neural response generation. A speaker model encodes personas in distributed\nembeddings that capture individual characteristics such as background\ninformation and speaking style. A dyadic speaker-addressee model captures\nproperties of interactions between two interlocutors. Our models yield\nqualitative performance improvements in both perplexity and BLEU scores over\nbaseline sequence-to-sequence models, with similar gains in speaker consistency\nas measured by human judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06503v1",
            "date": "2016-03-21 17:20:34+00:00",
            "title": "Static and Dynamic Feature Selection in Morphosyntactic Analyzers",
            "authors": [
                "Bernd Bohnet",
                "Miguel Ballesteros",
                "Ryan McDonald",
                "Joakim Nivre"
            ],
            "abstract": "We study the use of greedy feature selection methods for morphosyntactic\ntagging under a number of different conditions. We compare a static ordering of\nfeatures to a dynamic ordering based on mutual information statistics, and we\napply the techniques to standalone taggers as well as joint systems for tagging\nand parsing. Experiments on five languages show that feature selection can\nresult in more compact models as well as higher accuracy under all conditions,\nbut also that a dynamic ordering works better than a static ordering and that\njoint systems benefit more than standalone taggers. We also show that the same\ntechniques can be used to select which morphosyntactic categories to predict in\norder to maximize syntactic accuracy in a joint system. Our final results\nrepresent a substantial improvement of the state of the art for several\nlanguages, while at the same time reducing both the number of features and the\nrunning time by up to 80% in some cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06598v2",
            "date": "2016-03-21 20:12:44+00:00",
            "title": "Stack-propagation: Improved Representation Learning for Syntax",
            "authors": [
                "Yuan Zhang",
                "David Weiss"
            ],
            "abstract": "Traditional syntax models typically leverage part-of-speech (POS) information\nby constructing features from hand-tuned templates. We demonstrate that a\nbetter approach is to utilize POS tags as a regularizer of learned\nrepresentations. We propose a simple method for learning a stacked pipeline of\nmodels which we call \"stack-propagation\". We apply this to dependency parsing\nand tagging, where we use the hidden layer of the tagger network as a\nrepresentation of the input tokens for the parser. At test time, our parser\ndoes not require predicted POS tags. On 19 languages from the Universal\nDependencies, our method is 1.3% (absolute) more accurate than a\nstate-of-the-art graph-based approach and 2.7% more accurate than the most\ncomparable greedy model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07012v2",
            "date": "2016-03-22 22:15:10+00:00",
            "title": "Semi-supervised Word Sense Disambiguation with Neural Models",
            "authors": [
                "Dayu Yuan",
                "Julian Richardson",
                "Ryan Doherty",
                "Colin Evans",
                "Eric Altendorf"
            ],
            "abstract": "Determining the intended sense of words in text - word sense disambiguation\n(WSD) - is a long standing problem in natural language processing. Recently,\nresearchers have shown promising results using word vectors extracted from a\nneural network language model as features in WSD algorithms. However, a simple\naverage or concatenation of word vectors for each word in a text loses the\nsequential and syntactic information of the text. In this paper, we study WSD\nwith a sequence learning neural net, LSTM, to better capture the sequential and\nsyntactic patterns of the text. To alleviate the lack of training data in\nall-words WSD, we employ the same LSTM in a semi-supervised label propagation\nclassifier. We demonstrate state-of-the-art results, especially on verbs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07252v3",
            "date": "2016-03-23 16:05:46+00:00",
            "title": "Neural Summarization by Extracting Sentences and Words",
            "authors": [
                "Jianpeng Cheng",
                "Mirella Lapata"
            ],
            "abstract": "Traditional approaches to extractive summarization rely heavily on\nhuman-engineered features. In this work we propose a data-driven approach based\non neural networks and continuous sentence features. We develop a general\nframework for single-document summarization composed of a hierarchical document\nencoder and an attention-based extractor. This architecture allows us to\ndevelop different classes of summarization models which can extract sentences\nor words. We train our models on large scale corpora containing hundreds of\nthousands of document-summary pairs. Experimental results on two summarization\ndatasets demonstrate that our models obtain results comparable to the state of\nthe art without any access to linguistic annotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07253v2",
            "date": "2016-03-23 16:12:34+00:00",
            "title": "Evaluating semantic models with word-sentence relatedness",
            "authors": [
                "Kimberly Glasgow",
                "Matthew Roos",
                "Amy Haufler",
                "Mark Chevillet",
                "Michael Wolmetz"
            ],
            "abstract": "Semantic textual similarity (STS) systems are designed to encode and evaluate\nthe semantic similarity between words, phrases, sentences, and documents. One\nmethod for assessing the quality or authenticity of semantic information\nencoded in these systems is by comparison with human judgments. A data set for\nevaluating semantic models was developed consisting of 775 English\nword-sentence pairs, each annotated for semantic relatedness by human raters\nengaged in a Maximum Difference Scaling (MDS) task, as well as a faster\nalternative task. As a sample application of this relatedness data,\nbehavior-based relatedness was compared to the relatedness computed via four\noff-the-shelf STS models: n-gram, Latent Semantic Analysis (LSA), Word2Vec, and\nUMBC Ebiquity. Some STS models captured much of the variance in the human\njudgments collected, but they were not sensitive to the implicatures and\nentailments that were processed and considered by the participants. All text\nstimuli and judgment data have been made freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07603v1",
            "date": "2016-03-24 14:45:20+00:00",
            "title": "Semantic Regularities in Document Representations",
            "authors": [
                "Fei Sun",
                "Jiafeng Guo",
                "Yanyan Lan",
                "Jun Xu",
                "Xueqi Cheng"
            ],
            "abstract": "Recent work exhibited that distributed word representations are good at\ncapturing linguistic regularities in language. This allows vector-oriented\nreasoning based on simple linear algebra between words. Since many different\nmethods have been proposed for learning document representations, it is natural\nto ask whether there is also linear structure in these learned representations\nto allow similar reasoning at document level. To answer this question, we\ndesign a new document analogy task for testing the semantic regularities in\ndocument representations, and conduct empirical evaluations over several\nstate-of-the-art document representation models. The results reveal that neural\nembedding based document representations work better on this analogy task than\nconventional methods, and we provide some preliminary explanations over these\nobservations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07609v1",
            "date": "2016-03-24 14:59:45+00:00",
            "title": "Contrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL",
            "authors": [
                "Yevgeni Berzak",
                "Roi Reichart",
                "Boris Katz"
            ],
            "abstract": "This work examines the impact of cross-linguistic transfer on grammatical\nerrors in English as Second Language (ESL) texts. Using a computational\nframework that formalizes the theory of Contrastive Analysis (CA), we\ndemonstrate that language specific error distributions in ESL writing can be\npredicted from the typological properties of the native language and their\nrelation to the typology of English. Our typology driven model enables to\nobtain accurate estimates of such distributions without access to any ESL data\nfor the target languages. Furthermore, we present a strategy for adjusting our\nmethod to low-resource languages that lack typological documentation using a\nbootstrapping approach which approximates native language typology from ESL\ntexts. Finally, we show that our framework is instrumental for linguistic\ninquiry seeking to identify first language factors that contribute to a wide\nrange of difficulties in second language acquisition.",
            "journal ref": "Proceedings of the 19th Conference on Computational Language\n  Learning, pages 94-102, Beijing, China, July 30-31, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07695v1",
            "date": "2016-03-24 18:22:39+00:00",
            "title": "Part-of-Speech Relevance Weights for Learning Word Embeddings",
            "authors": [
                "Quan Liu",
                "Zhen-Hua Ling",
                "Hui Jiang",
                "Yu Hu"
            ],
            "abstract": "This paper proposes a model to learn word embeddings with weighted contexts\nbased on part-of-speech (POS) relevance weights. POS is a fundamental element\nin natural language. However, state-of-the-art word embedding models fail to\nconsider it. This paper proposes to use position-dependent POS relevance\nweighting matrices to model the inherent syntactic relationship among words\nwithin a context window. We utilize the POS relevance weights to model each\nword-context pairs during the word embedding training process. The model\nproposed in this paper paper jointly optimizes word vectors and the POS\nrelevance matrices. Experiments conducted on popular word analogy and word\nsimilarity tasks all demonstrated the effectiveness of the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07771v3",
            "date": "2016-03-24 22:40:00+00:00",
            "title": "Neural Text Generation from Structured Data with Application to the Biography Domain",
            "authors": [
                "Remi Lebret",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "This paper introduces a neural model for concept-to-text generation that\nscales to large, rich domains. We experiment with a new dataset of biographies\nfrom Wikipedia that is an order of magnitude larger than existing resources\nwith over 700k samples. The dataset is also vastly more diverse with a 400k\nvocabulary, compared to a few hundred words for Weathergov or Robocup. Our\nmodel builds upon recent work on conditional neural language model for text\ngeneration. To deal with the large vocabulary, we extend these models to mix a\nfixed vocabulary with copy actions that transfer sample-specific words from the\ninput database to the generated output sentence. Our neural model significantly\nout-performs a classical Kneser-Ney language model adapted to this task by\nnearly 15 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07954v3",
            "date": "2016-03-25 16:38:54+00:00",
            "title": "Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning",
            "authors": [
                "Karthik Narasimhan",
                "Adam Yala",
                "Regina Barzilay"
            ],
            "abstract": "Most successful information extraction systems operate with access to a large\ncollection of documents. In this work, we explore the task of acquiring and\nincorporating external evidence to improve extraction accuracy in domains where\nthe amount of training data is scarce. This process entails issuing search\nqueries, extraction from new sources and reconciliation of extracted values,\nwhich are repeated until sufficient evidence is collected. We approach the\nproblem using a reinforcement learning framework where our model learns to\nselect optimal actions based on contextual information. We employ a deep\nQ-network, trained to optimize a reward function that reflects extraction\naccuracy while penalizing extra effort. Our experiments on two databases -- of\nshooting incidents, and food adulteration cases -- demonstrate that our system\nsignificantly outperforms traditional extractors and a competitive\nmeta-classifier baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08016v2",
            "date": "2016-03-25 20:09:29+00:00",
            "title": "Classifying Syntactic Regularities for Hundreds of Languages",
            "authors": [
                "Reed Coke",
                "Ben King",
                "Dragomir Radev"
            ],
            "abstract": "This paper presents a comparison of classification methods for linguistic\ntypology for the purpose of expanding an extensive, but sparse language\nresource: the World Atlas of Language Structures (WALS) (Dryer and Haspelmath,\n2013). We experimented with a variety of regression and nearest-neighbor\nmethods for use in classification over a set of 325 languages and six syntactic\nrules drawn from WALS. To classify each rule, we consider the typological\nfeatures of the other five rules; linguistic features extracted from a\nword-aligned Bible in each language; and genealogical features (genus and\nfamily) of each language. In general, we find that propagating the majority\nlabel among all languages of the same genus achieves the best accuracy in label\npre- diction. Following this, a logistic regression model that combines\ntypological and linguistic features offers the next best performance.\nInterestingly, this model actually outperforms the majority labels among all\nlanguages of the same family.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08594v1",
            "date": "2016-03-29 00:06:11+00:00",
            "title": "Prepositional Attachment Disambiguation Using Bilingual Parsing and Alignments",
            "authors": [
                "Geetanjali Rakshit",
                "Sagar Sontakke",
                "Pushpak Bhattacharyya",
                "Gholamreza Haffari"
            ],
            "abstract": "In this paper, we attempt to solve the problem of Prepositional Phrase (PP)\nattachments in English. The motivation for the work comes from NLP applications\nlike Machine Translation, for which, getting the correct attachment of\nprepositions is very crucial. The idea is to correct the PP-attachments for a\nsentence with the help of alignments from parallel data in another language.\nThe novelty of our work lies in the formulation of the problem into a dual\ndecomposition based algorithm that enforces agreement between the parse trees\nfrom two languages as a constraint. Experiments were performed on the\nEnglish-Hindi language pair and the performance improved by 10% over the\nbaseline, where the baseline is the attachment predicted by the MSTParser model\ntrained for English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08701v1",
            "date": "2016-03-29 10:00:27+00:00",
            "title": "What a Nerd! Beating Students and Vector Cosine in the ESL and TOEFL Datasets",
            "authors": [
                "Enrico Santus",
                "Tin-Shing Chiu",
                "Qin Lu",
                "Alessandro Lenci",
                "Chu-Ren Huang"
            ],
            "abstract": "In this paper, we claim that Vector Cosine, which is generally considered one\nof the most efficient unsupervised measures for identifying word similarity in\nVector Space Models, can be outperformed by a completely unsupervised measure\nthat evaluates the extent of the intersection among the most associated\ncontexts of two target words, weighting such intersection according to the rank\nof the shared contexts in the dependency ranked lists. This claim comes from\nthe hypothesis that similar words do not simply occur in similar contexts, but\nthey share a larger portion of their most relevant contexts compared to other\nrelated words. To prove it, we describe and evaluate APSyn, a variant of\nAverage Precision that, independently of the adopted parameters, outperforms\nthe Vector Cosine and the co-occurrence on the ESL and TOEFL test sets. In the\nbest setting, APSyn reaches 0.73 accuracy on the ESL dataset and 0.70 accuracy\nin the TOEFL dataset, beating therefore the non-English US college applicants\n(whose average, as reported in the literature, is 64.50%) and several\nstate-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08702v1",
            "date": "2016-03-29 10:00:40+00:00",
            "title": "Nine Features in a Random Forest to Learn Taxonomical Semantic Relations",
            "authors": [
                "Enrico Santus",
                "Alessandro Lenci",
                "Tin-Shing Chiu",
                "Qin Lu",
                "Chu-Ren Huang"
            ],
            "abstract": "ROOT9 is a supervised system for the classification of hypernyms, co-hyponyms\nand random words that is derived from the already introduced ROOT13 (Santus et\nal., 2016). It relies on a Random Forest algorithm and nine unsupervised\ncorpus-based features. We evaluate it with a 10-fold cross validation on 9,600\npairs, equally distributed among the three classes and involving several\nParts-Of-Speech (i.e. adjectives, nouns and verbs). When all the classes are\npresent, ROOT9 achieves an F1 score of 90.7%, against a baseline of 57.2%\n(vector cosine). When the classification is binary, ROOT9 achieves the\nfollowing results against the baseline: hypernyms-co-hyponyms 95.7% vs. 69.8%,\nhypernyms-random 91.8% vs. 64.1% and co-hyponyms-random 97.8% vs. 79.4%. In\norder to compare the performance with the state-of-the-art, we have also\nevaluated ROOT9 in subsets of the Weeds et al. (2014) datasets, proving that it\nis in fact competitive. Finally, we investigated whether the system learns the\nsemantic relation or it simply learns the prototypical hypernyms, as claimed by\nLevy et al. (2015). The second possibility seems to be the most likely, even\nthough ROOT9 can be trained on negative examples (i.e., switched hypernyms) to\ndrastically reduce this bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08705v1",
            "date": "2016-03-29 10:05:05+00:00",
            "title": "ROOT13: Spotting Hypernyms, Co-Hyponyms and Randoms",
            "authors": [
                "Enrico Santus",
                "Tin-Shing Chiu",
                "Qin Lu",
                "Alessandro Lenci",
                "Chu-Ren Huang"
            ],
            "abstract": "In this paper, we describe ROOT13, a supervised system for the classification\nof hypernyms, co-hyponyms and random words. The system relies on a Random\nForest algorithm and 13 unsupervised corpus-based features. We evaluate it with\na 10-fold cross validation on 9,600 pairs, equally distributed among the three\nclasses and involving several Parts-Of-Speech (i.e. adjectives, nouns and\nverbs). When all the classes are present, ROOT13 achieves an F1 score of 88.3%,\nagainst a baseline of 57.6% (vector cosine). When the classification is binary,\nROOT13 achieves the following results: hypernyms-co-hyponyms (93.4% vs. 60.2%),\nhypernymsrandom (92.3% vs. 65.5%) and co-hyponyms-random (97.3% vs. 81.5%). Our\nresults are competitive with stateof-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08865v3",
            "date": "2016-03-29 17:58:46+00:00",
            "title": "Compilation as a Typed EDSL-to-EDSL Transformation",
            "authors": [
                "Emil Axelsson"
            ],
            "abstract": "This article is about an implementation and compilation technique that is\nused in RAW-Feldspar which is a complete rewrite of the Feldspar embedded\ndomain-specific language (EDSL) (Axelsson et al. 2010). Feldspar is high-level\nfunctional language that generates efficient C code to run on embedded targets.\nThe gist of the technique presented in this post is the following: rather\nwriting a back end that converts pure Feldspar expressions directly to C, we\ntranslate them to a low-level monadic EDSL. From the low-level EDSL, C code is\nthen generated. This approach has several advantages:\n  1. The translation is simpler to write than a complete C back end.\n  2. The translation is between two typed EDSLs, which rules out many potential\nerrors.\n  3. The low-level EDSL is reusable and can be shared between several\nhigh-level EDSLs.\n  Although the article contains a lot of code, most of it is in fact reusable.\nAs mentioned in Discussion, we can write the same implementation in less than\n50 lines of code using generic libraries that we have developed to support\nFeldspar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08868v1",
            "date": "2016-03-29 18:12:28+00:00",
            "title": "A Readable Read: Automatic Assessment of Language Learning Materials based on Linguistic Complexity",
            "authors": [
                "Ildik\u00f3 Pil\u00e1n",
                "Sowmya Vajjala",
                "Elena Volodina"
            ],
            "abstract": "Corpora and web texts can become a rich language learning resource if we have\na means of assessing whether they are linguistically appropriate for learners\nat a given proficiency level. In this paper, we aim at addressing this issue by\npresenting the first approach for predicting linguistic complexity for Swedish\nsecond language learning material on a 5-point scale. After showing that the\ntraditional Swedish readability measure, L\\\"asbarhetsindex (LIX), is not\nsuitable for this task, we propose a supervised machine learning model, based\non a range of linguistic features, that can reliably classify texts according\nto their difficulty level. Our model obtained an accuracy of 81.3% and an\nF-score of 0.8, which is comparable to the state of the art in English and is\nconsiderably higher than previously reported results for other languages. We\nfurther studied the utility of our features with single sentences instead of\nfull texts since sentences are a common linguistic unit in language learning\nexercises. We trained a separate model on sentence-level data with five\nclasses, which yielded 63.4% accuracy. Although this is lower than the document\nlevel performance, we achieved an adjacent accuracy of 92%. Furthermore, we\nfound that using a combination of different features, compared to using lexical\nfeatures alone, resulted in 7% improvement in classification accuracy at the\nsentence level, whereas at the document level, lexical features were more\ndominant. Our models are intended for use in a freely accessible web-based\nlanguage learning platform for the automatic generation of exercises.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08884v1",
            "date": "2016-03-29 18:52:46+00:00",
            "title": "A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data",
            "authors": [
                "Adam Trischler",
                "Zheng Ye",
                "Xingdi Yuan",
                "Jing He",
                "Phillip Bachman",
                "Kaheer Suleman"
            ],
            "abstract": "Understanding unstructured text is a major goal within natural language\nprocessing. Comprehension tests pose questions based on short text passages to\nevaluate such understanding. In this work, we investigate machine comprehension\non the challenging {\\it MCTest} benchmark. Partly because of its limited size,\nprior work on {\\it MCTest} has focused mainly on engineering better features.\nWe tackle the dataset with a neural approach, harnessing simple neural networks\narranged in a parallel hierarchy. The parallel hierarchy enables our model to\ncompare the passage, question, and answer from a variety of trainable\nperspectives, as opposed to using a manually designed, rigid feature set.\nPerspectives range from the word level to sentence fragments to sequences of\nsentences; the networks operate only on word-embedding representations of text.\nWhen trained with a methodology designed to help cope with limited training\ndata, our Parallel-Hierarchical model sets a new state of the art for {\\it\nMCTest}, outperforming previous feature-engineered approaches slightly and\nprevious neural approaches by a significant margin (over 15\\% absolute).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08887v2",
            "date": "2016-03-29 18:58:42+00:00",
            "title": "Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints",
            "authors": [
                "Greg Durrett",
                "Taylor Berg-Kirkpatrick",
                "Dan Klein"
            ],
            "abstract": "We present a discriminative model for single-document summarization that\nintegrally combines compression and anaphoricity constraints. Our model selects\ntextual units to include in the summary based on a rich set of sparse features\nwhose weights are learned on a large corpus. We allow for the deletion of\ncontent within a sentence when that deletion is licensed by compression rules;\nin our framework, these are implemented as dependencies between subsentential\nunits of text. Anaphoricity constraints then improve cross-sentence coherence\nby guaranteeing that, for each pronoun included in the summary, the pronoun's\nantecedent is included as well or the pronoun is rewritten as a full mention.\nWhen trained end-to-end, our final system outperforms prior work on both ROUGE\nas well as on human judgments of linguistic quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09054v1",
            "date": "2016-03-30 07:05:45+00:00",
            "title": "Unsupervised Measure of Word Similarity: How to Outperform Co-occurrence and Vector Cosine in VSMs",
            "authors": [
                "Enrico Santus",
                "Tin-Shing Chiu",
                "Qin Lu",
                "Alessandro Lenci",
                "Chu-Ren Huang"
            ],
            "abstract": "In this paper, we claim that vector cosine, which is generally considered\namong the most efficient unsupervised measures for identifying word similarity\nin Vector Space Models, can be outperformed by an unsupervised measure that\ncalculates the extent of the intersection among the most mutually dependent\ncontexts of the target words. To prove it, we describe and evaluate APSyn, a\nvariant of the Average Precision that, without any optimization, outperforms\nthe vector cosine and the co-occurrence on the standard ESL test set, with an\nimprovement ranging between +9.00% and +17.98%, depending on the number of\nchosen top contexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09457v1",
            "date": "2016-03-31 05:14:10+00:00",
            "title": "LSTM based Conversation Models",
            "authors": [
                "Yi Luan",
                "Yangfeng Ji",
                "Mari Ostendorf"
            ],
            "abstract": "In this paper, we present a conversational model that incorporates both\ncontext and participant role for two-party conversations. Different\narchitectures are explored for integrating participant role and context\ninformation into a Long Short-term Memory (LSTM) language model. The\nconversational model can function as a language model or a language generation\nmodel. Experiments on the Ubuntu Dialog Corpus show that our model can capture\nmultiple turn interaction between participants. The proposed method outperforms\na traditional LSTM model as measured by language model perplexity and response\nranking. Generated responses show characteristic differences between the two\nparticipant roles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00077v1",
            "date": "2016-03-31 23:17:46+00:00",
            "title": "Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection",
            "authors": [
                "Sheng-syun Shen",
                "Hung-yi Lee"
            ],
            "abstract": "Recurrent neural network architectures combining with attention mechanism, or\nneural attention model, have shown promising performance recently for the tasks\nincluding speech recognition, image caption generation, visual question\nanswering and machine translation. In this paper, neural attention model is\napplied on two sequence classification tasks, dialogue act detection and key\nterm extraction. In the sequence labeling tasks, the model input is a sequence,\nand the output is the label of the input sequence. The major difficulty of\nsequence labeling is that when the input sequence is long, it can include many\nnoisy or irrelevant part. If the information in the whole sequence is treated\nequally, the noisy or irrelevant part may degrade the classification\nperformance. The attention mechanism is helpful for sequence classification\ntask because it is capable of highlighting important part among the entire\nsequence for the classification task. The experimental results show that with\nthe attention mechanism, discernible improvements were achieved in the sequence\nlabeling task considered here. The roles of the attention mechanism in the\ntasks are further analyzed and visualized in this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00100v1",
            "date": "2016-04-01 01:51:34+00:00",
            "title": "A Compositional Approach to Language Modeling",
            "authors": [
                "Kushal Arora",
                "Anand Rangarajan"
            ],
            "abstract": "Traditional language models treat language as a finite state automaton on a\nprobability space over words. This is a very strong assumption when modeling\nsomething inherently complex such as language. In this paper, we challenge this\nby showing how the linear chain assumption inherent in previous work can be\ntranslated into a sequential composition tree. We then propose a new model that\nmarginalizes over all possible composition trees thereby removing any\nunderlying structural assumptions. As the partition function of this new model\nis intractable, we use a recently proposed sentence level evaluation metric\nContrastive Entropy to evaluate our model. Given this new evaluation metric, we\nreport more than 100% improvement across distortion levels over current state\nof the art recurrent neural network based language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00117v2",
            "date": "2016-04-01 03:24:32+00:00",
            "title": "Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding",
            "authors": [
                "Aaron Jaech",
                "Larry Heck",
                "Mari Ostendorf"
            ],
            "abstract": "The goal of this paper is to use multi-task learning to efficiently scale\nslot filling models for natural language understanding to handle multiple\ntarget tasks or domains. The key to scalability is reducing the amount of\ntraining data needed to learn a model for a new task. The proposed multi-task\nmodel delivers better performance with less data by leveraging patterns that it\nlearns from the other tasks. The approach supports an open vocabulary, which\nallows the models to generalize to unseen words, which is particularly\nimportant when very little training data is used. A newly collected\ncrowd-sourced data set, covering four different domains, is used to demonstrate\nthe effectiveness of the domain adaptation and open vocabulary techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00400v1",
            "date": "2016-04-01 20:06:46+00:00",
            "title": "Revisiting Summarization Evaluation for Scientific Articles",
            "authors": [
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "Evaluation of text summarization approaches have been mostly based on metrics\nthat measure similarities of system generated summaries with a set of human\nwritten gold-standard summaries. The most widely used metric in summarization\nevaluation has been the ROUGE family. ROUGE solely relies on lexical overlaps\nbetween the terms and phrases in the sentences; therefore, in cases of\nterminology variations and paraphrasing, ROUGE is not as effective. Scientific\narticle summarization is one such case that is different from general domain\nsummarization (e.g. newswire data). We provide an extensive analysis of ROUGE's\neffectiveness as an evaluation metric for scientific summarization; we show\nthat, contrary to the common belief, ROUGE is not much reliable in evaluating\nscientific summaries. We furthermore show how different variants of ROUGE\nresult in very different correlations with the manual Pyramid scores. Finally,\nwe propose an alternative metric for summarization evaluation which is based on\nthe content relevance between a system generated summary and the corresponding\nhuman written summaries. We call our metric SERA (Summarization Evaluation by\nRelevance Analysis). Unlike ROUGE, SERA consistently achieves high correlations\nwith manual scores which shows its effectiveness in evaluation of scientific\narticle summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00425v2",
            "date": "2016-04-01 22:18:51+00:00",
            "title": "Cross-lingual Models of Word Embeddings: An Empirical Comparison",
            "authors": [
                "Shyam Upadhyay",
                "Manaal Faruqui",
                "Chris Dyer",
                "Dan Roth"
            ],
            "abstract": "Despite interest in using cross-lingual knowledge to learn word embeddings\nfor various tasks, a systematic comparison of the possible approaches is\nlacking in the literature. We perform an extensive evaluation of four popular\napproaches of inducing cross-lingual embeddings, each requiring a different\nform of supervision, on four typographically different language pairs. Our\nevaluation setup spans four different tasks, including intrinsic evaluation on\nmono-lingual and cross-lingual similarity, and extrinsic evaluation on\ndownstream semantic and syntactic applications. We show that models which\nrequire expensive cross-lingual knowledge almost always perform better, but\ncheaply supervised models often prove competitive on certain tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00502v1",
            "date": "2016-04-02 13:52:23+00:00",
            "title": "Online Updating of Word Representations for Part-of-Speech Tagging",
            "authors": [
                "Wenpeng Yin",
                "Tobias Schnabel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We propose online unsupervised domain adaptation (DA), which is performed\nincrementally as data comes in and is applicable when batch DA is not possible.\nIn a part-of-speech (POS) tagging evaluation, we find that online unsupervised\nDA performs as well as batch DA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00503v1",
            "date": "2016-04-02 13:57:02+00:00",
            "title": "Discriminative Phrase Embedding for Paraphrase Identification",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This work, concerning paraphrase identification task, on one hand contributes\nto expanding deep learning embeddings to include continuous and discontinuous\nlinguistic phrases. On the other hand, it comes up with a new scheme TF-KLD-KNN\nto learn the discriminative weights of words and phrases specific to paraphrase\ntask, so that a weighted sum of embeddings can represent sentences more\neffectively. Based on these two innovations we get competitive state-of-the-art\nperformance on paraphrase identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00734v1",
            "date": "2016-04-04 03:58:31+00:00",
            "title": "Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks",
            "authors": [
                "Matthew Francis-Landau",
                "Greg Durrett",
                "Dan Klein"
            ],
            "abstract": "A key challenge in entity linking is making effective use of contextual\ninformation to disambiguate mentions that might refer to different entities in\ndifferent contexts. We present a model that uses convolutional neural networks\nto capture semantic correspondence between a mention's context and a proposed\ntarget entity. These convolutional networks operate at multiple granularities\nto exploit various kinds of topic information, and their rich parameterization\ngives them the capacity to learn which n-grams characterize different topics.\nWe combine these networks with a sparse linear model to achieve\nstate-of-the-art performance on multiple entity linking datasets, outperforming\nthe prior systems of Durrett and Klein (2014) and Nguyen et al. (2014).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00938v1",
            "date": "2016-04-04 16:33:15+00:00",
            "title": "Multi-Field Structural Decomposition for Question Answering",
            "authors": [
                "Tomasz Jurczyk",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents a precursory yet novel approach to the question answering\ntask using structural decomposition. Our system first generates linguistic\nstructures such as syntactic and semantic trees from text, decomposes them into\nmultiple fields, then indexes the terms in each field. For each question, it\ndecomposes the question into multiple fields, measures the relevance score of\neach field to the indexed ones, then ranks all documents by their relevance\nscores and weights associated with the fields, where the weights are learned\nthrough statistical modeling. Our final model gives an absolute improvement of\nover 40% to the baseline approach using simple search for detecting documents\ncontaining answers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01178v1",
            "date": "2016-04-05 08:50:27+00:00",
            "title": "Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks",
            "authors": [
                "Aliaksei Severyn",
                "Alessandro Moschitti"
            ],
            "abstract": "In this paper, we propose convolutional neural networks for learning an\noptimal representation of question and answer sentences. Their main aspect is\nthe use of relational information given by the matches between words from the\ntwo members of the pair. The matches are encoded as embeddings with additional\nparameters (dimensions), which are tuned by the network. These allows for\nbetter capturing interactions between questions and answers, resulting in a\nsignificant boost in accuracy. We test our models on two widely used answer\nsentence selection benchmarks. The results clearly show the effectiveness of\nour relational information, which allows our relatively simple network to\napproach the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01221v1",
            "date": "2016-04-05 11:34:11+00:00",
            "title": "Character-Level Neural Translation for Multilingual Media Monitoring in the SUMMA Project",
            "authors": [
                "Guntis Barzdins",
                "Steve Renals",
                "Didzis Gosko"
            ],
            "abstract": "The paper steps outside the comfort-zone of the traditional NLP tasks like\nautomatic speech recognition (ASR) and machine translation (MT) to addresses\ntwo novel problems arising in the automated multilingual news monitoring:\nsegmentation of the TV and radio program ASR transcripts into individual\nstories, and clustering of the individual stories coming from various sources\nand languages into storylines. Storyline clustering of stories covering the\nsame events is an essential task for inquisitorial media monitoring. We address\nthese two problems jointly by engaging the low-dimensional semantic\nrepresentation capabilities of the sequence to sequence neural translation\nmodels. To enable joint multi-task learning for multilingual neural translation\nof morphologically rich languages we replace the attention mechanism with the\nsliding-window mechanism and operate the sequence to sequence neural\ntranslation model on the character-level rather than on the word-level. The\nstory segmentation and storyline clustering problem is tackled by examining the\nlow-dimensional vectors produced as a side-product of the neural translation\nprocess. The results of this paper describe a novel approach to the automatic\nstory segmentation and storyline clustering problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01235v1",
            "date": "2016-04-05 12:42:31+00:00",
            "title": "A new TAG Formalism for Tamil and Parser Analytics",
            "authors": [
                "Vijay Krishna Menon",
                "S. Rajendran",
                "M. Anand Kumar",
                "K. P. Soman"
            ],
            "abstract": "Tree adjoining grammar (TAG) is specifically suited for morph rich and\nagglutinated languages like Tamil due to its psycho linguistic features and\nparse time dependency and morph resolution. Though TAG and LTAG formalisms have\nbeen known for about 3 decades, efforts on designing TAG Syntax for Tamil have\nnot been entirely successful due to the complexity of its specification and the\nrich morphology of Tamil language. In this paper we present a minimalistic TAG\nfor Tamil without much morphological considerations and also introduce a parser\nimplementation with some obvious variations from the XTAG system",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01278v1",
            "date": "2016-04-05 14:46:18+00:00",
            "title": "RIGA at SemEval-2016 Task 8: Impact of Smatch Extensions and Character-Level Neural Translation on AMR Parsing Accuracy",
            "authors": [
                "Guntis Barzdins",
                "Didzis Gosko"
            ],
            "abstract": "Two extensions to the AMR smatch scoring script are presented. The first\nextension com-bines the smatch scoring script with the C6.0 rule-based\nclassifier to produce a human-readable report on the error patterns frequency\nobserved in the scored AMR graphs. This first extension results in 4% gain over\nthe state-of-art CAMR baseline parser by adding to it a manually crafted\nwrapper fixing the identified CAMR parser errors. The second extension combines\na per-sentence smatch with an en-semble method for selecting the best AMR graph\namong the set of AMR graphs for the same sentence. This second modification\nau-tomatically yields further 0.4% gain when ap-plied to outputs of two\nnondeterministic AMR parsers: a CAMR+wrapper parser and a novel character-level\nneural translation AMR parser. For AMR parsing task the character-level neural\ntranslation attains surprising 7% gain over the carefully optimized word-level\nneural translation. Overall, we achieve smatch F1=62% on the SemEval-2016\nofficial scor-ing set and F1=67% on the LDC2015E86 test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01692v2",
            "date": "2016-04-06 16:58:35+00:00",
            "title": "An Ensemble Method to Produce High-Quality Word Embeddings (2016)",
            "authors": [
                "Robyn Speer",
                "Joshua Chin"
            ],
            "abstract": "A currently successful approach to computational semantics is to represent\nwords as embeddings in a machine-learned vector space. We present an ensemble\nmethod that combines embeddings produced by GloVe (Pennington et al., 2014) and\nword2vec (Mikolov et al., 2013) with structured knowledge from the semantic\nnetworks ConceptNet (Speer and Havasi, 2012) and PPDB (Ganitkevitch et al.,\n2013), merging their information into a common representation with a large,\nmultilingual vocabulary. The embeddings it produces achieve state-of-the-art\nperformance on many word-similarity evaluations. Its score of $\\rho = .596$ on\nan evaluation of rare words (Luong et al., 2013) is 16% higher than the\nprevious best known system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01904v2",
            "date": "2016-04-07 07:47:11+00:00",
            "title": "Neural Headline Generation with Sentence-wise Optimization",
            "authors": [
                "Ayana",
                "Shiqi Shen",
                "Yu Zhao",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Recently, neural models have been proposed for headline generation by\nlearning to map documents to headlines with recurrent neural networks.\nNevertheless, as traditional neural network utilizes maximum likelihood\nestimation for parameter optimization, it essentially constrains the expected\ntraining objective within word level rather than sentence level. Moreover, the\nperformance of model prediction significantly relies on training data\ndistribution. To overcome these drawbacks, we employ minimum risk training\nstrategy in this paper, which directly optimizes model parameters in sentence\nlevel with respect to evaluation metrics and leads to significant improvements\nfor headline generation. Experiment results show that our models outperforms\nstate-of-the-art systems on both English and Chinese headline generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02201v1",
            "date": "2016-04-08 00:16:35+00:00",
            "title": "Transfer Learning for Low-Resource Neural Machine Translation",
            "authors": [
                "Barret Zoph",
                "Deniz Yuret",
                "Jonathan May",
                "Kevin Knight"
            ],
            "abstract": "The encoder-decoder framework for neural machine translation (NMT) has been\nshown effective in large data scenarios, but is much less effective for\nlow-resource languages. We present a transfer learning method that\nsignificantly improves Bleu scores across a range of low-resource languages.\nOur key idea is to first train a high-resource language pair (the parent\nmodel), then transfer some of the learned parameters to the low-resource pair\n(the child model) to initialize and constrain training. Using our transfer\nlearning method we improve baseline NMT models by an average of 5.6 Bleu on\nfour low-resource language pairs. Ensembling and unknown word replacement add\nanother 2 Bleu which brings the NMT performance on low-resource machine\ntranslation close to a strong syntax based machine translation (SBMT) system,\nexceeding its performance on one language pair. Additionally, using the\ntransfer learning model for re-scoring, we can improve the SBMT system by an\naverage of 1.3 Bleu, improving the state-of-the-art on low-resource machine\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02612v1",
            "date": "2016-04-09 22:00:27+00:00",
            "title": "Fusing Audio, Textual and Visual Features for Sentiment Analysis of News Videos",
            "authors": [
                "Mois\u00e9s H. R. Pereira",
                "Fl\u00e1vio L. C. P\u00e1dua",
                "Adriano C. M. Pereira",
                "Fabr\u00edcio Benevenuto",
                "Daniel H. Dalip"
            ],
            "abstract": "This paper presents a novel approach to perform sentiment analysis of news\nvideos, based on the fusion of audio, textual and visual clues extracted from\ntheir contents. The proposed approach aims at contributing to the\nsemiodiscoursive study regarding the construction of the ethos (identity) of\nthis media universe, which has become a central part of the modern-day lives of\nmillions of people. To achieve this goal, we apply state-of-the-art\ncomputational methods for (1) automatic emotion recognition from facial\nexpressions, (2) extraction of modulations in the participants' speeches and\n(3) sentiment analysis from the closed caption associated to the videos of\ninterest. More specifically, we compute features, such as, visual intensities\nof recognized emotions, field sizes of participants, voicing probability, sound\nloudness, speech fundamental frequencies and the sentiment scores (polarities)\nfrom text sentences in the closed caption. Experimental results with a dataset\ncontaining 520 annotated news videos from three Brazilian and one American\npopular TV newscasts show that our approach achieves an accuracy of up to 84%\nin the sentiments (tension levels) classification task, thus demonstrating its\nhigh potential to be used by media analysts in several applications,\nespecially, in the journalistic domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02843v1",
            "date": "2016-04-11 08:56:58+00:00",
            "title": "Method of Tibetan Person Knowledge Extraction",
            "authors": [
                "Yuan Sun",
                "Zhen Zhu"
            ],
            "abstract": "Person knowledge extraction is the foundation of the Tibetan knowledge graph\nconstruction, which provides support for Tibetan question answering system,\ninformation retrieval, information extraction and other researches, and\npromotes national unity and social stability. This paper proposes a SVM and\ntemplate based approach to Tibetan person knowledge extraction. Through\nconstructing the training corpus, we build the templates based the shallow\nparsing analysis of Tibetan syntactic, semantic features and verbs. Using the\ntraining corpus, we design a hierarchical SVM classifier to realize the entity\nknowledge extraction. Finally, experimental results prove the method has\ngreater improvement in Tibetan person knowledge extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02993v2",
            "date": "2016-04-11 15:21:05+00:00",
            "title": "Using Sentence-Level LSTM Language Models for Script Inference",
            "authors": [
                "Karl Pichotta",
                "Raymond J. Mooney"
            ],
            "abstract": "There is a small but growing body of research on statistical scripts, models\nof event sequences that allow probabilistic inference of implicit events from\ndocuments. These systems operate on structured verb-argument events produced by\nan NLP pipeline. We compare these systems with recent Recurrent Neural Net\nmodels that directly operate on raw tokens to predict sentences, finding the\nlatter to be roughly comparable to the former in terms of predicting missing\nevents in documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03035v1",
            "date": "2016-04-11 17:15:34+00:00",
            "title": "Learning Global Features for Coreference Resolution",
            "authors": [
                "Sam Wiseman",
                "Alexander M. Rush",
                "Stuart M. Shieber"
            ],
            "abstract": "There is compelling evidence that coreference prediction would benefit from\nmodeling global information about entity-clusters. Yet, state-of-the-art\nperformance can be achieved with systems treating each mention prediction\nindependently, which we attribute to the inherent difficulty of crafting\ninformative cluster-level features. We instead propose to use recurrent neural\nnetworks (RNNs) to learn latent, global representations of entity clusters\ndirectly from their mentions. We show that such representations are especially\nuseful for the prediction of pronominal mentions, and can be incorporated into\nan end-to-end coreference system that outperforms the state of the art without\nrequiring any additional search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03136v1",
            "date": "2016-04-11 20:24:52+00:00",
            "title": "Shallow Parsing Pipeline for Hindi-English Code-Mixed Social Media Text",
            "authors": [
                "Arnav Sharma",
                "Sakshi Gupta",
                "Raveesh Motlani",
                "Piyush Bansal",
                "Manish Srivastava",
                "Radhika Mamidi",
                "Dipti M. Sharma"
            ],
            "abstract": "In this study, the problem of shallow parsing of Hindi-English code-mixed\nsocial media text (CSMT) has been addressed. We have annotated the data,\ndeveloped a language identifier, a normalizer, a part-of-speech tagger and a\nshallow parser. To the best of our knowledge, we are the first to attempt\nshallow parsing on CSMT. The pipeline developed has been made available to the\nresearch community with the goal of enabling better text analysis of Hindi\nEnglish CSMT. The pipeline is accessible at http://bit.ly/csmt-parser-api .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03209v1",
            "date": "2016-04-12 02:34:00+00:00",
            "title": "Disfluency Detection using a Bidirectional LSTM",
            "authors": [
                "Vicky Zayats",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce a new approach for disfluency detection using a Bidirectional\nLong-Short Term Memory neural network (BLSTM). In addition to the word\nsequence, the model takes as input pattern match features that were developed\nto reduce sensitivity to vocabulary size in training, which lead to improved\nperformance over the word sequence alone. The BLSTM takes advantage of explicit\nrepair states in addition to the standard reparandum states. The final output\nleverages integer linear programming to incorporate constraints of disfluency\nstructure. In experiments on the Switchboard corpus, the model achieves\nstate-of-the-art performance for both the standard disfluency detection task\nand the correction detection task. Analysis shows that the model has better\ndetection of non-repetition disfluencies, which tend to be much harder to\ndetect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03357v1",
            "date": "2016-04-12 11:57:05+00:00",
            "title": "Improving sentence compression by learning to predict gaze",
            "authors": [
                "Sigrid Klerke",
                "Yoav Goldberg",
                "Anders S\u00f8gaard"
            ],
            "abstract": "We show how eye-tracking corpora can be used to improve sentence compression\nmodels, presenting a novel multi-task learning algorithm based on multi-layer\nLSTMs. We obtain performance competitive with or better than state-of-the-art\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04677v1",
            "date": "2016-04-16 01:49:09+00:00",
            "title": "Sentence-Level Grammatical Error Identification as Sequence-to-Sequence Correction",
            "authors": [
                "Allen Schmaltz",
                "Yoon Kim",
                "Alexander M. Rush",
                "Stuart M. Shieber"
            ],
            "abstract": "We demonstrate that an attention-based encoder-decoder model can be used for\nsentence-level grammatical error identification for the Automated Evaluation of\nScientific Writing (AESW) Shared Task 2016. The attention-based encoder-decoder\nmodels can be used for the generation of corrections, in addition to error\nidentification, which is of interest for certain end-user applications. We show\nthat a character-based encoder-decoder model is particularly effective,\noutperforming other results on the AESW Shared Task on its own, and showing\ngains over a word-based counterpart. Our final model--a combination of three\ncharacter-based encoder-decoder models, one word-based encoder-decoder model,\nand a sentence-level CNN--is the highest performing system on the AESW 2016\nbinary prediction Shared Task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04873v1",
            "date": "2016-04-17 14:07:34+00:00",
            "title": "From Incremental Meaning to Semantic Unit (phrase by phrase)",
            "authors": [
                "Andreas Scherbakov",
                "Ekaterina Vylomova",
                "Fei Liu",
                "Timothy Baldwin"
            ],
            "abstract": "This paper describes an experimental approach to Detection of Minimal\nSemantic Units and their Meaning (DiMSUM), explored within the framework of\nSemEval 2016 Task 10. The approach is primarily based on a combination of word\nembeddings and parserbased features, and employs unidirectional incremental\ncomputation of compositional embeddings for multiword expressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05073v1",
            "date": "2016-04-18 10:27:49+00:00",
            "title": "Speed-Constrained Tuning for Statistical Machine Translation Using Bayesian Optimization",
            "authors": [
                "Daniel Beck",
                "Adri\u00e0 de Gispert",
                "Gonzalo Iglesias",
                "Aurelien Waite",
                "Bill Byrne"
            ],
            "abstract": "We address the problem of automatically finding the parameters of a\nstatistical machine translation system that maximize BLEU scores while ensuring\nthat decoding speed exceeds a minimum value. We propose the use of Bayesian\nOptimization to efficiently tune the speed-related decoding parameters by\neasily incorporating speed as a noisy constraint function. The obtained\nparameter values are guaranteed to satisfy the speed constraint with an\nassociated confidence margin. Across three language pairs and two speed\nconstraint values, we report overall optimization time reduction compared to\ngrid and random search. We also show that Bayesian Optimization can decouple\nspeed and BLEU measurements, resulting in a further reduction of overall\noptimization time as speed is measured over a small subset of sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05372v1",
            "date": "2016-04-18 22:56:13+00:00",
            "title": "Clustering Comparable Corpora of Russian and Ukrainian Academic Texts: Word Embeddings and Semantic Fingerprints",
            "authors": [
                "Andrey Kutuzov",
                "Mikhail Kopotev",
                "Tatyana Sviridenko",
                "Lyubov Ivanova"
            ],
            "abstract": "We present our experience in applying distributional semantics (neural word\nembeddings) to the problem of representing and clustering documents in a\nbilingual comparable corpus. Our data is a collection of Russian and Ukrainian\nacademic texts, for which topics are their academic fields. In order to build\nlanguage-independent semantic representations of these documents, we train\nneural distributional models on monolingual corpora and learn the optimal\nlinear transformation of vectors from one language to another. The resulting\nvectors are then used to produce `semantic fingerprints' of documents, serving\nas input to a clustering algorithm.\n  The presented method is compared to several baselines including `orthographic\ntranslation' with Levenshtein edit distance and outperforms them by a large\nmargin. We also show that language-independent `semantic fingerprints' are\nsuperior to multi-lingual clustering algorithms proposed in the previous work,\nat the same time requiring less linguistic resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05499v1",
            "date": "2016-04-19 10:08:49+00:00",
            "title": "Exploring Segment Representations for Neural Segmentation Models",
            "authors": [
                "Yijia Liu",
                "Wanxiang Che",
                "Jiang Guo",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Many natural language processing (NLP) tasks can be generalized into\nsegmentation problem. In this paper, we combine semi-CRF with neural network to\nsolve NLP segmentation tasks. Our model represents a segment both by composing\nthe input units and embedding the entire segment. We thoroughly study different\ncomposition functions and different segment embeddings. We conduct extensive\nexperiments on two typical segmentation tasks: named entity recognition (NER)\nand Chinese word segmentation (CWS). Experimental results show that our neural\nsemi-CRF model benefits from representing the entire segment and achieves the\nstate-of-the-art performance on CWS benchmark dataset and competitive results\non the CoNLL03 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05519v3",
            "date": "2016-04-19 11:09:20+00:00",
            "title": "M$^2$S-Net: Multi-Modal Similarity Metric Learning based Deep Convolutional Network for Answer Selection",
            "authors": [
                "Lingxun Meng",
                "Yan Li"
            ],
            "abstract": "Recent works using artificial neural networks based on distributed word\nrepresentation greatly boost performance on various natural language processing\ntasks, especially the answer selection problem. Nevertheless, most of the\nprevious works used deep learning methods (like LSTM-RNN, CNN, etc.) only to\ncapture semantic representation of each sentence separately, without\nconsidering the interdependence between each other. In this paper, we propose a\nnovel end-to-end learning framework which constitutes deep convolutional neural\nnetwork based on multi-modal similarity metric learning (M$^2$S-Net) on\npairwise tokens. The proposed model demonstrates its performance by surpassing\nprevious state-of-the-art systems on the answer selection benchmark, i.e.,\nTREC-QA dataset, in both MAP and MRR metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05525v1",
            "date": "2016-04-19 11:39:53+00:00",
            "title": "An Attentive Neural Architecture for Fine-grained Entity Type Classification",
            "authors": [
                "Sonse Shimaoka",
                "Pontus Stenetorp",
                "Kentaro Inui",
                "Sebastian Riedel"
            ],
            "abstract": "In this work we propose a novel attention-based neural network model for the\ntask of fine-grained entity type classification that unlike previously proposed\nmodels recursively composes representations of entity mention contexts. Our\nmodel achieves state-of-the-art performance with 74.94% loose micro F1-score on\nthe well-established FIGER dataset, a relative improvement of 2.59%. We also\ninvestigate the behavior of the attention mechanism of our model and observe\nthat it can learn contextual linguistic expressions that indicate the\nfine-grained category memberships of an entity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05529v3",
            "date": "2016-04-19 11:53:09+00:00",
            "title": "Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss",
            "authors": [
                "Barbara Plank",
                "Anders S\u00f8gaard",
                "Yoav Goldberg"
            ],
            "abstract": "Bidirectional long short-term memory (bi-LSTM) networks have recently proven\nsuccessful for various NLP sequence modeling tasks, but little is known about\ntheir reliance to input representations, target languages, data set size, and\nlabel noise. We address these issues and evaluate bi-LSTMs with word,\ncharacter, and unicode byte embeddings for POS tagging. We compare bi-LSTMs to\ntraditional POS taggers across languages and data sizes. We also present a\nnovel bi-LSTM model, which combines the POS tagging loss function with an\nauxiliary loss function that accounts for rare words. The model obtains\nstate-of-the-art performance across 22 languages, and works especially well for\nmorphologically complex languages. Our analysis suggests that bi-LSTMs are less\nsensitive to training data size and label corruptions (at small noise levels)\nthan previously assumed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05559v1",
            "date": "2016-04-18 18:51:18+00:00",
            "title": "Efficient Calculation of Bigram Frequencies in a Corpus of Short Texts",
            "authors": [
                "Melvyn Drag",
                "Gauthaman Vasudevan"
            ],
            "abstract": "We show that an efficient and popular method for calculating bigram\nfrequencies is unsuitable for bodies of short texts and offer a simple\nalternative. Our method has the same computational complexity as the old method\nand offers an exact count instead of an approximation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05747v1",
            "date": "2016-04-19 20:59:32+00:00",
            "title": "Syntactic and semantic classification of verb arguments using dependency-based and rich semantic features",
            "authors": [
                "Francesco Elia"
            ],
            "abstract": "Corpus Pattern Analysis (CPA) has been the topic of Semeval 2015 Task 15,\naimed at producing a system that can aid lexicographers in their efforts to\nbuild a dictionary of meanings for English verbs using the CPA annotation\nprocess. CPA parsing is one of the subtasks which this annotation process is\nmade of and it is the focus of this report. A supervised machine-learning\napproach has been implemented, in which syntactic features derived from parse\ntrees and semantic features derived from WordNet and word embeddings are used.\nIt is shown that this approach performs well, even with the data sparsity\nissues that characterize the dataset, and can obtain better results than other\nsystem by a margin of about 4% f-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05800v3",
            "date": "2016-04-20 03:03:12+00:00",
            "title": "A Deep Neural Network for Chinese Zero Pronoun Resolution",
            "authors": [
                "Qingyu Yin",
                "Weinan Zhang",
                "Yu Zhang",
                "Ting Liu"
            ],
            "abstract": "Existing approaches for Chinese zero pronoun resolution overlook semantic\ninformation. This is because zero pronouns have no descriptive information,\nwhich results in difficulty in explicitly capturing their semantic similarities\nwith antecedents. Moreover, when dealing with candidate antecedents,\ntraditional systems simply take advantage of the local information of a single\ncandidate antecedent while failing to consider the underlying information\nprovided by the other candidates from a global perspective. To address these\nweaknesses, we propose a novel zero pronoun-specific neural network, which is\ncapable of representing zero pronouns by utilizing the contextual information\nat the semantic level. In addition, when dealing with candidate antecedents, a\ntwo-level candidate encoder is employed to explicitly capture both the local\nand global information of candidate antecedents. We conduct experiments on the\nChinese portion of the OntoNotes 5.0 corpus. Experimental results show that our\napproach substantially outperforms the state-of-the-art method in various\nexperimental settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06045v7",
            "date": "2016-04-20 18:06:49+00:00",
            "title": "Dialog-based Language Learning",
            "authors": [
                "Jason Weston"
            ],
            "abstract": "A long-term goal of machine learning research is to build an intelligent\ndialog agent. Most research in natural language understanding has focused on\nlearning from fixed training sets of labeled data, with supervision either at\nthe word level (tagging, parsing tasks) or sentence level (question answering,\nmachine translation). This kind of supervision is not realistic of how humans\nlearn, where language is both learned by, and used for, communication. In this\nwork, we study dialog-based language learning, where supervision is given\nnaturally and implicitly in the response of the dialog partner during the\nconversation. We study this setup in two domains: the bAbI dataset of (Weston\net al., 2015) and large-scale question answering from (Dodge et al., 2015). We\nevaluate a set of baseline learning strategies on these tasks, and show that a\nnovel model incorporating predictive lookahead is a promising approach for\nlearning from a teacher's response. In particular, a surprising result is that\nit can learn to answer questions correctly without any reward-based supervision\nat all.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06113v1",
            "date": "2016-04-20 20:10:41+00:00",
            "title": "Speaker Cluster-Based Speaker Adaptive Training for Deep Neural Network Acoustic Modeling",
            "authors": [
                "Wei Chu",
                "Ruxin Chen"
            ],
            "abstract": "A speaker cluster-based speaker adaptive training (SAT) method under deep\nneural network-hidden Markov model (DNN-HMM) framework is presented in this\npaper. During training, speakers that are acoustically adjacent to each other\nare hierarchically clustered using an i-vector based distance metric. DNNs with\nspeaker dependent layers are then adaptively trained for each cluster of\nspeakers. Before decoding starts, an unseen speaker in test set is matched to\nthe closest speaker cluster through comparing i-vector based distances. The\npreviously trained DNN of the matched speaker cluster is used for decoding\nutterances of the test speaker. The performance of the proposed method on a\nlarge vocabulary spontaneous speech recognition task is evaluated on a training\nset of with 1500 hours of speech, and a test set of 24 speakers with 1774\nutterances. Comparing to a speaker independent DNN with a baseline word error\nrate of 11.6%, a relative 6.8% reduction in word error rate is observed from\nthe proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06274v2",
            "date": "2016-04-21 12:25:04+00:00",
            "title": "Chinese Song Iambics Generation with Neural Attention-based Model",
            "authors": [
                "Qixin Wang",
                "Tianyi Luo",
                "Dong Wang",
                "Chao Xing"
            ],
            "abstract": "Learning and generating Chinese poems is a charming yet challenging task.\nTraditional approaches involve various language modeling and machine\ntranslation techniques, however, they perform not as well when generating poems\nwith complex pattern constraints, for example Song iambics, a famous type of\npoems that involve variable-length sentences and strict rhythmic patterns. This\npaper applies the attention-based sequence-to-sequence model to generate\nChinese Song iambics. Specifically, we encode the cue sentences by a\nbi-directional Long-Short Term Memory (LSTM) model and then predict the entire\niambic with the information provided by the encoder, in the form of an\nattention-based LSTM that can regularize the generation process by the fine\nstructure of the input cues. Several techniques are investigated to improve the\nmodel, including global context integration, hybrid style training, character\nvector initialization and adaptation. Both the automatic and subjective\nevaluation results show that our model indeed can learn the complex structural\nand rhythmic patterns of Song iambics, and the generation is rather successful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06285v1",
            "date": "2016-04-21 12:55:29+00:00",
            "title": "A Novel Approach to Dropped Pronoun Translation",
            "authors": [
                "Longyue Wang",
                "Zhaopeng Tu",
                "Xiaojun Zhang",
                "Hang Li",
                "Andy Way",
                "Qun Liu"
            ],
            "abstract": "Dropped Pronouns (DP) in which pronouns are frequently dropped in the source\nlanguage but should be retained in the target language are challenge in machine\ntranslation. In response to this problem, we propose a semi-supervised approach\nto recall possibly missing pronouns in the translation. Firstly, we build\ntraining data for DP generation in which the DPs are automatically labelled\naccording to the alignment information from a parallel corpus. Secondly, we\nbuild a deep learning-based DP generator for input sentences in decoding when\nno corresponding references exist. More specifically, the generation is\ntwo-phase: (1) DP position detection, which is modeled as a sequential\nlabelling task with recurrent neural networks; and (2) DP prediction, which\nemploys a multilayer perceptron with rich features. Finally, we integrate the\nabove outputs into our translation system to recall missing pronouns by both\nextracting rules from the DP-labelled training data and translating the\nDP-generated input sentences. Experimental results show that our approach\nachieves a significant improvement of 1.58 BLEU points in translation\nperformance with 66% F-score for DP generation accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06361v1",
            "date": "2016-04-21 15:39:01+00:00",
            "title": "Row-less Universal Schema",
            "authors": [
                "Patrick Verga",
                "Andrew McCallum"
            ],
            "abstract": "Universal schema jointly embeds knowledge bases and textual patterns to\nreason about entities and relations for automatic knowledge base construction\nand information extraction. In the past, entity pairs and relations were\nrepresented as learned vectors with compatibility determined by a scoring\nfunction, limiting generalization to unseen text patterns and entities.\nRecently, 'column-less' versions of Universal Schema have used compositional\npattern encoders to generalize to all text patterns. In this work we take the\nnext step and propose a 'row-less' model of universal schema, removing explicit\nentity pair representations. Instead of learning vector representations for\neach entity pair in our training set, we treat an entity pair as a function of\nits relation types. In experimental results on the FB15k-237 benchmark we\ndemonstrate that we can match the performance of a comparable model with\nexplicit entity pair representations using a model of attention over relation\ntypes. We further demonstrate that the model per- forms with nearly the same\naccuracy on entity pairs never seen during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06583v1",
            "date": "2016-04-22 09:19:07+00:00",
            "title": "SweLL on the rise: Swedish Learner Language corpus for European Reference Level studies",
            "authors": [
                "Elena Volodina",
                "Ildik\u00f3 Pil\u00e1n",
                "Ingegerd Enstr\u00f6m",
                "Lorena Llozhi",
                "Peter Lundkvist",
                "Gunl\u00f6g Sundberg",
                "Monica Sandell"
            ],
            "abstract": "We present a new resource for Swedish, SweLL, a corpus of Swedish Learner\nessays linked to learners' performance according to the Common European\nFramework of Reference (CEFR). SweLL consists of three subcorpora - SpIn,\nSW1203 and Tisus, collected from three different educational establishments.\nThe common metadata for all subcorpora includes age, gender, native languages,\ntime of residence in Sweden, type of written task. Depending on the subcorpus,\nlearner texts may contain additional information, such as text genres, topics,\ngrades. Five of the six CEFR levels are represented in the corpus: A1, A2, B1,\nB2 and C1 comprising in total 339 essays. C2 level is not included since\ncourses at C2 level are not offered. The work flow consists of collection of\nessays and permits, essay digitization and registration, meta-data annotation,\nautomatic linguistic annotation. Inter-rater agreement is presented on the\nbasis of SW1203 subcorpus. The work on SweLL is still ongoing with more than\n100 essays waiting in the pipeline. This article both describes the resource\nand the \"how-to\" behind the compilation of SweLL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06648v1",
            "date": "2016-04-22 13:25:14+00:00",
            "title": "Automatic verbal aggression detection for Russian and American imageboards",
            "authors": [
                "Denis Gordeev"
            ],
            "abstract": "The problem of aggression for Internet communities is rampant. Anonymous\nforums usually called imageboards are notorious for their aggressive and\ndeviant behaviour even in comparison with other Internet communities. This\nstudy is aimed at studying ways of automatic detection of verbal expression of\naggression for the most popular American (4chan.org) and Russian (2ch.hk)\nimageboards. A set of 1,802,789 messages was used for this study. The machine\nlearning algorithm word2vec was applied to detect the state of aggression. A\ndecent result is obtained for English (88%), the results for Russian are yet to\nbe improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06650v1",
            "date": "2016-04-22 13:33:08+00:00",
            "title": "Detecting state of aggression in sentences using CNN",
            "authors": [
                "Rodmonga Potapova",
                "Denis Gordeev"
            ],
            "abstract": "In this article we study verbal expression of aggression and its detection\nusing machine learning and neural networks methods. We test our results using\nour corpora of messages from anonymous imageboards. We also compare Random\nforest classifier with convolutional neural network for \"Movie reviews with one\nsentence per review\" corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06896v2",
            "date": "2016-04-23 11:53:43+00:00",
            "title": "Why and How to Pay Different Attention to Phrase Alignments of Different Intensities",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This work studies comparatively two typical sentence pair classification\ntasks: textual entailment (TE) and answer selection (AS), observing that phrase\nalignments of different intensities contribute differently in these tasks. We\naddress the problems of identifying phrase alignments of flexible granularity\nand pooling alignments of different intensities for these tasks. Examples for\nflexible granularity are alignments between two single words, between a single\nword and a phrase and between a short phrase and a long phrase. By intensity we\nroughly mean the degree of match, it ranges from identity over surface-form\nco-occurrence, rephrasing and other semantic relatedness to unrelated words as\nin lots of parenthesis text. Prior work (i) has limitations in phrase\ngeneration and representation, or (ii) conducts alignment at word and phrase\nlevels by handcrafted features or (iii) utilizes a single attention mechanism\nover alignment intensities without considering the characteristics of specific\ntasks, which limits the system's effectiveness across tasks. We propose an\narchitecture based on Gated Recurrent Unit that supports (i) representation\nlearning of phrases of arbitrary granularity and (ii) task-specific focusing of\nphrase alignments between two sentences by attention pooling. Experimental\nresults on TE and AS match our observation and are state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.07370v2",
            "date": "2016-04-25 19:19:04+00:00",
            "title": "Parsing Argumentation Structures in Persuasive Essays",
            "authors": [
                "Christian Stab",
                "Iryna Gurevych"
            ],
            "abstract": "In this article, we present a novel approach for parsing argumentation\nstructures. We identify argument components using sequence labeling at the\ntoken level and apply a new joint model for detecting argumentation structures.\nThe proposed model globally optimizes argument component types and\nargumentative relations using integer linear programming. We show that our\nmodel considerably improves the performance of base classifiers and\nsignificantly outperforms challenging heuristic baselines. Moreover, we\nintroduce a novel corpus of persuasive essays annotated with argumentation\nstructures. We show that our annotation scheme and annotation guidelines\nsuccessfully guide human annotators to substantial agreement. This corpus and\nthe annotation guidelines are freely available for ensuring reproducibility and\nto encourage future research in computational argumentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.07809v1",
            "date": "2016-04-26 19:31:08+00:00",
            "title": "Entities as topic labels: Improving topic interpretability and evaluability combining Entity Linking and Labeled LDA",
            "authors": [
                "Federico Nanni",
                "Pablo Ruiz Fabo"
            ],
            "abstract": "In order to create a corpus exploration method providing topics that are\neasier to interpret than standard LDA topic models, here we propose combining\ntwo techniques called Entity linking and Labeled LDA. Our method identifies in\nan ontology a series of descriptive labels for each document in a corpus. Then\nit generates a specific topic for each label. Having a direct relation between\ntopics and labels makes interpretation easier; using an ontology as background\nknowledge limits label ambiguity. As our topics are described with a limited\nnumber of clear-cut labels, they promote interpretability, and this may help\nquantitative evaluation. We illustrate the potential of the approach by\napplying it in order to define the most relevant topics addressed by each party\nin the European Parliament's fifth mandate (1999-2004).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08120v1",
            "date": "2016-04-27 15:52:18+00:00",
            "title": "Extracting Temporal and Causal Relations between Events",
            "authors": [
                "Paramita Mirza"
            ],
            "abstract": "Structured information resulting from temporal information processing is\ncrucial for a variety of natural language processing tasks, for instance to\ngenerate timeline summarization of events from news documents, or to answer\ntemporal/causal-related questions about some events. In this thesis we present\na framework for an integrated temporal and causal relation extraction system.\nWe first develop a robust extraction component for each type of relations, i.e.\ntemporal order and causality. We then combine the two extraction components\ninto an integrated relation extraction system, CATENA---CAusal and Temporal\nrelation Extraction from NAtural language texts---, by utilizing the\npresumption about event precedence in causality, that causing events must\nhappened BEFORE resulting events. Several resources and techniques to improve\nour relation extraction systems are also discussed, including word embeddings\nand training data expansion. Finally, we report our adaptation efforts of\ntemporal information processing for languages other than English, namely\nItalian and Indonesian.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08242v2",
            "date": "2016-04-27 21:00:03+00:00",
            "title": "The IBM 2016 English Conversational Telephone Speech Recognition System",
            "authors": [
                "George Saon",
                "Tom Sercu",
                "Steven Rennie",
                "Hong-Kwang J. Kuo"
            ],
            "abstract": "We describe a collection of acoustic and language modeling techniques that\nlowered the word error rate of our English conversational telephone LVCSR\nsystem to a record 6.6% on the Switchboard subset of the Hub5 2000 evaluation\ntestset. On the acoustic side, we use a score fusion of three strong models:\nrecurrent nets with maxout activations, very deep convolutional nets with 3x3\nkernels, and bidirectional long short-term memory nets which operate on FMLLR\nand i-vector features. On the language modeling side, we use an updated model\n\"M\" and hierarchical neural network LMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08561v1",
            "date": "2016-04-28 19:10:47+00:00",
            "title": "Comparing Fifty Natural Languages and Twelve Genetic Languages Using Word Embedding Language Divergence (WELD) as a Quantitative Measure of Language Distance",
            "authors": [
                "Ehsaneddin Asgari",
                "Mohammad R. K. Mofrad"
            ],
            "abstract": "We introduce a new measure of distance between languages based on word\nembedding, called word embedding language divergence (WELD). WELD is defined as\ndivergence between unified similarity distribution of words between languages.\nUsing such a measure, we perform language comparison for fifty natural\nlanguages and twelve genetic languages. Our natural language dataset is a\ncollection of sentence-aligned parallel corpora from bible translations for\nfifty languages spanning a variety of language families. Although we use\nparallel corpora, which guarantees having the same content in all languages,\ninterestingly in many cases languages within the same family cluster together.\nIn addition to natural languages, we perform language comparison for the coding\nregions in the genomes of 12 different organisms (4 plants, 6 animals, and two\nhuman subjects). Our result confirms a significant high-level difference in the\ngenetic language model of humans/animals versus plants. The proposed method is\na step toward defining a quantitative measure of similarity between languages,\nwith applications in languages classification, genre identification, dialect\nidentification, and evaluation of translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08633v2",
            "date": "2016-04-28 22:09:49+00:00",
            "title": "Word Ordering Without Syntax",
            "authors": [
                "Allen Schmaltz",
                "Alexander M. Rush",
                "Stuart M. Shieber"
            ],
            "abstract": "Recent work on word ordering has argued that syntactic structure is\nimportant, or even required, for effectively recovering the order of a\nsentence. We find that, in fact, an n-gram language model with a simple\nheuristic gives strong results on this task. Furthermore, we show that a long\nshort-term memory (LSTM) language model is even more effective at recovering\norder, with our basic model outperforming a state-of-the-art syntactic model by\n11.5 BLEU points. Additional data and larger beams yield further gains, at the\nexpense of training and search time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08672v2",
            "date": "2016-04-29 02:44:02+00:00",
            "title": "Distance Metric Learning for Aspect Phrase Grouping",
            "authors": [
                "Shufeng Xiong",
                "Yue Zhang",
                "Donghong Ji",
                "Yinxia Lou"
            ],
            "abstract": "Aspect phrase grouping is an important task in aspect-level sentiment\nanalysis. It is a challenging problem due to polysemy and context dependency.\nWe propose an Attention-based Deep Distance Metric Learning (ADDML) method, by\nconsidering aspect phrase representation as well as context representation.\nFirst, leveraging the characteristics of the review text, we automatically\ngenerate aspect phrase sample pairs for distant supervision. Second, we feed\nword embeddings of aspect phrases and their contexts into an attention-based\nneural network to learn feature representation of contexts. Both aspect phrase\nembedding and context embedding are used to learn a deep feature subspace for\nmeasure the distances between aspect phrases for K-means clustering.\nExperiments on four review datasets show that the proposed method outperforms\nstate-of-the-art strong baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.00090v3",
            "date": "2016-04-30 10:48:29+00:00",
            "title": "Response Selection with Topic Clues for Retrieval-based Chatbots",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "We consider incorporating topic information into message-response matching to\nboost responses with rich content in retrieval-based chatbots. To this end, we\npropose a topic-aware convolutional neural tensor network (TACNTN). In TACNTN,\nmatching between a message and a response is not only conducted between a\nmessage vector and a response vector generated by convolutional neural\nnetworks, but also leverages extra topic information encoded in two topic\nvectors. The two topic vectors are linear combinations of topic words of the\nmessage and the response respectively, where the topic words are obtained from\na pre-trained LDA model and their weights are determined by themselves as well\nas the message vector and the response vector. The message vector, the response\nvector, and the two topic vectors are fed to neural tensors to calculate a\nmatching score. Empirical study on a public data set and a human annotated data\nset shows that TACNTN can significantly outperform state-of-the-art methods for\nmessage-response matching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.00482v3",
            "date": "2016-05-02 13:41:38+00:00",
            "title": "Compositional Sentence Representation from Character within Large Context Text",
            "authors": [
                "Geonmin Kim",
                "Hwaran Lee",
                "Jisu Choi",
                "Soo-young Lee"
            ],
            "abstract": "This paper describes a Hierarchical Composition Recurrent Network (HCRN)\nconsisting of a 3-level hierarchy of compositional models: character, word and\nsentence. This model is designed to overcome two problems of representing a\nsentence on the basis of a constituent word sequence. The first is a\ndata-sparsity problem in word embedding, and the other is a no usage of\ninter-sentence dependency. In the HCRN, word representations are built from\ncharacters, thus resolving the data-sparsity problem, and inter-sentence\ndependency is embedded into sentence representation at the level of sentence\ncomposition. We adopt a hierarchy-wise learning scheme in order to alleviate\nthe optimization difficulties of learning deep hierarchical recurrent network\nin end-to-end fashion. The HCRN was quantitatively and qualitatively evaluated\non a dialogue act classification task. Especially, sentence representations\nwith an inter-sentence dependency are able to capture both implicit and\nexplicit semantics of sentence, significantly improving performance. In the\nend, the HCRN achieved state-of-the-art performance with a test error rate of\n22.7% for dialogue act classification on the SWBD-DAMSL database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01655v1",
            "date": "2016-05-05 17:07:54+00:00",
            "title": "Stance and Sentiment in Tweets",
            "authors": [
                "Saif M. Mohammad",
                "Parinaz Sobhani",
                "Svetlana Kiritchenko"
            ],
            "abstract": "We can often detect from a person's utterances whether he/she is in favor of\nor against a given target entity -- their stance towards the target. However, a\nperson may express the same stance towards a target by using negative or\npositive language. Here for the first time we present a dataset of\ntweet--target pairs annotated for both stance and sentiment. The targets may or\nmay not be referred to in the tweets, and they may or may not be the target of\nopinion in the tweets. Partitions of this dataset were used as training and\ntest sets in a SemEval-2016 shared task competition. We propose a simple stance\ndetection system that outperforms submissions from all 19 teams that\nparticipated in the shared task. Additionally, access to both stance and\nsentiment annotations allows us to explore several research questions. We show\nthat while knowing the sentiment expressed by a tweet is beneficial for stance\nclassification, it alone is not sufficient. Finally, we use additional\nunlabeled data through distant supervision techniques and word embeddings to\nfurther improve stance classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01744v1",
            "date": "2016-05-05 20:11:57+00:00",
            "title": "Improving Automated Patent Claim Parsing: Dataset, System, and Experiments",
            "authors": [
                "Mengke Hu",
                "David Cinciruk",
                "John MacLaren Walsh"
            ],
            "abstract": "Off-the-shelf natural language processing software performs poorly when\nparsing patent claims owing to their use of irregular language relative to the\ncorpora built from news articles and the web typically utilized to train this\nsoftware. Stopping short of the extensive and expensive process of accumulating\na large enough dataset to completely retrain parsers for patent claims, a\nmethod of adapting existing natural language processing software towards patent\nclaims via forced part of speech tag correction is proposed. An Amazon\nMechanical Turk collection campaign organized to generate a public corpus to\ntrain such an improved claim parsing system is discussed, identifying lessons\nlearned during the campaign that can be of use in future NLP dataset collection\ncampaigns with AMT. Experiments utilizing this corpus and other patent claim\nsets measure the parsing performance improvement garnered via the claim parsing\nsystem. Finally, the utility of the improved claim parsing system within other\npatent processing applications is demonstrated via experiments showing improved\nautomated patent subject classification when the new claim parsing system is\nutilized to generate the features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01845v1",
            "date": "2016-05-06 07:30:53+00:00",
            "title": "Detecting Context Dependence in Exercise Item Candidates Selected from Corpora",
            "authors": [
                "Ildik\u00f3 Pil\u00e1n"
            ],
            "abstract": "We explore the factors influencing the dependence of single sentences on\ntheir larger textual context in order to automatically identify candidate\nsentences for language learning exercises from corpora which are presentable in\nisolation. An in-depth investigation of this question has not been previously\ncarried out. Understanding this aspect can contribute to a more efficient\nselection of candidate sentences which, besides reducing the time required for\nitem writing, can also ensure a higher degree of variability and authenticity.\nWe present a set of relevant aspects collected based on the qualitative\nanalysis of a smaller set of context-dependent corpus example sentences.\nFurthermore, we implemented a rule-based algorithm using these criteria which\nachieved an average precision of 0.76 for the identification of different\nissues related to context dependence. The method has also been evaluated\nempirically where 80% of the sentences in which our system did not detect\ncontext-dependent elements were also considered context-independent by human\nraters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02019v1",
            "date": "2016-05-06 18:13:18+00:00",
            "title": "Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec",
            "authors": [
                "Christopher E Moody"
            ],
            "abstract": "Distributed dense word vectors have been shown to be effective at capturing\ntoken-level semantic and syntactic regularities in language, while topic models\ncan form interpretable representations over documents. In this work, we\ndescribe lda2vec, a model that learns dense word vectors jointly with\nDirichlet-distributed latent document-level mixtures of topic vectors. In\ncontrast to continuous dense document representations, this formulation\nproduces sparse, interpretable document mixtures through a non-negative simplex\nconstraint. Our method is simple to incorporate into existing automatic\ndifferentiation frameworks and allows for unsupervised document representations\ngeared for use by scientists while simultaneously learning word vectors and the\nlinear relationships between them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02134v2",
            "date": "2016-05-07 02:41:54+00:00",
            "title": "Neural Recovery Machine for Chinese Dropped Pronoun",
            "authors": [
                "Wei-Nan Zhang",
                "Ting Liu",
                "Qingyu Yin",
                "Yu Zhang"
            ],
            "abstract": "Dropped pronouns (DPs) are ubiquitous in pro-drop languages like Chinese,\nJapanese etc. Previous work mainly focused on painstakingly exploring the\nempirical features for DPs recovery. In this paper, we propose a neural\nrecovery machine (NRM) to model and recover DPs in Chinese, so that to avoid\nthe non-trivial feature engineering process. The experimental results show that\nthe proposed NRM significantly outperforms the state-of-the-art approaches on\nboth two heterogeneous datasets. Further experiment results of Chinese zero\npronoun (ZP) resolution show that the performance of ZP resolution can also be\nimproved by recovering the ZPs to DPs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02150v1",
            "date": "2016-05-07 06:39:57+00:00",
            "title": "On Improving Informativity and Grammaticality for Multi-Sentence Compression",
            "authors": [
                "Elaheh ShafieiBavani",
                "Mohammad Ebrahimi",
                "Raymond Wong",
                "Fang Chen"
            ],
            "abstract": "Multi Sentence Compression (MSC) is of great value to many real world\napplications, such as guided microblog summarization, opinion summarization and\nnewswire summarization. Recently, word graph-based approaches have been\nproposed and become popular in MSC. Their key assumption is that redundancy\namong a set of related sentences provides a reliable way to generate\ninformative and grammatical sentences. In this paper, we propose an effective\napproach to enhance the word graph-based MSC and tackle the issue that most of\nthe state-of-the-art MSC approaches are confronted with: i.e., improving both\ninformativity and grammaticality at the same time. Our approach consists of\nthree main components: (1) a merging method based on Multiword Expressions\n(MWE); (2) a mapping strategy based on synonymy between words; (3) a re-ranking\nstep to identify the best compression candidates generated using a POS-based\nlanguage model (POS-LM). We demonstrate the effectiveness of this novel\napproach using a dataset made of clusters of English newswire sentences. The\nobserved improvements on informativity and grammaticality of the generated\ncompressions show that our approach is superior to state-of-the-art MSC\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02257v1",
            "date": "2016-05-08 01:38:34+00:00",
            "title": "A corpus of preposition supersenses in English web reviews",
            "authors": [
                "Nathan Schneider",
                "Jena D. Hwang",
                "Vivek Srikumar",
                "Meredith Green",
                "Kathryn Conger",
                "Tim O'Gorman",
                "Martha Palmer"
            ],
            "abstract": "We present the first corpus annotated with preposition supersenses,\nunlexicalized categories for semantic functions that can be marked by English\nprepositions (Schneider et al., 2015). That scheme improves upon its\npredecessors to better facilitate comprehensive manual annotation. Moreover,\nunlike the previous schemes, the preposition supersenses are organized\nhierarchically. Our data will be publicly released on the web upon publication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02276v3",
            "date": "2016-05-08 05:09:28+00:00",
            "title": "Problems With Evaluation of Word Embeddings Using Word Similarity Tasks",
            "authors": [
                "Manaal Faruqui",
                "Yulia Tsvetkov",
                "Pushpendre Rastogi",
                "Chris Dyer"
            ],
            "abstract": "Lacking standardized extrinsic evaluation methods for vector representations\nof words, the NLP community has relied heavily on word similarity tasks as a\nproxy for intrinsic evaluation of word vectors. Word similarity evaluation,\nwhich correlates the distance between vectors and human judgments of semantic\nsimilarity is attractive, because it is computationally inexpensive and fast.\nIn this paper we present several problems associated with the evaluation of\nword vectors on word similarity datasets, and summarize existing solutions. Our\nstudy suggests that the use of word similarity tasks for evaluation of word\nvectors is not sustainable and calls for further research on evaluation\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02457v1",
            "date": "2016-05-09 07:48:40+00:00",
            "title": "The Controlled Natural Language of Randall Munroe's Thing Explainer",
            "authors": [
                "Tobias Kuhn"
            ],
            "abstract": "It is rare that texts or entire books written in a Controlled Natural\nLanguage (CNL) become very popular, but exactly this has happened with a book\nthat has been published last year. Randall Munroe's Thing Explainer uses only\nthe 1'000 most often used words of the English language together with drawn\npictures to explain complicated things such as nuclear reactors, jet engines,\nthe solar system, and dishwashers. This restricted language is a very\ninteresting new case for the CNL community. I describe here its place in the\ncontext of existing approaches on Controlled Natural Languages, and I provide a\nfirst analysis from a scientific perspective, covering the word production\nrules and word distributions.",
            "journal ref": "Proceedings of the Fifth Workshop on Controlled Natural Language\n  (CNL 2016), Springer 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02592v1",
            "date": "2016-05-09 14:05:57+00:00",
            "title": "GLEU Without Tuning",
            "authors": [
                "Courtney Napoles",
                "Keisuke Sakaguchi",
                "Matt Post",
                "Joel Tetreault"
            ],
            "abstract": "The GLEU metric was proposed for evaluating grammatical error corrections\nusing n-gram overlap with a set of reference sentences, as opposed to\nprecision/recall of specific annotated errors (Napoles et al., 2015). This\npaper describes improvements made to the GLEU metric that address problems that\narise when using an increasing number of reference sets. Unlike the originally\npresented metric, the modified metric does not require tuning. We recommend\nthat this version be used instead of the original version.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03148v2",
            "date": "2016-05-10 18:44:34+00:00",
            "title": "Coverage Embedding Models for Neural Machine Translation",
            "authors": [
                "Haitao Mi",
                "Baskaran Sankaran",
                "Zhiguo Wang",
                "Abe Ittycheriah"
            ],
            "abstract": "In this paper, we enhance the attention-based neural machine translation\n(NMT) by adding explicit coverage embedding models to alleviate issues of\nrepeating and dropping translations in NMT. For each source word, our model\nstarts with a full coverage embedding vector to track the coverage status, and\nthen keeps updating it with neural networks as the translation goes.\nExperiments on the large-scale Chinese-to-English task show that our enhanced\nmodel improves the translation quality significantly on various test sets over\nthe strong large vocabulary NMT system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03209v1",
            "date": "2016-05-10 20:50:56+00:00",
            "title": "Vocabulary Manipulation for Neural Machine Translation",
            "authors": [
                "Haitao Mi",
                "Zhiguo Wang",
                "Abe Ittycheriah"
            ],
            "abstract": "In order to capture rich language phenomena, neural machine translation\nmodels have to use a large vocabulary size, which requires high computing time\nand large memory usage. In this paper, we alleviate this issue by introducing a\nsentence-level or batch-level vocabulary, which is only a very small sub-set of\nthe full output vocabulary. For each sentence or batch, we only predict the\ntarget words in its sentence-level or batch-level vocabulary. Thus, we reduce\nboth the computing time and the memory usage. Our method simply takes into\naccount the translation options of each word or phrase in the source sentence,\nand picks a very small target vocabulary for each sentence based on a\nword-to-word translation model or a bilingual phrase library learned from a\ntraditional machine translation model. Experimental results on the large-scale\nEnglish-to-French task show that our method achieves better translation\nperformance by 1 BLEU point over the large vocabulary neural machine\ntranslation system of Jean et al. (2015).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03284v2",
            "date": "2016-05-11 05:05:05+00:00",
            "title": "Machine Comprehension Based on Learning to Rank",
            "authors": [
                "Tian Tian",
                "Yuezhang Li"
            ],
            "abstract": "Machine comprehension plays an essential role in NLP and has been widely\nexplored with dataset like MCTest. However, this dataset is too simple and too\nsmall for learning true reasoning abilities. \\cite{hermann2015teaching}\ntherefore release a large scale news article dataset and propose a deep LSTM\nreader system for machine comprehension. However, the training process is\nexpensive. We therefore try feature-engineered approach with semantics on the\nnew dataset to see how traditional machine learning technique and semantics can\nhelp with machine comprehension. Meanwhile, our proposed L2R reader system\nachieves good performance with efficiency and less training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03664v1",
            "date": "2016-05-12 03:18:12+00:00",
            "title": "Real-Time Web Scale Event Summarization Using Sequential Decision Making",
            "authors": [
                "Chris Kedzie",
                "Fernando Diaz",
                "Kathleen McKeown"
            ],
            "abstract": "We present a system based on sequential decision making for the online\nsummarization of massive document streams, such as those found on the web.\nGiven an event of interest (e.g. \"Boston marathon bombing\"), our system is able\nto filter the stream for relevance and produce a series of short text updates\ndescribing the event as it unfolds over time. Unlike previous work, our\napproach is able to jointly model the relevance, comprehensiveness, novelty,\nand timeliness required by time-sensitive queries. We demonstrate a 28.3%\nimprovement in summary F1 and a 43.8% improvement in time-sensitive F1 metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03832v1",
            "date": "2016-05-12 14:37:51+00:00",
            "title": "Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning",
            "authors": [
                "Yulia Tsvetkov",
                "Sunayana Sitaram",
                "Manaal Faruqui",
                "Guillaume Lample",
                "Patrick Littell",
                "David Mortensen",
                "Alan W Black",
                "Lori Levin",
                "Chris Dyer"
            ],
            "abstract": "We introduce polyglot language models, recurrent neural network models\ntrained to predict symbol sequences in many different languages using shared\nrepresentations of symbols and conditioning on typological information about\nthe language to be predicted. We apply these to the problem of modeling phone\nsequences---a domain in which universal symbol inventories and\ncross-linguistically shared feature representations are a natural fit.\nIntrinsic evaluation on held-out perplexity, qualitative analysis of the\nlearned representations, and extrinsic evaluation in two downstream\napplications that make use of phonetic features show (i) that polyglot models\nbetter generalize to held-out data than comparable monolingual models and (ii)\nthat polyglot phonetic feature representations are of higher quality than those\nlearned monolingually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03852v2",
            "date": "2016-05-12 15:15:58+00:00",
            "title": "Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning",
            "authors": [
                "Yulia Tsvetkov",
                "Manaal Faruqui",
                "Wang Ling",
                "Brian MacWhinney",
                "Chris Dyer"
            ],
            "abstract": "We use Bayesian optimization to learn curricula for word representation\nlearning, optimizing performance on downstream tasks that depend on the learned\nrepresentations as features. The curricula are modeled by a linear ranking\nfunction which is the scalar product of a learned weight vector and an\nengineered feature vector that characterizes the different aspects of the\ncomplexity of each instance in the training corpus. We show that learning the\ncurriculum improves performance on a variety of downstream tasks over random\norders and in comparison to the natural corpus order.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03924v2",
            "date": "2016-05-12 18:45:18+00:00",
            "title": "Joint Embeddings of Hierarchical Categories and Entities",
            "authors": [
                "Yuezhang Li",
                "Ronghuo Zheng",
                "Tian Tian",
                "Zhiting Hu",
                "Rahul Iyer",
                "Katia Sycara"
            ],
            "abstract": "Due to the lack of structured knowledge applied in learning distributed\nrepresentation of categories, existing work cannot incorporate category\nhierarchies into entity information.~We propose a framework that embeds\nentities and categories into a semantic space by integrating structured\nknowledge and taxonomy hierarchy from large knowledge bases. The framework\nallows to compute meaningful semantic relatedness between entities and\ncategories.~Compared with the previous state of the art, our framework can\nhandle both single-word concepts and multiple-word concepts with superior\nperformance in concept categorization and semantic relatedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03956v1",
            "date": "2016-05-12 19:59:43+00:00",
            "title": "On the Convergent Properties of Word Embedding Methods",
            "authors": [
                "Yingtao Tian",
                "Vivek Kulkarni",
                "Bryan Perozzi",
                "Steven Skiena"
            ],
            "abstract": "Do word embeddings converge to learn similar things over different\ninitializations? How repeatable are experiments with word embeddings? Are all\nword embedding techniques equally reliable? In this paper we propose evaluating\nmethods for learning word representations by their consistency across\ninitializations. We propose a measure to quantify the similarity of the learned\nword representations under this setting (where they are subject to different\nrandom initializations). Our preliminary results illustrate that our metric not\nonly measures a intrinsic property of word embedding methods but also\ncorrelates well with other evaluation metrics on downstream tasks. We believe\nour methods are is useful in characterizing robustness -- an important property\nto consider when developing new word embedding methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04002v1",
            "date": "2016-05-12 22:42:48+00:00",
            "title": "Which Learning Algorithms Can Generalize Identity-Based Rules to Novel Inputs?",
            "authors": [
                "Paul Tupper",
                "Bobak Shahriari"
            ],
            "abstract": "We propose a novel framework for the analysis of learning algorithms that\nallows us to say when such algorithms can and cannot generalize certain\npatterns from training data to test data. In particular we focus on situations\nwhere the rule that must be learned concerns two components of a stimulus being\nidentical. We call such a basis for discrimination an identity-based rule.\nIdentity-based rules have proven to be difficult or impossible for certain\ntypes of learning algorithms to acquire from limited datasets. This is in\ncontrast to human behaviour on similar tasks. Here we provide a framework for\nrigorously establishing which learning algorithms will fail at generalizing\nidentity-based rules to novel stimuli. We use this framework to show that such\nalgorithms are unable to generalize identity-based rules to novel inputs unless\ntrained on virtually all possible inputs. We demonstrate these results\ncomputationally with a multilayer feedforward neural network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04238v1",
            "date": "2016-05-13 16:25:38+00:00",
            "title": "Semantic Spaces",
            "authors": [
                "Yuri Manin",
                "Matilde Marcolli"
            ],
            "abstract": "Any natural language can be considered as a tool for producing large\ndatabases (consisting of texts, written, or discursive). This tool for its\ndescription in turn requires other large databases (dictionaries, grammars\netc.). Nowadays, the notion of database is associated with computer processing\nand computer memory. However, a natural language resides also in human brains\nand functions in human communication, from interpersonal to intergenerational\none. We discuss in this survey/research paper mathematical, in particular\ngeometric, constructions, which help to bridge these two worlds. In particular,\nin this paper we consider the Vector Space Model of semantics based on\nfrequency matrices, as used in Natural Language Processing. We investigate\nunderlying geometries, formulated in terms of Grassmannians, projective spaces,\nand flag varieties. We formulate the relation between vector space models and\nsemantic spaces based on semic axes in terms of projectability of subvarieties\nin Grassmannians and projective spaces. We interpret Latent Semantics as a\ngeometric flow on Grassmannians. We also discuss how to formulate G\\\"ardenfors'\nnotion of \"meeting of minds\" in our geometric setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04278v2",
            "date": "2016-05-13 18:45:22+00:00",
            "title": "Universal Dependencies for Learner English",
            "authors": [
                "Yevgeni Berzak",
                "Jessica Kenney",
                "Carolyn Spadine",
                "Jing Xian Wang",
                "Lucia Lam",
                "Keiko Sophie Mori",
                "Sebastian Garza",
                "Boris Katz"
            ],
            "abstract": "We introduce the Treebank of Learner English (TLE), the first publicly\navailable syntactic treebank for English as a Second Language (ESL). The TLE\nprovides manually annotated POS tags and Universal Dependency (UD) trees for\n5,124 sentences from the Cambridge First Certificate in English (FCE) corpus.\nThe UD annotations are tied to a pre-existing error annotation of the FCE,\nwhereby full syntactic analyses are provided for both the original and error\ncorrected versions of each sentence. Further on, we delineate ESL annotation\nguidelines that allow for consistent syntactic treatment of ungrammatical\nEnglish. Finally, we benchmark POS tagging and dependency parsing performance\non the TLE dataset and measure the effect of grammatical errors on parsing\naccuracy. We envision the treebank to support a wide range of linguistic and\ncomputational research on second language acquisition as well as automatic\nprocessing of ungrammatical language. The treebank is available at\nuniversaldependencies.org. The annotation manual used in this project and a\ngraphical query engine are available at esltreebank.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04359v1",
            "date": "2016-05-14 01:13:48+00:00",
            "title": "Occurrence Statistics of Entities, Relations and Types on the Web",
            "authors": [
                "Aman Madaan",
                "Sunita Sarawagi"
            ],
            "abstract": "The problem of collecting reliable estimates of occurrence of entities on the\nopen web forms the premise for this report. The models learned for tagging\nentities cannot be expected to perform well when deployed on the web. This is\nowing to the severe mismatch in the distributions of such entities on the web\nand in the relatively diminutive training data. In this report, we build up the\ncase for maximum mean discrepancy for estimation of occurrence statistics of\nentities on the web, taking a review of named entity disambiguation techniques\nand related concepts along the way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04469v3",
            "date": "2016-05-14 21:30:57+00:00",
            "title": "Rationale-Augmented Convolutional Neural Networks for Text Classification",
            "authors": [
                "Ye Zhang",
                "Iain Marshall",
                "Byron C. Wallace"
            ],
            "abstract": "We present a new Convolutional Neural Network (CNN) model for text\nclassification that jointly exploits labels on documents and their component\nsentences. Specifically, we consider scenarios in which annotators explicitly\nmark sentences (or snippets) that support their overall document\ncategorization, i.e., they provide rationales. Our model exploits such\nsupervision via a hierarchical approach in which each document is represented\nby a linear combination of the vector representations of its component\nsentences. We propose a sentence-level convolutional model that estimates the\nprobability that a given sentence is a rationale, and we then scale the\ncontribution of each sentence to the aggregate document representation in\nproportion to these estimates. Experiments on five classification datasets that\nhave document labels and associated rationales demonstrate that our approach\nconsistently outperforms strong baselines. Moreover, our model naturally\nprovides explanations for its predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04475v1",
            "date": "2016-05-14 22:11:07+00:00",
            "title": "Capturing divergence in dependency trees to improve syntactic projection",
            "authors": [
                "Ryan Georgi",
                "Fei Xia",
                "William D. Lewis"
            ],
            "abstract": "Obtaining syntactic parses is a crucial part of many NLP pipelines. However,\nmost of the world's languages do not have large amounts of syntactically\nannotated corpora available for building parsers. Syntactic projection\ntechniques attempt to address this issue by using parallel corpora consisting\nof resource-poor and resource-rich language pairs, taking advantage of a parser\nfor the resource-rich language and word alignment between the languages to\nproject the parses onto the data for the resource-poor language. These\nprojection methods can suffer, however, when the two languages are divergent.\nIn this paper, we investigate the possibility of using small, parallel,\nannotated corpora to automatically detect divergent structural patterns between\ntwo languages. These patterns can then be used to improve structural projection\nalgorithms, allowing for better performing NLP tools for resource-poor\nlanguages, in particular those that may not have large amounts of annotated\ndata necessary for traditional, fully-supervised methods. While this detection\nprocess is not exhaustive, we demonstrate that common patterns of divergence\ncan be identified automatically without prior knowledge of a given language\npair, and the patterns can be used to improve performance of projection\nalgorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04481v3",
            "date": "2016-05-15 00:26:26+00:00",
            "title": "Anchoring and Agreement in Syntactic Annotations",
            "authors": [
                "Yevgeni Berzak",
                "Yan Huang",
                "Andrei Barbu",
                "Anna Korhonen",
                "Boris Katz"
            ],
            "abstract": "We present a study on two key characteristics of human syntactic annotations:\nanchoring and agreement. Anchoring is a well known cognitive bias in human\ndecision making, where judgments are drawn towards pre-existing values. We\nstudy the influence of anchoring on a standard approach to creation of\nsyntactic resources where syntactic annotations are obtained via human editing\nof tagger and parser output. Our experiments demonstrate a clear anchoring\neffect and reveal unwanted consequences, including overestimation of parsing\nperformance and lower quality of annotations in comparison with human-based\nannotations. Using sentences from the Penn Treebank WSJ, we also report\nsystematically obtained inter-annotator agreement estimates for English\ndependency parsing. Our agreement results control for parser bias, and are\nconsequential in that they are on par with state of the art parsing performance\nfor English newswire. We discuss the impact of our findings on strategies for\nfuture annotation efforts and parser evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04515v8",
            "date": "2016-05-15 09:41:00+00:00",
            "title": "Machine Translation Evaluation Resources and Methods: A Survey",
            "authors": [
                "Lifeng Han"
            ],
            "abstract": "We introduce the Machine Translation (MT) evaluation survey that contains\nboth manual and automatic evaluation methods. The traditional human evaluation\ncriteria mainly include the intelligibility, fidelity, fluency, adequacy,\ncomprehension, and informativeness. The advanced human assessments include\ntask-oriented measures, post-editing, segment ranking, and extended criteriea,\netc. We classify the automatic evaluation methods into two categories,\nincluding lexical similarity scenario and linguistic features application. The\nlexical similarity methods contain edit distance, precision, recall, F-measure,\nand word order. The linguistic features can be divided into syntactic features\nand semantic features respectively. The syntactic features include part of\nspeech tag, phrase types and sentence structures, and the semantic features\ninclude named entity, synonyms, textual entailment, paraphrase, semantic roles,\nand language models. The deep learning models for evaluation are very newly\nproposed. Subsequently, we also introduce the evaluation methods for MT\nevaluation including different correlation scores, and the recent quality\nestimation (QE) tasks for MT.\n  This paper differs from the existing works\n\\cite{GALEprogram2009,EuroMatrixProject2007} from several aspects, by\nintroducing some recent development of MT evaluation measures, the different\nclassifications from manual to automatic evaluation measures, the introduction\nof recent QE tasks of MT, and the concise construction of the content.\n  We hope this work will be helpful for MT researchers to easily pick up some\nmetrics that are best suitable for their specific MT model development, and\nhelp MT evaluation researchers to get a general clue of how MT evaluation\nresearch developed. Furthermore, hopefully, this work can also shine some light\non other evaluation tasks, except for translation, of NLP fields.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04553v2",
            "date": "2016-05-15 14:00:06+00:00",
            "title": "A Proposal for Linguistic Similarity Datasets Based on Commonality Lists",
            "authors": [
                "Dmitrijs Milajevs",
                "Sascha Griffiths"
            ],
            "abstract": "Similarity is a core notion that is used in psychology and two branches of\nlinguistics: theoretical and computational. The similarity datasets that come\nfrom the two fields differ in design: psychological datasets are focused around\na certain topic such as fruit names, while linguistic datasets contain words\nfrom various categories. The later makes humans assign low similarity scores to\nthe words that have nothing in common and to the words that have contrast in\nmeaning, making similarity scores ambiguous. In this work we discuss the\nsimilarity collection procedure for a multi-category dataset that avoids score\nambiguity and suggest changes to the evaluation procedure to reflect the\ninsights of psychological literature for word, phrase and sentence similarity.\nWe suggest to ask humans to provide a list of commonalities and differences\ninstead of numerical similarity scores and employ the structure of human\njudgements beyond pairwise similarity for model evaluation. We believe that the\nproposed approach will give rise to datasets that test meaning representation\nmodels more thoroughly with respect to the human treatment of similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04569v2",
            "date": "2016-05-15 15:53:02+00:00",
            "title": "Syntactically Guided Neural Machine Translation",
            "authors": [
                "Felix Stahlberg",
                "Eva Hasler",
                "Aurelien Waite",
                "Bill Byrne"
            ],
            "abstract": "We investigate the use of hierarchical phrase-based SMT lattices in\nend-to-end neural machine translation (NMT). Weight pushing transforms the\nHiero scores for complete translation hypotheses, with the full translation\ngrammar score and full n-gram language model score, into posteriors compatible\nwith NMT predictive probabilities. With a slightly modified NMT beam-search\ndecoder we find gains over both Hiero and NMT decoding alone, with practical\nadvantages in extending NMT to very large input and output vocabularies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04800v2",
            "date": "2016-05-16 15:15:05+00:00",
            "title": "Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz"
            ],
            "abstract": "This paper describes the submission of the AMU (Adam Mickiewicz University)\nteam to the Automatic Post-Editing (APE) task of WMT 2016. We explore the\napplication of neural translation models to the APE problem and achieve good\nresults by treating different models as components in a log-linear model,\nallowing for multiple inputs (the MT-output and the source) that are decoded to\nthe same target language (post-edited translations). A simple string-matching\npenalty integrated within the log-linear model is used to control for higher\nfaithfulness with regard to the raw machine translation output. To overcome the\nproblem of too little training data, we generate large amounts of artificial\ndata. Our submission improves over the uncorrected baseline on the unseen test\nset by -3.2\\% TER and +5.5\\% BLEU and outperforms any other system submitted to\nthe shared-task by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04809v3",
            "date": "2016-05-16 15:34:19+00:00",
            "title": "The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Tomasz Dwojak",
                "Rico Sennrich"
            ],
            "abstract": "This paper describes the AMU-UEDIN submissions to the WMT 2016 shared task on\nnews translation. We explore methods of decode-time integration of\nattention-based neural translation models with phrase-based statistical machine\ntranslation. Efficient batch-algorithms for GPU-querying are proposed and\nimplemented. For English-Russian, our system stays behind the state-of-the-art\npure neural models in terms of BLEU. Among restricted systems, manual\nevaluation places it in the first cluster tied with the pure neural model. For\nthe Russian-English task, our submission achieves the top BLEU result,\noutperforming the best pure neural system by 1.1 BLEU points and our own\nphrase-based baseline by 1.6 BLEU. After manual evaluation, this system is the\nbest restricted system in its own cluster. In follow-up experiments we improve\nresults by additional 0.8 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05101v1",
            "date": "2016-05-17 10:43:38+00:00",
            "title": "Recurrent Neural Network for Text Classification with Multi-Task Learning",
            "authors": [
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural network based methods have obtained great progress on a variety of\nnatural language processing tasks. However, in most previous works, the models\nare learned based on single-task supervised objectives, which often suffer from\ninsufficient training data. In this paper, we use the multi-task learning\nframework to jointly learn across multiple related tasks. Based on recurrent\nneural network, we propose three different mechanisms of sharing information to\nmodel text with task-specific and shared layers. The entire network is trained\njointly on all these tasks. Experiments on four benchmark text classification\ntasks show that our proposed models can improve the performance of a task with\nthe help of other related tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05110v2",
            "date": "2016-05-17 11:03:25+00:00",
            "title": "Incorporating Loose-Structured Knowledge into Conversation Modeling via Recall-Gate LSTM",
            "authors": [
                "Zhen Xu",
                "Bingquan Liu",
                "Baoxun Wang",
                "Chengjie Sun",
                "Xiaolong Wang"
            ],
            "abstract": "Modeling human conversations is the essence for building satisfying chat-bots\nwith multi-turn dialog ability. Conversation modeling will notably benefit from\ndomain knowledge since the relationships between sentences can be clarified due\nto semantic hints introduced by knowledge. In this paper, a deep neural network\nis proposed to incorporate background knowledge for conversation modeling.\nThrough a specially designed Recall gate, domain knowledge can be transformed\ninto the extra global memory of Long Short-Term Memory (LSTM), so as to enhance\nLSTM by cooperating with its local memory to capture the implicit semantic\nrelevance between sentences within conversations. In addition, this paper\nintroduces the loose structured domain knowledge base, which can be built with\nslight amount of manual work and easily adopted by the Recall gate. Our model\nis evaluated on the context-oriented response selecting task, and experimental\nresults on both two datasets have shown that our approach is promising for\nmodeling human conversations and building key components of automatic chatting\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05172v2",
            "date": "2016-05-17 14:07:43+00:00",
            "title": "Siamese convolutional networks based on phonetic features for cognate identification",
            "authors": [
                "Taraka Rama"
            ],
            "abstract": "In this paper, we explore the use of convolutional networks (ConvNets) for\nthe purpose of cognate identification. We compare our architecture with binary\nclassifiers based on string similarity measures on different language families.\nOur experiments show that convolutional networks achieve competitive results\nacross concepts and across language families at the task of cognate\nidentification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05416v1",
            "date": "2016-05-18 01:45:32+00:00",
            "title": "Leveraging Lexical Resources for Learning Entity Embeddings in Multi-Relational Data",
            "authors": [
                "Teng Long",
                "Ryan Lowe",
                "Jackie Chi Kit Cheung",
                "Doina Precup"
            ],
            "abstract": "Recent work in learning vector-space embeddings for multi-relational data has\nfocused on combining relational information derived from knowledge bases with\ndistributional information derived from large text corpora. We propose a simple\napproach that leverages the descriptions of entities or phrases available in\nlexical resources, in conjunction with distributional semantics, in order to\nderive a better initialization for training relational models. Applying this\ninitialization to the TransE model results in significant new state-of-the-art\nperformances on the WordNet dataset, decreasing the mean rank from the previous\nbest of 212 to 51. It also results in faster convergence of the entity\nrepresentations. We find that there is a trade-off between improving the mean\nrank and the hits@10 with this approach. This illustrates that much remains to\nbe understood regarding performance improvements in relational models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05573v2",
            "date": "2016-05-18 13:33:21+00:00",
            "title": "Modelling Interaction of Sentence Pair with coupled-LSTMs",
            "authors": [
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, there is rising interest in modelling the interactions of two\nsentences with deep neural networks. However, most of the existing methods\nencode two sequences with separate encoders, in which a sentence is encoded\nwith little or no information from the other sentence. In this paper, we\npropose a deep architecture to model the strong interaction of sentence pair\nwith two coupled-LSTMs. Specifically, we introduce two coupled ways to model\nthe interdependences of two LSTMs, coupling the local contextualized\ninteractions of two sentences. We then aggregate these interactions and use a\ndynamic pooling to select the most informative features. Experiments on two\nvery large datasets demonstrate the efficacy of our proposed architecture and\nits superiority to state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05906v1",
            "date": "2016-05-19 12:05:55+00:00",
            "title": "Automatic TM Cleaning through MT and POS Tagging: Autodesk's Submission to the NLP4TM 2016 Shared Task",
            "authors": [
                "Alena Zwahlen",
                "Olivier Carnal",
                "Samuel L\u00e4ubli"
            ],
            "abstract": "We describe a machine learning based method to identify incorrect entries in\ntranslation memories. It extends previous work by Barbu (2015) through\nincorporating recall-based machine translation and part-of-speech-tagging\nfeatures. Our system ranked first in the Binary Classification (II) task for\ntwo out of three language pairs: English-Italian and English-Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06353v2",
            "date": "2016-05-20 13:43:56+00:00",
            "title": "Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz"
            ],
            "abstract": "In this work, we study parameter tuning towards the M^2 metric, the standard\nmetric for automatic grammar error correction (GEC) tasks. After implementing\nM^2 as a scorer in the Moses tuning framework, we investigate interactions of\ndense and sparse features, different optimizers, and tuning strategies for the\nCoNLL-2014 shared task. We notice erratic behavior when optimizing sparse\nfeature weights with M^2 and offer partial solutions. We find that a bare-bones\nphrase-based SMT setup with task-specific parameter-tuning outperforms all\npreviously published results for the CoNLL-2014 test set by a large margin\n(46.37% M^2 over previously 41.75%, by an SMT system with neural features)\nwhile being trained on the same, publicly available data. Our newly introduced\ndense and sparse features widen that gap, and we improve the state-of-the-art\nto 49.49% M^2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06770v1",
            "date": "2016-05-22 10:50:07+00:00",
            "title": "Automatic Construction of Discourse Corpora for Dialogue Translation",
            "authors": [
                "Longyue Wang",
                "Xiaojun Zhang",
                "Zhaopeng Tu",
                "Andy Way",
                "Qun Liu"
            ],
            "abstract": "In this paper, a novel approach is proposed to automatically construct\nparallel discourse corpus for dialogue machine translation. Firstly, the\nparallel subtitle data and its corresponding monolingual movie script data are\ncrawled and collected from Internet. Then tags such as speaker and discourse\nboundary from the script data are projected to its subtitle data via an\ninformation retrieval approach in order to map monolingual discourse to\nbilingual texts. We not only evaluate the mapping results, but also integrate\nspeaker information into the translation. Experiments show our proposed method\ncan achieve 81.79% and 98.64% accuracy on speaker and dialogue boundary\nannotation, and speaker-based language model adaptation can obtain around 0.5\nBLEU points improvement in translation qualities. Finally, we publicly release\naround 100K parallel discourse data with manual speaker and dialogue boundary\nannotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07333v1",
            "date": "2016-05-24 08:20:12+00:00",
            "title": "Combining Recurrent and Convolutional Neural Networks for Relation Classification",
            "authors": [
                "Ngoc Thang Vu",
                "Heike Adel",
                "Pankaj Gupta",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper investigates two different neural architectures for the task of\nrelation classification: convolutional neural networks and recurrent neural\nnetworks. For both models, we demonstrate the effect of different architectural\nchoices. We present a new context representation for convolutional neural\nnetworks for relation classification (extended middle context). Furthermore, we\npropose connectionist bi-directional recurrent neural networks and introduce\nranking loss for their optimization. Finally, we show that combining\nconvolutional and recurrent neural networks using a simple voting scheme is\naccurate enough to improve results. Our neural models achieve state-of-the-art\nresults on the SemEval 2010 relation classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07346v1",
            "date": "2016-05-24 09:19:05+00:00",
            "title": "Multi-Level Analysis and Annotation of Arabic Corpora for Text-to-Sign Language MT",
            "authors": [
                "Abdelaziz Lakhfif",
                "Mohammed T. Laskri",
                "Eric Atwell"
            ],
            "abstract": "In this paper, we present an ongoing effort in lexical semantic analysis and\nannotation of Modern Standard Arabic (MSA) text, a semi automatic annotation\ntool concerned with the morphologic, syntactic, and semantic levels of\ndescription.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07366v1",
            "date": "2016-05-24 10:28:43+00:00",
            "title": "Experiments in Linear Template Combination using Genetic Algorithms",
            "authors": [
                "Nikhilesh Bhatnagar",
                "Radhika Mamidi"
            ],
            "abstract": "Natural Language Generation systems typically have two parts - strategic\n('what to say') and tactical ('how to say'). We present our experiments in\nbuilding an unsupervised corpus-driven template based tactical NLG system. We\nconsider templates as a sequence of words containing gaps. Our idea is based on\nthe observation that templates are grammatical locally (within their textual\nspan). We posit the construction of a sentence as a highly restricted sequence\nof such templates. This work is an attempt to explore the resulting search\nspace using Genetic Algorithms to arrive at acceptable solutions. We present a\nbaseline implementation of this approach which outputs gapped text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07515v2",
            "date": "2016-05-24 15:54:48+00:00",
            "title": "Neural Semantic Role Labeling with Dependency Path Embeddings",
            "authors": [
                "Michael Roth",
                "Mirella Lapata"
            ],
            "abstract": "This paper introduces a novel model for semantic role labeling that makes use\nof neural sequence modeling techniques. Our approach is motivated by the\nobservation that complex syntactic structures and related phenomena, such as\nnested subordinations and nominal predicates, are not handled well by existing\nmodels. Our model treats such instances as sub-sequences of lexicalized\ndependency paths and learns suitable embedding representations. We\nexperimentally demonstrate that such embeddings can improve results over\nprevious state-of-the-art semantic role labelers, and showcase qualitative\nimprovements obtained by our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07683v4",
            "date": "2016-05-24 23:09:58+00:00",
            "title": "Learning End-to-End Goal-Oriented Dialog",
            "authors": [
                "Antoine Bordes",
                "Y-Lan Boureau",
                "Jason Weston"
            ],
            "abstract": "Traditional dialog systems used in goal-oriented applications require a lot\nof domain-specific handcrafting, which hinders scaling up to new domains.\nEnd-to-end dialog systems, in which all components are trained from the dialogs\nthemselves, escape this limitation. But the encouraging success recently\nobtained in chit-chat dialog may not carry over to goal-oriented settings. This\npaper proposes a testbed to break down the strengths and shortcomings of\nend-to-end dialog systems in goal-oriented applications. Set in the context of\nrestaurant reservation, our tasks require manipulating sentences and symbols,\nso as to properly conduct conversations, issue API calls and use the outputs of\nsuch calls. We show that an end-to-end dialog system based on Memory Networks\ncan reach promising, yet imperfect, performance and learn to perform\nnon-trivial operations. We confirm those results by comparing our system to a\nhand-crafted slot-filling baseline on data from the second Dialog State\nTracking Challenge (Henderson et al., 2014a). We show similar result patterns\non data extracted from an online concierge service.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07766v1",
            "date": "2016-05-25 08:05:37+00:00",
            "title": "Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction",
            "authors": [
                "Kim Anh Nguyen",
                "Sabine Schulte im Walde",
                "Ngoc Thang Vu"
            ],
            "abstract": "We propose a novel vector representation that integrates lexical contrast\ninto distributional vectors and strengthens the most salient features for\ndetermining degrees of word similarity. The improved vectors significantly\noutperform standard models and distinguish antonyms from synonyms with an\naverage precision of 0.66-0.76 across word classes (adjectives, nouns, verbs).\nMoreover, we integrate the lexical contrast vectors into the objective function\nof a skip-gram model. The novel embedding outperforms state-of-the-art models\non predicting word similarities in SimLex-999, and on distinguishing antonyms\nfrom synonyms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07843v1",
            "date": "2016-05-25 12:01:46+00:00",
            "title": "Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction",
            "authors": [
                "Yichun Yin",
                "Furu Wei",
                "Li Dong",
                "Kaimeng Xu",
                "Ming Zhang",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we develop a novel approach to aspect term extraction based on\nunsupervised learning of distributed representations of words and dependency\npaths. The basic idea is to connect two words (w1 and w2) with the dependency\npath (r) between them in the embedding space. Specifically, our method\noptimizes the objective w1 + r = w2 in the low-dimensional space, where the\nmulti-hop dependency paths are treated as a sequence of grammatical relations\nand modeled by a recurrent neural network. Then, we design the embedding\nfeatures that consider linear context and dependency context information, for\nthe conditional random field (CRF) based aspect term extraction. Experimental\nresults on the SemEval datasets show that, (1) with only embedding features, we\ncan achieve state-of-the-art results; (2) our embedding method which\nincorporates the syntactic information among words yields better performance\nthan other representative ones in aspect term extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07869v2",
            "date": "2016-05-25 13:18:57+00:00",
            "title": "Variational Neural Machine Translation",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su",
                "Hong Duan",
                "Min Zhang"
            ],
            "abstract": "Models of neural machine translation are often from a discriminative family\nof encoderdecoders that learn a conditional distribution of a target sentence\ngiven a source sentence. In this paper, we propose a variational model to learn\nthis conditional distribution for neural machine translation: a variational\nencoderdecoder model that can be trained end-to-end. Different from the vanilla\nencoder-decoder model that generates target translations from hidden\nrepresentations of source sentences alone, the variational model introduces a\ncontinuous latent variable to explicitly model underlying semantics of source\nsentences and to guide the generation of target translations. In order to\nperform efficient posterior inference and large-scale training, we build a\nneural posterior approximator conditioned on both the source and the target\nsides, and equip it with a reparameterization technique to estimate the\nvariational lower bound. Experiments on both Chinese-English and English-\nGerman translation tasks show that the proposed variational neural machine\ntranslation achieves significant improvements over the vanilla neural machine\ntranslation baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07874v2",
            "date": "2016-05-25 13:29:07+00:00",
            "title": "BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su"
            ],
            "abstract": "In this paper, we propose a bidimensional attention based recursive\nautoencoder (BattRAE) to integrate clues and sourcetarget interactions at\nmultiple levels of granularity into bilingual phrase representations. We employ\nrecursive autoencoders to generate tree structures of phrases with embeddings\nat different levels of granularity (e.g., words, sub-phrases and phrases). Over\nthese embeddings on the source and target side, we introduce a bidimensional\nattention network to learn their interactions encoded in a bidimensional\nattention matrix, from which we extract two soft attention weight distributions\nsimultaneously. These weight distributions enable BattRAE to generate\ncompositive phrase representations via convolution. Based on the learned phrase\nrepresentations, we further use a bilinear neural model, trained via a\nmax-margin method, to measure bilingual semantic similarity. To evaluate the\neffectiveness of BattRAE, we incorporate this semantic similarity as an\nadditional feature into a state-of-the-art SMT system. Extensive experiments on\nNIST Chinese-English test sets show that our model achieves a substantial\nimprovement of up to 1.63 BLEU points on average over the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.08675v1",
            "date": "2016-05-27 14:57:37+00:00",
            "title": "Boosting Question Answering by Deep Entity Recognition",
            "authors": [
                "Piotr Przyby\u0142a"
            ],
            "abstract": "In this paper an open-domain factoid question answering system for Polish,\nRAFAEL, is presented. The system goes beyond finding an answering sentence; it\nalso extracts a single string, corresponding to the required entity. Herein the\nfocus is placed on different approaches to entity recognition, essential for\nretrieving information matching question constraints. Apart from traditional\napproach, including named entity recognition (NER) solutions, a novel\ntechnique, called Deep Entity Recognition (DeepER), is introduced and\nimplemented. It allows a comprehensive search of all forms of entity references\nmatching a given WordNet synset (e.g. an impressionist), based on a previously\nassembled entity library. It has been created by analysing the first sentences\nof encyclopaedia entries and disambiguation and redirect pages. DeepER also\nprovides automatic evaluation, which makes possible numerous experiments,\nincluding over a thousand questions from a quiz TV show answered on the grounds\nof Polish Wikipedia. The final results of a manual evaluation on a separate\nquestion set show that the strength of DeepER approach lies in its ability to\nanswer questions that demand answers beyond the traditional categories of named\nentities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.08889v2",
            "date": "2016-05-28 13:19:15+00:00",
            "title": "Building an Evaluation Scale using Item Response Theory",
            "authors": [
                "John P. Lalor",
                "Hao Wu",
                "Hong Yu"
            ],
            "abstract": "Evaluation of NLP methods requires testing against a previously vetted\ngold-standard test set and reporting standard metrics\n(accuracy/precision/recall/F1). The current assumption is that all items in a\ngiven test set are equal with regards to difficulty and discriminating power.\nWe propose Item Response Theory (IRT) from psychometrics as an alternative\nmeans for gold-standard test-set generation and NLP system evaluation. IRT is\nable to describe characteristics of individual items - their difficulty and\ndiscriminating power - and can account for these characteristics in its\nestimation of human intelligence or ability for an NLP task. In this paper, we\ndemonstrate IRT by generating a gold-standard test set for Recognizing Textual\nEntailment. By collecting a large number of human responses and fitting our IRT\nmodel, we show that our IRT model compares NLP systems with the performance in\na human population and is able to provide more insight into system performance\nthan standard evaluation metrics. We show that a high accuracy score does not\nalways imply a high IRT score, which depends on the item characteristics and\nthe response pattern.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.08900v2",
            "date": "2016-05-28 14:47:49+00:00",
            "title": "Aspect Level Sentiment Classification with Deep Memory Network",
            "authors": [
                "Duyu Tang",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "We introduce a deep memory network for aspect level sentiment classification.\nUnlike feature-based SVM and sequential neural models such as LSTM, this\napproach explicitly captures the importance of each context word when inferring\nthe sentiment polarity of an aspect. Such importance degree and text\nrepresentation are calculated with multiple computational layers, each of which\nis a neural attention model over an external memory. Experiments on laptop and\nrestaurant datasets demonstrate that our approach performs comparable to\nstate-of-art feature based SVM system, and substantially better than LSTM and\nattention-based LSTM architectures. On both datasets we show that multiple\ncomputational layers could improve the performance. Moreover, our approach is\nalso fast. The deep memory network with 9 layers is 15 times faster than LSTM\nwith a CPU implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.09090v1",
            "date": "2016-05-30 02:47:35+00:00",
            "title": "Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention",
            "authors": [
                "Yang Liu",
                "Chengjie Sun",
                "Lei Lin",
                "Xiaolong Wang"
            ],
            "abstract": "In this paper, we proposed a sentence encoding-based model for recognizing\ntext entailment. In our approach, the encoding of sentence is a two-stage\nprocess. Firstly, average pooling was used over word-level bidirectional LSTM\n(biLSTM) to generate a first-stage sentence representation. Secondly, attention\nmechanism was employed to replace average pooling on the same sentence for\nbetter representations. Instead of using target sentence to attend words in\nsource sentence, we utilized the sentence's first-stage representation to\nattend words appeared in itself, which is called \"Inner-Attention\" in our paper\n. Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus\nhas proved the effectiveness of \"Inner-Attention\" mechanism. With less number\nof parameters, our model outperformed the existing best sentence encoding-based\napproach by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.09096v6",
            "date": "2016-05-30 03:54:18+00:00",
            "title": "Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change",
            "authors": [
                "William L. Hamilton",
                "Jure Leskovec",
                "Dan Jurafsky"
            ],
            "abstract": "Understanding how words change their meanings over time is key to models of\nlanguage and cultural evolution, but historical data on meaning is scarce,\nmaking theories hard to develop and test. Word embeddings show promise as a\ndiachronic tool, but have not been carefully evaluated. We develop a robust\nmethodology for quantifying semantic change by evaluating word embeddings\n(PPMI, SVD, word2vec) against known historical changes. We then use this\nmethodology to reveal statistical laws of semantic evolution. Using six\nhistorical corpora spanning four languages and two centuries, we propose two\nquantitative laws of semantic change: (i) the law of conformity---the rate of\nsemantic change scales with an inverse power-law of word frequency; (ii) the\nlaw of innovation---independent of frequency, words that are more polysemous\nhave higher rates of semantic change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00066v1",
            "date": "2016-12-31 07:09:52+00:00",
            "title": "A POS Tagger for Code Mixed Indian Social Media Text - ICON-2016 NLP Tools Contest Entry from Surukam",
            "authors": [
                "Sree Harsha Ramesh",
                "Raveena R Kumar"
            ],
            "abstract": "Building Part-of-Speech (POS) taggers for code-mixed Indian languages is a\nparticularly challenging problem in computational linguistics due to a dearth\nof accurately annotated training corpora. ICON, as part of its NLP tools\ncontest has organized this challenge as a shared task for the second\nconsecutive year to improve the state-of-the-art. This paper describes the POS\ntagger built at Surukam to predict the coarse-grained and fine-grained POS tags\nfor three language pairs - Bengali-English, Telugu-English and Hindi-English,\nwith the text spanning three popular social media platforms - Facebook,\nWhatsApp and Twitter. We employed Conditional Random Fields as the sequence\ntagging algorithm and used a library called sklearn-crfsuite - a thin wrapper\naround CRFsuite for training our model. Among the features we used include -\ncharacter n-grams, language information and patterns for emoji, number,\npunctuation and web-address. Our submissions in the constrained\nenvironment,i.e., without making any use of monolingual POS taggers or the\nlike, obtained an overall average F1-score of 76.45%, which is comparable to\nthe 2015 winning score of 76.79%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00145v2",
            "date": "2016-12-31 17:00:08+00:00",
            "title": "Expanding Subjective Lexicons for Social Media Mining with Embedding Subspaces",
            "authors": [
                "Silvio Amir",
                "R\u00e1mon Astudillo",
                "Wang Ling",
                "Paula C. Carvalho",
                "M\u00e1rio J. Silva"
            ],
            "abstract": "Recent approaches for sentiment lexicon induction have capitalized on\npre-trained word embeddings that capture latent semantic properties. However,\nembeddings obtained by optimizing performance of a given task (e.g. predicting\ncontextual words) are sub-optimal for other applications. In this paper, we\naddress this problem by exploiting task-specific representations, induced via\nembedding sub-space projection. This allows us to expand lexicons describing\nmultiple semantic properties. For each property, our model jointly learns\nsuitable representations and the concomitant predictor. Experiments conducted\nover multiple subjective lexicons, show that our model outperforms previous\nwork and other baselines; even in low training data regimes. Furthermore,\nlexicon-based sentiment classifiers built on top of our lexicons outperform\nsimilar resources and yield performances comparable to those of supervised\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00168v1",
            "date": "2016-12-31 21:37:10+00:00",
            "title": "Social Media Argumentation Mining: The Quest for Deliberateness in Raucousness",
            "authors": [
                "Jan \u0160najder"
            ],
            "abstract": "Argumentation mining from social media content has attracted increasing\nattention. The task is both challenging and rewarding. The informal nature of\nuser-generated content makes the task dauntingly difficult. On the other hand,\nthe insights that could be gained by a large-scale analysis of social media\nargumentation make it a very worthwhile task. In this position paper I discuss\nthe motivation for social media argumentation mining, as well as the tasks and\nchallenges involved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00061v5",
            "date": "2016-05-31 22:02:01+00:00",
            "title": "Hierarchical Question-Image Co-Attention for Visual Question Answering",
            "authors": [
                "Jiasen Lu",
                "Jianwei Yang",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "A number of recent works have proposed attention models for Visual Question\nAnswering (VQA) that generate spatial maps highlighting image regions relevant\nto answering the question. In this paper, we argue that in addition to modeling\n\"where to look\" or visual attention, it is equally important to model \"what\nwords to listen to\" or question attention. We present a novel co-attention\nmodel for VQA that jointly reasons about image and question attention. In\naddition, our model reasons about the question (and consequently the image via\nthe co-attention mechanism) in a hierarchical fashion via a novel 1-dimensional\nconvolution neural networks (CNN). Our model improves the state-of-the-art on\nthe VQA dataset from 60.3% to 60.5%, and from 61.6% to 63.3% on the COCO-QA\ndataset. By using ResNet, the performance is further improved to 62.1% for VQA\nand 65.4% for COCO-QA.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.00372v1",
            "date": "2016-06-01 18:01:14+00:00",
            "title": "Conversational Contextual Cues: The Case of Personalization and History for Response Ranking",
            "authors": [
                "Rami Al-Rfou",
                "Marc Pickett",
                "Javier Snaider",
                "Yun-hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "We investigate the task of modeling open-domain, multi-turn, unstructured,\nmulti-participant, conversational dialogue. We specifically study the effect of\nincorporating different elements of the conversation. Unlike previous efforts,\nwhich focused on modeling messages and responses, we extend the modeling to\nlong context and participant's history. Our system does not rely on handwritten\nrules or engineered features; instead, we train deep neural networks on a large\nconversational dataset. In particular, we exploit the structure of Reddit\ncomments and posts to extract 2.1 billion messages and 133 million\nconversations. We evaluate our models on the task of predicting the next\nresponse in a conversation, and we find that modeling both context and\nparticipants improves prediction accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01280v4",
            "date": "2016-06-03 21:27:03+00:00",
            "title": "Dependency Parsing as Head Selection",
            "authors": [
                "Xingxing Zhang",
                "Jianpeng Cheng",
                "Mirella Lapata"
            ],
            "abstract": "Conventional graph-based dependency parsers guarantee a tree structure both\nduring training and inference. Instead, we formalize dependency parsing as the\nproblem of independently selecting the head of each word in a sentence. Our\nmodel which we call \\textsc{DeNSe} (as shorthand for {\\bf De}pendency {\\bf\nN}eural {\\bf Se}lection) produces a distribution over possible heads for each\nword using features obtained from a bidirectional recurrent neural network.\nWithout enforcing structural constraints during training, \\textsc{DeNSe}\ngenerates (at inference time) trees for the overwhelming majority of sentences,\nwhile non-tree outputs can be adjusted with a maximum spanning tree algorithm.\nWe evaluate \\textsc{DeNSe} on four languages (English, Chinese, Czech, and\nGerman) with varying degrees of non-projectivity. Despite the simplicity of the\napproach, our parsers are on par with the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01292v1",
            "date": "2016-06-03 22:26:01+00:00",
            "title": "An Attentional Neural Conversation Model with Improved Specificity",
            "authors": [
                "Kaisheng Yao",
                "Baolin Peng",
                "Geoffrey Zweig",
                "Kam-Fai Wong"
            ],
            "abstract": "In this paper we propose a neural conversation model for conducting\ndialogues. We demonstrate the use of this model to generate help desk\nresponses, where users are asking questions about PC applications. Our model is\ndistinguished by two characteristics. First, it models intention across turns\nwith a recurrent network, and incorporates an attention model that is\nconditioned on the representation of intention. Secondly, it avoids generating\nnon-specific responses by incorporating an IDF term in the objective function.\nThe model is evaluated both as a pure generation model in which a help-desk\nresponse is generated from scratch, and as a retrieval model with performance\nmeasured using recall rates of the correct response. Experimental results\nindicate that the model outperforms previously proposed neural conversation\narchitectures, and that using specificity in the objective function\nsignificantly improves performances for both generation and retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01549v3",
            "date": "2016-06-05 19:30:39+00:00",
            "title": "Gated-Attention Readers for Text Comprehension",
            "authors": [
                "Bhuwan Dhingra",
                "Hanxiao Liu",
                "Zhilin Yang",
                "William W. Cohen",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "In this paper we study the problem of answering cloze-style questions over\ndocuments. Our model, the Gated-Attention (GA) Reader, integrates a multi-hop\narchitecture with a novel attention mechanism, which is based on multiplicative\ninteractions between the query embedding and the intermediate states of a\nrecurrent neural network document reader. This enables the reader to build\nquery-specific representations of tokens in the document for accurate answer\nselection. The GA Reader obtains state-of-the-art results on three benchmarks\nfor this task--the CNN \\& Daily Mail news stories and the Who Did What dataset.\nThe effectiveness of multiplicative interaction is demonstrated by an ablation\nstudy, and by comparing to alternative compositional operators for implementing\nthe gated-attention. The code is available at\nhttps://github.com/bdhingra/ga-reader.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01720v1",
            "date": "2016-06-06 12:55:57+00:00",
            "title": "Proof nets for the Displacement calculus",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "We present a proof net calculus for the Displacement calculus and show its\ncorrectness. This is the first proof net calculus which models the Displacement\ncalculus directly and not by some sort of translation into another formalism.\nThe proof net calculus opens up new possibilities for parsing and proof search\nwith the Displacement calculus.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1606.02245v4",
            "date": "2016-06-07 18:25:48+00:00",
            "title": "Iterative Alternating Neural Attention for Machine Reading",
            "authors": [
                "Alessandro Sordoni",
                "Philip Bachman",
                "Adam Trischler",
                "Yoshua Bengio"
            ],
            "abstract": "We propose a novel neural attention architecture to tackle machine\ncomprehension tasks, such as answering Cloze-style queries with respect to a\ndocument. Unlike previous models, we do not collapse the query into a single\nvector, instead we deploy an iterative alternating attention mechanism that\nallows a fine-grained exploration of both the query and the document. Our model\noutperforms state-of-the-art baselines in standard machine comprehension\nbenchmarks such as CNN news articles and the Children's Book Test (CBT)\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02447v1",
            "date": "2016-06-08 08:27:09+00:00",
            "title": "Learning Language Games through Interaction",
            "authors": [
                "Sida I. Wang",
                "Percy Liang",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce a new language learning setting relevant to building adaptive\nnatural language interfaces. It is inspired by Wittgenstein's language games: a\nhuman wishes to accomplish some task (e.g., achieving a certain configuration\nof blocks), but can only communicate with a computer, who performs the actual\nactions (e.g., removing all red blocks). The computer initially knows nothing\nabout language and therefore must learn it from scratch through interaction,\nwhile the human adapts to the computer's capabilities. We created a game in a\nblocks world and collected interactions from 100 people playing it. First, we\nanalyze the humans' strategies, showing that using compositionality and\navoiding synonyms correlates positively with task performance. Second, we\ncompare computer strategies, showing how to quickly learn a semantic parsing\nmodel from scratch, and that modeling pragmatics further accelerates learning\nfor successful players.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02562v1",
            "date": "2016-06-08 14:08:21+00:00",
            "title": "DialPort: Connecting the Spoken Dialog Research Community to Real User Data",
            "authors": [
                "Tiancheng Zhao",
                "Kyusong Lee",
                "Maxine Eskenazi"
            ],
            "abstract": "This paper describes a new spoken dialog portal that connects systems\nproduced by the spoken dialog academic research community and gives them access\nto real users. We introduce a distributed, multi-modal, multi-agent prototype\ndialog framework that affords easy integration with various remote resources,\nranging from end-to-end dialog systems to external knowledge APIs. To date, the\nDialPort portal has successfully connected to the multi-domain spoken dialog\nsystem at Cambridge University, the NOAA (National Oceanic and Atmospheric\nAdministration) weather API and the Yelp API.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.02689v1",
            "date": "2016-06-08 19:03:06+00:00",
            "title": "Continuously Learning Neural Dialogue Management",
            "authors": [
                "Pei-Hao Su",
                "Milica Gasic",
                "Nikola Mrksic",
                "Lina Rojas-Barahona",
                "Stefan Ultes",
                "David Vandyke",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "We describe a two-step approach for dialogue management in task-oriented\nspoken dialogue systems. A unified neural network framework is proposed to\nenable the system to first learn by supervision from a set of dialogue data and\nthen continuously improve its behaviour via reinforcement learning, all using\ngradient-based algorithms on one single model. The experiments demonstrate the\nsupervised model's effectiveness in the corpus-based evaluation, with user\nsimulation, and with paid human subjects. The use of reinforcement learning\nfurther improves the model's performance in both interactive settings,\nespecially under higher-noise conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02858v2",
            "date": "2016-06-09 08:19:16+00:00",
            "title": "A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task",
            "authors": [
                "Danqi Chen",
                "Jason Bolton",
                "Christopher D. Manning"
            ],
            "abstract": "Enabling a computer to understand a document so that it can answer\ncomprehension questions is a central, yet unsolved goal of NLP. A key factor\nimpeding its solution by machine learned systems is the limited availability of\nhuman-annotated data. Hermann et al. (2015) seek to solve this problem by\ncreating over a million training examples by pairing CNN and Daily Mail news\narticles with their summarized bullet points, and show that a neural network\ncan then be trained to give good performance on this task. In this paper, we\nconduct a thorough examination of this new reading comprehension task. Our\nprimary aim is to understand what depth of language understanding is required\nto do well on this task. We approach this from one side by doing a careful\nhand-analysis of a small subset of the problems and from the other by showing\nthat simple, carefully designed systems can obtain accuracies of 73.6% and\n76.6% on these two datasets, exceeding current state-of-the-art results by\n7-10% and approaching what we believe is the ceiling for performance on this\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02976v1",
            "date": "2016-06-09 14:32:50+00:00",
            "title": "Large scale biomedical texts classification: a kNN and an ESA-based approaches",
            "authors": [
                "Khadim Dram\u00e9",
                "Fleur Mougin",
                "Gayo Diallo"
            ],
            "abstract": "With the large and increasing volume of textual data, automated methods for\nidentifying significant topics to classify textual documents have received a\ngrowing interest. While many efforts have been made in this direction, it still\nremains a real challenge. Moreover, the issue is even more complex as full\ntexts are not always freely available. Then, using only partial information to\nannotate these documents is promising but remains a very ambitious issue.\nMethodsWe propose two classification methods: a k-nearest neighbours\n(kNN)-based approach and an explicit semantic analysis (ESA)-based approach.\nAlthough the kNN-based approach is widely used in text classification, it needs\nto be improved to perform well in this specific classification problem which\ndeals with partial information. Compared to existing kNN-based methods, our\nmethod uses classical Machine Learning (ML) algorithms for ranking the labels.\nAdditional features are also investigated in order to improve the classifiers'\nperformance. In addition, the combination of several learning algorithms with\nvarious techniques for fixing the number of relevant topics is performed. On\nthe other hand, ESA seems promising for this classification task as it yielded\ninteresting results in related issues, such as semantic relatedness computation\nbetween texts and text classification. Unlike existing works, which use ESA for\nenriching the bag-of-words approach with additional knowledge-based features,\nour ESA-based method builds a standalone classifier. Furthermore, we\ninvestigate if the results of this method could be useful as a complementary\nfeature of our kNN-based approach.ResultsExperimental evaluations performed on\nlarge standard annotated datasets, provided by the BioASQ organizers, show that\nthe kNN-based method with the Random Forest learning algorithm achieves good\nperformances compared with the current state-of-the-art methods, reaching a\ncompetitive f-measure of 0.55% while the ESA-based approach surprisingly\nyielded reserved results.ConclusionsWe have proposed simple classification\nmethods suitable to annotate textual documents using only partial information.\nThey are therefore adequate for large multi-label classification and\nparticularly in the biomedical domain. Thus, our work contributes to the\nextraction of relevant information from unstructured documents in order to\nfacilitate their automated processing. Consequently, it could be used for\nvarious purposes, including document indexing, information retrieval, etc.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.03152v4",
            "date": "2016-06-10 01:02:19+00:00",
            "title": "Policy Networks with Two-Stage Training for Dialogue Systems",
            "authors": [
                "Mehdi Fatemi",
                "Layla El Asri",
                "Hannes Schulz",
                "Jing He",
                "Kaheer Suleman"
            ],
            "abstract": "In this paper, we propose to use deep policy networks which are trained with\nan advantage actor-critic method for statistically optimised dialogue systems.\nFirst, we show that, on summary state and action spaces, deep Reinforcement\nLearning (RL) outperforms Gaussian Processes methods. Summary state and action\nspaces lead to good performance but require pre-engineering effort, RL\nknowledge, and domain expertise. In order to remove the need to define such\nsummary spaces, we show that deep RL can also be trained efficiently on the\noriginal state and action spaces. Dialogue systems based on partially\nobservable Markov decision processes are known to require many dialogues to\ntrain, which makes them unappealing for practical deployment. We show that a\ndeep RL method based on an actor-critic architecture can exploit a small amount\nof data very efficiently. Indeed, with only a few hundred dialogues collected\nwith a handcrafted policy, the actor-critic deep learner is considerably\nbootstrapped from a combination of supervised and batch RL. In addition,\nconvergence to an optimal policy is significantly sped up compared to other\ndeep RL methods initialized on the data with batch RL. All experiments are\nperformed on a restaurant domain derived from the Dialogue State Tracking\nChallenge 2 (DSTC2) dataset.",
            "journal ref": "Proceedings of the SIGDIAL 2016 Conference, pages 101--110, Los\n  Angeles, USA, 13-15 September 2016. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03153v3",
            "date": "2016-06-10 01:22:32+00:00",
            "title": "Unsupervised Learning of Word-Sequence Representations from Scratch via Convolutional Tensor Decomposition",
            "authors": [
                "Furong Huang",
                "Animashree Anandkumar"
            ],
            "abstract": "Unsupervised text embeddings extraction is crucial for text understanding in\nmachine learning. Word2Vec and its variants have received substantial success\nin mapping words with similar syntactic or semantic meaning to vectors close to\neach other. However, extracting context-aware word-sequence embedding remains a\nchallenging task. Training over large corpus is difficult as labels are\ndifficult to get. More importantly, it is challenging for pre-trained models to\nobtain word-sequence embeddings that are universally good for all downstream\ntasks or for any new datasets. We propose a two-phased ConvDic+DeconvDec\nframework to solve the problem by combining a word-sequence dictionary learning\nmodel with a word-sequence embedding decode model. We propose a convolutional\ntensor decomposition mechanism to learn good word-sequence phrase dictionary in\nthe learning phase. It is proved to be more accurate and much more efficient\nthan the popular alternating minimization method. In the decode phase, we\nintroduce a deconvolution framework that is immune to the problem of varying\nsentence lengths. The word-sequence embeddings we extracted using\nConvDic+DeconvDec are universally good for a few downstream tasks we test on.\nThe framework requires neither pre-training nor prior/outside information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03254v2",
            "date": "2016-06-10 10:12:13+00:00",
            "title": "Natural Language Generation enhances human decision-making with uncertain information",
            "authors": [
                "Dimitra Gkatzia",
                "Oliver Lemon",
                "Verena Rieser"
            ],
            "abstract": "Decision-making is often dependent on uncertain data, e.g. data associated\nwith confidence scores or probabilities. We present a comparison of different\ninformation presentations for uncertain data and, for the first time, measure\ntheir effects on human decision-making. We show that the use of Natural\nLanguage Generation (NLG) improves decision-making under uncertainty, compared\nto state-of-the-art graphical-based representation methods. In a task-based\nstudy with 442 adults, we found that presentations using NLG lead to 24% better\ndecision-making on average than the graphical presentations, and to 44% better\ndecision-making when NLG is combined with graphics. We also show that women\nachieve significantly better results when presented with NLG output (an 87%\nincrease on average compared to graphical presentations).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03402v2",
            "date": "2016-06-10 17:30:46+00:00",
            "title": "Length bias in Encoder Decoder Models and a Case for Global Conditioning",
            "authors": [
                "Pavel Sountsov",
                "Sunita Sarawagi"
            ],
            "abstract": "Encoder-decoder networks are popular for modeling sequences probabilistically\nin many applications. These models use the power of the Long Short-Term Memory\n(LSTM) architecture to capture the full dependence among variables, unlike\nearlier models like CRFs that typically assumed conditional independence among\nnon-adjacent variables. However in practice encoder-decoder models exhibit a\nbias towards short sequences that surprisingly gets worse with increasing beam\nsize.\n  In this paper we show that such phenomenon is due to a discrepancy between\nthe full sequence margin and the per-element margin enforced by the locally\nconditioned training objective of a encoder-decoder model. The discrepancy more\nadversely impacts long sequences, explaining the bias towards predicting short\nsequences.\n  For the case where the predicted sequences come from a closed set, we show\nthat a globally conditioned model alleviates the above problems of\nencoder-decoder models. From a practical point of view, our proposed model also\neliminates the need for a beam-search during inference, which reduces to an\nefficient dot-product based search in a vector-space.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.03556v2",
            "date": "2016-06-11 05:41:10+00:00",
            "title": "Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?",
            "authors": [
                "Abhishek Das",
                "Harsh Agrawal",
                "C. Lawrence Zitnick",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We conduct large-scale studies on `human attention' in Visual Question\nAnswering (VQA) to understand where humans choose to look to answer questions\nabout images. We design and test multiple game-inspired novel\nattention-annotation interfaces that require the subject to sharpen regions of\na blurred image to answer a question. Thus, we introduce the VQA-HAT (Human\nATtention) dataset. We evaluate attention maps generated by state-of-the-art\nVQA models against human attention both qualitatively (via visualizations) and\nquantitatively (via rank-order correlation). Overall, our experiments show that\ncurrent attention models in VQA do not seem to be looking at the same regions\nas humans.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.03568v2",
            "date": "2016-06-11 08:12:02+00:00",
            "title": "Word Sense Disambiguation using a Bidirectional LSTM",
            "authors": [
                "Mikael K\u00e5geb\u00e4ck",
                "Hans Salomonsson"
            ],
            "abstract": "In this paper we present a clean, yet effective, model for word sense\ndisambiguation. Our approach leverage a bidirectional long short-term memory\nnetwork which is shared between all words. This enables the model to share\nstatistical strength and to scale well with vocabulary size. The model is\ntrained end-to-end, directly from the raw text to sense labels, and makes\neffective use of word order. We evaluate our approach on two standard datasets,\nusing identical hyperparameter settings, which are in turn tuned on a third set\nof held out data. We employ no external resources (e.g. knowledge graphs,\npart-of-speech tagging, etc), language specific features, or hand crafted\nrules, but still achieve statistically equivalent results to the best\nstate-of-the-art systems, that employ no such limitations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03784v1",
            "date": "2016-06-13 00:12:49+00:00",
            "title": "MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection",
            "authors": [
                "Guido Zarrella",
                "Amy Marsh"
            ],
            "abstract": "We describe MITRE's submission to the SemEval-2016 Task 6, Detecting Stance\nin Tweets. This effort achieved the top score in Task A on supervised stance\ndetection, producing an average F1 score of 67.8 when assessing whether a tweet\nauthor was in favor or against a topic. We employed a recurrent neural network\ninitialized with features learned via distant supervision on two large\nunlabeled datasets. We trained embeddings of words and phrases with the\nword2vec skip-gram method, then used those features to learn sentence\nrepresentations via a hashtag prediction auxiliary task. These sentence vectors\nwere then fine-tuned for stance detection on several hundred labeled examples.\nThe result was a high performing system that used transfer learning to maximize\nthe value of the available training data.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.04155v2",
            "date": "2016-06-13 22:10:23+00:00",
            "title": "Rationalizing Neural Predictions",
            "authors": [
                "Tao Lei",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "Prediction without justification has limited applicability. As a remedy, we\nlearn to extract pieces of input text as justifications -- rationales -- that\nare tailored to be short and coherent, yet sufficient for making the same\nprediction. Our approach combines two modular components, generator and\nencoder, which are trained to operate well together. The generator specifies a\ndistribution over text fragments as candidate rationales and these are passed\nthrough the encoder for prediction. Rationales are never given during training.\nInstead, the model is regularized by desiderata for rationales. We evaluate the\napproach on multi-aspect sentiment analysis against manually annotated test\ncases. Our approach outperforms attention-based baseline by a significant\nmargin. We also successfully illustrate the method on the question retrieval\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04199v3",
            "date": "2016-06-14 03:53:00+00:00",
            "title": "Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation",
            "authors": [
                "Jie Zhou",
                "Ying Cao",
                "Xuguang Wang",
                "Peng Li",
                "Wei Xu"
            ],
            "abstract": "Neural machine translation (NMT) aims at solving machine translation (MT)\nproblems using neural networks and has exhibited promising results in recent\nyears. However, most of the existing NMT models are shallow and there is still\na performance gap between a single NMT model and the best conventional MT\nsystem. In this work, we introduce a new type of linear connections, named\nfast-forward connections, based on deep Long Short-Term Memory (LSTM) networks,\nand an interleaved bi-directional architecture for stacking the LSTM layers.\nFast-forward connections play an essential role in propagating the gradients\nand building a deep topology of depth 16. On the WMT'14 English-to-French task,\nwe achieve BLEU=37.7 with a single attention model, which outperforms the\ncorresponding single shallow model by 6.2 BLEU points. This is the first time\nthat a single NMT model achieves state-of-the-art performance and outperforms\nthe best conventional model by 0.7 BLEU points. We can still achieve BLEU=36.3\neven without using an attention mechanism. After special handling of unknown\nwords and model ensembling, we obtain the best score reported to date on this\ntask with BLEU=40.4. Our models are also validated on the more difficult WMT'14\nEnglish-to-German task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04217v1",
            "date": "2016-06-14 07:04:37+00:00",
            "title": "Word Representation Models for Morphologically Rich Languages in Neural Machine Translation",
            "authors": [
                "Ekaterina Vylomova",
                "Trevor Cohn",
                "Xuanli He",
                "Gholamreza Haffari"
            ],
            "abstract": "Dealing with the complex word forms in morphologically rich languages is an\nopen problem in language processing, and is particularly important in\ntranslation. In contrast to most modern neural systems of translation, which\ndiscard the identity for rare words, in this paper we propose several\narchitectures for learning word representations from character and morpheme\nlevel word decompositions. We incorporate these representations in a novel\nmachine translation model which jointly learns word alignments and translations\nvia a hard attention mechanism. Evaluating on translating from several\nmorphologically rich languages into English, we show consistent improvements\nover strong baseline methods, of between 1 and 1.5 BLEU points.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1606.04429v1",
            "date": "2016-06-14 15:44:52+00:00",
            "title": "Using Fuzzy Logic to Leverage HTML Markup for Web Page Representation",
            "authors": [
                "Alberto P. Garc\u00eda-Plaza",
                "V\u00edctor Fresno",
                "Raquel Mart\u00ednez",
                "Arkaitz Zubiaga"
            ],
            "abstract": "The selection of a suitable document representation approach plays a crucial\nrole in the performance of a document clustering task. Being able to pick out\nrepresentative words within a document can lead to substantial improvements in\ndocument clustering. In the case of web documents, the HTML markup that defines\nthe layout of the content provides additional structural information that can\nbe further exploited to identify representative words. In this paper we\nintroduce a fuzzy term weighing approach that makes the most of the HTML\nstructure for document clustering. We set forth and build on the hypothesis\nthat a good representation can take advantage of how humans skim through\ndocuments to extract the most representative words. The authors of web pages\nmake use of HTML tags to convey the most important message of a web page\nthrough page elements that attract the readers' attention, such as page titles\nor emphasized elements. We define a set of criteria to exploit the information\nprovided by these page elements, and introduce a fuzzy combination of these\ncriteria that we evaluate within the context of a web page clustering task. Our\nproposed approach, called Abstract Fuzzy Combination of Criteria (AFCC), can\nadapt to datasets whose features are distributed differently, achieving good\nresults compared to other similar fuzzy logic based approaches and TF-IDF\nacross different datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.04582v6",
            "date": "2016-06-14 21:54:46+00:00",
            "title": "Query-Reduction Networks for Question Answering",
            "authors": [
                "Minjoon Seo",
                "Sewon Min",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "In this paper, we study the problem of question answering when reasoning over\nmultiple facts is required. We propose Query-Reduction Network (QRN), a variant\nof Recurrent Neural Network (RNN) that effectively handles both short-term\n(local) and long-term (global) sequential dependencies to reason over multiple\nfacts. QRN considers the context sentences as a sequence of state-changing\ntriggers, and reduces the original query to a more informed query as it\nobserves each trigger (context sentence) through time. Our experiments show\nthat QRN produces the state-of-the-art results in bAbI QA and dialog tasks, and\nin a real goal-oriented dialog dataset. In addition, QRN formulation allows\nparallelization on RNN's time axis, saving an order of magnitude in time\ncomplexity for training and inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04631v1",
            "date": "2016-06-15 03:26:53+00:00",
            "title": "Bidirectional Long-Short Term Memory for Video Description",
            "authors": [
                "Yi Bin",
                "Yang Yang",
                "Zi Huang",
                "Fumin Shen",
                "Xing Xu",
                "Heng Tao Shen"
            ],
            "abstract": "Video captioning has been attracting broad research attention in multimedia\ncommunity. However, most existing approaches either ignore temporal information\namong video frames or just employ local contextual temporal knowledge. In this\nwork, we propose a novel video captioning framework, termed as\n\\emph{Bidirectional Long-Short Term Memory} (BiLSTM), which deeply captures\nbidirectional global temporal structure in video. Specifically, we first devise\na joint visual modelling approach to encode video data by combining a forward\nLSTM pass, a backward LSTM pass, together with visual features from\nConvolutional Neural Networks (CNNs). Then, we inject the derived video\nrepresentation into the subsequent language model for initialization. The\nbenefits are in two folds: 1) comprehensively preserving sequential and visual\ninformation; and 2) adaptively learning dense visual features and sparse\nsemantic representations for videos and sentences, respectively. We verify the\neffectiveness of our proposed video captioning framework on a commonly-used\nbenchmark, i.e., Microsoft Video Description (MSVD) corpus, and the\nexperimental results demonstrate that the superiority of the proposed approach\nas compared to several state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1606.04672v2",
            "date": "2016-06-15 08:21:29+00:00",
            "title": "Constitutional Precedent of Amicus Briefs",
            "authors": [
                "Allen Huang",
                "Lars Roemheld"
            ],
            "abstract": "We investigate shared language between U.S. Supreme Court majority opinions\nand interest groups' corresponding amicus briefs. Specifically, we evaluate\nwhether language that originated in an amicus brief acquired legal precedent\nstatus by being cited in the Court's opinion. Using plagiarism detection\nsoftware, automated querying of a large legal database, and manual analysis, we\nestablish seven instances where interest group amici were able to formulate\nconstitutional case law, setting binding legal precedent. We discuss several\nsuch instances for their implications in the Supreme Court's creation of case\nlaw.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04686v1",
            "date": "2016-06-15 09:05:56+00:00",
            "title": "Natural Language Generation as Planning under Uncertainty Using Reinforcement Learning",
            "authors": [
                "Verena Rieser",
                "Oliver Lemon"
            ],
            "abstract": "We present and evaluate a new model for Natural Language Generation (NLG) in\nSpoken Dialogue Systems, based on statistical planning, given noisy feedback\nfrom the current generation context (e.g. a user and a surface realiser). We\nstudy its use in a standard NLG problem: how to present information (in this\ncase a set of search results) to users, given the complex trade- offs between\nutterance length, amount of information conveyed, and cognitive load. We set\nthese trade-offs by analysing existing MATCH data. We then train a NLG pol- icy\nusing Reinforcement Learning (RL), which adapts its behaviour to noisy feed-\nback from the current generation context. This policy is compared to several\nbase- lines derived from previous work in this area. The learned policy\nsignificantly out- performs all the prior approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05286v1",
            "date": "2016-06-16 17:31:13+00:00",
            "title": "Spectral decomposition method of dialog state tracking via collective matrix factorization",
            "authors": [
                "Julien Perez"
            ],
            "abstract": "The task of dialog management is commonly decomposed into two sequential\nsubtasks: dialog state tracking and dialog policy learning. In an end-to-end\ndialog system, the aim of dialog state tracking is to accurately estimate the\ntrue dialog state from noisy observations produced by the speech recognition\nand the natural language understanding modules. The state tracking task is\nprimarily meant to support a dialog policy. From a probabilistic perspective,\nthis is achieved by maintaining a posterior distribution over hidden dialog\nstates composed of a set of context dependent variables. Once a dialog policy\nis learned, it strives to select an optimal dialog act given the estimated\ndialog state and a defined reward function. This paper introduces a novel\nmethod of dialog state tracking based on a bilinear algebric decomposition\nmodel that provides an efficient inference schema through collective matrix\nfactorization. We evaluate the proposed approach on the second Dialog State\nTracking Challenge (DSTC-2) dataset and we show that the proposed tracker gives\nencouraging results compared to the state-of-the-art trackers that participated\nin this standard benchmark. Finally, we show that the prediction schema is\ncomputationally efficient in comparison to the previous approaches.",
            "journal ref": "Dialogue & Discourse 7(3) (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05467v2",
            "date": "2016-06-17 10:24:29+00:00",
            "title": "Gender Inference using Statistical Name Characteristics in Twitter",
            "authors": [
                "Juergen Mueller",
                "Gerd Stumme"
            ],
            "abstract": "Much attention has been given to the task of gender inference of Twitter\nusers. Although names are strong gender indicators, the names of Twitter users\nare rarely used as a feature; probably due to the high number of ill-formed\nnames, which cannot be found in any name dictionary. Instead of relying solely\non a name database, we propose a novel name classifier. Our approach extracts\ncharacteristics from the user names and uses those in order to assign the names\nto a gender. This enables us to classify international first names as well as\nill-formed names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05611v2",
            "date": "2016-06-17 17:52:31+00:00",
            "title": "Data-driven HR - R\u00e9sum\u00e9 Analysis Based on Natural Language Processing and Machine Learning",
            "authors": [
                "Tim Zimmermann",
                "Leo Kotschenreuther",
                "Karsten Schmidt"
            ],
            "abstract": "Recruiters usually spend less than a minute looking at each r\\'esum\\'e when\ndeciding whether it's worth continuing the recruitment process with the\ncandidate. Recruiters focus on keywords, and it's almost impossible to\nguarantee a fair process of candidate selection. The main scope of this paper\nis to tackle this issue by introducing a data-driven approach that shows how to\nprocess r\\'esum\\'es automatically and give recruiters more time to only examine\npromising candidates. Furthermore, we show how to leverage Machine Learning and\nNatural Language Processing in order to extract all required information from\nthe r\\'esum\\'es. Once the information is extracted, a ranking score is\ncalculated. The score describes how well the candidates fit based on their\neducation, work experience and skills. Later this paper illustrates a prototype\napplication that shows how this novel approach can increase the productivity of\nrecruiters. The application enables them to filter and rank candidates based on\npredefined job descriptions. Guided by the ranking, recruiters can get deeper\ninsights from candidate profiles and validate why and how the application\nranked them. This application shows how to improve the hiring process by giving\nan unbiased hiring decision support.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05694v1",
            "date": "2016-06-17 22:32:50+00:00",
            "title": "DeepStance at SemEval-2016 Task 6: Detecting Stance in Tweets Using Character and Word-Level CNNs",
            "authors": [
                "Prashanth Vijayaraghavan",
                "Ivan Sysoev",
                "Soroush Vosoughi",
                "Deb Roy"
            ],
            "abstract": "This paper describes our approach for the Detecting Stance in Tweets task\n(SemEval-2016 Task 6). We utilized recent advances in short text categorization\nusing deep learning to create word-level and character-level models. The choice\nbetween word-level and character-level models in each particular case was\ninformed through validation performance. Our final system is a combination of\nclassifiers using word-level or character-level models. We also employed novel\ndata augmentation techniques to expand and diversify our training dataset, thus\nmaking our system more robust. Our system achieved a macro-average precision,\nrecall and F1-scores of 0.67, 0.61 and 0.635 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06061v2",
            "date": "2016-06-20 10:54:51+00:00",
            "title": "Fast, Compact, and High Quality LSTM-RNN Based Statistical Parametric Speech Synthesizers for Mobile Devices",
            "authors": [
                "Heiga Zen",
                "Yannis Agiomyrgiannakis",
                "Niels Egberts",
                "Fergus Henderson",
                "Przemys\u0142aw Szczepaniak"
            ],
            "abstract": "Acoustic models based on long short-term memory recurrent neural networks\n(LSTM-RNNs) were applied to statistical parametric speech synthesis (SPSS) and\nshowed significant improvements in naturalness and latency over those based on\nhidden Markov models (HMMs). This paper describes further optimizations of\nLSTM-RNN-based SPSS for deployment on mobile devices; weight quantization,\nmulti-frame inference, and robust inference using an {\\epsilon}-contaminated\nGaussian loss function. Experimental results in subjective listening tests show\nthat these optimizations can make LSTM-RNN-based SPSS comparable to HMM-based\nSPSS in runtime speed while maintaining naturalness. Evaluations between\nLSTM-RNN- based SPSS and HMM-driven unit selection speech synthesis are also\npresented.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1606.06086v2",
            "date": "2016-06-20 12:31:13+00:00",
            "title": "Uncertainty in Neural Network Word Embedding: Exploration of Threshold for Similarity",
            "authors": [
                "Navid Rekabsaz",
                "Mihai Lupu",
                "Allan Hanbury"
            ],
            "abstract": "Word embedding, specially with its recent developments, promises a\nquantification of the similarity between terms. However, it is not clear to\nwhich extent this similarity value can be genuinely meaningful and useful for\nsubsequent tasks. We explore how the similarity score obtained from the models\nis really indicative of term relatedness. We first observe and quantify the\nuncertainty factor of the word embedding models regarding to the similarity\nvalue. Based on this factor, we introduce a general threshold on various\ndimensions which effectively filters the highly related terms. Our evaluation\non four information retrieval collections supports the effectiveness of our\napproach as the results of the introduced threshold are significantly better\nthan the baseline while being equal to or statistically indistinguishable from\nthe optimal results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06125v2",
            "date": "2016-06-20 14:05:59+00:00",
            "title": "Introducing a Calculus of Effects and Handlers for Natural Language Semantics",
            "authors": [
                "Jirka Mar\u0161\u00edk",
                "Maxime Amblard"
            ],
            "abstract": "In compositional model-theoretic semantics, researchers assemble\ntruth-conditions or other kinds of denotations using the lambda calculus. It\nwas previously observed that the lambda terms and/or the denotations studied\ntend to follow the same pattern: they are instances of a monad. In this paper,\nwe present an extension of the simply-typed lambda calculus that exploits this\nuniformity using the recently discovered technique of effect handlers. We prove\nthat our calculus exhibits some of the key formal properties of the lambda\ncalculus and we use it to construct a modular semantics for a small fragment\nthat involves multiple distinct semantic phenomena.",
            "journal ref": "Formal Grammar 2016, Aug 2016, Bozen-Bolzano, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06164v2",
            "date": "2016-06-20 15:08:22+00:00",
            "title": "Pragmatic factors in image description: the case of negations",
            "authors": [
                "Emiel van Miltenburg",
                "Roser Morante",
                "Desmond Elliott"
            ],
            "abstract": "We provide a qualitative analysis of the descriptions containing negations\n(no, not, n't, nobody, etc) in the Flickr30K corpus, and a categorization of\nnegation uses. Based on this analysis, we provide a set of requirements that an\nimage description system should have in order to generate negation sentences.\nAs a pilot experiment, we used our categorization to manually annotate\nsentences containing negations in the Flickr30K corpus, with an agreement score\nof K=0.67. With this paper, we hope to open up a broader discussion of\nsubjective language in image descriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06259v2",
            "date": "2016-06-20 19:23:53+00:00",
            "title": "MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos",
            "authors": [
                "Amir Zadeh",
                "Rowan Zellers",
                "Eli Pincus",
                "Louis-Philippe Morency"
            ],
            "abstract": "People are sharing their opinions, stories and reviews through online video\nsharing websites every day. Studying sentiment and subjectivity in these\nopinion videos is experiencing a growing attention from academia and industry.\nWhile sentiment analysis has been successful for text, it is an understudied\nresearch question for videos and multimedia content. The biggest setbacks for\nstudies in this direction are lack of a proper dataset, methodology, baselines\nand statistical analysis of how information from different modality sources\nrelate to each other. This paper introduces to the scientific community the\nfirst opinion-level annotated corpus of sentiment and subjectivity analysis in\nonline videos called Multimodal Opinion-level Sentiment Intensity dataset\n(MOSI). The dataset is rigorously annotated with labels for subjectivity,\nsentiment intensity, per-frame and per-opinion annotated visual features, and\nper-milliseconds annotated audio features. Furthermore, we present baselines\nfor future studies in this direction as well as a new multimodal fusion\napproach that jointly models spoken words and visual gestures.",
            "journal ref": "IEEE Intelligent Systems 31.6 (2016): 82-88",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06461v3",
            "date": "2016-06-21 07:54:35+00:00",
            "title": "Neighborhood Mixture Model for Knowledge Base Completion",
            "authors": [
                "Dat Quoc Nguyen",
                "Kairit Sirts",
                "Lizhen Qu",
                "Mark Johnson"
            ],
            "abstract": "Knowledge bases are useful resources for many natural language processing\ntasks, however, they are far from complete. In this paper, we define a novel\nentity representation as a mixture of its neighborhood in the knowledge base\nand apply this technique on TransE-a well-known embedding model for knowledge\nbase completion. Experimental results show that the neighborhood information\nsignificantly helps to improve the results of the TransE model, leading to\nbetter performance than obtained by other state-of-the-art embedding models on\nthree benchmark datasets for triple classification, entity prediction and\nrelation prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06623v1",
            "date": "2016-06-21 15:39:35+00:00",
            "title": "An empirical study on large scale text classification with skip-gram embeddings",
            "authors": [
                "Georgios Balikas",
                "Massih-Reza Amini"
            ],
            "abstract": "We investigate the integration of word embeddings as classification features\nin the setting of large scale text classification. Such representations have\nbeen used in a plethora of tasks, however their application in classification\nscenarios with thousands of classes has not been extensively researched,\npartially due to hardware limitations. In this work, we examine efficient\ncomposition functions to obtain document-level from word-level embeddings and\nwe subsequently investigate their combination with the traditional\none-hot-encoding representations. By presenting empirical evidence on large,\nmulti-class, multi-label classification problems, we demonstrate the efficiency\nand the performance benefits of this combination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06737v3",
            "date": "2016-06-21 20:00:01+00:00",
            "title": "Criticality in Formal Languages and Statistical Physics",
            "authors": [
                "Henry W. Lin",
                "Max Tegmark"
            ],
            "abstract": "We show that the mutual information between two symbols, as a function of the\nnumber of symbols between the two, decays exponentially in any probabilistic\nregular grammar, but can decay like a power law for a context-free grammar.\nThis result about formal languages is closely related to a well-known result in\nclassical statistical mechanics that there are no phase transitions in\ndimensions fewer than two. It is also related to the emergence of power-law\ncorrelations in turbulence and cosmological inflation through recursive\ngenerative processes. We elucidate these physics connections and comment on\npotential applications of our results to machine learning tasks like training\nartificial recurrent neural networks. Along the way, we introduce a useful\nquantity which we dub the rational mutual information and discuss\ngeneralizations of our claims involving more complicated Bayesian networks.",
            "journal ref": "Entropy, 19, 299 (2017)",
            "category": "cond-mat.dis-nn"
        },
        {
            "url": "http://arxiv.org/abs/1606.06900v2",
            "date": "2016-06-22 11:07:43+00:00",
            "title": "Inferring Logical Forms From Denotations",
            "authors": [
                "Panupong Pasupat",
                "Percy Liang"
            ],
            "abstract": "A core problem in learning semantic parsers from denotations is picking out\nconsistent logical forms--those that yield the correct denotation--from a\ncombinatorially large space. To control the search space, previous work relied\non restricted set of rules, which limits expressivity. In this paper, we\nconsider a much more expressive class of logical forms, and show how to use\ndynamic programming to efficiently represent the complete set of consistent\nlogical forms. Expressivity also introduces many more spurious logical forms\nwhich are consistent with the correct denotation but do not represent the\nmeaning of the utterance. To address this, we generate fictitious worlds and\nuse crowdsourced denotations on these worlds to filter out spurious logical\nforms. On the WikiTableQuestions dataset, we increase the coverage of\nanswerable questions from 53.5% to 76%, and the additional crowdsourced\nsupervision lets us rule out 92.1% of spurious logical forms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06905v2",
            "date": "2016-06-22 11:30:47+00:00",
            "title": "Learning text representation using recurrent convolutional neural network with highway layers",
            "authors": [
                "Ying Wen",
                "Weinan Zhang",
                "Rui Luo",
                "Jun Wang"
            ],
            "abstract": "Recently, the rapid development of word embedding and neural networks has\nbrought new inspiration to various NLP and IR tasks. In this paper, we describe\na staged hybrid model combining Recurrent Convolutional Neural Networks (RCNN)\nwith highway layers. The highway network module is incorporated in the middle\ntakes the output of the bi-directional Recurrent Neural Network (Bi-RNN) module\nin the first stage and provides the Convolutional Neural Network (CNN) module\nin the last stage with the input. The experiment shows that our model\noutperforms common neural network models (CNN, RNN, Bi-RNN) on a sentiment\nanalysis task. Besides, the analysis of how sequence length influences the RCNN\nwith highway layers shows that our model could learn good representation for\nthe long text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06950v2",
            "date": "2016-06-22 13:51:57+00:00",
            "title": "A segmental framework for fully-unsupervised large-vocabulary speech recognition",
            "authors": [
                "Herman Kamper",
                "Aren Jansen",
                "Sharon Goldwater"
            ],
            "abstract": "Zero-resource speech technology is a growing research area that aims to\ndevelop methods for speech processing in the absence of transcriptions,\nlexicons, or language modelling text. Early term discovery systems focused on\nidentifying isolated recurring patterns in a corpus, while more recent\nfull-coverage systems attempt to completely segment and cluster the audio into\nword-like units---effectively performing unsupervised speech recognition. This\narticle presents the first attempt we are aware of to apply such a system to\nlarge-vocabulary multi-speaker data. Our system uses a Bayesian modelling\nframework with segmental word representations: each word segment is represented\nas a fixed-dimensional acoustic embedding obtained by mapping the sequence of\nfeature frames to a single embedding vector. We compare our system on English\nand Xitsonga datasets to state-of-the-art baselines, using a variety of\nmeasures including word error rate (obtained by mapping the unsupervised output\nto ground truth transcriptions). Very high word error rates are reported---in\nthe order of 70--80% for speaker-dependent and 80--95% for speaker-independent\nsystems---highlighting the difficulty of this task. Nevertheless, in terms of\ncluster quality and word segmentation metrics, we show that by imposing a\nconsistent top-down segmentation while also using bottom-up knowledge from\ndetected syllable boundaries, both single-speaker and multi-speaker versions of\nour system outperform a purely bottom-up single-speaker syllable-based\napproach. We also show that the discovered clusters can be made less speaker-\nand gender-specific by using an unsupervised autoencoder-like feature extractor\nto learn better frame-level features (prior to embedding). Our system's\ndiscovered clusters are still less pure than those of unsupervised term\ndiscovery systems, but provide far greater coverage.",
            "journal ref": "Comput. Speech Lang. 46 (2017) 154-174",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06991v1",
            "date": "2016-06-22 15:53:29+00:00",
            "title": "Toward Word Embedding for Personalized Information Retrieval",
            "authors": [
                "Nawal Ould-Amer",
                "Philippe Mulhem",
                "Mathias Gery"
            ],
            "abstract": "This paper presents preliminary works on using Word Embedding (word2vec) for\nquery expansion in the context of Personalized Information Retrieval.\nTraditionally, word embeddings are learned on a general corpus, like Wikipedia.\nIn this work we try to personalize the word embeddings learning, by achieving\nthe learning on the user's profile. The word embeddings are then in the same\ncontext than the user interests. Our proposal is evaluated on the CLEF Social\nBook Search 2016 collection. The results obtained show that some efforts should\nbe made in the way to apply Word Embedding in the context of Personalized\nInformation Retrieval.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07006v3",
            "date": "2016-06-22 16:37:55+00:00",
            "title": "Using Word Embeddings in Twitter Election Classification",
            "authors": [
                "Xiao Yang",
                "Craig Macdonald",
                "Iadh Ounis"
            ],
            "abstract": "Word embeddings and convolutional neural networks (CNN) have attracted\nextensive attention in various classification tasks for Twitter, e.g. sentiment\nclassification. However, the effect of the configuration used to train and\ngenerate the word embeddings on the classification performance has not been\nstudied in the existing literature. In this paper, using a Twitter election\nclassification task that aims to detect election-related tweets, we investigate\nthe impact of the background dataset used to train the embedding models, the\ncontext window size and the dimensionality of word embeddings on the\nclassification performance. By comparing the classification results of two word\nembedding models, which are trained using different background corpora (e.g.\nWikipedia articles and Twitter microposts), we show that the background data\ntype should align with the Twitter classification dataset to achieve a better\nperformance. Moreover, by evaluating the results of word embeddings models\ntrained using various context window sizes and dimensionalities, we found that\nlarge context window and dimension sizes are preferable to improve the\nperformance. Our experimental results also show that using word embeddings and\nCNN leads to statistically significant improvements over various baselines such\nas random, SVM with TF-IDF and SVM with word embeddings.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07103v2",
            "date": "2016-06-22 20:58:08+00:00",
            "title": "Deep Feature Fusion Network for Answer Quality Prediction in Community Question Answering",
            "authors": [
                "Sai Praneeth Suggu",
                "Kushwanth N. Goutham",
                "Manoj K. Chinnakotla",
                "Manish Shrivastava"
            ],
            "abstract": "Community Question Answering (cQA) forums have become a popular medium for\nsoliciting direct answers to specific questions of users from experts or other\nexperienced users on a given topic. However, for a given question, users\nsometimes have to sift through a large number of low-quality or irrelevant\nanswers to find out the answer which satisfies their information need. To\nalleviate this, the problem of Answer Quality Prediction (AQP) aims to predict\nthe quality of an answer posted in response to a forum question. Current AQP\nsystems either learn models using - a) various hand-crafted features (HCF) or\nb) use deep learning (DL) techniques which automatically learn the required\nfeature representations.\n  In this paper, we propose a novel approach for AQP known as - \"Deep Feature\nFusion Network (DFFN)\" which leverages the advantages of both hand-crafted\nfeatures and deep learning based systems. Given a question-answer pair along\nwith its metadata, DFFN independently - a) learns deep features using a\nConvolutional Neural Network (CNN) and b) computes hand-crafted features using\nvarious external resources and then combines them using a deep neural network\ntrained to predict the final answer quality. DFFN achieves state-of-the-art\nperformance on the standard SemEval-2015 and SemEval-2016 benchmark datasets\nand outperforms baseline approaches which individually employ either HCF or DL\nbased techniques alone.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07211v1",
            "date": "2016-06-23 07:21:28+00:00",
            "title": "Toward a Deep Neural Approach for Knowledge-Based IR",
            "authors": [
                "Gia-Hung Nguyen",
                "Lynda Tamine",
                "Laure Soulier",
                "Nathalie Bricon-Souf"
            ],
            "abstract": "This paper tackles the problem of the semantic gap between a document and a\nquery within an ad-hoc information retrieval task. In this context, knowledge\nbases (KBs) have already been acknowledged as valuable means since they allow\nthe representation of explicit relations between entities. However, they do not\nnecessarily represent implicit relations that could be hidden in a corpora.\nThis latter issue is tackled by recent works dealing with deep representation\nlearn ing of texts. With this in mind, we argue that embedding KBs within deep\nneural architectures supporting documentquery matching would give rise to\nfine-grained latent representations of both words and their semantic relations.\nIn this paper, we review the main approaches of neural-based document ranking\nas well as those approaches for latent representation of entities and relations\nvia KBs. We then propose some avenues to incorporate KBs in deep neural\napproaches for document ranking. More particularly, this paper advocates that\nKBs can be used either to support enhanced latent representations of queries\nand documents based on both distributional and relational semantics or to serve\nas a semantic translator between their latent distributional representations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07470v1",
            "date": "2016-06-23 20:37:06+00:00",
            "title": "NN-grams: Unifying neural network and n-gram language models for Speech Recognition",
            "authors": [
                "Babak Damavandi",
                "Shankar Kumar",
                "Noam Shazeer",
                "Antoine Bruguier"
            ],
            "abstract": "We present NN-grams, a novel, hybrid language model integrating n-grams and\nneural networks (NN) for speech recognition. The model takes as input both word\nhistories as well as n-gram counts. Thus, it combines the memorization capacity\nand scalability of an n-gram model with the generalization ability of neural\nnetworks. We report experiments where the model is trained on 26B words.\nNN-grams are efficient at run-time since they do not include an output soft-max\nlayer. The model is trained using noise contrastive estimation (NCE), an\napproach that transforms the estimation problem of neural networks into one of\nbinary classification between data samples and noise samples. We present\nresults with noise samples derived from either an n-gram distribution or from\nspeech recognition lattices. NN-grams outperforms an n-gram model on an Italian\nspeech recognition dictation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07545v1",
            "date": "2016-06-24 02:28:24+00:00",
            "title": "Interactive Semantic Featuring for Text Classification",
            "authors": [
                "Camille Jandot",
                "Patrice Simard",
                "Max Chickering",
                "David Grangier",
                "Jina Suh"
            ],
            "abstract": "In text classification, dictionaries can be used to define\nhuman-comprehensible features. We propose an improvement to dictionary features\ncalled smoothed dictionary features. These features recognize document contexts\ninstead of n-grams. We describe a principled methodology to solicit dictionary\nfeatures from a teacher, and present results showing that models built using\nthese human-comprehensible features are competitive with models trained with\nBag of Words features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07565v1",
            "date": "2016-06-24 04:40:48+00:00",
            "title": "Adaptability of Neural Networks on Varying Granularity IR Tasks",
            "authors": [
                "Daniel Cohen",
                "Qingyao Ai",
                "W. Bruce Croft"
            ],
            "abstract": "Recent work in Information Retrieval (IR) using Deep Learning models has\nyielded state of the art results on a variety of IR tasks. Deep neural networks\n(DNN) are capable of learning ideal representations of data during the training\nprocess, removing the need for independently extracting features. However, the\nstructures of these DNNs are often tailored to perform on specific datasets. In\naddition, IR tasks deal with text at varying levels of granularity from single\nfactoids to documents containing thousands of words. In this paper, we examine\nthe role of the granularity on the performance of common state of the art DNN\nstructures in IR.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07770v3",
            "date": "2016-06-24 17:53:45+00:00",
            "title": "Captioning Images with Diverse Objects",
            "authors": [
                "Subhashini Venugopalan",
                "Lisa Anne Hendricks",
                "Marcus Rohrbach",
                "Raymond Mooney",
                "Trevor Darrell",
                "Kate Saenko"
            ],
            "abstract": "Recent captioning models are limited in their ability to scale and describe\nconcepts unseen in paired image-text corpora. We propose the Novel Object\nCaptioner (NOC), a deep visual semantic captioning model that can describe a\nlarge number of object categories not present in existing image-caption\ndatasets. Our model takes advantage of external sources -- labeled images from\nobject recognition datasets, and semantic knowledge extracted from unannotated\ntext. We propose minimizing a joint objective which can learn from these\ndiverse data sources and leverage distributional semantic embeddings, enabling\nthe model to generalize and describe novel objects outside of image-caption\ndatasets. We demonstrate that our model exploits semantic information to\ngenerate captions for hundreds of object categories in the ImageNet object\nrecognition dataset that are not observed in MSCOCO image-caption training\ndata, as well as many categories that are observed very rarely. Both automatic\nevaluations and human judgements show that our model considerably outperforms\nprior work in being able to describe many more categories of objects.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.07822v1",
            "date": "2016-06-24 20:05:53+00:00",
            "title": "Efficient Parallel Learning of Word2Vec",
            "authors": [
                "Jeroen B. P. Vuurens",
                "Carsten Eickhoff",
                "Arjen P. de Vries"
            ],
            "abstract": "Since its introduction, Word2Vec and its variants are widely used to learn\nsemantics-preserving representations of words or entities in an embedding\nspace, which can be used to produce state-of-art results for various Natural\nLanguage Processing tasks. Existing implementations aim to learn efficiently by\nrunning multiple threads in parallel while operating on a single model in\nshared memory, ignoring incidental memory update collisions. We show that these\ncollisions can degrade the efficiency of parallel learning, and propose a\nstraightforward caching strategy that improves the efficiency by a factor of 4.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07839v3",
            "date": "2016-06-24 21:48:55+00:00",
            "title": "Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles",
            "authors": [
                "Stefan Lee",
                "Senthil Purushwalkam",
                "Michael Cogswell",
                "Viresh Ranjan",
                "David Crandall",
                "Dhruv Batra"
            ],
            "abstract": "Many practical perception systems exist within larger processes that include\ninteractions with users or additional components capable of evaluating the\nquality of predicted solutions. In these contexts, it is beneficial to provide\nthese oracle mechanisms with multiple highly likely hypotheses rather than a\nsingle prediction. In this work, we pose the task of producing multiple outputs\nas a learning problem over an ensemble of deep networks -- introducing a novel\nstochastic gradient descent based approach to minimize the loss with respect to\nan oracle. Our method is simple to implement, agnostic to both architecture and\nloss function, and parameter-free. Our approach achieves lower oracle error\ncompared to existing methods on a wide range of tasks and deep architectures.\nWe also show qualitatively that the diverse solutions produced often provide\ninterpretable representations of task ambiguity.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.07955v1",
            "date": "2016-06-25 20:04:42+00:00",
            "title": "X575: writing rengas with web services",
            "authors": [
                "Daniel Winterstein",
                "Joseph Corneli"
            ],
            "abstract": "Our software system simulates the classical collaborative Japanese poetry\nform, renga, made of linked haikus. We used NLP methods wrapped up as web\nservices. Our experiments were only a partial success, since results fail to\nsatisfy classical constraints. To gather ideas for future work, we examine\nrelated research in semiotics, linguistics, and computing.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.08140v3",
            "date": "2016-06-27 06:50:10+00:00",
            "title": "STransE: a novel embedding model of entities and relationships in knowledge bases",
            "authors": [
                "Dat Quoc Nguyen",
                "Kairit Sirts",
                "Lizhen Qu",
                "Mark Johnson"
            ],
            "abstract": "Knowledge bases of real-world facts about entities and their relationships\nare useful resources for a variety of natural language processing tasks.\nHowever, because knowledge bases are typically incomplete, it is useful to be\nable to perform link prediction or knowledge base completion, i.e., predict\nwhether a relationship not in the knowledge base is likely to be true. This\npaper combines insights from several previous link prediction models into a new\nembedding model STransE that represents each entity as a low-dimensional\nvector, and each relation by two matrices and a translation vector. STransE is\na simple combination of the SE and TransE models, but it obtains better link\nprediction performance on two benchmark datasets than previous embedding\nmodels. Thus, STransE can serve as a new baseline for the more complex models\nin the link prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08689v1",
            "date": "2016-06-28 13:32:08+00:00",
            "title": "Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content",
            "authors": [
                "Nemanja Djuric",
                "Hao Wu",
                "Vladan Radosavljevic",
                "Mihajlo Grbovic",
                "Narayan Bhamidipati"
            ],
            "abstract": "We consider the problem of learning distributed representations for documents\nin data streams. The documents are represented as low-dimensional vectors and\nare jointly learned with distributed vector representations of word tokens\nusing a hierarchical framework with two embedded neural language models. In\nparticular, we exploit the context of documents in streams and use one of the\nlanguage models to model the document sequences, and the other to model word\nsequences within them. The models learn continuous vector representations for\nboth word tokens and documents such that semantically similar documents and\nwords are close in a common vector space. We discuss extensions to our model,\nwhich can be applied to personalized recommendation and social relationship\nmining by adding further user layers to the hierarchy, thus learning\nuser-specific vectors to represent individual preferences. We validated the\nlearned representations on a public movie rating data set from MovieLens, as\nwell as on a large-scale Yahoo News data comprising three months of user\nactivity logs collected on Yahoo servers. The results indicate that the\nproposed model can learn useful representations of both documents and word\ntokens, outperforming the current state-of-the-art by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08954v2",
            "date": "2016-06-29 05:01:56+00:00",
            "title": "Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs",
            "authors": [
                "Swabha Swayamdipta",
                "Miguel Ballesteros",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We present a transition-based parser that jointly produces syntactic and\nsemantic dependencies. It learns a representation of the entire algorithm\nstate, using stack long short-term memories. Our greedy inference algorithm has\nlinear time, including feature extraction. On the CoNLL 2008--9 English shared\ntasks, we obtain the best published parsing performance among models that\njointly learn syntax and semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09058v1",
            "date": "2016-06-29 12:08:51+00:00",
            "title": "A Distributional Semantics Approach to Implicit Language Learning",
            "authors": [
                "Dimitrios Alikaniotis",
                "John N. Williams"
            ],
            "abstract": "In the present paper we show that distributional information is particularly\nimportant when considering concept availability under implicit language\nlearning conditions. Based on results from different behavioural experiments we\nargue that the implicit learnability of semantic regularities depends on the\ndegree to which the relevant concept is reflected in language use. In our\nsimulations, we train a Vector-Space model on either an English or a Chinese\ncorpus and then feed the resulting representations to a feed-forward neural\nnetwork. The task of the neural network was to find a mapping between the word\nrepresentations and the novel words. Using datasets from four behavioural\nexperiments, which used different semantic manipulations, we were able to\nobtain learning patterns very similar to those obtained by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09403v1",
            "date": "2016-06-30 09:18:53+00:00",
            "title": "Learning Crosslingual Word Embeddings without Bilingual Corpora",
            "authors": [
                "Long Duong",
                "Hiroshi Kanayama",
                "Tengfei Ma",
                "Steven Bird",
                "Trevor Cohn"
            ],
            "abstract": "Crosslingual word embeddings represent lexical items from different languages\nin the same vector space, enabling transfer of NLP tools. However, previous\nattempts had expensive resource requirements, difficulty incorporating\nmonolingual data or were unable to handle polysemy. We address these drawbacks\nin our method which takes advantage of a high coverage dictionary in an EM\nstyle training algorithm over monolingual corpora in two languages. Our model\nachieves state-of-the-art performance on bilingual lexicon induction task\nexceeding models using large bilingual corpora, and competitive results on the\nmonolingual word similarity and cross-lingual document classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00272v3",
            "date": "2016-07-31 22:21:42+00:00",
            "title": "Modeling Context in Referring Expressions",
            "authors": [
                "Licheng Yu",
                "Patrick Poirson",
                "Shan Yang",
                "Alexander C. Berg",
                "Tamara L. Berg"
            ],
            "abstract": "Humans refer to objects in their environments all the time, especially in\ndialogue with other people. We explore generating and comprehending natural\nlanguage referring expressions for objects in images. In particular, we focus\non incorporating better measures of visual context into referring expression\nmodels and find that visual comparison to other objects within an image helps\nimprove performance significantly. We also develop methods to tie the language\ngeneration process together, so that we generate expressions for all objects of\na particular category jointly. Evaluation on three recent datasets - RefCOCO,\nRefCOCO+, and RefCOCOg, shows the advantages of our methods for both referring\nexpression generation and comprehension.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1608.00318v2",
            "date": "2016-08-01 04:42:49+00:00",
            "title": "A Neural Knowledge Language Model",
            "authors": [
                "Sungjin Ahn",
                "Heeyoul Choi",
                "Tanel P\u00e4rnamaa",
                "Yoshua Bengio"
            ],
            "abstract": "Current language models have a significant limitation in the ability to\nencode and decode factual knowledge. This is mainly because they acquire such\nknowledge from statistical co-occurrences although most of the knowledge words\nare rarely observed. In this paper, we propose a Neural Knowledge Language\nModel (NKLM) which combines symbolic knowledge provided by the knowledge graph\nwith the RNN language model. By predicting whether the word to generate has an\nunderlying fact or not, the model can generate such knowledge-related words by\ncopying from the description of the predicted fact. In experiments, we show\nthat the NKLM significantly improves the performance while generating a much\nsmaller number of unknown words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00470v2",
            "date": "2016-08-01 15:27:16+00:00",
            "title": "Labeling Topics with Images using Neural Networks",
            "authors": [
                "Nikolaos Aletras",
                "Arpit Mittal"
            ],
            "abstract": "Topics generated by topic models are usually represented by lists of $t$\nterms or alternatively using short phrases and images. The current\nstate-of-the-art work on labeling topics using images selects images by\nre-ranking a small set of candidates for a given topic. In this paper, we\npresent a more generic method that can estimate the degree of association\nbetween any arbitrary pair of an unseen topic and image using a deep neural\nnetwork. Our method has better runtime performance $O(n)$ compared to $O(n^2)$\nfor the current state-of-the-art method, and is also significantly more\naccurate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00938v1",
            "date": "2016-08-02 19:04:58+00:00",
            "title": "Evolutionary forces in language change",
            "authors": [
                "Christopher A. Ahern",
                "Mitchell G. Newberry",
                "Robin Clark",
                "Joshua B. Plotkin"
            ],
            "abstract": "Languages and genes are both transmitted from generation to generation, with\nopportunity for differential reproduction and survivorship of forms. Here we\napply a rigorous inference framework, drawn from population genetics, to\ndistinguish between two broad mechanisms of language change: drift and\nselection. Drift is change that results from stochasticity in transmission and\nit may occur in the absence of any intrinsic difference between linguistic\nforms; whereas selection is truly an evolutionary force arising from intrinsic\ndifferences -- for example, when one form is preferred by members of the\npopulation. Using large corpora of parsed texts spanning the 12th century to\nthe 21st century, we analyze three examples of grammatical changes in English:\nthe regularization of past-tense verbs, the rise of the periphrastic `do', and\nsyntactic variation in verbal negation. We show that we can reject stochastic\ndrift in favor of a selective force driving some of these language changes, but\nnot others. The strength of drift depends on a word's frequency, and so drift\nprovides an alternative explanation for why some words are more prone to change\nthan others. Our results suggest an important role for stochasticity in\nlanguage change, and they provide a null model against which selective theories\nof language evolution must be compared.",
            "journal ref": "None",
            "category": "q-bio.PE"
        },
        {
            "url": "http://arxiv.org/abs/1608.01238v1",
            "date": "2016-08-03 16:12:23+00:00",
            "title": "Improving Quality of Hierarchical Clustering for Large Data Series",
            "authors": [
                "Manuel R. Ciosici"
            ],
            "abstract": "Brown clustering is a hard, hierarchical, bottom-up clustering of words in a\nvocabulary. Words are assigned to clusters based on their usage pattern in a\ngiven corpus. The resulting clusters and hierarchical structure can be used in\nconstructing class-based language models and for generating features to be used\nin NLP tasks. Because of its high computational cost, the most-used version of\nBrown clustering is a greedy algorithm that uses a window to restrict its\nsearch space. Like other clustering algorithms, Brown clustering finds a\nsub-optimal, but nonetheless effective, mapping of words to clusters. Because\nof its ability to produce high-quality, human-understandable cluster, Brown\nclustering has seen high uptake the NLP research community where it is used in\nthe preprocessing and feature generation steps.\n  Little research has been done towards improving the quality of Brown\nclusters, despite the greedy and heuristic nature of the algorithm. The\napproaches tried so far have focused on: studying the effect of the\ninitialisation in a similar algorithm; tuning the parameters used to define the\ndesired number of clusters and the behaviour of the algorithm; and including a\nseparate parameter to differentiate the window from the desired number of\nclusters. However, some of these approaches have not yielded significant\nimprovements in cluster quality.\n  In this thesis, a close analysis of the Brown algorithm is provided,\nrevealing important under-specifications and weaknesses in the original\nalgorithm. These have serious effects on cluster quality and reproducibility of\nresearch using Brown clustering. In the second part of the thesis, two\nmodifications are proposed. Finally, a thorough evaluation is performed,\nconsidering both the optimization criterion of Brown clustering and the\nperformance of the resulting class-based language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01247v2",
            "date": "2016-08-03 16:33:32+00:00",
            "title": "Query Clustering using Segment Specific Context Embeddings",
            "authors": [
                "S. K Kolluru",
                "Prasenjit Mukherjee"
            ],
            "abstract": "This paper presents a novel query clustering approach to capture the broad\ninterest areas of users querying search engines. We make use of recent advances\nin NLP - word2vec and extend it to get query2vec, vector representations of\nqueries, based on query contexts, obtained from the top search results for the\nquery and use a highly scalable Divide & Merge clustering algorithm on top of\nthe query vectors, to get the clusters. We have tried this approach on a\nvariety of segments, including Retail, Travel, Health, Phones and found the\nclusters to be effective in discovering user's interest areas which have high\nmonetization potential.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.01281v1",
            "date": "2016-08-03 18:35:12+00:00",
            "title": "Learning Online Alignments with Continuous Rewards Policy Gradient",
            "authors": [
                "Yuping Luo",
                "Chung-Cheng Chiu",
                "Navdeep Jaitly",
                "Ilya Sutskever"
            ],
            "abstract": "Sequence-to-sequence models with soft attention had significant success in\nmachine translation, speech recognition, and question answering. Though capable\nand easy to use, they require that the entirety of the input sequence is\navailable at the beginning of inference, an assumption that is not valid for\ninstantaneous translation and speech recognition. To address this problem, we\npresent a new method for solving sequence-to-sequence problems using hard\nonline alignments instead of soft offline alignments. The online alignments\nmodel is able to start producing outputs without the need to first process the\nentire input sequence. A highly accurate online sequence-to-sequence model is\nuseful because it can be used to build an accurate voice-based instantaneous\ntranslator. Our model uses hard binary stochastic decisions to select the\ntimesteps at which outputs will be produced. The model is trained to produce\nthese stochastic decisions using a standard policy gradient method. In our\nexperiments, we show that this model achieves encouraging performance on TIMIT\nand Wall Street Journal (WSJ) speech recognition datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1608.01406v1",
            "date": "2016-08-04 00:37:16+00:00",
            "title": "Quantum Algorithms for Compositional Natural Language Processing",
            "authors": [
                "William Zeng",
                "Bob Coecke"
            ],
            "abstract": "We propose a new application of quantum computing to the field of natural\nlanguage processing. Ongoing work in this field attempts to incorporate\ngrammatical structure into algorithms that compute meaning. In (Coecke,\nSadrzadeh and Clark, 2010), the authors introduce such a model (the CSC model)\nbased on tensor product composition. While this algorithm has many advantages,\nits implementation is hampered by the large classical computational resources\nthat it requires. In this work we show how computational shortcomings of the\nCSC approach could be resolved using quantum computation (possibly in addition\nto existing techniques for dimension reduction). We address the value of\nquantum RAM (Giovannetti,2008) for this model and extend an algorithm from\nWiebe, Braun and Lloyd (2012) into a quantum algorithm to categorize sentences\nin CSC. Our new algorithm demonstrates a quadratic speedup over classical\nmethods under certain conditions.",
            "journal ref": "EPTCS 221, 2016, pp. 67-75",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01884v2",
            "date": "2016-08-05 13:39:08+00:00",
            "title": "Winograd Schemas and Machine Translation",
            "authors": [
                "Ernest Davis"
            ],
            "abstract": "A Winograd schema is a pair of sentences that differ in a single word and\nthat contain an ambiguous pronoun whose referent is different in the two\nsentences and requires the use of commonsense knowledge or world knowledge to\ndisambiguate. This paper discusses how Winograd schemas and other sentence\npairs could be used as challenges for machine translation using distinctions\nbetween pronouns, such as gender, that appear in the target language but not in\nthe source.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.01961v1",
            "date": "2016-08-05 18:14:19+00:00",
            "title": "De-Conflated Semantic Representations",
            "authors": [
                "Mohammad Taher Pilehvar",
                "Nigel Collier"
            ],
            "abstract": "One major deficiency of most semantic representation techniques is that they\nusually model a word type as a single point in the semantic space, hence\nconflating all the meanings that the word can have. Addressing this issue by\nlearning distinct representations for individual meanings of words has been the\nsubject of several research studies in the past few years. However, the\ngenerated sense representations are either not linked to any sense inventory or\nare unreliable for infrequent word senses. We propose a technique that tackles\nthese problems by de-conflating the representations of words based on the deep\nknowledge it derives from a semantic network. Our approach provides multiple\nadvantages in comparison to the past work, including its high coverage and the\nability to generate accurate representations even for infrequent word senses.\nWe carry out evaluations on six datasets across two semantic similarity tasks\nand report state-of-the-art results on most of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01972v2",
            "date": "2016-08-05 18:53:42+00:00",
            "title": "Bridging the Gap: Incorporating a Semantic Similarity Measure for Effectively Mapping PubMed Queries to Documents",
            "authors": [
                "Sun Kim",
                "Nicolas Fiorini",
                "W. John Wilbur",
                "Zhiyong Lu"
            ],
            "abstract": "The main approach of traditional information retrieval (IR) is to examine how\nmany words from a query appear in a document. A drawback of this approach,\nhowever, is that it may fail to detect relevant documents where no or only few\nwords from a query are found. The semantic analysis methods such as LSA (latent\nsemantic analysis) and LDA (latent Dirichlet allocation) have been proposed to\naddress the issue, but their performance is not superior compared to common IR\napproaches. Here we present a query-document similarity measure motivated by\nthe Word Mover's Distance. Unlike other similarity measures, the proposed\nmethod relies on neural word embeddings to compute the distance between words.\nThis process helps identify related words when no direct matches are found\nbetween a query and a document. Our method is efficient and straightforward to\nimplement. The experimental results on TREC Genomics data show that our\napproach outperforms the BM25 ranking function by an average of 12% in mean\naverage precision. Furthermore, for a real-world dataset collected from the\nPubMed search logs, we combine the semantic measure with BM25 using a learning\nto rank method, which leads to improved ranking scores by up to 25%. This\nexperiment demonstrates that the proposed approach and BM25 nicely complement\neach other and together produce superior performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02071v1",
            "date": "2016-08-06 06:24:59+00:00",
            "title": "Transferring Knowledge from Text to Predict Disease Onset",
            "authors": [
                "Yun Liu",
                "Kun-Ta Chuang",
                "Fu-Wen Liang",
                "Huey-Jen Su",
                "Collin M. Stultz",
                "John V. Guttag"
            ],
            "abstract": "In many domains such as medicine, training data is in short supply. In such\ncases, external knowledge is often helpful in building predictive models. We\npropose a novel method to incorporate publicly available domain expertise to\nbuild accurate models. Specifically, we use word2vec models trained on a\ndomain-specific corpus to estimate the relevance of each feature's text\ndescription to the prediction problem. We use these relevance estimates to\nrescale the features, causing more important features to experience weaker\nregularization.\n  We apply our method to predict the onset of five chronic diseases in the next\nfive years in two genders and two age groups. Our rescaling approach improves\nthe accuracy of the model, particularly when there are few positive examples.\nFurthermore, our method selects 60% fewer features, easing interpretation by\nphysicians. Our method is applicable to other domains where feature and outcome\ndescriptions are available.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1608.02153v2",
            "date": "2016-08-06 20:51:53+00:00",
            "title": "OCR of historical printings with an application to building diachronic corpora: A case study using the RIDGES herbal corpus",
            "authors": [
                "U. Springmann",
                "A. L\u00fcdeling"
            ],
            "abstract": "This article describes the results of a case study that applies Neural\nNetwork-based Optical Character Recognition (OCR) to scanned images of books\nprinted between 1487 and 1870 by training the OCR engine OCRopus\n[@breuel2013high] on the RIDGES herbal text corpus [@OdebrechtEtAlSubmitted].\nTraining specific OCR models was possible because the necessary *ground truth*\nis available as error-corrected diplomatic transcriptions. The OCR results have\nbeen evaluated for accuracy against the ground truth of unseen test sets.\nCharacter and word accuracies (percentage of correctly recognized items) for\nthe resulting machine-readable texts of individual documents range from 94% to\nmore than 99% (character level) and from 76% to 97% (word level). This includes\nthe earliest printed books, which were thought to be inaccessible by OCR\nmethods until recently. Furthermore, OCR models trained on one part of the\ncorpus consisting of books with different printing dates and different typesets\n*(mixed models)* have been tested for their predictive power on the books from\nthe other part containing yet other fonts, mostly yielding character accuracies\nwell above 90%. It therefore seems possible to construct generalized models\ntrained on a range of fonts that can be applied to a wide variety of historical\nprintings still giving good results. A moderate postcorrection effort of some\npages will then enable the training of individual models with even better\naccuracies. Using this method, diachronic corpora including early printings can\nbe constructed much faster and cheaper than by manual transcription. The OCR\nmethods reported here open up the possibility of transforming our printed\ntextual cultural heritage into electronic text by largely automatic means,\nwhich is a prerequisite for the mass conversion of scanned books.",
            "journal ref": "Digital Humanities Quarterly 11, 2 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02195v1",
            "date": "2016-08-07 09:04:59+00:00",
            "title": "Automating Political Bias Prediction",
            "authors": [
                "Felix Biessmann"
            ],
            "abstract": "Every day media generate large amounts of text. An unbiased view on media\nreports requires an understanding of the political bias of media content.\nAssistive technology for estimating the political bias of texts can be helpful\nin this context. This study proposes a simple statistical learning approach to\npredict political bias from text. Standard text features extracted from\nspeeches and manifestos of political parties are used to predict political bias\nin terms of political party affiliation and in terms of political views.\nResults indicate that political bias can be predicted with above chance\naccuracy. Mistakes of the model can be interpreted with respect to changes of\npolicies of political actors. Two approaches are presented to make the results\nmore interpretable: a) discriminative text features are related to the\npolitical orientation of a party and b) sentiment features of texts are\ncorrelated with a measure of political power. Political power appears to be\nstrongly correlated with positive sentiment of a text. To highlight some\npotential use cases a web application shows how the model can be used for texts\nfor which the political bias is not clear such as news articles.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1608.02254v2",
            "date": "2016-08-07 18:17:02+00:00",
            "title": "Reconciling Lambek's restriction, cut-elimination, and substitution in the presence of exponential modalities",
            "authors": [
                "Max Kanovich",
                "Stepan Kuznetsov",
                "Andre Scedrov"
            ],
            "abstract": "The Lambek calculus can be considered as a version of non-commutative\nintuitionistic linear logic. One of the interesting features of the Lambek\ncalculus is the so-called \"Lambek's restriction,\" that is, the antecedent of\nany provable sequent should be non-empty. In this paper we discuss ways of\nextending the Lambek calculus with the linear logic exponential modality while\nkeeping Lambek's restriction. Interestingly enough, we show that for any system\nequipped with a reasonable exponential modality the following holds: if the\nsystem enjoys cut elimination and substitution to the full extent, then the\nsystem necessarily violates Lambek's restriction. Nevertheless, we show that\ntwo of the three conditions can be implemented. Namely, we design a system with\nLambek's restriction and cut elimination and another system with Lambek's\nrestriction and substitution. For both calculi we prove that they are\nundecidable, even if we take only one of the two divisions provided by the\nLambek calculus. The system with cut elimination and substitution and without\nLambek's restriction is folklore and known to be undecidable.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1608.02272v1",
            "date": "2016-08-07 21:43:08+00:00",
            "title": "Incorporation of Speech Duration Information in Score Fusion of Speaker Recognition Systems",
            "authors": [
                "Ali Khodabakhsh",
                "Seyyed Saeed Sarfjoo",
                "Umut Uludag",
                "Osman Soyyigit",
                "Cenk Demiroglu"
            ],
            "abstract": "In recent years identity-vector (i-vector) based speaker verification (SV)\nsystems have become very successful. Nevertheless, environmental noise and\nspeech duration variability still have a significant effect on degrading the\nperformance of these systems. In many real-life applications, duration of\nrecordings are very short; as a result, extracted i-vectors cannot reliably\nrepresent the attributes of the speaker. Here, we investigate the effect of\nspeech duration on the performance of three state-of-the-art speaker\nrecognition systems. In addition, using a variety of available score fusion\nmethods, we investigate the effect of score fusion for those speaker\nverification techniques to benefit from the performance difference of different\nmethods under different enrollment and test speech duration conditions. This\ntechnique performed significantly better than the baseline score fusion\nmethods.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1608.02519v1",
            "date": "2016-08-08 17:03:03+00:00",
            "title": "Topic Modelling and Event Identification from Twitter Textual Data",
            "authors": [
                "Marina Sokolova",
                "Kanyi Huang",
                "Stan Matwin",
                "Joshua Ramisch",
                "Vera Sazonova",
                "Renee Black",
                "Chris Orwa",
                "Sidney Ochieng",
                "Nanjira Sambuli"
            ],
            "abstract": "The tremendous growth of social media content on the Internet has inspired\nthe development of the text analytics to understand and solve real-life\nproblems. Leveraging statistical topic modelling helps researchers and\npractitioners in better comprehension of textual content as well as provides\nuseful information for further analysis. Statistical topic modelling becomes\nespecially important when we work with large volumes of dynamic text, e.g.,\nFacebook or Twitter datasets. In this study, we summarize the message content\nof four data sets of Twitter messages relating to challenging social events in\nKenya. We use Latent Dirichlet Allocation (LDA) topic modelling to analyze the\ncontent. Our study uses two evaluation measures, Normalized Mutual Information\n(NMI) and topic coherence analysis, to select the best LDA models. The obtained\nLDA results show that the tool can be effectively used to extract discussion\ntopics and summarize them for further manual analysis",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1608.02689v2",
            "date": "2016-08-09 04:38:38+00:00",
            "title": "Multi-task Domain Adaptation for Sequence Tagging",
            "authors": [
                "Nanyun Peng",
                "Mark Dredze"
            ],
            "abstract": "Many domain adaptation approaches rely on learning cross domain shared\nrepresentations to transfer the knowledge learned in one domain to other\ndomains. Traditional domain adaptation only considers adapting for one task. In\nthis paper, we explore multi-task representation learning under the domain\nadaptation scenario. We propose a neural network framework that supports domain\nadaptation for multiple tasks simultaneously, and learns shared representations\nthat better generalize for domain adaptation. We apply the proposed framework\nto domain adaptation for sequence tagging problems considering two tasks:\nChinese word segmentation and named entity recognition. Experiments show that\nmulti-task domain adaptation works better than disjoint domain adaptation for\neach task, and achieves the state-of-the-art results for both tasks in the\nsocial media domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02904v4",
            "date": "2016-08-09 18:29:04+00:00",
            "title": "TweeTime: A Minimally Supervised Method for Recognizing and Normalizing Time Expressions in Twitter",
            "authors": [
                "Jeniya Tabassum",
                "Alan Ritter",
                "Wei Xu"
            ],
            "abstract": "We describe TweeTIME, a temporal tagger for recognizing and normalizing time\nexpressions in Twitter. Most previous work in social media analysis has to rely\non temporal resolvers that are designed for well-edited text, and therefore\nsuffer from the reduced performance due to domain mismatch. We present a\nminimally supervised method that learns from large quantities of unlabeled data\nand requires no hand-engineered rules or hand-annotated training corpora.\nTweeTIME achieves 0.68 F1 score on the end-to-end task of resolving date\nexpressions, outperforming a broad range of state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.03000v1",
            "date": "2016-08-09 23:05:03+00:00",
            "title": "Neural Generation of Regular Expressions from Natural Language with Minimal Domain Knowledge",
            "authors": [
                "Nicholas Locascio",
                "Karthik Narasimhan",
                "Eduardo DeLeon",
                "Nate Kushman",
                "Regina Barzilay"
            ],
            "abstract": "This paper explores the task of translating natural language queries into\nregular expressions which embody their meaning. In contrast to prior work, the\nproposed neural model does not utilize domain-specific crafting, learning to\ntranslate directly from a parallel corpus. To fully explore the potential of\nneural models, we propose a methodology for collecting a large corpus of\nregular expression, natural language pairs. Our resulting model achieves a\nperformance gain of 19.6% over previous state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03192v1",
            "date": "2016-08-10 14:46:15+00:00",
            "title": "Growing Graphs with Hyperedge Replacement Graph Grammars",
            "authors": [
                "Salvador Agui\u00f1aga",
                "Rodrigo Palacios",
                "David Chiang",
                "Tim Weninger"
            ],
            "abstract": "Discovering the underlying structures present in large real world graphs is a\nfundamental scientific problem. In this paper we show that a graph's clique\ntree can be used to extract a hyperedge replacement grammar. If we store an\nordering from the extraction process, the extracted graph grammar is guaranteed\nto generate an isomorphic copy of the original graph. Or, a stochastic\napplication of the graph grammar rules can be used to quickly create random\ngraphs. In experiments on large real world networks, we show that random\ngraphs, generated from extracted graph grammars, exhibit a wide range of\nproperties that are very similar to the original graphs. In addition to graph\nproperties like degree or eigenvector centrality, what a graph \"looks like\"\nultimately depends on small details in local graph substructures that are\ndifficult to define at a global level. We show that our generative graph model\nis able to preserve these local substructures when generating new graphs and\nperforms well on new and difficult tests of model robustness.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1608.03764v1",
            "date": "2016-08-12 11:58:00+00:00",
            "title": "Extracting Biological Pathway Models From NLP Event Representations",
            "authors": [
                "Michael Spranger",
                "Sucheendra K. Palaniappan",
                "Samik Ghosh"
            ],
            "abstract": "This paper describes an an open-source software system for the automatic\nconversion of NLP event representations to system biology structured data\ninterchange formats such as SBML and BioPAX. It is part of a larger effort to\nmake results of the NLP community available for system biology pathway\nmodelers.",
            "journal ref": "Proceedings of the 2015 Workshop on Biomedical Natural Language\n  Processing (BioNLP 2015), pages 42-51. Association for Computational\n  Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03767v1",
            "date": "2016-08-12 12:10:24+00:00",
            "title": "Measuring the State of the Art of Automated Pathway Curation Using Graph Algorithms - A Case Study of the mTOR Pathway",
            "authors": [
                "Michael Spranger",
                "Sucheendra K. Palaniappan",
                "Samik Ghosh"
            ],
            "abstract": "This paper evaluates the difference between human pathway curation and\ncurrent NLP systems. We propose graph analysis methods for quantifying the gap\nbetween human curated pathway maps and the output of state-of-the-art automatic\nNLP systems. Evaluation is performed on the popular mTOR pathway. Based on\nanalyzing where current systems perform well and where they fail, we identify\npossible avenues for progress.",
            "journal ref": "Proceedings of the 15th Workshop on Biomedical Natural Language\n  Processing, Berlin, Germany, 2016, pages 119-127. Association for\n  Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04020v2",
            "date": "2016-08-13 19:30:12+00:00",
            "title": "Undecidability of the Lambek calculus with subexponential and bracket modalities",
            "authors": [
                "Max Kanovich",
                "Stepan Kuznetsov",
                "Andre Scedrov"
            ],
            "abstract": "The Lambek calculus is a well-known logical formalism for modelling natural\nlanguage syntax. The original calculus covered a substantial number of\nintricate natural language phenomena, but only those restricted to the\ncontext-free setting. In order to address more subtle linguistic issues, the\nLambek calculus has been extended in various ways. In particular, Morrill and\nValentin (2015) introduce an extension with so-called exponential and bracket\nmodalities. Their extension is based on a non-standard contraction rule for the\nexponential that interacts with the bracket structure in an intricate way. The\nstandard contraction rule is not admissible in this calculus. In this paper we\nprove undecidability of the derivability problem in their calculus. We also\ninvestigate restricted decidable fragments considered by Morrill and Valentin\nand we show that these fragments belong to the NP class.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1608.04147v1",
            "date": "2016-08-14 22:34:22+00:00",
            "title": "Numerically Grounded Language Models for Semantic Error Correction",
            "authors": [
                "Georgios P. Spithourakis",
                "Isabelle Augenstein",
                "Sebastian Riedel"
            ],
            "abstract": "Semantic error detection and correction is an important task for applications\nsuch as fact checking, speech-to-text or grammatical error correction. Current\napproaches generally focus on relatively shallow semantics and do not account\nfor numeric quantities. Our approach uses language models grounded in numbers\nwithin the text. Such groundings are easily achieved for recurrent neural\nlanguage model architectures, which can be further conditioned on incomplete\nbackground knowledge bases. Our evaluation on clinical reports shows that\nnumerical grounding improves perplexity by 33% and F1 for semantic error\ncorrection by 5 points when compared to ungrounded approaches. Conditioning on\na knowledge base yields further improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04670v1",
            "date": "2016-08-15 03:34:13+00:00",
            "title": "Attribute Extraction from Product Titles in eCommerce",
            "authors": [
                "Ajinkya More"
            ],
            "abstract": "This paper presents a named entity extraction system for detecting attributes\nin product titles of eCommerce retailers like Walmart. The absence of syntactic\nstructure in such short pieces of text makes extracting attribute values a\nchallenging problem. We find that combining sequence labeling algorithms such\nas Conditional Random Fields and Structured Perceptron with a curated\nnormalization scheme produces an effective system for the task of extracting\nproduct attribute values from titles. To keep the discussion concrete, we will\nillustrate the mechanics of the system from the point of view of a particular\nattribute - brand. We also discuss the importance of an attribute extraction\nsystem in the context of retail websites with large product catalogs, compare\nour approach to other potential approaches to this problem and end the paper\nwith a discussion of the performance of our system for extracting attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04738v2",
            "date": "2016-08-16 07:44:02+00:00",
            "title": "An Efficient Character-Level Neural Machine Translation",
            "authors": [
                "Shenjian Zhao",
                "Zhihua Zhang"
            ],
            "abstract": "Neural machine translation aims at building a single large neural network\nthat can be trained to maximize translation performance. The encoder-decoder\narchitecture with an attention mechanism achieves a translation performance\ncomparable to the existing state-of-the-art phrase-based systems on the task of\nEnglish-to-French translation. However, the use of large vocabulary becomes the\nbottleneck in both training and improving the performance. In this paper, we\npropose an efficient architecture to train a deep character-level neural\nmachine translation by introducing a decimator and an interpolator. The\ndecimator is used to sample the source sequence before encoding while the\ninterpolator is used to resample after decoding. Such a deep model has two\nmajor advantages. It avoids the large vocabulary issue radically; at the same\ntime, it is much faster and more memory-efficient in training than conventional\ncharacter-based models. More interestingly, our model is able to translate the\nmisspelled word like human beings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04808v2",
            "date": "2016-08-16 23:37:43+00:00",
            "title": "Learning Latent Local Conversation Modes for Predicting Community Endorsement in Online Discussions",
            "authors": [
                "Hao Fang",
                "Hao Cheng",
                "Mari Ostendorf"
            ],
            "abstract": "Many social media platforms offer a mechanism for readers to react to\ncomments, both positively and negatively, which in aggregate can be thought of\nas community endorsement. This paper addresses the problem of predicting\ncommunity endorsement in online discussions, leveraging both the participant\nresponse structure and the text of the comment. The different types of features\nare integrated in a neural network that uses a novel architecture to learn\nlatent modes of discussion structure that perform as well as deep neural\nnetworks but are more interpretable. In addition, the latent modes can be used\nto weight text features thereby improving prediction accuracy.",
            "journal ref": "SocialNLP Workshop at Conf. Empirical Methods Natural Language\n  Process. (EMNLP), 2016",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1608.05813v5",
            "date": "2016-08-20 12:12:09+00:00",
            "title": "phi-LSTM: A Phrase-based Hierarchical LSTM Model for Image Captioning",
            "authors": [
                "Ying Hua Tan",
                "Chee Seng Chan"
            ],
            "abstract": "A picture is worth a thousand words. Not until recently, however, we noticed\nsome success stories in understanding of visual scenes: a model that is able to\ndetect/name objects, describe their attributes, and recognize their\nrelationships/interactions. In this paper, we propose a phrase-based\nhierarchical Long Short-Term Memory (phi-LSTM) model to generate image\ndescription. The proposed model encodes sentence as a sequence of combination\nof phrases and words, instead of a sequence of words alone as in those\nconventional solutions. The two levels of this model are dedicated to i) learn\nto generate image relevant noun phrases, and ii) produce appropriate image\ndescription from the phrases and other words in the corpus. Adopting a\nconvolutional neural network to learn image features and the LSTM to learn the\nword sequence in a sentence, the proposed model has shown better or competitive\nresults in comparison to the state-of-the-art models on Flickr8k and Flickr30k\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.05852v1",
            "date": "2016-08-20 17:34:38+00:00",
            "title": "Learning Word Embeddings from Intrinsic and Extrinsic Views",
            "authors": [
                "Jifan Chen",
                "Kan Chen",
                "Xipeng Qiu",
                "Qi Zhang",
                "Xuanjing Huang",
                "Zheng Zhang"
            ],
            "abstract": "While word embeddings are currently predominant for natural language\nprocessing, most of existing models learn them solely from their contexts.\nHowever, these context-based word embeddings are limited since not all words'\nmeaning can be learned based on only context. Moreover, it is also difficult to\nlearn the representation of the rare words due to data sparsity problem. In\nthis work, we address these issues by learning the representations of words by\nintegrating their intrinsic (descriptive) and extrinsic (contextual)\ninformation. To prove the effectiveness of our model, we evaluate it on four\ntasks, including word similarity, reverse dictionaries,Wiki link prediction,\nand document classification. Experiment results show that our model is powerful\nin both word and document modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06459v1",
            "date": "2016-08-23 10:55:23+00:00",
            "title": "Tracking Amendments to Legislation and Other Political Texts with a Novel Minimum-Edit-Distance Algorithm: DocuToads",
            "authors": [
                "Henrik Hermansson",
                "James P. Cross"
            ],
            "abstract": "Political scientists often find themselves tracking amendments to political\ntexts. As different actors weigh in, texts change as they are drafted and\nredrafted, reflecting political preferences and power. This study provides a\nnovel solution to the prob- lem of detecting amendments to political text based\nupon minimum edit distances. We demonstrate the usefulness of two\nlanguage-insensitive, transparent, and efficient minimum-edit-distance\nalgorithms suited for the task. These algorithms are capable of providing an\naccount of the types (insertions, deletions, substitutions, and trans-\npositions) and substantive amount of amendments made between version of texts.\nTo illustrate the usefulness and efficiency of the approach we replicate two\nexisting stud- ies from the field of legislative studies. Our results\ndemonstrate that minimum edit distance methods can produce superior measures of\ntext amendments to hand-coded efforts in a fraction of the time and resource\ncosts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06549v1",
            "date": "2016-08-23 15:34:55+00:00",
            "title": "Using Semantic Similarity for Input Topic Identification in Crawling-based Web Application Testing",
            "authors": [
                "Jun-Wei Lin",
                "Farn Wang"
            ],
            "abstract": "To automatically test web applications, crawling-based techniques are usually\nadopted to mine the behavior models, explore the state spaces or detect the\nviolated invariants of the applications. However, in existing crawlers, rules\nfor identifying the topics of input text fields, such as login ids, passwords,\nemails, dates and phone numbers, have to be manually configured. Moreover, the\nrules for one application are very often not suitable for another. In addition,\nwhen several rules conflict and match an input text field to more than one\ntopics, it can be difficult to determine which rule suggests a better match.\nThis paper presents a natural-language approach to automatically identify the\ntopics of encountered input fields during crawling by semantically comparing\ntheir similarities with the input fields in labeled corpus. In our evaluation\nwith 100 real-world forms, the proposed approach demonstrated comparable\nperformance to the rule-based one. Our experiments also show that the accuracy\nof the rule-based approach can be improved by up to 19% when integrated with\nour approach.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1608.06656v1",
            "date": "2016-08-23 21:07:50+00:00",
            "title": "Lexical Query Modeling in Session Search",
            "authors": [
                "Christophe Van Gysel",
                "Evangelos Kanoulas",
                "Maarten de Rijke"
            ],
            "abstract": "Lexical query modeling has been the leading paradigm for session search. In\nthis paper, we analyze TREC session query logs and compare the performance of\ndifferent lexical matching approaches for session search. Naive methods based\non term frequency weighing perform on par with specialized session models. In\naddition, we investigate the viability of lexical query models in the setting\nof session search. We give important insights into the potential and\nlimitations of lexical query modeling for session search and propose future\ndirections for the field of session search.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.07094v2",
            "date": "2016-08-25 11:46:06+00:00",
            "title": "A Novel Term_Class Relevance Measure for Text Categorization",
            "authors": [
                "D S Guru",
                "Mahamad Suhil"
            ],
            "abstract": "In this paper, we introduce a new measure called Term_Class relevance to\ncompute the relevancy of a term in classifying a document into a particular\nclass. The proposed measure estimates the degree of relevance of a given term,\nin placing an unlabeled document to be a member of a known class, as a product\nof Class_Term weight and Class_Term density; where the Class_Term weight is the\nratio of the number of documents of the class containing the term to the total\nnumber of documents containing the term and the Class_Term density is the\nrelative density of occurrence of the term in the class to the total occurrence\nof the term in the entire population. Unlike the other existing term weighting\nschemes such as TF-IDF and its variants, the proposed relevance measure takes\ninto account the degree of relative participation of the term across all\ndocuments of the class to the entire population. To demonstrate the\nsignificance of the proposed measure experimentation has been conducted on the\n20 Newsgroups dataset. Further, the superiority of the novel measure is brought\nout through a comparative analysis.",
            "journal ref": "Procedia Computer Science, vol.45, pp.13-22, 2015",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.07905v2",
            "date": "2016-08-29 03:42:50+00:00",
            "title": "Machine Comprehension Using Match-LSTM and Answer Pointer",
            "authors": [
                "Shuohang Wang",
                "Jing Jiang"
            ],
            "abstract": "Machine comprehension of text is an important problem in natural language\nprocessing. A recently released dataset, the Stanford Question Answering\nDataset (SQuAD), offers a large number of real questions and their answers\ncreated by humans through crowdsourcing. SQuAD provides a challenging testbed\nfor evaluating machine comprehension algorithms, partly because compared with\nprevious datasets, in SQuAD the answers do not come from a small set of\ncandidate answers and they have variable lengths. We propose an end-to-end\nneural architecture for the task. The architecture is based on match-LSTM, a\nmodel we proposed previously for textual entailment, and Pointer Net, a\nsequence-to-sequence model proposed by Vinyals et al.(2015) to constrain the\noutput tokens to be from the input sequences. We propose two ways of using\nPointer Net for our task. Our experiments show that both of our two models\nsubstantially outperform the best results obtained by Rajpurkar et al.(2016)\nusing logistic regression and manually crafted features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08339v1",
            "date": "2016-08-30 06:12:22+00:00",
            "title": "American Sign Language fingerspelling recognition from video: Methods for unrestricted recognition and signer-independence",
            "authors": [
                "Taehwan Kim"
            ],
            "abstract": "In this thesis, we study the problem of recognizing video sequences of\nfingerspelled letters in American Sign Language (ASL). Fingerspelling comprises\na significant but relatively understudied part of ASL, and recognizing it is\nchallenging for a number of reasons: It involves quick, small motions that are\noften highly coarticulated; it exhibits significant variation between signers;\nand there has been a dearth of continuous fingerspelling data collected. In\nthis work, we propose several types of recognition approaches, and explore the\nsigner variation problem. Our best-performing models are segmental\n(semi-Markov) conditional random fields using deep neural network-based\nfeatures. In the signer-dependent setting, our recognizers achieve up to about\n8% letter error rates. The signer-independent setting is much more challenging,\nbut with neural network adaptation we achieve up to 17% letter error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08515v1",
            "date": "2016-08-30 15:43:52+00:00",
            "title": "Language Detection For Short Text Messages In Social Media",
            "authors": [
                "Ivana Balazevic",
                "Mikio Braun",
                "Klaus-Robert M\u00fcller"
            ],
            "abstract": "With the constant growth of the World Wide Web and the number of documents in\ndifferent languages accordingly, the need for reliable language detection tools\nhas increased as well. Platforms such as Twitter with predominantly short texts\nare becoming important information resources, which additionally imposes the\nneed for short texts language detection algorithms. In this paper, we show how\nincorporating personalized user-specific information into the language\ndetection algorithm leads to an important improvement of detection results. To\nchoose the best algorithm for language detection for short text messages, we\ninvestigate several machine learning approaches. These approaches include the\nuse of the well-known classifiers such as SVM and logistic regression, a\ndictionary based approach, and a probabilistic model based on modified\nKneser-Ney smoothing. Furthermore, the extension of the probabilistic model to\ninclude additional user-specific information such as evidence accumulation per\nuser and user interface language is explored, with the goal of improving the\nclassification performance. The proposed approaches are evaluated on randomly\ncollected Twitter data containing Latin as well as non-Latin alphabet languages\nand the quality of the obtained results is compared, followed by the selection\nof the best performing algorithm. This algorithm is then evaluated against two\nalready existing general language detection tools: Chromium Compact Language\nDetector 2 (CLD2) and langid, where our method significantly outperforms the\nresults achieved by both of the mentioned methods. Additionally, a preview of\nbenefits and possible applications of having a reliable language detection\nalgorithm is given.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00456v2",
            "date": "2016-11-02 03:04:42+00:00",
            "title": "Measuring Asymmetric Opinions on Online Social Interrelationship with Language and Network Features",
            "authors": [
                "Bo Wang",
                "Yanshu Yu",
                "Yuan Wang"
            ],
            "abstract": "Instead of studying the properties of social relationship from an objective\nview, in this paper, we focus on individuals' subjective and asymmetric\nopinions on their interrelationships. Inspired by the theories from\nsociolinguistics, we investigate two individuals' opinions on their\ninterrelationship with their interactive language features. Eliminating the\ndifference of personal language style, we clarify that the asymmetry of\ninteractive language feature values can indicate individuals' asymmetric\nopinions on their interrelationship. We also discuss how the degree of\nopinions' asymmetry is related to the individuals' personality traits.\nFurthermore, to measure the individuals' asymmetric opinions on\ninterrelationship concretely, we develop a novel model synthetizing interactive\nlanguage and social network features. The experimental results with Enron email\ndataset provide multiple evidences of the asymmetric opinions on\ninterrelationship, and also verify the effectiveness of the proposed model in\nmeasuring the degree of opinions' asymmetry.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1611.00457v1",
            "date": "2016-11-02 03:11:10+00:00",
            "title": "Structure vs. Language: Investigating the Multi-factors of Asymmetric Opinions on Online Social Interrelationship with a Case Study",
            "authors": [
                "Bo Wang",
                "Yingjun Sun",
                "Yuan Wang"
            ],
            "abstract": "Though current researches often study the properties of online social\nrelationship from an objective view, we also need to understand individuals'\nsubjective opinions on their interrelationships in social computing studies.\nInspired by the theories from sociolinguistics, the latest work indicates that\ninteractive language can reveal individuals' asymmetric opinions on their\ninterrelationship. In this work, in order to explain the opinions' asymmetry on\ninterrelationship with more latent factors, we extend the investigation from\nsingle relationship to the structural context in online social network. We\nanalyze the correlation between interactive language features and the\nstructural context of interrelationships. The structural context of vertex,\nedges and triangles in social network are considered. With statistical analysis\non Enron email dataset, we find that individuals' opinions (measured by\ninteractive language features) on their interrelationship are related to some\nof their important structural context in social network. This result can help\nus to understand and measure the individuals' opinions on their\ninterrelationship with more intrinsic information.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1611.01547v5",
            "date": "2016-11-04 21:35:07+00:00",
            "title": "Automated Generation of Multilingual Clusters for the Evaluation of Distributed Representations",
            "authors": [
                "Philip Blair",
                "Yuval Merhav",
                "Joel Barry"
            ],
            "abstract": "We propose a language-agnostic way of automatically generating sets of\nsemantically similar clusters of entities along with sets of \"outlier\"\nelements, which may then be used to perform an intrinsic evaluation of word\nembeddings in the outlier detection task. We used our methodology to create a\ngold-standard dataset, which we call WikiSem500, and evaluated multiple\nstate-of-the-art embeddings. The results show a correlation between performance\non this dataset and performance on sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01587v5",
            "date": "2016-11-05 01:59:29+00:00",
            "title": "A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks",
            "authors": [
                "Kazuma Hashimoto",
                "Caiming Xiong",
                "Yoshimasa Tsuruoka",
                "Richard Socher"
            ],
            "abstract": "Transfer and multi-task learning have traditionally focused on either a\nsingle source-target pair or very few, similar tasks. Ideally, the linguistic\nlevels of morphology, syntax and semantics would benefit each other by being\ntrained in a single model. We introduce a joint many-task model together with a\nstrategy for successively growing its depth to solve increasingly complex\ntasks. Higher layers include shortcut connections to lower-level task\npredictions to reflect linguistic hierarchies. We use a simple regularization\nterm to allow for optimizing all model weights to improve one task's loss\nwithout exhibiting catastrophic interference of the other tasks. Our single\nend-to-end model obtains state-of-the-art or competitive results on five\ndifferent tasks from tagging, parsing, relatedness, and entailment tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01604v4",
            "date": "2016-11-05 04:53:40+00:00",
            "title": "Dynamic Coattention Networks For Question Answering",
            "authors": [
                "Caiming Xiong",
                "Victor Zhong",
                "Richard Socher"
            ],
            "abstract": "Several deep learning models have been proposed for question answering.\nHowever, due to their single-pass nature, they have no way to recover from\nlocal maxima corresponding to incorrect answers. To address this problem, we\nintroduce the Dynamic Coattention Network (DCN) for question answering. The DCN\nfirst fuses co-dependent representations of the question and the document in\norder to focus on relevant parts of both. Then a dynamic pointing decoder\niterates over potential answer spans. This iterative procedure enables the\nmodel to recover from initial local maxima corresponding to incorrect answers.\nOn the Stanford question answering dataset, a single DCN model improves the\nprevious state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains\n80.4% F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01714v1",
            "date": "2016-11-06 01:32:39+00:00",
            "title": "Beyond Fine Tuning: A Modular Approach to Learning on Small Data",
            "authors": [
                "Ark Anderson",
                "Kyle Shaffer",
                "Artem Yankov",
                "Court D. Corley",
                "Nathan O. Hodas"
            ],
            "abstract": "In this paper we present a technique to train neural network models on small\namounts of data. Current methods for training neural networks on small amounts\nof rich data typically rely on strategies such as fine-tuning a pre-trained\nneural network or the use of domain-specific hand-engineered features. Here we\ntake the approach of treating network layers, or entire networks, as modules\nand combine pre-trained modules with untrained modules, to learn the shift in\ndistributions between data sets. The central impact of using a modular approach\ncomes from adding new representations to a network, as opposed to replacing\nrepresentations via fine-tuning. Using this technique, we are able surpass\nresults using standard fine-tuning transfer learning approaches, and we are\nalso able to significantly increase performance over such approaches when using\nsmaller amounts of data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.01724v2",
            "date": "2016-11-06 03:17:42+00:00",
            "title": "Words or Characters? Fine-grained Gating for Reading Comprehension",
            "authors": [
                "Zhilin Yang",
                "Bhuwan Dhingra",
                "Ye Yuan",
                "Junjie Hu",
                "William W. Cohen",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Previous work combines word-level and character-level representations using\nconcatenation or scalar weighting, which is suboptimal for high-level tasks\nlike reading comprehension. We present a fine-grained gating mechanism to\ndynamically combine word-level and character-level representations based on\nproperties of the words. We also extend the idea of fine-grained gating to\nmodeling the interaction between questions and paragraphs for reading\ncomprehension. Experiments show that our approach can improve the performance\non reading comprehension tasks, achieving new state-of-the-art results on the\nChildren's Book Test dataset. To demonstrate the generality of our gating\nmechanism, we also show improved results on a social media tag prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01734v3",
            "date": "2016-11-06 07:26:38+00:00",
            "title": "Deep Biaffine Attention for Neural Dependency Parsing",
            "authors": [
                "Timothy Dozat",
                "Christopher D. Manning"
            ],
            "abstract": "This paper builds off recent work from Kiperwasser & Goldberg (2016) using\nneural attention in a simple graph-based dependency parser. We use a larger but\nmore thoroughly regularized parser than other recent BiLSTM-based approaches,\nwith biaffine classifiers to predict arcs and labels. Our parser gets state of\nthe art or near state of the art performance on standard treebanks for six\ndifferent languages, achieving 95.7% UAS and 94.1% LAS on the most popular\nEnglish PTB dataset. This makes it the highest-performing graph-based parser on\nthis benchmark---outperforming Kiperwasser Goldberg (2016) by 1.8% and\n2.2%---and comparable to the highest performing transition-based parser\n(Kuncoro et al., 2016), which achieves 95.8% UAS and 94.6% LAS. We also show\nwhich hyperparameter choices had a significant effect on parsing accuracy,\nallowing us to achieve large gains over other graph-based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01747v1",
            "date": "2016-11-06 09:50:24+00:00",
            "title": "A Compare-Aggregate Model for Matching Text Sequences",
            "authors": [
                "Shuohang Wang",
                "Jing Jiang"
            ],
            "abstract": "Many NLP tasks including machine comprehension, answer selection and text\nentailment require the comparison between sequences. Matching the important\nunits between sequences is a key to solve these problems. In this paper, we\npresent a general \"compare-aggregate\" framework that performs word-level\nmatching followed by aggregation using Convolutional Neural Networks. We\nparticularly focus on the different comparison functions we can use to match\ntwo vectors. We use four different datasets to evaluate the model. We find that\nsome simple comparison functions based on element-wise operations can work\nbetter than standard neural network and neural tensor network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01783v1",
            "date": "2016-11-06 14:00:14+00:00",
            "title": "Domain Adaptation For Formant Estimation Using Deep Learning",
            "authors": [
                "Yehoshua Dissen",
                "Joseph Keshet",
                "Jacob Goldberger",
                "Cynthia Clopper"
            ],
            "abstract": "In this paper we present a domain adaptation technique for formant estimation\nusing a deep network. We first train a deep learning network on a small read\nspeech dataset. We then freeze the parameters of the trained network and use\nseveral different datasets to train an adaptation layer that makes the obtained\nnetwork universal in the sense that it works well for a variety of speakers and\nspeech domains with very different characteristics. We evaluated our adapted\nnetwork on three datasets, each of which has different speaker characteristics\nand speech styles. The performance of our method compares favorably with\nalternative methods for formant estimation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01868v1",
            "date": "2016-11-07 01:08:11+00:00",
            "title": "Truth Discovery with Memory Network",
            "authors": [
                "Luyang Li",
                "Bing Qin",
                "Wenjing Ren",
                "Ting Liu"
            ],
            "abstract": "Truth discovery is to resolve conflicts and find the truth from\nmultiple-source statements. Conventional methods mostly research based on the\nmutual effect between the reliability of sources and the credibility of\nstatements, however, pay no attention to the mutual effect among the\ncredibility of statements about the same object. We propose memory network\nbased models to incorporate these two ideas to do the truth discovery. We use\nfeedforward memory network and feedback memory network to learn the\nrepresentation of the credibility of statements which are about the same\nobject. Specially, we adopt memory mechanism to learn source reliability and\nuse it through truth prediction. During learning models, we use multiple types\nof data (categorical data and continuous data) by assigning different weights\nautomatically in the loss function based on their own effect on truth discovery\nprediction. The experiment results show that the memory network based models\nmuch outperform the state-of-the-art method and other baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02695v1",
            "date": "2016-11-08 09:50:30+00:00",
            "title": "Automatic recognition of child speech for robotic applications in noisy environments",
            "authors": [
                "Samuel Fernando",
                "Roger K. Moore",
                "David Cameron",
                "Emily C. Collins",
                "Abigail Millings",
                "Amanda J. Sharkey",
                "Tony J. Prescott"
            ],
            "abstract": "Automatic speech recognition (ASR) allows a natural and intuitive interface\nfor robotic educational applications for children. However there are a number\nof challenges to overcome to allow such an interface to operate robustly in\nrealistic settings, including the intrinsic difficulties of recognising child\nspeech and high levels of background noise often present in classrooms. As part\nof the EU EASEL project we have provided several contributions to address these\nchallenges, implementing our own ASR module for use in robotics applications.\nWe used the latest deep neural network algorithms which provide a leap in\nperformance over the traditional GMM approach, and apply data augmentation\nmethods to improve robustness to noise and speaker variation. We provide a\nclose integration between the ASR module and the rest of the dialogue system,\nallowing the ASR to receive in real-time the language models relevant to the\ncurrent section of the dialogue, greatly improving the accuracy. We integrated\nour ASR module into an interactive, multimodal system using a small humanoid\nrobot to help children learn about exercise and energy. The system was\ninstalled at a public museum event as part of a research study where 320\nchildren (aged 3 to 14) interacted with the robot, with our ASR achieving 90%\naccuracy for fluent and near-fluent speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02815v1",
            "date": "2016-11-08 15:25:02+00:00",
            "title": "An Automated System for Essay Scoring of Online Exams in Arabic based on Stemming Techniques and Levenshtein Edit Operations",
            "authors": [
                "Emad Fawzi Al-Shalabi"
            ],
            "abstract": "In this article, an automated system is proposed for essay scoring in Arabic\nlanguage for online exams based on stemming techniques and Levenshtein edit\noperations. An online exam has been developed on the proposed mechanisms,\nexploiting the capabilities of light and heavy stemming. The implemented online\ngrading system has shown to be an efficient tool for automated scoring of essay\nquestions.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Volume 13,\n  Issue 5, September 2016",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.02944v1",
            "date": "2016-11-09 14:27:03+00:00",
            "title": "Increasing the throughput of machine translation systems using clouds",
            "authors": [
                "Jernej Vi\u010di\u010d",
                "Andrej Brodnik"
            ],
            "abstract": "The manuscript presents an experiment at implementation of a Machine\nTranslation system in a MapReduce model. The empirical evaluation was done\nusing fully implemented translation systems embedded into the MapReduce\nprogramming model. Two machine translation paradigms were studied: shallow\ntransfer Rule Based Machine Translation and Statistical Machine Translation.\n  The results show that the MapReduce model can be successfully used to\nincrease the throughput of a machine translation system. Furthermore this\nmethod enhances the throughput of a machine translation system without\ndecreasing the quality of the translation output.\n  Thus, the present manuscript also represents a contribution to the seminal\nwork in natural language processing, specifically Machine Translation. It first\npoints toward the importance of the definition of the metric of throughput of\ntranslation system and, second, the applicability of the machine translation\ntask to the MapReduce paradigm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03305v1",
            "date": "2016-11-08 14:28:40+00:00",
            "title": "Getting Started with Neural Models for Semantic Matching in Web Search",
            "authors": [
                "Kezban Dilek Onal",
                "Ismail Sengor Altingovde",
                "Pinar Karagoz",
                "Maarten de Rijke"
            ],
            "abstract": "The vocabulary mismatch problem is a long-standing problem in information\nretrieval. Semantic matching holds the promise of solving the problem. Recent\nadvances in language technology have given rise to unsupervised neural models\nfor learning representations of words as well as bigger textual units. Such\nrepresentations enable powerful semantic matching methods. This survey is meant\nas an introduction to the use of neural models for semantic matching. To remain\nfocused we limit ourselves to web search. We detail the required background and\nterminology, a taxonomy grouping the rapidly growing body of work in the area,\nand then survey work on neural models for semantic matching in the context of\nthree tasks: query suggestion, ad retrieval, and document retrieval. We include\na section on resources and best practices that we believe will help readers who\nare new to the area. We conclude with an assessment of the state-of-the-art and\nsuggestions for future work.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.03533v1",
            "date": "2016-11-10 22:11:16+00:00",
            "title": "Landmark-based consonant voicing detection on multilingual corpora",
            "authors": [
                "Xiang Kong",
                "Xuesong Yang",
                "Mark Hasegawa-Johnson",
                "Jeung-Yoon Choi",
                "Stefanie Shattuck-Hufnagel"
            ],
            "abstract": "This paper tests the hypothesis that distinctive feature classifiers anchored\nat phonetic landmarks can be transferred cross-lingually without loss of\naccuracy. Three consonant voicing classifiers were developed: (1) manually\nselected acoustic features anchored at a phonetic landmark, (2) MFCCs (either\naveraged across the segment or anchored at the landmark), and(3) acoustic\nfeatures computed using a convolutional neural network (CNN). All detectors are\ntrained on English data (TIMIT),and tested on English, Turkish, and Spanish\n(performance measured using F1 and accuracy). Experiments demonstrate that\nmanual features outperform all MFCC classifiers, while CNNfeatures outperform\nboth. MFCC-based classifiers suffer an F1reduction of 16% absolute when\ngeneralized from English to other languages. Manual features suffer only a 5%\nF1 reduction,and CNN features actually perform better in Turkish and Span-ish\nthan in the training language, demonstrating that features capable of\nrepresenting long-term spectral dynamics (CNN and landmark-based features) are\nable to generalize cross-lingually with little or no loss of accuracy",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03596v1",
            "date": "2016-11-11 06:36:53+00:00",
            "title": "Generalized Entropies and the Similarity of Texts",
            "authors": [
                "Eduardo G. Altmann",
                "Laercio Dias",
                "Martin Gerlach"
            ],
            "abstract": "We show how generalized Gibbs-Shannon entropies can provide new insights on\nthe statistical properties of texts. The universal distribution of word\nfrequencies (Zipf's law) implies that the generalized entropies, computed at\nthe word level, are dominated by words in a specific range of frequencies. Here\nwe show that this is the case not only for the generalized entropies but also\nfor the generalized (Jensen-Shannon) divergences, used to compute the\nsimilarity between different texts. This finding allows us to identify the\ncontribution of specific words (and word frequencies) for the different\ngeneralized entropies and also to estimate the size of the databases needed to\nobtain a reliable estimation of the divergences. We test our results in large\ndatabases of books (from the Google n-gram database) and scientific papers\n(indexed by Web of Science).",
            "journal ref": "J. Stat. Mech. 014002 (2017)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1611.03954v3",
            "date": "2016-11-12 04:28:04+00:00",
            "title": "Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment",
            "authors": [
                "Muhao Chen",
                "Yingtao Tian",
                "Mohan Yang",
                "Carlo Zaniolo"
            ],
            "abstract": "Many recent works have demonstrated the benefits of knowledge graph\nembeddings in completing monolingual knowledge graphs. Inasmuch as related\nknowledge bases are built in several different languages, achieving\ncross-lingual knowledge alignment will help people in constructing a coherent\nknowledge base, and assist machines in dealing with different expressions of\nentity relationships across diverse human languages. Unfortunately, achieving\nthis highly desirable crosslingual alignment by human labor is very costly and\nerrorprone. Thus, we propose MTransE, a translation-based model for\nmultilingual knowledge graph embeddings, to provide a simple and automated\nsolution. By encoding entities and relations of each language in a separated\nembedding space, MTransE provides transitions for each embedding vector to its\ncross-lingual counterparts in other spaces, while preserving the\nfunctionalities of monolingual embeddings. We deploy three different techniques\nto represent cross-lingual transitions, namely axis calibration, translation\nvectors, and linear transformations, and derive five variants for MTransE using\ndifferent loss functions. Our models can be trained on partially aligned\ngraphs, where just a small portion of triples are aligned with their\ncross-lingual counterparts. The experiments on cross-lingual entity matching\nand triple-wise alignment verification show promising results, with some\nvariants consistently outperforming others on different tasks. We also explore\nhow MTransE preserves the key properties of its monolingual counterpart TransE.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.04558v2",
            "date": "2016-11-14 20:24:39+00:00",
            "title": "Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation",
            "authors": [
                "Melvin Johnson",
                "Mike Schuster",
                "Quoc V. Le",
                "Maxim Krikun",
                "Yonghui Wu",
                "Zhifeng Chen",
                "Nikhil Thorat",
                "Fernanda Vi\u00e9gas",
                "Martin Wattenberg",
                "Greg Corrado",
                "Macduff Hughes",
                "Jeffrey Dean"
            ],
            "abstract": "We propose a simple solution to use a single Neural Machine Translation (NMT)\nmodel to translate between multiple languages. Our solution requires no change\nin the model architecture from our base system but instead introduces an\nartificial token at the beginning of the input sentence to specify the required\ntarget language. The rest of the model, which includes encoder, decoder and\nattention, remains unchanged and is shared across all languages. Using a shared\nwordpiece vocabulary, our approach enables Multilingual NMT using a single\nmodel without any increase in parameters, which is significantly simpler than\nprevious proposals for Multilingual NMT. Our method often improves the\ntranslation quality of all involved language pairs, even while keeping the\ntotal number of model parameters constant. On the WMT'14 benchmarks, a single\nmultilingual model achieves comparable performance for\nEnglish$\\rightarrow$French and surpasses state-of-the-art results for\nEnglish$\\rightarrow$German. Similarly, a single multilingual model surpasses\nstate-of-the-art results for French$\\rightarrow$English and\nGerman$\\rightarrow$English on WMT'14 and WMT'15 benchmarks respectively. On\nproduction corpora, multilingual models of up to twelve language pairs allow\nfor better translation of many individual pairs. In addition to improving the\ntranslation quality of language pairs that the model was trained with, our\nmodels can also learn to perform implicit bridging between language pairs never\nseen explicitly during training, showing that transfer learning and zero-shot\ntranslation is possible for neural translation. Finally, we show analyses that\nhints at a universal interlingua representation in our models and show some\ninteresting examples when mixing languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04841v2",
            "date": "2016-11-14 10:53:40+00:00",
            "title": "Quantitative Entropy Study of Language Complexity",
            "authors": [
                "R. R. Xie",
                "W. B. Deng",
                "D. J. Wang",
                "L. P. Csernai"
            ],
            "abstract": "We study the entropy of Chinese and English texts, based on characters in\ncase of Chinese texts and based on words for both languages. Significant\ndifferences are found between the languages and between different personal\nstyles of debating partners. The entropy analysis points in the direction of\nlower entropy, that is of higher complexity. Such a text analysis would be\napplied for individuals of different styles, a single individual at different\nage, as well as different groups of the population.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04842v1",
            "date": "2016-11-15 14:12:41+00:00",
            "title": "The Role of Word Length in Semantic Topology",
            "authors": [
                "Francesco Fumarola"
            ],
            "abstract": "A topological argument is presented concering the structure of semantic\nspace, based on the negative correlation between polysemy and word length. The\nresulting graph structure is applied to the modeling of free-recall\nexperiments, resulting in predictions on the comparative values of recall\nprobabilities. Associative recall is found to favor longer words whereas\nsequential recall is found to favor shorter words. Data from the PEERS\nexperiments of Lohnas et al. (2015) and Healey and Kahana (2016) confirm both\npredictons, with correlation coefficients $r_{seq}= -0.17$ and $r_{ass}=\n+0.17$. The argument is then applied to predicting global properties of list\nrecall, which leads to a novel explanation for the word-length effect based on\nthe optimization of retrieval strategies.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1611.04887v1",
            "date": "2016-11-15 15:34:47+00:00",
            "title": "Interpreting the Syntactic and Social Elements of the Tweet Representations via Elementary Property Prediction Tasks",
            "authors": [
                "J Ganesh",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Research in social media analysis is experiencing a recent surge with a large\nnumber of works applying representation learning models to solve high-level\nsyntactico-semantic tasks such as sentiment analysis, semantic textual\nsimilarity computation, hashtag prediction and so on. Although the performance\nof the representation learning models are better than the traditional baselines\nfor the tasks, little is known about the core properties of a tweet encoded\nwithin the representations. Understanding these core properties would empower\nus in making generalizable conclusions about the quality of representations.\nOur work presented here constitutes the first step in opening the black-box of\nvector embedding for social media posts, with emphasis on tweets in particular.\n  In order to understand the core properties encoded in a tweet representation,\nwe evaluate the representations to estimate the extent to which it can model\neach of those properties such as tweet length, presence of words, hashtags,\nmentions, capitalization, and so on. This is done with the help of multiple\nclassifiers which take the representation as input. Essentially, each\nclassifier evaluates one of the syntactic or social properties which are\narguably salient for a tweet. This is also the first holistic study on\nextensively analysing the ability to encode these properties for a wide variety\nof tweet representation models including the traditional unsupervised methods\n(BOW, LDA), unsupervised representation learning methods (Siamese CBOW,\nTweet2Vec) as well as supervised methods (CNN, BLSTM).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05104v2",
            "date": "2016-11-16 00:53:01+00:00",
            "title": "A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs",
            "authors": [
                "Shayne Longpre",
                "Sabeek Pradhan",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "LSTMs have become a basic building block for many deep NLP models. In recent\nyears, many improvements and variations have been proposed for deep sequence\nmodels in general, and LSTMs in particular. We propose and analyze a series of\naugmentations and modifications to LSTM networks resulting in improved\nperformance for text classification datasets. We observe compounding\nimprovements on traditional LSTMs using Monte Carlo test-time model averaging,\naverage pooling, and residual connections, along with four other suggested\nmodifications. Our analysis provides a simple, reliable, and high quality\nbaseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05118v2",
            "date": "2016-11-16 02:16:09+00:00",
            "title": "The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives",
            "authors": [
                "Mohit Iyyer",
                "Varun Manjunatha",
                "Anupam Guha",
                "Yogarshi Vyas",
                "Jordan Boyd-Graber",
                "Hal Daum\u00e9 III",
                "Larry Davis"
            ],
            "abstract": "Visual narrative is often a combination of explicit information and judicious\nomissions, relying on the viewer to supply missing details. In comics, most\nmovements in time and space are hidden in the \"gutters\" between panels. To\nfollow the story, readers logically connect panels together by inferring unseen\nactions through a process called \"closure\". While computers can now describe\nwhat is explicitly depicted in natural images, in this paper we examine whether\nthey can understand the closure-driven narratives conveyed by stylized artwork\nand dialogue in comic book panels. We construct a dataset, COMICS, that\nconsists of over 1.2 million panels (120 GB) paired with automatic textbox\ntranscriptions. An in-depth analysis of COMICS demonstrates that neither text\nnor image alone can tell a comic book story, so a computer must understand both\nmodalities to keep up with the plot. We introduce three cloze-style tasks that\nask models to predict narrative and character-centric aspects of a panel given\nn preceding panels as context. Various deep neural architectures underperform\nhuman baselines on these tasks, suggesting that COMICS contains fundamental\nchallenges for both vision and language.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.06478v1",
            "date": "2016-11-20 06:50:16+00:00",
            "title": "Visualizing Linguistic Shift",
            "authors": [
                "Salman Mahmood",
                "Rami Al-Rfou",
                "Klaus Mueller"
            ],
            "abstract": "Neural network based models are a very powerful tool for creating word\nembeddings, the objective of these models is to group similar words together.\nThese embeddings have been used as features to improve results in various\napplications such as document classification, named entity recognition, etc.\nNeural language models are able to learn word representations which have been\nused to capture semantic shifts across time and geography. The objective of\nthis paper is to first identify and then visualize how words change meaning in\ndifferent text corpus. We will train a neural language model on texts from a\ndiverse set of disciplines philosophy, religion, fiction etc. Each text will\nalter the embeddings of the words to represent the meaning of the word inside\nthat text. We will present a computational technique to detect words that\nexhibit significant linguistic shift in meaning and usage. We then use enhanced\nscatterplots and storyline visualization to visualize the linguistic shift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06492v2",
            "date": "2016-11-20 10:07:54+00:00",
            "title": "Recurrent Memory Addressing for describing videos",
            "authors": [
                "Arnav Kumar Jain",
                "Abhinav Agarwalla",
                "Kumar Krishna Agrawal",
                "Pabitra Mitra"
            ],
            "abstract": "In this paper, we introduce Key-Value Memory Networks to a multimodal setting\nand a novel key-addressing mechanism to deal with sequence-to-sequence models.\nThe proposed model naturally decomposes the problem of video captioning into\nvision and language segments, dealing with them as key-value pairs. More\nspecifically, we learn a semantic embedding (v) corresponding to each frame (k)\nin the video, thereby creating (k, v) memory slots. We propose to find the next\nstep attention weights conditioned on the previous attention distributions for\nthe key-value memory slots in the memory addressing schema. Exploiting this\nflexibility of the framework, we additionally capture spatial dependencies\nwhile mapping from the visual to semantic embedding. Experiments done on the\nYoutube2Text dataset demonstrate usefulness of recurrent key-addressing, while\nachieving competitive scores on BLEU@4, METEOR metrics against state-of-the-art\nmodels.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.06607v2",
            "date": "2016-11-20 23:10:51+00:00",
            "title": "A Hierarchical Approach for Generating Descriptive Image Paragraphs",
            "authors": [
                "Jonathan Krause",
                "Justin Johnson",
                "Ranjay Krishna",
                "Li Fei-Fei"
            ],
            "abstract": "Recent progress on image captioning has made it possible to generate novel\nsentences describing images in natural language, but compressing an image into\na single sentence can describe visual content in only coarse detail. While one\nnew captioning approach, dense captioning, can potentially describe images in\nfiner levels of detail by captioning many regions within an image, it in turn\nis unable to produce a coherent story for an image. In this paper we overcome\nthese limitations by generating entire paragraphs for describing images, which\ncan tell detailed, unified stories. We develop a model that decomposes both\nimages and paragraphs into their constituent parts, detecting semantic regions\nin images and using a hierarchical recurrent neural network to reason about\nlanguage. Linguistic analysis confirms the complexity of the paragraph\ngeneration task, and thorough experiments on a new dataset of image and\nparagraph pairs demonstrate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.06671v1",
            "date": "2016-11-21 07:32:56+00:00",
            "title": "Ontology Driven Disease Incidence Detection on Twitter",
            "authors": [
                "Mark Abraham Magumba",
                "Peter Nabende"
            ],
            "abstract": "In this work we address the issue of generic automated disease incidence\nmonitoring on twitter. We employ an ontology of disease related concepts and\nuse it to obtain a conceptual representation of tweets. Unlike previous key\nword based systems and topic modeling approaches, our ontological approach\nallows us to apply more stringent criteria for determining which messages are\nrelevant such as spatial and temporal characteristics whilst giving a stronger\nguarantee that the resulting models will perform well on new data that may be\nlexically divergent. We achieve this by training learners on concepts rather\nthan individual words. For training we use a dataset containing mentions of\ninfluenza and Listeria and use the learned models to classify datasets\ncontaining mentions of an arbitrary selection of other diseases. We show that\nour ontological approach achieves good performance on this task using a variety\nof Natural Language Processing Techniques. We also show that word vectors can\nbe learned directly from our concepts to achieve even better results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06997v1",
            "date": "2016-11-21 20:25:19+00:00",
            "title": "Coherent Dialogue with Attention-based Language Models",
            "authors": [
                "Hongyuan Mei",
                "Mohit Bansal",
                "Matthew R. Walter"
            ],
            "abstract": "We model coherent conversation continuation via RNN-based dialogue models\nequipped with a dynamic attention mechanism. Our attention-RNN language model\ndynamically increases the scope of attention on the history as the conversation\ncontinues, as opposed to standard attention (or alignment) models with a fixed\ninput scope in a sequence-to-sequence model. This allows each generated word to\nbe associated with the most relevant words in its corresponding conversation\nhistory. We evaluate the model on two popular dialogue datasets, the\nopen-domain MovieTriples dataset and the closed-domain Ubuntu Troubleshoot\ndataset, and achieve significant improvements over the state-of-the-art and\nbaselines on several metrics, including complementary diversity-based metrics,\nhuman evaluation, and qualitative visualizations. We also show that a vanilla\nRNN with dynamic attention outperforms more complex memory models (e.g., LSTM\nand GRU) by allowing for flexible, long-distance memory. We promote further\ncoherence via topic modeling-based reranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07174v2",
            "date": "2016-11-22 07:36:21+00:00",
            "title": "Deep Recurrent Convolutional Neural Network: Improving Performance For Speech Recognition",
            "authors": [
                "Zewang Zhang",
                "Zheng Sun",
                "Jiaqi Liu",
                "Jingwen Chen",
                "Zhao Huo",
                "Xiao Zhang"
            ],
            "abstract": "A deep learning approach has been widely applied in sequence modeling\nproblems. In terms of automatic speech recognition (ASR), its performance has\nsignificantly been improved by increasing large speech corpus and deeper neural\nnetwork. Especially, recurrent neural network and deep convolutional neural\nnetwork have been applied in ASR successfully. Given the arising problem of\ntraining speed, we build a novel deep recurrent convolutional network for\nacoustic modeling and then apply deep residual learning to it. Our experiments\nshow that it has not only faster convergence speed but better recognition\naccuracy over traditional deep convolutional recurrent network. In the\nexperiments, we compare the convergence speed of our novel deep recurrent\nconvolutional networks and traditional deep convolutional recurrent networks.\nWith faster convergence speed, our novel deep recurrent convolutional networks\ncan reach the comparable performance. We further show that applying deep\nresidual learning can boost the convergence speed of our novel deep recurret\nconvolutional networks. Finally, we evaluate all our experimental networks by\nphoneme error rate (PER) with our proposed bidirectional statistical n-gram\nlanguage model. Our evaluation results show that our newly proposed deep\nrecurrent convolutional network applied with deep residual learning can reach\nthe best PER of 17.33\\% with the fastest convergence speed on TIMIT database.\nThe outstanding performance of our novel deep recurrent convolutional neural\nnetwork with deep residual learning indicates that it can be potentially\nadopted in other sequential problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07837v3",
            "date": "2016-11-23 15:21:48+00:00",
            "title": "Adaptive Feature Abstraction for Translating Video to Text",
            "authors": [
                "Yunchen Pu",
                "Martin Renqiang Min",
                "Zhe Gan",
                "Lawrence Carin"
            ],
            "abstract": "Previous models for video captioning often use the output from a specific\nlayer of a Convolutional Neural Network (CNN) as video features. However, the\nvariable context-dependent semantics in the video may make it more appropriate\nto adaptively select features from the multiple CNN layers. We propose a new\napproach for generating adaptive spatiotemporal representations of videos for\nthe captioning task. A novel attention mechanism is developed, that adaptively\nand sequentially focuses on different layers of CNN features (levels of feature\n\"abstraction\"), as well as local spatiotemporal regions of the feature maps at\neach layer. The proposed approach is evaluated on three benchmark datasets:\nYouTube2Text, M-VAD and MSR-VTT. Along with visualizing the results and how the\nmodel works, these experiments quantitatively demonstrate the effectiveness of\nthe proposed adaptive spatiotemporal feature abstraction for translating videos\nto sentences with rich semantics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.07897v2",
            "date": "2016-11-23 17:32:23+00:00",
            "title": "Learning Generic Sentence Representations Using Convolutional Neural Networks",
            "authors": [
                "Zhe Gan",
                "Yunchen Pu",
                "Ricardo Henao",
                "Chunyuan Li",
                "Xiaodong He",
                "Lawrence Carin"
            ],
            "abstract": "We propose a new encoder-decoder approach to learn distributed sentence\nrepresentations that are applicable to multiple purposes. The model is learned\nby using a convolutional neural network as an encoder to map an input sentence\ninto a continuous vector, and using a long short-term memory recurrent neural\nnetwork as a decoder. Several tasks are considered, including sentence\nreconstruction and future sentence prediction. Further, a hierarchical\nencoder-decoder model is proposed to encode a sentence to predict multiple\nfuture sentences. By training our models on a large collection of novels, we\nobtain a highly generic convolutional sentence encoder that performs well in\npractice. Experimental results on several benchmark datasets, and across a\nbroad range of applications, demonstrate the superiority of the proposed model\nover competing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08034v2",
            "date": "2016-11-23 23:40:50+00:00",
            "title": "Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling",
            "authors": [
                "Zhe Gan",
                "Chunyuan Li",
                "Changyou Chen",
                "Yunchen Pu",
                "Qinliang Su",
                "Lawrence Carin"
            ],
            "abstract": "Recurrent neural networks (RNNs) have shown promising performance for\nlanguage modeling. However, traditional training of RNNs using back-propagation\nthrough time often suffers from overfitting. One reason for this is that\nstochastic optimization (used for large training sets) does not provide good\nestimates of model uncertainty. This paper leverages recent advances in\nstochastic gradient Markov Chain Monte Carlo (also appropriate for large\ntraining sets) to learn weight uncertainty in RNNs. It yields a principled\nBayesian learning algorithm, adding gradient noise during training (enhancing\nexploration of the model-parameter space) and model averaging when testing.\nExtensive experiments on various RNN models and across a broad range of\napplications demonstrate the superiority of the proposed approach over\nstochastic optimization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08096v1",
            "date": "2016-11-24 08:43:03+00:00",
            "title": "User Personalized Satisfaction Prediction via Multiple Instance Deep Learning",
            "authors": [
                "Zheqian Chen",
                "Ben Gao",
                "Huimin Zhang",
                "Zhou Zhao",
                "Deng Cai"
            ],
            "abstract": "Community based question answering services have arisen as a popular\nknowledge sharing pattern for netizens. With abundant interactions among users,\nindividuals are capable of obtaining satisfactory information. However, it is\nnot effective for users to attain answers within minutes. Users have to check\nthe progress over time until the satisfying answers submitted. We address this\nproblem as a user personalized satisfaction prediction task. Existing methods\nusually exploit manual feature selection. It is not desirable as it requires\ncareful design and is labor intensive. In this paper, we settle this issue by\ndeveloping a new multiple instance deep learning framework. Specifically, in\nour settings, each question follows a weakly supervised learning multiple\ninstance learning assumption, where its obtained answers can be regarded as\ninstance sets and we define the question resolved with at least one\nsatisfactory answer. We thus design an efficient framework exploiting multiple\ninstance learning property with deep learning to model the question answer\npairs. Extensive experiments on large scale datasets from Stack Exchange\ndemonstrate the feasibility of our proposed framework in predicting askers\npersonalized satisfaction. Our framework can be extended to numerous\napplications such as UI satisfaction Prediction, multi armed bandit problem,\nexpert finding and so on.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.08135v1",
            "date": "2016-11-24 11:01:32+00:00",
            "title": "Question Retrieval for Community-based Question Answering via Heterogeneous Network Integration Learning",
            "authors": [
                "Zheqian Chen",
                "Chi Zhang",
                "Zhou Zhao",
                "Deng Cai"
            ],
            "abstract": "Community based question answering platforms have attracted substantial users\nto share knowledge and learn from each other. As the rapid enlargement of CQA\nplatforms, quantities of overlapped questions emerge, which makes users\nconfounded to select a proper reference. It is urgent for us to take effective\nautomated algorithms to reuse historical questions with corresponding answers.\nIn this paper we focus on the problem with question retrieval, which aims to\nmatch historical questions that are relevant or semantically equivalent to\nresolve one s query directly. The challenges in this task are the lexical gaps\nbetween questions for the word ambiguity and word mismatch problem.\nFurthermore, limited words in queried sentences cause sparsity of word\nfeatures. To alleviate these challenges, we propose a novel framework named\nHNIL which encodes not only the question contents but also the askers social\ninteractions to enhance the question embedding performance. More specifically,\nwe apply random walk based learning method with recurrent neural network to\nmatch the similarities between askers question and historical questions\nproposed by other users. Extensive experiments on a large scale dataset from a\nreal world CQA site show that employing the heterogeneous social network\ninformation outperforms the other state of the art solutions in this task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.08807v2",
            "date": "2016-11-27 08:32:19+00:00",
            "title": "The polysemy of the words that children learn over time",
            "authors": [
                "Bernardino Casas",
                "Neus Catal\u00e0",
                "Ramon Ferrer-i-Cancho",
                "Antoni Hern\u00e1ndez-Fern\u00e1ndez",
                "Jaume Baixeries"
            ],
            "abstract": "Here we study polysemy as a potential learning bias in vocabulary learning in\nchildren. Words of low polysemy could be preferred as they reduce the\ndisambiguation effort for the listener. However, such preference could be a\nside-effect of another bias: the preference of children for nouns in\ncombination with the lower polysemy of nouns with respect to other\npart-of-speech categories. Our results show that mean polysemy in children\nincreases over time in two phases, i.e. a fast growth till the 31st month\nfollowed by a slower tendency towards adult speech. In contrast, this evolution\nis not found in adults interacting with children. This suggests that children\nhave a preference for non-polysemous words in their early stages of vocabulary\nacquisition. Interestingly, the evolutionary pattern described above weakens\nwhen controlling for syntactic category (noun, verb, adjective or adverb) but\nit does not disappear completely, suggesting that it could result from\nacombination of a standalone bias for low polysemy and a preference for nouns.",
            "journal ref": "Interaction Studies 19 (3), 389-426 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.08928v2",
            "date": "2016-11-27 22:42:13+00:00",
            "title": "A theory of interpretive clustering in free recall",
            "authors": [
                "Francesco Fumarola"
            ],
            "abstract": "A stochastic model of short-term verbal memory is proposed, in which the\npsychological state of the subject is encoded as the instantaneous position of\na particle diffusing over a semantic graph with a probabilistic structure. The\nmodel is particularly suitable for studying the dependence of free-recall\nobservables on semantic properties of the words to be recalled. Besides\npredicting some well-known experimental features (contiguity effect, forward\nasymmetry, word-length effect), a novel prediction is obtained on the\nrelationship between the contiguity effect and the syllabic length of words;\nshorter words, by way of their wider semantic range, are predicted to be\ncharacterized by stronger forward contiguity. A fresh analysis of archival data\nallows to confirm this prediction.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1611.09122v1",
            "date": "2016-11-18 03:48:04+00:00",
            "title": "Statistical Properties of European Languages and Voynich Manuscript Analysis",
            "authors": [
                "Andronik Arutyunov",
                "Leonid Borisov",
                "Sergey Fedorov",
                "Anastasiya Ivchenko",
                "Elizabeth Kirina-Lilinskaya",
                "Yurii Orlov",
                "Konstantin Osminin",
                "Sergey Shilin",
                "Dmitriy Zeniuk"
            ],
            "abstract": "The statistical properties of letters frequencies in European literature\ntexts are investigated. The determination of logarithmic dependence of letters\nsequence for one-language and two-language texts are examined. The pare of\nlanguages is suggested for Voynich Manuscript. The internal structure of\nManuscript is considered. The spectral portraits of two-letters distribution\nare constructed.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1611.09235v1",
            "date": "2016-11-28 16:49:37+00:00",
            "title": "Joint Copying and Restricted Generation for Paraphrase",
            "authors": [
                "Ziqiang Cao",
                "Chuwei Luo",
                "Wenjie Li",
                "Sujian Li"
            ],
            "abstract": "Many natural language generation tasks, such as abstractive summarization and\ntext simplification, are paraphrase-orientated. In these tasks, copying and\nrewriting are two main writing modes. Most previous sequence-to-sequence\n(Seq2Seq) models use a single decoder and neglect this fact. In this paper, we\ndevelop a novel Seq2Seq model to fuse a copying decoder and a restricted\ngenerative decoder. The copying decoder finds the position to be copied based\non a typical attention model. The generative decoder produces words limited in\nthe source-specific vocabulary. To combine the two decoders and determine the\nfinal output, we develop a predictor to predict the mode of copying or\nrewriting. This predictor can be guided by the actual writing mode in the\ntraining data. We conduct extensive experiments on two different paraphrase\ndatasets. The result shows that our model outperforms the state-of-the-art\napproaches in terms of both informativeness and language quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09238v1",
            "date": "2016-11-28 16:53:06+00:00",
            "title": "Improving Multi-Document Summarization via Text Classification",
            "authors": [
                "Ziqiang Cao",
                "Wenjie Li",
                "Sujian Li",
                "Furu Wei"
            ],
            "abstract": "Developed so far, multi-document summarization has reached its bottleneck due\nto the lack of sufficient training data and diverse categories of documents.\nText classification just makes up for these deficiencies. In this paper, we\npropose a novel summarization system called TCSum, which leverages plentiful\ntext classification data to improve the performance of multi-document\nsummarization. TCSum projects documents onto distributed representations which\nact as a bridge between text classification and summarization. It also utilizes\nthe classification results to produce summaries of different styles. Extensive\nexperiments on DUC generic multi-document summarization datasets show that,\nTCSum can achieve the state-of-the-art performance without using any\nhand-crafted features and has the capability to catch the variations of summary\nstyles with respect to different text categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09268v3",
            "date": "2016-11-28 18:14:11+00:00",
            "title": "MS MARCO: A Human Generated MAchine Reading COmprehension Dataset",
            "authors": [
                "Payal Bajaj",
                "Daniel Campos",
                "Nick Craswell",
                "Li Deng",
                "Jianfeng Gao",
                "Xiaodong Liu",
                "Rangan Majumder",
                "Andrew McNamara",
                "Bhaskar Mitra",
                "Tri Nguyen",
                "Mir Rosenberg",
                "Xia Song",
                "Alina Stoica",
                "Saurabh Tiwary",
                "Tong Wang"
            ],
            "abstract": "We introduce a large scale MAchine Reading COmprehension dataset, which we\nname MS MARCO. The dataset comprises of 1,010,916 anonymized\nquestions---sampled from Bing's search query logs---each with a human generated\nanswer and 182,669 completely human rewritten generated answers. In addition,\nthe dataset contains 8,841,823 passages---extracted from 3,563,535 web\ndocuments retrieved by Bing---that provide the information necessary for\ncurating the natural language answers. A question in the MS MARCO dataset may\nhave multiple answers or no answers at all. Using this dataset, we propose\nthree different tasks with varying levels of difficulty: (i) predict if a\nquestion is answerable given a set of context passages, and extract and\nsynthesize the answer as a human would (ii) generate a well-formed answer (if\npossible) based on the context passages that can be understood with the\nquestion and passage context, and finally (iii) rank a set of retrieved\npassages given a question. The size of the dataset and the fact that the\nquestions are derived from real user search queries distinguishes MS MARCO from\nother well-known publicly available datasets for machine reading comprehension\nand question-answering. We believe that the scale and the real-world nature of\nthis dataset makes it attractive for benchmarking machine reading comprehension\nand question-answering models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09441v1",
            "date": "2016-11-29 00:22:13+00:00",
            "title": "Sentiment Analysis for Twitter : Going Beyond Tweet Text",
            "authors": [
                "Lahari Poddar",
                "Kishaloy Halder",
                "Xianyan Jia"
            ],
            "abstract": "Analysing sentiment of tweets is important as it helps to determine the\nusers' opinion. Knowing people's opinion is crucial for several purposes\nstarting from gathering knowledge about customer base, e-governance,\ncampaigning and many more. In this report, we aim to develop a system to detect\nthe sentiment from tweets. We employ several linguistic features along with\nsome other external sources of information to detect the sentiment of a tweet.\nWe show that augmenting the 140 character-long tweet with information harvested\nfrom external urls shared in the tweet as well as Social Media features\nenhances the sentiment prediction accuracy significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09534v1",
            "date": "2016-11-29 09:05:11+00:00",
            "title": "Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce",
            "authors": [
                "Tom Zahavy",
                "Alessandro Magnani",
                "Abhinandan Krishnan",
                "Shie Mannor"
            ],
            "abstract": "Classifying products into categories precisely and efficiently is a major\nchallenge in modern e-commerce. The high traffic of new products uploaded daily\nand the dynamic nature of the categories raise the need for machine learning\nmodels that can reduce the cost and time of human editors. In this paper, we\npropose a decision level fusion approach for multi-modal product classification\nusing text and image inputs. We train input specific state-of-the-art deep\nneural networks for each input source, show the potential of forging them\ntogether into a multi-modal architecture and train a novel policy network that\nlearns to choose between them. Finally, we demonstrate that our multi-modal\nnetwork improves the top-1 accuracy % over both networks on a real-world\nlarge-scale product classification dataset that we collected fromWalmart.com.\nWhile we focus on image-text fusion that characterizes e-commerce domains, our\nalgorithms can be easily applied to other modalities such as audio, video,\nphysical sensors, etc.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.09703v1",
            "date": "2016-11-29 16:20:24+00:00",
            "title": "Semantic Parsing of Mathematics by Context-based Learning from Aligned Corpora and Theorem Proving",
            "authors": [
                "Cezary Kaliszyk",
                "Josef Urban",
                "Ji\u0159\u00ed Vysko\u010dil"
            ],
            "abstract": "We study methods for automated parsing of informal mathematical expressions\ninto formal ones, a main prerequisite for deep computer understanding of\ninformal mathematical texts. We propose a context-based parsing approach that\ncombines efficient statistical learning of deep parse trees with their semantic\npruning by type checking and large-theory automated theorem proving. We show\nthat the methods very significantly improve on previous results in parsing\ntheorems from the Flyspeck corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09823v3",
            "date": "2016-11-29 20:16:44+00:00",
            "title": "Dialogue Learning With Human-In-The-Loop",
            "authors": [
                "Jiwei Li",
                "Alexander H. Miller",
                "Sumit Chopra",
                "Marc'Aurelio Ranzato",
                "Jason Weston"
            ],
            "abstract": "An important aspect of developing conversational agents is to give a bot the\nability to improve through communicating with humans and to learn from the\nmistakes that it makes. Most research has focused on learning from fixed\ntraining sets of labeled data rather than interacting with a dialogue partner\nin an online fashion. In this paper we explore this direction in a\nreinforcement learning setting where the bot improves its question-answering\nability from feedback a teacher gives following its generated responses. We\nbuild a simulator that tests various aspects of such learning in a synthetic\nenvironment, and introduce models that work in this regime. Finally, real\nexperiments with Mechanical Turk validate the approach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.09830v3",
            "date": "2016-11-29 20:38:07+00:00",
            "title": "NewsQA: A Machine Comprehension Dataset",
            "authors": [
                "Adam Trischler",
                "Tong Wang",
                "Xingdi Yuan",
                "Justin Harris",
                "Alessandro Sordoni",
                "Philip Bachman",
                "Kaheer Suleman"
            ],
            "abstract": "We present NewsQA, a challenging machine comprehension dataset of over\n100,000 human-generated question-answer pairs. Crowdworkers supply questions\nand answers based on a set of over 10,000 news articles from CNN, with answers\nconsisting of spans of text from the corresponding articles. We collect this\ndataset through a four-stage process designed to solicit exploratory questions\nthat require reasoning. A thorough analysis confirms that NewsQA demands\nabilities beyond simple word matching and recognizing textual entailment. We\nmeasure human performance on the dataset and compare it to several strong\nneural models. The performance gap between humans and machines (0.198 in F1)\nindicates that significant progress can be made on NewsQA through future\nresearch. The dataset is freely available at\nhttps://datasets.maluuba.com/NewsQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00186v1",
            "date": "2016-07-01 10:01:11+00:00",
            "title": "Throwing fuel on the embers: Probability or Dichotomy, Cognitive or Linguistic?",
            "authors": [
                "David M. W. Powers"
            ],
            "abstract": "Prof. Robert Berwick's abstract for his forthcoming invited talk at the\nACL2016 workshop on Cognitive Aspects of Computational Language Learning\nrevives an ancient debate. Entitled \"Why take a chance?\", Berwick seems to\nrefer implicitly to Chomsky's critique of the statistical approach of Harris as\nwell as the currently dominant paradigms in CoNLL.\n  Berwick avoids Chomsky's use of \"innate\" but states that \"the debate over the\nexistence of sophisticated mental grammars was settled with Chomsky's Logical\nStructure of Linguistic Theory (1957/1975)\", acknowledging that \"this debate\nhas often been revived\".\n  This paper agrees with the view that this debate has long since been settled,\nbut with the opposite outcome! Given the embers have not yet died away, and the\nquestions remain fundamental, perhaps it is appropriate to refuel the debate,\nso I would like to join Bob in throwing fuel on this fire by reviewing the\nevidence against the Chomskian position!",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00570v1",
            "date": "2016-07-02 23:10:09+00:00",
            "title": "Representation learning for very short texts using weighted word embedding aggregation",
            "authors": [
                "Cedric De Boom",
                "Steven Van Canneyt",
                "Thomas Demeester",
                "Bart Dhoedt"
            ],
            "abstract": "Short text messages such as tweets are very noisy and sparse in their use of\nvocabulary. Traditional textual representations, such as tf-idf, have\ndifficulty grasping the semantic meaning of such texts, which is important in\napplications such as event detection, opinion mining, news recommendation, etc.\nWe constructed a method based on semantic word embeddings and frequency\ninformation to arrive at low-dimensional representations for short texts\ndesigned to capture semantic similarity. For this purpose we designed a\nweight-based model and a learning procedure based on a novel median-based loss\nfunction. This paper discusses the details of our model and the optimization\nmethods, together with the experimental results on both Wikipedia and Twitter\ndata. We find that our method outperforms the baseline approaches in the\nexperiments, and that it generalizes well on different word embeddings without\nretraining. Our method is therefore capable of retaining most of the semantic\ninformation in the text, and is applicable out-of-the-box.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1607.00970v2",
            "date": "2016-07-04 17:42:52+00:00",
            "title": "Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation",
            "authors": [
                "Lili Mou",
                "Yiping Song",
                "Rui Yan",
                "Ge Li",
                "Lu Zhang",
                "Zhi Jin"
            ],
            "abstract": "Using neural networks to generate replies in human-computer dialogue systems\nis attracting increasing attention over the past few years. However, the\nperformance is not satisfactory: the neural network tends to generate safe,\nuniversally relevant replies which carry little meaning. In this paper, we\npropose a content-introducing approach to neural network-based generative\ndialogue systems. We first use pointwise mutual information (PMI) to predict a\nnoun as a keyword, reflecting the main gist of the reply. We then propose\nseq2BF, a \"sequence to backward and forward sequences\" model, which generates a\nreply containing the given keyword. Experimental results show that our approach\nsignificantly outperforms traditional sequence-to-sequence models in terms of\nhuman evaluation and the entropy measure, and that the predicted keyword can\nappear at an appropriate position in the reply.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00976v2",
            "date": "2016-07-04 18:04:18+00:00",
            "title": "Modelling Context with User Embeddings for Sarcasm Detection in Social Media",
            "authors": [
                "Silvio Amir",
                "Byron C. Wallace",
                "Hao Lyu",
                "Paula Carvalho M\u00e1rio J. Silva"
            ],
            "abstract": "We introduce a deep neural network for automated sarcasm detection. Recent\nwork has emphasized the need for models to capitalize on contextual features,\nbeyond lexical and syntactic cues present in utterances. For example, different\nspeakers will tend to employ sarcasm regarding different subjects and, thus,\nsarcasm detection models ought to encode such speaker information. Current\nmethods have achieved this by way of laborious feature engineering. By\ncontrast, we propose to automatically learn and then exploit user embeddings,\nto be used in concert with lexical signals to recognize sarcasm. Our approach\ndoes not require elaborate feature engineering (and concomitant data scraping);\nfitting user embeddings requires only the text from their previous posts. The\nexperimental results show that our model outperforms a state-of-the-art\napproach leveraging an extensive set of carefully crafted features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00992v1",
            "date": "2016-07-04 19:02:47+00:00",
            "title": "Generic Statistical Relational Entity Resolution in Knowledge Graphs",
            "authors": [
                "Jay Pujara",
                "Lise Getoor"
            ],
            "abstract": "Entity resolution, the problem of identifying the underlying entity of\nreferences found in data, has been researched for many decades in many\ncommunities. A common theme in this research has been the importance of\nincorporating relational features into the resolution process. Relational\nentity resolution is particularly important in knowledge graphs (KGs), which\nhave a regular structure capturing entities and their interrelationships. We\nidentify three major problems in KG entity resolution: (1) intra-KG reference\nambiguity; (2) inter-KG reference ambiguity; and (3) ambiguity when extending\nKGs with new facts. We implement a framework that generalizes across these\nthree settings and exploits this regular structure of KGs. Our framework has\nmany advantages over custom solutions widely deployed in industry, including\ncollective inference, scalability, and interpretability. We apply our framework\nto two real-world KG entity resolution problems, ambiguity in NELL and merging\ndata from Freebase and MusicBrainz, demonstrating the importance of relational\nfeatures.",
            "journal ref": "In the Sixth International Workshop on Statistical Relational AI,\n  2016",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.01490v1",
            "date": "2016-07-06 06:58:31+00:00",
            "title": "Towards Self-explanatory Ontology Visualization with Contextual Verbalization",
            "authors": [
                "Ren\u0101rs Liepi\u0146\u0161",
                "Uldis Boj\u0101rs",
                "Normunds Gr\u016bz\u012btis",
                "K\u0101rlis \u010cer\u0101ns",
                "Edgars Celms"
            ],
            "abstract": "Ontologies are one of the core foundations of the Semantic Web. To\nparticipate in Semantic Web projects, domain experts need to be able to\nunderstand the ontologies involved. Visual notations can provide an overview of\nthe ontology and help users to understand the connections among entities.\nHowever, the users first need to learn the visual notation before they can\ninterpret it correctly. Controlled natural language representation would be\nreadable right away and might be preferred in case of complex axioms, however,\nthe structure of the ontology would remain less apparent. We propose to combine\nontology visualizations with contextual ontology verbalizations of selected\nontology (diagram) elements, displaying controlled natural language (CNL)\nexplanations of OWL axioms corresponding to the selected visual notation\nelements. Thus, the domain experts will benefit from both the high-level\noverview provided by the graphical notation and the detailed textual\nexplanations of particular elements in the diagram.",
            "journal ref": "Databases and Information Systems, Communications in Computer and\n  Information Science, Vol. 615, Springer, 2016, pp. 3-17",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.01628v1",
            "date": "2016-07-06 14:13:12+00:00",
            "title": "Guided Alignment Training for Topic-Aware Neural Machine Translation",
            "authors": [
                "Wenhu Chen",
                "Evgeny Matusov",
                "Shahram Khadivi",
                "Jan-Thorsten Peter"
            ],
            "abstract": "In this paper, we propose an effective way for biasing the attention\nmechanism of a sequence-to-sequence neural machine translation (NMT) model\ntowards the well-studied statistical word alignment models. We show that our\nnovel guided alignment training approach improves translation quality on\nreal-life e-commerce texts consisting of product titles and descriptions,\novercoming the problems posed by many unknown words and a large type/token\nratio. We also show that meta-data associated with input texts such as topic or\ncategory information can significantly improve translation quality when used as\nan additional signal to the decoder part of the network. With both novel\nfeatures, the BLEU score of the NMT system on a product title set improves from\n18.6 to 21.3%. Even larger MT quality gains are obtained through domain\nadaptation of a general domain NMT system to e-commerce data. The developed NMT\nsystem also performs well on the IWSLT speech translation task, where an\nensemble of four variant systems outperforms the phrase-based baseline by 2.1%\nBLEU absolute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01990v1",
            "date": "2016-07-07 12:35:31+00:00",
            "title": "A Maturity Model for Public Administration as Open Translation Data Providers",
            "authors": [
                "N\u00faria Bel",
                "Mikel L. Forcada",
                "Asunci\u00f3n G\u00f3mez-P\u00e9rez"
            ],
            "abstract": "Any public administration that produces translation data can be a provider of\nuseful reusable data to meet its own translation needs and the ones of other\npublic organizations and private companies that work with texts of the same\ndomain. These data can also be crucial to produce domain-tuned Machine\nTranslation systems. The organization's management of the translation process,\nthe characteristics of the archives of the generated resources and of the\ninfrastructure available to support them determine the efficiency and the\neffectiveness with which the materials produced can be converted into reusable\ndata. However, it is of utmost importance that the organizations themselves\nfirst become aware of the goods they are producing and, second, adapt their\ninternal processes to become optimal providers. In this article, we propose a\nMaturity Model to help these organizations to achieve it by identifying the\ndifferent stages of the management of translation data that determine the path\nto the aforementioned goal.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1607.02061v2",
            "date": "2016-07-07 16:00:33+00:00",
            "title": "Representing Verbs with Rich Contexts: an Evaluation on Verb Similarity",
            "authors": [
                "Emmanuele Chersoni",
                "Enrico Santus",
                "Alessandro Lenci",
                "Philippe Blache",
                "Chu-Ren Huang"
            ],
            "abstract": "Several studies on sentence processing suggest that the mental lexicon keeps\ntrack of the mutual expectations between words. Current DSMs, however,\nrepresent context words as separate features, thereby loosing important\ninformation for word expectations, such as word interrelations. In this paper,\nwe present a DSM that addresses this issue by defining verb contexts as joint\nsyntactic dependencies. We test our representation in a verb similarity task on\ntwo datasets, showing that joint contexts achieve performances comparable to\nsingle dependencies or even better. Moreover, they are able to overcome the\ndata sparsity problem of joint feature spaces, in spite of the limited size of\nour training corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02250v3",
            "date": "2016-07-08 06:46:48+00:00",
            "title": "Consensus Attention-based Neural Networks for Chinese Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Ting Liu",
                "Zhipeng Chen",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Reading comprehension has embraced a booming in recent NLP research. Several\ninstitutes have released the Cloze-style reading comprehension data, and these\nhave greatly accelerated the research of machine comprehension. In this work,\nwe firstly present Chinese reading comprehension datasets, which consist of\nPeople Daily news dataset and Children's Fairy Tale (CFT) dataset. Also, we\npropose a consensus attention-based neural network architecture to tackle the\nCloze-style reading comprehension problem, which aims to induce a consensus\nattention over every words in the query. Experimental results show that the\nproposed neural network significantly outperforms the state-of-the-art\nbaselines in several public datasets. Furthermore, we setup a baseline for\nChinese reading comprehension task, and hopefully this would speed up the\nprocess for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02310v3",
            "date": "2016-07-08 11:01:56+00:00",
            "title": "Collaborative Training of Tensors for Compositional Distributional Semantics",
            "authors": [
                "Tamara Polajnar"
            ],
            "abstract": "Type-based compositional distributional semantic models present an\ninteresting line of research into functional representations of linguistic\nmeaning. One of the drawbacks of such models, however, is the lack of training\ndata required to train each word-type combination. In this paper we address\nthis by introducing training methods that share parameters between similar\nwords. We show that these methods enable zero-shot learning for words that have\nno training data at all, as well as enabling construction of high-quality\ntensors from very few training examples per word.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02355v1",
            "date": "2016-07-08 13:20:35+00:00",
            "title": "Lexical Based Semantic Orientation of Online Customer Reviews and Blogs",
            "authors": [
                "Aurangzeb khan",
                "Khairullah khan",
                "Shakeel Ahmad",
                "Fazal Masood Kundi",
                "Irum Tareen",
                "Muhammad Zubair Asghar"
            ],
            "abstract": "Rapid increase in internet users along with growing power of online review\nsites and social media has given birth to sentiment analysis or opinion mining,\nwhich aims at determining what other people think and comment. Sentiments or\nOpinions contain public generated content about products, services, policies\nand politics. People are usually interested to seek positive and negative\nopinions containing likes and dislikes, shared by users for features of\nparticular product or service. This paper proposed sentence-level lexical based\ndomain independent sentiment classification method for different types of data\nsuch as reviews and blogs. The proposed method is based on general lexicons\ni.e. WordNet, SentiWordNet and user defined lexical dictionaries for semantic\norientation. The relations and glosses of these dictionaries provide solution\nto the domain portability problem. The method performs better than word and\ntext level corpus based machine learning methods for semantic orientation. The\nresults show the proposed method performs better as it shows precision of 87%\nand83% at document and sentence levels respectively for online comments.",
            "journal ref": "Journal of American Science 2014;10(8)\n  http://www.jofamericanscience.org",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02501v2",
            "date": "2016-07-08 19:53:56+00:00",
            "title": "Actionable and Political Text Classification using Word Embeddings and LSTM",
            "authors": [
                "Adithya Rao",
                "Nemanja Spasojevic"
            ],
            "abstract": "In this work, we apply word embeddings and neural networks with Long\nShort-Term Memory (LSTM) to text classification problems, where the\nclassification criteria are decided by the context of the application. We\nexamine two applications in particular. The first is that of Actionability,\nwhere we build models to classify social media messages from customers of\nservice providers as Actionable or Non-Actionable. We build models for over 30\ndifferent languages for actionability, and most of the models achieve accuracy\naround 85%, with some reaching over 90% accuracy. We also show that using LSTM\nneural networks with word embeddings vastly outperform traditional techniques.\nSecond, we explore classification of messages with respect to political\nleaning, where social media messages are classified as Democratic or\nRepublican. The model is able to classify messages with a high accuracy of\n87.57%. As part of our experiments, we vary different hyperparameters of the\nneural networks, and report the effect of such variation on the accuracy. These\nactionability models have been deployed to production and help company agents\nprovide customer support by prioritizing which messages to respond to. The\nmodel for political leaning has been opened and made available for wider use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02769v1",
            "date": "2016-07-10 18:11:27+00:00",
            "title": "Annotation Methodologies for Vision and Language Dataset Creation",
            "authors": [
                "Gitit Kehat",
                "James Pustejovsky"
            ],
            "abstract": "Annotated datasets are commonly used in the training and evaluation of tasks\ninvolving natural language and vision (image description generation, action\nrecognition and visual question answering). However, many of the existing\ndatasets reflect problems that emerge in the process of data selection and\nannotation. Here we point out some of the difficulties and problems one\nconfronts when creating and validating annotated vision and language datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1607.02784v1",
            "date": "2016-07-10 20:39:24+00:00",
            "title": "Open Information Extraction",
            "authors": [
                "Duc-Thuan Vo",
                "Ebrahim Bagheri"
            ],
            "abstract": "Open Information Extraction (Open IE) systems aim to obtain relation tuples\nwith highly scalable extraction in portable across domain by identifying a\nvariety of relation phrases and their arguments in arbitrary sentences. The\nfirst generation of Open IE learns linear chain models based on unlexicalized\nfeatures such as Part-of-Speech (POS) or shallow tags to label the intermediate\nwords between pair of potential arguments for identifying extractable\nrelations. Open IE currently is developed in the second generation that is able\nto extract instances of the most frequently observed relation types such as\nVerb, Noun and Prep, Verb and Prep, and Infinitive with deep linguistic\nanalysis. They expose simple yet principled ways in which verbs express\nrelationships in linguistics such as verb phrase-based extraction or\nclause-based extraction. They obtain a significantly higher performance over\nprevious systems in the first generation. In this paper, we describe an\noverview of two Open IE generations including strengths, weaknesses and\napplication areas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03055v1",
            "date": "2016-07-11 17:48:53+00:00",
            "title": "Exploring the Political Agenda of the European Parliament Using a Dynamic Topic Modeling Approach",
            "authors": [
                "Derek Greene",
                "James P. Cross"
            ],
            "abstract": "This study analyzes the political agenda of the European Parliament (EP)\nplenary, how it has evolved over time, and the manner in which Members of the\nEuropean Parliament (MEPs) have reacted to external and internal stimuli when\nmaking plenary speeches. To unveil the plenary agenda and detect latent themes\nin legislative speeches over time, MEP speech content is analyzed using a new\ndynamic topic modeling method based on two layers of Non-negative Matrix\nFactorization (NMF). This method is applied to a new corpus of all English\nlanguage legislative speeches in the EP plenary from the period 1999-2014. Our\nfindings suggest that two-layer NMF is a valuable alternative to existing\ndynamic topic modeling approaches found in the literature, and can unveil niche\ntopics and associated vocabularies not captured by existing methods.\nSubstantively, our findings suggest that the political agenda of the EP evolves\nsignificantly over time and reacts to exogenous events such as EU Treaty\nreferenda and the emergence of the Euro-crisis. MEP contributions to the\nplenary agenda are also found to be impacted upon by voting behaviour and the\ncommittee structure of the Parliament.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03316v3",
            "date": "2016-07-12 11:43:15+00:00",
            "title": "Separating Answers from Queries for Neural Reading Comprehension",
            "authors": [
                "Dirk Weissenborn"
            ],
            "abstract": "We present a novel neural architecture for answering queries, designed to\noptimally leverage explicit support in the form of query-answer memories. Our\nmodel is able to refine and update a given query while separately accumulating\nevidence for predicting the answer. Its architecture reflects this separation\nwith dedicated embedding matrices and loosely connected information pathways\n(modules) for updating the query and accumulating evidence. This separation of\nresponsibilities effectively decouples the search for query related support and\nthe prediction of the answer. On recent benchmark datasets for reading\ncomprehension, our model achieves state-of-the-art results. A qualitative\nanalysis reveals that the model effectively accumulates weighted evidence from\nthe query and over multiple support retrieval cycles which results in a robust\nanswer prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03707v1",
            "date": "2016-07-13 12:48:33+00:00",
            "title": "Re-presenting a Story by Emotional Factors using Sentimental Analysis Method",
            "authors": [
                "Hwiyeol Jo",
                "Yohan Moon",
                "Jong In Kim",
                "Jeong Ryu"
            ],
            "abstract": "Remembering an event is affected by personal emotional status. We examined\nthe psychological status and personal factors; depression (Center for\nEpidemiological Studies - Depression, Radloff, 1977), present affective\n(Positive Affective and Negative Affective Schedule, Watson et al., 1988), life\norient (Life Orient Test, Scheier & Carver, 1985), self-awareness (Core Self\nEvaluation Scale, Judge et al., 2003), and social factor (Social Support,\nSarason et al., 1983) of undergraduate students (N=64) and got summaries of a\nstory, Chronicle of a Death Foretold (Gabriel Garcia Marquez, 1981) from them.\nWe implement a sentimental analysis model based on convolutional neural network\n(LeCun & Bengio, 1995) to evaluate each summary. From the same vein used for\ntransfer learning (Pan & Yang, 2010), we collected 38,265 movie review data to\ntrain the model and then use them to score summaries of each student. The\nresults of CES-D and PANAS show the relationship between emotion and memory\nretrieval as follows: depressed people have shown a tendency of representing a\nstory more negatively, and they seemed less expressive. People with full of\nemotion - high in PANAS - have retrieved their memory more expressively than\nothers, using more negative words then others. The contributions of this study\ncan be summarized as follows: First, lightening the relationship between\nemotion and its effect during times of storing or retrieving a memory. Second,\nsuggesting objective methods to evaluate the intensity of emotion in natural\nlanguage format, using a sentimental analysis model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03766v3",
            "date": "2016-07-13 14:31:54+00:00",
            "title": "AudioPairBank: Towards A Large-Scale Tag-Pair-Based Audio Content Analysis",
            "authors": [
                "Sebastian Sager",
                "Benjamin Elizalde",
                "Damian Borth",
                "Christian Schulze",
                "Bhiksha Raj",
                "Ian Lane"
            ],
            "abstract": "Recently, sound recognition has been used to identify sounds, such as car and\nriver. However, sounds have nuances that may be better described by\nadjective-noun pairs such as slow car, and verb-noun pairs such as flying\ninsects, which are under explored. Therefore, in this work we investigate the\nrelation between audio content and both adjective-noun pairs and verb-noun\npairs. Due to the lack of datasets with these kinds of annotations, we\ncollected and processed the AudioPairBank corpus consisting of a combined total\nof 1,123 pairs and over 33,000 audio files. One contribution is the previously\nunavailable documentation of the challenges and implications of collecting\naudio recordings with these type of labels. A second contribution is to show\nthe degree of correlation between the audio content and the labels through\nsound recognition experiments, which yielded results of 70% accuracy, hence\nalso providing a performance benchmark. The results and study in this paper\nencourage further exploration of the nuances in audio and are meant to\ncomplement similar research performed on images and text in multimedia\nanalysis.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1607.03780v1",
            "date": "2016-07-13 15:08:26+00:00",
            "title": "A Vector Space for Distributional Semantics for Entailment",
            "authors": [
                "James Henderson",
                "Diana Nicoleta Popa"
            ],
            "abstract": "Distributional semantics creates vector-space representations that capture\nmany forms of semantic similarity, but their relation to semantic entailment\nhas been less clear. We propose a vector-space model which provides a formal\nfoundation for a distributional semantics of entailment. Using a mean-field\napproximation, we develop approximate inference procedures and entailment\noperators over vectors of probabilities of features being known (versus\nunknown). We use this framework to reinterpret an existing\ndistributional-semantic model (Word2Vec) as approximating an entailment-based\nmodel of the distributions of words in contexts, thereby predicting lexical\nentailment relations. In both unsupervised and semi-supervised experiments on\nhyponymy detection, we get substantial improvements over previous results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03895v1",
            "date": "2016-07-13 20:00:01+00:00",
            "title": "Tie-breaker: Using language models to quantify gender bias in sports journalism",
            "authors": [
                "Liye Fu",
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee"
            ],
            "abstract": "Gender bias is an increasingly important issue in sports journalism. In this\nwork, we propose a language-model-based approach to quantify differences in\nquestions posed to female vs. male athletes, and apply it to tennis post-match\ninterviews. We find that journalists ask male players questions that are\ngenerally more focused on the game when compared with the questions they ask\ntheir female counterparts. We also provide a fine-grained analysis of the\nextent to which the salience of this bias depends on various factors, such as\nquestion type, game outcome or player rank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04110v1",
            "date": "2016-07-14 12:45:07+00:00",
            "title": "Using Recurrent Neural Network for Learning Expressive Ontologies",
            "authors": [
                "Giulio Petrucci",
                "Chiara Ghidini",
                "Marco Rospocher"
            ],
            "abstract": "Recently, Neural Networks have been proven extremely effective in many\nnatural language processing tasks such as sentiment analysis, question\nanswering, or machine translation. Aiming to exploit such advantages in the\nOntology Learning process, in this technical report we present a detailed\ndescription of a Recurrent Neural Network based system to be used to pursue\nsuch goal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04423v4",
            "date": "2016-07-15 09:10:11+00:00",
            "title": "Attention-over-Attention Neural Networks for Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Zhipeng Chen",
                "Si Wei",
                "Shijin Wang",
                "Ting Liu",
                "Guoping Hu"
            ],
            "abstract": "Cloze-style queries are representative problems in reading comprehension.\nOver the past few months, we have seen much progress that utilizing neural\nnetwork approach to solve Cloze-style questions. In this paper, we present a\nnovel model called attention-over-attention reader for the Cloze-style reading\ncomprehension task. Our model aims to place another attention mechanism over\nthe document-level attention, and induces \"attended attention\" for final\npredictions. Unlike the previous works, our neural network model requires less\npre-defined hyper-parameters and uses an elegant architecture for modeling.\nExperimental results show that the proposed attention-over-attention model\nsignificantly outperforms various state-of-the-art systems by a large margin in\npublic datasets, such as CNN and Children's Book Test datasets.",
            "journal ref": "ACL 2017 Vol.1 Long Papers 593-602",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04576v1",
            "date": "2016-07-15 16:43:40+00:00",
            "title": "Neural Discourse Modeling of Conversations",
            "authors": [
                "John M. Pierre",
                "Mark Butler",
                "Jacob Portnoff",
                "Luis Aguilar"
            ],
            "abstract": "Deep neural networks have shown recent promise in many language-related tasks\nsuch as the modeling of conversations. We extend RNN-based sequence to sequence\nmodels to capture the long range discourse across many turns of conversation.\nWe perform a sensitivity analysis on how much additional context affects\nperformance, and provide quantitative and qualitative evidence that these\nmodels are able to capture discourse relationships across multiple utterances.\nOur results quantifies how adding an additional RNN layer for modeling\ndiscourse improves the quality of output utterances and providing more of the\nprevious conversation as input also improves performance. By searching the\ngenerated outputs for specific discourse markers we show how neural discourse\nmodels can exhibit increased coherence and cohesion in conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04606v2",
            "date": "2016-07-15 18:27:55+00:00",
            "title": "Enriching Word Vectors with Subword Information",
            "authors": [
                "Piotr Bojanowski",
                "Edouard Grave",
                "Armand Joulin",
                "Tomas Mikolov"
            ],
            "abstract": "Continuous word representations, trained on large unlabeled corpora are\nuseful for many natural language processing tasks. Popular models that learn\nsuch representations ignore the morphology of words, by assigning a distinct\nvector to each word. This is a limitation, especially for languages with large\nvocabularies and many rare words. In this paper, we propose a new approach\nbased on the skipgram model, where each word is represented as a bag of\ncharacter $n$-grams. A vector representation is associated to each character\n$n$-gram; words being represented as the sum of these representations. Our\nmethod is fast, allowing to train models on large corpora quickly and allows us\nto compute word representations for words that did not appear in the training\ndata. We evaluate our word representations on nine different languages, both on\nword similarity and analogy tasks. By comparing to recently proposed\nmorphological word representations, we show that our vectors achieve\nstate-of-the-art performance on these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04660v1",
            "date": "2016-05-16 12:55:36+00:00",
            "title": "Identification of promising research directions using machine learning aided medical literature analysis",
            "authors": [
                "Victor Andrei",
                "Ognjen Arandjelovic"
            ],
            "abstract": "The rapidly expanding corpus of medical research literature presents major\nchallenges in the understanding of previous work, the extraction of maximum\ninformation from collected data, and the identification of promising research\ndirections. We present a case for the use of advanced machine learning\ntechniques as an aide in this task and introduce a novel methodology that is\nshown to be capable of extracting meaningful information from large\nlongitudinal corpora, and of tracking complex temporal changes within it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.04683v2",
            "date": "2016-07-15 23:31:45+00:00",
            "title": "On the efficient representation and execution of deep acoustic models",
            "authors": [
                "Raziel Alvarez",
                "Rohit Prabhavalkar",
                "Anton Bakhtin"
            ],
            "abstract": "In this paper we present a simple and computationally efficient quantization\nscheme that enables us to reduce the resolution of the parameters of a neural\nnetwork from 32-bit floating point values to 8-bit integer values. The proposed\nquantization scheme leads to significant memory savings and enables the use of\noptimized hardware instructions for integer arithmetic, thus significantly\nreducing the cost of inference. Finally, we propose a \"quantization aware\"\ntraining process that applies the proposed scheme during network training and\nfind that it allows us to recover most of the loss in accuracy introduced by\nquantization. We validate the proposed techniques by applying them to a long\nshort-term memory-based acoustic model on an open-ended large vocabulary speech\nrecognition task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1607.05108v1",
            "date": "2016-07-18 14:44:26+00:00",
            "title": "Neural Machine Translation with Recurrent Attention Modeling",
            "authors": [
                "Zichao Yang",
                "Zhiting Hu",
                "Yuntian Deng",
                "Chris Dyer",
                "Alex Smola"
            ],
            "abstract": "Knowing which words have been attended to in previous time steps while\ngenerating a translation is a rich source of information for predicting what\nwords will be attended to in the future. We improve upon the attention model of\nBahdanau et al. (2014) by explicitly modeling the relationship between previous\nand subsequent attention levels for each word using one recurrent network per\ninput word. This architecture easily captures informative features, such as\nfertility and regularities in relative distortion. In experiments, we show our\nparameterization of attention improves translation quality.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1607.05422v1",
            "date": "2016-07-19 06:32:26+00:00",
            "title": "A Novel Information Theoretic Framework for Finding Semantic Similarity in WordNet",
            "authors": [
                "Abhijit Adhikari",
                "Shivang Singh",
                "Deepjyoti Mondal",
                "Biswanath Dutta",
                "Animesh Dutta"
            ],
            "abstract": "Information content (IC) based measures for finding semantic similarity is\ngaining preferences day by day. Semantics of concepts can be highly\ncharacterized by information theory. The conventional way for calculating IC is\nbased on the probability of appearance of concepts in corpora. Due to data\nsparseness and corpora dependency issues of those conventional approaches, a\nnew corpora independent intrinsic IC calculation measure has evolved. In this\npaper, we mainly focus on such intrinsic IC model and several topological\naspects of the underlying ontology. Accuracy of intrinsic IC calculation and\nsemantic similarity measure rely on these aspects deeply. Based on these\nanalysis we propose an information theoretic framework which comprises an\nintrinsic IC calculator and a semantic similarity model. Our approach is\ncompared with state of the art semantic similarity measures based on corpora\ndependent IC calculation as well as intrinsic IC based methods using several\nbenchmark data set. We also compare our model with the related Edge based,\nFeature based and Distributional approaches. Experimental results show that our\nintrinsic IC model gives high correlation value when applied to different\nsemantic similarity models. Our proposed semantic similarity model also\nachieves significant results when embedded with some state of the art IC models\nincluding ours.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1607.05650v2",
            "date": "2016-07-13 06:57:38+00:00",
            "title": "A Supervised Authorship Attribution Framework for Bengali Language",
            "authors": [
                "Shanta Phani",
                "Shibamouli Lahiri",
                "Arindam Biswas"
            ],
            "abstract": "Authorship Attribution is a long-standing problem in Natural Language\nProcessing. Several statistical and computational methods have been used to\nfind a solution to this problem. In this paper, we have proposed methods to\ndeal with the authorship attribution problem in Bengali.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05666v1",
            "date": "2016-07-19 17:17:58+00:00",
            "title": "Trainable Frontend For Robust and Far-Field Keyword Spotting",
            "authors": [
                "Yuxuan Wang",
                "Pascal Getreuer",
                "Thad Hughes",
                "Richard F. Lyon",
                "Rif A. Saurous"
            ],
            "abstract": "Robust and far-field speech recognition is critical to enable true hands-free\ncommunication. In far-field conditions, signals are attenuated due to distance.\nTo improve robustness to loudness variation, we introduce a novel frontend\ncalled per-channel energy normalization (PCEN). The key ingredient of PCEN is\nthe use of an automatic gain control based dynamic compression to replace the\nwidely used static (such as log or root) compression. We evaluate PCEN on the\nkeyword spotting task. On our large rerecorded noisy and far-field eval sets,\nwe show that PCEN significantly improves recognition performance. Furthermore,\nwe model PCEN as neural network layers and optimize high-dimensional PCEN\nparameters jointly with the keyword spotting acoustic model. The trained PCEN\nfrontend demonstrates significant further improvements without increasing model\ncomplexity or inference-time cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05809v1",
            "date": "2016-07-20 03:25:31+00:00",
            "title": "Neural Contextual Conversation Learning with Labeled Question-Answering Pairs",
            "authors": [
                "Kun Xiong",
                "Anqi Cui",
                "Zefeng Zhang",
                "Ming Li"
            ],
            "abstract": "Neural conversational models tend to produce generic or safe responses in\ndifferent contexts, e.g., reply \\textit{\"Of course\"} to narrative statements or\n\\textit{\"I don't know\"} to questions. In this paper, we propose an end-to-end\napproach to avoid such problem in neural generative models. Additional memory\nmechanisms have been introduced to standard sequence-to-sequence (seq2seq)\nmodels, so that context can be considered while generating sentences. Three\nseq2seq models, which memorize a fix-sized contextual vector from hidden input,\nhidden input/output and a gated contextual attention structure respectively,\nhave been trained and tested on a dataset of labeled question-answering pairs\nin Chinese. The model with contextual attention outperforms others including\nthe state-of-the-art seq2seq models on perplexity test. The novel contextual\nmodel generates diverse and robust responses, and is able to carry out\nconversations on a wide range of topics appropriately.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05968v1",
            "date": "2016-07-20 14:15:24+00:00",
            "title": "Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics and Construction Grammar for Spatial Language",
            "authors": [
                "Michael Spranger",
                "Jakob Suchan",
                "Mehul Bhatt"
            ],
            "abstract": "We present a system for generating and understanding of dynamic and static\nspatial relations in robotic interaction setups. Robots describe an environment\nof moving blocks using English phrases that include spatial relations such as\n\"across\" and \"in front of\". We evaluate the system in robot-robot interactions\nand show that the system can robustly deal with visual perception errors,\nlanguage omissions and ungrammatical utterances.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.06153v1",
            "date": "2016-07-20 23:26:33+00:00",
            "title": "Compositional Sequence Labeling Models for Error Detection in Learner Writing",
            "authors": [
                "Marek Rei",
                "Helen Yannakoudakis"
            ],
            "abstract": "In this paper, we present the first experiments using neural network models\nfor the task of error detection in learner writing. We perform a systematic\ncomparison of alternative compositional architectures and propose a framework\nfor error detection based on bidirectional LSTMs. Experiments on the CoNLL-14\nshared task dataset show the model is able to outperform other participants on\ndetecting errors in learner writing. Finally, the model is integrated with a\npublicly deployed self-assessment system, leading to performance comparable to\nhuman annotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06560v1",
            "date": "2016-07-22 05:38:37+00:00",
            "title": "Automated Prediction of Temporal Relations",
            "authors": [
                "Amol S Patwardhan",
                "Jacob Badeaux",
                "Siavash",
                "Gerald M Knapp"
            ],
            "abstract": "Background: There has been growing research interest in automated answering\nof questions or generation of summary of free form text such as news article.\nIn order to implement this task, the computer should be able to identify the\nsequence of events, duration of events, time at which event occurred and the\nrelationship type between event pairs, time pairs or event-time pairs. Specific\nProblem: It is important to accurately identify the relationship type between\ncombinations of event and time before the temporal ordering of events can be\ndefined. The machine learning approach taken in Mani et. al (2006) provides an\naccuracy of only 62.5 on the baseline data from TimeBank. The researchers used\nmaximum entropy classifier in their methodology. TimeML uses the TLINK\nannotation to tag a relationship type between events and time. The time\ncomplexity is quadratic when it comes to tagging documents with TLINK using\nhuman annotation. This research proposes using decision tree and parsing to\nimprove the relationship type tagging. This research attempts to solve the gaps\nin human annotation by automating the task of relationship type tagging in an\nattempt to improve the accuracy of event and time relationship in annotated\ndocuments. Scope information: The documents from the domain of news will be\nused. The tagging will be performed within the same document and not across\ndocuments. The relationship types will be identified only for a pair of event\nand time and not a chain of events. The research focuses on documents tagged\nusing the TimeML specification which contains tags such as EVENT, TLINK, and\nTIMEX. Each tag has attributes such as identifier, relation, POS, time etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07788v1",
            "date": "2016-02-05 17:59:55+00:00",
            "title": "How scientific literature has been evolving over the time? A novel statistical approach using tracking verbal-based methods",
            "authors": [
                "Daria Micaela Hernandez",
                "Monica Becue-Bertaut",
                "Igor Barahona"
            ],
            "abstract": "This paper provides a global vision of the scientific publications related\nwith the Systemic Lupus Erythematosus (SLE), taking as starting point abstracts\nof articles. Through the time, abstracts have been evolving towards higher\ncomplexity on used terminology, which makes necessary the use of sophisticated\nstatistical methods and answering questions including: how vocabulary is\nevolving through the time? Which ones are most influential articles? And which\none are the articles that introduced new terms and vocabulary? To answer these,\nwe analyze a dataset composed by 506 abstracts and downloaded from 115\ndifferent journals and cover a 18 year-period.",
            "journal ref": "JSM Proceedings (2014), Section on Statistical Learning and Data\n  Mining. Alexandria, VA. American Statistical Association. 1121-1131",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07956v1",
            "date": "2016-07-27 04:51:17+00:00",
            "title": "Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification",
            "authors": [
                "Yuezhang Li",
                "Ronghuo Zheng",
                "Tian Tian",
                "Zhiting Hu",
                "Rahul Iyer",
                "Katia Sycara"
            ],
            "abstract": "Due to the lack of structured knowledge applied in learning distributed\nrepresentation of cate- gories, existing work cannot incorporate category\nhierarchies into entity information. We propose a framework that embeds\nentities and categories into a semantic space by integrating structured\nknowledge and taxonomy hierarchy from large knowledge bases. The framework\nallows to com- pute meaningful semantic relatedness between entities and\ncategories. Our framework can han- dle both single-word concepts and\nmultiple-word concepts with superior performance on concept categorization and\nyield state of the art results on dataless hierarchical classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08074v1",
            "date": "2016-07-27 13:08:41+00:00",
            "title": "Mining Arguments from Cancer Documents Using Natural Language Processing and Ontologies",
            "authors": [
                "Adrian Groza",
                "Oana Popa"
            ],
            "abstract": "In the medical domain, the continuous stream of scientific research contains\ncontradictory results supported by arguments and counter-arguments. As medical\nexpertise occurs at different levels, part of the human agents have\ndifficulties to face the huge amount of studies, but also to understand the\nreasons and pieces of evidences claimed by the proponents and the opponents of\nthe debated topic. To better understand the supporting arguments for new\nfindings related to current state of the art in the medical domain we need\ntools able to identify arguments in scientific papers. Our work here aims to\nfill the above technological gap.\n  Quite aware of the difficulty of this task, we embark to this road by relying\non the well-known interleaving of domain knowledge with natural language\nprocessing. To formalise the existing medical knowledge, we rely on ontologies.\nTo structure the argumentation model we use also the expressivity and reasoning\ncapabilities of Description Logics. To perform argumentation mining we\nformalise various linguistic patterns in a rule-based language. We tested our\nsolution against a corpus of scientific papers related to breast cancer. The\nrun experiments show a F-measure between 0.71 and 0.86 for identifying\nconclusions of an argument and between 0.65 and 0.86 for identifying premises\nof an argument.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.08592v1",
            "date": "2016-07-28 19:47:25+00:00",
            "title": "Modeling selectional restrictions in a relational type system",
            "authors": [
                "Erkki Luuk"
            ],
            "abstract": "Selectional restrictions are semantic constraints on forming certain complex\ntypes in natural language. The paper gives an overview of modeling selectional\nrestrictions in a relational type system with morphological and syntactic\ntypes. We discuss some foundations of the system and ways of formalizing\nselectional restrictions.\n  Keywords: type theory, selectional restrictions, syntax, morphology",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08822v1",
            "date": "2016-07-29 14:26:27+00:00",
            "title": "SPICE: Semantic Propositional Image Caption Evaluation",
            "authors": [
                "Peter Anderson",
                "Basura Fernando",
                "Mark Johnson",
                "Stephen Gould"
            ],
            "abstract": "There is considerable interest in the task of automatically generating image\ncaptions. However, evaluation is challenging. Existing automatic evaluation\nmetrics are primarily sensitive to n-gram overlap, which is neither necessary\nnor sufficient for the task of simulating human judgment. We hypothesize that\nsemantic propositional content is an important component of human caption\nevaluation, and propose a new automated caption evaluation metric defined over\nscene graphs coined SPICE. Extensive evaluations across a range of models and\ndatasets indicate that SPICE captures human judgments over model-generated\ncaptions better than other automatic metrics (e.g., system-level correlation of\n0.88 with human judgments on the MS COCO dataset, versus 0.43 for CIDEr and\n0.53 for METEOR). Furthermore, SPICE can answer questions such as `which\ncaption-generator best understands colors?' and `can caption-generators count?'",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1607.08883v1",
            "date": "2016-07-29 18:20:24+00:00",
            "title": "Labeling of Query Words using Conditional Random Field",
            "authors": [
                "Satanu Ghosh",
                "Souvick Ghosh",
                "Dipankar Das"
            ],
            "abstract": "This paper describes our approach on Query Word Labeling as an attempt in the\nshared task on Mixed Script Information Retrieval at Forum for Information\nRetrieval Evaluation (FIRE) 2015. The query is written in Roman script and the\nwords were in English or transliterated from Indian regional languages. A total\nof eight Indian languages were present in addition to English. We also\nidentified the Named Entities and special symbols as part of our task. A CRF\nbased machine learning framework was used for labeling the individual words\nwith their corresponding language labels. We used a dictionary based approach\nfor language identification. We also took into account the context of the word\nwhile identifying the language. Our system demonstrated an overall accuracy of\n75.5% for token level language identification. The strict F-measure scores for\nthe identification of token level language labels for Bengali, English and\nHindi are 0.7486, 0.892 and 0.7972 respectively. The overall weighted F-measure\nof our system was 0.7498.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1607.08898v1",
            "date": "2016-07-29 19:16:08+00:00",
            "title": "Personalized Emphasis Framing for Persuasive Message Generation",
            "authors": [
                "Tao Ding",
                "Shimei Pan"
            ],
            "abstract": "In this paper, we present a study on personalized emphasis framing which can\nbe used to tailor the content of a message to enhance its appeal to different\nindividuals. With this framework, we directly model content selection decisions\nbased on a set of psychologically-motivated domain-independent personal traits\nincluding personality (e.g., extraversion and conscientiousness) and basic\nhuman values (e.g., self-transcendence and hedonism). We also demonstrate how\nthe analysis results can be used in automated personalized content selection\nfor persuasive message generation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.00372v2",
            "date": "2016-01-04 03:04:05+00:00",
            "title": "Mutual Information and Diverse Decoding Improve Neural Machine Translation",
            "authors": [
                "Jiwei Li",
                "Dan Jurafsky"
            ],
            "abstract": "Sequence-to-sequence neural translation models learn semantic and syntactic\nrelations between sentence pairs by optimizing the likelihood of the target\ngiven the source, i.e., $p(y|x)$, an objective that ignores other potentially\nuseful sources of information. We introduce an alternative objective function\nfor neural MT that maximizes the mutual information between the source and\ntarget sentences, modeling the bi-directional dependency of sources and\ntargets. We implement the model with a simple re-ranking method, and also\nintroduce a decoding algorithm that increases diversity in the N-best list\nproduced by the first pass. Applied to the WMT German/English and\nFrench/English tasks, the proposed models offers a consistent performance boost\non both standard LSTM and attention-based neural MT architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00770v3",
            "date": "2016-01-05 08:53:05+00:00",
            "title": "End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures",
            "authors": [
                "Makoto Miwa",
                "Mohit Bansal"
            ],
            "abstract": "We present a novel end-to-end neural model to extract entities and relations\nbetween them. Our recurrent neural network based model captures both word\nsequence and dependency tree substructure information by stacking bidirectional\ntree-structured LSTM-RNNs on bidirectional sequential LSTM-RNNs. This allows\nour model to jointly represent both entities and relations with shared\nparameters in a single model. We further encourage detection of entities during\ntraining and use of entity information in relation extraction via entity\npretraining and scheduled sampling. Our model improves over the\nstate-of-the-art feature-based model on end-to-end relation extraction,\nachieving 12.1% and 5.7% relative error reductions in F1-score on ACE2005 and\nACE2004, respectively. We also show that our LSTM-RNN based model compares\nfavorably to the state-of-the-art CNN based model (in F1-score) on nominal\nrelation classification (SemEval-2010 Task 8). Finally, we present an extensive\nablation analysis of several model components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00901v1",
            "date": "2016-01-05 16:56:28+00:00",
            "title": "Joint learning of ontology and semantic parser from text",
            "authors": [
                "Janez Starc",
                "Dunja Mladeni\u0107"
            ],
            "abstract": "Semantic parsing methods are used for capturing and representing semantic\nmeaning of text. Meaning representation capturing all the concepts in the text\nmay not always be available or may not be sufficiently complete. Ontologies\nprovide a structured and reasoning-capable way to model the content of a\ncollection of texts. In this work, we present a novel approach to joint\nlearning of ontology and semantic parser from text. The method is based on\nsemi-automatic induction of a context-free grammar from semantically annotated\ntext. The grammar parses the text into semantic trees. Both, the grammar and\nthe semantic trees are used to learn the ontology on several levels -- classes,\ninstances, taxonomic and non-taxonomic relations. The approach was evaluated on\nthe first sentences of Wikipedia pages describing people.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.01073v1",
            "date": "2016-01-06 04:00:50+00:00",
            "title": "Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism",
            "authors": [
                "Orhan Firat",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "We propose multi-way, multilingual neural machine translation. The proposed\napproach enables a single neural translation model to translate between\nmultiple languages, with a number of parameters that grows only linearly with\nthe number of languages. This is made possible by having a single attention\nmechanism that is shared across all language pairs. We train the proposed\nmulti-way, multilingual model on ten language pairs from WMT'15 simultaneously\nand observe clear performance improvements over models trained on only one\nlanguage pair. In particular, we observe that the proposed model significantly\nimproves the translation quality of low-resource language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.01887v1",
            "date": "2016-01-08 14:29:44+00:00",
            "title": "Research Project: Text Engineering Tool for Ontological Scientometry",
            "authors": [
                "Rustam Tagiew"
            ],
            "abstract": "The number of scientific papers grows exponentially in many disciplines. The\nshare of online available papers grows as well. At the same time, the period of\ntime for a paper to loose at chance to be cited anymore shortens. The decay of\nthe citing rate shows similarity to ultradiffusional processes as for other\nonline contents in social networks. The distribution of papers per author shows\nsimilarity to the distribution of posts per user in social networks. The rate\nof uncited papers for online available papers grows while some papers 'go\nviral' in terms of being cited. Summarized, the practice of scientific\npublishing moves towards the domain of social networks. The goal of this\nproject is to create a text engineering tool, which can semi-automatically\ncategorize a paper according to its type of contribution and extract\nrelationships between them into an ontological database. Semi-automatic\ncategorization means that the mistakes made by automatic pre-categorization and\nrelationship-extraction will be corrected through a wikipedia-like front-end by\nvolunteers from general public. This tool should not only help researchers and\nthe general public to find relevant supplementary material and peers faster,\nbut also provide more information for research funding agencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02539v1",
            "date": "2016-01-11 17:54:53+00:00",
            "title": "Investigating gated recurrent neural networks for speech synthesis",
            "authors": [
                "Zhizheng Wu",
                "Simon King"
            ],
            "abstract": "Recently, recurrent neural networks (RNNs) as powerful sequence models have\nre-emerged as a potential acoustic model for statistical parametric speech\nsynthesis (SPSS). The long short-term memory (LSTM) architecture is\nparticularly attractive because it addresses the vanishing gradient problem in\nstandard RNNs, making them easier to train. Although recent studies have\ndemonstrated that LSTMs can achieve significantly better performance on SPSS\nthan deep feed-forward neural networks, little is known about why. Here we\nattempt to answer two questions: a) why do LSTMs work well as a sequence model\nfor SPSS; b) which component (e.g., input gate, output gate, forget gate) is\nmost important. We present a visual analysis alongside a series of experiments,\nresulting in a proposal for a simplified architecture. The simplified\narchitecture has significantly fewer parameters than an LSTM, thus reducing\ngeneration complexity considerably without degrading quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03651v2",
            "date": "2016-01-14 16:30:41+00:00",
            "title": "Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation",
            "authors": [
                "Yan Xu",
                "Ran Jia",
                "Lili Mou",
                "Ge Li",
                "Yunchuan Chen",
                "Yangyang Lu",
                "Zhi Jin"
            ],
            "abstract": "Nowadays, neural networks play an important role in the task of relation\nclassification. By designing different neural architectures, researchers have\nimproved the performance to a large extent in comparison with traditional\nmethods. However, existing neural networks for relation classification are\nusually of shallow architectures (e.g., one-layer convolutional neural networks\nor recurrent networks). They may fail to explore the potential representation\nspace in different abstraction levels. In this paper, we propose deep recurrent\nneural networks (DRNNs) for relation classification to tackle this challenge.\nFurther, we propose a data augmentation method by leveraging the directionality\nof relations. We evaluated our DRNNs on the SemEval-2010 Task~8, and achieve an\nF1-score of 86.1%, outperforming previous state-of-the-art recorded results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03896v2",
            "date": "2016-01-15 12:50:32+00:00",
            "title": "Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures",
            "authors": [
                "Raffaella Bernardi",
                "Ruket Cakici",
                "Desmond Elliott",
                "Aykut Erdem",
                "Erkut Erdem",
                "Nazli Ikizler-Cinbis",
                "Frank Keller",
                "Adrian Muscat",
                "Barbara Plank"
            ],
            "abstract": "Automatic description generation from natural images is a challenging problem\nthat has recently received a large amount of interest from the computer vision\nand natural language processing communities. In this survey, we classify the\nexisting approaches based on how they conceptualize this problem, viz., models\nthat cast description as either generation problem or as a retrieval problem\nover a visual or multimodal representational space. We provide a detailed\nreview of existing models, highlighting their advantages and disadvantages.\nMoreover, we give an overview of the benchmark image datasets and the\nevaluation measures that have been developed to assess the quality of\nmachine-generated image descriptions. Finally we extrapolate future directions\nin the area of automatic image description generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04468v1",
            "date": "2016-01-18 11:09:02+00:00",
            "title": "Bandit Structured Prediction for Learning from Partial Feedback in Statistical Machine Translation",
            "authors": [
                "Artem Sokolov",
                "Stefan Riezler",
                "Tanguy Urvoy"
            ],
            "abstract": "We present an approach to structured prediction from bandit feedback, called\nBandit Structured Prediction, where only the value of a task loss function at a\nsingle predicted point, instead of a correct structure, is observed in\nlearning. We present an application to discriminative reranking in Statistical\nMachine Translation (SMT) where the learning algorithm only has access to a\n1-BLEU loss evaluation of a predicted translation instead of obtaining a gold\nstandard reference translation. In our experiment bandit feedback is obtained\nby evaluating BLEU on reference translations without revealing them to the\nalgorithm. This can be thought of as a simulation of interactive machine\ntranslation where an SMT system is personalized by a user who provides single\npoint feedback to predicted translations. Our experiments show that our\napproach improves translation quality and is comparable to approaches that\nemploy more informative feedback in learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04580v2",
            "date": "2016-01-18 15:46:00+00:00",
            "title": "Nonparametric Bayesian Storyline Detection from Microtexts",
            "authors": [
                "Vinodh Krishnan",
                "Jacob Eisenstein"
            ],
            "abstract": "News events and social media are composed of evolving storylines, which\ncapture public attention for a limited period of time. Identifying storylines\nrequires integrating temporal and linguistic information, and prior work takes\na largely heuristic approach. We present a novel online non-parametric Bayesian\nframework for storyline detection, using the distance-dependent Chinese\nRestaurant Process (dd-CRP). To ensure efficient linear-time inference, we\nemploy a fixed-lag Gibbs sampling procedure, which is novel for the dd-CRP. We\nevaluate on the TREC Twitter Timeline Generation (TTG), obtaining encouraging\nresults: despite using a weak baseline retrieval model, the dd-CRP story\nclustering method is competitive with the best entries in the 2014 TTG task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05194v1",
            "date": "2016-01-20 08:26:07+00:00",
            "title": "Improved Spoken Document Summarization with Coverage Modeling Techniques",
            "authors": [
                "Kuan-Yu Chen",
                "Shih-Hung Liu",
                "Berlin Chen",
                "Hsin-Min Wang"
            ],
            "abstract": "Extractive summarization aims at selecting a set of indicative sentences from\na source document as a summary that can express the major theme of the\ndocument. A general consensus on extractive summarization is that both\nrelevance and coverage are critical issues to address. The existing methods\ndesigned to model coverage can be characterized by either reducing redundancy\nor increasing diversity in the summary. Maximal margin relevance (MMR) is a\nwidely-cited method since it takes both relevance and redundancy into account\nwhen generating a summary for a given document. In addition to MMR, there is\nonly a dearth of research concentrating on reducing redundancy or increasing\ndiversity for the spoken document summarization task, as far as we are aware.\nMotivated by these observations, two major contributions are presented in this\npaper. First, in contrast to MMR, which considers coverage by reducing\nredundancy, we propose two novel coverage-based methods, which directly\nincrease diversity. With the proposed methods, a set of representative\nsentences, which not only are relevant to the given document but also cover\nmost of the important sub-themes of the document, can be selected\nautomatically. Second, we make a step forward to plug in several\ndocument/sentence representation methods into the proposed framework to further\nenhance the summarization performance. A series of empirical evaluations\ndemonstrate the effectiveness of our proposed methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05403v1",
            "date": "2016-01-20 20:37:47+00:00",
            "title": "Semantic Word Clusters Using Signed Normalized Graph Cuts",
            "authors": [
                "Jo\u00e3o Sedoc",
                "Jean Gallier",
                "Lyle Ungar",
                "Dean Foster"
            ],
            "abstract": "Vector space representations of words capture many aspects of word\nsimilarity, but such methods tend to make vector spaces in which antonyms (as\nwell as synonyms) are close to each other. We present a new signed spectral\nnormalized graph cut algorithm, signed clustering, that overlays existing\nthesauri upon distributionally derived vector representations of words, so that\nantonym relationships between word pairs are represented by negative weights.\nOur signed clustering algorithm produces clusters of words which simultaneously\ncapture distributional and synonym relations. We evaluate these clusters\nagainst the SimLex-999 dataset (Hill et al.,2014) of human judgments of word\npair similarities, and also show the benefit of using our clusters to predict\nthe sentiment of a given text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.05991v3",
            "date": "2016-01-22 13:22:10+00:00",
            "title": "Speech vocoding for laboratory phonology",
            "authors": [
                "Milos Cernak",
                "Stefan Benus",
                "Alexandros Lazaridis"
            ],
            "abstract": "Using phonological speech vocoding, we propose a platform for exploring\nrelations between phonology and speech processing, and in broader terms, for\nexploring relations between the abstract and physical structures of a speech\nsignal. Our goal is to make a step towards bridging phonology and speech\nprocessing and to contribute to the program of Laboratory Phonology. We show\nthree application examples for laboratory phonology: compositional phonological\nspeech modelling, a comparison of phonological systems and an experimental\nphonological parametric text-to-speech (TTS) system. The featural\nrepresentations of the following three phonological systems are considered in\nthis work: (i) Government Phonology (GP), (ii) the Sound Pattern of English\n(SPE), and (iii) the extended SPE (eSPE). Comparing GP- and eSPE-based vocoded\nspeech, we conclude that the latter achieves slightly better results than the\nformer. However, GP - the most compact phonological speech representation -\nperforms comparably to the systems with a higher number of phonological\nfeatures. The parametric TTS based on phonological speech representation, and\ntrained from an unlabelled audiobook in an unsupervised manner, achieves\nintelligibility of 85% of the state-of-the-art parametric speech synthesis. We\nenvision that the presented approach paves the way for researchers in both\nfields to form meaningful hypotheses that are explicitly testable using the\nconcepts developed and exemplified in this paper. On the one hand, laboratory\nphonologists might test the applied concepts of their theoretical models, and\non the other hand, the speech processing community may utilize the concepts\ndeveloped for the theoretical phonological models for improvements of the\ncurrent state-of-the-art applications.",
            "journal ref": "Computer Speech & Language, Volume 42, March 2017, Pages 100-121",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06303v4",
            "date": "2016-01-23 19:48:54+00:00",
            "title": "Undecidability of the Lambek calculus with a relevant modality",
            "authors": [
                "Max Kanovich",
                "Stepan Kuznetsov",
                "Andre Scedrov"
            ],
            "abstract": "Morrill and Valentin in the paper \"Computational coverage of TLG:\nNonlinearity\" considered an extension of the Lambek calculus enriched by a\nso-called \"exponential\" modality. This modality behaves in the \"relevant\"\nstyle, that is, it allows contraction and permutation, but not weakening.\nMorrill and Valentin stated an open problem whether this system is decidable.\nHere we show its undecidability. Our result remains valid if we consider the\nfragment where all division operations have one direction. We also show that\nthe derivability problem in a restricted case, where the modality can be\napplied only to variables (primitive types), is decidable and belongs to the NP\nclass.",
            "journal ref": "Proc. Formal Grammar 2015/2016, LNCS vol. 9804, Springer, 2016,\n  pp. 240-256",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1601.06733v7",
            "date": "2016-01-25 19:25:48+00:00",
            "title": "Long Short-Term Memory-Networks for Machine Reading",
            "authors": [
                "Jianpeng Cheng",
                "Li Dong",
                "Mirella Lapata"
            ],
            "abstract": "In this paper we address the question of how to render sequence-level\nnetworks better at handling structured input. We propose a machine reading\nsimulator which processes text incrementally from left to right and performs\nshallow reasoning with memory and attention. The reader extends the Long\nShort-Term Memory architecture with a memory network in place of a single\nmemory cell. This enables adaptive memory usage during recurrence with neural\nattention, offering a way to weakly induce relations among tokens. The system\nis initially designed to process a single sequence but we also demonstrate how\nto integrate it with an encoder-decoder architecture. Experiments on language\nmodeling, sentiment analysis, and natural language inference show that our\nmodel matches or outperforms the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06738v1",
            "date": "2016-01-25 19:38:37+00:00",
            "title": "A Label Semantics Approach to Linguistic Hedges",
            "authors": [
                "Martha Lewis",
                "Jonathan Lawry"
            ],
            "abstract": "We introduce a model for the linguistic hedges `very' and `quite' within the\nlabel semantics framework, and combined with the prototype and conceptual\nspaces theories of concepts. The proposed model emerges naturally from the\nrepresentational framework we use and as such, has a clear semantic grounding.\nWe give generalisations of these hedge models and show that they can be\ncomposed with themselves and with other functions, going on to examine their\nbehaviour in the limit of composition.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.07124v1",
            "date": "2016-01-26 18:19:00+00:00",
            "title": "LIA-RAG: a system based on graphs and divergence of probabilities applied to Speech-To-Text Summarization",
            "authors": [
                "Elvys Linhares Pontes",
                "Juan-Manuel Torres-Moreno",
                "Andr\u00e9a Carneiro Linhares"
            ],
            "abstract": "This paper aims to introduces a new algorithm for automatic speech-to-text\nsummarization based on statistical divergences of probabilities and graphs. The\ninput is a text from speech conversations with noise, and the output a compact\ntext summary. Our results, on the pilot task CCCS Multiling 2015 French corpus\nare very encouraging",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.07435v2",
            "date": "2016-01-27 16:37:08+00:00",
            "title": "Co-Occurrence Patterns in the Voynich Manuscript",
            "authors": [
                "Torsten Timm"
            ],
            "abstract": "The Voynich Manuscript is a medieval book written in an unknown script. This\npaper studies the distribution of similarly spelled words in the Voynich\nManuscript. It shows that the distribution of words within the manuscript is\nnot compatible with natural languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.08188v1",
            "date": "2016-01-29 16:48:07+00:00",
            "title": "Lipreading with Long Short-Term Memory",
            "authors": [
                "Michael Wand",
                "Jan Koutn\u00edk",
                "J\u00fcrgen Schmidhuber"
            ],
            "abstract": "Lipreading, i.e. speech recognition from visual-only recordings of a\nspeaker's face, can be achieved with a processing pipeline based solely on\nneural networks, yielding significantly better accuracy than conventional\nmethods. Feed-forward and recurrent neural network layers (namely Long\nShort-Term Memory; LSTM) are stacked to form a single structure which is\ntrained by back-propagating error gradients through all the layers. The\nperformance of such a stacked network was experimentally evaluated and compared\nto a standard Support Vector Machine classifier using conventional computer\nvision features (Eigenlips and Histograms of Oriented Gradients). The\nevaluation was performed on data from 19 speakers of the publicly available\nGRID corpus. With 51 different words to classify, we report a best word\naccuracy on held-out evaluation speakers of 79.6% using the end-to-end neural\nnetwork-based solution (11.6% improvement over the best feature-based solution\nevaluated).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1602.00104v1",
            "date": "2016-01-30 10:53:05+00:00",
            "title": "Extracting Keyword for Disambiguating Name Based on the Overlap Principle",
            "authors": [
                "Mahyuddin K. M. Nasution"
            ],
            "abstract": "Name disambiguation has become one of the main themes in the Semantic Web\nagenda. The semantic web is an extension of the current Web in which\ninformation is not only given well-defined meaning, but also has many purposes\nthat contain the ambiguous naturally or a lot of thing came with the overlap,\nmainly deals with the persons name. Therefore, we develop an approach to\nextract keywords from web snippet with utilizing the overlap principle, a\nconcept to understand things with ambiguous, whereby features of person are\ngenerated for dealing with the variety of web, the web is steadily gaining\nground in the semantic research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1602.00293v1",
            "date": "2016-01-31 18:12:46+00:00",
            "title": "WASSUP? LOL : Characterizing Out-of-Vocabulary Words in Twitter",
            "authors": [
                "Suman Kalyan Maity",
                "Chaitanya Sarda",
                "Anshit Chaudhary",
                "Abhijeet Patil",
                "Shraman Kumar",
                "Akash Mondal",
                "Animesh Mukherjee"
            ],
            "abstract": "Language in social media is mostly driven by new words and spellings that are\nconstantly entering the lexicon thereby polluting it and resulting in high\ndeviation from the formal written version. The primary entities of such\nlanguage are the out-of-vocabulary (OOV) words. In this paper, we study various\nsociolinguistic properties of the OOV words and propose a classification model\nto categorize them into at least six categories. We achieve 81.26% accuracy\nwith high precision and recall. We observe that the content features are the\nmost discriminative ones followed by lexical and context features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.00426v1",
            "date": "2016-02-01 08:37:56+00:00",
            "title": "An Iterative Deep Learning Framework for Unsupervised Discovery of Speech Features and Linguistic Units with Applications on Spoken Term Detection",
            "authors": [
                "Cheng-Tao Chung",
                "Cheng-Yu Tsai",
                "Hsiang-Hung Lu",
                "Chia-Hsiang Liu",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "In this work we aim to discover high quality speech features and linguistic\nunits directly from unlabeled speech data in a zero resource scenario. The\nresults are evaluated using the metrics and corpora proposed in the Zero\nResource Speech Challenge organized at Interspeech 2015. A Multi-layered\nAcoustic Tokenizer (MAT) was proposed for automatic discovery of multiple sets\nof acoustic tokens from the given corpus. Each acoustic token set is specified\nby a set of hyperparameters that describe the model configuration. These sets\nof acoustic tokens carry different characteristics fof the given corpus and the\nlanguage behind, thus can be mutually reinforced. The multiple sets of token\nlabels are then used as the targets of a Multi-target Deep Neural Network\n(MDNN) trained on low-level acoustic features. Bottleneck features extracted\nfrom the MDNN are then used as the feedback input to the MAT and the MDNN\nitself in the next iteration. We call this iterative deep learning framework\nthe Multi-layered Acoustic Tokenizing Deep Neural Network (MAT-DNN), which\ngenerates both high quality speech features for the Track 1 of the Challenge\nand acoustic tokens for the Track 2 of the Challenge. In addition, we performed\nextra experiments on the same corpora on the application of query-by-example\nspoken term detection. The experimental results showed the iterative deep\nlearning framework of MAT-DNN improved the detection performance due to better\nunderlying speech features and acoustic tokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.00515v2",
            "date": "2016-02-01 13:27:34+00:00",
            "title": "Marvin: Semantic annotation using multiple knowledge sources",
            "authors": [
                "Nikola Milosevic"
            ],
            "abstract": "People are producing more written material then anytime in the history. The\nincrease is so high that professionals from the various fields are no more able\nto cope with this amount of publications. Text mining tools can offer tools to\nhelp them and one of the tools that can aid information retrieval and\ninformation extraction is semantic text annotation. In this report we present\nMarvin, a text annotator written in Java, which can be used as a command line\ntool and as a Java library. Marvin is able to annotate text using multiple\nsources, including WordNet, MetaMap, DBPedia and thesauri represented as SKOS.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1602.01428v1",
            "date": "2016-02-03 19:44:37+00:00",
            "title": "\"Draw My Topics\": Find Desired Topics fast from large scale of Corpus",
            "authors": [
                "Jason Dou",
                "Ni Sun",
                "Xiaojun Zou"
            ],
            "abstract": "We develop the \"Draw My Topics\" toolkit, which provides a fast way to\nincorporate social scientists' interest into standard topic modelling. Instead\nof using raw corpus with primitive processing as input, an algorithm based on\nVector Space Model and Conditional Entropy are used to connect social\nscientists' willingness and unsupervised topic models' output. Space for users'\nadjustment on specific corpus of their interest is also accommodated. We\ndemonstrate the toolkit's use on the Diachronic People's Daily Corpus in\nChinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.01576v1",
            "date": "2016-02-04 07:53:11+00:00",
            "title": "A Factorized Recurrent Neural Network based architecture for medium to large vocabulary Language Modelling",
            "authors": [
                "Anantharaman Palacode Narayana Iyer"
            ],
            "abstract": "Statistical language models are central to many applications that use\nsemantics. Recurrent Neural Networks (RNN) are known to produce state of the\nart results for language modelling, outperforming their traditional n-gram\ncounterparts in many cases. To generate a probability distribution across a\nvocabulary, these models require a softmax output layer that linearly increases\nin size with the size of the vocabulary. Large vocabularies need a\ncommensurately large softmax layer and training them on typical laptops/PCs\nrequires significant time and machine resources. In this paper we present a new\ntechnique for implementing RNN based large vocabulary language models that\nsubstantially speeds up computation while optimally using the limited memory\nresources. Our technique, while building on the notion of factorizing the\noutput layer by having multiple output layers, improves on the earlier work by\nsubstantially optimizing on the individual output layer size and also\neliminating the need for a multistep prediction process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02047v1",
            "date": "2016-02-05 14:54:57+00:00",
            "title": "Utiliza\u00e7\u00e3o de Grafos e Matriz de Similaridade na Sumariza\u00e7\u00e3o Autom\u00e1tica de Documentos Baseada em Extra\u00e7\u00e3o de Frases",
            "authors": [
                "Elvys Linhares Pontes"
            ],
            "abstract": "The internet increased the amount of information available. However, the\nreading and understanding of this information are costly tasks. In this\nscenario, the Natural Language Processing (NLP) applications enable very\nimportant solutions, highlighting the Automatic Text Summarization (ATS), which\nproduce a summary from one or more source texts. Automatically summarizing one\nor more texts, however, is a complex task because of the difficulties inherent\nto the analysis and generation of this summary. This master's thesis describes\nthe main techniques and methodologies (NLP and heuristics) to generate\nsummaries. We have also addressed and proposed some heuristics based on graphs\nand similarity matrix to measure the relevance of judgments and to generate\nsummaries by extracting sentences. We used the multiple languages (English,\nFrench and Spanish), CSTNews (Brazilian Portuguese), RPM (French) and DECODA\n(French) corpus to evaluate the developped systems. The results obtained were\nquite interesting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02089v1",
            "date": "2016-02-05 16:40:44+00:00",
            "title": "Harmonic Grammar in a DisCo Model of Meaning",
            "authors": [
                "Martha Lewis",
                "Bob Coecke"
            ],
            "abstract": "The model of cognition developed in (Smolensky and Legendre, 2006) seeks to\nunify two levels of description of the cognitive process: the connectionist and\nthe symbolic. The theory developed brings together these two levels into the\nIntegrated Connectionist/Symbolic Cognitive architecture (ICS). Clark and\nPulman (2007) draw a parallel with semantics where meaning may be modelled on\nboth distributional and symbolic levels, developed by Coecke et al, 2010 into\nthe Distributional Compositional (DisCo) model of meaning. In the current work,\nwe revisit Smolensky and Legendre (S&L)'s model. We describe the DisCo\nframework, summarise the key ideas in S&L's architecture, and describe how\ntheir description of harmony as a graded measure of grammaticality may be\napplied in the DisCo model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1602.02133v1",
            "date": "2016-02-05 19:42:24+00:00",
            "title": "Mining Software Quality from Software Reviews: Research Trends and Open Issues",
            "authors": [
                "Issa Atoum",
                "Ahmed Otoom"
            ],
            "abstract": "Software review text fragments have considerably valuable information about\nusers experience. It includes a huge set of properties including the software\nquality. Opinion mining or sentiment analysis is concerned with analyzing\ntextual user judgments. The application of sentiment analysis on software\nreviews can find a quantitative value that represents software quality.\nAlthough many software quality methods are proposed they are considered\ndifficult to customize and many of them are limited. This article investigates\nthe application of opinion mining as an approach to extract software quality\nproperties. We found that the major issues of software reviews mining using\nsentiment analysis are due to software lifecycle and the diverse users and\nteams.",
            "journal ref": "International Journal of Computer Trends and Technology,Vol. 31,\n  No. 2, Jan 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02499v2",
            "date": "2016-02-08 09:23:19+00:00",
            "title": "The \"Sprekend Nederland\" project and its application to accent location",
            "authors": [
                "David A. van Leeuwen",
                "Rosemary Orr"
            ],
            "abstract": "This paper describes the data collection effort that is part of the project\nSprekend Nederland (The Netherlands Talking), and discusses its potential use\nin Automatic Accent Location. We define Automatic Accent Location as the task\nto describe the accent of a speaker in terms of the location of the speaker and\nits history. We discuss possible ways of describing accent location, the\nconsequence these have for the task of automatic accent location, and potential\nevaluation metrics.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1602.03483v1",
            "date": "2016-02-10 18:49:58+00:00",
            "title": "Learning Distributed Representations of Sentences from Unlabelled Data",
            "authors": [
                "Felix Hill",
                "Kyunghyun Cho",
                "Anna Korhonen"
            ],
            "abstract": "Unsupervised methods for learning distributed representations of words are\nubiquitous in today's NLP research, but far less is known about the best ways\nto learn distributed phrase or sentence representations from unlabelled data.\nThis paper is a systematic comparison of models that learn such\nrepresentations. We find that the optimal approach depends critically on the\nintended application. Deeper, more complex models are preferable for\nrepresentations to be used in supervised systems, but shallow log-linear models\nwork best for building representation spaces that can be decoded with simple\nspatial distance metrics. We also propose two new unsupervised\nrepresentation-learning objectives designed to optimise the trade-off between\ntraining time, domain portability and performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03551v1",
            "date": "2016-02-10 22:02:29+00:00",
            "title": "Knowledge Transfer with Medical Language Embeddings",
            "authors": [
                "Stephanie L. Hyland",
                "Theofanis Karaletsos",
                "Gunnar R\u00e4tsch"
            ],
            "abstract": "Identifying relationships between concepts is a key aspect of scientific\nknowledge synthesis. Finding these links often requires a researcher to\nlaboriously search through scien- tific papers and databases, as the size of\nthese resources grows ever larger. In this paper we describe how distributional\nsemantics can be used to unify structured knowledge graphs with unstructured\ntext to predict new relationships between medical concepts, using a\nprobabilistic generative model. Our approach is also designed to ameliorate\ndata sparsity and scarcity issues in the medical domain, which make language\nmodelling more challenging. Specifically, we integrate the medical relational\ndatabase (SemMedDB) with text from electronic health records (EHRs) to perform\nknowledge graph completion. We further demonstrate the ability of our model to\npredict relationships between tokens not appearing in the relational database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03606v1",
            "date": "2016-02-11 02:39:21+00:00",
            "title": "Variations of the Similarity Function of TextRank for Automated Summarization",
            "authors": [
                "Federico Barrios",
                "Federico L\u00f3pez",
                "Luis Argerich",
                "Rosa Wachenchauzer"
            ],
            "abstract": "This article presents new alternatives to the similarity function for the\nTextRank algorithm for automatic summarization of texts. We describe the\ngeneralities of the algorithm and the different functions we propose. Some of\nthese variants achieve a significative improvement using the same metrics and\ndataset as the original publication.",
            "journal ref": "44 JAIIO - ASAI 2015 - ISSN: 2451-7585, pages 65-72",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03609v1",
            "date": "2016-02-11 03:06:33+00:00",
            "title": "Attentive Pooling Networks",
            "authors": [
                "Cicero dos Santos",
                "Ming Tan",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "In this work, we propose Attentive Pooling (AP), a two-way attention\nmechanism for discriminative model training. In the context of pair-wise\nranking or classification with neural networks, AP enables the pooling layer to\nbe aware of the current input pair, in a way that information from the two\ninput items can directly influence the computation of each other's\nrepresentations. Along with such representations of the paired inputs, AP\njointly learns a similarity measure over projected segments (e.g. trigrams) of\nthe pair, and subsequently, derives the corresponding attention vector for each\ninput to guide the pooling. Our two-way attention mechanism is a general\nframework independent of the underlying representation learning, and it has\nbeen applied to both convolutional neural networks (CNNs) and recurrent neural\nnetworks (RNNs) in our studies. The empirical results, from three very\ndifferent benchmark tasks of question answering/answer selection, demonstrate\nthat our proposed models outperform a variety of strong baselines and achieve\nstate-of-the-art performance in all the benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.03661v1",
            "date": "2016-02-11 10:15:03+00:00",
            "title": "On the emergence of syntactic structures: quantifying and modelling duality of patterning",
            "authors": [
                "Vittorio Loreto",
                "Pietro Gravino",
                "Vito D. P. Servedio",
                "Francesca Tria"
            ],
            "abstract": "The complex organization of syntax in hierarchical structures is one of the\ncore design features of human language. Duality of patterning refers for\ninstance to the organization of the meaningful elements in a language at two\ndistinct levels: a combinatorial level where meaningless forms are combined\ninto meaningful forms and a compositional level where meaningful forms are\ncomposed into larger lexical units. The question remains wide open regarding\nhow such a structure could have emerged. Furthermore a clear mathematical\nframework to quantify this phenomenon is still lacking. The aim of this paper\nis that of addressing these two aspects in a self-consistent way. First, we\nintroduce suitable measures to quantify the level of combinatoriality and\ncompositionality in a language, and present a framework to estimate these\nobservables in human natural languages. Second, we show that the theoretical\npredictions of a multi-agents modeling scheme, namely the Blending Game, are in\nsurprisingly good agreement with empirical data. In the Blending Game a\npopulation of individuals plays language games aiming at success in\ncommunication. It is remarkable that the two sides of duality of patterning\nemerge simultaneously as a consequence of a pure cultural dynamics in a\nsimulated environment that contains meaningful relations, provided a simple\nconstraint on message transmission fidelity is also considered.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1602.04101v1",
            "date": "2016-02-12 16:18:00+00:00",
            "title": "An Empirical Study on Academic Commentary and Its Implications on Reading and Writing",
            "authors": [
                "Tai Wang",
                "Xiangen Hu",
                "Keith Shubeck",
                "Zhiqiang Cai",
                "Jie Tang"
            ],
            "abstract": "The relationship between reading and writing (RRW) is one of the major themes\nin learning science. One of its obstacles is that it is difficult to define or\nmeasure the latent background knowledge of the individual. However, in an\nacademic research setting, scholars are required to explicitly list their\nbackground knowledge in the citation sections of their manuscripts. This unique\nopportunity was taken advantage of to observe RRW, especially in the published\nacademic commentary scenario. RRW was visualized under a proposed topic process\nmodel by using a state of the art version of latent Dirichlet allocation (LDA).\nThe empirical study showed that the academic commentary is modulated both by\nits target paper and the author's background knowledge. Although this\nconclusion was obtained in a unique environment, we suggest its implications\ncan also shed light on other similar interesting areas, such as dialog and\nconversation, group discussion, and social media.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1602.04853v1",
            "date": "2016-02-04 14:32:57+00:00",
            "title": "Complex Networks of Words in Fables",
            "authors": [
                "Yurij Holovatch",
                "Vasyl Palchykov"
            ],
            "abstract": "In this chapter we give an overview of the application of complex network\ntheory to quantify some properties of language. Our study is based on two\nfables in Ukrainian, Mykyta the Fox and Abu-Kasym's slippers. It consists of\ntwo parts: the analysis of frequency-rank distributions of words and the\napplication of complex-network theory. The first part shows that the text sizes\nare sufficiently large to observe statistical properties. This supports their\nselection for the analysis of typical properties of the language networks in\nthe second part of the chapter. In describing language as a complex network,\nwhile words are usually associated with nodes, there is more variability in the\nchoice of links and different representations result in different networks.\nHere, we examine a number of such representations of the language network and\nperform a comparative analysis of their characteristics. Our results suggest\nthat, irrespective of link representation, the Ukrainian language network used\nin the selected fables is a strongly correlated, scale-free, small world. We\ndiscuss how such empirical approaches may help form a useful basis for a\ntheoretical description of language evolution and how they may be used in\nanalyses of other textual narratives.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1602.04874v1",
            "date": "2016-02-16 00:45:19+00:00",
            "title": "Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation",
            "authors": [
                "Yushi Yao",
                "Zheng Huang"
            ],
            "abstract": "Recurrent neural network(RNN) has been broadly applied to natural language\nprocessing(NLP) problems. This kind of neural network is designed for modeling\nsequential data and has been testified to be quite efficient in sequential\ntagging tasks. In this paper, we propose to use bi-directional RNN with long\nshort-term memory(LSTM) units for Chinese word segmentation, which is a crucial\npreprocess task for modeling Chinese sentences and articles. Classical methods\nfocus on designing and combining hand-craft features from context, whereas\nbi-directional LSTM network(BLSTM) does not need any prior knowledge or\npre-designing, and it is expert in keeping the contextual information in both\ndirections. Experiment result shows that our approach gets state-of-the-art\nperformance in word segmentation on both traditional Chinese datasets and\nsimplified Chinese datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1602.05292v1",
            "date": "2016-02-17 04:06:28+00:00",
            "title": "Authorship Attribution Using a Neural Network Language Model",
            "authors": [
                "Zhenhao Ge",
                "Yufang Sun",
                "Mark J. T. Smith"
            ],
            "abstract": "In practice, training language models for individual authors is often\nexpensive because of limited data resources. In such cases, Neural Network\nLanguage Models (NNLMs), generally outperform the traditional non-parametric\nN-gram models. Here we investigate the performance of a feed-forward NNLM on an\nauthorship attribution problem, with moderate author set size and relatively\nlimited data. We also consider how the text topics impact performance. Compared\nwith a well-constructed N-gram baseline method with Kneser-Ney smoothing, the\nproposed method achieves nearly 2:5% reduction in perplexity and increases\nauthor classification accuracy by 3:43% on average, given as few as 5 test\nsentences. The performance is very competitive with the state of the art in\nterms of accuracy and demand on test data. The source code, preprocessed\ndatasets, a detailed description of the methodology and results are available\nat https://github.com/zge/authorship-attribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05307v1",
            "date": "2016-02-17 05:26:47+00:00",
            "title": "Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding",
            "authors": [
                "Xiang Ren",
                "Wenqi He",
                "Meng Qu",
                "Clare R. Voss",
                "Heng Ji",
                "Jiawei Han"
            ],
            "abstract": "Current systems of fine-grained entity typing use distant supervision in\nconjunction with existing knowledge bases to assign categories (type labels) to\nentity mentions. However, the type labels so obtained from knowledge bases are\noften noisy (i.e., incorrect for the entity mention's local context). We define\na new task, Label Noise Reduction in Entity Typing (LNR), to be the automatic\nidentification of correct type labels (type-paths) for training examples, given\nthe set of candidate type labels obtained by distant supervision with a given\ntype hierarchy. The unknown type labels for individual entity mentions and the\nsemantic similarity between entity types pose unique challenges for solving the\nLNR task. We propose a general framework, called PLE, to jointly embed entity\nmentions, text features and entity types into the same low-dimensional space\nwhere, in that space, objects whose types are semantically close have similar\nrepresentations. Then we estimate the type-path for each training example in a\ntop-down manner using the learned embeddings. We formulate a global objective\nfor learning the embeddings from text corpora and knowledge bases, which adopts\na novel margin-based loss that is robust to noisy labels and faithfully models\ntype correlation derived from knowledge bases. Our experiments on three public\ntyping datasets demonstrate the effectiveness and robustness of PLE, with an\naverage of 25% improvement in accuracy compared to next best method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05753v1",
            "date": "2016-02-18 10:56:46+00:00",
            "title": "Overview of Annotation Creation: Processes & Tools",
            "authors": [
                "Mark A. Finlayson",
                "Toma\u017e Erjavec"
            ],
            "abstract": "Creating linguistic annotations requires more than just a reliable annotation\nscheme. Annotation can be a complex endeavour potentially involving many\npeople, stages, and tools. This chapter outlines the process of creating\nend-to-end linguistic annotations, identifying specific tasks that researchers\noften perform. Because tool support is so central to achieving high quality,\nreusable annotations with low cost, the focus is on identifying capabilities\nthat are necessary or useful for annotation tools, as well as common problems\nthese tools present that reduce their utility. Although examples of specific\ntools are provided in many cases, this chapter concentrates more on abstract\ncapabilities and problems because new tools appear continuously, while old\ntools disappear into disuse or disrepair. The two core capabilities tools must\nhave are support for the chosen annotation scheme and the ability to work on\nthe language under study. Additional capabilities are organized into three\ncategories: those that are widely provided; those that often useful but found\nin only a few tools; and those that have as yet little or no available tool\nsupport.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.05765v2",
            "date": "2016-02-18 11:37:50+00:00",
            "title": "Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoning",
            "authors": [
                "Shoaib Jameel",
                "Steven Schockaert"
            ],
            "abstract": "Conceptual spaces are geometric representations of conceptual knowledge, in\nwhich entities correspond to points, natural properties correspond to convex\nregions, and the dimensions of the space correspond to salient features. While\nconceptual spaces enable elegant models of various cognitive phenomena, the\nlack of automated methods for constructing such representations have so far\nlimited their application in artificial intelligence. To address this issue, we\npropose a method which learns a vector-space embedding of entities from\nWikipedia and constrains this embedding such that entities of the same semantic\ntype are located in some lower-dimensional subspace. We experimentally\ndemonstrate the usefulness of these subspaces as (approximate) conceptual space\nrepresentations by showing, among others, that important features can be\nmodelled as directions and that natural properties tend to correspond to convex\nregions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1602.05875v3",
            "date": "2016-02-18 16:55:30+00:00",
            "title": "Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data",
            "authors": [
                "Gil Keren",
                "Bj\u00f6rn Schuller"
            ],
            "abstract": "Traditional convolutional layers extract features from patches of data by\napplying a non-linearity on an affine function of the input. We propose a model\nthat enhances this feature extraction process for the case of sequential data,\nby feeding patches of the data into a recurrent neural network and using the\noutputs or hidden states of the recurrent units to compute the extracted\nfeatures. By doing so, we exploit the fact that a window containing a few\nframes of the sequential data is a sequence itself and this additional\nstructure might encapsulate valuable information. In addition, we allow for\nmore steps of computation in the feature extraction process, which is\npotentially beneficial as an affine function followed by a non-linearity can\nresult in too simple features. Using our convolutional recurrent layers we\nobtain an improvement in performance in two audio classification tasks,\ncompared to traditional convolutional layers. Tensorflow code for the\nconvolutional recurrent layers is publicly available in\nhttps://github.com/cruvadom/Convolutional-RNN.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1602.06359v1",
            "date": "2016-02-20 02:55:11+00:00",
            "title": "Text Matching as Image Recognition",
            "authors": [
                "Liang Pang",
                "Yanyan Lan",
                "Jiafeng Guo",
                "Jun Xu",
                "Shengxian Wan",
                "Xueqi Cheng"
            ],
            "abstract": "Matching two texts is a fundamental problem in many natural language\nprocessing tasks. An effective way is to extract meaningful matching patterns\nfrom words, phrases, and sentences to produce the matching score. Inspired by\nthe success of convolutional neural network in image recognition, where neurons\ncan capture many complicated patterns based on the extracted elementary visual\npatterns such as oriented edges and corners, we propose to model text matching\nas the problem of image recognition. Firstly, a matching matrix whose entries\nrepresent the similarities between words is constructed and viewed as an image.\nThen a convolutional neural network is utilized to capture rich matching\npatterns in a layer-by-layer way. We show that by resembling the compositional\nhierarchies of patterns in image recognition, our model can successfully\nidentify salient signals such as n-gram and n-term matchings. Experimental\nresults demonstrate its superiority against the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.06979v1",
            "date": "2016-02-22 21:47:43+00:00",
            "title": "Empath: Understanding Topic Signals in Large-Scale Text",
            "authors": [
                "Ethan Fast",
                "Binbin Chen",
                "Michael Bernstein"
            ],
            "abstract": "Human language is colored by a broad range of topics, but existing text\nanalysis tools only focus on a small number of them. We present Empath, a tool\nthat can generate and validate new lexical categories on demand from a small\nset of seed terms (like \"bleed\" and \"punch\" to generate the category violence).\nEmpath draws connotations between words and phrases by deep learning a neural\nembedding across more than 1.8 billion words of modern fiction. Given a small\nset of seed words that characterize a category, Empath uses its neural\nembedding to discover new related terms, then validates the category with a\ncrowd-powered filter. Empath also analyzes text across 200 built-in,\npre-validated categories we have generated from common topics in our web\ndataset, like neglect, government, and social media. We show that Empath's\ndata-driven, human validated categories are highly correlated (r=0.906) with\nsimilar categories in LIWC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07275v1",
            "date": "2016-02-22 17:55:04+00:00",
            "title": "Temporal Network Analysis of Literary Texts",
            "authors": [
                "Sandra D. Prado",
                "Silvio R. Dahmen",
                "Ana L. C. Bazzan",
                "Padraig Mac Carron",
                "Ralph Kenna"
            ],
            "abstract": "We study temporal networks of characters in literature focusing on \"Alice's\nAdventures in Wonderland\" (1865) by Lewis Carroll and the anonymous \"La Chanson\nde Roland\" (around 1100). The former, one of the most influential pieces of\nnonsense literature ever written, describes the adventures of Alice in a\nfantasy world with logic plays interspersed along the narrative. The latter, a\nsong of heroic deeds, depicts the Battle of Roncevaux in 778 A.D. during\nCharlemagne's campaign on the Iberian Peninsula. We apply methods recently\ndeveloped by Taylor and coworkers \\cite{Taylor+2015} to find time-averaged\neigenvector centralities, Freeman indices and vitalities of characters. We show\nthat temporal networks are more appropriate than static ones for studying\nstories, as they capture features that the time-independent approaches fail to\nyield.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1602.07394v1",
            "date": "2016-02-24 04:33:49+00:00",
            "title": "Improved Accent Classification Combining Phonetic Vowels with Acoustic Features",
            "authors": [
                "Zhenhao Ge"
            ],
            "abstract": "Researches have shown accent classification can be improved by integrating\nsemantic information into pure acoustic approach. In this work, we combine\nphonetic knowledge, such as vowels, with enhanced acoustic features to build an\nimproved accent classification system. The classifier is based on Gaussian\nMixture Model-Universal Background Model (GMM-UBM), with normalized Perceptual\nLinear Predictive (PLP) features. The features are further optimized by\nPrinciple Component Analysis (PCA) and Hetroscedastic Linear Discriminant\nAnalysis (HLDA). Using 7 major types of accented speech from the Foreign\nAccented English (FAE) corpus, the system achieves classification accuracy 54%\nwith input test data as short as 20 seconds, which is competitive to the state\nof the art in this field.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1602.07563v2",
            "date": "2016-02-24 15:34:22+00:00",
            "title": "Multilingual Twitter Sentiment Classification: The Role of Human Annotators",
            "authors": [
                "Igor Mozetic",
                "Miha Grcar",
                "Jasmina Smailovic"
            ],
            "abstract": "What are the limits of automated Twitter sentiment classification? We analyze\na large set of manually labeled tweets in different languages, use them as\ntraining data, and construct automated classification models. It turns out that\nthe quality of classification models depends much more on the quality and size\nof training data than on the type of the model trained. Experimental results\nindicate that there is no statistically significant difference between the\nperformance of the top classification models. We quantify the quality of\ntraining data by applying various annotator agreement measures, and identify\nthe weakest points of different datasets. We show that the model performance\napproaches the inter-annotator agreement when the size of the training set is\nsufficiently large. However, it is crucial to regularly monitor the self- and\ninter-annotator agreements since this improves the training datasets and\nconsequently the model performance. Finally, we show that there is strong\nevidence that humans perceive the sentiment classes (negative, neutral, and\npositive) as ordered.",
            "journal ref": "PLoS ONE 11(5): e0155036, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07618v1",
            "date": "2016-02-22 16:17:54+00:00",
            "title": "From quantum foundations via natural language meaning to a theory of everything",
            "authors": [
                "Bob Coecke"
            ],
            "abstract": "In this paper we argue for a paradigmatic shift from `reductionism' to\n`togetherness'. In particular, we show how interaction between systems in\nquantum theory naturally carries over to modelling how word meanings interact\nin natural language. Since meaning in natural language, depending on the\nsubject domain, encompasses discussions within any scientific discipline, we\nobtain a template for theories such as social interaction, animal behaviour,\nand many others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07776v4",
            "date": "2016-02-25 02:42:58+00:00",
            "title": "Recurrent Neural Network Grammars",
            "authors": [
                "Chris Dyer",
                "Adhiguna Kuncoro",
                "Miguel Ballesteros",
                "Noah A. Smith"
            ],
            "abstract": "We introduce recurrent neural network grammars, probabilistic models of\nsentences with explicit phrase structure. We explain efficient inference\nprocedures that allow application to both parsing and language modeling.\nExperiments show that they provide better parsing in English than any single\npreviously published supervised generative model and better language modeling\nthan state-of-the-art sequential RNNs in English and Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08742v3",
            "date": "2016-02-28 16:59:46+00:00",
            "title": "Optimizing the Learning Order of Chinese Characters Using a Novel Topological Sort Algorithm",
            "authors": [
                "James C. Loach",
                "Jinzhao Wang"
            ],
            "abstract": "We present a novel algorithm for optimizing the order in which Chinese\ncharacters are learned, one that incorporates the benefits of learning them in\norder of usage frequency and in order of their hierarchal structural\nrelationships. We show that our work outperforms previously published orders\nand algorithms. Our algorithm is applicable to any scheduling task where nodes\nhave intrinsic differences in importance and must be visited in topological\norder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.08952v2",
            "date": "2016-02-29 13:31:17+00:00",
            "title": "Representation of linguistic form and function in recurrent neural networks",
            "authors": [
                "\u00c1kos K\u00e1d\u00e1r",
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi"
            ],
            "abstract": "We present novel methods for analyzing the activation patterns of RNNs from a\nlinguistic point of view and explore the types of linguistic structure they\nlearn. As a case study, we use a multi-task gated recurrent network\narchitecture consisting of two parallel pathways with shared word embeddings\ntrained on predicting the representations of the visual scene corresponding to\nan input sentence, and predicting the next word in the same sentence. Based on\nour proposed method to estimate the amount of contribution of individual tokens\nin the input to the final prediction of the networks we show that the image\nprediction pathway: a) is sensitive to the information structure of the\nsentence b) pays selective attention to lexical categories and grammatical\nfunctions that carry semantic information c) learns to treat the same input\ntoken differently depending on its grammatical functions in the sentence. In\ncontrast the language model is comparatively more sensitive to words with a\nsyntactic function. Furthermore, we propose methods to ex- plore the function\nof individual hidden units in RNNs and show that the two pathways of the\narchitecture in our case study contain specialized units tuned to patterns\ninformative for the task, some of which can carry activations to later time\nsteps to encode long-term dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00081v1",
            "date": "2016-09-01 01:34:56+00:00",
            "title": "All Fingers are not Equal: Intensity of References in Scientific Articles",
            "authors": [
                "Tanmoy Chakraborty",
                "Ramasuri Narayanam"
            ],
            "abstract": "Research accomplishment is usually measured by considering all citations with\nequal importance, thus ignoring the wide variety of purposes an article is\nbeing cited for. Here, we posit that measuring the intensity of a reference is\ncrucial not only to perceive better understanding of research endeavor, but\nalso to improve the quality of citation-based applications. To this end, we\ncollect a rich annotated dataset with references labeled by the intensity, and\npropose a novel graph-based semi-supervised model, GraLap to label the\nintensity of references. Experiments with AAN datasets show a significant\nimprovement compared to the baselines to achieve the true labels of the\nreferences (46% better correlation). Finally, we provide four applications to\ndemonstrate how the knowledge of reference intensity leads to design better\nreal-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00425v1",
            "date": "2016-09-01 23:29:57+00:00",
            "title": "Identifying Dogmatism in Social Media: Signals and Models",
            "authors": [
                "Ethan Fast",
                "Eric Horvitz"
            ],
            "abstract": "We explore linguistic and behavioral features of dogmatism in social media\nand construct statistical models that can identify dogmatic comments. Our model\nis based on a corpus of Reddit posts, collected across a diverse set of\nconversational topics and annotated via paid crowdsourcing. We operationalize\nkey aspects of dogmatism described by existing psychology theories (such as\nover-confidence), finding they have predictive power. We also find evidence for\nnew signals of dogmatism, such as the tendency of dogmatic posts to refrain\nfrom signaling cognitive processes. When we use our predictive model to analyze\nmillions of other Reddit posts, we find evidence that suggests dogmatism is a\ndeeper personality trait, present for dogmatic users across many different\ndomains, and that users who engage on dogmatic comments tend to show increases\nin dogmatic posts themselves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00435v1",
            "date": "2016-09-02 00:40:15+00:00",
            "title": "Citation Classification for Behavioral Analysis of a Scientific Field",
            "authors": [
                "David Jurgens",
                "Srijan Kumar",
                "Raine Hoover",
                "Dan McFarland",
                "Dan Jurafsky"
            ],
            "abstract": "Citations are an important indicator of the state of a scientific field,\nreflecting how authors frame their work, and influencing uptake by future\nscholars. However, our understanding of citation behavior has been limited to\nsmall-scale manual citation analysis. We perform the largest behavioral study\nof citations to date, analyzing how citations are both framed and taken up by\nscholars in one entire field: natural language processing. We introduce a new\ndataset of nearly 2,000 citations annotated for function and centrality, and\nuse it to develop a state-of-the-art classifier and label the entire ACL\nReference Corpus. We then study how citations are framed by authors and use\nboth papers and online traces to track how citations are followed by readers.\nWe demonstrate that authors are sensitive to discourse structure and\npublication venue when citing, that online readers follow temporal links to\nprevious and future work rather than methodological links, and that how a paper\ncites related work is predictive of its citation count. Finally, we use changes\nin citation roles to show that the field of NLP is undergoing a significant\nincrease in consensus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00626v2",
            "date": "2016-09-02 14:42:18+00:00",
            "title": "SynsetRank: Degree-adjusted Random Walk for Relation Identification",
            "authors": [
                "Shinichi Nakajima",
                "Sebastian Krause",
                "Dirk Weissenborn",
                "Sven Schmeier",
                "Nico Goernitz",
                "Feiyu Xu"
            ],
            "abstract": "In relation extraction, a key process is to obtain good detectors that find\nrelevant sentences describing the target relation. To minimize the necessity of\nlabeled data for refining detectors, previous work successfully made use of\nBabelNet, a semantic graph structure expressing relationships between synsets,\nas side information or prior knowledge. The goal of this paper is to enhance\nthe use of graph structure in the framework of random walk with a few\nadjustable parameters. Actually, a straightforward application of random walk\ndegrades the performance even after parameter optimization. With the insight\nfrom this unsuccessful trial, we propose SynsetRank, which adjusts the initial\nprobability so that high degree nodes influence the neighbors as strong as low\ndegree nodes. In our experiment on 13 relations in the FB15K-237 dataset,\nSynsetRank significantly outperforms baselines and the plain random walk\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00777v3",
            "date": "2016-09-03 01:02:51+00:00",
            "title": "Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access",
            "authors": [
                "Bhuwan Dhingra",
                "Lihong Li",
                "Xiujun Li",
                "Jianfeng Gao",
                "Yun-Nung Chen",
                "Faisal Ahmed",
                "Li Deng"
            ],
            "abstract": "This paper proposes KB-InfoBot -- a multi-turn dialogue agent which helps\nusers search Knowledge Bases (KBs) without composing complicated queries. Such\ngoal-oriented dialogue agents typically need to interact with an external\ndatabase to access real-world knowledge. Previous systems achieved this by\nissuing a symbolic query to the KB to retrieve entries based on their\nattributes. However, such symbolic operations break the differentiability of\nthe system and prevent end-to-end training of neural dialogue agents. In this\npaper, we address this limitation by replacing symbolic queries with an induced\n\"soft\" posterior distribution over the KB that indicates which entities the\nuser is interested in. Integrating the soft retrieval process with a\nreinforcement learner leads to higher task success rate and reward in both\nsimulations and against real users. We also present a fully neural end-to-end\nagent, trained entirely from user feedback, and discuss its application towards\npersonalized dialogue agents. The source code is available at\nhttps://github.com/MiuLab/KB-InfoBot.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.00799v1",
            "date": "2016-09-03 07:24:08+00:00",
            "title": "Lexical-Morphological Modeling for Legal Text Analysis",
            "authors": [
                "Danilo S. Carvalho",
                "Minh-Tien Nguyen",
                "Tran Xuan Chien",
                "Minh Le Nguyen"
            ],
            "abstract": "In the context of the Competition on Legal Information Extraction/Entailment\n(COLIEE), we propose a method comprising the necessary steps for finding\nrelevant documents to a legal question and deciding on textual entailment\nevidence to provide a correct answer. The proposed method is based on the\ncombination of several lexical and morphological characteristics, to build a\nlanguage model and a set of features for Machine Learning algorithms. We\nprovide a detailed study on the proposed method performance and failure cases,\nindicating that it is competitive with state-of-the-art approaches on Legal\nInformation Retrieval and Question Answering, while not needing extensive\ntraining data nor depending on expert produced knowledge. The proposed method\nachieved significant results in the competition, indicating a substantial level\nof adequacy for the tasks addressed.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1609.01574v1",
            "date": "2016-09-06 14:30:18+00:00",
            "title": "Automatically extracting, ranking and visually summarizing the treatments for a disease",
            "authors": [
                "Prakash Reddy Putta",
                "John J. Dzak III",
                "Siddhartha R. Jonnalagadda"
            ],
            "abstract": "Clinicians are expected to have up-to-date and broad knowledge of disease\ntreatment options for a patient. Online health knowledge resources contain a\nwealth of information. However, because of the time investment needed to\ndisseminate and rank pertinent information, there is a need to summarize the\ninformation in a more concise format. Our aim of the study is to provide\nclinicians with a concise overview of popular treatments for a given disease\nusing information automatically computed from Medline abstracts. We analyzed\nthe treatments of two disorders - Atrial Fibrillation and Congestive Heart\nFailure. We calculated the precision, recall, and f-scores of our two ranking\nmethods to measure the accuracy of the results. For Atrial Fibrillation\ndisorder, maximum f-score for the New Treatments weighing method is 0.611,\nwhich occurs at 60 treatments. For Congestive Heart Failure disorder, maximum\nf-score for the New Treatments weighing method is 0.503, which occurs at 80\ntreatments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01580v1",
            "date": "2016-09-06 14:46:41+00:00",
            "title": "Using Natural Language Processing to Screen Patients with Active Heart Failure: An Exploration for Hospital-wide Surveillance",
            "authors": [
                "Shu Dong",
                "R Kannan Mutharasan",
                "Siddhartha Jonnalagadda"
            ],
            "abstract": "In this paper, we proposed two different approaches, a rule-based approach\nand a machine-learning based approach, to identify active heart failure cases\nautomatically by analyzing electronic health records (EHR). For the rule-based\napproach, we extracted cardiovascular data elements from clinical notes and\nmatched patients to different colors according their heart failure condition by\nusing rules provided by experts in heart failure. It achieved 69.4% accuracy\nand 0.729 F1-Score. For the machine learning approach, with bigram of clinical\nnotes as features, we tried four different models while SVM with linear kernel\nachieved the best performance with 87.5% accuracy and 0.86 F1-Score. Also, from\nthe classification comparison between the four different models, we believe\nthat linear models fit better for this problem. Once we combine the\nmachine-learning and rule-based algorithms, we will enable hospital-wide\nsurveillance of active heart failure through increased accuracy and\ninterpretability of the outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01586v1",
            "date": "2016-09-06 14:54:58+00:00",
            "title": "A Bootstrap Machine Learning Approach to Identify Rare Disease Patients from Electronic Health Records",
            "authors": [
                "Ravi Garg",
                "Shu Dong",
                "Sanjiv Shah",
                "Siddhartha R Jonnalagadda"
            ],
            "abstract": "Rare diseases are very difficult to identify among large number of other\npossible diagnoses. Better availability of patient data and improvement in\nmachine learning algorithms empower us to tackle this problem computationally.\nIn this paper, we target one such rare disease - cardiac amyloidosis. We aim to\nautomate the process of identifying potential cardiac amyloidosis patients with\nthe help of machine learning algorithms and also learn most predictive factors.\nWith the help of experienced cardiologists, we prepared a gold standard with 73\npositive (cardiac amyloidosis) and 197 negative instances. We achieved high\naverage cross-validation F1 score of 0.98 using an ensemble machine learning\nclassifier. Some of the predictive variables were: Age and Diagnosis of cardiac\narrest, chest pain, congestive heart failure, hypertension, prim open angle\nglaucoma, and shoulder arthritis. Further studies are needed to validate the\naccuracy of the system across an entire health system and its generalizability\nfor other diseases.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.01592v1",
            "date": "2016-09-06 15:02:03+00:00",
            "title": "CRTS: A type system for representing clinical recommendations",
            "authors": [
                "Ravi P Garg",
                "Kalpana Raja",
                "Siddhartha R Jonnalagadda"
            ],
            "abstract": "Background: Clinical guidelines and recommendations are the driving wheels of\nthe evidence-based medicine (EBM) paradigm, but these are available primarily\nas unstructured text and are generally highly heterogeneous in nature. This\nsignificantly reduces the dissemination and automatic application of these\nrecommendations at the point of care. A comprehensive structured representation\nof these recommendations is highly beneficial in this regard. Objective: The\nobjective of this paper to present Clinical Recommendation Type System (CRTS),\na common type system that can effectively represent a clinical recommendation\nin a structured form. Methods: CRTS is built by analyzing 125 recommendations\nand 195 research articles corresponding to 6 different diseases available from\nUpToDate, a publicly available clinical knowledge system, and from the National\nGuideline Clearinghouse, a public resource for evidence-based clinical practice\nguidelines. Results: We show that CRTS not only covers the recommendations but\nalso is flexible to be extended to represent information from primary\nliterature. We also describe how our developed type system can be applied for\nclinical decision support, medical knowledge summarization, and citation\nretrieval. Conclusion: We showed that our proposed type system is precise and\ncomprehensive in representing a large sample of recommendations available for\nvarious disorders. CRTS can now be used to build interoperable information\nextraction systems that automatically extract clinical recommendations and\nrelated data elements from clinical evidence resources, guidelines, systematic\nreviews and primary publications.\n  Keywords: guidelines and recommendations, type system, clinical decision\nsupport, evidence-based medicine, information storage and retrieval",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01594v1",
            "date": "2016-09-06 15:05:25+00:00",
            "title": "An Information Extraction Approach to Prescreen Heart Failure Patients for Clinical Trials",
            "authors": [
                "Abhishek Kalyan Adupa",
                "Ravi Prakash Garg",
                "Jessica Corona-Cox",
                "Sanjiv. J. Shah",
                "Siddhartha R. Jonnalagadda"
            ],
            "abstract": "To reduce the large amount of time spent screening, identifying, and\nrecruiting patients into clinical trials, we need prescreening systems that are\nable to automate the data extraction and decision-making tasks that are\ntypically relegated to clinical research study coordinators. However, a major\nobstacle is the vast amount of patient data available as unstructured free-form\ntext in electronic health records. Here we propose an information\nextraction-based approach that first automatically converts unstructured text\ninto a structured form. The structured data are then compared against a list of\neligibility criteria using a rule-based system to determine which patients\nqualify for enrollment in a heart failure clinical trial. We show that we can\nachieve highly accurate results, with recall and precision values of 0.95 and\n0.86, respectively. Our system allowed us to significantly reduce the time\nneeded for prescreening patients from a few weeks to a few minutes. Our\nopen-source information extraction modules are available for researchers and\ncould be tested and validated in other cardiovascular trials. An approach such\nas the one we demonstrate here may decrease costs and expedite clinical trials,\nand could enhance the reproducibility of trials across institutions and\npopulations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01597v1",
            "date": "2016-09-06 15:10:39+00:00",
            "title": "A Hybrid Citation Retrieval Algorithm for Evidence-based Clinical Knowledge Summarization: Combining Concept Extraction, Vector Similarity and Query Expansion for High Precision",
            "authors": [
                "Kalpana Raja",
                "Andrew J Sauer",
                "Ravi P Garg",
                "Melanie R Klerer",
                "Siddhartha R Jonnalagadda"
            ],
            "abstract": "Novel information retrieval methods to identify citations relevant to a\nclinical topic can overcome the knowledge gap existing between the primary\nliterature (MEDLINE) and online clinical knowledge resources such as UpToDate.\nSearching the MEDLINE database directly or with query expansion methods returns\na large number of citations that are not relevant to the query. The current\nstudy presents a citation retrieval system that retrieves citations for\nevidence-based clinical knowledge summarization. This approach combines query\nexpansion, concept-based screening algorithm, and concept-based vector\nsimilarity. We also propose an information extraction framework for automated\nconcept (Population, Intervention, Comparison, and Disease) extraction. We\nevaluated our proposed system on all topics (as queries) available from\nUpToDate for two diseases, heart failure (HF) and atrial fibrillation (AFib).\nThe system achieved an overall F-score of 41.2% on HF topics and 42.4% on AFib\ntopics on a gold standard of citations available in UpToDate. This is\nsignificantly high when compared to a query-expansion based baseline (F-score\nof 1.3% on HF and 2.2% on AFib) and a system that uses query expansion with\ndisease hyponyms and journal names, concept-based screening, and term-based\nvector similarity system (F-score of 37.5% on HF and 39.5% on AFib). Evaluating\nthe system with top K relevant citations, where K is the number of citations in\nthe gold standard achieved a much higher overall F-score of 69.9% on HF topics\nand 75.1% on AFib topics. In addition, the system retrieved up to 18 new\nrelevant citations per topic when tested on ten HF and six AFib clinical\ntopics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02043v1",
            "date": "2016-09-07 16:05:20+00:00",
            "title": "Feasibility of Post-Editing Speech Transcriptions with a Mismatched Crowd",
            "authors": [
                "Purushotam Radadia",
                "Shirish Karande"
            ],
            "abstract": "Manual correction of speech transcription can involve a selection from\nplausible transcriptions. Recent work has shown the feasibility of employing a\nmismatched crowd for speech transcription. However, it is yet to be established\nwhether a mismatched worker has sufficiently fine-granular speech perception to\nchoose among the phonetically proximate options that are likely to be generated\nfrom the trellis of an ASRU. Hence, we consider five languages, Arabic, German,\nHindi, Russian and Spanish. For each we generate synthetic, phonetically\nproximate, options which emulate post-editing scenarios of varying difficulty.\nWe consistently observe non-trivial crowd ability to choose among fine-granular\noptions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1609.02727v1",
            "date": "2016-09-09 09:58:45+00:00",
            "title": "Detecting Singleton Review Spammers Using Semantic Similarity",
            "authors": [
                "Vlad Sandulescu",
                "Martin Ester"
            ],
            "abstract": "Online reviews have increasingly become a very important resource for\nconsumers when making purchases. Though it is becoming more and more difficult\nfor people to make well-informed buying decisions without being deceived by\nfake reviews. Prior works on the opinion spam problem mostly considered\nclassifying fake reviews using behavioral user patterns. They focused on\nprolific users who write more than a couple of reviews, discarding one-time\nreviewers. The number of singleton reviewers however is expected to be high for\nmany review websites. While behavioral patterns are effective when dealing with\nelite users, for one-time reviewers, the review text needs to be exploited. In\nthis paper we tackle the problem of detecting fake reviews written by the same\nperson using multiple names, posting each review under a different name. We\npropose two methods to detect similar reviews and show the results generally\noutperform the vectorial similarity measures used in prior works. The first\nmethod extends the semantic similarity between words to the reviews level. The\nsecond method is based on topic modeling and exploits the similarity of the\nreviews topic distributions using two models: bag-of-words and\nbag-of-opinion-phrases. The experiments were conducted on reviews from three\ndifferent datasets: Yelp (57K reviews), Trustpilot (9K reviews) and Ott dataset\n(800 reviews).",
            "journal ref": "WWW '15 Companion Proceedings of the 24th International Conference\n  on World Wide Web, 2015, p.971-976",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02745v1",
            "date": "2016-09-09 11:16:15+00:00",
            "title": "A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "Opinion mining from customer reviews has become pervasive in recent years.\nSentences in reviews, however, are usually classified independently, even\nthough they form part of a review's argumentative structure. Intuitively,\nsentences in a review build and elaborate upon each other; knowledge of the\nreview structure and sentential context should thus inform the classification\nof each sentence. We demonstrate this hypothesis for the task of aspect-based\nsentiment analysis by modeling the interdependencies of sentences in a review\nwith a hierarchical bidirectional LSTM. We show that the hierarchical model\noutperforms two non-hierarchical baselines, obtains results competitive with\nthe state-of-the-art, and outperforms the state-of-the-art on five\nmultilingual, multi-domain datasets without any hand-engineered features or\nexternal resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02746v1",
            "date": "2016-09-09 11:16:56+00:00",
            "title": "INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "This paper describes our deep learning-based approach to sentiment analysis\nin Twitter as part of SemEval-2016 Task 4. We use a convolutional neural\nnetwork to determine sentiment and participate in all subtasks, i.e. two-point,\nthree-point, and five-point scale sentiment classification and two-point and\nfive-point scale sentiment quantification. We achieve competitive results for\ntwo-point scale sentiment classification and quantification, ranking fifth and\na close fourth (third and second by alternative metrics) respectively despite\nusing only pre-trained embeddings that contain no sentiment information. We\nachieve good performance on three-point scale sentiment classification, ranking\neighth out of 35, while performing poorly on five-point scale sentiment\nclassification and quantification. An error analysis reveals that this is due\nto low expressiveness of the model to capture negative sentiment as well as an\ninability to take into account ordinal information. We propose improvements in\norder to address these and other issues.",
            "journal ref": "Proceedings of SemEval (2016): 178-182",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02748v2",
            "date": "2016-09-09 11:23:51+00:00",
            "title": "INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "This paper describes our deep learning-based approach to multilingual\naspect-based sentiment analysis as part of SemEval 2016 Task 5. We use a\nconvolutional neural network (CNN) for both aspect extraction and aspect-based\nsentiment analysis. We cast aspect extraction as a multi-label classification\nproblem, outputting probabilities over aspects parameterized by a threshold. To\ndetermine the sentiment towards an aspect, we concatenate an aspect vector with\nevery word embedding and apply a convolution over it. Our constrained system\n(unconstrained for English) achieves competitive results across all languages\nand domains, placing first or second in 5 and 7 out of 11 language-domain pairs\nfor aspect category detection (slot 1) and sentiment polarity (slot 3)\nrespectively, thereby demonstrating the viability of a deep learning-based\napproach for multilingual aspect-based sentiment analysis.",
            "journal ref": "Proceedings of SemEval (2016): 330-336",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03148v1",
            "date": "2016-09-11 10:20:48+00:00",
            "title": "Divide and...conquer? On the limits of algorithmic approaches to syntactic semantic structure",
            "authors": [
                "Diego Gabriel Krivochen"
            ],
            "abstract": "In computer science, divide and conquer (D&C) is an algorithm design paradigm\nbased on multi-branched recursion. A D&C algorithm works by recursively and\nmonotonically breaking down a problem into sub problems of the same (or a\nrelated) type, until these become simple enough to be solved directly. The\nsolutions to the sub problems are then combined to give a solution to the\noriginal problem. The present work identifies D&C algorithms assumed within\ncontemporary syntactic theory, and discusses the limits of their applicability\nin the realms of the syntax semantics and syntax morphophonology interfaces. We\nwill propose that D&C algorithms, while valid for some processes, fall short on\nflexibility given a mixed approach to the structure of linguistic phrase\nmarkers. Arguments in favour of a computationally mixed approach to linguistic\nstructure will be presented as an alternative that offers advantages to uniform\nD&C approaches.",
            "journal ref": "Czech and Slovak Linguistic Review 1(2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03286v1",
            "date": "2016-09-12 07:29:59+00:00",
            "title": "Knowledge as a Teacher: Knowledge-Guided Structural Attention Networks",
            "authors": [
                "Yun-Nung Chen",
                "Dilek Hakkani-Tur",
                "Gokhan Tur",
                "Asli Celikyilmaz",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "Natural language understanding (NLU) is a core component of a spoken dialogue\nsystem. Recently recurrent neural networks (RNN) obtained strong results on NLU\ndue to their superior ability of preserving sequential information over time.\nTraditionally, the NLU module tags semantic slots for utterances considering\ntheir flat structures, as the underlying RNN structure is a linear chain.\nHowever, natural language exhibits linguistic properties that provide rich,\nstructured information for better understanding. This paper introduces a novel\nmodel, knowledge-guided structural attention networks (K-SAN), a generalization\nof RNN to additionally incorporate non-flat network topologies guided by prior\nknowledge. There are two characteristics: 1) important substructures can be\ncaptured from small training data, allowing the model to generalize to\npreviously unseen test data; 2) the model automatically figures out the salient\nsubstructures that are essential to predict the semantic tags of the given\nsentences, so that the understanding performance can be improved. The\nexperiments on the benchmark Air Travel Information System (ATIS) data show\nthat the proposed K-SAN architecture can effectively extract salient knowledge\nfrom substructures with an attention mechanism, and outperform the performance\nof the state-of-the-art neural network based frameworks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1609.03357v1",
            "date": "2016-09-12 11:58:59+00:00",
            "title": "Modelling Creativity: Identifying Key Components through a Corpus-Based Approach",
            "authors": [
                "Anna Jordanous",
                "Bill Keller"
            ],
            "abstract": "Creativity is a complex, multi-faceted concept encompassing a variety of\nrelated aspects, abilities, properties and behaviours. If we wish to study\ncreativity scientifically, then a tractable and well-articulated model of\ncreativity is required. Such a model would be of great value to researchers\ninvestigating the nature of creativity and in particular, those concerned with\nthe evaluation of creative practice. This paper describes a unique approach to\ndeveloping a suitable model of how creative behaviour emerges that is based on\nthe words people use to describe the concept. Using techniques from the field\nof statistical natural language processing, we identify a collection of\nfourteen key components of creativity through an analysis of a corpus of\nacademic papers on the topic. Words are identified which appear significantly\noften in connection with discussions of the concept. Using a measure of lexical\nsimilarity to help cluster these words, a number of distinct themes emerge,\nwhich collectively contribute to a comprehensive and multi-perspective model of\ncreativity. The components provide an ontology of creativity: a set of building\nblocks which can be used to model creative practice in a variety of domains.\nThe components have been employed in two case studies to evaluate the\ncreativity of computational systems and have proven useful in articulating\nachievements of this work and directions for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03528v2",
            "date": "2016-09-12 18:59:29+00:00",
            "title": "The Microsoft 2016 Conversational Speech Recognition System",
            "authors": [
                "W. Xiong",
                "J. Droppo",
                "X. Huang",
                "F. Seide",
                "M. Seltzer",
                "A. Stolcke",
                "D. Yu",
                "G. Zweig"
            ],
            "abstract": "We describe Microsoft's conversational speech recognition system, in which we\ncombine recent developments in neural-network-based acoustic and language\nmodeling to advance the state of the art on the Switchboard recognition task.\nInspired by machine learning ensemble techniques, the system uses a range of\nconvolutional and recurrent neural networks. I-vector modeling and lattice-free\nMMI training provide significant gains for all acoustic model architectures.\nLanguage model rescoring with multiple forward and backward running RNNLMs, and\nword posterior-based system combination provide a 20% boost. The best single\nsystem uses a ResNet architecture acoustic model with RNNLM rescoring, and\nachieves a word error rate of 6.9% on the NIST 2000 Switchboard task. The\ncombined system has an error rate of 6.2%, representing an improvement over\npreviously reported results on this benchmark task.",
            "journal ref": "Proc. IEEE ICASSP, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03632v1",
            "date": "2016-09-12 23:27:37+00:00",
            "title": "Joint Extraction of Events and Entities within a Document Context",
            "authors": [
                "Bishan Yang",
                "Tom Mitchell"
            ],
            "abstract": "Events and entities are closely related; entities are often actors or\nparticipants in events and events without entities are uncommon. The\ninterpretation of events and entities is highly contextually dependent.\nExisting work in information extraction typically models events separately from\nentities, and performs inference at the sentence level, ignoring the rest of\nthe document. In this paper, we propose a novel approach that models the\ndependencies among variables of events, entities, and their relations, and\nperforms joint inference of these variables across a document. The goal is to\nenable access to document-level contextual information and facilitate\ncontext-aware predictions. We demonstrate that our approach substantially\noutperforms the state-of-the-art methods for event extraction as well as a\nstrong baseline for entity extraction.",
            "journal ref": "Proceedings of NAACL-HLT 2016, pages 289-299",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03663v1",
            "date": "2016-09-13 03:02:32+00:00",
            "title": "An Experimental Study of LSTM Encoder-Decoder Model for Text Simplification",
            "authors": [
                "Tong Wang",
                "Ping Chen",
                "Kevin Amaral",
                "Jipeng Qiang"
            ],
            "abstract": "Text simplification (TS) aims to reduce the lexical and structural complexity\nof a text, while still retaining the semantic meaning. Current automatic TS\ntechniques are limited to either lexical-level applications or manually\ndefining a large amount of rules. Since deep neural networks are powerful\nmodels that have achieved excellent performance over many difficult tasks, in\nthis paper, we propose to use the Long Short-Term Memory (LSTM) Encoder-Decoder\nmodel for sentence level TS, which makes minimal assumptions about word\nsequence. We conduct preliminary experiments to find that the model is able to\nlearn operation rules such as reversing, sorting and replacing from sequence\npairs, which shows that the model may potentially discover and apply rules such\nas modifying sentence structure, substituting words, and removing words for TS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.03976v1",
            "date": "2016-09-13 18:46:03+00:00",
            "title": "Multimodal Attention for Neural Machine Translation",
            "authors": [
                "Ozan Caglayan",
                "Lo\u00efc Barrault",
                "Fethi Bougares"
            ],
            "abstract": "The attention mechanism is an important part of the neural machine\ntranslation (NMT) where it was reported to produce richer source representation\ncompared to fixed-length encoding sequence-to-sequence models. Recently, the\neffectiveness of attention has also been explored in the context of image\ncaptioning. In this work, we assess the feasibility of a multimodal attention\nmechanism that simultaneously focus over an image and its natural language\ndescription for generating a description in another language. We train several\nvariants of our proposed attention mechanism on the Multi30k multilingual image\ncaptioning dataset. We show that a dedicated attention for each modality\nachieves up to 1.6 points in BLEU and METEOR compared to a textual NMT\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04309v3",
            "date": "2016-09-14 15:15:08+00:00",
            "title": "Efficient softmax approximation for GPUs",
            "authors": [
                "Edouard Grave",
                "Armand Joulin",
                "Moustapha Ciss\u00e9",
                "David Grangier",
                "Herv\u00e9 J\u00e9gou"
            ],
            "abstract": "We propose an approximate strategy to efficiently train neural network based\nlanguage models over very large vocabularies. Our approach, called adaptive\nsoftmax, circumvents the linear dependency on the vocabulary size by exploiting\nthe unbalanced word distribution to form clusters that explicitly minimize the\nexpectation of computation time. Our approach further reduces the computational\ntime by exploiting the specificities of modern architectures and matrix-matrix\nvector operations, making it particularly suited for graphical processing\nunits. Our experiments carried out on standard benchmarks, such as EuroParl and\nOne Billion Word, show that our approach brings a large gain in efficiency over\nstandard approximations while achieving an accuracy close to that of the full\nsoftmax. The code of our method is available at\nhttps://github.com/facebookresearch/adaptive-softmax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04417v1",
            "date": "2016-09-14 20:02:42+00:00",
            "title": "An Adaptive Psychoacoustic Model for Automatic Speech Recognition",
            "authors": [
                "Peng Dai",
                "Xue Teng",
                "Frank Rudzicz",
                "Ing Yann Soon"
            ],
            "abstract": "Compared with automatic speech recognition (ASR), the human auditory system\nis more adept at handling noise-adverse situations, including environmental\nnoise and channel distortion. To mimic this adeptness, auditory models have\nbeen widely incorporated in ASR systems to improve their robustness. This paper\nproposes a novel auditory model which incorporates psychoacoustics and\notoacoustic emissions (OAEs) into ASR. In particular, we successfully implement\nthe frequency-dependent property of psychoacoustic models and effectively\nimprove resulting system performance. We also present a novel double-transform\nspectrum-analysis technique, which can qualitatively predict ASR performance\nfor different noise types. Detailed theoretical analysis is provided to show\nthe effectiveness of the proposed algorithm. Experiments are carried out on the\nAURORA2 database and show that the word recognition rate using our proposed\nfeature extraction method is significantly increased over the baseline. Given\nmodels trained with clean speech, our proposed method achieves up to 85.39%\nword recognition accuracy on noisy data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05104v2",
            "date": "2016-09-16 15:20:32+00:00",
            "title": "Intrinsic normalization and extrinsic denormalization of formant data of vowels",
            "authors": [
                "T. V. Ananthapadmanabha",
                "A. G. Ramakrishnan"
            ],
            "abstract": "Using a known speaker-intrinsic normalization procedure, formant data are\nscaled by the reciprocal of the geometric mean of the first three formant\nfrequencies. This reduces the influence of the talker but results in a\ndistorted vowel space. The proposed speaker-extrinsic procedure re-scales the\nnormalized values by the mean formant values of vowels. When tested on the\nformant data of vowels published by Peterson and Barney, the combined approach\nleads to well separated clusters by reducing the spread due to talkers. The\nproposed procedure performs better than two top-ranked normalization procedures\nbased on the accuracy of vowel classification as the objective measure.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.05180v2",
            "date": "2016-09-16 19:12:30+00:00",
            "title": "Grammatical Templates: Improving Text Difficulty Evaluation for Language Learners",
            "authors": [
                "Shuhan Wang",
                "Erik Andersen"
            ],
            "abstract": "Language students are most engaged while reading texts at an appropriate\ndifficulty level. However, existing methods of evaluating text difficulty focus\nmainly on vocabulary and do not prioritize grammatical features, hence they do\nnot work well for language learners with limited knowledge of grammar. In this\npaper, we introduce grammatical templates, the expert-identified units of\ngrammar that students learn from class, as an important feature of text\ndifficulty evaluation. Experimental classification results show that\ngrammatical template features significantly improve text difficulty prediction\naccuracy over baseline readability features by 7.4%. Moreover, we build a\nsimple and human-understandable text difficulty evaluation approach with 87.7%\naccuracy, using only 5 grammatical template features.",
            "journal ref": "The 26th International Conference on Computational Linguistics\n  (COLING), 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05234v1",
            "date": "2016-09-16 20:56:22+00:00",
            "title": "Interactive Spoken Content Retrieval by Deep Reinforcement Learning",
            "authors": [
                "Yen-Chen Wu",
                "Tzu-Hsiang Lin",
                "Yang-De Chen",
                "Hung-Yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "User-machine interaction is important for spoken content retrieval. For text\ncontent retrieval, the user can easily scan through and select on a list of\nretrieved item. This is impossible for spoken content retrieval, because the\nretrieved items are difficult to show on screen. Besides, due to the high\ndegree of uncertainty for speech recognition, the retrieval results can be very\nnoisy. One way to counter such difficulties is through user-machine\ninteraction. The machine can take different actions to interact with the user\nto obtain better retrieval results before showing to the user. The suitable\nactions depend on the retrieval status, for example requesting for extra\ninformation from the user, returning a list of topics for user to select, etc.\nIn our previous work, some hand-crafted states estimated from the present\nretrieval results are used to determine the proper actions. In this paper, we\npropose to use Deep-Q-Learning techniques instead to determine the machine\nactions for interactive spoken content retrieval. Deep-Q-Learning bypasses the\nneed for estimation of the hand-crafted states, and directly determine the best\naction base on the present retrieval status even without any human knowledge.\nIt is shown to achieve significantly better performance compared with the\nprevious hand-crafted states.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.05244v2",
            "date": "2016-09-16 21:33:42+00:00",
            "title": "Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis",
            "authors": [
                "Haohan Wang",
                "Aaksha Meghawat",
                "Louis-Philippe Morency",
                "Eric P. Xing"
            ],
            "abstract": "Multimodal sentiment analysis is drawing an increasing amount of attention\nthese days. It enables mining of opinions in video reviews which are now\navailable aplenty on online platforms. However, multimodal sentiment analysis\nhas only a few high-quality data sets annotated for training machine learning\nalgorithms. These limited resources restrict the generalizability of models,\nwhere, for example, the unique characteristics of a few speakers (e.g., wearing\nglasses) may become a confounding factor for the sentiment classification task.\nIn this paper, we propose a Select-Additive Learning (SAL) procedure that\nimproves the generalizability of trained neural networks for multimodal\nsentiment analysis. In our experiments, we show that our SAL approach improves\nprediction accuracy significantly in all three modalities (verbal, acoustic,\nvisual), as well as in their fusion. Our results show that SAL, even when\ntrained on one dataset, achieves good generalization across two new test\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06127v1",
            "date": "2016-09-20 12:29:15+00:00",
            "title": "A framework for mining process models from emails logs",
            "authors": [
                "Diana Jlailaty",
                "Daniela Grigori",
                "Khalid Belhajjame"
            ],
            "abstract": "Due to its wide use in personal, but most importantly, professional contexts,\nemail represents a valuable source of information that can be harvested for\nunderstanding, reengineering and repurposing undocumented business processes of\ncompanies and institutions. Towards this aim, a few researchers investigated\nthe problem of extracting process oriented information from email logs in order\nto take benefit of the many available process mining techniques and tools. In\nthis paper we go further in this direction, by proposing a new method for\nmining process models from email logs that leverage unsupervised machine\nlearning techniques with little human involvement. Moreover, our method allows\nto semi-automatically label emails with activity names, that can be used for\nactivity recognition in new incoming emails. A use case demonstrates the\nusefulness of the proposed solution using a modest in size, yet real-world,\ndataset containing emails that belong to two different process models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06268v1",
            "date": "2016-09-20 17:54:47+00:00",
            "title": "Semantic Similarity Strategies for Job Title Classification",
            "authors": [
                "Yun Zhu",
                "Faizan Javed",
                "Ozgur Ozturk"
            ],
            "abstract": "Automatic and accurate classification of items enables numerous downstream\napplications in many domains. These applications can range from faceted\nbrowsing of items to product recommendations and big data analytics. In the\nonline recruitment domain, we refer to classifying job ads to pre-defined or\ncustom occupation categories as job title classification. A large-scale job\ntitle classification system can power various downstream applications such as\nsemantic search, job recommendations and labor market analytics. In this paper,\nwe discuss experiments conducted to improve our in-house job title\nclassification system. The classification component of the system is composed\nof a two-stage coarse and fine level classifier cascade that classifies input\ntext such as job title and/or job ads to one of the thousands of job titles in\nour taxonomy. To improve classification accuracy and effectiveness, we\nexperiment with various semantic representation strategies such as average W2V\nvectors and document similarity measures such as Word Movers Distance (WMD).\nOur initial results show an overall improvement in accuracy of Carotene[1].",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1609.06380v1",
            "date": "2016-09-20 22:59:19+00:00",
            "title": "Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention",
            "authors": [
                "Yang Liu",
                "Sujian Li"
            ],
            "abstract": "Recognizing implicit discourse relations is a challenging but important task\nin the field of Natural Language Processing. For such a complex text processing\ntask, different from previous studies, we argue that it is necessary to\nrepeatedly read the arguments and dynamically exploit the efficient features\nuseful for recognizing discourse relations. To mimic the repeated reading\nstrategy, we propose the neural networks with multi-level attention (NNMA),\ncombining the attention mechanism and external memories to gradually fix the\nattention on some specific words helpful to judging the discourse relations.\nExperiments on the PDTB dataset show that our proposed method achieves the\nstate-of-art results. The visualization of the attention weights also\nillustrates the progress that our model observes the arguments on each level\nand progressively locates the important words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06404v1",
            "date": "2016-09-21 02:14:23+00:00",
            "title": "KU-ISPL Language Recognition System for NIST 2015 i-Vector Machine Learning Challenge",
            "authors": [
                "Suwon Shon",
                "Seongkyu Mun",
                "John H. L. Hansen",
                "Hanseok Ko"
            ],
            "abstract": "In language recognition, the task of rejecting/differentiating closely spaced\nversus acoustically far spaced languages remains a major challenge. For\nconfusable closely spaced languages, the system needs longer input test\nduration material to obtain sufficient information to distinguish between\nlanguages. Alternatively, if languages are distinct and not\nacoustically/linguistically similar to others, duration is not a sufficient\nremedy. The solution proposed here is to explore duration distribution analysis\nfor near/far languages based on the Language Recognition i-Vector Machine\nLearning Challenge 2015 (LRiMLC15) database. Using this knowledge, we propose a\nlikelihood ratio based fusion approach that leveraged both score and duration\ninformation. The experimental results show that the use of duration and score\nfusion improves language recognition performance by 5% relative in LRiMLC15\ncost.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.06657v1",
            "date": "2016-09-21 18:12:04+00:00",
            "title": "The Color of the Cat is Gray: 1 Million Full-Sentences Visual Question Answering (FSVQA)",
            "authors": [
                "Andrew Shin",
                "Yoshitaka Ushiku",
                "Tatsuya Harada"
            ],
            "abstract": "Visual Question Answering (VQA) task has showcased a new stage of interaction\nbetween language and vision, two of the most pivotal components of artificial\nintelligence. However, it has mostly focused on generating short and repetitive\nanswers, mostly single words, which fall short of rich linguistic capabilities\nof humans. We introduce Full-Sentence Visual Question Answering (FSVQA)\ndataset, consisting of nearly 1 million pairs of questions and full-sentence\nanswers for images, built by applying a number of rule-based natural language\nprocessing techniques to original VQA dataset and captions in the MS COCO\ndataset. This poses many additional complexities to conventional VQA task, and\nwe provide a baseline for approaching and evaluating the task, on top of which\nwe invite the research community to build further improvements.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1609.06686v1",
            "date": "2016-09-21 19:08:15+00:00",
            "title": "Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "Convolutional neural networks (CNNs) have demonstrated superior capability\nfor extracting information from raw signals in computer vision. Recently,\ncharacter-level and multi-channel CNNs have exhibited excellent performance for\nsentence classification tasks. We apply CNNs to large-scale authorship\nattribution, which aims to determine an unknown text's author among many\ncandidate authors, motivated by their ability to process character-level\nsignals and to differentiate between a large number of classes, while making\nfast predictions in comparison to state-of-the-art approaches. We extensively\nevaluate CNN-based approaches that leverage word and character channels and\ncompare them against state-of-the-art methods for a large range of author\nnumbers, shedding new light on traditional approaches. We show that\ncharacter-level CNNs outperform the state-of-the-art on four out of five\ndatasets in different domains. Additionally, we present the first application\nof authorship attribution to reddit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07028v2",
            "date": "2016-09-22 15:37:45+00:00",
            "title": "Image-embodied Knowledge Representation Learning",
            "authors": [
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Huanbo Luan",
                "Maosong Sun"
            ],
            "abstract": "Entity images could provide significant visual information for knowledge\nrepresentation learning. Most conventional methods learn knowledge\nrepresentations merely from structured triples, ignoring rich visual\ninformation extracted from entity images. In this paper, we propose a novel\nImage-embodied Knowledge Representation Learning model (IKRL), where knowledge\nrepresentations are learned with both triple facts and images. More\nspecifically, we first construct representations for all images of an entity\nwith a neural image encoder. These image representations are then integrated\ninto an aggregated image-based representation via an attention-based method. We\nevaluate our IKRL models on knowledge graph completion and triple\nclassification. Experimental results demonstrate that our models outperform all\nbaselines on both tasks, which indicates the significance of visual information\nfor knowledge representations and the capability of our models in learning\nknowledge representations with images.",
            "journal ref": "IJCAI-2017",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1609.07245v2",
            "date": "2016-09-23 07:03:32+00:00",
            "title": "A New Statistic Feature of the Short-Time Amplitude Spectrum Values for Human's Unvoiced Pronunciation",
            "authors": [
                "Xiaodong Zhuang"
            ],
            "abstract": "In this paper, a new statistic feature of the discrete short-time amplitude\nspectrum is discovered by experiments for the signals of unvoiced\npronunciation. For the random-varying short-time spectrum, this feature reveals\nthe relationship between the amplitude's average and its standard for every\nfrequency component. On the other hand, the association between the amplitude\ndistributions for different frequency components is also studied. A new model\nrepresenting such association is inspired by the normalized histogram of\namplitude. By mathematical analysis, the new statistic feature discovered is\nproved to be necessary evidence which supports the proposed model, and also can\nbe direct evidence for the widely used hypothesis of \"identical distribution of\namplitude for all frequencies\".",
            "journal ref": "WSEAS Transactions on Signal Processing, Volume 12, pp. 265-269,\n  2016",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.07317v2",
            "date": "2016-09-23 11:25:41+00:00",
            "title": "Language as a Latent Variable: Discrete Generative Models for Sentence Compression",
            "authors": [
                "Yishu Miao",
                "Phil Blunsom"
            ],
            "abstract": "In this work we explore deep generative models of text in which the latent\nrepresentation of a document is itself drawn from a discrete language model\ndistribution. We formulate a variational auto-encoder for inference in this\nmodel and apply it to the task of compressing sentences. In this application\nthe generative model first draws a latent summary sentence from a background\nlanguage model, and then subsequently draws the observed sentence conditioned\non this latent summary. In our empirical evaluation we show that generative\nformulations of both abstractive and extractive compression yield\nstate-of-the-art results when trained on a large amount of supervised data.\nFurther, we explore semi-supervised compression scenarios where we show that it\nis possible to achieve performance competitive with previously proposed\nsupervised models while training on a fraction of the supervised data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07498v1",
            "date": "2016-09-23 20:03:14+00:00",
            "title": "Speaker Recognition for Children's Speech",
            "authors": [
                "Saeid Safavi",
                "Maryam Najafian",
                "Abualsoud Hanani",
                "Martin J Russell",
                "Peter Jancovic",
                "Michael J Carey"
            ],
            "abstract": "This paper presents results on Speaker Recognition (SR) for children's\nspeech, using the OGI Kids corpus and GMM-UBM and GMM-SVM SR systems. Regions\nof the spectrum containing important speaker information for children are\nidentified by conducting SR experiments over 21 frequency bands. As for adults,\nthe spectrum can be split into four regions, with the first (containing primary\nvocal tract resonance information) and third (corresponding to high frequency\nspeech sounds) being most useful for SR. However, the frequencies at which\nthese regions occur are from 11% to 38% higher for children. It is also noted\nthat subband SR rates are lower for younger children. Finally results are\npresented of SR experiments to identify a child in a class (30 children,\nsimilar age) and school (288 children, varying ages). Class performance depends\non age, with accuracy varying from 90% for young children to 99% for older\nchildren. The identification rate achieved for a child in a school is 81%.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.07680v1",
            "date": "2016-09-24 23:22:50+00:00",
            "title": "Existence of Hierarchies and Human's Pursuit of Top Hierarchy Lead to Power Law",
            "authors": [
                "Shuiyuan Yu",
                "Junying Liang",
                "Haitao Liu"
            ],
            "abstract": "The power law is ubiquitous in natural and social phenomena, and is\nconsidered as a universal relationship between the frequency and its rank for\ndiverse social systems. However, a general model is still lacking to interpret\nwhy these seemingly unrelated systems share great similarity. Through a\ndetailed analysis of natural language texts and simulation experiments based on\nthe proposed 'Hierarchical Selection Model', we found that the existence of\nhierarchies and human's pursuit of top hierarchy lead to the power law.\nFurther, the power law is a statistical and emergent performance of\nhierarchies, and it is the universality of hierarchies that contributes to the\nubiquity of the power law.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07843v1",
            "date": "2016-09-26 04:06:13+00:00",
            "title": "Pointer Sentinel Mixture Models",
            "authors": [
                "Stephen Merity",
                "Caiming Xiong",
                "James Bradbury",
                "Richard Socher"
            ],
            "abstract": "Recent neural network sequence models with softmax classifiers have achieved\ntheir best language modeling performance only with very large hidden states and\nlarge vocabularies. Even then they struggle to predict rare or unseen words\neven if the context makes the prediction unambiguous. We introduce the pointer\nsentinel mixture architecture for neural sequence models which has the ability\nto either reproduce a word from the recent context or produce a word from a\nstandard softmax classifier. Our pointer sentinel-LSTM model achieves state of\nthe art language modeling performance on the Penn Treebank (70.9 perplexity)\nwhile using far fewer parameters than a standard softmax LSTM. In order to\nevaluate how well language models can exploit longer contexts and deal with\nmore realistic vocabularies and larger corpora we also introduce the freely\navailable WikiText corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.07876v1",
            "date": "2016-09-26 07:34:24+00:00",
            "title": "Lexicon-Free Fingerspelling Recognition from Video: Data, Models, and Signer Adaptation",
            "authors": [
                "Taehwan Kim",
                "Jonathan Keane",
                "Weiran Wang",
                "Hao Tang",
                "Jason Riggle",
                "Gregory Shakhnarovich",
                "Diane Brentari",
                "Karen Livescu"
            ],
            "abstract": "We study the problem of recognizing video sequences of fingerspelled letters\nin American Sign Language (ASL). Fingerspelling comprises a significant but\nrelatively understudied part of ASL. Recognizing fingerspelling is challenging\nfor a number of reasons: It involves quick, small motions that are often highly\ncoarticulated; it exhibits significant variation between signers; and there has\nbeen a dearth of continuous fingerspelling data collected. In this work we\ncollect and annotate a new data set of continuous fingerspelling videos,\ncompare several types of recognizers, and explore the problem of signer\nvariation. Our best-performing models are segmental (semi-Markov) conditional\nrandom fields using deep neural network-based features. In the signer-dependent\nsetting, our recognizers achieve up to about 92% letter accuracy. The\nmulti-signer setting is much more challenging, but with neural network\nadaptation we achieve up to 83% letter accuracies in this setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08210v1",
            "date": "2016-09-26 22:12:50+00:00",
            "title": "Learning to Translate for Multilingual Question Answering",
            "authors": [
                "Ferhan Ture",
                "Elizabeth Boschee"
            ],
            "abstract": "In multilingual question answering, either the question needs to be\ntranslated into the document language, or vice versa. In addition to direction,\nthere are multiple methods to perform the translation, four of which we explore\nin this paper: word-based, 10-best, context-based, and grammar-based. We build\na feature for each combination of translation direction and method, and train a\nmodel that learns optimal feature weights. On a large forum dataset consisting\nof posts in English, Arabic, and Chinese, our novel learn-to-translate approach\nwas more effective than a strong baseline (p<0.05): translating all text into\nEnglish, then training a classifier based only on English (original or\ntranslated) text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08389v2",
            "date": "2016-09-27 12:55:10+00:00",
            "title": "A Hackathon for Classical Tibetan",
            "authors": [
                "Orna Almogi",
                "Lena Dankin",
                "Nachum Dershowitz",
                "Lior Wolf"
            ],
            "abstract": "We describe the course of a hackathon dedicated to the development of\nlinguistic tools for Tibetan Buddhist studies. Over a period of five days, a\ngroup of seventeen scholars, scientists, and students developed and compared\nalgorithms for intertextual alignment and text classification, along with some\nbasic language tools, including a stemmer and word segmenter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08409v1",
            "date": "2016-09-27 13:25:10+00:00",
            "title": "Modelling Radiological Language with Bidirectional Long Short-Term Memory Networks",
            "authors": [
                "Savelie Cornegruta",
                "Robert Bakewell",
                "Samuel Withey",
                "Giovanni Montana"
            ],
            "abstract": "Motivated by the need to automate medical information extraction from\nfree-text radiological reports, we present a bi-directional long short-term\nmemory (BiLSTM) neural network architecture for modelling radiological\nlanguage. The model has been used to address two NLP tasks: medical\nnamed-entity recognition (NER) and negation detection. We investigate whether\nlearning several types of word embeddings improves BiLSTM's performance on\nthose tasks. Using a large dataset of chest x-ray reports, we compare the\nproposed model to a baseline dictionary-based NER system and a negation\ndetection system that leverages the hand-crafted rules of the NegEx algorithm\nand the grammatical relations obtained from the Stanford Dependency Parser.\nCompared to these more traditional rule-based systems, we argue that BiLSTM\noffers a strong alternative for both our tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08433v1",
            "date": "2016-09-27 13:37:13+00:00",
            "title": "Local Training for PLDA in Speaker Verification",
            "authors": [
                "Chenghui Zhao",
                "Lantian Li",
                "Dong Wang",
                "April Pu"
            ],
            "abstract": "PLDA is a popular normalization approach for the i-vector model, and it has\ndelivered state-of-the-art performance in speaker verification. However, PLDA\ntraining requires a large amount of labeled development data, which is highly\nexpensive in most cases. A possible approach to mitigate the problem is various\nunsupervised adaptation methods, which use unlabeled data to adapt the PLDA\nscattering matrices to the target domain.\n  In this paper, we present a new `local training' approach that utilizes\ninaccurate but much cheaper local labels to train the PLDA model. These local\nlabels discriminate speakers within a single conversion only, and so are much\neasier to obtain compared to the normal `global labels'. Our experiments show\nthat the proposed approach can deliver significant performance improvement,\nparticularly with limited globally-labeled data.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.08442v2",
            "date": "2016-09-27 13:48:01+00:00",
            "title": "Collaborative Learning for Language and Speaker Recognition",
            "authors": [
                "Lantian Li",
                "Zhiyuan Tang",
                "Dong Wang",
                "Andrew Abel",
                "Yang Feng",
                "Shiyue Zhang"
            ],
            "abstract": "This paper presents a unified model to perform language and speaker\nrecognition simultaneously and altogether. The model is based on a multi-task\nrecurrent neural network where the output of one task is fed as the input of\nthe other, leading to a collaborative learning framework that can improve both\nlanguage and speaker recognition by borrowing information from each other. Our\nexperiments demonstrated that the multi-task model outperforms the\ntask-specific models on both tasks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1609.08445v1",
            "date": "2016-09-27 13:50:13+00:00",
            "title": "AP16-OL7: A Multilingual Database for Oriental Languages and A Language Recognition Baseline",
            "authors": [
                "Dong Wang",
                "Lantian Li",
                "Difei Tang",
                "Qing Chen"
            ],
            "abstract": "We present the AP16-OL7 database which was released as the training and test\ndata for the oriental language recognition (OLR) challenge on APSIPA 2016.\nBased on the database, a baseline system was constructed on the basis of the\ni-vector model. We report the baseline results evaluated in various metrics\ndefined by the AP16-OLR evaluation plan and demonstrate that AP16-OL7 is a\nreasonable data resource for multilingual research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08492v2",
            "date": "2016-09-27 15:14:04+00:00",
            "title": "WS4A: a Biomedical Question and Answering System based on public Web Services and Ontologies",
            "authors": [
                "Miguel J. Rodrigues",
                "Miguel Fal\u00e9",
                "Andre Lamurias",
                "Francisco M. Couto"
            ],
            "abstract": "This paper describes our system, dubbed WS4A (Web Services for All), that\nparticipated in the fourth edition of the BioASQ challenge (2016). We used WS4A\nto perform the Question and Answering (QA) task 4b, which consisted on the\nretrieval of relevant concepts, documents, snippets, RDF triples, exact answers\nand ideal answers for each given question. The novelty in our approach consists\non the maximum exploitation of existing web services in each step of WS4A, such\nas the annotation of text, and the retrieval of metadata for each annotation.\nThe information retrieved included concept identifiers, ontologies, ancestors,\nand most importantly, PubMed identifiers. The paper describes the WS4A pipeline\nand also presents the precision, recall and f-measure values obtained in task\n4b. Our system achieved two second places in two subtasks on one of the five\nbatches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08779v1",
            "date": "2016-09-28 05:44:10+00:00",
            "title": "Using Natural Language Processing and Qualitative Analysis to Intervene in Gang Violence: A Collaboration Between Social Work Researchers and Data Scientists",
            "authors": [
                "Desmond Upton Patton",
                "Kathleen McKeown",
                "Owen Rambow",
                "Jamie Macbeth"
            ],
            "abstract": "The U.S. has the highest rate of firearm-related deaths when compared to\nother industrialized countries. Violence particularly affects low-income, urban\nneighborhoods in cities like Chicago, which saw a 40% increase in firearm\nviolence from 2014 to 2015 to more than 3,000 shooting victims. While recent\nstudies have found that urban, gang-involved individuals curate a unique and\ncomplex communication style within and between social media platforms,\norganizations focused on reducing gang violence are struggling to keep up with\nthe growing complexity of social media platforms and the sheer volume of data\nthey present. In this paper, describe the Digital Urban Violence Analysis\nApproach (DUVVA), a collaborative qualitative analysis method used in a\ncollaboration between data scientists and social work researchers to develop a\nsuite of systems for decoding the high- stress language of urban, gang-involved\nyouth. Our approach leverages principles of grounded theory when analyzing\napproximately 800 tweets posted by Chicago gang members and participation of\nyouth from Chicago neighborhoods to create a language resource for natural\nlanguage processing (NLP) methods. In uncovering the unique language and\ncommunication style, we developed automated tools with the potential to detect\naggressive language on social media and aid individuals and groups in\nperforming violence prevention and interruption.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1609.09007v1",
            "date": "2016-09-28 16:55:52+00:00",
            "title": "Unsupervised Neural Hidden Markov Models",
            "authors": [
                "Ke Tran",
                "Yonatan Bisk",
                "Ashish Vaswani",
                "Daniel Marcu",
                "Kevin Knight"
            ],
            "abstract": "In this work, we present the first results for neuralizing an Unsupervised\nHidden Markov Model. We evaluate our approach on tag in- duction. Our approach\noutperforms existing generative models and is competitive with the\nstate-of-the-art though with a simpler model easily extended to include\nadditional context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09028v2",
            "date": "2016-09-28 18:24:12+00:00",
            "title": "Stance Classification in Rumours as a Sequential Task Exploiting the Tree Structure of Social Media Conversations",
            "authors": [
                "Arkaitz Zubiaga",
                "Elena Kochkina",
                "Maria Liakata",
                "Rob Procter",
                "Michal Lukasik"
            ],
            "abstract": "Rumour stance classification, the task that determines if each tweet in a\ncollection discussing a rumour is supporting, denying, questioning or simply\ncommenting on the rumour, has been attracting substantial interest. Here we\nintroduce a novel approach that makes use of the sequence of transitions\nobserved in tree-structured conversation threads in Twitter. The conversation\nthreads are formed by harvesting users' replies to one another, which results\nin a nested tree-like structure. Previous work addressing the stance\nclassification task has treated each tweet as a separate unit. Here we analyse\ntweets by virtue of their position in a sequence and test two sequential\nclassifiers, Linear-Chain CRF and Tree CRF, each of which makes different\nassumptions about the conversational structure. We experiment with eight\nTwitter datasets, collected during breaking news, and show that exploiting the\nsequential structure of Twitter conversations achieves significant improvements\nover the non-sequential methods. Our work is the first to model Twitter\nconversations as a tree structure in this manner, introducing a novel way of\ntackling NLP tasks on Twitter conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09247v1",
            "date": "2016-09-29 08:12:14+00:00",
            "title": "Training Dependency Parsers with Partial Annotation",
            "authors": [
                "Zhenghua Li",
                "Yue Zhang",
                "Jiayuan Chao",
                "Min Zhang"
            ],
            "abstract": "Recently, these has been a surge on studying how to obtain partially\nannotated data for model supervision. However, there still lacks a systematic\nstudy on how to train statistical models with partial annotation (PA). Taking\ndependency parsing as our case study, this paper describes and compares two\nstraightforward approaches for three mainstream dependency parsers. The first\napproach is previously proposed to directly train a log-linear graph-based\nparser (LLGPar) with PA based on a forest-based objective. This work for the\nfirst time proposes the second approach to directly training a linear\ngraph-based parse (LGPar) and a linear transition-based parser (LTPar) with PA\nbased on the idea of constrained decoding. We conduct extensive experiments on\nPenn Treebank under three different settings for simulating PA, i.e., random\ndependencies, most uncertain dependencies, and dependencies with divergent\noutputs from the three parsers. The results show that LLGPar is most effective\nin learning from PA and LTPar lags behind the graph-based counterparts by large\nmargin. Moreover, LGPar and LTPar can achieve best performance by using LLGPar\nto complete PA into full annotation (FA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09405v2",
            "date": "2016-09-29 16:09:29+00:00",
            "title": "Evaluating Induced CCG Parsers on Grounded Semantic Parsing",
            "authors": [
                "Yonatan Bisk",
                "Siva Reddy",
                "John Blitzer",
                "Julia Hockenmaier",
                "Mark Steedman"
            ],
            "abstract": "We compare the effectiveness of four different syntactic CCG parsers for a\nsemantic slot-filling task to explore how much syntactic supervision is\nrequired for downstream semantic analysis. This extrinsic, task-based\nevaluation provides a unique window to explore the strengths and weaknesses of\nsemantics captured by unsupervised grammar induction systems. We release a new\nFreebase semantic parsing dataset called SPADES (Semantic PArsing of\nDEclarative Sentences) containing 93K cloze-style questions paired with\nanswers. We evaluate all our models on this dataset. Our code and data are\navailable at https://github.com/sivareddyg/graph-parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00388v3",
            "date": "2016-10-03 02:11:03+00:00",
            "title": "Learning to Translate in Real-time with Neural Machine Translation",
            "authors": [
                "Jiatao Gu",
                "Graham Neubig",
                "Kyunghyun Cho",
                "Victor O. K. Li"
            ],
            "abstract": "Translating in real-time, a.k.a. simultaneous translation, outputs\ntranslation words before the input sentence ends, which is a challenging\nproblem for conventional machine translation methods. We propose a neural\nmachine translation (NMT) framework for simultaneous translation in which an\nagent learns to make decisions on when to translate from the interaction with a\npre-trained NMT environment. To trade off quality and delay, we extensively\nexplore various targets for delay and design a method for beam-search\napplicable in the simultaneous MT setting. Experiments against state-of-the-art\nbaselines on two language pairs demonstrate the efficacy of the proposed\nframework both quantitatively and qualitatively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00572v1",
            "date": "2016-10-03 14:44:58+00:00",
            "title": "An Arabic-Hebrew parallel corpus of TED talks",
            "authors": [
                "Mauro Cettolo"
            ],
            "abstract": "We describe an Arabic-Hebrew parallel corpus of TED talks built upon WIT3,\nthe Web inventory that repurposes the original content of the TED website in a\nway which is more convenient for MT researchers. The benchmark consists of\nabout 2,000 talks, whose subtitles in Arabic and Hebrew have been accurately\naligned and rearranged in sentences, for a total of about 3.5M tokens per\nlanguage. Talks have been partitioned in train, development and test sets\nsimilarly in all respects to the MT tasks of the IWSLT 2016 evaluation\ncampaign. In addition to describing the benchmark, we list the problems\nencountered in preparing it and the novel methods designed to solve them.\nBaseline MT results and some measures on sentence length are provided as an\nextrinsic evaluation of the quality of the benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00852v1",
            "date": "2016-10-04 06:01:03+00:00",
            "title": "Ensemble Maximum Entropy Classification and Linear Regression for Author Age Prediction",
            "authors": [
                "Joey Hong",
                "Chris Mattmann",
                "Paul Ramirez"
            ],
            "abstract": "The evolution of the internet has created an abundance of unstructured data\non the web, a significant part of which is textual. The task of author\nprofiling seeks to find the demographics of people solely from their linguistic\nand content-based features in text. The ability to describe traits of authors\nclearly has applications in fields such as security and forensics, as well as\nmarketing. Instead of seeing age as just a classification problem, we also\nframe age as a regression one, but use an ensemble chain method that\nincorporates the power of both classification and regression to learn the\nauthors exact age.",
            "journal ref": "2017 IEEE International Conference on Information Reuse and\n  Integration (IRI)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1610.01247v1",
            "date": "2016-10-05 01:24:42+00:00",
            "title": "ECAT: Event Capture Annotation Tool",
            "authors": [
                "Tuan Do",
                "Nikhil Krishnaswamy",
                "James Pustejovsky"
            ],
            "abstract": "This paper introduces the Event Capture Annotation Tool (ECAT), a\nuser-friendly, open-source interface tool for annotating events and their\nparticipants in video, capable of extracting the 3D positions and orientations\nof objects in video captured by Microsoft's Kinect(R) hardware. The modeling\nlanguage VoxML (Pustejovsky and Krishnaswamy, 2016) underlies ECAT's object,\nprogram, and attribute representations, although ECAT uses its own spec for\nexplicit labeling of motion instances. The demonstration will show the tool's\nworkflow and the options available for capturing event-participant relations\nand browsing visual data. Mapping ECAT's output to VoxML will also be\naddressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01367v1",
            "date": "2016-10-05 11:34:36+00:00",
            "title": "Monaural Multi-Talker Speech Recognition using Factorial Speech Processing Models",
            "authors": [
                "Mahdi Khademian",
                "Mohammad Mehdi Homayounpour"
            ],
            "abstract": "A Pascal challenge entitled monaural multi-talker speech recognition was\ndeveloped, targeting the problem of robust automatic speech recognition against\nspeech like noises which significantly degrades the performance of automatic\nspeech recognition systems. In this challenge, two competing speakers say a\nsimple command simultaneously and the objective is to recognize speech of the\ntarget speaker. Surprisingly during the challenge, a team from IBM research,\ncould achieve a performance better than human listeners on this task. The\nproposed method of the IBM team, consist of an intermediate speech separation\nand then a single-talker speech recognition. This paper reconsiders the task of\nthis challenge based on gain adapted factorial speech processing models. It\ndevelops a joint-token passing algorithm for direct utterance decoding of both\ntarget and masker speakers, simultaneously. Comparing it to the challenge\nwinner, it uses maximum uncertainty during the decoding which cannot be used in\nthe past two-phased method. It provides detailed derivation of inference on\nthese models based on general inference procedures of probabilistic graphical\nmodels. As another improvement, it uses deep neural networks for joint-speaker\nidentification and gain estimation which makes these two steps easier than\nbefore producing competitive results for these steps. The proposed method of\nthis work outperforms past super-human results and even the results were\nachieved recently by Microsoft research, using deep neural networks. It\nachieved 5.5% absolute task performance improvement compared to the first\nsuper-human system and 2.7% absolute task performance improvement compared to\nits recent competitor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01382v1",
            "date": "2016-10-05 12:16:35+00:00",
            "title": "Divide-and-Conquer based Ensemble to Spot Emotions in Speech using MFCC and Random Forest",
            "authors": [
                "Abdul Malik Badshah",
                "Jamil Ahmad",
                "Mi Young Lee",
                "Sung Wook Baik"
            ],
            "abstract": "Besides spoken words, speech signals also carry information about speaker\ngender, age, and emotional state which can be used in a variety of speech\nanalysis applications. In this paper, a divide and conquer strategy for\nensemble classification has been proposed to recognize emotions in speech.\nIntrinsic hierarchy in emotions has been utilized to construct an emotions\ntree, which assisted in breaking down the emotion recognition task into smaller\nsub tasks. The proposed framework generates predictions in three phases.\nFirstly, emotions are detected in the input speech signal by classifying it as\nneutral or emotional. If the speech is classified as emotional, then in the\nsecond phase, it is further classified into positive and negative classes.\nFinally, individual positive or negative emotions are identified based on the\noutcomes of the previous stages. Several experiments have been performed on a\nwidely used benchmark dataset. The proposed method was able to achieve improved\nrecognition rates as compared to several other approaches.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1610.01520v2",
            "date": "2016-10-05 16:47:17+00:00",
            "title": "Comparative study of LSA vs Word2vec embeddings in small corpora: a case study in dreams database",
            "authors": [
                "Edgar Altszyler",
                "Mariano Sigman",
                "Sidarta Ribeiro",
                "Diego Fern\u00e1ndez Slezak"
            ],
            "abstract": "Word embeddings have been extensively studied in large text datasets.\nHowever, only a few studies analyze semantic representations of small corpora,\nparticularly relevant in single-person text production studies. In the present\npaper, we compare Skip-gram and LSA capabilities in this scenario, and we test\nboth techniques to extract relevant semantic patterns in single-series dreams\nreports. LSA showed better performance than Skip-gram in small size training\ncorpus in two semantic tests. As a study case, we show that LSA can capture\nrelevant words associations in dream reports series, even in cases of small\nnumber of dreams or low-frequency words. We propose that LSA can be used to\nexplore words associations in dreams reports, which could bring new insight\ninto this classic research area of psychology",
            "journal ref": "Conscious Cogn. 2017 Nov;56:178-187",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01561v1",
            "date": "2016-10-05 18:39:53+00:00",
            "title": "Summarizing Situational and Topical Information During Crises",
            "authors": [
                "Koustav Rudra",
                "Siddhartha Banerjee",
                "Niloy Ganguly",
                "Pawan Goyal",
                "Muhammad Imran",
                "Prasenjit Mitra"
            ],
            "abstract": "The use of microblogging platforms such as Twitter during crises has become\nwidespread. More importantly, information disseminated by affected people\ncontains useful information like reports of missing and found people, requests\nfor urgent needs etc. For rapid crisis response, humanitarian organizations\nlook for situational awareness information to understand and assess the\nseverity of the crisis. In this paper, we present a novel framework (i) to\ngenerate abstractive summaries useful for situational awareness, and (ii) to\ncapture sub-topics and present a short informative summary for each of these\ntopics. A summary is generated using a two stage framework that first extracts\na set of important tweets from the whole set of information through an\nInteger-linear programming (ILP) based optimization technique and then follows\na word graph and concept event based abstractive summarization technique to\nproduce the final summary. High accuracies obtained for all the tasks show the\neffectiveness of the proposed framework.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1610.01720v1",
            "date": "2016-10-06 02:39:29+00:00",
            "title": "Automatic Detection of Small Groups of Persons, Influential Members, Relations and Hierarchy in Written Conversations Using Fuzzy Logic",
            "authors": [
                "French Pope III",
                "Rouzbeh A. Shirvani",
                "Mugizi Robert Rwebangira",
                "Mohamed Chouikha",
                "Ayo Taylor",
                "Andres Alarcon Ramirez",
                "Amirsina Torfi"
            ],
            "abstract": "Nowadays a lot of data is collected in online forums. One of the key tasks is\nto determine the social structure of these online groups, for example the\nidentification of subgroups within a larger group. We will approach the\ngrouping of individual as a classification problem. The classifier will be\nbased on fuzzy logic. The input to the classifier will be linguistic features\nand degree of relationships (among individuals). The output of the classifiers\nare the groupings of individuals. We also incorporate a method that ranks the\nmembers of the detected subgroup to identify the hierarchies in each subgroup.\nData from the HBO television show The Wire is used to analyze the efficacy and\nusefulness of fuzzy logic based methods as alternative methods to classical\nstatistical methods usually used for these problems. The proposed methodology\ncould detect automatically the most influential members of each organization\nThe Wire with 90% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01858v1",
            "date": "2016-10-06 13:20:07+00:00",
            "title": "A Robust Framework for Classifying Evolving Document Streams in an Expert-Machine-Crowd Setting",
            "authors": [
                "Muhammad Imran",
                "Sanjay Chawla",
                "Carlos Castillo"
            ],
            "abstract": "An emerging challenge in the online classification of social media data\nstreams is to keep the categories used for classification up-to-date. In this\npaper, we propose an innovative framework based on an Expert-Machine-Crowd\n(EMC) triad to help categorize items by continuously identifying novel concepts\nin heterogeneous data streams often riddled with outliers. We unify constrained\nclustering and outlier detection by formulating a novel optimization problem:\nCOD-Means. We design an algorithm to solve the COD-Means problem and show that\nCOD-Means will not only help detect novel categories but also seamlessly\ndiscover human annotation errors and improve the overall quality of the\ncategorization process. Experiments on diverse real data sets demonstrate that\nour approach is both effective and efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02209v2",
            "date": "2016-10-07 09:59:13+00:00",
            "title": "Morphology Generation for Statistical Machine Translation using Deep Learning Techniques",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Carlos Escolano"
            ],
            "abstract": "Morphology in unbalanced languages remains a big challenge in the context of\nmachine translation. In this paper, we propose to de-couple machine translation\nfrom morphology generation in order to better deal with the problem. We\ninvestigate the morphology simplification with a reasonable trade-off between\nexpected gain and generation complexity. For the Chinese-Spanish task, optimum\nmorphological simplification is in gender and number. For this purpose, we\ndesign a new classification architecture which, compared to other standard\nmachine learning techniques, obtains the best results. This proposed\nneural-based architecture consists of several layers: an embedding, a\nconvolutional followed by a recurrent neural network and, finally, ends with\nsigmoid and softmax layers. We obtain classification results over 98% accuracy\nin gender classification, over 93% in number classification, and an overall\ntranslation improvement of 0.7 METEOR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02567v1",
            "date": "2016-10-08 19:43:23+00:00",
            "title": "Mining the Web for Pharmacovigilance: the Case Study of Duloxetine and Venlafaxine",
            "authors": [
                "Abbas Chokor",
                "Abeed Sarker",
                "Graciela Gonzalez"
            ],
            "abstract": "Adverse reactions caused by drugs following their release into the market are\namong the leading causes of death in many countries. The rapid growth of\nelectronically available health related information, and the ability to process\nlarge volumes of them automatically, using natural language processing (NLP)\nand machine learning algorithms, have opened new opportunities for\npharmacovigilance. Survey found that more than 70% of US Internet users consult\nthe Internet when they require medical information. In recent years, research\nin this area has addressed for Adverse Drug Reaction (ADR) pharmacovigilance\nusing social media, mainly Twitter and medical forums and websites. This paper\nwill show the information which can be collected from a variety of Internet\ndata sources and search engines, mainly Google Trends and Google Correlate.\nWhile considering the case study of two popular Major depressive Disorder (MDD)\ndrugs, Duloxetine and Venlafaxine, we will provide a comparative analysis for\ntheir reactions using publicly-available alternative data sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02683v1",
            "date": "2016-10-09 14:42:58+00:00",
            "title": "Interpreting Neural Networks to Improve Politeness Comprehension",
            "authors": [
                "Malika Aubakirova",
                "Mohit Bansal"
            ],
            "abstract": "We present an interpretable neural network approach to predicting and\nunderstanding politeness in natural language requests. Our models are based on\nsimple convolutional neural networks directly on raw text, avoiding any manual\nidentification of complex sentiment or syntactic features, while performing\nbetter than such feature-based models from previous work. More importantly, we\nuse the challenging task of politeness prediction as a testbed to next present\na much-needed understanding of what these successful networks are actually\nlearning. For this, we present several network visualizations based on\nactivation clusters, first derivative saliency, and embedding space\ntransformations, helping us automatically identify several subtle linguistics\nmarkers of politeness theories. Further, this analysis reveals multiple novel,\nhigh-scoring politeness strategies which, when added back as new features,\nreduce the accuracy gap between the original featurized system and the neural\nmodel, thus providing a clear quantitative interpretation of the success of\nthese neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02736v1",
            "date": "2016-10-09 22:41:19+00:00",
            "title": "Emergence of linguistic laws in human voice",
            "authors": [
                "Ivan Gonzalez Torre",
                "Bartolo Luque",
                "Lucas Lacasa",
                "Jordi Luque",
                "Antoni Hernandez-Fernandez"
            ],
            "abstract": "Linguistic laws constitute one of the quantitative cornerstones of modern\ncognitive sciences and have been routinely investigated in written corpora, or\nin the equivalent transcription of oral corpora. This means that inferences of\nstatistical patterns of language in acoustics are biased by the arbitrary,\nlanguage-dependent segmentation of the signal, and virtually precludes the\npossibility of making comparative studies between human voice and other animal\ncommunication systems. Here we bridge this gap by proposing a method that\nallows to measure such patterns in acoustic signals of arbitrary origin,\nwithout needs to have access to the language corpus underneath. The method has\nbeen applied to six different human languages, recovering successfully some\nwell-known laws of human communication at timescales even below the phoneme and\nfinding yet another link between complexity and criticality in a biological\nsystem. These methods further pave the way for new comparative studies in\nanimal communication or the analysis of signals of unknown code.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1610.02751v1",
            "date": "2016-10-10 02:14:16+00:00",
            "title": "A New Theoretical and Technological System of Imprecise-Information Processing",
            "authors": [
                "Shiyou Lian"
            ],
            "abstract": "Imprecise-information processing will play an indispensable role in\nintelligent systems, especially in the anthropomorphic intelligent systems (as\nintelligent robots). A new theoretical and technological system of\nimprecise-information processing has been founded in Principles of\nImprecise-Information Processing: A New Theoretical and Technological System[1]\nwhich is different from fuzzy technology. The system has clear hierarchy and\nrigorous structure, which results from the formation principle of imprecise\ninformation and has solid mathematical and logical bases, and which has many\nadvantages beyond fuzzy technology. The system provides a technological\nplatform for relevant applications and lays a theoretical foundation for\nfurther research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03009v1",
            "date": "2016-10-10 18:03:29+00:00",
            "title": "Investigation of Synthetic Speech Detection Using Frame- and Segment-Specific Importance Weighting",
            "authors": [
                "Ali Khodabakhsh",
                "Cenk Demiroglu"
            ],
            "abstract": "Speaker verification systems are vulnerable to spoofing attacks which\npresents a major problem in their real-life deployment. To date, most of the\nproposed synthetic speech detectors (SSDs) have weighted the importance of\ndifferent segments of speech equally. However, different attack methods have\ndifferent strengths and weaknesses and the traces that they leave may be short\nor long term acoustic artifacts. Moreover, those may occur for only particular\nphonemes or sounds. Here, we propose three algorithms that weigh\nlikelihood-ratio scores of individual frames, phonemes, and sound-classes\ndepending on their importance for the SSD. Significant improvement over the\nbaseline system has been obtained for known attack methods that were used in\ntraining the SSDs. However, improvement with unknown attack types was not\nsubstantial. Thus, the type of distortions that were caused by the unknown\nsystems were different and could not be captured better with the proposed SSD\ncompared to the baseline SSD.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1610.03017v3",
            "date": "2016-10-10 18:19:34+00:00",
            "title": "Fully Character-Level Neural Machine Translation without Explicit Segmentation",
            "authors": [
                "Jason Lee",
                "Kyunghyun Cho",
                "Thomas Hofmann"
            ],
            "abstract": "Most existing machine translation systems operate at the level of words,\nrelying on explicit segmentation to extract tokens. We introduce a neural\nmachine translation (NMT) model that maps a source character sequence to a\ntarget character sequence without any segmentation. We employ a character-level\nconvolutional network with max-pooling at the encoder to reduce the length of\nsource representation, allowing the model to be trained at a speed comparable\nto subword-level models while capturing local regularities. Our\ncharacter-to-character model outperforms a recently proposed baseline with a\nsubword-level encoder on WMT'15 DE-EN and CS-EN, and gives comparable\nperformance on FI-EN and RU-EN. We then demonstrate that it is possible to\nshare a single character-level encoder across multiple languages by training a\nmodel on a many-to-one translation task. In this multilingual setting, the\ncharacter-level encoder significantly outperforms the subword-level encoder on\nall the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality\nof the multilingual character-level translation even surpasses the models\nspecifically trained on that language pair alone, both in terms of BLEU score\nand human judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03120v2",
            "date": "2016-10-10 22:35:21+00:00",
            "title": "Correlation-Based Method for Sentiment Classification",
            "authors": [
                "Hussam Hamdan"
            ],
            "abstract": "The classic supervised classification algorithms are efficient, but\ntime-consuming, complicated and not interpretable, which makes it difficult to\nanalyze their results that limits the possibility to improve them based on real\nobservations. In this paper, we propose a new and a simple classifier to\npredict a sentiment label of a short text. This model keeps the capacity of\nhuman interpret-ability and can be extended to integrate NLP techniques in a\nmore interpretable way. Our model is based on a correlation metric which\nmeasures the degree of association between a sentiment label and a word. Ten\ncorrelation metrics are proposed and evaluated intrinsically. And then a\nclassifier based on each metric is proposed, evaluated and compared to the\nclassic classification algorithms which have proved their performance in many\nstudies. Our model outperforms these algorithms with several correlation\nmetrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03165v1",
            "date": "2016-10-11 02:48:13+00:00",
            "title": "Long Short-Term Memory based Convolutional Recurrent Neural Networks for Large Vocabulary Speech Recognition",
            "authors": [
                "Xiangang Li",
                "Xihong Wu"
            ],
            "abstract": "Long short-term memory (LSTM) recurrent neural networks (RNNs) have been\nshown to give state-of-the-art performance on many speech recognition tasks, as\nthey are able to provide the learned dynamically changing contextual window of\nall sequence history. On the other hand, the convolutional neural networks\n(CNNs) have brought significant improvements to deep feed-forward neural\nnetworks (FFNNs), as they are able to better reduce spectral variation in the\ninput signal. In this paper, a network architecture called as convolutional\nrecurrent neural network (CRNN) is proposed by combining the CNN and LSTM RNN.\nIn the proposed CRNNs, each speech frame, without adjacent context frames, is\norganized as a number of local feature patches along the frequency axis, and\nthen a LSTM network is performed on each feature patch along the time axis. We\ntrain and compare FFNNs, LSTM RNNs and the proposed LSTM CRNNs at various\nnumber of configurations. Experimental results show that the LSTM CRNNs can\nexceed state-of-the-art speech recognition performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03342v1",
            "date": "2016-10-11 14:00:28+00:00",
            "title": "From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learning",
            "authors": [
                "Lieke Gelderloos",
                "Grzegorz Chrupa\u0142a"
            ],
            "abstract": "We present a model of visually-grounded language learning based on stacked\ngated recurrent neural networks which learns to predict visual features given\nan image description in the form of a sequence of phonemes. The learning task\nresembles that faced by human language learners who need to discover both\nstructure and meaning from noisy and ambiguous data across modalities. We show\nthat our model indeed learns to predict features of the visual context given\nphonetically transcribed image descriptions, and show that it represents\nlinguistic information in a hierarchy of levels: lower layers in the stack are\ncomparatively more sensitive to form, whereas higher layers are more sensitive\nto meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03708v2",
            "date": "2016-10-12 13:42:03+00:00",
            "title": "Generating captions without looking beyond objects",
            "authors": [
                "Hendrik Heuer",
                "Christof Monz",
                "Arnold W. M. Smeulders"
            ],
            "abstract": "This paper explores new evaluation perspectives for image captioning and\nintroduces a noun translation task that achieves comparative image caption\ngeneration performance by translating from a set of nouns to captions. This\nimplies that in image captioning, all word categories other than nouns can be\nevoked by a powerful language model without sacrificing performance on n-gram\nprecision. The paper also investigates lower and upper bounds of how much\nindividual word categories in the captions contribute to the final BLEU score.\nA large possible improvement exists for nouns, verbs, and prepositions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1610.03750v1",
            "date": "2016-10-12 15:26:30+00:00",
            "title": "Semi-supervised Discovery of Informative Tweets During the Emerging Disasters",
            "authors": [
                "Shanshan Zhang",
                "Slobodan Vucetic"
            ],
            "abstract": "The first objective towards the effective use of microblogging services such\nas Twitter for situational awareness during the emerging disasters is discovery\nof the disaster-related postings. Given the wide range of possible disasters,\nusing a pre-selected set of disaster-related keywords for the discovery is\nsuboptimal. An alternative that we focus on in this work is to train a\nclassifier using a small set of labeled postings that are becoming available as\na disaster is emerging. Our hypothesis is that utilizing large quantities of\nhistorical microblogs could improve the quality of classification, as compared\nto training a classifier only on the labeled data. We propose to use unlabeled\nmicroblogs to cluster words into a limited number of clusters and use the word\nclusters as features for classification. To evaluate the proposed\nsemi-supervised approach, we used Twitter data from 6 different disasters. Our\nresults indicate that when the number of labeled tweets is 100 or less, the\nproposed approach is superior to the standard classification based on the bag\nor words feature representation. Our results also reveal that the choice of the\nunlabeled corpus, the choice of word clustering algorithm, and the choice of\nhyperparameters can have a significant impact on the classification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03950v1",
            "date": "2016-10-13 06:55:54+00:00",
            "title": "Compressing Neural Language Models by Sparse Word Representations",
            "authors": [
                "Yunchuan Chen",
                "Lili Mou",
                "Yan Xu",
                "Ge Li",
                "Zhi Jin"
            ],
            "abstract": "Neural networks are among the state-of-the-art techniques for language\nmodeling. Existing neural language models typically map discrete words to\ndistributed, dense vector representations. After information processing of the\npreceding context words by hidden layers, an output layer estimates the\nprobability of the next word. Such approaches are time- and memory-intensive\nbecause of the large numbers of parameters for word embeddings and the output\nlayer. In this paper, we propose to compress neural language models by sparse\nword representations. In the experiments, the number of parameters in our model\nincreases very slowly with the growth of the vocabulary size, which is almost\nimperceptible. Moreover, our approach not only reduces the parameter space to a\nlarge extent, but also improves the performance in terms of the perplexity\nmeasure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03955v1",
            "date": "2016-10-13 07:07:50+00:00",
            "title": "Dialogue Session Segmentation by Embedding-Enhanced TextTiling",
            "authors": [
                "Yiping Song",
                "Lili Mou",
                "Rui Yan",
                "Li Yi",
                "Zinan Zhu",
                "Xiaohua Hu",
                "Ming Zhang"
            ],
            "abstract": "In human-computer conversation systems, the context of a user-issued\nutterance is particularly important because it provides useful background\ninformation of the conversation. However, it is unwise to track all previous\nutterances in the current session as not all of them are equally important. In\nthis paper, we address the problem of session segmentation. We propose an\nembedding-enhanced TextTiling approach, inspired by the observation that\nconversation utterances are highly noisy, and that word embeddings provide a\nrobust way of capturing semantics. Experimental results show that our approach\nachieves better performance than the TextTiling, MMD approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04211v2",
            "date": "2016-10-13 19:38:03+00:00",
            "title": "Gated End-to-End Memory Networks",
            "authors": [
                "Julien Perez",
                "Fei Liu"
            ],
            "abstract": "Machine reading using differentiable reasoning models has recently shown\nremarkable progress. In this context, End-to-End trainable Memory Networks,\nMemN2N, have demonstrated promising performance on simple natural language\nbased reasoning tasks such as factual reasoning and basic deduction. However,\nother tasks, namely multi-fact question-answering, positional reasoning or\ndialog related tasks, remain challenging particularly due to the necessity of\nmore complex interactions between the memory and controller modules composing\nthis family of models. In this paper, we introduce a novel end-to-end memory\naccess regulation mechanism inspired by the current progress on the connection\nshort-cutting principle in the field of computer vision. Concretely, we develop\na Gated End-to-End trainable Memory Network architecture, GMemN2N. From the\nmachine learning perspective, this new capability is learned in an end-to-end\nfashion without the use of any additional supervision signal which is, as far\nas our knowledge goes, the first of its kind. Our experiments show significant\nimprovements on the most challenging tasks in the 20 bAbI dataset, without the\nuse of any domain knowledge. Then, we show improvements on the dialog bAbI\ntasks including the real human-bot conversion-based Dialog State Tracking\nChallenge (DSTC-2) dataset. On these two datasets, our model sets the new state\nof the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04345v1",
            "date": "2016-10-14 07:14:44+00:00",
            "title": "A Language-independent and Compositional Model for Personality Trait Recognition from Short Texts",
            "authors": [
                "Fei Liu",
                "Julien Perez",
                "Scott Nowson"
            ],
            "abstract": "Many methods have been used to recognize author personality traits from text,\ntypically combining linguistic feature engineering with shallow learning\nmodels, e.g. linear regression or Support Vector Machines. This work uses\ndeep-learning-based models and atomic features of text, the characters, to\nbuild hierarchical, vectorial word and sentence representations for trait\ninference. This method, applied to a corpus of tweets, shows state-of-the-art\nperformance across five traits and three languages (English, Spanish and\nItalian) compared with prior work in author profiling. The results, supported\nby preliminary visualisation work, are encouraging for the ability to detect\ncomplex human traits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04416v1",
            "date": "2016-10-14 11:52:19+00:00",
            "title": "Distributional Inclusion Hypothesis for Tensor-based Composition",
            "authors": [
                "Dimitri Kartsaklis",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "According to the distributional inclusion hypothesis, entailment between\nwords can be measured via the feature inclusions of their distributional\nvectors. In recent work, we showed how this hypothesis can be extended from\nwords to phrases and sentences in the setting of compositional distributional\nsemantics. This paper focuses on inclusion properties of tensors; its main\ncontribution is a theoretical and experimental analysis of how feature\ninclusion works in different concrete models of verb tensors. We present\nresults for relational, Frobenius, projective, and holistic methods and compare\nthem to the simple vector addition, multiplication, min, and max models. The\ndegrees of entailment thus obtained are evaluated via a variety of existing\nword-based measures, such as Weed's and Clarke's, KL-divergence, APinc,\nbalAPinc, and two of our previously proposed metrics at the phrase/sentence\nlevel. We perform experiments on three entailment datasets, investigating which\nversion of tensor-based composition achieves the highest performance when\ncombined with the sentence-level measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04533v1",
            "date": "2016-02-17 19:33:47+00:00",
            "title": "A Comprehensive Comparative Study of Word and Sentence Similarity Measures",
            "authors": [
                "Issa Atoum",
                "Ahmed Otoom",
                "Narayanan Kulathuramaiyer"
            ],
            "abstract": "Sentence similarity is considered the basis of many natural language tasks\nsuch as information retrieval, question answering and text summarization. The\nsemantic meaning between compared text fragments is based on the words semantic\nfeatures and their relationships. This article reviews a set of word and\nsentence similarity measures and compares them on benchmark datasets. On the\nstudied datasets, results showed that hybrid semantic measures perform better\nthan both knowledge and corpus based measures.",
            "journal ref": "International Journal of Computer Applications,2016,135(1),\n  Foundation of Computer Science (FCS), NY, USA",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1610.04718v2",
            "date": "2016-10-15 11:04:44+00:00",
            "title": "Generalization of metric classification algorithms for sequences classification and labelling",
            "authors": [
                "Roman Samarev",
                "Andrey Vasnetsov",
                "Elizaveta Smelkova"
            ],
            "abstract": "The article deals with the issue of modification of metric classification\nalgorithms. In particular, it studies the algorithm k-Nearest Neighbours for\nits application to sequential data. A method of generalization of metric\nclassification algorithms is proposed. As a part of it, there has been\ndeveloped an algorithm for solving the problem of classification and labelling\nof sequential data. The advantages of the developed algorithm of classification\nin comparison with the existing one are also discussed in the article. There is\na comparison of the effectiveness of the proposed algorithm with the algorithm\nof CRF in the task of chunking in the open data set CoNLL2000.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1610.04814v1",
            "date": "2016-10-16 03:40:13+00:00",
            "title": "Term-Class-Max-Support (TCMS): A Simple Text Document Categorization Approach Using Term-Class Relevance Measure",
            "authors": [
                "D S Guru",
                "Mahamad Suhil"
            ],
            "abstract": "In this paper, a simple text categorization method using term-class relevance\nmeasures is proposed. Initially, text documents are processed to extract\nsignificant terms present in them. For every term extracted from a document, we\ncompute its importance in preserving the content of a class through a novel\nterm-weighting scheme known as Term_Class Relevance (TCR) measure proposed by\nGuru and Suhil (2015) [1]. In this way, for every term, its relevance for all\nthe classes present in the corpus is computed and stored in the knowledgebase.\nDuring testing, the terms present in the test document are extracted and the\nterm-class relevance of each term is obtained from the stored knowledgebase. To\nachieve quick search of term weights, Btree indexing data structure has been\nadapted. Finally, the class which receives maximum support in terms of\nterm-class relevance is decided to be the class of the given test document. The\nproposed method works in logarithmic complexity in testing time and simple to\nimplement when compared to any other text categorization techniques available\nin literature. The experiments conducted on various benchmarking datasets have\nrevealed that the performance of the proposed method is satisfactory and\nencouraging.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1610.04989v1",
            "date": "2016-10-17 07:28:06+00:00",
            "title": "Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification",
            "authors": [
                "Jiacheng Xu",
                "Danlu Chen",
                "Xipeng Qiu",
                "Xuangjing Huang"
            ],
            "abstract": "Recently, neural networks have achieved great success on sentiment\nclassification due to their ability to alleviate feature engineering. However,\none of the remaining challenges is to model long texts in document-level\nsentiment classification under a recurrent architecture because of the\ndeficiency of the memory unit. To address this problem, we present a Cached\nLong Short-Term Memory neural networks (CLSTM) to capture the overall semantic\ninformation in long texts. CLSTM introduces a cache mechanism, which divides\nmemory into several groups with different forgetting rates and thus enables the\nnetwork to keep sentiment information better within a recurrent unit. The\nproposed CLSTM outperforms the state-of-the-art models on three publicly\navailable document-level sentiment analysis datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05256v2",
            "date": "2016-10-17 18:40:50+00:00",
            "title": "Achieving Human Parity in Conversational Speech Recognition",
            "authors": [
                "W. Xiong",
                "J. Droppo",
                "X. Huang",
                "F. Seide",
                "M. Seltzer",
                "A. Stolcke",
                "D. Yu",
                "G. Zweig"
            ],
            "abstract": "Conversational speech recognition has served as a flagship speech recognition\ntask since the release of the Switchboard corpus in the 1990s. In this paper,\nwe measure the human error rate on the widely used NIST 2000 test set, and find\nthat our latest automated system has reached human parity. The error rate of\nprofessional transcribers is 5.9% for the Switchboard portion of the data, in\nwhich newly acquainted pairs of people discuss an assigned topic, and 11.3% for\nthe CallHome portion where friends and family members have open-ended\nconversations. In both cases, our automated system establishes a new state of\nthe art, and edges past the human benchmark, achieving error rates of 5.8% and\n11.0%, respectively. The key to our system's performance is the use of various\nconvolutional and LSTM acoustic model architectures, combined with a novel\nspatial smoothing method and lattice-free MMI acoustic training, multiple\nrecurrent neural network language modeling approaches, and a systematic use of\nsystem combination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05654v2",
            "date": "2016-10-18 14:53:20+00:00",
            "title": "The infochemical core",
            "authors": [
                "Antoni Hern\u00e1ndez-Fern\u00e1ndez",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Vocalizations and less often gestures have been the object of linguistic\nresearch over decades. However, the development of a general theory of\ncommunication with human language as a particular case requires a clear\nunderstanding of the organization of communication through other means.\nInfochemicals are chemical compounds that carry information and are employed by\nsmall organisms that cannot emit acoustic signals of optimal frequency to\nachieve successful communication. Here the distribution of infochemicals across\nspecies is investigated when they are ranked by their degree or the number of\nspecies with which it is associated (because they produce or they are sensitive\nto it). The quality of the fit of different functions to the dependency between\ndegree and rank is evaluated with a penalty for the number of parameters of the\nfunction. Surprisingly, a double Zipf (a Zipf distribution with two regimes\nwith a different exponent each) is the model yielding the best fit although it\nis the function with the largest number of parameters. This suggests that the\nworld wide repertoire of infochemicals contains a chemical nucleus shared by\nmany species and reminiscent of the core vocabularies found for human language\nin dictionaries or large corpora.",
            "journal ref": "Journal of Quantitative Linguistics 23 (2), 133-153 (2016)",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1610.05670v2",
            "date": "2016-10-18 15:22:14+00:00",
            "title": "Stylometric Analysis of Early Modern Period English Plays",
            "authors": [
                "Mark Eisen",
                "Santiago Segarra",
                "Gabriel Egan",
                "Alejandro Ribeiro"
            ],
            "abstract": "Function word adjacency networks (WANs) are used to study the authorship of\nplays from the Early Modern English period. In these networks, nodes are\nfunction words and directed edges between two nodes represent the relative\nfrequency of directed co-appearance of the two words. For every analyzed play,\na WAN is constructed and these are aggregated to generate author profile\nnetworks. We first study the similarity of writing styles between Early English\nplaywrights by comparing the profile WANs. The accuracy of using WANs for\nauthorship attribution is then demonstrated by attributing known plays among\nsix popular playwrights. Moreover, the WAN method is shown to outperform other\nfrequency-based methods on attributing Early English plays. In addition, WANs\nare shown to be reliable classifiers even when attributing collaborative plays.\nFor several plays of disputed co-authorship, a deeper analysis is performed by\nattributing every act and scene separately, in which we both corroborate\nexisting breakdowns and provide evidence of new assignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05812v4",
            "date": "2016-10-18 22:06:01+00:00",
            "title": "Small-footprint Highway Deep Neural Networks for Speech Recognition",
            "authors": [
                "Liang Lu",
                "Steve Renals"
            ],
            "abstract": "State-of-the-art speech recognition systems typically employ neural network\nacoustic models. However, compared to Gaussian mixture models, deep neural\nnetwork (DNN) based acoustic models often have many more model parameters,\nmaking it challenging for them to be deployed on resource-constrained\nplatforms, such as mobile devices. In this paper, we study the application of\nthe recently proposed highway deep neural network (HDNN) for training\nsmall-footprint acoustic models. HDNNs are a depth-gated feedforward neural\nnetwork, which include two types of gate functions to facilitate the\ninformation flow through different layers. Our study demonstrates that HDNNs\nare more compact than regular DNNs for acoustic modeling, i.e., they can\nachieve comparable recognition accuracy with many fewer model parameters.\nFurthermore, HDNNs are more controllable than DNNs: the gate functions of an\nHDNN can control the behavior of the whole network using a very small number of\nmodel parameters. Finally, we show that HDNNs are more adaptable than DNNs. For\nexample, simply updating the gate functions using adaptation data can result in\nconsiderable gains in accuracy. We demonstrate these aspects by experiments\nusing the publicly available AMI corpus, which has around 80 hours of training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06540v1",
            "date": "2016-10-20 19:00:48+00:00",
            "title": "Jointly Learning to Align and Convert Graphemes to Phonemes with Neural Attention Models",
            "authors": [
                "Shubham Toshniwal",
                "Karen Livescu"
            ],
            "abstract": "We propose an attention-enabled encoder-decoder model for the problem of\ngrapheme-to-phoneme conversion. Most previous work has tackled the problem via\njoint sequence models that require explicit alignments for training. In\ncontrast, the attention-enabled encoder-decoder model allows for jointly\nlearning to align and convert characters to phonemes. We explore different\ntypes of attention models, including global and local attention, and our best\nmodels achieve state-of-the-art results on three standard data sets (CMUDict,\nPronlex, and NetTalk).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07708v2",
            "date": "2016-10-25 02:13:53+00:00",
            "title": "Knowledge will Propel Machine Understanding of Content: Extrapolating from Current Examples",
            "authors": [
                "Amit Sheth",
                "Sujan Perera",
                "Sanjaya Wijeratne"
            ],
            "abstract": "Machine Learning has been a big success story during the AI resurgence. One\nparticular stand out success relates to unsupervised learning from a massive\namount of data, albeit much of it relates to one modality/type of data at a\ntime. In spite of early assertions of the unreasonable effectiveness of data,\nthere is increasing recognition of utilizing knowledge whenever it is available\nor can be created purposefully. In this paper, we focus on discussing the\nindispensable role of knowledge for deeper understanding of complex text and\nmultimodal data in situations where (i) large amounts of training data\n(labeled/unlabeled) are not available or labor intensive to create, (ii) the\nobjects (particularly text) to be recognized are complex (i.e., beyond simple\nentity-person/location/organization names), such as implicit entities and\nhighly subjective content, and (iii) applications need to use complementary or\nrelated data in multiple modalities/media. What brings us to the cusp of rapid\nprogress is our ability to (a) create knowledge, varying from comprehensive or\ncross domain to domain or application specific, and (b) carefully exploit the\nknowledge to further empower or extend the applications of ML/NLP techniques.\nUsing the early results in several diverse situations - both in data types and\napplications - we seek to foretell unprecedented progress in our ability for\ndeeper understanding and exploitation of multimodal data.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.08078v1",
            "date": "2016-10-25 20:19:35+00:00",
            "title": "Dis-S2V: Discourse Informed Sen2Vec",
            "authors": [
                "Tanay Kumar Saha",
                "Shafiq Joty",
                "Naeemul Hassan",
                "Mohammad Al Hasan"
            ],
            "abstract": "Vector representation of sentences is important for many text processing\ntasks that involve clustering, classifying, or ranking sentences. Recently,\ndistributed representation of sentences learned by neural models from unlabeled\ndata has been shown to outperform the traditional bag-of-words representation.\nHowever, most of these learning methods consider only the content of a sentence\nand disregard the relations among sentences in a discourse by and large.\n  In this paper, we propose a series of novel models for learning latent\nrepresentations of sentences (Sen2Vec) that consider the content of a sentence\nas well as inter-sentence relations. We first represent the inter-sentence\nrelations with a language network and then use the network to induce contextual\ninformation into the content-based Sen2Vec models. Two different approaches are\nintroduced to exploit the information in the network. Our first approach\nretrofits (already trained) Sen2Vec vectors with respect to the network in two\ndifferent ways: (1) using the adjacency relations of a node, and (2) using a\nstochastic sampling method which is more flexible in sampling neighbors of a\nnode. The second approach uses a regularizer to encode the information in the\nnetwork into the existing Sen2Vec model. Experimental results show that our\nproposed models outperform existing methods in three fundamental information\nsystem tasks demonstrating the effectiveness of our approach. The models\nleverage the computational power of multi-core CPUs to achieve fine-grained\ncomputational efficiency. We make our code publicly available upon acceptance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08095v1",
            "date": "2016-10-25 21:08:15+00:00",
            "title": "Modeling Ambiguity, Subjectivity, and Diverging Viewpoints in Opinion Question Answering Systems",
            "authors": [
                "Mengting Wan",
                "Julian McAuley"
            ],
            "abstract": "Product review websites provide an incredible lens into the wide variety of\nopinions and experiences of different people, and play a critical role in\nhelping users discover products that match their personal needs and\npreferences. To help address questions that can't easily be answered by reading\nothers' reviews, some review websites also allow users to pose questions to the\ncommunity via a question-answering (QA) system. As one would expect, just as\nopinions diverge among different reviewers, answers to such questions may also\nbe subjective, opinionated, and divergent. This means that answering such\nquestions automatically is quite different from traditional QA tasks, where it\nis assumed that a single `correct' answer is available. While recent work\nintroduced the idea of question-answering using product reviews, it did not\naccount for two aspects that we consider in this paper: (1) Questions have\nmultiple, often divergent, answers, and this full spectrum of answers should\nsomehow be used to train the system; and (2) What makes a `good' answer depends\non the asker and the answerer, and these factors should be incorporated in\norder for the system to be more personalized. Here we build a new QA dataset\nwith 800 thousand questions---and over 3.1 million answers---and show that\nexplicitly accounting for personalization and ambiguity leads both to\nquantitatively better answers, but also a more nuanced view of the range of\nsupporting, but subjective, opinions.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1610.08229v1",
            "date": "2016-10-26 08:48:10+00:00",
            "title": "Word Embeddings and Their Use In Sentence Classification Tasks",
            "authors": [
                "Amit Mandelbaum",
                "Adi Shalev"
            ],
            "abstract": "This paper have two parts. In the first part we discuss word embeddings. We\ndiscuss the need for them, some of the methods to create them, and some of\ntheir interesting properties. We also compare them to image embeddings and see\nhow word embedding and image embedding can be combined to perform different\ntasks. In the second part we implement a convolutional neural network trained\non top of pre-trained word vectors. The network is used for several\nsentence-level classification tasks, and achieves state-of-art (or comparable)\nresults, demonstrating the great power of pre-trainted word embeddings over\nrandom ones.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1610.08613v2",
            "date": "2016-10-27 04:28:29+00:00",
            "title": "Can Active Memory Replace Attention?",
            "authors": [
                "\u0141ukasz Kaiser",
                "Samy Bengio"
            ],
            "abstract": "Several mechanisms to focus attention of a neural network on selected parts\nof its input or memory have been used successfully in deep learning models in\nrecent years. Attention has improved image classification, image captioning,\nspeech recognition, generative models, and learning algorithmic tasks, but it\nhad probably the largest impact on neural machine translation.\n  Recently, similar improvements have been obtained using alternative\nmechanisms that do not focus on a single part of a memory but operate on all of\nit in parallel, in a uniform way. Such mechanism, which we call active memory,\nimproved over attention in algorithmic tasks, image processing, and in\ngenerative modelling.\n  So far, however, active memory has not improved over attention for most\nnatural language processing tasks, in particular for machine translation. We\nanalyze this shortcoming in this paper and propose an extended model of active\nmemory that matches existing attention models on neural machine translation and\ngeneralizes better to longer sentences. We investigate this model and explain\nwhy previous active memory models did not succeed. Finally, we discuss when\nactive memory brings most benefits and where attention can be a better choice.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1610.08763v2",
            "date": "2016-10-27 13:20:25+00:00",
            "title": "CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases",
            "authors": [
                "Xiang Ren",
                "Zeqiu Wu",
                "Wenqi He",
                "Meng Qu",
                "Clare R. Voss",
                "Heng Ji",
                "Tarek F. Abdelzaher",
                "Jiawei Han"
            ],
            "abstract": "Extracting entities and relations for types of interest from text is\nimportant for understanding massive text corpora. Traditionally, systems of\nentity relation extraction have relied on human-annotated corpora for training\nand adopted an incremental pipeline. Such systems require additional human\nexpertise to be ported to a new domain, and are vulnerable to errors cascading\ndown the pipeline. In this paper, we investigate joint extraction of typed\nentities and relations with labeled data heuristically obtained from knowledge\nbases (i.e., distant supervision). As our algorithm for type labeling via\ndistant supervision is context-agnostic, noisy training data poses unique\nchallenges for the task. We propose a novel domain-independent framework,\ncalled CoType, that runs a data-driven text segmentation algorithm to extract\nentity mentions, and jointly embeds entity mentions, relation mentions, text\nfeatures and type labels into two low-dimensional spaces (for entity and\nrelation mentions respectively), where, in each space, objects whose types are\nclose will also have similar representations. CoType, then using these learned\nembeddings, estimates the types of test (unlinkable) mentions. We formulate a\njoint optimization problem to learn embeddings from text corpora and knowledge\nbases, adopting a novel partial-label loss function for noisy labeled data and\nintroducing an object \"translation\" function to capture the cross-constraints\nof entities and relations on each other. Experiments on three public datasets\ndemonstrate the effectiveness of CoType across different domains (e.g., news,\nbiomedical), with an average of 25% improvement in F1 score compared to the\nnext best method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09158v1",
            "date": "2016-10-28 10:26:40+00:00",
            "title": "Towards a continuous modeling of natural language domains",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "Humans continuously adapt their style and language to a variety of domains.\nHowever, a reliable definition of `domain' has eluded researchers thus far.\nAdditionally, the notion of discrete domains stands in contrast to the\nmultiplicity of heterogeneous domains that humans navigate, many of which\noverlap. In order to better understand the change and variation of human\nlanguage, we draw on research in domain adaptation and extend the notion of\ndiscrete domains to the continuous spectrum. We propose representation\nlearning-based models that can adapt to continuous domains and detail how these\ncan be used to investigate variation in language. To this end, we propose to\nuse dialogue modeling as a test bed due to its proximity to language modeling\nand its social component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09226v1",
            "date": "2016-10-28 14:09:33+00:00",
            "title": "Text Segmentation using Named Entity Recognition and Co-reference Resolution in English and Greek Texts",
            "authors": [
                "Pavlina Fragkou"
            ],
            "abstract": "In this paper we examine the benefit of performing named entity recognition\n(NER) and co-reference resolution to an English and a Greek corpus used for\ntext segmentation. The aim here is to examine whether the combination of text\nsegmentation and information extraction can be beneficial for the\nidentification of the various topics that appear in a document. NER was\nperformed manually in the English corpus and was compared with the output\nproduced by publicly available annotation tools while, an already existing tool\nwas used for the Greek corpus. Produced annotations from both corpora were\nmanually corrected and enriched to cover four types of named entities.\nCo-reference resolution i.e., substitution of every reference of the same\ninstance with the same named entity identifier was subsequently performed. The\nevaluation, using five text segmentation algorithms for the English corpus and\nfour for the Greek corpus leads to the conclusion that, the benefit highly\ndepends on the segment's topic, the number of named entity instances appearing\nin it, as well as the segment's length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09756v2",
            "date": "2016-10-31 01:31:52+00:00",
            "title": "Towards Deep Learning in Hindi NER: An approach to tackle the Labelled Data Scarcity",
            "authors": [
                "Vinayak Athavale",
                "Shreenivas Bharadwaj",
                "Monik Pamecha",
                "Ameya Prabhu",
                "Manish Shrivastava"
            ],
            "abstract": "In this paper we describe an end to end Neural Model for Named Entity\nRecognition NER) which is based on Bi-Directional RNN-LSTM. Almost all NER\nsystems for Hindi use Language Specific features and handcrafted rules with\ngazetteers. Our model is language independent and uses no domain specific\nfeatures or any handcrafted rules. Our models rely on semantic information in\nthe form of word vectors which are learnt by an unsupervised learning algorithm\non an unannotated corpus. Our model attained state of the art performance in\nboth English and Hindi without the use of any morphological analysis or without\nusing gazetteers of any sort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09889v2",
            "date": "2016-10-31 12:16:39+00:00",
            "title": "Chinese Poetry Generation with Planning based Neural Network",
            "authors": [
                "Zhe Wang",
                "Wei He",
                "Hua Wu",
                "Haiyang Wu",
                "Wei Li",
                "Haifeng Wang",
                "Enhong Chen"
            ],
            "abstract": "Chinese poetry generation is a very challenging task in natural language\nprocessing. In this paper, we propose a novel two-stage poetry generating\nmethod which first plans the sub-topics of the poem according to the user's\nwriting intent, and then generates each line of the poem sequentially, using a\nmodified recurrent neural network encoder-decoder framework. The proposed\nplanning-based method can ensure that the generated poem is coherent and\nsemantically consistent with the user's intent. A comprehensive evaluation with\nhuman judgments demonstrates that our proposed approach outperforms the\nstate-of-the-art poetry generating methods and the poem quality is somehow\ncomparable to human poets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09893v1",
            "date": "2016-10-31 12:24:13+00:00",
            "title": "LightRNN: Memory and Computation-Efficient Recurrent Neural Networks",
            "authors": [
                "Xiang Li",
                "Tao Qin",
                "Jian Yang",
                "Tie-Yan Liu"
            ],
            "abstract": "Recurrent neural networks (RNNs) have achieved state-of-the-art performances\nin many natural language processing tasks, such as language modeling and\nmachine translation. However, when the vocabulary is large, the RNN model will\nbecome very big (e.g., possibly beyond the memory capacity of a GPU device) and\nits training will become very inefficient. In this work, we propose a novel\ntechnique to tackle this challenge. The key idea is to use 2-Component (2C)\nshared embedding for word representations. We allocate every word in the\nvocabulary into a table, each row of which is associated with a vector, and\neach column associated with another vector. Depending on its position in the\ntable, a word is jointly represented by two components: a row vector and a\ncolumn vector. Since the words in the same row share the row vector and the\nwords in the same column share the column vector, we only need $2 \\sqrt{|V|}$\nvectors to represent a vocabulary of $|V|$ unique words, which are far less\nthan the $|V|$ vectors required by existing approaches. Based on the\n2-Component shared embedding, we design a new RNN algorithm and evaluate it\nusing the language modeling task on several benchmark datasets. The results\nshow that our algorithm significantly reduces the model size and speeds up the\ntraining process, without sacrifice of accuracy (it achieves similar, if not\nbetter, perplexity as compared to state-of-the-art language models).\nRemarkably, on the One-Billion-Word benchmark Dataset, our algorithm achieves\ncomparable perplexity to previous language models, whilst reducing the model\nsize by a factor of 40-100, and speeding up the training process by a factor of\n2. We name our proposed algorithm \\emph{LightRNN} to reflect its very small\nmodel size and very high training speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09964v1",
            "date": "2016-10-31 15:20:14+00:00",
            "title": "Ontology Verbalization using Semantic-Refinement",
            "authors": [
                "Vinu E. V",
                "P Sreenivasa Kumar"
            ],
            "abstract": "We propose a rule-based technique to generate redundancy-free NL descriptions\nof OWL entities.The existing approaches which address the problem of\nverbalizing OWL ontologies generate NL text segments which are close to their\ncounterpart OWL statements.Some of these approaches also perform grouping and\naggregating of these NL text segments to generate a more fluent and\ncomprehensive form of the content.Restricting our attention to description of\nindividuals and concepts, we find that the approach currently followed in the\navailable tools is that of determining the set of all logical conditions that\nare satisfied by the given individual/concept name and translate these\nconditions verbatim into corresponding NL descriptions.Human-understandability\nof such descriptions is affected by the presence of repetitions and\nredundancies, as they have high fidelity to their OWL representation.In the\nliterature, no efforts had been taken to remove redundancies and repetitions at\nthe logical-level before generating the NL descriptions of entities and we find\nthis to be the main reason for lack of readability of the generated\ntext.Herein, we propose a technique called semantic-refinement(SR) to generate\nmeaningful and easily-understandable descriptions of individuals and concepts\nof a given OWLontology.We identify the combinations of OWL/DL constructs that\nlead to repetitive/redundant descriptions and propose a series of refinement\nrules to rewrite the conditions that are satisfied by an individual/concept in\na meaning-preserving manner.The reduced set of conditions are then employed for\ngenerating NL descriptions.Our experiments show that, SR leads to significantly\nimproved descriptions of ontology entities.We also test the effectiveness and\nusefulness of the the generated descriptions for the purpose of validating the\nontologies and find that the proposed technique is indeed helpful in the\ncontext.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.09982v1",
            "date": "2016-10-31 15:45:41+00:00",
            "title": "Sentiment Analysis of Review Datasets Using Naive Bayes and K-NN Classifier",
            "authors": [
                "Lopamudra Dey",
                "Sanjay Chakraborty",
                "Anuraag Biswas",
                "Beepa Bose",
                "Sweta Tiwari"
            ],
            "abstract": "The advent of Web 2.0 has led to an increase in the amount of sentimental\ncontent available in the Web. Such content is often found in social media web\nsites in the form of movie or product reviews, user comments, testimonials,\nmessages in discussion forums etc. Timely discovery of the sentimental or\nopinionated web content has a number of advantages, the most important of all\nbeing monetization. Understanding of the sentiments of human masses towards\ndifferent entities and products enables better services for contextual\nadvertisements, recommendation systems and analysis of market trends. The focus\nof our project is sentiment focussed web crawling framework to facilitate the\nquick discovery of sentimental contents of movie reviews and hotel reviews and\nanalysis of the same. We use statistical methods to capture elements of\nsubjective style and the sentence polarity. The paper elaborately discusses two\nsupervised machine learning algorithms: K-Nearest Neighbour(K-NN) and Naive\nBayes and compares their overall accuracy, precisions as well as recall values.\nIt was seen that in case of movie reviews Naive Bayes gave far better results\nthan K-NN but for hotel reviews these algorithms gave lesser, almost same\naccuracies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1610.10099v2",
            "date": "2016-10-31 19:56:39+00:00",
            "title": "Neural Machine Translation in Linear Time",
            "authors": [
                "Nal Kalchbrenner",
                "Lasse Espeholt",
                "Karen Simonyan",
                "Aaron van den Oord",
                "Alex Graves",
                "Koray Kavukcuoglu"
            ],
            "abstract": "We present a novel neural network for processing sequences. The ByteNet is a\none-dimensional convolutional neural network that is composed of two parts, one\nto encode the source sequence and the other to decode the target sequence. The\ntwo network parts are connected by stacking the decoder on top of the encoder\nand preserving the temporal resolution of the sequences. To address the\ndiffering lengths of the source and the target, we introduce an efficient\nmechanism by which the decoder is dynamically unfolded over the representation\nof the encoder. The ByteNet uses dilation in the convolutional layers to\nincrease its receptive field. The resulting network has two core properties: it\nruns in time that is linear in the length of the sequences and it sidesteps the\nneed for excessive memorization. The ByteNet decoder attains state-of-the-art\nperformance on character-level language modelling and outperforms the previous\nbest results obtained with recurrent networks. The ByteNet also achieves\nstate-of-the-art performance on character-to-character machine translation on\nthe English-to-German WMT translation task, surpassing comparable neural\ntranslation models that are based on recurrent networks with attentional\npooling and run in quadratic time. We find that the latent alignment structure\ncontained in the representations reflects the expected alignment between the\ntokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00148v1",
            "date": "2016-12-01 05:08:53+00:00",
            "title": "Domain Adaptation for Named Entity Recognition in Online Media with Word Embeddings",
            "authors": [
                "Vivek Kulkarni",
                "Yashar Mehdad",
                "Troy Chevalier"
            ],
            "abstract": "Content on the Internet is heterogeneous and arises from various domains like\nNews, Entertainment, Finance and Technology. Understanding such content\nrequires identifying named entities (persons, places and organizations) as one\nof the key steps. Traditionally Named Entity Recognition (NER) systems have\nbeen built using available annotated datasets (like CoNLL, MUC) and demonstrate\nexcellent performance. However, these models fail to generalize onto other\ndomains like Sports and Finance where conventions and language use can differ\nsignificantly. Furthermore, several domains do not have large amounts of\nannotated labeled data for training robust Named Entity Recognition models. A\nkey step towards this challenge is to adapt models learned on domains where\nlarge amounts of annotated training data are available to domains with scarce\nannotated data.\n  In this paper, we propose methods to effectively adapt models learned on one\ndomain onto other domains using distributed word representations. First we\nanalyze the linguistic variation present across domains to identify key\nlinguistic insights that can boost performance across domains. We propose\nmethods to capture domain specific semantics of word usage in addition to\nglobal semantics. We then demonstrate how to effectively use such domain\nspecific knowledge to learn NER models that outperform previous baselines in\nthe domain adaptation setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00227v1",
            "date": "2016-12-01 12:58:02+00:00",
            "title": "On Coreferring Text-extracted Event Descriptions with the aid of Ontological Reasoning",
            "authors": [
                "Stefano Borgo",
                "Loris Bozzato",
                "Alessio Palmero Aprosio",
                "Marco Rospocher",
                "Luciano Serafini"
            ],
            "abstract": "Systems for automatic extraction of semantic information about events from\nlarge textual resources are now available: these tools are capable to generate\nRDF datasets about text extracted events and this knowledge can be used to\nreason over the recognized events. On the other hand, text based tasks for\nevent recognition, as for example event coreference (i.e. recognizing whether\ntwo textual descriptions refer to the same event), do not take into account\nontological information of the extracted events in their process. In this\npaper, we propose a method to derive event coreference on text extracted event\ndata using semantic based rule reasoning. We demonstrate our method considering\na limited (yet representative) set of event types: we introduce a formal\nanalysis on their ontological properties and, on the base of this, we define a\nset of coreference criteria. We then implement these criteria as RDF-based\nreasoning rules to be applied on text extracted event data. We evaluate the\neffectiveness of our approach over a standard coreference benchmark dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.00370v4",
            "date": "2016-12-01 18:10:48+00:00",
            "title": "Improved Image Captioning via Policy Gradient optimization of SPIDEr",
            "authors": [
                "Siqi Liu",
                "Zhenhai Zhu",
                "Ning Ye",
                "Sergio Guadarrama",
                "Kevin Murphy"
            ],
            "abstract": "Current image captioning methods are usually trained via (penalized) maximum\nlikelihood estimation. However, the log-likelihood score of a caption does not\ncorrelate well with human assessments of quality. Standard syntactic evaluation\nmetrics, such as BLEU, METEOR and ROUGE, are also not well correlated. The\nnewer SPICE and CIDEr metrics are better correlated, but have traditionally\nbeen hard to optimize for. In this paper, we show how to use a policy gradient\n(PG) method to directly optimize a linear combination of SPICE and CIDEr (a\ncombination we call SPIDEr): the SPICE score ensures our captions are\nsemantically faithful to the image, while CIDEr score ensures our captions are\nsyntactically fluent. The PG method we propose improves on the prior MIXER\napproach, by using Monte Carlo rollouts instead of mixing MLE training with PG.\nWe show empirically that our algorithm leads to easier optimization and\nimproved results compared to MIXER. Finally, we show that using our PG method\nwe can optimize any of the metrics, including the proposed SPIDEr metric which\nresults in image captions that are strongly preferred by human raters compared\nto captions generated by the same model but trained to optimize MLE or the COCO\nmetrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.00385v2",
            "date": "2016-12-01 19:11:24+00:00",
            "title": "Temporal Attention-Gated Model for Robust Sequence Classification",
            "authors": [
                "Wenjie Pei",
                "Tadas Baltru\u0161aitis",
                "David M. J. Tax",
                "Louis-Philippe Morency"
            ],
            "abstract": "Typical techniques for sequence classification are designed for\nwell-segmented sequences which have been edited to remove noisy or irrelevant\nparts. Therefore, such methods cannot be easily applied on noisy sequences\nexpected in real-world applications. In this paper, we present the Temporal\nAttention-Gated Model (TAGM) which integrates ideas from attention models and\ngated recurrent networks to better deal with noisy or unsegmented sequences.\nSpecifically, we extend the concept of attention model to measure the relevance\nof each observation (time step) of a sequence. We then use a novel gated\nrecurrent network to learn the hidden representation for the final prediction.\nAn important advantage of our approach is interpretability since the temporal\nattention weights provide a meaningful value for the salience of each time step\nin the sequence. We demonstrate the merits of our TAGM approach, both for\nprediction accuracy and interpretability, on three different tasks: spoken\ndigit recognition, text-based sentiment analysis and visual event recognition.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.00913v2",
            "date": "2016-12-03 02:13:18+00:00",
            "title": "End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager",
            "authors": [
                "Xuesong Yang",
                "Yun-Nung Chen",
                "Dilek Hakkani-Tur",
                "Paul Crook",
                "Xiujun Li",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "Natural language understanding and dialogue policy learning are both\nessential in conversational systems that predict the next system actions in\nresponse to a current user utterance. Conventional approaches aggregate\nseparate models of natural language understanding (NLU) and system action\nprediction (SAP) as a pipeline that is sensitive to noisy outputs of\nerror-prone NLU. To address the issues, we propose an end-to-end deep recurrent\nneural network with limited contextual dialogue memory by jointly training NLU\nand SAP on DSTC4 multi-domain human-human dialogues. Experiments show that our\nproposed model significantly outperforms the state-of-the-art pipeline models\nfor both NLU and SAP, which indicates that our joint model is capable of\nmitigating the affects of noisy NLU outputs, and NLU model can be refined by\nerror flows backpropagating from the extra supervised signals of system\nactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01340v2",
            "date": "2016-12-05 13:18:48+00:00",
            "title": "We used Neural Networks to Detect Clickbaits: You won't believe what happened Next!",
            "authors": [
                "Ankesh Anand",
                "Tanmoy Chakraborty",
                "Noseong Park"
            ],
            "abstract": "Online content publishers often use catchy headlines for their articles in\norder to attract users to their websites. These headlines, popularly known as\nclickbaits, exploit a user's curiosity gap and lure them to click on links that\noften disappoint them. Existing methods for automatically detecting clickbaits\nrely on heavy feature engineering and domain knowledge. Here, we introduce a\nneural network architecture based on Recurrent Neural Networks for detecting\nclickbaits. Our model relies on distributed word representations learned from a\nlarge unannotated corpora, and character embeddings learned via Convolutional\nNeural Networks. Experimental results on a dataset of news headlines show that\nour model outperforms existing techniques for clickbait detection with an\naccuracy of 0.98 with F1-score of 0.98 and ROC-AUC of 0.99.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01892v1",
            "date": "2016-12-06 16:26:33+00:00",
            "title": "Cross-Lingual Predicate Mapping Between Linked Data Ontologies",
            "authors": [
                "Gautam Singh",
                "Saemi Jang",
                "Mun Y. Yi"
            ],
            "abstract": "Ontologies in different natural languages often differ in quality in terms of\nrichness of schema or richness of internal links. This difference is markedly\nvisible when comparing a rich English language ontology with a non-English\nlanguage counterpart. Discovering alignment between them is a useful endeavor\nas it serves as a starting point in bridging the disparity. In particular, our\nwork is motivated by the absence of inter-language links for predicates in the\nlocalised versions of DBpedia. In this paper, we propose and demonstrate an\nad-hoc system to find possible owl:equivalentProperty links between predicates\nin ontologies of different natural languages. We seek to achieve this mapping\nby using pre-existing inter-language links of the resources connected by the\ngiven predicate. Thus, our methodology stresses on semantic similarity rather\nthan lexical. Moreover, through an evaluation, we show that our system is\ncapable of outperforming a baseline system that is similar to the one used in\nrecent OAEI campaigns.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.03231v1",
            "date": "2016-12-10 00:32:28+00:00",
            "title": "A natural language interface to a graph-based bibliographic information retrieval system",
            "authors": [
                "Yongjun Zhu",
                "Erjia Yan",
                "Il-Yeol Song"
            ],
            "abstract": "With the ever-increasing scientific literature, there is a need on a natural\nlanguage interface to bibliographic information retrieval systems to retrieve\nrelated information effectively. In this paper, we propose a natural language\ninterface, NLI-GIBIR, to a graph-based bibliographic information retrieval\nsystem. In designing NLI-GIBIR, we developed a novel framework that can be\napplicable to graph-based bibliographic information retrieval systems. Our\nframework integrates algorithms/heuristics for interpreting and analyzing\nnatural language bibliographic queries. NLI-GIBIR allows users to search for a\nvariety of bibliographic data through natural language. A series of text- and\nlinguistic-based techniques are used to analyze and answer natural language\nqueries, including tokenization, named entity recognition, and syntactic\nanalysis. We find that our framework can effectively represents and addresses\ncomplex bibliographic information needs. Thus, the contributions of this paper\nare as follows: First, to our knowledge, it is the first attempt to propose a\nnatural language interface to graph-based bibliographic information retrieval.\nSecond, we propose a novel customized natural language processing framework\nthat integrates a few original algorithms/heuristics for interpreting and\nanalyzing natural language bibliographic queries. Third, we show that the\nproposed framework and natural language interface provide a practical solution\nin building real-world natural language interface-based bibliographic\ninformation retrieval systems. Our experimental results show that the presented\nsystem can correctly answer 39 out of 40 example natural language queries with\nvarying lengths and complexities.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.03277v1",
            "date": "2016-12-10 10:54:45+00:00",
            "title": "Data Curation APIs",
            "authors": [
                "Seyed-Mehdi-Reza Beheshti",
                "Alireza Tabebordbar",
                "Boualem Benatallah",
                "Reza Nouri"
            ],
            "abstract": "Understanding and analyzing big data is firmly recognized as a powerful and\nstrategic priority. For deeper interpretation of and better intelligence with\nbig data, it is important to transform raw data (unstructured, semi-structured\nand structured data sources, e.g., text, video, image data sets) into curated\ndata: contextualized data and knowledge that is maintained and made available\nfor use by end-users and applications. In particular, data curation acts as the\nglue between raw data and analytics, providing an abstraction layer that\nrelieves users from time consuming, tedious and error prone curation tasks. In\nthis context, the data curation process becomes a vital analytics asset for\nincreasing added value and insights.\n  In this paper, we identify and implement a set of curation APIs and make them\navailable (on GitHub) to researchers and developers to assist them transforming\ntheir raw data into curated data. The curation APIs enable developers to easily\nadd features - such as extracting keyword, part of speech, and named entities\nsuch as Persons, Locations, Organizations, Companies, Products, Diseases,\nDrugs, etc.; providing synonyms and stems for extracted information items\nleveraging lexical knowledge bases for the English language such as WordNet;\nlinking extracted entities to external knowledge bases such as Google Knowledge\nGraph and Wikidata; discovering similarity among the extracted information\nitems, such as calculating similarity between string, number, date and time\ndata; classifying, sorting and categorizing data into various types, forms or\nany other distinct class; and indexing structured and unstructured data - into\ntheir applications.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.03551v3",
            "date": "2016-12-12 06:19:32+00:00",
            "title": "Reading Comprehension using Entity-based Memory Network",
            "authors": [
                "Xun Wang",
                "Katsuhito Sudoh",
                "Masaaki Nagata",
                "Tomohide Shibata",
                "Daisuke Kawahara",
                "Sadao Kurohashi"
            ],
            "abstract": "This paper introduces a novel neural network model for question answering,\nthe \\emph{entity-based memory network}. It enhances neural networks' ability of\nrepresenting and calculating information over a long period by keeping records\nof entities contained in text. The core component is a memory pool which\ncomprises entities' states. These entities' states are continuously updated\naccording to the input text. Questions with regard to the input text are used\nto search the memory pool for related entities and answers are further\npredicted based on the states of retrieved entities. Compared with previous\nmemory network models, the proposed model is capable of handling fine-grained\ninformation and more sophisticated relations based on entities. We formulated\nseveral different tasks as question answering problems and tested the proposed\nmodel. Experiments reported satisfying results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03597v1",
            "date": "2016-12-12 10:27:31+00:00",
            "title": "Search Personalization with Embeddings",
            "authors": [
                "Thanh Vu",
                "Dat Quoc Nguyen",
                "Mark Johnson",
                "Dawei Song",
                "Alistair Willis"
            ],
            "abstract": "Recent research has shown that the performance of search personalization\ndepends on the richness of user profiles which normally represent the user's\ntopical interests. In this paper, we propose a new embedding approach to\nlearning user profiles, where users are embedded on a topical interest space.\nWe then directly utilize the user profiles for search personalization.\nExperiments on query logs from a major commercial web search engine demonstrate\nthat our embedding approach improves the performance of the search engine and\nalso achieves better search performance than other strong baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.03628v1",
            "date": "2016-12-12 11:41:46+00:00",
            "title": "VIBIKNet: Visual Bidirectional Kernelized Network for Visual Question Answering",
            "authors": [
                "Marc Bola\u00f1os",
                "\u00c1lvaro Peris",
                "Francisco Casacuberta",
                "Petia Radeva"
            ],
            "abstract": "In this paper, we address the problem of visual question answering by\nproposing a novel model, called VIBIKNet. Our model is based on integrating\nKernelized Convolutional Neural Networks and Long-Short Term Memory units to\ngenerate an answer given a question about an image. We prove that VIBIKNet is\nan optimal trade-off between accuracy and computational load, in terms of\nmemory and time consumption. We validate our method on the VQA challenge\ndataset and compare it to the top performing methods in order to illustrate its\nperformance and speed.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.03651v1",
            "date": "2016-12-12 12:51:03+00:00",
            "title": "FastText.zip: Compressing text classification models",
            "authors": [
                "Armand Joulin",
                "Edouard Grave",
                "Piotr Bojanowski",
                "Matthijs Douze",
                "H\u00e9rve J\u00e9gou",
                "Tomas Mikolov"
            ],
            "abstract": "We consider the problem of producing compact architectures for text\nclassification, such that the full model fits in a limited amount of memory.\nAfter considering different solutions inspired by the hashing literature, we\npropose a method built upon product quantization to store word embeddings.\nWhile the original technique leads to a loss in accuracy, we adapt this method\nto circumvent quantization artefacts. Our experiments carried out on several\nbenchmarks show that our approach typically requires two orders of magnitude\nless memory than fastText while being only slightly inferior with respect to\naccuracy. As a result, it outperforms the state of the art by a good margin in\nterms of the compromise between memory usage and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03769v1",
            "date": "2016-12-12 16:25:08+00:00",
            "title": "Context-aware Sentiment Word Identification: sentiword2vec",
            "authors": [
                "Yushi Yao",
                "Guangjian Li"
            ],
            "abstract": "Traditional sentiment analysis often uses sentiment dictionary to extract\nsentiment information in text and classify documents. However, emerging\ninformal words and phrases in user generated content call for analysis aware to\nthe context. Usually, they have special meanings in a particular context.\nBecause of its great performance in representing inter-word relation, we use\nsentiment word vectors to identify the special words. Based on the distributed\nlanguage model word2vec, in this paper we represent a novel method about\nsentiment representation of word under particular context, to be detailed, to\nidentify the words with abnormal sentiment polarity in long answers. Result\nshows the improved model shows better performance in representing the words\nwith special meaning, while keep doing well in representing special idiomatic\npattern. Finally, we will discuss the meaning of vectors representing in the\nfield of sentiment, which may be different from general object-based\nconditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04061v1",
            "date": "2016-12-13 08:32:02+00:00",
            "title": "Learning to Hash-tag Videos with Tag2Vec",
            "authors": [
                "Aditya Singh",
                "Saurabh Saini",
                "Rajvi Shah",
                "PJ Narayanan"
            ],
            "abstract": "User-given tags or labels are valuable resources for semantic understanding\nof visual media such as images and videos. Recently, a new type of labeling\nmechanism known as hash-tags have become increasingly popular on social media\nsites. In this paper, we study the problem of generating relevant and useful\nhash-tags for short video clips. Traditional data-driven approaches for tag\nenrichment and recommendation use direct visual similarity for label transfer\nand propagation. We attempt to learn a direct low-cost mapping from video to\nhash-tags using a two step training process. We first employ a natural language\nprocessing (NLP) technique, skip-gram models with neural network training to\nlearn a low-dimensional vector representation of hash-tags (Tag2Vec) using a\ncorpus of 10 million hash-tags. We then train an embedding function to map\nvideo features to the low-dimensional Tag2vec space. We learn this embedding\nfor 29 categories of short video clips with hash-tags. A query video without\nany tag-information can then be directly mapped to the vector space of tags\nusing the learned embedding and relevant tags can be found by performing a\nsimple nearest-neighbor retrieval in the Tag2Vec space. We validate the\nrelevance of the tags suggested by our system qualitatively and quantitatively\nwith a user study.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.04426v1",
            "date": "2016-12-13 23:09:49+00:00",
            "title": "Improving Neural Language Models with a Continuous Cache",
            "authors": [
                "Edouard Grave",
                "Armand Joulin",
                "Nicolas Usunier"
            ],
            "abstract": "We propose an extension to neural network language models to adapt their\nprediction to the recent history. Our model is a simplified version of memory\naugmented networks, which stores past hidden activations as memory and accesses\nthem through a dot product with the current hidden activation. This mechanism\nis very efficient and scales to very large memory sizes. We also draw a link\nbetween the use of external memory in neural network and cache models used with\ncount based language models. We demonstrate on several language model datasets\nthat our approach performs significantly better than recent memory augmented\nnetworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04675v2",
            "date": "2016-12-14 15:07:51+00:00",
            "title": "Recurrent Deep Stacking Networks for Speech Recognition",
            "authors": [
                "Peidong Wang",
                "Zhongqiu Wang",
                "Deliang Wang"
            ],
            "abstract": "This paper presented our work on applying Recurrent Deep Stacking Networks\n(RDSNs) to Robust Automatic Speech Recognition (ASR) tasks. In the paper, we\nalso proposed a more efficient yet comparable substitute to RDSN, Bi- Pass\nStacking Network (BPSN). The main idea of these two models is to add\nphoneme-level information into acoustic models, transforming an acoustic model\nto the combination of an acoustic model and a phoneme-level N-gram model.\nExperiments showed that RDSN and BPsn can substantially improve the\nperformances over conventional DNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04936v4",
            "date": "2016-12-15 05:46:27+00:00",
            "title": "Learning through Dialogue Interactions by Asking Questions",
            "authors": [
                "Jiwei Li",
                "Alexander H. Miller",
                "Sumit Chopra",
                "Marc'Aurelio Ranzato",
                "Jason Weston"
            ],
            "abstract": "A good dialogue agent should have the ability to interact with users by both\nresponding to questions and by asking questions, and importantly to learn from\nboth types of interaction. In this work, we explore this direction by designing\na simulator and a set of synthetic tasks in the movie domain that allow such\ninteractions between a learner and a teacher. We investigate how a learner can\nbenefit from asking questions in both offline and online reinforcement learning\nsettings, and demonstrate that the learner improves when asking questions.\nFinally, real experiments with Mechanical Turk validate the approach. Our work\nrepresents a first step in developing such end-to-end learned interactive\ndialogue agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04949v1",
            "date": "2016-12-15 07:19:46+00:00",
            "title": "Recurrent Image Captioner: Describing Images with Spatial-Invariant Transformation and Attention Filtering",
            "authors": [
                "Hao Liu",
                "Yang Yang",
                "Fumin Shen",
                "Lixin Duan",
                "Heng Tao Shen"
            ],
            "abstract": "Along with the prosperity of recurrent neural network in modelling sequential\ndata and the power of attention mechanism in automatically identify salient\ninformation, image captioning, a.k.a., image description, has been remarkably\nadvanced in recent years. Nonetheless, most existing paradigms may suffer from\nthe deficiency of invariance to images with different scaling, rotation, etc.;\nand effective integration of standalone attention to form a holistic end-to-end\nsystem. In this paper, we propose a novel image captioning architecture, termed\nRecurrent Image Captioner (\\textbf{RIC}), which allows visual encoder and\nlanguage decoder to coherently cooperate in a recurrent manner. Specifically,\nwe first equip CNN-based visual encoder with a differentiable layer to enable\nspatially invariant transformation of visual signals. Moreover, we deploy an\nattention filter module (differentiable) between encoder and decoder to\ndynamically determine salient visual parts. We also employ bidirectional LSTM\nto preprocess sentences for generating better textual representations. Besides,\nwe propose to exploit variational inference to optimize the whole architecture.\nExtensive experimental results on three benchmark datasets (i.e., Flickr8k,\nFlickr30k and MS COCO) demonstrate the superiority of our proposed architecture\nas compared to most of the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.04988v1",
            "date": "2016-12-15 09:14:35+00:00",
            "title": "TeKnowbase: Towards Construction of a Knowledge-base of Technical Concepts",
            "authors": [
                "Prajna Upadhyay",
                "Tanuma Patra",
                "Ashwini Purkar",
                "Maya Ramanath"
            ],
            "abstract": "In this paper, we describe the construction of TeKnowbase, a knowledge-base\nof technical concepts in computer science. Our main information sources are\ntechnical websites such as Webopedia and Techtarget as well as Wikipedia and\nonline textbooks. We divide the knowledge-base construction problem into two\nparts -- the acquisition of entities and the extraction of relationships among\nthese entities. Our knowledge-base consists of approximately 100,000 triples.\nWe conducted an evaluation on a sample of triples and report an accuracy of a\nlittle over 90\\%. We additionally conducted classification experiments on\nStackOverflow data with features from TeKnowbase and achieved improved\nclassification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05348v1",
            "date": "2016-12-16 03:33:07+00:00",
            "title": "Machine Reading with Background Knowledge",
            "authors": [
                "Ndapandula Nakashole",
                "Tom M. Mitchell"
            ],
            "abstract": "Intelligent systems capable of automatically understanding natural language\ntext are important for many artificial intelligence applications including\nmobile phone voice assistants, computer vision, and robotics. Understanding\nlanguage often constitutes fitting new information into a previously acquired\nview of the world. However, many machine reading systems rely on the text alone\nto infer its meaning. In this paper, we pursue a different approach; machine\nreading methods that make use of background knowledge to facilitate language\nunderstanding. To this end, we have developed two methods: The first method\naddresses prepositional phrase attachment ambiguity. It uses background\nknowledge within a semi-supervised machine learning algorithm that learns from\nboth labeled and unlabeled data. This approach yields state-of-the-art results\non two datasets against strong baselines; The second method extracts\nrelationships from compound nouns. Our knowledge-aware method for compound noun\nanalysis accurately extracts relationships and significantly outperforms a\nbaseline that does not make use of background knowledge.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.06043v4",
            "date": "2016-12-19 04:23:22+00:00",
            "title": "An Empirical Study of Adequate Vision Span for Attention-Based Neural Machine Translation",
            "authors": [
                "Raphael Shu",
                "Hideki Nakayama"
            ],
            "abstract": "Recently, the attention mechanism plays a key role to achieve high\nperformance for Neural Machine Translation models. However, as it computes a\nscore function for the encoder states in all positions at each decoding step,\nthe attention model greatly increases the computational complexity. In this\npaper, we investigate the adequate vision span of attention models in the\ncontext of machine translation, by proposing a novel attention framework that\nis capable of reducing redundant score computation dynamically. The term\n\"vision span\" means a window of the encoder states considered by the attention\nmodel in one step. In our experiments, we found that the average window size of\nvision span can be reduced by over 50% with modest loss in accuracy on\nEnglish-Japanese and German-English translation tasks.% This results indicate\nthat the conventional attention mechanism performs a significant amount of\nredundant computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06062v1",
            "date": "2016-12-19 07:06:34+00:00",
            "title": "Improving Tweet Representations using Temporal and User Context",
            "authors": [
                "Ganesh J",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "In this work we propose a novel representation learning model which computes\nsemantic representations for tweets accurately. Our model systematically\nexploits the chronologically adjacent tweets ('context') from users' Twitter\ntimelines for this task. Further, we make our model user-aware so that it can\ndo well in modeling the target tweet by exploiting the rich knowledge about the\nuser such as the way the user writes the post and also summarizing the topics\non which the user writes. We empirically demonstrate that the proposed models\noutperform the state-of-the-art models in predicting the user profile\nattributes like spouse, education and job by 19.66%, 2.27% and 2.22%\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06530v2",
            "date": "2016-12-20 07:20:16+00:00",
            "title": "Automatic Generation of Grounded Visual Questions",
            "authors": [
                "Shijie Zhang",
                "Lizhen Qu",
                "Shaodi You",
                "Zhenglu Yang",
                "Jiawan Zhang"
            ],
            "abstract": "In this paper, we propose the first model to be able to generate visually\ngrounded questions with diverse types for a single image. Visual question\ngeneration is an emerging topic which aims to ask questions in natural language\nbased on visual input. To the best of our knowledge, it lacks automatic methods\nto generate meaningful questions with various types for the same visual input.\nTo circumvent the problem, we propose a model that automatically generates\nvisually grounded questions with varying types. Our model takes as input both\nimages and the captions generated by a dense caption model, samples the most\nprobable question types, and generates the questions in sequel. The\nexperimental results on two real world datasets show that our model outperforms\nthe strongest baseline in terms of both correctness and diversity with a wide\nmargin.",
            "journal ref": "IJCAI 2017",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.07040v1",
            "date": "2016-12-21 10:09:30+00:00",
            "title": "A deep learning approach for predicting the quality of online health expert question-answering services",
            "authors": [
                "Ze Hu",
                "Zhan Zhang",
                "Qing Chen",
                "Haiqin Yang",
                "Decheng Zuo"
            ],
            "abstract": "Currently, a growing number of health consumers are asking health-related\nquestions online, at any time and from anywhere, which effectively lowers the\ncost of health care. The most common approach is using online health expert\nquestion-answering (HQA) services, as health consumers are more willing to\ntrust answers from professional physicians. However, these answers can be of\nvarying quality depending on circumstance. In addition, as the available HQA\nservices grow, how to predict the answer quality of HQA services via machine\nlearning becomes increasingly important and challenging. In an HQA service,\nanswers are normally short texts, which are severely affected by the data\nsparsity problem. Furthermore, HQA services lack community features such as\nbest answer and user votes. Therefore, the wisdom of the crowd is not available\nto rate answer quality. To address these problems, in this paper, the\nprediction of HQA answer quality is defined as a classification task. First,\nbased on the characteristics of HQA services and feedback from medical experts,\na standard for HQA service answer quality evaluation is defined. Next, based on\nthe characteristics of HQA services, several novel non-textual features are\nproposed, including surface linguistic features and social features. Finally, a\ndeep belief network (DBN)-based HQA answer quality prediction framework is\nproposed to predict the quality of answers by learning the high-level hidden\nsemantic representation from the physicians' answers. Our results prove that\nthe proposed framework overcomes the problem of overly sparse textual features\nin short text answers and effectively identifies high-quality answers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.07411v2",
            "date": "2016-12-22 01:25:20+00:00",
            "title": "A Context-aware Attention Network for Interactive Question Answering",
            "authors": [
                "Huayu Li",
                "Martin Renqiang Min",
                "Yong Ge",
                "Asim Kadav"
            ],
            "abstract": "Neural network based sequence-to-sequence models in an encoder-decoder\nframework have been successfully applied to solve Question Answering (QA)\nproblems, predicting answers from statements and questions. However, almost all\nprevious models have failed to consider detailed context information and\nunknown states under which systems do not have enough information to answer\ngiven questions. These scenarios with incomplete or ambiguous information are\nvery common in the setting of Interactive Question Answering (IQA). To address\nthis challenge, we develop a novel model, employing context-dependent\nword-level attention for more accurate statement representations and\nquestion-guided sentence-level attention for better context modeling. We also\ngenerate unique IQA datasets to test our model, which will be made publicly\navailable. Employing these attention mechanisms, our model accurately\nunderstands when it can output an answer or when it requires generating a\nsupplementary question for additional input depending on different contexts.\nWhen available, user's feedback is encoded and directly applied to update\nsentence-level attention to infer an answer. Extensive experiments on QA and\nIQA datasets quantitatively demonstrate the effectiveness of our model with\nsignificant improvement over state-of-the-art conventional QA models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07600v1",
            "date": "2016-12-22 14:00:28+00:00",
            "title": "Re-evaluating Automatic Metrics for Image Captioning",
            "authors": [
                "Mert Kilickaya",
                "Aykut Erdem",
                "Nazli Ikizler-Cinbis",
                "Erkut Erdem"
            ],
            "abstract": "The task of generating natural language descriptions from images has received\na lot of attention in recent years. Consequently, it is becoming increasingly\nimportant to evaluate such image captioning approaches in an automatic manner.\nIn this paper, we provide an in-depth evaluation of the existing image\ncaptioning metrics through a series of carefully designed experiments.\nMoreover, we explore the utilization of the recently proposed Word Mover's\nDistance (WMD) document metric for the purpose of image captioning. Our\nfindings outline the differences and/or similarities between metrics and their\nrelative robustness by means of extensive correlation, accuracy and distraction\nbased evaluations. Our results also demonstrate that WMD provides strong\nadvantages over other metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07602v4",
            "date": "2016-12-22 14:08:08+00:00",
            "title": "Jointly Extracting Relations with Class Ties via Effective Deep Ranking",
            "authors": [
                "Hai Ye",
                "Wenhan Chao",
                "Zhunchen Luo",
                "Zhoujun Li"
            ],
            "abstract": "Connections between relations in relation extraction, which we call class\nties, are common. In distantly supervised scenario, one entity tuple may have\nmultiple relation facts. Exploiting class ties between relations of one entity\ntuple will be promising for distantly supervised relation extraction. However,\nprevious models are not effective or ignore to model this property. In this\nwork, to effectively leverage class ties, we propose to make joint relation\nextraction with a unified model that integrates convolutional neural network\n(CNN) with a general pairwise ranking framework, in which three novel ranking\nloss functions are introduced. Additionally, an effective method is presented\nto relieve the severe class imbalance problem from NR (not relation) for model\ntraining. Experiments on a widely used dataset show that leveraging class ties\nwill enhance extraction and demonstrate the effectiveness of our model to learn\nclass ties. Our model outperforms the baselines significantly, achieving\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.07833v1",
            "date": "2016-12-22 22:44:17+00:00",
            "title": "Understanding Image and Text Simultaneously: a Dual Vision-Language Machine Comprehension Task",
            "authors": [
                "Nan Ding",
                "Sebastian Goodman",
                "Fei Sha",
                "Radu Soricut"
            ],
            "abstract": "We introduce a new multi-modal task for computer systems, posed as a combined\nvision-language comprehension challenge: identifying the most suitable text\ndescribing a scene, given several similar options. Accomplishing the task\nentails demonstrating comprehension beyond just recognizing \"keywords\" (or\nkey-phrases) and their corresponding visual concepts. Instead, it requires an\nalignment between the representations of the two modalities that achieves a\nvisually-grounded \"understanding\" of various linguistic elements and their\ndependencies. This new task also admits an easy-to-compute and well-studied\nmetric: the accuracy in detecting the true target among the decoys.\n  The paper makes several contributions: an effective and extensible mechanism\nfor generating decoys from (human-created) image captions; an instance of\napplying this mechanism, yielding a large-scale machine comprehension dataset\n(based on the COCO images and captions) that we make publicly available; human\nevaluation results on this dataset, informing a performance upper-bound; and\nseveral baseline and competitive learning approaches that illustrate the\nutility of the proposed task and dataset in advancing both image and language\ncomprehension. We also show that, in a multi-task learning setting, the\nperformance on the proposed task is positively correlated with the end-to-end\ntask of image captioning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.07940v1",
            "date": "2016-12-23 11:32:37+00:00",
            "title": "Supervised Opinion Aspect Extraction by Exploiting Past Extraction Results",
            "authors": [
                "Lei Shu",
                "Bing Liu",
                "Hu Xu",
                "Annice Kim"
            ],
            "abstract": "One of the key tasks of sentiment analysis of product reviews is to extract\nproduct aspects or features that users have expressed opinions on. In this\nwork, we focus on using supervised sequence labeling as the base approach to\nperforming the task. Although several extraction methods using sequence\nlabeling methods such as Conditional Random Fields (CRF) and Hidden Markov\nModels (HMM) have been proposed, we show that this supervised approach can be\nsignificantly improved by exploiting the idea of concept sharing across\nmultiple domains. For example, \"screen\" is an aspect in iPhone, but not only\niPhone has a screen, many electronic devices have screens too. When \"screen\"\nappears in a review of a new domain (or product), it is likely to be an aspect\ntoo. Knowing this information enables us to do much better extraction in the\nnew domain. This paper proposes a novel extraction method exploiting this idea\nin the context of supervised sequence labeling. Experimental results show that\nit produces markedly better results than without using the past information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08178v1",
            "date": "2016-12-24 13:30:23+00:00",
            "title": "JU_KS_Group@FIRE 2016: Consumer Health Information Search",
            "authors": [
                "Kamal Sarkar",
                "Debanjan Das",
                "Indra Banerjee",
                "Mamta Kumari",
                "Prasenjit Biswas"
            ],
            "abstract": "In this paper, we describe the methodology used and the results obtained by\nus for completing the tasks given under the shared task on Consumer Health\nInformation Search (CHIS) collocated with the Forum for Information Retrieval\nEvaluation (FIRE) 2016, ISI Kolkata. The shared task consists of two sub-tasks\n- (1) task1: given a query and a document/set of documents associated with that\nquery, the task is to classify the sentences in the document as relevant to the\nquery or not and (2) task 2: the relevant sentences need to be further\nclassified as supporting the claim made in the query, or opposing the claim\nmade in the query. We have participated in both the sub-tasks. The percentage\naccuracy obtained by our developed system for task1 was 73.39 which is third\nhighest among the 9 teams participated in the shared task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.08205v1",
            "date": "2016-12-24 17:09:21+00:00",
            "title": "Predicting the Industry of Users on Social Media",
            "authors": [
                "Konstantinos Pappas",
                "Rada Mihalcea"
            ],
            "abstract": "Automatic profiling of social media users is an important task for supporting\na multitude of downstream applications. While a number of studies have used\nsocial media content to extract and study collective social attributes, there\nis a lack of substantial research that addresses the detection of a user's\nindustry. We frame this task as classification using both feature engineering\nand ensemble learning. Our industry-detection system uses both posted content\nand profile information to detect a user's industry with 64.3% accuracy,\nsignificantly outperforming the majority baseline in a taxonomy of fourteen\nindustry classes. Our qualitative analysis suggests that a person's industry\nnot only affects the words used and their perceived meanings, but also the\nnumber and type of emotions being expressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08504v1",
            "date": "2016-12-27 05:59:55+00:00",
            "title": "Classifying Patents Based on their Semantic Content",
            "authors": [
                "Antonin Bergeaud",
                "Yoann Potiron",
                "Juste Raimbault"
            ],
            "abstract": "In this paper, we extend some usual techniques of classification resulting\nfrom a large-scale data-mining and network approach. This new technology, which\nin particular is designed to be suitable to big data, is used to construct an\nopen consolidated database from raw data on 4 million patents taken from the US\npatent office from 1976 onward. To build the pattern network, not only do we\nlook at each patent title, but we also examine their full abstract and extract\nthe relevant keywords accordingly. We refer to this classification as semantic\napproach in contrast with the more common technological approach which consists\nin taking the topology when considering US Patent office technological classes.\nMoreover, we document that both approaches have highly different topological\nmeasures and strong statistical evidence that they feature a different model.\nThis suggests that our method is a useful tool to extract endogenous\ninformation.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1612.09213v1",
            "date": "2016-12-29 17:36:54+00:00",
            "title": "Verifying Heaps' law using Google Books Ngram data",
            "authors": [
                "Vladimir V. Bochkarev",
                "Eduard Yu. Lerner",
                "Anna V. Shevlyakova"
            ],
            "abstract": "This article is devoted to the verification of the empirical Heaps law in\nEuropean languages using Google Books Ngram corpus data. The connection between\nword distribution frequency and expected dependence of individual word number\non text size is analysed in terms of a simple probability model of text\ngeneration. It is shown that the Heaps exponent varies significantly within\ncharacteristic time intervals of 60-100 years.",
            "journal ref": "Uchenye Zapiski Kazanskogo Universiteta. Seriya\n  Fiziko-Matematicheskie Nauki, 2013, vol. 155, no. 4, pp. 16-23. (In Russian)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.09327v1",
            "date": "2016-04-11 06:09:36+00:00",
            "title": "Intelligent information extraction based on artificial neural network",
            "authors": [
                "Ahlam Ansari",
                "Moonish Maknojia",
                "Altamash Shaikh"
            ],
            "abstract": "Question Answering System (QAS) is used for information retrieval and natural\nlanguage processing (NLP) to reduce human effort. There are numerous QAS based\non the user documents present today, but they all are limited to providing\nobjective answers and process simple questions only. Complex questions cannot\nbe answered by the existing QAS, as they require interpretation of the current\nand old data as well as the question asked by the user. The above limitations\ncan be overcome by using deep cases and neural network. Hence we propose a\nmodified QAS in which we create a deep artificial neural network with\nassociative memory from text documents. The modified QAS processes the contents\nof the text document provided to it and find the answer to even complex\nquestions in the documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.09535v1",
            "date": "2016-12-30 17:10:29+00:00",
            "title": "PAMPO: using pattern matching and pos-tagging for effective Named Entities recognition in Portuguese",
            "authors": [
                "Concei\u00e7\u00e3o Rocha",
                "Al\u00edpio Jorge",
                "Roberta Sionara",
                "Paula Brito",
                "Carlos Pimenta",
                "Solange Rezende"
            ],
            "abstract": "This paper deals with the entity extraction task (named entity recognition)\nof a text mining process that aims at unveiling non-trivial semantic\nstructures, such as relationships and interaction between entities or\ncommunities. In this paper we present a simple and efficient named entity\nextraction algorithm. The method, named PAMPO (PAttern Matching and POs tagging\nbased algorithm for NER), relies on flexible pattern matching, part-of-speech\ntagging and lexical-based rules. It was developed to process texts written in\nPortuguese, however it is potentially applicable to other languages as well.\n  We compare our approach with current alternatives that support Named Entity\nRecognition (NER) for content written in Portuguese. These are Alchemy, Zemanta\nand Rembrandt. Evaluation of the efficacy of the entity extraction method on\nseveral texts written in Portuguese indicates a considerable improvement on\n$recall$ and $F_1$ measures.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1603.00260v1",
            "date": "2016-03-01 13:14:33+00:00",
            "title": "Event Search and Analytics: Detecting Events in Semantically Annotated Corpora for Search and Analytics",
            "authors": [
                "Dhruv Gupta"
            ],
            "abstract": "In this article, I present the questions that I seek to answer in my PhD\nresearch. I posit to analyze natural language text with the help of semantic\nannotations and mine important events for navigating large text corpora.\nSemantic annotations such as named entities, geographic locations, and temporal\nexpressions can help us mine events from the given corpora. These events thus\nprovide us with useful means to discover the locked knowledge in them. I pose\nthree problems that can help unlock this knowledge vault in semantically\nannotated text corpora: i. identifying important events; ii. semantic search;\nand iii. event analytics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1603.00892v1",
            "date": "2016-03-02 21:19:36+00:00",
            "title": "Counter-fitting Word Vectors to Linguistic Constraints",
            "authors": [
                "Nikola Mrk\u0161i\u0107",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Blaise Thomson",
                "Milica Ga\u0161i\u0107",
                "Lina Rojas-Barahona",
                "Pei-Hao Su",
                "David Vandyke",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "In this work, we present a novel counter-fitting method which injects\nantonymy and synonymy constraints into vector space representations in order to\nimprove the vectors' capability for judging semantic similarity. Applying this\nmethod to publicly available pre-trained word vectors leads to a new state of\nthe art performance on the SimLex-999 dataset. We also show how the method can\nbe used to tailor the word vector space for the downstream task of dialogue\nstate tracking, resulting in robust improvements across different dialogue\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01520v3",
            "date": "2016-03-04 16:13:24+00:00",
            "title": "Optimized Polynomial Evaluation with Semantic Annotations",
            "authors": [
                "Daniel Rubio Bonilla",
                "Colin W. Glass",
                "Jan Kuper"
            ],
            "abstract": "In this paper we discuss how semantic annotations can be used to introduce\nmathematical algorithmic information of the underlying imperative code to\nenable compilers to produce code transformations that will enable better\nperformance. By using this approaches not only good performance is achieved,\nbut also better programmability, maintainability and portability across\ndifferent hardware architectures. To exemplify this we will use polynomial\nequations of different degrees.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01595v1",
            "date": "2016-03-04 20:04:31+00:00",
            "title": "Sentiment Analysis in Scholarly Book Reviews",
            "authors": [
                "Hussam Hamdan",
                "Patrice Bellot",
                "Frederic Bechet"
            ],
            "abstract": "So far different studies have tackled the sentiment analysis in several\ndomains such as restaurant and movie reviews. But, this problem has not been\nstudied in scholarly book reviews which is different in terms of review style\nand size. In this paper, we propose to combine different features in order to\nbe presented to a supervised classifiers which extract the opinion target\nexpressions and detect their polarities in scholarly book reviews. We construct\na labeled corpus for training and evaluating our methods in French book\nreviews. We also evaluate them on English restaurant reviews in order to\nmeasure their robustness across the domains and languages. The evaluation shows\nthat our methods are enough robust for English restaurant reviews and French\nbook reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01833v1",
            "date": "2016-03-06 14:12:30+00:00",
            "title": "Semi-Automatic Data Annotation, POS Tagging and Mildly Context-Sensitive Disambiguation: the eXtended Revised AraMorph (XRAM)",
            "authors": [
                "Giuliano Lancioni",
                "Valeria Pettinari",
                "Laura Garofalo",
                "Marta Campanelli",
                "Ivana Pepe",
                "Simona Olivieri",
                "Ilaria Cicola"
            ],
            "abstract": "An extended, revised form of Tim Buckwalter's Arabic lexical and\nmorphological resource AraMorph, eXtended Revised AraMorph (henceforth XRAM),\nis presented which addresses a number of weaknesses and inconsistencies of the\noriginal model by allowing a wider coverage of real-world Classical and\ncontemporary (both formal and informal) Arabic texts. Building upon previous\nresearch, XRAM enhancements include (i) flag-selectable usage markers, (ii)\nprobabilistic mildly context-sensitive POS tagging, filtering, disambiguation\nand ranking of alternative morphological analyses, (iii) semi-automatic\nincrement of lexical coverage through extraction of lexical and morphological\ninformation from existing lexical resources. Testing of XRAM through a\nfront-end Python module showed a remarkable success level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01987v1",
            "date": "2016-03-07 09:54:11+00:00",
            "title": "A matter of words: NLP for quality evaluation of Wikipedia medical articles",
            "authors": [
                "Vittoria Cozza",
                "Marinella Petrocchi",
                "Angelo Spognardi"
            ],
            "abstract": "Automatic quality evaluation of Web information is a task with many fields of\napplications and of great relevance, especially in critical domains like the\nmedical one. We move from the intuition that the quality of content of medical\nWeb documents is affected by features related with the specific domain. First,\nthe usage of a specific vocabulary (Domain Informativeness); then, the adoption\nof specific codes (like those used in the infoboxes of Wikipedia articles) and\nthe type of document (e.g., historical and technical ones). In this paper, we\npropose to leverage specific domain features to improve the results of the\nevaluation of Wikipedia medical articles. In particular, we evaluate the\narticles adopting an \"actionable\" model, whose features are related to the\ncontent of the articles, so that the model can also directly suggest strategies\nfor improving a given article quality. We rely on Natural Language Processing\n(NLP) and dictionaries-based techniques in order to extract the bio-medical\nconcepts in a text. We prove the effectiveness of our approach by classifying\nthe medical articles of the Wikipedia Medicine Portal, which have been\npreviously manually labeled by the Wiki Project team. The results of our\nexperiments confirm that, by considering domain-oriented features, it is\npossible to obtain sensible improvements with respect to existing solutions,\nmainly for those articles that other approaches have less correctly classified.\nOther than being interesting by their own, the results call for further\nresearch in the area of domain specific features suitable for Web data quality\nassessment.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1603.02514v3",
            "date": "2016-03-08 13:24:45+00:00",
            "title": "Variational Autoencoders for Semi-supervised Text Classification",
            "authors": [
                "Weidi Xu",
                "Haoze Sun",
                "Chao Deng",
                "Ying Tan"
            ],
            "abstract": "Although semi-supervised variational autoencoder (SemiVAE) works in image\nclassification task, it fails in text classification task if using vanilla LSTM\nas its decoder. From a perspective of reinforcement learning, it is verified\nthat the decoder's capability to distinguish between different categorical\nlabels is essential. Therefore, Semi-supervised Sequential Variational\nAutoencoder (SSVAE) is proposed, which increases the capability by feeding\nlabel into its decoder RNN at each time-step. Two specific decoder structures\nare investigated and both of them are verified to be effective. Besides, in\norder to reduce the computational complexity in training, a novel optimization\nmethod is proposed, which estimates the gradient of the unlabeled objective\nfunction by sampling, along with two variance reduction techniques.\nExperimental results on Large Movie Review Dataset (IMDB) and AG's News corpus\nshow that the proposed approach significantly improves the classification\naccuracy compared with pure-supervised classifiers, and achieves competitive\nperformance against previous advanced methods. State-of-the-art results can be\nobtained by integrating other pretraining-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02618v2",
            "date": "2016-03-08 18:39:46+00:00",
            "title": "The red one!: On learning to refer to things based on their discriminative properties",
            "authors": [
                "Angeliki Lazaridou",
                "Nghia The Pham",
                "Marco Baroni"
            ],
            "abstract": "As a first step towards agents learning to communicate about their visual\nenvironment, we propose a system that, given visual representations of a\nreferent (cat) and a context (sofa), identifies their discriminative\nattributes, i.e., properties that distinguish them (has_tail). Moreover,\ndespite the lack of direct supervision at the attribute level, the model learns\nto assign plausible attributes to objects (sofa-has_cushion). Finally, we\npresent a preliminary experiment confirming the referential success of the\npredicted discriminative attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03112v1",
            "date": "2016-03-10 00:33:28+00:00",
            "title": "Building a Fine-Grained Entity Typing System Overnight for a New X (X = Language, Domain, Genre)",
            "authors": [
                "Lifu Huang",
                "Jonathan May",
                "Xiaoman Pan",
                "Heng Ji"
            ],
            "abstract": "Recent research has shown great progress on fine-grained entity typing. Most\nexisting methods require pre-defining a set of types and training a multi-class\nclassifier from a large labeled data set based on multi-level linguistic\nfeatures. They are thus limited to certain domains, genres and languages. In\nthis paper, we propose a novel unsupervised entity typing framework by\ncombining symbolic and distributional semantics. We start from learning general\nembeddings for each entity mention, compose the embeddings of specific contexts\nusing linguistic structures, link the mention to knowledge bases and learn its\nrelated knowledge representations. Then we develop a novel joint hierarchical\nclustering and linking algorithm to type all mentions using these\nrepresentations. This framework doesn't rely on any annotated data, predefined\ntyping schema, or hand-crafted features, therefore it can be quickly adapted to\na new domain, genre and language. Furthermore, it has great flexibility at\nincorporating linguistic structures (e.g., Abstract Meaning Representation\n(AMR), dependency relations) to improve specific context representation.\nExperiments on genres (news and discussion forum) show comparable performance\nwith state-of-the-art supervised typing systems trained from a large amount of\nlabeled data. Results on various languages (English, Chinese, Japanese, Hausa,\nand Yoruba) and domains (general and biomedical) demonstrate the portability of\nour framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03144v2",
            "date": "2016-03-10 04:27:15+00:00",
            "title": "Part-of-Speech Tagging for Historical English",
            "authors": [
                "Yi Yang",
                "Jacob Eisenstein"
            ],
            "abstract": "As more historical texts are digitized, there is interest in applying natural\nlanguage processing tools to these archives. However, the performance of these\ntools is often unsatisfactory, due to language change and genre differences.\nSpelling normalization heuristics are the dominant solution for dealing with\nhistorical texts, but this approach fails to account for changes in usage and\nvocabulary. In this empirical paper, we assess the capability of domain\nadaptation techniques to cope with historical texts, focusing on the classic\nbenchmark task of part-of-speech tagging. We evaluate several domain adaptation\nmethods on the task of tagging Early Modern English and Modern British English\ntexts in the Penn Corpora of Historical English. We demonstrate that the\nFeature Embedding method for unsupervised domain adaptation outperforms word\nembeddings and Brown clusters, showing the importance of embedding the entire\nfeature space, rather than just individual words. Feature Embeddings also give\nbetter performance than spelling normalization, but the combination of the two\nmethods is better still, yielding a 5% raw improvement in tagging accuracy on\nEarly Modern English texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03153v2",
            "date": "2016-03-10 06:01:28+00:00",
            "title": "Zipf's law emerges asymptotically during phase transitions in communicative systems",
            "authors": [
                "Bohdan B. Khomtchouk",
                "Claes Wahlestedt"
            ],
            "abstract": "Zipf's law predicts a power-law relationship between word rank and frequency\nin language communication systems, and is widely reported in texts yet remains\nenigmatic as to its origins. Computer simulations have shown that language\ncommunication systems emerge at an abrupt phase transition in the fidelity of\nmappings between symbols and objects. Since the phase transition approximates\nthe Heaviside or step function, we show that Zipfian scaling emerges\nasymptotically at high rank based on the Laplace transform. We thereby\ndemonstrate that Zipf's law gradually emerges from the moment of phase\ntransition in communicative systems. We show that this power-law scaling\nbehavior explains the emergence of natural languages at phase transitions. We\nfind that the emergence of Zipf's law during language communication suggests\nthat the use of rare words in a lexicon is critical for the construction of an\neffective communicative system at the phase transition.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1603.03610v1",
            "date": "2016-03-11 12:32:29+00:00",
            "title": "A short proof that $O_2$ is an MCFL",
            "authors": [
                "Mark-Jan Nederhof"
            ],
            "abstract": "We present a new proof that $O_2$ is a multiple context-free language. It\ncontrasts with a recent proof by Salvati (2015) in its avoidance of concepts\nthat seem specific to two-dimensional geometry, such as the complex exponential\nfunction. Our simple proof creates realistic prospects of widening the results\nto higher dimensions. This finding is of central importance to the relation\nbetween extreme free word order and classes of grammars used to describe the\nsyntax of natural language.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1603.04553v1",
            "date": "2016-03-15 04:39:15+00:00",
            "title": "Unsupervised Ranking Model for Entity Coreference Resolution",
            "authors": [
                "Xuezhe Ma",
                "Zhengzhong Liu",
                "Eduard Hovy"
            ],
            "abstract": "Coreference resolution is one of the first stages in deep language\nunderstanding and its importance has been well recognized in the natural\nlanguage processing community. In this paper, we propose a generative,\nunsupervised ranking model for entity coreference resolution by introducing\nresolution mode variables. Our unsupervised system achieves 58.44% F1 score of\nthe CoNLL metric on the English data from the CoNLL-2012 shared task (Pradhan\net al., 2012), outperforming the Stanford deterministic system (Lee et al.,\n2013) by 3.01%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05350v2",
            "date": "2016-03-17 03:54:41+00:00",
            "title": "Self-organization of vocabularies under different interaction orders",
            "authors": [
                "Javier Vera"
            ],
            "abstract": "Traditionally, the formation of vocabularies has been studied by agent-based\nmodels (specially, the Naming Game) in which random pairs of agents negotiate\nword-meaning associations at each discrete time step. This paper proposes a\nfirst approximation to a novel question: To what extent the negotiation of\nword-meaning associations is influenced by the order in which the individuals\ninteract? Automata Networks provide the adequate mathematical framework to\nexplore this question. Computer simulations suggest that on two-dimensional\nlattices the typical features of the formation of word-meaning associations are\nrecovered under random schemes that update small fractions of the population at\nthe same time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05354v2",
            "date": "2016-03-17 04:39:02+00:00",
            "title": "Modeling self-organization of vocabularies under phonological similarity effects",
            "authors": [
                "Javier Vera"
            ],
            "abstract": "This work develops a computational model (by Automata Networks) of\nphonological similarity effects involved in the formation of word-meaning\nassociations on artificial populations of speakers. Classical studies show that\nin recalling experiments memory performance was impaired for phonologically\nsimilar words versus dissimilar ones. Here, the individuals confound\nphonologically similar words according to a predefined parameter. The main\nhypothesis is that there is a critical range of the parameter, and with this,\nof working-memory mechanisms, which implies drastic changes in the final\nconsensus of the entire population. Theoretical results present proofs of\nconvergence for a particular case of the model within a worst-case complexity\nframework. Computer simulations describe the evolution of an energy function\nthat measures the amount of local agreement between individuals. The main\nfinding is the appearance of sudden changes in the energy function at critical\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05673v1",
            "date": "2016-03-17 20:20:32+00:00",
            "title": "Predicting health inspection results from online restaurant reviews",
            "authors": [
                "Samantha Wong",
                "Hamidreza Chinaei",
                "Frank Rudzicz"
            ],
            "abstract": "Informatics around public health are increasingly shifting from the\nprofessional to the public spheres. In this work, we apply linguistic analytics\nto restaurant reviews, from Yelp, in order to automatically predict official\nhealth inspection reports. We consider two types of feature sets, i.e., keyword\ndetection and topic model features, and use these in several classification\nmethods. Our empirical analysis shows that these extracted features can predict\npublic health inspection reports with over 90% accuracy using simple support\nvector machines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05962v1",
            "date": "2016-03-18 19:24:44+00:00",
            "title": "Document Neural Autoregressive Distribution Estimation",
            "authors": [
                "Stanislas Lauly",
                "Yin Zheng",
                "Alexandre Allauzen",
                "Hugo Larochelle"
            ],
            "abstract": "We present an approach based on feed-forward neural networks for learning the\ndistribution of textual documents. This approach is inspired by the Neural\nAutoregressive Distribution Estimator(NADE) model, which has been shown to be a\ngood estimator of the distribution of discrete-valued igh-dimensional vectors.\nIn this paper, we present how NADE can successfully be adapted to the case of\ntextual data, retaining from NADE the property that sampling or computing the\nprobability of observations can be done exactly and efficiently. The approach\ncan also be used to learn deep representations of documents that are\ncompetitive to those learned by the alternative topic modeling approaches.\nFinally, we describe how the approach can be combined with a regular neural\nnetwork N-gram model and substantially improve its performance, by making its\nlearned representation sensitive to the larger, document-specific context.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1603.06147v4",
            "date": "2016-03-19 21:35:04+00:00",
            "title": "A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation",
            "authors": [
                "Junyoung Chung",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "The existing machine translation systems, whether phrase-based or neural,\nhave relied almost exclusively on word-level modelling with explicit\nsegmentation. In this paper, we ask a fundamental question: can neural machine\ntranslation generate a character sequence without any explicit segmentation? To\nanswer this question, we evaluate an attention-based encoder-decoder with a\nsubword-level encoder and a character-level decoder on four language\npairs--En-Cs, En-De, En-Ru and En-Fi-- using the parallel corpora from WMT'15.\nOur experiments show that the models with a character-level decoder outperform\nthe ones with a subword-level decoder on all of the four language pairs.\nFurthermore, the ensembles of neural models with a character-level decoder\noutperform the state-of-the-art non-neural machine translation systems on\nEn-Cs, En-De and En-Fi and perform comparably on En-Ru.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06270v2",
            "date": "2016-03-20 21:15:56+00:00",
            "title": "Multi-Task Cross-Lingual Sequence Tagging from Scratch",
            "authors": [
                "Zhilin Yang",
                "Ruslan Salakhutdinov",
                "William Cohen"
            ],
            "abstract": "We present a deep hierarchical recurrent neural network for sequence tagging.\nGiven a sequence of words, our model employs deep gated recurrent units on both\ncharacter and word levels to encode morphology and context information, and\napplies a conditional random field layer to predict the tags. Our model is task\nindependent, language independent, and feature engineering free. We further\nextend our model to multi-task and cross-lingual joint training by sharing the\narchitecture and parameters. Our model achieves state-of-the-art results in\nmultiple languages on several benchmark tasks including POS tagging, chunking,\nand NER. We also demonstrate that multi-task and cross-lingual joint training\ncan improve the performance in various cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06571v3",
            "date": "2016-03-21 16:32:06+00:00",
            "title": "Bayesian Neural Word Embedding",
            "authors": [
                "Oren Barkan"
            ],
            "abstract": "Recently, several works in the domain of natural language processing\npresented successful methods for word embedding. Among them, the Skip-Gram with\nnegative sampling, known also as word2vec, advanced the state-of-the-art of\nvarious linguistics tasks. In this paper, we propose a scalable Bayesian neural\nword embedding algorithm. The algorithm relies on a Variational Bayes solution\nfor the Skip-Gram objective and a detailed step by step description is\nprovided. We present experimental results that demonstrate the performance of\nthe proposed algorithm for word analogy and similarity tasks on six different\ndatasets and show it is competitive with the original Skip-Gram method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06677v1",
            "date": "2016-03-22 05:07:16+00:00",
            "title": "Learning Executable Semantic Parsers for Natural Language Understanding",
            "authors": [
                "Percy Liang"
            ],
            "abstract": "For building question answering systems and natural language interfaces,\nsemantic parsing has emerged as an important and powerful paradigm. Semantic\nparsers map natural language into logical forms, the classic representation for\nmany important linguistic phenomena. The modern twist is that we are interested\nin learning semantic parsers from data, which introduces a new layer of\nstatistical and computational issues. This article lays out the components of a\nstatistical semantic parser, highlighting the key challenges. We will see that\nsemantic parsing is a rich fusion of the logical and the statistical world, and\nthat this fusion will play an integral role in the future of natural language\nunderstanding systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06744v2",
            "date": "2016-03-22 11:41:51+00:00",
            "title": "Latent Predictor Networks for Code Generation",
            "authors": [
                "Wang Ling",
                "Edward Grefenstette",
                "Karl Moritz Hermann",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Andrew Senior",
                "Fumin Wang",
                "Phil Blunsom"
            ],
            "abstract": "Many language generation tasks require the production of text conditioned on\nboth structured and unstructured inputs. We present a novel neural network\narchitecture which generates an output sequence conditioned on an arbitrary\nnumber of input functions. Crucially, our approach allows both the choice of\nconditioning context and the granularity of generation, for example characters\nor tokens, to be marginalised, thus permitting scalable and effective training.\nUsing this framework, we address the problem of generating programming code\nfrom a mixed natural language and structured specification. We create two new\ndata sets for this paradigm derived from the collectible trading card games\nMagic the Gathering and Hearthstone. On these, and a third preexisting corpus,\nwe demonstrate that marginalising multiple predictors allows our model to\noutperform strong benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06785v1",
            "date": "2016-03-22 13:34:28+00:00",
            "title": "Multi-domain machine translation enhancements by parallel data extraction from comparable corpora",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Emilia Rejmund",
                "Krzysztof Marasek"
            ],
            "abstract": "Parallel texts are a relatively rare language resource, however, they\nconstitute a very useful research material with a wide range of applications.\nThis study presents and analyses new methodologies we developed for obtaining\nsuch data from previously built comparable corpora. The methodologies are\nautomatic and unsupervised which makes them good for large scale research. The\ntask is highly practical as non-parallel multilingual data occur much more\nfrequently than parallel corpora and accessing them is easy, although parallel\nsentences are a considerably more useful resource. In this study, we propose a\nmethod of automatic web crawling in order to build topic-aligned comparable\ncorpora, e.g. based on the Wikipedia or Euronews.com. We also developed new\nmethods of obtaining parallel sentences from comparable data and proposed\nmethods of filtration of corpora capable of selecting inconsistent or only\npartially equivalent translations. Our methods are easily scalable to other\nlanguages. Evaluation of the quality of the created corpora was performed by\nanalysing the impact of their use on statistical machine translation systems.\nExperiments were presented on the basis of the Polish-English language pair for\ntexts from different domains, i.e. lectures, phrasebooks, film dialogues,\nEuropean Parliament proceedings and texts contained medicines leaflets. We also\ntested a second method of creating parallel corpora based on data from\ncomparable corpora which allows for automatically expanding the existing corpus\nof sentences about a given domain on the basis of analogies found between them.\nIt does not require, therefore, having past parallel resources in order to\ntrain a classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07185v1",
            "date": "2016-03-23 13:57:33+00:00",
            "title": "Enabling Cognitive Intelligence Queries in Relational Databases using Low-dimensional Word Embeddings",
            "authors": [
                "Rajesh Bordawekar",
                "Oded Shmueli"
            ],
            "abstract": "We apply distributed language embedding methods from Natural Language\nProcessing to assign a vector to each database entity associated token (for\nexample, a token may be a word occurring in a table row, or the name of a\ncolumn). These vectors, of typical dimension 200, capture the meaning of tokens\nbased on the contexts in which the tokens appear together. To form vectors, we\napply a learning method to a token sequence derived from the database. We\ndescribe various techniques for extracting token sequences from a database. The\ntechniques differ in complexity, in the token sequences they output and in the\ndatabase information used (e.g., foreign keys). The vectors can be used to\nalgebraically quantify semantic relationships between the tokens such as\nsimilarities and analogies. Vectors enable a dual view of the data: relational\nand (meaningful rather than purely syntactical) text. We introduce and explore\na new class of queries called cognitive intelligence (CI) queries that extract\ninformation from the database based, in part, on the relationships encoded by\nvectors. We have implemented a prototype system on top of Spark to exhibit the\npower of CI queries. Here, CI queries are realized via SQL UDFs. This power\ngoes far beyond text extensions to relational systems due to the information\nencoded in vectors. We also consider various extensions to the basic scheme,\nincluding using a collection of views derived from the database to focus on a\ndomain of interest, utilizing vectors and/or text from external sources,\nmaintaining vectors as the database evolves and exploring a database without\nutilizing its schema. For the latter, we consider minimal extensions to SQL to\nvastly improve query expressiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08091v1",
            "date": "2016-03-26 09:25:16+00:00",
            "title": "Measuring Book Impact Based on the Multi-granularity Online Review Mining",
            "authors": [
                "Qingqing Zhou",
                "Chengzhi Zhang",
                "Star X. Zhao",
                "Bikun Chen"
            ],
            "abstract": "As with articles and journals, the customary methods for measuring books'\nacademic impact mainly involve citations, which is easy but limited to\ninterrogating traditional citation databases and scholarly book reviews,\nResearchers have attempted to use other metrics, such as Google Books,\nlibcitation, and publisher prestige. However, these approaches lack\ncontent-level information and cannot determine the citation intentions of\nusers. Meanwhile, the abundant online review resources concerning academic\nbooks can be used to mine deeper information and content utilizing altmetric\nperspectives. In this study, we measure the impacts of academic books by\nmulti-granularity mining online reviews, and we identify factors that affect a\nbook's impact. First, online reviews of a sample of academic books on Amazon.cn\nare crawled and processed. Then, multi-granularity review mining is conducted\nto identify review sentiment polarities and aspects' sentiment values. Lastly,\nthe numbers of positive reviews and negative reviews, aspect sentiment values,\nstar values, and information regarding helpfulness are integrated via the\nentropy method, and lead to the calculation of the final book impact scores.\nThe results of a correlation analysis of book impact scores obtained via our\nmethod versus traditional book citations show that, although there are\nsubstantial differences between subject areas, online book reviews tend to\nreflect the academic impact. Thus, we infer that online reviews represent a\npromising source for mining book impact within the altmetric perspective and at\nthe multi-granularity content level. Moreover, our proposed method might also\nbe a means by which to measure other books besides academic publications.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08636v1",
            "date": "2016-03-29 04:34:39+00:00",
            "title": "Towards an Automated Requirements-driven Development of Smart Cyber-Physical Systems",
            "authors": [
                "Jiri Vinarek",
                "Petr Hnetynka"
            ],
            "abstract": "The Invariant Refinement Method for Self Adaptation (IRM-SA) is a design\nmethod targeting development of smart Cyber-Physical Systems (sCPS). It allows\nfor a systematic translation of the system requirements into the system\narchitecture expressed as an ensemble-based component system (EBCS). However,\nsince the requirements are captured using natural language, there exists the\ndanger of their misinterpretation due to natural language requirements'\nambiguity, which could eventually lead to design errors. Thus, automation and\nvalidation of the design process is desirable. In this paper, we (i) analyze\nthe translation process of natural language requirements into the IRM-SA model,\n(ii) identify individual steps that can be automated and/or validated using\nnatural language processing techniques, and (iii) propose suitable methods.",
            "journal ref": "EPTCS 205, 2016, pp. 59-68",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1603.08832v1",
            "date": "2016-03-29 16:24:46+00:00",
            "title": "Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Community",
            "authors": [
                "Ethan Fast",
                "Tina Vachovsky",
                "Michael S. Bernstein"
            ],
            "abstract": "Imagine a princess asleep in a castle, waiting for her prince to slay the\ndragon and rescue her. Tales like the famous Sleeping Beauty clearly divide up\ngender roles. But what about more modern stories, borne of a generation\nincreasingly aware of social constructs like sexism and racism? Do these\nstories tend to reinforce gender stereotypes, or counter them? In this paper,\nwe present a technique that combines natural language processing with a\ncrowdsourced lexicon of stereotypes to capture gender biases in fiction. We\napply this technique across 1.8 billion words of fiction from the Wattpad\nonline writing community, investigating gender representation in stories, how\nmale and female characters behave and are described, and how authors' use of\ngender stereotypes is associated with the community's ratings. We find that\nmale over-representation and traditional gender stereotypes (e.g., dominant men\nand submissive women) are common throughout nearly every genre in our corpus.\nHowever, only some of these stereotypes, like sexual or violent men, are\nassociated with highly rated stories. Finally, despite women often being the\ntarget of negative stereotypes, female authors are equally likely to write such\nstereotypes as men.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09188v1",
            "date": "2016-03-30 13:43:38+00:00",
            "title": "Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings",
            "authors": [
                "Spandana Gella",
                "Mirella Lapata",
                "Frank Keller"
            ],
            "abstract": "We introduce a new task, visual sense disambiguation for verbs: given an\nimage and a verb, assign the correct sense of the verb, i.e., the one that\ndescribes the action depicted in the image. Just as textual word sense\ndisambiguation is useful for a wide range of NLP tasks, visual sense\ndisambiguation can be useful for multimodal tasks such as image retrieval,\nimage description, and text illustration. We introduce VerSe, a new dataset\nthat augments existing multimodal datasets (COCO and TUHOI) with sense labels.\nWe propose an unsupervised algorithm based on Lesk which performs visual sense\ndisambiguation using textual, visual, or multimodal embeddings. We find that\ntextual embeddings perform well when gold-standard textual annotations (object\nlabels and image descriptions) are available, while multimodal embeddings\nperform well on unannotated images. We also verify our findings by using the\ntextual and multimodal embeddings as features in a supervised setting and\nanalyse the performance of visual sense disambiguation task. VerSe is made\npublicly available and can be downloaded at:\nhttps://github.com/spandanagella/verse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09460v2",
            "date": "2016-03-31 05:47:03+00:00",
            "title": "System Combination for Short Utterance Speaker Recognition",
            "authors": [
                "Lantian Li",
                "Dong Wang",
                "Xiaodong Zhang",
                "Thomas Fang Zheng",
                "Panshi Jin"
            ],
            "abstract": "For text-independent short-utterance speaker recognition (SUSR), the\nperformance often degrades dramatically. This paper presents a combination\napproach to the SUSR tasks with two phonetic-aware systems: one is the\nDNN-based i-vector system and the other is our recently proposed\nsubregion-based GMM-UBM system. The former employs phone posteriors to\nconstruct an i-vector model in which the shared statistics offers stronger\nrobustness against limited test data, while the latter establishes a\nphone-dependent GMM-UBM system which represents speaker characteristics with\nmore details. A score-level fusion is implemented to integrate the respective\nadvantages from the two systems. Experimental results show that for the\ntext-independent SUSR task, both the DNN-based i-vector system and the\nsubregion-based GMM-UBM system outperform their respective baselines, and the\nscore-level system combination delivers performance improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09630v2",
            "date": "2016-03-31 15:10:40+00:00",
            "title": "Differentiable Pooling for Unsupervised Acoustic Model Adaptation",
            "authors": [
                "Pawel Swietojanski",
                "Steve Renals"
            ],
            "abstract": "We present a deep neural network (DNN) acoustic model that includes\nparametrised and differentiable pooling operators. Unsupervised acoustic model\nadaptation is cast as the problem of updating the decision boundaries\nimplemented by each pooling operator. In particular, we experiment with two\ntypes of pooling parametrisations: learned $L_p$-norm pooling and weighted\nGaussian pooling, in which the weights of both operators are treated as\nspeaker-dependent. We perform investigations using three different large\nvocabulary speech recognition corpora: AMI meetings, TED talks and Switchboard\nconversational telephone speech. We demonstrate that differentiable pooling\noperators provide a robust and relatively low-dimensional way to adapt acoustic\nmodels, with relative word error rates reductions ranging from 5--20% with\nrespect to unadapted systems, which themselves are better than the baseline\nfully-connected DNN-based acoustic models. We also investigate how the proposed\ntechniques work under various adaptation conditions including the quality of\nadaptation data and complementarity to other feature- and model-space\nadaptation methods, as well as providing an analysis of the characteristics of\neach of the proposed approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09631v2",
            "date": "2016-03-31 15:13:51+00:00",
            "title": "Data Collection for Interactive Learning through the Dialog",
            "authors": [
                "Miroslav Vodol\u00e1n",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "This paper presents a dataset collected from natural dialogs which enables to\ntest the ability of dialog systems to learn new facts from user utterances\nthroughout the dialog. This interactive learning will help with one of the most\nprevailing problems of open domain dialog system, which is the sparsity of\nfacts a dialog system can reason about. The proposed dataset, consisting of\n1900 collected dialogs, allows simulation of an interactive gaining of\ndenotations and questions explanations from users which can be used for the\ninteractive learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09727v1",
            "date": "2016-03-31 19:16:54+00:00",
            "title": "Neural Language Correction with Character-Based Attention",
            "authors": [
                "Ziang Xie",
                "Anand Avati",
                "Naveen Arivazhagan",
                "Dan Jurafsky",
                "Andrew Y. Ng"
            ],
            "abstract": "Natural language correction has the potential to help language learners\nimprove their writing skills. While approaches with separate classifiers for\ndifferent error types have high precision, they do not flexibly handle errors\nsuch as redundancy or non-idiomatic phrasing. On the other hand, word and\nphrase-based machine translation methods are not designed to cope with\northographic errors, and have recently been outpaced by neural models.\nMotivated by these issues, we present a neural network-based approach to\nlanguage correction. The core component of our method is an encoder-decoder\nrecurrent neural network with an attention mechanism. By operating at the\ncharacter level, the network avoids the problem of out-of-vocabulary words. We\nillustrate the flexibility of our approach on dataset of noisy, user-generated\ntext collected from an English learner forum. When combined with a language\nmodel, our method achieves a state-of-the-art $F_{0.5}$-score on the CoNLL 2014\nShared Task. We further demonstrate that training the network on additional\ndata with synthesized errors can improve performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00125v2",
            "date": "2016-04-01 04:18:39+00:00",
            "title": "AttSum: Joint Learning of Focusing and Summarization with Neural Attention",
            "authors": [
                "Ziqiang Cao",
                "Wenjie Li",
                "Sujian Li",
                "Furu Wei",
                "Yanran Li"
            ],
            "abstract": "Query relevance ranking and sentence saliency ranking are the two main tasks\nin extractive query-focused summarization. Previous supervised summarization\nsystems often perform the two tasks in isolation. However, since reference\nsummaries are the trade-off between relevance and saliency, using them as\nsupervision, neither of the two rankers could be trained well. This paper\nproposes a novel summarization system called AttSum, which tackles the two\ntasks jointly. It automatically learns distributed representations for\nsentences as well as the document cluster. Meanwhile, it applies the attention\nmechanism to simulate the attentive reading of human behavior when a query is\ngiven. Extensive experiments are conducted on DUC query-focused summarization\nbenchmark datasets. Without using any hand-crafted features, AttSum achieves\ncompetitive performance. It is also observed that the sentences recognized to\nfocus on the query indeed meet the query need.",
            "journal ref": "COLING 2016",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1604.00466v3",
            "date": "2016-04-02 06:35:45+00:00",
            "title": "Automatic Annotation of Structured Facts in Images",
            "authors": [
                "Mohamed Elhoseiny",
                "Scott Cohen",
                "Walter Chang",
                "Brian Price",
                "Ahmed Elgammal"
            ],
            "abstract": "Motivated by the application of fact-level image understanding, we present an\nautomatic method for data collection of structured visual facts from images\nwith captions. Example structured facts include attributed objects (e.g.,\n<flower, red>), actions (e.g., <baby, smile>), interactions (e.g., <man,\nwalking, dog>), and positional information (e.g., <vase, on, table>). The\ncollected annotations are in the form of fact-image pairs (e.g.,<man, walking,\ndog> and an image region containing this fact). With a language approach, the\nproposed method is able to collect hundreds of thousands of visual fact\nannotations with accuracy of 83% according to human judgment. Our method\nautomatically collected more than 380,000 visual fact annotations and more than\n110,000 unique visual facts from images with captions and localized them in\nimages in less than one day of processing time on standard CPU platforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00562v2",
            "date": "2016-04-02 21:52:03+00:00",
            "title": "Reasoning About Pragmatics with Neural Listeners and Speakers",
            "authors": [
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "We present a model for pragmatically describing scenes, in which contrastive\nbehavior results from a combination of inference-driven pragmatics and learned\nsemantics. Like previous learned approaches to language generation, our model\nuses a simple feature-driven architecture (here a pair of neural \"listener\" and\n\"speaker\" models) to ground language in the world. Like inference-driven\napproaches to pragmatics, our model actively reasons about listener behavior\nwhen selecting utterances. For training, our approach requires only ordinary\ncaptions, annotated _without_ demonstration of the pragmatic behavior the model\nultimately exhibits. In human evaluations on a referring expression game, our\napproach succeeds 81% of the time, compared to a 69% success rate using\nexisting techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00788v2",
            "date": "2016-04-04 09:30:54+00:00",
            "title": "Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models",
            "authors": [
                "Minh-Thang Luong",
                "Christopher D. Manning"
            ],
            "abstract": "Nearly all previous work on neural machine translation (NMT) has used quite\nrestricted vocabularies, perhaps with a subsequent method to patch in unknown\nwords. This paper presents a novel word-character solution to achieving open\nvocabulary NMT. We build hybrid systems that translate mostly at the word level\nand consult the character components for rare words. Our character-level\nrecurrent neural networks compute source word representations and recover\nunknown target words when needed. The twofold advantage of such a hybrid\napproach is that it is much faster and easier to train than character-based\nones; at the same time, it never produces unknown words as in the case of\nword-based models. On the WMT'15 English to Czech translation task, this hybrid\napproach offers an addition boost of +2.1-11.4 BLEU points over models that\nalready handle unknown words. Our best system achieves a new state-of-the-art\nresult with 20.7 BLEU score. We demonstrate that our character models can\nsuccessfully learn to not only generate well-formed words for Czech, a\nhighly-inflected language with a very complex vocabulary, but also build\ncorrect representations for English source words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00834v2",
            "date": "2016-04-04 12:29:00+00:00",
            "title": "In narrative texts punctuation marks obey the same statistics as words",
            "authors": [
                "Andrzej Kulig",
                "Jaroslaw Kwapien",
                "Tomasz Stanisz",
                "Stanislaw Drozdz"
            ],
            "abstract": "From a grammar point of view, the role of punctuation marks in a sentence is\nformally defined and well understood. In semantic analysis punctuation plays\nalso a crucial role as a method of avoiding ambiguity of the meaning. A\ndifferent situation can be observed in the statistical analyses of language\nsamples, where the decision on whether the punctuation marks should be\nconsidered or should be neglected is seen rather as arbitrary and at present it\nbelongs to a researcher's preference. An objective of this work is to shed some\nlight onto this problem by providing us with an answer to the question whether\nthe punctuation marks may be treated as ordinary words and whether they should\nbe included in any analysis of the word co-occurences. We already know from our\nprevious study (S.~Dro\\.zd\\.z {\\it et al.}, Inf. Sci. 331 (2016) 32-44) that\nfull stops that determine the length of sentences are the main carrier of\nlong-range correlations. Now we extend that study and analyze statistical\nproperties of the most common punctuation marks in a few Indo-European\nlanguages, investigate their frequencies, and locate them accordingly in the\nZipf rank-frequency plots as well as study their role in the word-adjacency\nnetworks. We show that, from a statistical viewpoint, the punctuation marks\nreveal properties that are qualitatively similar to the properties of the most\nfrequent words like articles, conjunctions, pronouns, and prepositions. This\nrefers to both the Zipfian analysis and the network analysis. By adding the\npunctuation marks to the Zipf plots, we also show that these plots that are\nnormally described by the Zipf-Mandelbrot distribution largely restore the\npower-law Zipfian behaviour for the most frequent items.",
            "journal ref": "Information Sciences 375 (2017) 98-113",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00933v1",
            "date": "2016-04-04 16:18:44+00:00",
            "title": "Entity Type Recognition using an Ensemble of Distributional Semantic Models to Enhance Query Understanding",
            "authors": [
                "Walid Shalaby",
                "Khalifeh Al Jadda",
                "Mohammed Korayem",
                "Trey Grainger"
            ],
            "abstract": "We present an ensemble approach for categorizing search query entities in the\nrecruitment domain. Understanding the types of entities expressed in a search\nquery (Company, Skill, Job Title, etc.) enables more intelligent information\nretrieval based upon those entities compared to a traditional keyword-based\nsearch. Because search queries are typically very short, leveraging a\ntraditional bag-of-words model to identify entity types would be inappropriate\ndue to the lack of contextual information. Our approach instead combines clues\nfrom different sources of varying complexity in order to collect real-world\nknowledge about query entities. We employ distributional semantic\nrepresentations of query entities through two models: 1) contextual vectors\ngenerated from encyclopedic corpora like Wikipedia, and 2) high dimensional\nword embedding vectors generated from millions of job postings using word2vec.\nAdditionally, our approach utilizes both entity linguistic properties obtained\nfrom WordNet and ontological properties extracted from DBpedia. We evaluate our\napproach on a data set created at CareerBuilder; the largest job board in the\nUS. The data set contains entities extracted from millions of job\nseekers/recruiters search queries, job postings, and resume documents. After\nconstructing the distributional vectors of search entities, we use supervised\nmachine learning to infer search entity types. Empirical results show that our\napproach outperforms the state-of-the-art word2vec distributional semantics\nmodel trained on Wikipedia. Moreover, we achieve micro-averaged F 1 score of\n97% using the proposed distributional representations ensemble.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01537v1",
            "date": "2016-04-06 08:26:31+00:00",
            "title": "Generating Chinese Classical Poems with RNN Encoder-Decoder",
            "authors": [
                "Xiaoyuan Yi",
                "Ruoyu Li",
                "Maosong Sun"
            ],
            "abstract": "We take the generation of Chinese classical poem lines as a\nsequence-to-sequence learning problem, and build a novel system based on the\nRNN Encoder-Decoder structure to generate quatrains (Jueju in Chinese), with a\ntopic word as input. Our system can jointly learn semantic meaning within a\nsingle line, semantic relevance among lines in a poem, and the use of\nstructural, rhythmical and tonal patterns, without utilizing any constraint\ntemplates. Experimental results show that our system outperforms other\ncompetitive systems. We also find that the attention mechanism can capture the\nword associations in Chinese classical poetry and inverting target lines in\ntraining can improve performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01696v1",
            "date": "2016-04-06 17:15:10+00:00",
            "title": "A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories",
            "authors": [
                "Nasrin Mostafazadeh",
                "Nathanael Chambers",
                "Xiaodong He",
                "Devi Parikh",
                "Dhruv Batra",
                "Lucy Vanderwende",
                "Pushmeet Kohli",
                "James Allen"
            ],
            "abstract": "Representation and learning of commonsense knowledge is one of the\nfoundational problems in the quest to enable deep language understanding. This\nissue is particularly challenging for understanding casual and correlational\nrelationships between events. While this topic has received a lot of interest\nin the NLP community, research has been hindered by the lack of a proper\nevaluation framework. This paper attempts to address this problem with a new\nframework for evaluating story understanding and script learning: the 'Story\nCloze Test'. This test requires a system to choose the correct ending to a\nfour-sentence story. We created a new corpus of ~50k five-sentence commonsense\nstories, ROCStories, to enable this evaluation. This corpus is unique in two\nways: (1) it captures a rich set of causal and temporal commonsense relations\nbetween daily events, and (2) it is a high quality collection of everyday life\nstories that can also be used for story generation. Experimental evaluation\nshows that a host of baselines and state-of-the-art models based on shallow\nlanguage understanding struggle to achieve a high score on the Story Cloze\nTest. We discuss these implications for script and story learning, and offer\nsuggestions for deeper language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01729v2",
            "date": "2016-04-06 19:01:28+00:00",
            "title": "Improving LSTM-based Video Description with Linguistic Knowledge Mined from Text",
            "authors": [
                "Subhashini Venugopalan",
                "Lisa Anne Hendricks",
                "Raymond Mooney",
                "Kate Saenko"
            ],
            "abstract": "This paper investigates how linguistic knowledge mined from large text\ncorpora can aid the generation of natural language descriptions of videos.\nSpecifically, we integrate both a neural language model and distributional\nsemantics trained on large text corpora into a recent LSTM-based architecture\nfor video description. We evaluate our approach on a collection of Youtube\nvideos as well as two large movie description datasets showing significant\nimprovements in grammaticality while modestly improving descriptive quality.",
            "journal ref": "Proc.EMNLP (2016) pg.1961-1966",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02506v3",
            "date": "2016-04-09 01:14:05+00:00",
            "title": "Word embeddings and recurrent neural networks based on Long-Short Term Memory nodes in supervised biomedical word sense disambiguation",
            "authors": [
                "Antonio Jimeno Yepes"
            ],
            "abstract": "Word sense disambiguation helps identifying the proper sense of ambiguous\nwords in text. With large terminologies such as the UMLS Metathesaurus\nambiguities appear and highly effective disambiguation methods are required.\nSupervised learning algorithm methods are used as one of the approaches to\nperform disambiguation. Features extracted from the context of an ambiguous\nword are used to identify the proper sense of such a word. The type of features\nhave an impact on machine learning methods, thus affect disambiguation\nperformance. In this work, we have evaluated several types of features derived\nfrom the context of the ambiguous word and we have explored as well more global\nfeatures derived from MEDLINE using word embeddings. Results show that word\nembeddings improve the performance of more traditional features and allow as\nwell using recurrent neural network classifiers based on Long-Short Term Memory\n(LSTM) nodes. The combination of unigrams and word embeddings with an SVM sets\na new state of the art performance with a macro accuracy of 95.97 in the MSH\nWSD data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03249v1",
            "date": "2016-04-12 05:23:26+00:00",
            "title": "Attributes as Semantic Units between Natural Language and Visual Recognition",
            "authors": [
                "Marcus Rohrbach"
            ],
            "abstract": "Impressive progress has been made in the fields of computer vision and\nnatural language processing. However, it remains a challenge to find the best\npoint of interaction for these very different modalities. In this chapter we\ndiscuss how attributes allow us to exchange information between the two\nmodalities and in this way lead to an interaction on a semantic level.\nSpecifically we discuss how attributes allow using knowledge mined from\nlanguage resources for recognizing novel visual categories, how we can generate\nsentence description about images and video, how we can ground natural language\nin visual content, and finally, how we can answer natural language questions\nabout images.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1604.03318v1",
            "date": "2016-04-12 09:27:00+00:00",
            "title": "Applying Ontological Modeling on Quranic Nature Domain",
            "authors": [
                "A. B. M. Shamsuzzaman Sadi",
                "Towfique Anam",
                "Mohamed Abdirazak",
                "Abdillahi Hasan Adnan",
                "Sazid Zaman Khan",
                "Mohamed Mahmudur Rahman",
                "Ghassan Samara"
            ],
            "abstract": "The holy Quran is the holy book of the Muslims. It contains information about\nmany domains. Often people search for particular concepts of holy Quran based\non the relations among concepts. An ontological modeling of holy Quran can be\nuseful in such a scenario. In this paper, we have modeled nature related\nconcepts of holy Quran using OWL (Web Ontology Language) / RDF (Resource\nDescription Framework). Our methodology involves identifying nature related\nconcepts mentioned in holy Quran and identifying relations among those\nconcepts. These concepts and relations are represented as classes/instances and\nproperties of an OWL ontology. Later, in the result section it is shown that,\nusing the Ontological model, SPARQL queries can retrieve verses and concepts of\ninterest. Thus, this modeling helps semantic search and query on the holy\nQuran. In this work, we have used English translation of the holy Quran by\nSahih International, Protege OWL Editor and for querying we have used SPARQL.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1604.04383v3",
            "date": "2016-04-15 07:35:00+00:00",
            "title": "Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding",
            "authors": [
                "Milos Cernak",
                "Alexandros Lazaridis",
                "Afsaneh Asaei",
                "Philip N. Garner"
            ],
            "abstract": "Most current very low bit rate (VLBR) speech coding systems use hidden Markov\nmodel (HMM) based speech recognition/synthesis techniques. This allows\ntransmission of information (such as phonemes) segment by segment that\ndecreases the bit rate. However, the encoder based on a phoneme speech\nrecognition may create bursts of segmental errors. Segmental errors are further\npropagated to optional suprasegmental (such as syllable) information coding.\nTogether with the errors of voicing detection in pitch parametrization,\nHMM-based speech coding creates speech discontinuities and unnatural speech\nsound artefacts.\n  In this paper, we propose a novel VLBR speech coding framework based on\nneural networks (NNs) for end-to-end speech analysis and synthesis without\nHMMs. The speech coding framework relies on phonological (sub-phonetic)\nrepresentation of speech, and it is designed as a composition of deep and\nspiking NNs: a bank of phonological analysers at the transmitter, and a\nphonological synthesizer at the receiver, both realised as deep NNs, and a\nspiking NN as an incremental and robust encoder of syllable boundaries for\ncoding of continuous fundamental frequency (F0). A combination of phonological\nfeatures defines much more sound patterns than phonetic features defined by\nHMM-based speech coders, and the finer analysis/synthesis code contributes into\nsmoother encoded speech. Listeners significantly prefer the NN-based approach\ndue to fewer discontinuities and speech artefacts of the encoded speech. A\nsingle forward pass is required during the speech encoding and decoding. The\nproposed VLBR speech coding operates at a bit rate of approximately 360 bits/s.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  Volume: 24, Issue: 12, Dec. 2016",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1604.04802v1",
            "date": "2016-04-16 21:18:14+00:00",
            "title": "Supervised and Unsupervised Ensembling for Knowledge Base Population",
            "authors": [
                "Nazneen Fatema Rajani",
                "Raymond J. Mooney"
            ],
            "abstract": "We present results on combining supervised and unsupervised methods to\nensemble multiple systems for two popular Knowledge Base Population (KBP)\ntasks, Cold Start Slot Filling (CSSF) and Tri-lingual Entity Discovery and\nLinking (TEDL). We demonstrate that our combined system along with auxiliary\nfeatures outperforms the best performing system for both tasks in the 2015\ncompetition, several ensembling baselines, as well as the state-of-the-art\nstacking approach to ensembling KBP systems. The success of our technique on\ntwo different and challenging problems demonstrates the power and generality of\nour combined approach to ensembling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04835v3",
            "date": "2016-04-17 07:15:33+00:00",
            "title": "SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions",
            "authors": [
                "Han Xiao",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Knowledge representation is an important, long-history topic in AI, and there\nhave been a large amount of work for knowledge graph embedding which projects\nsymbolic entities and relations into low-dimensional, real-valued vector space.\nHowever, most embedding methods merely concentrate on data fitting and ignore\nthe explicit semantic expression, leading to uninterpretable representations.\nThus, traditional embedding methods have limited potentials for many\napplications such as question answering, and entity classification. To this\nend, this paper proposes a semantic representation method for knowledge graph\n\\textbf{(KSR)}, which imposes a two-level hierarchical generative process that\nglobally extracts many aspects and then locally assigns a specific category in\neach aspect for every triple. Since both aspects and categories are\nsemantics-relevant, the collection of categories in each aspect is treated as\nthe semantic representation of this triple. Extensive experiments justify our\nmodel outperforms other state-of-the-art baselines substantially.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.05875v1",
            "date": "2016-04-20 09:53:42+00:00",
            "title": "Distributed Entity Disambiguation with Per-Mention Learning",
            "authors": [
                "Tiep Mai",
                "Bichen Shi",
                "Patrick K. Nicholson",
                "Deepak Ajwani",
                "Alessandra Sala"
            ],
            "abstract": "Entity disambiguation, or mapping a phrase to its canonical representation in\na knowledge base, is a fundamental step in many natural language processing\napplications. Existing techniques based on global ranking models fail to\ncapture the individual peculiarities of the words and hence, either struggle to\nmeet the accuracy requirements of many real-world applications or they are too\ncomplex to satisfy real-time constraints of applications.\n  In this paper, we propose a new disambiguation system that learns specialized\nfeatures and models for disambiguating each ambiguous phrase in the English\nlanguage. To train and validate the hundreds of thousands of learning models\nfor this purpose, we use a Wikipedia hyperlink dataset with more than 170\nmillion labelled annotations. We provide an extensive experimental evaluation\nto show that the accuracy of our approach compares favourably with respect to\nmany state-of-the-art disambiguation systems. The training required for our\napproach can be easily distributed over a cluster. Furthermore, updating our\nsystem for new entities or calibrating it for special ones is a computationally\nfast process, that does not affect the disambiguation of the other entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06076v1",
            "date": "2016-04-20 19:48:07+00:00",
            "title": "Question Answering via Integer Programming over Semi-Structured Knowledge",
            "authors": [
                "Daniel Khashabi",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Peter Clark",
                "Oren Etzioni",
                "Dan Roth"
            ],
            "abstract": "Answering science questions posed in natural language is an important AI\nchallenge. Answering such questions often requires non-trivial inference and\nknowledge that goes beyond factoid retrieval. Yet, most systems for this task\nare based on relatively shallow Information Retrieval (IR) and statistical\ncorrelation techniques operating on large unstructured corpora. We propose a\nstructured inference system for this task, formulated as an Integer Linear\nProgram (ILP), that answers natural language questions using a semi-structured\nknowledge base derived from text, including questions requiring multi-step\ninference and a combination of multiple facts. On a dataset of real, unseen\nscience questions, our system significantly outperforms (+14%) the best\nprevious attempt at structured reasoning for this task, which used Markov Logic\nNetworks (MLNs). It also improves upon a previous ILP formulation by 17.7%.\nWhen combined with unstructured inference methods, the ILP system significantly\nboosts overall performance (+10%). Finally, we show our approach is\nsubstantially more robust to a simple answer perturbation compared to\nstatistical correlation methods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1604.06225v1",
            "date": "2016-04-21 09:25:11+00:00",
            "title": "OCR Error Correction Using Character Correction and Feature-Based Word Classification",
            "authors": [
                "Ido Kissos",
                "Nachum Dershowitz"
            ],
            "abstract": "This paper explores the use of a learned classifier for post-OCR text\ncorrection. Experiments with the Arabic language show that this approach, which\nintegrates a weighted confusion matrix and a shallow language model, improves\nthe vast majority of segmentation and recognition errors, the most frequent\ntypes of error on our dataset.",
            "journal ref": "Proceedings of the 12th IAPR International Workshop on Document\n  Analysis Systems (DAS 2016), Santorini, Greece, pp. 198-203 (2016)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1604.06952v3",
            "date": "2016-04-23 20:48:50+00:00",
            "title": "Visualization of Jacques Lacan's Registers of the Psychoanalytic Field, and Discovery of Metaphor and of Metonymy. Analytical Case Study of Edgar Allan Poe's \"The Purloined Letter\"",
            "authors": [
                "Fionn Murtagh",
                "Giuseppe Iurato"
            ],
            "abstract": "We start with a description of Lacan's work that we then take into our\nanalytics methodology. In a first investigation, a Lacan-motivated template of\nthe Poe story is fitted to the data. A segmentation of the storyline is used in\norder to map out the diachrony. Based on this, it will be shown how synchronous\naspects, potentially related to Lacanian registers, can be sought. This\ndemonstrates the effectiveness of an approach based on a model template of the\nstoryline narrative. In a second and more comprehensive investigation, we\ndevelop an approach for revealing, that is, uncovering, Lacanian register\nrelationships. Objectives of this work include the wide and general application\nof our methodology. This methodology is strongly based on the \"letting the data\nspeak\" Correspondence Analysis analytics platform of Jean-Paul Benz\\'ecri, that\nis also the geometric data analysis, both qualitative and quantitative\nanalytics, developed by Pierre Bourdieu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08095v1",
            "date": "2016-02-24 02:50:44+00:00",
            "title": "Accent Classification with Phonetic Vowel Representation",
            "authors": [
                "Zhenhao Ge",
                "Yingyi Tan",
                "Aravind Ganapathiraju"
            ],
            "abstract": "Previous accent classification research focused mainly on detecting accents\nwith pure acoustic information without recognizing accented speech. This work\ncombines phonetic knowledge such as vowels with acoustic information to build\nGuassian Mixture Model (GMM) classifier with Perceptual Linear Predictive (PLP)\nfeatures, optimized by Hetroscedastic Linear Discriminant Analysis (HLDA). With\ninput about 20-second accented speech, this system achieves classification rate\nof 51% on a 7-way classification system focusing on the major types of accents\nin English, which is competitive to the state-of-the-art results in this field.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1604.08504v2",
            "date": "2016-04-28 16:36:35+00:00",
            "title": "Detecting \"Smart\" Spammers On Social Network: A Topic Model Approach",
            "authors": [
                "Linqing Liu",
                "Yao Lu",
                "Ye Luo",
                "Renxian Zhang",
                "Laurent Itti",
                "Jianwei Lu"
            ],
            "abstract": "Spammer detection on social network is a challenging problem. The rigid\nanti-spam rules have resulted in emergence of \"smart\" spammers. They resemble\nlegitimate users who are difficult to identify. In this paper, we present a\nnovel spammer classification approach based on Latent Dirichlet\nAllocation(LDA), a topic model. Our approach extracts both the local and the\nglobal information of topic distribution patterns, which capture the essence of\nspamming. Tested on one benchmark dataset and one self-collected dataset, our\nproposed method outperforms other state-of-the-art methods in terms of averaged\nF1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.08781v2",
            "date": "2016-04-29 11:36:25+00:00",
            "title": "Teaching natural language to computers",
            "authors": [
                "Joseph Corneli",
                "Miriam Corneli"
            ],
            "abstract": "\"Natural Language,\" whether spoken and attended to by humans, or processed\nand generated by computers, requires networked structures that reflect creative\nprocesses in semantic, syntactic, phonetic, linguistic, social, emotional, and\ncultural modules. Being able to produce novel and useful behavior following\nrepeated practice gets to the root of both artificial intelligence and human\nlanguage. This paper investigates the modalities involved in language-like\napplications that computers -- and programmers -- engage with, and aims to fine\ntune the questions we ask to better account for context, self-awareness, and\nembodiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.00459v1",
            "date": "2016-05-02 12:38:03+00:00",
            "title": "Multi30K: Multilingual English-German Image Descriptions",
            "authors": [
                "Desmond Elliott",
                "Stella Frank",
                "Khalil Sima'an",
                "Lucia Specia"
            ],
            "abstract": "We introduce the Multi30K dataset to stimulate multilingual multimodal\nresearch. Recent advances in image description have been demonstrated on\nEnglish-language datasets almost exclusively, but image description should not\nbe limited to English. This dataset extends the Flickr30K dataset with i)\nGerman translations created by professional translators over a subset of the\nEnglish descriptions, and ii) descriptions crowdsourced independently of the\noriginal English descriptions. We outline how the data can be used for\nmultilingual image description and multimodal machine translation, but we\nanticipate the data will be useful for a broader range of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.00855v1",
            "date": "2016-05-03 12:13:26+00:00",
            "title": "Improving Image Captioning by Concept-based Sentence Reranking",
            "authors": [
                "Xirong Li",
                "Qin Jin"
            ],
            "abstract": "This paper describes our winning entry in the ImageCLEF 2015 image sentence\ngeneration task. We improve Google's CNN-LSTM model by introducing\nconcept-based sentence reranking, a data-driven approach which exploits the\nlarge amounts of concept-level annotations on Flickr. Different from previous\nusage of concept detection that is tailored to specific image captioning\nmodels, the propose approach reranks predicted sentences in terms of their\nmatches with detected concepts, essentially treating the underlying model as a\nblack box. This property makes the approach applicable to a number of existing\nsolutions. We also experiment with fine tuning on the deep language model,\nwhich improves the performance further. Scoring METEOR of 0.1875 on the\nImageCLEF 2015 test set, our system outperforms the runner-up (METEOR of\n0.1687) with a clear margin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1605.00942v2",
            "date": "2016-05-03 15:20:31+00:00",
            "title": "TheanoLM - An Extensible Toolkit for Neural Network Language Modeling",
            "authors": [
                "Seppo Enarvi",
                "Mikko Kurimo"
            ],
            "abstract": "We present a new tool for training neural network language models (NNLMs),\nscoring sentences, and generating text. The tool has been written using Python\nlibrary Theano, which allows researcher to easily extend it and tune any aspect\nof the training process. Regardless of the flexibility, Theano is able to\ngenerate extremely fast native code that can utilize a GPU or multiple CPU\ncores in order to parallelize the heavy numerical computations. The tool has\nbeen evaluated in difficult Finnish and English conversational speech\nrecognition tasks, and significant improvement was obtained over our best\nback-off n-gram models. The results that we obtained in the Finnish task were\ncompared to those from existing RNNLM and RWTHLM toolkits, and found to be as\ngood or better, while training times were an order of magnitude shorter.",
            "journal ref": "Proc. Interspeech 2016, pp. 3052-3056",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01194v1",
            "date": "2016-05-04 09:36:49+00:00",
            "title": "IISCNLP at SemEval-2016 Task 2: Interpretable STS with ILP based Multiple Chunk Aligner",
            "authors": [
                "Lavanya Sita Tekumalla",
                "Sharmistha"
            ],
            "abstract": "Interpretable semantic textual similarity (iSTS) task adds a crucial\nexplanatory layer to pairwise sentence similarity. We address various\ncomponents of this task: chunk level semantic alignment along with assignment\nof similarity type and score for aligned chunks with a novel system presented\nin this paper. We propose an algorithm, iMATCH, for the alignment of multiple\nnon-contiguous chunks based on Integer Linear Programming (ILP). Similarity\ntype and score assignment for pairs of chunks is done using a supervised\nmulticlass classification technique based on Random Forrest Classifier. Results\nshow that our algorithm iMATCH has low execution time and outperforms most\nother participating systems in terms of alignment score. Of the three datasets,\nwe are top ranked for answer- students dataset in terms of overall score and\nhave top alignment score for headlines dataset in the gold chunks track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01652v1",
            "date": "2016-05-05 17:00:44+00:00",
            "title": "LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues",
            "authors": [
                "Phong Le",
                "Marc Dymetman",
                "Jean-Michel Renders"
            ],
            "abstract": "We introduce an LSTM-based method for dynamically integrating several\nword-prediction experts to obtain a conditional language model which can be\ngood simultaneously at several subtasks. We illustrate this general approach\nwith an application to dialogue where we integrate a neural chat model, good at\nconversational aspects, with a neural question-answering model, good at\nretrieving precise information from a knowledge-base, and show how the\nintegration combines the strengths of the independent components. We hope that\nthis focused contribution will attract attention on the benefits of using such\nmixtures of experts in NLP.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1605.01661v1",
            "date": "2016-05-05 17:38:42+00:00",
            "title": "Parallels of human language in the behavior of bottlenose dolphins",
            "authors": [
                "R. Ferrer-i-Cancho",
                "D. Lusseau",
                "B. McCowan"
            ],
            "abstract": "A short review of similarities between dolphins and humans with the help of\nquantitative linguistics and information theory.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1605.02916v1",
            "date": "2016-05-10 10:03:48+00:00",
            "title": "Grammatical Case Based IS-A Relation Extraction with Boosting for Polish",
            "authors": [
                "Pawe\u0142 \u0141ozi\u0144ski",
                "Dariusz Czerski",
                "Mieczys\u0142aw A. K\u0142opotek"
            ],
            "abstract": "Pattern-based methods of IS-A relation extraction rely heavily on so called\nHearst patterns. These are ways of expressing instance enumerations of a class\nin natural language. While these lexico-syntactic patterns prove quite useful,\nthey may not capture all taxonomical relations expressed in text. Therefore in\nthis paper we describe a novel method of IS-A relation extraction from\npatterns, which uses morpho-syntactical annotations along with grammatical case\nof noun phrases that constitute entities participating in IS-A relation. We\nalso describe a method for increasing the number of extracted relations that we\ncall pseudo-subclass boosting which has potential application in any\npattern-based relation extraction method. Experiments were conducted on a\ncorpus of about 0.5 billion web documents in Polish language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02945v2",
            "date": "2016-05-10 11:29:28+00:00",
            "title": "The Yahoo Query Treebank, V. 1.0",
            "authors": [
                "Yuval Pinter",
                "Roi Reichart",
                "Idan Szpektor"
            ],
            "abstract": "A description and annotation guidelines for the Yahoo Webscope release of\nQuery Treebank, Version 1.0, May 2016.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02948v3",
            "date": "2016-05-10 11:33:33+00:00",
            "title": "Different approaches for identifying important concepts in probabilistic biomedical text summarization",
            "authors": [
                "Milad Moradi",
                "Nasser Ghadiri"
            ],
            "abstract": "Automatic text summarization tools help users in biomedical domain to acquire\ntheir intended information from various textual resources more efficiently.\nSome of the biomedical text summarization systems put the basis of their\nsentence selection approach on the frequency of concepts extracted from the\ninput text. However, it seems that exploring other measures rather than the\nfrequency for identifying the valuable content of the input document, and\nconsidering the correlations existing between concepts may be more useful for\nthis type of summarization. In this paper, we describe a Bayesian summarizer\nfor biomedical text documents. The Bayesian summarizer initially maps the input\ntext to the Unified Medical Language System (UMLS) concepts, then it selects\nthe important ones to be used as classification features. We introduce\ndifferent feature selection approaches to identify the most important concepts\nof the text and to select the most informative content according to the\ndistribution of these concepts. We show that with the use of an appropriate\nfeature selection approach, the Bayesian biomedical summarizer can improve the\nperformance of summarization. We perform extensive evaluations on a corpus of\nscientific papers in biomedical domain. The results show that the Bayesian\nsummarizer outperforms the biomedical summarizers that rely on the frequency of\nconcepts, the domain-independent and baseline methods based on the\nRecall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics. Moreover,\nthe results suggest that using the meaningfulness measure and considering the\ncorrelations of concepts in the feature selection step lead to a significant\nincrease in the performance of summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.03261v1",
            "date": "2016-05-11 02:31:21+00:00",
            "title": "Sensorimotor Input as a Language Generalisation Tool: A Neurorobotics Model for Generation and Generalisation of Noun-Verb Combinations with Sensorimotor Inputs",
            "authors": [
                "Junpei Zhong",
                "Martin Peniak",
                "Jun Tani",
                "Tetsuya Ogata",
                "Angelo Cangelosi"
            ],
            "abstract": "The paper presents a neurorobotics cognitive model to explain the\nunderstanding and generalisation of nouns and verbs combinations when a vocal\ncommand consisting of a verb-noun sentence is provided to a humanoid robot.\nThis generalisation process is done via the grounding process: different\nobjects are being interacted, and associated, with different motor behaviours,\nfollowing a learning approach inspired by developmental language acquisition in\ninfants. This cognitive model is based on Multiple Time-scale Recurrent Neural\nNetworks (MTRNN).With the data obtained from object manipulation tasks with a\nhumanoid robot platform, the robotic agent implemented with this model can\nground the primitive embodied structure of verbs through training with\nverb-noun combination samples. Moreover, we show that a functional hierarchical\narchitecture, based on MTRNN, is able to generalise and produce novel\ncombinations of noun-verb sentences. Further analyses of the learned network\ndynamics and representations also demonstrate how the generalisation is\npossible via the exploitation of this functional hierarchical recurrent\nnetwork.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1605.03481v2",
            "date": "2016-05-11 15:30:09+00:00",
            "title": "Tweet2Vec: Character-Based Distributed Representations for Social Media",
            "authors": [
                "Bhuwan Dhingra",
                "Zhong Zhou",
                "Dylan Fitzpatrick",
                "Michael Muehl",
                "William W. Cohen"
            ],
            "abstract": "Text from social media provides a set of challenges that can cause\ntraditional NLP approaches to fail. Informal language, spelling errors,\nabbreviations, and special characters are all commonplace in these posts,\nleading to a prohibitively large vocabulary size for word-level approaches. We\npropose a character composition model, tweet2vec, which finds vector-space\nrepresentations of whole tweets by learning complex, non-local dependencies in\ncharacter sequences. The proposed model outperforms a word-level baseline at\npredicting user-annotated hashtags associated with the posts, doing\nsignificantly better when the input contains many out-of-vocabulary words or\nunusual character sequences. Our tweet2vec encoder is publicly available.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1605.03705v1",
            "date": "2016-05-12 07:34:08+00:00",
            "title": "Movie Description",
            "authors": [
                "Anna Rohrbach",
                "Atousa Torabi",
                "Marcus Rohrbach",
                "Niket Tandon",
                "Christopher Pal",
                "Hugo Larochelle",
                "Aaron Courville",
                "Bernt Schiele"
            ],
            "abstract": "Audio Description (AD) provides linguistic descriptions of movies and allows\nvisually impaired people to follow a movie along with their peers. Such\ndescriptions are by design mainly visual and thus naturally form an interesting\ndata source for computer vision and computational linguistics. In this work we\npropose a novel dataset which contains transcribed ADs, which are temporally\naligned to full length movies. In addition we also collected and aligned movie\nscripts used in prior work and compare the two sources of descriptions. In\ntotal the Large Scale Movie Description Challenge (LSMDC) contains a parallel\ncorpus of 118,114 sentences and video clips from 202 movies. First we\ncharacterize the dataset by benchmarking different approaches for generating\nvideo descriptions. Comparing ADs to scripts, we find that ADs are indeed more\nvisual and describe precisely what is shown rather than what should happen\naccording to the scripts created prior to movie production. Furthermore, we\npresent and compare the results of several teams who participated in a\nchallenge organized in the context of the workshop \"Describing and\nUnderstanding Video & The Large Scale Movie Description Challenge (LSMDC)\", at\nICCV 2015.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1605.05087v3",
            "date": "2016-05-17 10:07:34+00:00",
            "title": "Word2Vec is a special case of Kernel Correspondence Analysis and Kernels for Natural Language Processing",
            "authors": [
                "Hirotaka Niitsuma",
                "Minho Lee"
            ],
            "abstract": "We show that correspondence analysis (CA) is equivalent to defining a Gini\nindex with appropriately scaled one-hot encoding. Using this relation, we\nintroduce a nonlinear kernel extension to CA. This extended CA gives a known\nanalysis for natural language via specialized kernels that use an appropriate\ncontingency table. We propose a semi-supervised CA, which is a special case of\nthe kernel extension to CA. Because CA requires excessive memory if applied to\nnumerous categories, CA has not been used for natural language processing. We\naddress this problem by introducing delayed evaluation to randomized singular\nvalue decomposition. The memory-efficient CA is then applied to a word-vector\nrepresentation task. We propose a tail-cut kernel, which is an extension to the\nskip-gram within the kernel extension to CA. Our tail-cut kernel outperforms\nexisting word-vector representation methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1605.05156v1",
            "date": "2016-05-17 13:31:14+00:00",
            "title": "Tweet Acts: A Speech Act Classifier for Twitter",
            "authors": [
                "Soroush Vosoughi",
                "Deb Roy"
            ],
            "abstract": "Speech acts are a way to conceptualize speech as action. This holds true for\ncommunication on any platform, including social media platforms such as\nTwitter. In this paper, we explored speech act recognition on Twitter by\ntreating it as a multi-class classification problem. We created a taxonomy of\nsix speech acts for Twitter and proposed a set of semantic and syntactic\nfeatures. We trained and tested a logistic regression classifier using a data\nset of manually labelled tweets. Our method achieved a state-of-the-art\nperformance with an average F1 score of more than $0.70$. We also explored\nclassifiers with three different granularities (Twitter-wide, type-specific and\ntopic-specific) in order to find the right balance between generalization and\noverfitting for our task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05303v1",
            "date": "2016-05-17 19:45:49+00:00",
            "title": "Fuzzy Sets Across the Natural Language Generation Pipeline",
            "authors": [
                "A. Ramos-Soto",
                "A. Bugar\u00edn",
                "S. Barro"
            ],
            "abstract": "We explore the implications of using fuzzy techniques (mainly those commonly\nused in the linguistic description/summarization of data discipline) from a\nnatural language generation perspective. For this, we provide an extensive\ndiscussion of some general convergence points and an exploration of the\nrelationship between the different tasks involved in the standard NLG system\npipeline architecture and the most common fuzzy approaches used in linguistic\nsummarization/description of data, such as fuzzy quantified statements,\nevaluation criteria or aggregation operators. Each individual discussion is\nillustrated with a related use case. Recent work made in the context of\ncross-fertilization of both research fields is also referenced. This paper\nencompasses general ideas that emerged as part of the PhD thesis \"Application\nof fuzzy sets in data-to-text systems\". It does not present a specific\napplication or a formal approach, but rather discusses current high-level\nissues and potential usages of fuzzy sets (focused on linguistic summarization\nof data) in natural language generation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1605.05414v2",
            "date": "2016-05-18 01:36:29+00:00",
            "title": "On the Evaluation of Dialogue Systems with Next Utterance Classification",
            "authors": [
                "Ryan Lowe",
                "Iulian V. Serban",
                "Mike Noseworthy",
                "Laurent Charlin",
                "Joelle Pineau"
            ],
            "abstract": "An open challenge in constructing dialogue systems is developing methods for\nautomatically learning dialogue strategies from large amounts of unlabelled\ndata. Recent work has proposed Next-Utterance-Classification (NUC) as a\nsurrogate task for building dialogue systems from text data. In this paper we\ninvestigate the performance of humans on this task to validate the relevance of\nNUC as a method of evaluation. Our results show three main findings: (1) humans\nare able to correctly classify responses at a rate much better than chance,\nthus confirming that the task is feasible, (2) human performance levels vary\nacross task domains (we consider 3 datasets) and expertise levels (novice vs\nexperts), thus showing that a range of performance is possible on this type of\ntask, (3) automated dialogue systems built using state-of-the-art machine\nlearning methods have similar performance to the human novices, but worse than\nthe experts, thus confirming the utility of this class of tasks for driving\nfurther research in automated dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05433v2",
            "date": "2016-05-18 04:10:41+00:00",
            "title": "Relations such as Hypernymy: Identifying and Exploiting Hearst Patterns in Distributional Vectors for Lexical Entailment",
            "authors": [
                "Stephen Roller",
                "Katrin Erk"
            ],
            "abstract": "We consider the task of predicting lexical entailment using distributional\nvectors. We perform a novel qualitative analysis of one existing model which\nwas previously shown to only measure the prototypicality of word pairs. We find\nthat the model strongly learns to identify hypernyms using Hearst patterns,\nwhich are well known to be predictive of lexical relations. We present a novel\nmodel which exploits this behavior as a method of feature extraction in an\niterative procedure similar to Principal Component Analysis. Our model combines\nthe extracted features with the strengths of other proposed models in the\nliterature, and matches or outperforms prior work on multiple data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06083v1",
            "date": "2016-05-19 19:17:23+00:00",
            "title": "Stereotyping and Bias in the Flickr30K Dataset",
            "authors": [
                "Emiel van Miltenburg"
            ],
            "abstract": "An untested assumption behind the crowdsourced descriptions of the images in\nthe Flickr30K dataset (Young et al., 2014) is that they \"focus only on the\ninformation that can be obtained from the image alone\" (Hodosh et al., 2013, p.\n859). This paper presents some evidence against this assumption, and provides a\nlist of biases and unwarranted inferences that can be found in the Flickr30K\ndataset. Finally, it considers methods to find examples of these, and discusses\nhow we should deal with stereotype-driven descriptions in future applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06319v1",
            "date": "2016-05-20 12:20:27+00:00",
            "title": "As Cool as a Cucumber: Towards a Corpus of Contemporary Similes in Serbian",
            "authors": [
                "Nikola Milosevic",
                "Goran Nenadic"
            ],
            "abstract": "Similes are natural language expressions used to compare unlikely things,\nwhere the comparison is not taken literally. They are often used in everyday\ncommunication and are an important part of cultural heritage. Having an\nup-to-date corpus of similes is challenging, as they are constantly coined\nand/or adapted to the contemporary times. In this paper we present a\nmethodology for semi-automated collection of similes from the world wide web\nusing text mining techniques. We expanded an existing corpus of traditional\nsimiles (containing 333 similes) by collecting 446 additional expressions. We,\nalso, explore how crowdsourcing can be used to extract and curate new similes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06799v1",
            "date": "2016-05-22 14:22:03+00:00",
            "title": "Textual Paralanguage and its Implications for Marketing Communications",
            "authors": [
                "Andrea Webb Luangrath",
                "Joann Peck",
                "Victor A. Barger"
            ],
            "abstract": "Both face-to-face communication and communication in online environments\nconvey information beyond the actual verbal message. In a traditional\nface-to-face conversation, paralanguage, or the ancillary meaning- and\nemotion-laden aspects of speech that are not actual verbal prose, gives\ncontextual information that allows interactors to more appropriately understand\nthe message being conveyed. In this paper, we conceptualize textual\nparalanguage (TPL), which we define as written manifestations of nonverbal\naudible, tactile, and visual elements that supplement or replace written\nlanguage and that can be expressed through words, symbols, images, punctuation,\ndemarcations, or any combination of these elements. We develop a typology of\ntextual paralanguage using data from Twitter, Facebook, and Instagram. We\npresent a conceptual framework of antecedents and consequences of brands' use\nof textual paralanguage. Implications for theory and practice are discussed.",
            "journal ref": "Journal of Consumer Psychology 27 (2017) 98-107",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07268v1",
            "date": "2016-05-24 02:39:26+00:00",
            "title": "Classifying discourse in a CSCL platform to evaluate correlations with Teacher Participation and Progress",
            "authors": [
                "Eliana Scheihing",
                "Matthieu Vernier",
                "Javiera Born",
                "Julio Guerra",
                "Luis Carcamo"
            ],
            "abstract": "In Computer-Supported learning, monitoring and engaging a group of learners\nis a complex task for teachers, especially when learners are working\ncollaboratively: Are my students motivated? What kind of progress are they\nmaking? Should I intervene? Is my communication and the didactic design adapted\nto my students? Our hypothesis is that the analysis of natural language\ninteractions between students, and between students and teachers, provide very\nvaluable information and could be used to produce qualitative indicators to\nhelp teachers' decisions. We develop an automatic approach in three steps (1)\nto explore the discursive functions of messages in a CSCL platform, (2) to\nclassify the messages automatically and (3) to evaluate correlations between\ndiscursive attitudes and other variables linked to the learning activity.\nResults tend to show that some types of discourse are correlated with a notion\nof Progress on the learning activities and the importance of emotive\nparticipation from the Teacher.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1605.07669v2",
            "date": "2016-05-24 21:56:08+00:00",
            "title": "On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems",
            "authors": [
                "Pei-Hao Su",
                "Milica Gasic",
                "Nikola Mrksic",
                "Lina Rojas-Barahona",
                "Stefan Ultes",
                "David Vandyke",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "The ability to compute an accurate reward function is essential for\noptimising a dialogue policy via reinforcement learning. In real-world\napplications, using explicit user feedback as the reward signal is often\nunreliable and costly to collect. This problem can be mitigated if the user's\nintent is known in advance or data is available to pre-train a task success\npredictor off-line. In practice neither of these apply for most real world\napplications. Here we propose an on-line learning framework whereby the\ndialogue policy is jointly trained alongside the reward model via active\nlearning with a Gaussian process model. This Gaussian process operates on a\ncontinuous space dialogue representation generated in an unsupervised fashion\nusing a recurrent neural network encoder-decoder. The experimental results\ndemonstrate that the proposed framework is able to significantly reduce data\nannotation costs and mitigate noisy user feedback in dialogue policy learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07852v2",
            "date": "2016-05-25 12:25:26+00:00",
            "title": "SS4MCT: A Statistical Stemmer for Morphologically Complex Texts",
            "authors": [
                "Javid Dadashkarimi",
                "Hossein Nasr Esfahani",
                "Heshaam Faili",
                "Azadeh Shakery"
            ],
            "abstract": "There have been multiple attempts to resolve various inflection matching\nproblems in information retrieval. Stemming is a common approach to this end.\nAmong many techniques for stemming, statistical stemming has been shown to be\neffective in a number of languages, particularly highly inflected languages. In\nthis paper we propose a method for finding affixes in different positions of a\nword. Common statistical techniques heavily rely on string similarity in terms\nof prefix and suffix matching. Since infixes are common in irregular/informal\ninflections in morphologically complex texts, it is required to find infixes\nfor stemming. In this paper we propose a method whose aim is to find\nstatistical inflectional rules based on minimum edit distance table of word\npairs and the likelihoods of the rules in a language. These rules are used to\nstatistically stem words and can be used in different text mining tasks.\nExperimental results on CLEF 2008 and CLEF 2009 English-Persian CLIR tasks\nindicate that the proposed method significantly outperforms all the baselines\nin terms of MAP.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1605.07891v2",
            "date": "2016-05-25 14:09:00+00:00",
            "title": "Query Expansion with Locally-Trained Word Embeddings",
            "authors": [
                "Fernando Diaz",
                "Bhaskar Mitra",
                "Nick Craswell"
            ],
            "abstract": "Continuous space word embeddings have received a great deal of attention in\nthe natural language processing and machine learning communities for their\nability to model term similarity and other relationships. We study the use of\nterm relatedness in the context of query expansion for ad hoc information\nretrieval. We demonstrate that word embeddings such as word2vec and GloVe, when\ntrained globally, underperform corpus and query specific embeddings for\nretrieval tasks. These results suggest that other tasks benefiting from global\nembeddings may also benefit from local embeddings.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.00253v1",
            "date": "2016-06-01 12:34:50+00:00",
            "title": "On a Topic Model for Sentences",
            "authors": [
                "Georgios Balikas",
                "Massih-Reza Amini",
                "Marianne Clausel"
            ],
            "abstract": "Probabilistic topic models are generative models that describe the content of\ndocuments by discovering the latent topics underlying them. However, the\nstructure of the textual input, and for instance the grouping of words in\ncoherent text spans such as sentences, contains much information which is\ngenerally lost with these models. In this paper, we propose sentenceLDA, an\nextension of LDA whose goal is to overcome this limitation by incorporating the\nstructure of the text in the generative and inference processes. We illustrate\nthe advantages of sentenceLDA by comparing it with LDA using both intrinsic\n(perplexity) and extrinsic (text classification) evaluation tasks on different\ntext collections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00411v1",
            "date": "2016-06-01 19:30:07+00:00",
            "title": "Temporal Topic Modeling to Assess Associations between News Trends and Infectious Disease Outbreaks",
            "authors": [
                "Saurav Ghosh",
                "Prithwish Chakraborty",
                "Elaine O. Nsoesie",
                "Emily Cohn",
                "Sumiko R. Mekaru",
                "John S. Brownstein",
                "Naren Ramakrishnan"
            ],
            "abstract": "In retrospective assessments, internet news reports have been shown to\ncapture early reports of unknown infectious disease transmission prior to\nofficial laboratory confirmation. In general, media interest and reporting\npeaks and wanes during the course of an outbreak. In this study, we quantify\nthe extent to which media interest during infectious disease outbreaks is\nindicative of trends of reported incidence. We introduce an approach that uses\nsupervised temporal topic models to transform large corpora of news articles\ninto temporal topic trends. The key advantages of this approach include,\napplicability to a wide range of diseases, and ability to capture disease\ndynamics - including seasonality, abrupt peaks and troughs. We evaluated the\nmethod using data from multiple infectious disease outbreaks reported in the\nUnited States of America (U.S.), China and India. We noted that temporal topic\ntrends extracted from disease-related news reports successfully captured the\ndynamics of multiple outbreaks such as whooping cough in U.S. (2012), dengue\noutbreaks in India (2013) and China (2014). Our observations also suggest that\nefficient modeling of temporal topic trends using time-series regression\ntechniques can estimate disease case counts with increased precision before\nofficial reports by health organizations.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1606.00577v3",
            "date": "2016-06-02 08:15:15+00:00",
            "title": "Source-LDA: Enhancing probabilistic topic models using prior knowledge sources",
            "authors": [
                "Justin Wood",
                "Patrick Tan",
                "Wei Wang",
                "Corey Arnold"
            ],
            "abstract": "A popular approach to topic modeling involves extracting co-occurring n-grams\nof a corpus into semantic themes. The set of n-grams in a theme represents an\nunderlying topic, but most topic modeling approaches are not able to label\nthese sets of words with a single n-gram. Such labels are useful for topic\nidentification in summarization systems. This paper introduces a novel approach\nto labeling a group of n-grams comprising an individual topic. The approach\ntaken is to complement the existing topic distributions over words with a known\ndistribution based on a predefined set of topics. This is done by integrating\nexisting labeled knowledge sources representing known potential topics into the\nprobabilistic topic model. These knowledge sources are translated into a\ndistribution and used to set the hyperparameters of the Dirichlet generated\ndistribution over words. In the inference these modified distributions guide\nthe convergence of the latent topics to conform with the complementary\ndistributions. This approach ensures that the topic inference process is\nconsistent with existing knowledge. The label assignment from the complementary\nknowledge sources are then transferred to the latent topics of the corpus. The\nresults show both accurate label assignment to topics as well as improved topic\ngeneration than those obtained using various labeling approaches based off\nLatent Dirichlet allocation (LDA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00739v2",
            "date": "2016-06-02 16:06:29+00:00",
            "title": "Stochastic Structured Prediction under Bandit Feedback",
            "authors": [
                "Artem Sokolov",
                "Julia Kreutzer",
                "Christopher Lo",
                "Stefan Riezler"
            ],
            "abstract": "Stochastic structured prediction under bandit feedback follows a learning\nprotocol where on each of a sequence of iterations, the learner receives an\ninput, predicts an output structure, and receives partial feedback in form of a\ntask loss evaluation of the predicted structure. We present applications of\nthis learning scenario to convex and non-convex objectives for structured\nprediction and analyze them as stochastic first-order methods. We present an\nexperimental evaluation on problems of natural language processing over\nexponential output spaces, and compare convergence speed across different\nobjectives under the practical criterion of optimal task performance on\ndevelopment data and the optimization-theoretic criterion of minimal squared\ngradient norm. Best results under both criteria are obtained for a non-convex\nobjective for pairwise preference learning under bandit feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00979v1",
            "date": "2016-06-03 06:40:14+00:00",
            "title": "Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information",
            "authors": [
                "Yuanzhe Zhang",
                "Kang Liu",
                "Shizhu He",
                "Guoliang Ji",
                "Zhanyi Liu",
                "Hua Wu",
                "Jun Zhao"
            ],
            "abstract": "With the rapid growth of knowledge bases (KBs) on the web, how to take full\nadvantage of them becomes increasingly important. Knowledge base-based question\nanswering (KB-QA) is one of the most promising approaches to access the\nsubstantial knowledge. Meantime, as the neural network-based (NN-based) methods\ndevelop, NN-based KB-QA has already achieved impressive results. However,\nprevious work did not put emphasis on question representation, and the question\nis converted into a fixed vector regardless of its candidate answers. This\nsimple representation strategy is unable to express the proper information of\nthe question. Hence, we present a neural attention-based model to represent the\nquestions dynamically according to the different focuses of various candidate\nanswer aspects. In addition, we leverage the global knowledge inside the\nunderlying KB, aiming at integrating the rich KB information into the\nrepresentation of the answers. And it also alleviates the out of vocabulary\n(OOV) problem, which helps the attention model to represent the question more\nprecisely. The experimental results on WEBQUESTIONS demonstrate the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.01219v1",
            "date": "2016-06-03 18:42:14+00:00",
            "title": "Learning Stylometric Representations for Authorship Analysis",
            "authors": [
                "Steven H. H. Ding",
                "Benjamin C. M. Fung",
                "Farkhund Iqbal",
                "William K. Cheung"
            ],
            "abstract": "Authorship analysis (AA) is the study of unveiling the hidden properties of\nauthors from a body of exponentially exploding textual data. It extracts an\nauthor's identity and sociolinguistic characteristics based on the reflected\nwriting styles in the text. It is an essential process for various areas, such\nas cybercrime investigation, psycholinguistics, political socialization, etc.\nHowever, most of the previous techniques critically depend on the manual\nfeature engineering process. Consequently, the choice of feature set has been\nshown to be scenario- or dataset-dependent. In this paper, to mimic the human\nsentence composition process using a neural network approach, we propose to\nincorporate different categories of linguistic features into distributed\nrepresentation of words in order to learn simultaneously the writing style\nrepresentations based on unlabeled texts for authorship analysis. In\nparticular, the proposed models allow topical, lexical, syntactical, and\ncharacter-level feature vectors of each document to be extracted as\nstylometrics. We evaluate the performance of our approach on the problems of\nauthorship characterization and authorship verification with the Twitter,\nnovel, and essay datasets. The experiments suggest that our proposed text\nrepresentation outperforms the bag-of-lexical-n-grams, Latent Dirichlet\nAllocation, Latent Semantic Analysis, PVDM, PVDBOW, and word2vec\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01269v1",
            "date": "2016-06-03 20:32:52+00:00",
            "title": "End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning",
            "authors": [
                "Jason D. Williams",
                "Geoffrey Zweig"
            ],
            "abstract": "This paper presents a model for end-to-end learning of task-oriented dialog\nsystems. The main component of the model is a recurrent neural network (an\nLSTM), which maps from raw dialog history directly to a distribution over\nsystem actions. The LSTM automatically infers a representation of dialog\nhistory, which relieves the system developer of much of the manual feature\nengineering of dialog state. In addition, the developer can provide software\nthat expresses business rules and provides access to programmatic APIs,\nenabling the LSTM to take actions in the real world on behalf of the user. The\nLSTM can be optimized using supervised learning (SL), where a domain expert\nprovides example dialogs which the LSTM should imitate; or using reinforcement\nlearning (RL), where the system improves by interacting directly with end\nusers. Experiments show that SL and RL are complementary: SL alone can derive a\nreasonable initial policy from a small number of training dialogs; and starting\nRL optimization with a policy trained with SL substantially accelerates the\nlearning rate of RL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01305v4",
            "date": "2016-06-03 23:31:47+00:00",
            "title": "Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations",
            "authors": [
                "David Krueger",
                "Tegan Maharaj",
                "J\u00e1nos Kram\u00e1r",
                "Mohammad Pezeshki",
                "Nicolas Ballas",
                "Nan Rosemary Ke",
                "Anirudh Goyal",
                "Yoshua Bengio",
                "Aaron Courville",
                "Chris Pal"
            ],
            "abstract": "We propose zoneout, a novel method for regularizing RNNs. At each timestep,\nzoneout stochastically forces some hidden units to maintain their previous\nvalues. Like dropout, zoneout uses random noise to train a pseudo-ensemble,\nimproving generalization. But by preserving instead of dropping hidden units,\ngradient information and state information are more readily propagated through\ntime, as in feedforward stochastic depth networks. We perform an empirical\ninvestigation of various RNN regularizers, and find that zoneout gives\nsignificant performance improvements across tasks. We achieve competitive\nresults with relatively simple models in character- and word-level language\nmodelling on the Penn Treebank and Text8 datasets, and combining with recurrent\nbatch normalization yields state-of-the-art results on permuted sequential\nMNIST.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1606.01404v1",
            "date": "2016-06-04 18:34:51+00:00",
            "title": "Generating Natural Language Inference Chains",
            "authors": [
                "Vladyslav Kolesnyk",
                "Tim Rockt\u00e4schel",
                "Sebastian Riedel"
            ],
            "abstract": "The ability to reason with natural language is a fundamental prerequisite for\nmany NLP tasks such as information extraction, machine translation and question\nanswering. To quantify this ability, systems are commonly tested whether they\ncan recognize textual entailment, i.e., whether one sentence can be inferred\nfrom another one. However, in most NLP applications only single source\nsentences instead of sentence pairs are available. Hence, we propose a new task\nthat measures how well a model can generate an entailed sentence from a source\nsentence. We take entailment-pairs of the Stanford Natural Language Inference\ncorpus and train an LSTM with attention. On a manually annotated test set we\nfound that 82% of generated sentences are correct, an improvement of 10.3% over\nan LSTM baseline. A qualitative analysis shows that this model is not only\ncapable of shortening input sentences, but also inferring new statements via\nparaphrasing and phrase entailment. We then apply this model recursively to\ninput-output pairs, thereby generating natural language inference chains that\ncan be used to automatically construct an entailment graph from source\nsentences. Finally, by swapping source and target sentences we can also train a\nmodel that given an input sentence invents additional information to generate a\nnew sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01515v2",
            "date": "2016-06-05 14:26:56+00:00",
            "title": "Coordination in Categorical Compositional Distributional Semantics",
            "authors": [
                "Dimitri Kartsaklis"
            ],
            "abstract": "An open problem with categorical compositional distributional semantics is\nthe representation of words that are considered semantically vacuous from a\ndistributional perspective, such as determiners, prepositions, relative\npronouns or coordinators. This paper deals with the topic of coordination\nbetween identical syntactic types, which accounts for the majority of\ncoordination cases in language. By exploiting the compact closed structure of\nthe underlying category and Frobenius operators canonically induced over the\nfixed basis of finite-dimensional vector spaces, we provide a morphism as\nrepresentation of a coordinator tensor, and we show how it lifts from atomic\ntypes to compound types. Linguistic intuitions are provided, and the importance\nof the Frobenius operators as an addition to the compact closed setting with\nregard to language is discussed.",
            "journal ref": "EPTCS 221, 2016, pp. 29-38",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01781v2",
            "date": "2016-06-06 15:14:50+00:00",
            "title": "Very Deep Convolutional Networks for Text Classification",
            "authors": [
                "Alexis Conneau",
                "Holger Schwenk",
                "Lo\u00efc Barrault",
                "Yann Lecun"
            ],
            "abstract": "The dominant approach for many NLP tasks are recurrent neural networks, in\nparticular LSTMs, and convolutional neural networks. However, these\narchitectures are rather shallow in comparison to the deep convolutional\nnetworks which have pushed the state-of-the-art in computer vision. We present\na new architecture (VDCNN) for text processing which operates directly at the\ncharacter level and uses only small convolutions and pooling operations. We are\nable to show that the performance of this model increases with depth: using up\nto 29 convolutional layers, we report improvements over the state-of-the-art on\nseveral public text classification tasks. To the best of our knowledge, this is\nthe first time that very deep convolutional nets have been applied to text\nprocessing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.01847v3",
            "date": "2016-06-06 17:59:56+00:00",
            "title": "Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding",
            "authors": [
                "Akira Fukui",
                "Dong Huk Park",
                "Daylen Yang",
                "Anna Rohrbach",
                "Trevor Darrell",
                "Marcus Rohrbach"
            ],
            "abstract": "Modeling textual or visual information with vector representations trained\nfrom large language or visual datasets has been successfully explored in recent\nyears. However, tasks such as visual question answering require combining these\nvector representations with each other. Approaches to multimodal pooling\ninclude element-wise product or sum, as well as concatenation of the visual and\ntextual representations. We hypothesize that these methods are not as\nexpressive as an outer product of the visual and textual vectors. As the outer\nproduct is typically infeasible due to its high dimensionality, we instead\npropose utilizing Multimodal Compact Bilinear pooling (MCB) to efficiently and\nexpressively combine multimodal features. We extensively evaluate MCB on the\nvisual question answering and grounding tasks. We consistently show the benefit\nof MCB over ablations without MCB. For visual question answering, we present an\narchitecture which uses MCB twice, once for predicting attention over spatial\nfeatures and again to combine the attended representation with the question\nrepresentation. This model outperforms the state-of-the-art on the Visual7W\ndataset and the VQA challenge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.02276v1",
            "date": "2016-06-07 19:40:00+00:00",
            "title": "Multilingual Visual Sentiment Concept Matching",
            "authors": [
                "Nikolaos Pappas",
                "Miriam Redi",
                "Mercan Topkara",
                "Brendan Jou",
                "Hongyi Liu",
                "Tao Chen",
                "Shih-Fu Chang"
            ],
            "abstract": "The impact of culture in visual emotion perception has recently captured the\nattention of multimedia research. In this study, we pro- vide powerful\ncomputational linguistics tools to explore, retrieve and browse a dataset of\n16K multilingual affective visual concepts and 7.3M Flickr images. First, we\ndesign an effective crowdsourc- ing experiment to collect human judgements of\nsentiment connected to the visual concepts. We then use word embeddings to\nrepre- sent these concepts in a low dimensional vector space, allowing us to\nexpand the meaning around concepts, and thus enabling insight about\ncommonalities and differences among different languages. We compare a variety\nof concept representations through a novel evaluation task based on the notion\nof visual semantic relatedness. Based on these representations, we design\nclustering schemes to group multilingual visual concepts, and evaluate them\nwith novel metrics based on the crowdsourced sentiment annotations as well as\nvisual semantic relatedness. The proposed clustering framework enables us to\nanalyze the full multilingual dataset in-depth and also show an application on\na facial data subset, exploring cultural in- sights of portrait-related\naffective visual concepts.",
            "journal ref": "Proceedings ICMR '16 Proceedings of the 2016 ACM on International\n  Conference on Multimedia Retrieval Pages 151-158",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02440v1",
            "date": "2016-06-08 08:11:54+00:00",
            "title": "On the Place of Text Data in Lifelogs, and Text Analysis via Semantic Facets",
            "authors": [
                "Gregory Grefenstette",
                "Lawrence Muchemi"
            ],
            "abstract": "Current research in lifelog data has not paid enough attention to analysis of\ncognitive activities in comparison to physical activities. We argue that as we\nlook into the future, wearable devices are going to be cheaper and more\nprevalent and textual data will play a more significant role. Data captured by\nlifelogging devices will increasingly include speech and text, potentially\nuseful in analysis of intellectual activities. Analyzing what a person hears,\nreads, and sees, we should be able to measure the extent of cognitive activity\ndevoted to a certain topic or subject by a learner. Test-based lifelog records\ncan benefit from semantic analysis tools developed for natural language\nprocessing. We show how semantic analysis of such text data can be achieved\nthrough the use of taxonomic subject facets and how these facets might be\nuseful in quantifying cognitive activity devoted to various topics in a\nperson's day. We are currently developing a method to automatically create\ntaxonomic topic vocabularies that can be applied to this detection of\nintellectual activity.",
            "journal ref": "iConference 2016 SIE on Lifelogging, Mar 2016, Philadelphia,\n  United States. iConference 2016 SIE on Lifelogging, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02555v1",
            "date": "2016-06-08 13:47:18+00:00",
            "title": "Improving Recurrent Neural Networks For Sequence Labelling",
            "authors": [
                "Marco Dinarelli",
                "Isabelle Tellier"
            ],
            "abstract": "In this paper we study different types of Recurrent Neural Networks (RNN) for\nsequence labeling tasks. We propose two new variants of RNNs integrating\nimprovements for sequence labeling, and we compare them to the more traditional\nElman and Jordan RNNs. We compare all models, either traditional or new, on\nfour distinct tasks of sequence labeling: two on Spoken Language Understanding\n(ATIS and MEDIA); and two of POS tagging for the French Treebank (FTB) and the\nPenn Treebank (PTB) corpora. The results show that our new variants of RNNs are\nalways more effective than the others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02560v2",
            "date": "2016-06-08 14:03:25+00:00",
            "title": "Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning",
            "authors": [
                "Tiancheng Zhao",
                "Maxine Eskenazi"
            ],
            "abstract": "This paper presents an end-to-end framework for task-oriented dialog systems\nusing a variant of Deep Recurrent Q-Networks (DRQN). The model is able to\ninterface with a relational database and jointly learn policies for both\nlanguage understanding and dialog strategy. Moreover, we propose a hybrid\nalgorithm that combines the strength of reinforcement learning and supervised\nlearning to achieve faster learning speed. We evaluated the proposed model on a\n20 Question Game conversational game simulator. Results show that the proposed\nmethod outperforms the modular-based baseline and learns a distributed\nrepresentation of the latent dialog state.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.02960v2",
            "date": "2016-06-09 13:29:34+00:00",
            "title": "Sequence-to-Sequence Learning as Beam-Search Optimization",
            "authors": [
                "Sam Wiseman",
                "Alexander M. Rush"
            ],
            "abstract": "Sequence-to-Sequence (seq2seq) modeling has rapidly become an important\ngeneral-purpose NLP tool that has proven effective for many text-generation and\nsequence-labeling tasks. Seq2seq builds on deep neural language modeling and\ninherits its remarkable accuracy in estimating local, next-word distributions.\nIn this work, we introduce a model and beam-search training scheme, based on\nthe work of Daume III and Marcu (2005), that extends seq2seq to learn global\nsequence scores. This structured approach avoids classical biases associated\nwith local training and unifies the training loss with the test-time usage,\nwhile preserving the proven model architecture of seq2seq and its efficient\ntraining approach. We show that our system outperforms a highly-optimized\nattention-based seq2seq system and other baselines on three different sequence\nto sequence tasks: word ordering, parsing, and machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03002v1",
            "date": "2016-06-09 15:41:17+00:00",
            "title": "MuFuRU: The Multi-Function Recurrent Unit",
            "authors": [
                "Dirk Weissenborn",
                "Tim Rockt\u00e4schel"
            ],
            "abstract": "Recurrent neural networks such as the GRU and LSTM found wide adoption in\nnatural language processing and achieve state-of-the-art results for many\ntasks. These models are characterized by a memory state that can be written to\nand read from by applying gated composition operations to the current input and\nthe previous state. However, they only cover a small subset of potentially\nuseful compositions. We propose Multi-Function Recurrent Units (MuFuRUs) that\nallow for arbitrary differentiable functions as composition operations.\nFurthermore, MuFuRUs allow for an input- and state-dependent choice of these\ncomposition operations that is learned. Our experiments demonstrate that the\nadditional functionality helps in different sequence modeling tasks, including\nthe evaluation of propositional logic formulae, language modeling and sentiment\nanalysis.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1606.03144v1",
            "date": "2016-06-09 23:42:45+00:00",
            "title": "Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays",
            "authors": [
                "Marek Rei",
                "Ronan Cummins"
            ],
            "abstract": "We investigate the task of assessing sentence-level prompt relevance in\nlearner essays. Various systems using word overlap, neural embeddings and\nneural compositional models are evaluated on two datasets of learner writing.\nWe propose a new method for sentence-level similarity calculation, which learns\nto adjust the weights of pre-trained word embeddings for a specific task,\nachieving substantially higher accuracy compared to other relevant baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03207v2",
            "date": "2016-06-10 06:44:21+00:00",
            "title": "Deep CNNs along the Time Axis with Intermap Pooling for Robustness to Spectral Variations",
            "authors": [
                "Hwaran Lee",
                "Geonmin Kim",
                "Ho-Gyeong Kim",
                "Sang-Hoon Oh",
                "Soo-Young Lee"
            ],
            "abstract": "Convolutional neural networks (CNNs) with convolutional and pooling\noperations along the frequency axis have been proposed to attain invariance to\nfrequency shifts of features. However, this is inappropriate with regard to the\nfact that acoustic features vary in frequency. In this paper, we contend that\nconvolution along the time axis is more effective. We also propose the addition\nof an intermap pooling (IMP) layer to deep CNNs. In this layer, filters in each\ngroup extract common but spectrally variant features, then the layer pools the\nfeature maps of each group. As a result, the proposed IMP CNN can achieve\ninsensitivity to spectral variations characteristic of different speakers and\nutterances. The effectiveness of the IMP CNN architecture is demonstrated on\nseveral LVCSR tasks. Even without speaker adaptation techniques, the\narchitecture achieved a WER of 12.7% on the SWB part of the Hub5'2000\nevaluation test set, which is competitive with other state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03333v1",
            "date": "2016-06-10 14:09:32+00:00",
            "title": "Automatic Genre and Show Identification of Broadcast Media",
            "authors": [
                "Mortaza Doulaty",
                "Oscar Saz",
                "Raymond W. M. Ng",
                "Thomas Hain"
            ],
            "abstract": "Huge amounts of digital videos are being produced and broadcast every day,\nleading to giant media archives. Effective techniques are needed to make such\ndata accessible further. Automatic meta-data labelling of broadcast media is an\nessential task for multimedia indexing, where it is standard to use multi-modal\ninput for such purposes. This paper describes a novel method for automatic\ndetection of media genre and show identities using acoustic features, textual\nfeatures or a combination thereof. Furthermore the inclusion of available\nmeta-data, such as time of broadcast, is shown to lead to very high\nperformance. Latent Dirichlet Allocation is used to model both acoustics and\ntext, yielding fixed dimensional representations of media recordings that can\nthen be used in Support Vector Machines based classification. Experiments are\nconducted on more than 1200 hours of TV broadcasts from the British\nBroadcasting Corporation (BBC), where the task is to categorise the broadcasts\ninto 8 genres or 133 show identities. On a 200-hour test set, accuracies of\n98.6% and 85.7% were achieved for genre and show identification respectively,\nusing a combination of acoustic and textual features with meta-data.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1606.03335v1",
            "date": "2016-06-10 14:12:47+00:00",
            "title": "WordNet2Vec: Corpora Agnostic Word Vectorization Method",
            "authors": [
                "Roman Bartusiak",
                "\u0141ukasz Augustyniak",
                "Tomasz Kajdanowicz",
                "Przemys\u0142aw Kazienko",
                "Maciej Piasecki"
            ],
            "abstract": "A complex nature of big data resources demands new methods for structuring\nespecially for textual content. WordNet is a good knowledge source for\ncomprehensive abstraction of natural language as its good implementations exist\nfor many languages. Since WordNet embeds natural language in the form of a\ncomplex network, a transformation mechanism WordNet2Vec is proposed in the\npaper. It creates vectors for each word from WordNet. These vectors encapsulate\ngeneral position - role of a given word towards all other words in the natural\nlanguage. Any list or set of such vectors contains knowledge about the context\nof its component within the whole language. Such word representation can be\neasily applied to many analytic tasks like classification or clustering. The\nusefulness of the WordNet2Vec method was demonstrated in sentiment analysis,\ni.e. classification with transfer learning for the real Amazon opinion textual\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03352v1",
            "date": "2016-06-10 14:56:19+00:00",
            "title": "Conditional Generation and Snapshot Learning in Neural Dialogue Systems",
            "authors": [
                "Tsung-Hsien Wen",
                "Milica Gasic",
                "Nikola Mrksic",
                "Lina M. Rojas-Barahona",
                "Pei-Hao Su",
                "Stefan Ultes",
                "David Vandyke",
                "Steve Young"
            ],
            "abstract": "Recently a variety of LSTM-based conditional language models (LM) have been\napplied across a range of language generation tasks. In this work we study\nvarious model architectures and different ways to represent and aggregate the\nsource information in an end-to-end neural dialogue system framework. A method\ncalled snapshot learning is also proposed to facilitate learning from\nsupervised sequential signals by applying a companion cross-entropy objective\nfunction to the conditioning vector. The experimental and analytical results\ndemonstrate firstly that competition occurs between the conditioning vector and\nthe LM, and the differing architectures provide different trade-offs between\nthe two. Secondly, the discriminative power and transparency of the\nconditioning vector is key to providing both model interpretability and better\nperformance. Thirdly, snapshot learning leads to consistent performance\nimprovements independent of which architecture is used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03475v1",
            "date": "2016-06-10 20:45:30+00:00",
            "title": "De-identification of Patient Notes with Recurrent Neural Networks",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee",
                "Ozlem Uzuner",
                "Peter Szolovits"
            ],
            "abstract": "Objective: Patient notes in electronic health records (EHRs) may contain\ncritical information for medical investigations. However, the vast majority of\nmedical investigators can only access de-identified notes, in order to protect\nthe confidentiality of patients. In the United States, the Health Insurance\nPortability and Accountability Act (HIPAA) defines 18 types of protected health\ninformation (PHI) that needs to be removed to de-identify patient notes. Manual\nde-identification is impractical given the size of EHR databases, the limited\nnumber of researchers with access to the non-de-identified notes, and the\nfrequent mistakes of human annotators. A reliable automated de-identification\nsystem would consequently be of high value.\n  Materials and Methods: We introduce the first de-identification system based\non artificial neural networks (ANNs), which requires no handcrafted features or\nrules, unlike existing systems. We compare the performance of the system with\nstate-of-the-art systems on two datasets: the i2b2 2014 de-identification\nchallenge dataset, which is the largest publicly available de-identification\ndataset, and the MIMIC de-identification dataset, which we assembled and is\ntwice as large as the i2b2 2014 dataset.\n  Results: Our ANN model outperforms the state-of-the-art systems. It yields an\nF1-score of 97.85 on the i2b2 2014 dataset, with a recall 97.38 and a precision\nof 97.32, and an F1-score of 99.23 on the MIMIC de-identification dataset, with\na recall 99.25 and a precision of 99.06.\n  Conclusion: Our findings support the use of ANNs for de-identification of\npatient notes, as they show better performance than previously published\nsystems while requiring no feature engineering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03667v4",
            "date": "2016-06-12 05:38:20+00:00",
            "title": "Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads",
            "authors": [
                "Ji He",
                "Mari Ostendorf",
                "Xiaodong He",
                "Jianshu Chen",
                "Jianfeng Gao",
                "Lihong Li",
                "Li Deng"
            ],
            "abstract": "We introduce an online popularity prediction and tracking task as a benchmark\ntask for reinforcement learning with a combinatorial, natural language action\nspace. A specified number of discussion threads predicted to be popular are\nrecommended, chosen from a fixed window of recent comments to track. Novel deep\nreinforcement learning architectures are studied for effective modeling of the\nvalue function associated with actions comprised of interdependent sub-actions.\nThe proposed model, which represents dependence between sub-actions through a\nbi-directional LSTM, gives the best performance across different experimental\nconfigurations and domains, and it also generalizes well with varying numbers\nof recommendation requests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03777v2",
            "date": "2016-06-12 22:59:14+00:00",
            "title": "Neural Belief Tracker: Data-Driven Dialogue State Tracking",
            "authors": [
                "Nikola Mrk\u0161i\u0107",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Tsung-Hsien Wen",
                "Blaise Thomson",
                "Steve Young"
            ],
            "abstract": "One of the core components of modern spoken dialogue systems is the belief\ntracker, which estimates the user's goal at every step of the dialogue.\nHowever, most current approaches have difficulty scaling to larger, more\ncomplex dialogue domains. This is due to their dependency on either: a) Spoken\nLanguage Understanding models that require large amounts of annotated training\ndata; or b) hand-crafted lexicons for capturing some of the linguistic\nvariation in users' language. We propose a novel Neural Belief Tracking (NBT)\nframework which overcomes these problems by building on recent advances in\nrepresentation learning. NBT models reason over pre-trained word vectors,\nlearning to compose them into distributed representations of user utterances\nand dialogue context. Our evaluation on two datasets shows that this approach\nsurpasses past limitations, matching the performance of state-of-the-art models\nwhich rely on hand-crafted semantic lexicons and outperforming them when such\nlexicons are not provided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.03783v1",
            "date": "2016-06-12 23:50:19+00:00",
            "title": "Retrieving and Ranking Similar Questions from Question-Answer Archives Using Topic Modelling and Topic Distribution Regression",
            "authors": [
                "Pedro Chahuara",
                "Thomas Lampert",
                "Pierre Gancarski"
            ],
            "abstract": "Presented herein is a novel model for similar question ranking within\ncollaborative question answer platforms. The presented approach integrates a\nregression stage to relate topics derived from questions to those derived from\nquestion-answer pairs. This helps to avoid problems caused by the differences\nin vocabulary used within questions and answers, and the tendency for questions\nto be shorter than answers. The performance of the model is shown to outperform\ntranslation methods and topic modelling (without regression) on several\nreal-world datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.03864v2",
            "date": "2016-06-13 09:08:04+00:00",
            "title": "Neural Associative Memory for Dual-Sequence Modeling",
            "authors": [
                "Dirk Weissenborn"
            ],
            "abstract": "Many important NLP problems can be posed as dual-sequence or\nsequence-to-sequence modeling tasks. Recent advances in building end-to-end\nneural architectures have been highly successful in solving such tasks. In this\nwork we propose a new architecture for dual-sequence modeling that is based on\nassociative memory. We derive AM-RNNs, a recurrent associative memory (AM)\nwhich augments generic recurrent neural networks (RNN). This architecture is\nextended to the Dual AM-RNN which operates on two AMs at once. Our models\nachieve very competitive results on textual entailment. A qualitative analysis\ndemonstrates that long range dependencies between source and target-sequence\ncan be bridged effectively using Dual AM-RNNs. However, an initial experiment\non auto-encoding reveals that these benefits are not exploited by the system\nwhen learning to solve sequence-to-sequence tasks which indicates that\nadditional supervision or regularization is needed.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1606.04052v5",
            "date": "2016-06-13 18:09:40+00:00",
            "title": "Dialog state tracking, a machine reading approach using Memory Network",
            "authors": [
                "Julien Perez",
                "Fei Liu"
            ],
            "abstract": "In an end-to-end dialog system, the aim of dialog state tracking is to\naccurately estimate a compact representation of the current dialog status from\na sequence of noisy observations produced by the speech recognition and the\nnatural language understanding modules. This paper introduces a novel method of\ndialog state tracking based on the general paradigm of machine reading and\nproposes to solve it using an End-to-End Memory Network, MemN2N, a\nmemory-enhanced neural network architecture. We evaluate the proposed approach\non the second Dialog State Tracking Challenge (DSTC-2) dataset. The corpus has\nbeen converted for the occasion in order to frame the hidden state variable\ninference as a question-answering task based on a sequence of utterances\nextracted from a dialog. We show that the proposed tracker gives encouraging\nresults. Then, we propose to extend the DSTC-2 dataset with specific reasoning\ncapabilities requirement like counting, list maintenance, yes-no question\nanswering and indefinite knowledge management. Finally, we present encouraging\nresults using our proposed MemN2N based tracking model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04081v1",
            "date": "2016-06-13 19:41:56+00:00",
            "title": "Graph-Community Detection for Cross-Document Topic Segment Relationship Identification",
            "authors": [
                "Pedro Mota",
                "Maxine Eskenazi",
                "Luisa Coheur"
            ],
            "abstract": "In this paper we propose a graph-community detection approach to identify\ncross-document relationships at the topic segment level. Given a set of related\ndocuments, we automatically find these relationships by clustering segments\nwith similar content (topics). In this context, we study how different\nweighting mechanisms influence the discovery of word communities that relate to\nthe different topics found in the documents. Finally, we test different mapping\nfunctions to assign topic segments to word communities, determining which topic\nsegments are considered equivalent.\n  By performing this task it is possible to enable efficient multi-document\nbrowsing, since when a user finds relevant content in one document we can\nprovide access to similar topics in other documents. We deploy our approach in\ntwo different scenarios. One is an educational scenario where equivalence\nrelationships between learning materials need to be found. The other consists\nof a series of dialogs in a social context where students discuss commonplace\ntopics. Results show that our proposed approach better discovered equivalence\nrelationships in learning material documents and obtained close results in the\nsocial speech domain, where the best performing approach was a clustering\ntechnique.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04289v2",
            "date": "2016-06-14 10:17:27+00:00",
            "title": "Automatic Text Scoring Using Neural Networks",
            "authors": [
                "Dimitrios Alikaniotis",
                "Helen Yannakoudakis",
                "Marek Rei"
            ],
            "abstract": "Automated Text Scoring (ATS) provides a cost-effective and consistent\nalternative to human marking. However, in order to achieve good performance,\nthe predictive features of the system need to be manually engineered by human\nexperts. We introduce a model that forms word representations by learning the\nextent to which specific words contribute to the text's score. Using Long-Short\nTerm Memory networks to represent the meaning of texts, we demonstrate that a\nfully automated framework is able to achieve excellent results over similar\napproaches. In an attempt to make our results more interpretable, and inspired\nby recent advances in visualizing neural networks, we introduce a novel method\nfor identifying the regions of the text that the model has found more\ndiscriminative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04351v1",
            "date": "2016-06-14 13:36:00+00:00",
            "title": "TwiSE at SemEval-2016 Task 4: Twitter Sentiment Classification",
            "authors": [
                "Georgios Balikas",
                "Massih-Reza Amini"
            ],
            "abstract": "This paper describes the participation of the team \"TwiSE\" in the SemEval\n2016 challenge. Specifically, we participated in Task 4, namely \"Sentiment\nAnalysis in Twitter\" for which we implemented sentiment classification systems\nfor subtasks A, B, C and D. Our approach consists of two steps. In the first\nstep, we generate and validate diverse feature sets for twitter sentiment\nevaluation, inspired by the work of participants of previous editions of such\nchallenges. In the second step, we focus on the optimization of the evaluation\nmeasures of the different subtasks. To this end, we examine different learning\nstrategies by validating them on the data provided by the task organisers. For\nour final submissions we used an ensemble learning approach (stacked\ngeneralization) for Subtask A and single linear models for the rest of the\nsubtasks. In the official leaderboard we were ranked 9/35, 8/19, 1/11 and 2/14\nfor subtasks A, B, C and D respectively.\\footnote{We make the code available\nfor research purposes at\n\\url{https://github.com/balikasg/SemEval2016-Twitter\\_Sentiment\\_Evaluation}.}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.04721v1",
            "date": "2016-06-15 11:08:24+00:00",
            "title": "Personality Traits and Echo Chambers on Facebook",
            "authors": [
                "Alessandro Bessi"
            ],
            "abstract": "In online social networks, users tend to select information that adhere to\ntheir system of beliefs and to form polarized groups of like minded people.\nPolarization as well as its effects on online social interactions have been\nextensively investigated. Still, the relation between group formation and\npersonality traits remains unclear. A better understanding of the cognitive and\npsychological determinants of online social dynamics might help to design more\nefficient communication strategies and to challenge the digital misinformation\nthreat. In this work, we focus on users commenting posts published by US\nFacebook pages supporting scientific and conspiracy-like narratives, and we\nclassify the personality traits of those users according to their online\nbehavior. We show that different and conflicting communities are populated by\nusers showing similar psychological profiles, and that the dominant personality\nmodel is the same in both scientific and conspiracy echo chambers. Moreover, we\nobserve that the permanence within echo chambers slightly shapes users'\npsychological profiles. Our results suggest that the presence of specific\npersonality traits in individuals lead to their considerable involvement in\nsupporting narratives inside virtual echo chambers.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1606.05007v1",
            "date": "2016-06-15 23:45:33+00:00",
            "title": "Automatic Pronunciation Generation by Utilizing a Semi-supervised Deep Neural Networks",
            "authors": [
                "Naoya Takahashi",
                "Tofigh Naghibi",
                "Beat Pfister"
            ],
            "abstract": "Phonemic or phonetic sub-word units are the most commonly used atomic\nelements to represent speech signals in modern ASRs. However they are not the\noptimal choice due to several reasons such as: large amount of effort required\nto handcraft a pronunciation dictionary, pronunciation variations, human\nmistakes and under-resourced dialects and languages. Here, we propose a\ndata-driven pronunciation estimation and acoustic modeling method which only\ntakes the orthographic transcription to jointly estimate a set of sub-word\nunits and a reliable dictionary. Experimental results show that the proposed\nmethod which is based on semi-supervised training of a deep neural network\nlargely outperforms phoneme based continuous speech recognition on the TIMIT\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05320v2",
            "date": "2016-06-16 19:13:52+00:00",
            "title": "Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models",
            "authors": [
                "Viktoriya Krakovna",
                "Finale Doshi-Velez"
            ],
            "abstract": "As deep neural networks continue to revolutionize various application\ndomains, there is increasing interest in making these powerful models more\nunderstandable and interpretable, and narrowing down the causes of good and bad\npredictions. We focus on recurrent neural networks (RNNs), state of the art\nmodels in speech recognition and translation. Our approach to increasing\ninterpretability is by combining an RNN with a hidden Markov model (HMM), a\nsimpler and more transparent model. We explore various combinations of RNNs and\nHMMs: an HMM trained on LSTM states; a hybrid model where an HMM is trained\nfirst, then a small LSTM is given HMM state distributions and trained to fill\nin gaps in the HMM's performance; and a jointly trained hybrid model. We find\nthat the LSTM and HMM learn complementary information about the features in the\ntext.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1606.05464v2",
            "date": "2016-06-17 09:39:47+00:00",
            "title": "Stance Detection with Bidirectional Conditional Encoding",
            "authors": [
                "Isabelle Augenstein",
                "Tim Rockt\u00e4schel",
                "Andreas Vlachos",
                "Kalina Bontcheva"
            ],
            "abstract": "Stance detection is the task of classifying the attitude expressed in a text\ntowards a target such as Hillary Clinton to be \"positive\", negative\" or\n\"neutral\". Previous work has assumed that either the target is mentioned in the\ntext or that training data for every target is given. This paper considers the\nmore challenging version of this task, where targets are not always mentioned\nand no training data is available for the test targets. We experiment with\nconditional LSTM encoding, which builds a representation of the tweet that is\ndependent on the target, and demonstrate that it outperforms encoding the tweet\nand the target independently. Performance is improved further when the\nconditional model is augmented with bidirectional encoding. We evaluate our\napproach on the SemEval 2016 Task 6 Twitter Stance Detection corpus achieving\nperformance second best only to a system trained on semi-automatically labelled\ntweets for the test target. When such weak supervision is added, our approach\nachieves state-of-the-art results.",
            "journal ref": "EMNLP 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05554v1",
            "date": "2016-06-17 15:15:18+00:00",
            "title": "SMS Spam Filtering using Probabilistic Topic Modelling and Stacked Denoising Autoencoder",
            "authors": [
                "Noura Al Moubayed",
                "Toby Breckon",
                "Peter Matthews",
                "A. Stephen McGough"
            ],
            "abstract": "In This paper we present a novel approach to spam filtering and demonstrate\nits applicability with respect to SMS messages. Our approach requires minimum\nfeatures engineering and a small set of la- belled data samples. Features are\nextracted using topic modelling based on latent Dirichlet allocation, and then\na comprehensive data model is created using a Stacked Denoising Autoencoder\n(SDA). Topic modelling summarises the data providing ease of use and high\ninterpretability by visualising the topics using word clouds. Given that the\nSMS messages can be regarded as either spam (unwanted) or ham (wanted), the SDA\nis able to model the messages and accurately discriminate between the two\nclasses without the need for a pre-labelled training set. The results are\ncompared against the state-of-the-art spam detection algorithms with our\nproposed approach achieving over 97% accuracy which compares favourably to the\nbest reported algorithms presented in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.05925v1",
            "date": "2016-06-19 23:40:51+00:00",
            "title": "Graph based manifold regularized deep neural networks for automatic speech recognition",
            "authors": [
                "Vikrant Singh Tomar",
                "Richard C. Rose"
            ],
            "abstract": "Deep neural networks (DNNs) have been successfully applied to a wide variety\nof acoustic modeling tasks in recent years. These include the applications of\nDNNs either in a discriminative feature extraction or in a hybrid acoustic\nmodeling scenario. Despite the rapid progress in this area, a number of\nchallenges remain in training DNNs. This paper presents an effective way of\ntraining DNNs using a manifold learning based regularization framework. In this\nframework, the parameters of the network are optimized to preserve underlying\nmanifold based relationships between speech feature vectors while minimizing a\nmeasure of loss between network outputs and targets. This is achieved by\nincorporating manifold based locality constraints in the objective criterion of\nDNNs. Empirical evidence is provided to demonstrate that training a network\nwith manifold constraints preserves structural compactness in the hidden layers\nof the network. Manifold regularization is applied to train bottleneck DNNs for\nfeature extraction in hidden Markov model (HMM) based speech recognition. The\nexperiments in this work are conducted on the Aurora-2 spoken digits and the\nAurora-4 read news large vocabulary continuous speech recognition tasks. The\nperformance is measured in terms of word error rate (WER) on these tasks. It is\nshown that the manifold regularized DNNs result in up to 37% reduction in WER\nrelative to standard DNNs.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1606.06031v1",
            "date": "2016-06-20 09:37:17+00:00",
            "title": "The LAMBADA dataset: Word prediction requiring a broad discourse context",
            "authors": [
                "Denis Paperno",
                "Germ\u00e1n Kruszewski",
                "Angeliki Lazaridou",
                "Quan Ngoc Pham",
                "Raffaella Bernardi",
                "Sandro Pezzelle",
                "Marco Baroni",
                "Gemma Boleda",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "We introduce LAMBADA, a dataset to evaluate the capabilities of computational\nmodels for text understanding by means of a word prediction task. LAMBADA is a\ncollection of narrative passages sharing the characteristic that human subjects\nare able to guess their last word if they are exposed to the whole passage, but\nnot if they only see the last sentence preceding the target word. To succeed on\nLAMBADA, computational models cannot simply rely on local context, but must be\nable to keep track of information in the broader discourse. We show that\nLAMBADA exemplifies a wide range of linguistic phenomena, and that none of\nseveral state-of-the-art language models reaches accuracy above 1% on this\nnovel benchmark. We thus propose LAMBADA as a challenging test set, meant to\nencourage the development of new models capable of genuine understanding of\nbroad context in natural language text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06083v2",
            "date": "2016-06-20 12:26:21+00:00",
            "title": "Product Classification in E-Commerce using Distributional Semantics",
            "authors": [
                "Vivek Gupta",
                "Harish Karnick",
                "Ashendra Bansal",
                "Pradhuman Jhala"
            ],
            "abstract": "Product classification is the task of automatically predicting a taxonomy\npath for a product in a predefined taxonomy hierarchy given a textual product\ndescription or title. For efficient product classification we require a\nsuitable representation for a document (the textual description of a product)\nfeature vector and efficient and fast algorithms for prediction. To address the\nabove challenges, we propose a new distributional semantics representation for\ndocument vector formation. We also develop a new two-level ensemble approach\nutilizing (with respect to the taxonomy tree) a path-wise, node-wise and\ndepth-wise classifiers for error reduction in the final product classification.\nOur experiments show the effectiveness of the distributional representation and\nthe ensemble approach on data sets from a leading e-commerce platform and\nachieve better results on various evaluation metrics compared to earlier\napproaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.06121v1",
            "date": "2016-06-20 13:58:45+00:00",
            "title": "Quantifying and Reducing Stereotypes in Word Embeddings",
            "authors": [
                "Tolga Bolukbasi",
                "Kai-Wei Chang",
                "James Zou",
                "Venkatesh Saligrama",
                "Adam Kalai"
            ],
            "abstract": "Machine learning algorithms are optimized to model statistical properties of\nthe training data. If the input data reflects stereotypes and biases of the\nbroader society, then the output of the learning algorithm also captures these\nstereotypes. In this paper, we initiate the study of gender stereotypes in {\\em\nword embedding}, a popular framework to represent text data. As their use\nbecomes increasingly common, applications can inadvertently amplify unwanted\nstereotypes. We show across multiple datasets that the embeddings contain\nsignificant gender stereotypes, especially with regard to professions. We\ncreated a novel gender analogy task and combined it with crowdsourcing to\nsystematically quantify the gender bias in a given embedding. We developed an\nefficient algorithm that reduces gender stereotype using just a handful of\ntraining examples while preserving the useful geometric properties of the\nembedding. We evaluated our algorithm on several metrics. While we focus on\nmale/female stereotypes, our framework may be applicable to other types of\nembedding biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06137v1",
            "date": "2016-06-20 14:26:33+00:00",
            "title": "LSTM-Based Predictions for Proactive Information Retrieval",
            "authors": [
                "Petri Luukkonen",
                "Markus Koskela",
                "Patrik Flor\u00e9en"
            ],
            "abstract": "We describe a method for proactive information retrieval targeted at\nretrieving relevant information during a writing task. In our method, the\ncurrent task and the needs of the user are estimated, and the potential next\nsteps are unobtrusively predicted based on the user's past actions. We focus on\nthe task of writing, in which the user is coalescing previously collected\ninformation into a text. Our proactive system automatically recommends the user\nrelevant background information. The proposed system incorporates text input\nprediction using a long short-term memory (LSTM) network. We present\nsimulations, which show that the system is able to reach higher precision\nvalues in an exploratory search setting compared to both a baseline and a\ncomparison system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.06142v1",
            "date": "2016-06-20 14:34:55+00:00",
            "title": "Comparing the hierarchy of keywords in on-line news portals",
            "authors": [
                "Gergely Tib\u00e9ly",
                "David Sousa-Rodrigues",
                "P\u00e9ter Pollner",
                "Gergely Palla"
            ],
            "abstract": "The tagging of on-line content with informative keywords is a widespread\nphenomenon from scientific article repositories through blogs to on-line news\nportals. In most of the cases, the tags on a given item are free words chosen\nby the authors independently. Therefore, relations among keywords in a\ncollection of news items is unknown. However, in most cases the topics and\nconcepts described by these keywords are forming a latent hierarchy, with the\nmore general topics and categories at the top, and more specialised ones at the\nbottom. Here we apply a recent, cooccurrence-based tag hierarchy extraction\nmethod to sets of keywords obtained from four different on-line news portals.\nThe resulting hierarchies show substantial differences not just in the topics\nrendered as important (being at the top of the hierarchy) or of less interest\n(categorised low in the hierarchy), but also in the underlying network\nstructure. This reveals discrepancies between the plausible keyword association\nframeworks in the studied news portals.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1606.06352v1",
            "date": "2016-06-20 22:30:19+00:00",
            "title": "Visualizing textual models with in-text and word-as-pixel highlighting",
            "authors": [
                "Abram Handler",
                "Su Lin Blodgett",
                "Brendan O'Connor"
            ],
            "abstract": "We explore two techniques which use color to make sense of statistical text\nmodels. One method uses in-text annotations to illustrate a model's view of\nparticular tokens in particular documents. Another uses a high-level,\n\"words-as-pixels\" graphic to display an entire corpus. Together, these methods\noffer both zoomed-in and zoomed-out perspectives into a model's understanding\nof text. We show how these interconnected methods help diagnose a classifier's\npoor performance on Twitter slang, and make sense of a topic model on\nhistorical political texts.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1606.06361v1",
            "date": "2016-06-20 23:29:55+00:00",
            "title": "A Probabilistic Generative Grammar for Semantic Parsing",
            "authors": [
                "Abulhair Saparov",
                "Tom M. Mitchell"
            ],
            "abstract": "We present a framework that couples the syntax and semantics of natural\nlanguage sentences in a generative model, in order to develop a semantic parser\nthat jointly infers the syntactic, morphological, and semantic representations\nof a given sentence under the guidance of background knowledge. To generate a\nsentence in our framework, a semantic statement is first sampled from a prior,\nsuch as from a set of beliefs in a knowledge base. Given this semantic\nstatement, a grammar probabilistically generates the output sentence. A joint\nsemantic-syntactic parser is derived that returns the $k$-best semantic and\nsyntactic parses for a given sentence. The semantic prior is flexible, and can\nbe used to incorporate background knowledge during parsing, in ways unlike\nprevious semantic parsing approaches. For example, semantic statements\ncorresponding to beliefs in a knowledge base can be given higher prior\nprobability, type-correct statements can be given somewhat lower probability,\nand beliefs outside the knowledge base can be given lower probability. The\nconstruction of our grammar invokes a novel application of hierarchical\nDirichlet processes (HDPs), which in turn, requires a novel and efficient\ninference approach. We present experimental results showing, for a simple\ngrammar, that our parser outperforms a state-of-the-art CCG semantic parser and\nscales to knowledge bases with millions of beliefs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06368v2",
            "date": "2016-06-20 23:59:25+00:00",
            "title": "Unanimous Prediction for 100% Precision with Application to Learning Semantic Mappings",
            "authors": [
                "Fereshte Khani",
                "Martin Rinard",
                "Percy Liang"
            ],
            "abstract": "Can we train a system that, on any new input, either says \"don't know\" or\nmakes a prediction that is guaranteed to be correct? We answer the question in\nthe affirmative provided our model family is well-specified. Specifically, we\nintroduce the unanimity principle: only predict when all models consistent with\nthe training data predict the same output. We operationalize this principle for\nsemantic parsing, the task of mapping utterances to logical forms. We develop a\nsimple, efficient method that reasons over the infinite set of all consistent\nmodels by only checking two of the models. We prove that our method obtains\n100% precision even with a modest amount of training data from a possibly\nadversarial distribution. Empirically, we demonstrate the effectiveness of our\napproach on the standard GeoQuery dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1606.06424v1",
            "date": "2016-06-21 04:56:33+00:00",
            "title": "A Novel Framework to Expedite Systematic Reviews by Automatically Building Information Extraction Training Corpora",
            "authors": [
                "Tanmay Basu",
                "Shraman Kumar",
                "Abhishek Kalyan",
                "Priyanka Jayaswal",
                "Pawan Goyal",
                "Stephen Pettifer",
                "Siddhartha R. Jonnalagadda"
            ],
            "abstract": "A systematic review identifies and collates various clinical studies and\ncompares data elements and results in order to provide an evidence based answer\nfor a particular clinical question. The process is manual and involves lot of\ntime. A tool to automate this process is lacking. The aim of this work is to\ndevelop a framework using natural language processing and machine learning to\nbuild information extraction algorithms to identify data elements in a new\nprimary publication, without having to go through the expensive task of manual\nannotation to build gold standards for each data element type. The system is\ndeveloped in two stages. Initially, it uses information contained in existing\nsystematic reviews to identify the sentences from the PDF files of the included\nreferences that contain specific data elements of interest using a modified\nJaccard similarity measure. These sentences have been treated as labeled data.A\nSupport Vector Machine (SVM) classifier is trained on this labeled data to\nextract data elements of interests from a new article. We conducted experiments\non Cochrane Database systematic reviews related to congestive heart failure\nusing inclusion criteria as an example data element. The empirical results show\nthat the proposed system automatically identifies sentences containing the data\nelement of interest with a high recall (93.75%) and reasonable precision\n(27.05% - which means the reviewers have to read only 3.7 sentences on\naverage). The empirical results suggest that the tool is retrieving valuable\ninformation from the reference articles, even when it is time-consuming to\nidentify them manually. Thus we hope that the tool will be useful for automatic\ndata extraction from biomedical research publications. The future scope of this\nwork is to generalize this information framework for all types of systematic\nreviews.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.06622v3",
            "date": "2016-06-21 15:38:27+00:00",
            "title": "Question Relevance in VQA: Identifying Non-Visual And False-Premise Questions",
            "authors": [
                "Arijit Ray",
                "Gordon Christie",
                "Mohit Bansal",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "Visual Question Answering (VQA) is the task of answering natural-language\nquestions about images. We introduce the novel problem of determining the\nrelevance of questions to images in VQA. Current VQA models do not reason about\nwhether a question is even related to the given image (e.g. What is the capital\nof Argentina?) or if it requires information from external resources to answer\ncorrectly. This can break the continuity of a dialogue in human-machine\ninteraction. Our approaches for determining relevance are composed of two\nstages. Given an image and a question, (1) we first determine whether the\nquestion is visual or not, (2) if visual, we determine whether the question is\nrelevant to the given image or not. Our approaches, based on LSTM-RNNs, VQA\nmodel uncertainty, and caption-question similarity, are able to outperform\nstrong baselines on both relevance tasks. We also present human studies showing\nthat VQA models augmented with such question relevance reasoning are perceived\nas more intelligent, reasonable, and human-like.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.06820v5",
            "date": "2016-06-22 05:23:01+00:00",
            "title": "Divergent discourse between protests and counter-protests: #BlackLivesMatter and #AllLivesMatter",
            "authors": [
                "Ryan J. Gallagher",
                "Andrew J. Reagan",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Since the shooting of Black teenager Michael Brown by White police officer\nDarren Wilson in Ferguson, Missouri, the protest hashtag #BlackLivesMatter has\namplified critiques of extrajudicial killings of Black Americans. In response\nto #BlackLivesMatter, other Twitter users have adopted #AllLivesMatter, a\ncounter-protest hashtag whose content argues that equal attention should be\ngiven to all lives regardless of race. Through a multi-level analysis of over\n860,000 tweets, we study how these protests and counter-protests diverge by\nquantifying aspects of their discourse. We find that #AllLivesMatter\nfacilitates opposition between #BlackLivesMatter and hashtags such as\n#PoliceLivesMatter and #BlueLivesMatter in such a way that historically echoes\nthe tension between Black protesters and law enforcement. In addition, we show\nthat a significant portion of #AllLivesMatter use stems from hijacking by\n#BlackLivesMatter advocates. Beyond simply injecting #AllLivesMatter with\n#BlackLivesMatter content, these hijackers use the hashtag to directly confront\nthe counter-protest notion of \"All lives matter.\" Our findings suggest that\nBlack Lives Matter movement was able to grow, exhibit diverse conversations,\nand avoid derailment on social media by making discussion of counter-protest\nopinions a central topic of #AllLivesMatter, rather than the movement itself.",
            "journal ref": "PLoS ONE, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06864v2",
            "date": "2016-06-22 09:29:40+00:00",
            "title": "A Curriculum Learning Method for Improved Noise Robustness in Automatic Speech Recognition",
            "authors": [
                "Stefan Braun",
                "Daniel Neil",
                "Shih-Chii Liu"
            ],
            "abstract": "The performance of automatic speech recognition systems under noisy\nenvironments still leaves room for improvement. Speech enhancement or feature\nenhancement techniques for increasing noise robustness of these systems usually\nadd components to the recognition system that need careful optimization. In\nthis work, we propose the use of a relatively simple curriculum training\nstrategy called accordion annealing (ACCAN). It uses a multi-stage training\nschedule where samples at signal-to-noise ratio (SNR) values as low as 0dB are\nfirst added and samples at increasing higher SNR values are gradually added up\nto an SNR value of 50dB. We also use a method called per-epoch noise mixing\n(PEM) that generates noisy training samples online during training and thus\nenables dynamically changing the SNR of our training data. Both the ACCAN and\nthe PEM methods are evaluated on a end-to-end speech recognition pipeline on\nthe Wall Street Journal corpus. ACCAN decreases the average word error rate\n(WER) on the 20dB to -10dB SNR range by up to 31.4% when compared to a\nconventional multi-condition training method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.06871v2",
            "date": "2016-06-22 10:00:14+00:00",
            "title": "A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition",
            "authors": [
                "Albert Zeyer",
                "Patrick Doetsch",
                "Paul Voigtlaender",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "We present a comprehensive study of deep bidirectional long short-term memory\n(LSTM) recurrent neural network (RNN) based acoustic models for automatic\nspeech recognition (ASR). We study the effect of size and depth and train\nmodels of up to 8 layers. We investigate the training aspect and study\ndifferent variants of optimization methods, batching, truncated\nbackpropagation, different regularization techniques such as dropout and $L_2$\nregularization, and different gradient clipping variants.\n  The major part of the experimental analysis was performed on the Quaero\ncorpus. Additional experiments also were performed on the Switchboard corpus.\nOur best LSTM model has a relative improvement in word error rate of over 14\\%\ncompared to our best feed-forward neural network (FFNN) baseline on the Quaero\ntask. On this task, we get our best result with an 8 layer bidirectional LSTM\nand we show that a pretraining scheme with layer-wise construction helps for\ndeep LSTMs.\n  Finally we compare the training calculation time of many of the presented\nexperiments in relation with recognition performance.\n  All the experiments were done with RETURNN, the RWTH extensible training\nframework for universal recurrent neural networks in combination with RASR, the\nRWTH ASR toolkit.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1606.07043v1",
            "date": "2016-06-22 19:00:38+00:00",
            "title": "Toward Interpretable Topic Discovery via Anchored Correlation Explanation",
            "authors": [
                "Kyle Reing",
                "David C. Kale",
                "Greg Ver Steeg",
                "Aram Galstyan"
            ],
            "abstract": "Many predictive tasks, such as diagnosing a patient based on their medical\nchart, are ultimately defined by the decisions of human experts. Unfortunately,\nencoding experts' knowledge is often time consuming and expensive. We propose a\nsimple way to use fuzzy and informal knowledge from experts to guide discovery\nof interpretable latent topics in text. The underlying intuition of our\napproach is that latent factors should be informative about both correlations\nin the data and a set of relevance variables specified by an expert.\nMathematically, this approach is a combination of the information bottleneck\nand Total Correlation Explanation (CorEx). We give a preliminary evaluation of\nAnchored CorEx, showing that it produces more coherent and interpretable topics\non two distinct corpora.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1606.07056v1",
            "date": "2016-06-22 19:55:24+00:00",
            "title": "Emulating Human Conversations using Convolutional Neural Network-based IR",
            "authors": [
                "Abhay Prakash",
                "Chris Brockett",
                "Puneet Agrawal"
            ],
            "abstract": "Conversational agents (\"bots\") are beginning to be widely used in\nconversational interfaces. To design a system that is capable of emulating\nhuman-like interactions, a conversational layer that can serve as a fabric for\nchat-like interaction with the agent is needed. In this paper, we introduce a\nmodel that employs Information Retrieval by utilizing convolutional deep\nstructured semantic neural network-based features in the ranker to present\nhuman-like responses in ongoing conversation with a user. In conversations,\naccounting for context is critical to the retrieval model; we show that our\ncontext-sensitive approach using a Convolutional Deep Structured Semantic Model\n(cDSSM) with character trigrams significantly outperforms several conventional\nbaselines in terms of the relevance of responses retrieved.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.07137v1",
            "date": "2016-06-22 23:35:59+00:00",
            "title": "Automated Extraction of Number of Subjects in Randomised Controlled Trials",
            "authors": [
                "Abeed Sarker"
            ],
            "abstract": "We present a simple approach for automatically extracting the number of\nsubjects involved in randomised controlled trials (RCT). Our approach first\napplies a set of rule-based techniques to extract candidate study sizes from\nthe abstracts of the articles. Supervised classification is then performed over\nthe candidates with support vector machines, using a small set of lexical,\nstructural, and contextual features. With only a small annotated training set\nof 201 RCTs, we obtained an accuracy of 88\\%. We believe that this system will\naid complex medical text processing tasks such as summarisation and question\nanswering.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.07189v1",
            "date": "2016-06-23 05:03:03+00:00",
            "title": "Gender and Interest Targeting for Sponsored Post Advertising at Tumblr",
            "authors": [
                "Mihajlo Grbovic",
                "Vladan Radosavljevic",
                "Nemanja Djuric",
                "Narayan Bhamidipati",
                "Ananth Nagarajan"
            ],
            "abstract": "As one of the leading platforms for creative content, Tumblr offers\nadvertisers a unique way of creating brand identity. Advertisers can tell their\nstory through images, animation, text, music, video, and more, and promote that\ncontent by sponsoring it to appear as an advertisement in the streams of Tumblr\nusers. In this paper we present a framework that enabled one of the key\ntargeted advertising components for Tumblr, specifically gender and interest\ntargeting. We describe the main challenges involved in development of the\nframework, which include creating the ground truth for training gender\nprediction models, as well as mapping Tumblr content to an interest taxonomy.\nFor purposes of inferring user interests we propose a novel semi-supervised\nneural language model for categorization of Tumblr content (i.e., post tags and\npost keywords). The model was trained on a large-scale data set consisting of\n6.8 billion user posts, with very limited amount of categorized keywords, and\nwas shown to have superior performance over the bag-of-words model. We\nsuccessfully deployed gender and interest targeting capability in Yahoo\nproduction systems, delivering inference for users that cover more than 90% of\ndaily activities at Tumblr. Online performance results indicate advantages of\nthe proposed approach, where we observed 20% lift in user engagement with\nsponsored posts as compared to untargeted campaigns.",
            "journal ref": "Proceedings of the 21th ACM SIGKDD International Conference on\n  Knowledge Discovery and Data Mining (KDD 2015), Sydney, Australia",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07287v1",
            "date": "2016-06-23 12:25:09+00:00",
            "title": "Picture It In Your Mind: Generating High Level Visual Representations From Textual Descriptions",
            "authors": [
                "Fabio Carrara",
                "Andrea Esuli",
                "Tiziano Fagni",
                "Fabrizio Falchi",
                "Alejandro Moreo Fern\u00e1ndez"
            ],
            "abstract": "In this paper we tackle the problem of image search when the query is a short\ntextual description of the image the user is looking for. We choose to\nimplement the actual search process as a similarity search in a visual feature\nspace, by learning to translate a textual query into a visual representation.\nSearching in the visual feature space has the advantage that any update to the\ntranslation model does not require to reprocess the, typically huge, image\ncollection on which the search is performed. We propose Text2Vis, a neural\nnetwork that generates a visual representation, in the visual feature space of\nthe fc6-fc7 layers of ImageNet, from a short descriptive text. Text2Vis\noptimizes two loss functions, using a stochastic loss-selection method. A\nvisual-focused loss is aimed at learning the actual text-to-visual feature\nmapping, while a text-focused loss is aimed at modeling the higher-level\nsemantic concepts expressed in language and countering the overfit on\nnon-relevant visual components of the visual loss. We report preliminary\nresults on the MS-COCO dataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1606.07356v2",
            "date": "2016-06-23 16:05:16+00:00",
            "title": "Analyzing the Behavior of Visual Question Answering Models",
            "authors": [
                "Aishwarya Agrawal",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "Recently, a number of deep-learning based models have been proposed for the\ntask of Visual Question Answering (VQA). The performance of most models is\nclustered around 60-70%. In this paper we propose systematic methods to analyze\nthe behavior of these models as a first step towards recognizing their\nstrengths and weaknesses, and identifying the most fruitful directions for\nprogress. We analyze two models, one each from two major classes of VQA models\n-- with-attention and without-attention and show the similarities and\ndifferences in the behavior of these models. We also analyze the winning entry\nof the VQA Challenge 2016.\n  Our behavior analysis reveals that despite recent progress, today's VQA\nmodels are \"myopic\" (tend to fail on sufficiently novel instances), often \"jump\nto conclusions\" (converge on a predicted answer after 'listening' to just half\nthe question), and are \"stubborn\" (do not change their answers across images).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07461v2",
            "date": "2016-06-23 20:20:39+00:00",
            "title": "LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks",
            "authors": [
                "Hendrik Strobelt",
                "Sebastian Gehrmann",
                "Hanspeter Pfister",
                "Alexander M. Rush"
            ],
            "abstract": "Recurrent neural networks, and in particular long short-term memory (LSTM)\nnetworks, are a remarkably effective tool for sequence modeling that learn a\ndense black-box hidden representation of their sequential input. Researchers\ninterested in better understanding these models have studied the changes in\nhidden state representations over time and noticed some interpretable patterns\nbut also significant noise. In this work, we present LSTMVIS, a visual analysis\ntool for recurrent neural networks with a focus on understanding these hidden\nstate dynamics. The tool allows users to select a hypothesis input range to\nfocus on local state changes, to match these states changes to similar patterns\nin a large data set, and to align these results with structural annotations\nfrom their domain. We show several use cases of the tool for analyzing specific\nhidden state properties on dataset containing nesting, phrase structure, and\nchord progressions, and demonstrate how the tool can be used to isolate\npatterns for further statistical analysis. We characterize the domain, the\ndifferent stakeholders, and their goals and tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07493v5",
            "date": "2016-06-23 21:54:44+00:00",
            "title": "Sort Story: Sorting Jumbled Images and Captions into Stories",
            "authors": [
                "Harsh Agrawal",
                "Arjun Chandrasekaran",
                "Dhruv Batra",
                "Devi Parikh",
                "Mohit Bansal"
            ],
            "abstract": "Temporal common sense has applications in AI tasks such as QA, multi-document\nsummarization, and human-AI communication. We propose the task of sequencing --\ngiven a jumbled set of aligned image-caption pairs that belong to a story, the\ntask is to sort them such that the output sequence forms a coherent story. We\npresent multiple approaches, via unary (position) and pairwise (order)\npredictions, and their ensemble-based combinations, achieving strong results on\nthis task. We use both text-based and image-based features, which depict\ncomplementary improvements. Using qualitative examples, we demonstrate that our\nmodels have learnt interesting aspects of temporal common sense.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07572v3",
            "date": "2016-06-24 06:00:42+00:00",
            "title": "Enriching Linked Datasets with New Object Properties",
            "authors": [
                "Subhashree S",
                "P Sreenivasa Kumar"
            ],
            "abstract": "Although several RDF knowledge bases are available through the LOD\ninitiative, the ontology schema of such linked datasets is not very rich. In\nparticular, they lack object properties. The problem of finding new object\nproperties (and their instances) between any two given classes has not been\ninvestigated in detail in the context of Linked Data. In this paper, we present\nDART (Detecting Arbitrary Relations for enriching T-Boxes of Linked Data) - an\nunsupervised solution to enrich the LOD cloud with new object properties\nbetween two given classes. DART exploits contextual similarity to identify text\npatterns from the web corpus that can potentially represent relations between\nindividuals. These text patterns are then clustered by means of paraphrase\ndetection to capture the object properties between the two given LOD classes.\nDART also performs fully automated mapping of the discovered relations to the\nproperties in the linked dataset. This serves many purposes such as\nidentification of completely new relations, elimination of irrelevant\nrelations, and generation of prospective property axioms. We have empirically\nevaluated our approach on several pairs of classes and found that the system\ncan indeed be used for enriching the linked datasets with new object properties\nand their instances. We compared DART with newOntExt system which is an\noffshoot of the NELL (Never-Ending Language Learning) effort. Our experiments\nreveal that DART gives better results than newOntExt with respect to both the\ncorrectness, as well as the number of relations.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1606.07711v4",
            "date": "2016-06-24 14:45:27+00:00",
            "title": "A Game-Theoretic Approach to Word Sense Disambiguation",
            "authors": [
                "Rocco Tripodi",
                "Marcello Pelillo"
            ],
            "abstract": "This paper presents a new model for word sense disambiguation formulated in\nterms of evolutionary game theory, where each word to be disambiguated is\nrepresented as a node on a graph whose edges represent word relations and\nsenses are represented as classes. The words simultaneously update their class\nmembership preferences according to the senses that neighboring words are\nlikely to choose. We use distributional information to weigh the influence that\neach word has on the decisions of the others and semantic similarity\ninformation to measure the strength of compatibility among the choices. With\nthis information we can formulate the word sense disambiguation problem as a\nconstraint satisfaction problem and solve it using tools derived from game\ntheory, maintaining the textual coherence. The model is based on two ideas:\nsimilar words should be assigned to similar classes and the meaning of a word\ndoes not depend on all the words in a text but just on some of them. The paper\nprovides an in-depth motivation of the idea of modeling the word sense\ndisambiguation problem in terms of game theory, which is illustrated by an\nexample. The conclusion presents an extensive analysis on the combination of\nsimilarity measures to use in the framework and a comparison with\nstate-of-the-art systems. The results show that our model outperforms\nstate-of-the-art algorithms and can be applied to different tasks and in\ndifferent scenarios.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1606.07947v4",
            "date": "2016-06-25 18:16:39+00:00",
            "title": "Sequence-Level Knowledge Distillation",
            "authors": [
                "Yoon Kim",
                "Alexander M. Rush"
            ],
            "abstract": "Neural machine translation (NMT) offers a novel alternative formulation of\ntranslation that is potentially simpler than statistical approaches. However to\nreach competitive performance, NMT models need to be exceedingly large. In this\npaper we consider applying knowledge distillation approaches (Bucila et al.,\n2006; Hinton et al., 2015) that have proven successful for reducing the size of\nneural models in other domains to the problem of NMT. We demonstrate that\nstandard knowledge distillation applied to word-level prediction can be\neffective for NMT, and also introduce two novel sequence-level versions of\nknowledge distillation that further improve performance, and somewhat\nsurprisingly, seem to eliminate the need for beam search (even when applied on\nthe original teacher model). Our best student model runs 10 times faster than\nits state-of-the-art teacher with little loss in performance. It is also\nsignificantly better than a baseline model trained without knowledge\ndistillation: by 4.2/1.7 BLEU with greedy decoding/beam search. Applying weight\npruning on top of knowledge distillation results in a student model that has 13\ntimes fewer parameters than the original teacher model, with a decrease of 0.4\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07953v2",
            "date": "2016-06-25 19:46:28+00:00",
            "title": "Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records",
            "authors": [
                "Abhyuday Jagannatha",
                "Hong Yu"
            ],
            "abstract": "Sequence labeling for extraction of medical events and their attributes from\nunstructured text in Electronic Health Record (EHR) notes is a key step towards\nsemantic understanding of EHRs. It has important applications in health\ninformatics including pharmacovigilance and drug surveillance. The state of the\nart supervised machine learning models in this domain are based on Conditional\nRandom Fields (CRFs) with features calculated from fixed context windows. In\nthis application, we explored various recurrent neural network frameworks and\nshow that they significantly outperformed the CRF models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.08359v2",
            "date": "2016-06-27 16:39:23+00:00",
            "title": "Lifted Rule Injection for Relation Embeddings",
            "authors": [
                "Thomas Demeester",
                "Tim Rockt\u00e4schel",
                "Sebastian Riedel"
            ],
            "abstract": "Methods based on representation learning currently hold the state-of-the-art\nin many natural language processing and knowledge base inference tasks. Yet, a\nmajor challenge is how to efficiently incorporate commonsense knowledge into\nsuch models. A recent approach regularizes relation and entity representations\nby propositionalization of first-order logic rules. However,\npropositionalization does not scale beyond domains with only few entities and\nrules. In this paper we present a highly efficient method for incorporating\nimplication rules into distributed representations for automated knowledge base\nconstruction. We map entity-tuple embeddings into an approximately Boolean\nspace and encourage a partial ordering over relation embeddings based on\nimplication rules mined from WordNet. Surprisingly, we find that the strong\nrestriction of the entity-tuple embedding space does not hurt the\nexpressiveness of the model and even acts as a regularizer that improves\ngeneralization. By incorporating few commonsense rules, we achieve an increase\nof 2 percentage points mean average precision over a matrix factorization\nbaseline, while observing a negligible increase in runtime.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1606.08777v1",
            "date": "2016-06-28 16:31:50+00:00",
            "title": "\"Show me the cup\": Reference with Continuous Representations",
            "authors": [
                "Gemma Boleda",
                "Sebastian Pad\u00f3",
                "Marco Baroni"
            ],
            "abstract": "One of the most basic functions of language is to refer to objects in a\nshared scene. Modeling reference with continuous representations is challenging\nbecause it requires individuation, i.e., tracking and distinguishing an\narbitrary number of referents. We introduce a neural network model that, given\na definite description and a set of objects represented by natural images,\npoints to the intended object if the expression has a unique referent, or\nindicates a failure, if it does not. The model, directly trained on reference\nacts, is competitive with a pipeline manually engineered to perform the same\ntask, both when referents are purely visual, and when they are characterized by\na combination of visual and linguistic properties.",
            "journal ref": "In: Gelbukh A. (eds) Computational Linguistics and Intelligent\n  Text Processing. CICLing 2017. Lecture Notes in Computer Science, vol 10761.\n  Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09163v1",
            "date": "2016-06-29 15:51:44+00:00",
            "title": "Optimising The Input Window Alignment in CD-DNN Based Phoneme Recognition for Low Latency Processing",
            "authors": [
                "Akash Kumar Dhaka",
                "Giampiero Salvi"
            ],
            "abstract": "We present a systematic analysis on the performance of a phonetic recogniser\nwhen the window of input features is not symmetric with respect to the current\nframe. The recogniser is based on Context Dependent Deep Neural Networks\n(CD-DNNs) and Hidden Markov Models (HMMs). The objective is to reduce the\nlatency of the system by reducing the number of future feature frames required\nto estimate the current output. Our tests performed on the TIMIT database show\nthat the performance does not degrade when the input window is shifted up to 5\nframes in the past compared to common practice (no future frame). This\ncorresponds to improving the latency by 50 ms in our settings. Our tests also\nshow that the best results are not obtained with the symmetric window commonly\nemployed, but with an asymmetric window with eight past and two future context\nframes, although this observation should be confirmed on other data sets. The\nreduction in latency suggested by our results is critical for specific\napplications such as real-time lip synchronisation for tele-presence, but may\nalso be beneficial in general applications to improve the lag in human-machine\nspoken interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09222v1",
            "date": "2016-06-29 19:06:48+00:00",
            "title": "Penambahan emosi menggunakan metode manipulasi prosodi untuk sistem text to speech bahasa Indonesia",
            "authors": [
                "Salita Ulitia Prini",
                "Ary Setijadi Prihatmanto"
            ],
            "abstract": "Adding an emotions using prosody manipulation method for Indonesian text to\nspeech system. Text To Speech (TTS) is a system that can convert text in one\nlanguage into speech, accordance with the reading of the text in the language\nused. The focus of this research is a natural sounding concept, the make\n\"humanize\" for the pronunciation of voice synthesis system Text To Speech.\nHumans have emotions / intonation that may affect the sound produced. The main\nrequirement for the system used Text To Speech in this research is eSpeak, the\ndatabase MBROLA using id1, Human Speech Corpus database from a website that\nsummarizes the words with the highest frequency (Most Common Words) used in a\ncountry. And there are 3 types of emotional / intonation designed base. There\nis a happy, angry and sad emotion. Method for develop the emotional filter is\nmanipulate the relevant features of prosody (especially pitch and duration\nvalue) using a predetermined rate factor that has been established by analyzing\nthe differences between the standard output Text To Speech and voice recording\nwith emotional prosody / a particular intonation. The test results for the\nperception tests of Human Speech Corpus for happy emotion is 95 %, 96.25 % for\nangry emotion and 98.75 % for sad emotions. For perception test system carried\nby intelligibility and naturalness test. Intelligibility test for the accuracy\nof sound with the original sentence is 93.3%, and for clarity rate for each\nsentence is 62.8%. For naturalness, accuracy emotional election amounted to\n75.6 % for happy emotion, 73.3 % for angry emotion, and 60 % for sad emotions.\n  -----\n  Text To Speech (TTS) merupakan suatu sistem yang dapat mengonversi teks dalam\nformat suatu bahasa menjadi ucapan sesuai dengan pembacaan teks dalam bahasa\nyang digunakan.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1606.09239v1",
            "date": "2016-06-29 19:52:53+00:00",
            "title": "Learning Concept Taxonomies from Multi-modal Data",
            "authors": [
                "Hao Zhang",
                "Zhiting Hu",
                "Yuntian Deng",
                "Mrinmaya Sachan",
                "Zhicheng Yan",
                "Eric P. Xing"
            ],
            "abstract": "We study the problem of automatically building hypernym taxonomies from\ntextual and visual data. Previous works in taxonomy induction generally ignore\nthe increasingly prominent visual data, which encode important perceptual\nsemantics. Instead, we propose a probabilistic model for taxonomy induction by\njointly leveraging text and images. To avoid hand-crafted feature engineering,\nwe design end-to-end features based on distributed representations of images\nand words. The model is discriminatively trained given a small set of existing\nontologies and is capable of building full taxonomies from scratch for a\ncollection of unseen conceptual label items with associated images. We evaluate\nour model and features on the WordNet hierarchies, where our system outperforms\nprevious approaches by a large gap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.09274v1",
            "date": "2016-06-29 20:36:23+00:00",
            "title": "Compression of Neural Machine Translation Models via Pruning",
            "authors": [
                "Abigail See",
                "Minh-Thang Luong",
                "Christopher D. Manning"
            ],
            "abstract": "Neural Machine Translation (NMT), like many other deep learning domains,\ntypically suffers from over-parameterization, resulting in large storage sizes.\nThis paper examines three simple magnitude-based pruning schemes to compress\nNMT models, namely class-blind, class-uniform, and class-distribution, which\ndiffer in terms of how pruning thresholds are computed for the different\nclasses of weights in the NMT architecture. We demonstrate the efficacy of\nweight pruning as a compression technique for a state-of-the-art NMT system. We\nshow that an NMT model with over 200 million parameters can be pruned by 40%\nwith very little performance loss as measured on the WMT'14 English-German\ntranslation task. This sheds light on the distribution of redundancy in the NMT\narchitecture. Our main result is that with retraining, we can recover and even\nsurpass the original performance with an 80%-pruned model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.00104v1",
            "date": "2016-07-30 11:53:04+00:00",
            "title": "World Knowledge as Indirect Supervision for Document Clustering",
            "authors": [
                "Chenguang Wang",
                "Yangqiu Song",
                "Dan Roth",
                "Ming Zhang",
                "Jiawei Han"
            ],
            "abstract": "One of the key obstacles in making learning protocols realistic in\napplications is the need to supervise them, a costly process that often\nrequires hiring domain experts. We consider the framework to use the world\nknowledge as indirect supervision. World knowledge is general-purpose\nknowledge, which is not designed for any specific domain. Then the key\nchallenges are how to adapt the world knowledge to domains and how to represent\nit for learning. In this paper, we provide an example of using world knowledge\nfor domain dependent document clustering. We provide three ways to specify the\nworld knowledge to domains by resolving the ambiguity of the entities and their\ntypes, and represent the data with world knowledge as a heterogeneous\ninformation network. Then we propose a clustering algorithm that can cluster\nmultiple types and incorporate the sub-type information as constraints. In the\nexperiments, we use two existing knowledge bases as our sources of world\nknowledge. One is Freebase, which is collaboratively collected knowledge about\nentities and their organizations. The other is YAGO2, a knowledge base\nautomatically extracted from Wikipedia and maps knowledge to the linguistic\nknowledge base, WordNet. Experimental results on two text benchmark datasets\n(20newsgroups and RCV1) show that incorporating world knowledge as indirect\nsupervision can significantly outperform the state-of-the-art clustering\nalgorithms as well as clustering algorithms enhanced with world knowledge\nfeatures.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1608.00255v2",
            "date": "2016-07-31 20:01:04+00:00",
            "title": "Continuation semantics for multi-quantifier sentences: operation-based approaches",
            "authors": [
                "Justyna Grudzinska",
                "Marek Zawadowski"
            ],
            "abstract": "Classical scope-assignment strategies for multi-quantifier sentences involve\nquantifier phrase (QP)-movement. More recent continuation-based approaches\nprovide a compelling alternative, for they interpret QP's in situ - without\nresorting to Logical Forms or any structures beyond the overt syntax. The\ncontinuation-based strategies can be divided into two groups: those that locate\nthe source of scope-ambiguity in the rules of semantic composition and those\nthat attribute it to the lexical entries for the quantifier words. In this\npaper, we focus on the former operation-based approaches and the nature of the\nsemantic operations involved. More specifically, we discuss three such possible\noperation-based strategies for multi-quantifier sentences, together with their\nrelative merits and costs.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1608.00329v2",
            "date": "2016-08-01 06:00:22+00:00",
            "title": "Keyphrase Extraction using Sequential Labeling",
            "authors": [
                "Sujatha Das Gollapalli",
                "Xiao-li Li"
            ],
            "abstract": "Keyphrases efficiently summarize a document's content and are used in various\ndocument processing and retrieval tasks. Several unsupervised techniques and\nclassifiers exist for extracting keyphrases from text documents. Most of these\nmethods operate at a phrase-level and rely on part-of-speech (POS) filters for\ncandidate phrase generation. In addition, they do not directly handle\nkeyphrases of varying lengths. We overcome these modeling shortcomings by\naddressing keyphrase extraction as a sequential labeling task in this paper. We\nexplore a basic set of features commonly used in NLP tasks as well as\npredictions from various unsupervised methods to train our taggers. In addition\nto a more natural modeling for the keyphrase extraction problem, we show that\ntagging models yield significant performance benefits over existing\nstate-of-the-art extraction methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00466v2",
            "date": "2016-08-01 15:14:08+00:00",
            "title": "Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification",
            "authors": [
                "Madhusudan Lakshmana",
                "Sundararajan Sellamanickam",
                "Shirish Shevade",
                "Keerthi Selvaraj"
            ],
            "abstract": "The state-of-the-art CNN models give good performance on sentence\nclassification tasks. The purpose of this work is to empirically study\ndesirable properties such as semantic coherence, attention mechanism and\nreusability of CNNs in these tasks. Semantically coherent kernels are\npreferable as they are a lot more interpretable for explaining the decision of\nthe learned CNN model. We observe that the learned kernels do not have semantic\ncoherence. Motivated by this observation, we propose to learn kernels with\nsemantic coherence using clustering scheme combined with Word2Vec\nrepresentation and domain knowledge such as SentiWordNet. We suggest a\ntechnique to visualize attention mechanism of CNNs for decision explanation\npurpose. Reusable property enables kernels learned on one problem to be used in\nanother problem. This helps in efficient learning as only a few additional\ndomain specific filters may have to be learned. We demonstrate the efficacy of\nour core ideas of learning semantically coherent kernels and leveraging\nreusable kernels for efficient learning on several benchmark datasets.\nExperimental results show the usefulness of our approach by achieving\nperformance close to the state-of-the-art methods but with semantic and\nreusable properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.00895v2",
            "date": "2016-08-02 16:43:27+00:00",
            "title": "RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks",
            "authors": [
                "Patrick Doetsch",
                "Albert Zeyer",
                "Paul Voigtlaender",
                "Ilya Kulikov",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "In this work we release our extensible and easily configurable neural network\ntraining software. It provides a rich set of functional layers with a\nparticular focus on efficient training of recurrent neural network topologies\non multiple GPUs. The source of the software package is public and freely\navailable for academic research purposes and can be used as a framework or as a\nstandalone tool which supports a flexible configuration. The software allows to\ntrain state-of-the-art deep bidirectional long short-term memory (LSTM) models\non both one dimensional data like speech or two dimensional data like\nhandwritten text and was used to develop successful submission systems in\nseveral evaluation campaigns.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1608.01018v1",
            "date": "2016-08-02 22:28:45+00:00",
            "title": "Proceedings of the 2016 Workshop on Semantic Spaces at the Intersection of NLP, Physics and Cognitive Science",
            "authors": [
                "Dimitrios Kartsaklis",
                "Martha Lewis",
                "Laura Rimell"
            ],
            "abstract": "This volume contains the Proceedings of the 2016 Workshop on Semantic Spaces\nat the Intersection of NLP, Physics and Cognitive Science (SLPCS 2016), which\nwas held on the 11th of June at the University of Strathclyde, Glasgow, and was\nco-located with Quantum Physics and Logic (QPL 2016). Exploiting the common\nground provided by the concept of a vector space, the workshop brought together\nresearchers working at the intersection of Natural Language Processing (NLP),\ncognitive science, and physics, offering them an appropriate forum for\npresenting their uniquely motivated work and ideas. The interplay between these\nthree disciplines inspired theoretically motivated approaches to the\nunderstanding of how word meanings interact with each other in sentences and\ndiscourse, how diagrammatic reasoning depicts and simplifies this interaction,\nhow language models are determined by input from the world, and how word and\nsentence meanings interact logically. This first edition of the workshop\nconsisted of three invited talks from distinguished speakers (Hans Briegel,\nPeter G\\\"ardenfors, Dominic Widdows) and eight presentations of selected\ncontributed papers. Each submission was refereed by at least three members of\nthe Programme Committee, who delivered detailed and insightful comments and\nsuggestions.",
            "journal ref": "EPTCS 221, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01298v1",
            "date": "2016-08-03 19:34:13+00:00",
            "title": "A Physical Metaphor to Study Semantic Drift",
            "authors": [
                "S\u00e1ndor Dar\u00e1nyi",
                "Peter Wittek",
                "Konstantinos Konstantinidis",
                "Symeon Papadopoulos",
                "Efstratios Kontopoulos"
            ],
            "abstract": "In accessibility tests for digital preservation, over time we experience\ndrifts of localized and labelled content in statistical models of evolving\nsemantics represented as a vector field. This articulates the need to detect,\nmeasure, interpret and model outcomes of knowledge dynamics. To this end we\nemploy a high-performance machine learning algorithm for the training of\nextremely large emergent self-organizing maps for exploratory data analysis.\nThe working hypothesis we present here is that the dynamics of semantic drifts\ncan be modeled on a relaxed version of Newtonian mechanics called social\nmechanics. By using term distances as a measure of semantic relatedness vs.\ntheir PageRank values indicating social importance and applied as variable\n`term mass', gravitation as a metaphor to express changes in the semantic\ncontent of a vector field lends a new perspective for experimentation. From\n`term gravitation' over time, one can compute its generating potential whose\nfluctuations manifest modifications in pairwise term similarity vs. social\nimportance, thereby updating Osgood's semantic differential. The dataset\nexamined is the public catalog metadata of Tate Galleries, London.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01401v1",
            "date": "2016-08-04 00:36:12+00:00",
            "title": "Dual Density Operators and Natural Language Meaning",
            "authors": [
                "Daniela Ashoush",
                "Bob Coecke"
            ],
            "abstract": "Density operators allow for representing ambiguity about a vector\nrepresentation, both in quantum theory and in distributional natural language\nmeaning. Formally equivalently, they allow for discarding part of the\ndescription of a composite system, where we consider the discarded part to be\nthe context. We introduce dual density operators, which allow for two\nindependent notions of context. We demonstrate the use of dual density\noperators within a grammatical-compositional distributional framework for\nnatural language meaning. We show that dual density operators can be used to\nsimultaneously represent: (i) ambiguity about word meanings (e.g. queen as a\nperson vs. queen as a band), and (ii) lexical entailment (e.g. tiger ->\nmammal). We provide a proof-of-concept example.",
            "journal ref": "EPTCS 221, 2016, pp. 1-10",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.01402v1",
            "date": "2016-08-04 00:36:21+00:00",
            "title": "Interacting Conceptual Spaces",
            "authors": [
                "Josef Bolt",
                "Bob Coecke",
                "Fabrizio Genovese",
                "Martha Lewis",
                "Daniel Marsden",
                "Robin Piedeleu"
            ],
            "abstract": "We propose applying the categorical compositional scheme of [6] to conceptual\nspace models of cognition. In order to do this we introduce the category of\nconvex relations as a new setting for categorical compositional semantics,\nemphasizing the convex structure important to conceptual space applications. We\nshow how conceptual spaces for composite types such as adjectives and verbs can\nbe constructed. We illustrate this new model on detailed examples.",
            "journal ref": "EPTCS 221, 2016, pp. 11-19",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.01404v1",
            "date": "2016-08-04 00:36:57+00:00",
            "title": "Quantifier Scope in Categorical Compositional Distributional Semantics",
            "authors": [
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "In previous work with J. Hedges, we formalised a generalised quantifiers\ntheory of natural language in categorical compositional distributional\nsemantics with the help of bialgebras. In this paper, we show how quantifier\nscope ambiguity can be represented in that setting and how this representation\ncan be generalised to branching quantifiers.",
            "journal ref": "EPTCS 221, 2016, pp. 49-57",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02076v2",
            "date": "2016-08-06 07:16:31+00:00",
            "title": "Bi-directional Attention with Agreement for Dependency Parsing",
            "authors": [
                "Hao Cheng",
                "Hao Fang",
                "Xiaodong He",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "We develop a novel bi-directional attention model for dependency parsing,\nwhich learns to agree on headword predictions from the forward and backward\nparsing directions. The parsing procedure for each direction is formulated as\nsequentially querying the memory component that stores continuous headword\nembeddings. The proposed parser makes use of {\\it soft} headword embeddings,\nallowing the model to implicitly capture high-order parsing history without\ndramatically increasing the computational complexity. We conduct experiments on\nEnglish, Chinese, and 12 other languages from the CoNLL 2006 shared task,\nshowing that the proposed model achieves state-of-the-art unlabeled attachment\nscores on 6 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.02289v1",
            "date": "2016-08-08 00:59:03+00:00",
            "title": "Detecting Sarcasm in Multimodal Social Platforms",
            "authors": [
                "Rossano Schifanella",
                "Paloma de Juan",
                "Joel Tetreault",
                "Liangliang Cao"
            ],
            "abstract": "Sarcasm is a peculiar form of sentiment expression, where the surface\nsentiment differs from the implied sentiment. The detection of sarcasm in\nsocial media platforms has been applied in the past mainly to textual\nutterances where lexical indicators (such as interjections and intensifiers),\nlinguistic markers, and contextual information (such as user profiles, or past\nconversations) were used to detect the sarcastic tone. However, modern social\nmedia platforms allow to create multimodal messages where audiovisual content\nis integrated with the text, making the analysis of a mode in isolation\npartial. In our work, we first study the relationship between the textual and\nvisual aspects in multimodal posts from three major social media platforms,\ni.e., Instagram, Tumblr and Twitter, and we run a crowdsourcing task to\nquantify the extent to which images are perceived as necessary by human\nannotators. Moreover, we propose two different computational frameworks to\ndetect sarcasm that integrate the textual and visual modalities. The first\napproach exploits visual semantics trained on an external dataset, and\nconcatenates the semantics features with state-of-the-art textual features. The\nsecond method adapts a visual neural network initialized with parameters\ntrained on ImageNet to multimodal sarcastic posts. Results show the positive\neffect of combining modalities for the detection of sarcasm across platforms\nand methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1608.02717v1",
            "date": "2016-08-09 08:24:02+00:00",
            "title": "Mean Box Pooling: A Rich Image Representation and Output Embedding for the Visual Madlibs Task",
            "authors": [
                "Ashkan Mokarian",
                "Mateusz Malinowski",
                "Mario Fritz"
            ],
            "abstract": "We present Mean Box Pooling, a novel visual representation that pools over\nCNN representations of a large number, highly overlapping object proposals. We\nshow that such representation together with nCCA, a successful multimodal\nembedding technique, achieves state-of-the-art performance on the Visual\nMadlibs task. Moreover, inspired by the nCCA's objective function, we extend\nclassical CNN+LSTM approach to train the network by directly maximizing the\nsimilarity between the internal representation of the deep learning\narchitecture and candidate answers. Again, such approach achieves a significant\nimprovement over the prior work that also uses CNN+LSTM approach on Visual\nMadlibs.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1608.02893v2",
            "date": "2016-08-08 01:30:45+00:00",
            "title": "Syntactically Informed Text Compression with Recurrent Neural Networks",
            "authors": [
                "David Cox"
            ],
            "abstract": "We present a self-contained system for constructing natural language models\nfor use in text compression. Our system improves upon previous neural network\nbased models by utilizing recent advances in syntactic parsing -- Google's\nSyntaxNet -- to augment character-level recurrent neural networks. RNNs have\nproven exceptional in modeling sequence data such as text, as their\narchitecture allows for modeling of long-term contextual information.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1608.02996v1",
            "date": "2016-08-09 22:24:16+00:00",
            "title": "Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders",
            "authors": [
                "Antonio Valerio Miceli Barone"
            ],
            "abstract": "Current approaches to learning vector representations of text that are\ncompatible between different languages usually require some amount of parallel\ntext, aligned at word, sentence or at least document level. We hypothesize\nhowever, that different natural languages share enough semantic structure that\nit should be possible, in principle, to learn compatible vector representations\njust by analyzing the monolingual distribution of words.\n  In order to evaluate this hypothesis, we propose a scheme to map word vectors\ntrained on a source language to vectors semantically compatible with word\nvectors trained on a target language using an adversarial autoencoder.\n  We present preliminary qualitative results and discuss possible future\ndevelopments of this technique, such as applications to cross-lingual sentence\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03785v1",
            "date": "2016-08-12 13:13:10+00:00",
            "title": "Compositional Distributional Cognition",
            "authors": [
                "Yaared Al-Mehairi",
                "Bob Coecke",
                "Martha Lewis"
            ],
            "abstract": "We accommodate the Integrated Connectionist/Symbolic Architecture (ICS) of\n[32] within the categorical compositional semantics (CatCo) of [13], forming a\nmodel of categorical compositional cognition (CatCog). This resolves intrinsic\nproblems with ICS such as the fact that representations inhabit an unbounded\nspace and that sentences with differing tree structures cannot be directly\ncompared. We do so in a way that makes the most of the grammatical structure\navailable, in contrast to strategies like circular convolution. Using the CatCo\nmodel also allows us to make use of tools developed for CatCo such as the\nrepresentation of ambiguity and logical reasoning via density matrices,\nstructural meanings for words such as relative pronouns, and addressing over-\nand under-extension, all of which are present in cognitive processes. Moreover\nthe CatCog framework is sufficiently flexible to allow for entirely different\nrepresentations of meaning, such as conceptual spaces. Interestingly, since the\nCatCo model was largely inspired by categorical quantum mechanics, so is\nCatCog.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.03902v1",
            "date": "2016-08-12 20:19:16+00:00",
            "title": "Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks",
            "authors": [
                "Dat Tien Nguyen",
                "Kamela Ali Al Mannai",
                "Shafiq Joty",
                "Hassan Sajjad",
                "Muhammad Imran",
                "Prasenjit Mitra"
            ],
            "abstract": "The role of social media, in particular microblogging platforms such as\nTwitter, as a conduit for actionable and tactical information during disasters\nis increasingly acknowledged. However, time-critical analysis of big crisis\ndata on social media streams brings challenges to machine learning techniques,\nespecially the ones that use supervised learning. The Scarcity of labeled data,\nparticularly in the early hours of a crisis, delays the machine learning\nprocess. The current state-of-the-art classification methods require a\nsignificant amount of labeled data specific to a particular event for training\nplus a lot of feature engineering to achieve best results. In this work, we\nintroduce neural network based classification methods for binary and\nmulti-class tweet classification task. We show that neural network based models\ndo not require any feature engineering and perform better than state-of-the-art\nmethods. In the early hours of a disaster when no labeled data is available,\nour proposed method makes the best use of the out-of-event data and achieves\ngood results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.03938v1",
            "date": "2016-08-13 04:02:38+00:00",
            "title": "Determining Health Utilities through Data Mining of Social Media",
            "authors": [
                "Christopher Thompson",
                "Josh Introne",
                "Clint Young"
            ],
            "abstract": "'Health utilities' measure patient preferences for perfect health compared to\nspecific unhealthy states, such as asthma, a fractured hip, or colon cancer.\nWhen integrated over time, these estimations are called quality adjusted life\nyears (QALYs). Until now, characterizing health utilities (HUs) required\ndetailed patient interviews or written surveys. While reliable and specific,\nthis data remained costly due to efforts to locate, enlist and coordinate\nparticipants. Thus the scope, context and temporality of diseases examined has\nremained limited.\n  Now that more than a billion people use social media, we propose a novel\nstrategy: use natural language processing to analyze public online\nconversations for signals of the severity of medical conditions and correlate\nthese to known HUs using machine learning. In this work, we filter a dataset\nthat originally contained 2 billion tweets for relevant content on 60 diseases.\nUsing this data, our algorithm successfully distinguished mild from severe\ndiseases, which had previously been categorized only by traditional techniques.\nThis represents progress towards two related applications: first, predicting\nHUs where such information is nonexistent; and second, (where rich HU data\nalready exists) estimating temporal or geographic patterns of disease severity\nthrough data mining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04089v1",
            "date": "2016-08-14 11:36:52+00:00",
            "title": "Viewpoint and Topic Modeling of Current Events",
            "authors": [
                "Kerry Zhang",
                "Jussi Karlgren",
                "Cheng Zhang",
                "Jens Lagergren"
            ],
            "abstract": "There are multiple sides to every story, and while statistical topic models\nhave been highly successful at topically summarizing the stories in corpora of\ntext documents, they do not explicitly address the issue of learning the\ndifferent sides, the viewpoints, expressed in the documents. In this paper, we\nshow how these viewpoints can be learned completely unsupervised and\nrepresented in a human interpretable form. We use a novel approach of applying\nCorrLDA2 for this purpose, which learns topic-viewpoint relations that can be\nused to form groups of topics, where each group represents a viewpoint. A\ncorpus of documents about the Israeli-Palestinian conflict is then used to\ndemonstrate how a Palestinian and an Israeli viewpoint can be learned. By\nleveraging the magnitudes and signs of the feature weights of a linear SVM, we\nintroduce a principled method to evaluate associations between topics and\nviewpoints. With this, we demonstrate, both quantitatively and qualitatively,\nthat the learned topic groups are contextually coherent, and form consistently\ncorrect topic-viewpoint associations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.04868v2",
            "date": "2016-08-17 06:24:46+00:00",
            "title": "Towards Music Captioning: Generating Music Playlist Descriptions",
            "authors": [
                "Keunwoo Choi",
                "George Fazekas",
                "Brian McFee",
                "Kyunghyun Cho",
                "Mark Sandler"
            ],
            "abstract": "Descriptions are often provided along with recommendations to help users'\ndiscovery. Recommending automatically generated music playlists (e.g.\npersonalised playlists) introduces the problem of generating descriptions. In\nthis paper, we propose a method for generating music playlist descriptions,\nwhich is called as music captioning. In the proposed method, audio content\nanalysis and natural language processing are adopted to utilise the information\nof each track.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1608.04917v2",
            "date": "2016-08-17 10:10:14+00:00",
            "title": "Cohesion and Coalition Formation in the European Parliament: Roll-Call Votes and Twitter Activities",
            "authors": [
                "Darko Cherepnalkoski",
                "Andreas Karpf",
                "Igor Mozetic",
                "Miha Grcar"
            ],
            "abstract": "We study the cohesion within and the coalitions between political groups in\nthe Eighth European Parliament (2014--2019) by analyzing two entirely different\naspects of the behavior of the Members of the European Parliament (MEPs) in the\npolicy-making processes. On one hand, we analyze their co-voting patterns and,\non the other, their retweeting behavior. We make use of two diverse datasets in\nthe analysis. The first one is the roll-call vote dataset, where cohesion is\nregarded as the tendency to co-vote within a group, and a coalition is formed\nwhen the members of several groups exhibit a high degree of co-voting agreement\non a subject. The second dataset comes from Twitter; it captures the retweeting\n(i.e., endorsing) behavior of the MEPs and implies cohesion (retweets within\nthe same group) and coalitions (retweets between groups) from a completely\ndifferent perspective.\n  We employ two different methodologies to analyze the cohesion and coalitions.\nThe first one is based on Krippendorff's Alpha reliability, used to measure the\nagreement between raters in data-analysis scenarios, and the second one is\nbased on Exponential Random Graph Models, often used in social-network\nanalysis. We give general insights into the cohesion of political groups in the\nEuropean Parliament, explore whether coalitions are formed in the same way for\ndifferent policy areas, and examine to what degree the retweeting behavior of\nMEPs corresponds to their co-voting patterns. A novel and interesting aspect of\nour work is the relationship between the co-voting and retweeting patterns.",
            "journal ref": "PLoS ONE 11(11): e0166586, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.06651v2",
            "date": "2016-08-23 20:55:09+00:00",
            "title": "Unsupervised, Efficient and Semantic Expertise Retrieval",
            "authors": [
                "Christophe Van Gysel",
                "Maarten de Rijke",
                "Marcel Worring"
            ],
            "abstract": "We introduce an unsupervised discriminative model for the task of retrieving\nexperts in online document collections. We exclusively employ textual evidence\nand avoid explicit feature engineering by learning distributed word\nrepresentations in an unsupervised way. We compare our model to\nstate-of-the-art unsupervised statistical vector space and probabilistic\ngenerative approaches. Our proposed log-linear model achieves the retrieval\nperformance levels of state-of-the-art document-centric methods with the low\ninference cost of so-called profile-centric approaches. It yields a\nstatistically significant improved ranking over vector space and generative\nmodels in most cases, matching the performance of supervised methods on various\nbenchmarks. That is, by using solely text we can do as well as methods that\nwork with external evidence and/or relevance feedback. A contrastive analysis\nof rankings produced by discriminative and generative approaches shows that\nthey have complementary strengths due to the ability of the unsupervised\ndiscriminative model to perform semantic matching.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.07187v4",
            "date": "2016-08-25 15:07:17+00:00",
            "title": "Semantics derived automatically from language corpora contain human-like biases",
            "authors": [
                "Aylin Caliskan",
                "Joanna J. Bryson",
                "Arvind Narayanan"
            ],
            "abstract": "Artificial intelligence and machine learning are in a period of astounding\ngrowth. However, there are concerns that these technologies may be used, either\nwith or without intention, to perpetuate the prejudice and unfairness that\nunfortunately characterizes many human institutions. Here we show for the first\ntime that human-like semantic biases result from the application of standard\nmachine learning to ordinary language---the same sort of language humans are\nexposed to every day. We replicate a spectrum of standard human biases as\nexposed by the Implicit Association Test and other well-known psychological\nstudies. We replicate these using a widely used, purely statistical\nmachine-learning model---namely, the GloVe word embedding---trained on a corpus\nof text from the Web. Our results indicate that language itself contains\nrecoverable and accurate imprints of our historic biases, whether these are\nmorally neutral as towards insects or flowers, problematic as towards race or\ngender, or even simply veridical, reflecting the {\\em status quo} for the\ndistribution of gender with respect to careers or first names. These\nregularities are captured by machine learning along with the rest of semantics.\nIn addition to our empirical findings concerning language, we also contribute\nnew methods for evaluating bias in text, the Word Embedding Association Test\n(WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results\nhave implications not only for AI and machine learning, but also for the fields\nof psychology, sociology, and human ethics, since they raise the possibility\nthat mere exposure to everyday language can account for the biases we replicate\nhere.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.07253v1",
            "date": "2016-08-25 18:57:50+00:00",
            "title": "Learning Latent Vector Spaces for Product Search",
            "authors": [
                "Christophe Van Gysel",
                "Maarten de Rijke",
                "Evangelos Kanoulas"
            ],
            "abstract": "We introduce a novel latent vector space model that jointly learns the latent\nrepresentations of words, e-commerce products and a mapping between the two\nwithout the need for explicit annotations. The power of the model lies in its\nability to directly model the discriminative relation between products and a\nparticular word. We compare our method to existing latent vector space models\n(LSI, LDA and word2vec) and evaluate it as a feature in a learning to rank\nsetting. Our latent vector space model achieves its enhanced performance as it\nlearns better product representations. Furthermore, the mapping from words to\nproducts and the representations of words benefit directly from the errors\npropagated back from the product representations during parameter estimation.\nWe provide an in-depth analysis of the performance of our model and analyze the\nstructure of the learned representations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1608.07639v1",
            "date": "2016-08-27 00:34:00+00:00",
            "title": "Learning to generalize to new compositions in image understanding",
            "authors": [
                "Yuval Atzmon",
                "Jonathan Berant",
                "Vahid Kezami",
                "Amir Globerson",
                "Gal Chechik"
            ],
            "abstract": "Recurrent neural networks have recently been used for learning to describe\nimages using natural language. However, it has been observed that these models\ngeneralize poorly to scenes that were not observed during training, possibly\ndepending too strongly on the statistics of the text in the training data. Here\nwe propose to describe images using short structured representations, aiming to\ncapture the crux of a description. These structured representations allow us to\ntease-out and evaluate separately two types of generalization: standard\ngeneralization to new images with similar scenes, and generalization to new\ncombinations of known entities. We compare two learning approaches on the\nMS-COCO dataset: a state-of-the-art recurrent network based on an LSTM (Show,\nAttend and Tell), and a simple structured prediction model on top of a deep\nnetwork. We find that the structured model generalizes to new compositions\nsubstantially better than the LSTM, ~7 times the accuracy of predicting\nstructured representations. By providing a concrete method to quantify\ngeneralization for unseen combinations, we argue that structured\nrepresentations and compositional splits are a useful benchmark for image\ncaptioning, and advocate compositional models that capture linguistic and\nvisual structure.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1608.07852v1",
            "date": "2016-08-28 20:31:37+00:00",
            "title": "Quantitative Analyses of Chinese Poetry of Tang and Song Dynasties: Using Changing Colors and Innovative Terms as Examples",
            "authors": [
                "Chao-Lin Liu"
            ],
            "abstract": "Tang (618-907 AD) and Song (960-1279) dynasties are two very important\nperiods in the development of Chinese literary. The most influential forms of\nthe poetry in Tang and Song were Shi and Ci, respectively. Tang Shi and Song Ci\nestablished crucial foundations of the Chinese literature, and their influences\nin both literary works and daily lives of the Chinese communities last until\ntoday.\n  We can analyze and compare the Complete Tang Shi and the Complete Song Ci\nfrom various viewpoints. In this presentation, we report our findings about the\ndifferences in their vocabularies. Interesting new words that started to appear\nin Song Ci and continue to be used in modern Chinese were identified. Colors\nare an important ingredient of the imagery in poetry, and we discuss the most\nfrequent color words that appeared in Tang Shi and Song Ci.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08176v4",
            "date": "2016-08-29 18:45:00+00:00",
            "title": "What is Wrong with Topic Modeling? (and How to Fix it Using Search-based Software Engineering)",
            "authors": [
                "Amritanshu Agrawal",
                "Wei Fu",
                "Tim Menzies"
            ],
            "abstract": "Context: Topic modeling finds human-readable structures in unstructured\ntextual data. A widely used topic modeler is Latent Dirichlet allocation. When\nrun on different datasets, LDA suffers from \"order effects\" i.e. different\ntopics are generated if the order of training data is shuffled. Such order\neffects introduce a systematic error for any study. This error can relate to\nmisleading results;specifically, inaccurate topic descriptions and a reduction\nin the efficacy of text mining classification results. Objective: To provide a\nmethod in which distributions generated by LDA are more stable and can be used\nfor further analysis. Method: We use LDADE, a search-based software engineering\ntool that tunes LDA's parameters using DE (Differential Evolution). LDADE is\nevaluated on data from a programmer information exchange site (Stackoverflow),\ntitle and abstract text of thousands ofSoftware Engineering (SE) papers, and\nsoftware defect reports from NASA. Results were collected across different\nimplementations of LDA (Python+Scikit-Learn, Scala+Spark); across different\nplatforms (Linux, Macintosh) and for different kinds of LDAs (VEM,or using\nGibbs sampling). Results were scored via topic stability and text mining\nclassification accuracy. Results: In all treatments: (i) standard LDA exhibits\nvery large topic instability; (ii) LDADE's tunings dramatically reduce cluster\ninstability; (iii) LDADE also leads to improved performances for supervised as\nwell as unsupervised learning. Conclusion: Due to topic instability, using\nstandard LDA with its \"off-the-shelf\" settings should now be depreciated. Also,\nin future, we should require SE papers that use LDA to test and (if needed)\nmitigate LDA topic instability. Finally, LDADE is a candidate technology for\neffectively and efficiently reducing that instability.",
            "journal ref": "Information and Software Technology Journal, 2018",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1608.08188v1",
            "date": "2016-08-29 19:24:25+00:00",
            "title": "Visual Question: Predicting If a Crowd Will Agree on the Answer",
            "authors": [
                "Danna Gurari",
                "Kristen Grauman"
            ],
            "abstract": "Visual question answering (VQA) systems are emerging from a desire to empower\nusers to ask any natural language question about visual content and receive a\nvalid answer in response. However, close examination of the VQA problem reveals\nan unavoidable, entangled problem that multiple humans may or may not always\nagree on a single answer to a visual question. We train a model to\nautomatically predict from a visual question whether a crowd would agree on a\nsingle answer. We then propose how to exploit this system in a novel\napplication to efficiently allocate human effort to collect answers to visual\nquestions. Specifically, we propose a crowdsourcing system that automatically\nsolicits fewer human responses when answer agreement is expected and more human\nresponses when answer disagreement is expected. Our system improves upon\nexisting crowdsourcing systems, typically eliminating at least 20% of human\neffort with no loss to the information collected from the crowd.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.08716v1",
            "date": "2016-08-31 02:56:00+00:00",
            "title": "Measuring Machine Intelligence Through Visual Question Answering",
            "authors": [
                "C. Lawrence Zitnick",
                "Aishwarya Agrawal",
                "Stanislaw Antol",
                "Margaret Mitchell",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "As machines have become more intelligent, there has been a renewed interest\nin methods for measuring their intelligence. A common approach is to propose\ntasks for which a human excels, but one which machines find difficult. However,\nan ideal task should also be easy to evaluate and not be easily gameable. We\nbegin with a case study exploring the recently popular task of image captioning\nand its limitations as a task for measuring machine intelligence. An\nalternative and more promising task is Visual Question Answering that tests a\nmachine's ability to reason about language and vision. We describe a dataset\nunprecedented in size created for the task that contains over 760,000 human\ngenerated questions about images. Using around 10 million human generated\nanswers, machines may be easily evaluated.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1608.08940v1",
            "date": "2016-08-31 17:01:09+00:00",
            "title": "Hash2Vec, Feature Hashing for Word Embeddings",
            "authors": [
                "Luis Argerich",
                "Joaqu\u00edn Torr\u00e9 Zaffaroni",
                "Mat\u00edas J Cano"
            ],
            "abstract": "In this paper we propose the application of feature hashing to create word\nembeddings for natural language processing. Feature hashing has been used\nsuccessfully to create document vectors in related tasks like document\nclassification. In this work we show that feature hashing can be applied to\nobtain word embeddings in linear time with the size of the data. The results\nshow that this algorithm, that does not need training, is able to capture the\nsemantic meaning of words. We compare the results against GloVe showing that\nthey are similar. As far as we know this is the first application of feature\nhashing to the word embeddings problem and the results indicate this is a\nscalable technique with practical results for NLP applications.",
            "journal ref": "45 JAIIO - ASAI 2016 - ISSN: 2451-7585 - Pages 33-40",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1608.08953v2",
            "date": "2016-08-31 17:20:09+00:00",
            "title": "Dynamic Allocation of Crowd Contributions for Sentiment Analysis during the 2016 U.S. Presidential Election",
            "authors": [
                "Mehrnoosh Sameki",
                "Mattia Gentil",
                "Kate K. Mays",
                "Lei Guo",
                "Margrit Betke"
            ],
            "abstract": "Opinions about the 2016 U.S. Presidential Candidates have been expressed in\nmillions of tweets that are challenging to analyze automatically. Crowdsourcing\nthe analysis of political tweets effectively is also difficult, due to large\ninter-rater disagreements when sarcasm is involved. Each tweet is typically\nanalyzed by a fixed number of workers and majority voting. We here propose a\ncrowdsourcing framework that instead uses a dynamic allocation of the number of\nworkers. We explore two dynamic-allocation methods: (1) The number of workers\nqueried to label a tweet is computed offline based on the predicted difficulty\nof discerning the sentiment of a particular tweet. (2) The number of crowd\nworkers is determined online, during an iterative crowd sourcing process, based\non inter-rater agreements between labels.We applied our approach to 1,000\ntwitter messages about the four U.S. presidential candidates Clinton, Cruz,\nSanders, and Trump, collected during February 2016. We implemented the two\nproposed methods using decision trees that allocate more crowd efforts to\ntweets predicted to be sarcastic. We show that our framework outperforms the\ntraditional static allocation scheme. It collects opinion labels from the crowd\nat a much lower cost while maintaining labeling accuracy.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1608.08974v2",
            "date": "2016-08-31 18:11:29+00:00",
            "title": "Towards Transparent AI Systems: Interpreting Visual Question Answering Models",
            "authors": [
                "Yash Goyal",
                "Akrit Mohapatra",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "Deep neural networks have shown striking progress and obtained\nstate-of-the-art results in many AI research fields in the recent years.\nHowever, it is often unsatisfying to not know why they predict what they do. In\nthis paper, we address the problem of interpreting Visual Question Answering\n(VQA) models. Specifically, we are interested in finding what part of the input\n(pixels in images or words in questions) the VQA model focuses on while\nanswering the question. To tackle this problem, we use two visualization\ntechniques -- guided backpropagation and occlusion -- to find important words\nin the question and important regions in the image. We then present qualitative\nand quantitative analyses of these importance maps. We found that even without\nexplicit attention mechanisms, VQA models may sometimes be implicitly attending\nto relevant regions in the image, and often to appropriate words in the\nquestion.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.00020v4",
            "date": "2016-10-31 20:07:23+00:00",
            "title": "Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision",
            "authors": [
                "Chen Liang",
                "Jonathan Berant",
                "Quoc Le",
                "Kenneth D. Forbus",
                "Ni Lao"
            ],
            "abstract": "Harnessing the statistical power of neural networks to perform language\nunderstanding and symbolic reasoning is difficult, when it requires executing\nefficient discrete operations against a large knowledge-base. In this work, we\nintroduce a Neural Symbolic Machine, which contains (a) a neural \"programmer\",\ni.e., a sequence-to-sequence model that maps language utterances to programs\nand utilizes a key-variable memory to handle compositionality (b) a symbolic\n\"computer\", i.e., a Lisp interpreter that performs program execution, and helps\nfind good programs by pruning the search space. We apply REINFORCE to directly\noptimize the task reward of this structured prediction problem. To train with\nweak supervision and improve the stability of REINFORCE, we augment it with an\niterative maximum-likelihood training process. NSM outperforms the\nstate-of-the-art on the WebQuestionsSP dataset when trained from\nquestion-answer pairs only, without requiring any feature engineering or\ndomain-specific knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00138v1",
            "date": "2016-11-01 06:05:49+00:00",
            "title": "MusicMood: Predicting the mood of music from song lyrics using machine learning",
            "authors": [
                "Sebastian Raschka"
            ],
            "abstract": "Sentiment prediction of contemporary music can have a wide-range of\napplications in modern society, for instance, selecting music for public\ninstitutions such as hospitals or restaurants to potentially improve the\nemotional well-being of personnel, patients, and customers, respectively. In\nthis project, music recommendation system built upon on a naive Bayes\nclassifier, trained to predict the sentiment of songs based on song lyrics\nalone. The experimental results show that music corresponding to a happy mood\ncan be detected with high precision based on text features obtained from song\nlyrics.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.00356v1",
            "date": "2016-11-01 19:59:48+00:00",
            "title": "Using Artificial Intelligence to Identify State Secrets",
            "authors": [
                "Renato Rocha Souza",
                "Flavio Codeco Coelho",
                "Rohan Shah",
                "Matthew Connelly"
            ],
            "abstract": "Whether officials can be trusted to protect national security information has\nbecome a matter of great public controversy, reigniting a long-standing debate\nabout the scope and nature of official secrecy. The declassification of\nmillions of electronic records has made it possible to analyze these issues\nwith greater rigor and precision. Using machine-learning methods, we examined\nnearly a million State Department cables from the 1970s to identify features of\nrecords that are more likely to be classified, such as international\nnegotiations, military operations, and high-level communications. Even with\nincomplete data, algorithms can use such features to identify 90% of classified\ncables with <11% false positives. But our results also show that there are\nlongstanding problems in the identification of sensitive information. Error\nanalysis reveals many examples of both overclassification and\nunderclassification. This indicates both the need for research on inter-coder\nreliability among officials as to what constitutes classified material and the\nopportunity to develop recommender systems to better manage both classification\nand declassification.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1611.00384v2",
            "date": "2016-11-01 20:48:34+00:00",
            "title": "CB2CF: A Neural Multiview Content-to-Collaborative Filtering Model for Completely Cold Item Recommendations",
            "authors": [
                "Oren Barkan",
                "Noam Koenigstein",
                "Eylon Yogev",
                "Ori Katz"
            ],
            "abstract": "In Recommender Systems research, algorithms are often characterized as either\nCollaborative Filtering (CF) or Content Based (CB). CF algorithms are trained\nusing a dataset of user preferences while CB algorithms are typically based on\nitem profiles. These approaches harness different data sources and therefore\nthe resulting recommended items are generally very different. This paper\npresents the CB2CF, a deep neural multiview model that serves as a bridge from\nitems content into their CF representations. CB2CF is a real-world algorithm\ndesigned for Microsoft Store services that handle around a billion users\nworldwide. CB2CF is demonstrated on movies and apps recommendations, where it\nis shown to outperform an alternative CB model on completely cold items.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.00440v1",
            "date": "2016-11-02 01:45:28+00:00",
            "title": "And the Winner is ...: Bayesian Twitter-based Prediction on 2016 U.S. Presidential Election",
            "authors": [
                "Elvyna Tunggawan",
                "Yustinus Eko Soelistio"
            ],
            "abstract": "This paper describes a Naive-Bayesian predictive model for 2016 U.S.\nPresidential Election based on Twitter data. We use 33,708 tweets gathered\nsince December 16, 2015 until February 29, 2016. We introduce a simpler data\npreprocessing method to label the data and train the model. The model achieves\n95.8% accuracy on 10-fold cross validation and predicts Ted Cruz and Bernie\nSanders as Republican and Democratic nominee respectively. It achieves a\ncomparable result to those in its competitor methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.00514v1",
            "date": "2016-11-02 09:24:10+00:00",
            "title": "The Intelligent Voice 2016 Speaker Recognition System",
            "authors": [
                "Abbas Khosravani",
                "Cornelius Glackin",
                "Nazim Dugan",
                "G\u00e9rard Chollet",
                "Nigel Cannings"
            ],
            "abstract": "This paper presents the Intelligent Voice (IV) system submitted to the NIST\n2016 Speaker Recognition Evaluation (SRE). The primary emphasis of SRE this\nyear was on developing speaker recognition technology which is robust for novel\nlanguages that are much more heterogeneous than those used in the current\nstate-of-the-art, using significantly less training data, that does not contain\nmeta-data from those languages. The system is based on the state-of-the-art\ni-vector/PLDA which is developed on the fixed training condition, and the\nresults are reported on the protocol defined on the development set of the\nchallenge.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1611.01259v1",
            "date": "2016-11-04 03:45:03+00:00",
            "title": "Generalized Topic Modeling",
            "authors": [
                "Avrim Blum",
                "Nika Haghtalab"
            ],
            "abstract": "Recently there has been significant activity in developing algorithms with\nprovable guarantees for topic modeling. In standard topic models, a topic (such\nas sports, business, or politics) is viewed as a probability distribution $\\vec\na_i$ over words, and a document is generated by first selecting a mixture $\\vec\nw$ over topics, and then generating words i.i.d. from the associated mixture\n$A{\\vec w}$. Given a large collection of such documents, the goal is to recover\nthe topic vectors and then to correctly classify new documents according to\ntheir topic mixture.\n  In this work we consider a broad generalization of this framework in which\nwords are no longer assumed to be drawn i.i.d. and instead a topic is a complex\ndistribution over sequences of paragraphs. Since one could not hope to even\nrepresent such a distribution in general (even if paragraphs are given using\nsome natural feature representation), we aim instead to directly learn a\ndocument classifier. That is, we aim to learn a predictor that given a new\ndocument, accurately predicts its topic mixture, without learning the\ndistributions explicitly. We present several natural conditions under which one\ncan do this efficiently and discuss issues such as noise tolerance and sample\ncomplexity in this model. More generally, our model can be viewed as a\ngeneralization of the multi-view or co-training setting in machine learning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.01462v3",
            "date": "2016-11-04 17:36:20+00:00",
            "title": "Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling",
            "authors": [
                "Hakan Inan",
                "Khashayar Khosravi",
                "Richard Socher"
            ],
            "abstract": "Recurrent neural networks have been very successful at predicting sequences\nof words in tasks such as language modeling. However, all such models are based\non the conventional classification framework, where the model is trained\nagainst one-hot targets, and each word is represented both as an input and as\nan output in isolation. This causes inefficiencies in learning both in terms of\nutilizing all of the information and in terms of the number of parameters\nneeded to train. We introduce a novel theoretical framework that facilitates\nbetter learning in language modeling, and show that our framework leads to\ntying together the input embedding and the output projection matrices, greatly\nreducing the number of trainable variables. Our framework leads to state of the\nart performance on the Penn Treebank with a variety of network models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.01576v2",
            "date": "2016-11-05 00:31:25+00:00",
            "title": "Quasi-Recurrent Neural Networks",
            "authors": [
                "James Bradbury",
                "Stephen Merity",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Recurrent neural networks are a powerful tool for modeling sequential data,\nbut the dependence of each timestep's computation on the previous timestep's\noutput limits parallelism and makes RNNs unwieldy for very long sequences. We\nintroduce quasi-recurrent neural networks (QRNNs), an approach to neural\nsequence modeling that alternates convolutional layers, which apply in parallel\nacross timesteps, and a minimalist recurrent pooling function that applies in\nparallel across channels. Despite lacking trainable recurrent layers, stacked\nQRNNs have better predictive accuracy than stacked LSTMs of the same hidden\nsize. Due to their increased parallelism, they are up to 16 times faster at\ntrain and test time. Experiments on language modeling, sentiment\nclassification, and character-level neural machine translation demonstrate\nthese advantages and underline the viability of QRNNs as a basic building block\nfor a variety of sequence tasks.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1611.01599v2",
            "date": "2016-11-05 04:05:18+00:00",
            "title": "LipNet: End-to-End Sentence-level Lipreading",
            "authors": [
                "Yannis M. Assael",
                "Brendan Shillingford",
                "Shimon Whiteson",
                "Nando de Freitas"
            ],
            "abstract": "Lipreading is the task of decoding text from the movement of a speaker's\nmouth. Traditional approaches separated the problem into two stages: designing\nor learning visual features, and prediction. More recent deep lipreading\napproaches are end-to-end trainable (Wand et al., 2016; Chung & Zisserman,\n2016a). However, existing work on models trained end-to-end perform only word\nclassification, rather than sentence-level sequence prediction. Studies have\nshown that human lipreading performance increases for longer words (Easton &\nBasala, 1982), indicating the importance of features capturing temporal context\nin an ambiguous communication channel. Motivated by this observation, we\npresent LipNet, a model that maps a variable-length sequence of video frames to\ntext, making use of spatiotemporal convolutions, a recurrent network, and the\nconnectionist temporal classification loss, trained entirely end-to-end. To the\nbest of our knowledge, LipNet is the first end-to-end sentence-level lipreading\nmodel that simultaneously learns spatiotemporal visual features and a sequence\nmodel. On the GRID corpus, LipNet achieves 95.2% accuracy in sentence-level,\noverlapped speaker split task, outperforming experienced human lipreaders and\nthe previous 86.4% word-level state-of-the-art accuracy (Gergen et al., 2016).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.01702v2",
            "date": "2016-11-05 21:25:07+00:00",
            "title": "TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency",
            "authors": [
                "Adji B. Dieng",
                "Chong Wang",
                "Jianfeng Gao",
                "John Paisley"
            ],
            "abstract": "In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based\nlanguage model designed to directly capture the global semantic meaning\nrelating words in a document via latent topics. Because of their sequential\nnature, RNNs are good at capturing the local structure of a word sequence -\nboth semantic and syntactic - but might face difficulty remembering long-range\ndependencies. Intuitively, these long-range dependencies are of semantic\nnature. In contrast, latent topic models are able to capture the global\nunderlying semantic structure of a document but do not account for word\nordering. The proposed TopicRNN model integrates the merits of RNNs and latent\ntopic models: it captures local (syntactic) dependencies using an RNN and\nglobal (semantic) dependencies using latent topics. Unlike previous work on\ncontextual RNN language modeling, our model is learned end-to-end. Empirical\nresults on word prediction show that TopicRNN outperforms existing contextual\nRNN baselines. In addition, TopicRNN can be used as an unsupervised feature\nextractor for documents. We do this for sentiment analysis on the IMDB movie\nreview dataset and report an error rate of $6.28\\%$. This is comparable to the\nstate-of-the-art $5.91\\%$ resulting from a semi-supervised approach. Finally,\nTopicRNN also yields sensible topics, making it a useful alternative to\ndocument models such as latent Dirichlet allocation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.01802v2",
            "date": "2016-11-06 16:08:21+00:00",
            "title": "Self-Wiring Question Answering Systems",
            "authors": [
                "Ricardo Usbeck",
                "Jonathan Huthmann",
                "Nico Duldhardt",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "Question answering (QA) has been the subject of a resurgence over the past\nyears. The said resurgence has led to a multitude of question answering (QA)\nsystems being developed both by companies and research facilities. While a few\ncomponents of QA systems get reused across implementations, most systems do not\nleverage the full potential of component reuse. Hence, the development of QA\nsystems is currently still a tedious and time-consuming process. We address the\nchallenge of accelerating the creation of novel or tailored QA systems by\npresenting a concept for a self-wiring approach to composing QA systems. Our\napproach will allow the reuse of existing, web-based QA systems or modules\nwhile developing new QA platforms. To this end, it will rely on QA modules\nbeing described using the Web Ontology Language. Based on these descriptions,\nour approach will be able to automatically compose QA systems using a\ndata-driven approach automatically.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.02266v2",
            "date": "2016-11-07 20:57:24+00:00",
            "title": "Gaussian Attention Model and Its Application to Knowledge Base Embedding and Question Answering",
            "authors": [
                "Liwen Zhang",
                "John Winn",
                "Ryota Tomioka"
            ],
            "abstract": "We propose the Gaussian attention model for content-based neural memory\naccess. With the proposed attention model, a neural network has the additional\ndegree of freedom to control the focus of its attention from a laser sharp\nattention to a broad attention. It is applicable whenever we can assume that\nthe distance in the latent space reflects some notion of semantics. We use the\nproposed attention model as a scoring function for the embedding of a knowledge\nbase into a continuous vector space and then train a model that performs\nquestion answering about the entities in the knowledge base. The proposed\nattention model can handle both the propagation of uncertainty when following a\nseries of relations and also the conjunction of conditions in a natural way. On\na dataset of soccer players who participated in the FIFA World Cup 2014, we\ndemonstrate that our model can handle both path queries and conjunctive queries\nwell.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1611.02337v1",
            "date": "2016-11-07 23:05:40+00:00",
            "title": "Balotage in Argentina 2015, a sentiment analysis of tweets",
            "authors": [
                "Daniel Robins",
                "Fernando Emmanuel Frati",
                "Jonatan Alvarez",
                "Jose Texier"
            ],
            "abstract": "Twitter social network contains a large amount of information generated by\nits users. That information is composed of opinions and comments that may\nreflect trends in social behavior. There is talk of trend when it is possible\nto identify opinions and comments geared towards the same shared by a lot of\npeople direction. To determine if two or more written opinions share the same\naddress, techniques Natural Language Processing (NLP) are used. This paper\nproposes a methodology for predicting reflected in Twitter from the use of\nsentiment analysis functions NLP based on social behaviors. The case study was\nselected the 2015 Presidential in Argentina, and a software architecture Big\nData composed Vertica data base with the component called Pulse was used.\nThrough the analysis it was possible to detect trends in voting intentions with\nregard to the presidential candidates, achieving greater accuracy in predicting\nthat achieved with traditional systems surveys.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.02554v2",
            "date": "2016-11-08 15:18:44+00:00",
            "title": "The Neural Noisy Channel",
            "authors": [
                "Lei Yu",
                "Phil Blunsom",
                "Chris Dyer",
                "Edward Grefenstette",
                "Tomas Kocisky"
            ],
            "abstract": "We formulate sequence to sequence transduction as a noisy channel decoding\nproblem and use recurrent neural networks to parameterise the source and\nchannel models. Unlike direct models which can suffer from explaining-away\neffects during training, noisy channel models must produce outputs that explain\ntheir inputs, and their component models can be trained with not only paired\ntraining samples but also unpaired samples from the marginal output\ndistribution. Using a latent variable to control how much of the conditioning\nsequence the channel model needs to read in order to generate a subsequent\nsymbol, we obtain a tractable and effective beam search decoder. Experimental\nresults on abstractive sentence summarisation, morphological inflection, and\nmachine translation show that noisy channel models outperform direct models,\nand that they significantly benefit from increased amounts of unpaired output\ndata that direct models cannot easily use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02654v2",
            "date": "2016-11-08 19:04:09+00:00",
            "title": "Sentence Ordering and Coherence Modeling using Recurrent Neural Networks",
            "authors": [
                "Lajanugen Logeswaran",
                "Honglak Lee",
                "Dragomir Radev"
            ],
            "abstract": "Modeling the structure of coherent texts is a key NLP problem. The task of\ncoherently organizing a given set of sentences has been commonly used to build\nand evaluate models that understand such structure. We propose an end-to-end\nunsupervised deep learning approach based on the set-to-sequence framework to\naddress this problem. Our model strongly outperforms prior methods in the order\ndiscrimination task and a novel task of ordering abstracts from scientific\narticles. Furthermore, our work shows that useful text representations can be\nobtained by learning to order sentences. Visualizing the learned sentence\nrepresentations shows that the model captures high-level logical structure in\nparagraphs. Our representations perform comparably to state-of-the-art\npre-training methods on sentence similarity and paraphrase detection tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02683v2",
            "date": "2016-11-08 20:42:26+00:00",
            "title": "Unsupervised Pretraining for Sequence to Sequence Learning",
            "authors": [
                "Prajit Ramachandran",
                "Peter J. Liu",
                "Quoc V. Le"
            ],
            "abstract": "This work presents a general unsupervised learning method to improve the\naccuracy of sequence to sequence (seq2seq) models. In our method, the weights\nof the encoder and decoder of a seq2seq model are initialized with the\npretrained weights of two language models and then fine-tuned with labeled\ndata. We apply this method to challenging benchmarks in machine translation and\nabstractive summarization and find that it significantly improves the\nsubsequent supervised models. Our main result is that pretraining improves the\ngeneralization of seq2seq models. We achieve state-of-the art results on the\nWMT English$\\rightarrow$German task, surpassing a range of methods using both\nphrase-based machine translation and neural machine translation. Our method\nachieves a significant improvement of 1.3 BLEU from the previous best models on\nboth WMT'14 and WMT'15 English$\\rightarrow$German. We also conduct human\nevaluations on abstractive summarization and find that our method outperforms a\npurely supervised learning baseline in a statistically significant manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.02879v1",
            "date": "2016-11-09 10:24:52+00:00",
            "title": "Audio Visual Speech Recognition using Deep Recurrent Neural Networks",
            "authors": [
                "Abhinav Thanda",
                "Shankar M Venkatesan"
            ],
            "abstract": "In this work, we propose a training algorithm for an audio-visual automatic\nspeech recognition (AV-ASR) system using deep recurrent neural network\n(RNN).First, we train a deep RNN acoustic model with a Connectionist Temporal\nClassification (CTC) objective function. The frame labels obtained from the\nacoustic model are then used to perform a non-linear dimensionality reduction\nof the visual features using a deep bottleneck network. Audio and visual\nfeatures are fused and used to train a fusion RNN. The use of bottleneck\nfeatures for visual modality helps the model to converge properly during\ntraining. Our system is evaluated on GRID corpus. Our results show that\npresence of visual modality gives significant improvement in character error\nrate (CER) at various levels of noise even when the model is trained without\nnoisy data. We also provide a comparison of two fusion methods: feature fusion\nand decision fusion.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.03218v4",
            "date": "2016-11-10 08:44:52+00:00",
            "title": "Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence",
            "authors": [
                "Emilio Jorge",
                "Mikael K\u00e5geb\u00e4ck",
                "Fredrik D. Johansson",
                "Emil Gustavsson"
            ],
            "abstract": "Acquiring your first language is an incredible feat and not easily\nduplicated. Learning to communicate using nothing but a few pictureless books,\na corpus, would likely be impossible even for humans. Nevertheless, this is the\ndominating approach in most natural language processing today. As an\nalternative, we propose the use of situated interactions between agents as a\ndriving force for communication, and the framework of Deep Recurrent Q-Networks\nfor evolving a shared language grounded in the provided environment. We task\nthe agents with interactive image search in the form of the game Guess Who?.\nThe images from the game provide a non trivial environment for the agents to\ndiscuss and a natural grounding for the concepts they decide to encode in their\ncommunication. Our experiments show that the agents learn not only to encode\nphysical concepts in their words, i.e. grounding, but also that the agents\nlearn to hold a multi-step dialogue remembering the state of the dialogue from\nstep to step.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.03558v1",
            "date": "2016-11-11 01:21:20+00:00",
            "title": "Neural Networks Models for Entity Discovery and Linking",
            "authors": [
                "Dan Liu",
                "Wei Lin",
                "Shiliang Zhang",
                "Si Wei",
                "Hui Jiang"
            ],
            "abstract": "This paper describes the USTC_NELSLIP systems submitted to the Trilingual\nEntity Detection and Linking (EDL) track in 2016 TAC Knowledge Base Population\n(KBP) contests. We have built two systems for entity discovery and mention\ndetection (MD): one uses the conditional RNNLM and the other one uses the\nattention-based encoder-decoder framework. The entity linking (EL) system\nconsists of two modules: a rule based candidate generation and a neural\nnetworks probability ranking model. Moreover, some simple string matching rules\nare used for NIL clustering. At the end, our best system has achieved an F1\nscore of 0.624 in the end-to-end typed mention ceaf plus metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.03599v1",
            "date": "2016-11-11 07:05:49+00:00",
            "title": "UTCNN: a Deep Learning Model of Stance Classificationon on Social Media Text",
            "authors": [
                "Wei-Fan Chen",
                "Lun-Wei Ku"
            ],
            "abstract": "Most neural network models for document classification on social media focus\non text infor-mation to the neglect of other information on these platforms. In\nthis paper, we classify post stance on social media channels and develop UTCNN,\na neural network model that incorporates user tastes, topic tastes, and user\ncomments on posts. UTCNN not only works on social media texts, but also\nanalyzes texts in forums and message boards. Experiments performed on Chinese\nFacebook data and English online debate forum data show that UTCNN achieves a\n0.755 macro-average f-score for supportive, neutral, and unsupportive stance\nclasses on Facebook data, which is significantly better than models in which\neither user, topic, or comment information is withheld. This model design\ngreatly mitigates the lack of data for the minor class without the use of\noversampling. In addition, UTCNN yields a 0.842 accuracy on English online\ndebate forum data, which also significantly outperforms results from previous\nwork as well as other deep learning models, showing that UTCNN performs well\nregardless of language or platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04033v1",
            "date": "2016-11-12 18:41:58+00:00",
            "title": "1.5 billion words Arabic Corpus",
            "authors": [
                "Ibrahim Abu El-khair"
            ],
            "abstract": "This study is an attempt to build a contemporary linguistic corpus for Arabic\nlanguage. The corpus produced, is a text corpus includes more than five million\nnewspaper articles. It contains over a billion and a half words in total, out\nof which, there is about three million unique words. The data were collected\nfrom newspaper articles in ten major news sources from eight Arabic countries,\nover a period of fourteen years. The corpus was encoded with two types of\nencoding, namely: UTF-8, and Windows CP-1256. Also it was marked with two\nmark-up languages, namely: SGML, and XML.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04361v1",
            "date": "2016-11-14 12:36:07+00:00",
            "title": "Attending to Characters in Neural Sequence Labeling Models",
            "authors": [
                "Marek Rei",
                "Gamal K. O. Crichton",
                "Sampo Pyysalo"
            ],
            "abstract": "Sequence labeling architectures use word embeddings for capturing similarity,\nbut suffer when handling previously unseen or rare words. We investigate\ncharacter-level extensions to such models and propose a novel architecture for\ncombining alternative word representations. By using an attention mechanism,\nthe model is able to dynamically decide how much information to use from a\nword- or character-level component. We evaluated different architectures on a\nrange of sequence labeling datasets, and character-level extensions were found\nto improve performance on every benchmark. In addition, the proposed\nattention-based architecture delivered the best results even with a smaller\nnumber of trainable parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04503v3",
            "date": "2016-11-14 18:07:54+00:00",
            "title": "Zero-resource Machine Translation by Multimodal Encoder-decoder Network with Multimedia Pivot",
            "authors": [
                "Hideki Nakayama",
                "Noriki Nishida"
            ],
            "abstract": "We propose an approach to build a neural machine translation system with no\nsupervised resources (i.e., no parallel corpora) using multimodal embedded\nrepresentation over texts and images. Based on the assumption that text\ndocuments are often likely to be described with other multimedia information\n(e.g., images) somewhat related to the content, we try to indirectly estimate\nthe relevance between two languages. Using multimedia as the \"pivot\", we\nproject all modalities into one common hidden space where samples belonging to\nsimilar semantic concepts should come close to each other, whatever the\nobserved space of each sample is. This modality-agnostic representation is the\nkey to bridging the gap between different modalities. Putting a decoder on top\nof it, our network can flexibly draw the outputs from any input modality.\nNotably, in the testing phase, we need only source language texts as the input\nfor translation. In experiments, we tested our method on two benchmarks to show\nthat it can achieve reasonable translation performance. We compared and\ninvestigated several possible implementations and found that an end-to-end\nmodel that simultaneously optimized both rank loss in multimodal encoders and\ncross-entropy loss in decoders performed the best.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.04642v5",
            "date": "2016-11-14 22:54:45+00:00",
            "title": "Link Prediction using Embedded Knowledge Graphs",
            "authors": [
                "Yelong Shen",
                "Po-Sen Huang",
                "Ming-Wei Chang",
                "Jianfeng Gao"
            ],
            "abstract": "Since large knowledge bases are typically incomplete, missing facts need to\nbe inferred from observed facts in a task called knowledge base completion. The\nmost successful approaches to this task have typically explored explicit paths\nthrough sequences of triples. These approaches have usually resorted to\nhuman-designed sampling procedures, since large knowledge graphs produce\nprohibitively large numbers of possible paths, most of which are uninformative.\nAs an alternative approach, we propose performing a single, short sequence of\ninteractive lookup operations on an embedded knowledge graph which has been\ntrained through end-to-end backpropagation to be an optimized and compressed\nversion of the initial knowledge base. Our proposed model, called Embedded\nKnowledge Graph Network (EKGN), achieves new state-of-the-art results on\npopular knowledge base completion benchmarks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.05010v1",
            "date": "2016-11-15 20:06:40+00:00",
            "title": "Anchor-Free Correlated Topic Modeling: Identifiability and Algorithm",
            "authors": [
                "Kejun Huang",
                "Xiao Fu",
                "Nicholas D. Sidiropoulos"
            ],
            "abstract": "In topic modeling, many algorithms that guarantee identifiability of the\ntopics have been developed under the premise that there exist anchor words --\ni.e., words that only appear (with positive probability) in one topic.\nFollow-up work has resorted to three or higher-order statistics of the data\ncorpus to relax the anchor word assumption. Reliable estimates of higher-order\nstatistics are hard to obtain, however, and the identification of topics under\nthose models hinges on uncorrelatedness of the topics, which can be\nunrealistic. This paper revisits topic modeling based on second-order moments,\nand proposes an anchor-free topic mining framework. The proposed approach\nguarantees the identification of the topics under a much milder condition\ncompared to the anchor-word assumption, thereby exhibiting much better\nrobustness in practice. The associated algorithm only involves one\neigen-decomposition and a few small linear programs. This makes it easy to\nimplement and scale up to very large problem instances. Experiments using the\nTDT2 and Reuters-21578 corpus demonstrate that the proposed anchor-free\napproach exhibits very favorable performance (measured using coherence,\nsimilarity count, and clustering accuracy metrics) compared to the prior art.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1611.05379v1",
            "date": "2016-11-16 17:32:10+00:00",
            "title": "PCT and Beyond: Towards a Computational Framework for `Intelligent' Communicative Systems",
            "authors": [
                "Prof. Roger K. Moore"
            ],
            "abstract": "Recent years have witnessed increasing interest in the potential benefits of\n`intelligent' autonomous machines such as robots. Honda's Asimo humanoid robot,\niRobot's Roomba robot vacuum cleaner and Google's driverless cars have fired\nthe imagination of the general public, and social media buzz with speculation\nabout a utopian world of helpful robot assistants or the coming robot\napocalypse! However, there is a long way to go before autonomous systems reach\nthe level of capabilities required for even the simplest of tasks involving\nhuman-robot interaction - especially if it involves communicative behaviour\nsuch as speech and language. Of course the field of Artificial Intelligence\n(AI) has made great strides in these areas, and has moved on from abstract\nhigh-level rule-based paradigms to embodied architectures whose operations are\ngrounded in real physical environments. What is still missing, however, is an\noverarching theory of intelligent communicative behaviour that informs\nsystem-level design decisions in order to provide a more coherent approach to\nsystem integration. This chapter introduces the beginnings of such a framework\ninspired by the principles of Perceptual Control Theory (PCT). In particular,\nit is observed that PCT has hitherto tended to view perceptual processes as a\nrelatively straightforward series of transformations from sensation to\nperception, and has overlooked the potential of powerful generative model-based\nsolutions that have emerged in practical fields such as visual or auditory\nscene analysis. Starting from first principles, a sequence of arguments is\npresented which not only shows how these ideas might be integrated into PCT,\nbut which also extend PCT towards a remarkably symmetric architecture for a\nneeds-driven communicative agent. It is concluded that, if behaviour is the\ncontrol of perception, then perception is the simulation of behaviour.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.05527v1",
            "date": "2016-11-17 01:43:01+00:00",
            "title": "Automatic Node Selection for Deep Neural Networks using Group Lasso Regularization",
            "authors": [
                "Tsubasa Ochiai",
                "Shigeki Matsuda",
                "Hideyuki Watanabe",
                "Shigeru Katagiri"
            ],
            "abstract": "We examine the effect of the Group Lasso (gLasso) regularizer in selecting\nthe salient nodes of Deep Neural Network (DNN) hidden layers by applying a\nDNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of\ngLasso regularization, one for outgoing weight vectors and another for incoming\nweight vectors, as well as two sizes of DNNs: 2048 hidden layer nodes and 4096\nnodes. Furthermore, we compare gLasso and L2 regularizers. Our experiment\nresults demonstrate that our DNN training, in which the gLasso regularizer was\nembedded, successfully selected the hidden layer nodes that are necessary and\nsufficient for achieving high classification power.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.05546v2",
            "date": "2016-11-17 03:21:00+00:00",
            "title": "Zero-Shot Visual Question Answering",
            "authors": [
                "Damien Teney",
                "Anton van den Hengel"
            ],
            "abstract": "Part of the appeal of Visual Question Answering (VQA) is its promise to\nanswer new questions about previously unseen images. Most current methods\ndemand training questions that illustrate every possible concept, and will\ntherefore never achieve this capability, since the volume of required training\ndata would be prohibitive. Answering general questions about images requires\nmethods capable of Zero-Shot VQA, that is, methods able to answer questions\nbeyond the scope of the training questions. We propose a new evaluation\nprotocol for VQA methods which measures their ability to perform Zero-Shot VQA,\nand in doing so highlights significant practical deficiencies of current\napproaches, some of which are masked by the biases in current datasets. We\npropose and evaluate several strategies for achieving Zero-Shot VQA, including\nmethods based on pretrained word embeddings, object classifiers with semantic\nembeddings, and test-time retrieval of example images. Our extensive\nexperiments are intended to serve as baselines for Zero-Shot VQA, and they also\nachieve state-of-the-art performance in the standard VQA evaluation setting.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.06188v2",
            "date": "2016-11-18 18:13:46+00:00",
            "title": "Variable Computation in Recurrent Neural Networks",
            "authors": [
                "Yacine Jernite",
                "Edouard Grave",
                "Armand Joulin",
                "Tomas Mikolov"
            ],
            "abstract": "Recurrent neural networks (RNNs) have been used extensively and with\nincreasing success to model various types of sequential data. Much of this\nprogress has been achieved through devising recurrent units and architectures\nwith the flexibility to capture complex statistics in the data, such as long\nrange dependency or localized attention phenomena. However, while many\nsequential data (such as video, speech or language) can have highly variable\ninformation flow, most recurrent models still consume input features at a\nconstant rate and perform a constant number of computations per time step,\nwhich can be detrimental to both speed and model capacity. In this paper, we\nexplore a modification to existing recurrent units which allows them to learn\nto vary the amount of computation they perform at each step, without prior\nknowledge of the sequence's time structure. We show experimentally that not\nonly do our models require fewer operations, they also lead to better\nperformance overall on evaluation tasks.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1611.06204v1",
            "date": "2016-11-18 19:38:59+00:00",
            "title": "Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks",
            "authors": [
                "Volkan Cirik",
                "Eduard Hovy",
                "Louis-Philippe Morency"
            ],
            "abstract": "Curriculum Learning emphasizes the order of training instances in a\ncomputational learning setup. The core hypothesis is that simpler instances\nshould be learned early as building blocks to learn more complex ones. Despite\nits usefulness, it is still unknown how exactly the internal representation of\nmodels are affected by curriculum learning. In this paper, we study the effect\nof curriculum learning on Long Short-Term Memory (LSTM) networks, which have\nshown strong competency in many Natural Language Processing (NLP) problems. Our\nexperiments on sentiment analysis task and a synthetic task similar to sequence\nprediction tasks in NLP show that curriculum learning has a positive effect on\nthe LSTM's internal states by biasing the model towards building constructive\nrepresentations i.e. the internal representation at the previous timesteps are\nused as building blocks for the final prediction. We also find that smaller\nmodels significantly improves when they are trained with curriculum learning.\nLastly, we show that curriculum learning helps more when the amount of training\ndata is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06216v1",
            "date": "2016-11-18 20:11:51+00:00",
            "title": "Generative Deep Neural Networks for Dialogue: A Short Review",
            "authors": [
                "Iulian Vlad Serban",
                "Ryan Lowe",
                "Laurent Charlin",
                "Joelle Pineau"
            ],
            "abstract": "Researchers have recently started investigating deep neural networks for\ndialogue applications. In particular, generative sequence-to-sequence (Seq2Seq)\nmodels have shown promising results for unstructured tasks, such as word-level\ndialogue response generation. The hope is that such models will be able to\nleverage massive amounts of data to learn meaningful natural language\nrepresentations and response generation strategies, while requiring a minimum\namount of domain knowledge and hand-crafting. An important challenge is to\ndevelop models that can effectively incorporate dialogue context and generate\nmeaningful and diverse responses. In support of this goal, we review recently\nproposed models based on generative encoder-decoder neural network\narchitectures, and show that these models have better ability to incorporate\nlong-term dialogue history, to model uncertainty and ambiguity in dialogue, and\nto generate responses with high-level compositional structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.06322v1",
            "date": "2016-11-19 07:23:10+00:00",
            "title": "Spotting Rumors via Novelty Detection",
            "authors": [
                "Yumeng Qin",
                "Dominik Wurzer",
                "Victor Lavrenko",
                "Cunchen Tang"
            ],
            "abstract": "Rumour detection is hard because the most accurate systems operate\nretrospectively, only recognizing rumours once they have collected repeated\nsignals. By then the rumours might have already spread and caused harm. We\nintroduce a new category of features based on novelty, tailored to detect\nrumours early on. To compensate for the absence of repeated signals, we make\nuse of news wire as an additional data source. Unconfirmed (novel) information\nwith respect to the news articles is considered as an indication of rumours.\nAdditionally we introduce pseudo feedback, which assumes that documents that\nare similar to previous rumours, are more likely to also be a rumour.\nComparison with other real-time approaches shows that novelty based features in\nconjunction with pseudo feedback perform significantly better, when detecting\nrumours instantly after their publication.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1611.06459v2",
            "date": "2016-11-20 01:21:31+00:00",
            "title": "Gendered Conversation in a Social Game-Streaming Platform",
            "authors": [
                "Supun Nakandala",
                "Giovanni Luca Ciampaglia",
                "Norman Makoto Su",
                "Yong-Yeol Ahn"
            ],
            "abstract": "Online social media and games are increasingly replacing offline social\nactivities. Social media is now an indispensable mode of communication; online\ngaming is not only a genuine social activity but also a popular spectator\nsport. With support for anonymity and larger audiences, online interaction\nshrinks social and geographical barriers. Despite such benefits, social\ndisparities such as gender inequality persist in online social media. In\nparticular, online gaming communities have been criticized for persistent\ngender disparities and objectification. As gaming evolves into a social\nplatform, persistence of gender disparity is a pressing question. Yet, there\nare few large-scale, systematic studies of gender inequality and\nobjectification in social gaming platforms. Here we analyze more than one\nbillion chat messages from Twitch, a social game-streaming platform, to study\nhow the gender of streamers is associated with the nature of conversation.\nUsing a combination of computational text analysis methods, we show that\ngendered conversation and objectification is prevalent in chats. Female\nstreamers receive significantly more objectifying comments while male streamers\nreceive more game-related comments. This difference is more pronounced for\npopular streamers. There also exists a large number of users who post only on\nfemale or male streams. Employing a neural vector-space embedding (paragraph\nvector) method, we analyze gendered chat messages and create prediction models\nthat (i) identify the gender of streamers based on messages posted in the\nchannel and (ii) identify the gender a viewer prefers to watch based on their\nchat messages. Our findings suggest that disparities in social game-streaming\nplatforms is a nuanced phenomenon that involves the gender of streamers as well\nas those who produce gendered and game-related conversation.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1611.06468v1",
            "date": "2016-11-20 04:06:47+00:00",
            "title": "Generating machine-executable plans from end-user's natural-language instructions",
            "authors": [
                "Rui Liu",
                "Xiaoli Zhang"
            ],
            "abstract": "It is critical for advanced manufacturing machines to autonomously execute a\ntask by following an end-user's natural language (NL) instructions. However, NL\ninstructions are usually ambiguous and abstract so that the machines may\nmisunderstand and incorrectly execute the task. To address this NL-based\nhuman-machine communication problem and enable the machines to appropriately\nexecute tasks by following the end-user's NL instructions, we developed a\nMachine-Executable-Plan-Generation (exePlan) method. The exePlan method\nconducts task-centered semantic analysis to extract task-related information\nfrom ambiguous NL instructions. In addition, the method specifies machine\nexecution parameters to generate a machine-executable plan by interpreting\nabstract NL instructions. To evaluate the exePlan method, an industrial robot\nBaxter was instructed by NL to perform three types of industrial tasks {'drill\na hole', 'clean a spot', 'install a screw'}. The experiment results proved that\nthe exePlan method was effective in generating machine-executable plans from\nthe end-user's NL instructions. Such a method has the promise to endow a\nmachine with the ability of NL-instructed task execution.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.06933v1",
            "date": "2016-11-21 18:30:17+00:00",
            "title": "Unsupervised Learning for Lexicon-Based Classification",
            "authors": [
                "Jacob Eisenstein"
            ],
            "abstract": "In lexicon-based classification, documents are assigned labels by comparing\nthe number of words that appear from two opposed lexicons, such as positive and\nnegative sentiment. Creating such words lists is often easier than labeling\ninstances, and they can be debugged by non-experts if classification\nperformance is unsatisfactory. However, there is little analysis or\njustification of this classification heuristic. This paper describes a set of\nassumptions that can be used to derive a probabilistic justification for\nlexicon-based classification, as well as an analysis of its expected accuracy.\nOne key assumption behind lexicon-based classification is that all words in\neach lexicon are equally predictive. This is rarely true in practice, which is\nwhy lexicon-based approaches are usually outperformed by supervised classifiers\nthat learn distinct weights on each word from labeled instances. This paper\nshows that it is possible to learn such weights without labeled data, by\nleveraging co-occurrence statistics across the lexicons. This offers the best\nof both worlds: light supervision in the form of lexicons, and data-driven\nclassification with higher accuracy than traditional word-counting heuristics.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.06950v1",
            "date": "2016-11-21 19:00:32+00:00",
            "title": "Statistical Learning for OCR Text Correction",
            "authors": [
                "Jie Mei",
                "Aminul Islam",
                "Yajing Wu",
                "Abidalrahman Moh'd",
                "Evangelos E. Milios"
            ],
            "abstract": "The accuracy of Optical Character Recognition (OCR) is crucial to the success\nof subsequent applications used in text analyzing pipeline. Recent models of\nOCR post-processing significantly improve the quality of OCR-generated text,\nbut are still prone to suggest correction candidates from limited observations\nwhile insufficiently accounting for the characteristics of OCR errors. In this\npaper, we show how to enlarge candidate suggestion space by using external\ncorpus and integrating OCR-specific features in a regression approach to\ncorrect OCR-generated errors. The evaluation results show that our model can\ncorrect 61.5% of the OCR-errors (considering the top 1 suggestion) and 71.5% of\nthe OCR-errors (considering the top 3 suggestions), for cases where the\ntheoretical correction upper-bound is 78%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.06986v1",
            "date": "2016-11-21 20:08:51+00:00",
            "title": "Robust end-to-end deep audiovisual speech recognition",
            "authors": [
                "Ramon Sanabria",
                "Florian Metze",
                "Fernando De La Torre"
            ],
            "abstract": "Speech is one of the most effective ways of communication among humans. Even\nthough audio is the most common way of transmitting speech, very important\ninformation can be found in other modalities, such as vision. Vision is\nparticularly useful when the acoustic signal is corrupted. Multi-modal speech\nrecognition however has not yet found wide-spread use, mostly because the\ntemporal alignment and fusion of the different information sources is\nchallenging.\n  This paper presents an end-to-end audiovisual speech recognizer (AVSR), based\non recurrent neural networks (RNN) with a connectionist temporal classification\n(CTC) loss function. CTC creates sparse \"peaky\" output activations, and we\nanalyze the differences in the alignments of output targets (phonemes or\nvisemes) between audio-only, video-only, and audio-visual feature\nrepresentations. We present the first such experiments on the large vocabulary\nIBM ViaVoice database, which outperform previously published approaches on\nphone accuracy in clean and noisy conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.07139v1",
            "date": "2016-11-22 03:42:44+00:00",
            "title": "A Natural Language Query Interface for Searching Personal Information on Smartwatches",
            "authors": [
                "Reza Rawassizadeh",
                "Chelsea Dobbins",
                "Manouchehr Nourizadeh",
                "Zahra Ghamchili",
                "Michael Pazzani"
            ],
            "abstract": "Currently, personal assistant systems, run on smartphones and use natural\nlanguage interfaces. However, these systems rely mostly on the web for finding\ninformation. Mobile and wearable devices can collect an enormous amount of\ncontextual personal data such as sleep and physical activities. These\ninformation objects and their applications are known as quantified-self, mobile\nhealth or personal informatics, and they can be used to provide a deeper\ninsight into our behavior. To our knowledge, existing personal assistant\nsystems do not support all types of quantified-self queries. In response to\nthis, we have undertaken a user study to analyze a set of \"textual\nquestions/queries\" that users have used to search their quantified-self or\nmobile health data. Through analyzing these questions, we have constructed a\nlight-weight natural language based query interface, including a text parser\nalgorithm and a user interface, to process the users' queries that have been\nused for searching quantified-self information. This query interface has been\ndesigned to operate on small devices, i.e. smartwatches, as well as augmenting\nthe personal assistant systems by allowing them to process end users' natural\nlanguage queries about their quantified-self data.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1611.08002v2",
            "date": "2016-11-23 21:22:22+00:00",
            "title": "Semantic Compositional Networks for Visual Captioning",
            "authors": [
                "Zhe Gan",
                "Chuang Gan",
                "Xiaodong He",
                "Yunchen Pu",
                "Kenneth Tran",
                "Jianfeng Gao",
                "Lawrence Carin",
                "Li Deng"
            ],
            "abstract": "A Semantic Compositional Network (SCN) is developed for image captioning, in\nwhich semantic concepts (i.e., tags) are detected from the image, and the\nprobability of each tag is used to compose the parameters in a long short-term\nmemory (LSTM) network. The SCN extends each weight matrix of the LSTM to an\nensemble of tag-dependent weight matrices. The degree to which each member of\nthe ensemble is used to generate an image caption is tied to the\nimage-dependent probability of the corresponding tag. In addition to captioning\nimages, we also extend the SCN to generate captions for video clips. We\nqualitatively analyze semantic composition in SCNs, and quantitatively evaluate\nthe algorithm on three benchmark datasets: COCO, Flickr30k, and Youtube2Text.\nExperimental results show that the proposed method significantly outperforms\nprior state-of-the-art approaches, across multiple evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.08307v1",
            "date": "2016-11-24 21:01:46+00:00",
            "title": "Learning Python Code Suggestion with a Sparse Pointer Network",
            "authors": [
                "Avishkar Bhoopchand",
                "Tim Rockt\u00e4schel",
                "Earl Barr",
                "Sebastian Riedel"
            ],
            "abstract": "To enhance developer productivity, all modern integrated development\nenvironments (IDEs) include code suggestion functionality that proposes likely\nnext tokens at the cursor. While current IDEs work well for statically-typed\nlanguages, their reliance on type annotations means that they do not provide\nthe same level of support for dynamic programming languages as for\nstatically-typed languages. Moreover, suggestion engines in modern IDEs do not\npropose expressions or multi-statement idiomatic code. Recent work has shown\nthat language models can improve code suggestion systems by learning from\nsoftware repositories. This paper introduces a neural language model with a\nsparse pointer network aimed at capturing very long-range dependencies. We\nrelease a large-scale code suggestion corpus of 41M lines of Python code\ncrawled from GitHub. On this corpus, we found standard neural language models\nto perform well at suggesting local phenomena, but struggle to refer to\nidentifiers that are introduced many tokens in the past. By augmenting a neural\nlanguage model with a pointer network specialized in referring to predefined\nclasses of identifiers, we obtain a much lower perplexity and a 5 percentage\npoints increase in accuracy for code suggestion compared to an LSTM baseline.\nIn fact, this increase in code suggestion accuracy is due to a 13 times more\naccurate prediction of identifiers. Furthermore, a qualitative analysis shows\nthis model indeed captures interesting long-range dependencies, like referring\nto a class member defined over 60 tokens in the past.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1611.08321v1",
            "date": "2016-11-24 23:15:56+00:00",
            "title": "Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images",
            "authors": [
                "Junhua Mao",
                "Jiajing Xu",
                "Yushi Jing",
                "Alan Yuille"
            ],
            "abstract": "In this paper, we focus on training and evaluating effective word embeddings\nwith both text and visual information. More specifically, we introduce a\nlarge-scale dataset with 300 million sentences describing over 40 million\nimages crawled and downloaded from publicly available Pins (i.e. an image with\nsentence descriptions uploaded by users) on Pinterest. This dataset is more\nthan 200 times larger than MS COCO, the standard large-scale image dataset with\nsentence descriptions. In addition, we construct an evaluation dataset to\ndirectly assess the effectiveness of word embeddings in terms of finding\nsemantically similar or related words and phrases. The word/phrase pairs in\nthis evaluation dataset are collected from the click data with millions of\nusers in an image search system, thus contain rich semantic relationships.\nBased on these datasets, we propose and compare several Recurrent Neural\nNetworks (RNNs) based multimodal (text and image) models. Experiments show that\nour model benefits from incorporating the visual information into the word\nembeddings, and a weight sharing strategy is crucial for learning such\nmultimodal embeddings. The project page is:\nhttp://www.stat.ucla.edu/~junhua.mao/multimodal_embedding.html",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.08373v1",
            "date": "2016-11-25 08:11:23+00:00",
            "title": "Bidirectional LSTM-CRF for Clinical Concept Extraction",
            "authors": [
                "Raghavendra Chalapathy",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "Automated extraction of concepts from patient clinical records is an\nessential facilitator of clinical research. For this reason, the 2010 i2b2/VA\nNatural Language Processing Challenges for Clinical Records introduced a\nconcept extraction task aimed at identifying and classifying concepts into\npredefined categories (i.e., treatments, tests and problems). State-of-the-art\nconcept extraction approaches heavily rely on handcrafted features and\ndomain-specific resources which are hard to collect and define. For this\nreason, this paper proposes an alternative, streamlined approach: a recurrent\nneural network (the bidirectional LSTM with CRF decoding) initialized with\ngeneral-purpose, off-the-shelf word embeddings. The experimental results\nachieved on the 2010 i2b2/VA reference corpora using the proposed framework\noutperform all recent methods and ranks closely to the best submission from the\noriginal 2010 i2b2/VA challenge.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1611.08669v5",
            "date": "2016-11-26 06:39:28+00:00",
            "title": "Visual Dialog",
            "authors": [
                "Abhishek Das",
                "Satwik Kottur",
                "Khushi Gupta",
                "Avi Singh",
                "Deshraj Yadav",
                "Jos\u00e9 M. F. Moura",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We introduce the task of Visual Dialog, which requires an AI agent to hold a\nmeaningful dialog with humans in natural, conversational language about visual\ncontent. Specifically, given an image, a dialog history, and a question about\nthe image, the agent has to ground the question in image, infer context from\nhistory, and answer the question accurately. Visual Dialog is disentangled\nenough from a specific downstream task so as to serve as a general test of\nmachine intelligence, while being grounded in vision enough to allow objective\nevaluation of individual responses and benchmark progress. We develop a novel\ntwo-person chat data-collection protocol to curate a large-scale Visual Dialog\ndataset (VisDial). VisDial v0.9 has been released and contains 1 dialog with 10\nquestion-answer pairs on ~120k images from COCO, with a total of ~1.2M dialog\nquestion-answer pairs.\n  We introduce a family of neural encoder-decoder models for Visual Dialog with\n3 encoders -- Late Fusion, Hierarchical Recurrent Encoder and Memory Network --\nand 2 decoders (generative and discriminative), which outperform a number of\nsophisticated baselines. We propose a retrieval-based evaluation protocol for\nVisual Dialog where the AI agent is asked to sort a set of candidate answers\nand evaluated on metrics such as mean-reciprocal-rank of human response. We\nquantify gap between machine and human performance on the Visual Dialog task\nvia human studies. Putting it all together, we demonstrate the first 'visual\nchatbot'! Our dataset, code, trained models and visual chatbot are available on\nhttps://visualdialog.org",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1611.08675v1",
            "date": "2016-11-26 07:53:22+00:00",
            "title": "Deep Reinforcement Learning for Multi-Domain Dialogue Systems",
            "authors": [
                "Heriberto Cuay\u00e1huitl",
                "Seunghak Yu",
                "Ashley Williamson",
                "Jacob Carse"
            ],
            "abstract": "Standard deep reinforcement learning methods such as Deep Q-Networks (DQN)\nfor multiple tasks (domains) face scalability problems. We propose a method for\nmulti-domain dialogue policy learning---termed NDQN, and apply it to an\ninformation-seeking spoken dialogue system in the domains of restaurants and\nhotels. Experimental results comparing DQN (baseline) versus NDQN (proposed)\nusing simulations report that our proposed method exhibits better scalability\nand is promising for optimising the behaviour of multi-domain dialogue systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.08737v1",
            "date": "2016-11-26 20:11:00+00:00",
            "title": "Structural Correspondence Learning for Cross-lingual Sentiment Classification with One-to-many Mappings",
            "authors": [
                "Nana Li",
                "Shuangfei Zhai",
                "Zhongfei Zhang",
                "Boying Liu"
            ],
            "abstract": "Structural correspondence learning (SCL) is an effective method for\ncross-lingual sentiment classification. This approach uses unlabeled documents\nalong with a word translation oracle to automatically induce task specific,\ncross-lingual correspondences. It transfers knowledge through identifying\nimportant features, i.e., pivot features. For simplicity, however, it assumes\nthat the word translation oracle maps each pivot feature in source language to\nexactly only one word in target language. This one-to-one mapping between words\nin different languages is too strict. Also the context is not considered at\nall. In this paper, we propose a cross-lingual SCL based on distributed\nrepresentation of words; it can learn meaningful one-to-many mappings for pivot\nwords using large amounts of monolingual data and a small dictionary. We\nconduct experiments on NLP\\&CC 2013 cross-lingual sentiment analysis dataset,\nemploying English as source language, and Chinese as target language. Our\nmethod does not rely on the parallel corpora and the experimental results show\nthat our approach is more competitive than the state-of-the-art methods in\ncross-lingual sentiment classification.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.08945v4",
            "date": "2016-11-28 00:54:34+00:00",
            "title": "Learning a Natural Language Interface with Neural Programmer",
            "authors": [
                "Arvind Neelakantan",
                "Quoc V. Le",
                "Martin Abadi",
                "Andrew McCallum",
                "Dario Amodei"
            ],
            "abstract": "Learning a natural language interface for database tables is a challenging\ntask that involves deep language understanding and multi-step reasoning. The\ntask is often approached by mapping natural language queries to logical forms\nor programs that provide the desired response when executed on the database. To\nour knowledge, this paper presents the first weakly supervised, end-to-end\nneural network model to induce such programs on a real-world dataset. We\nenhance the objective function of Neural Programmer, a neural network with\nbuilt-in discrete operations, and apply it on WikiTableQuestions, a natural\nlanguage question-answering dataset. The model is trained end-to-end with weak\nsupervision of question-answer pairs, and does not require domain-specific\ngrammars, rules, or annotations that are key elements in previous approaches to\nprogram induction. The main experimental result in this paper is that a single\nNeural Programmer model achieves 34.2% accuracy using only 10,000 examples with\nweak supervision. An ensemble of 15 models, with a trivial combination\ntechnique, achieves 37.7% accuracy, which is competitive to the current\nstate-of-the-art accuracy of 37.1% obtained by a traditional natural language\nsemantic parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09028v1",
            "date": "2016-11-28 08:56:04+00:00",
            "title": "Analyzing Features for the Detection of Happy Endings in German Novels",
            "authors": [
                "Fotis Jannidis",
                "Isabella Reger",
                "Albin Zehe",
                "Martin Becker",
                "Lena Hettinger",
                "Andreas Hotho"
            ],
            "abstract": "With regard to a computational representation of literary plot, this paper\nlooks at the use of sentiment analysis for happy ending detection in German\nnovels. Its focus lies on the investigation of previously proposed sentiment\nfeatures in order to gain insight about the relevance of specific features on\nthe one hand and the implications of their performance on the other hand.\nTherefore, we study various partitionings of novels, considering the highly\nvariable concept of \"ending\". We also show that our approach, even though still\nrather simple, can potentially lead to substantial findings relevant to\nliterary studies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.09207v1",
            "date": "2016-11-28 15:51:25+00:00",
            "title": "AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech",
            "authors": [
                "Brian Patton",
                "Yannis Agiomyrgiannakis",
                "Michael Terry",
                "Kevin Wilson",
                "Rif A. Saurous",
                "D. Sculley"
            ],
            "abstract": "Developers of text-to-speech synthesizers (TTS) often make use of human\nraters to assess the quality of synthesized speech. We demonstrate that we can\nmodel human raters' mean opinion scores (MOS) of synthesized speech using a\ndeep recurrent neural network whose inputs consist solely of a raw waveform.\nOur best models provide utterance-level estimates of MOS only moderately\ninferior to sampled human ratings, as shown by Pearson and Spearman\ncorrelations. When multiple utterances are scored and averaged, a scenario\ncommon in synthesizer quality assessment, AutoMOS achieves correlations\napproaching those of human raters. The AutoMOS model has a number of\napplications, such as the ability to explore the parameter space of a speech\nsynthesizer without requiring a human-in-the-loop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09288v2",
            "date": "2016-11-28 19:09:58+00:00",
            "title": "Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition",
            "authors": [
                "Tom Sercu",
                "Vaibhava Goel"
            ],
            "abstract": "In computer vision pixelwise dense prediction is the task of predicting a\nlabel for each pixel in the image. Convolutional neural networks achieve good\nperformance on this task, while being computationally efficient. In this paper\nwe carry these ideas over to the problem of assigning a sequence of labels to a\nset of speech frames, a task commonly known as framewise classification. We\nshow that dense prediction view of framewise classification offers several\nadvantages and insights, including computational efficiency and the ability to\napply batch normalization. When doing dense prediction we pay specific\nattention to strided pooling in time and introduce an asymmetric dilated\nconvolution, called time-dilated convolution, that allows for efficient and\nelegant implementation of pooling in time. We show results using time-dilated\nconvolutions in a very deep VGG-style CNN with batch normalization on the Hub5\nSwitchboard-2000 benchmark task. With a big n-gram language model, we achieve\n7.7% WER which is the best single model single-pass performance reported so\nfar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09434v2",
            "date": "2016-11-28 23:48:41+00:00",
            "title": "Input Switched Affine Networks: An RNN Architecture Designed for Interpretability",
            "authors": [
                "Jakob N. Foerster",
                "Justin Gilmer",
                "Jan Chorowski",
                "Jascha Sohl-Dickstein",
                "David Sussillo"
            ],
            "abstract": "There exist many problem domains where the interpretability of neural network\nmodels is essential for deployment. Here we introduce a recurrent architecture\ncomposed of input-switched affine transformations - in other words an RNN\nwithout any explicit nonlinearities, but with input-dependent recurrent\nweights. This simple form allows the RNN to be analyzed via straightforward\nlinear methods: we can exactly characterize the linear contribution of each\ninput to the model predictions; we can use a change-of-basis to disentangle\ninput, output, and computational hidden unit subspaces; we can fully\nreverse-engineer the architecture's solution to a simple task. Despite this\nease of interpretation, the input switched affine network achieves reasonable\nperformance on a text modeling tasks, and allows greater computational\nefficiency than networks with standard nonlinearities.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.09573v1",
            "date": "2016-11-29 11:28:59+00:00",
            "title": "Learning Concept Hierarchies through Probabilistic Topic Modeling",
            "authors": [
                "V. S. Anoop",
                "S. Asharaf",
                "P. Deepak"
            ],
            "abstract": "With the advent of semantic web, various tools and techniques have been\nintroduced for presenting and organizing knowledge. Concept hierarchies are one\nsuch technique which gained significant attention due to its usefulness in\ncreating domain ontologies that are considered as an integral part of semantic\nweb. Automated concept hierarchy learning algorithms focus on extracting\nrelevant concepts from unstructured text corpus and connect them together by\nidentifying some potential relations exist between them. In this paper, we\npropose a novel approach for identifying relevant concepts from plain text and\nthen learns hierarchy of concepts by exploiting subsumption relation between\nthem. To start with, we model topics using a probabilistic topic model and then\nmake use of some lightweight linguistic process to extract semantically rich\nconcepts. Then we connect concepts by identifying an \"is-a\" relationship\nbetween pair of concepts. The proposed method is completely unsupervised and\nthere is no need for a domain specific training corpus for concept extraction\nand learning. Experiments on large and real-world text corpora such as BBC News\ndataset and Reuters News corpus shows that the proposed method outperforms some\nof the existing methods for concept extraction and efficient concept hierarchy\nlearning is possible if the overall task is guided by a probabilistic topic\nmodeling algorithm.",
            "journal ref": "International Journal of Information Processing (IJIP), Volume 10,\n  Issue 3, 2016",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1611.09878v1",
            "date": "2016-11-29 21:12:04+00:00",
            "title": "Identity-sensitive Word Embedding through Heterogeneous Networks",
            "authors": [
                "Jian Tang",
                "Meng Qu",
                "Qiaozhu Mei"
            ],
            "abstract": "Most existing word embedding approaches do not distinguish the same words in\ndifferent contexts, therefore ignoring their contextual meanings. As a result,\nthe learned embeddings of these words are usually a mixture of multiple\nmeanings. In this paper, we acknowledge multiple identities of the same word in\ndifferent contexts and learn the \\textbf{identity-sensitive} word embeddings.\nBased on an identity-labeled text corpora, a heterogeneous network of words and\nword identities is constructed to model different-levels of word\nco-occurrences. The heterogeneous network is further embedded into a\nlow-dimensional space through a principled network embedding approach, through\nwhich we are able to obtain the embeddings of words and the embeddings of word\nidentities. We study three different types of word identities including topics,\nsentiments and categories. Experimental results on real-world data sets show\nthat the identity-sensitive word embeddings learned by our approach indeed\ncapture different meanings of words and outperforms competitive methods on\ntasks including text classification and word similarity computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.09921v2",
            "date": "2016-11-29 22:24:30+00:00",
            "title": "Less is More: Learning Prominent and Diverse Topics for Data Summarization",
            "authors": [
                "Jian Tang",
                "Cheng Li",
                "Ming Zhang",
                "Qiaozhu Mei"
            ],
            "abstract": "Statistical topic models efficiently facilitate the exploration of\nlarge-scale data sets. Many models have been developed and broadly used to\nsummarize the semantic structure in news, science, social media, and digital\nhumanities. However, a common and practical objective in data exploration tasks\nis not to enumerate all existing topics, but to quickly extract representative\nones that broadly cover the content of the corpus, i.e., a few topics that\nserve as a good summary of the data. Most existing topic models fit exactly the\nsame number of topics as a user specifies, which have imposed an unnecessary\nburden to the users who have limited prior knowledge. We instead propose new\nmodels that are able to learn fewer but more representative topics for the\npurpose of data summarization. We propose a reinforced random walk that allows\nprominent topics to absorb tokens from similar and smaller topics, thus\nenhances the diversity among the top topics extracted. With this reinforced\nrandom walk as a general process embedded in classical topic models, we obtain\n\\textit{diverse topic models} that are able to extract the most prominent and\ndiverse topics from data. The inference procedures of these diverse topic\nmodels remain as simple and efficient as the classical models. Experimental\nresults demonstrate that the diverse topic models not only discover topics that\nbetter summarize the data, but also require minimal prior knowledge of the\nusers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1607.00167v2",
            "date": "2016-07-01 09:15:13+00:00",
            "title": "SentiBubbles: Topic Modeling and Sentiment Visualization of Entity-centric Tweets",
            "authors": [
                "Jo\u00e3o Oliveira",
                "Mike Pinto",
                "Pedro Saleiro",
                "Jorge Teixeira"
            ],
            "abstract": "Social Media users tend to mention entities when reacting to news events. The\nmain purpose of this work is to create entity-centric aggregations of tweets on\na daily basis. By applying topic modeling and sentiment analysis, we create\ndata visualization insights about current events and people reactions to those\nevents from an entity-centric perspective.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1607.00325v2",
            "date": "2016-07-01 17:34:16+00:00",
            "title": "Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation",
            "authors": [
                "Dong Yu",
                "Morten Kolb\u00e6k",
                "Zheng-Hua Tan",
                "Jesper Jensen"
            ],
            "abstract": "We propose a novel deep learning model, which supports permutation invariant\ntraining (PIT), for speaker independent multi-talker speech separation,\ncommonly known as the cocktail-party problem. Different from most of the prior\narts that treat speech separation as a multi-class regression problem and the\ndeep clustering technique that considers it a segmentation (or clustering)\nproblem, our model optimizes for the separation regression error, ignoring the\norder of mixing sources. This strategy cleverly solves the long-lasting label\npermutation problem that has prevented progress on deep learning based\ntechniques for speech separation. Experiments on the equal-energy mixing setup\nof a Danish corpus confirms the effectiveness of PIT. We believe improvements\nbuilt upon PIT can eventually solve the cocktail-party problem and enable\nreal-world adoption of, e.g., automatic meeting transcription and multi-party\nhuman-computer interaction, where overlapping speech is common.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00410v1",
            "date": "2016-07-01 21:24:21+00:00",
            "title": "Domain Adaptation for Neural Networks by Parameter Augmentation",
            "authors": [
                "Yusuke Watanabe",
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "We propose a simple domain adaptation method for neural networks in a\nsupervised setting. Supervised domain adaptation is a way of improving the\ngeneralization performance on the target domain by using the source domain\ndataset, assuming that both of the datasets are labeled. Recently, recurrent\nneural networks have been shown to be successful on a variety of NLP tasks such\nas caption generation; however, the existing domain adaptation techniques are\nlimited to (1) tune the model parameters by the target dataset after the\ntraining by the source dataset, or (2) design the network to have dual output,\none for the source domain and the other for the target domain. Reformulating\nthe idea of the domain adaptation technique proposed by Daume (2007), we\npropose a simple domain adaptation method, which can be applied to neural\nnetworks trained with a cross-entropy loss. On captioning datasets, we show\nperformance improvements over other domain adaptation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00424v1",
            "date": "2016-07-01 22:11:38+00:00",
            "title": "Learning Relational Dependency Networks for Relation Extraction",
            "authors": [
                "Dileep Viswanathan",
                "Ameet Soni",
                "Jude Shavlik",
                "Sriraam Natarajan"
            ],
            "abstract": "We consider the task of KBP slot filling -- extracting relation information\nfrom newswire documents for knowledge base construction. We present our\npipeline, which employs Relational Dependency Networks (RDNs) to learn\nlinguistic patterns for relation extraction. Additionally, we demonstrate how\nseveral components such as weak supervision, word2vec features, joint learning\nand the use of human advice, can be incorporated in this relational framework.\nWe evaluate the different components in the benchmark KBP 2015 task and show\nthat RDNs effectively model a diverse set of features and perform competitively\nwith current state-of-the-art relation extraction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.01274v1",
            "date": "2016-07-04 01:16:55+00:00",
            "title": "Temporal Topic Analysis with Endogenous and Exogenous Processes",
            "authors": [
                "Baiyang Wang",
                "Diego Klabjan"
            ],
            "abstract": "We consider the problem of modeling temporal textual data taking endogenous\nand exogenous processes into account. Such text documents arise in real world\napplications, including job advertisements and economic news articles, which\nare influenced by the fluctuations of the general economy. We propose a\nhierarchical Bayesian topic model which imposes a \"group-correlated\"\nhierarchical structure on the evolution of topics over time incorporating both\nprocesses, and show that this model can be estimated from Markov chain Monte\nCarlo sampling methods. We further demonstrate that this model captures the\nintrinsic relationships between the topic distribution and the time-dependent\nfactors, and compare its performance with latent Dirichlet allocation (LDA) and\ntwo other related models. The model is applied to two collections of documents\nto illustrate its empirical performance: online job advertisements from\nDirectEmployers Association and journalists' postings on BusinessInsider.com.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01869v1",
            "date": "2016-07-07 03:43:12+00:00",
            "title": "Scalable Semantic Matching of Queries to Ads in Sponsored Search Advertising",
            "authors": [
                "Mihajlo Grbovic",
                "Nemanja Djuric",
                "Vladan Radosavljevic",
                "Fabrizio Silvestri",
                "Ricardo Baeza-Yates",
                "Andrew Feng",
                "Erik Ordentlich",
                "Lee Yang",
                "Gavin Owens"
            ],
            "abstract": "Sponsored search represents a major source of revenue for web search engines.\nThis popular advertising model brings a unique possibility for advertisers to\ntarget users' immediate intent communicated through a search query, usually by\ndisplaying their ads alongside organic search results for queries deemed\nrelevant to their products or services. However, due to a large number of\nunique queries it is challenging for advertisers to identify all such relevant\nqueries. For this reason search engines often provide a service of advanced\nmatching, which automatically finds additional relevant queries for advertisers\nto bid on. We present a novel advanced matching approach based on the idea of\nsemantic embeddings of queries and ads. The embeddings were learned using a\nlarge data set of user search sessions, consisting of search queries, clicked\nads and search links, while utilizing contextual information such as dwell time\nand skipped ads. To address the large-scale nature of our problem, both in\nterms of data and vocabulary size, we propose a novel distributed algorithm for\ntraining of the embeddings. Finally, we present an approach for overcoming a\ncold-start problem associated with new ads and queries. We report results of\neditorial evaluation and online tests on actual search traffic. The results\nshow that our approach significantly outperforms baselines in terms of\nrelevance, coverage, and incremental revenue. Lastly, we open-source learned\nquery embeddings to be used by researchers in computational advertising and\nrelated fields.",
            "journal ref": "39th International ACM SIGIR Conference on Research and\n  Development in Information Retrieval, SIGIR 2016, Pisa, Italy",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1607.01958v1",
            "date": "2016-07-07 10:48:34+00:00",
            "title": "Stock trend prediction using news sentiment analysis",
            "authors": [
                "Joshi Kalyani",
                "Prof. H. N. Bharathi",
                "Prof. Rao Jyothi"
            ],
            "abstract": "Efficient Market Hypothesis is the popular theory about stock prediction.\nWith its failure much research has been carried in the area of prediction of\nstocks. This project is about taking non quantifiable data such as financial\nnews articles about a company and predicting its future stock trend with news\nsentiment classification. Assuming that news articles have impact on stock\nmarket, this is an attempt to study relationship between news and stock trend.\nTo show this, we created three different classification models which depict\npolarity of news articles being positive or negative. Observations show that RF\nand SVM perform well in all types of testing. Na\\\"ive Bayes gives good result\nbut not compared to the other two. Experiments are conducted to evaluate\nvarious aspects of the proposed model and encouraging results are obtained in\nall of the experiments. The accuracy of the prediction model is more than 80%\nand in comparison with news random labeling with 50% of accuracy; the model has\nincreased the accuracy by 30%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.01963v5",
            "date": "2016-07-07 11:24:51+00:00",
            "title": "Sequence Training and Adaptation of Highway Deep Neural Networks",
            "authors": [
                "Liang Lu"
            ],
            "abstract": "Highway deep neural network (HDNN) is a type of depth-gated feedforward\nneural network, which has shown to be easier to train with more hidden layers\nand also generalise better compared to conventional plain deep neural networks\n(DNNs). Previously, we investigated a structured HDNN architecture for speech\nrecognition, in which the two gate functions were tied across all the hidden\nlayers, and we were able to train a much smaller model without sacrificing the\nrecognition accuracy. In this paper, we carry on the study of this architecture\nwith sequence-discriminative training criterion and speaker adaptation\ntechniques on the AMI meeting speech recognition corpus. We show that these two\ntechniques improve speech recognition accuracy on top of the model trained with\nthe cross entropy criterion. Furthermore, we demonstrate that the two gate\nfunctions that are tied across all the hidden layers are able to control the\ninformation flow over the whole network, and we can achieve considerable\nimprovements by only updating these gate functions in both sequence training\nand adaptation experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02109v2",
            "date": "2016-07-07 18:08:59+00:00",
            "title": "Predicting and Understanding Law-Making with Word Vectors and an Ensemble Model",
            "authors": [
                "John J. Nay"
            ],
            "abstract": "Out of nearly 70,000 bills introduced in the U.S. Congress from 2001 to 2015,\nonly 2,513 were enacted. We developed a machine learning approach to\nforecasting the probability that any bill will become law. Starting in 2001\nwith the 107th Congress, we trained models on data from previous Congresses,\npredicted all bills in the current Congress, and repeated until the 113th\nCongress served as the test. For prediction we scored each sentence of a bill\nwith a language model that embeds legislative vocabulary into a\nhigh-dimensional, semantic-laden vector space. This language representation\nenables our investigation into which words increase the probability of\nenactment for any topic. To test the relative importance of text and context,\nwe compared the text model to a context-only model that uses variables such as\nwhether the bill's sponsor is in the majority party. To test the effect of\nchanges to bills after their introduction on our ability to predict their final\noutcome, we compared using the bill text and meta-data available at the time of\nintroduction with using the most recent data. At the time of introduction\ncontext-only predictions outperform text-only, and with the newest data\ntext-only outperforms context-only. Combining text and context always performs\nbest. We conducted a global sensitivity analysis on the combined model to\ndetermine important variables predicting enactment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02436v1",
            "date": "2016-07-08 16:17:12+00:00",
            "title": "Document Clustering Games in Static and Dynamic Scenarios",
            "authors": [
                "Rocco Tripodi",
                "Marcello Pelillo"
            ],
            "abstract": "In this work we propose a game theoretic model for document clustering. Each\ndocument to be clustered is represented as a player and each cluster as a\nstrategy. The players receive a reward interacting with other players that they\ntry to maximize choosing their best strategies. The geometry of the data is\nmodeled with a weighted graph that encodes the pairwise similarity among\ndocuments, so that similar players are constrained to choose similar\nstrategies, updating their strategy preferences at each iteration of the games.\nWe used different approaches to find the prototypical elements of the clusters\nand with this information we divided the players into two disjoint sets, one\ncollecting players with a definite strategy and the other one collecting\nplayers that try to learn from others the correct strategy to play. The latter\nset of players can be considered as new data points that have to be clustered\naccording to previous information. This representation is useful in scenarios\nin which the data are streamed continuously. The evaluation of the system was\nconducted on 13 document datasets using different settings. It shows that the\nproposed method performs well compared to different document clustering\nalgorithms.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.02467v2",
            "date": "2016-07-08 17:35:51+00:00",
            "title": "Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge",
            "authors": [
                "Marc Dymetman",
                "Chunyang Xiao"
            ],
            "abstract": "We introduce LL-RNNs (Log-Linear RNNs), an extension of Recurrent Neural\nNetworks that replaces the softmax output layer by a log-linear output layer,\nof which the softmax is a special case. This conceptually simple move has two\nmain advantages. First, it allows the learner to combat training data sparsity\nby allowing it to model words (or more generally, output symbols) as complex\ncombinations of attributes without requiring that each combination is directly\nobserved in the training data (as the softmax does). Second, it permits the\ninclusion of flexible prior knowledge in the form of a priori specified modular\nfeatures, where the neural network component learns to dynamically control the\nweights of a log-linear distribution exploiting these features.\n  We conduct experiments in the domain of language modelling of French, that\nexploit morphological prior knowledge and show an important decrease in\nperplexity relative to a baseline RNN.\n  We provide other motivating iillustrations, and finally argue that the\nlog-linear and the neural-network components contribute complementary strengths\nto the LL-RNN: the LL aspect allows the model to incorporate rich prior\nknowledge, while the NN aspect, according to the \"representation learning\"\nparadigm, allows the model to discover novel combination of characteristics.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.02576v2",
            "date": "2016-07-09 07:11:43+00:00",
            "title": "Analysis of opinionated text for opinion mining",
            "authors": [
                "K Paramesha",
                "K C Ravishankar"
            ],
            "abstract": "In sentiment analysis, the polarities of the opinions expressed on an\nobject/feature are determined to assess the sentiment of a sentence or document\nwhether it is positive/negative/neutral. Naturally, the object/feature is a\nnoun representation which refers to a product or a component of a product, let\nus say, the \"lens\" in a camera and opinions emanating on it are captured in\nadjectives, verbs, adverbs and noun words themselves. Apart from such words,\nother meta-information and diverse effective features are also going to play an\nimportant role in influencing the sentiment polarity and contribute\nsignificantly to the performance of the system. In this paper, some of the\nassociated information/meta-data are explored and investigated in the sentiment\ntext. Based on the analysis results presented here, there is scope for further\nassessment and utilization of the meta-information as features in text\ncategorization, ranking text document, identification of spam documents and\npolarity classification problems.",
            "journal ref": "Machine Learning and Applications: An International Journal\n  (MLAIJ) Vol.3, No.2, June 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.02802v1",
            "date": "2016-07-11 01:20:57+00:00",
            "title": "Mapping distributional to model-theoretic semantic spaces: a baseline",
            "authors": [
                "Franck Dernoncourt"
            ],
            "abstract": "Word embeddings have been shown to be useful across state-of-the-art systems\nin many natural language processing tasks, ranging from question answering\nsystems to dependency parsing. (Herbelot and Vecchi, 2015) explored word\nembeddings and their utility for modeling language semantics. In particular,\nthey presented an approach to automatically map a standard distributional\nsemantic space onto a set-theoretic model using partial least squares\nregression. We show in this paper that a simple baseline achieves a +51%\nrelative improvement compared to their model on one of the two datasets they\nused, and yields competitive results on the second dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.03474v5",
            "date": "2016-07-12 19:36:50+00:00",
            "title": "Recurrent Highway Networks",
            "authors": [
                "Julian Georg Zilly",
                "Rupesh Kumar Srivastava",
                "Jan Koutn\u00edk",
                "J\u00fcrgen Schmidhuber"
            ],
            "abstract": "Many sequential processing tasks require complex nonlinear transition\nfunctions from one step to the next. However, recurrent neural networks with\n'deep' transition functions remain difficult to train, even when using Long\nShort-Term Memory (LSTM) networks. We introduce a novel theoretical analysis of\nrecurrent networks based on Gersgorin's circle theorem that illuminates several\nmodeling and optimization issues and improves our understanding of the LSTM\ncell. Based on this analysis we propose Recurrent Highway Networks, which\nextend the LSTM architecture to allow step-to-step transition depths larger\nthan one. Several language modeling experiments demonstrate that the proposed\narchitecture results in powerful and efficient models. On the Penn Treebank\ncorpus, solely increasing the transition depth from 1 to 10 improves word-level\nperplexity from 90.6 to 65.4 using the same number of parameters. On the larger\nWikipedia datasets for character prediction (text8 and enwik8), RHNs outperform\nall previous results and achieve an entropy of 1.27 bits per character.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1607.03827v2",
            "date": "2016-07-13 17:08:01+00:00",
            "title": "The KIT Motion-Language Dataset",
            "authors": [
                "Matthias Plappert",
                "Christian Mandery",
                "Tamim Asfour"
            ],
            "abstract": "Linking human motion and natural language is of great interest for the\ngeneration of semantic representations of human activities as well as for the\ngeneration of robot activities based on natural language input. However, while\nthere have been years of research in this area, no standardized and openly\navailable dataset exists to support the development and evaluation of such\nsystems. We therefore propose the KIT Motion-Language Dataset, which is large,\nopen, and extensible. We aggregate data from multiple motion capture databases\nand include them in our dataset using a unified representation that is\nindependent of the capture system or marker set, making it easy to work with\nthe data regardless of its origin. To obtain motion annotations in natural\nlanguage, we apply a crowd-sourcing approach and a web-based tool that was\nspecifically build for this purpose, the Motion Annotation Tool. We thoroughly\ndocument the annotation process itself and discuss gamification methods that we\nused to keep annotators motivated. We further propose a novel method,\nperplexity-based selection, which systematically selects motions for further\nannotation that are either under-represented in our dataset or that have\nerroneous annotations. We show that our method mitigates the two aforementioned\nproblems and ensures a systematic annotation process. We provide an in-depth\nanalysis of the structure and contents of our resulting dataset, which, as of\nOctober 10, 2016, contains 3911 motions with a total duration of 11.23 hours\nand 6278 annotations in natural language that contain 52,903 words. We believe\nthis makes our dataset an excellent choice that enables more transparent and\ncomparable research in this important area.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1607.04315v3",
            "date": "2016-07-14 20:58:26+00:00",
            "title": "Neural Semantic Encoders",
            "authors": [
                "Tsendsuren Munkhdalai",
                "Hong Yu"
            ],
            "abstract": "We present a memory augmented neural network for natural language\nunderstanding: Neural Semantic Encoders. NSE is equipped with a novel memory\nupdate rule and has a variable sized encoding memory that evolves over time and\nmaintains the understanding of input sequences through read}, compose and write\noperations. NSE can also access multiple and shared memories. In this paper, we\ndemonstrated the effectiveness and the flexibility of NSE on five different\nnatural language tasks: natural language inference, question answering,\nsentence classification, document sentiment analysis and machine translation\nwhere NSE achieved state-of-the-art performance when evaluated on publically\navailable benchmarks. For example, our shared-memory model showed an\nencouraging result on neural machine translation, improving an attention-based\nbaseline by approximately 1.0 BLEU.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1607.04492v2",
            "date": "2016-07-15 12:59:01+00:00",
            "title": "Neural Tree Indexers for Text Understanding",
            "authors": [
                "Tsendsuren Munkhdalai",
                "Hong Yu"
            ],
            "abstract": "Recurrent neural networks (RNNs) process input text sequentially and model\nthe conditional transition between word tokens. In contrast, the advantages of\nrecursive networks include that they explicitly model the compositionality and\nthe recursive structure of natural language. However, the current recursive\narchitecture is limited by its dependence on syntactic tree. In this paper, we\nintroduce a robust syntactic parsing-independent tree structured model, Neural\nTree Indexers (NTI) that provides a middle ground between the sequential RNNs\nand the syntactic treebased recursive models. NTI constructs a full n-ary tree\nby processing the input text with its node function in a bottom-up fashion.\nAttention mechanism can then be applied to both structure and node function. We\nimplemented and evaluated a binarytree model of NTI, showing the model achieved\nthe state-of-the-art performance on three different NLP tasks: natural language\ninference, answer sentence selection, and sentence classification,\noutperforming state-of-the-art recurrent and recursive neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.05174v1",
            "date": "2016-07-18 16:44:34+00:00",
            "title": "Is spoken language all-or-nothing? Implications for future speech-based human-machine interaction",
            "authors": [
                "Roger K. Moore"
            ],
            "abstract": "Recent years have seen significant market penetration for voice-based\npersonal assistants such as Apple's Siri. However, despite this success, user\ntake-up is frustratingly low. This position paper argues that there is a\nhabitability gap caused by the inevitable mismatch between the capabilities and\nexpectations of human users and the features and benefits provided by\ncontemporary technology. Suggestions are made as to how such problems might be\nmitigated, but a more worrisome question emerges: \"is spoken language\nall-or-nothing\"? The answer, based on contemporary views on the special nature\nof (spoken) language, is that there may indeed be a fundamental limit to the\ninteraction that can take place between mismatched interlocutors (such as\nhumans and machines). However, it is concluded that interactions between native\nand non-native speakers, or between adults and children, or even between humans\nand dogs, might provide critical inspiration for the design of future\nspeech-based human-machine interaction.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1607.05241v1",
            "date": "2016-07-18 19:01:00+00:00",
            "title": "Imitation Learning with Recurrent Neural Networks",
            "authors": [
                "Khanh Nguyen"
            ],
            "abstract": "We present a novel view that unifies two frameworks that aim to solve\nsequential prediction problems: learning to search (L2S) and recurrent neural\nnetworks (RNN). We point out equivalences between elements of the two\nframeworks. By complementing what is missing from one framework comparing to\nthe other, we introduce a more advanced imitation learning framework that, on\none hand, augments L2S s notion of search space and, on the other hand,\nenhances RNNs training procedure to be more robust to compounding errors\narising from training on highly correlated examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06025v2",
            "date": "2016-07-20 16:59:21+00:00",
            "title": "Constructing a Natural Language Inference Dataset using Generative Neural Networks",
            "authors": [
                "Janez Starc",
                "Dunja Mladeni\u0107"
            ],
            "abstract": "Natural Language Inference is an important task for Natural Language\nUnderstanding. It is concerned with classifying the logical relation between\ntwo sentences. In this paper, we propose several text generative neural\nnetworks for generating text hypothesis, which allows construction of new\nNatural Language Inference datasets. To evaluate the models, we propose a new\nmetric -- the accuracy of the classifier trained on the generated dataset. The\naccuracy obtained by our best generative model is only 2.7% lower than the\naccuracy of the classifier trained on the original, human crafted dataset.\nFurthermore, the best generated dataset combined with the original dataset\nachieves the highest accuracy. The best model learns a mapping embedding for\neach training example. By comparing various metrics we show that datasets that\nobtain higher ROUGE or METEOR scores do not necessarily yield higher\nclassification accuracies. We also provide analysis of what are the\ncharacteristics of a good dataset including the distinguishability of the\ngenerated datasets from the original one.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.06215v1",
            "date": "2016-07-21 07:20:44+00:00",
            "title": "A Comprehensive Survey on Cross-modal Retrieval",
            "authors": [
                "Kaiye Wang",
                "Qiyue Yin",
                "Wei Wang",
                "Shu Wu",
                "Liang Wang"
            ],
            "abstract": "In recent years, cross-modal retrieval has drawn much attention due to the\nrapid growth of multimodal data. It takes one type of data as the query to\nretrieve relevant data of another type. For example, a user can use a text to\nretrieve relevant pictures or videos. Since the query and its retrieved results\ncan be of different modalities, how to measure the content similarity between\ndifferent modalities of data remains a challenge. Various methods have been\nproposed to deal with such a problem. In this paper, we first review a number\nof representative methods for cross-modal retrieval and classify them into two\nmain groups: 1) real-valued representation learning, and 2) binary\nrepresentation learning. Real-valued representation learning methods aim to\nlearn real-valued common representations for different modalities of data. To\nspeed up the cross-modal retrieval, a number of binary representation learning\nmethods are proposed to map different modalities of data into a common Hamming\nspace. Then, we introduce several multimodal datasets in the community, and\nshow the experimental results on two commonly used multimodal datasets. The\ncomparison reveals the characteristic of different kinds of cross-modal\nretrieval methods, which is expected to benefit both practical applications and\nfuture research. Finally, we discuss open problems and future research\ndirections.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1607.06275v2",
            "date": "2016-07-21 11:40:50+00:00",
            "title": "Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering",
            "authors": [
                "Peng Li",
                "Wei Li",
                "Zhengyan He",
                "Xuguang Wang",
                "Ying Cao",
                "Jie Zhou",
                "Wei Xu"
            ],
            "abstract": "While question answering (QA) with neural network, i.e. neural QA, has\nachieved promising results in recent years, lacking of large scale real-word QA\ndataset is still a challenge for developing and evaluating neural QA system. To\nalleviate this problem, we propose a large scale human annotated real-world QA\ndataset WebQA with more than 42k questions and 556k evidences. As existing\nneural QA methods resolve QA either as sequence generation or\nclassification/ranking problem, they face challenges of expensive softmax\ncomputation, unseen answers handling or separate candidate answer generation\ncomponent. In this work, we cast neural QA as a sequence labeling problem and\npropose an end-to-end sequence labeling model, which overcomes all the above\nchallenges. Experimental results on WebQA show that our model outperforms the\nbaselines significantly with an F1 score of 74.69% with word-based input, and\nthe performance drops only 3.72 F1 points with more challenging character-based\ninput.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06520v1",
            "date": "2016-07-21 22:26:20+00:00",
            "title": "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings",
            "authors": [
                "Tolga Bolukbasi",
                "Kai-Wei Chang",
                "James Zou",
                "Venkatesh Saligrama",
                "Adam Kalai"
            ],
            "abstract": "The blind application of machine learning runs the risk of amplifying biases\npresent in data. Such a danger is facing us with word embedding, a popular\nframework to represent text data as vectors which has been used in many machine\nlearning and natural language processing tasks. We show that even word\nembeddings trained on Google News articles exhibit female/male gender\nstereotypes to a disturbing extent. This raises concerns because their\nwidespread use, as we describe, often tends to amplify these biases.\nGeometrically, gender bias is first shown to be captured by a direction in the\nword embedding. Second, gender neutral words are shown to be linearly separable\nfrom gender definition words in the word embedding. Using these properties, we\nprovide a methodology for modifying an embedding to remove gender stereotypes,\nsuch as the association between between the words receptionist and female,\nwhile maintaining desired associations such as between the words queen and\nfemale. We define metrics to quantify both direct and indirect gender biases in\nembeddings, and develop algorithms to \"debias\" the embedding. Using\ncrowd-worker evaluation as well as standard benchmarks, we empirically\ndemonstrate that our algorithms significantly reduce gender bias in embeddings\nwhile preserving the its useful properties such as the ability to cluster\nrelated concepts and to solve analogy tasks. The resulting embeddings can be\nused in applications without amplifying gender bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06532v1",
            "date": "2016-07-22 00:20:09+00:00",
            "title": "Novel Word Embedding and Translation-based Language Modeling for Extractive Speech Summarization",
            "authors": [
                "Kuan-Yu Chen",
                "Shih-Hung Liu",
                "Berlin Chen",
                "Hsin-Min Wang",
                "Hsin-Hsi Chen"
            ],
            "abstract": "Word embedding methods revolve around learning continuous distributed vector\nrepresentations of words with neural networks, which can capture semantic\nand/or syntactic cues, and in turn be used to induce similarity measures among\nwords, sentences and documents in context. Celebrated methods can be\ncategorized as prediction-based and count-based methods according to the\ntraining objectives and model architectures. Their pros and cons have been\nextensively analyzed and evaluated in recent studies, but there is relatively\nless work continuing the line of research to develop an enhanced learning\nmethod that brings together the advantages of the two model families. In\naddition, the interpretation of the learned word representations still remains\nsomewhat opaque. Motivated by the observations and considering the pressing\nneed, this paper presents a novel method for learning the word representations,\nwhich not only inherits the advantages of classic word embedding methods but\nalso offers a clearer and more rigorous interpretation of the learned word\nrepresentations. Built upon the proposed word embedding method, we further\nformulate a translation-based language modeling framework for the extractive\nspeech summarization task. A series of empirical evaluations demonstrate the\neffectiveness of the proposed word representation learning and language\nmodeling techniques in extractive speech summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.06875v2",
            "date": "2016-07-23 01:46:09+00:00",
            "title": "Processing Natural Language About Ongoing Actions",
            "authors": [
                "Steve Doubleday",
                "Sean Trott",
                "Jerome Feldman"
            ],
            "abstract": "Actions may not proceed as planned; they may be interrupted, resumed or\noverridden. This is a challenge to handle in a natural language understanding\nsystem. We describe extensions to an existing implementation for the control of\nautonomous systems by natural language, to enable such systems to handle\nincoming language requests regarding actions. Language Communication with\nAutonomous Systems (LCAS) has been extended with support for X-nets,\nparameterized executable schemas representing actions. X-nets enable the system\nto control actions at a desired level of granularity, while providing a\nmechanism for language requests to be processed asynchronously. Standard\nsemantics supported include requests to stop, continue, or override the\nexisting action. The specific domain demonstrated is the control of motion of a\nsimulated robot, but the approach is general, and could be applied to other\ndomains.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1607.07514v1",
            "date": "2016-07-26 00:58:14+00:00",
            "title": "Tweet2Vec: Learning Tweet Embeddings Using Character-level CNN-LSTM Encoder-Decoder",
            "authors": [
                "Soroush Vosoughi",
                "Prashanth Vijayaraghavan",
                "Deb Roy"
            ],
            "abstract": "We present Tweet2Vec, a novel method for generating general-purpose vector\nrepresentation of tweets. The model learns tweet embeddings using\ncharacter-level CNN-LSTM encoder-decoder. We trained our model on 3 million,\nrandomly selected English-language tweets. The model was evaluated using two\nmethods: tweet semantic similarity and tweet sentiment categorization,\noutperforming the previous state-of-the-art in both tasks. The evaluations\ndemonstrate the power of the tweet embeddings generated by our model for\nvarious tweet categorization tasks. The vector representations generated by our\nmodel are generic, and hence can be applied to a variety of tasks. Though the\nmodel presented in this paper is trained on English-language tweets, the method\npresented can be used to learn tweet embeddings for different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.07602v1",
            "date": "2016-07-26 09:19:46+00:00",
            "title": "OntoCat: Automatically categorizing knowledge in API Documentation",
            "authors": [
                "Niraj Kumar",
                "Premkumar Devanbu"
            ],
            "abstract": "Most application development happens in the context of complex APIs;\nreference documentation for APIs has grown tremendously in variety, complexity,\nand volume, and can be difficult to navigate. There is a growing need to\ndevelop well-organized ways to access the knowledge latent in the\ndocumentation; several research efforts deal with the organization (ontology)\nof API-related knowledge. Extensive knowledge-engineering work, supported by a\nrigorous qualitative analysis, by Maalej & Robillard [3] has identified a\nuseful taxonomy of API knowledge. Based on this taxonomy, we introduce a domain\nindependent technique to extract the knowledge types from the given API\nreference documentation. Our system, OntoCat, introduces total nine different\nfeatures and their semantic and statistical combinations to classify the\ndifferent knowledge types. We tested OntoCat on python API reference\ndocumentation. Our experimental results show the effectiveness of the system\nand opens the scope of probably related research areas (i.e., user behavior,\ndocumentation quality, etc.).",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1607.08720v2",
            "date": "2016-07-29 08:17:45+00:00",
            "title": "TopicResponse: A Marriage of Topic Modelling and Rasch Modelling for Automatic Measurement in MOOCs",
            "authors": [
                "Jiazhen He",
                "Benjamin I. P. Rubinstein",
                "James Bailey",
                "Rui Zhang",
                "Sandra Milligan"
            ],
            "abstract": "This paper explores the suitability of using automatically discovered topics\nfrom MOOC discussion forums for modelling students' academic abilities. The\nRasch model from psychometrics is a popular generative probabilistic model that\nrelates latent student skill, latent item difficulty, and observed student-item\nresponses within a principled, unified framework. According to scholarly\neducational theory, discovered topics can be regarded as appropriate\nmeasurement items if (1) students' participation across the discovered topics\nis well fit by the Rasch model, and if (2) the topics are interpretable to\nsubject-matter experts as being educationally meaningful. Such Rasch-scaled\ntopics, with associated difficulty levels, could be of potential benefit to\ncurriculum refinement, student assessment and personalised feedback. The\ntechnical challenge that remains, is to discover meaningful topics that\nsimultaneously achieve good statistical fit with the Rasch model. To address\nthis challenge, we combine the Rasch model with non-negative matrix\nfactorisation based topic modelling, jointly fitting both models. We\ndemonstrate the suitability of our approach with quantitative experiments on\ndata from three Coursera MOOCs, and with qualitative survey results on topic\ninterpretability on a Discrete Optimisation MOOC.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1607.08723v4",
            "date": "2016-07-29 08:33:10+00:00",
            "title": "Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner",
            "authors": [
                "Emmanuel Dupoux"
            ],
            "abstract": "During their first years of life, infants learn the language(s) of their\nenvironment at an amazing speed despite large cross cultural variations in\namount and complexity of the available language input. Understanding this\nsimple fact still escapes current cognitive and linguistic theories. Recently,\nspectacular progress in the engineering science, notably, machine learning and\nwearable technology, offer the promise of revolutionizing the study of\ncognitive development. Machine learning offers powerful learning algorithms\nthat can achieve human-like performance on many linguistic tasks. Wearable\nsensors can capture vast amounts of data, which enable the reconstruction of\nthe sensory experience of infants in their natural environment. The project of\n'reverse engineering' language development, i.e., of building an effective\nsystem that mimics infant's achievements appears therefore to be within reach.\nHere, we analyze the conditions under which such a project can contribute to\nour scientific understanding of early language development. We argue that\ninstead of defining a sub-problem or simplifying the data, computational models\nshould address the full complexity of the learning situation, and take as input\nthe raw sensory signals available to infants. This implies that (1) accessible\nbut privacy-preserving repositories of home data be setup and widely shared,\nand (2) models be evaluated at different linguistic levels through a benchmark\nof psycholinguist tests that can be passed by machines and humans alike, (3)\nlinguistically and psychologically plausible learning architectures be scaled\nup to real data using probabilistic/optimization principles from machine\nlearning. We discuss the feasibility of this approach and present preliminary\nresults.",
            "journal ref": "Dupoux, E. (2018). Cognitive science in the era of artificial\n  intelligence: A roadmap for reverse-engineering the infant language learner.\n  Cognition, 173, 43-59",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.08864v2",
            "date": "2016-07-29 16:26:54+00:00",
            "title": "The DLVHEX System for Knowledge Representation: Recent Advances (System Description)",
            "authors": [
                "Christoph Redl"
            ],
            "abstract": "The DLVHEX system implements the HEX-semantics, which integrates answer set\nprogramming (ASP) with arbitrary external sources. Since its first release ten\nyears ago, significant advancements were achieved. Most importantly, the\nexploitation of properties of external sources led to efficiency improvements\nand flexibility enhancements of the language, and technical improvements on the\nsystem side increased user's convenience. In this paper, we present the current\nstatus of the system and point out the most important recent enhancements over\nearly versions. While existing literature focuses on theoretical aspects and\nspecific components, a bird's eye view of the overall system is missing. In\norder to promote the system for real-world applications, we further present\napplications which were already successfully realized on top of DLVHEX. This\npaper is under consideration for acceptance in Theory and Practice of Logic\nProgramming.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.00816v1",
            "date": "2016-01-05 12:50:14+00:00",
            "title": "Open challenges in understanding development and evolution of speech forms: The roles of embodied self-organization, motivation and active exploration",
            "authors": [
                "Pierre-Yves Oudeyer"
            ],
            "abstract": "This article discusses open scientific challenges for understanding\ndevelopment and evolution of speech forms, as a commentary to Moulin-Frier et\nal. (Moulin-Frier et al., 2015). Based on the analysis of mathematical models\nof the origins of speech forms, with a focus on their assumptions , we study\nthe fundamental question of how speech can be formed out of non--speech, at\nboth developmental and evolutionary scales. In particular, we emphasize the\nimportance of embodied self-organization , as well as the role of mechanisms of\nmotivation and active curiosity-driven exploration in speech formation. Finally\n, we discuss an evolutionary-developmental perspective of the origins of\nspeech.",
            "journal ref": "Journal of Phonetics, Elsevier, 2015, 53, pp.5",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.01356v3",
            "date": "2016-01-07 00:09:37+00:00",
            "title": "From Word Embeddings to Item Recommendation",
            "authors": [
                "Makbule Gulcin Ozsoy"
            ],
            "abstract": "Social network platforms can use the data produced by their users to serve\nthem better. One of the services these platforms provide is recommendation\nservice. Recommendation systems can predict the future preferences of users\nusing their past preferences. In the recommendation systems literature there\nare various techniques, such as neighborhood based methods, machine-learning\nbased methods and matrix-factorization based methods. In this work, a set of\nwell known methods from natural language processing domain, namely Word2Vec, is\napplied to recommendation systems domain. Unlike previous works that use\nWord2Vec for recommendation, this work uses non-textual features, the\ncheck-ins, and it recommends venues to visit/check-in to the target users. For\nthe experiments, a Foursquare check-in dataset is used. The results show that\nuse of continuous vector space representations of items modeled by techniques\nof Word2Vec is promising for making recommendations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1601.01705v4",
            "date": "2016-01-07 21:21:59+00:00",
            "title": "Learning to Compose Neural Networks for Question Answering",
            "authors": [
                "Jacob Andreas",
                "Marcus Rohrbach",
                "Trevor Darrell",
                "Dan Klein"
            ],
            "abstract": "We describe a question answering model that applies to both images and\nstructured knowledge bases. The model uses natural language strings to\nautomatically assemble neural networks from a collection of composable modules.\nParameters for these modules are learned jointly with network-assembly\nparameters via reinforcement learning, with only (world, question, answer)\ntriples as supervision. Our approach, which we term a dynamic neural model\nnetwork, achieves state-of-the-art results on benchmark datasets in both visual\nand structured domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02543v1",
            "date": "2016-01-11 18:01:56+00:00",
            "title": "Evaluating the Performance of a Speech Recognition based System",
            "authors": [
                "Vinod Kumar Pandey",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Speech based solutions have taken center stage with growth in the services\nindustry where there is a need to cater to a very large number of people from\nall strata of the society. While natural language speech interfaces are the\ntalk in the research community, yet in practice, menu based speech solutions\nthrive. Typically in a menu based speech solution the user is required to\nrespond by speaking from a closed set of words when prompted by the system. A\nsequence of human speech response to the IVR prompts results in the completion\nof a transaction. A transaction is deemed successful if the speech solution can\ncorrectly recognize all the spoken utterances of the user whenever prompted by\nthe system. The usual mechanism to evaluate the performance of a speech\nsolution is to do an extensive test of the system by putting it to actual\npeople use and then evaluating the performance by analyzing the logs for\nsuccessful transactions. This kind of evaluation could lead to dissatisfied\ntest users especially if the performance of the system were to result in a poor\ntransaction completion rate. To negate this the Wizard of Oz approach is\nadopted during evaluation of a speech system. Overall this kind of evaluations\nis an expensive proposition both in terms of time and cost. In this paper, we\npropose a method to evaluate the performance of a speech solution without\nactually putting it to people use. We first describe the methodology and then\nshow experimentally that this can be used to identify the performance\nbottlenecks of the speech solution even before the system is actually used thus\nsaving evaluation time and expenses.",
            "journal ref": "ACC (3) 2011: 230-238",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02789v1",
            "date": "2016-01-12 10:06:52+00:00",
            "title": "Comparison and Adaptation of Automatic Evaluation Metrics for Quality Assessment of Re-Speaking",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Danijel Kor\u017einek"
            ],
            "abstract": "Re-speaking is a mechanism for obtaining high quality subtitles for use in\nlive broadcast and other public events. Because it relies on humans performing\nthe actual re-speaking, the task of estimating the quality of the results is\nnon-trivial. Most organisations rely on humans to perform the actual quality\nassessment, but purely automatic methods have been developed for other similar\nproblems, like Machine Translation. This paper will try to compare several of\nthese methods: BLEU, EBLEU, NIST, METEOR, METEOR-PL, TER and RIBES. These will\nthen be matched to the human-derived NER metric, commonly used in re-speaking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.02828v2",
            "date": "2016-01-12 12:33:56+00:00",
            "title": "Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation",
            "authors": [
                "Pawel Swietojanski",
                "Jinyu Li",
                "Steve Renals"
            ],
            "abstract": "This work presents a broad study on the adaptation of neural network acoustic\nmodels by means of learning hidden unit contributions (LHUC) -- a method that\nlinearly re-combines hidden units in a speaker- or environment-dependent manner\nusing small amounts of unsupervised adaptation data. We also extend LHUC to a\nspeaker adaptive training (SAT) framework that leads to a more adaptable DNN\nacoustic model, working both in a speaker-dependent and a speaker-independent\nmanner, without the requirements to maintain auxiliary speaker-dependent\nfeature extractors or to introduce significant speaker-dependent changes to the\nDNN structure. Through a series of experiments on four different speech\nrecognition benchmarks (TED talks, Switchboard, AMI meetings, and Aurora4)\ncomprising 270 test speakers, we show that LHUC in both its test-only and SAT\nvariants results in consistent word error rate reductions ranging from 5% to\n23% relative depending on the task and the degree of mismatch between training\nand test data. In addition, we have investigated the effect of the amount of\nadaptation data per speaker, the quality of unsupervised adaptation targets,\nthe complementarity to other adaptation techniques, one-shot adaptation, and an\nextension to adapting DNNs trained in a sequence discriminative manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03210v3",
            "date": "2016-01-13 11:48:25+00:00",
            "title": "The scarcity of crossing dependencies: a direct outcome of a specific constraint?",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "The structure of a sentence can be represented as a network where vertices\nare words and edges indicate syntactic dependencies. Interestingly, crossing\nsyntactic dependencies have been observed to be infrequent in human languages.\nThis leads to the question of whether the scarcity of crossings in languages\narises from an independent and specific constraint on crossings. We provide\nstatistical evidence suggesting that this is not the case, as the proportion of\ndependency crossings of sentences from a wide range of languages can be\naccurately estimated by a simple predictor based on a null hypothesis on the\nlocal probability that two dependencies cross given their lengths. The relative\nerror of this predictor never exceeds 5% on average, whereas the error of a\nbaseline predictor assuming a random ordering of the words of a sentence is at\nleast 6 times greater. Our results suggest that the low frequency of crossings\nin natural languages is neither originated by hidden knowledge of language nor\nby the undesirability of crossings per se, but as a mere side effect of the\nprinciple of dependency length minimization.",
            "journal ref": "Phys. Rev. E 96, 062304 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.03764v6",
            "date": "2016-01-14 22:02:18+00:00",
            "title": "Linear Algebraic Structure of Word Senses, with Applications to Polysemy",
            "authors": [
                "Sanjeev Arora",
                "Yuanzhi Li",
                "Yingyu Liang",
                "Tengyu Ma",
                "Andrej Risteski"
            ],
            "abstract": "Word embeddings are ubiquitous in NLP and information retrieval, but it is\nunclear what they represent when the word is polysemous. Here it is shown that\nmultiple word senses reside in linear superposition within the word embedding\nand simple sparse coding can recover vectors that approximately capture the\nsenses. The success of our approach, which applies to several embedding\nmethods, is mathematically explained using a variant of the random walk on\ndiscourses model (Arora et al., 2016). A novel aspect of our technique is that\neach extracted word sense is accompanied by one of about 2000 \"discourse atoms\"\nthat gives a succinct description of which other words co-occur with that word\nsense. Discourse atoms can be of independent interest, and make the method\npotentially more useful. Empirical tests are used to verify and support the\ntheory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04075v1",
            "date": "2016-01-15 21:01:12+00:00",
            "title": "Modification of Question Writing Style Influences Content Popularity in a Social Q&A System",
            "authors": [
                "Igor A. Podgorny"
            ],
            "abstract": "TurboTax AnswerXchange is a social Q&A system supporting users working on\nfederal and state tax returns. Using 2015 data, we demonstrate that content\npopularity (or number of views per AnswerXchange question) can be predicted\nwith reasonable accuracy based on attributes of the question alone. We also\nemploy probabilistic topic analysis and uplift modeling to identify question\nfeatures with the highest impact on popularity. We demonstrate that content\npopularity is driven by behavioral attributes of AnswerXchange users and\ndepends on complex interactions between search ranking algorithms,\npsycholinguistic factors and question writing style. Our findings provide a\nrationale for employing popularity predictions to guide the users into\nformulating better questions and editing the existing ones. For example,\nstarting question title with a question word or adding details to the question\nincrease number of views per question. Similar approach can be applied to\npromoting AnswerXchange content indexed by Google to drive organic traffic to\nTurboTax.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1601.05893v1",
            "date": "2016-01-22 07:09:54+00:00",
            "title": "GeoTextTagger: High-Precision Location Tagging of Textual Documents using a Natural Language Processing Approach",
            "authors": [
                "Shawn Brunsting",
                "Hans De Sterck",
                "Remco Dolman",
                "Teun van Sprundel"
            ],
            "abstract": "Location tagging, also known as geotagging or geolocation, is the process of\nassigning geographical coordinates to input data. In this paper we present an\nalgorithm for location tagging of textual documents. Our approach makes use of\nprevious work in natural language processing by using a state-of-the-art\npart-of-speech tagger and named entity recognizer to find blocks of text which\nmay refer to locations. A knowledge base (OpenStreatMap) is then used to find a\nlist of possible locations for each block. Finally, one location is chosen for\neach block by assigning distance-based scores to each location and repeatedly\nselecting the location and block with the best score. We tested our geolocation\nalgorithm with Wikipedia articles about topics with a well-defined geographical\nlocation that are geotagged by the articles' authors, where classification\napproaches have achieved median errors as low as 11 km, with attainable\naccuracy limited by the class size. Our approach achieved a 10th percentile\nerror of 490 metres and median error of 54 kilometres on the Wikipedia dataset\nwe used. When considering the five location tags with the greatest scores, 50%\nof articles were assigned at least one tag within 8.5 kilometres of the\narticle's author-assigned true location. We also tested our approach on Twitter\nmessages that are tagged with the location from which the message was sent.\nTwitter texts are challenging because they are short and unstructured and often\ndo not contain words referring to the location they were sent from, but we\nobtain potentially useful results. We explain how we use the Spark framework\nfor data analytics to collect and process our test data. In general,\nclassification-based approaches for location tagging may be reaching their\nupper accuracy limit, but our precision-focused approach has high accuracy for\nsome texts and shows significant potential for improvement overall.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.05936v1",
            "date": "2016-01-22 10:02:47+00:00",
            "title": "Exploiting Low-dimensional Structures to Enhance DNN Based Acoustic Modeling in Speech Recognition",
            "authors": [
                "Pranay Dighe",
                "Gil Luyet",
                "Afsaneh Asaei",
                "Herve Bourlard"
            ],
            "abstract": "We propose to model the acoustic space of deep neural network (DNN)\nclass-conditional posterior probabilities as a union of low-dimensional\nsubspaces. To that end, the training posteriors are used for dictionary\nlearning and sparse coding. Sparse representation of the test posteriors using\nthis dictionary enables projection to the space of training data. Relying on\nthe fact that the intrinsic dimensions of the posterior subspaces are indeed\nvery small and the matrix of all posteriors belonging to a class has a very low\nrank, we demonstrate how low-dimensional structures enable further enhancement\nof the posteriors and rectify the spurious errors due to mismatch conditions.\nThe enhanced acoustic modeling method leads to improvements in continuous\nspeech recognition task using hybrid DNN-HMM (hidden Markov model) framework in\nboth clean and noisy conditions, where upto 15.4% relative reduction in word\nerror rate (WER) is achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06081v1",
            "date": "2016-01-22 17:33:28+00:00",
            "title": "Why Do Urban Legends Go Viral?",
            "authors": [
                "Marco Guerini",
                "Carlo Strapparava"
            ],
            "abstract": "Urban legends are a genre of modern folklore, consisting of stories about\nrare and exceptional events, just plausible enough to be believed, which tend\nto propagate inexorably across communities. In our view, while urban legends\nrepresent a form of \"sticky\" deceptive text, they are marked by a tension\nbetween the credible and incredible. They should be credible like a news\narticle and incredible like a fairy tale to go viral. In particular we will\nfocus on the idea that urban legends should mimic the details of news (who,\nwhere, when) to be credible, while they should be emotional and readable like a\nfairy tale to be catchy and memorable. Using NLP tools we will provide a\nquantitative analysis of these prototypical characteristics. We also lay out\nsome machine learning experiments showing that it is possible to recognize an\nurban legend using just these simple features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06581v2",
            "date": "2016-01-25 12:51:46+00:00",
            "title": "Character-Level Incremental Speech Recognition with Recurrent Neural Networks",
            "authors": [
                "Kyuyeon Hwang",
                "Wonyong Sung"
            ],
            "abstract": "In real-time speech recognition applications, the latency is an important\nissue. We have developed a character-level incremental speech recognition (ISR)\nsystem that responds quickly even during the speech, where the hypotheses are\ngradually improved while the speaking proceeds. The algorithm employs a\nspeech-to-character unidirectional recurrent neural network (RNN), which is\nend-to-end trained with connectionist temporal classification (CTC), and an\nRNN-based character-level language model (LM). The output values of the\nCTC-trained RNN are character-level probabilities, which are processed by beam\nsearch decoding. The RNN LM augments the decoding by providing long-term\ndependency information. We propose tree-based online beam search with\nadditional depth-pruning, which enables the system to process infinitely long\ninput speech with low latency. This system not only responds quickly on speech\nbut also can dictate out-of-vocabulary (OOV) words according to pronunciation.\nThe proposed model achieves the word error rate (WER) of 8.90% on the Wall\nStreet Journal (WSJ) Nov'92 20K evaluation set when trained on the WSJ SI-284\ntraining set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.06732v1",
            "date": "2016-01-25 19:23:44+00:00",
            "title": "Concept Generation in Language Evolution",
            "authors": [
                "Martha Lewis",
                "Jonathan Lawry"
            ],
            "abstract": "This thesis investigates the generation of new concepts from combinations of\nexisting concepts as a language evolves. We give a method for combining\nconcepts, and will be investigating the utility of composite concepts in\nlanguage evolution and thence the utility of concept generation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.06755v1",
            "date": "2016-01-25 20:24:50+00:00",
            "title": "The Utility of Hedged Assertions in the Emergence of Shared Categorical Labels",
            "authors": [
                "Martha Lewis",
                "Jonathan Lawry"
            ],
            "abstract": "We investigate the emergence of shared concepts in a community of language\nusers using a multi-agent simulation. We extend results showing that negated\nassertions are of use in developing shared categories, to include assertions\nmodified by linguistic hedges. Results show that using hedged assertions\npositively affects the emergence of shared categories in two distinct ways.\nFirstly, using contraction hedges like `very' gives better convergence over\ntime. Secondly, using expansion hedges such as `quite' reduces concept overlap.\nHowever, both these improvements come at a cost of slower speed of development.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1601.06763v1",
            "date": "2016-01-25 20:40:55+00:00",
            "title": "Emerging Dimension Weights in a Conceptual Spaces Model of Concept Combination",
            "authors": [
                "Martha Lewis",
                "Jonathan Lawry"
            ],
            "abstract": "We investigate the generation of new concepts from combinations of properties\nas an artificial language develops. To do so, we have developed a new framework\nfor conjunctive concept combination. This framework gives a semantic grounding\nto the weighted sum approach to concept combination seen in the literature. We\nimplement the framework in a multi-agent simulation of language evolution and\nshow that shared combination weights emerge. The expected value and the\nvariance of these weights across agents may be predicted from the distribution\nof elements in the conceptual space, as determined by the underlying\nenvironment, together with the rate at which agents adopt others' concepts.\nWhen this rate is smaller, the agents are able to converge to weights with\nlower variance. However, the time taken to converge to a steady state\ndistribution of weights is longer.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1602.01103v2",
            "date": "2016-02-02 21:00:11+00:00",
            "title": "Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions",
            "authors": [
                "Chenhao Tan",
                "Vlad Niculae",
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee"
            ],
            "abstract": "Changing someone's opinion is arguably one of the most important challenges\nof social interaction. The underlying process proves difficult to study: it is\nhard to know how someone's opinions are formed and whether and how someone's\nviews shift. Fortunately, ChangeMyView, an active community on Reddit, provides\na platform where users present their own opinions and reasoning, invite others\nto contest them, and acknowledge when the ensuing discussions change their\noriginal views. In this work, we study these interactions to understand the\nmechanisms behind persuasion.\n  We find that persuasive arguments are characterized by interesting patterns\nof interaction dynamics, such as participant entry-order and degree of\nback-and-forth exchange. Furthermore, by comparing similar counterarguments to\nthe same opinion, we show that language factors play an essential role. In\nparticular, the interplay between the language of the opinion holder and that\nof the counterargument provides highly predictive cues of persuasiveness.\nFinally, since even in this favorable setting people may not be persuaded, we\ninvestigate the problem of determining whether someone's opinion is susceptible\nto being changed at all. For this more difficult task, we show that stylistic\nchoices in how the opinion is expressed carry predictive power.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1602.01208v3",
            "date": "2016-02-03 06:56:51+00:00",
            "title": "Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences",
            "authors": [
                "Akira Taniguchi",
                "Tadahiro Taniguchi",
                "Tetsunari Inamura"
            ],
            "abstract": "In this paper, we propose a novel unsupervised learning method for the\nlexical acquisition of words related to places visited by robots, from human\ncontinuous speech signals. We address the problem of learning novel words by a\nrobot that has no prior knowledge of these words except for a primitive\nacoustic model. Further, we propose a method that allows a robot to effectively\nuse the learned words and their meanings for self-localization tasks. The\nproposed method is nonparametric Bayesian spatial concept acquisition method\n(SpCoA) that integrates the generative model for self-localization and the\nunsupervised word segmentation in uttered sentences via latent variables\nrelated to the spatial concept. We implemented the proposed method SpCoA on\nSIGVerse, which is a simulation environment, and TurtleBot2, which is a mobile\nrobot in a real environment. Further, we conducted experiments for evaluating\nthe performance of SpCoA. The experimental results showed that SpCoA enabled\nthe robot to acquire the names of places from speech sentences. They also\nrevealed that the robot could effectively utilize the acquired spatial concepts\nand reduce the uncertainty in self-localization.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1602.01248v1",
            "date": "2016-02-03 10:19:19+00:00",
            "title": "Using Hadoop for Large Scale Analysis on Twitter: A Technical Report",
            "authors": [
                "Nikolaos Nodarakis",
                "Spyros Sioutas",
                "Athanasios Tsakalidis",
                "Giannis Tzimas"
            ],
            "abstract": "Sentiment analysis (or opinion mining) on Twitter data has attracted much\nattention recently. One of the system's key features, is the immediacy in\ncommunication with other users in an easy, user-friendly and fast way.\nConsequently, people tend to express their feelings freely, which makes Twitter\nan ideal source for accumulating a vast amount of opinions towards a wide\ndiversity of topics. This amount of information offers huge potential and can\nbe harnessed to receive the sentiment tendency towards these topics. However,\nsince none can invest an infinite amount of time to read through these tweets,\nan automated decision making approach is necessary. Nevertheless, most existing\nsolutions are limited in centralized environments only. Thus, they can only\nprocess at most a few thousand tweets. Such a sample, is not representative to\ndefine the sentiment polarity towards a topic due to the massive number of\ntweets published daily. In this paper, we go one step further and develop a\nnovel method for sentiment learning in the MapReduce framework. Our algorithm\nexploits the hashtags and emoticons inside a tweet, as sentiment labels, and\nproceeds to a classification procedure of diverse sentiment types in a parallel\nand distributed manner. Moreover, we utilize Bloom filters to compact the\nstorage size of intermediate data and boost the performance of our algorithm.\nThrough an extensive experimental evaluation, we prove that our solution is\nefficient, robust and scalable and confirm the quality of our sentiment\nidentification.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1602.01635v2",
            "date": "2016-02-04 11:15:28+00:00",
            "title": "A Generalised Quantifier Theory of Natural Language in Categorical Compositional Distributional Semantics with Bialgebras",
            "authors": [
                "Jules Hedges",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Categorical compositional distributional semantics is a model of natural\nlanguage; it combines the statistical vector space models of words with the\ncompositional models of grammar. We formalise in this model the generalised\nquantifier theory of natural language, due to Barwise and Cooper. The\nunderlying setting is a compact closed category with bialgebras. We start from\na generative grammar formalisation and develop an abstract categorical\ncompositional semantics for it, then instantiate the abstract setting to sets\nand relations and to finite dimensional vector spaces and linear maps. We prove\nthe equivalence of the relational instantiation to the truth theoretic\nsemantics of generalised quantifiers. The vector space instantiation formalises\nthe statistical usages of words and enables us to, for the first time, reason\nabout quantified phrases and sentences compositionally in distributional\nsemantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.01895v1",
            "date": "2016-02-05 00:17:18+00:00",
            "title": "Generate Image Descriptions based on Deep RNN and Memory Cells for Images Features",
            "authors": [
                "Shijian Tang",
                "Song Han"
            ],
            "abstract": "Generating natural language descriptions for images is a challenging task.\nThe traditional way is to use the convolutional neural network (CNN) to extract\nimage features, followed by recurrent neural network (RNN) to generate\nsentences. In this paper, we present a new model that added memory cells to\ngate the feeding of image features to the deep neural network. The intuition is\nenabling our model to memorize how much information from images should be fed\nat each stage of the RNN. Experiments on Flickr8K and Flickr30K datasets showed\nthat our model outperforms other state-of-the-art models with higher BLEU\nscores.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1602.02068v2",
            "date": "2016-02-05 15:49:02+00:00",
            "title": "From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification",
            "authors": [
                "Andr\u00e9 F. T. Martins",
                "Ram\u00f3n Fernandez Astudillo"
            ],
            "abstract": "We propose sparsemax, a new activation function similar to the traditional\nsoftmax, but able to output sparse probabilities. After deriving its\nproperties, we show how its Jacobian can be efficiently computed, enabling its\nuse in a network trained with backpropagation. Then, we propose a new smooth\nand convex loss function which is the sparsemax analogue of the logistic loss.\nWe reveal an unexpected connection between this new loss and the Huber\nclassification loss. We obtain promising empirical results in multi-label\nclassification problems and in attention-based neural networks for natural\nlanguage inference. For the latter, we achieve a similar performance as the\ntraditional softmax, but with a selective, more compact, attention focus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.02332v1",
            "date": "2016-02-07 02:49:27+00:00",
            "title": "Scalable Text Mining with Sparse Generative Models",
            "authors": [
                "Antti Puurula"
            ],
            "abstract": "The information age has brought a deluge of data. Much of this is in text\nform, insurmountable in scope for humans and incomprehensible in structure for\ncomputers. Text mining is an expanding field of research that seeks to utilize\nthe information contained in vast document collections. General data mining\nmethods based on machine learning face challenges with the scale of text data,\nposing a need for scalable text mining methods.\n  This thesis proposes a solution to scalable text mining: generative models\ncombined with sparse computation. A unifying formalization for generative text\nmodels is defined, bringing together research traditions that have used\nformally equivalent models, but ignored parallel developments. This framework\nallows the use of methods developed in different processing tasks such as\nretrieval and classification, yielding effective solutions across different\ntext mining tasks. Sparse computation using inverted indices is proposed for\ninference on probabilistic models. This reduces the computational complexity of\nthe common text mining operations according to sparsity, yielding probabilistic\nmodels with the scalability of modern search engines.\n  The proposed combination provides sparse generative models: a solution for\ntext mining that is general, effective, and scalable. Extensive experimentation\non text classification and ranked retrieval datasets are conducted, showing\nthat the proposed solution matches or outperforms the leading task-specific\nmethods in effectiveness, with a order of magnitude decrease in classification\ntimes for Wikipedia article categorization with a million classes. The\ndeveloped methods were further applied in two 2014 Kaggle data mining prize\ncompetitions with over a hundred competing teams, earning first and second\nplaces.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1602.02373v2",
            "date": "2016-02-07 14:05:58+00:00",
            "title": "Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings",
            "authors": [
                "Rie Johnson",
                "Tong Zhang"
            ],
            "abstract": "One-hot CNN (convolutional neural network) has been shown to be effective for\ntext categorization (Johnson & Zhang, 2015). We view it as a special case of a\ngeneral framework which jointly trains a linear model with a non-linear feature\ngenerator consisting of `text region embedding + pooling'. Under this\nframework, we explore a more sophisticated region embedding method using Long\nShort-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly\nlarge) sizes, whereas the region size needs to be fixed in a CNN. We seek\neffective and efficient use of LSTM for this purpose in the supervised and\nsemi-supervised settings. The best results were obtained by combining region\nembeddings in the form of LSTM and convolution layers trained on unlabeled\ndata. The results indicate that on this task, embeddings of text regions, which\ncan convey complex concepts, are more useful than embeddings of single words in\nisolation. We report performances exceeding the previous best results on four\nbenchmark datasets.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1602.02665v1",
            "date": "2016-02-08 17:46:18+00:00",
            "title": "The happiness paradox: your friends are happier than you",
            "authors": [
                "Johan Bollen",
                "Bruno Gon\u00e7alves",
                "Ingrid van de Leemput",
                "Guangchen Ruan"
            ],
            "abstract": "Most individuals in social networks experience a so-called Friendship\nParadox: they are less popular than their friends on average. This effect may\nexplain recent findings that widespread social network media use leads to\nreduced happiness. However the relation between popularity and happiness is\npoorly understood. A Friendship paradox does not necessarily imply a Happiness\nparadox where most individuals are less happy than their friends. Here we\nreport the first direct observation of a significant Happiness Paradox in a\nlarge-scale online social network of $39,110$ Twitter users. Our results reveal\nthat popular individuals are indeed happier and that a majority of individuals\nexperience a significant Happiness paradox. The magnitude of the latter effect\nis shaped by complex interactions between individual popularity, happiness, and\nthe fact that users cluster assortatively by level of happiness. Our results\nindicate that the topology of online social networks and the distribution of\nhappiness in some populations can cause widespread psycho-social effects that\naffect the well-being of billions of individuals.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1602.02850v1",
            "date": "2016-02-09 03:43:21+00:00",
            "title": "Toward Optimal Feature Selection in Naive Bayes for Text Categorization",
            "authors": [
                "Bo Tang",
                "Steven Kay",
                "Haibo He"
            ],
            "abstract": "Automated feature selection is important for text categorization to reduce\nthe feature size and to speed up the learning process of classifiers. In this\npaper, we present a novel and efficient feature selection framework based on\nthe Information Theory, which aims to rank the features with their\ndiscriminative capacity for classification. We first revisit two information\nmeasures: Kullback-Leibler divergence and Jeffreys divergence for binary\nhypothesis testing, and analyze their asymptotic properties relating to type I\nand type II errors of a Bayesian classifier. We then introduce a new divergence\nmeasure, called Jeffreys-Multi-Hypothesis (JMH) divergence, to measure\nmulti-distribution divergence for multi-class classification. Based on the\nJMH-divergence, we develop two efficient feature selection methods, termed\nmaximum discrimination ($MD$) and $MD-\\chi^2$ methods, for text categorization.\nThe promising results of extensive experiments demonstrate the effectiveness of\nthe proposed approaches.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1602.03001v2",
            "date": "2016-02-09 14:36:49+00:00",
            "title": "A Convolutional Attention Network for Extreme Summarization of Source Code",
            "authors": [
                "Miltiadis Allamanis",
                "Hao Peng",
                "Charles Sutton"
            ],
            "abstract": "Attention mechanisms in neural networks have proved useful for problems in\nwhich the input and output do not have fixed dimension. Often there exist\nfeatures that are locally translation invariant and would be valuable for\ndirecting the model's attention, but previous attentional architectures are not\nconstructed to learn such features specifically. We introduce an attentional\nneural network that employs convolution on the input tokens to detect local\ntime-invariant and long-range topical attention features in a context-dependent\nway. We apply this architecture to the problem of extreme summarization of\nsource code snippets into short, descriptive function name-like summaries.\nUsing those features, the model sequentially generates a summary by\nmarginalizing over two attention mechanisms: one that predicts the next summary\ntoken based on the attention weights of the input tokens and another that is\nable to copy a code token as-is directly into the summary. We demonstrate our\nconvolutional attention neural network's performance on 10 popular Java\nprojects showing that it achieves better performance compared to previous\nattentional mechanisms.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1602.04278v1",
            "date": "2016-02-13 03:30:34+00:00",
            "title": "Signer-independent Fingerspelling Recognition with Deep Neural Network Adaptation",
            "authors": [
                "Taehwan Kim",
                "Weiran Wang",
                "Hao Tang",
                "Karen Livescu"
            ],
            "abstract": "We study the problem of recognition of fingerspelled letter sequences in\nAmerican Sign Language in a signer-independent setting. Fingerspelled sequences\nare both challenging and important to recognize, as they are used for many\ncontent words such as proper nouns and technical terms. Previous work has shown\nthat it is possible to achieve almost 90% accuracies on fingerspelling\nrecognition in a signer-dependent setting. However, the more realistic\nsigner-independent setting presents challenges due to significant variations\namong signers, coupled with the dearth of available training data. We\ninvestigate this problem with approaches inspired by automatic speech\nrecognition. We start with the best-performing approaches from prior work,\nbased on tandem models and segmental conditional random fields (SCRFs), with\nfeatures based on deep neural network (DNN) classifiers of letters and\nphonological features. Using DNN adaptation, we find that it is possible to\nbridge a large part of the gap between signer-dependent and signer-independent\nperformance. Using only about 115 transcribed words for adaptation from the\ntarget signer, we obtain letter accuracies of up to 82.7% with frame-level\nadaptation labels and 69.7% with only word labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.04375v2",
            "date": "2016-02-13 20:13:48+00:00",
            "title": "Science Question Answering using Instructional Materials",
            "authors": [
                "Mrinmaya Sachan",
                "Avinava Dubey",
                "Eric P. Xing"
            ],
            "abstract": "We provide a solution for elementary science test using instructional\nmaterials. We posit that there is a hidden structure that explains the\ncorrectness of an answer given the question and instructional materials and\npresent a unified max-margin framework that learns to find these hidden\nstructures (given a corpus of question-answer pairs and instructional\nmaterials), and uses what it learns to answer novel elementary science\nquestions. Our evaluation shows that our framework outperforms several strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.04709v1",
            "date": "2016-02-12 19:56:49+00:00",
            "title": "Identifying Structures in Social Conversations in NSCLC Patients through the Semi-Automatic extraction of Topical Taxonomies",
            "authors": [
                "Giancarlo Crocetti",
                "Amir A. Delay",
                "Fatemeh Seyedmendhi"
            ],
            "abstract": "The exploration of social conversations for addressing patient's needs is an\nimportant analytical task in which many scholarly publications are contributing\nto fill the knowledge gap in this area. The main difficulty remains the\ninability to turn such contributions into pragmatic processes the\npharmaceutical industry can leverage in order to generate insight from social\nmedia data, which can be considered as one of the most challenging source of\ninformation available today due to its sheer volume and noise. This study is\nbased on the work by Scott Spangler and Jeffrey Kreulen and applies it to\nidentify structure in social media through the extraction of a topical taxonomy\nable to capture the latent knowledge in social conversations in health-related\nsites. The mechanism for automatically identifying and generating a taxonomy\nfrom social conversations is developed and pressured tested using public data\nfrom media sites focused on the needs of cancer patients and their families.\nMoreover, a novel method for generating the category's label and the\ndetermination of an optimal number of categories is presented which extends\nScott and Jeffrey's research in a meaningful way. We assume the reader is\nfamiliar with taxonomies, what they are and how they are used.",
            "journal ref": "Journal of Engineering Research and Applications, Vol. 6, Issue 1,\n  January 2016, pp.20-26",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1602.04930v1",
            "date": "2016-02-16 07:43:29+00:00",
            "title": "Generalized minimum dominating set and application in automatic text summarization",
            "authors": [
                "Yi-Zhi Xu",
                "Hai-Jun Zhou"
            ],
            "abstract": "For a graph formed by vertices and weighted edges, a generalized minimum\ndominating set (MDS) is a vertex set of smallest cardinality such that the\nsummed weight of edges from each outside vertex to vertices in this set is\nequal to or larger than certain threshold value. This generalized MDS problem\nreduces to the conventional MDS problem in the limiting case of all the edge\nweights being equal to the threshold value. We treat the generalized MDS\nproblem in the present paper by a replica-symmetric spin glass theory and\nderive a set of belief-propagation equations. As a practical application we\nconsider the problem of extracting a set of sentences that best summarize a\ngiven input text document. We carry out a preliminary test of the statistical\nphysics-inspired method to this automatic text summarization problem.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1602.06025v1",
            "date": "2016-02-19 02:07:20+00:00",
            "title": "Spectral Learning for Supervised Topic Models",
            "authors": [
                "Yong Ren",
                "Yining Wang",
                "Jun Zhu"
            ],
            "abstract": "Supervised topic models simultaneously model the latent topic structure of\nlarge collections of documents and a response variable associated with each\ndocument. Existing inference methods are based on variational approximation or\nMonte Carlo sampling, which often suffers from the local minimum defect.\nSpectral methods have been applied to learn unsupervised topic models, such as\nlatent Dirichlet allocation (LDA), with provable guarantees. This paper\ninvestigates the possibility of applying spectral methods to recover the\nparameters of supervised LDA (sLDA). We first present a two-stage spectral\nmethod, which recovers the parameters of LDA followed by a power update method\nto recover the regression model parameters. Then, we further present a\nsingle-phase spectral algorithm to jointly recover the topic distribution\nmatrix as well as the regression weights. Our spectral algorithms are provably\ncorrect and computationally efficient. We prove a sample complexity bound for\neach algorithm and subsequently derive a sufficient condition for the\nidentifiability of sLDA. Thorough experiments on synthetic and real-world\ndatasets verify the theory and demonstrate the practical effectiveness of the\nspectral algorithms. In fact, our results on a large-scale review rating\ndataset demonstrate that our single-phase spectral algorithm alone gets\ncomparable or even better performance than state-of-the-art methods, while\nprevious work on spectral methods has rarely reported such promising\nperformance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1602.06727v3",
            "date": "2016-02-22 11:11:04+00:00",
            "title": "Improving Trajectory Modelling for DNN-based Speech Synthesis by using Stacked Bottleneck Features and Minimum Generation Error Training",
            "authors": [
                "Zhizheng Wu",
                "Simon King"
            ],
            "abstract": "We propose two novel techniques --- stacking bottleneck features and minimum\ngeneration error training criterion --- to improve the performance of deep\nneural network (DNN)-based speech synthesis. The techniques address the related\nissues of frame-by-frame independence and ignorance of the relationship between\nstatic and dynamic features, within current typical DNN-based synthesis\nframeworks. Stacking bottleneck features, which are an acoustically--informed\nlinguistic representation, provides an efficient way to include more detailed\nlinguistic context at the input. The minimum generation error training\ncriterion minimises overall output trajectory error across an utterance, rather\nthan minimising the error per frame independently, and thus takes into account\nthe interaction between static and dynamic features. The two techniques can be\neasily combined to further improve performance. We present both objective and\nsubjective results that demonstrate the effectiveness of the proposed\ntechniques. The subjective results show that combining the two techniques leads\nto significantly more natural synthetic speech than from conventional DNN or\nlong short-term memory (LSTM) recurrent neural network (RNN) systems.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1602.06967v1",
            "date": "2016-02-22 21:22:49+00:00",
            "title": "Blind score normalization method for PLDA based speaker recognition",
            "authors": [
                "Danila Doroshin",
                "Nikolay Lubimov",
                "Marina Nastasenko",
                "Mikhail Kotov"
            ],
            "abstract": "Probabilistic Linear Discriminant Analysis (PLDA) has become state-of-the-art\nmethod for modeling $i$-vector space in speaker recognition task. However the\nperformance degradation is observed if enrollment data size differs from one\nspeaker to another. This paper presents a solution to such problem by\nintroducing new PLDA scoring normalization technique. Normalization parameters\nare derived in a blind way, so that, unlike traditional \\textit{ZT-norm}, no\nextra development data is required. Moreover, proposed method has shown to be\noptimal in terms of detection cost function. The experiments conducted on NIST\nSRE 2014 database demonstrate an improved accuracy in a mixed enrollment number\ncondition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07291v1",
            "date": "2016-02-23 20:39:40+00:00",
            "title": "The IBM 2016 Speaker Recognition System",
            "authors": [
                "Seyed Omid Sadjadi",
                "Sriram Ganapathy",
                "Jason W. Pelecanos"
            ],
            "abstract": "In this paper we describe the recent advancements made in the IBM i-vector\nspeaker recognition system for conversational speech. In particular, we\nidentify key techniques that contribute to significant improvements in\nperformance of our system, and quantify their contributions. The techniques\ninclude: 1) a nearest-neighbor discriminant analysis (NDA) approach that is\nformulated to alleviate some of the limitations associated with the\nconventional linear discriminant analysis (LDA) that assumes Gaussian\nclass-conditional distributions, 2) the application of speaker- and\nchannel-adapted features, which are derived from an automatic speech\nrecognition (ASR) system, for speaker recognition, and 3) the use of a deep\nneural network (DNN) acoustic model with a large number of output units (~10k\nsenones) to compute the frame-level soft alignments required in the i-vector\nestimation process. We evaluate these techniques on the NIST 2010 speaker\nrecognition evaluation (SRE) extended core conditions involving telephone and\nmicrophone trials. Experimental results indicate that: 1) the NDA is more\neffective (up to 35% relative improvement in terms of EER) than the traditional\nparametric LDA for speaker recognition, 2) when compared to raw acoustic\nfeatures (e.g., MFCCs), the ASR speaker-adapted features provide gains in\nspeaker recognition performance, and 3) increasing the number of output units\nin the DNN acoustic model (i.e., increasing the senone set size from 2k to 10k)\nprovides consistent improvements in performance (for example from 37% to 57%\nrelative EER gains over our baseline GMM i-vector system). To our knowledge,\nresults reported in this paper represent the best performances published to\ndate on the NIST SRE 2010 extended core tasks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1602.07393v1",
            "date": "2016-02-24 04:32:34+00:00",
            "title": "Domain Specific Author Attribution Based on Feedforward Neural Network Language Models",
            "authors": [
                "Zhenhao Ge",
                "Yufang Sun"
            ],
            "abstract": "Authorship attribution refers to the task of automatically determining the\nauthor based on a given sample of text. It is a problem with a long history and\nhas a wide range of application. Building author profiles using language models\nis one of the most successful methods to automate this task. New language\nmodeling methods based on neural networks alleviate the curse of dimensionality\nand usually outperform conventional N-gram methods. However, there have not\nbeen much research applying them to authorship attribution. In this paper, we\npresent a novel setup of a Neural Network Language Model (NNLM) and apply it to\na database of text samples from different authors. We investigate how the NNLM\nperforms on a task with moderate author set size and relatively limited\ntraining and test data, and how the topics of the text samples affect the\naccuracy. NNLM achieves nearly 2.5% reduction in perplexity, a measurement of\nfitness of a trained language model to the test data. Given 5 random test\nsentences, it also increases the author classification accuracy by 3.43% on\naverage, compared with the N-gram methods using SRILM tools. An open source\nimplementation of our methodology is freely available at\nhttps://github.com/zge/authorship-attribution/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.07807v2",
            "date": "2016-02-25 05:49:36+00:00",
            "title": "Data Cleaning for XML Electronic Dictionaries via Statistical Anomaly Detection",
            "authors": [
                "Michael Bloodgood",
                "Benjamin Strauss"
            ],
            "abstract": "Many important forms of data are stored digitally in XML format. Errors can\noccur in the textual content of the data in the fields of the XML. Fixing these\nerrors manually is time-consuming and expensive, especially for large amounts\nof data. There is increasing interest in the research, development, and use of\nautomated techniques for assisting with data cleaning. Electronic dictionaries\nare an important form of data frequently stored in XML format that frequently\nhave errors introduced through a mixture of manual typographical entry errors\nand optical character recognition errors. In this paper we describe methods for\nflagging statistical anomalies as likely errors in electronic dictionaries\nstored in XML format. We describe six systems based on different sources of\ninformation. The systems detect errors using various signals in the data\nincluding uncommon characters, text length, character-based language models,\nword-based language models, tied-field length ratios, and tied-field\ntransliteration models. Four of the systems detect errors based on expectations\nautomatically inferred from content within elements of a single field type. We\ncall these single-field systems. Two of the systems detect errors based on\ncorrespondence expectations automatically inferred from content within elements\nof multiple related field types. We call these tied-field systems. For each\nsystem, we provide an intuitive analysis of the type of error that it is\nsuccessful at detecting. Finally, we describe two larger-scale evaluations\nusing crowdsourcing with Amazon's Mechanical Turk platform and using the\nannotations of a domain expert. The evaluations consistently show that the\nsystems are useful for improving the efficiency with which errors in XML\nelectronic dictionaries can be detected.",
            "journal ref": "In Proceedings of the 2016 IEEE Tenth International Conference on\n  Semantic Computing (ICSC), pages 79-86, Laguna Hills, CA, USA, February 2016.\n  IEEE",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1602.08128v1",
            "date": "2016-02-25 21:48:56+00:00",
            "title": "PCA Method for Automated Detection of Mispronounced Words",
            "authors": [
                "Zhenhao Ge",
                "Sudhendu R. Sharma",
                "Mark J. T. Smith"
            ],
            "abstract": "This paper presents a method for detecting mispronunciations with the aim of\nimproving Computer Assisted Language Learning (CALL) tools used by foreign\nlanguage learners. The algorithm is based on Principle Component Analysis\n(PCA). It is hierarchical with each successive step refining the estimate to\nclassify the test word as being either mispronounced or correct. Preprocessing\nbefore detection, like normalization and time-scale modification, is\nimplemented to guarantee uniformity of the feature vectors input to the\ndetection system. The performance using various features including spectrograms\nand Mel-Frequency Cepstral Coefficients (MFCCs) are compared and evaluated.\nBest results were obtained using MFCCs, achieving up to 99% accuracy in word\nverification and 93% in native/non-native classification. Compared with Hidden\nMarkov Models (HMMs) which are used pervasively in recognition application,\nthis particular approach is computational efficient and effective when training\ndata is limited.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1602.08761v2",
            "date": "2016-02-28 19:44:57+00:00",
            "title": "Resource Constrained Structured Prediction",
            "authors": [
                "Tolga Bolukbasi",
                "Kai-Wei Chang",
                "Joseph Wang",
                "Venkatesh Saligrama"
            ],
            "abstract": "We study the problem of structured prediction under test-time budget\nconstraints. We propose a novel approach applicable to a wide range of\nstructured prediction problems in computer vision and natural language\nprocessing. Our approach seeks to adaptively generate computationally costly\nfeatures during test-time in order to reduce the computational cost of\nprediction while maintaining prediction performance. We show that training the\nadaptive feature generation system can be reduced to a series of structured\nlearning problems, resulting in efficient training using existing structured\nlearning algorithms. This framework provides theoretical justification for\nseveral existing heuristic approaches found in literature. We evaluate our\nproposed adaptive system on two structured prediction tasks, optical character\nrecognition (OCR) and dependency parsing and show strong performance in\nreduction of the feature costs without degrading accuracy.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1609.00464v2",
            "date": "2016-09-02 04:26:54+00:00",
            "title": "The Semantic Knowledge Graph: A compact, auto-generated model for real-time traversal and ranking of any relationship within a domain",
            "authors": [
                "Trey Grainger",
                "Khalifeh AlJadda",
                "Mohammed Korayem",
                "Andries Smith"
            ],
            "abstract": "This paper describes a new kind of knowledge representation and mining system\nwhich we are calling the Semantic Knowledge Graph. At its heart, the Semantic\nKnowledge Graph leverages an inverted index, along with a complementary\nuninverted index, to represent nodes (terms) and edges (the documents within\nintersecting postings lists for multiple terms/nodes). This provides a layer of\nindirection between each pair of nodes and their corresponding edge, enabling\nedges to materialize dynamically from underlying corpus statistics. As a\nresult, any combination of nodes can have edges to any other nodes materialize\nand be scored to reveal latent relationships between the nodes. This provides\nnumerous benefits: the knowledge graph can be built automatically from a\nreal-world corpus of data, new nodes - along with their combined edges - can be\ninstantly materialized from any arbitrary combination of preexisting nodes\n(using set operations), and a full model of the semantic relationships between\nall entities within a domain can be represented and dynamically traversed using\na highly compact representation of the graph. Such a system has widespread\napplications in areas as diverse as knowledge modeling and reasoning, natural\nlanguage processing, anomaly detection, data cleansing, semantic search,\nanalytics, data classification, root cause analysis, and recommendations\nsystems. The main contribution of this paper is the introduction of a novel\nsystem - the Semantic Knowledge Graph - which is able to dynamically discover\nand score interesting relationships between any arbitrary combination of\nentities (words, phrases, or extracted concepts) through dynamically\nmaterializing nodes and edges from a compact graphical representation built\nautomatically from a corpus of data representative of a knowledge domain.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1609.00514v1",
            "date": "2016-09-02 09:21:33+00:00",
            "title": "On Horizontal and Vertical Separation in Hierarchical Text Classification",
            "authors": [
                "Mostafa Dehghani",
                "Hosein Azarbonyad",
                "Jaap Kamps",
                "Maarten Marx"
            ],
            "abstract": "Hierarchy is a common and effective way of organizing data and representing\ntheir relationships at different levels of abstraction. However, hierarchical\ndata dependencies cause difficulties in the estimation of \"separable\" models\nthat can distinguish between the entities in the hierarchy. Extracting\nseparable models of hierarchical entities requires us to take their relative\nposition into account and to consider the different types of dependencies in\nthe hierarchy. In this paper, we present an investigation of the effect of\nseparability in text-based entity classification and argue that in hierarchical\nclassification, a separation property should be established between entities\nnot only in the same layer, but also in different layers. Our main findings are\nthe followings. First, we analyse the importance of separability on the data\nrepresentation in the task of classification and based on that, we introduce a\n\"Strong Separation Principle\" for optimizing expected effectiveness of\nclassifiers decision based on separation property. Second, we present\nHierarchical Significant Words Language Models (HSWLM) which capture all, and\nonly, the essential features of hierarchical entities according to their\nrelative position in the hierarchy resulting in horizontally and vertically\nseparable models. Third, we validate our claims on real-world data and\ndemonstrate that how HSWLM improves the accuracy of classification and how it\nprovides transferable models over time. Although discussions in this paper\nfocus on the classification problem, the models are applicable to any\ninformation access tasks on data that has, or can be mapped to, a hierarchical\nstructure.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1609.00718v1",
            "date": "2016-08-31 15:43:27+00:00",
            "title": "Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level",
            "authors": [
                "Rie Johnson",
                "Tong Zhang"
            ],
            "abstract": "This paper reports the performances of shallow word-level convolutional\nneural networks (CNN), our earlier work (2015), on the eight datasets with\nrelatively large training data that were used for testing the very deep\ncharacter-level CNN in Conneau et al. (2016). Our findings are as follows. The\nshallow word-level CNNs achieve better error rates than the error rates\nreported in Conneau et al., though the results should be interpreted with some\nconsideration due to the unique pre-processing of Conneau et al. The shallow\nword-level CNN uses more parameters and therefore requires more storage than\nthe deep character-level CNN; however, the shallow word-level CNN computes much\nfaster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.01962v1",
            "date": "2016-09-07 12:33:02+00:00",
            "title": "Using Gaussian Processes for Rumour Stance Classification in Social Media",
            "authors": [
                "Michal Lukasik",
                "Kalina Bontcheva",
                "Trevor Cohn",
                "Arkaitz Zubiaga",
                "Maria Liakata",
                "Rob Procter"
            ],
            "abstract": "Social media tend to be rife with rumours while new reports are released\npiecemeal during breaking news. Interestingly, one can mine multiple reactions\nexpressed by social media users in those situations, exploring their stance\ntowards rumours, ultimately enabling the flagging of highly disputed rumours as\nbeing potentially false. In this work, we set out to develop an automated,\nsupervised classifier that uses multi-task learning to classify the stance\nexpressed in each individual tweet in a rumourous conversation as either\nsupporting, denying or questioning the rumour. Using a classifier based on\nGaussian Processes, and exploring its effectiveness on two datasets with very\ndifferent characteristics and varying distributions of stances, we show that\nour approach consistently outperforms competitive baseline classifiers. Our\nclassifier is especially effective in estimating the distribution of different\ntypes of stance associated with a given rumour, which we set forth as a desired\ncharacteristic for a rumour-tracking system that will warn both ordinary users\nof Twitter and professional news practitioners when a rumour is being rebutted.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02075v1",
            "date": "2016-09-07 17:09:10+00:00",
            "title": "The Social Dynamics of Language Change in Online Networks",
            "authors": [
                "Rahul Goel",
                "Sandeep Soni",
                "Naman Goyal",
                "John Paparrizos",
                "Hanna Wallach",
                "Fernando Diaz",
                "Jacob Eisenstein"
            ],
            "abstract": "Language change is a complex social phenomenon, revealing pathways of\ncommunication and sociocultural influence. But, while language change has long\nbeen a topic of study in sociolinguistics, traditional linguistic research\nmethods rely on circumstantial evidence, estimating the direction of change\nfrom differences between older and younger speakers. In this paper, we use a\ndata set of several million Twitter users to track language changes in\nprogress. First, we show that language change can be viewed as a form of social\ninfluence: we observe complex contagion for phonetic spellings and \"netspeak\"\nabbreviations (e.g., lol), but not for older dialect markers from spoken\nlanguage. Next, we test whether specific types of social network connections\nare more influential than others, using a parametric Hawkes process model. We\nfind that tie strength plays an important role: densely embedded social ties\nare significantly better conduits of linguistic influence. Geographic locality\nappears to play a more limited role: we find relatively little evidence to\nsupport the hypothesis that individuals are more influenced by geographically\nlocal social ties, even in their usage of geographical dialect markers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.02082v1",
            "date": "2016-08-04 10:11:24+00:00",
            "title": "An improved uncertainty decoding scheme with weighted samples for DNN-HMM hybrid systems",
            "authors": [
                "Christian Huemmer",
                "Ram\u00f3n Fern\u00e1ndez Astudillo",
                "Walter Kellermann"
            ],
            "abstract": "In this paper, we advance a recently-proposed uncertainty decoding scheme for\nDNN-HMM (deep neural network - hidden Markov model) hybrid systems. This\nnumerical sampling concept averages DNN outputs produced by a finite set of\nfeature samples (drawn from a probabilistic distortion model) to approximate\nthe posterior likelihoods of the context-dependent HMM states. As main\ninnovation, we propose a weighted DNN-output averaging based on a minimum\nclassification error criterion and apply it to a probabilistic distortion model\nfor spatial diffuseness features. The experimental evaluation is performed on\nthe 8-channel REVERB Challenge task using a DNN-HMM hybrid system with\nmultichannel front-end signal enhancement. We show that the recognition\naccuracy of the DNN-HMM hybrid system improves by incorporating uncertainty\ndecoding based on random sampling and that the proposed weighted DNN-output\naveraging further reduces the word error rate scores.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.02116v2",
            "date": "2016-09-07 19:05:42+00:00",
            "title": "Ask the GRU: Multi-Task Learning for Deep Text Recommendations",
            "authors": [
                "Trapit Bansal",
                "David Belanger",
                "Andrew McCallum"
            ],
            "abstract": "In a variety of application domains the content to be recommended to users is\nassociated with text. This includes research papers, movies with associated\nplot summaries, news articles, blog posts, etc. Recommendation approaches based\non latent factor models can be extended naturally to leverage text by employing\nan explicit mapping from text to factors. This enables recommendations for new,\nunseen content, and may generalize better, since the factors for all items are\nproduced by a compactly-parametrized model. Previous work has used topic models\nor averages of word embeddings for this mapping. In this paper we present a\nmethod leveraging deep recurrent neural networks to encode the text sequence\ninto a latent vector, specifically gated recurrent units (GRUs) trained\nend-to-end on the collaborative filtering task. For the task of scientific\npaper recommendation, this yields models with significantly higher accuracy. In\ncold-start scenarios, we beat the previous state-of-the-art, all of which\nignore word order. Performance is further improved by multi-task learning,\nwhere the text encoder network is trained for a combination of content\nrecommendation and item metadata prediction. This regularizes the collaborative\nfiltering model, ameliorating the problem of sparsity of the observed rating\nmatrix.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1609.03193v2",
            "date": "2016-09-11 18:56:53+00:00",
            "title": "Wav2Letter: an End-to-End ConvNet-based Speech Recognition System",
            "authors": [
                "Ronan Collobert",
                "Christian Puhrsch",
                "Gabriel Synnaeve"
            ],
            "abstract": "This paper presents a simple end-to-end model for speech recognition,\ncombining a convolutional network based acoustic model and a graph decoding. It\nis trained to output letters, with transcribed speech, without the need for\nforce alignment of phonemes. We introduce an automatic segmentation criterion\nfor training from sequence annotation without alignment that is on par with CTC\nwhile being simpler. We show competitive results in word error rate on the\nLibrispeech corpus with MFCC features, and promising results from raw waveform.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.03207v2",
            "date": "2016-09-11 19:59:18+00:00",
            "title": "Multiplex lexical networks reveal patterns in early word acquisition in children",
            "authors": [
                "Massimo Stella",
                "Nicole M. Beckage",
                "Markus Brede"
            ],
            "abstract": "Network models of language have provided a way of linking cognitive processes\nto the structure and connectivity of language. However, one shortcoming of\ncurrent approaches is focusing on only one type of linguistic relationship at a\ntime, missing the complex multi-relational nature of language. In this work, we\novercome this limitation by modelling the mental lexicon of English-speaking\ntoddlers as a multiplex lexical network, i.e. a multi-layered network where\nN=529 words/nodes are connected according to four types of relationships: (i)\nfree associations, (ii) feature sharing, (iii) co-occurrence, and (iv)\nphonological similarity. We provide analysis of the topology of the resulting\nmultiplex and then proceed to evaluate single layers as well as the full\nmultiplex structure on their ability to predict empirically observed age of\nacquisition data of English speaking toddlers. We find that the emerging\nmultiplex network topology is an important proxy of the cognitive processes of\nacquisition, capable of capturing emergent lexicon structure. In fact, we show\nthat the multiplex topology is fundamentally more powerful than individual\nlayers in predicting the ordering with which words are acquired. Furthermore,\nmultiplex analysis allows for a quantification of distinct phases of lexical\nacquisition in early learners: while initially all the multiplex layers\ncontribute to word learning, after about month 23 free associations take the\nlead in driving word acquisition.",
            "journal ref": "Scientific Reports 7, Article number: 46730 (2017)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1609.03777v2",
            "date": "2016-09-13 11:41:48+00:00",
            "title": "Character-Level Language Modeling with Hierarchical Recurrent Neural Networks",
            "authors": [
                "Kyuyeon Hwang",
                "Wonyong Sung"
            ],
            "abstract": "Recurrent neural network (RNN) based character-level language models (CLMs)\nare extremely useful for modeling out-of-vocabulary words by nature. However,\ntheir performance is generally much worse than the word-level language models\n(WLMs), since CLMs need to consider longer history of tokens to properly\npredict the next one. We address this problem by proposing hierarchical RNN\narchitectures, which consist of multiple modules with different timescales.\nDespite the multi-timescale structures, the input and output layers operate\nwith the character-level clock, which allows the existing RNN CLM training\napproaches to be directly applicable without any modifications. Our CLM models\nshow better perplexity than Kneser-Ney (KN) 5-gram WLMs on the One Billion Word\nBenchmark with only 2% of parameters. Also, we present real-time\ncharacter-level end-to-end speech recognition examples on the Wall Street\nJournal (WSJ) corpus, where replacing traditional mono-clock RNN CLMs with the\nproposed models results in better recognition accuracies even though the number\nof parameters are reduced to 30%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.04628v1",
            "date": "2016-09-15 13:23:43+00:00",
            "title": "Context Aware Nonnegative Matrix Factorization Clustering",
            "authors": [
                "Rocco Tripodi",
                "Sebastiano Vascon",
                "Marcello Pelillo"
            ],
            "abstract": "In this article we propose a method to refine the clustering results obtained\nwith the nonnegative matrix factorization (NMF) technique, imposing consistency\nconstraints on the final labeling of the data. The research community focused\nits effort on the initialization and on the optimization part of this method,\nwithout paying attention to the final cluster assignments. We propose a game\ntheoretic framework in which each object to be clustered is represented as a\nplayer, which has to choose its cluster membership. The information obtained\nwith NMF is used to initialize the strategy space of the players and a weighted\ngraph is used to model the interactions among the players. These interactions\nallow the players to choose a cluster which is coherent with the clusters\nchosen by similar players, a property which is not guaranteed by NMF, since it\nproduces a soft clustering of the data. The results on common benchmarks show\nthat our model is able to improve the performances of many NMF formulations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1609.04904v2",
            "date": "2016-09-16 03:45:15+00:00",
            "title": "Long-Term Trends in the Public Perception of Artificial Intelligence",
            "authors": [
                "Ethan Fast",
                "Eric Horvitz"
            ],
            "abstract": "Analyses of text corpora over time can reveal trends in beliefs, interest,\nand sentiment about a topic. We focus on views expressed about artificial\nintelligence (AI) in the New York Times over a 30-year period. General\ninterest, awareness, and discussion about AI has waxed and waned since the\nfield was founded in 1956. We present a set of measures that captures levels of\nengagement, measures of pessimism and optimism, the prevalence of specific\nhopes and concerns, and topics that are linked to discussions about AI over\ndecades. We find that discussion of AI has increased sharply since 2009, and\nthat these discussions have been consistently more optimistic than pessimistic.\nHowever, when we examine specific concerns, we find that worries of loss of\ncontrol of AI, ethical concerns for AI, and the negative impact of AI on work\nhave grown in recent years. We also find that hopes for AI in healthcare and\neducation have increased over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.04938v2",
            "date": "2016-09-16 08:14:50+00:00",
            "title": "Image-to-Markup Generation with Coarse-to-Fine Attention",
            "authors": [
                "Yuntian Deng",
                "Anssi Kanervisto",
                "Jeffrey Ling",
                "Alexander M. Rush"
            ],
            "abstract": "We present a neural encoder-decoder model to convert images into\npresentational markup based on a scalable coarse-to-fine attention mechanism.\nOur method is evaluated in the context of image-to-LaTeX generation, and we\nintroduce a new dataset of real-world rendered mathematical expressions paired\nwith LaTeX markup. We show that unlike neural OCR techniques using CTC-based\nmodels, attention-based approaches can tackle this non-standard OCR task. Our\napproach outperforms classical mathematical OCR systems by a large margin on\nin-domain rendered data, and, with pretraining, also performs well on\nout-of-domain handwritten data. To reduce the inference complexity associated\nwith the attention-based approaches, we introduce a new coarse-to-fine\nattention layer that selects a support region before applying attention.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1609.05600v2",
            "date": "2016-09-19 05:21:36+00:00",
            "title": "Graph-Structured Representations for Visual Question Answering",
            "authors": [
                "Damien Teney",
                "Lingqiao Liu",
                "Anton van den Hengel"
            ],
            "abstract": "This paper proposes to improve visual question answering (VQA) with\nstructured representations of both scene contents and questions. A key\nchallenge in VQA is to require joint reasoning over the visual and text\ndomains. The predominant CNN/LSTM-based approach to VQA is limited by\nmonolithic vector representations that largely ignore structure in the scene\nand in the form of the question. CNN feature vectors cannot effectively capture\nsituations as simple as multiple object instances, and LSTMs process questions\nas series of words, which does not reflect the true complexity of language\nstructure. We instead propose to build graphs over the scene objects and over\nthe question words, and we describe a deep neural network that exploits the\nstructure in these representations. This shows significant benefit over the\nsequential processing of LSTMs. The overall efficacy of our approach is\ndemonstrated by significant improvements over the state-of-the-art, from 71.2%\nto 74.4% in accuracy on the \"abstract scenes\" multiple-choice benchmark, and\nfrom 34.7% to 39.1% in accuracy over pairs of \"balanced\" scenes, i.e. images\nwith fine-grained differences and opposite yes/no answers to a same question.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1609.06578v1",
            "date": "2016-09-21 14:25:23+00:00",
            "title": "Twitter Opinion Topic Model: Extracting Product Opinions from Tweets by Leveraging Hashtags and Sentiment Lexicon",
            "authors": [
                "Kar Wai Lim",
                "Wray Buntine"
            ],
            "abstract": "Aspect-based opinion mining is widely applied to review data to aggregate or\nsummarize opinions of a product, and the current state-of-the-art is achieved\nwith Latent Dirichlet Allocation (LDA)-based model. Although social media data\nlike tweets are laden with opinions, their \"dirty\" nature (as natural language)\nhas discouraged researchers from applying LDA-based opinion model for product\nreview mining. Tweets are often informal, unstructured and lacking labeled data\nsuch as categories and ratings, making it challenging for product opinion\nmining. In this paper, we propose an LDA-based opinion model named Twitter\nOpinion Topic Model (TOTM) for opinion mining and sentiment analysis. TOTM\nleverages hashtags, mentions, emoticons and strong sentiment words that are\npresent in tweets in its discovery process. It improves opinion prediction by\nmodeling the target-opinion interaction directly, thus discovering target\nspecific opinion words, neglected in existing approaches. Moreover, we propose\na new formulation of incorporating sentiment prior information into a topic\nmodel, by utilizing an existing public sentiment lexicon. This is novel in that\nit learns and updates with the data. We conduct experiments on 9 million tweets\non electronic products, and demonstrate the improved performance of TOTM in\nboth quantitative evaluations and qualitative analysis. We show that\naspect-based opinion analysis on massive volume of tweets provides useful\nopinions on products.",
            "journal ref": "Proceedings of the 23rd ACM International Conference on\n  Information and Knowledge Management (CIKM), pp. 1319-1328. ACM. 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06616v2",
            "date": "2016-09-21 16:09:12+00:00",
            "title": "Gov2Vec: Learning Distributed Representations of Institutions and Their Legal Text",
            "authors": [
                "John J. Nay"
            ],
            "abstract": "We compare policy differences across institutions by embedding\nrepresentations of the entire legal corpus of each institution and the\nvocabulary shared across all corpora into a continuous vector space. We apply\nour method, Gov2Vec, to Supreme Court opinions, Presidential actions, and\nofficial summaries of Congressional bills. The model discerns meaningful\ndifferences between government branches. We also learn representations for more\nfine-grained word sources: individual Presidents and (2-year) Congresses. The\nsimilarities between learned representations of Congresses over time and\nsitting Presidents are negatively correlated with the bill veto rate, and the\ntemporal ordering of Presidents and Congresses was implicitly learned from only\ntext. With the resulting vectors we answer questions such as: how does Obama\nand the 113th House differ in addressing climate change and how does this vary\nfrom environmental or economic perspectives? Our work illustrates\nvector-arithmetic-based investigations of complex relationships between word\nsources based on their texts. We are extending this to create a more\ncomprehensive legal semantic map.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.06783v1",
            "date": "2016-09-22 00:10:16+00:00",
            "title": "Nonparametric Bayesian Topic Modelling with the Hierarchical Pitman-Yor Processes",
            "authors": [
                "Kar Wai Lim",
                "Wray Buntine",
                "Changyou Chen",
                "Lan Du"
            ],
            "abstract": "The Dirichlet process and its extension, the Pitman-Yor process, are\nstochastic processes that take probability distributions as a parameter. These\nprocesses can be stacked up to form a hierarchical nonparametric Bayesian\nmodel. In this article, we present efficient methods for the use of these\nprocesses in this hierarchical context, and apply them to latent variable\nmodels for text analytics. In particular, we propose a general framework for\ndesigning these Bayesian models, which are called topic models in the computer\nscience community. We then propose a specific nonparametric Bayesian topic\nmodel for modelling text from social media. We focus on tweets (posts on\nTwitter) in this article due to their ease of access. We find that our\nnonparametric model performs better than existing parametric models in both\ngoodness of fit and real world applications.",
            "journal ref": "International Journal of Approximate Reasoning, Volume 78, pp.\n  172-191. Elsevier. 2016",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1609.06791v1",
            "date": "2016-09-22 01:08:31+00:00",
            "title": "Twitter-Network Topic Model: A Full Bayesian Treatment for Social Network and Text Modeling",
            "authors": [
                "Kar Wai Lim",
                "Changyou Chen",
                "Wray Buntine"
            ],
            "abstract": "Twitter data is extremely noisy -- each tweet is short, unstructured and with\ninformal language, a challenge for current topic modeling. On the other hand,\ntweets are accompanied by extra information such as authorship, hashtags and\nthe user-follower network. Exploiting this additional information, we propose\nthe Twitter-Network (TN) topic model to jointly model the text and the social\nnetwork in a full Bayesian nonparametric way. The TN topic model employs the\nhierarchical Poisson-Dirichlet processes (PDP) for text modeling and a Gaussian\nprocess random function model for social network modeling. We show that the TN\ntopic model significantly outperforms several existing nonparametric models due\nto its flexibility. Moreover, the TN topic model enables additional informative\ninference such as authors' interests, hashtag analysis, as well as leading to\nfurther applications such as author recommendation, automatic topic labeling\nand hashtag suggestion. Note our general inference framework can readily be\napplied to other topic models with embedded PDP nodes.",
            "journal ref": "NIPS 2013 Topic Models: Computation, Application, and Evaluation,\n  pp. 1-5. Google Sites. 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08144v2",
            "date": "2016-09-26 19:59:55+00:00",
            "title": "Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation",
            "authors": [
                "Yonghui Wu",
                "Mike Schuster",
                "Zhifeng Chen",
                "Quoc V. Le",
                "Mohammad Norouzi",
                "Wolfgang Macherey",
                "Maxim Krikun",
                "Yuan Cao",
                "Qin Gao",
                "Klaus Macherey",
                "Jeff Klingner",
                "Apurva Shah",
                "Melvin Johnson",
                "Xiaobing Liu",
                "\u0141ukasz Kaiser",
                "Stephan Gouws",
                "Yoshikiyo Kato",
                "Taku Kudo",
                "Hideto Kazawa",
                "Keith Stevens",
                "George Kurian",
                "Nishant Patil",
                "Wei Wang",
                "Cliff Young",
                "Jason Smith",
                "Jason Riesa",
                "Alex Rudnick",
                "Oriol Vinyals",
                "Greg Corrado",
                "Macduff Hughes",
                "Jeffrey Dean"
            ],
            "abstract": "Neural Machine Translation (NMT) is an end-to-end learning approach for\nautomated translation, with the potential to overcome many of the weaknesses of\nconventional phrase-based translation systems. Unfortunately, NMT systems are\nknown to be computationally expensive both in training and in translation\ninference. Also, most NMT systems have difficulty with rare words. These issues\nhave hindered NMT's use in practical deployments and services, where both\naccuracy and speed are essential. In this work, we present GNMT, Google's\nNeural Machine Translation system, which attempts to address many of these\nissues. Our model consists of a deep LSTM network with 8 encoder and 8 decoder\nlayers using attention and residual connections. To improve parallelism and\ntherefore decrease training time, our attention mechanism connects the bottom\nlayer of the decoder to the top layer of the encoder. To accelerate the final\ntranslation speed, we employ low-precision arithmetic during inference\ncomputations. To improve handling of rare words, we divide words into a limited\nset of common sub-word units (\"wordpieces\") for both input and output. This\nmethod provides a good balance between the flexibility of \"character\"-delimited\nmodels and the efficiency of \"word\"-delimited models, naturally handles\ntranslation of rare words, and ultimately improves the overall accuracy of the\nsystem. Our beam search technique employs a length-normalization procedure and\nuses a coverage penalty, which encourages generation of an output sentence that\nis most likely to cover all the words in the source sentence. On the WMT'14\nEnglish-to-French and English-to-German benchmarks, GNMT achieves competitive\nresults to state-of-the-art. Using a human side-by-side evaluation on a set of\nisolated simple sentences, it reduces translation errors by an average of 60%\ncompared to Google's phrase-based production system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08194v1",
            "date": "2016-09-26 21:13:49+00:00",
            "title": "Online Segment to Segment Neural Transduction",
            "authors": [
                "Lei Yu",
                "Jan Buys",
                "Phil Blunsom"
            ],
            "abstract": "We introduce an online neural sequence to sequence model that learns to\nalternate between encoding and decoding segments of the input as it is read. By\nindependently tracking the encoding and decoding representations our algorithm\npermits exact polynomial marginalization of the latent segmentation during\ntraining, and during decoding beam search is employed to find the best\nalignment path together with the predicted output sequence. Our model tackles\nthe bottleneck of vanilla encoder-decoders that have to read and memorize the\nentire input sequence in their fixed-length hidden states before producing any\noutput. It is different from previous attentive models in that, instead of\ntreating the attention weights as output of a deterministic function, our model\nassigns attention weights to a sequential latent variable which can be\nmarginalized out and permits online generation. Experiments on abstractive\nsentence summarization and morphological inflection show significant\nperformance gains over the baseline encoder-decoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08337v1",
            "date": "2016-09-27 09:56:09+00:00",
            "title": "Multi-task Recurrent Model for True Multilingual Speech Recognition",
            "authors": [
                "Zhiyuan Tang",
                "Lantian Li",
                "Dong Wang"
            ],
            "abstract": "Research on multilingual speech recognition remains attractive yet\nchallenging. Recent studies focus on learning shared structures under the\nmulti-task paradigm, in particular a feature sharing structure. This approach\nhas been found effective to improve performance on each individual language.\nHowever, this approach is only useful when the deployed system supports just\none language. In a true multilingual scenario where multiple languages are\nallowed, performance will be significantly reduced due to the competition among\nlanguages in the decoding space. This paper presents a multi-task recurrent\nmodel that involves a multilingual speech recognition (ASR) component and a\nlanguage recognition (LR) component, and the ASR component is informed of the\nlanguage information by the LR component, leading to a language-aware\nrecognition. We tested the approach on an English-Chinese bilingual recognition\ntask. The results show that the proposed multi-task recurrent model can improve\nperformance of multilingual recognition systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08441v2",
            "date": "2016-09-27 13:46:55+00:00",
            "title": "Weakly Supervised PLDA Training",
            "authors": [
                "Lantian Li",
                "Yixiang Chen",
                "Dong Wang",
                "Chenghui Zhao"
            ],
            "abstract": "PLDA is a popular normalization approach for the i-vector model, and it has\ndelivered state-of-the-art performance in speaker verification. However, PLDA\ntraining requires a large amount of labelled development data, which is highly\nexpensive in most cases. We present a cheap PLDA training approach, which\nassumes that speakers in the same session can be easily separated, and speakers\nin different sessions are simply different. This results in `weak labels' which\nare not fully accurate but cheap, leading to a weak PLDA training.\n  Our experimental results on real-life large-scale telephony customer service\nachieves demonstrated that the weak training can offer good performance when\nhuman-labelled data are limited. More interestingly, the weak training can be\nemployed as a discriminative adaptation approach, which is more efficient than\nthe prevailing unsupervised method when human-labelled data are insufficient.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.08496v1",
            "date": "2016-09-27 15:26:07+00:00",
            "title": "Topic Modeling over Short Texts by Incorporating Word Embeddings",
            "authors": [
                "Jipeng Qiang",
                "Ping Chen",
                "Tong Wang",
                "Xindong Wu"
            ],
            "abstract": "Inferring topics from the overwhelming amount of short texts becomes a\ncritical but challenging task for many content analysis tasks, such as content\ncharactering, user interest profiling, and emerging topic detecting. Existing\nmethods such as probabilistic latent semantic analysis (PLSA) and latent\nDirichlet allocation (LDA) cannot solve this prob- lem very well since only\nvery limited word co-occurrence information is available in short texts. This\npaper studies how to incorporate the external word correlation knowledge into\nshort texts to improve the coherence of topic modeling. Based on recent results\nin word embeddings that learn se- mantically representations for words from a\nlarge corpus, we introduce a novel method, Embedding-based Topic Model (ETM),\nto learn latent topics from short texts. ETM not only solves the problem of\nvery limited word co-occurrence information by aggregating short texts into\nlong pseudo- texts, but also utilizes a Markov Random Field regularized model\nthat gives correlated words a better chance to be put into the same topic. The\nexperiments on real-world datasets validate the effectiveness of our model\ncomparing with the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08703v1",
            "date": "2016-09-27 23:10:42+00:00",
            "title": "Optimizing Neural Network Hyperparameters with Gaussian Processes for Dialog Act Classification",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee"
            ],
            "abstract": "Systems based on artificial neural networks (ANNs) have achieved\nstate-of-the-art results in many natural language processing tasks. Although\nANNs do not require manually engineered features, ANNs have many\nhyperparameters to be optimized. The choice of hyperparameters significantly\nimpacts models' performances. However, the ANN hyperparameters are typically\nchosen by manual, grid, or random search, which either requires expert\nexperiences or is computationally expensive. Recent approaches based on\nBayesian optimization using Gaussian processes (GPs) is a more systematic way\nto automatically pinpoint optimal or near-optimal machine learning\nhyperparameters. Using a previously published ANN model yielding\nstate-of-the-art results for dialog act classification, we demonstrate that\noptimizing hyperparameters using GP further improves the results, and reduces\nthe computational time by a factor of 4 compared to a random search. Therefore\nit is a useful technique for tuning ANN models to yield the best performances\nfor natural language processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.08789v3",
            "date": "2016-09-28 06:26:16+00:00",
            "title": "Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition",
            "authors": [
                "Zhiyuan Tang",
                "Ying Shi",
                "Dong Wang",
                "Yang Feng",
                "Shiyue Zhang"
            ],
            "abstract": "Recurrent neural networks (RNNs) have shown clear superiority in sequence\nmodeling, particularly the ones with gated units, such as long short-term\nmemory (LSTM) and gated recurrent unit (GRU). However, the dynamic properties\nbehind the remarkable performance remain unclear in many applications, e.g.,\nautomatic speech recognition (ASR). This paper employs visualization techniques\nto study the behavior of LSTM and GRU when performing speech recognition tasks.\nOur experiments show some interesting patterns in the gated memory, and some of\nthem have inspired simple yet effective modifications on the network structure.\nWe report two of such modifications: (1) lazy cell update in LSTM, and (2)\nshortcut connections for residual learning. Both modifications lead to more\ncomprehensible and powerful networks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1609.08843v1",
            "date": "2016-09-28 10:03:05+00:00",
            "title": "Hierarchical Memory Networks for Answer Selection on Unknown Words",
            "authors": [
                "Jiaming Xu",
                "Jing Shi",
                "Yiqun Yao",
                "Suncong Zheng",
                "Bo Xu",
                "Bo Xu"
            ],
            "abstract": "Recently, end-to-end memory networks have shown promising results on Question\nAnswering task, which encode the past facts into an explicit memory and perform\nreasoning ability by making multiple computational steps on the memory.\nHowever, memory networks conduct the reasoning on sentence-level memory to\noutput coarse semantic vectors and do not further take any attention mechanism\nto focus on words, which may lead to the model lose some detail information,\nespecially when the answers are rare or unknown words. In this paper, we\npropose a novel Hierarchical Memory Networks, dubbed HMN. First, we encode the\npast facts into sentence-level memory and word-level memory respectively. Then,\n(k)-max pooling is exploited following reasoning module on the sentence-level\nmemory to sample the (k) most relevant sentences to a question and feed these\nsentences into attention mechanism on the word-level memory to focus the words\nin the selected sentences. Finally, the prediction is jointly learned over the\noutputs of the sentence-level reasoning module and the word-level attention\nmechanism. The experimental results demonstrate that our approach successfully\nconducts answer selection on unknown words and achieves a better performance\nthan memory networks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1609.09188v1",
            "date": "2016-09-29 03:22:01+00:00",
            "title": "Topic Browsing for Research Papers with Hierarchical Latent Tree Analysis",
            "authors": [
                "Leonard K. M. Poon",
                "Nevin L. Zhang"
            ],
            "abstract": "Academic researchers often need to face with a large collection of research\npapers in the literature. This problem may be even worse for postgraduate\nstudents who are new to a field and may not know where to start. To address\nthis problem, we have developed an online catalog of research papers where the\npapers have been automatically categorized by a topic model. The catalog\ncontains 7719 papers from the proceedings of two artificial intelligence\nconferences from 2000 to 2015. Rather than the commonly used Latent Dirichlet\nAllocation, we use a recently proposed method called hierarchical latent tree\nanalysis for topic modeling. The resulting topic model contains a hierarchy of\ntopics so that users can browse the topics from the top level to the bottom\nlevel. The topic model contains a manageable number of general topics at the\ntop level and allows thousands of fine-grained topics at the bottom level. It\nalso can detect topics that have emerged recently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1609.09315v1",
            "date": "2016-09-29 12:20:13+00:00",
            "title": "Semantic Parsing with Semi-Supervised Sequential Autoencoders",
            "authors": [
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "G\u00e1bor Melis",
                "Edward Grefenstette",
                "Chris Dyer",
                "Wang Ling",
                "Phil Blunsom",
                "Karl Moritz Hermann"
            ],
            "abstract": "We present a novel semi-supervised approach for sequence transduction and\napply it to semantic parsing. The unsupervised component is based on a\ngenerative model in which latent sentences generate the unpaired logical forms.\nWe apply this method to a number of semantic parsing tasks focusing on domains\nwith limited access to labelled training data and extend those datasets with\nsynthetically generated logical forms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00219v1",
            "date": "2016-10-02 03:35:11+00:00",
            "title": "Text Network Exploration via Heterogeneous Web of Topics",
            "authors": [
                "Junxian He",
                "Ying Huang",
                "Changfeng Liu",
                "Jiaming Shen",
                "Yuting Jia",
                "Xinbing Wang"
            ],
            "abstract": "A text network refers to a data type that each vertex is associated with a\ntext document and the relationship between documents is represented by edges.\nThe proliferation of text networks such as hyperlinked webpages and academic\ncitation networks has led to an increasing demand for quickly developing a\ngeneral sense of a new text network, namely text network exploration. In this\npaper, we address the problem of text network exploration through constructing\na heterogeneous web of topics, which allows people to investigate a text\nnetwork associating word level with document level. To achieve this, a\nprobabilistic generative model for text and links is proposed, where three\ndifferent relationships in the heterogeneous topic web are quantified. We also\ndevelop a prototype demo system named TopicAtlas to exhibit such heterogeneous\ntopic web, and demonstrate how this system can facilitate the task of text\nnetwork exploration. Extensive qualitative analyses are included to verify the\neffectiveness of this heterogeneous topic web. Besides, we validate our model\non real-life text networks, showing that it preserves good performance on\nobjective evaluation metrics.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1610.00369v2",
            "date": "2016-10-02 23:45:23+00:00",
            "title": "Sentiment Analysis on Bangla and Romanized Bangla Text (BRBT) using Deep Recurrent models",
            "authors": [
                "A. Hassan",
                "M. R. Amin",
                "N. Mohammed",
                "A. K. A. Azad"
            ],
            "abstract": "Sentiment Analysis (SA) is an action research area in the digital age. With\nrapid and constant growth of online social media sites and services, and the\nincreasing amount of textual data such as - statuses, comments, reviews etc.\navailable in them, application of automatic SA is on the rise. However, most of\nthe research works on SA in natural language processing (NLP) are based on\nEnglish language. Despite being the sixth most widely spoken language in the\nworld, Bangla still does not have a large and standard dataset. Because of\nthis, recent research works in Bangla have failed to produce results that can\nbe both comparable to works done by others and reusable as stepping stones for\nfuture researchers to progress in this field. Therefore, we first tried to\nprovide a textual dataset - that includes not just Bangla, but Romanized Bangla\ntexts as well, is substantial, post-processed and multiple validated, ready to\nbe used in SA experiments. We tested this dataset in Deep Recurrent model,\nspecifically, Long Short Term Memory (LSTM), using two types of loss functions\n- binary crossentropy and categorical crossentropy, and also did some\nexperimental pre-training by using data from one validation to pre-train the\nother and vice versa. Lastly, we documented the results along with some\nanalysis on them, which were promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00520v1",
            "date": "2016-10-03 12:52:26+00:00",
            "title": "Semi-supervised Learning with Sparse Autoencoders in Phone Classification",
            "authors": [
                "Akash Kumar Dhaka",
                "Giampiero Salvi"
            ],
            "abstract": "We propose the application of a semi-supervised learning method to improve\nthe performance of acoustic modelling for automatic speech recognition based on\ndeep neural net- works. As opposed to unsupervised initialisation followed by\nsupervised fine tuning, our method takes advantage of both unlabelled and\nlabelled data simultaneously through mini- batch stochastic gradient descent.\nWe tested the method with varying proportions of labelled vs unlabelled\nobservations in frame-based phoneme classification on the TIMIT database. Our\nexperiments show that the method outperforms standard supervised training for\nan equal amount of labelled data and provides competitive error rates compared\nto state-of-the-art graph-based semi-supervised learning techniques.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1610.00552v1",
            "date": "2016-09-30 10:44:32+00:00",
            "title": "FPGA-Based Low-Power Speech Recognition with Recurrent Neural Networks",
            "authors": [
                "Minjae Lee",
                "Kyuyeon Hwang",
                "Jinhwan Park",
                "Sungwook Choi",
                "Sungho Shin",
                "Wonyong Sung"
            ],
            "abstract": "In this paper, a neural network based real-time speech recognition (SR)\nsystem is developed using an FPGA for very low-power operation. The implemented\nsystem employs two recurrent neural networks (RNNs); one is a\nspeech-to-character RNN for acoustic modeling (AM) and the other is for\ncharacter-level language modeling (LM). The system also employs a statistical\nword-level LM to improve the recognition accuracy. The results of the AM, the\ncharacter-level LM, and the word-level LM are combined using a fairly simple\nN-best search algorithm instead of the hidden Markov model (HMM) based network.\nThe RNNs are implemented using massively parallel processing elements (PEs) for\nlow latency and high throughput. The weights are quantized to 6 bits to store\nall of them in the on-chip memory of an FPGA. The proposed algorithm is\nimplemented on a Xilinx XC7Z045, and the system can operate much faster than\nreal-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.00956v1",
            "date": "2016-10-04 12:48:51+00:00",
            "title": "Embracing data abundance: BookTest Dataset for Reading Comprehension",
            "authors": [
                "Ondrej Bajgar",
                "Rudolf Kadlec",
                "Jan Kleindienst"
            ],
            "abstract": "There is a practically unlimited amount of natural language data available.\nStill, recent work in text comprehension has focused on datasets which are\nsmall relative to current computing possibilities. This article is making a\ncase for the community to move to larger data and as a step in that direction\nit is proposing the BookTest, a new dataset similar to the popular Children's\nBook Test (CBT), however more than 60 times larger. We show that training on\nthe new data improves the accuracy of our Attention-Sum Reader model on the\noriginal CBT test data by a much larger margin than many recent attempts to\nimprove the model architecture. On one version of the dataset our ensemble even\nexceeds the human baseline provided by Facebook. We then show in our own human\nstudy that there is still space for further improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01030v2",
            "date": "2016-10-04 14:53:51+00:00",
            "title": "Applications of Online Deep Learning for Crisis Response Using Social Media Information",
            "authors": [
                "Dat Tien Nguyen",
                "Shafiq Joty",
                "Muhammad Imran",
                "Hassan Sajjad",
                "Prasenjit Mitra"
            ],
            "abstract": "During natural or man-made disasters, humanitarian response organizations\nlook for useful information to support their decision-making processes. Social\nmedia platforms such as Twitter have been considered as a vital source of\nuseful information for disaster response and management. Despite advances in\nnatural language processing techniques, processing short and informal Twitter\nmessages is a challenging task. In this paper, we propose to use Deep Neural\nNetwork (DNN) to address two types of information needs of response\norganizations: 1) identifying informative tweets and 2) classifying them into\ntopical classes. DNNs use distributed representation of words and learn the\nrepresentation as well as higher level features automatically for the\nclassification task. We propose a new online algorithm based on stochastic\ngradient descent to train DNNs in an online fashion during disaster situations.\nWe test our models using a crisis-related real-world Twitter dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01465v4",
            "date": "2016-10-05 14:58:36+00:00",
            "title": "Visual Question Answering: Datasets, Algorithms, and Future Challenges",
            "authors": [
                "Kushal Kafle",
                "Christopher Kanan"
            ],
            "abstract": "Visual Question Answering (VQA) is a recent problem in computer vision and\nnatural language processing that has garnered a large amount of interest from\nthe deep learning, computer vision, and natural language processing\ncommunities. In VQA, an algorithm needs to answer text-based questions about\nimages. Since the release of the first VQA dataset in 2014, additional datasets\nhave been released and many algorithms have been proposed. In this review, we\ncritically examine the current state of VQA in terms of problem formulation,\nexisting datasets, evaluation metrics, and algorithms. In particular, we\ndiscuss the limitations of current datasets with regard to their ability to\nproperly train and assess VQA algorithms. We then exhaustively review existing\nalgorithms for VQA. Finally, we discuss possible future directions for VQA and\nimage understanding research.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1610.01546v1",
            "date": "2016-09-22 05:46:49+00:00",
            "title": "Conversational Recommendation System with Unsupervised Learning",
            "authors": [
                "Yueming Sun",
                "Yi Zhang",
                "Yunfei Chen",
                "Roger Jin"
            ],
            "abstract": "We will demonstrate a conversational products recommendation agent. This\nsystem shows how we combine research in personalized recommendation systems\nwith research in dialogue systems to build a virtual sales agent. Based on new\ndeep learning technologies we developed, the virtual agent is capable of\nlearning how to interact with users, how to answer user questions, what is the\nnext question to ask, and what to recommend when chatting with a human user.\n  Normally a descent conversational agent for a particular domain requires tens\nof thousands of hand labeled conversational data or hand written rules. This is\na major barrier when launching a conversation agent for a new domain. We will\nexplore and demonstrate the effectiveness of the learning solution even when\nthere is no hand written rules or hand labeled training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.01891v1",
            "date": "2016-10-06 14:38:09+00:00",
            "title": "A New Data Representation Based on Training Data Characteristics to Extract Drug Named-Entity in Medical Text",
            "authors": [
                "Sadikin Mujiono",
                "Mohamad Ivan Fanany",
                "Chan Basaruddin"
            ],
            "abstract": "One essential task in information extraction from the medical corpus is drug\nname recognition. Compared with text sources come from other domains, the\nmedical text is special and has unique characteristics. In addition, the\nmedical text mining poses more challenges, e.g., more unstructured text, the\nfast growing of new terms addition, a wide range of name variation for the same\ndrug. The mining is even more challenging due to the lack of labeled dataset\nsources and external knowledge, as well as multiple token representations for a\nsingle drug name that is more common in the real application setting. Although\nmany approaches have been proposed to overwhelm the task, some problems\nremained with poor F-score performance (less than 0.75). This paper presents a\nnew treatment in data representation techniques to overcome some of those\nchallenges. We propose three data representation techniques based on the\ncharacteristics of word distribution and word similarities as a result of word\nembedding training. The first technique is evaluated with the standard NN\nmodel, i.e., MLP (Multi-Layer Perceptrons). The second technique involves two\ndeep network classifiers, i.e., DBN (Deep Belief Networks), and SAE (Stacked\nDenoising Encoders). The third technique represents the sentence as a sequence\nthat is evaluated with a recurrent NN model, i.e., LSTM (Long Short Term\nMemory). In extracting the drug name entities, the third technique gives the\nbest F-score performance compared to the state of the art, with its average\nF-score being 0.8645.",
            "journal ref": "Computational Intelligence and Neuroscience Volume 2016 (2016),\n  Article ID 3483528, 24 pages",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02424v2",
            "date": "2016-10-07 20:56:47+00:00",
            "title": "Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models",
            "authors": [
                "Ashwin K Vijayakumar",
                "Michael Cogswell",
                "Ramprasath R. Selvaraju",
                "Qing Sun",
                "Stefan Lee",
                "David Crandall",
                "Dhruv Batra"
            ],
            "abstract": "Neural sequence models are widely used to model time-series data. Equally\nubiquitous is the usage of beam search (BS) as an approximate inference\nalgorithm to decode output sequences from these models. BS explores the search\nspace in a greedy left-right fashion retaining only the top-B candidates -\nresulting in sequences that differ only slightly from each other. Producing\nlists of nearly identical sequences is not only computationally wasteful but\nalso typically fails to capture the inherent ambiguity of complex AI tasks. To\novercome this problem, we propose Diverse Beam Search (DBS), an alternative to\nBS that decodes a list of diverse outputs by optimizing for a\ndiversity-augmented objective. We observe that our method finds better top-1\nsolutions by controlling for the exploration and exploitation of the search\nspace - implying that DBS is a better search algorithm. Moreover, these gains\nare achieved with minimal computational or memory over- head as compared to\nbeam search. To demonstrate the broad applicability of our method, we present\nresults on image captioning, machine translation and visual question generation\nusing both standard quantitative metrics and qualitative human studies.\nFurther, we study the role of diversity for image-grounded language generation\ntasks as the complexity of the image changes. We observe that our method\nconsistently outperforms BS and previously proposed techniques for diverse\ndecoding from neural sequence models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.02692v1",
            "date": "2016-10-09 16:38:31+00:00",
            "title": "Open-Ended Visual Question-Answering",
            "authors": [
                "Issey Masuda",
                "Santiago Pascual de la Puente",
                "Xavier Giro-i-Nieto"
            ],
            "abstract": "This thesis report studies methods to solve Visual Question-Answering (VQA)\ntasks with a Deep Learning framework. As a preliminary step, we explore Long\nShort-Term Memory (LSTM) networks used in Natural Language Processing (NLP) to\ntackle Question-Answering (text based). We then modify the previous model to\naccept an image as an input in addition to the question. For this purpose, we\nexplore the VGG-16 and K-CNN convolutional neural networks to extract visual\nfeatures from the image. These are merged with the word embedding or with a\nsentence embedding of the question to predict the answer. This work was\nsuccessfully submitted to the Visual Question Answering Challenge 2016, where\nit achieved a 53,62% of accuracy in the test dataset. The developed software\nhas followed the best programming practices and Python code style, providing a\nconsistent baseline in Keras for different configurations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.02891v3",
            "date": "2016-10-10 12:51:05+00:00",
            "title": "Personalizing a Dialogue System with Transfer Reinforcement Learning",
            "authors": [
                "Kaixiang Mo",
                "Shuangyin Li",
                "Yu Zhang",
                "Jiajun Li",
                "Qiang Yang"
            ],
            "abstract": "It is difficult to train a personalized task-oriented dialogue system because\nthe data collected from each individual is often insufficient. Personalized\ndialogue systems trained on a small dataset can overfit and make it difficult\nto adapt to different user needs. One way to solve this problem is to consider\na collection of multiple users' data as a source domain and an individual\nuser's data as a target domain, and to perform a transfer learning from the\nsource to the target domain. By following this idea, we propose\n\"PETAL\"(PErsonalized Task-oriented diALogue), a transfer-learning framework\nbased on POMDP to learn a personalized dialogue system. The system first learns\ncommon dialogue knowledge from the source domain and then adapts this knowledge\nto the target user. This framework can avoid the negative transfer problem by\nconsidering differences between source and target users. The policy in the\npersonalized POMDP can learn to choose different actions appropriately for\ndifferent users. Experimental results on a real-world coffee-shopping data and\nsimulation data show that our personalized dialogue system can choose different\noptimal actions for different users, and thus effectively improve the dialogue\nquality under the personalized setting.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.02906v3",
            "date": "2016-10-10 13:27:01+00:00",
            "title": "A General Framework for Content-enhanced Network Representation Learning",
            "authors": [
                "Xiaofei Sun",
                "Jiang Guo",
                "Xiao Ding",
                "Ting Liu"
            ],
            "abstract": "This paper investigates the problem of network embedding, which aims at\nlearning low-dimensional vector representation of nodes in networks. Most\nexisting network embedding methods rely solely on the network structure, i.e.,\nthe linkage relationships between nodes, but ignore the rich content\ninformation associated with it, which is common in real world networks and\nbeneficial to describing the characteristics of a node. In this paper, we\npropose content-enhanced network embedding (CENE), which is capable of jointly\nleveraging the network structure and the content information. Our approach\nintegrates text modeling and structure modeling in a general framework by\ntreating the content information as a special kind of node. Experiments on\nseveral real world net- works with application to node classification show that\nour models outperform all existing network embedding methods, demonstrating the\nmerits of content information and joint learning.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1610.03035v6",
            "date": "2016-10-10 19:16:08+00:00",
            "title": "Latent Sequence Decompositions",
            "authors": [
                "William Chan",
                "Yu Zhang",
                "Quoc Le",
                "Navdeep Jaitly"
            ],
            "abstract": "We present the Latent Sequence Decompositions (LSD) framework. LSD decomposes\nsequences with variable lengthed output units as a function of both the input\nsequence and the output sequence. We present a training algorithm which samples\nvalid extensions and an approximate decoding algorithm. We experiment with the\nWall Street Journal speech recognition task. Our LSD model achieves 12.9% WER\ncompared to a character baseline of 14.8% WER. When combined with a\nconvolutional network on the encoder, we achieve 9.6% WER.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1610.03106v1",
            "date": "2016-10-10 21:52:47+00:00",
            "title": "Supervised Term Weighting Metrics for Sentiment Analysis in Short Text",
            "authors": [
                "Hussam Hamdan",
                "Patrice Bellot",
                "Frederic Bechet"
            ],
            "abstract": "Term weighting metrics assign weights to terms in order to discriminate the\nimportant terms from the less crucial ones. Due to this characteristic, these\nmetrics have attracted growing attention in text classification and recently in\nsentiment analysis. Using the weights given by such metrics could lead to more\naccurate document representation which may improve the performance of the\nclassification. While previous studies have focused on proposing or comparing\ndifferent weighting metrics at two-classes document level sentiment analysis,\nthis study propose to analyse the results given by each metric in order to find\nout the characteristics of good and bad weighting metrics. Therefore we present\nan empirical study of fifteen global supervised weighting metrics with four\nlocal weighting metrics adopted from information retrieval, we also give an\nanalysis to understand the behavior of each metric by observing and analysing\nhow each metric distributes the terms and deduce some characteristics which may\ndistinguish the good and bad metrics. The evaluation has been done using\nSupport Vector Machine on three different datasets: Twitter, restaurant and\nlaptop reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.03164v1",
            "date": "2016-10-11 02:47:09+00:00",
            "title": "Navigational Instruction Generation as Inverse Reinforcement Learning with Neural Machine Translation",
            "authors": [
                "Andrea F. Daniele",
                "Mohit Bansal",
                "Matthew R. Walter"
            ],
            "abstract": "Modern robotics applications that involve human-robot interaction require\nrobots to be able to communicate with humans seamlessly and effectively.\nNatural language provides a flexible and efficient medium through which robots\ncan exchange information with their human partners. Significant advancements\nhave been made in developing robots capable of interpreting free-form\ninstructions, but less attention has been devoted to endowing robots with the\nability to generate natural language. We propose a navigational guide model\nthat enables robots to generate natural language instructions that allow humans\nto navigate a priori unknown environments. We first decide which information to\nshare with the user according to their preferences, using a policy trained from\nhuman demonstrations via inverse reinforcement learning. We then \"translate\"\nthis information into a natural language instruction using a neural\nsequence-to-sequence model that learns to generate free-form instructions from\nnatural language corpora. We evaluate our method on a benchmark route\ninstruction dataset and achieve a BLEU score of 72.18% when compared to\nhuman-generated reference instructions. We additionally conduct navigation\nexperiments with human participants that demonstrate that our method generates\ninstructions that people follow as accurately and easily as those produced by\nhumans.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1610.03914v3",
            "date": "2016-10-13 02:20:45+00:00",
            "title": "Mapping Between fMRI Responses to Movies and their Natural Language Annotations",
            "authors": [
                "Kiran Vodrahalli",
                "Po-Hsuan Chen",
                "Yingyu Liang",
                "Christopher Baldassano",
                "Janice Chen",
                "Esther Yong",
                "Christopher Honey",
                "Uri Hasson",
                "Peter Ramadge",
                "Ken Norman",
                "Sanjeev Arora"
            ],
            "abstract": "Several research groups have shown how to correlate fMRI responses to the\nmeanings of presented stimuli. This paper presents new methods for doing so\nwhen only a natural language annotation is available as the description of the\nstimulus. We study fMRI data gathered from subjects watching an episode of BBCs\nSherlock [1], and learn bidirectional mappings between fMRI responses and\nnatural language representations. We show how to leverage data from multiple\nsubjects watching the same movie to improve the accuracy of the mappings,\nallowing us to succeed at a scene classification task with 72% accuracy (random\nguessing would give 4%) and at a scene ranking task with average rank in the\ntop 4% (random guessing would give 50%). The key ingredients are (a) the use of\nthe Shared Response Model (SRM) and its variant SRM-ICA [2, 3] to aggregate\nfMRI data from multiple subjects, both of which are shown to be superior to\nstandard PCA in producing low-dimensional representations for the tasks in this\npaper; (b) a sentence embedding technique adapted from the natural language\nprocessing (NLP) literature [4] that produces semantic vector representation of\nthe annotations; (c) using previous timestep information in the featurization\nof the predictor data.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1610.03934v1",
            "date": "2016-10-13 04:10:58+00:00",
            "title": "A Survey of Voice Translation Methodologies - Acoustic Dialect Decoder",
            "authors": [
                "Hans Krupakar",
                "Keerthika Rajvel",
                "Bharathi B",
                "Angel Deborah S",
                "Vallidevi Krishnamurthy"
            ],
            "abstract": "Speech Translation has always been about giving source text or audio input\nand waiting for system to give translated output in desired form. In this\npaper, we present the Acoustic Dialect Decoder (ADD) - a voice to voice\near-piece translation device. We introduce and survey the recent advances made\nin the field of Speech Engineering, to employ in the ADD, particularly focusing\non the three major processing steps of Recognition, Translation and Synthesis.\nWe tackle the problem of machine understanding of natural language by designing\na recognition unit for source audio to text, a translation unit for source\nlanguage text to target language text, and a synthesis unit for target language\ntext to target language speech. Speech from the surroundings will be recorded\nby the recognition unit present on the ear-piece and translation will start as\nsoon as one sentence is successfully read. This way, we hope to give translated\noutput as and when input is being read. The recognition unit will use Hidden\nMarkov Models (HMMs) Based Tool-Kit (HTK), hybrid RNN systems with gated memory\ncells, and the synthesis unit, HMM based speech synthesis system HTS. This\nsystem will initially be built as an English to Tamil translation device.",
            "journal ref": "2016 International Conference on Information Communication and\n  Embedded Systems (ICICES), Chennai, 2016, pp. 1-9",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04120v1",
            "date": "2016-10-13 15:11:40+00:00",
            "title": "Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understanding",
            "authors": [
                "Lina M. Rojas Barahona",
                "Milica Gasic",
                "Nikola Mrk\u0161i\u0107",
                "Pei-Hao Su",
                "Stefan Ultes",
                "Tsung-Hsien Wen",
                "Steve Young"
            ],
            "abstract": "This paper presents a deep learning architecture for the semantic decoder\ncomponent of a Statistical Spoken Dialogue System. In a slot-filling dialogue,\nthe semantic decoder predicts the dialogue act and a set of slot-value pairs\nfrom a set of n-best hypotheses returned by the Automatic Speech Recognition.\nMost current models for spoken language understanding assume (i) word-aligned\nsemantic annotations as in sequence taggers and (ii) delexicalisation, or a\nmapping of input words to domain-specific concepts using heuristics that try to\ncapture morphological variation but that do not scale to other domains nor to\nlanguage variation (e.g., morphology, synonyms, paraphrasing ). In this work\nthe semantic decoder is trained using unaligned semantic annotations and it\nuses distributed semantic representation learning to overcome the limitations\nof explicit delexicalisation. The proposed architecture uses a convolutional\nneural network for the sentence representation and a long-short term memory\nnetwork for the context representation. Results are presented for the publicly\navailable DSTC2 corpus and an In-car corpus which is similar to DSTC2 but has a\nsignificantly higher word error rate (WER).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.04377v1",
            "date": "2016-10-14 09:06:36+00:00",
            "title": "Civique: Using Social Media to Detect Urban Emergencies",
            "authors": [
                "Diptesh Kanojia",
                "Vishwajeet Kumar",
                "Krithi Ramamritham"
            ],
            "abstract": "We present the Civique system for emergency detection in urban areas by\nmonitoring micro blogs like Tweets. The system detects emergency related\nevents, and classifies them into appropriate categories like \"fire\",\n\"accident\", \"earthquake\", etc. We demonstrate our ideas by classifying Twitter\nposts in real time, visualizing the ongoing event on a map interface and\nalerting users with options to contact relevant authorities, both online and\noffline. We evaluate our classifiers for both the steps, i.e., emergency\ndetection and categorization, and obtain F-scores exceeding 70% and 90%,\nrespectively. We demonstrate Civique using a web interface and on an Android\napplication, in realtime, and show its use for both tweet detection and\nvisualization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.04658v2",
            "date": "2016-10-14 22:03:15+00:00",
            "title": "Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation",
            "authors": [
                "Yacine Jernite",
                "Anna Choromanska",
                "David Sontag"
            ],
            "abstract": "We consider multi-class classification where the predictor has a hierarchical\nstructure that allows for a very large number of labels both at train and test\ntime. The predictive power of such models can heavily depend on the structure\nof the tree, and although past work showed how to learn the tree structure, it\nexpected that the feature vectors remained static. We provide a novel algorithm\nto simultaneously perform representation learning for the input data and\nlearning of the hierarchi- cal predictor. Our approach optimizes an objec- tive\nfunction which favors balanced and easily- separable multi-way node partitions.\nWe theoret- ically analyze this objective, showing that it gives rise to a\nboosting style property and a bound on classification error. We next show how\nto extend the algorithm to conditional density estimation. We empirically\nvalidate both variants of the al- gorithm on text classification and language\nmod- eling, respectively, and show that they compare favorably to common\nbaselines in terms of accu- racy and running time.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1610.05688v1",
            "date": "2016-10-18 16:02:10+00:00",
            "title": "Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models",
            "authors": [
                "Pranay Dighe",
                "Afsaneh Asaei",
                "Herve Bourlard"
            ],
            "abstract": "Conventional deep neural networks (DNN) for speech acoustic modeling rely on\nGaussian mixture models (GMM) and hidden Markov model (HMM) to obtain binary\nclass labels as the targets for DNN training. Subword classes in speech\nrecognition systems correspond to context-dependent tied states or senones. The\npresent work addresses some limitations of GMM-HMM senone alignments for DNN\ntraining. We hypothesize that the senone probabilities obtained from a DNN\ntrained with binary labels can provide more accurate targets to learn better\nacoustic models. However, DNN outputs bear inaccuracies which are exhibited as\nhigh dimensional unstructured noise, whereas the informative components are\nstructured and low-dimensional. We exploit principle component analysis (PCA)\nand sparse coding to characterize the senone subspaces. Enhanced probabilities\nobtained from low-rank and sparse reconstructions are used as soft-targets for\nDNN acoustic modeling, that also enables training with untranscribed data.\nExperiments conducted on AMI corpus shows 4.6% relative reduction in word error\nrate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.05948v1",
            "date": "2016-10-19 10:16:46+00:00",
            "title": "A Bayesian Approach to Estimation of Speaker Normalization Parameters",
            "authors": [
                "Dhananjay Ram",
                "Debasis Kundu",
                "Rajesh M. Hegde"
            ],
            "abstract": "In this work, a Bayesian approach to speaker normalization is proposed to\ncompensate for the degradation in performance of a speaker independent speech\nrecognition system. The speaker normalization method proposed herein uses the\ntechnique of vocal tract length normalization (VTLN). The VTLN parameters are\nestimated using a novel Bayesian approach which utilizes the Gibbs sampler, a\nspecial type of Markov Chain Monte Carlo method. Additionally the\nhyperparameters are estimated using maximum likelihood approach. This model is\nused assuming that human vocal tract can be modeled as a tube of uniform cross\nsection. It captures the variation in length of the vocal tract of different\nspeakers more effectively, than the linear model used in literature. The work\nhas also investigated different methods like minimization of Mean Square Error\n(MSE) and Mean Absolute Error (MAE) for the estimation of VTLN parameters. Both\nsingle pass and two pass approaches are then used to build a VTLN based speech\nrecognizer. Experimental results on recognition of vowels and Hindi phrases\nfrom a medium vocabulary indicate that the Bayesian method improves the\nperformance by a considerable margin.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1610.06370v1",
            "date": "2016-10-20 11:48:30+00:00",
            "title": "Clinical Text Prediction with Numerically Grounded Conditional Language Models",
            "authors": [
                "Georgios P. Spithourakis",
                "Steffen E. Petersen",
                "Sebastian Riedel"
            ],
            "abstract": "Assisted text input techniques can save time and effort and improve text\nquality. In this paper, we investigate how grounded and conditional extensions\nto standard neural language models can bring improvements in the tasks of word\nprediction and completion. These extensions incorporate a structured knowledge\nbase and numerical values from the text into the context used to predict the\nnext word. Our automated evaluation on a clinical dataset shows extended models\nsignificantly outperform standard models. Our best system uses both\nconditioning and grounding, because of their orthogonal benefits. For word\nprediction with a list of 5 suggestions, it improves recall from 25.03% to\n71.28% and for word completion it improves keystroke savings from 34.35% to\n44.81%, where theoretical bound for this dataset is 58.78%. We also perform a\nqualitative investigation of how models with lower perplexity occasionally fare\nbetter at the tasks. We found that at test time numbers have more influence on\nthe document level than on individual word probabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06454v2",
            "date": "2016-10-20 15:17:04+00:00",
            "title": "Reasoning with Memory Augmented Neural Networks for Language Comprehension",
            "authors": [
                "Tsendsuren Munkhdalai",
                "Hong Yu"
            ],
            "abstract": "Hypothesis testing is an important cognitive process that supports human\nreasoning. In this paper, we introduce a computational hypothesis testing\napproach based on memory augmented neural networks. Our approach involves a\nhypothesis testing loop that reconsiders and progressively refines a previously\nformed hypothesis in order to generate new hypotheses to test. We apply the\nproposed approach to language comprehension task by using Neural Semantic\nEncoders (NSE). Our NSE models achieve the state-of-the-art results showing an\nabsolute improvement of 1.2% to 2.6% accuracy over previous results obtained by\nsingle and ensemble systems on standard machine comprehension benchmarks such\nas the Children's Book Test (CBT) and Who-Did-What (WDW) news article datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06620v2",
            "date": "2016-10-20 22:01:36+00:00",
            "title": "Proposing Plausible Answers for Open-ended Visual Question Answering",
            "authors": [
                "Omid Bakhshandeh",
                "Trung Bui",
                "Zhe Lin",
                "Walter Chang"
            ],
            "abstract": "Answering open-ended questions is an essential capability for any intelligent\nagent. One of the most interesting recent open-ended question answering\nchallenges is Visual Question Answering (VQA) which attempts to evaluate a\nsystem's visual understanding through its answers to natural language questions\nabout images. There exist many approaches to VQA, the majority of which do not\nexhibit deeper semantic understanding of the candidate answers they produce. We\nstudy the importance of generating plausible answers to a given question by\nintroducing the novel task of `Answer Proposal': for a given open-ended\nquestion, a system should generate a ranked list of candidate answers informed\nby the semantics of the question. We experiment with various models including a\nneural generative model as well as a semantic graph matching one. We provide\nboth intrinsic and extrinsic evaluations for the task of Answer Proposal,\nshowing that our best model learns to propose plausible answers with a high\nrecall and performs competitively with some other solutions to VQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06700v1",
            "date": "2016-10-21 08:45:35+00:00",
            "title": "End-to-End Training Approaches for Discriminative Segmental Models",
            "authors": [
                "Hao Tang",
                "Weiran Wang",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "Recent work on discriminative segmental models has shown that they can\nachieve competitive speech recognition performance, using features based on\ndeep neural frame classifiers. However, segmental models can be more\nchallenging to train than standard frame-based approaches. While some segmental\nmodels have been successfully trained end to end, there is a lack of\nunderstanding of their training under different settings and with different\nlosses.\n  We investigate a model class based on recent successful approaches,\nconsisting of a linear model that combines segmental features based on an LSTM\nframe classifier. Similarly to hybrid HMM-neural network models, segmental\nmodels of this class can be trained in two stages (frame classifier training\nfollowed by linear segmental model weight training), end to end (joint training\nof both frame classifier and linear weights), or with end-to-end fine-tuning\nafter two-stage training.\n  We study segmental models trained end to end with hinge loss, log loss,\nlatent hinge loss, and marginal log loss. We consider several losses for the\ncase where training alignments are available as well as where they are not.\n  We find that in general, marginal log loss provides the most consistent\nstrong performance without requiring ground-truth alignments. We also find that\ntraining with dropout is very important in obtaining good performance with\nend-to-end training. Finally, the best results are typically obtained by a\ncombination of two-stage training and fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.06856v1",
            "date": "2016-10-21 16:53:09+00:00",
            "title": "Automated Big Text Security Classification",
            "authors": [
                "Khudran Alzhrani",
                "Ethan M. Rudd",
                "Terrance E. Boult",
                "C. Edward Chow"
            ],
            "abstract": "In recent years, traditional cybersecurity safeguards have proven ineffective\nagainst insider threats. Famous cases of sensitive information leaks caused by\ninsiders, including the WikiLeaks release of diplomatic cables and the Edward\nSnowden incident, have greatly harmed the U.S. government's relationship with\nother governments and with its own citizens. Data Leak Prevention (DLP) is a\nsolution for detecting and preventing information leaks from within an\norganization's network. However, state-of-art DLP detection models are only\nable to detect very limited types of sensitive information, and research in the\nfield has been hindered due to the lack of available sensitive texts. Many\nresearchers have focused on document-based detection with artificially labeled\n\"confidential documents\" for which security labels are assigned to the entire\ndocument, when in reality only a portion of the document is sensitive. This\ntype of whole-document based security labeling increases the chances of\npreventing authorized users from accessing non-sensitive information within\nsensitive documents. In this paper, we introduce Automated Classification\nEnabled by Security Similarity (ACESS), a new and innovative detection model\nthat penetrates the complexity of big text security classification/detection.\nTo analyze the ACESS system, we constructed a novel dataset, containing\nformerly classified paragraphs from diplomatic cables made public by the\nWikiLeaks organization. To our knowledge this paper is the first to analyze a\ndataset that contains actual formerly sensitive information annotated at\nparagraph granularity.",
            "journal ref": "2016 IEEE International Conference on Intelligence and Security\n  Informatics (ISI)",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1610.07363v1",
            "date": "2016-10-24 11:25:24+00:00",
            "title": "Learning Reporting Dynamics during Breaking News for Rumour Detection in Social Media",
            "authors": [
                "Arkaitz Zubiaga",
                "Maria Liakata",
                "Rob Procter"
            ],
            "abstract": "Breaking news leads to situations of fast-paced reporting in social media,\nproducing all kinds of updates related to news stories, albeit with the caveat\nthat some of those early updates tend to be rumours, i.e., information with an\nunverified status at the time of posting. Flagging information that is\nunverified can be helpful to avoid the spread of information that may turn out\nto be false. Detection of rumours can also feed a rumour tracking system that\nultimately determines their veracity. In this paper we introduce a novel\napproach to rumour detection that learns from the sequential dynamics of\nreporting during breaking news in social media to detect rumours in new\nstories. Using Twitter datasets collected during five breaking news stories, we\nexperiment with Conditional Random Fields as a sequential classifier that\nleverages context learnt during an event for rumour detection, which we compare\nwith the state-of-the-art rumour detection system as well as other baselines.\nIn contrast to existing work, our classifier does not need to observe tweets\nquerying a piece of information to deem it a rumour, but instead we detect\nrumours from the tweet alone by exploiting context learnt during the event. Our\nclassifier achieves competitive performance, beating the state-of-the-art\nclassifier that relies on querying tweets with improved precision and recall,\nas well as outperforming our best baseline with nearly 40% improvement in terms\nof F1 score. The scale and diversity of our experiments reinforces the\ngeneralisability of our classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07365v1",
            "date": "2016-10-24 11:30:22+00:00",
            "title": "Introduction: Cognitive Issues in Natural Language Processing",
            "authors": [
                "Thierry Poibeau",
                "Shravan Vasishth"
            ],
            "abstract": "This special issue is dedicated to get a better picture of the relationships\nbetween computational linguistics and cognitive science. It specifically raises\ntwo questions: \"what is the potential contribution of computational language\nmodeling to cognitive science?\" and conversely: \"what is the influence of\ncognitive science in contemporary computational linguistics?\"",
            "journal ref": "Traitement Automatique des Langues, ATALA, 2014, Traitement\n  Automatique des Langues et Sciences Cognitives, 55 (3), pp.7-19",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07432v1",
            "date": "2016-10-24 14:37:27+00:00",
            "title": "Virtual Embodiment: A Scalable Long-Term Strategy for Artificial Intelligence Research",
            "authors": [
                "Douwe Kiela",
                "Luana Bulat",
                "Anita L. Vero",
                "Stephen Clark"
            ],
            "abstract": "Meaning has been called the \"holy grail\" of a variety of scientific\ndisciplines, ranging from linguistics to philosophy, psychology and the\nneurosciences. The field of Artifical Intelligence (AI) is very much a part of\nthat list: the development of sophisticated natural language semantics is a\nsine qua non for achieving a level of intelligence comparable to humans.\nEmbodiment theories in cognitive science hold that human semantic\nrepresentation depends on sensori-motor experience; the abundant evidence that\nhuman meaning representation is grounded in the perception of physical reality\nleads to the conclusion that meaning must depend on a fusion of multiple\n(perceptual) modalities. Despite this, AI research in general, and its\nsubdisciplines such as computational linguistics and computer vision in\nparticular, have focused primarily on tasks that involve a single modality.\nHere, we propose virtual embodiment as an alternative, long-term strategy for\nAI research that is multi-modal in nature and that allows for the kind of\nscalability required to develop the field coherently and incrementally, in an\nethically responsible fashion.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1610.07569v1",
            "date": "2016-10-24 19:35:29+00:00",
            "title": "Geometry of Polysemy",
            "authors": [
                "Jiaqi Mu",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Vector representations of words have heralded a transformational approach to\nclassical problems in NLP; the most popular example is word2vec. However, a\nsingle vector does not suffice to model the polysemous nature of many\n(frequent) words, i.e., words with multiple meanings. In this paper, we propose\na three-fold approach for unsupervised polysemy modeling: (a) context\nrepresentations, (b) sense induction and disambiguation and (c) lexeme (as a\nword and sense pair) representations. A key feature of our work is the finding\nthat a sentence containing a target word is well represented by a low rank\nsubspace, instead of a point in a vector space. We then show that the subspaces\nassociated with a particular sense of the target word tend to intersect over a\nline (one-dimensional subspace), which we use to disambiguate senses using a\nclustering algorithm that harnesses the Grassmannian geometry of the\nrepresentations. The disambiguation algorithm, which we call $K$-Grassmeans,\nleads to a procedure to label the different senses of the target word in the\ncorpus -- yielding lexeme vector representations, all in an unsupervised manner\nstarting from a large (Wikipedia) corpus in English. Apart from several\nprototypical target (word,sense) examples and a host of empirical studies to\nintuit and justify the various geometric representations, we validate our\nalgorithms on standard sense induction and disambiguation datasets and present\nnew state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.07647v2",
            "date": "2016-10-24 20:48:04+00:00",
            "title": "Learning to Reason With Adaptive Computation",
            "authors": [
                "Mark Neumann",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "Multi-hop inference is necessary for machine learning systems to successfully\nsolve tasks such as Recognising Textual Entailment and Machine Reading. In this\nwork, we demonstrate the effectiveness of adaptive computation for learning the\nnumber of inference steps required for examples of different complexity and\nthat learning the correct number of inference steps is difficult. We introduce\nthe first model involving Adaptive Computation Time which provides a small\nperformance benefit on top of a similar model without an adaptive component as\nwell as enabling considerable insight into the reasoning process of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.08597v1",
            "date": "2016-10-27 03:21:49+00:00",
            "title": "Word Embeddings to Enhance Twitter Gang Member Profile Identification",
            "authors": [
                "Sanjaya Wijeratne",
                "Lakshika Balasuriya",
                "Derek Doran",
                "Amit Sheth"
            ],
            "abstract": "Gang affiliates have joined the masses who use social media to share thoughts\nand actions publicly. Interestingly, they use this public medium to express\nrecent illegal actions, to intimidate others, and to share outrageous images\nand statements. Agencies able to unearth these profiles may thus be able to\nanticipate, stop, or hasten the investigation of gang-related crimes. This\npaper investigates the use of word embeddings to help identify gang members on\nTwitter. Building on our previous work, we generate word embeddings that\ntranslate what Twitter users post in their profile descriptions, tweets,\nprofile images, and linked YouTube content to a real vector format amenable for\nmachine learning classification. Our experimental results show that pre-trained\nword embeddings can boost the accuracy of supervised learning algorithms\ntrained over gang members social media posts.",
            "journal ref": "IJCAI Workshop on Semantic Machine Learning (SML 2016). pp. 18-24.\n  CEUR-WS, New York City, NY (07 2016)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1610.09225v1",
            "date": "2016-10-28 14:07:43+00:00",
            "title": "Sentiment Analysis of Twitter Data for Predicting Stock Market Movements",
            "authors": [
                "Venkata Sasank Pagolu",
                "Kamal Nayan Reddy Challa",
                "Ganapati Panda",
                "Babita Majhi"
            ],
            "abstract": "Predicting stock market movements is a well-known problem of interest.\nNow-a-days social media is perfectly representing the public sentiment and\nopinion about current events. Especially, twitter has attracted a lot of\nattention from researchers for studying the public sentiments. Stock market\nprediction on the basis of public sentiments expressed on twitter has been an\nintriguing field of research. Previous studies have concluded that the\naggregate public mood collected from twitter may well be correlated with Dow\nJones Industrial Average Index (DJIA). The thesis of this work is to observe\nhow well the changes in stock prices of a company, the rises and falls, are\ncorrelated with the public opinions being expressed in tweets about that\ncompany. Understanding author's opinion from a piece of text is the objective\nof sentiment analysis. The present paper have employed two different textual\nrepresentations, Word2vec and N-gram, for analyzing the public sentiments in\ntweets. In this paper, we have applied sentiment analysis and supervised\nmachine learning principles to the tweets extracted from twitter and analyze\nthe correlation between stock market movements of a company and sentiments in\ntweets. In an elaborate way, positive news and tweets in social media about a\ncompany would definitely encourage people to invest in the stocks of that\ncompany and as a result the stock price of that company would increase. At the\nend of the paper, it is shown that a strong correlation exists between the rise\nand falls in stock prices with the public sentiments in tweets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1610.09516v1",
            "date": "2016-10-29 14:30:57+00:00",
            "title": "Finding Street Gang Members on Twitter",
            "authors": [
                "Lakshika Balasuriya",
                "Sanjaya Wijeratne",
                "Derek Doran",
                "Amit Sheth"
            ],
            "abstract": "Most street gang members use Twitter to intimidate others, to present\noutrageous images and statements to the world, and to share recent illegal\nactivities. Their tweets may thus be useful to law enforcement agencies to\ndiscover clues about recent crimes or to anticipate ones that may occur.\nFinding these posts, however, requires a method to discover gang member Twitter\nprofiles. This is a challenging task since gang members represent a very small\npopulation of the 320 million Twitter users. This paper studies the problem of\nautomatically finding gang members on Twitter. It outlines a process to curate\none of the largest sets of verifiable gang member profiles that have ever been\nstudied. A review of these profiles establishes differences in the language,\nimages, YouTube links, and emojis gang members use compared to the rest of the\nTwitter population. Features from this review are used to train a series of\nsupervised classifiers. Our classifier achieves a promising F1 score with a low\nfalse positive rate.",
            "journal ref": "The 2016 IEEE/ACM Int. Conf. on Advances in Social Networks\n  Analysis and Mining. vol. 8, pp. 685-692. San Francisco, CA, USA (2016)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1610.09704v1",
            "date": "2016-10-30 20:09:46+00:00",
            "title": "Feature-Augmented Neural Networks for Patient Note De-identification",
            "authors": [
                "Ji Young Lee",
                "Franck Dernoncourt",
                "Ozlem Uzuner",
                "Peter Szolovits"
            ],
            "abstract": "Patient notes contain a wealth of information of potentially great interest\nto medical investigators. However, to protect patients' privacy, Protected\nHealth Information (PHI) must be removed from the patient notes before they can\nbe legally released, a process known as patient note de-identification. The\nmain objective for a de-identification system is to have the highest possible\nrecall. Recently, the first neural-network-based de-identification system has\nbeen proposed, yielding state-of-the-art results. Unlike other systems, it does\nnot rely on human-engineered features, which allows it to be quickly deployed,\nbut does not leverage knowledge from human experts or from electronic health\nrecords (EHRs). In this work, we explore a method to incorporate\nhuman-engineered features as well as features derived from EHRs to a\nneural-network-based de-identification system. Our results show that the\naddition of features, especially the EHR-derived features, further improves the\nstate-of-the-art in patient note de-identification, including for some of the\nmost sensitive PHI types such as patient names. Since in a real-life setting\npatient notes typically come with EHRs, we recommend developers of\nde-identification systems to leverage the information EHRs contain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1610.09975v1",
            "date": "2016-10-31 15:36:42+00:00",
            "title": "Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition",
            "authors": [
                "Hagen Soltau",
                "Hank Liao",
                "Hasim Sak"
            ],
            "abstract": "We present results that show it is possible to build a competitive, greatly\nsimplified, large vocabulary continuous speech recognition system with whole\nwords as acoustic units. We model the output vocabulary of about 100,000 words\ndirectly using deep bi-directional LSTM RNNs with CTC loss. The model is\ntrained on 125,000 hours of semi-supervised acoustic training data, which\nenables us to alleviate the data sparsity problem for word models. We show that\nthe CTC word models work very well as an end-to-end all-neural speech\nrecognition model without the use of traditional context-dependent sub-word\nphone units that require a pronunciation lexicon, and without any language\nmodel removing the need to decode. We demonstrate that the CTC word models\nperform better than a strong, more complex, state-of-the-art baseline with\nsub-word units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00347v1",
            "date": "2016-12-01 16:49:04+00:00",
            "title": "Bootstrapping incremental dialogue systems: using linguistic knowledge to learn from minimal data",
            "authors": [
                "Dimitrios Kalatzis",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "We present a method for inducing new dialogue systems from very small amounts\nof unannotated dialogue data, showing how word-level exploration using\nReinforcement Learning (RL), combined with an incremental and semantic grammar\n- Dynamic Syntax (DS) - allows systems to discover, generate, and understand\nmany new dialogue variants. The method avoids the use of expensive and\ntime-consuming dialogue act annotations, and supports more natural\n(incremental) dialogues than turn-based systems. Here, language generation and\ndialogue management are treated as a joint decision/optimisation problem, and\nthe MDP model for RL is constructed automatically. With an implemented system,\nwe show that this method enables a wide range of dialogue variations to be\nautomatically captured, even when the system is trained from only a single\ndialogue. The variants include question-answer pairs, over- and\nunder-answering, self- and other-corrections, clarification interaction,\nsplit-utterances, and ellipsis. This generalisation property results from the\nstructural knowledge and constraints present within the DS grammar, and\nhighlights some limitations of recent systems built using machine learning\ntechniques only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00377v4",
            "date": "2016-12-01 18:49:23+00:00",
            "title": "Piecewise Latent Variables for Neural Variational Text Processing",
            "authors": [
                "Iulian V. Serban",
                "Alexander G. Ororbia II",
                "Joelle Pineau",
                "Aaron Courville"
            ],
            "abstract": "Advances in neural variational inference have facilitated the learning of\npowerful directed graphical models with continuous latent variables, such as\nvariational autoencoders. The hope is that such models will learn to represent\nrich, multi-modal latent factors in real-world data, such as natural language\ntext. However, current models often assume simplistic priors on the latent\nvariables - such as the uni-modal Gaussian distribution - which are incapable\nof representing complex latent factors efficiently. To overcome this\nrestriction, we propose the simple, but highly flexible, piecewise constant\ndistribution. This distribution has the capacity to represent an exponential\nnumber of modes of a latent target distribution, while remaining mathematically\ntractable. Our results demonstrate that incorporating this new latent\ndistribution into different models yields substantial improvements in natural\nlanguage processing tasks such as document modeling and natural language\ngeneration for dialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.00837v3",
            "date": "2016-12-02 20:57:07+00:00",
            "title": "Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering",
            "authors": [
                "Yash Goyal",
                "Tejas Khot",
                "Douglas Summers-Stay",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "Problems at the intersection of vision and language are of significant\nimportance both as challenging research questions and for the rich set of\napplications they enable. However, inherent structure in our world and bias in\nour language tend to be a simpler signal for learning than visual modalities,\nresulting in models that ignore visual information, leading to an inflated\nsense of their capability.\n  We propose to counter these language priors for the task of Visual Question\nAnswering (VQA) and make vision (the V in VQA) matter! Specifically, we balance\nthe popular VQA dataset by collecting complementary images such that every\nquestion in our balanced dataset is associated with not just a single image,\nbut rather a pair of similar images that result in two different answers to the\nquestion. Our dataset is by construction more balanced than the original VQA\ndataset and has approximately twice the number of image-question pairs. Our\ncomplete balanced dataset is publicly available at www.visualqa.org as part of\nthe 2nd iteration of the Visual Question Answering Dataset and Challenge (VQA\nv2.0).\n  We further benchmark a number of state-of-art VQA models on our balanced\ndataset. All models perform significantly worse on our balanced dataset,\nsuggesting that these models have indeed learned to exploit language priors.\nThis finding provides the first concrete empirical evidence for what seems to\nbe a qualitative sense among practitioners.\n  Finally, our data collection protocol for identifying complementary images\nenables us to develop a novel interpretable model, which in addition to\nproviding an answer to the given (image, question) pair, also provides a\ncounter-example based explanation. Specifically, it identifies an image that is\nsimilar to the original image, but it believes has a different answer to the\nsame question. This can help in building trust for machines among their users.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.00944v1",
            "date": "2016-12-03 09:08:51+00:00",
            "title": "Using Discourse Signals for Robust Instructor Intervention Prediction",
            "authors": [
                "Muthu Kumar Chandrasekaran",
                "Carrie Demmans Epp",
                "Min-Yen Kan",
                "Diane Litman"
            ],
            "abstract": "We tackle the prediction of instructor intervention in student posts from\ndiscussion forums in Massive Open Online Courses (MOOCs). Our key finding is\nthat using automatically obtained discourse relations improves the prediction\nof when instructors intervene in student discussions, when compared with a\nstate-of-the-art, feature-rich baseline. Our supervised classifier makes use of\nan automatic discourse parser which outputs Penn Discourse Treebank (PDTB) tags\nthat represent in-post discourse features. We show PDTB relation-based features\nincrease the robustness of the classifier and complement baseline features in\nrecalling more diverse instructor intervention patterns. In comprehensive\nexperiments over 14 MOOC offerings from several disciplines, the PDTB discourse\nfeatures improve performance on average. The resultant models are less\ndependent on domain-specific vocabulary, allowing them to better generalize to\nnew courses.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.01197v1",
            "date": "2016-12-04 22:29:32+00:00",
            "title": "Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)",
            "authors": [
                "Chen Liang",
                "Jonathan Berant",
                "Quoc Le",
                "Kenneth D. Forbus",
                "Ni Lao"
            ],
            "abstract": "Extending the success of deep neural networks to natural language\nunderstanding and symbolic reasoning requires complex operations and external\nmemory. Recent neural program induction approaches have attempted to address\nthis problem, but are typically limited to differentiable memory, and\nconsequently cannot scale beyond small synthetic tasks. In this work, we\npropose the Manager-Programmer-Computer framework, which integrates neural\nnetworks with non-differentiable memory to support abstract, scalable and\nprecise operations through a friendly neural computer interface. Specifically,\nwe introduce a Neural Symbolic Machine, which contains a sequence-to-sequence\nneural \"programmer\", and a non-differentiable \"computer\" that is a Lisp\ninterpreter with code assist. To successfully apply REINFORCE for training, we\naugment it with approximate gold programs found by an iterative maximum\nlikelihood training process. NSM is able to learn a semantic parser from weak\nsupervision over a large knowledge base. It achieves new state-of-the-art\nperformance on WebQuestionsSP, a challenging semantic parsing dataset, with\nweak supervision. Compared to previous approaches, NSM is end-to-end, therefore\ndoes not rely on feature engineering or domain specific knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.01556v4",
            "date": "2016-12-05 21:28:06+00:00",
            "title": "The Evolution of Sentiment Analysis - A Review of Research Topics, Venues, and Top Cited Papers",
            "authors": [
                "Mika Viking M\u00e4ntyl\u00e4",
                "Daniel Graziotin",
                "Miikka Kuutila"
            ],
            "abstract": "Sentiment analysis is one of the fastest growing research areas in computer\nscience, making it challenging to keep track of all the activities in the area.\nWe present a computer-assisted literature review, where we utilize both text\nmining and qualitative coding, and analyze 6,996 papers from Scopus. We find\nthat the roots of sentiment analysis are in the studies on public opinion\nanalysis at the beginning of 20th century and in the text subjectivity analysis\nperformed by the computational linguistics community in 1990's. However, the\noutbreak of computer-based sentiment analysis only occurred with the\navailability of subjective texts on the Web. Consequently, 99% of the papers\nhave been published after 2004. Sentiment analysis papers are scattered to\nmultiple publication venues, and the combined number of papers in the top-15\nvenues only represent ca. 30% of the papers in total. We present the top-20\ncited papers from Google Scholar and Scopus and a taxonomy of research topics.\nIn recent years, sentiment analysis has shifted from analyzing online product\nreviews to social media texts from Twitter and Facebook. Many topics beyond\nproduct reviews like stock markets, elections, disasters, medicine, software\nengineering and cyberbullying extend the utilization of sentiment analysis",
            "journal ref": "Computer Science Review, Volume 27, February 2018, Pages 16-32",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02482v2",
            "date": "2016-12-07 23:20:53+00:00",
            "title": "Improving the Performance of Neural Machine Translation Involving Morphologically Rich Languages",
            "authors": [
                "Krupakar Hans",
                "R S Milton"
            ],
            "abstract": "The advent of the attention mechanism in neural machine translation models\nhas improved the performance of machine translation systems by enabling\nselective lookup into the source sentence. In this paper, the efficiencies of\ntranslation using bidirectional encoder attention decoder models were studied\nwith respect to translation involving morphologically rich languages. The\nEnglish - Tamil language pair was selected for this analysis. First, the use of\nWord2Vec embedding for both the English and Tamil words improved the\ntranslation results by 0.73 BLEU points over the baseline RNNSearch model with\n4.84 BLEU score. The use of morphological segmentation before word\nvectorization to split the morphologically rich Tamil words into their\nrespective morphemes before the translation, caused a reduction in the target\nvocabulary size by a factor of 8. Also, this model (RNNMorph) improved the\nperformance of neural machine translation by 7.05 BLEU points over the\nRNNSearch model used over the same corpus. Since the BLEU evaluation of the\nRNNMorph model might be unreliable due to an increase in the number of matching\ntokens per sentence, the performances of the translations were also compared by\nmeans of human evaluation metrics of adequacy, fluency and relative ranking.\nFurther, the use of morphological segmentation also improved the efficacy of\nthe attention mechanism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02695v1",
            "date": "2016-12-08 15:23:44+00:00",
            "title": "Towards better decoding and language model integration in sequence to sequence models",
            "authors": [
                "Jan Chorowski",
                "Navdeep Jaitly"
            ],
            "abstract": "The recently proposed Sequence-to-Sequence (seq2seq) framework advocates\nreplacing complex data processing pipelines, such as an entire automatic speech\nrecognition system, with a single neural network trained in an end-to-end\nfashion. In this contribution, we analyse an attention-based seq2seq speech\nrecognition system that directly transcribes recordings into characters. We\nobserve two shortcomings: overconfidence in its predictions and a tendency to\nproduce incomplete transcriptions when language models are used. We propose\npractical solutions to both problems achieving competitive speaker independent\nword error rates on the Wall Street Journal dataset: without separate language\nmodels we reach 10.6% WER, while together with a trigram language model, we\nreach 6.7% WER.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1612.03226v1",
            "date": "2016-12-10 00:09:45+00:00",
            "title": "Active Learning for Speech Recognition: the Power of Gradients",
            "authors": [
                "Jiaji Huang",
                "Rewon Child",
                "Vinay Rao",
                "Hairong Liu",
                "Sanjeev Satheesh",
                "Adam Coates"
            ],
            "abstract": "In training speech recognition systems, labeling audio clips can be\nexpensive, and not all data is equally valuable. Active learning aims to label\nonly the most informative samples to reduce cost. For speech recognition,\nconfidence scores and other likelihood-based active learning methods have been\nshown to be effective. Gradient-based active learning methods, however, are\nstill not well-understood. This work investigates the Expected Gradient Length\n(EGL) approach in active learning for end-to-end speech recognition. We justify\nEGL from a variance reduction perspective, and observe that EGL's measure of\ninformativeness picks novel samples uncorrelated with confidence scores.\nExperimentally, we show that EGL can reduce word errors by 11\\%, or\nalternatively, reduce the number of samples to label by 50\\%, when compared to\nrandom sampling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.03494v1",
            "date": "2016-12-11 22:27:37+00:00",
            "title": "Flu Detector: Estimating influenza-like illness rates from online user-generated content",
            "authors": [
                "Vasileios Lampos"
            ],
            "abstract": "We provide a brief technical description of an online platform for disease\nmonitoring, titled as the Flu Detector (fludetector.cs.ucl.ac.uk). Flu\nDetector, in its current version (v.0.5), uses either Twitter or Google search\ndata in conjunction with statistical Natural Language Processing models to\nestimate the rate of influenza-like illness in the population of England. Its\nback-end is a live service that collects online data, utilises modern\ntechnologies for large-scale text processing, and finally applies statistical\ninference models that are trained offline. The front-end visualises the various\ndisease rate estimates. Notably, the models based on Google data achieve a high\nlevel of accuracy with respect to the most recent four flu seasons in England\n(2012/13 to 2015/16). This highlighted Flu Detector as having a great potential\nof becoming a complementary source to the domestic traditional flu surveillance\nschemes.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1612.03929v5",
            "date": "2016-12-12 21:19:51+00:00",
            "title": "Deep Active Learning for Dialogue Generation",
            "authors": [
                "Nabiha Asghar",
                "Pascal Poupart",
                "Xin Jiang",
                "Hang Li"
            ],
            "abstract": "We propose an online, end-to-end, neural generative conversational model for\nopen-domain dialogue. It is trained using a unique combination of offline\ntwo-phase supervised learning and online human-in-the-loop active learning.\nWhile most existing research proposes offline supervision or hand-crafted\nreward functions for online reinforcement, we devise a novel interactive\nlearning mechanism based on hamming-diverse beam search for response generation\nand one-character user-feedback at each step. Experiments show that our model\ninherently promotes the generation of semantically relevant and interesting\nresponses, and can be used to train agents with customized personas, moods and\nconversational styles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04118v2",
            "date": "2016-12-13 12:12:20+00:00",
            "title": "Information Extraction with Character-level Neural Networks and Free Noisy Supervision",
            "authors": [
                "Philipp Meerkamp",
                "Zhengyi Zhou"
            ],
            "abstract": "We present an architecture for information extraction from text that augments\nan existing parser with a character-level neural network. The network is\ntrained using a measure of consistency of extracted data with existing\ndatabases as a form of noisy supervision. Our architecture combines the ability\nof constraint-based information extraction systems to easily incorporate domain\nknowledge and constraints with the ability of deep neural networks to leverage\nlarge amounts of data to learn complex features. Boosting the existing parser's\nprecision, the system led to large improvements over a mature and highly tuned\nconstraint-based production information extraction system used at Bloomberg for\nfinancial language text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04174v2",
            "date": "2016-12-13 13:55:39+00:00",
            "title": "Models of retrieval in sentence comprehension: A computational evaluation using Bayesian hierarchical modeling",
            "authors": [
                "Bruno Nicenboim",
                "Shravan Vasishth"
            ],
            "abstract": "Research on interference has provided evidence that the formation of\ndependencies between non-adjacent words relies on a cue-based retrieval\nmechanism. Two different models can account for one of the main predictions of\ninterference, i.e., a slowdown at a retrieval site, when several items share a\nfeature associated with a retrieval cue: Lewis and Vasishth's (2005)\nactivation-based model and McElree's (2000) direct access model. Even though\nthese two models have been used almost interchangeably, they are based on\ndifferent assumptions and predict differences in the relationship between\nreading times and response accuracy. The activation-based model follows the\nassumptions of ACT-R, and its retrieval process behaves as a lognormal race\nbetween accumulators of evidence with a single variance. Under this model,\naccuracy of the retrieval is determined by the winner of the race and retrieval\ntime by its rate of accumulation. In contrast, the direct access model assumes\na model of memory where only the probability of retrieval varies between items;\nin this model, differences in latencies are a by-product of the possibility and\nrepairing incorrect retrievals. We implemented both models in a Bayesian\nhierarchical framework in order to evaluate them and compare them. We show that\nsome aspects of the data are better fit under the direct access model than\nunder the activation-based model. We suggest that this finding does not rule\nout the possibility that retrieval may be behaving as a race model with\nassumptions that follow less closely the ones from the ACT-R framework. We show\nthat by introducing a modification of the activation model, i.e, by assuming\nthat the accumulation of evidence for retrieval of incorrect items is not only\nslower but noisier (i.e., different variances for the correct and incorrect\nitems), the model can provide a fit as good as the one of the direct access\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04403v1",
            "date": "2016-12-13 21:29:05+00:00",
            "title": "You Are What You Eat... Listen to, Watch, and Read",
            "authors": [
                "Mason Bretan"
            ],
            "abstract": "This article describes a data driven method for deriving the relationship\nbetween personality and media preferences. A qunatifiable representation of\nsuch a relationship can be leveraged for use in recommendation systems and\nameliorate the \"cold start\" problem. Here, the data is comprised of an original\ncollection of 1,316 Okcupid dating profiles. Of these profiles, 800 are labeled\nwith one of 16 possible Myers-Briggs Type Indicators (MBTI). A personality\nspecific topic model describing a person's favorite books, movies, shows,\nmusic, and food was generated using latent Dirichlet allocation (LDA). There\nwere several significant findings, for example, intuitive thinking types\npreferred sci-fi/fantasy entertainment, extraversion correlated positively with\nupbeat dance music, and jazz, folk, and international cuisine correlated\npositively with those characterized by openness to experience. Many other\ncorrelations confirmed previous findings describing the relationship among\npersonality, writing style, and personal preferences. (For complete\nword/personality type assocations see the Appendix).",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1612.04744v2",
            "date": "2016-12-14 17:40:02+00:00",
            "title": "Incorporating Language Level Information into Acoustic Models",
            "authors": [
                "Peidong Wang",
                "Deliang Wang"
            ],
            "abstract": "This paper proposed a class of novel Deep Recurrent Neural Networks which can\nincorporate language-level information into acoustic models. For simplicity, we\nnamed these networks Recurrent Deep Language Networks (RDLNs). Multiple\nvariants of RDLNs were considered, including two kinds of context information,\ntwo methods to process the context, and two methods to incorporate the\nlanguage-level information. RDLNs provided possible methods to fine-tune the\nwhole Automatic Speech Recognition (ASR) system in the acoustic modeling\nprocess.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.04757v2",
            "date": "2016-12-14 18:12:47+00:00",
            "title": "Attentive Explanations: Justifying Decisions and Pointing to the Evidence",
            "authors": [
                "Dong Huk Park",
                "Lisa Anne Hendricks",
                "Zeynep Akata",
                "Bernt Schiele",
                "Trevor Darrell",
                "Marcus Rohrbach"
            ],
            "abstract": "Deep models are the defacto standard in visual decision models due to their\nimpressive performance on a wide array of visual tasks. However, they are\nfrequently seen as opaque and are unable to explain their decisions. In\ncontrast, humans can justify their decisions with natural language and point to\nthe evidence in the visual world which led to their decisions. We postulate\nthat deep models can do this as well and propose our Pointing and Justification\n(PJ-X) model which can justify its decision with a sentence and point to the\nevidence by introspecting its decision and explanation process using an\nattention mechanism. Unfortunately there is no dataset available with reference\nexplanations for visual decision making. We thus collect two datasets in two\ndomains where it is interesting and challenging to explain decisions. First, we\nextend the visual question answering task to not only provide an answer but\nalso a natural language explanation for the answer. Second, we focus on\nexplaining human activities which is traditionally more challenging than object\nclassification. We extensively evaluate our PJ-X model, both on the\njustification and pointing tasks, by comparing it to prior models and ablations\nusing both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.04868v1",
            "date": "2016-12-14 22:22:33+00:00",
            "title": "Interpretable Semantic Textual Similarity: Finding and explaining differences between sentences",
            "authors": [
                "I. Lopez-Gazpio",
                "M. Maritxalar",
                "A. Gonzalez-Agirre",
                "G. Rigau",
                "L. Uria",
                "E. Agirre"
            ],
            "abstract": "User acceptance of artificial intelligence agents might depend on their\nability to explain their reasoning, which requires adding an interpretability\nlayer that fa- cilitates users to understand their behavior. This paper focuses\non adding an in- terpretable layer on top of Semantic Textual Similarity (STS),\nwhich measures the degree of semantic equivalence between two sentences. The\ninterpretability layer is formalized as the alignment between pairs of segments\nacross the two sentences, where the relation between the segments is labeled\nwith a relation type and a similarity score. We present a publicly available\ndataset of sentence pairs annotated following the formalization. We then\ndevelop a system trained on this dataset which, given a sentence pair, explains\nwhat is similar and different, in the form of graded and typed segment\nalignments. When evaluated on the dataset, the system performs better than an\ninformed baseline, showing that the dataset and task are well-defined and\nfeasible. Most importantly, two user studies show how the system output can be\nused to automatically produce explanations in natural language. Users performed\nbetter when having access to the explanations, pro- viding preliminary evidence\nthat our dataset and method to automatically produce explanations is useful in\nreal applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05251v1",
            "date": "2016-12-15 20:57:56+00:00",
            "title": "Neural Networks for Joint Sentence Classification in Medical Paper Abstracts",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee",
                "Peter Szolovits"
            ],
            "abstract": "Existing models based on artificial neural networks (ANNs) for sentence\nclassification often do not incorporate the context in which sentences appear,\nand classify sentences individually. However, traditional sentence\nclassification approaches have been shown to greatly benefit from jointly\nclassifying subsequent sentences, such as with conditional random fields. In\nthis work, we present an ANN architecture that combines the effectiveness of\ntypical ANN models to classify sentences in isolation, with the strength of\nstructured prediction. Our model achieves state-of-the-art results on two\ndifferent datasets for sequential sentence classification in medical abstracts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05270v1",
            "date": "2016-12-15 21:07:12+00:00",
            "title": "A Simple Approach to Multilingual Polarity Classification in Twitter",
            "authors": [
                "Eric S. Tellez",
                "Sabino Miranda Jim\u00e9nez",
                "Mario Graff",
                "Daniela Moctezuma",
                "Ranyart R. Su\u00e1rez",
                "Oscar S. Siordia"
            ],
            "abstract": "Recently, sentiment analysis has received a lot of attention due to the\ninterest in mining opinions of social media users. Sentiment analysis consists\nin determining the polarity of a given text, i.e., its degree of positiveness\nor negativeness. Traditionally, Sentiment Analysis algorithms have been\ntailored to a specific language given the complexity of having a number of\nlexical variations and errors introduced by the people generating content. In\nthis contribution, our aim is to provide a simple to implement and easy to use\nmultilingual framework, that can serve as a baseline for sentiment analysis\ncontests, and as starting point to build new sentiment analysis systems. We\ncompare our approach in eight different languages, three of them have important\ninternational contests, namely, SemEval (English), TASS (Spanish), and\nSENTIPOLC (Italian). Within the competitions our approach reaches from medium\nto high positions in the rankings; whereas in the remaining languages our\napproach outperforms the reported results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.05688v3",
            "date": "2016-12-17 01:03:55+00:00",
            "title": "A User Simulator for Task-Completion Dialogues",
            "authors": [
                "Xiujun Li",
                "Zachary C. Lipton",
                "Bhuwan Dhingra",
                "Lihong Li",
                "Jianfeng Gao",
                "Yun-Nung Chen"
            ],
            "abstract": "Despite widespread interests in reinforcement-learning for task-oriented\ndialogue systems, several obstacles can frustrate research and development\nprogress. First, reinforcement learners typically require interaction with the\nenvironment, so conventional dialogue corpora cannot be used directly. Second,\neach task presents specific challenges, requiring separate corpus of\ntask-specific annotated data. Third, collecting and annotating human-machine or\nhuman-human conversations for task-oriented dialogues requires extensive domain\nknowledge. Because building an appropriate dataset can be both financially\ncostly and time-consuming, one popular approach is to build a user simulator\nbased upon a corpus of example dialogues. Then, one can train reinforcement\nlearning agents in an online fashion as they interact with the simulator.\nDialogue agents trained on these simulators can serve as an effective starting\npoint. Once agents master the simulator, they may be deployed in a real\nenvironment to interact with humans, and continue to be trained online. To ease\nempirical algorithmic comparisons in dialogues, this paper introduces a new,\npublicly available simulation framework, where our simulator, designed for the\nmovie-booking domain, leverages both rules and collected data. The simulator\nsupports two tasks: movie ticket booking and movie seeking. Finally, we\ndemonstrate several agents and detail the procedure to add and test your own\nagent in the proposed framework.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1612.05734v1",
            "date": "2016-12-17 11:36:06+00:00",
            "title": "Web-based Semantic Similarity for Emotion Recognition in Web Objects",
            "authors": [
                "Valentina Franzoni",
                "Giulio Biondi",
                "Alfredo Milani",
                "Yuanxi Li"
            ],
            "abstract": "In this project we propose a new approach for emotion recognition using\nweb-based similarity (e.g. confidence, PMI and PMING). We aim to extract basic\nemotions from short sentences with emotional content (e.g. news titles, tweets,\ncaptions), performing a web-based quantitative evaluation of semantic proximity\nbetween each word of the analyzed sentence and each emotion of a psychological\nmodel (e.g. Plutchik, Ekman, Lovheim). The phases of the extraction include:\ntext preprocessing (tokenization, stop words, filtering), search engine\nautomated query, HTML parsing of results (i.e. scraping), estimation of\nsemantic proximity, ranking of emotions according to proximity measures. The\nmain idea is that, since it is possible to generalize semantic similarity under\nthe assumption that similar concepts co-occur in documents indexed in search\nengines, therefore also emotions can be generalized in the same way, through\ntags or terms that express them in a particular language, ranking emotions.\nTraining results are compared to human evaluation, then additional comparative\ntests on results are performed, both for the global ranking correlation (e.g.\nKendall, Spearman, Pearson) both for the evaluation of the emotion linked to\neach single word. Different from sentiment analysis, our approach works at a\ndeeper level of abstraction, aiming at recognizing specific emotions and not\nonly the positive/negative sentiment, in order to predict emotions as semantic\ndata.",
            "journal ref": "In Proc 9th International Conference on Utility and Cloud\n  Computing (UCC 2016). ACM, New York, NY, USA, 327-332",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.06212v1",
            "date": "2016-12-19 14:59:14+00:00",
            "title": "A recurrent neural network without chaos",
            "authors": [
                "Thomas Laurent",
                "James von Brecht"
            ],
            "abstract": "We introduce an exceptionally simple gated recurrent neural network (RNN)\nthat achieves performance comparable to well-known gated architectures, such as\nLSTMs and GRUs, on the word-level language modeling task. We prove that our\nmodel has simple, predicable and non-chaotic dynamics. This stands in stark\ncontrast to more standard gated architectures, whose underlying dynamical\nsystems exhibit chaotic behavior.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1612.06391v1",
            "date": "2016-12-19 21:00:02+00:00",
            "title": "Talk it up or play it down? (Un)expected correlations between (de-)emphasis and recurrence of discussion points in consequential U.S. economic policy meetings",
            "authors": [
                "Chenhao Tan",
                "Lillian Lee"
            ],
            "abstract": "In meetings where important decisions get made, what items receive more\nattention may influence the outcome. We examine how different types of\nrhetorical (de-)emphasis -- including hedges, superlatives, and contrastive\nconjunctions -- correlate with what gets revisited later, controlling for item\nfrequency and speaker. Our data consists of transcripts of recurring meetings\nof the Federal Reserve's Open Market Committee (FOMC), where important aspects\nof U.S. monetary policy are decided on. Surprisingly, we find that words\nappearing in the context of hedging, which is usually considered a way to\nexpress uncertainty, are more likely to be repeated in subsequent meetings,\nwhile strong emphasis indicated by superlatives has a slightly negative effect\non word recurrence in subsequent meetings. We also observe interesting patterns\nin how these effects vary depending on social factors such as status and gender\nof the speaker. For instance, the positive effects of hedging are more\npronounced for female speakers than for male speakers.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1612.06890v1",
            "date": "2016-12-20 21:40:40+00:00",
            "title": "CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning",
            "authors": [
                "Justin Johnson",
                "Bharath Hariharan",
                "Laurens van der Maaten",
                "Li Fei-Fei",
                "C. Lawrence Zitnick",
                "Ross Girshick"
            ],
            "abstract": "When building artificial intelligence systems that can reason and answer\nquestions about visual data, we need diagnostic tests to analyze our progress\nand discover shortcomings. Existing benchmarks for visual question answering\ncan help, but have strong biases that models can exploit to correctly answer\nquestions without reasoning. They also conflate multiple sources of error,\nmaking it hard to pinpoint model weaknesses. We present a diagnostic dataset\nthat tests a range of visual reasoning abilities. It contains minimal biases\nand has detailed annotations describing the kind of reasoning each question\nrequires. We use this dataset to analyze a variety of modern visual reasoning\nsystems, providing novel insights into their abilities and limitations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.07843v1",
            "date": "2016-12-23 00:31:30+00:00",
            "title": "\"What is Relevant in a Text Document?\": An Interpretable Machine Learning Approach",
            "authors": [
                "Leila Arras",
                "Franziska Horn",
                "Gr\u00e9goire Montavon",
                "Klaus-Robert M\u00fcller",
                "Wojciech Samek"
            ],
            "abstract": "Text documents can be described by a number of abstract concepts such as\nsemantic category, writing style, or sentiment. Machine learning (ML) models\nhave been trained to automatically map documents to these abstract concepts,\nallowing to annotate very large text collections, more than could be processed\nby a human in a lifetime. Besides predicting the text's category very\naccurately, it is also highly desirable to understand how and why the\ncategorization process takes place. In this paper, we demonstrate that such\nunderstanding can be achieved by tracing the classification decision back to\nindividual words using layer-wise relevance propagation (LRP), a recently\ndeveloped technique for explaining predictions of complex non-linear\nclassifiers. We train two word-based ML models, a convolutional neural network\n(CNN) and a bag-of-words SVM classifier, on a topic categorization task and\nadapt the LRP method to decompose the predictions of these models onto words.\nResulting scores indicate how much individual words contribute to the overall\nclassification decision. This enables one to distill relevant information from\ntext documents without an explicit semantic information extraction step. We\nfurther use the word-wise relevance scores for generating novel vector-based\ndocument representations which capture semantic information. Based on these\ndocument vectors, we introduce a measure of model explanatory power and show\nthat, although the SVM and CNN models perform similarly in terms of\nclassification accuracy, the latter exhibits a higher level of explainability\nwhich makes it more comprehensible for humans and potentially more useful for\nother applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08354v1",
            "date": "2016-12-26 09:51:18+00:00",
            "title": "Image-Text Multi-Modal Representation Learning by Adversarial Backpropagation",
            "authors": [
                "Gwangbeen Park",
                "Woobin Im"
            ],
            "abstract": "We present novel method for image-text multi-modal representation learning.\nIn our knowledge, this work is the first approach of applying adversarial\nlearning concept to multi-modal learning and not exploiting image-text pair\ninformation to learn multi-modal feature. We only use category information in\ncontrast with most previous methods using image-text pair information for\nmulti-modal embedding. In this paper, we show that multi-modal feature can be\nachieved without image-text pair information and our method makes more similar\ndistribution with image and text in multi-modal feature space than other\nmethods which use image-text pair information. And we show our multi-modal\nfeature has universal semantic information, even though it was trained for\ncategory prediction. Our model is end-to-end backpropagation, intuitive and\neasily extended to other multi-modal learning work.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.09268v1",
            "date": "2016-12-27 21:58:42+00:00",
            "title": "The ontogeny of discourse structure mimics the development of literature",
            "authors": [
                "Natalia Bezerra Mota",
                "Sylvia Pinheiro",
                "Mariano Sigman",
                "Diego Fernandez Slezak",
                "Guillermo Cecchi",
                "Mauro Copelli",
                "Sidarta Ribeiro"
            ],
            "abstract": "Discourse varies with age, education, psychiatric state and historical epoch,\nbut the ontogenetic and cultural dynamics of discourse structure remain to be\nquantitatively characterized. To this end we investigated word graphs obtained\nfrom verbal reports of 200 subjects ages 2-58, and 676 literary texts spanning\n~5,000 years. In healthy subjects, lexical diversity, graph size, and\nlong-range recurrence departed from initial near-random levels through a\nmonotonic asymptotic increase across ages, while short-range recurrence showed\na corresponding decrease. These changes were explained by education and suggest\na hierarchical development of discourse structure: short-range recurrence and\nlexical diversity stabilize after elementary school, but graph size and\nlong-range recurrence only stabilize after high school. This gradual maturation\nwas blurred in psychotic subjects, who maintained in adulthood a near-random\nstructure. In literature, monotonic asymptotic changes over time were\nremarkable: While lexical diversity, long-range recurrence and graph size\nincreased away from near-randomness, short-range recurrence declined, from\nabove to below random levels. Bronze Age texts are structurally similar to\nchildish or psychotic discourses, but subsequent texts converge abruptly to the\nhealthy adult pattern around the onset of the Axial Age (800-200 BC), a period\nof pivotal cultural change. Thus, individually as well as historically,\ndiscourse maturation increases the range of word recurrence away from\nrandomness.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1612.09542v2",
            "date": "2016-12-30 17:39:19+00:00",
            "title": "A Joint Speaker-Listener-Reinforcer Model for Referring Expressions",
            "authors": [
                "Licheng Yu",
                "Hao Tan",
                "Mohit Bansal",
                "Tamara L. Berg"
            ],
            "abstract": "Referring expressions are natural language constructions used to identify\nparticular objects within a scene. In this paper, we propose a unified\nframework for the tasks of referring expression comprehension and generation.\nOur model is composed of three modules: speaker, listener, and reinforcer. The\nspeaker generates referring expressions, the listener comprehends referring\nexpressions, and the reinforcer introduces a reward function to guide sampling\nof more discriminative expressions. The listener-speaker modules are trained\njointly in an end-to-end learning framework, allowing the modules to be aware\nof one another during learning while also benefiting from the discriminative\nreinforcer's feedback. We demonstrate that this unified framework and training\nachieves state-of-the-art results for both comprehension and generation on\nthree referring expression datasets. Project and demo page:\nhttps://vision.cs.unc.edu/refer",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.09574v1",
            "date": "2016-12-30 19:52:09+00:00",
            "title": "Automatic Data Deformation Analysis on Evolving Folksonomy Driven Environment",
            "authors": [
                "Massimiliano Dal Mas"
            ],
            "abstract": "The Folksodriven framework makes it possible for data scientists to define an\nontology environment where searching for buried patterns that have some kind of\npredictive power to build predictive models more effectively. It accomplishes\nthis through an abstractions that isolate parameters of the predictive modeling\nprocess searching for patterns and designing the feature set, too. To reflect\nthe evolving knowledge, this paper considers ontologies based on folksonomies\naccording to a new concept structure called \"Folksodriven\" to represent\nfolksonomies. So, the studies on the transformational regulation of the\nFolksodriven tags are regarded to be important for adaptive folksonomies\nclassifications in an evolving environment used by Intelligent Systems to\nrepresent the knowledge sharing. Folksodriven tags are used to categorize\nsalient data points so they can be fed to a machine-learning system and\n\"featurizing\" the data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1603.00106v2",
            "date": "2016-03-01 00:45:18+00:00",
            "title": "Characterizing Diseases from Unstructured Text: A Vocabulary Driven Word2vec Approach",
            "authors": [
                "Saurav Ghosh",
                "Prithwish Chakraborty",
                "Emily Cohn",
                "John S. Brownstein",
                "Naren Ramakrishnan"
            ],
            "abstract": "Traditional disease surveillance can be augmented with a wide variety of\nreal-time sources such as, news and social media. However, these sources are in\ngeneral unstructured and, construction of surveillance tools such as\ntaxonomical correlations and trace mapping involves considerable human\nsupervision. In this paper, we motivate a disease vocabulary driven word2vec\nmodel (Dis2Vec) to model diseases and constituent attributes as word embeddings\nfrom the HealthMap news corpus. We use these word embeddings to automatically\ncreate disease taxonomies and evaluate our model against corresponding human\nannotated taxonomies. We compare our model accuracies against several\nstate-of-the art word2vec methods. Our results demonstrate that Dis2Vec\noutperforms traditional distributed vector representations in its ability to\nfaithfully capture taxonomical attributes across different class of diseases\nsuch as endemic, emerging and rare.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1603.00223v2",
            "date": "2016-03-01 10:43:43+00:00",
            "title": "Segmental Recurrent Neural Networks for End-to-end Speech Recognition",
            "authors": [
                "Liang Lu",
                "Lingpeng Kong",
                "Chris Dyer",
                "Noah A. Smith",
                "Steve Renals"
            ],
            "abstract": "We study the segmental recurrent neural network for end-to-end acoustic\nmodelling. This model connects the segmental conditional random field (CRF)\nwith a recurrent neural network (RNN) used for feature extraction. Compared to\nmost previous CRF-based acoustic models, it does not rely on an external system\nto provide features or segmentation boundaries. Instead, this model\nmarginalises out all the possible segmentations, and features are extracted\nfrom the RNN trained together with the segmental CRF. In essence, this model is\nself-contained and can be trained end-to-end. In this paper, we discuss\npractical training and decoding issues as well as the method to speed up the\ntraining in the context of speech recognition. We performed experiments on the\nTIMIT dataset. We achieved 17.3 phone error rate (PER) from the first-pass\ndecoding --- the best reported result using CRFs, despite the fact that we only\nused a zeroth-order CRF and without using any language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.00423v1",
            "date": "2016-03-01 19:45:25+00:00",
            "title": "Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs",
            "authors": [
                "Phong Le",
                "Willem Zuidema"
            ],
            "abstract": "Recursive neural networks (RNN) and their recently proposed extension\nrecursive long short term memory networks (RLSTM) are models that compute\nrepresentations for sentences, by recursively combining word embeddings\naccording to an externally provided parse tree. Both models thus, unlike\nrecurrent networks, explicitly make use of the hierarchical structure of a\nsentence. In this paper, we demonstrate that RNNs nevertheless suffer from the\nvanishing gradient and long distance dependency problem, and that RLSTMs\ngreatly improve over RNN's on these problems. We present an artificial learning\ntask that allows us to quantify the severity of these problems for both models.\nWe further show that a ratio of gradients (at the root node and a focal leaf\nnode) is highly indicative of the success of backpropagation at optimizing the\nrelevant weights low in the tree. This paper thus provides an explanation for\nexisting, superior results of RLSTMs on tasks such as sentiment analysis, and\nsuggests that the benefits of including hierarchical structure and of including\nLSTM-style gating are complementary.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1603.00810v3",
            "date": "2016-03-02 18:01:57+00:00",
            "title": "Character-based Neural Machine Translation",
            "authors": [
                "Marta R. Costa-Juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Neural Machine Translation (MT) has reached state-of-the-art results.\nHowever, one of the main challenges that neural MT still faces is dealing with\nvery large vocabularies and morphologically rich languages. In this paper, we\npropose a neural MT system using character-based embeddings in combination with\nconvolutional and highway layers to replace the standard lookup-based word\nrepresentations. The resulting unlimited-vocabulary and affix-aware source word\nembeddings are tested in a state-of-the-art neural MT based on an\nattention-based bidirectional recurrent neural network. The proposed MT scheme\nprovides improved results even when the source language is not morphologically\nrich. Improvements up to 3 BLEU points are obtained in the German-English WMT\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01354v5",
            "date": "2016-03-04 05:55:02+00:00",
            "title": "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF",
            "authors": [
                "Xuezhe Ma",
                "Eduard Hovy"
            ],
            "abstract": "State-of-the-art sequence labeling systems traditionally require large\namounts of task-specific knowledge in the form of hand-crafted features and\ndata pre-processing. In this paper, we introduce a novel neutral network\narchitecture that benefits from both word- and character-level representations\nautomatically, by using combination of bidirectional LSTM, CNN and CRF. Our\nsystem is truly end-to-end, requiring no feature engineering or data\npre-processing, thus making it applicable to a wide range of sequence labeling\ntasks. We evaluate our system on two data sets for two sequence labeling tasks\n--- Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003\ncorpus for named entity recognition (NER). We obtain state-of-the-art\nperformance on both the two data --- 97.55\\% accuracy for POS tagging and\n91.21\\% F1 for NER.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1603.01417v1",
            "date": "2016-03-04 10:40:28+00:00",
            "title": "Dynamic Memory Networks for Visual and Textual Question Answering",
            "authors": [
                "Caiming Xiong",
                "Stephen Merity",
                "Richard Socher"
            ],
            "abstract": "Neural network architectures with memory and attention mechanisms exhibit\ncertain reasoning capabilities required for question answering. One such\narchitecture, the dynamic memory network (DMN), obtained high accuracy on a\nvariety of language tasks. However, it was not shown whether the architecture\nachieves strong results for question answering when supporting facts are not\nmarked during training or whether it could be applied to other modalities such\nas images. Based on an analysis of the DMN, we propose several improvements to\nits memory and input modules. Together with these changes we introduce a novel\ninput module for images in order to be able to answer visual questions. Our new\nDMN+ model improves the state of the art on both the Visual Question Answering\ndataset and the \\babi-10k text question-answering dataset without supporting\nfact supervision.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1603.01597v2",
            "date": "2016-03-04 20:13:56+00:00",
            "title": "Integrated Sequence Tagging for Medieval Latin Using Deep Representation Learning",
            "authors": [
                "Mike Kestemont",
                "Jeroen De Gussem"
            ],
            "abstract": "In this paper we consider two sequence tagging tasks for medieval Latin:\npart-of-speech tagging and lemmatization. These are both basic, yet\nfoundational preprocessing steps in applications such as text re-use detection.\nNevertheless, they are generally complicated by the considerable orthographic\nvariation which is typical of medieval Latin. In Digital Classics, these tasks\nare traditionally solved in a (i) cascaded and (ii) lexicon-dependent fashion.\nFor example, a lexicon is used to generate all the potential lemma-tag pairs\nfor a token, and next, a context-aware PoS-tagger is used to select the most\nappropriate tag-lemma pair. Apart from the problems with out-of-lexicon items,\nerror percolation is a major downside of such approaches. In this paper we\nexplore the possibility to elegantly solve these tasks using a single,\nintegrated approach. For this, we make use of a layered neural network\narchitecture from the field of deep representation learning.",
            "journal ref": "Journal of Data Mining & Digital Humanities, Special Issue on\n  Computer-Aided Processing of Intertextuality in Ancient Languages, Towards a\n  Digital Ecosystem: NLP. Corpus infrastructure. Methods for Retrieving Texts\n  and Computing Text Similarities (August 6, 2017) jdmdh:3835",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.01913v2",
            "date": "2016-03-07 01:54:56+00:00",
            "title": "A Latent Variable Recurrent Neural Network for Discourse Relation Language Models",
            "authors": [
                "Yangfeng Ji",
                "Gholamreza Haffari",
                "Jacob Eisenstein"
            ],
            "abstract": "This paper presents a novel latent variable recurrent neural network\narchitecture for jointly modeling sequences of words and (possibly latent)\ndiscourse relations between adjacent sentences. A recurrent neural network\ngenerates individual words, thus reaping the benefits of\ndiscriminatively-trained vector representations. The discourse relations are\nrepresented with a latent variable, which can be predicted or marginalized,\ndepending on the task. The resulting model can therefore employ a training\nobjective that includes not only discourse relation classification, but also\nword prediction. As a result, it outperforms state-of-the-art alternatives for\ntwo tasks: implicit discourse relation classification in the Penn Discourse\nTreebank, and dialog act classification in the Switchboard corpus. Furthermore,\nby marginalizing over latent discourse relations at test time, we obtain a\ndiscourse informed language model, which improves over a strong LSTM baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.02776v1",
            "date": "2016-03-09 03:13:37+00:00",
            "title": "Implicit Discourse Relation Classification via Multi-Task Neural Networks",
            "authors": [
                "Yang Liu",
                "Sujian Li",
                "Xiaodong Zhang",
                "Zhifang Sui"
            ],
            "abstract": "Without discourse connectives, classifying implicit discourse relations is a\nchallenging task and a bottleneck for building a practical discourse parser.\nPrevious research usually makes use of one kind of discourse framework such as\nPDTB or RST to improve the classification performance on discourse relations.\nActually, under different discourse annotation frameworks, there exist multiple\ncorpora which have internal connections. To exploit the combination of\ndifferent discourse corpora, we design related discourse classification tasks\nspecific to a corpus, and propose a novel Convolutional Neural Network embedded\nmulti-task learning system to synthesize these tasks by learning both unique\nand shared representations for each task. The experimental results on the PDTB\nimplicit discourse relation classification task demonstrate that our model\nachieves significant gains over baseline systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03170v2",
            "date": "2016-03-10 07:43:15+00:00",
            "title": "Data fluidity in DARIAH -- pushing the agenda forward",
            "authors": [
                "Laurent Romary",
                "Mike Mertens",
                "Anne Baillot"
            ],
            "abstract": "This paper provides both an update concerning the setting up of the European\nDARIAH infrastructure and a series of strong action lines related to the\ndevelopment of a data centred strategy for the humanities in the coming years.\nIn particular we tackle various aspect of data management: data hosting, the\nsetting up of a DARIAH seal of approval, the establishment of a charter between\ncultural heritage institutions and scholars and finally a specific view on\ncertification mechanisms for data.",
            "journal ref": "BIBLIOTHEK Forschung und Praxis, De Gruyter, 2016, 39 (3),\n  pp.350-357",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1603.03185v2",
            "date": "2016-03-10 08:51:51+00:00",
            "title": "Personalized Speech recognition on mobile devices",
            "authors": [
                "Ian McGraw",
                "Rohit Prabhavalkar",
                "Raziel Alvarez",
                "Montse Gonzalez Arenas",
                "Kanishka Rao",
                "David Rybach",
                "Ouais Alsharif",
                "Hasim Sak",
                "Alexander Gruenstein",
                "Francoise Beaufays",
                "Carolina Parada"
            ],
            "abstract": "We describe a large vocabulary speech recognition system that is accurate,\nhas low latency, and yet has a small enough memory and computational footprint\nto run faster than real-time on a Nexus 5 Android smartphone. We employ a\nquantized Long Short-Term Memory (LSTM) acoustic model trained with\nconnectionist temporal classification (CTC) to directly predict phoneme\ntargets, and further reduce its memory footprint using an SVD-based compression\nscheme. Additionally, we minimize our memory footprint by using a single\nlanguage model for both dictation and voice command domains, constructed using\nBayesian interpolation. Finally, in order to properly handle device-specific\ninformation, such as proper names and other context-dependent information, we\ninject vocabulary items into the decoder graph and bias the language model\non-the-fly. Our system achieves 13.5% word error rate on an open-ended\ndictation task, running with a median speed that is seven times faster than\nreal-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.03784v1",
            "date": "2016-03-11 21:09:19+00:00",
            "title": "Towards using social media to identify individuals at risk for preventable chronic illness",
            "authors": [
                "Dane Bell",
                "Daniel Fried",
                "Luwen Huangfu",
                "Mihai Surdeanu",
                "Stephen Kobourov"
            ],
            "abstract": "We describe a strategy for the acquisition of training data necessary to\nbuild a social-media-driven early detection system for individuals at risk for\n(preventable) type 2 diabetes mellitus (T2DM). The strategy uses a game-like\nquiz with data and questions acquired semi-automatically from Twitter. The\nquestions are designed to inspire participant engagement and collect relevant\ndata to train a public-health model applied to individuals. Prior systems\ndesigned to use social media such as Twitter to predict obesity (a risk factor\nfor T2DM) operate on entire communities such as states, counties, or cities,\nbased on statistics gathered by government agencies. Because there is\nconsiderable variation among individuals within these groups, training data on\nthe individual level would be more effective, but this data is difficult to\nacquire. The approach proposed here aims to address this issue. Our strategy\nhas two steps. First, we trained a random forest classifier on data gathered\nfrom (public) Twitter statuses and state-level statistics with state-of-the-art\naccuracy. We then converted this classifier into a 20-questions-style quiz and\nmade it available online. In doing so, we achieved high engagement with\nindividuals that took the quiz, while also building a training set of\nvoluntarily supplied individual-level data for future classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06042v2",
            "date": "2016-03-19 03:56:03+00:00",
            "title": "Globally Normalized Transition-Based Neural Networks",
            "authors": [
                "Daniel Andor",
                "Chris Alberti",
                "David Weiss",
                "Aliaksei Severyn",
                "Alessandro Presta",
                "Kuzman Ganchev",
                "Slav Petrov",
                "Michael Collins"
            ],
            "abstract": "We introduce a globally normalized transition-based neural network model that\nachieves state-of-the-art part-of-speech tagging, dependency parsing and\nsentence compression results. Our model is a simple feed-forward neural network\nthat operates on a task-specific transition system, yet achieves comparable or\nbetter accuracies than recurrent models. We discuss the importance of global as\nopposed to local normalization: a key insight is that the label bias problem\nimplies that globally normalized models can be strictly more expressive than\nlocally normalized models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06059v3",
            "date": "2016-03-19 07:27:15+00:00",
            "title": "Generating Natural Questions About an Image",
            "authors": [
                "Nasrin Mostafazadeh",
                "Ishan Misra",
                "Jacob Devlin",
                "Margaret Mitchell",
                "Xiaodong He",
                "Lucy Vanderwende"
            ],
            "abstract": "There has been an explosion of work in the vision & language community during\nthe past few years from image captioning to video transcription, and answering\nquestions about images. These tasks have focused on literal descriptions of the\nimage. To move beyond the literal, we choose to explore how questions about an\nimage are often directed at commonsense inference and the abstract events\nevoked by objects in the image. In this paper, we introduce the novel task of\nVisual Question Generation (VQG), where the system is tasked with asking a\nnatural and engaging question when shown an image. We provide three datasets\nwhich cover a variety of images from object-centric to event-centric, with\nconsiderably more abstract training data than provided to state-of-the-art\ncaptioning systems thus far. We train and test several generative and retrieval\nmodels to tackle the task of VQG. Evaluation results show that while such\nmodels ask reasonable questions for a variety of images, there is still a wide\ngap with human performance which motivates further work on connecting images\nwith commonsense knowledge and pragmatics. Our proposed task offers a new\nchallenge to the community which we hope furthers interest in exploring deeper\nconnections between vision & language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06111v2",
            "date": "2016-03-19 16:38:31+00:00",
            "title": "How Transferable are Neural Networks in NLP Applications?",
            "authors": [
                "Lili Mou",
                "Zhao Meng",
                "Rui Yan",
                "Ge Li",
                "Yan Xu",
                "Lu Zhang",
                "Zhi Jin"
            ],
            "abstract": "Transfer learning is aimed to make use of valuable knowledge in a source\ndomain to help model performance in a target domain. It is particularly\nimportant to neural networks, which are very likely to be overfitting. In some\nfields like image processing, many studies have shown the effectiveness of\nneural network-based transfer learning. For neural NLP, however, existing\nstudies have only casually applied transfer learning, and conclusions are\ninconsistent. In this paper, we conduct systematic case studies and provide an\nilluminating picture on the transferability of neural networks in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06127v4",
            "date": "2016-03-19 18:35:26+00:00",
            "title": "Sentence Pair Scoring: Towards Unified Framework for Text Comprehension",
            "authors": [
                "Petr Baudi\u0161",
                "Jan Pichl",
                "Tom\u00e1\u0161 Vysko\u010dil",
                "Jan \u0160ediv\u00fd"
            ],
            "abstract": "We review the task of Sentence Pair Scoring, popular in the literature in\nvarious forms - viewed as Answer Sentence Selection, Semantic Text Scoring,\nNext Utterance Ranking, Recognizing Textual Entailment, Paraphrasing or e.g. a\ncomponent of Memory Networks.\n  We argue that all such tasks are similar from the model perspective and\npropose new baselines by comparing the performance of common IR metrics and\npopular convolutional, recurrent and attention-based neural models across many\nSentence Pair Scoring tasks and datasets. We discuss the problem of evaluating\nrandomized models, propose a statistically grounded methodology, and attempt to\nimprove comparisons by releasing new datasets that are much harder than some of\nthe currently used well explored benchmarks. We introduce a unified open source\nsoftware framework with easily pluggable models and tasks, which enables us to\nexperiment with multi-task reusability of trained sentence model. We set a new\nstate-of-art in performance on the Ubuntu Dialogue dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06318v6",
            "date": "2016-03-21 03:33:20+00:00",
            "title": "Harnessing Deep Neural Networks with Logic Rules",
            "authors": [
                "Zhiting Hu",
                "Xuezhe Ma",
                "Zhengzhong Liu",
                "Eduard Hovy",
                "Eric Xing"
            ],
            "abstract": "Combining deep neural networks with structured logic rules is desirable to\nharness flexibility and reduce uninterpretability of the neural models. We\npropose a general framework capable of enhancing various types of neural\nnetworks (e.g., CNNs and RNNs) with declarative first-order logic rules.\nSpecifically, we develop an iterative distillation method that transfers the\nstructured information of logic rules into the weights of neural networks. We\ndeploy the framework on a CNN for sentiment analysis, and an RNN for named\nentity recognition. With a few highly intuitive rules, we obtain substantial\nimprovements and achieve state-of-the-art or comparable results to previous\nbest-performing systems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1603.06393v3",
            "date": "2016-03-21 11:35:08+00:00",
            "title": "Incorporating Copying Mechanism in Sequence-to-Sequence Learning",
            "authors": [
                "Jiatao Gu",
                "Zhengdong Lu",
                "Hang Li",
                "Victor O. K. Li"
            ],
            "abstract": "We address an important problem in sequence-to-sequence (Seq2Seq) learning\nreferred to as copying, in which certain segments in the input sequence are\nselectively replicated in the output sequence. A similar phenomenon is\nobservable in human language communication. For example, humans tend to repeat\nentity names or even long phrases in conversation. The challenge with regard to\ncopying in Seq2Seq is that new machinery is needed to decide when to perform\nthe operation. In this paper, we incorporate copying into neural network-based\nSeq2Seq learning and propose a new model called CopyNet with encoder-decoder\nstructure. CopyNet can nicely integrate the regular way of word generation in\nthe decoder with the new copying mechanism which can choose sub-sequences in\nthe input sequence and put them at proper places in the output sequence. Our\nempirical study on both synthetic data sets and real world data sets\ndemonstrates the efficacy of CopyNet. For example, CopyNet can outperform\nregular RNN-based model with remarkable margins on text summarization tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06485v1",
            "date": "2016-03-21 16:34:13+00:00",
            "title": "A System for Probabilistic Linking of Thesauri and Classification Systems",
            "authors": [
                "Lisa Posch",
                "Philipp Schaer",
                "Arnim Bleier",
                "Markus Strohmaier"
            ],
            "abstract": "This paper presents a system which creates and visualizes probabilistic\nsemantic links between concepts in a thesaurus and classes in a classification\nsystem. For creating the links, we build on the Polylingual Labeled Topic Model\n(PLL-TM). PLL-TM identifies probable thesaurus descriptors for each class in\nthe classification system by using information from the natural language text\nof documents, their assigned thesaurus descriptors and their designated\nclasses. The links are then presented to users of the system in an interactive\nvisualization, providing them with an automatically generated overview of the\nrelations between the thesaurus and the classification system.",
            "journal ref": "KI - K\\\"unstliche Intelligenz, 2015",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1603.06679v3",
            "date": "2016-03-22 05:59:00+00:00",
            "title": "Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis",
            "authors": [
                "Wenya Wang",
                "Sinno Jialin Pan",
                "Daniel Dahlmeier",
                "Xiaokui Xiao"
            ],
            "abstract": "In aspect-based sentiment analysis, extracting aspect terms along with the\nopinions being expressed from user-generated content is one of the most\nimportant subtasks. Previous studies have shown that exploiting connections\nbetween aspect and opinion terms is promising for this task. In this paper, we\npropose a novel joint model that integrates recursive neural networks and\nconditional random fields into a unified framework for explicit aspect and\nopinion terms co-extraction. The proposed model learns high-level\ndiscriminative features and double propagate information between aspect and\nopinion terms, simultaneously. Moreover, it is flexible to incorporate\nhand-crafted features into the proposed model to further boost its information\nextraction performance. Experimental results on the SemEval Challenge 2014\ndataset show the superiority of our proposed model over several baseline\nmethods as well as the winning systems of the challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.06807v2",
            "date": "2016-03-22 14:25:16+00:00",
            "title": "Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus",
            "authors": [
                "Iulian Vlad Serban",
                "Alberto Garc\u00eda-Dur\u00e1n",
                "Caglar Gulcehre",
                "Sungjin Ahn",
                "Sarath Chandar",
                "Aaron Courville",
                "Yoshua Bengio"
            ],
            "abstract": "Over the past decade, large-scale supervised learning corpora have enabled\nmachine learning researchers to make substantial advances. However, to this\ndate, there are no large-scale question-answer corpora available. In this paper\nwe present the 30M Factoid Question-Answer Corpus, an enormous question answer\npair corpus produced by applying a novel neural network architecture on the\nknowledge base Freebase to transduce facts into natural language questions. The\nproduced question answer pairs are evaluated both by human evaluators and using\nautomatic evaluation metrics, including well-established machine translation\nand sentence similarity metrics. Across all evaluation criteria the\nquestion-generation model outperforms the competing template-based baseline.\nFurthermore, when presented to human evaluators, the generated questions appear\ncomparable in quality to real human-generated questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07044v1",
            "date": "2016-03-23 01:52:54+00:00",
            "title": "Recurrent Neural Network Encoder with Attention for Community Question Answering",
            "authors": [
                "Wei-Ning Hsu",
                "Yu Zhang",
                "James Glass"
            ],
            "abstract": "We apply a general recurrent neural network (RNN) encoder framework to\ncommunity question answering (cQA) tasks. Our approach does not rely on any\nlinguistic processing, and can be applied to different languages or domains.\nFurther improvements are observed when we extend the RNN encoders with a neural\nattention mechanism that encourages reasoning over entire sequences. To deal\nwith practical issues such as data sparsity and imbalanced labels, we apply\nvarious techniques such as transfer learning and multitask learning. Our\nexperiments on the SemEval-2016 cQA task show 10% improvement on a MAP score\ncompared to an information retrieval-based approach, and achieve comparable\nperformance to a strong handcrafted feature-based method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07150v1",
            "date": "2016-03-23 12:02:12+00:00",
            "title": "The Anatomy of a Search and Mining System for Digital Archives",
            "authors": [
                "Martyn Harris",
                "Mark Levene",
                "Dell Zhang",
                "Dan Levene"
            ],
            "abstract": "Samtla (Search And Mining Tools with Linguistic Analysis) is a digital\nhumanities system designed in collaboration with historians and linguists to\nassist them with their research work in quantifying the content of any textual\ncorpora through approximate phrase search and document comparison. The\nretrieval engine uses a character-based n-gram language model rather than the\nconventional word-based one so as to achieve great flexibility in language\nagnostic query processing.\n  The index is implemented as a space-optimised character-based suffix tree\nwith an accompanying database of document content and metadata. A number of\ntext mining tools are integrated into the system to allow researchers to\ndiscover textual patterns, perform comparative analysis, and find out what is\ncurrently popular in the research community.\n  Herein we describe the system architecture, user interface, models and\nalgorithms, and data storage of the Samtla system. We also present several case\nstudies of its usage in practice together with an evaluation of the systems'\nranking performance through crowdsourcing.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07313v1",
            "date": "2016-03-23 19:26:20+00:00",
            "title": "CONDITOR1: Topic Maps and DITA labelling tool for textual documents with historical information",
            "authors": [
                "Piedad Garrido",
                "Jesus Tramullas",
                "Manuel Coll"
            ],
            "abstract": "Conditor is a software tool which works with textual documents containing\nhistorical information. The purpose of this work two-fold: firstly to show the\nvalidity of the developed engine to correctly identify and label the entities\nof the universe of discourse with a labelled-combined XTM-DITA model. Secondly\nto explain the improvements achieved in the information retrieval process\nthanks to the use of a object-oriented database (JPOX) as well as its\nintegration into the Lucene-type database search process to not only accomplish\nmore accurate searches, but to also help the future development of a\nrecommender system. We finish with a brief demo in a 3D-graph of the results of\nthe aforementioned search.",
            "journal ref": "Journal of Digital Information, 10, 4, 2009",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07624v1",
            "date": "2016-03-24 15:22:52+00:00",
            "title": "Semantic Properties of Customer Sentiment in Tweets",
            "authors": [
                "Eun Hee Ko",
                "Diego Klabjan"
            ],
            "abstract": "An increasing number of people are using online social networking services\n(SNSs), and a significant amount of information related to experiences in\nconsumption is shared in this new media form. Text mining is an emerging\ntechnique for mining useful information from the web. We aim at discovering in\nparticular tweets semantic patterns in consumers' discussions on social media.\nSpecifically, the purposes of this study are twofold: 1) finding similarity and\ndissimilarity between two sets of textual documents that include consumers'\nsentiment polarities, two forms of positive vs. negative opinions and 2)\ndriving actual content from the textual data that has a semantic trend. The\nconsidered tweets include consumers opinions on US retail companies (e.g.,\nAmazon, Walmart). Cosine similarity and K-means clustering methods are used to\nachieve the former goal, and Latent Dirichlet Allocation (LDA), a popular topic\nmodeling algorithm, is used for the latter purpose. This is the first study\nwhich discover semantic properties of textual data in consumption context\nbeyond sentiment analysis. In addition to major findings, we apply LDA (Latent\nDirichlet Allocations) to the same data and drew latent topics that represent\nconsumers' positive opinions and negative opinions on social media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.07646v1",
            "date": "2016-03-24 16:39:37+00:00",
            "title": "Recursive Neural Language Architecture for Tag Prediction",
            "authors": [
                "Saurabh Kataria"
            ],
            "abstract": "We consider the problem of learning distributed representations for tags from\ntheir associated content for the task of tag recommendation. Considering\ntagging information is usually very sparse, effective learning from content and\ntag association is very crucial and challenging task. Recently, various neural\nrepresentation learning models such as WSABIE and its variants show promising\nperformance, mainly due to compact feature representations learned in a\nsemantic space. However, their capacity is limited by a linear compositional\napproach for representing tags as sum of equal parts and hurt their\nperformance. In this work, we propose a neural feedback relevance model for\nlearning tag representations with weighted feature representations. Our\nexperiments on two widely used datasets show significant improvement for\nquality of recommendations over various baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1603.08023v2",
            "date": "2016-03-25 20:32:21+00:00",
            "title": "How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation",
            "authors": [
                "Chia-Wei Liu",
                "Ryan Lowe",
                "Iulian V. Serban",
                "Michael Noseworthy",
                "Laurent Charlin",
                "Joelle Pineau"
            ],
            "abstract": "We investigate evaluation metrics for dialogue response generation systems\nwhere supervised labels, such as task completion, are not available. Recent\nworks in response generation have adopted metrics from machine translation to\ncompare a model's generated response to a single target response. We show that\nthese metrics correlate very weakly with human judgements in the non-technical\nTwitter domain, and not at all in the technical Ubuntu domain. We provide\nquantitative and qualitative results highlighting specific weaknesses in\nexisting metrics, and provide recommendations for future development of better\nautomatic evaluation metrics for dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08042v2",
            "date": "2016-03-25 21:43:28+00:00",
            "title": "On the Compression of Recurrent Neural Networks with an Application to LVCSR acoustic modeling for Embedded Speech Recognition",
            "authors": [
                "Rohit Prabhavalkar",
                "Ouais Alsharif",
                "Antoine Bruguier",
                "Ian McGraw"
            ],
            "abstract": "We study the problem of compressing recurrent neural networks (RNNs). In\nparticular, we focus on the compression of RNN acoustic models, which are\nmotivated by the goal of building compact and accurate speech recognition\nsystems which can be run efficiently on mobile devices. In this work, we\npresent a technique for general recurrent model compression that jointly\ncompresses both recurrent and non-recurrent inter-layer weight matrices. We\nfind that the proposed technique allows us to reduce the size of our Long\nShort-Term Memory (LSTM) acoustic model to a third of its original size with\nnegligible loss in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08048v1",
            "date": "2016-03-25 22:36:40+00:00",
            "title": "\"Did I Say Something Wrong?\" A Word-Level Analysis of Wikipedia Articles for Deletion Discussions",
            "authors": [
                "Michael Ruster"
            ],
            "abstract": "This thesis focuses on gaining linguistic insights into textual discussions\non a word level. It was of special interest to distinguish messages that\nconstructively contribute to a discussion from those that are detrimental to\nthem. Thereby, we wanted to determine whether \"I\"- and \"You\"-messages are\nindicators for either of the two discussion styles. These messages are nowadays\noften used in guidelines for successful communication. Although their effects\nhave been successfully evaluated multiple times, a large-scale analysis has\nnever been conducted.\n  Thus, we used Wikipedia Articles for Deletion (short: AfD) discussions\ntogether with the records of blocked users and developed a fully automated\ncreation of an annotated data set. In this data set, messages were labelled\neither constructive or disruptive. We applied binary classifiers to the data to\ndetermine characteristic words for both discussion styles. Thereby, we also\ninvestigated whether function words like pronouns and conjunctions play an\nimportant role in distinguishing the two.\n  We found that \"You\"-messages were a strong indicator for disruptive messages\nwhich matches their attributed effects on communication. However, we found\n\"I\"-messages to be indicative for disruptive messages as well which is contrary\nto their attributed effects. The importance of function words could neither be\nconfirmed nor refuted. Other characteristic words for either communication\nstyle were not found. Yet, the results suggest that a different model might\nrepresent disruptive and constructive messages in textual discussions better.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08079v1",
            "date": "2016-03-26 06:49:33+00:00",
            "title": "Do You See What I Mean? Visual Resolution of Linguistic Ambiguities",
            "authors": [
                "Yevgeni Berzak",
                "Andrei Barbu",
                "Daniel Harari",
                "Boris Katz",
                "Shimon Ullman"
            ],
            "abstract": "Understanding language goes hand in hand with the ability to integrate\ncomplex contextual information obtained via perception. In this work, we\npresent a novel task for grounded language understanding: disambiguating a\nsentence given a visual scene which depicts one of the possible interpretations\nof that sentence. To this end, we introduce a new multimodal corpus containing\nambiguous sentences, representing a wide range of syntactic, semantic and\ndiscourse ambiguities, coupled with videos that visualize the different\ninterpretations for each sentence. We address this task by extending a vision\nmodel which determines if a sentence is depicted by a video. We demonstrate how\nsuch a model can be adjusted to recognize different interpretations of the same\nunderlying sentence, allowing to disambiguate sentences in a unified fashion\nacross the different ambiguity types.",
            "journal ref": "Conference on Empirical Methods in Natural Language Processing\n  (EMNLP), 2015, pages 1477--1487",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1603.08089v1",
            "date": "2016-03-26 08:57:22+00:00",
            "title": "Online shopping behavior study based on multi-granularity opinion mining: China vs. America",
            "authors": [
                "Qingqing Zhou",
                "Rui Xia",
                "Chengzhi Zhang"
            ],
            "abstract": "With the development of e-commerce, many products are now being sold\nworldwide, and manufacturers are eager to obtain a better understanding of\ncustomer behavior in various regions. To achieve this goal, most previous\nefforts have focused mainly on questionnaires, which are time-consuming and\ncostly. The tremendous volume of product reviews on e-commerce websites has\nseen a new trend emerge, whereby manufacturers attempt to understand user\npreferences by analyzing online reviews. Following this trend, this paper\naddresses the problem of studying customer behavior by exploiting recently\ndeveloped opinion mining techniques. This work is novel for three reasons.\nFirst, questionnaire-based investigation is automatically enabled by employing\nalgorithms for template-based question generation and opinion mining-based\nanswer extraction. Using this system, manufacturers are able to obtain reports\nof customer behavior featuring a much larger sample size, more direct\ninformation, a higher degree of automation, and a lower cost. Second,\ninternational customer behavior study is made easier by integrating tools for\nmultilingual opinion mining. Third, this is the first time an automatic\nquestionnaire investigation has been conducted to compare customer behavior in\nChina and America, where product reviews are written and read in Chinese and\nEnglish, respectively. Our study on digital cameras, smartphones, and tablet\ncomputers yields three findings. First, Chinese customers follow the Doctrine\nof the Mean, and often use euphemistic expressions, while American customers\nexpress their opinions more directly. Second, Chinese customers care more about\ngeneral feelings, while American customers pay more attention to product\ndetails. Third, Chinese customers focus on external features, while American\ncustomers care more about the internal features of products.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1603.08148v3",
            "date": "2016-03-26 22:31:57+00:00",
            "title": "Pointing the Unknown Words",
            "authors": [
                "Caglar Gulcehre",
                "Sungjin Ahn",
                "Ramesh Nallapati",
                "Bowen Zhou",
                "Yoshua Bengio"
            ],
            "abstract": "The problem of rare and unknown words is an important issue that can\npotentially influence the performance of many NLP systems, including both the\ntraditional count-based and the deep learning models. We propose a novel way to\ndeal with the rare and unseen words for the neural network models using\nattention. Our model uses two softmax layers in order to predict the next word\nin conditional language models: one predicts the location of a word in the\nsource sentence, and the other predicts a word in the shortlist vocabulary. At\neach time-step, the decision of which softmax layer to use choose adaptively\nmade by an MLP which is conditioned on the context.~We motivate our work from a\npsychological evidence that humans naturally have a tendency to point towards\nobjects in the context or the environment when the name of an object is not\nknown.~We observe improvements on two tasks, neural machine translation on the\nEuroparl English to French parallel corpora and text summarization on the\nGigaword dataset using our proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08321v1",
            "date": "2016-03-28 06:06:10+00:00",
            "title": "Audio Visual Emotion Recognition with Temporal Alignment and Perception Attention",
            "authors": [
                "Linlin Chao",
                "Jianhua Tao",
                "Minghao Yang",
                "Ya Li",
                "Zhengqi Wen"
            ],
            "abstract": "This paper focuses on two key problems for audio-visual emotion recognition\nin the video. One is the audio and visual streams temporal alignment for\nfeature level fusion. The other one is locating and re-weighting the perception\nattentions in the whole audio-visual stream for better recognition. The Long\nShort Term Memory Recurrent Neural Network (LSTM-RNN) is employed as the main\nclassification architecture. Firstly, soft attention mechanism aligns the audio\nand visual streams. Secondly, seven emotion embedding vectors, which are\ncorresponding to each classification emotion type, are added to locate the\nperception attentions. The locating and re-weighting process is also based on\nthe soft attention mechanism. The experiment results on EmotiW2015 dataset and\nthe qualitative analysis show the efficiency of the proposed two techniques.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1603.08458v3",
            "date": "2016-03-28 17:47:42+00:00",
            "title": "Longitudinal Analysis of Discussion Topics in an Online Breast Cancer Community using Convolutional Neural Networks",
            "authors": [
                "Shaodian Zhang",
                "Edouard Grave",
                "Elizabeth Sklar",
                "Noemie Elhadad"
            ],
            "abstract": "Identifying topics of discussions in online health communities (OHC) is\ncritical to various applications, but can be difficult because topics of OHC\ncontent are usually heterogeneous and domain-dependent. In this paper, we\nprovide a multi-class schema, an annotated dataset, and supervised classifiers\nbased on convolutional neural network (CNN) and other models for the task of\nclassifying discussion topics. We apply the CNN classifier to the most popular\nbreast cancer online community, and carry out a longitudinal analysis to show\ntopic distributions and topic changes throughout members' participation. Our\nexperimental results suggest that CNN outperforms other classifiers in the task\nof topic classification, and that certain trajectories can be detected with\nrespect to topic changes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08474v1",
            "date": "2016-03-28 18:38:46+00:00",
            "title": "Deep Embedding for Spatial Role Labeling",
            "authors": [
                "Oswaldo Ludwig",
                "Xiao Liu",
                "Parisa Kordjamshidi",
                "Marie-Francine Moens"
            ],
            "abstract": "This paper introduces the visually informed embedding of word (VIEW), a\ncontinuous vector representation for a word extracted from a deep neural model\ntrained using the Microsoft COCO data set to forecast the spatial arrangements\nbetween visual objects, given a textual description. The model is composed of a\ndeep multilayer perceptron (MLP) stacked on the top of a Long Short Term Memory\n(LSTM) network, the latter being preceded by an embedding layer. The VIEW is\napplied to transferring multimodal background knowledge to Spatial Role\nLabeling (SpRL) algorithms, which recognize spatial relations between objects\nmentioned in the text. This work also contributes with a new method to select\ncomplementary features and a fine-tuning method for MLP that improves the $F1$\nmeasure in classifying the words into spatial roles. The VIEW is evaluated with\nthe Task 3 of SemEval-2013 benchmark data set, SpaceEval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.08507v1",
            "date": "2016-03-28 19:54:12+00:00",
            "title": "Generating Visual Explanations",
            "authors": [
                "Lisa Anne Hendricks",
                "Zeynep Akata",
                "Marcus Rohrbach",
                "Jeff Donahue",
                "Bernt Schiele",
                "Trevor Darrell"
            ],
            "abstract": "Clearly explaining a rationale for a classification decision to an end-user\ncan be as important as the decision itself. Existing approaches for deep visual\nrecognition are generally opaque and do not output any justification text;\ncontemporary vision-language models can describe image content but fail to take\ninto account class-discriminative image aspects which justify visual\npredictions. We propose a new model that focuses on the discriminating\nproperties of the visible object, jointly predicts a class label, and explains\nwhy the predicted label is appropriate for the image. We propose a novel loss\nfunction based on sampling and reinforcement learning that learns to generate\nsentences that realize a global sentence property, such as class specificity.\nOur results on a fine-grained bird species classification dataset show that our\nmodel is able to generate explanations which are not only consistent with an\nimage but also more discriminative than descriptions produced by existing\ncaptioning methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1603.09128v1",
            "date": "2016-03-30 11:09:01+00:00",
            "title": "Bilingual Learning of Multi-sense Embeddings with Discrete Autoencoders",
            "authors": [
                "Simon \u0160uster",
                "Ivan Titov",
                "Gertjan van Noord"
            ],
            "abstract": "We present an approach to learning multi-sense word embeddings relying both\non monolingual and bilingual information. Our model consists of an encoder,\nwhich uses monolingual and bilingual context (i.e. a parallel sentence) to\nchoose a sense for a given word, and a decoder which predicts context words\nbased on the chosen sense. The two components are estimated jointly. We observe\nthat the word representations induced from bilingual data outperform the\nmonolingual counterparts across a range of evaluation tasks, even though\ncrosslingual information is not available at test time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09170v5",
            "date": "2016-03-30 13:09:20+00:00",
            "title": "Model Interpolation with Trans-dimensional Random Field Language Models for Speech Recognition",
            "authors": [
                "Bin Wang",
                "Zhijian Ou",
                "Yong He",
                "Akinori Kawamura"
            ],
            "abstract": "The dominant language models (LMs) such as n-gram and neural network (NN)\nmodels represent sentence probabilities in terms of conditionals. In contrast,\na new trans-dimensional random field (TRF) LM has been recently introduced to\nshow superior performances, where the whole sentence is modeled as a random\nfield. In this paper, we examine how the TRF models can be interpolated with\nthe NN models, and obtain 12.1\\% and 17.9\\% relative error rate reductions over\n6-gram LMs for English and Chinese speech recognition respectively through\nlog-linear combination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09381v1",
            "date": "2016-03-30 20:57:07+00:00",
            "title": "Clinical Information Extraction via Convolutional Neural Network",
            "authors": [
                "Peng Li",
                "Heng Huang"
            ],
            "abstract": "We report an implementation of a clinical information extraction tool that\nleverages deep neural network to annotate event spans and their attributes from\nraw clinical notes and pathology reports. Our approach uses context words and\ntheir part-of-speech tags and shape information as features. Then we hire\ntemporal (1D) convolutional neural network to learn hidden feature\nrepresentations. Finally, we use Multilayer Perceptron (MLP) to predict event\nspans. The empirical evaluation demonstrates that our approach significantly\noutperforms baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1603.09405v1",
            "date": "2016-03-30 22:39:59+00:00",
            "title": "Enhancing Sentence Relation Modeling with Auxiliary Character-level Embedding",
            "authors": [
                "Peng Li",
                "Heng Huang"
            ],
            "abstract": "Neural network based approaches for sentence relation modeling automatically\ngenerate hidden matching features from raw sentence pairs. However, the quality\nof matching feature representation may not be satisfied due to complex semantic\nrelations such as entailment or contradiction. To address this challenge, we\npropose a new deep neural network architecture that jointly leverage\npre-trained word embedding and auxiliary character embedding to learn sentence\nmeanings. The two kinds of word sequence representations as inputs into\nmulti-layer bidirectional LSTM to learn enhanced sentence representation. After\nthat, we construct matching features followed by another temporal CNN to learn\nhigh-level hidden matching feature representations. Experimental results\ndemonstrate that our approach consistently outperforms the existing methods on\nstandard evaluation datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09509v2",
            "date": "2016-03-31 09:54:44+00:00",
            "title": "Learning Multiscale Features Directly From Waveforms",
            "authors": [
                "Zhenyao Zhu",
                "Jesse H. Engel",
                "Awni Hannun"
            ],
            "abstract": "Deep learning has dramatically improved the performance of speech recognition\nsystems through learning hierarchies of features optimized for the task at\nhand. However, true end-to-end learning, where features are learned directly\nfrom waveforms, has only recently reached the performance of hand-tailored\nrepresentations based on the Fourier transform. In this paper, we detail an\napproach to use convolutional filters to push past the inherent tradeoff of\ntemporal and frequency resolution that exists for spectral representations. At\nincreased computational cost, we show that increasing temporal resolution via\nreduced stride and increasing frequency resolution via additional filters\ndelivers significant performance improvements. Further, we find more efficient\nrepresentations by simultaneously learning at multiple scales, leading to an\noverall decrease in word error rate on a difficult internal speech test set by\n20.7% relative to networks with the same number of parameters trained on\nspectrograms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.09643v4",
            "date": "2016-03-31 15:37:29+00:00",
            "title": "Multi-task Recurrent Model for Speech and Speaker Recognition",
            "authors": [
                "Zhiyuan Tang",
                "Lantian Li",
                "Dong Wang"
            ],
            "abstract": "Although highly correlated, speech and speaker recognition have been regarded\nas two independent tasks and studied by two communities. This is certainly not\nthe way that people behave: we decipher both speech content and speaker traits\nat the same time. This paper presents a unified model to perform speech and\nspeaker recognition simultaneously and altogether. The model is based on a\nunified neural network where the output of one task is fed to the input of the\nother, leading to a multi-task recurrent network. Experiments show that the\njoint model outperforms the task-specific models on both the two tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00119v3",
            "date": "2016-04-01 03:32:03+00:00",
            "title": "Semi-supervised and Unsupervised Methods for Categorizing Posts in Web Discussion Forums",
            "authors": [
                "Krish Perumal"
            ],
            "abstract": "Web discussion forums are used by millions of people worldwide to share\ninformation belonging to a variety of domains such as automotive vehicles,\npets, sports, etc. They typically contain posts that fall into different\ncategories such as problem, solution, feedback, spam, etc. Automatic\nidentification of these categories can aid information retrieval that is\ntailored for specific user requirements. Previously, a number of supervised\nmethods have attempted to solve this problem; however, these depend on the\navailability of abundant training data. A few existing unsupervised and\nsemi-supervised approaches are either focused on identifying a single category\nor do not report category-specific performance. In contrast, this work proposes\nunsupervised and semi-supervised methods that require no or minimal training\ndata to achieve this objective without compromising on performance. A\nfine-grained analysis is also carried out to discuss their limitations. The\nproposed methods are based on sequence models (specifically, Hidden Markov\nModels) that can model language for each category using word and part-of-speech\nprobability distributions, and manually specified features. Empirical\nevaluations across domains demonstrate that the proposed methods are better\nsuited for this task than existing ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00126v1",
            "date": "2016-04-01 04:36:58+00:00",
            "title": "Nonparametric Spherical Topic Modeling with Word Embeddings",
            "authors": [
                "Kayhan Batmanghelich",
                "Ardavan Saeedi",
                "Karthik Narasimhan",
                "Sam Gershman"
            ],
            "abstract": "Traditional topic models do not account for semantic regularities in\nlanguage. Recent distributional representations of words exhibit semantic\nconsistency over directional metrics such as cosine similarity. However,\nneither categorical nor Gaussian observational distributions used in existing\ntopic models are appropriate to leverage such correlations. In this paper, we\npropose to use the von Mises-Fisher distribution to model the density of words\nover a unit sphere. Such a representation is well-suited for directional data.\nWe use a Hierarchical Dirichlet Process for our base topic model and propose an\nefficient inference algorithm based on Stochastic Variational Inference. This\nmodel enables us to naturally exploit the semantic structures of word\nembeddings while flexibly discovering the number of topics. Experiments\ndemonstrate that our method outperforms competitive approaches in terms of\ntopic coherence on two different text corpora while offering efficient\ninference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00317v1",
            "date": "2016-04-01 16:26:57+00:00",
            "title": "A Semisupervised Approach for Language Identification based on Ladder Networks",
            "authors": [
                "Ehud Ben-Reuven",
                "Jacob Goldberger"
            ],
            "abstract": "In this study we address the problem of training a neuralnetwork for language\nidentification using both labeled and unlabeled speech samples in the form of\ni-vectors. We propose a neural network architecture that can also handle\nout-of-set languages. We utilize a modified version of the recently proposed\nLadder Network semisupervised training procedure that optimizes the\nreconstruction costs of a stack of denoising autoencoders. We show that this\napproach can be successfully applied to the case where the training dataset is\ncomposed of both labeled and unlabeled acoustic data. The results show enhanced\nlanguage identification on the NIST 2015 language identification dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00461v1",
            "date": "2016-04-02 04:59:21+00:00",
            "title": "Embedding Lexical Features via Low-Rank Tensors",
            "authors": [
                "Mo Yu",
                "Mark Dredze",
                "Raman Arora",
                "Matthew Gormley"
            ],
            "abstract": "Modern NLP models rely heavily on engineered features, which often combine\nword and contextual information into complex lexical features. Such combination\nresults in large numbers of features, which can lead to over-fitting. We\npresent a new model that represents complex lexical features---comprised of\nparts for words, contextual information and labels---in a tensor that captures\nconjunction information among these parts. We apply low-rank tensor\napproximations to the corresponding parameter tensors to reduce the parameter\nspace and improve prediction speed. Furthermore, we investigate two methods for\nhandling features that include $n$-grams of mixed lengths. Our model achieves\nstate-of-the-art results on tasks in relation extraction, PP-attachment, and\npreposition disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00727v4",
            "date": "2016-04-04 02:43:23+00:00",
            "title": "Character-Level Question Answering with Attention",
            "authors": [
                "David Golub",
                "Xiaodong He"
            ],
            "abstract": "We show that a character-level encoder-decoder framework can be successfully\napplied to question answering with a structured knowledge base. We use our\nmodel for single-relation question answering and demonstrate the effectiveness\nof our approach on the SimpleQuestions dataset (Bordes et al., 2015), where we\nimprove state-of-the-art accuracy from 63.9% to 70.9%, without use of\nensembles. Importantly, our character-level model has 16x fewer parameters than\nan equivalent word-level model, can be learned with significantly less data\ncompared to previous work, which relies on data augmentation, and is robust to\nnew entities in testing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.00790v3",
            "date": "2016-04-04 09:43:04+00:00",
            "title": "Image Captioning with Deep Bidirectional LSTMs",
            "authors": [
                "Cheng Wang",
                "Haojin Yang",
                "Christian Bartz",
                "Christoph Meinel"
            ],
            "abstract": "This work presents an end-to-end trainable deep bidirectional LSTM\n(Long-Short Term Memory) model for image captioning. Our model builds on a deep\nconvolutional neural network (CNN) and two separate LSTM networks. It is\ncapable of learning long term visual-language interactions by making use of\nhistory and future context information at high level semantic space. Two novel\ndeep bidirectional variant models, in which we increase the depth of\nnonlinearity transition in different way, are proposed to learn hierarchical\nvisual-language embeddings. Data augmentation techniques such as multi-crop,\nmulti-scale and vertical mirror are proposed to prevent overfitting in training\ndeep models. We visualize the evolution of bidirectional LSTM internal states\nover time and qualitatively analyze how our models \"translate\" image to\nsentence. Our proposed models are evaluated on caption generation and\nimage-sentence retrieval tasks with three benchmark datasets: Flickr8K,\nFlickr30K and MSCOCO datasets. We demonstrate that bidirectional LSTM models\nachieve highly competitive performance to the state-of-the-art results on\ncaption generation even without integrating additional mechanism (e.g. object\ndetection, attention model etc.) and significantly outperform recent methods on\nretrieval task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1604.01243v1",
            "date": "2016-04-05 13:04:35+00:00",
            "title": "Mental Lexicon Growth Modelling Reveals the Multiplexity of the English Language",
            "authors": [
                "Massimo Stella",
                "Markus Brede"
            ],
            "abstract": "In this work we extend previous analyses of linguistic networks by adopting a\nmulti-layer network framework for modelling the human mental lexicon, i.e. an\nabstract mental repository where words and concepts are stored together with\ntheir linguistic patterns. Across a three-layer linguistic multiplex, we model\nEnglish words as nodes and connect them according to (i) phonological\nsimilarities, (ii) synonym relationships and (iii) free word associations. Our\nmain aim is to exploit this multi-layered structure to explore the influence of\nphonological and semantic relationships on lexicon assembly over time. We\npropose a model of lexicon growth which is driven by the phonological layer:\nwords are suggested according to different orderings of insertion (e.g. shorter\nword length, highest frequency, semantic multiplex features) and accepted or\nrejected subject to constraints. We then measure times of network assembly and\ncompare these to empirical data about the age of acquisition of words. In\nagreement with empirical studies in psycholinguistics, our results provide\nquantitative evidence for the hypothesis that word acquisition is driven by\nfeatures at multiple levels of organisation within language.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1604.01485v1",
            "date": "2016-04-06 05:16:10+00:00",
            "title": "A Focused Dynamic Attention Model for Visual Question Answering",
            "authors": [
                "Ilija Ilievski",
                "Shuicheng Yan",
                "Jiashi Feng"
            ],
            "abstract": "Visual Question and Answering (VQA) problems are attracting increasing\ninterest from multiple research disciplines. Solving VQA problems requires\ntechniques from both computer vision for understanding the visual contents of a\npresented image or video, as well as the ones from natural language processing\nfor understanding semantics of the question and generating the answers.\nRegarding visual content modeling, most of existing VQA methods adopt the\nstrategy of extracting global features from the image or video, which\ninevitably fails in capturing fine-grained information such as spatial\nconfiguration of multiple objects. Extracting features from auto-generated\nregions -- as some region-based image recognition methods do -- cannot\nessentially address this problem and may introduce some overwhelming irrelevant\nfeatures with the question. In this work, we propose a novel Focused Dynamic\nAttention (FDA) model to provide better aligned image content representation\nwith proposed questions. Being aware of the key words in the question, FDA\nemploys off-the-shelf object detector to identify important regions and fuse\nthe information from the regions and global features via an LSTM unit. Such\nquestion-driven representations are then combined with question representation\nand fed into a reasoning unit for generating the answers. Extensive evaluation\non a large-scale benchmark dataset, VQA, clearly demonstrate the superior\nperformance of FDA over well-established baselines.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1604.01792v2",
            "date": "2016-04-06 20:07:52+00:00",
            "title": "Advances in Very Deep Convolutional Neural Networks for LVCSR",
            "authors": [
                "Tom Sercu",
                "Vaibhava Goel"
            ],
            "abstract": "Very deep CNNs with small 3x3 kernels have recently been shown to achieve\nvery strong performance as acoustic models in hybrid NN-HMM speech recognition\nsystems. In this paper we investigate how to efficiently scale these models to\nlarger datasets. Specifically, we address the design choice of pooling and\npadding along the time dimension which renders convolutional evaluation of\nsequences highly inefficient. We propose a new CNN design without timepadding\nand without timepooling, which is slightly suboptimal for accuracy, but has two\nsignificant advantages: it enables sequence training and deployment by allowing\nefficient convolutional evaluation of full utterances, and, it allows for batch\nnormalization to be straightforwardly adopted to CNNs on sequence data. Through\nbatch normalization, we recover the lost peformance from removing the\ntime-pooling, while keeping the benefit of efficient convolutional evaluation.\nWe demonstrate the performance of our models both on larger scale data than\nbefore, and after sequence training. Our very deep CNN model sequence trained\non the 2000h switchboard dataset obtains 9.4 word error rate on the Hub5\ntest-set, matching with a single model the performance of the 2015 IBM system\ncombination, which was the previous best published result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.02027v2",
            "date": "2016-04-07 15:04:16+00:00",
            "title": "Combinatorial Topic Models using Small-Variance Asymptotics",
            "authors": [
                "Ke Jiang",
                "Suvrit Sra",
                "Brian Kulis"
            ],
            "abstract": "Topic models have emerged as fundamental tools in unsupervised machine\nlearning. Most modern topic modeling algorithms take a probabilistic view and\nderive inference algorithms based on Latent Dirichlet Allocation (LDA) or its\nvariants. In contrast, we study topic modeling as a combinatorial optimization\nproblem, and propose a new objective function derived from LDA by passing to\nthe small-variance limit. We minimize the derived objective by using ideas from\ncombinatorial optimization, which results in a new, fast, and high-quality\ntopic modeling algorithm. In particular, we show that our results are\ncompetitive with popular LDA-based topic modeling approaches, and also discuss\nthe (dis)similarities between our approach and its probabilistic counterparts.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1604.02038v2",
            "date": "2016-04-07 15:29:45+00:00",
            "title": "Sentence Level Recurrent Topic Model: Letting Topics Speak for Themselves",
            "authors": [
                "Fei Tian",
                "Bin Gao",
                "Di He",
                "Tie-Yan Liu"
            ],
            "abstract": "We propose Sentence Level Recurrent Topic Model (SLRTM), a new topic model\nthat assumes the generation of each word within a sentence to depend on both\nthe topic of the sentence and the whole history of its preceding words in the\nsentence. Different from conventional topic models that largely ignore the\nsequential order of words or their topic coherence, SLRTM gives full\ncharacterization to them by using a Recurrent Neural Networks (RNN) based\nframework. Experimental results have shown that SLRTM outperforms several\nstrong baselines on various tasks. Furthermore, SLRTM can automatically\ngenerate sentences given a topic (i.e., topics to sentences), which is a key\ntechnology for real world applications such as personalized short text\nconversation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1604.02125v4",
            "date": "2016-04-07 19:26:56+00:00",
            "title": "Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes",
            "authors": [
                "Gordon Christie",
                "Ankit Laddha",
                "Aishwarya Agrawal",
                "Stanislaw Antol",
                "Yash Goyal",
                "Kevin Kochersberger",
                "Dhruv Batra"
            ],
            "abstract": "We present an approach to simultaneously perform semantic segmentation and\nprepositional phrase attachment resolution for captioned images. Some\nambiguities in language cannot be resolved without simultaneously reasoning\nabout an associated image. If we consider the sentence \"I shot an elephant in\nmy pajamas\", looking at language alone (and not using common sense), it is\nunclear if it is the person or the elephant wearing the pajamas or both. Our\napproach produces a diverse set of plausible hypotheses for both semantic\nsegmentation and prepositional phrase attachment resolution that are then\njointly reranked to select the most consistent pair. We show that our semantic\nsegmentation and prepositional phrase attachment resolution modules have\ncomplementary strengths, and that joint reasoning produces more accurate\nresults than any module operating in isolation. Multiple hypotheses are also\nshown to be crucial to improved multiple-module reasoning. Our vision and\nlanguage approach significantly outperforms the Stanford Parser (De Marneffe et\nal., 2006) by 17.91% (28.69% relative) and 12.83% (25.28% relative) in two\ndifferent experiments. We also make small improvements over DeepLab-CRF (Chen\net al., 2015).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1604.02594v1",
            "date": "2016-04-09 19:09:22+00:00",
            "title": "Learning Compact Recurrent Neural Networks",
            "authors": [
                "Zhiyun Lu",
                "Vikas Sindhwani",
                "Tara N. Sainath"
            ],
            "abstract": "Recurrent neural networks (RNNs), including long short-term memory (LSTM)\nRNNs, have produced state-of-the-art results on a variety of speech recognition\ntasks. However, these models are often too large in size for deployment on\nmobile devices with memory and latency constraints. In this work, we study\nmechanisms for learning compact RNNs and LSTMs via low-rank factorizations and\nparameter sharing schemes. Our goal is to investigate redundancies in recurrent\narchitectures where compression can be admitted without losing performance. A\nhybrid strategy of using structured matrices in the bottom layers and shared\nlow-rank factors on the top layers is found to be particularly effective,\nreducing the parameters of a standard LSTM by 75%, at a small cost of 0.3%\nincrease in WER, on a 2,000-hr English Voice Search task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1604.03029v1",
            "date": "2016-03-24 10:59:28+00:00",
            "title": "Mapping Out Narrative Structures and Dynamics Using Networks and Textual Information",
            "authors": [
                "Semi Min",
                "Juyong Park"
            ],
            "abstract": "Human communication is often executed in the form of a narrative, an account\nof connected events composed of characters, actions, and settings. A coherent\nnarrative structure is therefore a requisite for a well-formulated narrative --\nbe it fictional or nonfictional -- for informative and effective communication,\nopening up the possibility of a deeper understanding of a narrative by studying\nits structural properties. In this paper we present a network-based framework\nfor modeling and analyzing the structure of a narrative, which is further\nexpanded by incorporating methods from computational linguistics to utilize the\nnarrative text. Modeling a narrative as a dynamically unfolding system, we\ncharacterize its progression via the growth patterns of the character network,\nand use sentiment analysis and topic modeling to represent the actual content\nof the narrative in the form of interaction maps between characters with\nassociated sentiment values and keywords. This is a network framework advanced\nbeyond the simple occurrence-based one most often used until now, allowing one\nto utilize the unique characteristics of a given narrative to a high degree.\nGiven the ubiquity and importance of narratives, such advanced network-based\nrepresentation and analysis framework may lead to a more systematic modeling\nand understanding of narratives for social interactions, expression of human\nsentiments, and communication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03390v2",
            "date": "2016-04-12 13:09:01+00:00",
            "title": "Video Description using Bidirectional Recurrent Neural Networks",
            "authors": [
                "\u00c1lvaro Peris",
                "Marc Bola\u00f1os",
                "Petia Radeva",
                "Francisco Casacuberta"
            ],
            "abstract": "Although traditionally used in the machine translation field, the\nencoder-decoder framework has been recently applied for the generation of video\nand image descriptions. The combination of Convolutional and Recurrent Neural\nNetworks in these models has proven to outperform the previous state of the\nart, obtaining more accurate video descriptions. In this work we propose\npushing further this model by introducing two contributions into the encoding\nstage. First, producing richer image representations by combining object and\nlocation information from Convolutional Neural Networks and second, introducing\nBidirectional Recurrent Neural Networks for capturing both forward and backward\ntemporal relationships in the input frames.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1604.03627v1",
            "date": "2016-04-13 01:00:24+00:00",
            "title": "Dissecting a Social Botnet: Growth, Content and Influence in Twitter",
            "authors": [
                "Norah Abokhodair",
                "Daisy Yoo",
                "David W. McDonald"
            ],
            "abstract": "Social botnets have become an important phenomenon on social media. There are\nmany ways in which social bots can disrupt or influence online discourse, such\nas, spam hashtags, scam twitter users, and astroturfing. In this paper we\nconsidered one specific social botnet in Twitter to understand how it grows\nover time, how the content of tweets by the social botnet differ from regular\nusers in the same dataset, and lastly, how the social botnet may have\ninfluenced the relevant discussions. Our analysis is based on a qualitative\ncoding for approximately 3000 tweets in Arabic and English from the Syrian\nsocial bot that was active for 35 weeks on Twitter before it was shutdown. We\nfind that the growth, behavior and content of this particular botnet did not\nspecifically align with common conceptions of botnets. Further we identify\ninteresting aspects of the botnet that distinguish it from regular users.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1604.03968v1",
            "date": "2016-04-13 20:27:43+00:00",
            "title": "Visual Storytelling",
            "authors": [
                "Ting-Hao",
                "Huang",
                "Francis Ferraro",
                "Nasrin Mostafazadeh",
                "Ishan Misra",
                "Aishwarya Agrawal",
                "Jacob Devlin",
                "Ross Girshick",
                "Xiaodong He",
                "Pushmeet Kohli",
                "Dhruv Batra",
                "C. Lawrence Zitnick",
                "Devi Parikh",
                "Lucy Vanderwende",
                "Michel Galley",
                "Margaret Mitchell"
            ],
            "abstract": "We introduce the first dataset for sequential vision-to-language, and explore\nhow this data may be used for the task of visual storytelling. The first\nrelease of this dataset, SIND v.1, includes 81,743 unique photos in 20,211\nsequences, aligned to both descriptive (caption) and story language. We\nestablish several strong baselines for the storytelling task, and motivate an\nautomatic metric to benchmark progress. Modelling concrete description as well\nas figurative and social language, as provided in this dataset and the\nstorytelling task, has the potential to move artificial intelligence from basic\nunderstandings of typical visual scenes towards more and more human-like\nunderstanding of grounded event structure and subjective expression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04358v1",
            "date": "2016-04-15 05:51:29+00:00",
            "title": "StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation",
            "authors": [
                "Xiang Li",
                "Lili Mou",
                "Rui Yan",
                "Ming Zhang"
            ],
            "abstract": "Existing open-domain human-computer conversation systems are typically\npassive: they either synthesize or retrieve a reply provided a human-issued\nutterance. It is generally presumed that humans should take the role to lead\nthe conversation and introduce new content when a stalemate occurs, and that\nthe computer only needs to \"respond.\" In this paper, we propose\nStalemateBreaker, a conversation system that can proactively introduce new\ncontent when appropriate. We design a pipeline to determine when, what, and how\nto introduce new content during human-computer conversation. We further propose\na novel reranking algorithm Bi-PageRank-HITS to enable rich interaction between\nconversation context and candidate replies. Experiments show that both the\ncontent-introducing approach and the reranking algorithm are effective. Our\nfull StalemateBreaker model outperforms a state-of-the-practice conversation\nsystem by +14.4% p@1 when a stalemate occurs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04378v1",
            "date": "2016-04-15 07:23:53+00:00",
            "title": "Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN",
            "authors": [
                "Shengxian Wan",
                "Yanyan Lan",
                "Jun Xu",
                "Jiafeng Guo",
                "Liang Pang",
                "Xueqi Cheng"
            ],
            "abstract": "Semantic matching, which aims to determine the matching degree between two\ntexts, is a fundamental problem for many NLP applications. Recently, deep\nlearning approach has been applied to this problem and significant improvements\nhave been achieved. In this paper, we propose to view the generation of the\nglobal interaction between two texts as a recursive process: i.e. the\ninteraction of two texts at each position is a composition of the interactions\nbetween their prefixes as well as the word level interaction at the current\nposition. Based on this idea, we propose a novel deep architecture, namely\nMatch-SRNN, to model the recursive matching structure. Firstly, a tensor is\nconstructed to capture the word level interactions. Then a spatial RNN is\napplied to integrate the local interactions recursively, with importance\ndetermined by four types of gates. Finally, the matching score is calculated\nbased on the global interaction. We show that, after degenerated to the exact\nmatching scenario, Match-SRNN can approximate the dynamic programming process\nof longest common subsequence. Thus, there exists a clear interpretation for\nMatch-SRNN. Our experiments on two semantic matching tasks showed the\neffectiveness of Match-SRNN, and its ability of visualizing the learned\nmatching structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04562v3",
            "date": "2016-04-15 16:40:49+00:00",
            "title": "A Network-based End-to-End Trainable Task-oriented Dialogue System",
            "authors": [
                "Tsung-Hsien Wen",
                "David Vandyke",
                "Nikola Mrksic",
                "Milica Gasic",
                "Lina M. Rojas-Barahona",
                "Pei-Hao Su",
                "Stefan Ultes",
                "Steve Young"
            ],
            "abstract": "Teaching machines to accomplish tasks by conversing naturally with humans is\nchallenging. Currently, developing task-oriented dialogue systems requires\ncreating multiple components and typically this involves either a large amount\nof handcrafting, or acquiring costly labelled datasets to solve a statistical\nlearning problem for each component. In this work we introduce a neural\nnetwork-based text-in, text-out end-to-end trainable goal-oriented dialogue\nsystem along with a new way of collecting dialogue data based on a novel\npipe-lined Wizard-of-Oz framework. This approach allows us to develop dialogue\nsystems easily and without making too many assumptions about the task at hand.\nThe results show that the model can converse with human subjects naturally\nwhilst helping them to accomplish tasks in a restaurant search domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.04661v2",
            "date": "2016-04-15 23:40:04+00:00",
            "title": "Parallelizing Word2Vec in Shared and Distributed Memory",
            "authors": [
                "Shihao Ji",
                "Nadathur Satish",
                "Sheng Li",
                "Pradeep Dubey"
            ],
            "abstract": "Word2Vec is a widely used algorithm for extracting low-dimensional vector\nrepresentations of words. It generated considerable excitement in the machine\nlearning and natural language processing (NLP) communities recently due to its\nexceptional performance in many NLP applications such as named entity\nrecognition, sentiment analysis, machine translation and question answering.\nState-of-the-art algorithms including those by Mikolov et al. have been\nparallelized for multi-core CPU architectures but are based on vector-vector\noperations that are memory-bandwidth intensive and do not efficiently use\ncomputational resources. In this paper, we improve reuse of various data\nstructures in the algorithm through the use of minibatching, hence allowing us\nto express the problem using matrix multiply operations. We also explore\ndifferent techniques to distribute word2vec computation across nodes in a\ncompute cluster, and demonstrate good strong scalability up to 32 nodes. In\ncombination, these techniques allow us to scale up the computation near\nlinearly across cores and nodes, and process hundreds of millions of words per\nsecond, which is the fastest word2vec implementation to the best of our\nknowledge.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1604.05468v1",
            "date": "2016-04-19 08:31:23+00:00",
            "title": "Understanding Rating Behaviour and Predicting Ratings by Identifying Representative Users",
            "authors": [
                "Rahul Kamath",
                "Masanao Ochi",
                "Yutaka Matsuo"
            ],
            "abstract": "Online user reviews describing various products and services are now abundant\non the web. While the information conveyed through review texts and ratings is\neasily comprehensible, there is a wealth of hidden information in them that is\nnot immediately obvious. In this study, we unlock this hidden value behind user\nreviews to understand the various dimensions along which users rate products.\nWe learn a set of users that represent each of these dimensions and use their\nratings to predict product ratings. Specifically, we work with restaurant\nreviews to identify users whose ratings are influenced by dimensions like\n'Service', 'Atmosphere' etc. in order to predict restaurant ratings and\nunderstand the variation in rating behaviour across different cuisines. While\nprevious approaches to obtaining product ratings require either a large number\nof user ratings or a few review texts, we show that it is possible to predict\nratings with few user ratings and no review text. Our experiments show that our\napproach outperforms other conventional methods by 16-27% in terms of RMSE.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1604.05781v2",
            "date": "2016-04-20 01:06:50+00:00",
            "title": "What we write about when we write about causality: Features of causal statements across large-scale social discourse",
            "authors": [
                "Thomas C. McAndrew",
                "Joshua C. Bongard",
                "Christopher M. Danforth",
                "Peter S. Dodds",
                "Paul D. H. Hines",
                "James P. Bagrow"
            ],
            "abstract": "Identifying and communicating relationships between causes and effects is\nimportant for understanding our world, but is affected by language structure,\ncognitive and emotional biases, and the properties of the communication medium.\nDespite the increasing importance of social media, much remains unknown about\ncausal statements made online. To study real-world causal attribution, we\nextract a large-scale corpus of causal statements made on the Twitter social\nnetwork platform as well as a comparable random control corpus. We compare\ncausal and control statements using statistical language and sentiment analysis\ntools. We find that causal statements have a number of significant lexical and\ngrammatical differences compared with controls and tend to be more negative in\nsentiment than controls. Causal statements made online tend to focus on news\nand current events, medicine and health, or interpersonal relationships, as\nshown by topic models. By quantifying the features and potential biases of\ncausality communication, this study improves our understanding of the accuracy\nof information and opinions found online.",
            "journal ref": "2016 IEEE/ACM International Conference on Advances in Social\n  Networks Analysis and Mining (ASONAM), San Francisco, CA, 2016, pp. 519-524",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1604.05878v1",
            "date": "2016-04-20 09:58:56+00:00",
            "title": "A Factorization Machine Framework for Testing Bigram Embeddings in Knowledgebase Completion",
            "authors": [
                "Johannes Welbl",
                "Guillaume Bouchard",
                "Sebastian Riedel"
            ],
            "abstract": "Embedding-based Knowledge Base Completion models have so far mostly combined\ndistributed representations of individual entities or relations to compute\ntruth scores of missing links. Facts can however also be represented using\npairwise embeddings, i.e. embeddings for pairs of entities and relations. In\nthis paper we explore such bigram embeddings with a flexible Factorization\nMachine model and several ablations from it. We investigate the relevance of\nvarious bigram types on the fb15k237 dataset and find relative improvements\ncompared to a compositional model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06529v2",
            "date": "2016-04-22 03:20:24+00:00",
            "title": "Dependency Parsing with LSTMs: An Empirical Evaluation",
            "authors": [
                "Adhiguna Kuncoro",
                "Yuichiro Sawai",
                "Kevin Duh",
                "Yuji Matsumoto"
            ],
            "abstract": "We propose a transition-based dependency parser using Recurrent Neural\nNetworks with Long Short-Term Memory (LSTM) units. This extends the feedforward\nneural network parser of Chen and Manning (2014) and enables modelling of\nentire sequences of shift/reduce transition decisions. On the Google Web\nTreebank, our LSTM parser is competitive with the best feedforward parser on\noverall accuracy and notably achieves more than 3% improvement for long-range\ndependencies, which has proved difficult for previous transition-based parsers\ndue to error propagation and limited context information. Our findings\nadditionally suggest that dropout regularisation on the embedding layer is\ncrucial to improve the LSTM's generalisation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06635v1",
            "date": "2016-04-22 12:51:11+00:00",
            "title": "Bridging LSTM Architecture and the Neural Dynamics during Reading",
            "authors": [
                "Peng Qian",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, the long short-term memory neural network (LSTM) has attracted wide\ninterest due to its success in many tasks. LSTM architecture consists of a\nmemory cell and three gates, which looks similar to the neuronal networks in\nthe brain. However, there still lacks the evidence of the cognitive\nplausibility of LSTM architecture as well as its working mechanism. In this\npaper, we study the cognitive plausibility of LSTM by aligning its internal\narchitecture with the brain activity observed via fMRI when the subjects read a\nstory. Experiment results show that the artificial memory vector in LSTM can\naccurately predict the observed sequential brain activities, indicating the\ncorrelation between LSTM architecture and the cognitive process of story\nreading.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.06721v1",
            "date": "2016-04-22 15:58:18+00:00",
            "title": "Exploiting Deep Semantics and Compositionality of Natural Language for Human-Robot-Interaction",
            "authors": [
                "Manfred Eppe",
                "Sean Trott",
                "Jerome Feldman"
            ],
            "abstract": "We develop a natural language interface for human robot interaction that\nimplements reasoning about deep semantics in natural language. To realize the\nrequired deep analysis, we employ methods from cognitive linguistics, namely\nthe modular and compositional framework of Embodied Construction Grammar (ECG)\n[Feldman, 2009]. Using ECG, robots are able to solve fine-grained reference\nresolution problems and other issues related to deep semantics and\ncompositionality of natural language. This also includes verbal interaction\nwith humans to clarify commands and queries that are too ambiguous to be\nexecuted safely. We implement our NLU framework as a ROS package and present\nproof-of-concept scenarios with different robots, as well as a survey on the\nstate of the art.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1604.07236v3",
            "date": "2016-04-25 12:50:50+00:00",
            "title": "Towards Real-Time, Country-Level Location Classification of Worldwide Tweets",
            "authors": [
                "Arkaitz Zubiaga",
                "Alex Voss",
                "Rob Procter",
                "Maria Liakata",
                "Bo Wang",
                "Adam Tsakalidis"
            ],
            "abstract": "In contrast to much previous work that has focused on location classification\nof tweets restricted to a specific country, here we undertake the task in a\nbroader context by classifying global tweets at the country level, which is so\nfar unexplored in a real-time scenario. We analyse the extent to which a\ntweet's country of origin can be determined by making use of eight\ntweet-inherent features for classification. Furthermore, we use two datasets,\ncollected a year apart from each other, to analyse the extent to which a model\ntrained from historical tweets can still be leveraged for classification of new\ntweets. With classification experiments on all 217 countries in our datasets,\nas well as on the top 25 countries, we offer some insights into the best use of\ntweet-inherent features for an accurate country-level classification of tweets.\nWe find that the use of a single feature, such as the use of tweet content\nalone -- the most widely used feature in previous work -- leaves much to be\ndesired. Choosing an appropriate combination of both tweet content and metadata\ncan actually lead to substantial improvements of between 20\\% and 50\\%. We\nobserve that tweet content, the user's self-reported location and the user's\nreal name, all of which are inherent in a tweet and available in a real-time\nscenario, are particularly useful to determine the country of origin. We also\nexperiment on the applicability of a model trained on historical tweets to\nclassify new tweets, finding that the choice of a particular combination of\nfeatures whose utility does not fade over time can actually lead to comparable\nperformance, avoiding the need to retrain. However, the difficulty of achieving\naccurate classification increases slightly for countries with multiple\ncommonalities, especially for English and Spanish speaking countries.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1605.00122v1",
            "date": "2016-04-30 15:04:19+00:00",
            "title": "An Improved System for Sentence-level Novelty Detection in Textual Streams",
            "authors": [
                "Xinyu Fu",
                "Eugene Ch'ng",
                "Uwe Aickelin",
                "Lanyun Zhang"
            ],
            "abstract": "Novelty detection in news events has long been a difficult problem. A number\nof models performed well on specific data streams but certain issues are far\nfrom being solved, particularly in large data streams from the WWW where\nunpredictability of new terms requires adaptation in the vector space model. We\npresent a novel event detection system based on the Incremental Term\nFrequency-Inverse Document Frequency (TF-IDF) weighting incorporated with\nLocality Sensitive Hashing (LSH). Our system could efficiently and effectively\nadapt to the changes within the data streams of any new terms with continual\nupdates to the vector space model. Regarding miss probability, our proposed\nnovelty detection framework outperforms a recognised baseline system by\napproximately 16% when evaluating a benchmark dataset from Google News.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1605.00223v1",
            "date": "2016-05-01 09:01:13+00:00",
            "title": "Text-mining the NeuroSynth corpus using Deep Boltzmann Machines",
            "authors": [
                "Ricardo Pio Monti",
                "Romy Lorenz",
                "Robert Leech",
                "Christoforos Anagnostopoulos",
                "Giovanni Montana"
            ],
            "abstract": "Large-scale automated meta-analysis of neuroimaging data has recently\nestablished itself as an important tool in advancing our understanding of human\nbrain function. This research has been pioneered by NeuroSynth, a database\ncollecting both brain activation coordinates and associated text across a large\ncohort of neuroimaging research papers. One of the fundamental aspects of such\nmeta-analysis is text-mining. To date, word counts and more sophisticated\nmethods such as Latent Dirichlet Allocation have been proposed. In this work we\npresent an unsupervised study of the NeuroSynth text corpus using Deep\nBoltzmann Machines (DBMs). The use of DBMs yields several advantages over the\naforementioned methods, principal among which is the fact that it yields both\nword and document embeddings in a high-dimensional vector space. Such\nembeddings serve to facilitate the use of traditional machine learning\ntechniques on the text corpus. The proposed DBM model is shown to learn\nembeddings with a clear semantic structure.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1605.01326v2",
            "date": "2016-05-04 16:00:59+00:00",
            "title": "Compression and the origins of Zipf's law for word frequencies",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Here we sketch a new derivation of Zipf's law for word frequencies based on\noptimal coding. The structure of the derivation is reminiscent of Mandelbrot's\nrandom typing model but it has multiple advantages over random typing: (1) it\nstarts from realistic cognitive pressures (2) it does not require fine tuning\nof parameters and (3) it sheds light on the origins of other statistical laws\nof language and thus can lead to a compact theory of linguistic laws. Our\nfindings suggest that the recurrence of Zipf's law in human languages could\noriginate from pressure for easy and fast communication.",
            "journal ref": "Complexity 21, 409-411 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01478v1",
            "date": "2016-05-05 03:06:47+00:00",
            "title": "Modeling Rich Contexts for Sentiment Classification with LSTM",
            "authors": [
                "Minlie Huang",
                "Yujie Cao",
                "Chao Dong"
            ],
            "abstract": "Sentiment analysis on social media data such as tweets and weibo has become a\nvery important and challenging task. Due to the intrinsic properties of such\ndata, tweets are short, noisy, and of divergent topics, and sentiment\nclassification on these data requires to modeling various contexts such as the\nretweet/reply history of a tweet, and the social context about authors and\nrelationships. While few prior study has approached the issue of modeling\ncontexts in tweet, this paper proposes to use a hierarchical LSTM to model rich\ncontexts in tweet, particularly long-range context. Experimental results show\nthat contexts can help us to perform sentiment classification remarkably\nbetter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01635v1",
            "date": "2016-05-05 15:57:21+00:00",
            "title": "The IBM Speaker Recognition System: Recent Advances and Error Analysis",
            "authors": [
                "Seyed Omid Sadjadi",
                "Jason Pelecanos",
                "Sriram Ganapathy"
            ],
            "abstract": "We present the recent advances along with an error analysis of the IBM\nspeaker recognition system for conversational speech. Some of the key\nadvancements that contribute to our system include: a nearest-neighbor\ndiscriminant analysis (NDA) approach (as opposed to LDA) for intersession\nvariability compensation in the i-vector space, the application of speaker and\nchannel-adapted features derived from an automatic speech recognition (ASR)\nsystem for speaker recognition, and the use of a DNN acoustic model with a very\nlarge number of output units (~10k senones) to compute the frame-level soft\nalignments required in the i-vector estimation process. We evaluate these\ntechniques on the NIST 2010 SRE extended core conditions (C1-C9), as well as\nthe 10sec-10sec condition. To our knowledge, results achieved by our system\nrepresent the best performances published to date on these conditions. For\nexample, on the extended tel-tel condition (C5) the system achieves an EER of\n0.59%. To garner further understanding of the remaining errors (on C5), we\nexamine the recordings associated with the low scoring target trials, where\nvarious issues are identified for the problematic recordings/trials.\nInterestingly, it is observed that correcting the pathological recordings not\nonly improves the scores for the target trials but also for the nontarget\ntrials.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.01919v1",
            "date": "2016-05-06 12:52:09+00:00",
            "title": "User Reviews and Language: How Language Influences Ratings",
            "authors": [
                "Scott A. Hale"
            ],
            "abstract": "The number of user reviews of tourist attractions, restaurants, mobile apps,\netc. is increasing for all languages; yet, research is lacking on how reviews\nin multiple languages should be aggregated and displayed. Speakers of different\nlanguages may have consistently different experiences, e.g., different\ninformation available in different languages at tourist attractions or\ndifferent user experiences with software due to\ninternationalization/localization choices. This paper assesses the similarity\nin the ratings given by speakers of different languages to London tourist\nattractions on TripAdvisor. The correlations between different languages are\ngenerally high, but some language pairs are more correlated than others. The\nresults question the common practice of computing average ratings from reviews\nin many languages.",
            "journal ref": "Proceedings of the 2016 CHI Conference on Human Factors in\n  Computing Systems, CHI'16 Extended Abstracts",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1605.02129v1",
            "date": "2016-05-07 01:55:51+00:00",
            "title": "Adobe-MIT submission to the DSTC 4 Spoken Language Understanding pilot task",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee",
                "Trung H. Bui",
                "Hung H. Bui"
            ],
            "abstract": "The Dialog State Tracking Challenge 4 (DSTC 4) proposes several pilot tasks.\nIn this paper, we focus on the spoken language understanding pilot task, which\nconsists of tagging a given utterance with speech acts and semantic slots. We\ncompare different classifiers: the best system obtains 0.52 and 0.67 F1-scores\non the test set for speech act recognition for the tourist and the guide\nrespectively, and 0.52 F1-score for semantic tagging for both the guide and the\ntourist.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02130v1",
            "date": "2016-05-07 02:00:30+00:00",
            "title": "Robust Dialog State Tracking for Large Ontologies",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee",
                "Trung H. Bui",
                "Hung H. Bui"
            ],
            "abstract": "The Dialog State Tracking Challenge 4 (DSTC 4) differentiates itself from the\nprevious three editions as follows: the number of slot-value pairs present in\nthe ontology is much larger, no spoken language understanding output is given,\nand utterances are labeled at the subdialog level. This paper describes a novel\ndialog state tracking method designed to work robustly under these conditions,\nusing elaborate string matching, coreference resolution tailored for dialogs\nand a few other improvements. The method can correctly identify many values\nthat are not explicitly present in the utterance. On the final evaluation, our\nmethod came in first among 7 competing teams and 24 entries. The F1-score\nachieved by our method was 9 and 7 percentage points higher than that of the\nrunner-up for the utterance-level evaluation and for the subdialog-level\nevaluation, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.02442v1",
            "date": "2016-05-09 07:14:52+00:00",
            "title": "Machine Learning Techniques with Ontology for Subjective Answer Evaluation",
            "authors": [
                "M. Syamala Devi",
                "Himani Mittal"
            ],
            "abstract": "Computerized Evaluation of English Essays is performed using Machine learning\ntechniques like Latent Semantic Analysis (LSA), Generalized LSA, Bilingual\nEvaluation Understudy and Maximum Entropy. Ontology, a concept map of domain\nknowledge, can enhance the performance of these techniques. Use of Ontology\nmakes the evaluation process holistic as presence of keywords, synonyms, the\nright word combination and coverage of concepts can be checked. In this paper,\nthe above mentioned techniques are implemented both with and without Ontology\nand tested on common input data consisting of technical answers of Computer\nScience. Domain Ontology of Computer Graphics is designed and developed. The\nsoftware used for implementation includes Java Programming Language and tools\nsuch as MATLAB, Prot\\'eg\\'e, etc. Ten questions from Computer Graphics with\nsixty answers for each question are used for testing. The results are analyzed\nand it is concluded that the results are more accurate with use of Ontology.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1605.02697v2",
            "date": "2016-05-09 19:04:23+00:00",
            "title": "Ask Your Neurons: A Deep Learning Approach to Visual Question Answering",
            "authors": [
                "Mateusz Malinowski",
                "Marcus Rohrbach",
                "Mario Fritz"
            ],
            "abstract": "We address a question answering task on real-world images that is set up as a\nVisual Turing Test. By combining latest advances in image representation and\nnatural language processing, we propose Ask Your Neurons, a scalable, jointly\ntrained, end-to-end formulation to this problem.\n  In contrast to previous efforts, we are facing a multi-modal problem where\nthe language output (answer) is conditioned on visual and natural language\ninputs (image and question). We provide additional insights into the problem by\nanalyzing how much information is contained only in the language part for which\nwe provide a new human baseline. To study human consensus, which is related to\nthe ambiguities inherent in this challenging task, we propose two novel metrics\nand collect additional answers which extend the original DAQUAR dataset to\nDAQUAR-Consensus.\n  Moreover, we also extend our analysis to VQA, a large-scale question\nanswering about images dataset, where we investigate some particular design\nchoices and show the importance of stronger visual models. At the same time, we\nachieve strong performance of our model that still uses a global image\nrepresentation. Finally, based on such analysis, we refine our Ask Your Neurons\non DAQUAR, which also leads to a better performance on this challenging task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1605.03835v1",
            "date": "2016-05-12 14:39:50+00:00",
            "title": "Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model",
            "authors": [
                "Kyunghyun Cho"
            ],
            "abstract": "Recent advances in conditional recurrent language modelling have mainly\nfocused on network architectures (e.g., attention mechanism), learning\nalgorithms (e.g., scheduled sampling and sequence-level training) and novel\napplications (e.g., image/video description generation, speech recognition,\netc.) On the other hand, we notice that decoding algorithms/strategies have not\nbeen investigated as much, and it has become standard to use greedy or beam\nsearch. In this paper, we propose a novel decoding strategy motivated by an\nearlier observation that nonlinear hidden layers of a deep neural network\nstretch the data manifold. The proposed strategy is embarrassingly\nparallelizable without any communication overhead, while improving an existing\ndecoding algorithm. We extensively evaluate it with attention-based neural\nmachine translation on the task of En->Cz translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04013v2",
            "date": "2016-05-13 00:32:01+00:00",
            "title": "A Corpus-based Toy Model for DisCoCat",
            "authors": [
                "Stefano Gogioso"
            ],
            "abstract": "The categorical compositional distributional (DisCoCat) model of meaning\nrigorously connects distributional semantics and pregroup grammars, and has\nfound a variety of applications in computational linguistics. From a more\nabstract standpoint, the DisCoCat paradigm predicates the construction of a\nmapping from syntax to categorical semantics. In this work we present a\nconcrete construction of one such mapping, from a toy model of syntax for\ncorpora annotated with constituent structure trees, to categorical semantics\ntaking place in a category of free R-semimodules over an involutive commutative\nsemiring R.",
            "journal ref": "EPTCS 221, 2016, pp. 20-28",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04072v1",
            "date": "2016-05-13 07:31:50+00:00",
            "title": "Towards Empathetic Human-Robot Interactions",
            "authors": [
                "Pascale Fung",
                "Dario Bertero",
                "Yan Wan",
                "Anik Dey",
                "Ricky Ho Yin Chan",
                "Farhad Bin Siddique",
                "Yang Yang",
                "Chien-Sheng Wu",
                "Ruixi Lin"
            ],
            "abstract": "Since the late 1990s when speech companies began providing their\ncustomer-service software in the market, people have gotten used to speaking to\nmachines. As people interact more often with voice and gesture controlled\nmachines, they expect the machines to recognize different emotions, and\nunderstand other high level communication features such as humor, sarcasm and\nintention. In order to make such communication possible, the machines need an\nempathy module in them which can extract emotions from human speech and\nbehavior and can decide the correct response of the robot. Although research on\nempathetic robots is still in the early stage, we described our approach using\nsignal processing techniques, sentiment analysis and machine learning\nalgorithms to make robots that can \"understand\" human emotion. We propose Zara\nthe Supergirl as a prototype system of empathetic robots. It is a software\nbased virtual android, with an animated cartoon character to present itself on\nthe screen. She will get \"smarter\" and more empathetic through its deep\nlearning algorithms, and by gathering more data and learning from it. In this\npaper, we present our work so far in the areas of deep learning of emotion and\nsentiment recognition, as well as humor recognition. We hope to explore the\nfuture direction of android development and how it can help improve people's\nlives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04122v1",
            "date": "2016-05-13 10:46:22+00:00",
            "title": "Natural Language Semantics and Computability",
            "authors": [
                "Richard Moot",
                "Christian Retor\u00e9"
            ],
            "abstract": "This paper is a reflexion on the computability of natural language semantics.\nIt does not contain a new model or new results in the formal semantics of\nnatural language: it is rather a computational analysis of the logical models\nand algorithms currently used in natural language semantics, defined as the\nmapping of a statement to logical formulas - formulas, because a statement can\nbe ambiguous. We argue that as long as possible world semantics is left out,\none can compute the semantic representation(s) of a given statement, including\naspects of lexical meaning. We also discuss the algorithmic complexity of this\nprocess.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04227v3",
            "date": "2016-05-12 19:44:04+00:00",
            "title": "Relation Schema Induction using Tensor Factorization with Side Information",
            "authors": [
                "Madhav Nimishakavi",
                "Uday Singh Saini",
                "Partha Talukdar"
            ],
            "abstract": "Given a set of documents from a specific domain (e.g., medical research\njournals), how do we automatically build a Knowledge Graph (KG) for that\ndomain? Automatic identification of relations and their schemas, i.e., type\nsignature of arguments of relations (e.g., undergo(Patient, Surgery)), is an\nimportant first step towards this goal. We refer to this problem as Relation\nSchema Induction (RSI). In this paper, we propose Schema Induction using\nCoupled Tensor Factorization (SICTF), a novel tensor factorization method for\nrelation schema induction. SICTF factorizes Open Information Extraction\n(OpenIE) triples extracted from a domain corpus along with additional side\ninformation in a principled way to induce relation schemas. To the best of our\nknowledge, this is the first application of tensor factorization for the RSI\nproblem. Through extensive experiments on multiple real-world datasets, we find\nthat SICTF is not only more accurate than state-of-the-art baselines, but also\nsignificantly faster (about 14x faster).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1605.04462v3",
            "date": "2016-05-14 20:02:05+00:00",
            "title": "Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health",
            "authors": [
                "Tim Althoff",
                "Kevin Clark",
                "Jure Leskovec"
            ],
            "abstract": "Mental illness is one of the most pressing public health issues of our time.\nWhile counseling and psychotherapy can be effective treatments, our knowledge\nabout how to conduct successful counseling conversations has been limited due\nto lack of large-scale data with labeled outcomes of the conversations. In this\npaper, we present a large-scale, quantitative study on the discourse of\ntext-message-based counseling conversations. We develop a set of novel\ncomputational discourse analysis methods to measure how various linguistic\naspects of conversations are correlated with conversation outcomes. Applying\ntechniques such as sequence-based conversation models, language model\ncomparisons, message clustering, and psycholinguistics-inspired word frequency\nanalyses, we discover actionable conversation strategies that are associated\nwith better conversation outcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.04655v2",
            "date": "2016-05-16 05:50:54+00:00",
            "title": "Joint Learning of Sentence Embeddings for Relevance and Entailment",
            "authors": [
                "Petr Baudis",
                "Silvestr Stanko",
                "Jan Sedivy"
            ],
            "abstract": "We consider the problem of Recognizing Textual Entailment within an\nInformation Retrieval context, where we must simultaneously determine the\nrelevancy as well as degree of entailment for individual pieces of evidence to\ndetermine a yes/no answer to a binary natural language question.\n  We compare several variants of neural networks for sentence embeddings in a\nsetting of decision-making based on evidence of varying relevance. We propose a\nbasic model to integrate evidence for entailment, show that joint training of\nthe sentence embeddings to model relevance and entailment is feasible even with\nno explicit per-evidence supervision, and show the importance of evaluating\nstrong baselines. We also demonstrate the benefit of carrying over text\ncomprehension model trained on an unrelated task for our small datasets.\n  Our research is motivated primarily by a new open dataset we introduce,\nconsisting of binary questions and news-based evidence snippets. We also apply\nthe proposed relevance-entailment model on a similar task of ranking\nmultiple-choice test answers, evaluating it on a preliminary dataset of school\ntest questions as well as the standard MCTest dataset, where we improve the\nneural model state-of-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05134v1",
            "date": "2016-05-17 12:33:24+00:00",
            "title": "A Semi-automatic Method for Efficient Detection of Stories on Social Media",
            "authors": [
                "Soroush Vosoughi",
                "Deb Roy"
            ],
            "abstract": "Twitter has become one of the main sources of news for many people. As\nreal-world events and emergencies unfold, Twitter is abuzz with hundreds of\nthousands of stories about the events. Some of these stories are harmless,\nwhile others could potentially be life-saving or sources of malicious rumors.\nThus, it is critically important to be able to efficiently track stories that\nspread on Twitter during these events. In this paper, we present a novel\nsemi-automatic tool that enables users to efficiently identify and track\nstories about real-world events on Twitter. We ran a user study with 25\nparticipants, demonstrating that compared to more conventional methods, our\ntool can increase the speed and the accuracy with which users can track stories\nabout real-world events.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1605.05150v1",
            "date": "2016-05-17 13:06:49+00:00",
            "title": "Automatic Detection and Categorization of Election-Related Tweets",
            "authors": [
                "Prashanth Vijayaraghavan",
                "Soroush Vosoughi",
                "Deb Roy"
            ],
            "abstract": "With the rise in popularity of public social media and micro-blogging\nservices, most notably Twitter, the people have found a venue to hear and be\nheard by their peers without an intermediary. As a consequence, and aided by\nthe public nature of Twitter, political scientists now potentially have the\nmeans to analyse and understand the narratives that organically form, spread\nand decline among the public in a political campaign. However, the volume and\ndiversity of the conversation on Twitter, combined with its noisy and\nidiosyncratic nature, make this a hard task. Thus, advanced data mining and\nlanguage processing techniques are required to process and analyse the data. In\nthis paper, we present and evaluate a technical framework, based on recent\nadvances in deep neural networks, for identifying and analysing\nelection-related conversation on Twitter on a continuous, longitudinal basis.\nOur models can detect election-related tweets with an F-score of 0.92 and can\ncategorize these tweets into 22 topics with an F-score of 0.90.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05166v1",
            "date": "2016-05-17 13:47:24+00:00",
            "title": "Digital Stylometry: Linking Profiles Across Social Networks",
            "authors": [
                "Soroush Vosoughi",
                "Helen Zhou",
                "Deb Roy"
            ],
            "abstract": "There is an ever growing number of users with accounts on multiple social\nmedia and networking sites. Consequently, there is increasing interest in\nmatching user accounts and profiles across different social networks in order\nto create aggregate profiles of users. In this paper, we present models for\nDigital Stylometry, which is a method for matching users through stylometry\ninspired techniques. We experimented with linguistic, temporal, and combined\ntemporal-linguistic models for matching user accounts, using standard and novel\ntechniques. Using publicly available data, our best model, a combined\ntemporal-linguistic one, was able to correctly match the accounts of 31% of\n5,612 distinct users across Twitter and Facebook.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1605.05195v2",
            "date": "2016-05-17 14:51:54+00:00",
            "title": "Enhanced Twitter Sentiment Classification Using Contextual Information",
            "authors": [
                "Soroush Vosoughi",
                "Helen Zhou",
                "Deb Roy"
            ],
            "abstract": "The rise in popularity and ubiquity of Twitter has made sentiment analysis of\ntweets an important and well-covered area of research. However, the 140\ncharacter limit imposed on tweets makes it hard to use standard linguistic\nmethods for sentiment classification. On the other hand, what tweets lack in\nstructure they make up with sheer volume and rich metadata. This metadata\nincludes geolocation, temporal and author information. We hypothesize that\nsentiment is dependent on all these contextual factors. Different locations,\ntimes and authors have different emotional valences. In this paper, we explored\nthis hypothesis by utilizing distant supervision to collect millions of\nlabelled tweets from different locations, times and authors. We used this data\nto analyse the variation of tweet sentiments across different authors, times\nand locations. Once we explored and understood the relationship between these\nvariables and sentiment, we used a Bayesian approach to combine these variables\nwith more standard linguistic features such as n-grams to create a Twitter\nsentiment classifier. This combined classifier outperforms the purely\nlinguistic classifier, showing that integrating the rich contextual information\navailable on Twitter into sentiment classification is a promising direction of\nresearch.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1605.05362v1",
            "date": "2016-05-17 20:52:33+00:00",
            "title": "Yelp Dataset Challenge: Review Rating Prediction",
            "authors": [
                "Nabiha Asghar"
            ],
            "abstract": "Review websites, such as TripAdvisor and Yelp, allow users to post online\nreviews for various businesses, products and services, and have been recently\nshown to have a significant influence on consumer shopping behaviour. An online\nreview typically consists of free-form text and a star rating out of 5. The\nproblem of predicting a user's star rating for a product, given the user's text\nreview for that product, is called Review Rating Prediction and has lately\nbecome a popular, albeit hard, problem in machine learning. In this paper, we\ntreat Review Rating Prediction as a multi-class classification problem, and\nbuild sixteen different prediction models by combining four feature extraction\nmethods, (i) unigrams, (ii) bigrams, (iii) trigrams and (iv) Latent Semantic\nIndexing, with four machine learning algorithms, (i) logistic regression, (ii)\nNaive Bayes classification, (iii) perceptrons, and (iv) linear Support Vector\nClassification. We analyse the performance of each of these sixteen models to\ncome up with the best model for predicting the ratings from reviews. We use the\ndataset provided by Yelp for training and testing the models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.05894v2",
            "date": "2016-05-19 11:32:29+00:00",
            "title": "Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages",
            "authors": [
                "Muhammad Imran",
                "Prasenjit Mitra",
                "Carlos Castillo"
            ],
            "abstract": "Microblogging platforms such as Twitter provide active communication channels\nduring mass convergence and emergency events such as earthquakes, typhoons.\nDuring the sudden onset of a crisis situation, affected people post useful\ninformation on Twitter that can be used for situational awareness and other\nhumanitarian disaster response efforts, if processed timely and effectively.\nProcessing social media information pose multiple challenges such as parsing\nnoisy, brief and informal messages, learning information categories from the\nincoming stream of messages and classifying them into different classes among\nothers. One of the basic necessities of many of these tasks is the availability\nof data, in particular human-annotated data. In this paper, we present\nhuman-annotated Twitter corpora collected during 19 different crises that took\nplace between 2013 and 2015. To demonstrate the utility of the annotations, we\ntrain machine learning classifiers. Moreover, we publish first largest word2vec\nword embeddings trained on 52 million crisis-related tweets. To deal with\ntweets language issues, we present human-annotated normalized lexical resources\nfor different lexical variations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06069v3",
            "date": "2016-05-19 17:59:02+00:00",
            "title": "A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues",
            "authors": [
                "Iulian Vlad Serban",
                "Alessandro Sordoni",
                "Ryan Lowe",
                "Laurent Charlin",
                "Joelle Pineau",
                "Aaron Courville",
                "Yoshua Bengio"
            ],
            "abstract": "Sequential data often possesses a hierarchical structure with complex\ndependencies between subsequences, such as found between the utterances in a\ndialogue. In an effort to model this kind of generative process, we propose a\nneural network-based generative architecture, with latent stochastic variables\nthat span a variable number of time steps. We apply the proposed model to the\ntask of dialogue response generation and compare it with recent neural network\narchitectures. We evaluate the model performance through automatic evaluation\nmetrics and by carrying out a human evaluation. The experiments demonstrate\nthat our model improves upon recently proposed models and that the latent\nvariables facilitate the generation of long outputs and maintain the context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06304v2",
            "date": "2016-05-20 11:52:01+00:00",
            "title": "Local communities obstruct global consensus: Naming game on multi-local-world networks",
            "authors": [
                "Yang Lou",
                "Guanrong Chen",
                "Zhengping Fan",
                "Luna Xiang"
            ],
            "abstract": "Community structure is essential for social communications, where individuals\nbelonging to the same community are much more actively interacting and\ncommunicating with each other than those in different communities within the\nhuman society. Naming game, on the other hand, is a social communication model\nthat simulates the process of learning a name of an object within a community\nof humans, where the individuals can generally reach global consensus\nasymptotically through iterative pair-wise conversations. The underlying\nnetwork indicates the relationships among the individuals. In this paper, three\ntypical topologies, namely random-graph, small-world and scale-free networks,\nare employed, which are embedded with the multi-local-world community\nstructure, to study the naming game. Simulations show that 1) the convergence\nprocess to global consensus is getting slower as the community structure\nbecomes more prominent, and eventually might fail; 2) if the inter-community\nconnections are sufficiently dense, neither the number nor the size of the\ncommunities affects the convergence process; and 3) for different topologies\nwith the same average node-degree, local clustering of individuals obstruct or\nprohibit global consensus to take place. The results reveal the role of local\ncommunities in a global naming game in social network studies.",
            "journal ref": "Physica A 492 (2018) 1741-1752",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1605.06650v2",
            "date": "2016-05-21 14:36:33+00:00",
            "title": "Latent Tree Models for Hierarchical Topic Detection",
            "authors": [
                "Peixian Chen",
                "Nevin L. Zhang",
                "Tengfei Liu",
                "Leonard K. M. Poon",
                "Zhourong Chen",
                "Farhan Khawar"
            ],
            "abstract": "We present a novel method for hierarchical topic detection where topics are\nobtained by clustering documents in multiple ways. Specifically, we model\ndocument collections using a class of graphical models called hierarchical\nlatent tree models (HLTMs). The variables at the bottom level of an HLTM are\nobserved binary variables that represent the presence/absence of words in a\ndocument. The variables at other levels are binary latent variables, with those\nat the lowest latent level representing word co-occurrence patterns and those\nat higher levels representing co-occurrence of patterns at the level below.\nEach latent variable gives a soft partition of the documents, and document\nclusters in the partitions are interpreted as topics. Latent variables at high\nlevels of the hierarchy capture long-range word co-occurrence patterns and\nhence give thematically more general topics, while those at low levels of the\nhierarchy capture short-range word co-occurrence patterns and give thematically\nmore specific topics. Unlike LDA-based topic models, HLTMs do not refer to a\ndocument generation process and use word variables instead of token variables.\nThey use a tree structure to model the relationships between topics and words,\nwhich is conducive to the discovery of meaningful topics and topic hierarchies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.06778v1",
            "date": "2016-05-22 12:14:55+00:00",
            "title": "openXBOW - Introducing the Passau Open-Source Crossmodal Bag-of-Words Toolkit",
            "authors": [
                "Maximilian Schmitt",
                "Bj\u00f6rn W. Schuller"
            ],
            "abstract": "We introduce openXBOW, an open-source toolkit for the generation of\nbag-of-words (BoW) representations from multimodal input. In the BoW principle,\nword histograms were first used as features in document classification, but the\nidea was and can easily be adapted to, e.g., acoustic or visual low-level\ndescriptors, introducing a prior step of vector quantisation. The openXBOW\ntoolkit supports arbitrary numeric input features and text input and\nconcatenates computed subbags to a final bag. It provides a variety of\nextensions and options. To our knowledge, openXBOW is the first publicly\navailable toolkit for the generation of crossmodal bags-of-words. The\ncapabilities of the tool are exemplified in two sample scenarios:\ntime-continuous speech-based emotion recognition and sentiment analysis in\ntweets where improved results over other feature representation forms were\nobserved.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1605.07133v1",
            "date": "2016-05-23 18:46:46+00:00",
            "title": "Towards Multi-Agent Communication-Based Language Learning",
            "authors": [
                "Angeliki Lazaridou",
                "Nghia The Pham",
                "Marco Baroni"
            ],
            "abstract": "We propose an interactive multimodal framework for language learning. Instead\nof being passively exposed to large amounts of natural text, our learners\n(implemented as feed-forward neural networks) engage in cooperative referential\ngames starting from a tabula rasa setup, and thus develop their own language\nfrom the need to communicate in order to succeed at the game. Preliminary\nexperiments provide promising results, but also suggest that it is important to\nensure that agents trained in this way do not develop an adhoc communication\ncode only effective for the game they are playing",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07427v1",
            "date": "2016-05-24 12:48:19+00:00",
            "title": "Hierarchical Memory Networks",
            "authors": [
                "Sarath Chandar",
                "Sungjin Ahn",
                "Hugo Larochelle",
                "Pascal Vincent",
                "Gerald Tesauro",
                "Yoshua Bengio"
            ],
            "abstract": "Memory networks are neural networks with an explicit memory component that\ncan be both read and written to by the network. The memory is often addressed\nin a soft way using a softmax function, making end-to-end training with\nbackpropagation possible. However, this is not computationally scalable for\napplications which require the network to read from extremely large memories.\nOn the other hand, it is well known that hard attention mechanisms based on\nreinforcement learning are challenging to train successfully. In this paper, we\nexplore a form of hierarchical memory network, which can be considered as a\nhybrid between hard and soft attention memory networks. The memory is organized\nin a hierarchical structure such that reading from it is done with less\ncomputation than soft attention over a flat memory, while also being easier to\ntrain than hard attention over a flat memory. Specifically, we propose to\nincorporate Maximum Inner Product Search (MIPS) in the training and inference\nprocedures for our hierarchical memory network. We explore the use of various\nstate-of-the art approximate MIPS techniques and report results on\nSimpleQuestions, a challenging large scale factoid question answering task.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1605.07733v1",
            "date": "2016-05-25 04:57:42+00:00",
            "title": "On model architecture for a children's speech recognition interactive dialog system",
            "authors": [
                "Radoslava Kraleva",
                "Velin Kralev"
            ],
            "abstract": "This report presents a general model of the architecture of information\nsystems for the speech recognition of children. It presents a model of the\nspeech data stream and how it works. The result of these studies and presented\nveins architectural model shows that research needs to be focused on\nacoustic-phonetic modeling in order to improve the quality of children's speech\nrecognition and the sustainability of the systems to noise and changes in\ntransmission environment. Another important aspect is the development of more\naccurate algorithms for modeling of spontaneous child speech.",
            "journal ref": "Third International Scientific Conference \"Mathematics and Natural\n  Sciences\", Vol. (1), pp. 106-111, 2009",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1605.07735v1",
            "date": "2016-05-25 05:04:11+00:00",
            "title": "Design and development a children's speech database",
            "authors": [
                "Radoslava Kraleva"
            ],
            "abstract": "The report presents the process of planning, designing and the development of\na database of spoken children's speech whose native language is Bulgarian. The\nproposed model is designed for children between the age of 4 and 6 without\nspeech disorders, and reflects their specific capabilities. At this age most\nchildren cannot read, there is no sustained concentration, they are emotional,\netc. The aim is to unite all the media information accompanying the recording\nand processing of spoken speech, thereby to facilitate the work of researchers\nin the field of speech recognition. This database will be used for the\ndevelopment of systems for children's speech recognition, children's speech\nsynthesis systems, games which allow voice control, etc. As a result of the\nproposed model a prototype system for speech recognition is presented.",
            "journal ref": "Fourth International Scientific Conference \"Mathematics and\n  Natural Sciences\" 2011, Bulgaria, Vol. (2), pp. 41-48",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.07844v2",
            "date": "2016-05-25 12:04:43+00:00",
            "title": "Dimension Projection among Languages based on Pseudo-relevant Documents for Query Translation",
            "authors": [
                "Javid Dadashkarimi",
                "Mahsa S. Shahshahani",
                "Amirhossein Tebbifakhr",
                "Heshaam Faili",
                "Azadeh Shakery"
            ],
            "abstract": "Using top-ranked documents in response to a query has been shown to be an\neffective approach to improve the quality of query translation in\ndictionary-based cross-language information retrieval. In this paper, we\npropose a new method for dictionary-based query translation based on dimension\nprojection of embedded vectors from the pseudo-relevant documents in the source\nlanguage to their equivalents in the target language. To this end, first we\nlearn low-dimensional vectors of the words in the pseudo-relevant collections\nseparately and then aim to find a query-dependent transformation matrix between\nthe vectors of translation pairs appeared in the collections. At the next step,\nrepresentation of each query term is projected to the target language and then,\nafter using a softmax function, a query-dependent translation model is built.\nFinally, the model is used for query translation. Our experiments on four CLEF\ncollections in French, Spanish, German, and Italian demonstrate that the\nproposed method outperforms a word embedding baseline based on bilingual\nshuffling and a further number of competitive baselines. The proposed method\nreaches up to 87% performance of machine translation (MT) in short queries and\nconsiderable improvements in verbose queries.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1605.07895v1",
            "date": "2016-05-25 14:23:21+00:00",
            "title": "Automatic Extraction of Causal Relations from Natural Language Texts: A Comprehensive Survey",
            "authors": [
                "Nabiha Asghar"
            ],
            "abstract": "Automatic extraction of cause-effect relationships from natural language\ntexts is a challenging open problem in Artificial Intelligence. Most of the\nearly attempts at its solution used manually constructed linguistic and\nsyntactic rules on small and domain-specific data sets. However, with the\nadvent of big data, the availability of affordable computing power and the\nrecent popularization of machine learning, the paradigm to tackle this problem\nhas slowly shifted. Machines are now expected to learn generic causal\nextraction rules from labelled data with minimal supervision, in a domain\nindependent-manner. In this paper, we provide a comprehensive survey of causal\nrelation extraction techniques from both paradigms, and analyse their relative\nstrengths and weaknesses, with recommendations for future work.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1605.07912v4",
            "date": "2016-05-25 14:49:58+00:00",
            "title": "Review Networks for Caption Generation",
            "authors": [
                "Zhilin Yang",
                "Ye Yuan",
                "Yuexin Wu",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "We propose a novel extension of the encoder-decoder framework, called a\nreview network. The review network is generic and can enhance any existing\nencoder- decoder model: in this paper, we consider RNN decoders with both CNN\nand RNN encoders. The review network performs a number of review steps with\nattention mechanism on the encoder hidden states, and outputs a thought vector\nafter each review step; the thought vectors are used as the input of the\nattention mechanism in the decoder. We show that conventional encoder-decoders\nare a special case of our framework. Empirically, we show that our framework\nimproves over state-of- the-art encoder-decoder systems on the tasks of image\ncaptioning and source code captioning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1605.07918v1",
            "date": "2016-05-25 14:59:46+00:00",
            "title": "Automatic Open Knowledge Acquisition via Long Short-Term Memory Networks with Feedback Negative Sampling",
            "authors": [
                "Byungsoo Kim",
                "Hwanjo Yu",
                "Gary Geunbae Lee"
            ],
            "abstract": "Previous studies in Open Information Extraction (Open IE) are mainly based on\nextraction patterns. They manually define patterns or automatically learn them\nfrom a large corpus. However, these approaches are limited when grasping the\ncontext of a sentence, and they fail to capture implicit relations. In this\npaper, we address this problem with the following methods. First, we exploit\nlong short-term memory (LSTM) networks to extract higher-level features along\nthe shortest dependency paths, connecting headwords of relations and arguments.\nThe path-level features from LSTM networks provide useful clues regarding\ncontextual information and the validity of arguments. Second, we constructed\nsamples to train LSTM networks without the need for manual labeling. In\nparticular, feedback negative sampling picks highly negative samples among\nnon-positive samples through a model trained with positive samples. The\nexperimental results show that our approach produces more precise and abundant\nextractions than state-of-the-art open IE systems. To the best of our\nknowledge, this is the first work to apply deep learning to Open IE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.08535v3",
            "date": "2016-05-27 08:27:18+00:00",
            "title": "Deep API Learning",
            "authors": [
                "Xiaodong Gu",
                "Hongyu Zhang",
                "Dongmei Zhang",
                "Sunghun Kim"
            ],
            "abstract": "Developers often wonder how to implement a certain functionality (e.g., how\nto parse XML files) using APIs. Obtaining an API usage sequence based on an\nAPI-related natural language query is very helpful in this regard. Given a\nquery, existing approaches utilize information retrieval models to search for\nmatching API sequences. These approaches treat queries and APIs as bag-of-words\n(i.e., keyword matching or word-to-word alignment) and lack a deep\nunderstanding of the semantics of the query.\n  We propose DeepAPI, a deep learning based approach to generate API usage\nsequences for a given natural language query. Instead of a bags-of-words\nassumption, it learns the sequence of words in a query and the sequence of\nassociated APIs. DeepAPI adapts a neural language model named RNN\nEncoder-Decoder. It encodes a word sequence (user query) into a fixed-length\ncontext vector, and generates an API sequence based on the context vector. We\nalso augment the RNN Encoder-Decoder by considering the importance of\nindividual APIs. We empirically evaluate our approach with more than 7 million\nannotated code snippets collected from GitHub. The results show that our\napproach generates largely accurate API sequences and outperforms the related\napproaches.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1605.08764v1",
            "date": "2016-05-27 19:31:54+00:00",
            "title": "Stacking With Auxiliary Features",
            "authors": [
                "Nazneen Fatema Rajani",
                "Raymond J. Mooney"
            ],
            "abstract": "Ensembling methods are well known for improving prediction accuracy. However,\nthey are limited in the sense that they cannot discriminate among component\nmodels effectively. In this paper, we propose stacking with auxiliary features\nthat learns to fuse relevant information from multiple systems to improve\nperformance. Auxiliary features enable the stacker to rely on systems that not\njust agree on an output but also the provenance of the output. We demonstrate\nour approach on three very different and difficult problems -- the Cold Start\nSlot Filling, the Tri-lingual Entity Discovery and Linking and the ImageNet\nobject detection tasks. We obtain new state-of-the-art results on the first two\ntasks and substantial improvements on the detection task, thus verifying the\npower and generality of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.09186v4",
            "date": "2016-05-30 11:47:00+00:00",
            "title": "Does Multimodality Help Human and Machine for Translation and Image Captioning?",
            "authors": [
                "Ozan Caglayan",
                "Walid Aransa",
                "Yaxing Wang",
                "Marc Masana",
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Fethi Bougares",
                "Lo\u00efc Barrault",
                "Joost van de Weijer"
            ],
            "abstract": "This paper presents the systems developed by LIUM and CVC for the WMT16\nMultimodal Machine Translation challenge. We explored various comparative\nmethods, namely phrase-based systems and attentional recurrent neural networks\nmodels trained using monomodal or multimodal data. We also performed a human\nevaluation in order to estimate the usefulness of multimodal data for human\nmachine translation and image description generation. Our systems obtained the\nbest results for both tasks according to the automatic evaluation metrics BLEU\nand METEOR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1605.09211v1",
            "date": "2016-05-30 12:57:44+00:00",
            "title": "Going Deeper for Multilingual Visual Sentiment Detection",
            "authors": [
                "Brendan Jou",
                "Shih-Fu Chang"
            ],
            "abstract": "This technical report details several improvements to the visual concept\ndetector banks built on images from the Multilingual Visual Sentiment Ontology\n(MVSO). The detector banks are trained to detect a total of 9,918\nsentiment-biased visual concepts from six major languages: English, Spanish,\nItalian, French, German and Chinese. In the original MVSO release,\nadjective-noun pair (ANP) detectors were trained for the six languages using an\nAlexNet-styled architecture by fine-tuning from DeepSentiBank. Here, through a\nmore extensive set of experiments, parameter tuning, and training runs, we\ndetail and release higher accuracy models for detecting ANPs across six\nlanguages from the same image pool and setting as in the original release using\na more modern architecture, GoogLeNet, providing comparable or better\nperformance with reduced network parameter cost.\n  In addition, since the image pool in MVSO can be corrupted by user noise from\nsocial interactions, we partitioned out a sub-corpus of MVSO images based on\ntag-restricted queries for higher fidelity labels. We show that as a result of\nthese higher fidelity labels, higher performing AlexNet-styled ANP detectors\ncan be trained using the tag-restricted image subset as compared to the models\nin full corpus. We release all these newly trained models for public research\nuse along with the list of tag-restricted images from the MVSO dataset.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1605.09553v2",
            "date": "2016-05-31 10:04:20+00:00",
            "title": "Attention Correctness in Neural Image Captioning",
            "authors": [
                "Chenxi Liu",
                "Junhua Mao",
                "Fei Sha",
                "Alan Yuille"
            ],
            "abstract": "Attention mechanisms have recently been introduced in deep learning for\nvarious tasks in natural language processing and computer vision. But despite\ntheir popularity, the \"correctness\" of the implicitly-learned attention maps\nhas only been assessed qualitatively by visualization of several examples. In\nthis paper we focus on evaluating and improving the correctness of attention in\nneural image captioning models. Specifically, we propose a quantitative\nevaluation metric for the consistency between the generated attention maps and\nhuman annotations, using recently released datasets with alignment between\nregions in images and entities in captions. We then propose novel models with\ndifferent levels of explicit supervision for learning attention maps during\ntraining. The supervision can be strong when alignment between regions and\ncaption entities are available, or weak when only object segments and\ncategories are provided. We show on the popular Flickr30k and COCO datasets\nthat introducing supervision of attention maps during training solidly improves\nboth attention correctness and caption quality, showing the promise of making\nmachine perception more human-like.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1605.09564v1",
            "date": "2016-05-31 10:31:16+00:00",
            "title": "Determining the Characteristic Vocabulary for a Specialized Dictionary using Word2vec and a Directed Crawler",
            "authors": [
                "Gregory Grefenstette",
                "Lawrence Muchemi"
            ],
            "abstract": "Specialized dictionaries are used to understand concepts in specific domains,\nespecially where those concepts are not part of the general vocabulary, or\nhaving meanings that differ from ordinary languages. The first step in creating\na specialized dictionary involves detecting the characteristic vocabulary of\nthe domain in question. Classical methods for detecting this vocabulary involve\ngathering a domain corpus, calculating statistics on the terms found there, and\nthen comparing these statistics to a background or general language corpus.\nTerms which are found significantly more often in the specialized corpus than\nin the background corpus are candidates for the characteristic vocabulary of\nthe domain. Here we present two tools, a directed crawler, and a distributional\nsemantics package, that can be used together, circumventing the need of a\nbackground corpus. Both tools are available on the web.",
            "journal ref": "GLOBALEX 2016: Lexicographic Resources for Human Language\n  Technology, May 2016, Portoroz, Slovenia. 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00138v2",
            "date": "2016-12-31 16:41:43+00:00",
            "title": "Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization",
            "authors": [
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "This paper tackles the reduction of redundant repeating generation that is\noften observed in RNN-based encoder-decoder models. Our basic idea is to\njointly estimate the upper-bound frequency of each target vocabulary in the\nencoder and control the output words based on the estimation in the decoder.\nOur method shows significant improvement over a strong RNN-based\nencoder-decoder baseline and achieved its best results on an abstractive\nsummarization benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.00776v2",
            "date": "2016-06-02 17:37:31+00:00",
            "title": "Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation",
            "authors": [
                "Iulian Vlad Serban",
                "Tim Klinger",
                "Gerald Tesauro",
                "Kartik Talamadupula",
                "Bowen Zhou",
                "Yoshua Bengio",
                "Aaron Courville"
            ],
            "abstract": "We introduce the multiresolution recurrent neural network, which extends the\nsequence-to-sequence framework to model natural language generation as two\nparallel discrete stochastic processes: a sequence of high-level coarse tokens,\nand a sequence of natural language tokens. There are many ways to estimate or\nlearn the high-level coarse tokens, but we argue that a simple extraction\nprocedure is sufficient to capture a wealth of high-level discourse semantics.\nSuch procedure allows training the multiresolution recurrent neural network by\nmaximizing the exact joint log-likelihood over both sequences. In contrast to\nthe standard log- likelihood objective w.r.t. natural language tokens (word\nperplexity), optimizing the joint log-likelihood biases the model towards\nmodeling high-level abstractions. We apply the proposed model to the task of\ndialogue response generation in two challenging domains: the Ubuntu technical\nsupport domain, and Twitter conversations. On Ubuntu, the model outperforms\ncompeting approaches by a substantial margin, achieving state-of-the-art\nresults according to both automatic evaluation metrics and a human evaluation\nstudy. On Twitter, the model appears to generate more relevant and on-topic\nresponses according to automatic evaluation metrics. Finally, our experiments\ndemonstrate that the proposed model is more adept at overcoming the sparsity of\nnatural language and is better able to capture long-term structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.02979v2",
            "date": "2016-06-09 14:45:39+00:00",
            "title": "Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)",
            "authors": [
                "Shaohua Li",
                "Tat-Seng Chua",
                "Jun Zhu",
                "Chunyan Miao"
            ],
            "abstract": "Word embedding maps words into a low-dimensional continuous embedding space\nby exploiting the local word collocation patterns in a small context window. On\nthe other hand, topic modeling maps documents onto a low-dimensional topic\nspace, by utilizing the global word collocation patterns in the same document.\nThese two types of patterns are complementary. In this paper, we propose a\ngenerative topic embedding model to combine the two types of patterns. In our\nmodel, topics are represented by embedding vectors, and are shared across\ndocuments. The probability of each word is influenced by both its local context\nand its topic. A variational inference method yields the topic embeddings as\nwell as the topic mixing proportions for each document. Jointly they represent\nthe document in a low-dimensional continuous space. In two document\nclassification tasks, our method performs better than eight existing methods,\nwith fewer features. In addition, we illustrate with an example that our method\ncan generate coherent topics even based on only one document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07298v1",
            "date": "2016-06-23 12:53:31+00:00",
            "title": "Explaining Predictions of Non-Linear Classifiers in NLP",
            "authors": [
                "Leila Arras",
                "Franziska Horn",
                "Gr\u00e9goire Montavon",
                "Klaus-Robert M\u00fcller",
                "Wojciech Samek"
            ],
            "abstract": "Layer-wise relevance propagation (LRP) is a recently proposed technique for\nexplaining predictions of complex non-linear classifiers in terms of input\nvariables. In this paper, we apply LRP for the first time to natural language\nprocessing (NLP). More precisely, we use it to explain the predictions of a\nconvolutional neural network (CNN) trained on a topic categorization task. Our\nanalysis highlights which words are relevant for a specific prediction of the\nCNN. We compare our technique to standard sensitivity analysis, both\nqualitatively and quantitatively, using a \"word deleting\" perturbation\nexperiment, a PCA analysis, and various visualizations. All experiments\nvalidate the suitability of LRP for explaining the CNN predictions, which is\nalso in line with results reported in recent image classification studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1606.07496v1",
            "date": "2016-06-23 22:04:08+00:00",
            "title": "Is a Picture Worth Ten Thousand Words in a Review Dataset?",
            "authors": [
                "Roberto Camacho Barranco",
                "Laura M. Rodriguez",
                "Rebecca Urbina",
                "M. Shahriar Hossain"
            ],
            "abstract": "While textual reviews have become prominent in many recommendation-based\nsystems, automated frameworks to provide relevant visual cues against text\nreviews where pictures are not available is a new form of task confronted by\ndata mining and machine learning researchers. Suggestions of pictures that are\nrelevant to the content of a review could significantly benefit the users by\nincreasing the effectiveness of a review. We propose a deep learning-based\nframework to automatically: (1) tag the images available in a review dataset,\n(2) generate a caption for each image that does not have one, and (3) enhance\neach review by recommending relevant images that might not be uploaded by the\ncorresponding reviewer. We evaluate the proposed framework using the Yelp\nChallenge Dataset. While a subset of the images in this particular dataset are\ncorrectly captioned, the majority of the pictures do not have any associated\ntext. Moreover, there is no mapping between reviews and images. Each image has\na corresponding business-tag where the picture was taken, though. The overall\ndata setting and unavailability of crucial pieces required for a mapping make\nthe problem of recommending images for reviews a major challenge. Qualitative\nand quantitative evaluations indicate that our proposed framework provides high\nquality enhancements through automatic captioning, tagging, and recommendation\nfor mapping reviews and images.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1606.08207v3",
            "date": "2016-06-27 11:04:41+00:00",
            "title": "Semantic homophily in online communication: evidence from Twitter",
            "authors": [
                "Sanja \u0160\u0107epanovi\u0107",
                "Igor Mishkovski",
                "Bruno Gon\u00e7alves",
                "Nguyen Trung Hieu",
                "Pan Hui"
            ],
            "abstract": "People are observed to assortatively connect on a set of traits. This\nphenomenon, termed assortative mixing or sometimes homophily, can be quantified\nthrough assortativity coefficient in social networks. Uncovering the exact\ncauses of strong assortative mixing found in social networks has been a\nresearch challenge. Among the main suggested causes from sociology are the\ntendency of similar individuals to connect (often itself referred as homophily)\nand the social influence among already connected individuals. An important\nquestion to researchers and in practice can be tackled, as we present here:\nunderstanding the exact mechanisms of interplay between these tendencies and\nthe underlying social network structure. Namely, in addition to the mentioned\nassortativity coefficient, there are several other static and temporal network\nproperties and substructures that can be linked to the tendencies of homophily\nand social influence in the social network and we herein investigate those.\nConcretely, we tackle a computer-mediated \\textit{communication network} (based\non Twitter mentions) and a particular type of assortative mixing that can be\ninferred from the semantic features of communication content that we term\n\\textit{semantic homophily}. Our work, to the best of our knowledge, is the\nfirst to offer an in-depth analysis on semantic homophily in a communication\nnetwork and the interplay between them. We quantify diverse levels of semantic\nhomophily, identify the semantic aspects that are the drivers of observed\nhomophily, show insights in its temporal evolution and finally, we present its\nintricate interplay with the communication network on Twitter. By analyzing\nthese mechanisms we increase understanding on what are the semantic aspects\nthat shape and how they shape the human computer-mediated communication.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1608.08927v1",
            "date": "2016-08-31 16:23:07+00:00",
            "title": "The Generalized Smallest Grammar Problem",
            "authors": [
                "Payam Siyari",
                "Matthias Gall\u00e9"
            ],
            "abstract": "The Smallest Grammar Problem -- the problem of finding the smallest\ncontext-free grammar that generates exactly one given sequence -- has never\nbeen successfully applied to grammatical inference. We investigate the reasons\nand propose an extended formulation that seeks to minimize non-recursive\ngrammars, instead of straight-line programs. In addition, we provide very\nefficient algorithms that approximate the minimization problem of this class of\ngrammars. Our empirical evaluation shows that we are able to find smaller\nmodels than the current best approximations to the Smallest Grammar Problem on\nstandard benchmarks, and that the inferred rules capture much better the\nsyntactic structure of natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1611.00448v1",
            "date": "2016-11-02 02:32:05+00:00",
            "title": "Natural-Parameter Networks: A Class of Probabilistic Neural Networks",
            "authors": [
                "Hao Wang",
                "Xingjian Shi",
                "Dit-Yan Yeung"
            ],
            "abstract": "Neural networks (NN) have achieved state-of-the-art performance in various\napplications. Unfortunately in applications where training data is\ninsufficient, they are often prone to overfitting. One effective way to\nalleviate this problem is to exploit the Bayesian approach by using Bayesian\nneural networks (BNN). Another shortcoming of NN is the lack of flexibility to\ncustomize different distributions for the weights and neurons according to the\ndata, as is often done in probabilistic graphical models. To address these\nproblems, we propose a class of probabilistic neural networks, dubbed\nnatural-parameter networks (NPN), as a novel and lightweight Bayesian treatment\nof NN. NPN allows the usage of arbitrary exponential-family distributions to\nmodel the weights and neurons. Different from traditional NN and BNN, NPN takes\ndistributions as input and goes through layers of transformation before\nproducing distributions to match the target output distributions. As a Bayesian\ntreatment, efficient backpropagation (BP) is performed to learn the natural\nparameters for the distributions over both the weights and neurons. The output\ndistributions of each layer, as byproducts, may be used as second-order\nrepresentations for the associated tasks such as link prediction. Experiments\non real-world datasets show that NPN can achieve state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.00454v1",
            "date": "2016-11-02 02:49:44+00:00",
            "title": "Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks",
            "authors": [
                "Hao Wang",
                "Xingjian Shi",
                "Dit-Yan Yeung"
            ],
            "abstract": "Hybrid methods that utilize both content and rating information are commonly\nused in many recommender systems. However, most of them use either handcrafted\nfeatures or the bag-of-words representation as a surrogate for the content\ninformation but they are neither effective nor natural enough. To address this\nproblem, we develop a collaborative recurrent autoencoder (CRAE) which is a\ndenoising recurrent autoencoder (DRAE) that models the generation of content\nsequences in the collaborative filtering (CF) setting. The model generalizes\nrecent advances in recurrent deep learning from i.i.d. input to non-i.i.d.\n(CF-based) input and provides a new denoising scheme along with a novel\nlearnable pooling scheme for the recurrent autoencoder. To do this, we first\ndevelop a hierarchical Bayesian model for the DRAE and then generalize it to\nthe CF setting. The synergy between denoising and CF enables CRAE to make\naccurate recommendations while learning to fill in the blanks in sequences.\nExperiments on real-world datasets from different domains (CiteULike and\nNetflix) show that, by jointly modeling the order-aware generation of sequences\nfor the content information and performing CF for the ratings, CRAE is able to\nsignificantly outperform the state of the art on both the recommendation task\nbased on ratings and the sequence generation task based on content information.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1611.01400v1",
            "date": "2016-11-04 14:43:44+00:00",
            "title": "Learning to Rank Scientific Documents from the Crowd",
            "authors": [
                "Jesse M Lingeman",
                "Hong Yu"
            ],
            "abstract": "Finding related published articles is an important task in any science, but\nwith the explosion of new work in the biomedical domain it has become\nespecially challenging. Most existing methodologies use text similarity metrics\nto identify whether two articles are related or not. However biomedical\nknowledge discovery is hypothesis-driven. The most related articles may not be\nones with the highest text similarities. In this study, we first develop an\ninnovative crowd-sourcing approach to build an expert-annotated\ndocument-ranking corpus. Using this corpus as the gold standard, we then\nevaluate the approaches of using text similarity to rank the relatedness of\narticles. Finally, we develop and evaluate a new supervised model to\nautomatically rank related scientific articles. Our results show that authors'\nranking differ significantly from rankings by text-similarity-based models. By\ntraining a learning-to-rank model on a subset of the annotated corpus, we found\nthe best supervised learning-to-rank model (SVM-Rank) significantly surpassed\nstate-of-the-art baseline systems.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1611.10277v4",
            "date": "2016-11-30 17:32:17+00:00",
            "title": "Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge",
            "authors": [
                "Ryan J. Gallagher",
                "Kyle Reing",
                "David Kale",
                "Greg Ver Steeg"
            ],
            "abstract": "While generative models such as Latent Dirichlet Allocation (LDA) have proven\nfruitful in topic modeling, they often require detailed assumptions and careful\nspecification of hyperparameters. Such model complexity issues only compound\nwhen trying to generalize generative models to incorporate human input. We\nintroduce Correlation Explanation (CorEx), an alternative approach to topic\nmodeling that does not assume an underlying generative model, and instead\nlearns maximally informative topics through an information-theoretic framework.\nThis framework naturally generalizes to hierarchical and semi-supervised\nextensions with no additional modeling assumptions. In particular, word-level\ndomain knowledge can be flexibly incorporated within CorEx through anchor\nwords, allowing topic separability and representation to be promoted with\nminimal human intervention. Across a variety of datasets, metrics, and\nexperiments, we demonstrate that CorEx produces topics that are comparable in\nquality to those produced by unsupervised and semi-supervised variants of LDA.",
            "journal ref": "Transactions of the Association for Computational Linguistics\n  (TACL), Vol. 5, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1607.00623v1",
            "date": "2016-07-03 10:30:40+00:00",
            "title": "Visualizing Natural Language Descriptions: A Survey",
            "authors": [
                "Kaveh Hassani",
                "Won-Sook Lee"
            ],
            "abstract": "A natural language interface exploits the conceptual simplicity and\nnaturalness of the language to create a high-level user-friendly communication\nchannel between humans and machines. One of the promising applications of such\ninterfaces is generating visual interpretations of semantic content of a given\nnatural language that can be then visualized either as a static scene or a\ndynamic animation. This survey discusses requirements and challenges of\ndeveloping such systems and reports 26 graphical systems that exploit natural\nlanguage interfaces and addresses both artificial intelligence and\nvisualization aspects. This work serves as a frame of reference to researchers\nand to enable further advances in the field.",
            "journal ref": "ACM Computing Surveys, Volume 49 Issue 1, Article No. 17, June\n  2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1601.04908v2",
            "date": "2016-01-19 13:13:25+00:00",
            "title": "Graded Entailment for Compositional Distributional Semantics",
            "authors": [
                "Desislava Bankova",
                "Bob Coecke",
                "Martha Lewis",
                "Daniel Marsden"
            ],
            "abstract": "The categorical compositional distributional model of natural language\nprovides a conceptually motivated procedure to compute the meaning of\nsentences, given grammatical structure and the meanings of its words. This\napproach has outperformed other models in mainstream empirical language\nprocessing tasks. However, until recently it has lacked the crucial feature of\nlexical entailment -- as do other distributional models of meaning.\n  In this paper we solve the problem of entailment for categorical\ncompositional distributional semantics. Taking advantage of the abstract\ncategorical framework allows us to vary our choice of model. This enables the\nintroduction of a notion of entailment, exploiting ideas from the categorical\nsemantics of partial knowledge in quantum computation.\n  The new model of language uses density matrices, on which we introduce a\nnovel robust graded order capturing the entailment strength between concepts.\nThis graded measure emerges from a general framework for approximate\nentailment, induced by any commutative monoid. Quantum logic embeds in our\ngraded order.\n  Our main theorem shows that entailment strength lifts compositionally to the\nsentence level, giving a lower bound on sentence entailment. We describe the\nessential properties of graded entailment such as continuity, and provide a\nprocedure for calculating entailment strength.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1602.04983v1",
            "date": "2016-02-16 11:04:29+00:00",
            "title": "Contextual Media Retrieval Using Natural Language Queries",
            "authors": [
                "Sreyasi Nag Chowdhury",
                "Mateusz Malinowski",
                "Andreas Bulling",
                "Mario Fritz"
            ],
            "abstract": "The widespread integration of cameras in hand-held and head-worn devices as\nwell as the ability to share content online enables a large and diverse visual\ncapture of the world that millions of users build up collectively every day. We\nenvision these images as well as associated meta information, such as GPS\ncoordinates and timestamps, to form a collective visual memory that can be\nqueried while automatically taking the ever-changing context of mobile users\ninto account. As a first step towards this vision, in this work we present\nXplore-M-Ego: a novel media retrieval system that allows users to query a\ndynamic database of images and videos using spatio-temporal natural language\nqueries. We evaluate our system using a new dataset of real user queries as\nwell as through a usability study. One key finding is that there is a\nconsiderable amount of inter-user variability, for example in the resolution of\nspatial relations in natural language utterances. We show that our retrieval\nsystem can cope with this variability using personalisation through an online\nlearning-based retrieval formulation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1609.01926v1",
            "date": "2016-09-07 10:44:28+00:00",
            "title": "A modular architecture for transparent computation in Recurrent Neural Networks",
            "authors": [
                "Giovanni Sirio Carmantini",
                "Peter beim Graben",
                "Mathieu Desroches",
                "Serafim Rodrigues"
            ],
            "abstract": "Computation is classically studied in terms of automata, formal languages and\nalgorithms; yet, the relation between neural dynamics and symbolic\nrepresentations and operations is still unclear in traditional eliminative\nconnectionism. Therefore, we suggest a unique perspective on this central\nissue, to which we would like to refer as to transparent connectionism, by\nproposing accounts of how symbolic computation can be implemented in neural\nsubstrates. In this study we first introduce a new model of dynamics on a\nsymbolic space, the versatile shift, showing that it supports the real-time\nsimulation of a range of automata. We then show that the Goedelization of\nversatile shifts defines nonlinear dynamical automata, dynamical systems\nevolving on a vectorial space. Finally, we present a mapping between nonlinear\ndynamical automata and recurrent artificial neural networks. The mapping\ndefines an architecture characterized by its granular modularity, where data,\nsymbolic operations and their control are not only distinguishable in\nactivation space, but also spatially localizable in the network itself, while\nmaintaining a distributed encoding of symbolic representations. The resulting\nnetworks simulate automata in real-time and are programmed directly, in absence\nof network training. To discuss the unique characteristics of the architecture\nand their consequences, we present two examples: i) the design of a Central\nPattern Generator from a finite-state locomotive controller, and ii) the\ncreation of a network simulating a system of interactive automata that supports\nthe parsing of garden-path sentences as investigated in psycholinguistics\nexperiments.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1609.06492v1",
            "date": "2016-09-21 10:52:03+00:00",
            "title": "Document Image Coding and Clustering for Script Discrimination",
            "authors": [
                "Darko Brodic",
                "Alessia Amelio",
                "Zoran N. Milivojevic",
                "Milena Jevtic"
            ],
            "abstract": "The paper introduces a new method for discrimination of documents given in\ndifferent scripts. The document is mapped into a uniformly coded text of\nnumerical values. It is derived from the position of the letters in the text\nline, based on their typographical characteristics. Each code is considered as\na gray level. Accordingly, the coded text determines a 1-D image, on which\ntexture analysis by run-length statistics and local binary pattern is\nperformed. It defines feature vectors representing the script content of the\ndocument. A modified clustering approach employed on document feature vector\ngroups documents written in the same script. Experimentation performed on two\ncustom oriented databases of historical documents in old Cyrillic, angular and\nround Glagolitic as well as Antiqua and Fraktur scripts demonstrates the\nsuperiority of the proposed method with respect to well-known methods in the\nstate-of-the-art.",
            "journal ref": "ICIC Express Letters Vol. 10 n. 7 July 2016 pp. 1561-1566",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1610.01076v1",
            "date": "2016-10-04 16:29:28+00:00",
            "title": "Tutorial on Answering Questions about Images with Deep Learning",
            "authors": [
                "Mateusz Malinowski",
                "Mario Fritz"
            ],
            "abstract": "Together with the development of more accurate methods in Computer Vision and\nNatural Language Understanding, holistic architectures that answer on questions\nabout the content of real-world images have emerged. In this tutorial, we build\na neural-based approach to answer questions about images. We base our tutorial\non two datasets: (mostly on) DAQUAR, and (a bit on) VQA. With small tweaks the\nmodels that we present here can achieve a competitive performance on both\ndatasets, in fact, they are among the best methods that use a combination of\nLSTM with a global, full frame CNN representation of an image. We hope that\nafter reading this tutorial, the reader will be able to use Deep Learning\nframeworks, such as Keras and introduced Kraino, to build various architectures\nthat will lead to a further performance improvement on this challenging task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1612.01928v1",
            "date": "2016-11-27 22:20:51+00:00",
            "title": "Invariant Representations for Noisy Speech Recognition",
            "authors": [
                "Dmitriy Serdyuk",
                "Kartik Audhkhasi",
                "Phil\u00e9mon Brakel",
                "Bhuvana Ramabhadran",
                "Samuel Thomas",
                "Yoshua Bengio"
            ],
            "abstract": "Modern automatic speech recognition (ASR) systems need to be robust under\nacoustic variability arising from environmental, speaker, channel, and\nrecording conditions. Ensuring such robustness to variability is a challenge in\nmodern day neural network-based ASR systems, especially when all types of\nvariability are not seen during training. We attempt to address this problem by\nencouraging the neural network acoustic model to learn invariant feature\nrepresentations. We use ideas from recent research on image generation using\nGenerative Adversarial Networks and domain adaptation ideas extending\nadversarial gradient-based training. A recent work from Ganin et al. proposes\nto use adversarial training for image domain adaptation by using an\nintermediate representation from the main target classification network to\ndeteriorate the domain classifier performance through a separate neural\nnetwork. Our work focuses on investigating neural architectures which produce\nrepresentations invariant to noise conditions for ASR. We evaluate the proposed\narchitecture on the Aurora-4 task, a popular benchmark for noise robust ASR. We\nshow that our method generalizes better than the standard multi-condition\ntraining especially when only a few noise categories are seen during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.02741v4",
            "date": "2016-12-08 17:45:16+00:00",
            "title": "Coupling Distributed and Symbolic Execution for Natural Language Queries",
            "authors": [
                "Lili Mou",
                "Zhengdong Lu",
                "Hang Li",
                "Zhi Jin"
            ],
            "abstract": "Building neural networks to query a knowledge base (a table) with natural\nlanguage is an emerging research topic in deep learning. An executor for table\nquerying typically requires multiple steps of execution because queries may\nhave complicated structures. In previous studies, researchers have developed\neither fully distributed executors or symbolic executors for table querying. A\ndistributed executor can be trained in an end-to-end fashion, but is weak in\nterms of execution efficiency and explicit interpretability. A symbolic\nexecutor is efficient in execution, but is very difficult to train especially\nat initial stages. In this paper, we propose to couple distributed and symbolic\nexecution for natural language queries, where the symbolic executor is\npretrained with the distributed executor's intermediate execution results in a\nstep-by-step fashion. Experiments show that our approach significantly\noutperforms both distributed and symbolic executors, exhibiting high accuracy,\nhigh learning efficiency, high execution efficiency, and high interpretability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1612.04418v1",
            "date": "2016-12-13 22:09:24+00:00",
            "title": "User Model-Based Intent-Aware Metrics for Multilingual Search Evaluation",
            "authors": [
                "Alexey Drutsa",
                "Andrey Shutovich",
                "Philipp Pushnyakov",
                "Evgeniy Krokhalyov",
                "Gleb Gusev",
                "Pavel Serdyukov"
            ],
            "abstract": "Despite the growing importance of multilingual aspect of web search, no\nappropriate offline metrics to evaluate its quality are proposed so far. At the\nsame time, personal language preferences can be regarded as intents of a query.\nThis approach translates the multilingual search problem into a particular task\nof search diversification. Furthermore, the standard intent-aware approach\ncould be adopted to build a diversified metric for multilingual search on the\nbasis of a classical IR metric such as ERR. The intent-aware approach estimates\nuser satisfaction under a user behavior model. We show however that the\nunderlying user behavior models is not realistic in the multilingual case, and\nthe produced intent-aware metric do not appropriately estimate the user\nsatisfaction. We develop a novel approach to build intent-aware user behavior\nmodels, which overcome these limitations and convert to quality metrics that\nbetter correlate with standard online metrics of user satisfaction.",
            "journal ref": "NIPS 2016 Workshop \"What If? Inference and Learning of\n  Hypothetical and Counterfactual Interventions in Complex Systems\" (What If\n  2016) pre-print",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1612.07182v2",
            "date": "2016-12-21 15:27:06+00:00",
            "title": "Multi-Agent Cooperation and the Emergence of (Natural) Language",
            "authors": [
                "Angeliki Lazaridou",
                "Alexander Peysakhovich",
                "Marco Baroni"
            ],
            "abstract": "The current mainstream approach to train natural language systems is to\nexpose them to large amounts of text. This passive learning is problematic if\nwe are interested in developing interactive machines, such as conversational\nagents. We propose a framework for language learning that relies on multi-agent\ncommunication. We study this learning in the context of referential games. In\nthese games, a sender and a receiver see a pair of images. The sender is told\none of them is the target and is allowed to send a message from a fixed,\narbitrary vocabulary to the receiver. The receiver must rely on this message to\nidentify the target. Thus, the agents develop their own language interactively\nout of the need to communicate. We show that two networks with simple\nconfigurations are able to learn to coordinate in the referential game. We\nfurther explore how to make changes to the game environment to cause the \"word\nmeanings\" induced in the game to better reflect intuitive semantic properties\nof the images. In addition, we present a simple strategy for grounding the\nagents' code into natural language. Both of these are necessary steps towards\ndeveloping machines that are able to communicate with humans productively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1612.08543v1",
            "date": "2016-12-27 09:10:18+00:00",
            "title": "Distributed Real-Time Sentiment Analysis for Big Data Social Streams",
            "authors": [
                "Amir Hossein Akhavan Rahnama"
            ],
            "abstract": "Big data trend has enforced the data-centric systems to have continuous fast\ndata streams. In recent years, real-time analytics on stream data has formed\ninto a new research field, which aims to answer queries about\nwhat-is-happening-now with a negligible delay. The real challenge with\nreal-time stream data processing is that it is impossible to store instances of\ndata, and therefore online analytical algorithms are utilized. To perform\nreal-time analytics, pre-processing of data should be performed in a way that\nonly a short summary of stream is stored in main memory. In addition, due to\nhigh speed of arrival, average processing time for each instance of data should\nbe in such a way that incoming instances are not lost without being captured.\nLastly, the learner needs to provide high analytical accuracy measures.\nSentinel is a distributed system written in Java that aims to solve this\nchallenge by enforcing both the processing and learning process to be done in\ndistributed form. Sentinel is built on top of Apache Storm, a distributed\ncomputing platform. Sentinels learner, Vertical Hoeffding Tree, is a parallel\ndecision tree-learning algorithm based on the VFDT, with ability of enabling\nparallel classification in distributed environments. Sentinel also uses\nSpaceSaving to keep a summary of the data stream and stores its summary in a\nsynopsis data structure. Application of Sentinel on Twitter Public Stream API\nis shown and the results are discussed.",
            "journal ref": "IEEE 2014 International Conference on Control, Decision and\n  Information Technologies (CoDIT)",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1603.03827v1",
            "date": "2016-03-12 00:02:51+00:00",
            "title": "Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks",
            "authors": [
                "Ji Young Lee",
                "Franck Dernoncourt"
            ],
            "abstract": "Recent approaches based on artificial neural networks (ANNs) have shown\npromising results for short-text classification. However, many short texts\noccur in sequences (e.g., sentences in a document or utterances in a dialog),\nand most existing ANN-based systems do not leverage the preceding short texts\nwhen classifying a subsequent one. In this work, we present a model based on\nrecurrent neural networks and convolutional neural networks that incorporates\nthe preceding short texts. Our model achieves state-of-the-art results on three\ndifferent datasets for dialog act prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1603.05670v2",
            "date": "2016-03-17 20:06:27+00:00",
            "title": "Bank distress in the news: Describing events through deep learning",
            "authors": [
                "Samuel R\u00f6nnqvist",
                "Peter Sarlin"
            ],
            "abstract": "While many models are purposed for detecting the occurrence of significant\nevents in financial systems, the task of providing qualitative detail on the\ndevelopments is not usually as well automated. We present a deep learning\napproach for detecting relevant discussion in text and extracting natural\nlanguage descriptions of events. Supervised by only a small set of event\ninformation, comprising entity names and dates, the model is leveraged by\nunsupervised learning of semantic vector representations on extensive text\ndata. We demonstrate applicability to the study of financial risk based on news\n(6.6M articles), particularly bank distress and government interventions (243\nevents), where indices can signal the level of bank-stress-related reporting at\nthe entity level, or aggregated at national or European level, while being\ncoupled with explanations. Thus, we exemplify how text, as timely, widely\navailable and descriptive data, can serve as a useful complementary source of\ninformation for financial and systemic risk analytics.",
            "journal ref": "Neurocomputing, 264, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.01219v1",
            "date": "2016-04-05 11:18:04+00:00",
            "title": "Learning to Generate Posters of Scientific Papers",
            "authors": [
                "Yuting Qiang",
                "Yanwei Fu",
                "Yanwen Guo",
                "Zhi-Hua Zhou",
                "Leonid Sigal"
            ],
            "abstract": "Researchers often summarize their work in the form of posters. Posters\nprovide a coherent and efficient way to convey core ideas from scientific\npapers. Generating a good scientific poster, however, is a complex and time\nconsuming cognitive task, since such posters need to be readable, informative,\nand visually aesthetic. In this paper, for the first time, we study the\nchallenging problem of learning to generate posters from scientific papers. To\nthis end, a data-driven framework, that utilizes graphical models, is proposed.\nSpecifically, given content to display, the key elements of a good poster,\nincluding panel layout and attributes of each panel, are learned and inferred\nfrom data. Then, given inferred layout and attributes, composition of graphical\nelements within each panel is synthesized. To learn and validate our model, we\ncollect and make public a Poster-Paper dataset, which consists of scientific\npapers and corresponding posters with exhaustively labelled panels and\nattributes. Qualitative and quantitative results indicate the effectiveness of\nour approach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1604.01272v1",
            "date": "2016-04-05 14:32:48+00:00",
            "title": "Feature extraction using Latent Dirichlet Allocation and Neural Networks: A case study on movie synopses",
            "authors": [
                "Despoina Christou"
            ],
            "abstract": "Feature extraction has gained increasing attention in the field of machine\nlearning, as in order to detect patterns, extract information, or predict\nfuture observations from big data, the urge of informative features is crucial.\nThe process of extracting features is highly linked to dimensionality reduction\nas it implies the transformation of the data from a sparse high-dimensional\nspace, to higher level meaningful abstractions. This dissertation employs\nNeural Networks for distributed paragraph representations, and Latent Dirichlet\nAllocation to capture higher level features of paragraph vectors. Although\nNeural Networks for distributed paragraph representations are considered the\nstate of the art for extracting paragraph vectors, we show that a quick topic\nanalysis model such as Latent Dirichlet Allocation can provide meaningful\nfeatures too. We evaluate the two methods on the CMU Movie Summary Corpus, a\ncollection of 25,203 movie plot summaries extracted from Wikipedia. Finally,\nfor both approaches, we use K-Nearest Neighbors to discover similar movies, and\nplot the projected representations using T-Distributed Stochastic Neighbor\nEmbedding to depict the context similarities. These similarities, expressed as\nmovie distances, can be used for movies recommendation. The recommended movies\nof this approach are compared with the recommended movies from IMDB, which use\na collaborative filtering recommendation approach, to show that our two models\ncould constitute either an alternative or a supplementary recommendation\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.03114v1",
            "date": "2016-04-11 20:00:04+00:00",
            "title": "Conversational flow in Oxford-style debates",
            "authors": [
                "Justine Zhang",
                "Ravi Kumar",
                "Sujith Ravi",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Public debates are a common platform for presenting and juxtaposing diverging\nviews on important issues. In this work we propose a methodology for tracking\nhow ideas flow between participants throughout a debate. We use this approach\nin a case study of Oxford-style debates---a competitive format where the winner\nis determined by audience votes---and show how the outcome of a debate depends\non aspects of conversational flow. In particular, we find that winners tend to\nmake better use of a debate's interactive component than losers, by actively\npursuing their opponents' points rather than promoting their own ideas over the\ncourse of the conversation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1604.07407v1",
            "date": "2016-04-25 20:00:02+00:00",
            "title": "Conversational Markers of Constructive Discussions",
            "authors": [
                "Vlad Niculae",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Group discussions are essential for organizing every aspect of modern life,\nfrom faculty meetings to senate debates, from grant review panels to papal\nconclaves. While costly in terms of time and organization effort, group\ndiscussions are commonly seen as a way of reaching better decisions compared to\nsolutions that do not require coordination between the individuals (e.g.\nvoting)---through discussion, the sum becomes greater than the parts. However,\nthis assumption is not irrefutable: anecdotal evidence of wasteful discussions\nabounds, and in our own experiments we find that over 30% of discussions are\nunproductive.\n  We propose a framework for analyzing conversational dynamics in order to\ndetermine whether a given task-oriented discussion is worth having or not. We\nexploit conversational patterns reflecting the flow of ideas and the balance\nbetween the participants, as well as their linguistic choices. We apply this\nframework to conversations naturally occurring in an online collaborative world\nexploration game developed and deployed to support this research. Using this\nsetting, we show that linguistic cues and conversational patterns extracted\nfrom the first 20 seconds of a team discussion are predictive of whether it\nwill be a wasteful or a productive one.",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2017": [
        {
            "url": "http://arxiv.org/abs/1702.00167v2",
            "date": "2017-02-01 09:04:54+00:00",
            "title": "SMPOST: Parts of Speech Tagger for Code-Mixed Indic Social Media Text",
            "authors": [
                "Deepak Gupta",
                "Shubham Tripathi",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Use of social media has grown dramatically during the last few years. Users\nfollow informal languages in communicating through social media. The language\nof communication is often mixed in nature, where people transcribe their\nregional language with English and this technique is found to be extremely\npopular. Natural language processing (NLP) aims to infer the information from\nthese text where Part-of-Speech (PoS) tagging plays an important role in\ngetting the prosody of the written text. For the task of PoS tagging on\nCode-Mixed Indian Social Media Text, we develop a supervised system based on\nConditional Random Field classifier. In order to tackle the problem\neffectively, we have focused on extracting rich linguistic features. We\nparticipate in three different language pairs, ie. English-Hindi,\nEnglish-Bengali and English-Telugu on three different social media platforms,\nTwitter, Facebook & WhatsApp. The proposed system is able to successfully\nassign coarse as well as fine-grained PoS tag labels for a given a code-mixed\nsentence. Experiments show that our system is quite generic that shows\nencouraging performance levels on all the three language pairs in all the\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00500v4",
            "date": "2017-02-01 23:36:33+00:00",
            "title": "AMR-to-text Generation with Synchronous Node Replacement Grammar",
            "authors": [
                "Linfeng Song",
                "Xiaochang Peng",
                "Yue Zhang",
                "Zhiguo Wang",
                "Daniel Gildea"
            ],
            "abstract": "This paper addresses the task of AMR-to-text generation by leveraging\nsynchronous node replacement grammar. During training, graph-to-string rules\nare learned using a heuristic extraction algorithm. At test time, a graph\ntransducer is applied to collapse input AMRs and generate output sentences.\nEvaluated on SemEval-2016 Task 8, our method gives a BLEU score of 25.62, which\nis the best reported so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00716v1",
            "date": "2017-02-02 15:41:46+00:00",
            "title": "Analysing Temporal Evolution of Interlingual Wikipedia Article Pairs",
            "authors": [
                "Simon Gottschalk",
                "Elena Demidova"
            ],
            "abstract": "Wikipedia articles representing an entity or a topic in different language\neditions evolve independently within the scope of the language-specific user\ncommunities. This can lead to different points of views reflected in the\narticles, as well as complementary and inconsistent information. An analysis of\nhow the information is propagated across the Wikipedia language editions can\nprovide important insights in the article evolution along the temporal and\ncultural dimensions and support quality control. To facilitate such analysis,\nwe present MultiWiki - a novel web-based user interface that provides an\noverview of the similarities and differences across the article pairs\noriginating from different language editions on a timeline. MultiWiki enables\nusers to observe the changes in the interlingual article similarity over time\nand to perform a detailed visual comparison of the article snapshots at a\nparticular time point.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00764v2",
            "date": "2017-02-02 17:53:29+00:00",
            "title": "Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Survey",
            "authors": [
                "Lorenzo Ferrone",
                "Fabio Massimo Zanzotto"
            ],
            "abstract": "Natural language is inherently a discrete symbolic representation of human\nknowledge. Recent advances in machine learning (ML) and in natural language\nprocessing (NLP) seem to contradict the above intuition: discrete symbols are\nfading away, erased by vectors or tensors called distributed and distributional\nrepresentations. However, there is a strict link between\ndistributed/distributional representations and discrete symbols, being the\nfirst an approximation of the second. A clearer understanding of the strict\nlink between distributed/distributional representations and symbols may\ncertainly lead to radically new deep learning networks. In this paper we make a\nsurvey that aims to renew the link between symbolic representations and\ndistributed/distributional representations. This is the right time to\nrevitalize the area of interpreting how discrete symbols are represented inside\nneural networks.",
            "journal ref": "Frontiers in Robotics and AI, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00992v2",
            "date": "2017-02-03 13:06:25+00:00",
            "title": "Automatic Prediction of Discourse Connectives",
            "authors": [
                "Eric Malmi",
                "Daniele Pighin",
                "Sebastian Krause",
                "Mikhail Kozhevnikov"
            ],
            "abstract": "Accurate prediction of suitable discourse connectives (however, furthermore,\netc.) is a key component of any system aimed at building coherent and fluent\ndiscourses from shorter sentences and passages. As an example, a dialog system\nmight assemble a long and informative answer by sampling passages extracted\nfrom different documents retrieved from the Web. We formulate the task of\ndiscourse connective prediction and release a dataset of 2.9M sentence pairs\nseparated by discourse connectives for this task. Then, we evaluate the\nhardness of the task for human raters, apply a recently proposed decomposable\nattention (DA) model to this task and observe that the automatic predictor has\na higher F1 than human raters (32 vs. 30). Nevertheless, under specific\nconditions the raters still outperform the DA model, suggesting that there is\nheadroom for future improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01101v1",
            "date": "2017-02-03 18:19:47+00:00",
            "title": "Multilingual Multi-modal Embeddings for Natural Language Processing",
            "authors": [
                "Iacer Calixto",
                "Qun Liu",
                "Nick Campbell"
            ],
            "abstract": "We propose a novel discriminative model that learns embeddings from\nmultilingual and multi-modal data, meaning that our model can take advantage of\nimages and descriptions in multiple languages to improve embedding quality. To\nthat end, we introduce a modification of a pairwise contrastive estimation\noptimisation function as our training objective. We evaluate our embeddings on\nan image-sentence ranking (ISR), a semantic textual similarity (STS), and a\nneural machine translation (NMT) task. We find that the additional multilingual\nsignals lead to improvements on both the ISR and STS tasks, and the\ndiscriminative cost can also be used in re-ranking $n$-best lists produced by\nNMT models, yielding strong improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01147v2",
            "date": "2017-02-03 20:31:34+00:00",
            "title": "Predicting Target Language CCG Supertags Improves Neural Machine Translation",
            "authors": [
                "Maria Nadejde",
                "Siva Reddy",
                "Rico Sennrich",
                "Tomasz Dwojak",
                "Marcin Junczys-Dowmunt",
                "Philipp Koehn",
                "Alexandra Birch"
            ],
            "abstract": "Neural machine translation (NMT) models are able to partially learn syntactic\ninformation from sequential lexical information. Still, some complex syntactic\nphenomena such as prepositional phrase attachment are poorly modeled. This work\naims to answer two questions: 1) Does explicitly modeling target language\nsyntax help NMT? 2) Is tight integration of words and syntax better than\nmultitask training? We introduce syntactic information in the form of CCG\nsupertags in the decoder, by interleaving the target supertags with the word\nsequence. Our results on WMT data show that explicitly modeling target-syntax\nimproves machine translation quality for German->English, a high-resource pair,\nand for Romanian->English, a low-resource pair and also several syntactic\nphenomena including prepositional phrase attachment. Furthermore, a tight\ncoupling of words and syntax improves translation quality more than multitask\ntraining. By combining target-syntax with adding source-side dependency labels\nin the embedding layer, we obtain a total improvement of 0.9 BLEU for\nGerman->English and 1.2 BLEU for Romanian->English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01287v1",
            "date": "2017-02-04 13:46:53+00:00",
            "title": "Doubly-Attentive Decoder for Multi-modal Neural Machine Translation",
            "authors": [
                "Iacer Calixto",
                "Qun Liu",
                "Nick Campbell"
            ],
            "abstract": "We introduce a Multi-modal Neural Machine Translation model in which a\ndoubly-attentive decoder naturally incorporates spatial visual features\nobtained using pre-trained convolutional neural networks, bridging the gap\nbetween image description and translation. Our decoder learns to attend to\nsource-language words and parts of an image independently by means of two\nseparate attention mechanisms as it generates words in the target language. We\nfind that our model can efficiently exploit not just back-translated in-domain\nmulti-modal data but also large general-domain text-only MT corpora. We also\nreport state-of-the-art results on the Multi30k data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01360v1",
            "date": "2017-02-05 02:22:31+00:00",
            "title": "An Empirical Evaluation of Zero Resource Acoustic Unit Discovery",
            "authors": [
                "Chunxi Liu",
                "Jinyi Yang",
                "Ming Sun",
                "Santosh Kesiraju",
                "Alena Rott",
                "Lucas Ondel",
                "Pegah Ghahremani",
                "Najim Dehak",
                "Lukas Burget",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Acoustic unit discovery (AUD) is a process of automatically identifying a\ncategorical acoustic unit inventory from speech and producing corresponding\nacoustic unit tokenizations. AUD provides an important avenue for unsupervised\nacoustic model training in a zero resource setting where expert-provided\nlinguistic knowledge and transcribed speech are unavailable. Therefore, to\nfurther facilitate zero-resource AUD process, in this paper, we demonstrate\nacoustic feature representations can be significantly improved by (i)\nperforming linear discriminant analysis (LDA) in an unsupervised self-trained\nfashion, and (ii) leveraging resources of other languages through building a\nmultilingual bottleneck (BN) feature extractor to give effective cross-lingual\ngeneralization. Moreover, we perform comprehensive evaluations of AUD efficacy\non multiple downstream speech applications, and their correlated performance\nsuggests that AUD evaluations are feasible using different alternative language\nresources when only a subset of these evaluation resources can be available in\ntypical zero resource applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01466v1",
            "date": "2017-02-05 23:16:01+00:00",
            "title": "Prepositions in Context",
            "authors": [
                "Hongyu Gong",
                "Jiaqi Mu",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Prepositions are highly polysemous, and their variegated senses encode\nsignificant semantic information. In this paper we match each preposition's\ncomplement and attachment and their interplay crucially to the geometry of the\nword vectors to the left and right of the preposition. Extracting such features\nfrom the vast number of instances of each preposition and clustering them makes\nfor an efficient preposition sense disambigution (PSD) algorithm, which is\ncomparable to and better than state-of-the-art on two benchmark datasets. Our\nreliance on no external linguistic resource allows us to scale the PSD\nalgorithm to a large WikiCorpus and learn sense-specific preposition\nrepresentations -- which we show to encode semantic relations and paraphrasing\nof verb particle compounds, via simple vector operations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01517v1",
            "date": "2017-02-06 07:29:01+00:00",
            "title": "Opinion Recommendation using Neural Memory Model",
            "authors": [
                "Zhongqing Wang",
                "Yue Zhang"
            ],
            "abstract": "We present opinion recommendation, a novel task of jointly predicting a\ncustom review with a rating score that a certain user would give to a certain\nproduct or service, given existing reviews and rating scores to the product or\nservice by other users, and the reviews that the user has given to other\nproducts and services. A characteristic of opinion recommendation is the\nreliance of multiple data sources for multi-task joint learning, which is the\nstrength of neural models. We use a single neural network to model users and\nproducts, capturing their correlation and generating customised product\nrepresentations using a deep memory network, from which customised ratings and\nreviews are constructed jointly. Results show that our opinion recommendation\nsystem gives ratings that are closer to real user ratings on Yelp.com data\ncompared with Yelp's own ratings, and our methods give better results compared\nto several pipelines baselines using state-of-the-art sentiment rating and\nsummarization systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01569v2",
            "date": "2017-02-06 11:22:15+00:00",
            "title": "Neural Semantic Parsing over Multiple Knowledge-bases",
            "authors": [
                "Jonathan Herzig",
                "Jonathan Berant"
            ],
            "abstract": "A fundamental challenge in developing semantic parsers is the paucity of\nstrong supervision in the form of language utterances annotated with logical\nform. In this paper, we propose to exploit structural regularities in language\nin different domains, and train semantic parsers over multiple knowledge-bases\n(KBs), while sharing information across datasets. We find that we can\nsubstantially improve parsing accuracy by training a single\nsequence-to-sequence model over multiple KBs, when providing an encoding of the\ndomain at decoding time. Our model achieves state-of-the-art performance on the\nOvernight dataset (containing eight domains), improves performance over a\nsingle KB baseline from 75.6% to 79.6%, while obtaining a 7x reduction in the\nnumber of model parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01587v1",
            "date": "2017-02-06 12:19:14+00:00",
            "title": "A Hybrid Approach For Hindi-English Machine Translation",
            "authors": [
                "Omkar Dhariya",
                "Shrikant Malviya",
                "Uma Shanker Tiwary"
            ],
            "abstract": "In this paper, an extended combined approach of phrase based statistical\nmachine translation (SMT), example based MT (EBMT) and rule based MT (RBMT) is\nproposed to develop a novel hybrid data driven MT system capable of\noutperforming the baseline SMT, EBMT and RBMT systems from which it is derived.\nIn short, the proposed hybrid MT process is guided by the rule based MT after\ngetting a set of partial candidate translations provided by EBMT and SMT\nsubsystems. Previous works have shown that EBMT systems are capable of\noutperforming the phrase-based SMT systems and RBMT approach has the strength\nof generating structurally and morphologically more accurate results. This\nhybrid approach increases the fluency, accuracy and grammatical precision which\nimprove the quality of a machine translation system. A comparison of the\nproposed hybrid machine translation (HTM) model with renowned translators i.e.\nGoogle, BING and Babylonian is also presented which shows that the proposed\nmodel works better on sentences with ambiguity as well as comprised of idioms\nthan others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01711v1",
            "date": "2017-02-06 17:14:29+00:00",
            "title": "Q-WordNet PPV: Simple, Robust and (almost) Unsupervised Generation of Polarity Lexicons for Multiple Languages",
            "authors": [
                "I\u00f1aki San Vicente",
                "Rodrigo Agerri",
                "German Rigau"
            ],
            "abstract": "This paper presents a simple, robust and (almost) unsupervised\ndictionary-based method, qwn-ppv (Q-WordNet as Personalized PageRanking Vector)\nto automatically generate polarity lexicons. We show that qwn-ppv outperforms\nother automatically generated lexicons for the four extrinsic evaluations\npresented here. It also shows very competitive and robust results with respect\nto manually annotated ones. Results suggest that no single lexicon is best for\nevery task and dataset and that the intrinsic evaluation of polarity lexicons\nis not a good performance indicator on a Sentiment Analysis task. The qwn-ppv\nmethod allows to easily create quality polarity lexicons whenever no\ndomain-based annotated corpora are available for a given language.",
            "journal ref": "Proceedings of the 14th Conference of the European Chapter of the\n  Association for Computational Linguistics (EACL 2014), pages 88-97,\n  Gothenburg, Sweden, April 26-30 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01714v1",
            "date": "2017-02-06 17:21:39+00:00",
            "title": "DNN adaptation by automatic quality estimation of ASR hypotheses",
            "authors": [
                "Daniele Falavigna",
                "Marco Matassoni",
                "Shahab Jalalvand",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "In this paper we propose to exploit the automatic Quality Estimation (QE) of\nASR hypotheses to perform the unsupervised adaptation of a deep neural network\nmodeling acoustic probabilities. Our hypothesis is that significant\nimprovements can be achieved by: i)automatically transcribing the evaluation\ndata we are currently trying to recognise, and ii) selecting from it a subset\nof \"good quality\" instances based on the word error rate (WER) scores predicted\nby a QE component. To validate this hypothesis, we run several experiments on\nthe evaluation data sets released for the CHiME-3 challenge. First, we operate\nin oracle conditions in which manual transcriptions of the evaluation data are\navailable, thus allowing us to compute the \"true\" sentence WER. In this\nscenario, we perform the adaptation with variable amounts of data, which are\ncharacterised by different levels of quality. Then, we move to realistic\nconditions in which the manual transcriptions of the evaluation data are not\navailable. In this case, the adaptation is performed on data selected according\nto the WER scores \"predicted\" by a QE component. Our results indicate that: i)\nQE predictions allow us to closely approximate the adaptation results obtained\nin oracle conditions, and ii) the overall ASR performance based on the proposed\nQE-driven adaptation method is significantly better than the strong, most\nrecent, CHiME-3 baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01776v2",
            "date": "2017-02-06 19:55:51+00:00",
            "title": "Multi-task memory networks for category-specific aspect and opinion terms co-extraction",
            "authors": [
                "Wenya Wang",
                "Sinno Jialin Pan",
                "Daniel Dahlmeier"
            ],
            "abstract": "In aspect-based sentiment analysis, most existing methods either focus on\naspect/opinion terms extraction or aspect terms categorization. However, each\ntask by itself only provides partial information to end users. To generate more\ndetailed and structured opinion analysis, we propose a finer-grained problem,\nwhich we call category-specific aspect and opinion terms extraction. This\nproblem involves the identification of aspect and opinion terms within each\nsentence, as well as the categorization of the identified terms. To this end,\nwe propose an end-to-end multi-task attention model, where each task\ncorresponds to aspect/opinion terms extraction for a specific category. Our\nmodel benefits from exploring the commonalities and relationships among\ndifferent tasks to address the data sparsity issue. We demonstrate its\nstate-of-the-art performance on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01802v2",
            "date": "2017-02-06 21:49:12+00:00",
            "title": "Ensemble Distillation for Neural Machine Translation",
            "authors": [
                "Markus Freitag",
                "Yaser Al-Onaizan",
                "Baskaran Sankaran"
            ],
            "abstract": "Knowledge distillation describes a method for training a student network to\nperform better by learning from a stronger teacher network. Translating a\nsentence with an Neural Machine Translation (NMT) engine is time expensive and\nhaving a smaller model speeds up this process. We demonstrate how to transfer\nthe translation quality of an ensemble and an oracle BLEU teacher network into\na single NMT system. Further, we present translation improvements from a\nteacher network that has the same architecture and dimensions of the student\nnetwork. As the training of the student model is still expensive, we introduce\na data filtering method based on the knowledge of the teacher model that not\nonly speeds up the training, but also leads to better translation quality. Our\ntechniques need no code change and can be easily reproduced with any NMT\narchitecture to speed up the decoding process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01806v2",
            "date": "2017-02-06 22:08:46+00:00",
            "title": "Beam Search Strategies for Neural Machine Translation",
            "authors": [
                "Markus Freitag",
                "Yaser Al-Onaizan"
            ],
            "abstract": "The basic concept in Neural Machine Translation (NMT) is to train a large\nNeural Network that maximizes the translation performance on a given parallel\ncorpus. NMT is then using a simple left-to-right beam-search decoder to\ngenerate new translations that approximately maximize the trained conditional\nprobability. The current beam search strategy generates the target sentence\nword by word from left-to- right while keeping a fixed amount of active\ncandidates at each time step. First, this simple search is less adaptive as it\nalso expands candidates whose scores are much worse than the current best.\nSecondly, it does not expand hypotheses if they are not within the best scoring\ncandidates, even if their scores are close to the best one. The latter one can\nbe avoided by increasing the beam size until no performance improvement can be\nobserved. While you can reach better performance, this has the draw- back of a\nslower decoding speed. In this paper, we concentrate on speeding up the decoder\nby applying a more flexible beam search strategy whose candidate size may vary\nat each time step depending on the candidate scores. We speed up the original\ndecoder by up to 43% for the two language pairs German-English and\nChinese-English without losing any translation quality.",
            "journal ref": "Proceedings of the First Workshop on Neural Machine Translation,\n  2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01815v2",
            "date": "2017-02-06 22:50:49+00:00",
            "title": "Living a discrete life in a continuous world: Reference with distributed representations",
            "authors": [
                "Gemma Boleda",
                "Sebastian Pad\u00f3",
                "Nghia The Pham",
                "Marco Baroni"
            ],
            "abstract": "Reference is a crucial property of language that allows us to connect\nlinguistic expressions to the world. Modeling it requires handling both\ncontinuous and discrete aspects of meaning. Data-driven models excel at the\nformer, but struggle with the latter, and the reverse is true for symbolic\nmodels.\n  This paper (a) introduces a concrete referential task to test both aspects,\ncalled cross-modal entity tracking; (b) proposes a neural network architecture\nthat uses external memory to build an entity library inspired in the DRSs of\nDRT, with a mechanism to dynamically introduce new referents or add information\nto referents that are already in the library.\n  Our model shows promise: it beats traditional neural network architectures on\nthe task. However, it is still outperformed by Memory Networks, another model\nwith external memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01841v3",
            "date": "2017-02-07 01:39:02+00:00",
            "title": "The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task",
            "authors": [
                "Roy Schwartz",
                "Maarten Sap",
                "Ioannis Konstas",
                "Li Zilles",
                "Yejin Choi",
                "Noah A. Smith"
            ],
            "abstract": "A writer's style depends not just on personal traits but also on her intent\nand mental state. In this paper, we show how variants of the same writing task\ncan lead to measurable differences in writing style. We present a case study\nbased on the story cloze task (Mostafazadeh et al., 2016a), where annotators\nwere assigned similar writing tasks with different constraints: (1) writing an\nentire story, (2) adding a story ending for a given story context, and (3)\nadding an incoherent ending to a story. We show that a simple linear classifier\ninformed by stylistic features is able to successfully distinguish among the\nthree cases, without even looking at the story context. In addition, combining\nour stylistic features with language model predictions reaches state of the art\nperformance on the story cloze challenge. Our results demonstrate that\ndifferent task framings can dramatically affect the way people write.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01923v1",
            "date": "2017-02-07 08:33:35+00:00",
            "title": "Comparative Study of CNN and RNN for Natural Language Processing",
            "authors": [
                "Wenpeng Yin",
                "Katharina Kann",
                "Mo Yu",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Deep neural networks (DNN) have revolutionized the field of natural language\nprocessing (NLP). Convolutional neural network (CNN) and recurrent neural\nnetwork (RNN), the two main types of DNN architectures, are widely explored to\nhandle various NLP tasks. CNN is supposed to be good at extracting\nposition-invariant features and RNN at modeling units in sequence. The state of\nthe art on many NLP tasks often switches due to the battle between CNNs and\nRNNs. This work is the first systematic comparison of CNN and RNN on a wide\nrange of representative NLP tasks, aiming to give basic guidance for DNN\nselection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01932v2",
            "date": "2017-02-07 09:16:46+00:00",
            "title": "A Knowledge-Grounded Neural Conversation Model",
            "authors": [
                "Marjan Ghazvininejad",
                "Chris Brockett",
                "Ming-Wei Chang",
                "Bill Dolan",
                "Jianfeng Gao",
                "Wen-tau Yih",
                "Michel Galley"
            ],
            "abstract": "Neural network models are capable of generating extremely natural sounding\nconversational interactions. Nevertheless, these models have yet to demonstrate\nthat they can incorporate content in the form of factual information or\nentity-grounded opinion that would enable them to serve in more task-oriented\nconversational applications. This paper presents a novel, fully data-driven,\nand knowledge-grounded neural conversation model aimed at producing more\ncontentful responses without slot filling. We generalize the widely-used\nSeq2Seq approach by conditioning responses on both conversation history and\nexternal \"facts\", allowing the model to be versatile and applicable in an\nopen-domain setting. Our approach yields significant improvements over a\ncompetitive Seq2Seq baseline. Human judges found that our outputs are\nsignificantly more informative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01944v1",
            "date": "2017-02-07 10:18:07+00:00",
            "title": "EliXa: A Modular and Flexible ABSA Platform",
            "authors": [
                "I\u00f1aki San Vicente",
                "Xabier Saralegi",
                "Rodrigo Agerri"
            ],
            "abstract": "This paper presents a supervised Aspect Based Sentiment Analysis (ABSA)\nsystem. Our aim is to develop a modular platform which allows to easily conduct\nexperiments by replacing the modules or adding new features. We obtain the best\nresult in the Opinion Target Extraction (OTE) task (slot 2) using an\noff-the-shelf sequence labeler. The target polarity classification (slot 3) is\naddressed by means of a multiclass SVM algorithm which includes lexical based\nfeatures such as the polarity values obtained from domain and open polarity\nlexicons. The system obtains accuracies of 0.70 and 0.73 for the restaurant and\nlaptop domain respectively, and performs second best in the out-of-domain\nhotel, achieving an accuracy of 0.80.",
            "journal ref": "Proceedings of the 9th International Workshop on Semantic\n  Evaluation (SemEval 2015). Association for Computational Linguistics, June\n  2015, Denver, Colorado, pp.748-752",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02098v3",
            "date": "2017-02-07 16:58:18+00:00",
            "title": "Fast and Accurate Entity Recognition with Iterated Dilated Convolutions",
            "authors": [
                "Emma Strubell",
                "Patrick Verga",
                "David Belanger",
                "Andrew McCallum"
            ],
            "abstract": "Today when many practitioners run basic NLP on the entire web and\nlarge-volume traffic, faster methods are paramount to saving time and energy\ncosts. Recent advances in GPU hardware have led to the emergence of\nbi-directional LSTMs as a standard method for obtaining per-token vector\nrepresentations serving as input to labeling tasks such as NER (often followed\nby prediction in a linear-chain CRF). Though expressive and accurate, these\nmodels fail to fully exploit GPU parallelism, limiting their computational\nefficiency. This paper proposes a faster alternative to Bi-LSTMs for NER:\nIterated Dilated Convolutional Neural Networks (ID-CNNs), which have better\ncapacity than traditional CNNs for large context and structured prediction.\nUnlike LSTMs whose sequential processing on sentences of length N requires O(N)\ntime even in the face of parallelism, ID-CNNs permit fixed-depth convolutions\nto run in parallel across entire documents. We describe a distinct combination\nof network structure, parameter sharing and training procedures that enable\ndramatic 14-20x test-time speedups while retaining accuracy comparable to the\nBi-LSTM-CRF. Moreover, ID-CNNs trained to aggregate context from the entire\ndocument are even more accurate while maintaining 8x faster test time speeds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02170v1",
            "date": "2017-02-07 19:21:50+00:00",
            "title": "How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks",
            "authors": [
                "Stanis\u0142aw Jastrzebski",
                "Damian Le\u015bniak",
                "Wojciech Marian Czarnecki"
            ],
            "abstract": "Maybe the single most important goal of representation learning is making\nsubsequent learning faster. Surprisingly, this fact is not well reflected in\nthe way embeddings are evaluated. In addition, recent practice in word\nembeddings points towards importance of learning specialized representations.\nWe argue that focus of word representation evaluation should reflect those\ntrends and shift towards evaluating what useful information is easily\naccessible. Specifically, we propose that evaluation should focus on data\nefficiency and simple supervised tasks, where the amount of available data is\nvaried and scores of a supervised model are reported for each subset (as\ncommonly done in transfer learning).\n  In order to illustrate significance of such analysis, a comprehensive\nevaluation of selected word embeddings is presented. Proposed approach yields a\nmore complete picture and brings new insight into performance characteristics,\nfor instance information about word similarity or analogy tends to be\nnon--linearly encoded in the embedding space, which questions the cosine-based,\nunsupervised, evaluation methods. All results and analysis scripts are\navailable online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02171v6",
            "date": "2017-02-07 19:22:06+00:00",
            "title": "Question Answering through Transfer Learning from Large Fine-grained Supervision Data",
            "authors": [
                "Sewon Min",
                "Minjoon Seo",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We show that the task of question answering (QA) can significantly benefit\nfrom the transfer learning of models trained on a different large, fine-grained\nQA dataset. We achieve the state of the art in two well-studied QA datasets,\nWikiQA and SemEval-2016 (Task 3A), through a basic transfer learning technique\nfrom SQuAD. For WikiQA, our model outperforms the previous best model by more\nthan 8%. We demonstrate that finer supervision provides better guidance for\nlearning lexical and syntactic information than coarser supervision, through\nquantitative results and visual analysis. We also show that a similar transfer\nlearning procedure achieves the state of the art on an entailment task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02211v2",
            "date": "2017-02-07 21:43:21+00:00",
            "title": "Fixing the Infix: Unsupervised Discovery of Root-and-Pattern Morphology",
            "authors": [
                "Tarek Sakakini",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "We present an unsupervised and language-agnostic method for learning\nroot-and-pattern morphology in Semitic languages. This form of morphology,\nabundant in Semitic languages, has not been handled in prior unsupervised\napproaches. We harness the syntactico-semantic information in distributed word\nrepresentations to solve the long standing problem of root-and-pattern\ndiscovery in Semitic languages. Moreover, we construct an unsupervised root\nextractor based on the learned rules. We prove the validity of learned rules\nacross Arabic, Hebrew, and Amharic, alongside showing that our root extractor\ncompares favorably with a widely used, carefully engineered root extractor:\nISRI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02212v3",
            "date": "2017-02-07 21:49:13+00:00",
            "title": "MORSE: Semantic-ally Drive-n MORpheme SEgment-er",
            "authors": [
                "Tarek Sakakini",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "We present in this paper a novel framework for morpheme segmentation which\nuses the morpho-syntactic regularities preserved by word representations, in\naddition to orthographic features, to segment words into morphemes. This\nframework is the first to consider vocabulary-wide syntactico-semantic\ninformation for this task. We also analyze the deficiencies of available\nbenchmarking datasets and introduce our own dataset that was created on the\nbasis of compositionality. We validate our algorithm across datasets and\npresent state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02261v1",
            "date": "2017-02-08 03:19:57+00:00",
            "title": "Social media mining for identification and exploration of health-related information from pregnant women",
            "authors": [
                "Pramod Bharadwaj Chandrashekar",
                "Arjun Magge",
                "Abeed Sarker",
                "Graciela Gonzalez"
            ],
            "abstract": "Widespread use of social media has led to the generation of substantial\namounts of information about individuals, including health-related information.\nSocial media provides the opportunity to study health-related information about\nselected population groups who may be of interest for a particular study. In\nthis paper, we explore the possibility of utilizing social media to perform\ntargeted data collection and analysis from a particular population group --\npregnant women. We hypothesize that we can use social media to identify cohorts\nof pregnant women and follow them over time to analyze crucial health-related\ninformation. To identify potentially pregnant women, we employ simple\nrule-based searches that attempt to detect pregnancy announcements with\nmoderate precision. To further filter out false positives and noise, we employ\na supervised classifier using a small number of hand-annotated data. We then\ncollect their posts over time to create longitudinal health timelines and\nattempt to divide the timelines into different pregnancy trimesters. Finally,\nwe assess the usefulness of the timelines by performing a preliminary analysis\nto estimate drug intake patterns of our cohort at different trimesters. Our\nrule-based cohort identification technique collected 53,820 users over thirty\nmonths from Twitter. Our pregnancy announcement classification technique\nachieved an F-measure of 0.81 for the pregnancy class, resulting in 34,895 user\ntimelines. Analysis of the timelines revealed that pertinent health-related\ninformation, such as drug-intake and adverse reactions can be mined from the\ndata. Our approach to using user timelines in this fashion has produced very\nencouraging results and can be employed for other important tasks where\ncohorts, for which health-related information may not be available from other\nsources, are required to be followed over time to derive population-based\nestimates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02265v4",
            "date": "2017-02-08 03:32:23+00:00",
            "title": "Neural Machine Translation with Source-Side Latent Graph Parsing",
            "authors": [
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "This paper presents a novel neural machine translation model which jointly\nlearns translation and source-side latent graph representations of sentences.\nUnlike existing pipelined approaches using syntactic parsers, our end-to-end\nmodel learns a latent graph parser as part of the encoder of an attention-based\nneural machine translation model, and thus the parser is optimized according to\nthe translation objective. In experiments, we first show that our model\ncompares favorably with state-of-the-art sequential and pipelined syntax-based\nNMT models. We also show that the performance of our model can be further\nimproved by pre-training it with a small amount of treebank annotations. Our\nfinal ensemble model significantly outperforms the previous best models on the\nstandard English-to-Japanese translation dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02363v2",
            "date": "2017-02-08 10:45:23+00:00",
            "title": "Automatically Annotated Turkish Corpus for Named Entity Recognition and Text Categorization using Large-Scale Gazetteers",
            "authors": [
                "H. Bahadir Sahin",
                "Caglar Tirkaz",
                "Eray Yildiz",
                "Mustafa Tolga Eren",
                "Ozan Sonmez"
            ],
            "abstract": "Turkish Wikipedia Named-Entity Recognition and Text Categorization (TWNERTC)\ndataset is a collection of automatically categorized and annotated sentences\nobtained from Wikipedia. We constructed large-scale gazetteers by using a graph\ncrawler algorithm to extract relevant entity and domain information from a\nsemantic knowledge base, Freebase. The constructed gazetteers contains\napproximately 300K entities with thousands of fine-grained entity types under\n77 different domains. Since automated processes are prone to ambiguity, we also\nintroduce two new content specific noise reduction methodologies. Moreover, we\nmap fine-grained entity types to the equivalent four coarse-grained types:\nperson, loc, org, misc. Eventually, we construct six different dataset versions\nand evaluate the quality of annotations by comparing ground truths from human\nannotators. We make these datasets publicly available to support studies on\nTurkish named-entity recognition (NER) and text categorization (TC).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02367v1",
            "date": "2017-02-08 10:58:02+00:00",
            "title": "Iterative Multi-document Neural Attention for Multiple Answer Prediction",
            "authors": [
                "Claudio Greco",
                "Alessandro Suglia",
                "Pierpaolo Basile",
                "Gaetano Rossiello",
                "Giovanni Semeraro"
            ],
            "abstract": "People have information needs of varying complexity, which can be solved by\nan intelligent agent able to answer questions formulated in a proper way,\neventually considering user context and preferences. In a scenario in which the\nuser profile can be considered as a question, intelligent agents able to answer\nquestions can be used to find the most relevant answers for a given user. In\nthis work we propose a novel model based on Artificial Neural Networks to\nanswer questions with multiple answers by exploiting multiple facts retrieved\nfrom a knowledge base. The model is evaluated on the factoid Question Answering\nand top-n recommendation tasks of the bAbI Movie Dialog dataset. After\nassessing the performance of the model on both tasks, we try to define the\nlong-term goal of a conversational recommender system able to interact using\nnatural language and to support users in their information seeking processes in\na personalized way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02390v1",
            "date": "2017-02-08 12:11:41+00:00",
            "title": "A Hybrid Convolutional Variational Autoencoder for Text Generation",
            "authors": [
                "Stanislau Semeniuta",
                "Aliaksei Severyn",
                "Erhardt Barth"
            ],
            "abstract": "In this paper we explore the effect of architectural choices on learning a\nVariational Autoencoder (VAE) for text generation. In contrast to the\npreviously introduced VAE model for text where both the encoder and decoder are\nRNNs, we propose a novel hybrid architecture that blends fully feed-forward\nconvolutional and deconvolutional components with a recurrent language model.\nOur architecture exhibits several attractive properties such as faster run time\nand convergence, ability to better handle long sequences and, more importantly,\nit helps to avoid some of the major difficulties posed by training VAE models\non textual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02535v3",
            "date": "2017-02-08 17:30:51+00:00",
            "title": "Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization",
            "authors": [
                "Ye Zhang",
                "Matthew Lease",
                "Byron C. Wallace"
            ],
            "abstract": "A fundamental advantage of neural models for NLP is their ability to learn\nrepresentations from scratch. However, in practice this often means ignoring\nexisting external linguistic resources, e.g., WordNet or domain specific\nontologies such as the Unified Medical Language System (UMLS). We propose a\ngeneral, novel method for exploiting such resources via weight sharing. Prior\nwork on weight sharing in neural networks has considered it largely as a means\nof model compression. In contrast, we treat weight sharing as a flexible\nmechanism for incorporating prior knowledge into neural models. We show that\nthis approach consistently yields improved performance on classification tasks\ncompared to baseline strategies that do not exploit weight sharing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02584v2",
            "date": "2017-02-08 19:10:53+00:00",
            "title": "Predicting Audience's Laughter Using Convolutional Neural Network",
            "authors": [
                "Lei Chen",
                "Chong MIn Lee"
            ],
            "abstract": "For the purpose of automatically evaluating speakers' humor usage, we build a\npresentation corpus containing humorous utterances based on TED talks. Compared\nto previous data resources supporting humor recognition research, ours has\nseveral advantages, including (a) both positive and negative instances coming\nfrom a homogeneous data set, (b) containing a large number of speakers, and (c)\nbeing open. Focusing on using lexical cues for humor recognition, we\nsystematically compare a newly emerging text classification method based on\nConvolutional Neural Networks (CNNs) with a well-established conventional\nmethod using linguistic knowledge. The advantages of the CNN method are both\ngetting higher detection accuracies and being able to learn essential features\nautomatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03033v1",
            "date": "2017-02-10 01:27:00+00:00",
            "title": "Local System Voting Feature for Machine Translation System Combination",
            "authors": [
                "Markus Freitag",
                "Jan-Thorsten Peter",
                "Stephan Peitz",
                "Minwei Feng",
                "Hermann Ney"
            ],
            "abstract": "In this paper, we enhance the traditional confusion network system\ncombination approach with an additional model trained by a neural network. This\nwork is motivated by the fact that the commonly used binary system voting\nmodels only assign each input system a global weight which is responsible for\nthe global impact of each input system on all translations. This prevents\nindividual systems with low system weights from having influence on the system\ncombination output, although in some situations this could be helpful. Further,\nwords which have only been seen by one or few systems rarely have a chance of\nbeing present in the combined output. We train a local system voting model by a\nneural network which is based on the words themselves and the combinatorial\noccurrences of the different system outputs. This gives system combination the\noption to prefer other systems at different word positions even for the same\nsentence.",
            "journal ref": "Proceedings of the Tenth Workshop on Statistical Machine\n  Translation (WMT), 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03082v1",
            "date": "2017-02-10 07:22:08+00:00",
            "title": "UsingWord Embedding for Cross-Language Plagiarism Detection",
            "authors": [
                "J. Ferrero",
                "F. Agnes",
                "L. Besacier",
                "D. Schwab"
            ],
            "abstract": "This paper proposes to use distributed representation of words (word\nembeddings) in cross-language textual similarity detection. The main\ncontributions of this paper are the following: (a) we introduce new\ncross-language similarity detection methods based on distributed representation\nof words; (b) we combine the different methods proposed to verify their\ncomplementarity and finally obtain an overall F1 score of 89.15% for\nEnglish-French similarity detection at chunk level (88.5% at sentence level) on\na very challenging corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03196v4",
            "date": "2017-02-10 14:57:26+00:00",
            "title": "Universal Semantic Parsing",
            "authors": [
                "Siva Reddy",
                "Oscar T\u00e4ckstr\u00f6m",
                "Slav Petrov",
                "Mark Steedman",
                "Mirella Lapata"
            ],
            "abstract": "Universal Dependencies (UD) offer a uniform cross-lingual syntactic\nrepresentation, with the aim of advancing multilingual applications. Recent\nwork shows that semantic parsing can be accomplished by transforming syntactic\ndependencies to logical forms. However, this work is limited to English, and\ncannot process dependency graphs, which allow handling complex phenomena such\nas control. In this work, we introduce UDepLambda, a semantic interface for UD,\nwhich maps natural language to logical forms in an almost language-independent\nfashion and can process dependency graphs. We perform experiments on question\nanswering against Freebase and provide German and Spanish translations of the\nWebQuestions and GraphQuestions datasets to facilitate multilingual evaluation.\nResults show that UDepLambda outperforms strong baselines across languages and\ndatasets. For English, it achieves a 4.9 F1 point improvement over the\nstate-of-the-art on GraphQuestions. Our code and data can be downloaded at\nhttps://github.com/sivareddyg/udeplambda.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03305v1",
            "date": "2017-02-10 19:16:40+00:00",
            "title": "Universal Dependencies to Logical Forms with Negation Scope",
            "authors": [
                "Federico Fancellu",
                "Siva Reddy",
                "Adam Lopez",
                "Bonnie Webber"
            ],
            "abstract": "Many language technology applications would benefit from the ability to\nrepresent negation and its scope on top of widely-used linguistic resources. In\nthis paper, we investigate the possibility of obtaining a first-order logic\nrepresentation with negation scope marked using Universal Dependencies. To do\nso, we enhance UDepLambda, a framework that converts dependency graphs to\nlogical forms. The resulting UDepLambda$\\lnot$ is able to handle phenomena\nrelated to scope by means of an higher-order type theory, relevant not only to\nnegation but also to universal quantification and other complex semantic\nphenomena. The initial conversion we did for English is promising, in that one\ncan represent the scope of negation also in the presence of more complex\nphenomena such as universal quantifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03342v2",
            "date": "2017-02-10 22:44:59+00:00",
            "title": "Learning Concept Embeddings for Efficient Bag-of-Concepts Densification",
            "authors": [
                "Walid Shalaby",
                "Wlodek Zadrozny"
            ],
            "abstract": "Explicit concept space models have proven efficacy for text representation in\nmany natural language and text mining applications. The idea is to embed\ntextual structures into a semantic space of concepts which captures the main\nideas, objects, and the characteristics of these structures. The so called Bag\nof Concepts (BoC) representation suffers from data sparsity causing low\nsimilarity scores between similar texts due to low concept overlap. To address\nthis problem, we propose two neural embedding models to learn continuous\nconcept vectors. Once they are learned, we propose an efficient vector\naggregation method to generate fully continuous BoC representations. We\nevaluate our concept embedding models on three tasks: 1) measuring entity\nsemantic relatedness and ranking where we achieve 1.6% improvement in\ncorrelation scores, 2) dataless concept categorization where we achieve\nstate-of-the-art performance and reduce the categorization error rate by more\nthan 5% compared to five prior word and entity embedding models, and 3)\ndataless document classification where our models outperform the sparse BoC\nrepresentations. In addition, by exploiting our efficient linear time vector\naggregation method, we achieve better accuracy scores with much less concept\ndimensions compared to previous BoC densification methods which operate in\npolynomial time and require hundreds of dimensions in the BoC representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03470v1",
            "date": "2017-02-12 00:23:04+00:00",
            "title": "Vector Embedding of Wikipedia Concepts and Entities",
            "authors": [
                "Ehsan Sherkat",
                "Evangelos Milios"
            ],
            "abstract": "Using deep learning for different machine learning tasks such as image\nclassification and word embedding has recently gained many attentions. Its\nappealing performance reported across specific Natural Language Processing\n(NLP) tasks in comparison with other approaches is the reason for its\npopularity. Word embedding is the task of mapping words or phrases to a low\ndimensional numerical vector. In this paper, we use deep learning to embed\nWikipedia Concepts and Entities. The English version of Wikipedia contains more\nthan five million pages, which suggest its capability to cover many English\nEntities, Phrases, and Concepts. Each Wikipedia page is considered as a\nconcept. Some concepts correspond to entities, such as a person's name, an\norganization or a place. Contrary to word embedding, Wikipedia Concepts\nEmbedding is not ambiguous, so there are different vectors for concepts with\nsimilar surface form but different mentions. We proposed several approaches and\nevaluated their performance based on Concept Analogy and Concept Similarity\ntasks. The results show that proposed approaches have the performance\ncomparable and in some cases even higher than the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03525v2",
            "date": "2017-02-12 13:19:03+00:00",
            "title": "Learning to Parse and Translate Improves Neural Machine Translation",
            "authors": [
                "Akiko Eriguchi",
                "Yoshimasa Tsuruoka",
                "Kyunghyun Cho"
            ],
            "abstract": "There has been relatively little attention to incorporating linguistic prior\nto neural machine translation. Much of the previous work was further\nconstrained to considering linguistic prior on the source side. In this paper,\nwe propose a hybrid model, called NMT+RNNG, that learns to parse and translate\nby combining the recurrent neural network grammar into the attention-based\nneural machine translation. Our approach encourages the neural machine\ntranslation model to incorporate linguistic prior during training, and lets it\ntranslate on its own afterward. Extensive experiments with four language pairs\nshow the effectiveness of the proposed NMT+RNNG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03654v1",
            "date": "2017-02-13 07:08:28+00:00",
            "title": "A Morphology-aware Network for Morphological Disambiguation",
            "authors": [
                "Eray Yildiz",
                "Caglar Tirkaz",
                "H. Bahadir Sahin",
                "Mustafa Tolga Eren",
                "Ozan Sonmez"
            ],
            "abstract": "Agglutinative languages such as Turkish, Finnish and Hungarian require\nmorphological disambiguation before further processing due to the complex\nmorphology of words. A morphological disambiguator is used to select the\ncorrect morphological analysis of a word. Morphological disambiguation is\nimportant because it generally is one of the first steps of natural language\nprocessing and its performance affects subsequent analyses. In this paper, we\npropose a system that uses deep learning techniques for morphological\ndisambiguation. Many of the state-of-the-art results in computer vision, speech\nrecognition and natural language processing have been obtained through deep\nlearning models. However, applying deep learning techniques to morphologically\nrich languages is not well studied. In this work, while we focus on Turkish\nmorphological disambiguation we also present results for French and German in\norder to show that the proposed architecture achieves high accuracy with no\nlanguage-specific feature engineering or additional resource. In the\nexperiments, we achieve 84.12, 88.35 and 93.78 morphological disambiguation\naccuracy among the ambiguous words for Turkish, German and French respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03706v1",
            "date": "2017-02-13 10:24:55+00:00",
            "title": "Multitask Learning with Deep Neural Networks for Community Question Answering",
            "authors": [
                "Daniele Bonadiman",
                "Antonio Uva",
                "Alessandro Moschitti"
            ],
            "abstract": "In this paper, we developed a deep neural network (DNN) that learns to solve\nsimultaneously the three tasks of the cQA challenge proposed by the\nSemEval-2016 Task 3, i.e., question-comment similarity, question-question\nsimilarity and new question-comment similarity. The latter is the main task,\nwhich can exploit the previous two for achieving better results. Our DNN is\ntrained jointly on all the three cQA tasks and learns to encode questions and\ncomments into a single vector representation shared across the multiple tasks.\nThe results on the official challenge test set show that our approach produces\nhigher accuracy and faster convergence rates than the individual neural\nnetworks. Additionally, our method, which does not use any manual feature\nengineering, approaches the state of the art established with methods that make\nheavy use of it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03856v1",
            "date": "2017-02-13 16:30:23+00:00",
            "title": "Towards speech-to-text translation without speech recognition",
            "authors": [
                "Sameer Bansal",
                "Herman Kamper",
                "Adam Lopez",
                "Sharon Goldwater"
            ],
            "abstract": "We explore the problem of translating speech to text in low-resource\nscenarios where neither automatic speech recognition (ASR) nor machine\ntranslation (MT) are available, but we have training data in the form of audio\npaired with text translations. We present the first system for this problem\napplied to a realistic multi-speaker dataset, the CALLHOME Spanish-English\nspeech translation corpus. Our approach uses unsupervised term discovery (UTD)\nto cluster repeated patterns in the audio, creating a pseudotext, which we pair\nwith translations to create a parallel text and train a simple bag-of-words MT\nmodel. We identify the challenges faced by the system, finding that the\ndifficulty of cross-speaker UTD results in low recall, but that our system is\nstill able to correctly translate some content words in test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03964v1",
            "date": "2017-02-13 19:52:02+00:00",
            "title": "The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations",
            "authors": [
                "Lasha Abzianidze",
                "Johannes Bjerva",
                "Kilian Evang",
                "Hessel Haagsma",
                "Rik van Noord",
                "Pierre Ludmann",
                "Duc-Duy Nguyen",
                "Johan Bos"
            ],
            "abstract": "The Parallel Meaning Bank is a corpus of translations annotated with shared,\nformal meaning representations comprising over 11 million words divided over\nfour languages (English, German, Italian, and Dutch). Our approach is based on\ncross-lingual projection: automatically produced (and manually corrected)\nsemantic annotations for English sentences are mapped onto their word-aligned\ntranslations, assuming that the translations are meaning-preserving. The\nsemantic annotation consists of five main steps: (i) segmentation of the text\nin sentences and lexical items; (ii) syntactic parsing with Combinatory\nCategorial Grammar; (iii) universal semantic tagging; (iv) symbolization; and\n(v) compositional semantic analysis based on Discourse Representation Theory.\nThese steps are performed using statistical models trained in a semi-supervised\nmanner. The employed annotation models are all language-neutral. Our first\nresults are promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04066v1",
            "date": "2017-02-14 03:47:34+00:00",
            "title": "JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction",
            "authors": [
                "Courtney Napoles",
                "Keisuke Sakaguchi",
                "Joel Tetreault"
            ],
            "abstract": "We present a new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG) for\ndeveloping and evaluating grammatical error correction (GEC). Unlike other\ncorpora, it represents a broad range of language proficiency levels and uses\nholistic fluency edits to not only correct grammatical errors but also make the\noriginal text more native sounding. We describe the types of corrections made\nand benchmark four leading GEC systems on this corpus, identifying specific\nareas in which they do well and how they can improve. JFLEG fulfills the need\nfor a new gold standard to properly assess the current state of GEC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04333v1",
            "date": "2017-02-14 18:46:47+00:00",
            "title": "On the Relevance of Auditory-Based Gabor Features for Deep Learning in Automatic Speech Recognition",
            "authors": [
                "Angel Mario Castro Martinez",
                "Sri Harish Mallidi",
                "Bernd T. Meyer"
            ],
            "abstract": "Previous studies support the idea of merging auditory-based Gabor features\nwith deep learning architectures to achieve robust automatic speech\nrecognition, however, the cause behind the gain of such combination is still\nunknown. We believe these representations provide the deep learning decoder\nwith more discriminable cues. Our aim with this paper is to validate this\nhypothesis by performing experiments with three different recognition tasks\n(Aurora 4, CHiME 2 and CHiME 3) and assess the discriminability of the\ninformation encoded by Gabor filterbank features. Additionally, to identify the\ncontribution of low, medium and high temporal modulation frequencies subsets of\nthe Gabor filterbank were used as features (dubbed LTM, MTM and HTM\nrespectively). With temporal modulation frequencies between 16 and 25 Hz, HTM\nconsistently outperformed the remaining ones in every condition, highlighting\nthe robustness of these representations against channel distortions, low\nsignal-to-noise ratios and acoustically challenging real-life scenarios with\nrelative improvements from 11 to 56% against a Mel-filterbank-DNN baseline. To\nexplain the results, a measure of similarity between phoneme classes from DNN\nactivations is proposed and linked to their acoustic properties. We find this\nmeasure to be consistent with the observed error rates and highlight specific\ndifferences on phoneme level to pinpoint the benefit of the proposed features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04372v1",
            "date": "2017-02-14 20:06:15+00:00",
            "title": "A case study on using speech-to-translation alignments for language documentation",
            "authors": [
                "Antonios Anastasopoulos",
                "David Chiang"
            ],
            "abstract": "For many low-resource or endangered languages, spoken language resources are\nmore likely to be annotated with translations than with transcriptions. Recent\nwork exploits such annotations to produce speech-to-translation alignments,\nwithout access to any text transcriptions. We investigate whether providing\nsuch information can aid in producing better (mismatched) crowdsourced\ntranscriptions, which in turn could be valuable for training speech recognition\nsystems, and show that they can indeed be beneficial through a small-scale case\nstudy as a proof-of-concept. We also present a simple phonetically aware string\naveraging technique that produces transcriptions of higher quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04457v2",
            "date": "2017-02-15 03:35:03+00:00",
            "title": "Automated Phrase Mining from Massive Text Corpora",
            "authors": [
                "Jingbo Shang",
                "Jialu Liu",
                "Meng Jiang",
                "Xiang Ren",
                "Clare R Voss",
                "Jiawei Han"
            ],
            "abstract": "As one of the fundamental tasks in text analysis, phrase mining aims at\nextracting quality phrases from a text corpus. Phrase mining is important in\nvarious tasks such as information extraction/retrieval, taxonomy construction,\nand topic modeling. Most existing methods rely on complex, trained linguistic\nanalyzers, and thus likely have unsatisfactory performance on text corpora of\nnew domains and genres without extra but expensive adaption. Recently, a few\ndata-driven methods have been developed successfully for extraction of phrases\nfrom massive domain-specific text. However, none of the state-of-the-art models\nis fully automated because they require human experts for designing rules or\nlabeling phrases.\n  Since one can easily obtain many quality phrases from public knowledge bases\nto a scale that is much larger than that produced by human experts, in this\npaper, we propose a novel framework for automated phrase mining, AutoPhrase,\nwhich leverages this large amount of high-quality phrases in an effective way\nand achieves better performance compared to limited human labeled phrases. In\naddition, we develop a POS-guided phrasal segmentation model, which\nincorporates the shallow syntactic information in part-of-speech (POS) tags to\nfurther enhance the performance, when a POS tagger is available. Note that,\nAutoPhrase can support any language as long as a general knowledge base (e.g.,\nWikipedia) in that language is available, while benefiting from, but not\nrequiring, a POS tagger. Compared to the state-of-the-art methods, the new\nmethod has shown significant improvements in effectiveness on five real-world\ndatasets across different domains and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04488v5",
            "date": "2017-02-15 07:37:55+00:00",
            "title": "Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network",
            "authors": [
                "Jingjing Xu",
                "Xu Sun"
            ],
            "abstract": "Recent studies have shown effectiveness in using neural networks for Chinese\nword segmentation. However, these models rely on large-scale data and are less\neffective for low-resource datasets because of insufficient training data. We\npropose a transfer learning method to improve low-resource word segmentation by\nleveraging high-resource corpora. First, we train a teacher model on\nhigh-resource corpora and then use the learned knowledge to initialize a\nstudent model. Second, a weighted data similarity method is proposed to train\nthe student model on low-resource data. Experiment results show that our work\nsignificantly improves the performance on low-resource datasets: 2.3% and 1.5%\nF-score on PKU and CTB datasets. Furthermore, this paper achieves\nstate-of-the-art results: 96.1%, and 96.2% F-score on PKU and CTB datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04510v1",
            "date": "2017-02-15 09:08:21+00:00",
            "title": "A Dependency-Based Neural Reordering Model for Statistical Machine Translation",
            "authors": [
                "Christian Hadiwinoto",
                "Hwee Tou Ng"
            ],
            "abstract": "In machine translation (MT) that involves translating between two languages\nwith significant differences in word order, determining the correct word order\nof translated words is a major challenge. The dependency parse tree of a source\nsentence can help to determine the correct word order of the translated words.\nIn this paper, we present a novel reordering approach utilizing a neural\nnetwork and dependency-based embeddings to predict whether the translations of\ntwo source words linked by a dependency relation should remain in the same\norder or should be swapped in the translated sentence. Experiments on\nChinese-to-English translation show that our approach yields a statistically\nsignificant improvement of 0.57 BLEU point on benchmark NIST test sets,\ncompared to our prior state-of-the-art statistical MT system that uses sparse\ndependency-based reordering features.",
            "journal ref": "Proceedings of AAAI-17 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04615v1",
            "date": "2017-02-11 02:21:42+00:00",
            "title": "Automated Identification of Drug-Drug Interactions in Pediatric Congestive Heart Failure Patients",
            "authors": [
                "Daniel Miller"
            ],
            "abstract": "Congestive Heart Failure, or CHF, is a serious medical condition that can\nresult in fluid buildup in the body as a result of a weak heart. When the heart\ncan't pump enough blood to efficiently deliver nutrients and oxygen to the\nbody, kidney function may be impaired, resulting in fluid retention. CHF\npatients require a broad drug regimen to maintain the delicate system balance,\nparticularly between their heart and kidneys. These drugs include ACE\ninhibitors and Beta Blockers to control blood pressure, anticoagulants to\nprevent blood clots, and diuretics to reduce fluid overload. Many of these\ndrugs may interact, and potential effects of these interactions must be weighed\nagainst their benefits. For this project, we consider a set of 44 drugs\nidentified as specifically relevant for treating CHF by pediatric cardiologists\nat Lucile Packard Children's Hospital. This list was generated as part of our\ncurrent work at the LPCH Heart Center. The goal of this project is to identify\nand evaluate potentially harmful drug-drug interactions (DDIs) within pediatric\npatients with Congestive Heart Failure. This identification will be done\nautonomously, so that it may continuously update by evaluating newly published\nliterature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04811v3",
            "date": "2017-02-15 23:04:09+00:00",
            "title": "Understanding Deep Learning Performance through an Examination of Test Set Difficulty: A Psychometric Case Study",
            "authors": [
                "John P. Lalor",
                "Hao Wu",
                "Tsendsuren Munkhdalai",
                "Hong Yu"
            ],
            "abstract": "Interpreting the performance of deep learning models beyond test set accuracy\nis challenging. Characteristics of individual data points are often not\nconsidered during evaluation, and each data point is treated equally. We\nexamine the impact of a test set question's difficulty to determine if there is\na relationship between difficulty and performance. We model difficulty using\nwell-studied psychometric methods on human response patterns. Experiments on\nNatural Language Inference (NLI) and Sentiment Analysis (SA) show that the\nlikelihood of answering a question correctly is impacted by the question's\ndifficulty. As DNNs are trained with more data, easy examples are learned more\nquickly than hard examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04938v1",
            "date": "2017-02-16 12:10:18+00:00",
            "title": "Fast and unsupervised methods for multilingual cognate clustering",
            "authors": [
                "Taraka Rama",
                "Johannes Wahle",
                "Pavel Sofroniev",
                "Gerhard J\u00e4ger"
            ],
            "abstract": "In this paper we explore the use of unsupervised methods for detecting\ncognates in multilingual word lists. We use online EM to train sound segment\nsimilarity weights for computing similarity between two words. We tested our\nonline systems on geographically spread sixteen different language groups of\nthe world and show that the Online PMI system (Pointwise Mutual Information)\noutperforms a HMM based system and two linguistically motivated systems:\nLexStat and ALINE. Our results suggest that a PMI system trained in an online\nfashion can be used by historical linguists for fast and accurate\nidentification of cognates in not so well-studied language families.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05053v1",
            "date": "2017-02-16 17:09:12+00:00",
            "title": "Addressing the Data Sparsity Issue in Neural AMR Parsing",
            "authors": [
                "Xiaochang Peng",
                "Chuan Wang",
                "Daniel Gildea",
                "Nianwen Xue"
            ],
            "abstract": "Neural attention models have achieved great success in different NLP tasks.\nHow- ever, they have not fulfilled their promise on the AMR parsing task due to\nthe data sparsity issue. In this paper, we de- scribe a sequence-to-sequence\nmodel for AMR parsing and present different ways to tackle the data sparsity\nproblem. We show that our methods achieve significant improvement over a\nbaseline neural atten- tion model and our results are also compet- itive\nagainst state-of-the-art systems that do not use extra linguistic resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05398v1",
            "date": "2017-02-17 15:39:21+00:00",
            "title": "Experiment Segmentation in Scientific Discourse as Clause-level Structured Prediction using Recurrent Neural Networks",
            "authors": [
                "Pradeep Dasigi",
                "Gully A. P. C. Burns",
                "Eduard Hovy",
                "Anita de Waard"
            ],
            "abstract": "We propose a deep learning model for identifying structure within experiment\nnarratives in scientific literature. We take a sequence labeling approach to\nthis problem, and label clauses within experiment narratives to identify the\ndifferent parts of the experiment. Our dataset consists of paragraphs taken\nfrom open access PubMed papers labeled with rhetorical information as a result\nof our pilot annotation. Our model is a Recurrent Neural Network (RNN) with\nLong Short-Term Memory (LSTM) cells that labels clauses. The clause\nrepresentations are computed by combining word representations using a novel\nattention mechanism that involves a separate RNN. We compare this model against\nLSTMs where the input layer has simple or no attention and a feature rich CRF\nmodel. Furthermore, we describe how our work could be useful for information\nextraction from scientific literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05531v1",
            "date": "2017-02-17 22:10:28+00:00",
            "title": "Analysis and Optimization of fastText Linear Text Classifier",
            "authors": [
                "Vladimir Zolotov",
                "David Kung"
            ],
            "abstract": "The paper [1] shows that simple linear classifier can compete with complex\ndeep learning algorithms in text classification applications. Combining bag of\nwords (BoW) and linear classification techniques, fastText [1] attains same or\nonly slightly lower accuracy than deep learning algorithms [2-9] that are\norders of magnitude slower. We proved formally that fastText can be transformed\ninto a simpler equivalent classifier, which unlike fastText does not have any\nhidden layer. We also proved that the necessary and sufficient dimensionality\nof the word vector embedding space is exactly the number of document classes.\nThese results help constructing more optimal linear text classifiers with\nguaranteed maximum classification capabilities. The results are proven exactly\nby pure formal algebraic methods without attracting any empirical data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05624v1",
            "date": "2017-02-18 15:29:01+00:00",
            "title": "Reproducing and learning new algebraic operations on word embeddings using genetic programming",
            "authors": [
                "Roberto Santana"
            ],
            "abstract": "Word-vector representations associate a high dimensional real-vector to every\nword from a corpus. Recently, neural-network based methods have been proposed\nfor learning this representation from large corpora. This type of\nword-to-vector embedding is able to keep, in the learned vector space, some of\nthe syntactic and semantic relationships present in the original word corpus.\nThis, in turn, serves to address different types of language classification\ntasks by doing algebraic operations defined on the vectors. The general\npractice is to assume that the semantic relationships between the words can be\ninferred by the application of a-priori specified algebraic operations. Our\ngeneral goal in this paper is to show that it is possible to learn methods for\nword composition in semantic spaces. Instead of expressing the compositional\nmethod as an algebraic operation, we will encode it as a program, which can be\nlinear, nonlinear, or involve more intricate expressions. More remarkably, this\nprogram will be evolved from a set of initial random programs by means of\ngenetic programming (GP). We show that our method is able to reproduce the same\nbehavior as human-designed algebraic operators. Using a word analogy task as\nbenchmark, we also show that GP-generated programs are able to obtain accuracy\nvalues above those produced by the commonly used human-designed rule for\nalgebraic manipulation of word vectors. Finally, we show the robustness of our\napproach by executing the evolved programs on the word2vec GoogleNews vectors,\nlearned over 3 billion running words, and assessing their accuracy in the same\nword analogy task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05638v1",
            "date": "2017-02-18 18:10:04+00:00",
            "title": "A Stylometric Inquiry into Hyperpartisan and Fake News",
            "authors": [
                "Martin Potthast",
                "Johannes Kiesel",
                "Kevin Reinartz",
                "Janek Bevendorff",
                "Benno Stein"
            ],
            "abstract": "This paper reports on a writing style analysis of hyperpartisan (i.e.,\nextremely one-sided) news in connection to fake news. It presents a large\ncorpus of 1,627 articles that were manually fact-checked by professional\njournalists from BuzzFeed. The articles originated from 9 well-known political\npublishers, 3 each from the mainstream, the hyperpartisan left-wing, and the\nhyperpartisan right-wing. In sum, the corpus contains 299 fake news, 97% of\nwhich originated from hyperpartisan publishers.\n  We propose and demonstrate a new way of assessing style similarity between\ntext categories via Unmasking---a meta-learning approach originally devised for\nauthorship verification---, revealing that the style of left-wing and\nright-wing news have a lot more in common than any of the two have with the\nmainstream. Furthermore, we show that hyperpartisan news can be discriminated\nwell by its style from the mainstream (F1=0.78), as can be satire from both\n(F1=0.81). Unsurprisingly, style-based fake news detection does not live up to\nscratch (F1=0.46). Nevertheless, the former results are important to implement\npre-screening for fake news detectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05793v1",
            "date": "2017-02-19 20:37:40+00:00",
            "title": "Harmonic Grammar, Optimality Theory, and Syntax Learnability: An Empirical Exploration of Czech Word Order",
            "authors": [
                "Ann Irvine",
                "Mark Dredze"
            ],
            "abstract": "This work presents a systematic theoretical and empirical comparison of the\nmajor algorithms that have been proposed for learning Harmonic and Optimality\nTheory grammars (HG and OT, respectively). By comparing learning algorithms, we\nare also able to compare the closely related OT and HG frameworks themselves.\nExperimental results show that the additional expressivity of the HG framework\nover OT affords performance gains in the task of predicting the surface word\norder of Czech sentences. We compare the perceptron with the classic Gradual\nLearning Algorithm (GLA), which learns OT grammars, as well as the popular\nMaximum Entropy model. In addition to showing that the perceptron is\ntheoretically appealing, our work shows that the performance of the HG model it\nlearns approaches that of the upper bound in prediction accuracy on a held out\ntest set and that it is capable of accurately modeling observed variation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05821v1",
            "date": "2017-02-20 00:23:06+00:00",
            "title": "Post-edit Analysis of Collective Biography Generation",
            "authors": [
                "Bo Han",
                "Will Radford",
                "Ana\u00efs Cadilhac",
                "Art Harol",
                "Andrew Chisholm",
                "Ben Hachey"
            ],
            "abstract": "Text generation is increasingly common but often requires manual post-editing\nwhere high precision is critical to end users. However, manual editing is\nexpensive so we want to ensure this effort is focused on high-value tasks. And\nwe want to maintain stylistic consistency, a particular challenge in crowd\nsettings. We present a case study, analysing human post-editing in the context\nof a template-based biography generation system. An edit flow visualisation\ncombined with manual characterisation of edits helps identify and prioritise\nwork for improving end-to-end efficiency and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05962v1",
            "date": "2017-02-20 13:36:23+00:00",
            "title": "Latent Variable Dialogue Models and their Diversity",
            "authors": [
                "Kris Cao",
                "Stephen Clark"
            ],
            "abstract": "We present a dialogue generation model that directly captures the variability\nin possible responses to a given input, which reduces the `boring output' issue\nof deterministic dialogue models. Experiments show that our model generates\nmore diverse outputs than baseline models, and also generates more consistently\nacceptable output than sampling from a deterministic encoder-decoder model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06027v2",
            "date": "2017-02-20 15:53:56+00:00",
            "title": "Parent Oriented Teacher Selection Causes Language Diversity",
            "authors": [
                "Ibrahim Cimentepe",
                "Haluk O. Bingol"
            ],
            "abstract": "An evolutionary model for emergence of diversity in language is developed. We\ninvestigated the effects of two real life observations, namely, people prefer\npeople that they communicate with well, and people interact with people that\nare physically close to each other. Clearly these groups are relatively small\ncompared to the entire population. We restrict selection of the teachers from\nsuch small groups, called imitation sets, around parents. Then the child learns\nlanguage from a teacher selected within the imitation set of her parent. As a\nresult, there are subcommunities with their own languages developed. Within\nsubcommunity comprehension is found to be high. The number of languages is\nrelated to the relative size of imitation set by a power law.",
            "journal ref": "Journal of Theoretical Biology 429, 142-148, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06135v4",
            "date": "2017-02-20 19:00:06+00:00",
            "title": "Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages",
            "authors": [
                "Raj Dabre",
                "Fabien Cromieres",
                "Sadao Kurohashi"
            ],
            "abstract": "In this paper, we explore a simple solution to \"Multi-Source Neural Machine\nTranslation\" (MSNMT) which only relies on preprocessing a N-way multilingual\ncorpus without modifying the Neural Machine Translation (NMT) architecture or\ntraining procedure. We simply concatenate the source sentences to form a single\nlong multi-source input sentence while keeping the target side sentence as it\nis and train an NMT system using this preprocessed corpus. We evaluate our\nmethod in resource poor as well as resource rich settings and show its\neffectiveness (up to 4 BLEU using 2 source languages and up to 6 BLEU using 5\nsource languages). We also compare against existing methods for MSNMT and show\nthat our solution gives competitive results despite its simplicity. We also\nprovide some insights on how the NMT system leverages multilingual information\nin such a scenario by visualizing attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06235v1",
            "date": "2017-02-21 01:30:59+00:00",
            "title": "Learning to generate one-sentence biographies from Wikidata",
            "authors": [
                "Andrew Chisholm",
                "Will Radford",
                "Ben Hachey"
            ],
            "abstract": "We investigate the generation of one-sentence Wikipedia biographies from\nfacts derived from Wikidata slot-value pairs. We train a recurrent neural\nnetwork sequence-to-sequence model with attention to select facts and generate\ntextual summaries. Our model incorporates a novel secondary objective that\nhelps ensure it generates sentences that contain the input facts. The model\nachieves a BLEU score of 41, improving significantly upon the vanilla\nsequence-to-sequence model and scoring roughly twice that of a simple template\nbaseline. Human preference evaluation suggests the model is nearly as good as\nthe Wikipedia reference. Manual analysis explores content selection, suggesting\nthe model can trade the ability to infer knowledge against the risk of\nhallucinating incorrect information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06239v1",
            "date": "2017-02-21 02:18:38+00:00",
            "title": "Reinforcement Learning Based Argument Component Detection",
            "authors": [
                "Yang Gao",
                "Hao Wang",
                "Chen Zhang",
                "Wei Wang"
            ],
            "abstract": "Argument component detection (ACD) is an important sub-task in argumentation\nmining. ACD aims at detecting and classifying different argument components in\nnatural language texts. Historical annotations (HAs) are important features the\nhuman annotators consider when they manually perform the ACD task. However, HAs\nare largely ignored by existing automatic ACD techniques. Reinforcement\nlearning (RL) has proven to be an effective method for using HAs in some\nnatural language processing tasks. In this work, we propose a RL-based ACD\ntechnique, and evaluate its performance on two well-annotated corpora. Results\nsuggest that, in terms of classification accuracy, HAs-augmented RL outperforms\nplain RL by at most 17.85%, and outperforms the state-of-the-art supervised\nlearning algorithm by at most 11.94%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06336v1",
            "date": "2017-02-21 11:34:14+00:00",
            "title": "Hybrid Dialog State Tracker with ASR Features",
            "authors": [
                "Miroslav Vodol\u00e1n",
                "Rudolf Kadlec",
                "Jan Kleindienst"
            ],
            "abstract": "This paper presents a hybrid dialog state tracker enhanced by trainable\nSpoken Language Understanding (SLU) for slot-filling dialog systems. Our\narchitecture is inspired by previously proposed neural-network-based\nbelief-tracking systems. In addition, we extended some parts of our modular\narchitecture with differentiable rules to allow end-to-end training. We\nhypothesize that these rules allow our tracker to generalize better than pure\nmachine-learning based systems. For evaluation, we used the Dialog State\nTracking Challenge (DSTC) 2 dataset - a popular belief tracking testbed with\ndialogs from restaurant information system. To our knowledge, our hybrid\ntracker sets a new state-of-the-art result in three out of four categories\nwithin the DSTC2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06378v4",
            "date": "2017-02-21 13:39:35+00:00",
            "title": "Multitask Learning with CTC and Segmental CRF for Speech Recognition",
            "authors": [
                "Liang Lu",
                "Lingpeng Kong",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "Segmental conditional random fields (SCRFs) and connectionist temporal\nclassification (CTC) are two sequence labeling methods used for end-to-end\ntraining of speech recognition models. Both models define a transcription\nprobability by marginalizing decisions about latent segmentation alternatives\nto derive a sequence probability: the former uses a globally normalized joint\nmodel of segment labels and durations, and the latter classifies each frame as\neither an output symbol or a \"continuation\" of the previous label. In this\npaper, we train a recognition model by optimizing an interpolation between the\nSCRF and CTC losses, where the same recurrent neural network (RNN) encoder is\nused for feature extraction for both outputs. We find that this multitask\nobjective improves recognition accuracy when decoding with either the SCRF or\nCTC models. Additionally, we show that CTC can also be used to pretrain the RNN\nencoder, which improves the convergence rate when learning the joint model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06589v2",
            "date": "2017-02-21 21:24:26+00:00",
            "title": "Neural Multi-Step Reasoning for Question Answering on Semi-Structured Tables",
            "authors": [
                "Till Haug",
                "Octavian-Eugen Ganea",
                "Paulina Grnarova"
            ],
            "abstract": "Advances in natural language processing tasks have gained momentum in recent\nyears due to the increasingly popular neural network methods. In this paper, we\nexplore deep learning techniques for answering multi-step reasoning questions\nthat operate on semi-structured tables. Challenges here arise from the level of\nlogical compositionality expressed by questions, as well as the domain\nopenness. Our approach is weakly supervised, trained on question-answer-table\ntriples without requiring intermediate strong supervision. It performs two\nphases: first, machine understandable logical forms (programs) are generated\nfrom natural language questions following the work of [Pasupat and Liang,\n2015]. Second, paraphrases of logical forms and questions are embedded in a\njointly learned vector space using word and character convolutional neural\nnetworks. A neural scoring function is further used to rank and retrieve the\nmost probable logical form (interpretation) of a question. Our best single\nmodel achieves 34.8% accuracy on the WikiTableQuestions dataset, while the best\nensemble of our models pushes the state-of-the-art score on this task to 38.7%,\nthus slightly surpassing both the engineered feature scoring baseline, as well\nas the Neural Programmer model of [Neelakantan et al., 2016].",
            "journal ref": "European Conference on Information Retrieval (ECIR) 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06594v2",
            "date": "2017-02-21 21:36:51+00:00",
            "title": "On the Complexity of CCG Parsing",
            "authors": [
                "Marco Kuhlmann",
                "Giorgio Satta",
                "Peter Jonsson"
            ],
            "abstract": "We study the parsing complexity of Combinatory Categorial Grammar (CCG) in\nthe formalism of Vijay-Shanker and Weir (1994). As our main result, we prove\nthat any parsing algorithm for this formalism will take in the worst case\nexponential time when the size of the grammar, and not only the length of the\ninput sentence, is included in the analysis. This sets the formalism of\nVijay-Shanker and Weir (1994) apart from weakly equivalent formalisms such as\nTree-Adjoining Grammar (TAG), for which parsing can be performed in time\npolynomial in the combined size of grammar and input sentence. Our results\ncontribute to a refined understanding of the class of mildly context-sensitive\ngrammars, and inform the search for new, mildly context-sensitive versions of\nCCG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06672v1",
            "date": "2017-02-22 04:30:09+00:00",
            "title": "Calculating Probabilities Simplifies Word Learning",
            "authors": [
                "Aida Nematzadeh",
                "Barend Beekhuizen",
                "Shanshan Huang",
                "Suzanne Stevenson"
            ],
            "abstract": "Children can use the statistical regularities of their environment to learn\nword meanings, a mechanism known as cross-situational learning. We take a\ncomputational approach to investigate how the information present during each\nobservation in a cross-situational framework can affect the overall acquisition\nof word meanings. We do so by formulating various in-the-moment learning\nmechanisms that are sensitive to different statistics of the environment, such\nas counts and conditional probabilities. Each mechanism introduces a unique\nsource of competition or mutual exclusivity bias to the model; the mechanism\nthat maximally uses the model's knowledge of word meanings performs the best.\nMoreover, the gap between this mechanism and others is amplified in more\nchallenging learning scenarios, such as learning from few examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06675v1",
            "date": "2017-02-22 04:50:23+00:00",
            "title": "Context-Aware Prediction of Derivational Word-forms",
            "authors": [
                "Ekaterina Vylomova",
                "Ryan Cotterell",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Derivational morphology is a fundamental and complex characteristic of\nlanguage. In this paper we propose the new task of predicting the derivational\nform of a given base-form lemma that is appropriate for a given context. We\npresent an encoder--decoder style neural network to produce a derived form\ncharacter-by-character, based on its corresponding character-level\nrepresentation of the base form and the context. We demonstrate that our model\nis able to generate valid context-sensitive derivations from known base forms,\nbut is less accurate under a lexicon agnostic setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06696v1",
            "date": "2017-02-22 07:41:08+00:00",
            "title": "One Representation per Word - Does it make Sense for Composition?",
            "authors": [
                "Thomas Kober",
                "Julie Weeds",
                "John Wilkie",
                "Jeremy Reffin",
                "David Weir"
            ],
            "abstract": "In this paper, we investigate whether an a priori disambiguation of word\nsenses is strictly necessary or whether the meaning of a word in context can be\ndisambiguated through composition alone. We evaluate the performance of\noff-the-shelf single-vector and multi-sense vector models on a benchmark phrase\nsimilarity task and a novel task for word-sense discrimination. We find that\nsingle-sense vector models perform as well or better than multi-sense vector\nmodels despite arguably less clean elementary representations. Our findings\nfurthermore show that simple composition functions such as pointwise addition\nare able to recover sense specific information from a single-sense vector model\nremarkably well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06703v1",
            "date": "2017-02-22 08:32:47+00:00",
            "title": "Data Distillation for Controlling Specificity in Dialogue Generation",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Dan Jurafsky"
            ],
            "abstract": "People speak at different levels of specificity in different situations.\nDepending on their knowledge, interlocutors, mood, etc.} A conversational agent\nshould have this ability and know when to be specific and when to be general.\nWe propose an approach that gives a neural network--based conversational agent\nthis ability. Our approach involves alternating between \\emph{data\ndistillation} and model training : removing training examples that are closest\nto the responses most commonly produced by the model trained from the last\nround and then retrain the model on the remaining dataset. Dialogue generation\nmodels trained with different degrees of data distillation manifest different\nlevels of specificity.\n  We then train a reinforcement learning system for selecting among this pool\nof generation models, to choose the best level of specificity for a given\ninput. Compared to the original generative model trained without distillation,\nthe proposed system is capable of generating more interesting and\nhigher-quality responses, in addition to appropriately adjusting specificity\ndepending on the context.\n  Our research constitutes a specific case of a broader approach involving\ntraining multiple subsystems from a single dataset distinguished by differences\nin a specific property one wishes to model. We show that from such a set of\nsubsystems, one can use reinforcement learning to build a system that tailors\nits output to different input contexts at test time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06709v1",
            "date": "2017-02-22 08:59:37+00:00",
            "title": "Fine-Grained Entity Type Classification by Jointly Learning Representations and Label Embeddings",
            "authors": [
                "Abhishek",
                "Ashish Anand",
                "Amit Awekar"
            ],
            "abstract": "Fine-grained entity type classification (FETC) is the task of classifying an\nentity mention to a broad set of types. Distant supervision paradigm is\nextensively used to generate training data for this task. However, generated\ntraining data assigns same set of labels to every mention of an entity without\nconsidering its local context. Existing FETC systems have two major drawbacks:\nassuming training data to be noise free and use of hand crafted features. Our\nwork overcomes both drawbacks. We propose a neural network model that jointly\nlearns entity mentions and their context representation to eliminate use of\nhand crafted features. Our model treats training data as noisy and uses\nnon-parametric variant of hinge loss function. Experiments show that the\nproposed model outperforms previous state-of-the-art methods on two publicly\navailable datasets, namely FIGER (GOLD) and BBN with an average relative\nimprovement of 2.69% in micro-F1 score. Knowledge learnt by our model on one\ndataset can be transferred to other datasets while using same model or other\nFETC systems. These approaches of transferring knowledge further improve the\nperformance of respective models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06733v1",
            "date": "2017-02-22 10:10:44+00:00",
            "title": "Improving a Strong Neural Parser with Conjunction-Specific Features",
            "authors": [
                "Jessica Ficler",
                "Yoav Goldberg"
            ],
            "abstract": "While dependency parsers reach very high overall accuracy, some dependency\nrelations are much harder than others. In particular, dependency parsers\nperform poorly in coordination construction (i.e., correctly attaching the\n\"conj\" relation). We extend a state-of-the-art dependency parser with\nconjunction-specific features, focusing on the similarity between the conjuncts\nhead words. Training the extended parser yields an improvement in \"conj\"\nattachment as well as in overall dependency parsing accuracy on the Stanford\ndependency conversion of the Penn TreeBank.",
            "journal ref": "EACL 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06740v3",
            "date": "2017-02-22 10:34:47+00:00",
            "title": "Improving Chinese SRL with Heterogeneous Annotations",
            "authors": [
                "Qiaolin Xia",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "Previous studies on Chinese semantic role labeling (SRL) have concentrated on\nsingle semantically annotated corpus. But the training data of single corpus is\noften limited. Meanwhile, there usually exists other semantically annotated\ncorpora for Chinese SRL scattered across different annotation frameworks. Data\nsparsity remains a bottleneck. This situation calls for larger training\ndatasets, or effective approaches which can take advantage of highly\nheterogeneous data. In these papers, we focus mainly on the latter, that is, to\nimprove Chinese SRL by using heterogeneous corpora together. We propose a novel\nprogressive learning model which augments the Progressive Neural Network with\nGated Recurrent Adapters. The model can accommodate heterogeneous inputs and\neffectively transfer knowledge between them. We also release a new corpus,\nChinese SemBank, for Chinese SRL. Experiments on CPB 1.0 show that ours model\noutperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06794v1",
            "date": "2017-02-22 13:49:18+00:00",
            "title": "Tackling Error Propagation through Reinforcement Learning: A Case of Greedy Dependency Parsing",
            "authors": [
                "Minh Le",
                "Antske Fokkens"
            ],
            "abstract": "Error propagation is a common problem in NLP. Reinforcement learning explores\nerroneous states during training and can therefore be more robust when mistakes\nare made early in a process. In this paper, we apply reinforcement learning to\ngreedy dependency parsing which is known to suffer from error propagation.\nReinforcement learning improves accuracy of both labeled and unlabeled\ndependencies of the Stanford Neural Dependency Parser, a high performance\ngreedy parser, while maintaining its efficiency. We investigate the portion of\nerrors which are the result of error propagation and confirm that reinforcement\nlearning reduces the occurrence of error propagation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06891v1",
            "date": "2017-02-22 16:50:25+00:00",
            "title": "EVE: Explainable Vector Based Embedding Technique Using Wikipedia",
            "authors": [
                "M. Atif Qureshi",
                "Derek Greene"
            ],
            "abstract": "We present an unsupervised explainable word embedding technique, called EVE,\nwhich is built upon the structure of Wikipedia. The proposed model defines the\ndimensions of a semantic vector representing a word using human-readable\nlabels, thereby it readily interpretable. Specifically, each vector is\nconstructed using the Wikipedia category graph structure together with the\nWikipedia article link structure. To test the effectiveness of the proposed\nword embedding model, we consider its usefulness in three fundamental tasks: 1)\nintruder detection - to evaluate its ability to identify a non-coherent vector\nfrom a list of coherent vectors, 2) ability to cluster - to evaluate its\ntendency to group related vectors together while keeping unrelated vectors in\nseparate clusters, and 3) sorting relevant items first - to evaluate its\nability to rank vectors (items) relevant to the query in the top order of the\nresult. For each task, we also propose a strategy to generate a task-specific\nhuman-interpretable explanation from the model. These demonstrate the overall\neffectiveness of the explainable embeddings generated by EVE. Finally, we\ncompare EVE with the Word2Vec, FastText, and GloVe embedding techniques across\nthe three tasks, and report improvements over the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07015v1",
            "date": "2017-02-22 21:44:02+00:00",
            "title": "Unsupervised Learning of Morphological Forests",
            "authors": [
                "Jiaming Luo",
                "Karthik Narasimhan",
                "Regina Barzilay"
            ],
            "abstract": "This paper focuses on unsupervised modeling of morphological families,\ncollectively comprising a forest over the language vocabulary. This formulation\nenables us to capture edgewise properties reflecting single-step morphological\nderivations, along with global distributional properties of the entire forest.\nThese global properties constrain the size of the affix set and encourage\nformation of tight morphological families. The resulting objective is solved\nusing Integer Linear Programming (ILP) paired with contrastive estimation. We\ntrain the model by alternating between optimizing the local log-linear model\nand the global ILP objective. We evaluate our system on three tasks: root\ndetection, clustering of morphological families and segmentation. Our\nexperiments demonstrate that our model yields consistent gains in all three\ntasks compared with the best published results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07046v1",
            "date": "2017-02-22 23:39:03+00:00",
            "title": "Feature Generation for Robust Semantic Role Labeling",
            "authors": [
                "Travis Wolfe",
                "Mark Dredze",
                "Benjamin Van Durme"
            ],
            "abstract": "Hand-engineered feature sets are a well understood method for creating robust\nNLP models, but they require a lot of expertise and effort to create. In this\nwork we describe how to automatically generate rich feature sets from simple\nunits called featlets, requiring less engineering. Using information gain to\nguide the generation process, we train models which rival the state of the art\non two standard Semantic Role Labeling datasets with almost no task or\nlinguistic insight.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07117v1",
            "date": "2017-02-23 07:16:03+00:00",
            "title": "LTSG: Latent Topical Skip-Gram for Mutually Learning Topic Model and Vector Representations",
            "authors": [
                "Jarvan Law",
                "Hankz Hankui Zhuo",
                "Junhua He",
                "Erhu Rong"
            ],
            "abstract": "Topic models have been widely used in discovering latent topics which are\nshared across documents in text mining. Vector representations, word embeddings\nand topic embeddings, map words and topics into a low-dimensional and dense\nreal-value vector space, which have obtained high performance in NLP tasks.\nHowever, most of the existing models assume the result trained by one of them\nare perfect correct and used as prior knowledge for improving the other model.\nSome other models use the information trained from external large corpus to\nhelp improving smaller corpus. In this paper, we aim to build such an algorithm\nframework that makes topic models and vector representations mutually improve\neach other within the same corpus. An EM-style algorithm framework is employed\nto iteratively optimize both topic model and vector representations.\nExperimental results show that our model outperforms state-of-art methods on\nvarious NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07203v2",
            "date": "2017-02-23 13:13:53+00:00",
            "title": "Utilizing Lexical Similarity between Related, Low-resource Languages for Pivot-based SMT",
            "authors": [
                "Anoop Kunchukuttan",
                "Maulik Shah",
                "Pradyot Prakash",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "We investigate pivot-based translation between related languages in a low\nresource, phrase-based SMT setting. We show that a subword-level pivot-based\nSMT model using a related pivot language is substantially better than word and\nmorpheme-level pivot models. It is also highly competitive with the best direct\ntranslation model, which is encouraging as no direct source-target training\ncorpus is used. We also show that combining multiple related language pivot\nmodels can rival a direct translation model. Thus, the use of subwords as\ntranslation units coupled with multiple related pivot languages can compensate\nfor the lack of a direct parallel corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07285v2",
            "date": "2017-02-23 16:47:01+00:00",
            "title": "Are Emojis Predictable?",
            "authors": [
                "Francesco Barbieri",
                "Miguel Ballesteros",
                "Horacio Saggion"
            ],
            "abstract": "Emojis are ideograms which are naturally combined with plain text to visually\ncomplement or condense the meaning of a message. Despite being widely used in\nsocial media, their underlying semantics have received little attention from a\nNatural Language Processing standpoint. In this paper, we investigate the\nrelation between words and emojis, studying the novel task of predicting which\nemojis are evoked by text-based tweet messages. We train several models based\non Long Short-Term Memory networks (LSTMs) in this task. Our experimental\nresults show that our neural model outperforms two baselines as well as humans\nsolving the same task, suggesting that computational models are able to better\ncapture the underlying semantics of emojis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07324v1",
            "date": "2017-02-23 18:19:35+00:00",
            "title": "Inherent Biases of Recurrent Neural Networks for Phonological Assimilation and Dissimilation",
            "authors": [
                "Amanda Doucette"
            ],
            "abstract": "A recurrent neural network model of phonological pattern learning is\nproposed. The model is a relatively simple neural network with one recurrent\nlayer, and displays biases in learning that mimic observed biases in human\nlearning. Single-feature patterns are learned faster than two-feature patterns,\nand vowel or consonant-only patterns are learned faster than patterns involving\nvowels and consonants, mimicking the results of laboratory learning\nexperiments. In non-recurrent models, capturing these biases requires the use\nof alpha features or some other representation of repeated features, but with a\nrecurrent neural network, these elaborations are not necessary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07495v1",
            "date": "2017-02-24 08:35:10+00:00",
            "title": "Dirichlet-vMF Mixture Model",
            "authors": [
                "Shaohua Li"
            ],
            "abstract": "This document is about the multi-document Von-Mises-Fisher mixture model with\na Dirichlet prior, referred to as VMFMix. VMFMix is analogous to Latent\nDirichlet Allocation (LDA) in that they can capture the co-occurrence patterns\nacorss multiple documents. The difference is that in VMFMix, the topic-word\ndistribution is defined on a continuous n-dimensional hypersphere. Hence VMFMix\nis used to derive topic embeddings, i.e., representative vectors, from multiple\nsets of embedding vectors. An efficient Variational Expectation-Maximization\ninference algorithm is derived. The performance of VMFMix on two document\nclassification tasks is reported, with some preliminary analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07507v1",
            "date": "2017-02-24 09:26:10+00:00",
            "title": "Use Generalized Representations, But Do Not Forget Surface Features",
            "authors": [
                "Nafise Sadat Moosavi",
                "Michael Strube"
            ],
            "abstract": "Only a year ago, all state-of-the-art coreference resolvers were using an\nextensive amount of surface features. Recently, there was a paradigm shift\ntowards using word embeddings and deep neural networks, where the use of\nsurface features is very limited. In this paper, we show that a simple SVM\nmodel with surface features outperforms more complex neural models for\ndetecting anaphoric mentions. Our analysis suggests that using generalized\nrepresentations and surface features have different strength that should be\nboth taken into account for improving coreference resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07793v1",
            "date": "2017-02-24 22:49:13+00:00",
            "title": "Residual Convolutional CTC Networks for Automatic Speech Recognition",
            "authors": [
                "Yisen Wang",
                "Xuejiao Deng",
                "Songbai Pu",
                "Zhiheng Huang"
            ],
            "abstract": "Deep learning approaches have been widely used in Automatic Speech\nRecognition (ASR) and they have achieved a significant accuracy improvement.\nEspecially, Convolutional Neural Networks (CNNs) have been revisited in ASR\nrecently. However, most CNNs used in existing work have less than 10 layers\nwhich may not be deep enough to capture all human speech signal information. In\nthis paper, we propose a novel deep and wide CNN architecture denoted as\nRCNN-CTC, which has residual connections and Connectionist Temporal\nClassification (CTC) loss function. RCNN-CTC is an end-to-end system which can\nexploit temporal and spectral structures of speech signals simultaneously.\nFurthermore, we introduce a CTC-based system combination, which is different\nfrom the conventional frame-wise senone-based one. The basic subsystems adopted\nin the combination are different types and thus mutually complementary to each\nother. Experimental results show that our proposed single system RCNN-CTC can\nachieve the lowest word error rate (WER) on WSJ and Tencent Chat data sets,\ncompared to several widely used neural network systems in ASR. In addition, the\nproposed system combination can offer a further error reduction on these two\ndata sets, resulting in relative WER reductions of $14.91\\%$ and $6.52\\%$ on\nWSJ dev93 and Tencent Chat data sets respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07835v1",
            "date": "2017-02-25 05:16:15+00:00",
            "title": "Critical Survey of the Freely Available Arabic Corpora",
            "authors": [
                "Wajdi Zaghouani"
            ],
            "abstract": "The availability of corpora is a major factor in building natural language\nprocessing applications. However, the costs of acquiring corpora can prevent\nsome researchers from going further in their endeavours. The ease of access to\nfreely available corpora is urgent needed in the NLP research community\nespecially for language such as Arabic. Currently, there is not easy was to\naccess to a comprehensive and updated list of freely available Arabic corpora.\nWe present in this paper, the results of a recent survey conducted to identify\nthe list of the freely available Arabic corpora and language resources. Our\npreliminary results showed an initial list of 66 sources. We presents our\nfindings in the various categories studied and we provided the direct links to\nget the data when possible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07998v1",
            "date": "2017-02-26 08:07:26+00:00",
            "title": "Detecting (Un)Important Content for Single-Document News Summarization",
            "authors": [
                "Yinfei Yang",
                "Forrest Sheng Bao",
                "Ani Nenkova"
            ],
            "abstract": "We present a robust approach for detecting intrinsic sentence importance in\nnews, by training on two corpora of document-summary pairs. When used for\nsingle-document summarization, our approach, combined with the \"beginning of\ndocument\" heuristic, outperforms a state-of-the-art summarizer and the\nbeginning-of-article baseline in both automatic and manual evaluations. These\nresults represent an important advance because in the absence of cross-document\nrepetition, single document summarizers for news have not been able to\nconsistently outperform the strong beginning-of-article baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08021v1",
            "date": "2017-02-26 11:22:41+00:00",
            "title": "Friends and Enemies of Clinton and Trump: Using Context for Detecting Stance in Political Tweets",
            "authors": [
                "Mirko Lai",
                "Delia Iraz\u00fa Hern\u00e1ndez Far\u00edas",
                "Viviana Patti",
                "Paolo Rosso"
            ],
            "abstract": "Stance detection, the task of identifying the speaker's opinion towards a\nparticular target, has attracted the attention of researchers. This paper\ndescribes a novel approach for detecting stance in Twitter. We define a set of\nfeatures in order to consider the context surrounding a target of interest with\nthe final aim of training a model for predicting the stance towards the\nmentioned targets. In particular, we are interested in investigating political\ndebates in social media. For this reason we evaluated our approach focusing on\ntwo targets of the SemEval-2016 Task6 on Detecting stance in tweets, which are\nrelated to the political campaign for the 2016 U.S. presidential elections:\nHillary Clinton vs. Donald Trump. For the sake of comparison with the state of\nthe art, we evaluated our model against the dataset released in the\nSemEval-2016 Task 6 shared task competition. Our results outperform the best\nones obtained by participating teams, and show that information about enemies\nand friends of politicians help in detecting stance towards them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08217v1",
            "date": "2017-02-27 10:09:46+00:00",
            "title": "A case study on English-Malayalam Machine Translation",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper we present our work on a case study on Statistical Machine\nTranslation (SMT) and Rule based machine translation (RBMT) for translation\nfrom English to Malayalam and Malayalam to English. One of the motivations of\nour study is to make a three way performance comparison, such as, a) SMT and\nRBMT b) English to Malayalam SMT and Malayalam to English SMT c) English to\nMalayalam RBMT and Malayalam to English RBMT. We describe the development of\nEnglish to Malayalam and Malayalam to English baseline phrase based SMT system\nand the evaluation of its performance compared against the RBMT system. Based\non our study the observations are: a) SMT systems outperform RBMT systems, b)\nIn the case of SMT, English - Malayalam systems perform better than that of\nMalayalam - English systems, c) In the case RBMT, Malayalam to English systems\nare performing better than English to Malayalam systems. Based on our\nevaluations and detailed error analysis, we describe the requirements of\nincorporating morphological processing into the SMT to improve the accuracy of\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08303v1",
            "date": "2017-02-27 14:37:21+00:00",
            "title": "Identifying beneficial task relations for multi-task learning in deep neural networks",
            "authors": [
                "Joachim Bingel",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Multi-task learning (MTL) in deep neural networks for NLP has recently\nreceived increasing interest due to some compelling benefits, including its\npotential to efficiently regularize models and to reduce the need for labeled\ndata. While it has brought significant improvements in a number of NLP tasks,\nmixed results have been reported, and little is known about the conditions\nunder which MTL leads to gains in NLP. This paper sheds light on the specific\ntask relations that can lead to gains from MTL models over single-task setups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08450v1",
            "date": "2017-02-27 13:37:15+00:00",
            "title": "A Knowledge-Based Approach to Word Sense Disambiguation by distributional selection and semantic features",
            "authors": [
                "Mokhtar Billami"
            ],
            "abstract": "Word sense disambiguation improves many Natural Language Processing (NLP)\napplications such as Information Retrieval, Information Extraction, Machine\nTranslation, or Lexical Simplification. Roughly speaking, the aim is to choose\nfor each word in a text its best sense. One of the most popular method\nestimates local semantic similarity relatedness between two word senses and\nthen extends it to all words from text. The most direct method computes a rough\nscore for every pair of word senses and chooses the lexical chain that has the\nbest score (we can imagine the exponential complexity that returns this\ncomprehensive approach). In this paper, we propose to use a combinatorial\noptimization metaheuristic for choosing the nearest neighbors obtained by\ndistributional selection around the word to disambiguate. The test and the\nevaluation of our method concern a corpus written in French by means of the\nsemantic network BabelNet. The obtained accuracy rate is 78 % on all names and\nverbs chosen for the evaluation.",
            "journal ref": "22\\`eme Conf\\'erence sur le Traitement Automatique des Langues\n  Naturelles et 17\\`eme Rencontre des \\'Etudiants Chercheurs en Informatique\n  pour le Traitement Automatique des Langues, Jun 2015, CAEN, France. 22\\`eme\n  Conf\\'erence sur le Traitement Automatique des Langues Naturelles et 17\\`eme\n  Rencontre des \\'Etudiants Chercheurs en Informatique pour le Traitement\n  Automatique des Langues, pp.13--24, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08451v1",
            "date": "2017-02-27 13:38:08+00:00",
            "title": "Approches d'analyse distributionnelle pour am\u00e9liorer la d\u00e9sambigu\u00efsation s\u00e9mantique",
            "authors": [
                "Mokhtar Billami",
                "N\u00faria Gala"
            ],
            "abstract": "Word sense disambiguation (WSD) improves many Natural Language Processing\n(NLP) applications such as Information Retrieval, Machine Translation or\nLexical Simplification. WSD is the ability of determining a word sense among\ndifferent ones within a polysemic lexical unit taking into account the context.\nThe most straightforward approach uses a semantic proximity measure between the\nword sense candidates of the target word and those of its context. Such a\nmethod very easily entails a combinatorial explosion. In this paper, we propose\ntwo methods based on distributional analysis which enable to reduce the\nexponential complexity without losing the coherence. We present a comparison\nbetween the selection of distributional neighbors and the linearly nearest\nneighbors. The figures obtained show that selecting distributional neighbors\nleads to better results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08563v3",
            "date": "2017-02-27 22:25:45+00:00",
            "title": "Soft Label Memorization-Generalization for Natural Language Inference",
            "authors": [
                "John P. Lalor",
                "Hao Wu",
                "Hong Yu"
            ],
            "abstract": "Often when multiple labels are obtained for a training example it is assumed\nthat there is an element of noise that must be accounted for. It has been shown\nthat this disagreement can be considered signal instead of noise. In this work\nwe investigate using soft labels for training data to improve generalization in\nmachine learning models. However, using soft labels for training Deep Neural\nNetworks (DNNs) is not practical due to the costs involved in obtaining\nmultiple labels for large data sets. We propose soft label\nmemorization-generalization (SLMG), a fine-tuning approach to using soft labels\nfor training DNNs. We assume that differences in labels provided by human\nannotators represent ambiguity about the true label instead of noise.\nExperiments with SLMG demonstrate improved generalization performance on the\nNatural Language Inference (NLI) task. Our experiments show that by injecting a\nsmall percentage of soft label training data (0.03% of training set size) we\ncan improve generalization performance over several baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08653v2",
            "date": "2017-02-28 05:43:10+00:00",
            "title": "Scaffolding Networks: Incremental Learning and Teaching Through Questioning",
            "authors": [
                "Asli Celikyilmaz",
                "Li Deng",
                "Lihong Li",
                "Chong Wang"
            ],
            "abstract": "We introduce a new paradigm of learning for reasoning, understanding, and\nprediction, as well as the scaffolding network to implement this paradigm. The\nscaffolding network embodies an incremental learning approach that is\nformulated as a teacher-student network architecture to teach machines how to\nunderstand text and do reasoning. The key to our computational scaffolding\napproach is the interactions between the teacher and the student through\nsequential questioning. The student observes each sentence in the text\nincrementally, and it uses an attention-based neural net to discover and\nregister the key information in relation to its current memory. Meanwhile, the\nteacher asks questions about the observed text, and the student network gets\nrewarded by correctly answering these questions. The entire network is updated\ncontinually using reinforcement learning. Our experimental results on synthetic\nand real datasets show that the scaffolding network not only outperforms\nstate-of-the-art methods but also learns to do reasoning in a scalable way even\nwith little human generated input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08866v1",
            "date": "2017-02-24 21:49:45+00:00",
            "title": "Studying Positive Speech on Twitter",
            "authors": [
                "Marina Sokolova",
                "Vera Sazonova",
                "Kanyi Huang",
                "Rudraneel Chakraboty",
                "Stan Matwin"
            ],
            "abstract": "We present results of empirical studies on positive speech on Twitter. By\npositive speech we understand speech that works for the betterment of a given\nsituation, in this case relations between different communities in a\nconflict-prone country. We worked with four Twitter data sets. Through\nsemi-manual opinion mining, we found that positive speech accounted for < 1% of\nthe data . In fully automated studies, we tested two approaches: unsupervised\nstatistical analysis, and supervised text classification based on distributed\nword representation. We discuss benefits and challenges of those approaches and\nreport empirical evidence obtained in the study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00089v1",
            "date": "2017-02-28 23:54:57+00:00",
            "title": "A Joint Identification Approach for Argumentative Writing Revisions",
            "authors": [
                "Fan Zhang",
                "Diane Litman"
            ],
            "abstract": "Prior work on revision identification typically uses a pipeline method:\nrevision extraction is first conducted to identify the locations of revisions\nand revision classification is then conducted on the identified revisions. Such\na setting propagates the errors of the revision extraction step to the revision\nclassification step. This paper proposes an approach that identifies the\nrevision location and the revision type jointly to solve the issue of error\npropagation. It utilizes a sequence representation of revisions and conducts\nsequence labeling for revision identification. A mutation-based approach is\nutilized to update identification sequences. Results demonstrate that our\nproposed approach yields better performance on both revision location\nextraction and revision type classification compared to a pipeline baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00538v2",
            "date": "2017-03-01 22:37:02+00:00",
            "title": "Unsupervised Ensemble Ranking of Terms in Electronic Health Record Notes Based on Their Importance to Patients",
            "authors": [
                "Jinying Chen",
                "Hong Yu"
            ],
            "abstract": "Background: Electronic health record (EHR) notes contain abundant medical\njargon that can be difficult for patients to comprehend. One way to help\npatients is to reduce information overload and help them focus on medical terms\nthat matter most to them.\n  Objective: The aim of this work was to develop FIT (Finding Important Terms\nfor patients), an unsupervised natural language processing (NLP) system that\nranks medical terms in EHR notes based on their importance to patients.\n  Methods: We built FIT on a new unsupervised ensemble ranking model derived\nfrom the biased random walk algorithm to combine heterogeneous information\nresources for ranking candidate terms from each EHR note. Specifically, FIT\nintegrates four single views for term importance: patient use of medical\nconcepts, document-level term salience, word-occurrence based term relatedness,\nand topic coherence. It also incorporates partial information of term\nimportance as conveyed by terms' unfamiliarity levels and semantic types. We\nevaluated FIT on 90 expert-annotated EHR notes and compared it with three\nbenchmark unsupervised ensemble ranking methods.\n  Results: FIT achieved 0.885 AUC-ROC for ranking candidate terms from EHR\nnotes to identify important terms. When including term identification, the\nperformance of FIT for identifying important terms from EHR notes was 0.813\nAUC-ROC. It outperformed the three ensemble rankers for most metrics. Its\nperformance is relatively insensitive to its parameter.\n  Conclusions: FIT can automatically identify EHR terms important to patients\nand may help develop personalized interventions to improve quality of care. By\nusing unsupervised learning as well as a robust and flexible framework for\ninformation fusion, FIT can be readily applied to other domains and\napplications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00572v3",
            "date": "2017-03-02 01:08:10+00:00",
            "title": "Structural Embedding of Syntactic Trees for Machine Comprehension",
            "authors": [
                "Rui Liu",
                "Junjie Hu",
                "Wei Wei",
                "Zi Yang",
                "Eric Nyberg"
            ],
            "abstract": "Deep neural networks for machine comprehension typically utilizes only word\nor character embeddings without explicitly taking advantage of structured\nlinguistic information such as constituency trees and dependency trees. In this\npaper, we propose structural embedding of syntactic trees (SEST), an algorithm\nframework to utilize structured information and encode them into vector\nrepresentations that can boost the performance of algorithms for the machine\ncomprehension. We evaluate our approach using a state-of-the-art neural\nattention model on the SQuAD dataset. Experimental results demonstrate that our\nmodel can accurately identify the syntactic boundaries of the sentences and\nextract answers that are syntactically coherent over the baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00782v1",
            "date": "2017-03-02 13:49:23+00:00",
            "title": "Lock-Free Parallel Perceptron for Graph-based Dependency Parsing",
            "authors": [
                "Xu Sun",
                "Shuming Ma"
            ],
            "abstract": "Dependency parsing is an important NLP task. A popular approach for\ndependency parsing is structured perceptron. Still, graph-based dependency\nparsing has the time complexity of $O(n^3)$, and it suffers from slow training.\nTo deal with this problem, we propose a parallel algorithm called parallel\nperceptron. The parallel algorithm can make full use of a multi-core computer\nwhich saves a lot of training time. Based on experiments we observe that\ndependency parsing with parallel perceptron can achieve 8-fold faster training\nspeed than traditional structured perceptron methods when using 10 threads, and\nwith no loss at all in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00993v1",
            "date": "2017-03-02 23:58:54+00:00",
            "title": "A Comparative Study of Word Embeddings for Reading Comprehension",
            "authors": [
                "Bhuwan Dhingra",
                "Hanxiao Liu",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "The focus of past machine learning research for Reading Comprehension tasks\nhas been primarily on the design of novel deep learning architectures. Here we\nshow that seemingly minor choices made on (1) the use of pre-trained word\nembeddings, and (2) the representation of out-of-vocabulary tokens at test\ntime, can turn out to have a larger impact than architectural choices on the\nfinal performance. We systematically explore several options for these choices,\nand provide recommendations to researchers working in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01024v1",
            "date": "2017-03-03 03:14:28+00:00",
            "title": "Exponential Moving Average Model in Parallel Speech Recognition Training",
            "authors": [
                "Xu Tian",
                "Jun Zhang",
                "Zejun Ma",
                "Yi He",
                "Juan Wei"
            ],
            "abstract": "As training data rapid growth, large-scale parallel training with multi-GPUs\ncluster is widely applied in the neural network model learning currently.We\npresent a new approach that applies exponential moving average method in\nlarge-scale parallel training of neural network model. It is a non-interference\nstrategy that the exponential moving average model is not broadcasted to\ndistributed workers to update their local models after model synchronization in\nthe training process, and it is implemented as the final model of the training\nsystem. Fully-connected feed-forward neural networks (DNNs) and deep\nunidirectional Long short-term memory (LSTM) recurrent neural networks (RNNs)\nare successfully trained with proposed method for large vocabulary continuous\nspeech recognition on Shenma voice search data in Mandarin. The character error\nrate (CER) of Mandarin speech recognition further degrades than\nstate-of-the-art approaches of parallel training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01485v1",
            "date": "2017-03-04 15:55:01+00:00",
            "title": "Lexical Resources for Hindi Marathi MT",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper we describe some ways to utilize various lexical resources to\nimprove the quality of statistical machine translation system. We have\naugmented the training corpus with various lexical resources such as\nIndoWordnet semantic relation set, function words, kridanta pairs and verb\nphrases etc. Our research on the usage of lexical resources mainly focused on\ntwo ways such as augmenting parallel corpus with more vocabulary and augmenting\nwith various word forms. We have described case studies, evaluations and\ndetailed error analysis for both Marathi to Hindi and Hindi to Marathi machine\ntranslation systems. From the evaluations we observed that, there is an\nincremental growth in the quality of machine translation as the usage of\nvarious lexical resources increases. Moreover usage of various lexical\nresources helps to improve the coverage and quality of machine translation\nwhere limited parallel corpus is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01694v2",
            "date": "2017-03-06 00:38:51+00:00",
            "title": "Word forms - not just their lengths- are optimized for efficient communication",
            "authors": [
                "Stephan C. Meylan",
                "Thomas L. Griffiths"
            ],
            "abstract": "The inverse relationship between the length of a word and the frequency of\nits use, first identified by G.K. Zipf in 1935, is a classic empirical law that\nholds across a wide range of human languages. We demonstrate that length is one\naspect of a much more general property of words: how distinctive they are with\nrespect to other words in a language. Distinctiveness plays a critical role in\nrecognizing words in fluent speech, in that it reflects the strength of\npotential competitors when selecting the best candidate for an ambiguous\nsignal. Phonological information content, a measure of a word's string\nprobability under a statistical model of a language's sound or character\nsequences, concisely captures distinctiveness. Examining large-scale corpora\nfrom 13 languages, we find that distinctiveness significantly outperforms word\nlength as a predictor of frequency. This finding provides evidence that\nlisteners' processing constraints shape fine-grained aspects of word forms\nacross languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01726v1",
            "date": "2017-03-06 05:07:12+00:00",
            "title": "A Novel Comprehensive Approach for Estimating Concept Semantic Similarity in WordNet",
            "authors": [
                "Xiao-gang Zhang",
                "Shou-qian Sun",
                "Ke-jun Zhang"
            ],
            "abstract": "Computation of semantic similarity between concepts is an important\nfoundation for many research works. This paper focuses on IC computing methods\nand IC measures, which estimate the semantic similarities between concepts by\nexploiting the topological parameters of the taxonomy. Based on analyzing\nrepresentative IC computing methods and typical semantic similarity measures,\nwe propose a new hybrid IC computing method. Through adopting the parameter\ndhyp and lch, we utilize the new IC computing method and propose a novel\ncomprehensive measure of semantic similarity between concepts. An experiment\nbased on WordNet \"is a\" taxonomy has been designed to test representative\nmeasures and our measure on benchmark dataset R&G, and the results show that\nour measure can obviously improve the similarity accuracy. We evaluate the\nproposed approach by comparing the correlation coefficients between five\nmeasures and the artificial data. The results show that our proposal\noutperforms the previous measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02019v1",
            "date": "2017-03-06 18:44:49+00:00",
            "title": "Performing Stance Detection on Twitter Data using Computational Linguistics Techniques",
            "authors": [
                "Gourav G. Shenoy",
                "Erika H. Dsouza",
                "Sandra K\u00fcbler"
            ],
            "abstract": "As humans, we can often detect from a persons utterances if he or she is in\nfavor of or against a given target entity (topic, product, another person,\netc). But from the perspective of a computer, we need means to automatically\ndeduce the stance of the tweeter, given just the tweet text. In this paper, we\npresent our results of performing stance detection on twitter data using a\nsupervised approach. We begin by extracting bag-of-words to perform\nclassification using TIMBL, then try and optimize the features to improve\nstance detection accuracy, followed by extending the dataset with two sets of\nlexicons - arguing, and MPQA subjectivity; next we explore the MALT parser and\nconstruct features using its dependency triples, finally we perform analysis\nusing Scikit-learn Random Forest implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02031v1",
            "date": "2017-03-05 15:07:10+00:00",
            "title": "Random vector generation of a semantic space",
            "authors": [
                "Jean-Fran\u00e7ois Delpech",
                "Sabine Ploux"
            ],
            "abstract": "We show how random vectors and random projection can be implemented in the\nusual vector space model to construct a Euclidean semantic space from a French\nsynonym dictionary. We evaluate theoretically the resulting noise and show the\nexperimental distribution of the similarities of terms in a neighborhood\naccording to the choice of parameters. We also show that the Schmidt\northogonalization process is applicable and can be used to separate homonyms\nwith distinct semantic meanings. Neighboring terms are easily arranged into\nsemantically significant clusters which are well suited to the generation of\nrealistic lists of synonyms and to such applications as word selection for\nautomatic text generation. This process, applicable to any language, can easily\nbe extended to collocations, is extremely fast and can be updated in real time,\nwhenever new synonyms are proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02136v1",
            "date": "2017-03-06 22:37:43+00:00",
            "title": "English Conversational Telephone Speech Recognition by Humans and Machines",
            "authors": [
                "George Saon",
                "Gakuto Kurata",
                "Tom Sercu",
                "Kartik Audhkhasi",
                "Samuel Thomas",
                "Dimitrios Dimitriadis",
                "Xiaodong Cui",
                "Bhuvana Ramabhadran",
                "Michael Picheny",
                "Lynn-Li Lim",
                "Bergul Roomi",
                "Phil Hall"
            ],
            "abstract": "One of the most difficult speech recognition tasks is accurate recognition of\nhuman to human communication. Advances in deep learning over the last few years\nhave produced major speech recognition improvements on the representative\nSwitchboard conversational corpus. Word error rates that just a few years ago\nwere 14% have dropped to 8.0%, then 6.6% and most recently 5.8%, and are now\nbelieved to be within striking range of human performance. This then raises two\nissues - what IS human performance, and how far down can we still drive speech\nrecognition error rates? A recent paper by Microsoft suggests that we have\nalready achieved human performance. In trying to verify this statement, we\nperformed an independent set of human performance measurements on two\nconversational tasks and found that human performance may be considerably\nbetter than what was earlier reported, giving the community a significantly\nharder goal to achieve. We also report on our own efforts in this area,\npresenting a set of acoustic and language modeling techniques that lowered the\nword error rate of our own English conversational telephone LVCSR system to the\nlevel of 5.5%/10.3% on the Switchboard/CallHome subsets of the Hub5 2000\nevaluation, which - at least at the writing of this paper - is a new\nperformance milestone (albeit not at what we measure to be human performance!).\nOn the acoustic side, we use a score fusion of three models: one LSTM with\nmultiple feature inputs, a second LSTM trained with speaker-adversarial\nmulti-task learning and a third residual net (ResNet) with 25 convolutional\nlayers and time-dilated convolutions. On the language modeling side, we use\nword and character LSTMs and convolutional WaveNet-style language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02166v1",
            "date": "2017-03-07 01:13:39+00:00",
            "title": "Building a Syllable Database to Solve the Problem of Khmer Word Segmentation",
            "authors": [
                "Nam Tran Van"
            ],
            "abstract": "Word segmentation is a basic problem in natural language processing. With the\nlanguages having the complex writing system like the Khmer language in Southern\nof Vietnam, this problem really very intractable, posing the significant\nchallenges. Although there are some experts in Vietnam as well as international\nhaving deeply researched this problem, there are still no reasonable results\nmeeting the demand, in particular, no treated thoroughly the ambiguous\nphenomenon, in the process of Khmer language processing so far. This paper\npresent a solution based on the syllable division into component clusters using\ntwo syllable models proposed, thereby building a Khmer syllable database, is\nstill not actually available. This method using a lexical database updated from\nthe online Khmer dictionaries and some supported dictionaries serving role of\ntraining data and complementary linguistic characteristics. Each component\ncluster is labelled and located by the first and last letter to identify\nentirety a syllable. This approach is workable and the test results achieve\nhigh accuracy, eliminate the ambiguity, contribute to solving the problem of\nword segmentation and applying efficiency in Khmer language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02517v1",
            "date": "2017-03-07 18:35:33+00:00",
            "title": "Learning opacity in Stratal Maximum Entropy Grammar",
            "authors": [
                "Aleksei Nazarov",
                "Joe Pater"
            ],
            "abstract": "Opaque phonological patterns are sometimes claimed to be difficult to learn;\nspecific hypotheses have been advanced about the relative difficulty of\nparticular kinds of opaque processes (Kiparsky 1971, 1973), and the kind of\ndata that will be helpful in learning an opaque pattern (Kiparsky 2000). In\nthis paper, we present a computationally implemented learning theory for one\ngrammatical theory of opacity: a Maximum Entropy version of Stratal OT\n(Berm\\'udez-Otero 1999, Kiparsky 2000), and test it on simplified versions of\nopaque French tense-lax vowel alternations and the opaque interaction of\ndiphthong raising and flapping in Canadian English. We find that the difficulty\nof opacity can be influenced by evidence for stratal affiliation: the Canadian\nEnglish case is easier if the learner encounters application of raising outside\nthe flapping context, or non-application of raising between words (i.e., <life>\nwith a raised vowel; <lie for> with a non-raised vowel).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02620v1",
            "date": "2017-03-07 22:13:17+00:00",
            "title": "Linguistic Knowledge as Memory for Recurrent Neural Networks",
            "authors": [
                "Bhuwan Dhingra",
                "Zhilin Yang",
                "William W. Cohen",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Training recurrent neural networks to model long term dependencies is\ndifficult. Hence, we propose to use external linguistic knowledge as an\nexplicit signal to inform the model which memories it should utilize.\nSpecifically, external knowledge is used to augment a sequence with typed edges\nbetween arbitrarily distant elements, and the resulting graph is decomposed\ninto directed acyclic subgraphs. We introduce a model that encodes such graphs\nas explicit memory in recurrent neural networks, and use it to model\ncoreference relations in text. We apply our model to several text comprehension\ntasks and achieve new state-of-the-art results on all considered benchmarks,\nincluding CNN, bAbi, and LAMBADA. On the bAbi QA tasks, our model solves 15 out\nof the 20 tasks with only 1000 training examples per task. Analysis of the\nlearned representations further demonstrates the ability of our model to encode\nfine-grained entity information across a document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02859v2",
            "date": "2017-03-08 14:51:09+00:00",
            "title": "A World of Difference: Divergent Word Interpretations among People",
            "authors": [
                "Tianran Hu",
                "Ruihua Song",
                "Maya Abtahian",
                "Philip Ding",
                "Xing Xie",
                "Jiebo Luo"
            ],
            "abstract": "Divergent word usages reflect differences among people. In this paper, we\npresent a novel angle for studying word usage divergence -- word\ninterpretations. We propose an approach that quantifies semantic differences in\ninterpretations among different groups of people. The effectiveness of our\napproach is validated by quantitative evaluations. Experiment results indicate\nthat divergences in word interpretations exist. We further apply the approach\nto two well studied types of differences between people -- gender and region.\nThe detected words with divergent interpretations reveal the unique features of\nspecific groups of people. For gender, we discover that certain different\ninterests, social attitudes, and characters between males and females are\nreflected in their divergent interpretations of many words. For region, we find\nthat specific interpretations of certain words reveal the geographical and\ncultural features of different regions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03091v1",
            "date": "2017-03-09 01:04:07+00:00",
            "title": "Deep Learning applied to NLP",
            "authors": [
                "Marc Moreno Lopez",
                "Jugal Kalita"
            ],
            "abstract": "Convolutional Neural Network (CNNs) are typically associated with Computer\nVision. CNNs are responsible for major breakthroughs in Image Classification\nand are the core of most Computer Vision systems today. More recently CNNs have\nbeen applied to problems in Natural Language Processing and gotten some\ninteresting results. In this paper, we will try to explain the basics of CNNs,\nits different variations and how they have been applied to NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03149v1",
            "date": "2017-03-09 06:20:49+00:00",
            "title": "Detecting Sockpuppets in Deceptive Opinion Spam",
            "authors": [
                "Marjan Hosseinia",
                "Arjun Mukherjee"
            ],
            "abstract": "This paper explores the problem of sockpuppet detection in deceptive opinion\nspam using authorship attribution and verification approaches. Two methods are\nexplored. The first is a feature subsampling scheme that uses the KL-Divergence\non stylistic language models of an author to find discriminative features. The\nsecond is a transduction scheme, spy induction that leverages the diversity of\nauthors in the unlabeled test set by sending a set of spies (positive samples)\nfrom the training set to retrieve hidden samples in the unlabeled test set\nusing nearest and farthest neighbors. Experiments using ground truth sockpuppet\ndata show the effectiveness of the proposed schemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03200v2",
            "date": "2017-03-09 09:46:56+00:00",
            "title": "Turkish PoS Tagging by Reducing Sparsity with Morpheme Tags in Small Datasets",
            "authors": [
                "Burcu Can",
                "Ahmet \u00dcst\u00fcn",
                "Murathan Kurfal\u0131"
            ],
            "abstract": "Sparsity is one of the major problems in natural language processing. The\nproblem becomes even more severe in agglutinating languages that are highly\nprone to be inflected. We deal with sparsity in Turkish by adopting\nmorphological features for part-of-speech tagging. We learn inflectional and\nderivational morpheme tags in Turkish by using conditional random fields (CRF)\nand we employ the morpheme tags in part-of-speech (PoS) tagging by using hidden\nMarkov models (HMMs) to mitigate sparsity. Results show that using morpheme\ntags in PoS tagging helps alleviate the sparsity in emission probabilities. Our\nmodel outperforms other hidden Markov model based PoS tagging models for small\ntraining datasets in Turkish. We obtain an accuracy of 94.1% in morpheme\ntagging and 89.2% in PoS tagging on a 5K training dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03640v1",
            "date": "2017-03-10 11:58:48+00:00",
            "title": "A Study of Metrics of Distance and Correlation Between Ranked Lists for Compositionality Detection",
            "authors": [
                "Christina Lioma",
                "Niels Dalum Hansen"
            ],
            "abstract": "Compositionality in language refers to how much the meaning of some phrase\ncan be decomposed into the meaning of its constituents and the way these\nconstituents are combined. Based on the premise that substitution by synonyms\nis meaning-preserving, compositionality can be approximated as the semantic\nsimilarity between a phrase and a version of that phrase where words have been\nreplaced by their synonyms. Different ways of representing such phrases exist\n(e.g., vectors [1] or language models [2]), and the choice of representation\naffects the measurement of semantic similarity.\n  We propose a new compositionality detection method that represents phrases as\nranked lists of term weights. Our method approximates the semantic similarity\nbetween two ranked list representations using a range of well-known distance\nand correlation metrics. In contrast to most state-of-the-art approaches in\ncompositionality detection, our method is completely unsupervised. Experiments\nwith a publicly available dataset of 1048 human-annotated phrases shows that,\ncompared to strong supervised baselines, our approach provides superior\nmeasurement of compositionality using any of the distance and correlation\nmetrics considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03666v1",
            "date": "2017-03-10 12:59:52+00:00",
            "title": "Comparison of SMT and RBMT; The Requirement of Hybridization for Marathi-Hindi MT",
            "authors": [
                "Sreelekha. S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "We present in this paper our work on comparison between Statistical Machine\nTranslation (SMT) and Rule-based machine translation for translation from\nMarathi to Hindi. Rule Based systems although robust take lots of time to\nbuild. On the other hand statistical machine translation systems are easier to\ncreate, maintain and improve upon. We describe the development of a basic\nMarathi-Hindi SMT system and evaluate its performance. Through a detailed error\nanalysis, we, point out the relative strengths and weaknesses of both systems.\nEffectively, we shall see that even with a small amount of training corpus a\nstatistical machine translation system has many advantages for high quality\ndomain specific machine translation over that of a rule-based counterpart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03771v1",
            "date": "2017-03-10 17:27:38+00:00",
            "title": "Coping with Construals in Broad-Coverage Semantic Annotation of Adpositions",
            "authors": [
                "Jena D. Hwang",
                "Archna Bhatia",
                "Na-Rae Han",
                "Tim O'Gorman",
                "Vivek Srikumar",
                "Nathan Schneider"
            ],
            "abstract": "We consider the semantics of prepositions, revisiting a broad-coverage\nannotation scheme used for annotating all 4,250 preposition tokens in a 55,000\nword corpus of English. Attempts to apply the scheme to adpositions and case\nmarkers in other languages, as well as some problematic cases in English, have\nled us to reconsider the assumption that a preposition's lexical contribution\nis equivalent to the role/relation that it mediates. Our proposal is to embrace\nthe potential for construal in adposition use, expressing such phenomena\ndirectly at the token level to manage complexity and avoid sense proliferation.\nWe suggest a framework to represent both the scene role and the adposition's\nlexical function so they can be annotated at scale---supporting automatic,\nstatistical processing of domain-general language---and sketch how this\nrepresentation would inform a constructional analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03842v2",
            "date": "2017-03-10 20:54:11+00:00",
            "title": "Effects of Limiting Memory Capacity on the Behaviour of Exemplar Dynamics",
            "authors": [
                "B. Goodman",
                "P. F. Tupper"
            ],
            "abstract": "Exemplar models are a popular class of models used to describe language\nchange. Here we study how limiting the memory capacity of an individual in\nthese models affects the system's behaviour. In particular we demonstrate the\neffect this change has on the extinction of categories. Previous work in\nexemplar dynamics has not addressed this question. In order to investigate\nthis, we will inspect a simplified exemplar model. We will prove for the\nsimplified model that all the sound categories but one will always become\nextinct, whether memory storage is limited or not. However, computer\nsimulations show that changing the number of stored memories alters how fast\ncategories become extinct.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03906v2",
            "date": "2017-03-11 04:17:46+00:00",
            "title": "Massive Exploration of Neural Machine Translation Architectures",
            "authors": [
                "Denny Britz",
                "Anna Goldie",
                "Minh-Thang Luong",
                "Quoc Le"
            ],
            "abstract": "Neural Machine Translation (NMT) has shown remarkable progress over the past\nfew years with production systems now being deployed to end-users. One major\ndrawback of current architectures is that they are expensive to train,\ntypically requiring days to weeks of GPU time to converge. This makes\nexhaustive hyperparameter search, as is commonly done with other neural network\narchitectures, prohibitively expensive. In this work, we present the first\nlarge-scale analysis of NMT architecture hyperparameters. We report empirical\nresults and variance numbers for several hundred experimental runs,\ncorresponding to over 250,000 GPU hours on the standard WMT English to German\ntranslation task. Our experiments lead to novel insights and practical advice\nfor building and extending NMT architectures. As part of this contribution, we\nrelease an open-source NMT framework that enables researchers to easily\nexperiment with novel techniques and reproduce state of the art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04009v1",
            "date": "2017-03-11 18:20:13+00:00",
            "title": "Automated Hate Speech Detection and the Problem of Offensive Language",
            "authors": [
                "Thomas Davidson",
                "Dana Warmsley",
                "Michael Macy",
                "Ingmar Weber"
            ],
            "abstract": "A key challenge for automatic hate-speech detection on social media is the\nseparation of hate speech from other instances of offensive language. Lexical\ndetection methods tend to have low precision because they classify all messages\ncontaining particular terms as hate speech and previous work using supervised\nlearning has failed to distinguish between the two categories. We used a\ncrowd-sourced hate speech lexicon to collect tweets containing hate speech\nkeywords. We use crowd-sourcing to label a sample of these tweets into three\ncategories: those containing hate speech, only offensive language, and those\nwith neither. We train a multi-class classifier to distinguish between these\ndifferent categories. Close analysis of the predictions and the errors shows\nwhen we can reliably separate hate speech from other offensive language and\nwhen this differentiation is more difficult. We find that racist and homophobic\ntweets are more likely to be classified as hate speech but that sexist tweets\nare generally classified as offensive. Tweets without explicit hate keywords\nare also more difficult to classify.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04178v2",
            "date": "2017-03-12 21:07:54+00:00",
            "title": "Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations",
            "authors": [
                "Jose Camacho-Collados"
            ],
            "abstract": "The study of taxonomies and hypernymy relations has been extensive on the\nNatural Language Processing (NLP) literature. However, the evaluation of\ntaxonomy learning approaches has been traditionally troublesome, as it mainly\nrelies on ad-hoc experiments which are hardly reproducible and manually\nexpensive. Partly because of this, current research has been lately focusing on\nthe hypernymy detection task. In this paper we reflect on this trend, analyzing\nissues related to current evaluation procedures. Finally, we propose three\npotential avenues for future work so that is-a relations and resources based on\nthem play a more important role in downstream NLP applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04213v2",
            "date": "2017-03-13 01:06:19+00:00",
            "title": "MetaPAD: Meta Pattern Discovery from Massive Text Corpora",
            "authors": [
                "Meng Jiang",
                "Jingbo Shang",
                "Taylor Cassidy",
                "Xiang Ren",
                "Lance M. Kaplan",
                "Timothy P. Hanratty",
                "Jiawei Han"
            ],
            "abstract": "Mining textual patterns in news, tweets, papers, and many other kinds of text\ncorpora has been an active theme in text mining and NLP research. Previous\nstudies adopt a dependency parsing-based pattern discovery approach. However,\nthe parsing results lose rich context around entities in the patterns, and the\nprocess is costly for a corpus of large scale. In this study, we propose a\nnovel typed textual pattern structure, called meta pattern, which is extended\nto a frequent, informative, and precise subsequence pattern in certain context.\nWe propose an efficient framework, called MetaPAD, which discovers meta\npatterns from massive corpora with three techniques: (1) it develops a\ncontext-aware segmentation method to carefully determine the boundaries of\npatterns with a learnt pattern quality assessment function, which avoids costly\ndependency parsing and generates high-quality patterns; (2) it identifies and\ngroups synonymous meta patterns from multiple facets---their types, contexts,\nand extractions; and (3) it examines type distributions of entities in the\ninstances extracted by each group of patterns, and looks for appropriate type\nlevels to make discovered patterns precise. Experiments demonstrate that our\nproposed framework discovers high-quality typed textual patterns efficiently\nfrom different genres of massive corpora and facilitates information\nextraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04330v1",
            "date": "2017-03-13 11:03:40+00:00",
            "title": "Story Cloze Ending Selection Baselines and Data Examination",
            "authors": [
                "Todor Mihaylov",
                "Anette Frank"
            ],
            "abstract": "This paper describes two supervised baseline systems for the Story Cloze Test\nShared Task (Mostafazadeh et al., 2016a). We first build a classifier using\nfeatures based on word embeddings and semantic similarity computation. We\nfurther implement a neural LSTM system with different encoding strategies that\ntry to model the relation between the story and the provided endings. Our\nexperiments show that a model using representation features based on average\nword embedding vectors over the given story words and the candidate ending\nsentences words, joint with similarity features between the story and candidate\nending representations performed better than the neural models. Our best model\nachieves an accuracy of 72.42, ranking 3rd in the official evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04357v1",
            "date": "2017-03-13 12:28:03+00:00",
            "title": "Nematus: a Toolkit for Neural Machine Translation",
            "authors": [
                "Rico Sennrich",
                "Orhan Firat",
                "Kyunghyun Cho",
                "Alexandra Birch",
                "Barry Haddow",
                "Julian Hitschler",
                "Marcin Junczys-Dowmunt",
                "Samuel L\u00e4ubli",
                "Antonio Valerio Miceli Barone",
                "Jozef Mokry",
                "Maria N\u0103dejde"
            ],
            "abstract": "We present Nematus, a toolkit for Neural Machine Translation. The toolkit\nprioritizes high translation accuracy, usability, and extensibility. Nematus\nhas been used to build top-performing submissions to shared translation tasks\nat WMT and IWSLT, and has been used to train systems for production\nenvironments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04474v1",
            "date": "2017-03-13 16:36:38+00:00",
            "title": "DRAGNN: A Transition-based Framework for Dynamically Connected Neural Networks",
            "authors": [
                "Lingpeng Kong",
                "Chris Alberti",
                "Daniel Andor",
                "Ivan Bogatyy",
                "David Weiss"
            ],
            "abstract": "In this work, we present a compact, modular framework for constructing novel\nrecurrent neural architectures. Our basic module is a new generic unit, the\nTransition Based Recurrent Unit (TBRU). In addition to hidden layer\nactivations, TBRUs have discrete state dynamics that allow network connections\nto be built dynamically as a function of intermediate activations. By\nconnecting multiple TBRUs, we can extend and combine commonly used\narchitectures such as sequence-to-sequence, attention mechanisms, and\nre-cursive tree-structured models. A TBRU can also serve as both an encoder for\ndownstream tasks and as a decoder for its own task simultaneously, resulting in\nmore accurate multi-task learning. We call our approach Dynamic Recurrent\nAcyclic Graphical Neural Networks, or DRAGNN. We show that DRAGNN is\nsignificantly more accurate and efficient than seq2seq with attention for\nsyntactic dependency parsing and yields more accurate multi-task learning for\nextractive summarization tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04481v1",
            "date": "2017-03-13 16:50:36+00:00",
            "title": "Geometrical morphology",
            "authors": [
                "John Goldsmith",
                "Eric Rosen"
            ],
            "abstract": "We explore inflectional morphology as an example of the relationship of the\ndiscrete and the continuous in linguistics. The grammar requests a form of a\nlexeme by specifying a set of feature values, which corresponds to a corner M\nof a hypercube in feature value space. The morphology responds to that request\nby providing a morpheme, or a set of morphemes, whose vector sum is\ngeometrically closest to the corner M. In short, the chosen morpheme $\\mu$ is\nthe morpheme (or set of morphemes) that maximizes the inner product of $\\mu$\nand M.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04617v2",
            "date": "2017-03-14 17:43:25+00:00",
            "title": "Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering",
            "authors": [
                "Junbei Zhang",
                "Xiaodan Zhu",
                "Qian Chen",
                "Lirong Dai",
                "Si Wei",
                "Hui Jiang"
            ],
            "abstract": "The last several years have seen intensive interest in exploring\nneural-network-based models for machine comprehension (MC) and question\nanswering (QA). In this paper, we approach the problems by closely modelling\nquestions in a neural network framework. We first introduce syntactic\ninformation to help encode questions. We then view and model different types of\nquestions and the information shared among them as an adaptation task and\nproposed adaptation models for them. On the Stanford Question Answering Dataset\n(SQuAD), we show that these approaches can help attain better results over a\ncompetitive baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04650v3",
            "date": "2017-03-14 18:26:12+00:00",
            "title": "Joint Learning of Correlated Sequence Labelling Tasks Using Bidirectional Recurrent Neural Networks",
            "authors": [
                "Vardaan Pahuja",
                "Anirban Laha",
                "Shachar Mirkin",
                "Vikas Raykar",
                "Lili Kotlerman",
                "Guy Lev"
            ],
            "abstract": "The stream of words produced by Automatic Speech Recognition (ASR) systems is\ntypically devoid of punctuations and formatting. Most natural language\nprocessing applications expect segmented and well-formatted texts as input,\nwhich is not available in ASR output. This paper proposes a novel technique of\njointly modeling multiple correlated tasks such as punctuation and\ncapitalization using bidirectional recurrent neural networks, which leads to\nimproved performance for each of these tasks. This method could be extended for\njoint modeling of any other correlated sequence labeling tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04718v1",
            "date": "2017-03-11 07:37:37+00:00",
            "title": "Extending Automatic Discourse Segmentation for Texts in Spanish to Catalan",
            "authors": [
                "Iria da Cunha",
                "Eric SanJuan",
                "Juan-Manuel Torres-Moreno",
                "Irene Castell\u00f3n"
            ],
            "abstract": "At present, automatic discourse analysis is a relevant research topic in the\nfield of NLP. However, discourse is one of the phenomena most difficult to\nprocess. Although discourse parsers have been already developed for several\nlanguages, this tool does not exist for Catalan. In order to implement this\nkind of parser, the first step is to develop a discourse segmenter. In this\narticle we present the first discourse segmenter for texts in Catalan. This\nsegmenter is based on Rhetorical Structure Theory (RST) for Spanish, and uses\nlexical and syntactic information to translate rules valid for Spanish into\nrules for Catalan. We have evaluated the system by using a gold standard corpus\nincluding manually segmented texts and results are promising.",
            "journal ref": "Proceedings of the First Workshop on Modeling, Learning and Mining\n  for Cross/Multilinguality (MultiLingMine 2016), 38th European Conference on\n  Information Retrieval (ECIR 2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04879v1",
            "date": "2017-03-15 01:54:52+00:00",
            "title": "Sparse Named Entity Classification using Factorization Machines",
            "authors": [
                "Ai Hirata",
                "Mamoru Komachi"
            ],
            "abstract": "Named entity classification is the task of classifying text-based elements\ninto various categories, including places, names, dates, times, and monetary\nvalues. A bottleneck in named entity classification, however, is the data\nproblem of sparseness, because new named entities continually emerge, making it\nrather difficult to maintain a dictionary for named entity classification.\nThus, in this paper, we address the problem of named entity classification\nusing matrix factorization to overcome the problem of feature sparsity.\nExperimental results show that our proposed model, with fewer features and a\nsmaller size, achieves competitive accuracy to state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04887v4",
            "date": "2017-03-15 02:26:25+00:00",
            "title": "Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets",
            "authors": [
                "Zhen Yang",
                "Wei Chen",
                "Feng Wang",
                "Bo Xu"
            ],
            "abstract": "This paper proposes an approach for applying GANs to NMT. We build a\nconditional sequence generative adversarial net which comprises of two\nadversarial sub models, a generator and a discriminator. The generator aims to\ngenerate sentences which are hard to be discriminated from human-translated\nsentences (i.e., the golden target sentences), And the discriminator makes\nefforts to discriminate the machine-generated sentences from human-translated\nones. The two sub models play a mini-max game and achieve the win-win situation\nwhen they reach a Nash Equilibrium. Additionally, the static sentence-level\nBLEU is utilized as the reinforced objective for the generator, which biases\nthe generation towards high BLEU points. During training, both the dynamic\ndiscriminator and the static BLEU objective are employed to evaluate the\ngenerated sentences and feedback the evaluations to guide the learning of the\ngenerator. Experimental results show that the proposed model consistently\noutperforms the traditional RNNSearch and the newly emerged state-of-the-art\nTransformer on English-German and Chinese-English translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04929v1",
            "date": "2017-03-15 04:57:17+00:00",
            "title": "SyntaxNet Models for the CoNLL 2017 Shared Task",
            "authors": [
                "Chris Alberti",
                "Daniel Andor",
                "Ivan Bogatyy",
                "Michael Collins",
                "Dan Gillick",
                "Lingpeng Kong",
                "Terry Koo",
                "Ji Ma",
                "Mark Omernick",
                "Slav Petrov",
                "Chayut Thanapirom",
                "Zora Tung",
                "David Weiss"
            ],
            "abstract": "We describe a baseline dependency parsing system for the CoNLL2017 Shared\nTask. This system, which we call \"ParseySaurus,\" uses the DRAGNN framework\n[Kong et al, 2017] to combine transition-based recurrent parsing and tagging\nwith character-based word representations. On the v1.3 Universal Dependencies\nTreebanks, the new system outpeforms the publicly available, state-of-the-art\n\"Parsey's Cousins\" models by 3.47% absolute Labeled Accuracy Score (LAS) across\n52 treebanks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05122v1",
            "date": "2017-03-15 12:32:34+00:00",
            "title": "Is this word borrowed? An automatic approach to quantify the likeliness of borrowing in social media",
            "authors": [
                "Jasabanta Patro",
                "Bidisha Samanta",
                "Saurabh Singh",
                "Prithwish Mukherjee",
                "Monojit Choudhury",
                "Animesh Mukherjee"
            ],
            "abstract": "Code-mixing or code-switching are the effortless phenomena of natural\nswitching between two or more languages in a single conversation. Use of a\nforeign word in a language; however, does not necessarily mean that the speaker\nis code-switching because often languages borrow lexical items from other\nlanguages. If a word is borrowed, it becomes a part of the lexicon of a\nlanguage; whereas, during code-switching, the speaker is aware that the\nconversation involves foreign words or phrases. Identifying whether a foreign\nword used by a bilingual speaker is due to borrowing or code-switching is a\nfundamental importance to theories of multilingualism, and an essential\nprerequisite towards the development of language and speech technologies for\nmultilingual communities. In this paper, we present a series of novel\ncomputational methods to identify the borrowed likeliness of a word, based on\nthe social media signals. We first propose context based clustering method to\nsample a set of candidate words from the social media data.Next, we propose\nthree novel and similar metrics based on the usage of these words by the users\nin different tweets; these metrics were used to score and rank the candidate\nwords indicating their borrowed likeliness. We compare these rankings with a\nground truth ranking constructed through a human judgment experiment. The\nSpearman's rank correlation between the two rankings (nearly 0.62 for all the\nthree metric variants) is more than double the value (0.26) of the most\ncompetitive existing baseline reported in the literature. Some other striking\nobservations are, (i) the correlation is higher for the ground truth data\nelicited from the younger participants (age less than 30) than that from the\nolder participants, and (ii )those participants who use mixed-language for\ntweeting the least, provide the best signals of borrowing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05423v1",
            "date": "2017-03-15 23:34:20+00:00",
            "title": "End-to-end optimization of goal-driven and visually grounded dialogue systems",
            "authors": [
                "Florian Strub",
                "Harm de Vries",
                "Jeremie Mary",
                "Bilal Piot",
                "Aaron Courville",
                "Olivier Pietquin"
            ],
            "abstract": "End-to-end design of dialogue systems has recently become a popular research\ntopic thanks to powerful tools such as encoder-decoder architectures for\nsequence-to-sequence learning. Yet, most current approaches cast human-machine\ndialogue management as a supervised learning problem, aiming at predicting the\nnext utterance of a participant given the full history of the dialogue. This\nvision is too simplistic to render the intrinsic planning problem inherent to\ndialogue as well as its grounded nature, making the context of a dialogue\nlarger than the sole history. This is why only chit-chat and question answering\ntasks have been addressed so far using end-to-end architectures. In this paper,\nwe introduce a Deep Reinforcement Learning method to optimize visually grounded\ntask-oriented dialogues, based on the policy gradient algorithm. This approach\nis tested on a dataset of 120k dialogues collected through Mechanical Turk and\nprovides encouraging results at solving both the problem of generating natural\ndialogues and the task of discovering a specific object in a complex picture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05465v1",
            "date": "2017-03-16 03:15:22+00:00",
            "title": "Neobility at SemEval-2017 Task 1: An Attention-based Sentence Similarity Model",
            "authors": [
                "Wenli Zhuang",
                "Ernie Chang"
            ],
            "abstract": "This paper describes a neural-network model which performed competitively\n(top 6) at the SemEval 2017 cross-lingual Semantic Textual Similarity (STS)\ntask. Our system employs an attention-based recurrent neural network model that\noptimizes the sentence similarity. In this paper, we describe our participation\nin the multilingual STS task which measures similarity across English, Spanish,\nand Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05916v2",
            "date": "2017-03-17 07:53:03+00:00",
            "title": "Construction of a Japanese Word Similarity Dataset",
            "authors": [
                "Yuya Sakaizawa",
                "Mamoru Komachi"
            ],
            "abstract": "An evaluation of distributed word representation is generally conducted using\na word similarity task and/or a word analogy task. There are many datasets\nreadily available for these tasks in English. However, evaluating distributed\nrepresentation in languages that do not have such resources (e.g., Japanese) is\ndifficult. Therefore, as a first step toward evaluating distributed\nrepresentations in Japanese, we constructed a Japanese word similarity dataset.\nTo the best of our knowledge, our dataset is the first resource that can be\nused to evaluate distributed representations in Japanese. Moreover, our dataset\ncontains various parts of speech and includes rare words in addition to common\nwords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.06501v1",
            "date": "2017-03-19 19:56:25+00:00",
            "title": "M\u00e9todos de Otimiza\u00e7\u00e3o Combinat\u00f3ria Aplicados ao Problema de Compress\u00e3o MultiFrases",
            "authors": [
                "Elvys Linhares Pontes",
                "Thiago Gouveia da Silva",
                "Andr\u00e9a Carneiro Linhares",
                "Juan-Manuel Torres-Moreno",
                "St\u00e9phane Huet"
            ],
            "abstract": "The Internet has led to a dramatic increase in the amount of available\ninformation. In this context, reading and understanding this flow of\ninformation have become costly tasks. In the last years, to assist people to\nunderstand textual data, various Natural Language Processing (NLP) applications\nbased on Combinatorial Optimization have been devised. However, for\nMulti-Sentences Compression (MSC), method which reduces the sentence length\nwithout removing core information, the insertion of optimization methods\nrequires further study to improve the performance of MSC. This article\ndescribes a method for MSC using Combinatorial Optimization and Graph Theory to\ngenerate more informative sentences while maintaining their grammaticality. An\nexperiment led on a corpus of 40 clusters of sentences shows that our system\nhas achieved a very good quality and is better than the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.06541v1",
            "date": "2017-03-19 23:42:28+00:00",
            "title": "Native Language Identification using Stacked Generalization",
            "authors": [
                "Shervin Malmasi",
                "Mark Dras"
            ],
            "abstract": "Ensemble methods using multiple classifiers have proven to be the most\nsuccessful approach for the task of Native Language Identification (NLI),\nachieving the current state of the art. However, a systematic examination of\nensemble methods for NLI has yet to be conducted. Additionally, deeper ensemble\narchitectures such as classifier stacking have not been closely evaluated. We\npresent a set of experiments using three ensemble-based models, testing each\nwith multiple configurations and algorithms. This includes a rigorous\napplication of meta-classification models for NLI, achieving state-of-the-art\nresults on three datasets from different languages. We also present the first\nuse of statistical significance testing for comparing NLI systems, showing that\nour results are significantly better than the previous state of the art. We\nmake available a collection of test set predictions to facilitate future\nstatistical tests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.07090v1",
            "date": "2017-03-21 08:24:50+00:00",
            "title": "Deep LSTM for Large Vocabulary Continuous Speech Recognition",
            "authors": [
                "Xu Tian",
                "Jun Zhang",
                "Zejun Ma",
                "Yi He",
                "Juan Wei",
                "Peihao Wu",
                "Wenchang Situ",
                "Shuai Li",
                "Yang Zhang"
            ],
            "abstract": "Recurrent neural networks (RNNs), especially long short-term memory (LSTM)\nRNNs, are effective network for sequential task like speech recognition. Deeper\nLSTM models perform well on large vocabulary continuous speech recognition,\nbecause of their impressive learning ability. However, it is more difficult to\ntrain a deeper network. We introduce a training framework with layer-wise\ntraining and exponential moving average methods for deeper LSTM models. It is a\ncompetitive framework that LSTM models of more than 7 layers are successfully\ntrained on Shenma voice search data in Mandarin and they outperform the deep\nLSTM models trained by conventional approach. Moreover, in order for online\nstreaming speech recognition applications, the shallow model with low real time\nfactor is distilled from the very deep model. The recognition accuracy have\nlittle loss in the distillation process. Therefore, the model trained with the\nproposed training framework reduces relative 14\\% character error rate,\ncompared to original model which has the similar real-time capability.\nFurthermore, the novel transfer learning strategy with segmental Minimum\nBayes-Risk is also introduced in the framework. The strategy makes it possible\nthat training with only a small part of dataset could outperform full dataset\ntraining from the beginning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.07438v2",
            "date": "2017-03-21 21:36:28+00:00",
            "title": "The NLTK FrameNet API: Designing for Discoverability with a Rich Linguistic Resource",
            "authors": [
                "Nathan Schneider",
                "Chuck Wooters"
            ],
            "abstract": "A new Python API, integrated within the NLTK suite, offers access to the\nFrameNet 1.7 lexical database. The lexicon (structured in terms of frames) as\nwell as annotated sentences can be processed programatically, or browsed with\nhuman-readable displays via the interactive Python prompt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.07476v2",
            "date": "2017-03-22 00:37:33+00:00",
            "title": "Topic Identification for Speech without ASR",
            "authors": [
                "Chunxi Liu",
                "Jan Trmal",
                "Matthew Wiesner",
                "Craig Harman",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Modern topic identification (topic ID) systems for speech use automatic\nspeech recognition (ASR) to produce speech transcripts, and perform supervised\nclassification on such ASR outputs. However, under resource-limited conditions,\nthe manually transcribed speech required to develop standard ASR systems can be\nseverely limited or unavailable. In this paper, we investigate alternative\nunsupervised solutions to obtaining tokenizations of speech in terms of a\nvocabulary of automatically discovered word-like or phoneme-like units, without\ndepending on the supervised training of ASR systems. Moreover, using automatic\nphoneme-like tokenizations, we demonstrate that a convolutional neural network\nbased framework for learning spoken document representations provides\ncompetitive performance compared to a standard bag-of-words representation, as\nevidenced by comprehensive topic ID evaluations on both single-label and\nmulti-label classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.07713v2",
            "date": "2017-03-22 15:42:28+00:00",
            "title": "Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection",
            "authors": [
                "Zhao Meng",
                "Lili Mou",
                "Zhi Jin"
            ],
            "abstract": "Speaker change detection (SCD) is an important task in dialog modeling. Our\npaper addresses the problem of text-based SCD, which differs from existing\naudio-based studies and is useful in various scenarios, for example, processing\ndialog transcripts where speaker identities are missing (e.g., OpenSubtitle),\nand enhancing audio SCD with textual information. We formulate text-based SCD\nas a matching problem of utterances before and after a certain decision point;\nwe propose a hierarchical recurrent neural network (RNN) with static\nsentence-level attention. Experimental results show that neural networks\nconsistently achieve better performance than feature-based approaches, and that\nour attention-based model significantly outperforms non-attention neural\nnetworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08068v1",
            "date": "2017-03-23 13:48:45+00:00",
            "title": "Sequential Recurrent Neural Networks for Language Modeling",
            "authors": [
                "Youssef Oualil",
                "Clayton Greenberg",
                "Mittul Singh",
                "Dietrich Klakow"
            ],
            "abstract": "Feedforward Neural Network (FNN)-based language models estimate the\nprobability of the next word based on the history of the last N words, whereas\nRecurrent Neural Networks (RNN) perform the same task based only on the last\nword and some context information that cycles in the network. This paper\npresents a novel approach, which bridges the gap between these two categories\nof networks. In particular, we propose an architecture which takes advantage of\nthe explicit, sequential enumeration of the word history in FNN structure while\nenhancing each word representation at the projection layer through recurrent\ncontext information that evolves in the network. The context integration is\nperformed using an additional word-dependent weight matrix that is also learned\nduring the training. Extensive experiments conducted on the Penn Treebank (PTB)\nand the Large Text Compression Benchmark (LTCB) corpus showed a significant\nreduction of the perplexity when compared to state-of-the-art feedforward as\nwell as recurrent neural network architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08084v1",
            "date": "2017-03-23 14:20:52+00:00",
            "title": "Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation",
            "authors": [
                "Jean-Benoit Delbrouck",
                "Stephane Dupont"
            ],
            "abstract": "In state-of-the-art Neural Machine Translation, an attention mechanism is\nused during decoding to enhance the translation. At every step, the decoder\nuses this mechanism to focus on different parts of the source sentence to\ngather the most useful information before outputting its target word. Recently,\nthe effectiveness of the attention mechanism has also been explored for\nmultimodal tasks, where it becomes possible to focus both on sentence parts and\nimage regions. Approaches to pool two modalities usually include element-wise\nproduct, sum or concatenation. In this paper, we evaluate the more advanced\nMultimodal Compact Bilinear pooling method, which takes the outer product of\ntwo vectors to combine the attention features for the two modalities. This has\nbeen previously investigated for visual question answering. We try out this\napproach for multimodal image caption translation and show improvements\ncompared to basic combination methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08088v2",
            "date": "2017-03-23 14:37:15+00:00",
            "title": "Rapid-Rate: A Framework for Semi-supervised Real-time Sentiment Trend Detection in Unstructured Big Data",
            "authors": [
                "Vineet John"
            ],
            "abstract": "Commercial establishments like restaurants, service centres and retailers\nhave several sources of customer feedback about products and services, most of\nwhich need not be as structured as rated reviews provided by services like\nYelp, or Amazon, in terms of sentiment conveyed. For instance, Amazon provides\na fine-grained score on a numeric scale for product reviews. Some sources,\nhowever, like social media (Twitter, Facebook), mailing lists (Google Groups)\nand forums (Quora) contain text data that is much more voluminous, but\nunstructured and unlabelled. It might be in the best interests of a business\nestablishment to assess the general sentiment towards their brand on these\nplatforms as well. This text could be pipelined into a system with a built-in\nprediction model, with the objective of generating real-time graphs on opinion\nand sentiment trends. Although such tasks like the one described about have\nbeen explored with respect to document classification problems in the past, the\nimplementation described in this paper, by virtue of learning a continuous\nfunction rather than a discrete one, offers a lot more depth of insight as\ncompared to document classification approaches. This study aims to explore the\nvalidity of such a continuous function predicting model to quantify sentiment\nabout an entity, without the additional overhead of manual labelling, and\ncomputational preprocessing & feature extraction. This research project also\naims to design and implement a re-usable document regression pipeline as a\nframework, Rapid-Rate, that can be used to predict document scores in\nreal-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08244v1",
            "date": "2017-03-23 22:20:45+00:00",
            "title": "TokTrack: A Complete Token Provenance and Change Tracking Dataset for the English Wikipedia",
            "authors": [
                "Fabian Fl\u00f6ck",
                "Kenan Erdogan",
                "Maribel Acosta"
            ],
            "abstract": "We present a dataset that contains every instance of all tokens (~ words)\never written in undeleted, non-redirect English Wikipedia articles until\nOctober 2016, in total 13,545,349,787 instances. Each token is annotated with\n(i) the article revision it was originally created in, and (ii) lists with all\nthe revisions in which the token was ever deleted and (potentially) re-added\nand re-deleted from its article, enabling a complete and straightforward\ntracking of its history. This data would be exceedingly hard to create by an\naverage potential user as it is (i) very expensive to compute and as (ii)\naccurately tracking the history of each token in revisioned documents is a\nnon-trivial task. Adapting a state-of-the-art algorithm, we have produced a\ndataset that allows for a range of analyses and metrics, already popular in\nresearch and going beyond, to be generated on complete-Wikipedia scale;\nensuring quality and allowing researchers to forego expensive text-comparison\ncomputation, which so far has hindered scalable usage. We show how this data\nenables, on token-level, computation of provenance, measuring survival of\ncontent over time, very detailed conflict metrics, and fine-grained\ninteractions of editors like partial reverts, re-additions and other metrics,\nin the process gaining several novel insights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08537v1",
            "date": "2017-03-24 17:55:33+00:00",
            "title": "Crowdsourcing Universal Part-Of-Speech Tags for Code-Switching",
            "authors": [
                "Victor Soto",
                "Julia Hirschberg"
            ],
            "abstract": "Code-switching is the phenomenon by which bilingual speakers switch between\nmultiple languages during communication. The importance of developing language\ntechnologies for codeswitching data is immense, given the large populations\nthat routinely code-switch. High-quality linguistic annotations are extremely\nvaluable for any NLP task, and performance is often limited by the amount of\nhigh-quality labeled data. However, little such data exists for code-switching.\nIn this paper, we describe crowd-sourcing universal part-of-speech tags for the\nMiami Bangor Corpus of Spanish-English code-switched speech. We split the\nannotation task into three subtasks: one in which a subset of tokens are\nlabeled automatically, one in which questions are specifically designed to\ndisambiguate a subset of high frequency words, and a more general cascaded\napproach for the remaining data in which questions are displayed to the worker\nfollowing a decision tree structure. Each subtask is extended and adapted for a\nmultilingual setting and the universal tagset. The quality of the annotation\nprocess is measured using hidden check questions annotated with gold labels.\nThe overall agreement between gold standard labels and the majority vote is\nbetween 0.95 and 0.96 for just three labels and the average recall across\npart-of-speech tags is between 0.87 and 0.99, depending on the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08646v1",
            "date": "2017-03-25 04:25:21+00:00",
            "title": "Simplifying the Bible and Wikipedia Using Statistical Machine Translation",
            "authors": [
                "Yohan Jo"
            ],
            "abstract": "I started this work with the hope of generating a text synthesizer (like a\nmusical synthesizer) that can imitate certain linguistic styles. Most of the\nreport focuses on text simplification using statistical machine translation\n(SMT) techniques. I applied MOSES to a parallel corpus of the Bible (King James\nVersion and Easy-to-Read Version) and that of Wikipedia articles (normal and\nsimplified). I report the importance of the three main components of\nSMT---phrase translation, language model, and recording---by changing their\nweights and comparing the resulting quality of simplified text in terms of\nMETEOR and BLEU. Toward the end of the report will be presented some examples\nof text \"synthesized\" into the King James style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08701v1",
            "date": "2017-03-25 14:56:27+00:00",
            "title": "Morphological Analysis for the Maltese Language: The Challenges of a Hybrid System",
            "authors": [
                "Claudia Borg",
                "Albert Gatt"
            ],
            "abstract": "Maltese is a morphologically rich language with a hybrid morphological system\nwhich features both concatenative and non-concatenative processes. This paper\nanalyses the impact of this hybridity on the performance of machine learning\ntechniques for morphological labelling and clustering. In particular, we\nanalyse a dataset of morphologically related word clusters to evaluate the\ndifference in results for concatenative and nonconcatenative clusters. We also\ndescribe research carried out in morphological labelling, with a particular\nfocus on the verb category. Two evaluations were carried out, one using an\nunseen dataset, and another one using a gold standard dataset which was\nmanually labelled. The gold standard dataset was split into concatenative and\nnon-concatenative to analyse the difference in results between the two\nmorphological systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08748v1",
            "date": "2017-03-26 00:30:38+00:00",
            "title": "LEPOR: An Augmented Machine Translation Evaluation Metric",
            "authors": [
                "Aaron Li-Feng Han"
            ],
            "abstract": "Machine translation (MT) was developed as one of the hottest research topics\nin the natural language processing (NLP) literature. One important issue in MT\nis that how to evaluate the MT system reasonably and tell us whether the\ntranslation system makes an improvement or not. The traditional manual judgment\nmethods are expensive, time-consuming, unrepeatable, and sometimes with low\nagreement. On the other hand, the popular automatic MT evaluation methods have\nsome weaknesses. Firstly, they tend to perform well on the language pairs with\nEnglish as the target language, but weak when English is used as source.\nSecondly, some methods rely on many additional linguistic features to achieve\ngood performance, which makes the metric unable to replicate and apply to other\nlanguage pairs easily. Thirdly, some popular metrics utilize incomprehensive\nfactors, which result in low performance on some practical tasks. In this\nthesis, to address the existing problems, we design novel MT evaluation methods\nand investigate their performances on different languages. Firstly, we design\naugmented factors to yield highly accurate evaluation.Secondly, we design a\ntunable evaluation model where weighting of factors can be optimised according\nto the characteristics of languages. Thirdly, in the enhanced version of our\nmethods, we design concise linguistic feature using POS to show that our\nmethods can yield even higher performance when using some external linguistic\nresources. Finally, we introduce the practical performance of our metrics in\nthe ACL-WMT workshop shared tasks, which show that the proposed methods are\nrobust across different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08864v4",
            "date": "2017-03-26 20:02:44+00:00",
            "title": "Learning Simpler Language Models with the Differential State Framework",
            "authors": [
                "Alexander G. Ororbia II",
                "Tomas Mikolov",
                "David Reitter"
            ],
            "abstract": "Learning useful information across long time lags is a critical and difficult\nproblem for temporal neural models in tasks such as language modeling. Existing\narchitectures that address the issue are often complex and costly to train. The\nDifferential State Framework (DSF) is a simple and high-performing design that\nunifies previously introduced gated neural models. DSF models maintain\nlonger-term memory by learning to interpolate between a fast-changing\ndata-driven representation and a slowly changing, implicitly stable state. This\nrequires hardly any more parameters than a classical, simple recurrent network.\nWithin the DSF framework, a new architecture is presented, the Delta-RNN. In\nlanguage modeling at the word and character levels, the Delta-RNN outperforms\npopular complex architectures, such as the Long Short Term Memory (LSTM) and\nthe Gated Recurrent Unit (GRU), and, when regularized, performs comparably to\nseveral state-of-the-art baselines. At the subword level, the Delta-RNN's\nperformance is comparable to that of complex gated architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08885v1",
            "date": "2017-03-26 23:48:06+00:00",
            "title": "Question Answering from Unstructured Text by Retrieval and Comprehension",
            "authors": [
                "Yusuke Watanabe",
                "Bhuwan Dhingra",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Open domain Question Answering (QA) systems must interact with external\nknowledge sources, such as web pages, to find relevant information. Information\nsources like Wikipedia, however, are not well structured and difficult to\nutilize in comparison with Knowledge Bases (KBs). In this work we present a\ntwo-step approach to question answering from unstructured text, consisting of a\nretrieval step and a comprehension step. For comprehension, we present an RNN\nbased attention model with a novel mixture mechanism for selecting answers from\neither retrieved articles or a fixed vocabulary. For retrieval we introduce a\nhand-crafted model and a neural model for ranking relevant articles. We achieve\nstate-of-the-art performance on W IKI M OVIES dataset, reducing the error by\n40%. Our experimental results further demonstrate the importance of each of the\nintroduced components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09013v1",
            "date": "2017-03-27 11:15:58+00:00",
            "title": "A Sentence Simplification System for Improving Relation Extraction",
            "authors": [
                "Christina Niklaus",
                "Bernhard Bermeitinger",
                "Siegfried Handschuh",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "In this demo paper, we present a text simplification approach that is\ndirected at improving the performance of state-of-the-art Open Relation\nExtraction (RE) systems. As syntactically complex sentences often pose a\nchallenge for current Open RE approaches, we have developed a simplification\nframework that performs a pre-processing step by taking a single sentence as\ninput and using a set of syntactic-based transformation rules to create a\ntextual input that is easier to process for subsequently applied Open RE\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09817v3",
            "date": "2017-03-28 21:47:16+00:00",
            "title": "Learning Similarity Functions for Pronunciation Variations",
            "authors": [
                "Einat Naaman",
                "Yossi Adi",
                "Joseph Keshet"
            ],
            "abstract": "A significant source of errors in Automatic Speech Recognition (ASR) systems\nis due to pronunciation variations which occur in spontaneous and\nconversational speech. Usually ASR systems use a finite lexicon that provides\none or more pronunciations for each word. In this paper, we focus on learning a\nsimilarity function between two pronunciations. The pronunciations can be the\ncanonical and the surface pronunciations of the same word or they can be two\nsurface pronunciations of different words. This task generalizes problems such\nas lexical access (the problem of learning the mapping between words and their\npossible pronunciations), and defining word neighborhoods. It can also be used\nto dynamically increase the size of the pronunciation lexicon, or in predicting\nASR errors. We propose two methods, which are based on recurrent neural\nnetworks, to learn the similarity function. The first is based on binary\nclassification, and the second is based on learning the ranking of the\npronunciations. We demonstrate the efficiency of our approach on the task of\nlexical access using a subset of the Switchboard conversational speech corpus.\nResults suggest that on this task our methods are superior to previous methods\nwhich are based on graphical Bayesian methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09825v1",
            "date": "2017-03-28 22:05:20+00:00",
            "title": "Semi-Supervised Affective Meaning Lexicon Expansion Using Semantic and Distributed Word Representations",
            "authors": [
                "Areej Alhothali",
                "Jesse Hoey"
            ],
            "abstract": "In this paper, we propose an extension to graph-based sentiment lexicon\ninduction methods by incorporating distributed and semantic word\nrepresentations in building the similarity graph to expand a three-dimensional\nsentiment lexicon. We also implemented and evaluated the label propagation\nusing four different word representations and similarity metrics. Our\ncomprehensive evaluation of the four approaches was performed on a single data\nset, demonstrating that all four methods can generate a significant number of\nnew sentiment assignments with high accuracy. The highest correlations\n(tau=0.51) and the lowest error (mean absolute error < 1.1%), obtained by\ncombining both the semantic and the distributional features, outperformed the\ndistributional-based and semantic-based label-propagation models and approached\na supervised algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10065v1",
            "date": "2017-03-29 14:26:13+00:00",
            "title": "Hierarchical Classification for Spoken Arabic Dialect Identification using Prosody: Case of Algerian Dialects",
            "authors": [
                "Soumia Bougrine",
                "Hadda Cherroun",
                "Djelloul Ziadi"
            ],
            "abstract": "In daily communications, Arabs use local dialects which are hard to identify\nautomatically using conventional classification methods. The dialect\nidentification challenging task becomes more complicated when dealing with an\nunder-resourced dialects belonging to a same county/region. In this paper, we\nstart by analyzing statistically Algerian dialects in order to capture their\nspecificities related to prosody information which are extracted at utterance\nlevel after a coarse-grained consonant/vowel segmentation. According to these\nanalysis findings, we propose a Hierarchical classification approach for spoken\nArabic algerian Dialect IDentification (HADID). It takes advantage from the\nfact that dialects have an inherent property of naturally structured into\nhierarchy. Within HADID, a top-down hierarchical classification is applied, in\nwhich we use Deep Neural Networks (DNNs) method to build a local classifier for\nevery parent node into the hierarchy dialect structure. Our framework is\nimplemented and evaluated on Algerian Arabic dialects corpus. Whereas, the\nhierarchy dialect structure is deduced from historic and linguistic knowledges.\nThe results reveal that within {\\HD}, the best classifier is DNNs compared to\nSupport Vector Machine. In addition, compared with a baseline Flat\nclassification system, our HADID gives an improvement of 63.5% in term of\nprecision. Furthermore, overall results evidence the suitability of our\nprosody-based HADID for speaker independent dialect identification while\nrequiring less than 6s test utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10152v1",
            "date": "2017-03-29 17:33:27+00:00",
            "title": "Automatic Argumentative-Zoning Using Word2vec",
            "authors": [
                "Haixia Liu"
            ],
            "abstract": "In comparison with document summarization on the articles from social media\nand newswire, argumentative zoning (AZ) is an important task in scientific\npaper analysis. Traditional methodology to carry on this task relies on feature\nengineering from different levels. In this paper, three models of generating\nsentence vectors for the task of sentence classification were explored and\ncompared. The proposed approach builds sentence representations using learned\nembeddings based on neural network. The learned word embeddings formed a\nfeature space, to which the examined sentence is mapped to. Those features are\ninput into the classifiers for supervised classification. Using\n10-cross-validation scheme, evaluation was conducted on the\nArgumentative-Zoning (AZ) annotated articles. The results showed that simply\naveraging the word vectors in a sentence works better than the paragraph to\nvector algorithm and by integrating specific cuewords into the loss function of\nthe neural network can improve the classification performance. In comparison\nwith the hand-crafted features, the word2vec method won for most of the\ncategories. However, the hand-crafted features showed their strength on\nclassifying some of the categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10186v2",
            "date": "2017-03-29 18:15:06+00:00",
            "title": "Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding",
            "authors": [
                "Will Monroe",
                "Robert X. D. Hawkins",
                "Noah D. Goodman",
                "Christopher Potts"
            ],
            "abstract": "We present a model of pragmatic referring expression interpretation in a\ngrounded communication task (identifying colors from descriptions) that draws\nupon predictions from two recurrent neural network classifiers, a speaker and a\nlistener, unified by a recursive pragmatic reasoning framework. Experiments\nshow that this combined pragmatic model interprets color descriptions more\naccurately than the classifiers from which it is built, and that much of this\nimprovement results from combining the speaker and listener perspectives. We\nobserve that pragmatic reasoning helps primarily in the hardest cases: when the\nmodel must distinguish very similar colors, or when few utterances adequately\nexpress the target color. Our findings make use of a newly-collected corpus of\nhuman utterances in color reference games, which exhibit a variety of pragmatic\nbehaviors. We also show that the embedded speaker model reproduces many of\nthese pragmatic behaviors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10661v1",
            "date": "2017-02-22 07:57:14+00:00",
            "title": "BanglaLekha-Isolated: A Comprehensive Bangla Handwritten Character Dataset",
            "authors": [
                "Mithun Biswas",
                "Rafiqul Islam",
                "Gautam Kumar Shom",
                "Md Shopon",
                "Nabeel Mohammed",
                "Sifat Momen",
                "Md Anowarul Abedin"
            ],
            "abstract": "Bangla handwriting recognition is becoming a very important issue nowadays.\nIt is potentially a very important task specially for Bangla speaking\npopulation of Bangladesh and West Bengal. By keeping that in our mind we are\nintroducing a comprehensive Bangla handwritten character dataset named\nBanglaLekha-Isolated. This dataset contains Bangla handwritten numerals, basic\ncharacters and compound characters. This dataset was collected from multiple\ngeographical location within Bangladesh and includes sample collected from a\nvariety of aged groups. This dataset can also be used for other classification\nproblems i.e: gender, age, district. This is the largest dataset on Bangla\nhandwritten characters yet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10724v2",
            "date": "2017-03-31 01:21:40+00:00",
            "title": "N-gram Language Modeling using Recurrent Neural Network Estimation",
            "authors": [
                "Ciprian Chelba",
                "Mohammad Norouzi",
                "Samy Bengio"
            ],
            "abstract": "We investigate the effective memory depth of RNN models by using them for\n$n$-gram language model (LM) smoothing.\n  Experiments on a small corpus (UPenn Treebank, one million words of training\ndata and 10k vocabulary) have found the LSTM cell with dropout to be the best\nmodel for encoding the $n$-gram state when compared with feed-forward and\nvanilla RNN models. When preserving the sentence independence assumption the\nLSTM $n$-gram matches the LSTM LM performance for $n=9$ and slightly\noutperforms it for $n=13$. When allowing dependencies across sentence\nboundaries, the LSTM $13$-gram almost matches the perplexity of the unlimited\nhistory LSTM LM.\n  LSTM $n$-gram smoothing also has the desirable property of improving with\nincreasing $n$-gram order, unlike the Katz or Kneser-Ney back-off estimators.\nUsing multinomial distributions as targets in training instead of the usual\none-hot target is only slightly beneficial for low $n$-gram orders.\n  Experiments on the One Billion Words benchmark show that the results hold at\nlarger scale: while LSTM smoothing for short $n$-gram contexts does not provide\nsignificant advantages over classic N-gram models, it becomes effective with\nlong contexts ($n > 5$); depending on the task and amount of data it can match\nfully recurrent LSTM models at about $n=13$. This may have implications when\nmodeling short-format text, e.g. voice search/query LMs.\n  Building LSTM $n$-gram LMs may be appealing for some practical situations:\nthe state in a $n$-gram LM can be succinctly represented with $(n-1)*4$ bytes\nstoring the identity of the words in the context and batches of $n$-gram\ncontexts can be processed in parallel. On the downside, the $n$-gram context\nencoding computed by the LSTM is discarded, making the model more expensive\nthan a regular recurrent LSTM LM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10772v1",
            "date": "2017-03-31 07:10:30+00:00",
            "title": "Joining Hands: Exploiting Monolingual Treebanks for Parsing of Code-mixing Data",
            "authors": [
                "Irshad Ahmad Bhat",
                "Riyaz Ahmad Bhat",
                "Manish Shrivastava",
                "Dipti Misra Sharma"
            ],
            "abstract": "In this paper, we propose efficient and less resource-intensive strategies\nfor parsing of code-mixed data. These strategies are not constrained by\nin-domain annotations, rather they leverage pre-existing monolingual annotated\nresources for training. We show that these methods can produce significantly\nbetter results as compared to an informed baseline. Besides, we also present a\ndata set of 450 Hindi and English code-mixed tweets of Hindi multilingual\nspeakers for evaluation. The data set is manually annotated with Universal\nDependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00045v2",
            "date": "2017-04-28 19:29:54+00:00",
            "title": "Understanding and Detecting Supporting Arguments of Diverse Types",
            "authors": [
                "Xinyu Hua",
                "Lu Wang"
            ],
            "abstract": "We investigate the problem of sentence-level supporting argument detection\nfrom relevant documents for user-specified claims. A dataset containing claims\nand associated citation articles is collected from online debate website\nidebate.org. We then manually label sentence-level supporting arguments from\nthe documents along with their types as study, factual, opinion, or reasoning.\nWe further characterize arguments of different types, and explore whether\nleveraging type information can facilitate the supporting arguments detection\ntask. Experimental results show that LambdaMART (Burges, 2010) ranker that uses\nfeatures informed by argument types yields better performance than the same\nranker trained without type information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00108v1",
            "date": "2017-04-29 01:13:04+00:00",
            "title": "Semi-supervised sequence tagging with bidirectional language models",
            "authors": [
                "Matthew E. Peters",
                "Waleed Ammar",
                "Chandra Bhagavatula",
                "Russell Power"
            ],
            "abstract": "Pre-trained word embeddings learned from unlabeled text have become a\nstandard component of neural network architectures for NLP tasks. However, in\nmost cases, the recurrent network that operates on word-level representations\nto produce context sensitive representations is trained on relatively little\nlabeled data. In this paper, we demonstrate a general semi-supervised approach\nfor adding pre- trained context embeddings from bidirectional language models\nto NLP systems and apply it to sequence labeling tasks. We evaluate our model\non two standard datasets for named entity recognition (NER) and chunking, and\nin both cases achieve state of the art results, surpassing previous systems\nthat use other forms of transfer or joint learning with additional labeled data\nand task specific gazetteers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00251v1",
            "date": "2017-04-29 23:33:13+00:00",
            "title": "Lifelong Learning CRF for Supervised Aspect Extraction",
            "authors": [
                "Lei Shu",
                "Hu Xu",
                "Bing Liu"
            ],
            "abstract": "This paper makes a focused contribution to supervised aspect extraction. It\nshows that if the system has performed aspect extraction from many past domains\nand retained their results as knowledge, Conditional Random Fields (CRF) can\nleverage this knowledge in a lifelong learning manner to extract in a new\ndomain markedly better than the traditional CRF without using this prior\nknowledge. The key innovation is that even after CRF training, the model can\nstill improve its extraction with experiences in its applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00316v4",
            "date": "2017-04-30 13:52:49+00:00",
            "title": "A Conditional Variational Framework for Dialog Generation",
            "authors": [
                "Xiaoyu Shen",
                "Hui Su",
                "Yanran Li",
                "Wenjie Li",
                "Shuzi Niu",
                "Yang Zhao",
                "Akiko Aizawa",
                "Guoping Long"
            ],
            "abstract": "Deep latent variable models have been shown to facilitate the response\ngeneration for open-domain dialog systems. However, these latent variables are\nhighly randomized, leading to uncontrollable generated responses. In this\npaper, we propose a framework allowing conditional response generation based on\nspecific attributes. These attributes can be either manually assigned or\nautomatically detected. Moreover, the dialog states for both speakers are\nmodeled separately in order to reflect personal features. We validate this\nframework on two different scenarios, where the attribute refers to genericness\nand sentiment states respectively. The experiment result testified the\npotential of our model, where meaningful responses can be generated in\naccordance with the specified attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00364v1",
            "date": "2017-04-30 19:18:22+00:00",
            "title": "Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings",
            "authors": [
                "John Wieting",
                "Kevin Gimpel"
            ],
            "abstract": "We consider the problem of learning general-purpose, paraphrastic sentence\nembeddings, revisiting the setting of Wieting et al. (2016b). While they found\nLSTM recurrent networks to underperform word averaging, we present several\ndevelopments that together produce the opposite conclusion. These include\ntraining on sentence pairs rather than phrase pairs, averaging states to\nrepresent sequences, and regularizing aggressively. These improve LSTMs in both\ntransfer learning and supervised settings. We also introduce a new recurrent\narchitecture, the Gated Recurrent Averaging Network, that is inspired by\naveraging and LSTMs while outperforming them both. We analyze our learned\nmodels, finding evidence of preferences for particular parts of speech and\ndependency relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00390v1",
            "date": "2017-05-01 00:36:59+00:00",
            "title": "Duluth at SemEval--2016 Task 14 : Extending Gloss Overlaps to Enrich Semantic Taxonomies",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth systems that participated in Task 14 of\nSemEval 2016, Semantic Taxonomy Enrichment. There were three related systems in\nthe formal evaluation which are discussed here, along with numerous\npost--evaluation runs. All of these systems identified synonyms between WordNet\nand other dictionaries by measuring the gloss overlaps between them. These\nsystems perform better than the random baseline and one post--evaluation\nvariation was within a respectable margin of the median result attained by all\nparticipating systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00403v2",
            "date": "2017-05-01 02:39:00+00:00",
            "title": "Dependency Parsing with Dilated Iterated Graph CNNs",
            "authors": [
                "Emma Strubell",
                "Andrew McCallum"
            ],
            "abstract": "Dependency parses are an effective way to inject linguistic knowledge into\nmany downstream tasks, and many practitioners wish to efficiently parse\nsentences at scale. Recent advances in GPU hardware have enabled neural\nnetworks to achieve significant gains over the previous best models, these\nmodels still fail to leverage GPUs' capability for massive parallelism due to\ntheir requirement of sequential processing of the sentence. In response, we\npropose Dilated Iterated Graph Convolutional Neural Networks (DIG-CNNs) for\ngraph-based dependency parsing, a graph convolutional architecture that allows\nfor efficient end-to-end GPU parsing. In experiments on the English Penn\nTreeBank benchmark, we show that DIG-CNNs perform on par with some of the best\nneural network parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00424v1",
            "date": "2017-05-01 05:58:56+00:00",
            "title": "Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionary",
            "authors": [
                "Meng Fang",
                "Trevor Cohn"
            ],
            "abstract": "Cross-lingual model transfer is a compelling and popular method for\npredicting annotations in a low-resource language, whereby parallel corpora\nprovide a bridge to a high-resource language and its associated annotated\ncorpora. However, parallel data is not readily available for many languages,\nlimiting the applicability of these approaches. We address these drawbacks in\nour framework which takes advantage of cross-lingual word embeddings trained\nsolely on a high coverage bilingual dictionary. We propose a novel neural\nnetwork model for joint training from both sources of data based on\ncross-lingual word embeddings, and show substantial empirical improvements over\nbaseline techniques. We also propose several active learning heuristics, which\nresult in improvements over competitive benchmark methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00440v1",
            "date": "2017-05-01 08:12:15+00:00",
            "title": "Data Augmentation for Low-Resource Neural Machine Translation",
            "authors": [
                "Marzieh Fadaee",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "The quality of a Neural Machine Translation system depends substantially on\nthe availability of sizable parallel corpora. For low-resource language pairs\nthis is not the case, resulting in poor translation quality. Inspired by work\nin computer vision, we propose a novel data augmentation approach that targets\nlow-frequency words by generating new sentence pairs containing rare words in\nnew, synthetically created contexts. Experimental results on simulated\nlow-resource settings show that our method improves translation quality by up\nto 2.9 BLEU points over the baseline and up to 3.2 BLEU over back-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00441v1",
            "date": "2017-05-01 08:16:56+00:00",
            "title": "Learning Topic-Sensitive Word Representations",
            "authors": [
                "Marzieh Fadaee",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "Distributed word representations are widely used for modeling words in NLP\ntasks. Most of the existing models generate one representation per word and do\nnot consider different meanings of a word. We present two approaches to learn\nmultiple topic-sensitive representations per word by using Hierarchical\nDirichlet Process. We observe that by modeling topics and integrating topic\ndistributions for each document we obtain representations that are able to\ndistinguish between different meanings of a given word. Our models yield\nstatistically significant improvements for the lexical substitution task\nindicating that commonly used single word representations, even when combined\nwith contextual information, are insufficient for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00571v1",
            "date": "2017-05-01 15:57:41+00:00",
            "title": "Lancaster A at SemEval-2017 Task 5: Evaluation metrics matter: predicting sentiment from financial news headlines",
            "authors": [
                "Andrew Moore",
                "Paul Rayson"
            ],
            "abstract": "This paper describes our participation in Task 5 track 2 of SemEval 2017 to\npredict the sentiment of financial news headlines for a specific company on a\ncontinuous scale between -1 and 1. We tackled the problem using a number of\napproaches, utilising a Support Vector Regression (SVR) and a Bidirectional\nLong Short-Term Memory (BLSTM). We found an improvement of 4-6% using the LSTM\nmodel over the SVR and came fourth in the track. We report a number of\ndifferent evaluations using a finance specific word embedding model and reflect\non the effects of using different evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00652v1",
            "date": "2017-05-01 18:24:15+00:00",
            "title": "Efficient Natural Language Response Suggestion for Smart Reply",
            "authors": [
                "Matthew Henderson",
                "Rami Al-Rfou",
                "Brian Strope",
                "Yun-hsuan Sung",
                "Laszlo Lukacs",
                "Ruiqi Guo",
                "Sanjiv Kumar",
                "Balint Miklos",
                "Ray Kurzweil"
            ],
            "abstract": "This paper presents a computationally efficient machine-learned method for\nnatural language response suggestion. Feed-forward neural networks using n-gram\nembedding features encode messages into vectors which are optimized to give\nmessage-response pairs a high dot-product value. An optimized search finds\nresponse suggestions. The method is evaluated in a large-scale commercial\ne-mail application, Inbox by Gmail. Compared to a sequence-to-sequence\napproach, the new system achieves the same quality at a small fraction of the\ncomputational requirements and latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00746v2",
            "date": "2017-05-02 00:23:43+00:00",
            "title": "Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems",
            "authors": [
                "Satoshi Akasaki",
                "Nobuhiro Kaji"
            ],
            "abstract": "Recently emerged intelligent assistants on smartphones and home electronics\n(e.g., Siri and Alexa) can be seen as novel hybrids of domain-specific\ntask-oriented spoken dialogue systems and open-domain non-task-oriented ones.\nTo realize such hybrid dialogue systems, this paper investigates determining\nwhether or not a user is going to have a chat with the system. To address the\nlack of benchmark datasets for this task, we construct a new dataset consisting\nof 15; 160 utterances collected from the real log data of a commercial\nintelligent assistant (and will release the dataset to facilitate future\nresearch activity). In addition, we investigate using tweets and Web search\nqueries for handling open-domain user utterances, which characterize the task\nof chat detection. Experiments demonstrated that, while simple supervised\nmethods are effective, the use of the tweets and search queries further\nimproves the F1-score from 86.21 to 87.53.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00753v1",
            "date": "2017-05-02 01:14:06+00:00",
            "title": "A Teacher-Student Framework for Zero-Resource Neural Machine Translation",
            "authors": [
                "Yun Chen",
                "Yang Liu",
                "Yong Cheng",
                "Victor O. K. Li"
            ],
            "abstract": "While end-to-end neural machine translation (NMT) has made remarkable\nprogress recently, it still suffers from the data scarcity problem for\nlow-resource language pairs and domains. In this paper, we propose a method for\nzero-resource NMT by assuming that parallel sentences have close probabilities\nof generating a sentence in a third language. Based on this assumption, our\nmethod is able to train a source-to-target NMT model (\"student\") without\nparallel corpora available, guided by an existing pivot-to-target NMT model\n(\"teacher\") on a source-pivot parallel corpus. Experimental results show that\nthe proposed method significantly improves over a baseline pivot-based model by\n+3.0 BLEU points across various language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01020v1",
            "date": "2017-05-02 15:21:46+00:00",
            "title": "Modeling Source Syntax for Neural Machine Translation",
            "authors": [
                "Junhui Li",
                "Deyi Xiong",
                "Zhaopeng Tu",
                "Muhua Zhu",
                "Min Zhang",
                "Guodong Zhou"
            ],
            "abstract": "Even though a linguistics-free sequence to sequence model in neural machine\ntranslation (NMT) has certain capability of implicitly learning syntactic\ninformation of source sentences, this paper shows that source syntax can be\nexplicitly incorporated into NMT effectively to provide further improvements.\nSpecifically, we linearize parse trees of source sentences to obtain structural\nlabel sequences. On the basis, we propose three different sorts of encoders to\nincorporate source syntax into NMT: 1) Parallel RNN encoder that learns word\nand label annotation vectors parallelly; 2) Hierarchical RNN encoder that\nlearns word and label annotation vectors in a two-level hierarchy; and 3) Mixed\nRNN encoder that stitchingly learns word and label annotation vectors over\nsequences where words and labels are mixed. Experimentation on\nChinese-to-English translation demonstrates that all the three proposed\nsyntactic encoders are able to improve translation accuracy. It is interesting\nto note that the simplest RNN encoder, i.e., Mixed RNN encoder yields the best\nperformance with an significant improvement of 1.4 BLEU points. Moreover, an\nin-depth analysis from several perspectives is provided to reveal how source\nsyntax benefits NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01042v1",
            "date": "2017-05-02 16:06:03+00:00",
            "title": "Entity Linking with people entity on Wikipedia",
            "authors": [
                "Weiqian Yan",
                "Kanchan Khurad"
            ],
            "abstract": "This paper introduces a new model that uses named entity recognition,\ncoreference resolution, and entity linking techniques, to approach the task of\nlinking people entities on Wikipedia people pages to their corresponding\nWikipedia pages if applicable. Our task is different from general and\ntraditional entity linking because we are working in a limited domain, namely,\npeople entities, and we are including pronouns as entities, whereas in the\npast, pronouns were never considered as entities in entity linking. We have\nbuilt 2 models, both outperforms our baseline model significantly. The purpose\nof our project is to build a model that could be use to generate cleaner data\nfor future entity linking tasks. Our contribution include a clean data set\nconsisting of 50Wikipedia people pages, and 2 entity linking models,\nspecifically tuned for this domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01214v2",
            "date": "2017-05-03 01:05:14+00:00",
            "title": "A Hybrid Architecture for Multi-Party Conversational Systems",
            "authors": [
                "Maira Gatti de Bayser",
                "Paulo Cavalin",
                "Renan Souza",
                "Alan Braz",
                "Heloisa Candello",
                "Claudio Pinhanez",
                "Jean-Pierre Briot"
            ],
            "abstract": "Multi-party Conversational Systems are systems with natural language\ninteraction between one or more people or systems. From the moment that an\nutterance is sent to a group, to the moment that it is replied in the group by\na member, several activities must be done by the system: utterance\nunderstanding, information search, reasoning, among others. In this paper we\npresent the challenges of designing and building multi-party conversational\nsystems, the state of the art, our proposed hybrid architecture using both\nrules and machine learning and some insights after implementing and evaluating\none on the finance domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01265v2",
            "date": "2017-05-03 06:33:37+00:00",
            "title": "On the effectiveness of feature set augmentation using clusters of word embeddings",
            "authors": [
                "Georgios Balikas",
                "Ioannis Partalas"
            ],
            "abstract": "Word clusters have been empirically shown to offer important performance\nimprovements on various tasks. Despite their importance, their incorporation in\nthe standard pipeline of feature engineering relies more on a trial-and-error\nprocedure where one evaluates several hyper-parameters, like the number of\nclusters to be used. In order to better understand the role of such features we\nsystematically evaluate their effect on four tasks, those of named entity\nsegmentation and classification as well as, those of five-point sentiment\nclassification and quantification. Our results strongly suggest that cluster\nmembership features improve the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01452v1",
            "date": "2017-05-03 14:39:56+00:00",
            "title": "Chunk-Based Bi-Scale Decoder for Neural Machine Translation",
            "authors": [
                "Hao Zhou",
                "Zhaopeng Tu",
                "Shujian Huang",
                "Xiaohua Liu",
                "Hang Li",
                "Jiajun Chen"
            ],
            "abstract": "In typical neural machine translation~(NMT), the decoder generates a sentence\nword by word, packing all linguistic granularities in the same time-scale of\nRNN. In this paper, we propose a new type of decoder for NMT, which splits the\ndecode state into two parts and updates them in two different time-scales.\nSpecifically, we first predict a chunk time-scale state for phrasal modeling,\non top of which multiple word time-scale states are generated. In this way, the\ntarget sentence is translated hierarchically from chunks to words, with\ninformation in different granularities being leveraged. Experiments show that\nour proposed model significantly improves the translation performance over the\nstate-of-the-art NMT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01684v1",
            "date": "2017-05-04 03:13:03+00:00",
            "title": "Probabilistic Typology: Deep Generative Models of Vowel Inventories",
            "authors": [
                "Ryan Cotterell",
                "Jason Eisner"
            ],
            "abstract": "Linguistic typology studies the range of structures present in human\nlanguage. The main goal of the field is to discover which sets of possible\nphenomena are universal, and which are merely frequent. For example, all\nlanguages have vowels, while most---but not all---languages have an /u/ sound.\nIn this paper we present the first probabilistic treatment of a basic question\nin phonological typology: What makes a natural vowel inventory? We introduce a\nseries of deep stochastic point processes, and contrast them with previous\ncomputational, simulation-based approaches. We provide a comprehensive suite of\nexperiments on over 200 distinct languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01833v1",
            "date": "2017-05-04 13:33:00+00:00",
            "title": "A Finite State and Rule-based Akshara to Prosodeme (A2P) Converter in Hindi",
            "authors": [
                "Somnath Roy"
            ],
            "abstract": "This article describes a software module called Akshara to Prosodeme (A2P)\nconverter in Hindi. It converts an input grapheme into prosedeme (sequence of\nphonemes with the specification of syllable boundaries and prosodic labels).\nThe software is based on two proposed finite state machines\\textemdash one for\nthe syllabification and another for the syllable labeling. In addition to that,\nit also uses a set of nonlinear phonological rules proposed for foot formation\nin Hindi, which encompass solutions to schwa-deletion in simple, compound,\nderived and inflected words. The nonlinear phonological rules are based on\nmetrical phonology with the provision of recursive foot structure. A software\nmodule is implemented in Python. The testing of the software for\nsyllabification, syllable labeling, schwa deletion and prosodic labeling yield\nan accuracy of more than 99% on a lexicon of size 28664 words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01991v1",
            "date": "2017-05-04 19:50:35+00:00",
            "title": "Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU",
            "authors": [
                "Jacob Devlin"
            ],
            "abstract": "Attentional sequence-to-sequence models have become the new standard for\nmachine translation, but one challenge of such models is a significant increase\nin training and decoding cost compared to phrase-based systems. Here, we focus\non efficient decoding, with a goal of achieving accuracy close the\nstate-of-the-art in neural machine translation (NMT), while achieving CPU\ndecoding speed/throughput close to that of a phrasal decoder.\n  We approach this problem from two angles: First, we describe several\ntechniques for speeding up an NMT beam search decoder, which obtain a 4.4x\nspeedup over a very efficient baseline decoder without changing the decoder\noutput. Second, we propose a simple but powerful network architecture which\nuses an RNN (GRU/LSTM) layer at bottom, followed by a series of stacked\nfully-connected layers applied at every timestep. This architecture achieves\nsimilar accuracy to a deep recurrent model, at a small fraction of the training\nand decoding cost. By combining these techniques, our best system achieves a\nvery competitive accuracy of 38.3 BLEU on WMT English-French NewsTest2014,\nwhile decoding at 100 words/sec on single-threaded CPU. We believe this is the\nbest published accuracy/speed trade-off of an NMT system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02012v2",
            "date": "2017-05-04 20:58:06+00:00",
            "title": "Machine Comprehension by Text-to-Text Neural Question Generation",
            "authors": [
                "Xingdi Yuan",
                "Tong Wang",
                "Caglar Gulcehre",
                "Alessandro Sordoni",
                "Philip Bachman",
                "Sandeep Subramanian",
                "Saizheng Zhang",
                "Adam Trischler"
            ],
            "abstract": "We propose a recurrent neural model that generates natural-language questions\nfrom documents, conditioned on answers. We show how to train the model using a\ncombination of supervised and reinforcement learning. After teacher forcing for\nstandard maximum likelihood training, we fine-tune the model using policy\ngradient techniques to maximize several rewards that measure question quality.\nMost notably, one of these rewards is the performance of a question-answering\nsystem. We motivate question generation as a means to improve the performance\nof question answering systems. Our model is trained and evaluated on the recent\nquestion-answering dataset SQuAD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02023v1",
            "date": "2017-05-04 21:13:24+00:00",
            "title": "Senti17 at SemEval-2017 Task 4: Ten Convolutional Neural Network Voters for Tweet Polarity Classification",
            "authors": [
                "Hussam Hamdan"
            ],
            "abstract": "This paper presents Senti17 system which uses ten convolutional neural\nnetworks (ConvNet) to assign a sentiment label to a tweet. The network consists\nof a convolutional layer followed by a fully-connected layer and a Softmax on\ntop. Ten instances of this network are initialized with the same word\nembeddings as inputs but with different initializations for the network\nweights. We combine the results of all instances by selecting the sentiment\nlabel given by the majority of the ten voters. This system is ranked fourth in\nSemEval-2017 Task4 over 38 systems with 67.4%",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02073v2",
            "date": "2017-05-05 03:36:11+00:00",
            "title": "Cross-lingual Distillation for Text Classification",
            "authors": [
                "Ruochen Xu",
                "Yiming Yang"
            ],
            "abstract": "Cross-lingual text classification(CLTC) is the task of classifying documents\nwritten in different languages into the same taxonomy of categories. This paper\npresents a novel approach to CLTC that builds on model distillation, which\nadapts and extends a framework originally proposed for model compression. Using\nsoft probabilistic predictions for the documents in a label-rich language as\nthe (induced) supervisory labels in a parallel corpus of documents, we train\nclassifiers successfully for new languages in which labeled training data are\nnot available. An adversarial feature adaptation technique is also applied\nduring the model training to reduce distribution mismatch. We conducted\nexperiments on two benchmark CLTC datasets, treating English as the source\nlanguage and German, French, Japan and Chinese as the unlabeled target\nlanguages. The proposed approach had the advantageous or comparable performance\nof the other state-of-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02077v1",
            "date": "2017-05-05 03:43:35+00:00",
            "title": "Crowdsourcing Argumentation Structures in Chinese Hotel Reviews",
            "authors": [
                "Mengxue Li",
                "Shiqiang Geng",
                "Yang Gao",
                "Haijing Liu",
                "Hao Wang"
            ],
            "abstract": "Argumentation mining aims at automatically extracting the premises-claim\ndiscourse structures in natural language texts. There is a great demand for\nargumentation corpora for customer reviews. However, due to the controversial\nnature of the argumentation annotation task, there exist very few large-scale\nargumentation corpora for customer reviews. In this work, we novelly use the\ncrowdsourcing technique to collect argumentation annotations in Chinese hotel\nreviews. As the first Chinese argumentation dataset, our corpus includes 4814\nargument component annotations and 411 argument relation annotations, and its\nannotations qualities are comparable to some widely used argumentation corpora\nin other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02131v1",
            "date": "2017-05-05 08:49:14+00:00",
            "title": "Joint RNN Model for Argument Component Boundary Detection",
            "authors": [
                "Minglan Li",
                "Yang Gao",
                "Hui Wen",
                "Yang Du",
                "Haijing Liu",
                "Hao Wang"
            ],
            "abstract": "Argument Component Boundary Detection (ACBD) is an important sub-task in\nargumentation mining; it aims at identifying the word sequences that constitute\nargument components, and is usually considered as the first sub-task in the\nargumentation mining pipeline. Existing ACBD methods heavily depend on\ntask-specific knowledge, and require considerable human efforts on\nfeature-engineering. To tackle these problems, in this work, we formulate ACBD\nas a sequence labeling problem and propose a variety of Recurrent Neural\nNetwork (RNN) based methods, which do not use domain specific or handcrafted\nfeatures beyond the relative position of the sentence in the document. In\nparticular, we propose a novel joint RNN model that can predict whether\nsentences are argumentative or not, and use the predicted results to more\nprecisely detect the argument component boundaries. We evaluate our techniques\non two corpora from two different genres; results suggest that our joint RNN\nmodel obtain the state-of-the-art performance on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02304v1",
            "date": "2017-05-05 17:10:16+00:00",
            "title": "Deep Speaker: an End-to-End Neural Speaker Embedding System",
            "authors": [
                "Chao Li",
                "Xiaokong Ma",
                "Bing Jiang",
                "Xiangang Li",
                "Xuewei Zhang",
                "Xiao Liu",
                "Ying Cao",
                "Ajay Kannan",
                "Zhenyao Zhu"
            ],
            "abstract": "We present Deep Speaker, a neural speaker embedding system that maps\nutterances to a hypersphere where speaker similarity is measured by cosine\nsimilarity. The embeddings generated by Deep Speaker can be used for many\ntasks, including speaker identification, verification, and clustering. We\nexperiment with ResCNN and GRU architectures to extract the acoustic features,\nthen mean pool to produce utterance-level speaker embeddings, and train using\ntriplet loss based on cosine similarity. Experiments on three distinct datasets\nsuggest that Deep Speaker outperforms a DNN-based i-vector baseline. For\nexample, Deep Speaker reduces the verification equal error rate by 50%\n(relatively) and improves the identification accuracy by 60% (relatively) on a\ntext-independent dataset. We also present results that suggest adapting from a\nmodel trained with Mandarin can improve accuracy for English speaker\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02314v1",
            "date": "2017-05-05 17:30:50+00:00",
            "title": "Building Morphological Chains for Agglutinative Languages",
            "authors": [
                "Serkan Ozen",
                "Burcu Can"
            ],
            "abstract": "In this paper, we build morphological chains for agglutinative languages by\nusing a log-linear model for the morphological segmentation task. The model is\nbased on the unsupervised morphological segmentation system called\nMorphoChains. We extend MorphoChains log linear model by expanding the\ncandidate space recursively to cover more split points for agglutinative\nlanguages such as Turkish, whereas in the original model candidates are\ngenerated by considering only binary segmentation of each word. The results\nshow that we improve the state-of-art Turkish scores by 12% having a F-measure\nof 72% and we improve the English scores by 3% having a F-measure of 74%.\nEventually, the system outperforms both MorphoChains and other well-known\nunsupervised morphological segmentation systems. The results indicate that\ncandidate generation plays an important role in such an unsupervised log-linear\nmodel that is learned using contrastive estimation with negative samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02364v5",
            "date": "2017-05-05 18:54:39+00:00",
            "title": "Supervised Learning of Universal Sentence Representations from Natural Language Inference Data",
            "authors": [
                "Alexis Conneau",
                "Douwe Kiela",
                "Holger Schwenk",
                "Loic Barrault",
                "Antoine Bordes"
            ],
            "abstract": "Many modern NLP systems rely on word embeddings, previously trained in an\nunsupervised manner on large corpora, as base features. Efforts to obtain\nembeddings for larger chunks of text, such as sentences, have however not been\nso successful. Several attempts at learning unsupervised representations of\nsentences have not reached satisfactory enough performance to be widely\nadopted. In this paper, we show how universal sentence representations trained\nusing the supervised data of the Stanford Natural Language Inference datasets\ncan consistently outperform unsupervised methods like SkipThought vectors on a\nwide range of transfer tasks. Much like how computer vision uses ImageNet to\nobtain features, which can then be transferred to other tasks, our work tends\nto indicate the suitability of natural language inference for transfer learning\nto other NLP tasks. Our encoder is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02452v1",
            "date": "2017-05-06 06:35:58+00:00",
            "title": "A Generative Model of a Pronunciation Lexicon for Hindi",
            "authors": [
                "Pramod Pandey",
                "Somnath Roy"
            ],
            "abstract": "Voice browser applications in Text-to- Speech (TTS) and Automatic Speech\nRecognition (ASR) systems crucially depend on a pronunciation lexicon. The\npresent paper describes the model of pronunciation lexicon of Hindi developed\nto automatically generate the output forms of Hindi at two levels, the\n<phoneme> and the <PS> (PS, in short for Prosodic Structure). The latter level\ninvolves both syllable-division and stress placement. The paper describes the\ntool developed for generating the two-level outputs of lexica in Hindi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02700v1",
            "date": "2017-05-07 21:16:35+00:00",
            "title": "Generating Memorable Mnemonic Encodings of Numbers",
            "authors": [
                "Vincent Fiorentini",
                "Megan Shao",
                "Julie Medero"
            ],
            "abstract": "The major system is a mnemonic system that can be used to memorize sequences\nof numbers. In this work, we present a method to automatically generate\nsentences that encode a given number. We propose several encoding models and\ncompare the most promising ones in a password memorability study. The results\nof the study show that a model combining part-of-speech sentence templates with\nan $n$-gram language model produces the most memorable password\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02750v1",
            "date": "2017-05-08 05:50:47+00:00",
            "title": "Density Estimation for Geolocation via Convolutional Mixture Density Network",
            "authors": [
                "Hayate Iso",
                "Shoko Wakamiya",
                "Eiji Aramaki"
            ],
            "abstract": "Nowadays, geographic information related to Twitter is crucially important\nfor fine-grained applications. However, the amount of geographic information\navail- able on Twitter is low, which makes the pursuit of many applications\nchallenging. Under such circumstances, estimating the location of a tweet is an\nimportant goal of the study. Unlike most previous studies that estimate the\npre-defined district as the classification task, this study employs a\nprobability distribution to represent richer information of the tweet, not only\nthe location but also its ambiguity. To realize this modeling, we propose the\nconvolutional mixture density network (CMDN), which uses text data to estimate\nthe mixture model parameters. Experimentally obtained results reveal that CMDN\nachieved the highest prediction performance among the method for predicting the\nexact coordinates. It also provides a quantitative representation of the\nlocation ambiguity for each tweet that properly works for extracting the\nreliable location estimations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02798v6",
            "date": "2017-05-08 09:43:05+00:00",
            "title": "Reinforced Mnemonic Reader for Machine Reading Comprehension",
            "authors": [
                "Minghao Hu",
                "Yuxing Peng",
                "Zhen Huang",
                "Xipeng Qiu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we introduce the Reinforced Mnemonic Reader for machine\nreading comprehension tasks, which enhances previous attentive readers in two\naspects. First, a reattention mechanism is proposed to refine current\nattentions by directly accessing to past attentions that are temporally\nmemorized in a multi-round alignment architecture, so as to avoid the problems\nof attention redundancy and attention deficiency. Second, a new optimization\napproach, called dynamic-critical reinforcement learning, is introduced to\nextend the standard supervised method. It always encourages to predict a more\nacceptable answer so as to address the convergence suppression problem occurred\nin traditional reinforcement learning algorithms. Extensive experiments on the\nStanford Question Answering Dataset (SQuAD) show that our model achieves\nstate-of-the-art results. Meanwhile, our model outperforms previous systems by\nover 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02925v1",
            "date": "2017-05-08 15:40:51+00:00",
            "title": "Ontology-Aware Token Embeddings for Prepositional Phrase Attachment",
            "authors": [
                "Pradeep Dasigi",
                "Waleed Ammar",
                "Chris Dyer",
                "Eduard Hovy"
            ],
            "abstract": "Type-level word embeddings use the same set of parameters to represent all\ninstances of a word regardless of its context, ignoring the inherent lexical\nambiguity in language. Instead, we embed semantic concepts (or synsets) as\ndefined in WordNet and represent a word token in a particular context by\nestimating a distribution over relevant semantic concepts. We use the new,\ncontext-sensitive embeddings in a model for predicting prepositional phrase(PP)\nattachments and jointly learn the concept embeddings and model parameters. We\nshow that using context-sensitive embeddings improves the accuracy of the PP\nattachment model by 5.4% absolute points, which amounts to a 34.4% relative\nreduction in errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03122v3",
            "date": "2017-05-08 23:25:30+00:00",
            "title": "Convolutional Sequence to Sequence Learning",
            "authors": [
                "Jonas Gehring",
                "Michael Auli",
                "David Grangier",
                "Denis Yarats",
                "Yann N. Dauphin"
            ],
            "abstract": "The prevalent approach to sequence to sequence learning maps an input\nsequence to a variable length output sequence via recurrent neural networks. We\nintroduce an architecture based entirely on convolutional neural networks.\nCompared to recurrent models, computations over all elements can be fully\nparallelized during training and optimization is easier since the number of\nnon-linearities is fixed and independent of the input length. Our use of gated\nlinear units eases gradient propagation and we equip each decoder layer with a\nseparate attention module. We outperform the accuracy of the deep LSTM setup of\nWu et al. (2016) on both WMT'14 English-German and WMT'14 English-French\ntranslation at an order of magnitude faster speed, both on GPU and CPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03202v2",
            "date": "2017-05-09 06:46:21+00:00",
            "title": "Does William Shakespeare REALLY Write Hamlet? Knowledge Representation Learning with Confidence",
            "authors": [
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Fen Lin",
                "Leyu Lin"
            ],
            "abstract": "Knowledge graphs (KGs), which could provide essential relational information\nbetween entities, have been widely utilized in various knowledge-driven\napplications. Since the overall human knowledge is innumerable that still grows\nexplosively and changes frequently, knowledge construction and update\ninevitably involve automatic mechanisms with less human supervision, which\nusually bring in plenty of noises and conflicts to KGs. However, most\nconventional knowledge representation learning methods assume that all triple\nfacts in existing KGs share the same significance without any noises. To\naddress this problem, we propose a novel confidence-aware knowledge\nrepresentation learning framework (CKRL), which detects possible noises in KGs\nwhile learning knowledge representations with confidence simultaneously.\nSpecifically, we introduce the triple confidence to conventional\ntranslation-based methods for knowledge representation learning. To make triple\nconfidence more flexible and universal, we only utilize the internal structural\ninformation in KGs, and propose three kinds of triple confidences considering\nboth local and global structural information. In experiments, We evaluate our\nmodels on knowledge graph noise detection, knowledge graph completion and\ntriple classification. Experimental results demonstrate that our\nconfidence-aware models achieve significant and consistent improvements on all\ntasks, which confirms the capability of CKRL modeling confidence with\nstructural information in both KG noise detection and knowledge representation\nlearning.",
            "journal ref": "AAAI-2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03247v1",
            "date": "2017-05-09 09:35:22+00:00",
            "title": "A Systematic Review of Hindi Prosody",
            "authors": [
                "Somnath Roy"
            ],
            "abstract": "Prosody describes both form and function of a sentence using the\nsuprasegmental features of speech. Prosody phenomena are explored in the domain\nof higher phonological constituents such as word, phonological phrase and\nintonational phrase. The study of prosody at the word level is called word\nprosody and above word level is called sentence prosody. Word Prosody describes\nstress pattern by comparing the prosodic features of its constituent syllables.\nSentence Prosody involves the study on phrasing pattern and intonatonal pattern\nof a language. The aim of this study is to summarize the existing works on\nHindi prosody carried out in different domain of language and speech\nprocessing. The review is presented in a systematic fashion so that it could be\na useful resource for one who wants to build on the existing works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03261v2",
            "date": "2017-05-09 10:22:48+00:00",
            "title": "Drug-drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers",
            "authors": [
                "Zibo Yi",
                "Shasha Li",
                "Jie Yu",
                "Qingbo Wu"
            ],
            "abstract": "Drug-drug interaction (DDI) is a vital information when physicians and\npharmacists intend to co-administer two or more drugs. Thus, several DDI\ndatabases are constructed to avoid mistakenly combined use. In recent years,\nautomatically extracting DDIs from biomedical text has drawn researchers'\nattention. However, the existing work utilize either complex feature\nengineering or NLP tools, both of which are insufficient for sentence\ncomprehension. Inspired by the deep learning approaches in natural language\nprocessing, we propose a recur- rent neural network model with multiple\nattention layers for DDI classification. We evaluate our model on 2013 SemEval\nDDIExtraction dataset. The experiments show that our model classifies most of\nthe drug pairs into correct DDI categories, which outperforms the existing NLP\nor deep learning methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03389v1",
            "date": "2017-05-09 15:35:25+00:00",
            "title": "Logical Parsing from Natural Language Based on a Neural Translation Model",
            "authors": [
                "Liang Li",
                "Pengyu Li",
                "Yifan Liu",
                "Tao Wan",
                "Zengchang Qin"
            ],
            "abstract": "Semantic parsing has emerged as a significant and powerful paradigm for\nnatural language interface and question answering systems. Traditional methods\nof building a semantic parser rely on high-quality lexicons, hand-crafted\ngrammars and linguistic features which are limited by applied domain or\nrepresentation. In this paper, we propose a general approach to learn from\ndenotations based on Seq2Seq model augmented with attention mechanism. We\nencode input sequence into vectors and use dynamic programming to infer\ncandidate logical forms. We utilize the fact that similar utterances should\nhave similar logical forms to help reduce the searching space. Under our\nlearning policy, the Seq2Seq model can learn mappings gradually with noises.\nCurriculum learning is adopted to make the learning smoother. We test our\nmethod on the arithmetic domain which shows our model can successfully infer\nthe correct logical forms and learn the word meanings, compositionality and\noperation orders simultaneously.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03551v2",
            "date": "2017-05-09 21:35:07+00:00",
            "title": "TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension",
            "authors": [
                "Mandar Joshi",
                "Eunsol Choi",
                "Daniel S. Weld",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present TriviaQA, a challenging reading comprehension dataset containing\nover 650K question-answer-evidence triples. TriviaQA includes 95K\nquestion-answer pairs authored by trivia enthusiasts and independently gathered\nevidence documents, six per question on average, that provide high quality\ndistant supervision for answering the questions. We show that, in comparison to\nother recently introduced large-scale datasets, TriviaQA (1) has relatively\ncomplex, compositional questions, (2) has considerable syntactic and lexical\nvariability between questions and corresponding answer-evidence sentences, and\n(3) requires more cross sentence reasoning to find answers. We also present two\nbaseline algorithms: a feature-based classifier and a state-of-the-art neural\nnetwork, that performs well on SQuAD reading comprehension. Neither approach\ncomes close to human performance (23% and 40% vs. 80%), suggesting that\nTriviaQA is a challenging testbed that is worth significant future study. Data\nand code available at -- http://nlp.cs.washington.edu/triviaqa/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03645v1",
            "date": "2017-05-10 08:05:44+00:00",
            "title": "A Survey of Deep Learning Methods for Relation Extraction",
            "authors": [
                "Shantanu Kumar"
            ],
            "abstract": "Relation Extraction is an important sub-task of Information Extraction which\nhas the potential of employing deep learning (DL) models with the creation of\nlarge datasets using distant supervision. In this review, we compare the\ncontributions and pitfalls of the various DL models that have been used for the\ntask, to help guide the path ahead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03802v1",
            "date": "2017-05-10 14:42:54+00:00",
            "title": "Analysing Data-To-Text Generation Benchmarks",
            "authors": [
                "Laura Perez-Beltrachini",
                "Claire Gardent"
            ],
            "abstract": "Recently, several data-sets associating data to text have been created to\ntrain data-to-text surface realisers. It is unclear however to what extent the\nsurface realisation task exercised by these data-sets is linguistically\nchallenging. Do these data-sets provide enough variety to encourage the\ndevelopment of generic, high-quality data-to-text surface realisers ? In this\npaper, we argue that these data-sets have important drawbacks. We back up our\nclaim using statistics, metrics and manual evaluation. We conclude by eliciting\na set of criteria for the creation of a data-to-text benchmark which could help\nbetter support the development, evaluation and comparison of linguistically\nsophisticated data-to-text surface realisers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03919v1",
            "date": "2017-05-10 18:44:15+00:00",
            "title": "A Minimal Span-Based Neural Constituency Parser",
            "authors": [
                "Mitchell Stern",
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "In this work, we present a minimal neural model for constituency parsing\nbased on independent scoring of labels and spans. We show that this model is\nnot only compatible with classical dynamic programming techniques, but also\nadmits a novel greedy top-down inference algorithm based on recursive\npartitioning of the input. We demonstrate empirically that both prediction\nschemes are competitive with recent work, and when combined with basic\nextensions to the scoring model are capable of achieving state-of-the-art\nsingle-model performance on the Penn Treebank (91.79 F1) and strong performance\non the French Treebank (82.23 F1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03995v1",
            "date": "2017-05-11 02:56:29+00:00",
            "title": "Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix",
            "authors": [
                "Bingfeng Luo",
                "Yansong Feng",
                "Zheng Wang",
                "Zhanxing Zhu",
                "Songfang Huang",
                "Rui Yan",
                "Dongyan Zhao"
            ],
            "abstract": "Distant supervision significantly reduces human efforts in building training\ndata for many classification tasks. While promising, this technique often\nintroduces noise to the generated training data, which can severely affect the\nmodel performance. In this paper, we take a deep look at the application of\ndistant supervision in relation extraction. We show that the dynamic transition\nmatrix can effectively characterize the noise in the training data built by\ndistant supervision. The transition matrix can be effectively trained using a\nnovel curriculum learning based method without any direct supervision about the\nnoise. We thoroughly evaluate our approach under a wide range of extraction\nscenarios. Experimental results show that our approach consistently improves\nthe extraction results and outperforms the state-of-the-art in various\nevaluation scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04003v1",
            "date": "2017-05-11 04:04:33+00:00",
            "title": "Content-based Approach for Vietnamese Spam SMS Filtering",
            "authors": [
                "Thai-Hoang Pham",
                "Phuong Le-Hong"
            ],
            "abstract": "Short Message Service (SMS) spam is a serious problem in Vietnam because of\nthe availability of very cheap pre-paid SMS packages. There are some systems to\ndetect and filter spam messages for English, most of which use machine learning\ntechniques to analyze the content of messages and classify them. For\nVietnamese, there is some research on spam email filtering but none focused on\nSMS. In this work, we propose the first system for filtering Vietnamese spam\nSMS. We first propose an appropriate preprocessing method since existing tools\nfor Vietnamese preprocessing cannot give good accuracy on our dataset. We then\nexperiment with vector representations and classifiers to find the best model\nfor this problem. Our system achieves an accuracy of 94% when labelling spam\nmessages while the misclassification rate of legitimate messages is relatively\nsmall, about only 0.4%. This is an encouraging result compared to that of\nEnglish and can be served as a strong baseline for future development of\nVietnamese SMS spam prevention systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04038v1",
            "date": "2017-05-11 07:08:30+00:00",
            "title": "Building a Semantic Role Labelling System for Vietnamese",
            "authors": [
                "Thai-Hoang Pham",
                "Xuan-Khoai Pham",
                "Phuong Le-Hong"
            ],
            "abstract": "Semantic role labelling (SRL) is a task in natural language processing which\ndetects and classifies the semantic arguments associated with the predicates of\na sentence. It is an important step towards understanding the meaning of a\nnatural language. There exists SRL systems for well-studied languages like\nEnglish, Chinese or Japanese but there is not any such system for the\nVietnamese language. In this paper, we present the first SRL system for\nVietnamese with encouraging accuracy. We first demonstrate that a simple\napplication of SRL techniques developed for English could not give a good\naccuracy for Vietnamese. We then introduce a new algorithm for extracting\ncandidate syntactic constituents, which is much more accurate than the common\nnode-mapping algorithm usually used in the identification step. Finally, in the\nclassification step, in addition to the common linguistic features, we propose\nnovel and useful features for use in SRL. Our SRL system achieves an $F_1$\nscore of 73.53\\% on the Vietnamese PropBank corpus. This system, including\nsoftware and corpus, is available as an open source project and we believe that\nit is a good baseline for the development of future Vietnamese SRL systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04044v3",
            "date": "2017-05-11 07:31:39+00:00",
            "title": "End-to-end Recurrent Neural Network Models for Vietnamese Named Entity Recognition: Word-level vs. Character-level",
            "authors": [
                "Thai-Hoang Pham",
                "Phuong Le-Hong"
            ],
            "abstract": "This paper demonstrates end-to-end neural network architectures for\nVietnamese named entity recognition. Our best model is a combination of\nbidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network\n(CNN), Conditional Random Field (CRF), using pre-trained word embeddings as\ninput, which achieves an F1 score of 88.59% on a standard test set. Our system\nis able to achieve a comparable performance to the first-rank system of the\nVLSP campaign without using any syntactic or hand-crafted features. We also\ngive an extensive empirical study on using common deep learning models for\nVietnamese NER, at both word and character level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04153v1",
            "date": "2017-05-11 13:11:23+00:00",
            "title": "Dynamic Compositional Neural Networks over Tree Structure",
            "authors": [
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Tree-structured neural networks have proven to be effective in learning\nsemantic representations by exploiting syntactic information. In spite of their\nsuccess, most existing models suffer from the underfitting problem: they\nrecursively use the same shared compositional function throughout the whole\ncompositional process and lack expressive power due to inability to capture the\nrichness of compositionality. In this paper, we address this issue by\nintroducing the dynamic compositional neural networks over tree structure\n(DC-TreeNN), in which the compositional function is dynamically generated by a\nmeta network. The role of meta-network is to capture the metaknowledge across\nthe different compositional rules and formulate them. Experimental results on\ntwo typical tasks show the effectiveness of the proposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04253v2",
            "date": "2017-05-11 15:53:00+00:00",
            "title": "Sketching Word Vectors Through Hashing",
            "authors": [
                "Behrang QasemiZadeh",
                "Laura Kallmeyer"
            ],
            "abstract": "We propose a new fast word embedding technique using hash functions. The\nmethod is a derandomization of a new type of random projections: By\ndisregarding the classic constraint used in designing random projections (i.e.,\npreserving pairwise distances in a particular normed space), our solution\nexploits extremely sparse non-negative random projections. Our experiments show\nthat the proposed method can achieve competitive results, comparable to neural\nembedding learning techniques, however, with only a fraction of the\ncomputational complexity of these methods. While the proposed derandomization\nenhances the computational and space complexity of our method, the possibility\nof applying weighting methods such as positive pointwise mutual information\n(PPMI) to our models after their construction (and at a reduced dimensionality)\nimparts a high discriminatory power to the resulting embeddings. Obviously,\nthis method comes with other known benefits of random projection-based\ntechniques such as ease of update.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04304v3",
            "date": "2017-05-11 17:39:35+00:00",
            "title": "A Deep Reinforced Model for Abstractive Summarization",
            "authors": [
                "Romain Paulus",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Attentional, RNN-based encoder-decoder models for abstractive summarization\nhave achieved good performance on short input and output sequences. For longer\ndocuments and summaries however these models often include repetitive and\nincoherent phrases. We introduce a neural network model with a novel\nintra-attention that attends over the input and continuously generated output\nseparately, and a new training method that combines standard supervised word\nprediction and reinforcement learning (RL). Models trained only with supervised\nlearning often exhibit \"exposure bias\" - they assume ground truth is provided\nat each step during training. However, when standard word prediction is\ncombined with the global sequence prediction training of RL the resulting\nsummaries become more readable. We evaluate this model on the CNN/Daily Mail\nand New York Times datasets. Our model obtains a 41.16 ROUGE-1 score on the\nCNN/Daily Mail dataset, an improvement over previous state-of-the-art models.\nHuman evaluation also shows that our model produces higher quality summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04400v1",
            "date": "2017-05-11 23:34:42+00:00",
            "title": "Reducing Bias in Production Speech Models",
            "authors": [
                "Eric Battenberg",
                "Rewon Child",
                "Adam Coates",
                "Christopher Fougner",
                "Yashesh Gaur",
                "Jiaji Huang",
                "Heewoo Jun",
                "Ajay Kannan",
                "Markus Kliegl",
                "Atul Kumar",
                "Hairong Liu",
                "Vinay Rao",
                "Sanjeev Satheesh",
                "David Seetapun",
                "Anuroop Sriram",
                "Zhenyao Zhu"
            ],
            "abstract": "Replacing hand-engineered pipelines with end-to-end deep learning systems has\nenabled strong results in applications like speech and object recognition.\nHowever, the causality and latency constraints of production systems put\nend-to-end speech models back into the underfitting regime and expose biases in\nthe model that we show cannot be overcome by \"scaling up\", i.e., training\nbigger models on more data. In this work we systematically identify and address\nsources of bias, reducing error rates by up to 20% while remaining practical\nfor deployment. We achieve this by utilizing improved neural architectures for\nstreaming inference, solving optimization issues, and employing strategies that\nincrease audio and label modelling versatility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04416v2",
            "date": "2017-05-12 01:26:23+00:00",
            "title": "Evaluating vector-space models of analogy",
            "authors": [
                "Dawn Chen",
                "Joshua C. Peterson",
                "Thomas L. Griffiths"
            ],
            "abstract": "Vector-space representations provide geometric tools for reasoning about the\nsimilarity of a set of objects and their relationships. Recent machine learning\nmethods for deriving vector-space embeddings of words (e.g., word2vec) have\nachieved considerable success in natural language processing. These vector\nspaces have also been shown to exhibit a surprising capacity to capture verbal\nanalogies, with similar results for natural images, giving new life to a\nclassic model of analogies as parallelograms that was first proposed by\ncognitive scientists. We evaluate the parallelogram model of analogy as applied\nto modern word embeddings, providing a detailed analysis of the extent to which\nthis approach captures human relational similarity judgments in a large\nbenchmark dataset. We find that that some semantic relationships are better\ncaptured than others. We then provide evidence for deeper limitations of the\nparallelogram model based on the intrinsic geometric constraints of vector\nspaces, paralleling classic results for first-order similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04434v1",
            "date": "2017-05-12 03:44:34+00:00",
            "title": "Arc-swift: A Novel Transition System for Dependency Parsing",
            "authors": [
                "Peng Qi",
                "Christopher D. Manning"
            ],
            "abstract": "Transition-based dependency parsers often need sequences of local shift and\nreduce operations to produce certain attachments. Correct individual decisions\nhence require global information about the sentence context and mistakes cause\nerror propagation. This paper proposes a novel transition system, arc-swift,\nthat enables direct attachments between tokens farther apart with a single\ntransition. This allows the parser to leverage lexical information more\ndirectly in transition decisions. Hence, arc-swift can achieve significantly\nbetter performance with a very small beam size. Our parsers reduce error by\n3.7--7.6% relative to those using existing transition systems on the Penn\nTreebank dependency parsing task and English Universal Dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04815v1",
            "date": "2017-05-13 12:29:39+00:00",
            "title": "Learning Semantic Correspondences in Technical Documentation",
            "authors": [
                "Kyle Richardson",
                "Jonas Kuhn"
            ],
            "abstract": "We consider the problem of translating high-level textual descriptions to\nformal representations in technical documentation as part of an effort to model\nthe meaning of such documentation. We focus specifically on the problem of\nlearning translational correspondences between text descriptions and grounded\nrepresentations in the target documentation, such as formal representation of\nfunctions or code templates. Our approach exploits the parallel nature of such\ndocumentation, or the tight coupling between high-level text and the low-level\nrepresentations we aim to learn. Data is collected by mining technical\ndocuments for such parallel text-representation pairs, which we use to train a\nsimple semantic parsing model. We report new baseline results on sixteen novel\ndatasets, including the standard library documentation for nine popular\nprogramming languages across seven natural languages, and a small collection of\nUnix utility manuals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04839v3",
            "date": "2017-05-13 14:49:08+00:00",
            "title": "Annotating and Modeling Empathy in Spoken Conversations",
            "authors": [
                "Firoj Alam",
                "Morena Danieli",
                "Giuseppe Riccardi"
            ],
            "abstract": "Empathy, as defined in behavioral sciences, expresses the ability of human\nbeings to recognize, understand and react to emotions, attitudes and beliefs of\nothers. The lack of an operational definition of empathy makes it difficult to\nmeasure it. In this paper, we address two related problems in automatic\naffective behavior analysis: the design of the annotation protocol and the\nautomatic recognition of empathy from spoken conversations. We propose and\nevaluate an annotation scheme for empathy inspired by the modal model of\nemotions. The annotation scheme was evaluated on a corpus of real-life, dyadic\nspoken conversations. In the context of behavioral analysis, we designed an\nautomatic segmentation and classification system for empathy. Given the\ndifferent speech and language levels of representation where empathy may be\ncommunicated, we investigated features derived from the lexical and acoustic\nspaces. The feature development process was designed to support both the fusion\nand automatic selection of relevant features from high dimensional space. The\nautomatic classification system was evaluated on call center conversations\nwhere it showed significantly better performance than the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05039v1",
            "date": "2017-05-14 23:45:49+00:00",
            "title": "Joint Modeling of Content and Discourse Relations in Dialogues",
            "authors": [
                "Kechen Qin",
                "Lu Wang",
                "Joseph Kim"
            ],
            "abstract": "We present a joint modeling approach to identify salient discussion points in\nspoken meetings as well as to label the discourse relations between speaker\nturns. A variation of our model is also discussed when discourse relations are\ntreated as latent variables. Experimental results on two popular meeting\ncorpora show that our joint model can outperform state-of-the-art approaches\nfor both phrase-based content selection and discourse relation prediction\ntasks. We also evaluate our model on predicting the consistency among team\nmembers' understanding of their group decisions. Classifiers trained with\nfeatures constructed from our model achieve significant better predictive\nperformance than the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05040v1",
            "date": "2017-05-15 00:21:03+00:00",
            "title": "Winning on the Merits: The Joint Effects of Content and Style on Debate Outcomes",
            "authors": [
                "Lu Wang",
                "Nick Beauchamp",
                "Sarah Shugars",
                "Kechen Qin"
            ],
            "abstract": "Debate and deliberation play essential roles in politics and government, but\nmost models presume that debates are won mainly via superior style or agenda\ncontrol. Ideally, however, debates would be won on the merits, as a function of\nwhich side has the stronger arguments. We propose a predictive model of debate\nthat estimates the effects of linguistic features and the latent persuasive\nstrengths of different topics, as well as the interactions between the two.\nUsing a dataset of 118 Oxford-style debates, our model's combination of content\n(as latent topics) and style (as linguistic features) allows us to predict\naudience-adjudicated winners with 74% accuracy, significantly outperforming\nlinguistic features alone (66%). Our model finds that winning sides employ\nstronger arguments, and allows us to identify the linguistic features\nassociated with strong or weak arguments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05183v2",
            "date": "2017-05-15 12:29:52+00:00",
            "title": "Representation learning of drug and disease terms for drug repositioning",
            "authors": [
                "Sahil Manchanda",
                "Ashish Anand"
            ],
            "abstract": "Drug repositioning (DR) refers to identification of novel indications for the\napproved drugs. The requirement of huge investment of time as well as money and\nrisk of failure in clinical trials have led to surge in interest in drug\nrepositioning. DR exploits two major aspects associated with drugs and\ndiseases: existence of similarity among drugs and among diseases due to their\nshared involved genes or pathways or common biological effects. Existing\nmethods of identifying drug-disease association majorly rely on the information\navailable in the structured databases only. On the other hand, abundant\ninformation available in form of free texts in biomedical research articles are\nnot being fully exploited. Word-embedding or obtaining vector representation of\nwords from a large corpora of free texts using neural network methods have been\nshown to give significant performance for several natural language processing\ntasks. In this work we propose a novel way of representation learning to obtain\nfeatures of drugs and diseases by combining complementary information available\nin unstructured texts and structured datasets. Next we use matrix completion\napproach on these feature vectors to learn projection matrix between drug and\ndisease vector spaces. The proposed method has shown competitive performance\nwith state-of-the-art methods. Further, the case studies on Alzheimer's and\nHypertension diseases have shown that the predicted associations are matching\nwith the existing knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05414v2",
            "date": "2017-05-15 19:11:12+00:00",
            "title": "Key-Value Retrieval Networks for Task-Oriented Dialogue",
            "authors": [
                "Mihail Eric",
                "Christopher D. Manning"
            ],
            "abstract": "Neural task-oriented dialogue systems often struggle to smoothly interface\nwith a knowledge base. In this work, we seek to address this problem by\nproposing a new neural dialogue agent that is able to effectively sustain\ngrounded, multi-domain discourse through a novel key-value retrieval mechanism.\nThe model is end-to-end differentiable and does not need to explicitly model\ndialogue state or belief trackers. We also release a new dataset of 3,031\ndialogues that are grounded through underlying knowledge bases and span three\ndistinct tasks in the in-car personal assistant space: calendar scheduling,\nweather information retrieval, and point-of-interest navigation. Our\narchitecture is simultaneously trained on data from all domains and\nsignificantly outperforms a competitive rule-based system and other existing\nneural dialogue architectures on the provided domains according to both\nautomatic and human evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05437v1",
            "date": "2017-05-10 10:00:00+00:00",
            "title": "A Biomedical Information Extraction Primer for NLP Researchers",
            "authors": [
                "Surag Nair"
            ],
            "abstract": "Biomedical Information Extraction is an exciting field at the crossroads of\nNatural Language Processing, Biology and Medicine. It encompasses a variety of\ndifferent tasks that require application of state-of-the-art NLP techniques,\nsuch as NER and Relation Extraction. This paper provides an overview of the\nproblems in the field and discusses some of the techniques used for solving\nthem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05940v3",
            "date": "2017-05-16 22:13:45+00:00",
            "title": "Subregular Complexity and Deep Learning",
            "authors": [
                "Enes Avcu",
                "Chihiro Shibata",
                "Jeffrey Heinz"
            ],
            "abstract": "This paper argues that the judicial use of formal language theory and\ngrammatical inference are invaluable tools in understanding how deep neural\nnetworks can and cannot represent and learn long-term dependencies in temporal\nsequences. Learning experiments were conducted with two types of Recurrent\nNeural Networks (RNNs) on six formal languages drawn from the Strictly Local\n(SL) and Strictly Piecewise (SP) classes. The networks were Simple RNNs\n(s-RNNs) and Long Short-Term Memory RNNs (LSTMs) of varying sizes. The SL and\nSP classes are among the simplest in a mathematically well-understood hierarchy\nof subregular classes. They encode local and long-term dependencies,\nrespectively. The grammatical inference algorithm Regular Positive and Negative\nInference (RPNI) provided a baseline. According to earlier research, the LSTM\narchitecture should be capable of learning long-term dependencies and should\noutperform s-RNNs. The results of these experiments challenge this narrative.\nFirst, the LSTMs' performance was generally worse in the SP experiments than in\nthe SL ones. Second, the s-RNNs out-performed the LSTMs on the most complex SP\nexperiment and performed comparably to them on the others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05952v2",
            "date": "2017-05-16 23:09:00+00:00",
            "title": "A Novel Neural Network Model for Joint POS Tagging and Graph-based Dependency Parsing",
            "authors": [
                "Dat Quoc Nguyen",
                "Mark Dras",
                "Mark Johnson"
            ],
            "abstract": "We present a novel neural network model that learns POS tagging and\ngraph-based dependency parsing jointly. Our model uses bidirectional LSTMs to\nlearn feature representations shared for both POS tagging and dependency\nparsing tasks, thus handling the feature-engineering problem. Our extensive\nexperiments, on 19 languages from the Universal Dependencies project, show that\nour model outperforms the state-of-the-art neural network-based\nStack-propagation model for joint POS tagging and transition-based dependency\nparsing, resulting in a new state of the art. Our code is open-source and\navailable together with pre-trained models at:\nhttps://github.com/datquocnguyen/jPTDP",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05992v1",
            "date": "2017-05-17 02:34:27+00:00",
            "title": "Frame Stacking and Retaining for Recurrent Neural Network Acoustic Model",
            "authors": [
                "Xu Tian",
                "Jun Zhang",
                "Zejun Ma",
                "Yi He",
                "Juan Wei"
            ],
            "abstract": "Frame stacking is broadly applied in end-to-end neural network training like\nconnectionist temporal classification (CTC), and it leads to more accurate\nmodels and faster decoding. However, it is not well-suited to conventional\nneural network based on context-dependent state acoustic model, if the decoder\nis unchanged. In this paper, we propose a novel frame retaining method which is\napplied in decoding. The system which combined frame retaining with frame\nstacking could reduces the time consumption of both training and decoding. Long\nshort-term memory (LSTM) recurrent neural networks (RNNs) using it achieve\nalmost linear training speedup and reduces relative 41\\% real time factor\n(RTF). At the same time, recognition performance is no degradation or improves\nsightly on Shenma voice search dataset in Mandarin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06106v2",
            "date": "2017-05-17 11:48:15+00:00",
            "title": "Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models",
            "authors": [
                "Katharina Kann",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present a semi-supervised way of training a character-based\nencoder-decoder recurrent neural network for morphological reinflection, the\ntask of generating one inflected word form from another. This is achieved by\nusing unlabeled tokens or random strings as training data for an autoencoding\ntask, adapting a network for morphological reinflection, and performing\nmulti-task training. We thus use limited labeled data more effectively,\nobtaining up to 9.9% improvement over state-of-the-art baselines for 8\ndifferent languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06353v1",
            "date": "2017-05-17 21:29:08+00:00",
            "title": "Political Footprints: Political Discourse Analysis using Pre-Trained Word Vectors",
            "authors": [
                "Christophe Bruchansky"
            ],
            "abstract": "In this paper, we discuss how machine learning could be used to produce a\nsystematic and more objective political discourse analysis. Political\nfootprints are vector space models (VSMs) applied to political discourse. Each\nof their vectors represents a word, and is produced by training the English\nlexicon on large text corpora. This paper presents a simple implementation of\npolitical footprints, some heuristics on how to use them, and their application\nto four cases: the U.N. Kyoto Protocol and Paris Agreement, and two U.S.\npresidential elections. The reader will be offered a number of reasons to\nbelieve that political footprints produce meaningful results, along with some\nsuggestions on how to improve their implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06369v3",
            "date": "2017-05-17 23:32:12+00:00",
            "title": "Decoding Sentiment from Distributed Representations of Sentences",
            "authors": [
                "Edoardo Maria Ponti",
                "Ivan Vuli\u0107",
                "Anna Korhonen"
            ],
            "abstract": "Distributed representations of sentences have been developed recently to\nrepresent their meaning as real-valued vectors. However, it is not clear how\nmuch information such representations retain about the polarity of sentences.\nTo study this question, we decode sentiment from unsupervised sentence\nrepresentations learned with different architectures (sensitive to the order of\nwords, the order of sentences, or none) in 9 typologically diverse languages.\nSentiment results from the (recursive) composition of lexical items and\ngrammatical strategies such as negation and concession. The results are\nmanifold: we show that there is no `one-size-fits-all' representation\narchitecture outperforming the others across the board. Rather, the top-ranking\narchitectures depend on the language and data at hand. Moreover, we find that\nin several cases the additive composition model based on skip-gram word vectors\nmay surpass supervised state-of-art architectures such as bidirectional LSTMs.\nFinally, we provide a possible explanation of the observed variation based on\nthe type of negative constructions in each language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06457v1",
            "date": "2017-05-18 08:16:20+00:00",
            "title": "Information Density as a Factor for Variation in the Embedding of Relative Clauses",
            "authors": [
                "Augustin Speyer",
                "Robin Lemke"
            ],
            "abstract": "In German, relative clauses can be positioned in-situ or extraposed. A\npotential factor for the variation might be information density. In this study,\nthis hypothesis is tested with a corpus of 17th century German funeral sermons.\nFor each referent in the relative clauses and their matrix clauses, the\nattention state was determined (first calculation). In a second calculation,\nfor each word the surprisal values were determined, using a bi-gram language\nmodel. In a third calculation, the surprisal values were accommodated as to\nwhether it is the first occurrence of the word in question or not. All three\ncalculations pointed in the same direction: With in-situ relative clauses, the\nrate of new referents was lower and the average surprisal values were lower,\nespecially the accommodated surprisal values, than with extraposed relative\nclauses. This indicated that in-formation density is a factor governing the\nchoice between in-situ and extraposed relative clauses. The study also sheds\nlight on the intrinsic relation-ship between the information theoretic concept\nof information density and in-formation structural concepts such as givenness\nwhich are used under a more linguistic perspective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06463v1",
            "date": "2017-05-18 08:27:42+00:00",
            "title": "Universal Dependencies Parsing for Colloquial Singaporean English",
            "authors": [
                "Hongmin Wang",
                "Yue Zhang",
                "GuangYong Leonard Chan",
                "Jie Yang",
                "Hai Leong Chieu"
            ],
            "abstract": "Singlish can be interesting to the ACL community both linguistically as a\nmajor creole based on English, and computationally for information extraction\nand sentiment analysis of regional social media. We investigate dependency\nparsing of Singlish by constructing a dependency treebank under the Universal\nDependencies scheme, and then training a neural network model by integrating\nEnglish syntactic knowledge into a state-of-the-art parser trained on the\nSinglish treebank. Results show that English knowledge can lead to 25% relative\nerror reduction, resulting in a parser of 84.47% accuracies. To the best of our\nknowledge, we are the first to use neural stacking to improve cross-lingual\ndependency parsing on low-resource languages. We make both our annotation and\nparser available for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06476v4",
            "date": "2017-05-18 08:54:47+00:00",
            "title": "ParlAI: A Dialog Research Software Platform",
            "authors": [
                "Alexander H. Miller",
                "Will Feng",
                "Adam Fisch",
                "Jiasen Lu",
                "Dhruv Batra",
                "Antoine Bordes",
                "Devi Parikh",
                "Jason Weston"
            ],
            "abstract": "We introduce ParlAI (pronounced \"par-lay\"), an open-source software platform\nfor dialog research implemented in Python, available at http://parl.ai. Its\ngoal is to provide a unified framework for sharing, training and testing of\ndialog models, integration of Amazon Mechanical Turk for data collection, human\nevaluation, and online/reinforcement learning; and a repository of machine\nlearning models for comparing with others' models, and improving upon existing\narchitectures. Over 20 tasks are supported in the first release, including\npopular datasets such as SQuAD, bAbI tasks, MCTest, WikiQA, QACNN, QADailyMail,\nCBT, bAbI Dialog, Ubuntu, OpenSubtitles and VQA. Several models are integrated,\nincluding neural models such as memory networks, seq2seq and attentive LSTMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07008v1",
            "date": "2017-05-19 14:17:31+00:00",
            "title": "A Lightweight Regression Method to Infer Psycholinguistic Properties for Brazilian Portuguese",
            "authors": [
                "Leandro B. dos Santos",
                "Magali S. Duran",
                "Nathan S. Hartmann",
                "Arnaldo Candido Jr.",
                "Gustavo H. Paetzold",
                "Sandra M. Aluisio"
            ],
            "abstract": "Psycholinguistic properties of words have been used in various approaches to\nNatural Language Processing tasks, such as text simplification and readability\nassessment. Most of these properties are subjective, involving costly and\ntime-consuming surveys to be gathered. Recent approaches use the limited\ndatasets of psycholinguistic properties to extend them automatically to large\nlexicons. However, some of the resources used by such approaches are not\navailable to most languages. This study presents a method to infer\npsycholinguistic properties for Brazilian Portuguese (BP) using regressors\nbuilt with a light set of features usually available for less resourced\nlanguages: word length, frequency lists, lexical databases composed of school\ndictionaries and word embedding models. The correlations between the properties\ninferred are close to those obtained by related works. The resulting resource\ncontains 26,874 words in BP annotated with concreteness, age of acquisition,\nimageability and subjective frequency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07371v2",
            "date": "2017-05-21 00:14:07+00:00",
            "title": "Spelling Correction as a Foreign Language",
            "authors": [
                "Yingbo Zhou",
                "Utkarsh Porwal",
                "Roberto Konow"
            ],
            "abstract": "In this paper, we reformulated the spell correction problem as a machine\ntranslation task under the encoder-decoder framework. This reformulation\nenabled us to use a single model for solving the problem that is traditionally\nformulated as learning a language model and an error model. This model employs\nmulti-layer recurrent neural networks as an encoder and a decoder. We\ndemonstrate the effectiveness of this model using an internal dataset, where\nthe training data is automatically obtained from user logs. The model offers\ncompetitive performance as compared to the state of the art methods but does\nnot require any feature engineering nor hand tuning between models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07393v2",
            "date": "2017-05-21 04:42:11+00:00",
            "title": "Recurrent Additive Networks",
            "authors": [
                "Kenton Lee",
                "Omer Levy",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce recurrent additive networks (RANs), a new gated RNN which is\ndistinguished by the use of purely additive latent state updates. At every time\nstep, the new state is computed as a gated component-wise sum of the input and\nthe previous state, without any of the non-linearities commonly used in RNN\ntransition dynamics. We formally show that RAN states are weighted sums of the\ninput vectors, and that the gates only contribute to computing the weights of\nthese sums. Despite this relatively simple functional form, experiments\ndemonstrate that RANs perform on par with LSTMs on benchmark language modeling\nproblems. This result shows that many of the non-linear computations in LSTMs\nand related networks are not essential, at least for the problems we consider,\nand suggests that the gates are doing more of the computational work than\npreviously understood.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07687v2",
            "date": "2017-05-22 12:01:10+00:00",
            "title": "W2VLDA: Almost Unsupervised System for Aspect Based Sentiment Analysis",
            "authors": [
                "Aitor Garc\u00eda-Pablos",
                "Montse Cuadros",
                "German Rigau"
            ],
            "abstract": "With the increase of online customer opinions in specialised websites and\nsocial networks, the necessity of automatic systems to help to organise and\nclassify customer reviews by domain-specific aspect/categories and sentiment\npolarity is more important than ever. Supervised approaches to Aspect Based\nSentiment Analysis obtain good results for the domain/language their are\ntrained on, but having manually labelled data for training supervised systems\nfor all domains and languages are usually very costly and time consuming. In\nthis work we describe W2VLDA, an almost unsupervised system based on topic\nmodelling, that combined with some other unsupervised methods and a minimal\nconfiguration, performs aspect/category classifiation,\naspect-terms/opinion-words separation and sentiment polarity classification for\nany given domain and language. We evaluate the performance of the aspect and\nsentiment classification in the multilingual SemEval 2016 task 5 (ABSA)\ndataset. We show competitive results for several languages (English, Spanish,\nFrench and Dutch) and domains (hotels, restaurants, electronic-devices).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08018v1",
            "date": "2017-05-22 21:53:05+00:00",
            "title": "Use of Knowledge Graph in Rescoring the N-Best List in Automatic Speech Recognition",
            "authors": [
                "Ashwini Jaya Kumar",
                "Camilo Morales",
                "Maria-Esther Vidal",
                "Christoph Schmidt",
                "S\u00f6ren Auer"
            ],
            "abstract": "With the evolution of neural network based methods, automatic speech\nrecognition (ASR) field has been advanced to a level where building an\napplication with speech interface is a reality. In spite of these advances,\nbuilding a real-time speech recogniser faces several problems such as low\nrecognition accuracy, domain constraint, and out-of-vocabulary words. The low\nrecognition accuracy problem is addressed by improving the acoustic model,\nlanguage model, decoder and by rescoring the N-best list at the output of the\ndecoder. We are considering the N-best list rescoring approach to improve the\nrecognition accuracy. Most of the methods in the literature use the\ngrammatical, lexical, syntactic and semantic connection between the words in a\nrecognised sentence as a feature to rescore. In this paper, we have tried to\nsee the semantic relatedness between the words in a sentence to rescore the\nN-best list. Semantic relatedness is computed using\nTransE~\\cite{bordes2013translating}, a method for low dimensional embedding of\na triple in a knowledge graph. The novelty of the paper is the application of\nsemantic web to automatic speech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08038v1",
            "date": "2017-05-22 23:13:02+00:00",
            "title": "Latent Human Traits in the Language of Social Media: An Open-Vocabulary Approach",
            "authors": [
                "Vivek Kulkarni",
                "Margaret L. Kern",
                "David Stillwell",
                "Michal Kosinski",
                "Sandra Matz",
                "Lyle Ungar",
                "Steven Skiena",
                "H. Andrew Schwartz"
            ],
            "abstract": "Over the past century, personality theory and research has successfully\nidentified core sets of characteristics that consistently describe and explain\nfundamental differences in the way people think, feel and behave. Such\ncharacteristics were derived through theory, dictionary analyses, and survey\nresearch using explicit self-reports. The availability of social media data\nspanning millions of users now makes it possible to automatically derive\ncharacteristics from language use -- at large scale. Taking advantage of\nlinguistic information available through Facebook, we study the process of\ninferring a new set of potential human traits based on unprompted language use.\nWe subject these new traits to a comprehensive set of evaluations and compare\nthem with a popular five factor model of personality. We find that our\nlanguage-based trait construct is often more generalizable in that it often\npredicts non-questionnaire-based outcomes better than questionnaire-based\ntraits (e.g. entities someone likes, income and intelligence quotient), while\nthe factors remain nearly as stable as traditional factors. Our approach\nsuggests a value in new constructs of personality derived from everyday human\nlanguage use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08091v2",
            "date": "2017-05-23 06:32:36+00:00",
            "title": "Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Recently, encoder-decoder neural networks have shown impressive performance\non many sequence-related tasks. The architecture commonly uses an attentional\nmechanism which allows the model to learn alignments between the source and the\ntarget sequence. Most attentional mechanisms used today is based on a global\nattention property which requires a computation of a weighted summarization of\nthe whole input sequence generated by encoder states. However, it is\ncomputationally expensive and often produces misalignment on the longer input\nsequence. Furthermore, it does not fit with monotonous or left-to-right nature\nin several tasks, such as automatic speech recognition (ASR),\ngrapheme-to-phoneme (G2P), etc. In this paper, we propose a novel attention\nmechanism that has local and monotonic properties. Various ways to control\nthose properties are also explored. Experimental results on ASR, G2P and\nmachine translation between two languages with similar sentence structures,\ndemonstrate that the proposed encoder-decoder model with local monotonic\nattention could achieve significant performance improvements and reduce the\ncomputational complexity in comparison with the one that used the standard\nglobal attention architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08432v2",
            "date": "2017-05-23 17:40:14+00:00",
            "title": "Question-Answering with Grammatically-Interpretable Representations",
            "authors": [
                "Hamid Palangi",
                "Paul Smolensky",
                "Xiaodong He",
                "Li Deng"
            ],
            "abstract": "We introduce an architecture, the Tensor Product Recurrent Network (TPRN). In\nour application of TPRN, internal representations learned by end-to-end\noptimization in a deep neural network performing a textual question-answering\n(QA) task can be interpreted using basic concepts from linguistic theory. No\nperformance penalty need be paid for this increased interpretability: the\nproposed model performs comparably to a state-of-the-art system on the SQuAD QA\ntask. The internal representation which is interpreted is a Tensor Product\nRepresentation: for each input word, the model selects a symbol to encode the\nword, and a role in which to place the symbol, and binds the two together. The\nselection is via soft attention. The overall interpretation is built from\ninterpretations of the symbols, as recruited by the trained model, and\ninterpretations of the roles as used by the model. We find support for our\ninitial hypothesis that symbols can be interpreted as lexical-semantic word\nmeanings, while roles can be interpreted as approximations of grammatical roles\n(or categories) such as subject, wh-word, determiner, etc. Fine-grained\nanalysis reveals specific correspondences between the learned roles and parts\nof speech as assigned by a standard tagger (Toutanova et al. 2003), and finds\nseveral discrepancies in the model's favor. In this sense, the model learns\nsignificant aspects of grammar, after having been exposed solely to\nlinguistically unannotated text, questions, and answers: no prior linguistic\nknowledge is given to the model. What is given is the means to build\nrepresentations using symbols and roles, with an inductive bias favoring use of\nthese in an approximately discrete manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08828v1",
            "date": "2017-05-24 15:50:47+00:00",
            "title": "Deep Investigation of Cross-Language Plagiarism Detection Methods",
            "authors": [
                "Jeremy Ferrero",
                "Laurent Besacier",
                "Didier Schwab",
                "Frederic Agnes"
            ],
            "abstract": "This paper is a deep investigation of cross-language plagiarism detection\nmethods on a new recently introduced open dataset, which contains parallel and\ncomparable collections of documents with multiple characteristics (different\ngenres, languages and sizes of texts). We investigate cross-language plagiarism\ndetection methods for 6 language pairs on 2 granularities of text units in\norder to draw robust conclusions on the best methods while deeply analyzing\ncorrelations across document styles and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08843v2",
            "date": "2017-05-24 16:22:13+00:00",
            "title": "Parsing with CYK over Distributed Representations",
            "authors": [
                "Fabio Massimo Zanzotto",
                "Giordano Cristini",
                "Giorgio Satta"
            ],
            "abstract": "Syntactic parsing is a key task in natural language processing. This task has\nbeen dominated by symbolic, grammar-based parsers. Neural networks, with their\ndistributed representations, are challenging these methods. In this article we\nshow that existing symbolic parsing algorithms can cross the border and be\nentirely formulated over distributed representations. To this end we introduce\na version of the traditional Cocke-Younger-Kasami (CYK) algorithm, called\nD-CYK, which is entirely defined over distributed representations. Our D-CYK\nuses matrix multiplication on real number matrices of size independent of the\nlength of the input string. These operations are compatible with traditional\nneural networks. Experiments show that our D-CYK approximates the original CYK\nalgorithm. By showing that CYK can be entirely performed on distributed\nrepresentations, we open the way to the definition of recurrent layers of\nCYK-informed neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08942v1",
            "date": "2017-05-24 19:44:35+00:00",
            "title": "Joint PoS Tagging and Stemming for Agglutinative Languages",
            "authors": [
                "Necva B\u00f6l\u00fcc\u00fc",
                "Burcu Can"
            ],
            "abstract": "The number of word forms in agglutinative languages is theoretically infinite\nand this variety in word forms introduces sparsity in many natural language\nprocessing tasks. Part-of-speech tagging (PoS tagging) is one of these tasks\nthat often suffers from sparsity. In this paper, we present an unsupervised\nBayesian model using Hidden Markov Models (HMMs) for joint PoS tagging and\nstemming for agglutinative languages. We use stemming to reduce sparsity in PoS\ntagging. Two tasks are jointly performed to provide a mutual benefit in both\ntasks. Our results show that joint POS tagging and stemming improves PoS\ntagging scores. We present results for Turkish and Finnish as agglutinative\nlanguages and English as a morphologically poor language.",
            "journal ref": "CICLING 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08947v2",
            "date": "2017-05-24 19:53:13+00:00",
            "title": "Deep Voice 2: Multi-Speaker Neural Text-to-Speech",
            "authors": [
                "Sercan Arik",
                "Gregory Diamos",
                "Andrew Gibiansky",
                "John Miller",
                "Kainan Peng",
                "Wei Ping",
                "Jonathan Raiman",
                "Yanqi Zhou"
            ],
            "abstract": "We introduce a technique for augmenting neural text-to-speech (TTS) with\nlowdimensional trainable speaker embeddings to generate different voices from a\nsingle model. As a starting point, we show improvements over the two\nstate-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and\nTacotron. We introduce Deep Voice 2, which is based on a similar pipeline with\nDeep Voice 1, but constructed with higher performance building blocks and\ndemonstrates a significant audio quality improvement over Deep Voice 1. We\nimprove Tacotron by introducing a post-processing neural vocoder, and\ndemonstrate a significant audio quality improvement. We then demonstrate our\ntechnique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron\non two multi-speaker TTS datasets. We show that a single neural TTS system can\nlearn hundreds of unique voices from less than half an hour of data per\nspeaker, while achieving high audio quality synthesis and preserving the\nspeaker identities almost perfectly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09054v1",
            "date": "2017-05-25 05:45:42+00:00",
            "title": "Max-Cosine Matching Based Neural Models for Recognizing Textual Entailment",
            "authors": [
                "Zhipeng Xie",
                "Junfeng Hu"
            ],
            "abstract": "Recognizing textual entailment is a fundamental task in a variety of text\nmining or natural language processing applications. This paper proposes a\nsimple neural model for RTE problem. It first matches each word in the\nhypothesis with its most-similar word in the premise, producing an augmented\nrepresentation of the hypothesis conditioned on the premise as a sequence of\nword pairs. The LSTM model is then used to model this augmented sequence, and\nthe final output from the LSTM is fed into a softmax layer to make the\nprediction. Besides the base model, in order to enhance its performance, we\nalso proposed three techniques: the integration of multiple word-embedding\nlibrary, bi-way integration, and ensemble based on model averaging.\nExperimental results on the SNLI dataset have shown that the three techniques\nare effective in boosting the predicative accuracy and that our method\noutperforms several state-of-the-state ones.",
            "journal ref": "DASFAA (1) 2017: 295-308",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09189v1",
            "date": "2017-05-25 14:09:48+00:00",
            "title": "Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs",
            "authors": [
                "Jean Maillard",
                "Stephen Clark",
                "Dani Yogatama"
            ],
            "abstract": "We introduce a neural network that represents sentences by composing their\nwords according to induced binary parse trees. We use Tree-LSTM as our\ncomposition function, applied along a tree structure found by a fully\ndifferentiable natural language chart parser. Our model simultaneously\noptimises both the composition function and the parser, thus eliminating the\nneed for externally-provided parse trees which are normally required for\nTree-LSTM. It can therefore be seen as a tree-based RNN that is unsupervised\nwith respect to the parse trees. As it is fully differentiable, our model is\neasily trained with an off-the-shelf gradient descent method and\nbackpropagation. We demonstrate that it achieves better performance compared to\nvarious supervised Tree-LSTM architectures on a textual entailment task and a\nreverse dictionary task.",
            "journal ref": "Natural Language Engineering 25, no. 4 (2019): 433-49",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09516v1",
            "date": "2017-05-26 10:36:12+00:00",
            "title": "Biomedical Event Trigger Identification Using Bidirectional Recurrent Neural Network Based Models",
            "authors": [
                "Patchigolla V S S Rahul",
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Biomedical events describe complex interactions between various biomedical\nentities. Event trigger is a word or a phrase which typically signifies the\noccurrence of an event. Event trigger identification is an important first step\nin all event extraction methods. However many of the current approaches either\nrely on complex hand-crafted features or consider features only within a\nwindow. In this paper we propose a method that takes the advantage of recurrent\nneural network (RNN) to extract higher level features present across the\nsentence. Thus hidden state representation of RNN along with word and entity\ntype embedding as features avoid relying on the complex hand-crafted features\ngenerated using various NLP toolkits. Our experiments have shown to achieve\nstate-of-art F1-score on Multi Level Event Extraction (MLEE) corpus. We have\nalso performed category-wise analysis of the result and discussed the\nimportance of various features in trigger identification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09585v1",
            "date": "2017-05-26 13:44:54+00:00",
            "title": "Detecting and Explaining Crisis",
            "authors": [
                "Rohan Kshirsagar",
                "Robert Morris",
                "Sam Bowman"
            ],
            "abstract": "Individuals on social media may reveal themselves to be in various states of\ncrisis (e.g. suicide, self-harm, abuse, or eating disorders). Detecting crisis\nfrom social media text automatically and accurately can have profound\nconsequences. However, detecting a general state of crisis without explaining\nwhy has limited applications. An explanation in this context is a coherent,\nconcise subset of the text that rationalizes the crisis detection. We explore\nseveral methods to detect and explain crisis using a combination of neural and\nnon-neural techniques. We evaluate these techniques on a unique data set\nobtained from Koko, an anonymous emotional support network available through\nvarious messaging applications. We annotate a small subset of the samples\nlabeled with crisis with corresponding explanations. Our best technique\nsignificantly outperforms the baseline for detection and explanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09724v1",
            "date": "2017-05-26 21:10:15+00:00",
            "title": "Semi-Supervised Model Training for Unbounded Conversational Speech Recognition",
            "authors": [
                "Shane Walker",
                "Morten Pedersen",
                "Iroro Orife",
                "Jason Flaks"
            ],
            "abstract": "For conversational large-vocabulary continuous speech recognition (LVCSR)\ntasks, up to about two thousand hours of audio is commonly used to train state\nof the art models. Collection of labeled conversational audio however, is\nprohibitively expensive, laborious and error-prone. Furthermore, academic\ncorpora like Fisher English (2004) or Switchboard (1992) are inadequate to\ntrain models with sufficient accuracy in the unbounded space of conversational\nspeech. These corpora are also timeworn due to dated acoustic telephony\nfeatures and the rapid advancement of colloquial vocabulary and idiomatic\nspeech over the last decades. Utilizing the colossal scale of our unlabeled\ntelephony dataset, we propose a technique to construct a modern, high quality\nconversational speech training corpus on the order of hundreds of millions of\nutterances (or tens of thousands of hours) for both acoustic and language model\ntraining. We describe the data collection, selection and training, evaluating\nthe results of our updated speech recognition system on a test corpus of 7K\nmanually transcribed utterances. We show relative word error rate (WER)\nreductions of {35%, 19%} on {agent, caller} utterances over our seed model and\n5% absolute WER improvements over IBM Watson STT on this conversational speech\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09837v2",
            "date": "2017-05-27 16:19:37+00:00",
            "title": "On the relation between dependency distance, crossing dependencies, and parsing. Comment on \"Dependency distance: a new perspective on syntactic patterns in natural languages\" by Haitao Liu et al",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Liu et al. (2017) provide a comprehensive account of research on dependency\ndistance in human languages. While the article is a very rich and useful report\non this complex subject, here I will expand on a few specific issues where\nresearch in computational linguistics (specifically natural language\nprocessing) can inform DDM research, and vice versa. These aspects have not\nbeen explored much in the article by Liu et al. or elsewhere, probably due to\nthe little overlap between both research communities, but they may provide\ninteresting insights for improving our understanding of the evolution of human\nlanguages, the mechanisms by which the brain processes and understands\nlanguage, and the construction of effective computer systems to achieve this\ngoal.",
            "journal ref": "Physics of Life Reviews, 21:200-203, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09899v2",
            "date": "2017-05-28 06:59:07+00:00",
            "title": "Understanding Abuse: A Typology of Abusive Language Detection Subtasks",
            "authors": [
                "Zeerak Waseem",
                "Thomas Davidson",
                "Dana Warmsley",
                "Ingmar Weber"
            ],
            "abstract": "As the body of research on abusive language detection and analysis grows,\nthere is a need for critical consideration of the relationships between\ndifferent subtasks that have been grouped under this label. Based on work on\nhate speech, cyberbullying, and online abuse we propose a typology that\ncaptures central similarities and differences between subtasks and we discuss\nits implications for data annotation and feature construction. We emphasize the\npractical actions that can be taken by researchers to best approach their\nabusive language detection subtask of interest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09906v1",
            "date": "2017-05-28 07:48:14+00:00",
            "title": "Listen, Interact and Talk: Learning to Speak via Interaction",
            "authors": [
                "Haichao Zhang",
                "Haonan Yu",
                "Wei Xu"
            ],
            "abstract": "One of the long-term goals of artificial intelligence is to build an agent\nthat can communicate intelligently with human in natural language. Most\nexisting work on natural language learning relies heavily on training over a\npre-collected dataset with annotated labels, leading to an agent that\nessentially captures the statistics of the fixed external training data. As the\ntraining data is essentially a static snapshot representation of the knowledge\nfrom the annotator, the agent trained this way is limited in adaptiveness and\ngeneralization of its behavior. Moreover, this is very different from the\nlanguage learning process of humans, where language is acquired during\ncommunication by taking speaking action and learning from the consequences of\nspeaking action in an interactive manner. This paper presents an interactive\nsetting for grounded natural language learning, where an agent learns natural\nlanguage by interacting with a teacher and learning from feedback, thus\nlearning and improving language skills while taking part in the conversation.\nTo achieve this goal, we propose a model which incorporates both imitation and\nreinforcement by leveraging jointly sentence and reward feedbacks from the\nteacher. Experiments are conducted to validate the effectiveness of the\nproposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09980v2",
            "date": "2017-05-28 19:41:09+00:00",
            "title": "Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations",
            "authors": [
                "Rik van Noord",
                "Johan Bos"
            ],
            "abstract": "We evaluate the character-level translation method for neural semantic\nparsing on a large corpus of sentences annotated with Abstract Meaning\nRepresentations (AMRs). Using a sequence-to-sequence model, and some trivial\npreprocessing and postprocessing of AMRs, we obtain a baseline accuracy of 53.1\n(F-score on AMR-triples). We examine five different approaches to improve this\nbaseline result: (i) reordering AMR branches to match the word order of the\ninput sentence increases performance to 58.3; (ii) adding part-of-speech tags\n(automatically produced) to the input shows improvement as well (57.2); (iii)\nSo does the introduction of super characters (conflating frequent sequences of\ncharacters to a single character), reaching 57.4; (iv) optimizing the training\nprocess by using pre-training and averaging a set of models increases\nperformance to 58.7; (v) adding silver-standard training data obtained by an\noff-the-shelf parser yields the biggest improvement, resulting in an F-score of\n64.0. Combining all five techniques leads to an F-score of 71.0 on holdout\ndata, which is state-of-the-art in AMR parsing. This is remarkable because of\nthe relative simplicity of the approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09995v1",
            "date": "2017-05-28 22:16:35+00:00",
            "title": "Subject Specific Stream Classification Preprocessing Algorithm for Twitter Data Stream",
            "authors": [
                "Nisansa de Silva",
                "Danaja Maldeniya",
                "Chamilka Wijeratne"
            ],
            "abstract": "Micro-blogging service Twitter is a lucrative source for data mining\napplications on global sentiment. But due to the omnifariousness of the\nsubjects mentioned in each data item; it is inefficient to run a data mining\nalgorithm on the raw data. This paper discusses an algorithm to accurately\nclassify the entire stream in to a given number of mutually exclusive\ncollectively exhaustive streams upon each of which the data mining algorithm\ncan be run separately yielding more relevant results with a high efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10030v1",
            "date": "2017-05-29 03:45:25+00:00",
            "title": "Supervised Complementary Entity Recognition with Augmented Key-value Pairs of Knowledge",
            "authors": [
                "Hu Xu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Extracting opinion targets is an important task in sentiment analysis on\nproduct reviews and complementary entities (products) are one important type of\nopinion targets that may work together with the reviewed product. In this\npaper, we address the problem of Complementary Entity Recognition (CER) as a\nsupervised sequence labeling with the capability of expanding domain knowledge\nas key-value pairs from unlabeled reviews, by automatically learning and\nenhancing knowledge-based features. We use Conditional Random Field (CRF) as\nthe base learner and augment CRF with knowledge-based features (called the\nKnowledge-based CRF or KCRF for short). We conduct experiments to show that\nKCRF effectively improves the performance of supervised CER task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10112v1",
            "date": "2017-05-29 10:51:20+00:00",
            "title": "Dynamics of core of language vocabulary",
            "authors": [
                "Valery D. Solovyev",
                "Vladimir V. Bochkarev",
                "Anna V. Shevlyakova"
            ],
            "abstract": "Studies of the overall structure of vocabulary and its dynamics became\npossible due to creation of diachronic text corpora, especially Google Books\nNgram. This article discusses the question of core change rate and the degree\nto which the core words cover the texts. Different periods of the last three\ncenturies and six main European languages presented in Google Books Ngram are\ncompared. The main result is high stability of core change rate, which is\nanalogous to stability of the Swadesh list.",
            "journal ref": "CEUR Workshop Proceedings 1886, 122-129 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10130v1",
            "date": "2017-05-29 11:37:58+00:00",
            "title": "An Automatic Contextual Analysis and Clustering Classifiers Ensemble approach to Sentiment Analysis",
            "authors": [
                "Murtadha Talib AL-Sharuee",
                "Fei Liu",
                "Mahardhika Pratama"
            ],
            "abstract": "Products reviews are one of the major resources to determine the public\nsentiment. The existing literature on reviews sentiment analysis mainly\nutilizes supervised paradigm, which needs labeled data to be trained on and\nsuffers from domain-dependency. This article addresses these issues by\ndescribes a completely automatic approach for sentiment analysis based on\nunsupervised ensemble learning. The method consists of two phases. The first\nphase is contextual analysis, which has five processes, namely (1) data\npreparation; (2) spelling correction; (3) intensifier handling; (4) negation\nhandling and (5) contrast handling. The second phase comprises the unsupervised\nlearning approach, which is an ensemble of clustering classifiers using a\nmajority voting mechanism with different weight schemes. The base classifier of\nthe ensemble method is a modified k-means algorithm. The base classifier is\nmodified by extracting initial centroids from the feature set via using\nSentWordNet (SWN). We also introduce new sentiment analysis problems of\nAustralian airlines and home builders which offer potential benchmark problems\nin the sentiment analysis field. Our experiments on datasets from different\ndomains show that contextual analysis and the ensemble phases improve the\nclustering performance in term of accuracy, stability and generalization\nability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10272v1",
            "date": "2017-05-29 16:20:21+00:00",
            "title": "Who's to say what's funny? A computer using Language Models and Deep Learning, That's Who!",
            "authors": [
                "Xinru Yan",
                "Ted Pedersen"
            ],
            "abstract": "Humor is a defining characteristic of human beings. Our goal is to develop\nmethods that automatically detect humorous statements and rank them on a\ncontinuous scale. In this paper we report on results using a Language Model\napproach, and outline our plans for using methods from Deep Learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10415v1",
            "date": "2017-05-29 23:34:03+00:00",
            "title": "On the \"Calligraphy\" of Books",
            "authors": [
                "Vanessa Q. Marinho",
                "Henrique F. de Arruda",
                "Thales S. Lima",
                "Luciano F. Costa",
                "Diego R. Amancio"
            ],
            "abstract": "Authorship attribution is a natural language processing task that has been\nwidely studied, often by considering small order statistics. In this paper, we\nexplore a complex network approach to assign the authorship of texts based on\ntheir mesoscopic representation, in an attempt to capture the flow of the\nnarrative. Indeed, as reported in this work, such an approach allowed the\nidentification of the dominant narrative structure of the studied authors. This\nhas been achieved due to the ability of the mesoscopic approach to take into\naccount relationships between different, not necessarily adjacent, parts of the\ntext, which is able to capture the story flow. The potential of the proposed\napproach has been illustrated through principal component analysis, a\ncomparison with the chance baseline method, and network visualization. Such\nvisualizations reveal individual characteristics of the authors, which can be\nunderstood as a kind of calligraphy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10610v4",
            "date": "2017-05-29 08:10:15+00:00",
            "title": "The Importance of Automatic Syntactic Features in Vietnamese Named Entity Recognition",
            "authors": [
                "Thai-Hoang Pham",
                "Phuong Le-Hong"
            ],
            "abstract": "This paper presents a state-of-the-art system for Vietnamese Named Entity\nRecognition (NER). By incorporating automatic syntactic features with word\nembeddings as input for bidirectional Long Short-Term Memory (Bi-LSTM), our\nsystem, although simpler than some deep learning architectures, achieves a much\nbetter result for Vietnamese NER. The proposed method achieves an overall F1\nscore of 92.05% on the test set of an evaluation campaign, organized in late\n2016 by the Vietnamese Language and Speech Processing (VLSP) community. Our\nnamed entity recognition system outperforms the best previous systems for\nVietnamese NER by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10754v1",
            "date": "2017-05-30 17:07:45+00:00",
            "title": "A Low Dimensionality Representation for Language Variety Identification",
            "authors": [
                "Francisco Rangel",
                "Marc Franco-Salvador",
                "Paolo Rosso"
            ],
            "abstract": "Language variety identification aims at labelling texts in a native language\n(e.g. Spanish, Portuguese, English) with its specific variation (e.g.\nArgentina, Chile, Mexico, Peru, Spain; Brazil, Portugal; UK, US). In this work\nwe propose a low dimensionality representation (LDR) to address this task with\nfive different varieties of Spanish: Argentina, Chile, Mexico, Peru and Spain.\nWe compare our LDR method with common state-of-the-art representations and show\nan increase in accuracy of ~35%. Furthermore, we compare LDR with two reference\ndistributed representation models. Experimental results show competitive\nperformance while dramatically reducing the dimensionality --and increasing the\nbig data suitability-- to only 6 features per variety. Additionally, we analyse\nthe behaviour of the employed machine learning algorithms and the most\ndiscriminating features. Finally, we employ an alternative dataset to test the\nrobustness of our low dimensionality representation with another set of similar\nlanguages.",
            "journal ref": "CICLing - Computational Linguistics and Intelligent Text\n  Processing, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10814v1",
            "date": "2017-05-30 18:23:18+00:00",
            "title": "Character Composition Model with Convolutional Neural Networks for Dependency Parsing on Morphologically Rich Languages",
            "authors": [
                "Xiang Yu",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present a transition-based dependency parser that uses a convolutional\nneural network to compose word representations from characters. The character\ncomposition model shows great improvement over the word-lookup model,\nespecially for parsing agglutinative languages. These improvements are even\nbetter than using pre-trained word embeddings from extra data. On the SPMRL\ndata sets, our system outperforms the previous best greedy parser (Ballesteros\net al., 2015) by a margin of 3% on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10900v1",
            "date": "2017-05-31 00:43:04+00:00",
            "title": "Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representations",
            "authors": [
                "Paul Michel",
                "Abhilasha Ravichander",
                "Shruti Rijhwani"
            ],
            "abstract": "We investigate the pertinence of methods from algebraic topology for text\ndata analysis. These methods enable the development of\nmathematically-principled isometric-invariant mappings from a set of vectors to\na document embedding, which is stable with respect to the geometry of the\ndocument in the selected metric space. In this work, we evaluate the utility of\nthese topology-based document representations in traditional NLP tasks,\nspecifically document clustering and sentiment classification. We find that the\nembeddings do not benefit text analysis. In fact, performance is worse than\nsimple techniques like $\\textit{tf-idf}$, indicating that the geometry of the\ndocument does not provide enough variability for classification on the basis of\ntopic or sentiment in the chosen datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10962v1",
            "date": "2017-05-31 07:32:29+00:00",
            "title": "Analysis of the Effect of Dependency Information on Predicate-Argument Structure Analysis and Zero Anaphora Resolution",
            "authors": [
                "Koichiro Yoshino",
                "Shinsuke Mori",
                "Satoshi Nakamura"
            ],
            "abstract": "This paper investigates and analyzes the effect of dependency information on\npredicate-argument structure analysis (PASA) and zero anaphora resolution (ZAR)\nfor Japanese, and shows that a straightforward approach of PASA and ZAR works\neffectively even if dependency information was not available. We constructed an\nanalyzer that directly predicts relationships of predicates and arguments with\ntheir semantic roles from a POS-tagged corpus. The features of the system are\ndesigned to compensate for the absence of syntactic information by using\nfeatures used in dependency parsing as a reference. We also constructed\nanalyzers that use the oracle dependency and the real dependency parsing\nresults, and compared with the system that does not use any syntactic\ninformation to verify that the improvement provided by dependencies is not\ncrucial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.11160v1",
            "date": "2017-05-31 16:05:49+00:00",
            "title": "Learning When to Attend for Neural Machine Translation",
            "authors": [
                "Junhui Li",
                "Muhua Zhu"
            ],
            "abstract": "In the past few years, attention mechanisms have become an indispensable\ncomponent of end-to-end neural machine translation models. However, previous\nattention models always refer to some source words when predicting a target\nword, which contradicts with the fact that some target words have no\ncorresponding source words. Motivated by this observation, we propose a novel\nattention model that has the capability of determining when a decoder should\nattend to source words and when it should not. Experimental results on NIST\nChinese-English translation tasks show that the new model achieves an\nimprovement of 0.8 BLEU score over a state-of-the-art baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.11168v1",
            "date": "2017-05-31 16:31:54+00:00",
            "title": "Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning",
            "authors": [
                "Li Lucy",
                "Jon Gauthier"
            ],
            "abstract": "Distributional word representation methods exploit word co-occurrences to\nbuild compact vector encodings of words. While these representations enjoy\nwidespread use in modern natural language processing, it is unclear whether\nthey accurately encode all necessary facets of conceptual meaning. In this\npaper, we evaluate how well these representations can predict perceptual and\nconceptual features of concrete concepts, drawing on two semantic norm datasets\nsourced from human participants. We find that several standard word\nrepresentations fail to encode many salient perceptual features of concepts,\nand show that these deficits correlate with word-word similarity prediction\nerrors. Our analyses provide motivation for grounded and embodied language\nlearning approaches, which may help to remedy these deficits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00094v1",
            "date": "2017-08-31 22:03:53+00:00",
            "title": "Linguistic Reflexes of Well-Being and Happiness in Echo",
            "authors": [
                "Jiaqi Wu",
                "Marilyn Walker",
                "Pranav Anand",
                "Steve Whittaker"
            ],
            "abstract": "Different theories posit different sources for feelings of well-being and\nhappiness. Appraisal theory grounds our emotional responses in our goals and\ndesires and their fulfillment, or lack of fulfillment. Self Determination\ntheory posits that the basis for well-being rests on our assessment of our\ncompetence, autonomy, and social connection. And surveys that measure happiness\nempirically note that people require their basic needs to be met for food and\nshelter, but beyond that tend to be happiest when socializing, eating or having\nsex. We analyze a corpus of private microblogs from a well-being application\ncalled ECHO, where users label each written post about daily events with a\nhappiness score between 1 and 9. Our goal is to ground the linguistic\ndescriptions of events that users experience in theories of well-being and\nhappiness, and then examine the extent to which different theoretical accounts\ncan explain the variance in the happiness scores. We show that recurrent event\ntypes, such as OBLIGATION and INCOMPETENCE, which affect people's feelings of\nwell-being are not captured in current lexical or semantic resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00224v1",
            "date": "2017-09-01 09:55:44+00:00",
            "title": "Variational Inference for Logical Inference",
            "authors": [
                "Guy Emerson",
                "Ann Copestake"
            ],
            "abstract": "Functional Distributional Semantics is a framework that aims to learn, from\ntext, semantic representations which can be interpreted in terms of truth. Here\nwe make two contributions to this framework. The first is to show how a type of\nlogical inference can be performed by evaluating conditional probabilities. The\nsecond is to make these calculations tractable by means of a variational\napproximation. This approximation also enables faster convergence during\ntraining, allowing us to close the gap with state-of-the-art vector space\nmodels when evaluating on semantic similarity. We demonstrate promising\nperformance on two tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00226v1",
            "date": "2017-09-01 10:01:52+00:00",
            "title": "Semantic Composition via Probabilistic Model Theory",
            "authors": [
                "Guy Emerson",
                "Ann Copestake"
            ],
            "abstract": "Semantic composition remains an open problem for vector space models of\nsemantics. In this paper, we explain how the probabilistic graphical model used\nin the framework of Functional Distributional Semantics can be interpreted as a\nprobabilistic version of model theory. Building on this, we explain how various\nsemantic phenomena can be recast in terms of conditional probabilities in the\ngraphical model. This connection between formal semantics and machine learning\nis helpful in both directions: it gives us an explicit mechanism for modelling\ncontext-dependent meanings (a challenge for formal semantics), and also gives\nus well-motivated techniques for composing distributed representations (a\nchallenge for distributional semantics). We present results on two datasets\nthat go beyond word similarity, showing how these semantically-motivated\ntechniques improve on the performance of vector models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00489v1",
            "date": "2017-09-01 21:38:28+00:00",
            "title": "Arc-Standard Spinal Parsing with Stack-LSTMs",
            "authors": [
                "Miguel Ballesteros",
                "Xavier Carreras"
            ],
            "abstract": "We present a neural transition-based parser for spinal trees, a dependency\nrepresentation of constituent trees. The parser uses Stack-LSTMs that compose\nconstituent nodes with dependency-based derivations. In experiments, we show\nthat this model adapts to different styles of dependency relations, but this\nchoice has little effect for predicting constituent structure, suggesting that\nLSTMs induce useful states by themselves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00616v1",
            "date": "2017-09-02 18:45:48+00:00",
            "title": "Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging",
            "authors": [
                "Hassan Sajjad",
                "Fahim Dalvi",
                "Nadir Durrani",
                "Ahmed Abdelali",
                "Yonatan Belinkov",
                "Stephan Vogel"
            ],
            "abstract": "Word segmentation plays a pivotal role in improving any Arabic NLP\napplication. Therefore, a lot of research has been spent in improving its\naccuracy. Off-the-shelf tools, however, are: i) complicated to use and ii)\ndomain/dialect dependent. We explore three language-independent alternatives to\nmorphological segmentation using: i) data-driven sub-word units, ii) characters\nas a unit of learning, and iii) word embeddings learned using a character CNN\n(Convolution Neural Network). On the tasks of Machine Translation and POS\ntagging, we found these methods to achieve close to, and occasionally surpass\nstate-of-the-art performance. In our analysis, we show that a neural machine\ntranslation system is sensitive to the ratio of source and target tokens, and a\nratio close to 1 or greater, gives optimal performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00659v2",
            "date": "2017-09-03 04:00:49+00:00",
            "title": "Investigating how well contextual features are captured by bi-directional recurrent neural network models",
            "authors": [
                "Kushal Chawla",
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Learning algorithms for natural language processing (NLP) tasks traditionally\nrely on manually defined relevant contextual features. On the other hand,\nneural network models using an only distributional representation of words have\nbeen successfully applied for several NLP tasks. Such models learn features\nautomatically and avoid explicit feature engineering. Across several domains,\nneural models become a natural choice specifically when limited characteristics\nof data are known. However, this flexibility comes at the cost of\ninterpretability. In this paper, we define three different methods to\ninvestigate ability of bi-directional recurrent neural networks (RNNs) in\ncapturing contextual features. In particular, we analyze RNNs for sequence\ntagging tasks. We perform a comprehensive analysis on general as well as\nbiomedical domain datasets. Our experiments focus on important contextual words\nas features, which can easily be extended to analyze various other feature\ntypes. We also investigate positional effects of context words and show how the\ndeveloped methods can be used for error analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00678v1",
            "date": "2017-09-03 07:42:17+00:00",
            "title": "Disentangling ASR and MT Errors in Speech Translation",
            "authors": [
                "Ngoc-Tien Le",
                "Benjamin Lecouteux",
                "Laurent Besacier"
            ],
            "abstract": "The main aim of this paper is to investigate automatic quality assessment for\nspoken language translation (SLT). More precisely, we investigate SLT errors\nthat can be due to transcription (ASR) or to translation (MT) modules. This\npaper investigates automatic detection of SLT errors using a single classifier\nbased on joint ASR and MT features. We evaluate both 2-class (good/bad) and\n3-class (good/badASR/badMT ) labeling tasks. The 3-class problem necessitates\nto disentangle ASR and MT errors in the speech translation output and we\npropose two label extraction methods for this non trivial step. This enables -\nas a by-product - qualitative analysis on the SLT errors and their origin (are\nthey due to transcription or to translation step?) on our large in-house corpus\nfor French-to-English speech translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00813v1",
            "date": "2017-09-04 05:38:35+00:00",
            "title": "From Review to Rating: Exploring Dependency Measures for Text Classification",
            "authors": [
                "Samuel Cunningham-Nelson",
                "Mahsa Baktashmotlagh",
                "Wageeh Boles"
            ],
            "abstract": "Various text analysis techniques exist, which attempt to uncover unstructured\ninformation from text. In this work, we explore using statistical dependence\nmeasures for textual classification, representing text as word vectors. Student\nsatisfaction scores on a 3-point scale and their free text comments written\nabout university subjects are used as the dataset. We have compared two textual\nrepresentations: a frequency word representation and term frequency\nrelationship to word vectors, and found that word vectors provide a greater\naccuracy. However, these word vectors have a large number of features which\naggravates the burden of computational complexity. Thus, we explored using a\nnon-linear dependency measure for feature selection by maximizing the\ndependence between the text reviews and corresponding scores. Our quantitative\nand qualitative analysis on a student satisfaction dataset shows that our\napproach achieves comparable accuracy to the full feature vector, while being\nan order of magnitude faster in testing. These text analysis and feature\nreduction techniques can be used for other textual data applications such as\nsentiment analysis.",
            "journal ref": "Under Consideration by Pattern Recognition Letters (PRL) 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00831v1",
            "date": "2017-09-04 06:29:36+00:00",
            "title": "Hypothesis Testing based Intrinsic Evaluation of Word Embeddings",
            "authors": [
                "Nishant Gurnani"
            ],
            "abstract": "We introduce the cross-match test - an exact, distribution free,\nhigh-dimensional hypothesis test as an intrinsic evaluation metric for word\nembeddings. We show that cross-match is an effective means of measuring\ndistributional similarity between different vector representations and of\nevaluating the statistical significance of different vector embedding models.\nAdditionally, we find that cross-match can be used to provide a quantitative\nmeasure of linguistic similarity for selecting bridge languages for machine\ntranslation. We demonstrate that the results of the hypothesis test align with\nour expectations and note that the framework of two sample hypothesis testing\nis not limited to word embeddings and can be extended to all vector\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01042v1",
            "date": "2017-09-04 16:54:35+00:00",
            "title": "Getting Reliable Annotations for Sarcasm in Online Dialogues",
            "authors": [
                "Reid Swanson",
                "Stephanie Lukin",
                "Luke Eisenberg",
                "Thomas Chase Corcoran",
                "Marilyn A. Walker"
            ],
            "abstract": "The language used in online forums differs in many ways from that of\ntraditional language resources such as news. One difference is the use and\nfrequency of nonliteral, subjective dialogue acts such as sarcasm. Whether the\naim is to develop a theory of sarcasm in dialogue, or engineer automatic\nmethods for reliably detecting sarcasm, a major challenge is simply the\ndifficulty of getting enough reliably labelled examples. In this paper we\ndescribe our work on methods for achieving highly reliable sarcasm annotations\nfrom untrained annotators on Mechanical Turk. We explore the use of a number of\ncommon statistical reliability measures, such as Kappa, Karger's, Majority\nClass, and EM. We show that more sophisticated measures do not appear to yield\nbetter results for our data than simple measures such as assuming that the\ncorrect label is the one that a majority of Turkers apply.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01058v2",
            "date": "2017-09-04 17:54:49+00:00",
            "title": "A Unified Query-based Generative Model for Question Generation and Question Answering",
            "authors": [
                "Linfeng Song",
                "Zhiguo Wang",
                "Wael Hamza"
            ],
            "abstract": "We propose a query-based generative model for solving both tasks of question\ngeneration (QG) and question an- swering (QA). The model follows the classic\nencoder- decoder framework. The encoder takes a passage and a query as input\nthen performs query understanding by matching the query with the passage from\nmultiple per- spectives. The decoder is an attention-based Long Short Term\nMemory (LSTM) model with copy and coverage mechanisms. In the QG task, a\nquestion is generated from the system given the passage and the target answer,\nwhereas in the QA task, the answer is generated given the question and the\npassage. During the training stage, we leverage a policy-gradient reinforcement\nlearning algorithm to overcome exposure bias, a major prob- lem resulted from\nsequence learning with cross-entropy loss. For the QG task, our experiments\nshow higher per- formances than the state-of-the-art results. When used as\nadditional training data, the automatically generated questions even improve\nthe performance of a strong ex- tractive QA system. In addition, our model\nshows bet- ter performance than the state-of-the-art baselines of the\ngenerative QA task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01121v2",
            "date": "2017-09-04 19:05:39+00:00",
            "title": "Do latent tree learning models identify meaningful structure in sentences?",
            "authors": [
                "Adina Williams",
                "Andrew Drozdov",
                "Samuel R. Bowman"
            ],
            "abstract": "Recent work on the problem of latent tree learning has made it possible to\ntrain neural networks that learn to both parse a sentence and use the resulting\nparse to interpret the sentence, all without exposure to ground-truth parse\ntrees at training time. Surprisingly, these models often perform better at\nsentence understanding tasks than models that use parse trees from conventional\nparsers. This paper aims to investigate what these latent tree learning models\nlearn. We replicate two such models in a shared codebase and find that (i) only\none of these models outperforms conventional tree-structured models on sentence\nclassification, (ii) its parsing strategies are not especially consistent\nacross random restarts, (iii) the parses it produces tend to be shallower than\nstandard Penn Treebank (PTB) parses, and (iv) they do not resemble those of PTB\nor any other semantic or syntactic formalism that the authors are aware of.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01186v1",
            "date": "2017-09-04 22:52:59+00:00",
            "title": "Learning Neural Word Salience Scores",
            "authors": [
                "Krasen Samardzhiev",
                "Andrew Gargett",
                "Danushka Bollegala"
            ],
            "abstract": "Measuring the salience of a word is an essential step in numerous NLP tasks.\nHeuristic approaches such as tfidf have been used so far to estimate the\nsalience of words. We propose \\emph{Neural Word Salience} (NWS) scores, unlike\nheuristics, are learnt from a corpus. Specifically, we learn word salience\nscores such that, using pre-trained word embeddings as the input, can\naccurately predict the words that appear in a sentence, given the words that\nappear in the sentences preceding or succeeding that sentence. Experimental\nresults on sentence similarity prediction show that the learnt word salience\nscores perform comparably or better than some of the state-of-the-art\napproaches for representing sentences on benchmark datasets for sentence\nsimilarity, while using only a fraction of the training and prediction times\nrequired by prior methods. Moreover, our NWS scores positively correlate with\npsycholinguistic measures such as concreteness, and imageability implying a\nclose connection to the salience as perceived by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01189v1",
            "date": "2017-09-04 23:06:36+00:00",
            "title": "Satirical News Detection and Analysis using Attention Mechanism and Linguistic Features",
            "authors": [
                "Fan Yang",
                "Arjun Mukherjee",
                "Eduard Dragut"
            ],
            "abstract": "Satirical news is considered to be entertainment, but it is potentially\ndeceptive and harmful. Despite the embedded genre in the article, not everyone\ncan recognize the satirical cues and therefore believe the news as true news.\nWe observe that satirical cues are often reflected in certain paragraphs rather\nthan the whole document. Existing works only consider document-level features\nto detect the satire, which could be limited. We consider paragraph-level\nlinguistic features to unveil the satire by incorporating neural network and\nattention mechanism. We investigate the difference between paragraph-level\nfeatures and document-level features, and analyze them on a large satirical\nnews dataset. The evaluation shows that the proposed model detects satirical\nnews effectively and reveals what features are important at which level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01193v1",
            "date": "2017-09-04 23:30:22+00:00",
            "title": "Compositional Approaches for Representing Relations Between Words: A Comparative Study",
            "authors": [
                "Huda Hakami",
                "Danushka Bollegala"
            ],
            "abstract": "Identifying the relations that exist between words (or entities) is important\nfor various natural language processing tasks such as, relational search,\nnoun-modifier classification and analogy detection. A popular approach to\nrepresent the relations between a pair of words is to extract the patterns in\nwhich the words co-occur with from a corpus, and assign each word-pair a vector\nof pattern frequencies. Despite the simplicity of this approach, it suffers\nfrom data sparseness, information scalability and linguistic creativity as the\nmodel is unable to handle previously unseen word pairs in a corpus. In\ncontrast, a compositional approach for representing relations between words\novercomes these issues by using the attributes of each individual word to\nindirectly compose a representation for the common relations that hold between\nthe two words. This study aims to compare different operations for creating\nrelation representations from word-level representations. We investigate the\nperformance of the compositional methods by measuring the relational\nsimilarities using several benchmark datasets for word analogy. Moreover, we\nevaluate the different relation representations in a knowledge base completion\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01199v1",
            "date": "2017-09-05 00:25:58+00:00",
            "title": "Using $k$-way Co-occurrences for Learning Word Embeddings",
            "authors": [
                "Danushka Bollegala",
                "Yuichi Yoshida",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Co-occurrences between two words provide useful insights into the semantics\nof those words. Consequently, numerous prior work on word embedding learning\nhave used co-occurrences between two words as the training signal for learning\nword embeddings. However, in natural language texts it is common for multiple\nwords to be related and co-occurring in the same context. We extend the notion\nof co-occurrences to cover $k(\\geq\\!\\!2)$-way co-occurrences among a set of\n$k$-words. Specifically, we prove a theoretical relationship between the joint\nprobability of $k(\\geq\\!\\!2)$ words, and the sum of $\\ell_2$ norms of their\nembeddings. Next, we propose a learning objective motivated by our theoretical\nresult that utilises $k$-way co-occurrences for learning word embeddings. Our\nexperimental results show that the derived theoretical relationship does indeed\nhold empirically, and despite data sparsity, for some smaller $k$ values,\n$k$-way embeddings perform comparably or better than $2$-way embeddings in a\nrange of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01562v2",
            "date": "2017-09-05 19:27:22+00:00",
            "title": "Optimizing for Measure of Performance in Max-Margin Parsing",
            "authors": [
                "Alexander Bauer",
                "Shinichi Nakajima",
                "Nico G\u00f6rnitz",
                "Klaus-Robert M\u00fcller"
            ],
            "abstract": "Many statistical learning problems in the area of natural language processing\nincluding sequence tagging, sequence segmentation and syntactic parsing has\nbeen successfully approached by means of structured prediction methods. An\nappealing property of the corresponding discriminative learning algorithms is\ntheir ability to integrate the loss function of interest directly into the\noptimization process, which potentially can increase the resulting performance\naccuracy. Here, we demonstrate on the example of constituency parsing how to\noptimize for F1-score in the max-margin framework of structural SVM. In\nparticular, the optimization is with respect to the original (not binarized)\ntrees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01634v2",
            "date": "2017-09-06 00:15:37+00:00",
            "title": "The Voynich Manuscript is Written in Natural Language: The Pahlavi Hypothesis",
            "authors": [
                "J. Michael Herrmann"
            ],
            "abstract": "The late medieval Voynich Manuscript (VM) has resisted decryption and was\nconsidered a meaningless hoax or an unsolvable cipher. Here, we provide\nevidence that the VM is written in natural language by establishing a relation\nof the Voynich alphabet and the Iranian Pahlavi script. Many of the Voynich\ncharacters are upside-down versions of their Pahlavi counterparts, which may be\nan effect of different writing directions. Other Voynich letters can be\nexplained as ligatures or departures from Pahlavi with the intent to cope with\nknown problems due to the stupendous ambiguity of Pahlavi text. While a\ntranslation of the VM text is not attempted here, we can confirm the\nVoynich-Pahlavi relation at the character level by the transcription of many\nwords from the VM illustrations and from parts of the main text. Many of the\ntranscribed words can be identified as terms from Zoroastrian cosmology which\nis in line with the use of Pahlavi script in Zoroastrian communities from\nmedieval times.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01679v2",
            "date": "2017-09-06 05:23:37+00:00",
            "title": "A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse",
            "authors": [
                "Sosuke Kobayashi",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "This study addresses the problem of identifying the meaning of unknown words\nor entities in a discourse with respect to the word embedding approaches used\nin neural language models. We proposed a method for on-the-fly construction and\nexploitation of word embeddings in both the input and output layers of a neural\nmodel by tracking contexts. This extends the dynamic entity representation used\nin Kobayashi et al. (2016) and incorporates a copy mechanism proposed\nindependently by Gu et al. (2016) and Gulcehre et al. (2016). In addition, we\nconstruct a new task and dataset called Anonymized Language Modeling for\nevaluating the ability to capture word meanings while reading. Experiments\nconducted using our novel dataset show that the proposed variant of RNN\nlanguage model outperformed the baseline model. Furthermore, the experiments\nalso demonstrate that dynamic updates of an output layer help a model predict\nreappearing entities, whereas those of an input layer are effective to predict\nwords following reappearing entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01766v3",
            "date": "2017-09-06 11:13:50+00:00",
            "title": "Information-Propogation-Enhanced Neural Machine Translation by Relation Model",
            "authors": [
                "Wen Zhang",
                "Jiawei Hu",
                "Yang Feng",
                "Qun Liu"
            ],
            "abstract": "Even though sequence-to-sequence neural machine translation (NMT) model have\nachieved state-of-art performance in the recent fewer years, but it is widely\nconcerned that the recurrent neural network (RNN) units are very hard to\ncapture the long-distance state information, which means RNN can hardly find\nthe feature with long term dependency as the sequence becomes longer.\nSimilarly, convolutional neural network (CNN) is introduced into NMT for\nspeeding recently, however, CNN focus on capturing the local feature of the\nsequence; To relieve this issue, we incorporate a relation network into the\nstandard encoder-decoder framework to enhance information-propogation in neural\nnetwork, ensuring that the information of the source sentence can flow into the\ndecoder adequately. Experiments show that proposed framework outperforms the\nstatistical MT model and the state-of-art NMT model significantly on two data\nsets with different scales.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01848v1",
            "date": "2017-09-06 14:50:42+00:00",
            "title": "Depression and Self-Harm Risk Assessment in Online Forums",
            "authors": [
                "Andrew Yates",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "Users suffering from mental health conditions often turn to online resources\nfor support, including specialized online support communities or general\ncommunities such as Twitter and Reddit. In this work, we present a neural\nframework for supporting and studying users in both types of communities. We\npropose methods for identifying posts in support communities that may indicate\na risk of self-harm, and demonstrate that our approach outperforms strong\npreviously proposed methods for identifying such posts. Self-harm is closely\nrelated to depression, which makes identifying depressed users on general\nforums a crucial related task. We introduce a large-scale general forum dataset\n(\"RSDD\") consisting of users with self-reported depression diagnoses matched\nwith control users. We show how our method can be applied to effectively\nidentify depressed users from their use of language alone. We demonstrate that\nour method outperforms strong baselines on this general forum dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01895v1",
            "date": "2017-09-03 04:31:11+00:00",
            "title": "A Semi-Supervised Approach to Detecting Stance in Tweets",
            "authors": [
                "Amita Misra",
                "Brian Ecker",
                "Theodore Handleman",
                "Nicolas Hahn",
                "Marilyn Walker"
            ],
            "abstract": "Stance classification aims to identify, for a particular issue under\ndiscussion, whether the speaker or author of a conversational turn has Pro\n(Favor) or Con (Against) stance on the issue. Detecting stance in tweets is a\nnew task proposed for SemEval-2016 Task6, involving predicting stance for a\ndataset of tweets on the topics of abortion, atheism, climate change, feminism\nand Hillary Clinton. Given the small size of the dataset, our team created our\nown topic-specific training corpus by developing a set of high precision\nhashtags for each topic that were used to query the twitter API, with the aim\nof developing a large training corpus without additional human labeling of\ntweets for stance. The hashtags selected for each topic were predicted to be\nstance-bearing on their own. Experimental results demonstrate good performance\nfor our features for opinion-target pairs based on generalizing dependency\nfeatures using sentiment lexicons.",
            "journal ref": "International Workshop on Semantic Evaluation 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01950v1",
            "date": "2017-09-06 18:09:15+00:00",
            "title": "\"Having 2 hours to write a paper is fun!\": Detecting Sarcasm in Numerical Portions of Text",
            "authors": [
                "Lakshya Kumar",
                "Arpan Somani",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Sarcasm occurring due to the presence of numerical portions in text has been\nquoted as an error made by automatic sarcasm detection approaches in the past.\nWe present a first study in detecting sarcasm in numbers, as in the case of the\nsentence 'Love waking up at 4 am'. We analyze the challenges of the problem,\nand present Rule-based, Machine Learning and Deep Learning approaches to detect\nsarcasm in numerical portions of text. Our Deep Learning approach outperforms\nfour past works for sarcasm detection and Rule-based and Machine learning\napproaches on a dataset of tweets, obtaining an F1-score of 0.93. This shows\nthat special attention to text containing numbers may be useful to improve\nstate-of-the-art in sarcasm detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02184v3",
            "date": "2017-09-07 11:31:09+00:00",
            "title": "Translating Terminological Expressions in Knowledge Bases with Neural Machine Translation",
            "authors": [
                "Mihael Arcan",
                "Daniel Torregrosa",
                "Paul Buitelaar"
            ],
            "abstract": "Our work presented in this paper focuses on the translation of terminological\nexpressions represented in semantically structured resources, like ontologies\nor knowledge graphs. The challenge of translating ontology labels or\nterminological expressions documented in knowledge bases lies in the highly\nspecific vocabulary and the lack of contextual information, which can guide a\nmachine translation system to translate ambiguous words into the targeted\ndomain. Due to these challenges, we evaluate the translation quality of\ndomain-specific expressions in the medical and financial domain with\nstatistical as well as with neural machine translation methods and experiment\ndomain adaptation of the translation models with terminological expressions\nonly. Furthermore, we perform experiments on the injection of external\nterminological expressions into the translation systems. Through these\nexperiments, we observed a significant advantage in domain adaptation for the\ndomain-specific resource in the medical and financial domain and the benefit of\nsubword models over word-based neural machine translation models for\nterminology translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02271v1",
            "date": "2017-09-07 14:22:50+00:00",
            "title": "Leveraging Discourse Information Effectively for Authorship Attribution",
            "authors": [
                "Su Wang",
                "Elisa Ferracane",
                "Raymond J. Mooney"
            ],
            "abstract": "We explore techniques to maximize the effectiveness of discourse information\nin the task of authorship attribution. We present a novel method to embed\ndiscourse features in a Convolutional Neural Network text classifier, which\nachieves a state-of-the-art result by a substantial margin. We empirically\ninvestigate several featurization methods to understand the conditions under\nwhich discourse features contribute non-trivial performance gains, and analyze\ndiscourse embeddings.",
            "journal ref": "The 8th International Joint Conference on Natural Language\n  Processing (IJCNLP 2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02279v1",
            "date": "2017-09-07 14:30:50+00:00",
            "title": "Cynical Selection of Language Model Training Data",
            "authors": [
                "Amittai Axelrod"
            ],
            "abstract": "The Moore-Lewis method of \"intelligent selection of language model training\ndata\" is very effective, cheap, efficient... and also has structural problems.\n(1) The method defines relevance by playing language models trained on the\nin-domain and the out-of-domain (or data pool) corpora against each other. This\npowerful idea-- which we set out to preserve-- treats the two corpora as the\nopposing ends of a single spectrum. This lack of nuance does not allow for the\ntwo corpora to be very similar. In the extreme case where the come from the\nsame distribution, all of the sentences have a Moore-Lewis score of zero, so\nthere is no resulting ranking. (2) The selected sentences are not guaranteed to\nbe able to model the in-domain data, nor to even cover the in-domain data. They\nare simply well-liked by the in-domain model; this is necessary, but not\nsufficient. (3) There is no way to tell what is the optimal number of sentences\nto select, short of picking various thresholds and building the systems.\n  We present a greedy, lazy, approximate, and generally efficient\ninformation-theoretic method of accomplishing the same goal using only\nvocabulary counts. The method has the following properties: (1) Is responsive\nto the extent to which two corpora differ. (2) Quickly reaches near-optimal\nvocabulary coverage. (3) Takes into account what has already been selected. (4)\nDoes not involve defining any kind of domain, nor any kind of classifier. (6)\nKnows approximately when to stop. This method can be used as an\ninherently-meaningful measure of similarity, as it measures the bits of\ninformation to be gained by adding one text to another.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02783v1",
            "date": "2017-09-08 17:12:16+00:00",
            "title": "A Statistical Comparison of Some Theories of NP Word Order",
            "authors": [
                "Richard Futrell",
                "Roger Levy",
                "Matthew Dryer"
            ],
            "abstract": "A frequent object of study in linguistic typology is the order of elements\n{demonstrative, adjective, numeral, noun} in the noun phrase. The goal is to\npredict the relative frequencies of these orders across languages. Here we use\nPoisson regression to statistically compare some prominent accounts of this\nvariation. We compare feature systems derived from Cinque (2005) to feature\nsystems given in Cysouw (2010) and Dryer (in prep). In this setting, we do not\nfind clear reasons to prefer the model of Cinque (2005) or Dryer (in prep), but\nwe find both of these models have substantially better fit to the typological\ndata than the model from Cysouw (2010).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02828v1",
            "date": "2017-09-08 18:27:50+00:00",
            "title": "Globally Normalized Reader",
            "authors": [
                "Jonathan Raiman",
                "John Miller"
            ],
            "abstract": "Rapid progress has been made towards question answering (QA) systems that can\nextract answers from text. Existing neural approaches make use of expensive\nbi-directional attention mechanisms or score all possible answer spans,\nlimiting scalability. We propose instead to cast extractive QA as an iterative\nsearch problem: select the answer's sentence, start word, and end word. This\nrepresentation reduces the space of each search step and allows computation to\nbe conditionally allocated to promising search paths. We show that globally\nnormalizing the decision process and back-propagating through beam search makes\nthis representation viable and learning efficient. We empirically demonstrate\nthe benefits of this approach using our model, Globally Normalized Reader\n(GNR), which achieves the second highest single model performance on the\nStanford Question Answering Dataset (68.4 EM, 76.21 F1 dev) and is 24.7x faster\nthan bi-attention-flow. We also introduce a data-augmentation method to produce\nsemantically valid examples by aligning named entities to a knowledge base and\nswapping them with new entities of the same type. This method improves the\nperformance of all models considered in this work and is of independent\ninterest for a variety of NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02842v1",
            "date": "2017-09-08 19:30:09+00:00",
            "title": "Combining LSTM and Latent Topic Modeling for Mortality Prediction",
            "authors": [
                "Yohan Jo",
                "Lisa Lee",
                "Shruti Palaskar"
            ],
            "abstract": "There is a great need for technologies that can predict the mortality of\npatients in intensive care units with both high accuracy and accountability. We\npresent joint end-to-end neural network architectures that combine long\nshort-term memory (LSTM) and a latent topic model to simultaneously train a\nclassifier for mortality prediction and learn latent topics indicative of\nmortality from textual clinical notes. For topic interpretability, the topic\nmodeling layer has been carefully designed as a single-layer network with\nconstraints inspired by LDA. Experiments on the MIMIC-III dataset show that our\nmodels significantly outperform prior models that are based on LDA topics in\nmortality prediction. However, we achieve limited success with our method for\ninterpreting topics from the trained models by looking at the neural network\nweights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02843v1",
            "date": "2017-09-08 19:34:02+00:00",
            "title": "CLaC at SemEval-2016 Task 11: Exploring linguistic and psycho-linguistic Features for Complex Word Identification",
            "authors": [
                "Elnaz Davoodi",
                "Leila Kosseim"
            ],
            "abstract": "This paper describes the system deployed by the CLaC-EDLK team to the\n\"SemEval 2016, Complex Word Identification task\". The goal of the task is to\nidentify if a given word in a given context is \"simple\" or \"complex\". Our\nsystem relies on linguistic features and cognitive complexity. We used several\nsupervised models, however the Random Forest model outperformed the others.\nOverall our best configuration achieved a G-score of 68.8% in the task, ranking\nour system 21 out of 45.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02911v1",
            "date": "2017-09-09 05:04:19+00:00",
            "title": "Semi-Supervised Instance Population of an Ontology using Word Vector Embeddings",
            "authors": [
                "Vindula Jayawardana",
                "Dimuthu Lakmal",
                "Nisansa de Silva",
                "Amal Shehan Perera",
                "Keet Sugathadasa",
                "Buddhi Ayesha",
                "Madhavi Perera"
            ],
            "abstract": "In many modern day systems such as information extraction and knowledge\nmanagement agents, ontologies play a vital role in maintaining the concept\nhierarchies of the selected domain. However, ontology population has become a\nproblematic process due to its nature of heavy coupling with manual human\nintervention. With the use of word embeddings in the field of natural language\nprocessing, it became a popular topic due to its ability to cope up with\nsemantic sensitivity. Hence, in this study, we propose a novel way of\nsemi-supervised ontology population through word embeddings as the basis. We\nbuilt several models including traditional benchmark models and new types of\nmodels which are based on word embeddings. Finally, we ensemble them together\nto come up with a synergistic model with better accuracy. We demonstrate that\nour ensemble model can outperform the individual models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03010v1",
            "date": "2017-09-09 22:03:11+00:00",
            "title": "Steering Output Style and Topic in Neural Response Generation",
            "authors": [
                "Di Wang",
                "Nebojsa Jojic",
                "Chris Brockett",
                "Eric Nyberg"
            ],
            "abstract": "We propose simple and flexible training and decoding methods for influencing\noutput style and topic in neural encoder-decoder based language generation.\nThis capability is desirable in a variety of applications, including\nconversational systems, where successful agents need to produce language in a\nspecific style and generate responses steered by a human puppeteer or external\nknowledge. We decompose the neural generation process into empirically easier\nsub-problems: a faithfulness model and a decoding method based on\nselective-sampling. We also describe training and sampling algorithms that bias\nthe generation process with a specific language style restriction, or a topic\nrestriction. Human evaluation results show that our proposed methods are able\nto restrict style and topic without degrading output quality in conversational\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03064v2",
            "date": "2017-09-10 07:56:44+00:00",
            "title": "AppTechMiner: Mining Applications and Techniques from Scientific Articles",
            "authors": [
                "Mayank Singh",
                "Soham Dan",
                "Sanyam Agarwal",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "This paper presents AppTechMiner, a rule-based information extraction\nframework that automatically constructs a knowledge base of all application\nareas and problem solving techniques. Techniques include tools, methods,\ndatasets or evaluation metrics. We also categorize individual research articles\nbased on their application areas and the techniques proposed/improved in the\narticle. Our system achieves high average precision (~82%) and recall (~84%) in\nknowledge base creation. It also performs well in application and technique\nassignment to an individual article (average accuracy ~66%). In the end, we\nfurther present two use cases presenting a trivial information retrieval system\nand an extensive temporal analysis of the usage of techniques and application\nareas. At present, we demonstrate the framework for the domain of computational\nlinguistics but this can be easily generalized to any other field of research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03167v1",
            "date": "2017-09-10 20:22:30+00:00",
            "title": "Debbie, the Debate Bot of the Future",
            "authors": [
                "Geetanjali Rakshit",
                "Kevin K. Bowden",
                "Lena Reed",
                "Amita Misra",
                "Marilyn Walker"
            ],
            "abstract": "Chatbots are a rapidly expanding application of dialogue systems with\ncompanies switching to bot services for customer support, and new applications\nfor users interested in casual conversation. One style of casual conversation\nis argument, many people love nothing more than a good argument. Moreover,\nthere are a number of existing corpora of argumentative dialogues, annotated\nfor agreement and disagreement, stance, sarcasm and argument quality. This\npaper introduces Debbie, a novel arguing bot, that selects arguments from\nconversational corpora, and aims to use them appropriately in context. We\npresent an initial working prototype of Debbie, with some preliminary\nevaluation and describe future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03190v1",
            "date": "2017-09-10 22:37:18+00:00",
            "title": "Data-Driven Dialogue Systems for Social Agents",
            "authors": [
                "Kevin K. Bowden",
                "Shereen Oraby",
                "Amita Misra",
                "Jiaqi Wu",
                "Stephanie Lukin"
            ],
            "abstract": "In order to build dialogue systems to tackle the ambitious task of holding\nsocial conversations, we argue that we need a data driven approach that\nincludes insight into human conversational chit chat, and which incorporates\ndifferent natural language processing modules. Our strategy is to analyze and\nindex large corpora of social media data, including Twitter conversations,\nonline debates, dialogues between friends, and blog posts, and then to couple\nthis data retrieval with modules that perform tasks such as sentiment and style\nanalysis, topic modeling, and summarization. We aim for personal assistants\nthat can learn more nuanced human language, and to grow from task-oriented\nagents to more personable social bots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03544v1",
            "date": "2017-09-11 18:54:15+00:00",
            "title": "KnowNER: Incremental Multilingual Knowledge in Named Entity Recognition",
            "authors": [
                "Dominic Seyler",
                "Tatiana Dembelova",
                "Luciano Del Corro",
                "Johannes Hoffart",
                "Gerhard Weikum"
            ],
            "abstract": "KnowNER is a multilingual Named Entity Recognition (NER) system that\nleverages different degrees of external knowledge. A novel modular framework\ndivides the knowledge into four categories according to the depth of knowledge\nthey convey. Each category consists of a set of features automatically\ngenerated from different information sources (such as a knowledge-base, a list\nof names or document-specific semantic annotations) and is used to train a\nconditional random field (CRF). Since those information sources are usually\nmultilingual, KnowNER can be easily trained for a wide range of languages. In\nthis paper, we show that the incorporation of deeper knowledge systematically\nboosts accuracy and compare KnowNER with state-of-the-art NER approaches across\nthree languages (i.e., English, German and Spanish) performing amongst\nstate-of-the art systems in all of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03637v2",
            "date": "2017-09-12 00:57:04+00:00",
            "title": "Capturing Long-range Contextual Dependencies with Memory-enhanced Conditional Random Fields",
            "authors": [
                "Fei Liu",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Despite successful applications across a broad range of NLP tasks,\nconditional random fields (\"CRFs\"), in particular the linear-chain variant, are\nonly able to model local features. While this has important benefits in terms\nof inference tractability, it limits the ability of the model to capture\nlong-range dependencies between items. Attempts to extend CRFs to capture\nlong-range dependencies have largely come at the cost of computational\ncomplexity and approximate inference. In this work, we propose an extension to\nCRFs by integrating external memory, taking inspiration from memory networks,\nthereby allowing CRFs to incorporate information far beyond neighbouring steps.\nExperiments across two tasks show substantial improvements over strong CRF and\nLSTM baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03665v1",
            "date": "2017-09-12 02:52:54+00:00",
            "title": "Small-footprint Keyword Spotting Using Deep Neural Network and Connectionist Temporal Classifier",
            "authors": [
                "Zhiming Wang",
                "Xiaolong Li",
                "Jun Zhou"
            ],
            "abstract": "Mainly for the sake of solving the lack of keyword-specific data, we propose\none Keyword Spotting (KWS) system using Deep Neural Network (DNN) and\nConnectionist Temporal Classifier (CTC) on power-constrained small-footprint\nmobile devices, taking full advantage of general corpus from continuous speech\nrecognition which is of great amount. DNN is to directly predict the posterior\nof phoneme units of any personally customized key-phrase, and CTC to produce a\nconfidence score of the given phoneme sequence as responsive decision-making\nmechanism. The CTC-KWS has competitive performance in comparison with purely\nDNN based keyword specific KWS, but not increasing any computational\ncomplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03756v1",
            "date": "2017-09-12 09:23:55+00:00",
            "title": "Cross-lingual Word Segmentation and Morpheme Segmentation as Sequence Labelling",
            "authors": [
                "Yan Shao"
            ],
            "abstract": "This paper presents our segmentation system developed for the MLP 2017 shared\ntasks on cross-lingual word segmentation and morpheme segmentation. We model\nboth word and morpheme segmentation as character-level sequence labelling\ntasks. The prevalent bidirectional recurrent neural network with conditional\nrandom fields as the output interface is adapted as the baseline system, which\nis further improved via ensemble decoding. Our universal system is applied to\nand extensively evaluated on all the official data sets without any\nlanguage-specific adjustment. The official evaluation results indicate that the\nproposed model achieves outstanding accuracies both for word and morpheme\nsegmentation on all the languages in various types when compared to the other\nparticipating systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03759v1",
            "date": "2017-09-12 09:27:12+00:00",
            "title": "Language Models of Spoken Dutch",
            "authors": [
                "Lyan Verwimp",
                "Joris Pelemans",
                "Marieke Lycke",
                "Hugo Van hamme",
                "Patrick Wambacq"
            ],
            "abstract": "In Flanders, all TV shows are subtitled. However, the process of subtitling\nis a very time-consuming one and can be sped up by providing the output of a\nspeech recognizer run on the audio of the TV show, prior to the subtitling.\nNaturally, this speech recognition will perform much better if the employed\nlanguage model is adapted to the register and the topic of the program. We\npresent several language models trained on subtitles of television shows\nprovided by the Flemish public-service broadcaster VRT. This data was gathered\nin the context of the project STON which has as purpose to facilitate the\nprocess of subtitling TV shows. One model is trained on all available data (46M\nword tokens), but we also trained models on a specific type of TV show or\ndomain/topic. Language models of spoken language are quite rare due to the lack\nof training data. The size of this corpus is relatively large for a corpus of\nspoken language (compare with e.g. CGN which has 9M words), but still rather\nsmall for a language model. Thus, in practice it is advised to interpolate\nthese models with a large background language model trained on written\nlanguage. The models can be freely downloaded on\nhttp://www.esat.kuleuven.be/psi/spraak/downloads/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03814v1",
            "date": "2017-09-12 12:47:05+00:00",
            "title": "SYSTRAN Purely Neural MT Engines for WMT2017",
            "authors": [
                "Yongchao Deng",
                "Jungi Kim",
                "Guillaume Klein",
                "Catherine Kobus",
                "Natalia Segal",
                "Christophe Servan",
                "Bo Wang",
                "Dakun Zhang",
                "Josep Crego",
                "Jean Senellart"
            ],
            "abstract": "This paper describes SYSTRAN's systems submitted to the WMT 2017 shared news\ntranslation task for English-German, in both translation directions. Our\nsystems are built using OpenNMT, an open-source neural machine translation\nsystem, implementing sequence-to-sequence models with LSTM encoder/decoders and\nattention. We experimented using monolingual data automatically\nback-translated. Our resulting models are further hyper-specialised with an\nadaptation technique that finely tunes models according to the evaluation test\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03815v1",
            "date": "2017-09-12 12:58:07+00:00",
            "title": "OpenNMT: Open-source Toolkit for Neural Machine Translation",
            "authors": [
                "Guillaume Klein",
                "Yoon Kim",
                "Yuntian Deng",
                "Josep Crego",
                "Jean Senellart",
                "Alexander M. Rush"
            ],
            "abstract": "We introduce an open-source toolkit for neural machine translation (NMT) to\nsupport research into model architectures, feature representations, and source\nmodalities, while maintaining competitive performance, modularity and\nreasonable training requirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03856v5",
            "date": "2017-09-12 14:16:56+00:00",
            "title": "StarSpace: Embed All The Things!",
            "authors": [
                "Ledell Wu",
                "Adam Fisch",
                "Sumit Chopra",
                "Keith Adams",
                "Antoine Bordes",
                "Jason Weston"
            ],
            "abstract": "We present StarSpace, a general-purpose neural embedding model that can solve\na wide variety of problems: labeling tasks such as text classification, ranking\ntasks such as information retrieval/web search, collaborative filtering-based\nor content-based recommendation, embedding of multi-relational graphs, and\nlearning word, sentence or document level embeddings. In each case the model\nworks by embedding those entities comprised of discrete features and comparing\nthem against each other -- learning similarities dependent on the task.\nEmpirical results on a number of tasks show that StarSpace is highly\ncompetitive with existing methods, whilst also being generally applicable to\nnew cases where those methods are not.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03925v1",
            "date": "2017-09-12 15:57:22+00:00",
            "title": "Human Associations Help to Detect Conventionalized Multiword Expressions",
            "authors": [
                "Natalia Loukachevitch",
                "Anastasia Gerasimova"
            ],
            "abstract": "In this paper we show that if we want to obtain human evidence about\nconventionalization of some phrases, we should ask native speakers about\nassociations they have to a given phrase and its component words. We have shown\nthat if component words of a phrase have each other as frequent associations,\nthen this phrase can be considered as conventionalized. Another type of\nconventionalized phrases can be revealed using two factors: low entropy of\nphrase associations and low intersection of component word and phrase\nassociations. The association experiments were performed for the Russian\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03933v1",
            "date": "2017-09-12 16:13:10+00:00",
            "title": "Hash Embeddings for Efficient Word Representations",
            "authors": [
                "Dan Svenstrup",
                "Jonas Meinertz Hansen",
                "Ole Winther"
            ],
            "abstract": "We present hash embeddings, an efficient method for representing words in a\ncontinuous vector form. A hash embedding may be seen as an interpolation\nbetween a standard word embedding and a word embedding created using a random\nhash function (the hashing trick). In hash embeddings each token is represented\nby $k$ $d$-dimensional embeddings vectors and one $k$ dimensional weight\nvector. The final $d$ dimensional representation of the token is the product of\nthe two. Rather than fitting the embedding vectors for each token these are\nselected by the hashing trick from a shared pool of $B$ embedding vectors. Our\nexperiments show that hash embeddings can easily deal with huge vocabularies\nconsisting of millions of tokens. When using a hash embedding there is no need\nto create a dictionary before training nor to perform any kind of vocabulary\npruning after training. We show that models trained using hash embeddings\nexhibit at least the same level of performance as models trained using regular\nembeddings across a wide range of tasks. Furthermore, the number of parameters\nneeded by such an embedding is only a fraction of what is required by a regular\nembedding. Since standard embeddings and embeddings constructed using the\nhashing trick are actually just special cases of a hash embedding, hash\nembeddings can be considered an extension and improvement over the existing\nregular embedding types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04005v2",
            "date": "2017-09-12 18:19:51+00:00",
            "title": "Addressee and Response Selection in Multi-Party Conversations with Speaker Interaction RNNs",
            "authors": [
                "Rui Zhang",
                "Honglak Lee",
                "Lazaros Polymenakos",
                "Dragomir Radev"
            ],
            "abstract": "In this paper, we study the problem of addressee and response selection in\nmulti-party conversations. Understanding multi-party conversations is\nchallenging because of complex speaker interactions: multiple speakers exchange\nmessages with each other, playing different roles (sender, addressee,\nobserver), and these roles vary across turns. To tackle this challenge, we\npropose the Speaker Interaction Recurrent Neural Network (SI-RNN). Whereas the\nprevious state-of-the-art system updated speaker embeddings only for the\nsender, SI-RNN uses a novel dialog encoder to update speaker embeddings in a\nrole-sensitive way. Additionally, unlike the previous work that selected the\naddressee and response separately, SI-RNN selects them jointly by viewing the\ntask as a sequence prediction problem. Experimental results show that SI-RNN\nsignificantly improves the accuracy of addressee and response selection,\nparticularly in complex conversations with many speakers and responses to\ndistant messages many turns in the past.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04250v2",
            "date": "2017-09-13 11:01:02+00:00",
            "title": "Dialogue Act Sequence Labeling using Hierarchical encoder with CRF",
            "authors": [
                "Harshit Kumar",
                "Arvind Agarwal",
                "Riddhiman Dasgupta",
                "Sachindra Joshi",
                "Arun Kumar"
            ],
            "abstract": "Dialogue Act recognition associate dialogue acts (i.e., semantic labels) to\nutterances in a conversation. The problem of associating semantic labels to\nutterances can be treated as a sequence labeling problem. In this work, we\nbuild a hierarchical recurrent neural network using bidirectional LSTM as a\nbase unit and the conditional random field (CRF) as the top layer to classify\neach utterance into its corresponding dialogue act. The hierarchical network\nlearns representations at multiple levels, i.e., word level, utterance level,\nand conversation level. The conversation level representations are input to the\nCRF layer, which takes into account not only all previous utterances but also\ntheir dialogue acts, thus modeling the dependency among both, labels and\nutterances, an important consideration of natural dialogue. We validate our\napproach on two different benchmark data sets, Switchboard and Meeting Recorder\nDialogue Act, and show performance improvement over the state-of-the-art\nmethods by $2.2\\%$ and $4.1\\%$ absolute points, respectively. It is worth\nnoting that the inter-annotator agreement on Switchboard data set is $84\\%$,\nand our method is able to achieve the accuracy of about $79\\%$ despite being\ntrained on the noisy data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04264v1",
            "date": "2017-09-13 11:59:06+00:00",
            "title": "Flexible End-to-End Dialogue System for Knowledge Grounded Conversation",
            "authors": [
                "Wenya Zhu",
                "Kaixiang Mo",
                "Yu Zhang",
                "Zhangbin Zhu",
                "Xuezheng Peng",
                "Qiang Yang"
            ],
            "abstract": "In knowledge grounded conversation, domain knowledge plays an important role\nin a special domain such as Music. The response of knowledge grounded\nconversation might contain multiple answer entities or no entity at all.\nAlthough existing generative question answering (QA) systems can be applied to\nknowledge grounded conversation, they either have at most one entity in a\nresponse or cannot deal with out-of-vocabulary entities. We propose a fully\ndata-driven generative dialogue system GenDS that is capable of generating\nresponses based on input message and related knowledge base (KB). To generate\narbitrary number of answer entities even when these entities never appear in\nthe training set, we design a dynamic knowledge enquirer which selects\ndifferent answer entities at different positions in a single response,\naccording to different local context. It does not rely on the representations\nof entities, enabling our model deal with out-of-vocabulary entities. We\ncollect a human-human conversation data (ConversMusic) with knowledge\nannotations. The proposed method is evaluated on CoversMusic and a public\nquestion answering dataset. Our proposed GenDS system outperforms baseline\nmethods significantly in terms of the BLEU, entity accuracy, entity recall and\nhuman evaluation. Moreover,the experiments also demonstrate that GenDS works\nbetter even on small datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04348v2",
            "date": "2017-09-13 14:22:14+00:00",
            "title": "Natural Language Inference over Interaction Space",
            "authors": [
                "Yichen Gong",
                "Heng Luo",
                "Jian Zhang"
            ],
            "abstract": "Natural Language Inference (NLI) task requires an agent to determine the\nlogical relationship between a natural language premise and a natural language\nhypothesis. We introduce Interactive Inference Network (IIN), a novel class of\nneural network architectures that is able to achieve high-level understanding\nof the sentence pair by hierarchically extracting semantic features from\ninteraction space. We show that an interaction tensor (attention weight)\ncontains semantic information to solve natural language inference, and a denser\ninteraction tensor contains richer semantic information. One instance of such\narchitecture, Densely Interactive Inference Network (DIIN), demonstrates the\nstate-of-the-art performance on large scale NLI copora and large-scale NLI\nalike corpus. It's noteworthy that DIIN achieve a greater than 20% error\nreduction on the challenging Multi-Genre NLI (MultiNLI) dataset with respect to\nthe strongest published system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04359v1",
            "date": "2017-09-13 14:46:50+00:00",
            "title": "Linguistic Features of Genre and Method Variation in Translation: A Computational Perspective",
            "authors": [
                "Ekaterina Lapshninova-Koltunski",
                "Marcos Zampieri"
            ],
            "abstract": "In this paper we describe the use of text classification methods to\ninvestigate genre and method variation in an English - German translation\ncorpus. For this purpose we use linguistically motivated features representing\ntexts using a combination of part-of-speech tags arranged in bigrams, trigrams,\nand 4-grams. The classification method used in this paper is a Bayesian\nclassifier with Laplace smoothing. We use the output of the classifiers to\ncarry out an extensive feature analysis on the main difference between genres\nand methods of translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04409v1",
            "date": "2017-09-13 16:31:44+00:00",
            "title": "A Review of Evaluation Techniques for Social Dialogue Systems",
            "authors": [
                "Amanda Cercas Curry",
                "Helen Hastie",
                "Verena Rieser"
            ],
            "abstract": "In contrast with goal-oriented dialogue, social dialogue has no clear measure\nof task success. Consequently, evaluation of these systems is notoriously hard.\nIn this paper, we review current evaluation methods, focusing on automatic\nmetrics. We conclude that turn-based metrics often ignore the context and do\nnot account for the fact that several replies are valid, while end-of-dialogue\nrewards are mainly hand-crafted. Both lack grounding in human perceptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04491v1",
            "date": "2017-09-13 18:17:56+00:00",
            "title": "Method for Aspect-Based Sentiment Annotation Using Rhetorical Analysis",
            "authors": [
                "\u0141ukasz Augustyniak",
                "Krzysztof Rajda",
                "Tomasz Kajdanowicz"
            ],
            "abstract": "This paper fills a gap in aspect-based sentiment analysis and aims to present\na new method for preparing and analysing texts concerning opinion and\ngenerating user-friendly descriptive reports in natural language. We present a\ncomprehensive set of techniques derived from Rhetorical Structure Theory and\nsentiment analysis to extract aspects from textual opinions and then build an\nabstractive summary of a set of opinions. Moreover, we propose aspect-aspect\ngraphs to evaluate the importance of aspects and to filter out unimportant ones\nfrom the summary. Additionally, the paper presents a prototype solution of data\nflow with interesting and valuable results. The proposed method's results\nproved the high accuracy of aspect detection when applied to the gold standard\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04682v1",
            "date": "2017-09-14 09:36:48+00:00",
            "title": "Towards an Arabic-English Machine-Translation Based on Semantic Web",
            "authors": [
                "Neama Abdulaziz Dahan",
                "Fadl Mutaher Ba-Alwi",
                "Ibrahim Ahmed Al-Baltah",
                "Ghaleb H. Al-gapheri"
            ],
            "abstract": "Communication tools make the world like a small village and as a consequence\npeople can contact with others who are from different societies or who speak\ndifferent languages. This communication cannot happen effectively without\nMachine Translation because they can be found anytime and everywhere. There are\na number of studies that have developed Machine Translation for the English\nlanguage with so many other languages except the Arabic it has not been\nconsidered yet. Therefore we aim to highlight a roadmap for our proposed\ntranslation machine to provide an enhanced Arabic English translation based on\nSemantic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04685v1",
            "date": "2017-09-14 09:46:15+00:00",
            "title": "Machine-Translation History and Evolution: Survey for Arabic-English Translations",
            "authors": [
                "Nabeel T. Alsohybe",
                "Neama Abdulaziz Dahan",
                "Fadl Mutaher Ba-Alwi"
            ],
            "abstract": "As a result of the rapid changes in information and communication technology\n(ICT), the world has become a small village where people from all over the\nworld connect with each other in dialogue and communication via the Internet.\nAlso, communications have become a daily routine activity due to the new\nglobalization where companies and even universities become global residing\ncross countries borders. As a result, translation becomes a needed activity in\nthis connected world. ICT made it possible to have a student in one country\ntake a course or even a degree from a different country anytime anywhere\neasily. The resulted communication still needs a language as a means that helps\nthe receiver understands the contents of the sent message. People need an\nautomated translation application because human translators are hard to find\nall the times, and the human translations are very expensive comparing to the\ntranslations automated process. Several types of research describe the\nelectronic process of the Machine-Translation. In this paper, the authors are\ngoing to study some of these previous researches, and they will explore some of\nthe needed tools for the Machine-Translation. This research is going to\ncontribute to the Machine-Translation area by helping future researchers to\nhave a summary for the Machine-Translation groups of research and to let lights\non the importance of the translation mechanism.",
            "journal ref": "CJAST 23(4): 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04820v1",
            "date": "2017-09-14 14:44:30+00:00",
            "title": "Synapse at CAp 2017 NER challenge: Fasttext CRF",
            "authors": [
                "Damien Sileo",
                "Camille Pradel",
                "Philippe Muller",
                "Tim Van de Cruys"
            ],
            "abstract": "We present our system for the CAp 2017 NER challenge which is about named\nentity recognition on French tweets. Our system leverages unsupervised learning\non a larger dataset of French tweets to learn features feeding a CRF model. It\nwas ranked first without using any gazetteer or structured external data, with\nan F-measure of 58.89\\%. To the best of our knowledge, it is the first system\nto use fasttext embeddings (which include subword representations) and an\nembedding-based sentence representation for NER.",
            "journal ref": "CAP2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04849v5",
            "date": "2017-09-14 16:06:45+00:00",
            "title": "Self-Attentive Residual Decoder for Neural Machine Translation",
            "authors": [
                "Lesly Miculicich Werlen",
                "Nikolaos Pappas",
                "Dhananjay Ram",
                "Andrei Popescu-Belis"
            ],
            "abstract": "Neural sequence-to-sequence networks with attention have achieved remarkable\nperformance for machine translation. One of the reasons for their effectiveness\nis their ability to capture relevant source-side contextual information at each\ntime-step prediction through an attention mechanism. However, the target-side\ncontext is solely based on the sequence model which, in practice, is prone to a\nrecency bias and lacks the ability to capture effectively non-sequential\ndependencies among words. To address this limitation, we propose a\ntarget-side-attentive residual recurrent network for decoding, where attention\nover previous words contributes directly to the prediction of the next word.\nThe residual learning facilitates the flow of information from the distant past\nand is able to emphasize any of the previously translated words, hence it gains\naccess to a wider context. The proposed model outperforms a neural MT baseline\nas well as a memory and self-attention network on three language pairs. The\nanalysis of the attention learned by the decoder confirms that it emphasizes a\nwider context, and that it captures syntactic-like structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04969v1",
            "date": "2017-09-14 20:28:27+00:00",
            "title": "Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications",
            "authors": [
                "Fred Morstatter",
                "Kai Shu",
                "Suhang Wang",
                "Huan Liu"
            ],
            "abstract": "Most social media platforms are largely based on text, and users often write\nposts to describe where they are, what they are seeing, and how they are\nfeeling. Because written text lacks the emotional cues of spoken and\nface-to-face dialogue, ambiguities are common in written language. This problem\nis exacerbated in the short, informal nature of many social media posts. To\nbypass this issue, a suite of special characters called \"emojis,\" which are\nsmall pictograms, are embedded within the text. Many emojis are small\ndepictions of facial expressions designed to help disambiguate the emotional\nmeaning of the text. However, a new ambiguity arises in the way that emojis are\nrendered. Every platform (Windows, Mac, and Android, to name a few) renders\nemojis according to their own style. In fact, it has been shown that some\nemojis can be rendered so differently that they look \"happy\" on some platforms,\nand \"sad\" on others. In this work, we use real-world data to verify the\nexistence of this problem. We verify that the usage of the same emoji can be\nsignificantly different across platforms, with some emojis exhibiting different\nsentiment polarities on different platforms. We propose a solution to identify\nthe intended emoji based on the platform-specific nature of the emoji used by\nthe author of a social media post. We apply our solution to sentiment analysis,\na task that can benefit from the emoji calibration technique we use in this\nwork. We conduct experiments to evaluate the effectiveness of the mapping in\nthis task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05074v1",
            "date": "2017-09-15 06:58:13+00:00",
            "title": "A Deep Generative Framework for Paraphrase Generation",
            "authors": [
                "Ankush Gupta",
                "Arvind Agarwal",
                "Prawaan Singh",
                "Piyush Rai"
            ],
            "abstract": "Paraphrase generation is an important problem in NLP, especially in question\nanswering, information retrieval, information extraction, conversation systems,\nto name a few. In this paper, we address the problem of generating paraphrases\nautomatically. Our proposed method is based on a combination of deep generative\nmodels (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases,\ngiven an input sentence. Traditional VAEs when combined with recurrent neural\nnetworks can generate free text but they are not suitable for paraphrase\ngeneration for a given sentence. We address this problem by conditioning the\nboth, encoder and decoder sides of VAE, on the original sentence, so that it\ncan generate the given sentence's paraphrases. Unlike most existing models, our\nmodel is simple, modular and can generate multiple paraphrases, for a given\nsentence. Quantitative evaluation of the proposed method on a benchmark\nparaphrase dataset demonstrates its efficacy, and its performance improvement\nover the state-of-the-art methods by a significant margin, whereas qualitative\nhuman evaluation indicate that the generated paraphrases are well-formed,\ngrammatically correct, and are relevant to the input sentence. Furthermore, we\nevaluate our method on a newly released question paraphrase dataset, and\nestablish a new baseline for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05094v1",
            "date": "2017-09-15 08:12:58+00:00",
            "title": "Unsupervised Aspect Term Extraction with B-LSTM & CRF using Automatically Labelled Datasets",
            "authors": [
                "Athanasios Giannakopoulos",
                "Claudiu Musat",
                "Andreea Hossmann",
                "Michael Baeriswyl"
            ],
            "abstract": "Aspect Term Extraction (ATE) identifies opinionated aspect terms in texts and\nis one of the tasks in the SemEval Aspect Based Sentiment Analysis (ABSA)\ncontest. The small amount of available datasets for supervised ATE and the\ncostly human annotation for aspect term labelling give rise to the need for\nunsupervised ATE. In this paper, we introduce an architecture that achieves\ntop-ranking performance for supervised ATE. Moreover, it can be used\nefficiently as feature extractor and classifier for unsupervised ATE. Our\nsecond contribution is a method to automatically construct datasets for ATE. We\ntrain a classifier on our automatically labelled datasets and evaluate it on\nthe human annotated SemEval ABSA test sets. Compared to a strong rule-based\nbaseline, we obtain a dramatically higher F-score and attain precision values\nabove 80%. Our unsupervised method beats the supervised ABSA baseline from\nSemEval, while preserving high precision scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05227v1",
            "date": "2017-09-15 14:19:29+00:00",
            "title": "Transcribing Against Time",
            "authors": [
                "Matthias Sperber",
                "Graham Neubig",
                "Jan Niehues",
                "Satoshi Nakamura",
                "Alex Waibel"
            ],
            "abstract": "We investigate the problem of manually correcting errors from an automatic\nspeech transcript in a cost-sensitive fashion. This is done by specifying a\nfixed time budget, and then automatically choosing location and size of\nsegments for correction such that the number of corrected errors is maximized.\nThe core components, as suggested by previous research [1], are a utility model\nthat estimates the number of errors in a particular segment, and a cost model\nthat estimates annotation effort for the segment. In this work we propose a\ndynamic updating framework that allows for the training of cost models during\nthe ongoing transcription process. This removes the need for transcriber\nenrollment prior to the actual transcription, and improves correction\nefficiency by allowing highly transcriber-adaptive cost modeling. We first\nconfirm and analyze the improvements afforded by this method in a simulated\nstudy. We then conduct a realistic user study, observing efficiency\nimprovements of 15% relative on average, and 42% for the participants who\ndeviated most strongly from our initial, transcriber-agnostic cost model.\nMoreover, we find that our updating framework can capture dynamically changing\nfactors, such as transcriber fatigue and topic familiarity, which we observe to\nhave a large influence on the transcriber's working behavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05295v1",
            "date": "2017-09-15 16:28:46+00:00",
            "title": "And That's A Fact: Distinguishing Factual and Emotional Argumentation in Online Dialogue",
            "authors": [
                "Shereen Oraby",
                "Lena Reed",
                "Ryan Compton",
                "Ellen Riloff",
                "Marilyn Walker",
                "Steve Whittaker"
            ],
            "abstract": "We investigate the characteristics of factual and emotional argumentation\nstyles observed in online debates. Using an annotated set of \"factual\" and\n\"feeling\" debate forum posts, we extract patterns that are highly correlated\nwith factual and emotional arguments, and then apply a bootstrapping\nmethodology to find new patterns in a larger pool of unannotated forum posts.\nThis process automatically produces a large set of patterns representing\nlinguistic expressions that are highly correlated with factual and emotional\nlanguage. Finally, we analyze the most discriminating patterns to better\nunderstand the defining characteristics of factual and emotional arguments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05305v1",
            "date": "2017-09-15 16:54:02+00:00",
            "title": "Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog",
            "authors": [
                "Shereen Oraby",
                "Vrindavan Harrison",
                "Amita Misra",
                "Ellen Riloff",
                "Marilyn Walker"
            ],
            "abstract": "Effective models of social dialog must understand a broad range of rhetorical\nand figurative devices. Rhetorical questions (RQs) are a type of figurative\nlanguage whose aim is to achieve a pragmatic goal, such as structuring an\nargument, being persuasive, emphasizing a point, or being ironic. While there\nare computational models for other forms of figurative language, rhetorical\nquestions have received little attention to date. We expand a small dataset\nfrom previous work, presenting a corpus of 10,270 RQs from debate forums and\nTwitter that represent different discourse functions. We show that we can\nclearly distinguish between RQs and sincere questions (0.76 F1). We then show\nthat RQs can be used both sarcastically and non-sarcastically, observing that\nnon-sarcastic (other) uses of RQs are frequently argumentative in forums, and\npersuasive in tweets. We present experiments to distinguish between these uses\nof RQs using SVM and LSTM models that represent linguistic features and\npost-level context, achieving results as high as 0.76 F1 for \"sarcastic\" and\n0.77 F1 for \"other\" in forums, and 0.83 F1 for both \"sarcastic\" and \"other\" in\ntweets. We supplement our quantitative experiments with an in-depth\ncharacterization of the linguistic variation in RQs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05308v1",
            "date": "2017-09-15 16:59:20+00:00",
            "title": "Harvesting Creative Templates for Generating Stylistically Varied Restaurant Reviews",
            "authors": [
                "Shereen Oraby",
                "Sheideh Homayon",
                "Marilyn Walker"
            ],
            "abstract": "Many of the creative and figurative elements that make language exciting are\nlost in translation in current natural language generation engines. In this\npaper, we explore a method to harvest templates from positive and negative\nreviews in the restaurant domain, with the goal of vastly expanding the types\nof stylistic variation available to the natural language generator. We learn\nhyperbolic adjective patterns that are representative of the strongly-valenced\nexpressive language commonly used in either positive or negative reviews. We\nthen identify and delexicalize entities, and use heuristics to extract\ngeneration templates from review sentences. We evaluate the learned templates\nagainst more traditional review templates, using subjective measures of\n\"convincingness\", \"interestingness\", and \"naturalness\". Our results show that\nthe learned templates score highly on these measures. Finally, we analyze the\nlinguistic categories that characterize the learned positive and negative\ntemplates. We plan to use the learned templates to improve the conversational\nstyle of dialogue systems in the restaurant domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05404v1",
            "date": "2017-09-15 21:01:57+00:00",
            "title": "Creating and Characterizing a Diverse Corpus of Sarcasm in Dialogue",
            "authors": [
                "Shereen Oraby",
                "Vrindavan Harrison",
                "Lena Reed",
                "Ernesto Hernandez",
                "Ellen Riloff",
                "Marilyn Walker"
            ],
            "abstract": "The use of irony and sarcasm in social media allows us to study them at scale\nfor the first time. However, their diversity has made it difficult to construct\na high-quality corpus of sarcasm in dialogue. Here, we describe the process of\ncreating a large- scale, highly-diverse corpus of online debate forums\ndialogue, and our novel methods for operationalizing classes of sarcasm in the\nform of rhetorical questions and hyperbole. We show that we can use\nlexico-syntactic cues to reliably retrieve sarcastic utterances with high\naccuracy. To demonstrate the properties and quality of our corpus, we conduct\nsupervised learning experiments with simple features, and show that we achieve\nboth higher precision and F than previous work on sarcasm in debate forums\ndialogue. We apply a weakly-supervised linguistic pattern learner and\nqualitatively analyze the linguistic differences in each class.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05411v1",
            "date": "2017-09-15 21:12:59+00:00",
            "title": "Combining Search with Structured Data to Create a More Engaging User Experience in Open Domain Dialogue",
            "authors": [
                "Kevin K. Bowden",
                "Shereen Oraby",
                "Jiaqi Wu",
                "Amita Misra",
                "Marilyn Walker"
            ],
            "abstract": "The greatest challenges in building sophisticated open-domain conversational\nagents arise directly from the potential for ongoing mixed-initiative\nmulti-turn dialogues, which do not follow a particular plan or pursue a\nparticular fixed information need. In order to make coherent conversational\ncontributions in this context, a conversational agent must be able to track the\ntypes and attributes of the entities under discussion in the conversation and\nknow how they are related. In some cases, the agent can rely on structured\ninformation sources to help identify the relevant semantic relations and\nproduce a turn, but in other cases, the only content available comes from\nsearch, and it may be unclear which semantic relations hold between the search\nresults and the discourse context. A further constraint is that the system must\nproduce its contribution to the ongoing conversation in real-time. This paper\ndescribes our experience building SlugBot for the 2017 Alexa Prize, and\ndiscusses how we leveraged search and structured data from different sources to\nhelp SlugBot produce dialogic turns and carry on conversations whose length\nover the semi-finals user evaluation period averaged 8:17 minutes.",
            "journal ref": "Search-Oriented Conversational AI (SCAI) 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05413v1",
            "date": "2017-09-15 21:24:48+00:00",
            "title": "\"How May I Help You?\": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts",
            "authors": [
                "Shereen Oraby",
                "Pritam Gundecha",
                "Jalal Mahmud",
                "Mansurul Bhuiyan",
                "Rama Akkiraju"
            ],
            "abstract": "Given the increasing popularity of customer service dialogue on Twitter,\nanalysis of conversation data is essential to understand trends in customer and\nagent behavior for the purpose of automating customer service interactions. In\nthis work, we develop a novel taxonomy of fine-grained \"dialogue acts\"\nfrequently observed in customer service, showcasing acts that are more suited\nto the domain than the more generic existing taxonomies. Using a sequential\nSVM-HMM model, we model conversation flow, predicting the dialogue act of a\ngiven turn in real-time. We characterize differences between customer and agent\nbehavior in Twitter customer service conversations, and investigate the effect\nof testing our system on different customer service industries. Finally, we use\na data-driven approach to predict important conversation outcomes: customer\nsatisfaction, customer frustration, and overall problem resolution. We show\nthat the type and location of certain dialogue acts in a conversation have a\nsignificant effect on the probability of desirable and undesirable outcomes,\nand present actionable rules based on our findings. The patterns and rules we\nderive can be used as guidelines for outcome-driven automated customer service\nplatforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05475v2",
            "date": "2017-09-16 07:39:53+00:00",
            "title": "Order-Preserving Abstractive Summarization for Spoken Content Based on Connectionist Temporal Classification",
            "authors": [
                "Bo-Ru Lu",
                "Frank Shyu",
                "Yun-Nung Chen",
                "Hung-Yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Connectionist temporal classification (CTC) is a powerful approach for\nsequence-to-sequence learning, and has been popularly used in speech\nrecognition. The central ideas of CTC include adding a label \"blank\" during\ntraining. With this mechanism, CTC eliminates the need of segment alignment,\nand hence has been applied to various sequence-to-sequence learning problems.\nIn this work, we applied CTC to abstractive summarization for spoken content.\nThe \"blank\" in this case implies the corresponding input data are less\nimportant or noisy; thus it can be ignored. This approach was shown to\noutperform the existing methods in term of ROUGE scores over Chinese Gigaword\nand MATBN corpora. This approach also has the nice property that the ordering\nof words or characters in the input documents can be better preserved in the\ngenerated summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05487v1",
            "date": "2017-09-16 09:40:36+00:00",
            "title": "Role of Morphology Injection in Statistical Machine Translation",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Phrase-based Statistical models are more commonly used as they perform\noptimally in terms of both, translation quality and complexity of the system.\nHindi and in general all Indian languages are morphologically richer than\nEnglish. Hence, even though Phrase-based systems perform very well for the less\ndivergent language pairs, for English to Indian language translation, we need\nmore linguistic information (such as morphology, parse tree, parts of speech\ntags, etc.) on the source side. Factored models seem to be useful in this case,\nas Factored models consider word as a vector of factors. These factors can\ncontain any information about the surface word and use it while translating.\nHence, the objective of this work is to handle morphological inflections in\nHindi and Marathi using Factored translation models while translating from\nEnglish. SMT approaches face the problem of data sparsity while translating\ninto a morphologically rich language. It is very unlikely for a parallel corpus\nto contain all morphological forms of words. We propose a solution to generate\nthese unseen morphological forms and inject them into original training\ncorpora. In this paper, we study factored models and the problem of sparseness\nin context of translation to morphologically rich languages. We propose a\nsimple and effective solution which is based on enriching the input with\nvarious morphological forms of words. We observe that morphology injection\nimproves the quality of translation in terms of both adequacy and fluency. We\nverify this with the experiments on two morphologically rich languages: Hindi\nand Marathi, while translating from English.",
            "journal ref": "ACM Transactions on Asian and Low-Resource Language Information\n  Processing (TALLIP), Volume 17 Issue 1, September 2017 Issue-in-Progress,\n  Article No. 1",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05522v1",
            "date": "2017-09-16 14:33:27+00:00",
            "title": "AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline",
            "authors": [
                "Hui Bu",
                "Jiayu Du",
                "Xingyu Na",
                "Bengu Wu",
                "Hao Zheng"
            ],
            "abstract": "An open-source Mandarin speech corpus called AISHELL-1 is released. It is by\nfar the largest corpus which is suitable for conducting the speech recognition\nresearch and building speech recognition systems for Mandarin. The recording\nprocedure, including audio capturing devices and environments are presented in\ndetails. The preparation of the related resources, including transcriptions and\nlexicon are described. The corpus is released with a Kaldi recipe. Experimental\nresults implies that the quality of audio recordings and transcriptions are\npromising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05563v1",
            "date": "2017-09-16 20:16:56+00:00",
            "title": "Data Innovation for International Development: An overview of natural language processing for qualitative data analysis",
            "authors": [
                "Philipp Broniecki",
                "Anna Hanchar",
                "Slava J. Mikhaylov"
            ],
            "abstract": "Availability, collection and access to quantitative data, as well as its\nlimitations, often make qualitative data the resource upon which development\nprograms heavily rely. Both traditional interview data and social media\nanalysis can provide rich contextual information and are essential for\nresearch, appraisal, monitoring and evaluation. These data may be difficult to\nprocess and analyze both systematically and at scale. This, in turn, limits the\nability of timely data driven decision-making which is essential in fast\nevolving complex social systems. In this paper, we discuss the potential of\nusing natural language processing to systematize analysis of qualitative data,\nand to inform quick decision-making in the development context. We illustrate\nthis with interview data generated in a format of micro-narratives for the UNDP\nFragments of Impact project.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05599v1",
            "date": "2017-09-17 02:56:40+00:00",
            "title": "Hierarchical Gated Recurrent Neural Tensor Network for Answer Triggering",
            "authors": [
                "Wei Li",
                "Yunfang Wu"
            ],
            "abstract": "In this paper, we focus on the problem of answer triggering ad-dressed by\nYang et al. (2015), which is a critical component for a real-world question\nanswering system. We employ a hierarchical gated recurrent neural tensor\n(HGRNT) model to capture both the context information and the deep\nin-teractions between the candidate answers and the question. Our result on F\nval-ue achieves 42.6%, which surpasses the baseline by over 10 %.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05631v2",
            "date": "2017-09-17 09:36:40+00:00",
            "title": "Unwritten Languages Demand Attention Too! Word Discovery with Encoder-Decoder Models",
            "authors": [
                "Marcely Zanon Boito",
                "Alexandre Berard",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "Word discovery is the task of extracting words from unsegmented text. In this\npaper we examine to what extent neural networks can be applied to this task in\na realistic unwritten language scenario, where only small corpora and limited\nannotations are available. We investigate two scenarios: one with no\nsupervision and another with limited supervision with access to the most\nfrequent words. Obtained results show that it is possible to retrieve at least\n27% of the gold standard vocabulary by training an encoder-decoder neural\nmachine translation system with only 5,157 sentences. This result is close to\nthose obtained with a task-specific Bayesian nonparametric model. Moreover, our\napproach has the advantage of generating translation alignments, which could be\nused to create a bilingual lexicon. As a future perspective, this approach is\nalso well suited to work directly from speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05820v2",
            "date": "2017-09-18 08:46:20+00:00",
            "title": "Toward a full-scale neural machine translation in production: the Booking.com use case",
            "authors": [
                "Pavel Levin",
                "Nishikant Dhanuka",
                "Talaat Khalil",
                "Fedor Kovalev",
                "Maxim Khalilov"
            ],
            "abstract": "While some remarkable progress has been made in neural machine translation\n(NMT) research, there have not been many reports on its development and\nevaluation in practice. This paper tries to fill this gap by presenting some of\nour findings from building an in-house travel domain NMT system in a large\nscale E-commerce setting. The three major topics that we cover are optimization\nand training (including different optimization strategies and corpus sizes),\nhandling real-world content and evaluating results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05914v1",
            "date": "2017-09-18 13:28:40+00:00",
            "title": "Limitations of Cross-Lingual Learning from Image Search",
            "authors": [
                "Mareike Hartmann",
                "Anders Soegaard"
            ],
            "abstract": "Cross-lingual representation learning is an important step in making NLP\nscale to all the world's languages. Recent work on bilingual lexicon induction\nsuggests that it is possible to learn cross-lingual representations of words\nbased on similarities between images associated with these words. However, that\nwork focused on the translation of selected nouns only. In our work, we\ninvestigate whether the meaning of other parts-of-speech, in particular\nadjectives and verbs, can be learned in the same way. We also experiment with\ncombining the representations learned from visual data with embeddings learned\nfrom textual data. Our experiments across five language pairs indicate that\nprevious work does not scale to the problem of learning cross-lingual\nrepresentations beyond simple nouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06033v1",
            "date": "2017-09-18 16:34:18+00:00",
            "title": "Sequence to Sequence Learning for Event Prediction",
            "authors": [
                "Dai Quoc Nguyen",
                "Dat Quoc Nguyen",
                "Cuong Xuan Chu",
                "Stefan Thater",
                "Manfred Pinkal"
            ],
            "abstract": "This paper presents an approach to the task of predicting an event\ndescription from a preceding sentence in a text. Our approach explores\nsequence-to-sequence learning using a bidirectional multi-layer recurrent\nneural network. Our approach substantially outperforms previous work in terms\nof the BLEU score on two datasets derived from WikiHow and DeScript\nrespectively. Since the BLEU score is not easy to interpret as a measure of\nevent prediction, we complement our study with a second evaluation that\nexploits the rich linguistic annotation of gold paraphrase sets of events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06136v1",
            "date": "2017-09-18 19:45:51+00:00",
            "title": "Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "In this paper, we present a deep reinforcement learning (RL) framework for\niterative dialog policy optimization in end-to-end task-oriented dialog\nsystems. Popular approaches in learning dialog policy with RL include letting a\ndialog agent to learn against a user simulator. Building a reliable user\nsimulator, however, is not trivial, often as difficult as building a good\ndialog agent. We address this challenge by jointly optimizing the dialog agent\nand the user simulator with deep RL by simulating dialogs between the two\nagents. We first bootstrap a basic dialog agent and a basic user simulator by\nlearning directly from dialog corpora with supervised training. We then improve\nthem further by letting the two agents to conduct task-oriented dialogs and\niteratively optimizing their policies with deep RL. Both the dialog agent and\nthe user simulator are designed with neural network models that can be trained\nend-to-end. Our experiment results show that the proposed method leads to\npromising improvements on task success rate and total task reward comparing to\nsupervised training and single-agent RL training baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06162v1",
            "date": "2017-09-18 20:40:11+00:00",
            "title": "Paraphrasing verbal metonymy through computational methods",
            "authors": [
                "Alberto Mor\u00f3n Hern\u00e1ndez"
            ],
            "abstract": "Verbal metonymy has received relatively scarce attention in the field of\ncomputational linguistics despite the fact that a model to accurately\nparaphrase metonymy has applications both in academia and the technology\nsector. The method described in this paper makes use of data from the British\nNational Corpus in order to create word vectors, find instances of verbal\nmetonymy and generate potential paraphrases. Two different ways of creating\nword vectors are evaluated in this study: Continuous bag of words and\nSkip-grams. Skip-grams are found to outperform the Continuous bag of words\napproach. Furthermore, the Skip-gram model is found to operate with\nbetter-than-chance accuracy and there is a strong positive relationship (phi\ncoefficient = 0.61) between the model's classification and human judgement of\nthe ranked paraphrases. This study lends credence to the viability of modelling\nverbal metonymy through computational methods based on distributional\nsemantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06265v2",
            "date": "2017-09-19 06:07:12+00:00",
            "title": "Dynamic Oracle for Neural Machine Translation in Decoding Phase",
            "authors": [
                "Zi-Yi Dou",
                "Hao Zhou",
                "Shu-Jian Huang",
                "Xin-Yu Dai",
                "Jia-Jun Chen"
            ],
            "abstract": "The past several years have witnessed the rapid progress of end-to-end Neural\nMachine Translation (NMT). However, there exists discrepancy between training\nand inference in NMT when decoding, which may lead to serious problems since\nthe model might be in a part of the state space it has never seen during\ntraining. To address the issue, Scheduled Sampling has been proposed. However,\nthere are certain limitations in Scheduled Sampling and we propose two dynamic\noracle-based methods to improve it. We manage to mitigate the discrepancy by\nchanging the training process towards a less guided scheme and meanwhile\naggregating the oracle's demonstrations. Experimental results show that the\nproposed approaches improve translation quality over standard NMT system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06307v2",
            "date": "2017-09-19 09:16:09+00:00",
            "title": "A Fast and Accurate Vietnamese Word Segmenter",
            "authors": [
                "Dat Quoc Nguyen",
                "Dai Quoc Nguyen",
                "Thanh Vu",
                "Mark Dras",
                "Mark Johnson"
            ],
            "abstract": "We propose a novel approach to Vietnamese word segmentation. Our approach is\nbased on the Single Classification Ripple Down Rules methodology (Compton and\nJansen, 1990), where rules are stored in an exception structure and new rules\nare only added to correct segmentation errors given by existing rules.\nExperimental results on the benchmark Vietnamese treebank show that our\napproach outperforms previous state-of-the-art approaches JVnSegmenter,\nvnTokenizer, DongDu and UETsegmenter in terms of both accuracy and performance\nspeed. Our code is open-source and available at:\nhttps://github.com/datquocnguyen/RDRsegmenter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06309v1",
            "date": "2017-09-19 09:22:12+00:00",
            "title": "Aspect-Based Relational Sentiment Analysis Using a Stacked Neural Network Architecture",
            "authors": [
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "Sentiment analysis can be regarded as a relation extraction problem in which\nthe sentiment of some opinion holder towards a certain aspect of a product,\ntheme or event needs to be extracted. We present a novel neural architecture\nfor sentiment analysis as a relation extraction problem that addresses this\nproblem by dividing it into three subtasks: i) identification of aspect and\nopinion terms, ii) labeling of opinion terms with a sentiment, and iii)\nextraction of relations between opinion terms and aspect terms. For each\nsubtask, we propose a neural network based component and combine all of them\ninto a complete system for relational sentiment analysis. The component for\naspect and opinion term extraction is a hybrid architecture consisting of a\nrecurrent neural network stacked on top of a convolutional neural network. This\napproach outperforms a standard convolutional deep neural architecture as well\nas a recurrent network architecture and performs competitively compared to\nother methods on two datasets of annotated customer reviews. To extract\nsentiments for individual opinion terms, we propose a recurrent architecture in\ncombination with word distance features and achieve promising results,\noutperforming a majority baseline by 18% accuracy and providing the first\nresults for the USAGE dataset. Our relation extraction component outperforms\nthe current state-of-the-art in aspect-opinion relation extraction by 15%\nF-Measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06311v1",
            "date": "2017-09-19 09:30:35+00:00",
            "title": "Aspect-Based Sentiment Analysis Using a Two-Step Neural Network Architecture",
            "authors": [
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "The World Wide Web holds a wealth of information in the form of unstructured\ntexts such as customer reviews for products, events and more. By extracting and\nanalyzing the expressed opinions in customer reviews in a fine-grained way,\nvaluable opportunities and insights for customers and businesses can be gained.\nWe propose a neural network based system to address the task of Aspect-Based\nSentiment Analysis to compete in Task 2 of the ESWC-2016 Challenge on Semantic\nSentiment Analysis. Our proposed architecture divides the task in two subtasks:\naspect term extraction and aspect-specific sentiment extraction. This approach\nis flexible in that it allows to address each subtask independently. As a first\nstep, a recurrent neural network is used to extract aspects from a text by\nframing the problem as a sequence labeling task. In a second step, a recurrent\nnetwork processes each extracted aspect with respect to its context and\npredicts a sentiment label. The system uses pretrained semantic word embedding\nfeatures which we experimentally enhance with semantic knowledge extracted from\nWordNet. Further features extracted from SenticNet prove to be beneficial for\nthe extraction of sentiment labels. As the best performing system in its\ncategory, our proposed system proves to be an effective approach for the\nAspect-Based Sentiment Analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06317v1",
            "date": "2017-09-19 09:46:11+00:00",
            "title": "Improving Opinion-Target Extraction with Character-Level Word Embeddings",
            "authors": [
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "Fine-grained sentiment analysis is receiving increasing attention in recent\nyears. Extracting opinion target expressions (OTE) in reviews is often an\nimportant step in fine-grained, aspect-based sentiment analysis. Retrieving\nthis information from user-generated text, however, can be difficult. Customer\nreviews, for instance, are prone to contain misspelled words and are difficult\nto process due to their domain-specific language. In this work, we investigate\nwhether character-level models can improve the performance for the\nidentification of opinion target expressions. We integrate information about\nthe character structure of a word into a sequence labeling system using\ncharacter-level word embeddings and show their positive impact on the system's\nperformance. Specifically, we obtain an increase by 3.3 points F1-score with\nrespect to our baseline model. In further experiments, we reveal encoded\ncharacter patterns of the learned embeddings and give a nuanced view of the\nperformance differences of both models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06436v1",
            "date": "2017-09-19 14:04:15+00:00",
            "title": "Language Modeling with Highway LSTM",
            "authors": [
                "Gakuto Kurata",
                "Bhuvana Ramabhadran",
                "George Saon",
                "Abhinav Sethy"
            ],
            "abstract": "Language models (LMs) based on Long Short Term Memory (LSTM) have shown good\ngains in many automatic speech recognition tasks. In this paper, we extend an\nLSTM by adding highway networks inside an LSTM and use the resulting Highway\nLSTM (HW-LSTM) model for language modeling. The added highway networks increase\nthe depth in the time dimension. Since a typical LSTM has two internal states,\na memory cell and a hidden state, we compare various types of HW-LSTM by adding\nhighway networks onto the memory cell and/or the hidden state. Experimental\nresults on English broadcast news and conversational telephone speech\nrecognition show that the proposed HW-LSTM LM improves speech recognition\naccuracy on top of a strong LSTM LM baseline. We report 5.1% and 9.9% on the\nSwitchboard and CallHome subsets of the Hub5 2000 evaluation, which reaches the\nbest performance numbers reported on these tasks to date.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06438v2",
            "date": "2017-09-19 14:09:03+00:00",
            "title": "A Recorded Debating Dataset",
            "authors": [
                "Shachar Mirkin",
                "Michal Jacovi",
                "Tamar Lavee",
                "Hong-Kwang Kuo",
                "Samuel Thomas",
                "Leslie Sager",
                "Lili Kotlerman",
                "Elad Venezian",
                "Noam Slonim"
            ],
            "abstract": "This paper describes an English audio and textual dataset of debating\nspeeches, a unique resource for the growing research field of computational\nargumentation and debating technologies. We detail the process of speech\nrecording by professional debaters, the transcription of the speeches with an\nAutomatic Speech Recognition (ASR) system, their consequent automatic\nprocessing to produce a text that is more \"NLP-friendly\", and in parallel --\nthe manual transcription of the speeches in order to produce gold-standard\n\"reference\" transcripts. We release 60 speeches on various controversial\ntopics, each in five formats corresponding to the different stages in the\nproduction of the data. The intention is to allow utilizing this resource for\nmultiple research purposes, be it the addition of in-domain training data for a\ndebate-specific ASR system, or applying argumentation mining on either noisy or\nclean debate transcripts. We intend to make further releases of this data in\nthe future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06901v2",
            "date": "2017-09-20 14:30:17+00:00",
            "title": "De-identification of medical records using conditional random fields and long short-term memory networks",
            "authors": [
                "Zhipeng Jiang",
                "Chao Zhao",
                "Bin He",
                "Yi Guan",
                "Jingchi Jiang"
            ],
            "abstract": "The CEGS N-GRID 2016 Shared Task 1 in Clinical Natural Language Processing\nfocuses on the de-identification of psychiatric evaluation records. This paper\ndescribes two participating systems of our team, based on conditional random\nfields (CRFs) and long short-term memory networks (LSTMs). A pre-processing\nmodule was introduced for sentence detection and tokenization before\nde-identification. For CRFs, manually extracted rich features were utilized to\ntrain the model. For LSTMs, a character-level bi-directional LSTM network was\napplied to represent tokens and classify tags for each token, following which a\ndecoding layer was stacked to decode the most probable protected health\ninformation (PHI) terms. The LSTM-based system attained an i2b2 strict\nmicro-F_1 measure of 89.86%, which was higher than that of the CRF-based\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07104v2",
            "date": "2017-09-20 23:56:24+00:00",
            "title": "On the Use of Machine Translation-Based Approaches for Vietnamese Diacritic Restoration",
            "authors": [
                "Thai-Hoang Pham",
                "Xuan-Khoai Pham",
                "Phuong Le-Hong"
            ],
            "abstract": "This paper presents an empirical study of two machine translation-based\napproaches for Vietnamese diacritic restoration problem, including phrase-based\nand neural-based machine translation models. This is the first work that\napplies neural-based machine translation method to this problem and gives a\nthorough comparison to the phrase-based machine translation method which is the\ncurrent state-of-the-art method for this problem. On a large dataset, the\nphrase-based approach has an accuracy of 97.32% while that of the neural-based\napproach is 96.15%. While the neural-based method has a slightly lower\naccuracy, it is about twice faster than the phrase-based method in terms of\ninference speed. Moreover, neural-based machine translation method has much\nroom for future improvement such as incorporating pre-trained word embeddings\nand collecting more training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07276v1",
            "date": "2017-09-21 12:10:43+00:00",
            "title": "Speech Recognition Challenge in the Wild: Arabic MGB-3",
            "authors": [
                "Ahmed Ali",
                "Stephan Vogel",
                "Steve Renals"
            ],
            "abstract": "This paper describes the Arabic MGB-3 Challenge - Arabic Speech Recognition\nin the Wild. Unlike last year's Arabic MGB-2 Challenge, for which the\nrecognition task was based on more than 1,200 hours broadcast TV news\nrecordings from Aljazeera Arabic TV programs, MGB-3 emphasises dialectal Arabic\nusing a multi-genre collection of Egyptian YouTube videos. Seven genres were\nused for the data collection: comedy, cooking, family/kids, fashion, drama,\nsports, and science (TEDx). A total of 16 hours of videos, split evenly across\nthe different genres, were divided into adaptation, development and evaluation\ndata sets. The Arabic MGB-Challenge comprised two tasks: A) Speech\ntranscription, evaluated on the MGB-3 test set, along with the 10 hour MGB-2\ntest set to report progress on the MGB-2 evaluation; B) Arabic dialect\nidentification, introduced this year in order to distinguish between four major\nArabic dialects - Egyptian, Levantine, North African, Gulf, as well as Modern\nStandard Arabic. Two hours of audio per dialect were released for development\nand a further two hours were used for evaluation. For dialect identification,\nboth lexical features and i-vector bottleneck features were shared with\nparticipants in addition to the raw audio recordings. Overall, thirteen teams\nsubmitted ten systems to the challenge. We outline the approaches adopted in\neach system, and summarise the evaluation results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07357v1",
            "date": "2017-09-21 14:51:51+00:00",
            "title": "Retrofitting Concept Vector Representations of Medical Concepts to Improve Estimates of Semantic Similarity and Relatedness",
            "authors": [
                "Zhiguo Yu",
                "Byron C. Wallace",
                "Todd Johnson",
                "Trevor Cohen"
            ],
            "abstract": "Estimation of semantic similarity and relatedness between biomedical concepts\nhas utility for many informatics applications. Automated methods fall into two\ncategories: methods based on distributional statistics drawn from text corpora,\nand methods using the structure of existing knowledge resources. Methods in the\nformer category disregard taxonomic structure, while those in the latter fail\nto consider semantically relevant empirical information. In this paper, we\npresent a method that retrofits distributional context vector representations\nof biomedical concepts using structural information from the UMLS\nMetathesaurus, such that the similarity between vector representations of\nlinked concepts is augmented. We evaluated it on the UMNSRS benchmark. Our\nresults demonstrate that retrofitting of concept vector representations leads\nto better correlation with human raters for both similarity and relatedness,\nsurpassing the best results reported to date. They also demonstrate a clear\nimprovement in performance on this reference standard for retrofitted vector\nrepresentations, as compared to those without retrofitting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07403v2",
            "date": "2017-09-21 16:40:46+00:00",
            "title": "Inducing Distant Supervision in Suggestion Mining through Part-of-Speech Embeddings",
            "authors": [
                "Sapna Negi",
                "Paul Buitelaar"
            ],
            "abstract": "Mining suggestion expressing sentences from a given text is a less\ninvestigated sentence classification task, and therefore lacks hand labeled\nbenchmark datasets. In this work, we propose and evaluate two approaches for\ndistant supervision in suggestion mining. The distant supervision is obtained\nthrough a large silver standard dataset, constructed using the text from\nwikiHow and Wikipedia. Both the approaches use a LSTM based neural network\narchitecture to learn a classification model for suggestion mining, but vary in\ntheir method to use the silver standard dataset. The first approach directly\ntrains the classifier using this dataset, while the second approach only learns\nword embeddings from this dataset. In the second approach, we also learn POS\nembeddings, which interestingly gives the best classification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07470v1",
            "date": "2017-09-21 18:17:36+00:00",
            "title": "Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts",
            "authors": [
                "Arpita Roy",
                "Youngja Park",
                "SHimei Pan"
            ],
            "abstract": "Word embedding is a Natural Language Processing (NLP) technique that\nautomatically maps words from a vocabulary to vectors of real numbers in an\nembedding space. It has been widely used in recent years to boost the\nperformance of a vari-ety of NLP tasks such as Named Entity Recognition,\nSyntac-tic Parsing and Sentiment Analysis. Classic word embedding methods such\nas Word2Vec and GloVe work well when they are given a large text corpus. When\nthe input texts are sparse as in many specialized domains (e.g.,\ncybersecurity), these methods often fail to produce high-quality vectors. In\nthis pa-per, we describe a novel method to train domain-specificword embeddings\nfrom sparse texts. In addition to domain texts, our method also leverages\ndiverse types of domain knowledge such as domain vocabulary and semantic\nrelations. Specifi-cally, we first propose a general framework to encode\ndiverse types of domain knowledge as text annotations. Then we de-velop a novel\nWord Annotation Embedding (WAE) algorithm to incorporate diverse types of text\nannotations in word em-bedding. We have evaluated our method on two\ncybersecurity text corpora: a malware description corpus and a Common\nVulnerability and Exposure (CVE) corpus. Our evaluation re-sults have\ndemonstrated the effectiveness of our method in learning domain-specific word\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07484v1",
            "date": "2017-09-21 18:37:36+00:00",
            "title": "WERd: Using Social Text Spelling Variants for Evaluating Dialectal Speech Recognition",
            "authors": [
                "Ahmed Ali",
                "Preslav Nakov",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "We study the problem of evaluating automatic speech recognition (ASR) systems\nthat target dialectal speech input. A major challenge in this case is that the\northography of dialects is typically not standardized. From an ASR evaluation\nperspective, this means that there is no clear gold standard for the expected\noutput, and several possible outputs could be considered correct according to\ndifferent human annotators, which makes standard word error rate (WER)\ninadequate as an evaluation metric. Such a situation is typical for machine\ntranslation (MT), and thus we borrow ideas from an MT evaluation metric, namely\nTERp, an extension of translation error rate which is closely-related to WER.\nIn particular, in the process of comparing a hypothesis to a reference, we make\nuse of spelling variants for words and phrases, which we mine from Twitter in\nan unsupervised fashion. Our experiments with evaluating ASR output for\nEgyptian Arabic, and further manual analysis, show that the resulting WERd\n(i.e., WER for dialects) metric, a variant of TERp, is more adequate than WER\nfor evaluating dialectal ASR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07758v1",
            "date": "2017-09-22 13:59:17+00:00",
            "title": "Improving Language Modelling with Noise-contrastive estimation",
            "authors": [
                "Farhana Ferdousi Liza",
                "Marek Grzes"
            ],
            "abstract": "Neural language models do not scale well when the vocabulary is large.\nNoise-contrastive estimation (NCE) is a sampling-based method that allows for\nfast learning with large vocabularies. Although NCE has shown promising\nperformance in neural machine translation, it was considered to be an\nunsuccessful approach for language modelling. A sufficient investigation of the\nhyperparameters in the NCE-based neural language models was also missing. In\nthis paper, we showed that NCE can be a successful approach in neural language\nmodelling when the hyperparameters of a neural network are tuned appropriately.\nWe introduced the 'search-then-converge' learning rate schedule for NCE and\ndesigned a heuristic that specifies how to use this schedule. The impact of the\nother important hyperparameters, such as the dropout rate and the weight\ninitialisation range, was also demonstrated. We showed that appropriate tuning\nof NCE-based neural language models outperforms the state-of-the-art\nsingle-model methods on a popular benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07777v2",
            "date": "2017-09-22 14:28:44+00:00",
            "title": "Sentence Correction Based on Large-scale Language Modelling",
            "authors": [
                "Ji Wen"
            ],
            "abstract": "With the further development of informatization, more and more data is stored\nin the form of text. There are some loss of text during their generation and\ntransmission. The paper aims to establish a language model based on the\nlarge-scale corpus to complete the restoration of missing text. In this paper,\nwe introduce a novel measurement to find the missing words, and a way of\nestablishing a comprehensive candidate lexicon to insert the correct choice of\nwords. The paper also introduces some effective optimization methods, which\nlargely improve the efficiency of the text restoration and shorten the time of\ndealing with 1000 sentences into 3.6 seconds. \\keywords{ language model,\nsentence correction, word imputation, parallel optimization",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07809v1",
            "date": "2017-09-22 15:28:24+00:00",
            "title": "Neural Machine Translation",
            "authors": [
                "Philipp Koehn"
            ],
            "abstract": "Draft of textbook chapter on neural machine translation. a comprehensive\ntreatment of the topic, ranging from introduction to neural networks,\ncomputation graphs, description of the currently dominant attentional\nsequence-to-sequence model, recent refinements, alternative architectures and\nchallenges. Written as chapter for the textbook Statistical Machine\nTranslation. Used in the JHU Fall 2017 class on machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07840v1",
            "date": "2017-09-22 16:43:48+00:00",
            "title": "Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena",
            "authors": [
                "Igor Shalyminov",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis;\nand it contains many sorts of disfluency such as mid-utterance/sentence\nhesitations, interruptions, and self-corrections. But training data for machine\nlearning approaches to dialogue processing is often either cleaned-up or wholly\nsynthetic in order to avoid such phenomena. The question then arises of how\nwell systems trained on such clean data generalise to real spontaneous\ndialogue, or indeed whether they are trainable at all on naturally occurring\ndialogue data. To answer this question, we created a new corpus called bAbI+ by\nsystematically adding natural spontaneous incremental dialogue phenomena such\nas restarts and self-corrections to the Facebook AI Research's bAbI dialogues\ndataset. We then explore the performance of a state-of-the-art retrieval model,\nMemN2N, on this more natural dataset. Results show that the semantic accuracy\nof the MemN2N model drops drastically; and that although it is in principle\nable to learn to process the constructions in bAbI+, it needs an impractical\namount of training data to do so. Finally, we go on to show that an\nincremental, semantic parser -- DyLan -- shows 100% semantic accuracy on both\nbAbI and bAbI+, highlighting the generalisation properties of linguistically\ninformed dialogue models.",
            "journal ref": "Proceedings of the 21st Workshop on the Semantics and Pragmatics\n  of Dialogue (ISSN 2308-2275), pp 125-133. Saarbrucken, Germany, 15-17 August\n  2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07858v1",
            "date": "2017-09-22 17:24:33+00:00",
            "title": "Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars",
            "authors": [
                "Arash Eshghi",
                "Igor Shalyminov",
                "Oliver Lemon"
            ],
            "abstract": "We investigate an end-to-end method for automatically inducing task-based\ndialogue systems from small amounts of unannotated dialogue data. It combines\nan incremental semantic grammar - Dynamic Syntax and Type Theory with Records\n(DS-TTR) - with Reinforcement Learning (RL), where language generation and\ndialogue management are a joint decision problem. The systems thus produced are\nincremental: dialogues are processed word-by-word, shown previously to be\nessential in supporting natural, spontaneous dialogue. We hypothesised that the\nrich linguistic knowledge within the grammar should enable a combinatorially\nlarge number of dialogue variations to be processed, even when trained on very\nfew dialogues. Our experiments show that our model can process 74% of the\nFacebook AI bAbI dataset even when trained on only 0.13% of the data (5\ndialogues). It can in addition process 65% of bAbI+, a corpus we created by\nsystematically adding incremental dialogue phenomena such as restarts and\nself-corrections to bAbI. We compare our model with a state-of-the-art\nretrieval model, MemN2N. We find that, in terms of semantic accuracy, MemN2N\nshows very poor robustness to the bAbI+ transformations even when trained on\nthe full bAbI dataset.",
            "journal ref": "Proceedings of the 2017 Conference on Empirical Methods in Natural\n  Language Processing (ISBN 978-1-945626-83-8), pp 2210-2220. Copenhagen,\n  Denmark September 7-11, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07862v2",
            "date": "2017-09-22 17:33:32+00:00",
            "title": "Mitigating the Impact of Speech Recognition Errors on Chatbot using Sequence-to-Sequence Model",
            "authors": [
                "Pin-Jung Chen",
                "I-Hung Hsu",
                "Yi-Yao Huang",
                "Hung-Yi Lee"
            ],
            "abstract": "We apply sequence-to-sequence model to mitigate the impact of speech\nrecognition errors on open domain end-to-end dialog generation. We cast the\ntask as a domain adaptation problem where ASR transcriptions and original text\nare in two different domains. In this paper, our proposed model includes two\nindividual encoders for each domain data and make their hidden states similar\nto ensure the decoder predict the same dialog text. The method shows that the\nsequence-to-sequence model can learn the ASR transcriptions and original text\npair having the same meaning and eliminate the speech recognition errors.\nExperimental results on Cornell movie dialog dataset demonstrate that the\ndomain adaption system help the spoken dialog system generate more similar\nresponses with the original text answers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08011v3",
            "date": "2017-09-23 06:15:37+00:00",
            "title": "Long Short-Term Memory for Japanese Word Segmentation",
            "authors": [
                "Yoshiaki Kitagawa",
                "Mamoru Komachi"
            ],
            "abstract": "This study presents a Long Short-Term Memory (LSTM) neural network approach\nto Japanese word segmentation (JWS). Previous studies on Chinese word\nsegmentation (CWS) succeeded in using recurrent neural networks such as LSTM\nand gated recurrent units (GRU). However, in contrast to Chinese, Japanese\nincludes several character types, such as hiragana, katakana, and kanji, that\nproduce orthographic variations and increase the difficulty of word\nsegmentation. Additionally, it is important for JWS tasks to consider a global\ncontext, and yet traditional JWS approaches rely on local features. In order to\naddress this problem, this study proposes employing an LSTM-based approach to\nJWS. The experimental results indicate that the proposed model achieves\nstate-of-the-art accuracy with respect to various Japanese corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08074v1",
            "date": "2017-09-23 16:43:31+00:00",
            "title": "Language Independent Acquisition of Abbreviations",
            "authors": [
                "Michael R. Glass",
                "Md Faisal Mahbub Chowdhury",
                "Alfio M. Gliozzo"
            ],
            "abstract": "This paper addresses automatic extraction of abbreviations (encompassing\nacronyms and initialisms) and corresponding long-form expansions from plain\nunstructured text. We create and are going to release a multilingual resource\nfor abbreviations and their corresponding expansions, built automatically by\nexploiting Wikipedia redirect and disambiguation pages, that can be used as a\nbenchmark for evaluation. We address a shortcoming of previous work where only\nthe redirect pages were used, and so every abbreviation had only a single\nexpansion, even though multiple different expansions are possible for many of\nthe abbreviations. We also develop a principled machine learning based approach\nto scoring expansion candidates using different techniques such as indicators\nof near synonymy, topical relatedness, and surface similarity. We show improved\nperformance over seven languages, including two with a non-Latin alphabet,\nrelative to strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08196v1",
            "date": "2017-09-24 13:48:36+00:00",
            "title": "Identifying Phrasemes via Interlingual Association Measures -- A Data-driven Approach on Dependency-parsed and Word-aligned Parallel Corpora",
            "authors": [
                "Johannes Gra\u00ebn"
            ],
            "abstract": "This is a preprint of the article \"Identifying Phrasemes via Interlingual\nAssociation Measures\" that was presented in February 2016 at the LeKo (Lexical\ncombinations and typified speech in a multilingual context) conference in\nInnsbruck.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08299v2",
            "date": "2017-09-25 03:14:01+00:00",
            "title": "Dataset for the First Evaluation on Chinese Machine Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Ting Liu",
                "Zhipeng Chen",
                "Wentao Ma",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Machine Reading Comprehension (MRC) has become enormously popular recently\nand has attracted a lot of attention. However, existing reading comprehension\ndatasets are mostly in English. To add diversity in reading comprehension\ndatasets, in this paper we propose a new Chinese reading comprehension dataset\nfor accelerating related research in the community. The proposed dataset\ncontains two different types: cloze-style reading comprehension and user query\nreading comprehension, associated with large-scale training data as well as\nhuman-annotated validation and hidden test set. Along with this dataset, we\nalso hosted the first Evaluation on Chinese Machine Reading Comprehension\n(CMRC-2017) and successfully attracted tens of participants, which suggest the\npotential impact of this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08521v1",
            "date": "2017-09-25 14:40:28+00:00",
            "title": "Using objective words in the reviews to improve the colloquial arabic sentiment analysis",
            "authors": [
                "Omar Al-Harbi"
            ],
            "abstract": "One of the main difficulties in sentiment analysis of the Arabic language is\nthe presence of the colloquialism. In this paper, we examine the effect of\nusing objective words in conjunction with sentimental words on sentiment\nclassification for the colloquial Arabic reviews, specifically Jordanian\ncolloquial reviews. The reviews often include both sentimental and objective\nwords, however, the most existing sentiment analysis models ignore the\nobjective words as they are considered useless. In this work, we created two\nlexicons: the first includes the colloquial sentimental words and compound\nphrases, while the other contains the objective words associated with values of\nsentiment tendency based on a particular estimation method. We used these\nlexicons to extract sentiment features that would be training input to the\nSupport Vector Machines (SVM) to classify the sentiment polarity of the\nreviews. The reviews dataset have been collected manually from JEERAN website.\nThe results of the experiments show that the proposed approach improves the\npolarity classification in comparison to two baseline models, with accuracy\n95.6%.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  6, No.3, June 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08698v1",
            "date": "2017-09-20 03:51:19+00:00",
            "title": "Identifying Restaurant Features via Sentiment Analysis on Yelp Reviews",
            "authors": [
                "Boya Yu",
                "Jiaxu Zhou",
                "Yi Zhang",
                "Yunong Cao"
            ],
            "abstract": "Many people use Yelp to find a good restaurant. Nonetheless, with only an\noverall rating for each restaurant, Yelp offers not enough information for\nindependently judging its various aspects such as environment, service or\nflavor. In this paper, we introduced a machine learning based method to\ncharacterize such aspects for particular types of restaurants. The main\napproach used in this paper is to use a support vector machine (SVM) model to\ndecipher the sentiment tendency of each review from word frequency. Word scores\ngenerated from the SVM models are further processed into a polarity index\nindicating the significance of each word for special types of restaurant.\nCustomers overall tend to express more sentiment regarding service. As for the\ndistinction between different cuisines, results that match the common sense are\nobtained: Japanese cuisines are usually fresh, some French cuisines are\noverpriced while Italian Restaurants are often famous for their pizzas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08716v1",
            "date": "2017-09-25 20:36:42+00:00",
            "title": "DOC: Deep Open Classification of Text Documents",
            "authors": [
                "Lei Shu",
                "Hu Xu",
                "Bing Liu"
            ],
            "abstract": "Traditional supervised learning makes the closed-world assumption that the\nclasses appeared in the test data must have appeared in training. This also\napplies to text learning or text classification. As learning is used\nincreasingly in dynamic open environments where some new/test documents may not\nbelong to any of the training classes, identifying these novel documents during\nclassification presents an important problem. This problem is called open-world\nclassification or open classification. This paper proposes a novel deep\nlearning based approach. It outperforms existing state-of-the-art techniques\ndramatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08898v2",
            "date": "2017-09-26 09:04:29+00:00",
            "title": "Improving a Multi-Source Neural Machine Translation Model with Corpus Extension for Low-Resource Languages",
            "authors": [
                "Gyu-Hyeon Choi",
                "Jong-Hun Shin",
                "Young-Kil Kim"
            ],
            "abstract": "In machine translation, we often try to collect resources to improve\nperformance. However, most of the language pairs, such as Korean-Arabic and\nKorean-Vietnamese, do not have enough resources to train machine translation\nsystems. In this paper, we propose the use of synthetic methods for extending a\nlow-resource corpus and apply it to a multi-source neural machine translation\nmodel. We showed the improvement of machine translation performance through\ncorpus extension using the synthetic method. We specifically focused on how to\ncreate source sentences that can make better target sentences, including the\nuse of synthetic methods. We found that the corpus extension could also improve\nthe performance of multi-source neural machine translation. We showed the\ncorpus extension and multi-source model to be efficient methods for a\nlow-resource language pair. Furthermore, when both methods were used together,\nwe found better machine translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08907v2",
            "date": "2017-09-26 09:28:49+00:00",
            "title": "Input-to-Output Gate to Improve RNN Language Models",
            "authors": [
                "Sho Takase",
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "This paper proposes a reinforcing method that refines the output layers of\nexisting Recurrent Neural Network (RNN) language models. We refer to our\nproposed method as Input-to-Output Gate (IOG). IOG has an extremely simple\nstructure, and thus, can be easily combined with any RNN language models. Our\nexperiments on the Penn Treebank and WikiText-2 datasets demonstrate that IOG\nconsistently boosts the performance of several different types of current\ntopline RNN language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09220v1",
            "date": "2017-09-26 18:54:39+00:00",
            "title": "Dataset Construction via Attention for Aspect Term Extraction with Distant Supervision",
            "authors": [
                "Athanasios Giannakopoulos",
                "Diego Antognini",
                "Claudiu Musat",
                "Andreea Hossmann",
                "Michael Baeriswyl"
            ],
            "abstract": "Aspect Term Extraction (ATE) detects opinionated aspect terms in sentences or\ntext spans, with the end goal of performing aspect-based sentiment analysis.\nThe small amount of available datasets for supervised ATE and the fact that\nthey cover only a few domains raise the need for exploiting other data sources\nin new and creative ways. Publicly available review corpora contain a plethora\nof opinionated aspect terms and cover a larger domain spectrum. In this paper,\nwe first propose a method for using such review corpora for creating a new\ndataset for ATE. Our method relies on an attention mechanism to select\nsentences that have a high likelihood of containing actual opinionated aspects.\nWe thus improve the quality of the extracted aspects. We then use the\nconstructed dataset to train a model and perform ATE with distant supervision.\nBy evaluating on human annotated datasets, we prove that our method achieves a\nsignificantly improved performance over various unsupervised and supervised\nbaselines. Finally, we prove that sentence selection matters when it comes to\ncreating new datasets for ATE. Specifically, we show that, using a set of\nselected sentences leads to higher ATE performance compared to using the whole\nsentence set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09239v1",
            "date": "2017-09-26 19:59:16+00:00",
            "title": "Predicting Disease-Gene Associations using Cross-Document Graph-based Features",
            "authors": [
                "Hendrik ter Horst",
                "Matthias Hartung",
                "Roman Klinger",
                "Matthias Zwick",
                "Philipp Cimiano"
            ],
            "abstract": "In the context of personalized medicine, text mining methods pose an\ninteresting option for identifying disease-gene associations, as they can be\nused to generate novel links between diseases and genes which may complement\nknowledge from structured databases. The most straightforward approach to\nextract such links from text is to rely on a simple assumption postulating an\nassociation between all genes and diseases that co-occur within the same\ndocument. However, this approach (i) tends to yield a number of spurious\nassociations, (ii) does not capture different relevant types of associations,\nand (iii) is incapable of aggregating knowledge that is spread across\ndocuments. Thus, we propose an approach in which disease-gene co-occurrences\nand gene-gene interactions are represented in an RDF graph. A machine\nlearning-based classifier is trained that incorporates features extracted from\nthe graph to separate disease-gene pairs into valid disease-gene associations\nand spurious ones. On the manually curated Genetic Testing Registry, our\napproach yields a 30 points increase in F1 score over a plain co-occurrence\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09254v1",
            "date": "2017-09-26 20:28:18+00:00",
            "title": "Learning to Explain Non-Standard English Words and Phrases",
            "authors": [
                "Ke Ni",
                "William Yang Wang"
            ],
            "abstract": "We describe a data-driven approach for automatically explaining new,\nnon-standard English expressions in a given sentence, building on a large\ndataset that includes 15 years of crowdsourced examples from\nUrbanDictionary.com. Unlike prior studies that focus on matching keywords from\na slang dictionary, we investigate the possibility of learning a neural\nsequence-to-sequence model that generates explanations of unseen non-standard\nEnglish expressions given context. We propose a dual encoder approach---a\nword-level encoder learns the representation of context, and a second\ncharacter-level encoder to learn the hidden representation of the target\nnon-standard expression. Our model can produce reasonable definitions of new\nnon-standard English expressions given their context with certain confidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09360v3",
            "date": "2017-09-27 07:06:05+00:00",
            "title": "Learning of Colors from Color Names: Distribution and Point Estimation",
            "authors": [
                "Lyndon White",
                "Roberto Togneri",
                "Wei Liu",
                "Mohammed Bennamoun"
            ],
            "abstract": "Color names are often made up of multiple words. As a task in natural\nlanguage understanding we investigate in depth the capacity of neural networks\nbased on sums of word embeddings (SOWE), recurrence (LSTM and GRU based RNNs)\nand convolution (CNN), to estimate colors from sequences of terms. We consider\nboth point and distribution estimates of color. We argue that the latter has a\nparticular value as there is no clear agreement between people as to what a\nparticular color describes -- different people have a different idea of what it\nmeans to be ``very dark orange'', for example. Surprisingly, despite it's\nsimplicity, the sum of word embeddings generally performs the best on almost\nall evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09443v1",
            "date": "2017-09-27 10:50:12+00:00",
            "title": "Prosodic Features from Large Corpora of Child-Directed Speech as Predictors of the Age of Acquisition of Words",
            "authors": [
                "Lea Frermann",
                "Michael C. Frank"
            ],
            "abstract": "The impressive ability of children to acquire language is a widely studied\nphenomenon, and the factors influencing the pace and patterns of word learning\nremains a subject of active research. Although many models predicting the age\nof acquisition of words have been proposed, little emphasis has been directed\nto the raw input children achieve. In this work we present a comparatively\nlarge-scale multi-modal corpus of prosody-text aligned child directed speech.\nOur corpus contains automatically extracted word-level prosodic features, and\nwe investigate the utility of this information as predictors of age of\nacquisition. We show that prosody features boost predictive power in a\nregularized regression, and demonstrate their utility in the context of a\nmulti-modal factorized language models trained and tested on child-directed\nspeech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09500v1",
            "date": "2017-09-27 13:31:41+00:00",
            "title": "Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets",
            "authors": [
                "Rotem Dror",
                "Gili Baumer",
                "Marina Bogomolov",
                "Roi Reichart"
            ],
            "abstract": "With the ever-growing amounts of textual data from a large variety of\nlanguages, domains, and genres, it has become standard to evaluate NLP\nalgorithms on multiple datasets in order to ensure consistent performance\nacross heterogeneous setups. However, such multiple comparisons pose\nsignificant challenges to traditional statistical analysis methods in NLP and\ncan lead to erroneous conclusions. In this paper, we propose a Replicability\nAnalysis framework for a statistically sound analysis of multiple comparisons\nbetween algorithms for NLP tasks. We discuss the theoretical advantages of this\nframework over the current, statistically unjustified, practice in the NLP\nliterature, and demonstrate its empirical value across four applications:\nmulti-domain dependency parsing, multilingual POS tagging, cross-domain\nsentiment classification and word similarity prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09590v2",
            "date": "2017-09-27 15:50:53+00:00",
            "title": "An attentive neural architecture for joint segmentation and parsing and its application to real estate ads",
            "authors": [
                "Giannis Bekoulis",
                "Johannes Deleu",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "In processing human produced text using natural language processing (NLP)\ntechniques, two fundamental subtasks that arise are (i) segmentation of the\nplain text into meaningful subunits (e.g., entities), and (ii) dependency\nparsing, to establish relations between subunits. In this paper, we develop a\nrelatively simple and effective neural joint model that performs both\nsegmentation and dependency parsing together, instead of one after the other as\nin most state-of-the-art works. We will focus in particular on the real estate\nad setting, aiming to convert an ad to a structured description, which we name\nproperty tree, comprising the tasks of (1) identifying important entities of a\nproperty (e.g., rooms) from classifieds and (2) structuring them into a tree\nformat. In this work, we propose a new joint model that is able to tackle the\ntwo tasks simultaneously and construct the property tree by (i) avoiding the\nerror propagation that would arise from the subtasks one after the other in a\npipelined fashion, and (ii) exploiting the interactions between the subtasks.\nFor this purpose, we perform an extensive comparative study of the pipeline\nmethods and the new proposed joint model, reporting an improvement of over\nthree percentage points in the overall edge F1 score of the property tree.\nAlso, we propose attention methods, to encourage our model to focus on salient\ntokens during the construction of the property tree. Thus we experimentally\ndemonstrate the usefulness of attentive neural architectures for the proposed\njoint model, showcasing a further improvement of two percentage points in edge\nF1 score for our application.",
            "journal ref": "Expert Systems with Applications, Volume 102, 15 July 2018, Pages\n  100-112, ISSN 0957-4174",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09686v2",
            "date": "2017-09-27 18:18:32+00:00",
            "title": "Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition",
            "authors": [
                "L. T. Anh",
                "M. Y. Arkhipov",
                "M. S. Burtsev"
            ],
            "abstract": "Named Entity Recognition (NER) is one of the most common tasks of the natural\nlanguage processing. The purpose of NER is to find and classify tokens in text\ndocuments into predefined categories called tags, such as person names,\nquantity expressions, percentage expressions, names of locations,\norganizations, as well as expression of time, currency and others. Although\nthere is a number of approaches have been proposed for this task in Russian\nlanguage, it still has a substantial potential for the better solutions. In\nthis work, we studied several deep neural network models starting from vanilla\nBi-directional Long Short-Term Memory (Bi-LSTM) then supplementing it with\nConditional Random Fields (CRF) as well as highway networks and finally adding\nexternal word embeddings. All models were evaluated across three datasets:\nGareev's dataset, Person-1000, FactRuEval-2016. We found that extension of\nBi-LSTM model with CRF significantly increased the quality of predictions.\nEncoding input tokens with external word embeddings reduced training time and\nallowed to achieve state of the art for the Russian NER task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09783v1",
            "date": "2017-09-28 02:09:04+00:00",
            "title": "A Deep Neural Network Approach To Parallel Sentence Extraction",
            "authors": [
                "Francis Gr\u00e9goire",
                "Philippe Langlais"
            ],
            "abstract": "Parallel sentence extraction is a task addressing the data sparsity problem\nfound in multilingual natural language processing applications. We propose an\nend-to-end deep neural network approach to detect translational equivalence\nbetween sentences in two different languages. In contrast to previous\napproaches, which typically rely on multiples models and various word alignment\nfeatures, by leveraging continuous vector representation of sentences we remove\nthe need of any domain specific feature engineering. Using a siamese\nbidirectional recurrent neural networks, our results against a strong baseline\nbased on a state-of-the-art parallel sentence extraction system show a\nsignificant improvement in both the quality of the extracted parallel sentences\nand the translation performance of statistical machine translation systems. We\nbelieve this study is the first one to investigate deep learning for the\nparallel sentence extraction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09885v2",
            "date": "2017-09-28 10:35:41+00:00",
            "title": "Sentiment Classification with Word Attention based on Weakly Supervised Learning with a Convolutional Neural Network",
            "authors": [
                "Gichang Lee",
                "Jaeyun Jeong",
                "Seungwan Seo",
                "CzangYeob Kim",
                "Pilsung Kang"
            ],
            "abstract": "In order to maximize the applicability of sentiment analysis results, it is\nnecessary to not only classify the overall sentiment (positive/negative) of a\ngiven document but also to identify the main words that contribute to the\nclassification. However, most datasets for sentiment analysis only have the\nsentiment label for each document or sentence. In other words, there is no\ninformation about which words play an important role in sentiment\nclassification. In this paper, we propose a method for identifying key words\ndiscriminating positive and negative sentences by using a weakly supervised\nlearning method based on a convolutional neural network (CNN). In our model,\neach word is represented as a continuous-valued vector and each sentence is\nrepresented as a matrix whose rows correspond to the word vector used in the\nsentence. Then, the CNN model is trained using these sentence matrices as\ninputs and the sentiment labels as the output. Once the CNN model is trained,\nwe implement the word attention mechanism that identifies high-contributing\nwords to classification results with a class activation map, using the weights\nfrom the fully connected layer at the end of the learned CNN model. In order to\nverify the proposed methodology, we evaluated the classification accuracy and\ninclusion rate of polarity words using two movie review datasets. Experimental\nresult show that the proposed model can not only correctly classify the\nsentence polarity but also successfully identify the corresponding words with\nhigh polarity scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10053v2",
            "date": "2017-09-28 16:44:22+00:00",
            "title": "Graph Convolutional Networks for Named Entity Recognition",
            "authors": [
                "A. Cetoli",
                "S. Bragaglia",
                "A. D. O'Harney",
                "M. Sloan"
            ],
            "abstract": "In this paper we investigate the role of the dependency tree in a named\nentity recognizer upon using a set of GCN. We perform a comparison among\ndifferent NER architectures and show that the grammar of a sentence positively\ninfluences the results. Experiments on the ontonotes dataset demonstrate\nconsistent performance improvements, without requiring heavy feature\nengineering nor additional language-specific knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10159v1",
            "date": "2017-09-28 20:31:30+00:00",
            "title": "A Web of Hate: Tackling Hateful Speech in Online Social Spaces",
            "authors": [
                "Haji Mohammad Saleem",
                "Kelly P Dillon",
                "Susan Benesch",
                "Derek Ruths"
            ],
            "abstract": "Online social platforms are beset with hateful speech - content that\nexpresses hatred for a person or group of people. Such content can frighten,\nintimidate, or silence platform users, and some of it can inspire other users\nto commit violence. Despite widespread recognition of the problems posed by\nsuch content, reliable solutions even for detecting hateful speech are lacking.\nIn the present work, we establish why keyword-based methods are insufficient\nfor detection. We then propose an approach to detecting hateful speech that\nuses content produced by self-identifying hateful communities as training data.\nOur approach bypasses the expensive annotation process often required to train\nkeyword systems and performs well across several established platforms, making\nsubstantial improvements over current state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10191v1",
            "date": "2017-09-28 22:40:07+00:00",
            "title": "Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks",
            "authors": [
                "Mingbo Ma",
                "Kai Zhao",
                "Liang Huang",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "Sentence-level classification and sequential labeling are two fundamental\ntasks in language understanding. While these two tasks are usually modeled\nseparately, in reality, they are often correlated, for example in intent\nclassification and slot filling, or in topic classification and named-entity\nrecognition. In order to utilize the potential benefits from their\ncorrelations, we propose a jointly trained model for learning the two tasks\nsimultaneously via Long Short-Term Memory (LSTM) networks. This model predicts\nthe sentence-level category and the word-level label sequence from the stepwise\noutput hidden representations of LSTM. We also introduce a novel mechanism of\n\"sparse attention\" to weigh words differently based on their semantic relevance\nto sentence-level classification. The proposed method outperforms baseline\nmodels on ATIS and TREC datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10217v2",
            "date": "2017-09-29 02:35:28+00:00",
            "title": "The First Evaluation of Chinese Human-Computer Dialogue Technology",
            "authors": [
                "Wei-Nan Zhang",
                "Zhigang Chen",
                "Wanxiang Che",
                "Guoping Hu",
                "Ting Liu"
            ],
            "abstract": "In this paper, we introduce the first evaluation of Chinese human-computer\ndialogue technology. We detail the evaluation scheme, tasks, metrics and how to\ncollect and annotate the data for training, developing and test. The evaluation\nincludes two tasks, namely user intent classification and online testing of\ntask-oriented dialogue. To consider the different sources of the data for\ntraining and developing, the first task can also be divided into two sub tasks.\nBoth the two tasks are coming from the real problems when using the\napplications developed by industry. The evaluation data is provided by the\niFLYTEK Corporation. Meanwhile, in this paper, we publish the evaluation\nresults to present the current performance of the participants in the two tasks\nof Chinese human-computer dialogue technology. Moreover, we analyze the\nexisting problems of human-computer dialogue as well as the evaluation scheme\nitself.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10381v1",
            "date": "2017-09-29 12:58:05+00:00",
            "title": "Towards Universal Semantic Tagging",
            "authors": [
                "Lasha Abzianidze",
                "Johan Bos"
            ],
            "abstract": "The paper proposes the task of universal semantic tagging---tagging word\ntokens with language-neutral, semantically informative tags. We argue that the\ntask, with its independent nature, contributes to better semantic analysis for\nwide-coverage multilingual text. We present the initial version of the semantic\ntagset and show that (a) the tags provide semantically fine-grained\ninformation, and (b) they are suitable for cross-lingual semantic parsing. An\napplication of the semantic tagging in the Parallel Meaning Bank supports both\nof these points as the tags contribute to formal lexical semantics and their\ncross-lingual projection. As a part of the application, we annotate a small\ncorpus with the semantic tags and present new baseline result for universal\nsemantic tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10486v1",
            "date": "2017-09-29 16:36:53+00:00",
            "title": "Symbol, Conversational, and Societal Grounding with a Toy Robot",
            "authors": [
                "Casey Kennington",
                "Sarah Plane"
            ],
            "abstract": "Essential to meaningful interaction is grounding at the symbolic,\nconversational, and societal levels. We present ongoing work with Anki's Cozmo\ntoy robot as a research platform where we leverage the recent\nwords-as-classifiers model of lexical semantics in interactive reference\nresolution tasks for language grounding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00049v2",
            "date": "2017-12-29 22:21:39+00:00",
            "title": "Personal Names in Modern Turkey",
            "authors": [
                "Ama\u00e7 Herda\u011fdelen"
            ],
            "abstract": "We analyzed the most common 5000 male and 5000 female Turkish names based on\ntheir etymological, morphological, and semantic attributes. The name statistics\nare based on all Turkish citizens who were alive in 2014 and they cover 90% of\nall population. To the best of our knowledge, this study is the most\ncomprehensive data-driven analysis of Turkish personal names. Female names have\na greater diversity than male names (e.g., top 15 male names cover 25% of the\nmale population, whereas top 28 female names cover 25% of the female\npopulation). Despite their diversity, female names exhibit predictable\npatterns. For example, certain roots such as g\\\"ul and nar (rose and\npomegranate/red, respectively) are used to generate hundreds of unique female\nnames. Turkish personal names have their origins mainly in Arabic, followed by\nTurkish and Persian. We computed overall frequencies of names according to\nbroad semantic themes that were identified in previous studies. We found that\nforeign-origin names such as olga and khaled, pastoral names such as ya\\u{g}mur\nand deniz (rain and sea, respectively), and names based on fruits and plants\nsuch as filiz and menek\\c{s}e (sprout and violet, respectively) are more\nfrequently observed among females. Among males, names based on animals such as\narslan and yunus (lion and dolphin, respectively) and names based on famous\nand/or historical figures such as mustafa kemal and o\\u{g}uz ka\\u{g}an (founder\nof the Turkish Republic and the founder of the Turks in Turkish mythology,\nrespectively) are observed more frequently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00059v2",
            "date": "2017-12-29 23:31:05+00:00",
            "title": "The CAPIO 2017 Conversational Speech Recognition System",
            "authors": [
                "Kyu J. Han",
                "Akshay Chandrashekaran",
                "Jungsuk Kim",
                "Ian Lane"
            ],
            "abstract": "In this paper we show how we have achieved the state-of-the-art performance\non the industry-standard NIST 2000 Hub5 English evaluation set. We explore\ndensely connected LSTMs, inspired by the densely connected convolutional\nnetworks recently introduced for image classification tasks. We also propose an\nacoustic model adaptation scheme that simply averages the parameters of a seed\nneural network acoustic model and its adapted version. This method was applied\nwith the CallHome training corpus and improved individual system performances\nby on average 6.1% (relative) against the CallHome portion of the evaluation\nset with no performance loss on the Switchboard portion. With RNN-LM rescoring\nand lattice combination on the 5 systems trained across three different phone\nsets, our 2017 speech recognition system has obtained 5.0% and 9.1% on\nSwitchboard and CallHome, respectively, both of which are the best word error\nrates reported thus far. According to IBM in their latest work to compare human\nand machine transcriptions, our reported Switchboard word error rate can be\nconsidered to surpass the human parity (5.1%) of transcribing conversational\ntelephone speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00076v6",
            "date": "2017-12-30 02:38:07+00:00",
            "title": "Bidirectional Attention for SQL Generation",
            "authors": [
                "Tong Guo",
                "Huilin Gao"
            ],
            "abstract": "Generating structural query language (SQL) queries from natural language is a\nlong-standing open problem. Answering a natural language question about a\ndatabase table requires modeling complex interactions between the columns of\nthe table and the question. In this paper, we apply the synthesizing approach\nto solve this problem. Based on the structure of SQL queries, we break down the\nmodel to three sub-modules and design specific deep neural networks for each of\nthem. Taking inspiration from the similar machine reading task, we employ the\nbidirectional attention mechanisms and character-level embedding with\nconvolutional neural networks (CNNs) to improve the result. Experimental\nevaluations show that our model achieves the state-of-the-art results in\nWikiSQL dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00254v1",
            "date": "2017-12-31 08:44:51+00:00",
            "title": "A New Approach for Measuring Sentiment Orientation based on Multi-Dimensional Vector Space",
            "authors": [
                "Youngsam Kim",
                "Hyopil Shin"
            ],
            "abstract": "This study implements a vector space model approach to measure the sentiment\norientations of words. Two representative vectors for positive/negative\npolarity are constructed using high-dimensional vec-tor space in both an\nunsupervised and a semi-supervised manner. A sentiment ori-entation value per\nword is determined by taking the difference between the cosine distances\nagainst the two reference vec-tors. These two conditions (unsupervised and\nsemi-supervised) are compared against an existing unsupervised method (Turney,\n2002). As a result of our experi-ment, we demonstrate that this novel ap-proach\nsignificantly outperforms the pre-vious unsupervised approach and is more\npractical and data efficient as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00051v2",
            "date": "2017-03-31 20:39:10+00:00",
            "title": "Reading Wikipedia to Answer Open-Domain Questions",
            "authors": [
                "Danqi Chen",
                "Adam Fisch",
                "Jason Weston",
                "Antoine Bordes"
            ],
            "abstract": "This paper proposes to tackle open- domain question answering using Wikipedia\nas the unique knowledge source: the answer to any factoid question is a text\nspan in a Wikipedia article. This task of machine reading at scale combines the\nchallenges of document retrieval (finding the relevant articles) with that of\nmachine comprehension of text (identifying the answer spans from those\narticles). Our approach combines a search component based on bigram hashing and\nTF-IDF matching with a multi-layer recurrent neural network model trained to\ndetect answers in Wikipedia paragraphs. Our experiments on multiple existing QA\ndatasets indicate that (1) both modules are highly competitive with respect to\nexisting counterparts and (2) multitask learning using distant supervision on\ntheir combination is an effective complete system on this challenging task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00052v1",
            "date": "2017-03-31 20:39:38+00:00",
            "title": "One-Shot Neural Cross-Lingual Transfer for Paradigm Completion",
            "authors": [
                "Katharina Kann",
                "Ryan Cotterell",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present a novel cross-lingual transfer method for paradigm completion, the\ntask of mapping a lemma to its inflected forms, using a neural encoder-decoder\nmodel, the state of the art for the monolingual task. We use labeled data from\na high-resource language to increase performance on a low-resource language. In\nexperiments on 21 language pairs from four different language families, we\nobtain up to 58% higher accuracy than without transfer and show that even\nzero-shot and one-shot learning are possible. We further find that the degree\nof language relatedness strongly influences the ability to transfer\nmorphological knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00057v2",
            "date": "2017-03-31 21:03:58+00:00",
            "title": "Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systems",
            "authors": [
                "Layla El Asri",
                "Hannes Schulz",
                "Shikhar Sharma",
                "Jeremie Zumer",
                "Justin Harris",
                "Emery Fine",
                "Rahul Mehrotra",
                "Kaheer Suleman"
            ],
            "abstract": "This paper presents the Frames dataset (Frames is available at\nhttp://datasets.maluuba.com/Frames), a corpus of 1369 human-human dialogues\nwith an average of 15 turns per dialogue. We developed this dataset to study\nthe role of memory in goal-oriented dialogue systems. Based on Frames, we\nintroduce a task called frame tracking, which extends state tracking to a\nsetting where several states are tracked simultaneously. We propose a baseline\nmodel for this task. We show that Frames can also be used to study memory in\ndialogue management and information presentation through natural language\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00177v1",
            "date": "2017-04-01 14:53:54+00:00",
            "title": "Sentiment Analysis of Citations Using Word2vec",
            "authors": [
                "Haixia Liu"
            ],
            "abstract": "Citation sentiment analysis is an important task in scientific paper\nanalysis. Existing machine learning techniques for citation sentiment analysis\nare focusing on labor-intensive feature engineering, which requires large\nannotated corpus. As an automatic feature extraction tool, word2vec has been\nsuccessfully applied to sentiment analysis of short texts. In this work, I\nconducted empirical research with the question: how well does word2vec work on\nthe sentiment analysis of citations? The proposed method constructed sentence\nvectors (sent2vec) by averaging the word embeddings, which were learned from\nAnthology Collections (ACL-Embeddings). I also investigated polarity-specific\nword embeddings (PS-Embeddings) for classifying positive and negative\ncitations. The sentence vectors formed a feature space, to which the examined\ncitation sentence was mapped to. Those features were input into classifiers\n(support vector machines) for supervised classification. Using\n10-cross-validation scheme, evaluation was conducted on a set of annotated\ncitations. The results showed that word embeddings are effective on classifying\npositive and negative citations. However, hand-crafted features performed\nbetter for the overall classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00200v3",
            "date": "2017-04-01 17:05:35+00:00",
            "title": "Towards Building Large Scale Multimodal Domain-Aware Conversation Systems",
            "authors": [
                "Amrita Saha",
                "Mitesh Khapra",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "While multimodal conversation agents are gaining importance in several\ndomains such as retail, travel etc., deep learning research in this area has\nbeen limited primarily due to the lack of availability of large-scale, open\nchatlogs. To overcome this bottleneck, in this paper we introduce the task of\nmultimodal, domain-aware conversations, and propose the MMD benchmark dataset.\nThis dataset was gathered by working in close coordination with large number of\ndomain experts in the retail domain. These experts suggested various\nconversations flows and dialog states which are typically seen in multimodal\nconversations in the fashion domain. Keeping these flows and states in mind, we\ncreated a dataset consisting of over 150K conversation sessions between\nshoppers and sales agents, with the help of in-house annotators using a\nsemi-automated manually intense iterative process. With this dataset, we\npropose 5 new sub-tasks for multimodal conversations along with their\nevaluation methodology. We also propose two multimodal neural models in the\nencode-attend-decode paradigm and demonstrate their performance on two of the\nsub-tasks, namely text response generation and best image response selection.\nThese experiments serve to establish baseline performance and open new research\ndirections for each of these sub-tasks. Further, for each of the sub-tasks, we\npresent a `per-state evaluation' of 9 most significant dialog states, which\nwould enable more focused research into understanding the challenges and\ncomplexities involved in each of these states.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00253v4",
            "date": "2017-04-02 05:54:14+00:00",
            "title": "Building a Neural Machine Translation System Using Only Synthetic Parallel Data",
            "authors": [
                "Jaehong Park",
                "Jongyoon Song",
                "Sungroh Yoon"
            ],
            "abstract": "Recent works have shown that synthetic parallel data automatically generated\nby translation models can be effective for various neural machine translation\n(NMT) issues. In this study, we build NMT systems using only synthetic parallel\ndata. As an efficient alternative to real parallel data, we also present a new\ntype of synthetic parallel corpus. The proposed pseudo parallel data are\ndistinct from previous works in that ground truth and synthetic examples are\nmixed on both sides of sentence pairs. Experiments on Czech-German and\nFrench-German translations demonstrate the efficacy of the proposed pseudo\nparallel corpus, which shows not only enhanced results for bidirectional\ntranslation tasks but also substantial improvement with the aid of a ground\ntruth real parallel corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00380v1",
            "date": "2017-04-02 22:36:56+00:00",
            "title": "Word-Alignment-Based Segment-Level Machine Translation Evaluation using Word Embeddings",
            "authors": [
                "Junki Matsuo",
                "Mamoru Komachi",
                "Katsuhito Sudoh"
            ],
            "abstract": "One of the most important problems in machine translation (MT) evaluation is\nto evaluate the similarity between translation hypotheses with different\nsurface forms from the reference, especially at the segment level. We propose\nto use word embeddings to perform word alignment for segment-level MT\nevaluation. We performed experiments with three types of alignment methods\nusing word embeddings. We evaluated our proposed methods with various\ntranslation datasets. Experimental results show that our proposed methods\noutperform previous word embeddings-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00405v2",
            "date": "2017-04-03 02:10:19+00:00",
            "title": "Syntax Aware LSTM Model for Chinese Semantic Role Labeling",
            "authors": [
                "Feng Qian",
                "Lei Sha",
                "Baobao Chang",
                "Lu-chen Liu",
                "Ming Zhang"
            ],
            "abstract": "As for semantic role labeling (SRL) task, when it comes to utilizing parsing\ninformation, both traditional methods and recent recurrent neural network (RNN)\nbased methods use the feature engineering way. In this paper, we propose Syntax\nAware Long Short Time Memory(SA-LSTM). The structure of SA-LSTM modifies\naccording to dependency parsing information in order to model parsing\ninformation directly in an architecture engineering way instead of feature\nengineering way. We experimentally demonstrate that SA-LSTM gains more\nimprovement from the model architecture. Furthermore, SA-LSTM outperforms the\nstate-of-the-art on CPB 1.0 significantly according to Student t-test\n($p<0.05$).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00440v1",
            "date": "2017-04-03 06:22:04+00:00",
            "title": "Combining Lexical and Syntactic Features for Detecting Content-dense Texts in News",
            "authors": [
                "Yinfei Yang",
                "Ani Nenkova"
            ],
            "abstract": "Content-dense news report important factual information about an event in\ndirect, succinct manner. Information seeking applications such as information\nextraction, question answering and summarization normally assume all text they\ndeal with is content-dense. Here we empirically test this assumption on news\narticles from the business, U.S. international relations, sports and science\njournalism domains. Our findings clearly indicate that about half of the news\ntexts in our study are in fact not content-dense and motivate the development\nof a supervised content-density detector. We heuristically label a large\ntraining corpus for the task and train a two-layer classifying model based on\nlexical and unlexicalized syntactic features. On manually annotated data, we\ncompare the performance of domain-specific classifiers, trained on data only\nfrom a given news domain and a general classifier in which data from all four\ndomains is pooled together. Our annotation and prediction experiments\ndemonstrate that the concept of content density varies depending on the domain\nand that naive annotators provide judgement biased toward the stereotypical\ndomain label. Domain-specific classifiers are more accurate for domains in\nwhich content-dense texts are typically fewer. Domain independent classifiers\nreproduce better naive crowdsourced judgements. Classification prediction is\nhigh across all conditions, around 80%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00552v2",
            "date": "2017-04-03 12:40:54+00:00",
            "title": "A Transition-Based Directed Acyclic Graph Parser for UCCA",
            "authors": [
                "Daniel Hershcovich",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "We present the first parser for UCCA, a cross-linguistically applicable\nframework for semantic representation, which builds on extensive typological\nwork and supports rapid annotation. UCCA poses a challenge for existing parsing\ntechniques, as it exhibits reentrancy (resulting in DAG structures),\ndiscontinuous structures and non-terminal nodes corresponding to complex\nsemantic units. To our knowledge, the conjunction of these formal properties is\nnot supported by any existing parser. Our transition-based parser, which uses a\nnovel transition set and features based on bidirectional LSTMs, has value not\njust for UCCA parsing: its ability to handle more general graph structures can\ninform the development of parsers for other semantic DAG structures, and in\nlanguages that frequently use discontinuous structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00559v2",
            "date": "2017-04-03 13:03:40+00:00",
            "title": "Neural Lattice-to-Sequence Models for Uncertain Inputs",
            "authors": [
                "Matthias Sperber",
                "Graham Neubig",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "The input to a neural sequence-to-sequence model is often determined by an\nup-stream system, e.g. a word segmenter, part of speech tagger, or speech\nrecognizer. These up-stream models are potentially error-prone. Representing\ninputs through word lattices allows making this uncertainty explicit by\ncapturing alternative sequences and their posterior probabilities in a compact\nform. In this work, we extend the TreeLSTM (Tai et al., 2015) into a\nLatticeLSTM that is able to consume word lattices, and can be used as encoder\nin an attentional encoder-decoder model. We integrate lattice posterior scores\ninto this architecture by extending the TreeLSTM's child-sum and forget gates\nand introducing a bias term into the attention mechanism. We experiment with\nspeech translation lattices and report consistent improvements over baselines\nthat translate either the 1-best hypothesis or the lattice without posterior\nscores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00774v3",
            "date": "2017-04-03 19:17:58+00:00",
            "title": "Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality",
            "authors": [
                "Alexandre Salle",
                "Aline Villavicencio"
            ],
            "abstract": "Increasing the capacity of recurrent neural networks (RNN) usually involves\naugmenting the size of the hidden layer, with significant increase of\ncomputational cost. Recurrent neural tensor networks (RNTN) increase capacity\nusing distinct hidden layer weights for each word, but with greater costs in\nmemory usage. In this paper, we introduce restricted recurrent neural tensor\nnetworks (r-RNTN) which reserve distinct hidden layer weights for frequent\nvocabulary words while sharing a single set of weights for infrequent words.\nPerplexity evaluations show that for fixed hidden layer sizes, r-RNTNs improve\nlanguage model performance over RNNs using only a small fraction of the\nparameters of unrestricted RNTNs. These results hold for r-RNTNs using Gated\nRecurrent Units and Long Short-Term Memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00849v3",
            "date": "2017-04-04 01:47:14+00:00",
            "title": "Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks",
            "authors": [
                "Chin-Cheng Hsu",
                "Hsin-Te Hwang",
                "Yi-Chiao Wu",
                "Yu Tsao",
                "Hsin-Min Wang"
            ],
            "abstract": "Building a voice conversion (VC) system from non-parallel speech corpora is\nchallenging but highly valuable in real application scenarios. In most\nsituations, the source and the target speakers do not repeat the same texts or\nthey may even speak different languages. In this case, one possible, although\nindirect, solution is to build a generative model for speech. Generative models\nfocus on explaining the observations with latent variables instead of learning\na pairwise transformation function, thereby bypassing the requirement of speech\nframe alignment. In this paper, we propose a non-parallel VC framework with a\nvariational autoencoding Wasserstein generative adversarial network (VAW-GAN)\nthat explicitly considers a VC objective when building the speech model.\nExperimental results corroborate the capability of our framework for building a\nVC system from unaligned data, and demonstrate improved conversion quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00898v2",
            "date": "2017-04-04 07:24:15+00:00",
            "title": "Interpretation of Semantic Tweet Representations",
            "authors": [
                "J Ganesh",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Research in analysis of microblogging platforms is experiencing a renewed\nsurge with a large number of works applying representation learning models for\napplications like sentiment analysis, semantic textual similarity computation,\nhashtag prediction, etc. Although the performance of the representation\nlearning models has been better than the traditional baselines for such tasks,\nlittle is known about the elementary properties of a tweet encoded within these\nrepresentations, or why particular representations work better for certain\ntasks. Our work presented here constitutes the first step in opening the\nblack-box of vector embeddings for tweets. Traditional feature engineering\nmethods for high-level applications have exploited various elementary\nproperties of tweets. We believe that a tweet representation is effective for\nan application because it meticulously encodes the application-specific\nelementary properties of tweets. To understand the elementary properties\nencoded in a tweet representation, we evaluate the representations on the\naccuracy to which they can model each of those properties such as tweet length,\npresence of particular words, hashtags, mentions, capitalization, etc. Our\nsystematic extensive study of nine supervised and four unsupervised tweet\nrepresentations against most popular eight textual and five social elementary\nproperties reveal that Bi-directional LSTMs (BLSTMs) and Skip-Thought Vectors\n(STV) best encode the textual and social properties of tweets respectively.\nFastText is the best model for low resource settings, providing very little\ndegradation with reduction in embedding size. Finally, we draw interesting\ninsights by correlating the model performance obtained for elementary property\nprediction tasks with the highlevel downstream applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00924v2",
            "date": "2017-04-04 09:08:46+00:00",
            "title": "Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention",
            "authors": [
                "Ryosuke Miyazaki",
                "Mamoru Komachi"
            ],
            "abstract": "Previous approaches to training syntax-based sentiment classification models\nrequired phrase-level annotated corpora, which are not readily available in\nmany languages other than English. Thus, we propose the use of tree-structured\nLong Short-Term Memory with an attention mechanism that pays attention to each\nsubtree of the parse tree. Experimental results indicate that our model\nachieves the state-of-the-art performance in a Japanese sentiment\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01314v3",
            "date": "2017-04-05 08:58:44+00:00",
            "title": "Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF",
            "authors": [
                "Yan Shao",
                "Christian Hardmeier",
                "J\u00f6rg Tiedemann",
                "Joakim Nivre"
            ],
            "abstract": "We present a character-based model for joint segmentation and POS tagging for\nChinese. The bidirectional RNN-CRF architecture for general sequence tagging is\nadapted and applied with novel vector representations of Chinese characters\nthat capture rich contextual information and lower-than-character level\nfeatures. The proposed model is extensively evaluated and compared with a\nstate-of-the-art tagger respectively on CTB5, CTB9 and UD Chinese. The\nexperimental results indicate that our model is accurate and robust across\ndatasets in different sizes, genres and annotation schemes. We obtain\nstate-of-the-art performance on CTB5, achieving 94.38 F1-score for joint\nsegmentation and POS tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01346v1",
            "date": "2017-04-05 10:07:22+00:00",
            "title": "CompiLIG at SemEval-2017 Task 1: Cross-Language Plagiarism Detection Methods for Semantic Textual Similarity",
            "authors": [
                "Jeremy Ferrero",
                "Frederic Agnes",
                "Laurent Besacier",
                "Didier Schwab"
            ],
            "abstract": "We present our submitted systems for Semantic Textual Similarity (STS) Track\n4 at SemEval-2017. Given a pair of Spanish-English sentences, each system must\nestimate their semantic similarity by a score between 0 and 5. In our\nsubmission, we use syntax-based, dictionary-based, context-based, and MT-based\nmethods. We also combine these methods in unsupervised and supervised way. Our\nbest run ranked 1st on track 4a with a correlation of 83.02% with human\nannotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01419v1",
            "date": "2017-04-05 13:38:01+00:00",
            "title": "Linear Ensembles of Word Embedding Models",
            "authors": [
                "Avo Murom\u00e4gi",
                "Kairit Sirts",
                "Sven Laur"
            ],
            "abstract": "This paper explores linear methods for combining several word embedding\nmodels into an ensemble. We construct the combined models using an iterative\nmethod based on either ordinary least squares regression or the solution to the\northogonal Procrustes problem.\n  We evaluate the proposed approaches on Estonian---a morphologically complex\nlanguage, for which the available corpora for training word embeddings are\nrelatively small. We compare both combined models with each other and with the\ninput word embedding models using synonym and analogy tests. The results show\nthat while using the ordinary least squares regression performs poorly in our\nexperiments, using orthogonal Procrustes to combine several word embedding\nmodels into an ensemble model leads to 7-10% relative improvements over the\nmean result of the initial models in synonym tests and 19-47% in analogy tests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01653v3",
            "date": "2017-04-05 21:10:07+00:00",
            "title": "Automatic Measurement of Pre-aspiration",
            "authors": [
                "Yaniv Sheena",
                "M\u00ed\u0161a Hejn\u00e1",
                "Yossi Adi",
                "Joseph Keshet"
            ],
            "abstract": "Pre-aspiration is defined as the period of glottal friction occurring in\nsequences of vocalic/consonantal sonorants and phonetically voiceless\nobstruents. We propose two machine learning methods for automatic measurement\nof pre-aspiration duration: a feedforward neural network, which works at the\nframe level; and a structured prediction model, which relies on manually\ndesigned feature functions, and works at the segment level. The input for both\nalgorithms is a speech signal of an arbitrary length containing a single\nobstruent, and the output is a pair of times which constitutes the\npre-aspiration boundaries. We train both models on a set of manually annotated\nexamples. Results suggest that the structured model is superior to the\nframe-based model as it yields higher accuracy in predicting the boundaries and\ngeneralizes to new speakers and new languages. Finally, we demonstrate the\napplicability of our structured prediction algorithm by replicating linguistic\nanalysis of pre-aspiration in Aberystwyth English with high correlation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01748v3",
            "date": "2017-04-06 08:32:16+00:00",
            "title": "MRA - Proof of Concept of a Multilingual Report Annotator Web Application",
            "authors": [
                "Lu\u00eds Campos",
                "Francisco Couto"
            ],
            "abstract": "MRA (Multilingual Report Annotator) is a web application that translates\nRadiology text and annotates it with RadLex terms. Its goal is to explore the\nsolution of translating non-English Radiology reports as a way to solve the\nproblem of most of the Text Mining tools being developed for English. In this\nbrief paper we explain the language barrier problem and shortly describe the\napplication. MRA can be found at https://github.com/lasigeBioTM/MRA .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01792v3",
            "date": "2017-04-06 11:44:07+00:00",
            "title": "Neural Question Generation from Text: A Preliminary Study",
            "authors": [
                "Qingyu Zhou",
                "Nan Yang",
                "Furu Wei",
                "Chuanqi Tan",
                "Hangbo Bao",
                "Ming Zhou"
            ],
            "abstract": "Automatic question generation aims to generate questions from a text passage\nwhere the generated questions can be answered by certain sub-spans of the given\npassage. Traditional methods mainly use rigid heuristic rules to transform a\nsentence into related questions. In this work, we propose to apply the neural\nencoder-decoder model to generate meaningful and diverse questions from natural\nlanguage sentences. The encoder reads the input text and the answer position,\nto produce an answer-aware input representation, which is fed to the decoder to\ngenerate an answer focused question. We conduct a preliminary study on neural\nquestion generation from text with the SQuAD dataset, and the experiment\nresults show that our method can produce fluent and diverse questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01938v1",
            "date": "2017-04-06 17:07:40+00:00",
            "title": "The Interplay of Semantics and Morphology in Word Embeddings",
            "authors": [
                "Oded Avraham",
                "Yoav Goldberg"
            ],
            "abstract": "We explore the ability of word embeddings to capture both semantic and\nmorphological similarity, as affected by the different types of linguistic\nproperties (surface form, lemma, morphological tag) used to compose the\nrepresentation of each word. We train several models, where each uses a\ndifferent subset of these properties to compose its representations. By\nevaluating the models on semantic and morphological measures, we reveal some\nuseful insights on the relationship between semantics and morphology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02080v1",
            "date": "2017-04-07 03:27:54+00:00",
            "title": "Conversation Modeling on Reddit using a Graph-Structured LSTM",
            "authors": [
                "Vicky Zayats",
                "Mari Ostendorf"
            ],
            "abstract": "This paper presents a novel approach for modeling threaded discussions on\nsocial media using a graph-structured bidirectional LSTM which represents both\nhierarchical and temporal conversation structure. In experiments with a task of\npredicting popularity of comments in Reddit discussions, the proposed model\noutperforms a node-independent architecture for different sets of input\nfeatures. Analyses show a benefit to the model over the full course of the\ndiscussion, improving detection in both early and late stages. Further, the use\nof language cues with the bidirectional tree state updates helps with\nidentifying controversial comments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02134v6",
            "date": "2017-04-07 08:42:45+00:00",
            "title": "Adposition and Case Supersenses v2.5: Guidelines for English",
            "authors": [
                "Nathan Schneider",
                "Jena D. Hwang",
                "Archna Bhatia",
                "Vivek Srikumar",
                "Na-Rae Han",
                "Tim O'Gorman",
                "Sarah R. Moeller",
                "Omri Abend",
                "Adi Shalev",
                "Austin Blodgett",
                "Jakob Prange"
            ],
            "abstract": "This document offers a detailed linguistic description of SNACS (Semantic\nNetwork of Adposition and Case Supersenses; Schneider et al., 2018), an\ninventory of 50 semantic labels (\"supersenses\") that characterize the use of\nadpositions and case markers at a somewhat coarse level of granularity, as\ndemonstrated in the STREUSLE corpus (https://github.com/nert-gu/streusle/;\nversion 4.3 tracks guidelines version 2.5). Though the SNACS inventory aspires\nto be universal, this document is specific to English; documentation for other\nlanguages will be published separately.\n  Version 2 is a revision of the supersense inventory proposed for English by\nSchneider et al. (2015, 2016) (henceforth \"v1\"), which in turn was based on\nprevious schemes. The present inventory was developed after extensive review of\nthe v1 corpus annotations for English, plus previously unanalyzed genitive case\npossessives (Blodgett and Schneider, 2018), as well as consideration of\nadposition and case phenomena in Hebrew, Hindi, Korean, and German. Hwang et\nal. (2017) present the theoretical underpinnings of the v2 scheme. Schneider et\nal. (2018) summarize the scheme, its application to English corpus data, and an\nautomatic disambiguation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02156v2",
            "date": "2017-04-07 09:37:36+00:00",
            "title": "The Meaning Factory at SemEval-2017 Task 9: Producing AMRs with Neural Semantic Parsing",
            "authors": [
                "Rik van Noord",
                "Johan Bos"
            ],
            "abstract": "We evaluate a semantic parser based on a character-based sequence-to-sequence\nmodel in the context of the SemEval-2017 shared task on semantic parsing for\nAMRs. With data augmentation, super characters, and POS-tagging we gain major\nimprovements in performance compared to a baseline character-level model.\nAlthough we improve on previous character-based neural semantic parsing models,\nthe overall accuracy is still lower than a state-of-the-art AMR parser. An\nensemble combining our neural semantic parser with an existing, traditional\nparser, yields a small gain in performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02215v2",
            "date": "2017-04-07 13:07:15+00:00",
            "title": "EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION",
            "authors": [
                "Steffen Eger",
                "Erik-L\u00e2n Do Dinh",
                "Ilia Kuznetsov",
                "Masoud Kiaeeha",
                "Iryna Gurevych"
            ],
            "abstract": "This paper describes our approach to the SemEval 2017 Task 10: \"Extracting\nKeyphrases and Relations from Scientific Publications\", specifically to Subtask\n(B): \"Classification of identified keyphrases\". We explored three different\ndeep learning approaches: a character-level convolutional neural network (CNN),\na stacked learner with an MLP meta-classifier, and an attention based Bi-LSTM.\nFrom these approaches, we created an ensemble of differently\nhyper-parameterized systems, achieving a micro-F1-score of 0.63 on the test\ndata. Our approach ranks 2nd (score of 1st placed system: 0.64) out of four\naccording to this official score. However, we erroneously trained 2 out of 3\nneural nets (the stacker and the CNN) on only roughly 15% of the full data,\nnamely, the original development set. When trained on the full data\n(training+development), our ensemble has a micro-F1-score of 0.69. Our code is\navailable from https://github.com/UKPLab/semeval2017-scienceie.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02293v1",
            "date": "2017-04-07 17:04:51+00:00",
            "title": "Comparison of Global Algorithms in Word Sense Disambiguation",
            "authors": [
                "Lo\u00efc Vial",
                "Andon Tchechmedjiev",
                "Didier Schwab"
            ],
            "abstract": "This article compares four probabilistic algorithms (global algorithms) for\nWord Sense Disambiguation (WSD) in terms of the number of scorer calls (local\nalgo- rithm) and the F1 score as determined by a gold-standard scorer. Two\nalgorithms come from the state of the art, a Simulated Annealing Algorithm\n(SAA) and a Genetic Algorithm (GA) as well as two algorithms that we first\nadapt from WSD that are state of the art probabilistic search algorithms,\nnamely a Cuckoo search algorithm (CSA) and a Bat Search algorithm (BS). As WSD\nrequires to evaluate exponentially many word sense combinations (with branching\nfactors of up to 6 or more), probabilistic algorithms allow to find approximate\nsolution in a tractable time by sampling the search space. We find that CSA, GA\nand SA all eventually converge to similar results (0.98 F1 score), but CSA gets\nthere faster (in fewer scorer calls) and reaches up to 0.95 F1 before SA in\nfewer scorer calls. In BA a strict convergence criterion prevents it from\nreaching above 0.89 F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02362v2",
            "date": "2017-03-17 15:57:45+00:00",
            "title": "Fostering User Engagement: Rhetorical Devices for Applause Generation Learnt from TED Talks",
            "authors": [
                "Zhe Liu",
                "Anbang Xu",
                "Mengdi Zhang",
                "Jalal Mahmud",
                "Vibha Sinha"
            ],
            "abstract": "One problem that every presenter faces when delivering a public discourse is\nhow to hold the listeners' attentions or to keep them involved. Therefore, many\nstudies in conversation analysis work on this issue and suggest qualitatively\ncon-structions that can effectively lead to audience's applause. To investigate\nthese proposals quantitatively, in this study we an-alyze the transcripts of\n2,135 TED Talks, with a particular fo-cus on the rhetorical devices that are\nused by the presenters for applause elicitation. Through conducting regression\nanal-ysis, we identify and interpret 24 rhetorical devices as triggers of\naudience applauding. We further build models that can rec-ognize\napplause-evoking sentences and conclude this work with potential implications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02385v1",
            "date": "2017-04-07 22:08:39+00:00",
            "title": "A Trolling Hierarchy in Social Media and A Conditional Random Field For Trolling Detection",
            "authors": [
                "Luis Gerardo Mojica"
            ],
            "abstract": "An-ever increasing number of social media websites, electronic newspapers and\nInternet forums allow visitors to leave comments for others to read and\ninteract. This exchange is not free from participants with malicious\nintentions, which do not contribute with the written conversation. Among\ndifferent communities users adopt strategies to handle such users. In this\npaper we present a comprehensive categorization of the trolling phenomena\nresource, inspired by politeness research and propose a model that jointly\npredicts four crucial aspects of trolling: intention, interpretation, intention\ndisclosure and response strategy. Finally, we present a new annotated dataset\ncontaining excerpts of conversations involving trolls and the interactions with\nother users that we hope will be a useful resource for the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02497v1",
            "date": "2017-04-08 13:56:55+00:00",
            "title": "On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models",
            "authors": [
                "Steffen Eger",
                "Alexander Mehler"
            ],
            "abstract": "We consider two graph models of semantic change. The first is a time-series\nmodel that relates embedding vectors from one time period to embedding vectors\nof previous time periods. In the second, we construct one graph for each word:\nnodes in this graph correspond to time points and edge weights to the\nsimilarity of the word's meaning across two time points. We apply our two\nmodels to corpora across three different languages. We find that semantic\nchange is linear in two senses. Firstly, today's embedding vectors (= meaning)\nof words can be derived as linear combinations of embedding vectors of their\nneighbors in previous time periods. Secondly, self-similarity of words decays\nlinearly in time. We consider both findings as new laws/hypotheses of semantic\nchange.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02565v2",
            "date": "2017-04-09 06:17:08+00:00",
            "title": "Prosody: The Rhythms and Melodies of Speech",
            "authors": [
                "Dafydd Gibbon"
            ],
            "abstract": "The present contribution is a tutorial on selected aspects of prosody, the\nrhythms and melodies of speech, based on a course of the same name at the\nSummer School on Contemporary Phonetics and Phonology at Tongji University,\nShanghai, China in July 2016. The tutorial is not intended as an introduction\nto experimental methodology or as an overview of the literature on the topic,\nbut as an outline of observationally accessible aspects of fundamental\nfrequency and timing patterns with the aid of computational visualisation,\nsituated in a semiotic framework of sign ranks and interpretations. After an\ninformal introduction to the basic concepts of prosody in the introduction and\na discussion of the place of prosody in the architecture of language, a\nselection of acoustic phonetic topics in phonemic tone and accent prosody, word\nprosody, phrasal prosody and discourse prosody are discussed, and a stylisation\nmethod for visualising aspects of prosody is introduced. Examples are taken\nfrom a number of typologically different languages: Anyi/Agni (Niger-Congo>Kwa,\nIvory Coast), English, Kuki-Thadou (Sino-Tibetan, North-East India and\nMyanmar), Mandarin Chinese, Tem (Niger-Congo>Gur, Togo) and Farsi. The main\nfocus is on fundamental frequency patterns, but issues of timing and rhythm are\nalso discussed. In the final section, further reading and possible future\nresearch directions are outlined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02709v2",
            "date": "2017-04-10 04:48:53+00:00",
            "title": "Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments",
            "authors": [
                "Quynh Ngoc Thi Do",
                "Steven Bethard",
                "Marie-Francine Moens"
            ],
            "abstract": "Implicit semantic role labeling (iSRL) is the task of predicting the semantic\nroles of a predicate that do not appear as explicit arguments, but rather\nregard common sense knowledge or are mentioned earlier in the discourse. We\nintroduce an approach to iSRL based on a predictive recurrent neural semantic\nframe model (PRNSFM) that uses a large unannotated corpus to learn the\nprobability of a sequence of semantic arguments given a predicate. We leverage\nthe sequence probabilities predicted by the PRNSFM to estimate selectional\npreferences for predicates and their arguments. On the NomBank iSRL test set,\nour approach improves state-of-the-art performance on implicit semantic role\nlabeling with less reliance than prior work on manually constructed language\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02788v3",
            "date": "2017-04-10 10:19:53+00:00",
            "title": "Entity Linking for Queries by Searching Wikipedia Sentences",
            "authors": [
                "Chuanqi Tan",
                "Furu Wei",
                "Pengjie Ren",
                "Weifeng Lv",
                "Ming Zhou"
            ],
            "abstract": "We present a simple yet effective approach for linking entities in queries.\nThe key idea is to search sentences similar to a query from Wikipedia articles\nand directly use the human-annotated entities in the similar sentences as\ncandidate entities for the query. Then, we employ a rich set of features, such\nas link-probability, context-matching, word embeddings, and relatedness among\ncandidate entities as well as their related entities, to rank the candidates\nunder a regression based framework. The advantages of our approach lie in two\naspects, which contribute to the ranking process and final linking result.\nFirst, it can greatly reduce the number of candidate entities by filtering out\nirrelevant entities with the words in the query. Second, we can obtain the\nquery sensitive prior probability in addition to the static link-probability\nderived from all Wikipedia articles. We conduct experiments on two benchmark\ndatasets on entity linking for queries, namely the ERD14 dataset and the GERDAQ\ndataset. Experimental results show that our method outperforms state-of-the-art\nsystems and yields 75.0% in F1 on the ERD14 dataset and 56.9% on the GERDAQ\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02813v1",
            "date": "2017-04-10 11:42:09+00:00",
            "title": "Character-Word LSTM Language Models",
            "authors": [
                "Lyan Verwimp",
                "Joris Pelemans",
                "Hugo Van hamme",
                "Patrick Wambacq"
            ],
            "abstract": "We present a Character-Word Long Short-Term Memory Language Model which both\nreduces the perplexity with respect to a baseline word-level language model and\nreduces the number of parameters of the model. Character information can reveal\nstructural (dis)similarities between words and can even be used when a word is\nout-of-vocabulary, thus improving the modeling of infrequent and unknown words.\nBy concatenating word and character embeddings, we achieve up to 2.77% relative\nimprovement on English compared to a baseline model with a similar amount of\nparameters and 4.57% on Dutch. Moreover, we also outperform baseline word-level\nmodels with a larger number of parameters.",
            "journal ref": "European Chapter of the Association for Computational Linguistics\n  (EACL) 2017, Valencia, Spain, pp. 417-427",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03013v1",
            "date": "2017-04-10 18:44:08+00:00",
            "title": "Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years",
            "authors": [
                "Nathan Siegle Hartmann",
                "Livia Cucatto",
                "Danielle Brants",
                "Sandra Alu\u00edsio"
            ],
            "abstract": "Recent research shows that most Brazilian students have serious problems\nregarding their reading skills. The full development of this skill is key for\nthe academic and professional future of every citizen. Tools for classifying\nthe complexity of reading materials for children aim to improve the quality of\nthe model of teaching reading and text comprehension. For English, Fengs work\n[11] is considered the state-of-art in grade level prediction and achieved 74%\nof accuracy in automatically classifying 4 levels of textual complexity for\nclose school grades. There are no classifiers for nonfiction texts for close\ngrades in Portuguese. In this article, we propose a scheme for manual\nannotation of texts in 5 grade levels, which will be used for customized\nreading to avoid the lack of interest by students who are more advanced in\nreading and the blocking of those that still need to make further progress. We\nobtained 52% of accuracy in classifying texts into 5 levels and 74% in 3\nlevels. The results prove to be promising when compared to the state-of-art\nwork.9",
            "journal ref": "Hartmann N., Cucatto L., Brants D., Alu\\'isio S. (2016) Automatic\n  Classification of the Complexity of Nonfiction Texts in Portuguese for Early\n  School Years. In: Computational Processing of the Portuguese Language. PROPOR\n  2016. Springer",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03016v1",
            "date": "2017-04-10 19:02:18+00:00",
            "title": "Automatic semantic role labeling on non-revised syntactic trees of journalistic texts",
            "authors": [
                "Nathan Siegle Hartmann",
                "Magali Sanches Duran",
                "Sandra Maria Alu\u00edsio"
            ],
            "abstract": "Semantic Role Labeling (SRL) is a Natural Language Processing task that\nenables the detection of events described in sentences and the participants of\nthese events. For Brazilian Portuguese (BP), there are two studies recently\nconcluded that perform SRL in journalistic texts. [1] obtained F1-measure\nscores of 79.6, using the PropBank.Br corpus, which has syntactic trees\nmanually revised, [8], without using a treebank for training, obtained\nF1-measure scores of 68.0 for the same corpus. However, the use of manually\nrevised syntactic trees for this task does not represent a real scenario of\napplication. The goal of this paper is to evaluate the performance of SRL on\nrevised and non-revised syntactic trees using a larger and balanced corpus of\nBP journalistic texts. First, we have shown that [1]'s system also performs\nbetter than [8]'s system on the larger corpus. Second, the SRL system trained\non non-revised syntactic trees performs better over non-revised trees than a\nsystem trained on gold-standard data.",
            "journal ref": "PROPOR 2016. Springer. Lecture Notes in Computer Science volume\n  9727 (2016) pgs. 202-212",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03169v2",
            "date": "2017-04-11 06:48:45+00:00",
            "title": "Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation",
            "authors": [
                "Raphael Shu",
                "Hideki Nakayama"
            ],
            "abstract": "For extended periods of time, sequence generation models rely on beam search\nalgorithm to generate output sequence. However, the correctness of beam search\ndegrades when the a model is over-confident about a suboptimal prediction. In\nthis paper, we propose to perform minimum Bayes-risk (MBR) decoding for some\nextra steps at a later stage. In order to speed up MBR decoding, we compute the\nBayes risks on GPU in batch mode. In our experiments, we found that MBR\nreranking works with a large beam size. Later-stage MBR decoding is shown to\noutperform simple MBR reranking in machine translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03242v1",
            "date": "2017-04-11 11:20:19+00:00",
            "title": "Automatic Keyword Extraction for Text Summarization: A Survey",
            "authors": [
                "Santosh Kumar Bharti",
                "Korra Sathya Babu"
            ],
            "abstract": "In recent times, data is growing rapidly in every domain such as news, social\nmedia, banking, education, etc. Due to the excessiveness of data, there is a\nneed of automatic summarizer which will be capable to summarize the data\nespecially textual data in original document without losing any critical\npurposes. Text summarization is emerged as an important research area in recent\npast. In this regard, review of existing work on text summarization process is\nuseful for carrying out further research. In this paper, recent literature on\nautomatic keyword extraction and text summarization are presented since text\nsummarization process is highly depend on keyword extraction. This literature\nincludes the discussion about different methodology used for keyword extraction\nand text summarization. It also discusses about different databases used for\ntext summarization in several domains along with evaluation matrices. Finally,\nit discusses briefly about issues and research challenges faced by researchers\nalong with future direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03279v2",
            "date": "2017-04-11 13:27:00+00:00",
            "title": "Unfolding and Shrinking Neural Machine Translation Ensembles",
            "authors": [
                "Felix Stahlberg",
                "Bill Byrne"
            ],
            "abstract": "Ensembling is a well-known technique in neural machine translation (NMT) to\nimprove system performance. Instead of a single neural net, multiple neural\nnets with the same topology are trained separately, and the decoder generates\npredictions by averaging over the individual models. Ensembling often improves\nthe quality of the generated translations drastically. However, it is not\nsuitable for production systems because it is cumbersome and slow. This work\naims to reduce the runtime to be on par with a single system without\ncompromising the translation quality. First, we show that the ensemble can be\nunfolded into a single large neural network which imitates the output of the\nensemble system. We show that unfolding can already improve the runtime in\npractice since more work can be done on the GPU. We proceed by describing a set\nof techniques to shrink the unfolded network by reducing the dimensionality of\nlayers. On Japanese-English we report that the resulting network has the size\nand decoding speed of a single NMT network but performs on the level of a\n3-ensemble system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03471v3",
            "date": "2017-04-11 18:01:07+00:00",
            "title": "What do Neural Machine Translation Models Learn about Morphology?",
            "authors": [
                "Yonatan Belinkov",
                "Nadir Durrani",
                "Fahim Dalvi",
                "Hassan Sajjad",
                "James Glass"
            ],
            "abstract": "Neural machine translation (MT) models obtain state-of-the-art performance\nwhile maintaining a simple, end-to-end architecture. However, little is known\nabout what these models learn about source and target languages during the\ntraining process. In this work, we analyze the representations learned by\nneural MT models at various levels of granularity and empirically evaluate the\nquality of the representations for learning morphology through extrinsic\npart-of-speech and morphological tagging tasks. We conduct a thorough\ninvestigation along several parameters: word-based vs. character-based\nrepresentations, depth of the encoding layer, the identity of the target\nlanguage, and encoder vs. decoder representations. Our data-driven,\nquantitative evaluation sheds light on important aspects in the neural MT\nsystem and its ability to capture word structure.",
            "journal ref": "ACL 55 (2017), volume 1, 861-872",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03560v2",
            "date": "2017-04-11 22:44:35+00:00",
            "title": "ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge",
            "authors": [
                "Robyn Speer",
                "Joanna Lowry-Duda"
            ],
            "abstract": "This paper describes Luminoso's participation in SemEval 2017 Task 2,\n\"Multilingual and Cross-lingual Semantic Word Similarity\", with a system based\non ConceptNet. ConceptNet is an open, multilingual knowledge graph that focuses\non general knowledge that relates the meanings of words and phrases. Our\nsubmission to SemEval was an update of previous work that builds high-quality,\nmultilingual word embeddings from a combination of ConceptNet and\ndistributional semantics. Our system took first place in both subtasks. It\nranked first in 4 out of 5 of the separate languages, and also ranked first in\nall 10 of the cross-lingual language pairs.",
            "journal ref": "Proceedings of the 11th International Workshop on Semantic\n  Evaluation (SemEval-2017), p. 85-89",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03693v1",
            "date": "2017-04-12 10:47:10+00:00",
            "title": "Trainable Referring Expression Generation using Overspecification Preferences",
            "authors": [
                "Thiago castro Ferreira",
                "Ivandre Paraboni"
            ],
            "abstract": "Referring expression generation (REG) models that use speaker-dependent\ninformation require a considerable amount of training data produced by every\nindividual speaker, or may otherwise perform poorly. In this work we present a\nsimple REG experiment that allows the use of larger training data sets by\ngrouping speakers according to their overspecification preferences. Intrinsic\nevaluation shows that this method generally outperforms the personalised method\nfound in previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03956v2",
            "date": "2017-04-13 00:36:33+00:00",
            "title": "Incremental Skip-gram Model with Negative Sampling",
            "authors": [
                "Nobuhiro Kaji",
                "Hayato Kobayashi"
            ],
            "abstract": "This paper explores an incremental training strategy for the skip-gram model\nwith negative sampling (SGNS) from both empirical and theoretical perspectives.\nExisting methods of neural word embeddings, including SGNS, are multi-pass\nalgorithms and thus cannot perform incremental model update. To address this\nproblem, we present a simple incremental extension of SGNS and provide a\nthorough theoretical analysis to demonstrate its validity. Empirical\nexperiments demonstrated the correctness of the theoretical analysis as well as\nthe practical usefulness of the incremental algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03987v1",
            "date": "2017-04-13 04:00:07+00:00",
            "title": "Mobile Keyboard Input Decoding with Finite-State Transducers",
            "authors": [
                "Tom Ouyang",
                "David Rybach",
                "Fran\u00e7oise Beaufays",
                "Michael Riley"
            ],
            "abstract": "We propose a finite-state transducer (FST) representation for the models used\nto decode keyboard inputs on mobile devices. Drawing from learnings from the\nfield of speech recognition, we describe a decoding framework that can satisfy\nthe strict memory and latency constraints of keyboard input. We extend this\nframework to support functionalities typically not present in speech\nrecognition, such as literal decoding, autocorrections, word completions, and\nnext word predictions.\n  We describe the general framework of what we call for short the keyboard \"FST\ndecoder\" as well as the implementation details that are new compared to a\nspeech FST decoder. We demonstrate that the FST decoder enables new UX features\nsuch as post-corrections. Finally, we sketch how this decoder can support\nadvanced features such as personalization and contextualization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04008v3",
            "date": "2017-04-13 06:35:55+00:00",
            "title": "A Neural Model for User Geolocation and Lexical Dialectology",
            "authors": [
                "Afshin Rahimi",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "We propose a simple yet effective text- based user geolocation model based on\na neural network with one hidden layer, which achieves state of the art\nperformance over three Twitter benchmark geolocation datasets, in addition to\nproducing word and phrase embeddings in the hidden layer that we show to be\nuseful for detecting dialectal terms. As part of our analysis of dialectal\nterms, we release DAREDS, a dataset for evaluating dialect term detection\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04100v2",
            "date": "2017-04-13 12:54:30+00:00",
            "title": "Cross-lingual and cross-domain discourse segmentation of entire documents",
            "authors": [
                "Chlo\u00e9 Braud",
                "Oph\u00e9lie Lacroix",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Discourse segmentation is a crucial step in building end-to-end discourse\nparsers. However, discourse segmenters only exist for a few languages and\ndomains. Typically they only detect intra-sentential segment boundaries,\nassuming gold standard sentence and token segmentation, and relying on\nhigh-quality syntactic parses and rich heuristics that are not generally\navailable across languages and domains. In this paper, we propose statistical\ndiscourse segmenters for five languages and three domains that do not rely on\ngold pre-annotations. We also consider the problem of learning discourse\nsegmenters when no labeled data is available for a language. Our fully\nsupervised system obtains 89.5% F1 for English newswire, with slight drops in\nperformance on other domains, and we report supervised and unsupervised\n(cross-lingual) results for five languages in total.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04154v2",
            "date": "2017-04-13 14:40:40+00:00",
            "title": "Learning Joint Multilingual Sentence Representations with Neural Machine Translation",
            "authors": [
                "Holger Schwenk",
                "Matthijs Douze"
            ],
            "abstract": "In this paper, we use the framework of neural machine translation to learn\njoint sentence representations across six very different languages. Our aim is\nthat a representation which is independent of the language, is likely to\ncapture the underlying semantics. We define a new cross-lingual similarity\nmeasure, compare up to 1.4M sentence representations and study the\ncharacteristics of close sentences. We provide experimental evidence that\nsentences that are close in embedding space are indeed semantically highly\nrelated, but often have quite different structure and syntax. These relations\nalso hold when comparing sentences in different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04198v1",
            "date": "2017-04-13 16:21:18+00:00",
            "title": "Room for improvement in automatic image description: an error analysis",
            "authors": [
                "Emiel van Miltenburg",
                "Desmond Elliott"
            ],
            "abstract": "In recent years we have seen rapid and significant progress in automatic\nimage description but what are the open problems in this area? Most work has\nbeen evaluated using text-based similarity metrics, which only indicate that\nthere have been improvements, without explaining what has improved. In this\npaper, we present a detailed error analysis of the descriptions generated by a\nstate-of-the-art attention-based model. Our analysis operates on two levels:\nfirst we check the descriptions for accuracy, and then we categorize the types\nof errors we observe in the inaccurate descriptions. We find only 20% of the\ndescriptions are free from errors, and surprisingly that 26% are unrelated to\nthe image. Finally, we manually correct the most frequently occurring error\ntypes (e.g. gender identification) to estimate the performance reward for\naddressing these errors, observing gains of 0.2--1 BLEU point per type.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04259v1",
            "date": "2017-04-13 19:23:43+00:00",
            "title": "Identity and Granularity of Events in Text",
            "authors": [
                "Piek Vossen",
                "Agata Cybulska"
            ],
            "abstract": "In this paper we describe a method to detect event descrip- tions in\ndifferent news articles and to model the semantics of events and their\ncomponents using RDF representations. We compare these descriptions to solve a\ncross-document event coreference task. Our com- ponent approach to event\nsemantics defines identity and granularity of events at different levels. It\nperforms close to state-of-the-art approaches on the cross-document event\ncoreference task, while outperforming other works when assuming similar quality\nof event detection. We demonstrate how granularity and identity are\ninterconnected and we discuss how se- mantic anomaly could be used to define\ndifferences between coreference, subevent and topical relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04336v1",
            "date": "2017-04-14 02:41:13+00:00",
            "title": "An entity-driven recursive neural network model for chinese discourse coherence modeling",
            "authors": [
                "Fan Xu",
                "Shujing Du",
                "Maoxi Li",
                "Mingwen Wang"
            ],
            "abstract": "Chinese discourse coherence modeling remains a challenge taskin Natural\nLanguage Processing field.Existing approaches mostlyfocus on the need for\nfeature engineering, whichadoptthe sophisticated features to capture the logic\nor syntactic or semantic relationships acrosssentences within a text.In this\npaper, we present an entity-drivenrecursive deep modelfor the Chinese discourse\ncoherence evaluation based on current English discourse coherenceneural network\nmodel. Specifically, to overcome the shortage of identifying the entity(nouns)\noverlap across sentences in the currentmodel, Our combined modelsuccessfully\ninvestigatesthe entities information into the recursive neural network\nfreamework.Evaluation results on both sentence ordering and machine translation\ncoherence rating task show the effectiveness of the proposed model, which\nsignificantly outperforms the existing strong baseline.",
            "journal ref": "International Journal of Artificial Intelligence and Applications\n  (IJAIA), Vol.8, No.2, March 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04347v3",
            "date": "2017-04-14 04:56:36+00:00",
            "title": "Exploiting Cross-Sentence Context for Neural Machine Translation",
            "authors": [
                "Longyue Wang",
                "Zhaopeng Tu",
                "Andy Way",
                "Qun Liu"
            ],
            "abstract": "In translation, considering the document as a whole can help to resolve\nambiguities and inconsistencies. In this paper, we propose a cross-sentence\ncontext-aware approach and investigate the influence of historical contextual\ninformation on the performance of neural machine translation (NMT). First, this\nhistory is summarized in a hierarchical way. We then integrate the historical\nrepresentation into NMT in two strategies: 1) a warm-start of encoder and\ndecoder states, and 2) an auxiliary context source for updating decoder states.\nExperimental results on a large Chinese-English translation task show that our\napproach significantly improves upon a strong attention-based NMT system by up\nto +2.1 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04368v2",
            "date": "2017-04-14 07:55:19+00:00",
            "title": "Get To The Point: Summarization with Pointer-Generator Networks",
            "authors": [
                "Abigail See",
                "Peter J. Liu",
                "Christopher D. Manning"
            ],
            "abstract": "Neural sequence-to-sequence models have provided a viable new approach for\nabstractive text summarization (meaning they are not restricted to simply\nselecting and rearranging passages from the original text). However, these\nmodels have two shortcomings: they are liable to reproduce factual details\ninaccurately, and they tend to repeat themselves. In this work we propose a\nnovel architecture that augments the standard sequence-to-sequence attentional\nmodel in two orthogonal ways. First, we use a hybrid pointer-generator network\nthat can copy words from the source text via pointing, which aids accurate\nreproduction of information, while retaining the ability to produce novel words\nthrough the generator. Second, we use coverage to keep track of what has been\nsummarized, which discourages repetition. We apply our model to the CNN / Daily\nMail summarization task, outperforming the current abstractive state-of-the-art\nby at least 2 ROUGE points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04441v1",
            "date": "2017-04-14 14:43:44+00:00",
            "title": "How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?",
            "authors": [
                "Georg Heigold",
                "G\u00fcnter Neumann",
                "Josef van Genabith"
            ],
            "abstract": "This paper investigates the robustness of NLP against perturbed word forms.\nWhile neural approaches can achieve (almost) human-like accuracy for certain\ntasks and conditions, they often are sensitive to small changes in the input\nsuch as non-canonical input (e.g., typos). Yet both stability and robustness\nare desired properties in applications involving user-generated content, and\nthe more as humans easily cope with such noisy or adversary conditions. In this\npaper, we study the impact of noisy input. We consider different noise\ndistributions (one type of noise, combination of noise types) and mismatched\nnoise distributions for training and testing. Moreover, we empirically evaluate\nthe robustness of different models (convolutional neural networks, recurrent\nneural networks, non-neural models), different basic units (characters, byte\npair encoding units), and different NLP tasks (morphological tagging, machine\ntranslation).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04452v2",
            "date": "2017-04-14 15:23:45+00:00",
            "title": "Bringing Structure into Summaries: Crowdsourcing a Benchmark Corpus of Concept Maps",
            "authors": [
                "Tobias Falke",
                "Iryna Gurevych"
            ],
            "abstract": "Concept maps can be used to concisely represent important information and\nbring structure into large document collections. Therefore, we study a variant\nof multi-document summarization that produces summaries in the form of concept\nmaps. However, suitable evaluation datasets for this task are currently\nmissing. To close this gap, we present a newly created corpus of concept maps\nthat summarize heterogeneous collections of web documents on educational\ntopics. It was created using a novel crowdsourcing approach that allows us to\nefficiently determine important elements in large document collections. We\nrelease the corpus along with a baseline system and proposed evaluation\nprotocol to enable further research on this variant of summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04455v2",
            "date": "2017-04-14 15:28:06+00:00",
            "title": "Cardinal Virtues: Extracting Relation Cardinalities from Text",
            "authors": [
                "Paramita Mirza",
                "Simon Razniewski",
                "Fariz Darari",
                "Gerhard Weikum"
            ],
            "abstract": "Information extraction (IE) from text has largely focused on relations\nbetween individual entities, such as who has won which award. However, some\nfacts are never fully mentioned, and no IE method has perfect recall. Thus, it\nis beneficial to also tap contents about the cardinalities of these relations,\nfor example, how many awards someone has won. We introduce this novel problem\nof extracting cardinalities and discusses the specific challenges that set it\napart from standard IE. We present a distant supervision method using\nconditional random fields. A preliminary evaluation results in precision\nbetween 3% and 55%, depending on the difficulty of relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04520v6",
            "date": "2017-04-14 19:35:14+00:00",
            "title": "Neural Machine Translation Model with a Large Vocabulary Selected by Branching Entropy",
            "authors": [
                "Zi Long",
                "Ryuichiro Kimura",
                "Takehito Utsuro",
                "Tomoharu Mitsuhashi",
                "Mikio Yamamoto"
            ],
            "abstract": "Neural machine translation (NMT), a new approach to machine translation, has\nachieved promising results comparable to those of traditional approaches such\nas statistical machine translation (SMT). Despite its recent success, NMT\ncannot handle a larger vocabulary because the training complexity and decoding\ncomplexity proportionally increase with the number of target words. This\nproblem becomes even more serious when translating patent documents, which\ncontain many technical terms that are observed infrequently. In this paper, we\npropose to select phrases that contain out-of-vocabulary words using the\nstatistical approach of branching entropy. This allows the proposed NMT system\nto be applied to a translation task of any language pair without any\nlanguage-specific knowledge about technical term identification. The selected\nphrases are then replaced with tokens during training and post-translated by\nthe phrase translation table of SMT. Evaluation on Japanese-to-Chinese,\nChinese-to-Japanese, Japanese-to-English and English-to-Japanese patent\nsentence translation proved the effectiveness of phrases selected with\nbranching entropy, where the proposed NMT model achieves a substantial\nimprovement over a baseline NMT model without our proposed technique. Moreover,\nthe number of translation errors of under-translation by the baseline NMT model\nwithout our proposed technique reduces to around half by the proposed NMT\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04521v1",
            "date": "2017-04-14 19:36:54+00:00",
            "title": "Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation",
            "authors": [
                "Zi Long",
                "Takehito Utsuro",
                "Tomoharu Mitsuhashi",
                "Mikio Yamamoto"
            ],
            "abstract": "Neural machine translation (NMT), a new approach to machine translation, has\nachieved promising results comparable to those of traditional approaches such\nas statistical machine translation (SMT). Despite its recent success, NMT\ncannot handle a larger vocabulary because training complexity and decoding\ncomplexity proportionally increase with the number of target words. This\nproblem becomes even more serious when translating patent documents, which\ncontain many technical terms that are observed infrequently. In NMTs, words\nthat are out of vocabulary are represented by a single unknown token. In this\npaper, we propose a method that enables NMT to translate patent sentences\ncomprising a large vocabulary of technical terms. We train an NMT system on\nbilingual data wherein technical terms are replaced with technical term tokens;\nthis allows it to translate most of the source sentences except technical\nterms. Further, we use it as a decoder to translate source sentences with\ntechnical term tokens and replace the tokens with technical term translations\nusing SMT. We also use it to rerank the 1,000-best SMT translations on the\nbasis of the average of the SMT score and that of the NMT rescoring of the\ntranslated sentences with technical term tokens. Our experiments on\nJapanese-Chinese patent sentences show that the proposed NMT system achieves a\nsubstantial improvement of up to 3.1 BLEU points and 2.3 RIBES points over\ntraditional SMT systems and an improvement of approximately 0.6 BLEU points and\n0.8 RIBES points over an equivalent NMT system without our proposed technique.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04530v2",
            "date": "2017-04-14 20:29:23+00:00",
            "title": "Neural Extractive Summarization with Side Information",
            "authors": [
                "Shashi Narayan",
                "Nikos Papasarantopoulos",
                "Shay B. Cohen",
                "Mirella Lapata"
            ],
            "abstract": "Most extractive summarization methods focus on the main body of the document\nfrom which sentences need to be extracted. However, the gist of the document\nmay lie in side information, such as the title and image captions which are\noften available for newswire articles. We propose to explore side information\nin the context of single-document extractive summarization. We develop a\nframework for single-document summarization composed of a hierarchical document\nencoder and an attention-based extractor with attention over side information.\nWe evaluate our model on a large scale news dataset. We show that extractive\nsummarization with side information consistently outperforms its counterpart\nthat does not use any side information, in terms of both informativeness and\nfluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04539v2",
            "date": "2017-04-14 20:41:27+00:00",
            "title": "Cross-lingual Abstract Meaning Representation Parsing",
            "authors": [
                "Marco Damonte",
                "Shay B. Cohen"
            ],
            "abstract": "Abstract Meaning Representation (AMR) annotation efforts have mostly focused\non English. In order to train parsers on other languages, we propose a method\nbased on annotation projection, which involves exploiting annotations in a\nsource language and a parallel corpus of the source language and a target\nlanguage. Using English as the source language, we show promising results for\nItalian, Spanish, German and Chinese as target languages. Besides evaluating\nthe target parsers on non-gold datasets, we further propose an evaluation\nmethod that exploits the English gold annotations and does not require access\nto gold annotations for the target languages. This is achieved by inverting the\nprojection process: a new English parser is learned from the target language\nparser and evaluated on the existing English gold standard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04550v4",
            "date": "2017-04-14 22:29:27+00:00",
            "title": "Distributional Modeling on a Diet: One-shot Word Learning from Text Only",
            "authors": [
                "Su Wang",
                "Stephen Roller",
                "Katrin Erk"
            ],
            "abstract": "We test whether distributional models can do one-shot learning of\ndefinitional properties from text only. Using Bayesian models, we find that\nfirst learning overarching structure in the known data, regularities in textual\ncontexts and in properties, helps one-shot learning, and that individual\ncontext items can be highly informative. Our experiments show that our model\ncan learn properties from a single exposure when given an informative\nutterance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04565v2",
            "date": "2017-04-15 02:09:31+00:00",
            "title": "Neural Paraphrase Identification of Questions with Noisy Pretraining",
            "authors": [
                "Gaurav Singh Tomar",
                "Thyago Duque",
                "Oscar T\u00e4ckstr\u00f6m",
                "Jakob Uszkoreit",
                "Dipanjan Das"
            ],
            "abstract": "We present a solution to the problem of paraphrase identification of\nquestions. We focus on a recent dataset of question pairs annotated with binary\nparaphrase labels and show that a variant of the decomposable attention model\n(Parikh et al., 2016) results in accurate performance on this task, while being\nfar simpler than many competing neural architectures. Furthermore, when the\nmodel is pretrained on a noisy dataset of automatically collected question\nparaphrases, it obtains the best reported performance on the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04601v2",
            "date": "2017-04-15 07:36:49+00:00",
            "title": "MUSE: Modularizing Unsupervised Sense Embeddings",
            "authors": [
                "Guang-He Lee",
                "Yun-Nung Chen"
            ],
            "abstract": "This paper proposes to address the word sense ambiguity issue in an\nunsupervised manner, where word sense representations are learned along a word\nsense selection mechanism given contexts. Prior work focused on designing a\nsingle model to deliver both mechanisms, and thus suffered from either\ncoarse-grained representation learning or inefficient sense selection. The\nproposed modular approach, MUSE, implements flexible modules to optimize\ndistinct mechanisms, achieving the first purely sense-level representation\nlearning system with linear-time sense selection. We leverage reinforcement\nlearning to enable joint training on the proposed modules, and introduce\nvarious exploration techniques on sense selection for better robustness. The\nexperiments on benchmark data show that the proposed approach achieves the\nstate-of-the-art performance on synonym selection as well as on contextual word\nsimilarities in terms of MaxSimC.",
            "journal ref": "Proceedings of the 2017 Conference on Empirical Methods in Natural\n  Language Processing, pages 327-337",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04675v4",
            "date": "2017-04-15 19:04:59+00:00",
            "title": "Graph Convolutional Encoders for Syntax-aware Neural Machine Translation",
            "authors": [
                "Jasmijn Bastings",
                "Ivan Titov",
                "Wilker Aziz",
                "Diego Marcheggiani",
                "Khalil Sima'an"
            ],
            "abstract": "We present a simple and effective approach to incorporating syntactic\nstructure into neural attention-based encoder-decoder models for machine\ntranslation. We rely on graph-convolutional networks (GCNs), a recent class of\nneural networks developed for modeling graph-structured data. Our GCNs use\npredicted syntactic dependency trees of source sentences to produce\nrepresentations of words (i.e. hidden states of the encoder) that are sensitive\nto their syntactic neighborhoods. GCNs take word representations as input and\nproduce word representations as output, so they can easily be incorporated as\nlayers into standard encoders (e.g., on top of bidirectional RNNs or\nconvolutional neural networks). We evaluate their effectiveness with\nEnglish-German and English-Czech translation experiments for different types of\nencoders and observe substantial improvements over their syntax-agnostic\nversions in all the considered setups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04743v3",
            "date": "2017-04-16 09:54:50+00:00",
            "title": "Towards String-to-Tree Neural Machine Translation",
            "authors": [
                "Roee Aharoni",
                "Yoav Goldberg"
            ],
            "abstract": "We present a simple method to incorporate syntactic information about the\ntarget language in a neural machine translation system by translating into\nlinearized, lexicalized constituency trees. An experiment on the WMT16\nGerman-English news translation task resulted in an improved BLEU score when\ncompared to a syntax-agnostic NMT baseline trained on the same dataset. An\nanalysis of the translations from the syntax-aware system shows that it\nperforms more reordering during translation in comparison to the baseline. A\nsmall-scale human evaluation also showed an advantage to the syntax-aware\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04856v1",
            "date": "2017-04-17 03:20:07+00:00",
            "title": "A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes",
            "authors": [
                "Pablo Loyola",
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "We propose a model to automatically describe changes introduced in the source\ncode of a program using natural language. Our method receives as input a set of\ncode commits, which contains both the modifications and message introduced by\nan user. These two modalities are used to train an encoder-decoder\narchitecture. We evaluated our approach on twelve real world open source\nprojects from four different programming languages. Quantitative and\nqualitative results showed that the proposed approach can generate feasible and\nsemantically sound descriptions not only in standard in-project settings, but\nalso in a cross-project setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04859v2",
            "date": "2017-04-17 03:30:30+00:00",
            "title": "Learning Character-level Compositionality with Visual Features",
            "authors": [
                "Frederick Liu",
                "Han Lu",
                "Chieh Lo",
                "Graham Neubig"
            ],
            "abstract": "Previous work has modeled the compositionality of words by creating\ncharacter-level models of meaning, reducing problems of sparsity for rare\nwords. However, in many writing systems compositionality has an effect even on\nthe character-level: the meaning of a character is derived by the sum of its\nparts. In this paper, we model this effect by creating embeddings for\ncharacters based on their visual characteristics, creating an image for the\ncharacter and running it through a convolutional neural network to produce a\nvisual character embedding. Experiments on a text classification task\ndemonstrate that such model allows for better processing of instances with rare\ncharacters in languages such as Chinese, Japanese, and Korean. Additionally,\nqualitative analyses demonstrate that our proposed model learns to focus on the\nparts of characters that carry semantic content, resulting in embeddings that\nare coherent in visual space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04920v3",
            "date": "2017-04-17 10:18:32+00:00",
            "title": "Deep Joint Entity Disambiguation with Local Neural Attention",
            "authors": [
                "Octavian-Eugen Ganea",
                "Thomas Hofmann"
            ],
            "abstract": "We propose a novel deep learning model for joint document-level entity\ndisambiguation, which leverages learned neural representations. Key components\nare entity embeddings, a neural attention mechanism over local context windows,\nand a differentiable joint inference stage for disambiguation. Our approach\nthereby combines benefits of deep learning with more traditional approaches\nsuch as graphical models and probabilistic mention-entity maps. Extensive\nexperiments show that we are able to obtain competitive or state-of-the-art\naccuracy at moderate computational costs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05162v1",
            "date": "2017-04-18 01:04:49+00:00",
            "title": "Automatic Disambiguation of French Discourse Connectives",
            "authors": [
                "Majid Laali",
                "Leila Kosseim"
            ],
            "abstract": "Discourse connectives (e.g. however, because) are terms that can explicitly\nconvey a discourse relation within a text. While discourse connectives have\nbeen shown to be an effective clue to automatically identify discourse\nrelations, they are not always used to convey such relations, thus they should\nfirst be disambiguated between discourse-usage non-discourse-usage. In this\npaper, we investigate the applicability of features proposed for the\ndisambiguation of English discourse connectives for French. Our results with\nthe French Discourse Treebank (FDTB) show that syntactic and lexical features\ndeveloped for English texts are as effective for French and allow the\ndisambiguation of French discourse connectives with an accuracy of 94.2%.",
            "journal ref": "International Journal of Computational Linguistics and\n  Applications, vol. 7, no. 1, 2016, pp. 11-30",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05179v3",
            "date": "2017-04-18 02:42:17+00:00",
            "title": "SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine",
            "authors": [
                "Matthew Dunn",
                "Levent Sagun",
                "Mike Higgins",
                "V. Ugur Guney",
                "Volkan Cirik",
                "Kyunghyun Cho"
            ],
            "abstract": "We publicly release a new large-scale dataset, called SearchQA, for machine\ncomprehension, or question-answering. Unlike recently released datasets, such\nas DeepMind CNN/DailyMail and SQuAD, the proposed SearchQA was constructed to\nreflect a full pipeline of general question-answering. That is, we start not\nfrom an existing article and generate a question-answer pair, but start from an\nexisting question-answer pair, crawled from J! Archive, and augment it with\ntext snippets retrieved by Google. Following this approach, we built SearchQA,\nwhich consists of more than 140k question-answer pairs with each pair having\n49.6 snippets on average. Each question-answer-context tuple of the SearchQA\ncomes with additional meta-data such as the snippet's URL, which we believe\nwill be valuable resources for future research. We conduct human evaluation as\nwell as test two baseline methods, one simple word selection and the other deep\nlearning based, on the SearchQA. We show that there is a meaningful gap between\nthe human and machine performances. This suggests that the proposed dataset\ncould well serve as a benchmark for question-answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05228v3",
            "date": "2017-04-18 08:24:20+00:00",
            "title": "Sentiment analysis based on rhetorical structure theory: Learning deep neural networks from discourse trees",
            "authors": [
                "Mathias Kraus",
                "Stefan Feuerriegel"
            ],
            "abstract": "Prominent applications of sentiment analysis are countless, covering areas\nsuch as marketing, customer service and communication. The conventional\nbag-of-words approach for measuring sentiment merely counts term frequencies;\nhowever, it neglects the position of the terms within the discourse. As a\nremedy, we develop a discourse-aware method that builds upon the discourse\nstructure of documents. For this purpose, we utilize rhetorical structure\ntheory to label (sub-)clauses according to their hierarchical relationships and\nthen assign polarity scores to individual leaves. To learn from the resulting\nrhetorical structure, we propose a tensor-based, tree-structured deep neural\nnetwork (named Discourse-LSTM) in order to process the complete discourse tree.\nThe underlying tensors infer the salient passages of narrative materials. In\naddition, we suggest two algorithms for data augmentation (node reordering and\nartificial leaf insertion) that increase our training set and reduce\noverfitting. Our benchmarks demonstrate the superior performance of our\napproach. Moreover, our tensor structure reveals the salient text passages and\nthereby provides explanatory insights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05347v2",
            "date": "2017-04-18 14:12:37+00:00",
            "title": "Baselines and test data for cross-lingual inference",
            "authors": [
                "\u017deljko Agi\u0107",
                "Natalie Schluter"
            ],
            "abstract": "The recent years have seen a revival of interest in textual entailment,\nsparked by i) the emergence of powerful deep neural network learners for\nnatural language processing and ii) the timely development of large-scale\nevaluation datasets such as SNLI. Recast as natural language inference, the\nproblem now amounts to detecting the relation between pairs of statements: they\neither contradict or entail one another, or they are mutually neutral. Current\nresearch in natural language inference is effectively exclusive to English. In\nthis paper, we propose to advance the research in SNLI-style natural language\ninference toward multilingual evaluation. To that end, we provide test data for\nfour major languages: Arabic, French, Spanish, and Russian. We experiment with\na set of baselines. Our systems are based on cross-lingual word embeddings and\nmachine translation. While our best system scores an average accuracy of just\nover 75%, we focus largely on enabling further research in multilingual\ninference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05358v1",
            "date": "2017-04-18 14:30:32+00:00",
            "title": "Representing Sentences as Low-Rank Subspaces",
            "authors": [
                "Jiaqi Mu",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Sentences are important semantic units of natural language. A generic,\ndistributional representation of sentences that can capture the latent\nsemantics is beneficial to multiple downstream applications. We observe a\nsimple geometry of sentences -- the word representations of a given sentence\n(on average 10.23 words in all SemEval datasets with a standard deviation 4.84)\nroughly lie in a low-rank subspace (roughly, rank 4). Motivated by this\nobservation, we represent a sentence by the low-rank subspace spanned by its\nword vectors. Such an unsupervised representation is empirically validated via\nsemantic textual similarity tasks on 19 different datasets, where it\noutperforms the sophisticated neural network models, including skip-thought\nvectors, by 15% on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05415v2",
            "date": "2017-04-18 16:38:01+00:00",
            "title": "An Empirical Analysis of NMT-Derived Interlingual Embeddings and their Use in Parallel Sentence Identification",
            "authors": [
                "Cristina Espa\u00f1a-Bonet",
                "\u00c1d\u00e1m Csaba Varga",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Josef van Genabith"
            ],
            "abstract": "End-to-end neural machine translation has overtaken statistical machine\ntranslation in terms of translation quality for some language pairs, specially\nthose with large amounts of parallel data. Besides this palpable improvement,\nneural networks provide several new properties. A single system can be trained\nto translate between many languages at almost no additional cost other than\ntraining time. Furthermore, internal representations learned by the network\nserve as a new semantic representation of words -or sentences- which, unlike\nstandard word embeddings, are learned in an essentially bilingual or even\nmultilingual context. In view of these properties, the contribution of the\npresent work is two-fold. First, we systematically study the NMT context\nvectors, i.e. output of the encoder, and their power as an interlingua\nrepresentation of a sentence. We assess their quality and effectiveness by\nmeasuring similarities across translations, as well as semantically related and\nsemantically unrelated sentence pairs. Second, as extrinsic evaluation of the\nfirst point, we identify parallel sentences in comparable corpora, obtaining an\nF1=98.2% on data from a shared task when using only NMT context vectors. Using\ncontext vectors jointly with similarity measures F1 reaches 98.9%.",
            "journal ref": "IEEE Journal of Selected Topics in Signal Processing, vol. 11, no.\n  8, pp. 1340-1350, December 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05426v4",
            "date": "2017-04-18 17:10:13+00:00",
            "title": "A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference",
            "authors": [
                "Adina Williams",
                "Nikita Nangia",
                "Samuel R. Bowman"
            ],
            "abstract": "This paper introduces the Multi-Genre Natural Language Inference (MultiNLI)\ncorpus, a dataset designed for use in the development and evaluation of machine\nlearning models for sentence understanding. In addition to being one of the\nlargest corpora available for the task of NLI, at 433k examples, this corpus\nimproves upon available resources in its coverage: it offers data from ten\ndistinct genres of written and spoken English--making it possible to evaluate\nsystems on nearly the full complexity of the language--and it offers an\nexplicit setting for the evaluation of cross-genre domain adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05571v1",
            "date": "2017-04-19 00:55:23+00:00",
            "title": "Predicting Role Relevance with Minimal Domain Expertise in a Financial Domain",
            "authors": [
                "Mayank Kejriwal"
            ],
            "abstract": "Word embeddings have made enormous inroads in recent years in a wide variety\nof text mining applications. In this paper, we explore a word embedding-based\narchitecture for predicting the relevance of a role between two financial\nentities within the context of natural language sentences. In this extended\nabstract, we propose a pooled approach that uses a collection of sentences to\ntrain word embeddings using the skip-gram word2vec architecture. We use the\nword embeddings to obtain context vectors that are assigned one or more labels\nbased on manual annotations. We train a machine learning classifier using the\nlabeled context vectors, and use the trained classifier to predict contextual\nrole relevance on test data. Our approach serves as a good minimal-expertise\nbaseline for the task as it is simple and intuitive, uses open-source modules,\nrequires little feature crafting effort and performs well across roles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05611v1",
            "date": "2017-04-19 05:02:05+00:00",
            "title": "Dependency resolution and semantic mining using Tree Adjoining Grammars for Tamil Language",
            "authors": [
                "Vijay Krishna Menon",
                "S Rajendran",
                "M Anandkumar",
                "K P Soman"
            ],
            "abstract": "Tree adjoining grammars (TAGs) provide an ample tool to capture syntax of\nmany Indian languages. Tamil represents a special challenge to computational\nformalisms as it has extensive agglutinative morphology and a comparatively\ndifficult argument structure. Modelling Tamil syntax and morphology using TAG\nis an interesting problem which has not been in focus even though TAGs are over\n4 decades old, since its inception. Our research with Tamil TAGs have shown us\nthat we can not only represent syntax of the language, but to an extent mine\nout semantics through dependency resolution of the sentence. But in order to\ndemonstrate this phenomenal property, we need to parse Tamil language sentences\nusing TAGs we have built and through parsing obtain a derivation we could use\nto resolve dependencies, thus proving the semantic property. We use an in-house\ndeveloped pseudo lexical TAG chart parser; algorithm given by Schabes and Joshi\n(1988), for generating derivations of sentences. We do not use any statistics\nto rank out ambiguous derivations but rather use all of them to understand the\nmentioned semantic relation with in TAGs for Tamil. We shall also present a\nbrief parser analysis for the completeness of our discussions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05742v1",
            "date": "2017-04-19 14:17:25+00:00",
            "title": "Adversarial Multi-task Learning for Text Classification",
            "authors": [
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural network models have shown their promising opportunities for multi-task\nlearning, which focus on learning the shared layers to extract the common and\ntask-invariant features. However, in most existing approaches, the extracted\nshared features are prone to be contaminated by task-specific features or the\nnoise brought by other tasks. In this paper, we propose an adversarial\nmulti-task learning framework, alleviating the shared and private latent\nfeature spaces from interfering with each other. We conduct extensive\nexperiments on 16 different text classification tasks, which demonstrates the\nbenefits of our approach. Besides, we show that the shared knowledge learned by\nour proposed model can be regarded as off-the-shelf knowledge and easily\ntransferred to new tasks. The datasets of all 16 tasks are publicly available\nat \\url{http://nlp.fudan.edu.cn/data/}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05781v1",
            "date": "2017-04-19 15:41:34+00:00",
            "title": "Redefining Context Windows for Word Embedding Models: An Experimental Study",
            "authors": [
                "Pierre Lison",
                "Andrey Kutuzov"
            ],
            "abstract": "Distributional semantic models learn vector representations of words through\nthe contexts they occur in. Although the choice of context (which often takes\nthe form of a sliding window) has a direct influence on the resulting\nembeddings, the exact role of this model component is still not fully\nunderstood. This paper presents a systematic analysis of context windows based\non a set of four distinct hyper-parameters. We train continuous Skip-Gram\nmodels on two English-language corpora for various combinations of these\nhyper-parameters, and evaluate them on both lexical similarity and analogy\ntasks. Notable experimental results are the positive impact of cross-sentential\ncontexts and the surprisingly good performance of right-context windows.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05958v2",
            "date": "2017-04-19 23:54:46+00:00",
            "title": "Global Relation Embedding for Relation Extraction",
            "authors": [
                "Yu Su",
                "Honglei Liu",
                "Semih Yavuz",
                "Izzeddin Gur",
                "Huan Sun",
                "Xifeng Yan"
            ],
            "abstract": "We study the problem of textual relation embedding with distant supervision.\nTo combat the wrong labeling problem of distant supervision, we propose to\nembed textual relations with global statistics of relations, i.e., the\nco-occurrence statistics of textual and knowledge base relations collected from\nthe entire corpus. This approach turns out to be more robust to the training\nnoise introduced by distant supervision. On a popular relation extraction\ndataset, we show that the learned textual relation embedding can be used to\naugment existing relation extraction models and significantly improve their\nperformance. Most remarkably, for the top 1,000 relational facts discovered by\nthe best existing model, the precision can be improved from 83.9% to 89.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05974v2",
            "date": "2017-04-20 01:26:23+00:00",
            "title": "Cross-domain Semantic Parsing via Paraphrasing",
            "authors": [
                "Yu Su",
                "Xifeng Yan"
            ],
            "abstract": "Existing studies on semantic parsing mainly focus on the in-domain setting.\nWe formulate cross-domain semantic parsing as a domain adaptation problem:\ntrain a semantic parser on some source domains and then adapt it to the target\ndomain. Due to the diversity of logical forms in different domains, this\nproblem presents unique and intriguing challenges. By converting logical forms\ninto canonical utterances in natural language, we reduce semantic parsing to\nparaphrasing, and develop an attentive sequence-to-sequence paraphrase model\nthat is general and flexible to adapt to different domains. We discover two\nproblems, small micro variance and large macro variance, of pre-trained word\nembeddings that hinder their direct use in neural networks, and propose\nstandardization techniques as a remedy. On the popular Overnight dataset, which\ncontains eight domains, we show that both cross-domain training and\nstandardized pre-trained word embeddings can bring significant improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06104v2",
            "date": "2017-04-20 12:20:43+00:00",
            "title": "Neural End-to-End Learning for Computational Argumentation Mining",
            "authors": [
                "Steffen Eger",
                "Johannes Daxenberger",
                "Iryna Gurevych"
            ],
            "abstract": "We investigate neural techniques for end-to-end computational argumentation\nmining (AM). We frame AM both as a token-based dependency parsing and as a\ntoken-based sequence tagging problem, including a multi-task learning setup.\nContrary to models that operate on the argument component level, we find that\nframing AM as dependency parsing leads to subpar performance results. In\ncontrast, less complex (local) tagging models based on BiLSTMs perform robustly\nacross classification scenarios, being able to catch long-range dependencies\ninherent to the AM problem. Moreover, we find that jointly learning 'natural'\nsubtasks, in a multi-task learning setup, improves performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06217v1",
            "date": "2017-04-20 16:30:39+00:00",
            "title": "Reinforcement Learning with External Knowledge and Two-Stage Q-functions for Predicting Popular Reddit Threads",
            "authors": [
                "Ji He",
                "Mari Ostendorf",
                "Xiaodong He"
            ],
            "abstract": "This paper addresses the problem of predicting popularity of comments in an\nonline discussion forum using reinforcement learning, particularly addressing\ntwo challenges that arise from having natural language state and action spaces.\nFirst, the state representation, which characterizes the history of comments\ntracked in a discussion at a particular point, is augmented to incorporate the\nglobal context represented by discussions on world events available in an\nexternal knowledge source. Second, a two-stage Q-learning framework is\nintroduced, making it feasible to search the combinatorial action space while\nalso accounting for redundancy among sub-actions. We experiment with five\nReddit communities, showing that the two methods improve over previous reported\nresults on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06360v1",
            "date": "2017-04-20 23:02:14+00:00",
            "title": "SwellShark: A Generative Model for Biomedical Named Entity Recognition without Labeled Data",
            "authors": [
                "Jason Fries",
                "Sen Wu",
                "Alex Ratner",
                "Christopher R\u00e9"
            ],
            "abstract": "We present SwellShark, a framework for building biomedical named entity\nrecognition (NER) systems quickly and without hand-labeled data. Our approach\nviews biomedical resources like lexicons as function primitives for\nautogenerating weak supervision. We then use a generative model to unify and\ndenoise this supervision and construct large-scale, probabilistically labeled\ndatasets for training high-accuracy NER taggers. In three biomedical NER tasks,\nSwellShark achieves competitive scores with state-of-the-art supervised\nbenchmarks using no hand-labeled training data. In a drug name extraction task\nusing patient medical records, one domain expert using SwellShark achieved\nwithin 5.1% of a crowdsourced annotation approach -- which originally utilized\n20 teams over the course of several weeks -- in 24 hours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06380v1",
            "date": "2017-04-21 02:27:26+00:00",
            "title": "Improving Context Aware Language Models",
            "authors": [
                "Aaron Jaech",
                "Mari Ostendorf"
            ],
            "abstract": "Increased adaptability of RNN language models leads to improved predictions\nthat benefit many applications. However, current methods do not take full\nadvantage of the RNN structure. We show that the most widely-used approach to\nadaptation (concatenating the context with the word embedding at the input to\nthe recurrent layer) is outperformed by a model that has some low-cost\nimprovements: adaptation of both the hidden and output layers. and a feature\nhashing bias term to capture context idiosyncrasies. Experiments on language\nmodeling and classification tasks using three different corpora demonstrate the\nadvantages of the proposed techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06393v1",
            "date": "2017-04-21 04:36:55+00:00",
            "title": "Neural System Combination for Machine Translation",
            "authors": [
                "Long Zhou",
                "Wenpeng Hu",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Neural machine translation (NMT) becomes a new approach to machine\ntranslation and generates much more fluent results compared to statistical\nmachine translation (SMT).\n  However, SMT is usually better than NMT in translation adequacy. It is\ntherefore a promising direction to combine the advantages of both NMT and SMT.\n  In this paper, we propose a neural system combination framework leveraging\nmulti-source NMT, which takes as input the outputs of NMT and SMT systems and\nproduces the final translation.\n  Extensive experiments on the Chinese-to-English translation task show that\nour model archives significant improvement by 5.3 BLEU points over the best\nsingle system output and 3.4 BLEU points over the state-of-the-art traditional\nsystem combination methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06692v1",
            "date": "2017-04-21 19:47:30+00:00",
            "title": "Improving Semantic Composition with Offset Inference",
            "authors": [
                "Thomas Kober",
                "Julie Weeds",
                "Jeremy Reffin",
                "David Weir"
            ],
            "abstract": "Count-based distributional semantic models suffer from sparsity due to\nunobserved but plausible co-occurrences in any text collection. This problem is\namplified for models like Anchored Packed Trees (APTs), that take the\ngrammatical type of a co-occurrence into account. We therefore introduce a\nnovel form of distributional inference that exploits the rich type structure in\nAPTs and infers missing data by the same mechanism that is used for semantic\ncomposition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06779v1",
            "date": "2017-04-22 09:59:42+00:00",
            "title": "Lexical Features in Coreference Resolution: To be Used With Caution",
            "authors": [
                "Nafise Sadat Moosavi",
                "Michael Strube"
            ],
            "abstract": "Lexical features are a major source of information in state-of-the-art\ncoreference resolvers. Lexical features implicitly model some of the linguistic\nphenomena at a fine granularity level. They are especially useful for\nrepresenting the context of mentions. In this paper we investigate a drawback\nof using many lexical features in state-of-the-art coreference resolvers. We\nshow that if coreference resolvers mainly rely on lexical features, they can\nhardly generalize to unseen domains. Furthermore, we show that the current\ncoreference resolution evaluation is clearly flawed by only evaluating on a\nspecific split of a specific dataset in which there is a notable overlap\nbetween the training, development and test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06836v1",
            "date": "2017-04-22 18:59:25+00:00",
            "title": "Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation",
            "authors": [
                "Lotem Peled",
                "Roi Reichart"
            ],
            "abstract": "Sarcasm is a form of speech in which speakers say the opposite of what they\ntruly mean in order to convey a strong sentiment. In other words, \"Sarcasm is\nthe giant chasm between what I say, and the person who doesn't get it.\". In\nthis paper we present the novel task of sarcasm interpretation, defined as the\ngeneration of a non-sarcastic utterance conveying the same message as the\noriginal sarcastic one. We introduce a novel dataset of 3000 sarcastic tweets,\neach interpreted by five human judges. Addressing the task as monolingual\nmachine translation (MT), we experiment with MT algorithms and evaluation\nmeasures. We then present SIGN: an MT based sarcasm interpretation algorithm\nthat targets sentiment words, a defining element of textual sarcasm. We show\nthat while the scores of n-gram based automatic measures are similar for all\ninterpretation models, SIGN's interpretations are scored higher by humans for\nadequacy and sentiment polarity. We conclude with a discussion on future\nresearch directions for our new task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06841v1",
            "date": "2017-04-22 19:39:32+00:00",
            "title": "Medical Text Classification using Convolutional Neural Networks",
            "authors": [
                "Mark Hughes",
                "Irene Li",
                "Spyros Kotoulas",
                "Toyotaro Suzumura"
            ],
            "abstract": "We present an approach to automatically classify clinical text at a sentence\nlevel. We are using deep convolutional neural networks to represent complex\nfeatures. We train the network on a dataset providing a broad categorization of\nhealth information. Through a detailed evaluation, we demonstrate that our\nmethod outperforms several approaches widely used in natural language\nprocessing tasks by about 15%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06851v1",
            "date": "2017-04-22 21:10:10+00:00",
            "title": "Affect-LM: A Neural Language Model for Customizable Affective Text Generation",
            "authors": [
                "Sayan Ghosh",
                "Mathieu Chollet",
                "Eugene Laksana",
                "Louis-Philippe Morency",
                "Stefan Scherer"
            ],
            "abstract": "Human verbal communication includes affective messages which are conveyed\nthrough use of emotionally colored words. There has been a lot of research in\nthis direction but the problem of integrating state-of-the-art neural language\nmodels with affective information remains an area ripe for exploration. In this\npaper, we propose an extension to an LSTM (Long Short-Term Memory) language\nmodel for generating conversational text, conditioned on affect categories. Our\nproposed model, Affect-LM enables us to customize the degree of emotional\ncontent in generated sentences through an additional design parameter.\nPerception studies conducted using Amazon Mechanical Turk show that Affect-LM\ngenerates naturally looking emotional sentences without sacrificing grammatical\ncorrectness. Affect-LM also learns affect-discriminative word representations,\nand perplexity experiments show that additional affective information in\nconversational text can improve language model prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06855v2",
            "date": "2017-04-22 22:56:04+00:00",
            "title": "Deep Multitask Learning for Semantic Dependency Parsing",
            "authors": [
                "Hao Peng",
                "Sam Thomson",
                "Noah A. Smith"
            ],
            "abstract": "We present a deep neural architecture that parses sentences into three\nsemantic dependency graph formalisms. By using efficient, nearly arc-factored\ninference and a bidirectional-LSTM composed with a multi-layer perceptron, our\nbase system is able to significantly improve the state of the art for semantic\ndependency parsing, without using hand-engineered features or syntax. We then\nexplore two multitask learning approaches---one that shares parameters across\nformalisms, and one that uses higher-order structures to predict the graphs\njointly. We find that both approaches improve performance across formalisms on\naverage, achieving a new state of the art. Our code is open-source and\navailable at https://github.com/Noahs-ARK/NeurboParser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06869v1",
            "date": "2017-04-23 01:14:55+00:00",
            "title": "Argument Mining with Structured SVMs and RNNs",
            "authors": [
                "Vlad Niculae",
                "Joonsuk Park",
                "Claire Cardie"
            ],
            "abstract": "We propose a novel factor graph model for argument mining, designed for\nsettings in which the argumentative relations in a document do not necessarily\nform a tree structure. (This is the case in over 20% of the web comments\ndataset we release.) Our model jointly learns elementary unit type\nclassification and argumentative relation prediction. Moreover, our model\nsupports SVM and RNN parametrizations, can enforce structure constraints (e.g.,\ntransitivity), and can express dependencies between adjacent relations and\npropositions. Our approaches outperform unstructured baselines in both web\ncomments and argumentative essay datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06879v3",
            "date": "2017-04-23 04:34:26+00:00",
            "title": "Deep Keyphrase Generation",
            "authors": [
                "Rui Meng",
                "Sanqiang Zhao",
                "Shuguang Han",
                "Daqing He",
                "Peter Brusilovsky",
                "Yu Chi"
            ],
            "abstract": "Keyphrase provides highly-condensed information that can be effectively used\nfor understanding, organizing and retrieving text content. Though previous\nstudies have provided many workable solutions for automated keyphrase\nextraction, they commonly divided the to-be-summarized content into multiple\ntext chunks, then ranked and selected the most meaningful ones. These\napproaches could neither identify keyphrases that do not appear in the text,\nnor capture the real semantic meaning behind the text. We propose a generative\nmodel for keyphrase prediction with an encoder-decoder framework, which can\neffectively overcome the above drawbacks. We name it as deep keyphrase\ngeneration since it attempts to capture the deep semantic meaning of the\ncontent with a deep learning method. Empirical analysis on six datasets\ndemonstrates that our proposed model not only achieves a significant\nperformance boost on extracting keyphrases that appear in the source text, but\nalso can generate absent keyphrases based on the semantic meaning of the text.\nCode and dataset are available at\nhttps://github.com/memray/OpenNMT-kpg-release.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06918v1",
            "date": "2017-04-23 12:38:13+00:00",
            "title": "Neural Machine Translation via Binary Code Prediction",
            "authors": [
                "Yusuke Oda",
                "Philip Arthur",
                "Graham Neubig",
                "Koichiro Yoshino",
                "Satoshi Nakamura"
            ],
            "abstract": "In this paper, we propose a new method for calculating the output layer in\nneural machine translation systems. The method is based on predicting a binary\ncode for each word and can reduce computation time/memory requirements of the\noutput layer to be logarithmic in vocabulary size in the best case. In\naddition, we also introduce two advanced approaches to improve the robustness\nof the proposed model: using error-correcting codes and combining softmax and\nbinary codes. Experiments on two English-Japanese bidirectional translation\ntasks show proposed models achieve BLEU scores that approach the softmax, while\nreducing memory usage to the order of less than 1/10 and improving decoding\nspeed on CPUs by x5 to x10.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06936v1",
            "date": "2017-04-23 15:16:53+00:00",
            "title": "A* CCG Parsing with a Supertag and Dependency Factored Model",
            "authors": [
                "Masashi Yoshikawa",
                "Hiroshi Noji",
                "Yuji Matsumoto"
            ],
            "abstract": "We propose a new A* CCG parsing model in which the probability of a tree is\ndecomposed into factors of CCG categories and its syntactic dependencies both\ndefined on bi-directional LSTMs. Our factored model allows the precomputation\nof all probabilities and runs very efficiently, while modeling sentence\nstructures explicitly via dependencies. Our model achieves the state-of-the-art\nresults on English and Japanese CCG parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06986v1",
            "date": "2017-04-23 21:31:22+00:00",
            "title": "Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling",
            "authors": [
                "Kazuya Kawakami",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "Fixed-vocabulary language models fail to account for one of the most\ncharacteristic statistical facts of natural language: the frequent creation and\nreuse of new word types. Although character-level language models offer a\npartial solution in that they can create word types not attested in the\ntraining corpus, they do not capture the \"bursty\" distribution of such words.\nIn this paper, we augment a hierarchical LSTM language model that generates\nsequences of word tokens character by character with a caching mechanism that\nlearns to reuse previously generated words. To validate our model we construct\na new open-vocabulary language modeling corpus (the Multilingual Wikipedia\nCorpus, MWC) from comparable Wikipedia articles in 7 typologically diverse\nlanguages and demonstrate the effectiveness of our model across this range of\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07047v1",
            "date": "2017-04-24 05:50:29+00:00",
            "title": "Fast and Accurate Neural Word Segmentation for Chinese",
            "authors": [
                "Deng Cai",
                "Hai Zhao",
                "Zhisong Zhang",
                "Yuan Xin",
                "Yongjian Wu",
                "Feiyue Huang"
            ],
            "abstract": "Neural models with minimal feature engineering have achieved competitive\nperformance against traditional methods for the task of Chinese word\nsegmentation. However, both training and working procedures of the current\nneural models are computationally inefficient. This paper presents a greedy\nneural word segmenter with balanced word and character embedding inputs to\nalleviate the existing drawbacks. Our segmenter is truly end-to-end, capable of\nperforming segmentation much faster and even more accurate than\nstate-of-the-art neural models on Chinese benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07073v1",
            "date": "2017-04-24 07:57:37+00:00",
            "title": "Selective Encoding for Abstractive Sentence Summarization",
            "authors": [
                "Qingyu Zhou",
                "Nan Yang",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "We propose a selective encoding model to extend the sequence-to-sequence\nframework for abstractive sentence summarization. It consists of a sentence\nencoder, a selective gate network, and an attention equipped decoder. The\nsentence encoder and decoder are built with recurrent neural networks. The\nselective gate network constructs a second level sentence representation by\ncontrolling the information flow from encoder to decoder. The second level\nrepresentation is tailored for sentence summarization task, which leads to\nbetter performance. We evaluate our model on the English Gigaword, DUC 2004 and\nMSR abstractive sentence summarization datasets. The experimental results show\nthat the proposed selective encoding model outperforms the state-of-the-art\nbaseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07092v2",
            "date": "2017-04-24 08:50:15+00:00",
            "title": "Robust Incremental Neural Semantic Graph Parsing",
            "authors": [
                "Jan Buys",
                "Phil Blunsom"
            ],
            "abstract": "Parsing sentences to linguistically-expressive semantic representations is a\nkey goal of Natural Language Processing. Yet statistical parsing has focused\nalmost exclusively on bilexical dependencies or domain-specific logical forms.\nWe propose a neural encoder-decoder transition-based parser which is the first\nfull-coverage semantic graph parser for Minimal Recursion Semantics (MRS). The\nmodel architecture uses stack-based embedding features, predicting graphs\njointly with unlexicalized predicates and their token alignments. Our parser is\nmore accurate than attention-based baselines on MRS, and on an additional\nAbstract Meaning Representation (AMR) benchmark, and GPU batch processing makes\nit an order of magnitude faster than a high-precision grammar-based parser.\nFurther, the 86.69% Smatch score of our MRS parser is higher than the\nupper-bound on AMR parsing, making MRS an attractive choice as a semantic\nrepresentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07130v1",
            "date": "2017-04-24 10:38:24+00:00",
            "title": "Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings",
            "authors": [
                "He He",
                "Anusha Balakrishnan",
                "Mihail Eric",
                "Percy Liang"
            ],
            "abstract": "We study a symmetric collaborative dialogue setting in which two agents, each\nwith private knowledge, must strategically communicate to achieve a common\ngoal. The open-ended dialogue state in this setting poses new challenges for\nexisting dialogue systems. We collected a dataset of 11K human-human dialogues,\nwhich exhibits interesting lexical, semantic, and strategic elements. To model\nboth structured knowledge and unstructured language, we propose a neural model\nwith dynamic knowledge graph embeddings that evolve as the dialogue progresses.\nAutomatic and human evaluations show that our model is both more effective at\nachieving the goal and more human-like than baseline neural and rule-based\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07138v2",
            "date": "2017-04-24 10:55:20+00:00",
            "title": "Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search",
            "authors": [
                "Chris Hokamp",
                "Qun Liu"
            ],
            "abstract": "We present Grid Beam Search (GBS), an algorithm which extends beam search to\nallow the inclusion of pre-specified lexical constraints. The algorithm can be\nused with any model that generates a sequence $ \\mathbf{\\hat{y}} =\n\\{y_{0}\\ldots y_{T}\\} $, by maximizing $ p(\\mathbf{y} | \\mathbf{x}) =\n\\prod\\limits_{t}p(y_{t} | \\mathbf{x}; \\{y_{0} \\ldots y_{t-1}\\}) $. Lexical\nconstraints take the form of phrases or words that must be present in the\noutput sequence. This is a very general way to incorporate additional knowledge\ninto a model's output without requiring any modification of the model\nparameters or training data. We demonstrate the feasibility and flexibility of\nLexically Constrained Decoding by conducting experiments on Neural\nInteractive-Predictive Translation, as well as Domain Adaptation for Neural\nMachine Translation. Experiments show that GBS can provide large improvements\nin translation quality in interactive scenarios, and that, even without any\nuser input, GBS can be used to achieve significant gains in performance in\ndomain adaptation scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07146v1",
            "date": "2017-04-24 11:14:20+00:00",
            "title": "Found in Translation: Reconstructing Phylogenetic Language Trees from Translations",
            "authors": [
                "Ella Rabinovich",
                "Noam Ordan",
                "Shuly Wintner"
            ],
            "abstract": "Translation has played an important role in trade, law, commerce, politics,\nand literature for thousands of years. Translators have always tried to be\ninvisible; ideal translations should look as if they were written originally in\nthe target language. We show that traces of the source language remain in the\ntranslation product to the extent that it is possible to uncover the history of\nthe source language by looking only at the translation. Specifically, we\nautomatically reconstruct phylogenetic language trees from monolingual texts\n(translated from several source languages). The signal of the source language\nis so powerful that it is retained even after two phases of translation. This\nstrongly indicates that source language interference is the most dominant\ncharacteristic of translated texts, overshadowing the more subtle signals of\nuniversal properties of translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07157v1",
            "date": "2017-04-24 11:49:08+00:00",
            "title": "Watset: Automatic Induction of Synsets from a Graph of Synonyms",
            "authors": [
                "Dmitry Ustalov",
                "Alexander Panchenko",
                "Chris Biemann"
            ],
            "abstract": "This paper presents a new graph-based approach that induces synsets using\nsynonymy dictionaries and word embeddings. First, we build a weighted graph of\nsynonyms extracted from commonly available resources, such as Wiktionary.\nSecond, we apply word sense induction to deal with ambiguous words. Finally, we\ncluster the disambiguated version of the ambiguous input graph into synsets.\nOur meta-clustering approach lets us use an efficient hard clustering algorithm\nto perform a fuzzy clustering of the graph. Despite its simplicity, our\napproach shows excellent results, outperforming five competitive\nstate-of-the-art methods in terms of F-score on three gold standard datasets\nfor English and Russian derived from large-scale manually constructed lexical\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07203v3",
            "date": "2017-04-24 13:13:30+00:00",
            "title": "What is the Essence of a Claim? Cross-Domain Claim Identification",
            "authors": [
                "Johannes Daxenberger",
                "Steffen Eger",
                "Ivan Habernal",
                "Christian Stab",
                "Iryna Gurevych"
            ],
            "abstract": "Argument mining has become a popular research area in NLP. It typically\nincludes the identification of argumentative components, e.g. claims, as the\ncentral component of an argument. We perform a qualitative analysis across six\ndifferent datasets and show that these appear to conceptualize claims quite\ndifferently. To learn about the consequences of such different\nconceptualizations of claim for practical applications, we carried out\nextensive experiments using state-of-the-art feature-rich and deep learning\nsystems, to identify claims in a cross-domain fashion. While the divergent\nperception of claims in different datasets is indeed harmful to cross-domain\nclassification, we show that there are shared properties on the lexical level\nas well as system configurations that can help to overcome these gaps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07329v1",
            "date": "2017-04-24 17:07:26+00:00",
            "title": "A Trie-Structured Bayesian Model for Unsupervised Morphological Segmentation",
            "authors": [
                "Murathan Kurfal\u0131",
                "Ahmet \u00dcst\u00fcn",
                "Burcu Can"
            ],
            "abstract": "In this paper, we introduce a trie-structured Bayesian model for unsupervised\nmorphological segmentation. We adopt prior information from different sources\nin the model. We use neural word embeddings to discover words that are\nmorphologically derived from each other and thereby that are semantically\nsimilar. We use letter successor variety counts obtained from tries that are\nbuilt by neural word embeddings. Our results show that using different\ninformation sources such as neural word embeddings and letter successor variety\nas prior information improves morphological segmentation in a Bayesian model.\nOur model outperforms other unsupervised morphological segmentation models on\nTurkish and gives promising results on English and German for scarce resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07398v2",
            "date": "2017-04-24 18:04:17+00:00",
            "title": "Predicting Native Language from Gaze",
            "authors": [
                "Yevgeni Berzak",
                "Chie Nakamura",
                "Suzanne Flynn",
                "Boris Katz"
            ],
            "abstract": "A fundamental question in language learning concerns the role of a speaker's\nfirst language in second language acquisition. We present a novel methodology\nfor studying this question: analysis of eye-movement patterns in second\nlanguage reading of free-form text. Using this methodology, we demonstrate for\nthe first time that the native language of English learners can be predicted\nfrom their gaze fixations when reading English. We provide analysis of\nclassifier uncertainty and learned features, which indicates that differences\nin English reading are likely to be rooted in linguistic divergences across\nnative languages. The presented framework complements production studies and\noffers new ground for advancing research on multilingualism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07415v1",
            "date": "2017-04-24 18:49:38+00:00",
            "title": "Ruminating Reader: Reasoning with Gated Multi-Hop Attention",
            "authors": [
                "Yichen Gong",
                "Samuel R. Bowman"
            ],
            "abstract": "To answer the question in machine comprehension (MC) task, the models need to\nestablish the interaction between the question and the context. To tackle the\nproblem that the single-pass model cannot reflect on and correct its answer, we\npresent Ruminating Reader. Ruminating Reader adds a second pass of attention\nand a novel information fusion component to the Bi-Directional Attention Flow\nmodel (BiDAF). We propose novel layer structures that construct an query-aware\ncontext vector representation and fuse encoding representation with\nintermediate representation on top of BiDAF model. We show that a multi-hop\nattention mechanism can be applied to a bi-directional attention structure. In\nexperiments on SQuAD, we find that the Reader outperforms the BiDAF baseline by\na substantial margin, and matches or surpasses the performance of all other\npublished systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07427v1",
            "date": "2017-04-24 19:28:52+00:00",
            "title": "Recognizing Descriptive Wikipedia Categories for Historical Figures",
            "authors": [
                "Yanqing Chen",
                "Steven Skiena"
            ],
            "abstract": "Wikipedia is a useful knowledge source that benefits many applications in\nlanguage processing and knowledge representation. An important feature of\nWikipedia is that of categories. Wikipedia pages are assigned different\ncategories according to their contents as human-annotated labels which can be\nused in information retrieval, ad hoc search improvements, entity ranking and\ntag recommendations. However, important pages are usually assigned too many\ncategories, which makes it difficult to recognize the most important ones that\ngive the best descriptions.\n  In this paper, we propose an approach to recognize the most descriptive\nWikipedia categories. We observe that historical figures in a precise category\npresumably are mutually similar and such categorical coherence could be\nevaluated via texts or Wikipedia links of corresponding members in the\ncategory. We rank descriptive level of Wikipedia categories according to their\ncoherence and our ranking yield an overall agreement of 88.27% compared with\nhuman wisdom.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07431v5",
            "date": "2017-04-24 19:34:38+00:00",
            "title": "A Challenge Set Approach to Evaluating Machine Translation",
            "authors": [
                "Pierre Isabelle",
                "Colin Cherry",
                "George Foster"
            ],
            "abstract": "Neural machine translation represents an exciting leap forward in translation\nquality. But what longstanding weaknesses does it resolve, and which remain? We\naddress these questions with a challenge set approach to translation evaluation\nand error analysis. A challenge set consists of a small set of sentences, each\nhand-designed to probe a system's capacity to bridge a particular structural\ndivergence between languages. To exemplify this approach, we present an\nEnglish-French challenge set, and use it to analyze phrase-based and neural\nsystems. The resulting analysis provides not only a more fine-grained picture\nof the strengths of neural systems, but also insight into which linguistic\nphenomena remain out of reach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07441v1",
            "date": "2017-04-24 20:04:25+00:00",
            "title": "Detecting English Writing Styles For Non Native Speakers",
            "authors": [
                "Yanging Chen",
                "Rami Al-Rfou'",
                "Yejin Choi"
            ],
            "abstract": "This paper presents the first attempt, up to our knowledge, to classify\nEnglish writing styles on this scale with the challenge of classifying day to\nday language written by writers with different backgrounds covering various\nareas of topics.The paper proposes simple machine learning algorithms and\nsimple to generate features to solve hard problems. Relying on the scale of the\ndata available from large sources of knowledge like Wikipedia. We believe such\nsources of data are crucial to generate robust solutions for the web with high\naccuracy and easy to deploy in practice. The paper achieves 74\\% accuracy\nclassifying native versus non native speakers writing styles.\n  Moreover, the paper shows some interesting observations on the similarity\nbetween different languages measured by the similarity of their users English\nwriting styles. This technique could be used to show some well known facts\nabout languages as in grouping them into families, which our experiments\nsupport.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07463v1",
            "date": "2017-04-24 20:55:33+00:00",
            "title": "Streaming Word Embeddings with the Space-Saving Algorithm",
            "authors": [
                "Chandler May",
                "Kevin Duh",
                "Benjamin Van Durme",
                "Ashwin Lall"
            ],
            "abstract": "We develop a streaming (one-pass, bounded-memory) word embedding algorithm\nbased on the canonical skip-gram with negative sampling algorithm implemented\nin word2vec. We compare our streaming algorithm to word2vec empirically by\nmeasuring the cosine similarity between word pairs under each algorithm and by\napplying each algorithm in the downstream task of hashtag prediction on a\ntwo-month interval of the Twitter sample stream. We then discuss the results of\nthese experiments, concluding they provide partial validation of our approach\nas a streaming replacement for word2vec. Finally, we discuss potential failure\nmodes and suggest directions for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07556v1",
            "date": "2017-04-25 06:42:24+00:00",
            "title": "Adversarial Multi-Criteria Learning for Chinese Word Segmentation",
            "authors": [
                "Xinchi Chen",
                "Zhan Shi",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Different linguistic perspectives causes many diverse segmentation criteria\nfor Chinese word segmentation (CWS). Most existing methods focus on improve the\nperformance for each single criterion. However, it is interesting to exploit\nthese different criteria and mining their common underlying knowledge. In this\npaper, we propose adversarial multi-criteria learning for CWS by integrating\nshared knowledge from multiple heterogeneous segmentation criteria. Experiments\non eight corpora with heterogeneous segmentation criteria show that the\nperformance of each corpus obtains a significant improvement, compared to\nsingle-criterion learning. Source codes of this paper are available on Github.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07616v1",
            "date": "2017-04-25 10:22:57+00:00",
            "title": "Joint POS Tagging and Dependency Parsing with Transition-based Neural Networks",
            "authors": [
                "Liner Yang",
                "Meishan Zhang",
                "Yang Liu",
                "Nan Yu",
                "Maosong Sun",
                "Guohong Fu"
            ],
            "abstract": "While part-of-speech (POS) tagging and dependency parsing are observed to be\nclosely related, existing work on joint modeling with manually crafted feature\ntemplates suffers from the feature sparsity and incompleteness problems. In\nthis paper, we propose an approach to joint POS tagging and dependency parsing\nusing transition-based neural networks. Three neural network based classifiers\nare designed to resolve shift/reduce, tagging, and labeling conflicts.\nExperiments show that our approach significantly outperforms previous methods\nfor joint POS tagging and dependency parsing across a variety of natural\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07875v1",
            "date": "2017-04-25 19:26:36+00:00",
            "title": "Automatic Compositor Attribution in the First Folio of Shakespeare",
            "authors": [
                "Maria Ryskina",
                "Hannah Alpert-Abrams",
                "Dan Garrette",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Compositor attribution, the clustering of pages in a historical printed\ndocument by the individual who set the type, is a bibliographic task that\nrelies on analysis of orthographic variation and inspection of visual details\nof the printed page. In this paper, we introduce a novel unsupervised model\nthat jointly describes the textual and visual features needed to distinguish\ncompositors. Applied to images of Shakespeare's First Folio, our model predicts\nattributions that agree with the manual judgements of bibliographers with an\naccuracy of 87%, even on text that is the output of OCR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07986v1",
            "date": "2017-04-26 07:04:46+00:00",
            "title": "Other Topics You May Also Agree or Disagree: Modeling Inter-Topic Preferences using Tweets and Matrix Factorization",
            "authors": [
                "Akira Sasaki",
                "Kazuaki Hanawa",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "We present in this paper our approach for modeling inter-topic preferences of\nTwitter users: for example, those who agree with the Trans-Pacific Partnership\n(TPP) also agree with free trade. This kind of knowledge is useful not only for\nstance detection across multiple topics but also for various real-world\napplications including public opinion surveys, electoral predictions, electoral\ncampaigns, and online debates. In order to extract users' preferences on\nTwitter, we design linguistic patterns in which people agree and disagree about\nspecific topics (e.g., \"A is completely wrong\"). By applying these linguistic\npatterns to a collection of tweets, we extract statements agreeing and\ndisagreeing with various topics. Inspired by previous work on item\nrecommendation, we formalize the task of modeling inter-topic preferences as\nmatrix factorization: representing users' preferences as a user-topic matrix\nand mapping both users and topics onto a latent feature space that abstracts\nthe preferences. Our experimental results demonstrate both that our proposed\napproach is useful in predicting missing preferences of users and that the\nlatent vector representations of topics successfully encode inter-topic\npreferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08012v2",
            "date": "2017-04-26 08:33:14+00:00",
            "title": "Topically Driven Neural Language Model",
            "authors": [
                "Jey Han Lau",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Language models are typically applied at the sentence level, without access\nto the broader document context. We present a neural language model that\nincorporates document context in the form of a topic model-like architecture,\nthus providing a succinct representation of the broader document context\noutside of the current sentence. Experiments over a range of datasets\ndemonstrate that our model outperforms a pure sentence-based model in terms of\nlanguage model perplexity, and leads to topics that are potentially more\ncoherent than those produced by a standard LDA topic model. Our model also has\nthe ability to generate related sentences for a topic, providing another way to\ninterpret topics.",
            "journal ref": "In Proceedings of the 55th Annual Meeting of the Association for\n  Computational Linguistics (ACL 2017), pp. 355--365",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08059v1",
            "date": "2017-04-26 11:17:51+00:00",
            "title": "Riemannian Optimization for Skip-Gram Negative Sampling",
            "authors": [
                "Alexander Fonarev",
                "Oleksii Hrinchuk",
                "Gleb Gusev",
                "Pavel Serdyukov",
                "Ivan Oseledets"
            ],
            "abstract": "Skip-Gram Negative Sampling (SGNS) word embedding model, well known by its\nimplementation in \"word2vec\" software, is usually optimized by stochastic\ngradient descent. However, the optimization of SGNS objective can be viewed as\na problem of searching for a good matrix with the low-rank constraint. The most\nstandard way to solve this type of problems is to apply Riemannian optimization\nframework to optimize the SGNS objective over the manifold of required low-rank\nmatrices. In this paper, we propose an algorithm that optimizes SGNS objective\nusing Riemannian optimization and demonstrates its superiority over popular\ncompetitors, such as the original method to train SGNS and SVD over SPPMI\nmatrix.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08088v1",
            "date": "2017-04-26 13:06:25+00:00",
            "title": "Enriching Complex Networks with Word Embeddings for Detecting Mild Cognitive Impairment from Speech Transcripts",
            "authors": [
                "Leandro B. dos Santos",
                "Edilson A. Corr\u00eaa Jr",
                "Osvaldo N. Oliveira Jr",
                "Diego R. Amancio",
                "Let\u00edcia L. Mansur",
                "Sandra M. Alu\u00edsio"
            ],
            "abstract": "Mild Cognitive Impairment (MCI) is a mental disorder difficult to diagnose.\nLinguistic features, mainly from parsers, have been used to detect MCI, but\nthis is not suitable for large-scale assessments. MCI disfluencies produce\nnon-grammatical speech that requires manual or high precision automatic\ncorrection of transcripts. In this paper, we modeled transcripts into complex\nnetworks and enriched them with word embedding (CNE) to better represent short\ntexts produced in neuropsychological assessments. The network measurements were\napplied with well-known classifiers to automatically identify MCI in\ntranscripts, in a binary classification task. A comparison was made with the\nperformance of traditional approaches using Bag of Words (BoW) and linguistic\nfeatures for three datasets: DementiaBank in English, and Cinderella and\nArizona-Battery in Portuguese. Overall, CNE provided higher accuracy than using\nonly complex networks, while Support Vector Machine was superior to other\nclassifiers. CNE provided the highest accuracies for DementiaBank and\nCinderella, but BoW was more efficient for the Arizona-Battery dataset probably\nowing to its short narratives. The approach using linguistic features yielded\nhigher accuracy if the transcriptions of the Cinderella dataset were manually\nrevised. Taken together, the results indicate that complex networks enriched\nwith embedding is promising for detecting MCI in large-scale assessments",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08300v2",
            "date": "2017-04-26 19:06:37+00:00",
            "title": "Diversity driven Attention Model for Query-based Abstractive Summarization",
            "authors": [
                "Preksha Nema",
                "Mitesh Khapra",
                "Anirban Laha",
                "Balaraman Ravindran"
            ],
            "abstract": "Abstractive summarization aims to generate a shorter version of the document\ncovering all the salient points in a compact and coherent fashion. On the other\nhand, query-based summarization highlights those points that are relevant in\nthe context of a given query. The encode-attend-decode paradigm has achieved\nnotable success in machine translation, extractive summarization, dialog\nsystems, etc. But it suffers from the drawback of generation of repeated\nphrases. In this work we propose a model for the query-based summarization task\nbased on the encode-attend-decode paradigm with two key additions (i) a query\nattention model (in addition to document attention model) which learns to focus\non different portions of the query at different time steps (instead of using a\nstatic representation for the query) and (ii) a new diversity based attention\nmodel which aims to alleviate the problem of repeating phrases in the summary.\nIn order to enable the testing of this model we introduce a new query-based\nsummarization dataset building on debatepedia. Our experiments show that with\nthese two additions the proposed model clearly outperforms vanilla\nencode-attend-decode models with a gain of 28% (absolute) in ROUGE-L scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08352v1",
            "date": "2017-04-26 21:10:53+00:00",
            "title": "From Characters to Words to in Between: Do We Capture Morphology?",
            "authors": [
                "Clara Vania",
                "Adam Lopez"
            ],
            "abstract": "Words can be represented by composing the representations of subword units\nsuch as word segments, characters, and/or character n-grams. While such\nrepresentations are effective and may capture the morphological regularities of\nwords, they have not been systematically compared, and it is not understood how\nthey interact with different morphological typologies. On a language modeling\ntask, we present experiments that systematically vary (1) the basic unit of\nrepresentation, (2) the composition of these representations, and (3) the\nmorphological typology of the language modeled. Our results extend previous\nfindings that character representations are effective across typologies, and we\nfind that a previously unstudied combination of character trigram\nrepresentations composed with bi-LSTMs outperforms most others. But we also\nfind room for improvement: none of the character-level models match the\npredictive accuracy of a model with access to true morphological analyses, even\nwhen learned from an order of magnitude more data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08381v3",
            "date": "2017-04-26 23:53:34+00:00",
            "title": "Neural AMR: Sequence-to-Sequence Models for Parsing and Generation",
            "authors": [
                "Ioannis Konstas",
                "Srinivasan Iyer",
                "Mark Yatskar",
                "Yejin Choi",
                "Luke Zettlemoyer"
            ],
            "abstract": "Sequence-to-sequence models have shown strong performance across a broad\nrange of applications. However, their application to parsing and generating\ntext usingAbstract Meaning Representation (AMR)has been limited, due to the\nrelatively limited amount of labeled data and the non-sequential nature of the\nAMR graphs. We present a novel training procedure that can lift this limitation\nusing millions of unlabeled sentences and careful preprocessing of the AMR\ngraphs. For AMR parsing, our model achieves competitive results of 62.1SMATCH,\nthe current best score reported without significant use of external semantic\nresources. For AMR generation, our model establishes a new state-of-the-art\nperformance of BLEU 33.8. We present extensive ablative and qualitative\nanalysis including strong evidence that sequence-based AMR models are robust\nagainst ordering variations of graph-to-sequence conversions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08384v1",
            "date": "2017-04-27 00:03:02+00:00",
            "title": "Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks",
            "authors": [
                "Rajarshi Das",
                "Manzil Zaheer",
                "Siva Reddy",
                "Andrew McCallum"
            ],
            "abstract": "Existing question answering methods infer answers either from a knowledge\nbase or from raw text. While knowledge base (KB) methods are good at answering\ncompositional questions, their performance is often affected by the\nincompleteness of the KB. Au contraire, web text contains millions of facts\nthat are absent in the KB, however in an unstructured form. {\\it Universal\nschema} can support reasoning on the union of both structured KBs and\nunstructured text by aligning them in a common embedded space. In this paper we\nextend universal schema to natural language question answering, employing\n\\emph{memory networks} to attend to the large body of facts in the combination\nof text and KB. Our models can be trained in an end-to-end fashion on\nquestion-answer pairs. Evaluation results on \\spades fill-in-the-blank question\nanswering dataset show that exploiting universal schema for question answering\nis better than using either a KB or text alone. This model also outperforms the\ncurrent state-of-the-art by 8.5 $F_1$ points.\\footnote{Code and data available\nin \\url{https://rajarshd.github.io/TextKBQA}}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08387v3",
            "date": "2017-04-27 00:24:20+00:00",
            "title": "Learning Structured Natural Language Representations for Semantic Parsing",
            "authors": [
                "Jianpeng Cheng",
                "Siva Reddy",
                "Vijay Saraswat",
                "Mirella Lapata"
            ],
            "abstract": "We introduce a neural semantic parser that converts natural language\nutterances to intermediate representations in the form of predicate-argument\nstructures, which are induced with a transition system and subsequently mapped\nto target domains. The semantic parser is trained end-to-end using annotated\nlogical forms or their denotations. We obtain competitive results on various\ndatasets. The induced predicate-argument structures shed light on the types of\nrepresentations useful for semantic parsing and how these are different from\nlinguistically motivated ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08388v2",
            "date": "2017-04-27 00:29:17+00:00",
            "title": "Duluth at Semeval-2017 Task 7 : Puns upon a midnight dreary, Lexical Semantics for the weak and weary",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth systems that participated in SemEval-2017\nTask 7 : Detection and Interpretation of English Puns. The Duluth systems\nparticipated in all three subtasks, and relied on methods that included word\nsense disambiguation and measures of semantic relatedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08390v1",
            "date": "2017-04-27 00:40:33+00:00",
            "title": "Duluth at SemEval-2017 Task 6: Language Models in Humor Detection",
            "authors": [
                "Xinru Yan",
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth system that participated in SemEval-2017 Task\n6 #HashtagWars: Learning a Sense of Humor. The system participated in Subtasks\nA and B using N-gram language models, ranking highly in the task evaluation.\nThis paper discusses the results of our system in the development and\nevaluation stages and from two post-evaluation runs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08430v2",
            "date": "2017-04-27 04:25:41+00:00",
            "title": "A GRU-Gated Attention Model for Neural Machine Translation",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su"
            ],
            "abstract": "Neural machine translation (NMT) heavily relies on an attention network to\nproduce a context vector for each target word prediction. In practice, we find\nthat context vectors for different target words are quite similar to one\nanother and therefore are insufficient in discriminatively predicting target\nwords. The reason for this might be that context vectors produced by the\nvanilla attention network are just a weighted sum of source representations\nthat are invariant to decoder states. In this paper, we propose a novel\nGRU-gated attention model (GAtt) for NMT which enhances the degree of\ndiscrimination of context vectors by enabling source representations to be\nsensitive to the partial translation generated by the decoder. GAtt uses a\ngated recurrent unit (GRU) to combine two types of information: treating a\nsource annotation vector originally produced by the bidirectional encoder as\nthe history state while the corresponding previous decoder state as the input\nto the GRU. The GRU-combined information forms a new source annotation vector.\nIn this way, we can obtain translation-sensitive source representations which\nare then feed into the attention network to generate discriminative context\nvectors. We further propose a variant that regards a source annotation vector\nas the current input while the previous decoder state as the history.\nExperiments on NIST Chinese-English translation tasks show that both GAtt-based\nmodels achieve significant improvements over the vanilla attentionbased NMT.\nFurther analyses on attention weights and context vectors demonstrate the\neffectiveness of GAtt in improving the discrimination power of representations\nand handling the challenging issue of over-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08531v1",
            "date": "2017-04-27 12:27:25+00:00",
            "title": "A Survey of Neural Network Techniques for Feature Extraction from Text",
            "authors": [
                "Vineet John"
            ],
            "abstract": "This paper aims to catalyze the discussions about text feature extraction\ntechniques using neural network architectures. The research questions discussed\nin the paper focus on the state-of-the-art neural network techniques that have\nproven to be useful tools for language processing, language generation, text\nclassification and other computational linguistics tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08760v1",
            "date": "2017-04-27 22:05:06+00:00",
            "title": "Learning a Neural Semantic Parser from User Feedback",
            "authors": [
                "Srinivasan Iyer",
                "Ioannis Konstas",
                "Alvin Cheung",
                "Jayant Krishnamurthy",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present an approach to rapidly and easily build natural language\ninterfaces to databases for new domains, whose performance improves over time\nbased on user feedback, and requires minimal intervention. To achieve this, we\nadapt neural sequence models to map utterances directly to SQL with its full\nexpressivity, bypassing any intermediate meaning representations. These models\nare immediately deployed online to solicit feedback from real users to flag\nincorrect queries. Finally, the popularity of SQL facilitates gathering\nannotations for incorrect predictions using the crowd, which is directly used\nto improve our models. This complete feedback loop, without intermediate\nrepresentations or database specific engineering, opens up new ways of building\nhigh quality semantic parsers. Experiments suggest that this approach can be\ndeployed quickly for any new target domain, as we show by learning a semantic\nparser for an online academic database from scratch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08795v2",
            "date": "2017-04-28 03:12:57+00:00",
            "title": "Mapping Instructions and Visual Observations to Actions with Reinforcement Learning",
            "authors": [
                "Dipendra Misra",
                "John Langford",
                "Yoav Artzi"
            ],
            "abstract": "We propose to directly map raw visual observations and text input to actions\nfor instruction execution. While existing approaches assume access to\nstructured environment representations or use a pipeline of separately trained\nmodels, we learn a single model to jointly reason about linguistic and visual\ninput. We use reinforcement learning in a contextual bandit setting to train a\nneural network agent. To guide the agent's exploration, we use reward shaping\nwith different forms of supervision. Our approach does not require intermediate\nrepresentations, planning procedures, or training different models. We evaluate\nin a simulated environment, and show significant improvements over supervised\nlearning and common reinforcement learning variants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08798v1",
            "date": "2017-04-28 03:33:16+00:00",
            "title": "Word Affect Intensities",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "Words often convey affect -- emotions, feelings, and attitudes. Lexicons of\nword-affect association have applications in automatic emotion analysis and\nnatural language generation. However, existing lexicons indicate only coarse\ncategories of affect association. Here, for the first time, we create an affect\nintensity lexicon with real-valued scores of association. We use a technique\ncalled best-worst scaling that improves annotation consistency and obtains\nreliable fine-grained scores. The lexicon includes terms common from both\ngeneral English and terms specific to social media communications. It has close\nto 6,000 entries for four basic emotions. We will be adding entries for other\naffect dimensions shortly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08893v2",
            "date": "2017-04-28 12:09:31+00:00",
            "title": "How compatible are our discourse annotations? Insights from mapping RST-DT and PDTB annotations",
            "authors": [
                "Vera Demberg",
                "Fatemeh Torabi Asr",
                "Merel Scholman"
            ],
            "abstract": "Discourse-annotated corpora are an important resource for the community, but\nthey are often annotated according to different frameworks. This makes\ncomparison of the annotations difficult, thereby also preventing researchers\nfrom searching the corpora in a unified way, or using all annotated data\njointly to train computational systems. Several theoretical proposals have\nrecently been made for mapping the relational labels of different frameworks to\neach other, but these proposals have so far not been validated against existing\nannotations. The two largest discourse relation annotated resources, the Penn\nDiscourse Treebank and the Rhetorical Structure Theory Discourse Treebank, have\nhowever been annotated on the same text, allowing for a direct comparison of\nthe annotation layers. We propose a method for automatically aligning the\ndiscourse segments, and then evaluate existing mapping proposals by comparing\nthe empirically observed against the proposed mappings. Our analysis highlights\nthe influence of segmentation on subsequent discourse relation labeling, and\nshows that while agreement between frameworks is reasonable for explicit\nrelations, agreement on implicit relations is low. We identify several sources\nof systematic discrepancies between the two annotation schemes and discuss\nconsequences of these discrepancies for future annotation and for the training\nof automatic discourse relation labellers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08960v1",
            "date": "2017-04-28 14:46:25+00:00",
            "title": "Neural Word Segmentation with Rich Pretraining",
            "authors": [
                "Jie Yang",
                "Yue Zhang",
                "Fei Dong"
            ],
            "abstract": "Neural word segmentation research has benefited from large-scale raw texts by\nleveraging them for pretraining character and word embeddings. On the other\nhand, statistical segmentation research has exploited richer sources of\nexternal information, such as punctuation, automatic segmentation and POS. We\ninvestigate the effectiveness of a range of external training sources for\nneural word segmentation by building a modular segmentation model, pretraining\nthe most important submodule using rich external sources. Results show that\nsuch pretraining significantly improves the model, leading to accuracies\ncompetitive to the best methods on six benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00134v4",
            "date": "2017-06-01 00:37:46+00:00",
            "title": "Semantic Refinement GRU-based Neural Language Generation for Spoken Dialogue Systems",
            "authors": [
                "Van-Khanh Tran",
                "Le-Minh Nguyen"
            ],
            "abstract": "Natural language generation (NLG) plays a critical role in spoken dialogue\nsystems. This paper presents a new approach to NLG by using recurrent neural\nnetworks (RNN), in which a gating mechanism is applied before RNN computation.\nThis allows the proposed model to generate appropriate sentences. The RNN-based\ngenerator can be learned from unaligned data by jointly training sentence\nplanning and surface realization to produce natural language responses. The\nmodel was extensively evaluated on four different NLG domains. The results show\nthat the proposed generator achieved better performance on all the NLG domains\ncompared to previous generators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00139v3",
            "date": "2017-06-01 01:06:17+00:00",
            "title": "Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks",
            "authors": [
                "Van-Khanh Tran",
                "Le-Minh Nguyen"
            ],
            "abstract": "Natural language generation (NLG) is a critical component in a spoken\ndialogue system. This paper presents a Recurrent Neural Network based\nEncoder-Decoder architecture, in which an LSTM-based decoder is introduced to\nselect, aggregate semantic elements produced by an attention mechanism over the\ninput elements, and to produce the required utterances. The proposed generator\ncan be jointly trained both sentence planning and surface realization to\nproduce natural language sentences. The proposed model was extensively\nevaluated on four different NLG datasets. The experimental results showed that\nthe proposed generators not only consistently outperform the previous methods\nacross all the NLG domains but also show an ability to generalize from a new,\nunseen domain and learn from multi-domain datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00245v1",
            "date": "2017-06-01 10:27:07+00:00",
            "title": "Polish Read Speech Corpus for Speech Tools and Services",
            "authors": [
                "Danijel Kor\u017einek",
                "Krzysztof Marasek",
                "\u0141ukasz Brocki",
                "Krzysztof Wo\u0142k"
            ],
            "abstract": "This paper describes the speech processing activities conducted at the Polish\nconsortium of the CLARIN project. The purpose of this segment of the project\nwas to develop specific tools that would allow for automatic and semi-automatic\nprocessing of large quantities of acoustic speech data. The tools include the\nfollowing: grapheme-to-phoneme conversion, speech-to-text alignment, voice\nactivity detection, speaker diarization, keyword spotting and automatic speech\ntranscription. Furthermore, in order to develop these tools, a large\nhigh-quality studio speech corpus was recorded and released under an open\nlicense, to encourage development in the area of Polish speech research.\nAnother purpose of the corpus was to serve as a reference for studies in\nphonetics and pronunciation. All the tools and resources were released on the\nthe Polish CLARIN website. This paper discusses the current status and future\nplans for the project.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00321v1",
            "date": "2017-06-01 14:30:19+00:00",
            "title": "Using of heterogeneous corpora for training of an ASR system",
            "authors": [
                "Jan Trmal",
                "Gaurav Kumar",
                "Vimal Manohar",
                "Sanjeev Khudanpur",
                "Matt Post",
                "Paul McNamee"
            ],
            "abstract": "The paper summarizes the development of the LVCSR system built as a part of\nthe Pashto speech-translation system at the SCALE (Summer Camp for Applied\nLanguage Exploration) 2015 workshop on \"Speech-to-text-translation for\nlow-resource languages\". The Pashto language was chosen as a good \"proxy\"\nlow-resource language, exhibiting multiple phenomena which make the\nspeech-recognition and and speech-to-text-translation systems development hard.\n  Even when the amount of data is seemingly sufficient, given the fact that the\ndata originates from multiple sources, the preliminary experiments reveal that\nthere is little to no benefit in merging (concatenating) the corpora and more\nelaborate ways of making use of all of the data must be worked out.\n  This paper concentrates only on the LVCSR part and presents a range of\ndifferent techniques that were found to be useful in order to benefit from\nmultiple different corpora",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00377v1",
            "date": "2017-06-01 16:31:20+00:00",
            "title": "Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules",
            "authors": [
                "Ivan Vuli\u0107",
                "Nikola Mrk\u0161i\u0107",
                "Roi Reichart",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Steve Young",
                "Anna Korhonen"
            ],
            "abstract": "Morphologically rich languages accentuate two properties of distributional\nvector space models: 1) the difficulty of inducing accurate representations for\nlow-frequency word forms; and 2) insensitivity to distinct lexical relations\nthat have similar distributional signatures. These effects are detrimental for\nlanguage understanding systems, which may infer that 'inexpensive' is a\nrephrasing for 'expensive' or may not associate 'acquire' with 'acquires'. In\nthis work, we propose a novel morph-fitting procedure which moves past the use\nof curated semantic lexicons for improving distributional vector spaces.\nInstead, our method injects morphological constraints generated using simple\nlanguage-specific rules, pulling inflectional forms of the same word close\ntogether and pushing derivational antonyms far apart. In intrinsic evaluation\nover four languages, we show that our approach: 1) improves low-frequency word\nestimates; and 2) boosts the semantic quality of the entire word vector\ncollection. Finally, we show that morph-fitted vectors yield large gains in the\ndownstream task of dialogue state tracking, highlighting the importance of\nmorphology for tackling long-tail phenomena in language understanding tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00457v1",
            "date": "2017-06-01 18:57:39+00:00",
            "title": "NMTPY: A Flexible Toolkit for Advanced Neural Machine Translation Systems",
            "authors": [
                "Ozan Caglayan",
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Adrien Bardet",
                "Walid Aransa",
                "Fethi Bougares",
                "Lo\u00efc Barrault"
            ],
            "abstract": "In this paper, we present nmtpy, a flexible Python toolkit based on Theano\nfor training Neural Machine Translation and other neural sequence-to-sequence\narchitectures. nmtpy decouples the specification of a network from the training\nand inference utilities to simplify the addition of a new architecture and\nreduce the amount of boilerplate code to be written. nmtpy has been used for\nLIUM's top-ranked submissions to WMT Multimodal Machine Translation and News\nTranslation tasks in 2016 and 2017.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00465v1",
            "date": "2017-06-01 19:20:50+00:00",
            "title": "Machine Assisted Analysis of Vowel Length Contrasts in Wolof",
            "authors": [
                "Elodie Gauthier",
                "Laurent Besacier",
                "Sylvie Voisin"
            ],
            "abstract": "Growing digital archives and improving algorithms for automatic analysis of\ntext and speech create new research opportunities for fundamental research in\nphonetics. Such empirical approaches allow statistical evaluation of a much\nlarger set of hypothesis about phonetic variation and its conditioning factors\n(among them geographical / dialectal variants). This paper illustrates this\nvision and proposes to challenge automatic methods for the analysis of a not\neasily observable phenomenon: vowel length contrast. We focus on Wolof, an\nunder-resourced language from Sub-Saharan Africa. In particular, we propose\nmultiple features to make a fine evaluation of the degree of length contrast\nunder different factors such as: read vs semi spontaneous speech ; standard vs\ndialectal Wolof. Our measures made fully automatically on more than 20k vowel\ntokens show that our proposed features can highlight different degrees of\ncontrast for each vowel considered. We notably show that contrast is weaker in\nsemi-spontaneous speech and in a non standard semi-spontaneous dialect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00468v2",
            "date": "2017-06-01 19:26:32+00:00",
            "title": "Function Assistant: A Tool for NL Querying of APIs",
            "authors": [
                "Kyle Richardson",
                "Jonas Kuhn"
            ],
            "abstract": "In this paper, we describe Function Assistant, a lightweight Python-based\ntoolkit for querying and exploring source code repositories using natural\nlanguage. The toolkit is designed to help end-users of a target API quickly\nfind information about functions through high-level natural language queries\nand descriptions. For a given text query and background API, the tool finds\ncandidate functions by performing a translation from the text to known\nrepresentations in the API using the semantic parsing approach of Richardson\nand Kuhn (2017). Translations are automatically learned from example text-code\npairs in example APIs. The toolkit includes features for building translation\npipelines and query engines for arbitrary source code projects. To explore this\nlast feature, we perform new experiments on 27 well-known Python projects\nhosted on Github.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00506v1",
            "date": "2017-06-01 21:59:47+00:00",
            "title": "Morphological Embeddings for Named Entity Recognition in Morphologically Rich Languages",
            "authors": [
                "Onur Gungor",
                "Eray Yildiz",
                "Suzan Uskudarli",
                "Tunga Gungor"
            ],
            "abstract": "In this work, we present new state-of-the-art results of 93.59,% and 79.59,%\nfor Turkish and Czech named entity recognition based on the model of (Lample et\nal., 2016). We contribute by proposing several schemes for representing the\nmorphological analysis of a word in the context of named entity recognition. We\nshow that a concatenation of this representation with the word and character\nembeddings improves the performance. The effect of these representation schemes\non the tagging performance is also investigated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00612v1",
            "date": "2017-06-02 10:12:52+00:00",
            "title": "Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech",
            "authors": [
                "Michael Neumann",
                "Ngoc Thang Vu"
            ],
            "abstract": "Speech emotion recognition is an important and challenging task in the realm\nof human-computer interaction. Prior work proposed a variety of models and\nfeature sets for training a system. In this work, we conduct extensive\nexperiments using an attentive convolutional neural network with multi-view\nlearning objective function. We compare system performance using different\nlengths of the input signal, different types of acoustic features and different\ntypes of emotion speech (improvised/scripted). Our experimental results on the\nInteractive Emotional Motion Capture (IEMOCAP) database reveal that the\nrecognition performance strongly depends on the type of speech data independent\nof the choice of input features. Furthermore, we achieved state-of-the-art\nresults on the improvised speech data of IEMOCAP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00741v1",
            "date": "2017-06-02 16:20:19+00:00",
            "title": "Prosodic Event Recognition using Convolutional Neural Networks with Context Information",
            "authors": [
                "Sabrina Stehwien",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper demonstrates the potential of convolutional neural networks (CNN)\nfor detecting and classifying prosodic events on words, specifically pitch\naccents and phrase boundary tones, from frame-based acoustic features. Typical\napproaches use not only feature representations of the word in question but\nalso its surrounding context. We show that adding position features indicating\nthe current word benefits the CNN. In addition, this paper discusses the\ngeneralization from a speaker-dependent modelling approach to a\nspeaker-independent setup. The proposed method is simple and efficient and\nyields strong results not only in speaker-dependent but also\nspeaker-independent cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00927v3",
            "date": "2017-06-03 10:46:50+00:00",
            "title": "Concept Transfer Learning for Adaptive Language Understanding",
            "authors": [
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "Concept definition is important in language understanding (LU) adaptation\nsince literal definition difference can easily lead to data sparsity even if\ndifferent data sets are actually semantically correlated. To address this\nissue, in this paper, a novel concept transfer learning approach is proposed.\nHere, substructures within literal concept definition are investigated to\nreveal the relationship between concepts. A hierarchical semantic\nrepresentation for concepts is proposed, where a semantic slot is represented\nas a composition of {\\em atomic concepts}. Based on this new hierarchical\nrepresentation, transfer learning approaches are developed for adaptive LU. The\napproaches are applied to two tasks: value set mismatch and domain adaptation,\nand evaluated on two LU benchmarks: ATIS and DSTC 2\\&3. Thorough empirical\nstudies validate both the efficiency and effectiveness of the proposed method.\nIn particular, we achieve state-of-the-art performance ($F_1$-score 96.08\\%) on\nATIS by only using lexicon features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01069v1",
            "date": "2017-06-04 13:12:45+00:00",
            "title": "CRNN: A Joint Neural Network for Redundancy Detection",
            "authors": [
                "Xinyu Fu",
                "Eugene Ch'ng",
                "Uwe Aickelin",
                "Simon See"
            ],
            "abstract": "This paper proposes a novel framework for detecting redundancy in supervised\nsentence categorisation. Unlike traditional singleton neural network, our model\nincorporates character-aware convolutional neural network (Char-CNN) with\ncharacter-aware recurrent neural network (Char-RNN) to form a convolutional\nrecurrent neural network (CRNN). Our model benefits from Char-CNN in that only\nsalient features are selected and fed into the integrated Char-RNN. Char-RNN\neffectively learns long sequence semantics via sophisticated update mechanism.\nWe compare our framework against the state-of-the-art text classification\nalgorithms on four popular benchmarking corpus. For instance, our model\nachieves competing precision rate, recall ratio, and F1 score on the\nGoogle-news data-set. For twenty-news-groups data stream, our algorithm obtains\nthe optimum on precision rate, recall ratio, and F1 score. For Brown Corpus,\nour framework obtains the best F1 score and almost equivalent precision rate\nand recall ratio over the top competitor. For the question classification\ncollection, CRNN produces the optimal recall rate and F1 score and comparable\nprecision rate. We also analyse three different RNN hidden recurrent cells'\nimpact on performance and their runtime efficiency. We observe that MGU\nachieves the optimal runtime and comparable performance against GRU and LSTM.\nFor TFIDF based algorithms, we experiment with word2vec, GloVe, and sent2vec\nembeddings and report their performance differences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01206v1",
            "date": "2017-06-05 06:20:23+00:00",
            "title": "One-step and Two-step Classification for Abusive Language Detection on Twitter",
            "authors": [
                "Ji Ho Park",
                "Pascale Fung"
            ],
            "abstract": "Automatic abusive language detection is a difficult but important task for\nonline social media. Our research explores a two-step approach of performing\nclassification on abusive language and then classifying into specific types and\ncompares it with one-step approach of doing one multi-class classification for\ndetecting sexist and racist languages. With a public English Twitter corpus of\n20 thousand tweets in the type of sexism and racism, our approach shows a\npromising performance of 0.827 F-measure by using HybridCNN in one-step and\n0.824 F-measure by using logistic regression in two-steps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01399v3",
            "date": "2017-06-05 16:10:58+00:00",
            "title": "Language Generation with Recurrent Generative Adversarial Networks without Pre-training",
            "authors": [
                "Ofir Press",
                "Amir Bar",
                "Ben Bogin",
                "Jonathan Berant",
                "Lior Wolf"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have shown great promise recently in\nimage generation. Training GANs for language generation has proven to be more\ndifficult, because of the non-differentiable nature of generating text with\nrecurrent neural networks. Consequently, past work has either resorted to\npre-training with maximum-likelihood or used convolutional networks for\ngeneration. In this work, we show that recurrent neural networks can be trained\nto generate text with GANs from scratch using curriculum learning, by slowly\nteaching the model to generate sequences of increasing and variable length. We\nempirically show that our approach vastly improves the quality of generated\nsequences compared to a convolutional baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01570v2",
            "date": "2017-06-06 00:55:25+00:00",
            "title": "Acquisition of Translation Lexicons for Historically Unwritten Languages via Bridging Loanwords",
            "authors": [
                "Michael Bloodgood",
                "Benjamin Strauss"
            ],
            "abstract": "With the advent of informal electronic communications such as social media,\ncolloquial languages that were historically unwritten are being written for the\nfirst time in heavily code-switched environments. We present a method for\ninducing portions of translation lexicons through the use of expert knowledge\nin these settings where there are approximately zero resources available other\nthan a language informant, potentially not even large amounts of monolingual\ndata. We investigate inducing a Moroccan Darija-English translation lexicon via\nFrench loanwords bridging into English and find that a useful lexicon is\ninduced for human-assisted translation and statistical machine translation.",
            "journal ref": "In Proceedings of the 10th Workshop on Building and Using\n  Comparable Corpora, pages 21-25, Vancouver, Canada, August 2017. Association\n  for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01678v3",
            "date": "2017-06-06 10:04:45+00:00",
            "title": "Text Summarization using Abstract Meaning Representation",
            "authors": [
                "Shibhansh Dohare",
                "Harish Karnick",
                "Vivek Gupta"
            ],
            "abstract": "With an ever increasing size of text present on the Internet, automatic\nsummary generation remains an important problem for natural language\nunderstanding. In this work we explore a novel full-fledged pipeline for text\nsummarization with an intermediate step of Abstract Meaning Representation\n(AMR). The pipeline proposed by us first generates an AMR graph of an input\nstory, through which it extracts a summary graph and finally, generate summary\nsentences from this summary graph. Our proposed method achieves\nstate-of-the-art results compared to the other text summarization routines\nbased on AMR. We also point out some significant problems in the existing\nevaluation methods, which make them unsuitable for evaluating summary quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01690v1",
            "date": "2017-06-06 10:48:29+00:00",
            "title": "A Frame Tracking Model for Memory-Enhanced Dialogue Systems",
            "authors": [
                "Hannes Schulz",
                "Jeremie Zumer",
                "Layla El Asri",
                "Shikhar Sharma"
            ],
            "abstract": "Recently, resources and tasks were proposed to go beyond state tracking in\ndialogue systems. An example is the frame tracking task, which requires\nrecording multiple frames, one for each user goal set during the dialogue. This\nallows a user, for instance, to compare items corresponding to different goals.\nThis paper proposes a model which takes as input the list of frames created so\nfar during the dialogue, the current user utterance as well as the dialogue\nacts, slot types, and slot values associated with this utterance. The model\nthen outputs the frame being referenced by each triple of dialogue act, slot\ntype, and slot value. We show that on the recently published Frames dataset,\nthis model significantly outperforms a previously proposed rule-based baseline.\nIn addition, we propose an extensive analysis of the frame tracking task by\ndividing it into sub-tasks and assessing their difficulty with respect to our\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01723v1",
            "date": "2017-06-06 12:11:50+00:00",
            "title": "A General-Purpose Tagger with Convolutional Neural Networks",
            "authors": [
                "Xiang Yu",
                "Agnieszka Fale\u0144ska",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present a general-purpose tagger based on convolutional neural networks\n(CNN), used for both composing word vectors and encoding context information.\nThe CNN tagger is robust across different tagging tasks: without task-specific\ntuning of hyper-parameters, it achieves state-of-the-art results in\npart-of-speech tagging, morphological tagging and supertagging. The CNN tagger\nis also robust against the out-of-vocabulary problem, it performs well on\nartificially unnormalized texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01740v1",
            "date": "2017-06-06 13:10:49+00:00",
            "title": "Label-Dependencies Aware Recurrent Neural Networks",
            "authors": [
                "Yoann Dupont",
                "Marco Dinarelli",
                "Isabelle Tellier"
            ],
            "abstract": "In the last few years, Recurrent Neural Networks (RNNs) have proved effective\non several NLP tasks. Despite such great success, their ability to model\n\\emph{sequence labeling} is still limited. This lead research toward solutions\nwhere RNNs are combined with models which already proved effective in this\ndomain, such as CRFs. In this work we propose a solution far simpler but very\neffective: an evolution of the simple Jordan RNN, where labels are re-injected\nas input into the network, and converted into embeddings, in the same way as\nwords. We compare this RNN variant to all the other RNN models, Elman and\nJordan RNN, LSTM and GRU, on two well-known tasks of Spoken Language\nUnderstanding (SLU). Thanks to label embeddings and their combination at the\nhidden layer, the proposed variant, which uses more parameters than Elman and\nJordan RNNs, but far fewer than LSTM and GRU, is more effective than other\nRNNs, but also outperforms sophisticated CRF models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01839v1",
            "date": "2017-06-06 16:16:51+00:00",
            "title": "Assessing the Linguistic Productivity of Unsupervised Deep Neural Networks",
            "authors": [
                "Lawrence Phillips",
                "Nathan Hodas"
            ],
            "abstract": "Increasingly, cognitive scientists have demonstrated interest in applying\ntools from deep learning. One use for deep learning is in language acquisition\nwhere it is useful to know if a linguistic phenomenon can be learned through\ndomain-general means. To assess whether unsupervised deep learning is\nappropriate, we first pose a smaller question: Can unsupervised neural networks\napply linguistic rules productively, using them in novel situations? We draw\nfrom the literature on determiner/noun productivity by training an\nunsupervised, autoencoder network measuring its ability to combine nouns with\ndeterminers. Our simple autoencoder creates combinations it has not previously\nencountered and produces a degree of overlap matching adults. While this\npreliminary work does not provide conclusive evidence for productivity, it\nwarrants further investigation with more complex models. Further, this work\nhelps lay the foundations for future collaboration between the deep learning\nand cognitive science communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01847v1",
            "date": "2017-06-06 16:36:41+00:00",
            "title": "Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext",
            "authors": [
                "John Wieting",
                "Jonathan Mallinson",
                "Kevin Gimpel"
            ],
            "abstract": "We consider the problem of learning general-purpose, paraphrastic sentence\nembeddings in the setting of Wieting et al. (2016b). We use neural machine\ntranslation to generate sentential paraphrases via back-translation of\nbilingual sentence pairs. We evaluate the paraphrase pairs by their ability to\nserve as training data for learning paraphrastic sentence embeddings. We find\nthat the data quality is stronger than prior work based on bitext and on par\nwith manually-written English paraphrase pairs, with the advantage that our\napproach can scale up to generate large training sets for many languages and\ndomains. We experiment with several language pairs and data sources, and\ndevelop a variety of data filtering techniques. In the process, we explore how\nneural machine translation output differs from human-written sentences, finding\nclear differences in length, the amount of repetition, and the use of rare\nwords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01967v2",
            "date": "2017-06-06 20:45:30+00:00",
            "title": "Synergistic Union of Word2Vec and Lexicon for Domain Specific Semantic Similarity",
            "authors": [
                "Keet Sugathadasa",
                "Buddhi Ayesha",
                "Nisansa de Silva",
                "Amal Shehan Perera",
                "Vindula Jayawardana",
                "Dimuthu Lakmal",
                "Madhavi Perera"
            ],
            "abstract": "Semantic similarity measures are an important part in Natural Language\nProcessing tasks. However Semantic similarity measures built for general use do\nnot perform well within specific domains. Therefore in this study we introduce\na domain specific semantic similarity measure that was created by the\nsynergistic union of word2vec, a word embedding method that is used for\nsemantic similarity calculation and lexicon based (lexical) semantic similarity\nmethods. We prove that this proposed methodology out performs word embedding\nmethods trained on generic corpus and methods trained on domain specific corpus\nbut do not use lexical semantic similarity methods to augment the results.\nFurther, we prove that text lemmatization can improve the performance of word\nembedding methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02027v2",
            "date": "2017-06-07 02:06:58+00:00",
            "title": "Question Answering and Question Generation as Dual Tasks",
            "authors": [
                "Duyu Tang",
                "Nan Duan",
                "Tao Qin",
                "Zhao Yan",
                "Ming Zhou"
            ],
            "abstract": "We study the problem of joint question answering (QA) and question generation\n(QG) in this paper.\n  Our intuition is that QA and QG have intrinsic connections and these two\ntasks could improve each other.\n  On one side, the QA model judges whether the generated question of a QG model\nis relevant to the answer.\n  On the other side, the QG model provides the probability of generating a\nquestion given the answer, which is a useful evidence that in turn facilitates\nQA.\n  In this paper we regard QA and QG as dual tasks.\n  We propose a training framework that trains the models of QA and QG\nsimultaneously, and explicitly leverages their probabilistic correlation to\nguide the training process of both models.\n  We implement a QG model based on sequence-to-sequence learning, and a QA\nmodel based on recurrent neural network.\n  As all the components of the QA and QG models are differentiable, all the\nparameters involved in these two models could be conventionally learned with\nback propagation.\n  We conduct experiments on three datasets. Empirical results show that our\ntraining framework improves both QA and QG tasks.\n  The improved QA model performs comparably with strong baseline approaches on\nall three datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02095v2",
            "date": "2017-06-07 09:04:29+00:00",
            "title": "Macquarie University at BioASQ 5b -- Query-based Summarisation Techniques for Selecting the Ideal Answers",
            "authors": [
                "Diego Molla-Aliod"
            ],
            "abstract": "Macquarie University's contribution to the BioASQ challenge (Task 5b Phase B)\nfocused on the use of query-based extractive summarisation techniques for the\ngeneration of the ideal answers. Four runs were submitted, with approaches\nranging from a trivial system that selected the first $n$ snippets, to the use\nof deep learning approaches under a regression framework. Our experiments and\nthe ROUGE results of the five test batches of BioASQ indicate surprisingly good\nresults for the trivial approach. Overall, most of our runs on the first three\ntest batches achieved the best ROUGE-SU4 results in the challenge.",
            "journal ref": "Proceedings of the BioNLP 2017 Workshop (Vancouver, Canada), pages\n  67-75 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02241v1",
            "date": "2017-06-07 16:24:32+00:00",
            "title": "Insights into Analogy Completion from the Biomedical Domain",
            "authors": [
                "Denis Newman-Griffis",
                "Albert M Lai",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Analogy completion has been a popular task in recent years for evaluating the\nsemantic properties of word embeddings, but the standard methodology makes a\nnumber of assumptions about analogies that do not always hold, either in recent\nbenchmark datasets or when expanding into other domains. Through an analysis of\nanalogies in the biomedical domain, we identify three assumptions: that of a\nSingle Answer for any given analogy, that the pairs involved describe the Same\nRelationship, and that each pair is Informative with respect to the other. We\npropose modifying the standard methodology to relax these assumptions by\nallowing for multiple correct answers, reporting MAP and MRR in addition to\naccuracy, and using multiple example pairs. We further present BMASS, a novel\ndataset for evaluating linguistic regularities in biomedical embeddings, and\ndemonstrate that the relationships described in the dataset pose significant\nsemantic challenges to current word embedding methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02427v1",
            "date": "2017-06-08 02:03:32+00:00",
            "title": "Content-Based Table Retrieval for Web Queries",
            "authors": [
                "Zhao Yan",
                "Duyu Tang",
                "Nan Duan",
                "Junwei Bao",
                "Yuanhua Lv",
                "Ming Zhou",
                "Zhoujun Li"
            ],
            "abstract": "Understanding the connections between unstructured text and semi-structured\ntable is an important yet neglected problem in natural language processing. In\nthis work, we focus on content-based table retrieval. Given a query, the task\nis to find the most relevant table from a collection of tables. Further\nprogress towards improving this area requires powerful models of semantic\nmatching and richer training and evaluation resources. To remedy this, we\npresent a ranking based approach, and implement both carefully designed\nfeatures and neural network architectures to measure the relevance between a\nquery and the content of a table. Furthermore, we release an open-domain\ndataset that includes 21,113 web queries for 273,816 tables. We conduct\ncomprehensive experiments on both real world and synthetic datasets. Results\nverify the effectiveness of our approach and present the challenges for this\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02459v1",
            "date": "2017-06-08 07:05:56+00:00",
            "title": "Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization",
            "authors": [
                "Shuming Ma",
                "Xu Sun",
                "Jingjing Xu",
                "Houfeng Wang",
                "Wenjie Li",
                "Qi Su"
            ],
            "abstract": "Current Chinese social media text summarization models are based on an\nencoder-decoder framework. Although its generated summaries are similar to\nsource texts literally, they have low semantic relevance. In this work, our\ngoal is to improve semantic relevance between source texts and summaries for\nChinese social media summarization. We introduce a Semantic Relevance Based\nneural model to encourage high semantic similarity between texts and summaries.\nIn our model, the source text is represented by a gated attention encoder,\nwhile the summary representation is produced by a decoder. Besides, the\nsimilarity score between the representations is maximized during training. Our\nexperiments show that the proposed model outperforms baseline systems on a\nsocial media corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02551v1",
            "date": "2017-06-08 12:48:15+00:00",
            "title": "The Algorithmic Inflection of Russian and Generation of Grammatically Correct Text",
            "authors": [
                "T. M. Sadykov",
                "T. A. Zhukov"
            ],
            "abstract": "We present a deterministic algorithm for Russian inflection. This algorithm\nis implemented in a publicly available web-service www.passare.ru which\nprovides functions for inflection of single words, word matching and synthesis\nof grammatically correct Russian text. The inflectional functions have been\ntested against the annotated corpus of Russian language OpenCorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02737v1",
            "date": "2017-06-08 19:30:02+00:00",
            "title": "Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM",
            "authors": [
                "Takaaki Hori",
                "Shinji Watanabe",
                "Yu Zhang",
                "William Chan"
            ],
            "abstract": "We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR)\nmodel. We learn to listen and write characters with a joint Connectionist\nTemporal Classification (CTC) and attention-based encoder-decoder network. The\nencoder is a deep Convolutional Neural Network (CNN) based on the VGG network.\nThe CTC network sits on top of the encoder and is jointly trained with the\nattention-based decoder. During the beam search process, we combine the CTC\npredictions, the attention-based decoder predictions and a separately trained\nLSTM language model. We achieve a 5-10\\% error reduction compared to prior\nsystems on spontaneous Japanese and Chinese speech, and our end-to-end model\nbeats out traditional hybrid ASR systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02807v2",
            "date": "2017-06-09 01:39:12+00:00",
            "title": "Learning to Embed Words in Context for Syntactic Tasks",
            "authors": [
                "Lifu Tu",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We present models for embedding words in the context of surrounding words.\nSuch models, which we refer to as token embeddings, represent the\ncharacteristics of a word that are specific to a given context, such as word\nsense, syntactic category, and semantic role. We explore simple, efficient\ntoken embedding models based on standard neural network architectures. We learn\ntoken embeddings on a large amount of unannotated text and evaluate them as\nfeatures for part-of-speech taggers and dependency parsers trained on much\nsmaller amounts of annotated data. We find that predictors endowed with token\nembeddings consistently outperform baseline predictors across a range of\ncontext window and training set sizes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02861v3",
            "date": "2017-06-09 08:13:21+00:00",
            "title": "Assigning personality/identity to a chatting machine for coherent conversation generation",
            "authors": [
                "Qiao Qian",
                "Minlie Huang",
                "Haizhou Zhao",
                "Jingfang Xu",
                "Xiaoyan Zhu"
            ],
            "abstract": "Endowing a chatbot with personality or an identity is quite challenging but\ncritical to deliver more realistic and natural conversations. In this paper, we\naddress the issue of generating responses that are coherent to a pre-specified\nagent profile. We design a model consisting of three modules: a profile\ndetector to decide whether a post should be responded using the profile and\nwhich key should be addressed, a bidirectional decoder to generate responses\nforward and backward starting from a selected profile value, and a position\ndetector that predicts a word position from which decoding should start given a\nselected profile value. We show that general conversation data from social\nmedia can be used to generate profile-coherent responses. Manual and automatic\nevaluation shows that our model can deliver more coherent, natural, and\ndiversified responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02883v1",
            "date": "2017-06-09 10:17:24+00:00",
            "title": "Overview of the NLPCC 2017 Shared Task: Chinese News Headline Categorization",
            "authors": [
                "Xipeng Qiu",
                "Jingjing Gong",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we give an overview for the shared task at the CCF Conference\non Natural Language Processing \\& Chinese Computing (NLPCC 2017): Chinese News\nHeadline Categorization. The dataset of this shared task consists 18 classes,\n12,000 short texts along with corresponded labels for each class. The dataset\nand example code can be accessed at\nhttps://github.com/FudanNLP/nlpcc2017_news_headline_categorization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02909v1",
            "date": "2017-06-08 03:01:37+00:00",
            "title": "Deriving a Representative Vector for Ontology Classes with Instance Word Vector Embeddings",
            "authors": [
                "Vindula Jayawardana",
                "Dimuthu Lakmal",
                "Nisansa de Silva",
                "Amal Shehan Perera",
                "Keet Sugathadasa",
                "Buddhi Ayesha"
            ],
            "abstract": "Selecting a representative vector for a set of vectors is a very common\nrequirement in many algorithmic tasks. Traditionally, the mean or median vector\nis selected. Ontology classes are sets of homogeneous instance objects that can\nbe converted to a vector space by word vector embeddings. This study proposes a\nmethodology to derive a representative vector for ontology classes whose\ninstances were converted to the vector space. We start by deriving five\ncandidate vectors which are then used to train a machine learning model that\nwould calculate a representative vector for the class. We show that our\nmethodology out-performs the traditional mean and median vector\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03148v1",
            "date": "2017-06-09 22:44:31+00:00",
            "title": "Trimming and Improving Skip-thought Vectors",
            "authors": [
                "Shuai Tang",
                "Hailin Jin",
                "Chen Fang",
                "Zhaowen Wang",
                "Virginia R. de Sa"
            ],
            "abstract": "The skip-thought model has been proven to be effective at learning sentence\nrepresentations and capturing sentence semantics. In this paper, we propose a\nsuite of techniques to trim and improve it. First, we validate a hypothesis\nthat, given a current sentence, inferring the previous and inferring the next\nsentence provide similar supervision power, therefore only one decoder for\npredicting the next sentence is preserved in our trimmed skip-thought model.\nSecond, we present a connection layer between encoder and decoder to help the\nmodel to generalize better on semantic relatedness tasks. Third, we found that\na good word embedding initialization is also essential for learning better\nsentence representations. We train our model unsupervised on a large corpus\nwith contiguous sentences, and then evaluate the trained model on 7 supervised\ntasks, which includes semantic relatedness, paraphrase detection, and text\nclassification benchmarks. We empirically show that, our proposed model is a\nfaster, lighter-weight and equally powerful alternative to the original\nskip-thought model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03191v2",
            "date": "2017-06-10 07:02:57+00:00",
            "title": "Classification of Questions and Learning Outcome Statements (LOS) Into Blooms Taxonomy (BT) By Similarity Measurements Towards Extracting Of Learning Outcome from Learning Material",
            "authors": [
                "Shadi Diab",
                "Badie Sartawi"
            ],
            "abstract": "Blooms Taxonomy (BT) have been used to classify the objectives of learning\noutcome by dividing the learning into three different domains; the cognitive\ndomain, the effective domain and the psychomotor domain. In this paper, we are\nintroducing a new approach to classify the questions and learning outcome\nstatements (LOS) into Blooms taxonomy (BT) and to verify BT verb lists, which\nare being cited and used by academicians to write questions and (LOS). An\nexperiment was designed to investigate the semantic relationship between the\naction verbs used in both questions and LOS to obtain more accurate\nclassification of the levels of BT. A sample of 775 different action verbs\ncollected from different universities allows us to measure an accurate and\nclear-cut cognitive level for the action verb. It is worth mentioning that\nnatural language processing techniques were used to develop our rules as to\ninduce the questions into chunks in order to extract the action verbs. Our\nproposed solution was able to classify the action verb into a precise level of\nthe cognitive domain. We, on our side, have tested and evaluated our proposed\nsolution using confusion matrix. The results of evaluation tests yielded 97%\nfor the macro average of precision and 90% for F1. Thus, the outcome of the\nresearch suggests that it is crucial to analyse and verify the action verbs\ncited and used by academicians to write LOS and classify their questions based\non blooms taxonomy in order to obtain a definite and more accurate\nclassification.",
            "journal ref": "International Journal of Managing Information Technology (IJMIT)\n  Vol.9, No.2, May 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03216v2",
            "date": "2017-06-10 10:25:20+00:00",
            "title": "Articulation rate in Swedish child-directed speech increases as a function of the age of the child even when surprisal is controlled for",
            "authors": [
                "Johan Sjons",
                "Thomas H\u00f6rberg",
                "Robert \u00d6stling",
                "Johannes Bjerva"
            ],
            "abstract": "In earlier work, we have shown that articulation rate in Swedish\nchild-directed speech (CDS) increases as a function of the age of the child,\neven when utterance length and differences in articulation rate between\nsubjects are controlled for. In this paper we show on utterance level in\nspontaneous Swedish speech that i) for the youngest children, articulation rate\nin CDS is lower than in adult-directed speech (ADS), ii) there is a significant\nnegative correlation between articulation rate and surprisal (the negative log\nprobability) in ADS, and iii) the increase in articulation rate in Swedish CDS\nas a function of the age of the child holds, even when surprisal along with\nutterance length and differences in articulation rate between speakers are\ncontrolled for. These results indicate that adults adjust their articulation\nrate to make it fit the linguistic capacity of the child.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03335v3",
            "date": "2017-06-11 10:18:46+00:00",
            "title": "Exploring Automated Essay Scoring for Nonnative English Speakers",
            "authors": [
                "Amber Nigam"
            ],
            "abstract": "Automated Essay Scoring (AES) has been quite popular and is being widely\nused. However, lack of appropriate methodology for rating nonnative English\nspeakers' essays has meant a lopsided advancement in this field. In this paper,\nwe report initial results of our experiments with nonnative AES that learns\nfrom manual evaluation of nonnative essays. For this purpose, we conducted an\nexercise in which essays written by nonnative English speakers in test\nenvironment were rated both manually and by the automated system designed for\nthe experiment. In the process, we experimented with a few features to learn\nabout nuances linked to nonnative evaluation. The proposed methodology of\nautomated essay evaluation has yielded a correlation coefficient of 0.750 with\nthe manual evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03367v1",
            "date": "2017-06-11 16:04:42+00:00",
            "title": "A Full Non-Monotonic Transition System for Unrestricted Non-Projective Parsing",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Restricted non-monotonicity has been shown beneficial for the projective\narc-eager dependency parser in previous research, as posterior decisions can\nrepair mistakes made in previous states due to the lack of information. In this\npaper, we propose a novel, fully non-monotonic transition system based on the\nnon-projective Covington algorithm. As a non-monotonic system requires\nexploration of erroneous actions during the training process, we develop\nseveral non-monotonic variants of the recently defined dynamic oracle for the\nCovington parser, based on tight approximations of the loss. Experiments on\ndatasets from the CoNLL-X and CoNLL-XI shared tasks show that a non-monotonic\ndynamic oracle outperforms the monotonic version in the majority of languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03441v1",
            "date": "2017-06-12 02:24:33+00:00",
            "title": "Dialog Structure Through the Lens of Gender, Gender Environment, and Power",
            "authors": [
                "Vinodkumar Prabhakaran",
                "Owen Rambow"
            ],
            "abstract": "Understanding how the social context of an interaction affects our dialog\nbehavior is of great interest to social scientists who study human behavior, as\nwell as to computer scientists who build automatic methods to infer those\nsocial contexts. In this paper, we study the interaction of power, gender, and\ndialog behavior in organizational interactions. In order to perform this study,\nwe first construct the Gender Identified Enron Corpus of emails, in which we\nsemi-automatically assign the gender of around 23,000 individuals who authored\naround 97,000 email messages in the Enron corpus. This corpus, which is made\nfreely available, is orders of magnitude larger than previously existing gender\nidentified corpora in the email domain. Next, we use this corpus to perform a\nlarge-scale data-oriented study of the interplay of gender and manifestations\nof power. We argue that, in addition to one's own gender, the \"gender\nenvironment\" of an interaction, i.e., the gender makeup of one's interlocutors,\nalso affects the way power is manifested in dialog. We focus especially on\nmanifestations of power in the dialog structure --- both, in a shallow sense\nthat disregards the textual content of messages (e.g., how often do the\nparticipants contribute, how often do they get replies etc.), as well as the\nstructure that is expressed within the textual content (e.g., who issues\nrequests and how are they made, whose requests get responses etc.). We find\nthat both gender and gender environment affect the ways power is manifested in\ndialog, resulting in patterns that reveal the underlying factors. Finally, we\nshow the utility of gender information in the problem of automatically\npredicting the direction of power between pairs of participants in email\ninteractions.",
            "journal ref": "Journal for Dialogue & Discourse 8(2) (2017) 21-55",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03499v1",
            "date": "2017-06-12 08:08:00+00:00",
            "title": "SU-RUG at the CoNLL-SIGMORPHON 2017 shared task: Morphological Inflection with Attentional Sequence-to-Sequence Models",
            "authors": [
                "Robert \u00d6stling",
                "Johannes Bjerva"
            ],
            "abstract": "This paper describes the Stockholm University/University of Groningen\n(SU-RUG) system for the SIGMORPHON 2017 shared task on morphological\ninflection. Our system is based on an attentional sequence-to-sequence neural\nnetwork model using Long Short-Term Memory (LSTM) cells, with joint training of\nmorphological inflection and the inverse transformation, i.e. lemmatization and\nmorphological analysis. Our system outperforms the baseline with a large\nmargin, and our submission ranks as the 4th best team for the track we\nparticipate in (task 1, high-resource).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03530v1",
            "date": "2017-06-12 09:21:45+00:00",
            "title": "Candidate sentence selection for language learning exercises: from a comprehensive framework to an empirical evaluation",
            "authors": [
                "Ildik\u00f3 Pil\u00e1n",
                "Elena Volodina",
                "Lars Borin"
            ],
            "abstract": "We present a framework and its implementation relying on Natural Language\nProcessing methods, which aims at the identification of exercise item\ncandidates from corpora. The hybrid system combining heuristics and machine\nlearning methods includes a number of relevant selection criteria. We focus on\ntwo fundamental aspects: linguistic complexity and the dependence of the\nextracted sentences on their original context. Previous work on exercise\ngeneration addressed these two criteria only to a limited extent, and a refined\noverall candidate sentence selection framework appears also to be lacking. In\naddition to a detailed description of the system, we present the results of an\nempirical evaluation conducted with language teachers and learners which\nindicate the usefulness of the system for educational purposes. We have\nintegrated our system into a freely available online learning platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03542v1",
            "date": "2017-06-12 10:00:47+00:00",
            "title": "Exploring the Syntactic Abilities of RNNs with Multi-task Learning",
            "authors": [
                "Emile Enguehard",
                "Yoav Goldberg",
                "Tal Linzen"
            ],
            "abstract": "Recent work has explored the syntactic abilities of RNNs using the\nsubject-verb agreement task, which diagnoses sensitivity to sentence structure.\nRNNs performed this task well in common cases, but faltered in complex\nsentences (Linzen et al., 2016). We test whether these errors are due to\ninherent limitations of the architecture or to the relatively indirect\nsupervision provided by most agreement dependencies in a corpus. We trained a\nsingle RNN to perform both the agreement task and an additional task, either\nCCG supertagging or language modeling. Multi-task training led to significantly\nlower error rates, in particular on complex sentences, suggesting that RNNs\nhave the ability to evolve more sophisticated syntactic representations than\nshown before. We also show that easily available agreement training data can\nimprove performance on other syntactic tasks, in particular when only a limited\namount of training data is available for those tasks. The multi-task paradigm\ncan also be leveraged to inject grammatical knowledge into language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03747v1",
            "date": "2017-06-12 17:35:41+00:00",
            "title": "Acoustic data-driven lexicon learning based on a greedy pronunciation selection framework",
            "authors": [
                "Xiaohui Zhang",
                "Vimal Manohar",
                "Daniel Povey",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Speech recognition systems for irregularly-spelled languages like English\nnormally require hand-written pronunciations. In this paper, we describe a\nsystem for automatically obtaining pronunciations of words for which\npronunciations are not available, but for which transcribed data exists. Our\nmethod integrates information from the letter sequence and from the acoustic\nevidence. The novel aspect of the problem that we address is the problem of how\nto prune entries from such a lexicon (since, empirically, lexicons with too\nmany entries do not tend to be good for ASR performance). Experiments on\nvarious ASR tasks show that, with the proposed framework, starting with an\ninitial lexicon of several thousand words, we are able to learn a lexicon which\nperforms close to a full expert lexicon in terms of WER performance on test\ndata, and is better than lexicons built using G2P alone or with a pruning\ncriterion based on pronunciation probability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03799v2",
            "date": "2017-06-12 18:24:25+00:00",
            "title": "Verb Physics: Relative Physical Knowledge of Actions and Objects",
            "authors": [
                "Maxwell Forbes",
                "Yejin Choi"
            ],
            "abstract": "Learning commonsense knowledge from natural language text is nontrivial due\nto reporting bias: people rarely state the obvious, e.g., \"My house is bigger\nthan me.\" However, while rarely stated explicitly, this trivial everyday\nknowledge does influence the way people talk about the world, which provides\nindirect clues to reason about the world. For example, a statement like, \"Tyler\nentered his house\" implies that his house is bigger than Tyler.\n  In this paper, we present an approach to infer relative physical knowledge of\nactions and objects along five dimensions (e.g., size, weight, and strength)\nfrom unstructured natural language text. We frame knowledge acquisition as\njoint inference over two closely related problems: learning (1) relative\nphysical knowledge of object pairs and (2) physical implications of actions\nwhen applied to those object pairs. Empirical results demonstrate that it is\npossible to extract knowledge of actions and objects from language and that\njoint inference over different types of knowledge improves performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03818v1",
            "date": "2017-06-12 19:30:57+00:00",
            "title": "Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings",
            "authors": [
                "Shane Settle",
                "Keith Levin",
                "Herman Kamper",
                "Karen Livescu"
            ],
            "abstract": "Query-by-example search often uses dynamic time warping (DTW) for comparing\nqueries and proposed matching segments. Recent work has shown that comparing\nspeech segments by representing them as fixed-dimensional vectors --- acoustic\nword embeddings --- and measuring their vector distance (e.g., cosine distance)\ncan discriminate between words more accurately than DTW-based approaches. We\nconsider an approach to query-by-example search that embeds both the query and\ndatabase segments according to a neural model, followed by nearest-neighbor\nsearch to find the matching segments. Earlier work on embedding-based\nquery-by-example, using template-based acoustic word embeddings, achieved\ncompetitive performance. We find that our embeddings, based on recurrent neural\nnetworks trained to optimize word discrimination, achieve substantial\nimprovements in performance and run-time efficiency over the previous\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03824v1",
            "date": "2017-06-12 19:51:00+00:00",
            "title": "Attention-based Vocabulary Selection for NMT Decoding",
            "authors": [
                "Baskaran Sankaran",
                "Markus Freitag",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Neural Machine Translation (NMT) models usually use large target vocabulary\nsizes to capture most of the words in the target language. The vocabulary size\nis a big factor when decoding new sentences as the final softmax layer\nnormalizes over all possible target words. To address this problem, it is\nwidely common to restrict the target vocabulary with candidate lists based on\nthe source sentence. Usually, the candidate lists are a combination of external\nword-to-word aligner, phrase table entries or most frequent words. In this\nwork, we propose a simple and yet novel approach to learn candidate lists\ndirectly from the attention layer during NMT training. The candidate lists are\nhighly optimized for the current NMT model and do not need any external\ncomputation of the candidate pool. We show significant decoding speedup\ncompared with using the entire vocabulary, without losing any translation\nquality for two language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03872v1",
            "date": "2017-06-12 23:57:48+00:00",
            "title": "Six Challenges for Neural Machine Translation",
            "authors": [
                "Philipp Koehn",
                "Rebecca Knowles"
            ],
            "abstract": "We explore six challenges for neural machine translation: domain mismatch,\namount of training data, rare words, long sentences, word alignment, and beam\nsearch. We show both deficiencies and improvements over the quality of\nphrase-based statistical machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03952v2",
            "date": "2017-06-13 08:28:39+00:00",
            "title": "Modelling prosodic structure using Artificial Neural Networks",
            "authors": [
                "Jean-Philippe Bernardy",
                "Charalambos Themistocleous"
            ],
            "abstract": "The ability to accurately perceive whether a speaker is asking a question or\nis making a statement is crucial for any successful interaction. However,\nlearning and classifying tonal patterns has been a challenging task for\nautomatic speech recognition and for models of tonal representation, as tonal\ncontours are characterized by significant variation. This paper provides a\nclassification model of Cypriot Greek questions and statements. We evaluate two\nstate-of-the-art network architectures: a Long Short-Term Memory (LSTM) network\nand a convolutional network (ConvNet). The ConvNet outperforms the LSTM in the\nclassification task and exhibited an excellent performance with 95%\nclassification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04138v2",
            "date": "2017-06-13 15:55:02+00:00",
            "title": "An Exploration of Neural Sequence-to-Sequence Architectures for Automatic Post-Editing",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz"
            ],
            "abstract": "In this work, we explore multiple neural architectures adapted for the task\nof automatic post-editing of machine translation output. We focus on neural\nend-to-end models that combine both inputs $mt$ (raw MT output) and $src$\n(source language input) in a single neural architecture, modeling $\\{mt, src\\}\n\\rightarrow pe$ directly. Apart from that, we investigate the influence of\nhard-attention models which seem to be well-suited for monolingual tasks, as\nwell as combinations of both ideas. We report results on data sets provided\nduring the WMT-2016 shared task on automatic post-editing and can demonstrate\nthat dual-attention models that incorporate all available data in the APE\nscenario in a single model improve on the best shared task system and on all\nother published results after the shared task. Dual-attention models that are\ncombined with hard attention remain competitive despite applying fewer changes\nto the input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04389v1",
            "date": "2017-06-14 09:59:47+00:00",
            "title": "Fine-grained human evaluation of neural versus phrase-based machine translation",
            "authors": [
                "Filip Klubi\u010dka",
                "Antonio Toral",
                "V\u00edctor M. S\u00e1nchez-Cartagena"
            ],
            "abstract": "We compare three approaches to statistical machine translation (pure\nphrase-based, factored phrase-based and neural) by performing a fine-grained\nmanual evaluation via error annotation of the systems' outputs. The error types\nin our annotation are compliant with the multidimensional quality metrics\n(MQM), and the annotation is performed by two annotators. Inter-annotator\nagreement is high for such a task, and results show that the best performing\nsystem (neural) reduces the errors produced by the worst system (phrase-based)\nby 54%.",
            "journal ref": "The Prague Bulletin of Mathematical Linguistics No. 108, pp.\n  121-132 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04473v1",
            "date": "2017-06-14 13:18:08+00:00",
            "title": "Idea density for predicting Alzheimer's disease from transcribed speech",
            "authors": [
                "Kairit Sirts",
                "Olivier Piguet",
                "Mark Johnson"
            ],
            "abstract": "Idea Density (ID) measures the rate at which ideas or elementary predications\nare expressed in an utterance or in a text. Lower ID is found to be associated\nwith an increased risk of developing Alzheimer's disease (AD) (Snowdon et al.,\n1996; Engelman et al., 2010). ID has been used in two different versions:\npropositional idea density (PID) counts the expressed ideas and can be applied\nto any text while semantic idea density (SID) counts pre-defined information\ncontent units and is naturally more applicable to normative domains, such as\npicture description tasks. In this paper, we develop DEPID, a novel\ndependency-based method for computing PID, and its version DEPID-R that enables\nto exclude repeating ideas---a feature characteristic to AD speech. We conduct\nthe first comparison of automatically extracted PID and SID in the diagnostic\nclassification task on two different AD datasets covering both closed-topic and\nfree-recall domains. While SID performs better on the normative dataset, adding\nPID leads to a small but significant improvement (+1.7 F-score). On the\nfree-topic dataset, PID performs better than SID as expected (77.6 vs 72.3 in\nF-score) but adding the features derived from the word embedding clustering\nunderlying the automatic SID increases the results considerably, leading to an\nF-score of 84.8.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04815v6",
            "date": "2017-06-15 11:10:33+00:00",
            "title": "S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension",
            "authors": [
                "Chuanqi Tan",
                "Furu Wei",
                "Nan Yang",
                "Bowen Du",
                "Weifeng Lv",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we present a novel approach to machine reading comprehension\nfor the MS-MARCO dataset. Unlike the SQuAD dataset that aims to answer a\nquestion with exact text spans in a passage, the MS-MARCO dataset defines the\ntask as answering a question from multiple passages and the words in the answer\nare not necessary in the passages. We therefore develop an\nextraction-then-synthesis framework to synthesize answers from extraction\nresults. Specifically, the answer extraction model is first employed to predict\nthe most important sub-spans from the passage as evidence, and the answer\nsynthesis model takes the evidence as additional features along with the\nquestion and passage to further elaborate the final answers. We build the\nanswer extraction model with state-of-the-art neural networks for single\npassage reading comprehension, and propose an additional task of passage\nranking to help answer extraction in multiple passages. The answer synthesis\nmodel is based on the sequence-to-sequence neural networks with extracted\nevidences as features. Experiments show that our extraction-then-synthesis\nmethod outperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04971v1",
            "date": "2017-06-15 17:14:17+00:00",
            "title": "German in Flux: Detecting Metaphoric Change via Word Entropy",
            "authors": [
                "Dominik Schlechtweg",
                "Stefanie Eckmann",
                "Enrico Santus",
                "Sabine Schulte im Walde",
                "Daniel Hole"
            ],
            "abstract": "This paper explores the information-theoretic measure entropy to detect\nmetaphoric change, transferring ideas from hypernym detection to research on\nlanguage change. We also build the first diachronic test set for German as a\nstandard for metaphoric change annotation. Our model shows high performance, is\nunsupervised, language-independent and generalizable to other processes of\nsemantic change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04997v1",
            "date": "2017-06-15 07:24:23+00:00",
            "title": "Extracting Formal Models from Normative Texts",
            "authors": [
                "John J. Camilleri",
                "Normunds Gr\u016bz\u012btis",
                "Gerardo Schneider"
            ],
            "abstract": "We are concerned with the analysis of normative texts - documents based on\nthe deontic notions of obligation, permission, and prohibition. Our goal is to\nmake queries about these notions and verify that a text satisfies certain\nproperties concerning causality of actions and timing constraints. This\nrequires taking the original text and building a representation (model) of it\nin a formal language, in our case the C-O Diagram formalism. We present an\nexperimental, semi-automatic aid that helps to bridge the gap between a\nnormative text in natural language and its C-O Diagram representation. Our\napproach consists of using dependency structures obtained from the\nstate-of-the-art Stanford Parser, and applying our own rules and heuristics in\norder to extract the relevant components. The result is a tabular data\nstructure where each sentence is split into suitable fields, which can then be\nconverted into a C-O Diagram. The process is not fully automatic however, and\nsome post-editing is generally required of the user. We apply our tool and\nperform experiments on documents from different domains, and report an initial\nevaluation of the accuracy and feasibility of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05083v2",
            "date": "2017-06-15 20:47:03+00:00",
            "title": "Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation",
            "authors": [
                "Chris Hokamp"
            ],
            "abstract": "This work presents a novel approach to Automatic Post-Editing (APE) and\nWord-Level Quality Estimation (QE) using ensembles of specialized Neural\nMachine Translation (NMT) systems. Word-level features that have proven\neffective for QE are included as input factors, expanding the representation of\nthe original source and the machine translation hypothesis, which are used to\ngenerate an automatically post-edited hypothesis. We train a suite of NMT\nmodels that use different input representations, but share the same output\nspace. These models are then ensembled together, and tuned for both the APE and\nthe QE task. We thus attempt to connect the state-of-the-art approaches to APE\nand QE within a single framework. Our models achieve state-of-the-art results\nin both tasks, with the only difference in the tuning step which learns weights\nfor each component of the ensemble.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05111v1",
            "date": "2017-06-15 23:07:06+00:00",
            "title": "A Mixture Model for Learning Multi-Sense Word Embeddings",
            "authors": [
                "Dai Quoc Nguyen",
                "Dat Quoc Nguyen",
                "Ashutosh Modi",
                "Stefan Thater",
                "Manfred Pinkal"
            ],
            "abstract": "Word embeddings are now a standard technique for inducing meaning\nrepresentations for words. For getting good representations, it is important to\ntake into account different senses of a word. In this paper, we propose a\nmixture model for learning multi-sense word embeddings. Our model generalizes\nthe previous works in that it allows to induce different weights of different\nsenses of a word. The experimental results show that our model outperforms\nprevious models on standard evaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05140v1",
            "date": "2017-06-16 03:53:38+00:00",
            "title": "An Automatic Approach for Document-level Topic Model Evaluation",
            "authors": [
                "Shraey Bhatia",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "Topic models jointly learn topics and document-level topic distribution.\nExtrinsic evaluation of topic models tends to focus exclusively on topic-level\nevaluation, e.g. by assessing the coherence of topics. We demonstrate that\nthere can be large discrepancies between topic- and document-level model\nquality, and that basing model evaluation on topic-level analysis can be highly\nmisleading. We propose a method for automatically predicting topic model\nquality based on analysis of document-level topic allocations, and provide\nempirical evidence for its robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05674v2",
            "date": "2017-06-18 15:53:59+00:00",
            "title": "Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach",
            "authors": [
                "Takuo Hamaguchi",
                "Hidekazu Oiwa",
                "Masashi Shimbo",
                "Yuji Matsumoto"
            ],
            "abstract": "Knowledge base completion (KBC) aims to predict missing information in a\nknowledge base.In this paper, we address the out-of-knowledge-base (OOKB)\nentity problem in KBC:how to answer queries concerning test entities not\nobserved at training time. Existing embedding-based KBC models assume that all\ntest entities are available at training time, making it unclear how to obtain\nembeddings for new entities without costly retraining. To solve the OOKB entity\nproblem without retraining, we use graph neural networks (Graph-NNs) to compute\nthe embeddings of OOKB entities, exploiting the limited auxiliary knowledge\nprovided at test time.The experimental results show the effectiveness of our\nproposed model in the OOKB setting.Additionally, in the standard KBC setting in\nwhich OOKB entities are not involved, our model achieves state-of-the-art\nperformance on the WordNet dataset. The code and dataset are available at\nhttps://github.com/takuo-h/GNN-for-OOKB",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05723v1",
            "date": "2017-06-18 20:39:24+00:00",
            "title": "Detecting Large Concept Extensions for Conceptual Analysis",
            "authors": [
                "Louis Chartrand",
                "Jackie C. K. Cheung",
                "Mohamed Bouguessa"
            ],
            "abstract": "When performing a conceptual analysis of a concept, philosophers are\ninterested in all forms of expression of a concept in a text---be it direct or\nindirect, explicit or implicit. In this paper, we experiment with topic-based\nmethods of automating the detection of concept expressions in order to\nfacilitate philosophical conceptual analysis. We propose six methods based on\nLDA, and evaluate them on a new corpus of court decision that we had annotated\nby experts and non-experts. Our results indicate that these methods can yield\nimportant improvements over the keyword heuristic, which is often used as a\nconcept detection heuristic in many contexts. While more work remains to be\ndone, this indicates that detecting concepts through topics can serve as a\ngeneral-purpose method for at least some forms of concept expression that are\nnot captured using naive keyword approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05765v1",
            "date": "2017-06-19 02:38:01+00:00",
            "title": "An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation",
            "authors": [
                "Makoto Morishita",
                "Yusuke Oda",
                "Graham Neubig",
                "Koichiro Yoshino",
                "Katsuhito Sudoh",
                "Satoshi Nakamura"
            ],
            "abstract": "Training of neural machine translation (NMT) models usually uses mini-batches\nfor efficiency purposes. During the mini-batched training process, it is\nnecessary to pad shorter sentences in a mini-batch to be equal in length to the\nlongest sentence therein for efficient computation. Previous work has noted\nthat sorting the corpus based on the sentence length before making mini-batches\nreduces the amount of padding and increases the processing speed. However,\ndespite the fact that mini-batch creation is an essential step in NMT training,\nwidely used NMT toolkits implement disparate strategies for doing so, which\nhave not been empirically validated or compared. This work investigates\nmini-batch creation strategies with experiments over two different datasets.\nOur results suggest that the choice of a mini-batch creation strategy has a\nlarge effect on NMT training and some length-based sorting strategies do not\nalways work well compared with simple shuffling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06177v1",
            "date": "2017-06-19 21:04:22+00:00",
            "title": "Topic Modeling for Classification of Clinical Reports",
            "authors": [
                "Efsun Sarioglu Kayi",
                "Kabir Yadav",
                "James M. Chamberlain",
                "Hyeong-Ah Choi"
            ],
            "abstract": "Electronic health records (EHRs) contain important clinical information about\npatients. Efficient and effective use of this information could supplement or\neven replace manual chart review as a means of studying and improving the\nquality and safety of healthcare delivery. However, some of these clinical data\nare in the form of free text and require pre-processing before use in automated\nsystems. A common free text data source is radiology reports, typically\ndictated by radiologists to explain their interpretations. We sought to\ndemonstrate machine learning classification of computed tomography (CT) imaging\nreports into binary outcomes, i.e. positive and negative for fracture, using\nregular text classification and classifiers based on topic modeling. Topic\nmodeling provides interpretable themes (topic distributions) in reports, a\nrepresentation that is more compact than the commonly used bag-of-words\nrepresentation and can be processed faster than raw text in subsequent\nautomated processes. We demonstrate new classifiers based on this topic\nmodeling representation of the reports. Aggregate topic classifier (ATC) and\nconfidence-based topic classifier (CTC) use a single topic that is determined\nfrom the training dataset based on different measures to classify the reports\non the test dataset. Alternatively, similarity-based topic classifier (STC)\nmeasures the similarity between the reports' topic distributions to determine\nthe predicted class. Our proposed topic modeling-based classifier systems are\nshown to be competitive with existing text classification techniques and\nprovides an efficient and interpretable representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06363v1",
            "date": "2017-06-20 11:13:06+00:00",
            "title": "Improving text classification with vectors of reduced precision",
            "authors": [
                "Krzysztof Wr\u00f3bel",
                "Maciej Wielgosz",
                "Marcin Pietro\u0144",
                "Micha\u0142 Karwatowski",
                "Aleksander Smywi\u0144ski-Pohl"
            ],
            "abstract": "This paper presents the analysis of the impact of a floating-point number\nprecision reduction on the quality of text classification. The precision\nreduction of the vectors representing the data (e.g. TF-IDF representation in\nour case) allows for a decrease of computing time and memory footprint on\ndedicated hardware platforms. The impact of precision reduction on the\nclassification quality was performed on 5 corpora, using 4 different\nclassifiers. Also, dimensionality reduction was taken into account. Results\nindicate that the precision reduction improves classification accuracy for most\ncases (up to 25% of error reduction). In general, the reduction from 64 to 4\nbits gives the best scores and ensures that the results will not be worse than\nwith the full floating-point representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06415v1",
            "date": "2017-06-20 13:29:16+00:00",
            "title": "THUMT: An Open Source Toolkit for Neural Machine Translation",
            "authors": [
                "Jiacheng Zhang",
                "Yanzhuo Ding",
                "Shiqi Shen",
                "Yong Cheng",
                "Maosong Sun",
                "Huanbo Luan",
                "Yang Liu"
            ],
            "abstract": "This paper introduces THUMT, an open-source toolkit for neural machine\ntranslation (NMT) developed by the Natural Language Processing Group at\nTsinghua University. THUMT implements the standard attention-based\nencoder-decoder framework on top of Theano and supports three training\ncriteria: maximum likelihood estimation, minimum risk training, and\nsemi-supervised training. It features a visualization tool for displaying the\nrelevance between hidden states in neural networks and contextual words, which\nhelps to analyze the internal workings of NMT. Experiments on Chinese-English\ndatasets show that THUMT using minimum risk training significantly outperforms\nGroundHog, a state-of-the-art toolkit for NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06542v2",
            "date": "2017-06-12 04:54:41+00:00",
            "title": "Extract with Order for Coherent Multi-Document Summarization",
            "authors": [
                "Mir Tafseer Nayeem",
                "Yllias Chali"
            ],
            "abstract": "In this work, we aim at developing an extractive summarizer in the\nmulti-document setting. We implement a rank based sentence selection using\ncontinuous vector representations along with key-phrases. Furthermore, we\npropose a model to tackle summary coherence for increasing readability. We\nconduct experiments on the Document Understanding Conference (DUC) 2004\ndatasets using ROUGE toolkit. Our experiments demonstrate that the methods\nbring significant improvements over the state of the art methods in terms of\ninformativity and coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06749v1",
            "date": "2017-06-21 06:11:59+00:00",
            "title": "Cross-language Learning with Adversarial Neural Networks: Application to Community Question Answering",
            "authors": [
                "Shafiq Joty",
                "Preslav Nakov",
                "Llu\u00eds M\u00e0rquez",
                "Israa Jaradat"
            ],
            "abstract": "We address the problem of cross-language adaptation for question-question\nsimilarity reranking in community question answering, with the objective to\nport a system trained on one input language to another input language given\nlabeled training data for the first language and only unlabeled data for the\nsecond language. In particular, we propose to use adversarial training of\nneural networks to learn high-level features that are discriminative for the\nmain learning task, and at the same time are invariant across the input\nlanguages. The evaluation results show sizable improvements for our\ncross-language adversarial neural network (CLANN) model over a strong\nnon-adversarial system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06802v1",
            "date": "2017-06-21 09:14:02+00:00",
            "title": "JaTeCS an open-source JAva TExt Categorization System",
            "authors": [
                "Andrea Esuli",
                "Tiziano Fagni",
                "Alejandro Moreo Fernandez"
            ],
            "abstract": "JaTeCS is an open source Java library that supports research on automatic\ntext categorization and other related problems, such as ordinal regression and\nquantification, which are of special interest in opinion mining applications.\nIt covers all the steps of an experimental activity, from reading the corpus to\nthe evaluation of the experimental results. As JaTeCS is focused on text as the\nmain input data, it provides the user with many text-dedicated tools, e.g.:\ndata readers for many formats, including the most commonly used text corpora\nand lexical resources, natural language processing tools, multi-language\nsupport, methods for feature selection and weighting, the implementation of\nmany machine learning algorithms as well as wrappers for well-known external\nsoftware (e.g., SVM_light) which enable their full control from code. JaTeCS\nsupport its expansion by abstracting through interfaces many of the typical\ntools and procedures used in text processing tasks. The library also provides a\nnumber of \"template\" implementations of typical experimental setups (e.g.,\ntrain-test, k-fold validation, grid-search optimization, randomized runs) which\nenable fast realization of experiments just by connecting the templates with\ndata readers, learning algorithms and evaluation measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06894v1",
            "date": "2017-06-21 13:32:21+00:00",
            "title": "Stance Detection in Turkish Tweets",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck"
            ],
            "abstract": "Stance detection is a classification problem in natural language processing\nwhere for a text and target pair, a class result from the set {Favor, Against,\nNeither} is expected. It is similar to the sentiment analysis problem but\ninstead of the sentiment of the text author, the stance expressed for a\nparticular target is investigated in stance detection. In this paper, we\npresent a stance detection tweet data set for Turkish comprising stance\nannotations of these tweets for two popular sports clubs as targets.\nAdditionally, we provide the evaluation results of SVM classifiers for each\ntarget on this data set, where the classifiers use unigram, bigram, and hashtag\nfeatures. This study is significant as it presents one of the initial stance\ndetection data sets proposed so far and the first one for Turkish language, to\nthe best of our knowledge. The data set and the evaluation results of the\ncorresponding SVM-based approaches will form plausible baselines for the\ncomparison of future studies on stance detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06896v1",
            "date": "2017-06-20 09:44:52+00:00",
            "title": "Effective Spoken Language Labeling with Deep Recurrent Neural Networks",
            "authors": [
                "Marco Dinarelli",
                "Yoann Dupont",
                "Isabelle Tellier"
            ],
            "abstract": "Understanding spoken language is a highly complex problem, which can be\ndecomposed into several simpler tasks. In this paper, we focus on Spoken\nLanguage Understanding (SLU), the module of spoken dialog systems responsible\nfor extracting a semantic interpretation from the user utterance. The task is\ntreated as a labeling problem. In the past, SLU has been performed with a wide\nvariety of probabilistic models. The rise of neural networks, in the last\ncouple of years, has opened new interesting research directions in this domain.\nRecurrent Neural Networks (RNNs) in particular are able not only to represent\nseveral pieces of information as embeddings but also, thanks to their recurrent\narchitecture, to encode as embeddings relatively long contexts. Such long\ncontexts are in general out of reach for models previously used for SLU. In\nthis paper we propose novel RNNs architectures for SLU which outperform\nprevious ones. Starting from a published idea as base block, we design new deep\nRNNs achieving state-of-the-art results on two widely used corpora for SLU:\nATIS (Air Traveling Information System), in English, and MEDIA (Hotel\ninformation and reservation in France), in French.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07238v1",
            "date": "2017-06-22 10:11:34+00:00",
            "title": "Automatic Quality Estimation for ASR System Combination",
            "authors": [
                "Shahab Jalalvand",
                "Matteo Negri",
                "Daniele Falavigna",
                "Marco Matassoni",
                "Marco Turchi"
            ],
            "abstract": "Recognizer Output Voting Error Reduction (ROVER) has been widely used for\nsystem combination in automatic speech recognition (ASR). In order to select\nthe most appropriate words to insert at each position in the output\ntranscriptions, some ROVER extensions rely on critical information such as\nconfidence scores and other ASR decoder features. This information, which is\nnot always available, highly depends on the decoding process and sometimes\ntends to over estimate the real quality of the recognized words. In this paper\nwe propose a novel variant of ROVER that takes advantage of ASR quality\nestimation (QE) for ranking the transcriptions at \"segment level\" instead of:\ni) relying on confidence scores, or ii) feeding ROVER with randomly ordered\nhypotheses. We first introduce an effective set of features to compensate for\nthe absence of ASR decoder information. Then, we apply QE techniques to perform\naccurate hypothesis ranking at segment-level before starting the fusion\nprocess. The evaluation is carried out on two different tasks, in which we\nrespectively combine hypotheses coming from independent ASR systems and\nmulti-microphone recordings. In both tasks, it is assumed that the ASR decoder\ninformation is not available. The proposed approach significantly outperforms\nstandard ROVER and it is competitive with two strong oracles that e xploit\nprior knowledge about the real quality of the hypotheses to be combined.\nCompared to standard ROVER, the abs olute WER improvements in the two\nevaluation scenarios range from 0.5% to 7.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07440v2",
            "date": "2017-06-22 18:00:34+00:00",
            "title": "End-to-end Conversation Modeling Track in DSTC6",
            "authors": [
                "Chiori Hori",
                "Takaaki Hori"
            ],
            "abstract": "End-to-end training of neural networks is a promising approach to automatic\nconstruction of dialog systems using a human-to-human dialog corpus. Recently,\nVinyals et al. tested neural conversation models using OpenSubtitles. Lowe et\nal. released the Ubuntu Dialogue Corpus for researching unstructured multi-turn\ndialogue systems. Furthermore, the approach has been extended to accomplish\ntask oriented dialogs to provide information properly with natural\nconversation. For example, Ghazvininejad et al. proposed a knowledge grounded\nneural conversation model [3], where the research is aiming at combining\nconversational dialogs with task-oriented knowledge using unstructured data\nsuch as Twitter data for conversation and Foursquare data for external\nknowledge.However, the task is still limited to a restaurant information\nservice, and has not yet been tested with a wide variety of dialog tasks. In\naddition, it is still unclear how to create intelligent dialog systems that can\nrespond like a human agent.\n  In consideration of these problems, we proposed a challenge track to the 6th\ndialog system technology challenges (DSTC6) using human-to-human dialog data to\nmimic human dialog behaviors. The focus of the challenge track is to train\nend-to-end conversation models from human-to-human conversation and accomplish\nend-to-end dialog tasks in various situations assuming a customer service, in\nwhich a system plays a role of human agent and generates natural and\ninformative sentences in response to user's questions or comments given dialog\ncontext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07518v1",
            "date": "2017-06-22 22:54:25+00:00",
            "title": "Neural Machine Translation with Gumbel-Greedy Decoding",
            "authors": [
                "Jiatao Gu",
                "Daniel Jiwoong Im",
                "Victor O. K. Li"
            ],
            "abstract": "Previous neural machine translation models used some heuristic search\nalgorithms (e.g., beam search) in order to avoid solving the maximum a\nposteriori problem over translation sentences at test time. In this paper, we\npropose the Gumbel-Greedy Decoding which trains a generative network to predict\ntranslation under a trained model. We solve such a problem using the\nGumbel-Softmax reparameterization, which makes our generative network\ndifferentiable and trainable through standard stochastic gradient methods. We\nempirically demonstrate that our proposed model is effective for generating\nsequences of discrete words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07598v2",
            "date": "2017-06-23 08:33:38+00:00",
            "title": "Named Entity Recognition with stack residual LSTM and trainable bias decoding",
            "authors": [
                "Quan Tran",
                "Andrew MacKinlay",
                "Antonio Jimeno Yepes"
            ],
            "abstract": "Recurrent Neural Network models are the state-of-the-art for Named Entity\nRecognition (NER). We present two innovations to improve the performance of\nthese models. The first innovation is the introduction of residual connections\nbetween the Stacked Recurrent Neural Network model to address the degradation\nproblem of deep neural networks. The second innovation is a bias decoding\nmechanism that allows the trained system to adapt to non-differentiable and\nexternally computed objectives, such as the entity-based F-measure. Our work\nimproves the state-of-the-art results for both Spanish and English languages on\nthe standard train/development/test split of the CoNLL 2003 Shared Task NER\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07786v1",
            "date": "2017-06-23 17:43:20+00:00",
            "title": "Comparison of Modified Kneser-Ney and Witten-Bell Smoothing Techniques in Statistical Language Model of Bahasa Indonesia",
            "authors": [
                "Ismail Rusli"
            ],
            "abstract": "Smoothing is one technique to overcome data sparsity in statistical language\nmodel. Although in its mathematical definition there is no explicit dependency\nupon specific natural language, different natures of natural languages result\nin different effects of smoothing techniques. This is true for Russian language\nas shown by Whittaker (1998). In this paper, We compared Modified Kneser-Ney\nand Witten-Bell smoothing techniques in statistical language model of Bahasa\nIndonesia. We used train sets of totally 22M words that we extracted from\nIndonesian version of Wikipedia. As far as we know, this is the largest train\nset used to build statistical language model for Bahasa Indonesia. The\nexperiments with 3-gram, 5-gram, and 7-gram showed that Modified Kneser-Ney\nconsistently outperforms Witten-Bell smoothing technique in term of perplexity\nvalues. It is interesting to note that our experiments showed 5-gram model for\nModified Kneser-Ney smoothing technique outperforms that of 7-gram. Meanwhile,\nWitten-Bell smoothing is consistently improving over the increase of n-gram\norder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07905v1",
            "date": "2017-06-24 04:08:11+00:00",
            "title": "Encoder-Decoder Shift-Reduce Syntactic Parsing",
            "authors": [
                "Jiangming Liu",
                "Yue Zhang"
            ],
            "abstract": "Starting from NMT, encoder-decoder neu- ral networks have been used for many\nNLP problems. Graph-based models and transition-based models borrowing the en-\ncoder components achieve state-of-the-art performance on dependency parsing and\nconstituent parsing, respectively. How- ever, there has not been work\nempirically studying the encoder-decoder neural net- works for transition-based\nparsing. We apply a simple encoder-decoder to this end, achieving comparable\nresults to the parser of Dyer et al. (2015) on standard de- pendency parsing,\nand outperforming the parser of Vinyals et al. (2015) on con- stituent parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08032v1",
            "date": "2017-06-25 04:05:09+00:00",
            "title": "A Deep Neural Architecture for Sentence-level Sentiment Classification in Twitter Social Networking",
            "authors": [
                "Huy Nguyen",
                "Minh-Le Nguyen"
            ],
            "abstract": "This paper introduces a novel deep learning framework including a\nlexicon-based approach for sentence-level prediction of sentiment label\ndistribution. We propose to first apply semantic rules and then use a Deep\nConvolutional Neural Network (DeepCNN) for character-level embeddings in order\nto increase information for word-level embedding. After that, a Bidirectional\nLong Short-Term Memory Network (Bi-LSTM) produces a sentence-wide feature\nrepresentation from the word-level embedding. We evaluate our approach on three\nTwitter sentiment classification datasets. Experimental results show that our\nmodel can improve the classification accuracy of sentence-level sentiment\nanalysis in Twitter social networking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08162v1",
            "date": "2017-06-25 20:12:28+00:00",
            "title": "Automated text summarisation and evidence-based medicine: A survey of two domains",
            "authors": [
                "Abeed Sarker",
                "Diego Molla",
                "Cecile Paris"
            ],
            "abstract": "The practice of evidence-based medicine (EBM) urges medical practitioners to\nutilise the latest research evidence when making clinical decisions. Because of\nthe massive and growing volume of published research on various medical topics,\npractitioners often find themselves overloaded with information. As such,\nnatural language processing research has recently commenced exploring\ntechniques for performing medical domain-specific automated text summarisation\n(ATS) techniques-- targeted towards the task of condensing large medical texts.\nHowever, the development of effective summarisation techniques for this task\nrequires cross-domain knowledge. We present a survey of EBM, the\ndomain-specific needs for EBM, automated summarisation techniques, and how they\nhave been applied hitherto. We envision that this survey will serve as a first\nresource for the development of future operational text summarisation\ntechniques for EBM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08186v1",
            "date": "2017-06-25 23:10:26+00:00",
            "title": "Automatic Synonym Discovery with Knowledge Bases",
            "authors": [
                "Meng Qu",
                "Xiang Ren",
                "Jiawei Han"
            ],
            "abstract": "Recognizing entity synonyms from text has become a crucial task in many\nentity-leveraging applications. However, discovering entity synonyms from\ndomain-specific text corpora (e.g., news articles, scientific papers) is rather\nchallenging. Current systems take an entity name string as input to find out\nother names that are synonymous, ignoring the fact that often times a name\nstring can refer to multiple entities (e.g., \"apple\" could refer to both Apple\nInc and the fruit apple). Moreover, most existing methods require training data\nmanually created by domain experts to construct supervised-learning systems. In\nthis paper, we study the problem of automatic synonym discovery with knowledge\nbases, that is, identifying synonyms for knowledge base entities in a given\ndomain-specific corpus. The manually-curated synonyms for each entity stored in\na knowledge base not only form a set of name strings to disambiguate the\nmeaning for each other, but also can serve as \"distant\" supervision to help\ndetermine important features for the task. We propose a novel framework, called\nDPE, to integrate two kinds of mutually-complementing signals for synonym\ndiscovery, i.e., distributional features based on corpus-level statistics and\ntextual patterns based on local contexts. In particular, DPE jointly optimizes\nthe two kinds of signals in conjunction with distant supervision, so that they\ncan mutually enhance each other in the training stage. At the inference stage,\nboth signals will be utilized to discover synonyms for the given entities.\nExperimental results prove the effectiveness of the proposed framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08198v1",
            "date": "2017-06-26 00:55:04+00:00",
            "title": "English-Japanese Neural Machine Translation with Encoder-Decoder-Reconstructor",
            "authors": [
                "Yukio Matsumura",
                "Takayuki Sato",
                "Mamoru Komachi"
            ],
            "abstract": "Neural machine translation (NMT) has recently become popular in the field of\nmachine translation. However, NMT suffers from the problem of repeating or\nmissing words in the translation. To address this problem, Tu et al. (2017)\nproposed an encoder-decoder-reconstructor framework for NMT using\nback-translation. In this method, they selected the best forward translation\nmodel in the same manner as Bahdanau et al. (2015), and then trained a\nbi-directional translation model as fine-tuning. Their experiments show that it\noffers significant improvement in BLEU scores in Chinese-English translation\ntask. We confirm that our re-implementation also shows the same tendency and\nalleviates the problem of repeating and missing words in the translation on a\nEnglish-Japanese task too. In addition, we evaluate the effectiveness of\npre-training by comparing it with a jointly-trained model of forward\ntranslation and back-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08683v1",
            "date": "2017-06-27 06:33:52+00:00",
            "title": "Memory-augmented Chinese-Uyghur Neural Machine Translation",
            "authors": [
                "Shiyue Zhang",
                "Gulnigar Mahmut",
                "Dong Wang",
                "Askar Hamdulla"
            ],
            "abstract": "Neural machine translation (NMT) has achieved notable performance recently.\nHowever, this approach has not been widely applied to the translation task\nbetween Chinese and Uyghur, partly due to the limited parallel data resource\nand the large proportion of rare words caused by the agglutinative nature of\nUyghur. In this paper, we collect ~200,000 sentence pairs and show that with\nthis middle-scale database, an attention-based NMT can perform very well on\nChinese-Uyghur/Uyghur-Chinese translation. To tackle rare words, we propose a\nnovel memory structure to assist the NMT inference. Our experiments\ndemonstrated that the memory-augmented NMT (M-NMT) outperforms both the vanilla\nNMT and the phrase-based statistical machine translation (SMT). Interestingly,\nthe memory structure provides an elegant way for dealing with words that are\nout of vocabulary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09031v2",
            "date": "2017-06-27 20:02:34+00:00",
            "title": "CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages",
            "authors": [
                "Ryan Cotterell",
                "Christo Kirov",
                "John Sylak-Glassman",
                "G\u00e9raldine Walther",
                "Ekaterina Vylomova",
                "Patrick Xia",
                "Manaal Faruqui",
                "Sandra K\u00fcbler",
                "David Yarowsky",
                "Jason Eisner",
                "Mans Hulden"
            ],
            "abstract": "The CoNLL-SIGMORPHON 2017 shared task on supervised morphological generation\nrequired systems to be trained and tested in each of 52 typologically diverse\nlanguages. In sub-task 1, submitted systems were asked to predict a specific\ninflected form of a given lemma. In sub-task 2, systems were given a lemma and\nsome of its specific inflected forms, and asked to complete the inflectional\nparadigm by predicting all of the remaining inflected forms. Both sub-tasks\nincluded high, medium, and low-resource conditions. Sub-task 1 received 24\nsystem submissions, while sub-task 2 received 3 system submissions. Following\nthe success of neural sequence-to-sequence models in the SIGMORPHON 2016 shared\ntask, all but one of the submissions included a neural component. The results\nshow that high performance can be achieved with small training datasets, so\nlong as models have appropriate inductive bias or make use of additional\nunlabeled data or synthetic data. However, different biasing and data\naugmentation resulted in disjoint sets of inflected forms being predicted\ncorrectly, suggesting that there is room for future improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09147v2",
            "date": "2017-06-28 07:26:55+00:00",
            "title": "Named Entity Disambiguation for Noisy Text",
            "authors": [
                "Yotam Eshel",
                "Noam Cohen",
                "Kira Radinsky",
                "Shaul Markovitch",
                "Ikuya Yamada",
                "Omer Levy"
            ],
            "abstract": "We address the task of Named Entity Disambiguation (NED) for noisy text. We\npresent WikilinksNED, a large-scale NED dataset of text fragments from the web,\nwhich is significantly noisier and more challenging than existing news-based\ndatasets. To capture the limited and noisy local context surrounding each\nmention, we design a neural model and train it with a novel method for sampling\ninformative negative examples. We also describe a new way of initializing word\nand entity embeddings that significantly improves performance. Our model\nsignificantly outperforms existing state-of-the-art methods on WikilinksNED\nwhile achieving comparable performance on a smaller newswire dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09254v2",
            "date": "2017-06-28 12:38:53+00:00",
            "title": "The E2E Dataset: New Challenges For End-to-End Generation",
            "authors": [
                "Jekaterina Novikova",
                "Ond\u0159ej Du\u0161ek",
                "Verena Rieser"
            ],
            "abstract": "This paper describes the E2E data, a new dataset for training end-to-end,\ndata-driven natural language generation systems in the restaurant domain, which\nis ten times bigger than existing, frequently used datasets in this area. The\nE2E dataset poses new challenges: (1) its human reference texts show more\nlexical richness and syntactic variation, including discourse phenomena; (2)\ngenerating from this set requires content selection. As such, learning from\nthis dataset promises more natural, varied and less template-like system\nutterances. We also establish a baseline on this dataset, which illustrates\nsome of the difficulties associated with this data.",
            "journal ref": "Proceedings of the SIGDIAL 2017 Conference, pages 201-206,\n  Saarbr\\\"ucken, Germany, 15-17 August 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09335v1",
            "date": "2017-06-28 15:50:26+00:00",
            "title": "Generating Appealing Brand Names",
            "authors": [
                "Gaurush Hiranandani",
                "Pranav Maneriker",
                "Harsh Jhamtani"
            ],
            "abstract": "Providing appealing brand names to newly launched products, newly formed\ncompanies or for renaming existing companies is highly important as it can play\na crucial role in deciding its success or failure. In this work, we propose a\ncomputational method to generate appealing brand names based on the description\nof such entities. We use quantitative scores for readability, pronounceability,\nmemorability and uniqueness of the generated names to rank order them. A set of\ndiverse appealing names is recommended to the user for the brand naming task.\nExperimental results show that the names generated by our approach are more\nappealing than names which prior approaches and recruited humans could come up.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09433v1",
            "date": "2017-06-28 18:17:30+00:00",
            "title": "Data-driven Natural Language Generation: Paving the Road to Success",
            "authors": [
                "Jekaterina Novikova",
                "Ond\u0159ej Du\u0161ek",
                "Verena Rieser"
            ],
            "abstract": "We argue that there are currently two major bottlenecks to the commercial use\nof statistical machine learning approaches for natural language generation\n(NLG): (a) The lack of reliable automatic evaluation metrics for NLG, and (b)\nThe scarcity of high quality in-domain corpora. We address the first problem by\nthoroughly analysing current evaluation metrics and motivating the need for a\nnew, more reliable metric. The second problem is addressed by presenting a\nnovel framework for developing and evaluating a high quality corpus for NLG\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09453v2",
            "date": "2017-06-28 19:41:25+00:00",
            "title": "Toward Computation and Memory Efficient Neural Network Acoustic Models with Binary Weights and Activations",
            "authors": [
                "Liang Lu"
            ],
            "abstract": "Neural network acoustic models have significantly advanced state of the art\nspeech recognition over the past few years. However, they are usually\ncomputationally expensive due to the large number of matrix-vector\nmultiplications and nonlinearity operations. Neural network models also require\nsignificant amounts of memory for inference because of the large model size.\nFor these two reasons, it is challenging to deploy neural network based speech\nrecognizers on resource-constrained platforms such as embedded devices. This\npaper investigates the use of binary weights and activations for computation\nand memory efficient neural network acoustic models. Compared to real-valued\nweight matrices, binary weights require much fewer bits for storage, thereby\ncutting down the memory footprint. Furthermore, with binary weights or\nactivations, the matrix-vector multiplications are turned into addition and\nsubtraction operations, which are computationally much faster and more energy\nefficient for hardware platforms. In this paper, we study the applications of\nbinary weights and activations for neural network acoustic modeling, reporting\nencouraging results on the WSJ and AMI corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09528v1",
            "date": "2017-06-29 00:42:10+00:00",
            "title": "Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold",
            "authors": [
                "Swabha Swayamdipta",
                "Sam Thomson",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We present a new, efficient frame-semantic parser that labels semantic\narguments to FrameNet predicates. Built using an extension to the segmental RNN\nthat emphasizes recall, our basic system achieves competitive performance\nwithout any calls to a syntactic parser. We then introduce a method that uses\nphrase-syntactic annotations from the Penn Treebank during training only,\nthrough a multitask objective; no parsing is required at training or test time.\nThis \"syntactic scaffold\" offers a cheaper alternative to traditional syntactic\npipelining, and achieves state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09562v1",
            "date": "2017-06-29 03:19:39+00:00",
            "title": "Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles",
            "authors": [
                "Francis Ferraro",
                "Adam Poliak",
                "Ryan Cotterell",
                "Benjamin Van Durme"
            ],
            "abstract": "We study how different frame annotations complement one another when learning\ncontinuous lexical semantics. We learn the representations from a tensorized\nskip-gram model that consistently encodes syntactic-semantic content better,\nwith multiple 10% gains over baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09569v2",
            "date": "2017-06-29 03:53:55+00:00",
            "title": "Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition",
            "authors": [
                "Inigo Jauregi Unanue",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "Background. Previous state-of-the-art systems on Drug Name Recognition (DNR)\nand Clinical Concept Extraction (CCE) have focused on a combination of text\n\"feature engineering\" and conventional machine learning algorithms such as\nconditional random fields and support vector machines. However, developing good\nfeatures is inherently heavily time-consuming. Conversely, more modern machine\nlearning approaches such as recurrent neural networks (RNNs) have proved\ncapable of automatically learning effective features from either random\nassignments or automated word \"embeddings\". Objectives. (i) To create a highly\naccurate DNR and CCE system that avoids conventional, time-consuming feature\nengineering. (ii) To create richer, more specialized word embeddings by using\nhealth domain datasets such as MIMIC-III. (iii) To evaluate our systems over\nthree contemporary datasets. Methods. Two deep learning methods, namely the\nBidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model\nis set as the baseline to compare the deep learning systems to a traditional\nmachine learning approach. The same features are used for all the models.\nResults. We have obtained the best results with the Bidirectional LSTM-CRF\nmodel, which has outperformed all previously proposed systems. The specialized\nembeddings have helped to cover unusual words in DDI-DrugBank and DDI-MedLine,\nbut not in the 2010 i2b2/VA IRB Revision dataset. Conclusion. We present a\nstate-of-the-art system for DNR and CCE. Automated word embeddings has allowed\nus to avoid costly feature engineering and achieve higher accuracy.\nNevertheless, the embeddings need to be retrained over datasets that are\nadequate for the domain, in order to adequately cover the domain-specific\nvocabulary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09673v1",
            "date": "2017-06-29 10:54:43+00:00",
            "title": "Improving Distributed Representations of Tweets - Present and Future",
            "authors": [
                "Ganesh J"
            ],
            "abstract": "Unsupervised representation learning for tweets is an important research\nfield which helps in solving several business applications such as sentiment\nanalysis, hashtag prediction, paraphrase detection and microblog ranking. A\ngood tweet representation learning model must handle the idiosyncratic nature\nof tweets which poses several challenges such as short length, informal words,\nunusual grammar and misspellings. However, there is a lack of prior work which\nsurveys the representation learning models with a focus on tweets. In this\nwork, we organize the models based on its objective function which aids the\nunderstanding of the literature. We also provide interesting future directions,\nwhich we believe are fruitful in advancing this field by building high-quality\ntweet representation learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09733v1",
            "date": "2017-06-29 13:02:46+00:00",
            "title": "Stronger Baselines for Trustable Results in Neural Machine Translation",
            "authors": [
                "Michael Denkowski",
                "Graham Neubig"
            ],
            "abstract": "Interest in neural machine translation has grown rapidly as its effectiveness\nhas been demonstrated across language and data scenarios. New research\nregularly introduces architectural and algorithmic improvements that lead to\nsignificant gains over \"vanilla\" NMT implementations. However, these new\ntechniques are rarely evaluated in the context of previously published\ntechniques, specifically those that are widely used in state-of-theart\nproduction and shared-task systems. As a result, it is often difficult to\ndetermine whether improvements from research will carry over to systems\ndeployed for real-world use. In this work, we recommend three specific methods\nthat are relatively easy to implement and result in much stronger experimental\nsystems. Beyond reporting significantly higher BLEU scores, we conduct an\nin-depth analysis of where improvements originate and what inherent weaknesses\nof basic NMT models are being addressed. We then compare the relative gains\nafforded by several other techniques proposed in the literature when starting\nwith vanilla systems versus our stronger baselines, showing that experimental\nconclusions may change depending on the baseline chosen. This indicates that\nchoosing a strong baseline is crucial for reporting reliable experimental\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09742v1",
            "date": "2017-06-28 13:37:29+00:00",
            "title": "AP17-OLR Challenge: Data, Plan, and Baseline",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Yixiang Chen",
                "Qing Chen"
            ],
            "abstract": "We present the data profile and the evaluation plan of the second oriental\nlanguage recognition (OLR) challenge AP17-OLR. Compared to the event last year\n(AP16-OLR), the new challenge involves more languages and focuses more on short\nutterances. The data is offered by SpeechOcean and the NSFC M2ASR project. Two\ntypes of baselines are constructed to assist the participants, one is based on\nthe i-vector model and the other is based on various neural networks. We report\nthe baseline results evaluated with various metrics defined by the AP17-OLR\nevaluation plan and demonstrate that the combined database is a reasonable data\nresource for multilingual research. All the data is free for participants, and\nthe Kaldi recipes for the baselines have been published online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09789v3",
            "date": "2017-06-29 14:58:47+00:00",
            "title": "Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension",
            "authors": [
                "David Golub",
                "Po-Sen Huang",
                "Xiaodong He",
                "Li Deng"
            ],
            "abstract": "We develop a technique for transfer learning in machine comprehension (MC)\nusing a novel two-stage synthesis network (SynNet). Given a high-performing MC\nmodel in one domain, our technique aims to answer questions about documents in\nanother domain, where we use no labeled data of question-answer pairs. Using\nthe proposed SynNet with a pretrained model from the SQuAD dataset on the\nchallenging NewsQA dataset, we achieve an F1 measure of 44.3% with a single\nmodel and 46.6% with an ensemble, approaching performance of in-domain models\n(F1 measure of 50.0%) and outperforming the out-of-domain baseline of 7.6%,\nwithout use of provided annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09799v1",
            "date": "2017-06-29 15:14:07+00:00",
            "title": "Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation",
            "authors": [
                "Shikhar Sharma",
                "Layla El Asri",
                "Hannes Schulz",
                "Jeremie Zumer"
            ],
            "abstract": "Automated metrics such as BLEU are widely used in the machine translation\nliterature. They have also been used recently in the dialogue community for\nevaluating dialogue response generation. However, previous work in dialogue\nresponse generation has shown that these metrics do not correlate strongly with\nhuman judgment in the non task-oriented dialogue setting. Task-oriented\ndialogue responses are expressed on narrower domains and exhibit lower\ndiversity. It is thus reasonable to think that these automated metrics would\ncorrelate well with human judgment in the task-oriented setting where the\ngeneration task consists of translating dialogue acts into a sentence. We\nconduct an empirical study to confirm whether this is the case. Our findings\nindicate that these automated metrics have stronger correlation with human\njudgments in the task-oriented setting compared to what has been observed in\nthe non task-oriented setting. We also observe that these metrics correlate\neven better for datasets which provide multiple ground truth reference\nsentences. In addition, we show that some of the currently available corpora\nfor task-oriented language generation can be solved with simple models and\nadvocate for more challenging datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09856v1",
            "date": "2017-06-29 17:04:48+00:00",
            "title": "Automatic Mapping of French Discourse Connectives to PDTB Discourse Relations",
            "authors": [
                "Majid Laali",
                "Leila Kosseim"
            ],
            "abstract": "In this paper, we present an approach to exploit phrase tables generated by\nstatistical machine translation in order to map French discourse connectives to\ndiscourse relations. Using this approach, we created ConcoLeDisCo, a lexicon of\nFrench discourse connectives and their PDTB relations. When evaluated against\nLEXCONN, ConcoLeDisCo achieves a recall of 0.81 and an Average Precision of\n0.68 for the Concession and Condition relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00055v1",
            "date": "2017-07-31 20:12:06+00:00",
            "title": "SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation",
            "authors": [
                "Daniel Cer",
                "Mona Diab",
                "Eneko Agirre",
                "I\u00f1igo Lopez-Gazpio",
                "Lucia Specia"
            ],
            "abstract": "Semantic Textual Similarity (STS) measures the meaning similarity of\nsentences. Applications include machine translation (MT), summarization,\ngeneration, question answering (QA), short answer grading, semantic search,\ndialog and conversational systems. The STS shared task is a venue for assessing\nthe current state-of-the-art. The 2017 task focuses on multilingual and\ncross-lingual pairs with one sub-track exploring MT quality estimation (MTQE)\ndata. The task obtained strong participation from 31 teams, with 17\nparticipating in all language tracks. We summarize performance and review a\nselection of well performing methods. Analysis highlights common errors,\nproviding insight into the limitations of existing models. To support ongoing\nwork on semantic representations, the STS Benchmark is introduced as a new\nshared training and evaluation set carefully selected from the corpus of\nEnglish STS shared task data (2012-2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00098v1",
            "date": "2017-07-31 23:29:41+00:00",
            "title": "The Code2Text Challenge: Text Generation in Source Code Libraries",
            "authors": [
                "Kyle Richardson",
                "Sina Zarrie\u00df",
                "Jonas Kuhn"
            ],
            "abstract": "We propose a new shared task for tactical data-to-text generation in the\ndomain of source code libraries. Specifically, we focus on text generation of\nfunction descriptions from example software projects. Data is drawn from\nexisting resources used for studying the related problem of semantic parser\ninduction (Richardson and Kuhn, 2017b; Richardson and Kuhn, 2017a), and spans a\nwide variety of both natural languages and programming languages. In this\npaper, we describe these existing resources, which will serve as training and\ndevelopment data for the task, and discuss plans for building new independent\ntest sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00160v2",
            "date": "2017-08-01 05:09:34+00:00",
            "title": "Using Linguistic Features to Improve the Generalization Capability of Neural Coreference Resolvers",
            "authors": [
                "Nafise Sadat Moosavi",
                "Michael Strube"
            ],
            "abstract": "Coreference resolution is an intermediate step for text understanding. It is\nused in tasks and domains for which we do not necessarily have coreference\nannotated corpora. Therefore, generalization is of special importance for\ncoreference resolution. However, while recent coreference resolvers have\nnotable improvements on the CoNLL dataset, they struggle to generalize properly\nto new domains or datasets. In this paper, we investigate the role of\nlinguistic features in building more generalizable coreference resolvers. We\nshow that generalization improves only slightly by merely using a set of\nadditional linguistic features. However, employing features and subsets of\ntheir values that are informative for coreference resolution, considerably\nimproves generalization. Thanks to better generalization, our system achieves\nstate-of-the-art results in out-of-domain evaluations, e.g., on WikiCoref, our\nsystem, which is trained on CoNLL, achieves on-par performance with a system\ndesigned for this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00179v1",
            "date": "2017-08-01 06:35:24+00:00",
            "title": "An Investigation into the Pedagogical Features of Documents",
            "authors": [
                "Emily Sheng",
                "Prem Natarajan",
                "Jonathan Gordon",
                "Gully Burns"
            ],
            "abstract": "Characterizing the content of a technical document in terms of its learning\nutility can be useful for applications related to education, such as generating\nreading lists from large collections of documents. We refer to this learning\nutility as the \"pedagogical value\" of the document to the learner. While\npedagogical value is an important concept that has been studied extensively\nwithin the education domain, there has been little work exploring it from a\ncomputational, i.e., natural language processing (NLP), perspective. To allow a\ncomputational exploration of this concept, we introduce the notion of\n\"pedagogical roles\" of documents (e.g., Tutorial and Survey) as an intermediary\ncomponent for the study of pedagogical value. Given the lack of available\ncorpora for our exploration, we create the first annotated corpus of\npedagogical roles and use it to test baseline techniques for automatic\nprediction of such roles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00241v1",
            "date": "2017-08-01 10:56:17+00:00",
            "title": "Improving Part-of-Speech Tagging for NLP Pipelines",
            "authors": [
                "Vishaal Jatav",
                "Ravi Teja",
                "Srini Bharadwaj",
                "Venkat Srinivasan"
            ],
            "abstract": "This paper outlines the results of sentence level linguistics based rules for\nimproving part-of-speech tagging. It is well known that the performance of\ncomplex NLP systems is negatively affected if one of the preliminary stages is\nless than perfect. Errors in the initial stages in the pipeline have a\nsnowballing effect on the pipeline's end performance. We have created a set of\nlinguistics based rules at the sentence level which adjust part-of-speech tags\nfrom state-of-the-art taggers. Comparison with state-of-the-art taggers on\nwidely used benchmarks demonstrate significant improvements in tagging accuracy\nand consequently in the quality and accuracy of NLP systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00391v1",
            "date": "2017-08-01 15:41:51+00:00",
            "title": "A Continuously Growing Dataset of Sentential Paraphrases",
            "authors": [
                "Wuwei Lan",
                "Siyu Qiu",
                "Hua He",
                "Wei Xu"
            ],
            "abstract": "A major challenge in paraphrase research is the lack of parallel corpora. In\nthis paper, we present a new method to collect large-scale sentential\nparaphrases from Twitter by linking tweets through shared URLs. The main\nadvantage of our method is its simplicity, as it gets rid of the classifier or\nhuman in the loop needed to select data before annotation and subsequent\napplication of paraphrase identification algorithms in the previous work. We\npresent the largest human-labeled paraphrase corpus to date of 51,524 sentence\npairs and the first cross-domain benchmarking for automatic paraphrase\nidentification. In addition, we show that more than 30,000 new sentential\nparaphrases can be easily and continuously captured every month at ~70%\nprecision, and demonstrate their utility for downstream NLP tasks through\nphrasal paraphrase extraction. We make our code and data freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00415v2",
            "date": "2017-08-01 17:02:45+00:00",
            "title": "A Generative Parser with a Discriminative Recognition Algorithm",
            "authors": [
                "Jianpeng Cheng",
                "Adam Lopez",
                "Mirella Lapata"
            ],
            "abstract": "Generative models defining joint distributions over parse trees and sentences\nare useful for parsing and language modeling, but impose restrictions on the\nscope of features and are often outperformed by discriminative models. We\npropose a framework for parsing and language modeling which marries a\ngenerative model with a discriminative recognition model in an encoder-decoder\nsetting. We provide interpretations of the framework based on expectation\nmaximization and variational inference, and show that it enables parsing and\nlanguage modeling within a single implementation. On the English Penn\nTreen-bank, our framework obtains competitive performance on constituency\nparsing while matching the state-of-the-art single-model language modeling\nscore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00416v1",
            "date": "2017-08-01 17:05:32+00:00",
            "title": "Deriving Verb Predicates By Clustering Verbs with Arguments",
            "authors": [
                "Joao Sedoc",
                "Derry Wijaya",
                "Masoud Rouhizadeh",
                "Andy Schwartz",
                "Lyle Ungar"
            ],
            "abstract": "Hand-built verb clusters such as the widely used Levin classes (Levin, 1993)\nhave proved useful, but have limited coverage. Verb classes automatically\ninduced from corpus data such as those from VerbKB (Wijaya, 2016), on the other\nhand, can give clusters with much larger coverage, and can be adapted to\nspecific corpora such as Twitter. We present a method for clustering the\noutputs of VerbKB: verbs with their multiple argument types, e.g.\n\"marry(person, person)\", \"feel(person, emotion).\" We make use of a novel\nlow-dimensional embedding of verbs and their arguments to produce high quality\nclusters in which the same verb can be in different clusters depending on its\nargument type. The resulting verb clusters do a better job than hand-built\nclusters of predicting sarcasm, sentiment, and locus of control in tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00553v1",
            "date": "2017-08-02 00:05:10+00:00",
            "title": "Low-Rank Hidden State Embeddings for Viterbi Sequence Labeling",
            "authors": [
                "Dung Thai",
                "Shikhar Murty",
                "Trapit Bansal",
                "Luke Vilnis",
                "David Belanger",
                "Andrew McCallum"
            ],
            "abstract": "In textual information extraction and other sequence labeling tasks it is now\ncommon to use recurrent neural networks (such as LSTM) to form rich embedded\nrepresentations of long-term input co-occurrence patterns. Representation of\noutput co-occurrence patterns is typically limited to a hand-designed graphical\nmodel, such as a linear-chain CRF representing short-term Markov dependencies\namong successive labels. This paper presents a method that learns embedded\nrepresentations of latent output structure in sequence data. Our model takes\nthe form of a finite-state machine with a large number of latent states per\nlabel (a latent variable CRF), where the state-transition matrix is\nfactorized---effectively forming an embedded representation of\nstate-transitions capable of enforcing long-term label dependencies, while\nsupporting exact Viterbi inference over output labels. We demonstrate accuracy\nimprovements and interpretable latent structure in a synthetic but complex task\nbased on CoNLL named entity recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00563v1",
            "date": "2017-08-02 00:48:35+00:00",
            "title": "Analyzing Neural MT Search and Model Performance",
            "authors": [
                "Jan Niehues",
                "Eunah Cho",
                "Thanh-Le Ha",
                "Alex Waibel"
            ],
            "abstract": "In this paper, we offer an in-depth analysis about the modeling and search\nperformance. We address the question if a more complex search algorithm is\nnecessary. Furthermore, we investigate the question if more complex models\nwhich might only be applicable during rescoring are promising.\n  By separating the search space and the modeling using $n$-best list\nreranking, we analyze the influence of both parts of an NMT system\nindependently. By comparing differently performing NMT systems, we show that\nthe better translation is already in the search space of the translation\nsystems with less performance. This results indicate that the current search\nalgorithms are sufficient for the NMT systems. Furthermore, we could show that\neven a relatively small $n$-best list of $50$ hypotheses already contain\nnotably better translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00712v1",
            "date": "2017-08-02 11:55:57+00:00",
            "title": "Dynamic Data Selection for Neural Machine Translation",
            "authors": [
                "Marlies van der Wees",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "Intelligent selection of training data has proven a successful technique to\nsimultaneously increase training efficiency and translation performance for\nphrase-based machine translation (PBMT). With the recent increase in popularity\nof neural machine translation (NMT), we explore in this paper to what extent\nand how NMT can also benefit from data selection. While state-of-the-art data\nselection (Axelrod et al., 2011) consistently performs well for PBMT, we show\nthat gains are substantially lower for NMT. Next, we introduce dynamic data\nselection for NMT, a method in which we vary the selected subset of training\ndata between different training epochs. Our experiments show that the best\nresults are achieved when applying a technique we call gradual fine-tuning,\nwith improvements up to +2.6 BLEU over the original data selection approach and\nup to +3.1 BLEU over a general baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00726v1",
            "date": "2017-08-02 12:48:32+00:00",
            "title": "The University of Edinburgh's Neural MT Systems for WMT17",
            "authors": [
                "Rico Sennrich",
                "Alexandra Birch",
                "Anna Currey",
                "Ulrich Germann",
                "Barry Haddow",
                "Kenneth Heafield",
                "Antonio Valerio Miceli Barone",
                "Philip Williams"
            ],
            "abstract": "This paper describes the University of Edinburgh's submissions to the WMT17\nshared news translation and biomedical translation tasks. We participated in 12\ntranslation directions for news, translating between English and Czech, German,\nLatvian, Russian, Turkish and Chinese. For the biomedical task we submitted\nsystems for English to Czech, German, Polish and Romanian. Our systems are\nneural machine translation systems trained with Nematus, an attentional\nencoder-decoder. We follow our setup from last year and build BPE-based models\nwith parallel and back-translated monolingual training data. Novelties this\nyear include the use of deep architectures, layer normalization, and more\ncompact models due to weight tying and improvements in BPE segmentations. We\nperform extensive ablative experiments, reporting on the effectivenes of layer\nnormalization, deep architectures, and different ensembling techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00790v2",
            "date": "2017-08-02 15:10:28+00:00",
            "title": "Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition",
            "authors": [
                "Yong Jiang",
                "Wenjuan Han",
                "Kewei Tu"
            ],
            "abstract": "Unsupervised dependency parsing aims to learn a dependency parser from\nunannotated sentences. Existing work focuses on either learning generative\nmodels using the expectation-maximization algorithm and its variants, or\nlearning discriminative models using the discriminative clustering algorithm.\nIn this paper, we propose a new learning strategy that learns a generative\nmodel and a discriminative model jointly based on the dual decomposition\nmethod. Our method is simple and general, yet effective to capture the\nadvantages of both models and improve their learning results. We tested our\nmethod on the UD treebank and achieved a state-of-the-art performance on thirty\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00801v1",
            "date": "2017-08-02 15:43:30+00:00",
            "title": "Dependency Grammar Induction with Neural Lexicalization and Big Training Data",
            "authors": [
                "Wenjuan Han",
                "Yong Jiang",
                "Kewei Tu"
            ],
            "abstract": "We study the impact of big models (in terms of the degree of lexicalization)\nand big data (in terms of the training corpus size) on dependency grammar\ninduction. We experimented with L-DMV, a lexicalized version of Dependency\nModel with Valence and L-NDMV, our lexicalized extension of the Neural\nDependency Model with Valence. We find that L-DMV only benefits from very small\ndegrees of lexicalization and moderate sizes of training corpora. L-NDMV can\nbenefit from big training data and lexicalization of greater degrees,\nespecially when enhanced with good model initialization, and it achieves a\nresult that is competitive with the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00818v1",
            "date": "2017-08-02 16:51:01+00:00",
            "title": "Enterprise to Computer: Star Trek chatbot",
            "authors": [
                "Grishma Jena",
                "Mansi Vashisht",
                "Abheek Basu",
                "Lyle Ungar",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Human interactions and human-computer interactions are strongly influenced by\nstyle as well as content. Adding a persona to a chatbot makes it more\nhuman-like and contributes to a better and more engaging user experience. In\nthis work, we propose a design for a chatbot that captures the \"style\" of Star\nTrek by incorporating references from the show along with peculiar tones of the\nfictional characters therein. Our Enterprise to Computer bot (E2Cbot) treats\nStar Trek dialog style and general dialog style differently, using two\nrecurrent neural network Encoder-Decoder models. The Star Trek dialog style\nuses sequence to sequence (SEQ2SEQ) models (Sutskever et al., 2014; Bahdanau et\nal., 2014) trained on Star Trek dialogs. The general dialog style uses Word\nGraph to shift the response of the SEQ2SEQ model into the Star Trek domain. We\nevaluate the bot both in terms of perplexity and word overlap with Star Trek\nvocabulary and subjectively using human evaluators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00850v1",
            "date": "2017-08-02 17:54:32+00:00",
            "title": "Towards Semantic Modeling of Contradictions and Disagreements: A Case Study of Medical Guidelines",
            "authors": [
                "Wlodek Zadrozny",
                "Hossein Hematialam",
                "Luciana Garbayo"
            ],
            "abstract": "We introduce a formal distinction between contradictions and disagreements in\nnatural language texts, motivated by the need to formally reason about\ncontradictory medical guidelines. This is a novel and potentially very useful\ndistinction, and has not been discussed so far in NLP and logic. We also\ndescribe a NLP system capable of automated finding contradictory medical\nguidelines; the system uses a combination of text analysis and information\nretrieval modules. We also report positive evaluation results on a small corpus\nof contradictory medical recommendations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00897v1",
            "date": "2017-08-02 19:04:52+00:00",
            "title": "Domain Aware Neural Dialog System",
            "authors": [
                "Sajal Choudhary",
                "Prerna Srivastava",
                "Lyle Ungar",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "We investigate the task of building a domain aware chat system which\ngenerates intelligent responses in a conversation comprising of different\ndomains. The domain, in this case, is the topic or theme of the conversation.\nTo achieve this, we present DOM-Seq2Seq, a domain aware neural network model\nbased on the novel technique of using domain-targeted sequence-to-sequence\nmodels (Sutskever et al., 2014) and a domain classifier. The model captures\nfeatures from current utterance and domains of the previous utterances to\nfacilitate the formation of relevant responses. We evaluate our model on\nautomatic metrics and compare our performance with the Seq2Seq model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00993v1",
            "date": "2017-08-03 04:30:37+00:00",
            "title": "Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning",
            "authors": [
                "Jan Niehues",
                "Eunah Cho"
            ],
            "abstract": "Linguistic resources such as part-of-speech (POS) tags have been extensively\nused in statistical machine translation (SMT) frameworks and have yielded\nbetter performances. However, usage of such linguistic annotations in neural\nmachine translation (NMT) systems has been left under-explored.\n  In this work, we show that multi-task learning is a successful and a easy\napproach to introduce an additional knowledge into an end-to-end neural\nattentional model. By jointly training several natural language processing\n(NLP) tasks in one system, we are able to leverage common information and\nimprove the performance of the individual task.\n  We analyze the impact of three design decisions in multi-task learning: the\ntasks used in training, the training schedule, and the degree of parameter\nsharing across the tasks, which is defined by the network architecture. The\nexperiments are conducted for an German to English translation task. As\nadditional linguistic resources, we exploit POS information and named-entities\n(NE). Experiments show that the translation quality can be improved by up to\n1.5 BLEU points under the low-resource condition. The performance of the POS\ntagger is also improved using the multi-task learning scheme.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01018v1",
            "date": "2017-08-03 06:45:31+00:00",
            "title": "CRF Autoencoder for Unsupervised Dependency Parsing",
            "authors": [
                "Jiong Cai",
                "Yong Jiang",
                "Kewei Tu"
            ],
            "abstract": "Unsupervised dependency parsing, which tries to discover linguistic\ndependency structures from unannotated data, is a very challenging task. Almost\nall previous work on this task focuses on learning generative models. In this\npaper, we develop an unsupervised dependency parsing model based on the CRF\nautoencoder. The encoder part of our model is discriminative and globally\nnormalized which allows us to use rich features as well as universal linguistic\npriors. We propose an exact algorithm for parsing as well as a tractable\nlearning algorithm. We evaluated the performance of our model on eight\nmultilingual treebanks and found that our model achieved comparable performance\nwith state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01353v1",
            "date": "2017-08-04 01:55:18+00:00",
            "title": "Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference",
            "authors": [
                "Qian Chen",
                "Xiaodan Zhu",
                "Zhen-Hua Ling",
                "Si Wei",
                "Hui Jiang",
                "Diana Inkpen"
            ],
            "abstract": "The RepEval 2017 Shared Task aims to evaluate natural language understanding\nmodels for sentence representation, in which a sentence is represented as a\nfixed-length vector with neural networks and the quality of the representation\nis tested with a natural language inference task. This paper describes our\nsystem (alpha) that is ranked among the top in the Shared Task, on both the\nin-domain test set (obtaining a 74.9% accuracy) and on the cross-domain test\nset (also attaining a 74.9% accuracy), demonstrating that the model generalizes\nwell to the cross-domain data. Our model is equipped with intra-sentence\ngated-attention composition which helps achieve a better performance. In\naddition to submitting our model to the Shared Task, we have also tested it on\nthe Stanford Natural Language Inference (SNLI) dataset. We obtain an accuracy\nof 85.5%, which is the best reported result on SNLI when cross-sentence\nattention is not allowed, the same condition enforced in RepEval 2017.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01464v1",
            "date": "2017-08-04 11:57:02+00:00",
            "title": "Massively Multilingual Neural Grapheme-to-Phoneme Conversion",
            "authors": [
                "Ben Peters",
                "Jon Dehdari",
                "Josef van Genabith"
            ],
            "abstract": "Grapheme-to-phoneme conversion (g2p) is necessary for text-to-speech and\nautomatic speech recognition systems. Most g2p systems are monolingual: they\nrequire language-specific data or handcrafting of rules. Such systems are\ndifficult to extend to low resource languages, for which data and handcrafted\nrules are not available. As an alternative, we present a neural\nsequence-to-sequence approach to g2p which is trained on\nspelling--pronunciation pairs in hundreds of languages. The system shares a\nsingle encoder and decoder across all languages, allowing it to utilize the\nintrinsic similarities between different writing systems. We show an 11%\nimprovement in phoneme error rate over an approach based on adapting\nhigh-resource monolingual g2p models to low-resource languages. Our model is\nalso much more compact relative to previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01681v1",
            "date": "2017-08-04 23:10:31+00:00",
            "title": "Predicting the Law Area and Decisions of French Supreme Court Cases",
            "authors": [
                "Octavia-Maria Sulea",
                "Marcos Zampieri",
                "Mihaela Vela",
                "Josef van Genabith"
            ],
            "abstract": "In this paper, we investigate the application of text classification methods\nto predict the law area and the decision of cases judged by the French Supreme\nCourt. We also investigate the influence of the time period in which a ruling\nwas made over the textual form of the case description and the extent to which\nit is necessary to mask the judge's motivation for a ruling to emulate a\nreal-world test scenario. We report results of 96% f1 score in predicting a\ncase ruling, 90% f1 score in predicting the law area of a case, and 75.9% f1\nscore in estimating the time span when a ruling has been issued using a linear\nSupport Vector Machine (SVM) classifier trained on lexical features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01713v1",
            "date": "2017-08-05 05:50:44+00:00",
            "title": "Automatic Question-Answering Using A Deep Similarity Neural Network",
            "authors": [
                "Shervin Minaee",
                "Zhu Liu"
            ],
            "abstract": "Automatic question-answering is a classical problem in natural language\nprocessing, which aims at designing systems that can automatically answer a\nquestion, in the same way as human does. In this work, we propose a deep\nlearning based model for automatic question-answering. First the questions and\nanswers are embedded using neural probabilistic modeling. Then a deep\nsimilarity neural network is trained to find the similarity score of a pair of\nanswer and question. Then for each question, the best answer is found as the\none with the highest similarity score. We first train this model on a\nlarge-scale public question-answering database, and then fine-tune it to\ntransfer to the customer-care chat data. We have also tested our framework on a\npublic question-answering database and achieved very good performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01759v1",
            "date": "2017-08-05 12:24:04+00:00",
            "title": "Referenceless Quality Estimation for Natural Language Generation",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Jekaterina Novikova",
                "Verena Rieser"
            ],
            "abstract": "Traditional automatic evaluation measures for natural language generation\n(NLG) use costly human-authored references to estimate the quality of a system\noutput. In this paper, we propose a referenceless quality estimation (QE)\napproach based on recurrent neural networks, which predicts a quality score for\na NLG system output by comparing it to the source meaning representation only.\nOur method outperforms traditional metrics and a constant baseline in most\nrespects; we also show that synthetic data helps to increase correlation\nresults by 21% compared to the base system. Our results are comparable to\nresults obtained in similar QE tasks despite the more challenging setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01766v1",
            "date": "2017-08-05 13:21:01+00:00",
            "title": "A Syllable-based Technique for Word Embeddings of Korean Words",
            "authors": [
                "Sanghyuk Choi",
                "Taeuk Kim",
                "Jinseok Seol",
                "Sang-goo Lee"
            ],
            "abstract": "Word embedding has become a fundamental component to many NLP tasks such as\nnamed entity recognition and machine translation. However, popular models that\nlearn such embeddings are unaware of the morphology of words, so it is not\ndirectly applicable to highly agglutinative languages such as Korean. We\npropose a syllable-based learning model for Korean using a convolutional neural\nnetwork, in which word representation is composed of trained syllable vectors.\nOur model successfully produces morphologically meaningful representation of\nKorean words compared to the original Skip-gram embeddings. The results also\nshow that it is quite robust to the Out-of-Vocabulary problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01769v1",
            "date": "2017-08-05 13:32:58+00:00",
            "title": "Extractive Multi Document Summarization using Dynamical Measurements of Complex Networks",
            "authors": [
                "Jorge V. Tohalino",
                "Diego R. Amancio"
            ],
            "abstract": "Due to the large amount of textual information available on Internet, it is\nof paramount relevance to use techniques that find relevant and concise\ncontent. A typical task devoted to the identification of informative sentences\nin documents is the so called extractive document summarization task. In this\npaper, we use complex network concepts to devise an extractive Multi Document\nSummarization (MDS) method, which extracts the most central sentences from\nseveral textual sources. In the proposed model, texts are represented as\nnetworks, where nodes represent sentences and the edges are established based\non the number of shared words. Differently from previous works, the\nidentification of relevant terms is guided by the characterization of nodes via\ndynamical measurements of complex networks, including symmetry, accessibility\nand absorption time. The evaluation of the proposed system revealed that\nexcellent results were obtained with particular dynamical measurements,\nincluding those based on the exploration of networks via random walks.",
            "journal ref": "2017 Brazilian Conference on Intelligent Systems (BRACIS),\n  Uberlandia, 2017, pp. 366-371",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01771v1",
            "date": "2017-08-05 13:38:10+00:00",
            "title": "Neural Machine Translation with Word Predictions",
            "authors": [
                "Rongxiang Weng",
                "Shujian Huang",
                "Zaixiang Zheng",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "In the encoder-decoder architecture for neural machine translation (NMT), the\nhidden states of the recurrent structures in the encoder and decoder carry the\ncrucial information about the sentence.These vectors are generated by\nparameters which are updated by back-propagation of translation errors through\ntime. We argue that propagating errors through the end-to-end recurrent\nstructures are not a direct way of control the hidden vectors. In this paper,\nwe propose to use word predictions as a mechanism for direct supervision. More\nspecifically, we require these vectors to be able to predict the vocabulary in\ntarget sentence. Our simple mechanism ensures better representations in the\nencoder and decoder without using any extra data or annotation. It is also\nhelpful in reducing the target side vocabulary and improving the decoding\nefficiency. Experiments on Chinese-English and German-English machine\ntranslation tasks show BLEU improvements by 4.53 and 1.3, respectively",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01809v1",
            "date": "2017-08-05 19:06:00+00:00",
            "title": "A Comparison of Neural Models for Word Ordering",
            "authors": [
                "Eva Hasler",
                "Felix Stahlberg",
                "Marcus Tomalin",
                "Adri`a de Gispert",
                "Bill Byrne"
            ],
            "abstract": "We compare several language models for the word-ordering task and propose a\nnew bag-to-sequence neural model based on attention-based sequence-to-sequence\nmodels. We evaluate the model on a large German WMT data set where it\nsignificantly outperforms existing models. We also describe a novel search\nstrategy for LM-based word ordering and report results on the English Penn\nTreebank. Our best model setup outperforms prior work both in terms of speed\nand quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01980v1",
            "date": "2017-08-07 03:46:48+00:00",
            "title": "Translating Phrases in Neural Machine Translation",
            "authors": [
                "Xing Wang",
                "Zhaopeng Tu",
                "Deyi Xiong",
                "Min Zhang"
            ],
            "abstract": "Phrases play an important role in natural language understanding and machine\ntranslation (Sag et al., 2002; Villavicencio et al., 2005). However, it is\ndifficult to integrate them into current neural machine translation (NMT) which\nreads and generates sentences word by word. In this work, we propose a method\nto translate phrases in NMT by integrating a phrase memory storing target\nphrases from a phrase-based statistical machine translation (SMT) system into\nthe encoder-decoder architecture of NMT. At each decoding step, the phrase\nmemory is first re-written by the SMT model, which dynamically generates\nrelevant target phrases with contextual information provided by the NMT model.\nThen the proposed model reads the phrase memory to make probability estimations\nfor all phrases in the phrase memory. If phrase generation is carried on, the\nNMT decoder selects an appropriate phrase from the memory to perform phrase\ntranslation and updates its decoding state by consuming the words in the\nselected phrase. Otherwise, the NMT decoder generates a word from the\nvocabulary as the general NMT decoder does. Experiment results on the Chinese\nto English translation show that the proposed model achieves significant\nimprovements over the baseline on various test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02005v1",
            "date": "2017-08-07 06:47:23+00:00",
            "title": "Memory-augmented Neural Machine Translation",
            "authors": [
                "Yang Feng",
                "Shiyue Zhang",
                "Andi Zhang",
                "Dong Wang",
                "Andrew Abel"
            ],
            "abstract": "Neural machine translation (NMT) has achieved notable success in recent\ntimes, however it is also widely recognized that this approach has limitations\nwith handling infrequent words and word pairs. This paper presents a novel\nmemory-augmented NMT (M-NMT) architecture, which stores knowledge about how\nwords (usually infrequently encountered ones) should be translated in a memory\nand then utilizes them to assist the neural model. We use this memory mechanism\nto combine the knowledge learned from a conventional statistical machine\ntranslation system and the rules learned by an NMT system, and also propose a\nsolution for out-of-vocabulary (OOV) words based on this framework. Our\nexperiments on two Chinese-English translation tasks demonstrated that the\nM-NMT architecture outperformed the NMT baseline by $9.0$ and $2.7$ BLEU points\non the two tasks, respectively. Additionally, we found this architecture\nresulted in a much more effective OOV treatment compared to competitive\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02267v2",
            "date": "2017-08-07 19:00:25+00:00",
            "title": "ISS-MULT: Intelligent Sample Selection for Multi-Task Learning in Question Answering",
            "authors": [
                "Ali Ahmadvand",
                "Jinho D. Choi"
            ],
            "abstract": "Transferring knowledge from a source domain to another domain is useful,\nespecially when gathering new data is very expensive and time-consuming. Deep\nnetworks have been well-studied for question answering tasks in recent years;\nhowever, no prominent research for transfer learning through deep neural\nnetworks exists in the question answering field. In this paper, two main\nmethods (INIT and MULT) in this field are examined. Then, a new method named\nIntelligent sample selection (ISS-MULT) is proposed to improve the MULT method\nfor question answering tasks. Different datasets, specificay SQuAD, SelQA,\nWikiQA, NewWikiQA and InforBoxQA, are used for evaluation. Moreover, two\ndifferent tasks of question answering - answer selection and answer triggering\n- are evaluated to examine the effectiveness of transfer learning. The results\nshow that using transfer learning generally improves the performance if the\ncorpora are related and are based on the same policy. In addition, using\nISS-MULT could finely improve the MULT method for question answering tasks, and\nthese improvements prove more significant in the answer triggering task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02275v2",
            "date": "2017-08-07 19:41:42+00:00",
            "title": "Corpus-level Fine-grained Entity Typing",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper addresses the problem of corpus-level entity typing, i.e.,\ninferring from a large corpus that an entity is a member of a class such as\n\"food\" or \"artist\". The application of entity typing we are interested in is\nknowledge base completion, specifically, to learn which classes an entity is a\nmember of. We propose FIGMENT to tackle this problem. FIGMENT is embedding-\nbased and combines (i) a global model that scores based on aggregated\ncontextual information of an entity and (ii) a context model that first scores\nthe individual occurrences of an entity and then aggregates the scores. Each of\nthe two proposed models has some specific properties. For the global model,\nlearning high quality entity representations is crucial because it is the only\nsource used for the predictions. Therefore, we introduce representations using\nname and contexts of entities on the three levels of entity, word, and\ncharacter. We show each has complementary information and a multi-level\nrepresentation is the best. For the context model, we need to use distant\nsupervision since the context-level labels are not available for entities.\nDistant supervised labels are noisy and this harms the performance of models.\nTherefore, we introduce and apply new algorithms for noise mitigation using\nmulti-instance learning. We show the effectiveness of our models in a large\nentity typing dataset, built from Freebase.",
            "journal ref": "JAIR, Vol 61 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02420v1",
            "date": "2017-08-08 09:27:55+00:00",
            "title": "Mining fine-grained opinions on closed captions of YouTube videos with an attention-RNN",
            "authors": [
                "Edison Marrese-Taylor",
                "Jorge A. Balazs",
                "Yutaka Matsuo"
            ],
            "abstract": "Video reviews are the natural evolution of written product reviews. In this\npaper we target this phenomenon and introduce the first dataset created from\nclosed captions of YouTube product review videos as well as a new attention-RNN\nmodel for aspect extraction and joint aspect extraction and sentiment\nclassification. Our model provides state-of-the-art performance on aspect\nextraction without requiring the usage of hand-crafted features on the SemEval\nABSA corpus, while it outperforms the baseline on the joint task. In our\ndataset, the attention-RNN model outperforms the baseline for both tasks, but\nwe observe important performance drops for all models in comparison to SemEval.\nThese results, as well as further experiments on domain adaptation for aspect\nextraction, suggest that differences between speech and written text, which\nhave been discussed extensively in the literature, also extend to the domain of\nproduct reviews, where they are relevant for fine-grained opinion mining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02561v1",
            "date": "2017-08-08 17:03:25+00:00",
            "title": "Neural-based Context Representation Learning for Dialog Act Classification",
            "authors": [
                "Daniel Ortega",
                "Ngoc Thang Vu"
            ],
            "abstract": "We explore context representation learning methods in neural-based models for\ndialog act classification. We propose and compare extensively different methods\nwhich combine recurrent neural network architectures and attention mechanisms\n(AMs) at different context levels. Our experimental results on two benchmark\ndatasets show consistent improvements compared to the models without contextual\ninformation and reveal that the most suitable AM in the architecture depends on\nthe nature of the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02709v8",
            "date": "2017-08-09 04:02:17+00:00",
            "title": "Recent Trends in Deep Learning Based Natural Language Processing",
            "authors": [
                "Tom Young",
                "Devamanyu Hazarika",
                "Soujanya Poria",
                "Erik Cambria"
            ],
            "abstract": "Deep learning methods employ multiple processing layers to learn hierarchical\nrepresentations of data and have produced state-of-the-art results in many\ndomains. Recently, a variety of model designs and methods have blossomed in the\ncontext of natural language processing (NLP). In this paper, we review\nsignificant deep learning related models and methods that have been employed\nfor numerous NLP tasks and provide a walk-through of their evolution. We also\nsummarize, compare and contrast the various models and put forward a detailed\nunderstanding of the past, present and future of deep learning in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02989v1",
            "date": "2017-08-09 19:58:55+00:00",
            "title": "Identifying Reference Spans: Topic Modeling and Word Embeddings help IR",
            "authors": [
                "Luis Moraes",
                "Shahryar Baki",
                "Rakesh Verma",
                "Daniel Lee"
            ],
            "abstract": "The CL-SciSumm 2016 shared task introduced an interesting problem: given a\ndocument D and a piece of text that cites D, how do we identify the text spans\nof D being referenced by the piece of text? The shared task provided the first\nannotated dataset for studying this problem. We present an analysis of our\ncontinued work in improving our system's performance on this task. We\ndemonstrate how topic models and word embeddings can be used to surpass the\npreviously best performing system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03105v3",
            "date": "2017-08-10 07:43:13+00:00",
            "title": "Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical Language Models",
            "authors": [
                "Hussein S. Al-Olimat",
                "Krishnaprasad Thirunarayan",
                "Valerie Shalin",
                "Amit Sheth"
            ],
            "abstract": "Extracting location names from informal and unstructured social media data\nrequires the identification of referent boundaries and partitioning compound\nnames. Variability, particularly systematic variability in location names\n(Carroll, 1983), challenges the identification task. Some of this variability\ncan be anticipated as operations within a statistical language model, in this\ncase drawn from gazetteers such as OpenStreetMap (OSM), Geonames, and DBpedia.\nThis permits evaluation of an observed n-gram in Twitter targeted text as a\nlegitimate location name variant from the same location-context. Using n-gram\nstatistics and location-related dictionaries, our Location Name Extraction tool\n(LNEx) handles abbreviations and automatically filters and augments the\nlocation names in gazetteers (handling name contractions and auxiliary\ncontents) to help detect the boundaries of multi-word location names and\nthereby delimit them in texts.\n  We evaluated our approach on 4,500 event-specific tweets from three targeted\nstreams to compare the performance of LNEx against that of ten state-of-the-art\ntaggers that rely on standard semantic, syntactic and/or orthographic features.\nLNEx improved the average F-Score by 33-179%, outperforming all taggers.\nFurther, LNEx is capable of stream processing.",
            "journal ref": "In The 27th International Conference on Computational Linguistics\n  (COLING 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03152v2",
            "date": "2017-08-10 10:21:31+00:00",
            "title": "Towards Neural Speaker Modeling in Multi-Party Conversation: The Task, Dataset, and Models",
            "authors": [
                "Zhao Meng",
                "Lili Mou",
                "Zhi Jin"
            ],
            "abstract": "Neural network-based dialog systems are attracting increasing attention in\nboth academia and industry. Recently, researchers have begun to realize the\nimportance of speaker modeling in neural dialog systems, but there lacks\nestablished tasks and datasets. In this paper, we propose speaker\nclassification as a surrogate task for general speaker modeling, and collect\nmassive data to facilitate research in this direction. We further investigate\ntemporal-based and content-based models of speakers, and propose several\nhybrids of them. Experiments show that speaker classification is feasible, and\nthat hybrid models outperform each single component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03186v1",
            "date": "2017-08-10 12:48:07+00:00",
            "title": "Neural and Statistical Methods for Leveraging Meta-information in Machine Translation",
            "authors": [
                "Shahram Khadivi",
                "Patrick Wilken",
                "Leonard Dahlmann",
                "Evgeny Matusov"
            ],
            "abstract": "In this paper, we discuss different methods which use meta information and\nricher context that may accompany source language input to improve machine\ntranslation quality. We focus on category information of input text as meta\ninformation, but the proposed methods can be extended to all textual and\nnon-textual meta information that might be available for the input text or\nautomatically predicted using the text content. The main novelty of this work\nis to use state-of-the-art neural network methods to tackle this problem within\na statistical machine translation (SMT) framework. We observe translation\nquality improvements up to 3% in terms of BLEU score in some text categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03271v1",
            "date": "2017-08-10 15:48:33+00:00",
            "title": "Neural Machine Translation Leveraging Phrase-based Models in a Hybrid Search",
            "authors": [
                "Leonard Dahlmann",
                "Evgeny Matusov",
                "Pavel Petrushkov",
                "Shahram Khadivi"
            ],
            "abstract": "In this paper, we introduce a hybrid search for attention-based neural\nmachine translation (NMT). A target phrase learned with statistical MT models\nextends a hypothesis in the NMT beam search when the attention of the NMT model\nfocuses on the source words translated by this phrase. Phrases added in this\nway are scored with the NMT model, but also with SMT features including\nphrase-level translation probabilities and a target language model.\nExperimental results on German->English news domain and English->Russian\ne-commerce domain translation tasks show that using phrase-based models in NMT\nsearch improves MT quality by up to 2.3% BLEU absolute as compared to a strong\nNMT baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03312v1",
            "date": "2017-08-10 17:46:28+00:00",
            "title": "Radical-level Ideograph Encoder for RNN-based Sentiment Analysis of Chinese and Japanese",
            "authors": [
                "Yuanzhi Ke",
                "Masafumi Hagiwara"
            ],
            "abstract": "The character vocabulary can be very large in non-alphabetic languages such\nas Chinese and Japanese, which makes neural network models huge to process such\nlanguages. We explored a model for sentiment classification that takes the\nembeddings of the radicals of the Chinese characters, i.e, hanzi of Chinese and\nkanji of Japanese. Our model is composed of a CNN word feature encoder and a\nbi-directional RNN document feature encoder. The results achieved are on par\nwith the character embedding-based models, and close to the state-of-the-art\nword embedding-based models, with 90% smaller vocabulary, and at least 13% and\n80% fewer parameters than the character embedding-based models and word\nembedding-based models respectively. The results suggest that the radical\nembedding-based approach is cost-effective for machine learning on Chinese and\nJapanese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03390v1",
            "date": "2017-08-10 21:27:57+00:00",
            "title": "Making Sense of Word Embeddings",
            "authors": [
                "Maria Pelevina",
                "Nikolay Arefyev",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We present a simple yet effective approach for learning word sense\nembeddings. In contrast to existing techniques, which either directly learn\nsense representations from corpora or rely on sense inventories from lexical\nresources, our approach can induce a sense inventory from existing word\nembeddings via clustering of ego-networks of related words. An integrated WSD\nmechanism enables labeling of words in context with learned sense vectors,\nwhich gives rise to downstream applications. Experiments show that the\nperformance of our method is comparable to state-of-the-art unsupervised WSD\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03421v1",
            "date": "2017-08-11 02:27:26+00:00",
            "title": "N-gram and Neural Language Models for Discriminating Similar Languages",
            "authors": [
                "Andre Cianflone",
                "Leila Kosseim"
            ],
            "abstract": "This paper describes our submission (named clac) to the 2016 Discriminating\nSimilar Languages (DSL) shared task. We participated in the closed Sub-task 1\n(Set A) with two separate machine learning techniques. The first approach is a\ncharacter based Convolution Neural Network with a bidirectional long short term\nmemory (BiLSTM) layer (CLSTM), which achieved an accuracy of 78.45% with\nminimal tuning. The second approach is a character-based n-gram model. This\nlast approach achieved an accuracy of 88.45% which is close to the accuracy of\n89.38% achieved by the best submission, and allowed us to rank #7 overall.",
            "journal ref": "Proceedings of the Third Workshop on NLP for Similar Languages,\n  Varieties and Dialects (VarDial3). A workshop of the 26th International\n  Conference on Computational Linguistics (COLING 2016, Osaka, Japan), pp\n  243-250 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03425v2",
            "date": "2017-08-11 03:46:31+00:00",
            "title": "Argument Labeling of Explicit Discourse Relations using LSTM Neural Networks",
            "authors": [
                "Sohail Hooda",
                "Leila Kosseim"
            ],
            "abstract": "Argument labeling of explicit discourse relations is a challenging task. The\nstate of the art systems achieve slightly above 55% F-measure but require\nhand-crafted features. In this paper, we propose a Long Short Term Memory\n(LSTM) based model for argument labeling. We experimented with multiple\nconfigurations of our model. Using the PDTB dataset, our best model achieved an\nF1 measure of 23.05% without any feature engineering. This is significantly\nhigher than the 20.52% achieved by the state of the art RNN approach, but\nsignificantly lower than the feature based state of the art systems. On the\nother hand, because our approach learns only from the raw dataset, it is more\nwidely applicable to multiple textual genres and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03446v2",
            "date": "2017-08-11 06:21:22+00:00",
            "title": "What matters in a transferable neural network model for relation classification in the biomedical domain?",
            "authors": [
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Lack of sufficient labeled data often limits the applicability of advanced\nmachine learning algorithms to real life problems. However efficient use of\nTransfer Learning (TL) has been shown to be very useful across domains. TL\nutilizes valuable knowledge learned in one task (source task), where sufficient\ndata is available, to the task of interest (target task). In biomedical and\nclinical domain, it is quite common that lack of sufficient training data do\nnot allow to fully exploit machine learning models. In this work, we present\ntwo unified recurrent neural models leading to three transfer learning\nframeworks for relation classification tasks. We systematically investigate\neffectiveness of the proposed frameworks in transferring the knowledge under\nmultiple aspects related to source and target tasks, such as, similarity or\nrelatedness between source and target tasks, and size of training data for\nsource task. Our empirical results show that the proposed frameworks in general\nimprove the model performance, however these improvements do depend on aspects\nrelated to source and target tasks. This dependence then finally determine the\nchoice of a particular TL framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03447v1",
            "date": "2017-08-11 06:30:23+00:00",
            "title": "Unified Neural Architecture for Drug, Disease and Clinical Entity Recognition",
            "authors": [
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Most existing methods for biomedical entity recognition task rely on explicit\nfeature engineering where many features either are specific to a particular\ntask or depends on output of other existing NLP tools. Neural architectures\nhave been shown across various domains that efforts for explicit feature design\ncan be reduced. In this work we propose an unified framework using\nbi-directional long short term memory network (BLSTM) for named entity\nrecognition (NER) tasks in biomedical and clinical domains. Three important\ncharacteristics of the framework are as follows - (1) model learns contextual\nas well as morphological features using two different BLSTM in hierarchy, (2)\nmodel uses first order linear conditional random field (CRF) in its output\nlayer in cascade of BLSTM to infer label or tag sequence, (3) model does not\nuse any domain specific features or dictionary, i.e., in another words, same\nset of features are used in the three NER tasks, namely, disease name\nrecognition (Disease NER), drug name recognition (Drug NER) and clinical entity\nrecognition (Clinical NER). We compare performance of the proposed model with\nexisting state-of-the-art models on the standard benchmark datasets of the\nthree tasks. We show empirically that the proposed framework outperforms all\nexisting models. Further our analysis of CRF layer and word-embedding obtained\nusing character based embedding show their importance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03492v1",
            "date": "2017-08-11 09:58:39+00:00",
            "title": "Break it Down for Me: A Study in Automated Lyric Annotation",
            "authors": [
                "Lucas Sterckx",
                "Jason Naradowsky",
                "Bill Byrne",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "Comprehending lyrics, as found in songs and poems, can pose a challenge to\nhuman and machine readers alike. This motivates the need for systems that can\nunderstand the ambiguity and jargon found in such creative texts, and provide\ncommentary to aid readers in reaching the correct interpretation. We introduce\nthe task of automated lyric annotation (ALA). Like text simplification, a goal\nof ALA is to rephrase the original text in a more easily understandable manner.\nHowever, in ALA the system must often include additional information to clarify\nniche terminology and abstract concepts. To stimulate research on this task, we\nrelease a large collection of crowdsourced annotations for song lyrics. We\nanalyze the performance of translation and retrieval models on this task,\nmeasuring performance with both automated and human evaluation. We find that\neach model captures a unique type of information important to the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03541v1",
            "date": "2017-08-11 13:45:48+00:00",
            "title": "Automatic Identification of AltLexes using Monolingual Parallel Corpora",
            "authors": [
                "Elnaz Davoodi",
                "Leila Kosseim"
            ],
            "abstract": "The automatic identification of discourse relations is still a challenging\ntask in natural language processing. Discourse connectives, such as \"since\" or\n\"but\", are the most informative cues to identify explicit relations; however\ndiscourse parsers typically use a closed inventory of such connectives. As a\nresult, discourse relations signaled by markers outside these inventories (i.e.\nAltLexes) are not detected as effectively. In this paper, we propose a novel\nmethod to leverage parallel corpora in text simplification and lexical\nresources to automatically identify alternative lexicalizations that signal\ndiscourse relation. When applied to the Simple Wikipedia and Newsela corpora\nalong with WordNet and the PPDB, the method allowed the automatic discovery of\n91 AltLexes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03629v3",
            "date": "2017-08-11 17:52:36+00:00",
            "title": "Simple and Effective Dimensionality Reduction for Word Embeddings",
            "authors": [
                "Vikas Raunak"
            ],
            "abstract": "Word embeddings have become the basic building blocks for several natural\nlanguage processing and information retrieval tasks. Pre-trained word\nembeddings are used in several downstream applications as well as for\nconstructing representations for sentences, paragraphs and documents. Recently,\nthere has been an emphasis on further improving the pre-trained word vectors\nthrough post-processing algorithms. One such area of improvement is the\ndimensionality reduction of the word embeddings. Reducing the size of word\nembeddings through dimensionality reduction can improve their utility in memory\nconstrained devices, benefiting several real-world applications. In this work,\nwe present a novel algorithm that effectively combines PCA based dimensionality\nreduction with a recently proposed post-processing algorithm, to construct word\nembeddings of lower dimensions. Empirical evaluations on 12 standard word\nsimilarity benchmarks show that our algorithm reduces the embedding\ndimensionality by 50%, while achieving similar or (more often) better\nperformance than the higher dimension embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03696v1",
            "date": "2017-08-11 20:33:02+00:00",
            "title": "Emotion Intensities in Tweets",
            "authors": [
                "Saif M. Mohammad",
                "Felipe Bravo-Marquez"
            ],
            "abstract": "This paper examines the task of detecting intensity of emotion from text. We\ncreate the first datasets of tweets annotated for anger, fear, joy, and sadness\nintensities. We use a technique called best--worst scaling (BWS) that improves\nannotation consistency and obtains reliable fine-grained scores. We show that\nemotion-word hashtags often impact emotion intensity, usually conveying a more\nintense emotion. Finally, we create a benchmark regression system and conduct\nexperiments to determine: which features are useful for detecting emotion\nintensity, and, the extent to which two emotions are similar in terms of how\nthey manifest in language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03699v1",
            "date": "2017-08-11 20:37:27+00:00",
            "title": "Improved Abusive Comment Moderation with User Embeddings",
            "authors": [
                "John Pavlopoulos",
                "Prodromos Malakasiotis",
                "Juli Bakagianni",
                "Ion Androutsopoulos"
            ],
            "abstract": "Experimenting with a dataset of approximately 1.6M user comments from a Greek\nnews sports portal, we explore how a state of the art RNN-based moderation\nmethod can be improved by adding user embeddings, user type embeddings, user\nbiases, or user type biases. We observe improvements in all cases, with user\nembeddings leading to the biggest performance gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03700v1",
            "date": "2017-08-11 20:40:07+00:00",
            "title": "WASSA-2017 Shared Task on Emotion Intensity",
            "authors": [
                "Saif M. Mohammad",
                "Felipe Bravo-Marquez"
            ],
            "abstract": "We present the first shared task on detecting the intensity of emotion felt\nby the speaker of a tweet. We create the first datasets of tweets annotated for\nanger, fear, joy, and sadness intensities using a technique called best--worst\nscaling (BWS). We show that the annotations lead to reliable fine-grained\nintensity scores (rankings of tweets by intensity). The data was partitioned\ninto training, development, and test sets for the competition. Twenty-two teams\nparticipated in the shared task, with the best system obtaining a Pearson\ncorrelation of 0.747 with the gold intensity scores. We summarize the machine\nlearning setups, resources, and tools used by the participating teams, with a\nfocus on the techniques and resources that are particularly useful for the\ntask. The emotion intensity dataset and the shared task are helping improve our\nunderstanding of how we convey more or less intense emotions through language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03743v1",
            "date": "2017-08-12 04:33:52+00:00",
            "title": "Cross-Sentence N-ary Relation Extraction with Graph LSTMs",
            "authors": [
                "Nanyun Peng",
                "Hoifung Poon",
                "Chris Quirk",
                "Kristina Toutanova",
                "Wen-tau Yih"
            ],
            "abstract": "Past work in relation extraction has focused on binary relations in single\nsentences. Recent NLP inroads in high-value domains have sparked interest in\nthe more general setting of extracting n-ary relations that span multiple\nsentences. In this paper, we explore a general relation extraction framework\nbased on graph long short-term memory networks (graph LSTMs) that can be easily\nextended to cross-sentence n-ary relation extraction. The graph formulation\nprovides a unified way of exploring different LSTM approaches and incorporating\nvarious intra-sentential and inter-sentential dependencies, such as sequential,\nsyntactic, and discourse relations. A robust contextual representation is\nlearned for the entities, which serves as input to the relation classifier.\nThis simplifies handling of relations with arbitrary arity, and enables\nmulti-task learning with related relations. We evaluate this framework in two\nimportant precision medicine settings, demonstrating its effectiveness with\nboth conventional supervised learning and distant supervision. Cross-sentence\nextraction produced larger knowledge bases. and multi-task learning\nsignificantly improved extraction accuracy. A thorough analysis of various LSTM\napproaches yielded useful insight the impact of linguistic analysis on\nextraction accuracy.",
            "journal ref": "Transactions of the Association for Computational Linguistics\n  (TACL) 2017, Vol 5",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03920v1",
            "date": "2017-08-13 15:09:47+00:00",
            "title": "Towards Speech Emotion Recognition \"in the wild\" using Aggregated Corpora and Deep Multi-Task Learning",
            "authors": [
                "Jaebok Kim",
                "Gwenn Englebienne",
                "Khiet P. Truong",
                "Vanessa Evers"
            ],
            "abstract": "One of the challenges in Speech Emotion Recognition (SER) \"in the wild\" is\nthe large mismatch between training and test data (e.g. speakers and tasks). In\norder to improve the generalisation capabilities of the emotion models, we\npropose to use Multi-Task Learning (MTL) and use gender and naturalness as\nauxiliary tasks in deep neural networks. This method was evaluated in\nwithin-corpus and various cross-corpus classification experiments that simulate\nconditions \"in the wild\". In comparison to Single-Task Learning (STL) based\nstate of the art methods, we found that our MTL method proposed improved\nperformance significantly. Particularly, models using both gender and\nnaturalness achieved more gains than those using either gender or naturalness\nseparately. This benefit was also found in the high-level representations of\nthe feature space, obtained from our method proposed, where discriminative\nemotional clusters could be observed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04299v1",
            "date": "2017-08-14 20:01:44+00:00",
            "title": "Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks",
            "authors": [
                "Sayyed M. Zahiri",
                "Jinho D. Choi"
            ],
            "abstract": "While there have been significant advances in detecting emotions from speech\nand image recognition, emotion detection on text is still under-explored and\nremained as an active research field. This paper introduces a corpus for\ntext-based emotion detection on multiparty dialogue as well as deep neural\nmodels that outperform the existing approaches for document classification. We\nfirst present a new corpus that provides annotation of seven emotions on\nconsecutive utterances in dialogues extracted from the show, Friends. We then\nsuggest four types of sequence-based convolutional neural network models with\nattention that leverage the sequence information encapsulated in dialogue. Our\nbest model shows the accuracies of 37.9% and 54% for fine- and coarse-grained\nemotions, respectively. Given the difficulty of this task, this is promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04390v1",
            "date": "2017-08-15 03:46:31+00:00",
            "title": "Fluency-Guided Cross-Lingual Image Captioning",
            "authors": [
                "Weiyu Lan",
                "Xirong Li",
                "Jianfeng Dong"
            ],
            "abstract": "Image captioning has so far been explored mostly in English, as most\navailable datasets are in this language. However, the application of image\ncaptioning should not be restricted by language. Only few studies have been\nconducted for image captioning in a cross-lingual setting. Different from these\nworks that manually build a dataset for a target language, we aim to learn a\ncross-lingual captioning model fully from machine-translated sentences. To\nconquer the lack of fluency in the translated sentences, we propose in this\npaper a fluency-guided learning framework. The framework comprises a module to\nautomatically estimate the fluency of the sentences and another module to\nutilize the estimated fluency scores to effectively train an image captioning\nmodel for the target language. As experiments on two bilingual\n(English-Chinese) datasets show, our approach improves both fluency and\nrelevance of the generated captions in Chinese, but without using any manually\nwritten sentences from the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04469v1",
            "date": "2017-08-15 12:05:02+00:00",
            "title": "Comparison of Decoding Strategies for CTC Acoustic Models",
            "authors": [
                "Thomas Zenkel",
                "Ramon Sanabria",
                "Florian Metze",
                "Jan Niehues",
                "Matthias Sperber",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "Connectionist Temporal Classification has recently attracted a lot of\ninterest as it offers an elegant approach to building acoustic models (AMs) for\nspeech recognition. The CTC loss function maps an input sequence of observable\nfeature vectors to an output sequence of symbols. Output symbols are\nconditionally independent of each other under CTC loss, so a language model\n(LM) can be incorporated conveniently during decoding, retaining the\ntraditional separation of acoustic and linguistic components in ASR. For fixed\nvocabularies, Weighted Finite State Transducers provide a strong baseline for\nefficient integration of CTC AMs with n-gram LMs. Character-based neural LMs\nprovide a straight forward solution for open vocabulary speech recognition and\nall-neural models, and can be decoded with beam search. Finally,\nsequence-to-sequence models can be used to translate a sequence of individual\nsounds into a word string. We compare the performance of these three\napproaches, and analyze their error patterns, which provides insightful\nguidance for future research and development in this important area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04559v1",
            "date": "2017-08-12 18:55:12+00:00",
            "title": "Statistical Vs Rule Based Machine Translation; A Case Study on Indian Language Perspective",
            "authors": [
                "Sreelekha S"
            ],
            "abstract": "In this paper we present our work on a case study between Statistical Machien\nTransaltion (SMT) and Rule-Based Machine Translation (RBMT) systems on\nEnglish-Indian langugae and Indian to Indian langugae perspective. Main\nobjective of our study is to make a five way performance compariosn; such as,\na) SMT and RBMT b) SMT on English-Indian langugae c) RBMT on English-Indian\nlangugae d) SMT on Indian to Indian langugae perspective e) RBMT on Indian to\nIndian langugae perspective. Through a detailed analysis we describe the Rule\nBased and the Statistical Machine Translation system developments and its\nevaluations. Through a detailed error analysis, we point out the relative\nstrengths and weaknesses of both systems. The observations based on our study\nare: a) SMT systems outperforms RBMT b) In the case of SMT, English to Indian\nlanguage MT systmes performs better than Indian to English langugae MT systems\nc) In the case of RBMT, English to Indian langugae MT systems perofrms better\nthan Indian to Englsih Language MT systems d) SMT systems performs better for\nIndian to Indian language MT systems compared to RBMT. Effectively, we shall\nsee that even with a small amount of training corpus a statistical machine\ntranslation system has many advantages for high quality domain specific machine\ntranslation over that of a rule-based counterpart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04704v1",
            "date": "2017-08-15 22:12:59+00:00",
            "title": "Evaluating Word Embeddings for Sentence Boundary Detection in Speech Transcripts",
            "authors": [
                "Marcos V. Treviso",
                "Christopher D. Shulby",
                "Sandra M. Aluisio"
            ],
            "abstract": "This paper is motivated by the automation of neuropsychological tests\ninvolving discourse analysis in the retellings of narratives by patients with\npotential cognitive impairment. In this scenario the task of sentence boundary\ndetection in speech transcripts is important as discourse analysis involves the\napplication of Natural Language Processing tools, such as taggers and parsers,\nwhich depend on the sentence as a processing unit. Our aim in this paper is to\nverify which embedding induction method works best for the sentence boundary\ndetection task, specifically whether it be those which were proposed to capture\nsemantic, syntactic or morphological similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04755v1",
            "date": "2017-08-16 03:17:57+00:00",
            "title": "Learning Chinese Word Representations From Glyphs Of Characters",
            "authors": [
                "Tzu-Ray Su",
                "Hung-Yi Lee"
            ],
            "abstract": "In this paper, we propose new methods to learn Chinese word representations.\nChinese characters are composed of graphical components, which carry rich\nsemantics. It is common for a Chinese learner to comprehend the meaning of a\nword from these graphical components. As a result, we propose models that\nenhance word representations by character glyphs. The character glyph features\nare directly learned from the bitmaps of characters by convolutional\nauto-encoder(convAE), and the glyph features improve Chinese word\nrepresentations which are already enhanced by character embeddings. Another\ncontribution in this paper is that we created several evaluation datasets in\ntraditional Chinese and made them public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04765v1",
            "date": "2017-08-16 04:27:09+00:00",
            "title": "Dialogue Act Segmentation for Vietnamese Human-Human Conversational Texts",
            "authors": [
                "Thi Lan Ngo",
                "Khac Linh Pham",
                "Minh Son Cao",
                "Son Bao Pham",
                "Xuan Hieu Phan"
            ],
            "abstract": "Dialog act identification plays an important role in understanding\nconversations. It has been widely applied in many fields such as dialogue\nsystems, automatic machine translation, automatic speech recognition, and\nespecially useful in systems with human-computer natural language dialogue\ninterfaces such as virtual assistants and chatbots. The first step of\nidentifying dialog act is identifying the boundary of the dialog act in\nutterances. In this paper, we focus on segmenting the utterance according to\nthe dialog act boundaries, i.e. functional segments identification, for\nVietnamese utterances. We investigate carefully functional segment\nidentification in two approaches: (1) machine learning approach using maximum\nentropy (ME) and conditional random fields (CRFs); (2) deep learning approach\nusing bidirectional Long Short-Term Memory (LSTM) with a CRF layer\n(Bi-LSTM-CRF) on two different conversational datasets: (1) Facebook messages\n(Message data); (2) transcription from phone conversations (Phone data). To the\nbest of our knowledge, this is the first work that applies deep learning based\napproach to dialog act segmentation. As the results show, deep learning\napproach performs appreciably better as to compare with traditional machine\nlearning approaches. Moreover, it is also the first study that tackles dialog\nact and functional segment identification for Vietnamese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05148v1",
            "date": "2017-08-17 06:42:03+00:00",
            "title": "Natural Language Processing: State of The Art, Current Trends and Challenges",
            "authors": [
                "Diksha Khurana",
                "Aditya Koli",
                "Kiran Khatter",
                "Sukhdev Singh"
            ],
            "abstract": "Natural language processing (NLP) has recently gained much attention for\nrepresenting and analysing human language computationally. It has spread its\napplications in various fields such as machine translation, email spam\ndetection, information extraction, summarization, medical, and question\nanswering etc. The paper distinguishes four phases by discussing different\nlevels of NLP and components of Natural Language Generation (NLG) followed by\npresenting the history and evolution of NLP, state of the art presenting the\nvarious applications of NLP and current trends and challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05269v1",
            "date": "2017-08-17 13:50:39+00:00",
            "title": "Towards Syntactic Iberian Polarity Classification",
            "authors": [
                "David Vilares",
                "Marcos Garcia",
                "Miguel A. Alonso",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Lexicon-based methods using syntactic rules for polarity classification rely\non parsers that are dependent on the language and on treebank guidelines. Thus,\nrules are also dependent and require adaptation, especially in multilingual\nscenarios. We tackle this challenge in the context of the Iberian Peninsula,\nreleasing the first symbolic syntax-based Iberian system with rules shared\nacross five official languages: Basque, Catalan, Galician, Portuguese and\nSpanish. The model is made available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05286v2",
            "date": "2017-08-17 14:06:58+00:00",
            "title": "Simple Open Stance Classification for Rumour Analysis",
            "authors": [
                "Ahmet Aker",
                "Leon Derczynski",
                "Kalina Bontcheva"
            ],
            "abstract": "Stance classification determines the attitude, or stance, in a (typically\nshort) text. The task has powerful applications, such as the detection of fake\nnews or the automatic extraction of attitudes toward entities or events in the\nmedia. This paper describes a surprisingly simple and efficient classification\napproach to open stance classification in Twitter, for rumour and veracity\nclassification. The approach profits from a novel set of automatically\nidentifiable problem-specific features, which significantly boost classifier\naccuracy and achieve above state-of-the-art results on recent benchmark\ndatasets. This calls into question the value of using complex sophisticated\nmodels for stance classification without first doing informed feature\nextraction.",
            "journal ref": "In RANLP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05449v1",
            "date": "2017-08-17 21:57:47+00:00",
            "title": "An Annotated Corpus of Relational Strategies in Customer Service",
            "authors": [
                "Ian Beaver",
                "Cynthia Freeman",
                "Abdullah Mueen"
            ],
            "abstract": "We create and release the first publicly available commercial customer\nservice corpus with annotated relational segments. Human-computer data from\nthree live customer service Intelligent Virtual Agents (IVAs) in the domains of\ntravel and telecommunications were collected, and reviewers marked all text\nthat was deemed unnecessary to the determination of user intention. After\nmerging the selections of multiple reviewers to create highlighted texts, a\nsecond round of annotation was done to determine the classes of language\npresent in the highlighted sections such as the presence of Greetings,\nBackstory, Justification, Gratitude, Rants, or Emotions. This resulting corpus\nis a valuable resource for improving the quality and relational abilities of\nIVAs. As well as discussing the corpus itself, we compare the usage of such\nlanguage in human-human interactions on TripAdvisor forums. We show that\nremoval of this language from task-based inputs has a positive effect on IVA\nunderstanding by both an increase in confidence and improvement in responses,\ndemonstrating the need for automated methods of its discovery.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05466v1",
            "date": "2017-08-17 23:37:18+00:00",
            "title": "Large-Scale Domain Adaptation via Teacher-Student Learning",
            "authors": [
                "Jinyu Li",
                "Michael L. Seltzer",
                "Xi Wang",
                "Rui Zhao",
                "Yifan Gong"
            ],
            "abstract": "High accuracy speech recognition requires a large amount of transcribed data\nfor supervised training. In the absence of such data, domain adaptation of a\nwell-trained acoustic model can be performed, but even here, high accuracy\nusually requires significant labeled data from the target domain. In this work,\nwe propose an approach to domain adaptation that does not require\ntranscriptions but instead uses a corpus of unlabeled parallel data, consisting\nof pairs of samples from the source domain of the well-trained model and the\ndesired target domain. To perform adaptation, we employ teacher/student (T/S)\nlearning, in which the posterior probabilities generated by the source-domain\nmodel can be used in lieu of labels to train the target-domain model. We\nevaluate the proposed approach in two scenarios, adapting a clean acoustic\nmodel to noisy speech and adapting an adults speech acoustic model to children\nspeech. Significant improvements in accuracy are obtained, with reductions in\nword error rate of up to 44% over the original source model without the need\nfor transcribed data in the target domain. Moreover, we show that increasing\nthe amount of unlabeled data results in additional model robustness, which is\nparticularly beneficial when using simulated training data in the\ntarget-domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05482v2",
            "date": "2017-08-18 02:07:36+00:00",
            "title": "A Question Answering Approach to Emotion Cause Extraction",
            "authors": [
                "Lin Gui",
                "Jiannan Hu",
                "Yulan He",
                "Ruifeng Xu",
                "Qin Lu",
                "Jiachen Du"
            ],
            "abstract": "Emotion cause extraction aims to identify the reasons behind a certain\nemotion expressed in text. It is a much more difficult task compared to emotion\nclassification. Inspired by recent advances in using deep memory networks for\nquestion answering (QA), we propose a new approach which considers emotion\ncause identification as a reading comprehension task in QA. Inspired by\nconvolutional neural networks, we propose a new mechanism to store relevant\ncontext in different memory slots to model context information. Our proposed\napproach can extract both word level sequence features and lexical features.\nPerformance evaluation shows that our method achieves the state-of-the-art\nperformance on a recently released emotion cause dataset, outperforming a\nnumber of competitive baselines by at least 3.01% in F-measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05515v1",
            "date": "2017-08-18 06:02:16+00:00",
            "title": "Syllable-level Neural Language Model for Agglutinative Language",
            "authors": [
                "Seunghak Yu",
                "Nilesh Kulkarni",
                "Haejun Lee",
                "Jihie Kim"
            ],
            "abstract": "Language models for agglutinative languages have always been hindered in past\ndue to myriad of agglutinations possible to any given word through various\naffixes. We propose a method to diminish the problem of out-of-vocabulary words\nby introducing an embedding derived from syllables and morphemes which\nleverages the agglutinative property. Our model outperforms character-level\nembedding in perplexity by 16.87 with 9.50M parameters. Proposed method\nachieves state of the art performance over existing input prediction methods in\nterms of Key Stroke Saving and has been commercialized.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05521v1",
            "date": "2017-08-18 06:59:16+00:00",
            "title": "EmoAtt at EmoInt-2017: Inner attention sentence embedding for Emotion Intensity",
            "authors": [
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "In this paper we describe a deep learning system that has been designed and\nbuilt for the WASSA 2017 Emotion Intensity Shared Task. We introduce a\nrepresentation learning approach based on inner attention on top of an RNN.\nResults show that our model offers good capabilities and is able to\nsuccessfully identify emotion-bearing words to predict intensity without\nleveraging on lexicons, obtaining the 13th place among 22 shared task\ncompetitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05536v1",
            "date": "2017-08-18 08:43:52+00:00",
            "title": "Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution",
            "authors": [
                "E. Manjavacas",
                "J. de Gussem",
                "W. Daelemans",
                "M. Kestemont"
            ],
            "abstract": "Recent applications of neural language models have led to an increased\ninterest in the automatic generation of natural language. However impressive,\nthe evaluation of neurally generated text has so far remained rather informal\nand anecdotal. Here, we present an attempt at the systematic assessment of one\naspect of the quality of neurally generated text. We focus on a specific aspect\nof neural language generation: its ability to reproduce authorial writing\nstyles. Using established models for authorship attribution, we empirically\nassess the stylistic qualities of neurally generated text. In comparison to\nconventional language models, neural models generate fuzzier text that is\nrelatively harder to attribute correctly. Nevertheless, our results also\nsuggest that neurally generated text offers more valuable perspectives for the\naugmentation of training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05582v1",
            "date": "2017-08-18 12:34:11+00:00",
            "title": "Agree to Disagree: Improving Disagreement Detection with Dual GRUs",
            "authors": [
                "Sushant Hiray",
                "Venkatesh Duppada"
            ],
            "abstract": "This paper presents models for detecting agreement/disagreement in online\ndiscussions. In this work we show that by using a Siamese inspired architecture\nto encode the discussions, we no longer need to rely on hand-crafted features\nto exploit the meta thread structure. We evaluate our model on existing online\ndiscussion corpora - ABCD, IAC and AWTP. Experimental results on ABCD dataset\nshow that by fusing lexical and word embedding features, our model achieves the\nstate of the art performance of 0.804 average F1 score. We also show that the\nmodel trained on ABCD dataset performs competitively on relatively smaller\nannotated datasets (IAC and AWTP).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05592v1",
            "date": "2017-08-18 13:11:22+00:00",
            "title": "Future Word Contexts in Neural Network Language Models",
            "authors": [
                "Xie Chen",
                "Xunying Liu",
                "Anton Ragni",
                "Yu Wang",
                "Mark Gales"
            ],
            "abstract": "Recently, bidirectional recurrent network language models (bi-RNNLMs) have\nbeen shown to outperform standard, unidirectional, recurrent neural network\nlanguage models (uni-RNNLMs) on a range of speech recognition tasks. This\nindicates that future word context information beyond the word history can be\nuseful. However, bi-RNNLMs pose a number of challenges as they make use of the\ncomplete previous and future word context information. This impacts both\ntraining efficiency and their use within a lattice rescoring framework. In this\npaper these issues are addressed by proposing a novel neural network structure,\nsucceeding word RNNLMs (su-RNNLMs). Instead of using a recurrent unit to\ncapture the complete future word contexts, a feedforward unit is used to model\na finite number of succeeding, future, words. This model can be trained much\nmore efficiently than bi-RNNLMs and can also be used for lattice rescoring.\nExperimental results on a meeting transcription task (AMI) show the proposed\nmodel consistently outperformed uni-RNNLMs and yield only a slight degradation\ncompared to bi-RNNLMs in N-best rescoring. Additionally, performance\nimprovements can be obtained using lattice rescoring and subsequent confusion\nnetwork decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05719v1",
            "date": "2017-08-18 18:00:05+00:00",
            "title": "Cross-Lingual Dependency Parsing for Closely Related Languages - Helsinki's Submission to VarDial 2017",
            "authors": [
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "This paper describes the submission from the University of Helsinki to the\nshared task on cross-lingual dependency parsing at VarDial 2017. We present\nwork on annotation projection and treebank translation that gave good results\nfor all three target languages in the test set. In particular, Slovak seems to\nwork well with information coming from the Czech treebank, which is in line\nwith related work. The attachment scores for cross-lingual models even surpass\nthe fully supervised models trained on the target language treebank. Croatian\nis the most difficult language in the test set and the improvements over the\nbaseline are rather modest. Norwegian works best with information coming from\nSwedish whereas Danish contributes surprisingly little.",
            "journal ref": "In Proceedings of the Fourth Workshop on NLP for Similar\n  Languages, Varieties and Dialects at EACL 2017, Valencia/Spain, pp. 131-136",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05729v1",
            "date": "2017-08-18 18:16:23+00:00",
            "title": "Neural machine translation for low-resource languages",
            "authors": [
                "Robert \u00d6stling",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Neural machine translation (NMT) approaches have improved the state of the\nart in many machine translation settings over the last couple of years, but\nthey require large amounts of training data to produce sensible output. We\ndemonstrate that NMT can be used for low-resource languages as well, by\nintroducing more local dependencies and using word alignments to learn sentence\nreordering during translation. In addition to our novel model, we also present\nan empirical evaluation of low-resource phrase-based statistical machine\ntranslation (SMT) and NMT to investigate the lower limits of the respective\ntechnologies. We find that while SMT remains the best option for low-resource\nsettings, our method can produce acceptable translations with only 70000 tokens\nof training data, a level where the baseline NMT system fails completely.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05763v1",
            "date": "2017-08-18 21:27:34+00:00",
            "title": "The Natural Stories Corpus",
            "authors": [
                "Richard Futrell",
                "Edward Gibson",
                "Hal Tily",
                "Idan Blank",
                "Anastasia Vishnevetsky",
                "Steven T. Piantadosi",
                "Evelina Fedorenko"
            ],
            "abstract": "It is now a common practice to compare models of human language processing by\npredicting participant reactions (such as reading times) to corpora consisting\nof rich naturalistic linguistic materials. However, many of the corpora used in\nthese studies are based on naturalistic text and thus do not contain many of\nthe low-frequency syntactic constructions that are often required to\ndistinguish processing theories. Here we describe a new corpus consisting of\nEnglish texts edited to contain many low-frequency syntactic constructions\nwhile still sounding fluent to native speakers. The corpus is annotated with\nhand-corrected parse trees and includes self-paced reading time data. Here we\ngive an overview of the content of the corpus and release the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05797v1",
            "date": "2017-08-19 03:18:50+00:00",
            "title": "CLaC @ QATS: Quality Assessment for Text Simplification",
            "authors": [
                "Elnaz Davoodi",
                "Leila Kosseim"
            ],
            "abstract": "This paper describes our approach to the 2016 QATS quality assessment shared\ntask. We trained three independent Random Forest classifiers in order to assess\nthe quality of the simplified texts in terms of grammaticality, meaning\npreservation and simplicity. We used the language model of Google-Ngram as\nfeature to predict the grammaticality. Meaning preservation is predicted using\ntwo complementary approaches based on word embedding and WordNet synonyms. A\nwider range of features including TF-IDF, sentence length and frequency of cue\nphrases are used to evaluate the simplicity aspect. Overall, the accuracy of\nthe system ranges from 33.33% for the overall aspect to 58.73% for\ngrammaticality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05798v1",
            "date": "2017-08-19 03:19:40+00:00",
            "title": "The CLaC Discourse Parser at CoNLL-2016",
            "authors": [
                "Majid Laali",
                "Andre Cianflone",
                "Leila Kosseim"
            ],
            "abstract": "This paper describes our submission \"CLaC\" to the CoNLL-2016 shared task on\nshallow discourse parsing. We used two complementary approaches for the task. A\nstandard machine learning approach for the parsing of explicit relations, and a\ndeep learning approach for non-explicit relations. Overall, our parser achieves\nan F1-score of 0.2106 on the identification of discourse relations (0.3110 for\nexplicit relations and 0.1219 for non-explicit relations) on the blind\nCoNLL-2016 test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05800v1",
            "date": "2017-08-19 03:20:43+00:00",
            "title": "On the Contribution of Discourse Structure on Text Complexity Assessment",
            "authors": [
                "Elnaz Davoodi",
                "Leila Kosseim"
            ],
            "abstract": "This paper investigates the influence of discourse features on text\ncomplexity assessment. To do so, we created two data sets based on the Penn\nDiscourse Treebank and the Simple English Wikipedia corpora and compared the\ninfluence of coherence, cohesion, surface, lexical and syntactic features to\nassess text complexity.\n  Results show that with both data sets coherence features are more correlated\nto text complexity than the other types of features. In addition, feature\nselection revealed that with both data sets the top most discriminating feature\nis a coherence feature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05801v1",
            "date": "2017-08-19 03:37:38+00:00",
            "title": "ClaC: Semantic Relatedness of Words and Phrases",
            "authors": [
                "Reda Siblini",
                "Leila Kosseim"
            ],
            "abstract": "The measurement of phrasal semantic relatedness is an important metric for\nmany natural language processing applications. In this paper, we present three\napproaches for measuring phrasal semantics, one based on a semantic network\nmodel, another on a distributional similarity model, and a hybrid between the\ntwo. Our hybrid approach achieved an F-measure of 77.4% on the task of\nevaluating the semantic similarity of words and compositional phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05803v1",
            "date": "2017-08-19 04:01:43+00:00",
            "title": "Measuring the Effect of Discourse Relations on Blog Summarization",
            "authors": [
                "Shamima Mithun",
                "Leila Kosseim"
            ],
            "abstract": "The work presented in this paper attempts to evaluate and quantify the use of\ndiscourse relations in the context of blog summarization and compare their use\nto more traditional and factual texts. Specifically, we measured the usefulness\nof 6 discourse relations - namely comparison, contingency, illustration,\nattribution, topic-opinion, and attributive for the task of text summarization\nfrom blogs. We have evaluated the effect of each relation using the TAC 2008\nopinion summarization dataset and compared them with the results with the DUC\n2007 dataset. The results show that in both textual genres, contingency,\ncomparison, and illustration relations provide a significant improvement on\nsummarization content; while attribution, topic-opinion, and attributive\nrelations do not provide a consistent and significant improvement. These\nresults indicate that, at least for summarization, discourse relations are just\nas useful for informal and affective texts as for more traditional news\narticles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05857v1",
            "date": "2017-08-19 14:43:50+00:00",
            "title": "The CLaC Discourse Parser at CoNLL-2015",
            "authors": [
                "Majid Laali",
                "Elnaz Davoodi",
                "Leila Kosseim"
            ],
            "abstract": "This paper describes our submission (kosseim15) to the CoNLL-2015 shared task\non shallow discourse parsing. We used the UIMA framework to develop our parser\nand used ClearTK to add machine learning functionality to the UIMA framework.\nOverall, our parser achieves a result of 17.3 F1 on the identification of\ndiscourse relations on the blind CoNLL-2015 test set, ranking in sixth place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05891v1",
            "date": "2017-08-19 19:52:36+00:00",
            "title": "Arabic Multi-Dialect Segmentation: bi-LSTM-CRF vs. SVM",
            "authors": [
                "Mohamed Eldesouki",
                "Younes Samih",
                "Ahmed Abdelali",
                "Mohammed Attia",
                "Hamdy Mubarak",
                "Kareem Darwish",
                "Kallmeyer Laura"
            ],
            "abstract": "Arabic word segmentation is essential for a variety of NLP applications such\nas machine translation and information retrieval. Segmentation entails breaking\nwords into their constituent stems, affixes and clitics. In this paper, we\ncompare two approaches for segmenting four major Arabic dialects using only\nseveral thousand training examples for each dialect. The two approaches involve\nposing the problem as a ranking problem, where an SVM ranker picks the best\nsegmentation, and as a sequence labeling problem, where a bi-LSTM RNN coupled\nwith CRF determines where best to segment words. We are able to achieve solid\nsegmentation results for all dialects using rather limited training data. We\nalso show that employing Modern Standard Arabic data for domain adaptation and\nassuming context independence improve overall results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05942v1",
            "date": "2017-08-20 09:24:46+00:00",
            "title": "The Helsinki Neural Machine Translation System",
            "authors": [
                "Robert \u00d6stling",
                "Yves Scherrer",
                "J\u00f6rg Tiedemann",
                "Gongbo Tang",
                "Tommi Nieminen"
            ],
            "abstract": "We introduce the Helsinki Neural Machine Translation system (HNMT) and how it\nis applied in the news translation task at WMT 2017, where it ranked first in\nboth the human and automatic evaluations for English--Finnish. We discuss the\nsuccess of English--Finnish translations and the overall advantage of NMT over\na strong SMT baseline. We also discuss our submissions for English--Latvian,\nEnglish--Chinese and Chinese--English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05943v1",
            "date": "2017-08-20 09:31:49+00:00",
            "title": "Neural Machine Translation with Extended Context",
            "authors": [
                "J\u00f6rg Tiedemann",
                "Yves Scherrer"
            ],
            "abstract": "We investigate the use of extended context in attention-based neural machine\ntranslation. We base our experiments on translated movie subtitles and discuss\nthe effect of increasing the segments beyond single translation units. We study\nthe use of extended source language context as well as bilingual context\nextensions. The models learn to distinguish between information from different\nsegments and are surprisingly robust with respect to translation quality. In\nthis pilot study, we observe interesting cross-sentential attention patterns\nthat improve textual coherence in translation at least in some selected cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05956v1",
            "date": "2017-08-20 12:00:16+00:00",
            "title": "An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "We present a novel end-to-end trainable neural network model for\ntask-oriented dialog systems. The model is able to track dialog state, issue\nAPI calls to knowledge base (KB), and incorporate structured KB query results\ninto system responses to successfully complete task-oriented dialogs. The\nproposed model produces well-structured system responses by jointly learning\nbelief tracking and KB result processing conditioning on the dialog history. We\nevaluate the model in a restaurant search domain using a dataset that is\nconverted from the second Dialog State Tracking Challenge (DSTC2) corpus.\nExperiment results show that the proposed model can robustly track dialog state\ngiven the dialog history. Moreover, our model demonstrates promising results in\nproducing appropriate system responses, outperforming prior end-to-end\ntrainable neural network models using per-response accuracy evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05992v2",
            "date": "2017-08-20 16:29:54+00:00",
            "title": "Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient?",
            "authors": [
                "Piotr \u017belasko"
            ],
            "abstract": "In this paper, the problem of recovery of morphological information lost in\nabbreviated forms is addressed with a focus on highly inflected languages.\nEvidence is presented that the correct inflected form of an expanded\nabbreviation can in many cases be deduced solely from the morphosyntactic tags\nof the context. The prediction model is a deep bidirectional LSTM network with\ntag embedding. The training and evaluation data are gathered by finding the\nwords which could have been abbreviated and using their corresponding\nmorphosyntactic tags as the labels, while the tags of the context words are\nused as the input features for classification. The network is trained on over\n10 million words from the Polish Sejm Corpus and achieves 74.2% prediction\naccuracy on a smaller, but more general National Corpus of Polish. The analysis\nof errors suggests that performance in this task may improve if some prior\nknowledge about the abbreviated word is incorporated into the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06022v1",
            "date": "2017-08-20 21:19:01+00:00",
            "title": "Learning to Paraphrase for Question Answering",
            "authors": [
                "Li Dong",
                "Jonathan Mallinson",
                "Siva Reddy",
                "Mirella Lapata"
            ],
            "abstract": "Question answering (QA) systems are sensitive to the many different ways\nnatural language expresses the same information need. In this paper we turn to\nparaphrases as a means of capturing this knowledge and present a general\nframework which learns felicitous paraphrases for various QA tasks. Our method\nis trained end-to-end using question-answer pairs as a supervision signal. A\nquestion and its paraphrases serve as input to a neural scoring model which\nassigns higher weights to linguistic expressions most likely to yield correct\nanswers. We evaluate our approach on QA over Freebase and answer sentence\nselection. Experimental results on three datasets show that our framework\nconsistently improves performance, achieving competitive results despite the\nuse of simple QA models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06025v1",
            "date": "2017-08-20 21:21:37+00:00",
            "title": "Portuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks",
            "authors": [
                "Nathan Hartmann",
                "Erick Fonseca",
                "Christopher Shulby",
                "Marcos Treviso",
                "Jessica Rodrigues",
                "Sandra Aluisio"
            ],
            "abstract": "Word embeddings have been found to provide meaningful representations for\nwords in an efficient way; therefore, they have become common in Natural\nLanguage Processing sys- tems. In this paper, we evaluated different word\nembedding models trained on a large Portuguese corpus, including both Brazilian\nand European variants. We trained 31 word embedding models using FastText,\nGloVe, Wang2Vec and Word2Vec. We evaluated them intrinsically on syntactic and\nsemantic analogies and extrinsically on POS tagging and sentence semantic\nsimilarity tasks. The obtained results suggest that word analogies are not\nappropriate for word embedding evaluation; task-specific evaluations appear to\nbe a better option.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06073v2",
            "date": "2017-08-21 03:17:23+00:00",
            "title": "The Microsoft 2017 Conversational Speech Recognition System",
            "authors": [
                "W. Xiong",
                "L. Wu",
                "F. Alleva",
                "J. Droppo",
                "X. Huang",
                "A. Stolcke"
            ],
            "abstract": "We describe the 2017 version of Microsoft's conversational speech recognition\nsystem, in which we update our 2016 system with recent developments in\nneural-network-based acoustic and language modeling to further advance the\nstate of the art on the Switchboard speech recognition task. The system adds a\nCNN-BLSTM acoustic model to the set of model architectures we combined\npreviously, and includes character-based and dialog session aware LSTM language\nmodels in rescoring. For system combination we adopt a two-stage approach,\nwhereby subsets of acoustic models are first combined at the senone/frame\nlevel, followed by a word-level voting via confusion networks. We also added a\nconfusion network rescoring step after system combination. The resulting system\nyields a 5.1\\% word error rate on the 2000 Switchboard evaluation set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06075v1",
            "date": "2017-08-21 03:33:58+00:00",
            "title": "Scientific Information Extraction with Semi-supervised Neural Tagging",
            "authors": [
                "Yi Luan",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "This paper addresses the problem of extracting keyphrases from scientific\narticles and categorizing them as corresponding to a task, process, or\nmaterial. We cast the problem as sequence tagging and introduce semi-supervised\nmethods to a neural tagging model, which builds on recent advances in named\nentity recognition. Since annotated training data is scarce in this domain, we\nintroduce a graph-based semi-supervised algorithm together with a data\nselection scheme to leverage unannotated articles. Both inductive and\ntransductive semi-supervised learning strategies outperform state-of-the-art\ninformation extraction performance on the 2017 SemEval Task 10 ScienceIE task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06185v1",
            "date": "2017-08-21 12:30:48+00:00",
            "title": "Seernet at EmoInt-2017: Tweet Emotion Intensity Estimator",
            "authors": [
                "Venkatesh Duppada",
                "Sushant Hiray"
            ],
            "abstract": "The paper describes experiments on estimating emotion intensity in tweets\nusing a generalized regressor system. The system combines lexical, syntactic\nand pre-trained word embedding features, trains them on general regressors and\nfinally combines the best performing models to create an ensemble. The proposed\nsystem stood 3rd out of 22 systems in the leaderboard of WASSA-2017 Shared Task\non Emotion Intensity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06426v1",
            "date": "2017-08-21 21:28:07+00:00",
            "title": "Cold Fusion: Training Seq2Seq Models Together with Language Models",
            "authors": [
                "Anuroop Sriram",
                "Heewoo Jun",
                "Sanjeev Satheesh",
                "Adam Coates"
            ],
            "abstract": "Sequence-to-sequence (Seq2Seq) models with attention have excelled at tasks\nwhich involve generating natural language sentences such as machine\ntranslation, image captioning and speech recognition. Performance has further\nbeen improved by leveraging unlabeled data, often in the form of a language\nmodel. In this work, we present the Cold Fusion method, which leverages a\npre-trained language model during training, and show its effectiveness on the\nspeech recognition task. We show that Seq2Seq models with Cold Fusion are able\nto better utilize language information enjoying i) faster convergence and\nbetter generalization, and ii) almost complete transfer to a new domain while\nusing less than 10% of the labeled training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06510v2",
            "date": "2017-08-22 06:48:27+00:00",
            "title": "Handling Homographs in Neural Machine Translation",
            "authors": [
                "Frederick Liu",
                "Han Lu",
                "Graham Neubig"
            ],
            "abstract": "Homographs, words with different meanings but the same surface form, have\nlong caused difficulty for machine translation systems, as it is difficult to\nselect the correct translation based on the context. However, with the advent\nof neural machine translation (NMT) systems, which can theoretically take into\naccount global sentential context, one may hypothesize that this problem has\nbeen alleviated. In this paper, we first provide empirical evidence that\nexisting NMT systems in fact still have significant problems in properly\ntranslating ambiguous words. We then proceed to describe methods, inspired by\nthe word sense disambiguation literature, that model the context of the input\nword with context-aware word embeddings that help to differentiate the word\nsense be- fore feeding it into the encoder. Experiments on three language pairs\ndemonstrate that such models improve the performance of NMT systems both in\nterms of BLEU score and in the accuracy of translating homographs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06708v1",
            "date": "2017-08-19 18:18:22+00:00",
            "title": "A rule based algorithm for detecting negative words in Persian",
            "authors": [
                "Reza Takhshid",
                "Adel Rahimi"
            ],
            "abstract": "In this paper, we present a novel method for detecting negative words in\nPersian. We first used an algorithm to an exceptions list which was later\nmodified by hand. We then used the mentioned lists and a Persian polarity\ncorpus in our rule based algorithm to detect negative words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07104v1",
            "date": "2017-08-23 17:12:03+00:00",
            "title": "Automatic Detection of Fake News",
            "authors": [
                "Ver\u00f3nica P\u00e9rez-Rosas",
                "Bennett Kleinberg",
                "Alexandra Lefevre",
                "Rada Mihalcea"
            ],
            "abstract": "The proliferation of misleading information in everyday access media outlets\nsuch as social media feeds, news blogs, and online newspapers have made it\nchallenging to identify trustworthy news sources, thus increasing the need for\ncomputational tools able to provide insights into the reliability of online\ncontent. In this paper, we focus on the automatic identification of fake\ncontent in online news. Our contribution is twofold. First, we introduce two\nnovel datasets for the task of fake news detection, covering seven different\nnews domains. We describe the collection, annotation, and validation process in\ndetail and present several exploratory analysis on the identification of\nlinguistic differences in fake and legitimate news content. Second, we conduct\na set of learning experiments to build accurate fake news detectors. In\naddition, we provide comparative analyses of the automatic and manual\nidentification of fake news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07241v5",
            "date": "2017-08-24 01:26:48+00:00",
            "title": "NNVLP: A Neural Network-Based Vietnamese Language Processing Toolkit",
            "authors": [
                "Thai-Hoang Pham",
                "Xuan-Khoai Pham",
                "Tuan-Anh Nguyen",
                "Phuong Le-Hong"
            ],
            "abstract": "This paper demonstrates neural network-based toolkit namely NNVLP for\nessential Vietnamese language processing tasks including part-of-speech (POS)\ntagging, chunking, named entity recognition (NER). Our toolkit is a combination\nof bidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network\n(CNN), Conditional Random Field (CRF), using pre-trained word embeddings as\ninput, which achieves state-of-the-art results on these three tasks. We provide\nboth API and web demo for this toolkit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07279v1",
            "date": "2017-08-24 05:24:26+00:00",
            "title": "Combining Discrete and Neural Features for Sequence Labeling",
            "authors": [
                "Jie Yang",
                "Zhiyang Teng",
                "Meishan Zhang",
                "Yue Zhang"
            ],
            "abstract": "Neural network models have recently received heated research attention in the\nnatural language processing community. Compared with traditional models with\ndiscrete features, neural models have two main advantages. First, they take\nlow-dimensional, real-valued embedding vectors as inputs, which can be trained\nover large raw data, thereby addressing the issue of feature sparsity in\ndiscrete models. Second, deep neural networks can be used to automatically\ncombine input features, and including non-local features that capture semantic\npatterns that cannot be expressed using discrete indicator features. As a\nresult, neural network models have achieved competitive accuracies compared\nwith the best discrete models for a range of NLP tasks.\n  On the other hand, manual feature templates have been carefully investigated\nfor most NLP tasks over decades and typically cover the most useful indicator\npattern for solving the problems. Such information can be complementary the\nfeatures automatically induced from neural networks, and therefore combining\ndiscrete and neural features can potentially lead to better accuracy compared\nwith models that leverage discrete or neural features only.\n  In this paper, we systematically investigate the effect of discrete and\nneural feature combination for a range of fundamental NLP tasks based on\nsequence labeling, including word segmentation, POS tagging and named entity\nrecognition for Chinese and English, respectively. Our results on standard\nbenchmarks show that state-of-the-art neural models can give accuracies\ncomparable to the best discrete models in the literature for most tasks and\ncombing discrete and neural features unanimously yield better results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07403v1",
            "date": "2017-08-24 13:40:06+00:00",
            "title": "CloudScan - A configuration-free invoice analysis system using recurrent neural networks",
            "authors": [
                "Rasmus Berg Palm",
                "Ole Winther",
                "Florian Laws"
            ],
            "abstract": "We present CloudScan; an invoice analysis system that requires zero\nconfiguration or upfront annotation. In contrast to previous work, CloudScan\ndoes not rely on templates of invoice layout, instead it learns a single global\nmodel of invoices that naturally generalizes to unseen invoice layouts. The\nmodel is trained using data automatically extracted from end-user provided\nfeedback. This automatic training data extraction removes the requirement for\nusers to annotate the data precisely. We describe a recurrent neural network\nmodel that can capture long range context and compare it to a baseline logistic\nregression model corresponding to the current CloudScan production system. We\ntrain and evaluate the system on 8 important fields using a dataset of 326,471\ninvoices. The recurrent neural network and baseline model achieve 0.891 and\n0.887 average F1 scores respectively on seen invoice layouts. For the harder\ntask of unseen invoice layouts, the recurrent neural network model outperforms\nthe baseline with 0.840 average F1 compared to 0.788.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07476v1",
            "date": "2017-08-24 16:06:18+00:00",
            "title": "M2D: Monolog to Dialog Generation for Conversational Story Telling",
            "authors": [
                "Kevin K. Bowden",
                "Grace I. Lin",
                "Lena I. Reed",
                "Marilyn A. Walker"
            ],
            "abstract": "Storytelling serves many different social functions, e.g. stories are used to\npersuade, share troubles, establish shared values, learn social behaviors, and\nentertain. Moreover, stories are often told conversationally through dialog,\nand previous work suggests that information provided dialogically is more\nengaging than when provided in monolog. In this paper, we present algorithms\nfor converting a deep representation of a story into a dialogic storytelling,\nthat can vary aspects of the telling, including the personality of the\nstorytellers. We conduct several experiments to test whether dialogic\nstorytellings are more engaging, and whether automatically generated variants\nin linguistic form that correspond to personality differences can be recognized\nin an extended storytelling dialog.",
            "journal ref": "ICIDS 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07690v1",
            "date": "2017-08-25 11:21:44+00:00",
            "title": "Revisiting the Centroid-based Method: A Strong Baseline for Multi-Document Summarization",
            "authors": [
                "Demian Gholipour Ghalandari"
            ],
            "abstract": "The centroid-based model for extractive document summarization is a simple\nand fast baseline that ranks sentences based on their similarity to a centroid\nvector. In this paper, we apply this ranking to possible summaries instead of\nsentences and use a simple greedy algorithm to find the best summary.\nFurthermore, we show possi- bilities to scale up to larger input docu- ment\ncollections by selecting a small num- ber of sentences from each document prior\nto constructing the summary. Experiments were done on the DUC2004 dataset for\nmulti-document summarization. We ob- serve a higher performance over the orig-\ninal model, on par with more complex state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07950v3",
            "date": "2017-08-26 08:12:35+00:00",
            "title": "Machine Translation in Indian Languages: Challenges and Resolution",
            "authors": [
                "Raj Nath Patel",
                "Prakash B. Pimpale",
                "M Sasikumar"
            ],
            "abstract": "English to Indian language machine translation poses the challenge of\nstructural and morphological divergence. This paper describes English to Indian\nlanguage statistical machine translation using pre-ordering and suffix\nseparation. The pre-ordering uses rules to transfer the structure of the source\nsentences prior to training and translation. This syntactic restructuring helps\nstatistical machine translation to tackle the structural divergence and hence\nbetter translation quality. The suffix separation is used to tackle the\nmorphological divergence between English and highly agglutinative Indian\nlanguages. We demonstrate that the use of pre-ordering and suffix separation\nhelps in improving the quality of English to Indian Language machine\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08484v1",
            "date": "2017-08-28 18:57:50+00:00",
            "title": "Joint Syntacto-Discourse Parsing and the Syntacto-Discourse Treebank",
            "authors": [
                "Kai Zhao",
                "Liang Huang"
            ],
            "abstract": "Discourse parsing has long been treated as a stand-alone problem independent\nfrom constituency or dependency parsing. Most attempts at this problem are\npipelined rather than end-to-end, sophisticated, and not self-contained: they\nassume gold-standard text segmentations (Elementary Discourse Units), and use\nexternal parsers for syntactic features. In this paper we propose the first\nend-to-end discourse parser that jointly parses in both syntax and discourse\nlevels, as well as the first syntacto-discourse treebank by integrating the\nPenn Treebank with the RST Treebank. Built upon our recent span-based\nconstituency parser, this joint syntacto-discourse parser requires no\npreprocessing whatsoever (such as segmentation or feature extraction), achieves\nthe state-of-the-art end-to-end discourse parsing accuracy.",
            "journal ref": "Proceedings of EMNLP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08572v1",
            "date": "2017-08-29 02:05:14+00:00",
            "title": "Really? Well. Apparently Bootstrapping Improves the Performance of Sarcasm and Nastiness Classifiers for Online Dialogue",
            "authors": [
                "Stephanie Lukin",
                "Marilyn Walker"
            ],
            "abstract": "More and more of the information on the web is dialogic, from Facebook\nnewsfeeds, to forum conversations, to comment threads on news articles. In\ncontrast to traditional, monologic Natural Language Processing resources such\nas news, highly social dialogue is frequent in social media, making it a\nchallenging context for NLP. This paper tests a bootstrapping method,\noriginally proposed in a monologic domain, to train classifiers to identify two\ndifferent types of subjective language in dialogue: sarcasm and nastiness. We\nexplore two methods of developing linguistic indicators to be used in a first\nlevel classifier aimed at maximizing precision at the expense of recall. The\nbest performing classifier for the first phase achieves 54% precision and 38%\nrecall for sarcastic utterances. We then use general syntactic patterns from\nprevious work to create more general sarcasm indicators, improving precision to\n62% and recall to 52%. To further test the generality of the method, we then\napply it to bootstrapping a classifier for nastiness dialogic acts. Our first\nphase, using crowdsourced nasty indicators, achieves 58% precision and 49%\nrecall, which increases to 75% precision and 62% recall when we bootstrap over\nthe first level with generalized syntactic patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08573v1",
            "date": "2017-08-29 02:05:56+00:00",
            "title": "Generating Different Story Tellings from Semantic Representations of Narrative",
            "authors": [
                "Elena Rishes",
                "Stephanie M. Lukin",
                "David K. Elson",
                "Marilyn A. Walker"
            ],
            "abstract": "In order to tell stories in different voices for different audiences,\ninteractive story systems require: (1) a semantic representation of story\nstructure, and (2) the ability to automatically generate story and dialogue\nfrom this semantic representation using some form of Natural Language\nGeneration (NLG). However, there has been limited research on methods for\nlinking story structures to narrative descriptions of scenes and story events.\nIn this paper we present an automatic method for converting from Scheherazade's\nstory intention graph, a semantic representation, to the input required by the\nPersonage NLG engine. Using 36 Aesop Fables distributed in DramaBank, a\ncollection of story encodings, we train translation rules on one story and then\ntest these rules by generating text for the remaining 35. The results are\nmeasured in terms of the string similarity metrics Levenshtein Distance and\nBLEU score. The results show that we can generate the 35 stories with correct\ncontent: the test set stories on average are close to the output of the\nScheherazade realizer, which was customized to this semantic representation. We\nprovide some examples of story variations generated by personage. In future\nwork, we will experiment with measuring the quality of the same stories\ngenerated in different voices, and with techniques for making storytelling\ninteractive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08575v1",
            "date": "2017-08-29 02:21:48+00:00",
            "title": "Identifying Subjective and Figurative Language in Online Dialogue",
            "authors": [
                "Stephanie M. Lukin",
                "Luke Eisenberg",
                "Thomas Corcoran",
                "Marilyn A. Walker"
            ],
            "abstract": "More and more of the information on the web is dialogic, from Facebook\nnewsfeeds, to forum conversations, to comment threads on news articles. In\ncontrast to traditional, monologic resources such as news, highly social\ndialogue is very frequent in social media. We aim to automatically identify\nsarcastic and nasty utterances in unannotated online dialogue, extending a\nbootstrapping method previously applied to the classification of monologic\nsubjective sentences in Riloff and Weibe 2003. We have adapted the method to\nfit the sarcastic and nasty dialogic domain. Our method is as follows: 1)\nExplore methods for identifying sarcastic and nasty cue words and phrases in\ndialogues; 2) Use the learned cues to train a sarcastic (nasty) Cue-Based\nClassifier; 3) Learn general syntactic extraction patterns from the sarcastic\n(nasty) utterances and define fine-tuned sarcastic patterns to create a\nPattern-Based Classifier; 4) Combine both Cue-Based and fine-tuned\nPattern-Based Classifiers to maximize precision at the expense of recall and\ntest on unannotated utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08580v1",
            "date": "2017-08-29 03:11:20+00:00",
            "title": "Generating Sentence Planning Variations for Story Telling",
            "authors": [
                "Stephanie M. Lukin",
                "Lena I. Reed",
                "Marilyn A. Walker"
            ],
            "abstract": "There has been a recent explosion in applications for dialogue interaction\nranging from direction-giving and tourist information to interactive story\nsystems. Yet the natural language generation (NLG) component for many of these\nsystems remains largely handcrafted. This limitation greatly restricts the\nrange of applications; it also means that it is impossible to take advantage of\nrecent work in expressive and statistical language generation that can\ndynamically and automatically produce a large number of variations of given\ncontent. We propose that a solution to this problem lies in new methods for\ndeveloping language generation resources. We describe the ES-Translator, a\ncomputational language generator that has previously been applied only to\nfables, and quantitatively evaluate the domain independence of the EST by\napplying it to personal narratives from weblogs. We then take advantage of\nrecent work on language generation to create a parameterized sentence planner\nfor story generation that provides aggregation operations, variations in\ndiscourse and in point of view. Finally, we present a user evaluation of\ndifferent personal narrative retellings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08585v1",
            "date": "2017-08-29 03:51:56+00:00",
            "title": "Narrative Variations in a Virtual Storyteller",
            "authors": [
                "Stephanie M. Lukin",
                "Marilyn A. Walker"
            ],
            "abstract": "Research on storytelling over the last 100 years has distinguished at least\ntwo levels of narrative representation (1) story, or fabula; and (2) discourse,\nor sujhet. We use this distinction to create Fabula Tales, a computational\nframework for a virtual storyteller that can tell the same story in different\nways through the implementation of general narratological variations, such as\nvarying direct vs. indirect speech, character voice (style), point of view, and\nfocalization. A strength of our computational framework is that it is based on\nvery general methods for re-using existing story content, either from fables or\nfrom personal narratives collected from blogs. We first explain how a simple\nannotation tool allows naive annotators to easily create a deep representation\nof fabula called a story intention graph, and show how we use this\nrepresentation to generate story tellings automatically. Then we present\nresults of two studies testing our narratological parameters, and showing that\ndifferent tellings affect the reader's perception of the story and characters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08615v1",
            "date": "2017-08-29 07:21:39+00:00",
            "title": "Comparing Human and Machine Errors in Conversational Speech Transcription",
            "authors": [
                "Andreas Stolcke",
                "Jasha Droppo"
            ],
            "abstract": "Recent work in automatic recognition of conversational telephone speech (CTS)\nhas achieved accuracy levels comparable to human transcribers, although there\nis some debate how to precisely quantify human performance on this task, using\nthe NIST 2000 CTS evaluation set. This raises the question what systematic\ndifferences, if any, may be found differentiating human from machine\ntranscription errors. In this paper we approach this question by comparing the\noutput of our most accurate CTS recognition system to that of a standard speech\ntranscription vendor pipeline. We find that the most frequent substitution,\ndeletion and insertion error types of both outputs show a high degree of\noverlap. The only notable exception is that the automatic recognizer tends to\nconfuse filled pauses (\"uh\") and backchannel acknowledgments (\"uhhuh\"). Humans\ntend not to make this error, presumably due to the distinctive and opposing\npragmatic functions attached to these words. Furthermore, we quantify the\ncorrelation between human and machine errors at the speaker level, and\ninvestigate the effect of speaker overlap between training and test data.\nFinally, we report on an informal \"Turing test\" asking humans to discriminate\nbetween automatic and human transcription error cases.",
            "journal ref": "Proc. Interspeech 2017, pp. 137-141",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08712v3",
            "date": "2017-08-29 11:56:41+00:00",
            "title": "Neural Machine Translation Training in a Multi-Domain Scenario",
            "authors": [
                "Hassan Sajjad",
                "Nadir Durrani",
                "Fahim Dalvi",
                "Yonatan Belinkov",
                "Stephan Vogel"
            ],
            "abstract": "In this paper, we explore alternative ways to train a neural machine\ntranslation system in a multi-domain scenario. We investigate data\nconcatenation (with fine tuning), model stacking (multi-level fine tuning),\ndata selection and multi-model ensemble. Our findings show that the best\ntranslation quality can be achieved by building an initial system on a\nconcatenation of available out-of-domain data and then fine-tuning it on\nin-domain data. Model stacking works best when training begins with the\nfurthest out-of-domain data and the model is incrementally fine-tuned with the\nnext furthest domain and so on. Data selection did not give the best results,\nbut can be considered as a decent compromise between training time and\ntranslation quality. A weighted ensemble of different individual models\nperformed better than data selection. It is beneficial in a scenario when there\nis no time for fine-tuning an already trained model.",
            "journal ref": "Proceedings of the 14th International Workshop on Spoken Language\n  Translation (IWSLT), 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.08959v2",
            "date": "2017-08-29 18:25:35+00:00",
            "title": "A Simple LSTM model for Transition-based Dependency Parsing",
            "authors": [
                "Mohab Elkaref",
                "Bernd Bohnet"
            ],
            "abstract": "We present a simple LSTM-based transition-based dependency parser. Our model\nis composed of a single LSTM hidden layer replacing the hidden layer in the\nusual feed-forward network architecture. We also propose a new initialization\nmethod that uses the pre-trained weights from a feed-forward neural network to\ninitialize our LSTM-based model. We also show that using dropout on the input\nlayer has a positive effect on performance. Our final parser achieves a 93.06%\nunlabeled and 91.01% labeled attachment score on the Penn Treebank. We\nadditionally replace LSTMs with GRUs and Elman units in our model and explore\nthe effectiveness of our initialization method on individual gates constituting\nall three types of RNN units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09082v1",
            "date": "2017-08-30 01:56:23+00:00",
            "title": "PersonaBank: A Corpus of Personal Narratives and Their Story Intention Graphs",
            "authors": [
                "Stephanie M. Lukin",
                "Kevin Bowden",
                "Casey Barackman",
                "Marilyn A. Walker"
            ],
            "abstract": "We present a new corpus, PersonaBank, consisting of 108 personal stories from\nweblogs that have been annotated with their Story Intention Graphs, a deep\nrepresentation of the fabula of a story. We describe the topics of the stories\nand the basis of the Story Intention Graph representation, as well as the\nprocess of annotating the stories to produce the Story Intention Graphs and the\nchallenges of adapting the tool to this new personal narrative domain We also\ndiscuss how the corpus can be used in applications that retell the story using\ndifferent styles of tellings, co-tellings, or as a content planner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09085v1",
            "date": "2017-08-30 02:01:30+00:00",
            "title": "Argument Strength is in the Eye of the Beholder: Audience Effects in Persuasion",
            "authors": [
                "Stephanie M. Lukin",
                "Pranav Anand",
                "Marilyn Walker",
                "Steve Whittaker"
            ],
            "abstract": "Americans spend about a third of their time online, with many participating\nin online conversations on social and political issues. We hypothesize that\nsocial media arguments on such issues may be more engaging and persuasive than\ntraditional media summaries, and that particular types of people may be more or\nless convinced by particular styles of argument, e.g. emotional arguments may\nresonate with some personalities while factual arguments resonate with others.\nWe report a set of experiments testing at large scale how audience variables\ninteract with argument style to affect the persuasiveness of an argument, an\nunder-researched topic within natural language processing. We show that belief\nchange is affected by personality factors, with conscientious, open and\nagreeable people being more convinced by emotional arguments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09090v1",
            "date": "2017-08-30 02:19:39+00:00",
            "title": "Automating Direct Speech Variations in Stories and Games",
            "authors": [
                "Stephanie M. Lukin",
                "James O. Ryan",
                "Marilyn A. Walker"
            ],
            "abstract": "Dialogue authoring in large games requires not only content creation but the\nsubtlety of its delivery, which can vary from character to character. Manually\nauthoring this dialogue can be tedious, time-consuming, or even altogether\ninfeasible. This paper utilizes a rich narrative representation for modeling\ndialogue and an expressive natural language generation engine for realizing it,\nand expands upon a translation tool that bridges the two. We add functionality\nto the translator to allow direct speech to be modeled by the narrative\nrepresentation, whereas the original translator supports only narratives told\nby a third person narrator. We show that we can perform character substitution\nin dialogues. We implement and evaluate a potential application to dialogue\nimplementation: generating dialogue for games with big, dynamic, or\nprocedurally-generated open worlds. We present a pilot study on human\nperceptions of the personalities of characters using direct speech, assuming\nunknown personality types at the time of authoring.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09151v1",
            "date": "2017-08-30 07:55:57+00:00",
            "title": "Paradigm Completion for Derivational Morphology",
            "authors": [
                "Ryan Cotterell",
                "Ekaterina Vylomova",
                "Huda Khayrallah",
                "Christo Kirov",
                "David Yarowsky"
            ],
            "abstract": "The generation of complex derived word forms has been an overlooked problem\nin NLP; we fill this gap by applying neural sequence-to-sequence models to the\ntask. We overview the theoretical motivation for a paradigmatic treatment of\nderivational morphology, and introduce the task of derivational paradigm\ncompletion as a parallel to inflectional paradigm completion. State-of-the-art\nneural models, adapted from the inflection task, are able to learn a range of\nderivation patterns, and outperform a non-neural baseline by 16.4%. However,\ndue to semantic, historical, and lexical considerations involved in\nderivational morphology, future work will be needed to achieve performance\nparity with inflection-generating systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09157v3",
            "date": "2017-08-30 08:14:34+00:00",
            "title": "Cross-lingual, Character-Level Neural Morphological Tagging",
            "authors": [
                "Ryan Cotterell",
                "Georg Heigold"
            ],
            "abstract": "Even for common NLP tasks, sufficient supervision is not available in many\nlanguages -- morphological tagging is no exception. In the work presented here,\nwe explore a transfer learning scheme, whereby we train character-level\nrecurrent neural taggers to predict morphological taggings for high-resource\nlanguages and low-resource languages together. Learning joint character\nrepresentations among multiple related languages successfully enables knowledge\ntransfer from the high-resource languages to the low-resource ones, improving\naccuracy by up to 30%",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09163v1",
            "date": "2017-08-30 08:32:32+00:00",
            "title": "An Empirical Study of Discriminative Sequence Labeling Models for Vietnamese Text Processing",
            "authors": [
                "Phuong Le-Hong",
                "Minh Pham Quang Nhat",
                "Thai-Hoang Pham",
                "Tuan-Anh Tran",
                "Dang-Minh Nguyen"
            ],
            "abstract": "This paper presents an empirical study of two widely-used sequence prediction\nmodels, Conditional Random Fields (CRFs) and Long Short-Term Memory Networks\n(LSTMs), on two fundamental tasks for Vietnamese text processing, including\npart-of-speech tagging and named entity recognition. We show that a strong\nlower bound for labeling accuracy can be obtained by relying only on simple\nword-based features with minimal hand-crafted feature engineering, of 90.65\\%\nand 86.03\\% performance scores on the standard test sets for the two tasks\nrespectively. In particular, we demonstrate empirically the surprising\nefficiency of word embeddings in both of the two tasks, with both of the two\nmodels. We point out that the state-of-the-art LSTMs model does not always\noutperform significantly the traditional CRFs model, especially on\nmoderate-sized data sets. Finally, we give some suggestions and discussions for\nefficient use of sequence labeling models in practical applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09217v1",
            "date": "2017-08-30 11:27:02+00:00",
            "title": "Look-ahead Attention for Generation in Neural Machine Translation",
            "authors": [
                "Long Zhou",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "The attention model has become a standard component in neural machine\ntranslation (NMT) and it guides translation process by selectively focusing on\nparts of the source sentence when predicting each target word. However, we find\nthat the generation of a target word does not only depend on the source\nsentence, but also rely heavily on the previous generated target words,\nespecially the distant words which are difficult to model by using recurrent\nneural networks. To solve this problem, we propose in this paper a novel\nlook-ahead attention mechanism for generation in NMT, which aims at directly\ncapturing the dependency relationship between target words. We further design\nthree patterns to integrate our look-ahead attention into the conventional\nattention model. Experiments on NIST Chinese-to-English and WMT\nEnglish-to-German translation tasks show that our proposed look-ahead attention\nmechanism achieves substantial improvements over state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09230v3",
            "date": "2017-08-30 12:10:20+00:00",
            "title": "TANKER: Distributed Architecture for Named Entity Recognition and Disambiguation",
            "authors": [
                "Sandro A. Coelho",
                "Diego Moussallem",
                "Gustavo C. Publio",
                "Diego Esteves"
            ],
            "abstract": "Named Entity Recognition and Disambiguation (NERD) systems have recently been\nwidely researched to deal with the significant growth of the Web. NERD systems\nare crucial for several Natural Language Processing (NLP) tasks such as\nsummarization, understanding, and machine translation. However, there is no\nstandard interface specification, i.e. these systems may vary significantly\neither for exporting their outputs or for processing the inputs. Thus, when a\ngiven company desires to implement more than one NERD system, the process is\nquite exhaustive and prone to failure. In addition, industrial solutions demand\ncritical requirements, e.g., large-scale processing, completeness, versatility,\nand licenses. Commonly, these requirements impose a limitation, making good\nNERD models to be ignored by companies. This paper presents TANKER, a\ndistributed architecture which aims to overcome scalability, reliability and\nfailure tolerance limitations related to industrial needs by combining NERD\nsystems. To this end, TANKER relies on a micro-services oriented architecture,\nwhich enables agile development and delivery of complex enterprise\napplications. In addition, TANKER provides a standardized API which makes\npossible to combine several NERD systems at once.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09234v1",
            "date": "2017-08-30 12:29:04+00:00",
            "title": "Fighting with the Sparsity of Synonymy Dictionaries",
            "authors": [
                "Dmitry Ustalov",
                "Mikhail Chernoskutov",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "Graph-based synset induction methods, such as MaxMax and Watset, induce\nsynsets by performing a global clustering of a synonymy graph. However, such\nmethods are sensitive to the structure of the input synonymy graph: sparseness\nof the input dictionary can substantially reduce the quality of the extracted\nsynsets. In this paper, we propose two different approaches designed to\nalleviate the incompleteness of the input dictionaries. The first one performs\na pre-processing of the graph by adding missing edges, while the second one\nperforms a post-processing by merging similar synset clusters. We evaluate\nthese approaches on two datasets for the Russian language and discuss their\nimpact on the performance of synset induction methods. Finally, we perform an\nextensive error analysis of each approach and discuss prominent alternative\nmethods for coping with the problem of the sparsity of the synonymy\ndictionaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09403v1",
            "date": "2017-08-30 18:01:08+00:00",
            "title": "Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set",
            "authors": [
                "Tianze Shi",
                "Liang Huang",
                "Lillian Lee"
            ],
            "abstract": "We first present a minimal feature set for transition-based dependency\nparsing, continuing a recent trend started by Kiperwasser and Goldberg (2016a)\nand Cross and Huang (2016a) of using bi-directional LSTM features. We plug our\nminimal feature set into the dynamic-programming framework of Huang and Sagae\n(2010) and Kuhlmann et al. (2011) to produce the first implementation of\nworst-case O(n^3) exact decoders for arc-hybrid and arc-eager transition\nsystems. With our minimal features, we also present O(n^3) global training\nmethods. Finally, using ensembles including our new parsers, we achieve the\nbest unlabeled attachment score reported (to our knowledge) on the Chinese\nTreebank and the \"second-best-in-class\" result on the English Penn Treebank.",
            "journal ref": "Proceedings of EMNLP, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09417v1",
            "date": "2017-08-30 18:22:28+00:00",
            "title": "LangPro: Natural Language Theorem Prover",
            "authors": [
                "Lasha Abzianidze"
            ],
            "abstract": "LangPro is an automated theorem prover for natural language\n(https://github.com/kovvalsky/LangPro). Given a set of premises and a\nhypothesis, it is able to prove semantic relations between them. The prover is\nbased on a version of analytic tableau method specially designed for natural\nlogic. The proof procedure operates on logical forms that preserve linguistic\nexpressions to a large extent. %This property makes the logical forms easily\nobtainable from syntactic trees. %, in particular, Combinatory Categorial\nGrammar derivation trees. The nature of proofs is deductive and transparent. On\nthe FraCaS and SICK textual entailment datasets, the prover achieves high\nresults comparable to state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09496v1",
            "date": "2017-08-30 22:51:08+00:00",
            "title": "Inferring Narrative Causality between Event Pairs in Films",
            "authors": [
                "Zhichao Hu",
                "Marilyn A. Walker"
            ],
            "abstract": "To understand narrative, humans draw inferences about the underlying\nrelations between narrative events. Cognitive theories of narrative\nunderstanding define these inferences as four different types of causality,\nthat include pairs of events A, B where A physically causes B (X drop, X\nbreak), to pairs of events where A causes emotional state B (Y saw X, Y felt\nfear). Previous work on learning narrative relations from text has either\nfocused on \"strict\" physical causality, or has been vague about what relation\nis being learned. This paper learns pairs of causal events from a corpus of\nfilm scene descriptions which are action rich and tend to be told in\nchronological order. We show that event pairs induced using our methods are of\nhigh quality and are judged to have a stronger causal relation than event pairs\nfrom Rel-grams.",
            "journal ref": "Proceedings of the SIGDIAL 2017 Conference pages 342-351",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09497v1",
            "date": "2017-08-30 23:02:06+00:00",
            "title": "Unsupervised Induction of Contingent Event Pairs from Film Scenes",
            "authors": [
                "Zhichao Hu",
                "Elahe Rahimtoroghi",
                "Larissa Munishkina",
                "Reid Swanson",
                "Marilyn A. Walker"
            ],
            "abstract": "Human engagement in narrative is partially driven by reasoning about\ndiscourse relations between narrative events, and the expectations about what\nis likely to happen next that results from such reasoning. Researchers in NLP\nhave tackled modeling such expectations from a range of perspectives, including\ntreating it as the inference of the contingent discourse relation, or as a type\nof common-sense causal reasoning. Our approach is to model likelihood between\nevents by drawing on several of these lines of previous work. We implement and\nevaluate different unsupervised methods for learning event pairs that are\nlikely to be contingent on one another. We refine event pairs that we learn\nfrom a corpus of film scene descriptions utilizing web search counts, and\nevaluate our results by collecting human judgments of contingency. Our results\nindicate that the use of web search counts increases the average accuracy of\nour best method to 85.64% over a baseline of 50%, as compared to an average\naccuracy of 75.15% without web search.",
            "journal ref": "In Proceedings of Conference on Empirical Methods in Natural\n  Language Processing (EMNLP 2013)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09609v1",
            "date": "2017-08-31 08:18:12+00:00",
            "title": "Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation",
            "authors": [
                "Greg Durrett",
                "Jonathan K. Kummerfeld",
                "Taylor Berg-Kirkpatrick",
                "Rebecca S. Portnoff",
                "Sadia Afroz",
                "Damon McCoy",
                "Kirill Levchenko",
                "Vern Paxson"
            ],
            "abstract": "One weakness of machine-learned NLP models is that they typically perform\npoorly on out-of-domain data. In this work, we study the task of identifying\nproducts being bought and sold in online cybercrime forums, which exhibits\nparticularly challenging cross-domain effects. We formulate a task that\nrepresents a hybrid of slot-filling information extraction and named entity\nrecognition and annotate data from four different forums. Each of these forums\nconstitutes its own \"fine-grained domain\" in that the forums cover different\nmarket sectors with different properties, even though all forums are in the\nbroad domain of cybercrime. We characterize these domain differences in the\ncontext of a learning-based system: supervised models see decreased accuracy\nwhen applied to new forums, and standard techniques for semi-supervised\nlearning and domain adaptation have limited effectiveness on this data, which\nsuggests the need to improve these techniques. We release a dataset of 1,938\nannotated posts from across the four forums.",
            "journal ref": "EMNLP (2017) 2598-2607",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09702v1",
            "date": "2017-08-31 13:33:04+00:00",
            "title": "Human and Machine Judgements for Russian Semantic Relatedness",
            "authors": [
                "Alexander Panchenko",
                "Dmitry Ustalov",
                "Nikolay Arefyev",
                "Denis Paperno",
                "Natalia Konstantinova",
                "Natalia Loukachevitch",
                "Chris Biemann"
            ],
            "abstract": "Semantic relatedness of terms represents similarity of meaning by a numerical\nscore. On the one hand, humans easily make judgments about semantic\nrelatedness. On the other hand, this kind of information is useful in language\nprocessing systems. While semantic relatedness has been extensively studied for\nEnglish using numerous language resources, such as associative norms, human\njudgments, and datasets generated from lexical databases, no evaluation\nresources of this kind have been available for Russian to date. Our\ncontribution addresses this problem. We present five language resources of\ndifferent scale and purpose for Russian semantic relatedness, each being a list\nof triples (word_i, word_j, relatedness_ij). Four of them are designed for\nevaluation of systems for computing semantic relatedness, complementing each\nother in terms of the semantic relation type they represent. These benchmarks\nwere used to organize a shared task on Russian semantic relatedness, which\nattracted 19 teams. We use one of the best approaches identified in this\ncompetition to generate the fifth high-coverage resource, the first open\ndistributional thesaurus of Russian. Multiple evaluations of this thesaurus,\nincluding a large-scale crowdsourcing study involving native speakers, indicate\nits high accuracy.",
            "journal ref": "In Proceedings of the 5th Conference on Analysis of Images, Social\n  Networks, and Texts (AIST'2016): Springer Communications in Computer and\n  Information Sciences (CCIS)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09789v1",
            "date": "2017-08-31 16:04:26+00:00",
            "title": "Learning Lexico-Functional Patterns for First-Person Affect",
            "authors": [
                "Lena Reed",
                "Jiaqi Wu",
                "Shereen Oraby",
                "Pranav Anand",
                "Marilyn Walker"
            ],
            "abstract": "Informal first-person narratives are a unique resource for computational\nmodels of everyday events and people's affective reactions to them. People\nblogging about their day tend not to explicitly say I am happy. Instead they\ndescribe situations from which other humans can readily infer their affective\nreactions. However current sentiment dictionaries are missing much of the\ninformation needed to make similar inferences. We build on recent work that\nmodels affect in terms of lexical predicate functions and affect on the\npredicate's arguments. We present a method to learn proxies for these functions\nfrom first-person narratives. We construct a novel fine-grained test set, and\nshow that the patterns we learn improve our ability to predict first-person\naffective reactions to everyday events, from a Stanford sentiment baseline of\n.67F to .75F.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09803v2",
            "date": "2017-08-31 16:34:38+00:00",
            "title": "Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation",
            "authors": [
                "Toan Q. Nguyen",
                "David Chiang"
            ],
            "abstract": "We present a simple method to improve neural translation of a low-resource\nlanguage pair using parallel data from a related, also low-resource, language\npair. The method is based on the transfer method of Zoph et al., but whereas\ntheir method ignores any source vocabulary overlap, ours exploits it. First, we\nsplit words using Byte Pair Encoding (BPE) to increase vocabulary overlap.\nThen, we train a model on the first language pair and transfer its parameters,\nincluding its source word embeddings, to another model and continue training on\nthe second language pair. Our experiments show that transfer learning helps\nword-based translation only slightly, but when used on top of a much stronger\nBPE baseline, it yields larger improvements of up to 4.3 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00164v1",
            "date": "2017-09-30 08:46:03+00:00",
            "title": "Speaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning",
            "authors": [
                "Ta-Chung Chi",
                "Po-Chun Chen",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Language understanding (LU) and dialogue policy learning are two essential\ncomponents in conversational systems. Human-human dialogues are not\nwell-controlled and often random and unpredictable due to their own goals and\nspeaking habits. This paper proposes a role-based contextual model to consider\ndifferent speaker roles independently based on the various speaking patterns in\nthe multi-turn dialogues. The experiments on the benchmark dataset show that\nthe proposed role-based model successfully learns role-specific behavioral\npatterns for contextual encoding and then significantly improves language\nunderstanding and dialogue policy learning tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00165v2",
            "date": "2017-09-30 08:50:16+00:00",
            "title": "Dynamic Time-Aware Attention to Speaker Roles and Contexts for Spoken Language Understanding",
            "authors": [
                "Po-Chun Chen",
                "Ta-Chung Chi",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Spoken language understanding (SLU) is an essential component in\nconversational systems. Most SLU component treats each utterance independently,\nand then the following components aggregate the multi-turn information in the\nseparate phases. In order to avoid error propagation and effectively utilize\ncontexts, prior work leveraged history for contextual SLU. However, the\nprevious model only paid attention to the content in history utterances without\nconsidering their temporal information and speaker roles. In the dialogues, the\nmost recent utterances should be more important than the least recent ones.\nFurthermore, users usually pay attention to 1) self history for reasoning and\n2) others' utterances for listening, the speaker of the utterances may provides\ninformative cues to help understanding. Therefore, this paper proposes an\nattention-based network that additionally leverages temporal information and\nspeaker role for better SLU, where the attention to contexts and speaker roles\ncan be automatically learned in an end-to-end manner. The experiments on the\nbenchmark Dialogue State Tracking Challenge 4 (DSTC4) dataset show that the\ntime-aware dynamic role attention networks significantly improve the\nunderstanding performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00205v1",
            "date": "2017-09-30 14:21:12+00:00",
            "title": "Bag-of-Vector Embeddings of Dependency Graphs for Semantic Induction",
            "authors": [
                "Diana Nicoleta Popa",
                "James Henderson"
            ],
            "abstract": "Vector-space models, from word embeddings to neural network parsers, have\nmany advantages for NLP. But how to generalise from fixed-length word vectors\nto a vector space for arbitrary linguistic structures is still unclear. In this\npaper we propose bag-of-vector embeddings of arbitrary linguistic graphs. A\nbag-of-vector space is the minimal nonparametric extension of a vector space,\nallowing the representation to grow with the size of the graph, but not tying\nthe representation to any specific tree or graph structure. We propose\nefficient training and inference algorithms based on tensor factorisation for\nembedding arbitrary graphs in a bag-of-vector space. We demonstrate the\nusefulness of this representation by training bag-of-vector embeddings of\ndependency graphs and evaluating them on unsupervised semantic induction for\nthe Semantic Textual Similarity and Natural Language Inference tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00273v1",
            "date": "2017-10-01 00:04:10+00:00",
            "title": "What Words Do We Use to Lie?: Word Choice in Deceptive Messages",
            "authors": [
                "Jason Dou",
                "Michelle Liu",
                "Haaris Muneer",
                "Adam Schlussel"
            ],
            "abstract": "Text messaging is the most widely used form of computer- mediated\ncommunication (CMC). Previous findings have shown that linguistic factors can\nreliably indicate messages as deceptive. For example, users take longer and use\nmore words to craft deceptive messages than they do truthful messages. Existing\nresearch has also examined how factors, such as student status and gender,\naffect rates of deception and word choice in deceptive messages. However, this\nresearch has been limited by small sample sizes and has returned contradicting\nfindings. This paper aims to address these issues by using a dataset of text\nmessages collected from a large and varied set of participants using an Android\nmessaging application. The results of this paper show significant differences\nin word choice and frequency of deceptive messages between male and female\nparticipants, as well as between students and non-students.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00341v1",
            "date": "2017-10-01 12:54:50+00:00",
            "title": "Fully Automated Fact Checking Using External Sources",
            "authors": [
                "Georgi Karadzhov",
                "Preslav Nakov",
                "Lluis Marquez",
                "Alberto Barron-Cedeno",
                "Ivan Koychev"
            ],
            "abstract": "Given the constantly growing proliferation of false claims online in recent\nyears, there has been also a growing research interest in automatically\ndistinguishing false rumors from factually true claims. Here, we propose a\ngeneral-purpose framework for fully-automatic fact checking using external\nsources, tapping the potential of the entire Web as a knowledge source to\nconfirm or reject a claim. Our framework uses a deep neural network with LSTM\ntext encoding to combine semantic kernels with task-specific embeddings that\nencode a claim together with pieces of potentially-relevant text fragments from\nthe Web, taking the source reliability into account. The evaluation results\nshow good performance on two different tasks and datasets: (i) rumor detection\nand (ii) fact checking of the answers to a question in community question\nanswering forums.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00346v1",
            "date": "2017-10-01 13:18:48+00:00",
            "title": "Robust Tuning Datasets for Statistical Machine Translation",
            "authors": [
                "Preslav Nakov",
                "Stephan Vogel"
            ],
            "abstract": "We explore the idea of automatically crafting a tuning dataset for\nStatistical Machine Translation (SMT) that makes the hyper-parameters of the\nSMT system more robust with respect to some specific deficiencies of the\nparameter tuning algorithms. This is an under-explored research direction,\nwhich can allow better parameter tuning. In this paper, we achieve this goal by\nselecting a subset of the available sentence pairs, which are more suitable for\nspecific combinations of optimizers, objective functions, and evaluation\nmeasures. We demonstrate the potential of the idea with the pairwise ranking\noptimization (PRO) optimizer, which is known to yield too short translations.\nWe show that the learning problem can be alleviated by tuning on a subset of\nthe development set, selected based on sentence length. In particular, using\nthe longest 50% of the tuning sentences, we achieve two-fold tuning speedup,\nand improvements in BLEU score that rival those of alternatives, which fix\nBLEU+1's smoothing instead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00453v1",
            "date": "2017-10-02 01:52:05+00:00",
            "title": "Visual Reasoning with Natural Language",
            "authors": [
                "Stephanie Zhou",
                "Alane Suhr",
                "Yoav Artzi"
            ],
            "abstract": "Natural language provides a widely accessible and expressive interface for\nrobotic agents. To understand language in complex environments, agents must\nreason about the full range of language inputs and their correspondence to the\nworld. Such reasoning over language and vision is an open problem that is\nreceiving increasing attention. While existing data sets focus on visual\ndiversity, they do not display the full range of natural language expressions,\nsuch as counting, set reasoning, and comparisons.\n  We propose a simple task for natural language visual reasoning, where images\nare paired with descriptive statements. The task is to predict if a statement\nis true for the given scene. This abstract describes our existing synthetic\nimages corpus and our current work on collecting real vision data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00477v4",
            "date": "2017-10-02 04:16:36+00:00",
            "title": "A Crowd-Annotated Spanish Corpus for Humor Analysis",
            "authors": [
                "Santiago Castro",
                "Luis Chiruzzo",
                "Aiala Ros\u00e1",
                "Diego Garat",
                "Guillermo Moncecchi"
            ],
            "abstract": "Computational Humor involves several tasks, such as humor recognition, humor\ngeneration, and humor scoring, for which it is useful to have human-curated\ndata. In this work we present a corpus of 27,000 tweets written in Spanish and\ncrowd-annotated by their humor value and funniness score, with about four\nannotations per tweet, tagged by 1,300 people over the Internet. It is equally\ndivided between tweets coming from humorous and non-humorous accounts. The\ninter-annotator agreement Krippendorff's alpha value is 0.5710. The dataset is\navailable for general use and can serve as a basis for humor detection and as a\nfirst step to tackle subjectivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00519v2",
            "date": "2017-10-02 07:58:19+00:00",
            "title": "Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In NLP, convolutional neural networks (CNNs) have benefited less than\nrecurrent neural networks (RNNs) from attention mechanisms. We hypothesize that\nthis is because the attention in CNNs has been mainly implemented as attentive\npooling (i.e., it is applied to pooling) rather than as attentive convolution\n(i.e., it is integrated into convolution). Convolution is the differentiator of\nCNNs in that it can powerfully model the higher-level representation of a word\nby taking into account its local fixed-size context in the input text t^x. In\nthis work, we propose an attentive convolution network, ATTCONV. It extends the\ncontext scope of the convolution operation, deriving higher-level features for\na word not only from local context, but also information extracted from\nnonlocal context by the attention mechanism commonly used in RNNs. This\nnonlocal context can come (i) from parts of the input text t^x that are distant\nor (ii) from extra (i.e., external) contexts t^y. Experiments on sentence\nmodeling with zero-context (sentiment analysis), single-context (textual\nentailment) and multiple-context (claim verification) demonstrate the\neffectiveness of ATTCONV in sentence representation learning with the\nincorporation of context. In particular, attentive convolution outperforms\nattentive pooling and is a strong competitor to popular attentive RNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00689v1",
            "date": "2017-10-02 14:34:25+00:00",
            "title": "Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics",
            "authors": [
                "Martin Boyanov",
                "Ivan Koychev",
                "Preslav Nakov",
                "Alessandro Moschitti",
                "Giovanni Da San Martino"
            ],
            "abstract": "We propose to use question answering (QA) data from Web forums to train\nchatbots from scratch, i.e., without dialog training data. First, we extract\npairs of question and answer sentences from the typically much longer texts of\nquestions and answers in a forum. We then use these shorter texts to train\nseq2seq models in a more efficient way. We further improve the parameter\noptimization using a new model selection strategy based on QA measures.\nFinally, we propose to use extrinsic evaluation with respect to a QA task as an\nautomatic evaluation method for chatbots. The evaluation shows that the model\nachieves a MAP of 63.5% on the extrinsic task. Moreover, it can answer\ncorrectly 49.5% of the questions when they are similar to questions asked in\nthe forum, and 47.3% of the questions when they are more conversational in\nstyle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00803v1",
            "date": "2017-10-02 17:18:37+00:00",
            "title": "Compiling and Processing Historical and Contemporary Portuguese Corpora",
            "authors": [
                "Marcos Zampieri"
            ],
            "abstract": "This technical report describes the framework used for processing three large\nPortuguese corpora. Two corpora contain texts from newspapers, one published in\nBrazil and the other published in Portugal. The third corpus is Colonia, a\nhistorical Portuguese collection containing texts written between the 16th and\nthe early 20th century. The report presents pre-processing methods,\nsegmentation, and annotation of the corpora as well as indexing and querying\nmethods. Finally, it presents published research papers using the corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00880v3",
            "date": "2017-10-02 19:33:46+00:00",
            "title": "Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection",
            "authors": [
                "Haw-Shiuan Chang",
                "ZiYun Wang",
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "Modeling hypernymy, such as poodle is-a dog, is an important generalization\naid to many NLP tasks, such as entailment, coreference, relation extraction,\nand question answering. Supervised learning from labeled hypernym sources, such\nas WordNet, limits the coverage of these models, which can be addressed by\nlearning hypernyms from unlabeled text. Existing unsupervised methods either do\nnot scale to large vocabularies or yield unacceptably poor accuracy. This paper\nintroduces distributional inclusion vector embedding (DIVE), a\nsimple-to-implement unsupervised method of hypernym discovery via per-word\nnon-negative vector embeddings which preserve the inclusion property of word\ncontexts in a low-dimensional and interpretable space. In experimental\nevaluations more comprehensive than any previous literature of which we are\naware-evaluating on 11 datasets using multiple existing as well as newly\nproposed scoring functions-we find that our method provides up to double the\nprecision of previous unsupervised embeddings, and the highest average\nperformance, using a much more compact word representation, and yielding many\nnew state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00923v1",
            "date": "2017-10-02 21:56:16+00:00",
            "title": "Minimal Dependency Translation: a Framework for Computer-Assisted Translation for Under-Resourced Languages",
            "authors": [
                "Michael Gasser"
            ],
            "abstract": "This paper introduces Minimal Dependency Translation (MDT), an ongoing\nproject to develop a rule-based framework for the creation of rudimentary\nbilingual lexicon-grammars for machine translation and computer-assisted\ntranslation into and out of under-resourced languages as well as initial steps\ntowards an implementation of MDT for English-to-Amharic translation. The basic\nunits in MDT, called groups, are headed multi-item sequences. In addition to\nwordforms, groups may contain lexemes, syntactic-semantic categories, and\ngrammatical features. Each group is associated with one or more translations,\neach of which is a group in a target language. During translation, constraint\nsatisfaction is used to select a set of source-language groups for the input\nsentence and to sequence the words in the associated target-language groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00936v1",
            "date": "2017-10-02 23:02:17+00:00",
            "title": "Identifying Nominals with No Head Match Co-references Using Deep Learning",
            "authors": [
                "M. Stone",
                "R. Arora"
            ],
            "abstract": "Identifying nominals with no head match is a long-standing challenge in\ncoreference resolution with current systems performing significantly worse than\nhumans. In this paper we present a new neural network architecture which\noutperforms the current state-of-the-art system on the English portion of the\nCoNLL 2012 Shared Task. This is done by using a logistic regression on features\nproduced by two submodels, one of which is has the architecture proposed in\n[CM16a] while the other combines domain specific embeddings of the antecedent\nand the mention. We also propose some simple additional features which seem to\nimprove performance for all models substantially, increasing F1 by almost 4% on\nbasic logistic regression and other complex models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00969v1",
            "date": "2017-10-03 03:24:28+00:00",
            "title": "Event Identification as a Decision Process with Non-linear Representation of Text",
            "authors": [
                "Yukun Yan",
                "Daqi Zheng",
                "Zhengdong Lu",
                "Sen Song"
            ],
            "abstract": "We propose scale-free Identifier Network(sfIN), a novel model for event\nidentification in documents. In general, sfIN first encodes a document into\nmulti-scale memory stacks, then extracts special events via conducting\nmulti-scale actions, which can be considered as a special type of sequence\nlabelling. The design of large scale actions makes it more efficient processing\na long document. The whole model is trained with both supervised learning and\nreinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00987v1",
            "date": "2017-10-03 05:19:12+00:00",
            "title": "Annotation and Detection of Emotion in Text-based Dialogue Systems with CNN",
            "authors": [
                "Jialiang Zhao",
                "Qi Gao"
            ],
            "abstract": "Knowledge of users' emotion states helps improve human-computer interaction.\nIn this work, we presented EmoNet, an emotion detector of Chinese daily\ndialogues based on deep convolutional neural networks. In order to maintain the\noriginal linguistic features, such as the order, commonly used methods like\nsegmentation and keywords extraction were not adopted, instead we increased the\ndepth of CNN and tried to let CNN learn inner linguistic relationships. Our\nmain contribution is that we presented a new model and a new pipeline which can\nbe used in multi-language environment to solve sentimental problems.\nExperimental results shows EmoNet has a great capacity in learning the emotion\nof dialogues and achieves a better result than other state of art detectors do.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00998v1",
            "date": "2017-10-03 06:14:31+00:00",
            "title": "Is Structure Necessary for Modeling Argument Expectations in Distributional Semantics?",
            "authors": [
                "Emmanuele Chersoni",
                "Enrico Santus",
                "Philippe Blache",
                "Alessandro Lenci"
            ],
            "abstract": "Despite the number of NLP studies dedicated to thematic fit estimation,\nlittle attention has been paid to the related task of composing and updating\nverb argument expectations. The few exceptions have mostly modeled this\nphenomenon with structured distributional models, implicitly assuming a\nsimilarly structured representation of events. Recent experimental evidence,\nhowever, suggests that human processing system could also exploit an\nunstructured \"bag-of-arguments\" type of event representation to predict\nupcoming input. In this paper, we re-implement a traditional structured model\nand adapt it to compare the different hypotheses concerning the degree of\nstructure in our event knowledge, evaluating their relative performance in the\ntask of the argument expectations update.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01025v3",
            "date": "2017-10-03 08:19:24+00:00",
            "title": "MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processing",
            "authors": [
                "Raj Dabre",
                "Sadao Kurohashi"
            ],
            "abstract": "Multilinguality is gradually becoming ubiquitous in the sense that more and\nmore researchers have successfully shown that using additional languages help\nimprove the results in many Natural Language Processing tasks. Multilingual\nMultiway Corpora (MMC) contain the same sentence in multiple languages. Such\ncorpora have been primarily used for Multi-Source and Pivot Language Machine\nTranslation but are also useful for developing multilingual sequence taggers by\ntransfer learning. While these corpora are available, they are not organized\nfor multilingual experiments and researchers need to write boilerplate code\nevery time they want to use said corpora. Moreover, because there is no\nofficial MMC collection it becomes difficult to compare against existing\napproaches. As such we present our work on creating a unified and\nsystematically organized repository of MMC spanning a large number of\nlanguages. We also provide training, development and test splits for corpora\nwhere official splits are unavailable. We hope that this will help speed up the\npace of multilingual NLP research and ensure that NLP researchers obtain\nresults that are more trustable since they can be compared easily. We indicate\ncorpora sources, extraction procedures if any and relevant statistics. We also\nmake our collection public for research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01095v1",
            "date": "2017-10-03 11:48:37+00:00",
            "title": "Towards an Inferential Lexicon of Event Selecting Predicates for French",
            "authors": [
                "Ingrid Falk",
                "Fabienne Martin"
            ],
            "abstract": "We present a manually constructed seed lexicon encoding the inferential\nprofiles of French event selecting predicates across different uses. The\ninferential profile (Karttunen, 1971a) of a verb is designed to capture the\ninferences triggered by the use of this verb in context. It reflects the\ninfluence of the clause-embedding verb on the factuality of the event described\nby the embedded clause. The resource developed provides evidence for the\nfollowing three hypotheses: (i) French implicative verbs have an aspect\ndependent profile (their inferential profile varies with outer aspect), while\nfactive verbs have an aspect independent profile (they keep the same\ninferential profile with both imperfective and perfective aspect); (ii)\nimplicativity decreases with imperfective aspect: the inferences triggered by\nFrench implicative verbs combined with perfective aspect are often weakened\nwhen the same verbs are combined with imperfective aspect; (iii) implicativity\ndecreases with an animate (deep) subject: the inferences triggered by a verb\nwhich is implicative with an inanimate subject are weakened when the same verb\nis used with an animate subject. The resource additionally shows that verbs\nwith different inferential profiles display clearly distinct sub-categorisation\npatterns. In particular, verbs that have both factive and implicative readings\nare shown to prefer infinitival clauses in their implicative reading, and\ntensed clauses in their factive reading.",
            "journal ref": "International Conference on Computational Semantics, Sep 2017,\n  Montpellier, France. 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01329v3",
            "date": "2017-10-03 18:15:10+00:00",
            "title": "Improving Lexical Choice in Neural Machine Translation",
            "authors": [
                "Toan Q. Nguyen",
                "David Chiang"
            ],
            "abstract": "We explore two solutions to the problem of mistranslating rare words in\nneural machine translation. First, we argue that the standard output layer,\nwhich computes the inner product of a vector representing the context with all\npossible output word embeddings, rewards frequent words disproportionately, and\nwe propose to fix the norms of both vectors to a constant value. Second, we\nintegrate a simple lexical module which is jointly trained with the rest of the\nmodel. We evaluate our approaches on eight language pairs with data sizes\nranging from 100k to 8M words, and achieve improvements of up to +4.3 BLEU,\nsurpassing phrase-based translation in nearly all settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01411v1",
            "date": "2017-10-03 22:40:05+00:00",
            "title": "Transferring Semantic Roles Using Translation and Syntactic Information",
            "authors": [
                "Maryam Aminian",
                "Mohammad Sadegh Rasooli",
                "Mona Diab"
            ],
            "abstract": "Our paper addresses the problem of annotation projection for semantic role\nlabeling for resource-poor languages using supervised annotations from a\nresource-rich language through parallel data. We propose a transfer method that\nemploys information from source and target syntactic dependencies as well as\nword alignment density to improve the quality of an iterative bootstrapping\nmethod. Our experiments yield a $3.5$ absolute labeled F-score improvement over\na standard annotation projection method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01487v1",
            "date": "2017-10-04 07:23:23+00:00",
            "title": "Cross-Language Question Re-Ranking",
            "authors": [
                "Giovanni Da San Martino",
                "Salvatore Romeo",
                "Alberto Barron-Cedeno",
                "Shafiq Joty",
                "Lluis Marquez",
                "Alessandro Moschitti",
                "Preslav Nakov"
            ],
            "abstract": "We study how to find relevant questions in community forums when the language\nof the new questions is different from that of the existing questions in the\nforum. In particular, we explore the Arabic-English language pair. We compare a\nkernel-based system with a feed-forward neural network in a scenario where a\nlarge parallel corpus is available for training a machine translation system,\nbilingual dictionaries, and cross-language word embeddings. We observe that\nboth approaches degrade the performance of the system when working on the\ntranslated text, especially the kernel-based system, which depends heavily on a\nsyntactic kernel. We address this issue using a cross-language tree kernel,\nwhich compares the original Arabic tree to the English trees of the related\nquestions. We show that this kernel almost closes the performance gap with\nrespect to the monolingual system. On the neural network side, we use the\nparallel corpus to train cross-language embeddings, which we then use to\nrepresent the Arabic input and the English related questions in the same space.\nThe results also improve to close to those of the monolingual neural network.\nOverall, the kernel system shows a better performance compared to the neural\nnetwork in all cases.",
            "journal ref": "SIGIR 2017: 1145-1148",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01492v1",
            "date": "2017-10-04 07:57:59+00:00",
            "title": "Semantic Sentiment Analysis of Twitter Data",
            "authors": [
                "Preslav Nakov"
            ],
            "abstract": "Internet and the proliferation of smart mobile devices have changed the way\ninformation is created, shared, and spreads, e.g., microblogs such as Twitter,\nweblogs such as LiveJournal, social networks such as Facebook, and instant\nmessengers such as Skype and WhatsApp are now commonly used to share thoughts\nand opinions about anything in the surrounding world. This has resulted in the\nproliferation of social media content, thus creating new opportunities to study\npublic opinion at a scale that was never possible before. Naturally, this\nabundance of data has quickly attracted business and research interest from\nvarious fields including marketing, political science, and social studies,\namong many others, which are interested in questions like these: Do people like\nthe new Apple Watch? Do Americans support ObamaCare? How do Scottish feel about\nthe Brexit? Answering these questions requires studying the sentiment of\nopinions people express in social media, which has given rise to the fast\ngrowth of the field of sentiment analysis in social media, with Twitter being\nespecially popular for research due to its scale, representativeness, variety\nof topics discussed, as well as ease of public access to its messages. Here we\npresent an overview of work on sentiment analysis on Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01504v1",
            "date": "2017-10-04 08:28:24+00:00",
            "title": "Discourse Structure in Machine Translation Evaluation",
            "authors": [
                "Shafiq Joty",
                "Francisco Guzm\u00e1n",
                "Llu\u00eds M\u00e0rquez",
                "Preslav Nakov"
            ],
            "abstract": "In this article, we explore the potential of using sentence-level discourse\nstructure for machine translation evaluation. We first design discourse-aware\nsimilarity measures, which use all-subtree kernels to compare discourse parse\ntrees in accordance with the Rhetorical Structure Theory (RST). Then, we show\nthat a simple linear combination with these measures can help improve various\nexisting machine translation evaluation metrics regarding correlation with\nhuman judgments both at the segment- and at the system-level. This suggests\nthat discourse information is complementary to the information used by many of\nthe existing evaluation metrics, and thus it could be taken into account when\ndeveloping richer evaluation metrics, such as the WMT-14 winning combined\nmetric DiscoTKparty. We also provide a detailed analysis of the relevance of\nvarious discourse elements and relations from the RST parse trees for machine\ntranslation evaluation. In particular we show that: (i) all aspects of the RST\ntree are relevant, (ii) nuclearity is more useful than relation type, and (iii)\nthe similarity of the translation RST tree to the reference tree is positively\ncorrelated with translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01779v2",
            "date": "2017-10-04 19:42:37+00:00",
            "title": "Building a Web-Scale Dependency-Parsed Corpus from CommonCrawl",
            "authors": [
                "Alexander Panchenko",
                "Eugen Ruppert",
                "Stefano Faralli",
                "Simone Paolo Ponzetto",
                "Chris Biemann"
            ],
            "abstract": "We present DepCC, the largest-to-date linguistically analyzed corpus in\nEnglish including 365 million documents, composed of 252 billion tokens and 7.5\nbillion of named entity occurrences in 14.3 billion sentences from a web-scale\ncrawl of the \\textsc{Common Crawl} project. The sentences are processed with a\ndependency parser and with a named entity tagger and contain provenance\ninformation, enabling various applications ranging from training syntax-based\nword embeddings to open information extraction and question answering. We built\nan index of all sentences and their linguistic meta-data enabling quick search\nacross the corpus. We demonstrate the utility of this corpus on the verb\nsimilarity task by showing that a distributional model trained on our corpus\nyields better results than models trained on smaller corpora, like Wikipedia.\nThis distributional model outperforms the state of art models of verb\nsimilarity trained on smaller corpora on the SimVerb3500 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01789v1",
            "date": "2017-10-04 20:13:43+00:00",
            "title": "Enhanced Neural Machine Translation by Learning from Draft",
            "authors": [
                "Aodong Li",
                "Shiyue Zhang",
                "Dong Wang",
                "Thomas Fang Zheng"
            ],
            "abstract": "Neural machine translation (NMT) has recently achieved impressive results. A\npotential problem of the existing NMT algorithm, however, is that the decoding\nis conducted from left to right, without considering the right context. This\npaper proposes an two-stage approach to solve the problem. In the first stage,\na conventional attention-based NMT system is used to produce a draft\ntranslation, and in the second stage, a novel double-attention NMT system is\nused to refine the translation, by looking at the original input as well as the\ndraft translation. This drafting-and-refinement can obtain the right-context\ninformation from the draft, hence producing more consistent translations. We\nevaluated this approach using two Chinese-English translation tasks, one with\n44k pairs and 1M pairs respectively. The experiments showed that our approach\nachieved positive improvements over the conventional NMT system: the\nimprovements are 2.4 and 0.9 BLEU points on the small-scale and large-scale\ntasks, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01799v1",
            "date": "2017-10-04 20:49:52+00:00",
            "title": "Counterfactual Language Model Adaptation for Suggesting Phrases",
            "authors": [
                "Kenneth C. Arnold",
                "Kai-Wei Chang",
                "Adam T. Kalai"
            ],
            "abstract": "Mobile devices use language models to suggest words and phrases for use in\ntext entry. Traditional language models are based on contextual word frequency\nin a static corpus of text. However, certain types of phrases, when offered to\nwriters as suggestions, may be systematically chosen more often than their\nfrequency would predict. In this paper, we propose the task of generating\nsuggestions that writers accept, a related but distinct task to making accurate\npredictions. Although this task is fundamentally interactive, we propose a\ncounterfactual setting that permits offline training and evaluation. We find\nthat even a simple language model can capture text characteristics that improve\nacceptability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01809v1",
            "date": "2017-10-04 21:21:30+00:00",
            "title": "Syntactic and Semantic Features For Code-Switching Factored Language Models",
            "authors": [
                "Heike Adel",
                "Ngoc Thang Vu",
                "Katrin Kirchhoff",
                "Dominic Telaar",
                "Tanja Schultz"
            ],
            "abstract": "This paper presents our latest investigations on different features for\nfactored language models for Code-Switching speech and their effect on\nautomatic speech recognition (ASR) performance. We focus on syntactic and\nsemantic features which can be extracted from Code-Switching text data and\nintegrate them into factored language models. Different possible factors, such\nas words, part-of-speech tags, Brown word clusters, open class words and\nclusters of open class word embeddings are explored. The experimental results\nreveal that Brown word clusters, part-of-speech tags and open-class words are\nthe most effective at reducing the perplexity of factored language models on\nthe Mandarin-English Code-Switching corpus SEAME. In ASR experiments, the model\ncontaining Brown word clusters and part-of-speech tags and the model also\nincluding clusters of open class word embeddings yield the best mixed error\nrate results. In summary, the best language model can significantly reduce the\nperplexity on the SEAME evaluation set by up to 10.8% relative and the mixed\nerror rate by up to 3.4% relative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.01977v1",
            "date": "2017-10-05 11:54:34+00:00",
            "title": "Machine Learning Based Detection of Clickbait Posts in Social Media",
            "authors": [
                "Xinyue Cao",
                "Thai Le",
                "Jason",
                "Zhang"
            ],
            "abstract": "Clickbait (headlines) make use of misleading titles that hide critical\ninformation from or exaggerate the content on the landing target pages to\nentice clicks. As clickbaits often use eye-catching wording to attract viewers,\ntarget contents are often of low quality. Clickbaits are especially widespread\non social media such as Twitter, adversely impacting user experience by causing\nimmense dissatisfaction. Hence, it has become increasingly important to put\nforward a widely applicable approach to identify and detect clickbaits. In this\npaper, we make use of a dataset from the clickbait challenge 2017\n(clickbait-challenge.com) comprising of over 21,000 headlines/titles, each of\nwhich is annotated by at least five judgments from crowdsourcing on how\nclickbait it is. We attempt to build an effective computational clickbait\ndetection model on this dataset. We first considered a total of 331 features,\nfiltered out many features to avoid overfitting and improve the running time of\nlearning, and eventually selected the 60 most important features for our final\nmodel. Using these features, Random Forest Regression achieved the following\nresults: MSE=0.035 MSE, Accuracy=0.82, and F1-sore=0.61 on the clickbait class.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02076v1",
            "date": "2017-10-05 15:29:33+00:00",
            "title": "On the Effective Use of Pretraining for Natural Language Inference",
            "authors": [
                "Ignacio Cases",
                "Minh-Thang Luong",
                "Christopher Potts"
            ],
            "abstract": "Neural networks have excelled at many NLP tasks, but there remain open\nquestions about the performance of pretrained distributed word representations\nand their interaction with weight initialization and other hyperparameters. We\naddress these questions empirically using attention-based sequence-to-sequence\nmodels for natural language inference (NLI). Specifically, we compare three\ntypes of embeddings: random, pretrained (GloVe, word2vec), and retrofitted\n(pretrained plus WordNet information). We show that pretrained embeddings\noutperform both random and retrofitted ones in a large NLI corpus. Further\nexperiments on more controlled data sets shed light on the contexts for which\nretrofitted embeddings can be useful. We also explore two principled approaches\nto initializing the rest of the model parameters, Gaussian and orthogonal,\nshowing that the latter yields gains of up to 2.9% in the NLI task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02086v2",
            "date": "2017-10-05 16:03:42+00:00",
            "title": "Indowordnets help in Indian Language Machine Translation",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Being less resource languages, Indian-Indian and English-Indian language MT\nsystem developments faces the difficulty to translate various lexical\nphenomena. In this paper, we present our work on a comparative study of 440\nphrase-based statistical trained models for 110 language pairs across 11 Indian\nlanguages. We have developed 110 baseline Statistical Machine Translation\nsystems. Then we have augmented the training corpus with Indowordnet synset\nword entries of lexical database and further trained 110 models on top of the\nbaseline system. We have done a detailed performance comparison using various\nevaluation metrics such as BLEU score, METEOR and TER. We observed significant\nimprovement in evaluations of translation quality across all the 440 models\nafter using the Indowordnet. These experiments give a detailed insight in two\nways : (1) usage of lexical database with synset mapping for resource poor\nlanguages (2) efficient usage of Indowordnet sysnset mapping. More over, synset\nmapped lexical entries helped the SMT system to handle the ambiguity to a great\nextent during the translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02093v3",
            "date": "2017-10-05 16:15:23+00:00",
            "title": "Morphology Generation for Statistical Machine Translation",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "When translating into morphologically rich languages, Statistical MT\napproaches face the problem of data sparsity. The severity of the sparseness\nproblem will be high when the corpus size of morphologically richer language is\nless. Even though we can use factored models to correctly generate\nmorphological forms of words, the problem of data sparseness limits their\nperformance. In this paper, we describe a simple and effective solution which\nis based on enriching the input corpora with various morphological forms of\nwords. We use this method with the phrase-based and factor-based experiments on\ntwo morphologically rich languages: Hindi and Marathi when translating from\nEnglish. We evaluate the performance of our experiments both in terms automatic\nevaluation and subjective evaluation such as adequacy and fluency. We observe\nthat the morphology injection method helps in improving the quality of\ntranslation. We further analyze that the morph injection method helps in\nhandling the data sparseness problem to a great level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02095v1",
            "date": "2017-10-05 16:18:08+00:00",
            "title": "Machine Translation Evaluation with Neural Networks",
            "authors": [
                "Francisco Guzm\u00e1n",
                "Shafiq R. Joty",
                "Llu\u00eds M\u00e0rquez",
                "Preslav Nakov"
            ],
            "abstract": "We present a framework for machine translation evaluation using neural\nnetworks in a pairwise setting, where the goal is to select the better\ntranslation from a pair of hypotheses, given the reference translation. In this\nframework, lexical, syntactic and semantic information from the reference and\nthe two hypotheses is embedded into compact distributed vector representations,\nand fed into a multi-layer neural network that models nonlinear interactions\nbetween each of the hypotheses and the reference, as well as between the two\nhypotheses. We experiment with the benchmark datasets from the WMT Metrics\nshared task, on which we obtain the best results published so far, with the\nbasic network configuration. We also perform a series of experiments to analyze\nand understand the contribution of the different components of the network. We\nevaluate variants and extensions, including fine-tuning of the semantic\nembeddings, and sentence-based representations modeled with convolutional and\nrecurrent neural networks. In summary, the proposed framework is flexible and\ngeneralizable, allows for efficient learning and scoring, and provides an MT\nevaluation metric that correlates with human judgments, and is on par with the\nstate of the art.",
            "journal ref": "Computer Speech & Language 45: 180-200 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02100v3",
            "date": "2017-10-05 16:21:55+00:00",
            "title": "Phrase Pair Mappings for Hindi-English Statistical Machine Translation",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we present our work on the creation of lexical resources for\nthe Machine Translation between English and Hindi. We describes the development\nof phrase pair mappings for our experiments and the comparative performance\nevaluation between different trained models on top of the baseline Statistical\nMachine Translation system. We focused on augmenting the parallel corpus with\nmore vocabulary as well as with various inflected forms by exploring different\nways. We have augmented the training corpus with various lexical resources such\nas lexical words, synset words, function words and verb phrases. We have\ndescribed the case studies, automatic and subjective evaluations, detailed\nerror analysis for both the English to Hindi and Hindi to English machine\ntranslation systems. We further analyzed that, there is an incremental growth\nin the quality of machine translation with the usage of various lexical\nresources. Thus lexical resources do help uplift the translation quality of\nresource poor langugaes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02187v1",
            "date": "2017-10-05 19:24:07+00:00",
            "title": "BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages",
            "authors": [
                "Benjamin Heinzerling",
                "Michael Strube"
            ],
            "abstract": "We present BPEmb, a collection of pre-trained subword unit embeddings in 275\nlanguages, based on Byte-Pair Encoding (BPE). In an evaluation using\nfine-grained entity typing as testbed, BPEmb performs competitively, and for\nsome languages bet- ter than alternative subword approaches, while requiring\nvastly fewer resources and no tokenization. BPEmb is available at\nhttps://github.com/bheinzerling/bpemb",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02318v1",
            "date": "2017-10-06 09:06:33+00:00",
            "title": "A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification",
            "authors": [
                "Shuming Ma",
                "Xu Sun"
            ],
            "abstract": "Text summarization and text simplification are two major ways to simplify the\ntext for poor readers, including children, non-native speakers, and the\nfunctionally illiterate. Text summarization is to produce a brief summary of\nthe main ideas of the text, while text simplification aims to reduce the\nlinguistic complexity of the text and retain the original meaning. Recently,\nmost approaches for text summarization and text simplification are based on the\nsequence-to-sequence model, which achieves much success in many text generation\ntasks. However, although the generated simplified texts are similar to source\ntexts literally, they have low semantic relevance. In this work, our goal is to\nimprove semantic relevance between source texts and simplified texts for text\nsummarization and text simplification. We introduce a Semantic Relevance Based\nneural model to encourage high semantic similarity between texts and summaries.\nIn our model, the source text is represented by a gated attention encoder,\nwhile the summary representation is produced by a decoder. Besides, the\nsimilarity score between the representations is maximized during training. Our\nexperiments show that the proposed model outperforms the state-of-the-art\nsystems on two benchmark corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02365v2",
            "date": "2017-10-06 12:22:44+00:00",
            "title": "Czech Text Document Corpus v 2.0",
            "authors": [
                "Pavel Kr\u00e1l",
                "Ladislav Lenc"
            ],
            "abstract": "This paper introduces \"Czech Text Document Corpus v 2.0\", a collection of\ntext documents for automatic document classification in Czech language. It is\ncomposed of the text documents provided by the Czech News Agency and is freely\navailable for research purposes at http://ctdc.kiv.zcu.cz/. This corpus was\ncreated in order to facilitate a straightforward comparison of the document\nclassification approaches on Czech data. It is particularly dedicated to\nevaluation of multi-label document classification approaches, because one\ndocument is usually labelled with more than one label. Besides the information\nabout the document classes, the corpus is also annotated at the morphological\nlayer. This paper further shows the results of selected state-of-the-art\nmethods on this corpus to offer the possibility of an easy comparison with\nthese approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02398v3",
            "date": "2017-10-05 16:26:43+00:00",
            "title": "Bilingual Words and Phrase Mappings for Marathi and Hindi SMT",
            "authors": [
                "Sreelekha S",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Lack of proper linguistic resources is the major challenges faced by the\nMachine Translation system developments when dealing with the resource poor\nlanguages. In this paper, we describe effective ways to utilize the lexical\nresources to improve the quality of statistical machine translation. Our\nresearch on the usage of lexical resources mainly focused on two ways, such as;\naugmenting the parallel corpus with more vocabulary and to provide various word\nforms. We have augmented the training corpus with various lexical resources\nsuch as lexical words, function words, kridanta pairs and verb phrases. We have\ndescribed the case studies, evaluations and detailed error analysis for both\nMarathi to Hindi and Hindi to Marathi machine translation systems. From the\nevaluations we observed that, there is an incremental growth in the quality of\nmachine translation as the usage of various lexical resources increases.\nMoreover, usage of various lexical resources helps to improve the coverage and\nquality of machine translation where limited parallel corpus is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02437v1",
            "date": "2017-10-06 14:54:04+00:00",
            "title": "Learning Word Embeddings for Hyponymy with Entailment-Based Distributional Semantics",
            "authors": [
                "James Henderson"
            ],
            "abstract": "Lexical entailment, such as hyponymy, is a fundamental issue in the semantics\nof natural language. This paper proposes distributional semantic models which\nefficiently learn word embeddings for entailment, using a recently-proposed\nframework for modelling entailment in a vector-space. These models postulate a\nlatent vector for a pseudo-phrase containing two neighbouring word vectors. We\ninvestigate both modelling words as the evidence they contribute about this\nphrase vector, or as the posterior distribution of a one-word phrase vector,\nand find that the posterior vectors perform better. The resulting word\nembeddings outperform the best previous results on predicting hyponymy between\nwords, in unsupervised and semi-supervised experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02514v1",
            "date": "2017-10-06 17:46:53+00:00",
            "title": "On the Challenges of Sentiment Analysis for Dynamic Events",
            "authors": [
                "Monireh Ebrahimi",
                "Amir Hossein Yazdavar",
                "Amit Sheth"
            ],
            "abstract": "With the proliferation of social media over the last decade, determining\npeople's attitude with respect to a specific topic, document, interaction or\nevents has fueled research interest in natural language processing and\nintroduced a new channel called sentiment and emotion analysis. For instance,\nbusinesses routinely look to develop systems to automatically understand their\ncustomer conversations by identifying the relevant content to enhance marketing\ntheir products and managing their reputations. Previous efforts to assess\npeople's sentiment on Twitter have suggested that Twitter may be a valuable\nresource for studying political sentiment and that it reflects the offline\npolitical landscape. According to a Pew Research Center report, in January 2016\n44 percent of US adults stated having learned about the presidential election\nthrough social media. Furthermore, 24 percent reported use of social media\nposts of the two candidates as a source of news and information, which is more\nthan the 15 percent who have used both candidates' websites or emails combined.\nThe first presidential debate between Trump and Hillary was the most tweeted\ndebate ever with 17.1 million tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02569v1",
            "date": "2017-10-06 19:57:59+00:00",
            "title": "Low-resource bilingual lexicon extraction using graph based word embeddings",
            "authors": [
                "Ximena Gutierrez-Vasques",
                "Victor Mijangos"
            ],
            "abstract": "In this work we focus on the task of automatically extracting bilingual\nlexicon for the language pair Spanish-Nahuatl. This is a low-resource setting\nwhere only a small amount of parallel corpus is available. Most of the\ndownstream methods do not work well under low-resources conditions. This is\nspecially true for the approaches that use vectorial representations like\nWord2Vec. Our proposal is to construct bilingual word vectors from a graph.\nThis graph is generated using translation pairs obtained from an unsupervised\nword alignment method.\n  We show that, in a low-resource setting, these type of vectors are successful\nin representing words in a bilingual semantic space. Moreover, when a linear\ntransformation is applied to translate words from one language to another, our\ngraph based representations considerably outperform the popular setting that\nuses Word2Vec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02603v2",
            "date": "2017-10-06 22:40:36+00:00",
            "title": "Low-Rank RNN Adaptation for Context-Aware Language Modeling",
            "authors": [
                "Aaron Jaech",
                "Mari Ostendorf"
            ],
            "abstract": "A context-aware language model uses location, user and/or domain metadata\n(context) to adapt its predictions. In neural language models, context\ninformation is typically represented as an embedding and it is given to the RNN\nas an additional input, which has been shown to be useful in many applications.\nWe introduce a more powerful mechanism for using context to adapt an RNN by\nletting the context vector control a low-rank transformation of the recurrent\nlayer weight matrix. Experiments show that allowing a greater fraction of the\nmodel parameters to be adjusted has benefits in terms of perplexity and\nclassification for several different types of context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02717v1",
            "date": "2017-10-07 18:30:38+00:00",
            "title": "Group Sparse CNNs for Question Classification with Answer Sets",
            "authors": [
                "Mingbo Ma",
                "Liang Huang",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "Question classification is an important task with wide applications. However,\ntraditional techniques treat questions as general sentences, ignoring the\ncorresponding answer data. In order to consider answer information into\nquestion modeling, we first introduce novel group sparse autoencoders which\nrefine question representation by utilizing group information in the answer\nset. We then propose novel group sparse CNNs which naturally learn question\nrepresentation with respect to their answers by implanting group sparse\nautoencoders into traditional CNNs. The proposed model significantly outperform\nstrong baselines on four datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02718v2",
            "date": "2017-10-07 18:37:08+00:00",
            "title": "OSU Multimodal Machine Translation System Report",
            "authors": [
                "Mingbo Ma",
                "Dapeng Li",
                "Kai Zhao",
                "Liang Huang"
            ],
            "abstract": "This paper describes Oregon State University's submissions to the shared\nWMT'17 task \"multimodal translation task I\". In this task, all the sentence\npairs are image captions in different languages. The key difference between\nthis task and conventional machine translation is that we have corresponding\nimages as additional information for each sentence pair. In this paper, we\nintroduce a simple but effective system which takes an image shared between\ndifferent languages, feeding it into the both encoding and decoding side. We\nreport our system's performance for English-French and English-German with\nFlickr30K (in-domain) and MSCOCO (out-of-domain) datasets. Our system achieves\nthe best performance in TER for English-German for MSCOCO dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02745v2",
            "date": "2017-10-07 20:43:27+00:00",
            "title": "Multi-Document Summarization using Distributed Bag-of-Words Model",
            "authors": [
                "Kaustubh Mani",
                "Ishan Verma",
                "Hardik Meisheri",
                "Lipika Dey"
            ],
            "abstract": "As the number of documents on the web is growing exponentially,\nmulti-document summarization is becoming more and more important since it can\nprovide the main ideas in a document set in short time. In this paper, we\npresent an unsupervised centroid-based document-level reconstruction framework\nusing distributed bag of words model. Specifically, our approach selects\nsummary sentences in order to minimize the reconstruction error between the\nsummary and the documents. We apply sentence selection and beam search, to\nfurther improve the performance of our model. Experimental results on two\ndifferent datasets show significant performance gains compared with the\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02855v2",
            "date": "2017-10-08 16:56:05+00:00",
            "title": "The IIT Bombay English-Hindi Parallel Corpus",
            "authors": [
                "Anoop Kunchukuttan",
                "Pratik Mehta",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "We present the IIT Bombay English-Hindi Parallel Corpus. The corpus is a\ncompilation of parallel corpora previously available in the public domain as\nwell as new parallel corpora we collected. The corpus contains 1.49 million\nparallel segments, of which 694k segments were not previously available in the\npublic domain. The corpus has been pre-processed for machine translation, and\nwe report baseline phrase-based SMT and NMT translation results on this corpus.\nThis corpus has been used in two editions of shared tasks at the Workshop on\nAsian Language Translation (2016 and 2017). The corpus is freely available for\nnon-commercial research. To the best of our knowledge, this is the largest\npublicly available English-Hindi parallel corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02925v1",
            "date": "2017-10-09 03:07:54+00:00",
            "title": "Natural Language Inference from Multiple Premises",
            "authors": [
                "Alice Lai",
                "Yonatan Bisk",
                "Julia Hockenmaier"
            ],
            "abstract": "We define a novel textual entailment task that requires inference over\nmultiple premise sentences. We present a new dataset for this task that\nminimizes trivial lexical inferences, emphasizes knowledge of everyday events,\nand presents a more challenging setting for textual entailment. We evaluate\nseveral strong neural baselines and analyze how the multiple premise task\ndiffers from standard textual entailment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03006v3",
            "date": "2017-10-09 09:29:55+00:00",
            "title": "Page Stream Segmentation with Convolutional Neural Nets Combining Textual and Visual Features",
            "authors": [
                "Gregor Wiedemann",
                "Gerhard Heyer"
            ],
            "abstract": "In recent years, (retro-)digitizing paper-based files became a major\nundertaking for private and public archives as well as an important task in\nelectronic mailroom applications. As a first step, the workflow involves\nscanning and Optical Character Recognition (OCR) of documents. Preservation of\ndocument contexts of single page scans is a major requirement in this context.\nTo facilitate workflows involving very large amounts of paper scans, page\nstream segmentation (PSS) is the task to automatically separate a stream of\nscanned images into multi-page documents. In a digitization project together\nwith a German federal archive, we developed a novel approach based on\nconvolutional neural networks (CNN) combining image and text features to\nachieve optimal document separation results. Evaluation shows that our PSS\narchitecture achieves an accuracy up to 93 % which can be regarded as a new\nstate-of-the-art for this task.",
            "journal ref": "Proceedings of the 11th International Conference on Language\n  Resources and Evaluation (LREC 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03203v2",
            "date": "2017-10-09 17:30:12+00:00",
            "title": "Deep Learning Paradigm with Transformed Monolingual Word Embeddings for Multilingual Sentiment Analysis",
            "authors": [
                "Yujie Lu",
                "Tatsunori Mori"
            ],
            "abstract": "The surge of social media use brings huge demand of multilingual sentiment\nanalysis (MSA) for unveiling cultural difference. So far, traditional methods\nresorted to machine translation---translating texts in other languages to\nEnglish, and then adopt the methods once worked in English. However, this\nparadigm is conditioned by the quality of machine translation. In this paper,\nwe propose a new deep learning paradigm to assimilate the differences between\nlanguages for MSA. We first pre-train monolingual word embeddings separately,\nthen map word embeddings in different spaces into a shared embedding space, and\nthen finally train a parameter-sharing deep neural network for MSA. The\nexperimental results show that our paradigm is effective. Especially, our CNN\nmodel outperforms a state-of-the-art baseline by around 2.1% in terms of\nclassification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03348v1",
            "date": "2017-10-09 23:21:34+00:00",
            "title": "What does Attention in Neural Machine Translation Pay Attention to?",
            "authors": [
                "Hamidreza Ghader",
                "Christof Monz"
            ],
            "abstract": "Attention in neural machine translation provides the possibility to encode\nrelevant parts of the source sentence at each translation step. As a result,\nattention is considered to be an alignment model as well. However, there is no\nwork that specifically studies attention and provides analysis of what is being\nlearned by attention models. Thus, the question still remains that how\nattention is similar or different from the traditional alignment. In this\npaper, we provide detailed analysis of attention and compare it to traditional\nalignment. We answer the question of whether attention is only capable of\nmodelling translational equivalent or it captures more information. We show\nthat attention is different from alignment in some cases and is capturing\nuseful information other than alignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03476v1",
            "date": "2017-10-10 09:41:46+00:00",
            "title": "MoNoise: Modeling Noise Using a Modular Normalization System",
            "authors": [
                "Rob van der Goot",
                "Gertjan van Noord"
            ],
            "abstract": "We propose MoNoise: a normalization model focused on generalizability and\nefficiency, it aims at being easily reusable and adaptable. Normalization is\nthe task of translating texts from a non- canonical domain to a more canonical\ndomain, in our case: from social media data to standard language. Our proposed\nmodel is based on a modular candidate generation in which each module is\nresponsible for a different type of normalization action. The most important\ngeneration modules are a spelling correction system and a word embeddings\nmodule. Depending on the definition of the normalization task, a static lookup\nlist can be crucial for performance. We train a random forest classifier to\nrank the candidates, which generalizes well to all different types of\nnormaliza- tion actions. Most features for the ranking originate from the\ngeneration modules; besides these features, N-gram features prove to be an\nimportant source of information. We show that MoNoise beats the\nstate-of-the-art on different normalization benchmarks for English and Dutch,\nwhich all define the task of normalization slightly different.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03501v3",
            "date": "2017-10-10 10:39:20+00:00",
            "title": "A Very Low Resource Language Speech Corpus for Computational Language Documentation Experiments",
            "authors": [
                "P. Godard",
                "G. Adda",
                "M. Adda-Decker",
                "J. Benjumea",
                "L. Besacier",
                "J. Cooper-Leavitt",
                "G-N. Kouarata",
                "L. Lamel",
                "H. Maynard",
                "M. Mueller",
                "A. Rialland",
                "S. Stueker",
                "F. Yvon",
                "M. Zanon-Boito"
            ],
            "abstract": "Most speech and language technologies are trained with massive amounts of\nspeech and text information. However, most of the world languages do not have\nsuch resources or stable orthography. Systems constructed under these almost\nzero resource conditions are not only promising for speech technology but also\nfor computational language documentation. The goal of computational language\ndocumentation is to help field linguists to (semi-)automatically analyze and\nannotate audio recordings of endangered and unwritten languages. Example tasks\nare automatic phoneme discovery or lexicon discovery from the speech signal.\nThis paper presents a speech corpus collected during a realistic language\ndocumentation process. It is made up of 5k speech utterances in Mboshi (Bantu\nC25) aligned to French text translations. Speech transcriptions are also made\navailable: they correspond to a non-standard graphemic form close to the\nlanguage phonology. We present how the data was collected, cleaned and\nprocessed and we illustrate its use through a zero-resource task: spoken term\ndiscovery. The dataset is made available to the community for reproducible\ncomputational language documentation experiments and their evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03743v1",
            "date": "2017-10-10 17:47:41+00:00",
            "title": "Confidence through Attention",
            "authors": [
                "Mat\u012bss Rikters",
                "Mark Fishel"
            ],
            "abstract": "Attention distributions of the generated translations are a useful bi-product\nof attention-based recurrent neural network translation models and can be\ntreated as soft alignments between the input and output tokens. In this work,\nwe use attention distributions as a confidence metric for output translations.\nWe present two strategies of using the attention distributions: filtering out\nbad translations from a large back-translated corpus, and selecting the best\ntranslation in a hybrid setup of two different translation systems. While\nmanual evaluation indicated only a weak correlation between our confidence\nscore and human judgments, the use-cases showed improvements of up to 2.22 BLEU\npoints for filtering and 0.99 points for hybrid translation, tested on\nEnglish<->German and English<->Latvian translation.",
            "journal ref": "Machine Translation Summit XVI, Nagoya, Japan, September 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03838v1",
            "date": "2017-10-10 22:12:46+00:00",
            "title": "The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages",
            "authors": [
                "Dingquan Wang",
                "Jason Eisner"
            ],
            "abstract": "We release Galactic Dependencies 1.0---a large set of synthetic languages not\nfound on Earth, but annotated in Universal Dependencies format. This new\nresource aims to provide training and development data for NLP methods that aim\nto adapt to unfamiliar languages. Each synthetic treebank is produced from a\nreal treebank by stochastically permuting the dependents of nouns and/or verbs\nto match the word order of other real languages. We discuss the usefulness,\nrealism, parsability, perplexity, and diversity of the synthetic languages. As\na simple demonstration of the use of Galactic Dependencies, we consider\nsingle-source transfer, which attempts to parse a real target language using a\nparser trained on a \"nearby\" source language. We find that including synthetic\nsource languages somewhat increases the diversity of the source pool, which\nsignificantly improves results for most target languages.",
            "journal ref": "Transactions of the Association of Computational Linguistics\n  (TACL), 4:491--505, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03877v1",
            "date": "2017-10-11 01:47:21+00:00",
            "title": "Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning",
            "authors": [
                "Dingquan Wang",
                "Jason Eisner"
            ],
            "abstract": "We show how to predict the basic word-order facts of a novel language given\nonly a corpus of part-of-speech (POS) sequences. We predict how often direct\nobjects follow their verbs, how often adjectives follow their nouns, and in\ngeneral the directionalities of all dependency relations. Such typological\nproperties could be helpful in grammar induction. While such a problem is\nusually regarded as unsupervised learning, our innovation is to treat it as\nsupervised learning, using a large collection of realistic synthetic languages\nas training data. The supervised learner must identify surface features of a\nlanguage's POS sequence (hand-engineered or neural features) that correlate\nwith the language's deeper structure (latent trees). In the experiment, we\nshow: 1) Given a small set of real languages, it helps to add many synthetic\nlanguages to the training data. 2) Our system is robust even when the POS\nsequences include noise. 3) Our system on this task outperforms a grammar\ninduction baseline by a large margin.",
            "journal ref": "Transactions of the Association of Computational Linguistics\n  (TACL), 5:147--161, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03954v1",
            "date": "2017-10-11 08:22:10+00:00",
            "title": "Decision support from financial disclosures with deep neural networks and transfer learning",
            "authors": [
                "Mathias Kraus",
                "Stefan Feuerriegel"
            ],
            "abstract": "Company disclosures greatly aid in the process of financial decision-making;\ntherefore, they are consulted by financial investors and automated traders\nbefore exercising ownership in stocks. While humans are usually able to\ncorrectly interpret the content, the same is rarely true of computerized\ndecision support systems, which struggle with the complexity and ambiguity of\nnatural language. A possible remedy is represented by deep learning, which\novercomes several shortcomings of traditional methods of text mining. For\ninstance, recurrent neural networks, such as long short-term memories, employ\nhierarchical structures, together with a large number of hidden layers, to\nautomatically extract features from ordered sequences of words and capture\nhighly non-linear relationships such as context-dependent meanings. However,\ndeep learning has only recently started to receive traction, possibly because\nits performance is largely untested. Hence, this paper studies the use of deep\nneural networks for financial decision support. We additionally experiment with\ntransfer learning, in which we pre-train the network on a different corpus with\na length of 139.1 million words. Our results reveal a higher directional\naccuracy as compared to traditional machine learning when predicting stock\nprice movements in response to financial disclosures. Our work thereby helps to\nhighlight the business value of deep learning and provides recommendations to\npractitioners and executives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03957v1",
            "date": "2017-10-11 08:30:30+00:00",
            "title": "DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset",
            "authors": [
                "Yanran Li",
                "Hui Su",
                "Xiaoyu Shen",
                "Wenjie Li",
                "Ziqiang Cao",
                "Shuzi Niu"
            ],
            "abstract": "We develop a high-quality multi-turn dialog dataset, DailyDialog, which is\nintriguing in several aspects. The language is human-written and less noisy.\nThe dialogues in the dataset reflect our daily communication way and cover\nvarious topics about our daily life. We also manually label the developed\ndataset with communication intention and emotion information. Then, we evaluate\nexisting approaches on DailyDialog dataset and hope it benefit the research\nfield of dialog systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04087v3",
            "date": "2017-10-11 14:24:28+00:00",
            "title": "Word Translation Without Parallel Data",
            "authors": [
                "Alexis Conneau",
                "Guillaume Lample",
                "Marc'Aurelio Ranzato",
                "Ludovic Denoyer",
                "Herv\u00e9 J\u00e9gou"
            ],
            "abstract": "State-of-the-art methods for learning cross-lingual word embeddings have\nrelied on bilingual dictionaries or parallel corpora. Recent studies showed\nthat the need for parallel data supervision can be alleviated with\ncharacter-level information. While these methods showed encouraging results,\nthey are not on par with their supervised counterparts and are limited to pairs\nof languages sharing a common alphabet. In this work, we show that we can build\na bilingual dictionary between two languages without using any parallel\ncorpora, by aligning monolingual word embedding spaces in an unsupervised way.\nWithout using any character information, our model even outperforms existing\nsupervised methods on cross-lingual tasks for some language pairs. Our\nexperiments demonstrate that our method works very well also for distant\nlanguage pairs, like English-Russian or English-Chinese. We finally describe\nexperiments on the English-Esperanto low-resource language pair, on which there\nonly exists a limited amount of parallel data, to show the potential impact of\nour method in fully unsupervised machine translation. Our code, embeddings and\ndictionaries are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04344v1",
            "date": "2017-10-12 02:39:45+00:00",
            "title": "Using Context Events in Neural Network Models for Event Temporal Status Identification",
            "authors": [
                "Zeyu Dai",
                "Wenlin Yao",
                "Ruihong Huang"
            ],
            "abstract": "Focusing on the task of identifying event temporal status, we find that\nevents directly or indirectly governing the target event in a dependency tree\nare most important contexts. Therefore, we extract dependency chains containing\ncontext events and use them as input in neural network models, which\nconsistently outperform previous models using local context words as input.\nVisualization verifies that the dependency chain representation can effectively\ncapture the context events which are closely related to the target event and\nplay key roles in predicting event temporal status.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04437v1",
            "date": "2017-10-12 10:36:41+00:00",
            "title": "Revisiting the Design Issues of Local Models for Japanese Predicate-Argument Structure Analysis",
            "authors": [
                "Yuichiroh Matsubayashi",
                "Kentaro Inui"
            ],
            "abstract": "The research trend in Japanese predicate-argument structure (PAS) analysis is\nshifting from pointwise prediction models with local features to global models\ndesigned to search for globally optimal solutions. However, the existing global\nmodels tend to employ only relatively simple local features; therefore, the\noverall performance gains are rather limited. The importance of designing a\nlocal model is demonstrated in this study by showing that the performance of a\nsophisticated local model can be considerably improved with recent feature\nembedding methods and a feature combination learning based on a neural network,\noutperforming the state-of-the-art global models in $F_1$ on a common benchmark\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04515v1",
            "date": "2017-10-12 13:40:43+00:00",
            "title": "Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR",
            "authors": [
                "Dan Lim"
            ],
            "abstract": "This thesis introduces the sequence to sequence model with Luong's attention\nmechanism for end-to-end ASR. It also describes various neural network\nalgorithms including Batch normalization, Dropout and Residual network which\nconstitute the convolutional attention-based seq2seq neural network. Finally\nthe proposed model proved its effectiveness for speech recognition achieving\n15.8% phoneme error rate on TIMIT dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04600v1",
            "date": "2017-10-12 16:33:01+00:00",
            "title": "Auto Analysis of Customer Feedback using CNN and GRU Network",
            "authors": [
                "Deepak Gupta",
                "Pabitra Lenka",
                "Harsimran Bedi",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Analyzing customer feedback is the best way to channelize the data into new\nmarketing strategies that benefit entrepreneurs as well as customers. Therefore\nan automated system which can analyze the customer behavior is in great demand.\nUsers may write feedbacks in any language, and hence mining appropriate\ninformation often becomes intractable. Especially in a traditional\nfeature-based supervised model, it is difficult to build a generic system as\none has to understand the concerned language for finding the relevant features.\nIn order to overcome this, we propose deep Convolutional Neural Network (CNN)\nand Recurrent Neural Network (RNN) based approaches that do not require\nhandcrafting of features. We evaluate these techniques for analyzing customer\nfeedback sentences in four languages, namely English, French, Japanese and\nSpanish. Our empirical analysis shows that our models perform well in all the\nfour languages on the setups of IJCNLP Shared Task on Customer Feedback\nAnalysis. Our model achieved the second rank in French, with an accuracy of\n71.75% and third ranks for all the other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04802v1",
            "date": "2017-10-13 04:51:18+00:00",
            "title": "End-to-end Network for Twitter Geolocation Prediction and Hashing",
            "authors": [
                "Jey Han Lau",
                "Lianhua Chi",
                "Khoi-Nguyen Tran",
                "Trevor Cohn"
            ],
            "abstract": "We propose an end-to-end neural network to predict the geolocation of a\ntweet. The network takes as input a number of raw Twitter metadata such as the\ntweet message and associated user account information. Our model is language\nindependent, and despite minimal feature engineering, it is interpretable and\ncapable of learning location indicative words and timing patterns. Compared to\nstate-of-the-art systems, our model outperforms them by 2%-6%. Additionally, we\npropose extensions to the model to compress representation learnt by the\nnetwork into binary codes. Experiments show that it produces compact codes\ncompared to benchmark hashing algorithms. An implementation of the model is\nreleased publicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04989v1",
            "date": "2017-10-13 16:24:23+00:00",
            "title": "Complex Word Identification: Challenges in Data Annotation and System Performance",
            "authors": [
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Gustavo Paetzold",
                "Lucia Specia"
            ],
            "abstract": "This paper revisits the problem of complex word identification (CWI)\nfollowing up the SemEval CWI shared task. We use ensemble classifiers to\ninvestigate how well computational methods can discriminate between complex and\nnon-complex words. Furthermore, we analyze the classification performance to\nunderstand what makes lexical complexity challenging. Our findings show that\nmost systems performed poorly on the SemEval CWI dataset, and one of the\nreasons for that is the way in which human annotation was performed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05094v1",
            "date": "2017-10-13 22:55:43+00:00",
            "title": "Learning Phrase Embeddings from Paraphrases with GRUs",
            "authors": [
                "Zhihao Zhou",
                "Lifu Huang",
                "Heng Ji"
            ],
            "abstract": "Learning phrase representations has been widely explored in many Natural\nLanguage Processing (NLP) tasks (e.g., Sentiment Analysis, Machine Translation)\nand has shown promising improvements. Previous studies either learn\nnon-compositional phrase representations with general word embedding learning\ntechniques or learn compositional phrase representations based on syntactic\nstructures, which either require huge amounts of human annotations or cannot be\neasily generalized to all phrases. In this work, we propose to take advantage\nof large-scaled paraphrase database and present a pair-wise gated recurrent\nunits (pairwise-GRU) framework to generate compositional phrase\nrepresentations. Our framework can be re-used to generate representations for\nany phrases. Experimental results show that our framework achieves\nstate-of-the-art results on several phrase similarity tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05364v1",
            "date": "2017-10-15 17:32:59+00:00",
            "title": "Clickbait Detection in Tweets Using Self-attentive Network",
            "authors": [
                "Yiwei Zhou"
            ],
            "abstract": "Clickbait detection in tweets remains an elusive challenge. In this paper, we\ndescribe the solution for the Zingel Clickbait Detector at the Clickbait\nChallenge 2017, which is capable of evaluating each tweet's level of click\nbaiting. We first reformat the regression problem as a multi-classification\nproblem, based on the annotation scheme. To perform multi-classification, we\napply a token-level, self-attentive mechanism on the hidden states of\nbi-directional Gated Recurrent Units (biGRU), which enables the model to\ngenerate tweets' task-specific vector representations by attending to important\ntokens. The self-attentive neural network can be trained end-to-end, without\ninvolving any manual feature engineering. Our detector ranked first in the\nfinal evaluation of Clickbait Challenge 2017.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05370v1",
            "date": "2017-10-15 18:15:35+00:00",
            "title": "NoReC: The Norwegian Review Corpus",
            "authors": [
                "Erik Velldal",
                "Lilja \u00d8vrelid",
                "Eivind Alexander Bergem",
                "Cathrine Stadsnes",
                "Samia Touileb",
                "Fredrik J\u00f8rgensen"
            ],
            "abstract": "This paper presents the Norwegian Review Corpus (NoReC), created for training\nand evaluating models for document-level sentiment analysis. The full-text\nreviews have been collected from major Norwegian news sources and cover a range\nof different domains, including literature, movies, video games, restaurants,\nmusic and theater, in addition to product reviews across a range of categories.\nEach review is labeled with a manually assigned score of 1-6, as provided by\nthe rating of the original author. This first release of the corpus comprises\nmore than 35,000 reviews. It is distributed using the CoNLL-U format,\npre-processed using UDPipe, along with a rich set of metadata. The work\nreported in this paper forms part of the SANT initiative (Sentiment Analysis\nfor Norwegian Text), a project seeking to provide resources and tools for\nsentiment analysis and opinion mining for Norwegian. As resources for sentiment\nanalysis have so far been unavailable for Norwegian, NoReC represents a highly\nvaluable and sought-after addition to Norwegian language technology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05429v1",
            "date": "2017-10-16 00:52:32+00:00",
            "title": "Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media",
            "authors": [
                "Amir Hossein Yazdavar",
                "Hussein S. Al-Olimat",
                "Monireh Ebrahimi",
                "Goonmeet Bajaj",
                "Tanvi Banerjee",
                "Krishnaprasad Thirunarayan",
                "Jyotishman Pathak",
                "Amit Sheth"
            ],
            "abstract": "With the rise of social media, millions of people are routinely expressing\ntheir moods, feelings, and daily struggles with mental health issues on social\nmedia platforms like Twitter. Unlike traditional observational cohort studies\nconducted through questionnaires and self-reported surveys, we explore the\nreliable detection of clinical depression from tweets obtained unobtrusively.\nBased on the analysis of tweets crawled from users with self-reported\ndepressive symptoms in their Twitter profiles, we demonstrate the potential for\ndetecting clinical depression symptoms which emulate the PHQ-9 questionnaire\nclinicians use today. Our study uses a semi-supervised statistical model to\nevaluate how the duration of these symptoms and their expression on Twitter (in\nterms of word usage patterns and topical preferences) align with the medical\nfindings reported via the PHQ-9. Our proactive and automatic screening tool is\nable to identify clinical depressive symptoms with an accuracy of 68% and\nprecision of 72%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05519v2",
            "date": "2017-10-16 05:49:29+00:00",
            "title": "BKTreebank: Building a Vietnamese Dependency Treebank",
            "authors": [
                "Kiem-Hieu Nguyen"
            ],
            "abstract": "Dependency treebank is an important resource in any language. In this paper,\nwe present our work on building BKTreebank, a dependency treebank for\nVietnamese. Important points on designing POS tagset, dependency relations, and\nannotation guidelines are discussed. We describe experiments on POS tagging and\ndependency parsing on the treebank. Experimental results show that the treebank\nis a useful resource for Vietnamese language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05709v2",
            "date": "2017-10-16 14:11:38+00:00",
            "title": "Aligning Script Events with Narrative Texts",
            "authors": [
                "Simon Ostermann",
                "Michael Roth",
                "Stefan Thater",
                "Manfred Pinkal"
            ],
            "abstract": "Script knowledge plays a central role in text understanding and is relevant\nfor a variety of downstream tasks. In this paper, we consider two recent\ndatasets which provide a rich and general representation of script events in\nterms of paraphrase sets. We introduce the task of mapping event mentions in\nnarrative texts to such script event types, and present a model for this task\nthat exploits rich linguistic representations as well as information on\ntemporal ordering. The results of our experiments demonstrate that this complex\ntask is indeed feasible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.05978v1",
            "date": "2017-10-16 20:03:08+00:00",
            "title": "Convolutional Neural Networks for Sentiment Classification on Business Reviews",
            "authors": [
                "Andreea Salinca"
            ],
            "abstract": "Recently Convolutional Neural Networks (CNNs) models have proven remarkable\nresults for text classification and sentiment analysis. In this paper, we\npresent our approach on the task of classifying business reviews using word\nembeddings on a large-scale dataset provided by Yelp: Yelp 2017 challenge\ndataset. We compare word-based CNN using several pre-trained word embeddings\nand end-to-end vector representations for text reviews classification. We\nconduct several experiments to capture the semantic relationship between\nbusiness reviews and we use deep learning techniques that prove that the\nobtained results are competitive with traditional methods.",
            "journal ref": "Proceedings of IJCAI Workshop on Semantic Machine Learning (SML\n  2017): 35-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06112v1",
            "date": "2017-10-17 06:05:50+00:00",
            "title": "CASICT Tibetan Word Segmentation System for MLWS2017",
            "authors": [
                "Jiawei Hu",
                "Qun Liu"
            ],
            "abstract": "We participated in the MLWS 2017 on Tibetan word segmentation task, our\nsystem is trained in a unrestricted way, by introducing a baseline system and\n76w tibetan segmented sentences of ours. In the system character sequence is\nprocessed by the baseline system into word sequence, then a subword unit (BPE\nalgorithm) split rare words into subwords with its corresponding features,\nafter that a neural network classifier is adopted to token each subword into\n\"B,M,E,S\" label, in decoding step a simple rule is used to recover a final word\nsequence. The candidate system for submition is selected by evaluating the\nF-score in dev set pre-extracted from the 76w sentences. Experiment shows that\nthis method can fix segmentation errors of baseline system and result in a\nsignificant performance gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06313v2",
            "date": "2017-10-17 14:27:36+00:00",
            "title": "Paying Attention to Multi-Word Expressions in Neural Machine Translation",
            "authors": [
                "Mat\u012bss Rikters",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Processing of multi-word expressions (MWEs) is a known problem for any\nnatural language processing task. Even neural machine translation (NMT)\nstruggles to overcome it. This paper presents results of experiments on\ninvestigating NMT attention allocation to the MWEs and improving automated\ntranslation of sentences that contain MWEs in English->Latvian and\nEnglish->Czech NMT systems. Two improvement strategies were explored -(1)\nbilingual pairs of automatically extracted MWE candidates were added to the\nparallel corpus used to train the NMT system, and (2) full sentences containing\nthe automatically extracted MWE candidates were added to the parallel corpus.\nBoth approaches allowed to increase automated evaluation results. The best\nresult - 0.99 BLEU point increase - has been reached with the first approach,\nwhile with the second approach minimal improvements achieved. We also provide\nopen-source software and tools used for MWE extraction and alignment\ninspection.",
            "journal ref": "Published in Machine Translation Summit XVI, Nagoya, Japan,\n  September 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06371v2",
            "date": "2017-10-17 16:39:09+00:00",
            "title": "Specialising Word Vectors for Lexical Entailment",
            "authors": [
                "Ivan Vuli\u0107",
                "Nikola Mrk\u0161i\u0107"
            ],
            "abstract": "We present LEAR (Lexical Entailment Attract-Repel), a novel post-processing\nmethod that transforms any input word vector space to emphasise the asymmetric\nrelation of lexical entailment (LE), also known as the IS-A or\nhyponymy-hypernymy relation. By injecting external linguistic constraints\n(e.g., WordNet links) into the initial vector space, the LE specialisation\nprocedure brings true hyponymy-hypernymy pairs closer together in the\ntransformed Euclidean space. The proposed asymmetric distance measure adjusts\nthe norms of word vectors to reflect the actual WordNet-style hierarchy of\nconcepts. Simultaneously, a joint objective enforces semantic similarity using\nthe symmetric cosine distance, yielding a vector space specialised for both\nlexical relations at once. LEAR specialisation achieves state-of-the-art\nperformance in the tasks of hypernymy directionality, hypernymy detection, and\ngraded lexical entailment, demonstrating the effectiveness and robustness of\nthe proposed asymmetric specialisation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06393v1",
            "date": "2017-10-17 17:02:54+00:00",
            "title": "RETUYT in TASS 2017: Sentiment Analysis for Spanish Tweets using SVM and CNN",
            "authors": [
                "Aiala Ros\u00e1",
                "Luis Chiruzzo",
                "Mathias Etcheverry",
                "Santiago Castro"
            ],
            "abstract": "This article presents classifiers based on SVM and Convolutional Neural\nNetworks (CNN) for the TASS 2017 challenge on tweets sentiment analysis. The\nclassifier with the best performance in general uses a combination of SVM and\nCNN. The use of word embeddings was particularly useful for improving the\nclassifiers performance.",
            "journal ref": "ISSN 1613-0073, TASS 2017: Workshop on Semantic Analysis at SEPLN,\n  Sep 2017, pages 77-83",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06524v2",
            "date": "2017-10-17 23:13:32+00:00",
            "title": "Unsupervised Sentence Representations as Word Information Series: Revisiting TF--IDF",
            "authors": [
                "Ignacio Arroyo-Fern\u00e1ndez",
                "Carlos-Francisco M\u00e9ndez-Cruz",
                "Gerardo Sierra",
                "Juan-Manuel Torres-Moreno",
                "Grigori Sidorov"
            ],
            "abstract": "Sentence representation at the semantic level is a challenging task for\nNatural Language Processing and Artificial Intelligence. Despite the advances\nin word embeddings (i.e. word vector representations), capturing sentence\nmeaning is an open question due to complexities of semantic interactions among\nwords. In this paper, we present an embedding method, which is aimed at\nlearning unsupervised sentence representations from unlabeled text. We propose\nan unsupervised method that models a sentence as a weighted series of word\nembeddings. The weights of the word embeddings are fitted by using Shannon's\nword entropies provided by the Term Frequency--Inverse Document Frequency\n(TF--IDF) transform. The hyperparameters of the model can be selected according\nto the properties of data (e.g. sentence length and textual gender).\nHyperparameter selection involves word embedding methods and dimensionalities,\nas well as weighting schemata. Our method offers advantages over existing\nmethods: identifiable modules, short-term training, online inference of\n(unseen) sentence representations, as well as independence from domain,\nexternal knowledge and language resources. Results showed that our model\noutperformed the state of the art in well-known Semantic Textual Similarity\n(STS) benchmarks. Moreover, our model reached state-of-the-art performance when\ncompared to supervised and knowledge-based STS systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06536v1",
            "date": "2017-10-18 00:26:24+00:00",
            "title": "Basic tasks of sentiment analysis",
            "authors": [
                "Iti Chaturvedi",
                "Soujanya Poria",
                "Erik Cambria"
            ],
            "abstract": "Subjectivity detection is the task of identifying objective and subjective\nsentences. Objective sentences are those which do not exhibit any sentiment.\nSo, it is desired for a sentiment analysis engine to find and separate the\nobjective sentences for further analysis, e.g., polarity detection. In\nsubjective sentences, opinions can often be expressed on one or multiple\ntopics. Aspect extraction is a subtask of sentiment analysis that consists in\nidentifying opinion targets in opinionated text, i.e., in detecting the\nspecific aspects of a product or service the opinion holder is either praising\nor complaining about.",
            "journal ref": "Encyclopedia of Social Network Analysis and Mining, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06554v2",
            "date": "2017-10-18 01:47:29+00:00",
            "title": "Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting",
            "authors": [
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "We describe Honk, an open-source PyTorch reimplementation of convolutional\nneural networks for keyword spotting that are included as examples in\nTensorFlow. These models are useful for recognizing \"command triggers\" in\nspeech-based interfaces (e.g., \"Hey Siri\"), which serve as explicit cues for\naudio recordings of utterances that are sent to the cloud for full speech\nrecognition. Evaluation on Google's recently released Speech Commands Dataset\nshows that our reimplementation is comparable in accuracy and provides a\nstarting point for future work on the keyword spotting task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06632v1",
            "date": "2017-10-18 09:13:06+00:00",
            "title": "Towards a Seamless Integration of Word Senses into Downstream NLP Applications",
            "authors": [
                "Mohammad Taher Pilehvar",
                "Jose Camacho-Collados",
                "Roberto Navigli",
                "Nigel Collier"
            ],
            "abstract": "Lexical ambiguity can impede NLP systems from accurate understanding of\nsemantics. Despite its potential benefits, the integration of sense-level\ninformation into NLP systems has remained understudied. By incorporating a\nnovel disambiguation algorithm into a state-of-the-art classification model, we\ncreate a pipeline to integrate sense-level information into downstream NLP\napplications. We show that a simple disambiguation of the input text can lead\nto consistent performance improvement on multiple topic categorization and\npolarity detection datasets, particularly when the fine granularity of the\nunderlying sense inventory is reduced and the document is sufficiently large.\nOur results also point to the need for sense representation research to focus\nmore on in vivo evaluations which target the performance in downstream NLP\napplications rather than artificial benchmarks.",
            "journal ref": "Proceedings of the 55th Annual Meeting of the Association for\n  Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada (2017),\n  pages 1857-1869",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06700v1",
            "date": "2017-10-18 12:36:06+00:00",
            "title": "Build Fast and Accurate Lemmatization for Arabic",
            "authors": [
                "Hamdy Mubarak"
            ],
            "abstract": "In this paper we describe the complexity of building a lemmatizer for Arabic\nwhich has a rich and complex derivational morphology, and we discuss the need\nfor a fast and accurate lammatization to enhance Arabic Information Retrieval\n(IR) results. We also introduce a new data set that can be used to test\nlemmatization accuracy, and an efficient lemmatization algorithm that\noutperforms state-of-the-art Arabic lemmatization in terms of accuracy and\nspeed. We share the data set and the code for public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06917v2",
            "date": "2017-10-08 21:12:27+00:00",
            "title": "Annotating High-Level Structures of Short Stories and Personal Anecdotes",
            "authors": [
                "Boyang Li",
                "Beth Cardier",
                "Tong Wang",
                "Florian Metze"
            ],
            "abstract": "Stories are a vital form of communication in human culture; they are employed\ndaily to persuade, to elicit sympathy, or to convey a message. Computational\nunderstanding of human narratives, especially high-level narrative structures,\nremain limited to date. Multiple literary theories for narrative structures\nexist, but operationalization of the theories has remained a challenge. We\ndeveloped an annotation scheme by consolidating and extending existing\nnarratological theories, including Labov and Waletsky's (1967) functional\ncategorization scheme and Freytag's (1863) pyramid of dramatic tension, and\npresent 360 annotated short stories collected from online sources. In the\nfuture, this research will support an approach that enables systems to\nintelligently sustain complex communications with humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06931v2",
            "date": "2017-10-18 20:48:18+00:00",
            "title": "OhioState at IJCNLP-2017 Task 4: Exploring Neural Architectures for Multilingual Customer Feedback Analysis",
            "authors": [
                "Dushyanta Dhyani"
            ],
            "abstract": "This paper describes our systems for IJCNLP 2017 Shared Task on Customer\nFeedback Analysis. We experimented with simple neural architectures that gave\ncompetitive performance on certain tasks. This includes shallow CNN and\nBi-Directional LSTM architectures with Facebook's Fasttext as a baseline model.\nOur best performing model was in the Top 5 systems using the Exact-Accuracy and\nMicro-Average-F1 metrics for the Spanish (85.28% for both) and French (70% and\n73.17% respectively) task, and outperformed all the other models on comment\n(87.28%) and meaningless (51.85%) tags using Micro Average F1 by Tags metric\nfor the French task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06937v1",
            "date": "2017-10-17 17:10:11+00:00",
            "title": "Embedding-Based Speaker Adaptive Training of Deep Neural Networks",
            "authors": [
                "Xiaodong Cui",
                "Vaibhava Goel",
                "George Saon"
            ],
            "abstract": "An embedding-based speaker adaptive training (SAT) approach is proposed and\ninvestigated in this paper for deep neural network acoustic modeling. In this\napproach, speaker embedding vectors, which are a constant given a particular\nspeaker, are mapped through a control network to layer-dependent element-wise\naffine transformations to canonicalize the internal feature representations at\nthe output of hidden layers of a main network. The control network for\ngenerating the speaker-dependent mappings is jointly estimated with the main\nnetwork for the overall speaker adaptive acoustic modeling. Experiments on\nlarge vocabulary continuous speech recognition (LVCSR) tasks show that the\nproposed SAT scheme can yield superior performance over the widely-used\nspeaker-aware training using i-vectors with speaker-adapted input features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07032v1",
            "date": "2017-10-19 08:13:19+00:00",
            "title": "SLING: A framework for frame semantic parsing",
            "authors": [
                "Michael Ringgaard",
                "Rahul Gupta",
                "Fernando C. N. Pereira"
            ],
            "abstract": "We describe SLING, a framework for parsing natural language into semantic\nframes. SLING supports general transition-based, neural-network parsing with\nbidirectional LSTM input encoding and a Transition Based Recurrent Unit (TBRU)\nfor output decoding. The parsing model is trained end-to-end using only the\ntext tokens as input. The transition system has been designed to output frame\ngraphs directly without any intervening symbolic representation. The SLING\nframework includes an efficient and scalable frame store implementation as well\nas a neural network JIT compiler for fast inference during parsing. SLING is\nimplemented in C++ and it is available for download on GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07045v1",
            "date": "2017-10-19 09:15:37+00:00",
            "title": "Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings",
            "authors": [
                "Pieter Fivez",
                "Simon \u0160uster",
                "Walter Daelemans"
            ],
            "abstract": "We present an unsupervised context-sensitive spelling correction method for\nclinical free-text that uses word and character n-gram embeddings. Our method\ngenerates misspelling replacement candidates and ranks them according to their\nsemantic fit, by calculating a weighted cosine similarity between the\nvectorized representation of a candidate and the misspelling context. To tune\nthe parameters of this model, we generate self-induced spelling error corpora.\nWe perform our experiments for two languages. For English, we greatly\noutperform off-the-shelf spelling correction tools on a manually annotated\nMIMIC-III test set, and counter the frequency bias of a noisy channel model,\nshowing that neural embeddings can be successfully exploited to improve upon\nthe state-of-the-art. For Dutch, we also outperform an off-the-shelf spelling\ncorrection tool on manually annotated clinical records from the Antwerp\nUniversity Hospital, but can offer no empirical evidence that our method\ncounters the frequency bias of a noisy channel model in this case as well.\nHowever, both our context-sensitive model and our implementation of the noisy\nchannel model obtain high scores on the test set, establishing a\nstate-of-the-art for Dutch clinical spelling correction with the noisy channel\nmodel.",
            "journal ref": "CLIN Journal, Volume 7, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07210v1",
            "date": "2017-10-17 23:35:58+00:00",
            "title": "Multi-Task Label Embedding for Text Classification",
            "authors": [
                "Honglun Zhang",
                "Liqiang Xiao",
                "Wenqing Chen",
                "Yongkun Wang",
                "Yaohui Jin"
            ],
            "abstract": "Multi-task learning in text classification leverages implicit correlations\namong related tasks to extract common features and yield performance gains.\nHowever, most previous works treat labels of each task as independent and\nmeaningless one-hot vectors, which cause a loss of potential information and\nmakes it difficult for these models to jointly learn three or more tasks. In\nthis paper, we propose Multi-Task Label Embedding to convert labels in text\nclassification into semantic vectors, thereby turning the original tasks into\nvector matching tasks. We implement unsupervised, supervised and\nsemi-supervised models of Multi-Task Label Embedding, all utilizing semantic\ncorrelations among tasks and making it particularly convenient to scale and\ntransfer as more tasks are involved. Extensive experiments on five benchmark\ndatasets for text classification show that our models can effectively improve\nperformances of related tasks with semantic representations of labels and\nadditional information from each other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07388v1",
            "date": "2017-10-20 01:31:31+00:00",
            "title": "Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models",
            "authors": [
                "Yi Luan",
                "Chris Brockett",
                "Bill Dolan",
                "Jianfeng Gao",
                "Michel Galley"
            ],
            "abstract": "Building a persona-based conversation agent is challenging owing to the lack\nof large amounts of speaker-specific conversation data for model training. This\npaper addresses the problem by proposing a multi-task learning approach to\ntraining neural conversation models that leverages both conversation data\nacross speakers and other types of data pertaining to the speaker and speaker\nroles to be modeled. Experiments show that our approach leads to significant\nimprovements over baseline model quality, generating responses that capture\nmore precisely speakers' traits and speaking styles. The model offers the\nbenefits of being algorithmically simple and easy to implement, and not relying\non large quantities of data representing specific individual speakers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07394v2",
            "date": "2017-10-20 02:11:06+00:00",
            "title": "Recognizing Explicit and Implicit Hate Speech Using a Weakly Supervised Two-path Bootstrapping Approach",
            "authors": [
                "Lei Gao",
                "Alexis Kuppersmith",
                "Ruihong Huang"
            ],
            "abstract": "In the wake of a polarizing election, social media is laden with hateful\ncontent. To address various limitations of supervised hate speech\nclassification methods including corpus bias and huge cost of annotation, we\npropose a weakly supervised two-path bootstrapping approach for an online hate\nspeech detection model leveraging large-scale unlabeled data. This system\nsignificantly outperforms hate speech detection systems that are trained in a\nsupervised manner using manually annotated data. Applying this model on a large\nquantity of tweets collected before, after, and on election day reveals\nmotivations and patterns of inflammatory language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07395v2",
            "date": "2017-10-20 02:11:21+00:00",
            "title": "Detecting Online Hate Speech Using Context Aware Models",
            "authors": [
                "Lei Gao",
                "Ruihong Huang"
            ],
            "abstract": "In the wake of a polarizing election, the cyber world is laden with hate\nspeech. Context accompanying a hate speech text is useful for identifying hate\nspeech, which however has been largely overlooked in existing datasets and hate\nspeech detection models. In this paper, we provide an annotated corpus of hate\nspeech with context information well kept. Then we propose two types of hate\nspeech detection models that incorporate context information, a logistic\nregression model with context features and a neural network model with learning\ncomponents for context. Our evaluation shows that both models outperform a\nstrong baseline by around 3% to 4% in F1 score and combining these two models\nfurther improve the performance by another 7% in F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07441v1",
            "date": "2017-10-20 07:38:54+00:00",
            "title": "A Semantically Motivated Approach to Compute ROUGE Scores",
            "authors": [
                "Elaheh ShafieiBavani",
                "Mohammad Ebrahimi",
                "Raymond Wong",
                "Fang Chen"
            ],
            "abstract": "ROUGE is one of the first and most widely used evaluation metrics for text\nsummarization. However, its assessment merely relies on surface similarities\nbetween peer and model summaries. Consequently, ROUGE is unable to fairly\nevaluate abstractive summaries including lexical variations and paraphrasing.\nExploring the effectiveness of lexical resource-based models to address this\nissue, we adopt a graph-based algorithm into ROUGE to capture the semantic\nsimilarities between peer and model summaries. Our semantically motivated\napproach computes ROUGE scores based on both lexical and semantic similarities.\nExperiment results over TAC AESOP datasets indicate that exploiting the\nlexico-semantic similarity of the words used in summaries would significantly\nhelp ROUGE correlate better with human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07503v4",
            "date": "2017-10-20 12:22:15+00:00",
            "title": "Local Word Vectors Guiding Keyphrase Extraction",
            "authors": [
                "Eirini Papagiannopoulou",
                "Grigorios Tsoumakas"
            ],
            "abstract": "Automated keyphrase extraction is a fundamental textual information\nprocessing task concerned with the selection of representative phrases from a\ndocument that summarize its content. This work presents a novel unsupervised\nmethod for keyphrase extraction, whose main innovation is the use of local word\nembeddings (in particular GloVe vectors), i.e., embeddings trained from the\nsingle document under consideration. We argue that such local representation of\nwords and keyphrases are able to accurately capture their semantics in the\ncontext of the document they are part of, and therefore can help in improving\nkeyphrase extraction quality. Empirical results offer evidence that indeed\nlocal representations lead to better keyphrase extraction results compared to\nboth embeddings trained on very large third corpora or larger corpora\nconsisting of several documents of the same scientific field and to other\nstate-of-the-art unsupervised keyphrase extraction methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07695v1",
            "date": "2017-10-20 20:23:45+00:00",
            "title": "Verb Pattern: A Probabilistic Semantic Representation on Verbs",
            "authors": [
                "Wanyun Cui",
                "Xiyou Zhou",
                "Hangyu Lin",
                "Yanghua Xiao",
                "Haixun Wang",
                "Seung-won Hwang",
                "Wei Wang"
            ],
            "abstract": "Verbs are important in semantic understanding of natural language.\nTraditional verb representations, such as FrameNet, PropBank, VerbNet, focus on\nverbs' roles. These roles are too coarse to represent verbs' semantics. In this\npaper, we introduce verb patterns to represent verbs' semantics, such that each\npattern corresponds to a single semantic of the verb. First we analyze the\nprinciples for verb patterns: generality and specificity. Then we propose a\nnonparametric model based on description length. Experimental results prove the\nhigh effectiveness of verb patterns. We further apply verb patterns to\ncontext-aware conceptualization, to show that verb patterns are helpful in\nsemantic-related tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07770v1",
            "date": "2017-10-21 08:06:48+00:00",
            "title": "Text Coherence Analysis Based on Deep Neural Network",
            "authors": [
                "Baiyun Cui",
                "Yingming Li",
                "Yaqing Zhang",
                "Zhongfei Zhang"
            ],
            "abstract": "In this paper, we propose a novel deep coherence model (DCM) using a\nconvolutional neural network architecture to capture the text coherence. The\ntext coherence problem is investigated with a new perspective of learning\nsentence distributional representation and text coherence modeling\nsimultaneously. In particular, the model captures the interactions between\nsentences by computing the similarities of their distributional\nrepresentations. Further, it can be easily trained in an end-to-end fashion.\nThe proposed model is evaluated on a standard Sentence Ordering task. The\nexperimental results demonstrate its effectiveness and promise in coherence\nassessment showing a significant improvement over the state-of-the-art by a\nwide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07960v1",
            "date": "2017-10-22 15:12:43+00:00",
            "title": "How big is big enough? Unsupervised word sense disambiguation using a very large corpus",
            "authors": [
                "Piotr Przyby\u0142a"
            ],
            "abstract": "In this paper, the problem of disambiguating a target word for Polish is\napproached by searching for related words with known meaning. These relatives\nare used to build a training corpus from unannotated text. This technique is\nimproved by proposing new rich sources of replacements that substitute the\ntraditional requirement of monosemy with heuristics based on wordnet relations.\nThe na\\\"ive Bayesian classifier has been modified to account for an unknown\ndistribution of senses. A corpus of 600 million web documents (594 billion\ntokens), gathered by the NEKST search engine allows us to assess the\nrelationship between training set size and disambiguation accuracy. The\nclassifier is evaluated using both a wordnet baseline and a corpus with 17,314\nmanually annotated occurrences of 54 ambiguous words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08015v1",
            "date": "2017-10-22 21:03:28+00:00",
            "title": "Bringing Semantic Structures to User Intent Detection in Online Medical Queries",
            "authors": [
                "Chenwei Zhang",
                "Nan Du",
                "Wei Fan",
                "Yaliang Li",
                "Chun-Ta Lu",
                "Philip S. Yu"
            ],
            "abstract": "The Internet has revolutionized healthcare by offering medical information\nubiquitously to patients via web search. The healthcare status, complex medical\ninformation needs of patients are expressed diversely and implicitly in their\nmedical text queries. Aiming to better capture a focused picture of user's\nmedical-related information search and shed insights on their healthcare\ninformation access strategies, it is challenging yet rewarding to detect\nstructured user intentions from their diversely expressed medical text queries.\nWe introduce a graph-based formulation to explore structured concept\ntransitions for effective user intent detection in medical queries, where each\nnode represents a medical concept mention and each directed edge indicates a\nmedical concept transition. A deep model based on multi-task learning is\nintroduced to extract structured semantic transitions from user queries, where\nthe model extracts word-level medical concept mentions as well as\nsentence-level concept transitions collectively. A customized graph-based\nmutual transfer loss function is designed to impose explicit constraints and\nfurther exploit the contribution of mentioning a medical concept word to the\nimplication of a semantic transition. We observe an 8% relative improvement in\nAUC and 23% relative reduction in coverage error by comparing the proposed\nmodel with the best baseline model for the concept transition inference task on\nreal-world medical text queries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08048v1",
            "date": "2017-10-23 00:26:50+00:00",
            "title": "A First Step in Combining Cognitive Event Features and Natural Language Representations to Predict Emotions",
            "authors": [
                "Andres Campero",
                "Bjarke Felbo",
                "Joshua B. Tenenbaum",
                "Rebecca Saxe"
            ],
            "abstract": "We explore the representational space of emotions by combining methods from\ndifferent academic fields. Cognitive science has proposed appraisal theory as a\nview on human emotion with previous research showing how human-rated abstract\nevent features can predict fine-grained emotions and capture the similarity\nspace of neural patterns in mentalizing brain regions. At the same time,\nnatural language processing (NLP) has demonstrated how transfer and multitask\nlearning can be used to cope with scarcity of annotated data for text modeling.\n  The contribution of this work is to show that appraisal theory can be\ncombined with NLP for mutual benefit. First, fine-grained emotion prediction\ncan be improved to human-level performance by using NLP representations in\naddition to appraisal features. Second, using the appraisal features as\nauxiliary targets during training can improve predictions even when only text\nis available as input. Third, we obtain a representation with a similarity\nmatrix that better correlates with the neural activity across regions. Best\nresults are achieved when the model is trained to simultaneously predict\nappraisals, emotions and emojis using a shared representation.\n  While these results are preliminary, the integration of cognitive\nneuroscience and NLP techniques opens up an interesting direction for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08246v1",
            "date": "2017-10-23 12:58:12+00:00",
            "title": "Testing the limits of unsupervised learning for semantic similarity",
            "authors": [
                "Richa Sharma",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "Semantic Similarity between two sentences can be defined as a way to\ndetermine how related or unrelated two sentences are. The task of Semantic\nSimilarity in terms of distributed representations can be thought to be\ngenerating sentence embeddings (dense vectors) which take both context and\nmeaning of sentence in account. Such embeddings can be produced by multiple\nmethods, in this paper we try to evaluate LSTM auto encoders for generating\nthese embeddings. Unsupervised algorithms (auto encoders to be specific) just\ntry to recreate their inputs, but they can be forced to learn order (and some\ninherent meaning to some extent) by creating proper bottlenecks. We try to\nevaluate how properly can algorithms trained just on plain English Sentences\nlearn to figure out Semantic Similarity, without giving them any sense of what\nmeaning of a sentence is.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08312v2",
            "date": "2017-10-23 14:46:58+00:00",
            "title": "Attending to All Mention Pairs for Full Abstract Biological Relation Extraction",
            "authors": [
                "Patrick Verga",
                "Emma Strubell",
                "Ofer Shai",
                "Andrew McCallum"
            ],
            "abstract": "Most work in relation extraction forms a prediction by looking at a short\nspan of text within a single sentence containing a single entity pair mention.\nHowever, many relation types, particularly in biomedical text, are expressed\nacross sentences or require a large context to disambiguate. We propose a model\nto consider all mention and entity pairs simultaneously in order to make a\nprediction. We encode full paper abstracts using an efficient self-attention\nencoder and form pairwise predictions between all mentions with a bi-affine\noperation. An entity-pair wise pooling aggregates mention pair scores to make a\nfinal prediction while alleviating training noise by performing within document\nmulti-instance learning. We improve our model's performance by jointly training\nthe model to predict named entities and adding an additional corpus of weakly\nlabeled data. We demonstrate our model's effectiveness by achieving the state\nof the art on the Biocreative V Chemical Disease Relation dataset for models\nwithout KB resources, outperforming ensembles of models which use hand-crafted\nfeatures and additional linguistic resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08451v1",
            "date": "2017-10-23 18:34:37+00:00",
            "title": "Combining Lexical Features and a Supervised Learning Approach for Arabic Sentiment Analysis",
            "authors": [
                "Samhaa R. El-Beltagy",
                "Talaat Khalil",
                "Amal Halaby",
                "Muhammad Hammad"
            ],
            "abstract": "The importance of building sentiment analysis tools for Arabic social media\nhas been recognized during the past couple of years, especially with the rapid\nincrease in the number of Arabic social media users. One of the main\ndifficulties in tackling this problem is that text within social media is\nmostly colloquial, with many dialects being used within social media platforms.\nIn this paper, we present a set of features that were integrated with a machine\nlearning based sentiment analysis model and applied on Egyptian, Saudi,\nLevantine, and MSA Arabic social media datasets. Many of the proposed features\nwere derived through the use of an Arabic Sentiment Lexicon. The model also\npresents emoticon based features, as well as input text related features such\nas the number of segments within the text, the length of the text, whether the\ntext ends with a question mark or not, etc. We show that the presented features\nhave resulted in an increased accuracy across six of the seven datasets we've\nexperimented with and which are all benchmarked. Since the developed model\nout-performs all existing Arabic sentiment analysis systems that have publicly\navailable datasets, we can state that this model presents state-of-the-art in\nArabic sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08458v1",
            "date": "2017-10-23 18:55:19+00:00",
            "title": "NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis",
            "authors": [
                "Samhaa R. El-Beltagy",
                "Mona El Kalamawy",
                "Abu Bakr Soliman"
            ],
            "abstract": "This paper describes two systems that were used by the authors for addressing\nArabic Sentiment Analysis as part of SemEval-2017, task 4. The authors\nparticipated in three Arabic related subtasks which are: Subtask A (Message\nPolarity Classification), Sub-task B (Topic-Based Message Polarity\nclassification) and Subtask D (Tweet quantification) using the team name of\nNileTMRG. For subtask A, we made use of our previously developed sentiment\nanalyzer which we augmented with a scored lexicon. For subtasks B and D, we\nused an ensemble of three different classifiers. The first classifier was a\nconvolutional neural network for which we trained (word2vec) word embeddings.\nThe second classifier consisted of a MultiLayer Perceptron, while the third\nclassifier was a Logistic regression model that takes the same input as the\nsecond classifier. Voting between the three classifiers was used to determine\nthe final outcome. The output from task B, was quantified to produce the\nresults for task D. In all three Arabic related tasks in which NileTMRG\nparticipated, the team ranked at number one.",
            "journal ref": "Proceedings of the 11th International Workshop on Semantic\n  Evaluations (SemEval-2017), pages 790-795, Vancouver, Canada, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08691v3",
            "date": "2017-10-24 10:15:58+00:00",
            "title": "BENGAL: An Automatic Benchmark Generator for Entity Recognition and Linking",
            "authors": [
                "Axel-Cyrille Ngonga Ngomo",
                "Michael R\u00f6der",
                "Diego Moussallem",
                "Ricardo Usbeck",
                "Ren\u00e9 Speck"
            ],
            "abstract": "The manual creation of gold standards for named entity recognition and entity\nlinking is time- and resource-intensive. Moreover, recent works show that such\ngold standards contain a large proportion of mistakes in addition to being\ndifficult to maintain. We hence present BENGAL, a novel automatic generation of\nsuch gold standards as a complement to manually created benchmarks. The main\nadvantage of our benchmarks is that they can be readily generated at any time.\nThey are also cost-effective while being guaranteed to be free of annotation\nerrors. We compare the performance of 11 tools on benchmarks in English\ngenerated by BENGAL and on 16benchmarks created manually. We show that our\napproach can be ported easily across languages by presenting results achieved\nby 4 tools on both Brazilian Portuguese and Spanish. Overall, our results\nsuggest that our automatic benchmark generation approach can create varied\nbenchmarks that have characteristics similar to those of existing benchmarks.\nOur approach is open-source. Our experimental results are available at\nhttp://faturl.com/bengalexpinlg and the code at\nhttps://github.com/dice-group/BENGAL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08721v1",
            "date": "2017-10-24 12:11:07+00:00",
            "title": "Clickbait Identification using Neural Networks",
            "authors": [
                "Philippe Thomas"
            ],
            "abstract": "This paper presents the results of our participation in the Clickbait\nDetection Challenge 2017. The system relies on a fusion of neural networks,\nincorporating different types of available informations. It does not require\nany linguistic preprocessing, and hence generalizes more easily to new domains\nand languages. The final combined model achieves a mean squared error of\n0.0428, an accuracy of 0.826, and a F1 score of 0.564. According to the\nofficial evaluation metric the system ranked 6th of the 13 participating teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09137v1",
            "date": "2017-10-25 09:45:58+00:00",
            "title": "Linking Tweets with Monolingual and Cross-Lingual News using Transformed Word Embeddings",
            "authors": [
                "Aditya Mogadala",
                "Dominik Jung",
                "Achim Rettinger"
            ],
            "abstract": "Social media platforms have grown into an important medium to spread\ninformation about an event published by the traditional media, such as news\narticles. Grouping such diverse sources of information that discuss the same\ntopic in varied perspectives provide new insights. But the gap in word usage\nbetween informal social media content such as tweets and diligently written\ncontent (e.g. news articles) make such assembling difficult. In this paper, we\npropose a transformation framework to bridge the word usage gap between tweets\nand online news articles across languages by leveraging their word embeddings.\nUsing our framework, word embeddings extracted from tweets and news articles\nare aligned closer to each other across languages, thus facilitating the\nidentification of similarity between news articles and tweets. Experimental\nresults show a notable improvement over baselines for monolingual tweets and\nnews articles comparison, while new findings are reported for cross-lingual\ncomparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09233v1",
            "date": "2017-10-24 16:08:58+00:00",
            "title": "A Simple Text Analytics Model To Assist Literary Criticism: comparative approach and example on James Joyce against Shakespeare and the Bible",
            "authors": [
                "Renato Fabbri",
                "Luis Henrique Garcia"
            ],
            "abstract": "Literary analysis, criticism or studies is a largely valued field with\ndedicated journals and researchers which remains mostly within the humanities\nscope. Text analytics is the computer-aided process of deriving information\nfrom texts. In this article we describe a simple and generic model for\nperforming literary analysis using text analytics. The method relies on\nstatistical measures of: 1) token and sentence sizes and 2) Wordnet synset\nfeatures. These measures are then used in Principal Component Analysis where\nthe texts to be analyzed are observed against Shakespeare and the Bible,\nregarded as reference literature. The model is validated by analyzing selected\nworks from James Joyce (1882-1941), one of the most important writers of the\n20th century. We discuss the consistency of this approach, the reasons why we\ndid not use other techniques (e.g. part-of-speech tagging) and the ways by\nwhich the analysis model might be adapted and enhanced.",
            "journal ref": "Anais do XX ENMC - Encontro Nacional de Modelagem Computacional e\n  VIII ECTM - Encontro de Ci\\^encias e Tecnologia de Materiais, Nova Friburgo,\n  RJ - 16 a 19 Outubro 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09306v1",
            "date": "2017-10-25 15:34:52+00:00",
            "title": "Exploring the Use of Text Classification in the Legal Domain",
            "authors": [
                "Octavia-Maria Sulea",
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Mihaela Vela",
                "Liviu P. Dinu",
                "Josef van Genabith"
            ],
            "abstract": "In this paper, we investigate the application of text classification methods\nto support law professionals. We present several experiments applying machine\nlearning techniques to predict with high accuracy the ruling of the French\nSupreme Court and the law area to which a case belongs to. We also investigate\nthe influence of the time period in which a ruling was made on the form of the\ncase description and the extent to which we need to mask information in a full\ncase ruling to automatically obtain training and test data that resembles case\ndescriptions. We developed a mean probability ensemble system combining the\noutput of multiple SVM classifiers. We report results of 98% average F1 score\nin predicting a case ruling, 96% F1 score for predicting the law area of a\ncase, and 87.07% F1 score on estimating the date of a ruling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09340v3",
            "date": "2017-10-25 16:57:51+00:00",
            "title": "Non-Projective Dependency Parsing with Non-Local Transitions",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present a novel transition system, based on the Covington non-projective\nparser, introducing non-local transitions that can directly create arcs\ninvolving nodes to the left of the current focus positions. This avoids the\nneed for long sequences of No-Arc transitions to create long-distance arcs,\nthus alleviating error propagation. The resulting parser outperforms the\noriginal version and achieves the best accuracy on the Stanford Dependencies\nconversion of the Penn Treebank among greedy transition-based algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09589v1",
            "date": "2017-10-26 08:41:50+00:00",
            "title": "ALL-IN-1: Short Text Classification with One Model for All Languages",
            "authors": [
                "Barbara Plank"
            ],
            "abstract": "We present ALL-IN-1, a simple model for multilingual text classification that\ndoes not require any parallel data. It is based on a traditional Support Vector\nMachine classifier exploiting multilingual word embeddings and character\nn-grams. Our model is simple, easily extendable yet very effective, overall\nranking 1st (out of 12 teams) in the IJCNLP 2017 shared task on customer\nfeedback analysis in four languages: English, French, Japanese and Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09617v1",
            "date": "2017-10-26 09:50:16+00:00",
            "title": "Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models",
            "authors": [
                "Yanzhang He",
                "Rohit Prabhavalkar",
                "Kanishka Rao",
                "Wei Li",
                "Anton Bakhtin",
                "Ian McGraw"
            ],
            "abstract": "We develop streaming keyword spotting systems using a recurrent neural\nnetwork transducer (RNN-T) model: an all-neural, end-to-end trained,\nsequence-to-sequence model which jointly learns acoustic and language model\ncomponents. Our models are trained to predict either phonemes or graphemes as\nsubword units, thus allowing us to detect arbitrary keyword phrases, without\nany out-of-vocabulary words. In order to adapt the models to the requirements\nof keyword spotting, we propose a novel technique which biases the RNN-T system\ntowards a specific keyword of interest.\n  Our systems are compared against a strong sequence-trained, connectionist\ntemporal classification (CTC) based \"keyword-filler\" baseline, which is\naugmented with a separate phoneme language model. Overall, our RNN-T system\nwith the proposed biasing technique significantly improves performance over the\nbaseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09753v1",
            "date": "2017-10-26 15:25:37+00:00",
            "title": "Impact of Coreference Resolution on Slot Filling",
            "authors": [
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In this paper, we demonstrate the importance of coreference resolution for\nnatural language processing on the example of the TAC Slot Filling shared task.\nWe illustrate the strengths and weaknesses of automatic coreference resolution\nsystems and provide experimental results to show that they improve performance\nin the slot filling end-to-end setting. Finally, we publish KBPchains, a\nresource containing automatically extracted coreference chains from the TAC\nsource corpus in order to support other researchers working on this topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.09942v1",
            "date": "2017-10-26 23:16:31+00:00",
            "title": "CANDiS: Coupled & Attention-Driven Neural Distant Supervision",
            "authors": [
                "Tushar Nagarajan",
                "Sharmistha",
                "Partha Talukdar"
            ],
            "abstract": "Distant Supervision for Relation Extraction uses heuristically aligned text\ndata with an existing knowledge base as training data. The unsupervised nature\nof this technique allows it to scale to web-scale relation extraction tasks, at\nthe expense of noise in the training data. Previous work has explored\nrelationships among instances of the same entity-pair to reduce this noise, but\nrelationships among instances across entity-pairs have not been fully\nexploited. We explore the use of inter-instance couplings based on verb-phrase\nand entity type similarities. We propose a novel technique, CANDiS, which casts\ndistant supervision using inter-instance coupling into an end-to-end neural\nnetwork model. CANDiS incorporates an attention module at the instance-level to\nmodel the multi-instance nature of this problem. CANDiS outperforms existing\nstate-of-the-art techniques on a standard benchmark dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10361v2",
            "date": "2017-10-28 00:43:01+00:00",
            "title": "Deep Residual Learning for Small-Footprint Keyword Spotting",
            "authors": [
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "We explore the application of deep residual learning and dilated convolutions\nto the keyword spotting task, using the recently-released Google Speech\nCommands Dataset as our benchmark. Our best residual network (ResNet)\nimplementation significantly outperforms Google's previous convolutional neural\nnetworks in terms of accuracy. By varying model depth and width, we can achieve\ncompact models that also outperform previous small-footprint variants. To our\nknowledge, we are the first to examine these approaches for keyword spotting,\nand our results establish an open-source state-of-the-art reference to support\nthe development of future speech-based interfaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10398v2",
            "date": "2017-10-28 06:24:36+00:00",
            "title": "A Study of All-Convolutional Encoders for Connectionist Temporal Classification",
            "authors": [
                "Kalpesh Krishna",
                "Liang Lu",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "Connectionist temporal classification (CTC) is a popular sequence prediction\napproach for automatic speech recognition that is typically used with models\nbased on recurrent neural networks (RNNs). We explore whether deep\nconvolutional neural networks (CNNs) can be used effectively instead of RNNs as\nthe \"encoder\" in CTC. CNNs lack an explicit representation of the entire\nsequence, but have the advantage that they are much faster to train. We present\nan exploration of CNNs as encoders for CTC models, in the context of\ncharacter-based (lexicon-free) automatic speech recognition. In particular, we\nexplore a range of one-dimensional convolutional layers, which are particularly\nefficient. We compare the performance of our CNN-based models against typical\nRNNbased models in terms of training time, decoding time, model size and word\nerror rate (WER) on the Switchboard Eval2000 corpus. We find that our CNN-based\nmodels are close in performance to LSTMs, while not matching them, and are much\nfaster to train and decode.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10453v2",
            "date": "2017-10-28 12:00:09+00:00",
            "title": "Inducing Regular Grammars Using Recurrent Neural Networks",
            "authors": [
                "Mor Cohen",
                "Avi Caciularu",
                "Idan Rejwan",
                "Jonathan Berant"
            ],
            "abstract": "Grammar induction is the task of learning a grammar from a set of examples.\nRecently, neural networks have been shown to be powerful learning machines that\ncan identify patterns in streams of data. In this work we investigate their\neffectiveness in inducing a regular grammar from data, without any assumptions\nabout the grammar. We train a recurrent neural network to distinguish between\nstrings that are in or outside a regular language, and utilize an algorithm for\nextracting the learned finite-state automaton. We apply this method to several\nregular languages and find unexpected results regarding the connections between\nthe network's states that may be regarded as evidence for generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10504v2",
            "date": "2017-10-28 17:28:04+00:00",
            "title": "Phase Conductor on Multi-layered Attentions for Machine Comprehension",
            "authors": [
                "Rui Liu",
                "Wei Wei",
                "Weiguang Mao",
                "Maria Chikina"
            ],
            "abstract": "Attention models have been intensively studied to improve NLP tasks such as\nmachine comprehension via both question-aware passage attention model and\nself-matching attention model. Our research proposes phase conductor\n(PhaseCond) for attention models in two meaningful ways. First, PhaseCond, an\narchitecture of multi-layered attention models, consists of multiple phases\neach implementing a stack of attention layers producing passage representations\nand a stack of inner or outer fusion layers regulating the information flow.\nSecond, we extend and improve the dot-product attention function for PhaseCond\nby simultaneously encoding multiple question and passage embedding layers from\ndifferent perspectives. We demonstrate the effectiveness of our proposed model\nPhaseCond on the SQuAD dataset, showing that our model significantly\noutperforms both state-of-the-art single-layered and multiple-layered attention\nmodels. We deepen our results with new findings via both detailed qualitative\nanalysis and visualized examples showing the dynamic changes through\nmulti-layered attention models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10520v1",
            "date": "2017-10-28 19:40:47+00:00",
            "title": "A Dual Encoder Sequence to Sequence Model for Open-Domain Dialogue Modeling",
            "authors": [
                "Sharath T. S.",
                "Shubhangi Tandon",
                "Ryan Bauer"
            ],
            "abstract": "Ever since the successful application of sequence to sequence learning for\nneural machine translation systems, interest has surged in its applicability\ntowards language generation in other problem domains. Recent work has\ninvestigated the use of these neural architectures towards modeling open-domain\nconversational dialogue, where it has been found that although these models are\ncapable of learning a good distributional language model, dialogue coherence is\nstill of concern. Unlike translation, conversation is much more a one-to-many\nmapping from utterance to a response, and it is even more pressing that the\nmodel be aware of the preceding flow of conversation. In this paper we propose\nto tackle this problem by introducing previous conversational context in terms\nof latent representations of dialogue acts over time. We inject the latent\ncontext representations into a sequence to sequence neural network in the form\nof dialog acts using a second encoder to enhance the quality and the coherence\nof the conversations generated. The main task of this research work is to show\nthat adding latent variables that capture discourse relations does indeed\nresult in more coherent responses when compared to conventional sequence to\nsequence models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10574v1",
            "date": "2017-10-29 08:04:24+00:00",
            "title": "Personalized word representations Carrying Personalized Semantics Learned from Social Network Posts",
            "authors": [
                "Zih-Wei Lin",
                "Tzu-Wei Sung",
                "Hung-Yi Lee",
                "Lin-Shan Lee"
            ],
            "abstract": "Distributed word representations have been shown to be very useful in various\nnatural language processing (NLP) application tasks. These word vectors learned\nfrom huge corpora very often carry both semantic and syntactic information of\nwords. However, it is well known that each individual user has his own language\npatterns because of different factors such as interested topics, friend groups,\nsocial activities, wording habits, etc., which may imply some kind of\npersonalized semantics. With such personalized semantics, the same word may\nimply slightly differently for different users. For example, the word\n\"Cappuccino\" may imply \"Leisure\", \"Joy\", \"Excellent\" for a user enjoying\ncoffee, by only a kind of drink for someone else. Such personalized semantics\nof course cannot be carried by the standard universal word vectors trained with\nhuge corpora produced by many people. In this paper, we propose a framework to\ntrain different personalized word vectors for different users based on the very\nsuccessful continuous skip-gram model using the social network data posted by\nmany individual users. In this framework, universal background word vectors are\nfirst learned from the background corpora, and then adapted by the personalized\ncorpus for each individual user to learn the personalized word vectors. We use\ntwo application tasks to evaluate the quality of the personalized word vectors\nobtained in this way, the user prediction task and the sentence completion\ntask. These personalized word vectors were shown to carry some personalized\nsemantics and offer improved performance on these two evaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10585v2",
            "date": "2017-10-29 09:24:56+00:00",
            "title": "Path-Based Attention Neural Model for Fine-Grained Entity Typing",
            "authors": [
                "Denghui Zhang",
                "Pengshan Cai",
                "Yantao Jia",
                "Manling Li",
                "Yuanzhuo Wang",
                "Xueqi Cheng"
            ],
            "abstract": "Fine-grained entity typing aims to assign entity mentions in the free text\nwith types arranged in a hierarchical structure. Traditional distant\nsupervision based methods employ a structured data source as a weak supervision\nand do not need hand-labeled data, but they neglect the label noise in the\nautomatically labeled training corpus. Although recent studies use many\nfeatures to prune wrong data ahead of training, they suffer from error\npropagation and bring much complexity. In this paper, we propose an end-to-end\ntyping model, called the path-based attention neural model (PAN), to learn a\nnoise- robust performance by leveraging the hierarchical structure of types.\nExperiments demonstrate its effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10586v1",
            "date": "2017-10-29 09:37:02+00:00",
            "title": "Evaluation of Automatic Video Captioning Using Direct Assessment",
            "authors": [
                "Yvette Graham",
                "George Awad",
                "Alan Smeaton"
            ],
            "abstract": "We present Direct Assessment, a method for manually assessing the quality of\nautomatically-generated captions for video. Evaluating the accuracy of video\ncaptions is particularly difficult because for any given video clip there is no\ndefinitive ground truth or correct answer against which to measure. Automatic\nmetrics for comparing automatic video captions against a manual caption such as\nBLEU and METEOR, drawn from techniques used in evaluating machine translation,\nwere used in the TRECVid video captioning task in 2016 but these are shown to\nhave weaknesses. The work presented here brings human assessment into the\nevaluation by crowdsourcing how well a caption describes a video. We\nautomatically degrade the quality of some sample captions which are assessed\nmanually and from this we are able to rate the quality of the human assessors,\na factor we take into account in the evaluation. Using data from the TRECVid\nvideo-to-text task in 2016, we show how our direct assessment method is\nreplicable and robust and should scale to where there many caption-generation\ntechniques to be evaluated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10609v1",
            "date": "2017-10-29 13:21:18+00:00",
            "title": "Finding Dominant User Utterances And System Responses in Conversations",
            "authors": [
                "Dhiraj Madan",
                "Sachindra Joshi"
            ],
            "abstract": "There are several dialog frameworks which allow manual specification of\nintents and rule based dialog flow. The rule based framework provides good\ncontrol to dialog designers at the expense of being more time consuming and\nlaborious. The job of a dialog designer can be reduced if we could identify\npairs of user intents and corresponding responses automatically from prior\nconversations between users and agents. In this paper we propose an approach to\nfind these frequent user utterances (which serve as examples for intents) and\ncorresponding agent responses. We propose a novel SimCluster algorithm that\nextends standard K-means algorithm to simultaneously cluster user utterances\nand agent utterances by taking their adjacency information into account. The\nmethod also aligns these clusters to provide pairs of intents and response\ngroups. We compare our results with those produced by using simple Kmeans\nclustering on a real dataset and observe upto 10% absolute improvement in\nF1-scores. Through our experiments on synthetic dataset, we show that our\nalgorithm gains more advantage over K-means algorithm when the data has large\nvariance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10639v4",
            "date": "2017-10-29 16:15:30+00:00",
            "title": "JESC: Japanese-English Subtitle Corpus",
            "authors": [
                "Reid Pryzant",
                "Yongjoo Chung",
                "Dan Jurafsky",
                "Denny Britz"
            ],
            "abstract": "In this paper we describe the Japanese-English Subtitle Corpus (JESC). JESC\nis a large Japanese-English parallel corpus covering the underrepresented\ndomain of conversational dialogue. It consists of more than 3.2 million\nexamples, making it the largest freely available dataset of its kind. The\ncorpus was assembled by crawling and aligning subtitles found on the web. The\nassembly process incorporates a number of novel preprocessing elements to\nensure high monolingual fluency and accurate bilingual alignments. We summarize\nits contents and evaluate its quality using human experts and baseline machine\ntranslation (MT) systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10723v2",
            "date": "2017-10-29 23:47:49+00:00",
            "title": "Simple and Effective Multi-Paragraph Reading Comprehension",
            "authors": [
                "Christopher Clark",
                "Matt Gardner"
            ],
            "abstract": "We consider the problem of adapting neural paragraph-level question answering\nmodels to the case where entire documents are given as input. Our proposed\nsolution trains models to produce well calibrated confidence scores for their\nresults on individual paragraphs. We sample multiple paragraphs from the\ndocuments during training, and use a shared-normalization training objective\nthat encourages the model to produce globally correct output. We combine this\nmethod with a state-of-the-art pipeline for training models on document QA\ndata. Experiments demonstrate strong performance on several document QA\ndatasets. Overall, we are able to achieve a score of 71.3 F1 on the web portion\nof TriviaQA, a large improvement from the 56.7 F1 of the previous best system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11035v2",
            "date": "2017-10-30 16:04:04+00:00",
            "title": "Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German",
            "authors": [
                "Pierre-Edouard Honnet",
                "Andrei Popescu-Belis",
                "Claudiu Musat",
                "Michael Baeriswyl"
            ],
            "abstract": "The goal of this work is to design a machine translation (MT) system for a\nlow-resource family of dialects, collectively known as Swiss German, which are\nwidely spoken in Switzerland but seldom written. We collected a significant\nnumber of parallel written resources to start with, up to a total of about 60k\nwords. Moreover, we identified several other promising data sources for Swiss\nGerman. Then, we designed and compared three strategies for normalizing Swiss\nGerman input in order to address the regional diversity. We found that\ncharacter-based neural MT was the best solution for text normalization. In\ncombination with phrase-based statistical MT, our solution reached 36% BLEU\nscore when translating from the Bernese dialect. This value, however, decreases\nas the testing data becomes more remote from the training one, geographically\nand topically. These resources and normalization techniques are a first step\ntowards full MT of Swiss German dialects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11154v1",
            "date": "2017-10-30 18:05:57+00:00",
            "title": "Creation of an Annotated Corpus of Spanish Radiology Reports",
            "authors": [
                "Viviana Cotik",
                "Dar\u00edo Filippo",
                "Roland Roller",
                "Hans Uszkoreit",
                "Feiyu Xu"
            ],
            "abstract": "This paper presents a new annotated corpus of 513 anonymized radiology\nreports written in Spanish. Reports were manually annotated with entities,\nnegation and uncertainty terms and relations. The corpus was conceived as an\nevaluation resource for named entity recognition and relation extraction\nalgorithms, and as input for the use of supervised methods. Biomedical\nannotated resources are scarce due to confidentiality issues and associated\ncosts. This work provides some guidelines that could help other researchers to\nundertake similar tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11332v1",
            "date": "2017-10-31 05:43:52+00:00",
            "title": "Improving Social Media Text Summarization by Learning Sentence Weight Distribution",
            "authors": [
                "Jingjing Xu"
            ],
            "abstract": "Recently, encoder-decoder models are widely used in social media text\nsummarization. However, these models sometimes select noise words in irrelevant\nsentences as part of a summary by error, thus declining the performance. In\norder to inhibit irrelevant sentences and focus on key information, we propose\nan effective approach by learning sentence weight distribution. In our model,\nwe build a multi-layer perceptron to predict sentence weights. During training,\nwe use the ROUGE score as an alternative to the estimated sentence weight, and\ntry to minimize the gap between estimated weights and predicted weights. In\nthis way, we encourage our model to focus on the key sentences, which have high\nrelevance with the summary. Experimental results show that our approach\noutperforms baselines on a large-scale social media corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11334v1",
            "date": "2017-10-31 05:47:38+00:00",
            "title": "Shallow Discourse Parsing with Maximum Entropy Model",
            "authors": [
                "Jingjing Xu"
            ],
            "abstract": "In recent years, more research has been devoted to studying the subtask of\nthe complete shallow discourse parsing, such as indentifying discourse\nconnective and arguments of connective. There is a need to design a full\ndiscourse parser to pull these subtasks together. So we develop a discourse\nparser turning the free text into discourse relations. The parser includes\nconnective identifier, arguments identifier, sense classifier and non-explicit\nidentifier, which connects with each other in pipeline. Each component applies\nthe maximum entropy model with abundant lexical and syntax features extracted\nfrom the Penn Discourse Tree-bank. The head-based representation of the PDTB is\nadopted in the arguments identifier, which turns the problem of indentifying\nthe arguments of discourse connective into finding the head and end of the\narguments. In the non-explicit identifier, the contextual type features like\nwords which have high frequency and can reflect the discourse relation are\nintroduced to improve the performance of non-explicit identifier. Compared with\nother methods, experimental results achieve the considerable performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11344v1",
            "date": "2017-10-31 06:29:11+00:00",
            "title": "A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Chen Xing",
                "Can Xu",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "We study the problem of response selection for multi-turn conversation in\nretrieval-based chatbots. The task requires matching a response candidate with\na conversation context, whose challenges include how to recognize important\nparts of the context, and how to model the relationships among utterances in\nthe context. Existing matching methods may lose important information in\ncontexts as we can interpret them with a unified framework in which contexts\nare transformed to fixed-length vectors without any interaction with responses\nbefore matching. The analysis motivates us to propose a new matching framework\nthat can sufficiently carry the important information in contexts to matching\nand model the relationships among utterances at the same time. The new\nframework, which we call a sequential matching framework (SMF), lets each\nutterance in a context interacts with a response candidate at the first step\nand transforms the pair to a matching vector. The matching vectors are then\naccumulated following the order of the utterances in the context with a\nrecurrent neural network (RNN) which models the relationships among the\nutterances. The context-response matching is finally calculated with the hidden\nstates of the RNN. Under SMF, we propose a sequential convolutional network and\nsequential attention network and conduct experiments on two public data sets to\ntest their performance. Experimental results show that both models can\nsignificantly outperform the state-of-the-art matching methods. We also show\nthat the models are interpretable with visualizations that provide us insights\non how they capture and leverage the important information in contexts for\nmatching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11350v3",
            "date": "2017-10-31 07:09:14+00:00",
            "title": "Grammar Induction for Minimalist Grammars using Variational Bayesian Inference : A Technical Report",
            "authors": [
                "Eva Portelance",
                "Amelia Bruno",
                "Daniel Harasim",
                "Leon Bergen",
                "Timothy J. O'Donnell"
            ],
            "abstract": "The following technical report presents a formal approach to probabilistic\nminimalist grammar parameter estimation. We describe a formalization of a\nminimalist grammar. We then present an algorithm for the application of\nvariational Bayesian inference to this formalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11475v2",
            "date": "2017-10-29 09:18:51+00:00",
            "title": "A Neural-Symbolic Approach to Design of CAPTCHA",
            "authors": [
                "Qiuyuan Huang",
                "Paul Smolensky",
                "Xiaodong He",
                "Li Deng",
                "Dapeng Wu"
            ],
            "abstract": "CAPTCHAs based on reading text are susceptible to machine-learning-based\nattacks due to recent significant advances in deep learning (DL). To address\nthis, this paper promotes image/visual captioning based CAPTCHAs, which is\nrobust against machine-learning-based attacks. To develop\nimage/visual-captioning-based CAPTCHAs, this paper proposes a new image\ncaptioning architecture by exploiting tensor product representations (TPR), a\nstructured neural-symbolic framework developed in cognitive science over the\npast 20 years, with the aim of integrating DL with explicit language structures\nand rules. We call it the Tensor Product Generation Network (TPGN). The key\nideas of TPGN are: 1) unsupervised learning of role-unbinding vectors of words\nvia a TPR-based deep neural network, and 2) integration of TPR with typical DL\narchitectures including Long Short-Term Memory (LSTM) models. The novelty of\nour approach lies in its ability to generate a sentence and extract partial\ngrammatical structure of the sentence by using role-unbinding vectors, which\nare obtained in an unsupervised manner. Experimental results demonstrate the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00188v2",
            "date": "2017-01-01 03:04:33+00:00",
            "title": "Aspect-augmented Adversarial Networks for Domain Adaptation",
            "authors": [
                "Yuan Zhang",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "We introduce a neural method for transfer learning between two (source and\ntarget) classification tasks or aspects over the same domain. Rather than\ntraining on target labels, we use a few keywords pertaining to source and\ntarget aspects indicating sentence relevance instead of document class labels.\nDocuments are encoded by learning to embed and softly select relevant sentences\nin an aspect-dependent manner. A shared classifier is trained on the source\nencoded documents and labels, and applied to target encoded documents. We\nensure transfer through aspect-adversarial training so that encoded documents\nare, as sets, aspect-invariant. Experimental results demonstrate that our\napproach outperforms different baselines and model variants on two datasets,\nyielding an improvement of 27% on a pathology dataset and 5% on a review\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00504v1",
            "date": "2017-01-02 19:23:55+00:00",
            "title": "Stance detection in online discussions",
            "authors": [
                "Peter Krejzl",
                "Barbora Hourov\u00e1",
                "Josef Steinberger"
            ],
            "abstract": "This paper describes our system created to detect stance in online\ndiscussions. The goal is to identify whether the author of a comment is in\nfavor of the given target or against. Our approach is based on a maximum\nentropy classifier, which uses surface-level, sentiment and domain-specific\nfeatures. The system was originally developed to detect stance in English\ntweets. We adapted it to process Czech news commentaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00576v1",
            "date": "2017-01-03 04:15:51+00:00",
            "title": "Shortcut Sequence Tagging",
            "authors": [
                "Huijia Wu",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Deep stacked RNNs are usually hard to train. Adding shortcut connections\nacross different layers is a common way to ease the training of stacked\nnetworks. However, extra shortcuts make the recurrent step more complicated. To\nsimply the stacked architecture, we propose a framework called shortcut block,\nwhich is a marriage of the gating mechanism and shortcuts, while discarding the\nself-connected part in LSTM cell. We present extensive empirical experiments\nshowing that this design makes training easy and improves generalization. We\npropose various shortcut block topologies and compositions to explore its\neffectiveness. Based on this architecture, we obtain a 6% relatively\nimprovement over the state-of-the-art on CCGbank supertagging dataset. We also\nget comparable results on POS tagging task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00728v1",
            "date": "2017-01-03 15:49:35+00:00",
            "title": "On (Commercial) Benefits of Automatic Text Summarization Systems in the News Domain: A Case of Media Monitoring and Media Response Analysis",
            "authors": [
                "Pashutan Modaresi",
                "Philipp Gross",
                "Siavash Sefidrodi",
                "Mirja Eckhof",
                "Stefan Conrad"
            ],
            "abstract": "In this work, we present the results of a systematic study to investigate the\n(commercial) benefits of automatic text summarization systems in a real world\nscenario. More specifically, we define a use case in the context of media\nmonitoring and media response analysis and claim that even using a simple\nquery-based extractive approach can dramatically save the processing time of\nthe employees without significantly reducing the quality of their work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00798v1",
            "date": "2017-01-03 19:41:24+00:00",
            "title": "Fuzzy Based Implicit Sentiment Analysis on Quantitative Sentences",
            "authors": [
                "Amir Hossein Yazdavar",
                "Monireh Ebrahimi",
                "Naomie Salim"
            ],
            "abstract": "With the rapid growth of social media on the web, emotional polarity\ncomputation has become a flourishing frontier in the text mining community.\nHowever, it is challenging to understand the latest trends and summarize the\nstate or general opinions about products due to the big diversity and size of\nsocial media data and this creates the need of automated and real time opinion\nextraction and mining. On the other hand, the bulk of current research has been\ndevoted to study the subjective sentences which contain opinion keywords and\nlimited work has been reported for objective statements that imply sentiment.\nIn this paper, fuzzy based knowledge engineering model has been developed for\nsentiment classification of special group of such sentences including the\nchange or deviation from desired range or value. Drug reviews are the rich\nsource of such statements. Therefore, in this research, some experiments were\ncarried out on patient's reviews on several different cholesterol lowering\ndrugs to determine their sentiment polarity. The main conclusion through this\nstudy is, in order to increase the accuracy level of existing drug opinion\nmining systems, objective sentences which imply opinion should be taken into\naccount. Our experimental results demonstrate that our proposed model obtains\nover 72 percent F1 value.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00946v3",
            "date": "2017-01-04 10:13:02+00:00",
            "title": "Joint Semantic Synthesis and Morphological Analysis of the Derived Word",
            "authors": [
                "Ryan Cotterell",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Much like sentences are composed of words, words themselves are composed of\nsmaller units. For example, the English word questionably can be analyzed as\nquestion+able+ly. However, this structural decomposition of the word does not\ndirectly give us a semantic representation of the word's meaning. Since\nmorphology obeys the principle of compositionality, the semantics of the word\ncan be systematically derived from the meaning of its parts. In this work, we\npropose a novel probabilistic model of word formation that captures both the\nanalysis of a word w into its constituents segments and the synthesis of the\nmeaning of w from the meanings of those segments. Our model jointly learns to\nsegment words into morphemes and compose distributional semantic vectors of\nthose morphemes. We experiment with the model on English CELEX data and German\nDerivBase (Zeller et al., 2013) data. We show that jointly modeling semantics\nincreases both segmentation accuracy and morpheme F1 by between 3% and 5%.\nAdditionally, we investigate different models of vector composition, showing\nthat recurrent neural networks yield an improvement over simple additive\nmodels. Finally, we study the degree to which the representations correspond to\na linguist's notion of morphological productivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01126v1",
            "date": "2017-01-04 19:14:37+00:00",
            "title": "Textual Entailment with Structured Attentions and Composition",
            "authors": [
                "Kai Zhao",
                "Liang Huang",
                "Mingbo Ma"
            ],
            "abstract": "Deep learning techniques are increasingly popular in the textual entailment\ntask, overcoming the fragility of traditional discrete models with hard\nalignments and logics. In particular, the recently proposed attention models\n(Rockt\\\"aschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art\naccuracy by computing soft word alignments between the premise and hypothesis\nsentences. However, there remains a major limitation: this line of work\ncompletely ignores syntax and recursion, which is helpful in many traditional\nefforts. We show that it is beneficial to extend the attention model to tree\nnodes between premise and hypothesis. More importantly, this subtree-level\nattention reveals information about entailment relation. We study the recursive\ncomposition of this subtree-level entailment relation, which can be viewed as a\nsoft version of the Natural Logic framework (MacCartney and Manning, 2009).\nExperiments show that our structured attention and entailment composition model\ncan correctly identify and infer entailment relations from the bottom up, and\nbring significant improvements in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01505v2",
            "date": "2017-01-05 23:35:12+00:00",
            "title": "Crime Topic Modeling",
            "authors": [
                "Da Kuang",
                "P. Jeffrey Brantingham",
                "Andrea L. Bertozzi"
            ],
            "abstract": "The classification of crime into discrete categories entails a massive loss\nof information. Crimes emerge out of a complex mix of behaviors and situations,\nyet most of these details cannot be captured by singular crime type labels.\nThis information loss impacts our ability to not only understand the causes of\ncrime, but also how to develop optimal crime prevention strategies. We apply\nmachine learning methods to short narrative text descriptions accompanying\ncrime records with the goal of discovering ecologically more meaningful latent\ncrime classes. We term these latent classes \"crime topics\" in reference to\ntext-based topic modeling methods that produce them. We use topic distributions\nto measure clustering among formally recognized crime types. Crime topics\nreplicate broad distinctions between violent and property crime, but also\nreveal nuances linked to target characteristics, situational conditions and the\ntools and methods of attack. Formal crime types are not discrete in topic\nspace. Rather, crime types are distributed across a range of crime topics.\nSimilarly, individual crime topics are distributed across a range of formal\ncrime types. Key ecological groups include identity theft, shoplifting,\nburglary and theft, car crimes and vandalism, criminal threats and confidence\ncrimes, and violent crimes. Though not a replacement for formal legal crime\nclassifications, crime topics provide a unique window into the heterogeneous\ncausal processes underlying crime.",
            "journal ref": "Kuang, D., Brantingham, P. J., & Bertozzi, A. L. (2017). Crime\n  topic modeling. Crime Science, 6(1), 12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01565v1",
            "date": "2017-01-06 08:18:38+00:00",
            "title": "Replication issues in syntax-based aspect extraction for opinion mining",
            "authors": [
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "Reproducing experiments is an important instrument to validate previous work\nand build upon existing approaches. It has been tackled numerous times in\ndifferent areas of science. In this paper, we introduce an empirical\nreplicability study of three well-known algorithms for syntactic centric\naspect-based opinion mining. We show that reproducing results continues to be a\ndifficult endeavor, mainly due to the lack of details regarding preprocessing\nand parameter setting, as well as due to the absence of available\nimplementations that clarify these details. We consider these are important\nthreats to validity of the research on the field, specifically when compared to\nother problems in NLP where public datasets and code availability are critical\nvalidity components. We conclude by encouraging code-based research, which we\nthink has a key role in helping researchers to understand the meaning of the\nstate-of-the-art better and to generate continuous advances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01574v1",
            "date": "2017-01-06 08:52:41+00:00",
            "title": "Real Multi-Sense or Pseudo Multi-Sense: An Approach to Improve Word Representation",
            "authors": [
                "Haoyue Shi",
                "Caihua Li",
                "Junfeng Hu"
            ],
            "abstract": "Previous researches have shown that learning multiple representations for\npolysemous words can improve the performance of word embeddings on many tasks.\nHowever, this leads to another problem. Several vectors of a word may actually\npoint to the same meaning, namely pseudo multi-sense. In this paper, we\nintroduce the concept of pseudo multi-sense, and then propose an algorithm to\ndetect such cases. With the consideration of the detected pseudo multi-sense\ncases, we try to refine the existing word embeddings to eliminate the influence\nof pseudo multi-sense. Moreover, we apply our algorithm on previous released\nmulti-sense word embeddings and tested it on artificial word similarity tasks\nand the analogy task. The result of the experiments shows that diminishing\npseudo multi-sense can improve the quality of word representations. Thus, our\nmethod is actually an efficient way to reduce linguistic complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01614v1",
            "date": "2017-01-06 12:28:15+00:00",
            "title": "Enumeration of Extractive Oracle Summaries",
            "authors": [
                "Tsutomu Hirao",
                "Masaaki Nishino",
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "To analyze the limitations and the future directions of the extractive\nsummarization paradigm, this paper proposes an Integer Linear Programming (ILP)\nformulation to obtain extractive oracle summaries in terms of ROUGE-N. We also\npropose an algorithm that enumerates all of the oracle summaries for a set of\nreference summaries to exploit F-measures that evaluate which system summaries\ncontain how many sentences that are extracted as an oracle summary. Our\nexperimental results obtained from Document Understanding Conference (DUC)\ncorpora demonstrated the following: (1) room still exists to improve the\nperformance of extractive summarization; (2) the F-measures derived from the\nenumerated oracle summaries have significantly stronger correlations with human\njudgment than those derived from single oracle summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01623v1",
            "date": "2017-01-06 12:54:48+00:00",
            "title": "Cross-Lingual Dependency Parsing with Late Decoding for Truly Low-Resource Languages",
            "authors": [
                "Michael Sejr Schlichtkrull",
                "Anders S\u00f8gaard"
            ],
            "abstract": "In cross-lingual dependency annotation projection, information is often lost\nduring transfer because of early decoding. We present an end-to-end graph-based\nneural network dependency parser that can be trained to reproduce matrices of\nedge scores, which can be directly projected across word alignments. We show\nthat our approach to cross-lingual dependency parsing is not only simpler, but\nalso achieves an absolute improvement of 2.25% averaged across 10 languages\ncompared to the previous state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01854v1",
            "date": "2017-01-07 16:27:44+00:00",
            "title": "Neural Machine Translation on Scarce-Resource Condition: A case-study on Persian-English",
            "authors": [
                "Mohaddeseh Bastan",
                "Shahram Khadivi",
                "Mohammad Mehdi Homayounpour"
            ],
            "abstract": "Neural Machine Translation (NMT) is a new approach for Machine Translation\n(MT), and due to its success, it has absorbed the attention of many researchers\nin the field. In this paper, we study NMT model on Persian-English language\npairs, to analyze the model and investigate the appropriateness of the model\nfor scarce-resourced scenarios, the situation that exists for Persian-centered\ntranslation systems. We adjust the model for the Persian language and find the\nbest parameters and hyper parameters for two tasks: translation and\ntransliteration. We also apply some preprocessing task on the Persian dataset\nwhich yields to increase for about one point in terms of BLEU score. Also, we\nhave modified the loss function to enhance the word alignment of the model.\nThis new loss function yields a total of 1.87 point improvements in terms of\nBLEU score in the translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.01908v1",
            "date": "2017-01-08 03:13:37+00:00",
            "title": "Sentence-level dialects identification in the greater China region",
            "authors": [
                "Fan Xu",
                "Mingwen Wang",
                "Maoxi Li"
            ],
            "abstract": "Identifying the different varieties of the same language is more challenging\nthan unrelated languages identification. In this paper, we propose an approach\nto discriminate language varieties or dialects of Mandarin Chinese for the\nMainland China, Hong Kong, Taiwan, Macao, Malaysia and Singapore, a.k.a., the\nGreater China Region (GCR). When applied to the dialects identification of the\nGCR, we find that the commonly used character-level or word-level uni-gram\nfeature is not very efficient since there exist several specific problems such\nas the ambiguity and context-dependent characteristic of words in the dialects\nof the GCR. To overcome these challenges, we use not only the general features\nlike character-level n-gram, but also many new word-level features, including\nPMI-based and word alignment-based features. A series of evaluation results on\nboth the news and open-domain dataset from Wikipedia show the effectiveness of\nthe proposed approach.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.\n  5, No.6, December 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02073v2",
            "date": "2017-01-09 06:42:57+00:00",
            "title": "Neural Personalized Response Generation as Domain Adaptation",
            "authors": [
                "Weinan Zhang",
                "Ting Liu",
                "Yifa Wang",
                "Qingfu Zhu"
            ],
            "abstract": "In this paper, we focus on the personalized response generation for\nconversational systems. Based on the sequence to sequence learning, especially\nthe encoder-decoder framework, we propose a two-phase approach, namely\ninitialization then adaptation, to model the responding style of human and then\ngenerate personalized responses. For evaluation, we propose a novel human aided\nmethod to evaluate the performance of the personalized response generation\nmodels by online real-time conversation and offline human judgement. Moreover,\nthe lexical divergence of the responses generated by the 5 personalized models\nindicates that the proposed two-phase approach achieves good results on\nmodeling the responding style of human and generating personalized responses\nfor the conversational systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02149v1",
            "date": "2017-01-09 12:03:11+00:00",
            "title": "Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This work studies comparatively two typical sentence matching tasks: textual\nentailment (TE) and answer selection (AS), observing that weaker phrase\nalignments are more critical in TE, while stronger phrase alignments deserve\nmore attention in AS. The key to reach this observation lies in phrase\ndetection, phrase representation, phrase alignment, and more importantly how to\nconnect those aligned phrases of different matching degrees with the final\nclassifier. Prior work (i) has limitations in phrase generation and\nrepresentation, or (ii) conducts alignment at word and phrase levels by\nhandcrafted features or (iii) utilizes a single framework of alignment without\nconsidering the characteristics of specific tasks, which limits the framework's\neffectiveness across tasks. We propose an architecture based on Gated Recurrent\nUnit that supports (i) representation learning of phrases of arbitrary\ngranularity and (ii) task-specific attentive pooling of phrase alignments\nbetween two sentences. Experimental results on TE and AS match our observation\nand show the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02795v1",
            "date": "2017-01-10 21:45:56+00:00",
            "title": "Bidirectional American Sign Language to English Translation",
            "authors": [
                "Hardie Cate",
                "Zeshan Hussain"
            ],
            "abstract": "We outline a bidirectional translation system that converts sentences from\nAmerican Sign Language (ASL) to English, and vice versa. To perform machine\ntranslation between ASL and English, we utilize a generative approach.\nSpecifically, we employ an adjustment to the IBM word-alignment model 1 (IBM\nWAM1), where we define language models for English and ASL, as well as a\ntranslation model, and attempt to generate a translation that maximizes the\nposterior distribution defined by these models. Then, using these models, we\nare able to quantify the concepts of fluency and faithfulness of a translation\nbetween languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02877v2",
            "date": "2017-01-11 08:02:40+00:00",
            "title": "Generalisation in Named Entity Recognition: A Quantitative Analysis",
            "authors": [
                "Isabelle Augenstein",
                "Leon Derczynski",
                "Kalina Bontcheva"
            ],
            "abstract": "Named Entity Recognition (NER) is a key NLP task, which is all the more\nchallenging on Web and user-generated content with their diverse and\ncontinuously changing language. This paper aims to quantify how this diversity\nimpacts state-of-the-art NER methods, by measuring named entity (NE) and\ncontext variability, feature sparsity, and their effects on precision and\nrecall. In particular, our findings indicate that NER approaches struggle to\ngeneralise in diverse genres with limited training data. Unseen NEs, in\nparticular, play an important role, which have a higher incidence in diverse\ngenres such as social media than in more regular genres such as newswire.\nCoupled with a higher incidence of unseen features more generally and the lack\nof large training corpora, this leads to significantly lower F1 scores for\ndiverse genres as compared to more regular ones. We also find that leading\nsystems rely heavily on surface forms found in training data, having problems\ngeneralising beyond these, and offer explanations for this observation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02901v1",
            "date": "2017-01-11 09:32:47+00:00",
            "title": "A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions",
            "authors": [
                "Antonio Toral",
                "V\u00edctor M. S\u00e1nchez-Cartagena"
            ],
            "abstract": "We aim to shed light on the strengths and weaknesses of the newly introduced\nneural machine translation paradigm. To that end, we conduct a multifaceted\nevaluation in which we compare outputs produced by state-of-the-art neural\nmachine translation and phrase-based machine translation systems for 9 language\ndirections across a number of dimensions. Specifically, we measure the\nsimilarity of the outputs, their fluency and amount of reordering, the effect\nof sentence length and performance across different error categories. We find\nout that translations produced by neural machine translation systems are\nconsiderably different, more fluent and more accurate in terms of word order\ncompared to those produced by phrase-based systems. Neural machine translation\nsystems are also more accurate at producing inflected forms, but they perform\npoorly when translating very long sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02925v1",
            "date": "2017-01-11 11:12:24+00:00",
            "title": "Question Analysis for Arabic Question Answering Systems",
            "authors": [
                "Waheeb Ahmed",
                "Dr. Anto P Babu"
            ],
            "abstract": "The first step of processing a question in Question Answering(QA) Systems is\nto carry out a detailed analysis of the question for the purpose of determining\nwhat it is asking for and how to perfectly approach answering it. Our Question\nanalysis uses several techniques to analyze any question given in natural\nlanguage: a Stanford POS Tagger & parser for Arabic language, a named entity\nrecognizer, tokenizer,Stop-word removal, Question expansion, Question\nclassification and Question focus extraction components. We employ numerous\ndetection rules and trained classifier using features from this analysis to\ndetect important elements of the question, including: 1) the portion of the\nquestion that is a referring to the answer (the focus); 2) different terms in\nthe question that identify what type of entity is being asked for (the lexical\nanswer types); 3) Question expansion ; 4) a process of classifying the question\ninto one or more of several and different types; and We describe how these\nelements are identified and evaluate the effect of accurate detection on our\nquestion-answering system using the Mean Reciprocal Rank(MRR) accuracy measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02946v1",
            "date": "2017-01-11 12:16:25+00:00",
            "title": "Cross-lingual RST Discourse Parsing",
            "authors": [
                "Chlo\u00e9 Braud",
                "Maximin Coavoux",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Discourse parsing is an integral part of understanding information flow and\nargumentative structure in documents. Most previous research has focused on\ninducing and evaluating models from the English RST Discourse Treebank.\nHowever, discourse treebanks for other languages exist, including Spanish,\nGerman, Basque, Dutch and Brazilian Portuguese. The treebanks share the same\nunderlying linguistic theory, but differ slightly in the way documents are\nannotated. In this paper, we present (a) a new discourse parser which is\nsimpler, yet competitive (significantly better on 2/3 metrics) to state of the\nart for English, (b) a harmonization of discourse treebanks across languages,\nenabling us to present (c) what to the best of our knowledge are the first\nexperiments on cross-lingual discourse parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02962v1",
            "date": "2017-01-11 13:11:48+00:00",
            "title": "Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network",
            "authors": [
                "Kim Anh Nguyen",
                "Sabine Schulte im Walde",
                "Ngoc Thang Vu"
            ],
            "abstract": "Distinguishing between antonyms and synonyms is a key task to achieve high\nperformance in NLP systems. While they are notoriously difficult to distinguish\nby distributional co-occurrence models, pattern-based methods have proven\neffective to differentiate between the relations. In this paper, we present a\nnovel neural network model AntSynNET that exploits lexico-syntactic patterns\nfrom syntactic parse trees. In addition to the lexical and syntactic\ninformation, we successfully integrate the distance between the related words\nalong the syntactic path as a new pattern feature. The results from\nclassification experiments show that AntSynNET improves the performance over\nprior pattern-based methods.",
            "journal ref": "EACL2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03092v1",
            "date": "2017-01-11 18:42:09+00:00",
            "title": "Job Detection in Twitter",
            "authors": [
                "Besat Kassaie"
            ],
            "abstract": "In this report, we propose a new application for twitter data called\n\\textit{job detection}. We identify people's job category based on their\ntweets. As a preliminary work, we limited our task to identify only IT workers\nfrom other job holders. We have used and compared both simple bag of words\nmodel and a document representation based on Skip-gram model. Our results show\nthat the model based on Skip-gram, achieves a 76\\% precision and 82\\% recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03129v1",
            "date": "2017-01-11 19:22:56+00:00",
            "title": "De-identification In practice",
            "authors": [
                "Besat Kassaie"
            ],
            "abstract": "We report our effort to identify the sensitive information, subset of data\nitems listed by HIPAA (Health Insurance Portability and Accountability), from\nmedical text using the recent advances in natural language processing and\nmachine learning techniques. We represent the words with high dimensional\ncontinuous vectors learned by a variant of Word2Vec called Continous Bag Of\nWords (CBOW). We feed the word vectors into a simple neural network with a Long\nShort-Term Memory (LSTM) architecture. Without any attempts to extract manually\ncrafted features and considering that our medical dataset is too small to be\nfed into neural network, we obtained promising results. The results thrilled us\nto think about the larger scale of the project with precise parameter tuning\nand other possible improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03163v1",
            "date": "2017-01-11 20:56:29+00:00",
            "title": "Parsing Universal Dependencies without training",
            "authors": [
                "H\u00e9ctor Mart\u00ednez Alonso",
                "\u017deljko Agi\u0107",
                "Barbara Plank",
                "Anders S\u00f8gaard"
            ],
            "abstract": "We propose UDP, the first training-free parser for Universal Dependencies\n(UD). Our algorithm is based on PageRank and a small set of head attachment\nrules. It features two-step decoding to guarantee that function words are\nattached as leaf nodes. The parser requires no training, and it is competitive\nwith a delexicalized transfer system. UDP offers a linguistically sound\nunsupervised alternative to cross-lingual parsing for UD, which can be used as\na baseline for such systems. The parser has very few parameters and is\ndistinctly robust to domain change across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03185v2",
            "date": "2017-01-11 22:55:04+00:00",
            "title": "Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models",
            "authors": [
                "Louis Shao",
                "Stephan Gouws",
                "Denny Britz",
                "Anna Goldie",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "Sequence-to-sequence models have been applied to the conversation response\ngeneration problem where the source sequence is the conversation history and\nthe target sequence is the response. Unlike translation, conversation\nresponding is inherently creative. The generation of long, informative,\ncoherent, and diverse responses remains a hard task. In this work, we focus on\nthe single turn setting. We add self-attention to the decoder to maintain\ncoherence in longer responses, and we propose a practical approach, called the\nglimpse-model, for scaling to large datasets. We introduce a stochastic\nbeam-search algorithm with segment-by-segment reranking which lets us inject\ndiversity earlier in the generation process. We trained on a combined data set\nof over 2.3B conversation messages mined from the web. In human evaluation\nstudies, our method produces longer responses overall, with a higher proportion\nrated as acceptable and excellent as length increases, compared to baseline\nsequence-to-sequence models with explicit length-promotion. A back-off strategy\nproduces better responses overall, in the full spectrum of lengths.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03214v2",
            "date": "2017-01-12 02:37:09+00:00",
            "title": "An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation",
            "authors": [
                "Chenhui Chu",
                "Raj Dabre",
                "Sadao Kurohashi"
            ],
            "abstract": "In this paper, we propose a novel domain adaptation method named \"mixed fine\ntuning\" for neural machine translation (NMT). We combine two existing\napproaches namely fine tuning and multi domain NMT. We first train an NMT model\non an out-of-domain parallel corpus, and then fine tune it on a parallel corpus\nwhich is a mix of the in-domain and out-of-domain corpora. All corpora are\naugmented with artificial tags to indicate specific domains. We empirically\ncompare our proposed method against fine tuning and multi domain methods and\ndiscuss its benefits and shortcomings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03231v1",
            "date": "2017-01-12 04:55:36+00:00",
            "title": "Single-Pass, Adaptive Natural Language Filtering: Measuring Value in User Generated Comments on Large-Scale, Social Media News Forums",
            "authors": [
                "Manuel Amunategui"
            ],
            "abstract": "There are large amounts of insight and social discovery potential in mining\ncrowd-sourced comments left on popular news forums like Reddit.com, Tumblr.com,\nFacebook.com and Hacker News. Unfortunately, due the overwhelming amount of\nparticipation with its varying quality of commentary, extracting value out of\nsuch data isn't always obvious nor timely. By designing efficient, single-pass\nand adaptive natural language filters to quickly prune spam, noise, copy-cats,\nmarketing diversions, and out-of-context posts, we can remove over a third of\nentries and return the comments with a higher probability of relatedness to the\noriginal article in question. The approach presented here uses an adaptive,\ntwo-step filtering process. It first leverages the original article posted in\nthe thread as a starting corpus to parse comments by matching intersecting\nwords and term-ratio balance per sentence then grows the corpus by adding new\nwords harvested from high-matching comments to increase filtering accuracy over\ntime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03329v2",
            "date": "2017-01-12 13:03:49+00:00",
            "title": "A Data-Oriented Model of Literary Language",
            "authors": [
                "Andreas van Cranenburgh",
                "Rens Bod"
            ],
            "abstract": "We consider the task of predicting how literary a text is, with a gold\nstandard from human ratings. Aside from a standard bigram baseline, we apply\nrich syntactic tree fragments, mined from the training set, and a series of\nhand-picked features. Our model is the first to distinguish degrees of highly\nand less literary novels using a variety of lexical and syntactic features, and\nexplains 76.0 % of the variation in literary ratings.",
            "journal ref": "Proceedings of EACL 2017, pp. 1228-1238",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03338v2",
            "date": "2017-01-12 13:41:08+00:00",
            "title": "LanideNN: Multilingual Language Identification on Character Window",
            "authors": [
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "In language identification, a common first step in natural language\nprocessing, we want to automatically determine the language of some input text.\nMonolingual language identification assumes that the given document is written\nin one language. In multilingual language identification, the document is\nusually in two or three languages and we just want their names. We aim one step\nfurther and propose a method for textual language identification where\nlanguages can change arbitrarily and the goal is to identify the spans of each\nof the languages. Our method is based on Bidirectional Recurrent Neural\nNetworks and it performs well in monolingual and multilingual language\nidentification tasks on six datasets covering 131 languages. The method keeps\nthe accuracy also for short documents and across domains, so it is ideal for\noff-the-shelf use without preparation of training data.",
            "journal ref": "Proceedings of the 15th Conference of the European Chapter of the\n  Association for Computational Linguistics: Volume 1, Long Papers, 927-936\n  (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03434v1",
            "date": "2017-01-12 17:51:12+00:00",
            "title": "SMARTies: Sentiment Models for Arabic Target Entities",
            "authors": [
                "Noura Farra",
                "Kathleen McKeown"
            ],
            "abstract": "We consider entity-level sentiment analysis in Arabic, a morphologically rich\nlanguage with increasing resources. We present a system that is applied to\ncomplex posts written in response to Arabic newspaper articles. Our goal is to\nidentify important entity \"targets\" within the post along with the polarity\nexpressed about each target. We achieve significant improvements over multiple\nbaselines, demonstrating that the use of specific morphological representations\nimproves the performance of identifying both important targets and their\nsentiment, and that the use of distributional semantic clusters further boosts\nperformances for these representations, especially when richer linguistic\nresources are not available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03849v3",
            "date": "2017-01-13 23:23:12+00:00",
            "title": "Deep Neural Networks for Czech Multi-label Document Classification",
            "authors": [
                "Ladislav Lenc",
                "Pavel Kr\u00e1l"
            ],
            "abstract": "This paper is focused on automatic multi-label document classification of\nCzech text documents. The current approaches usually use some pre-processing\nwhich can have negative impact (loss of information, additional implementation\nwork, etc). Therefore, we would like to omit it and use deep neural networks\nthat learn from simple features. This choice was motivated by their successful\nusage in many other machine learning fields. Two different networks are\ncompared: the first one is a standard multi-layer perceptron, while the second\none is a popular convolutional network. The experiments on a Czech newspaper\ncorpus show that both networks significantly outperform baseline method which\nuses a rich set of features with maximum entropy classifier. We have also shown\nthat convolutional network gives the best results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03924v1",
            "date": "2017-01-14 14:18:54+00:00",
            "title": "QCRI Machine Translation Systems for IWSLT 16",
            "authors": [
                "Nadir Durrani",
                "Fahim Dalvi",
                "Hassan Sajjad",
                "Stephan Vogel"
            ],
            "abstract": "This paper describes QCRI's machine translation systems for the IWSLT 2016\nevaluation campaign. We participated in the Arabic->English and English->Arabic\ntracks. We built both Phrase-based and Neural machine translation models, in an\neffort to probe whether the newly emerged NMT framework surpasses the\ntraditional phrase-based systems in Arabic-English language pairs. We trained a\nvery strong phrase-based system including, a big language model, the Operation\nSequence Model, Neural Network Joint Model and Class-based models along with\ndifferent domain adaptation techniques such as MML filtering, mixture modeling\nand using fine tuning over NNJM model. However, a Neural MT system, trained by\nstacking data from different genres through fine-tuning, and applying ensemble\nover 8 models, beat our very strong phrase-based system by a significant 2 BLEU\npoints margin in Arabic->English direction. We did not obtain similar gains in\nthe other direction but were still able to outperform the phrase-based system.\nWe also applied system combination on phrase-based and NMT outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.04027v1",
            "date": "2017-01-15 11:08:28+00:00",
            "title": "Neural Models for Sequence Chunking",
            "authors": [
                "Feifei Zhai",
                "Saloni Potdar",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "Many natural language understanding (NLU) tasks, such as shallow parsing\n(i.e., text chunking) and semantic slot filling, require the assignment of\nrepresentative labels to the meaningful chunks in a sentence. Most of the\ncurrent deep neural network (DNN) based methods consider these tasks as a\nsequence labeling problem, in which a word, rather than a chunk, is treated as\nthe basic unit for labeling. These chunks are then inferred by the standard IOB\n(Inside-Outside-Beginning) labels. In this paper, we propose an alternative\napproach by investigating the use of DNN for sequence chunking, and propose\nthree neural models so that each chunk can be treated as a complete unit for\nlabeling. Experimental results show that the proposed neural sequence chunking\nmodels can achieve start-of-the-art performance on both the text chunking and\nslot filling tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.04056v1",
            "date": "2017-01-15 15:10:29+00:00",
            "title": "Dialog Context Language Modeling with Recurrent Neural Networks",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "In this work, we propose contextual language models that incorporate dialog\nlevel discourse information into language modeling. Previous works on\ncontextual language model treat preceding utterances as a sequence of inputs,\nwithout considering dialog interactions. We design recurrent neural network\n(RNN) based contextual language models that specially track the interactions\nbetween speakers in a dialog. Experiment results on Switchboard Dialog Act\nCorpus show that the proposed model outperforms conventional single turn based\nRNN language model by 3.3% on perplexity. The proposed models also demonstrate\nadvantageous performance over other competitive contextual language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.04189v1",
            "date": "2017-01-16 06:49:01+00:00",
            "title": "Deep Memory Networks for Attitude Identification",
            "authors": [
                "Cheng Li",
                "Xiaoxiao Guo",
                "Qiaozhu Mei"
            ],
            "abstract": "We consider the task of identifying attitudes towards a given set of entities\nfrom text. Conventionally, this task is decomposed into two separate subtasks:\ntarget detection that identifies whether each entity is mentioned in the text,\neither explicitly or implicitly, and polarity classification that classifies\nthe exact sentiment towards an identified entity (the target) into positive,\nnegative, or neutral.\n  Instead, we show that attitude identification can be solved with an\nend-to-end machine learning architecture, in which the two subtasks are\ninterleaved by a deep memory network. In this way, signals produced in target\ndetection provide clues for polarity classification, and reversely, the\npredicted polarity provides feedback to the identification of targets.\nMoreover, the treatments for the set of targets also influence each other --\nthe learned representations may share the same semantics for some targets but\nvary for others. The proposed deep memory network, the AttNet, outperforms\nmethods that do not consider the interactions between the subtasks or those\namong the targets, including conventional machine learning methods and the\nstate-of-the-art deep learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.04290v1",
            "date": "2017-01-16 13:55:01+00:00",
            "title": "Machine Translation Approaches and Survey for Indian Languages",
            "authors": [
                "Nadeem Jadoon Khan",
                "Waqas Anwar",
                "Nadir Durrani"
            ],
            "abstract": "In this study, we present an analysis regarding the performance of the\nstate-of-art Phrase-based Statistical Machine Translation (SMT) on multiple\nIndian languages. We report baseline systems on several language pairs. The\nmotivation of this study is to promote the development of SMT and linguistic\nresources for these language pairs, as the current state-of-the-art is quite\nbleak due to sparse data resources. The success of an SMT system is contingent\non the availability of a large parallel corpus. Such data is necessary to\nreliably estimate translation probabilities. We report the performance of\nbaseline systems translating from Indian languages (Bengali, Guajarati, Hindi,\nMalayalam, Punjabi, Tamil, Telugu and Urdu) into English with average 10%\naccurate results for all the language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05011v1",
            "date": "2017-01-18 11:10:59+00:00",
            "title": "Assessing User Expertise in Spoken Dialog System Interactions",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Fernando Batista",
                "Isabel Trancoso",
                "Jos\u00e9 Lopes",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "Identifying the level of expertise of its users is important for a system\nsince it can lead to a better interaction through adaptation techniques.\nFurthermore, this information can be used in offline processes of root cause\nanalysis. However, not much effort has been put into automatically identifying\nthe level of expertise of an user, especially in dialog-based interactions. In\nthis paper we present an approach based on a specific set of task related\nfeatures. Based on the distribution of the features among the two classes -\nNovice and Expert - we used Random Forests as a classification approach.\nFurthermore, we used a Support Vector Machine classifier, in order to perform a\nresult comparison. By applying these approaches on data from a real system,\nLet's Go, we obtained preliminary results that we consider positive, given the\ndifficulty of the task and the lack of competing approaches for comparison.",
            "journal ref": "Advances in Speech and Language Technologies for Iberian\n  Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal,\n  November 23-25, pp. 245-254",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05343v1",
            "date": "2017-01-19 09:32:08+00:00",
            "title": "A Joint Framework for Argumentative Text Analysis Incorporating Domain Knowledge",
            "authors": [
                "Zhongyu Wei",
                "Chen Li",
                "Yang Liu"
            ],
            "abstract": "For argumentation mining, there are several sub-tasks such as argumentation\ncomponent type classification, relation classification. Existing research tends\nto solve such sub-tasks separately, but ignore the close relation between them.\nIn this paper, we present a joint framework incorporating logical relation\nbetween sub-tasks to improve the performance of argumentation structure\ngeneration. We design an objective function to combine the predictions from\nindividual models for each sub-task and solve the problem with some constraints\nconstructed from background knowledge. We evaluate our proposed model on two\npublic corpora and the experiment results show that our model can outperform\nthe baseline that uses a separate model significantly for each sub-task. Our\nmodel also shows advantages on component-related sub-tasks compared to a\nstate-of-the-art joint model based on the evidence graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05574v1",
            "date": "2017-01-19 19:32:06+00:00",
            "title": "Harnessing Cognitive Features for Sarcasm Detection",
            "authors": [
                "Abhijit Mishra",
                "Diptesh Kanojia",
                "Seema Nagar",
                "Kuntal Dey",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we propose a novel mechanism for enriching the feature vector,\nfor the task of sarcasm detection, with cognitive features extracted from\neye-movement patterns of human readers. Sarcasm detection has been a\nchallenging research problem, and its importance for NLP applications such as\nreview summarization, dialog systems and sentiment analysis is well recognized.\nSarcasm can often be traced to incongruity that becomes apparent as the full\nsentence unfolds. This presence of incongruity- implicit or explicit- affects\nthe way readers eyes move through the text. We observe the difference in the\nbehaviour of the eye, while reading sarcastic and non sarcastic sentences.\nMotivated by his observation, we augment traditional linguistic and stylistic\nfeatures for sarcasm detection with the cognitive features obtained from\nreaders eye movement data. We perform statistical classification using the\nenhanced feature set so obtained. The augmented cognitive features improve\nsarcasm detection by 3.7% (in terms of F-score), over the performance of the\nbest reported system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05581v1",
            "date": "2017-01-19 19:58:26+00:00",
            "title": "Leveraging Cognitive Features for Sentiment Analysis",
            "authors": [
                "Abhijit Mishra",
                "Diptesh Kanojia",
                "Seema Nagar",
                "Kuntal Dey",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Sentiments expressed in user-generated short text and sentences are nuanced\nby subtleties at lexical, syntactic, semantic and pragmatic levels. To address\nthis, we propose to augment traditional features used for sentiment analysis\nand sarcasm detection, with cognitive features derived from the eye-movement\npatterns of readers. Statistical classification using our enhanced feature set\nimproves the performance (F-score) of polarity detection by a maximum of 3.7%\nand 9.3% on two datasets, over the systems that use only traditional features.\nWe perform feature significance analysis, and experiment on a held-out dataset,\nshowing that cognitive features indeed empower sentiment analyzers to handle\ncomplex constructs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05625v2",
            "date": "2017-01-19 22:14:25+00:00",
            "title": "CEVO: Comprehensive EVent Ontology Enhancing Cognitive Annotation",
            "authors": [
                "Saeedeh Shekarpour",
                "Faisal Alshargi",
                "Valerie Shalin",
                "Krishnaprasad Thirunarayan",
                "Amit P. Sheth"
            ],
            "abstract": "While the general analysis of named entities has received substantial\nresearch attention on unstructured as well as structured data, the analysis of\nrelations among named entities has received limited focus. In fact, a review of\nthe literature revealed a deficiency in research on the abstract\nconceptualization required to organize relations. We believe that such an\nabstract conceptualization can benefit various communities and applications\nsuch as natural language processing, information extraction, machine learning,\nand ontology engineering. In this paper, we present Comprehensive EVent\nOntology (CEVO), built on Levin's conceptual hierarchy of English verbs that\ncategorizes verbs with shared meaning, and syntactic behavior. We present the\nfundamental concepts and requirements for this ontology. Furthermore, we\npresent three use cases employing the CEVO ontology on annotation tasks: (i)\nannotating relations in plain text, (ii) annotating ontological properties, and\n(iii) linking textual relations to ontological properties. These use-cases\ndemonstrate the benefits of using CEVO for annotation: (i) annotating English\nverbs from an abstract conceptualization, (ii) playing the role of an upper\nontology for organizing ontological properties, and (iii) facilitating the\nannotation of text relations using any underlying vocabulary. This resource is\navailable at https://shekarpour.github.io/cevo.io/ using https://w3id.org/cevo\nnamespace.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.06521v1",
            "date": "2017-01-23 17:43:23+00:00",
            "title": "Incorporating Global Visual Features into Attention-Based Neural Machine Translation",
            "authors": [
                "Iacer Calixto",
                "Qun Liu",
                "Nick Campbell"
            ],
            "abstract": "We introduce multi-modal, attention-based neural machine translation (NMT)\nmodels which incorporate visual features into different parts of both the\nencoder and the decoder. We utilise global image features extracted using a\npre-trained convolutional neural network and incorporate them (i) as words in\nthe source sentence, (ii) to initialise the encoder hidden state, and (iii) as\nadditional data to initialise the decoder hidden state. In our experiments, we\nevaluate how these different strategies to incorporate global image features\ncompare and which ones perform best. We also study the impact that adding\nsynthetic multi-modal, multilingual data brings and find that the additional\ndata have a positive impact on multi-modal models. We report new\nstate-of-the-art results and our best models also significantly improve on a\ncomparable phrase-based Statistical MT (PBSMT) model trained on the Multi30k\ndata set according to all metrics evaluated. To the best of our knowledge, it\nis the first time a purely neural model significantly improves over a PBSMT\nmodel on all metrics evaluated on this data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.06547v5",
            "date": "2017-01-23 18:32:27+00:00",
            "title": "Adversarial Learning for Neural Dialogue Generation",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Tianlin Shi",
                "S\u00e9bastien Jean",
                "Alan Ritter",
                "Dan Jurafsky"
            ],
            "abstract": "In this paper, drawing intuition from the Turing test, we propose using\nadversarial training for open-domain dialogue generation: the system is trained\nto produce sequences that are indistinguishable from human-generated dialogue\nutterances. We cast the task as a reinforcement learning (RL) problem where we\njointly train two systems, a generative model to produce response sequences,\nand a discriminator---analagous to the human evaluator in the Turing test--- to\ndistinguish between the human-generated dialogues and the machine-generated\nones. The outputs from the discriminator are then used as rewards for the\ngenerative model, pushing the system to generate dialogues that mostly resemble\nhuman dialogues.\n  In addition to adversarial training we describe a model for adversarial {\\em\nevaluation} that uses success in fooling an adversary as a dialogue evaluation\nmetric, while avoiding a number of potential pitfalls. Experimental results on\nseveral metrics, including adversarial evaluation, demonstrate that the\nadversarially-trained system generates higher-quality responses than previous\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.06549v2",
            "date": "2017-01-23 18:36:37+00:00",
            "title": "Learning to Decode for Future Success",
            "authors": [
                "Jiwei Li",
                "Will Monroe",
                "Dan Jurafsky"
            ],
            "abstract": "We introduce a simple, general strategy to manipulate the behavior of a\nneural decoder that enables it to generate outputs that have specific\nproperties of interest (e.g., sequences of a pre-specified length). The model\ncan be thought of as a simple version of the actor-critic model that uses an\ninterpolation of the actor (the MLE-based token generation policy) and the\ncritic (a value function that estimates the future values of the desired\nproperty) for decision making. We demonstrate that the approach is able to\nincorporate a variety of properties that cannot be handled by standard neural\nsequence decoders, such as sequence length and backward probability\n(probability of sources given targets), in addition to yielding consistent\nimprovements in abstractive summarization and machine translation when the\nproperty to be optimized is BLEU or ROUGE scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.07149v1",
            "date": "2017-01-25 03:04:31+00:00",
            "title": "Hierarchical Recurrent Attention Network for Response Generation",
            "authors": [
                "Chen Xing",
                "Wei Wu",
                "Yu Wu",
                "Ming Zhou",
                "Yalou Huang",
                "Wei-Ying Ma"
            ],
            "abstract": "We study multi-turn response generation in chatbots where a response is\ngenerated according to a conversation context. Existing work has modeled the\nhierarchy of the context, but does not pay enough attention to the fact that\nwords and utterances in the context are differentially important. As a result,\nthey may lose important information in context and generate irrelevant\nresponses. We propose a hierarchical recurrent attention network (HRAN) to\nmodel both aspects in a unified framework. In HRAN, a hierarchical attention\nmechanism attends to important parts within and among utterances with word\nlevel attention and utterance level attention respectively. With the word level\nattention, hidden vectors of a word level encoder are synthesized as utterance\nvectors and fed to an utterance level encoder to construct hidden\nrepresentations of the context. The hidden vectors of the context are then\nprocessed by the utterance level attention and formed as context vectors for\ndecoding the response. Empirical studies on both automatic evaluation and human\njudgment show that HRAN can significantly outperform state-of-the-art models\nfor multi-turn response generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.07880v1",
            "date": "2017-01-26 21:18:32+00:00",
            "title": "emLam -- a Hungarian Language Modeling baseline",
            "authors": [
                "D\u00e1vid M\u00e1rk Nemeskey"
            ],
            "abstract": "This paper aims to make up for the lack of documented baselines for Hungarian\nlanguage modeling. Various approaches are evaluated on three publicly available\nHungarian corpora. Perplexity values comparable to models of similar-sized\nEnglish corpora are reported. A new, freely downloadable Hungar- ian benchmark\ncorpus is introduced.",
            "journal ref": "In Proceedings of the 13th Conference on Hungarian Computational\n  Linguistics (MSZNY), pp. 91-102. Szeged, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08071v2",
            "date": "2017-01-27 14:50:36+00:00",
            "title": "Emotion Recognition From Speech With Recurrent Neural Networks",
            "authors": [
                "Vladimir Chernykh",
                "Pavel Prikhodko"
            ],
            "abstract": "In this paper the task of emotion recognition from speech is considered.\nProposed approach uses deep recurrent neural network trained on a sequence of\nacoustic features calculated over small speech intervals. At the same time\nspecial probabilistic-nature CTC loss function allows to consider long\nutterances containing both emotional and neutral parts. The effectiveness of\nsuch an approach is shown in two ways. Firstly, the comparison with recent\nadvances in this field is carried out. Secondly, human performance on the same\ntask is measured. Both criteria show the high quality of the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08118v1",
            "date": "2017-01-27 17:09:07+00:00",
            "title": "Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis",
            "authors": [
                "Bj\u00f6rn Ross",
                "Michael Rist",
                "Guillermo Carbonell",
                "Benjamin Cabrera",
                "Nils Kurowsky",
                "Michael Wojatzki"
            ],
            "abstract": "Some users of social media are spreading racist, sexist, and otherwise\nhateful content. For the purpose of training a hate speech detection system,\nthe reliability of the annotations is crucial, but there is no universally\nagreed-upon definition. We collected potentially hateful messages and asked two\ngroups of internet users to determine whether they were hate speech or not,\nwhether they should be banned or not and to rate their degree of offensiveness.\nOne of the groups was shown a definition prior to completing the survey. We\naimed to assess whether hate speech can be annotated reliably, and the extent\nto which existing definitions are in accordance with subjective ratings. Our\nresults indicate that showing users a definition caused them to partially align\ntheir own opinion with the definition but did not improve reliability, which\nwas very low overall. We conclude that the presence of hate speech should\nperhaps not be considered a binary yes-or-no decision, and raters need more\ndetailed instructions for the annotation.",
            "journal ref": "Proceedings of NLP4CMC III: 3rd Workshop on Natural Language\n  Processing for Computer-Mediated Communication (Bochum), Bochumer\n  Linguistische Arbeitsberichte, vol. 17, sep 2016, pp. 6-9",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08198v1",
            "date": "2017-01-27 21:28:57+00:00",
            "title": "Adversarial Evaluation of Dialogue Models",
            "authors": [
                "Anjuli Kannan",
                "Oriol Vinyals"
            ],
            "abstract": "The recent application of RNN encoder-decoder models has resulted in\nsubstantial progress in fully data-driven dialogue systems, but evaluation\nremains a challenge. An adversarial loss could be a way to directly evaluate\nthe extent to which generated dialogue responses sound like they came from a\nhuman. This could reduce the need for human evaluation, while more directly\nevaluating on a generative task. In this work, we investigate this idea by\ntraining an RNN to discriminate a dialogue model's samples from human-generated\nsamples. Although we find some evidence this setup could be viable, we also\nnote that many issues remain in its practical application. We discuss both\naspects and conclude that future work is warranted.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08303v2",
            "date": "2017-01-28 17:04:21+00:00",
            "title": "Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network",
            "authors": [
                "Sunil Kumar Sahu",
                "Ashish Anand"
            ],
            "abstract": "Simultaneous administration of multiple drugs can have synergistic or\nantagonistic effects as one drug can affect activities of other drugs.\nSynergistic effects lead to improved therapeutic outcomes, whereas,\nantagonistic effects can be life-threatening, may lead to increased healthcare\ncost, or may even cause death. Thus identification of unknown drug-drug\ninteraction (DDI) is an important concern for efficient and effective\nhealthcare. Although multiple resources for DDI exist, they are often unable to\nkeep pace with rich amount of information available in fast growing biomedical\ntexts. Most existing methods model DDI extraction from text as a classification\nproblem and mainly rely on handcrafted features. Some of these features further\ndepend on domain specific tools. Recently neural network models using latent\nfeatures have been shown to give similar or better performance than the other\nexisting models dependent on handcrafted features. In this paper, we present\nthree models namely, {\\it B-LSTM}, {\\it AB-LSTM} and {\\it Joint AB-LSTM} based\non long short-term memory (LSTM) network. All three models utilize word and\nposition embedding as latent features and thus do not rely on explicit feature\nengineering. Further use of bidirectional long short-term memory (Bi-LSTM)\nnetworks allow implicit feature extraction from the whole sentence. The two\nmodels, {\\it AB-LSTM} and {\\it Joint AB-LSTM} also use attentive pooling in the\noutput of Bi-LSTM layer to assign weights to features. Our experimental results\non the SemEval-2013 DDI extraction dataset show that the {\\it Joint AB-LSTM}\nmodel outperforms all the existing methods, including those relying on\nhandcrafted features. The other two proposed LSTM models also perform\ncompetitively with state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08339v1",
            "date": "2017-01-29 00:22:24+00:00",
            "title": "Using English as Pivot to Extract Persian-Italian Parallel Sentences from Non-Parallel Corpora",
            "authors": [
                "Ebrahim Ansari",
                "M. H. Sadreddini",
                "Mostafa Sheikhalishahi",
                "Richard Wallace",
                "Fatemeh Alimardani"
            ],
            "abstract": "The effectiveness of a statistical machine translation system (SMT) is very\ndependent upon the amount of parallel corpus used in the training phase. For\nlow-resource language pairs there are not enough parallel corpora to build an\naccurate SMT. In this paper, a novel approach is presented to extract bilingual\nPersian-Italian parallel sentences from a non-parallel (comparable) corpus. In\nthis study, English is used as the pivot language to compute the matching\nscores between source and target sentences and candidate selection phase.\nAdditionally, a new monolingual sentence similarity metric, Normalized Google\nDistance (NGD) is proposed to improve the matching process. Moreover, some\nextensions of the baseline system are applied to improve the quality of\nextracted sentences measured with BLEU. Experimental results show that using\nthe new pivot based extraction can increase the quality of bilingual corpus\nsignificantly and consequently improves the performance of the Persian-Italian\nSMT system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08340v2",
            "date": "2017-01-29 00:28:20+00:00",
            "title": "Extracting Bilingual Persian Italian Lexicon from Comparable Corpora Using Different Types of Seed Dictionaries",
            "authors": [
                "Ebrahim Ansari",
                "M. H. Sadreddini",
                "Lucio Grandinetti",
                "Mahsa Radinmehr",
                "Ziba Khosravan",
                "Mehdi Sheikhalishahi"
            ],
            "abstract": "Bilingual dictionaries are very important in various fields of natural\nlanguage processing. In recent years, research on extracting new bilingual\nlexicons from non-parallel (comparable) corpora have been proposed. Almost all\nuse a small existing dictionary or other resources to make an initial list\ncalled the \"seed dictionary\". In this paper, we discuss the use of different\ntypes of dictionaries as the initial starting list for creating a bilingual\nPersian-Italian lexicon from a comparable corpus. Our experiments apply\nstate-of-the-art techniques on three different seed dictionaries; an existing\ndictionary, a dictionary created with pivot-based schema, and a dictionary\nextracted from a small Persian-Italian parallel text. The interesting challenge\nof our approach is to find a way to combine different dictionaries together in\norder to produce a better and more accurate lexicon. In order to combine seed\ndictionaries, we propose two different combination models and examine the\neffect of our novel combination models on various comparable corpora that have\ndiffering degrees of comparability. We conclude with a proposal for a new\nweighting system to improve the extracted lexicon. The experimental results\nproduced by our implementation show the efficiency of our proposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08533v1",
            "date": "2017-01-30 10:28:49+00:00",
            "title": "Graph-Based Semi-Supervised Conditional Random Fields For Spoken Language Understanding Using Unaligned Data",
            "authors": [
                "Mohammad Aliannejadi",
                "Masoud Kiaeeha",
                "Shahram Khadivi",
                "Saeed Shiry Ghidary"
            ],
            "abstract": "We experiment graph-based Semi-Supervised Learning (SSL) of Conditional\nRandom Fields (CRF) for the application of Spoken Language Understanding (SLU)\non unaligned data. The aligned labels for examples are obtained using IBM\nModel. We adapt a baseline semi-supervised CRF by defining new feature set and\naltering the label propagation algorithm. Our results demonstrate that our\nproposed approach significantly improves the performance of the supervised\nmodel by utilizing the knowledge gained from the graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08655v2",
            "date": "2017-01-30 15:37:25+00:00",
            "title": "Structural Analysis of Hindi Phonetics and A Method for Extraction of Phonetically Rich Sentences from a Very Large Hindi Text Corpus",
            "authors": [
                "Shrikant Malviya",
                "Rohit Mishra",
                "Uma Shanker Tiwary"
            ],
            "abstract": "Automatic speech recognition (ASR) and Text to speech (TTS) are two prominent\narea of research in human computer interaction nowadays. A set of phonetically\nrich sentences is in a matter of importance in order to develop these two\ninteractive modules of HCI. Essentially, the set of phonetically rich sentences\nhas to cover all possible phone units distributed uniformly. Selecting such a\nset from a big corpus with maintaining phonetic characteristic based similarity\nis still a challenging problem. The major objective of this paper is to devise\na criteria in order to select a set of sentences encompassing all phonetic\naspects of a corpus with size as minimum as possible. First, this paper\npresents a statistical analysis of Hindi phonetics by observing the structural\ncharacteristics. Further a two stage algorithm is proposed to extract\nphonetically rich sentences with a high variety of triphones from the EMILLE\nHindi corpus. The algorithm consists of a distance measuring criteria to select\na sentence in order to improve the triphone distribution. Moreover, a special\npreprocessing method is proposed to score each triphone in terms of inverse\nprobability in order to fasten the algorithm. The results show that the\napproach efficiently build uniformly distributed phonetically-rich corpus with\noptimum number of sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08702v1",
            "date": "2017-01-27 18:43:31+00:00",
            "title": "Bangla Word Clustering Based on Tri-gram, 4-gram and 5-gram Language Model",
            "authors": [
                "Dipaloke Saha",
                "Md Saddam Hossain",
                "MD. Saiful Islam",
                "Sabir Ismail"
            ],
            "abstract": "In this paper, we describe a research method that generates Bangla word\nclusters on the basis of relating to meaning in language and contextual\nsimilarity. The importance of word clustering is in parts of speech (POS)\ntagging, word sense disambiguation, text classification, recommender system,\nspell checker, grammar checker, knowledge discover and for many others Natural\nLanguage Processing (NLP) applications. In the history of word clustering,\nEnglish and some other languages have already implemented some methods on word\nclustering efficiently. But due to lack of the resources, word clustering in\nBangla has not been still implemented efficiently. Presently, its\nimplementation is in the beginning stage. In some research of word clustering\nin English based on preceding and next five words of a key word they found an\nefficient result. Now, we are trying to implement the tri-gram, 4-gram and\n5-gram model of word clustering for Bangla to observe which one is the best\namong them. We have started our research with quite a large corpus of\napproximate 1 lakh Bangla words. We are using a machine learning technique in\nthis research. We will generate word clusters and analyze the clusters by\ntesting some different threshold values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00079v2",
            "date": "2017-07-01 01:21:22+00:00",
            "title": "Synthetic Data for Neural Machine Translation of Spoken-Dialects",
            "authors": [
                "Hany Hassan",
                "Mostafa Elaraby",
                "Ahmed Tawfik"
            ],
            "abstract": "In this paper, we introduce a novel approach to generate synthetic data for\ntraining Neural Machine Translation systems. The proposed approach transforms a\ngiven parallel corpus between a written language and a target language to a\nparallel corpus between a spoken dialect variant and the target language. Our\napproach is language independent and can be used to generate data for any\nvariant of the source language such as slang or spoken dialect or even for a\ndifferent language that is closely related to the source language.\n  The proposed approach is based on local embedding projection of distributed\nrepresentations which utilizes monolingual embeddings to transform parallel\ndata across language variants. We report experimental results on Levantine to\nEnglish translation using Neural Machine Translation. We show that the\ngenerated data can improve a very large scale system by more than 2.8 Bleu\npoints using synthetic spoken data which shows that it can be used to provide a\nreliable translation system for a spoken dialect that does not have sufficient\nparallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00110v1",
            "date": "2017-07-01 08:16:24+00:00",
            "title": "Efficient Attention using a Fixed-Size Memory Representation",
            "authors": [
                "Denny Britz",
                "Melody Y. Guan",
                "Minh-Thang Luong"
            ],
            "abstract": "The standard content-based attention mechanism typically used in\nsequence-to-sequence models is computationally expensive as it requires the\ncomparison of large encoder and decoder states at each time step. In this work,\nwe propose an alternative attention mechanism based on a fixed size memory\nrepresentation that is more efficient. Our technique predicts a compact set of\nK attention contexts during encoding and lets the decoder compute an efficient\nlookup that does not need to consult the memory. We show that our approach\nperforms on-par with the standard attention mechanism while yielding inference\nspeedups of 20% for real-world translation tasks and more for tasks with longer\nsequences. By visualizing attention scores we demonstrate that our models learn\ndistinct, meaningful alignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00166v2",
            "date": "2017-07-01 15:23:23+00:00",
            "title": "Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach",
            "authors": [
                "Liyuan Liu",
                "Xiang Ren",
                "Qi Zhu",
                "Shi Zhi",
                "Huan Gui",
                "Heng Ji",
                "Jiawei Han"
            ],
            "abstract": "Relation extraction is a fundamental task in information extraction. Most\nexisting methods have heavy reliance on annotations labeled by human experts,\nwhich are costly and time-consuming. To overcome this drawback, we propose a\nnovel framework, REHession, to conduct relation extractor learning using\nannotations from heterogeneous information source, e.g., knowledge base and\ndomain heuristics. These annotations, referred as heterogeneous supervision,\noften conflict with each other, which brings a new challenge to the original\nrelation extraction task: how to infer the true label from noisy labels for a\ngiven instance. Identifying context information as the backbone of both\nrelation extraction and true label discovery, we adopt embedding techniques to\nlearn the distributed representations of context, which bridges all components\nwith mutual enhancement in an iterative fashion. Extensive experimental results\ndemonstrate the superiority of REHession over the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00248v1",
            "date": "2017-07-02 07:41:10+00:00",
            "title": "DAG-based Long Short-Term Memory for Neural Word Segmentation",
            "authors": [
                "Xinchi Chen",
                "Zhan Shi",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural word segmentation has attracted more and more research interests for\nits ability to alleviate the effort of feature engineering and utilize the\nexternal resource by the pre-trained character or word embeddings. In this\npaper, we propose a new neural model to incorporate the word-level information\nfor Chinese word segmentation. Unlike the previous word-based models, our model\nstill adopts the framework of character-based sequence labeling, which has\nadvantages on both effectiveness and efficiency at the inference stage. To\nutilize the word-level information, we also propose a new long short-term\nmemory (LSTM) architecture over directed acyclic graph (DAG). Experimental\nresults demonstrate that our model leads to better performances than the\nbaseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00299v1",
            "date": "2017-07-02 14:39:21+00:00",
            "title": "Grammatical Error Correction with Neural Reinforcement Learning",
            "authors": [
                "Keisuke Sakaguchi",
                "Matt Post",
                "Benjamin Van Durme"
            ],
            "abstract": "We propose a neural encoder-decoder model with reinforcement learning (NRL)\nfor grammatical error correction (GEC). Unlike conventional maximum likelihood\nestimation (MLE), the model directly optimizes towards an objective that\nconsiders a sentence-level, task-specific evaluation metric, avoiding the\nexposure bias issue in MLE. We demonstrate that NRL outperforms MLE both in\nhuman and automated evaluation metrics, achieving the state-of-the-art on a\nfluency-oriented GEC corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00621v1",
            "date": "2017-07-03 16:06:16+00:00",
            "title": "Including Dialects and Language Varieties in Author Profiling",
            "authors": [
                "Alina Maria Ciobanu",
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Liviu P. Dinu"
            ],
            "abstract": "This paper presents a computational approach to author profiling taking\ngender and language variety into account. We apply an ensemble system with the\noutput of multiple linear SVM classifiers trained on character and word\n$n$-grams. We evaluate the system using the dataset provided by the organizers\nof the 2017 PAN lab on author profiling. Our approach achieved 75% average\naccuracy on gender identification on tweets written in four languages and 97%\naccuracy on language variety identification for Portuguese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00722v2",
            "date": "2017-07-03 18:25:51+00:00",
            "title": "Improving LSTM-CTC based ASR performance in domains with limited training data",
            "authors": [
                "Jayadev Billa"
            ],
            "abstract": "This paper addresses the observed performance gap between automatic speech\nrecognition (ASR) systems based on Long Short Term Memory (LSTM) neural\nnetworks trained with the connectionist temporal classification (CTC) loss\nfunction and systems based on hybrid Deep Neural Networks (DNNs) trained with\nthe cross entropy (CE) loss function on domains with limited data. We step\nthrough a number of experiments that show incremental improvements on a\nbaseline EESEN toolkit based LSTM-CTC ASR system trained on the Librispeech\n100hr (train-clean-100) corpus. Our results show that with effective\ncombination of data augmentation and regularization, a LSTM-CTC based system\ncan exceed the performance of a strong Kaldi based baseline trained on the same\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00896v4",
            "date": "2017-07-04 10:28:04+00:00",
            "title": "Multilingual Hierarchical Attention Networks for Document Classification",
            "authors": [
                "Nikolaos Pappas",
                "Andrei Popescu-Belis"
            ],
            "abstract": "Hierarchical attention networks have recently achieved remarkable performance\nfor document classification in a given language. However, when multilingual\ndocument collections are considered, training such models separately for each\nlanguage entails linear parameter growth and lack of cross-language transfer.\nLearning a single multilingual model with fewer parameters is therefore a\nchallenging but potentially beneficial objective. To this end, we propose\nmultilingual hierarchical attention networks for learning document structures,\nwith shared encoders and/or shared attention mechanisms across languages, using\nmulti-task learning and an aligned semantic space as input. We evaluate the\nproposed models on multilingual document classification with disjoint label\nsets, on a large dataset which we provide, with 600k news documents in 8\nlanguages, and 5k labels. The multilingual models outperform monolingual ones\nin low-resource as well as full-resource settings, and use fewer parameters,\nthus confirming their computational efficiency and the utility of\ncross-language transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00995v1",
            "date": "2017-07-04 13:57:04+00:00",
            "title": "An empirical study on the effectiveness of images in Multimodal Neural Machine Translation",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont"
            ],
            "abstract": "In state-of-the-art Neural Machine Translation (NMT), an attention mechanism\nis used during decoding to enhance the translation. At every step, the decoder\nuses this mechanism to focus on different parts of the source sentence to\ngather the most useful information before outputting its target word. Recently,\nthe effectiveness of the attention mechanism has also been explored for\nmultimodal tasks, where it becomes possible to focus both on sentence parts and\nimage regions that they describe. In this paper, we compare several attention\nmechanism on the multimodal translation task (English, image to German) and\nevaluate the ability of the model to make use of images to improve translation.\nWe surpass state-of-the-art scores on the Multi30k data set, we nevertheless\nidentify and report different misbehavior of the machine while translating.",
            "journal ref": "Proceedings of the 2017 Conference on Empirical Methods in Natural\n  Language Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01009v5",
            "date": "2017-07-04 14:26:56+00:00",
            "title": "Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont",
                "Omar Seddati"
            ],
            "abstract": "In Multimodal Neural Machine Translation (MNMT), a neural model generates a\ntranslated sentence that describes an image, given the image itself and one\nsource descriptions in English. This is considered as the multimodal image\ncaption translation task. The images are processed with Convolutional Neural\nNetwork (CNN) to extract visual features exploitable by the translation model.\nSo far, the CNNs used are pre-trained on object detection and localization\ntask. We hypothesize that richer architecture, such as dense captioning models,\nmay be more suitable for MNMT and could lead to improved translations. We\nextend this intuition to the word-embeddings, where we compute both linguistic\nand visual representation for our corpus vocabulary. We combine and compare\ndifferent confi",
            "journal ref": "Proc. GLU 2017 International Workshop on Grounding Language\n  Understanding",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01066v1",
            "date": "2017-07-04 16:48:28+00:00",
            "title": "Zero-Shot Transfer Learning for Event Extraction",
            "authors": [
                "Lifu Huang",
                "Heng Ji",
                "Kyunghyun Cho",
                "Clare R. Voss"
            ],
            "abstract": "Most previous event extraction studies have relied heavily on features\nderived from annotated event mentions, thus cannot be applied to new event\ntypes without annotation effort. In this work, we take a fresh look at event\nextraction and model it as a grounding problem. We design a transferable neural\narchitecture, mapping event mentions and types jointly into a shared semantic\nspace using structural and compositional neural networks, where the type of\neach event mention can be determined by the closest of all candidate types . By\nleveraging (1)~available manual annotations for a small set of existing event\ntypes and (2)~existing event ontologies, our framework applies to new event\ntypes without requiring additional annotation. Experiments on both existing\nevent types (e.g., ACE, ERE) and new event types (e.g., FrameNet) demonstrate\nthe effectiveness of our approach. \\textit{Without any manual annotations} for\n23 new event types, our zero-shot framework achieved performance comparable to\na state-of-the-art supervised model which is trained from the annotations of\n500 event mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01075v1",
            "date": "2017-07-04 17:18:50+00:00",
            "title": "Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structures",
            "authors": [
                "Lifu Huang",
                "Avirup Sil",
                "Heng Ji",
                "Radu Florian"
            ],
            "abstract": "Slot Filling (SF) aims to extract the values of certain types of attributes\n(or slots, such as person:cities\\_of\\_residence) for a given entity from a\nlarge collection of source documents. In this paper we propose an effective DNN\narchitecture for SF with the following new strategies: (1). Take a regularized\ndependency graph instead of a raw sentence as input to DNN, to compress the\nwide contexts between query and candidate filler; (2). Incorporate two\nattention mechanisms: local attention learned from query and candidate filler,\nand global attention learned from external knowledge bases, to guide the model\nto better select indicative contexts to determine slot type. Experiments show\nthat this framework outperforms state-of-the-art on both relation extraction\n(16\\% absolute F-score gain) and slot filling validation for each individual\nsystem (up to 8.5\\% absolute F-score gain).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01161v2",
            "date": "2017-07-04 21:42:55+00:00",
            "title": "Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models",
            "authors": [
                "Harsh Jhamtani",
                "Varun Gangal",
                "Eduard Hovy",
                "Eric Nyberg"
            ],
            "abstract": "Variations in writing styles are commonly used to adapt the content to a\nspecific context, audience, or purpose. However, applying stylistic variations\nis still by and large a manual process, and there have been little efforts\ntowards automating it. In this paper we explore automated methods to transform\ntext from modern English to Shakespearean English using an end to end trainable\nneural model with pointers to enable copy action. To tackle limited amount of\nparallel data, we pre-train embeddings of words by leveraging external\ndictionaries mapping Shakespearean words to modern English words as well as\nadditional text. Our methods are able to get a BLEU score of 31+, an\nimprovement of ~6 points above the strongest baseline. We publicly release our\ncode to foster further research in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01176v2",
            "date": "2017-07-04 23:11:52+00:00",
            "title": "CharManteau: Character Embedding Models For Portmanteau Creation",
            "authors": [
                "Varun Gangal",
                "Harsh Jhamtani",
                "Graham Neubig",
                "Eduard Hovy",
                "Eric Nyberg"
            ],
            "abstract": "Portmanteaus are a word formation phenomenon where two words are combined to\nform a new word. We propose character-level neural sequence-to-sequence (S2S)\nmethods for the task of portmanteau generation that are end-to-end-trainable,\nlanguage independent, and do not explicitly use additional phonetic\ninformation. We propose a noisy-channel-style model, which allows for the\nincorporation of unsupervised word lists, improving performance over a standard\nsource-to-target model. This model is made possible by an exhaustive candidate\ngeneration strategy specifically enabled by the features of the portmanteau\ntask. Experiments find our approach superior to a state-of-the-art FST-based\nbaseline with respect to ground truth accuracy and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01265v2",
            "date": "2017-07-05 08:55:28+00:00",
            "title": "Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification",
            "authors": [
                "Jonggu Kim",
                "Jong-Hyeok Lee"
            ],
            "abstract": "Most of neural approaches to relation classification have focused on finding\nshort patterns that represent the semantic relation using Convolutional Neural\nNetworks (CNNs) and those approaches have generally achieved better\nperformances than using Recurrent Neural Networks (RNNs). In a similar\nintuition to the CNN models, we propose a novel RNN-based model that strongly\nfocuses on only important parts of a sentence using multiple range-restricted\nbidirectional layers and attention for relation classification. Experimental\nresults on the SemEval-2010 relation classification task show that our model is\ncomparable to the state-of-the-art CNN-based and RNN-based models that use\nadditional linguistic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01321v1",
            "date": "2017-07-05 11:09:31+00:00",
            "title": "The Influence of Feature Representation of Text on the Performance of Document Classification",
            "authors": [
                "Sanda Martin\u010di\u0107-Ip\u0161i\u0107",
                "Tanja Mili\u010di\u0107",
                "Ljup\u010do Todorovski"
            ],
            "abstract": "In this paper we perform a comparative analysis of three models for feature\nrepresentation of text documents in the context of document classification. In\nparticular, we consider the most often used family of models bag-of-words,\nrecently proposed continuous space models word2vec and doc2vec, and the model\nbased on the representation of text documents as language networks. While the\nbag-of-word models have been extensively used for the document classification\ntask, the performance of the other two models for the same task have not been\nwell understood. This is especially true for the network-based model that have\nbeen rarely considered for representation of text documents for classification.\nIn this study, we measure the performance of the document classifiers trained\nusing the method of random forests for features generated the three models and\ntheir variants. The results of the empirical comparison show that the commonly\nused bag-of-words model has performance comparable to the one obtained by the\nemerging continuous-space model of doc2vec. In particular, the low-dimensional\nvariants of doc2vec generating up to 75 features are among the top-performing\ndocument representation models. The results finally point out that doc2vec\nshows a superior performance in the tasks of classifying large documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01355v2",
            "date": "2017-07-05 12:24:55+00:00",
            "title": "Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection",
            "authors": [
                "Peter Makarov",
                "Tatiana Ruzsics",
                "Simon Clematide"
            ],
            "abstract": "This paper presents the submissions by the University of Zurich to the\nSIGMORPHON 2017 shared task on morphological reinflection. The task is to\npredict the inflected form given a lemma and a set of morpho-syntactic\nfeatures. We focus on neural network approaches that can tackle the task in a\nlimited-resource setting. As the transduction of the lemma into the inflected\nform is dominated by copying over lemma characters, we propose two recurrent\nneural network architectures with hard monotonic attention that are strong at\ncopying and, yet, substantially different in how they achieve this. The first\napproach is an encoder-decoder model with a copy mechanism. The second approach\nis a neural state-transition system over a set of explicit edit actions,\nincluding a designated COPY action. We experiment with character alignment and\nfind that naive, greedy alignment consistently produces strong results for some\nlanguages. Our best system combination is the overall winner of the SIGMORPHON\n2017 Shared Task 1 without external resources. At a setting with 100 training\nsamples, both our approaches, as ensembles of models, outperform the next best\ncompetitor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01378v4",
            "date": "2017-07-05 13:08:03+00:00",
            "title": "An Attention Mechanism for Answer Selection Using a Combined Global and Local View",
            "authors": [
                "Yoram Bachrach",
                "Andrej Zukov-Gregoric",
                "Sam Coope",
                "Ed Tovell",
                "Bogdan Maksak",
                "Jose Rodriguez",
                "Conan McMurtie"
            ],
            "abstract": "We propose a new attention mechanism for neural based question answering,\nwhich depends on varying granularities of the input. Previous work focused on\naugmenting recurrent neural networks with simple attention mechanisms which are\na function of the similarity between a question embedding and an answer\nembeddings across time. We extend this by making the attention mechanism\ndependent on a global embedding of the answer attained using a separate\nnetwork.\n  We evaluate our system on InsuranceQA, a large question answering dataset.\nOur model outperforms current state-of-the-art results on InsuranceQA. Further,\nwe visualize which sections of text our attention mechanism focuses on, and\nexplore its performance across different parameter settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01521v1",
            "date": "2017-07-05 18:18:37+00:00",
            "title": "Context Aware Document Embedding",
            "authors": [
                "Zhaocheng Zhu",
                "Junfeng Hu"
            ],
            "abstract": "Recently, doc2vec has achieved excellent results in different tasks. In this\npaper, we present a context aware variant of doc2vec. We introduce a novel\nweight estimating mechanism that generates weights for each word occurrence\naccording to its contribution in the context, using deep neural networks. Our\ncontext aware model can achieve similar results compared to doc2vec initialized\nbyWikipedia trained vectors, while being much more efficient and free from\nheavy external corpus. Analysis of context aware weights shows they are a kind\nof enhanced IDF weights that capture sub-topic level keywords in documents.\nThey might result from deep neural networks that learn hidden representations\nwith the least entropy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01626v2",
            "date": "2017-07-06 03:50:38+00:00",
            "title": "Cross-Lingual Sentiment Analysis Without (Good) Translation",
            "authors": [
                "Mohamed Abdalla",
                "Graeme Hirst"
            ],
            "abstract": "Current approaches to cross-lingual sentiment analysis try to leverage the\nwealth of labeled English data using bilingual lexicons, bilingual vector space\nembeddings, or machine translation systems. Here we show that it is possible to\nuse a single linear transformation, with as few as 2000 word pairs, to capture\nfine-grained sentiment relationships between words in a cross-lingual setting.\nWe apply these cross-lingual sentiment models to a diverse set of tasks to\ndemonstrate their functionality in a non-English context. By effectively\nleveraging English sentiment knowledge without the need for accurate\ntranslation, we can analyze and extract features from other languages with\nscarce data at a very low cost, thus making sentiment and related analyses for\nmany languages inexpensive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01662v1",
            "date": "2017-07-06 07:39:06+00:00",
            "title": "An Embedded Deep Learning based Word Prediction",
            "authors": [
                "Seunghak Yu",
                "Nilesh Kulkarni",
                "Haejun Lee",
                "Jihie Kim"
            ],
            "abstract": "Recent developments in deep learning with application to language modeling\nhave led to success in tasks of text processing, summarizing and machine\ntranslation. However, deploying huge language models for mobile device such as\non-device keyboards poses computation as a bottle-neck due to their puny\ncomputation capacities. In this work we propose an embedded deep learning based\nword prediction method that optimizes run-time memory and also provides a real\ntime prediction environment. Our model size is 7.40MB and has average\nprediction time of 6.47 ms. We improve over the existing methods for word\nprediction in terms of key stroke savings and word prediction rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01793v2",
            "date": "2017-07-06 13:54:01+00:00",
            "title": "A Simple Approach to Learn Polysemous Word Embeddings",
            "authors": [
                "Yifan Sun",
                "Nikhil Rao",
                "Weicong Ding"
            ],
            "abstract": "Many NLP applications require disambiguating polysemous words. Existing\nmethods that learn polysemous word vector representations involve first\ndetecting various senses and optimizing the sense-specific embeddings\nseparately, which are invariably more involved than single sense learning\nmethods such as word2vec. Evaluating these methods is also problematic, as\nrigorous quantitative evaluations in this space is limited, especially when\ncompared with single-sense embeddings. In this paper, we propose a simple\nmethod to learn a word representation, given any context. Our method only\nrequires learning the usual single sense representation, and coefficients that\ncan be learnt via a single pass over the data. We propose several new test sets\nfor evaluating word sense induction, relevance detection, and contextual word\nsimilarity, significantly supplementing the currently available tests. Results\non these and other tests show that while our method is embarrassingly simple,\nit achieves excellent results when compared to the state of the art models for\nunsupervised polysemous word representation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01830v2",
            "date": "2017-07-06 15:12:55+00:00",
            "title": "Single-Queue Decoding for Neural Machine Translation",
            "authors": [
                "Raphael Shu",
                "Hideki Nakayama"
            ],
            "abstract": "Neural machine translation models rely on the beam search algorithm for\ndecoding. In practice, we found that the quality of hypotheses in the search\nspace is negatively affected owing to the fixed beam size. To mitigate this\nproblem, we store all hypotheses in a single priority queue and use a universal\nscore function for hypothesis selection. The proposed algorithm is more\nflexible as the discarded hypotheses can be revisited in a later step. We\nfurther design a penalty function to punish the hypotheses that tend to produce\na final translation that is much longer or shorter than expected. Despite its\nsimplicity, we show that the proposed decoding algorithm is able to select\nhypotheses with better qualities and improve the translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02026v2",
            "date": "2017-07-07 03:10:32+00:00",
            "title": "A Nested Attention Neural Hybrid Model for Grammatical Error Correction",
            "authors": [
                "Jianshu Ji",
                "Qinlong Wang",
                "Kristina Toutanova",
                "Yongen Gong",
                "Steven Truong",
                "Jianfeng Gao"
            ],
            "abstract": "Grammatical error correction (GEC) systems strive to correct both global\nerrors in word order and usage, and local errors in spelling and inflection.\nFurther developing upon recent work on neural machine translation, we propose a\nnew hybrid neural model with nested attention layers for GEC. Experiments show\nthat the new model can effectively correct errors of both types by\nincorporating word and character-level information,and that the model\nsignificantly outperforms previous neural models for GEC as measured on the\nstandard CoNLL-14 benchmark dataset. Further analysis also shows that the\nsuperiority of the proposed model can be largely attributed to the use of the\nnested attention mechanism, which has proven particularly effective in\ncorrecting local errors that involve small edits in orthography.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02063v1",
            "date": "2017-07-07 07:46:54+00:00",
            "title": "External Evaluation of Event Extraction Classifiers for Automatic Pathway Curation: An extended study of the mTOR pathway",
            "authors": [
                "Wojciech Kusa",
                "Michael Spranger"
            ],
            "abstract": "This paper evaluates the impact of various event extraction systems on\nautomatic pathway curation using the popular mTOR pathway. We quantify the\nimpact of training data sets as well as different machine learning classifiers\nand show that some improve the quality of automatically extracted pathways.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02230v1",
            "date": "2017-07-07 15:34:08+00:00",
            "title": "Computational Models of Tutor Feedback in Language Acquisition",
            "authors": [
                "Jens Nevens",
                "Michael Spranger"
            ],
            "abstract": "This paper investigates the role of tutor feedback in language learning using\ncomputational models. We compare two dominant paradigms in language learning:\ninteractive learning and cross-situational learning - which differ primarily in\nthe role of social feedback such as gaze or pointing. We analyze the\nrelationship between these two paradigms and propose a new mixed paradigm that\ncombines the two paradigms and allows to test algorithms in experiments that\ncombine no feedback and social feedback. To deal with mixed feedback\nexperiments, we develop new algorithms and show how they perform with respect\nto traditional knn and prototype approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02268v3",
            "date": "2017-07-07 16:55:56+00:00",
            "title": "Text Summarization Techniques: A Brief Survey",
            "authors": [
                "Mehdi Allahyari",
                "Seyedamin Pouriyeh",
                "Mehdi Assefi",
                "Saeid Safaei",
                "Elizabeth D. Trippe",
                "Juan B. Gutierrez",
                "Krys Kochut"
            ],
            "abstract": "In recent years, there has been a explosion in the amount of text data from a\nvariety of sources. This volume of text is an invaluable source of information\nand knowledge which needs to be effectively summarized to be useful. In this\nreview, the main approaches to automatic text summarization are described. We\nreview the different processes for summarization and describe the effectiveness\nand shortcomings of the different methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02633v1",
            "date": "2017-07-09 20:21:36+00:00",
            "title": "Controlling Linguistic Style Aspects in Neural Language Generation",
            "authors": [
                "Jessica Ficler",
                "Yoav Goldberg"
            ],
            "abstract": "Most work on neural natural language generation (NNLG) focus on controlling\nthe content of the generated text. We experiment with controlling several\nstylistic aspects of the generated text, in addition to its content. The method\nis based on conditioned RNN language model, where the desired content as well\nas the stylistic parameters serve as conditioning contexts. We demonstrate the\napproach on the movie reviews domain and show that it is successful in\ngenerating coherent sentences corresponding to the required linguistic style\nand content.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02786v4",
            "date": "2017-07-10 10:18:08+00:00",
            "title": "Learning to Compose Task-Specific Tree Structures",
            "authors": [
                "Jihun Choi",
                "Kang Min Yoo",
                "Sang-goo Lee"
            ],
            "abstract": "For years, recursive neural networks (RvNNs) have been shown to be suitable\nfor representing text into fixed-length vectors and achieved good performance\non several natural language processing tasks. However, the main drawback of\nRvNNs is that they require structured input, which makes data preparation and\nmodel implementation hard. In this paper, we propose Gumbel Tree-LSTM, a novel\ntree-structured long short-term memory architecture that learns how to compose\ntask-specific tree structures only from plain text data efficiently. Our model\nuses Straight-Through Gumbel-Softmax estimator to decide the parent node among\ncandidates dynamically and to calculate gradients of the discrete decision. We\nevaluate the proposed model on natural language inference and sentiment\nanalysis, and show that our model outperforms or is at least comparable to\nprevious models. We also find that our model converges significantly faster\nthan other models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02892v1",
            "date": "2017-07-10 14:58:53+00:00",
            "title": "A Generalized Recurrent Neural Architecture for Text Classification with Multi-Task Learning",
            "authors": [
                "Honglun Zhang",
                "Liqiang Xiao",
                "Yongkun Wang",
                "Yaohui Jin"
            ],
            "abstract": "Multi-task learning leverages potential correlations among related tasks to\nextract common features and yield performance gains. However, most previous\nworks only consider simple or weak interactions, thereby failing to model\ncomplex correlations among three or more tasks. In this paper, we propose a\nmulti-task learning architecture with four types of recurrent neural layers to\nfuse information across multiple related tasks. The architecture is\nstructurally flexible and considers various interactions among tasks, which can\nbe regarded as a generalized case of many previous works. Extensive experiments\non five benchmark datasets for text classification show that our model can\nsignificantly improve performances of related tasks with additional information\nfrom others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03058v1",
            "date": "2017-07-10 20:47:33+00:00",
            "title": "Improving Neural Parsing by Disentangling Model Combination and Reranking Effects",
            "authors": [
                "Daniel Fried",
                "Mitchell Stern",
                "Dan Klein"
            ],
            "abstract": "Recent work has proposed several generative neural models for constituency\nparsing that achieve state-of-the-art results. Since direct search in these\ngenerative models is difficult, they have primarily been used to rescore\ncandidate outputs from base parsers in which decoding is more straightforward.\nWe first present an algorithm for direct search in these generative models. We\nthen demonstrate that the rescoring results are at least partly due to implicit\nmodel combination rather than reranking effects. Finally, we show that explicit\nmodel combination can improve performance even further, resulting in new\nstate-of-the-art numbers on the PTB of 94.25 F1 when training only on gold data\nand 94.66 F1 when using external data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03103v2",
            "date": "2017-07-11 02:02:09+00:00",
            "title": "Refining Raw Sentence Representations for Textual Entailment Recognition via Attention",
            "authors": [
                "Jorge A. Balazs",
                "Edison Marrese-Taylor",
                "Pablo Loyola",
                "Yutaka Matsuo"
            ],
            "abstract": "In this paper we present the model used by the team Rivercorners for the 2017\nRepEval shared task. First, our model separately encodes a pair of sentences\ninto variable-length representations by using a bidirectional LSTM. Later, it\ncreates fixed-length raw representations by means of simple aggregation\nfunctions, which are then refined using an attention mechanism. Finally it\ncombines the refined representations of both sentences into a single vector to\nbe used for classification. With this model we obtained test accuracies of\n72.057% and 72.055% in the matched and mismatched evaluation tracks\nrespectively, outperforming the LSTM baseline, and obtaining performances\nsimilar to a model that relies on shared information between sentences (ESIM).\nWhen using an ensemble both accuracies increased to 72.247% and 72.827%\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03172v1",
            "date": "2017-07-11 08:33:55+00:00",
            "title": "Dataset for a Neural Natural Language Interface for Databases (NNLIDB)",
            "authors": [
                "Florin Brad",
                "Radu Iacob",
                "Ionel Hosu",
                "Traian Rebedea"
            ],
            "abstract": "Progress in natural language interfaces to databases (NLIDB) has been slow\nmainly due to linguistic issues (such as language ambiguity) and domain\nportability. Moreover, the lack of a large corpus to be used as a standard\nbenchmark has made data-driven approaches difficult to develop and compare. In\nthis paper, we revisit the problem of NLIDBs and recast it as a sequence\ntranslation problem. To this end, we introduce a large dataset extracted from\nthe Stack Exchange Data Explorer website, which can be used for training neural\nnatural language interfaces for databases. We also report encouraging baseline\nresults on a smaller manually annotated test corpus, obtained using an\nattention-based sequence-to-sequence neural network.",
            "journal ref": "Proceedings of the Eighth International Joint Conference on\n  Natural Language Processing (Volume 1: Long Papers), 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03228v1",
            "date": "2017-07-11 11:44:31+00:00",
            "title": "A non-projective greedy dependency parser with bidirectional LSTMs",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "The LyS-FASTPARSE team presents BIST-COVINGTON, a neural implementation of\nthe Covington (2001) algorithm for non-projective dependency parsing. The\nbidirectional LSTM approach by Kipperwasser and Goldberg (2016) is used to\ntrain a greedy parser with a dynamic oracle to mitigate error propagation. The\nmodel participated in the CoNLL 2017 UD Shared Task. In spite of not using any\nensemble methods and using the baseline segmentation and PoS tagging, the\nparser obtained good results on both macro-average LAS and UAS in the big\ntreebanks category (55 languages), ranking 7th out of 33 teams. In the all\ntreebanks category (LAS and UAS) we ranked 16th and 12th. The gap between the\nall and big categories is mainly due to the poor performance on four parallel\nPUD treebanks, suggesting that some `suffixed' treebanks (e.g. Spanish-AnCora)\nperform poorly on cross-treebank settings, which does not occur with the\ncorresponding `unsuffixed' treebank (e.g. Spanish). By changing that, we obtain\nthe 11th best LAS among all runs (official and unofficial). The code is made\navailable at https://github.com/CoNLL-UD-2017/LyS-FASTPARSE",
            "journal ref": "In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing\n  from Raw Text to Universal Dependencies, pages 152-162, Vancouver, Canada,\n  2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03253v1",
            "date": "2017-07-11 13:04:15+00:00",
            "title": "Leipzig Corpus Miner - A Text Mining Infrastructure for Qualitative Data Analysis",
            "authors": [
                "Andreas Niekler",
                "Gregor Wiedemann",
                "Gerhard Heyer"
            ],
            "abstract": "This paper presents the \"Leipzig Corpus Miner\", a technical infrastructure\nfor supporting qualitative and quantitative content analysis. The\ninfrastructure aims at the integration of 'close reading' procedures on\nindividual documents with procedures of 'distant reading', e.g. lexical\ncharacteristics of large document collections. Therefore information retrieval\nsystems, lexicometric statistics and machine learning procedures are combined\nin a coherent framework which enables qualitative data analysts to make use of\nstate-of-the-art Natural Language Processing techniques on very large document\ncollections. Applicability of the framework ranges from social sciences to\nmedia studies and market research. As an example we introduce the usage of the\nframework in a political science study on post-democracy and neoliberalism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03255v1",
            "date": "2017-07-11 13:11:45+00:00",
            "title": "Modeling the dynamics of domain specific terminology in diachronic corpora",
            "authors": [
                "Gerhard Heyer",
                "Cathleen Kantner",
                "Andreas Niekler",
                "Max Overbeck",
                "Gregor Wiedemann"
            ],
            "abstract": "In terminology work, natural language processing, and digital humanities,\nseveral studies address the analysis of variations in context and meaning of\nterms in order to detect semantic change and the evolution of terms. We\ndistinguish three different approaches to describe contextual variations:\nmethods based on the analysis of patterns and linguistic clues, methods\nexploring the latent semantic space of single words, and methods for the\nanalysis of topic membership. The paper presents the notion of context\nvolatility as a new measure for detecting semantic change and applies it to key\nterm extraction in a political science case study. The measure quantifies the\ndynamics of a term's contextual variation within a diachronic corpus to\nidentify periods of time that are characterised by intense controversial\ndebates or substantial semantic transformations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03264v2",
            "date": "2017-07-11 13:44:51+00:00",
            "title": "A simple but tough-to-beat baseline for the Fake News Challenge stance detection task",
            "authors": [
                "Benjamin Riedel",
                "Isabelle Augenstein",
                "Georgios P. Spithourakis",
                "Sebastian Riedel"
            ],
            "abstract": "Identifying public misinformation is a complicated and challenging task. An\nimportant part of checking the veracity of a specific claim is to evaluate the\nstance different news sources take towards the assertion. Automatic stance\nevaluation, i.e. stance detection, would arguably facilitate the process of\nfact checking. In this paper, we present our stance detection system which\nclaimed third place in Stage 1 of the Fake News Challenge. Despite our\nstraightforward approach, our system performs at a competitive level with the\ncomplex ensembles of the top two winning teams. We therefore propose our system\nas the 'simple but tough-to-beat baseline' for the Fake News Challenge stance\ndetection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03550v2",
            "date": "2017-07-12 05:41:06+00:00",
            "title": "Geospatial Semantics",
            "authors": [
                "Yingjie Hu"
            ],
            "abstract": "Geospatial semantics is a broad field that involves a variety of research\nareas. The term semantics refers to the meaning of things, and is in contrast\nwith the term syntactics. Accordingly, studies on geospatial semantics usually\nfocus on understanding the meaning of geographic entities as well as their\ncounterparts in the cognitive and digital world, such as cognitive geographic\nconcepts and digital gazetteers. Geospatial semantics can also facilitate the\ndesign of geographic information systems (GIS) by enhancing the\ninteroperability of distributed systems and developing more intelligent\ninterfaces for user interactions. During the past years, a lot of research has\nbeen conducted, approaching geospatial semantics from different perspectives,\nusing a variety of methods, and targeting different problems. Meanwhile, the\narrival of big geo data, especially the large amount of unstructured text data\non the Web, and the fast development of natural language processing methods\nenable new research directions in geospatial semantics. This chapter,\ntherefore, provides a systematic review on the existing geospatial semantic\nresearch. Six major research areas are identified and discussed, including\nsemantic interoperability, digital gazetteers, geographic information\nretrieval, geospatial Semantic Web, place semantics, and cognitive geographic\nconcepts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03736v2",
            "date": "2017-07-12 14:27:00+00:00",
            "title": "The Case for Being Average: A Mediocrity Approach to Style Masking and Author Obfuscation",
            "authors": [
                "Georgi Karadjov",
                "Tsvetomila Mihaylova",
                "Yasen Kiprov",
                "Georgi Georgiev",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Users posting online expect to remain anonymous unless they have logged in,\nwhich is often needed for them to be able to discuss freely on various topics.\nPreserving the anonymity of a text's writer can be also important in some other\ncontexts, e.g., in the case of witness protection or anonymity programs.\nHowever, each person has his/her own style of writing, which can be analyzed\nusing stylometry, and as a result, the true identity of the author of a piece\nof text can be revealed even if s/he has tried to hide it. Thus, it could be\nhelpful to design automatic tools that can help a person obfuscate his/her\nidentity when writing text. In particular, here we propose an approach that\nchanges the text, so that it is pushed towards average values for some general\nstylometric characteristics, thus making the use of these characteristics less\ndiscriminative. The approach consists of three main steps: first, we calculate\nthe values for some popular stylometric metrics that can indicate authorship;\nthen we apply various transformations to the text, so that these metrics are\nadjusted towards the average level, while preserving the semantics and the\nsoundness of the text; and finally, we add random noise. This approach turned\nout to be very efficient, and yielded the best performance on the Author\nObfuscation task at the PAN-2016 competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03764v1",
            "date": "2017-07-12 15:34:21+00:00",
            "title": "N-GrAM: New Groningen Author-profiling Model",
            "authors": [
                "Angelo Basile",
                "Gareth Dwyer",
                "Maria Medvedeva",
                "Josine Rawee",
                "Hessel Haagsma",
                "Malvina Nissim"
            ],
            "abstract": "We describe our participation in the PAN 2017 shared task on Author\nProfiling, identifying authors' gender and language variety for English,\nSpanish, Arabic and Portuguese. We describe both the final, submitted system,\nand a series of negative results. Our aim was to create a single model for both\ngender and language, and for all language varieties. Our best-performing system\n(on cross-validated results) is a linear support vector machine (SVM) with word\nunigrams and character 3- to 5-grams as features. A set of additional features,\nincluding POS tags, additional datasets, geographic entities, and Twitter\nhandles, hurt, rather than improve, performance. Results from cross-validation\nindicated high performance overall and results on the test set confirmed them,\nat 0.86 averaged accuracy, with performance on sub-tasks ranging from 0.68 to\n0.98.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03819v1",
            "date": "2017-07-12 10:09:32+00:00",
            "title": "A Critique of a Critique of Word Similarity Datasets: Sanity Check or Unnecessary Confusion?",
            "authors": [
                "Minh Le"
            ],
            "abstract": "Critical evaluation of word similarity datasets is very important for\ncomputational lexical semantics. This short report concerns the sanity check\nproposed in Batchkarov et al. (2016) to evaluate several popular datasets such\nas MC, RG and MEN -- the first two reportedly failed. I argue that this test is\nunstable, offers no added insight, and needs major revision in order to fulfill\nits purported goal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03903v2",
            "date": "2017-07-12 20:47:47+00:00",
            "title": "Negative Sampling Improves Hypernymy Extraction Based on Projection Learning",
            "authors": [
                "Dmitry Ustalov",
                "Nikolay Arefyev",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We present a new approach to extraction of hypernyms based on projection\nlearning and word embeddings. In contrast to classification-based approaches,\nprojection-based methods require no candidate hyponym-hypernym pairs. While it\nis natural to use both positive and negative training examples in supervised\nrelation extraction, the impact of negative examples on hypernym prediction was\nnot studied so far. In this paper, we show that explicit negative examples used\nfor regularization of the model significantly improve performance compared to\nthe state-of-the-art approach of Fu et al. (2014) on three datasets from\ndifferent languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03968v1",
            "date": "2017-07-13 03:47:32+00:00",
            "title": "Predicting Causes of Reformulation in Intelligent Assistants",
            "authors": [
                "Shumpei Sano",
                "Nobuhiro Kaji",
                "Manabu Sassano"
            ],
            "abstract": "Intelligent assistants (IAs) such as Siri and Cortana conversationally\ninteract with users and execute a wide range of actions (e.g., searching the\nWeb, setting alarms, and chatting). IAs can support these actions through the\ncombination of various components such as automatic speech recognition, natural\nlanguage understanding, and language generation. However, the complexity of\nthese components hinders developers from determining which component causes an\nerror. To remove this hindrance, we focus on reformulation, which is a useful\nsignal of user dissatisfaction, and propose a method to predict the\nreformulation causes. We evaluate the method using the user logs of a\ncommercial IA. The experimental results have demonstrated that features\ndesigned to detect the error of a specific component improve the performance of\nreformulation cause detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04095v1",
            "date": "2017-07-13 12:51:39+00:00",
            "title": "Is writing style predictive of scientific fraud?",
            "authors": [
                "Chlo\u00e9 Braud",
                "Anders S\u00f8gaard"
            ],
            "abstract": "The problem of detecting scientific fraud using machine learning was recently\nintroduced, with initial, positive results from a model taking into account\nvarious general indicators. The results seem to suggest that writing style is\npredictive of scientific fraud. We revisit these initial experiments, and show\nthat the leave-one-out testing procedure they used likely leads to a slight\nover-estimate of the predictability, but also that simple models can outperform\ntheir proposed model by some margin. We go on to explore more abstract\nlinguistic features, such as linguistic complexity and discourse structure,\nonly to obtain negative results. Upon analyzing our models, we do see some\ninteresting patterns, though: Scientific fraud, for examples, contains less\ncomparison, as well as different types of hedging and ways of presenting\nlogical reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04108v1",
            "date": "2017-07-13 13:18:52+00:00",
            "title": "Do Convolutional Networks need to be Deep for Text Classification ?",
            "authors": [
                "Hoa T. Le",
                "Christophe Cerisara",
                "Alexandre Denis"
            ],
            "abstract": "We study in this work the importance of depth in convolutional models for\ntext classification, either when character or word inputs are considered. We\nshow on 5 standard text classification and sentiment analysis tasks that deep\nmodels indeed give better performances than shallow networks when the text\ninput is represented as a sequence of characters. However, a simple\nshallow-and-wide network outperforms deep models such as DenseNet with word\ninputs. Our shallow word model further establishes new state-of-the-art\nperformances on two datasets: Yelp Binary (95.9\\%) and Yelp Full (64.9\\%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04408v1",
            "date": "2017-07-14 08:00:07+00:00",
            "title": "Developing a concept-level knowledge base for sentiment analysis in Singlish",
            "authors": [
                "Rajiv Bajpai",
                "Soujanya Poria",
                "Danyun Ho",
                "Erik Cambria"
            ],
            "abstract": "In this paper, we present Singlish sentiment lexicon, a concept-level\nknowledge base for sentiment analysis that associates multiword expressions to\na set of emotion labels and a polarity value. Unlike many other sentiment\nanalysis resources, this lexicon is not built by manually labeling pieces of\nknowledge coming from general NLP resources such as WordNet or DBPedia.\nInstead, it is automatically constructed by applying graph-mining and\nmulti-dimensional scaling techniques on the affective common-sense knowledge\ncollected from three different sources. This knowledge is represented\nredundantly at three levels: semantic network, matrix, and vector space.\nSubsequently, the concepts are labeled by emotions and polarity through the\nensemble application of spreading activation, neural networks and an emotion\ncategorization model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04412v1",
            "date": "2017-07-14 08:25:36+00:00",
            "title": "Evaluating Semantic Parsing against a Simple Web-based Question Answering Model",
            "authors": [
                "Alon Talmor",
                "Mor Geva",
                "Jonathan Berant"
            ],
            "abstract": "Semantic parsing shines at analyzing complex natural language that involves\ncomposition and computation over multiple pieces of evidence. However, datasets\nfor semantic parsing contain many factoid questions that can be answered from a\nsingle web document. In this paper, we propose to evaluate semantic\nparsing-based question answering models by comparing them to a question\nanswering baseline that queries the web and extracts the answer only from web\nsnippets, without access to the target knowledge-base. We investigate this\napproach on COMPLEXQUESTIONS, a dataset designed to focus on compositional\nlanguage, and find that our model obtains reasonable performance (35 F1\ncompared to 41 F1 of state-of-the-art). We find in our analysis that our model\nperforms well on complex questions involving conjunctions, but struggles on\nquestions that involve relation composition and superlatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04481v1",
            "date": "2017-07-14 12:33:25+00:00",
            "title": "LIUM-CVC Submissions for WMT17 Multimodal Translation Task",
            "authors": [
                "Ozan Caglayan",
                "Walid Aransa",
                "Adrien Bardet",
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Fethi Bougares",
                "Lo\u00efc Barrault",
                "Marc Masana",
                "Luis Herranz",
                "Joost van de Weijer"
            ],
            "abstract": "This paper describes the monomodal and multimodal Neural Machine Translation\nsystems developed by LIUM and CVC for WMT17 Shared Task on Multimodal\nTranslation. We mainly explored two multimodal architectures where either\nglobal visual features or convolutional feature maps are integrated in order to\nbenefit from visual context. Our final systems ranked first for both En-De and\nEn-Fr language pairs according to the automatic evaluation metrics METEOR and\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04499v1",
            "date": "2017-07-14 13:10:22+00:00",
            "title": "LIUM Machine Translation Systems for WMT17 News Translation Task",
            "authors": [
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Ozan Caglayan",
                "Walid Aransa",
                "Adrien Bardet",
                "Fethi Bougares",
                "Lo\u00efc Barrault"
            ],
            "abstract": "This paper describes LIUM submissions to WMT17 News Translation Task for\nEnglish-German, English-Turkish, English-Czech and English-Latvian language\npairs. We train BPE-based attentive Neural Machine Translation systems with and\nwithout factored outputs using the open source nmtpy framework. Competitive\nscores were obtained by ensembling various systems and exploiting the\navailability of target monolingual corpora for back-translation. The impact of\nback-translation quantity and quality is also analyzed for English-Turkish\nwhere our post-deadline submission surpassed the best entry by +1.6 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04538v1",
            "date": "2017-07-14 15:21:25+00:00",
            "title": "Cross-genre Document Retrieval: Matching between Conversational and Formal Writings",
            "authors": [
                "Tomasz Jurczyk",
                "Jinho D. Choi"
            ],
            "abstract": "This paper challenges a cross-genre document retrieval task, where the\nqueries are in formal writing and the target documents are in conversational\nwriting. In this task, a query, is a sentence extracted from either a summary\nor a plot of an episode in a TV show, and the target document consists of\ntranscripts from the corresponding episode. To establish a strong baseline, we\nemploy the current state-of-the-art search engine to perform document retrieval\non the dataset collected for this work. We then introduce a structure reranking\napproach to improve the initial ranking by utilizing syntactic and semantic\nstructures generated by NLP tools. Our evaluation shows an improvement of more\nthan 4% when the structure reranking is applied, which is very promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04662v1",
            "date": "2017-07-14 23:26:35+00:00",
            "title": "Rotations and Interpretability of Word Embeddings: the Case of the Russian Language",
            "authors": [
                "Alexey Zobnin"
            ],
            "abstract": "Consider a continuous word embedding model. Usually, the cosines between word\nvectors are used as a measure of similarity of words. These cosines do not\nchange under orthogonal transformations of the embedding space. We demonstrate\nthat, using some canonical orthogonal transformations from SVD, it is possible\nboth to increase the meaning of some components and to make the components more\nstable under re-learning. We study the interpretability of components for\npublicly available models for the Russian language (RusVectores, fastText,\nRDT).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04817v1",
            "date": "2017-07-16 04:17:49+00:00",
            "title": "Open-Set Language Identification",
            "authors": [
                "Shervin Malmasi"
            ],
            "abstract": "We present the first open-set language identification experiments using\none-class classification. We first highlight the shortcomings of traditional\nfeature extraction methods and propose a hashing-based feature vectorization\napproach as a solution. Using a dataset of 10 languages from different writing\nsystems, we train a One- Class Support Vector Machine using only a monolingual\ncorpus for each language. Each model is evaluated against a test set of data\nfrom all 10 languages and we achieve an average F-score of 0.99, highlighting\nthe effectiveness of this approach for open-set language identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04848v2",
            "date": "2017-07-16 09:08:42+00:00",
            "title": "Do Neural Nets Learn Statistical Laws behind Natural Language?",
            "authors": [
                "Shuntaro Takahashi",
                "Kumiko Tanaka-Ishii"
            ],
            "abstract": "The performance of deep learning in natural language processing has been\nspectacular, but the reasons for this success remain unclear because of the\ninherent complexity of deep learning. This paper provides empirical evidence of\nits effectiveness and of a limitation of neural networks for language\nengineering. Precisely, we demonstrate that a neural language model based on\nlong short-term memory (LSTM) effectively reproduces Zipf's law and Heaps' law,\ntwo representative statistical properties underlying natural language. We\ndiscuss the quality of reproducibility and the emergence of Zipf's law and\nHeaps' law as training progresses. We also point out that the neural language\nmodel has a limitation in reproducing long-range correlation, another\nstatistical property of natural language. This understanding could provide a\ndirection for improving the architectures of neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04860v1",
            "date": "2017-07-16 11:08:08+00:00",
            "title": "Automated Detection of Non-Relevant Posts on the Russian Imageboard \"2ch\": Importance of the Choice of Word Representations",
            "authors": [
                "Amir Bakarov",
                "Olga Gureenkova"
            ],
            "abstract": "This study considers the problem of automated detection of non-relevant posts\non Web forums and discusses the approach of resolving this problem by\napproximation it with the task of detection of semantic relatedness between the\ngiven post and the opening post of the forum discussion thread. The\napproximated task could be resolved through learning the supervised classifier\nwith a composed word embeddings of two posts. Considering that the success in\nthis task could be quite sensitive to the choice of word representations, we\npropose a comparison of the performance of different word embedding models. We\ntrain 7 models (Word2Vec, Glove, Word2Vec-f, Wang2Vec, AdaGram, FastText,\nSwivel), evaluate embeddings produced by them on dataset of human judgements\nand compare their performance on the task of non-relevant posts detection. To\nmake the comparison, we propose a dataset of semantic relatedness with posts\nfrom one of the most popular Russian Web forums, imageboard \"2ch\", which has\nchallenging lexical and grammatical features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04913v1",
            "date": "2017-07-16 16:57:36+00:00",
            "title": "End-to-End Information Extraction without Token-Level Supervision",
            "authors": [
                "Rasmus Berg Palm",
                "Dirk Hovy",
                "Florian Laws",
                "Ole Winther"
            ],
            "abstract": "Most state-of-the-art information extraction approaches rely on token-level\nlabels to find the areas of interest in text. Unfortunately, these labels are\ntime-consuming and costly to create, and consequently, not available for many\nreal-life IE tasks. To make matters worse, token-level labels are usually not\nthe desired output, but just an intermediary step. End-to-end (E2E) models,\nwhich take raw text as input and produce the desired output directly, need not\ndepend on token-level labels. We propose an E2E model based on pointer\nnetworks, which can be trained directly on pairs of raw input and output text.\nWe evaluate our model on the ATIS data set, MIT restaurant corpus and the MIT\nmovie corpus and compare to neural baselines that do use token-level labels. We\nachieve competitive results, within a few percentage points of the baselines,\nshowing the feasibility of E2E information extraction without the need for\ntoken-level labels. This opens up new possibilities, as for many tasks\ncurrently addressed by human extractors, raw input and output data are\navailable, but not token-level labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05000v1",
            "date": "2017-07-17 04:27:11+00:00",
            "title": "In-Order Transition-based Constituent Parsing",
            "authors": [
                "Jiangming Liu",
                "Yue Zhang"
            ],
            "abstract": "Both bottom-up and top-down strategies have been used for neural\ntransition-based constituent parsing. The parsing strategies differ in terms of\nthe order in which they recognize productions in the derivation tree, where\nbottom-up strategies and top-down strategies take post-order and pre-order\ntraversal over trees, respectively. Bottom-up parsers benefit from rich\nfeatures from readily built partial parses, but lack lookahead guidance in the\nparsing process; top-down parsers benefit from non-local guidance for local\ndecisions, but rely on a strong encoder over the input to predict a constituent\nhierarchy before its construction.To mitigate both issues, we propose a novel\nparsing system based on in-order traversal over syntactic trees, designing a\nset of transition actions to find a compromise between bottom-up constituent\ninformation and top-down lookahead information. Based on stack-LSTM, our\npsycholinguistically motivated constituent parsing system achieves 91.8 F1 on\nWSJ benchmark. Furthermore, the system achieves 93.6 F1 with supervised\nreranking and 94.2 F1 with semi-supervised reranking, which are the best\nresults on the WSJ benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05114v1",
            "date": "2017-07-17 12:09:08+00:00",
            "title": "Towards Bidirectional Hierarchical Representations for Attention-Based Neural Machine Translation",
            "authors": [
                "Baosong Yang",
                "Derek F. Wong",
                "Tong Xiao",
                "Lidia S. Chao",
                "Jingbo Zhu"
            ],
            "abstract": "This paper proposes a hierarchical attentional neural translation model which\nfocuses on enhancing source-side hierarchical representations by covering both\nlocal and global semantic information using a bidirectional tree-based encoder.\nTo maximize the predictive likelihood of target words, a weighted variant of an\nattention mechanism is used to balance the attentive information between\nlexical and phrase vectors. Using a tree-based rare word encoding, the proposed\nmodel is extended to sub-word level to alleviate the out-of-vocabulary (OOV)\nproblem. Empirical results reveal that the proposed model significantly\noutperforms sequence-to-sequence attention-based and tree-based neural\ntranslation models in English-Chinese translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05116v1",
            "date": "2017-07-17 12:20:53+00:00",
            "title": "To Normalize, or Not to Normalize: The Impact of Normalization on Part-of-Speech Tagging",
            "authors": [
                "Rob van der Goot",
                "Barbara Plank",
                "Malvina Nissim"
            ],
            "abstract": "Does normalization help Part-of-Speech (POS) tagging accuracy on noisy,\nnon-canonical data? To the best of our knowledge, little is known on the actual\nimpact of normalization in a real-world scenario, where gold error detection is\nnot available. We investigate the effect of automatic normalization on POS\ntagging of tweets. We also compare normalization to strategies that leverage\nlarge amounts of unlabeled data kept in its raw form. Our results show that\nnormalization helps, but does not add consistently beyond just word embedding\nlayer initialization. The latter approach yields a tagging model that is\ncompetitive with a Twitter state-of-the-art tagger.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05118v1",
            "date": "2017-07-17 12:25:52+00:00",
            "title": "LIG-CRIStAL System for the WMT17 Automatic Post-Editing Task",
            "authors": [
                "Alexandre Berard",
                "Olivier Pietquin",
                "Laurent Besacier"
            ],
            "abstract": "This paper presents the LIG-CRIStAL submission to the shared Automatic Post-\nEditing task of WMT 2017. We propose two neural post-editing models: a\nmonosource model with a task-specific attention mechanism, which performs\nparticularly well in a low-resource scenario; and a chained architecture which\nmakes use of the source sentence to provide extra context. This latter\narchitecture manages to slightly improve our results when more training data is\navailable. We present and discuss our results on two datasets (en-de and de-en)\nthat are made available for the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05127v1",
            "date": "2017-07-17 12:57:53+00:00",
            "title": "Neural Reranking for Named Entity Recognition",
            "authors": [
                "Jie Yang",
                "Yue Zhang",
                "Fei Dong"
            ],
            "abstract": "We propose a neural reranking system for named entity recognition (NER). The\nbasic idea is to leverage recurrent neural network models to learn\nsentence-level patterns that involve named entity mentions. In particular,\ngiven an output sentence produced by a baseline NER model, we replace all\nentity mentions, such as \\textit{Barack Obama}, into their entity types, such\nas \\textit{PER}. The resulting sentence patterns contain direct output\ninformation, yet is less sparse without specific named entities. For example,\n\"PER was born in LOC\" can be such a pattern. LSTM and CNN structures are\nutilised for learning deep representations of such sentences for reranking.\nResults show that our system can significantly improve the NER accuracies over\ntwo different baselines, giving the best reported results on a standard\nbenchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05261v1",
            "date": "2017-07-17 16:12:34+00:00",
            "title": "Exploring text datasets by visualizing relevant words",
            "authors": [
                "Franziska Horn",
                "Leila Arras",
                "Gr\u00e9goire Montavon",
                "Klaus-Robert M\u00fcller",
                "Wojciech Samek"
            ],
            "abstract": "When working with a new dataset, it is important to first explore and\nfamiliarize oneself with it, before applying any advanced machine learning\nalgorithms. However, to the best of our knowledge, no tools exist that quickly\nand reliably give insight into the contents of a selection of documents with\nrespect to what distinguishes them from other documents belonging to different\ncategories. In this paper we propose to extract `relevant words' from a\ncollection of texts, which summarize the contents of documents belonging to a\ncertain class (or discovered cluster in the case of unlabeled datasets), and\nvisualize them in word clouds to allow for a survey of salient features at a\nglance. We compare three methods for extracting relevant words and demonstrate\nthe usefulness of the resulting word clouds by providing an overview of the\nclasses contained in a dataset of scientific publications as well as by\ndiscovering trending topics from recent New York Times article snippets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05266v1",
            "date": "2017-07-17 16:21:46+00:00",
            "title": "A Simple Language Model based on PMI Matrix Approximations",
            "authors": [
                "Oren Melamud",
                "Ido Dagan",
                "Jacob Goldberger"
            ],
            "abstract": "In this study, we introduce a new approach for learning language models by\ntraining them to estimate word-context pointwise mutual information (PMI), and\nthen deriving the desired conditional probabilities from PMI at test time.\nSpecifically, we show that with minor modifications to word2vec's algorithm, we\nget principled language models that are closely related to the well-established\nNoise Contrastive Estimation (NCE) based language models. A compelling aspect\nof our approach is that our models are trained with the same simple negative\nsampling objective function that is commonly used in word2vec to learn word\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05288v3",
            "date": "2017-07-17 17:14:52+00:00",
            "title": "MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach",
            "authors": [
                "Diego Moussallem",
                "Ricardo Usbeck",
                "Michael R\u00f6der",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "Entity linking has recently been the subject of a significant body of\nresearch. Currently, the best performing approaches rely on trained\nmono-lingual models. Porting these approaches to other languages is\nconsequently a difficult endeavor as it requires corresponding training data\nand retraining of the models. We address this drawback by presenting a novel\nmultilingual, knowledge-based agnostic and deterministic approach to entity\nlinking, dubbed MAG. MAG is based on a combination of context-based retrieval\non structured knowledge bases and graph algorithms. We evaluate MAG on 23 data\nsets and in 7 languages. Our results show that the best approach trained on\nEnglish datasets (PBOH) achieves a micro F-measure that is up to 4 times worse\non datasets in other languages. MAG, on the other hand, achieves\nstate-of-the-art performance on English datasets and reaches a micro F-measure\nthat is up to 0.6 higher than that of PBOH on non-English languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05315v1",
            "date": "2017-07-17 10:20:15+00:00",
            "title": "Unsupervised Iterative Deep Learning of Speech Features and Acoustic Tokens with Applications to Spoken Term Detection",
            "authors": [
                "Cheng-Tao Chung",
                "Cheng-Yu Tsai",
                "Chia-Hsiang Liu",
                "Lin-Shan Lee"
            ],
            "abstract": "In this paper we aim to automatically discover high quality frame-level\nspeech features and acoustic tokens directly from unlabeled speech data. A\nMulti-granular Acoustic Tokenizer (MAT) was proposed for automatic discovery of\nmultiple sets of acoustic tokens from the given corpus. Each acoustic token set\nis specified by a set of hyperparameters describing the model configuration.\nThese different sets of acoustic tokens carry different characteristics for the\ngiven corpus and the language behind, thus can be mutually reinforced. The\nmultiple sets of token labels are then used as the targets of a Multi-target\nDeep Neural Network (MDNN) trained on frame-level acoustic features. Bottleneck\nfeatures extracted from the MDNN are then used as the feedback input to the MAT\nand the MDNN itself in the next iteration. The multi-granular acoustic token\nsets and the frame-level speech features can be iteratively optimized in the\niterative deep learning framework. We call this framework the Multi-granular\nAcoustic Tokenizing Deep Neural Network (MATDNN). The results were evaluated\nusing the metrics and corpora defined in the Zero Resource Speech Challenge\norganized at Interspeech 2015, and improved performance was obtained with a set\nof experiments of query-by-example spoken term detection on the same corpora.\nVisualization for the discovered tokens against the English phonemes was also\nshown.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05436v1",
            "date": "2017-07-18 01:53:58+00:00",
            "title": "Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder",
            "authors": [
                "Huadong Chen",
                "Shujian Huang",
                "David Chiang",
                "Jiajun Chen"
            ],
            "abstract": "Most neural machine translation (NMT) models are based on the sequential\nencoder-decoder framework, which makes no use of syntactic information. In this\npaper, we improve this model by explicitly incorporating source-side syntactic\ntrees. More specifically, we propose (1) a bidirectional tree encoder which\nlearns both sequential and tree structured representations; (2) a tree-coverage\nmodel that lets the attention depend on the source-side syntax. Experiments on\nChinese-English translation demonstrate that our proposed models outperform the\nsequential attentional model as well as a stronger baseline with a bottom-up\ntree encoder and word coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05438v1",
            "date": "2017-07-18 02:04:37+00:00",
            "title": "Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation",
            "authors": [
                "Huadong Chen",
                "Shujian Huang",
                "David Chiang",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Pairwise ranking methods are the basis of many widely used discriminative\ntraining approaches for structure prediction problems in natural language\nprocessing(NLP). Decomposing the problem of ranking hypotheses into pairwise\ncomparisons enables simple and efficient solutions. However, neglecting the\nglobal ordering of the hypothesis list may hinder learning. We propose a\nlistwise learning framework for structure prediction problems such as machine\ntranslation. Our framework directly models the entire translation list's\nordering to learn parameters which may better fit the given listwise samples.\nFurthermore, we propose top-rank enhanced loss functions, which are more\nsensitive to ranking errors at higher positions. Experiments on a large-scale\nChinese-English translation task show that both our listwise learning framework\nand top-rank enhanced listwise losses lead to significant improvements in\ntranslation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05468v1",
            "date": "2017-07-18 05:04:03+00:00",
            "title": "Detecting Intentional Lexical Ambiguity in English Puns",
            "authors": [
                "Elena Mikhalkova",
                "Yuri Karyakin"
            ],
            "abstract": "The article describes a model of automatic analysis of puns, where a word is\nintentionally used in two meanings at the same time (the target word). We\nemploy Roget's Thesaurus to discover two groups of words which, in a pun, form\naround two abstract bits of meaning (semes). They become a semantic vector,\nbased on which an SVM classifier learns to recognize puns, reaching a score\n0.73 for F-measure. We apply several rule-based methods to locate intentionally\nambiguous (target) words, based on structural and semantic criteria. It appears\nthat the structural criterion is more effective, although it possibly\ncharacterizes only the tested dataset. The results we get correlate with the\nresults of other teams at SemEval-2017 competition (Task 7 Detection and\nInterpretation of English Puns) considering effects of using supervised\nlearning models and word statistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05479v1",
            "date": "2017-07-18 05:40:18+00:00",
            "title": "PunFields at SemEval-2017 Task 7: Employing Roget's Thesaurus in Automatic Pun Recognition and Interpretation",
            "authors": [
                "Elena Mikhalkova",
                "Yuri Karyakin"
            ],
            "abstract": "The article describes a model of automatic interpretation of English puns,\nbased on Roget's Thesaurus, and its implementation, PunFields. In a pun, the\nalgorithm discovers two groups of words that belong to two main semantic\nfields. The fields become a semantic vector based on which an SVM classifier\nlearns to recognize puns. A rule-based model is then applied for recognition of\nintentionally ambiguous (target) words and their definitions. In SemEval Task 7\nPunFields shows a considerably good result in pun classification, but requires\nimprovement in searching for the target word and its definition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05501v2",
            "date": "2017-07-18 07:08:31+00:00",
            "title": "Story Generation from Sequence of Independent Short Descriptions",
            "authors": [
                "Parag Jain",
                "Priyanka Agrawal",
                "Abhijit Mishra",
                "Mohak Sukhwani",
                "Anirban Laha",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "Existing Natural Language Generation (NLG) systems are weak AI systems and\nexhibit limited capabilities when language generation tasks demand higher\nlevels of creativity, originality and brevity. Effective solutions or, at least\nevaluations of modern NLG paradigms for such creative tasks have been elusive,\nunfortunately. This paper introduces and addresses the task of coherent story\ngeneration from independent descriptions, describing a scene or an event.\nTowards this, we explore along two popular text-generation paradigms -- (1)\nStatistical Machine Translation (SMT), posing story generation as a translation\nproblem and (2) Deep Learning, posing story generation as a sequence to\nsequence learning problem. In SMT, we chose two popular methods such as phrase\nbased SMT (PB-SMT) and syntax based SMT (SYNTAX-SMT) to `translate' the\nincoherent input text into stories. We then implement a deep recurrent neural\nnetwork (RNN) architecture that encodes sequence of variable length input\ndescriptions to corresponding latent representations and decodes them to\nproduce well formed comprehensive story like summaries. The efficacy of the\nsuggested approaches is demonstrated on a publicly available dataset with the\nhelp of popular machine translation and summarization evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05589v2",
            "date": "2017-07-18 12:35:53+00:00",
            "title": "On the State of the Art of Evaluation in Neural Language Models",
            "authors": [
                "G\u00e1bor Melis",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "Ongoing innovations in recurrent neural network architectures have provided a\nsteady influx of apparently state-of-the-art results on language modelling\nbenchmarks. However, these have been evaluated using differing code bases and\nlimited computational resources, which represent uncontrolled sources of\nexperimental variation. We reevaluate several popular architectures and\nregularisation methods with large-scale automatic black-box hyperparameter\ntuning and arrive at the somewhat surprising conclusion that standard LSTM\narchitectures, when properly regularised, outperform more recent models. We\nestablish a new state of the art on the Penn Treebank and Wikitext-2 corpora,\nas well as strong baselines on the Hutter Prize dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05635v1",
            "date": "2017-07-18 14:19:50+00:00",
            "title": "Spherical Paragraph Model",
            "authors": [
                "Ruqing Zhang",
                "Jiafeng Guo",
                "Yanyan Lan",
                "Jun Xu",
                "Xueqi Cheng"
            ],
            "abstract": "Representing texts as fixed-length vectors is central to many language\nprocessing tasks. Most traditional methods build text representations based on\nthe simple Bag-of-Words (BoW) representation, which loses the rich semantic\nrelations between words. Recent advances in natural language processing have\nshown that semantically meaningful representations of words can be efficiently\nacquired by distributed models, making it possible to build text\nrepresentations based on a better foundation called the Bag-of-Word-Embedding\n(BoWE) representation. However, existing text representation methods using BoWE\noften lack sound probabilistic foundations or cannot well capture the semantic\nrelatedness encoded in word vectors. To address these problems, we introduce\nthe Spherical Paragraph Model (SPM), a probabilistic generative model based on\nBoWE, for text representation. SPM has good probabilistic interpretability and\ncan fully leverage the rich semantics of words, the word co-occurrence\ninformation as well as the corpus-wide information to help the representation\nlearning of texts. Experimental results on topical classification and sentiment\nanalysis demonstrate that SPM can achieve new state-of-the-art performances on\nseveral benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05850v3",
            "date": "2017-07-18 20:38:42+00:00",
            "title": "A Short Survey of Biomedical Relation Extraction Techniques",
            "authors": [
                "Elham Shahab"
            ],
            "abstract": "Biomedical information is growing rapidly in the recent years and retrieving\nuseful data through information extraction system is getting more attention. In\nthe current research, we focus on different aspects of relation extraction\ntechniques in biomedical domain and briefly describe the state-of-the-art for\nrelation extraction between a variety of biological elements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05853v2",
            "date": "2017-07-18 20:47:06+00:00",
            "title": "Encoding Word Confusion Networks with Recurrent Neural Networks for Dialog State Tracking",
            "authors": [
                "Glorianna Jagfeld",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper presents our novel method to encode word confusion networks, which\ncan represent a rich hypothesis space of automatic speech recognition systems,\nvia recurrent neural networks. We demonstrate the utility of our approach for\nthe task of dialog state tracking in spoken dialog systems that relies on\nautomatic speech recognition output. Encoding confusion networks outperforms\nencoding the best hypothesis of the automatic speech recognition in a neural\nsystem for dialog state tracking on the well-known second Dialog State Tracking\nChallenge dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05928v3",
            "date": "2017-07-19 03:18:40+00:00",
            "title": "Deep Active Learning for Named Entity Recognition",
            "authors": [
                "Yanyao Shen",
                "Hyokun Yun",
                "Zachary C. Lipton",
                "Yakov Kronrod",
                "Animashree Anandkumar"
            ],
            "abstract": "Deep learning has yielded state-of-the-art performance on many natural\nlanguage processing tasks including named entity recognition (NER). However,\nthis typically requires large amounts of labeled data. In this work, we\ndemonstrate that the amount of labeled training data can be drastically reduced\nwhen deep learning is combined with active learning. While active learning is\nsample-efficient, it can be computationally expensive since it requires\niterative retraining. To speed this up, we introduce a lightweight architecture\nfor NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and\nword encoders and a long short term memory (LSTM) tag decoder. The model\nachieves nearly state-of-the-art performance on standard datasets for the task\nwhile being computationally much more efficient than best performing models. We\ncarry out incremental active learning, during the training process, and are\nable to nearly match state-of-the-art performance with just 25\\% of the\noriginal training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05967v2",
            "date": "2017-07-19 07:51:05+00:00",
            "title": "Measuring Thematic Fit with Distributional Feature Overlap",
            "authors": [
                "Enrico Santus",
                "Emmanuele Chersoni",
                "Alessandro Lenci",
                "Philippe Blache"
            ],
            "abstract": "In this paper, we introduce a new distributional method for modeling\npredicate-argument thematic fit judgments. We use a syntax-based DSM to build a\nprototypical representation of verb-specific roles: for every verb, we extract\nthe most salient second order contexts for each of its roles (i.e. the most\nsalient dimensions of typical role fillers), and then we compute thematic fit\nas a weighted overlap between the top features of candidate fillers and role\nprototypes. Our experiments show that our method consistently outperforms a\nbaseline re-implementing a state-of-the-art system, and achieves better or\ncomparable results to those reported in the literature for the other\nunsupervised systems. Moreover, it provides an explicit representation of the\nfeatures characterizing verb-specific semantic roles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06002v1",
            "date": "2017-07-19 10:08:51+00:00",
            "title": "Argotario: Computational Argumentation Meets Serious Games",
            "authors": [
                "Ivan Habernal",
                "Raffael Hannemann",
                "Christian Pollak",
                "Christopher Klamm",
                "Patrick Pauli",
                "Iryna Gurevych"
            ],
            "abstract": "An important skill in critical thinking and argumentation is the ability to\nspot and recognize fallacies. Fallacious arguments, omnipresent in\nargumentative discourse, can be deceptive, manipulative, or simply leading to\n`wrong moves' in a discussion. Despite their importance, argumentation scholars\nand NLP researchers with focus on argumentation quality have not yet\ninvestigated fallacies empirically. The nonexistence of resources dealing with\nfallacious argumentation calls for scalable approaches to data acquisition and\nannotation, for which the serious games methodology offers an appealing, yet\nunexplored, alternative. We present Argotario, a serious game that deals with\nfallacies in everyday argumentation. Argotario is a multilingual, open-source,\nplatform-independent application with strong educational aspects, accessible at\nwww.argotario.net.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06012v2",
            "date": "2017-07-19 10:47:22+00:00",
            "title": "Modeling Target-Side Inflection in Neural Machine Translation",
            "authors": [
                "Ale\u0161 Tamchyna",
                "Marion Weller-Di Marco",
                "Alexander Fraser"
            ],
            "abstract": "NMT systems have problems with large vocabulary sizes. Byte-pair encoding\n(BPE) is a popular approach to solving this problem, but while BPE allows the\nsystem to generate any target-side word, it does not enable effective\ngeneralization over the rich vocabulary in morphologically rich languages with\nstrong inflectional phenomena. We introduce a simple approach to overcome this\nproblem by training a system to produce the lemma of a word and its\nmorphologically rich POS tag, which is then followed by a deterministic\ngeneration step. We apply this strategy for English-Czech and English-German\ntranslation scenarios, obtaining improvements in both settings. We furthermore\nshow that the improvement is not due to only adding explicit morphological\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06100v1",
            "date": "2017-07-18 15:46:47+00:00",
            "title": "Discovering topics in text datasets by visualizing relevant words",
            "authors": [
                "Franziska Horn",
                "Leila Arras",
                "Gr\u00e9goire Montavon",
                "Klaus-Robert M\u00fcller",
                "Wojciech Samek"
            ],
            "abstract": "When dealing with large collections of documents, it is imperative to quickly\nget an overview of the texts' contents. In this paper we show how this can be\nachieved by using a clustering algorithm to identify topics in the dataset and\nthen selecting and visualizing relevant words, which distinguish a group of\ndocuments from the rest of the texts, to summarize the contents of the\ndocuments belonging to each topic. We demonstrate our approach by discovering\ntrending topics in a collection of New York Times article snippets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06130v1",
            "date": "2017-07-19 14:49:35+00:00",
            "title": "Improving Language Modeling using Densely Connected Recurrent Neural Networks",
            "authors": [
                "Fr\u00e9deric Godin",
                "Joni Dambre",
                "Wesley De Neve"
            ],
            "abstract": "In this paper, we introduce the novel concept of densely connected layers\ninto recurrent neural networks. We evaluate our proposed architecture on the\nPenn Treebank language modeling task. We show that we can obtain similar\nperplexity scores with six times fewer parameters compared to a standard\nstacked 2-layer LSTM model trained with dropout (Zaremba et al. 2014). In\ncontrast with the current usage of skip connections, we show that densely\nconnecting only a few stacked layers with skip connections already yields\nsignificant perplexity reductions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06151v1",
            "date": "2017-07-19 15:24:20+00:00",
            "title": "Expect the unexpected: Harnessing Sentence Completion for Sarcasm Detection",
            "authors": [
                "Aditya Joshi",
                "Samarth Agrawal",
                "Pushpak Bhattacharyya",
                "Mark Carman"
            ],
            "abstract": "The trigram `I love being' is expected to be followed by positive words such\nas `happy'. In a sarcastic sentence, however, the word `ignored' may be\nobserved. The expected and the observed words are, thus, incongruous. We model\nsarcasm detection as the task of detecting incongruity between an observed and\nan expected word. In order to obtain the expected word, we use Context2Vec, a\nsentence completion library based on Bidirectional LSTM. However, since the\nexact word where such an incongruity occurs may not be known in advance, we\npresent two approaches: an All-words approach (which consults sentence\ncompletion for every content word) and an Incongruous words-only approach\n(which consults sentence completion for the 50% most incongruous content\nwords). The approaches outperform reported values for tweets but not for\ndiscussion forum posts. This is likely to be because of redundant consultation\nof sentence completion for discussion forum posts. Therefore, we consider an\noracle case where the exact incongruous word is manually labeled in a corpus\nreported in past work. In this case, the performance is higher than the\nall-words approach. This sets up the promise for using sentence completion for\nsarcasm detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06167v1",
            "date": "2017-07-19 15:48:27+00:00",
            "title": "Sentence-level quality estimation by predicting HTER as a multi-component metric",
            "authors": [
                "Eleftherios Avramidis"
            ],
            "abstract": "This submission investigates alternative machine learning models for\npredicting the HTER score on the sentence level. Instead of directly predicting\nthe HTER score, we suggest a model that jointly predicts the amount of the 4\ndistinct post-editing operations, which are then used to calculate the HTER\nscore. This also gives the possibility to correct invalid (e.g. negative)\npredicted values prior to the calculation of the HTER score. Without any\nfeature exploration, a multi-layer perceptron with 4 outputs yields small but\nsignificant improvements over the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06195v1",
            "date": "2017-07-19 17:03:34+00:00",
            "title": "Fast and Accurate OOV Decoder on High-Level Features",
            "authors": [
                "Yuri Khokhlov",
                "Natalia Tomashenko",
                "Ivan Medennikov",
                "Alexei Romanenko"
            ],
            "abstract": "This work proposes a novel approach to out-of-vocabulary (OOV) keyword search\n(KWS) task. The proposed approach is based on using high-level features from an\nautomatic speech recognition (ASR) system, so called phoneme posterior based\n(PPB) features, for decoding. These features are obtained by calculating\ntime-dependent phoneme posterior probabilities from word lattices, followed by\ntheir smoothing. For the PPB features we developed a special novel very fast,\nsimple and efficient OOV decoder. Experimental results are presented on the\nGeorgian language from the IARPA Babel Program, which was the test language in\nthe OpenKWS 2016 evaluation campaign. The results show that in terms of maximum\nterm weighted value (MTWV) metric and computational speed, for single ASR\nsystems, the proposed approach significantly outperforms the state-of-the-art\napproach based on using in-vocabulary proxies for OOV keywords in the indexed\ndatabase. The comparison of the two OOV KWS approaches on the fusion results of\nthe nine different ASR systems demonstrates that the proposed OOV decoder\noutperforms the proxy-based approach in terms of MTWV metric given the\ncomparable processing speed. Other important advantages of the OOV decoder\ninclude extremely low memory consumption and simplicity of its implementation\nand parameter optimization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06341v2",
            "date": "2017-07-20 02:09:23+00:00",
            "title": "A Sub-Character Architecture for Korean Language Processing",
            "authors": [
                "Karl Stratos"
            ],
            "abstract": "We introduce a novel sub-character architecture that exploits a unique\ncompositional structure of the Korean language. Our method decomposes each\ncharacter into a small set of primitive phonetic units called jamo letters from\nwhich character- and word-level representations are induced. The jamo letters\ndivulge syntactic and semantic information that is difficult to access with\nconventional character-level units. They greatly alleviate the data sparsity\nproblem, reducing the observation space to 1.6% of the original while\nincreasing accuracy in our experiments. We apply our architecture to dependency\nparsing and achieve dramatic improvement over strong lexical baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06357v1",
            "date": "2017-07-20 03:17:19+00:00",
            "title": "Improving Discourse Relation Projection to Build Discourse Annotated Corpora",
            "authors": [
                "Majid Laali",
                "Leila Kosseim"
            ],
            "abstract": "The naive approach to annotation projection is not effective to project\ndiscourse annotations from one language to another because implicit discourse\nrelations are often changed to explicit ones and vice-versa in the translation.\nIn this paper, we propose a novel approach based on the intersection between\nstatistical word-alignment models to identify unsupported discourse\nannotations. This approach identified 65% of the unsupported annotations in the\nEnglish-French parallel sentences from Europarl. By filtering out these\nunsupported annotations, we induced the first PDTB-style discourse annotated\ncorpus for French from Europarl. We then used this corpus to train a classifier\nto identify the discourse-usage of French discourse connectives and show a 15%\nimprovement of F1-score compared to the classifier trained on the non-filtered\nannotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06378v1",
            "date": "2017-07-20 05:40:37+00:00",
            "title": "Large-Scale Goodness Polarity Lexicons for Community Question Answering",
            "authors": [
                "Todor Mihaylov",
                "Daniel Belchev",
                "Yasen Kiprov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "We transfer a key idea from the field of sentiment analysis to a new domain:\ncommunity question answering (cQA). The cQA task we are interested in is the\nfollowing: given a question and a thread of comments, we want to re-rank the\ncomments so that the ones that are good answers to the question would be ranked\nhigher than the bad ones. We notice that good vs. bad comments use specific\nvocabulary and that one can often predict the goodness/badness of a comment\neven ignoring the question, based on the comment contents only. This leads us\nto the idea to build a good/bad polarity lexicon as an analogy to the\npositive/negative sentiment polarity lexicons, commonly used in sentiment\nanalysis. In particular, we use pointwise mutual information in order to build\nlarge-scale goodness polarity lexicons in a semi-supervised manner starting\nwith a small number of initial seeds. The evaluation results show an\nimprovement of 0.7 MAP points absolute over a very strong baseline and\nstate-of-the art performance on SemEval-2016 Task 3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06456v1",
            "date": "2017-07-20 11:54:37+00:00",
            "title": "Revisiting Selectional Preferences for Coreference Resolution",
            "authors": [
                "Benjamin Heinzerling",
                "Nafise Sadat Moosavi",
                "Michael Strube"
            ],
            "abstract": "Selectional preferences have long been claimed to be essential for\ncoreference resolution. However, they are mainly modeled only implicitly by\ncurrent coreference resolvers. We propose a dependency-based embedding model of\nselectional preferences which allows fine-grained compatibility judgments with\nhigh coverage. We show that the incorporation of our model improves coreference\nresolution performance on the CoNLL dataset, matching the state-of-the-art\nresults of a more complex system. However, it comes with a cost that makes it\ndebatable how worthwhile such improvements are.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06799v2",
            "date": "2017-07-21 08:36:31+00:00",
            "title": "Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "Selecting optimal parameters for a neural network architecture can often make\nthe difference between mediocre and state-of-the-art performance. However,\nlittle is published which parameters and design choices should be evaluated or\nselected making the correct hyperparameter optimization often a \"black art that\nrequires expert experiences\" (Snoek et al., 2012). In this paper, we evaluate\nthe importance of different network design choices and hyperparameters for five\ncommon linguistic sequence tagging tasks (POS, Chunking, NER, Entity\nRecognition, and Event Detection). We evaluated over 50.000 different setups\nand found, that some parameters, like the pre-trained word embeddings or the\nlast layer of the network, have a large impact on the performance, while other\nparameters, for example the number of LSTM layers or the number of recurrent\nunits, are of minor importance. We give a recommendation on a configuration\nthat performs well among different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06806v1",
            "date": "2017-07-21 09:02:55+00:00",
            "title": "Shallow reading with Deep Learning: Predicting popularity of online content using only its title",
            "authors": [
                "Wociech Stokowiec",
                "Tomasz Trzcinski",
                "Krzysztof Wolk",
                "Krzysztof Marasek",
                "Przemyslaw Rokita"
            ],
            "abstract": "With the ever decreasing attention span of contemporary Internet users, the\ntitle of online content (such as a news article or video) can be a major factor\nin determining its popularity. To take advantage of this phenomenon, we propose\na new method based on a bidirectional Long Short-Term Memory (LSTM) neural\nnetwork designed to predict the popularity of online content using only its\ntitle. We evaluate the proposed architecture on two distinct datasets of news\narticles and news videos distributed in social media that contain over 40,000\nsamples in total. On those datasets, our approach improves the performance over\ntraditional shallow approaches by a margin of 15%. Additionally, we show that\nusing pre-trained word vectors in the embedding layer improves the results of\nLSTM models, especially when the training set is small. To our knowledge, this\nis the first attempt of applying popularity prediction using only textual\ninformation from the title.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06875v1",
            "date": "2017-07-21 12:47:03+00:00",
            "title": "Why We Need New Evaluation Metrics for NLG",
            "authors": [
                "Jekaterina Novikova",
                "Ond\u0159ej Du\u0161ek",
                "Amanda Cercas Curry",
                "Verena Rieser"
            ],
            "abstract": "The majority of NLG evaluation relies on automatic metrics, such as BLEU . In\nthis paper, we motivate the need for novel, system- and data-independent\nautomatic evaluation methods: We investigate a wide range of metrics, including\nstate-of-the-art word-based and novel grammar-based ones, and demonstrate that\nthey only weakly reflect human judgements of system outputs as generated by\ndata-driven, end-to-end NLG. We also show that metric performance is data- and\nsystem-specific. Nevertheless, our results also suggest that automatic metrics\nperform reliably at system-level and can support system development by finding\ncases where a system performs poorly.",
            "journal ref": "Proceedings of the 2017 Conference on Empirical Methods in Natural\n  Language Processing, pages 2231-2242, Copenhagen, Denmark, September 7-11,\n  2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06878v1",
            "date": "2017-07-21 12:56:06+00:00",
            "title": "Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation",
            "authors": [
                "Alexander Panchenko",
                "Fide Marten",
                "Eugen Ruppert",
                "Stefano Faralli",
                "Dmitry Ustalov",
                "Simone Paolo Ponzetto",
                "Chris Biemann"
            ],
            "abstract": "Interpretability of a predictive model is a powerful feature that gains the\ntrust of users in the correctness of the predictions. In word sense\ndisambiguation (WSD), knowledge-based systems tend to be much more\ninterpretable than knowledge-free counterparts as they rely on the wealth of\nmanually-encoded elements representing word senses, such as hypernyms, usage\nexamples, and images. We present a WSD system that bridges the gap between\nthese two so far disconnected groups of methods. Namely, our system, providing\naccess to several state-of-the-art WSD models, aims to be interpretable as a\nknowledge-based system while it remains completely unsupervised and\nknowledge-free. The presented tool features a Web interface for all-word\ndisambiguation of texts that makes the sense predictions human readable by\nproviding interpretable word sense inventories, sense representations, and\ndisambiguation results. We provide a public API, enabling seamless integration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06885v1",
            "date": "2017-07-21 13:14:25+00:00",
            "title": "SGNMT -- A Flexible NMT Decoding Platform for Quick Prototyping of New Models and Search Strategies",
            "authors": [
                "Felix Stahlberg",
                "Eva Hasler",
                "Danielle Saunders",
                "Bill Byrne"
            ],
            "abstract": "This paper introduces SGNMT, our experimental platform for machine\ntranslation research. SGNMT provides a generic interface to neural and symbolic\nscoring modules (predictors) with left-to-right semantic such as translation\nmodels like NMT, language models, translation lattices, $n$-best lists or other\nkinds of scores and constraints. Predictors can be combined with other\npredictors to form complex decoding tasks. SGNMT implements a number of search\nstrategies for traversing the space spanned by the predictors which are\nappropriate for different predictor constellations. Adding new predictors or\ndecoding strategies is particularly easy, making it a very efficient tool for\nprototyping new research ideas. SGNMT is actively being used by students in the\nMPhil program in Machine Learning, Speech and Language Technology at the\nUniversity of Cambridge for course work and theses, as well as for most of the\nresearch work in our group.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06945v1",
            "date": "2017-07-21 15:52:54+00:00",
            "title": "Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation",
            "authors": [
                "Ivan Vuli\u0107",
                "Nikola Mrk\u0161i\u0107",
                "Anna Korhonen"
            ],
            "abstract": "Existing approaches to automatic VerbNet-style verb classification are\nheavily dependent on feature engineering and therefore limited to languages\nwith mature NLP pipelines. In this work, we propose a novel cross-lingual\ntransfer method for inducing VerbNets for multiple languages. To the best of\nour knowledge, this is the first study which demonstrates how the architectures\nfor learning word embeddings can be applied to this challenging\nsyntactic-semantic task. Our method uses cross-lingual translation pairs to tie\neach of the six target languages into a bilingual vector space with English,\njointly specialising the representations to encode the relational information\nfrom English VerbNet. A standard clustering algorithm is then run on top of the\nVerbNet-specialised representations, using vector dimensions as features for\nlearning verb classes. Our results show that the proposed cross-lingual\ntransfer approach sets new state-of-the-art verb classification performance\nacross all six target languages explored in this work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06957v1",
            "date": "2017-07-21 16:10:51+00:00",
            "title": "Reconstruction of Word Embeddings from Sub-Word Parameters",
            "authors": [
                "Karl Stratos"
            ],
            "abstract": "Pre-trained word embeddings improve the performance of a neural model at the\ncost of increasing the model size. We propose to benefit from this resource\nwithout paying the cost by operating strictly at the sub-lexical level. Our\napproach is quite simple: before task-specific training, we first optimize\nsub-word parameters to reconstruct pre-trained word embeddings using various\ndistance measures. We report interesting results on a variety of tasks: word\nsimilarity, word analogy, and part-of-speech tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06961v1",
            "date": "2017-07-21 16:18:10+00:00",
            "title": "Mimicking Word Embeddings using Subword RNNs",
            "authors": [
                "Yuval Pinter",
                "Robert Guthrie",
                "Jacob Eisenstein"
            ],
            "abstract": "Word embeddings improve generalization over lexical features by placing each\nword in a lower-dimensional space, using distributional information obtained\nfrom unlabeled data. However, the effectiveness of word embeddings for\ndownstream NLP tasks is limited by out-of-vocabulary (OOV) words, for which\nembeddings do not exist. In this paper, we present MIMICK, an approach to\ngenerating OOV word embeddings compositionally, by learning a function from\nspellings to distributional embeddings. Unlike prior work, MIMICK does not\nrequire re-training on the original word embedding corpus; instead, learning is\nperformed at the type level. Intrinsic and extrinsic evaluations demonstrate\nthe power of this simple approach. On 23 languages, MIMICK improves performance\nover a word-based baseline for tagging part-of-speech and morphosyntactic\nattributes. It is competitive with (and complementary to) a supervised\ncharacter-based model in low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06971v1",
            "date": "2017-07-21 16:47:56+00:00",
            "title": "Split and Rephrase",
            "authors": [
                "Shashi Narayan",
                "Claire Gardent",
                "Shay B. Cohen",
                "Anastasia Shimorina"
            ],
            "abstract": "We propose a new sentence simplification task (Split-and-Rephrase) where the\naim is to split a complex sentence into a meaning preserving sequence of\nshorter sentences. Like sentence simplification, splitting-and-rephrasing has\nthe potential of benefiting both natural language processing and societal\napplications. Because shorter sentences are generally better processed by NLP\nsystems, it could be used as a preprocessing step which facilitates and\nimproves the performance of parsers, semantic role labellers and machine\ntranslation systems. It should also be of use for people with reading\ndisabilities because it allows the conversion of longer sentences into shorter\nones. This paper makes two contributions towards this new task. First, we\ncreate and make available a benchmark consisting of 1,066,115 tuples mapping a\nsingle complex sentence to a sequence of sentences expressing the same meaning.\nSecond, we propose five models (vanilla sequence-to-sequence to\nsemantically-motivated models) to understand the difficulty of the proposed\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06996v4",
            "date": "2017-07-21 11:52:45+00:00",
            "title": "A Sentiment-and-Semantics-Based Approach for Emotion Detection in Textual Conversations",
            "authors": [
                "Umang Gupta",
                "Ankush Chatterjee",
                "Radhakrishnan Srikanth",
                "Puneet Agrawal"
            ],
            "abstract": "Emotions are physiological states generated in humans in reaction to internal\nor external events. They are complex and studied across numerous fields\nincluding computer science. As humans, on reading \"Why don't you ever text me!\"\nwe can either interpret it as a sad or angry emotion and the same ambiguity\nexists for machines. Lack of facial expressions and voice modulations make\ndetecting emotions from text a challenging problem. However, as humans\nincreasingly communicate using text messaging applications, and digital agents\ngain popularity in our society, it is essential that these digital agents are\nemotion aware, and respond accordingly.\n  In this paper, we propose a novel approach to detect emotions like happy, sad\nor angry in textual conversations using an LSTM based Deep Learning model. Our\napproach consists of semi-automated techniques to gather training data for our\nmodel. We exploit advantages of semantic and sentiment based embeddings and\npropose a solution combining both. Our work is evaluated on real-world\nconversations and significantly outperforms traditional Machine Learning\nbaselines as well as other off-the-shelf Deep Learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07045v2",
            "date": "2017-07-21 21:05:04+00:00",
            "title": "End-to-end Neural Coreference Resolution",
            "authors": [
                "Kenton Lee",
                "Luheng He",
                "Mike Lewis",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce the first end-to-end coreference resolution model and show that\nit significantly outperforms all previous work without using a syntactic parser\nor hand-engineered mention detector. The key idea is to directly consider all\nspans in a document as potential mentions and learn distributions over possible\nantecedents for each. The model computes span embeddings that combine\ncontext-dependent boundary representations with a head-finding attention\nmechanism. It is trained to maximize the marginal likelihood of gold antecedent\nspans from coreference clusters and is factored to enable aggressive pruning of\npotential mentions. Experiments demonstrate state-of-the-art performance, with\na gain of 1.5 F1 on the OntoNotes benchmark and by 3.1 F1 using a 5-model\nensemble, despite the fact that this is the first approach to be successfully\ntrained with no external resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07062v1",
            "date": "2017-07-21 22:42:52+00:00",
            "title": "A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization",
            "authors": [
                "Xinyu Hua",
                "Lu Wang"
            ],
            "abstract": "We study the problem of domain adaptation for neural abstractive\nsummarization. We make initial efforts in investigating what information can be\ntransferred to a new domain. Experimental results on news stories and opinion\narticles indicate that neural summarization model benefits from pre-training\nbased on extractive summaries. We also find that the combination of in-domain\nand out-of-domain setup yields better summaries when in-domain data is\ninsufficient. Further analysis shows that, the model is capable to select\nsalient content even trained on out-of-domain data, but requires in-domain data\nto capture the style for a target domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07086v1",
            "date": "2017-07-22 01:47:36+00:00",
            "title": "Identifying civilians killed by police with distantly supervised entity-event extraction",
            "authors": [
                "Katherine A. Keith",
                "Abram Handler",
                "Michael Pinkham",
                "Cara Magliozzi",
                "Joshua McDuffie",
                "Brendan O'Connor"
            ],
            "abstract": "We propose a new, socially-impactful task for natural language processing:\nfrom a news corpus, extract names of persons who have been killed by police. We\npresent a newly collected police fatality corpus, which we release publicly,\nand present a model to solve this problem that uses EM-based distant\nsupervision with logistic regression and convolutional neural network\nclassifiers. Our model outperforms two off-the-shelf event extractor systems,\nand it can suggest candidate victim names in some cases faster than one of the\nmajor manually-collected police fatality databases.",
            "journal ref": "Proceedings of the 2017 Conference on Empirical Methods in Natural\n  Language Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07129v2",
            "date": "2017-07-22 09:35:10+00:00",
            "title": "Predicting the Gender of Indonesian Names",
            "authors": [
                "Ali Akbar Septiandri"
            ],
            "abstract": "We investigated a way to predict the gender of a name using character-level\nLong-Short Term Memory (char-LSTM). We compared our method with some\nconventional machine learning methods, namely Naive Bayes, logistic regression,\nand XGBoost with n-grams as the features. We evaluated the models on a dataset\nconsisting of the names of Indonesian people. It is not common to use a family\nname as the surname in Indonesian culture, except in some ethnicities.\nTherefore, we inferred the gender from both full names and first names. The\nresults show that we can achieve 92.25% accuracy from full names, while using\nfirst names only yields 90.65% accuracy. These results are better than the ones\nfrom applying the classical machine learning algorithms to n-grams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07182v1",
            "date": "2017-07-22 16:04:49+00:00",
            "title": "Native Language Identification on Text and Speech",
            "authors": [
                "Marcos Zampieri",
                "Alina Maria Ciobanu",
                "Liviu P. Dinu"
            ],
            "abstract": "This paper presents an ensemble system combining the output of multiple SVM\nclassifiers to native language identification (NLI). The system was submitted\nto the NLI Shared Task 2017 fusion track which featured students essays and\nspoken responses in form of audio transcriptions and iVectors by non-native\nEnglish speakers of eleven native languages. Our system competed in the\nchallenge under the team name ZCD and was based on an ensemble of SVM\nclassifiers trained on character n-grams achieving 83.58% accuracy and ranking\n3rd in the shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07212v3",
            "date": "2017-07-22 20:34:53+00:00",
            "title": "\"i have a feeling trump will win..................\": Forecasting Winners and Losers from User Predictions on Twitter",
            "authors": [
                "Sandesh Swamy",
                "Alan Ritter",
                "Marie-Catherine de Marneffe"
            ],
            "abstract": "Social media users often make explicit predictions about upcoming events.\nSuch statements vary in the degree of certainty the author expresses toward the\noutcome:\"Leonardo DiCaprio will win Best Actor\" vs. \"Leonardo DiCaprio may win\"\nor \"No way Leonardo wins!\". Can popular beliefs on social media predict who\nwill win? To answer this question, we build a corpus of tweets annotated for\nveridicality on which we train a log-linear classifier that detects positive\nveridicality with high precision. We then forecast uncertain outcomes using the\nwisdom of crowds, by aggregating users' explicit predictions. Our method for\nforecasting winners is fully automated, relying only on a set of contenders as\ninput. It requires no training data of past outcomes and outperforms sentiment\nand tweet volume baselines on a broad range of contest prediction tasks. We\nfurther demonstrate how our approach can be used to measure the reliability of\nindividual accounts' predictions and retrospectively identify surprise\noutcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07250v1",
            "date": "2017-07-23 05:54:20+00:00",
            "title": "Tensor Fusion Network for Multimodal Sentiment Analysis",
            "authors": [
                "Amir Zadeh",
                "Minghai Chen",
                "Soujanya Poria",
                "Erik Cambria",
                "Louis-Philippe Morency"
            ],
            "abstract": "Multimodal sentiment analysis is an increasingly popular research area, which\nextends the conventional language-based definition of sentiment analysis to a\nmultimodal setup where other relevant modalities accompany language. In this\npaper, we pose the problem of multimodal sentiment analysis as modeling\nintra-modality and inter-modality dynamics. We introduce a novel model, termed\nTensor Fusion Network, which learns both such dynamics end-to-end. The proposed\napproach is tailored for the volatile nature of spoken language in online\nvideos as well as accompanying gestures and voice. In the experiments, our\nmodel outperforms state-of-the-art approaches for both multimodal and unimodal\nsentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07265v1",
            "date": "2017-07-23 08:12:59+00:00",
            "title": "Composing Distributed Representations of Relational Patterns",
            "authors": [
                "Sho Takase",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "Learning distributed representations for relation instances is a central\ntechnique in downstream NLP applications. In order to address semantic modeling\nof relational patterns, this paper constructs a new dataset that provides\nmultiple similarity ratings for every pair of relational patterns on the\nexisting dataset. In addition, we conduct a comparative study of different\nencoders including additive composition, RNN, LSTM, and GRU for composing\ndistributed representations of relational patterns. We also present Gated\nAdditive Composition, which is an enhancement of additive composition with the\ngating mechanism. Experiments show that the new dataset does not only enable\ndetailed analyses of the different encoders, but also provides a gauge to\npredict successes of distributed representations of relational patterns in the\nrelation classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07273v1",
            "date": "2017-07-23 09:55:48+00:00",
            "title": "Hierarchical Embeddings for Hypernymy Detection and Directionality",
            "authors": [
                "Kim Anh Nguyen",
                "Maximilian K\u00f6per",
                "Sabine Schulte im Walde",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present a novel neural model HyperVec to learn hierarchical embeddings for\nhypernymy detection and directionality. While previous embeddings have shown\nlimitations on prototypical hypernyms, HyperVec represents an unsupervised\nmeasure where embeddings are learned in a specific order and capture the\nhypernym$-$hyponym distributional hierarchy. Moreover, our model is able to\ngeneralize over unseen hypernymy pairs, when using only small sets of training\ndata, and by mapping to other languages. Results on benchmark datasets show\nthat HyperVec outperforms both state$-$of$-$the$-$art unsupervised measures and\nembedding models on hypernymy detection and directionality, and on predicting\ngraded lexical entailment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07278v1",
            "date": "2017-07-23 10:43:26+00:00",
            "title": "Fine Grained Citation Span for References in Wikipedia",
            "authors": [
                "Besnik Fetahu",
                "Katja Markert",
                "Avishek Anand"
            ],
            "abstract": "\\emph{Verifiability} is one of the core editing principles in Wikipedia,\neditors being encouraged to provide citations for the added content. For a\nWikipedia article, determining the \\emph{citation span} of a citation, i.e.\nwhat content is covered by a citation, is important as it helps decide for\nwhich content citations are still missing.\n  We are the first to address the problem of determining the \\emph{citation\nspan} in Wikipedia articles. We approach this problem by classifying which\ntextual fragments in an article are covered by a citation. We propose a\nsequence classification approach where for a paragraph and a citation, we\ndetermine the citation span at a fine-grained level.\n  We provide a thorough experimental evaluation and compare our approach\nagainst baselines adopted from the scientific domain, where we show improvement\nfor all evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07279v1",
            "date": "2017-07-23 10:51:30+00:00",
            "title": "Using Argument-based Features to Predict and Analyse Review Helpfulness",
            "authors": [
                "Haijing Liu",
                "Yang Gao",
                "Pin Lv",
                "Mengxue Li",
                "Shiqiang Geng",
                "Minglan Li",
                "Hao Wang"
            ],
            "abstract": "We study the helpful product reviews identification problem in this paper. We\nobserve that the evidence-conclusion discourse relations, also known as\narguments, often appear in product reviews, and we hypothesise that some\nargument-based features, e.g. the percentage of argumentative sentences, the\nevidences-conclusions ratios, are good indicators of helpful reviews. To\nvalidate this hypothesis, we manually annotate arguments in 110 hotel reviews,\nand investigate the effectiveness of several combinations of argument-based\nfeatures. Experiments suggest that, when being used together with the\nargument-based features, the state-of-the-art baseline features can enjoy a\nperformance boost (in terms of F1) of 11.01\\% in average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07331v1",
            "date": "2017-07-23 18:42:24+00:00",
            "title": "Rule-Based Spanish Morphological Analyzer Built From Spell Checking Lexicon",
            "authors": [
                "Natalie Ahn"
            ],
            "abstract": "Preprocessing tools for automated text analysis have become more widely\navailable in major languages, but non-English tools are often still limited in\ntheir functionality. When working with Spanish-language text, researchers can\neasily find tools for tokenization and stemming, but may not have the means to\nextract more complex word features like verb tense or mood. Yet Spanish is a\nmorphologically rich language in which such features are often identifiable\nfrom word form. Conjugation rules are consistent, but many special verbs and\nnouns take on different rules. While building a complete dictionary of known\nwords and their morphological rules would be labor intensive, resources to do\nso already exist, in spell checkers designed to generate valid forms of known\nwords. This paper introduces a set of tools for Spanish-language morphological\nanalysis, built using the COES spell checking tools, to label person, mood,\ntense, gender and number, derive a word's root noun or verb infinitive, and\nconvert verbs to their nominal form.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07343v1",
            "date": "2017-07-23 20:47:02+00:00",
            "title": "A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events",
            "authors": [
                "Prafulla Kumar Choubey",
                "Ruihong Huang"
            ],
            "abstract": "We present a sequential model for temporal relation classification between\nintra-sentence events. The key observation is that the overall syntactic\nstructure and compositional meanings of the multi-word context between events\nare important for distinguishing among fine-grained temporal relations.\nSpecifically, our approach first extracts a sequence of context words that\nindicates the temporal relation between two events, which well align with the\ndependency path between two event mentions. The context word sequence, together\nwith a parts-of-speech tag sequence and a dependency relation sequence that are\ngenerated corresponding to the word sequence, are then provided as input to\nbidirectional recurrent neural network (LSTM) models. The neural nets learn\ncompositional syntactic and semantic representations of contexts surrounding\nthe two events and predict the temporal relation between them. Evaluation of\nthe proposed approach on TimeBank corpus shows that sequential modeling is\ncapable of accurately recognizing temporal relations between events, which\noutperforms a neural net model using various discrete features as input that\nimitates previous feature based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07344v1",
            "date": "2017-07-23 20:49:18+00:00",
            "title": "Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events",
            "authors": [
                "Prafulla Kumar Choubey",
                "Ruihong Huang"
            ],
            "abstract": "We introduce a novel iterative approach for event coreference resolution that\ngradually builds event clusters by exploiting inter-dependencies among event\nmentions within the same chain as well as across event chains. Among event\nmentions in the same chain, we distinguish within- and cross-document event\ncoreference links by using two distinct pairwise classifiers, trained\nseparately to capture differences in feature distributions of within- and\ncross-document event clusters. Our event coreference approach alternates\nbetween WD and CD clustering and combines arguments from both event clusters\nafter every merge, continuing till no more merge can be made. And then it\nperforms further merging between event chains that are both closely related to\na set of other chains of events. Experiments on the ECB+ corpus show that our\nmodel outperforms state-of-the-art methods in joint task of WD and CD event\ncoreference resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07499v1",
            "date": "2017-07-24 11:49:00+00:00",
            "title": "Analysing Errors of Open Information Extraction Systems",
            "authors": [
                "Rudolf Schneider",
                "Tom Oberhauser",
                "Tobias Klatt",
                "Felix A. Gers",
                "Alexander L\u00f6ser"
            ],
            "abstract": "We report results on benchmarking Open Information Extraction (OIE) systems\nusing RelVis, a toolkit for benchmarking Open Information Extraction systems.\nOur comprehensive benchmark contains three data sets from the news domain and\none data set from Wikipedia with overall 4522 labeled sentences and 11243\nbinary or n-ary OIE relations. In our analysis on these data sets we compared\nthe performance of four popular OIE systems, ClausIE, OpenIE 4.2, Stanford\nOpenIE and PredPatt. In addition, we evaluated the impact of five common error\nclasses on a subset of 749 n-ary tuples. From our deep analysis we unreveal\nimportant research directions for a next generation of OIE systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07568v1",
            "date": "2017-07-24 14:20:07+00:00",
            "title": "CAp 2017 challenge: Twitter Named Entity Recognition",
            "authors": [
                "C\u00e9dric Lopez",
                "Ioannis Partalas",
                "Georgios Balikas",
                "Nadia Derbas",
                "Am\u00e9lie Martin",
                "Coralie Reutenauer",
                "Fr\u00e9d\u00e9rique Segond",
                "Massih-Reza Amini"
            ],
            "abstract": "The paper describes the CAp 2017 challenge. The challenge concerns the\nproblem of Named Entity Recognition (NER) for tweets written in French. We\nfirst present the data preparation steps we followed for constructing the\ndataset released in the framework of the challenge. We begin by demonstrating\nwhy NER for tweets is a challenging problem especially when the number of\nentities increases. We detail the annotation process and the necessary\ndecisions we made. We provide statistics on the inter-annotator agreement, and\nwe conclude the data description part with examples and statistics for the\ndata. We, then, describe the participation in the challenge, where 8 teams\nparticipated, with a focus on the methods employed by the challenge\nparticipants and the scores achieved in terms of F$_1$ measure. Importantly,\nthe constructed dataset comprising $\\sim$6,000 tweets annotated for 13 types of\nentities, which to the best of our knowledge is the first such dataset in\nFrench, is publicly available at \\url{http://cap2017.imag.fr/competition.html} .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07591v1",
            "date": "2017-07-24 14:52:32+00:00",
            "title": "Transition-Based Generation from Abstract Meaning Representations",
            "authors": [
                "Timo Schick"
            ],
            "abstract": "This work addresses the task of generating English sentences from Abstract\nMeaning Representation (AMR) graphs. To cope with this task, we transform each\ninput AMR graph into a structure similar to a dependency tree and annotate it\nwith syntactic information by applying various predefined actions to it.\nSubsequently, a sentence is obtained from this tree structure by visiting its\nnodes in a specific order. We train maximum entropy models to estimate the\nprobability of each individual action and devise an algorithm that efficiently\napproximates the best sequence of actions to be applied. Using a substandard\nlanguage model, our generator achieves a Bleu score of 27.4 on the LDC2014T12\ntest set, the best result reported so far without using silver standard\nannotations from another corpus as additional training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07628v1",
            "date": "2017-07-24 16:07:01+00:00",
            "title": "Improve Lexicon-based Word Embeddings By Word Sense Disambiguation",
            "authors": [
                "Yuanzhi Ke",
                "Masafumi Hagiwara"
            ],
            "abstract": "There have been some works that learn a lexicon together with the corpus to\nimprove the word embeddings. However, they either model the lexicon separately\nbut update the neural networks for both the corpus and the lexicon by the same\nlikelihood, or minimize the distance between all of the synonym pairs in the\nlexicon. Such methods do not consider the relatedness and difference of the\ncorpus and the lexicon, and may not be the best optimized. In this paper, we\npropose a novel method that considers the relatedness and difference of the\ncorpus and the lexicon. It trains word embeddings by learning the corpus to\npredicate a word and its corresponding synonym under the context at the same\ntime. For polysemous words, we use a word sense disambiguation filter to\neliminate the synonyms that have different meanings for the context. To\nevaluate the proposed method, we compare the performance of the word embeddings\ntrained by our proposed model, the control groups without the filter or the\nlexicon, and the prior works in the word similarity tasks and text\nclassification task. The experimental results show that the proposed model\nprovides better embeddings for polysemous words and improves the performance\nfor text classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07631v1",
            "date": "2017-07-24 16:19:59+00:00",
            "title": "Deep Architectures for Neural Machine Translation",
            "authors": [
                "Antonio Valerio Miceli Barone",
                "Jind\u0159ich Helcl",
                "Rico Sennrich",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "It has been shown that increasing model depth improves the quality of neural\nmachine translation. However, different architectural variants to increase\nmodel depth have been proposed, and so far, there has been no thorough\ncomparative study.\n  In this work, we describe and evaluate several existing approaches to\nintroduce depth in neural machine translation. Additionally, we explore novel\narchitectural variants, including deep transition RNNs, and we vary how\nattention is used in the deep decoder. We introduce a novel \"BiDeep\" RNN\narchitecture that combines deep transition RNNs and stacked RNNs.\n  Our evaluation is carried out on the English to German WMT news translation\ndataset, using a single-GPU machine for both training and inference. We find\nthat several of our proposed architectures improve upon existing approaches in\nterms of speed and translation quality. We obtain best improvements with a\nBiDeep RNN of combined depth 8, obtaining an average improvement of 1.5 BLEU\nover a strong shallow baseline.\n  We release our code for ease of adoption.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07719v3",
            "date": "2017-07-24 19:39:22+00:00",
            "title": "Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification",
            "authors": [
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We introduce globally normalized convolutional neural networks for joint\nentity classification and relation extraction. In particular, we propose a way\nto utilize a linear-chain conditional random field output layer for predicting\nentity types and relations between entities at the same time. Our experiments\nshow that global normalization outperforms a locally normalized softmax layer\non a benchmark dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07755v2",
            "date": "2017-07-24 21:33:21+00:00",
            "title": "AMR Parsing using Stack-LSTMs",
            "authors": [
                "Miguel Ballesteros",
                "Yaser Al-Onaizan"
            ],
            "abstract": "We present a transition-based AMR parser that directly generates AMR parses\nfrom plain text. We use Stack-LSTMs to represent our parser state and make\ndecisions greedily. In our experiments, we show that our parser achieves very\ncompetitive scores on English using only AMR training data. Adding additional\ninformation, such as POS tags and dependency trees, improves the results\nfurther.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07806v2",
            "date": "2017-07-25 03:42:40+00:00",
            "title": "Macro Grammars and Holistic Triggering for Efficient Semantic Parsing",
            "authors": [
                "Yuchen Zhang",
                "Panupong Pasupat",
                "Percy Liang"
            ],
            "abstract": "To learn a semantic parser from denotations, a learning algorithm must search\nover a combinatorially large space of logical forms for ones consistent with\nthe annotated denotations. We propose a new online learning algorithm that\nsearches faster as training progresses. The two key ideas are using macro\ngrammars to cache the abstract patterns of useful logical forms found thus far,\nand holistic triggering to efficiently retrieve the most relevant patterns\nbased on sentence similarity. On the WikiTableQuestions dataset, we first\nexpand the search space of an existing model to improve the state-of-the-art\naccuracy from 38.7% to 42.7%, and then use macro grammars and holistic\ntriggering to achieve an 11x speedup and an accuracy of 43.7%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07911v1",
            "date": "2017-07-25 10:51:19+00:00",
            "title": "Machine Translation at Booking.com: Journey and Lessons Learned",
            "authors": [
                "Pavel Levin",
                "Nishikant Dhanuka",
                "Maxim Khalilov"
            ],
            "abstract": "We describe our recently developed neural machine translation (NMT) system\nand benchmark it against our own statistical machine translation (SMT) system\nas well as two other general purpose online engines (statistical and neural).\nWe present automatic and human evaluation results of the translation output\nprovided by each system. We also analyze the effect of sentence length on the\nquality of output for SMT and NMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07922v2",
            "date": "2017-07-25 11:34:14+00:00",
            "title": "Question Dependent Recurrent Entity Network for Question Answering",
            "authors": [
                "Andrea Madotto",
                "Giuseppe Attardi"
            ],
            "abstract": "Question Answering is a task which requires building models capable of\nproviding answers to questions expressed in human language. Full question\nanswering involves some form of reasoning ability. We introduce a neural\nnetwork architecture for this task, which is a form of $Memory\\ Network$, that\nrecognizes entities and their relations to answers through a focus attention\nmechanism. Our model is named $Question\\ Dependent\\ Recurrent\\ Entity\\ Network$\nand extends $Recurrent\\ Entity\\ Network$ by exploiting aspects of the question\nduring the memorization process. We validate the model on both synthetic and\nreal datasets: the $bAbI$ question answering dataset and the $CNN\\ \\&\\ Daily\\\nNews$ $reading\\ comprehension$ dataset. In our experiments, the models achieved\na State-of-The-Art in the former and competitive results in the latter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08041v1",
            "date": "2017-07-25 15:28:59+00:00",
            "title": "Synthesising Sign Language from semantics, approaching \"from the target and back\"",
            "authors": [
                "Michael Filhol",
                "Gilles Falquet"
            ],
            "abstract": "We present a Sign Language modelling approach allowing to build grammars and\ncreate linguistic input for Sign synthesis through avatars. We comment on the\ntype of grammar it allows to build, and observe a resemblance between the\nresulting expressions and traditional semantic representations. Comparing the\nways in which the paradigms are designed, we name and contrast two essentially\ndifferent strategies for building higher-level linguistic input:\n\"source-and-forward\" vs. \"target-and-back\". We conclude by favouring the\nlatter, acknowledging the power of being able to automatically generate output\nfrom semantically relevant input straight into articulations of the target\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08052v1",
            "date": "2017-07-25 15:42:25+00:00",
            "title": "Challenges in Data-to-Document Generation",
            "authors": [
                "Sam Wiseman",
                "Stuart M. Shieber",
                "Alexander M. Rush"
            ],
            "abstract": "Recent neural models have shown significant progress on the problem of\ngenerating short descriptive texts conditioned on a small number of database\nrecords. In this work, we suggest a slightly more difficult data-to-text\ngeneration task, and investigate how effective current approaches are on this\ntask. In particular, we introduce a new, large-scale corpus of data records\npaired with descriptive documents, propose a series of extractive evaluation\nmethods for analyzing performance, and obtain baseline results using current\nneural generation methods. Experiments show that these models produce fluent\ntext, but fail to convincingly approximate human-generated documents. Moreover,\neven templated baselines exceed the performance of these neural models on some\nmetrics, though copy- and reconstruction-based extensions lead to noticeable\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08081v2",
            "date": "2017-07-25 16:41:49+00:00",
            "title": "Learning Word Relatedness over Time",
            "authors": [
                "Guy D. Rosin",
                "Eytan Adar",
                "Kira Radinsky"
            ],
            "abstract": "Search systems are often focused on providing relevant results for the \"now\",\nassuming both corpora and user needs that focus on the present. However, many\ncorpora today reflect significant longitudinal collections ranging from 20\nyears of the Web to hundreds of years of digitized newspapers and books.\nUnderstanding the temporal intent of the user and retrieving the most relevant\nhistorical content has become a significant challenge. Common search features,\nsuch as query expansion, leverage the relationship between terms but cannot\nfunction well across all times when relationships vary temporally. In this\nwork, we introduce a temporal relationship model that is extracted from\nlongitudinal data collections. The model supports the task of identifying,\ngiven two words, when they relate to each other. We present an algorithmic\nframework for this task and show its application for the task of query\nexpansion, achieving high gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08084v1",
            "date": "2017-07-25 16:56:53+00:00",
            "title": "ShotgunWSD: An unsupervised algorithm for global word sense disambiguation inspired by DNA sequencing",
            "authors": [
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu",
                "Florentina Hristea"
            ],
            "abstract": "In this paper, we present a novel unsupervised algorithm for word sense\ndisambiguation (WSD) at the document level. Our algorithm is inspired by a\nwidely-used approach in the field of genetics for whole genome sequencing,\nknown as the Shotgun sequencing technique. The proposed WSD algorithm is based\non three main steps. First, a brute-force WSD algorithm is applied to short\ncontext windows (up to 10 words) selected from the document in order to\ngenerate a short list of likely sense configurations for each window. In the\nsecond step, these local sense configurations are assembled into longer\ncomposite configurations based on suffix and prefix matching. The resulted\nconfigurations are ranked by their length, and the sense of each word is chosen\nbased on a voting scheme that considers only the top k configurations in which\nthe word appears. We compare our algorithm with other state-of-the-art\nunsupervised WSD algorithms and demonstrate better performance, sometimes by a\nvery large margin. We also show that our algorithm can yield better performance\nthan the Most Common Sense (MCS) baseline on one data set. Moreover, our\nalgorithm has a very small number of parameters, is robust to parameter tuning,\nand, unlike other bio-inspired methods, it gives a deterministic solution (it\ndoes not involve random choices).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08098v1",
            "date": "2017-07-25 17:29:18+00:00",
            "title": "From Image to Text Classification: A Novel Approach based on Clustering Word Embeddings",
            "authors": [
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this paper, we propose a novel approach for text classification based on\nclustering word embeddings, inspired by the bag of visual words model, which is\nwidely used in computer vision. After each word in a collection of documents is\nrepresented as word vector using a pre-trained word embeddings model, a k-means\nalgorithm is applied on the word vectors in order to obtain a fixed-size set of\nclusters. The centroid of each cluster is interpreted as a super word embedding\nthat embodies all the semantically related word vectors in a certain region of\nthe embedding space. Every embedded word in the collection of documents is then\nassigned to the nearest cluster centroid. In the end, each document is\nrepresented as a bag of super word embeddings by computing the frequency of\neach super word embedding in the respective document. We also diverge from the\nidea of building a single vocabulary for the entire collection of documents,\nand propose to build class-specific vocabularies for better performance. Using\nthis kind of representation, we report results on two text mining tasks, namely\ntext categorization by topic and polarity classification. On both tasks, our\nmodel yields better performance than the standard bag of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08172v1",
            "date": "2017-07-25 19:27:05+00:00",
            "title": "The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations",
            "authors": [
                "Nikita Nangia",
                "Adina Williams",
                "Angeliki Lazaridou",
                "Samuel R. Bowman"
            ],
            "abstract": "This paper presents the results of the RepEval 2017 Shared Task, which\nevaluated neural network sentence representation learning models on the\nMulti-Genre Natural Language Inference corpus (MultiNLI) recently introduced by\nWilliams et al. (2017). All of the five participating teams beat the\nbidirectional LSTM (BiLSTM) and continuous bag of words baselines reported in\nWilliams et al.. The best single model used stacked BiLSTMs with residual\nconnections to extract sentence features and reached 74.5% accuracy on the\ngenre-matched test set. Surprisingly, the results of the competition were\nfairly consistent across the genre-matched and genre-mismatched test sets, and\nacross subsets of the test data representing a variety of linguistic phenomena,\nsuggesting that all of the submitted systems learned reasonably\ndomain-independent representations for sentence meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08290v1",
            "date": "2017-07-26 05:11:58+00:00",
            "title": "Fast calculation of entropy with Zhang's estimator",
            "authors": [
                "Antoni Lozano",
                "Bernardino Casas",
                "Chris Bentz",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Entropy is a fundamental property of a repertoire. Here, we present an\nefficient algorithm to estimate the entropy of types with the help of Zhang's\nestimator. The algorithm takes advantage of the fact that the number of\ndifferent frequencies in a text is in general much smaller than the number of\ntypes. We justify the convenience of the algorithm by means of an analysis of\nthe statistical properties of texts from more than 1000 languages. Our work\nopens up various possibilities for future research.",
            "journal ref": "Issues in Quantitative Linguistics 4. E. Kelih, R. Knight, J.\n  Macutek and A. Wilson (eds.). No. 23 of the series \"Studies in Quantitative\n  Linguistics\". L\\\"udenscheid: RAM-Verlag. pp. 273-285",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08349v2",
            "date": "2017-07-26 10:03:40+00:00",
            "title": "Can string kernels pass the test of time in Native Language Identification?",
            "authors": [
                "Radu Tudor Ionescu",
                "Marius Popescu"
            ],
            "abstract": "We describe a machine learning approach for the 2017 shared task on Native\nLanguage Identification (NLI). The proposed approach combines several kernels\nusing multiple kernel learning. While most of our kernels are based on\ncharacter p-grams (also known as n-grams) extracted from essays or speech\ntranscripts, we also use a kernel based on i-vectors, a low-dimensional\nrepresentation of audio recordings, provided by the shared task organizers. For\nthe learning stage, we choose Kernel Discriminant Analysis (KDA) over Kernel\nRidge Regression (KRR), because the former classifier obtains better results\nthan the latter one on the development set. In our previous work, we have used\na similar machine learning approach to achieve state-of-the-art NLI results.\nThe goal of this paper is to demonstrate that our shallow and simple approach\nbased on string kernels (with minor improvements) can pass the test of time and\nreach state-of-the-art performance in the 2017 NLI shared task, despite the\nrecent advances in natural language processing. We participated in all three\ntracks, in which the competitors were allowed to use only the essays (essay\ntrack), only the speech transcripts (speech track), or both (fusion track).\nUsing only the data provided by the organizers for training our models, we have\nreached a macro F1 score of 86.95% in the closed essay track, a macro F1 score\nof 87.55% in the closed speech track, and a macro F1 score of 93.19% in the\nclosed fusion track. With these scores, our team (UnibucKernel) ranked in the\nfirst group of teams in all three tracks, while attaining the best scores in\nthe speech and the fusion tracks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08435v5",
            "date": "2017-07-26 13:40:21+00:00",
            "title": "SPEECH-COCO: 600k Visually Grounded Spoken Captions Aligned to MSCOCO Data Set",
            "authors": [
                "William Havard",
                "Laurent Besacier",
                "Olivier Rosec"
            ],
            "abstract": "This paper presents an augmentation of MSCOCO dataset where speech is added\nto image and text. Speech captions are generated using text-to-speech (TTS)\nsynthesis resulting in 616,767 spoken captions (more than 600h) paired with\nimages. Disfluencies and speed perturbation are added to the signal in order to\nsound more natural. Each speech signal (WAV) is paired with a JSON file\ncontaining exact timecode for each word/syllable/phoneme in the spoken caption.\nSuch a corpus could be used for Language and Vision (LaVi) tasks including\nspeech input or output instead of text. Investigating multimodal learning\nschemes for unsupervised speech pattern discovery is also possible with this\ncorpus, as demonstrated by a preliminary study conducted on a subset of the\ncorpus (10h, 10k spoken captions). The dataset is available on Zenodo:\nhttps://zenodo.org/record/4282267",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08446v2",
            "date": "2017-07-25 04:17:42+00:00",
            "title": "All that is English may be Hindi: Enhancing language identification through automatic ranking of likeliness of word borrowing in social media",
            "authors": [
                "Jasabanta Patro",
                "Bidisha Samanta",
                "Saurabh Singh",
                "Abhipsa Basu",
                "Prithwish Mukherjee",
                "Monojit Choudhury",
                "Animesh Mukherjee"
            ],
            "abstract": "In this paper, we present a set of computational methods to identify the\nlikeliness of a word being borrowed, based on the signals from social media. In\nterms of Spearman correlation coefficient values, our methods perform more than\ntwo times better (nearly 0.62) in predicting the borrowing likeliness compared\nto the best performing baseline (nearly 0.26) reported in literature. Based on\nthis likeliness estimate we asked annotators to re-annotate the language tags\nof foreign words in predominantly native contexts. In 88 percent of cases the\nannotators felt that the foreign language tag should be replaced by native\nlanguage tag, thus indicating a huge scope for improvement of automatic\nlanguage identification systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08458v1",
            "date": "2017-07-26 14:20:12+00:00",
            "title": "Men Are from Mars, Women Are from Venus: Evaluation and Modelling of Verbal Associations",
            "authors": [
                "Ekaterina Vylomova",
                "Andrei Shcherbakov",
                "Yuriy Philippovich",
                "Galina Cherkasova"
            ],
            "abstract": "We present a quantitative analysis of human word association pairs and study\nthe types of relations presented in the associations. We put our main focus on\nthe correlation between response types and respondent characteristics such as\noccupation and gender by contrasting syntagmatic and paradigmatic associations.\nFinally, we propose a personalised distributed word association model and show\nthe importance of incorporating demographic factors into the models commonly\nused in natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08608v3",
            "date": "2017-07-26 19:00:10+00:00",
            "title": "Gradient-based Inference for Networks with Output Constraints",
            "authors": [
                "Jay Yoon Lee",
                "Sanket Vaibhav Mehta",
                "Michael Wick",
                "Jean-Baptiste Tristan",
                "Jaime Carbonell"
            ],
            "abstract": "Practitioners apply neural networks to increasingly complex problems in\nnatural language processing, such as syntactic parsing and semantic role\nlabeling that have rich output structures. Many such structured-prediction\nproblems require deterministic constraints on the output values; for example,\nin sequence-to-sequence syntactic parsing, we require that the sequential\noutputs encode valid trees. While hidden units might capture such properties,\nthe network is not always able to learn such constraints from the training data\nalone, and practitioners must then resort to post-processing. In this paper, we\npresent an inference method for neural networks that enforces deterministic\nconstraints on outputs without performing rule-based post-processing or\nexpensive discrete search. Instead, in the spirit of gradient-based training,\nwe enforce constraints with gradient-based inference (GBI): for each input at\ntest-time, we nudge continuous model weights until the network's unconstrained\ninference procedure generates an output that satisfies the constraints. We\nstudy the efficacy of GBI on three tasks with hard constraints: semantic role\nlabeling, syntactic parsing, and sequence transduction. In each case, the\nalgorithm not only satisfies constraints but improves accuracy, even when the\nunderlying network is state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08660v1",
            "date": "2017-07-26 23:02:11+00:00",
            "title": "Temporal dynamics of semantic relations in word embeddings: an application to predicting armed conflict participants",
            "authors": [
                "Andrey Kutuzov",
                "Erik Velldal",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "This paper deals with using word embedding models to trace the temporal\ndynamics of semantic relations between pairs of words. The set-up is similar to\nthe well-known analogies task, but expanded with a time dimension. To this end,\nwe apply incremental updating of the models with new training texts, including\nincremental vocabulary expansion, coupled with learned transformation matrices\nthat let us map between members of the relation. The proposed approach is\nevaluated on the task of predicting insurgent armed groups based on\ngeographical locations. The gold standard data for the time span 1994--2010 is\nextracted from the UCDP Armed Conflicts dataset. The results show that the\nmethod is feasible and outperforms the baselines, but also that important work\nstill remains to be done.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08713v1",
            "date": "2017-07-27 05:49:51+00:00",
            "title": "Determining Semantic Textual Similarity using Natural Deduction Proofs",
            "authors": [
                "Hitomi Yanaka",
                "Koji Mineshima",
                "Pascual Martinez-Gomez",
                "Daisuke Bekki"
            ],
            "abstract": "Determining semantic textual similarity is a core research subject in natural\nlanguage processing. Since vector-based models for sentence representation\noften use shallow information, capturing accurate semantics is difficult. By\ncontrast, logical semantic representations capture deeper levels of sentence\nsemantics, but their symbolic nature does not offer graded notions of textual\nsimilarity. We propose a method for determining semantic textual similarity by\ncombining shallow features with features extracted from natural deduction\nproofs of bidirectional entailment relations between sentence pairs. For the\nnatural deduction proofs, we use ccg2lambda, a higher-order automatic inference\nsystem, which converts Combinatory Categorial Grammar (CCG) derivation trees\ninto semantic representations and conducts natural deduction proofs.\nExperiments show that our system was able to outperform other logic-based\nsystems and that features derived from the proofs are effective for learning\ntextual similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08939v1",
            "date": "2017-07-26 17:30:57+00:00",
            "title": "Strawman: an Ensemble of Deep Bag-of-Ngrams for Sentiment Analysis",
            "authors": [
                "Kyunghyun Cho"
            ],
            "abstract": "This paper describes a builder entry, named \"strawman\", to the sentence-level\nsentiment analysis task of the \"Build It, Break It\" shared task of the First\nWorkshop on Building Linguistically Generalizable NLP Systems. The goal of a\nbuilder is to provide an automated sentiment analyzer that would serve as a\ntarget for breakers whose goal is to find pairs of minimally-differing\nsentences that break the analyzer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08976v1",
            "date": "2017-07-27 18:01:18+00:00",
            "title": "Effective Inference for Generative Neural Parsing",
            "authors": [
                "Mitchell Stern",
                "Daniel Fried",
                "Dan Klein"
            ],
            "abstract": "Generative neural models have recently achieved state-of-the-art results for\nconstituency parsing. However, without a feasible search procedure, their use\nhas so far been limited to reranking the output of external parsers in which\ndecoding is more tractable. We describe an alternative to the conventional\naction-level beam search used for discriminative neural models that enables us\nto decode directly in these generative models. We then show that by improving\nour basic candidate selection strategy and using a coarse pruning function, we\ncan improve accuracy while exploring significantly less of the search space.\nApplied to the model of Choe and Charniak (2016), our inference procedure\nobtains 92.56 F1 on section 23 of the Penn Treebank, surpassing prior\nstate-of-the-art results for single-model systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08998v1",
            "date": "2017-07-26 07:48:46+00:00",
            "title": "ASDA : Analyseur Syntaxique du Dialecte Alg{\u00e9}rien dans un but d'analyse s{\u00e9}mantique",
            "authors": [
                "Im\u00e8ne Guellil",
                "Fai\u00e7al Azouaou"
            ],
            "abstract": "Opinion mining and sentiment analysis in social media is a research issue\nhaving a great interest in the scientific community. However, before begin this\nanalysis, we are faced with a set of problems. In particular, the problem of\nthe richness of languages and dialects within these media. To address this\nproblem, we propose in this paper an approach of construction and\nimplementation of Syntactic analyzer named ASDA. This tool represents a parser\nfor the Algerian dialect that label the terms of a given corpus. Thus, we\nconstruct a labeling table containing for each term its stem, different\nprefixes and suffixes, allowing us to determine the different grammatical parts\na sort of POS tagging. This labeling will serve us later in the semantic\nprocessing of the Algerian dialect, like the automatic translation of this\ndialect or sentiment analysis",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09067v1",
            "date": "2017-07-27 22:50:55+00:00",
            "title": "Adapting Sequence Models for Sentence Correction",
            "authors": [
                "Allen Schmaltz",
                "Yoon Kim",
                "Alexander M. Rush",
                "Stuart M. Shieber"
            ],
            "abstract": "In a controlled experiment of sequence-to-sequence approaches for the task of\nsentence correction, we find that character-based models are generally more\neffective than word-based models and models that encode subword information via\nconvolutions, and that modeling the output data as a series of diffs improves\neffectiveness over standard approaches. Our strongest sequence-to-sequence\nmodel improves over our strongest phrase-based statistical machine translation\nmodel, with access to the same data, by 6 M2 (0.5 GLEU) points. Additionally,\nin the data environment of the standard CoNLL-2014 setup, we demonstrate that\nmodeling (and tuning against) diffs yields similar or better M2 scores with\nsimpler models and/or significantly less data than previous\nsequence-to-sequence approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09168v1",
            "date": "2017-07-28 09:46:29+00:00",
            "title": "Learning to Predict Charges for Criminal Cases with Legal Basis",
            "authors": [
                "Bingfeng Luo",
                "Yansong Feng",
                "Jianbo Xu",
                "Xiang Zhang",
                "Dongyan Zhao"
            ],
            "abstract": "The charge prediction task is to determine appropriate charges for a given\ncase, which is helpful for legal assistant systems where the user input is fact\ndescription. We argue that relevant law articles play an important role in this\ntask, and therefore propose an attention-based neural network method to jointly\nmodel the charge prediction task and the relevant article extraction task in a\nunified framework. The experimental results show that, besides providing legal\nbasis, the relevant articles can also clearly improve the charge prediction\nresults, and our full model can effectively predict appropriate charges for\ncases with different expression styles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09231v1",
            "date": "2017-07-28 13:49:46+00:00",
            "title": "Improving coreference resolution with automatically predicted prosodic information",
            "authors": [
                "Ina R\u00f6siger",
                "Sabrina Stehwien",
                "Arndt Riester",
                "Ngoc Thang Vu"
            ],
            "abstract": "Adding manually annotated prosodic information, specifically pitch accents\nand phrasing, to the typical text-based feature set for coreference resolution\nhas previously been shown to have a positive effect on German data. Practical\napplications on spoken language, however, would rely on automatically predicted\nprosodic information. In this paper we predict pitch accents (and phrase\nboundaries) using a convolutional neural network (CNN) model from acoustic\nfeatures extracted from the speech signal. After an assessment of the quality\nof these automatic prosodic annotations, we show that they also significantly\nimprove coreference resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09406v1",
            "date": "2017-07-28 20:27:35+00:00",
            "title": "Online Deception Detection Refueled by Real World Data Collection",
            "authors": [
                "Wenlin Yao",
                "Zeyu Dai",
                "Ruihong Huang",
                "James Caverlee"
            ],
            "abstract": "The lack of large realistic datasets presents a bottleneck in online\ndeception detection studies. In this paper, we apply a data collection method\nbased on social network analysis to quickly identify high-quality deceptive and\ntruthful online reviews from Amazon. The dataset contains more than 10,000\ndeceptive reviews and is diverse in product domains and reviewers. Using this\ndataset, we explore effective general features for online deception detection\nthat perform well across domains. We demonstrate that with generalized features\n- advertising speak and writing complexity scores - deception detection\nperformance can be further improved by adding additional deceptive reviews from\nassorted domains in training. Finally, reviewer level evaluation gives an\ninteresting insight into different deceptive reviewers' writing styles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09410v1",
            "date": "2017-07-28 20:38:15+00:00",
            "title": "A Weakly Supervised Approach to Train Temporal Relation Classifiers and Acquire Regular Event Pairs Simultaneously",
            "authors": [
                "Wenlin Yao",
                "Saipravallika Nettyam",
                "Ruihong Huang"
            ],
            "abstract": "Capabilities of detecting temporal relations between two events can benefit\nmany applications. Most of existing temporal relation classifiers were trained\nin a supervised manner. Instead, we explore the observation that regular event\npairs show a consistent temporal relation despite of their various contexts,\nand these rich contexts can be used to train a contextual temporal relation\nclassifier, which can further recognize new temporal relation contexts and\nidentify new regular event pairs. We focus on detecting after and before\ntemporal relations and design a weakly supervised learning approach that\nextracts thousands of regular event pairs and learns a contextual temporal\nrelation classifier simultaneously. Evaluation shows that the acquired regular\nevent pairs are of high quality and contain rich commonsense knowledge and\ndomain specific knowledge. In addition, the weakly supervised trained temporal\nrelation classifier achieves comparable performance with the state-of-the-art\nsupervised systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09443v1",
            "date": "2017-07-29 00:46:23+00:00",
            "title": "Bilingual Document Alignment with Latent Semantic Indexing",
            "authors": [
                "Ulrich Germann"
            ],
            "abstract": "We apply cross-lingual Latent Semantic Indexing to the Bilingual Document\nAlignment Task at WMT16. Reduced-rank singular value decomposition of a\nbilingual term-document matrix derived from known English/French page pairs in\nthe training data allows us to map monolingual documents into a joint semantic\nspace. Two variants of cosine similarity between the vectors that place each\ndocument into the joint semantic space are combined with a measure of string\nsimilarity between corresponding URLs to produce 1:1 alignments of\nEnglish/French web pages in a variety of domains. The system achieves a recall\nof ca. 88% if no in-domain data is used for building the latent semantic model,\nand 93% if such data is included.\n  Analysing the system's errors on the training data, we argue that evaluating\naligner performance based on exact URL matches under-estimates their true\nperformance and propose an alternative that is able to account for duplicates\nand near-duplicates in the underlying data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09448v1",
            "date": "2017-07-29 01:47:53+00:00",
            "title": "Sentiment Analysis on Financial News Headlines using Training Dataset Augmentation",
            "authors": [
                "Vineet John",
                "Olga Vechtomova"
            ],
            "abstract": "This paper discusses the approach taken by the UWaterloo team to arrive at a\nsolution for the Fine-Grained Sentiment Analysis problem posed by Task 5 of\nSemEval 2017. The paper describes the document vectorization and sentiment\nscore prediction techniques used, as well as the design and implementation\ndecisions taken while building the system for this task. The system uses text\nvectorization models, such as N-gram, TF-IDF and paragraph embeddings, coupled\nwith regression model variants to predict the sentiment scores. Amongst the\nmethods examined, unigrams and bigrams coupled with simple linear regression\nobtained the best baseline accuracy. The paper also explores data augmentation\nmethods to supplement the training dataset. This system was designed for\nSubtask 2 (News Statements and Headlines).",
            "journal ref": "Association for Computational Linguistics, Proceedings of the 11th\n  International Workshop on Semantic Evaluation (SemEval-2017), 869-873",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09533v1",
            "date": "2017-07-29 15:58:55+00:00",
            "title": "Curriculum Learning and Minibatch Bucketing in Neural Machine Translation",
            "authors": [
                "Tom Kocmi",
                "Ondrej Bojar"
            ],
            "abstract": "We examine the effects of particular orderings of sentence pairs on the\non-line training of neural machine translation (NMT). We focus on two types of\nsuch orderings: (1) ensuring that each minibatch contains sentences similar in\nsome aspect and (2) gradual inclusion of some sentence types as the training\nprogresses (so called \"curriculum learning\"). In our English-to-Czech\nexperiments, the internal homogeneity of minibatches has no effect on the\ntraining but some of our \"curricula\" achieve a small improvement over the\nbaseline.",
            "journal ref": "Proceedings of the International Conference Recent Advances in\n  Natural Language Processing, RANLP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09569v1",
            "date": "2017-07-29 23:38:25+00:00",
            "title": "Learning Language Representations for Typology Prediction",
            "authors": [
                "Chaitanya Malaviya",
                "Graham Neubig",
                "Patrick Littell"
            ],
            "abstract": "One central mystery of neural NLP is what neural models \"know\" about their\nsubject matter. When a neural machine translation system learns to translate\nfrom one language to another, does it learn the syntax or semantics of the\nlanguages? Can this knowledge be extracted from the system to fill holes in\nhuman scientific knowledge? Existing typological databases contain relatively\nfull feature specifications for only a few hundred languages. Exploiting the\nexistence of parallel texts in more than a thousand languages, we build a\nmassive many-to-one neural machine translation (NMT) system from 1017 languages\ninto English, and use this to predict information missing from typological\ndatabases. Experiments show that the proposed method is able to infer not only\nsyntactic, but also phonological and phonetic inventory features, and improves\nover a baseline that has access to information about the languages' geographic\nand phylogenetic neighbors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09611v1",
            "date": "2017-07-30 11:34:40+00:00",
            "title": "Joint Named Entity Recognition and Stance Detection in Tweets",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck"
            ],
            "abstract": "Named entity recognition (NER) is a well-established task of information\nextraction which has been studied for decades. More recently, studies reporting\nNER experiments on social media texts have emerged. On the other hand, stance\ndetection is a considerably new research topic usually considered within the\nscope of sentiment analysis. Stance detection studies are mostly applied to\ntexts of online debates where the stance of the text owner for a particular\ntarget, either explicitly or implicitly mentioned in text, is explored. In this\nstudy, we investigate the possible contribution of named entities to the stance\ndetection task in tweets. We report the evaluation results of NER experiments\nas well as that of the subsequent stance detection experiments using named\nentities, on a publicly-available stance-annotated data set of tweets. Our\nresults indicate that named entities obtained with a high-performance NER\nsystem can contribute to stance detection performance on tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09751v3",
            "date": "2017-07-31 08:10:49+00:00",
            "title": "Skill2vec: Machine Learning Approach for Determining the Relevant Skills from Job Description",
            "authors": [
                "Le Van-Duyet",
                "Vo Minh Quan",
                "Dang Quang An"
            ],
            "abstract": "Unsupervise learned word embeddings have seen tremendous success in numerous\nNatural Language Processing (NLP) tasks in recent years. The main contribution\nof this paper is to develop a technique called Skill2vec, which applies machine\nlearning techniques in recruitment to enhance the search strategy to find\ncandidates possessing the appropriate skills. Skill2vec is a neural network\narchitecture inspired by Word2vec, developed by Mikolov et al. in 2013. It\ntransforms skills to new vector space, which has the characteristics of\ncalculation and presents skills relationships. We conducted an experiment\nevaluation manually by a recruitment company's domain experts to demonstrate\nthe effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09769v1",
            "date": "2017-07-31 08:56:32+00:00",
            "title": "Low-Resource Neural Headline Generation",
            "authors": [
                "Ottokar Tilk",
                "Tanel Alum\u00e4e"
            ],
            "abstract": "Recent neural headline generation models have shown great results, but are\ngenerally trained on very large datasets. We focus our efforts on improving\nheadline quality on smaller datasets by the means of pretraining. We propose\nnew methods that enable pre-training all the parameters of the model and\nutilize all available text, resulting in improvements by up to 32.4% relative\nin perplexity and 2.84 points in ROUGE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09816v1",
            "date": "2017-07-31 12:32:16+00:00",
            "title": "Combining Thesaurus Knowledge and Probabilistic Topic Models",
            "authors": [
                "Natalia Loukachevitch",
                "Michael Nokel",
                "Kirill Ivanov"
            ],
            "abstract": "In this paper we present the approach of introducing thesaurus knowledge into\nprobabilistic topic models. The main idea of the approach is based on the\nassumption that the frequencies of semantically related words and phrases,\nwhich are met in the same texts, should be enhanced: this action leads to their\nlarger contribution into topics found in these texts. We have conducted\nexperiments with several thesauri and found that for improving topic models, it\nis useful to utilize domain-specific knowledge. If a general thesaurus, such as\nWordNet, is used, the thesaurus-based improvement of topic models can be\nachieved with excluding hyponymy relations in combined topic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09879v1",
            "date": "2017-07-31 14:31:01+00:00",
            "title": "Linguistically Motivated Vocabulary Reduction for Neural Machine Translation from Turkish to English",
            "authors": [
                "Duygu Ataman",
                "Matteo Negri",
                "Marco Turchi",
                "Marcello Federico"
            ],
            "abstract": "The necessity of using a fixed-size word vocabulary in order to control the\nmodel complexity in state-of-the-art neural machine translation (NMT) systems\nis an important bottleneck on performance, especially for morphologically rich\nlanguages. Conventional methods that aim to overcome this problem by using\nsub-word or character-level representations solely rely on statistics and\ndisregard the linguistic properties of words, which leads to interruptions in\nthe word structure and causes semantic and syntactic losses. In this paper, we\npropose a new vocabulary reduction method for NMT, which can reduce the\nvocabulary of a given input corpus at any rate while also considering the\nmorphological properties of the language. Our method is based on unsupervised\nmorphology learning and can be, in principle, used for pre-processing any\nlanguage pair. We also present an alternative word segmentation method based on\nsupervised morphological analysis, which aids us in measuring the accuracy of\nour model. We evaluate our method in Turkish-to-English NMT task where the\ninput language is morphologically rich and agglutinative. We analyze different\nrepresentation methods in terms of translation accuracy as well as the semantic\nand syntactic properties of the generated output. Our method obtains a\nsignificant improvement of 2.3 BLEU points over the conventional vocabulary\nreduction technique, showing that it can provide better accuracy in open\nvocabulary translation of morphologically rich languages.",
            "journal ref": "The Prague Bulletin of Mathematical Linguistics. No. 108, 2017,\n  pp. 331-342",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09920v1",
            "date": "2017-07-31 15:31:12+00:00",
            "title": "Regularization techniques for fine-tuning in neural machine translation",
            "authors": [
                "Antonio Valerio Miceli Barone",
                "Barry Haddow",
                "Ulrich Germann",
                "Rico Sennrich"
            ],
            "abstract": "We investigate techniques for supervised domain adaptation for neural machine\ntranslation where an existing model trained on a large out-of-domain dataset is\nadapted to a small in-domain dataset. In this scenario, overfitting is a major\nchallenge. We investigate a number of techniques to reduce overfitting and\nimprove transfer learning, including regularization techniques such as dropout\nand L2-regularization towards an out-of-domain prior. In addition, we introduce\ntuneout, a novel regularization technique inspired by dropout. We apply these\ntechniques, alone and in combination, to neural machine translation, obtaining\nimprovements on IWSLT datasets for English->German and English->Russian. We\nalso investigate the amounts of in-domain training data needed for domain\nadaptation in NMT, and find a logarithmic relationship between the amount of\ntraining data and gain in BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00092v1",
            "date": "2017-10-31 20:24:07+00:00",
            "title": "Summarizing Dialogic Arguments from Social Media",
            "authors": [
                "Amita Misra",
                "Shereen Oraby",
                "Shubhangi Tandon",
                "Sharath TS",
                "Pranav Anand",
                "Marilyn Walker"
            ],
            "abstract": "Online argumentative dialog is a rich source of information on popular\nbeliefs and opinions that could be useful to companies as well as governmental\nor public policy agencies. Compact, easy to read, summaries of these dialogues\nwould thus be highly valuable. A priori, it is not even clear what form such a\nsummary should take. Previous work on summarization has primarily focused on\nsummarizing written texts, where the notion of an abstract of the text is well\ndefined. We collect gold standard training data consisting of five human\nsummaries for each of 161 dialogues on the topics of Gay Marriage, Gun Control\nand Abortion. We present several different computational models aimed at\nidentifying segments of the dialogues whose content should be used for the\nsummary, using linguistic features and Word2vec features with both SVMs and\nBidirectional LSTMs. We show that we can identify the most important arguments\nby using the dialog context with a best F-measure of 0.74 for gun control, 0.71\nfor gay marriage, and 0.67 for abortion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00155v1",
            "date": "2017-11-01 01:08:49+00:00",
            "title": "Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples",
            "authors": [
                "Pavlos Vougiouklis",
                "Hady Elsahar",
                "Lucie-Aim\u00e9e Kaffee",
                "Christoph Gravier",
                "Frederique Laforest",
                "Jonathon Hare",
                "Elena Simperl"
            ],
            "abstract": "Most people do not interact with Semantic Web data directly. Unless they have\nthe expertise to understand the underlying technology, they need textual or\nvisual interfaces to help them make sense of it. We explore the problem of\ngenerating natural language summaries for Semantic Web data. This is\nnon-trivial, especially in an open-domain context. To address this problem, we\nexplore the use of neural networks. Our system encodes the information from a\nset of triples into a vector of fixed dimensionality and generates a textual\nsummary by conditioning the output on the encoded vector. We train and evaluate\nour models on two corpora of loosely aligned Wikipedia snippets and DBpedia and\nWikidata triples with promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00179v1",
            "date": "2017-11-01 02:56:27+00:00",
            "title": "Keyword-based Query Comprehending via Multiple Optimized-Demand Augmentation",
            "authors": [
                "Boyuan Pan",
                "Hao Li",
                "Zhou Zhao",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "In this paper, we consider the problem of machine reading task when the\nquestions are in the form of keywords, rather than natural language. In recent\nyears, researchers have achieved significant success on machine reading\ncomprehension tasks, such as SQuAD and TriviaQA. These datasets provide a\nnatural language question sentence and a pre-selected passage, and the goal is\nto answer the question according to the passage. However, in the situation of\ninteracting with machines by means of text, people are more likely to raise a\nquery in form of several keywords rather than a complete sentence. The\nkeyword-based query comprehension is a new challenge, because small variations\nto a question may completely change its semantical information, thus yield\ndifferent answers. In this paper, we propose a novel neural network system that\nconsists a Demand Optimization Model based on a passage-attention neural\nmachine translation and a Reader Model that can find the answer given the\noptimized question. The Demand Optimization Model optimizes the original query\nand output multiple reconstructed questions, then the Reader Model takes the\nnew questions as input and locate the answers from the passage. To make\npredictions robust, an evaluation mechanism will score the reconstructed\nquestions so the final answer strike a good balance between the quality of both\nthe Demand Optimization Model and the Reader Model. Experimental results on\nseveral datasets show that our framework significantly improves multiple strong\nbaselines on this challenging task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00247v1",
            "date": "2017-11-01 08:27:46+00:00",
            "title": "Improved Text Language Identification for the South African Languages",
            "authors": [
                "Bernardt Duvenhage",
                "Mfundo Ntini",
                "Phala Ramonyai"
            ],
            "abstract": "Virtual assistants and text chatbots have recently been gaining popularity.\nGiven the short message nature of text-based chat interactions, the language\nidentification systems of these bots might only have 15 or 20 characters to\nmake a prediction. However, accurate text language identification is important,\nespecially in the early stages of many multilingual natural language processing\npipelines.\n  This paper investigates the use of a naive Bayes classifier, to accurately\npredict the language family that a piece of text belongs to, combined with a\nlexicon based classifier to distinguish the specific South African language\nthat the text is written in. This approach leads to a 31% reduction in the\nlanguage detection error.\n  In the spirit of reproducible research the training and testing datasets as\nwell as the code are published on github. Hopefully it will be useful to create\na text language identification shared task for South African languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00279v3",
            "date": "2017-11-01 10:40:42+00:00",
            "title": "Paraphrase Generation with Deep Reinforcement Learning",
            "authors": [
                "Zichao Li",
                "Xin Jiang",
                "Lifeng Shang",
                "Hang Li"
            ],
            "abstract": "Automatic generation of paraphrases from a given sentence is an important yet\nchallenging task in natural language processing (NLP), and plays a key role in\na number of applications such as question answering, search, and dialogue. In\nthis paper, we present a deep reinforcement learning approach to paraphrase\ngeneration. Specifically, we propose a new framework for the task, which\nconsists of a \\textit{generator} and an \\textit{evaluator}, both of which are\nlearned from data. The generator, built as a sequence-to-sequence learning\nmodel, can produce paraphrases given a sentence. The evaluator, constructed as\na deep matching model, can judge whether two sentences are paraphrases of each\nother. The generator is first trained by deep learning and then further\nfine-tuned by reinforcement learning in which the reward is given by the\nevaluator. For the learning of the evaluator, we propose two methods based on\nsupervised learning and inverse reinforcement learning respectively, depending\non the type of available training data. Empirical study shows that the learned\nevaluator can guide the generator to produce more accurate paraphrases.\nExperimental results demonstrate the proposed models (the generators)\noutperform the state-of-the-art methods in paraphrase generation in both\nautomatic evaluation and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00294v1",
            "date": "2017-11-01 11:29:27+00:00",
            "title": "Towards Automatic Generation of Entertaining Dialogues in Chinese Crosstalks",
            "authors": [
                "Shikang Du",
                "Xiaojun Wan",
                "Yajie Ye"
            ],
            "abstract": "Crosstalk, also known by its Chinese name xiangsheng, is a traditional\nChinese comedic performing art featuring jokes and funny dialogues, and one of\nChina's most popular cultural elements. It is typically in the form of a\ndialogue between two performers for the purpose of bringing laughter to the\naudience, with one person acting as the leading comedian and the other as the\nsupporting role. Though general dialogue generation has been widely explored in\nprevious studies, it is unknown whether such entertaining dialogues can be\nautomatically generated or not. In this paper, we for the first time\ninvestigate the possibility of automatic generation of entertaining dialogues\nin Chinese crosstalks. Given the utterance of the leading comedian in each\ndialogue, our task aims to generate the replying utterance of the supporting\nrole. We propose a humor-enhanced translation model to address this task and\nhuman evaluation results demonstrate the efficacy of our proposed model. The\nfeasibility of automatic entertaining dialogue generation is also verified.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00309v1",
            "date": "2017-11-01 12:25:20+00:00",
            "title": "Improving Neural Machine Translation through Phrase-based Forced Decoding",
            "authors": [
                "Jingyi Zhang",
                "Masao Utiyama",
                "Eiichro Sumita",
                "Graham Neubig",
                "Satoshi Nakamura"
            ],
            "abstract": "Compared to traditional statistical machine translation (SMT), neural machine\ntranslation (NMT) often sacrifices adequacy for the sake of fluency. We propose\na method to combine the advantages of traditional SMT and NMT by exploiting an\nexisting phrase-based SMT model to compute the phrase-based decoding cost for\nan NMT output and then using this cost to rerank the n-best NMT outputs. The\nmain challenge in implementing this approach is that NMT outputs may not be in\nthe search space of the standard phrase-based decoding algorithm, because the\nsearch space of phrase-based SMT is limited by the phrase-based translation\nrule table. We propose a soft forced decoding algorithm, which can always\nsuccessfully find a decoding path for any NMT output. We show that using the\nforced decoding cost to rerank the NMT outputs can successfully improve\ntranslation quality on four different language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00331v3",
            "date": "2017-11-01 13:22:02+00:00",
            "title": "Semantic Structure and Interpretability of Word Embeddings",
            "authors": [
                "Lutfi Kerem Senel",
                "Ihsan Utlu",
                "Veysel Yucesoy",
                "Aykut Koc",
                "Tolga Cukur"
            ],
            "abstract": "Dense word embeddings, which encode semantic meanings of words to low\ndimensional vector spaces have become very popular in natural language\nprocessing (NLP) research due to their state-of-the-art performances in many\nNLP tasks. Word embeddings are substantially successful in capturing semantic\nrelations among words, so a meaningful semantic structure must be present in\nthe respective vector spaces. However, in many cases, this semantic structure\nis broadly and heterogeneously distributed across the embedding dimensions,\nwhich makes interpretation a big challenge. In this study, we propose a\nstatistical method to uncover the latent semantic structure in the dense word\nembeddings. To perform our analysis we introduce a new dataset (SEMCAT) that\ncontains more than 6500 words semantically grouped under 110 categories. We\nfurther propose a method to quantify the interpretability of the word\nembeddings; the proposed method is a practical alternative to the classical\nword intrusion test that requires human intervention.",
            "journal ref": "L. K. \\c{S}enel, \\.I. Utlu, V. Y\\\"ucesoy, A. Ko\\c{c} and T.\n  \\c{C}ukur, \"Semantic Structure and Interpretability of Word Embeddings,\" in\n  IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no.\n  10, pp. 1769-1779, Oct. 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00354v1",
            "date": "2017-10-28 05:28:01+00:00",
            "title": "JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis",
            "authors": [
                "Ryosuke Sonobe",
                "Shinnosuke Takamichi",
                "Hiroshi Saruwatari"
            ],
            "abstract": "Thanks to improvements in machine learning techniques including deep\nlearning, a free large-scale speech corpus that can be shared between academic\ninstitutions and commercial companies has an important role. However, such a\ncorpus for Japanese speech synthesis does not exist. In this paper, we designed\na novel Japanese speech corpus, named the \"JSUT corpus,\" that is aimed at\nachieving end-to-end speech synthesis. The corpus consists of 10 hours of\nreading-style speech data and its transcription and covers all of the main\npronunciations of daily-use Japanese characters. In this paper, we describe how\nwe designed and analyzed the corpus. The corpus is freely available online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00513v3",
            "date": "2017-11-01 19:00:22+00:00",
            "title": "Evaluating Discourse Phenomena in Neural Machine Translation",
            "authors": [
                "Rachel Bawden",
                "Rico Sennrich",
                "Alexandra Birch",
                "Barry Haddow"
            ],
            "abstract": "For machine translation to tackle discourse phenomena, models must have\naccess to extra-sentential linguistic context. There has been recent interest\nin modelling context in neural machine translation (NMT), but models have been\nprincipally evaluated with standard automatic metrics, poorly adapted to\nevaluating discourse phenomena. In this article, we present hand-crafted,\ndiscourse test sets, designed to test the models' ability to exploit previous\nsource and target sentences. We investigate the performance of recently\nproposed multi-encoder NMT models trained on subtitles for English to French.\nWe also explore a novel way of exploiting context from the previous sentence.\nDespite gains using BLEU, multi-encoder models give limited improvement in the\nhandling of discourse phenomena: 50% accuracy on our coreference test set and\n53.5% for coherence/cohesion (compared to a non-contextual baseline of 50%). A\nsimple strategy of decoding the concatenation of the previous and current\nsentence leads to good performance, and our novel strategy of multi-encoding\nand decoding of two sentences leads to the best performance (72.5% for\ncoreference and 57% for coherence/cohesion), highlighting the importance of\ntarget-side context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00529v2",
            "date": "2017-11-01 20:24:39+00:00",
            "title": "Text Annotation Graphs: Annotating Complex Natural Language Phenomena",
            "authors": [
                "Angus G. Forbes",
                "Kristine Lee",
                "Gus Hahn-Powell",
                "Marco A. Valenzuela-Esc\u00e1rcega",
                "Mihai Surdeanu"
            ],
            "abstract": "This paper introduces a new web-based software tool for annotating text, Text\nAnnotation Graphs, or TAG. It provides functionality for representing complex\nrelationships between words and word phrases that are not available in other\nsoftware tools, including the ability to define and visualize relationships\nbetween the relationships themselves (semantic hypergraphs). Additionally, we\ninclude an approach to representing text annotations in which annotation\nsubgraphs, or semantic summaries, are used to show relationships outside of the\nsequential context of the text itself. Users can use these subgraphs to quickly\nfind similar structures within the current document or external annotated\ndocuments. Initially, TAG was developed to support information extraction tasks\non a large database of biomedical articles. However, our software is flexible\nenough to support a wide range of annotation tasks for any domain. Examples are\nprovided that showcase TAG's capabilities on morphological parsing and event\nextraction tasks. The TAG software is available at: https://github.com/\nCreativeCodingLab/TextAnnotationGraphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00768v3",
            "date": "2017-11-02 14:47:00+00:00",
            "title": "SRL4ORL: Improving Opinion Role Labeling using Multi-task Learning with Semantic Role Labeling",
            "authors": [
                "Ana Marasovi\u0107",
                "Anette Frank"
            ],
            "abstract": "For over a decade, machine learning has been used to extract\nopinion-holder-target structures from text to answer the question \"Who\nexpressed what kind of sentiment towards what?\". Recent neural approaches do\nnot outperform the state-of-the-art feature-based models for Opinion Role\nLabeling (ORL). We suspect this is due to the scarcity of labeled training data\nand address this issue using different multi-task learning (MTL) techniques\nwith a related task which has substantially more data, i.e. Semantic Role\nLabeling (SRL). We show that two MTL models improve significantly over the\nsingle-task model for labeling of both holders and targets, on the development\nand the test sets. We found that the vanilla MTL model which makes predictions\nusing only shared ORL and SRL features, performs the best. With deeper analysis\nwe determine what works and what might be done to make further improvements for\nORL.",
            "journal ref": "Proceedings of the 2018 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language Technologies\n  (NAACL-HLT)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00894v2",
            "date": "2017-11-02 19:13:55+00:00",
            "title": "Multi-Mention Learning for Reading Comprehension with Neural Cascades",
            "authors": [
                "Swabha Swayamdipta",
                "Ankur P. Parikh",
                "Tom Kwiatkowski"
            ],
            "abstract": "Reading comprehension is a challenging task, especially when executed across\nlonger or across multiple evidence documents, where the answer is likely to\nreoccur. Existing neural architectures typically do not scale to the entire\nevidence, and hence, resort to selecting a single passage in the document\n(either via truncation or other means), and carefully searching for the answer\nwithin that passage. However, in some cases, this strategy can be suboptimal,\nsince by focusing on a specific passage, it becomes difficult to leverage\nmultiple mentions of the same answer throughout the document. In this work, we\ntake a different approach by constructing lightweight models that are combined\nin a cascade to find the answer. Each submodel consists only of feed-forward\nnetworks equipped with an attention mechanism, making it trivially\nparallelizable. We show that our approach can scale to approximately an order\nof magnitude larger evidence documents and can aggregate information at the\nrepresentation level from multiple mentions of each answer candidate across the\ndocument. Empirically, our approach achieves state-of-the-art performance on\nboth the Wikipedia and web domains of the TriviaQA dataset, outperforming more\ncomplex, recurrent architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00938v1",
            "date": "2017-11-02 21:15:46+00:00",
            "title": "A Comparison of Feature-Based and Neural Scansion of Poetry",
            "authors": [
                "Manex Agirrezabal",
                "I\u00f1aki Alegria",
                "Mans Hulden"
            ],
            "abstract": "Automatic analysis of poetic rhythm is a challenging task that involves\nlinguistics, literature, and computer science. When the language to be analyzed\nis known, rule-based systems or data-driven methods can be used. In this paper,\nwe analyze poetic rhythm in English and Spanish. We show that the\nrepresentations of data learned from character-based neural models are more\ninformative than the ones from hand-crafted features, and that a\nBi-LSTM+CRF-model produces state-of-the art accuracy on scansion of poetry in\ntwo languages. Results also show that the information about whole word\nstructure, and not just independent syllables, is highly informative for\nperforming scansion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01006v1",
            "date": "2017-11-03 03:15:44+00:00",
            "title": "Towards Neural Machine Translation with Partially Aligned Corpora",
            "authors": [
                "Yining Wang",
                "Yang Zhao",
                "Jiajun Zhang",
                "Chengqing Zong",
                "Zhengshan Xue"
            ],
            "abstract": "While neural machine translation (NMT) has become the new paradigm, the\nparameter optimization requires large-scale parallel data which is scarce in\nmany domains and language pairs. In this paper, we address a new translation\nscenario in which there only exists monolingual corpora and phrase pairs. We\npropose a new method towards translation with partially aligned sentence pairs\nwhich are derived from the phrase pairs and monolingual corpora. To make full\nuse of the partially aligned corpora, we adapt the conventional NMT training\nmethod in two aspects. On one hand, different generation strategies are\ndesigned for aligned and unaligned target words. On the other hand, a different\nobjective function is designed to model the partially aligned parts. The\nexperiments demonstrate that our method can achieve a relatively good result in\nsuch a translation scenario, and tiny bitexts can boost translation quality to\na large extent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01048v2",
            "date": "2017-11-03 07:56:31+00:00",
            "title": "Dual Language Models for Code Switched Speech Recognition",
            "authors": [
                "Saurabh Garg",
                "Tanmay Parekh",
                "Preethi Jyothi"
            ],
            "abstract": "In this work, we present a simple and elegant approach to language modeling\nfor bilingual code-switched text. Since code-switching is a blend of two or\nmore different languages, a standard bilingual language model can be improved\nupon by using structures of the monolingual language models. We propose a novel\ntechnique called dual language models, which involves building two\ncomplementary monolingual language models and combining them using a\nprobabilistic model for switching between the two. We evaluate the efficacy of\nour approach using a conversational Mandarin-English speech corpus. We prove\nthe robustness of our model by showing significant improvements in perplexity\nmeasures over the standard bilingual language model without the use of any\nexternal information. Similar consistent improvements are also reflected in\nautomatic speech recognition error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01068v2",
            "date": "2017-11-03 09:05:44+00:00",
            "title": "Compressing Word Embeddings via Deep Compositional Code Learning",
            "authors": [
                "Raphael Shu",
                "Hideki Nakayama"
            ],
            "abstract": "Natural language processing (NLP) models often require a massive number of\nparameters for word embeddings, resulting in a large storage or memory\nfootprint. Deploying neural NLP models to mobile devices requires compressing\nthe word embeddings without any significant sacrifices in performance. For this\npurpose, we propose to construct the embeddings with few basis vectors. For\neach word, the composition of basis vectors is determined by a hash code. To\nmaximize the compression rate, we adopt the multi-codebook quantization\napproach instead of binary coding scheme. Each code is composed of multiple\ndiscrete numbers, such as (3, 2, 1, 8), where the value of each component is\nlimited to a fixed range. We propose to directly learn the discrete codes in an\nend-to-end neural network by applying the Gumbel-softmax trick. Experiments\nshow the compression rate achieves 98% in a sentiment analysis task and 94% ~\n99% in machine translation tasks without performance loss. In both tasks, the\nproposed method can improve the model performance by slightly lowering the\ncompression rate. Compared to other approaches such as character-level\nsegmentation, the proposed method is language-independent and does not require\nmodifications to the network architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01100v1",
            "date": "2017-11-03 10:53:05+00:00",
            "title": "One Model to Rule them all: Multitask and Multilingual Modelling for Lexical Analysis",
            "authors": [
                "Johannes Bjerva"
            ],
            "abstract": "When learning a new skill, you take advantage of your preexisting skills and\nknowledge. For instance, if you are a skilled violinist, you will likely have\nan easier time learning to play cello. Similarly, when learning a new language\nyou take advantage of the languages you already speak. For instance, if your\nnative language is Norwegian and you decide to learn Dutch, the lexical overlap\nbetween these two languages will likely benefit your rate of language\nacquisition. This thesis deals with the intersection of learning multiple tasks\nand learning multiple languages in the context of Natural Language Processing\n(NLP), which can be defined as the study of computational processing of human\nlanguage. Although these two types of learning may seem different on the\nsurface, we will see that they share many similarities.\n  The traditional approach in NLP is to consider a single task for a single\nlanguage at a time. However, recent advances allow for broadening this\napproach, by considering data for multiple tasks and languages simultaneously.\nThis is an important approach to explore further as the key to improving the\nreliability of NLP, especially for low-resource languages, is to take advantage\nof all relevant data whenever possible. In doing so, the hope is that in the\nlong term, low-resource languages can benefit from the advances made in NLP\nwhich are currently to a large extent reserved for high-resource languages.\nThis, in turn, may then have positive consequences for, e.g., language\npreservation, as speakers of minority languages will have a lower degree of\npressure to using high-resource languages. In the short term, answering the\nspecific research questions posed should be of use to NLP researchers working\ntowards the same goal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01161v2",
            "date": "2017-11-03 13:56:53+00:00",
            "title": "Learning Filterbanks from Raw Speech for Phone Recognition",
            "authors": [
                "Neil Zeghidour",
                "Nicolas Usunier",
                "Iasonas Kokkinos",
                "Thomas Schatz",
                "Gabriel Synnaeve",
                "Emmanuel Dupoux"
            ],
            "abstract": "We train a bank of complex filters that operates on the raw waveform and is\nfed into a convolutional neural network for end-to-end phone recognition. These\ntime-domain filterbanks (TD-filterbanks) are initialized as an approximation of\nmel-filterbanks, and then fine-tuned jointly with the remaining convolutional\narchitecture. We perform phone recognition experiments on TIMIT and show that\nfor several architectures, models trained on TD-filterbanks consistently\noutperform their counterparts trained on comparable mel-filterbanks. We get our\nbest performance by learning all front-end steps, from pre-emphasis up to\naveraging. Finally, we observe that the filters at convergence have an\nasymmetric impulse response, and that some of them remain almost analytic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01362v1",
            "date": "2017-11-03 23:48:18+00:00",
            "title": "\"Attention\" for Detecting Unreliable News in the Information Age",
            "authors": [
                "Venkatesh Duppada"
            ],
            "abstract": "An Unreliable news is any piece of information which is false or misleading,\ndeliberately spread to promote political, ideological and financial agendas.\nRecently the problem of unreliable news has got a lot of attention as the\nnumber instances of using news and social media outlets for propaganda have\nincreased rapidly. This poses a serious threat to society, which calls for\ntechnology to automatically and reliably identify unreliable news sources. This\npaper is an effort made in this direction to build systems for detecting\nunreliable news articles. In this paper, various NLP algorithms were built and\nevaluated on Unreliable News Data 2017 dataset. Variants of hierarchical\nattention networks (HAN) are presented for encoding and classifying news\narticles which achieve the best results of 0.944 ROC-AUC. Finally, Attention\nlayer weights are visualized to understand and give insight into the decisions\nmade by HANs. The results obtained are very promising and encouraging to deploy\nand use these systems in the real world to mitigate the problem of unreliable\nnews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01386v3",
            "date": "2017-11-04 03:04:40+00:00",
            "title": "Predicting Discharge Medications at Admission Time Based on Deep Learning",
            "authors": [
                "Yuan Yang",
                "Pengtao Xie",
                "Xin Gao",
                "Carol Cheng",
                "Christy Li",
                "Hongbao Zhang",
                "Eric Xing"
            ],
            "abstract": "Predicting discharge medications right after a patient being admitted is an\nimportant clinical decision, which provides physicians with guidance on what\ntype of medication regimen to plan for and what possible changes on initial\nmedication may occur during an inpatient stay. It also facilitates medication\nreconciliation process with easy detection of medication discrepancy at\ndischarge time to improve patient safety. However, since the information\navailable upon admission is limited and patients' condition may evolve during\nan inpatient stay, these predictions could be a difficult decision for\nphysicians to make. In this work, we investigate how to leverage deep learning\ntechnologies to assist physicians in predicting discharge medications based on\ninformation documented in the admission note. We build a convolutional neural\nnetwork which takes an admission note as input and predicts the medications\nplaced on the patient at discharge time. Our method is able to distill semantic\npatterns from unstructured and noisy texts, and is capable of capturing the\npharmacological correlations among medications. We evaluate our method on 25K\npatient visits and compare with 4 strong baselines. Our methods demonstrate a\n20% increase in macro-averaged F1 score than the best baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01427v1",
            "date": "2017-11-04 12:24:26+00:00",
            "title": "Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning",
            "authors": [
                "Jingjing Xu",
                "Xu Sun",
                "Sujian Li",
                "Xiaoyan Cai",
                "Bingzhen Wei"
            ],
            "abstract": "In recent years, neural networks have proven to be effective in Chinese word\nsegmentation. However, this promising performance relies on large-scale\ntraining data. Neural networks with conventional architectures cannot achieve\nthe desired results in low-resource datasets due to the lack of labelled\ntraining data. In this paper, we propose a deep stacking framework to improve\nthe performance on word segmentation tasks with insufficient data by\nintegrating datasets from diverse domains. Our framework consists of two parts,\ndomain-based models and deep stacking networks. The domain-based models are\nused to learn knowledge from different datasets. The deep stacking networks are\ndesigned to integrate domain-based models. To reduce model conflicts, we\ninnovatively add communication paths among models and design various structures\nof deep stacking networks, including Gaussian-based Stacking Networks,\nConcatenate-based Stacking Networks, Sequence-based Stacking Networks and\nTree-based Stacking Networks. We conduct experiments on six low-resource\ndatasets from various domains. Our proposed framework shows significant\nperformance improvements on all datasets compared with several strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01505v1",
            "date": "2017-11-04 22:46:54+00:00",
            "title": "Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task",
            "authors": [
                "Allyson Ettinger",
                "Sudha Rao",
                "Hal Daum\u00e9 III",
                "Emily M. Bender"
            ],
            "abstract": "This paper presents a summary of the first Workshop on Building\nLinguistically Generalizable Natural Language Processing Systems, and the\nassociated Build It Break It, The Language Edition shared task. The goal of\nthis workshop was to bring together researchers in NLP and linguistics with a\nshared task aimed at testing the generalizability of NLP systems beyond the\ndistributions of their training data. We describe the motivation, setup, and\nparticipation of the shared task, provide discussion of some highlighted\nresults, and discuss lessons learned.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01515v1",
            "date": "2017-11-05 01:36:58+00:00",
            "title": "Learning Word Embeddings from Speech",
            "authors": [
                "Yu-An Chung",
                "James Glass"
            ],
            "abstract": "In this paper, we propose a novel deep neural network architecture,\nSequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length\nvector representations of audio segments excised from a speech corpus, where\nthe vectors contain semantic information pertaining to the segments, and are\nclose to other vectors in the embedding space if their corresponding segments\nare semantically similar. The design of the proposed model is based on the RNN\nEncoder-Decoder framework, and borrows the methodology of continuous skip-grams\nfor training. The learned vector representations are evaluated on 13 widely\nused word similarity benchmarks, and achieved competitive results to that of\nGloVe. The biggest advantage of the proposed model is its capability of\nextracting semantic information of audio segments taken directly from raw\nspeech, without relying on any other modalities such as text or images, which\nare challenging and expensive to collect and annotate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01684v1",
            "date": "2017-11-06 00:39:31+00:00",
            "title": "Authorship Analysis of Xenophon's Cyropaedia",
            "authors": [
                "Anjalie Field"
            ],
            "abstract": "In the past several decades, many authorship attribution studies have used\ncomputational methods to determine the authors of disputed texts. Disputed\nauthorship is a common problem in Classics, since little information about\nancient documents has survived the centuries. Many scholars have questioned the\nauthenticity of the final chapter of Xenophon's Cyropaedia, a 4th century B.C.\nhistorical text. In this study, we use N-grams frequency vectors with a cosine\nsimilarity function and word frequency vectors with Naive Bayes Classifiers\n(NBC) and Support Vector Machines (SVM) to analyze the authorship of the\nCyropaedia. Although the N-gram analysis shows that the epilogue of the\nCyropaedia differs slightly from the rest of the work, comparing the analysis\nof Xenophon with analyses of Aristotle and Plato suggests that this difference\nis not significant. Both NBC and SVM analyses of word frequencies show that the\nfinal chapter of the Cyropaedia is closely related to the other chapters of the\nCyropaedia. Therefore, this analysis suggests that the disputed chapter was\nwritten by Xenophon. This information can help scholars better understand the\nCyropaedia and also demonstrates the usefulness of applying modern authorship\nanalysis techniques to classical literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01701v1",
            "date": "2017-11-06 03:05:05+00:00",
            "title": "Distributed Representation for Traditional Chinese Medicine Herb via Deep Learning Models",
            "authors": [
                "Wei Li",
                "Zheng Yang"
            ],
            "abstract": "Traditional Chinese Medicine (TCM) has accumulated a big amount of precious\nresource in the long history of development. TCM prescriptions that consist of\nTCM herbs are an important form of TCM treatment, which are similar to natural\nlanguage documents, but in a weakly ordered fashion. Directly adapting language\nmodeling style methods to learn the embeddings of the herbs can be problematic\nas the herbs are not strictly in order, the herbs in the front of the\nprescription can be connected to the very last ones. In this paper, we propose\nto represent TCM herbs with distributed representations via Prescription Level\nLanguage Modeling (PLLM). In one of our experiments, the correlation between\nour calculated similarity between medicines and the judgment of professionals\nachieves a Spearman score of 55.35 indicating a strong correlation, which\nsurpasses human beginners (TCM related field bachelor student) by a big margin\n(over 10%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01731v3",
            "date": "2017-11-06 05:20:54+00:00",
            "title": "A Survey on Dialogue Systems: Recent Advances and New Frontiers",
            "authors": [
                "Hongshen Chen",
                "Xiaorui Liu",
                "Dawei Yin",
                "Jiliang Tang"
            ],
            "abstract": "Dialogue systems have attracted more and more attention. Recent advances on\ndialogue systems are overwhelmingly contributed by deep learning techniques,\nwhich have been employed to enhance a wide range of big data applications such\nas computer vision, natural language processing, and recommender systems. For\ndialogue systems, deep learning can leverage a massive amount of data to learn\nmeaningful feature representations and response generation strategies, while\nrequiring a minimum amount of hand-crafting. In this article, we give an\noverview to these recent advances on dialogue systems from various perspectives\nand discuss some possible research directions. In particular, we generally\ndivide existing dialogue systems into task-oriented and non-task-oriented\nmodels, then detail how deep learning techniques help them with representative\nalgorithms and finally discuss some appealing research directions that can\nbring the dialogue system research into a new frontier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01804v2",
            "date": "2017-11-06 09:40:41+00:00",
            "title": "Evaluation of Croatian Word Embeddings",
            "authors": [
                "Lukas Svoboda",
                "Slobodan Beliga"
            ],
            "abstract": "Croatian is poorly resourced and highly inflected language from Slavic\nlanguage family. Nowadays, research is focusing mostly on English. We created a\nnew word analogy corpus based on the original English Word2vec word analogy\ncorpus and added some of the specific linguistic aspects from Croatian\nlanguage. Next, we created Croatian WordSim353 and RG65 corpora for a basic\nevaluation of word similarities. We compared created corpora on two popular\nword representation models, based on Word2Vec tool and fastText tool. Models\nhas been trained on 1.37B tokens training data corpus and tested on a new\nrobust Croatian word analogy corpus. Results show that models are able to\ncreate meaningful word representation. This research has shown that free word\norder and the higher morphological complexity of Croatian language influences\nthe quality of resulting word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01985v1",
            "date": "2017-11-03 14:52:08+00:00",
            "title": "Fine-tuning Tree-LSTM for phrase-level sentiment classification on a Polish dependency treebank. Submission to PolEval task 2",
            "authors": [
                "Tomasz Korbak",
                "Paulina \u017bak"
            ],
            "abstract": "We describe a variant of Child-Sum Tree-LSTM deep neural network (Tai et al,\n2015) fine-tuned for working with dependency trees and morphologically rich\nlanguages using the example of Polish. Fine-tuning included applying a custom\nregularization technique (zoneout, described by (Krueger et al., 2016), and\nfurther adapted for Tree-LSTMs) as well as using pre-trained word embeddings\nenhanced with sub-word information (Bojanowski et al., 2016). The system was\nimplemented in PyTorch and evaluated on phrase-level sentiment labeling task as\npart of the PolEval competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02085v3",
            "date": "2017-11-06 18:58:46+00:00",
            "title": "Neural Speed Reading via Skim-RNN",
            "authors": [
                "Minjoon Seo",
                "Sewon Min",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Inspired by the principles of speed reading, we introduce Skim-RNN, a\nrecurrent neural network (RNN) that dynamically decides to update only a small\nfraction of the hidden state for relatively unimportant input tokens. Skim-RNN\ngives computational advantage over an RNN that always updates the entire hidden\nstate. Skim-RNN uses the same input and output interfaces as a standard RNN and\ncan be easily used instead of RNNs in existing models. In our experiments, we\nshow that Skim-RNN can achieve significantly reduced computational cost without\nlosing accuracy compared to standard RNNs across five different natural\nlanguage tasks. In addition, we demonstrate that the trade-off between accuracy\nand speed of Skim-RNN can be dynamically controlled during inference time in a\nstable manner. Our analysis also shows that Skim-RNN running on a single CPU\noffers lower latency compared to standard RNNs on GPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02162v2",
            "date": "2017-11-06 20:30:50+00:00",
            "title": "TAMU at KBP 2017: Event Nugget Detection and Coreference Resolution",
            "authors": [
                "Prafulla Kumar Choubey",
                "Ruihong Huang"
            ],
            "abstract": "In this paper, we describe TAMU's system submitted to the TAC KBP 2017 event\nnugget detection and coreference resolution task. Our system builds on the\nstatistical and empirical observations made on training and development data.\nWe found that modifiers of event nuggets tend to have unique syntactic\ndistribution. Their parts-of-speech tags and dependency relations provides them\nessential characteristics that are useful in identifying their span and also\ndefining their types and realis status. We further found that the joint\nmodeling of event span detection and realis status identification performs\nbetter than the individual models for both tasks. Our simple system designed\nusing minimal features achieved the micro-average F1 scores of 57.72, 44.27 and\n42.47 for event span detection, type identification and realis status\nclassification tasks respectively. Also, our system achieved the CoNLL F1 score\nof 27.20 in event coreference resolution task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02207v1",
            "date": "2017-11-06 22:48:55+00:00",
            "title": "Towards Language-Universal End-to-End Speech Recognition",
            "authors": [
                "Suyoun Kim",
                "Michael L. Seltzer"
            ],
            "abstract": "Building speech recognizers in multiple languages typically involves\nreplicating a monolingual training recipe for each language, or utilizing a\nmulti-task learning approach where models for different languages have separate\noutput labels but share some internal parameters. In this work, we exploit\nrecent progress in end-to-end speech recognition to create a single\nmultilingual speech recognition system capable of recognizing any of the\nlanguages seen in training. To do so, we propose the use of a universal\ncharacter set that is shared among all languages. We also create a\nlanguage-specific gating mechanism within the network that can modulate the\nnetwork's internal representations in a language-specific way. We evaluate our\nproposed approach on the Microsoft Cortana task across three languages and show\nthat our system outperforms both the individual monolingual systems and systems\nbuilt with a multi-task learning approach. We also show that this model can be\nused to initialize a monolingual speech recognizer, and can be used to create a\nbilingual model for use in code-switching scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02212v2",
            "date": "2017-11-06 22:59:48+00:00",
            "title": "Improved training for online end-to-end speech recognition systems",
            "authors": [
                "Suyoun Kim",
                "Michael L. Seltzer",
                "Jinyu Li",
                "Rui Zhao"
            ],
            "abstract": "Achieving high accuracy with end-to-end speech recognizers requires careful\nparameter initialization prior to training. Otherwise, the networks may fail to\nfind a good local optimum. This is particularly true for online networks, such\nas unidirectional LSTMs. Currently, the best strategy to train such systems is\nto bootstrap the training from a tied-triphone system. However, this is time\nconsuming, and more importantly, is impossible for languages without a\nhigh-quality pronunciation lexicon. In this work, we propose an initialization\nstrategy that uses teacher-student learning to transfer knowledge from a large,\nwell-trained, offline end-to-end speech recognition model to an online\nend-to-end model, eliminating the need for a lexicon or any other linguistic\nresources. We also explore curriculum learning and label smoothing and show how\nthey can be combined with the proposed teacher-student learning for further\nimprovements. We evaluate our methods on a Microsoft Cortana personal assistant\ntask and show that the proposed method results in a 19 % relative improvement\nin word error rate compared to a randomly-initialized baseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02509v1",
            "date": "2017-11-06 06:21:01+00:00",
            "title": "Structure Regularized Bidirectional Recurrent Convolutional Neural Network for Relation Classification",
            "authors": [
                "Ji Wen"
            ],
            "abstract": "Relation classification is an important semantic processing task in the field\nof natural language processing (NLP). In this paper, we present a novel model,\nStructure Regularized Bidirectional Recurrent Convolutional Neural\nNetwork(SR-BRCNN), to classify the relation of two entities in a sentence, and\nthe new dataset of Chinese Sanwen for named entity recognition and relation\nclassification. Some state-of-the-art systems concentrate on modeling the\nshortest dependency path (SDP) between two entities leveraging convolutional or\nrecurrent neural networks. We further explore how to make full use of the\ndependency relations information in the SDP and how to improve the model by the\nmethod of structure regularization. We propose a structure regularized model to\nlearn relation representations along the SDP extracted from the forest formed\nby the structure regularized dependency tree, which benefits reducing the\ncomplexity of the whole model and helps improve the $F_{1}$ score by 10.3.\nExperimental results show that our method outperforms the state-of-the-art\napproaches on the Chinese Sanwen task and performs as well on the SemEval-2010\nTask 8 dataset\\footnote{The Chinese Sanwen corpus this paper developed and used\nwill be released in the further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02608v1",
            "date": "2017-11-07 17:01:55+00:00",
            "title": "Extractive Multi-document Summarization Using Multilayer Networks",
            "authors": [
                "Jorge V. Tohalino",
                "Diego R. Amancio"
            ],
            "abstract": "Huge volumes of textual information has been produced every single day. In\norder to organize and understand such large datasets, in recent years,\nsummarization techniques have become popular. These techniques aims at finding\nrelevant, concise and non-redundant content from such a big data. While network\nmethods have been adopted to model texts in some scenarios, a systematic\nevaluation of multilayer network models in the multi-document summarization\ntask has been limited to a few studies. Here, we evaluate the performance of a\nmultilayer-based method to select the most relevant sentences in the context of\nan extractive multi document summarization (MDS) task. In the adopted model,\nnodes represent sentences and edges are created based on the number of shared\nwords between sentences. Differently from previous studies in multi-document\nsummarization, we make a distinction between edges linking sentences from\ndifferent documents (inter-layer) and those connecting sentences from the same\ndocument (intra-layer). As a proof of principle, our results reveal that such a\ndiscrimination between intra- and inter-layer in a multilayered representation\nis able to improve the quality of the generated summaries. This piece of\ninformation could be used to improve current statistical methods and related\ntextual models.",
            "journal ref": "Physica A 503, 526-539 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02781v3",
            "date": "2017-11-08 00:57:39+00:00",
            "title": "RubyStar: A Non-Task-Oriented Mixture Model Dialog System",
            "authors": [
                "Huiting Liu",
                "Tao Lin",
                "Hanfei Sun",
                "Weijian Lin",
                "Chih-Wei Chang",
                "Teng Zhong",
                "Alexander Rudnicky"
            ],
            "abstract": "RubyStar is a dialog system designed to create \"human-like\" conversation by\ncombining different response generation strategies. RubyStar conducts a\nnon-task-oriented conversation on general topics by using an ensemble of\nrule-based, retrieval-based and generative methods. Topic detection, engagement\nmonitoring, and context tracking are used for managing interaction. Predictable\nelements of conversation, such as the bot's backstory and simple question\nanswering are handled by separate modules. We describe a rating scheme we\ndeveloped for evaluating response generation. We find that character-level RNN\nis an effective generation model for general responses, with proper parameter\nsettings; however other kinds of conversation topics might benefit from using\nother models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02918v2",
            "date": "2017-11-08 12:29:18+00:00",
            "title": "Improving Hypernymy Extraction with Distributional Semantic Classes",
            "authors": [
                "Alexander Panchenko",
                "Dmitry Ustalov",
                "Stefano Faralli",
                "Simone P. Ponzetto",
                "Chris Biemann"
            ],
            "abstract": "In this paper, we show how distributionally-induced semantic classes can be\nhelpful for extracting hypernyms. We present methods for inducing sense-aware\nsemantic classes using distributional semantics and using these induced\nsemantic classes for filtering noisy hypernymy relations. Denoising of\nhypernyms is performed by labeling each semantic class with its hypernyms. On\nthe one hand, this allows us to filter out wrong extractions using the global\nstructure of distributionally similar senses. On the other hand, we infer\nmissing hypernyms via label propagation to cluster terms. We conduct a\nlarge-scale crowdsourcing study showing that processing of automatically\nextracted hypernyms using our approach improves the quality of the hypernymy\nextraction in terms of both precision and recall. Furthermore, we show the\nutility of our method in the domain taxonomy induction task, achieving the\nstate-of-the-art results on a SemEval'16 task on taxonomy induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03226v2",
            "date": "2017-11-09 01:42:14+00:00",
            "title": "Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning",
            "authors": [
                "Meng Qu",
                "Xiang Ren",
                "Yu Zhang",
                "Jiawei Han"
            ],
            "abstract": "Extracting relations from text corpora is an important task in text mining.\nIt becomes particularly challenging when focusing on weakly-supervised relation\nextraction, that is, utilizing a few relation instances (i.e., a pair of\nentities and their relation) as seeds to extract more instances from corpora.\nExisting distributional approaches leverage the corpus-level co-occurrence\nstatistics of entities to predict their relations, and require large number of\nlabeled instances to learn effective relation classifiers. Alternatively,\npattern-based approaches perform bootstrapping or apply neural networks to\nmodel the local contexts, but still rely on large number of labeled instances\nto build reliable models. In this paper, we study integrating the\ndistributional and pattern-based methods in a weakly-supervised setting, such\nthat the two types of methods can provide complementary supervision for each\nother to build an effective, unified model. We propose a novel co-training\nframework with a distributional module and a pattern module. During training,\nthe distributional module helps the pattern module discriminate between the\ninformative patterns and other patterns, and the pattern module generates some\nhighly-confident instances to improve the distributional module. The whole\nframework can be effectively optimized by iterating between improving the\npattern module and updating the distributional module. We conduct experiments\non two tasks: knowledge base completion with text corpora and corpus-level\nrelation extraction. Experimental results prove the effectiveness of our\nframework in the weakly-supervised setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03230v1",
            "date": "2017-11-09 01:56:00+00:00",
            "title": "An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks",
            "authors": [
                "Yelong Shen",
                "Xiaodong Liu",
                "Kevin Duh",
                "Jianfeng Gao"
            ],
            "abstract": "Reading comprehension (RC) is a challenging task that requires synthesis of\ninformation across sentences and multiple turns of reasoning. Using a\nstate-of-the-art RC model, we empirically investigate the performance of\nsingle-turn and multiple-turn reasoning on the SQuAD and MS MARCO datasets. The\nRC model is an end-to-end neural network with iterative attention, and uses\nreinforcement learning to dynamically control the number of turns. We find that\nmultiple-turn reasoning outperforms single-turn reasoning for all question and\nanswer types; further, we observe that enabling a flexible number of turns\ngenerally improves upon a fixed multiple-turn strategy. %across all question\ntypes, and is particularly beneficial to questions with lengthy, descriptive\nanswers. We achieve results competitive to the state-of-the-art on these two\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03381v2",
            "date": "2017-11-09 14:10:45+00:00",
            "title": "Tracking of enriched dialog states for flexible conversational information access",
            "authors": [
                "Yinpei Dai",
                "Zhijian Ou",
                "Dawei Ren",
                "Pengfei Yu"
            ],
            "abstract": "Dialog state tracking (DST) is a crucial component in a task-oriented dialog\nsystem for conversational information access. A common practice in current\ndialog systems is to define the dialog state by a set of slot-value pairs. Such\nrepresentation of dialog states and the slot-filling based DST have been widely\nemployed, but suffer from three drawbacks. (1) The dialog state can contain\nonly a single value for a slot, and (2) can contain only users' affirmative\npreference over the values for a slot. (3) Current task-based dialog systems\nmainly focus on the searching task, while the enquiring task is also very\ncommon in practice. The above observations motivate us to enrich current\nrepresentation of dialog states and collect a brand new dialog dataset about\nmovies, based upon which we build a new DST, called enriched DST (EDST), for\nflexible accessing movie information. The EDST supports the searching task, the\nenquiring task and their mixed task. We show that the new EDST method not only\nachieves good results on Iqiyi dataset, but also outperforms other\nstate-of-the-art DST methods on the traditional dialog datasets, WOZ2.0 and\nDSTC2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03541v1",
            "date": "2017-11-09 06:29:28+00:00",
            "title": "Language Modeling for Code-Switched Data: Challenges and Approaches",
            "authors": [
                "Ganji Sreeram",
                "Rohit Sinha"
            ],
            "abstract": "Lately, the problem of code-switching has gained a lot of attention and has\nemerged as an active area of research. In bilingual communities, the speakers\ncommonly embed the words and phrases of a non-native language into the syntax\nof a native language in their day-to-day communications. The code-switching is\na global phenomenon among multilingual communities, still very limited acoustic\nand linguistic resources are available as yet. For developing effective speech\nbased applications, the ability of the existing language technologies to deal\nwith the code-switched data can not be over emphasized. The code-switching is\nbroadly classified into two modes: inter-sentential and intra-sentential\ncode-switching. In this work, we have studied the intra-sentential problem in\nthe context of code-switching language modeling task. The salient contributions\nof this paper includes: (i) the creation of Hindi-English code-switching text\ncorpus by crawling a few blogging sites educating about the usage of the\nInternet (ii) the exploration of the parts-of-speech features towards more\neffective modeling of Hindi-English code-switched data by the monolingual\nlanguage model (LM) trained on native (Hindi) language data, and (iii) the\nproposal of a novel textual factor referred to as the code-switch factor\n(CS-factor), which allows the LM to predict the code-switching instances. In\nthe context of recognition of the code-switching data, the substantial\nreduction in the PPL is achieved with the use of POS factors and also the\nproposed CS-factor provides independent as well as additive gain in the PPL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03602v2",
            "date": "2017-11-09 20:58:07+00:00",
            "title": "The Lifted Matrix-Space Model for Semantic Composition",
            "authors": [
                "WooJin Chung",
                "Sheng-Fu Wang",
                "Samuel R. Bowman"
            ],
            "abstract": "Tree-structured neural network architectures for sentence encoding draw\ninspiration from the approach to semantic composition generally seen in formal\nlinguistics, and have shown empirical improvements over comparable sequence\nmodels by doing so. Moreover, adding multiplicative interaction terms to the\ncomposition functions in these models can yield significant further\nimprovements. However, existing compositional approaches that adopt such a\npowerful composition function scale poorly, with parameter counts exploding as\nmodel dimension or vocabulary size grows. We introduce the Lifted Matrix-Space\nmodel, which uses a global transformation to map vector word embeddings to\nmatrices, which can then be composed via an operation based on matrix-matrix\nmultiplication. Its composition function effectively transmits a larger number\nof activations across layers with relatively few model parameters. We evaluate\nour model on the Stanford NLI corpus, the Multi-Genre NLI corpus, and the\nStanford Sentiment Treebank and find that it consistently outperforms TreeLSTM\n(Tai et al., 2015), the previous best known composition function for\ntree-structured models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03688v2",
            "date": "2017-11-10 04:35:14+00:00",
            "title": "Document Context Neural Machine Translation with Memory Networks",
            "authors": [
                "Sameen Maruf",
                "Gholamreza Haffari"
            ],
            "abstract": "We present a document-level neural machine translation model which takes both\nsource and target document context into account using memory networks. We model\nthe problem as a structured prediction problem with interdependencies among the\nobserved and hidden variables, i.e., the source sentences and their unobserved\ntarget translations in the document. The resulting structured prediction\nproblem is tackled with a neural translation model equipped with two memory\ncomponents, one each for the source and target side, to capture the documental\ninterdependencies. We train the model end-to-end, and propose an iterative\ndecoding algorithm based on block coordinate descent. Experimental results of\nEnglish translations from French, German, and Estonian documents show that our\nmodel is effective in exploiting both source and target document context, and\nstatistically significantly outperforms the previous work in terms of BLEU and\nMETEOR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03697v1",
            "date": "2017-11-10 05:27:44+00:00",
            "title": "Integrating User and Agent Models: A Deep Task-Oriented Dialogue System",
            "authors": [
                "Weiyan Wang",
                "Yuxiang WU",
                "Yu Zhang",
                "Zhongqi Lu",
                "Kaixiang Mo",
                "Qiang Yang"
            ],
            "abstract": "Task-oriented dialogue systems can efficiently serve a large number of\ncustomers and relieve people from tedious works. However, existing\ntask-oriented dialogue systems depend on handcrafted actions and states or\nextra semantic labels, which sometimes degrades user experience despite the\nintensive human intervention. Moreover, current user simulators have limited\nexpressive ability so that deep reinforcement Seq2Seq models have to rely on\nselfplay and only work in some special cases. To address those problems, we\npropose a uSer and Agent Model IntegrAtion (SAMIA) framework inspired by an\nobservation that the roles of the user and agent models are asymmetric.\nFirstly, this SAMIA framework model the user model as a Seq2Seq learning\nproblem instead of ranking or designing rules. Then the built user model is\nused as a leverage to train the agent model by deep reinforcement learning. In\nthe test phase, the output of the agent model is filtered by the user model to\nenhance the stability and robustness. Experiments on a real-world coffee\nordering dataset verify the effectiveness of the proposed SAMIA framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03754v1",
            "date": "2017-11-10 10:13:51+00:00",
            "title": "Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension",
            "authors": [
                "Todor Mihaylov",
                "Zornitsa Kozareva",
                "Anette Frank"
            ],
            "abstract": "Reading comprehension is a challenging task in natural language processing\nand requires a set of skills to be solved. While current approaches focus on\nsolving the task as a whole, in this paper, we propose to use a neural network\n`skill' transfer approach. We transfer knowledge from several lower-level\nlanguage tasks (skills) including textual entailment, named entity recognition,\nparaphrase detection and question type classification into the reading\ncomprehension model.\n  We conduct an empirical evaluation and show that transferring language skill\nknowledge leads to significant improvements for the task with much fewer steps\ncompared to the baseline model. We also show that the skill transfer approach\nis effective even with small amounts of training data. Another finding of this\nwork is that using token-wise deep label supervision for text classification\nimproves the performance of transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03759v3",
            "date": "2017-11-10 10:18:02+00:00",
            "title": "YEDDA: A Lightweight Collaborative Text Span Annotation Tool",
            "authors": [
                "Jie Yang",
                "Yue Zhang",
                "Linwei Li",
                "Xingxuan Li"
            ],
            "abstract": "In this paper, we introduce \\textsc{Yedda}, a lightweight but efficient and\ncomprehensive open-source tool for text span annotation. \\textsc{Yedda}\nprovides a systematic solution for text span annotation, ranging from\ncollaborative user annotation to administrator evaluation and analysis. It\novercomes the low efficiency of traditional text annotation tools by annotating\nentities through both command line and shortcut keys, which are configurable\nwith custom labels. \\textsc{Yedda} also gives intelligent recommendations by\nlearning the up-to-date annotated text. An administrator client is developed to\nevaluate annotation quality of multiple annotators and generate detailed\ncomparison report for each annotator pair. Experiments show that the proposed\nsystem can reduce the annotation time by half compared with existing annotation\ntools. And the annotation time can be further compressed by 16.47\\% through\nintelligent recommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03859v2",
            "date": "2017-11-10 15:14:49+00:00",
            "title": "Towards the Use of Deep Reinforcement Learning with Global Policy For Query-based Extractive Summarisation",
            "authors": [
                "Diego Molla"
            ],
            "abstract": "Supervised approaches for text summarisation suffer from the problem of\nmismatch between the target labels/scores of individual sentences and the\nevaluation score of the final summary. Reinforcement learning can solve this\nproblem by providing a learning mechanism that uses the score of the final\nsummary as a guide to determine the decisions made at the time of selection of\neach sentence. In this paper we present a proof-of-concept approach that\napplies a policy-gradient algorithm to learn a stochastic policy using an\nundiscounted reward. The method has been applied to a policy consisting of a\nsimple neural network and simple features. The resulting deep reinforcement\nlearning system is able to learn a global policy and obtain encouraging\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04075v3",
            "date": "2017-11-11 04:34:51+00:00",
            "title": "Towards Automated ICD Coding Using Deep Learning",
            "authors": [
                "Haoran Shi",
                "Pengtao Xie",
                "Zhiting Hu",
                "Ming Zhang",
                "Eric P. Xing"
            ],
            "abstract": "International Classification of Diseases(ICD) is an authoritative health care\nclassification system of different diseases and conditions for clinical and\nmanagement purposes. Considering the complicated and dedicated process to\nassign correct codes to each patient admission based on overall diagnosis, we\npropose a hierarchical deep learning model with attention mechanism which can\nautomatically assign ICD diagnostic codes given written diagnosis. We utilize\ncharacter-aware neural language models to generate hidden representations of\nwritten diagnosis descriptions and ICD codes, and design an attention mechanism\nto address the mismatch between the numbers of descriptions and corresponding\ncodes. Our experimental results show the strong potential of automated ICD\ncoding from diagnosis descriptions. Our best model achieves 0.53 and 0.90 of F1\nscore and area under curve of receiver operating characteristic respectively.\nThe result outperforms those achieved using character-unaware encoding method\nor without attention mechanism. It indicates that our proposed deep learning\nmodel can code automatically in a reasonable way and provide a framework for\ncomputer-auxiliary ICD coding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04115v1",
            "date": "2017-11-11 10:55:38+00:00",
            "title": "Discovering conversational topics and emotions associated with Demonetization tweets in India",
            "authors": [
                "Mitodru Niyogi",
                "Asim K. Pal"
            ],
            "abstract": "Social media platforms contain great wealth of information which provides us\nopportunities explore hidden patterns or unknown correlations, and understand\npeople's satisfaction with what they are discussing. As one showcase, in this\npaper, we summarize the data set of Twitter messages related to recent\ndemonetization of all Rs. 500 and Rs. 1000 notes in India and explore insights\nfrom Twitter's data. Our proposed system automatically extracts the popular\nlatent topics in conversations regarding demonetization discussed in Twitter\nvia the Latent Dirichlet Allocation (LDA) based topic model and also identifies\nthe correlated topics across different categories. Additionally, it also\ndiscovers people's opinions expressed through their tweets related to the event\nunder consideration via the emotion analyzer. The system also employs an\nintuitive and informative visualization to show the uncovered insight.\nFurthermore, we use an evaluation measure, Normalized Mutual Information (NMI),\nto select the best LDA models. The obtained LDA results show that the tool can\nbe effectively used to extract discussion topics and summarize them for further\nmanual analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04154v1",
            "date": "2017-11-11 15:35:22+00:00",
            "title": "Interpretable probabilistic embeddings: bridging the gap between topic models and neural networks",
            "authors": [
                "Anna Potapenko",
                "Artem Popov",
                "Konstantin Vorontsov"
            ],
            "abstract": "We consider probabilistic topic models and more recent word embedding\ntechniques from a perspective of learning hidden semantic representations.\nInspired by a striking similarity of the two approaches, we merge them and\nlearn probabilistic embeddings with online EM-algorithm on word co-occurrence\ndata. The resulting embeddings perform on par with Skip-Gram Negative Sampling\n(SGNS) on word similarity tasks and benefit in the interpretability of the\ncomponents. Next, we learn probabilistic document embeddings that outperform\nparagraph2vec on a document similarity task and require less memory and time\nfor training. Finally, we employ multimodal Additive Regularization of Topic\nModels (ARTM) to obtain a high sparsity and learn embeddings for other\nmodalities, such as timestamps and categories. We observe further improvement\nof word similarity performance and meaningful inter-modality similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04231v3",
            "date": "2017-11-12 03:35:48+00:00",
            "title": "Syntax-Directed Attention for Neural Machine Translation",
            "authors": [
                "Kehai Chen",
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Tiejun Zhao"
            ],
            "abstract": "Attention mechanism, including global attention and local attention, plays a\nkey role in neural machine translation (NMT). Global attention attends to all\nsource words for word prediction. In comparison, local attention selectively\nlooks at fixed-window source words. However, alignment weights for the current\ntarget word often decrease to the left and right by linear distance centering\non the aligned source position and neglect syntax-directed distance\nconstraints. In this paper, we extend local attention with syntax-distance\nconstraint, to focus on syntactically related source words with the predicted\ntarget word, thus learning a more effective context vector for word prediction.\nMoreover, we further propose a double context NMT architecture, which consists\nof a global context vector and a syntax-directed context vector over the global\nattention, to provide more translation performance for NMT from source\nrepresentation. The experiments on the large-scale Chinese-to-English and\nEnglish-to-Germen translation tasks show that the proposed approach achieves a\nsubstantial and significant improvement over the baseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04289v3",
            "date": "2017-11-12 13:12:19+00:00",
            "title": "Neural Natural Language Inference Models Enhanced with External Knowledge",
            "authors": [
                "Qian Chen",
                "Xiaodan Zhu",
                "Zhen-Hua Ling",
                "Diana Inkpen",
                "Si Wei"
            ],
            "abstract": "Modeling natural language inference is a very challenging task. With the\navailability of large annotated data, it has recently become feasible to train\ncomplex models such as neural-network-based inference models, which have shown\nto achieve the state-of-the-art performance. Although there exist relatively\nlarge annotated data, can machines learn all knowledge needed to perform\nnatural language inference (NLI) from these data? If not, how can\nneural-network-based NLI models benefit from external knowledge and how to\nbuild NLI models to leverage it? In this paper, we enrich the state-of-the-art\nneural natural language inference models with external knowledge. We\ndemonstrate that the proposed models improve neural NLI models to achieve the\nstate-of-the-art performance on the SNLI and MultiNLI datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04352v1",
            "date": "2017-11-12 20:40:25+00:00",
            "title": "Fast Reading Comprehension with ConvNets",
            "authors": [
                "Felix Wu",
                "Ni Lao",
                "John Blitzer",
                "Guandao Yang",
                "Kilian Weinberger"
            ],
            "abstract": "State-of-the-art deep reading comprehension models are dominated by recurrent\nneural nets. Their sequential nature is a natural fit for language, but it also\nprecludes parallelization within an instances and often becomes the bottleneck\nfor deploying such models to latency critical scenarios. This is particularly\nproblematic for longer texts. Here we present a convolutional architecture as\nan alternative to these recurrent architectures. Using simple dilated\nconvolutional units in place of recurrent ones, we achieve results comparable\nto the state of the art on two question answering tasks, while at the same time\nachieving up to two orders of magnitude speedups for question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04411v1",
            "date": "2017-11-13 03:47:52+00:00",
            "title": "Convolutional Neural Network with Word Embeddings for Chinese Word Segmentation",
            "authors": [
                "Chunqi Wang",
                "Bo Xu"
            ],
            "abstract": "Character-based sequence labeling framework is flexible and efficient for\nChinese word segmentation (CWS). Recently, many character-based neural models\nhave been applied to CWS. While they obtain good performance, they have two\nobvious weaknesses. The first is that they heavily rely on manually designed\nbigram feature, i.e. they are not good at capturing n-gram features\nautomatically. The second is that they make no use of full word information.\nFor the first weakness, we propose a convolutional neural model, which is able\nto capture rich n-gram features without any feature engineering. For the second\none, we propose an effective approach to integrate the proposed model with word\nembeddings. We evaluate the model on two benchmark datasets: PKU and MSR.\nWithout any feature engineering, the model obtains competitive performance --\n95.7% on PKU and 97.3% on MSR. Armed with word embeddings, the model achieves\nstate-of-the-art performance on both datasets -- 96.5% on PKU and 98.0% on MSR,\nwithout using any external labeled resource.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04457v1",
            "date": "2017-11-13 07:42:56+00:00",
            "title": "Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT",
            "authors": [
                "Yining Wang",
                "Long Zhou",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Neural machine translation (NMT), a new approach to machine translation, has\nbeen proved to outperform conventional statistical machine translation (SMT)\nacross a variety of language pairs. Translation is an open-vocabulary problem,\nbut most existing NMT systems operate with a fixed vocabulary, which causes the\nincapability of translating rare words. This problem can be alleviated by using\ndifferent translation granularities, such as character, subword and hybrid\nword-character. Translation involving Chinese is one of the most difficult\ntasks in machine translation, however, to the best of our knowledge, there has\nnot been any other work exploring which translation granularity is most\nsuitable for Chinese in NMT. In this paper, we conduct an extensive comparison\nusing Chinese-English NMT as a case study. Furthermore, we discuss the\nadvantages and disadvantages of various translation granularities in detail.\nOur experiments show that subword model performs best for Chinese-to-English\ntranslation with the vocabulary which is not so big while hybrid word-character\nmodel is most suitable for English-to-Chinese translation. Moreover,\nexperiments of different granularities show that Hybrid_BPE method can achieve\nbest result on Chinese-to-English translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04731v2",
            "date": "2017-11-13 17:59:26+00:00",
            "title": "Evaluating prose style transfer with the Bible",
            "authors": [
                "Keith Carlson",
                "Allen Riddell",
                "Daniel Rockmore"
            ],
            "abstract": "In the prose style transfer task a system, provided with text input and a\ntarget prose style, produces output which preserves the meaning of the input\ntext but alters the style. These systems require parallel data for evaluation\nof results and usually make use of parallel data for training. Currently, there\nare few publicly available corpora for this task. In this work, we identify a\nhigh-quality source of aligned, stylistically distinct text in different\nversions of the Bible. We provide a standardized split, into training,\ndevelopment and testing data, of the public domain versions in our corpus. This\ncorpus is highly parallel since many Bible versions are included. Sentences are\naligned due to the presence of chapter and verse numbers within all versions of\nthe text. In addition to the corpus, we present the results, as measured by the\nBLEU and PINC metrics, of several models trained on our data which can serve as\nbaselines for future research. While we present these data as a style transfer\ncorpus, we believe that it is of unmatched quality and may be useful for other\nnatural language tasks as well.",
            "journal ref": "Carlson, Keith, Allen Riddell, and Daniel Rockmore. \"Evaluating\n  prose style transfer with the Bible.\" Royal Society open science 5.10 (2018):\n  171920",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04805v2",
            "date": "2017-11-13 19:23:47+00:00",
            "title": "QuickEdit: Editing Text & Translations by Crossing Words Out",
            "authors": [
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "We propose a framework for computer-assisted text editing. It applies to\ntranslation post-editing and to paraphrasing. Our proposal relies on very\nsimple interactions: a human editor modifies a sentence by marking tokens they\nwould like the system to change. Our model then generates a new sentence which\nreformulates the initial sentence by avoiding marked words. The approach builds\nupon neural sequence-to-sequence modeling and introduces a neural network which\ntakes as input a sentence along with change markers. Our model is trained on\ntranslation bitext by simulating post-edits. We demonstrate the advantage of\nour approach for translation post-editing through simulated post-edits. We also\nevaluate our model for paraphrasing through a user study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04951v1",
            "date": "2017-11-14 05:19:45+00:00",
            "title": "From Word Segmentation to POS Tagging for Vietnamese",
            "authors": [
                "Dat Quoc Nguyen",
                "Thanh Vu",
                "Dai Quoc Nguyen",
                "Mark Dras",
                "Mark Johnson"
            ],
            "abstract": "This paper presents an empirical comparison of two strategies for Vietnamese\nPart-of-Speech (POS) tagging from unsegmented text: (i) a pipeline strategy\nwhere we consider the output of a word segmenter as the input of a POS tagger,\nand (ii) a joint strategy where we predict a combined segmentation and POS tag\nfor each syllable. We also make a comparison between state-of-the-art (SOTA)\nfeature-based and neural network-based models. On the benchmark Vietnamese\ntreebank (Nguyen et al., 2009), experimental results show that the pipeline\nstrategy produces better scores of POS tagging from unsegmented text than the\njoint strategy, and the highest accuracy is obtained by using a feature-based\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04956v5",
            "date": "2017-11-14 05:47:08+00:00",
            "title": "Classical Structured Prediction Losses for Sequence to Sequence Learning",
            "authors": [
                "Sergey Edunov",
                "Myle Ott",
                "Michael Auli",
                "David Grangier",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "There has been much recent work on training neural attention models at the\nsequence-level using either reinforcement learning-style methods or by\noptimizing the beam. In this paper, we survey a range of classical objective\nfunctions that have been widely used to train linear models for structured\nprediction and apply them to neural sequence to sequence models. Our\nexperiments show that these losses can perform surprisingly well by slightly\noutperforming beam search optimization in a like for like setup. We also report\nnew state of the art results on both IWSLT'14 German-English translation as\nwell as Gigaword abstractive summarization. On the larger WMT'14 English-French\ntranslation task, sequence-level training achieves 41.5 BLEU which is on par\nwith the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04964v2",
            "date": "2017-11-14 06:17:54+00:00",
            "title": "Dynamic Fusion Networks for Machine Reading Comprehension",
            "authors": [
                "Yichong Xu",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Yelong Shen",
                "Xiaodong Liu"
            ],
            "abstract": "This paper presents a novel neural model - Dynamic Fusion Network (DFN), for\nmachine reading comprehension (MRC). DFNs differ from most state-of-the-art\nmodels in their use of a dynamic multi-strategy attention process, in which\npassages, questions and answer candidates are jointly fused into attention\nvectors, along with a dynamic multi-step reasoning module for generating\nanswers. With the use of reinforcement learning, for each input sample that\nconsists of a question, a passage and a list of candidate answers, an instance\nof DFN with a sample-specific network architecture can be dynamically\nconstructed by determining what attention strategy to apply and how many\nreasoning steps to take. Experiments show that DFNs achieve the best result\nreported on RACE, a challenging MRC dataset that contains real human reading\nquestions in a wide variety of types. A detailed empirical analysis also\ndemonstrates that DFNs can produce attention vectors that summarize information\nfrom questions, passages and answer candidates more effectively than other\npopular MRC models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04987v3",
            "date": "2017-11-14 07:55:39+00:00",
            "title": "Unified Pragmatic Models for Generating and Following Instructions",
            "authors": [
                "Daniel Fried",
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "We show that explicit pragmatic inference aids in correctly generating and\nfollowing natural language instructions for complex, sequential tasks. Our\npragmatics-enabled models reason about why speakers produce certain\ninstructions, and about how listeners will react upon hearing them. Like\nprevious pragmatic models, we use learned base listener and speaker models to\nbuild a pragmatic speaker that uses the base listener to simulate the\ninterpretation of candidate descriptions, and a pragmatic listener that reasons\ncounterfactually about alternative descriptions. We extend these models to\ntasks with sequential structure. Evaluation of language generation and\ninterpretation shows that pragmatic inference improves state-of-the-art\nlistener models (at correctly interpreting human instructions) and speaker\nmodels (at producing instructions correctly interpreted by humans) in diverse\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05066v2",
            "date": "2017-11-14 12:00:36+00:00",
            "title": "Learning an Executable Neural Semantic Parser",
            "authors": [
                "Jianpeng Cheng",
                "Siva Reddy",
                "Vijay Saraswat",
                "Mirella Lapata"
            ],
            "abstract": "This paper describes a neural semantic parser that maps natural language\nutterances onto logical forms which can be executed against a task-specific\nenvironment, such as a knowledge base or a database, to produce a response. The\nparser generates tree-structured logical forms with a transition-based approach\nwhich combines a generic tree-generation algorithm with domain-general\noperations defined by the logical language. The generation process is modeled\nby structured recurrent neural networks, which provide a rich encoding of the\nsentential context and generation history for making predictions. To tackle\nmismatches between natural language and logical form tokens, various attention\nmechanisms are explored. Finally, we consider different training settings for\nthe neural semantic parser, including a fully supervised training where\nannotated logical forms are given, weakly-supervised training where denotations\nare provided, and distant supervision where only unlabeled sentences and a\nknowledge base are available. Experiments across a wide range of datasets\ndemonstrate the effectiveness of our parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05073v4",
            "date": "2017-11-14 12:13:44+00:00",
            "title": "DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications",
            "authors": [
                "Wei He",
                "Kai Liu",
                "Jing Liu",
                "Yajuan Lyu",
                "Shiqi Zhao",
                "Xinyan Xiao",
                "Yuan Liu",
                "Yizhong Wang",
                "Hua Wu",
                "Qiaoqiao She",
                "Xuan Liu",
                "Tian Wu",
                "Haifeng Wang"
            ],
            "abstract": "This paper introduces DuReader, a new large-scale, open-domain Chinese ma-\nchine reading comprehension (MRC) dataset, designed to address real-world MRC.\nDuReader has three advantages over previous MRC datasets: (1) data sources:\nquestions and documents are based on Baidu Search and Baidu Zhidao; answers are\nmanually generated. (2) question types: it provides rich annotations for more\nquestion types, especially yes-no and opinion questions, that leaves more\nopportunity for the research community. (3) scale: it contains 200K questions,\n420K answers and 1M documents; it is the largest Chinese MRC dataset so far.\nExperiments show that human performance is well above current state-of-the-art\nbaseline systems, leaving plenty of room for the community to make\nimprovements. To help the community make these improvements, both DuReader and\nbaseline systems have been posted online. We also organize a shared competition\nto encourage the exploration of more models. Since the release of the task,\nthere are significant improvements over the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05186v2",
            "date": "2017-11-14 16:50:40+00:00",
            "title": "False Positive and Cross-relation Signals in Distant Supervision Data",
            "authors": [
                "Anca Dumitrache",
                "Lora Aroyo",
                "Chris Welty"
            ],
            "abstract": "Distant supervision (DS) is a well-established method for relation extraction\nfrom text, based on the assumption that when a knowledge-base contains a\nrelation between a term pair, then sentences that contain that pair are likely\nto express the relation. In this paper, we use the results of a crowdsourcing\nrelation extraction task to identify two problems with DS data quality: the\nwidely varying degree of false positives across different relations, and the\nobserved causal connection between relations that are not considered by the DS\nmethod. The crowdsourcing data aggregation is performed using ambiguity-aware\nCrowdTruth metrics, that are used to capture and interpret inter-annotator\ndisagreement. We also present preliminary results of using the crowd to enhance\nDS training data for a relation classification model, without requiring the\ncrowd to annotate the entire set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05198v1",
            "date": "2017-11-14 17:05:51+00:00",
            "title": "Unsupervised patient representations from clinical notes with interpretable classification decisions",
            "authors": [
                "Madhumita Sushil",
                "Simon \u0160uster",
                "Kim Luyckx",
                "Walter Daelemans"
            ],
            "abstract": "We have two main contributions in this work: 1. We explore the usage of a\nstacked denoising autoencoder, and a paragraph vector model to learn\ntask-independent dense patient representations directly from clinical notes. We\nevaluate these representations by using them as features in multiple supervised\nsetups, and compare their performance with those of sparse representations. 2.\nTo understand and interpret the representations, we explore the best encoded\nfeatures within the patient representations obtained from the autoencoder\nmodel. Further, we calculate the significance of the input features of the\ntrained classifiers when we use these pretrained representations as input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05217v2",
            "date": "2017-11-14 17:30:26+00:00",
            "title": "Controllable Abstractive Summarization",
            "authors": [
                "Angela Fan",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "Current models for document summarization disregard user preferences such as\nthe desired length, style, the entities that the user might be interested in,\nor how much of the document the user has already read. We present a neural\nsummarization model with a simple but effective mechanism to enable users to\nspecify these high level attributes in order to control the shape of the final\nsummaries to better suit their needs. With user input, our system can produce\nhigh quality summaries that follow user preferences. Without user input, we set\nthe control variables automatically. On the full text CNN-Dailymail dataset, we\noutperform state of the art abstractive systems (both in terms of F1-ROUGE1\n40.38 vs. 39.53 and human evaluation).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05294v1",
            "date": "2017-11-14 19:42:55+00:00",
            "title": "Modeling Semantic Relatedness using Global Relation Vectors",
            "authors": [
                "Shoaib Jameel",
                "Zied Bouraoui",
                "Steven Schockaert"
            ],
            "abstract": "Word embedding models such as GloVe rely on co-occurrence statistics from a\nlarge corpus to learn vector representations of word meaning. These vectors\nhave proven to capture surprisingly fine-grained semantic and syntactic\ninformation. While we may similarly expect that co-occurrence statistics can be\nused to capture rich information about the relationships between different\nwords, existing approaches for modeling such relationships have mostly relied\non manipulating pre-trained word vectors. In this paper, we introduce a novel\nmethod which directly learns relation vectors from co-occurrence statistics. To\nthis end, we first introduce a variant of GloVe, in which there is an explicit\nconnection between word vectors and PMI weighted co-occurrence vectors. We then\nshow how relation vectors can be naturally embedded into the resulting vector\nspace.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05313v2",
            "date": "2017-11-14 21:07:38+00:00",
            "title": "Simulating Action Dynamics with Neural Process Networks",
            "authors": [
                "Antoine Bosselut",
                "Omer Levy",
                "Ari Holtzman",
                "Corin Ennis",
                "Dieter Fox",
                "Yejin Choi"
            ],
            "abstract": "Understanding procedural language requires anticipating the causal effects of\nactions, even when they are not explicitly stated. In this work, we introduce\nNeural Process Networks to understand procedural text through (neural)\nsimulation of action dynamics. Our model complements existing memory\narchitectures with dynamic entity tracking by explicitly modeling actions as\nstate transformers. The model updates the states of the entities by executing\nlearned action operators. Empirical results demonstrate that our proposed model\ncan reason about the unstated causal effects of actions, allowing it to provide\nmore accurate contextual information for understanding and generating\nprocedural text, all while offering more interpretable internal representations\nthan existing alternatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05345v3",
            "date": "2017-11-14 22:57:24+00:00",
            "title": "Supervised and Unsupervised Transfer Learning for Question Answering",
            "authors": [
                "Yu-An Chung",
                "Hung-Yi Lee",
                "James Glass"
            ],
            "abstract": "Although transfer learning has been shown to be successful for tasks like\nobject and speech recognition, its applicability to question answering (QA) has\nyet to be well-studied. In this paper, we conduct extensive experiments to\ninvestigate the transferability of knowledge learned from a source QA dataset\nto a target dataset using two QA models. The performance of both models on a\nTOEFL listening comprehension test (Tseng et al., 2016) and MCTest (Richardson\net al., 2013) is significantly improved via a simple transfer learning\ntechnique from MovieQA (Tapaswi et al., 2016). In particular, one of the models\nachieves the state-of-the-art on all target datasets; for the TOEFL listening\ncomprehension test, it outperforms the previous best model by 7%. Finally, we\nshow that transfer learning is helpful even in unsupervised scenarios when\ncorrect answers for target QA dataset examples are not available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05350v1",
            "date": "2017-11-14 23:10:59+00:00",
            "title": "A Deep Learning Approach for Expert Identification in Question Answering Communities",
            "authors": [
                "Chen Zheng",
                "Shuangfei Zhai",
                "Zhongfei Zhang"
            ],
            "abstract": "In this paper, we describe an effective convolutional neural network\nframework for identifying the expert in question answering community. This\napproach uses the convolutional neural network and combines user feature\nrepresentations with question feature representations to compute scores that\nthe user who gets the highest score is the expert on this question. Unlike\nprior work, this method does not measure expert based on measure answer content\nquality to identify the expert but only require question sentence and user\nembedding feature to identify the expert. Remarkably, Our model can be applied\nto different languages and different domains. The proposed framework is trained\non two datasets, The first dataset is Stack Overflow and the second one is\nZhihu. The Top-1 accuracy results of our experiments show that our framework\noutperforms the best baseline framework for expert identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05380v4",
            "date": "2017-11-15 02:12:34+00:00",
            "title": "Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings",
            "authors": [
                "Shaohui Kuang",
                "Junhui Li",
                "Ant\u00f3nio Branco",
                "Weihua Luo",
                "Deyi Xiong"
            ],
            "abstract": "In neural machine translation, a source sequence of words is encoded into a\nvector from which a target sequence is generated in the decoding phase.\nDifferently from statistical machine translation, the associations between\nsource words and their possible target counterparts are not explicitly stored.\nSource and target words are at the two ends of a long information processing\nprocedure, mediated by hidden states at both the source encoding and the target\ndecoding phases. This makes it possible that a source word is incorrectly\ntranslated into a target word that is not any of its admissible equivalent\ncounterparts in the target language.\n  In this paper, we seek to somewhat shorten the distance between source and\ntarget words in that procedure, and thus strengthen their association, by means\nof a method we term bridging source and target word embeddings. We experiment\nwith three strategies: (1) a source-side bridging model, where source word\nembeddings are moved one step closer to the output target sequence; (2) a\ntarget-side bridging model, which explores the more relevant source word\nembeddings for the prediction of the target sequence; and (3) a direct bridging\nmodel, which directly connects source and target word embeddings seeking to\nminimize errors in the translation of ones by the others.\n  Experiments and analysis presented in this paper demonstrate that the\nproposed bridging models are able to significantly improve quality of both\nsentence translation, in general, and alignment and translation of individual\nsource words with target words, in particular.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05433v1",
            "date": "2017-11-15 07:13:13+00:00",
            "title": "A Sequential Neural Encoder with Latent Structured Description for Modeling Sentences",
            "authors": [
                "Yu-Ping Ruan",
                "Qian Chen",
                "Zhen-Hua Ling"
            ],
            "abstract": "In this paper, we propose a sequential neural encoder with latent structured\ndescription (SNELSD) for modeling sentences. This model introduces latent\nchunk-level representations into conventional sequential neural encoders, i.e.,\nrecurrent neural networks (RNNs) with long short-term memory (LSTM) units, to\nconsider the compositionality of languages in semantic modeling. An SNELSD\nmodel has a hierarchical structure that includes a detection layer and a\ndescription layer. The detection layer predicts the boundaries of latent word\nchunks in an input sentence and derives a chunk-level vector for each word. The\ndescription layer utilizes modified LSTM units to process these chunk-level\nvectors in a recurrent manner and produces sequential encoding outputs. These\noutput vectors are further concatenated with word vectors or the outputs of a\nchain LSTM encoder to obtain the final sentence representation. All the model\nparameters are learned in an end-to-end manner without a dependency on\nadditional text chunking or syntax parsing. A natural language inference (NLI)\ntask and a sentiment analysis (SA) task are adopted to evaluate the performance\nof our proposed model. The experimental results demonstrate the effectiveness\nof the proposed SNELSD model on exploring task-dependent chunking patterns\nduring the semantic modeling of sentences. Furthermore, the proposed method\nachieves better performance than conventional chain LSTMs and tree-structured\nLSTMs on both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05467v1",
            "date": "2017-11-15 09:17:34+00:00",
            "title": "Aicyber's System for NLPCC 2017 Shared Task 2: Voting of Baselines",
            "authors": [
                "Du Steven",
                "Xi Zhang"
            ],
            "abstract": "This paper presents Aicyber's system for NLPCC 2017 shared task 2. It is\nformed by a voting of three deep learning based system trained on\ncharacter-enhanced word vectors and a well known bag-of-word model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05468v1",
            "date": "2017-11-15 09:20:43+00:00",
            "title": "Tracking Typological Traits of Uralic Languages in Distributed Language Representations",
            "authors": [
                "Johannes Bjerva",
                "Isabelle Augenstein"
            ],
            "abstract": "Although linguistic typology has a long history, computational approaches\nhave only recently gained popularity. The use of distributed representations in\ncomputational linguistics has also become increasingly popular. A recent\ndevelopment is to learn distributed representations of language, such that\ntypologically similar languages are spatially close to one another. Although\nempirical successes have been shown for such language representations, they\nhave not been subjected to much typological probing. In this paper, we first\nlook at whether this type of language representations are empirically useful\nfor model transfer between Uralic languages in deep neural networks. We then\ninvestigate which typological features are encoded in these representations by\nattempting to predict features in the World Atlas of Language Structures, at\nvarious stages of fine-tuning of the representations. We focus on Uralic\nlanguages, and find that some typological traits can be automatically inferred\nwith accuracies well above a strong baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05516v2",
            "date": "2017-11-15 12:18:44+00:00",
            "title": "Investigating Inner Properties of Multimodal Representation and Semantic Compositionality with Brain-based Componential Semantics",
            "authors": [
                "Shaonan Wang",
                "Jiajun Zhang",
                "Nan Lin",
                "Chengqing Zong"
            ],
            "abstract": "Multimodal models have been proven to outperform text-based approaches on\nlearning semantic representations. However, it still remains unclear what\nproperties are encoded in multimodal representations, in what aspects do they\noutperform the single-modality representations, and what happened in the\nprocess of semantic compositionality in different input modalities. Considering\nthat multimodal models are originally motivated by human concept\nrepresentations, we assume that correlating multimodal representations with\nbrain-based semantics would interpret their inner properties to answer the\nabove questions. To that end, we propose simple interpretation methods based on\nbrain-based componential semantics. First we investigate the inner properties\nof multimodal representations by correlating them with corresponding\nbrain-based property vectors. Then we map the distributed vector space to the\ninterpretable brain-based componential space to explore the inner properties of\nsemantic compositionality. Ultimately, the present paper sheds light on the\nfundamental questions of natural language understanding, such as how to\nrepresent the meaning of words and how to combine word meanings into larger\nunits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05538v1",
            "date": "2017-11-15 12:42:48+00:00",
            "title": "Detecting and assessing contextual change in diachronic text documents using context volatility",
            "authors": [
                "Christian Kahmann",
                "Andreas Niekler",
                "Gerhard Heyer"
            ],
            "abstract": "Terms in diachronic text corpora may exhibit a high degree of semantic\ndynamics that is only partially captured by the common notion of semantic\nchange. The new measure of context volatility that we propose models the degree\nby which terms change context in a text collection over time. The computation\nof context volatility for a word relies on the significance-values of its\nco-occurrent terms and the corresponding co-occurrence ranks in sequential time\nspans. We define a baseline and present an efficient computational approach in\norder to overcome problems related to computational issues in the data\nstructure. Results are evaluated both, on synthetic documents that are used to\nsimulate contextual changes, and a real example based on British newspaper\ntexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05568v1",
            "date": "2017-11-15 13:43:02+00:00",
            "title": "Dialogue Act Recognition via CRF-Attentive Structured Network",
            "authors": [
                "Zheqian Chen",
                "Rongqin Yang",
                "Zhou Zhao",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "Dialogue Act Recognition (DAR) is a challenging problem in dialogue\ninterpretation, which aims to attach semantic labels to utterances and\ncharacterize the speaker's intention. Currently, many existing approaches\nformulate the DAR problem ranging from multi-classification to structured\nprediction, which suffer from handcrafted feature extensions and attentive\ncontextual structural dependencies. In this paper, we consider the problem of\nDAR from the viewpoint of extending richer Conditional Random Field (CRF)\nstructural dependencies without abandoning end-to-end training. We incorporate\nhierarchical semantic inference with memory mechanism on the utterance\nmodeling. We then extend structured attention network to the linear-chain\nconditional random field layer which takes into account both contextual\nutterances and corresponding dialogue acts. The extensive experiments on two\nmajor benchmark datasets Switchboard Dialogue Act (SWDA) and Meeting Recorder\nDialogue Act (MRDA) datasets show that our method achieves better performance\nthan other state-of-the-art solutions to the problem. It is a remarkable fact\nthat our method is nearly close to the human annotator's performance on SWDA\nwithin 2% gap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05603v1",
            "date": "2017-11-15 14:51:20+00:00",
            "title": "Words are Malleable: Computing Semantic Shifts in Political and Media Discourse",
            "authors": [
                "Hosein Azarbonyad",
                "Mostafa Dehghani",
                "Kaspar Beelen",
                "Alexandra Arkut",
                "Maarten Marx",
                "Jaap Kamps"
            ],
            "abstract": "Recently, researchers started to pay attention to the detection of temporal\nshifts in the meaning of words. However, most (if not all) of these approaches\nrestricted their efforts to uncovering change over time, thus neglecting other\nvaluable dimensions such as social or political variability. We propose an\napproach for detecting semantic shifts between different viewpoints--broadly\ndefined as a set of texts that share a specific metadata feature, which can be\na time-period, but also a social entity such as a political party. For each\nviewpoint, we learn a semantic space in which each word is represented as a low\ndimensional neural embedded vector. The challenge is to compare the meaning of\na word in one space to its meaning in another space and measure the size of the\nsemantic shifts. We compare the effectiveness of a measure based on optimal\ntransformations between the two spaces with a measure based on the similarity\nof the neighbors of the word in the respective spaces. Our experiments\ndemonstrate that the combination of these two performs best. We show that the\nsemantic shifts not only occur over time, but also along different viewpoints\nin a short period of time. For evaluation, we demonstrate how this approach\ncaptures meaningful semantic shifts and can help improve other tasks such as\nthe contrastive viewpoint summarization and ideology detection (measured as\nclassification accuracy) in political texts. We also show that the two laws of\nsemantic change which were empirically shown to hold for temporal shifts also\nhold for shifts across viewpoints. These laws state that frequent words are\nless likely to shift meaning while words with many senses are more likely to do\nso.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05678v1",
            "date": "2017-11-15 17:14:44+00:00",
            "title": "Unsupervised Morphological Expansion of Small Datasets for Improving Word Embeddings",
            "authors": [
                "Syed Sarfaraz Akhtar",
                "Arihant Gupta",
                "Avijit Vajpayee",
                "Arjit Srivastava",
                "Manish Shrivastava"
            ],
            "abstract": "We present a language independent, unsupervised method for building word\nembeddings using morphological expansion of text. Our model handles the problem\nof data sparsity and yields improved word embeddings by relying on training\nword embeddings on artificially generated sentences. We evaluate our method\nusing small sized training sets on eleven test sets for the word similarity\ntask across seven languages. Further, for English, we evaluated the impacts of\nour approach using a large training set on three standard test sets. Our method\nimproved results across all languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05680v2",
            "date": "2017-11-15 17:15:05+00:00",
            "title": "An Unsupervised Approach for Mapping between Vector Spaces",
            "authors": [
                "Syed Sarfaraz Akhtar",
                "Arihant Gupta",
                "Avijit Vajpayee",
                "Arjit Srivastava",
                "Madan Gopal Jhawar",
                "Manish Shrivastava"
            ],
            "abstract": "We present a language independent, unsupervised approach for transforming\nword embeddings from source language to target language using a transformation\nmatrix. Our model handles the problem of data scarcity which is faced by many\nlanguages in the world and yields improved word embeddings for words in the\ntarget language by relying on transformed embeddings of words of the source\nlanguage. We initially evaluate our approach via word similarity tasks on a\nsimilar language pair - Hindi as source and Urdu as the target language, while\nwe also evaluate our method on French and German as target languages and\nEnglish as source language. Our approach improves the current state of the art\nresults - by 13% for French and 19% for German. For Urdu, we saw an increment\nof 16% over our initial baseline score. We further explore the prospects of our\napproach by applying it on multiple models of the same language and\ntransferring words between the two models, thus solving the problem of missing\nwords in a model. We evaluate this on word similarity and word analogy tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05732v2",
            "date": "2017-11-15 18:59:29+00:00",
            "title": "ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations",
            "authors": [
                "John Wieting",
                "Kevin Gimpel"
            ],
            "abstract": "We describe PARANMT-50M, a dataset of more than 50 million English-English\nsentential paraphrase pairs. We generated the pairs automatically by using\nneural machine translation to translate the non-English side of a large\nparallel corpus, following Wieting et al. (2017). Our hope is that ParaNMT-50M\ncan be a valuable resource for paraphrase generation and can provide a rich\nsource of semantic knowledge to improve downstream natural language\nunderstanding tasks. To show its utility, we use ParaNMT-50M to train\nparaphrastic sentence embeddings that outperform all supervised systems on\nevery SemEval semantic textual similarity competition, in addition to showing\nhow it can be used for paraphrase generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05780v2",
            "date": "2017-11-15 19:56:24+00:00",
            "title": "Detecting Egregious Conversations between Customers and Virtual Agents",
            "authors": [
                "Tommy Sandbank",
                "Michal Shmueli-Scheuer",
                "Jonathan Herzig",
                "David Konopnicki",
                "John Richards",
                "David Piorkowski"
            ],
            "abstract": "Virtual agents are becoming a prominent channel of interaction in customer\nservice. Not all customer interactions are smooth, however, and some can become\nalmost comically bad. In such instances, a human agent might need to step in\nand salvage the conversation. Detecting bad conversations is important since\ndisappointing customer service may threaten customer loyalty and impact\nrevenue. In this paper, we outline an approach to detecting such egregious\nconversations, using behavioral cues from the user, patterns in agent\nresponses, and user-agent interaction. Using logs of two commercial systems, we\nshow that using these features improves the detection F1-score by around 20%\nover using textual features alone. In addition, we show that those features are\ncommon across two quite different domains and, arguably, universal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05885v1",
            "date": "2017-11-16 01:53:19+00:00",
            "title": "Crowdsourcing Question-Answer Meaning Representations",
            "authors": [
                "Julian Michael",
                "Gabriel Stanovsky",
                "Luheng He",
                "Ido Dagan",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce Question-Answer Meaning Representations (QAMRs), which represent\nthe predicate-argument structure of a sentence as a set of question-answer\npairs. We also develop a crowdsourcing scheme to show that QAMRs can be labeled\nwith very little training, and gather a dataset with over 5,000 sentences and\n100,000 questions. A detailed qualitative analysis demonstrates that the\ncrowd-generated question-answer pairs cover the vast majority of\npredicate-argument relationships in existing datasets (including PropBank,\nNomBank, QA-SRL, and AMR) along with many previously under-resourced ones,\nincluding implicit arguments and relations. The QAMR data and annotation code\nis made publicly available to enable future work on how best to model these\ncomplex phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06061v2",
            "date": "2017-11-16 12:21:57+00:00",
            "title": "An Encoder-Decoder Framework Translating Natural Language to Database Queries",
            "authors": [
                "Ruichu Cai",
                "Boyan Xu",
                "Xiaoyan Yang",
                "Zhenjie Zhang",
                "Zijian Li",
                "Zhihao Liang"
            ],
            "abstract": "Machine translation is going through a radical revolution, driven by the\nexplosive development of deep learning techniques using Convolutional Neural\nNetwork (CNN) and Recurrent Neural Network (RNN). In this paper, we consider a\nspecial case in machine translation problems, targeting to convert natural\nlanguage into Structured Query Language (SQL) for data retrieval over\nrelational database. Although generic CNN and RNN learn the grammar structure\nof SQL when trained with sufficient samples, the accuracy and training\nefficiency of the model could be dramatically improved, when the translation\nmodel is deeply integrated with the grammar rules of SQL. We present a new\nencoder-decoder framework, with a suite of new approaches, including new\nsemantic features fed into the encoder, grammar-aware states injected into the\nmemory of decoder, as well as recursive state management for sub-queries. These\ntechniques help the neural network better focus on understanding semantics of\noperations in natural language and save the efforts on SQL grammar learning.\nThe empirical evaluation on real world database and queries show that our\napproach outperform state-of-the-art solution by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06141v2",
            "date": "2017-11-16 15:27:53+00:00",
            "title": "ConvAMR: Abstract meaning representation parsing for legal document",
            "authors": [
                "Lai Dac Viet",
                "Vu Trong Sinh",
                "Nguyen Le Minh",
                "Ken Satoh"
            ],
            "abstract": "Convolutional neural networks (CNN) have recently achieved remarkable\nperformance in a wide range of applications. In this research, we equip\nconvolutional sequence-to-sequence (seq2seq) model with an efficient graph\nlinearization technique for abstract meaning representation parsing. Our\nlinearization method is better than the prior method at signaling the turn of\ngraph traveling. Additionally, convolutional seq2seq model is more appropriate\nand considerably faster than the recurrent neural network models in this task.\nOur method outperforms previous methods by a large margin on both the standard\ndataset LDC2014T12. Our result indicates that future works still have a room\nfor improving parsing model using graph linearization approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06238v2",
            "date": "2017-11-16 18:34:16+00:00",
            "title": "A Generative Approach to Question Answering",
            "authors": [
                "Rajarshee Mitra"
            ],
            "abstract": "Question Answering has come a long way from answer sentence selection,\nrelational QA to reading and comprehension. We shift our attention to\ngenerative question answering (gQA) by which we facilitate machine to read\npassages and answer questions by learning to generate the answers. We frame the\nproblem as a generative task where the encoder being a network that models the\nrelationship between question and passage and encoding them to a vector thus\nfacilitating the decoder to directly form an abstraction of the answer. Not\nbeing able to retain facts and making repetitions are common mistakes that\naffect the overall legibility of answers. To counter these issues, we employ\ncopying mechanism and maintenance of coverage vector in our model respectively.\nOur results on MS-MARCO demonstrate it's superiority over baselines and we also\nshow qualitative examples where we improved in terms of correctness and\nreadability",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06729v1",
            "date": "2017-11-17 21:17:20+00:00",
            "title": "Phonological (un)certainty weights lexical activation",
            "authors": [
                "Laura Gwilliams",
                "David Poeppel",
                "Alec Marantz",
                "Tal Linzen"
            ],
            "abstract": "Spoken word recognition involves at least two basic computations. First is\nmatching acoustic input to phonological categories (e.g. /b/, /p/, /d/). Second\nis activating words consistent with those phonological categories. Here we test\nthe hypothesis that the listener's probability distribution over lexical items\nis weighted by the outcome of both computations: uncertainty about phonological\ndiscretisation and the frequency of the selected word(s). To test this, we\nrecord neural responses in auditory cortex using magnetoencephalography, and\nmodel this activity as a function of the size and relative activation of\nlexical candidates. Our findings indicate that towards the beginning of a word,\nthe processing system indeed weights lexical candidates by both phonological\ncertainty and lexical frequency; however, later into the word, activation is\nweighted by frequency alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06826v1",
            "date": "2017-11-18 07:52:40+00:00",
            "title": "Low-dimensional Embeddings for Interpretable Anchor-based Topic Inference",
            "authors": [
                "Moontae Lee",
                "David Mimno"
            ],
            "abstract": "The anchor words algorithm performs provably efficient topic model inference\nby finding an approximate convex hull in a high-dimensional word co-occurrence\nspace. However, the existing greedy algorithm often selects poor anchor words,\nreducing topic quality and interpretability. Rather than finding an approximate\nconvex hull in a high-dimensional space, we propose to find an exact convex\nhull in a visualizable 2- or 3-dimensional space. Such low-dimensional\nembeddings both improve topics and clearly show users why the algorithm selects\ncertain words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06861v2",
            "date": "2017-11-18 13:33:15+00:00",
            "title": "Style Transfer in Text: Exploration and Evaluation",
            "authors": [
                "Zhenxin Fu",
                "Xiaoye Tan",
                "Nanyun Peng",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Style transfer is an important problem in natural language processing (NLP).\nHowever, the progress in language style transfer is lagged behind other\ndomains, such as computer vision, mainly because of the lack of parallel data\nand principle evaluation metrics. In this paper, we propose to learn style\ntransfer with non-parallel data. We explore two models to achieve this goal,\nand the key idea behind the proposed models is to learn separate content\nrepresentations and style representations using adversarial networks. We also\npropose novel evaluation metrics which measure two aspects of style transfer:\ntransfer strength and content preservation. We access our models and the\nevaluation metrics on two tasks: paper-news title transfer, and\npositive-negative review transfer. Results show that the proposed content\npreservation metric is highly correlate to human judgments, and the proposed\nmodels are able to generate sentences with higher style transfer strength and\nsimilar content preservation score comparing to auto-encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06872v2",
            "date": "2017-11-18 15:28:17+00:00",
            "title": "Automatically Extracting Action Graphs from Materials Science Synthesis Procedures",
            "authors": [
                "Sheshera Mysore",
                "Edward Kim",
                "Emma Strubell",
                "Ao Liu",
                "Haw-Shiuan Chang",
                "Srikrishna Kompella",
                "Kevin Huang",
                "Andrew McCallum",
                "Elsa Olivetti"
            ],
            "abstract": "Computational synthesis planning approaches have achieved recent success in\norganic chemistry, where tabulated synthesis procedures are readily available\nfor supervised learning. The syntheses of inorganic materials, however, exist\nprimarily as natural language narratives contained within scientific journal\narticles. This synthesis information must first be extracted from the text in\norder to enable analogous synthesis planning methods for inorganic materials.\nIn this work, we present a system for automatically extracting structured\nrepresentations of synthesis procedures from the texts of materials science\njournal articles that describe explicit, experimental syntheses of inorganic\ncompounds. We define the structured representation as a set of linked events\nmade up of extracted scientific entities and evaluate two unsupervised\napproaches for extracting these structures on expert-annotated articles: a\nstrong heuristic baseline and a generative model of procedural text. We also\nevaluate a variety of supervised models for extracting scientific entities. Our\nresults provide insight into the nature of the data and directions for further\nwork in this exciting new area of research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06895v1",
            "date": "2017-11-18 17:01:24+00:00",
            "title": "Is China Entering WTO or shijie maoyi zuzhi--a Corpus Study of English Acronyms in Chinese Newspapers",
            "authors": [
                "Hai Hu"
            ],
            "abstract": "This is one of the first studies that quantitatively examine the usage of\nEnglish acronyms (e.g. WTO) in Chinese texts. Using newspaper corpora, I try to\nanswer 1) for all instances of a concept that has an English acronym (e.g.\nWorld Trade Organization), what percentage is expressed in the English acronym\n(WTO), and what percentage in its Chinese translation (shijie maoyi zuzhi), and\n2) what factors are at play in language users' choice between the English and\nChinese forms? Results show that different concepts have different percentage\nfor English acronyms (PercentOfEn), ranging from 2% to 98%. Linear models show\nthat PercentOfEn for individual concepts can be predicted by language economy\n(how long the Chinese translation is), concept frequency, and whether the first\nappearance of the concept in Chinese newspapers is the English acronym or its\nChinese translation (all p < .05).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07010v5",
            "date": "2017-11-19 12:29:59+00:00",
            "title": "A Discourse-Level Named Entity Recognition and Relation Extraction Dataset for Chinese Literature Text",
            "authors": [
                "Jingjing Xu",
                "Ji Wen",
                "Xu Sun",
                "Qi Su"
            ],
            "abstract": "Named Entity Recognition and Relation Extraction for Chinese literature text\nis regarded as the highly difficult problem, partially because of the lack of\ntagging sets. In this paper, we build a discourse-level dataset from hundreds\nof Chinese literature articles for improving this task. To build a high quality\ndataset, we propose two tagging methods to solve the problem of data\ninconsistency, including a heuristic tagging method and a machine auxiliary\ntagging method. Based on this corpus, we also introduce several widely used\nmodels to conduct experiments. Experimental results not only show the\nusefulness of the proposed dataset, but also provide baselines for further\nresearch. The dataset is available at\nhttps://github.com/lancopku/Chinese-Literature-NER-RE-Dataset",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07019v2",
            "date": "2017-11-19 13:57:30+00:00",
            "title": "Incorporating Syntactic Uncertainty in Neural Machine Translation with Forest-to-Sequence Model",
            "authors": [
                "Poorya Zaremoodi",
                "Gholamreza Haffari"
            ],
            "abstract": "Incorporating syntactic information in Neural Machine Translation models is a\nmethod to compensate their requirement for a large amount of parallel training\ntext, especially for low-resource language pairs. Previous works on using\nsyntactic information provided by (inevitably error-prone) parsers has been\npromising. In this paper, we propose a forest-to-sequence Attentional Neural\nMachine Translation model to make use of exponentially many parse trees of the\nsource sentence to compensate for the parser errors. Our method represents the\ncollection of parse trees as a packed forest, and learns a neural attentional\ntransduction model from the forest to the target sentence. Experiments on\nEnglish to German, Chinese and Persian translation show the superiority of our\nmethod over the tree-to-sequence and vanilla sequence-to-sequence neural\ntranslation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07265v1",
            "date": "2017-11-20 11:54:59+00:00",
            "title": "Fast BTG-Forest-Based Hierarchical Sub-sentential Alignment",
            "authors": [
                "Hao Wang",
                "Yves Lepage"
            ],
            "abstract": "In this paper, we propose a novel BTG-forest-based alignment method. Based on\na fast unsupervised initialization of parameters using variational IBM models,\nwe synchronously parse parallel sentences top-down and align hierarchically\nunder the constraint of BTG. Our two-step method can achieve the same run-time\nand comparable translation performance as fast_align while it yields smaller\nphrase tables. Final SMT results show that our method even outperforms in the\nexperiment of distantly related languages, e.g., English-Japanese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07404v1",
            "date": "2017-11-20 16:47:28+00:00",
            "title": "Non-Contextual Modeling of Sarcasm using a Neural Network Benchmark",
            "authors": [
                "N. Dianna Radpour",
                "Vinay Ashokkumar"
            ],
            "abstract": "One of the most crucial components of natural human-robot interaction is\nartificial intuition and its influence on dialog systems. The intuitive\ncapability that humans have is undeniably extraordinary, and so remains one of\nthe greatest challenges for natural communicative dialogue between humans and\nrobots. In this paper, we introduce a novel probabilistic modeling framework of\nidentifying, classifying and learning features of sarcastic text via training a\nneural network with human-informed sarcastic benchmarks. This is necessary for\nestablishing a comprehensive sentiment analysis schema that is sensitive to the\nnuances of sarcasm-ridden text by being trained on linguistic cues. We show\nthat our model provides a good fit for this type of real-world informed data,\nwith potential to achieve as accurate, if not more, than alternatives. Though\nthe implementation and benchmarking is an extensive task, it can be extended\nvia the same method that we present to capture different forms of nuances in\ncommunication and making for much more natural and engaging dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07611v1",
            "date": "2017-11-21 03:04:02+00:00",
            "title": "Event Representations with Tensor-based Compositions",
            "authors": [
                "Noah Weber",
                "Niranjan Balasubramanian",
                "Nathanael Chambers"
            ],
            "abstract": "Robust and flexible event representations are important to many core areas in\nlanguage understanding. Scripts were proposed early on as a way of representing\nsequences of events for such understanding, and has recently attracted renewed\nattention. However, obtaining effective representations for modeling\nscript-like event sequences is challenging. It requires representations that\ncan capture event-level and scenario-level semantics. We propose a new\ntensor-based composition method for creating event representations. The method\ncaptures more subtle semantic interactions between an event and its entities\nand yields representations that are effective at multiple event-related tasks.\nWith the continuous representations, we also devise a simple schema generation\nmethod which produces better schemas compared to a prior discrete\nrepresentation based method. Our analysis shows that the tensors capture\ndistinct usages of a predicate even when there are only subtle differences in\ntheir surface realizations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07646v3",
            "date": "2017-11-21 06:34:06+00:00",
            "title": "Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method",
            "authors": [
                "Yutong Shao",
                "Rico Sennrich",
                "Bonnie Webber",
                "Federico Fancellu"
            ],
            "abstract": "Idiom translation is a challenging problem in machine translation because the\nmeaning of idioms is non-compositional, and a literal (word-by-word)\ntranslation is likely to be wrong. In this paper, we focus on evaluating the\nquality of idiom translation of MT systems. We introduce a new evaluation\nmethod based on an idiom-specific blacklist of literal translations, based on\nthe insight that the occurrence of any blacklisted words in the translation\noutput indicates a likely translation error. We introduce a dataset, CIBB\n(Chinese Idioms Blacklists Bank), and perform an evaluation of a\nstate-of-the-art Chinese-English neural MT system. Our evaluation confirms that\na sizable number of idioms in our test set are mistranslated (46.1%), that\nliteral translation error is a common error type, and that our blacklist method\nis effective at identifying literal translation errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07893v2",
            "date": "2017-11-21 16:39:21+00:00",
            "title": "Effective Strategies in Zero-Shot Neural Machine Translation",
            "authors": [
                "Thanh-Le Ha",
                "Jan Niehues",
                "Alexander Waibel"
            ],
            "abstract": "In this paper, we proposed two strategies which can be applied to a\nmultilingual neural machine translation system in order to better tackle\nzero-shot scenarios despite not having any parallel corpus. The experiments\nshow that they are effective in terms of both performance and computing\nresources, especially in multilingual translation of unbalanced data in real\nzero-resourced condition when they alleviate the language bias problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07908v3",
            "date": "2017-11-21 16:55:18+00:00",
            "title": "Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition",
            "authors": [
                "Devendra Singh Sachan",
                "Pengtao Xie",
                "Mrinmaya Sachan",
                "Eric P Xing"
            ],
            "abstract": "Biomedical named entity recognition (NER) is a fundamental task in text\nmining of medical documents and has many applications. Deep learning based\napproaches to this task have been gaining increasing attention in recent years\nas their parameters can be learned end-to-end without the need for\nhand-engineered features. However, these approaches rely on high-quality\nlabeled data, which is expensive to obtain. To address this issue, we\ninvestigate how to use unlabeled text data to improve the performance of NER\nmodels. Specifically, we train a bidirectional language model (BiLM) on\nunlabeled data and transfer its weights to \"pretrain\" an NER model with the\nsame architecture as the BiLM, which results in a better parameter\ninitialization of the NER model. We evaluate our approach on four benchmark\ndatasets for biomedical NER and show that it leads to a substantial improvement\nin the F1 scores compared with the state-of-the-art approaches. We also show\nthat BiLM weight transfer leads to a faster model training and the pretrained\nmodel requires fewer training examples to achieve a particular F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07915v1",
            "date": "2017-11-21 17:04:33+00:00",
            "title": "10Sent: A Stable Sentiment Analysis Method Based on the Combination of Off-The-Shelf Approaches",
            "authors": [
                "Philipe F. Melo",
                "Daniel H. Dalip",
                "Manoel M. Junior",
                "Marcos A. Gon\u00e7alves",
                "Fabr\u00edcio Benevenuto"
            ],
            "abstract": "Sentiment analysis has become a very important tool for analysis of social\nmedia data. There are several methods developed for this research field, many\nof them working very differently from each other, covering distinct aspects of\nthe problem and disparate strategies. Despite the large number of existent\ntechniques, there is no single one which fits well in all cases or for all data\nsources. Supervised approaches may be able to adapt to specific situations but\nthey require manually labeled training, which is very cumbersome and expensive\nto acquire, mainly for a new application. In this context, in here, we propose\nto combine several very popular and effective state-of-the-practice sentiment\nanalysis methods, by means of an unsupervised bootstrapped strategy for\npolarity classification. One of our main goals is to reduce the large\nvariability (lack of stability) of the unsupervised methods across different\ndomains (datasets). Our solution was thoroughly tested considering thirteen\ndifferent datasets in several domains such as opinions, comments, and social\nmedia. The experimental results demonstrate that our combined method (aka,\n10SENT) improves the effectiveness of the classification task, but more\nimportantly, it solves a key problem in the field. It is consistently among the\nbest methods in many data types, meaning that it can produce the best (or close\nto best) results in almost all considered contexts, without any additional\ncosts (e.g., manual labeling). Our self-learning approach is also very\nindependent of the base methods, which means that it is highly extensible to\nincorporate any new additional method that can be envisioned in the future.\nFinally, we also investigate a transfer learning approach for sentiment\nanalysis as a means to gather additional (unsupervised) information for the\nproposed approach and we show the potential of this technique to improve our\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07950v3",
            "date": "2017-11-21 18:21:16+00:00",
            "title": "Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent",
            "authors": [
                "Zhilin Yang",
                "Saizheng Zhang",
                "Jack Urbanek",
                "Will Feng",
                "Alexander H. Miller",
                "Arthur Szlam",
                "Douwe Kiela",
                "Jason Weston"
            ],
            "abstract": "Contrary to most natural language processing research, which makes use of\nstatic datasets, humans learn language interactively, grounded in an\nenvironment. In this work we propose an interactive learning procedure called\nMechanical Turker Descent (MTD) and use it to train agents to execute natural\nlanguage commands grounded in a fantasy text adventure game. In MTD, Turkers\ncompete to train better agents in the short term, and collaborate by sharing\ntheir agents' skills in the long term. This results in a gamified, engaging\nexperience for the Turkers and a better quality teaching signal for the agents\ncompared to static datasets, as the Turkers naturally adapt the training data\nto the agent's abilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08083v1",
            "date": "2017-11-21 23:22:15+00:00",
            "title": "Application of Natural Language Processing to Determine User Satisfaction in Public Services",
            "authors": [
                "Radoslaw Kowalski",
                "Marc Esteve",
                "Slava J. Mikhaylov"
            ],
            "abstract": "Research on customer satisfaction has increased substantially in recent\nyears. However, the relative importance and relationships between different\ndeterminants of satisfaction remains uncertain. Moreover, quantitative studies\nto date tend to test for significance of pre-determined factors thought to have\nan influence with no scalable means to identify other causes of user\nsatisfaction. The gaps in knowledge make it difficult to use available\nknowledge on user preference for public service improvement. Meanwhile, digital\ntechnology development has enabled new methods to collect user feedback, for\nexample through online forums where users can comment freely on their\nexperience. New tools are needed to analyze large volumes of such feedback. Use\nof topic models is proposed as a feasible solution to aggregate open-ended user\nopinions that can be easily deployed in the public sector. Generated insights\ncan contribute to a more inclusive decision-making process in public service\nprovision. This novel methodological approach is applied to a case of service\nreviews of publicly-funded primary care practices in England. Findings from the\nanalysis of 145,000 reviews covering almost 7,700 primary care centers indicate\nthat the quality of interactions with staff and bureaucratic exigencies are the\nkey issues driving user satisfaction across England.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08231v3",
            "date": "2017-11-22 11:18:31+00:00",
            "title": "Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data?",
            "authors": [
                "Yi Zhang",
                "Xu Sun",
                "Shuming Ma",
                "Yang Yang",
                "Xuancheng Ren"
            ],
            "abstract": "Existing neural models usually predict the tag of the current token\nindependent of the neighboring tags. The popular LSTM-CRF model considers the\ntag dependencies between every two consecutive tags. However, it is hard for\nexisting neural models to take longer distance dependencies of tags into\nconsideration. The scalability is mainly limited by the complex model\nstructures and the cost of dynamic programming during training. In our work, we\nfirst design a new model called \"high order LSTM\" to predict multiple tags for\nthe current token which contains not only the current tag but also the previous\nseveral tags. We call the number of tags in one prediction as \"order\". Then we\npropose a new method called Multi-Order BiLSTM (MO-BiLSTM) which combines low\norder and high order LSTMs together. MO-BiLSTM keeps the scalability to high\norder models with a pruning technique. We evaluate MO-BiLSTM on all-phrase\nchunking and NER datasets. Experiment results show that MO-BiLSTM achieves the\nstate-of-the-art result in chunking and highly competitive results in two NER\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08493v1",
            "date": "2017-11-22 20:15:01+00:00",
            "title": "Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models",
            "authors": [
                "Bing Liu",
                "Tong Yu",
                "Ian Lane",
                "Ole J. Mengshoel"
            ],
            "abstract": "Dialog response selection is an important step towards natural response\ngeneration in conversational agents. Existing work on neural conversational\nmodels mainly focuses on offline supervised learning using a large set of\ncontext-response pairs. In this paper, we focus on online learning of response\nselection in retrieval-based dialog systems. We propose a contextual\nmulti-armed bandit model with a nonlinear reward function that uses distributed\nrepresentation of text for online response selection. A bidirectional LSTM is\nused to produce the distributed representations of dialog context and\nresponses, which serve as the input to a contextual bandit. In learning the\nbandit, we propose a customized Thompson sampling method that is applied to a\npolynomial feature space in approximating the reward. Experimental results on\nthe Ubuntu Dialogue Corpus demonstrate significant performance gains of the\nproposed method over conventional linear contextual bandits. Moreover, we\nreport encouraging response selection performance of the proposed neural bandit\nmodel using the Recall@k metric for a small set of online training samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08726v1",
            "date": "2017-11-23 15:00:43+00:00",
            "title": "Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce",
            "authors": [
                "Jianfei Yu",
                "Minghui Qiu",
                "Jing Jiang",
                "Jun Huang",
                "Shuangyong Song",
                "Wei Chu",
                "Haiqing Chen"
            ],
            "abstract": "In this paper, we study transfer learning for the PI and NLI problems, aiming\nto propose a general framework, which can effectively and efficiently adapt the\nshared knowledge learned from a resource-rich source domain to a resource- poor\ntarget domain. Specifically, since most existing transfer learning methods only\nfocus on learning a shared feature space across domains while ignoring the\nrelationship between the source and target domains, we propose to\nsimultaneously learn shared representations and domain relationships in a\nunified framework. Furthermore, we propose an efficient and effective hybrid\nmodel by combining a sentence encoding- based method and a sentence\ninteraction-based method as our base model. Extensive experiments on both\nparaphrase identification and natural language inference demonstrate that our\nbase model is efficient and has promising performance compared to the competing\nmodels, and our transfer learning method can help to significantly boost the\nperformance. Further analysis shows that the inter-domain and intra-domain\nrelationship captured by our model are insightful. Last but not least, we\ndeploy our transfer learning model for PI into our online chatbot system, which\ncan bring in significant improvements over our existing system. Finally, we\nlaunch our new system on the chatbot platform Eva in our E-commerce site\nAliExpress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08792v1",
            "date": "2017-11-23 18:00:29+00:00",
            "title": "SPINE: SParse Interpretable Neural Embeddings",
            "authors": [
                "Anant Subramanian",
                "Danish Pruthi",
                "Harsh Jhamtani",
                "Taylor Berg-Kirkpatrick",
                "Eduard Hovy"
            ],
            "abstract": "Prediction without justification has limited utility. Much of the success of\nneural models can be attributed to their ability to learn rich, dense and\nexpressive representations. While these representations capture the underlying\ncomplexity and latent trends in the data, they are far from being\ninterpretable. We propose a novel variant of denoising k-sparse autoencoders\nthat generates highly efficient and interpretable distributed word\nrepresentations (word embeddings), beginning with existing word representations\nfrom state-of-the-art methods like GloVe and word2vec. Through large scale\nhuman evaluation, we report that our resulting word embedddings are much more\ninterpretable than the original GloVe and word2vec embeddings. Moreover, our\nembeddings outperform existing popular word embeddings on a diverse suite of\nbenchmark downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09050v1",
            "date": "2017-11-24 17:14:34+00:00",
            "title": "Ethical Challenges in Data-Driven Dialogue Systems",
            "authors": [
                "Peter Henderson",
                "Koustuv Sinha",
                "Nicolas Angelard-Gontier",
                "Nan Rosemary Ke",
                "Genevieve Fried",
                "Ryan Lowe",
                "Joelle Pineau"
            ],
            "abstract": "The use of dialogue systems as a medium for human-machine interaction is an\nincreasingly prevalent paradigm. A growing number of dialogue systems use\nconversation strategies that are learned from large datasets. There are well\ndocumented instances where interactions with these system have resulted in\nbiased or even offensive conversations due to the data-driven training process.\nHere, we highlight potential ethical issues that arise in dialogue systems\nresearch, including: implicit biases in data-driven systems, the rise of\nadversarial examples, potential sources of privacy violations, safety concerns,\nspecial considerations for reinforcement learning systems, and reproducibility\nconcerns. We also suggest areas stemming from these issues that deserve further\ninvestigation. Through this initial survey, we hope to spur research leading to\nrobust, safe, and ethically sound dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09160v1",
            "date": "2017-11-24 22:31:01+00:00",
            "title": "An Exploration of Word Embedding Initialization in Deep-Learning Tasks",
            "authors": [
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Word embeddings are the interface between the world of discrete units of text\nprocessing and the continuous, differentiable world of neural networks. In this\nwork, we examine various random and pretrained initialization methods for\nembeddings used in deep networks and their effect on the performance on four\nNLP tasks with both recurrent and convolutional architectures. We confirm that\npretrained embeddings are a little better than random initialization,\nespecially considering the speed of learning. On the other hand, we do not see\nany significant difference between various methods of random initialization, as\nlong as the variance is kept reasonably low. High-variance initialization\nprevents the network to use the space of embeddings and forces it to use other\nfree parameters to accomplish the task. We support this hypothesis by observing\nthe performance in learning lexical relations and by the fact that the network\ncan learn to perform reasonably in its task even with fixed random embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09181v2",
            "date": "2017-11-25 02:58:59+00:00",
            "title": "Towards Accurate Deceptive Opinion Spam Detection based on Word Order-preserving CNN",
            "authors": [
                "Siyuan Zhao",
                "Zhiwei Xu",
                "Limin Liu",
                "Mengjie Guo"
            ],
            "abstract": "Nowadays, deep learning has been widely used. In natural language learning,\nthe analysis of complex semantics has been achieved because of its high degree\nof flexibility. The deceptive opinions detection is an important application\narea in deep learning model, and related mechanisms have been given attention\nand researched. On-line opinions are quite short, varied types and content. In\norder to effectively identify deceptive opinions, we need to comprehensively\nstudy the characteristics of deceptive opinions, and explore novel\ncharacteristics besides the textual semantics and emotional polarity that have\nbeen widely used in text analysis. The detection mechanism based on deep\nlearning has better self-adaptability and can effectively identify all kinds of\ndeceptive opinions. In this paper, we optimize the convolution neural network\nmodel by embedding the word order characteristics in its convolution layer and\npooling layer, which makes convolution neural network more suitable for various\ntext classification and deceptive opinions detection. The TensorFlow-based\nexperiments demonstrate that the detection mechanism proposed in this paper\nachieve more accurate deceptive opinion detection results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09271v3",
            "date": "2017-11-25 18:44:00+00:00",
            "title": "Acronym Disambiguation: A Domain Independent Approach",
            "authors": [
                "Aditya Thakker",
                "Suhail Barot",
                "Sudhir Bagul"
            ],
            "abstract": "Acronyms are omnipresent. They usually express information that is repetitive\nand well known. But acronyms can also be ambiguous because there can be\nmultiple expansions for the same acronym. In this paper, we propose a general\nsystem for acronym disambiguation that can work on any acronym given some\ncontext information. We present methods for retrieving all the possible\nexpansions of an acronym from Wikipedia and AcronymsFinder.com. We propose to\nuse these expansions to collect all possible contexts in which these acronyms\nare used and then score them using a paragraph embedding technique called\nDoc2Vec. This method collectively led to achieving an accuracy of 90.9% in\nselecting the correct expansion for given acronym, on a dataset we scraped from\nWikipedia with 707 distinct acronyms and 14,876 disambiguations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09285v1",
            "date": "2017-11-25 20:36:39+00:00",
            "title": "Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity",
            "authors": [
                "Samira Abnar",
                "Rasyan Ahmed",
                "Max Mijnheer",
                "Willem Zuidema"
            ],
            "abstract": "We evaluate 8 different word embedding models on their usefulness for\npredicting the neural activation patterns associated with concrete nouns. The\nmodels we consider include an experiential model, based on crowd-sourced\nassociation data, several popular neural and distributional models, and a model\nthat reflects the syntactic context of words (based on dependency parses). Our\ngoal is to assess the cognitive plausibility of these various embedding models,\nand understand how we can further improve our methods for interpreting brain\nimaging data.\n  We show that neural word embedding models exhibit superior performance on the\ntasks we consider, beating experiential word representation model. The\nsyntactically informed model gives the overall best performance when predicting\nbrain activation patterns from word embeddings; whereas the GloVe\ndistributional method gives the overall best performance when predicting in the\nreverse direction (words vectors from brain images). Interestingly, however,\nthe error patterns of these different models are markedly different. This may\nsupport the idea that the brain uses different systems for processing different\nkinds of words. Moreover, we suggest that taking the relative strengths of\ndifferent embedding models into account will lead to better models of the brain\nactivity associated with words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09367v1",
            "date": "2017-11-26 10:44:55+00:00",
            "title": "Learning to Remember Translation History with a Continuous Cache",
            "authors": [
                "Zhaopeng Tu",
                "Yang Liu",
                "Shuming Shi",
                "Tong Zhang"
            ],
            "abstract": "Existing neural machine translation (NMT) models generally translate\nsentences in isolation, missing the opportunity to take advantage of\ndocument-level information. In this work, we propose to augment NMT models with\na very light-weight cache-like memory network, which stores recent hidden\nrepresentations as translation history. The probability distribution over\ngenerated words is updated online depending on the translation history\nretrieved from the memory, endowing NMT models with the capability to\ndynamically adapt over time. Experiments on multiple domains with different\ntopics and styles show the effectiveness of the proposed approach with\nnegligible impact on the computational cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09476v3",
            "date": "2017-11-26 22:30:31+00:00",
            "title": "Machine Translation using Semantic Web Technologies: A Survey",
            "authors": [
                "Diego Moussallem",
                "Matthias Wauer",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "A large number of machine translation approaches have recently been developed\nto facilitate the fluid migration of content across languages. However, the\nliterature suggests that many obstacles must still be dealt with to achieve\nbetter automatic translations. One of these obstacles is lexical and syntactic\nambiguity. A promising way of overcoming this problem is using Semantic Web\ntechnologies. This article presents the results of a systematic review of\nmachine translation approaches that rely on Semantic Web technologies for\ntranslating texts. Overall, our survey suggests that while Semantic Web\ntechnologies can enhance the quality of machine translation outputs for various\nproblems, the combination of both is still in its infancy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09502v2",
            "date": "2017-11-27 01:42:00+00:00",
            "title": "Modeling Past and Future for Neural Machine Translation",
            "authors": [
                "Zaixiang Zheng",
                "Hao Zhou",
                "Shujian Huang",
                "Lili Mou",
                "Xinyu Dai",
                "Jiajun Chen",
                "Zhaopeng Tu"
            ],
            "abstract": "Existing neural machine translation systems do not explicitly model what has\nbeen translated and what has not during the decoding phase. To address this\nproblem, we propose a novel mechanism that separates the source information\ninto two parts: translated Past contents and untranslated Future contents,\nwhich are modeled by two additional recurrent layers. The Past and Future\ncontents are fed to both the attention model and the decoder states, which\noffers NMT systems the knowledge of translated and untranslated contents.\nExperimental results show that the proposed approach significantly improves\ntranslation performance in Chinese-English, German-English and English-German\ntranslation tasks. Specifically, the proposed model outperforms the\nconventional coverage model in both of the translation quality and the\nalignment error rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09824v1",
            "date": "2017-11-27 16:50:15+00:00",
            "title": "Lexical-semantic resources: yet powerful resources for automatic personality classification",
            "authors": [
                "Xuan-Son Vu",
                "Lucie Flekova",
                "Lili Jiang",
                "Iryna Gurevych"
            ],
            "abstract": "In this paper, we aim to reveal the impact of lexical-semantic resources,\nused in particular for word sense disambiguation and sense-level semantic\ncategorization, on automatic personality classification task. While stylistic\nfeatures (e.g., part-of-speech counts) have been shown their power in this\ntask, the impact of semantics beyond targeted word lists is relatively\nunexplored. We propose and extract three types of lexical-semantic features,\nwhich capture high-level concepts and emotions, overcoming the lexical gap of\nword n-grams. Our experimental results are comparable to state-of-the-art\nmethods, while no personality-specific resources are required.",
            "journal ref": "GWC 2018 The 9th Global WordNet Conference GWC 2018 The 9th Global\n  WordNet Conference GWC 2018 The 9th Global WordNet Conference GWC 2018, the\n  9th Global WordNet Conference",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09873v2",
            "date": "2017-11-27 18:43:52+00:00",
            "title": "Slim Embedding Layers for Recurrent Neural Language Models",
            "authors": [
                "Zhongliang Li",
                "Raymond Kulhanek",
                "Shaojun Wang",
                "Yunxin Zhao",
                "Shuang Wu"
            ],
            "abstract": "Recurrent neural language models are the state-of-the-art models for language\nmodeling. When the vocabulary size is large, the space taken to store the model\nparameters becomes the bottleneck for the use of recurrent neural language\nmodels. In this paper, we introduce a simple space compression method that\nrandomly shares the structured parameters at both the input and output\nembedding layers of the recurrent neural language models to significantly\nreduce the size of model parameters, but still compactly represent the original\ninput and output embedding layers. The method is easy to implement and tune.\nExperiments on several data sets show that the new method can get similar\nperplexity and BLEU score results while only using a very tiny fraction of\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10093v1",
            "date": "2017-11-28 02:56:12+00:00",
            "title": "Surfacing contextual hate speech words within social media",
            "authors": [
                "Jherez Taylor",
                "Melvyn Peignon",
                "Yi-Shin Chen"
            ],
            "abstract": "Social media platforms have recently seen an increase in the occurrence of\nhate speech discourse which has led to calls for improved detection methods.\nMost of these rely on annotated data, keywords, and a classification technique.\nWhile this approach provides good coverage, it can fall short when dealing with\nnew terms produced by online extremist communities which act as original\nsources of words which have alternate hate speech meanings. These code words\n(which can be both created and adopted words) are designed to evade automatic\ndetection and often have benign meanings in regular discourse. As an example,\n\"skypes\", \"googles\", and \"yahoos\" are all instances of words which have an\nalternate meaning that can be used for hate speech. This overlap introduces\nadditional challenges when relying on keywords for both the collection of data\nthat is specific to hate speech, and downstream classification. In this work,\nwe develop a community detection approach for finding extremist hate speech\ncommunities and collecting data from their members. We also develop a word\nembedding model that learns the alternate hate speech meaning of words and\ndemonstrate the candidacy of our code words with several annotation\nexperiments, designed to determine if it is possible to recognize a word as\nbeing used for hate speech without knowing its alternate meaning. We report an\ninter-annotator agreement rate of K=0.871, and K=0.676 for data drawn from our\nextremist community and the keyword approach respectively, supporting our claim\nthat hate speech detection is a contextual task and does not depend on a fixed\nlist of keywords. Our goal is to advance the domain by providing a high quality\nhate speech dataset in addition to learned code words that can be fed into\nexisting classification approaches, thus improving the accuracy of automated\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10122v3",
            "date": "2017-11-28 04:42:24+00:00",
            "title": "End-to-end Adversarial Learning for Generative Conversational Agents",
            "authors": [
                "Oswaldo Ludwig"
            ],
            "abstract": "This paper presents a new adversarial learning method for generative\nconversational agents (GCA) besides a new model of GCA. Similar to previous\nworks on adversarial learning for dialogue generation, our method assumes the\nGCA as a generator that aims at fooling a discriminator that labels dialogues\nas human-generated or machine-generated; however, in our approach, the\ndiscriminator performs token-level classification, i.e. it indicates whether\nthe current token was generated by humans or machines. To do so, the\ndiscriminator also receives the context utterances (the dialogue history) and\nthe incomplete answer up to the current token as input. This new approach makes\npossible the end-to-end training by backpropagation. A self-conversation\nprocess enables to produce a set of generated data with more diversity for the\nadversarial training. This approach improves the performance on questions not\nrelated to the training data. Experimental results with human and adversarial\nevaluations show that the adversarial method yields significant performance\ngains over the usual teacher forcing training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10124v1",
            "date": "2017-11-28 04:51:25+00:00",
            "title": "Vietnamese Semantic Role Labelling",
            "authors": [
                "Phuong Le-Hong",
                "Thai Hoang Pham",
                "Xuan Khoai Pham",
                "Thi Minh Huyen Nguyen",
                "Thi Luong Nguyen",
                "Minh Hiep Nguyen"
            ],
            "abstract": "In this paper, we study semantic role labelling (SRL), a subtask of semantic\nparsing of natural language sentences and its application for the Vietnamese\nlanguage. We present our effort in building Vietnamese PropBank, the first\nVietnamese SRL corpus and a software system for labelling semantic roles of\nVietnamese texts. In particular, we present a novel constituent extraction\nalgorithm in the argument candidate identification step which is more suitable\nand more accurate than the common node-mapping method. In the machine learning\npart, our system integrates distributed word features produced by two recent\nunsupervised learning models in two learned statistical classifiers and makes\nuse of integer linear programming inference procedure to improve the accuracy.\nThe system is evaluated in a series of experiments and achieves a good result,\nan $F_1$ score of 74.77%. Our system, including corpus and software, is\navailable as an open source project for free research and we believe that it is\na good baseline for the development of future Vietnamese SRL systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10133v1",
            "date": "2017-11-28 05:43:44+00:00",
            "title": "Unsupervised Discovery of Structured Acoustic Tokens with Applications to Spoken Term Detection",
            "authors": [
                "Cheng-Tao Chung",
                "Lin-Shan Lee"
            ],
            "abstract": "In this paper, we compare two paradigms for unsupervised discovery of\nstructured acoustic tokens directly from speech corpora without any human\nannotation. The Multigranular Paradigm seeks to capture all available\ninformation in the corpora with multiple sets of tokens for different model\ngranularities. The Hierarchical Paradigm attempts to jointly learn several\nlevels of signal representations in a hierarchical structure. The two paradigms\nare unified within a theoretical framework in this paper. Query-by-Example\nSpoken Term Detection (QbE-STD) experiments on the QUESST dataset of MediaEval\n2015 verifies the competitiveness of the acoustic tokens. The Enhanced\nRelevance Score (ERS) proposed in this work improves both paradigms for the\ntask of QbE-STD. We also list results on the ABX evaluation task of the Zero\nResource Challenge 2015 for comparison of the Paradigms.",
            "journal ref": "IEEE Transactions on Audio, Speech, and Language Processing 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10136v1",
            "date": "2017-11-28 05:55:11+00:00",
            "title": "Acoustic-To-Word Model Without OOV",
            "authors": [
                "Jinyu Li",
                "Guoli Ye",
                "Rui Zhao",
                "Jasha Droppo",
                "Yifan Gong"
            ],
            "abstract": "Recently, the acoustic-to-word model based on the Connectionist Temporal\nClassification (CTC) criterion was shown as a natural end-to-end model directly\ntargeting words as output units. However, this type of word-based CTC model\nsuffers from the out-of-vocabulary (OOV) issue as it can only model limited\nnumber of words in the output layer and maps all the remaining words into an\nOOV output node. Therefore, such word-based CTC model can only recognize the\nfrequent words modeled by the network output nodes. It also cannot easily\nhandle the hot-words which emerge after the model is trained. In this study, we\nimprove the acoustic-to-word model with a hybrid CTC model which can predict\nboth words and characters at the same time. With a shared-hidden-layer\nstructure and modular design, the alignments of words generated from the\nword-based CTC and the character-based CTC are synchronized. Whenever the\nacoustic-to-word model emits an OOV token, we back off that OOV segment to the\nword output generated from the character-based CTC, hence solving the OOV or\nhot-words issue. Evaluated on a Microsoft Cortana voice assistant task, the\nproposed model can reduce the errors introduced by the OOV output token in the\nacoustic-to-word model by 30%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10163v2",
            "date": "2017-11-28 07:58:34+00:00",
            "title": "Hybrid Oracle: Making Use of Ambiguity in Transition-based Chinese Dependency Parsing",
            "authors": [
                "Xuancheng Ren",
                "Xu Sun"
            ],
            "abstract": "In the training of transition-based dependency parsers, an oracle is used to\npredict a transition sequence for a sentence and its gold tree. However, the\ntransition system may exhibit ambiguity, that is, there can be multiple correct\ntransition sequences that form the gold tree. We propose to make use of the\nproperty in the training of neural dependency parsers, and present the Hybrid\nOracle. The new oracle gives all the correct transitions for a parsing state,\nwhich are used in the cross entropy loss function to provide better supervisory\nsignal. It is also used to generate different transition sequences for a\nsentence to better explore the training data and improve the generalization\nability of the parser. Evaluations show that the parsers trained using the\nhybrid oracle outperform the parsers using the traditional oracle in Chinese\ndependency parsing. We provide analysis from a linguistic view. The code is\navailable at https://github.com/lancopku/nndep .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10203v2",
            "date": "2017-11-28 09:41:34+00:00",
            "title": "Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure",
            "authors": [
                "Dieuwke Hupkes",
                "Sara Veldhoen",
                "Willem Zuidema"
            ],
            "abstract": "We investigate how neural networks can learn and process languages with\nhierarchical, compositional semantics. To this end, we define the artificial\ntask of processing nested arithmetic expressions, and study whether different\ntypes of neural networks can learn to compute their meaning. We find that\nrecursive neural networks can find a generalising solution to this problem, and\nwe visualise this solution by breaking it up in three steps: project, sum and\nsquash. As a next step, we investigate recurrent neural networks, and show that\na gated recurrent unit, that processes its input incrementally, also performs\nvery well on this task. To develop an understanding of what the recurrent\nnetwork encodes, visualisation techniques alone do not suffice. Therefore, we\ndevelop an approach where we formulate and test multiple hypotheses on the\ninformation encoded and processed by the network. For each hypothesis, we\nderive predictions about features of the hidden state representations at each\ntime step, and train 'diagnostic classifiers' to test those predictions. Our\nresults indicate that the networks follow a strategy similar to our\nhypothesised 'cumulative strategy', which explains the high accuracy of the\nnetwork on novel expressions, the generalisation to longer expressions than\nseen in training, and the mild deterioration with increasing length. This is\nturn shows that diagnostic classifiers can be a useful technique for opening up\nthe black box of neural networks. We argue that diagnostic classification,\nunlike most visualisation techniques, does scale up from small networks in a\ntoy domain, to larger and deeper recurrent networks dealing with real-life\ndata, and may therefore contribute to a better understanding of the internal\ndynamics of current state-of-the-art models in natural language processing.",
            "journal ref": "Journal of Artificial Intelligence Research 61 (2018) 907-926",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10705v1",
            "date": "2017-11-29 06:55:59+00:00",
            "title": "Speaker-Sensitive Dual Memory Networks for Multi-Turn Slot Tagging",
            "authors": [
                "Young-Bum Kim",
                "Sungjin Lee",
                "Ruhi Sarikaya"
            ],
            "abstract": "In multi-turn dialogs, natural language understanding models can introduce\nobvious errors by being blind to contextual information. To incorporate dialog\nhistory, we present a neural architecture with Speaker-Sensitive Dual Memory\nNetworks which encode utterances differently depending on the speaker. This\naddresses the different extents of information available to the system - the\nsystem knows only the surface form of user utterances while it has the exact\nsemantics of system output. We performed experiments on real user data from\nMicrosoft Cortana, a commercial personal assistant. The result showed a\nsignificant performance improvement over the state-of-the-art slot tagging\nmodels using contextual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10712v2",
            "date": "2017-11-29 07:38:07+00:00",
            "title": "End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning",
            "authors": [
                "Bing Liu",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Pararth Shah",
                "Larry Heck"
            ],
            "abstract": "In this paper, we present a neural network based task-oriented dialogue\nsystem that can be optimized end-to-end with deep reinforcement learning (RL).\nThe system is able to track dialogue state, interface with knowledge bases, and\nincorporate query results into agent's responses to successfully complete\ntask-oriented dialogues. Dialogue policy learning is conducted with a hybrid\nsupervised and deep RL methods. We first train the dialogue agent in a\nsupervised manner by learning directly from task-oriented dialogue corpora, and\nfurther optimize it with deep RL during its interaction with users. In the\nexperiments on two different dialogue task domains, our model demonstrates\nrobust performance in tracking dialogue state and producing reasonable system\nresponses. We show that deep RL based optimization leads to significant\nimprovement on task success rate and reduction in dialogue length comparing to\nsupervised training model. We further show benefits of training task-oriented\ndialogue model end-to-end comparing to component-wise optimization with\nexperiment results on dialogue simulations and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10837v1",
            "date": "2017-11-29 13:21:22+00:00",
            "title": "Curriculum Q-Learning for Visual Vocabulary Acquisition",
            "authors": [
                "Ahmed H. Zaidi",
                "Russell Moore",
                "Ted Briscoe"
            ],
            "abstract": "The structure of curriculum plays a vital role in our learning process, both\nas children and adults. Presenting material in ascending order of difficulty\nthat also exploits prior knowledge can have a significant impact on the rate of\nlearning. However, the notion of difficulty and prior knowledge differs from\nperson to person. Motivated by the need for a personalised curriculum, we\npresent a novel method of curriculum learning for vocabulary words in the form\nof visual prompts. We employ a reinforcement learning model grounded in\npedagogical theories that emulates the actions of a tutor. We simulate three\nstudents with different levels of vocabulary knowledge in order to evaluate the\nhow well our model adapts to the environment. The results of the simulation\nreveal that through interaction, the model is able to identify the areas of\nweakness, as well as push students to the edge of their ZPD. We hypothesise\nthat these methods can also be effective in training agents to learn language\nrepresentations in a simulated environment where it has previously been shown\nthat order of words and prior knowledge play an important role in the efficacy\nof language learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10960v1",
            "date": "2017-11-17 21:39:19+00:00",
            "title": "Identifying Patterns of Associated-Conditions through Topic Models of Electronic Medical Records",
            "authors": [
                "Moumita Bhattacharya",
                "Claudine Jurkovitz",
                "Hagit Shatkay"
            ],
            "abstract": "Multiple adverse health conditions co-occurring in a patient are typically\nassociated with poor prognosis and increased office or hospital visits.\nDeveloping methods to identify patterns of co-occurring conditions can assist\nin diagnosis. Thus identifying patterns of associations among co-occurring\nconditions is of growing interest. In this paper, we report preliminary results\nfrom a data-driven study, in which we apply a machine learning method, namely,\ntopic modeling, to electronic medical records, aiming to identify patterns of\nassociated conditions. Specifically, we use the well established latent\ndirichlet allocation, a method based on the idea that documents can be modeled\nas a mixture of latent topics, where each topic is a distribution over words.\nIn our study, we adapt the LDA model to identify latent topics in patients'\nEMRs. We evaluate the performance of our method both qualitatively, and show\nthat the obtained topics indeed align well with distinct medical phenomena\ncharacterized by co-occurring conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11081v1",
            "date": "2017-11-10 20:04:23+00:00",
            "title": "Improved Twitter Sentiment Analysis Using Naive Bayes and Custom Language Model",
            "authors": [
                "Angela Lin"
            ],
            "abstract": "In the last couple decades, social network services like Twitter have\ngenerated large volumes of data about users and their interests, providing\nmeaningful business intelligence so organizations can better understand and\nengage their customers. All businesses want to know who is promoting their\nproducts, who is complaining about them, and how are these opinions bringing or\ndiminishing value to a company. Companies want to be able to identify their\nhigh-value customers and quantify the value each user brings. Many businesses\nuse social media metrics to calculate the user contribution score, which\nenables them to quantify the value that influential users bring on social\nmedia, so the businesses can offer them more differentiated services. However,\nthe score calculation can be refined to provide a better illustration of a\nuser's contribution. Using Microsoft Azure as a case study, we conducted\nTwitter sentiment analysis to develop a machine learning classification model\nthat identifies tweet contents and sentiments most illustrative of\npositive-value user contribution. Using data mining and AI-powered cognitive\ntools, we analyzed factors of social influence and specifically, promotional\nlanguage in the developer community. Our predictive model was a combination of\na traditional supervised machine learning algorithm and a custom-developed\nnatural language model for identifying promotional tweets, that identifies a\nproduct-specific promotion on Twitter with a 90% accuracy rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11118v1",
            "date": "2017-11-29 21:40:59+00:00",
            "title": "Multimodal Attribute Extraction",
            "authors": [
                "Robert L. Logan IV",
                "Samuel Humeau",
                "Sameer Singh"
            ],
            "abstract": "The broad goal of information extraction is to derive structured information\nfrom unstructured data. However, most existing methods focus solely on text,\nignoring other types of unstructured data such as images, video and audio which\ncomprise an increasing portion of the information on the web. To address this\nshortcoming, we propose the task of multimodal attribute extraction. Given a\ncollection of unstructured and semi-structured contextual information about an\nentity (such as a textual description, or visual depictions) the task is to\nextract the entity's underlying attributes. In this paper, we provide a dataset\ncontaining mixed-media data for over 2 million product items along with 7\nmillion attribute-value pairs describing the items which can be used to train\nattribute extractors in a weakly supervised manner. We provide a variety of\nbaselines which demonstrate the relative effectiveness of the individual modes\nof information towards solving the task, as well as study human performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11125v1",
            "date": "2017-11-29 21:50:40+00:00",
            "title": "Predicting and Explaining Human Semantic Search in a Cognitive Model",
            "authors": [
                "Filip Miscevic",
                "Aida Nematzadeh",
                "Suzanne Stevenson"
            ],
            "abstract": "Recent work has attempted to characterize the structure of semantic memory\nand the search algorithms which, together, best approximate human patterns of\nsearch revealed in a semantic fluency task. There are a number of models that\nseek to capture semantic search processes over networks, but they vary in the\ncognitive plausibility of their implementation. Existing work has also\nneglected to consider the constraints that the incremental process of language\nacquisition must place on the structure of semantic memory. Here we present a\nmodel that incrementally updates a semantic network, with limited computational\nsteps, and replicates many patterns found in human semantic fluency using a\nsimple random walk. We also perform thorough analyses showing that a\ncombination of both structural and semantic features are correlated with human\nperformance patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11191v1",
            "date": "2017-11-30 02:06:47+00:00",
            "title": "Neural Response Generation with Dynamic Vocabularies",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Dejian Yang",
                "Can Xu",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "We study response generation for open domain conversation in chatbots.\nExisting methods assume that words in responses are generated from an identical\nvocabulary regardless of their inputs, which not only makes them vulnerable to\ngeneric patterns and irrelevant noise, but also causes a high cost in decoding.\nWe propose a dynamic vocabulary sequence-to-sequence (DVS2S) model which allows\neach input to possess their own vocabulary in decoding. In training, vocabulary\nconstruction and response generation are jointly learned by maximizing a lower\nbound of the true objective with a Monte Carlo sampling method. In inference,\nthe model dynamically allocates a small vocabulary for an input with the word\nprediction model, and conducts decoding only with the small vocabulary. Because\nof the dynamic vocabulary mechanism, DVS2S eludes many generic patterns and\nirrelevant words in generation, and enjoys efficient decoding at the same time.\nExperimental results on both automatic metrics and human annotations show that\nDVS2S can significantly outperform state-of-the-art methods in terms of\nresponse quality, but only requires 60% decoding time compared to the most\nefficient baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11221v3",
            "date": "2017-11-30 04:30:53+00:00",
            "title": "Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches",
            "authors": [
                "Shaohui Kuang",
                "Deyi Xiong",
                "Weihua Luo",
                "Guodong Zhou"
            ],
            "abstract": "Sentences in a well-formed text are connected to each other via various links\nto form the cohesive structure of the text. Current neural machine translation\n(NMT) systems translate a text in a conventional sentence-by-sentence fashion,\nignoring such cross-sentence links and dependencies. This may lead to generate\nan incoherent target text for a coherent source text. In order to handle this\nissue, we propose a cache-based approach to modeling coherence for neural\nmachine translation by capturing contextual information either from recently\ntranslated sentences or the entire document. Particularly, we explore two types\nof caches: a dynamic cache, which stores words from the best translation\nhypotheses of preceding sentences, and a topic cache, which maintains a set of\ntarget-side topical words that are semantically related to the document to be\ntranslated. On this basis, we build a new layer to score target words in these\ntwo caches with a cache-based neural model. Here the estimated probabilities\nfrom the cache-based neural model are combined with NMT probabilities into the\nfinal word prediction probabilities via a gating mechanism. Finally, the\nproposed cache-based neural model is trained jointly with NMT system in an\nend-to-end manner. Experiments and analysis presented in this paper demonstrate\nthat the proposed cache-based model achieves substantial improvements over\nseveral state-of-the-art SMT and NMT baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11310v1",
            "date": "2017-11-30 10:37:56+00:00",
            "title": "Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "The goal of this paper is to learn cross-domain representations for slot\nfilling task in spoken language understanding (SLU). Most of the recently\npublished SLU models are domain-specific ones that work on individual task\ndomains. Annotating data for each individual task domain is both financially\ncostly and non-scalable. In this work, we propose an adversarial training\nmethod in learning common features and representations that can be shared\nacross multiple domains. Model that produces such shared representations can be\ncombined with models trained on individual domain SLU data to reduce the amount\nof training samples required for developing a new domain. In our experiments\nusing data sets from multiple domains, we show that adversarial training helps\nin learning better domain-general SLU models, leading to improved slot filling\nF1 scores. We further show that applying adversarial learning on domain-general\nmodel also helps in achieving higher slot filling performance when the model is\njointly optimized with domain-specific models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11513v2",
            "date": "2017-11-30 17:02:52+00:00",
            "title": "Lexical and Derivational Meaning in Vector-Based Models of Relativisation",
            "authors": [
                "Michael Moortgat",
                "Gijs Wijnholds"
            ],
            "abstract": "Sadrzadeh et al (2013) present a compositional distributional analysis of\nrelative clauses in English in terms of the Frobenius algebraic structure of\nfinite dimensional vector spaces. The analysis relies on distinct type\nassignments and lexical recipes for subject vs object relativisation. The\nsituation for Dutch is different: because of the verb final nature of Dutch,\nrelative clauses are ambiguous between a subject vs object relativisation\nreading. Using an extended version of Lambek calculus, we present a\ncompositional distributional framework that accounts for this derivational\nambiguity, and that allows us to give a single meaning recipe for the relative\npronoun reconciling the Frobenius semantics with the demands of Dutch\nderivational syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.00069v1",
            "date": "2017-11-30 20:36:50+00:00",
            "title": "On the importance of normative data in speech-based assessment",
            "authors": [
                "Zeinab Noorian",
                "Chlo\u00e9 Pou-Prom",
                "Frank Rudzicz"
            ],
            "abstract": "Data sets for identifying Alzheimer's disease (AD) are often relatively\nsparse, which limits their ability to train generalizable models. Here, we\naugment such a data set, DementiaBank, with each of two normative data sets,\nthe Wisconsin Longitudinal Study and Talk2Me, each of which employs a\nspeech-based picture-description assessment. Through minority class\noversampling with ADASYN, we outperform state-of-the-art results in binary\nclassification of people with and without AD in DementiaBank. This work\nhighlights the effectiveness of combining sparse and difficult-to-acquire\npatient data with relatively large and easily accessible normative datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.00170v2",
            "date": "2017-12-01 03:14:51+00:00",
            "title": "Text Generation Based on Generative Adversarial Nets with Latent Variable",
            "authors": [
                "Heng Wang",
                "Zengchang Qin",
                "Tao Wan"
            ],
            "abstract": "In this paper, we propose a model using generative adversarial net (GAN) to\ngenerate realistic text. Instead of using standard GAN, we combine variational\nautoencoder (VAE) with generative adversarial net. The use of high-level latent\nrandom variables is helpful to learn the data distribution and solve the\nproblem that generative adversarial net always emits the similar data. We\npropose the VGAN model where the generative model is composed of recurrent\nneural network and VAE. The discriminative model is a convolutional neural\nnetwork. We train the model via policy gradient. We apply the proposed model to\nthe task of text generation and compare it to other recent neural network based\nmodels, such as recurrent neural network language model and SeqGAN. We evaluate\nthe performance of the model by calculating negative log-likelihood and the\nBLEU score. We conduct experiments on three benchmark datasets, and results\nshow that our model outperforms other previous models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.00609v1",
            "date": "2017-12-02 14:14:50+00:00",
            "title": "Improving Visually Grounded Sentence Representations with Self-Attention",
            "authors": [
                "Kang Min Yoo",
                "Youhyun Shin",
                "Sang-goo Lee"
            ],
            "abstract": "Sentence representation models trained only on language could potentially\nsuffer from the grounding problem. Recent work has shown promising results in\nimproving the qualities of sentence representations by jointly training them\nwith associated image features. However, the grounding capability is limited\ndue to distant connection between input sentences and image features by the\ndesign of the architecture. In order to further close the gap, we propose\napplying self-attention mechanism to the sentence encoder to deepen the\ngrounding effect. Our results on transfer tasks show that self-attentive\nencoders are better for visual grounding, as they exploit specific words with\nstrong visual associations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01460v1",
            "date": "2017-12-05 03:11:07+00:00",
            "title": "AWE-CM Vectors: Augmenting Word Embeddings with a Clinical Metathesaurus",
            "authors": [
                "Willie Boag",
                "Hassan Kan\u00e9"
            ],
            "abstract": "In recent years, word embeddings have been surprisingly effective at\ncapturing intuitive characteristics of the words they represent. These vectors\nachieve the best results when training corpora are extremely large, sometimes\nbillions of words. Clinical natural language processing datasets, however, tend\nto be much smaller. Even the largest publicly-available dataset of medical\nnotes is three orders of magnitude smaller than the dataset of the oft-used\n\"Google News\" word vectors. In order to make up for limited training data\nsizes, we encode expert domain knowledge into our embeddings. Building on a\nprevious extension of word2vec, we show that generalizing the notion of a\nword's \"context\" to include arbitrary features creates an avenue for encoding\ndomain knowledge into word embeddings. We show that the word vectors produced\nby this method outperform their text-only counterparts across the board in\ncorrelation with clinical experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01476v1",
            "date": "2017-12-05 04:49:58+00:00",
            "title": "Sequence Mining and Pattern Analysis in Drilling Reports with Deep Natural Language Processing",
            "authors": [
                "J\u00falio Hoffimann",
                "Youli Mao",
                "Avinash Wesley",
                "Aimee Taylor"
            ],
            "abstract": "Drilling activities in the oil and gas industry have been reported over\ndecades for thousands of wells on a daily basis, yet the analysis of this text\nat large-scale for information retrieval, sequence mining, and pattern analysis\nis very challenging. Drilling reports contain interpretations written by\ndrillers from noting measurements in downhole sensors and surface equipment,\nand can be used for operation optimization and accident mitigation. In this\ninitial work, a methodology is proposed for automatic classification of\nsentences written in drilling reports into three relevant labels (EVENT,\nSYMPTOM and ACTION) for hundreds of wells in an actual field. Some of the main\nchallenges in the text corpus were overcome, which include the high frequency\nof technical symbols, mistyping/abbreviation of technical terms, and the\npresence of incomplete sentences in the drilling reports. We obtain\nstate-of-the-art classification accuracy within this technical language and\nillustrate advanced queries enabled by the tool.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01586v1",
            "date": "2017-12-05 11:48:51+00:00",
            "title": "Deep Semantic Role Labeling with Self-Attention",
            "authors": [
                "Zhixing Tan",
                "Mingxuan Wang",
                "Jun Xie",
                "Yidong Chen",
                "Xiaodong Shi"
            ],
            "abstract": "Semantic Role Labeling (SRL) is believed to be a crucial step towards natural\nlanguage understanding and has been widely studied. Recent years, end-to-end\nSRL with recurrent neural networks (RNN) has gained increasing attention.\nHowever, it remains a major challenge for RNNs to handle structural information\nand long range dependencies. In this paper, we present a simple and effective\narchitecture for SRL which aims to address these problems. Our model is based\non self-attention which can directly capture the relationships between two\ntokens regardless of their distance. Our single model achieves F$_1=83.4$ on\nthe CoNLL-2005 shared task dataset and F$_1=82.7$ on the CoNLL-2012 shared task\ndataset, which outperforms the previous state-of-the-art results by $1.8$ and\n$1.0$ F$_1$ score respectively. Besides, our model is computationally\nefficient, and the parsing speed is 50K tokens per second on a single Titan X\nGPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01719v2",
            "date": "2017-12-05 15:55:27+00:00",
            "title": "Phylogenetics of Indo-European Language families via an Algebro-Geometric Analysis of their Syntactic Structures",
            "authors": [
                "Kevin Shu",
                "Andrew Ortegaray",
                "Robert Berwick",
                "Matilde Marcolli"
            ],
            "abstract": "Using Phylogenetic Algebraic Geometry, we analyze computationally the\nphylogenetic tree of subfamilies of the Indo-European language family, using\ndata of syntactic structures. The two main sources of syntactic data are the\nSSWL database and Longobardi's recent data of syntactic parameters. We compute\nphylogenetic invariants and likelihood functions for two sets of Germanic\nlanguages, a set of Romance languages, a set of Slavic languages and a set of\nearly Indo-European languages, and we compare the results with what is known\nthrough historical linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01741v1",
            "date": "2017-12-05 16:28:37+00:00",
            "title": "Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scaling",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad"
            ],
            "abstract": "Access to word-sentiment associations is useful for many applications,\nincluding sentiment analysis, stance detection, and linguistic analysis.\nHowever, manually assigning fine-grained sentiment association scores to words\nhas many challenges with respect to keeping annotations consistent. We apply\nthe annotation technique of Best-Worst Scaling to obtain real-valued sentiment\nassociation scores for words and phrases in three different domains: general\nEnglish, English Twitter, and Arabic Twitter. We show that on all three domains\nthe ranking of words by sentiment remains remarkably consistent even when the\nannotation process is repeated with a different set of annotators. We also, for\nthe first time, determine the minimum difference in sentiment association that\nis perceptible to native speakers of a language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01765v1",
            "date": "2017-12-05 17:13:48+00:00",
            "title": "Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad"
            ],
            "abstract": "Rating scales are a widely used method for data annotation; however, they\npresent several challenges, such as difficulty in maintaining inter- and\nintra-annotator consistency. Best-worst scaling (BWS) is an alternative method\nof annotation that is claimed to produce high-quality annotations while keeping\nthe required number of annotations similar to that of rating scales. However,\nthe veracity of this claim has never been systematically established. Here for\nthe first time, we set up an experiment that directly compares the rating scale\nmethod with BWS. We show that with the same total number of annotations, BWS\nproduces significantly more reliable results than the rating scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01794v1",
            "date": "2017-12-05 18:17:43+00:00",
            "title": "The Effect of Negators, Modals, and Degree Adverbs on Sentiment Composition",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad"
            ],
            "abstract": "Negators, modals, and degree adverbs can significantly affect the sentiment\nof the words they modify. Often, their impact is modeled with simple\nheuristics; although, recent work has shown that such heuristics do not capture\nthe true sentiment of multi-word phrases. We created a dataset of phrases that\ninclude various negators, modals, and degree adverbs, as well as their\ncombinations. Both the phrases and their constituent content words were\nannotated with real-valued scores of sentiment association. Using phrasal terms\nin the created dataset, we analyze the impact of individual modifiers and the\naverage effect of the groups of modifiers on overall sentiment. We find that\nthe effect of modifiers varies substantially among the members of the same\ngroup. Furthermore, each individual modifier can affect sentiment words in\ndifferent ways. Therefore, solutions based on statistical learning seem more\npromising than fixed hand-crafted rules on the task of automatic sentiment\nprediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01797v1",
            "date": "2017-12-05 18:21:24+00:00",
            "title": "One for All: Towards Language Independent Named Entity Linking",
            "authors": [
                "Avirup Sil",
                "Radu Florian"
            ],
            "abstract": "Entity linking (EL) is the task of disambiguating mentions in text by\nassociating them with entries in a predefined database of mentions (persons,\norganizations, etc). Most previous EL research has focused mainly on one\nlanguage, English, with less attention being paid to other languages, such as\nSpanish or Chinese. In this paper, we introduce LIEL, a Language Independent\nEntity Linking system, which provides an EL framework which, once trained on\none language, works remarkably well on a number of different languages without\nchange. LIEL makes a joint global prediction over the entire document,\nemploying a discriminative reranking framework with many domain and\nlanguage-independent feature functions. Experiments on numerous benchmark\ndatasets, show that the proposed system, once trained on one language, English,\noutperforms several state-of-the-art systems in English (by 4 points) and the\ntrained model also works very well on Spanish (14 points better than a\ncompetitor system), demonstrating the viability of the approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01813v1",
            "date": "2017-12-05 18:43:57+00:00",
            "title": "Neural Cross-Lingual Entity Linking",
            "authors": [
                "Avirup Sil",
                "Gourab Kundu",
                "Radu Florian",
                "Wael Hamza"
            ],
            "abstract": "A major challenge in Entity Linking (EL) is making effective use of\ncontextual information to disambiguate mentions to Wikipedia that might refer\nto different entities in different contexts. The problem exacerbates with\ncross-lingual EL which involves linking mentions written in non-English\ndocuments to entries in the English Wikipedia: to compare textual clues across\nlanguages we need to compute similarity between textual fragments across\nlanguages. In this paper, we propose a neural EL model that trains fine-grained\nsimilarities and dissimilarities between the query and candidate document from\nmultiple perspectives, combined with convolution and tensor networks. Further,\nwe show that this English-trained system can be applied, in zero-shot learning,\nto other languages by making surprisingly effective use of multi-lingual\nembeddings. The proposed system has strong empirical evidence yielding\nstate-of-the-art results in English as well as cross-lingual: Spanish and\nChinese TAC 2015 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01821v1",
            "date": "2017-12-05 18:53:49+00:00",
            "title": "Neural Machine Translation by Generating Multiple Linguistic Factors",
            "authors": [
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Lo\u00efc Barrault",
                "Fethi Bougares"
            ],
            "abstract": "Factored neural machine translation (FNMT) is founded on the idea of using\nthe morphological and grammatical decomposition of the words (factors) at the\noutput side of the neural network. This architecture addresses two well-known\nproblems occurring in MT, namely the size of target language vocabulary and the\nnumber of unknown tokens produced in the translation. FNMT system is designed\nto manage larger vocabulary and reduce the training time (for systems with\nequivalent target language vocabulary size). Moreover, we can produce\ngrammatically correct words that are not part of the vocabulary. FNMT model is\nevaluated on IWSLT'15 English to French task and compared to the baseline\nword-based and BPE-based NMT systems. Promising qualitative and quantitative\nresults (in terms of BLEU and METEOR) are reported.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01969v2",
            "date": "2017-12-05 23:38:00+00:00",
            "title": "Strong Baselines for Simple Question Answering over Knowledge Graphs with and without Neural Networks",
            "authors": [
                "Salman Mohammed",
                "Peng Shi",
                "Jimmy Lin"
            ],
            "abstract": "We examine the problem of question answering over knowledge graphs, focusing\non simple questions that can be answered by the lookup of a single fact.\nAdopting a straightforward decomposition of the problem into entity detection,\nentity linking, relation prediction, and evidence combination, we explore\nsimple yet strong baselines. On the popular SimpleQuestions dataset, we find\nthat basic LSTMs and GRUs plus a few heuristics yield accuracies that approach\nthe state of the art, and techniques that do not use neural networks also\nperform reasonably well. These results show that gains from sophisticated deep\nlearning techniques proposed in the literature are quite modest and that some\nprevious models exhibit unnecessary complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02016v1",
            "date": "2017-12-06 03:11:51+00:00",
            "title": "Dual Attention Network for Product Compatibility and Function Satisfiability Analysis",
            "authors": [
                "Hu Xu",
                "Sihong Xie",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Product compatibility and their functionality are of utmost importance to\ncustomers when they purchase products, and to sellers and manufacturers when\nthey sell products. Due to the huge number of products available online, it is\ninfeasible to enumerate and test the compatibility and functionality of every\nproduct. In this paper, we address two closely related problems: product\ncompatibility analysis and function satisfiability analysis, where the second\nproblem is a generalization of the first problem (e.g., whether a product works\nwith another product can be considered as a special function). We first\nidentify a novel question and answering corpus that is up-to-date regarding\nproduct compatibility and functionality information. To allow automatic\ndiscovery product compatibility and functionality, we then propose a deep\nlearning model called Dual Attention Network (DAN). Given a QA pair for a\nto-be-purchased product, DAN learns to 1) discover complementary products (or\nfunctions), and 2) accurately predict the actual compatibility (or\nsatisfiability) of the discovered products (or functions). The challenges\naddressed by the model include the briefness of QAs, linguistic patterns\nindicating compatibility, and the appropriate fusion of questions and answers.\nWe conduct experiments to quantitatively and qualitatively show that the\nidentified products and functions have both high coverage and accuracy,\ncompared with a wide spectrum of baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02109v1",
            "date": "2017-12-06 09:59:43+00:00",
            "title": "Multi-channel Encoder for Neural Machine Translation",
            "authors": [
                "Hao Xiong",
                "Zhongjun He",
                "Xiaoguang Hu",
                "Hua Wu"
            ],
            "abstract": "Attention-based Encoder-Decoder has the effective architecture for neural\nmachine translation (NMT), which typically relies on recurrent neural networks\n(RNN) to build the blocks that will be lately called by attentive reader during\nthe decoding process. This design of encoder yields relatively uniform\ncomposition on source sentence, despite the gating mechanism employed in\nencoding RNN. On the other hand, we often hope the decoder to take pieces of\nsource sentence at varying levels suiting its own linguistic structure: for\nexample, we may want to take the entity name in its raw form while taking an\nidiom as a perfectly composed unit. Motivated by this demand, we propose\nMulti-channel Encoder (MCE), which enhances encoding components with different\nlevels of composition. More specifically, in addition to the hidden state of\nencoding RNN, MCE takes 1) the original word embedding for raw encoding with no\ncomposition, and 2) a particular design of external memory in Neural Turing\nMachine (NTM) for more complex composition, while all three encoding strategies\nare properly blended during decoding. Empirical study on Chinese-English\ntranslation shows that our model can improve by 6.52 BLEU points upon a strong\nopen source NMT system: DL4MT1. On the WMT14 English- French task, our single\nshallow system achieves BLEU=38.8, comparable with the state-of-the-art deep\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02121v2",
            "date": "2017-12-06 10:41:47+00:00",
            "title": "A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network",
            "authors": [
                "Dai Quoc Nguyen",
                "Tu Dinh Nguyen",
                "Dat Quoc Nguyen",
                "Dinh Phung"
            ],
            "abstract": "In this paper, we propose a novel embedding model, named ConvKB, for\nknowledge base completion. Our model ConvKB advances state-of-the-art models by\nemploying a convolutional neural network, so that it can capture global\nrelationships and transitional characteristics between entities and relations\nin knowledge bases. In ConvKB, each triple (head entity, relation, tail entity)\nis represented as a 3-column matrix where each column vector represents a\ntriple element. This 3-column matrix is then fed to a convolution layer where\nmultiple filters are operated on the matrix to generate different feature maps.\nThese feature maps are then concatenated into a single feature vector\nrepresenting the input triple. The feature vector is multiplied with a weight\nvector via a dot product to return a score. This score is then used to predict\nwhether the triple is valid or not. Experiments show that ConvKB achieves\nbetter link prediction performance than previous state-of-the-art embedding\nmodels on two benchmark datasets WN18RR and FB15k-237.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02186v1",
            "date": "2017-12-06 13:48:57+00:00",
            "title": "Product Function Need Recognition via Semi-supervised Attention Network",
            "authors": [
                "Hu Xu",
                "Sihong Xie",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Functionality is of utmost importance to customers when they purchase\nproducts. However, it is unclear to customers whether a product can really\nsatisfy their needs on functions. Further, missing functions may be\nintentionally hidden by the manufacturers or the sellers. As a result, a\ncustomer needs to spend a fair amount of time before purchasing or just\npurchase the product on his/her own risk. In this paper, we first identify a\nnovel QA corpus that is dense on product functionality information\n\\footnote{The annotated corpus can be found at\n\\url{https://www.cs.uic.edu/~hxu/}.}. We then design a neural network called\nSemi-supervised Attention Network (SAN) to discover product functions from\nquestions. This model leverages unlabeled data as contextual information to\nperform semi-supervised sequence labeling. We conduct experiments to show that\nthe extracted function have both high coverage and accuracy, compared with a\nwide spectrum of baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02480v1",
            "date": "2017-12-07 03:17:51+00:00",
            "title": "A Corpus of Deep Argumentative Structures as an Explanation to Argumentative Relations",
            "authors": [
                "Paul Reisert",
                "Naoya Inoue",
                "Naoaki Okazaki",
                "Kentaro Inui"
            ],
            "abstract": "In this paper, we compose a new task for deep argumentative structure\nanalysis that goes beyond shallow discourse structure analysis. The idea is\nthat argumentative relations can reasonably be represented with a small set of\npredefined patterns. For example, using value judgment and bipolar causality,\nwe can explain a support relation between two argumentative segments as\nfollows: Segment 1 states that something is good, and Segment 2 states that it\nis good because it promotes something good when it happens. We are motivated by\nthe following questions: (i) how do we formulate the task?, (ii) can a\nreasonable pattern set be created?, and (iii) do the patterns work? To examine\nthe task feasibility, we conduct a three-stage, detailed annotation study using\n357 argumentative relations from the argumentative microtext corpus, a small,\nbut highly reliable corpus. We report the coverage of explanations captured by\nour patterns on a test set composed of 270 relations. Our coverage result of\n74.6% indicates that argumentative relations can reasonably be explained by our\nsmall pattern set. Our agreement result of 85.9% shows that a reasonable\ninter-annotator agreement can be achieved. To assist with future work in\ncomputational argumentation, the annotated corpus is made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02555v3",
            "date": "2017-12-07 10:02:56+00:00",
            "title": "Hungarian Layer: Logics Empowered Neural Architecture",
            "authors": [
                "Han Xiao",
                "Yidong Chen",
                "Xiaodong Shi"
            ],
            "abstract": "Neural architecture is a purely numeric framework, which fits the data as a\ncontinuous function. However, lacking of logic flow (e.g. \\textit{if, for,\nwhile}), traditional algorithms (e.g. \\textit{Hungarian algorithm, A$^*$\nsearching, decision tress algorithm}) could not be embedded into this paradigm,\nwhich limits the theories and applications. In this paper, we reform the\ncalculus graph as a dynamic process, which is guided by logic flow. Within our\nnovel methodology, traditional algorithms could empower numerical neural\nnetwork. Specifically, regarding the subject of sentence matching, we\nreformulate this issue as the form of task-assignment, which is solved by\nHungarian algorithm. First, our model applies BiLSTM to parse the sentences.\nThen Hungarian layer aligns the matching positions. Last, we transform the\nmatching results for soft-max regression by another BiLSTM. Extensive\nexperiments show that our model outperforms other state-of-the-art baselines\nsubstantially.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02768v1",
            "date": "2017-12-06 08:39:29+00:00",
            "title": "Convolutional Neural Networks for Medical Diagnosis from Admission Notes",
            "authors": [
                "Christy Li",
                "Dimitris Konomis",
                "Graham Neubig",
                "Pengtao Xie",
                "Carol Cheng",
                "Eric Xing"
            ],
            "abstract": "$\\textbf{Objective}$ Develop an automatic diagnostic system which only uses\ntextual admission information from Electronic Health Records (EHRs) and assist\nclinicians with a timely and statistically proved decision tool. The hope is\nthat the tool can be used to reduce mis-diagnosis.\n  $\\textbf{Materials and Methods}$ We use the real-world clinical notes from\nMIMIC-III, a freely available dataset consisting of clinical data of more than\nforty thousand patients who stayed in intensive care units of the Beth Israel\nDeaconess Medical Center between 2001 and 2012. We proposed a Convolutional\nNeural Network model to learn semantic features from unstructured textual input\nand automatically predict primary discharge diagnosis.\n  $\\textbf{Results}$ The proposed model achieved an overall 96.11% accuracy and\n80.48% weighted F1 score values on 10 most frequent disease classes,\nsignificantly outperforming four strong baseline models by at least 12.7% in\nweighted F1 score.\n  $\\textbf{Discussion}$ Experimental results imply that the CNN model is\nsuitable for supporting diagnosis decision making in the presence of complex,\nnoisy and unstructured clinical data while at the same time using fewer layers\nand parameters that other traditional Deep Network models.\n  $\\textbf{Conclusion}$ Our model demonstrated capability of representing\ncomplex medical meaningful features from unstructured clinical notes and\nprediction power for commonly misdiagnosed frequent diseases. It can use easily\nadopted in clinical setting to provide timely and statistically proved decision\nsupport.\n  $\\textbf{Keywords}$ Convolutional neural network, text classification,\ndischarge diagnosis prediction, admission information from EHRs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02856v2",
            "date": "2017-12-07 20:48:15+00:00",
            "title": "Effective Neural Solution for Multi-Criteria Word Segmentation",
            "authors": [
                "Han He",
                "Lei Wu",
                "Hua Yan",
                "Zhimin Gao",
                "Yi Feng",
                "George Townsend"
            ],
            "abstract": "We present a simple yet elegant solution to train a single joint model on\nmulti-criteria corpora for Chinese Word Segmentation (CWS). Our novel design\nrequires no private layers in model architecture, instead, introduces two\nartificial tokens at the beginning and ending of input sentence to specify the\nrequired target criteria. The rest of the model including Long Short-Term\nMemory (LSTM) layer and Conditional Random Fields (CRFs) layer remains\nunchanged and is shared across all datasets, keeping the size of parameter\ncollection minimal and constant. On Bakeoff 2005 and Bakeoff 2008 datasets, our\ninnovative design has surpassed both single-criterion and multi-criteria\nstate-of-the-art learning results. To the best knowledge, our design is the\nfirst one that has achieved the latest high performance on such large scale\ndatasets. Source codes and corpora of this paper are available on GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02959v1",
            "date": "2017-12-08 06:36:54+00:00",
            "title": "Sequence to Sequence Networks for Roman-Urdu to Urdu Transliteration",
            "authors": [
                "Mehreen Alam",
                "Sibt ul Hussain"
            ],
            "abstract": "Neural Machine Translation models have replaced the conventional phrase based\nstatistical translation methods since the former takes a generic, scalable,\ndata-driven approach rather than relying on manual, hand-crafted features. The\nneural machine translation system is based on one neural network that is\ncomposed of two parts, one that is responsible for input language sentence and\nother part that handles the desired output language sentence. This model based\non encoder-decoder architecture also takes as input the distributed\nrepresentations of the source language which enriches the learnt dependencies\nand gives a warm start to the network. In this work, we transform Roman-Urdu to\nUrdu transliteration into sequence to sequence learning problem. To this end,\nwe make the following contributions. We create the first ever parallel corpora\nof Roman-Urdu to Urdu, create the first ever distributed representation of\nRoman-Urdu and present the first neural machine translation model that\ntransliterates text from Roman-Urdu to Urdu language. Our model has achieved\nthe state-of-the-art results using BLEU as the evaluation metric. Precisely,\nour model is able to correctly predict sentences up to length 10 while\nachieving BLEU score of 48.6 on the test set. We are hopeful that our model and\nour results shall serve as the baseline for further work in the domain of\nneural machine translation for Roman-Urdu to Urdu using distributed\nrepresentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03199v1",
            "date": "2017-12-08 17:52:32+00:00",
            "title": "Characterizing the hyper-parameter space of LSTM language models for mixed context applications",
            "authors": [
                "Victor Akinwande",
                "Sekou L. Remy"
            ],
            "abstract": "Applying state of the art deep learning models to novel real world datasets\ngives a practical evaluation of the generalizability of these models. Of\nimportance in this process is how sensitive the hyper parameters of such models\nare to novel datasets as this would affect the reproducibility of a model. We\npresent work to characterize the hyper parameter space of an LSTM for language\nmodeling on a code-mixed corpus. We observe that the evaluated model shows\nminimal sensitivity to our novel dataset bar a few hyper parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03376v2",
            "date": "2017-12-09 10:47:19+00:00",
            "title": "Word Sense Disambiguation with LSTM: Do We Really Need 100 Billion Words?",
            "authors": [
                "Minh Le",
                "Marten Postma",
                "Jacopo Urbani"
            ],
            "abstract": "Recently, Yuan et al. (2016) have shown the effectiveness of using Long\nShort-Term Memory (LSTM) for performing Word Sense Disambiguation (WSD). Their\nproposed technique outperformed the previous state-of-the-art with several\nbenchmarks, but neither the training data nor the source code was released.\nThis paper presents the results of a reproduction study of this technique using\nonly openly available datasets (GigaWord, SemCore, OMSTI) and software\n(TensorFlow). From them, it emerged that state-of-the-art results can be\nobtained with much less data than hinted by Yuan et al. All code and trained\nmodels are made freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03430v1",
            "date": "2017-12-09 20:06:52+00:00",
            "title": "Aspect Extraction and Sentiment Classification of Mobile Apps using App-Store Reviews",
            "authors": [
                "Sharmistha Dey"
            ],
            "abstract": "Understanding of customer sentiment can be useful for product development. On\ntop of that if the priorities for the development order can be known, then\ndevelopment procedure become simpler. This work has tried to address this issue\nin the mobile app domain. Along with aspect and opinion extraction this work\nhas also categorized the extracted aspects ac-cording to their importance. This\ncan help developers to focus their time and energy at the right place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03449v1",
            "date": "2017-12-09 23:17:22+00:00",
            "title": "Modulating and attending the source image during encoding improves Multimodal Translation",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont"
            ],
            "abstract": "We propose a new and fully end-to-end approach for multimodal translation\nwhere the source text encoder modulates the entire visual input processing\nusing conditional batch normalization, in order to compute the most informative\nimage features for our task. Additionally, we propose a new attention mechanism\nderived from this original idea, where the attention model for the visual input\nis conditioned on the source text encoder representations. In the paper, we\ndetail our models as well as the image analysis pipeline. Finally, we report\nexperimental results. They are, as far as we know, the new state of the art on\nthree different test sets.",
            "journal ref": "Visually-Grounded Interaction and Language, NIPS 2017 Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03463v2",
            "date": "2017-12-10 00:55:16+00:00",
            "title": "Learning Interpretable Spatial Operations in a Rich 3D Blocks World",
            "authors": [
                "Yonatan Bisk",
                "Kevin J. Shih",
                "Yejin Choi",
                "Daniel Marcu"
            ],
            "abstract": "In this paper, we study the problem of mapping natural language instructions\nto complex spatial actions in a 3D blocks world. We first introduce a new\ndataset that pairs complex 3D spatial operations to rich natural language\ndescriptions that require complex spatial and pragmatic interpretations such as\n\"mirroring\", \"twisting\", and \"balancing\". This dataset, built on the simulation\nenvironment of Bisk, Yuret, and Marcu (2016), attains language that is\nsignificantly richer and more complex, while also doubling the size of the\noriginal dataset in the 2D environment with 100 new world configurations and\n250,000 tokens. In addition, we propose a new neural architecture that achieves\ncompetitive results while automatically discovering an inventory of\ninterpretable spatial operations (Figure 5)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03538v1",
            "date": "2017-12-10 14:27:26+00:00",
            "title": "Multi-Task Learning for Mental Health using Social Media Text",
            "authors": [
                "Adrian Benton",
                "Margaret Mitchell",
                "Dirk Hovy"
            ],
            "abstract": "We introduce initial groundwork for estimating suicide risk and mental health\nin a deep learning framework. By modeling multiple conditions, the system\nlearns to make predictions about suicide risk and mental health at a low false\npositive rate. Conditions are modeled as tasks in a multi-task learning (MTL)\nframework, with gender prediction as an additional auxiliary task. We\ndemonstrate the effectiveness of multi-task learning by comparison to a\nwell-tuned single-task baseline with the same number of parameters. Our best\nMTL model predicts potential suicide attempt, as well as the presence of\natypical mental health, with AUC > 0.8. We also find additional large\nimprovements using multi-task learning on mental health tasks with limited\ntraining data.",
            "journal ref": "Proceedings of the 15th Conference of the EACL (2017) 152-162",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03547v1",
            "date": "2017-12-10 15:02:05+00:00",
            "title": "Inducing Interpretability in Knowledge Graph Embeddings",
            "authors": [
                "Chandrahas",
                "Tathagata Sengupta",
                "Cibi Pragadeesh",
                "Partha Pratim Talukdar"
            ],
            "abstract": "We study the problem of inducing interpretability in KG embeddings.\nSpecifically, we explore the Universal Schema (Riedel et al., 2013) and propose\na method to induce interpretability. There have been many vector space models\nproposed for the problem, however, most of these methods don't address the\ninterpretability (semantics) of individual dimensions. In this work, we study\nthis problem and propose a method for inducing interpretability in KG\nembeddings using entity co-occurrence statistics. The proposed method\nsignificantly improves the interpretability, while maintaining comparable\nperformance in other KG tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03556v2",
            "date": "2017-12-10 16:28:33+00:00",
            "title": "Stochastic Answer Networks for Machine Reading Comprehension",
            "authors": [
                "Xiaodong Liu",
                "Yelong Shen",
                "Kevin Duh",
                "Jianfeng Gao"
            ],
            "abstract": "We propose a simple yet robust stochastic answer network (SAN) that simulates\nmulti-step reasoning in machine reading comprehension. Compared to previous\nwork such as ReasoNet which used reinforcement learning to determine the number\nof steps, the unique feature is the use of a kind of stochastic prediction\ndropout on the answer module (final layer) of the neural network during the\ntraining. We show that this simple trick improves robustness and achieves\nresults competitive to the state-of-the-art on the Stanford Question Answering\nDataset (SQuAD), the Adversarial SQuAD, and the Microsoft MAchine Reading\nCOmprehension Dataset (MS MARCO).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03609v4",
            "date": "2017-12-10 23:16:02+00:00",
            "title": "Contextualized Word Representations for Reading Comprehension",
            "authors": [
                "Shimi Salant",
                "Jonathan Berant"
            ],
            "abstract": "Reading a document and extracting an answer to a question about its content\nhas attracted substantial attention recently. While most work has focused on\nthe interaction between the question and the document, in this work we evaluate\nthe importance of context when the question and document are processed\nindependently. We take a standard neural architecture for this task, and show\nthat by providing rich contextualized word representations from a large\npre-trained language model as well as allowing the model to choose between\ncontext-dependent and context-independent word representations, we can obtain\ndramatic improvements and reach performance comparable to state-of-the-art on\nthe competitive SQuAD dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03665v1",
            "date": "2017-12-11 07:41:28+00:00",
            "title": "Scale Up Event Extraction Learning via Automatic Training Data Generation",
            "authors": [
                "Ying Zeng",
                "Yansong Feng",
                "Rong Ma",
                "Zheng Wang",
                "Rui Yan",
                "Chongde Shi",
                "Dongyan Zhao"
            ],
            "abstract": "The task of event extraction has long been investigated in a supervised\nlearning paradigm, which is bound by the number and the quality of the training\ninstances. Existing training data must be manually generated through a\ncombination of expert domain knowledge and extensive human involvement.\nHowever, due to drastic efforts required in annotating text, the resultant\ndatasets are usually small, which severally affects the quality of the learned\nmodel, making it hard to generalize. Our work develops an automatic approach\nfor generating training data for event extraction. Our approach allows us to\nscale up event extraction training instances from thousands to hundreds of\nthousands, and it does this at a much lower cost than a manual approach. We\nachieve this by employing distant supervision to automatically create event\nannotations from unlabelled text using existing structured knowledge bases or\ntables.We then develop a neural network model with post inference to transfer\nthe knowledge extracted from structured knowledge bases to automatically\nannotate typed events with corresponding arguments in text.We evaluate our\napproach by using the knowledge extracted from Freebase to label texts from\nWikipedia articles. Experimental results show that our approach can generate a\nlarge number of high quality training instances. We show that this large volume\nof training data not only leads to a better event extractor, but also allows us\nto detect multiple typed events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03935v1",
            "date": "2017-12-11 18:32:11+00:00",
            "title": "On the Benefit of Combining Neural, Statistical and External Features for Fake News Identification",
            "authors": [
                "Gaurav Bhatt",
                "Aman Sharma",
                "Shivam Sharma",
                "Ankush Nagpal",
                "Balasubramanian Raman",
                "Ankush Mittal"
            ],
            "abstract": "Identifying the veracity of a news article is an interesting problem while\nautomating this process can be a challenging task. Detection of a news article\nas fake is still an open question as it is contingent on many factors which the\ncurrent state-of-the-art models fail to incorporate. In this paper, we explore\na subtask to fake news identification, and that is stance detection. Given a\nnews article, the task is to determine the relevance of the body and its claim.\nWe present a novel idea that combines the neural, statistical and external\nfeatures to provide an efficient solution to this problem. We compute the\nneural embedding from the deep recurrent model, statistical features from the\nweighted n-gram bag-of-words model and handcrafted external features with the\nhelp of feature engineering heuristics. Finally, using deep neural layer all\nthe features are combined, thereby classifying the headline-body news pair as\nagree, disagree, discuss, or unrelated. We compare our proposed technique with\nthe current state-of-the-art models on the fake news challenge dataset. Through\nextensive experiments, we find that the proposed model outperforms all the\nstate-of-the-art techniques including the submissions to the fake news\nchallenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04158v1",
            "date": "2017-12-12 08:03:17+00:00",
            "title": "Tracing a Loose Wordhood for Chinese Input Method Engine",
            "authors": [
                "Xihu Zhang",
                "Chu Wei",
                "Hai Zhao"
            ],
            "abstract": "Chinese input methods are used to convert pinyin sequence or other Latin\nencoding systems into Chinese character sentences. For more effective\npinyin-to-character conversion, typical Input Method Engines (IMEs) rely on a\npredefined vocabulary that demands manually maintenance on schedule. For the\npurpose of removing the inconvenient vocabulary setting, this work focuses on\nautomatic wordhood acquisition by fully considering that Chinese inputting is a\nfree human-computer interaction procedure. Instead of strictly defining words,\na loose word likelihood is introduced for measuring how likely a character\nsequence can be a user-recognized word with respect to using IME. Then an\nonline algorithm is proposed to adjust the word likelihood or generate new\nwords by comparing user true choice for inputting and the algorithm prediction.\nThe experimental results show that the proposed solution can agilely adapt to\ndiverse typings and demonstrate performance approaching highly-optimized IME\nwith fixed vocabulary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04313v1",
            "date": "2017-12-12 14:36:15+00:00",
            "title": "The Zero Resource Speech Challenge 2017",
            "authors": [
                "Ewan Dunbar",
                "Xuan Nga Cao",
                "Juan Benjumea",
                "Julien Karadayi",
                "Mathieu Bernard",
                "Laurent Besacier",
                "Xavier Anguera",
                "Emmanuel Dupoux"
            ],
            "abstract": "We describe a new challenge aimed at discovering subword and word units from\nraw speech. This challenge is the followup to the Zero Resource Speech\nChallenge 2015. It aims at constructing systems that generalize across\nlanguages and adapt to new speakers. The design features and evaluation metrics\nof the challenge are presented and the results of seventeen models are\ndiscussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04762v3",
            "date": "2017-12-11 20:03:22+00:00",
            "title": "Social Media Writing Style Fingerprint",
            "authors": [
                "Himank Yadav",
                "Juliang Li"
            ],
            "abstract": "We present our approach for computer-aided social media text authorship\nattribution based on recent advances in short text authorship verification. We\nuse various natural language techniques to create word-level and\ncharacter-level models that act as hidden layers to simulate a simple neural\nnetwork. The choice of word-level and character-level models in each layer was\ninformed through validation performance. The output layer of our system uses an\nunweighted majority vote vector to arrive at a conclusion. We also considered\nwriting bias in social media posts while collecting our training dataset to\nincrease system robustness. Our system achieved a precision, recall, and\nF-measure of 0.82, 0.926 and 0.869 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04853v3",
            "date": "2017-12-13 16:41:08+00:00",
            "title": "A User-Study on Online Adaptation of Neural Machine Translation to Human Post-Edits",
            "authors": [
                "Sariya Karimova",
                "Patrick Simianer",
                "Stefan Riezler"
            ],
            "abstract": "The advantages of neural machine translation (NMT) have been extensively\nvalidated for offline translation of several language pairs for different\ndomains of spoken and written language. However, research on interactive\nlearning of NMT by adaptation to human post-edits has so far been confined to\nsimulation experiments. We present the first user study on online adaptation of\nNMT to user post-edits in the domain of patent translation. Our study involves\n29 human subjects (translation students) whose post-editing effort and\ntranslation quality were measured on about 4,500 interactions of a human\npost-editor and a machine translation system integrating an online adaptive\nlearning algorithm. Our experimental results show a significant reduction of\nhuman post-editing effort due to online adaptation in NMT according to several\nevaluation metrics, including hTER, hBLEU, and KSMR. Furthermore, we found\nsignificant improvements in BLEU/TER between NMT outputs and professional\ntranslations in granted patents, providing further evidence for the advantages\nof online adaptive NMT in an interactive setup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05128v1",
            "date": "2017-12-14 08:40:10+00:00",
            "title": "Passing the Brazilian OAB Exam: data preparation and some experiments",
            "authors": [
                "Pedro Delfino",
                "Bruno Cuconato",
                "Edward Hermann Haeusler",
                "Alexandre Rademaker"
            ],
            "abstract": "In Brazil, all legal professionals must demonstrate their knowledge of the\nlaw and its application by passing the OAB exams, the national bar exams. The\nOAB exams therefore provide an excellent benchmark for the performance of legal\ninformation systems since passing the exam would arguably signal that the\nsystem has acquired capacity of legal reasoning comparable to that of a human\nlawyer. This article describes the construction of a new data set and some\npreliminary experiments on it, treating the problem of finding the\njustification for the answers to questions. The results provide a baseline\nperformance measure against which to evaluate future improvements. We discuss\nthe reasons to the poor performance and propose next steps.",
            "journal ref": "Legal Knowledge and Information Systems 302 (2017) 89-94",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05483v1",
            "date": "2017-12-15 00:12:47+00:00",
            "title": "Learning when to skim and when to read",
            "authors": [
                "Alexander Rosenberg Johansen",
                "Richard Socher"
            ],
            "abstract": "Many recent advances in deep learning for natural language processing have\ncome at increasing computational cost, but the power of these state-of-the-art\nmodels is not needed for every example in a dataset. We demonstrate two\napproaches to reducing unnecessary computation in cases where a fast but weak\nbaseline classier and a stronger, slower model are both available. Applying an\nAUC-based metric to the task of sentiment classification, we find significant\nefficiency gains with both a probability-threshold method for reducing\ncomputational cost and one that uses a secondary decision network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05626v2",
            "date": "2017-12-15 11:32:29+00:00",
            "title": "Avoiding Echo-Responses in a Retrieval-Based Conversation System",
            "authors": [
                "Denis Fedorenko",
                "Nikita Smetanin",
                "Artem Rodichev"
            ],
            "abstract": "Retrieval-based conversation systems generally tend to highly rank responses\nthat are semantically similar or even identical to the given conversation\ncontext. While the system's goal is to find the most appropriate response,\nrather than the most semantically similar one, this tendency results in\nlow-quality responses. We refer to this challenge as the echoing problem. To\nmitigate this problem, we utilize a hard negative mining approach at the\ntraining stage. The evaluation shows that the resulting model reduces echoing\nand achieves better results in terms of Average Precision and Recall@N metrics,\ncompared to the models trained without the proposed approach.",
            "journal ref": "In: Artificial Intelligence and Natural Language. AINL 2018. Vol\n  930. Springer",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05846v2",
            "date": "2017-12-15 21:33:07+00:00",
            "title": "Hierarchical Text Generation and Planning for Strategic Dialogue",
            "authors": [
                "Denis Yarats",
                "Mike Lewis"
            ],
            "abstract": "End-to-end models for goal-orientated dialogue are challenging to train,\nbecause linguistic and strategic aspects are entangled in latent state vectors.\nWe introduce an approach to learning representations of messages in dialogues\nby maximizing the likelihood of subsequent sentences and actions, which\ndecouples the semantics of the dialogue utterance from its linguistic\nrealization. We then use these latent sentence representations for hierarchical\nlanguage generation, planning and reinforcement learning. Experiments show that\nour approach increases the end-task reward achieved by the model, improves the\neffectiveness of long-term planning using rollouts, and allows self-play\nreinforcement learning to improve decision making without diverging from human\nlanguage. Our hierarchical latent-variable model outperforms previous work both\nlinguistically and strategically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05884v2",
            "date": "2017-12-16 00:51:40+00:00",
            "title": "Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions",
            "authors": [
                "Jonathan Shen",
                "Ruoming Pang",
                "Ron J. Weiss",
                "Mike Schuster",
                "Navdeep Jaitly",
                "Zongheng Yang",
                "Zhifeng Chen",
                "Yu Zhang",
                "Yuxuan Wang",
                "RJ Skerry-Ryan",
                "Rif A. Saurous",
                "Yannis Agiomyrgiannakis",
                "Yonghui Wu"
            ],
            "abstract": "This paper describes Tacotron 2, a neural network architecture for speech\nsynthesis directly from text. The system is composed of a recurrent\nsequence-to-sequence feature prediction network that maps character embeddings\nto mel-scale spectrograms, followed by a modified WaveNet model acting as a\nvocoder to synthesize timedomain waveforms from those spectrograms. Our model\nachieves a mean opinion score (MOS) of $4.53$ comparable to a MOS of $4.58$ for\nprofessionally recorded speech. To validate our design choices, we present\nablation studies of key components of our system and evaluate the impact of\nusing mel spectrograms as the input to WaveNet instead of linguistic, duration,\nand $F_0$ features. We further demonstrate that using a compact acoustic\nintermediate representation enables significant simplification of the WaveNet\narchitecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05898v2",
            "date": "2017-12-16 03:43:42+00:00",
            "title": "NegBio: a high-performance tool for negation and uncertainty detection in radiology reports",
            "authors": [
                "Yifan Peng",
                "Xiaosong Wang",
                "Le Lu",
                "Mohammadhadi Bagheri",
                "Ronald Summers",
                "Zhiyong Lu"
            ],
            "abstract": "Negative and uncertain medical findings are frequent in radiology reports,\nbut discriminating them from positive findings remains challenging for\ninformation extraction. Here, we propose a new algorithm, NegBio, to detect\nnegative and uncertain findings in radiology reports. Unlike previous\nrule-based methods, NegBio utilizes patterns on universal dependencies to\nidentify the scope of triggers that are indicative of negation or uncertainty.\nWe evaluated NegBio on four datasets, including two public benchmarking corpora\nof radiology reports, a new radiology corpus that we annotated for this work,\nand a public corpus of general clinical texts. Evaluation on these datasets\ndemonstrates that NegBio is highly accurate for detecting negative and\nuncertain findings and compares favorably to a widely-used state-of-the-art\nsystem NegEx (an average of 9.5% improvement in precision and 5.1% in\nF1-score).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05972v2",
            "date": "2017-12-16 15:17:07+00:00",
            "title": "Train Once, Test Anywhere: Zero-Shot Learning for Text Classification",
            "authors": [
                "Pushpankar Kumar Pushp",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "Zero-shot Learners are models capable of predicting unseen classes. In this\nwork, we propose a Zero-shot Learning approach for text categorization. Our\nmethod involves training model on a large corpus of sentences to learn the\nrelationship between a sentence and embedding of sentence's tags. Learning such\nrelationship makes the model generalize to unseen sentences, tags, and even new\ndatasets provided they can be put into same embedding space. The model learns\nto predict whether a given sentence is related to a tag or not; unlike other\nclassifiers that learn to classify the sentence as one of the possible classes.\nWe propose three different neural networks for the task and report their\naccuracy on the test set of the dataset used for training them as well as two\nother standard datasets for which no retraining was done. We show that our\nmodels generalize well across new unseen classes in both cases. Although the\nmodels do not achieve the accuracy level of the state of the art supervised\nmodels, yet it evidently is a step forward towards general intelligence in\nnatural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06100v1",
            "date": "2017-12-17 12:42:51+00:00",
            "title": "Query-Based Abstractive Summarization Using Neural Networks",
            "authors": [
                "Johan Hasselqvist",
                "Niklas Helmertz",
                "Mikael K\u00e5geb\u00e4ck"
            ],
            "abstract": "In this paper, we present a model for generating summaries of text documents\nwith respect to a query. This is known as query-based summarization. We adapt\nan existing dataset of news article summaries for the task and train a\npointer-generator model using this dataset. The generated summaries are\nevaluated by measuring similarity to reference summaries. Our results show that\na neural network summarization model, similar to existing neural network models\nfor abstractive summarization, can be constructed to make use of queries to\nproduce targeted summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06163v1",
            "date": "2017-12-17 19:33:36+00:00",
            "title": "Towards a science of human stories: using sentiment analysis and emotional arcs to understand the building blocks of complex social systems",
            "authors": [
                "Andrew J. Reagan"
            ],
            "abstract": "Given the growing assortment of sentiment measuring instruments, it is\nimperative to understand which aspects of sentiment dictionaries contribute to\nboth their classification accuracy and their ability to provide richer\nunderstanding of texts. Here, we perform detailed, quantitative tests and\nqualitative assessments of 6 dictionary-based methods applied, and briefly\nexamine a further 20 methods. We show that while inappropriate for sentences,\ndictionary-based methods are generally robust in their classification accuracy\nfor longer texts.\n  Stories often following distinct emotional trajectories, forming patterns\nthat are meaningful to us. By classifying the emotional arcs for a filtered\nsubset of 4,803 stories from Project Gutenberg's fiction collection, we find a\nset of six core trajectories which form the building blocks of complex\nnarratives. Of profound scientific interest will be the degree to which we can\neventually understand the full landscape of human stories, and data driven\napproaches will play a crucial role.\n  Finally, we utilize web-scale data from Twitter to study the limits of what\nsocial data can tell us about public health, mental illness, discourse around\nthe protest movement of #BlackLivesMatter, discourse around climate change, and\nhidden networks. We conclude with a review of published works in complex\nsystems that separately analyze charitable donations, the happiness of words in\n10 languages, 100 years of daily temperature data across the United States, and\nAustralian Rules Football games.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06273v1",
            "date": "2017-12-18 07:04:10+00:00",
            "title": "Low Resourced Machine Translation via Morpho-syntactic Modeling: The Case of Dialectal Arabic",
            "authors": [
                "Alexander Erdmann",
                "Nizar Habash",
                "Dima Taji",
                "Houda Bouamor"
            ],
            "abstract": "We present the second ever evaluated Arabic dialect-to-dialect machine\ntranslation effort, and the first to leverage external resources beyond a small\nparallel corpus. The subject has not previously received serious attention due\nto lack of naturally occurring parallel data; yet its importance is evidenced\nby dialectal Arabic's wide usage and breadth of inter-dialect variation,\ncomparable to that of Romance languages. Our results suggest that modeling\nmorphology and syntax significantly improves dialect-to-dialect translation,\nthough optimizing such data-sparse models requires consideration of the\nlinguistic differences between dialects and the nature of available data and\nresources. On a single-reference blind test set where untranslated input scores\n6.5 BLEU and a model trained only on parallel data reaches 14.6, pivot\ntechniques and morphosyntactic modeling significantly improve performance to\n17.5.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06289v1",
            "date": "2017-12-18 08:24:12+00:00",
            "title": "A Chinese Dataset with Negative Full Forms for General Abbreviation Prediction",
            "authors": [
                "Yi Zhang",
                "Xu Sun"
            ],
            "abstract": "Abbreviation is a common phenomenon across languages, especially in Chinese.\nIn most cases, if an expression can be abbreviated, its abbreviation is used\nmore often than its fully expanded forms, since people tend to convey\ninformation in a most concise way. For various language processing tasks,\nabbreviation is an obstacle to improving the performance, as the textual form\nof an abbreviation does not express useful information, unless it's expanded to\nthe full form. Abbreviation prediction means associating the fully expanded\nforms with their abbreviations. However, due to the deficiency in the\nabbreviation corpora, such a task is limited in current studies, especially\nconsidering general abbreviation prediction should also include those full form\nexpressions that do not have valid abbreviations, namely the negative full\nforms (NFFs). Corpora incorporating negative full forms for general\nabbreviation prediction are few in number. In order to promote the research in\nthis area, we build a dataset for general Chinese abbreviation prediction,\nwhich needs a few preprocessing steps, and evaluate several different models on\nthe built dataset. The dataset is available at\nhttps://github.com/lancopku/Chinese-abbreviation-dataset",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06427v2",
            "date": "2017-12-18 14:39:57+00:00",
            "title": "Detecting Hate Speech in Social Media",
            "authors": [
                "Shervin Malmasi",
                "Marcos Zampieri"
            ],
            "abstract": "In this paper we examine methods to detect hate speech in social media, while\ndistinguishing this from general profanity. We aim to establish lexical\nbaselines for this task by applying supervised classification methods using a\nrecently released dataset annotated for this purpose. As features, our system\nuses character n-grams, word n-grams and word skip-grams. We obtain results of\n78% accuracy in identifying posts across three classes. Results demonstrate\nthat the main challenge lies in discriminating profanity and hate speech from\neach other. A number of directions for future work are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06674v1",
            "date": "2017-12-18 21:06:42+00:00",
            "title": "word representation or word embedding in Persian text",
            "authors": [
                "Siamak Sarmady",
                "Erfan Rahmani"
            ],
            "abstract": "Text processing is one of the sub-branches of natural language processing.\nRecently, the use of machine learning and neural networks methods has been\ngiven greater consideration. For this reason, the representation of words has\nbecome very important. This article is about word representation or converting\nwords into vectors in Persian text. In this research GloVe, CBOW and skip-gram\nmethods are updated to produce embedded vectors for Persian words. In order to\ntrain a neural networks, Bijankhan corpus, Hamshahri corpus and UPEC corpus\nhave been compound and used. Finally, we have 342,362 words that obtained\nvectors in all three models for this words. These vectors have many usage for\nPersian natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06855v2",
            "date": "2017-12-19 10:29:28+00:00",
            "title": "Subword and Crossword Units for CTC Acoustic Models",
            "authors": [
                "Thomas Zenkel",
                "Ramon Sanabria",
                "Florian Metze",
                "Alex Waibel"
            ],
            "abstract": "This paper proposes a novel approach to create an unit set for CTC based\nspeech recognition systems. By using Byte Pair Encoding we learn an unit set of\nan arbitrary size on a given training text. In contrast to using characters or\nwords as units this allows us to find a good trade-off between the size of our\nunit set and the available training data. We evaluate both Crossword units,\nthat may span multiple word, and Subword units. By combining this approach with\ndecoding methods using a separate language model we are able to achieve state\nof the art results for grapheme based CTC systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06880v1",
            "date": "2017-12-19 11:47:53+00:00",
            "title": "Analogy Mining for Specific Design Needs",
            "authors": [
                "Karni Gilon",
                "Felicia Y Ng",
                "Joel Chan",
                "Hila Lifshitz Assaf",
                "Aniket Kittur",
                "Dafna Shahaf"
            ],
            "abstract": "Finding analogical inspirations in distant domains is a powerful way of\nsolving problems. However, as the number of inspirations that could be matched\nand the dimensions on which that matching could occur grow, it becomes\nchallenging for designers to find inspirations relevant to their needs.\nFurthermore, designers are often interested in exploring specific aspects of a\nproduct-- for example, one designer might be interested in improving the\nbrewing capability of an outdoor coffee maker, while another might wish to\noptimize for portability. In this paper we introduce a novel system for\ntargeting analogical search for specific needs. Specifically, we contribute a\nnovel analogical search engine for expressing and abstracting specific design\nneeds that returns more distant yet relevant inspirations than alternate\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06961v2",
            "date": "2017-12-19 14:52:47+00:00",
            "title": "Unsupervised Word Mapping Using Structural Similarities in Monolingual Embeddings",
            "authors": [
                "Hanan Aldarmaki",
                "Mahesh Mohan",
                "Mona Diab"
            ],
            "abstract": "Most existing methods for automatic bilingual dictionary induction rely on\nprior alignments between the source and target languages, such as parallel\ncorpora or seed dictionaries. For many language pairs, such supervised\nalignments are not readily available. We propose an unsupervised approach for\nlearning a bilingual dictionary for a pair of languages given their\nindependently-learned monolingual word embeddings. The proposed method exploits\nlocal and global structures in monolingual vector spaces to align them such\nthat similar words are mapped to each other. We show empirically that the\nperformance of bilingual correspondents learned using our proposed unsupervised\nmethod is comparable to that of using supervised bilingual correspondents from\na seed dictionary.",
            "journal ref": "TACL 6 (2018) 185-196",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06994v1",
            "date": "2017-12-17 18:31:26+00:00",
            "title": "DeepNorm-A Deep Learning Approach to Text Normalization",
            "authors": [
                "Maryam Zare",
                "Shaurya Rohatgi"
            ],
            "abstract": "This paper presents an simple yet sophisticated approach to the challenge by\nSproat and Jaitly (2016)- given a large corpus of written text aligned to its\nnormalized spoken form, train an RNN to learn the correct normalization\nfunction. Text normalization for a token seems very straightforward without\nit's context. But given the context of the used token and then normalizing\nbecomes tricky for some classes. We present a novel approach in which the\nprediction of our classification algorithm is used by our sequence to sequence\nmodel to predict the normalized text of the input token. Our approach takes\nvery less time to learn and perform well unlike what has been reported by\nGoogle (5 days on their GPU cluster). We have achieved an accuracy of 97.62\nwhich is impressive given the resources we use. Our approach is using the best\nof both worlds, gradient boosting - state of the art in most classification\ntasks and sequence to sequence learning - state of the art in machine\ntranslation. We present our experiments and report results with various\nparameter settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07229v1",
            "date": "2017-12-19 21:43:59+00:00",
            "title": "Attentive Memory Networks: Efficient Machine Reading for Conversational Search",
            "authors": [
                "Tom Kenter",
                "Maarten de Rijke"
            ],
            "abstract": "Recent advances in conversational systems have changed the search paradigm.\nTraditionally, a user poses a query to a search engine that returns an answer\nbased on its index, possibly leveraging external knowledge bases and\nconditioning the response on earlier interactions in the search session. In a\nnatural conversation, there is an additional source of information to take into\naccount: utterances produced earlier in a conversation can also be referred to\nand a conversational IR system has to keep track of information conveyed by the\nuser during the conversation, even if it is implicit.\n  We argue that the process of building a representation of the conversation\ncan be framed as a machine reading task, where an automated system is presented\nwith a number of statements about which it should answer questions. The\nquestions should be answered solely by referring to the statements provided,\nwithout consulting external knowledge. The time is right for the information\nretrieval community to embrace this task, both as a stand-alone task and\nintegrated in a broader conversational search setting.\n  In this paper, we focus on machine reading as a stand-alone task and present\nthe Attentive Memory Network (AMN), an end-to-end trainable machine reading\nalgorithm. Its key contribution is in efficiency, achieved by having an\nhierarchical input encoder, iterating over the input only once. Speed is an\nimportant requirement in the setting of conversational search, as gaps between\nconversational turns have a detrimental effect on naturalness. On 20 datasets\ncommonly used for evaluating machine reading algorithms we show that the AMN\nachieves performance comparable to the state-of-the-art models, while using\nconsiderably fewer computations.",
            "journal ref": "Proceedings of 1st International Workshop on Conversational\n  Approaches to Information Retrieval, Tokyo, Japan, August 11, 2017 (CAIR'17)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07558v1",
            "date": "2017-12-20 16:28:48+00:00",
            "title": "An Ensemble Model with Ranking for Social Dialogue",
            "authors": [
                "Ioannis Papaioannou",
                "Amanda Cercas Curry",
                "Jose L. Part",
                "Igor Shalyminov",
                "Xinnuo Xu",
                "Yanchao Yu",
                "Ond\u0159ej Du\u0161ek",
                "Verena Rieser",
                "Oliver Lemon"
            ],
            "abstract": "Open-domain social dialogue is one of the long-standing goals of Artificial\nIntelligence. This year, the Amazon Alexa Prize challenge was announced for the\nfirst time, where real customers get to rate systems developed by leading\nuniversities worldwide. The aim of the challenge is to converse \"coherently and\nengagingly with humans on popular topics for 20 minutes\". We describe our Alexa\nPrize system (called 'Alana') consisting of an ensemble of bots, combining\nrule-based and machine learning systems, and using a contextual ranking\nmechanism to choose a system response. The ranker was trained on real user\nfeedback received during the competition, where we address the problem of how\nto train on the noisy and sparse feedback obtained during the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07794v1",
            "date": "2017-12-21 05:16:51+00:00",
            "title": "The Character Thinks Ahead: creative writing with deep learning nets and its stylistic assessment",
            "authors": [
                "Roger T. Dean",
                "Hazel Smith"
            ],
            "abstract": "We discuss how to control outputs from deep learning models of text corpora\nso as to create contemporary poetic works. We assess whether these controls are\nsuccessful in the immediate sense of creating stylo- metric distinctiveness.\nThe specific context is our piece The Character Thinks Ahead (2016/17); the\npotential applications are broad.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08207v3",
            "date": "2017-12-21 20:47:27+00:00",
            "title": "Variational Attention for Sequence-to-Sequence Models",
            "authors": [
                "Hareesh Bahuleyan",
                "Lili Mou",
                "Olga Vechtomova",
                "Pascal Poupart"
            ],
            "abstract": "The variational encoder-decoder (VED) encodes source information as a set of\nrandom variables using a neural network, which in turn is decoded into target\ndata using another neural network. In natural language processing,\nsequence-to-sequence (Seq2Seq) models typically serve as encoder-decoder\nnetworks. When combined with a traditional (deterministic) attention mechanism,\nthe variational latent space may be bypassed by the attention model, and thus\nbecomes ineffective. In this paper, we propose a variational attention\nmechanism for VED, where the attention vector is also modeled as Gaussian\ndistributed random variables. Results on two experiments show that, without\nloss of quality, our proposed method alleviates the bypassing phenomenon as it\nincreases the diversity of generated sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08291v1",
            "date": "2017-12-22 03:21:05+00:00",
            "title": "TFW, DamnGina, Juvie, and Hotsie-Totsie: On the Linguistic and Social Aspects of Internet Slang",
            "authors": [
                "Vivek Kulkarni",
                "William Yang Wang"
            ],
            "abstract": "Slang is ubiquitous on the Internet. The emergence of new social contexts\nlike micro-blogs, question-answering forums, and social networks has enabled\nslang and non-standard expressions to abound on the web. Despite this, slang\nhas been traditionally viewed as a form of non-standard language -- a form of\nlanguage that is not the focus of linguistic analysis and has largely been\nneglected. In this work, we use UrbanDictionary to conduct the first\nlarge-scale linguistic analysis of slang and its social aspects on the Internet\nto yield insights into this variety of language that is increasingly used all\nover the world online.\n  We begin by computationally analyzing the phonological, morphological and\nsyntactic properties of slang. We then study linguistic patterns in four\nspecific categories of slang namely alphabetisms, blends, clippings, and\nreduplicatives. Our analysis reveals that slang demonstrates extra-grammatical\nrules of phonological and morphological formation that markedly distinguish it\nfrom the standard form shedding insight into its generative patterns. Next, we\nanalyze the social aspects of slang by studying subject restriction and\nstereotyping in slang usage. Analyzing tens of thousands of such slang words\nreveals that the majority of slang on the Internet belongs to two major\ncategories: sex and drugs. We also noted that not only is slang usage not\nimmune to prevalent social biases and prejudices but also reflects such biases\nand stereotypes more intensely than the standard variety.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08302v1",
            "date": "2017-12-22 04:36:07+00:00",
            "title": "Source-side Prediction for Neural Headline Generation",
            "authors": [
                "Shun Kiyono",
                "Sho Takase",
                "Jun Suzuki",
                "Naoaki Okazaki",
                "Kentaro Inui",
                "Masaaki Nagata"
            ],
            "abstract": "The encoder-decoder model is widely used in natural language generation\ntasks. However, the model sometimes suffers from repeated redundant generation,\nmisses important phrases, and includes irrelevant entities. Toward solving\nthese problems we propose a novel source-side token prediction module. Our\nmethod jointly estimates the probability distributions over source and target\nvocabularies to capture a correspondence between source and target tokens. The\nexperiments show that the proposed model outperforms the current\nstate-of-the-art method in the headline generation task. Additionally, we show\nthat our method has an ability to learn a reasonable token-wise correspondence\nwithout knowing any true alignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08439v1",
            "date": "2017-12-22 13:40:41+00:00",
            "title": "Novel Ranking-Based Lexical Similarity Measure for Word Embedding",
            "authors": [
                "Jakub Dutkiewicz",
                "Czes\u0142aw J\u0119drzejek"
            ],
            "abstract": "Distributional semantics models derive word space from linguistic items in\ncontext. Meaning is obtained by defining a distance measure between vectors\ncorresponding to lexical entities. Such vectors present several problems. In\nthis paper we provide a guideline for post process improvements to the baseline\nvectors. We focus on refining the similarity aspect, address imperfections of\nthe model by applying the hubness reduction method, implementing relational\nknowledge into the model, and providing a new ranking similarity definition\nthat give maximum weight to the top 1 component value. This feature ranking is\nsimilar to the one used in information retrieval. All these enrichments\noutperform current literature results for joint ESL and TOEF sets comparison.\nSince single word embedding is a basic element of any semantic task one can\nexpect a significant improvement of results for these tasks. Moreover, our\nimproved method of text processing can be translated to continuous distributed\nrepresentation of biological sequences for deep proteomics and genomics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08793v1",
            "date": "2017-12-23 15:55:54+00:00",
            "title": "Are words easier to learn from infant- than adult-directed speech? A quantitative corpus-based investigation",
            "authors": [
                "Adriana Guevara-Rukoz",
                "Alejandrina Cristia",
                "Bogdan Ludusan",
                "Roland Thiolli\u00e8re",
                "Andrew Martin",
                "Reiko Mazuka",
                "Emmanuel Dupoux"
            ],
            "abstract": "We investigate whether infant-directed speech (IDS) could facilitate word\nform learning when compared to adult-directed speech (ADS). To study this, we\nexamine the distribution of word forms at two levels, acoustic and\nphonological, using a large database of spontaneous speech in Japanese. At the\nacoustic level we show that, as has been documented before for phonemes, the\nrealizations of words are more variable and less discriminable in IDS than in\nADS. At the phonological level, we find an effect in the opposite direction:\nthe IDS lexicon contains more distinctive words (such as onomatopoeias) than\nthe ADS counterpart. Combining the acoustic and phonological metrics together\nin a global discriminability score reveals that the bigger separation of\nlexical categories in the phonological space does not compensate for the\nopposite effect observed at the acoustic level. As a result, IDS word forms are\nstill globally less discriminable than ADS word forms, even though the effect\nis numerically small. We discuss the implication of these findings for the view\nthat the functional role of IDS is to improve language learnability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08819v1",
            "date": "2017-12-23 18:46:58+00:00",
            "title": "A Framework for Enriching Lexical Semantic Resources with Distributional Semantics",
            "authors": [
                "Chris Biemann",
                "Stefano Faralli",
                "Alexander Panchenko",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "We present an approach to combining distributional semantic representations\ninduced from text corpora with manually constructed lexical-semantic networks.\nWhile both kinds of semantic resources are available with high lexical\ncoverage, our aligned resource combines the domain specificity and availability\nof contextual information from distributional models with the conciseness and\nhigh quality of manually crafted lexical networks. We start with a\ndistributional representation of induced senses of vocabulary terms, which are\naccompanied with rich context information given by related lexical items. We\nthen automatically disambiguate such representations to obtain a full-fledged\nproto-conceptualization, i.e. a typed graph of induced word senses. In a final\nstep, this proto-conceptualization is aligned to a lexical ontology, resulting\nin a hybrid aligned resource. Moreover, unmapped induced senses are associated\nwith a semantic type in order to connect them to the core resource. Manual\nevaluations against ground-truth judgments for different stages of our method\nas well as an extrinsic evaluation on a knowledge-based Word Sense\nDisambiguation benchmark all indicate the high quality of the new hybrid\nresource. Additionally, we show the benefits of enriching top-down lexical\nknowledge resources with bottom-up distributional information from text for\naddressing high-end knowledge acquisition tasks such as cleaning hypernym\ngraphs and learning taxonomies from scratch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08841v2",
            "date": "2017-12-23 21:12:23+00:00",
            "title": "Dual Long Short-Term Memory Networks for Sub-Character Representation Learning",
            "authors": [
                "Han He",
                "Lei Wu",
                "Xiaokun Yang",
                "Hua Yan",
                "Zhimin Gao",
                "Yi Feng",
                "George Townsend"
            ],
            "abstract": "Characters have commonly been regarded as the minimal processing unit in\nNatural Language Processing (NLP). But many non-latin languages have\nhieroglyphic writing systems, involving a big alphabet with thousands or\nmillions of characters. Each character is composed of even smaller parts, which\nare often ignored by the previous work. In this paper, we propose a novel\narchitecture employing two stacked Long Short-Term Memory Networks (LSTMs) to\nlearn sub-character level representation and capture deeper level of semantic\nmeanings. To build a concrete study and substantiate the efficiency of our\nneural architecture, we take Chinese Word Segmentation as a research case\nexample. Among those languages, Chinese is a typical case, for which every\ncharacter contains several components called radicals. Our networks employ a\nshared radical level embedding to solve both Simplified and Traditional Chinese\nWord Segmentation, without extra Traditional to Simplified Chinese conversion,\nin such a highly end-to-end way the word segmentation can be significantly\nsimplified compared to the previous work. Radical level embeddings can also\ncapture deeper semantic meaning below character level and improve the system\nperformance of learning. By tying radical and character embeddings together,\nthe parameter count is reduced whereas semantic knowledge is shared and\ntransferred between two levels, boosting the performance largely. On 3 out of 4\nBakeoff 2005 datasets, our method surpassed state-of-the-art results by up to\n0.4%. Our results are reproducible, source codes and corpora are available on\nGitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08917v1",
            "date": "2017-12-24 13:23:58+00:00",
            "title": "Building a Sentiment Corpus of Tweets in Brazilian Portuguese",
            "authors": [
                "Henrico Bertini Brum",
                "Maria das Gra\u00e7as Volpe Nunes"
            ],
            "abstract": "The large amount of data available in social media, forums and websites\nmotivates researches in several areas of Natural Language Processing, such as\nsentiment analysis. The popularity of the area due to its subjective and\nsemantic characteristics motivates research on novel methods and approaches for\nclassification. Hence, there is a high demand for datasets on different domains\nand different languages. This paper introduces TweetSentBR, a sentiment corpora\nfor Brazilian Portuguese manually annotated with 15.000 sentences on TV show\ndomain. The sentences were labeled in three classes (positive, neutral and\nnegative) by seven annotators, following literature guidelines for ensuring\nreliability on the annotation. We also ran baseline experiments on polarity\nclassification using three machine learning methods, reaching 80.99% on\nF-Measure and 82.06% on accuracy in binary classification, and 59.85% F-Measure\nand 64.62% on accuracy on three point classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08933v1",
            "date": "2017-12-24 14:54:22+00:00",
            "title": "Semi-automatic definite description annotation: a first report",
            "authors": [
                "Danillo da Silva Rocha",
                "Alex Gwo Jen Lan",
                "Ivandre Paraboni"
            ],
            "abstract": "Studies in Referring Expression Generation (REG) often make use of corpora of\ndefinite descriptions produced by human subjects in controlled experiments.\nExperiments of this kind, which are essential for the study of reference\nphenomena and many others, may however include a considerable amount of noise.\nHuman subjects may easily lack attention, or may simply misunderstand the task\nat hand and, as a result, the elicited data may include large proportions of\nambiguous or ill-formed descriptions. In addition to that, REG corpora are\nusually collected for the study of semantics-related phenomena, and it is often\nthe case that the elicited descriptions (and their input contexts) need to be\nannotated with their corresponding semantic properties. This, as in many other\nfields, may require considerable time and skilled annotators. As a means to\ntackle both kinds of difficulties - poor data quality and high annotation costs\n- this work discusses a semi-automatic method for the annotation of definite\ndescriptions produced by human subjects in REG data collection experiments. The\nmethod makes use of simple rules to establish associations between words and\nmeanings, and is intended to facilitate the design of experiments that produce\nREG corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09127v1",
            "date": "2017-12-25 20:16:07+00:00",
            "title": "Generative Adversarial Nets for Multiple Text Corpora",
            "authors": [
                "Baiyang Wang",
                "Diego Klabjan"
            ],
            "abstract": "Generative adversarial nets (GANs) have been successfully applied to the\nartificial generation of image data. In terms of text data, much has been done\non the artificial generation of natural language from a single corpus. We\nconsider multiple text corpora as the input data, for which there can be two\napplications of GANs: (1) the creation of consistent cross-corpus word\nembeddings given different word embeddings per corpus; (2) the generation of\nrobust bag-of-words document embeddings for each corpora. We demonstrate our\nGAN models on real-world text data sets from different corpora, and show that\nembeddings from both models lead to improvements in supervised learning\nproblems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09185v1",
            "date": "2017-12-26 06:02:36+00:00",
            "title": "Actionable Email Intent Modeling with Reparametrized RNNs",
            "authors": [
                "Chu-Cheng Lin",
                "Dongyeop Kang",
                "Michael Gamon",
                "Madian Khabsa",
                "Ahmed Hassan Awadallah",
                "Patrick Pantel"
            ],
            "abstract": "Emails in the workplace are often intentional calls to action for its\nrecipients. We propose to annotate these emails for what action its recipient\nwill take. We argue that our approach of action-based annotation is more\nscalable and theory-agnostic than traditional speech-act-based email intent\nannotation, while still carrying important semantic and pragmatic information.\nWe show that our action-based annotation scheme achieves good inter-annotator\nagreement. We also show that we can leverage threaded messages from other\ndomains, which exhibit comparable intents in their conversation, with domain\nadaptive RAINBOW (Recurrently AttentIve Neural Bag-Of-Words). On a collection\nof datasets consisting of IRC, Reddit, and email, our reparametrized RNNs\noutperform common multitask/multidomain approaches on several speech act\nrelated tasks. We also experiment with a minimally supervised scenario of email\nrecipient action classification, and find the reparametrized RNNs learn a\nuseful representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09391v1",
            "date": "2017-12-26 20:21:09+00:00",
            "title": "Mapping to Declarative Knowledge for Word Problem Solving",
            "authors": [
                "Subhro Roy",
                "Dan Roth"
            ],
            "abstract": "Math word problems form a natural abstraction to a range of quantitative\nreasoning problems, such as understanding financial news, sports results, and\ncasualties of war. Solving such problems requires the understanding of several\nmathematical concepts such as dimensional analysis, subset relationships, etc.\nIn this paper, we develop declarative rules which govern the translation of\nnatural language description of these concepts to math expressions. We then\npresent a framework for incorporating such declarative knowledge into word\nproblem solving. Our method learns to map arithmetic word problem text to math\nexpressions, by learning to select the relevant declarative knowledge for each\noperation of the solution expression. This provides a way to handle multiple\nconcepts in the same problem while, at the same time, support interpretability\nof the answer expression. Our method models the mapping to declarative\nknowledge as a latent variable, thus removing the need for expensive\nannotations. Experimental evaluation suggests that our domain knowledge based\nsolver outperforms all other systems, and that it generalizes better in the\nrealistic case where the training data it is exposed to is biased in a\ndifferent way than the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09405v1",
            "date": "2017-12-26 21:00:04+00:00",
            "title": "Advances in Pre-Training Distributed Word Representations",
            "authors": [
                "Tomas Mikolov",
                "Edouard Grave",
                "Piotr Bojanowski",
                "Christian Puhrsch",
                "Armand Joulin"
            ],
            "abstract": "Many Natural Language Processing applications nowadays rely on pre-trained\nword representations estimated from large text corpora such as news\ncollections, Wikipedia and Web Crawl. In this paper, we show how to train\nhigh-quality word vector representations by using a combination of known tricks\nthat are however rarely used together. The main result of our work is the new\nset of publicly available pre-trained models that outperform the current state\nof the art by a large margin on a number of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09509v1",
            "date": "2017-12-27 06:44:02+00:00",
            "title": "A Gap-Based Framework for Chinese Word Segmentation via Very Deep Convolutional Networks",
            "authors": [
                "Zhiqing Sun",
                "Gehui Shen",
                "Zhihong Deng"
            ],
            "abstract": "Most previous approaches to Chinese word segmentation can be roughly\nclassified into character-based and word-based methods. The former regards this\ntask as a sequence-labeling problem, while the latter directly segments\ncharacter sequence into words. However, if we consider segmenting a given\nsentence, the most intuitive idea is to predict whether to segment for each gap\nbetween two consecutive characters, which in comparison makes previous\napproaches seem too complex. Therefore, in this paper, we propose a gap-based\nframework to implement this intuitive idea. Moreover, very deep convolutional\nneural networks, namely, ResNets and DenseNets, are exploited in our\nexperiments. Results show that our approach outperforms the best\ncharacter-based and word-based methods on 5 benchmarks, without any further\npost-processing module (e.g. Conditional Random Fields) nor beam search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09518v1",
            "date": "2017-12-27 08:02:26+00:00",
            "title": "Improving Text Normalization by Optimizing Nearest Neighbor Matching",
            "authors": [
                "Salman Ahmad Ansari",
                "Usman Zafar",
                "Asim Karim"
            ],
            "abstract": "Text normalization is an essential task in the processing and analysis of\nsocial media that is dominated with informal writing. It aims to map informal\nwords to their intended standard forms. Previously proposed text normalization\napproaches typically require manual selection of parameters for improved\nperformance. In this paper, we present an automatic optimizationbased nearest\nneighbor matching approach for text normalization. This approach is motivated\nby the observation that text normalization is essentially a matching problem\nand nearest neighbor matching with an adaptive similarity function is the most\ndirect procedure for it. Our similarity function incorporates weighted\ncontributions of contextual, string, and phonetic similarity, and the nearest\nneighbor matching involves a minimum similarity threshold. These four\nparameters are tuned efficiently using grid search. We evaluate the performance\nof our approach on two benchmark datasets. The results demonstrate that\nparameter tuning on small sized labeled datasets produce state-of-the-art text\nnormalization performances. Thus, this approach allows practically easy\nconstruction of evolving domain-specific normalization lexicons",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09827v1",
            "date": "2017-12-28 11:15:30+00:00",
            "title": "A Syntactic Approach to Domain-Specific Automatic Question Generation",
            "authors": [
                "Guy Danon",
                "Mark Last"
            ],
            "abstract": "Factoid questions are questions that require short fact-based answers.\nAutomatic generation (AQG) of factoid questions from a given text can\ncontribute to educational activities, interactive question answering systems,\nsearch engines, and other applications. The goal of our research is to generate\nfactoid source-question-answer triplets based on a specific domain. We propose\na four-component pipeline, which obtains as input a training corpus of\ndomain-specific documents, along with a set of declarative sentences from the\nsame domain, and generates as output a set of factoid questions that refer to\nthe source sentences but are slightly different from them, so that a\nquestion-answering system or a person can be asked a question that requires a\ndeeper understanding and knowledge than a simple word-matching. Contrary to\nexisting domain-specific AQG systems that utilize the template-based approach\nto question generation, we propose to transform each source sentence into a set\nof questions by applying a series of domain-independent rules (a\nsyntactic-based approach). Our pipeline was evaluated in the domain of cyber\nsecurity using a series of experiments on each component of the pipeline\nseparately and on the end-to-end system. The proposed approach generated a\nhigher percentage of acceptable questions than a prior state-of-the-art AQG\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.10066v1",
            "date": "2017-12-22 11:18:35+00:00",
            "title": "Disentangled Representations for Manipulation of Sentiment in Text",
            "authors": [
                "Maria Larsson",
                "Amanda Nilsson",
                "Mikael K\u00e5geb\u00e4ck"
            ],
            "abstract": "The ability to change arbitrary aspects of a text while leaving the core\nmessage intact could have a strong impact in fields like marketing and politics\nby enabling e.g. automatic optimization of message impact and personalized\nlanguage adapted to the receiver's profile. In this paper we take a first step\ntowards such a system by presenting an algorithm that can manipulate the\nsentiment of a text while preserving its semantics using disentangled\nrepresentations. Validation is performed by examining trajectories in embedding\nspace and analyzing transformed sentences for semantic preservation while\nexpression of desired sentiment shift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.10224v2",
            "date": "2017-12-29 13:34:46+00:00",
            "title": "Scalable Multi-Domain Dialogue State Tracking",
            "authors": [
                "Abhinav Rastogi",
                "Dilek Hakkani-Tur",
                "Larry Heck"
            ],
            "abstract": "Dialogue state tracking (DST) is a key component of task-oriented dialogue\nsystems. DST estimates the user's goal at each user turn given the interaction\nuntil then. State of the art approaches for state tracking rely on deep\nlearning methods, and represent dialogue state as a distribution over all\npossible slot values for each slot present in the ontology. Such a\nrepresentation is not scalable when the set of possible values are unbounded\n(e.g., date, time or location) or dynamic (e.g., movies or usernames).\nFurthermore, training of such models requires labeled data, where each user\nturn is annotated with the dialogue state, which makes building models for new\ndomains challenging. In this paper, we present a scalable multi-domain deep\nlearning based approach for DST. We introduce a novel framework for state\ntracking which is independent of the slot value set, and represent the dialogue\nstate as a distribution over a set of values of interest (candidate set)\nderived from the dialogue history or knowledge. Restricting these candidate\nsets to be bounded in size addresses the problem of slot-scalability.\nFurthermore, by leveraging the slot-independent architecture and transfer\nlearning, we show that our proposed approach facilitates quick adaptation to\nnew domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00700v1",
            "date": "2017-02-02 14:44:17+00:00",
            "title": "Multilingual and Cross-lingual Timeline Extraction",
            "authors": [
                "Egoitz Laparra",
                "Rodrigo Agerri",
                "Itziar Aldabe",
                "German Rigau"
            ],
            "abstract": "In this paper we present an approach to extract ordered timelines of events,\ntheir participants, locations and times from a set of multilingual and\ncross-lingual data sources. Based on the assumption that event-related\ninformation can be recovered from different documents written in different\nlanguages, we extend the Cross-document Event Ordering task presented at\nSemEval 2015 by specifying two new tasks for, respectively, Multilingual and\nCross-lingual Timeline Extraction. We then develop three deterministic\nalgorithms for timeline extraction based on two main ideas. First, we address\nimplicit temporal relations at document level since explicit time-anchors are\ntoo scarce to build a wide coverage timeline extraction system. Second, we\nleverage several multilingual resources to obtain a single, inter-operable,\nsemantic representation of events across documents and across languages. The\nresult is a highly competitive system that strongly outperforms the current\nstate-of-the-art. Nonetheless, further analysis of the results reveals that\nlinking the event mentions with their target entities and time-anchors remains\na difficult challenge. The systems, resources and scorers are freely available\nto facilitate its use and guarantee the reproducibility of results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00956v2",
            "date": "2017-02-03 10:15:29+00:00",
            "title": "KU-ISPL Speaker Recognition Systems under Language mismatch condition for NIST 2016 Speaker Recognition Evaluation",
            "authors": [
                "Suwon Shon",
                "Hanseok Ko"
            ],
            "abstract": "Korea University Intelligent Signal Processing Lab. (KU-ISPL) developed\nspeaker recognition system for SRE16 fixed training condition. Data for\nevaluation trials are collected from outside North America, spoken in Tagalog\nand Cantonese while training data only is spoken English. Thus, main issue for\nSRE16 is compensating the discrepancy between different languages. As\ndevelopment dataset which is spoken in Cebuano and Mandarin, we could prepare\nthe evaluation trials through preliminary experiments to compensate the\nlanguage mismatched condition. Our team developed 4 different approaches to\nextract i-vectors and applied state-of-the-art techniques as backend. To\ncompensate language mismatch, we investigated and endeavored unique method such\nas unsupervised language clustering, inter language variability compensation\nand gender/language dependent score normalization.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1702.01172v1",
            "date": "2017-02-03 21:36:46+00:00",
            "title": "Insights into Entity Name Evolution on Wikipedia",
            "authors": [
                "Helge Holzmann",
                "Thomas Risse"
            ],
            "abstract": "Working with Web archives raises a number of issues caused by their temporal\ncharacteristics. Depending on the age of the content, additional knowledge\nmight be needed to find and understand older texts. Especially facts about\nentities are subject to change. Most severe in terms of information retrieval\nare name changes. In order to find entities that have changed their name over\ntime, search engines need to be aware of this evolution. We tackle this problem\nby analyzing Wikipedia in terms of entity evolutions mentioned in articles\nregardless the structural elements. We gathered statistics and automatically\nextracted minimum excerpts covering name changes by incorporating lists\ndedicated to that subject. In future work, these excerpts are going to be used\nto discover patterns and detect changes in other sources. In this work we\ninvestigate whether or not Wikipedia is a suitable source for extracting the\nrequired knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01176v1",
            "date": "2017-02-03 21:44:26+00:00",
            "title": "Named Entity Evolution Analysis on Wikipedia",
            "authors": [
                "Helge Holzmann",
                "Thomas Risse"
            ],
            "abstract": "Accessing Web archives raises a number of issues caused by their temporal\ncharacteristics. Additional knowledge is needed to find and understand older\ntexts. Especially entities mentioned in texts are subject to change. Most\nsevere in terms of information retrieval are name changes. In order to find\nentities that have changed their name over time, search engines need to be\naware of this evolution. We tackle this problem by analyzing Wikipedia in terms\nof entity evolutions mentioned in articles. We present statistical data on\nexcerpts covering name changes, which will be used to discover similar text\npassages and extract evolution knowledge in future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01179v1",
            "date": "2017-02-03 21:48:18+00:00",
            "title": "Extraction of Evolution Descriptions from the Web",
            "authors": [
                "Helge Holzmann",
                "Thomas Risse"
            ],
            "abstract": "The evolution of named entities affects exploration and retrieval tasks in\ndigital libraries. An information retrieval system that is aware of name\nchanges can actively support users in finding former occurrences of evolved\nentities. However, current structured knowledge bases, such as DBpedia or\nFreebase, do not provide enough information about evolutions, even though the\ndata is available on their resources, like Wikipedia. Our \\emph{Evolution Base}\nprototype will demonstrate how excerpts describing name evolutions can be\nidentified on these websites with a promising precision. The descriptions are\nclassified by means of models that we trained based on a recent analysis of\nnamed entity evolutions on Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01187v1",
            "date": "2017-02-03 22:07:57+00:00",
            "title": "Named Entity Evolution Recognition on the Blogosphere",
            "authors": [
                "Helge Holzmann",
                "Nina Tahmasebi",
                "Thomas Risse"
            ],
            "abstract": "Advancements in technology and culture lead to changes in our language. These\nchanges create a gap between the language known by users and the language\nstored in digital archives. It affects user's possibility to firstly find\ncontent and secondly interpret that content. In previous work we introduced our\napproach for Named Entity Evolution Recognition~(NEER) in newspaper\ncollections. Lately, increasing efforts in Web preservation lead to increased\navailability of Web archives covering longer time spans. However, language on\nthe Web is more dynamic than in traditional media and many of the basic\nassumptions from the newspaper domain do not hold for Web data. In this paper\nwe discuss the limitations of existing methodology for NEER. We approach these\nby adapting an existing NEER method to work on noisy data like the Web and the\nBlogosphere in particular. We develop novel filters that reduce the noise and\nmake use of Semantic Web resources to obtain more information about terms. Our\nevaluation shows the potentials of the proposed approach.",
            "journal ref": "International Journal on Digital Libraries 2015, Volume 15, Issue\n  2, pp 209-235",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01417v2",
            "date": "2017-02-05 15:43:07+00:00",
            "title": "All-but-the-Top: Simple and Effective Postprocessing for Word Representations",
            "authors": [
                "Jiaqi Mu",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Real-valued word representations have transformed NLP applications; popular\nexamples are word2vec and GloVe, recognized for their ability to capture\nlinguistic regularities. In this paper, we demonstrate a {\\em very simple}, and\nyet counter-intuitive, postprocessing technique -- eliminate the common mean\nvector and a few top dominating directions from the word vectors -- that\nrenders off-the-shelf representations {\\em even stronger}. The postprocessing\nis empirically validated on a variety of lexical-level intrinsic tasks (word\nsimilarity, concept categorization, word analogy) and sentence-level tasks\n(semantic textural similarity and { text classification}) on multiple datasets\nand with a variety of representation methods and hyperparameter choices in\nmultiple languages; in each case, the processed representations are\nconsistently better than the original ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01829v2",
            "date": "2017-02-07 00:26:56+00:00",
            "title": "Neural Discourse Structure for Text Categorization",
            "authors": [
                "Yangfeng Ji",
                "Noah Smith"
            ],
            "abstract": "We show that discourse structure, as defined by Rhetorical Structure Theory\nand provided by an existing discourse parser, benefits text categorization. Our\napproach uses a recursive neural network and a newly proposed attention\nmechanism to compute a representation of the text that focuses on salient\ncontent, from the perspective of both RST and the task. Experiments consider\nvariants of the approach and illustrate its strengths and weaknesses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01925v1",
            "date": "2017-02-07 08:49:58+00:00",
            "title": "Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study",
            "authors": [
                "Ibrahim Abu El-Khair"
            ],
            "abstract": "The effectiveness of three stop words lists for Arabic Information\nRetrieval---General Stoplist, Corpus-Based Stoplist, Combined Stoplist ---were\ninvestigated in this study. Three popular weighting schemes were examined: the\ninverse document frequency weight, probabilistic weighting, and statistical\nlanguage modelling. The Idea is to combine the statistical approaches with\nlinguistic approaches to reach an optimal performance, and compare their effect\non retrieval. The LDC (Linguistic Data Consortium) Arabic Newswire data set was\nused with the Lemur Toolkit. The Best Match weighting scheme used in the Okapi\nretrieval system had the best overall performance of the three weighting\nalgorithms used in the study, stoplists improved retrieval effectiveness\nespecially when used with the BM25 weight. The overall performance of a general\nstoplist was better than the other two lists.",
            "journal ref": "Abu El-Khair, Ibrahim. (2006). Effects of stop words elimination\n  for Arabic information retrieval: a comparative study. International Journal\n  of Computing & Information Sciences 4 (3), 119-133",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02052v1",
            "date": "2017-02-07 14:59:45+00:00",
            "title": "Knowledge Adaptation: Teaching to Adapt",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "Domain adaptation is crucial in many real-world applications where the\ndistribution of the training data differs from the distribution of the test\ndata. Previous Deep Learning-based approaches to domain adaptation need to be\ntrained jointly on source and target domain data and are therefore unappealing\nin scenarios where models need to be adapted to a large number of domains or\nwhere a domain is evolving, e.g. spam detection where attackers continuously\nchange their tactics.\n  To fill this gap, we propose Knowledge Adaptation, an extension of Knowledge\nDistillation (Bucilua et al., 2006; Hinton et al., 2015) to the domain\nadaptation scenario. We show how a student model achieves state-of-the-art\nresults on unsupervised domain adaptation from multiple sources on a standard\nsentiment analysis benchmark by taking into account the domain-specific\nexpertise of multiple teachers and the similarities between their domains.\n  When learning from a single teacher, using domain similarity to gauge\ntrustworthiness is inadequate. To this end, we propose a simple metric that\ncorrelates well with the teacher's accuracy in the target domain. We\ndemonstrate that incorporating high-confidence examples selected by this metric\nenables the student model to achieve state-of-the-art performance in the\nsingle-source scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02206v2",
            "date": "2017-02-07 21:23:01+00:00",
            "title": "Semi-Supervised QA with Generative Domain-Adaptive Nets",
            "authors": [
                "Zhilin Yang",
                "Junjie Hu",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "We study the problem of semi-supervised question answering----utilizing\nunlabeled text to boost the performance of question answering models. We\npropose a novel training framework, the Generative Domain-Adaptive Nets. In\nthis framework, we train a generative model to generate questions based on the\nunlabeled text, and combine model-generated questions with human-generated\nquestions for training question answering models. We develop novel domain\nadaptation algorithms, based on reinforcement learning, to alleviate the\ndiscrepancy between the model-generated data distribution and the\nhuman-generated data distribution. Experiments show that our proposed framework\nobtains substantial improvement from unlabeled text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02426v1",
            "date": "2017-02-08 13:49:59+00:00",
            "title": "Data Selection Strategies for Multi-Domain Sentiment Analysis",
            "authors": [
                "Sebastian Ruder",
                "Parsa Ghaffari",
                "John G. Breslin"
            ],
            "abstract": "Domain adaptation is important in sentiment analysis as sentiment-indicating\nwords vary between domains. Recently, multi-domain adaptation has become more\npervasive, but existing approaches train on all available source domains\nincluding dissimilar ones. However, the selection of appropriate training data\nis as important as the choice of algorithm. We undertake -- to our knowledge\nfor the first time -- an extensive study of domain similarity metrics in the\ncontext of sentiment analysis and propose novel representations, metrics, and a\nnew scope for data selection. We evaluate the proposed methods on two\nlarge-scale multi-domain adaptation settings on tweets and reviews and\ndemonstrate that they consistently outperform strong random and balanced\nbaselines, while our proposed selection strategy outperforms instance-level\nselection and yields the best score on a large reviews corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02429v1",
            "date": "2017-02-08 13:56:16+00:00",
            "title": "Trainable Greedy Decoding for Neural Machine Translation",
            "authors": [
                "Jiatao Gu",
                "Kyunghyun Cho",
                "Victor O. K. Li"
            ],
            "abstract": "Recent research in neural machine translation has largely focused on two\naspects; neural network architectures and end-to-end learning algorithms. The\nproblem of decoding, however, has received relatively little attention from the\nresearch community. In this paper, we solely focus on the problem of decoding\ngiven a trained neural machine translation model. Instead of trying to build a\nnew decoding algorithm for any specific decoding objective, we propose the idea\nof trainable decoding algorithm in which we train a decoding algorithm to find\na translation that maximizes an arbitrary decoding objective. More\nspecifically, we design an actor that observes and manipulates the hidden state\nof the neural machine translation decoder and propose to train it using a\nvariant of deterministic policy gradient. We extensively evaluate the proposed\nalgorithm using four language pairs and two decoding objectives and show that\nwe can indeed train a trainable greedy decoder that generates a better\ntranslation (in terms of a target decoding objective) with minimal\ncomputational overhead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02640v1",
            "date": "2017-02-08 22:24:14+00:00",
            "title": "Character-level Deep Conflation for Business Data Analytics",
            "authors": [
                "Zhe Gan",
                "P. D. Singh",
                "Ameet Joshi",
                "Xiaodong He",
                "Jianshu Chen",
                "Jianfeng Gao",
                "Li Deng"
            ],
            "abstract": "Connecting different text attributes associated with the same entity\n(conflation) is important in business data analytics since it could help merge\ntwo different tables in a database to provide a more comprehensive profile of\nan entity. However, the conflation task is challenging because two text strings\nthat describe the same entity could be quite different from each other for\nreasons such as misspelling. It is therefore critical to develop a conflation\nmodel that is able to truly understand the semantic meaning of the strings and\nmatch them at the semantic level. To this end, we develop a character-level\ndeep conflation model that encodes the input text strings from character level\ninto finite dimension feature vectors, which are then used to compute the\ncosine similarity between the text strings. The model is trained in an\nend-to-end manner using back propagation and stochastic gradient descent to\nmaximize the likelihood of the correct association. Specifically, we propose\ntwo variants of the deep conflation model, based on long-short-term memory\n(LSTM) recurrent neural network (RNN) and convolutional neural network (CNN),\nrespectively. Both models perform well on a real-world business analytics\ndataset and significantly outperform the baseline bag-of-character (BoC) model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02736v1",
            "date": "2017-02-09 08:00:09+00:00",
            "title": "Challenges in Providing Automatic Affective Feedback in Instant Messaging Applications",
            "authors": [
                "Chieh-Yang Huang",
                "Ting-Hao",
                "Huang",
                "Lun-Wei Ku"
            ],
            "abstract": "Instant messaging is one of the major channels of computer mediated\ncommunication. However, humans are known to be very limited in understanding\nothers' emotions via text-based communication. Aiming on introducing emotion\nsensing technologies to instant messaging, we developed EmotionPush, a system\nthat automatically detects the emotions of the messages end-users received on\nFacebook Messenger and provides colored cues on their smartphones accordingly.\nWe conducted a deployment study with 20 participants during a time span of two\nweeks. In this paper, we revealed five challenges, along with examples, that we\nobserved in our study based on both user's feedback and chat logs, including\n(i)the continuum of emotions, (ii)multi-user conversations, (iii)different\ndynamics between different users, (iv)misclassification of emotions and\n(v)unconventional content. We believe this discussion will benefit the future\nexploration of affective computing for instant messaging, and also shed light\non research of conversational emotion sensing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02737v1",
            "date": "2017-02-09 08:03:16+00:00",
            "title": "Mining User/Movie Preferred Features Based on Reviews for Video Recommendation System",
            "authors": [
                "Xuan-Son Vu",
                "Seong-Bae Park"
            ],
            "abstract": "In this work, we present an approach for mining user preferences and\nrecommendation based on reviews. There have been various studies worked on\nrecommendation problem. However, most of the studies beyond one aspect user\ngenerated- content such as user ratings, user feedback and so on to state user\npreferences. There is a prob- lem in one aspect mining is lacking for stating\nuser preferences. As a demonstration, in collaborative filter recommendation,\nwe try to figure out the preference trend of crowded users, then use that trend\nto predict current user preference. Therefore, there is a gap between real user\npreferences and the trend of the crowded people. Additionally, user preferences\ncan be addressed from mining user reviews since user often comment about\nvarious aspects of products. To solve this problem, we mainly focus on mining\nproduct aspects and user aspects inside user reviews to directly state user\npreferences. We also take into account Social Network Analysis for cold-start\nitem problem. With cold-start user problem, collaborative filter algorithm is\nemployed in our work. The framework is general enough to be applied to\ndifferent recommendation domains. Theoretically, our method would achieve a\nsignificant enhancement.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1702.03274v2",
            "date": "2017-02-10 18:24:13+00:00",
            "title": "Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning",
            "authors": [
                "Jason D. Williams",
                "Kavosh Asadi",
                "Geoffrey Zweig"
            ],
            "abstract": "End-to-end learning of recurrent neural networks (RNNs) is an attractive\nsolution for dialog systems; however, current techniques are data-intensive and\nrequire thousands of dialogs to learn simple behaviors. We introduce Hybrid\nCode Networks (HCNs), which combine an RNN with domain-specific knowledge\nencoded as software and system action templates. Compared to existing\nend-to-end approaches, HCNs considerably reduce the amount of training data\nrequired, while retaining the key benefit of inferring a latent representation\nof dialog state. In addition, HCNs can be optimized with supervised learning,\nreinforcement learning, or a mixture of both. HCNs attain state-of-the-art\nperformance on the bAbI dialog dataset, and outperform two commercially\ndeployed customer-facing dialog systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1702.03402v1",
            "date": "2017-02-11 09:50:40+00:00",
            "title": "Parallel Long Short-Term Memory for Multi-stream Classification",
            "authors": [
                "Mohamed Bouaziz",
                "Mohamed Morchid",
                "Richard Dufour",
                "Georges Linar\u00e8s",
                "Renato De Mori"
            ],
            "abstract": "Recently, machine learning methods have provided a broad spectrum of original\nand efficient algorithms based on Deep Neural Networks (DNN) to automatically\npredict an outcome with respect to a sequence of inputs. Recurrent hidden cells\nallow these DNN-based models to manage long-term dependencies such as Recurrent\nNeural Networks (RNN) and Long Short-Term Memory (LSTM). Nevertheless, these\nRNNs process a single input stream in one (LSTM) or two (Bidirectional LSTM)\ndirections. But most of the information available nowadays is from multistreams\nor multimedia documents, and require RNNs to process these information\nsynchronously during the training. This paper presents an original LSTM-based\narchitecture, named Parallel LSTM (PLSTM), that carries out multiple parallel\nsynchronized input sequences in order to predict a common output. The proposed\nPLSTM method could be used for parallel sequence classification purposes. The\nPLSTM approach is evaluated on an automatic telecast genre sequences\nclassification task and compared with different state-of-the-art architectures.\nResults show that the proposed PLSTM method outperforms the baseline n-gram\nmodels as well as the state-of-the-art LSTM approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1702.03814v3",
            "date": "2017-02-13 15:26:27+00:00",
            "title": "Bilateral Multi-Perspective Matching for Natural Language Sentences",
            "authors": [
                "Zhiguo Wang",
                "Wael Hamza",
                "Radu Florian"
            ],
            "abstract": "Natural language sentence matching is a fundamental technology for a variety\nof tasks. Previous approaches either match sentences from a single direction or\nonly apply single granular (word-by-word or sentence-by-sentence) matching. In\nthis work, we propose a bilateral multi-perspective matching (BiMPM) model\nunder the \"matching-aggregation\" framework. Given two sentences $P$ and $Q$,\nour model first encodes them with a BiLSTM encoder. Next, we match the two\nencoded sentences in two directions $P \\rightarrow Q$ and $P \\leftarrow Q$. In\neach matching direction, each time step of one sentence is matched against all\ntime-steps of the other sentence from multiple perspectives. Then, another\nBiLSTM layer is utilized to aggregate the matching results into a fix-length\nmatching vector. Finally, based on the matching vector, the decision is made\nthrough a fully connected layer. We evaluate our model on three tasks:\nparaphrase identification, natural language inference and answer sentence\nselection. Experimental results on standard benchmark datasets show that our\nmodel achieves the state-of-the-art performance on all tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1702.05512v3",
            "date": "2017-02-17 20:26:50+00:00",
            "title": "soc2seq: Social Embedding meets Conversation Model",
            "authors": [
                "Parminder Bhatia",
                "Marsal Gavalda",
                "Arash Einolghozati"
            ],
            "abstract": "While liking or upvoting a post on a mobile app is easy to do, replying with\na written note is much more difficult, due to both the cognitive load of coming\nup with a meaningful response as well as the mechanics of entering the text.\nHere we present a novel textual reply generation model that goes beyond the\ncurrent auto-reply and predictive text entry models by taking into account the\ncontent preferences of the user, the idiosyncrasies of their conversational\nstyle, and even the structure of their social graph. Specifically, we have\ndeveloped two types of models for personalized user interactions: a\ncontent-based conversation model, which makes use of location together with\nuser information, and a social-graph-based conversation model, which combines\ncontent-based conversation models with social graphs.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1702.06478v1",
            "date": "2017-02-21 17:14:56+00:00",
            "title": "Syst\u00e8mes du LIA \u00e0 DEFT'13",
            "authors": [
                "Xavier Bost",
                "Ilaria Brunetti",
                "Luis Adri\u00e1n Cabrera-Diego",
                "Jean-Val\u00e8re Cossu",
                "Andr\u00e9a Linhares",
                "Mohamed Morchid",
                "Juan-Manuel Torres-Moreno",
                "Marc El-B\u00e8ze",
                "Richard Dufour"
            ],
            "abstract": "The 2013 D\\'efi de Fouille de Textes (DEFT) campaign is interested in two\ntypes of language analysis tasks, the document classification and the\ninformation extraction in the specialized domain of cuisine recipes. We present\nthe systems that the LIA has used in DEFT 2013. Our systems show interesting\nresults, even though the complexity of the proposed tasks.",
            "journal ref": "Proceedings of the Ninth DEFT Workshop, DEFT2013, Les\n  Sables-d'Olonne, France, 21st June 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06510v1",
            "date": "2017-02-21 18:24:52+00:00",
            "title": "Algorithmes de classification et d'optimisation: participation du LIA/ADOC \u00e1 DEFT'14",
            "authors": [
                "Luis Adri\u00e1n Cabrera-Diego",
                "St\u00e9phane Huet",
                "Bassam Jabaian",
                "Alejandro Molina",
                "Juan-Manuel Torres-Moreno",
                "Marc El-B\u00e8ze",
                "Barth\u00e9l\u00e9my Durette"
            ],
            "abstract": "This year, the DEFT campaign (D\\'efi Fouilles de Textes) incorporates a task\nwhich aims at identifying the session in which articles of previous TALN\nconferences were presented. We describe the three statistical systems developed\nat LIA/ADOC for this task. A fusion of these systems enables us to obtain\ninteresting results (micro-precision score of 0.76 measured on the test corpus)",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1702.06663v1",
            "date": "2017-02-22 03:14:36+00:00",
            "title": "Guided Deep List: Automating the Generation of Epidemiological Line Lists from Open Sources",
            "authors": [
                "Saurav Ghosh",
                "Prithwish Chakraborty",
                "Bryan L. Lewis",
                "Maimuna S. Majumder",
                "Emily Cohn",
                "John S. Brownstein",
                "Madhav V. Marathe",
                "Naren Ramakrishnan"
            ],
            "abstract": "Real-time monitoring and responses to emerging public health threats rely on\nthe availability of timely surveillance data. During the early stages of an\nepidemic, the ready availability of line lists with detailed tabular\ninformation about laboratory-confirmed cases can assist epidemiologists in\nmaking reliable inferences and forecasts. Such inferences are crucial to\nunderstand the epidemiology of a specific disease early enough to stop or\ncontrol the outbreak. However, construction of such line lists requires\nconsiderable human supervision and therefore, difficult to generate in\nreal-time. In this paper, we motivate Guided Deep List, the first tool for\nbuilding automated line lists (in near real-time) from open source reports of\nemerging disease outbreaks. Specifically, we focus on deriving epidemiological\ncharacteristics of an emerging disease and the affected population from reports\nof illness. Guided Deep List uses distributed vector representations (ala\nword2vec) to discover a set of indicators for each line list feature. This\ndiscovery of indicators is followed by the use of dependency parsing based\ntechniques for final extraction in tabular form. We evaluate the performance of\nGuided Deep List against a human annotated line list provided by HealthMap\ncorresponding to MERS outbreaks in Saudi Arabia. We demonstrate that Guided\nDeep List extracts line list features with increased accuracy compared to a\nbaseline method. We further show how these automatically extracted line list\nfeatures can be used for making epidemiological inferences, such as inferring\ndemographics and symptoms-to-hospitalization period of affected individuals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07071v1",
            "date": "2017-02-23 02:31:03+00:00",
            "title": "Pronunciation recognition of English phonemes /\\textipa{@}/, /\u00e6/, /\\textipa{A}:/ and /\\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients",
            "authors": [
                "Keith Y. Patarroyo",
                "Vladimir Vargas-Calder\u00f3n"
            ],
            "abstract": "The Vocal Joystick Vowel Corpus, by Washington University, was used to study\nmonophthongs pronounced by native English speakers. The objective of this study\nwas to quantitatively measure the extent at which speech recognition methods\ncan distinguish between similar sounding vowels. In particular, the phonemes\n/\\textipa{@}/, /{\\ae}/, /\\textipa{A}:/ and /\\textipa{2}/ were analysed. 748\nsound files from the corpus were used and subjected to Linear Predictive Coding\n(LPC) to compute their formants, and to Mel Frequency Cepstral Coefficients\n(MFCC) algorithm, to compute the cepstral coefficients. A Decision Tree\nClassifier was used to build a predictive model that learnt the patterns of the\ntwo first formants measured in the data set, as well as the patterns of the 13\ncepstral coefficients. An accuracy of 70\\% was achieved using formants for the\nmentioned phonemes. For the MFCC analysis an accuracy of 52 \\% was achieved and\nan accuracy of 71\\% when /\\textipa{@}/ was ignored. The results obtained show\nthat the studied algorithms are far from mimicking the ability of\ndistinguishing subtle differences in sounds like human hearing does.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07092v1",
            "date": "2017-02-23 04:27:49+00:00",
            "title": "A Neural Attention Model for Categorizing Patient Safety Events",
            "authors": [
                "Arman Cohan",
                "Allan Fong",
                "Nazli Goharian",
                "Raj Ratwani"
            ],
            "abstract": "Medical errors are leading causes of death in the US and as such, prevention\nof these errors is paramount to promoting health care. Patient Safety Event\nreports are narratives describing potential adverse events to the patients and\nare important in identifying and preventing medical errors. We present a neural\nnetwork architecture for identifying the type of safety events which is the\nfirst step in understanding these narratives. Our proposed model is based on a\nsoft neural attention model to improve the effectiveness of encoding long\nsequences. Empirical results on two large-scale real-world datasets of patient\nsafety reports demonstrate the effectiveness of our method with significant\nimprovements over existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.08388v3",
            "date": "2017-02-27 17:19:03+00:00",
            "title": "Political Homophily in Independence Movements: Analysing and Classifying Social Media Users by National Identity",
            "authors": [
                "Arkaitz Zubiaga",
                "Bo Wang",
                "Maria Liakata",
                "Rob Procter"
            ],
            "abstract": "Social media and data mining are increasingly being used to analyse political\nand societal issues. Here we undertake the classification of social media users\nas supporting or opposing ongoing independence movements in their territories.\nIndependence movements occur in territories whose citizens have conflicting\nnational identities; users with opposing national identities will then support\nor oppose the sense of being part of an independent nation that differs from\nthe officially recognised country. We describe a methodology that relies on\nusers' self-reported location to build large-scale datasets for three\nterritories -- Catalonia, the Basque Country and Scotland. An analysis of these\ndatasets shows that homophily plays an important role in determining who people\nconnect with, as users predominantly choose to follow and interact with others\nfrom the same national identity. We show that a classifier relying on users'\nfollow networks can achieve accurate, language-independent classification\nperformances ranging from 85% to 97% for the three territories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00203v3",
            "date": "2017-03-01 10:02:04+00:00",
            "title": "Frequency patterns of semantic change: Corpus-based evidence of a near-critical dynamics in language change",
            "authors": [
                "Quentin Feltgen",
                "Benjamin Fagard",
                "Jean-Pierre Nadal"
            ],
            "abstract": "It is generally believed that, when a linguistic item acquires a new meaning,\nits overall frequency of use in the language rises with time with an S-shaped\ngrowth curve. Yet, this claim has only been supported by a limited number of\ncase studies. In this paper, we provide the first corpus-based quantitative\nconfirmation of the genericity of the S-curve in language change. Moreover, we\nuncover another generic pattern, a latency phase of variable duration preceding\nthe S-growth, during which the frequency of use of the semantically expanding\nword remains low and more or less constant. We also propose a usage-based model\nof language change supported by cognitive considerations, which predicts that\nboth phases, the latency and the fast S-growth, take place. The driving\nmechanism is a stochastic dynamics, a random walk in the space of frequency of\nuse. The underlying deterministic dynamics highlights the role of a control\nparameter, the strength of the cognitive impetus governing the onset of change,\nwhich tunes the system at the vicinity of a saddle-node bifurcation. In the\nneighborhood of the critical point, the latency phase corresponds to the\ndiffusion time over the critical region, and the S-growth to the fast\nconvergence that follows. The duration of the two phases is computed as\nspecific first passage times of the random walk process, leading to\ndistributions that fit well the ones extracted from our dataset. We argue that\nour results are not specific to the studied corpus, but apply to semantic\nchange in general.",
            "journal ref": "R. Soc. open sci. 2017 4 170830; DOI: 10.1098/rsos.170830.\n  Published 8 November 2017",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1703.00317v1",
            "date": "2017-03-01 14:40:22+00:00",
            "title": "Tracing Linguistic Relations in Winning and Losing Sides of Explicit Opposing Groups",
            "authors": [
                "Ceyda Sanli",
                "Anupam Mondal",
                "Erik Cambria"
            ],
            "abstract": "Linguistic relations in oral conversations present how opinions are\nconstructed and developed in a restricted time. The relations bond ideas,\narguments, thoughts, and feelings, re-shape them during a speech, and finally\nbuild knowledge out of all information provided in the conversation. Speakers\nshare a common interest to discuss. It is expected that each speaker's reply\nincludes duplicated forms of words from previous speakers. However, linguistic\nadaptation is observed and evolves in a more complex path than just\ntransferring slightly modified versions of common concepts. A conversation\naiming a benefit at the end shows an emergent cooperation inducing the\nadaptation. Not only cooperation, but also competition drives the adaptation or\nan opposite scenario and one can capture the dynamic process by tracking how\nthe concepts are linguistically linked. To uncover salient complex dynamic\nevents in verbal communications, we attempt to discover self-organized\nlinguistic relations hidden in a conversation with explicitly stated winners\nand losers. We examine open access data of the United States Supreme Court. Our\nunderstanding is crucial in big data research to guide how transition states in\nopinion mining and decision-making should be modeled and how this required\nknowledge to guide the model should be pinpointed, by filtering large amount of\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00565v3",
            "date": "2017-03-02 00:48:15+00:00",
            "title": "Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ",
            "authors": [
                "Jason S. Kessler"
            ],
            "abstract": "Scattertext is an open source tool for visualizing linguistic variation\nbetween document categories in a language-independent way. The tool presents a\nscatterplot, where each axis corresponds to the rank-frequency a term occurs in\na category of documents. Through a tie-breaking strategy, the tool is able to\ndisplay thousands of visible term-representing points and find space to legibly\nlabel hundreds of them. Scattertext also lends itself to a query-based\nvisualization of how the use of terms with similar embeddings differs between\ndocument categories, as well as a visualization for comparing the importance\nscores of bag-of-words features to univariate metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00607v2",
            "date": "2017-03-02 03:59:18+00:00",
            "title": "Dynamic Word Embeddings for Evolving Semantic Discovery",
            "authors": [
                "Zijun Yao",
                "Yifan Sun",
                "Weicong Ding",
                "Nikhil Rao",
                "Hui Xiong"
            ],
            "abstract": "Word evolution refers to the changing meanings and associations of words\nthroughout time, as a byproduct of human language evolution. By studying word\nevolution, we can infer social trends and language constructs over different\nperiods of human history. However, traditional techniques such as word\nrepresentation learning do not adequately capture the evolving language\nstructure and vocabulary. In this paper, we develop a dynamic statistical model\nto learn time-aware word vector representation. We propose a model that\nsimultaneously learns time-aware embeddings and solves the resulting \"alignment\nproblem\". This model is trained on a crawled NYTimes dataset. Additionally, we\ndevelop multiple intuitive evaluation strategies of temporal word embeddings.\nOur qualitative and quantitative tests indicate that our method not only\nreliably captures this evolution over time, but also consistently outperforms\nstate-of-the-art temporal embedding approaches on both semantic accuracy and\nalignment quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00786v1",
            "date": "2017-03-02 13:52:47+00:00",
            "title": "A Generic Online Parallel Learning Framework for Large Margin Models",
            "authors": [
                "Shuming Ma",
                "Xu Sun"
            ],
            "abstract": "To speed up the training process, many existing systems use parallel\ntechnology for online learning algorithms. However, most research mainly focus\non stochastic gradient descent (SGD) instead of other algorithms. We propose a\ngeneric online parallel learning framework for large margin models, and also\nanalyze our framework on popular large margin algorithms, including MIRA and\nStructured Perceptron. Our framework is lock-free and easy to implement on\nexisting systems. Experiments show that systems with our framework can gain\nnear linear speed up by increasing running threads, and with no loss in\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01008v4",
            "date": "2017-03-03 01:29:11+00:00",
            "title": "End-to-End Task-Completion Neural Dialogue Systems",
            "authors": [
                "Xiujun Li",
                "Yun-Nung Chen",
                "Lihong Li",
                "Jianfeng Gao",
                "Asli Celikyilmaz"
            ],
            "abstract": "One of the major drawbacks of modularized task-completion dialogue systems is\nthat each module is trained individually, which presents several challenges.\nFor example, downstream modules are affected by earlier modules, and the\nperformance of the entire system is not robust to the accumulated errors. This\npaper presents a novel end-to-end learning framework for task-completion\ndialogue systems to tackle such issues. Our neural dialogue system can directly\ninteract with a structured database to assist users in accessing information\nand accomplishing certain tasks. The reinforcement learning based dialogue\nmanager offers robust capabilities to handle noises caused by other components\nof the dialogue system. Our experiments in a movie-ticket booking domain show\nthat our end-to-end system not only outperforms modularized dialogue system\nbaselines for both objective and subjective evaluation, but also is robust to\nnoises as demonstrated by several systematic experiments with different error\ngranularity and rates specific to the language understanding module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01671v2",
            "date": "2017-03-05 21:57:25+00:00",
            "title": "Controlling for Unobserved Confounds in Classification Using Correlational Constraints",
            "authors": [
                "Virgile Landeiro",
                "Aron Culotta"
            ],
            "abstract": "As statistical classifiers become integrated into real-world applications, it\nis important to consider not only their accuracy but also their robustness to\nchanges in the data distribution. In this paper, we consider the case where\nthere is an unobserved confounding variable $z$ that influences both the\nfeatures $\\mathbf{x}$ and the class variable $y$. When the influence of $z$\nchanges from training to testing data, we find that the classifier accuracy can\ndegrade rapidly. In our approach, we assume that we can predict the value of\n$z$ at training time with some error. The prediction for $z$ is then fed to\nPearl's back-door adjustment to build our model. Because of the attenuation\nbias caused by measurement error in $z$, standard approaches to controlling for\n$z$ are ineffective. In response, we propose a method to properly control for\nthe influence of $z$ by first estimating its relationship with the class\nvariable $y$, then updating predictions for $z$ to match that estimated\nrelationship. By adjusting the influence of $z$, we show that we can build a\nmodel that exceeds competing baselines on accuracy as well as on robustness\nover a range of confounding relationships.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1703.02573v1",
            "date": "2017-03-07 19:56:26+00:00",
            "title": "Data Noising as Smoothing in Neural Network Language Models",
            "authors": [
                "Ziang Xie",
                "Sida I. Wang",
                "Jiwei Li",
                "Daniel L\u00e9vy",
                "Aiming Nie",
                "Dan Jurafsky",
                "Andrew Y. Ng"
            ],
            "abstract": "Data noising is an effective technique for regularizing neural network\nmodels. While noising is widely adopted in application domains such as vision\nand speech, commonly used noising primitives have not been developed for\ndiscrete sequence-level settings such as language modeling. In this paper, we\nderive a connection between input noising in neural network language models and\nsmoothing in $n$-gram models. Using this connection, we draw upon ideas from\nsmoothing to develop effective noising schemes. We demonstrate performance\ngains when applying the proposed schemes to language modeling and machine\ntranslation. Finally, we provide empirical analysis validating the relationship\nbetween noising and smoothing.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1703.02860v1",
            "date": "2017-03-08 14:57:10+00:00",
            "title": "Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji",
            "authors": [
                "Tianran Hu",
                "Han Guo",
                "Hao Sun",
                "Thuy-vy Thi Nguyen",
                "Jiebo Luo"
            ],
            "abstract": "Emojis, as a new way of conveying nonverbal cues, are widely adopted in\ncomputer-mediated communications. In this paper, first from a message sender\nperspective, we focus on people's motives in using four types of emojis --\npositive, neutral, negative, and non-facial. We compare the willingness levels\nof using these emoji types for seven typical intentions that people usually\napply nonverbal cues for in communication. The results of extensive statistical\nhypothesis tests not only report the popularities of the intentions, but also\nuncover the subtle differences between emoji types in terms of intended uses.\nSecond, from a perspective of message recipients, we further study the\nsentiment effects of emojis, as well as their duplications, on verbal messages.\nDifferent from previous studies in emoji sentiment, we study the sentiments of\nemojis and their contexts as a whole. The experiment results indicate that the\npowers of conveying sentiment are different between four emoji types, and the\nsentiment effects of emojis vary in the contexts of different valences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03097v1",
            "date": "2017-03-09 01:28:00+00:00",
            "title": "Information Extraction in Illicit Domains",
            "authors": [
                "Mayank Kejriwal",
                "Pedro Szekely"
            ],
            "abstract": "Extracting useful entities and attribute values from illicit domains such as\nhuman trafficking is a challenging problem with the potential for widespread\nsocial impact. Such domains employ atypical language models, have `long tails'\nand suffer from the problem of concept drift. In this paper, we propose a\nlightweight, feature-agnostic Information Extraction (IE) paradigm specifically\ndesigned for such domains. Our approach uses raw, unlabeled text from an\ninitial corpus, and a few (12-120) seed annotations per domain-specific\nattribute, to learn robust IE models for unobserved pages and websites.\nEmpirically, we demonstrate that our approach can outperform feature-centric\nConditional Random Field baselines by over 18\\% F-Measure on five annotated\nsets of real-world human trafficking datasets in both low-supervision and\nhigh-supervision settings. We also show that our approach is demonstrably\nrobust to concept drift, and can be efficiently bootstrapped even in a serial\ncomputing environment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03386v3",
            "date": "2017-03-09 18:37:50+00:00",
            "title": "Loyalty in Online Communities",
            "authors": [
                "William L. Hamilton",
                "Justine Zhang",
                "Cristian Danescu-Niculescu-Mizil",
                "Dan Jurafsky",
                "Jure Leskovec"
            ],
            "abstract": "Loyalty is an essential component of multi-community engagement. When users\nhave the choice to engage with a variety of different communities, they often\nbecome loyal to just one, focusing on that community at the expense of others.\nHowever, it is unclear how loyalty is manifested in user behavior, or whether\nloyalty is encouraged by certain community characteristics.\n  In this paper we operationalize loyalty as a user-community relation: users\nloyal to a community consistently prefer it over all others; loyal communities\nretain their loyal users over time. By exploring this relation using a large\ndataset of discussion communities from Reddit, we reveal that loyalty is\nmanifested in remarkably consistent behaviors across a wide spectrum of\ncommunities. Loyal users employ language that signals collective identity and\nengage with more esoteric, less popular content, indicating they may play a\ncurational role in surfacing new material. Loyal communities have denser\nuser-user interaction networks and lower rates of triadic closure, suggesting\nthat community-level loyalty is associated with more cohesive interactions and\nless fragmentation into subgroups. We exploit these general patterns to predict\nfuture rates of loyalty. Our results show that a user's propensity to become\nloyal is apparent from their first interactions with a community, suggesting\nthat some users are intrinsically loyal from the very beginning.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1703.03429v1",
            "date": "2017-03-09 19:16:14+00:00",
            "title": "What can you do with a rock? Affordance extraction via word embeddings",
            "authors": [
                "Nancy Fulda",
                "Daniel Ricks",
                "Ben Murdoch",
                "David Wingate"
            ],
            "abstract": "Autonomous agents must often detect affordances: the set of behaviors enabled\nby a situation. Affordance detection is particularly helpful in domains with\nlarge action spaces, allowing the agent to prune its search space by avoiding\nfutile behaviors. This paper presents a method for affordance extraction via\nword embeddings trained on a Wikipedia corpus. The resulting word vectors are\ntreated as a common knowledge database which can be queried using linear\nalgebra. We apply this method to a reinforcement learning agent in a text-only\nenvironment and show that affordance-based action selection improves\nperformance most of the time. Our method increases the computational complexity\nof each learning step but significantly reduces the total number of steps\nneeded. In addition, the agent's action selections begin to resemble those a\nhuman would choose.",
            "journal ref": "Proceedings of the Twenty-Sixth International Joint Conference on\n  Artificial Intelligence (IJCAI), Pages 1039-1045, 2017",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1703.03442v2",
            "date": "2017-03-09 19:50:00+00:00",
            "title": "The cognitive roots of regularization in language",
            "authors": [
                "Vanessa Ferdinand",
                "Simon Kirby",
                "Kenny Smith"
            ],
            "abstract": "Regularization occurs when the output a learner produces is less variable\nthan the linguistic data they observed. In an artificial language learning\nexperiment, we show that there exist at least two independent sources of\nregularization bias in cognition: a domain-general source based on cognitive\nload and a domain-specific source triggered by linguistic stimuli. Both of\nthese factors modulate how frequency information is encoded and produced, but\nonly the production-side modulations result in regularization (i.e. cause\nlearners to eliminate variation from the observed input). We formalize the\ndefinition of regularization as the reduction of entropy and find that entropy\nmeasures are better at identifying regularization behavior than frequency-based\nanalyses. Using our experimental data and a model of cultural transmission, we\ngenerate predictions for the amount of regularity that would develop in each\nexperimental condition if the artificial language were transmitted over several\ngenerations of learners. Here we find that the effect of cognitive constraints\ncan become more complex when put into the context of cultural evolution:\nalthough learning biases certainly carry information about the course of\nlanguage evolution, we should not expect a one-to-one correspondence between\nthe micro-level processes that regularize linguistic datasets and the\nmacro-level evolution of linguistic regularity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03923v1",
            "date": "2017-03-11 07:35:28+00:00",
            "title": "A German Corpus for Text Similarity Detection Tasks",
            "authors": [
                "Juan-Manuel Torres-Moreno",
                "Gerardo Sierra",
                "Peter Peinl"
            ],
            "abstract": "Text similarity detection aims at measuring the degree of similarity between\na pair of texts. Corpora available for text similarity detection are designed\nto evaluate the algorithms to assess the paraphrase level among documents. In\nthis paper we present a textual German corpus for similarity detection. The\npurpose of this corpus is to automatically assess the similarity between a pair\nof texts and to evaluate different similarity measures, both for whole\ndocuments or for individual sentences. Therefore we have calculated several\nsimple measures on our corpus based on a library of similarity functions.",
            "journal ref": "Preprint of International Journal of Computational Linguistics and\n  Applications, vol. 5, no. 2, 2014, pp. 9-24",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.04001v1",
            "date": "2017-03-11 17:14:55+00:00",
            "title": "Language Use Matters: Analysis of the Linguistic Structure of Question Texts Can Characterize Answerability in Quora",
            "authors": [
                "Suman Kalyan Maity",
                "Aman Kharb",
                "Animesh Mukherjee"
            ],
            "abstract": "Quora is one of the most popular community Q&A sites of recent times.\nHowever, many question posts on this Q&A site often do not get answered. In\nthis paper, we quantify various linguistic activities that discriminates an\nanswered question from an unanswered one. Our central finding is that the way\nusers use language while writing the question text can be a very effective\nmeans to characterize answerability. This characterization helps us to predict\nearly if a question remaining unanswered for a specific time period t will\neventually be answered or not and achieve an accuracy of 76.26% (t = 1 month)\nand 68.33% (t = 3 months). Notably, features representing the language use\npatterns of the users are most discriminative and alone account for an accuracy\nof 74.18%. We also compare our method with some of the similar works (Dror et\nal., Yang et al.) achieving a maximum improvement of ~39% in terms of accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04247v1",
            "date": "2017-03-13 04:55:19+00:00",
            "title": "DeepFM: A Factorization-Machine based Neural Network for CTR Prediction",
            "authors": [
                "Huifeng Guo",
                "Ruiming Tang",
                "Yunming Ye",
                "Zhenguo Li",
                "Xiuqiang He"
            ],
            "abstract": "Learning sophisticated feature interactions behind user behaviors is critical\nin maximizing CTR for recommender systems. Despite great progress, existing\nmethods seem to have a strong bias towards low- or high-order interactions, or\nrequire expertise feature engineering. In this paper, we show that it is\npossible to derive an end-to-end learning model that emphasizes both low- and\nhigh-order feature interactions. The proposed model, DeepFM, combines the power\nof factorization machines for recommendation and deep learning for feature\nlearning in a new neural network architecture. Compared to the latest Wide \\&\nDeep model from Google, DeepFM has a shared input to its \"wide\" and \"deep\"\nparts, with no need of feature engineering besides raw features. Comprehensive\nexperiments are conducted to demonstrate the effectiveness and efficiency of\nDeepFM over the existing models for CTR prediction, on both benchmark data and\ncommercial data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.04336v1",
            "date": "2017-03-13 11:19:56+00:00",
            "title": "A Visual Representation of Wittgenstein's Tractatus Logico-Philosophicus",
            "authors": [
                "Anca Bucur",
                "Sergiu Nisioi"
            ],
            "abstract": "In this paper we present a data visualization method together with its\npotential usefulness in digital humanities and philosophy of language. We\ncompile a multilingual parallel corpus from different versions of\nWittgenstein's Tractatus Logico-Philosophicus, including the original in German\nand translations into English, Spanish, French, and Russian. Using this corpus,\nwe compute a similarity measure between propositions and render a visual\nnetwork of relations for different languages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.04417v1",
            "date": "2017-03-13 14:34:23+00:00",
            "title": "El Lenguaje Natural como Lenguaje Formal",
            "authors": [
                "Franco M. Luque"
            ],
            "abstract": "Formal languages theory is useful for the study of natural language. In\nparticular, it is of interest to study the adequacy of the grammatical\nformalisms to express syntactic phenomena present in natural language. First,\nit helps to draw hypothesis about the nature and complexity of the\nspeaker-hearer linguistic competence, a fundamental question in linguistics and\nother cognitive sciences. Moreover, from an engineering point of view, it\nallows the knowledge of practical limitations of applications based on those\nformalisms. In this article I introduce the adequacy problem of grammatical\nformalisms for natural language, also introducing some formal language theory\nconcepts required for this discussion. Then, I review the formalisms that have\nbeen proposed in history, and the arguments that have been given to support or\nreject their adequacy.\n  -----\n  La teor\\'ia de lenguajes formales es \\'util para el estudio de los lenguajes\nnaturales. En particular, resulta de inter\\'es estudiar la adecuaci\\'on de los\nformalismos gramaticales para expresar los fen\\'omenos sint\\'acticos presentes\nen el lenguaje natural. Primero, ayuda a trazar hip\\'otesis acerca de la\nnaturaleza y complejidad de las competencias ling\\\"u\\'isticas de los\nhablantes-oyentes del lenguaje, un interrogante fundamental de la\nling\\\"u\\'istica y otras ciencias cognitivas. Adem\\'as, desde el punto de vista\nde la ingenier\\'ia, permite conocer limitaciones pr\\'acticas de las\naplicaciones basadas en dichos formalismos. En este art\\'iculo hago una\nintroducci\\'on al problema de la adecuaci\\'on de los formalismos gramaticales\npara el lenguaje natural, introduciendo tambi\\'en algunos conceptos de la\nteor\\'ia de lenguajes formales necesarios para esta discusi\\'on. Luego, hago un\nrepaso de los formalismos que han sido propuestos a lo largo de la historia, y\nde los argumentos que se han dado para sostener o refutar su adecuaci\\'on.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04489v1",
            "date": "2017-03-13 17:13:51+00:00",
            "title": "Reinforcement Learning for Transition-Based Mention Detection",
            "authors": [
                "Georgiana Dinu",
                "Wael Hamza",
                "Radu Florian"
            ],
            "abstract": "This paper describes an application of reinforcement learning to the mention\ndetection task. We define a novel action-based formulation for the mention\ndetection task, in which a model can flexibly revise past labeling decisions by\ngrouping together tokens and assigning partial mention labels. We devise a\nmethod to create mention-level episodes and we train a model by rewarding\ncorrectly labeled complete mentions, irrespective of the inner structure\ncreated. The model yields results which are on par with a competitive\nsupervised counterpart while being more flexible in terms of achieving targeted\nbehavior through reward modeling and generating internal mention structure,\nespecially on longer mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04512v1",
            "date": "2017-02-24 07:00:51+00:00",
            "title": "Normalisation de la langue et de lecriture arabe : enjeux culturels regionaux et mondiaux",
            "authors": [
                "Henri Hudrisier",
                "Ben Henda Mokhtar"
            ],
            "abstract": "Arabic language and writing are now facing a resurgence of international\nnormative solutions that challenge most of their local or network based\noperating principles. Even if the multilingual digital coding solutions,\nespecially those proposed by Unicode, have solved many difficulties of Arabic\nwriting, the linguistic aspect is still in search of more adapted solutions.\nTerminology is one of the sectors in which the Arabic language requires a deep\nmodernization of its classical productivity models. The normative approach, in\nparticular that of the ISO TC37, is proposed as one of the solutions that would\nallow it to combine with international standards to better integrate the\nknowledge society under construction.\n  La langue et lecriture arabe sont aujourdhui confrontees a une recrudescence\nde solutions normatives internationales qui remettent en cause la plupart de\nleurs principes de fonctionnement en site ou sur les reseaux. Meme si les\nsolutions du codage numerique multilingue, notamment celles proposees par\nUnicode, ont resolu beaucoup de difficultes de lecriture arabe, le volet\nlinguistique est encore en quete de solutions plus adaptees. La terminologie\nest lun des secteurs dans lequel la langue arabe necessite une modernisation\nprofonde de ses modeles classiques de production. La voie normative, notamment\ncelle du TC37 de ISO, est proposee comme une des solutions qui lui permettrait\nde se mettre en synergie avec les referentiels internationaux pour mieux\nintegrer la societe du savoir en voie de construction.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1703.04677v2",
            "date": "2017-03-14 19:14:32+00:00",
            "title": "A computational investigation of sources of variability in sentence comprehension difficulty in aphasia",
            "authors": [
                "Paul M\u00e4tzig",
                "Shravan Vasishth",
                "Felix Engelmann",
                "David Caplan"
            ],
            "abstract": "We present a computational evaluation of three hypotheses about sources of\ndeficit in sentence comprehension in aphasia: slowed processing, intermittent\ndeficiency, and resource reduction. The ACT-R based Lewis and Vasishth (2005)\nmodel is used to implement these three proposals. Slowed processing is\nimplemented as slowed default production-rule firing time; intermittent\ndeficiency as increased random noise in activation of chunks in memory; and\nresource reduction as reduced goal activation. As data, we considered subject\nvs. object rela- tives whose matrix clause contained either an NP or a\nreflexive, presented in a self-paced listening modality to 56 individuals with\naphasia (IWA) and 46 matched controls. The participants heard the sentences and\ncarried out a picture verification task to decide on an interpretation of the\nsentence. These response accuracies are used to identify the best parameters\n(for each participant) that correspond to the three hypotheses mentioned above.\nWe show that controls have more tightly clustered (less variable) parameter\nvalues than IWA; specifically, compared to controls, among IWA there are more\nindividuals with low goal activations, high noise, and slow default action\ntimes. This suggests that (i) individual patients show differential amounts of\ndeficit along the three dimensions of slowed processing, intermittent\ndeficient, and resource reduction, (ii) overall, there is evidence for all\nthree sources of deficit playing a role, and (iii) IWA have a more variable\nrange of parameter values than controls. In sum, this study contributes a proof\nof concept of a quantitative implementation of, and evidence for, these three\naccounts of comprehension deficits in aphasia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04783v1",
            "date": "2017-03-14 22:28:51+00:00",
            "title": "Multichannel End-to-end Speech Recognition",
            "authors": [
                "Tsubasa Ochiai",
                "Shinji Watanabe",
                "Takaaki Hori",
                "John R. Hershey"
            ],
            "abstract": "The field of speech recognition is in the midst of a paradigm shift:\nend-to-end neural networks are challenging the dominance of hidden Markov\nmodels as a core technology. Using an attention mechanism in a recurrent\nencoder-decoder architecture solves the dynamic time alignment problem,\nallowing joint end-to-end training of the acoustic and language modeling\ncomponents. In this paper we extend the end-to-end framework to encompass\nmicrophone array signal processing for noise suppression and speech enhancement\nwithin the acoustic encoding network. This allows the beamforming components to\nbe optimized jointly within the recognition architecture to improve the\nend-to-end speech recognition objective. Experiments on the noisy speech\nbenchmarks (CHiME-4 and AMI) show that our multichannel end-to-end system\noutperformed the attention-based baseline with input from a conventional\nadaptive beamformer.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1703.04826v4",
            "date": "2017-03-14 23:25:34+00:00",
            "title": "Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling",
            "authors": [
                "Diego Marcheggiani",
                "Ivan Titov"
            ],
            "abstract": "Semantic role labeling (SRL) is the task of identifying the\npredicate-argument structure of a sentence. It is typically regarded as an\nimportant step in the standard NLP pipeline. As the semantic representations\nare closely related to syntactic ones, we exploit syntactic information in our\nmodel. We propose a version of graph convolutional networks (GCNs), a recent\nclass of neural networks operating on graphs, suited to model syntactic\ndependency graphs. GCNs over syntactic dependency trees are used as sentence\nencoders, producing latent feature representations of words in a sentence. We\nobserve that GCN layers are complementary to LSTM ones: when we stack both GCN\nand LSTM layers, we obtain a substantial improvement over an already\nstate-of-the-art LSTM SRL model, resulting in the best reported scores on the\nstandard benchmark (CoNLL-2009) both for Chinese and English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04854v1",
            "date": "2017-03-15 00:47:28+00:00",
            "title": "Distributed-Representation Based Hybrid Recommender System with Short Item Descriptions",
            "authors": [
                "Junhua He",
                "Hankz Hankui Zhuo",
                "Jarvan Law"
            ],
            "abstract": "Collaborative filtering (CF) aims to build a model from users' past behaviors\nand/or similar decisions made by other users, and use the model to recommend\nitems for users. Despite of the success of previous collaborative filtering\napproaches, they are all based on the assumption that there are sufficient\nrating scores available for building high-quality recommendation models. In\nreal world applications, however, it is often difficult to collect sufficient\nrating scores, especially when new items are introduced into the system, which\nmakes the recommendation task challenging. We find that there are often \"short\"\ntexts describing features of items, based on which we can approximate the\nsimilarity of items and make recommendation together with rating scores. In\nthis paper we \"borrow\" the idea of vector representation of words to capture\nthe information of short texts and embed it into a matrix factorization\nframework. We empirically show that our approach is effective by comparing it\nwith state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.04908v2",
            "date": "2017-03-15 03:30:13+00:00",
            "title": "Emergence of Grounded Compositional Language in Multi-Agent Populations",
            "authors": [
                "Igor Mordatch",
                "Pieter Abbeel"
            ],
            "abstract": "By capturing statistical patterns in large corpora, machine learning has\nenabled significant advances in natural language processing, including in\nmachine translation, question answering, and sentiment analysis. However, for\nagents to intelligently interact with humans, simply capturing the statistical\npatterns is insufficient. In this paper we investigate if, and how, grounded\ncompositional language can emerge as a means to achieve goals in multi-agent\npopulations. Towards this end, we propose a multi-agent learning environment\nand learning methods that bring about emergence of a basic compositional\nlanguage. This language is represented as streams of abstract discrete symbols\nuttered by agents over time, but nonetheless has a coherent structure that\npossesses a defined vocabulary and syntax. We also observe emergence of\nnon-verbal communication such as pointing and guiding when language\ncommunication is unavailable.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1703.04914v2",
            "date": "2017-03-15 04:00:27+00:00",
            "title": "Ensemble of Neural Classifiers for Scoring Knowledge Base Triples",
            "authors": [
                "Ikuya Yamada",
                "Motoki Sato",
                "Hiroyuki Shindo"
            ],
            "abstract": "This paper describes our approach for the triple scoring task at the WSDM Cup\n2017. The task required participants to assign a relevance score for each pair\nof entities and their types in a knowledge base in order to enhance the ranking\nresults in entity retrieval tasks. We propose an approach wherein the outputs\nof multiple neural network classifiers are combined using a supervised machine\nlearning model. The experimental results showed that our proposed method\nachieved the best performance in one out of three measures (i.e., Kendall's\ntau), and performed competitively in the other two measures (i.e., accuracy and\naverage score difference).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05123v2",
            "date": "2017-03-15 12:37:22+00:00",
            "title": "Character-based Neural Embeddings for Tweet Clustering",
            "authors": [
                "Svitlana Vakulenko",
                "Lyndon Nixon",
                "Mihai Lupu"
            ],
            "abstract": "In this paper we show how the performance of tweet clustering can be improved\nby leveraging character-based neural networks. The proposed approach overcomes\nthe limitations related to the vocabulary explosion in the word-based models\nand allows for the seamless processing of the multilingual content. Our\nevaluation results and code are available on-line at\nhttps://github.com/vendi12/tweet2vec_clustering",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.05260v1",
            "date": "2017-03-15 17:01:20+00:00",
            "title": "InScript: Narrative texts annotated with script information",
            "authors": [
                "Ashutosh Modi",
                "Tatjana Anikina",
                "Simon Ostermann",
                "Manfred Pinkal"
            ],
            "abstract": "This paper presents the InScript corpus (Narrative Texts Instantiating Script\nstructure). InScript is a corpus of 1,000 stories centered around 10 different\nscenarios. Verbs and noun phrases are annotated with event and participant\ntypes, respectively. Additionally, the text is annotated with coreference\ninformation. The corpus shows rich lexical variation and will serve as a unique\nresource for the study of the role of script knowledge in natural language\nprocessing.",
            "journal ref": "LREC 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05320v1",
            "date": "2017-03-16 01:06:07+00:00",
            "title": "Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network",
            "authors": [
                "Phong-Khac Do",
                "Huy-Tien Nguyen",
                "Chien-Xuan Tran",
                "Minh-Tien Nguyen",
                "Minh-Le Nguyen"
            ],
            "abstract": "This paper presents a study of employing Ranking SVM and Convolutional Neural\nNetwork for two missions: legal information retrieval and question answering in\nthe Competition on Legal Information Extraction/Entailment. For the first task,\nour proposed model used a triple of features (LSI, Manhattan, Jaccard), and is\nbased on paragraph level instead of article level as in previous studies. In\nfact, each single-paragraph article corresponds to a particular paragraph in a\nhuge multiple-paragraph article. For the legal question answering task,\nadditional statistical features from information retrieval task integrated into\nConvolutional Neural Network contribute to higher accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05706v2",
            "date": "2017-03-16 16:32:06+00:00",
            "title": "Improving Document Clustering by Eliminating Unnatural Language",
            "authors": [
                "Myungha Jang",
                "Jinho D. Choi",
                "James Allan"
            ],
            "abstract": "Technical documents contain a fair amount of unnatural language, such as\ntables, formulas, pseudo-codes, etc. Unnatural language can be an important\nfactor of confusing existing NLP tools. This paper presents an effective method\nof distinguishing unnatural language from natural language, and evaluates the\nimpact of unnatural language detection on NLP tasks such as document\nclustering. We view this problem as an information extraction task and build a\nmulticlass classification model identifying unnatural language components into\nfour categories. First, we create a new annotated corpus by collecting slides\nand papers in various formats, PPT, PDF, and HTML, where unnatural language\ncomponents are annotated into four categories. We then explore features\navailable from plain text to build a statistical model that can handle any\nformat as long as it is converted into plain text. Our experiments show that\nremoving unnatural language components gives an absolute improvement in\ndocument clustering up to 15%. Our corpus and tool are publicly available.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.05851v2",
            "date": "2017-03-17 00:02:42+00:00",
            "title": "Temporal Information Extraction for Question Answering Using Syntactic Dependencies in an LSTM-based Architecture",
            "authors": [
                "Yuanliang Meng",
                "Anna Rumshisky",
                "Alexey Romanov"
            ],
            "abstract": "In this paper, we propose to use a set of simple, uniform in architecture\nLSTM-based models to recover different kinds of temporal relations from text.\nUsing the shortest dependency path between entities as input, the same\narchitecture is used to extract intra-sentence, cross-sentence, and document\ncreation time relations. A \"double-checking\" technique reverses entity pairs in\nclassification, boosting the recall of positive cases and reducing\nmisclassifications between opposite classes. An efficient pruning algorithm\nresolves conflicts globally. Evaluated on QA-TempEval (SemEval2015 Task 5), our\nproposed technique outperforms state-of-the-art methods by a large margin.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.06345v1",
            "date": "2017-03-18 20:21:44+00:00",
            "title": "Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks",
            "authors": [
                "Zhilin Yang",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "Recent papers have shown that neural networks obtain state-of-the-art\nperformance on several different sequence tagging tasks. One appealing property\nof such systems is their generality, as excellent performance can be achieved\nwith a unified architecture and without task-specific feature engineering.\nHowever, it is unclear if such systems can be used for tasks without large\namounts of training data. In this paper we explore the problem of transfer\nlearning for neural sequence taggers, where a source task with plentiful\nannotations (e.g., POS tagging on Penn Treebank) is used to improve performance\non a target task with fewer available annotations (e.g., POS tagging for\nmicroblogs). We examine the effects of transfer learning for deep hierarchical\nrecurrent networks across domains, applications, and languages, and show that\nsignificant improvement can often be obtained. These improvements lead to\nimprovements over the current state-of-the-art on several well-studied tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.06492v2",
            "date": "2017-03-19 19:14:55+00:00",
            "title": "VQABQ: Visual Question Answering by Basic Questions",
            "authors": [
                "Jia-Hong Huang",
                "Modar Alfadly",
                "Bernard Ghanem"
            ],
            "abstract": "Taking an image and question as the input of our method, it can output the\ntext-based answer of the query question about the given image, so called Visual\nQuestion Answering (VQA). There are two main modules in our algorithm. Given a\nnatural language question about an image, the first module takes the question\nas input and then outputs the basic questions of the main given question. The\nsecond module takes the main question, image and these basic questions as input\nand then outputs the text-based answer of the main question. We formulate the\nbasic questions generation problem as a LASSO optimization problem, and also\npropose a criterion about how to exploit these basic questions to help answer\nmain question. Our method is evaluated on the challenging VQA dataset and\nyields state-of-the-art accuracy, 60.34% in open-ended task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.06630v1",
            "date": "2017-03-20 08:19:43+00:00",
            "title": "Automatic Text Summarization Approaches to Speed up Topic Model Learning Process",
            "authors": [
                "Mohamed Morchid",
                "Juan-Manuel Torres-Moreno",
                "Richard Dufour",
                "Javier Ram\u00edrez-Rodr\u00edguez",
                "Georges Linar\u00e8s"
            ],
            "abstract": "The number of documents available into Internet moves each day up. For this\nreason, processing this amount of information effectively and expressibly\nbecomes a major concern for companies and scientists. Methods that represent a\ntextual document by a topic representation are widely used in Information\nRetrieval (IR) to process big data such as Wikipedia articles. One of the main\ndifficulty in using topic model on huge data collection is related to the\nmaterial resources (CPU time and memory) required for model estimate. To deal\nwith this issue, we propose to build topic spaces from summarized documents. In\nthis paper, we present a study of topic space representation in the context of\nbig data. The topic space representation behavior is analyzed on different\nlanguages. Experiments show that topic spaces estimated from text summaries are\nas relevant as those estimated from the complete documents. The real advantage\nof such an approach is the processing time gain: we showed that the processing\ntime can be drastically reduced using summarized documents (more than 60\\% in\ngeneral). This study finally points out the differences between thematic\nrepresentations of documents depending on the targeted languages such as\nEnglish or latin languages.",
            "journal ref": "International Journal of Computational Linguistics and\n  Applications, 7(2):87-109, 2016",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.06676v3",
            "date": "2017-03-20 11:11:38+00:00",
            "title": "I2T2I: Learning Text to Image Synthesis with Textual Data Augmentation",
            "authors": [
                "Hao Dong",
                "Jingqing Zhang",
                "Douglas McIlwraith",
                "Yike Guo"
            ],
            "abstract": "Translating information between text and image is a fundamental problem in\nartificial intelligence that connects natural language processing and computer\nvision. In the past few years, performance in image caption generation has seen\nsignificant improvement through the adoption of recurrent neural networks\n(RNN). Meanwhile, text-to-image generation begun to generate plausible images\nusing datasets of specific categories like birds and flowers. We've even seen\nimage generation from multi-category datasets such as the Microsoft Common\nObjects in Context (MSCOCO) through the use of generative adversarial networks\n(GANs). Synthesizing objects with a complex shape, however, is still\nchallenging. For example, animals and humans have many degrees of freedom,\nwhich means that they can take on many complex shapes. We propose a new\ntraining method called Image-Text-Image (I2T2I) which integrates text-to-image\nand image-to-text (image captioning) synthesis to improve the performance of\ntext-to-image synthesis. We demonstrate that %the capability of our method to\nunderstand the sentence descriptions, so as to I2T2I can generate better\nmulti-categories images using MSCOCO than the state-of-the-art. We also\ndemonstrate that I2T2I can achieve transfer learning by using a pre-trained\nimage captioning module to generate human images on the MPII Human Pose",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.07805v1",
            "date": "2017-03-22 18:20:07+00:00",
            "title": "Supervised Typing of Big Graphs using Semantic Embeddings",
            "authors": [
                "Mayank Kejriwal",
                "Pedro Szekely"
            ],
            "abstract": "We propose a supervised algorithm for generating type embeddings in the same\nsemantic vector space as a given set of entity embeddings. The algorithm is\nagnostic to the derivation of the underlying entity embeddings. It does not\nrequire any manual feature engineering, generalizes well to hundreds of types\nand achieves near-linear scaling on Big Graphs containing many millions of\ntriples and instances by virtue of an incremental execution. We demonstrate the\nutility of the embeddings on a type recommendation task, outperforming a\nnon-parametric feature-agnostic baseline while achieving 15x speedup and\nnear-constant memory usage on a full partition of DBpedia. Using\nstate-of-the-art visualization, we illustrate the agreement of our\nextensionally derived DBpedia type embeddings with the manually curated domain\nontology. Finally, we use the embeddings to probabilistically cluster about 4\nmillion DBpedia instances into 415 types in the DBpedia ontology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08002v1",
            "date": "2017-03-23 11:02:47+00:00",
            "title": "A network of deep neural networks for distant speech recognition",
            "authors": [
                "Mirco Ravanelli",
                "Philemon Brakel",
                "Maurizio Omologo",
                "Yoshua Bengio"
            ],
            "abstract": "Despite the remarkable progress recently made in distant speech recognition,\nstate-of-the-art technology still suffers from a lack of robustness, especially\nwhen adverse acoustic conditions characterized by non-stationary noises and\nreverberation are met. A prominent limitation of current systems lies in the\nlack of matching and communication between the various technologies involved in\nthe distant speech recognition process. The speech enhancement and speech\nrecognition modules are, for instance, often trained independently. Moreover,\nthe speech enhancement normally helps the speech recognizer, but the output of\nthe latter is not commonly used, in turn, to improve the speech enhancement. To\naddress both concerns, we propose a novel architecture based on a network of\ndeep neural networks, where all the components are jointly trained and better\ncooperate with each other thanks to a full communication scheme between them.\nExperiments, conducted using different datasets, tasks and acoustic conditions,\nrevealed that the proposed framework can overtake other competitive solutions,\nincluding recent joint training approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08052v1",
            "date": "2017-03-23 13:00:14+00:00",
            "title": "Dynamic Bernoulli Embeddings for Language Evolution",
            "authors": [
                "Maja Rudolph",
                "David Blei"
            ],
            "abstract": "Word embeddings are a powerful approach for unsupervised analysis of\nlanguage. Recently, Rudolph et al. (2016) developed exponential family\nembeddings, which cast word embeddings in a probabilistic framework. Here, we\ndevelop dynamic embeddings, building on exponential family embeddings to\ncapture how the meanings of words change over time. We use dynamic embeddings\nto analyze three large collections of historical texts: the U.S. Senate\nspeeches from 1858 to 2009, the history of computer science ACM abstracts from\n1951 to 2014, and machine learning papers on the Arxiv from 2007 to 2015. We\nfind dynamic embeddings provide better fits than classical embeddings and\ncapture interesting patterns about how language changes.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1703.08120v1",
            "date": "2017-03-23 15:57:23+00:00",
            "title": "Recurrent and Contextual Models for Visual Question Answering",
            "authors": [
                "Abhijit Sharang",
                "Eric Lau"
            ],
            "abstract": "We propose a series of recurrent and contextual neural network models for\nmultiple choice visual question answering on the Visual7W dataset. Motivated by\ndivergent trends in model complexities in the literature, we explore the\nbalance between model expressiveness and simplicity by studying incrementally\nmore complex architectures. We start with LSTM-encoding of input questions and\nanswers; build on this with context generation by LSTM-encodings of neural\nimage and question representations and attention over images; and evaluate the\ndiversity and predictive power of our models and the ensemble thereof. All\nmodels are evaluated against a simple baseline inspired by the current\nstate-of-the-art, consisting of involving simple concatenation of bag-of-words\nand CNN representations for the text and images, respectively. Generally, we\nobserve marked variation in image-reasoning performance between our models not\nobvious from their overall performance, as well as evidence of dataset bias.\nOur standalone models achieve accuracies up to $64.6\\%$, while the ensemble of\nall models achieves the best accuracy of $66.67\\%$, within $0.5\\%$ of the\ncurrent state-of-the-art for Visual7W.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08135v2",
            "date": "2017-03-23 16:45:22+00:00",
            "title": "An embedded segmental K-means model for unsupervised segmentation and clustering of speech",
            "authors": [
                "Herman Kamper",
                "Karen Livescu",
                "Sharon Goldwater"
            ],
            "abstract": "Unsupervised segmentation and clustering of unlabelled speech are core\nproblems in zero-resource speech processing. Most approaches lie at\nmethodological extremes: some use probabilistic Bayesian models with\nconvergence guarantees, while others opt for more efficient heuristic\ntechniques. Despite competitive performance in previous work, the full Bayesian\napproach is difficult to scale to large speech corpora. We introduce an\napproximation to a recent Bayesian model that still has a clear objective\nfunction but improves efficiency by using hard clustering and segmentation\nrather than full Bayesian inference. Like its Bayesian counterpart, this\nembedded segmental K-means model (ES-KMeans) represents arbitrary-length word\nsegments as fixed-dimensional acoustic word embeddings. We first compare\nES-KMeans to previous approaches on common English and Xitsonga data sets (5\nand 2.5 hours of speech): ES-KMeans outperforms a leading heuristic method in\nword segmentation, giving similar scores to the Bayesian model while being 5\ntimes faster with fewer hyperparameters. However, its clusters are less pure\nthan those of the other models. We then show that ES-KMeans scales to larger\ncorpora by applying it to the 5 languages of the Zero Resource Speech Challenge\n2017 (up to 45 hours), where it performs competitively compared to the\nchallenge baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08136v2",
            "date": "2017-03-23 16:46:00+00:00",
            "title": "Visually grounded learning of keyword prediction from untranscribed speech",
            "authors": [
                "Herman Kamper",
                "Shane Settle",
                "Gregory Shakhnarovich",
                "Karen Livescu"
            ],
            "abstract": "During language acquisition, infants have the benefit of visual cues to\nground spoken language. Robots similarly have access to audio and visual\nsensors. Recent work has shown that images and spoken captions can be mapped\ninto a meaningful common space, allowing images to be retrieved using speech\nand vice versa. In this setting of images paired with untranscribed spoken\ncaptions, we consider whether computer vision systems can be used to obtain\ntextual labels for the speech. Concretely, we use an image-to-words multi-label\nvisual classifier to tag images with soft textual labels, and then train a\nneural network to map from the speech to these soft targets. We show that the\nresulting speech system is able to predict which words occur in an\nutterance---acting as a spoken bag-of-words classifier---without seeing any\nparallel speech and text. We find that the model often confuses semantically\nrelated words, e.g. \"man\" and \"person\", making it even more effective as a\nsemantic keyword spotter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08314v2",
            "date": "2017-03-24 08:46:48+00:00",
            "title": "Interacting Conceptual Spaces I : Grammatical Composition of Concepts",
            "authors": [
                "Joe Bolt",
                "Bob Coecke",
                "Fabrizio Genovese",
                "Martha Lewis",
                "Dan Marsden",
                "Robin Piedeleu"
            ],
            "abstract": "The categorical compositional approach to meaning has been successfully\napplied in natural language processing, outperforming other models in\nmainstream empirical language processing tasks. We show how this approach can\nbe generalized to conceptual space models of cognition. In order to do this,\nfirst we introduce the category of convex relations as a new setting for\ncategorical compositional semantics, emphasizing the convex structure important\nto conceptual space applications. We then show how to construct conceptual\nspaces for various types such as nouns, adjectives and verbs. Finally we show\nby means of examples how concepts can be systematically combined to establish\nthe meanings of composite phrases from the meanings of their constituent parts.\nThis provides the mathematical underpinnings of a new compositional approach to\ncognition.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1703.08471v1",
            "date": "2017-03-24 15:40:19+00:00",
            "title": "Batch-normalized joint training for DNN-based distant speech recognition",
            "authors": [
                "Mirco Ravanelli",
                "Philemon Brakel",
                "Maurizio Omologo",
                "Yoshua Bengio"
            ],
            "abstract": "Improving distant speech recognition is a crucial step towards flexible\nhuman-machine interfaces. Current technology, however, still exhibits a lack of\nrobustness, especially when adverse acoustic conditions are met. Despite the\nsignificant progress made in the last years on both speech enhancement and\nspeech recognition, one potential limitation of state-of-the-art technology\nlies in composing modules that are not well matched because they are not\ntrained jointly. To address this concern, a promising approach consists in\nconcatenating a speech enhancement and a speech recognition deep neural network\nand to jointly update their parameters as if they were within a single bigger\nnetwork. Unfortunately, joint training can be difficult because the output\ndistribution of the speech enhancement system may change substantially during\nthe optimization procedure. The speech recognition module would have to deal\nwith an input distribution that is non-stationary and unnormalized. To mitigate\nthis issue, we propose a joint training approach based on a fully\nbatch-normalized architecture. Experiments, conducted using different datasets,\ntasks and acoustic conditions, revealed that the proposed framework\nsignificantly overtakes other competitive solutions, especially in challenging\nenvironments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08513v2",
            "date": "2017-03-24 17:13:08+00:00",
            "title": "Interactive Natural Language Acquisition in a Multi-modal Recurrent Neural Architecture",
            "authors": [
                "Stefan Heinrich",
                "Stefan Wermter"
            ],
            "abstract": "For the complex human brain that enables us to communicate in natural\nlanguage, we gathered good understandings of principles underlying language\nacquisition and processing, knowledge about socio-cultural conditions, and\ninsights about activity patterns in the brain. However, we were not yet able to\nunderstand the behavioural and mechanistic characteristics for natural language\nand how mechanisms in the brain allow to acquire and process language. In\nbridging the insights from behavioural psychology and neuroscience, the goal of\nthis paper is to contribute a computational understanding of appropriate\ncharacteristics that favour language acquisition. Accordingly, we provide\nconcepts and refinements in cognitive modelling regarding principles and\nmechanisms in the brain and propose a neurocognitively plausible model for\nembodied language acquisition from real world interaction of a humanoid robot\nwith its environment. In particular, the architecture consists of a continuous\ntime recurrent neural network, where parts have different leakage\ncharacteristics and thus operate on multiple timescales for every modality and\nthe association of the higher level nodes of all modalities into cell\nassemblies. The model is capable of learning language production grounded in\nboth, temporal dynamic somatosensation and vision, and features hierarchical\nconcept abstraction, concept decomposition, multi-modal integration, and\nself-organisation of latent representations.",
            "journal ref": "Connection Science, vol 30, No 1, pp. 99-133, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08544v2",
            "date": "2017-03-24 14:49:58+00:00",
            "title": "Data-Mining Textual Responses to Uncover Misconception Patterns",
            "authors": [
                "Joshua J. Michalenko",
                "Andrew S. Lan",
                "Richard G. Baraniuk"
            ],
            "abstract": "An important, yet largely unstudied, problem in student data analysis is to\ndetect misconceptions from students' responses to open-response questions.\nMisconception detection enables instructors to deliver more targeted feedback\non the misconceptions exhibited by many students in their class, thus improving\nthe quality of instruction. In this paper, we propose a new natural language\nprocessing-based framework to detect the common misconceptions among students'\ntextual responses to short-answer questions. We propose a probabilistic model\nfor students' textual responses involving misconceptions and experimentally\nvalidate it on a real-world student-response dataset. Experimental results show\nthat our proposed framework excels at classifying whether a response exhibits\none or more misconceptions. More importantly, it can also automatically detect\nthe common misconceptions exhibited across responses from multiple students to\nmultiple questions; this property is especially important at large scale, since\ninstructors will no longer need to manually specify all possible misconceptions\nthat students might exhibit.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1703.09046v1",
            "date": "2017-03-27 13:11:33+00:00",
            "title": "Bootstrapping a Lexicon for Emotional Arousal in Software Engineering",
            "authors": [
                "Mika V. M\u00e4ntyl\u00e4",
                "Nicole Novielli",
                "Filippo Lanubile",
                "Ma\u00eblick Claes",
                "Miikka Kuutila"
            ],
            "abstract": "Emotional arousal increases activation and performance but may also lead to\nburnout in software development. We present the first version of a Software\nEngineering Arousal lexicon (SEA) that is specifically designed to address the\nproblem of emotional arousal in the software developer ecosystem. SEA is built\nusing a bootstrapping approach that combines word embedding model trained on\nissue-tracking data and manual scoring of items in the lexicon. We show that\nour lexicon is able to differentiate between issue priorities, which are a\nsource of emotional activation and then act as a proxy for arousal. The best\nperformance is obtained by combining SEA (428 words) with a previously created\ngeneral purpose lexicon by Warriner et al. (13,915 words) and it achieves\nCohen's d effect sizes up to 0.5.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1703.09398v1",
            "date": "2017-03-28 04:47:11+00:00",
            "title": "This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News",
            "authors": [
                "Benjamin D. Horne",
                "Sibel Adali"
            ],
            "abstract": "The problem of fake news has gained a lot of attention as it is claimed to\nhave had a significant impact on 2016 US Presidential Elections. Fake news is\nnot a new problem and its spread in social networks is well-studied. Often an\nunderlying assumption in fake news discussion is that it is written to look\nlike real news, fooling the reader who does not check for reliability of the\nsources or the arguments in its content. Through a unique study of three data\nsets and features that capture the style and the language of articles, we show\nthat this assumption is not true. Fake news in most cases is more similar to\nsatire than to real news, leading us to conclude that persuasion in fake news\nis achieved through heuristics rather than the strength of arguments. We show\noverall title structure and the use of proper nouns in titles are very\nsignificant in differentiating fake from real. This leads us to conclude that\nfake news is targeted for audiences who are not likely to read beyond titles\nand is aimed at creating mental associations between entities and claims.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1703.09400v1",
            "date": "2017-03-28 05:07:38+00:00",
            "title": "Diving Deep into Clickbaits: Who Use Them to What Extents in Which Topics with What Effects?",
            "authors": [
                "Md Main Uddin Rony",
                "Naeemul Hassan",
                "Mohammad Yousuf"
            ],
            "abstract": "The use of alluring headlines (clickbait) to tempt the readers has become a\ngrowing practice nowadays. For the sake of existence in the highly competitive\nmedia industry, most of the on-line media including the mainstream ones, have\nstarted following this practice. Although the wide-spread practice of clickbait\nmakes the reader's reliability on media vulnerable, a large scale analysis to\nreveal this fact is still absent. In this paper, we analyze 1.67 million\nFacebook posts created by 153 media organizations to understand the extent of\nclickbait practice, its impact and user engagement by using our own developed\nclickbait detection model. The model uses distributed sub-word embeddings\nlearned from a large corpus. The accuracy of the model is 98.3%. Powered with\nthis model, we further study the distribution of topics in clickbait and\nnon-clickbait contents.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1703.09439v1",
            "date": "2017-03-28 07:47:27+00:00",
            "title": "A practical approach to dialogue response generation in closed domains",
            "authors": [
                "Yichao Lu",
                "Phillip Keung",
                "Shaonan Zhang",
                "Jason Sun",
                "Vikas Bhardwaj"
            ],
            "abstract": "We describe a prototype dialogue response generation model for the customer\nservice domain at Amazon. The model, which is trained in a weakly supervised\nfashion, measures the similarity between customer questions and agent answers\nusing a dual encoder network, a Siamese-like neural network architecture.\nAnswer templates are extracted from embeddings derived from past agent answers,\nwithout turn-by-turn annotations. Responses to customer inquiries are generated\nby selecting the best template from the final set of templates. We show that,\nin a closed domain like customer service, the selected templates cover $>$70\\%\nof past customer inquiries. Furthermore, the relevance of the model-selected\ntemplates is significantly higher than templates selected by a standard tf-idf\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09527v1",
            "date": "2017-03-28 12:08:46+00:00",
            "title": "Is This a Joke? Detecting Humor in Spanish Tweets",
            "authors": [
                "Santiago Castro",
                "Mat\u00edas Cubero",
                "Diego Garat",
                "Guillermo Moncecchi"
            ],
            "abstract": "While humor has been historically studied from a psychological, cognitive and\nlinguistic standpoint, its study from a computational perspective is an area\nyet to be explored in Computational Linguistics. There exist some previous\nworks, but a characterization of humor that allows its automatic recognition\nand generation is far from being specified. In this work we build a\ncrowdsourced corpus of labeled tweets, annotated according to its humor value,\nletting the annotators subjectively decide which are humorous. A humor\nclassifier for Spanish tweets is assembled based on supervised learning,\nreaching a precision of 84% and a recall of 69%.",
            "journal ref": "Presented in Iberamia 2016. The final publication is available at\n  link.springer.com:\n  https://link.springer.com/chapter/10.1007%2F978-3-319-47955-2_12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09570v2",
            "date": "2017-03-27 02:18:36+00:00",
            "title": "A Tidy Data Model for Natural Language Processing using cleanNLP",
            "authors": [
                "Taylor Arnold"
            ],
            "abstract": "The package cleanNLP provides a set of fast tools for converting a textual\ncorpus into a set of normalized tables. The underlying natural language\nprocessing pipeline utilizes Stanford's CoreNLP library, exposing a number of\nannotation tasks for text written in English, French, German, and Spanish.\nAnnotators include tokenization, part of speech tagging, named entity\nrecognition, entity linking, sentiment analysis, dependency parsing,\ncoreference resolution, and information extraction.",
            "journal ref": "The R Journal, 9.2, 248-267 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09831v3",
            "date": "2017-03-28 22:29:53+00:00",
            "title": "A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment",
            "authors": [
                "Haonan Yu",
                "Haichao Zhang",
                "Wei Xu"
            ],
            "abstract": "We tackle a task where an agent learns to navigate in a 2D maze-like\nenvironment called XWORLD. In each session, the agent perceives a sequence of\nraw-pixel frames, a natural language command issued by a teacher, and a set of\nrewards. The agent learns the teacher's language from scratch in a grounded and\ncompositional manner, such that after training it is able to correctly execute\nzero-shot commands: 1) the combination of words in the command never appeared\nbefore, and/or 2) the command contains new object concepts that are learned\nfrom another task but never learned from navigation. Our deep framework for the\nagent is trained end to end: it learns simultaneously the visual\nrepresentations of the environment, the syntax and semantics of the language,\nand the action module that outputs actions. The zero-shot learning capability\nof our framework results from its compositionality and modularity with\nparameter tying. We visualize the intermediate outputs of the framework,\ndemonstrating that the agent truly understands how to solve the problem. We\nbelieve that our results provide some preliminary insights on how to train an\nagent with similar abilities in a 3D environment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10090v1",
            "date": "2017-03-29 15:12:10+00:00",
            "title": "A Short Review of Ethical Challenges in Clinical Natural Language Processing",
            "authors": [
                "Simon \u0160uster",
                "St\u00e9phan Tulkens",
                "Walter Daelemans"
            ],
            "abstract": "Clinical NLP has an immense potential in contributing to how clinical\npractice will be revolutionized by the advent of large scale processing of\nclinical records. However, this potential has remained largely untapped due to\nslow progress primarily caused by strict data access policies for researchers.\nIn this paper, we discuss the concern for privacy and the measures it entails.\nWe also suggest sources of less sensitive data. Finally, we draw attention to\nbiases that can compromise the validity of empirical research and lead to\nsocially harmful applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10698v1",
            "date": "2017-03-30 21:57:37+00:00",
            "title": "Neutral evolution and turnover over centuries of English word popularity",
            "authors": [
                "Damian Ruck",
                "R. Alexander Bentley",
                "Alberto Acerbi",
                "Philip Garnett",
                "Daniel J. Hruschka"
            ],
            "abstract": "Here we test Neutral models against the evolution of English word frequency\nand vocabulary at the population scale, as recorded in annual word frequencies\nfrom three centuries of English language books. Against these data, we test\nboth static and dynamic predictions of two neutral models, including the\nrelation between corpus size and vocabulary size, frequency distributions, and\nturnover within those frequency distributions. Although a commonly used Neutral\nmodel fails to replicate all these emergent properties at once, we find that\nmodified two-stage Neutral model does replicate the static and dynamic\nproperties of the corpus data. This two-stage model is meant to represent a\nrelatively small corpus (population) of English books, analogous to a `canon',\nsampled by an exponentially increasing corpus of books in the wider population\nof authors. More broadly, this mode -- a smaller neutral model within a larger\nneutral model -- could represent more broadly those situations where mass\nattention is focused on a small subset of the cultural variants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10931v2",
            "date": "2017-03-31 15:05:45+00:00",
            "title": "Sentence Simplification with Deep Reinforcement Learning",
            "authors": [
                "Xingxing Zhang",
                "Mirella Lapata"
            ],
            "abstract": "Sentence simplification aims to make sentences easier to read and understand.\nMost recent approaches draw on insights from machine translation to learn\nsimplification rewrites from monolingual corpora of complex and simple\nsentences. We address the simplification problem with an encoder-decoder model\ncoupled with a deep reinforcement learning framework. Our model, which we call\n{\\sc Dress} (as shorthand for {\\bf D}eep {\\bf RE}inforcement {\\bf S}entence\n{\\bf S}implification), explores the space of possible simplifications while\nlearning to optimize a reward function that encourages outputs which are\nsimple, fluent, and preserve the meaning of the input. Experiments on three\ndatasets demonstrate that our model outperforms competitive simplification\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10960v3",
            "date": "2017-03-31 15:55:00+00:00",
            "title": "Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders",
            "authors": [
                "Tiancheng Zhao",
                "Ran Zhao",
                "Maxine Eskenazi"
            ],
            "abstract": "While recent neural encoder-decoder models have shown great promise in\nmodeling open-domain conversations, they often generate dull and generic\nresponses. Unlike past work that has focused on diversifying the output of the\ndecoder at word-level to alleviate this problem, we present a novel framework\nbased on conditional variational autoencoders that captures the discourse-level\ndiversity in the encoder. Our model uses latent variables to learn a\ndistribution over potential conversational intents and generates diverse\nresponses using only greedy decoders. We have further developed a novel variant\nthat is integrated with linguistic prior knowledge for better performance.\nFinally, the training procedure is improved by introducing a bag-of-word loss.\nOur proposed models have been validated to generate significantly more diverse\nresponses than baseline approaches and exhibit competence in discourse-level\ndecision-making.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00106v1",
            "date": "2017-04-29 01:08:48+00:00",
            "title": "Learning to Ask: Neural Question Generation for Reading Comprehension",
            "authors": [
                "Xinya Du",
                "Junru Shao",
                "Claire Cardie"
            ],
            "abstract": "We study automatic question generation for sentences from text passages in\nreading comprehension. We introduce an attention-based sequence learning model\nfor the task and investigate the effect of encoding sentence- vs.\nparagraph-level information. In contrast to all previous work, our model does\nnot rely on hand-crafted rules or a sophisticated NLP pipeline; it is instead\ntrainable end-to-end via sequence-to-sequence learning. Automatic evaluation\nresults show that our system significantly outperforms the state-of-the-art\nrule-based system. In human evaluations, questions generated by our system are\nalso rated as being more natural (i.e., grammaticality, fluency) and as more\ndifficult to answer (in terms of syntactic and lexical divergence from the\noriginal text and reasoning needed to answer).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00217v1",
            "date": "2017-04-29 17:50:02+00:00",
            "title": "Extending and Improving Wordnet via Unsupervised Word Embeddings",
            "authors": [
                "Mikhail Khodak",
                "Andrej Risteski",
                "Christiane Fellbaum",
                "Sanjeev Arora"
            ],
            "abstract": "This work presents an unsupervised approach for improving WordNet that builds\nupon recent advances in document and sense representation via distributional\nsemantics. We apply our methods to construct Wordnets in French and Russian,\nlanguages which both lack good manual constructions.1 These are evaluated on\ntwo new 600-word test sets for word-to-synset matching and found to improve\ngreatly upon synset recall, outperforming the best automated Wordnets in\nF-score. Our methods require very few linguistic resources, thus being\napplicable for Wordnet construction in low-resources languages, and may further\nbe applied to sense clustering and other Wordnet improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00464v2",
            "date": "2017-05-01 10:43:28+00:00",
            "title": "Speech-Based Visual Question Answering",
            "authors": [
                "Ted Zhang",
                "Dengxin Dai",
                "Tinne Tuytelaars",
                "Marie-Francine Moens",
                "Luc Van Gool"
            ],
            "abstract": "This paper introduces speech-based visual question answering (VQA), the task\nof generating an answer given an image and a spoken question. Two methods are\nstudied: an end-to-end, deep neural network that directly uses audio waveforms\nas input versus a pipelined approach that performs ASR (Automatic Speech\nRecognition) on the question, followed by text-based visual question answering.\nFurthermore, we investigate the robustness of both methods by injecting various\nlevels of noise into the spoken question and find both methods to be tolerate\nnoise at similar levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00545v3",
            "date": "2017-05-01 14:36:21+00:00",
            "title": "Labelled network subgraphs reveal stylistic subtleties in written texts",
            "authors": [
                "Vanessa Q. Marinho",
                "Graeme Hirst",
                "Diego R. Amancio"
            ],
            "abstract": "The vast amount of data and increase of computational capacity have allowed\nthe analysis of texts from several perspectives, including the representation\nof texts as complex networks. Nodes of the network represent the words, and\nedges represent some relationship, usually word co-occurrence. Even though\nnetworked representations have been applied to study some tasks, such\napproaches are not usually combined with traditional models relying upon\nstatistical paradigms. Because networked models are able to grasp textual\npatterns, we devised a hybrid classifier, called labelled subgraphs, that\ncombines the frequency of common words with small structures found in the\ntopology of the network, known as motifs. Our approach is illustrated in two\ncontexts, authorship attribution and translationese identification. In the\nformer, a set of novels written by different authors is analyzed. To identify\ntranslationese, texts from the Canadian Hansard and the European parliament\nwere classified as to original and translated instances. Our results suggest\nthat labelled subgraphs are able to represent texts and it should be further\nexplored in other tasks, such as the analysis of text complexity, language\nproficiency, and machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00601v2",
            "date": "2017-05-01 17:41:37+00:00",
            "title": "The Promise of Premise: Harnessing Question Premises in Visual Question Answering",
            "authors": [
                "Aroma Mahendru",
                "Viraj Prabhu",
                "Akrit Mohapatra",
                "Dhruv Batra",
                "Stefan Lee"
            ],
            "abstract": "In this paper, we make a simple observation that questions about images often\ncontain premises - objects and relationships implied by the question - and that\nreasoning about premises can help Visual Question Answering (VQA) models\nrespond more intelligently to irrelevant or previously unseen questions. When\npresented with a question that is irrelevant to an image, state-of-the-art VQA\nmodels will still answer purely based on learned language biases, resulting in\nnon-sensical or even misleading answers. We note that a visual question is\nirrelevant to an image if at least one of its premises is false (i.e. not\ndepicted in the image). We leverage this observation to construct a dataset for\nQuestion Relevance Prediction and Explanation (QRPE) by searching for false\npremises. We train novel question relevance detection models and show that\nmodels that reason about premises consistently outperform models that do not.\nWe also find that forcing standard VQA models to reason about premises during\ntraining can lead to improvements on tasks requiring compositional reasoning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.00648v1",
            "date": "2017-05-01 18:20:47+00:00",
            "title": "\"Liar, Liar Pants on Fire\": A New Benchmark Dataset for Fake News Detection",
            "authors": [
                "William Yang Wang"
            ],
            "abstract": "Automatic fake news detection is a challenging problem in deception\ndetection, and it has tremendous real-world political and social impacts.\nHowever, statistical approaches to combating fake news has been dramatically\nlimited by the lack of labeled benchmark datasets. In this paper, we present\nliar: a new, publicly available dataset for fake news detection. We collected a\ndecade-long, 12.8K manually labeled short statements in various contexts from\nPolitiFact.com, which provides detailed analysis report and links to source\ndocuments for each case. This dataset can be used for fact-checking research as\nwell. Notably, this new dataset is an order of magnitude larger than previously\nlargest public fake news datasets of similar type. Empirically, we investigate\nautomatic fake news detection based on surface-level linguistic patterns. We\nhave designed a novel, hybrid convolutional neural network to integrate\nmeta-data with text. We show that this hybrid approach can improve a text-only\ndeep learning model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00823v1",
            "date": "2017-05-02 07:07:55+00:00",
            "title": "STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset",
            "authors": [
                "Yuya Yoshikawa",
                "Yutaro Shigeto",
                "Akikazu Takeuchi"
            ],
            "abstract": "In recent years, automatic generation of image descriptions (captions), that\nis, image captioning, has attracted a great deal of attention. In this paper,\nwe particularly consider generating Japanese captions for images. Since most\navailable caption datasets have been constructed for English language, there\nare few datasets for Japanese. To tackle this problem, we construct a\nlarge-scale Japanese image caption dataset based on images from MS-COCO, which\nis called STAIR Captions. STAIR Captions consists of 820,310 Japanese captions\nfor 164,062 images. In the experiment, we show that a neural network trained\nusing STAIR Captions can generate more natural and better Japanese captions,\ncompared to those generated using English-Japanese machine translation after\ngenerating English captions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00861v1",
            "date": "2017-05-02 08:58:17+00:00",
            "title": "Deep Neural Machine Translation with Linear Associative Unit",
            "authors": [
                "Mingxuan Wang",
                "Zhengdong Lu",
                "Jie Zhou",
                "Qun Liu"
            ],
            "abstract": "Deep Neural Networks (DNNs) have provably enhanced the state-of-the-art\nNeural Machine Translation (NMT) with their capability in modeling complex\nfunctions and capturing complex linguistic structures. However NMT systems with\ndeep architecture in their encoder or decoder RNNs often suffer from severe\ngradient diffusion due to the non-linear recurrent activations, which often\nmake the optimization much more difficult. To address this problem we propose\nnovel linear associative units (LAU) to reduce the gradient propagation length\ninside the recurrent unit. Different from conventional approaches (LSTM unit\nand GRU), LAUs utilizes linear associative connections between input and output\nof the recurrent unit, which allows unimpeded information flow through both\nspace and time direction. The model is quite simple, but it is surprisingly\neffective. Our empirical study on Chinese-English translation shows that our\nmodel with proper configuration can improve by 11.7 BLEU upon Groundhog and the\nbest reported results in the same setting. On WMT14 English-German task and a\nlarger WMT14 English-French task, our model achieves comparable results with\nthe state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01253v1",
            "date": "2017-05-03 04:46:33+00:00",
            "title": "The Forgettable-Watcher Model for Video Question Answering",
            "authors": [
                "Hongyang Xue",
                "Zhou Zhao",
                "Deng Cai"
            ],
            "abstract": "A number of visual question answering approaches have been proposed recently,\naiming at understanding the visual scenes by answering the natural language\nquestions. While the image question answering has drawn significant attention,\nvideo question answering is largely unexplored.\n  Video-QA is different from Image-QA since the information and the events are\nscattered among multiple frames. In order to better utilize the temporal\nstructure of the videos and the phrasal structures of the answers, we propose\ntwo mechanisms: the re-watching and the re-reading mechanisms and combine them\ninto the forgettable-watcher model. Then we propose a TGIF-QA dataset for video\nquestion answering with the help of automatic question generation. Finally, we\nevaluate the models on our dataset. The experimental results show the\neffectiveness of our proposed models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.01306v1",
            "date": "2017-05-03 08:50:56+00:00",
            "title": "Amobee at SemEval-2017 Task 4: Deep Learning System for Sentiment Detection on Twitter",
            "authors": [
                "Alon Rozental",
                "Daniel Fleischer"
            ],
            "abstract": "This paper describes the Amobee sentiment analysis system, adapted to compete\nin SemEval 2017 task 4. The system consists of two parts: a supervised training\nof RNN models based on a Twitter sentiment treebank, and the use of feedforward\nNN, Naive Bayes and logistic regression classifiers to produce predictions for\nthe different sub-tasks. The algorithm reached the 3rd place on the 5-label\nclassification task (sub-task C).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02269v2",
            "date": "2017-05-05 15:37:11+00:00",
            "title": "Sequential Attention: A Context-Aware Alignment Function for Machine Reading",
            "authors": [
                "Sebastian Brarda",
                "Philip Yeres",
                "Samuel R. Bowman"
            ],
            "abstract": "In this paper we propose a neural network model with a novel Sequential\nAttention layer that extends soft attention by assigning weights to words in an\ninput sequence in a way that takes into account not just how well that word\nmatches a query, but how well surrounding words match. We evaluate this\napproach on the task of reading comprehension (on the Who did What and CNN\ndatasets) and show that it dramatically improves a strong baseline--the\nStanford Reader--and is competitive with the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02315v5",
            "date": "2017-05-05 17:31:12+00:00",
            "title": "ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases",
            "authors": [
                "Xiaosong Wang",
                "Yifan Peng",
                "Le Lu",
                "Zhiyong Lu",
                "Mohammadhadi Bagheri",
                "Ronald M. Summers"
            ],
            "abstract": "The chest X-ray is one of the most commonly accessible radiological\nexaminations for screening and diagnosis of many lung diseases. A tremendous\nnumber of X-ray imaging studies accompanied by radiological reports are\naccumulated and stored in many modern hospitals' Picture Archiving and\nCommunication Systems (PACS). On the other side, it is still an open question\nhow this type of hospital-size knowledge database containing invaluable imaging\ninformatics (i.e., loosely labeled) can be used to facilitate the data-hungry\ndeep learning paradigms in building truly large-scale high precision\ncomputer-aided diagnosis (CAD) systems.\n  In this paper, we present a new chest X-ray database, namely \"ChestX-ray8\",\nwhich comprises 108,948 frontal-view X-ray images of 32,717 unique patients\nwith the text-mined eight disease image labels (where each image can have\nmulti-labels), from the associated radiological reports using natural language\nprocessing. Importantly, we demonstrate that these commonly occurring thoracic\ndiseases can be detected and even spatially-located via a unified\nweakly-supervised multi-label image classification and disease localization\nframework, which is validated using our proposed dataset. Although the initial\nquantitative results are promising as reported, deep convolutional neural\nnetwork based \"reading chest X-rays\" (i.e., recognizing and locating the common\ndisease patterns trained with only image-level labels) remains a strenuous task\nfor fully-automated high precision CAD systems. Data download link:\nhttps://nihcc.app.box.com/v/ChestXray-NIHCC",
            "journal ref": "IEEE CVPR 2017, pp. 2097-2106 (2017)",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.02494v3",
            "date": "2017-05-06 15:11:30+00:00",
            "title": "Learning Distributed Representations of Texts and Entities from Knowledge Base",
            "authors": [
                "Ikuya Yamada",
                "Hiroyuki Shindo",
                "Hideaki Takeda",
                "Yoshiyasu Takefuji"
            ],
            "abstract": "We describe a neural network model that jointly learns distributed\nrepresentations of texts and knowledge base (KB) entities. Given a text in the\nKB, we train our proposed model to predict entities that are relevant to the\ntext. Our model is designed to be generic with the ability to address various\nNLP tasks with ease. We train the model using a large corpus of texts and their\nentity annotations extracted from Wikipedia. We evaluated the model on three\nimportant NLP tasks (i.e., sentence textual similarity, entity linking, and\nfactoid question answering) involving both unsupervised and supervised\nsettings. As a result, we achieved state-of-the-art results on all three of\nthese tasks. Our code and trained models are publicly available for further\nacademic research.",
            "journal ref": "Transactions of the Association for Computational Linguistics, 5\n  (2017), 397-411",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02735v1",
            "date": "2017-05-08 03:48:01+00:00",
            "title": "Combating Human Trafficking with Deep Multimodal Models",
            "authors": [
                "Edmund Tong",
                "Amir Zadeh",
                "Cara Jones",
                "Louis-Philippe Morency"
            ],
            "abstract": "Human trafficking is a global epidemic affecting millions of people across\nthe planet. Sex trafficking, the dominant form of human trafficking, has seen a\nsignificant rise mostly due to the abundance of escort websites, where human\ntraffickers can openly advertise among at-will escort advertisements. In this\npaper, we take a major step in the automatic detection of advertisements\nsuspected to pertain to human trafficking. We present a novel dataset called\nTrafficking-10k, with more than 10,000 advertisements annotated for this task.\nThe dataset contains two sources of information per advertisement: text and\nimages. For the accurate detection of trafficking advertisements, we designed\nand trained a deep multimodal model called the Human Trafficking Deep Network\n(HTDN).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03127v4",
            "date": "2017-05-09 00:09:38+00:00",
            "title": "Word and Phrase Translation with word2vec",
            "authors": [
                "Stefan Jansen"
            ],
            "abstract": "Word and phrase tables are key inputs to machine translations, but costly to\nproduce. New unsupervised learning methods represent words and phrases in a\nhigh-dimensional vector space, and these monolingual embeddings have been shown\nto encode syntactic and semantic relationships between language elements. The\ninformation captured by these embeddings can be exploited for bilingual\ntranslation by learning a transformation matrix that allows matching relative\npositions across two monolingual vector spaces. This method aims to identify\nhigh-quality candidates for word and phrase translation more cost-effectively\nfrom unlabeled data.\n  This paper expands the scope of previous attempts of bilingual translation to\nfour languages (English, German, Spanish, and French). It shows how to process\nthe source data, train a neural network to learn the high-dimensional\nembeddings for individual languages and expands the framework for testing their\nquality beyond the English language. Furthermore, it shows how to learn\nbilingual transformation matrices and obtain candidates for word and phrase\ntranslation, and assess their quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03454v2",
            "date": "2017-05-08 19:34:23+00:00",
            "title": "The Pragmatics of Indirect Commands in Collaborative Discourse",
            "authors": [
                "Matthew Lamm",
                "Mihail Eric"
            ],
            "abstract": "Today's artificial assistants are typically prompted to perform tasks through\ndirect, imperative commands such as \\emph{Set a timer} or \\emph{Pick up the\nbox}. However, to progress toward more natural exchanges between humans and\nthese assistants, it is important to understand the way non-imperative\nutterances can indirectly elicit action of an addressee. In this paper, we\ninvestigate command types in the setting of a grounded, collaborative game. We\nfocus on a less understood family of utterances for eliciting agent action,\nlocatives like \\emph{The chair is in the other room}, and demonstrate how these\nutterances indirectly command in specific game state contexts. Our work shows\nthat models with domain-specific grounding can effectively realize the\npragmatic reasoning that is necessary for more robust natural language\ninteraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03487v2",
            "date": "2017-05-06 18:35:35+00:00",
            "title": "A neural network system for transformation of regional cuisine style",
            "authors": [
                "Masahiro Kazama",
                "Minami Sugimoto",
                "Chizuru Hosokawa",
                "Keisuke Matsushima",
                "Lav R. Varshney",
                "Yoshiki Ishikawa"
            ],
            "abstract": "We propose a novel system which can transform a recipe into any selected\nregional style (e.g., Japanese, Mediterranean, or Italian). This system has two\ncharacteristics. First the system can identify the degree of regional cuisine\nstyle mixture of any selected recipe and visualize such regional cuisine style\nmixtures using barycentric Newton diagrams. Second, the system can suggest\ningredient substitutions through an extended word2vec model, such that a recipe\nbecomes more authentic for any selected regional cuisine style. Drawing on a\nlarge number of recipes from Yummly, an example shows how the proposed system\ncan transform a traditional Japanese recipe, Sukiyaki, into French style.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1705.03557v2",
            "date": "2017-05-09 22:12:19+00:00",
            "title": "DeepTingle",
            "authors": [
                "Ahmed Khalifa",
                "Gabriella A. B. Barros",
                "Julian Togelius"
            ],
            "abstract": "DeepTingle is a text prediction and classification system trained on the\ncollected works of the renowned fantastic gay erotica author Chuck Tingle.\nWhereas the writing assistance tools you use everyday (in the form of\npredictive text, translation, grammar checking and so on) are trained on\ngeneric, purportedly \"neutral\" datasets, DeepTingle is trained on a very\nspecific, internally consistent but externally arguably eccentric dataset. This\nallows us to foreground and confront the norms embedded in data-driven\ncreativity and productivity assistance tools. As such tools effectively\nfunction as extensions of our cognition into technology, it is important to\nidentify the norms they embed within themselves and, by extension, us.\nDeepTingle is realized as a web application based on LSTM networks and the\nGloVe word embedding, implemented in JavaScript with Keras-JS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03751v1",
            "date": "2017-05-10 13:19:38+00:00",
            "title": "A Survey of Distant Supervision Methods using PGMs",
            "authors": [
                "Gagan Madan"
            ],
            "abstract": "Relation Extraction refers to the task of populating a database with tuples\nof the form $r(e_1, e_2)$, where $r$ is a relation and $e_1$, $e_2$ are\nentities. Distant supervision is one such technique which tries to\nautomatically generate training examples based on an existing KB such as\nFreebase. This paper is a survey of some of the techniques in distant\nsupervision which primarily rely on Probabilistic Graphical Models (PGMs).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.03773v1",
            "date": "2017-05-10 13:55:53+00:00",
            "title": "Flexible and Creative Chinese Poetry Generation Using Neural Memory",
            "authors": [
                "Jiyuan Zhang",
                "Yang Feng",
                "Dong Wang",
                "Yang Wang",
                "Andrew Abel",
                "Shiyue Zhang",
                "Andi Zhang"
            ],
            "abstract": "It has been shown that Chinese poems can be successfully generated by\nsequence-to-sequence neural models, particularly with the attention mechanism.\nA potential problem of this approach, however, is that neural models can only\nlearn abstract rules, while poem generation is a highly creative process that\ninvolves not only rules but also innovations for which pure statistical models\nare not appropriate in principle. This work proposes a memory-augmented neural\nmodel for Chinese poem generation, where the neural model and the augmented\nmemory work together to balance the requirements of linguistic accordance and\naesthetic innovation, leading to innovative generations that are still\nrule-compliant. In addition, it is found that the memory mechanism provides\ninteresting flexibility that can be used to generate poems with different\nstyles.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.04187v1",
            "date": "2017-05-11 14:00:10+00:00",
            "title": "On the role of words in the network structure of texts: application to authorship attribution",
            "authors": [
                "Camilo Akimushkin",
                "Diego R. Amancio",
                "Osvaldo N. Oliveira Jr"
            ],
            "abstract": "Well-established automatic analyses of texts mainly consider frequencies of\nlinguistic units, e.g. letters, words and bigrams, while methods based on\nco-occurrence networks consider the structure of texts regardless of the nodes\nlabel (i.e. the words semantics). In this paper, we reconcile these distinct\nviewpoints by introducing a generalized similarity measure to compare texts\nwhich accounts for both the network structure of texts and the role of\nindividual words in the networks. We use the similarity measure for authorship\nattribution of three collections of books, each composed of 8 authors and 10\nbooks per author. High accuracy rates were obtained with typical values from\n90% to 98.75%, much higher than with the traditional the TF-IDF approach for\nthe same collections. These accuracies are also higher than taking only the\ntopology of networks into account. We conclude that the different properties of\nspecific words on the macroscopic scale structure of a whole text are as\nrelevant as their frequency of appearance; conversely, considering the identity\nof nodes brings further knowledge about a piece of text represented as a\nnetwork.",
            "journal ref": "Physica A v. 495, p. 49-58, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04350v2",
            "date": "2017-05-11 18:49:17+00:00",
            "title": "Imagination improves Multimodal Translation",
            "authors": [
                "Desmond Elliott",
                "\u00c1kos K\u00e1d\u00e1r"
            ],
            "abstract": "We decompose multimodal translation into two sub-tasks: learning to translate\nand learning visually grounded representations. In a multitask learning\nframework, translations are learned in an attention-based encoder-decoder, and\ngrounded representations are learned through image representation prediction.\nOur approach improves translation performance compared to the state of the art\non the Multi30K dataset. Furthermore, it is equally effective if we train the\nimage prediction task on the external MS COCO dataset, and we find improvements\nif we train the translation model on the external News Commentary parallel\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05311v2",
            "date": "2017-05-15 16:07:35+00:00",
            "title": "Using Titles vs. Full-text as Source for Automated Semantic Document Annotation",
            "authors": [
                "Lukas Galke",
                "Florian Mai",
                "Alan Schelten",
                "Dennis Brunsch",
                "Ansgar Scherp"
            ],
            "abstract": "A significant part of the largest Knowledge Graph today, the Linked Open Data\ncloud, consists of metadata about documents such as publications, news reports,\nand other media articles. While the widespread access to the document metadata\nis a tremendous advancement, it is yet not so easy to assign semantic\nannotations and organize the documents along semantic concepts. Providing\nsemantic annotations like concepts in SKOS thesauri is a classical research\ntopic, but typically it is conducted on the full-text of the documents. For the\nfirst time, we offer a systematic comparison of classification approaches to\ninvestigate how far semantic annotations can be conducted using just the\nmetadata of the documents such as titles published as labels on the Linked Open\nData cloud. We compare the classifications obtained from analyzing the\ndocuments' titles with semantic annotations obtained from analyzing the\nfull-text. Apart from the prominent text classification baselines kNN and SVM,\nwe also compare recent techniques of Learning to Rank and neural networks and\nrevisit the traditional methods logistic regression, Rocchio, and Naive Bayes.\nThe results show that across three of our four datasets, the performance of the\nclassifications using only titles reaches over 90% of the quality compared to\nthe classification performance when using the full-text. Thus, conducting\ndocument classification by just using the titles is a reasonable approach for\nautomated semantic annotation and opens up new possibilities for enriching\nKnowledge Graphs.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06031v2",
            "date": "2017-05-17 07:24:02+00:00",
            "title": "Learning to Identify Ambiguous and Misleading News Headlines",
            "authors": [
                "Wei Wei",
                "Xiaojun Wan"
            ],
            "abstract": "Accuracy is one of the basic principles of journalism. However, it is\nincreasingly hard to manage due to the diversity of news media. Some editors of\nonline news tend to use catchy headlines which trick readers into clicking.\nThese headlines are either ambiguous or misleading, degrading the reading\nexperience of the audience. Thus, identifying inaccurate news headlines is a\ntask worth studying. Previous work names these headlines \"clickbaits\" and\nmainly focus on the features extracted from the headlines, which limits the\nperformance since the consistency between headlines and news bodies is\nunderappreciated. In this paper, we clearly redefine the problem and identify\nambiguous and misleading headlines separately. We utilize class sequential\nrules to exploit structure information when detecting ambiguous headlines. For\nthe identification of misleading headlines, we extract features based on the\ncongruence between headlines and bodies. To make use of the large unlabeled\ndata set, we apply a co-training method and gain an increase in performance.\nThe experiment results show the effectiveness of our methods. Then we use our\nclassifiers to detect inaccurate headlines crawled from different sources and\nconduct a data analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06262v2",
            "date": "2017-05-17 17:08:11+00:00",
            "title": "Utility of General and Specific Word Embeddings for Classifying Translational Stages of Research",
            "authors": [
                "Vincent Major",
                "Alisa Surkis",
                "Yindalon Aphinyanaphongs"
            ],
            "abstract": "Conventional text classification models make a bag-of-words assumption\nreducing text into word occurrence counts per document. Recent algorithms such\nas word2vec are capable of learning semantic meaning and similarity between\nwords in an entirely unsupervised manner using a contextual window and doing so\nmuch faster than previous methods. Each word is projected into vector space\nsuch that similar meaning words such as \"strong\" and \"powerful\" are projected\ninto the same general Euclidean space. Open questions about these embeddings\ninclude their utility across classification tasks and the optimal properties\nand source of documents to construct broadly functional embeddings. In this\nwork, we demonstrate the usefulness of pre-trained embeddings for\nclassification in our task and demonstrate that custom word embeddings, built\nin the domain and for the tasks, can improve performance over word embeddings\nlearnt on more general data including news articles or Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07318v1",
            "date": "2017-05-20 15:04:34+00:00",
            "title": "Formalized Lambek Calculus in Higher Order Logic (HOL4)",
            "authors": [
                "Chun Tian"
            ],
            "abstract": "In this project, a rather complete proof-theoretical formalization of Lambek\nCalculus (non-associative with arbitrary extensions) has been ported from Coq\nproof assistent to HOL4 theorem prover, with some improvements and new\ntheorems.\n  Three deduction systems (Syntactic Calculus, Natural Deduction and Sequent\nCalculus) of Lambek Calculus are defined with many related theorems proved. The\nequivalance between these systems are formally proved. Finally, a formalization\nof Sequent Calculus proofs (where Coq has built-in supports) has been designed\nand implemented in HOL4. Some basic results including the sub-formula\nproperties of the so-called \"cut-free\" proofs are formally proved.\n  This work can be considered as the preliminary work towards a language parser\nbased on category grammars which is not multimodal but still has ability to\nsupport context-sensitive languages through customized extensions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07425v2",
            "date": "2017-05-21 10:16:49+00:00",
            "title": "Learning Semantic Relatedness From Human Feedback Using Metric Learning",
            "authors": [
                "Thomas Niebler",
                "Martin Becker",
                "Christian P\u00f6litz",
                "Andreas Hotho"
            ],
            "abstract": "Assessing the degree of semantic relatedness between words is an important\ntask with a variety of semantic applications, such as ontology learning for the\nSemantic Web, semantic search or query expansion. To accomplish this in an\nautomated fashion, many relatedness measures have been proposed. However, most\nof these metrics only encode information contained in the underlying corpus and\nthus do not directly model human intuition. To solve this, we propose to\nutilize a metric learning approach to improve existing semantic relatedness\nmeasures by learning from additional information, such as explicit human\nfeedback. For this, we argue to use word embeddings instead of traditional\nhigh-dimensional vector representations in order to leverage their semantic\ndensity and to reduce computational cost. We rigorously test our approach on\nseveral domains including tagging data as well as publicly available embeddings\nbased on Wikipedia texts and navigation. Human feedback about semantic\nrelatedness for learning and evaluation is extracted from publicly available\ndatasets such as MEN or WS-353. We find that our method can significantly\nimprove semantic relatedness measures by learning from additional information,\nsuch as explicit human feedback. For tagging data, we are the first to generate\nand study embeddings. Our results are of special interest for ontology and\nrecommendation engineers, but also for any other researchers and practitioners\nof Semantic Web techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07830v3",
            "date": "2017-05-22 16:19:21+00:00",
            "title": "Ask the Right Questions: Active Question Reformulation with Reinforcement Learning",
            "authors": [
                "Christian Buck",
                "Jannis Bulian",
                "Massimiliano Ciaramita",
                "Wojciech Gajewski",
                "Andrea Gesmundo",
                "Neil Houlsby",
                "Wei Wang"
            ],
            "abstract": "We frame Question Answering (QA) as a Reinforcement Learning task, an\napproach that we call Active Question Answering. We propose an agent that sits\nbetween the user and a black box QA system and learns to reformulate questions\nto elicit the best possible answers. The agent probes the system with,\npotentially many, natural language reformulations of an initial question and\naggregates the returned evidence to yield the best answer. The reformulation\nsystem is trained end-to-end to maximize answer quality using policy gradient.\nWe evaluate on SearchQA, a dataset of complex questions extracted from\nJeopardy!. The agent outperforms a state-of-the-art base model, playing the\nrole of the environment, and other benchmarks. We also analyze the language\nthat the agent has learned while interacting with the question answering\nsystem. We find that successful question reformulations look quite different\nfrom natural language paraphrases. The agent is able to discover non-trivial\nreformulation strategies that resemble classic information retrieval techniques\nsuch as term re-weighting (tf-idf) and stemming.",
            "journal ref": "Sixth International Conference on Learning Representations (ICLR),\n  2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08063v1",
            "date": "2017-05-23 02:55:56+00:00",
            "title": "Contextualizing Citations for Scientific Summarization using Word Embeddings and Domain Knowledge",
            "authors": [
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "Citation texts are sometimes not very informative or in some cases inaccurate\nby themselves; they need the appropriate context from the referenced paper to\nreflect its exact contributions. To address this problem, we propose an\nunsupervised model that uses distributed representation of words as well as\ndomain knowledge to extract the appropriate context from the reference paper.\nEvaluation results show the effectiveness of our model by significantly\noutperforming the state-of-the-art. We furthermore demonstrate how an effective\ncontextualization method results in improving citation-based summarization of\nthe scientific articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08094v1",
            "date": "2017-05-23 06:49:12+00:00",
            "title": "TwiInsight: Discovering Topics and Sentiments from Social Media Datasets",
            "authors": [
                "Zhengkui Wang",
                "Guangdong Bai",
                "Soumyadeb Chowdhury",
                "Quanqing Xu",
                "Zhi Lin Seow"
            ],
            "abstract": "Social media platforms contain a great wealth of information which provides\nopportunities for us to explore hidden patterns or unknown correlations, and\nunderstand people's satisfaction with what they are discussing. As one\nshowcase, in this paper, we present a system, TwiInsight which explores the\ninsight of Twitter data. Different from other Twitter analysis systems,\nTwiInsight automatically extracts the popular topics under different categories\n(e.g., healthcare, food, technology, sports and transport) discussed in Twitter\nvia topic modeling and also identifies the correlated topics across different\ncategories. Additionally, it also discovers the people's opinions on the tweets\nand topics via the sentiment analysis. The system also employs an intuitive and\ninformative visualization to show the uncovered insight. Furthermore, we also\ndevelop and compare six most popular algorithms - three for sentiment analysis\nand three for topic modeling.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1705.08321v1",
            "date": "2017-05-02 17:04:42+00:00",
            "title": "Increasing Papers' Discoverability with Precise Semantic Labeling: the sci.AI Platform",
            "authors": [
                "Roman Gurinovich",
                "Alexander Pashuk",
                "Yuriy Petrovskiy",
                "Alex Dmitrievskij",
                "Oleg Kuryan",
                "Alexei Scerbacov",
                "Antonia Tiggre",
                "Elena Moroz",
                "Yuri Nikolsky"
            ],
            "abstract": "The number of published findings in biomedicine increases continually. At the\nsame time, specifics of the domain's terminology complicates the task of\nrelevant publications retrieval. In the current research, we investigate\ninfluence of terms' variability and ambiguity on a paper's likelihood of being\nretrieved. We obtained statistics that demonstrate significance of the issue\nand its challenges, followed by presenting the sci.AI platform, which allows\nprecise terms labeling as a resolution.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1705.08488v1",
            "date": "2017-05-23 19:12:05+00:00",
            "title": "Second-Order Word Embeddings from Nearest Neighbor Topological Features",
            "authors": [
                "Denis Newman-Griffis",
                "Eric Fosler-Lussier"
            ],
            "abstract": "We introduce second-order vector representations of words, induced from\nnearest neighborhood topological features in pre-trained contextual word\nembeddings. We then analyze the effects of using second-order embeddings as\ninput features in two deep natural language processing models, for named entity\nrecognition and recognizing textual entailment, as well as a linear model for\nparaphrase recognition. Surprisingly, we find that nearest neighbor information\nalone is sufficient to capture most of the performance benefits derived from\nusing pre-trained word embeddings. Furthermore, second-order embeddings are\nable to handle highly heterogeneous data better than first-order\nrepresentations, though at the cost of some specificity. Additionally,\naugmenting contextual embeddings with second-order information further improves\nmodel performance in some cases. Due to variance in the random initializations\nof word embeddings, utilizing nearest neighbor features from multiple\nfirst-order embedding samples can also contribute to downstream performance\ngains. Finally, we identify intriguing characteristics of second-order\nembedding spaces for further research, including much higher density and\ndifferent semantic interpretations of cosine similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09207v4",
            "date": "2017-05-25 14:54:07+00:00",
            "title": "Learning Structured Text Representations",
            "authors": [
                "Yang Liu",
                "Mirella Lapata"
            ],
            "abstract": "In this paper, we focus on learning structure-aware document representations\nfrom data without recourse to a discourse parser or additional annotations.\nDrawing inspiration from recent efforts to empower neural networks with a\nstructural bias, we propose a model that can encode a document while\nautomatically inducing rich structural dependencies. Specifically, we embed a\ndifferentiable non-projective parsing algorithm into a neural model and use\nattention mechanisms to incorporate the structural biases. Experimental\nevaluation across different tasks and datasets shows that the proposed model\nachieves state-of-the-art results on document modeling tasks while inducing\nintermediate structures which are both interpretable and meaningful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09222v1",
            "date": "2017-05-23 17:54:32+00:00",
            "title": "Towards a Knowledge Graph based Speech Interface",
            "authors": [
                "Ashwini Jaya Kumar",
                "S\u00f6ren Auer",
                "Christoph Schmidt",
                "Joachim k\u00f6hler"
            ],
            "abstract": "Applications which use human speech as an input require a speech interface\nwith high recognition accuracy. The words or phrases in the recognised text are\nannotated with a machine-understandable meaning and linked to knowledge graphs\nfor further processing by the target application. These semantic annotations of\nrecognised words can be represented as a subject-predicate-object triples which\ncollectively form a graph often referred to as a knowledge graph. This type of\nknowledge representation facilitates to use speech interfaces with any spoken\ninput application, since the information is represented in logical, semantic\nform, retrieving and storing can be followed using any web standard query\nlanguages. In this work, we develop a methodology for linking speech input to\nknowledge graphs and study the impact of recognition errors in the overall\nprocess. We show that for a corpus with lower WER, the annotation and linking\nof entities to the DBpedia knowledge graph is considerable. DBpedia Spotlight,\na tool to interlink text documents with the linked open data is used to link\nthe speech recognition output to the DBpedia knowledge graph. Such a\nknowledge-based speech recognition interface is useful for applications such as\nquestion answering or spoken dialog systems.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1705.09296v2",
            "date": "2017-05-25 18:00:03+00:00",
            "title": "Neural Models for Documents with Metadata",
            "authors": [
                "Dallas Card",
                "Chenhao Tan",
                "Noah A. Smith"
            ],
            "abstract": "Most real-world document collections involve various types of metadata, such\nas author, source, and date, and yet the most commonly-used approaches to\nmodeling text corpora ignore this information. While specialized models have\nbeen developed for particular applications, few are widely used in practice, as\ncustomization typically requires derivation of a custom inference algorithm. In\nthis paper, we build on recent advances in variational inference methods and\npropose a general neural framework, based on topic models, to enable flexible\nincorporation of metadata and allow for rapid exploration of alternative\nmodels. Our approach achieves strong performance, with a manageable tradeoff\nbetween perplexity, coherence, and sparsity. Finally, we demonstrate the\npotential of our framework through an exploration of a corpus of articles about\nUS immigration.",
            "journal ref": "Dallas Card, Chenhao Tan, and Noah A. Smith. (2018). Neural Models\n  for Documents with Metadata. In Proceedings of the 56th Annual Meeting of the\n  Association for Computational Linguistics (Volume 1: Long Papers)",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1705.09655v2",
            "date": "2017-05-26 17:40:12+00:00",
            "title": "Style Transfer from Non-Parallel Text by Cross-Alignment",
            "authors": [
                "Tianxiao Shen",
                "Tao Lei",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "This paper focuses on style transfer on the basis of non-parallel text. This\nis an instance of a broad family of problems including machine translation,\ndecipherment, and sentiment modification. The key challenge is to separate the\ncontent from other aspects such as style. We assume a shared latent content\ndistribution across different text corpora, and propose a method that leverages\nrefined alignment of latent representations to perform style transfer. The\ntransferred sentences from one style should match example sentences from the\nother style as a population. We demonstrate the effectiveness of this\ncross-alignment method on three tasks: sentiment modification, decipherment of\nword substitution ciphers, and recovery of word order.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09755v1",
            "date": "2017-05-27 02:23:18+00:00",
            "title": "word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA",
            "authors": [
                "Andrew J. Landgraf",
                "Jeremy Bellay"
            ],
            "abstract": "We show that the skip-gram formulation of word2vec trained with negative\nsampling is equivalent to a weighted logistic PCA. This connection allows us to\nbetter understand the objective, compare it to other word embedding methods,\nand extend it to higher dimensional models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09975v1",
            "date": "2017-05-28 18:22:15+00:00",
            "title": "A Deep Multi-View Learning Framework for City Event Extraction from Twitter Data Streams",
            "authors": [
                "Nazli Farajidavar",
                "Sefki Kolozali",
                "Payam Barnaghi"
            ],
            "abstract": "Cities have been a thriving place for citizens over the centuries due to\ntheir complex infrastructure. The emergence of the Cyber-Physical-Social\nSystems (CPSS) and context-aware technologies boost a growing interest in\nanalysing, extracting and eventually understanding city events which\nsubsequently can be utilised to leverage the citizen observations of their\ncities. In this paper, we investigate the feasibility of using Twitter textual\nstreams for extracting city events. We propose a hierarchical multi-view deep\nlearning approach to contextualise citizen observations of various city systems\nand services. Our goal has been to build a flexible architecture that can learn\nrepresentations useful for tasks, thus avoiding excessive task-specific feature\nengineering. We apply our approach on a real-world dataset consisting of event\nreports and tweets of over four months from San Francisco Bay Area dataset and\nadditional datasets collected from London. The results of our evaluations show\nthat our proposed solution outperforms the existing models and can be used for\nextracting city related events with an averaged accuracy of 81% over all\nclasses. To further evaluate the impact of our Twitter event extraction model,\nwe have used two sources of authorised reports through collecting road traffic\ndisruptions data from Transport for London API, and parsing the Time Out London\nwebsite for sociocultural events. The analysis showed that 49.5% of the Twitter\ntraffic comments are reported approximately five hours prior to the authorities\nofficial records. Moreover, we discovered that amongst the scheduled\nsociocultural event topics; tweets reporting transportation, cultural and\nsocial events are 31.75% more likely to influence the distribution of the\nTwitter comments than sport, weather and crime topics.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1705.09993v2",
            "date": "2017-05-28 21:12:56+00:00",
            "title": "Deep Learning for User Comment Moderation",
            "authors": [
                "John Pavlopoulos",
                "Prodromos Malakasiotis",
                "Ion Androutsopoulos"
            ],
            "abstract": "Experimenting with a new dataset of 1.6M user comments from a Greek news\nportal and existing datasets of English Wikipedia comments, we show that an RNN\noutperforms the previous state of the art in moderation. A deep,\nclassification-specific attention mechanism improves further the overall\nperformance of the RNN. We also compare against a CNN and a word-list baseline,\nconsidering both fully automatic and semi-automatic moderation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10586v1",
            "date": "2017-05-29 15:53:00+00:00",
            "title": "Character-Based Text Classification using Top Down Semantic Model for Sentence Representation",
            "authors": [
                "Zhenzhou Wu",
                "Xin Zheng",
                "Daniel Dahlmeier"
            ],
            "abstract": "Despite the success of deep learning on many fronts especially image and\nspeech, its application in text classification often is still not as good as a\nsimple linear SVM on n-gram TF-IDF representation especially for smaller\ndatasets. Deep learning tends to emphasize on sentence level semantics when\nlearning a representation with models like recurrent neural network or\nrecursive neural network, however from the success of TF-IDF representation, it\nseems a bag-of-words type of representation has its strength. Taking advantage\nof both representions, we present a model known as TDSM (Top Down Semantic\nModel) for extracting a sentence representation that considers both the\nword-level semantics by linearly combining the words with attention weights and\nthe sentence-level semantics with BiLSTM and use it on text classification. We\napply the model on characters and our results show that our model is better\nthan all the other character-based and word-based convolutional neural network\nmodels by \\cite{zhang15} across seven different datasets with only 1\\% of their\nparameters. We also demonstrate that this model beats traditional linear models\non TF-IDF vectors on small and polished datasets like news article in which\ntypically deep learning models surrender.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.11001v3",
            "date": "2017-05-31 09:21:04+00:00",
            "title": "Adversarial Ranking for Language Generation",
            "authors": [
                "Kevin Lin",
                "Dianqi Li",
                "Xiaodong He",
                "Zhengyou Zhang",
                "Ming-Ting Sun"
            ],
            "abstract": "Generative adversarial networks (GANs) have great successes on synthesizing\ndata. However, the existing GANs restrict the discriminator to be a binary\nclassifier, and thus limit their learning capacity for tasks that need to\nsynthesize output with rich structures such as natural language descriptions.\nIn this paper, we propose a novel generative adversarial network, RankGAN, for\ngenerating high-quality language descriptions. Rather than training the\ndiscriminator to learn and assign absolute binary predicate for individual data\nsample, the proposed RankGAN is able to analyze and rank a collection of\nhuman-written and machine-written sentences by giving a reference group. By\nviewing a set of data samples collectively and evaluating their quality through\nrelative ranking scores, the discriminator is able to make better assessment\nwhich in turn helps to learn a better generator. The proposed RankGAN is\noptimized through the policy gradient technique. Experimental results on\nmultiple public datasets clearly demonstrate the effectiveness of the proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00023v2",
            "date": "2017-08-31 18:08:35+00:00",
            "title": "R$^3$: Reinforced Reader-Ranker for Open-Domain Question Answering",
            "authors": [
                "Shuohang Wang",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Zhiguo Wang",
                "Tim Klinger",
                "Wei Zhang",
                "Shiyu Chang",
                "Gerald Tesauro",
                "Bowen Zhou",
                "Jing Jiang"
            ],
            "abstract": "In recent years researchers have achieved considerable success applying\nneural network methods to question answering (QA). These approaches have\nachieved state of the art results in simplified closed-domain settings such as\nthe SQuAD (Rajpurkar et al., 2016) dataset, which provides a pre-selected\npassage, from which the answer to a given question may be extracted. More\nrecently, researchers have begun to tackle open-domain QA, in which the model\nis given a question and access to a large corpus (e.g., wikipedia) instead of a\npre-selected passage (Chen et al., 2017a). This setting is more complex as it\nrequires large-scale search for relevant passages by an information retrieval\ncomponent, combined with a reading comprehension model that \"reads\" the\npassages to generate an answer to the question. Performance in this setting\nlags considerably behind closed-domain performance. In this paper, we present a\nnovel open-domain QA system called Reinforced Ranker-Reader $(R^3)$, based on\ntwo algorithmic innovations. First, we propose a new pipeline for open-domain\nQA with a Ranker component, which learns to rank retrieved passages in terms of\nlikelihood of generating the ground-truth answer to a given question. Second,\nwe propose a novel method that jointly trains the Ranker along with an\nanswer-generation Reader model, based on reinforcement learning. We report\nextensive experimental results showing that our method significantly improves\non the state of the art for multiple open-domain QA datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00028v1",
            "date": "2017-08-31 18:19:08+00:00",
            "title": "Glyph-aware Embedding of Chinese Characters",
            "authors": [
                "Falcon Z. Dai",
                "Zheng Cai"
            ],
            "abstract": "Given the advantage and recent success of English character-level and\nsubword-unit models in several NLP tasks, we consider the equivalent modeling\nproblem for Chinese. Chinese script is logographic and many Chinese logograms\nare composed of common substructures that provide semantic, phonetic and\nsyntactic hints. In this work, we propose to explicitly incorporate the visual\nappearance of a character's glyph in its representation, resulting in a novel\nglyph-aware embedding of Chinese characters. Being inspired by the success of\nconvolutional neural networks in computer vision, we use them to incorporate\nthe spatio-structural patterns of Chinese glyphs as rendered in raw pixels. In\nthe context of two basic Chinese NLP tasks of language modeling and word\nsegmentation, the model learns to represent each character's task-relevant\nsemantic and syntactic information in the character-level embedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00071v1",
            "date": "2017-08-31 20:36:07+00:00",
            "title": "Weather impacts expressed sentiment",
            "authors": [
                "Patrick Baylis",
                "Nick Obradovich",
                "Yury Kryvasheyeu",
                "Haohui Chen",
                "Lorenzo Coviello",
                "Esteban Moro",
                "Manuel Cebrian",
                "James H. Fowler"
            ],
            "abstract": "We conduct the largest ever investigation into the relationship between\nmeteorological conditions and the sentiment of human expressions. To do this,\nwe employ over three and a half billion social media posts from tens of\nmillions of individuals from both Facebook and Twitter between 2009 and 2016.\nWe find that cold temperatures, hot temperatures, precipitation, narrower daily\ntemperature ranges, humidity, and cloud cover are all associated with worsened\nexpressions of sentiment, even when excluding weather-related posts. We compare\nthe magnitude of our estimates with the effect sizes associated with notable\nhistorical events occurring within our data.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1709.00103v7",
            "date": "2017-08-31 23:12:15+00:00",
            "title": "Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning",
            "authors": [
                "Victor Zhong",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "A significant amount of the world's knowledge is stored in relational\ndatabases. However, the ability for users to retrieve facts from a database is\nlimited due to a lack of understanding of query languages such as SQL. We\npropose Seq2SQL, a deep neural network for translating natural language\nquestions to corresponding SQL queries. Our model leverages the structure of\nSQL queries to significantly reduce the output space of generated queries.\nMoreover, we use rewards from in-the-loop query execution over the database to\nlearn a policy to generate unordered parts of the query, which we show are less\nsuitable for optimization via cross entropy loss. In addition, we will publish\nWikiSQL, a dataset of 80654 hand-annotated examples of questions and SQL\nqueries distributed across 24241 tables from Wikipedia. This dataset is\nrequired to train our model and is an order of magnitude larger than comparable\ndatasets. By applying policy-based reinforcement learning with a query\nexecution environment to WikiSQL, our model Seq2SQL outperforms attentional\nsequence to sequence models, improving execution accuracy from 35.9% to 59.4%\nand logical form accuracy from 23.4% to 48.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00354v2",
            "date": "2017-09-01 14:56:53+00:00",
            "title": "Query-by-example Spoken Term Detection using Attention-based Multi-hop Networks",
            "authors": [
                "Chia-Wei Ao",
                "Hung-yi Lee"
            ],
            "abstract": "Retrieving spoken content with spoken queries, or query-by- example spoken\nterm detection (STD), is attractive because it makes possible the matching of\nsignals directly on the acoustic level without transcribing them into text.\nHere, we propose an end-to-end query-by-example STD model based on an\nattention-based multi-hop network, whose input is a spoken query and an audio\nsegment containing several utterances; the output states whether the audio\nsegment includes the query. The model can be trained in either a supervised\nscenario using labeled data, or in an unsupervised fashion. In the supervised\nscenario, we find that the attention mechanism and multiple hops improve\nperformance, and that the attention weights indicate the time span of the\ndetected terms. In the unsupervised setting, the model mimics the behavior of\nthe existing query-by-example STD system, yielding performance comparable to\nthe existing system but with a lower search time complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00389v1",
            "date": "2017-08-30 04:24:06+00:00",
            "title": "End-to-end Learning for Short Text Expansion",
            "authors": [
                "Jian Tang",
                "Yue Wang",
                "Kai Zheng",
                "Qiaozhu Mei"
            ],
            "abstract": "Effectively making sense of short texts is a critical task for many real\nworld applications such as search engines, social media services, and\nrecommender systems. The task is particularly challenging as a short text\ncontains very sparse information, often too sparse for a machine learning\nalgorithm to pick up useful signals. A common practice for analyzing short text\nis to first expand it with external information, which is usually harvested\nfrom a large collection of longer texts. In literature, short text expansion\nhas been done with all kinds of heuristics. We propose an end-to-end solution\nthat automatically learns how to expand short text to optimize a given learning\ntask. A novel deep memory network is proposed to automatically find relevant\ninformation from a collection of longer documents and reformulate the short\ntext through a gating mechanism. Using short text classification as a\ndemonstrating task, we show that the deep memory network significantly\noutperforms classical text expansion methods with comprehensive experiments on\nreal world data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00541v1",
            "date": "2017-09-02 07:00:22+00:00",
            "title": "Patterns versus Characters in Subword-aware Neural Language Modeling",
            "authors": [
                "Rustem Takhanov",
                "Zhenisbek Assylbekov"
            ],
            "abstract": "Words in some natural languages can have a composite structure. Elements of\nthis structure include the root (that could also be composite), prefixes and\nsuffixes with which various nuances and relations to other words can be\nexpressed. Thus, in order to build a proper word representation one must take\ninto account its internal structure. From a corpus of texts we extract a set of\nfrequent subwords and from the latter set we select patterns, i.e. subwords\nwhich encapsulate information on character $n$-gram regularities. The selection\nis made using the pattern-based Conditional Random Field model with $l_1$\nregularization. Further, for every word we construct a new sequence over an\nalphabet of patterns. The new alphabet's symbols confine a local statistical\ncontext stronger than the characters, therefore they allow better\nrepresentations in ${\\mathbb{R}}^n$ and are better building blocks for word\nrepresentation. In the task of subword-aware language modeling, pattern-based\nmodels outperform character-based analogues by 2-20 perplexity points. Also, a\nrecurrent neural network in which a word is represented as a sum of embeddings\nof its patterns is on par with a competitive and significantly more\nsophisticated character-based convolutional architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00661v1",
            "date": "2017-09-03 04:16:15+00:00",
            "title": "Topic Independent Identification of Agreement and Disagreement in Social Media Dialogue",
            "authors": [
                "Amita Misra",
                "Marilyn Walker"
            ],
            "abstract": "Research on the structure of dialogue has been hampered for years because\nlarge dialogue corpora have not been available. This has impacted the dialogue\nresearch community's ability to develop better theories, as well as good off\nthe shelf tools for dialogue processing. Happily, an increasing amount of\ninformation and opinion exchange occur in natural dialogue in online forums,\nwhere people share their opinions about a vast range of topics. In particular\nwe are interested in rejection in dialogue, also called disagreement and\ndenial, where the size of available dialogue corpora, for the first time,\noffers an opportunity to empirically test theoretical accounts of the\nexpression and inference of rejection in dialogue. In this paper, we test\nwhether topic-independent features motivated by theoretical predictions can be\nused to recognize rejection in online forums in a topic independent way. Our\nresults show that our theoretically motivated features achieve 66% accuracy, an\nimprovement over a unigram baseline of an absolute 6%.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.00662v1",
            "date": "2017-09-03 04:16:25+00:00",
            "title": "Using Summarization to Discover Argument Facets in Online Ideological Dialog",
            "authors": [
                "Amita Misra",
                "Pranav Anand",
                "Jean E Fox Tree",
                "Marilyn Walker"
            ],
            "abstract": "More and more of the information available on the web is dialogic, and a\nsignificant portion of it takes place in online forum conversations about\ncurrent social and political topics. We aim to develop tools to summarize what\nthese conversations are about. What are the CENTRAL PROPOSITIONS associated\nwith different stances on an issue, what are the abstract objects under\ndiscussion that are central to a speaker's argument? How can we recognize that\ntwo CENTRAL PROPOSITIONS realize the same FACET of the argument? We hypothesize\nthat the CENTRAL PROPOSITIONS are exactly those arguments that people find most\nsalient, and use human summarization as a probe for discovering them. We\ndescribe our corpus of human summaries of opinionated dialogs, then show how we\ncan identify similar repeated arguments, and group them into FACETS across many\ndiscussions of a topic. We define a new task, ARGUMENT FACET SIMILARITY (AFS),\nand show that we can predict AFS with a .54 correlation score, versus an ngram\nsystem baseline of .39 and a semantic textual similarity system baseline of\n.45.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.00728v1",
            "date": "2017-09-03 14:54:29+00:00",
            "title": "Formalising Type-Logical Grammars in Agda",
            "authors": [
                "Wen Kokke"
            ],
            "abstract": "In recent years, the interest in using proof assistants to formalise and\nreason about mathematics and programming languages has grown. Type-logical\ngrammars, being closely related to type theories and systems used in functional\nprogramming, are a perfect candidate to next apply this curiosity to. The\nadvantages of using proof assistants is that they allow one to write formally\nverified proofs about one's type-logical systems, and that any theory, once\nimplemented, can immediately be computed with. The downside is that in many\ncases the formal proofs are written as an afterthought, are incomplete, or use\nobtuse syntax. This makes it that the verified proofs are often much more\ndifficult to read than the pen-and-paper proofs, and almost never directly\npublished. In this paper, we will try to remedy that by example.\n  Concretely, we use Agda to model the Lambek-Grishin calculus, a grammar logic\nwith a rich vocabulary of type-forming operations. We then present a verified\nprocedure for cut elimination in this system. Then we briefly outline a CPS\ntranslation from proofs in the Lambek-Grishin calculus to programs in Agda. And\nfinally, we will put our system to use in the analysis of a simple example\nsentence.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1709.00893v1",
            "date": "2017-09-04 10:34:34+00:00",
            "title": "Interactive Attention Networks for Aspect-Level Sentiment Classification",
            "authors": [
                "Dehong Ma",
                "Sujian Li",
                "Xiaodong Zhang",
                "Houfeng Wang"
            ],
            "abstract": "Aspect-level sentiment classification aims at identifying the sentiment\npolarity of specific target in its context. Previous approaches have realized\nthe importance of targets in sentiment classification and developed various\nmethods with the goal of precisely modeling their contexts via generating\ntarget-specific representations. However, these studies always ignore the\nseparate modeling of targets. In this paper, we argue that both targets and\ncontexts deserve special treatment and need to be learned their own\nrepresentations via interactive learning. Then, we propose the interactive\nattention networks (IAN) to interactively learn attentions in the contexts and\ntargets, and generate the representations for targets and contexts separately.\nWith this design, the IAN model can well represent a target and its collocative\ncontext, which is helpful to sentiment classification. Experimental results on\nSemEval 2014 Datasets demonstrate the effectiveness of our model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.00917v1",
            "date": "2017-09-04 12:25:18+00:00",
            "title": "Using Optimal Ratio Mask as Training Target for Supervised Speech Separation",
            "authors": [
                "Shasha Xia",
                "Hao Li",
                "Xueliang Zhang"
            ],
            "abstract": "Supervised speech separation uses supervised learning algorithms to learn a\nmapping from an input noisy signal to an output target. With the fast\ndevelopment of deep learning, supervised separation has become the most\nimportant direction in speech separation area in recent years. For the\nsupervised algorithm, training target has a significant impact on the\nperformance. Ideal ratio mask is a commonly used training target, which can\nimprove the speech intelligibility and quality of the separated speech.\nHowever, it does not take into account the correlation between noise and clean\nspeech. In this paper, we use the optimal ratio mask as the training target of\nthe deep neural network (DNN) for speech separation. The experiments are\ncarried out under various noise environments and signal to noise ratio (SNR)\nconditions. The results show that the optimal ratio mask outperforms other\ntraining targets in general.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1709.00947v1",
            "date": "2017-09-04 13:30:23+00:00",
            "title": "Learning Word Embeddings from the Portuguese Twitter Stream: A Study of some Practical Aspects",
            "authors": [
                "Pedro Saleiro",
                "Lu\u00eds Sarmento",
                "Eduarda Mendes Rodrigues",
                "Carlos Soares",
                "Eug\u00e9nio Oliveira"
            ],
            "abstract": "This paper describes a preliminary study for producing and distributing a\nlarge-scale database of embeddings from the Portuguese Twitter stream. We start\nby experimenting with a relatively small sample and focusing on three\nchallenges: volume of training data, vocabulary size and intrinsic evaluation\nmetrics. Using a single GPU, we were able to scale up vocabulary size from 2048\nwords embedded and 500K training examples to 32768 words over 10M training\nexamples while keeping a stable validation loss and approximately linear trend\non training time per epoch. We also observed that using less than 50\\% of the\navailable training examples for each vocabulary size might result in\noverfitting. Results on intrinsic evaluation show promising performance for a\nvocabulary size of 32768 words. Nevertheless, intrinsic evaluation metrics\nsuffer from over-sensitivity to their corresponding cosine similarity\nthresholds, indicating that a wider range of metrics need to be developed to\ntrack progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01188v1",
            "date": "2017-09-04 23:06:05+00:00",
            "title": "Storytelling Agents with Personality and Adaptivity",
            "authors": [
                "Zhichao Hu",
                "Marilyn A. Walker",
                "Michael Neff",
                "Jean E. Fox Tree"
            ],
            "abstract": "We explore the expression of personality and adaptivity through the gestures\nof virtual agents in a storytelling task. We conduct two experiments using four\ndifferent dialogic stories. We manipulate agent personality on the extraversion\nscale, whether the agents adapt to one another in their gestural performance\nand agent gender. Our results show that subjects are able to perceive the\nintended variation in extraversion between different virtual agents,\nindependently of the story they are telling and the gender of the agent. A\nsecond study shows that subjects also prefer adaptive to nonadaptive virtual\nagents.",
            "journal ref": "In International Conference on Intelligent Virtual Agents, pp.\n  181-193. Springer, Cham, 2015",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1709.01256v2",
            "date": "2017-09-05 06:47:03+00:00",
            "title": "Semantic Document Distance Measures and Unsupervised Document Revision Detection",
            "authors": [
                "Xiaofeng Zhu",
                "Diego Klabjan",
                "Patrick Bless"
            ],
            "abstract": "In this paper, we model the document revision detection problem as a minimum\ncost branching problem that relies on computing document distances.\nFurthermore, we propose two new document distance measures, word vector-based\nDynamic Time Warping (wDTW) and word vector-based Tree Edit Distance (wTED).\nOur revision detection system is designed for a large scale corpus and\nimplemented in Apache Spark. We demonstrate that our system can more precisely\ndetect revisions than state-of-the-art methods by utilizing the Wikipedia\nrevision dumps https://snap.stanford.edu/data/wiki-meta.html and simulated data\nsets.",
            "journal ref": "IJCNLP 2017",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.01687v1",
            "date": "2017-09-06 06:42:22+00:00",
            "title": "Semi-Supervised Recurrent Neural Network for Adverse Drug Reaction Mention Extraction",
            "authors": [
                "Shashank Gupta",
                "Sachin Pawar",
                "Nitin Ramrakhiyani",
                "Girish Palshikar",
                "Vasudeva Varma"
            ],
            "abstract": "Social media is an useful platform to share health-related information due to\nits vast reach. This makes it a good candidate for public-health monitoring\ntasks, specifically for pharmacovigilance. We study the problem of extraction\nof Adverse-Drug-Reaction (ADR) mentions from social media, particularly from\ntwitter. Medical information extraction from social media is challenging,\nmainly due to short and highly information nature of text, as compared to more\ntechnical and formal medical reports.\n  Current methods in ADR mention extraction relies on supervised learning\nmethods, which suffers from labeled data scarcity problem. The State-of-the-art\nmethod uses deep neural networks, specifically a class of Recurrent Neural\nNetwork (RNN) which are Long-Short-Term-Memory networks (LSTMs)\n\\cite{hochreiter1997long}. Deep neural networks, due to their large number of\nfree parameters relies heavily on large annotated corpora for learning the end\ntask. But in real-world, it is hard to get large labeled data, mainly due to\nheavy cost associated with manual annotation. Towards this end, we propose a\nnovel semi-supervised learning based RNN model, which can leverage unlabeled\ndata also present in abundance on social media. Through experiments we\ndemonstrate the effectiveness of our method, achieving state-of-the-art\nperformance in ADR mention extraction.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.01713v3",
            "date": "2017-09-06 08:27:59+00:00",
            "title": "Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art",
            "authors": [
                "Yuan Gao",
                "Brij Mohan Lal Srivastava",
                "James Salsman"
            ],
            "abstract": "We use automatic speech recognition to assess spoken English learner\npronunciation based on the authentic intelligibility of the learners' spoken\nresponses determined from support vector machine (SVM) classifier or deep\nlearning neural network model predictions of transcription correctness. Using\nnumeric features produced by PocketSphinx alignment mode and many recognition\npasses searching for the substitution and deletion of each expected phoneme and\ninsertion of unexpected phonemes in sequence, the SVM models achieve 82 percent\nagreement with the accuracy of Amazon Mechanical Turk crowdworker\ntranscriptions, up from 75 percent reported by multiple independent\nresearchers. Using such features with SVM classifier probability prediction\nmodels can help computer-aided pronunciation teaching (CAPT) systems provide\nintelligibility remediation.",
            "journal ref": "IEEE Advanced Information Management, Communications, Electronic\n  and Automation Control Conference 2 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01887v1",
            "date": "2017-09-06 17:15:49+00:00",
            "title": "Measuring the Similarity of Sentential Arguments in Dialog",
            "authors": [
                "Amita Misra",
                "Brian Ecker",
                "Marilyn A. Walker"
            ],
            "abstract": "When people converse about social or political topics, similar arguments are\noften paraphrased by different speakers, across many different conversations.\nDebate websites produce curated summaries of arguments on such topics; these\nsummaries typically consist of lists of sentences that represent frequently\nparaphrased propositions, or labels capturing the essence of one particular\naspect of an argument, e.g. Morality or Second Amendment. We call these\nfrequently paraphrased propositions ARGUMENT FACETS. Like these curated sites,\nour goal is to induce and identify argument facets across multiple\nconversations, and produce summaries. However, we aim to do this automatically.\nWe frame the problem as consisting of two steps: we first extract sentences\nthat express an argument from raw social media dialogs, and then rank the\nextracted arguments in terms of their similarity to one another. Sets of\nsimilar arguments are used to represent argument facets. We show here that we\ncan predict ARGUMENT FACET SIMILARITY with a correlation averaging 0.63\ncompared to a human topline averaging 0.68 over three debate topics, easily\nbeating several reasonable baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01888v1",
            "date": "2017-09-05 02:38:44+00:00",
            "title": "Language Modeling by Clustering with Word Embeddings for Text Readability Assessment",
            "authors": [
                "Miriam Cha",
                "Youngjune Gwon",
                "H. T. Kung"
            ],
            "abstract": "We present a clustering-based language model using word embeddings for text\nreadability prediction. Presumably, an Euclidean semantic space hypothesis\nholds true for word embeddings whose training is done by observing word\nco-occurrences. We argue that clustering with word embeddings in the metric\nspace should yield feature representations in a higher semantic space\nappropriate for text regression. Also, by representing features in terms of\nhistograms, our approach can naturally address documents of varying lengths. An\nempirical evaluation using the Common Core Standards corpus reveals that the\nfeatures formed on our clustering-based language model significantly improve\nthe previously known results for the same corpus in readability prediction. We\nalso evaluate the task of sentence matching based on semantic relatedness using\nthe Wiki-SimpleWiki corpus and find that our features lead to superior matching\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01915v1",
            "date": "2017-09-06 17:44:53+00:00",
            "title": "Towards Neural Machine Translation with Latent Tree Attention",
            "authors": [
                "James Bradbury",
                "Richard Socher"
            ],
            "abstract": "Building models that take advantage of the hierarchical structure of language\nwithout a priori annotation is a longstanding goal in natural language\nprocessing. We introduce such a model for the task of machine translation,\npairing a recurrent neural network grammar encoder with a novel attentional\nRNNG decoder and applying policy gradient reinforcement learning to induce\nunsupervised tree structures on both the source and target. When trained on\ncharacter-level datasets with no explicit segmentation or parse annotation, the\nmodel learns a plausible segmentation and shallow parse, obtaining performance\nclose to an attentional baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01991v1",
            "date": "2017-08-05 08:30:48+00:00",
            "title": "Semi-Automatic Terminology Ontology Learning Based on Topic Modeling",
            "authors": [
                "Monika Rani",
                "Amit Kumar Dhar",
                "O. P. Vyas"
            ],
            "abstract": "Ontologies provide features like a common vocabulary, reusability,\nmachine-readable content, and also allows for semantic search, facilitate agent\ninteraction and ordering & structuring of knowledge for the Semantic Web (Web\n3.0) application. However, the challenge in ontology engineering is automatic\nlearning, i.e., the there is still a lack of fully automatic approach from a\ntext corpus or dataset of various topics to form ontology using machine\nlearning techniques. In this paper, two topic modeling algorithms are explored,\nnamely LSI & SVD and Mr.LDA for learning topic ontology. The objective is to\ndetermine the statistical relationship between document and terms to build a\ntopic ontology and ontology graph with minimum human intervention. Experimental\nanalysis on building a topic ontology and semantic retrieving corresponding\ntopic ontology for the user's query demonstrating the effectiveness of the\nproposed approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.02755v5",
            "date": "2017-09-08 16:02:30+00:00",
            "title": "Simple Recurrent Units for Highly Parallelizable Recurrence",
            "authors": [
                "Tao Lei",
                "Yu Zhang",
                "Sida I. Wang",
                "Hui Dai",
                "Yoav Artzi"
            ],
            "abstract": "Common recurrent neural architectures scale poorly due to the intrinsic\ndifficulty in parallelizing their state computations. In this work, we propose\nthe Simple Recurrent Unit (SRU), a light recurrent unit that balances model\ncapacity and scalability. SRU is designed to provide expressive recurrence,\nenable highly parallelized implementation, and comes with careful\ninitialization to facilitate training of deep models. We demonstrate the\neffectiveness of SRU on multiple NLP tasks. SRU achieves 5--9x speed-up over\ncuDNN-optimized LSTM on classification and question answering datasets, and\ndelivers stronger results than LSTM and convolutional models. We also obtain an\naverage of 0.7 BLEU improvement over the Transformer model on translation by\nincorporating SRU into the architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02984v2",
            "date": "2017-09-09 17:28:10+00:00",
            "title": "Sentiment Polarity Detection for Software Development",
            "authors": [
                "Fabio Calefato",
                "Filippo Lanubile",
                "Federico Maiorano",
                "Nicole Novielli"
            ],
            "abstract": "The role of sentiment analysis is increasingly emerging to study software\ndevelopers' emotions by mining crowd-generated content within social software\nengineering tools. However, off-the-shelf sentiment analysis tools have been\ntrained on non-technical domains and general-purpose social media, thus\nresulting in misclassifications of technical jargon and problem reports. Here,\nwe present Senti4SD, a classifier specifically trained to support sentiment\nanalysis in developers' communication channels. Senti4SD is trained and\nvalidated using a gold standard of Stack Overflow questions, answers, and\ncomments manually annotated for sentiment polarity. It exploits a suite of both\nlexicon- and keyword-based features, as well as semantic features based on word\nembedding. With respect to a mainstream off-the-shelf tool, which we use as a\nbaseline, Senti4SD reduces the misclassifications of neutral and positive posts\nas emotionally negative. To encourage replications, we release a lab package\nincluding the classifier, the word embedding space, and the gold standard with\nannotation guidelines.",
            "journal ref": "Empirical Software Engineering, June 2018, Volume 23, Issue 3, pp\n  1352 - 1382",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1709.03036v1",
            "date": "2017-09-10 03:42:43+00:00",
            "title": "Abductive Matching in Question Answering",
            "authors": [
                "Kedar Dhamdhere",
                "Kevin S. McCurley",
                "Mukund Sundararajan",
                "Ankur Taly"
            ],
            "abstract": "We study question-answering over semi-structured data. We introduce a new way\nto apply the technique of semantic parsing by applying machine learning only to\nprovide annotations that the system infers to be missing; all the other parsing\nlogic is in the form of manually authored rules. In effect, the machine\nlearning is used to provide non-syntactic matches, a step that is ill-suited to\nmanual rules. The advantage of this approach is in its debuggability and in its\ntransparency to the end-user. We demonstrate the effectiveness of the approach\nby achieving state-of-the-art performance of 40.42% accuracy on a standard\nbenchmark dataset over tables from Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03742v1",
            "date": "2017-09-12 08:54:02+00:00",
            "title": "Dependencies: Formalising Semantic Catenae for Information Retrieval",
            "authors": [
                "Christina Lioma"
            ],
            "abstract": "Building machines that can understand text like humans is an AI-complete\nproblem. A great deal of research has already gone into this, with astounding\nresults, allowing everyday people to discuss with their telephones, or have\ntheir reading materials analysed and classified by computers. A prerequisite\nfor processing text semantics, common to the above examples, is having some\ncomputational representation of text as an abstract object. Operations on this\nrepresentation practically correspond to making semantic inferences, and by\nextension simulating understanding text. The complexity and granularity of\nsemantic processing that can be realised is constrained by the mathematical and\ncomputational robustness, expressiveness, and rigour of the tools used.\n  This dissertation contributes a series of such tools, diverse in their\nmathematical formulation, but common in their application to model semantic\ninferences when machines process text. These tools are principally expressed in\nnine distinct models that capture aspects of semantic dependence in highly\ninterpretable and non-complex ways. This dissertation further reflects on\npresent and future problems with the current research paradigm in this area,\nand makes recommendations on how to overcome them.\n  The amalgamation of the body of work presented in this dissertation advances\nthe complexity and granularity of semantic inferences that can be made\nautomatically by machines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.04109v4",
            "date": "2017-09-13 02:13:25+00:00",
            "title": "Empower Sequence Labeling with Task-Aware Neural Language Model",
            "authors": [
                "Liyuan Liu",
                "Jingbo Shang",
                "Frank F. Xu",
                "Xiang Ren",
                "Huan Gui",
                "Jian Peng",
                "Jiawei Han"
            ],
            "abstract": "Linguistic sequence labeling is a general modeling approach that encompasses\na variety of problems, such as part-of-speech tagging and named entity\nrecognition. Recent advances in neural networks (NNs) make it possible to build\nreliable models without handcrafted features. However, in many cases, it is\nhard to obtain sufficient annotations to train these models. In this study, we\ndevelop a novel neural framework to extract abundant knowledge hidden in raw\ntexts to empower the sequence labeling task. Besides word-level knowledge\ncontained in pre-trained word embeddings, character-aware neural language\nmodels are incorporated to extract character-level knowledge. Transfer learning\ntechniques are further adopted to mediate different components and guide the\nlanguage model towards the key knowledge. Comparing to previous methods, these\ntask-specific knowledge allows us to adopt a more concise model and conduct\nmore efficient training. Different from most transfer learning methods, the\nproposed framework does not rely on any additional supervision. It extracts\nknowledge from self-contained order information of training sequences.\nExtensive experiments on benchmark datasets demonstrate the effectiveness of\nleveraging character-level knowledge and the efficiency of co-training. For\nexample, on the CoNLL03 NER task, model training completes in about 6 hours on\na single GPU, reaching F1 score of 91.71$\\pm$0.10 without using any extra\nannotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04219v1",
            "date": "2017-09-13 09:43:02+00:00",
            "title": "Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets",
            "authors": [
                "Jeremy Barnes",
                "Roman Klinger",
                "Sabine Schulte im Walde"
            ],
            "abstract": "There has been a good amount of progress in sentiment analysis over the past\n10 years, including the proposal of new methods and the creation of benchmark\ndatasets. In some papers, however, there is a tendency to compare models only\non one or two datasets, either because of time restraints or because the model\nis tailored to a specific task. Accordingly, it is hard to understand how well\na certain model generalizes across different tasks and datasets. In this paper,\nwe contribute to this situation by comparing several models on six different\nbenchmarks, which belong to different domains and additionally have different\nlevels of granularity (binary, 3-class, 4-class and 5-class). We show that\nBi-LSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are\nparticularly good at fine-grained sentiment tasks (i. e., with more than two\nclasses). Incorporating sentiment information into word embeddings during\ntraining gives good results for datasets that are lexically similar to the\ntraining data. With our experiments, we contribute to a better understanding of\nthe performance of different model architectures on different data sets.\nConsequently, we detect novel state-of-the-art results on the SenTube datasets.",
            "journal ref": "In Proceedings of WASSA (2017). 2 - 12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04558v2",
            "date": "2017-09-13 22:48:39+00:00",
            "title": "Using NLU in Context for Question Answering: Improving on Facebook's bAbI Tasks",
            "authors": [
                "John S. Ball"
            ],
            "abstract": "For the next step in human to machine interaction, Artificial Intelligence\n(AI) should interact predominantly using natural language because, if it\nworked, it would be the fastest way to communicate. Facebook's toy tasks (bAbI)\nprovide a useful benchmark to compare implementations for conversational AI.\nWhile the published experiments so far have been based on exploiting the\ndistributional hypothesis with machine learning, our model exploits natural\nlanguage understanding (NLU) with the decomposition of language based on Role\nand Reference Grammar (RRG) and the brain-based Patom theory. Our combinatorial\nsystem for conversational AI based on linguistics has many advantages: passing\nbAbI task tests without parsing or statistics while increasing scalability. Our\nmodel validates both the training and test data to find 'garbage' input and\noutput (GIGO). It is not rules-based, nor does it use parts of speech, but\ninstead relies on meaning. While Deep Learning is difficult to debug and fix,\nevery step in our model can be understood and changed like any non-statistical\ncomputer program. Deep Learning's lack of explicable reasoning has raised\nopposition to AI, partly due to fear of the unknown. To support the goals of\nAI, we propose extended tasks to use human-level statements with tense, aspect\nand voice, and embedded clauses with junctures: and answers to be natural\nlanguage generation (NLG) instead of keywords. While machine learning permits\ninvalid training data to produce incorrect test responses, our system cannot\nbecause the context tracking would need to be intentionally broken. We believe\nno existing learning systems can currently solve these extended natural\nlanguage tests. There appears to be a knowledge gap between NLP researchers and\nlinguists, but ongoing competitive results such as these promise to narrow that\ngap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04625v3",
            "date": "2017-09-14 06:11:09+00:00",
            "title": "Robustness Analysis of Visual QA Models by Basic Questions",
            "authors": [
                "Jia-Hong Huang",
                "Cuong Duc Dao",
                "Modar Alfadly",
                "C. Huck Yang",
                "Bernard Ghanem"
            ],
            "abstract": "Visual Question Answering (VQA) models should have both high robustness and\naccuracy. Unfortunately, most of the current VQA research only focuses on\naccuracy because there is a lack of proper methods to measure the robustness of\nVQA models. There are two main modules in our algorithm. Given a natural\nlanguage question about an image, the first module takes the question as input\nand then outputs the ranked basic questions, with similarity scores, of the\nmain given question. The second module takes the main question, image and these\nbasic questions as input and then outputs the text-based answer of the main\nquestion about the given image. We claim that a robust VQA model is one, whose\nperformance is not changed much when related basic questions as also made\navailable to it as input. We formulate the basic questions generation problem\nas a LASSO optimization, and also propose a large scale Basic Question Dataset\n(BQD) and Rscore (novel robustness measure), for analyzing the robustness of\nVQA models. We hope our BQD will be used as a benchmark for to evaluate the\nrobustness of VQA models, so as to help the community build more robust and\naccurate VQA models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1709.04696v3",
            "date": "2017-09-14 10:42:44+00:00",
            "title": "DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding",
            "authors": [
                "Tao Shen",
                "Tianyi Zhou",
                "Guodong Long",
                "Jing Jiang",
                "Shirui Pan",
                "Chengqi Zhang"
            ],
            "abstract": "Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely\nused on NLP tasks to capture the long-term and local dependencies,\nrespectively. Attention mechanisms have recently attracted enormous interest\ndue to their highly parallelizable computation, significantly less training\ntime, and flexibility in modeling dependencies. We propose a novel attention\nmechanism in which the attention between elements from input sequence(s) is\ndirectional and multi-dimensional (i.e., feature-wise). A light-weight neural\nnet, \"Directional Self-Attention Network (DiSAN)\", is then proposed to learn\nsentence embedding, based solely on the proposed attention without any RNN/CNN\nstructure. DiSAN is only composed of a directional self-attention with temporal\norder encoded, followed by a multi-dimensional attention that compresses the\nsequence into a vector representation. Despite its simple form, DiSAN\noutperforms complicated RNN models on both prediction quality and time\nefficiency. It achieves the best test accuracy among all sentence encoding\nmethods and improves the most recent best result by 1.02% on the Stanford\nNatural Language Inference (SNLI) dataset, and shows state-of-the-art test\naccuracy on the Stanford Sentiment Treebank (SST), Multi-Genre natural language\ninference (MultiNLI), Sentences Involving Compositional Knowledge (SICK),\nCustomer Review, MPQA, TREC question-type classification and Subjectivity\n(SUBJ) datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04710v1",
            "date": "2017-09-14 11:31:19+00:00",
            "title": "Embedded-Graph Theory",
            "authors": [
                "Atsushi Yokoyama"
            ],
            "abstract": "In this paper, we propose a new type of graph, denoted as \"embedded-graph\",\nand its theory, which employs a distributed representation to describe the\nrelations on the graph edges. Embedded-graphs can express linguistic and\ncomplicated relations, which cannot be expressed by the existing edge-graphs or\nweighted-graphs. We introduce the mathematical definition of embedded-graph,\ntranslation, edge distance, and graph similarity. We can transform an\nembedded-graph into a weighted-graph and a weighted-graph into an edge-graph by\nthe translation method and by threshold calculation, respectively. The edge\ndistance of an embedded-graph is a distance based on the components of a target\nvector, and it is calculated through cosine similarity with the target vector.\nThe graph similarity is obtained considering the relations with linguistic\ncomplexity. In addition, we provide some examples and data structures for\nembedded-graphs in this paper.",
            "journal ref": "None",
            "category": "cs.DM"
        },
        {
            "url": "http://arxiv.org/abs/1709.04857v1",
            "date": "2017-09-10 13:18:58+00:00",
            "title": "A New Semantic Theory of Natural Language",
            "authors": [
                "Kun Xing"
            ],
            "abstract": "Formal Semantics and Distributional Semantics are two important semantic\nframeworks in Natural Language Processing (NLP). Cognitive Semantics belongs to\nthe movement of Cognitive Linguistics, which is based on contemporary cognitive\nscience. Each framework could deal with some meaning phenomena, but none of\nthem fulfills all requirements proposed by applications. A unified semantic\ntheory characterizing all important language phenomena has both theoretical and\npractical significance; however, although many attempts have been made in\nrecent years, no existing theory has achieved this goal yet.\n  This article introduces a new semantic theory that has the potential to\ncharacterize most of the important meaning phenomena of natural language and to\nfulfill most of the necessary requirements for philosophical analysis and for\nNLP applications. The theory is based on a unified representation of\ninformation, and constructs a kind of mathematical model called cognitive model\nto interpret natural language expressions in a compositional manner. It accepts\nthe empirical assumption of Cognitive Semantics, and overcomes most\nshortcomings of Formal Semantics and of Distributional Semantics. The theory,\nhowever, is not a simple combination of existing theories, but an extensive\ngeneralization of classic logic and Formal Semantics. It inherits nearly all\nadvantages of Formal Semantics, and also provides descriptive contents for\nobjects and events as fine-gram as possible, descriptive contents which\nrepresent the results of human cognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05014v2",
            "date": "2017-09-15 00:12:08+00:00",
            "title": "WOAH: Preliminaries to Zero-shot Ontology Learning for Conversational Agents",
            "authors": [
                "Gonzalo Estr\u00e1n Buyo"
            ],
            "abstract": "The present paper presents the Weighted Ontology Approximation Heuristic\n(WOAH), a novel zero-shot approach to ontology estimation for conversational\nagents development environments. This methodology extracts verbs and nouns\nseparately from data by distilling the dependencies obtained and applying\nsimilarity and sparsity metrics to generate an ontology estimation configurable\nin terms of the level of generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05036v2",
            "date": "2017-09-15 02:25:57+00:00",
            "title": "Query-based Attention CNN for Text Similarity Map",
            "authors": [
                "Tzu-Chien Liu",
                "Yu-Hsueh Wu",
                "Hung-Yi Lee"
            ],
            "abstract": "In this paper, we introduce Query-based Attention CNN(QACNN) for Text\nSimilarity Map, an end-to-end neural network for question answering. This\nnetwork is composed of compare mechanism, two-staged CNN architecture with\nattention mechanism, and a prediction layer. First, the compare mechanism\ncompares between the given passage, query, and multiple answer choices to build\nsimilarity maps. Then, the two-staged CNN architecture extracts features\nthrough word-level and sentence-level. At the same time, attention mechanism\nhelps CNN focus more on the important part of the passage based on the query\ninformation. Finally, the prediction layer find out the most possible answer\nchoice. We conduct this model on the MovieQA dataset using Plot Synopses only,\nand achieve 79.99% accuracy which is the state of the art on the dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.05453v3",
            "date": "2017-09-16 04:14:53+00:00",
            "title": "Augmenting End-to-End Dialog Systems with Commonsense Knowledge",
            "authors": [
                "Tom Young",
                "Erik Cambria",
                "Iti Chaturvedi",
                "Minlie Huang",
                "Hao Zhou",
                "Subham Biswas"
            ],
            "abstract": "Building dialog agents that can converse naturally with humans is a\nchallenging yet intriguing problem of artificial intelligence. In open-domain\nhuman-computer conversation, where the conversational agent is expected to\nrespond to human responses in an interesting and engaging way, commonsense\nknowledge has to be integrated into the model effectively. In this paper, we\ninvestigate the impact of providing commonsense knowledge about the concepts\ncovered in the dialog. Our model represents the first attempt to integrating a\nlarge commonsense knowledge base into end-to-end conversational models. In the\nretrieval-based scenario, we propose the Tri-LSTM model to jointly take into\naccount message and commonsense for selecting an appropriate response. Our\nexperiments suggest that the knowledge-augmented models are superior to their\nknowledge-free counterparts in automatic evaluation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.05467v1",
            "date": "2017-09-16 05:54:54+00:00",
            "title": "Acquiring Background Knowledge to Improve Moral Value Prediction",
            "authors": [
                "Ying Lin",
                "Joe Hoover",
                "Morteza Dehghani",
                "Marlon Mooijman",
                "Heng Ji"
            ],
            "abstract": "In this paper, we address the problem of detecting expressions of moral\nvalues in tweets using content analysis. This is a particularly challenging\nproblem because moral values are often only implicitly signaled in language,\nand tweets contain little contextual information due to length constraints. To\naddress these obstacles, we present a novel approach to automatically acquire\nbackground knowledge from an external knowledge base to enrich input texts and\nthus improve moral value prediction. By combining basic text features with\nbackground knowledge, our overall context-aware framework achieves performance\ncomparable to a single human annotator. To the best of our knowledge, this is\nthe first attempt to incorporate background knowledge for the prediction of\nimplicit psychological variables in the area of computational social science.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05587v1",
            "date": "2017-09-17 01:15:03+00:00",
            "title": "Character Distributions of Classical Chinese Literary Texts: Zipf's Law, Genres, and Epochs",
            "authors": [
                "Chao-Lin Liu",
                "Shuhua Zhang",
                "Yuanli Geng",
                "Huei-ling Lai",
                "Hongsu Wang"
            ],
            "abstract": "We collect 14 representative corpora for major periods in Chinese history in\nthis study. These corpora include poetic works produced in several dynasties,\nnovels of the Ming and Qing dynasties, and essays and news reports written in\nmodern Chinese. The time span of these corpora ranges between 1046 BCE and 2007\nCE. We analyze their character and word distributions from the viewpoint of the\nZipf's law, and look for factors that affect the deviations and similarities\nbetween their Zipfian curves. Genres and epochs demonstrated their influences\nin our analyses. Specifically, the character distributions for poetic works of\nbetween 618 CE and 1644 CE exhibit striking similarity. In addition, although\ntexts of the same dynasty may tend to use the same set of characters, their\ncharacter distributions still deviate from each other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05700v2",
            "date": "2017-09-17 17:56:24+00:00",
            "title": "Morphology-based Entity and Relational Entity Extraction Framework for Arabic",
            "authors": [
                "Amin Jaber",
                "Fadi A. Zaraket"
            ],
            "abstract": "Rule-based techniques to extract relational entities from documents allow\nusers to specify desired entities with natural language questions, finite state\nautomata, regular expressions and structured query language. They require\nlinguistic and programming expertise and lack support for Arabic morphological\nanalysis. We present a morphology-based entity and relational entity extraction\nframework for Arabic (MERF). MERF requires basic knowledge of linguistic\nfeatures and regular expressions, and provides the ability to interactively\nspecify Arabic morphological and synonymity features, tag types associated with\nregular expressions, and relations and code actions defined over matches of\nsubexpressions. MERF constructs entities and relational entities from matches\nof the specifications. We evaluated MERF with several case studies. The results\nshow that MERF requires shorter development time and effort compared to\nexisting application specific techniques and produces reasonably accurate\nresults within a reasonable overhead in run time.",
            "journal ref": "Traitement Automatique des Langues (TAL). 58.3 (2017): 97-121",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.05729v1",
            "date": "2017-09-18 00:01:07+00:00",
            "title": "Flexible Computing Services for Comparisons and Analyses of Classical Chinese Poetry",
            "authors": [
                "Chao-Lin Liu"
            ],
            "abstract": "We collect nine corpora of representative Chinese poetry for the time span of\n1046 BCE and 1644 CE for studying the history of Chinese words, collocations,\nand patterns. By flexibly integrating our own tools, we are able to provide new\nperspectives for approaching our goals. We illustrate the ideas with two\nexamples. The first example show a new way to compare word preferences of\npoets, and the second example demonstrates how we can utilize our corpora in\nhistorical studies of the Chinese words. We show the viability of the tools for\nacademic research, and we wish to make it helpful for enriching existing\nChinese dictionary as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05743v1",
            "date": "2017-09-18 02:09:08+00:00",
            "title": "Towards Building a Knowledge Base of Monetary Transactions from a News Collection",
            "authors": [
                "Jan R. Benetka",
                "Krisztian Balog",
                "Kjetil N\u00f8rv\u00e5g"
            ],
            "abstract": "We address the problem of extracting structured representations of economic\nevents from a large corpus of news articles, using a combination of natural\nlanguage processing and machine learning techniques. The developed techniques\nallow for semi-automatic population of a financial knowledge base, which, in\nturn, may be used to support a range of data mining and exploration tasks. The\nkey challenge we face in this domain is that the same event is often reported\nmultiple times, with varying correctness of details. We address this challenge\nby first collecting all information pertinent to a given event from the entire\ncorpus, then considering all possible representations of the event, and\nfinally, using a supervised learning method, to rank these representations by\nthe associated confidence scores. A main innovative element of our approach is\nthat it jointly extracts and stores all attributes of the event as a single\nrepresentation (quintuple). Using a purpose-built test set we demonstrate that\nour supervised learning approach can achieve 25% improvement in F1-score over\nbaseline methods that consider the earliest, the latest or the most frequent\nreporting of the event.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.05778v1",
            "date": "2017-09-18 05:00:34+00:00",
            "title": "Word Vector Enrichment of Low Frequency Words in the Bag-of-Words Model for Short Text Multi-class Classification Problems",
            "authors": [
                "Bradford Heap",
                "Michael Bain",
                "Wayne Wobcke",
                "Alfred Krzywicki",
                "Susanne Schmeidl"
            ],
            "abstract": "The bag-of-words model is a standard representation of text for many linear\nclassifier learners. In many problem domains, linear classifiers are preferred\nover more complex models due to their efficiency, robustness and\ninterpretability, and the bag-of-words text representation can capture\nsufficient information for linear classifiers to make highly accurate\npredictions. However in settings where there is a large vocabulary, large\nvariance in the frequency of terms in the training corpus, many classes and\nvery short text (e.g., single sentences or document titles) the bag-of-words\nrepresentation becomes extremely sparse, and this can reduce the accuracy of\nclassifiers. A particular issue in such settings is that short texts tend to\ncontain infrequently occurring or rare terms which lack class-conditional\nevidence. In this work we introduce a method for enriching the bag-of-words\nmodel by complementing such rare term information with related terms from both\ngeneral and domain-specific Word Vector models. By reducing sparseness in the\nbag-of-words models, our enrichment approach achieves improved classification\nover several baseline classifiers in a variety of text classification problems.\nOur approach is also efficient because it requires no change to the linear\nclassifier before or during training, since bag-of-words enrichment applies\nonly to text being classified.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06365v1",
            "date": "2017-09-19 12:09:21+00:00",
            "title": "MetaLDA: a Topic Model that Efficiently Incorporates Meta information",
            "authors": [
                "He Zhao",
                "Lan Du",
                "Wray Buntine",
                "Gang Liu"
            ],
            "abstract": "Besides the text content, documents and their associated words usually come\nwith rich sets of meta informa- tion, such as categories of documents and\nsemantic/syntactic features of words, like those encoded in word embeddings.\nIncorporating such meta information directly into the generative process of\ntopic models can improve modelling accuracy and topic quality, especially in\nthe case where the word-occurrence information in the training data is\ninsufficient. In this paper, we present a topic model, called MetaLDA, which is\nable to leverage either document or word meta information, or both of them\njointly. With two data argumentation techniques, we can derive an efficient\nGibbs sampling algorithm, which benefits from the fully local conjugacy of the\nmodel. Moreover, the algorithm is favoured by the sparsity of the meta\ninformation. Extensive experiments on several real world datasets demonstrate\nthat our model achieves comparable or improved performance in terms of both\nperplexity and topic quality, particularly in handling sparse texts. In\naddition, compared with other models using meta information, our model runs\nsignificantly faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06429v1",
            "date": "2017-09-19 13:52:28+00:00",
            "title": "Neural Networks for Text Correction and Completion in Keyboard Decoding",
            "authors": [
                "Shaona Ghosh",
                "Per Ola Kristensson"
            ],
            "abstract": "Despite the ubiquity of mobile and wearable text messaging applications, the\nproblem of keyboard text decoding is not tackled sufficiently in the light of\nthe enormous success of the deep learning Recurrent Neural Network (RNN) and\nConvolutional Neural Networks (CNN) for natural language understanding. In\nparticular, considering that the keyboard decoders should operate on devices\nwith memory and processor resource constraints, makes it challenging to deploy\nindustrial scale deep neural network (DNN) models. This paper proposes a\nsequence-to-sequence neural attention network system for automatic text\ncorrection and completion. Given an erroneous sequence, our model encodes\ncharacter level hidden representations and then decodes the revised sequence\nthus enabling auto-correction and completion. We achieve this by a combination\nof character level CNN and gated recurrent unit (GRU) encoder along with and a\nword level gated recurrent unit (GRU) attention decoder. Unlike traditional\nlanguage models that learn from billions of words, our corpus size is only 12\nmillion words; an order of magnitude smaller. The memory footprint of our\nlearnt model for inference and prediction is also an order of magnitude smaller\nthan the conventional language model based text decoders. We report baseline\nperformance for neural keyboard decoders in such limited domain. Our models\nachieve a word level accuracy of $90\\%$ and a character error rate CER of\n$2.4\\%$ over the Twitter typo dataset. We present a novel dataset of noisy to\ncorrected mappings by inducing the noise distribution from the Twitter data\nover the OpenSubtitles 2009 dataset; on which our model predicts with a word\nlevel accuracy of $98\\%$ and sequence accuracy of $68.9\\%$. In our user study,\nour model achieved an average CER of $2.6\\%$ with the state-of-the-art\nnon-neural touch-screen keyboard decoder at CER of $1.6\\%$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06918v1",
            "date": "2017-09-20 15:03:51+00:00",
            "title": "Constructing a Hierarchical User Interest Structure based on User Profiles",
            "authors": [
                "Chao Zhao",
                "Min Zhao",
                "Yi Guan"
            ],
            "abstract": "The interests of individual internet users fall into a hierarchical structure\nwhich is useful in regards to building personalized searches and\nrecommendations. Most studies on this subject construct the interest hierarchy\nof a single person from the document perspective. In this study, we constructed\nthe user interest hierarchy via user profiles. We organized 433,397 user\ninterests, referred to here as \"attentions\", into a user attention network\n(UAN) from 200 million user profiles; we then applied the Louvain algorithm to\ndetect hierarchical clusters in these attentions. Finally, a 26-level hierarchy\nwith 34,676 clusters was obtained. We found that these attention clusters were\naggregated according to certain topics as opposed to the hyponymy-relation\nbased conceptual ontologies. The topics can be entities or concepts, and the\nrelations were not restrained by hyponymy. The concept relativity encapsulated\nin the user's interest can be captured by labeling the attention clusters with\ncorresponding concepts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07432v2",
            "date": "2017-09-21 17:50:04+00:00",
            "title": "Dynamic Evaluation of Neural Sequence Models",
            "authors": [
                "Ben Krause",
                "Emmanuel Kahembwe",
                "Iain Murray",
                "Steve Renals"
            ],
            "abstract": "We present methodology for using dynamic evaluation to improve neural\nsequence models. Models are adapted to recent history via a gradient descent\nbased mechanism, causing them to assign higher probabilities to re-occurring\nsequential patterns. Dynamic evaluation outperforms existing adaptation\napproaches in our comparisons. Dynamic evaluation improves the state-of-the-art\nword-level perplexities on the Penn Treebank and WikiText-2 datasets to 51.1\nand 44.3 respectively, and the state-of-the-art character-level cross-entropies\non the text8 and Hutter Prize datasets to 1.19 bits/char and 1.08 bits/char\nrespectively.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1709.07642v2",
            "date": "2017-09-22 09:08:47+00:00",
            "title": "Code Attention: Translating Code to Comments by Exploiting Domain Features",
            "authors": [
                "Wenhao Zheng",
                "Hong-Yu Zhou",
                "Ming Li",
                "Jianxin Wu"
            ],
            "abstract": "Appropriate comments of code snippets provide insight for code functionality,\nwhich are helpful for program comprehension. However, due to the great cost of\nauthoring with the comments, many code projects do not contain adequate\ncomments. Automatic comment generation techniques have been proposed to\ngenerate comments from pieces of code in order to alleviate the human efforts\nin annotating the code. Most existing approaches attempt to exploit certain\ncorrelations (usually manually given) between code and generated comments,\nwhich could be easily violated if the coding patterns change and hence the\nperformance of comment generation declines. In this paper, we first build\nC2CGit, a large dataset from open projects in GitHub, which is more than\n20$\\times$ larger than existing datasets. Then we propose a new attention\nmodule called Code Attention to translate code to comments, which is able to\nutilize the domain features of code snippets, such as symbols and identifiers.\nWe make ablation studies to determine effects of different parts in Code\nAttention. Experimental results demonstrate that the proposed module has better\nperformance over existing approaches in both BLEU and METEOR.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.08366v1",
            "date": "2017-09-25 08:28:23+00:00",
            "title": "\"Let me convince you to buy my product ... \": A Case Study of an Automated Persuasive System for Fashion Products",
            "authors": [
                "Vitobha Munigala",
                "Srikanth Tamilselvam",
                "Anush Sankaran"
            ],
            "abstract": "Persuasivenes is a creative art aimed at making people believe in certain set\nof beliefs. Many a times, such creativity is about adapting richness of one\ndomain into another to strike a chord with the target audience. In this\nresearch, we present PersuAIDE! - A persuasive system based on linguistic\ncreativity to transform given sentence to generate various forms of persuading\nsentences. These various forms cover multiple focus of persuasion such as\nmemorability and sentiment. For a given simple product line, the algorithm is\ncomposed of several steps including: (i) select an appropriate well-known\nexpression for the target domain to add memorability, (ii) identify keywords\nand entities in the given sentence and expression and transform it to produce\ncreative persuading sentence, and (iii) adding positive or negative sentiment\nfor further persuasion. The persuasive conversion were manually verified using\nqualitative results and the effectiveness of the proposed approach is\nempirically discussed.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.08448v3",
            "date": "2017-09-25 12:22:51+00:00",
            "title": "Extracting Ontological Knowledge from Textual Descriptions",
            "authors": [
                "Kevin Alex Mathews",
                "P Sreenivasa Kumar"
            ],
            "abstract": "Authoring of OWL-DL ontologies is intellectually challenging and to make this\nprocess simpler, many systems accept natural language text as input. A\ntext-based ontology authoring approach can be successful only when it is\ncombined with an effective method for extracting ontological axioms from text.\nExtracting axioms from unrestricted English input is a substantially\nchallenging task due to the richness of the language. Controlled natural\nlanguages (CNLs) have been proposed in this context and these tend to be highly\nrestrictive. In this paper, we propose a new CNL called TEDEI (TExtual\nDEscription Identifier) whose grammar is inspired by the different ways OWL-DL\nconstructs are expressed in English. We built a system that transforms TEDEI\nsentences into corresponding OWL-DL axioms. Now, ambiguity due to different\npossible lexicalizations of sentences and semantic ambiguity present in\nsentences are challenges in this context. We find that the best way to handle\nthese challenges is to construct axioms corresponding to alternative\nformalizations of the sentence so that the end-user can make an appropriate\nchoice. The output is compared against human-authored axioms and in substantial\nnumber of cases, human-authored axiom is indeed one of the alternatives given\nby the system. The proposed system substantially enhances the types of sentence\nstructures that can be used for ontology authoring.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.08600v3",
            "date": "2017-09-25 17:10:46+00:00",
            "title": "EZLearn: Exploiting Organic Supervision in Large-Scale Data Annotation",
            "authors": [
                "Maxim Grechkin",
                "Hoifung Poon",
                "Bill Howe"
            ],
            "abstract": "Many real-world applications require automated data annotation, such as\nidentifying tissue origins based on gene expressions and classifying images\ninto semantic categories. Annotation classes are often numerous and subject to\nchanges over time, and annotating examples has become the major bottleneck for\nsupervised learning methods. In science and other high-value domains, large\nrepositories of data samples are often available, together with two sources of\norganic supervision: a lexicon for the annotation classes, and text\ndescriptions that accompany some data samples. Distant supervision has emerged\nas a promising paradigm for exploiting such indirect supervision by\nautomatically annotating examples where the text description contains a class\nmention in the lexicon. However, due to linguistic variations and ambiguities,\nsuch training data is inherently noisy, which limits the accuracy of this\napproach. In this paper, we introduce an auxiliary natural language processing\nsystem for the text modality, and incorporate co-training to reduce noise and\naugment signal in distant supervision. Without using any manually labeled data,\nour EZLearn system learned to accurately annotate data samples in functional\ngenomics and scientific figure comprehension, substantially outperforming\nstate-of-the-art supervised methods trained on tens of thousands of annotated\nexamples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08694v1",
            "date": "2017-09-19 18:02:51+00:00",
            "title": "Methodology and Results for the Competition on Semantic Similarity Evaluation and Entailment Recognition for PROPOR 2016",
            "authors": [
                "Luciano Barbosa",
                "Paulo R. Cavalin",
                "Victor Guimaraes",
                "Matthias Kormaksson"
            ],
            "abstract": "In this paper, we present the methodology and the results obtained by our\nteams, dubbed Blue Man Group, in the ASSIN (from the Portuguese {\\it\nAvalia\\c{c}\\~ao de Similaridade Sem\\^antica e Infer\\^encia Textual})\ncompetition, held at PROPOR 2016\\footnote{International Conference on the\nComputational Processing of the Portuguese Language -\nhttp://propor2016.di.fc.ul.pt/}. Our team's strategy consisted of evaluating\nmethods based on semantic word vectors, following two distinct directions: 1)\nto make use of low-dimensional, compact, feature sets, and 2) deep\nlearning-based strategies dealing with high-dimensional feature vectors.\nEvaluation results demonstrated that the first strategy was more promising, so\nthat the results from the second strategy have been discarded. As a result, by\nconsidering the best run of each of the six teams, we have been able to achieve\nthe best accuracy and F1 values in entailment recognition, in the Brazilian\nPortuguese set, and the best F1 score overall. In the semantic similarity task,\nour team was ranked second in the Brazilian Portuguese set, and third\nconsidering both sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08858v1",
            "date": "2017-09-26 06:51:36+00:00",
            "title": "Polysemy Detection in Distributed Representation of Word Sense",
            "authors": [
                "Kana Oomoto",
                "Haruka Oikawa",
                "Eiko Yamamoto",
                "Mitsuo Yoshida",
                "Masayuki Okabe",
                "Kyoji Umemura"
            ],
            "abstract": "In this paper, we propose a statistical test to determine whether a given\nword is used as a polysemic word or not. The statistic of the word in this test\nroughly corresponds to the fluctuation in the senses of the neighboring words a\nnd the word itself. Even though the sense of a word corresponds to a single\nvector, we discuss how polysemy of the words affects the position of vectors.\nFinally, we also explain the method to detect this effect.",
            "journal ref": "None",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/1709.09118v5",
            "date": "2017-09-26 16:32:20+00:00",
            "title": "Tensor Product Generation Networks for Deep NLP Modeling",
            "authors": [
                "Qiuyuan Huang",
                "Paul Smolensky",
                "Xiaodong He",
                "Li Deng",
                "Dapeng Wu"
            ],
            "abstract": "We present a new approach to the design of deep networks for natural language\nprocessing (NLP), based on the general technique of Tensor Product\nRepresentations (TPRs) for encoding and processing symbol structures in\ndistributed neural networks. A network architecture --- the Tensor Product\nGeneration Network (TPGN) --- is proposed which is capable in principle of\ncarrying out TPR computation, but which uses unconstrained deep learning to\ndesign its internal representations. Instantiated in a model for image-caption\ngeneration, TPGN outperforms LSTM baselines when evaluated on the COCO dataset.\nThe TPR-capable structure enables interpretation of internal representations\nand operations, which prove to contain considerable grammatical content. Our\ncaption-generation model can be interpreted as generating sequences of\ngrammatical categories and retrieving words by their categories from a plan\nencoded as a distributed representation.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1709.09119v1",
            "date": "2017-09-26 16:33:59+00:00",
            "title": "Integration of Japanese Papers Into the DBLP Data Set",
            "authors": [
                "Paul Christian Sommerhoff"
            ],
            "abstract": "If someone is looking for a certain publication in the field of computer\nscience, the searching person is likely to use the DBLP to find the desired\npublication. The DBLP data set is continuously extended with new publications,\nor rather their metadata, for example the names of involved authors, the title\nand the publication date. While the size of the data set is already remarkable,\nspecific areas can still be improved. The DBLP offers a huge collection of\nEnglish papers because most papers concerning computer science are published in\nEnglish. Nevertheless, there are official publications in other languages which\nare supposed to be added to the data set. One kind of these are Japanese\npapers. This diploma thesis will show a way to automatically process\npublication lists of Japanese papers and to make them ready for an import into\nthe DBLP data set. Especially important are the problems along the way of\nprocessing, such as transcription handling and Personal Name Matching with\nJapanese names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09250v1",
            "date": "2017-09-26 20:24:10+00:00",
            "title": "Lexical Disambiguation in Natural Language Questions (NLQs)",
            "authors": [
                "Omar Al-Harbi",
                "Shaidah Jusoh",
                "Norita Md Norwawi"
            ],
            "abstract": "Question processing is a fundamental step in a question answering (QA)\napplication, and its quality impacts the performance of QA application. The\nmajor challenging issue in processing question is how to extract semantic of\nnatural language questions (NLQs). A human language is ambiguous. Ambiguity may\noccur at two levels; lexical and syntactic. In this paper, we propose a new\napproach for resolving lexical ambiguity problem by integrating context\nknowledge and concepts knowledge of a domain, into shallow natural language\nprocessing (SNLP) techniques. Concepts knowledge is modeled using ontology,\nwhile context knowledge is obtained from WordNet, and it is determined based on\nneighborhood words in a question. The approach will be applied to a university\nQA system.",
            "journal ref": "IJCSI International Journal of Computer Science Issues, Vol. 8,\n  Issue 4, No 2, July 2011 (143-150)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09345v4",
            "date": "2017-09-27 06:02:57+00:00",
            "title": "A Read-Write Memory Network for Movie Story Understanding",
            "authors": [
                "Seil Na",
                "Sangho Lee",
                "Jisung Kim",
                "Gunhee Kim"
            ],
            "abstract": "We propose a novel memory network model named Read-Write Memory Network\n(RWMN) to perform question and answering tasks for large-scale, multimodal\nmovie story understanding. The key focus of our RWMN model is to design the\nread network and the write network that consist of multiple convolutional\nlayers, which enable memory read and write operations to have high capacity and\nflexibility. While existing memory-augmented network models treat each memory\nslot as an independent block, our use of multi-layered CNNs allows the model to\nread and write sequential memory cells as chunks, which is more reasonable to\nrepresent a sequential story because adjacent memory blocks often have strong\ncorrelations. For evaluation, we apply our model to all the six tasks of the\nMovieQA benchmark, and achieve the best accuracies on several tasks, especially\non the visual QA task. Our model shows a potential to better understand not\nonly the content in the story, but also more abstract information, such as\nrelationships between characters and the reasons for their actions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1709.09404v1",
            "date": "2017-09-27 09:20:06+00:00",
            "title": "A Preliminary Study for Building an Arabic Corpus of Pair Questions-Texts from the Web: AQA-Webcorp",
            "authors": [
                "Wided Bakari",
                "Patrice Bellot",
                "Mahmoud Neji"
            ],
            "abstract": "With the development of electronic media and the heterogeneity of Arabic data\non the Web, the idea of building a clean corpus for certain applications of\nnatural language processing, including machine translation, information\nretrieval, question answer, become more and more pressing. In this manuscript,\nwe seek to create and develop our own corpus of pair's questions-texts. This\nconstitution then will provide a better base for our experimentation step.\nThus, we try to model this constitution by a method for Arabic insofar as it\nrecovers texts from the web that could prove to be answers to our factual\nquestions. To do this, we had to develop a java script that can extract from a\ngiven query a list of html pages. Then clean these pages to the extent of\nhaving a data base of texts and a corpus of pair's question-texts. In addition,\nwe give preliminary results of our proposal method. Some investigations for the\nconstruction of Arabic corpus are also presented in this document.",
            "journal ref": "International Journal of Recent Contributions from Engineering,\n  Science \\& IT (iJES), kassel university press GmbH, 2016, 4 (2), pp.38-45",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09587v3",
            "date": "2017-09-27 15:46:07+00:00",
            "title": "Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment",
            "authors": [
                "Tal Baumel",
                "Jumana Nassour-Kassis",
                "Raphael Cohen",
                "Michael Elhadad",
                "No`emie Elhadad"
            ],
            "abstract": "In the context of the Electronic Health Record, automated diagnosis coding of\npatient notes is a useful task, but a challenging one due to the large number\nof codes and the length of patient notes. We investigate four models for\nassigning multiple ICD codes to discharge summaries taken from both MIMIC II\nand III. We present Hierarchical Attention-GRU (HA-GRU), a hierarchical\napproach to tag a document by identifying the sentences relevant for each\nlabel. HA-GRU achieves state-of-the art results. Furthermore, the learned\nsentence-level attention layer highlights the model decision process, allows\neasier error analysis, and suggests future directions for improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09816v1",
            "date": "2017-09-28 06:13:33+00:00",
            "title": "Edina: Building an Open Domain Socialbot with Self-dialogues",
            "authors": [
                "Ben Krause",
                "Marco Damonte",
                "Mihai Dobre",
                "Daniel Duma",
                "Joachim Fainberg",
                "Federico Fancellu",
                "Emmanuel Kahembwe",
                "Jianpeng Cheng",
                "Bonnie Webber"
            ],
            "abstract": "We present Edina, the University of Edinburgh's social bot for the Amazon\nAlexa Prize competition. Edina is a conversational agent whose responses\nutilize data harvested from Amazon Mechanical Turk (AMT) through an innovative\nnew technique we call self-dialogues. These are conversations in which a single\nAMT Worker plays both participants in a dialogue. Such dialogues are\nsurprisingly natural, efficient to collect and reflective of relevant and/or\ntrending topics. These self-dialogues provide training data for a generative\nneural network as well as a basis for soft rules used by a matching score\ncomponent. Each match of a soft rule against a user utterance is associated\nwith a confidence score which we show is strongly indicative of reply quality,\nallowing this component to self-censor and be effectively integrated with other\ncomponents. Edina's full architecture features a rule-based system backing off\nto a matching score, backing off to a generative neural network. Our hybrid\ndata-driven methodology thus addresses both coverage limitations of a strictly\nrule-based approach and the lack of guarantees of a strictly machine-learning\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10445v2",
            "date": "2017-09-29 15:10:20+00:00",
            "title": "Synonym Discovery with Etymology-based Word Embeddings",
            "authors": [
                "Seunghyun Yoon",
                "Pablo Estrada",
                "Kyomin Jung"
            ],
            "abstract": "We propose a novel approach to learn word embeddings based on an extended\nversion of the distributional hypothesis. Our model derives word embedding\nvectors using the etymological composition of words, rather than the context in\nwhich they appear. It has the strength of not requiring a large text corpus,\nbut instead it requires reliable access to etymological roots of words, making\nit specially fit for languages with logographic writing systems. The model\nconsists on three steps: (1) building an etymological graph, which is a\nbipartite network of words and etymological roots, (2) obtaining the\nbiadjacency matrix of the etymological graph and reducing its dimensionality,\n(3) using columns/rows of the resulting matrices as embedding vectors. We test\nour model in the Chinese and Sino-Korean vocabularies. Our graphs are formed by\na set of 117,000 Chinese words, and a set of 135,000 Sino-Korean words. In both\ncases we show that our model performs well in the task of synonym discovery.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00102v2",
            "date": "2017-12-30 08:54:16+00:00",
            "title": "Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "This paper presents a new deep learning architecture for Natural Language\nInference (NLI). Firstly, we introduce a new architecture where alignment pairs\nare compared, compressed and then propagated to upper layers for enhanced\nrepresentation learning. Secondly, we adopt factorization layers for efficient\nand expressive compression of alignment vectors into scalar features, which are\nthen used to augment the base word representations. The design of our approach\nis aimed to be conceptually simple, compact and yet powerful. We conduct\nexperiments on three popular benchmarks, SNLI, MultiNLI and SciTail, achieving\ncompetitive performance on all. A lightweight parameterization of our model\nalso enjoys a $\\approx 3$ times reduction in parameter size compared to the\nexisting state-of-the-art models, e.g., ESIM and DIIN, while maintaining\ncompetitive performance. Additionally, visual analysis shows that our\npropagated features are highly interpretable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00016v2",
            "date": "2017-03-31 18:05:44+00:00",
            "title": "Opinion Mining on Non-English Short Text",
            "authors": [
                "Esra Akbas"
            ],
            "abstract": "As the type and the number of such venues increase, automated analysis of\nsentiment on textual resources has become an essential data mining task. In\nthis paper, we investigate the problem of mining opinions on the collection of\ninformal short texts. Both positive and negative sentiment strength of texts\nare detected. We focus on a non-English language that has few resources for\ntext mining. This approach would help enhance the sentiment analysis in\nlanguages where a list of opinionated words does not exist. We propose a new\nmethod projects the text into dense and low dimensional feature vectors\naccording to the sentiment strength of the words. We detect the mixture of\npositive and negative sentiments on a multi-variant scale. Empirical evaluation\nof the proposed framework on Turkish tweets shows that our approach gets good\nresults for opinion mining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00135v2",
            "date": "2017-04-01 08:16:20+00:00",
            "title": "Topic modeling of public repositories at scale using names in source code",
            "authors": [
                "Vadim Markovtsev",
                "Eiso Kant"
            ],
            "abstract": "Programming languages themselves have a limited number of reserved keywords\nand character based tokens that define the language specification. However,\nprogrammers have a rich use of natural language within their code through\ncomments, text literals and naming entities. The programmer defined names that\ncan be found in source code are a rich source of information to build a high\nlevel understanding of the project. The goal of this paper is to apply topic\nmodeling to names used in over 13.6 million repositories and perceive the\ninferred topics. One of the problems in such a study is the occurrence of\nduplicate repositories not officially marked as forks (obscure forks). We show\nhow to address it using the same identifiers which are extracted for topic\nmodeling.\n  We open with a discussion on naming in source code, we then elaborate on our\napproach to remove exact duplicate and fuzzy duplicate repositories using\nLocality Sensitive Hashing on the bag-of-words model and then discuss our work\non topic modeling; and finally present the results from our data analysis\ntogether with open-access to the source code, tools and datasets.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00784v2",
            "date": "2017-04-03 19:45:27+00:00",
            "title": "Online and Linear-Time Attention by Enforcing Monotonic Alignments",
            "authors": [
                "Colin Raffel",
                "Minh-Thang Luong",
                "Peter J. Liu",
                "Ron J. Weiss",
                "Douglas Eck"
            ],
            "abstract": "Recurrent neural network models with an attention mechanism have proven to be\nextremely effective on a wide variety of sequence-to-sequence problems.\nHowever, the fact that soft attention mechanisms perform a pass over the entire\ninput sequence when producing each element in the output sequence precludes\ntheir use in online settings and results in a quadratic time complexity. Based\non the insight that the alignment between input and output sequence elements is\nmonotonic in many problems of interest, we propose an end-to-end differentiable\nmethod for learning monotonic alignments which, at test time, enables computing\nattention online and in linear time. We validate our approach on sentence\nsummarization, machine translation, and online speech recognition problems and\nachieve results competitive with existing sequence-to-sequence models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1704.00939v1",
            "date": "2017-04-04 10:01:47+00:00",
            "title": "Fortia-FBK at SemEval-2017 Task 5: Bullish or Bearish? Inferring Sentiment towards Brands from Financial News Headlines",
            "authors": [
                "Youness Mansar",
                "Lorenzo Gatti",
                "Sira Ferradans",
                "Marco Guerini",
                "Jacopo Staiano"
            ],
            "abstract": "In this paper, we describe a methodology to infer Bullish or Bearish\nsentiment towards companies/brands. More specifically, our approach leverages\naffective lexica and word embeddings in combination with convolutional neural\nnetworks to infer the sentiment of financial news headlines towards a target\ncompany. Such architecture was used and evaluated in the context of the SemEval\n2017 challenge (task 5, subtask 2), in which it obtained the best performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01074v4",
            "date": "2017-04-04 15:44:48+00:00",
            "title": "Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory",
            "authors": [
                "Hao Zhou",
                "Minlie Huang",
                "Tianyang Zhang",
                "Xiaoyan Zhu",
                "Bing Liu"
            ],
            "abstract": "Perception and expression of emotion are key factors to the success of\ndialogue systems or conversational agents. However, this problem has not been\nstudied in large-scale conversation generation so far. In this paper, we\npropose Emotional Chatting Machine (ECM) that can generate appropriate\nresponses not only in content (relevant and grammatical) but also in emotion\n(emotionally consistent). To the best of our knowledge, this is the first work\nthat addresses the emotion factor in large-scale conversation generation. ECM\naddresses the factor using three new mechanisms that respectively (1) models\nthe high-level abstraction of emotion expressions by embedding emotion\ncategories, (2) captures the change of implicit internal emotion states, and\n(3) uses explicit emotion expressions with an external emotion vocabulary.\nExperiments show that the proposed model can generate responses appropriate not\nonly in content but also in emotion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01599v1",
            "date": "2017-04-05 18:21:39+00:00",
            "title": "Rhetorical relations for information retrieval",
            "authors": [
                "Christina Lioma",
                "Birger Larsen",
                "Wei Lu"
            ],
            "abstract": "Typically, every part in most coherent text has some plausible reason for its\npresence, some function that it performs to the overall semantics of the text.\nRhetorical relations, e.g. contrast, cause, explanation, describe how the parts\nof a text are linked to each other. Knowledge about this socalled discourse\nstructure has been applied successfully to several natural language processing\ntasks. This work studies the use of rhetorical relations for Information\nRetrieval (IR): Is there a correlation between certain rhetorical relations and\nretrieval performance? Can knowledge about a document's rhetorical relations be\nuseful to IR? We present a language model modification that considers\nrhetorical relations when estimating the relevance of a document to a query.\nEmpirical evaluation of different versions of our model on TREC settings shows\nthat certain rhetorical relations can benefit retrieval effectiveness notably\n(> 10% in mean average precision over a state-of-the-art baseline).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.01631v2",
            "date": "2017-04-05 19:44:23+00:00",
            "title": "Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition",
            "authors": [
                "Shubham Toshniwal",
                "Hao Tang",
                "Liang Lu",
                "Karen Livescu"
            ],
            "abstract": "End-to-end training of deep learning-based models allows for implicit\nlearning of intermediate representations based on the final task loss. However,\nthe end-to-end approach ignores the useful domain knowledge encoded in explicit\nintermediate-level supervision. We hypothesize that using intermediate\nrepresentations as auxiliary supervision at lower levels of deep networks may\nbe a good way of combining the advantages of end-to-end training and more\ntraditional pipeline approaches. We present experiments on conversational\nspeech recognition where we use lower-level tasks, such as phoneme recognition,\nin a multitask training approach with an encoder-decoder model for direct\ncharacter transcription. We compare multiple types of lower-level tasks and\nanalyze the effects of the auxiliary tasks. Our results on the Switchboard\ncorpus show that this approach improves recognition accuracy over a standard\nencoder-decoder model on the Eval2000 test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01691v2",
            "date": "2017-04-06 02:36:56+00:00",
            "title": "Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction",
            "authors": [
                "Chunting Zhou",
                "Graham Neubig"
            ],
            "abstract": "Labeled sequence transduction is a task of transforming one sequence into\nanother sequence that satisfies desiderata specified by a set of labels. In\nthis paper we propose multi-space variational encoder-decoders, a new model for\nlabeled sequence transduction with semi-supervised learning. The generative\nmodel can use neural networks to handle both discrete and continuous latent\nvariables to exploit various features of data. Experiments show that our model\nprovides not only a powerful supervised framework but also can effectively take\nadvantage of the unlabeled data. On the SIGMORPHON morphological inflection\nbenchmark, our model outperforms single-model state-of-art results by a large\nmargin for the majority of languages.",
            "journal ref": "ACL 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02090v1",
            "date": "2017-04-07 05:12:38+00:00",
            "title": "Conceptualization Topic Modeling",
            "authors": [
                "Yi-Kun Tang",
                "Xian-Ling Mao",
                "Heyan Huang",
                "Guihua Wen"
            ],
            "abstract": "Recently, topic modeling has been widely used to discover the abstract topics\nin text corpora. Most of the existing topic models are based on the assumption\nof three-layer hierarchical Bayesian structure, i.e. each document is modeled\nas a probability distribution over topics, and each topic is a probability\ndistribution over words. However, the assumption is not optimal. Intuitively,\nit's more reasonable to assume that each topic is a probability distribution\nover concepts, and then each concept is a probability distribution over words,\ni.e. adding a latent concept layer between topic layer and word layer in\ntraditional three-layer assumption. In this paper, we verify the proposed\nassumption by incorporating the new assumption in two representative topic\nmodels, and obtain two novel topic models. Extensive experiments were conducted\namong the proposed models and corresponding baselines, and the results show\nthat the proposed models significantly outperform the baselines in terms of\ncase study and perplexity, which means the new assumption is more reasonable\nthan traditional one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02263v1",
            "date": "2017-04-07 15:27:10+00:00",
            "title": "NILC-USP at SemEval-2017 Task 4: A Multi-view Ensemble for Twitter Sentiment Analysis",
            "authors": [
                "Edilson A. Corr\u00eaa Jr.",
                "Vanessa Queiroz Marinho",
                "Leandro Borges dos Santos"
            ],
            "abstract": "This paper describes our multi-view ensemble approach to SemEval-2017 Task 4\non Sentiment Analysis in Twitter, specifically, the Message Polarity\nClassification subtask for English (subtask A). Our system is a voting\nensemble, where each base classifier is trained in a different feature space.\nThe first space is a bag-of-words model and has a Linear SVM as base\nclassifier. The second and third spaces are two different strategies of\ncombining word embeddings to represent sentences and use a Linear SVM and a\nLogistic Regressor as base classifiers. The proposed system was ranked 18th out\nof 38 systems considering F1 score and 20th considering recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02841v1",
            "date": "2017-04-04 12:06:51+00:00",
            "title": "From Modal to Multimodal Ambiguities: a Classification Approach",
            "authors": [
                "Maria Chiara Caschera",
                "Fernando Ferri",
                "Patrizia Grifoni"
            ],
            "abstract": "This paper deals with classifying ambiguities for Multimodal Languages. It\nevolves the classifications and the methods of the literature on ambiguities\nfor Natural Language and Visual Language, empirically defining an original\nclassification of ambiguities for multimodal interaction using a linguistic\nperspective. This classification distinguishes between Semantic and Syntactic\nmultimodal ambiguities and their subclasses, which are intercepted using a\nrule-based method implemented in a software module. The experimental results\nhave achieved an accuracy of the obtained classification compared to the\nexpected one, which are defined by the human judgment, of 94.6% for the\nsemantic ambiguities classes, and 92.1% for the syntactic ambiguities classes.",
            "journal ref": "JNIT (Journal of Next Generation Information Technology), Volume 4\n  Issue 5, July, 2013,Pages 87-109, ISSN 2092-8637. GlobalCIS (Convergence\n  Information Society, Republic of Korea)",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1704.02963v1",
            "date": "2017-04-10 17:37:22+00:00",
            "title": "Exploring Word Embeddings for Unsupervised Textual User-Generated Content Normalization",
            "authors": [
                "Thales Felipe Costa Bertaglia",
                "Maria das Gra\u00e7as Volpe Nunes"
            ],
            "abstract": "Text normalization techniques based on rules, lexicons or supervised training\nrequiring large corpora are not scalable nor domain interchangeable, and this\nmakes them unsuitable for normalizing user-generated content (UGC). Current\ntools available for Brazilian Portuguese make use of such techniques. In this\nwork we propose a technique based on distributed representation of words (or\nword embeddings). It generates continuous numeric vectors of\nhigh-dimensionality to represent words. The vectors explicitly encode many\nlinguistic regularities and patterns, as well as syntactic and semantic word\nrelationships. Words that share semantic similarity are represented by similar\nvectors. Based on these features, we present a totally unsupervised, expandable\nand language and domain independent method for learning normalization lexicons\nfrom word embeddings. Our approach obtains high correction rate of orthographic\nerrors and internet slang in product reviews, outperforming the current\navailable tools for Brazilian Portuguese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03407v2",
            "date": "2017-04-11 16:42:55+00:00",
            "title": "What we really want to find by Sentiment Analysis: The Relationship between Computational Models and Psychological State",
            "authors": [
                "Hwiyeol Jo",
                "Soo-Min Kim",
                "Jeong Ryu"
            ],
            "abstract": "As the first step to model emotional state of a person, we build sentiment\nanalysis models with existing deep neural network algorithms and compare the\nmodels with psychological measurements to enlighten the relationship. In the\nexperiments, we first examined psychological state of 64 participants and asked\nthem to summarize the story of a book, Chronicle of a Death Foretold (Marquez,\n1981). Secondly, we trained models using crawled 365,802 movie review data;\nthen we evaluated participants' summaries using the pretrained model as a\nconcept of transfer learning. With the background that emotion affects on\nmemories, we investigated the relationship between the evaluation score of the\nsummaries from computational models and the examined psychological\nmeasurements. The result shows that although CNN performed the best among other\ndeep neural network algorithms (LSTM, GRU), its results are not related to the\npsychological state. Rather, GRU shows more explainable results depending on\nthe psychological state. The contribution of this paper can be summarized as\nfollows: (1) we enlighten the relationship between computational models and\npsychological measurements. (2) we suggest this framework as objective methods\nto evaluate the emotion; the real sentiment analysis of a person.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03617v1",
            "date": "2017-04-12 04:38:18+00:00",
            "title": "Representation Stability as a Regularizer for Improved Text Analytics Transfer Learning",
            "authors": [
                "Matthew Riemer",
                "Elham Khabiri",
                "Richard Goodwin"
            ],
            "abstract": "Although neural networks are well suited for sequential transfer learning\ntasks, the catastrophic forgetting problem hinders proper integration of prior\nknowledge. In this work, we propose a solution to this problem by using a\nmulti-task objective based on the idea of distillation and a mechanism that\ndirectly penalizes forgetting at the shared representation layer during the\nknowledge integration phase of training. We demonstrate our approach on a\nTwitter domain sentiment analysis task with sequential knowledge transfer from\nfour related tasks. We show that our technique outperforms networks fine-tuned\nto the target task. Additionally, we show both through empirical evidence and\nexamples that it does not forget useful knowledge from the source task that is\nforgotten during standard fine-tuning. Surprisingly, we find that first\ndistilling a human made rule based sentiment engine into a recurrent neural\nnetwork and then integrating the knowledge with the target task data leads to a\nsubstantial gain in generalization performance. Our experiments demonstrate the\npower of multi-source transfer techniques in practical text analytics problems\nwhen paired with distillation. In particular, for the SemEval 2016 Task 4\nSubtask A (Nakov et al., 2016) dataset we surpass the state of the art\nestablished during the competition with a comparatively simple model\narchitecture that is not even competitive when trained on only the labeled task\nspecific data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03940v3",
            "date": "2017-04-12 21:56:59+00:00",
            "title": "PACRR: A Position-Aware Neural IR Model for Relevance Matching",
            "authors": [
                "Kai Hui",
                "Andrew Yates",
                "Klaus Berberich",
                "Gerard de Melo"
            ],
            "abstract": "In order to adopt deep learning for information retrieval, models are needed\nthat can capture all relevant information required to assess the relevance of a\ndocument to a given user query. While previous works have successfully captured\nunigram term matches, how to fully employ position-dependent information such\nas proximity and term dependencies has been insufficiently explored. In this\nwork, we propose a novel neural IR model named PACRR aiming at better modeling\nposition-dependent interactions between a query and a document. Extensive\nexperiments on six years' TREC Web Track data confirm that the proposed model\nyields better results under multiple benchmarks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.05091v1",
            "date": "2017-04-17 18:48:00+00:00",
            "title": "FEUP at SemEval-2017 Task 5: Predicting Sentiment Polarity and Intensity with Financial Word Embeddings",
            "authors": [
                "Pedro Saleiro",
                "Eduarda Mendes Rodrigues",
                "Carlos Soares",
                "Eug\u00e9nio Oliveira"
            ],
            "abstract": "This paper presents the approach developed at the Faculty of Engineering of\nUniversity of Porto, to participate in SemEval 2017, Task 5: Fine-grained\nSentiment Analysis on Financial Microblogs and News. The task consisted in\npredicting a real continuous variable from -1.0 to +1.0 representing the\npolarity and intensity of sentiment concerning companies/stocks mentioned in\nshort texts. We modeled the task as a regression analysis problem and combined\ntraditional techniques such as pre-processing short texts, bag-of-words\nrepresentations and lexical-based features with enhanced financial specific\nbag-of-embeddings. We used an external collection of tweets and news headlines\nmentioning companies/stocks from S\\&P 500 to create financial word embeddings\nwhich are able to capture domain-specific syntactic and semantic similarities.\nThe resulting approach obtained a cosine similarity score of 0.69 in sub-task\n5.1 - Microblogs and 0.68 in sub-task 5.2 - News Headlines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05119v2",
            "date": "2017-04-17 20:42:05+00:00",
            "title": "Exploring Sparsity in Recurrent Neural Networks",
            "authors": [
                "Sharan Narang",
                "Erich Elsen",
                "Gregory Diamos",
                "Shubho Sengupta"
            ],
            "abstract": "Recurrent Neural Networks (RNN) are widely used to solve a variety of\nproblems and as the quantity of data and the amount of available compute have\nincreased, so have model sizes. The number of parameters in recent\nstate-of-the-art networks makes them hard to deploy, especially on mobile\nphones and embedded devices. The challenge is due to both the size of the model\nand the time it takes to evaluate it. In order to deploy these RNNs\nefficiently, we propose a technique to reduce the parameters of a network by\npruning weights during the initial training of the network. At the end of\ntraining, the parameters of the network are sparse while accuracy is still\nclose to the original dense neural network. The network size is reduced by 8x\nand the time required to train the model remains constant. Additionally, we can\nprune a larger dense network to achieve better than baseline performance while\nstill reducing the total number of parameters significantly. Pruning RNNs\nreduces the size of the model and can also help achieve significant inference\ntime speed-up using sparse matrix multiply. Benchmarks show that using our\ntechnique model size can be reduced by 90% and speed-up is around 2x to 7x.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1704.05295v1",
            "date": "2017-04-18 12:24:26+00:00",
            "title": "Semantic Similarity from Natural Language and Ontology Analysis",
            "authors": [
                "S\u00e9bastien Harispe",
                "Sylvie Ranwez",
                "Stefan Janaqi",
                "Jacky Montmain"
            ],
            "abstract": "Artificial Intelligence federates numerous scientific fields in the aim of\ndeveloping machines able to assist human operators performing complex\ntreatments -- most of which demand high cognitive skills (e.g. learning or\ndecision processes). Central to this quest is to give machines the ability to\nestimate the likeness or similarity between things in the way human beings\nestimate the similarity between stimuli.\n  In this context, this book focuses on semantic measures: approaches designed\nfor comparing semantic entities such as units of language, e.g. words,\nsentences, or concepts and instances defined into knowledge bases. The aim of\nthese measures is to assess the similarity or relatedness of such semantic\nentities by taking into account their semantics, i.e. their meaning --\nintuitively, the words tea and coffee, which both refer to stimulating\nbeverage, will be estimated to be more semantically similar than the words\ntoffee (confection) and coffee, despite that the last pair has a higher\nsyntactic similarity. The two state-of-the-art approaches for estimating and\nquantifying semantic similarities/relatedness of semantic entities are\npresented in detail: the first one relies on corpora analysis and is based on\nNatural Language Processing techniques and semantic models while the second is\nbased on more or less formal, computer-readable and workable forms of knowledge\nsuch as semantic networks, thesaurus or ontologies. (...) Beyond a simple\ninventory and categorization of existing measures, the aim of this monograph is\nto convey novices as well as researchers of these domains towards a better\nunderstanding of semantic similarity estimation and more generally semantic\nmeasures.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1704.05550v1",
            "date": "2017-04-18 22:21:22+00:00",
            "title": "Extractive Summarization: Limits, Compression, Generalized Model and Heuristics",
            "authors": [
                "Rakesh Verma",
                "Daniel Lee"
            ],
            "abstract": "Due to its promise to alleviate information overload, text summarization has\nattracted the attention of many researchers. However, it has remained a serious\nchallenge. Here, we first prove empirical limits on the recall (and F1-scores)\nof extractive summarizers on the DUC datasets under ROUGE evaluation for both\nthe single-document and multi-document summarization tasks. Next we define the\nconcept of compressibility of a document and present a new model of\nsummarization, which generalizes existing models in the literature and\nintegrates several dimensions of the summarization, viz., abstractive versus\nextractive, single versus multi-document, and syntactic versus semantic.\nFinally, we examine some new and existing single-document summarization\nalgorithms in a single framework and compare with state of the art summarizers\non DUC data.",
            "journal ref": "Computaci\\'on y Sistemas 21(4) (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05572v1",
            "date": "2017-04-19 01:07:56+00:00",
            "title": "Answering Complex Questions Using Open Information Extraction",
            "authors": [
                "Tushar Khot",
                "Ashish Sabharwal",
                "Peter Clark"
            ],
            "abstract": "While there has been substantial progress in factoid question-answering (QA),\nanswering complex questions remains challenging, typically requiring both a\nlarge body of knowledge and inference techniques. Open Information Extraction\n(Open IE) provides a way to generate semi-structured knowledge for QA, but to\ndate such knowledge has only been used to answer simple questions with\nretrieval-based methods. We overcome this limitation by presenting a method for\nreasoning with Open IE knowledge, allowing more complex questions to be\nhandled. Using a recently proposed support graph optimization framework for QA,\nwe develop a new inference model for Open IE, in particular one that can work\neffectively with multiple short facts, noise, and the relational structure of\ntuples. Our model significantly outperforms a state-of-the-art structured\nsolver on complex questions of varying difficulty, while also removing the\nreliance on manually curated knowledge.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1704.05753v2",
            "date": "2017-04-19 14:41:21+00:00",
            "title": "Understanding Task Design Trade-offs in Crowdsourced Paraphrase Collection",
            "authors": [
                "Youxuan Jiang",
                "Jonathan K. Kummerfeld",
                "Walter S. Lasecki"
            ],
            "abstract": "Linguistically diverse datasets are critical for training and evaluating\nrobust machine learning systems, but data collection is a costly process that\noften requires experts. Crowdsourcing the process of paraphrase generation is\nan effective means of expanding natural language datasets, but there has been\nlimited analysis of the trade-offs that arise when designing tasks. In this\npaper, we present the first systematic study of the key factors in\ncrowdsourcing paraphrase collection. We consider variations in instructions,\nincentives, data domains, and workflows. We manually analyzed paraphrases for\ncorrectness, grammaticality, and linguistic diversity. Our observations provide\nnew insight into the trade-offs between accuracy and diversity in crowd\nresponses that arise as a result of task design, providing guidance for future\nparaphrase generation procedures.",
            "journal ref": "ACL (2017) 103-109",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05972v1",
            "date": "2017-04-20 01:21:20+00:00",
            "title": "SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours",
            "authors": [
                "Leon Derczynski",
                "Kalina Bontcheva",
                "Maria Liakata",
                "Rob Procter",
                "Geraldine Wong Sak Hoi",
                "Arkaitz Zubiaga"
            ],
            "abstract": "Media is full of false claims. Even Oxford Dictionaries named \"post-truth\" as\nthe word of 2016. This makes it more important than ever to build systems that\ncan identify the veracity of a story, and the kind of discourse there is around\nit. RumourEval is a SemEval shared task that aims to identify and handle\nrumours and reactions to them, in text. We present an annotation scheme, a\nlarge dataset covering multiple topics - each having their own families of\nclaims and replies - and use these to pose two concrete challenges as well as\nthe results achieved by participants on these challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05973v1",
            "date": "2017-04-20 01:22:57+00:00",
            "title": "Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection",
            "authors": [
                "Tong Chen",
                "Lin Wu",
                "Xue Li",
                "Jun Zhang",
                "Hongzhi Yin",
                "Yang Wang"
            ],
            "abstract": "The proliferation of social media in communication and information\ndissemination has made it an ideal platform for spreading rumors. Automatically\ndebunking rumors at their stage of diffusion is known as \\textit{early rumor\ndetection}, which refers to dealing with sequential posts regarding disputed\nfactual claims with certain variations and highly textual duplication over\ntime. Thus, identifying trending rumors demands an efficient yet flexible model\nthat is able to capture long-range dependencies among postings and produce\ndistinct representations for the accurate early detection. However, it is a\nchallenging task to apply conventional classification algorithms to rumor\ndetection in earliness since they rely on hand-crafted features which require\nintensive manual efforts in the case of large amount of posts. This paper\npresents a deep attention model on the basis of recurrent neural networks (RNN)\nto learn \\textit{selectively} temporal hidden representations of sequential\nposts for identifying rumors. The proposed model delves soft-attention into the\nrecurrence to simultaneously pool out distinct features with particular focus\nand produce hidden representations that capture contextual variations of\nrelevant posts over time. Extensive experiments on real datasets collected from\nsocial media websites demonstrate that (1) the deep attention based RNN model\noutperforms state-of-the-arts that rely on hand-crafted features; (2) the\nintroduction of soft attention mechanism can effectively distill relevant parts\nto rumors from original posts in advance; (3) the proposed method detects\nrumors more quickly and accurately than competitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06125v1",
            "date": "2017-04-20 13:10:25+00:00",
            "title": "BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs",
            "authors": [
                "Mathieu Cliche"
            ],
            "abstract": "In this paper we describe our attempt at producing a state-of-the-art Twitter\nsentiment classifier using Convolutional Neural Networks (CNNs) and Long Short\nTerm Memory (LSTMs) networks. Our system leverages a large amount of unlabeled\ndata to pre-train word embeddings. We then use a subset of the unlabeled data\nto fine tune the embeddings using distant supervision. The final CNNs and LSTMs\nare trained on the SemEval-2017 Twitter dataset where the embeddings are fined\ntuned again. To boost performances we ensemble several CNNs and LSTMs together.\nOur approach achieved first rank on all of the five English subtasks amongst 40\nteams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06259v2",
            "date": "2017-04-21 15:32:01+00:00",
            "title": "A Semantic QA-Based Approach for Text Summarization Evaluation",
            "authors": [
                "Ping Chen",
                "Fei Wu",
                "Tong Wang",
                "Wei Ding"
            ],
            "abstract": "Many Natural Language Processing and Computational Linguistics applications\ninvolves the generation of new texts based on some existing texts, such as\nsummarization, text simplification and machine translation. However, there has\nbeen a serious problem haunting these applications for decades, that is, how to\nautomatically and accurately assess quality of these applications. In this\npaper, we will present some preliminary results on one especially useful and\nchallenging problem in NLP system evaluation: how to pinpoint content\ndifferences of two text passages (especially for large pas-sages such as\narticles and books). Our idea is intuitive and very different from existing\napproaches. We treat one text passage as a small knowledge base, and ask it a\nlarge number of questions to exhaustively identify all content points in it. By\ncomparing the correctly answered questions from two text passages, we will be\nable to compare their content precisely. The experiment using 2007 DUC\nsummarization corpus clearly shows promising results.",
            "journal ref": "AAAI 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06358v3",
            "date": "2017-04-20 22:28:14+00:00",
            "title": "Stability and Fluctuations in a Simple Model of Phonetic Category Change",
            "authors": [
                "Benjamin Goodman",
                "Paul Tupper"
            ],
            "abstract": "In spoken languages, speakers divide up the space of phonetic possibilities\ninto different regions, corresponding to different phonemes. We consider a\nsimple exemplar model of how this division of phonetic space varies over time\namong a population of language users. In the particular model we consider, we\nshow that, once the system is initialized with a given set of phonemes, that\nphonemes do not become extinct: all phonemes will be maintained in the system\nfor all time. This is in contrast to what is observed in more complex models.\nFurthermore, we show that the boundaries between phonemes fluctuate and we\nquantitatively study the fluctuations in a simple instance of our model. These\nresults prepare the ground for more sophisticated models in which some phonemes\ngo extinct or new phonemes emerge through other processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06485v2",
            "date": "2017-04-21 11:29:07+00:00",
            "title": "Attend to You: Personalized Image Captioning with Context Sequence Memory Networks",
            "authors": [
                "Cesc Chunseong Park",
                "Byeongchang Kim",
                "Gunhee Kim"
            ],
            "abstract": "We address personalization issues of image captioning, which have not been\ndiscussed yet in previous research. For a query image, we aim to generate a\ndescriptive sentence, accounting for prior knowledge such as the user's active\nvocabularies in previous documents. As applications of personalized image\ncaptioning, we tackle two post automation tasks: hashtag prediction and post\ngeneration, on our newly collected Instagram dataset, consisting of 1.1M posts\nfrom 6.3K users. We propose a novel captioning model named Context Sequence\nMemory Network (CSMN). Its unique updates over previous memory network models\ninclude (i) exploiting memory as a repository for multiple types of context\ninformation, (ii) appending previously generated words into memory to capture\nlong-term information without suffering from the vanishing gradient problem,\nand (iii) adopting CNN memory structure to jointly represent nearby ordered\nmemory slots for better context understanding. With quantitative evaluation and\nuser studies via Amazon Mechanical Turk, we show the effectiveness of the three\nnovel features of CSMN and its performance enhancement for personalized image\ncaptioning over state-of-the-art captioning models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1704.06567v1",
            "date": "2017-04-21 14:39:27+00:00",
            "title": "Attention Strategies for Multi-Source Sequence-to-Sequence Learning",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Jind\u0159ich Helcl"
            ],
            "abstract": "Modeling attention in neural multi-source sequence-to-sequence learning\nremains a relatively unexplored area, despite its usefulness in tasks that\nincorporate multiple source languages or modalities. We propose two novel\napproaches to combine the outputs of attention mechanisms over each source\nsequence, flat and hierarchical. We compare the proposed methods with existing\ntechniques and present results of systematic evaluation of those methods on the\nWMT16 Multimodal Translation and Automatic Post-editing tasks. We show that the\nproposed methods achieve competitive results on both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06619v1",
            "date": "2017-04-21 16:17:58+00:00",
            "title": "Scientific Article Summarization Using Citation-Context and Article's Discourse Structure",
            "authors": [
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "We propose a summarization approach for scientific articles which takes\nadvantage of citation-context and the document discourse model. While citations\nhave been previously used in generating scientific summaries, they lack the\nrelated context from the referenced article and therefore do not accurately\nreflect the article's content. Our method overcomes the problem of\ninconsistency between the citation summary and the article's content by\nproviding context for each citation. We also leverage the inherent scientific\narticle's discourse for producing better summaries. We show that our proposed\nmethod effectively improves over existing summarization approaches (greater\nthan 30% improvement over the best performing baseline) in terms of\n\\textsc{Rouge} scores on TAC2014 scientific summarization dataset. While the\ndataset we use for evaluation is in the biomedical domain, most of our\napproaches are general and therefore adaptable to other domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06877v2",
            "date": "2017-04-23 03:54:22+00:00",
            "title": "Learning to Skim Text",
            "authors": [
                "Adams Wei Yu",
                "Hongrae Lee",
                "Quoc V. Le"
            ],
            "abstract": "Recurrent Neural Networks are showing much promise in many sub-areas of\nnatural language processing, ranging from document classification to machine\ntranslation to automatic question answering. Despite their promise, many\nrecurrent models have to read the whole text word by word, making it slow to\nhandle long documents. For example, it is difficult to use a recurrent network\nto read a book and answer questions about it. In this paper, we present an\napproach of reading text while skipping irrelevant information if needed. The\nunderlying model is a recurrent network that learns how far to jump after\nreading a few words of the input text. We employ a standard policy gradient\nmethod to train the model to make discrete jumping decisions. In our benchmarks\non four different tasks, including number prediction, sentiment analysis, news\narticle classification and automatic Q\\&A, our proposed model, a modified LSTM\nwith jumping, is up to 6 times faster than the standard sequential LSTM, while\nmaintaining the same or even better accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06913v2",
            "date": "2017-04-23 11:27:53+00:00",
            "title": "Learning weakly supervised multimodal phoneme embeddings",
            "authors": [
                "Rahma Chaabouni",
                "Ewan Dunbar",
                "Neil Zeghidour",
                "Emmanuel Dupoux"
            ],
            "abstract": "Recent works have explored deep architectures for learning multimodal speech\nrepresentation (e.g. audio and images, articulation and audio) in a supervised\nway. Here we investigate the role of combining different speech modalities,\ni.e. audio and visual information representing the lips movements, in a weakly\nsupervised way using Siamese networks and lexical same-different side\ninformation. In particular, we ask whether one modality can benefit from the\nother to provide a richer representation for phone recognition in a weakly\nsupervised setting. We introduce mono-task and multi-task methods for merging\nspeech and visual modalities for phone recognition. The mono-task learning\nconsists in applying a Siamese network on the concatenation of the two\nmodalities, while the multi-task learning receives several different\ncombinations of modalities at train time. We show that multi-task learning\nenhances discriminability for visual and multimodal inputs while minimally\nimpacting auditory inputs. Furthermore, we present a qualitative analysis of\nthe obtained phone embeddings, and show that cross-modal visual input can\nimprove the discriminability of phonological features which are visually\ndiscernable (rounding, open/close, labial place of articulation), resulting in\nrepresentations that are closer to abstract linguistic features than those\nbased on audio only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06960v5",
            "date": "2017-04-23 18:46:42+00:00",
            "title": "Translating Neuralese",
            "authors": [
                "Jacob Andreas",
                "Anca Dragan",
                "Dan Klein"
            ],
            "abstract": "Several approaches have recently been proposed for learning decentralized\ndeep multiagent policies that coordinate via a differentiable communication\nchannel. While these policies are effective for many tasks, interpretation of\ntheir induced communication strategies has remained a challenge. Here we\npropose to interpret agents' messages by translating them. Unlike in typical\nmachine translation problems, we have no parallel data to learn from. Instead\nwe develop a translation model based on the insight that agent messages and\nnatural language strings mean the same thing if they induce the same belief\nabout the world in a listener. We present theoretical guarantees and empirical\nevidence that our approach preserves both the semantics and pragmatics of\nmessages by ensuring that players communicating through a translation layer do\nnot suffer a substantial loss in reward relative to players with a common\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07129v1",
            "date": "2017-04-24 10:38:23+00:00",
            "title": "An Analysis of Action Recognition Datasets for Language and Vision Tasks",
            "authors": [
                "Spandana Gella",
                "Frank Keller"
            ],
            "abstract": "A large amount of recent research has focused on tasks that combine language\nand vision, resulting in a proliferation of datasets and methods. One such task\nis action recognition, whose applications include image annotation, scene\nunder- standing and image retrieval. In this survey, we categorize the existing\nap- proaches based on how they conceptualize this problem and provide a\ndetailed review of existing datasets, highlighting their di- versity as well as\nadvantages and disad- vantages. We focus on recently devel- oped datasets which\nlink visual informa- tion with linguistic resources and provide a fine-grained\nsyntactic and semantic anal- ysis of actions in images.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07221v1",
            "date": "2017-04-24 13:41:25+00:00",
            "title": "Turing at SemEval-2017 Task 8: Sequential Approach to Rumour Stance Classification with Branch-LSTM",
            "authors": [
                "Elena Kochkina",
                "Maria Liakata",
                "Isabelle Augenstein"
            ],
            "abstract": "This paper describes team Turing's submission to SemEval 2017 RumourEval:\nDetermining rumour veracity and support for rumours (SemEval 2017 Task 8,\nSubtask A). Subtask A addresses the challenge of rumour stance classification,\nwhich involves identifying the attitude of Twitter users towards the\ntruthfulness of the rumour they are discussing. Stance classification is\nconsidered to be an important step towards rumour verification, therefore\nperforming well in this task is expected to be useful in debunking false\nrumours. In this work we classify a set of Twitter posts discussing rumours\ninto either supporting, denying, questioning or commenting on the underlying\nrumours. We propose a LSTM-based sequential model that, through modelling the\nconversational structure of tweets, which achieves an accuracy of 0.784 on the\nRumourEval test set outperforming all other systems in Subtask A.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07759v1",
            "date": "2017-04-25 16:07:34+00:00",
            "title": "Email Babel: Does Language Affect Criminal Activity in Compromised Webmail Accounts?",
            "authors": [
                "Emeric Bernard-Jones",
                "Jeremiah Onaolapo",
                "Gianluca Stringhini"
            ],
            "abstract": "We set out to understand the effects of differing language on the ability of\ncybercriminals to navigate webmail accounts and locate sensitive information in\nthem. To this end, we configured thirty Gmail honeypot accounts with English,\nRomanian, and Greek language settings. We populated the accounts with email\nmessages in those languages by subscribing them to selected online newsletters.\nWe hid email messages about fake bank accounts in fifteen of the accounts to\nmimic real-world webmail users that sometimes store sensitive information in\ntheir accounts. We then leaked credentials to the honey accounts via paste\nsites on the Surface Web and the Dark Web, and collected data for fifteen days.\nOur statistical analyses on the data show that cybercriminals are more likely\nto discover sensitive information (bank account information) in the Greek\naccounts than the remaining accounts, contrary to the expectation that Greek\nought to constitute a barrier to the understanding of non-Greek visitors to the\nGreek accounts. We also extracted the important words among the emails that\ncybercriminals accessed (as an approximation of the keywords that they searched\nfor within the honey accounts), and found that financial terms featured among\nthe top words. In summary, we show that language plays a significant role in\nthe ability of cybercriminals to access sensitive information hidden in\ncompromised webmail accounts.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1704.08619v1",
            "date": "2017-04-27 15:14:33+00:00",
            "title": "End-to-End Multimodal Emotion Recognition using Deep Neural Networks",
            "authors": [
                "Panagiotis Tzirakis",
                "George Trigeorgis",
                "Mihalis A. Nicolaou",
                "Bj\u00f6rn Schuller",
                "Stefanos Zafeiriou"
            ],
            "abstract": "Automatic affect recognition is a challenging task due to the various\nmodalities emotions can be expressed with. Applications can be found in many\ndomains including multimedia retrieval and human computer interaction. In\nrecent years, deep neural networks have been used with great success in\ndetermining emotional states. Inspired by this success, we propose an emotion\nrecognition system using auditory and visual modalities. To capture the\nemotional content for various styles of speaking, robust features need to be\nextracted. To this purpose, we utilize a Convolutional Neural Network (CNN) to\nextract features from the speech, while for the visual modality a deep residual\nnetwork (ResNet) of 50 layers. In addition to the importance of feature\nextraction, a machine learning algorithm needs also to be insensitive to\noutliers while being able to model the context. To tackle this problem, Long\nShort-Term Memory (LSTM) networks are utilized. The system is then trained in\nan end-to-end fashion where - by also taking advantage of the correlations of\nthe each of the streams - we manage to significantly outperform the traditional\napproaches based on auditory and visual handcrafted features for the prediction\nof spontaneous and natural emotions on the RECOLA database of the AVEC 2016\nresearch challenge on emotion recognition.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1704.08966v2",
            "date": "2017-04-28 14:57:29+00:00",
            "title": "Not All Dialogues are Created Equal: Instance Weighting for Neural Conversational Models",
            "authors": [
                "Pierre Lison",
                "Serge Bibauw"
            ],
            "abstract": "Neural conversational models require substantial amounts of dialogue data for\ntheir parameter estimation and are therefore usually learned on large corpora\nsuch as chat forums or movie subtitles. These corpora are, however, often\nchallenging to work with, notably due to their frequent lack of turn\nsegmentation and the presence of multiple references external to the dialogue\nitself. This paper shows that these challenges can be mitigated by adding a\nweighting model into the architecture. The weighting model, which is itself\nestimated from dialogue data, associates each training example to a numerical\nweight that reflects its intrinsic quality for dialogue modelling. At training\ntime, these sample weights are included into the empirical loss to be\nminimised. Evaluation results on retrieval-based models trained on movie and TV\nsubtitles demonstrate that the inclusion of such a weighting model improves the\nmodel performance on unsupervised metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00188v1",
            "date": "2017-06-01 07:25:22+00:00",
            "title": "Deep Learning for Hate Speech Detection in Tweets",
            "authors": [
                "Pinkesh Badjatiya",
                "Shashank Gupta",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Hate speech detection on Twitter is critical for applications like\ncontroversial event extraction, building AI chatterbots, content\nrecommendation, and sentiment analysis. We define this task as being able to\nclassify a tweet as racist, sexist or neither. The complexity of the natural\nlanguage constructs makes this task very challenging. We perform extensive\nexperiments with multiple deep learning architectures to learn semantic word\nembeddings to handle this complexity. Our experiments on a benchmark dataset of\n16K annotated tweets show that such deep learning methods outperform\nstate-of-the-art char/word n-gram methods by ~18 F1 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00286v3",
            "date": "2017-06-01 13:12:15+00:00",
            "title": "Learning to Compute Word Embeddings On the Fly",
            "authors": [
                "Dzmitry Bahdanau",
                "Tom Bosc",
                "Stanis\u0142aw Jastrz\u0119bski",
                "Edward Grefenstette",
                "Pascal Vincent",
                "Yoshua Bengio"
            ],
            "abstract": "Words in natural language follow a Zipfian distribution whereby some words\nare frequent but most are rare. Learning representations for words in the \"long\ntail\" of this distribution requires enormous amounts of data. Representations\nof rare words trained directly on end tasks are usually poor, requiring us to\npre-train embeddings on external data, or treat all rare words as\nout-of-vocabulary words with a unique representation. We provide a method for\npredicting embeddings of rare words on the fly from small amounts of auxiliary\ndata with a network trained end-to-end for the downstream task. We show that\nthis improves results against baselines where embeddings are trained on the end\ntask for reading comprehension, recognizing textual entailment and language\nmodeling.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.01038v2",
            "date": "2017-06-04 06:57:09+00:00",
            "title": "Improving Legal Information Retrieval by Distributional Composition with Term Order Probabilities",
            "authors": [
                "Danilo S. Carvalho",
                "Duc-Vu Tran",
                "Van-Khanh Tran",
                "Le-Nguyen Minh"
            ],
            "abstract": "Legal professionals worldwide are currently trying to get up-to-pace with the\nexplosive growth in legal document availability through digital means. This\ndrives a need for high efficiency Legal Information Retrieval (IR) and Question\nAnswering (QA) methods. The IR task in particular has a set of unique\nchallenges that invite the use of semantic motivated NLP techniques. In this\nwork, a two-stage method for Legal Information Retrieval is proposed, combining\nlexical statistics and distributional sentence representations in the context\nof Competition on Legal Information Extraction/Entailment (COLIEE). The\ncombination is done with the use of disambiguation rules, applied over the\nrankings obtained through n-gram statistics. After the ranking is done, its\nresults are evaluated for ambiguity, and disambiguation is done if a result is\ndecided to be unreliable for a given query. Competition and experimental\nresults indicate small gains in overall retrieval performance using the\nproposed approach. Additionally, an analysis of error and improvement cases is\npresented for a better understanding of the contributions.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.01427v1",
            "date": "2017-06-05 17:17:18+00:00",
            "title": "A simple neural network module for relational reasoning",
            "authors": [
                "Adam Santoro",
                "David Raposo",
                "David G. T. Barrett",
                "Mateusz Malinowski",
                "Razvan Pascanu",
                "Peter Battaglia",
                "Timothy Lillicrap"
            ],
            "abstract": "Relational reasoning is a central component of generally intelligent\nbehavior, but has proven difficult for neural networks to learn. In this paper\nwe describe how to use Relation Networks (RNs) as a simple plug-and-play module\nto solve problems that fundamentally hinge on relational reasoning. We tested\nRN-augmented networks on three tasks: visual question answering using a\nchallenging dataset called CLEVR, on which we achieve state-of-the-art,\nsuper-human performance; text-based question answering using the bAbI suite of\ntasks; and complex reasoning about dynamic physical systems. Then, using a\ncurated dataset called Sort-of-CLEVR we show that powerful convolutional\nnetworks do not have a general capacity to solve relational questions, but can\ngain this capacity when augmented with RNs. Our work shows how a deep learning\narchitecture equipped with an RN module can implicitly discover and learn to\nreason about entities and their relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01758v1",
            "date": "2017-05-26 08:03:10+00:00",
            "title": "A WL-SPPIM Semantic Model for Document Classification",
            "authors": [
                "Ming Li",
                "Peilun Xiao",
                "Ju Zhang"
            ],
            "abstract": "In this paper, we explore SPPIM-based text classification method, and the\nexperiment reveals that the SPPIM method is equal to or even superior than SGNS\nmethod in text classification task on three international and standard text\ndatasets, namely 20newsgroups, Reuters52 and WebKB. Comparing to SGNS, although\nSPPMI provides a better solution, it is not necessarily better than SGNS in\ntext classification tasks. Based on our analysis, SGNS takes into the\nconsideration of weight calculation during decomposition process, so it has\nbetter performance than SPPIM in some standard datasets. Inspired by this, we\npropose a WL-SPPIM semantic model based on SPPIM model, and experiment shows\nthat WL-SPPIM approach has better classification and higher scalability in the\ntext classification task compared with LDA, SGNS and SPPIM approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01863v2",
            "date": "2017-06-06 17:25:36+00:00",
            "title": "Marmara Turkish Coreference Corpus and Coreference Resolution Baseline",
            "authors": [
                "Peter Sch\u00fcller",
                "K\u00fcbra C\u0131ng\u0131ll\u0131",
                "Ferit Tun\u00e7er",
                "Bar\u0131\u015f G\u00fcn S\u00fcrmeli",
                "Ay\u015feg\u00fcl Pekel",
                "Ay\u015fe Hande Karatay",
                "Hacer Ezgi Karaka\u015f"
            ],
            "abstract": "We describe the Marmara Turkish Coreference Corpus, which is an annotation of\nthe whole METU-Sabanci Turkish Treebank with mentions and coreference chains.\nCollecting eight or more independent annotations for each document allowed for\nfully automatic adjudication. We provide a baseline system for Turkish mention\ndetection and coreference resolution and evaluate it on the corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02141v3",
            "date": "2017-06-07 12:03:07+00:00",
            "title": "How Important is Syntactic Parsing Accuracy? An Empirical Evaluation on Rule-Based Sentiment Analysis",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Iago Alonso-Alonso",
                "David Vilares"
            ],
            "abstract": "Syntactic parsing, the process of obtaining the internal structure of\nsentences in natural languages, is a crucial task for artificial intelligence\napplications that need to extract meaning from natural language text or speech.\nSentiment analysis is one example of application for which parsing has recently\nproven useful.\n  In recent years, there have been significant advances in the accuracy of\nparsing algorithms. In this article, we perform an empirical, task-oriented\nevaluation to determine how parsing accuracy influences the performance of a\nstate-of-the-art rule-based sentiment analysis system that determines the\npolarity of sentences from their parse trees. In particular, we evaluate the\nsystem using four well-known dependency parsers, including both current models\nwith state-of-the-art accuracy and more innacurate models which, however,\nrequire less computational resources.\n  The experiments show that all of the parsers produce similarly good results\nin the sentiment analysis task, without their accuracy having any relevant\ninfluence on the results. Since parsing is currently a task with a relatively\nhigh computational cost that varies strongly between algorithms, this suggests\nthat sentiment analysis researchers and users should prioritize speed over\naccuracy when choosing a parser; and parsing researchers should investigate\nmodels that improve speed further, even at some cost to accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02256v2",
            "date": "2017-06-07 16:58:59+00:00",
            "title": "A Mention-Ranking Model for Abstract Anaphora Resolution",
            "authors": [
                "Ana Marasovi\u0107",
                "Leo Born",
                "Juri Opitz",
                "Anette Frank"
            ],
            "abstract": "Resolving abstract anaphora is an important, but difficult task for text\nunderstanding. Yet, with recent advances in representation learning this task\nbecomes a more tangible aim. A central property of abstract anaphora is that it\nestablishes a relation between the anaphor embedded in the anaphoric sentence\nand its (typically non-nominal) antecedent. We propose a mention-ranking model\nthat learns how abstract anaphors relate to their antecedents with an\nLSTM-Siamese Net. We overcome the lack of training data by generating\nartificial anaphoric sentence--antecedent pairs. Our model outperforms\nstate-of-the-art results on shell noun resolution. We also report first\nbenchmark results on an abstract anaphora subset of the ARRAU corpus. This\ncorpus presents a greater challenge due to a mixture of nominal and pronominal\nanaphors and a greater range of confounders. We found model variants that\noutperform the baselines for nominal anaphors, without training on individual\nanaphor data, but still lag behind for pronominal anaphors. Our model selects\nsyntactically plausible candidates and -- if disregarding syntax --\ndiscriminates candidates using deeper features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03059v2",
            "date": "2017-06-09 17:59:16+00:00",
            "title": "Depthwise Separable Convolutions for Neural Machine Translation",
            "authors": [
                "Lukasz Kaiser",
                "Aidan N. Gomez",
                "Francois Chollet"
            ],
            "abstract": "Depthwise separable convolutions reduce the number of parameters and\ncomputation used in convolutional operations while increasing representational\nefficiency. They have been shown to be successful in image classification\nmodels, both in obtaining better models than previously possible for a given\nparameter count (the Xception architecture) and considerably reducing the\nnumber of parameters required to perform at a given level (the MobileNets\nfamily of architectures). Recently, convolutional sequence-to-sequence networks\nhave been applied to machine translation tasks with good results. In this work,\nwe study how depthwise separable convolutions can be applied to neural machine\ntranslation. We introduce a new architecture inspired by Xception and ByteNet,\ncalled SliceNet, which enables a significant reduction of the parameter count\nand amount of computation needed to obtain results like ByteNet, and, with a\nsimilar parameter count, achieves new state-of-the-art results. In addition to\nshowing that depthwise separable convolutions perform well for machine\ntranslation, we investigate the architectural changes that they enable: we\nobserve that thanks to depthwise separability, we can increase the length of\nconvolution windows, removing the need for filter dilation. We also introduce a\nnew \"super-separable\" convolution operation that further reduces the number of\nparameters and computational cost for obtaining state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03196v1",
            "date": "2017-06-10 07:41:22+00:00",
            "title": "Online Learning for Neural Machine Translation Post-editing",
            "authors": [
                "\u00c1lvaro Peris",
                "Luis Cebri\u00e1n",
                "Francisco Casacuberta"
            ],
            "abstract": "Neural machine translation has meant a revolution of the field. Nevertheless,\npost-editing the outputs of the system is mandatory for tasks requiring high\ntranslation quality. Post-editing offers a unique opportunity for improving\nneural machine translation systems, using online learning techniques and\ntreating the post-edited translations as new, fresh training data. We review\nclassical learning methods and propose a new optimization algorithm. We\nthoroughly compare online learning algorithms in a post-editing scenario.\nResults show significant improvements in translation quality and effort\nreduction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.03357v1",
            "date": "2017-06-11 14:37:22+00:00",
            "title": "Generic Axiomatization of Families of Noncrossing Graphs in Dependency Parsing",
            "authors": [
                "Anssi Yli-Jyr\u00e4",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present a simple encoding for unlabeled noncrossing graphs and show how\nits latent counterpart helps us to represent several families of directed and\nundirected graphs used in syntactic and semantic parsing of natural language as\ncontext-free languages. The families are separated purely on the basis of\nforbidden patterns in latent encoding, eliminating the need to differentiate\nthe families of non-crossing graphs in inference algorithms: one algorithm\nworks for all when the search space can be controlled in parser input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03449v1",
            "date": "2017-06-12 03:21:38+00:00",
            "title": "Scientific document summarization via citation contextualization and scientific discourse",
            "authors": [
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "The rapid growth of scientific literature has made it difficult for the\nresearchers to quickly learn about the developments in their respective fields.\nScientific document summarization addresses this challenge by providing\nsummaries of the important contributions of scientific papers. We present a\nframework for scientific summarization which takes advantage of the citations\nand the scientific discourse structure. Citation texts often lack the evidence\nand context to support the content of the cited paper and are even sometimes\ninaccurate. We first address the problem of inaccuracy of the citation texts by\nfinding the relevant context from the cited paper. We propose three approaches\nfor contextualizing citations which are based on query reformulation, word\nembeddings, and supervised learning. We then train a model to identify the\ndiscourse facets for each citation. We finally propose a method for summarizing\nscientific papers by leveraging the faceted citations and their corresponding\ncontexts. We evaluate our proposed method on two scientific summarization\ndatasets in the biomedical and computational linguistics domains. Extensive\nevaluation results show that our methods can improve over the state of the art\nby large margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03762v5",
            "date": "2017-06-12 17:57:34+00:00",
            "title": "Attention Is All You Need",
            "authors": [
                "Ashish Vaswani",
                "Noam Shazeer",
                "Niki Parmar",
                "Jakob Uszkoreit",
                "Llion Jones",
                "Aidan N. Gomez",
                "Lukasz Kaiser",
                "Illia Polosukhin"
            ],
            "abstract": "The dominant sequence transduction models are based on complex recurrent or\nconvolutional neural networks in an encoder-decoder configuration. The best\nperforming models also connect the encoder and decoder through an attention\nmechanism. We propose a new simple network architecture, the Transformer, based\nsolely on attention mechanisms, dispensing with recurrence and convolutions\nentirely. Experiments on two machine translation tasks show these models to be\nsuperior in quality while being more parallelizable and requiring significantly\nless time to train. Our model achieves 28.4 BLEU on the WMT 2014\nEnglish-to-German translation task, improving over the existing best results,\nincluding ensembles by over 2 BLEU. On the WMT 2014 English-to-French\ntranslation task, our model establishes a new single-model state-of-the-art\nBLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction\nof the training costs of the best models from the literature. We show that the\nTransformer generalizes well to other tasks by applying it successfully to\nEnglish constituency parsing both with large and limited training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04206v2",
            "date": "2017-06-13 18:02:27+00:00",
            "title": "Identifying Condition-Action Statements in Medical Guidelines Using Domain-Independent Features",
            "authors": [
                "Hossein Hematialam",
                "Wlodek Zadrozny"
            ],
            "abstract": "This paper advances the state of the art in text understanding of medical\nguidelines by releasing two new annotated clinical guidelines datasets, and\nestablishing baselines for using machine learning to extract condition-action\npairs. In contrast to prior work that relies on manually created rules, we\nreport experiment with several supervised machine learning techniques to\nclassify sentences as to whether they express conditions and actions. We show\nthe limitations and possible extensions of this work on text mining of medical\nguidelines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04326v1",
            "date": "2017-06-14 05:53:51+00:00",
            "title": "Transfer Learning for Neural Semantic Parsing",
            "authors": [
                "Xing Fan",
                "Emilio Monti",
                "Lambert Mathias",
                "Markus Dreyer"
            ],
            "abstract": "The goal of semantic parsing is to map natural language to a machine\ninterpretable meaning representation language (MRL). One of the constraints\nthat limits full exploration of deep learning technologies for semantic parsing\nis the lack of sufficient annotation training data. In this paper, we propose\nusing sequence-to-sequence in a multi-task setup for semantic parsing with a\nfocus on transfer learning. We explore three multi-task architectures for\nsequence-to-sequence modeling and compare their performance with an\nindependently trained model. Our experiments show that the multi-task setup\naids transfer learning from an auxiliary task with large labeled data to a\ntarget task with smaller labeled data. We see absolute accuracy gains ranging\nfrom 1.0% to 4.4% in our in- house data set, and we also see good gains ranging\nfrom 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and\nsemantic auxiliary tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04872v1",
            "date": "2017-06-15 14:01:40+00:00",
            "title": "Towards a theory of word order. Comment on \"Dependency distance: a new perspective on syntactic patterns in natural language\" by Haitao Liu et al",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "Comment on \"Dependency distance: a new perspective on syntactic patterns in\nnatural language\" by Haitao Liu et al",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04902v4",
            "date": "2017-06-15 14:46:56+00:00",
            "title": "A Survey Of Cross-lingual Word Embedding Models",
            "authors": [
                "Sebastian Ruder",
                "Ivan Vuli\u0107",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Cross-lingual representations of words enable us to reason about word meaning\nin multilingual contexts and are a key facilitator of cross-lingual transfer\nwhen developing natural language processing models for low-resource languages.\nIn this survey, we provide a comprehensive typology of cross-lingual word\nembedding models. We compare their data requirements and objective functions.\nThe recurring theme of the survey is that many of the models presented in the\nliterature optimize for the same objectives, and that seemingly different\nmodels are often equivalent modulo optimization strategies, hyper-parameters,\nand such. We also discuss the different ways cross-lingual word embeddings are\nevaluated, as well as future challenges and research horizons.",
            "journal ref": "JAIR 65 (2019) 569-631",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04922v2",
            "date": "2017-06-15 15:24:32+00:00",
            "title": "DSRIM: A Deep Neural Information Retrieval Model Enhanced by a Knowledge Resource Driven Representation of Documents",
            "authors": [
                "Gia-Hung Nguyen",
                "Laure Soulier",
                "Lynda Tamine",
                "Nathalie Bricon-Souf"
            ],
            "abstract": "The state-of-the-art solutions to the vocabulary mismatch in information\nretrieval (IR) mainly aim at leveraging either the relational semantics\nprovided by external resources or the distributional semantics, recently\ninvestigated by deep neural approaches. Guided by the intuition that the\nrelational semantics might improve the effectiveness of deep neural approaches,\nwe propose the Deep Semantic Resource Inference Model (DSRIM) that relies on:\n1) a representation of raw-data that models the relational semantics of text by\njointly considering objects and relations expressed in a knowledge resource,\nand 2) an end-to-end neural architecture that learns the query-document\nrelevance by leveraging the distributional and relational semantics of\ndocuments and queries. The experimental evaluation carried out on two TREC\ndatasets from TREC Terabyte and TREC CDS tracks relying respectively on WordNet\nand MeSH resources, indicates that our model outperforms state-of-the-art\nsemantic and deep neural IR models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.05087v2",
            "date": "2017-06-13 23:11:04+00:00",
            "title": "Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder",
            "authors": [
                "Caglar Gulcehre",
                "Francis Dutil",
                "Adam Trischler",
                "Yoshua Bengio"
            ],
            "abstract": "We investigate the integration of a planning mechanism into an\nencoder-decoder architecture with an explicit alignment for character-level\nmachine translation. We develop a model that plans ahead when it computes\nalignments between the source and target sequences, constructing a matrix of\nproposed future alignments and a commitment vector that governs whether to\nfollow or recompute the plan. This mechanism is inspired by the strategic\nattentive reader and writer (STRAW) model. Our proposed model is end-to-end\ntrainable with fully differentiable operations. We show that it outperforms a\nstrong baseline on three character-level decoder neural machine translation on\nWMT'15 corpus. Our analysis demonstrates that our model can compute\nqualitatively intuitive alignments and achieves superior performance with fewer\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05089v1",
            "date": "2017-06-14 13:00:08+00:00",
            "title": "Number game",
            "authors": [
                "Go Sugimoto"
            ],
            "abstract": "CLARIN (Common Language Resources and Technology Infrastructure) is regarded\nas one of the most important European research infrastructures, offering and\npromoting a wide array of useful services for (digital) research in linguistics\nand humanities. However, the assessment of the users for its core technical\ndevelopment has been highly limited, therefore, it is unclear if the community\nis thoroughly aware of the status-quo of the growing infrastructure. In\naddition, CLARIN does not seem to be fully materialised marketing and business\nplans and strategies despite its strong technical assets. This article analyses\nthe web traffic of the Virtual Language Observatory, one of the main web\napplications of CLARIN and a symbol of pan-European re-search cooperation, to\nevaluate the users and performance of the service in a transparent and\nscientific way. It is envisaged that the paper can raise awareness of the\npressing issues on objective and transparent operation of the infrastructure\nthough Open Evaluation, and the synergy between marketing and technical\ndevelopment. It also investigates the \"science of web analytics\" in an attempt\nto document the research process for the purpose of reusability and\nreproducibility, thus to find universal lessons for the use of a web analytics,\nrather than to merely produce a statistical report of a particular website\nwhich loses its value outside its context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05125v1",
            "date": "2017-06-16 01:26:09+00:00",
            "title": "Deal or No Deal? End-to-End Learning for Negotiation Dialogues",
            "authors": [
                "Mike Lewis",
                "Denis Yarats",
                "Yann N. Dauphin",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "Much of human dialogue occurs in semi-cooperative settings, where agents with\ndifferent goals attempt to agree on common decisions. Negotiations require\ncomplex communication and reasoning skills, but success is easy to measure,\nmaking this an interesting task for AI. We gather a large dataset of\nhuman-human negotiations on a multi-issue bargaining task, where agents who\ncannot observe each other's reward functions must reach an agreement (or a\ndeal) via natural language dialogue. For the first time, we show it is possible\nto train end-to-end models for negotiation, which must learn both linguistic\nand reasoning skills with no annotated dialogue states. We also introduce\ndialogue rollouts, in which the model plans ahead by simulating possible\ncomplete continuations of the conversation, and find that this technique\ndramatically improves performance. Our code and dataset are publicly available\n(https://github.com/facebookresearch/end-to-end-negotiator).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1706.05349v4",
            "date": "2017-06-16 17:07:24+00:00",
            "title": "Active learning in annotating micro-blogs dealing with e-reputation",
            "authors": [
                "Jean-Val\u00e8re Cossu",
                "Alejandro Molina-Villegas",
                "Mariana Tello-Signoret"
            ],
            "abstract": "Elections unleash strong political views on Twitter, but what do people\nreally think about politics? Opinion and trend mining on micro blogs dealing\nwith politics has recently attracted researchers in several fields including\nInformation Retrieval and Machine Learning (ML). Since the performance of ML\nand Natural Language Processing (NLP) approaches are limited by the amount and\nquality of data available, one promising alternative for some tasks is the\nautomatic propagation of expert annotations. This paper intends to develop a\nso-called active learning process for automatically annotating French language\ntweets that deal with the image (i.e., representation, web reputation) of\npoliticians. Our main focus is on the methodology followed to build an original\nannotated dataset expressing opinion from two French politicians over time. We\ntherefore review state of the art NLP-based ML algorithms to automatically\nannotate tweets using a manual initiation step as bootstrap. This paper focuses\non key issues about active learning while building a large annotated data set\nfrom noise. This will be introduced by human annotators, abundance of data and\nthe label distribution across data and entities. In turn, we show that Twitter\ncharacteristics such as the author's name or hashtags can be considered as the\nbearing point to not only improve automatic systems for Opinion Mining (OM) and\nTopic Classification but also to reduce noise in human annotations. However, a\nlater thorough analysis shows that reducing noise might induce the loss of\ncrucial information.",
            "journal ref": "Journal of Interdisciplinary Methodologies and Issues in Sciences,\n  Digital Contextualization (October 3, 2017) jimis:3970",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1706.05549v1",
            "date": "2017-06-17 15:06:58+00:00",
            "title": "A Large-Scale CNN Ensemble for Medication Safety Analysis",
            "authors": [
                "Liliya Akhtyamova",
                "Andrey Ignatov",
                "John Cardiff"
            ],
            "abstract": "Revealing Adverse Drug Reactions (ADR) is an essential part of post-marketing\ndrug surveillance, and data from health-related forums and medical communities\ncan be of a great significance for estimating such effects. In this paper, we\npropose an end-to-end CNN-based method for predicting drug safety on user\ncomments from healthcare discussion forums. We present an architecture that is\nbased on a vast ensemble of CNNs with varied structural parameters, where the\nprediction is determined by the majority vote. To evaluate the performance of\nthe proposed solution, we present a large-scale dataset collected from a\nmedical website that consists of over 50 thousand reviews for more than 4000\ndrugs. The results demonstrate that our model significantly outperforms\nconventional approaches and predicts medicine safety with an accuracy of 87.17%\nfor binary and 62.88% for multi-classification tasks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.05565v8",
            "date": "2017-06-17 17:36:23+00:00",
            "title": "Towards Neural Phrase-based Machine Translation",
            "authors": [
                "Po-Sen Huang",
                "Chong Wang",
                "Sitao Huang",
                "Dengyong Zhou",
                "Li Deng"
            ],
            "abstract": "In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our\nmethod explicitly models the phrase structures in output sequences using\nSleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence\nmodeling method. To mitigate the monotonic alignment requirement of SWAN, we\nintroduce a new layer to perform (soft) local reordering of input sequences.\nDifferent from existing neural machine translation (NMT) approaches, NPMT does\nnot use attention-based decoding mechanisms. Instead, it directly outputs\nphrases in a sequential order and can decode in linear time. Our experiments\nshow that NPMT achieves superior performances on IWSLT 2014\nGerman-English/English-German and IWSLT 2015 English-Vietnamese machine\ntranslation tasks compared with strong NMT baselines. We also observe that our\nmethod produces meaningful phrases in output languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05656v6",
            "date": "2017-06-18 14:04:09+00:00",
            "title": "Lexical representation explains cortical entrainment during speech comprehension",
            "authors": [
                "Stefan Frank",
                "Jinbiao Yang"
            ],
            "abstract": "Results from a recent neuroimaging study on spoken sentence comprehension\nhave been interpreted as evidence for cortical entrainment to hierarchical\nsyntactic structure. We present a simple computational model that predicts the\npower spectra from this study, even though the model's linguistic knowledge is\nrestricted to the lexical level, and word-level representations are not\ncombined into higher-level units (phrases or sentences). Hence, the cortical\nentrainment results can also be explained from the lexical properties of the\nstimuli, without recourse to hierarchical syntax.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1706.05719v1",
            "date": "2017-06-18 20:29:15+00:00",
            "title": "Towards the Improvement of Automated Scientific Document Categorization by Deep Learning",
            "authors": [
                "Thomas Krause"
            ],
            "abstract": "This master thesis describes an algorithm for automated categorization of\nscientific documents using deep learning techniques and compares the results to\nthe results of existing classification algorithms. As an additional goal a\nreusable API is to be developed allowing the automation of classification tasks\nin existing software. A design will be proposed using a convolutional neural\nnetwork as a classifier and integrating this into a REST based API. This is\nthen used as the basis for an actual proof of concept implementation presented\nas well in this thesis. It will be shown that the deep learning classifier\nprovides very good result in the context of multi-class document categorization\nand that it is feasible to integrate such classifiers into a larger ecosystem\nusing REST based services.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.06210v2",
            "date": "2017-06-19 23:15:22+00:00",
            "title": "Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning",
            "authors": [
                "Pawe\u0142 Budzianowski",
                "Stefan Ultes",
                "Pei-Hao Su",
                "Nikola Mrk\u0161i\u0107",
                "Tsung-Hsien Wen",
                "I\u00f1igo Casanueva",
                "Lina Rojas-Barahona",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Human conversation is inherently complex, often spanning many different\ntopics/domains. This makes policy learning for dialogue systems very\nchallenging. Standard flat reinforcement learning methods do not provide an\nefficient framework for modelling such dialogues. In this paper, we focus on\nthe under-explored problem of multi-domain dialogue management. First, we\npropose a new method for hierarchical reinforcement learning using the option\nframework. Next, we show that the proposed architecture learns faster and\narrives at a better policy than the existing flat ones do. Moreover, we show\nhow pretrained policies can be adapted to more complex systems with an\nadditional set of new actions. In doing that, we show that our approach has the\npotential to facilitate policy optimisation for more sophisticated multi-domain\ndialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06613v1",
            "date": "2017-06-20 18:19:54+00:00",
            "title": "End-to-End Neural Ad-hoc Ranking with Kernel Pooling",
            "authors": [
                "Chenyan Xiong",
                "Zhuyun Dai",
                "Jamie Callan",
                "Zhiyuan Liu",
                "Russell Power"
            ],
            "abstract": "This paper proposes K-NRM, a kernel based neural model for document ranking.\nGiven a query and a set of documents, K-NRM uses a translation matrix that\nmodels word-level similarities via word embeddings, a new kernel-pooling\ntechnique that uses kernels to extract multi-level soft match features, and a\nlearning-to-rank layer that combines those features into the final ranking\nscore. The whole model is trained end-to-end. The ranking layer learns desired\nfeature patterns from the pairwise ranking loss. The kernels transfer the\nfeature patterns into soft-match targets at each similarity level and enforce\nthem on the translation matrix. The word embeddings are tuned accordingly so\nthat they can produce the desired soft matches. Experiments on a commercial\nsearch engine's query log demonstrate the improvements of K-NRM over prior\nfeature-based and neural-based states-of-the-art, and explain the source of\nK-NRM's advantage: Its kernel-guided embedding encodes a similarity metric\ntailored for matching query words to document words, and provides effective\nmulti-level soft matches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.06681v3",
            "date": "2017-06-20 22:12:14+00:00",
            "title": "Graph-based Neural Multi-Document Summarization",
            "authors": [
                "Michihiro Yasunaga",
                "Rui Zhang",
                "Kshitijh Meelu",
                "Ayush Pareek",
                "Krishnan Srinivasan",
                "Dragomir Radev"
            ],
            "abstract": "We propose a neural multi-document summarization (MDS) system that\nincorporates sentence relation graphs. We employ a Graph Convolutional Network\n(GCN) on the relation graphs, with sentence embeddings obtained from Recurrent\nNeural Networks as input node features. Through multiple layer-wise\npropagation, the GCN generates high-level hidden sentence features for salience\nestimation. We then use a greedy heuristic to extract salient sentences while\navoiding redundancy. In our experiments on DUC 2004, we consider three types of\nsentence relation graphs and demonstrate the advantage of combining sentence\nrelations in graphs with the representation power of deep neural networks. Our\nmodel improves upon traditional graph-based extractive approaches and the\nvanilla GRU sequence model with no graph, and it achieves competitive results\nagainst other state-of-the-art multi-document summarization systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06714v3",
            "date": "2017-06-21 01:07:02+00:00",
            "title": "Neural-based Natural Language Generation in Dialogue using RNN Encoder-Decoder with Semantic Aggregation",
            "authors": [
                "Van-Khanh Tran",
                "Le-Minh Nguyen"
            ],
            "abstract": "Natural language generation (NLG) is an important component in spoken\ndialogue systems. This paper presents a model called Encoder-Aggregator-Decoder\nwhich is an extension of an Recurrent Neural Network based Encoder-Decoder\narchitecture. The proposed Semantic Aggregator consists of two components: an\nAligner and a Refiner. The Aligner is a conventional attention calculated over\nthe encoded input information, while the Refiner is another attention or gating\nmechanism stacked over the attentive Aligner in order to further select and\naggregate the semantic elements. The proposed model can be jointly trained both\nsentence planning and surface realization to produce natural language\nutterances. The model was extensively assessed on four different NLG domains,\nin which the experimental results showed that the proposed generator\nconsistently outperforms the previous methods on all the NLG domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06987v1",
            "date": "2017-06-21 16:19:59+00:00",
            "title": "A Generative Model of Group Conversation",
            "authors": [
                "Hannah Morrison",
                "Chris Martens"
            ],
            "abstract": "Conversations with non-player characters (NPCs) in games are typically\nconfined to dialogue between a human player and a virtual agent, where the\nconversation is initiated and controlled by the player. To create richer, more\nbelievable environments for players, we need conversational behavior to reflect\ninitiative on the part of the NPCs, including conversations that include\nmultiple NPCs who interact with one another as well as the player. We describe\na generative computational model of group conversation between agents, an\nabstract simulation of discussion in a small group setting. We define\nconversational interactions in terms of rules for turn taking and interruption,\nas well as belief change, sentiment change, and emotional response, all of\nwhich are dependent on agent personality, context, and relationships. We\nevaluate our model using a parameterized expressive range analysis, observing\ncorrelations between simulation parameters and features of the resulting\nconversations. This analysis confirms, for example, that character\npersonalities will predict how often they speak, and that heterogeneous groups\nof characters will generate more belief change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06996v2",
            "date": "2017-06-21 16:37:54+00:00",
            "title": "Statistical Inferences for Polarity Identification in Natural Language",
            "authors": [
                "Nicolas Pr\u00f6llochs",
                "Stefan Feuerriegel",
                "Dirk Neumann"
            ],
            "abstract": "Information forms the basis for all human behavior, including the ubiquitous\ndecision-making that people constantly perform in their every day lives. It is\nthus the mission of researchers to understand how humans process information to\nreach decisions. In order to facilitate this task, this work proposes a novel\nmethod of studying the reception of granular expressions in natural language.\nThe approach utilizes LASSO regularization as a statistical tool to extract\ndecisive words from textual content and draw statistical inferences based on\nthe correspondence between the occurrences of words and an exogenous response\nvariable. Accordingly, the method immediately suggests significant implications\nfor social sciences and Information Systems research: everyone can now identify\ntext segments and word choices that are statistically relevant to authors or\nreaders and, based on this knowledge, test hypotheses from behavioral research.\nWe demonstrate the contribution of our method by examining how authors\ncommunicate subjective information through narrative materials. This allows us\nto answer the question of which words to choose when communicating negative\ninformation. On the other hand, we show that investors trade not only upon\nfacts in financial disclosures but are distracted by filler words and\nnon-informative language. Practitioners - for example those in the fields of\ninvestor communications or marketing - can exploit our insights to enhance\ntheir writings based on the true perception of word choice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07179v2",
            "date": "2017-06-22 06:59:07+00:00",
            "title": "RelNet: End-to-End Modeling of Entities & Relations",
            "authors": [
                "Trapit Bansal",
                "Arvind Neelakantan",
                "Andrew McCallum"
            ],
            "abstract": "We introduce RelNet: a new model for relational reasoning. RelNet is a memory\naugmented neural network which models entities as abstract memory slots and is\nequipped with an additional relational memory which models relations between\nall memory pairs. The model thus builds an abstract knowledge graph on the\nentities and relations present in a document which can then be used to answer\nquestions about the document. It is trained end-to-end: only supervision to the\nmodel is in the form of correct answers to the questions. We test the model on\nthe 20 bAbI question-answering tasks with 10k examples per task and find that\nit solves all the tasks with a mean error of 0.3%, achieving 0% error on 11 of\nthe 20 tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07503v3",
            "date": "2017-06-22 22:09:14+00:00",
            "title": "Personalization in Goal-Oriented Dialog",
            "authors": [
                "Chaitanya K. Joshi",
                "Fei Mi",
                "Boi Faltings"
            ],
            "abstract": "The main goal of modeling human conversation is to create agents which can\ninteract with people in both open-ended and goal-oriented scenarios. End-to-end\ntrained neural dialog systems are an important line of research for such\ngeneralized dialog models as they do not resort to any situation-specific\nhandcrafting of rules. However, incorporating personalization into such systems\nis a largely unexplored topic as there are no existing corpora to facilitate\nsuch work. In this paper, we present a new dataset of goal-oriented dialogs\nwhich are influenced by speaker profiles attached to them. We analyze the\nshortcomings of an existing end-to-end dialog system based on Memory Networks\nand propose modifications to the architecture which enable personalization. We\nalso investigate personalization in dialog as a multi-task learning problem,\nand show that a single model which shares features among various profiles\noutperforms separate models for each profile.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07859v1",
            "date": "2017-06-22 04:33:59+00:00",
            "title": "Deep Speaker Verification: Do We Need End to End?",
            "authors": [
                "Dong Wang",
                "Lantian Li",
                "Zhiyuan Tang",
                "Thomas Fang Zheng"
            ],
            "abstract": "End-to-end learning treats the entire system as a whole adaptable black box,\nwhich, if sufficient data are available, may learn a system that works very\nwell for the target task. This principle has recently been applied to several\nprototype research on speaker verification (SV), where the feature learning and\nclassifier are learned together with an objective function that is consistent\nwith the evaluation metric. An opposite approach to end-to-end is feature\nlearning, which firstly trains a feature learning model, and then constructs a\nback-end classifier separately to perform SV. Recently, both approaches\nachieved significant performance gains on SV, mainly attributed to the smart\nutilization of deep neural networks. However, the two approaches have not been\ncarefully compared, and their respective advantages have not been well\ndiscussed. In this paper, we compare the end-to-end and feature learning\napproaches on a text-independent SV task. Our experiments on a dataset sampled\nfrom the Fisher database and involving 5,000 speakers demonstrated that the\nfeature learning approach outperformed the end-to-end approach. This is a\nstrong support for the feature learning approach, at least with data and\ncomputation resources similar to ours.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1706.07860v1",
            "date": "2017-06-22 04:26:39+00:00",
            "title": "Speaker Recognition with Cough, Laugh and \"Wei\"",
            "authors": [
                "Miao Zhang",
                "Yixiang Chen",
                "Lantian Li",
                "Dong Wang"
            ],
            "abstract": "This paper proposes a speaker recognition (SRE) task with trivial speech\nevents, such as cough and laugh. These trivial events are ubiquitous in\nconversations and less subjected to intentional change, therefore offering\nvaluable particularities to discover the genuine speaker from disguised speech.\nHowever, trivial events are often short and idiocratic in spectral patterns,\nmaking SRE extremely difficult. Fortunately, we found a very powerful deep\nfeature learning structure that can extract highly speaker-sensitive features.\nBy employing this tool, we studied the SRE performance on three types of\ntrivial events: cough, laugh and \"Wei\" (a short Chinese \"Hello\"). The results\nshow that there is rich speaker information within these trivial events, even\nfor cough that is intuitively less speaker distinguishable. With the deep\nfeature approach, the EER can reach 10%-14% with the three trivial events,\ndespite their extremely short durations (0.2-1.0 seconds).",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1706.07861v1",
            "date": "2017-06-22 04:32:40+00:00",
            "title": "Cross-lingual Speaker Verification with Deep Feature Learning",
            "authors": [
                "Lantian Li",
                "Dong Wang",
                "Askar Rozi",
                "Thomas Fang Zheng"
            ],
            "abstract": "Existing speaker verification (SV) systems often suffer from performance\ndegradation if there is any language mismatch between model training, speaker\nenrollment, and test. A major cause of this degradation is that most existing\nSV methods rely on a probabilistic model to infer the speaker factor, so any\nsignificant change on the distribution of the speech signal will impact the\ninference. Recently, we proposed a deep learning model that can learn how to\nextract the speaker factor by a deep neural network (DNN). By this feature\nlearning, an SV system can be constructed with a very simple back-end model. In\nthis paper, we investigate the robustness of the feature-based SV system in\nsituations with language mismatch. Our experiments were conducted on a complex\ncross-lingual scenario, where the model training was in English, and the\nenrollment and test were in Chinese or Uyghur. The experiments demonstrated\nthat the feature-based system outperformed the i-vector system with a large\nmargin, particularly with language mismatch between enrollment and test.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1706.07912v1",
            "date": "2017-06-24 06:04:21+00:00",
            "title": "Cluster Based Symbolic Representation for Skewed Text Categorization",
            "authors": [
                "Lavanya Narayana Raju",
                "Mahamad Suhil",
                "D S Guru",
                "Harsha S Gowda"
            ],
            "abstract": "In this work, a problem associated with imbalanced text corpora is addressed.\nA method of converting an imbalanced text corpus into a balanced one is\npresented. The presented method employs a clustering algorithm for conversion.\nInitially to avoid curse of dimensionality, an effective representation scheme\nbased on term class relevancy measure is adapted, which drastically reduces the\ndimension to the number of classes in the corpus. Subsequently, the samples of\nlarger sized classes are grouped into a number of subclasses of smaller sizes\nto make the entire corpus balanced. Each subclass is then given a single\nsymbolic vector representation by the use of interval valued features. This\nsymbolic representation in addition to being compact helps in reducing the\nspace requirement and also the classification time. The proposed model has been\nempirically demonstrated for its superiority on bench marking datasets viz.,\nReuters 21578 and TDT2. Further, it has been compared against several other\nexisting contemporary models including model based on support vector machine.\nThe comparative analysis indicates that the proposed model outperforms the\nother existing models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.08160v1",
            "date": "2017-06-25 20:00:54+00:00",
            "title": "Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context",
            "authors": [
                "Shyam Upadhyay",
                "Kai-Wei Chang",
                "Matt Taddy",
                "Adam Kalai",
                "James Zou"
            ],
            "abstract": "Word embeddings, which represent a word as a point in a vector space, have\nbecome ubiquitous to several NLP tasks. A recent line of work uses bilingual\n(two languages) corpora to learn a different vector for each sense of a word,\nby exploiting crosslingual signals to aid sense identification. We present a\nmulti-view Bayesian non-parametric algorithm which improves multi-sense word\nembeddings by (a) using multilingual (i.e., more than two languages) corpora to\nsignificantly improve sense embeddings beyond what one achieves with bilingual\ninformation, and (b) uses a principled approach to learn a variable number of\nsenses per word, in a data-driven manner. Ours is the first approach with the\nability to leverage multilingual corpora efficiently for multi-sense\nrepresentation learning. Experiments show that multilingual training\nsignificantly improves performance over monolingual and bilingual training, by\nallowing us to combine different parallel corpora to leverage multilingual\ncontext. Multilingual training yields comparable performance to a state of the\nart mono-lingual model trained on five times more training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08476v1",
            "date": "2017-06-26 16:52:42+00:00",
            "title": "Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability",
            "authors": [
                "Tiancheng Zhao",
                "Allen Lu",
                "Kyusong Lee",
                "Maxine Eskenazi"
            ],
            "abstract": "Generative encoder-decoder models offer great promise in developing\ndomain-general dialog systems. However, they have mainly been applied to\nopen-domain conversations. This paper presents a practical and novel framework\nfor building task-oriented dialog systems based on encoder-decoder models. This\nframework enables encoder-decoder models to accomplish slot-value independent\ndecision-making and interact with external databases. Moreover, this paper\nshows the flexibility of the proposed method by interleaving chatting\ncapability with a slot-filling system for better out-of-domain recovery. The\nmodels were trained on both real-user data from a bus information system and\nhuman-human chat data. Results show that the proposed framework achieves good\nperformance in both offline evaluation metrics and in task success rate with\nhuman users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08609v2",
            "date": "2017-06-26 21:34:29+00:00",
            "title": "The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics",
            "authors": [
                "Artemy Kolchinsky",
                "Nakul Dhande",
                "Kengjeun Park",
                "Yong-Yeol Ahn"
            ],
            "abstract": "We investigate the association between musical chords and lyrics by analyzing\na large dataset of user-contributed guitar tablatures. Motivated by the idea\nthat the emotional content of chords is reflected in the words used in\ncorresponding lyrics, we analyze associations between lyrics and chord\ncategories. We also examine the usage patterns of chords and lyrics in\ndifferent musical genres, historical eras, and geographical regions. Our\noverall results confirms a previously known association between Major chords\nand positive valence. We also report a wide variation in this association\nacross regions, genres, and eras. Our results suggest possible existence of\ndifferent emotional associations for other types of chords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08746v2",
            "date": "2017-06-27 09:23:37+00:00",
            "title": "DE-PACRR: Exploring Layers Inside the PACRR Model",
            "authors": [
                "Andrew Yates",
                "Kai Hui"
            ],
            "abstract": "Recent neural IR models have demonstrated deep learning's utility in ad-hoc\ninformation retrieval. However, deep models have a reputation for being black\nboxes, and the roles of a neural IR model's components may not be obvious at\nfirst glance. In this work, we attempt to shed light on the inner workings of a\nrecently proposed neural IR model, namely the PACRR model, by visualizing the\noutput of intermediate layers and by investigating the relationship between\nintermediate weights and the ultimate relevance score produced. We highlight\nseveral insights, hoping that such insights will be generally applicable.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.09055v1",
            "date": "2017-06-27 21:28:31+00:00",
            "title": "Acoustic Modeling Using a Shallow CNN-HTSVM Architecture",
            "authors": [
                "Christopher Dane Shulby",
                "Martha Dais Ferreira",
                "Rodrigo F. de Mello",
                "Sandra Maria Aluisio"
            ],
            "abstract": "High-accuracy speech recognition is especially challenging when large\ndatasets are not available. It is possible to bridge this gap with careful and\nknowledge-driven parsing combined with the biologically inspired CNN and the\nlearning guarantees of the Vapnik Chervonenkis (VC) theory. This work presents\na Shallow-CNN-HTSVM (Hierarchical Tree Support Vector Machine classifier)\narchitecture which uses a predefined knowledge-based set of rules with\nstatistical machine learning techniques. Here we show that gross errors present\neven in state-of-the-art systems can be avoided and that an accurate acoustic\nmodel can be built in a hierarchical fashion. The CNN-HTSVM acoustic model\noutperforms traditional GMM-HMM models and the HTSVM structure outperforms a\nMLP multi-class classifier. More importantly we isolate the performance of the\nacoustic model and provide results on both the frame and phoneme level\nconsidering the true robustness of the model. We show that even with a small\namount of data accurate and robust recognition rates can be obtained.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1706.10192v3",
            "date": "2017-06-30 13:39:03+00:00",
            "title": "Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval",
            "authors": [
                "Kai Hui",
                "Andrew Yates",
                "Klaus Berberich",
                "Gerard de Melo"
            ],
            "abstract": "Neural IR models, such as DRMM and PACRR, have achieved strong results by\nsuccessfully capturing relevance matching signals. We argue that the context of\nthese matching signals is also important. Intuitively, when extracting,\nmodeling, and combining matching signals, one would like to consider the\nsurrounding text (local context) as well as other signals from the same\ndocument that can contribute to the overall relevance score. In this work, we\nhighlight three potential shortcomings caused by not considering context\ninformation and propose three neural ingredients to address them: a\ndisambiguation component, cascade k-max pooling, and a shuffling combination\nlayer. Incorporating these components into the PACRR model yields Co-PACRR, a\nnovel context-aware neural IR model. Extensive comparisons with established\nmodels on Trec Web Track data confirm that the proposed model can achieve\nsuperior search results. In addition, an ablation analysis is conducted to gain\ninsights into the impact of and interactions between different components. We\nrelease our code to enable future comparisons.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.00214v1",
            "date": "2017-08-01 09:13:44+00:00",
            "title": "Natural Language Processing with Small Feed-Forward Networks",
            "authors": [
                "Jan A. Botha",
                "Emily Pitler",
                "Ji Ma",
                "Anton Bakalov",
                "Alex Salcianu",
                "David Weiss",
                "Ryan McDonald",
                "Slav Petrov"
            ],
            "abstract": "We show that small and shallow feed-forward neural networks can achieve near\nstate-of-the-art results on a range of unstructured and structured language\nprocessing tasks while being considerably cheaper in memory and computational\nrequirements than deep recurrent models. Motivated by resource-constrained\nenvironments like mobile phones, we showcase simple techniques for obtaining\nsuch small neural network models, and investigate different tradeoffs when\ndeciding how to allocate a small memory budget.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00481v1",
            "date": "2017-08-01 19:27:02+00:00",
            "title": "A Lightweight Front-end Tool for Interactive Entity Population",
            "authors": [
                "Hidekazu Oiwa",
                "Yoshihiko Suhara",
                "Jiyu Komiya",
                "Andrei Lopatenko"
            ],
            "abstract": "Entity population, a task of collecting entities that belong to a particular\ncategory, has attracted attention from vertical domains. There is still a high\ndemand for creating entity dictionaries in vertical domains, which are not\ncovered by existing knowledge bases. We develop a lightweight front-end tool\nfor facilitating interactive entity population. We implement key components\nnecessary for effective interactive entity population: 1) GUI-based dashboards\nto quickly modify an entity dictionary, and 2) entity highlighting on documents\nfor quickly viewing the current progress. We aim to reduce user cost from\nbeginning to end, including package installation and maintenance. The\nimplementation enables users to use this tool on their web browsers without any\nadditional packages --- users can focus on their missions to create entity\ndictionaries. Moreover, an entity expansion module is implemented as external\nAPIs. This design makes it easy to continuously improve interactive entity\npopulation pipelines. We are making our demo publicly available\n(http://bit.ly/luwak-demo).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00549v1",
            "date": "2017-08-01 23:05:57+00:00",
            "title": "Improved Representation Learning for Predicting Commonsense Ontologies",
            "authors": [
                "Xiang Li",
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "Recent work in learning ontologies (hierarchical and partially-ordered\nstructures) has leveraged the intrinsic geometry of spaces of learned\nrepresentations to make predictions that automatically obey complex structural\nconstraints. We explore two extensions of one such model, the order-embedding\nmodel for hierarchical relation learning, with an aim towards improved\nperformance on text data for commonsense knowledge representation. Our first\nmodel jointly learns ordering relations and non-hierarchical knowledge in the\nform of raw text. Our second extension exploits the partial order structure of\nthe training data to find long-distance triplet constraints among embeddings\nwhich are poorly enforced by the pairwise training procedure. We find that both\nincorporating free text and augmented training constraints improve over the\noriginal order-embedding model and other strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00625v1",
            "date": "2017-08-02 07:47:14+00:00",
            "title": "Deep Recurrent Generative Decoder for Abstractive Text Summarization",
            "authors": [
                "Piji Li",
                "Wai Lam",
                "Lidong Bing",
                "Zihao Wang"
            ],
            "abstract": "We propose a new framework for abstractive text summarization based on a\nsequence-to-sequence oriented encoder-decoder model equipped with a deep\nrecurrent generative decoder (DRGN).\n  Latent structure information implied in the target summaries is learned based\non a recurrent latent random model for improving the summarization quality.\n  Neural variational inference is employed to address the intractable posterior\ninference for the recurrent latent variables.\n  Abstractive summaries are generated based on both the generative latent\nvariables and the discriminative deterministic states.\n  Extensive experiments on some benchmark datasets in different languages show\nthat DRGN achieves improvements over the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00667v1",
            "date": "2017-08-02 09:40:42+00:00",
            "title": "Deep Reinforcement Learning for Inquiry Dialog Policies with Logical Formula Embeddings",
            "authors": [
                "Takuya Hiraoka",
                "Masaaki Tsuchida",
                "Yotaro Watanabe"
            ],
            "abstract": "This paper is the first attempt to learn the policy of an inquiry dialog\nsystem (IDS) by using deep reinforcement learning (DRL). Most IDS frameworks\nrepresent dialog states and dialog acts with logical formulae. In order to make\nlearning inquiry dialog policies more effective, we introduce a logical formula\nembedding framework based on a recursive neural network. The results of\nexperiments to evaluate the effect of 1) the DRL and 2) the logical formula\nembedding framework show that the combination of the two are as effective or\neven better than existing rule-based methods for inquiry dialog policies.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1708.00781v1",
            "date": "2017-08-02 14:49:03+00:00",
            "title": "Dynamic Entity Representations in Neural Language Models",
            "authors": [
                "Yangfeng Ji",
                "Chenhao Tan",
                "Sebastian Martschat",
                "Yejin Choi",
                "Noah A. Smith"
            ],
            "abstract": "Understanding a long document requires tracking how entities are introduced\nand evolve over time. We present a new type of language model, EntityNLM, that\ncan explicitly model entities, dynamically update their representations, and\ncontextually generate their mentions. Our model is generative and flexible; it\ncan model an arbitrary number of entities in context while generating each\nentity mention at an arbitrary length. In addition, it can be used for several\ndifferent tasks such as language modeling, coreference resolution, and entity\nprediction. Experimental results with all these tasks demonstrate that our\nmodel consistently outperforms strong baselines and prior work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01009v1",
            "date": "2017-08-03 05:53:53+00:00",
            "title": "Revisiting Activation Regularization for Language RNNs",
            "authors": [
                "Stephen Merity",
                "Bryan McCann",
                "Richard Socher"
            ],
            "abstract": "Recurrent neural networks (RNNs) serve as a fundamental building block for\nmany sequence tasks across natural language processing. Recent research has\nfocused on recurrent dropout techniques or custom RNN cells in order to improve\nperformance. Both of these can require substantial modifications to the machine\nlearning model or to the underlying RNN configurations. We revisit traditional\nregularization techniques, specifically L2 regularization on RNN activations\nand slowness regularization over successive hidden states, to improve the\nperformance of RNNs on the task of language modeling. Both of these techniques\nrequire minimal modification to existing RNN architectures and result in\nperformance improvements comparable or superior to more complicated\nregularization techniques or custom cell architectures. These regularization\ntechniques can be used without any modification on optimized LSTM\nimplementations such as the NVIDIA cuDNN LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01065v1",
            "date": "2017-08-03 09:18:16+00:00",
            "title": "Reader-Aware Multi-Document Summarization: An Enhanced Model and The First Dataset",
            "authors": [
                "Piji Li",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "We investigate the problem of reader-aware multi-document summarization\n(RA-MDS) and introduce a new dataset for this problem. To tackle RA-MDS, we\nextend a variational auto-encodes (VAEs) based MDS framework by jointly\nconsidering news documents and reader comments. To conduct evaluation for\nsummarization performance, we prepare a new dataset. We describe the methods\nfor data collection, aspect annotation, and summary writing as well as\nscrutinizing by experts. Experimental results show that reader comments can\nimprove the summarization performance, which also demonstrates the usefulness\nof the proposed dataset. The annotated dataset for RA-MDS is available online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01336v1",
            "date": "2017-08-04 00:17:48+00:00",
            "title": "MemexQA: Visual Memex Question Answering",
            "authors": [
                "Lu Jiang",
                "Junwei Liang",
                "Liangliang Cao",
                "Yannis Kalantidis",
                "Sachin Farfade",
                "Alexander Hauptmann"
            ],
            "abstract": "This paper proposes a new task, MemexQA: given a collection of photos or\nvideos from a user, the goal is to automatically answer questions that help\nusers recover their memory about events captured in the collection. Towards\nsolving the task, we 1) present the MemexQA dataset, a large, realistic\nmultimodal dataset consisting of real personal photos and crowd-sourced\nquestions/answers, 2) propose MemexNet, a unified, end-to-end trainable network\narchitecture for image, text and video question answering. Experimental results\non the MemexQA dataset demonstrate that MemexNet outperforms strong baselines\nand yields the state-of-the-art on this novel and challenging task. The\npromising results on TextQA and VideoQA suggest MemexNet's efficacy and\nscalability across various QA tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.01372v1",
            "date": "2017-08-04 03:47:35+00:00",
            "title": "Hashtag Healthcare: From Tweets to Mental Health Journals Using Deep Transfer Learning",
            "authors": [
                "Benjamin Shickel",
                "Martin Heesacker",
                "Sherry Benton",
                "Parisa Rashidi"
            ],
            "abstract": "As the popularity of social media platforms continues to rise, an\never-increasing amount of human communication and self- expression takes place\nonline. Most recent research has focused on mining social media for public user\nopinion about external entities such as product reviews or sentiment towards\npolitical news. However, less attention has been paid to analyzing users'\ninternalized thoughts and emotions from a mental health perspective. In this\npaper, we quantify the semantic difference between public Tweets and private\nmental health journals used in online cognitive behavioral therapy. We will use\ndeep transfer learning techniques for analyzing the semantic gap between the\ntwo domains. We show that for the task of emotional valence prediction, social\nmedia can be successfully harnessed to create more accurate, robust, and\npersonalized mental health models. Our results suggest that the semantic gap\nbetween public and private self-expression is small, and that utilizing the\nabundance of available social media is one way to overcome the small sample\nsizes of mental health data, which are commonly limited by availability and\nprivacy concerns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01425v4",
            "date": "2017-08-04 08:46:03+00:00",
            "title": "The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants",
            "authors": [
                "Ivan Habernal",
                "Henning Wachsmuth",
                "Iryna Gurevych",
                "Benno Stein"
            ],
            "abstract": "Reasoning is a crucial part of natural language argumentation. To comprehend\nan argument, one must analyze its warrant, which explains why its claim follows\nfrom its premises. As arguments are highly contextualized, warrants are usually\npresupposed and left implicit. Thus, the comprehension does not only require\nlanguage understanding and logic skills, but also depends on common sense. In\nthis paper we develop a methodology for reconstructing warrants systematically.\nWe operationalize it in a scalable crowdsourcing process, resulting in a freely\nlicensed dataset with warrants for 2k authentic arguments from news comments.\nOn this basis, we present a new challenging task, the argument reasoning\ncomprehension task. Given an argument with a claim and a premise, the goal is\nto choose the correct implicit warrant from two options. Both warrants are\nplausible and lexically close, but lead to contradicting claims. A solution to\nthis task will define a substantial step towards automatic warrant\nreconstruction. However, experiments with several neural attention and language\nmodels reveal that current approaches do not suffice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01565v1",
            "date": "2017-08-04 15:57:38+00:00",
            "title": "Improving Speaker-Independent Lipreading with Domain-Adversarial Training",
            "authors": [
                "Michael Wand",
                "Juergen Schmidhuber"
            ],
            "abstract": "We present a Lipreading system, i.e. a speech recognition system using only\nvisual features, which uses domain-adversarial training for speaker\nindependence. Domain-adversarial training is integrated into the optimization\nof a lipreader based on a stack of feedforward and LSTM (Long Short-Term\nMemory) recurrent neural networks, yielding an end-to-end trainable system\nwhich only requires a very small number of frames of untranscribed target data\nto substantially improve the recognition accuracy on the target speaker. On\npairs of different source and target speakers, we achieve a relative accuracy\nimprovement of around 40% with only 15 to 20 seconds of untranscribed target\nspeech data. On multi-speaker training setups, the accuracy improvements are\nsmaller but still substantial.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.01944v1",
            "date": "2017-08-06 22:20:02+00:00",
            "title": "Rookie: A unique approach for exploring news archives",
            "authors": [
                "Abram Handler",
                "Brendan O'Connor"
            ],
            "abstract": "News archives are an invaluable primary source for placing current events in\nhistorical context. But current search engine tools do a poor job at uncovering\nbroad themes and narratives across documents. We present Rookie: a practical\nsoftware system which uses natural language processing (NLP) to help readers,\nreporters and editors uncover broad stories in news archives. Unlike prior\nwork, Rookie's design emerged from 18 months of iterative development in\nconsultation with editors and computational journalists. This process lead to a\ndramatically different approach from previous academic systems with similar\ngoals. Our efforts offer a generalizable case study for others building\nreal-world journalism software using NLP.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1708.02210v1",
            "date": "2017-08-07 17:21:20+00:00",
            "title": "Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowd-sourced Time-Sync Comments",
            "authors": [
                "Qing Ping",
                "Chaomei Chen"
            ],
            "abstract": "With the prevalence of video sharing, there are increasing demands for\nautomatic video digestion such as highlight detection. Recently, platforms with\ncrowdsourced time-sync video comments have emerged worldwide, providing a good\nopportunity for highlight detection. However, this task is non-trivial: (1)\ntime-sync comments often lag behind their corresponding shot; (2) time-sync\ncomments are semantically sparse and noisy; (3) to determine which shots are\nhighlights is highly subjective. The present paper aims to tackle these\nchallenges by proposing a framework that (1) uses concept-mapped lexical-chains\nfor lag calibration; (2) models video highlights based on comment intensity and\ncombination of emotion and concept concentration of each shot; (3) summarize\neach detected highlight using improved SumBasic with emotion and concept\nmapping. Experiments on large real-world datasets show that our highlight\ndetection method and summarization method both outperform other benchmarks with\nconsiderable margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02657v2",
            "date": "2017-08-08 21:24:44+00:00",
            "title": "Which Encoding is the Best for Text Classification in Chinese, English, Japanese and Korean?",
            "authors": [
                "Xiang Zhang",
                "Yann LeCun"
            ],
            "abstract": "This article offers an empirical study on the different ways of encoding\nChinese, Japanese, Korean (CJK) and English languages for text classification.\nDifferent encoding levels are studied, including UTF-8 bytes, characters,\nwords, romanized characters and romanized words. For all encoding levels,\nwhenever applicable, we provide comparisons with linear models, fastText and\nconvolutional networks. For convolutional networks, we compare between encoding\nmechanisms using character glyph images, one-hot (or one-of-n) encoding, and\nembedding. In total there are 473 models, using 14 large-scale text\nclassification datasets in 4 languages including Chinese, English, Japanese and\nKorean. Some conclusions from these results include that byte-level one-hot\nencoding based on UTF-8 consistently produces competitive results for\nconvolutional networks, that word-level n-grams linear models are competitive\neven without perfect word segmentation, and that fastText provides the best\nresult using character-level n-gram encoding but can overfit when the features\nare overly rich.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02702v4",
            "date": "2017-08-09 03:21:20+00:00",
            "title": "Neural Vector Spaces for Unsupervised Information Retrieval",
            "authors": [
                "Christophe Van Gysel",
                "Maarten de Rijke",
                "Evangelos Kanoulas"
            ],
            "abstract": "We propose the Neural Vector Space Model (NVSM), a method that learns\nrepresentations of documents in an unsupervised manner for news article\nretrieval. In the NVSM paradigm, we learn low-dimensional representations of\nwords and documents from scratch using gradient descent and rank documents\naccording to their similarity with query representations that are composed from\nword representations. We show that NVSM performs better at document ranking\nthan existing latent semantic vector space methods. The addition of NVSM to a\nmixture of lexical language models and a state-of-the-art baseline vector space\nmodel yields a statistically significant increase in retrieval effectiveness.\nConsequently, NVSM adds a complementary relevance signal. Next to semantic\nmatching, we find that NVSM performs well in cases where lexical matching is\nneeded.\n  NVSM learns a notion of term specificity directly from the document\ncollection without feature engineering. We also show that NVSM learns\nregularities related to Luhn significance. Finally, we give advice on how to\ndeploy NVSM in situations where model selection (e.g., cross-validation) is\ninfeasible. We find that an unsupervised ensemble of multiple models trained\nwith different hyperparameter values performs better than a single\ncross-validated model. Therefore, NVSM can safely be used for ranking documents\nwithout supervised relevance judgments.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.02711v1",
            "date": "2017-08-09 04:19:42+00:00",
            "title": "Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge",
            "authors": [
                "Damien Teney",
                "Peter Anderson",
                "Xiaodong He",
                "Anton van den Hengel"
            ],
            "abstract": "This paper presents a state-of-the-art model for visual question answering\n(VQA), which won the first place in the 2017 VQA Challenge. VQA is a task of\nsignificant importance for research in artificial intelligence, given its\nmultimodal nature, clear evaluation protocol, and potential real-world\napplications. The performance of deep neural networks for VQA is very dependent\non choices of architectures and hyperparameters. To help further research in\nthe area, we describe in detail our high-performing, though relatively simple\nmodel. Through a massive exploration of architectures and hyperparameters\nrepresenting more than 3,000 GPU-hours, we identified tips and tricks that lead\nto its success, namely: sigmoid outputs, soft training targets, image features\nfrom bottom-up attention, gated tanh activations, output embeddings initialized\nusing GloVe and Google Images, large mini-batches, and smart shuffling of\ntraining data. We provide a detailed analysis of their impact on performance to\nassist others in making an appropriate selection.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.02912v1",
            "date": "2017-08-09 17:04:34+00:00",
            "title": "KeyXtract Twitter Model - An Essential Keywords Extraction Model for Twitter Designed using NLP Tools",
            "authors": [
                "Tharindu Weerasooriya",
                "Nandula Perera",
                "S. R. Liyanage"
            ],
            "abstract": "Since a tweet is limited to 140 characters, it is ambiguous and difficult for\ntraditional Natural Language Processing (NLP) tools to analyse. This research\npresents KeyXtract which enhances the machine learning based Stanford CoreNLP\nPart-of-Speech (POS) tagger with the Twitter model to extract essential\nkeywords from a tweet. The system was developed using rule-based parsers and\ntwo corpora. The data for the research was obtained from a Twitter profile of a\ntelecommunication company. The system development consisted of two stages. At\nthe initial stage, a domain specific corpus was compiled after analysing the\ntweets. The POS tagger extracted the Noun Phrases and Verb Phrases while the\nparsers removed noise and extracted any other keywords missed by the POS\ntagger. The system was evaluated using the Turing Test. After it was tested and\ncompared against Stanford CoreNLP, the second stage of the system was developed\naddressing the shortcomings of the first stage. It was enhanced using Named\nEntity Recognition and Lemmatization. The second stage was also tested using\nthe Turing test and its pass rate increased from 50.00% to 83.33%. The\nperformance of the final system output was measured using the F1 score.\nStanford CoreNLP with the Twitter model had an average F1 of 0.69 while the\nimproved system had a F1 of 0.77. The accuracy of the system could be improved\nby using a complete domain specific corpus. Since the system used linguistic\nfeatures of a sentence, it could be applied to other NLP tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03246v1",
            "date": "2017-08-10 15:03:12+00:00",
            "title": "SESA: Supervised Explicit Semantic Analysis",
            "authors": [
                "Dasha Bogdanova",
                "Majid Yazdani"
            ],
            "abstract": "In recent years supervised representation learning has provided state of the\nart or close to the state of the art results in semantic analysis tasks\nincluding ranking and information retrieval. The core idea is to learn how to\nembed items into a latent space such that they optimize a supervised objective\nin that latent space. The dimensions of the latent space have no clear\nsemantics, and this reduces the interpretability of the system. For example, in\npersonalization models, it is hard to explain why a particular item is ranked\nhigh for a given user profile. We propose a novel model of representation\nlearning called Supervised Explicit Semantic Analysis (SESA) that is trained in\na supervised fashion to embed items to a set of dimensions with explicit\nsemantics. The model learns to compare two objects by representing them in this\nexplicit space, where each dimension corresponds to a concept from a knowledge\nbase. This work extends Explicit Semantic Analysis (ESA) with a supervised\nmodel for ranking problems. We apply this model to the task of Job-Profile\nrelevance in LinkedIn in which a set of skills defines our explicit dimensions\nof the space. Every profile and job are encoded to this set of skills their\nsimilarity is calculated in this space. We use RNNs to embed text input into\nthis space. In addition to interpretability, our model makes use of the\nweb-scale collaborative skills data that is provided by users for each LinkedIn\nprofile. Our model provides state of the art result while it remains\ninterpretable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03569v1",
            "date": "2017-08-11 15:19:53+00:00",
            "title": "Semantic Word Clouds with Background Corpus Normalization and t-distributed Stochastic Neighbor Embedding",
            "authors": [
                "Erich Schubert",
                "Andreas Spitz",
                "Michael Weiler",
                "Johanna Gei\u00df",
                "Michael Gertz"
            ],
            "abstract": "Many word clouds provide no semantics to the word placement, but use a random\nlayout optimized solely for aesthetic purposes. We propose a novel approach to\nmodel word significance and word affinity within a document, and in comparison\nto a large background corpus. We demonstrate its usefulness for generating more\nmeaningful word clouds as a visual summary of a given document. We then select\nkeywords based on their significance and construct the word cloud based on the\nderived affinity. Based on a modified t-distributed stochastic neighbor\nembedding (t-SNE), we generate a semantic word placement. For words that\ncooccur significantly, we include edges, and cluster the words according to\ntheir cooccurrence. For this we designed a scalable and memory-efficient\nsketch-based approach usable on commodity hardware to aggregate the required\ncorpus statistics needed for normalization, and for identifying keywords as\nwell as significant cooccurences. We empirically validate our approch using a\nlarge Wikipedia corpus.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.03892v2",
            "date": "2017-08-13 11:55:02+00:00",
            "title": "EmoTxt: A Toolkit for Emotion Recognition from Text",
            "authors": [
                "Fabio Calefato",
                "Filippo Lanubile",
                "Nicole Novielli"
            ],
            "abstract": "We present EmoTxt, a toolkit for emotion recognition from text, trained and\ntested on a gold standard of about 9K question, answers, and comments from\nonline interactions. We provide empirical evidence of the performance of\nEmoTxt. To the best of our knowledge, EmoTxt is the first open-source toolkit\nsupporting both emotion recognition from text and training of custom emotion\nclassification models.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1708.03994v1",
            "date": "2017-08-14 02:34:17+00:00",
            "title": "Data Sets: Word Embeddings Learned from Tweets and General Data",
            "authors": [
                "Quanzhi Li",
                "Sameena Shah",
                "Xiaomo Liu",
                "Armineh Nourbakhsh"
            ],
            "abstract": "A word embedding is a low-dimensional, dense and real- valued vector\nrepresentation of a word. Word embeddings have been used in many NLP tasks.\nThey are usually gener- ated from a large text corpus. The embedding of a word\ncap- tures both its syntactic and semantic aspects. Tweets are short, noisy and\nhave unique lexical and semantic features that are different from other types\nof text. Therefore, it is necessary to have word embeddings learned\nspecifically from tweets. In this paper, we present ten word embedding data\nsets. In addition to the data sets learned from just tweet data, we also built\nembedding sets from the general data and the combination of tweets with the\ngeneral data. The general data consist of news articles, Wikipedia data and\nother web data. These ten embedding models were learned from about 400 million\ntweets and 7 billion words from the general text. In this paper, we also\npresent two experiments demonstrating how to use the data sets in some NLP\ntasks, such as tweet sentiment analysis and tweet topic classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04120v2",
            "date": "2017-07-28 09:35:45+00:00",
            "title": "Putting Self-Supervised Token Embedding on the Tables",
            "authors": [
                "Marc Szafraniec",
                "Gautier Marti",
                "Philippe Donnat"
            ],
            "abstract": "Information distribution by electronic messages is a privileged means of\ntransmission for many businesses and individuals, often under the form of\nplain-text tables. As their number grows, it becomes necessary to use an\nalgorithm to extract text and numbers instead of a human. Usual methods are\nfocused on regular expressions or on a strict structure in the data, but are\nnot efficient when we have many variations, fuzzy structure or implicit labels.\nIn this paper we introduce SC2T, a totally self-supervised model for\nconstructing vector representations of tokens in semi-structured messages by\nusing characters and context levels that address these issues. It can then be\nused for an unsupervised labeling of tokens, or be the basis for a\nsemi-supervised information extraction system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.04134v1",
            "date": "2017-08-04 03:44:35+00:00",
            "title": "A Measure for Dialog Complexity and its Application in Streamlining Service Operations",
            "authors": [
                "Q Vera Liao",
                "Biplav Srivastava",
                "Pavan Kapanipathi"
            ],
            "abstract": "Dialog is a natural modality for interaction between customers and businesses\nin the service industry. As customers call up the service provider, their\ninteractions may be routine or extraordinary. We believe that these\ninteractions, when seen as dialogs, can be analyzed to obtain a better\nunderstanding of customer needs and how to efficiently address them. We\nintroduce the idea of a dialog complexity measure to characterize multi-party\ninteractions, propose a general data-driven method to calculate it, use it to\ndiscover insights in public and enterprise dialog datasets, and demonstrate its\nbeneficial usage in facilitating better handling of customer requests and\nevaluating service agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04681v1",
            "date": "2017-08-15 20:38:37+00:00",
            "title": "Identifying Harm Events in Clinical Care through Medical Narratives",
            "authors": [
                "Arman Cohan",
                "Allan Fong",
                "Raj Ratwani",
                "Nazli Goharian"
            ],
            "abstract": "Preventable medical errors are estimated to be among the leading causes of\ninjury and death in the United States. To prevent such errors, healthcare\nsystems have implemented patient safety and incident reporting systems. These\nsystems enable clinicians to report unsafe conditions and cases where patients\nhave been harmed due to errors in medical care. These reports are narratives in\nnatural language and while they provide detailed information about the\nsituation, it is non-trivial to perform large scale analysis for identifying\ncommon causes of errors and harm to the patients. In this work, we present a\nmethod based on attentive convolutional and recurrent networks for identifying\nharm events in patient care and categorize the harm based on its severity\nlevel. We demonstrate that our methods can significantly improve the\nperformance over existing methods in identifying harm in clinical care.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04923v1",
            "date": "2017-08-16 15:12:20+00:00",
            "title": "mAnI: Movie Amalgamation using Neural Imitation",
            "authors": [
                "Naveen Panwar",
                "Shreya Khare",
                "Neelamadhav Gantayat",
                "Rahul Aralikatte",
                "Senthil Mani",
                "Anush Sankaran"
            ],
            "abstract": "Cross-modal data retrieval has been the basis of various creative tasks\nperformed by Artificial Intelligence (AI). One such highly challenging task for\nAI is to convert a book into its corresponding movie, which most of the\ncreative film makers do as of today. In this research, we take the first step\ntowards it by visualizing the content of a book using its corresponding movie\nvisuals. Given a set of sentences from a book or even a fan-fiction written in\nthe same universe, we employ deep learning models to visualize the input by\nstitching together relevant frames from the movie. We studied and compared\nthree different types of setting to match the book with the movie content: (i)\nDialog model: using only the dialog from the movie, (ii) Visual model: using\nonly the visual content from the movie, and (iii) Hybrid model: using the\ndialog and the visual content from the movie. Experiments on the publicly\navailable MovieBook dataset shows the effectiveness of the proposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04968v2",
            "date": "2017-08-16 16:37:52+00:00",
            "title": "Fault in your stars: An Analysis of Android App Reviews",
            "authors": [
                "Rahul Aralikatte",
                "Giriprasad Sridhara",
                "Neelamadhav Gantayat",
                "Senthil Mani"
            ],
            "abstract": "Mobile app distribution platforms such as Google Play Store allow users to\nshare their feedback about downloaded apps in the form of a review comment and\na corresponding star rating. Typically, the star rating ranges from one to five\nstars, with one star denoting a high sense of dissatisfaction with the app and\nfive stars denoting a high sense of satisfaction.\n  Unfortunately, due to a variety of reasons, often the star rating provided by\na user is inconsistent with the opinion expressed in the review. For example,\nconsider the following review for the Facebook App on Android; \"Awesome App\".\nOne would reasonably expect the rating for this review to be five stars, but\nthe actual rating is one star!\n  Such inconsistent ratings can lead to a deflated (or inflated) overall\naverage rating of an app which can affect user downloads, as typically users\nlook at the average star ratings while making a decision on downloading an app.\nAlso, the app developers receive a biased feedback about the application that\ndoes not represent ground reality. This is especially significant for small\napps with a few thousand downloads as even a small number of mismatched reviews\ncan bring down the average rating drastically.\n  In this paper, we conducted a study on this review-rating mismatch problem.\nWe manually examined 8600 reviews from 10 popular Android apps and found that\n20% of the ratings in our dataset were inconsistent with the review. Further,\nwe developed three systems; two of which were based on traditional machine\nlearning and one on deep learning to automatically identify reviews whose\nrating did not match with the opinion expressed in the review. Our deep\nlearning system performed the best and had an accuracy of 92% in identifying\nthe correct star rating to be associated with a given review.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.05071v1",
            "date": "2017-08-14 17:32:06+00:00",
            "title": "Learning spectro-temporal features with 3D CNNs for speech emotion recognition",
            "authors": [
                "Jaebok Kim",
                "Khiet P. Truong",
                "Gwenn Englebienne",
                "Vanessa Evers"
            ],
            "abstract": "In this paper, we propose to use deep 3-dimensional convolutional networks\n(3D CNNs) in order to address the challenge of modelling spectro-temporal\ndynamics for speech emotion recognition (SER). Compared to a hybrid of\nConvolutional Neural Network and Long-Short-Term-Memory (CNN-LSTM), our\nproposed 3D CNNs simultaneously extract short-term and long-term spectral\nfeatures with a moderate number of parameters. We evaluated our proposed and\nother state-of-the-art methods in a speaker-independent manner using aggregated\ncorpora that give a large and diverse set of speakers. We found that 1) shallow\ntemporal and moderately deep spectral kernels of a homogeneous architecture are\noptimal for the task; and 2) our 3D CNNs are more effective for\nspectro-temporal feature learning compared to other methods. Finally, we\nvisualised the feature space obtained with our proposed method using\nt-distributed stochastic neighbour embedding (T-SNE) and could observe distinct\nclusters of emotions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05271v1",
            "date": "2017-08-17 13:51:39+00:00",
            "title": "Incorporating Copying Mechanism in Image Captioning for Learning Novel Objects",
            "authors": [
                "Ting Yao",
                "Yingwei Pan",
                "Yehao Li",
                "Tao Mei"
            ],
            "abstract": "Image captioning often requires a large set of training image-sentence pairs.\nIn practice, however, acquiring sufficient training pairs is always expensive,\nmaking the recent captioning models limited in their ability to describe\nobjects outside of training corpora (i.e., novel objects). In this paper, we\npresent Long Short-Term Memory with Copying Mechanism (LSTM-C) --- a new\narchitecture that incorporates copying into the Convolutional Neural Networks\n(CNN) plus Recurrent Neural Networks (RNN) image captioning framework, for\ndescribing novel objects in captions. Specifically, freely available object\nrecognition datasets are leveraged to develop classifiers for novel objects.\nOur LSTM-C then nicely integrates the standard word-by-word sentence generation\nby a decoder RNN with copying mechanism which may instead select words from\nnovel objects at proper places in the output sentence. Extensive experiments\nare conducted on both MSCOCO image captioning and ImageNet datasets,\ndemonstrating the ability of our proposed LSTM-C architecture to describe novel\nobjects. Furthermore, superior results are reported when compared to\nstate-of-the-art deep models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.05873v1",
            "date": "2017-08-19 16:39:19+00:00",
            "title": "What Drives the International Development Agenda? An NLP Analysis of the United Nations General Debate 1970-2016",
            "authors": [
                "Alexander Baturo",
                "Niheer Dasandi",
                "Slava J. Mikhaylov"
            ],
            "abstract": "There is surprisingly little known about agenda setting for international\ndevelopment in the United Nations (UN) despite it having a significant\ninfluence on the process and outcomes of development efforts. This paper\naddresses this shortcoming using a novel approach that applies natural language\nprocessing techniques to countries' annual statements in the UN General Debate.\nEvery year UN member states deliver statements during the General Debate on\ntheir governments' perspective on major issues in world politics. These\nspeeches provide invaluable information on state preferences on a wide range of\nissues, including international development, but have largely been overlooked\nin the study of global politics. This paper identifies the main international\ndevelopment topics that states raise in these speeches between 1970 and 2016,\nand examine the country-specific drivers of international development rhetoric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05997v2",
            "date": "2017-08-20 17:48:35+00:00",
            "title": "A Batch Noise Contrastive Estimation Approach for Training Large Vocabulary Language Models",
            "authors": [
                "Youssef Oualil",
                "Dietrich Klakow"
            ],
            "abstract": "Training large vocabulary Neural Network Language Models (NNLMs) is a\ndifficult task due to the explicit requirement of the output layer\nnormalization, which typically involves the evaluation of the full softmax\nfunction over the complete vocabulary. This paper proposes a Batch Noise\nContrastive Estimation (B-NCE) approach to alleviate this problem. This is\nachieved by reducing the vocabulary, at each time step, to the target words in\nthe batch and then replacing the softmax by the noise contrastive estimation\napproach, where these words play the role of targets and noise samples at the\nsame time. In doing so, the proposed approach can be fully formulated and\nimplemented using optimal dense matrix operations. Applying B-NCE to train\ndifferent NNLMs on the Large Text Compression Benchmark (LTCB) and the One\nBillion Word Benchmark (OBWB) shows a significant reduction of the training\ntime with no noticeable degradation of the models performance. This paper also\npresents a new baseline comparative study of different standard NNLMs on the\nlarge OBWB on a single Titan-X GPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06266v1",
            "date": "2017-08-21 14:52:10+00:00",
            "title": "Probabilistic Relation Induction in Vector Space Embeddings",
            "authors": [
                "Zied Bouraoui",
                "Shoaib Jameel",
                "Steven Schockaert"
            ],
            "abstract": "Word embeddings have been found to capture a surprisingly rich amount of\nsyntactic and semantic knowledge. However, it is not yet sufficiently\nwell-understood how the relational knowledge that is implicitly encoded in word\nembeddings can be extracted in a reliable way. In this paper, we propose two\nprobabilistic models to address this issue. The first model is based on the\ncommon relations-as-translations view, but is cast in a probabilistic setting.\nOur second model is based on the much weaker assumption that there is a linear\nrelationship between the vector representations of related words. Compared to\nexisting approaches, our models lead to more accurate predictions, and they are\nmore explicit about what can and cannot be extracted from the word embedding.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1708.06550v1",
            "date": "2017-08-22 09:39:29+00:00",
            "title": "Golden Years, Golden Shores: A Study of Elders in Online Travel Communities",
            "authors": [
                "Bart\u0142omiej Balcerzak",
                "Rados\u0142aw Nielek"
            ],
            "abstract": "In this paper we present our exploratory findings related to extracting\nknowledge and experiences from a community of senior tourists. By using tools\nof qualitative analysis as well as review of literature, we managed to verify a\nset of hypotheses related to the content created by senior tourists when\nparticipating in on-line communities. We also produced a codebook, representing\nvarious themes one may encounter in such communities. This codebook, derived\nfrom our own qualitative research, as well a literature review will serve as a\nbasis for further development of automated tools of knowledge extraction. We\nalso managed to find that older adults more often than other poster in tourists\nforums, mention their age in discussion, more often share their experiences and\nmotivation to travel, however they do not differ in relation to describing\nbarriers encountered while traveling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06555v1",
            "date": "2017-08-22 10:26:41+00:00",
            "title": "Long-Short Range Context Neural Networks for Language Modeling",
            "authors": [
                "Youssef Oualil",
                "Mittul Singh",
                "Clayton Greenberg",
                "Dietrich Klakow"
            ],
            "abstract": "The goal of language modeling techniques is to capture the statistical and\nstructural properties of natural languages from training corpora. This task\ntypically involves the learning of short range dependencies, which generally\nmodel the syntactic properties of a language and/or long range dependencies,\nwhich are semantic in nature. We propose in this paper a new multi-span\narchitecture, which separately models the short and long context information\nwhile it dynamically merges them to perform the language modeling task. This is\ndone through a novel recurrent Long-Short Range Context (LSRC) network, which\nexplicitly models the local (short) and global (long) context using two\nseparate hidden states that evolve in time. This new architecture is an\nadaptation of the Long-Short Term Memory network (LSTM) to take into account\nthe linguistic properties. Extensive experiments conducted on the Penn Treebank\n(PTB) and the Large Text Compression Benchmark (LTCB) corpus showed a\nsignificant reduction of the perplexity when compared to state-of-the-art\nlanguage modeling techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06989v1",
            "date": "2017-08-23 13:27:16+00:00",
            "title": "A Neural Network Approach for Mixing Language Models",
            "authors": [
                "Youssef Oualil",
                "Dietrich Klakow"
            ],
            "abstract": "The performance of Neural Network (NN)-based language models is steadily\nimproving due to the emergence of new architectures, which are able to learn\ndifferent natural language characteristics. This paper presents a novel\nframework, which shows that a significant improvement can be achieved by\ncombining different existing heterogeneous models in a single architecture.\nThis is done through 1) a feature layer, which separately learns different\nNN-based models and 2) a mixture layer, which merges the resulting model\nfeatures. In doing so, this architecture benefits from the learning\ncapabilities of each model with no noticeable increase in the number of model\nparameters or the training time. Extensive experiments conducted on the Penn\nTreebank (PTB) and the Large Text Compression Benchmark (LTCB) corpus showed a\nsignificant reduction of the perplexity when compared to state-of-the-art\nfeedforward as well as recurrent neural network architectures.",
            "journal ref": "IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), New Orleans, LA, 2017, pp. 5710-5714",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07252v1",
            "date": "2017-08-24 02:14:50+00:00",
            "title": "A Study on Neural Network Language Modeling",
            "authors": [
                "Dengliang Shi"
            ],
            "abstract": "An exhaustive study on neural network language modeling (NNLM) is performed\nin this paper. Different architectures of basic neural network language models\nare described and examined. A number of different improvements over basic\nneural network language models, including importance sampling, word classes,\ncaching and bidirectional recurrent neural network (BiRNN), are studied\nseparately, and the advantages and disadvantages of every technique are\nevaluated. Then, the limits of neural network language modeling are explored\nfrom the aspects of model architecture and knowledge representation. Part of\nthe statistical information from a word sequence will loss when it is processed\nword by word in a certain order, and the mechanism of training neural network\nby updating weight matrixes and vectors imposes severe restrictions on any\nsignificant enhancement of NNLM. For knowledge representation, the knowledge\nrepresented by neural network language models is the approximate probabilistic\ndistribution of word sequences from a certain training data set rather than the\nknowledge of a language itself or the information conveyed by word sequences in\na natural language. Finally, some directions for improving neural network\nlanguage modeling further is discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07265v1",
            "date": "2017-08-24 03:12:22+00:00",
            "title": "An Image Analysis Approach to the Calligraphy of Books",
            "authors": [
                "Henrique F. de Arruda",
                "Vanessa Q. Marinho",
                "Thales S. Lima",
                "Diego R. Amancio",
                "Luciano da F. Costa"
            ],
            "abstract": "Text network analysis has received increasing attention as a consequence of\nits wide range of applications. In this work, we extend a previous work founded\non the study of topological features of mesoscopic networks. Here, the\ngeometrical properties of visualized networks are quantified in terms of\nseveral image analysis techniques and used as subsidies for authorship\nattribution. It was found that the visual features account for performance\nsimilar to that achieved by using topological measurements. In addition, the\ncombination of these two types of features improved the performance.",
            "journal ref": "Physica A 510, 110--120 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07624v2",
            "date": "2017-08-25 06:41:55+00:00",
            "title": "SPARQL as a Foreign Language",
            "authors": [
                "Tommaso Soru",
                "Edgard Marx",
                "Diego Moussallem",
                "Gustavo Publio",
                "Andr\u00e9 Valdestilhas",
                "Diego Esteves",
                "Ciro Baron Neto"
            ],
            "abstract": "In the last years, the Linked Data Cloud has achieved a size of more than 100\nbillion facts pertaining to a multitude of domains. However, accessing this\ninformation has been significantly challenging for lay users. Approaches to\nproblems such as Question Answering on Linked Data and Link Discovery have\nnotably played a role in increasing information access. These approaches are\noften based on handcrafted and/or statistical models derived from data\nobservation. Recently, Deep Learning architectures based on Neural Networks\ncalled seq2seq have shown to achieve state-of-the-art results at translating\nsequences into sequences. In this direction, we propose Neural SPARQL Machines,\nend-to-end deep architectures to translate any natural language expression into\nsentences encoding SPARQL queries. Our preliminary results, restricted on\nselected DBpedia classes, show that Neural SPARQL Machines are a promising\napproach for Question Answering on Linked Data, as they can deal with known\nproblems such as vocabulary mismatch and perform graph pattern composition.",
            "journal ref": "SEMANTiCS CEUR Workshop Proceedings 2044 (2017) Paper 14",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07863v1",
            "date": "2017-08-25 19:04:25+00:00",
            "title": "$k$-Nearest Neighbor Augmented Neural Networks for Text Classification",
            "authors": [
                "Zhiguo Wang",
                "Wael Hamza",
                "Linfeng Song"
            ],
            "abstract": "In recent years, many deep-learning based models are proposed for text\nclassification. This kind of models well fits the training set from the\nstatistical point of view. However, it lacks the capacity of utilizing\ninstance-level information from individual instances in the training set. In\nthis work, we propose to enhance neural network models by allowing them to\nleverage information from $k$-nearest neighbor (kNN) of the input text. Our\nmodel employs a neural network that encodes texts into text embeddings.\nMoreover, we also utilize $k$-nearest neighbor of the input text as an external\nmemory, and utilize it to capture instance-level information from the training\nset. The final prediction is made based on features from both the neural\nnetwork encoder and the kNN memory. Experimental results on several standard\nbenchmark datasets show that our model outperforms the baseline model on all\nthe datasets, and it even beats a very deep neural network model (with 29\nlayers) in several datasets. Our model also shows superior performance when\ntraining instances are scarce, and when the training set is severely\nunbalanced. Our model also leverages techniques such as semi-supervised\ntraining and transfer learning quite well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07903v1",
            "date": "2017-08-25 22:03:09+00:00",
            "title": "Nationality Classification Using Name Embeddings",
            "authors": [
                "Junting Ye",
                "Shuchu Han",
                "Yifan Hu",
                "Baris Coskun",
                "Meizhu Liu",
                "Hong Qin",
                "Steven Skiena"
            ],
            "abstract": "Nationality identification unlocks important demographic information, with\nmany applications in biomedical and sociological research. Existing name-based\nnationality classifiers use name substrings as features and are trained on\nsmall, unrepresentative sets of labeled names, typically extracted from\nWikipedia. As a result, these methods achieve limited performance and cannot\nsupport fine-grained classification.\n  We exploit the phenomena of homophily in communication patterns to learn name\nembeddings, a new representation that encodes gender, ethnicity, and\nnationality which is readily applicable to building classifiers and other\nsystems. Through our analysis of 57M contact lists from a major Internet\ncompany, we are able to design a fine-grained nationality classifier covering\n39 groups representing over 90% of the world population. In an evaluation\nagainst other published systems over 13 common classes, our F1 score (0.795) is\nsubstantial better than our closest competitor Ethnea (0.580). To the best of\nour knowledge, this is the most accurate, fine-grained nationality classifier\navailable.\n  As a social media application, we apply our classifiers to the followers of\nmajor Twitter celebrities over six different domains. We demonstrate stark\ndifferences in the ethnicities of the followers of Trump and Obama, and in the\nsports and entertainments favored by different groups. Finally, we identify an\nanomalous political figure whose presumably inflated following appears largely\nincapable of reading the language he posts in.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1708.08123v1",
            "date": "2017-08-27 18:50:48+00:00",
            "title": "Impact of Feature Selection on Micro-Text Classification",
            "authors": [
                "Ankit Vadehra",
                "Maura R. Grossman",
                "Gordon V. Cormack"
            ],
            "abstract": "Social media datasets, especially Twitter tweets, are popular in the field of\ntext classification. Tweets are a valuable source of micro-text (sometimes\nreferred to as \"micro-blogs\"), and have been studied in domains such as\nsentiment analysis, recommendation systems, spam detection, clustering, among\nothers. Tweets often include keywords referred to as \"Hashtags\" that can be\nused as labels for the tweet. Using tweets encompassing 50 labels, we studied\nthe impact of word versus character-level feature selection and extraction on\ndifferent learners to solve a multi-class classification task. We show that\nfeature extraction of simple character-level groups performs better than simple\nword groups and pre-processing methods like normalizing using Porter's Stemming\nand Part-of-Speech (\"POS\")-Lemmatization.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.09450v1",
            "date": "2017-08-30 20:01:34+00:00",
            "title": "Learning Fine-Grained Knowledge about Contingent Relations between Everyday Events",
            "authors": [
                "Elahe Rahimtoroghi",
                "Ernesto Hernandez",
                "Marilyn A Walker"
            ],
            "abstract": "Much of the user-generated content on social media is provided by ordinary\npeople telling stories about their daily lives. We develop and test a novel\nmethod for learning fine-grained common-sense knowledge from these stories\nabout contingent (causal and conditional) relationships between everyday\nevents. This type of knowledge is useful for text and story understanding,\ninformation extraction, question answering, and text summarization. We test and\ncompare different methods for learning contingency relation, and compare what\nis learned from topic-sorted story collections vs. general-domain stories. Our\nexperiments show that using topic-specific datasets enables learning\nfiner-grained knowledge about events and results in significant improvement\nover the baselines. An evaluation on Amazon Mechanical Turk shows 82% of the\nrelations between events that we learn from topic-sorted stories are judged as\ncontingent.",
            "journal ref": "17th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL\n  2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09453v1",
            "date": "2017-08-30 20:12:01+00:00",
            "title": "Inference of Fine-Grained Event Causality from Blogs and Films",
            "authors": [
                "Zhichao Hu",
                "Elahe Rahimtoroghi",
                "Marilyn A Walker"
            ],
            "abstract": "Human understanding of narrative is mainly driven by reasoning about causal\nrelations between events and thus recognizing them is a key capability for\ncomputational models of language understanding. Computational work in this area\nhas approached this via two different routes: by focusing on acquiring a\nknowledge base of common causal relations between events, or by attempting to\nunderstand a particular story or macro-event, along with its storyline. In this\nposition paper, we focus on knowledge acquisition approach and claim that\nnewswire is a relatively poor source for learning fine-grained causal relations\nbetween everyday events. We describe experiments using an unsupervised method\nto learn causal relations between events in the narrative genres of\nfirst-person narratives and film scene descriptions. We show that our method\nlearns fine-grained causal relations, judged by humans as likely to be causal\nover 80% of the time. We also demonstrate that the learned event pairs do not\nexist in publicly available event-pair datasets extracted from newswire.",
            "journal ref": "Events and Stories in the News Workshop, ACL 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09492v1",
            "date": "2017-08-30 22:26:48+00:00",
            "title": "Automatically Generating Commit Messages from Diffs using Neural Machine Translation",
            "authors": [
                "Siyuan Jiang",
                "Ameer Armaly",
                "Collin McMillan"
            ],
            "abstract": "Commit messages are a valuable resource in comprehension of software\nevolution, since they provide a record of changes such as feature additions and\nbug repairs. Unfortunately, programmers often neglect to write good commit\nmessages. Different techniques have been proposed to help programmers by\nautomatically writing these messages. These techniques are effective at\ndescribing what changed, but are often verbose and lack context for\nunderstanding the rationale behind a change. In contrast, humans write messages\nthat are short and summarize the high level rationale. In this paper, we adapt\nNeural Machine Translation (NMT) to automatically \"translate\" diffs into commit\nmessages. We trained an NMT algorithm using a corpus of diffs and human-written\ncommit messages from the top 1k Github projects. We designed a filter to help\nensure that we only trained the algorithm on higher-quality commit messages.\nOur evaluation uncovered a pattern in which the messages we generate tend to be\neither very high or very low quality. Therefore, we created a quality-assurance\nfilter to detect cases in which we are unable to produce good messages, and\nreturn a warning instead.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1708.09666v2",
            "date": "2017-08-31 11:17:53+00:00",
            "title": "Generating Video Descriptions with Topic Guidance",
            "authors": [
                "Shizhe Chen",
                "Jia Chen",
                "Qin Jin"
            ],
            "abstract": "Generating video descriptions in natural language (a.k.a. video captioning)\nis a more challenging task than image captioning as the videos are\nintrinsically more complicated than images in two aspects. First, videos cover\na broader range of topics, such as news, music, sports and so on. Second,\nmultiple topics could coexist in the same video. In this paper, we propose a\nnovel caption model, topic-guided model (TGM), to generate topic-oriented\ndescriptions for videos in the wild via exploiting topic information. In\naddition to predefined topics, i.e., category tags crawled from the web, we\nalso mine topics in a data-driven way based on training captions by an\nunsupervised topic mining model. We show that data-driven topics reflect a\nbetter topic schema than the predefined topics. As for testing video topic\nprediction, we treat the topic mining model as teacher to train the student,\nthe topic prediction model, by utilizing the full multi-modalities in the video\nespecially the speech modality. We propose a series of caption models to\nexploit topic guidance, including implicitly using the topics as input features\nto generate words related to the topic and explicitly modifying the weights in\nthe decoder with topics to function as an ensemble of topic-aware language\ndecoders. Our comprehensive experimental results on the current largest video\ncaption dataset MSR-VTT prove the effectiveness of our topic-guided model,\nwhich significantly surpasses the winning performance in the 2016 MSR video to\nlanguage challenge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.09667v2",
            "date": "2017-08-31 11:18:28+00:00",
            "title": "Video Captioning with Guidance of Multimodal Latent Topics",
            "authors": [
                "Shizhe Chen",
                "Jia Chen",
                "Qin Jin",
                "Alexander Hauptmann"
            ],
            "abstract": "The topic diversity of open-domain videos leads to various vocabularies and\nlinguistic expressions in describing video contents, and therefore, makes the\nvideo captioning task even more challenging. In this paper, we propose an\nunified caption framework, M&M TGM, which mines multimodal topics in\nunsupervised fashion from data and guides the caption decoder with these\ntopics. Compared to pre-defined topics, the mined multimodal topics are more\nsemantically and visually coherent and can reflect the topic distribution of\nvideos better. We formulate the topic-aware caption generation as a multi-task\nlearning problem, in which we add a parallel task, topic prediction, in\naddition to the caption task. For the topic prediction task, we use the mined\ntopics as the teacher to train a student topic prediction model, which learns\nto predict the latent topics from multimodal contents of videos. The topic\nprediction provides intermediate supervision to the learning process. As for\nthe caption task, we propose a novel topic-aware decoder to generate more\naccurate and detailed video descriptions with the guidance from latent topics.\nThe entire learning procedure is end-to-end and it optimizes both tasks\nsimultaneously. The results from extensive experiments conducted on the MSR-VTT\nand Youtube2Text datasets demonstrate the effectiveness of our proposed model.\nM&M TGM not only outperforms prior state-of-the-art methods on multiple\nevaluation metrics and on both benchmark datasets, but also achieves better\ngeneralization ability.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1710.00284v1",
            "date": "2017-10-01 03:36:45+00:00",
            "title": "Efficient and Effective Single-Document Summarizations and A Word-Embedding Measurement of Quality",
            "authors": [
                "Liqun Shao",
                "Hao Zhang",
                "Ming Jia",
                "Jie Wang"
            ],
            "abstract": "Our task is to generate an effective summary for a given document with\nspecific realtime requirements. We use the softplus function to enhance keyword\nrankings to favor important sentences, based on which we present a number of\nsummarization algorithms using various keyword extraction and topic clustering\nmethods. We show that our algorithms meet the realtime requirements and yield\nthe best ROUGE recall scores on DUC-02 over all previously-known algorithms. We\nshow that our algorithms meet the realtime requirements and yield the best\nROUGE recall scores on DUC-02 over all previously-known algorithms. To evaluate\nthe quality of summaries without human-generated benchmarks, we define a\nmeasure called WESM based on word-embedding using Word Mover's Distance. We\nshow that the orderings of the ROUGE and WESM scores of our algorithms are\nhighly comparable, suggesting that WESM may serve as a viable alternative for\nmeasuring the quality of a summary.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.00286v1",
            "date": "2017-10-01 03:52:37+00:00",
            "title": "DTATG: An Automatic Title Generator based on Dependency Trees",
            "authors": [
                "Liqun Shao",
                "Jie Wang"
            ],
            "abstract": "We study automatic title generation for a given block of text and present a\nmethod called DTATG to generate titles. DTATG first extracts a small number of\ncentral sentences that convey the main meanings of the text and are in a\nsuitable structure for conversion into a title. DTATG then constructs a\ndependency tree for each of these sentences and removes certain branches using\na Dependency Tree Compression Model we devise. We also devise a title test to\ndetermine if a sentence can be used as a title. If a trimmed sentence passes\nthe title test, then it becomes a title candidate. DTATG selects the title\ncandidate with the highest ranking score as the final title. Our experiments\nshowed that DTATG can generate adequate titles. We also showed that\nDTATG-generated titles have higher F1 scores than those generated by the\nprevious methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.00372v2",
            "date": "2017-10-01 15:55:17+00:00",
            "title": "Mathematical foundations of matrix syntax",
            "authors": [
                "Roman Orus",
                "Roger Martin",
                "Juan Uriagereka"
            ],
            "abstract": "Matrix syntax is a formal model of syntactic relations in language. The\npurpose of this paper is to explain its mathematical foundations, for an\naudience with some formal background. We make an axiomatic presentation,\nmotivating each axiom on linguistic and practical grounds. The resulting\nmathematical structure resembles some aspects of quantum mechanics. Matrix\nsyntax allows us to describe a number of language phenomena that are otherwise\nvery difficult to explain, such as linguistic chains, and is arguably a more\neconomical theory of language than most of the theories proposed in the context\nof the minimalist program in linguistics. In particular, sentences are\nnaturally modelled as vectors in a Hilbert space with a tensor product\nstructure, built from 2x2 matrices belonging to some specific group.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02772v1",
            "date": "2017-10-08 02:51:26+00:00",
            "title": "Smarnet: Teaching Machines to Read and Comprehend Like Human",
            "authors": [
                "Zheqian Chen",
                "Rongqin Yang",
                "Bin Cao",
                "Zhou Zhao",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "Machine Comprehension (MC) is a challenging task in Natural Language\nProcessing field, which aims to guide the machine to comprehend a passage and\nanswer the given question. Many existing approaches on MC task are suffering\nthe inefficiency in some bottlenecks, such as insufficient lexical\nunderstanding, complex question-passage interaction, incorrect answer\nextraction and so on. In this paper, we address these problems from the\nviewpoint of how humans deal with reading tests in a scientific way.\nSpecifically, we first propose a novel lexical gating mechanism to dynamically\ncombine the words and characters representations. We then guide the machines to\nread in an interactive way with attention mechanism and memory network. Finally\nwe add a checking layer to refine the answer for insurance. The extensive\nexperiments on two popular datasets SQuAD and TriviaQA show that our method\nexceeds considerable performance than most state-of-the-art solutions at the\ntime of submission.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02861v1",
            "date": "2017-10-08 17:34:03+00:00",
            "title": "Clickbait detection using word embeddings",
            "authors": [
                "Vijayasaradhi Indurthi",
                "Subba Reddy Oota"
            ],
            "abstract": "Clickbait is a pejorative term describing web content that is aimed at\ngenerating online advertising revenue, especially at the expense of quality or\naccuracy, relying on sensationalist headlines or eye-catching thumbnail\npictures to attract click-throughs and to encourage forwarding of the material\nover online social networks. We use distributed word representations of the\nwords in the title as features to identify clickbaits in online news media. We\ntrain a machine learning model using linear regression to predict the cickbait\nscore of a given tweet. Our methods achieve an F1-score of 64.98\\% and an MSE\nof 0.0791. Compared to other methods, our method is simple, fast to train, does\nnot require extensive feature engineering and yet moderately effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02973v1",
            "date": "2017-10-09 07:36:18+00:00",
            "title": "LD-SDS: Towards an Expressive Spoken Dialogue System based on Linked-Data",
            "authors": [
                "Alexandros Papangelis",
                "Panagiotis Papadakos",
                "Margarita Kotti",
                "Yannis Stylianou",
                "Yannis Tzitzikas",
                "Dimitris Plexousakis"
            ],
            "abstract": "In this work we discuss the related challenges and describe an approach\ntowards the fusion of state-of-the-art technologies from the Spoken Dialogue\nSystems (SDS) and the Semantic Web and Information Retrieval domains. We\nenvision a dialogue system named LD-SDS that will support advanced, expressive,\nand engaging user requests, over multiple, complex, rich, and open-domain data\nsources that will leverage the wealth of the available Linked Data.\nSpecifically, we focus on: a) improving the identification, disambiguation and\nlinking of entities occurring in data sources and user input; b) offering\nadvanced query services for exploiting the semantics of the data, with\nreasoning and exploratory capabilities; and c) expanding the typical\ninformation seeking dialogue model (slot filling) to better reflect real-world\nconversational search scenarios.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.03255v2",
            "date": "2017-10-09 18:21:57+00:00",
            "title": "Multitask training with unlabeled data for end-to-end sign language fingerspelling recognition",
            "authors": [
                "Bowen Shi",
                "Karen Livescu"
            ],
            "abstract": "We address the problem of automatic American Sign Language fingerspelling\nrecognition from video. Prior work has largely relied on frame-level labels,\nhand-crafted features, or other constraints, and has been hampered by the\nscarcity of data for this task. We introduce a model for fingerspelling\nrecognition that addresses these issues. The model consists of an\nauto-encoder-based feature extractor and an attention-based neural\nencoder-decoder, which are trained jointly. The model receives a sequence of\nimage frames and outputs the fingerspelled word, without relying on any\nframe-level training labels or hand-crafted features. In addition, the\nauto-encoder subcomponent makes it possible to leverage unlabeled data to\nimprove the feature learning. The model achieves 11.6% and 4.4% absolute letter\naccuracy improvement respectively in signer-independent and signer-adapted\nfingerspelling recognition over previous approaches that required frame-level\ntraining labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03430v3",
            "date": "2017-10-10 07:26:50+00:00",
            "title": "Learning to Rank Question-Answer Pairs using Hierarchical Recurrent Encoder with Latent Topic Clustering",
            "authors": [
                "Seunghyun Yoon",
                "Joongbo Shin",
                "Kyomin Jung"
            ],
            "abstract": "In this paper, we propose a novel end-to-end neural architecture for ranking\ncandidate answers, that adapts a hierarchical recurrent neural network and a\nlatent topic clustering module. With our proposed model, a text is encoded to a\nvector representation from an word-level to a chunk-level to effectively\ncapture the entire meaning. In particular, by adapting the hierarchical\nstructure, our model shows very small performance degradations in longer text\ncomprehension while other state-of-the-art recurrent neural network models\nsuffer from it. Additionally, the latent topic clustering module extracts\nsemantic information from target samples. This clustering module is useful for\nany text related tasks by allowing each data sample to find its nearest topic\ncluster, thus helping the neural network model analyze the entire data. We\nevaluate our models on the Ubuntu Dialogue Corpus and consumer electronic\ndomain question answering dataset, which is related to Samsung products. The\nproposed model shows state-of-the-art results for ranking question-answer\npairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04142v1",
            "date": "2017-10-11 15:51:13+00:00",
            "title": "Bollywood Movie Corpus for Text, Images and Videos",
            "authors": [
                "Nishtha Madaan",
                "Sameep Mehta",
                "Mayank Saxena",
                "Aditi Aggarwal",
                "Taneea S Agrawaal",
                "Vrinda Malhotra"
            ],
            "abstract": "In past few years, several data-sets have been released for text and images.\nWe present an approach to create the data-set for use in detecting and removing\ngender bias from text. We also include a set of challenges we have faced while\ncreating this corpora. In this work, we have worked with movie data from\nWikipedia plots and movie trailers from YouTube. Our Bollywood Movie corpus\ncontains 4000 movies extracted from Wikipedia and 880 trailers extracted from\nYouTube which were released from 1970-2017. The corpus contains csv files with\nthe following data about each movie - Wikipedia title of movie, cast, plot\ntext, co-referenced plot text, soundtrack information, link to movie poster,\ncaption of movie poster, number of males in poster, number of females in\nposter. In addition to that, corresponding to each cast member the following\ndata is available - cast name, cast gender, cast verbs, cast adjectives, cast\nrelations, cast centrality, cast mentions. We present some preliminary results\non the task of bias removal which suggest that the data-set is quite useful for\nperforming such tasks.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1710.04203v1",
            "date": "2017-10-04 21:38:48+00:00",
            "title": "Crowdsourcing for Beyond Polarity Sentiment Analysis A Pure Emotion Lexicon",
            "authors": [
                "Giannis Haralabopoulos",
                "Elena Simperl"
            ],
            "abstract": "Sentiment analysis aims to uncover emotions conveyed through information. In\nits simplest form, it is performed on a polarity basis, where the goal is to\nclassify information with positive or negative emotion. Recent research has\nexplored more nuanced ways to capture emotions that go beyond polarity. For\nthese methods to work, they require a critical resource: a lexicon that is\nappropriate for the task at hand, in terms of the range of emotions it captures\ndiversity. In the past, sentiment analysis lexicons have been created by\nexperts, such as linguists and behavioural scientists, with strict rules.\nLexicon evaluation was also performed by experts or gold standards. In our\npaper, we propose a crowdsourcing method for lexicon acquisition, which is\nscalable, cost-effective, and doesn't require experts or gold standards. We\nalso compare crowd and expert evaluations of the lexicon, to assess the overall\nlexicon quality, and the evaluation capabilities of the crowd.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.04334v4",
            "date": "2017-10-12 00:56:13+00:00",
            "title": "DisSent: Sentence Representation Learning from Explicit Discourse Relations",
            "authors": [
                "Allen Nie",
                "Erin D. Bennett",
                "Noah D. Goodman"
            ],
            "abstract": "Learning effective representations of sentences is one of the core missions\nof natural language understanding. Existing models either train on a vast\namount of text, or require costly, manually curated sentence relation datasets.\nWe show that with dependency parsing and rule-based rubrics, we can curate a\nhigh quality sentence relation task by leveraging explicit discourse relations.\nWe show that our curated dataset provides an excellent signal for learning\nvector representations of sentence meaning, representing relations that can\nonly be determined when the meanings of two sentences are combined. We\ndemonstrate that the automatically curated corpus allows a bidirectional LSTM\nsentence encoder to yield high quality sentence embeddings and can serve as a\nsupervised fine-tuning dataset for larger models such as BERT. Our fixed\nsentence embeddings achieve high performance on a variety of transfer tasks,\nincluding SentEval, and we achieve state-of-the-art results on Penn Discourse\nTreebank's implicit relation prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06280v2",
            "date": "2017-10-17 13:46:59+00:00",
            "title": "Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions",
            "authors": [
                "Jun Hatori",
                "Yuta Kikuchi",
                "Sosuke Kobayashi",
                "Kuniyuki Takahashi",
                "Yuta Tsuboi",
                "Yuya Unno",
                "Wilson Ko",
                "Jethro Tan"
            ],
            "abstract": "Comprehension of spoken natural language is an essential component for robots\nto communicate with human effectively. However, handling unconstrained spoken\ninstructions is challenging due to (1) complex structures including a wide\nvariety of expressions used in spoken language and (2) inherent ambiguity in\ninterpretation of human instructions. In this paper, we propose the first\ncomprehensive system that can handle unconstrained spoken language and is able\nto effectively resolve ambiguity in spoken instructions. Specifically, we\nintegrate deep-learning-based object detection together with natural language\nprocessing technologies to handle unconstrained spoken instructions, and\npropose a method for robots to resolve instruction ambiguity through dialogue.\nThrough our experiments on both a simulated environment as well as a physical\nindustrial robot arm, we demonstrate the ability of our system to understand\nnatural instructions from human operators effectively, and how higher success\nrates of the object picking task can be achieved through an interactive\nclarification process.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1710.06303v1",
            "date": "2017-10-17 14:11:37+00:00",
            "title": "Describing Natural Images Containing Novel Objects with Knowledge Guided Assitance",
            "authors": [
                "Aditya Mogadala",
                "Umanga Bista",
                "Lexing Xie",
                "Achim Rettinger"
            ],
            "abstract": "Images in the wild encapsulate rich knowledge about varied abstract concepts\nand cannot be sufficiently described with models built only using image-caption\npairs containing selected objects. We propose to handle such a task with the\nguidance of a knowledge base that incorporate many abstract concepts. Our\nmethod is a two-step process where we first build a multi-entity-label image\nrecognition model to predict abstract concepts as image labels and then\nleverage them in the second step as an external semantic attention and\nconstrained inference in the caption generation model for describing images\nthat depict unseen/novel objects. Evaluations show that our models outperform\nmost of the prior work for out-of-domain captioning on MSCOCO and are useful\nfor integration of knowledge and vision in general.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1710.06481v2",
            "date": "2017-10-17 19:35:07+00:00",
            "title": "Constructing Datasets for Multi-hop Reading Comprehension Across Documents",
            "authors": [
                "Johannes Welbl",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "Most Reading Comprehension methods limit themselves to queries which can be\nanswered using a single sentence, paragraph, or document. Enabling models to\ncombine disjoint pieces of textual evidence would extend the scope of machine\ncomprehension methods, but currently there exist no resources to train and test\nthis capability. We propose a novel task to encourage the development of models\nfor text understanding across multiple documents and to investigate the limits\nof existing methods. In our task, a model learns to seek and combine evidence -\neffectively performing multi-hop (alias multi-step) inference. We devise a\nmethodology to produce datasets for this task, given a collection of\nquery-answer pairs and thematically linked documents. Two datasets from\ndifferent domains are induced, and we identify potential pitfalls and devise\ncircumvention strategies. We evaluate two previously proposed competitive\nmodels and find that one can integrate information across documents. However,\nboth models struggle to select relevant information, as providing documents\nguaranteed to be relevant greatly improves their performance. While the models\noutperform several strong baselines, their best accuracy reaches 42.9% compared\nto human performance at 74.0% - leaving ample room for improvement.",
            "journal ref": "Transactions of the Association for Computational Linguistics\n  (TACL), Vol 6 (2018), pages 287-302",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06922v2",
            "date": "2017-10-12 00:37:27+00:00",
            "title": "Emergent Translation in Multi-Agent Communication",
            "authors": [
                "Jason Lee",
                "Kyunghyun Cho",
                "Jason Weston",
                "Douwe Kiela"
            ],
            "abstract": "While most machine translation systems to date are trained on large parallel\ncorpora, humans learn language in a different way: by being grounded in an\nenvironment and interacting with other humans. In this work, we propose a\ncommunication game where two agents, native speakers of their own respective\nlanguages, jointly learn to solve a visual referential task. We find that the\nability to understand and translate a foreign language emerges as a means to\nachieve shared goals. The emergent translation is interactive and multimodal,\nand crucially does not require parallel corpora, but only monolingual,\nindependent text and corresponding images. Our proposed translation model\nachieves this by grounding the source and target languages into a shared visual\nmodality, and outperforms several baselines on both word-level and\nsentence-level translation tasks. Furthermore, we show that agents in a\nmultilingual community learn to translate better and faster than in a bilingual\ncommunication setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06923v1",
            "date": "2017-10-12 12:18:16+00:00",
            "title": "Adapting general-purpose speech recognition engine output for domain-specific natural language question answering",
            "authors": [
                "C. Anantaram",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Speech-based natural language question-answering interfaces to enterprise\nsystems are gaining a lot of attention. General-purpose speech engines can be\nintegrated with NLP systems to provide such interfaces. Usually,\ngeneral-purpose speech engines are trained on large `general' corpus. However,\nwhen such engines are used for specific domains, they may not recognize\ndomain-specific words well, and may produce erroneous output. Further, the\naccent and the environmental conditions in which the speaker speaks a sentence\nmay induce the speech engine to inaccurately recognize certain words. The\nsubsequent natural language question-answering does not produce the requisite\nresults as the question does not accurately represent what the speaker\nintended. Thus, the speech engine's output may need to be adapted for a domain\nbefore further natural language processing is carried out. We present two\nmechanisms for such an adaptation, one based on evolutionary development and\nthe other based on machine learning, and show how we can repair the\nspeech-output to make the subsequent natural language question-answering\nbetter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07177v1",
            "date": "2017-10-19 15:20:14+00:00",
            "title": "Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description",
            "authors": [
                "Desmond Elliott",
                "Stella Frank",
                "Lo\u00efc Barrault",
                "Fethi Bougares",
                "Lucia Specia"
            ],
            "abstract": "We present the results from the second shared task on multimodal machine\ntranslation and multilingual image description. Nine teams submitted 19 systems\nto two tasks. The multimodal translation task, in which the source sentence is\nsupplemented by an image, was extended with a new language (French) and two new\ntest sets. The multilingual image description task was changed such that at\ntest time, only the image is given. Compared to last year, multimodal systems\nimproved, but text-only systems remain competitive.",
            "journal ref": "Proceedings of the Second Conference on Machine Translation, 2017,\n  pp. 215--233",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07729v1",
            "date": "2017-10-20 23:43:26+00:00",
            "title": "Is space a word, too?",
            "authors": [
                "Jake Ryland Williams",
                "Giovanni C. Santia"
            ],
            "abstract": "For words, rank-frequency distributions have long been heralded for adherence\nto a potentially-universal phenomenon known as Zipf's law. The hypothetical\nform of this empirical phenomenon was refined by Ben\\^{i}ot Mandelbrot to that\nwhich is presently referred to as the Zipf-Mandelbrot law. Parallel to this,\nHerbet Simon proposed a selection model potentially explaining Zipf's law.\nHowever, a significant dispute between Simon and Mandelbrot, notable empirical\nexceptions, and the lack of a strong empirical connection between Simon's model\nand the Zipf-Mandelbrot law have left the questions of universality and\nmechanistic generation open. We offer a resolution to these issues by\nexhibiting how the dark matter of word segmentation, i.e., space, punctuation,\netc., connect the Zipf-Mandelbrot law to Simon's mechanistic process. This\nexplains Mandelbrot's refinement as no more than a fudge factor, accommodating\nthe effects of the exclusion of the rank-frequency dark matter. Thus,\nintegrating these non-word objects resolves a more-generalized rank-frequency\nlaw. Since this relies upon the integration of space, etc., we find support for\nthe hypothesis that $all$ are generated by common processes, indicating from a\nphysical perspective that space is a word, too.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08321v1",
            "date": "2017-10-23 15:08:38+00:00",
            "title": "Content Based Document Recommender using Deep Learning",
            "authors": [
                "Nishant Nikhil",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "With the recent advancements in information technology there has been a huge\nsurge in amount of data available. But information retrieval technology has not\nbeen able to keep up with this pace of information generation resulting in over\nspending of time for retrieving relevant information. Even though systems exist\nfor assisting users to search a database along with filtering and recommending\nrelevant information, but recommendation system which uses content of documents\nfor recommendation still have a long way to mature. Here we present a Deep\nLearning based supervised approach to recommend similar documents based on the\nsimilarity of content. We combine the C-DSSM model with Word2Vec distributed\nrepresentations of words to create a novel model to classify a document pair as\nrelevant/irrelavant by assigning a score to it. Using our model retrieval of\ndocuments can be done in O(1) time and the memory complexity is O(n), where n\nis number of documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08396v1",
            "date": "2017-10-23 17:24:12+00:00",
            "title": "Deep Health Care Text Classification",
            "authors": [
                "Vinayakumar R",
                "Barathi Ganesh HB",
                "Anand Kumar M",
                "Soman KP"
            ],
            "abstract": "Health related social media mining is a valuable apparatus for the early\nrecognition of the diverse antagonistic medicinal conditions. Mostly, the\nexisting methods are based on machine learning with knowledge-based learning.\nThis working note presents the Recurrent neural network (RNN) and Long\nshort-term memory (LSTM) based embedding for automatic health text\nclassification in the social media mining. For each task, two systems are built\nand that classify the tweet at the tweet level. RNN and LSTM are used for\nextracting features and non-linear activation function at the last layer\nfacilitates to distinguish the tweets of different categories. The experiments\nare conducted on 2nd Social Media Mining for Health Applications Shared Task at\nAMIA 2017. The experiment results are considerable; however the proposed method\nis appropriate for the health text classification. This is primarily due to the\nreason that, it doesn't rely on any feature engineering mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.08528v1",
            "date": "2017-10-23 22:12:51+00:00",
            "title": "A Two-Level Classification Approach for Detecting Clickbait Posts using Text-Based Features",
            "authors": [
                "Olga Papadopoulou",
                "Markos Zampoglou",
                "Symeon Papadopoulos",
                "Ioannis Kompatsiaris"
            ],
            "abstract": "The emergence of social media as news sources has led to the rise of\nclickbait posts attempting to attract users to click on article links without\ninforming them on the actual article content. This paper presents our efforts\nto create a clickbait detector inspired by fake news detection algorithms, and\nour submission to the Clickbait Challenge 2017. The detector is based almost\nexclusively on text-based features taken from previous work on clickbait\ndetection, our own work on fake post detection, and features we designed\nspecifically for the challenge. We use a two-level classification approach,\ncombining the outputs of 65 first-level classifiers in a second-level feature\nvector. We present our exploratory results with individual features and their\ncombinations, taken from the post text and the target article title, as well as\nfeature selection. While our own blind tests with the dataset led to an F-score\nof 0.63, our final evaluation in the Challenge only achieved an F-score of\n0.43. We explore the possible causes of this, and lay out potential future\nsteps to achieve more successful results.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1710.08634v1",
            "date": "2017-10-24 07:40:16+00:00",
            "title": "Using Multi-Label Classification for Improved Question Answering",
            "authors": [
                "Ricardo Usbeck",
                "Michael Hoffmann",
                "Michael R\u00f6der",
                "Jens Lehmann",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "A plethora of diverse approaches for question answering over RDF data have\nbeen developed in recent years. While the accuracy of these systems has\nincreased significantly over time, most systems still focus on particular types\nof questions or particular challenges in question answering. What is a curse\nfor single systems is a blessing for the combination of these systems. We show\nin this paper how machine learning techniques can be applied to create a more\naccurate question answering metasystem by reusing existing systems. In\nparticular, we develop a multi-label classification-based metasystem for\nquestion answering over 6 existing systems using an innovative set of 14\nquestion features. The metasystem outperforms the best single system by 14%\nF-measure on the recent QALD-6 benchmark. Furthermore, we analyzed the\ninfluence and correlation of the underlying features on the metasystem quality.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.09085v1",
            "date": "2017-10-25 06:26:28+00:00",
            "title": "Re-evaluating the need for Modelling Term-Dependence in Text Classification Problems",
            "authors": [
                "Sounak Banerjee",
                "Prasenjit Majumder",
                "Mandar Mitra"
            ],
            "abstract": "A substantial amount of research has been carried out in developing machine\nlearning algorithms that account for term dependence in text classification.\nThese algorithms offer acceptable performance in most cases but they are\nassociated with a substantial cost. They require significantly greater\nresources to operate. This paper argues against the justification of the higher\ncosts of these algorithms, based on their performance in text classification\nproblems. In order to prove the conjecture, the performance of one of the best\ndependence models is compared to several well established algorithms in text\nclassification. A very specific collection of datasets have been designed,\nwhich would best reflect the disparity in the nature of text data, that are\npresent in real world applications. The results show that even one of the best\nterm dependence models, performs decent at best when compared to other\nindependence models. Coupled with their substantially greater requirement for\nhardware resources for operation, this makes them an impractical choice for\nbeing used in real world scenarios.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.10498v1",
            "date": "2017-10-28 17:13:49+00:00",
            "title": "Topic Based Sentiment Analysis Using Deep Learning",
            "authors": [
                "Sharath T. S.",
                "Shubhangi Tandon"
            ],
            "abstract": "In this paper , we tackle Sentiment Analysis conditioned on a Topic in\nTwitter data using Deep Learning . We propose a 2-tier approach : In the first\nphase we create our own Word Embeddings and see that they do perform better\nthan state-of-the-art embeddings when used with standard classifiers. We then\nperform inference on these embeddings to learn more about a word with respect\nto all the topics being considered, and also the top n-influencing words for\neach topic. In the second phase we use these embeddings to predict the\nsentiment of the tweet with respect to a given topic, and all other topics\nunder discussion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10739v1",
            "date": "2017-10-30 01:55:10+00:00",
            "title": "Learning neural trans-dimensional random field language models with noise-contrastive estimation",
            "authors": [
                "Bin Wang",
                "Zhijian Ou"
            ],
            "abstract": "Trans-dimensional random field language models (TRF LMs) where sentences are\nmodeled as a collection of random fields, have shown close performance with\nLSTM LMs in speech recognition and are computationally more efficient in\ninference. However, the training efficiency of neural TRF LMs is not\nsatisfactory, which limits the scalability of TRF LMs on large training corpus.\nIn this paper, several techniques on both model formulation and parameter\nestimation are proposed to improve the training efficiency and the performance\nof neural TRF LMs. First, TRFs are reformulated in the form of exponential\ntilting of a reference distribution. Second, noise-contrastive estimation (NCE)\nis introduced to jointly estimate the model parameters and normalization\nconstants. Third, we extend the neural TRF LMs by marrying the deep\nconvolutional neural network (CNN) and the bidirectional LSTM into the\npotential function to extract the deep hierarchical features and\nbidirectionally sequential features. Utilizing all the above techniques enables\nthe successful and efficient training of neural TRF LMs on a 40x larger\ntraining set with only 1/3 training time and further reduces the WER with\nrelative reduction of 4.7% on top of a strong LSTM LM baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10777v1",
            "date": "2017-10-30 05:37:25+00:00",
            "title": "Understanding Hidden Memories of Recurrent Neural Networks",
            "authors": [
                "Yao Ming",
                "Shaozu Cao",
                "Ruixiang Zhang",
                "Zhen Li",
                "Yuanzhe Chen",
                "Yangqiu Song",
                "Huamin Qu"
            ],
            "abstract": "Recurrent neural networks (RNNs) have been successfully applied to various\nnatural language processing (NLP) tasks and achieved better results than\nconventional methods. However, the lack of understanding of the mechanisms\nbehind their effectiveness limits further improvements on their architectures.\nIn this paper, we present a visual analytics method for understanding and\ncomparing RNN models for NLP tasks. We propose a technique to explain the\nfunction of individual hidden state units based on their expected response to\ninput texts. We then co-cluster hidden state units and words based on the\nexpected response and visualize co-clustering results as memory chips and word\nclouds to provide more structured knowledge on RNNs' hidden states. We also\npropose a glyph-based sequence visualization based on aggregate information to\nanalyze the behavior of an RNN's hidden state at the sentence-level. The\nusability and effectiveness of our method are demonstrated through case studies\nand reviews from domain experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10994v3",
            "date": "2017-10-30 14:55:25+00:00",
            "title": "Conceptual Text Summarizer: A new model in continuous vector space",
            "authors": [
                "Mohammad Ebrahim Khademi",
                "Mohammad Fakhredanesh",
                "Seyed Mojtaba Hoseini"
            ],
            "abstract": "Traditional methods of summarization are not cost-effective and possible\ntoday. Extractive summarization is a process that helps to extract the most\nimportant sentences from a text automatically and generates a short informative\nsummary. In this work, we propose an unsupervised method to summarize Persian\ntexts. This method is a novel hybrid approach that clusters the concepts of the\ntext using deep learning and traditional statistical methods. First we produce\na word embedding based on Hamshahri2 corpus and a dictionary of word\nfrequencies. Then the proposed algorithm extracts the keywords of the document,\nclusters its concepts, and finally ranks the sentences to produce the summary.\nWe evaluated the proposed method on Pasokh single-document corpus using the\nROUGE evaluation measure. Without using any hand-crafted features, our proposed\nmethod achieves state-of-the-art results. We compared our unsupervised method\nwith the best supervised Persian methods and we achieved an overall improvement\nof ROUGE-2 recall score of 7.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11027v1",
            "date": "2017-10-30 15:56:03+00:00",
            "title": "Named Entity Recognition in Twitter using Images and Text",
            "authors": [
                "Diego Esteves",
                "Rafael Peres",
                "Jens Lehmann",
                "Giulio Napolitano"
            ],
            "abstract": "Named Entity Recognition (NER) is an important subtask of information\nextraction that seeks to locate and recognise named entities. Despite recent\nachievements, we still face limitations with correctly detecting and\nclassifying entities, prominently in short and noisy text, such as Twitter. An\nimportant negative aspect in most of NER approaches is the high dependency on\nhand-crafted features and domain-specific knowledge, necessary to achieve\nstate-of-the-art results. Thus, devising models to deal with such\nlinguistically complex contexts is still challenging. In this paper, we propose\na novel multi-level architecture that does not rely on any specific linguistic\nresource or encoded rule. Unlike traditional approaches, we use features\nextracted from images and text to classify named entities. Experimental tests\nagainst state-of-the-art NER for Twitter on the Ritter dataset present\ncompetitive results (0.59 F-measure), indicating that this approach may lead\ntowards better NER models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.11169v2",
            "date": "2017-10-30 18:27:19+00:00",
            "title": "Indirect Supervision for Relation Extraction using Question-Answer Pairs",
            "authors": [
                "Zeqiu Wu",
                "Xiang Ren",
                "Frank F. Xu",
                "Ji Li",
                "Jiawei Han"
            ],
            "abstract": "Automatic relation extraction (RE) for types of interest is of great\nimportance for interpreting massive text corpora in an efficient manner.\nTraditional RE models have heavily relied on human-annotated corpus for\ntraining, which can be costly in generating labeled data and become obstacles\nwhen dealing with more relation types. Thus, more RE extraction systems have\nshifted to be built upon training data automatically acquired by linking to\nknowledge bases (distant supervision). However, due to the incompleteness of\nknowledge bases and the context-agnostic labeling, the training data collected\nvia distant supervision (DS) can be very noisy. In recent years, as increasing\nattention has been brought to tackling question-answering (QA) tasks, user\nfeedback or datasets of such tasks become more accessible. In this paper, we\npropose a novel framework, ReQuest, to leverage question-answer pairs as an\nindirect source of supervision for relation extraction, and study how to use\nsuch supervision to reduce noise induced from DS. Our model jointly embeds\nrelation mentions, types, QA entity mention pairs and text features in two\nlow-dimensional spaces (RE and QA), where objects with same relation types or\nsemantically similar question-answer pairs have similar representations. Shared\nfeatures connect these two spaces, carrying clearer semantic knowledge from\nboth sources. ReQuest, then use these learned embeddings to estimate the types\nof test relation mentions. We formulate a global objective function and adopt a\nnovel margin-based QA loss to reduce noise in DS by exploiting semantic\nevidence from the QA dataset. Our experimental results achieve an average of\n11% improvement in F1 score on two public RE datasets combined with TREC QA\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11301v3",
            "date": "2017-10-31 02:23:15+00:00",
            "title": "A generalized parsing framework for Abstract Grammars",
            "authors": [
                "Daniel Harasim",
                "Chris Bruno",
                "Eva Portelance",
                "Martin Rohrmeier",
                "Timothy J. O'Donnell"
            ],
            "abstract": "This technical report presents a general framework for parsing a variety of\ngrammar formalisms. We develop a grammar formalism, called an Abstract Grammar,\nwhich is general enough to represent grammars at many levels of the hierarchy,\nincluding Context Free Grammars, Minimalist Grammars, and Generalized\nContext-free Grammars. We then develop a single parsing framework which is\ncapable of parsing grammars which are at least up to GCFGs on the hierarchy.\nOur parsing framework exposes a grammar interface, so that it can parse any\nparticular grammar formalism that can be reduced to an Abstract Grammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00185v1",
            "date": "2017-01-01 01:57:59+00:00",
            "title": "Self-Taught Convolutional Neural Networks for Short Text Clustering",
            "authors": [
                "Jiaming Xu",
                "Bo Xu",
                "Peng Wang",
                "Suncong Zheng",
                "Guanhua Tian",
                "Jun Zhao",
                "Bo Xu"
            ],
            "abstract": "Short text clustering is a challenging problem due to its sparseness of text\nrepresentation. Here we propose a flexible Self-Taught Convolutional neural\nnetwork framework for Short Text Clustering (dubbed STC^2), which can flexibly\nand successfully incorporate more useful semantic features and learn non-biased\ndeep text representation in an unsupervised manner. In our framework, the\noriginal raw text features are firstly embedded into compact binary codes by\nusing one existing unsupervised dimensionality reduction methods. Then, word\nembeddings are explored and fed into convolutional neural networks to learn\ndeep feature representations, meanwhile the output units are used to fit the\npre-trained binary codes in the training process. Finally, we get the optimal\nclusters by employing K-means to cluster the learned representations. Extensive\nexperimental results demonstrate that the proposed framework is effective,\nflexible and outperform several popular clustering methods when tested on three\npublic short text datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.00562v1",
            "date": "2017-01-03 01:15:53+00:00",
            "title": "End-to-End Attention based Text-Dependent Speaker Verification",
            "authors": [
                "Shi-Xiong Zhang",
                "Zhuo Chen",
                "Yong Zhao",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "A new type of End-to-End system for text-dependent speaker verification is\npresented in this paper. Previously, using the phonetically\ndiscriminative/speaker discriminative DNNs as feature extractors for speaker\nverification has shown promising results. The extracted frame-level (DNN\nbottleneck, posterior or d-vector) features are equally weighted and aggregated\nto compute an utterance-level speaker representation (d-vector or i-vector). In\nthis work we use speaker discriminative CNNs to extract the noise-robust\nframe-level features. These features are smartly combined to form an\nutterance-level speaker vector through an attention mechanism. The proposed\nattention model takes the speaker discriminative information and the phonetic\ninformation to learn the weights. The whole system, including the CNN and\nattention model, is joint optimized using an end-to-end criterion. The training\nalgorithm imitates exactly the evaluation process --- directly mapping a test\nutterance and a few target speaker utterances into a single verification score.\nThe algorithm can automatically select the most similar impostor for each\ntarget speaker to train the network. We demonstrated the effectiveness of the\nproposed end-to-end system on Windows $10$ \"Hey Cortana\" speaker verification\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00749v1",
            "date": "2017-01-03 17:17:34+00:00",
            "title": "Pyndri: a Python Interface to the Indri Search Engine",
            "authors": [
                "Christophe Van Gysel",
                "Evangelos Kanoulas",
                "Maarten de Rijke"
            ],
            "abstract": "We introduce pyndri, a Python interface to the Indri search engine. Pyndri\nallows to access Indri indexes from Python at two levels: (1) dictionary and\ntokenized document collection, (2) evaluating queries on the index. We hope\nthat with the release of pyndri, we will stimulate reproducible, open and\nfast-paced IR research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.00851v1",
            "date": "2017-01-03 22:26:10+00:00",
            "title": "Unsupervised neural and Bayesian models for zero-resource speech processing",
            "authors": [
                "Herman Kamper"
            ],
            "abstract": "In settings where only unlabelled speech data is available, zero-resource\nspeech technology needs to be developed without transcriptions, pronunciation\ndictionaries, or language modelling text. There are two central problems in\nzero-resource speech processing: (i) finding frame-level feature\nrepresentations which make it easier to discriminate between linguistic units\n(phones or words), and (ii) segmenting and clustering unlabelled speech into\nmeaningful units. In this thesis, we argue that a combination of top-down and\nbottom-up modelling is advantageous in tackling these two problems.\n  To address the problem of frame-level representation learning, we present the\ncorrespondence autoencoder (cAE), a neural network trained with weak top-down\nsupervision from an unsupervised term discovery system. By combining this\ntop-down supervision with unsupervised bottom-up initialization, the cAE yields\nmuch more discriminative features than previous approaches. We then present our\nunsupervised segmental Bayesian model that segments and clusters unlabelled\nspeech into hypothesized words. By imposing a consistent top-down segmentation\nwhile also using bottom-up knowledge from detected syllable boundaries, our\nsystem outperforms several others on multi-speaker conversational English and\nXitsonga speech data. Finally, we show that the clusters discovered by the\nsegmental Bayesian model can be made less speaker- and gender-specific by using\nfeatures from the cAE instead of traditional acoustic features.\n  In summary, the different models and systems presented in this thesis show\nthat both top-down and bottom-up modelling can improve representation learning,\nsegmentation and clustering of unlabelled speech data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00991v1",
            "date": "2017-01-04 13:06:18+00:00",
            "title": "World Literature According to Wikipedia: Introduction to a DBpedia-Based Framework",
            "authors": [
                "Christoph Hube",
                "Frank Fischer",
                "Robert J\u00e4schke",
                "Gerhard Lauer",
                "Mads Rosendahl Thomsen"
            ],
            "abstract": "Among the manifold takes on world literature, it is our goal to contribute to\nthe discussion from a digital point of view by analyzing the representation of\nworld literature in Wikipedia with its millions of articles in hundreds of\nlanguages. As a preliminary, we introduce and compare three different\napproaches to identify writers on Wikipedia using data from DBpedia, a\ncommunity project with the goal of extracting and providing structured\ninformation from Wikipedia. Equipped with our basic set of writers, we analyze\nhow they are represented throughout the 15 biggest Wikipedia language versions.\nWe combine intrinsic measures (mostly examining the connectedness of articles)\nwith extrinsic ones (analyzing how often articles are frequented by readers)\nand develop methods to evaluate our results. The better part of our findings\nseems to convey a rather conservative, old-fashioned version of world\nliterature, but a version derived from reproducible facts revealing an implicit\nliterary canon based on the editing and reading behavior of millions of people.\nWhile still having to solve some known issues, the introduced methods will help\nus build an observatory of world literature to further investigate its\nrepresentativeness and biases.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.01417v1",
            "date": "2017-01-05 18:36:26+00:00",
            "title": "Exploration of Proximity Heuristics in Length Normalization",
            "authors": [
                "Pranav Agrawal"
            ],
            "abstract": "Ranking functions used in information retrieval are primarily used in the\nsearch engines and they are often adopted for various language processing\napplications. However, features used in the construction of ranking functions\nshould be analyzed before applying it on a data set. This paper gives\nguidelines on construction of generalized ranking functions with\napplication-dependent features. The paper prescribes a specific case of a\ngeneralized function for recommendation system using feature engineering\nguidelines on the given data set. The behavior of both generalized and specific\nfunctions are studied and implemented on the unstructured textual data. The\nproximity feature based ranking function has outperformed by 52% from regular\nBM25.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.01811v1",
            "date": "2017-01-07 09:58:49+00:00",
            "title": "Structural Attention Neural Networks for improved sentiment analysis",
            "authors": [
                "Filippos Kokkinos",
                "Alexandros Potamianos"
            ],
            "abstract": "We introduce a tree-structured attention neural network for sentences and\nsmall phrases and apply it to the problem of sentiment classification. Our\nmodel expands the current recursive models by incorporating structural\ninformation around a node of a syntactic tree using both bottom-up and top-down\ninformation propagation. Also, the model utilizes structural attention to\nidentify the most salient representations during the construction of the\nsyntactic tree. To our knowledge, the proposed models achieve state of the art\nperformance on the Stanford Sentiment Treebank dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02025v2",
            "date": "2017-01-08 22:20:22+00:00",
            "title": "Multi-level Representations for Fine-Grained Typing of Knowledge Base Entities",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Entities are essential elements of natural language. In this paper, we\npresent methods for learning multi-level representations of entities on three\ncomplementary levels: character (character patterns in entity names extracted,\ne.g., by neural networks), word (embeddings of words in entity names) and\nentity (entity embeddings). We investigate state-of-the-art learning methods on\neach level and find large differences, e.g., for deep learning models,\ntraditional ngram features and the subword model of fasttext (Bojanowski et\nal., 2016) on the character level; for word2vec (Mikolov et al., 2013) on the\nword level; and for the order-aware model wang2vec (Ling et al., 2015a) on the\nentity level. We confirm experimentally that each level of representation\ncontributes complementary information and a joint representation of all three\nlevels improves the existing embedding based baseline for fine-grained entity\ntyping by a large margin. Additionally, we show that adding information from\nentity descriptions further improves multi-level representations of entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02185v2",
            "date": "2017-01-09 14:13:23+00:00",
            "title": "Crowdsourcing Ground Truth for Medical Relation Extraction",
            "authors": [
                "Anca Dumitrache",
                "Lora Aroyo",
                "Chris Welty"
            ],
            "abstract": "Cognitive computing systems require human labeled data for evaluation, and\noften for training. The standard practice used in gathering this data minimizes\ndisagreement between annotators, and we have found this results in data that\nfails to account for the ambiguity inherent in language. We have proposed the\nCrowdTruth method for collecting ground truth through crowdsourcing, that\nreconsiders the role of people in machine learning based on the observation\nthat disagreement between annotators provides a useful signal for phenomena\nsuch as ambiguity in the text. We report on using this method to build an\nannotated data set for medical relation extraction for the $cause$ and $treat$\nrelations, and how this data performed in a supervised training experiment. We\ndemonstrate that by modeling ambiguity, labeled data gathered from crowd\nworkers can (1) reach the level of quality of domain experts for this task\nwhile reducing the cost, and (2) provide better training data at scale than\ndistant supervision. We further propose and validate new weighted measures for\nprecision, recall, and F-measure, that account for ambiguity in both human and\nmachine performance on this task.",
            "journal ref": "ACM Transactions on Interactive Intelligent Systems (TIIS) Volume\n  8 Issue 2, July 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02481v3",
            "date": "2017-01-10 08:59:38+00:00",
            "title": "Implicitly Incorporating Morphological Information into Word Embedding",
            "authors": [
                "Yang Xu",
                "Jiawei Liu"
            ],
            "abstract": "In this paper, we propose three novel models to enhance word embedding by\nimplicitly using morphological information. Experiments on word similarity and\nsyntactic analogy show that the implicit models are superior to traditional\nexplicit ones. Our models outperform all state-of-the-art baselines and\nsignificantly improve the performance on both tasks. Moreover, our performance\non the smallest corpus is similar to the performance of CBOW on the corpus\nwhich is five times the size of ours. Parameter analysis indicates that the\nimplicit models can supplement semantic information during the word embedding\ntraining process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02593v2",
            "date": "2017-01-10 14:01:47+00:00",
            "title": "A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling",
            "authors": [
                "Diego Marcheggiani",
                "Anton Frolov",
                "Ivan Titov"
            ],
            "abstract": "We introduce a simple and accurate neural model for dependency-based semantic\nrole labeling. Our model predicts predicate-argument dependencies relying on\nstates of a bidirectional LSTM encoder. The semantic role labeler achieves\ncompetitive performance on English, even without any kind of syntactic\ninformation and only using local inference. However, when automatically\npredicted part-of-speech tags are provided as input, it substantially\noutperforms all previous local models and approaches the best reported results\non the English CoNLL-2009 dataset. We also consider Chinese, Czech and Spanish\nwhere our approach also achieves competitive results. Syntactic parsers are\nunreliable on out-of-domain data, so standard (i.e., syntactically-informed)\nSRL models are hindered when tested in this setting. Our syntax-agnostic model\nappears more robust, resulting in the best reported results on standard\nout-of-domain test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02854v4",
            "date": "2017-01-11 06:02:44+00:00",
            "title": "Towards Decoding as Continuous Optimization in Neural Machine Translation",
            "authors": [
                "Cong Duy Vu Hoang",
                "Gholamreza Haffari",
                "Trevor Cohn"
            ],
            "abstract": "We propose a novel decoding approach for neural machine translation (NMT)\nbased on continuous optimisation. We convert decoding - basically a discrete\noptimization problem - into a continuous optimization problem. The resulting\nconstrained continuous optimisation problem is then tackled using\ngradient-based methods. Our powerful decoding framework enables decoding\nintractable models such as the intersection of left-to-right and right-to-left\n(bidirectional) as well as source-to-target and target-to-source (bilingual)\nNMT models. Our empirical results show that our decoding framework is\neffective, and leads to substantial improvements in translations generated from\nthe intersected models where the typical greedy or beam search is not feasible.\nWe also compare our framework against reranking, and analyse its advantages and\ndisadvantages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03038v2",
            "date": "2017-01-11 16:07:27+00:00",
            "title": "Decoding with Finite-State Transducers on GPUs",
            "authors": [
                "Arturo Argueta",
                "David Chiang"
            ],
            "abstract": "Weighted finite automata and transducers (including hidden Markov models and\nconditional random fields) are widely used in natural language processing (NLP)\nto perform tasks such as morphological analysis, part-of-speech tagging,\nchunking, named entity recognition, speech recognition, and others.\nParallelizing finite state algorithms on graphics processing units (GPUs) would\nbenefit many areas of NLP. Although researchers have implemented GPU versions\nof basic graph algorithms, limited previous work, to our knowledge, has been\ndone on GPU algorithms for weighted finite automata. We introduce a GPU\nimplementation of the Viterbi and forward-backward algorithm, achieving\ndecoding speedups of up to 5.2x over our serial implementation running on\ndifferent computer architectures and 6093x over OpenFST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03492v1",
            "date": "2017-01-12 20:14:52+00:00",
            "title": "Scalable, Trie-based Approximate Entity Extraction for Real-Time Financial Transaction Screening",
            "authors": [
                "Emrah Budur"
            ],
            "abstract": "Financial institutions have to screen their transactions to ensure that they\nare not affiliated with terrorism entities. Developing appropriate solutions to\ndetect such affiliations precisely while avoiding any kind of interruption to\nlarge amount of legitimate transactions is essential. In this paper, we present\nbuilding blocks of a scalable solution that may help financial institutions to\nbuild their own software to extract terrorism entities out of both structured\nand unstructured financial messages in real time and with approximate\nsimilarity matching approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03578v1",
            "date": "2017-01-13 07:26:00+00:00",
            "title": "Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network",
            "authors": [
                "Seunghyun Yoon",
                "Hyeongu Yun",
                "Yuna Kim",
                "Gyu-tae Park",
                "Kyomin Jung"
            ],
            "abstract": "In this paper, we propose an efficient transfer leaning methods for training\na personalized language model using a recurrent neural network with long\nshort-term memory architecture. With our proposed fast transfer learning\nschemes, a general language model is updated to a personalized language model\nwith a small amount of user data and a limited computing resource. These\nmethods are especially useful for a mobile device environment while the data is\nprevented from transferring out of the device for privacy purposes. Through\nexperiments on dialogue data in a drama, it is verified that our transfer\nlearning methods have successfully generated the personalized language model,\nwhose output is more similar to the personal language style in both qualitative\nand quantitative aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03682v1",
            "date": "2017-01-13 14:20:06+00:00",
            "title": "LIDE: Language Identification from Text Documents",
            "authors": [
                "Priyank Mathur",
                "Arkajyoti Misra",
                "Emrah Budur"
            ],
            "abstract": "The increase in the use of microblogging came along with the rapid growth on\nshort linguistic data. On the other hand deep learning is considered to be the\nnew frontier to extract meaningful information out of large amount of raw data\nin an automated manner. In this study, we engaged these two emerging fields to\ncome up with a robust language identifier on demand, namely Language\nIdentification Engine (LIDE). As a result, we achieved 95.12% accuracy in\nDiscriminating between Similar Languages (DSL) Shared Task 2015 dataset, which\nis comparable to the maximum reported accuracy of 95.54% achieved so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03947v1",
            "date": "2017-01-14 16:47:51+00:00",
            "title": "Balancing Novelty and Salience: Adaptive Learning to Rank Entities for Timeline Summarization of High-impact Events",
            "authors": [
                "Tuan Tran",
                "Claudia Nieder\u00e9e",
                "Nattiya Kanhabua",
                "Ujwal Gadiraju",
                "Avishek Anand"
            ],
            "abstract": "Long-running, high-impact events such as the Boston Marathon bombing often\ndevelop through many stages and involve a large number of entities in their\nunfolding. Timeline summarization of an event by key sentences eases story\ndigestion, but does not distinguish between what a user remembers and what she\nmight want to re-check. In this work, we present a novel approach for timeline\nsummarization of high-impact events, which uses entities instead of sentences\nfor summarizing the event at each individual point in time. Such entity\nsummaries can serve as both (1) important memory cues in a retrospective event\nconsideration and (2) pointers for personalized event exploration. In order to\nautomatically create such summaries, it is crucial to identify the \"right\"\nentities for inclusion. We propose to learn a ranking function for entities,\nwith a dynamically adapted trade-off between the in-document salience of\nentities and the informativeness of entities across documents, i.e., the level\nof new information associated with an entity for a time point under\nconsideration. Furthermore, for capturing collective attention for an entity we\nuse an innovative soft labeling approach based on Wikipedia. Our experiments on\na real large news datasets confirm the effectiveness of the proposed methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.04024v3",
            "date": "2017-01-15 10:38:17+00:00",
            "title": "A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue",
            "authors": [
                "Mihail Eric",
                "Christopher D. Manning"
            ],
            "abstract": "Task-oriented dialogue focuses on conversational agents that participate in\nuser-initiated dialogues on domain-specific topics. In contrast to chatbots,\nwhich simply seek to sustain open-ended meaningful discourse, existing\ntask-oriented agents usually explicitly model user intent and belief states.\nThis paper examines bypassing such an explicit representation by depending on a\nlatent neural embedding of state and learning selective attention to dialogue\nhistory together with copying to incorporate relevant prior context. We\ncomplement recent work by showing the effectiveness of simple\nsequence-to-sequence neural architectures with a copy mechanism. Our model\noutperforms more complex memory-augmented models by 7% in per-response\ngeneration and is on par with the current state-of-the-art on DSTC2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.04039v2",
            "date": "2017-01-15 13:34:43+00:00",
            "title": "The Birth of Collective Memories: Analyzing Emerging Entities in Text Streams",
            "authors": [
                "David Graus",
                "Daan Odijk",
                "Maarten de Rijke"
            ],
            "abstract": "We study how collective memories are formed online. We do so by tracking\nentities that emerge in public discourse, that is, in online text streams such\nas social media and news streams, before they are incorporated into Wikipedia,\nwhich, we argue, can be viewed as an online place for collective memory. By\ntracking how entities emerge in public discourse, i.e., the temporal patterns\nbetween their first mention in online text streams and subsequent incorporation\ninto collective memory, we gain insights into how the collective remembrance\nprocess happens online. Specifically, we analyze nearly 80,000 entities as they\nemerge in online text streams before they are incorporated into Wikipedia. The\nonline text streams we use for our analysis comprise of social media and news\nstreams, and span over 579 million documents in a timespan of 18 months. We\ndiscover two main emergence patterns: entities that emerge in a \"bursty\"\nfashion, i.e., that appear in public discourse without a precedent, blast into\nactivity and transition into collective memory. Other entities display a\n\"delayed\" pattern, where they appear in public discourse, experience a period\nof inactivity, and then resurface before transitioning into our cultural\ncollective memory.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.04292v1",
            "date": "2017-01-16 14:02:19+00:00",
            "title": "Semantic classifier approach to document classification",
            "authors": [
                "Piotr Borkowski",
                "Krzysztof Ciesielski",
                "Mieczys\u0142aw A. K\u0142opotek"
            ],
            "abstract": "In this paper we propose a new document classification method, bridging\ndiscrepancies (so-called semantic gap) between the training set and the\napplication sets of textual data. We demonstrate its superiority over classical\ntext classification approaches, including traditional classifier ensembles. The\nmethod consists in combining a document categorization technique with a single\nclassifier or a classifier ensemble (SEMCOM algorithm - Committee with Semantic\nCategorizer).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.04653v1",
            "date": "2017-01-17 12:53:19+00:00",
            "title": "Community Question Answering Platforms vs. Twitter for Predicting Characteristics of Urban Neighbourhoods",
            "authors": [
                "Marzieh Saeidi",
                "Alessandro Venerandi",
                "Licia Capra",
                "Sebastian Riedel"
            ],
            "abstract": "In this paper, we investigate whether text from a Community Question\nAnswering (QA) platform can be used to predict and describe real-world\nattributes. We experiment with predicting a wide range of 62 demographic\nattributes for neighbourhoods of London. We use the text from QA platform of\nYahoo! Answers and compare our results to the ones obtained from Twitter\nmicroblogs. Outcomes show that the correlation between the predicted\ndemographic attributes using text from Yahoo! Answers discussions and the\nobserved demographic attributes can reach an average Pearson correlation\ncoefficient of \\r{ho} = 0.54, slightly higher than the predictions obtained\nusing Twitter data. Our qualitative analysis indicates that there is semantic\nrelatedness between the highest correlated terms extracted from both datasets\nand their relative demographic attributes. Furthermore, the correlations\nhighlight the different natures of the information contained in Yahoo! Answers\nand Twitter. While the former seems to offer a more encyclopedic content, the\nlatter provides information related to the current sociocultural aspects or\nphenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05334v1",
            "date": "2017-01-19 08:50:37+00:00",
            "title": "Fuzzy Ontology-Based Sentiment Analysis of Transportation and City Feature Reviews for Safe Traveling",
            "authors": [
                "Farman Ali",
                "D. Kwak",
                "Pervez Khan",
                "S. M. Riazul Islam",
                "K. H. Kim",
                "K. S. Kwak"
            ],
            "abstract": "Traffic congestion is rapidly increasing in urban areas, particularly in mega\ncities. To date, there exist a few sensor network based systems to address this\nproblem. However, these techniques are not suitable enough in terms of\nmonitoring an entire transportation system and delivering emergency services\nwhen needed. These techniques require real-time data and intelligent ways to\nquickly determine traffic activity from useful information. In addition, these\nexisting systems and websites on city transportation and travel rely on rating\nscores for different factors (e.g., safety, low crime rate, cleanliness, etc.).\nThese rating scores are not efficient enough to deliver precise information,\nwhereas reviews or tweets are significant, because they help travelers and\ntransportation administrators to know about each aspect of the city. However,\nit is difficult for travelers to read, and for transportation systems to\nprocess, all reviews and tweets to obtain expressive sentiments regarding the\nneeds of the city. The optimum solution for this kind of problem is analyzing\nthe information available on social network platforms and performing sentiment\nanalysis. On the other hand, crisp ontology-based frameworks cannot extract\nblurred information from tweets and reviews; therefore, they produce inadequate\nresults. In this regard, this paper proposes fuzzy ontology-based sentiment\nanalysis and SWRL rule-based decision-making to monitor transportation\nactivities and to make a city- feature polarity map for travelers. This system\nretrieves reviews and tweets related to city features and transportation\nactivities. The feature opinions are extracted from these retrieved data, and\nthen fuzzy ontology is used to determine the transportation and city-feature\npolarity. A fuzzy ontology and an intelligent system prototype are developed by\nusing Prot\\'eg\\'e OWL and Java, respectively.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1701.05847v1",
            "date": "2017-01-20 16:36:09+00:00",
            "title": "End-To-End Visual Speech Recognition With LSTMs",
            "authors": [
                "Stavros Petridis",
                "Zuwei Li",
                "Maja Pantic"
            ],
            "abstract": "Traditional visual speech recognition systems consist of two stages, feature\nextraction and classification. Recently, several deep learning approaches have\nbeen presented which automatically extract features from the mouth images and\naim to replace the feature extraction stage. However, research on joint\nlearning of features and classification is very limited. In this work, we\npresent an end-to-end visual speech recognition system based on Long-Short\nMemory (LSTM) networks. To the best of our knowledge, this is the first model\nwhich simultaneously learns to extract features directly from the pixels and\nperform classification and also achieves state-of-the-art performance in visual\nspeech classification. The model consists of two streams which extract features\ndirectly from the mouth and difference images, respectively. The temporal\ndynamics in each stream are modelled by an LSTM and the fusion of the two\nstreams takes place via a Bidirectional LSTM (BLSTM). An absolute improvement\nof 9.7% over the base line is reported on the OuluVS2 database, and 1.5% on the\nCUAVE database when compared with other methods which use a similar visual\nfront-end.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1701.07481v3",
            "date": "2017-01-25 20:40:56+00:00",
            "title": "Learning Word-Like Units from Joint Audio-Visual Analysis",
            "authors": [
                "David Harwath",
                "James R. Glass"
            ],
            "abstract": "Given a collection of images and spoken audio captions, we present a method\nfor discovering word-like acoustic units in the continuous speech signal and\ngrounding them to semantically relevant image regions. For example, our model\nis able to detect spoken instances of the word 'lighthouse' within an utterance\nand associate them with image regions containing lighthouses. We do not use any\nform of conventional automatic speech recognition, nor do we use any text\ntranscriptions or conventional linguistic annotations. Our model effectively\nimplements a form of spoken language acquisition, in which the computer learns\nnot only to recognize word categories by sound, but also to enrich the words it\nlearns with semantics by grounding them in images.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.07795v1",
            "date": "2017-01-26 17:59:38+00:00",
            "title": "Match-Tensor: a Deep Relevance Model for Search",
            "authors": [
                "Aaron Jaech",
                "Hetunandan Kamisetty",
                "Eric Ringger",
                "Charlie Clarke"
            ],
            "abstract": "The application of Deep Neural Networks for ranking in search engines may\nobviate the need for the extensive feature engineering common to current\nlearning-to-rank methods. However, we show that combining simple relevance\nmatching features like BM25 with existing Deep Neural Net models often\nsubstantially improves the accuracy of these models, indicating that they do\nnot capture essential local relevance matching signals. We describe a novel\ndeep Recurrent Neural Net-based model that we call Match-Tensor. The\narchitecture of the Match-Tensor model simultaneously accounts for both local\nrelevance matching and global topicality signals allowing for a rich interplay\nbetween them when computing the relevance of a document to a query. On a large\nheld-out test set consisting of social media documents, we demonstrate not only\nthat Match-Tensor outperforms BM25 and other classes of DNNs but also that it\nlargely subsumes signals present in these models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.07955v1",
            "date": "2017-01-27 06:30:21+00:00",
            "title": "Statistical Analysis on Bangla Newspaper Data to Extract Trending Topic and Visualize Its Change Over Time",
            "authors": [
                "Syed Mehedi Hasan Nirob",
                "Md. Kazi Nayeem",
                "Md. Saiful Islam"
            ],
            "abstract": "Trending topic of newspapers is an indicator to understand the situation of a\ncountry and also a way to evaluate the particular newspaper. This paper\nrepresents a model describing few techniques to select trending topics from\nBangla Newspaper. Topics that are discussed more frequently than other in\nBangla newspaper will be marked and how a very famous topic loses its\nimportance with the change of time and another topic takes its place will be\ndemonstrated. Data from two popular Bangla Newspaper with date and time were\ncollected. Statistical analysis was performed after on these data after\npreprocessing. Popular and most used keywords were extracted from the stream of\nBangla keyword with this analysis. This model can also cluster category wise\nnews trend or a list of news trend in daily or weekly basis with enough data. A\npattern can be found on their news trend too. Comparison among past news trend\nof Bangla newspapers will give a visualization of the situation of Bangladesh.\nThis visualization will be helpful to predict future trending topics of Bangla\nNewspaper.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.08156v2",
            "date": "2017-01-27 12:38:47+00:00",
            "title": "A Comprehensive Survey on Bengali Phoneme Recognition",
            "authors": [
                "Sadia Tasnim Swarna",
                "Shamim Ehsan",
                "Md. Saiful Islam",
                "Marium E Jannat"
            ],
            "abstract": "Hidden Markov model based various phoneme recognition methods for Bengali\nlanguage is reviewed. Automatic phoneme recognition for Bengali language using\nmultilayer neural network is reviewed. Usefulness of multilayer neural network\nover single layer neural network is discussed. Bangla phonetic feature table\nconstruction and enhancement for Bengali speech recognition is also discussed.\nComparison among these methods is discussed.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1701.08694v1",
            "date": "2017-01-27 13:08:08+00:00",
            "title": "A Comparative Study on Different Types of Approaches to Bengali document Categorization",
            "authors": [
                "Md. Saiful Islam",
                "Fazla Elahi Md Jubayer",
                "Syed Ikhtiar Ahmed"
            ],
            "abstract": "Document categorization is a technique where the category of a document is\ndetermined. In this paper three well-known supervised learning techniques which\nare Support Vector Machine(SVM), Na\\\"ive Bayes(NB) and Stochastic Gradient\nDescent(SGD) compared for Bengali document categorization. Besides classifier,\nclassification also depends on how feature is selected from dataset. For\nanalyzing those classifier performances on predicting a document against twelve\ncategories several feature selection techniques are also applied in this\narticle namely Chi square distribution, normalized TFIDF (term\nfrequency-inverse document frequency) with word analyzer. So, we attempt to\nexplore the efficiency of those three-classification algorithms by using two\ndifferent feature selection techniques in this article.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08706v1",
            "date": "2017-01-27 12:54:52+00:00",
            "title": "Document Decomposition of Bangla Printed Text",
            "authors": [
                "Md. Fahad Hasan",
                "Tasmin Afroz",
                "Sabir Ismail",
                "Md. Saiful Islam"
            ],
            "abstract": "Today all kind of information is getting digitized and along with all this\ndigitization, the huge archive of various kinds of documents is being digitized\ntoo. We know that, Optical Character Recognition is the method through which,\nnewspapers and other paper documents convert into digital resources. But, it is\na fact that this method works on texts only. As a result, if we try to process\nany document which contains non-textual zones, then we will get garbage texts\nas output. That is why; in order to digitize documents properly they should be\nprepossessed carefully. And while preprocessing, segmenting document in\ndifferent regions according to the category properly is most important. But,\nthe Optical Character Recognition processes available for Bangla language have\nno such algorithm that can categorize a newspaper/book page fully. So we worked\nto decompose a document into its several parts like headlines, sub headlines,\ncolumns, images etc. And if the input is skewed and rotated, then the input was\nalso deskewed and de-rotated. To decompose any Bangla document we found out the\nedges of the input image. Then we find out the horizontal and vertical area of\nevery pixel where it lies in. Later on the input image was cut according to\nthese areas. Then we pick each and every sub image and found out their\nheight-width ratio, line height. Then according to these values the sub images\nwere categorized. To deskew the image we found out the skew angle and de skewed\nthe image according to this angle. To de-rotate the image we used the line\nheight, matra line, pixel ratio of matra line.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1701.09123v1",
            "date": "2017-01-31 16:36:06+00:00",
            "title": "Robust Multilingual Named Entity Recognition with Shallow Semi-Supervised Features",
            "authors": [
                "Rodrigo Agerri",
                "German Rigau"
            ],
            "abstract": "We present a multilingual Named Entity Recognition approach based on a robust\nand general set of features across languages and datasets. Our system combines\nshallow local information with clustering semi-supervised features induced on\nlarge amounts of unlabeled text. Understanding via empirical experimentation\nhow to effectively combine various types of clustering features allows us to\nseamlessly export our system to other datasets and languages. The result is a\nsimple but highly competitive system which obtains state of the art results\nacross five languages and twelve datasets. The results are reported on standard\nshared task evaluation data such as CoNLL for English, Spanish and Dutch.\nFurthermore, and despite the lack of linguistically motivated features, we also\nreport best results for languages such as Basque and German. In addition, we\ndemonstrate that our method also obtains very competitive results even when the\namount of supervised data is cut by half, alleviating the dependency on\nmanually annotated data. Finally, the results show that our emphasis on\nclustering features is crucial to develop robust out-of-domain models. The\nsystem and models are freely available to facilitate its use and guarantee the\nreproducibility of results.",
            "journal ref": "Artificial Intelligence, 238, 63-82 (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00061v1",
            "date": "2017-06-30 22:57:50+00:00",
            "title": "Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English",
            "authors": [
                "Su Lin Blodgett",
                "Brendan O'Connor"
            ],
            "abstract": "We highlight an important frontier in algorithmic fairness: disparity in the\nquality of natural language processing algorithms when applied to language from\nauthors of different social groups. For example, current systems sometimes\nanalyze the language of females and minorities more poorly than they do of\nwhites and males. We conduct an empirical analysis of racial disparity in\nlanguage identification for tweets written in African-American English, and\ndiscuss implications of disparity in NLP.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1707.00189v3",
            "date": "2017-07-01 18:42:29+00:00",
            "title": "Content-Based Weak Supervision for Ad-Hoc Re-Ranking",
            "authors": [
                "Sean MacAvaney",
                "Andrew Yates",
                "Kai Hui",
                "Ophir Frieder"
            ],
            "abstract": "One challenge with neural ranking is the need for a large amount of\nmanually-labeled relevance judgments for training. In contrast with prior work,\nwe examine the use of weak supervision sources for training that yield pseudo\nquery-document pairs that already exhibit relevance (e.g., newswire\nheadline-content pairs and encyclopedic heading-paragraph pairs). We also\npropose filtering techniques to eliminate training samples that are too far out\nof domain using two techniques: a heuristic-based approach and novel supervised\nfilter that re-purposes a neural ranker. Using several leading neural ranking\narchitectures and multiple weak supervision datasets, we show that these\nsources of training pairs are effective on their own (outperforming prior weak\nsupervision techniques), and that filtering can further improve performance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.00201v2",
            "date": "2017-07-01 20:50:33+00:00",
            "title": "Rank-1 Constrained Multichannel Wiener Filter for Speech Recognition in Noisy Environments",
            "authors": [
                "Ziteng Wang",
                "Emmanuel Vincent",
                "Romain Serizel",
                "Yonghong Yan"
            ],
            "abstract": "Multichannel linear filters, such as the Multichannel Wiener Filter (MWF) and\nthe Generalized Eigenvalue (GEV) beamformer are popular signal processing\ntechniques which can improve speech recognition performance. In this paper, we\npresent an experimental study on these linear filters in a specific speech\nrecognition task, namely the CHiME-4 challenge, which features real recordings\nin multiple noisy environments. Specifically, the rank-1 MWF is employed for\nnoise reduction and a new constant residual noise power constraint is derived\nwhich enhances the recognition performance. To fulfill the underlying rank-1\nassumption, the speech covariance matrix is reconstructed based on eigenvectors\nor generalized eigenvectors. Then the rank-1 constrained MWF is evaluated with\nalternative multichannel linear filters under the same framework, which\ninvolves a Bidirectional Long Short-Term Memory (BLSTM) network for mask\nestimation. The proposed filter outperforms alternative ones, leading to a 40%\nrelative Word Error Rate (WER) reduction compared with the baseline Weighted\nDelay and Sum (WDAS) beamformer on the real test set, and a 15% relative WER\nreduction compared with the GEV-BAN method. The results also suggest that the\nspeech recognition accuracy correlates more with the Mel-frequency cepstral\ncoefficients (MFCC) feature variance than with the noise reduction or the\nspeech distortion level.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1707.01090v1",
            "date": "2017-07-04 09:58:40+00:00",
            "title": "Hidden-Markov-Model Based Speech Enhancement",
            "authors": [
                "Daniel Dzibela",
                "Armin Sehr"
            ],
            "abstract": "The goal of this contribution is to use a parametric speech synthesis system\nfor reducing background noise and other interferences from recorded speech\nsignals. In a first step, Hidden Markov Models of the synthesis system are\ntrained.\n  Two adequate training corpora consisting of text and corresponding speech\nfiles have been set up and cleared of various faults, including inaudible\nutterances or incorrect assignments between audio and text data. Those are\ntested and compared against each other regarding e.g. flaws in the synthesized\nspeech, it's naturalness and intelligibility. Thus different voices have been\nsynthesized, whose quality depends less on the number of training samples used,\nbut much more on the cleanliness and signal-to-noise ratio of those.\nGeneralized voice models have been used for synthesis and the results greatly\ndiffer between the two speech corpora.\n  Tests regarding the adaptation to different speakers show that a resemblance\nto the original speaker is audible throughout all recordings, yet the\nsynthesized voices sound robotic and unnatural in smaller parts. The spoken\ntext, however, is usually intelligible, which shows that the models are working\nwell.\n  In a novel approach, speech is synthesized using side information of the\noriginal audio signal, particularly the pitch frequency. Results show an\nincrease of speech quality and intelligibility in comparison to speech\nsynthesized solely from text, up to the point of being nearly indistinguishable\nfrom the original.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1707.01183v1",
            "date": "2017-07-04 23:29:31+00:00",
            "title": "Complexity Metric for Code-Mixed Social Media Text",
            "authors": [
                "Souvick Ghosh",
                "Satanu Ghosh",
                "Dipankar Das"
            ],
            "abstract": "An evaluation metric is an absolute necessity for measuring the performance\nof any system and complexity of any data. In this paper, we have discussed how\nto determine the level of complexity of code-mixed social media texts that are\ngrowing rapidly due to multilingual interference. In general, texts written in\nmultiple languages are often hard to comprehend and analyze. At the same time,\nin order to meet the demands of analysis, it is also necessary to determine the\ncomplexity of a particular document or a text segment. Thus, in the present\npaper, we have discussed the existing metrics for determining the code-mixing\ncomplexity of a corpus, their advantages, and shortcomings as well as proposed\nseveral improvements on the existing metrics. The new index better reflects the\nvariety and complexity of a multilingual document. Also, the index can be\napplied to a sentence and seamlessly extended to a paragraph or an entire\ndocument. We have employed two existing code-mixed corpora to suit the\nrequirements of our study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01450v1",
            "date": "2017-07-05 16:11:31+00:00",
            "title": "The Complex Negotiation Dialogue Game",
            "authors": [
                "Romain Laroche"
            ],
            "abstract": "This position paper formalises an abstract model for complex negotiation\ndialogue. This model is to be used for the benchmark of optimisation algorithms\nranging from Reinforcement Learning to Stochastic Games, through Transfer\nLearning, One-Shot Learning or others.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.01561v1",
            "date": "2017-07-04 14:52:41+00:00",
            "title": "Automatic Generation of Natural Language Explanations",
            "authors": [
                "Felipe Costa",
                "Sixun Ouyang",
                "Peter Dolog",
                "Aonghus Lawlor"
            ],
            "abstract": "An important task for recommender system is to generate explanations\naccording to a user's preferences. Most of the current methods for explainable\nrecommendations use structured sentences to provide descriptions along with the\nrecommendations they produce. However, those methods have neglected the\nreview-oriented way of writing a text, even though it is known that these\nreviews have a strong influence over user's decision.\n  In this paper, we propose a method for the automatic generation of natural\nlanguage explanations, for predicting how a user would write about an item,\nbased on user ratings from different items' features. We design a\ncharacter-level recurrent neural network (RNN) model, which generates an item's\nreview explanations using long-short term memories (LSTM). The model generates\ntext reviews given a combination of the review and ratings score that express\nopinions about different factors or aspects of an item. Our network is trained\non a sub-sample from the large real-world dataset BeerAdvocate. Our empirical\nevaluation using natural language processing metrics shows the generated text's\nquality is close to a real user written review, identifying negation,\nmisspellings, and domain specific vocabulary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01780v3",
            "date": "2017-07-06 13:31:13+00:00",
            "title": "On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis",
            "authors": [
                "Jose Camacho-Collados",
                "Mohammad Taher Pilehvar"
            ],
            "abstract": "Text preprocessing is often the first step in the pipeline of a Natural\nLanguage Processing (NLP) system, with potential impact in its final\nperformance. Despite its importance, text preprocessing has not received much\nattention in the deep learning literature. In this paper we investigate the\nimpact of simple text preprocessing decisions (particularly tokenizing,\nlemmatizing, lowercasing and multiword grouping) on the performance of a\nstandard neural text classifier. We perform an extensive evaluation on standard\nbenchmarks from text categorization and sentiment analysis. While our\nexperiments show that a simple tokenization of input text is generally\nadequate, they also highlight significant degrees of variability across\npreprocessing techniques. This reveals the importance of paying attention to\nthis usually-overlooked step in the pipeline, particularly when comparing\ndifferent models. Finally, our evaluation provides insights into the best\npreprocessing practices for training word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01917v2",
            "date": "2017-07-06 18:02:12+00:00",
            "title": "Higher-order Relation Schema Induction using Tensor Factorization with Back-off and Aggregation",
            "authors": [
                "Madhav Nimishakavi",
                "Partha Talukdar"
            ],
            "abstract": "Relation Schema Induction (RSI) is the problem of identifying type signatures\nof arguments of relations from unlabeled text. Most of the previous work in\nthis area have focused only on binary RSI, i.e., inducing only the subject and\nobject type signatures per relation. However, in practice, many relations are\nhigh-order, i.e., they have more than two arguments and inducing type\nsignatures of all arguments is necessary. For example, in the sports domain,\ninducing a schema win(WinningPlayer, OpponentPlayer, Tournament, Location) is\nmore informative than inducing just win(WinningPlayer, OpponentPlayer). We\nrefer to this problem as Higher-order Relation Schema Induction (HRSI). In this\npaper, we propose Tensor Factorization with Back-off and Aggregation (TFBA), a\nnovel framework for the HRSI problem. To the best of our knowledge, this is the\nfirst attempt at inducing higher-order relation schemata from unlabeled text.\nUsing the experimental analysis on three real world datasets, we show how TFBA\nhelps in dealing with sparsity and induce higher order schemata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01961v1",
            "date": "2017-07-06 20:48:42+00:00",
            "title": "Long-Term Memory Networks for Question Answering",
            "authors": [
                "Fenglong Ma",
                "Radha Chitta",
                "Saurabh Kataria",
                "Jing Zhou",
                "Palghat Ramesh",
                "Tong Sun",
                "Jing Gao"
            ],
            "abstract": "Question answering is an important and difficult task in the natural language\nprocessing domain, because many basic natural language processing tasks can be\ncast into a question answering task. Several deep neural network architectures\nhave been developed recently, which employ memory and inference components to\nmemorize and reason over text information, and generate answers to questions.\nHowever, a major drawback of many such models is that they are capable of only\ngenerating single-word answers. In addition, they require large amount of\ntraining data to generate accurate answers. In this paper, we introduce the\nLong-Term Memory Network (LTMN), which incorporates both an external memory\nmodule and a Long Short-Term Memory (LSTM) module to comprehend the input data\nand generate multi-word answers. The LTMN model can be trained end-to-end using\nback-propagation and requires minimal supervision. We test our model on two\nsynthetic data sets (based on Facebook's bAbI data set) and the real-world\nStanford question answering data set, and show that it can achieve\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02275v1",
            "date": "2017-07-07 17:15:27+00:00",
            "title": "A parallel corpus of Python functions and documentation strings for automated code documentation and code generation",
            "authors": [
                "Antonio Valerio Miceli Barone",
                "Rico Sennrich"
            ],
            "abstract": "Automated documentation of programming source code and automated code\ngeneration from natural language are challenging tasks of both practical and\nscientific interest. Progress in these areas has been limited by the low\navailability of parallel corpora of code and natural language descriptions,\nwhich tend to be small and constrained to specific domains.\n  In this work we introduce a large and diverse parallel corpus of a hundred\nthousands Python functions with their documentation strings (\"docstrings\")\ngenerated by scraping open source repositories on GitHub. We describe baseline\nresults for the code documentation and code generation tasks obtained by neural\nmachine translation. We also experiment with data augmentation techniques to\nfurther increase the amount of training data.\n  We release our datasets and processing scripts in order to stimulate research\nin these areas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02363v1",
            "date": "2017-07-07 21:21:33+00:00",
            "title": "Towards Zero-Shot Frame Semantic Parsing for Domain Scaling",
            "authors": [
                "Ankur Bapna",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Larry Heck"
            ],
            "abstract": "State-of-the-art slot filling models for goal-oriented human/machine\nconversational language understanding systems rely on deep learning methods.\nWhile multi-task training of such models alleviates the need for large\nin-domain annotated datasets, bootstrapping a semantic parsing model for a new\ndomain using only the semantic frame, such as the back-end API or knowledge\ngraph schema, is still one of the holy grail tasks of language understanding\nfor dialogue systems. This paper proposes a deep learning based approach that\ncan utilize only the slot description in context without the need for any\nlabeled or unlabeled in-domain examples, to quickly bootstrap a new domain. The\nmain idea of this paper is to leverage the encoding of the slot names and\ndescriptions within a multi-task deep learned slot filling model, to implicitly\nalign slots across domains. The proposed approach is promising for solving the\ndomain scaling problem and eliminating the need for any manually annotated data\nor explicit schema alignment. Furthermore, our experiments on multiple domains\nshow that this approach results in significantly better slot-filling\nperformance when compared to using only in-domain data, especially in the low\ndata regime.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.02377v1",
            "date": "2017-07-08 00:57:01+00:00",
            "title": "Efficient Vector Representation for Documents through Corruption",
            "authors": [
                "Minmin Chen"
            ],
            "abstract": "We present an efficient document representation learning framework, Document\nVector through Corruption (Doc2VecC). Doc2VecC represents each document as a\nsimple average of word embeddings. It ensures a representation generated as\nsuch captures the semantic meanings of the document during learning. A\ncorruption model is included, which introduces a data-dependent regularization\nthat favors informative or rare words while forcing the embeddings of common\nand non-discriminative ones to be close to zero. Doc2VecC produces\nsignificantly better word embeddings than Word2Vec. We compare Doc2VecC with\nseveral state-of-the-art document representation learning algorithms. The\nsimple model architecture introduced by Doc2VecC matches or out-performs the\nstate-of-the-art in generating high-quality document representations for\nsentiment analysis, document classification as well as semantic relatedness\ntasks. The simplicity of the model enables training on billions of words per\nhour on a single machine. At the same time, the model is very efficient in\ngenerating representations of unseen documents at test time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02483v1",
            "date": "2017-07-08 19:45:47+00:00",
            "title": "Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection",
            "authors": [
                "Jian Ni",
                "Georgiana Dinu",
                "Radu Florian"
            ],
            "abstract": "The state-of-the-art named entity recognition (NER) systems are supervised\nmachine learning models that require large amounts of manually annotated data\nto achieve high accuracy. However, annotating NER data by human is expensive\nand time-consuming, and can be quite difficult for a new language. In this\npaper, we present two weakly supervised approaches for cross-lingual NER with\nno human annotation in a target language. The first approach is to create\nautomatically labeled NER data for a target language via annotation projection\non comparable corpora, where we develop a heuristic scheme that effectively\nselects good-quality projection-labeled data from noisy data. The second\napproach is to project distributed representations of words (word embeddings)\nfrom a target language to a source language, so that the source-language NER\nsystem can be applied to the target language without re-training. We also\ndesign two co-decoding schemes that effectively combine the outputs of the two\nprojection-based approaches. We evaluate the performance of the proposed\napproaches on both in-house and open NER data for several target languages. The\nresults show that the combined systems outperform three other weakly supervised\napproaches on the CoNLL data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02499v1",
            "date": "2017-07-08 21:30:40+00:00",
            "title": "Predicting the Quality of Short Narratives from Social Media",
            "authors": [
                "Tong Wang",
                "Ping Chen",
                "Boyang Li"
            ],
            "abstract": "An important and difficult challenge in building computational models for\nnarratives is the automatic evaluation of narrative quality. Quality evaluation\nconnects narrative understanding and generation as generation systems need to\nevaluate their own products. To circumvent difficulties in acquiring\nannotations, we employ upvotes in social media as an approximate measure for\nstory quality. We collected 54,484 answers from a crowd-powered\nquestion-and-answer website, Quora, and then used active learning to build a\nclassifier that labeled 28,320 answers as stories. To predict the number of\nupvotes without the use of social network features, we create neural networks\nthat model textual regions and the interdependence among regions, which serve\nas strong benchmarks for future research. To our best knowledge, this is the\nfirst large-scale study for automatic evaluation of narrative quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02575v1",
            "date": "2017-07-09 12:51:47+00:00",
            "title": "Neural Machine Translation between Herbal Prescriptions and Diseases",
            "authors": [
                "Sun-Chong Wang"
            ],
            "abstract": "The current study applies deep learning to herbalism. Toward the goal, we\nacquired the de-identified health insurance reimbursements that were claimed in\na 10-year period from 2004 to 2013 in the National Health Insurance Database of\nTaiwan, the total number of reimbursement records equaling 340 millions. Two\nartificial intelligence techniques were applied to the dataset: residual\nconvolutional neural network multitask classifier and attention-based recurrent\nneural network. The former works to translate from herbal prescriptions to\ndiseases; and the latter from diseases to herbal prescriptions. Analysis of the\nclassification results indicates that herbal prescriptions are specific to:\nanatomy, pathophysiology, sex and age of the patient, and season and year of\nthe prescription. Further analysis identifies temperature and gross domestic\nproduct as the meteorological and socioeconomic factors that are associated\nwith herbal prescriptions. Analysis of the neural machine transitional result\nindicates that the recurrent neural network learnt not only syntax but also\nsemantics of diseases and herbal prescriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03457v1",
            "date": "2017-07-11 20:48:57+00:00",
            "title": "Multiple Context-Free Tree Grammars: Lexicalization and Characterization",
            "authors": [
                "Joost Engelfriet",
                "Andreas Maletti",
                "Sebastian Maneth"
            ],
            "abstract": "Multiple (simple) context-free tree grammars are investigated, where \"simple\"\nmeans \"linear and nondeleting\". Every multiple context-free tree grammar that\nis finitely ambiguous can be lexicalized; i.e., it can be transformed into an\nequivalent one (generating the same tree language) in which each rule of the\ngrammar contains a lexical symbol. Due to this transformation, the rank of the\nnonterminals increases at most by 1, and the multiplicity (or fan-out) of the\ngrammar increases at most by the maximal rank of the lexical symbols; in\nparticular, the multiplicity does not increase when all lexical symbols have\nrank 0. Multiple context-free tree grammars have the same tree generating power\nas multi-component tree adjoining grammars (provided the latter can use a\nroot-marker). Moreover, every multi-component tree adjoining grammar that is\nfinitely ambiguous can be lexicalized. Multiple context-free tree grammars have\nthe same string generating power as multiple context-free (string) grammars and\npolynomial time parsing algorithms. A tree language can be generated by a\nmultiple context-free tree grammar if and only if it is the image of a regular\ntree language under a deterministic finite-copying macro tree transducer.\nMultiple context-free tree grammars can be used as a synchronous translation\ndevice.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03997v1",
            "date": "2017-07-13 07:22:18+00:00",
            "title": "A Web-Based Tool for Analysing Normative Documents in English",
            "authors": [
                "John J. Camilleri",
                "Mohammad Reza Haghshenas",
                "Gerardo Schneider"
            ],
            "abstract": "Our goal is to use formal methods to analyse normative documents written in\nEnglish, such as privacy policies and service-level agreements. This requires\nthe combination of a number of different elements, including information\nextraction from natural language, formal languages for model representation,\nand an interface for property specification and verification. We have worked on\na collection of components for this task: a natural language extraction tool, a\nsuitable formalism for representing such documents, an interface for building\nmodels in this formalism, and methods for answering queries asked of a given\nmodel. In this work, each of these concerns is brought together in a web-based\ntool, providing a single interface for analysing normative texts in English.\nThrough the use of a running example, we describe each component and\ndemonstrate the workflow established by our tool.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04221v1",
            "date": "2017-07-13 16:52:08+00:00",
            "title": "Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation",
            "authors": [
                "Jonathan K. Kummerfeld",
                "Dan Klein"
            ],
            "abstract": "General treebank analyses are graph structured, but parsers are typically\nrestricted to tree structures for efficiency and modeling reasons. We propose a\nnew representation and algorithm for a class of graph structures that is\nflexible enough to cover almost all treebank structures, while still admitting\nefficient learning and inference. In particular, we consider directed, acyclic,\none-endpoint-crossing graph structures, which cover most long-distance\ndislocation, shared argumentation, and similar tree-violating linguistic\nphenomena. We describe how to convert phrase structure parses, including\ntraces, to our new representation in a reversible manner. Our dynamic program\nuniquely decomposes structures, is sound and complete, and covers 97.3% of the\nPenn English Treebank. We also implement a proof-of-concept parser that\nrecovers a range of null elements and trace types.",
            "journal ref": "TACL 5 (2017) 441-454",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04227v5",
            "date": "2017-07-13 17:16:16+00:00",
            "title": "Automatic Speech Recognition with Very Large Conversational Finnish and Estonian Vocabularies",
            "authors": [
                "Seppo Enarvi",
                "Peter Smit",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "Today, the vocabulary size for language models in large vocabulary speech\nrecognition is typically several hundreds of thousands of words. While this is\nalready sufficient in some applications, the out-of-vocabulary words are still\nlimiting the usability in others. In agglutinative languages the vocabulary for\nconversational speech should include millions of word forms to cover the\nspelling variations due to colloquial pronunciations, in addition to the word\ncompounding and inflections. Very large vocabularies are also needed, for\nexample, when the recognition of rare proper names is important.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  vol. 25, no. 11, pp. 2085-2097, November 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04546v1",
            "date": "2017-07-14 15:43:45+00:00",
            "title": "Linguistic Markers of Influence in Informal Interactions",
            "authors": [
                "Shrimai Prabhumoye",
                "Samridhi Choudhary",
                "Evangelia Spiliopoulou",
                "Christopher Bogart",
                "Carolyn Penstein Rose",
                "Alan W Black"
            ],
            "abstract": "There has been a long standing interest in understanding `Social Influence'\nboth in Social Sciences and in Computational Linguistics. In this paper, we\npresent a novel approach to study and measure interpersonal influence in daily\ninteractions. Motivated by the basic principles of influence, we attempt to\nidentify indicative linguistic features of the posts in an online knitting\ncommunity. We present the scheme used to operationalize and label the posts\nwith indicator features. Experiments with the identified features show an\nimprovement in the classification accuracy of influence by 3.15%. Our results\nillustrate the important correlation between the characteristics of the\nlanguage and its potential to influence others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04550v1",
            "date": "2017-07-14 15:58:47+00:00",
            "title": "CUNI System for the WMT17 Multimodal Translation Task",
            "authors": [
                "Jind\u0159ich Helcl",
                "Jind\u0159ich Libovick\u00fd"
            ],
            "abstract": "In this paper, we describe our submissions to the WMT17 Multimodal\nTranslation Task. For Task 1 (multimodal translation), our best scoring system\nis a purely textual neural translation of the source image caption to the\ntarget language. The main feature of the system is the use of additional data\nthat was acquired by selecting similar sentences from parallel corpora and by\ndata synthesis with back-translation. For Task 2 (cross-lingual image\ncaptioning), our best submitted system generates an English caption which is\nthen translated by the best system used in Task 1. We also present negative\nresults, which are based on ideas that we believe have potential of making\nimprovements, but did not prove to be useful in our particular setup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04596v1",
            "date": "2017-07-14 18:05:49+00:00",
            "title": "DocTag2Vec: An Embedding Based Multi-label Learning Approach for Document Tagging",
            "authors": [
                "Sheng Chen",
                "Akshay Soni",
                "Aasish Pappu",
                "Yashar Mehdad"
            ],
            "abstract": "Tagging news articles or blog posts with relevant tags from a collection of\npredefined ones is coined as document tagging in this work. Accurate tagging of\narticles can benefit several downstream applications such as recommendation and\nsearch. In this work, we propose a novel yet simple approach called DocTag2Vec\nto accomplish this task. We substantially extend Word2Vec and Doc2Vec---two\npopular models for learning distributed representation of words and documents.\nIn DocTag2Vec, we simultaneously learn the representation of words, documents,\nand tags in a joint vector space during training, and employ the simple\n$k$-nearest neighbor search to predict tags for unseen documents. In contrast\nto previous multi-label learning methods, DocTag2Vec directly deals with raw\ntext instead of provided feature vector, and in addition, enjoys advantages\nlike the learning of tag representation, and the ability of handling newly\ncreated tags. To demonstrate the effectiveness of our approach, we conduct\nexperiments on several datasets and show promising results against\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04652v1",
            "date": "2017-07-14 22:02:05+00:00",
            "title": "EmojiNet: An Open Service and API for Emoji Sense Discovery",
            "authors": [
                "Sanjaya Wijeratne",
                "Lakshika Balasuriya",
                "Amit Sheth",
                "Derek Doran"
            ],
            "abstract": "This paper presents the release of EmojiNet, the largest machine-readable\nemoji sense inventory that links Unicode emoji representations to their English\nmeanings extracted from the Web. EmojiNet is a dataset consisting of: (i)\n12,904 sense labels over 2,389 emoji, which were extracted from the web and\nlinked to machine-readable sense definitions seen in BabelNet, (ii) context\nwords associated with each emoji sense, which are inferred through word\nembedding models trained over Google News corpus and a Twitter message corpus\nfor each emoji sense definition, and (iii) recognizing discrepancies in the\npresentation of emoji on different platforms, specification of the most likely\nplatform-based emoji sense for a selected set of emoji. The dataset is hosted\nas an open service with a REST API and is available at\nhttp://emojinet.knoesis.org/. The development of this dataset, evaluation of\nits quality, and its applications including emoji sense disambiguation and\nemoji sense similarity are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04653v1",
            "date": "2017-07-14 22:08:15+00:00",
            "title": "A Semantics-Based Measure of Emoji Similarity",
            "authors": [
                "Sanjaya Wijeratne",
                "Lakshika Balasuriya",
                "Amit Sheth",
                "Derek Doran"
            ],
            "abstract": "Emoji have grown to become one of the most important forms of communication\non the web. With its widespread use, measuring the similarity of emoji has\nbecome an important problem for contemporary text processing since it lies at\nthe heart of sentiment analysis, search, and interface design tasks. This paper\npresents a comprehensive analysis of the semantic similarity of emoji through\nembedding models that are learned over machine-readable emoji meanings in the\nEmojiNet knowledge base. Using emoji descriptions, emoji sense labels and emoji\nsense definitions, and with different training corpora obtained from Twitter\nand Google News, we develop and test multiple embedding models to measure emoji\nsimilarity. To evaluate our work, we create a new dataset called EmoSim508,\nwhich assigns human-annotated semantic similarity scores to a set of 508\ncarefully selected emoji pairs. After validation with EmoSim508, we present a\nreal-world use-case of our emoji embedding models using a sentiment analysis\ntask and show that our models outperform the previous best-performing emoji\nembedding model on this task. The EmoSim508 dataset and our emoji embedding\nmodels are publicly released with this paper and can be downloaded from\nhttp://emojinet.knoesis.org/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04968v2",
            "date": "2017-07-17 00:42:56+00:00",
            "title": "Visual Question Answering with Memory-Augmented Networks",
            "authors": [
                "Chao Ma",
                "Chunhua Shen",
                "Anthony Dick",
                "Qi Wu",
                "Peng Wang",
                "Anton van den Hengel",
                "Ian Reid"
            ],
            "abstract": "In this paper, we exploit a memory-augmented neural network to predict\naccurate answers to visual questions, even when those answers occur rarely in\nthe training set. The memory network incorporates both internal and external\nmemory blocks and selectively pays attention to each training exemplar. We show\nthat memory-augmented neural networks are able to maintain a relatively\nlong-term memory of scarce training exemplars, which is important for visual\nquestion answering due to the heavy-tailed distribution of answers in a general\nVQA setting. Experimental results on two large-scale benchmark datasets show\nthe favorable performance of the proposed algorithm with a comparison to state\nof the art.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.05015v1",
            "date": "2017-07-17 06:55:43+00:00",
            "title": "Iris: A Conversational Agent for Complex Tasks",
            "authors": [
                "Ethan Fast",
                "Binbin Chen",
                "Julia Mendelsohn",
                "Jonathan Bassen",
                "Michael Bernstein"
            ],
            "abstract": "Today's conversational agents are restricted to simple standalone commands.\nIn this paper, we present Iris, an agent that draws on human conversational\nstrategies to combine commands, allowing it to perform more complex tasks that\nit has not been explicitly designed to support: for example, composing one\ncommand to \"plot a histogram\" with another to first \"log-transform the data\".\nTo enable this complexity, we introduce a domain specific language that\ntransforms commands into automata that Iris can compose, sequence, and execute\ndynamically by interacting with a user through natural language, as well as a\nconversational type system that manages what kinds of commands can be combined.\nWe have designed Iris to help users with data science tasks, a domain that\nrequires support for command combination. In evaluation, we find that data\nscientists complete a predictive modeling task significantly faster (2.6 times\nspeedup) with Iris than a modern non-conversational programming environment.\nIris supports the same kinds of commands as today's agents, but empowers users\nto weave together these commands to accomplish complex goals.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1707.05236v1",
            "date": "2017-07-17 15:38:09+00:00",
            "title": "Artificial Error Generation with Machine Translation and Syntactic Patterns",
            "authors": [
                "Marek Rei",
                "Mariano Felice",
                "Zheng Yuan",
                "Ted Briscoe"
            ],
            "abstract": "Shortage of available training data is holding back progress in the area of\nautomated error detection. This paper investigates two alternative methods for\nartificially generating writing errors, in order to create additional\nresources. We propose treating error generation as a machine translation task,\nwhere grammatically correct text is translated to contain errors. In addition,\nwe explore a system for extracting textual patterns from an annotated corpus,\nwhich can then be used to insert errors into grammatically correct sentences.\nOur experiments show that the inclusion of artificially generated errors\nsignificantly improves error detection accuracy on both FCE and CoNLL 2014\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05246v1",
            "date": "2017-07-17 15:53:18+00:00",
            "title": "Learning to select data for transfer learning with Bayesian Optimization",
            "authors": [
                "Sebastian Ruder",
                "Barbara Plank"
            ],
            "abstract": "Domain similarity measures can be used to gauge adaptability and select\nsuitable data for transfer learning, but existing approaches define ad hoc\nmeasures that are deemed suitable for respective tasks. Inspired by work on\ncurriculum learning, we propose to \\emph{learn} data selection measures using\nBayesian Optimization and evaluate them across models, domains and tasks. Our\nlearned measures outperform existing domain similarity measures significantly\non three tasks: sentiment analysis, part-of-speech tagging, and parsing. We\nshow the importance of complementing similarity with diversity, and that\nlearned measures are -- to some degree -- transferable across models, domains,\nand even tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05254v1",
            "date": "2017-07-12 23:18:58+00:00",
            "title": "Explainable Entity-based Recommendations with Knowledge Graphs",
            "authors": [
                "Rose Catherine",
                "Kathryn Mazaitis",
                "Maxine Eskenazi",
                "William Cohen"
            ],
            "abstract": "Explainable recommendation is an important task. Many methods have been\nproposed which generate explanations from the content and reviews written for\nitems. When review text is unavailable, generating explanations is still a hard\nproblem. In this paper, we illustrate how explanations can be generated in such\na scenario by leveraging external knowledge in the form of knowledge graphs.\nOur method jointly ranks items and knowledge graph entities using a\nPersonalized PageRank procedure to produce recommendations together with their\nexplanations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.06065v1",
            "date": "2017-07-19 13:04:09+00:00",
            "title": "Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition",
            "authors": [
                "Taesup Kim",
                "Inchul Song",
                "Yoshua Bengio"
            ],
            "abstract": "Layer normalization is a recently introduced technique for normalizing the\nactivities of neurons in deep neural networks to improve the training speed and\nstability. In this paper, we introduce a new layer normalization technique\ncalled Dynamic Layer Normalization (DLN) for adaptive neural acoustic modeling\nin speech recognition. By dynamically generating the scaling and shifting\nparameters in layer normalization, DLN adapts neural acoustic models to the\nacoustic variability arising from various factors such as speakers, channel\nnoises, and environments. Unlike other adaptive acoustic models, our proposed\napproach does not require additional adaptation data or speaker information\nsuch as i-vectors. Moreover, the model size is fixed as it dynamically\ngenerates adaptation parameters. We apply our proposed DLN to deep\nbidirectional LSTM acoustic models and evaluate them on two benchmark datasets\nfor large vocabulary ASR experiments: WSJ and TED-LIUM release 2. The\nexperimental results show that our DLN improves neural acoustic models in terms\nof transcription accuracy by dynamically adapting to various speakers and\nenvironments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06265v2",
            "date": "2017-07-19 19:10:44+00:00",
            "title": "Unsupervised Domain Adaptation for Robust Speech Recognition via Variational Autoencoder-Based Data Augmentation",
            "authors": [
                "Wei-Ning Hsu",
                "Yu Zhang",
                "James Glass"
            ],
            "abstract": "Domain mismatch between training and testing can lead to significant\ndegradation in performance in many machine learning scenarios. Unfortunately,\nthis is not a rare situation for automatic speech recognition deployments in\nreal-world applications. Research on robust speech recognition can be regarded\nas trying to overcome this domain mismatch issue. In this paper, we address the\nunsupervised domain adaptation problem for robust speech recognition, where\nboth source and target domain speech are presented, but word transcripts are\nonly available for the source domain speech. We present novel\naugmentation-based methods that transform speech in a way that does not change\nthe transcripts. Specifically, we first train a variational autoencoder on both\nsource and target domain data (without supervision) to learn a latent\nrepresentation of speech. We then transform nuisance attributes of speech that\nare irrelevant to recognition by modifying the latent representations, in order\nto augment labeled training data with additional data whose distribution is\nmore similar to the target domain. The proposed method is evaluated on the\nCHiME-4 dataset and reduces the absolute word error rate (WER) by as much as\n35% compared to the non-adapted baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06299v1",
            "date": "2017-07-19 21:21:03+00:00",
            "title": "Reward-Balancing for Statistical Spoken Dialogue Systems using Multi-objective Reinforcement Learning",
            "authors": [
                "Stefan Ultes",
                "Pawe\u0142 Budzianowski",
                "I\u00f1igo Casanueva",
                "Nikola Mrk\u0161i\u0107",
                "Lina Rojas-Barahona",
                "Pei-Hao Su",
                "Tsung-Hsien Wen",
                "Milica Ga\u0161i\u0107",
                "Steve Young"
            ],
            "abstract": "Reinforcement learning is widely used for dialogue policy optimization where\nthe reward function often consists of more than one component, e.g., the\ndialogue success and the dialogue length. In this work, we propose a structured\nmethod for finding a good balance between these components by searching for the\noptimal reward component weighting. To render this search feasible, we use\nmulti-objective reinforcement learning to significantly reduce the number of\ntraining dialogues required. We apply our proposed method to find optimized\ncomponent weights for six domains and compare them to a default baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06320v2",
            "date": "2017-07-19 23:12:57+00:00",
            "title": "Learning Visually Grounded Sentence Representations",
            "authors": [
                "Douwe Kiela",
                "Alexis Conneau",
                "Allan Jabri",
                "Maximilian Nickel"
            ],
            "abstract": "We introduce a variety of models, trained on a supervised image captioning\ncorpus to predict the image features for a given caption, to perform sentence\nrepresentation grounding. We train a grounded sentence encoder that achieves\ngood performance on COCO caption and image retrieval and subsequently show that\nthis encoder can successfully be transferred to various NLP tasks, with\nimproved performance over text-only models. Lastly, we analyze the contribution\nof grounding, and show that word embeddings learned by this system outperform\nnon-grounded ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06519v1",
            "date": "2017-07-19 10:54:00+00:00",
            "title": "Language Transfer of Audio Word2Vec: Learning Audio Segment Representations without Target Language Data",
            "authors": [
                "Chia-Hao Shen",
                "Janet Y. Sung",
                "Hung-Yi Lee"
            ],
            "abstract": "Audio Word2Vec offers vector representations of fixed dimensionality for\nvariable-length audio segments using Sequence-to-sequence Autoencoder (SA).\nThese vector representations are shown to describe the sequential phonetic\nstructures of the audio segments to a good degree, with real world applications\nsuch as query-by-example Spoken Term Detection (STD). This paper examines the\ncapability of language transfer of Audio Word2Vec. We train SA from one\nlanguage (source language) and use it to extract the vector representation of\nthe audio segments of another language (target language). We found that SA can\nstill catch phonetic structure from the audio segments of the target language\nif the source and target languages are similar. In query-by-example STD, we\nobtain the vector representations from the SA learned from a large amount of\nsource language data, and found them surpass the representations from naive\nencoder and SA directly learned from a small amount of target language data.\nThe result shows that it is possible to learn Audio Word2Vec model from\nhigh-resource languages and use it on low-resource languages. This further\nexpands the usability of Audio Word2Vec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06556v1",
            "date": "2017-07-20 15:02:14+00:00",
            "title": "High-risk learning: acquiring new word vectors from tiny data",
            "authors": [
                "Aurelie Herbelot",
                "Marco Baroni"
            ],
            "abstract": "Distributional semantics models are known to struggle with small data. It is\ngenerally accepted that in order to learn 'a good vector' for a word, a model\nmust have sufficient examples of its usage. This contradicts the fact that\nhumans can guess the meaning of a word from a few occurrences only. In this\npaper, we show that a neural language model such as Word2Vec only necessitates\nminor modifications to its standard architecture to learn new terms from tiny\ndata, using background knowledge from a previously learnt semantic space. We\ntest our model on word definitions and on a nonce task involving 2-6 sentences'\nworth of context, showing a large increase in performance over state-of-the-art\nmodels on the definitional task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06562v1",
            "date": "2017-07-20 15:06:43+00:00",
            "title": "From Task Classification Towards Similarity Measures for Recommendation in Crowdsourcing Systems",
            "authors": [
                "Steffen Schnitzer",
                "Svenja Neitzel",
                "Christoph Rensing"
            ],
            "abstract": "Task selection in micro-task markets can be supported by recommender systems\nto help individuals to find appropriate tasks. Previous work showed that for\nthe selection process of a micro-task the semantic aspects, such as the\nrequired action and the comprehensibility, are rated more important than\nfactual aspects, such as the payment or the required completion time. This work\ngives a foundation to create such similarity measures. Therefore, we show that\nan automatic classification based on task descriptions is possible.\nAdditionally, we propose similarity measures to cluster micro-tasks according\nto semantic aspects.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.06598v2",
            "date": "2017-07-20 16:33:48+00:00",
            "title": "Toward Incorporation of Relevant Documents in word2vec",
            "authors": [
                "Navid Rekabsaz",
                "Bhaskar Mitra",
                "Mihai Lupu",
                "Allan Hanbury"
            ],
            "abstract": "Recent advances in neural word embedding provide significant benefit to\nvarious information retrieval tasks. However as shown by recent studies,\nadapting the embedding models for the needs of IR tasks can bring considerable\nfurther improvements. The embedding models in general define the term\nrelatedness by exploiting the terms' co-occurrences in short-window contexts.\nAn alternative (and well-studied) approach in IR for related terms to a query\nis using local information i.e. a set of top-retrieved documents. In view of\nthese two methods of term relatedness, in this work, we report our study on\nincorporating the local information of the query in the word embeddings. One\nmain challenge in this direction is that the dense vectors of word embeddings\nand their estimation of term-to-term relatedness remain difficult to interpret\nand hard to analyze. As an alternative, explicit word representations propose\nvectors whose dimensions are easily interpretable, and recent methods show\ncompetitive performance to the dense vectors. We introduce a neural-based\nexplicit representation, rooted in the conceptual ideas of the word2vec\nSkip-Gram model. The method provides interpretable explicit vectors while\nkeeping the effectiveness of the Skip-Gram model. The evaluation of various\nexplicit representations on word association collections shows that the newly\nproposed method out- performs the state-of-the-art explicit representations\nwhen tasked with ranking highly similar terms. Based on the introduced ex-\nplicit representation, we discuss our approaches on integrating local documents\nin globally-trained embedding models and discuss the preliminary results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.06690v3",
            "date": "2017-07-20 19:39:23+00:00",
            "title": "DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning",
            "authors": [
                "Wenhan Xiong",
                "Thien Hoang",
                "William Yang Wang"
            ],
            "abstract": "We study the problem of learning to reason in large scale knowledge graphs\n(KGs). More specifically, we describe a novel reinforcement learning framework\nfor learning multi-hop relational paths: we use a policy-based agent with\ncontinuous states based on knowledge graph embeddings, which reasons in a KG\nvector space by sampling the most promising relation to extend its path. In\ncontrast to prior work, our approach includes a reward function that takes the\naccuracy, diversity, and efficiency into consideration. Experimentally, we show\nthat our proposed method outperforms a path-ranking based algorithm and\nknowledge graph embedding methods on Freebase and Never-Ending Language\nLearning datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07048v2",
            "date": "2017-07-21 21:21:09+00:00",
            "title": "Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition",
            "authors": [
                "Zhehuai Chen",
                "Jasha Droppo",
                "Jinyu Li",
                "Wayne Xiong"
            ],
            "abstract": "Unsupervised single-channel overlapped speech recognition is one of the\nhardest problems in automatic speech recognition (ASR). Permutation invariant\ntraining (PIT) is a state of the art model-based approach, which applies a\nsingle neural network to solve this single-input, multiple-output modeling\nproblem. We propose to advance the current state of the art by imposing a\nmodular structure on the neural network, applying a progressive pretraining\nregimen, and improving the objective function with transfer learning and a\ndiscriminative training criterion. The modular structure splits the problem\ninto three sub-tasks: frame-wise interpreting, utterance-level speaker tracing,\nand speech recognition. The pretraining regimen uses these modules to solve\nprogressively harder tasks. Transfer learning leverages parallel clean speech\nto improve the training targets for the network. Our discriminative training\nformulation is a modification of standard formulations, that also penalizes\ncompeting outputs of the system. Experiments are conducted on the artificial\noverlapped Switchboard and hub5e-swb dataset. The proposed framework achieves\nover 30% relative improvement of WER over both a strong jointly trained system,\nPIT for ASR, and a separately optimized system, PIT for speech separation with\nclean speech ASR model. The improvement comes from better model generalization,\ntraining efficiency and the sequence level linguistic knowledge integration.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  26 (2018) 184-196",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07102v1",
            "date": "2017-07-22 04:17:42+00:00",
            "title": "OBJ2TEXT: Generating Visually Descriptive Language from Object Layouts",
            "authors": [
                "Xuwang Yin",
                "Vicente Ordonez"
            ],
            "abstract": "Generating captions for images is a task that has recently received\nconsiderable attention. In this work we focus on caption generation for\nabstract scenes, or object layouts where the only information provided is a set\nof objects and their locations. We propose OBJ2TEXT, a sequence-to-sequence\nmodel that encodes a set of objects and their locations as an input sequence\nusing an LSTM network, and decodes this representation using an LSTM language\nmodel. We show that our model, despite encoding object layouts as a sequence,\ncan represent spatial relationships between objects, and generate descriptions\nthat are globally coherent and semantically relevant. We test our approach in a\ntask of object-layout captioning by using only object annotations as inputs. We\nadditionally show that our model, combined with a state-of-the-art object\ndetector, improves an image captioning model from 0.863 to 0.950 (CIDEr score)\nin the test benchmark of the standard MS-COCO Captioning task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.07167v3",
            "date": "2017-07-22 13:53:28+00:00",
            "title": "Attention-Based End-to-End Speech Recognition on Voice Search",
            "authors": [
                "Changhao Shan",
                "Junbo Zhang",
                "Yujun Wang",
                "Lei Xie"
            ],
            "abstract": "Recently, there has been a growing interest in end-to-end speech recognition\nthat directly transcribes speech to text without any predefined alignments. In\nthis paper, we explore the use of attention-based encoder-decoder model for\nMandarin speech recognition on a voice search task. Previous attempts have\nshown that applying attention-based encoder-decoder to Mandarin speech\nrecognition was quite difficult due to the logographic orthography of Mandarin,\nthe large vocabulary and the conditional dependency of the attention model. In\nthis paper, we use character embedding to deal with the large vocabulary.\nSeveral tricks are used for effective model training, including L2\nregularization, Gaussian weight noise and frame skipping. We compare two\nattention mechanisms and use attention smoothing to cover long context in the\nattention model. Taken together, these tricks allow us to finally achieve a\ncharacter error rate (CER) of 3.58% and a sentence error rate (SER) of 7.43% on\nthe MiTV voice search dataset. While together with a trigram language model,\nCER and SER reach 2.81% and 5.77%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07191v1",
            "date": "2017-07-22 16:32:16+00:00",
            "title": "MoodSwipe: A Soft Keyboard that Suggests Messages Based on User-Specified Emotions",
            "authors": [
                "Chieh-Yang Huang",
                "Tristan Labetoulle",
                "Ting-Hao Kenneth Huang",
                "Yi-Pei Chen",
                "Hung-Chen Chen",
                "Vallari Srivastava",
                "Lun-Wei Ku"
            ],
            "abstract": "We present MoodSwipe, a soft keyboard that suggests text messages given the\nuser-specified emotions utilizing the real dialog data. The aim of MoodSwipe is\nto create a convenient user interface to enjoy the technology of emotion\nclassification and text suggestion, and at the same time to collect labeled\ndata automatically for developing more advanced technologies. While users\nselect the MoodSwipe keyboard, they can type as usual but sense the emotion\nconveyed by their text and receive suggestions for their message as a benefit.\nIn MoodSwipe, the detected emotions serve as the medium for suggested texts,\nwhere viewing the latter is the incentive to correcting the former. We conduct\nseveral experiments to show the superiority of the emotion classification\nmodels trained on the dialog data, and further to verify good emotion cues are\nimportant context for text suggestion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07270v1",
            "date": "2017-07-23 09:36:50+00:00",
            "title": "MatchZoo: A Toolkit for Deep Text Matching",
            "authors": [
                "Yixing Fan",
                "Liang Pang",
                "JianPeng Hou",
                "Jiafeng Guo",
                "Yanyan Lan",
                "Xueqi Cheng"
            ],
            "abstract": "In recent years, deep neural models have been widely adopted for text\nmatching tasks, such as question answering and information retrieval, showing\nimproved performance as compared with previous methods. In this paper, we\nintroduce the MatchZoo toolkit that aims to facilitate the designing, comparing\nand sharing of deep text matching models. Specifically, the toolkit provides a\nunified data preparation module for different text matching problems, a\nflexible layer-based model construction process, and a variety of training\nobjectives and evaluation metrics. In addition, the toolkit has implemented two\nschools of representative deep text matching models, namely\nrepresentation-focused models and interaction-focused models. Finally, users\ncan easily modify existing models, create and share their own models for text\nmatching in MatchZoo.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07328v1",
            "date": "2017-07-23 18:26:29+00:00",
            "title": "Adversarial Examples for Evaluating Reading Comprehension Systems",
            "authors": [
                "Robin Jia",
                "Percy Liang"
            ],
            "abstract": "Standard accuracy metrics indicate that reading comprehension systems are\nmaking rapid progress, but the extent to which these systems truly understand\nlanguage remains unclear. To reward systems with real language understanding\nabilities, we propose an adversarial evaluation scheme for the Stanford\nQuestion Answering Dataset (SQuAD). Our method tests whether systems can answer\nquestions about paragraphs that contain adversarially inserted sentences, which\nare automatically generated to distract computer systems without changing the\ncorrect answer or misleading humans. In this adversarial setting, the accuracy\nof sixteen published models drops from an average of $75\\%$ F1 score to $36\\%$;\nwhen the adversary is allowed to add ungrammatical sequences of words, average\naccuracy on four models decreases further to $7\\%$. We hope our insights will\nmotivate the development of new models that understand language more precisely.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07413v1",
            "date": "2017-07-24 06:05:21+00:00",
            "title": "Exploring Neural Transducers for End-to-End Speech Recognition",
            "authors": [
                "Eric Battenberg",
                "Jitong Chen",
                "Rewon Child",
                "Adam Coates",
                "Yashesh Gaur",
                "Yi Li",
                "Hairong Liu",
                "Sanjeev Satheesh",
                "David Seetapun",
                "Anuroop Sriram",
                "Zhenyao Zhu"
            ],
            "abstract": "In this work, we perform an empirical comparison among the CTC,\nRNN-Transducer, and attention-based Seq2Seq models for end-to-end speech\nrecognition. We show that, without any language model, Seq2Seq and\nRNN-Transducer models both outperform the best reported CTC models with a\nlanguage model, on the popular Hub5'00 benchmark. On our internal diverse\ndataset, these trends continue - RNNTransducer models rescored with a language\nmodel after beam search outperform our best CTC models. These results simplify\nthe speech recognition pipeline so that decoding can now be expressed purely as\nneural network operations. We also study how the choice of encoder architecture\naffects the performance of the three models - when all encoder layers are\nforward only, and when encoders downsample the input representation\naggressively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07469v1",
            "date": "2017-07-24 10:35:46+00:00",
            "title": "Character-level Intra Attention Network for Natural Language Inference",
            "authors": [
                "Han Yang",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Natural language inference (NLI) is a central problem in language\nunderstanding. End-to-end artificial neural networks have reached\nstate-of-the-art performance in NLI field recently.\n  In this paper, we propose Character-level Intra Attention Network (CIAN) for\nthe NLI task. In our model, we use the character-level convolutional network to\nreplace the standard word embedding layer, and we use the intra attention to\ncapture the intra-sentence semantics. The proposed CIAN model provides improved\nresults based on a newly published MNLI corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07554v1",
            "date": "2017-07-24 13:38:00+00:00",
            "title": "Learning Rare Word Representations using Semantic Bridging",
            "authors": [
                "Victor Prokhorov",
                "Mohammad Taher Pilehvar",
                "Dimitri Kartsaklis",
                "Pietro Li\u00f3",
                "Nigel Collier"
            ],
            "abstract": "We propose a methodology that adapts graph embedding techniques (DeepWalk\n(Perozzi et al., 2014) and node2vec (Grover and Leskovec, 2016)) as well as\ncross-lingual vector space mapping approaches (Least Squares and Canonical\nCorrelation Analysis) in order to merge the corpus and ontological sources of\nlexical knowledge. We also perform comparative analysis of the used algorithms\nin order to identify the best combination for the proposed system. We then\napply this to the task of enhancing the coverage of an existing word\nembedding's vocabulary with rare and unseen words. We show that our technique\ncan provide considerable extra coverage (over 99%), leading to consistent\nperformance gain (around 10% absolute gain is achieved with w2v-gn-500K cf.\\S\n3.3) on the Rare Word Similarity dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07601v1",
            "date": "2017-07-24 15:08:13+00:00",
            "title": "Image Pivoting for Learning Multilingual Multimodal Representations",
            "authors": [
                "Spandana Gella",
                "Rico Sennrich",
                "Frank Keller",
                "Mirella Lapata"
            ],
            "abstract": "In this paper we propose a model to learn multimodal multilingual\nrepresentations for matching images and sentences in different languages, with\nthe aim of advancing multilingual versions of image search and image\nunderstanding. Our model learns a common representation for images and their\ndescriptions in two different languages (which need not be parallel) by\nconsidering the image as a pivot between two languages. We introduce a new\npairwise ranking loss function which can handle both symmetric and asymmetric\nsimilarity between the two modalities. We evaluate our models on\nimage-description ranking for German and English, and on semantic textual\nsimilarity of image descriptions in English. In both cases we achieve\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07660v2",
            "date": "2017-07-24 17:42:32+00:00",
            "title": "Thread Reconstruction in Conversational Data using Neural Coherence Models",
            "authors": [
                "Dat Tien Nguyen",
                "Shafiq Joty",
                "Basma El Amel Boussaha",
                "Maarten de Rijke"
            ],
            "abstract": "Discussion forums are an important source of information. They are often used\nto answer specific questions a user might have and to discover more about a\ntopic of interest. Discussions in these forums may evolve in intricate ways,\nmaking it difficult for users to follow the flow of ideas. We propose a novel\napproach for automatically identifying the underlying thread structure of a\nforum discussion. Our approach is based on a neural model that computes\ncoherence scores of possible reconstructions and then selects the highest\nscoring, i.e., the most coherent one. Preliminary experiments demonstrate\npromising results outperforming a number of strong baseline methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07792v1",
            "date": "2017-07-25 02:29:31+00:00",
            "title": "Integrating Lexical and Temporal Signals in Neural Ranking Models for Searching Social Media Streams",
            "authors": [
                "Jinfeng Rao",
                "Hua He",
                "Haotian Zhang",
                "Ferhan Ture",
                "Royal Sequiera",
                "Salman Mohammed",
                "Jimmy Lin"
            ],
            "abstract": "Time is an important relevance signal when searching streams of social media\nposts. The distribution of document timestamps from the results of an initial\nquery can be leveraged to infer the distribution of relevant documents, which\ncan then be used to rerank the initial results. Previous experiments have shown\nthat kernel density estimation is a simple yet effective implementation of this\nidea. This paper explores an alternative approach to mining temporal signals\nwith recurrent neural networks. Our intuition is that neural networks provide a\nmore expressive framework to capture the temporal coherence of neighboring\ndocuments in time. To our knowledge, we are the first to integrate lexical and\ntemporal signals in an end-to-end neural network architecture, in which\nexisting neural ranking models are used to generate query-document similarity\nvectors that feed into a bidirectional LSTM layer for temporal modeling. Our\nresults are mixed: existing neural models for document ranking alone yield\nlimited improvements over simple baselines, but the integration of lexical and\ntemporal signals yield significant improvements over competitive temporal\nbaselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07804v1",
            "date": "2017-07-25 03:38:57+00:00",
            "title": "Exploring the Effectiveness of Convolutional Neural Networks for Answer Selection in End-to-End Question Answering",
            "authors": [
                "Royal Sequiera",
                "Gaurav Baruah",
                "Zhucheng Tu",
                "Salman Mohammed",
                "Jinfeng Rao",
                "Haotian Zhang",
                "Jimmy Lin"
            ],
            "abstract": "Most work on natural language question answering today focuses on answer\nselection: given a candidate list of sentences, determine which contains the\nanswer. Although important, answer selection is only one stage in a standard\nend-to-end question answering pipeline. This paper explores the effectiveness\nof convolutional neural networks (CNNs) for answer selection in an end-to-end\ncontext using the standard TrecQA dataset. We observe that a simple\nidf-weighted word overlap algorithm forms a very strong baseline, and that\ndespite substantial efforts by the community in applying deep learning to\ntackle answer selection, the gains are modest at best on this dataset.\nFurthermore, it is unclear if a CNN is more effective than the baseline in an\nend-to-end context based on standard retrieval metrics. To further explore this\nfinding, we conducted a manual user evaluation, which confirms that answers\nfrom the CNN are detectably better than those from idf-weighted word overlap.\nThis result suggests that users are sensitive to relatively small differences\nin answer selection quality.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07835v1",
            "date": "2017-07-25 06:57:39+00:00",
            "title": "Towards Semantic Query Segmentation",
            "authors": [
                "Ajinkya Kale",
                "Thrivikrama Taula",
                "Sanjika Hewavitharana",
                "Amit Srivastava"
            ],
            "abstract": "Query Segmentation is one of the critical components for understanding users'\nsearch intent in Information Retrieval tasks. It involves grouping tokens in\nthe search query into meaningful phrases which help downstream tasks like\nsearch relevance and query understanding. In this paper, we propose a novel\napproach to segment user queries using distributed query embeddings. Our key\ncontribution is a supervised approach to the segmentation task using\nlow-dimensional feature vectors for queries, getting rid of traditional hand\ntuned and heuristic NLP features which are quite expensive.\n  We benchmark on a 50,000 human-annotated web search engine query corpus\nachieving comparable accuracy to state-of-the-art techniques. The advantage of\nour technique is its fast and does not use external knowledge-base like\nWikipedia for score boosting. This helps us generalize our approach to other\ndomains like eCommerce without any fine-tuning. We demonstrate the\neffectiveness of this method on another 50,000 human-annotated eCommerce query\ncorpus from eBay search logs. Our approach is easy to implement and generalizes\nwell across different search domains proving the power of low-dimensional\nembeddings in query segmentation task, opening up a new direction of research\nfor this problem.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07847v3",
            "date": "2017-07-25 08:21:30+00:00",
            "title": "Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "The dominant neural architectures in question answer retrieval are based on\nrecurrent or convolutional encoders configured with complex word matching\nlayers. Given that recent architectural innovations are mostly new word\ninteraction layers or attention-based matching mechanisms, it seems to be a\nwell-established fact that these components are mandatory for good performance.\nUnfortunately, the memory and computation cost incurred by these complex\nmechanisms are undesirable for practical applications. As such, this paper\ntackles the question of whether it is possible to achieve competitive\nperformance with simple neural architectures. We propose a simple but novel\ndeep learning architecture for fast and efficient question-answer ranking and\nretrieval. More specifically, our proposed model, \\textsc{HyperQA}, is a\nparameter efficient neural network that outperforms other parameter intensive\nmodels such as Attentive Pooling BiLSTMs and Multi-Perspective CNNs on multiple\nQA benchmarks. The novelty behind \\textsc{HyperQA} is a pairwise ranking\nobjective that models the relationship between question and answer embeddings\nin Hyperbolic space instead of Euclidean space. This empowers our model with a\nself-organizing ability and enables automatic discovery of latent hierarchies\nwhile learning embeddings of questions and answers. Our model requires no\nfeature engineering, no similarity matrix matching, no complicated attention\nmechanisms nor over-parameterized layers and yet outperforms and remains\ncompetitive to many models that have these functionalities on multiple\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.08139v1",
            "date": "2017-07-25 18:10:48+00:00",
            "title": "Analogs of Linguistic Structure in Deep Representations",
            "authors": [
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "We investigate the compositional structure of message vectors computed by a\ndeep network trained on a communication game. By comparing truth-conditional\nrepresentations of encoder-produced message vectors to human-produced referring\nexpressions, we are able to identify aligned (vector, utterance) pairs with the\nsame meaning. We then search for structured relationships among these aligned\npairs to discover simple vector space transformations corresponding to\nnegation, conjunction, and disjunction. Our results suggest that neural\nrepresentations are capable of spontaneously developing a \"syntax\" with\nfunctional analogues to qualitative properties of natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08470v1",
            "date": "2017-07-26 14:36:58+00:00",
            "title": "Implicit Entity Linking in Tweets",
            "authors": [
                "Sujan Perera",
                "Pablo N. Mendes",
                "Adarsh Alex",
                "Amit Sheth",
                "Krishnaprasad Thirunarayan"
            ],
            "abstract": "Over the years, Twitter has become one of the largest communication platforms\nproviding key data to various applications such as brand monitoring, trend\ndetection, among others. Entity linking is one of the major tasks in natural\nlanguage understanding from tweets and it associates entity mentions in text to\ncorresponding entries in knowledge bases in order to provide unambiguous\ninterpretation and additional con- text. State-of-the-art techniques have\nfocused on linking explicitly mentioned entities in tweets with reasonable\nsuccess. However, we argue that in addition to explicit mentions i.e. The movie\nGravity was more ex- pensive than the mars orbiter mission entities (movie\nGravity) can also be mentioned implicitly i.e. This new space movie is crazy.\nyou must watch it!. This paper introduces the problem of implicit entity\nlinking in tweets. We propose an approach that models the entities by\nexploiting their factual and contextual knowledge. We demonstrate how to use\nthese models to perform implicit entity linking on a ground truth dataset with\n397 tweets from two domains, namely, Movie and Book. Specifically, we show: 1)\nthe importance of linking implicit entities and its value addition to the\nstandard entity linking task, and 2) the importance of exploiting contextual\nknowledge associated with an entity for linking their implicit mentions. We\nalso make the ground truth dataset publicly available to foster the research in\nthis new research area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08588v1",
            "date": "2017-07-26 18:01:32+00:00",
            "title": "Self-organized Hierarchical Softmax",
            "authors": [
                "Yikang Shen",
                "Shawn Tan",
                "Chrisopher Pal",
                "Aaron Courville"
            ],
            "abstract": "We propose a new self-organizing hierarchical softmax formulation for\nneural-network-based language models over large vocabularies. Instead of using\na predefined hierarchical structure, our approach is capable of learning word\nclusters with clear syntactical and semantic meaning during the language model\ntraining process. We provide experiments on standard benchmarks for language\nmodeling and sentence compression tasks. We find that this approach is as fast\nas other efficient softmax approximations, while achieving comparable or even\nbetter performance relative to similar full softmax models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08668v1",
            "date": "2017-07-26 23:57:29+00:00",
            "title": "A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions",
            "authors": [
                "Siddharth Karamcheti",
                "Edward C. Williams",
                "Dilip Arumugam",
                "Mina Rhee",
                "Nakul Gopalan",
                "Lawson L. S. Wong",
                "Stefanie Tellex"
            ],
            "abstract": "Robots operating alongside humans in diverse, stochastic environments must be\nable to accurately interpret natural language commands. These instructions\noften fall into one of two categories: those that specify a goal condition or\ntarget state, and those that specify explicit actions, or how to perform a\ngiven task. Recent approaches have used reward functions as a semantic\nrepresentation of goal-based commands, which allows for the use of a\nstate-of-the-art planner to find a policy for the given task. However, these\nreward functions cannot be directly used to represent action-oriented commands.\nWe introduce a new hybrid approach, the Deep Recurrent Action-Goal Grounding\nNetwork (DRAGGN), for task grounding and execution that handles natural\nlanguage from either category as input, and generalizes to unseen environments.\nOur robot-simulation results demonstrate that a system successfully\ninterpreting both goal-oriented and action-oriented task specifications brings\nus closer to robust natural language understanding for human-robot interaction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.08783v4",
            "date": "2017-07-27 08:56:29+00:00",
            "title": "Analysis of Italian Word Embeddings",
            "authors": [
                "Rocco Tripodi",
                "Stefano Li Pira"
            ],
            "abstract": "In this work we analyze the performances of two of the most used word\nembeddings algorithms, skip-gram and continuous bag of words on Italian\nlanguage. These algorithms have many hyper-parameter that have to be carefully\ntuned in order to obtain accurate word representation in vectorial space. We\nprovide an accurate analysis and an evaluation, showing what are the best\nconfiguration of parameters for specific tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08866v1",
            "date": "2017-07-27 13:56:36+00:00",
            "title": "Deep Residual Learning for Weakly-Supervised Relation Extraction",
            "authors": [
                "Yi Yao Huang",
                "William Yang Wang"
            ],
            "abstract": "Deep residual learning (ResNet) is a new method for training very deep neural\nnetworks using identity map-ping for shortcut connections. ResNet has won the\nImageNet ILSVRC 2015 classification task, and achieved state-of-the-art\nperformances in many computer vision tasks. However, the effect of residual\nlearning on noisy natural language processing tasks is still not well\nunderstood. In this paper, we design a novel convolutional neural network (CNN)\nwith residual learning, and investigate its impacts on the task of distantly\nsupervised noisy relation extraction. In contradictory to popular beliefs that\nResNet only works well for very deep networks, we found that even with 9 layers\nof CNNs, using identity mapping could significantly improve the performance for\ndistantly-supervised relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09050v1",
            "date": "2017-07-27 21:16:41+00:00",
            "title": "A Shared Task on Bandit Learning for Machine Translation",
            "authors": [
                "Artem Sokolov",
                "Julia Kreutzer",
                "Kellen Sunderland",
                "Pavel Danchenko",
                "Witold Szymaniak",
                "Hagen F\u00fcrstenau",
                "Stefan Riezler"
            ],
            "abstract": "We introduce and describe the results of a novel shared task on bandit\nlearning for machine translation. The task was organized jointly by Amazon and\nHeidelberg University for the first time at the Second Conference on Machine\nTranslation (WMT 2017). The goal of the task is to encourage research on\nlearning machine translation from weak user feedback instead of human\nreferences or post-edits. On each of a sequence of rounds, a machine\ntranslation system is required to propose a translation for an input, and\nreceives a real-valued estimate of the quality of the proposed translation for\nlearning. This paper describes the shared task's learning and evaluation setup,\nusing services hosted on Amazon Web Services (AWS), the data and evaluation\nmetrics, and the results of various machine translation architectures and\nlearning protocols.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09098v1",
            "date": "2017-07-28 03:41:18+00:00",
            "title": "MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension",
            "authors": [
                "Boyuan Pan",
                "Hao Li",
                "Zhou Zhao",
                "Bin Cao",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "Machine comprehension(MC) style question answering is a representative\nproblem in natural language processing. Previous methods rarely spend time on\nthe improvement of encoding layer, especially the embedding of syntactic\ninformation and name entity of the words, which are very crucial to the quality\nof encoding. Moreover, existing attention methods represent each query word as\na vector or use a single vector to represent the whole query sentence, neither\nof them can handle the proper weight of the key words in query sentence. In\nthis paper, we introduce a novel neural network architecture called Multi-layer\nEmbedding with Memory Network(MEMEN) for machine reading task. In the encoding\nlayer, we employ classic skip-gram model to the syntactic and semantic\ninformation of the words to train a new kind of embedding layer. We also\npropose a memory network of full-orientation matching of the query and passage\nto catch more pivotal information. Experiments show that our model has\ncompetitive results both from the perspectives of precision and efficiency in\nStanford Question Answering Dataset(SQuAD) among all published results and\nachieves the state-of-the-art results on TriviaQA dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.09468v2",
            "date": "2017-07-29 06:05:52+00:00",
            "title": "Zero-Shot Activity Recognition with Verb Attribute Induction",
            "authors": [
                "Rowan Zellers",
                "Yejin Choi"
            ],
            "abstract": "In this paper, we investigate large-scale zero-shot activity recognition by\nmodeling the visual and linguistic attributes of action verbs. For example, the\nverb \"salute\" has several properties, such as being a light movement, a social\nact, and short in duration. We use these attributes as the internal mapping\nbetween visual and textual representations to reason about a previously unseen\naction. In contrast to much prior work that assumes access to gold standard\nattributes for zero-shot classes and focuses primarily on object attributes,\nour model uniquely learns to infer action attributes from dictionary\ndefinitions and distributed word representations. Experimental results confirm\nthat action attributes inferred from language can provide a predictive signal\nfor zero-shot prediction of previously unseen activities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09538v1",
            "date": "2017-07-29 16:40:50+00:00",
            "title": "Benchmarking Multimodal Sentiment Analysis",
            "authors": [
                "Erik Cambria",
                "Devamanyu Hazarika",
                "Soujanya Poria",
                "Amir Hussain",
                "R. B. V. Subramaanyam"
            ],
            "abstract": "We propose a framework for multimodal sentiment analysis and emotion\nrecognition using convolutional neural network-based feature extraction from\ntext and visual modalities. We obtain a performance improvement of 10% over the\nstate of the art by combining visual, text and audio features. We also discuss\nsome major issues frequently ignored in multimodal sentiment analysis research:\nthe role of speaker-independent models, importance of the modalities and\ngeneralizability. The paper thus serve as a new benchmark for further research\nin multimodal sentiment analysis and also demonstrates the different facets of\nanalysis to be considered while performing such tasks.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1707.09823v1",
            "date": "2017-07-31 12:48:45+00:00",
            "title": "Familia: An Open-Source Toolkit for Industrial Topic Modeling",
            "authors": [
                "Di Jiang",
                "Zeyu Chen",
                "Rongzhong Lian",
                "Siqi Bao",
                "Chen Li"
            ],
            "abstract": "Familia is an open-source toolkit for pragmatic topic modeling in industry.\nFamilia abstracts the utilities of topic modeling in industry as two paradigms:\nsemantic representation and semantic matching. Efficient implementations of the\ntwo paradigms are made publicly available for the first time. Furthermore, we\nprovide off-the-shelf topic models trained on large-scale industrial corpora,\nincluding Latent Dirichlet Allocation (LDA), SentenceLDA and Topical Word\nEmbedding (TWE). We further describe typical applications which are\nsuccessfully powered by topic modeling, in order to ease the confusions and\ndifficulties of software engineers during topic model selection and\nutilization.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.09861v1",
            "date": "2017-07-31 14:25:24+00:00",
            "title": "Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "In this paper we show that reporting a single performance score is\ninsufficient to compare non-deterministic approaches. We demonstrate for common\nsequence tagging tasks that the seed value for the random number generator can\nresult in statistically significant (p < 10^-4) differences for\nstate-of-the-art systems. For two recent systems for NER, we observe an\nabsolute difference of one percentage point F1-score depending on the selected\nseed value, making these systems perceived either as state-of-the-art or\nmediocre. Instead of publishing and reporting single performance scores, we\npropose to compare score distributions based on multiple executions. Based on\nthe evaluation of 50.000 LSTM-networks for five sequence tagging tasks, we\npresent network architectures that produce both superior performance as well as\nare more stable with respect to the remaining hyperparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00043v2",
            "date": "2017-10-31 18:31:11+00:00",
            "title": "Unsupervised Machine Translation Using Monolingual Corpora Only",
            "authors": [
                "Guillaume Lample",
                "Alexis Conneau",
                "Ludovic Denoyer",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Machine translation has recently achieved impressive performance thanks to\nrecent advances in deep learning and the availability of large-scale parallel\ncorpora. There have been numerous attempts to extend these successes to\nlow-resource language pairs, yet requiring tens of thousands of parallel\nsentences. In this work, we take this research direction to the extreme and\ninvestigate whether it is possible to learn to translate even without any\nparallel data. We propose a model that takes sentences from monolingual corpora\nin two different languages and maps them into the same latent space. By\nlearning to reconstruct in both languages from this shared feature space, the\nmodel effectively learns to translate without using any labeled data. We\ndemonstrate our model on two widely used datasets and two language pairs,\nreporting BLEU scores of 32.8 and 15.1 on the Multi30k and WMT English-French\ndatasets, without using even a single parallel sentence at training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00106v2",
            "date": "2017-10-31 20:53:42+00:00",
            "title": "DCN+: Mixed Objective and Deep Residual Coattention for Question Answering",
            "authors": [
                "Caiming Xiong",
                "Victor Zhong",
                "Richard Socher"
            ],
            "abstract": "Traditional models for question answering optimize using cross entropy loss,\nwhich encourages exact answers at the cost of penalizing nearby or overlapping\nanswers that are sometimes equally accurate. We propose a mixed objective that\ncombines cross entropy loss with self-critical policy learning. The objective\nuses rewards derived from word overlap to solve the misalignment between\nevaluation metric and optimization objective. In addition to the mixed\nobjective, we improve dynamic coattention networks (DCN) with a deep residual\ncoattention encoder that is inspired by recent work in deep self-attention and\nresidual networks. Our proposals improve model performance across question\ntypes and input lengths, especially for long questions that requires the\nability to capture long-term dependencies. On the Stanford Question Answering\nDataset, our model achieves state-of-the-art results with 75.1% exact match\naccuracy and 83.1% F1, while the ensemble obtains 78.9% exact match accuracy\nand 86.0% F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00482v1",
            "date": "2017-11-01 18:00:22+00:00",
            "title": "Learning with Latent Language",
            "authors": [
                "Jacob Andreas",
                "Dan Klein",
                "Sergey Levine"
            ],
            "abstract": "The named concepts and compositional operators present in natural language\nprovide a rich source of information about the kinds of abstractions humans use\nto navigate the world. Can this linguistic background knowledge improve the\ngenerality and efficiency of learned classifiers and control policies? This\npaper aims to show that using the space of natural language strings as a\nparameter space is an effective way to capture natural task structure. In a\npretraining phase, we learn a language interpretation model that transforms\ninputs (e.g. images) into outputs (e.g. labels) given natural language\ndescriptions. To learn a new concept (e.g. a classifier), we search directly in\nthe space of descriptions to minimize the interpreter's loss on training\nexamples. Crucially, our models do not require language data to learn these\nconcepts: language is used only in pretraining to impose structure on\nsubsequent learning. Results on image classification, text editing, and\nreinforcement learning show that, in all settings, models with a linguistic\nparameterization outperform those without.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00520v1",
            "date": "2017-11-01 19:40:00+00:00",
            "title": "Uncovering Latent Style Factors for Expressive Speech Synthesis",
            "authors": [
                "Yuxuan Wang",
                "RJ Skerry-Ryan",
                "Ying Xiao",
                "Daisy Stanton",
                "Joel Shor",
                "Eric Battenberg",
                "Rob Clark",
                "Rif A. Saurous"
            ],
            "abstract": "Prosodic modeling is a core problem in speech synthesis. The key challenge is\nproducing desirable prosody from textual input containing only phonetic\ninformation. In this preliminary study, we introduce the concept of \"style\ntokens\" in Tacotron, a recently proposed end-to-end neural speech synthesis\nmodel. Using style tokens, we aim to extract independent prosodic styles from\ntraining data. We show that without annotation data or an explicit supervision\nsignal, our approach can automatically learn a variety of prosodic variations\nin a purely data-driven way. Importantly, each style token corresponds to a\nfixed style factor regardless of the given text sequence. As a result, we can\ncontrol the prosodic style of synthetic speech in a somewhat predictable and\nglobally consistent way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00681v3",
            "date": "2017-11-02 11:00:09+00:00",
            "title": "Extracting an English-Persian Parallel Corpus from Comparable Corpora",
            "authors": [
                "Akbar Karimi",
                "Ebrahim Ansari",
                "Bahram Sadeghi Bigham"
            ],
            "abstract": "Parallel data are an important part of a reliable Statistical Machine\nTranslation (SMT) system. The more of these data are available, the better the\nquality of the SMT system. However, for some language pairs such as\nPersian-English, parallel sources of this kind are scarce. In this paper, a\nbidirectional method is proposed to extract parallel sentences from English and\nPersian document aligned Wikipedia. Two machine translation systems are\nemployed to translate from Persian to English and the reverse after which an IR\nsystem is used to measure the similarity of the translated sentences. Adding\nthe extracted sentences to the training data of the existing SMT systems is\nshown to improve the quality of the translation. Furthermore, the proposed\nmethod slightly outperforms the one-directional approach. The extracted corpus\nconsists of about 200,000 sentences which have been sorted by their degree of\nsimilarity calculated by the IR system and is freely available for public\naccess on the Web.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01563v1",
            "date": "2017-11-05 11:34:46+00:00",
            "title": "Multi-label Dataless Text Classification with Topic Modeling",
            "authors": [
                "Daochen Zha",
                "Chenliang Li"
            ],
            "abstract": "Manually labeling documents is tedious and expensive, but it is essential for\ntraining a traditional text classifier. In recent years, a few dataless text\nclassification techniques have been proposed to address this problem. However,\nexisting works mainly center on single-label classification problems, that is,\neach document is restricted to belonging to a single category. In this paper,\nwe propose a novel Seed-guided Multi-label Topic Model, named SMTM. With a few\nseed words relevant to each category, SMTM conducts multi-label classification\nfor a collection of documents without any labeled document. In SMTM, each\ncategory is associated with a single category-topic which covers the meaning of\nthe category. To accommodate with multi-labeled documents, we explicitly model\nthe category sparsity in SMTM by using spike and slab prior and weak smoothing\nprior. That is, without using any threshold tuning, SMTM automatically selects\nthe relevant categories for each document. To incorporate the supervision of\nthe seed words, we propose a seed-guided biased GPU (i.e., generalized Polya\nurn) sampling procedure to guide the topic inference of SMTM. Experiments on\ntwo public datasets show that SMTM achieves better classification accuracy than\nstate-of-the-art alternatives and even outperforms supervised solutions in some\nscenarios.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.01567v1",
            "date": "2017-11-05 12:00:18+00:00",
            "title": "Robust Speech Recognition Using Generative Adversarial Networks",
            "authors": [
                "Anuroop Sriram",
                "Heewoo Jun",
                "Yashesh Gaur",
                "Sanjeev Satheesh"
            ],
            "abstract": "This paper describes a general, scalable, end-to-end framework that uses the\ngenerative adversarial network (GAN) objective to enable robust speech\nrecognition. Encoders trained with the proposed approach enjoy improved\ninvariance by learning to map noisy audio to the same embedding space as that\nof clean audio. Unlike previous methods, the new framework does not rely on\ndomain expertise or simplifying assumptions as are often needed in signal\nprocessing, and directly encourages robustness in a data-driven way. We show\nthe new approach improves simulated far-field speech recognition of vanilla\nsequence-to-sequence models without specialized front-ends or preprocessing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02012v1",
            "date": "2017-11-02 20:04:06+00:00",
            "title": "Hi, how can I help you?: Automating enterprise IT support help desks",
            "authors": [
                "Senthil Mani",
                "Neelamadhav Gantayat",
                "Rahul Aralikatte",
                "Monika Gupta",
                "Sampath Dechu",
                "Anush Sankaran",
                "Shreya Khare",
                "Barry Mitchell",
                "Hemamalini Subramanian",
                "Hema Venkatarangan"
            ],
            "abstract": "Question answering is one of the primary challenges of natural language\nunderstanding. In realizing such a system, providing complex long answers to\nquestions is a challenging task as opposed to factoid answering as the former\nneeds context disambiguation. The different methods explored in the literature\ncan be broadly classified into three categories namely: 1) classification\nbased, 2) knowledge graph based and 3) retrieval based. Individually, none of\nthem address the need of an enterprise wide assistance system for an IT support\nand maintenance domain. In this domain the variance of answers is large ranging\nfrom factoid to structured operating procedures; the knowledge is present\nacross heterogeneous data sources like application specific documentation,\nticket management systems and any single technique for a general purpose\nassistance is unable to scale for such a landscape. To address this, we have\nbuilt a cognitive platform with capabilities adopted for this domain. Further,\nwe have built a general purpose question answering system leveraging the\nplatform that can be instantiated for multiple products, technologies in the\nsupport domain. The system uses a novel hybrid answering model that\norchestrates across a deep learning classifier, a knowledge graph based context\ndisambiguation module and a sophisticated bag-of-words search system. This\norchestration performs context switching for a provided question and also does\na smooth hand-off of the question to a human expert if none of the automated\ntechniques can provide a confident answer. This system has been deployed across\n675 internal enterprise IT support and maintenance projects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02013v2",
            "date": "2017-11-02 23:02:52+00:00",
            "title": "Neural Language Modeling by Jointly Learning Syntax and Lexicon",
            "authors": [
                "Yikang Shen",
                "Zhouhan Lin",
                "Chin-Wei Huang",
                "Aaron Courville"
            ],
            "abstract": "We propose a neural language model capable of unsupervised syntactic\nstructure induction. The model leverages the structure information to form\nbetter semantic representations and better language modeling. Standard\nrecurrent neural networks are limited by their structure and fail to\nefficiently use syntactic information. On the other hand, tree-structured\nrecursive networks usually require additional structural supervision at the\ncost of human expert annotation. In this paper, We propose a novel neural\nlanguage model, called the Parsing-Reading-Predict Networks (PRPN), that can\nsimultaneously induce the syntactic structure from unannotated sentences and\nleverage the inferred structure to learn a better language model. In our model,\nthe gradient can be directly back-propagated from the language model loss into\nthe neural parsing network. Experiments show that the proposed model can\ndiscover the underlying syntactic structure and achieve state-of-the-art\nperformance on word/character-level language model tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02132v1",
            "date": "2017-11-06 19:35:00+00:00",
            "title": "Weighted Transformer Network for Machine Translation",
            "authors": [
                "Karim Ahmed",
                "Nitish Shirish Keskar",
                "Richard Socher"
            ],
            "abstract": "State-of-the-art results on neural machine translation often use attentional\nsequence-to-sequence models with some form of convolution or recursion. Vaswani\net al. (2017) propose a new architecture that avoids recurrence and convolution\ncompletely. Instead, it uses only self-attention and feed-forward layers. While\nthe proposed architecture achieves state-of-the-art results on several machine\ntranslation tasks, it requires a large number of parameters and training\niterations to converge. We propose Weighted Transformer, a Transformer with\nmodified attention layers, that not only outperforms the baseline network in\nBLEU score but also converges 15-40% faster. Specifically, we replace the\nmulti-head attention by multiple self-attention branches that the model learns\nto combine during the training process. Our model improves the state-of-the-art\nperformance by 0.5 BLEU points on the WMT 2014 English-to-German translation\ntask and by 0.4 on the English-to-French translation task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.02173v2",
            "date": "2017-11-06 20:59:58+00:00",
            "title": "Synthetic and Natural Noise Both Break Neural Machine Translation",
            "authors": [
                "Yonatan Belinkov",
                "Yonatan Bisk"
            ],
            "abstract": "Character-based neural machine translation (NMT) models alleviate\nout-of-vocabulary issues, learn morphology, and move us closer to completely\nend-to-end translation systems. Unfortunately, they are also very brittle and\neasily falter when presented with noisy data. In this paper, we confront NMT\nmodels with synthetic and natural sources of noise. We find that\nstate-of-the-art models fail to translate even moderately noisy texts that\nhumans have no trouble comprehending. We explore two approaches to increase\nmodel robustness: structure-invariant word representations and robust training\non noisy texts. We find that a model based on a character convolutional neural\nnetwork is able to simultaneously learn representations robust to multiple\nkinds of noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02281v2",
            "date": "2017-11-07 04:42:48+00:00",
            "title": "Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Jiatao Gu",
                "James Bradbury",
                "Caiming Xiong",
                "Victor O. K. Li",
                "Richard Socher"
            ],
            "abstract": "Existing approaches to neural machine translation condition each output word\non previously generated outputs. We introduce a model that avoids this\nautoregressive property and produces its outputs in parallel, allowing an order\nof magnitude lower latency during inference. Through knowledge distillation,\nthe use of input token fertilities as a latent variable, and policy gradient\nfine-tuning, we achieve this at a cost of as little as 2.0 BLEU points relative\nto the autoregressive Transformer network used as a teacher. We demonstrate\nsubstantial cumulative improvements associated with each of the three aspects\nof our training strategy, and validate our approach on IWSLT 2016\nEnglish-German and two WMT language pairs. By sampling fertilities in parallel\nat inference time, our non-autoregressive model achieves near-state-of-the-art\nperformance of 29.8 BLEU on WMT 2016 English-Romanian.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.02604v1",
            "date": "2017-11-07 16:51:52+00:00",
            "title": "Unbounded cache model for online language modeling with open vocabulary",
            "authors": [
                "Edouard Grave",
                "Moustapha Cisse",
                "Armand Joulin"
            ],
            "abstract": "Recently, continuous cache models were proposed as extensions to recurrent\nneural network language models, to adapt their predictions to local changes in\nthe data distribution. These models only capture the local context, of up to a\nfew thousands tokens. In this paper, we propose an extension of continuous\ncache models, which can scale to larger contexts. In particular, we use a large\nscale non-parametric memory component that stores all the hidden activations\nseen in the past. We leverage recent advances in approximate nearest neighbor\nsearch and quantization algorithms to store millions of representations while\nsearching them efficiently. We conduct extensive experiments showing that our\napproach significantly improves the perplexity of pre-trained language models\non new distributions, and can scale efficiently to much larger contexts than\npreviously proposed local cache models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.03225v3",
            "date": "2017-11-09 01:41:12+00:00",
            "title": "Large-scale Cloze Test Dataset Created by Teachers",
            "authors": [
                "Qizhe Xie",
                "Guokun Lai",
                "Zihang Dai",
                "Eduard Hovy"
            ],
            "abstract": "Cloze tests are widely adopted in language exams to evaluate students'\nlanguage proficiency. In this paper, we propose the first large-scale\nhuman-created cloze test dataset CLOTH, containing questions used in\nmiddle-school and high-school language exams. With missing blanks carefully\ncreated by teachers and candidate choices purposely designed to be nuanced,\nCLOTH requires a deeper language understanding and a wider attention span than\npreviously automatically-generated cloze datasets. We test the performance of\ndedicatedly designed baseline models including a language model trained on the\nOne Billion Word Corpus and show humans outperform them by a significant\nmargin. We investigate the source of the performance gap, trace model\ndeficiencies to some distinct properties of CLOTH, and identify the limited\nability of comprehending the long-term context to be the key bottleneck.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03373v3",
            "date": "2017-11-09 13:39:21+00:00",
            "title": "SemRe-Rank: Improving Automatic Term Extraction By Incorporating Semantic Relatedness With Personalised PageRank",
            "authors": [
                "Ziqi Zhang",
                "Jie Gao",
                "Fabio Ciravegna"
            ],
            "abstract": "Automatic Term Extraction deals with the extraction of terminology from a\ndomain specific corpus, and has long been an established research area in data\nand knowledge acquisition. ATE remains a challenging task as it is known that\nthere is no existing ATE methods that can consistently outperform others in any\ndomain. This work adopts a refreshed perspective to this problem: instead of\nsearching for such a 'one-size-fit-all' solution that may never exist, we\npropose to develop generic methods to 'enhance' existing ATE methods. We\nintroduce SemRe-Rank, the first method based on this principle, to incorporate\nsemantic relatedness - an often overlooked venue - into an existing ATE method\nto further improve its performance. SemRe-Rank incorporates word embeddings\ninto a personalised PageRank process to compute 'semantic importance' scores\nfor candidate terms from a graph of semantically related words (nodes), which\nare then used to revise the scores of candidate terms computed by a base ATE\nalgorithm. Extensively evaluated with 13 state-of-the-art base ATE methods on\nfour datasets of diverse nature, it is shown to have achieved widespread\nimprovement over all base methods and across all datasets, with up to 15\npercentage points when measured by the Precision in the top ranked K candidate\nterms (the average for a set of K's), or up to 28 percentage points in F1\nmeasured at a K that equals to the expected real terms in the candidates (F1 in\nshort). Compared to an alternative approach built on the well-known TextRank\nalgorithm, SemRe-Rank can potentially outperform by up to 8 points in Precision\nat top K, or up to 17 points in F1.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.03438v1",
            "date": "2017-11-09 15:58:55+00:00",
            "title": "Open-World Knowledge Graph Completion",
            "authors": [
                "Baoxu Shi",
                "Tim Weninger"
            ],
            "abstract": "Knowledge Graphs (KGs) have been applied to many tasks including Web search,\nlink prediction, recommendation, natural language processing, and entity\nlinking. However, most KGs are far from complete and are growing at a rapid\npace. To address these problems, Knowledge Graph Completion (KGC) has been\nproposed to improve KGs by filling in its missing connections. Unlike existing\nmethods which hold a closed-world assumption, i.e., where KGs are fixed and new\nentities cannot be easily added, in the present work we relax this assumption\nand propose a new open-world KGC task. As a first attempt to solve this task we\nintroduce an open-world KGC model called ConMask. This model learns embeddings\nof the entity's name and parts of its text-description to connect unseen\nentities to the KG. To mitigate the presence of noisy text descriptions,\nConMask uses a relationship-dependent content masking to extract relevant\nsnippets and then trains a fully convolutional neural network to fuse the\nextracted snippets with entities in the KG. Experiments on large data sets,\nboth old and new, show that ConMask performs well in the open-world KGC task\nand even outperforms existing KGC models on the standard closed-world KGC task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.03953v4",
            "date": "2017-11-10 18:29:00+00:00",
            "title": "Breaking the Softmax Bottleneck: A High-Rank RNN Language Model",
            "authors": [
                "Zhilin Yang",
                "Zihang Dai",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "We formulate language modeling as a matrix factorization problem, and show\nthat the expressiveness of Softmax-based models (including the majority of\nneural language models) is limited by a Softmax bottleneck. Given that natural\nlanguage is highly context-dependent, this further implies that in practice\nSoftmax with distributed word embeddings does not have enough capacity to model\nnatural language. We propose a simple and effective method to address this\nissue, and improve the state-of-the-art perplexities on Penn Treebank and\nWikiText-2 to 47.69 and 40.68 respectively. The proposed method also excels on\nthe large-scale 1B Word dataset, outperforming the baseline by over 5.6 points\nin perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04071v3",
            "date": "2017-11-11 03:46:53+00:00",
            "title": "KBGAN: Adversarial Learning for Knowledge Graph Embeddings",
            "authors": [
                "Liwei Cai",
                "William Yang Wang"
            ],
            "abstract": "We introduce KBGAN, an adversarial learning framework to improve the\nperformances of a wide range of existing knowledge graph embedding models.\nBecause knowledge graphs typically only contain positive facts, sampling useful\nnegative training examples is a non-trivial task. Replacing the head or tail\nentity of a fact with a uniformly randomly selected entity is a conventional\nmethod for generating negative facts, but the majority of the generated\nnegative facts can be easily discriminated from positive facts, and will\ncontribute little towards the training. Inspired by generative adversarial\nnetworks (GANs), we use one knowledge graph embedding model as a negative\nsample generator to assist the training of our desired model, which acts as the\ndiscriminator in GANs. This framework is independent of the concrete form of\ngenerator and discriminator, and therefore can utilize a wide variety of\nknowledge graph embedding models as its building blocks. In experiments, we\nadversarially train two translation-based models, TransE and TransD, each with\nassistance from one of the two probability-based models, DistMult and ComplEx.\nWe evaluate the performances of KBGAN on the link prediction task, using three\nknowledge base completion datasets: FB15k-237, WN18 and WN18RR. Experimental\nresults show that adversarial training substantially improves the performances\nof target embedding models under various settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04079v1",
            "date": "2017-11-11 05:14:02+00:00",
            "title": "Fine Grained Knowledge Transfer for Personalized Task-oriented Dialogue Systems",
            "authors": [
                "Kaixiang Mo",
                "Yu Zhang",
                "Qiang Yang",
                "Pascale Fung"
            ],
            "abstract": "Training a personalized dialogue system requires a lot of data, and the data\ncollected for a single user is usually insufficient. One common practice for\nthis problem is to share training dialogues between different users and train\nmultiple sequence-to-sequence dialogue models together with transfer learning.\nHowever, current sequence-to-sequence transfer learning models operate on the\nentire sentence, which might cause negative transfer if different personal\ninformation from different users is mixed up. We propose a personalized decoder\nmodel to transfer finer granularity phrase-level knowledge between different\nusers while keeping personal preferences of each user intact. A novel personal\ncontrol gate is introduced, enabling the personalized decoder to switch between\ngenerating personalized phrases and shared phrases. The proposed personalized\ndecoder model can be easily combined with various deep models and can be\ntrained with reinforcement learning. Real-world experimental results\ndemonstrate that the phrase-level personalized decoder improves the BLEU over\nmultiple sentence-level transfer baseline models by as much as 7.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04090v2",
            "date": "2017-11-11 07:20:51+00:00",
            "title": "MojiTalk: Generating Emotional Responses at Scale",
            "authors": [
                "Xianda Zhou",
                "William Yang Wang"
            ],
            "abstract": "Generating emotional language is a key step towards building empathetic\nnatural language processing agents. However, a major challenge for this line of\nresearch is the lack of large-scale labeled training data, and previous studies\nare limited to only small sets of human annotated sentiment labels.\nAdditionally, explicitly controlling the emotion and sentiment of generated\ntext is also difficult. In this paper, we take a more radical approach: we\nexploit the idea of leveraging Twitter data that are naturally labeled with\nemojis. More specifically, we collect a large corpus of Twitter conversations\nthat include emojis in the response, and assume the emojis convey the\nunderlying emotions of the sentence. We then introduce a reinforced conditional\nvariational encoder approach to train a deep generative model on these\nconversations, which allows us to use emojis to control the emotion of the\ngenerated text. Experimentally, we show in our quantitative and qualitative\nanalyses that the proposed models can successfully generate high-quality\nabstractive conversation responses in accordance with designated emotions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04204v3",
            "date": "2017-11-11 22:25:55+00:00",
            "title": "Automatic Extraction of Commonsense LocatedNear Knowledge",
            "authors": [
                "Frank F. Xu",
                "Bill Yuchen Lin",
                "Kenny Q. Zhu"
            ],
            "abstract": "LocatedNear relation is a kind of commonsense knowledge describing two\nphysical objects that are typically found near each other in real life. In this\npaper, we study how to automatically extract such relationship through a\nsentence-level relation classifier and aggregating the scores of entity pairs\nfrom a large corpus. Also, we release two benchmark datasets for evaluation and\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04434v1",
            "date": "2017-11-13 06:34:29+00:00",
            "title": "Faithful to the Original: Fact Aware Neural Abstractive Summarization",
            "authors": [
                "Ziqiang Cao",
                "Furu Wei",
                "Wenjie Li",
                "Sujian Li"
            ],
            "abstract": "Unlike extractive summarization, abstractive summarization has to fuse\ndifferent parts of the source text, which inclines to create fake facts. Our\npreliminary study reveals nearly 30% of the outputs from a state-of-the-art\nneural summarization system suffer from this problem. While previous\nabstractive summarization approaches usually focus on the improvement of\ninformativeness, we argue that faithfulness is also a vital prerequisite for a\npractical abstractive summarization system. To avoid generating fake facts in a\nsummary, we leverage open information extraction and dependency parse\ntechnologies to extract actual fact descriptions from the source text. The\ndual-attention sequence-to-sequence framework is then proposed to force the\ngeneration conditioned on both the source text and the extracted fact\ndescriptions. Experiments on the Gigaword benchmark dataset demonstrate that\nour model can greatly reduce fake summaries by 80%. Notably, the fact\ndescriptions also bring significant improvement on informativeness since they\noften condense the meaning of the source text.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.04903v2",
            "date": "2017-11-14 01:50:30+00:00",
            "title": "Robust Multilingual Part-of-Speech Tagging via Adversarial Training",
            "authors": [
                "Michihiro Yasunaga",
                "Jungo Kasai",
                "Dragomir Radev"
            ],
            "abstract": "Adversarial training (AT) is a powerful regularization method for neural\nnetworks, aiming to achieve robustness to input perturbations. Yet, the\nspecific effects of the robustness obtained from AT are still unclear in the\ncontext of natural language processing. In this paper, we propose and analyze a\nneural POS tagging model that exploits AT. In our experiments on the Penn\nTreebank WSJ corpus and the Universal Dependencies (UD) dataset (27 languages),\nwe find that AT not only improves the overall tagging accuracy, but also 1)\nprevents over-fitting well in low resource languages and 2) boosts tagging\naccuracy for rare / unseen words. We also demonstrate that 3) the improved\ntagging performance by AT contributes to the downstream task of dependency\nparsing, and that 4) AT helps the model to learn cleaner word representations.\n5) The proposed AT model is generally effective in different sequence labeling\ntasks. These positive results motivate further use of AT for natural language\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04981v1",
            "date": "2017-11-14 07:20:23+00:00",
            "title": "SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring",
            "authors": [
                "Yi Tay",
                "Minh C. Phan",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Deep learning has demonstrated tremendous potential for Automatic Text\nScoring (ATS) tasks. In this paper, we describe a new neural architecture that\nenhances vanilla neural network models with auxiliary neural coherence\nfeatures. Our new method proposes a new \\textsc{SkipFlow} mechanism that models\nrelationships between snapshots of the hidden representations of a long\nshort-term memory (LSTM) network as it reads. Subsequently, the semantic\nrelationships between multiple snapshots are used as auxiliary features for\nprediction. This has two main benefits. Firstly, essays are typically long\nsequences and therefore the memorization capability of the LSTM network may be\ninsufficient. Implicit access to multiple snapshots can alleviate this problem\nby acting as a protection against vanishing gradients. The parameters of the\n\\textsc{SkipFlow} mechanism also acts as an auxiliary memory. Secondly,\nmodeling relationships between multiple positions allows our model to learn\nfeatures that represent and approximate textual coherence. In our model, we\ncall this \\textit{neural coherence} features. Overall, we present a unified\ndeep learning architecture that generates neural coherence features as it reads\nin an end-to-end fashion. Our approach demonstrates state-of-the-art\nperformance on the benchmark ASAP dataset, outperforming not only feature\nengineering baselines but also other deep learning models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.05116v2",
            "date": "2017-11-14 14:39:51+00:00",
            "title": "Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering",
            "authors": [
                "Shuohang Wang",
                "Mo Yu",
                "Jing Jiang",
                "Wei Zhang",
                "Xiaoxiao Guo",
                "Shiyu Chang",
                "Zhiguo Wang",
                "Tim Klinger",
                "Gerald Tesauro",
                "Murray Campbell"
            ],
            "abstract": "A popular recent approach to answering open-domain questions is to first\nsearch for question-related passages and then apply reading comprehension\nmodels to extract answers. Existing methods usually extract answers from single\npassages independently. But some questions require a combination of evidence\nfrom across different sources to answer correctly. In this paper, we propose\ntwo models which make use of multiple passages to generate their answers. Both\nuse an answer-reranking approach which reorders the answer candidates generated\nby an existing state-of-the-art QA model. We propose two methods, namely,\nstrength-based re-ranking and coverage-based re-ranking, to make use of the\naggregated evidence from different passages to better determine the answer. Our\nmodels have achieved state-of-the-art results on three public open-domain QA\ndatasets: Quasar-T, SearchQA and the open-domain version of TriviaQA, with\nabout 8 percentage points of improvement over the former two datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05472v1",
            "date": "2017-11-15 09:33:44+00:00",
            "title": "Can clone detection support quality assessments of requirements specifications?",
            "authors": [
                "Elmar Juergens",
                "Florian Deissenboeck",
                "Martin Feilkas",
                "Benjamin Hummel",
                "Bernhard Schaetz",
                "Stefan Wagner",
                "Christoph Domann",
                "Jonathan Streit"
            ],
            "abstract": "Due to their pivotal role in software engineering, considerable effort is\nspent on the quality assurance of software requirements specifications. As they\nare mainly described in natural language, relatively few means of automated\nquality assessment exist. However, we found that clone detection, a technique\nwidely applied to source code, is promising to assess one important quality\naspect in an automated way, namely redundancy that stems from copy&paste\noperations. This paper describes a large-scale case study that applied clone\ndetection to 28 requirements specifications with a total of 8,667 pages. We\nreport on the amount of redundancy found in real-world specifications, discuss\nits nature as well as its consequences and evaluate in how far existing code\nclone detection approaches can be applied to assess the quality of requirements\nspecifications in practice.",
            "journal ref": "Proceedings of the 32nd ACM/IEEE International Conference on\n  Software Engineering - Volume 2. Pages 79-88. ACM, 2010",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1711.05789v1",
            "date": "2017-11-15 20:26:42+00:00",
            "title": "CMU LiveMedQA at TREC 2017 LiveQA: A Consumer Health Question Answering System",
            "authors": [
                "Yuan Yang",
                "Jingcheng Yu",
                "Ye Hu",
                "Xiaoyao Xu",
                "Eric Nyberg"
            ],
            "abstract": "In this paper, we present LiveMedQA, a question answering system that is\noptimized for consumer health question. On top of the general QA system\npipeline, we introduce several new features that aim to exploit domain-specific\nknowledge and entity structures for better performance. This includes a\nquestion type/focus analyzer based on deep text classification model, a\ntree-based knowledge graph for answer generation and a complementary\nstructure-aware searcher for answer retrieval. LiveMedQA system is evaluated in\nthe TREC 2017 LiveQA medical subtask, where it received an average score of\n0.356 on a 3 point scale. Evaluation results revealed 3 substantial drawbacks\nin current LiveMedQA system, based on which we provide a detailed discussion\nand propose a few solutions that constitute the main focus of our subsequent\nwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05795v1",
            "date": "2017-11-15 20:37:44+00:00",
            "title": "Finer Grained Entity Typing with TypeNet",
            "authors": [
                "Shikhar Murty",
                "Patrick Verga",
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "We consider the challenging problem of entity typing over an extremely fine\ngrained set of types, wherein a single mention or entity can have many\nsimultaneous and often hierarchically-structured types. Despite the importance\nof the problem, there is a relative lack of resources in the form of\nfine-grained, deep type hierarchies aligned to existing knowledge bases. In\nresponse, we introduce TypeNet, a dataset of entity types consisting of over\n1941 types organized in a hierarchy, obtained by manually annotating a mapping\nfrom 1081 Freebase types to WordNet. We also experiment with several models\ncomparable to state-of-the-art systems and explore techniques to incorporate a\nstructure loss on the hierarchy with the standard mention typing loss, as a\nfirst step towards future research on this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05851v2",
            "date": "2017-11-15 23:45:18+00:00",
            "title": "Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning",
            "authors": [
                "Rajarshi Das",
                "Shehzaad Dhuliawala",
                "Manzil Zaheer",
                "Luke Vilnis",
                "Ishan Durugkar",
                "Akshay Krishnamurthy",
                "Alex Smola",
                "Andrew McCallum"
            ],
            "abstract": "Knowledge bases (KB), both automatically and manually constructed, are often\nincomplete --- many valid facts can be inferred from the KB by synthesizing\nexisting information. A popular approach to KB completion is to infer new\nrelations by combinatory reasoning over the information found along other paths\nconnecting a pair of entities. Given the enormous size of KBs and the\nexponential number of paths, previous path-based models have considered only\nthe problem of predicting a missing relation given two entities or evaluating\nthe truth of a proposed triple. Additionally, these methods have traditionally\nused random paths between fixed entity pairs or more recently learned to pick\npaths between them. We propose a new algorithm MINERVA, which addresses the\nmuch more difficult and practical task of answering questions where the\nrelation is known, but only one entity. Since random walks are impractical in a\nsetting with combinatorially many destinations from a start node, we present a\nneural reinforcement learning approach which learns how to navigate the graph\nconditioned on the input query to find predictive paths. Empirically, this\napproach obtains state-of-the-art results on several datasets, significantly\noutperforming prior methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06095v1",
            "date": "2017-11-10 12:47:52+00:00",
            "title": "Depression Severity Estimation from Multiple Modalities",
            "authors": [
                "Evgeny Stepanov",
                "Stephane Lathuiliere",
                "Shammur Absar Chowdhury",
                "Arindam Ghosh",
                "Radu-Laurentiu Vieriu",
                "Nicu Sebe",
                "Giuseppe Riccardi"
            ],
            "abstract": "Depression is a major debilitating disorder which can affect people from all\nages. With a continuous increase in the number of annual cases of depression,\nthere is a need to develop automatic techniques for the detection of the\npresence and extent of depression. In this AVEC challenge we explore different\nmodalities (speech, language and visual features extracted from face) to design\nand develop automatic methods for the detection of depression. In psychology\nliterature, the PHQ-8 questionnaire is well established as a tool for measuring\nthe severity of depression. In this paper we aim to automatically predict the\nPHQ-8 scores from features extracted from the different modalities. We show\nthat visual features extracted from facial landmarks obtain the best\nperformance in terms of estimating the PHQ-8 results with a mean absolute error\n(MAE) of 4.66 on the development set. Behavioral characteristics from speech\nprovide an MAE of 4.73. Language features yield a slightly higher MAE of 5.17.\nWhen switching to the test set, our Turn Features derived from audio\ntranscriptions achieve the best performance, scoring an MAE of 4.11\n(corresponding to an RMSE of 4.94), which makes our system the winner of the\nAVEC 2017 depression sub-challenge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.06196v3",
            "date": "2017-11-16 16:59:33+00:00",
            "title": "Addressing Cross-Lingual Word Sense Disambiguation on Low-Density Languages: Application to Persian",
            "authors": [
                "Navid Rekabsaz",
                "Mihai Lupu",
                "Allan Hanbury",
                "Andres Duque"
            ],
            "abstract": "We explore the use of unsupervised methods in Cross-Lingual Word Sense\nDisambiguation (CL-WSD) with the application of English to Persian. Our\nproposed approach targets the languages with scarce resources (low-density) by\nexploiting word embedding and semantic similarity of the words in context. We\nevaluate the approach on a recent evaluation benchmark and compare it with the\nstate-of-the-art unsupervised system (CO-Graph). The results show that our\napproach outperforms both the standard baseline and the CO-Graph system in both\nof the task evaluation metrics (Out-Of-Five and Best result).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06232v3",
            "date": "2017-11-16 18:27:49+00:00",
            "title": "A Novel Framework for Robustness Analysis of Visual QA Models",
            "authors": [
                "Jia-Hong Huang",
                "Cuong Duc Dao",
                "Modar Alfadly",
                "Bernard Ghanem"
            ],
            "abstract": "Deep neural networks have been playing an essential role in many computer\nvision tasks including Visual Question Answering (VQA). Until recently, the\nstudy of their accuracy was the main focus of research but now there is a trend\ntoward assessing the robustness of these models against adversarial attacks by\nevaluating their tolerance to varying noise levels. In VQA, adversarial attacks\ncan target the image and/or the proposed main question and yet there is a lack\nof proper analysis of the later. In this work, we propose a flexible framework\nthat focuses on the language part of VQA that uses semantically relevant\nquestions, dubbed basic questions, acting as controllable noise to evaluate the\nrobustness of VQA models. We hypothesize that the level of noise is positively\ncorrelated to the similarity of a basic question to the main question. Hence,\nto apply noise on any given main question, we rank a pool of basic questions\nbased on their similarity by casting this ranking task as a LASSO optimization\nproblem. Then, we propose a novel robustness measure, R_score, and two\nlarge-scale basic question datasets (BQDs) in order to standardize robustness\nanalysis for VQA models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.06744v4",
            "date": "2017-11-17 22:02:53+00:00",
            "title": "Learning to Organize Knowledge and Answer Questions with N-Gram Machines",
            "authors": [
                "Fan Yang",
                "Jiazhong Nie",
                "William W. Cohen",
                "Ni Lao"
            ],
            "abstract": "Though deep neural networks have great success in natural language\nprocessing, they are limited at more knowledge intensive AI tasks, such as\nopen-domain Question Answering (QA). Existing end-to-end deep QA models need to\nprocess the entire text after observing the question, and therefore their\ncomplexity in responding a question is linear in the text size. This is\nprohibitive for practical tasks such as QA from Wikipedia, a novel, or the Web.\nWe propose to solve this scalability issue by using symbolic meaning\nrepresentations, which can be indexed and retrieved efficiently with complexity\nthat is independent of the text size. We apply our approach, called the N-Gram\nMachine (NGM), to three representative tasks. First as proof-of-concept, we\ndemonstrate that NGM successfully solves the bAbI tasks of synthetic text.\nSecond, we show that NGM scales to large corpus by experimenting on \"life-long\nbAbI\", a special version of bAbI that contains millions of sentences. Lastly on\nthe WikiMovies dataset, we use NGM to induce latent structure (i.e. schema) and\nanswer questions from natural language Wikipedia text, with only QA pairs as\nweak supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06968v1",
            "date": "2017-11-19 05:14:36+00:00",
            "title": "Intelligent Word Embeddings of Free-Text Radiology Reports",
            "authors": [
                "Imon Banerjee",
                "Sriraman Madhavan",
                "Roger Eric Goldman",
                "Daniel L. Rubin"
            ],
            "abstract": "Radiology reports are a rich resource for advancing deep learning\napplications in medicine by leveraging the large volume of data continuously\nbeing updated, integrated, and shared. However, there are significant\nchallenges as well, largely due to the ambiguity and subtlety of natural\nlanguage. We propose a hybrid strategy that combines semantic-dictionary\nmapping and word2vec modeling for creating dense vector embeddings of free-text\nradiology reports. Our method leverages the benefits of both\nsemantic-dictionary mapping as well as unsupervised learning. Using the vector\nrepresentation, we automatically classify the radiology reports into three\nclasses denoting confidence in the diagnosis of intracranial hemorrhage by the\ninterpreting radiologist. We performed experiments with varying hyperparameter\nsettings of the word embeddings and a range of different classifiers. Best\nperformance achieved was a weighted precision of 88% and weighted recall of\n90%. Our work offers the potential to leverage unstructured electronic health\nrecord data by allowing direct analysis of narrative clinical notes.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.07341v2",
            "date": "2017-11-16 03:52:41+00:00",
            "title": "FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension",
            "authors": [
                "Hsin-Yuan Huang",
                "Chenguang Zhu",
                "Yelong Shen",
                "Weizhu Chen"
            ],
            "abstract": "This paper introduces a new neural structure called FusionNet, which extends\nexisting attention approaches from three perspectives. First, it puts forward a\nnovel concept of \"history of word\" to characterize attention information from\nthe lowest word-level embedding up to the highest semantic-level\nrepresentation. Second, it introduces an improved attention scoring function\nthat better utilizes the \"history of word\" concept. Third, it proposes a\nfully-aware multi-level attention mechanism to capture the complete information\nin one text (such as a question) and exploit it in its counterpart (such as\ncontext or passage) layer by layer. We apply FusionNet to the Stanford Question\nAnswering Dataset (SQuAD) and it achieves the first position for both single\nand ensemble model on the official SQuAD leaderboard at the time of writing\n(Oct. 4th, 2017). Meanwhile, we verify the generalization of FusionNet with two\nadversarial SQuAD datasets and it sets up the new state-of-the-art on both\ndatasets: on AddSent, FusionNet increases the best F1 metric from 46.6% to\n51.4%; on AddOneSent, FusionNet boosts the best F1 metric from 56.0% to 60.7%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08195v3",
            "date": "2017-11-22 09:45:51+00:00",
            "title": "On the Automatic Generation of Medical Imaging Reports",
            "authors": [
                "Baoyu Jing",
                "Pengtao Xie",
                "Eric Xing"
            ],
            "abstract": "Medical imaging is widely used in clinical practice for diagnosis and\ntreatment. Report-writing can be error-prone for unexperienced physicians, and\ntime- consuming and tedious for experienced physicians. To address these\nissues, we study the automatic generation of medical imaging reports. This task\npresents several challenges. First, a complete report contains multiple\nheterogeneous forms of information, including findings and tags. Second,\nabnormal regions in medical images are difficult to identify. Third, the re-\nports are typically long, containing multiple sentences. To cope with these\nchallenges, we (1) build a multi-task learning framework which jointly performs\nthe pre- diction of tags and the generation of para- graphs, (2) propose a\nco-attention mechanism to localize regions containing abnormalities and\ngenerate narrations for them, (3) develop a hierarchical LSTM model to generate\nlong paragraphs. We demonstrate the effectiveness of the proposed methods on\ntwo publicly available datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08412v1",
            "date": "2017-11-22 17:39:58+00:00",
            "title": "Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes",
            "authors": [
                "Nikhil Garg",
                "Londa Schiebinger",
                "Dan Jurafsky",
                "James Zou"
            ],
            "abstract": "Word embeddings use vectors to represent words such that the geometry between\nvectors captures semantic relationship between the words. In this paper, we\ndevelop a framework to demonstrate how the temporal dynamics of the embedding\ncan be leveraged to quantify changes in stereotypes and attitudes toward women\nand ethnic minorities in the 20th and 21st centuries in the United States. We\nintegrate word embeddings trained on 100 years of text data with the U.S.\nCensus to show that changes in the embedding track closely with demographic and\noccupation shifts over time. The embedding captures global social shifts --\ne.g., the women's movement in the 1960s and Asian immigration into the U.S --\nand also illuminates how specific adjectives and occupations became more\nclosely associated with certain populations over time. Our framework for\ntemporal analysis of word embedding opens up a powerful new intersection\nbetween machine learning and quantitative social science.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08521v1",
            "date": "2017-11-22 22:24:20+00:00",
            "title": "EMFET: E-mail Features Extraction Tool",
            "authors": [
                "Wadi' Hijawi",
                "Hossam Faris",
                "Ja'far Alqatawna",
                "Ibrahim Aljarah",
                "Ala' M. Al-Zoubi",
                "Maria Habib"
            ],
            "abstract": "EMFET is an open source and flexible tool that can be used to extract a large\nnumber of features from any email corpus with emails saved in EML format. The\nextracted features can be categorized into three main groups: header features,\npayload (body) features, and attachment features. The purpose of the tool is to\nhelp practitioners and researchers to build datasets that can be used for\ntraining machine learning models for spam detection. So far, 140 features can\nbe extracted using EMFET. EMFET is extensible and easy to use. The source code\nof EMFET is publicly available at GitHub\n(https://github.com/WadeaHijjawi/EmailFeaturesExtraction)",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.08609v1",
            "date": "2017-11-23 08:25:23+00:00",
            "title": "Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis",
            "authors": [
                "Seyed Mahdi Rezaeinia",
                "Ali Ghodsi",
                "Rouhollah Rahmani"
            ],
            "abstract": "Sentiment analysis is one of the well-known tasks and fast growing research\nareas in natural language processing (NLP) and text classifications. This\ntechnique has become an essential part of a wide range of applications\nincluding politics, business, advertising and marketing. There are various\ntechniques for sentiment analysis, but recently word embeddings methods have\nbeen widely used in sentiment classification tasks. Word2Vec and GloVe are\ncurrently among the most accurate and usable word embedding methods which can\nconvert words into meaningful vectors. However, these methods ignore sentiment\ninformation of texts and need a huge corpus of texts for training and\ngenerating exact vectors which are used as inputs of deep learning models. As a\nresult, because of the small size of some corpuses, researcher often have to\nuse pre-trained word embeddings which were trained on other large text corpus\nsuch as Google News with about 100 billion words. The increasing accuracy of\npre-trained word embeddings has a great impact on sentiment analysis research.\nIn this paper we propose a novel method, Improved Word Vectors (IWV), which\nincreases the accuracy of pre-trained word embeddings in sentiment analysis.\nOur method is based on Part-of-Speech (POS) tagging techniques, lexicon-based\napproaches and Word2Vec/GloVe methods. We tested the accuracy of our method via\ndifferent deep learning models and sentiment datasets. Our experiment results\nshow that Improved Word Vectors (IWV) are very effective for sentiment\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09357v1",
            "date": "2017-11-26 09:45:04+00:00",
            "title": "Generative Adversarial Network for Abstractive Text Summarization",
            "authors": [
                "Linqing Liu",
                "Yao Lu",
                "Min Yang",
                "Qiang Qu",
                "Jia Zhu",
                "Hongyan Li"
            ],
            "abstract": "In this paper, we propose an adversarial process for abstractive text\nsummarization, in which we simultaneously train a generative model G and a\ndiscriminative model D. In particular, we build the generator G as an agent of\nreinforcement learning, which takes the raw text as input and predicts the\nabstractive summarization. We also build a discriminator which attempts to\ndistinguish the generated summary from the ground truth summary. Extensive\nexperiments demonstrate that our model achieves competitive ROUGE scores with\nthe state-of-the-art methods on CNN/Daily Mail dataset. Qualitatively, we show\nthat our model is able to generate more abstractive, readable and diverse\nsummaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09573v2",
            "date": "2017-11-27 08:17:16+00:00",
            "title": "Code Completion with Neural Attention and Pointer Networks",
            "authors": [
                "Jian Li",
                "Yue Wang",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Intelligent code completion has become an essential research task to\naccelerate modern software development. To facilitate effective code completion\nfor dynamically-typed programming languages, we apply neural language models by\nlearning from large codebases, and develop a tailored attention mechanism for\ncode completion. However, standard neural language models even with attention\nmechanism cannot correctly predict the out-of-vocabulary (OoV) words that\nrestrict the code completion performance. In this paper, inspired by the\nprevalence of locally repeated terms in program source code, and the recently\nproposed pointer copy mechanism, we propose a pointer mixture network for\nbetter predicting OoV words in code completion. Based on the context, the\npointer mixture network learns to either generate a within-vocabulary word\nthrough an RNN component, or regenerate an OoV word from local context through\na pointer component. Experiments on two benchmarked datasets demonstrate the\neffectiveness of our attention mechanism and pointer mixture network on the\ncode completion task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09684v1",
            "date": "2017-11-27 13:40:15+00:00",
            "title": "Production Ready Chatbots: Generate if not Retrieve",
            "authors": [
                "Aniruddha Tammewar",
                "Monik Pamecha",
                "Chirag Jain",
                "Apurva Nagvenkar",
                "Krupal Modi"
            ],
            "abstract": "In this paper, we present a hybrid model that combines a neural\nconversational model and a rule-based graph dialogue system that assists users\nin scheduling reminders through a chat conversation. The graph based system has\nhigh precision and provides a grammatically accurate response but has a low\nrecall. The neural conversation model can cater to a variety of requests, as it\ngenerates the responses word by word as opposed to using canned responses. The\nhybrid system shows significant improvements over the existing baseline system\nof rule based approach and caters to complex queries with a domain-restricted\nneural model. Restricting the conversation topic and combination of graph based\nretrieval system with a neural generative model makes the final system robust\nenough for a real world application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09724v1",
            "date": "2017-11-27 14:55:17+00:00",
            "title": "Table-to-text Generation by Structure-aware Seq2seq Learning",
            "authors": [
                "Tianyu Liu",
                "Kexiang Wang",
                "Lei Sha",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "Table-to-text generation aims to generate a description for a factual table\nwhich can be viewed as a set of field-value records. To encode both the content\nand the structure of a table, we propose a novel structure-aware seq2seq\narchitecture which consists of field-gating encoder and description generator\nwith dual attention. In the encoding phase, we update the cell memory of the\nLSTM unit by a field gate and its corresponding field value in order to\nincorporate field information into table representation. In the decoding phase,\ndual attention mechanism which contains word level attention and field level\nattention is proposed to model the semantic relevance between the generated\ndescription and the table. We conduct experiments on the \\texttt{WIKIBIO}\ndataset which contains over 700k biographies and corresponding infoboxes from\nWikipedia. The attention visualizations and case studies show that our model is\ncapable of generating coherent and informative descriptions based on the\ncomprehensive understanding of both the content and the structure of a table.\nAutomatic evaluations also show our model outperforms the baselines by a great\nmargin. Code for this work is available on\nhttps://github.com/tyliupku/wiki2bio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.10307v2",
            "date": "2017-11-28 14:28:54+00:00",
            "title": "Semantic Technology-Assisted Review (STAR) Document analysis and monitoring using random vectors",
            "authors": [
                "Jean-Fran\u00e7ois Delpech"
            ],
            "abstract": "The review and analysis of large collections of documents and the periodic\nmonitoring of new additions thereto has greatly benefited from new developments\nin computer software. This paper demonstrates how using random vectors to\nconstruct a low-dimensional Euclidean space embedding words and documents\nenables fast and accurate computation of semantic similarities between them.\nWith this technique of Semantic Technology-Assisted Review (STAR), documents\ncan be selected, compared, classified, summarized and evaluated very quickly\nwith minimal expert involvement and high-quality results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.10377v2",
            "date": "2017-11-15 17:32:59+00:00",
            "title": "Sentiment analysis of twitter data",
            "authors": [
                "Hamid Bagheri",
                "Md Johirul Islam"
            ],
            "abstract": "Social networks are the main resources to gather information about people's\nopinion and sentiments towards different topics as they spend hours daily on\nsocial media and share their opinion. In this technical paper, we show the\napplication of sentimental analysis and how to connect to Twitter and run\nsentimental analysis queries. We run experiments on different queries from\npolitics to humanity and show the interesting results. We realized that the\nneutral sentiments for tweets are significantly high which clearly shows the\nlimitations of the current works.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1712.00733v1",
            "date": "2017-12-03 08:41:35+00:00",
            "title": "Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks",
            "authors": [
                "Guohao Li",
                "Hang Su",
                "Wenwu Zhu"
            ],
            "abstract": "Visual Question Answering (VQA) has attracted much attention since it offers\ninsight into the relationships between the multi-modal analysis of images and\nnatural language. Most of the current algorithms are incapable of answering\nopen-domain questions that require to perform reasoning beyond the image\ncontents. To address this issue, we propose a novel framework which endows the\nmodel capabilities in answering more complex questions by leveraging massive\nexternal knowledge with dynamic memory networks. Specifically, the questions\nalong with the corresponding images trigger a process to retrieve the relevant\ninformation in external knowledge bases, which are embedded into a continuous\nvector space by preserving the entity-relation structures. Afterwards, we\nemploy dynamic memory networks to attend to the large body of facts in the\nknowledge graph and images, and then perform reasoning over these facts to\ngenerate corresponding answers. Extensive experiments demonstrate that our\nmodel not only achieves the state-of-the-art performance in the visual question\nanswering task, but can also answer open-domain questions effectively by\nleveraging the external knowledge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.00991v1",
            "date": "2017-12-04 10:30:18+00:00",
            "title": "Mining Supervisor Evaluation and Peer Feedback in Performance Appraisals",
            "authors": [
                "Girish Keshav Palshikar",
                "Sachin Pawar",
                "Saheb Chourasia",
                "Nitin Ramrakhiyani"
            ],
            "abstract": "Performance appraisal (PA) is an important HR process to periodically measure\nand evaluate every employee's performance vis-a-vis the goals established by\nthe organization. A PA process involves purposeful multi-step multi-modal\ncommunication between employees, their supervisors and their peers, such as\nself-appraisal, supervisor assessment and peer feedback. Analysis of the\nstructured data and text produced in PA is crucial for measuring the quality of\nappraisals and tracking actual improvements. In this paper, we apply text\nmining techniques to produce insights from PA text. First, we perform sentence\nclassification to identify strengths, weaknesses and suggestions of\nimprovements found in the supervisor assessments and then use clustering to\ndiscover broad categories among them. Next we use multi-class multi-label\nclassification techniques to match supervisor assessments to predefined broad\nperspectives on performance. Finally, we propose a short-text summarization\ntechnique to produce a summary of peer feedback comments for a given employee\nand compare it with manual summaries. All techniques are illustrated using a\nreal-life dataset of supervisor assessment and peer feedback text produced\nduring the PA of 4528 employees in a large multi-national IT company.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01097v1",
            "date": "2017-11-29 21:20:51+00:00",
            "title": "Generalized Grounding Graphs: A Probabilistic Framework for Understanding Grounded Commands",
            "authors": [
                "Thomas Kollar",
                "Stefanie Tellex",
                "Matthew Walter",
                "Albert Huang",
                "Abraham Bachrach",
                "Sachi Hemachandra",
                "Emma Brunskill",
                "Ashis Banerjee",
                "Deb Roy",
                "Seth Teller",
                "Nicholas Roy"
            ],
            "abstract": "Many task domains require robots to interpret and act upon natural language\ncommands which are given by people and which refer to the robot's physical\nsurroundings. Such interpretation is known variously as the symbol grounding\nproblem, grounded semantics and grounded language acquisition. This problem is\nchallenging because people employ diverse vocabulary and grammar, and because\nrobots have substantial uncertainty about the nature and contents of their\nsurroundings, making it difficult to associate the constitutive language\nelements (principally noun phrases and spatial relations) of the command text\nto elements of those surroundings. Symbolic models capture linguistic structure\nbut have not scaled successfully to handle the diverse language produced by\nuntrained users. Existing statistical approaches can better handle diversity,\nbut have not to date modeled complex linguistic structure, limiting achievable\naccuracy. Recent hybrid approaches have addressed limitations in scaling and\ncomplexity, but have not effectively associated linguistic and perceptual\nfeatures. Our framework, called Generalized Grounding Graphs (G^3), addresses\nthese issues by defining a probabilistic graphical model dynamically according\nto the linguistic parse structure of a natural language command. This approach\nscales effectively, handles linguistic diversity, and enables the system to\nassociate parts of a command with the specific objects, places, and events in\nthe external world to which they refer. We show that robots can learn word\nmeanings and use those learned meanings to robustly follow natural language\ncommands produced by untrained users. We demonstrate our approach for both\nmobility commands and mobile manipulation commands involving a variety of\nsemi-autonomous robotic platforms, including a wheelchair, a micro-air vehicle,\na forklift, and the Willow Garage PR2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01213v1",
            "date": "2017-12-04 17:39:51+00:00",
            "title": "An Encoder-Decoder Model for ICD-10 Coding of Death Certificates",
            "authors": [
                "Elena Tutubalina",
                "Zulfat Miftahutdinov"
            ],
            "abstract": "Information extraction from textual documents such as hospital records and\nhealthrelated user discussions has become a topic of intense interest. The task\nof medical concept coding is to map a variable length text to medical concepts\nand corresponding classification codes in some external system or ontology. In\nthis work, we utilize recurrent neural networks to automatically assign ICD-10\ncodes to fragments of death certificates written in English. We develop\nend-to-end neural architectures directly tailored to the task, including basic\nencoder-decoder architecture for statistical translation. In order to\nincorporate prior knowledge, we concatenate cosine similarities vector among\nthe text and dictionary entry to the encoded state. Being applied to a standard\nbenchmark from CLEF eHealth 2017 challenge, our model achieved F-measure of\n85.01% on a full test set with significant improvement as compared to the\naverage score of 62.2% for all official participants approaches.",
            "journal ref": "KFU at CLEF eHealth 2017 Task 1: ICD-10 Coding of English Death\n  Certificates with Recurrent Neural Networks, CEUR Workshop Proceedings, Vol\n  1866, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01411v1",
            "date": "2017-12-04 23:27:11+00:00",
            "title": "#anorexia, #anarexia, #anarexyia: Characterizing Online Community Practices with Orthographic Variation",
            "authors": [
                "Ian Stewart",
                "Stevie Chancellor",
                "Munmun De Choudhury",
                "Jacob Eisenstein"
            ],
            "abstract": "Distinctive linguistic practices help communities build solidarity and\ndifferentiate themselves from outsiders. In an online community, one such\npractice is variation in orthography, which includes spelling, punctuation, and\ncapitalization. Using a dataset of over two million Instagram posts, we\ninvestigate orthographic variation in a community that shares pro-eating\ndisorder (pro-ED) content. We find that not only does orthographic variation\ngrow more frequent over time, it also becomes more profound or deep, with\nvariants becoming increasingly distant from the original: as, for example,\n#anarexyia is more distant than #anarexia from the original spelling #anorexia.\nThese changes are driven by newcomers, who adopt the most extreme linguistic\npractices as they enter the community. Moreover, this behavior correlates with\nengagement: the newcomers who adopt deeper orthographic variants tend to remain\nactive for longer in the community, and the posts that contain deeper variation\nreceive more positive feedback in the form of \"likes.\" Previous work has linked\ncommunity membership change with language change, and our work casts this\nconnection in a new light, with newcomers driving an evolving practice, rather\nthan adapting to it. We also demonstrate the utility of orthographic variation\nas a new lens to study sociolinguistic change in online communities,\nparticularly when the change results from an exogenous force such as a content\nban.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02047v1",
            "date": "2017-12-06 05:38:29+00:00",
            "title": "Distance-based Self-Attention Network for Natural Language Inference",
            "authors": [
                "Jinbae Im",
                "Sungzoon Cho"
            ],
            "abstract": "Attention mechanism has been used as an ancillary means to help RNN or CNN.\nHowever, the Transformer (Vaswani et al., 2017) recently recorded the\nstate-of-the-art performance in machine translation with a dramatic reduction\nin training time by solely using attention. Motivated by the Transformer,\nDirectional Self Attention Network (Shen et al., 2017), a fully attention-based\nsentence encoder, was proposed. It showed good performance with various data by\nusing forward and backward directional information in a sentence. But in their\nstudy, not considered at all was the distance between words, an important\nfeature when learning the local dependency to help understand the context of\ninput text. We propose Distance-based Self-Attention Network, which considers\nthe word distance by using a simple distance mask in order to model the local\ndependency without losing the ability of modeling global dependency which\nattention has inherent. Our model shows good performance with NLI data, and it\nrecords the new state-of-the-art result with SNLI data. Additionally, we show\nthat our model has a strength in long sentences or documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02223v1",
            "date": "2017-12-06 15:15:21+00:00",
            "title": "Discourse-Aware Rumour Stance Classification in Social Media Using Sequential Classifiers",
            "authors": [
                "Arkaitz Zubiaga",
                "Elena Kochkina",
                "Maria Liakata",
                "Rob Procter",
                "Michal Lukasik",
                "Kalina Bontcheva",
                "Trevor Cohn",
                "Isabelle Augenstein"
            ],
            "abstract": "Rumour stance classification, defined as classifying the stance of specific\nsocial media posts into one of supporting, denying, querying or commenting on\nan earlier post, is becoming of increasing interest to researchers. While most\nprevious work has focused on using individual tweets as classifier inputs, here\nwe report on the performance of sequential classifiers that exploit the\ndiscourse features inherent in social media interactions or 'conversational\nthreads'. Testing the effectiveness of four sequential classifiers -- Hawkes\nProcesses, Linear-Chain Conditional Random Fields (Linear CRF), Tree-Structured\nConditional Random Fields (Tree CRF) and Long Short Term Memory networks (LSTM)\n-- on eight datasets associated with breaking news stories, and looking at\ndifferent types of local and contextual features, our work sheds new light on\nthe development of accurate stance classifiers. We show that sequential\nclassifiers that exploit the use of discourse properties in social media\nconversations while using only local features, outperform non-sequential\nclassifiers. Furthermore, we show that LSTM using a reduced set of features can\noutperform the other sequential classifiers; this performance is consistent\nacross datasets and across types of stances. To conclude, our work also\nanalyses the different features under study, identifying those that best help\ncharacterise and distinguish between stances, such as supporting tweets being\nmore likely to be accompanied by evidence than denying tweets. We also set\nforth a number of directions for future research.",
            "journal ref": "Information Processing & Management, Volume 54, Issue 2, March\n  2018, Pages 273-290",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02250v1",
            "date": "2017-12-06 16:00:45+00:00",
            "title": "Why Do Neural Dialog Systems Generate Short and Meaningless Replies? A Comparison between Dialog and Translation",
            "authors": [
                "Bolin Wei",
                "Shuai Lu",
                "Lili Mou",
                "Hao Zhou",
                "Pascal Poupart",
                "Ge Li",
                "Zhi Jin"
            ],
            "abstract": "This paper addresses the question: Why do neural dialog systems generate\nshort and meaningless replies? We conjecture that, in a dialog system, an\nutterance may have multiple equally plausible replies, causing the deficiency\nof neural networks in the dialog application. We propose a systematic way to\nmimic the dialog scenario in a machine translation system, and manage to\nreproduce the phenomenon of generating short and less meaningful sentences in\nthe translation setting, showing evidence of our conjecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02259v1",
            "date": "2017-12-06 16:18:31+00:00",
            "title": "An innovative solution for breast cancer textual big data analysis",
            "authors": [
                "Nicolas Thiebaut",
                "Antoine Simoulin",
                "Karl Neuberger",
                "Issam Ibnouhsein",
                "Nicolas Bousquet",
                "Nathalie Reix",
                "S\u00e9bastien Moli\u00e8re",
                "Carole Mathelin"
            ],
            "abstract": "The digitalization of stored information in hospitals now allows for the\nexploitation of medical data in text format, as electronic health records\n(EHRs), initially gathered for other purposes than epidemiology. Manual search\nand analysis operations on such data become tedious. In recent years, the use\nof natural language processing (NLP) tools was highlighted to automatize the\nextraction of information contained in EHRs, structure it and perform\nstatistical analysis on this structured information. The main difficulties with\nthe existing approaches is the requirement of synonyms or ontology\ndictionaries, that are mostly available in English only and do not include\nlocal or custom notations. In this work, a team composed of oncologists as\ndomain experts and data scientists develop a custom NLP-based system to process\nand structure textual clinical reports of patients suffering from breast\ncancer. The tool relies on the combination of standard text mining techniques\nand an advanced synonym detection method. It allows for a global analysis by\nretrieval of indicators such as medical history, tumor characteristics,\ntherapeutic responses, recurrences and prognosis. The versatility of the method\nallows to obtain easily new indicators, thus opening up the way for\nretrospective studies with a substantial reduction of the amount of manual\nwork. With no need for biomedical annotators or pre-defined ontologies, this\nlanguage-agnostic method reached an good extraction accuracy for several\nconcepts of interest, according to a comparison with a manually structured\nfile, without requiring any existing corpus with local or new notations.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1712.02767v1",
            "date": "2017-12-04 06:05:21+00:00",
            "title": "Topics and Label Propagation: Best of Both Worlds for Weakly Supervised Text Classification",
            "authors": [
                "Sachin Pawar",
                "Nitin Ramrakhiyani",
                "Swapnil Hingmire",
                "Girish K. Palshikar"
            ],
            "abstract": "We propose a Label Propagation based algorithm for weakly supervised text\nclassification. We construct a graph where each document is represented by a\nnode and edge weights represent similarities among the documents. Additionally,\nwe discover underlying topics using Latent Dirichlet Allocation (LDA) and\nenrich the document graph by including the topics in the form of additional\nnodes. The edge weights between a topic and a text document represent level of\n\"affinity\" between them. Our approach does not require document level\nlabelling, instead it expects manual labels only for topic nodes. This\nsignificantly minimizes the level of supervision needed as only a few topics\nare observed to be enough for achieving sufficiently high accuracy. The Label\nPropagation Algorithm is employed on this enriched graph to propagate labels\namong the nodes. Our approach combines the advantages of Label Propagation\n(through document-document similarities) and Topic Modelling (for minimal but\nsmart supervision). We demonstrate the effectiveness of our approach on various\ndatasets and compare with state-of-the-art weakly supervised text\nclassification approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.02896v1",
            "date": "2017-12-08 00:27:08+00:00",
            "title": "Audio-Visual Sentiment Analysis for Learning Emotional Arcs in Movies",
            "authors": [
                "Eric Chu",
                "Deb Roy"
            ],
            "abstract": "Stories can have tremendous power -- not only useful for entertainment, they\ncan activate our interests and mobilize our actions. The degree to which a\nstory resonates with its audience may be in part reflected in the emotional\njourney it takes the audience upon. In this paper, we use machine learning\nmethods to construct emotional arcs in movies, calculate families of arcs, and\ndemonstrate the ability for certain arcs to predict audience engagement. The\nsystem is applied to Hollywood films and high quality shorts found on the web.\nWe begin by using deep convolutional neural networks for audio and visual\nsentiment analysis. These models are trained on both new and existing\nlarge-scale datasets, after which they can be used to compute separate audio\nand visual emotional arcs. We then crowdsource annotations for 30-second video\nclips extracted from highs and lows in the arcs in order to assess the\nmicro-level precision of the system, with precision measured in terms of\nagreement in polarity between the system's predictions and annotators' ratings.\nThese annotations are also used to combine the audio and visual predictions.\nNext, we look at macro-level characterizations of movies by investigating\nwhether there exist `universal shapes' of emotional arcs. In particular, we\ndevelop a clustering approach to discover distinct classes of emotional arcs.\nFinally, we show on a sample corpus of short web videos that certain emotional\narcs are statistically significant predictors of the number of comments a video\nreceives. These results suggest that the emotional arcs learned by our approach\nsuccessfully represent macroscopic aspects of a video story that drive audience\nengagement. Such machine understanding could be used to predict audience\nreactions to video stories, ultimately improving our ability as storytellers to\ncommunicate with each other.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.03645v1",
            "date": "2017-12-11 04:48:43+00:00",
            "title": "Long-Range Correlation Underlying Childhood Language and Generative Models",
            "authors": [
                "Kumiko Tanaka-Ishii"
            ],
            "abstract": "Long-range correlation, a property of time series exhibiting long-term\nmemory, is mainly studied in the statistical physics domain and has been\nreported to exist in natural language. Using a state-of-the-art method for such\nanalysis, long-range correlation is first shown to occur in long CHILDES data\nsets. To understand why, Bayesian generative models of language, originally\nproposed in the cognitive scientific domain, are investigated. Among\nrepresentative models, the Simon model was found to exhibit surprisingly good\nlong-range correlation, but not the Pitman-Yor model. Since the Simon model is\nknown not to correctly reflect the vocabulary growth of natural language, a\nsimple new model is devised as a conjunct of the Simon and Pitman-Yor models,\nsuch that long-range correlation holds with a correct vocabulary growth rate.\nThe investigation overall suggests that uniform sampling is one cause of\nlong-range correlation and could thus have a relation with actual linguistic\nprocesses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03903v1",
            "date": "2017-12-11 17:24:13+00:00",
            "title": "A Novel Way of Identifying Cyber Predators",
            "authors": [
                "Dan Liu",
                "Ching Yee Suen",
                "Olga Ormandjieva"
            ],
            "abstract": "Recurrent Neural Networks with Long Short-Term Memory cell (LSTM-RNN) have\nimpressive ability in sequence data processing, particularly for language model\nbuilding and text classification. This research proposes the combination of\nsentiment analysis, new approach of sentence vectors and LSTM-RNN as a novel\nway for Sexual Predator Identification (SPI). LSTM-RNN language model is\napplied to generate sentence vectors which are the last hidden states in the\nlanguage model. Sentence vectors are fed into another LSTM-RNN classifier, so\nas to capture suspicious conversations. Hidden state enables to generate\nvectors for sentences never seen before. Fasttext is used to filter the\ncontents of conversations and generate a sentiment score so as to identify\npotential predators. The experiment achieves a record-breaking accuracy and\nprecision of 100% with recall of 81.10%, exceeding the top-ranked result in the\nSPI competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04034v1",
            "date": "2017-12-11 21:22:01+00:00",
            "title": "Learning Robust Dialog Policies in Noisy Environments",
            "authors": [
                "Maryam Fazel-Zarandi",
                "Shang-Wen Li",
                "Jin Cao",
                "Jared Casale",
                "Peter Henderson",
                "David Whitney",
                "Alborz Geramifard"
            ],
            "abstract": "Modern virtual personal assistants provide a convenient interface for\ncompleting daily tasks via voice commands. An important consideration for these\nassistants is the ability to recover from automatic speech recognition (ASR)\nand natural language understanding (NLU) errors. In this paper, we focus on\nlearning robust dialog policies to recover from these errors. To this end, we\ndevelop a user simulator which interacts with the assistant through voice\ncommands in realistic scenarios with noisy audio, and use it to learn dialog\npolicies through deep reinforcement learning. We show that dialogs generated by\nour simulator are indistinguishable from human generated dialogs, as determined\nby human evaluators. Furthermore, preliminary experimental results show that\nthe learned policies in noisy environments achieve the same execution success\nrate with fewer dialog turns compared to fixed rule-based policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04708v4",
            "date": "2017-12-13 11:17:37+00:00",
            "title": "Differentiable lower bound for expected BLEU score",
            "authors": [
                "Vlad Zhukov",
                "Eugene Golikov",
                "Maksim Kretov"
            ],
            "abstract": "In natural language processing tasks performance of the models is often\nmeasured with some non-differentiable metric, such as BLEU score. To use\nefficient gradient-based methods for optimization, it is a common workaround to\noptimize some surrogate loss function. This approach is effective if\noptimization of such loss also results in improving target metric. The\ncorresponding problem is referred to as loss-evaluation mismatch. In the\npresent work we propose a method for calculation of differentiable lower bound\nof expected BLEU score that does not involve computationally expensive sampling\nprocedure such as the one required when using REINFORCE rule from reinforcement\nlearning (RL) framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04787v2",
            "date": "2017-12-13 14:29:09+00:00",
            "title": "Creating New Language and Voice Components for the Updated MaryTTS Text-to-Speech Synthesis Platform",
            "authors": [
                "Ingmar Steiner",
                "S\u00e9bastien Le Maguer"
            ],
            "abstract": "We present a new workflow to create components for the MaryTTS text-to-speech\nsynthesis platform, which is popular with researchers and developers, extending\nit to support new languages and custom synthetic voices. This workflow replaces\nthe previous toolkit with an efficient, flexible process that leverages modern\nbuild automation and cloud-hosted infrastructure. Moreover, it is compatible\nwith the updated MaryTTS architecture, enabling new features and\nstate-of-the-art paradigms such as synthesis based on deep neural networks\n(DNNs). Like MaryTTS itself, the new tools are free, open source software\n(FOSS), and promote the use of open data.",
            "journal ref": "Proc. LREC 11 (2018) 3171-3175",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04798v3",
            "date": "2017-12-13 14:45:29+00:00",
            "title": "A Multimodal Corpus of Expert Gaze and Behavior during Phonetic Segmentation Tasks",
            "authors": [
                "Arif Khan",
                "Ingmar Steiner",
                "Yusuke Sugano",
                "Andreas Bulling",
                "Ross Macdonald"
            ],
            "abstract": "Phonetic segmentation is the process of splitting speech into distinct\nphonetic units. Human experts routinely perform this task manually by analyzing\nauditory and visual cues using analysis software, which is an extremely\ntime-consuming process. Methods exist for automatic segmentation, but these are\nnot always accurate enough. In order to improve automatic segmentation, we need\nto model it as close to the manual segmentation as possible. This corpus is an\neffort to capture the human segmentation behavior by recording experts\nperforming a segmentation task. We believe that this data will enable us to\nhighlight the important aspects of manual segmentation, which can be used in\nautomatic segmentation to improve its accuracy.",
            "journal ref": "Proc. LREC 11 (2018) 4277-4281",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1712.05382v2",
            "date": "2017-12-14 18:29:42+00:00",
            "title": "Monotonic Chunkwise Attention",
            "authors": [
                "Chung-Cheng Chiu",
                "Colin Raffel"
            ],
            "abstract": "Sequence-to-sequence models with soft attention have been successfully\napplied to a wide variety of problems, but their decoding process incurs a\nquadratic time and space cost and is inapplicable to real-time sequence\ntransduction. To address these issues, we propose Monotonic Chunkwise Attention\n(MoChA), which adaptively splits the input sequence into small chunks over\nwhich soft attention is computed. We show that models utilizing MoChA can be\ntrained efficiently with standard backpropagation while allowing online and\nlinear-time decoding at test time. When applied to online speech recognition,\nwe obtain state-of-the-art results and match the performance of a model using\nan offline soft attention mechanism. In document summarization experiments\nwhere we do not expect monotonic alignments, we show significantly improved\nperformance compared to a baseline monotonic attention-based model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06074v1",
            "date": "2017-12-17 08:53:13+00:00",
            "title": "Benford's Law and First Letter of Word",
            "authors": [
                "Xiaoyong Yan",
                "Seong-Gyu Yang",
                "Beom Jun Kim",
                "Petter Minnhagen"
            ],
            "abstract": "A universal First-Letter Law (FLL) is derived and described. It predicts the\npercentages of first letters for words in novels. The FLL is akin to Benford's\nlaw (BL) of first digits, which predicts the percentages of first digits in a\ndata collection of numbers. Both are universal in the sense that FLL only\ndepends on the numbers of letters in the alphabet, whereas BL only depends on\nthe number of digits in the base of the number system. The existence of these\ntypes of universal laws appears counter-intuitive. Nonetheless both describe\ndata very well. Relations to some earlier works are given. FLL predicts that an\nEnglish author on the average starts about 16 out of 100 words with the English\nletter `t'. This is corroborated by data, yet an author can freely write\nanything. Fuller implications and the applicability of FLL remain for the\nfuture.",
            "journal ref": "Physica A 512, 305-315 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06204v2",
            "date": "2017-12-17 23:11:28+00:00",
            "title": "Probabilistic Semantic Retrieval for Surveillance Videos with Activity Graphs",
            "authors": [
                "Yuting Chen",
                "Joseph Wang",
                "Yannan Bai",
                "Gregory Casta\u00f1\u00f3n",
                "Venkatesh Saligrama"
            ],
            "abstract": "We present a novel framework for finding complex activities matching\nuser-described queries in cluttered surveillance videos. The wide diversity of\nqueries coupled with unavailability of annotated activity data limits our\nability to train activity models. To bridge the semantic gap we propose to let\nusers describe an activity as a semantic graph with object attributes and\ninter-object relationships associated with nodes and edges, respectively. We\nlearn node/edge-level visual predictors during training and, at test-time,\npropose to retrieve activity by identifying likely locations that match the\nsemantic graph. We formulate a novel CRF based probabilistic activity\nlocalization objective that accounts for mis-detections, mis-classifications\nand track-losses, and outputs a likelihood score for a candidate grounded\nlocation of the query in the video. We seek groundings that maximize overall\nprecision and recall. To handle the combinatorial search over all\nhigh-probability groundings, we propose a highest precision subgraph matching\nalgorithm. Our method outperforms existing retrieval methods on benchmarked\ndatasets.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1712.06751v2",
            "date": "2017-12-19 02:15:19+00:00",
            "title": "HotFlip: White-Box Adversarial Examples for Text Classification",
            "authors": [
                "Javid Ebrahimi",
                "Anyi Rao",
                "Daniel Lowd",
                "Dejing Dou"
            ],
            "abstract": "We propose an efficient method to generate white-box adversarial examples to\ntrick a character-level neural classifier. We find that only a few\nmanipulations are needed to greatly decrease the accuracy. Our method relies on\nan atomic flip operation, which swaps one token for another, based on the\ngradients of the one-hot input vectors. Due to efficiency of our method, we can\nperform adversarial training which makes the model more robust to attacks at\ntest time. With the use of a few semantics-preserving constraints, we\ndemonstrate that HotFlip can be adapted to attack a word-level classifier as\nwell.",
            "journal ref": "ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07512v1",
            "date": "2017-12-20 15:03:04+00:00",
            "title": "Ethical Questions in NLP Research: The (Mis)-Use of Forensic Linguistics",
            "authors": [
                "Anil Kumar Singh",
                "Akhilesh Sudhakar"
            ],
            "abstract": "Ideas from forensic linguistics are now being used frequently in Natural\nLanguage Processing (NLP), using machine learning techniques. While the role of\nforensic linguistics was more benign earlier, it is now being used for purposes\nwhich are questionable. Certain methods from forensic linguistics are employed,\nwithout considering their scientific limitations and ethical concerns. While we\ntake the specific case of forensic linguistics as an example of such trends in\nNLP and machine learning, the issue is a larger one and present in many other\nscientific and data-driven domains. We suggest that such trends indicate that\nsome of the applied sciences are exceeding their legal and scientific briefs.\nWe highlight how carelessly implemented practices are serving to short-circuit\nthe due processes of law as well breach ethical codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07745v1",
            "date": "2017-12-20 23:10:21+00:00",
            "title": "Context-aware Path Ranking for Knowledge Base Completion",
            "authors": [
                "Sahisnu Mazumder",
                "Bing Liu"
            ],
            "abstract": "Knowledge base (KB) completion aims to infer missing facts from existing ones\nin a KB. Among various approaches, path ranking (PR) algorithms have received\nincreasing attention in recent years. PR algorithms enumerate paths between\nentity pairs in a KB and use those paths as features to train a model for\nmissing fact prediction. Due to their good performances and high model\ninterpretability, several methods have been proposed. However, most existing\nmethods suffer from scalability (high RAM consumption) and feature explosion\n(trains on an exponentially large number of features) problems. This paper\nproposes a Context-aware Path Ranking (C-PR) algorithm to solve these problems\nby introducing a selective path exploration strategy. C-PR learns global\nsemantics of entities in the KB using word embedding and leverages the\nknowledge of entity semantics to enumerate contextually relevant paths using\nbidirectional random walk. Experimental results on three large KBs show that\nthe path features (fewer in number) discovered by C-PR not only improve\npredictive performance but also are more interpretable than existing baselines.",
            "journal ref": "Published in IJCAI 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08349v2",
            "date": "2017-12-22 09:01:00+00:00",
            "title": "Tracking the Diffusion of Named Entities",
            "authors": [
                "Leon Derczynski",
                "Matthew Rowe"
            ],
            "abstract": "Existing studies of how information diffuses across social networks have thus\nfar concentrated on analysing and recovering the spread of deterministic\ninnovations such as URLs, hashtags, and group membership. However investigating\nhow mentions of real-world entities appear and spread has yet to be explored,\nlargely due to the computationally intractable nature of performing large-scale\nentity extraction. In this paper we present, to the best of our knowledge, one\nof the first pieces of work to closely examine the diffusion of named entities\non social media, using Reddit as our case study platform. We first investigate\nhow named entities can be accurately recognised and extracted from discussion\nposts. We then use these extracted entities to study the patterns of entity\ncascades and how the probability of a user adopting an entity (i.e. mentioning\nit) is associated with exposures to the entity. We put these pieces together by\npresenting a parallelised diffusion model that can forecast the probability of\nentity adoption, finding that the influence of adoption between users can be\ncharacterised by their prior interactions -- as opposed to whether the users\npropagated entity-adoptions beforehand. Our findings have important\nimplications for researchers studying influence and language, and for community\nanalysts who wish to understand entity-level influence dynamics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08636v1",
            "date": "2017-12-22 19:58:01+00:00",
            "title": "Find the Conversation Killers: a Predictive Study of Thread-ending Posts",
            "authors": [
                "Yunhao Jiao",
                "Cheng Li",
                "Fei Wu",
                "Qiaozhu Mei"
            ],
            "abstract": "How to improve the quality of conversations in online communities has\nattracted considerable attention recently. Having engaged, urbane, and reactive\nonline conversations has a critical effect on the social life of Internet\nusers. In this study, we are particularly interested in identifying a post in a\nmulti-party conversation that is unlikely to be further replied to, which\ntherefore kills that thread of the conversation. For this purpose, we propose a\ndeep learning model called the ConverNet. ConverNet is attractive due to its\ncapability of modeling the internal structure of a long conversation and its\nappropriate encoding of the contextual information of the conversation, through\neffective integration of attention mechanisms. Empirical experiments on\nreal-world datasets demonstrate the effectiveness of the proposal model. For\nthe widely concerned topic, our analysis also offers implications for improving\nthe quality and user experience of online conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08862v1",
            "date": "2017-12-24 00:27:09+00:00",
            "title": "Neural Network Multitask Learning for Traffic Flow Forecasting",
            "authors": [
                "Feng Jin",
                "Shiliang Sun"
            ],
            "abstract": "Traditional neural network approaches for traffic flow forecasting are\nusually single task learning (STL) models, which do not take advantage of the\ninformation provided by related tasks. In contrast to STL, multitask learning\n(MTL) has the potential to improve generalization by transferring information\nin training signals of extra tasks. In this paper, MTL based neural networks\nare used for traffic flow forecasting. For neural network MTL, a\nbackpropagation (BP) network is constructed by incorporating traffic flows at\nseveral contiguous time instants into an output layer. Nodes in the output\nlayer can be seen as outputs of different but closely related STL tasks.\nComprehensive experiments on urban vehicular traffic flow data and comparisons\nwith STL show that MTL in BP neural networks is a promising and effective\napproach for traffic flow forecasting.",
            "journal ref": "Proceedings of the International Joint Conference on Neural\n  Networks (IJCNN), 2008. pp. 1898-1902",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1712.08941v1",
            "date": "2017-12-24 16:16:07+00:00",
            "title": "Comparative Opinion Mining: A Review",
            "authors": [
                "Kasturi Dewi Varathan",
                "Anastasia Giachanou",
                "Fabio Crestani"
            ],
            "abstract": "Opinion mining refers to the use of natural language processing, text\nanalysis and computational linguistics to identify and extract subjective\ninformation in textual material. Opinion mining, also known as sentiment\nanalysis, has received a lot of attention in recent times, as it provides a\nnumber of tools to analyse the public opinion on a number of different topics.\nComparative opinion mining is a subfield of opinion mining that deals with\nidentifying and extracting information that is expressed in a comparative form\n(e.g.~\"paper X is better than the Y\"). Comparative opinion mining plays a very\nimportant role when ones tries to evaluate something, as it provides a\nreference point for the comparison. This paper provides a review of the area of\ncomparative opinion mining. It is the first review that cover specifically this\ntopic as all previous reviews dealt mostly with general opinion mining. This\nsurvey covers comparative opinion mining from two different angles. One from\nperspective of techniques and the other from perspective of comparative opinion\nelements. It also incorporates preprocessing tools as well as dataset that were\nused by the past researchers that can be useful to the future researchers in\nthe field of comparative opinion mining.",
            "journal ref": "Journal of the Association for Information Science and Technology,\n  68(4), 2017",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1712.09359v3",
            "date": "2017-12-26 18:43:32+00:00",
            "title": "Basic concepts and tools for the Toki Pona minimal and constructed language: description of the language and main issues; analysis of the vocabulary; text synthesis and syntax highlighting; Wordnet synsets",
            "authors": [
                "Renato Fabbri"
            ],
            "abstract": "A minimal constructed language (conlang) is useful for experiments and\ncomfortable for making tools. The Toki Pona (TP) conlang is minimal both in the\nvocabulary (with only 14 letters and 124 lemmas) and in the (about) 10 syntax\nrules. The language is useful for being a used and somewhat established minimal\nconlang with at least hundreds of fluent speakers. This article exposes current\nconcepts and resources for TP, and makes available Python (and Vim) scripted\nroutines for the analysis of the language, synthesis of texts, syntax\nhighlighting schemes, and the achievement of a preliminary TP Wordnet. Focus is\non the analysis of the basic vocabulary, as corpus analyses were found. The\nsynthesis is based on sentence templates, relates to context by keeping track\nof used words, and renders larger texts by using a fixed number of phonemes\n(e.g. for poems) and number of sentences, words and letters (e.g. for\nparagraphs). Syntax highlighting reflects morphosyntactic classes given in the\nofficial dictionary and different solutions are described and implemented in\nthe well-established Vim text editor. The tentative TP Wordnet is made\navailable in three patterns of relations between synsets and word lemmas. In\nsummary, this text holds potentially novel conceptualizations about, and tools\nand results in analyzing, synthesizing and syntax highlighting the TP language.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1712.09444v2",
            "date": "2017-12-22 17:42:15+00:00",
            "title": "Letter-Based Speech Recognition with Gated ConvNets",
            "authors": [
                "Vitaliy Liptchinsky",
                "Gabriel Synnaeve",
                "Ronan Collobert"
            ],
            "abstract": "In the recent literature, \"end-to-end\" speech systems often refer to\nletter-based acoustic models trained in a sequence-to-sequence manner, either\nvia a recurrent model or via a structured output learning approach (such as\nCTC). In contrast to traditional phone (or senone)-based approaches, these\n\"end-to-end'' approaches alleviate the need of word pronunciation modeling, and\ndo not require a \"forced alignment\" step at training time. Phone-based\napproaches remain however state of the art on classical benchmarks. In this\npaper, we propose a letter-based speech recognition system, leveraging a\nConvNet acoustic model. Key ingredients of the ConvNet are Gated Linear Units\nand high dropout. The ConvNet is trained to map audio sequences to their\ncorresponding letter transcriptions, either via a classical CTC approach, or\nvia a recent variant called ASG. Coupled with a simple decoder at inference\ntime, our system matches the best existing letter-based systems on WSJ (in word\nerror rate), and shows near state of the art performance on LibriSpeech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09783v3",
            "date": "2017-12-28 08:05:48+00:00",
            "title": "Topic Compositional Neural Language Model",
            "authors": [
                "Wenlin Wang",
                "Zhe Gan",
                "Wenqi Wang",
                "Dinghan Shen",
                "Jiaji Huang",
                "Wei Ping",
                "Sanjeev Satheesh",
                "Lawrence Carin"
            ],
            "abstract": "We propose a Topic Compositional Neural Language Model (TCNLM), a novel\nmethod designed to simultaneously capture both the global semantic meaning and\nthe local word ordering structure in a document. The TCNLM learns the global\nsemantic coherence of a document via a neural topic model, and the probability\nof each learned latent topic is further used to build a Mixture-of-Experts\n(MoE) language model, where each expert (corresponding to one topic) is a\nrecurrent neural network (RNN) that accounts for learning the local structure\nof a word sequence. In order to train the MoE model efficiently, a matrix\nfactorization method is applied, by extending each weight matrix of the RNN to\nbe an ensemble of topic-dependent weight matrices. The degree to which each\nmember of the ensemble is used is tied to the document-dependent probability of\nthe corresponding topics. Experimental results on several corpora show that the\nproposed approach outperforms both a pure RNN-based model and other\ntopic-guided language models. Further, our model yields sensible topics, and\nalso has the capacity to generate meaningful sentences conditioned on given\ntopics.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1712.09929v1",
            "date": "2017-12-28 16:49:40+00:00",
            "title": "On the Challenges of Detecting Rude Conversational Behaviour",
            "authors": [
                "Karan Grewal",
                "Khai N. Truong"
            ],
            "abstract": "In this study, we aim to identify moments of rudeness between two\nindividuals. In particular, we segment all occurrences of rudeness in\nconversations into three broad, distinct categories and try to identify each.\nWe show how machine learning algorithms can be used to identify rudeness based\non acoustic and semantic signals extracted from conversations. Furthermore, we\nmake note of our shortcomings in this task and highlight what makes this\nproblem inherently difficult. Finally, we provide next steps which are needed\nto ensure further success in identifying rudeness in conversations.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1712.10054v1",
            "date": "2017-12-28 20:56:16+00:00",
            "title": "Corpus specificity in LSA and Word2vec: the role of out-of-domain documents",
            "authors": [
                "Edgar Altszyler",
                "Mariano Sigman",
                "Diego Fernandez Slezak"
            ],
            "abstract": "Latent Semantic Analysis (LSA) and Word2vec are some of the most widely used\nword embeddings. Despite the popularity of these techniques, the precise\nmechanisms by which they acquire new semantic relations between words remain\nunclear. In the present article we investigate whether LSA and Word2vec\ncapacity to identify relevant semantic dimensions increases with size of\ncorpus. One intuitive hypothesis is that the capacity to identify relevant\ndimensions should increase as the amount of data increases. However, if corpus\nsize grow in topics which are not specific to the domain of interest, signal to\nnoise ratio may weaken. Here we set to examine and distinguish these\nalternative hypothesis. To investigate the effect of corpus specificity and\nsize in word-embeddings we study two ways for progressive elimination of\ndocuments: the elimination of random documents vs. the elimination of documents\nunrelated to a specific task. We show that Word2vec can take advantage of all\nthe documents, obtaining its best performance when it is trained with the whole\ncorpus. On the contrary, the specialization (removal of out-of-domain\ndocuments) of the training corpus, accompanied by a decrease of dimensionality,\ncan increase LSA word-representation quality while speeding up the processing\ntime. Furthermore, we show that the specialization without the decrease in LSA\ndimensionality can produce a strong performance reduction in specific tasks.\nFrom a cognitive-modeling point of view, we point out that LSA's word-knowledge\nacquisitions may not be efficiently exploiting higher-order co-occurrences and\nglobal relations, whereas Word2vec does.",
            "journal ref": "Proceedings of the 3rd Workshop on Representation Learning for\n  NLP, pages 1-10, 2018, ACL",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.10190v2",
            "date": "2017-12-29 11:46:13+00:00",
            "title": "Detecting Cross-Lingual Plagiarism Using Simulated Word Embeddings",
            "authors": [
                "Victor Thompson"
            ],
            "abstract": "Cross-lingual plagiarism (CLP) occurs when texts written in one language are\ntranslated into a different language and used without acknowledging the\noriginal sources. One of the most common methods for detecting CLP requires\nonline machine translators (such as Google or Microsoft translate) which are\nnot always available, and given that plagiarism detection typically involves\nlarge document comparison, the amount of translations required would overwhelm\nan online machine translator, especially when detecting plagiarism over the\nweb. In addition, when translated texts are replaced with their synonyms, using\nonline machine translators to detect CLP would result in poor performance. This\npaper addresses the problem of cross-lingual plagiarism detection (CLPD) by\nproposing a model that uses simulated word embeddings to reproduce the\npredictions of an online machine translator (Google translate) when detecting\nCLP. The simulated embeddings comprise of translated words in different\nlanguages mapped in a common space, and replicated to increase the prediction\nprobability of retrieving the translations of a word (and their synonyms) from\nthe model. Unlike most existing models, the proposed model does not require\nparallel corpora, and accommodates multiple languages (multi-lingual). We\ndemonstrated the effectiveness of the proposed model in detecting CLP in\nstandard datasets that contain CLP cases, and evaluated its performance against\na state-of-the-art baseline that relies on online machine translator (T+MA\nmodel). Evaluation results revealed that the proposed model is not only\neffective in detecting CLP, it outperformed the baseline. The results indicate\nthat CLP could be detected with state-of-the-art performances by leveraging the\nprediction accuracy of an internet translator with word embeddings, without\nrelying on internet translators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.10309v1",
            "date": "2017-12-29 18:53:12+00:00",
            "title": "Methods for Detecting Paraphrase Plagiarism",
            "authors": [
                "Victor Thompson"
            ],
            "abstract": "Paraphrase plagiarism is one of the difficult challenges facing plagiarism\ndetection systems. Paraphrasing occur when texts are lexically or syntactically\naltered to look different, but retain their original meaning. Most plagiarism\ndetection systems (many of which are commercial based) are designed to detect\nword co-occurrences and light modifications, but are unable to detect severe\nsemantic and structural alterations such as what is seen in many academic\ndocuments. Hence many paraphrase plagiarism cases go undetected. In this paper,\nwe approached the problem of paraphrase plagiarism by proposing methods for\ndetecting the most common techniques (phenomena) used in paraphrasing texts\n(namely; lexical substitution, insertion/deletion and word and phrase\nreordering), and combined the methods into a paraphrase detection model. We\nevaluated our proposed methods and model on collections containing paraphrase\ntexts. Experimental results show significant improvement in performance when\nthe methods were combined (the proposed model) as opposed to running them\nindividually. The results also show that the proposed paraphrase detection\nmodel outperformed a standard baseline (based on greedy string tilling), and\nprevious studies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1702.00210v1",
            "date": "2017-02-01 11:18:47+00:00",
            "title": "Foreign-language Reviews: Help or Hindrance?",
            "authors": [
                "Scott A. Hale",
                "Irene Eleta"
            ],
            "abstract": "The number and quality of user reviews greatly affects consumer purchasing\ndecisions. While reviews in all languages are increasing, it is still often the\ncase (especially for non-English speakers) that there are only a few reviews in\na person's first language. Using an online experiment, we examine the value\nthat potential purchasers receive from interfaces showing additional reviews in\na second language. The results paint a complicated picture with both positive\nand negative reactions to the inclusion of foreign-language reviews. Roughly\n26-28% of subjects clicked to see translations of the foreign-language content\nwhen given the opportunity, and those who did so were more likely to select the\nproduct with foreign-language reviews than those who did not.",
            "journal ref": "Proceedings of the SIGCHI Conference on Human Factors in Computing\n  Systems, CHI 2017",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1702.00523v1",
            "date": "2017-02-02 01:56:22+00:00",
            "title": "Deep Learning the Indus Script",
            "authors": [
                "Satish Palaniappan",
                "Ronojoy Adhikari"
            ],
            "abstract": "Standardized corpora of undeciphered scripts, a necessary starting point for\ncomputational epigraphy, requires laborious human effort for their preparation\nfrom raw archaeological records. Automating this process through machine\nlearning algorithms can be of significant aid to epigraphical research. Here,\nwe take the first steps in this direction and present a deep learning pipeline\nthat takes as input images of the undeciphered Indus script, as found in\narchaeological artifacts, and returns as output a string of graphemes, suitable\nfor inclusion in a standard corpus. The image is first decomposed into regions\nusing Selective Search and these regions are classified as containing textual\nand/or graphical information using a convolutional neural network. Regions\nclassified as potentially containing text are hierarchically merged and trimmed\nto remove non-textual information. The remaining textual part of the image is\nsegmented using standard image processing techniques to isolate individual\ngraphemes. This set is finally passed to a second convolutional neural network\nto classify the graphemes, based on a standard corpus. The classifier can\nidentify the presence or absence of the most frequent Indus grapheme, the \"jar\"\nsign, with an accuracy of 92%. Our results demonstrate the great potential of\ndeep learning approaches in computational epigraphy and, more generally, in the\ndigital humanities.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1702.00564v2",
            "date": "2017-02-02 07:48:58+00:00",
            "title": "Modelling dependency completion in sentence comprehension as a Bayesian hierarchical mixture process: A case study involving Chinese relative clauses",
            "authors": [
                "Shravan Vasishth",
                "Nicolas Chopin",
                "Robin Ryder",
                "Bruno Nicenboim"
            ],
            "abstract": "We present a case-study demonstrating the usefulness of Bayesian hierarchical\nmixture modelling for investigating cognitive processes. In sentence\ncomprehension, it is widely assumed that the distance between linguistic\nco-dependents affects the latency of dependency resolution: the longer the\ndistance, the longer the retrieval time (the distance-based account). An\nalternative theory, direct-access, assumes that retrieval times are a mixture\nof two distributions: one distribution represents successful retrievals (these\nare independent of dependency distance) and the other represents an initial\nfailure to retrieve the correct dependent, followed by a reanalysis that leads\nto successful retrieval. We implement both models as Bayesian hierarchical\nmodels and show that the direct-access model explains Chinese relative clause\nreading time data better than the distance account.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1702.00887v3",
            "date": "2017-02-03 01:40:45+00:00",
            "title": "Structured Attention Networks",
            "authors": [
                "Yoon Kim",
                "Carl Denton",
                "Luong Hoang",
                "Alexander M. Rush"
            ],
            "abstract": "Attention networks have proven to be an effective approach for embedding\ncategorical inference within a deep neural network. However, for many tasks we\nmay want to model richer structural dependencies without abandoning end-to-end\ntraining. In this work, we experiment with incorporating richer structural\ndistributions, encoded using graphical models, within deep networks. We show\nthat these structured attention networks are simple extensions of the basic\nattention procedure, and that they allow for extending attention beyond the\nstandard soft-selection approach, such as attending to partial segmentations or\nto subtrees. We experiment with two different classes of structured attention\nnetworks: a linear-chain conditional random field and a graph-based parsing\nmodel, and describe how these models can be practically implemented as neural\nnetwork layers. Experiments show that this approach is effective for\nincorporating structural biases, and structured attention networks outperform\nbaseline attention models on a variety of synthetic and real tasks: tree\ntransduction, neural machine translation, question answering, and natural\nlanguage inference. We further find that models trained in this way learn\ninteresting unsupervised hidden representations that generalize simple\nattention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01090v2",
            "date": "2017-02-03 17:36:19+00:00",
            "title": "Multi-level computational methods for interdisciplinary research in the HathiTrust Digital Library",
            "authors": [
                "Jaimie Murdock",
                "Colin Allen",
                "Katy B\u00f6rner",
                "Robert Light",
                "Simon McAlister",
                "Andrew Ravenscroft",
                "Robert Rose",
                "Doori Rose",
                "Jun Otsuka",
                "David Bourget",
                "John Lawrence",
                "Chris Reed"
            ],
            "abstract": "We show how faceted search using a combination of traditional classification\nsystems and mixed-membership topic models can go beyond keyword search to\ninform resource discovery, hypothesis formulation, and argument extraction for\ninterdisciplinary research. Our test domain is the history and philosophy of\nscientific work on animal mind and cognition. The methods can be generalized to\nother research areas and ultimately support a system for semi-automatic\nidentification of argument structures. We provide a case study for the\napplication of the methods to the problem of identifying and extracting\narguments about anthropomorphism during a critical period in the development of\ncomparative psychology. We show how a combination of classification systems and\nmixed-membership models trained over large digital libraries can inform\nresource discovery in this domain. Through a novel approach of \"drill-down\"\ntopic modeling---simultaneously reducing both the size of the corpus and the\nunit of analysis---we are able to reduce a large collection of fulltext volumes\nto a much smaller set of pages within six focal volumes containing arguments of\ninterest to historians and philosophers of comparative psychology. The volumes\nidentified in this way did not appear among the first ten results of the\nkeyword search in the HathiTrust digital library and the pages bear the kind of\n\"close reading\" needed to generate original interpretations that is the heart\nof scholarly work in the humanities. Zooming back out, we provide a way to\nplace the books onto a map of science originally constructed from very\ndifferent data and for different purposes. The multilevel approach advances\nunderstanding of the intellectual and societal contexts in which writings are\ninterpreted.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1702.01991v3",
            "date": "2017-02-07 13:02:09+00:00",
            "title": "Representations of language in a model of visually grounded speech signal",
            "authors": [
                "Grzegorz Chrupa\u0142a",
                "Lieke Gelderloos",
                "Afra Alishahi"
            ],
            "abstract": "We present a visually grounded model of speech perception which projects\nspoken utterances and images to a joint semantic space. We use a multi-layer\nrecurrent highway network to model the temporal nature of spoken speech, and\nshow that it learns to extract both form and meaning-based linguistic knowledge\nfrom the input signal. We carry out an in-depth analysis of the representations\nused by different components of the trained model and show that encoding of\nsemantic aspects tends to become richer as we go up the hierarchy of layers,\nwhereas encoding of form-related aspects of the language input tends to\ninitially increase and then plateau or decrease.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02092v1",
            "date": "2017-01-23 11:18:06+00:00",
            "title": "Characterisation of speech diversity using self-organising maps",
            "authors": [
                "Tom A. F. Anderson",
                "David M. W. Powers"
            ],
            "abstract": "We report investigations into speaker classification of larger quantities of\nunlabelled speech data using small sets of manually phonemically annotated\nspeech. The Kohonen speech typewriter is a semi-supervised method comprised of\nself-organising maps (SOMs) that achieves low phoneme error rates. A SOM is a\n2D array of cells that learn vector representations of the data based on\nneighbourhoods. In this paper, we report a method to evaluate pronunciation\nusing multilevel SOMs with /hVd/ single syllable utterances for the study of\nvowels, for Australian pronunciation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.02287v4",
            "date": "2017-02-08 04:54:09+00:00",
            "title": "Name Disambiguation in Anonymized Graphs using Network Embedding",
            "authors": [
                "Baichuan Zhang",
                "Mohammad Al Hasan"
            ],
            "abstract": "In real-world, our DNA is unique but many people share names. This phenomenon\noften causes erroneous aggregation of documents of multiple persons who are\nnamesake of one another. Such mistakes deteriorate the performance of document\nretrieval, web search, and more seriously, cause improper attribution of credit\nor blame in digital forensic. To resolve this issue, the name disambiguation\ntask is designed which aims to partition the documents associated with a name\nreference such that each partition contains documents pertaining to a unique\nreal-life person. Existing solutions to this task substantially rely on feature\nengineering, such as biographical feature extraction, or construction of\nauxiliary features from Wikipedia. However, for many scenarios, such features\nmay be costly to obtain or unavailable due to the risk of privacy violation. In\nthis work, we propose a novel name disambiguation method. Our proposed method\nis non-intrusive of privacy because instead of using attributes pertaining to a\nreal-life person, our method leverages only relational data in the form of\nanonymized graphs. In the methodological aspect, the proposed method uses a\nnovel representation learning model to embed each document in a low dimensional\nvector space where name disambiguation can be solved by a hierarchical\nagglomerative clustering algorithm. Our experimental results demonstrate that\nthe proposed method is significantly better than the existing name\ndisambiguation methods working in a similar setting.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1702.02540v2",
            "date": "2017-02-08 17:46:37+00:00",
            "title": "Automatic Rule Extraction from Long Short Term Memory Networks",
            "authors": [
                "W. James Murdoch",
                "Arthur Szlam"
            ],
            "abstract": "Although deep learning models have proven effective at solving problems in\nnatural language processing, the mechanism by which they come to their\nconclusions is often unclear. As a result, these models are generally treated\nas black boxes, yielding no insight of the underlying learned patterns. In this\npaper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new\napproach for tracking the importance of a given input to the LSTM for a given\noutput. By identifying consistently important patterns of words, we are able to\ndistill state of the art LSTMs on sentiment analysis and question answering\ninto a set of representative phrases. This representation is then\nquantitatively validated by using the extracted phrases to construct a simple,\nrule-based classifier which approximates the output of the LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03121v1",
            "date": "2017-02-10 10:31:57+00:00",
            "title": "Modeling Semantic Expectation: Using Script Knowledge for Referent Prediction",
            "authors": [
                "Ashutosh Modi",
                "Ivan Titov",
                "Vera Demberg",
                "Asad Sayeed",
                "Manfred Pinkal"
            ],
            "abstract": "Recent research in psycholinguistics has provided increasing evidence that\nhumans predict upcoming content. Prediction also affects perception and might\nbe a key to robustness in human language processing. In this paper, we\ninvestigate the factors that affect human prediction by building a\ncomputational model that can predict upcoming discourse referents based on\nlinguistic knowledge alone vs. linguistic knowledge jointly with common-sense\nknowledge in the form of scripts. We find that script knowledge significantly\nimproves model estimates of human predictions. In a second study, we test the\nhighly controversial hypothesis that predictability influences referring\nexpression type but do not find evidence for such an effect.",
            "journal ref": "Transactions of ACL, Volume-5, Pg 31-44 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03197v1",
            "date": "2017-02-10 14:59:14+00:00",
            "title": "Arabic Language Sentiment Analysis on Health Services",
            "authors": [
                "Abdulaziz M. Alayba",
                "Vasile Palade",
                "Matthew England",
                "Rahat Iqbal"
            ],
            "abstract": "The social media network phenomenon leads to a massive amount of valuable\ndata that is available online and easy to access. Many users share images,\nvideos, comments, reviews, news and opinions on different social networks\nsites, with Twitter being one of the most popular ones. Data collected from\nTwitter is highly unstructured, and extracting useful information from tweets\nis a challenging task. Twitter has a huge number of Arabic users who mostly\npost and write their tweets using the Arabic language. While there has been a\nlot of research on sentiment analysis in English, the amount of researches and\ndatasets in Arabic language is limited. This paper introduces an Arabic\nlanguage dataset which is about opinions on health services and has been\ncollected from Twitter. The paper will first detail the process of collecting\nthe data from Twitter and also the process of filtering, pre-processing and\nannotating the Arabic text in order to build a big sentiment analysis dataset\nin Arabic. Several Machine Learning algorithms (Naive Bayes, Support Vector\nMachine and Logistic Regression) alongside Deep and Convolutional Neural\nNetworks were utilized in our experiments of sentiment analysis on our health\ndataset.",
            "journal ref": "Proc. 1st International Workshop on Arabic Script Analysis and\n  Recognition (ASAR '17), pp. 114-118. IEEE, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.03859v1",
            "date": "2017-02-13 16:31:06+00:00",
            "title": "Offline bilingual word vectors, orthogonal transformations and the inverted softmax",
            "authors": [
                "Samuel L. Smith",
                "David H. P. Turban",
                "Steven Hamblin",
                "Nils Y. Hammerla"
            ],
            "abstract": "Usually bilingual word vectors are trained \"online\". Mikolov et al. showed\nthey can also be found \"offline\", whereby two pre-trained embeddings are\naligned with a linear transformation, using dictionaries compiled from expert\nknowledge. In this work, we prove that the linear transformation between two\nspaces should be orthogonal. This transformation can be obtained using the\nsingular value decomposition. We introduce a novel \"inverted softmax\" for\nidentifying translation pairs, with which we improve the precision @1 of\nMikolov's original mapping from 34% to 43%, when translating a test set\ncomposed of both common and rare English words into Italian. Orthogonal\ntransformations are more robust to noise, enabling us to learn the\ntransformation without expert bilingual signal by constructing a\n\"pseudo-dictionary\" from the identical character strings which appear in both\nlanguages, achieving 40% precision on the same test set. Finally, we extend our\nmethod to retrieve the true translations of English sentences from a corpus of\n200k Italian sentences with a precision @1 of 68%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04521v1",
            "date": "2017-02-15 09:45:23+00:00",
            "title": "Frustratingly Short Attention Spans in Neural Language Modeling",
            "authors": [
                "Micha\u0142 Daniluk",
                "Tim Rockt\u00e4schel",
                "Johannes Welbl",
                "Sebastian Riedel"
            ],
            "abstract": "Neural language models predict the next token using a latent representation\nof the immediate token history. Recently, various methods for augmenting neural\nlanguage models with an attention mechanism over a differentiable memory have\nbeen proposed. For predicting the next token, these models query information\nfrom a memory of the recent history which can facilitate learning mid- and\nlong-range dependencies. However, conventional attention mechanisms used in\nmemory-augmented neural language models produce a single output vector per time\nstep. This vector is used both for predicting the next token as well as for the\nkey and value of a differentiable memory of a token history. In this paper, we\npropose a neural language model with a key-value attention mechanism that\noutputs separate representations for the key and value of a differentiable\nmemory, as well as for encoding the next-word distribution. This model\noutperforms existing memory-augmented neural language models on two corpora.\nYet, we found that our method mainly utilizes a memory of the five most recent\noutput representations. This led to the unexpected main finding that a much\nsimpler model based only on the concatenation of recent output representations\nfrom previous time steps is on par with more sophisticated memory-augmented\nneural language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.04770v1",
            "date": "2017-02-15 20:56:30+00:00",
            "title": "Training Language Models Using Target-Propagation",
            "authors": [
                "Sam Wiseman",
                "Sumit Chopra",
                "Marc'Aurelio Ranzato",
                "Arthur Szlam",
                "Ruoyu Sun",
                "Soumith Chintala",
                "Nicolas Vasilache"
            ],
            "abstract": "While Truncated Back-Propagation through Time (BPTT) is the most popular\napproach to training Recurrent Neural Networks (RNNs), it suffers from being\ninherently sequential (making parallelization difficult) and from truncating\ngradient flow between distant time-steps. We investigate whether Target\nPropagation (TPROP) style approaches can address these shortcomings.\nUnfortunately, extensive experiments suggest that TPROP generally underperforms\nBPTT, and we end with an analysis of this phenomenon, and suggestions for\nfuture work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.05270v1",
            "date": "2017-02-17 09:26:10+00:00",
            "title": "Be Precise or Fuzzy: Learning the Meaning of Cardinals and Quantifiers from Vision",
            "authors": [
                "Sandro Pezzelle",
                "Marco Marelli",
                "Raffaella Bernardi"
            ],
            "abstract": "People can refer to quantities in a visual scene by using either exact\ncardinals (e.g. one, two, three) or natural language quantifiers (e.g. few,\nmost, all). In humans, these two processes underlie fairly different cognitive\nand neural mechanisms. Inspired by this evidence, the present study proposes\ntwo models for learning the objective meaning of cardinals and quantifiers from\nvisual scenes containing multiple objects. We show that a model capitalizing on\na 'fuzzy' measure of similarity is effective for learning quantifiers, whereas\nthe learning of exact cardinals is better accomplished when information about\nnumber is provided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06216v2",
            "date": "2017-02-20 23:48:39+00:00",
            "title": "Filtering Tweets for Social Unrest",
            "authors": [
                "Alan Mishler",
                "Kevin Wonus",
                "Wendy Chambers",
                "Michael Bloodgood"
            ],
            "abstract": "Since the events of the Arab Spring, there has been increased interest in\nusing social media to anticipate social unrest. While efforts have been made\ntoward automated unrest prediction, we focus on filtering the vast volume of\ntweets to identify tweets relevant to unrest, which can be provided to\ndownstream users for further analysis. We train a supervised classifier that is\nable to label Arabic language tweets as relevant to unrest with high\nreliability. We examine the relationship between training data size and\nperformance and investigate ways to optimize the model building process while\nminimizing cost. We also explore how confidence thresholds can be set to\nachieve desired levels of performance.",
            "journal ref": "In Proceedings of the 2017 IEEE 11th International Conference on\n  Semantic Computing (ICSC), pages 17-23, San Diego, CA, USA, January 2017.\n  IEEE",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06467v1",
            "date": "2017-02-21 16:26:54+00:00",
            "title": "Efficient Social Network Multilingual Classification using Character, POS n-grams and Dynamic Normalization",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Juan-Manuel Torres-Moreno",
                "Azucena Montes Rend\u00f3n",
                "Gerardo Sierra"
            ],
            "abstract": "In this paper we describe a dynamic normalization process applied to social\nnetwork multilingual documents (Facebook and Twitter) to improve the\nperformance of the Author profiling task for short texts. After the\nnormalization process, $n$-grams of characters and n-grams of POS tags are\nobtained to extract all the possible stylistic information encoded in the\ndocuments (emoticons, character flooding, capital letters, references to other\nusers, hyperlinks, hashtags, etc.). Experiments with SVM showed up to 90% of\nperformance.",
            "journal ref": "Proceedings of the 8th International Joint Conference on Knowledge\n  Discovery, Knowledge Engineering and Knowledge Management, Vol 1: KDIR,\n  307-314, 2016, Porto, Portugal",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1702.06677v1",
            "date": "2017-02-22 04:54:43+00:00",
            "title": "Discussion quality diffuses in the digital public square",
            "authors": [
                "George Berry",
                "Sean J. Taylor"
            ],
            "abstract": "Studies of online social influence have demonstrated that friends have\nimportant effects on many types of behavior in a wide variety of settings.\nHowever, we know much less about how influence works among relative strangers\nin digital public squares, despite important conversations happening in such\nspaces. We present the results of a study on large public Facebook pages where\nwe randomly used two different methods--most recent and social feedback--to\norder comments on posts. We find that the social feedback condition results in\nhigher quality viewed comments and response comments. After measuring the\naverage quality of comments written by users before the study, we find that\nsocial feedback has a positive effect on response quality for both low and high\nquality commenters. We draw on a theoretical framework of social norms to\nexplain this empirical result. In order to examine the influence mechanism\nfurther, we measure the similarity between comments viewed and written during\nthe study, finding that similarity increases for the highest quality\ncontributors under the social feedback condition. This suggests that, in\naddition to norms, some individuals may respond with increased relevance to\nhigh-quality comments.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1702.06700v1",
            "date": "2017-02-22 08:19:38+00:00",
            "title": "Task-driven Visual Saliency and Attention-based Visual Question Answering",
            "authors": [
                "Yuetan Lin",
                "Zhangyang Pang",
                "Donghui Wang",
                "Yueting Zhuang"
            ],
            "abstract": "Visual question answering (VQA) has witnessed great progress since May, 2015\nas a classic problem unifying visual and textual data into a system. Many\nenlightening VQA works explore deep into the image and question encodings and\nfusing methods, of which attention is the most effective and infusive\nmechanism. Current attention based methods focus on adequate fusion of visual\nand textual features, but lack the attention to where people focus to ask\nquestions about the image. Traditional attention based methods attach a single\nvalue to the feature at each spatial location, which losses many useful\ninformation. To remedy these problems, we propose a general method to perform\nsaliency-like pre-selection on overlapped region features by the interrelation\nof bidirectional LSTM (BiLSTM), and use a novel element-wise multiplication\nbased attention method to capture more competent correlation information\nbetween visual and textual features. We conduct experiments on the large-scale\nCOCO-VQA dataset and analyze the effectiveness of our model demonstrated by\nstrong empirical results.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1702.06777v1",
            "date": "2017-02-22 12:42:06+00:00",
            "title": "Dialectometric analysis of language variation in Twitter",
            "authors": [
                "Gonzalo Donoso",
                "David Sanchez"
            ],
            "abstract": "In the last few years, microblogging platforms such as Twitter have given\nrise to a deluge of textual data that can be used for the analysis of informal\ncommunication between millions of individuals. In this work, we propose an\ninformation-theoretic approach to geographic language variation using a corpus\nbased on Twitter. We test our models with tens of concepts and their associated\nkeywords detected in Spanish tweets geolocated in Spain. We employ\ndialectometric measures (cosine similarity and Jensen-Shannon divergence) to\nquantify the linguistic distance on the lexical level between cells created in\na uniform grid over the map. This can be done for a single concept or in the\ngeneral case taking into account an average of the considered variants. The\nlatter permits an analysis of the dialects that naturally emerge from the data.\nInterestingly, our results reveal the existence of two dialect macrovarieties.\nThe first group includes a region-specific speech spoken in small towns and\nrural areas whereas the second cluster encompasses cities that tend to use a\nmore uniform variety. Since the results obtained with the two different metrics\nqualitatively agree, our work suggests that social media corpora can be\nefficiently used for dialectometric analyses.",
            "journal ref": "Proceedings of the Fourth Workshop on NLP for Similar Languages,\n  Varieties and Dialects (VarDial), pp. 16-25, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.06875v1",
            "date": "2017-02-22 16:14:12+00:00",
            "title": "Triaging Content Severity in Online Mental Health Forums",
            "authors": [
                "Arman Cohan",
                "Sydney Young",
                "Andrew Yates",
                "Nazli Goharian"
            ],
            "abstract": "Mental health forums are online communities where people express their issues\nand seek help from moderators and other users. In such forums, there are often\nposts with severe content indicating that the user is in acute distress and\nthere is a risk of attempted self-harm. Moderators need to respond to these\nsevere posts in a timely manner to prevent potential self-harm. However, the\nlarge volume of daily posted content makes it difficult for the moderators to\nlocate and respond to these critical posts. We present a framework for triaging\nuser content into four severity categories which are defined based on\nindications of self-harm ideation. Our models are based on a feature-rich\nclassification framework which includes lexical, psycholinguistic, contextual\nand topic modeling features. Our approaches improve the state of the art in\ntriaging the content severity in mental health forums by large margins (up to\n17% improvement over the F-1 scores). Using the proposed model, we analyze the\nmental state of users and we show that overall, long-term users of the forum\ndemonstrate a decreased severity of risk over time. Our analysis on the\ninteraction of the moderators with the users further indicates that without an\nautomatic way to identify critical content, it is indeed challenging for the\nmoderators to provide timely response to the users in need.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07186v2",
            "date": "2017-02-23 12:00:10+00:00",
            "title": "Stability of Topic Modeling via Matrix Factorization",
            "authors": [
                "Mark Belford",
                "Brian Mac Namee",
                "Derek Greene"
            ],
            "abstract": "Topic models can provide us with an insight into the underlying latent\nstructure of a large corpus of documents. A range of methods have been proposed\nin the literature, including probabilistic topic models and techniques based on\nmatrix factorization. However, in both cases, standard implementations rely on\nstochastic elements in their initialization phase, which can potentially lead\nto different results being generated on the same corpus when using the same\nparameter values. This corresponds to the concept of \"instability\" which has\npreviously been studied in the context of $k$-means clustering. In many\napplications of topic modeling, this problem of instability is not considered\nand topic models are treated as being definitive, even though the results may\nchange considerably if the initialization process is altered. In this paper we\ndemonstrate the inherent instability of popular topic modeling approaches,\nusing a number of new measures to assess stability. To address this issue in\nthe context of matrix factorization for topic modeling, we propose the use of\nensemble learning strategies. Based on experiments performed on annotated text\ncorpora, we show that a K-Fold ensemble strategy, combining both ensembles and\nstructured initialization, can significantly reduce instability, while\nsimultaneously yielding more accurate topic models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1702.07680v1",
            "date": "2017-02-24 17:40:28+00:00",
            "title": "Consistent Alignment of Word Embedding Models",
            "authors": [
                "Cem Safak Sahin",
                "Rajmonda S. Caceres",
                "Brandon Oselio",
                "William M. Campbell"
            ],
            "abstract": "Word embedding models offer continuous vector representations that can\ncapture rich contextual semantics based on their word co-occurrence patterns.\nWhile these word vectors can provide very effective features used in many NLP\ntasks such as clustering similar words and inferring learning relationships,\nmany challenges and open research questions remain. In this paper, we propose a\nsolution that aligns variations of the same model (or different models) in a\njoint low-dimensional latent space leveraging carefully generated synthetic\ndata points. This generative process is inspired by the observation that a\nvariety of linguistic relationships is captured by simple linear operations in\nembedded space. We demonstrate that our approach can lead to substantial\nimprovements in recovering embeddings of local neighborhoods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07825v2",
            "date": "2017-02-25 03:11:04+00:00",
            "title": "Deep Voice: Real-time Neural Text-to-Speech",
            "authors": [
                "Sercan O. Arik",
                "Mike Chrzanowski",
                "Adam Coates",
                "Gregory Diamos",
                "Andrew Gibiansky",
                "Yongguo Kang",
                "Xian Li",
                "John Miller",
                "Andrew Ng",
                "Jonathan Raiman",
                "Shubho Sengupta",
                "Mohammad Shoeybi"
            ],
            "abstract": "We present Deep Voice, a production-quality text-to-speech system constructed\nentirely from deep neural networks. Deep Voice lays the groundwork for truly\nend-to-end neural speech synthesis. The system comprises five major building\nblocks: a segmentation model for locating phoneme boundaries, a\ngrapheme-to-phoneme conversion model, a phoneme duration prediction model, a\nfundamental frequency prediction model, and an audio synthesis model. For the\nsegmentation model, we propose a novel way of performing phoneme boundary\ndetection with deep neural networks using connectionist temporal classification\n(CTC) loss. For the audio synthesis model, we implement a variant of WaveNet\nthat requires fewer parameters and trains faster than the original. By using a\nneural network for each component, our system is simpler and more flexible than\ntraditional text-to-speech systems, where each component requires laborious\nfeature engineering and extensive domain expertise. Finally, we show that\ninference with our system can be performed faster than real time and describe\noptimized WaveNet inference kernels on both CPU and GPU that achieve up to 400x\nspeedups over existing implementations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07826v2",
            "date": "2017-02-25 03:20:49+00:00",
            "title": "Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations",
            "authors": [
                "Upol Ehsan",
                "Brent Harrison",
                "Larry Chan",
                "Mark O. Riedl"
            ],
            "abstract": "We introduce AI rationalization, an approach for generating explanations of\nautonomous system behavior as if a human had performed the behavior. We\ndescribe a rationalization technique that uses neural machine translation to\ntranslate internal state-action representations of an autonomous agent into\nnatural language. We evaluate our technique in the Frogger game environment,\ntraining an autonomous game playing agent to rationalize its action choices\nusing natural language. A natural language training corpus is collected from\nhuman players thinking out loud as they play the game. We motivate the use of\nrationalization as an approach to explanation generation and show the results\nof two experiments evaluating the effectiveness of rationalization. Results of\nthese evaluations show that neural machine translation is able to accurately\ngenerate rationalizations that describe agent behavior, and that\nrationalizations are more satisfying to humans than other alternative methods\nof explanation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1702.07983v1",
            "date": "2017-02-26 03:19:13+00:00",
            "title": "Maximum-Likelihood Augmented Discrete Generative Adversarial Networks",
            "authors": [
                "Tong Che",
                "Yanran Li",
                "Ruixiang Zhang",
                "R Devon Hjelm",
                "Wenjie Li",
                "Yangqiu Song",
                "Yoshua Bengio"
            ],
            "abstract": "Despite the successes in capturing continuous distributions, the application\nof generative adversarial networks (GANs) to discrete settings, like natural\nlanguage tasks, is rather restricted. The fundamental reason is the difficulty\nof back-propagation through discrete random variables combined with the\ninherent instability of the GAN training objective. To address these problems,\nwe propose Maximum-Likelihood Augmented Discrete Generative Adversarial\nNetworks. Instead of directly optimizing the GAN objective, we derive a novel\nand low-variance objective using the discriminator's output that follows\ncorresponds to the log-likelihood. Compared with the original, the new\nobjective is proved to be consistent in theory and beneficial in practice. The\nexperimental results on various discrete datasets demonstrate the effectiveness\nof the proposed approach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1702.08139v2",
            "date": "2017-02-27 04:16:01+00:00",
            "title": "Improved Variational Autoencoders for Text Modeling using Dilated Convolutions",
            "authors": [
                "Zichao Yang",
                "Zhiting Hu",
                "Ruslan Salakhutdinov",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Recent work on generative modeling of text has found that variational\nauto-encoders (VAE) incorporating LSTM decoders perform worse than simpler LSTM\nlanguage models (Bowman et al., 2015). This negative result is so far poorly\nunderstood, but has been attributed to the propensity of LSTM decoders to\nignore conditioning information from the encoder. In this paper, we experiment\nwith a new type of decoder for VAE: a dilated CNN. By changing the decoder's\ndilation architecture, we control the effective context from previously\ngenerated words. In experiments, we find that there is a trade off between the\ncontextual capacity of the decoder and the amount of encoding information used.\nWe show that with the right decoder, VAE can outperform LSTM language models.\nWe demonstrate perplexity gains on two datasets, representing the first\npositive experimental result on the use VAE for generative modeling of text.\nFurther, we conduct an in-depth investigation of the use of VAE (with our new\ndecoding architecture) for semi-supervised and unsupervised labeling tasks,\ndemonstrating gains over several strong baselines.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1703.00050v1",
            "date": "2017-02-28 20:47:47+00:00",
            "title": "SceneSeer: 3D Scene Design with Natural Language",
            "authors": [
                "Angel X. Chang",
                "Mihail Eric",
                "Manolis Savva",
                "Christopher D. Manning"
            ],
            "abstract": "Designing 3D scenes is currently a creative task that requires significant\nexpertise and effort in using complex 3D design interfaces. This effortful\ndesign process starts in stark contrast to the easiness with which people can\nuse language to describe real and imaginary environments. We present SceneSeer:\nan interactive text to 3D scene generation system that allows a user to design\n3D scenes using natural language. A user provides input text from which we\nextract explicit constraints on the objects that should appear in the scene.\nGiven these explicit constraints, the system then uses a spatial knowledge base\nlearned from an existing database of 3D scenes and 3D object models to infer an\narrangement of the objects forming a natural scene matching the input\ndescription. Using textual commands the user can then iteratively refine the\ncreated scene by adding, removing, replacing, and manipulating objects. We\nevaluate the quality of 3D scenes generated by SceneSeer in a perceptual\nevaluation experiment where we compare against manually designed scenes and\nsimpler baselines for 3D scene generation. We demonstrate how the generated\nscenes can be iteratively refined through simple natural language commands.",
            "journal ref": "None",
            "category": "cs.GR"
        },
        {
            "url": "http://arxiv.org/abs/1703.00096v2",
            "date": "2017-03-01 00:59:17+00:00",
            "title": "Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling",
            "authors": [
                "Hairong Liu",
                "Zhenyao Zhu",
                "Xiangang Li",
                "Sanjeev Satheesh"
            ],
            "abstract": "Most existing sequence labelling models rely on a fixed decomposition of a\ntarget sequence into a sequence of basic units. These methods suffer from two\nmajor drawbacks: 1) the set of basic units is fixed, such as the set of words,\ncharacters or phonemes in speech recognition, and 2) the decomposition of\ntarget sequences is fixed. These drawbacks usually result in sub-optimal\nperformance of modeling sequences. In this pa- per, we extend the popular CTC\nloss criterion to alleviate these limitations, and propose a new loss function\ncalled Gram-CTC. While preserving the advantages of CTC, Gram-CTC automatically\nlearns the best set of basic units (grams), as well as the most suitable\ndecomposition of tar- get sequences. Unlike CTC, Gram-CTC allows the model to\noutput variable number of characters at each time step, which enables the model\nto capture longer term dependency and improves the computational efficiency. We\ndemonstrate that the proposed Gram-CTC improves CTC in terms of both\nperformance and efficiency on the large vocabulary speech recognition task at\nmultiple scales of data, and that with Gram-CTC we can outperform the\nstate-of-the-art on a standard speech benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00099v1",
            "date": "2017-03-01 01:27:32+00:00",
            "title": "Learning Conversational Systems that Interleave Task and Non-Task Content",
            "authors": [
                "Zhou Yu",
                "Alan W Black",
                "Alexander I. Rudnicky"
            ],
            "abstract": "Task-oriented dialog systems have been applied in various tasks, such as\nautomated personal assistants, customer service providers and tutors. These\nsystems work well when users have clear and explicit intentions that are\nwell-aligned to the systems' capabilities. However, they fail if users\nintentions are not explicit. To address this shortcoming, we propose a\nframework to interleave non-task content (i.e. everyday social conversation)\ninto task conversations. When the task content fails, the system can still keep\nthe user engaged with the non-task content. We trained a policy using\nreinforcement learning algorithms to promote long-turn conversation coherence\nand consistency, so that the system can have smooth transitions between task\nand non-task content. To test the effectiveness of the proposed framework, we\ndeveloped a movie promotion dialog system. Experiments with human users\nindicate that a system that interleaves social and task content achieves a\nbetter task success rate and is also rated as more engaging compared to a pure\ntask-oriented system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.00948v1",
            "date": "2017-03-02 20:55:20+00:00",
            "title": "DAWT: Densely Annotated Wikipedia Texts across multiple languages",
            "authors": [
                "Nemanja Spasojevic",
                "Preeti Bhargava",
                "Guoning Hu"
            ],
            "abstract": "In this work, we open up the DAWT dataset - Densely Annotated Wikipedia Texts\nacross multiple languages. The annotations include labeled text mentions\nmapping to entities (represented by their Freebase machine ids) as well as the\ntype of the entity. The data set contains total of 13.6M articles, 5.0B tokens,\n13.8M mention entity co-occurrences. DAWT contains 4.8 times more anchor text\nto entity links than originally present in the Wikipedia markup. Moreover, it\nspans several languages including English, Spanish, Italian, German, French and\nArabic. We also present the methodology used to generate the dataset which\nenriches Wikipedia markup in order to increase number of links. In addition to\nthe main dataset, we open up several derived datasets including mention entity\nco-occurrence counts and entity embeddings, as well as mappings between\nFreebase ids and Wikidata item ids. We also discuss two applications of these\ndatasets and hope that opening them up would prove useful for the Natural\nLanguage Processing and Information Retrieval communities, as well as\nfacilitate multi-lingual research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.00955v4",
            "date": "2017-03-02 21:23:47+00:00",
            "title": "Toward Controlled Generation of Text",
            "authors": [
                "Zhiting Hu",
                "Zichao Yang",
                "Xiaodan Liang",
                "Ruslan Salakhutdinov",
                "Eric P. Xing"
            ],
            "abstract": "Generic generation and manipulation of text is challenging and has limited\nsuccess compared to recent deep generative modeling in visual domain. This\npaper aims at generating plausible natural language sentences, whose attributes\nare dynamically controlled by learning disentangled latent representations with\ndesignated semantics. We propose a new neural generative model which combines\nvariational auto-encoders and holistic attribute discriminators for effective\nimposition of semantic structures. With differentiable approximation to\ndiscrete text samples, explicit constraints on independent attribute controls,\nand efficient collaborative learning of generator and discriminators, our model\nlearns highly interpretable representations from even only word annotations,\nand produces realistic sentences with desired attributes. Quantitative\nevaluation validates the accuracy of sentence and attribute generation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1703.01557v2",
            "date": "2017-03-05 04:43:41+00:00",
            "title": "Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning",
            "authors": [
                "Lidong Bing",
                "William W. Cohen",
                "Bhuwan Dhingra"
            ],
            "abstract": "We propose a general approach to modeling semi-supervised learning (SSL)\nalgorithms. Specifically, we present a declarative language for modeling both\ntraditional supervised classification tasks and many SSL heuristics, including\nboth well-known heuristics such as co-training and novel domain-specific\nheuristics. In addition to representing individual SSL heuristics, we show that\nmultiple heuristics can be automatically combined using Bayesian optimization\nmethods. We experiment with two classes of tasks, link-based text\nclassification and relation extraction. We show modest improvements on\nwell-studied link-based classification benchmarks, and state-of-the-art results\non relation-extraction tasks for two realistic domains.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1703.01619v1",
            "date": "2017-03-05 16:10:11+00:00",
            "title": "Neural Machine Translation and Sequence-to-sequence Models: A Tutorial",
            "authors": [
                "Graham Neubig"
            ],
            "abstract": "This tutorial introduces a new and powerful set of techniques variously\ncalled \"neural machine translation\" or \"neural sequence-to-sequence models\".\nThese techniques have been used in a number of tasks regarding the handling of\nhuman language, and can be a powerful tool in the toolbox of anyone who wants\nto model sequential data of some sort. The tutorial assumes that the reader\nknows the basics of math and programming, but does not assume any particular\nexperience with neural networks or natural language processing. It attempts to\nexplain the intuition behind the various methods covered, then delves into them\nwith enough mathematical detail to understand them concretely, and culiminates\nwith a suggestion for an implementation exercise, where readers can test that\nthey understood the content in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01720v4",
            "date": "2017-03-06 04:30:12+00:00",
            "title": "Sound-Word2Vec: Learning Word Representations Grounded in Sounds",
            "authors": [
                "Ashwin K Vijayakumar",
                "Ramakrishna Vedantam",
                "Devi Parikh"
            ],
            "abstract": "To be able to interact better with humans, it is crucial for machines to\nunderstand sound - a primary modality of human perception. Previous works have\nused sound to learn embeddings for improved generic textual similarity\nassessment. In this work, we treat sound as a first-class citizen, studying\ndownstream textual tasks which require aural grounding. To this end, we propose\nsound-word2vec - a new embedding scheme that learns specialized word embeddings\ngrounded in sounds. For example, we learn that two seemingly (semantically)\nunrelated concepts, like leaves and paper are similar due to the similar\nrustling sounds they make. Our embeddings prove useful in textual tasks\nrequiring aural reasoning like text-based sound retrieval and discovering foley\nsound effects (used in movies). Moreover, our embedding space captures\ninteresting dependencies between words and onomatopoeia and outperforms prior\nwork on aurally-relevant word relatedness datasets such as AMEN and ASLex.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.01725v1",
            "date": "2017-03-06 04:56:19+00:00",
            "title": "Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity",
            "authors": [
                "Jack Hessel",
                "Lillian Lee",
                "David Mimno"
            ],
            "abstract": "The content of today's social media is becoming more and more rich,\nincreasingly mixing text, images, videos, and audio. It is an intriguing\nresearch question to model the interplay between these different modes in\nattracting user attention and engagement. But in order to pursue this study of\nmultimodal content, we must also account for context: timing effects, community\npreferences, and social factors (e.g., which authors are already popular) also\naffect the amount of feedback and reaction that social-media posts receive. In\nthis work, we separate out the influence of these non-content factors in\nseveral ways. First, we focus on ranking pairs of submissions posted to the\nsame community in quick succession, e.g., within 30 seconds, this framing\nencourages models to focus on time-agnostic and community-specific content\nfeatures. Within that setting, we determine the relative performance of author\nvs. content features. We find that victory usually belongs to \"cats and\ncaptions,\" as visual and textual features together tend to outperform\nidentity-based features. Moreover, our experiments show that when considered in\nisolation, simple unigram text features and deep neural network visual features\nyield the highest accuracy individually, and that the combination of the two\nmodalities generally leads to the best accuracies overall.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1703.01898v2",
            "date": "2017-03-06 14:40:09+00:00",
            "title": "Generative and Discriminative Text Classification with Recurrent Neural Networks",
            "authors": [
                "Dani Yogatama",
                "Chris Dyer",
                "Wang Ling",
                "Phil Blunsom"
            ],
            "abstract": "We empirically characterize the performance of discriminative and generative\nLSTM models for text classification. We find that although RNN-based generative\nmodels are more powerful than their bag-of-words ancestors (e.g., they account\nfor conditional dependencies across words in a document), they have higher\nasymptotic error rates than discriminatively trained RNN models. However we\nalso find that generative models approach their asymptotic error rate more\nrapidly than their discriminative counterparts---the same pattern that Ng &\nJordan (2001) proved holds for linear classification models that make more\nnaive conditional independence assumptions. Building on this finding, we\nhypothesize that RNN-based generative classification models will be more robust\nto shifts in the data distribution. This hypothesis is confirmed in a series of\nexperiments in zero-shot and continual learning settings that show that\ngenerative models substantially outperform discriminative models.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1703.02504v1",
            "date": "2017-03-07 18:15:57+00:00",
            "title": "Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification",
            "authors": [
                "Jan Deriu",
                "Aurelien Lucchi",
                "Valeria De Luca",
                "Aliaksei Severyn",
                "Simon M\u00fcller",
                "Mark Cieliebak",
                "Thomas Hofmann",
                "Martin Jaggi"
            ],
            "abstract": "This paper presents a novel approach for multi-lingual sentiment\nclassification in short texts. This is a challenging task as the amount of\ntraining data in languages other than English is very limited. Previously\nproposed multi-lingual approaches typically require to establish a\ncorrespondence to English for which powerful classifiers are already available.\nIn contrast, our method does not require such supervision. We leverage large\namounts of weakly-supervised data in various languages to train a multi-layer\nconvolutional network and demonstrate the importance of using pre-training of\nsuch networks. We thoroughly evaluate our approach on various multi-lingual\ndatasets, including the recent SemEval-2016 sentiment prediction benchmark\n(Task 4), where we achieved state-of-the-art performance. We also compare the\nperformance of our model trained individually for each language to a variant\ntrained for all languages at once. We show that the latter model reaches\nslightly worse - but still acceptable - performance when compared to the single\nlanguage model, while benefiting from better generalization properties across\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02507v3",
            "date": "2017-03-07 18:19:11+00:00",
            "title": "Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features",
            "authors": [
                "Matteo Pagliardini",
                "Prakhar Gupta",
                "Martin Jaggi"
            ],
            "abstract": "The recent tremendous success of unsupervised word embeddings in a multitude\nof applications raises the obvious question if similar methods could be derived\nto improve embeddings (i.e. semantic representations) of word sequences as\nwell. We present a simple but efficient unsupervised objective to train\ndistributed representations of sentences. Our method outperforms the\nstate-of-the-art unsupervised models on most benchmark tasks, highlighting the\nrobustness of the produced general-purpose sentence embeddings.",
            "journal ref": "NAACL 2018 - Conference of the North American Chapter of the\n  Association for Computational Linguistics, pages 528-540",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03130v1",
            "date": "2017-03-09 04:42:30+00:00",
            "title": "A Structured Self-attentive Sentence Embedding",
            "authors": [
                "Zhouhan Lin",
                "Minwei Feng",
                "Cicero Nogueira dos Santos",
                "Mo Yu",
                "Bing Xiang",
                "Bowen Zhou",
                "Yoshua Bengio"
            ],
            "abstract": "This paper proposes a new model for extracting an interpretable sentence\nembedding by introducing self-attention. Instead of using a vector, we use a\n2-D matrix to represent the embedding, with each row of the matrix attending on\na different part of the sentence. We also propose a self-attention mechanism\nand a special regularization term for the model. As a side effect, the\nembedding comes with an easy way of visualizing what specific parts of the\nsentence are encoded into the embedding. We evaluate our model on 3 different\ntasks: author profiling, sentiment classification, and textual entailment.\nResults show that our model yields a significant performance gain compared to\nother sentence embedding methods in all of the 3 tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03609v1",
            "date": "2017-03-10 10:17:27+00:00",
            "title": "NetSpam: a Network-based Spam Detection Framework for Reviews in Online Social Media",
            "authors": [
                "Saeedreza Shehnepoor",
                "Mostafa Salehi",
                "Reza Farahbakhsh",
                "Noel Crespi"
            ],
            "abstract": "Nowadays, a big part of people rely on available content in social media in\ntheir decisions (e.g. reviews and feedback on a topic or product). The\npossibility that anybody can leave a review provide a golden opportunity for\nspammers to write spam reviews about products and services for different\ninterests. Identifying these spammers and the spam content is a hot topic of\nresearch and although a considerable number of studies have been done recently\ntoward this end, but so far the methodologies put forth still barely detect\nspam reviews, and none of them show the importance of each extracted feature\ntype. In this study, we propose a novel framework, named NetSpam, which\nutilizes spam features for modeling review datasets as heterogeneous\ninformation networks to map spam detection procedure into a classification\nproblem in such networks. Using the importance of spam features help us to\nobtain better results in terms of different metrics experimented on real-world\nreview datasets from Yelp and Amazon websites. The results show that NetSpam\noutperforms the existing methods and among four categories of features;\nincluding review-behavioral, user-behavioral, reviewlinguistic,\nuser-linguistic, the first type of features performs better than the other\ncategories.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1703.03714v1",
            "date": "2017-03-10 15:27:45+00:00",
            "title": "Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialogue",
            "authors": [
                "Matthew Marge",
                "Claire Bonial",
                "Brendan Byrne",
                "Taylor Cassidy",
                "A. William Evans",
                "Susan G. Hill",
                "Clare Voss"
            ],
            "abstract": "Our overall program objective is to provide more natural ways for soldiers to\ninteract and communicate with robots, much like how soldiers communicate with\nother soldiers today. We describe how the Wizard-of-Oz (WOz) method can be\napplied to multimodal human-robot dialogue in a collaborative exploration task.\nWhile the WOz method can help design robot behaviors, traditional approaches\nplace the burden of decisions on a single wizard. In this work, we consider two\nwizards to stand in for robot navigation and dialogue management software\ncomponents. The scenario used to elicit data is one in which a human-robot team\nis tasked with exploring an unknown environment: a human gives verbal\ninstructions from a remote location and the robot follows them, clarifying\npossible misunderstandings as needed via dialogue. We found the division of\nlabor between wizards to be workable, which holds promise for future software\ndevelopment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.03939v1",
            "date": "2017-03-11 10:05:19+00:00",
            "title": "Ask Me Even More: Dynamic Memory Tensor Networks (Extended Model)",
            "authors": [
                "Govardana Sachithanandam Ramachandran",
                "Ajay Sohmshetty"
            ],
            "abstract": "We examine Memory Networks for the task of question answering (QA), under\ncommon real world scenario where training examples are scarce and under weakly\nsupervised scenario, that is only extrinsic labels are available for training.\nWe propose extensions for the Dynamic Memory Network (DMN), specifically within\nthe attention mechanism, we call the resulting Neural Architecture as Dynamic\nMemory Tensor Network (DMTN). Ultimately, we see that our proposed extensions\nresults in over 80% improvement in the number of task passed against the\nbaselined standard DMN and 20% more task passed compared to state-of-the-art\nEnd-to-End Memory Network for Facebook's single task weakly trained 1K bAbi\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.04081v2",
            "date": "2017-03-12 08:11:29+00:00",
            "title": "Feature overwriting as a finite mixture process: Evidence from comprehension data",
            "authors": [
                "Shravan Vasishth",
                "Lena A. J\u00e4ger",
                "Bruno Nicenboim"
            ],
            "abstract": "The ungrammatical sentence \"The key to the cabinets are on the table\" is\nknown to lead to an illusion of grammaticality. As discussed in the\nmeta-analysis by Jaeger et al., 2017, faster reading times are observed at the\nverb are in the agreement-attraction sentence above compared to the equally\nungrammatical sentence \"The key to the cabinet are on the table\". One\nexplanation for this facilitation effect is the feature percolation account:\nthe plural feature on cabinets percolates up to the head noun key, leading to\nthe illusion. An alternative account is in terms of cue-based retrieval (Lewis\n& Vasishth, 2005), which assumes that the non-subject noun cabinets is\nmisretrieved due to a partial feature-match when a dependency completion\nprocess at the auxiliary initiates a memory access for a subject with plural\nmarking. We present evidence for yet another explanation for the observed\nfacilitation. Because the second sentence has two nouns with identical number,\nit is possible that these are, in some proportion of trials, more difficult to\nkeep distinct, leading to slower reading times at the verb in the first\nsentence above; this is the feature overwriting account of Nairne, 1990. We\nshow that the feature overwriting proposal can be implemented as a finite\nmixture process. We reanalysed ten published data-sets, fitting hierarchical\nBayesian mixture models to these data assuming a two-mixture distribution. We\nshow that in nine out of the ten studies, a mixture distribution corresponding\nto feature overwriting furnishes a superior fit over both the feature\npercolation and the cue-based retrieval accounts.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1703.04498v1",
            "date": "2017-03-13 17:34:18+00:00",
            "title": "High-Throughput and Language-Agnostic Entity Disambiguation and Linking on User Generated Data",
            "authors": [
                "Preeti Bhargava",
                "Nemanja Spasojevic",
                "Guoning Hu"
            ],
            "abstract": "The Entity Disambiguation and Linking (EDL) task matches entity mentions in\ntext to a unique Knowledge Base (KB) identifier such as a Wikipedia or Freebase\nid. It plays a critical role in the construction of a high quality information\nnetwork, and can be further leveraged for a variety of information retrieval\nand NLP tasks such as text categorization and document tagging. EDL is a\ncomplex and challenging problem due to ambiguity of the mentions and real world\ntext being multi-lingual. Moreover, EDL systems need to have high throughput\nand should be lightweight in order to scale to large datasets and run on\noff-the-shelf machines. More importantly, these systems need to be able to\nextract and disambiguate dense annotations from the data in order to enable an\nInformation Retrieval or Extraction task running on the data to be more\nefficient and accurate. In order to address all these challenges, we present\nthe Lithium EDL system and algorithm - a high-throughput, lightweight,\nlanguage-agnostic EDL system that extracts and correctly disambiguates 75% more\nentities than state-of-the-art EDL systems and is significantly faster than\nthem.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.04816v3",
            "date": "2017-03-14 23:09:45+00:00",
            "title": "Making Neural QA as Simple as Possible but not Simpler",
            "authors": [
                "Dirk Weissenborn",
                "Georg Wiese",
                "Laura Seiffe"
            ],
            "abstract": "Recent development of large-scale question answering (QA) datasets triggered\na substantial amount of research into end-to-end neural architectures for QA.\nIncreasingly complex systems have been conceived without comparison to simpler\nneural baseline systems that would justify their complexity. In this work, we\npropose a simple heuristic that guides the development of neural baseline\nsystems for the extractive QA task. We find that there are two ingredients\nnecessary for building a high-performing neural QA system: first, the awareness\nof question words while processing the context and second, a composition\nfunction that goes beyond simple bag-of-words modeling, such as recurrent\nneural networks. Our results show that FastQA, a system that meets these two\nrequirements, can achieve very competitive performance compared with existing\nmodels. We argue that this surprising finding puts results of previous systems\nand the complexity of recent QA datasets into perspective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05390v3",
            "date": "2017-03-15 21:20:44+00:00",
            "title": "Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting",
            "authors": [
                "Sercan O. Arik",
                "Markus Kliegl",
                "Rewon Child",
                "Joel Hestness",
                "Andrew Gibiansky",
                "Chris Fougner",
                "Ryan Prenger",
                "Adam Coates"
            ],
            "abstract": "Keyword spotting (KWS) constitutes a major component of human-technology\ninterfaces. Maximizing the detection accuracy at a low false alarm (FA) rate,\nwhile minimizing the footprint size, latency and complexity are the goals for\nKWS. Towards achieving them, we study Convolutional Recurrent Neural Networks\n(CRNNs). Inspired by large-scale state-of-the-art speech recognition systems,\nwe combine the strengths of convolutional layers and recurrent layers to\nexploit local structure and long-range context. We analyze the effect of\narchitecture parameters, and propose training strategies to improve\nperformance. With only ~230k parameters, our CRNN model yields acceptably low\nlatency, and achieves 97.71% accuracy at 0.5 FA/hour for 5 dB signal-to-noise\nratio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05880v2",
            "date": "2017-03-17 03:38:48+00:00",
            "title": "Empirical Evaluation of Parallel Training Algorithms on Acoustic Modeling",
            "authors": [
                "Wenpeng Li",
                "BinBin Zhang",
                "Lei Xie",
                "Dong Yu"
            ],
            "abstract": "Deep learning models (DLMs) are state-of-the-art techniques in speech\nrecognition. However, training good DLMs can be time consuming especially for\nproduction-size models and corpora. Although several parallel training\nalgorithms have been proposed to improve training efficiency, there is no clear\nguidance on which one to choose for the task in hand due to lack of systematic\nand fair comparison among them. In this paper we aim at filling this gap by\ncomparing four popular parallel training algorithms in speech recognition,\nnamely asynchronous stochastic gradient descent (ASGD), blockwise model-update\nfiltering (BMUF), bulk synchronous parallel (BSP) and elastic averaging\nstochastic gradient descent (EASGD), on 1000-hour LibriSpeech corpora using\nfeed-forward deep neural networks (DNNs) and convolutional, long short-term\nmemory, DNNs (CLDNNs). Based on our experiments, we recommend using BMUF as the\ntop choice to train acoustic models since it is most stable, scales well with\nnumber of GPUs, can achieve reproducible results, and in many cases even\noutperforms single-GPU SGD. ASGD can be used as a substitute in some cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.05908v2",
            "date": "2017-03-17 06:59:51+00:00",
            "title": "Learning Robust Visual-Semantic Embeddings",
            "authors": [
                "Yao-Hung Hubert Tsai",
                "Liang-Kang Huang",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Many of the existing methods for learning joint embedding of images and text\nuse only supervised information from paired images and its textual attributes.\nTaking advantage of the recent success of unsupervised learning in deep neural\nnetworks, we propose an end-to-end learning framework that is able to extract\nmore robust multi-modal representations across domains. The proposed method\ncombines representation learning models (i.e., auto-encoders) together with\ncross-domain learning criteria (i.e., Maximum Mean Discrepancy loss) to learn\njoint embeddings for semantic and visual features. A novel technique of\nunsupervised-data adaptation inference is introduced to construct more\ncomprehensive embeddings for both labeled and unlabeled data. We evaluate our\nmethod on Animals with Attributes and Caltech-UCSD Birds 200-2011 dataset with\na wide range of applications, including zero and few-shot image recognition and\nretrieval, from inductive to transductive settings. Empirically, we show that\nour framework improves over the current state of the art on many of the\nconsidered tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.06108v1",
            "date": "2017-03-17 17:16:02+00:00",
            "title": "Global Entity Ranking Across Multiple Languages",
            "authors": [
                "Prantik Bhattacharyya",
                "Nemanja Spasojevic"
            ],
            "abstract": "We present work on building a global long-tailed ranking of entities across\nmultiple languages using Wikipedia and Freebase knowledge bases. We identify\nmultiple features and build a model to rank entities using a ground-truth\ndataset of more than 10 thousand labels. The final system ranks 27 million\nentities with 75% precision and 48% F1 score. We provide performance evaluation\nand empirical evidence of the quality of ranking across languages, and open the\nfinal ranked lists for future research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.06585v2",
            "date": "2017-03-20 03:50:57+00:00",
            "title": "Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning",
            "authors": [
                "Abhishek Das",
                "Satwik Kottur",
                "Jos\u00e9 M. F. Moura",
                "Stefan Lee",
                "Dhruv Batra"
            ],
            "abstract": "We introduce the first goal-driven training for visual question answering and\ndialog agents. Specifically, we pose a cooperative 'image guessing' game\nbetween two agents -- Qbot and Abot -- who communicate in natural language\ndialog so that Qbot can select an unseen image from a lineup of images. We use\ndeep reinforcement learning (RL) to learn the policies of these agents\nend-to-end -- from pixels to multi-agent multi-round dialog to game reward.\n  We demonstrate two experimental results.\n  First, as a 'sanity check' demonstration of pure RL (from scratch), we show\nresults on a synthetic world, where the agents communicate in ungrounded\nvocabulary, i.e., symbols with no pre-specified meanings (X, Y, Z). We find\nthat two bots invent their own communication protocol and start using certain\nsymbols to ask/answer about certain visual attributes (shape/color/style).\nThus, we demonstrate the emergence of grounded language and communication among\n'visual' dialog agents with no human supervision.\n  Second, we conduct large-scale real-image experiments on the VisDial dataset,\nwhere we pretrain with supervised dialog data and show that the RL 'fine-tuned'\nagents significantly outperform SL agents. Interestingly, the RL Qbot learns to\nask questions that Abot is good at, ultimately resulting in more informative\ndialog and a better team.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.06642v2",
            "date": "2017-03-20 09:28:38+00:00",
            "title": "Towards a Quantum World Wide Web",
            "authors": [
                "Diederik Aerts",
                "Jonito Aerts Arguelles",
                "Lester Beltran",
                "Lyneth Beltran",
                "Isaac Distrito",
                "Massimiliano Sassoli de Bianchi",
                "Sandro Sozzo",
                "Tomas Veloz"
            ],
            "abstract": "We elaborate a quantum model for the meaning associated with corpora of\nwritten documents, like the pages forming the World Wide Web. To that end, we\nare guided by how physicists constructed quantum theory for microscopic\nentities, which unlike classical objects cannot be fully represented in our\nspatial theater. We suggest that a similar construction needs to be carried out\nby linguists and computational scientists, to capture the full meaning carried\nby collections of documental entities. More precisely, we show how to associate\na quantum-like 'entity of meaning' to a 'language entity formed by printed\ndocuments', considering the latter as the collection of traces that are left by\nthe former, in specific results of search actions that we describe as\nmeasurements. In other words, we offer a perspective where a collection of\ndocuments, like the Web, is described as the space of manifestation of a more\ncomplex entity - the QWeb - which is the object of our modeling, drawing its\ninspiration from previous studies on operational-realistic approaches to\nquantum physics and quantum modeling of human cognition and decision-making. We\nemphasize that a consistent QWeb model needs to account for the observed\ncorrelations between words appearing in printed documents, e.g.,\nco-occurrences, as the latter would depend on the 'meaning connections'\nexisting between the concepts that are associated with these words. In that\nrespect, we show that both 'context and interference (quantum) effects' are\nrequired to explain the probabilities calculated by counting the relative\nnumber of documents containing certain words and co-ocurrrences of words.",
            "journal ref": "Theoretical Computer Science, 752, pp. 116-131 (2018)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1703.07055v1",
            "date": "2017-03-21 04:56:14+00:00",
            "title": "Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems",
            "authors": [
                "Xiujun Li",
                "Yun-Nung Chen",
                "Lihong Li",
                "Jianfeng Gao",
                "Asli Celikyilmaz"
            ],
            "abstract": "Language understanding is a key component in a spoken dialogue system. In\nthis paper, we investigate how the language understanding module influences the\ndialogue system performance by conducting a series of systematic experiments on\na task-oriented neural dialogue system in a reinforcement learning based\nsetting. The empirical study shows that among different types of language\nunderstanding errors, slot-level errors can have more impact on the overall\nperformance of a dialogue system compared to intent-level errors. In addition,\nour experiments demonstrate that the reinforcement learning based dialogue\nsystem is able to learn when and what to confirm in order to achieve better\nperformance and greater robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.07588v2",
            "date": "2017-03-22 10:08:51+00:00",
            "title": "Gate Activation Signal Analysis for Gated Recurrent Neural Networks and Its Correlation with Phoneme Boundaries",
            "authors": [
                "Yu-Hsuan Wang",
                "Cheng-Tao Chung",
                "Hung-yi Lee"
            ],
            "abstract": "In this paper we analyze the gate activation signals inside the gated\nrecurrent neural networks, and find the temporal structure of such signals is\nhighly correlated with the phoneme boundaries. This correlation is further\nverified by a set of experiments for phoneme segmentation, in which better\nresults compared to standard approaches were obtained.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1703.07754v1",
            "date": "2017-03-22 17:17:16+00:00",
            "title": "Direct Acoustics-to-Word Models for English Conversational Speech Recognition",
            "authors": [
                "Kartik Audhkhasi",
                "Bhuvana Ramabhadran",
                "George Saon",
                "Michael Picheny",
                "David Nahamoo"
            ],
            "abstract": "Recent work on end-to-end automatic speech recognition (ASR) has shown that\nthe connectionist temporal classification (CTC) loss can be used to convert\nacoustics to phone or character sequences. Such systems are used with a\ndictionary and separately-trained Language Model (LM) to produce word\nsequences. However, they are not truly end-to-end in the sense of mapping\nacoustics directly to words without an intermediate phone representation. In\nthis paper, we present the first results employing direct acoustics-to-word CTC\nmodels on two well-known public benchmark tasks: Switchboard and CallHome.\nThese models do not require an LM or even a decoder at run-time and hence\nrecognize speech with minimal complexity. However, due to the large number of\nword output units, CTC word models require orders of magnitude more data to\ntrain reliably compared to traditional systems. We present some techniques to\nmitigate this issue. Our CTC word model achieves a word error rate of\n13.0%/18.8% on the Hub5-2000 Switchboard/CallHome test sets without any LM or\ndecoder compared with 9.6%/16.0% for phone-based CTC with a 4-gram LM. We also\npresent rescoring results on CTC word model lattices to quantify the\nperformance benefits of a LM, and contrast the performance of word and phone\nCTC models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08098v9",
            "date": "2017-03-23 15:15:26+00:00",
            "title": "A survey of embedding models of entities and relationships for knowledge graph completion",
            "authors": [
                "Dat Quoc Nguyen"
            ],
            "abstract": "Knowledge graphs (KGs) of real-world facts about entities and their\nrelationships are useful resources for a variety of natural language processing\ntasks. However, because knowledge graphs are typically incomplete, it is useful\nto perform knowledge graph completion or link prediction, i.e. predict whether\na relationship not in the knowledge graph is likely to be true. This paper\nserves as a comprehensive survey of embedding models of entities and\nrelationships for knowledge graph completion, summarizing up-to-date\nexperimental results on standard benchmark datasets and pointing out potential\nfuture research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08324v2",
            "date": "2017-03-24 09:32:23+00:00",
            "title": "Are crossing dependencies really scarce?",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Carlos Gomez-Rodriguez",
                "J. L. Esteban"
            ],
            "abstract": "The syntactic structure of a sentence can be modelled as a tree, where\nvertices correspond to words and edges indicate syntactic dependencies. It has\nbeen claimed recurrently that the number of edge crossings in real sentences is\nsmall. However, a baseline or null hypothesis has been lacking. Here we\nquantify the amount of crossings of real sentences and compare it to the\npredictions of a series of baselines. We conclude that crossings are really\nscarce in real sentences. Their scarcity is unexpected by the hubiness of the\ntrees. Indeed, real sentences are close to linear trees, where the potential\nnumber of crossings is maximized.",
            "journal ref": "Physica A: Statistical Mechanics and its Applications,\n  493:311-329, 2018",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1703.08428v1",
            "date": "2017-03-24 14:40:31+00:00",
            "title": "Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop",
            "authors": [
                "Justin Cranshaw",
                "Emad Elwany",
                "Todd Newman",
                "Rafal Kocielnik",
                "Bowen Yu",
                "Sandeep Soni",
                "Jaime Teevan",
                "Andr\u00e9s Monroy-Hern\u00e1ndez"
            ],
            "abstract": "Although information workers may complain about meetings, they are an\nessential part of their work life. Consequently, busy people spend a\nsignificant amount of time scheduling meetings. We present Calendar.help, a\nsystem that provides fast, efficient scheduling through structured workflows.\nUsers interact with the system via email, delegating their scheduling needs to\nthe system as if it were a human personal assistant. Common scheduling\nscenarios are broken down using well-defined workflows and completed as a\nseries of microtasks that are automated when possible and executed by a human\notherwise. Unusual scenarios fall back to a trained human assistant who\nexecutes them as unstructured macrotasks. We describe the iterative approach we\nused to develop Calendar.help, and share the lessons learned from scheduling\nthousands of meetings during a year of real-world deployments. Our findings\nprovide insight into how complex information tasks can be broken down into\nrepeatable components that can be executed efficiently to improve productivity.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1703.08581v2",
            "date": "2017-03-24 19:45:24+00:00",
            "title": "Sequence-to-Sequence Models Can Directly Translate Foreign Speech",
            "authors": [
                "Ron J. Weiss",
                "Jan Chorowski",
                "Navdeep Jaitly",
                "Yonghui Wu",
                "Zhifeng Chen"
            ],
            "abstract": "We present a recurrent encoder-decoder deep neural network architecture that\ndirectly translates speech in one language into text in another. The model does\nnot explicitly transcribe the speech into text in the source language, nor does\nit require supervision from the ground truth source language transcription\nduring training. We apply a slightly modified sequence-to-sequence with\nattention architecture that has previously been used for speech recognition and\nshow that it can be repurposed for this more complex task, illustrating the\npower of attention-based models. A single model trained end-to-end obtains\nstate-of-the-art performance on the Fisher Callhome Spanish-English speech\ntranslation task, outperforming a cascade of independently trained\nsequence-to-sequence speech recognition and machine translation models by 1.8\nBLEU points on the Fisher test set. In addition, we find that making use of the\ntraining data in both languages by multi-task training sequence-to-sequence\nspeech translation and recognition models with a shared encoder network can\nimprove performance by a further 1.4 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.08705v1",
            "date": "2017-03-25 15:37:09+00:00",
            "title": "Comparing Rule-Based and Deep Learning Models for Patient Phenotyping",
            "authors": [
                "Sebastian Gehrmann",
                "Franck Dernoncourt",
                "Yeran Li",
                "Eric T. Carlson",
                "Joy T. Wu",
                "Jonathan Welt",
                "John Foote Jr.",
                "Edward T. Moseley",
                "David W. Grant",
                "Patrick D. Tyler",
                "Leo Anthony Celi"
            ],
            "abstract": "Objective: We investigate whether deep learning techniques for natural\nlanguage processing (NLP) can be used efficiently for patient phenotyping.\nPatient phenotyping is a classification task for determining whether a patient\nhas a medical condition, and is a crucial part of secondary analysis of\nhealthcare data. We assess the performance of deep learning algorithms and\ncompare them with classical NLP approaches.\n  Materials and Methods: We compare convolutional neural networks (CNNs),\nn-gram models, and approaches based on cTAKES that extract pre-defined medical\nconcepts from clinical notes and use them to predict patient phenotypes. The\nperformance is tested on 10 different phenotyping tasks using 1,610 discharge\nsummaries extracted from the MIMIC-III database.\n  Results: CNNs outperform other phenotyping algorithms in all 10 tasks. The\naverage F1-score of our model is 76 (PPV of 83, and sensitivity of 71) with our\nmodel having an F1-score up to 37 points higher than alternative approaches. We\nadditionally assess the interpretability of our model by presenting a method\nthat extracts the most salient phrases for a particular prediction.\n  Conclusion: We show that NLP methods based on deep learning improve the\nperformance of patient phenotyping. Our CNN-based algorithm automatically\nlearns the phrases associated with each patient phenotype. As such, it reduces\nthe annotation complexity for clinical domain experts, who are normally\nrequired to develop task-specific annotation rules and identify relevant\nphrases. Our method performs well in terms of both performance and\ninterpretability, which indicates that deep learning is an effective approach\nto patient phenotyping based on clinicians' notes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.09137v2",
            "date": "2017-03-27 15:13:49+00:00",
            "title": "Where to put the Image in an Image Caption Generator",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Kenneth P. Camilleri"
            ],
            "abstract": "When a recurrent neural network language model is used for caption\ngeneration, the image information can be fed to the neural network either by\ndirectly incorporating it in the RNN -- conditioning the language model by\n`injecting' image features -- or in a layer following the RNN -- conditioning\nthe language model by `merging' image features. While both options are attested\nin the literature, there is as yet no systematic comparison between the two. In\nthis paper we empirically show that it is not especially detrimental to\nperformance whether one architecture is used or another. The merge architecture\ndoes have practical advantages, as conditioning by merging allows the RNN's\nhidden state vector to shrink in size by up to four times. Our results suggest\nthat the visual and linguistic modalities for caption generation need not be\njointly encoded by the RNN as that yields large, memory-intensive models with\nfew tangible advantages in performance; rather, the multimodal integration\nshould be delayed to a subsequent stage.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1703.09684v2",
            "date": "2017-03-28 17:48:07+00:00",
            "title": "An Analysis of Visual Question Answering Algorithms",
            "authors": [
                "Kushal Kafle",
                "Christopher Kanan"
            ],
            "abstract": "In visual question answering (VQA), an algorithm must answer text-based\nquestions about images. While multiple datasets for VQA have been created since\nlate 2014, they all have flaws in both their content and the way algorithms are\nevaluated on them. As a result, evaluation scores are inflated and\npredominantly determined by answering easier questions, making it difficult to\ncompare different methods. In this paper, we analyze existing VQA algorithms\nusing a new dataset. It contains over 1.6 million questions organized into 12\ndifferent categories. We also introduce questions that are meaningless for a\ngiven image to force a VQA system to reason about image content. We propose new\nevaluation schemes that compensate for over-represented question-types and make\nit easier to study the strengths and weaknesses of algorithms. We analyze the\nperformance of both baseline and state-of-the-art VQA models, including\nmulti-modal compact bilinear pooling (MCB), neural module networks, and\nrecurrent answering units. Our experiments establish how attention helps\ncertain categories more than others, determine which models work better than\nothers, and explain how simple models (e.g. MLP) can surpass more complex\nmodels (MCB) by simply learning to answer large, easy question categories.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.09749v1",
            "date": "2017-03-21 15:34:28+00:00",
            "title": "Developpement de Methodes Automatiques pour la Reutilisation des Composants Logiciels",
            "authors": [
                "Kouakou Ive Arsene Koffi",
                "Konan Marcellin Brou",
                "Souleymane Oumtanaga"
            ],
            "abstract": "The large amount of information and the increasing complexity of applications\nconstrain developers to have stand-alone and reusable components from libraries\nand component markets.Our approach consists in developing methods to evaluate\nthe quality of the software component of these libraries, on the one hand and\nmoreover to optimize the financial cost and the adaptation's time of these\nselected components. Our objective function defines a metric that maximizes the\nvalue of the software component quality by minimizing the financial cost and\nmaintenance time. This model should make it possible to classify the components\nand order them in order to choose the most optimized.\n  MOTS-CLES : d{\\'e}veloppement de m{\\'e}thode, r{\\'e}utilisation, composants\nlogiciels, qualit{\\'e} de composant\n  KEYWORDS:method development, reuse, software components, component quality .",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1703.09902v4",
            "date": "2017-03-29 06:51:00+00:00",
            "title": "Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation",
            "authors": [
                "Albert Gatt",
                "Emiel Krahmer"
            ],
            "abstract": "This paper surveys the current state of the art in Natural Language\nGeneration (NLG), defined as the task of generating text or speech from\nnon-linguistic input. A survey of NLG is timely in view of the changes that the\nfield has undergone over the past decade or so, especially in relation to new\n(usually data-driven) methods, as well as new applications of NLG technology.\nThis survey therefore aims to (a) give an up-to-date synthesis of research on\nthe core tasks in NLG and the architectures adopted in which such tasks are\norganised; (b) highlight a number of relatively recent research topics that\nhave arisen partly as a result of growing synergies between NLG and other areas\nof artificial intelligence; (c) draw attention to the challenges in NLG\nevaluation, relating them to similar challenges faced in other areas of Natural\nLanguage Processing, with an emphasis on different evaluation methods and the\nrelationships between them.",
            "journal ref": "Journal of AI Research, volume 60, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10135v2",
            "date": "2017-03-29 16:55:13+00:00",
            "title": "Tacotron: Towards End-to-End Speech Synthesis",
            "authors": [
                "Yuxuan Wang",
                "RJ Skerry-Ryan",
                "Daisy Stanton",
                "Yonghui Wu",
                "Ron J. Weiss",
                "Navdeep Jaitly",
                "Zongheng Yang",
                "Ying Xiao",
                "Zhifeng Chen",
                "Samy Bengio",
                "Quoc Le",
                "Yannis Agiomyrgiannakis",
                "Rob Clark",
                "Rif A. Saurous"
            ],
            "abstract": "A text-to-speech synthesis system typically consists of multiple stages, such\nas a text analysis frontend, an acoustic model and an audio synthesis module.\nBuilding these components often requires extensive domain expertise and may\ncontain brittle design choices. In this paper, we present Tacotron, an\nend-to-end generative text-to-speech model that synthesizes speech directly\nfrom characters. Given <text, audio> pairs, the model can be trained completely\nfrom scratch with random initialization. We present several key techniques to\nmake the sequence-to-sequence framework perform well for this challenging task.\nTacotron achieves a 3.82 subjective 5-scale mean opinion score on US English,\noutperforming a production parametric system in terms of naturalness. In\naddition, since Tacotron generates speech at the frame level, it's\nsubstantially faster than sample-level autoregressive methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10252v1",
            "date": "2017-03-28 15:20:52+00:00",
            "title": "Linguistic Matrix Theory",
            "authors": [
                "Dimitrios Kartsaklis",
                "Sanjaye Ramgoolam",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "Recent research in computational linguistics has developed algorithms which\nassociate matrices with adjectives and verbs, based on the distribution of\nwords in a corpus of text. These matrices are linear operators on a vector\nspace of context words. They are used to construct the meaning of composite\nexpressions from that of the elementary constituents, forming part of a\ncompositional distributional approach to semantics. We propose a Matrix Theory\napproach to this data, based on permutation symmetry along with Gaussian\nweights and their perturbations. A simple Gaussian model is tested against word\nmatrices created from a large corpus of text. We characterize the cubic and\nquartic departures from the model, which we propose, alongside the Gaussian\nparameters, as signatures for comparison of linguistic corpora. We propose that\nperturbed Gaussian models with permutation symmetry provide a promising\nframework for characterizing the nature of universality in the statistical\nproperties of word matrices. The matrix theory framework developed here\nexploits the view of statistics as zero dimensional perturbative quantum field\ntheory. It perceives language as a physical system realizing a universality\nclass of matrix statistics characterized by permutation symmetry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.10339v2",
            "date": "2017-03-30 07:48:31+00:00",
            "title": "Finding News Citations for Wikipedia",
            "authors": [
                "Besnik Fetahu",
                "Katja Markert",
                "Wolfgang Nejdl",
                "Avishek Anand"
            ],
            "abstract": "An important editing policy in Wikipedia is to provide citations for added\nstatements in Wikipedia pages, where statements can be arbitrary pieces of\ntext, ranging from a sentence to a paragraph. In many cases citations are\neither outdated or missing altogether.\n  In this work we address the problem of finding and updating news citations\nfor statements in entity pages. We propose a two-stage supervised approach for\nthis problem. In the first step, we construct a classifier to find out whether\nstatements need a news citation or other kinds of citations (web, book,\njournal, etc.). In the second step, we develop a news citation algorithm for\nWikipedia statements, which recommends appropriate citations from a given news\ncollection. Apart from IR techniques that use the statement to query the news\ncollection, we also formalize three properties of an appropriate citation,\nnamely: (i) the citation should entail the Wikipedia statement, (ii) the\nstatement should be central to the citation, and (iii) the citation should be\nfrom an authoritative source.\n  We perform an extensive evaluation of both steps, using 20 million articles\nfrom a real-world news collection. Our results are quite promising, and show\nthat we can perform this task with high precision and at scale.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.10344v1",
            "date": "2017-03-30 07:56:42+00:00",
            "title": "Automated News Suggestions for Populating Wikipedia Entity Pages",
            "authors": [
                "Besnik Fetahu",
                "Katja Markert",
                "Avishek Anand"
            ],
            "abstract": "Wikipedia entity pages are a valuable source of information for direct\nconsumption and for knowledge-base construction, update and maintenance. Facts\nin these entity pages are typically supported by references. Recent studies\nshow that as much as 20\\% of the references are from online news sources.\nHowever, many entity pages are incomplete even if relevant information is\nalready available in existing news articles. Even for the already present\nreferences, there is often a delay between the news article publication time\nand the reference time. In this work, we therefore look at Wikipedia through\nthe lens of news and propose a novel news-article suggestion task to improve\nnews coverage in Wikipedia, and reduce the lag of newsworthy references. Our\nwork finds direct application, as a precursor, to Wikipedia page generation and\nknowledge-base acceleration tasks that rely on relevant and high quality input\nsources.\n  We propose a two-stage supervised approach for suggesting news articles to\nentity pages for a given state of Wikipedia. First, we suggest news articles to\nWikipedia entities (article-entity placement) relying on a rich set of features\nwhich take into account the \\emph{salience} and \\emph{relative authority} of\nentities, and the \\emph{novelty} of news articles to entity pages. Second, we\ndetermine the exact section in the entity page for the input article\n(article-section placement) guided by class-based section templates. We perform\nan extensive evaluation of our approach based on ground-truth data that is\nextracted from external references in Wikipedia. We achieve a high precision\nvalue of up to 93\\% in the \\emph{article-entity} suggestion stage and upto 84\\%\nfor the \\emph{article-section placement}. Finally, we compare our approach\nagainst competitive baselines and show significant improvements.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1703.10356v2",
            "date": "2017-03-30 08:40:19+00:00",
            "title": "Simplified End-to-End MMI Training and Voting for ASR",
            "authors": [
                "Lior Fritz",
                "David Burshtein"
            ],
            "abstract": "A simplified speech recognition system that uses the maximum mutual\ninformation (MMI) criterion is considered. End-to-end training using gradient\ndescent is suggested, similarly to the training of connectionist temporal\nclassification (CTC). We use an MMI criterion with a simple language model in\nthe training stage, and a standard HMM decoder. Our method compares favorably\nto CTC in terms of performance, robustness, decoding time, disk footprint and\nquality of alignments. The good alignments enable the use of a straightforward\nensemble method, obtained by simply averaging the predictions of several neural\nnetwork models, that were trained separately end-to-end. The ensemble method\nyields a considerable reduction in the word error rate.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1703.10476v2",
            "date": "2017-03-30 13:54:51+00:00",
            "title": "Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training",
            "authors": [
                "Rakshith Shetty",
                "Marcus Rohrbach",
                "Lisa Anne Hendricks",
                "Mario Fritz",
                "Bernt Schiele"
            ],
            "abstract": "While strong progress has been made in image captioning over the last years,\nmachine and human captions are still quite distinct. A closer look reveals that\nthis is due to the deficiencies in the generated word distribution, vocabulary\nsize, and strong bias in the generators towards frequent captions. Furthermore,\nhumans -- rightfully so -- generate multiple, diverse captions, due to the\ninherent ambiguity in the captioning task which is not considered in today's\nsystems.\n  To address these challenges, we change the training objective of the caption\ngenerator from reproducing groundtruth captions to generating a set of captions\nthat is indistinguishable from human generated captions. Instead of\nhandcrafting such a learning target, we employ adversarial training in\ncombination with an approximate Gumbel sampler to implicitly match the\ngenerated distribution to the human one. While our method achieves comparable\nperformance to the state-of-the-art in terms of the correctness of the\ncaptions, we generate a set of diverse captions, that are significantly less\nbiased and match the word statistics better in several aspects.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1703.10722v3",
            "date": "2017-03-31 00:50:37+00:00",
            "title": "Factorization tricks for LSTM networks",
            "authors": [
                "Oleksii Kuchaiev",
                "Boris Ginsburg"
            ],
            "abstract": "We present two simple ways of reducing the number of parameters and\naccelerating the training of large Long Short-Term Memory (LSTM) networks: the\nfirst one is \"matrix factorization by design\" of LSTM matrix into the product\nof two smaller matrices, and the second one is partitioning of LSTM matrix, its\ninputs and states into the independent groups. Both approaches allow us to\ntrain large LSTM networks significantly faster to the near state-of the art\nperplexity while using significantly less RNN parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00105v4",
            "date": "2017-04-29 01:03:40+00:00",
            "title": "Representation Learning and Pairwise Ranking for Implicit Feedback in Recommendation Systems",
            "authors": [
                "Sumit Sidana",
                "Mikhail Trofimov",
                "Oleg Horodnitskii",
                "Charlotte Laclau",
                "Yury Maximov",
                "Massih-Reza Amini"
            ],
            "abstract": "In this paper, we propose a novel ranking framework for collaborative\nfiltering with the overall aim of learning user preferences over items by\nminimizing a pairwise ranking loss. We show the minimization problem involves\ndependent random variables and provide a theoretical analysis by proving the\nconsistency of the empirical risk minimization in the worst case where all\nusers choose a minimal number of positive and negative items. We further derive\na Neural-Network model that jointly learns a new representation of users and\nitems in an embedded space as well as the preference relation of users over the\npairs of items. The learning objective is based on three scenarios of ranking\nlosses that control the ability of the model to maintain the ordering over the\nitems induced from the users' preferences, as well as, the capacity of the\ndot-product defined in the learned embedded space to produce the ordering. The\nproposed model is by nature suitable for implicit feedback and involves the\nestimation of only very few parameters. Through extensive experiments on\nseveral real-world benchmarks on implicit data, we show the interest of\nlearning the preference and the embedding simultaneously when compared to\nlearning those separately. We also demonstrate that our approach is very\ncompetitive with the best state-of-the-art collaborative filtering techniques\nproposed for implicit feedback.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1705.00321v4",
            "date": "2017-04-30 15:09:10+00:00",
            "title": "Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation",
            "authors": [
                "Ganbin Zhou",
                "Ping Luo",
                "Rongyu Cao",
                "Yijun Xiao",
                "Fen Lin",
                "Bo Chen",
                "Qing He"
            ],
            "abstract": "Different from other sequential data, sentences in natural language are\nstructured by linguistic grammars. Previous generative conversational models\nwith chain-structured decoder ignore this structure in human language and might\ngenerate plausible responses with less satisfactory relevance and fluency. In\nthis study, we aim to incorporate the results from linguistic analysis into the\nprocess of sentence generation for high-quality conversation generation.\nSpecifically, we use a dependency parser to transform each response sentence\ninto a dependency tree and construct a training corpus of sentence-tree pairs.\nA tree-structured decoder is developed to learn the mapping from a sentence to\nits tree, where different types of hidden states are used to depict the local\ndependencies from an internal tree node to its children. For training\nacceleration, we propose a tree canonicalization method, which transforms trees\ninto equivalent ternary trees. Then, with a proposed tree-structured search\nmethod, the model is able to generate the most probable responses in the form\nof dependency trees, which are finally flattened into sequences as the system\noutput. Experimental results demonstrate that the proposed X2Tree framework\noutperforms baseline methods over 11.15% increase of acceptance ratio.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.00335v1",
            "date": "2017-04-30 16:12:28+00:00",
            "title": "Quantifying Mental Health from Social Media with Neural User Embeddings",
            "authors": [
                "Silvio Amir",
                "Glen Coppersmith",
                "Paula Carvalho",
                "M\u00e1rio J. Silva",
                "Byron C. Wallace"
            ],
            "abstract": "Mental illnesses adversely affect a significant proportion of the population\nworldwide. However, the methods traditionally used for estimating and\ncharacterizing the prevalence of mental health conditions are time-consuming\nand expensive. Consequently, best-available estimates concerning the prevalence\nof mental health conditions are often years out of date. Automated approaches\nto supplement these survey methods with broad, aggregated information derived\nfrom social media content provides a potential means for near real-time\nestimates at scale. These may, in turn, provide grist for supporting,\nevaluating and iteratively improving upon public health programs and\ninterventions.\n  We propose a novel model for automated mental health status quantification\nthat incorporates user embeddings. This builds upon recent work exploring\nrepresentation learning methods that induce embeddings by leveraging social\nmedia post histories. Such embeddings capture latent characteristics of\nindividuals (e.g., political leanings) and encode a soft notion of homophily.\nIn this paper, we investigate whether user embeddings learned from twitter post\nhistories encode information that correlates with mental health statuses. To\nthis end, we estimated user embeddings for a set of users known to be affected\nby depression and post-traumatic stress disorder (PTSD), and for a set of\ndemographically matched `control' users. We then evaluated these embeddings\nwith respect to: (i) their ability to capture homophilic relations with respect\nto mental health status; and (ii) the performance of downstream mental health\nprediction models based on these features. Our experimental results demonstrate\nthat the user embeddings capture similarities between users with respect to\nmental conditions, and are predictive of mental health.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00557v1",
            "date": "2017-04-23 09:15:35+00:00",
            "title": "Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning",
            "authors": [
                "Yacine Jernite",
                "Samuel R. Bowman",
                "David Sontag"
            ],
            "abstract": "This work presents a novel objective function for the unsupervised training\nof neural network sentence encoders. It exploits signals from paragraph-level\ndiscourse coherence to train these models to understand text. Our objective is\npurely discriminative, allowing us to train models many times faster than was\npossible under prior methods, and it yields models which perform well in\nextrinsic evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00581v2",
            "date": "2017-05-01 16:28:18+00:00",
            "title": "Query-adaptive Video Summarization via Quality-aware Relevance Estimation",
            "authors": [
                "Arun Balajee Vasudevan",
                "Michael Gygli",
                "Anna Volokitin",
                "Luc Van Gool"
            ],
            "abstract": "Although the problem of automatic video summarization has recently received a\nlot of attention, the problem of creating a video summary that also highlights\nelements relevant to a search query has been less studied. We address this\nproblem by posing query-relevant summarization as a video frame subset\nselection problem, which lets us optimise for summaries which are\nsimultaneously diverse, representative of the entire video, and relevant to a\ntext query. We quantify relevance by measuring the distance between frames and\nqueries in a common textual-visual semantic embedding space induced by a neural\nnetwork. In addition, we extend the model to capture query-independent\nproperties, such as frame quality. We compare our method against previous state\nof the art on textual-visual embeddings for thumbnail selection and show that\nour model outperforms them on relevance prediction. Furthermore, we introduce a\nnew dataset, annotated with diversity and query-specific relevance labels. On\nthis dataset, we train and test our complete model for video summarization and\nshow that it outperforms standard baselines such as Maximal Marginal Relevance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.00694v2",
            "date": "2017-05-01 20:12:11+00:00",
            "title": "A polynomial time algorithm for the Lambek calculus with brackets of bounded order",
            "authors": [
                "Max Kanovich",
                "Stepan Kuznetsov",
                "Glyn Morrill",
                "Andre Scedrov"
            ],
            "abstract": "Lambek calculus is a logical foundation of categorial grammar, a linguistic\nparadigm of grammar as logic and parsing as deduction. Pentus (2010) gave a\npolynomial-time algorithm for determ- ining provability of bounded depth\nformulas in the Lambek calculus with empty antecedents allowed. Pentus'\nalgorithm is based on tabularisation of proof nets. Lambek calculus with\nbrackets is a conservative extension of Lambek calculus with bracket\nmodalities, suitable for the modeling of syntactical domains. In this paper we\ngive an algorithm for provability the Lambek calculus with brackets allowing\nempty antecedents. Our algorithm runs in polynomial time when both the formula\ndepth and the bracket nesting depth are bounded. It combines a Pentus-style\ntabularisation of proof nets with an automata-theoretic treatment of\nbracketing.",
            "journal ref": "Proc. FSCD 2017, LIPIcs vol. 84, 22:1-22:17",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1705.00697v1",
            "date": "2017-05-01 20:23:13+00:00",
            "title": "From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing",
            "authors": [
                "Juan Andr\u00e9s Laura",
                "Gabriel Masi",
                "Luis Argerich"
            ],
            "abstract": "In recent studies [1][13][12] Recurrent Neural Networks were used for\ngenerative processes and their surprising performance can be explained by their\nability to create good predictions. In addition, data compression is also based\non predictions. What the problem comes down to is whether a data compressor\ncould be used to perform as well as recurrent neural networks in natural\nlanguage processing tasks. If this is possible,then the problem comes down to\ndetermining if a compression algorithm is even more intelligent than a neural\nnetwork in specific tasks related to human language. In our journey we\ndiscovered what we think is the fundamental difference between a Data\nCompression Algorithm and a Recurrent Neural Network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.00995v2",
            "date": "2017-05-02 14:29:14+00:00",
            "title": "Fuzzy Approach Topic Discovery in Health and Medical Corpora",
            "authors": [
                "Amir Karami",
                "Aryya Gangopadhyay",
                "Bin Zhou",
                "Hadi Kharrazi"
            ],
            "abstract": "The majority of medical documents and electronic health records (EHRs) are in\ntext format that poses a challenge for data processing and finding relevant\ndocuments. Looking for ways to automatically retrieve the enormous amount of\nhealth and medical knowledge has always been an intriguing topic. Powerful\nmethods have been developed in recent years to make the text processing\nautomatic. One of the popular approaches to retrieve information based on\ndiscovering the themes in health & medical corpora is topic modeling, however,\nthis approach still needs new perspectives. In this research we describe fuzzy\nlatent semantic analysis (FLSA), a novel approach in topic modeling using fuzzy\nperspective. FLSA can handle health & medical corpora redundancy issue and\nprovides a new method to estimate the number of topics. The quantitative\nevaluations show that FLSA produces superior performance and features to latent\nDirichlet allocation (LDA), the most popular topic model.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1705.01346v1",
            "date": "2017-05-03 10:22:22+00:00",
            "title": "Going Wider: Recurrent Neural Network With Parallel Cells",
            "authors": [
                "Danhao Zhu",
                "Si Shen",
                "Xin-Yu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Recurrent Neural Network (RNN) has been widely applied for sequence modeling.\nIn RNN, the hidden states at current step are full connected to those at\nprevious step, thus the influence from less related features at previous step\nmay potentially decrease model's learning ability. We propose a simple\ntechnique called parallel cells (PCs) to enhance the learning ability of\nRecurrent Neural Network (RNN). In each layer, we run multiple small RNN cells\nrather than one single large cell. In this paper, we evaluate PCs on 2 tasks.\nOn language modeling task on PTB (Penn Tree Bank), our model outperforms state\nof art models by decreasing perplexity from 78.6 to 75.3. On Chinese-English\ntranslation task, our model increases BLEU score for 0.39 points than baseline\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.01359v1",
            "date": "2017-05-03 11:07:13+00:00",
            "title": "FOIL it! Find One mismatch between Image and Language caption",
            "authors": [
                "Ravi Shekhar",
                "Sandro Pezzelle",
                "Yauhen Klimovich",
                "Aurelie Herbelot",
                "Moin Nabi",
                "Enver Sangineto",
                "Raffaella Bernardi"
            ],
            "abstract": "In this paper, we aim to understand whether current language and vision\n(LaVi) models truly grasp the interaction between the two modalities. To this\nend, we propose an extension of the MSCOCO dataset, FOIL-COCO, which associates\nimages with both correct and \"foil\" captions, that is, descriptions of the\nimage that are highly similar to the original ones, but contain one single\nmistake (\"foil word\"). We show that current LaVi models fall into the traps of\nthis data and perform badly on three tasks: a) caption classification (correct\nvs. foil); b) foil word detection; c) foil word correction. Humans, in\ncontrast, have near-perfect performance on those tasks. We demonstrate that\nmerely utilising language cues is not enough to model FOIL-COCO and that it\nchallenges the state-of-the-art by requiring a fine-grained understanding of\nthe relation between text and image.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.02203v1",
            "date": "2017-04-18 13:24:41+00:00",
            "title": "Analysis of Computational Science Papers from ICCS 2001-2016 using Topic Modeling and Graph Theory",
            "authors": [
                "Tesfamariam M. Abuhay",
                "Sergey V. Kovalchuk",
                "Klavdiya O. Bochenina",
                "George Kampis",
                "Valeria V. Krzhizhanovskaya",
                "Michael H. Lees"
            ],
            "abstract": "This paper presents results of topic modeling and network models of topics\nusing the International Conference on Computational Science corpus, which\ncontains domain-specific (computational science) papers over sixteen years (a\ntotal of 5695 papers). We discuss topical structures of International\nConference on Computational Science, how these topics evolve over time in\nresponse to the topicality of various problems, technologies and methods, and\nhow all these topics relate to one another. This analysis illustrates\nmultidisciplinary research and collaborations among scientific communities, by\nconstructing static and dynamic networks from the topic modeling results and\nthe keywords of authors. The results of this study give insights about the past\nand future trends of core discussion topics in computational science. We used\nthe Non-negative Matrix Factorization topic modeling algorithm to discover\ntopics and labeled and grouped results hierarchically.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02394v1",
            "date": "2017-04-22 18:28:25+00:00",
            "title": "Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks",
            "authors": [
                "Jonathan Chang",
                "Stefan Scherer"
            ],
            "abstract": "Automatically assessing emotional valence in human speech has historically\nbeen a difficult task for machine learning algorithms. The subtle changes in\nthe voice of the speaker that are indicative of positive or negative emotional\nstates are often \"overshadowed\" by voice characteristics relating to emotional\nintensity or emotional activation. In this work we explore a representation\nlearning approach that automatically derives discriminative representations of\nemotional speech. In particular, we investigate two machine learning strategies\nto improve classifier performance: (1) utilization of unlabeled data using a\ndeep convolutional generative adversarial network (DCGAN), and (2) multitask\nlearning. Within our extensive experiments we leverage a multitask annotated\nemotional corpus as well as a large unlabeled meeting corpus (around 100\nhours). Our speaker-independent classification experiments show that in\nparticular the use of unlabeled data in our investigations improves performance\nof the classifiers and both fully supervised baseline approaches are\noutperformed considerably. We improve the classification of emotional valence\non a discrete 5-point scale to 43.88% and on a 3-point scale to 49.80%, which\nis competitive to state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02395v1",
            "date": "2017-04-26 20:47:36+00:00",
            "title": "On Using Active Learning and Self-Training when Mining Performance Discussions on Stack Overflow",
            "authors": [
                "Markus Borg",
                "Iben Lennerstad",
                "Rasmus Ros",
                "Elizabeth Bjarnason"
            ],
            "abstract": "Abundant data is the key to successful machine learning. However, supervised\nlearning requires annotated data that are often hard to obtain. In a\nclassification task with limited resources, Active Learning (AL) promises to\nguide annotators to examples that bring the most value for a classifier. AL can\nbe successfully combined with self-training, i.e., extending a training set\nwith the unlabelled examples for which a classifier is the most certain. We\nreport our experiences on using AL in a systematic manner to train an SVM\nclassifier for Stack Overflow posts discussing performance of software\ncomponents. We show that the training examples deemed as the most valuable to\nthe classifier are also the most difficult for humans to annotate. Despite\ncarefully evolved annotation criteria, we report low inter-rater agreement, but\nwe also propose mitigation strategies. Finally, based on one annotator's work,\nwe show that self-training can improve the classification accuracy. We conclude\nthe paper by discussing implication for future text miners aspiring to use AL\nand self-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02411v1",
            "date": "2017-05-05 22:36:04+00:00",
            "title": "Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting",
            "authors": [
                "Ming Sun",
                "Anirudh Raju",
                "George Tucker",
                "Sankaran Panchapagesan",
                "Gengshen Fu",
                "Arindam Mandal",
                "Spyros Matsoukas",
                "Nikko Strom",
                "Shiv Vitaladevuni"
            ],
            "abstract": "We propose a max-pooling based loss function for training Long Short-Term\nMemory (LSTM) networks for small-footprint keyword spotting (KWS), with low\nCPU, memory, and latency requirements. The max-pooling loss training can be\nfurther guided by initializing with a cross-entropy loss trained network. A\nposterior smoothing based evaluation approach is employed to measure keyword\nspotting performance. Our experimental results show that LSTM models trained\nusing cross-entropy loss or max-pooling loss outperform a cross-entropy loss\ntrained baseline feed-forward Deep Neural Network (DNN). In addition,\nmax-pooling loss trained LSTM with randomly initialized network performs better\ncompared to cross-entropy loss trained LSTM. Finally, the max-pooling loss\ntrained LSTM initialized with a cross-entropy pre-trained network shows the\nbest performance, which yields $67.6\\%$ relative reduction compared to baseline\nfeed-forward DNN in Area Under the Curve (AUC) measure.",
            "journal ref": "Spoken Language Technology Workshop (SLT), 2016 IEEE (pp.\n  474-480). IEEE",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.02426v2",
            "date": "2017-05-06 01:40:28+00:00",
            "title": "Analogical Inference for Multi-Relational Embeddings",
            "authors": [
                "Hanxiao Liu",
                "Yuexin Wu",
                "Yiming Yang"
            ],
            "abstract": "Large-scale multi-relational embedding refers to the task of learning the\nlatent representations for entities and relations in large knowledge graphs. An\neffective and scalable solution for this problem is crucial for the true\nsuccess of knowledge-based inference in a broad range of applications. This\npaper proposes a novel framework for optimizing the latent representations with\nrespect to the \\textit{analogical} properties of the embedded entities and\nrelations. By formulating the learning objective in a differentiable fashion,\nour model enjoys both theoretical power and computational scalability, and\nsignificantly outperformed a large number of representative baseline methods on\nbenchmark datasets. Furthermore, the model offers an elegant unification of\nseveral well-known methods in multi-relational embedding, which can be proven\nto be special instantiations of our framework.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.03151v3",
            "date": "2017-05-09 02:46:21+00:00",
            "title": "Phonetic Temporal Neural Model for Language Identification",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Yixiang Chen",
                "Lantian Li",
                "Andrew Abel"
            ],
            "abstract": "Deep neural models, particularly the LSTM-RNN model, have shown great\npotential for language identification (LID). However, the use of phonetic\ninformation has been largely overlooked by most existing neural LID methods,\nalthough this information has been used very successfully in conventional\nphonetic LID systems. We present a phonetic temporal neural model for LID,\nwhich is an LSTM-RNN LID system that accepts phonetic features produced by a\nphone-discriminative DNN as the input, rather than raw acoustic features. This\nnew model is similar to traditional phonetic LID methods, but the phonetic\nknowledge here is much richer: it is at the frame level and involves compacted\ninformation of all phones. Our experiments conducted on the Babel database and\nthe AP16-OLR database demonstrate that the temporal phonetic neural approach is\nvery effective, and significantly outperforms existing acoustic neural models.\nIt also outperforms the conventional i-vector approach on short utterances and\nin noisy conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03152v2",
            "date": "2017-05-09 02:47:22+00:00",
            "title": "Phone-aware Neural Language Identification",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Yixiang Chen",
                "Ying Shi",
                "Lantian Li"
            ],
            "abstract": "Pure acoustic neural models, particularly the LSTM-RNN model, have shown\ngreat potential in language identification (LID). However, the phonetic\ninformation has been largely overlooked by most of existing neural LID models,\nalthough this information has been used in the conventional phonetic LID\nsystems with a great success. We present a phone-aware neural LID architecture,\nwhich is a deep LSTM-RNN LID system but accepts output from an RNN-based ASR\nsystem. By utilizing the phonetic knowledge, the LID performance can be\nsignificantly improved. Interestingly, even if the test language is not\ninvolved in the ASR training, the phonetic knowledge still presents a large\ncontribution. Our experiments conducted on four languages within the Babel\ncorpus demonstrated that the phone-aware approach is highly effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03455v3",
            "date": "2017-05-08 20:57:30+00:00",
            "title": "Sequential Dialogue Context Modeling for Spoken Language Understanding",
            "authors": [
                "Ankur Bapna",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Larry Heck"
            ],
            "abstract": "Spoken Language Understanding (SLU) is a key component of goal oriented\ndialogue systems that would parse user utterances into semantic frame\nrepresentations. Traditionally SLU does not utilize the dialogue history beyond\nthe previous system turn and contextual ambiguities are resolved by the\ndownstream components. In this paper, we explore novel approaches for modeling\ndialogue context in a recurrent neural network (RNN) based language\nunderstanding system. We propose the Sequential Dialogue Encoder Network, that\nallows encoding context from the dialogue history in chronological order. We\ncompare the performance of our proposed architecture with two context models,\none that uses just the previous turn context and another that encodes dialogue\ncontext in a memory network, but loses the order of utterances in the dialogue\nhistory. Experiments with a multi-domain dialogue dataset demonstrate that the\nproposed architecture results in reduced semantic frame error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03508v1",
            "date": "2017-05-06 17:01:57+00:00",
            "title": "DeepDeath: Learning to Predict the Underlying Cause of Death with Big Data",
            "authors": [
                "Hamid Reza Hassanzadeh",
                "Ying Sha",
                "May D. Wang"
            ],
            "abstract": "Multiple cause-of-death data provides a valuable source of information that\ncan be used to enhance health standards by predicting health related\ntrajectories in societies with large populations. These data are often\navailable in large quantities across U.S. states and require Big Data\ntechniques to uncover complex hidden patterns. We design two different classes\nof models suitable for large-scale analysis of mortality data, a Hadoop-based\nensemble of random forests trained over N-grams, and the DeepDeath, a deep\nclassifier based on the recurrent neural network (RNN). We apply both classes\nto the mortality data provided by the National Center for Health Statistics and\nshow that while both perform significantly better than the random classifier,\nthe deep model that utilizes long short-term memory networks (LSTMs), surpasses\nthe N-gram based models and is capable of learning the temporal aspect of the\ndata without a need for building ad-hoc, expert-driven features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.03556v2",
            "date": "2017-05-09 22:09:01+00:00",
            "title": "Relevance-based Word Embedding",
            "authors": [
                "Hamed Zamani",
                "W. Bruce Croft"
            ],
            "abstract": "Learning a high-dimensional dense representation for vocabulary terms, also\nknown as a word embedding, has recently attracted much attention in natural\nlanguage processing and information retrieval tasks. The embedding vectors are\ntypically learned based on term proximity in a large corpus. This means that\nthe objective in well-known word embedding algorithms, e.g., word2vec, is to\naccurately predict adjacent word(s) for a given word or context. However, this\nobjective is not necessarily equivalent to the goal of many information\nretrieval (IR) tasks. The primary objective in various IR tasks is to capture\nrelevance instead of term proximity, syntactic, or even semantic similarity.\nThis is the motivation for developing unsupervised relevance-based word\nembedding models that learn word representations based on query-document\nrelevance information. In this paper, we propose two learning models with\ndifferent objective functions; one learns a relevance distribution over the\nvocabulary set for each query, and the other classifies each term as belonging\nto the relevant or non-relevant class for each query. To train our models, we\nused over six million unique queries and the top ranked documents retrieved in\nresponse to each query, which are assumed to be relevant to the query. We\nextrinsically evaluate our learned word representation models using two IR\ntasks: query expansion and query classification. Both query expansion\nexperiments on four TREC collections and query classification experiments on\nthe KDD Cup 2005 dataset suggest that the relevance-based word embedding models\nsignificantly outperform state-of-the-art proximity-based embedding models,\nsuch as word2vec and GloVe.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1705.03633v1",
            "date": "2017-05-10 07:08:23+00:00",
            "title": "Inferring and Executing Programs for Visual Reasoning",
            "authors": [
                "Justin Johnson",
                "Bharath Hariharan",
                "Laurens van der Maaten",
                "Judy Hoffman",
                "Li Fei-Fei",
                "C. Lawrence Zitnick",
                "Ross Girshick"
            ],
            "abstract": "Existing methods for visual reasoning attempt to directly map inputs to\noutputs using black-box architectures without explicitly modeling the\nunderlying reasoning processes. As a result, these black-box models often learn\nto exploit biases in the data rather than learning to perform visual reasoning.\nInspired by module networks, this paper proposes a model for visual reasoning\nthat consists of a program generator that constructs an explicit representation\nof the reasoning process to be performed, and an execution engine that executes\nthe resulting program to produce an answer. Both the program generator and the\nexecution engine are implemented by neural networks, and are trained using a\ncombination of backpropagation and REINFORCE. Using the CLEVR benchmark for\nvisual reasoning, we show that our model significantly outperforms strong\nbaselines and generalizes better in a variety of settings.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1705.03670v1",
            "date": "2017-05-10 09:30:42+00:00",
            "title": "Deep Speaker Feature Learning for Text-independent Speaker Verification",
            "authors": [
                "Lantian Li",
                "Yixiang Chen",
                "Ying Shi",
                "Zhiyuan Tang",
                "Dong Wang"
            ],
            "abstract": "Recently deep neural networks (DNNs) have been used to learn speaker\nfeatures. However, the quality of the learned features is not sufficiently\ngood, so a complex back-end model, either neural or probabilistic, has to be\nused to address the residual uncertainty when applied to speaker verification,\njust as with raw features. This paper presents a convolutional time-delay deep\nneural network structure (CT-DNN) for speaker feature learning. Our\nexperimental results on the Fisher database demonstrated that this CT-DNN can\nproduce high-quality speaker features: even with a single feature (0.3 seconds\nincluding the context), the EER can be as low as 7.68%. This effectively\nconfirmed that the speaker trait is largely a deterministic short-time property\nrather than a long-time distributional pattern, and therefore can be extracted\nfrom just dozens of frames.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1705.03865v2",
            "date": "2017-05-10 17:30:17+00:00",
            "title": "Survey of Visual Question Answering: Datasets and Techniques",
            "authors": [
                "Akshay Kumar Gupta"
            ],
            "abstract": "Visual question answering (or VQA) is a new and exciting problem that\ncombines natural language processing and computer vision techniques. We present\na survey of the various datasets and models that have been used to tackle this\ntask. The first part of the survey details the various datasets for VQA and\ncompares them along some common factors. The second part of this survey details\nthe different approaches for VQA, classified into four types: non-deep learning\nmodels, deep learning models without attention, deep learning models with\nattention, and other models which do not fit into the first three. Finally, we\ncompare the performances of these approaches and provide some directions for\nfuture work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.04146v3",
            "date": "2017-05-11 13:04:47+00:00",
            "title": "Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems",
            "authors": [
                "Wang Ling",
                "Dani Yogatama",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "Solving algebraic word problems requires executing a series of arithmetic\noperations---a program---to obtain a final answer. However, since programs can\nbe arbitrarily complicated, inducing them directly from question-answer pairs\nis a formidable challenge. To make this task more feasible, we solve these\nproblems by generating answer rationales, sequences of natural language and\nhuman-readable mathematical expressions that derive the final answer through a\nseries of small steps. Although rationales do not explicitly specify programs,\nthey provide a scaffolding for their structure via intermediate milestones. To\nevaluate our approach, we have created a new 100,000-sample dataset of\nquestions, answers and rationales. Experimental results show that indirect\nsupervision of program learning via answer rationales is a promising strategy\nfor inducing arithmetic programs.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.05487v1",
            "date": "2017-05-16 00:03:19+00:00",
            "title": "NeuroNER: an easy-to-use program for named-entity recognition based on neural networks",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee",
                "Peter Szolovits"
            ],
            "abstract": "Named-entity recognition (NER) aims at identifying entities of interest in a\ntext. Artificial neural networks (ANNs) have recently been shown to outperform\nexisting NER systems. However, ANNs remain challenging to use for non-expert\nusers. In this paper, we present NeuroNER, an easy-to-use named-entity\nrecognition tool based on ANNs. Users can annotate entities using a graphical\nweb-based user interface (BRAT): the annotations are then used to train an ANN,\nwhich in turn predict entities' locations and categories in new texts. NeuroNER\nmakes this annotation-training-prediction flow smooth and accessible to anyone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05633v2",
            "date": "2017-05-16 10:37:52+00:00",
            "title": "Social Media-based Substance Use Prediction",
            "authors": [
                "Tao Ding",
                "Warren K. Bickel",
                "Shimei Pan"
            ],
            "abstract": "In this paper, we demonstrate how the state-of-the-art machine learning and\ntext mining techniques can be used to build effective social media-based\nsubstance use detection systems. Since a substance use ground truth is\ndifficult to obtain on a large scale, to maximize system performance, we\nexplore different feature learning methods to take advantage of a large amount\nof unsupervised social media data. We also demonstrate the benefit of using\nmulti-view unsupervised feature learning to combine heterogeneous user\ninformation such as Facebook `\"likes\" and \"status updates\" to enhance system\nperformance. Based on our evaluation, our best models achieved 86% AUC for\npredicting tobacco use, 81% for alcohol use and 84% for drug use, all of which\nsignificantly outperformed existing methods. Our investigation has also\nuncovered interesting relations between a user's social media behavior (e.g.,\nword usage) and substance use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.05742v3",
            "date": "2017-05-16 14:53:02+00:00",
            "title": "Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs",
            "authors": [
                "Rakshit Trivedi",
                "Hanjun Dai",
                "Yichen Wang",
                "Le Song"
            ],
            "abstract": "The availability of large scale event data with time stamps has given rise to\ndynamically evolving knowledge graphs that contain temporal information for\neach edge. Reasoning over time in such dynamic knowledge graphs is not yet well\nunderstood. To this end, we present Know-Evolve, a novel deep evolutionary\nknowledge network that learns non-linearly evolving entity representations over\ntime. The occurrence of a fact (edge) is modeled as a multivariate point\nprocess whose intensity function is modulated by the score for that fact\ncomputed based on the learned entity embeddings. We demonstrate significantly\nimproved performance over various relational learning approaches on two large\nscale real-world datasets. Further, our method effectively predicts occurrence\nor recurrence time of a fact which is novel compared to prior reasoning\napproaches in multi-relational setting.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.05762v2",
            "date": "2017-05-16 15:23:14+00:00",
            "title": "A decentralized route to the origins of scaling in human language",
            "authors": [
                "Felipe Urbina",
                "Javier Vera"
            ],
            "abstract": "The Zipf's law establishes that if the words of a (large) text are ordered by\ndecreasing frequency, the frequency versus the rank decreases as a power law\nwith exponent close to $-1$. Previous work has stressed that this pattern\narises from a conflict of interests of the participants of communication. The\nchallenge here is to define a computational multi-agent language game, mainly\nbased on a parameter that measures the relative participant's interests.\nNumerical simulations suggest that at critical values of the parameter a\nhuman-like vocabulary, exhibiting scaling properties, seems to appear. The\nappearance of an intermediate distribution of frequencies at some critical\nvalues of the parameter suggests that on a population of artificial agents the\nemergence of scaling partly arises as a self-organized process only from local\ninteractions between agents.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1705.06273v1",
            "date": "2017-05-17 17:45:15+00:00",
            "title": "Transfer Learning for Named-Entity Recognition with Neural Networks",
            "authors": [
                "Ji Young Lee",
                "Franck Dernoncourt",
                "Peter Szolovits"
            ],
            "abstract": "Recent approaches based on artificial neural networks (ANNs) have shown\npromising results for named-entity recognition (NER). In order to achieve high\nperformances, ANNs need to be trained on a large labeled dataset. However,\nlabels might be difficult to obtain for the dataset on which the user wants to\nperform NER: label scarcity is particularly pronounced for patient note\nde-identification, which is an instance of NER. In this work, we analyze to\nwhat extent transfer learning may address this issue. In particular, we\ndemonstrate that transferring an ANN model trained on a large labeled dataset\nto another dataset with a limited number of labels improves upon the\nstate-of-the-art results on two different datasets for patient note\nde-identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.06400v2",
            "date": "2017-05-18 02:50:40+00:00",
            "title": "Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks",
            "authors": [
                "Matthias Plappert",
                "Christian Mandery",
                "Tamim Asfour"
            ],
            "abstract": "Linking human whole-body motion and natural language is of great interest for\nthe generation of semantic representations of observed human behaviors as well\nas for the generation of robot behaviors based on natural language input. While\nthere has been a large body of research in this area, most approaches that\nexist today require a symbolic representation of motions (e.g. in the form of\nmotion primitives), which have to be defined a-priori or require complex\nsegmentation algorithms. In contrast, recent advances in the field of neural\nnetworks and especially deep learning have demonstrated that sub-symbolic\nrepresentations that can be learned end-to-end usually outperform more\ntraditional approaches, for applications such as machine translation. In this\npaper we propose a generative model that learns a bidirectional mapping between\nhuman whole-body motion and natural language using deep recurrent neural\nnetworks (RNNs) and sequence-to-sequence learning. Our approach does not\nrequire any segmentation or manual feature engineering and learns a distributed\nrepresentation, which is shared for all motions and descriptions. We evaluate\nour approach on 2,846 human whole-body motions and 6,187 natural language\ndescriptions thereof from the KIT Motion-Language Dataset. Our results clearly\ndemonstrate the effectiveness of the proposed model: We show that our model\ngenerates a wide variety of realistic motions only from descriptions thereof in\nform of a single sentence. Conversely, our model is also capable of generating\ncorrect and detailed natural language descriptions from human motions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.06510v2",
            "date": "2017-05-18 10:24:03+00:00",
            "title": "Entropic selection of concepts unveils hidden topics in documents corpora",
            "authors": [
                "Andrea Martini",
                "Alessio Cardillo",
                "Paolo De Los Rios"
            ],
            "abstract": "The organization and evolution of science has recently become itself an\nobject of scientific quantitative investigation, thanks to the wealth of\ninformation that can be extracted from scientific documents, such as citations\nbetween papers and co-authorship between researchers. However, only few studies\nhave focused on the concepts that characterize full documents and that can be\nextracted and analyzed, revealing the deeper organization of scientific\nknowledge. Unfortunately, several concepts can be so common across documents\nthat they hinder the emergence of the underlying topical structure of the\ndocument corpus, because they give rise to a large amount of spurious and\ntrivial relations among documents. To identify and remove common concepts, we\nintroduce a method to gauge their relevance according to an objective\ninformation-theoretic measure related to the statistics of their occurrence\nacross the document corpus. After progressively removing concepts that,\naccording to this metric, can be considered as generic, we find that the topic\norganization displays a correspondingly more refined structure.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1705.06824v2",
            "date": "2017-05-18 22:51:44+00:00",
            "title": "Learning Convolutional Text Representations for Visual Question Answering",
            "authors": [
                "Zhengyang Wang",
                "Shuiwang Ji"
            ],
            "abstract": "Visual question answering is a recently proposed artificial intelligence task\nthat requires a deep understanding of both images and texts. In deep learning,\nimages are typically modeled through convolutional neural networks, and texts\nare typically modeled through recurrent neural networks. While the requirement\nfor modeling images is similar to traditional computer vision tasks, such as\nobject recognition and image classification, visual question answering raises a\ndifferent need for textual representation as compared to other natural language\nprocessing tasks. In this work, we perform a detailed analysis on natural\nlanguage questions in visual question answering. Based on the analysis, we\npropose to rely on convolutional neural networks for learning textual\nrepresentations. By exploring the various properties of convolutional neural\nnetworks specialized for text data, such as width and depth, we present our\n\"CNN Inception + Gate\" model. We show that our model improves question\nrepresentations and thus the overall accuracy of visual question answering\nmodels. We also show that the text representation requirement in visual\nquestion answering is more complicated and comprehensive than that in\nconventional natural language processing tasks, making it a better task to\nevaluate textual representation methods. Shallow models like fastText, which\ncan obtain comparable results with deep learning models in tasks like text\nclassification, are not suitable in visual question answering.",
            "journal ref": "In proceedings of the 2018 SIAM International Conference on Data\n  Mining (pp. 594-602). 2018",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.07136v3",
            "date": "2017-05-19 18:17:00+00:00",
            "title": "Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML",
            "authors": [
                "Xuezhe Ma",
                "Pengcheng Yin",
                "Jingzhou Liu",
                "Graham Neubig",
                "Eduard Hovy"
            ],
            "abstract": "Reward augmented maximum likelihood (RAML), a simple and effective learning\nframework to directly optimize towards the reward function in structured\nprediction tasks, has led to a number of impressive empirical successes. RAML\nincorporates task-specific reward by performing maximum-likelihood updates on\ncandidate outputs sampled according to an exponentiated payoff distribution,\nwhich gives higher probabilities to candidates that are close to the reference\noutput. While RAML is notable for its simplicity, efficiency, and its\nimpressive empirical successes, the theoretical properties of RAML, especially\nthe behavior of the exponentiated payoff distribution, has not been examined\nthoroughly. In this work, we introduce softmax Q-distribution estimation, a\nnovel theoretical interpretation of RAML, which reveals the relation between\nRAML and Bayesian decision theory. The softmax Q-distribution can be regarded\nas a smooth approximation of the Bayes decision boundary, and the Bayes\ndecision rule is achieved by decoding with this Q-distribution. We further show\nthat RAML is equivalent to approximately estimating the softmax Q-distribution,\nwith the temperature $\\tau$ controlling approximation error. We perform two\nexperiments, one on synthetic data of multi-class classification and one on\nreal data of image captioning, to demonstrate the relationship between RAML and\nthe proposed softmax Q-distribution estimation method, verifying our\ntheoretical analysis. Additional experiments on three structured prediction\ntasks with rewards defined on sequential (named entity recognition), tree-based\n(dependency parsing) and irregular (machine translation) structures show\nnotable improvements over maximum likelihood baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.07267v2",
            "date": "2017-05-20 06:53:09+00:00",
            "title": "Search Engine Guided Non-Parametric Neural Machine Translation",
            "authors": [
                "Jiatao Gu",
                "Yong Wang",
                "Kyunghyun Cho",
                "Victor O. K. Li"
            ],
            "abstract": "In this paper, we extend an attention-based neural machine translation (NMT)\nmodel by allowing it to access an entire training set of parallel sentence\npairs even after training. The proposed approach consists of two stages. In the\nfirst stage--retrieval stage--, an off-the-shelf, black-box search engine is\nused to retrieve a small subset of sentence pairs from a training set given a\nsource sentence. These pairs are further filtered based on a fuzzy matching\nscore based on edit distance. In the second stage--translation stage--, a novel\ntranslation model, called translation memory enhanced NMT (TM-NMT), seamlessly\nuses both the source sentence and a set of retrieved sentence pairs to perform\nthe translation. Empirical evaluation on three language pairs (En-Fr, En-De,\nand En-Es) shows that the proposed approach significantly outperforms the\nbaseline approach and the improvement is more significant when more relevant\nsentence pairs were retrieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07368v3",
            "date": "2017-05-20 23:45:54+00:00",
            "title": "Mixed Membership Word Embeddings for Computational Social Science",
            "authors": [
                "James Foulds"
            ],
            "abstract": "Word embeddings improve the performance of NLP systems by revealing the\nhidden structural relationships between words. Despite their success in many\napplications, word embeddings have seen very little use in computational social\nscience NLP tasks, presumably due to their reliance on big data, and to a lack\nof interpretability. I propose a probabilistic model-based word embedding\nmethod which can recover interpretable embeddings, without big data. The key\ninsight is to leverage mixed membership modeling, in which global\nrepresentations are shared, but individual entities (i.e. dictionary words) are\nfree to use these representations to uniquely differing degrees. I show how to\ntrain the model using a combination of state-of-the-art training techniques for\nword embeddings and topic models. The experimental results show an improvement\nin predictive language modeling of up to 63% in MRR over the skip-gram, and\ndemonstrate that the representations are beneficial for supervised learning. I\nillustrate the interpretability of the models with computational social science\ncase studies on State of the Union addresses and NIPS articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.07704v3",
            "date": "2017-05-22 13:11:24+00:00",
            "title": "A Regularized Framework for Sparse and Structured Neural Attention",
            "authors": [
                "Vlad Niculae",
                "Mathieu Blondel"
            ],
            "abstract": "Modern neural networks are often augmented with an attention mechanism, which\ntells the network where to focus within the input. We propose in this paper a\nnew framework for sparse and structured attention, building upon a smoothed max\noperator. We show that the gradient of this operator defines a mapping from\nreal values to probabilities, suitable as an attention mechanism. Our framework\nincludes softmax and a slight generalization of the recently-proposed sparsemax\nas special cases. However, we also show how our framework can incorporate\nmodern structured penalties, resulting in more interpretable attention\nmechanisms, that focus on entire segments or groups of an input. We derive\nefficient algorithms to compute the forward and backward passes of our\nattention mechanisms, enabling their use in a neural network trained with\nbackpropagation. To showcase their potential as a drop-in replacement for\nexisting ones, we evaluate our attention mechanisms on three large-scale tasks:\ntextual entailment, machine translation, and sentence summarization. Our\nattention mechanisms improve interpretability without sacrificing performance;\nnotably, on textual entailment and summarization, we outperform the standard\nattention mechanisms based on softmax and sparsemax.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1705.07860v1",
            "date": "2017-05-22 17:04:56+00:00",
            "title": "On-the-fly Operation Batching in Dynamic Computation Graphs",
            "authors": [
                "Graham Neubig",
                "Yoav Goldberg",
                "Chris Dyer"
            ],
            "abstract": "Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer\nmore flexibility for implementing models that cope with data of varying\ndimensions and structure, relative to toolkits that operate on statically\ndeclared computations (e.g., TensorFlow, CNTK, and Theano). However, existing\ntoolkits - both static and dynamic - require that the developer organize the\ncomputations into the batches necessary for exploiting high-performance\nalgorithms and hardware. This batching task is generally difficult, but it\nbecomes a major hurdle as architectures become complex. In this paper, we\npresent an algorithm, and its implementation in the DyNet toolkit, for\nautomatically batching operations. Developers simply write minibatch\ncomputations as aggregations of single instance computations, and the batching\nalgorithm seamlessly executes them, on the fly, using computationally efficient\nbatched operations. On a variety of tasks, we obtain throughput similar to that\nobtained with manual batches, as well as comparable speedups over\nsingle-instance learning on architectures that are impractical to batch\nmanually.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.08386v2",
            "date": "2017-05-23 16:06:32+00:00",
            "title": "Better Text Understanding Through Image-To-Text Transfer",
            "authors": [
                "Karol Kurach",
                "Sylvain Gelly",
                "Michal Jastrzebski",
                "Philip Haeusser",
                "Olivier Teytaud",
                "Damien Vincent",
                "Olivier Bousquet"
            ],
            "abstract": "Generic text embeddings are successfully used in a variety of tasks. However,\nthey are often learnt by capturing the co-occurrence structure from pure text\ncorpora, resulting in limitations of their ability to generalize. In this\npaper, we explore models that incorporate visual information into the text\nrepresentation. Based on comprehensive ablation studies, we propose a\nconceptually simple, yet well performing architecture. It outperforms previous\nmultimodal approaches on a set of well established benchmarks. We also improve\nthe state-of-the-art results for image-related text datasets, using orders of\nmagnitude less data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.08557v1",
            "date": "2017-05-23 23:17:49+00:00",
            "title": "Grounded Recurrent Neural Networks",
            "authors": [
                "Ankit Vani",
                "Yacine Jernite",
                "David Sontag"
            ],
            "abstract": "In this work, we present the Grounded Recurrent Neural Network (GRNN), a\nrecurrent neural network architecture for multi-label prediction which\nexplicitly ties labels to specific dimensions of the recurrent hidden state (we\ncall this process \"grounding\"). The approach is particularly well-suited for\nextracting large numbers of concepts from text. We apply the new model to\naddress an important problem in healthcare of understanding what medical\nconcepts are discussed in clinical text. Using a publicly available dataset\nderived from Intensive Care Units, we learn to label a patient's diagnoses and\nprocedures from their discharge summary. Our evaluation shows a clear advantage\nto using our proposed architecture over a variety of strong baselines.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1705.08992v2",
            "date": "2017-05-24 22:53:56+00:00",
            "title": "Matroids Hitting Sets and Unsupervised Dependency Grammar Induction",
            "authors": [
                "Nicholas Harvey",
                "Vahab Mirrokni",
                "David Karger",
                "Virginia Savova",
                "Leonid Peshkin"
            ],
            "abstract": "This paper formulates a novel problem on graphs: find the minimal subset of\nedges in a fully connected graph, such that the resulting graph contains all\nspanning trees for a set of specifed sub-graphs. This formulation is motivated\nby an un-supervised grammar induction problem from computational linguistics.\nWe present a reduction to some known problems and algorithms from graph theory,\nprovide computational complexity results, and describe an approximation\nalgorithm.",
            "journal ref": "None",
            "category": "cs.DM"
        },
        {
            "url": "http://arxiv.org/abs/1705.09037v3",
            "date": "2017-05-25 03:58:10+00:00",
            "title": "Deriving Neural Architectures from Sequence and Graph Kernels",
            "authors": [
                "Tao Lei",
                "Wengong Jin",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "The design of neural architectures for structured objects is typically guided\nby experimental insights rather than a formal process. In this work, we appeal\nto kernels over combinatorial structures, such as sequences and graphs, to\nderive appropriate neural operations. We introduce a class of deep recurrent\nneural operations and formally characterize their associated kernel spaces. Our\nrecurrent modules compare the input to virtual reference objects (cf. filters\nin CNN) via the kernels. Similar to traditional neural operations, these\nreference objects are parameterized and directly optimized in end-to-end\ntraining. We empirically evaluate the proposed class of neural architectures on\nstandard applications such as language modeling and molecular graph regression,\nachieving state-of-the-art results across these applications.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1705.09515v1",
            "date": "2017-05-26 10:34:24+00:00",
            "title": "ASR error management for improving spoken language understanding",
            "authors": [
                "Edwin Simonnet",
                "Sahar Ghannay",
                "Nathalie Camelin",
                "Yannick Est\u00e8ve",
                "Renato De Mori"
            ],
            "abstract": "This paper addresses the problem of automatic speech recognition (ASR) error\ndetection and their use for improving spoken language understanding (SLU)\nsystems. In this study, the SLU task consists in automatically extracting, from\nASR transcriptions , semantic concepts and concept/values pairs in a e.g\ntouristic information system. An approach is proposed for enriching the set of\nsemantic labels with error specific labels and by using a recently proposed\nneural approach based on word embeddings to compute well calibrated ASR\nconfidence measures. Experimental results are reported showing that it is\npossible to decrease significantly the Concept/Value Error Rate with a state of\nthe art system, outperforming previously published results performance on the\nsame experimental data. It also shown that combining an SLU approach based on\nconditional random fields with a neural encoder/decoder attention based\narchitecture , it is possible to effectively identifying confidence islands and\nuncertain semantic output segments useful for deciding appropriate error\nhandling actions by the dialogue manager strategy .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09656v1",
            "date": "2017-05-26 17:40:58+00:00",
            "title": "Helping News Editors Write Better Headlines: A Recommender to Improve the Keyword Contents & Shareability of News Headlines",
            "authors": [
                "Terrence Szymanski",
                "Claudia Orellana-Rodriguez",
                "Mark T. Keane"
            ],
            "abstract": "We present a software tool that employs state-of-the-art natural language\nprocessing (NLP) and machine learning techniques to help newspaper editors\ncompose effective headlines for online publication. The system identifies the\nmost salient keywords in a news article and ranks them based on both their\noverall popularity and their direct relevance to the article. The system also\nuses a supervised regression model to identify headlines that are likely to be\nwidely shared on social media. The user interface is designed to simplify and\nspeed the editor's decision process on the composition of the headline. As\nsuch, the tool provides an efficient way to combine the benefits of automated\npredictors of engagement and search-engine optimization (SEO) with human\njudgments of overall headline quality.",
            "journal ref": "Natural Language Processing meets Journalism. IJCAI-16 workshop.\n  Pages 30-34. 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.09665v1",
            "date": "2017-05-26 18:00:02+00:00",
            "title": "Community Identity and User Engagement in a Multi-Community Landscape",
            "authors": [
                "Justine Zhang",
                "William L. Hamilton",
                "Cristian Danescu-Niculescu-Mizil",
                "Dan Jurafsky",
                "Jure Leskovec"
            ],
            "abstract": "A community's identity defines and shapes its internal dynamics. Our current\nunderstanding of this interplay is mostly limited to glimpses gathered from\nisolated studies of individual communities. In this work we provide a\nsystematic exploration of the nature of this relation across a wide variety of\nonline communities. To this end we introduce a quantitative, language-based\ntypology reflecting two key aspects of a community's identity: how distinctive,\nand how temporally dynamic it is. By mapping almost 300 Reddit communities into\nthe landscape induced by this typology, we reveal regularities in how patterns\nof user engagement vary with the characteristics of a community.\n  Our results suggest that the way new and existing users engage with a\ncommunity depends strongly and systematically on the nature of the collective\nidentity it fosters, in ways that are highly consequential to community\nmaintainers. For example, communities with distinctive and highly dynamic\nidentities are more likely to retain their users. However, such niche\ncommunities also exhibit much larger acculturation gaps between existing users\nand newcomers, which potentially hinder the integration of the latter.\n  More generally, our methodology reveals differences in how various social\nphenomena manifest across communities, and shows that structuring the\nmulti-community landscape can lead to a better understanding of the systematic\nnature of this diversity.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1705.09731v3",
            "date": "2017-05-26 22:18:39+00:00",
            "title": "Multiplex model of mental lexicon reveals explosive learning in humans",
            "authors": [
                "Massimo Stella",
                "Nicole M. Beckage",
                "Markus Brede",
                "Manlio De Domenico"
            ],
            "abstract": "Word similarities affect language acquisition and use in a multi-relational\nway barely accounted for in the literature. We propose a multiplex network\nrepresentation of this mental lexicon of word similarities as a natural\nframework for investigating large-scale cognitive patterns. Our representation\naccounts for semantic, taxonomic, and phonological interactions and it\nidentifies a cluster of words which are used with greater frequency, are\nidentified, memorised, and learned more easily, and have more meanings than\nexpected at random. This cluster emerges around age 7 through an explosive\ntransition not reproduced by null models. We relate this explosive emergence to\npolysemy -- redundancy in word meanings. Results indicate that the word cluster\nacts as a core for the lexicon, increasing both lexical navigability and\nrobustness to linguistic degradation. Our findings provide quantitative\nconfirmation of existing conjectures about core structure in the mental lexicon\nand the importance of integrating multi-relational word-word interactions in\npsycholinguistic frameworks.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1705.09932v3",
            "date": "2017-05-28 12:19:03+00:00",
            "title": "The placement of the head that maximizes predictability. An information theoretic approach",
            "authors": [
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "The minimization of the length of syntactic dependencies is a\nwell-established principle of word order and the basis of a mathematical theory\nof word order. Here we complete that theory from the perspective of information\ntheory, adding a competing word order principle: the maximization of\npredictability of a target element. These two principles are in conflict: to\nmaximize the predictability of the head, the head should appear last, which\nmaximizes the costs with respect to dependency length minimization. The\nimplications of such a broad theoretical framework to understand the\noptimality, diversity and evolution of the six possible orderings of subject,\nobject and verb are reviewed.",
            "journal ref": "Glottometrics 39, 38-71",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10209v1",
            "date": "2017-05-29 14:24:08+00:00",
            "title": "On Multilingual Training of Neural Dependency Parsers",
            "authors": [
                "Micha\u0142 Zapotoczny",
                "Pawe\u0142 Rychlikowski",
                "Jan Chorowski"
            ],
            "abstract": "We show that a recently proposed neural dependency parser can be improved by\njoint training on multiple languages from the same family. The parser is\nimplemented as a deep neural network whose only input is orthographic\nrepresentations of words. In order to successfully parse, the network has to\ndiscover how linguistically relevant concepts can be inferred from word\nspellings. We analyze the representations of characters and words that are\nlearned by the network to establish which properties of languages were\naccounted for. In particular we show that the parser has approximately learned\nto associate Latin characters with their Cyrillic counterparts and that it can\ngroup Polish and Russian words that have a similar grammatical function.\nFinally, we evaluate the parser on selected languages from the Universal\nDependencies dataset and show that it is competitive with other recently\nproposed state-of-the art methods, while having a simple structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10229v1",
            "date": "2017-05-29 15:01:44+00:00",
            "title": "Latent Intention Dialogue Models",
            "authors": [
                "Tsung-Hsien Wen",
                "Yishu Miao",
                "Phil Blunsom",
                "Steve Young"
            ],
            "abstract": "Developing a dialogue agent that is capable of making autonomous decisions\nand communicating by natural language is one of the long-term goals of machine\nlearning research. Traditional approaches either rely on hand-crafting a small\nstate-action set for applying reinforcement learning that is not scalable or\nconstructing deterministic models for learning dialogue sentences that fail to\ncapture natural conversational variability. In this paper, we propose a Latent\nIntention Dialogue Model (LIDM) that employs a discrete latent variable to\nlearn underlying dialogue intentions in the framework of neural variational\ninference. In a goal-oriented dialogue scenario, these latent intentions can be\ninterpreted as actions guiding the generation of machine responses, which can\nbe further refined autonomously by reinforcement learning. The experimental\nevaluation of LIDM shows that the model out-performs published benchmarks for\nboth corpus-based and human evaluation, demonstrating the effectiveness of\ndiscrete latent variable models for learning goal-oriented dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.10874v3",
            "date": "2017-05-30 21:31:25+00:00",
            "title": "Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments",
            "authors": [
                "Zixing Zhang",
                "J\u00fcrgen Geiger",
                "Jouni Pohjalainen",
                "Amr El-Desoky Mousa",
                "Wenyu Jin",
                "Bj\u00f6rn Schuller"
            ],
            "abstract": "Eliminating the negative effect of non-stationary environmental noise is a\nlong-standing research topic for automatic speech recognition that stills\nremains an important challenge. Data-driven supervised approaches, including\nones based on deep neural networks, have recently emerged as potential\nalternatives to traditional unsupervised approaches and with sufficient\ntraining, can alleviate the shortcomings of the unsupervised methods in various\nreal-life acoustic environments. In this light, we review recently developed,\nrepresentative deep learning approaches for tackling non-stationary additive\nand convolutional degradation of speech with the aim of providing guidelines\nfor those involved in the development of environmentally robust speech\nrecognition systems. We separately discuss single- and multi-channel techniques\ndeveloped for the front-end and back-end of speech recognition systems, as well\nas joint front-end and back-end training frameworks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1705.10929v1",
            "date": "2017-05-31 03:06:39+00:00",
            "title": "Adversarial Generation of Natural Language",
            "authors": [
                "Sai Rajeswar",
                "Sandeep Subramanian",
                "Francis Dutil",
                "Christopher Pal",
                "Aaron Courville"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have gathered a lot of attention from\nthe computer vision community, yielding impressive results for image\ngeneration. Advances in the adversarial generation of natural language from\nnoise however are not commensurate with the progress made in generating images,\nand still lag far behind likelihood based methods. In this paper, we take a\nstep towards generating natural language with a GAN objective alone. We\nintroduce a simple baseline that addresses the discrete output space problem\nwithout relying on gradient estimators and show that it is able to achieve\nstate-of-the-art results on a Chinese poem generation dataset. We present\nquantitative results on generating sentences from context-free and\nprobabilistic context-free grammars, and qualitative language modeling results.\nA conditional version is also described that can generate sequences conditioned\non sentence characteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1705.11122v3",
            "date": "2017-05-31 14:57:33+00:00",
            "title": "Controllable Invariance through Adversarial Feature Learning",
            "authors": [
                "Qizhe Xie",
                "Zihang Dai",
                "Yulun Du",
                "Eduard Hovy",
                "Graham Neubig"
            ],
            "abstract": "Learning meaningful representations that maintain the content necessary for a\nparticular task while filtering away detrimental variations is a problem of\ngreat interest in machine learning. In this paper, we tackle the problem of\nlearning representations invariant to a specific factor or trait of data. The\nrepresentation learning process is formulated as an adversarial minimax game.\nWe analyze the optimal equilibrium of such a game and find that it amounts to\nmaximizing the uncertainty of inferring the detrimental factor given the\nrepresentation while maximizing the certainty of making task-specific\npredictions. On three benchmark tasks, namely fair and bias-free\nclassification, language-independent generation, and lighting-independent image\nclassification, we show that the proposed framework induces an invariant\nrepresentation, and leads to better generalization evidenced by the improved\nperformance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.11192v2",
            "date": "2017-05-31 17:47:55+00:00",
            "title": "Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols",
            "authors": [
                "Serhii Havrylov",
                "Ivan Titov"
            ],
            "abstract": "Learning to communicate through interaction, rather than relying on explicit\nsupervision, is often considered a prerequisite for developing a general AI. We\nstudy a setting where two agents engage in playing a referential game and, from\nscratch, develop a communication protocol necessary to succeed in this game.\nUnlike previous work, we require that messages they exchange, both at train and\ntest time, are in the form of a language (i.e. sequences of discrete symbols).\nWe compare a reinforcement learning approach and one using a differentiable\nrelaxation (straight-through Gumbel-softmax estimator) and observe that the\nlatter is much faster to converge and it results in more effective protocols.\nInterestingly, we also observe that the protocol we induce by optimizing the\ncommunication success exhibits a degree of compositionality and variability\n(i.e. the same information can be phrased in different ways), both properties\ncharacteristic of natural languages. As the ultimate goal is to ensure that\ncommunication is accomplished in natural language, we also perform experiments\nwhere we inject prior information about natural language into our model and\nstudy properties of the resulting protocol.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.00149v1",
            "date": "2017-09-01 04:09:42+00:00",
            "title": "Learning what to read: Focused machine reading",
            "authors": [
                "Enrique Noriega-Atala",
                "Marco A. Valenzuela-Escarcega",
                "Clayton T. Morrison",
                "Mihai Surdeanu"
            ],
            "abstract": "Recent efforts in bioinformatics have achieved tremendous progress in the\nmachine reading of biomedical literature, and the assembly of the extracted\nbiochemical interactions into large-scale models such as protein signaling\npathways. However, batch machine reading of literature at today's scale (PubMed\nalone indexes over 1 million papers per year) is unfeasible due to both cost\nand processing overhead. In this work, we introduce a focused reading approach\nto guide the machine reading of biomedical literature towards what literature\nshould be read to answer a biomedical query as efficiently as possible. We\nintroduce a family of algorithms for focused reading, including an intuitive,\nstrong baseline, and a second approach which uses a reinforcement learning (RL)\nframework that learns when to explore (widen the search) or exploit (narrow\nit). We demonstrate that the RL approach is capable of answering more queries\nthan the baseline, while being more efficient, i.e., reading fewer documents.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.00155v1",
            "date": "2017-09-01 04:46:10+00:00",
            "title": "Order-Planning Neural Text Generation From Structured Data",
            "authors": [
                "Lei Sha",
                "Lili Mou",
                "Tianyu Liu",
                "Pascal Poupart",
                "Sujian Li",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "Generating texts from structured data (e.g., a table) is important for\nvarious natural language processing tasks such as question answering and dialog\nsystems. In recent studies, researchers use neural language models and\nencoder-decoder frameworks for table-to-text generation. However, these neural\nnetwork-based approaches do not model the order of contents during text\ngeneration. When a human writes a summary based on a given table, he or she\nwould probably consider the content order before wording. In a biography, for\nexample, the nationality of a person is typically mentioned before occupation\nin a biography. In this paper, we propose an order-planning text generation\nmodel to capture the relationship between different fields and use such\nrelationship to make the generated text more fluent and smooth. We conducted\nexperiments on the WikiBio dataset and achieve significantly higher performance\nthan previous methods in terms of BLEU, ROUGE, and NIST scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00345v4",
            "date": "2017-09-01 14:38:21+00:00",
            "title": "Making \"fetch\" happen: The influence of social and linguistic context on nonstandard word growth and decline",
            "authors": [
                "Ian Stewart",
                "Jacob Eisenstein"
            ],
            "abstract": "In an online community, new words come and go: today's \"haha\" may be replaced\nby tomorrow's \"lol.\" Changes in online writing are usually studied as a social\nprocess, with innovations diffusing through a network of individuals in a\nspeech community. But unlike other types of innovation, language change is\nshaped and constrained by the system in which it takes part. To investigate the\nlinks between social and structural factors in language change, we undertake a\nlarge-scale analysis of nonstandard word growth in the online community Reddit.\nWe find that dissemination across many linguistic contexts is a sign of growth:\nwords that appear in more linguistic contexts grow faster and survive longer.\nWe also find that social dissemination likely plays a less important role in\nexplaining word growth and decline than previously hypothesized.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00387v1",
            "date": "2017-08-28 14:20:02+00:00",
            "title": "MIT-QCRI Arabic Dialect Identification System for the 2017 Multi-Genre Broadcast Challenge",
            "authors": [
                "Suwon Shon",
                "Ahmed Ali",
                "James Glass"
            ],
            "abstract": "In order to successfully annotate the Arabic speech con- tent found in\nopen-domain media broadcasts, it is essential to be able to process a diverse\nset of Arabic dialects. For the 2017 Multi-Genre Broadcast challenge (MGB-3)\nthere were two possible tasks: Arabic speech recognition, and Arabic Dialect\nIdentification (ADI). In this paper, we describe our efforts to create an ADI\nsystem for the MGB-3 challenge, with the goal of distinguishing amongst four\nmajor Arabic dialects, as well as Modern Standard Arabic. Our research fo-\ncused on dialect variability and domain mismatches between the training and\ntest domain. In order to achieve a robust ADI system, we explored both Siamese\nneural network models to learn similarity and dissimilarities among Arabic\ndialects, as well as i-vector post-processing to adapt domain mismatches. Both\nAcoustic and linguistic features were used for the final MGB-3 submissions,\nwith the best primary system achieving 75% accuracy on the official 10hr test\nset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00575v1",
            "date": "2017-09-02 13:13:06+00:00",
            "title": "Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection",
            "authors": [
                "Marek Rei",
                "Luana Bulat",
                "Douwe Kiela",
                "Ekaterina Shutova"
            ],
            "abstract": "The ubiquity of metaphor in our everyday communication makes it an important\nproblem for natural language understanding. Yet, the majority of metaphor\nprocessing systems to date rely on hand-engineered features and there is still\nno consensus in the field as to which features are optimal for this task. In\nthis paper, we present the first deep learning architecture designed to capture\nmetaphorical composition. Our results demonstrate that it outperforms the\nexisting approaches in the metaphor identification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.00770v1",
            "date": "2017-09-03 21:38:56+00:00",
            "title": "Understanding the Logical and Semantic Structure of Large Documents",
            "authors": [
                "Muhammad Mahbubur Rahman",
                "Tim Finin"
            ],
            "abstract": "Current language understanding approaches focus on small documents, such as\nnewswire articles, blog posts, product reviews and discussion forum entries.\nUnderstanding and extracting information from large documents like legal\nbriefs, proposals, technical manuals and research articles is still a\nchallenging task. We describe a framework that can analyze a large document and\nhelp people to know where a particular information is in that document. We aim\nto automatically identify and classify semantic sections of documents and\nassign consistent and human-understandable labels to similar sections across\ndocuments. A key contribution of our research is modeling the logical and\nsemantic structure of an electronic document. We apply machine learning\ntechniques, including deep learning, in our prototype system. We also make\navailable a dataset of information about a collection of scholarly articles\nfrom the arXiv eprints collection that includes a wide range of metadata for\neach article, including a table of contents, section labels, section\nsummarizations and more. We hope that this dataset will be a useful resource\nfor the machine learning and NLP communities in information retrieval,\ncontent-based question answering and language modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.01144v2",
            "date": "2017-08-29 10:03:05+00:00",
            "title": "Information Theoretic Analysis of DNN-HMM Acoustic Modeling",
            "authors": [
                "Pranay Dighe",
                "Afsaneh Asaei",
                "Herv\u00e9 Bourlard"
            ],
            "abstract": "We propose an information theoretic framework for quantitative assessment of\nacoustic modeling for hidden Markov model (HMM) based automatic speech\nrecognition (ASR). Acoustic modeling yields the probabilities of HMM sub-word\nstates for a short temporal window of speech acoustic features. We cast ASR as\na communication channel where the input sub-word probabilities convey the\ninformation about the output HMM state sequence. The quality of the acoustic\nmodel is thus quantified in terms of the information transmitted through this\nchannel. The process of inferring the most likely HMM state sequence from the\nsub-word probabilities is known as decoding. HMM based decoding assumes that an\nacoustic model yields accurate state-level probabilities and the data\ndistribution given the underlying hidden state is independent of any other\nstate in the sequence. We quantify 1) the acoustic model accuracy and 2) its\nrobustness to mismatch between data and the HMM conditional independence\nassumption in terms of some mutual information quantities. In this context,\nexploiting deep neural network (DNN) posterior probabilities leads to a simple\nand straightforward analysis framework to assess shortcomings of the acoustic\nmodel for HMM based decoding. This analysis enables us to evaluate the Gaussian\nmixture acoustic model (GMM) and the importance of many hidden layers in DNNs\nwithout any need of explicit speech recognition. In addition, it sheds light on\nthe contribution of low-dimensional models to enhance acoustic modeling for\nbetter compliance with the HMM based decoding requirements.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1709.01572v3",
            "date": "2017-09-05 19:53:35+00:00",
            "title": "Sequence Prediction with Neural Segmental Models",
            "authors": [
                "Hao Tang"
            ],
            "abstract": "Segments that span contiguous parts of inputs, such as phonemes in speech,\nnamed entities in sentences, actions in videos, occur frequently in sequence\nprediction problems. Segmental models, a class of models that explicitly\nhypothesizes segments, have allowed the exploration of rich segment features\nfor sequence prediction. However, segmental models suffer from slow decoding,\nhampering the use of computationally expensive features.\n  In this thesis, we introduce discriminative segmental cascades, a multi-pass\ninference framework that allows us to improve accuracy by adding higher-order\nfeatures and neural segmental features while maintaining efficiency. We also\nshow that instead of including more features to obtain better accuracy,\nsegmental cascades can be used to speed up training and decoding.\n  Segmental models, similarly to conventional speech recognizers, are typically\ntrained in multiple stages. In the first stage, a frame classifier is trained\nwith manual alignments, and then in the second stage, segmental models are\ntrained with manual alignments and the out- puts of the frame classifier.\nHowever, obtaining manual alignments are time-consuming and expensive. We\nexplore end-to-end training for segmental models with various loss functions,\nand show how end-to-end training with marginal log loss can eliminate the need\nfor detailed manual alignments.\n  We draw the connections between the marginal log loss and a popular\nend-to-end training approach called connectionist temporal classification. We\npresent a unifying framework for various end-to-end graph search-based models,\nsuch as hidden Markov models, connectionist temporal classification, and\nsegmental models. Finally, we discuss possible extensions of segmental models\nto large-vocabulary sequence prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.02076v1",
            "date": "2017-09-07 05:39:00+00:00",
            "title": "Composition by Conversation",
            "authors": [
                "Donya Quick",
                "Clayton T. Morrison"
            ],
            "abstract": "Most musical programming languages are developed purely for coding virtual\ninstruments or algorithmic compositions. Although there has been some work in\nthe domain of musical query languages for music information retrieval, there\nhas been little attempt to unify the principles of musical programming and\nquery languages with cognitive and natural language processing models that\nwould facilitate the activity of composition by conversation. We present a\nprototype framework, called MusECI, that merges these domains, permitting\nscore-level algorithmic composition in a text editor while also supporting\nconnectivity to existing natural language processing frameworks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1709.02968v1",
            "date": "2017-09-09 16:01:08+00:00",
            "title": "Matrix and Graph Operations for Relationship Inference: An Illustration with the Kinship Inference in the China Biographical Database",
            "authors": [
                "Chao-Lin Liu",
                "Hongsu Wang"
            ],
            "abstract": "Biographical databases contain diverse information about individuals. Person\nnames, birth information, career, friends, family and special achievements are\nsome possible items in the record for an individual. The relationships between\nindividuals, such as kinship and friendship, provide invaluable insights about\nhidden communities which are not directly recorded in databases. We show that\nsome simple matrix and graph-based operations are effective for inferring\nrelationships among individuals, and illustrate the main ideas with the China\nBiographical Database (CBDB).",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03406v1",
            "date": "2017-09-11 14:30:35+00:00",
            "title": "Social Media Text Processing and Semantic Analysis for Smart Cities",
            "authors": [
                "Jo\u00e3o Filipe Figueiredo Pereira"
            ],
            "abstract": "With the rise of Social Media, people obtain and share information almost\ninstantly on a 24/7 basis. Many research areas have tried to gain valuable\ninsights from these large volumes of freely available user generated content.\n  With the goal of extracting knowledge from social media streams that might be\nuseful in the context of intelligent transportation systems and smart cities,\nwe designed and developed a framework that provides functionalities for\nparallel collection of geo-located tweets from multiple pre-defined bounding\nboxes (cities or regions), including filtering of non-complying tweets, text\npre-processing for Portuguese and English language, topic modeling, and\ntransportation-specific text classifiers, as well as, aggregation and data\nvisualization.\n  We performed an exploratory data analysis of geo-located tweets in 5\ndifferent cities: Rio de Janeiro, S\\~ao Paulo, New York City, London and\nMelbourne, comprising a total of more than 43 million tweets in a period of 3\nmonths. Furthermore, we performed a large scale topic modelling comparison\nbetween Rio de Janeiro and S\\~ao Paulo. Interestingly, most of the topics are\nshared between both cities which despite being in the same country are\nconsidered very different regarding population, economy and lifestyle.\n  We take advantage of recent developments in word embeddings and train such\nrepresentations from the collections of geo-located tweets. We then use a\ncombination of bag-of-embeddings and traditional bag-of-words to train\ntravel-related classifiers in both Portuguese and English to filter\ntravel-related content from non-related. We created specific gold-standard data\nto perform empirical evaluation of the resulting classifiers. Results are in\nline with research work in other application areas by showing the robustness of\nusing word embeddings to learn word similarities that bag-of-words is not able\nto capture.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1709.03980v3",
            "date": "2017-09-12 13:38:11+00:00",
            "title": "Refining Source Representations with Relation Networks for Neural Machine Translation",
            "authors": [
                "Wen Zhang",
                "Jiawei Hu",
                "Yang Feng",
                "Qun Liu"
            ],
            "abstract": "Although neural machine translation (NMT) with the encoder-decoder framework\nhas achieved great success in recent times, it still suffers from some\ndrawbacks: RNNs tend to forget old information which is often useful and the\nencoder only operates through words without considering word relationship. To\nsolve these problems, we introduce a relation networks (RN) into NMT to refine\nthe encoding representations of the source. In our method, the RN first\naugments the representation of each source word with its neighbors and reasons\nall the possible pairwise relations between them. Then the source\nrepresentations and all the relations are fed to the attention module and the\ndecoder together, keeping the main encoder-decoder architecture unchanged.\nExperiments on two Chinese-to-English data sets in different scales both show\nthat our method can outperform the competitive baselines significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04071v5",
            "date": "2017-09-12 22:27:34+00:00",
            "title": "Variational Reasoning for Question Answering with Knowledge Graph",
            "authors": [
                "Yuyu Zhang",
                "Hanjun Dai",
                "Zornitsa Kozareva",
                "Alexander J. Smola",
                "Le Song"
            ],
            "abstract": "Knowledge graph (KG) is known to be helpful for the task of question\nanswering (QA), since it provides well-structured relational information\nbetween entities, and allows one to further infer indirect facts. However, it\nis challenging to build QA systems which can learn to reason over knowledge\ngraphs based on question-answer pairs alone. First, when people ask questions,\ntheir expressions are noisy (for example, typos in texts, or variations in\npronunciations), which is non-trivial for the QA system to match those\nmentioned entities to the knowledge graph. Second, many questions require\nmulti-hop logic reasoning over the knowledge graph to retrieve the answers. To\naddress these challenges, we propose a novel and unified deep learning\narchitecture, and an end-to-end variational learning algorithm which can handle\nnoise in questions, and learn multi-hop reasoning simultaneously. Our method\nachieves state-of-the-art performance on a recent benchmark dataset in the\nliterature. We also derive a series of new benchmark datasets, including\nquestions for multi-hop reasoning, questions paraphrased by neural translation\nmodel, and questions in human voice. Our method yields very promising results\non all these challenging datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.04380v2",
            "date": "2017-09-13 15:26:50+00:00",
            "title": "Neural Network Based Nonlinear Weighted Finite Automata",
            "authors": [
                "Tianyu Li",
                "Guillaume Rabusseau",
                "Doina Precup"
            ],
            "abstract": "Weighted finite automata (WFA) can expressively model functions defined over\nstrings but are inherently linear models. Given the recent successes of\nnonlinear models in machine learning, it is natural to wonder whether\nex-tending WFA to the nonlinear setting would be beneficial. In this paper, we\npropose a novel model of neural network based nonlinearWFA model (NL-WFA) along\nwith a learning algorithm. Our learning algorithm is inspired by the spectral\nlearning algorithm for WFAand relies on a nonlinear decomposition of the\nso-called Hankel matrix, by means of an auto-encoder network. The expressive\npower of NL-WFA and the proposed learning algorithm are assessed on both\nsynthetic and real-world data, showing that NL-WFA can lead to smaller model\nsizes and infer complex grammatical structures from data.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1709.04482v1",
            "date": "2017-09-13 18:02:53+00:00",
            "title": "Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems",
            "authors": [
                "Yonatan Belinkov",
                "James Glass"
            ],
            "abstract": "Neural models have become ubiquitous in automatic speech recognition systems.\nWhile neural networks are typically used as acoustic models in more complex\nsystems, recent studies have explored end-to-end speech recognition systems\nbased on neural networks, which can be trained to directly predict text from\ninput acoustic features. Although such systems are conceptually elegant and\nsimpler than traditional systems, it is less obvious how to interpret the\ntrained models. In this work, we analyze the speech representations learned by\na deep end-to-end model that is based on convolutional and recurrent layers,\nand trained with a connectionist temporal classification (CTC) loss. We use a\npre-trained model to generate frame-level features which are given to a\nclassifier that is trained on frame classification into phones. We evaluate\nrepresentations from different layers of the deep model and compare their\nquality for predicting phone labels. Our experiments shed light on important\naspects of the end-to-end model such as layer depth, model complexity, and\nother design choices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.05027v7",
            "date": "2017-09-15 01:10:23+00:00",
            "title": "Learning Intrinsic Sparse Structures within Long Short-Term Memory",
            "authors": [
                "Wei Wen",
                "Yuxiong He",
                "Samyam Rajbhandari",
                "Minjia Zhang",
                "Wenhan Wang",
                "Fang Liu",
                "Bin Hu",
                "Yiran Chen",
                "Hai Li"
            ],
            "abstract": "Model compression is significant for the wide adoption of Recurrent Neural\nNetworks (RNNs) in both user devices possessing limited resources and business\nclusters requiring quick responses to large-scale service requests. This work\naims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the\nsizes of basic structures within LSTM units, including input updates, gates,\nhidden states, cell states and outputs. Independently reducing the sizes of\nbasic structures can result in inconsistent dimensions among them, and\nconsequently, end up with invalid LSTM units. To overcome the problem, we\npropose Intrinsic Sparse Structures (ISS) in LSTMs. Removing a component of ISS\nwill simultaneously decrease the sizes of all basic structures by one and\nthereby always maintain the dimension consistency. By learning ISS within LSTM\nunits, the obtained LSTMs remain regular while having much smaller basic\nstructures. Based on group Lasso regularization, our method achieves 10.59x\nspeedup without losing any perplexity of a language modeling of Penn TreeBank\ndataset. It is also successfully evaluated through a compact model with only\n2.69M weights for machine Question Answering of SQuAD dataset. Our approach is\nsuccessfully extended to non- LSTM RNNs, like Recurrent Highway Networks\n(RHNs). Our source code is publicly available at\nhttps://github.com/wenwei202/iss-rnns",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.05038v1",
            "date": "2017-09-15 02:53:16+00:00",
            "title": "Self-Guiding Multimodal LSTM - when we do not have a perfect training dataset for image captioning",
            "authors": [
                "Yang Xian",
                "Yingli Tian"
            ],
            "abstract": "In this paper, a self-guiding multimodal LSTM (sg-LSTM) image captioning\nmodel is proposed to handle uncontrolled imbalanced real-world image-sentence\ndataset. We collect FlickrNYC dataset from Flickr as our testbed with 306,165\nimages and the original text descriptions uploaded by the users are utilized as\nthe ground truth for training. Descriptions in FlickrNYC dataset vary\ndramatically ranging from short term-descriptions to long\nparagraph-descriptions and can describe any visual aspects, or even refer to\nobjects that are not depicted. To deal with the imbalanced and noisy situation\nand to fully explore the dataset itself, we propose a novel guiding textual\nfeature extracted utilizing a multimodal LSTM (m-LSTM) model. Training of\nm-LSTM is based on the portion of data in which the image content and the\ncorresponding descriptions are strongly bonded. Afterwards, during the training\nof sg-LSTM on the rest training data, this guiding information serves as\nadditional input to the network along with the image representations and the\nground-truth descriptions. By integrating these input components into a\nmultimodal block, we aim to form a training scheme with the textual information\ntightly coupled with the image content. The experimental results demonstrate\nthat the proposed sg-LSTM model outperforms the traditional state-of-the-art\nmultimodal RNN captioning framework in successfully describing the key\ncomponents of the input images.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1709.05278v1",
            "date": "2017-09-15 15:47:23+00:00",
            "title": "Algorithms and Architecture for Real-time Recommendations at News UK",
            "authors": [
                "Dion Bailey",
                "Tom Pajak",
                "Daoud Clarke",
                "Carlos Rodriguez"
            ],
            "abstract": "Recommendation systems are recognised as being hugely important in industry,\nand the area is now well understood. At News UK, there is a requirement to be\nable to quickly generate recommendations for users on news items as they are\npublished. However, little has been published about systems that can generate\nrecommendations in response to changes in recommendable items and user\nbehaviour in a very short space of time. In this paper we describe a new\nalgorithm for updating collaborative filtering models incrementally, and\ndemonstrate its effectiveness on clickstream data from The Times. We also\ndescribe the architecture that allows recommendations to be generated on the\nfly, and how we have made each component scalable. The system is currently\nbeing used in production at News UK.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.05576v1",
            "date": "2017-09-16 22:58:13+00:00",
            "title": "SKOS Concepts and Natural Language Concepts: an Analysis of Latent Relationships in KOSs",
            "authors": [
                "Anna Mastora",
                "Manolis Peponakis",
                "Sarantos Kapidakis"
            ],
            "abstract": "The vehicle to represent Knowledge Organization Systems (KOSs) in the\nenvironment of the Semantic Web and linked data is the Simple Knowledge\nOrganization System (SKOS). SKOS provides a way to assign a URI to each\nconcept, and this URI functions as a surrogate for the concept. This fact makes\nof main concern the need to clarify the URIs' ontological meaning. The aim of\nthis study is to investigate the relation between the ontological substance of\nKOS concepts and concepts revealed through the grammatical and syntactic\nformalisms of natural language. For this purpose, we examined the dividableness\nof concepts in specific KOSs (i.e. a thesaurus, a subject headings system and a\nclassification scheme) by applying Natural Language Processing (NLP) techniques\n(i.e. morphosyntactic analysis) to the lexical representations (i.e. RDF\nliterals) of SKOS concepts. The results of the comparative analysis reveal\nthat, despite the use of multi-word units, thesauri tend to represent concepts\nin a way that can hardly be further divided conceptually, while Subject\nHeadings and Classification Schemes - to a certain extent - comprise terms that\ncan be decomposed into more conceptual constituents. Consequently, SKOS\nconcepts deriving from thesauri are more likely to represent atomic conceptual\nunits and thus be more appropriate tools for inference and reasoning. Since\nidentifiers represent the meaning of a concept, complex concepts are neither\nthe most appropriate nor the most efficient way of modelling a KOS for the\nSemantic Web.",
            "journal ref": "Journal of Information Science, 43(4), 492-508 (2017)",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06671v1",
            "date": "2017-09-19 22:58:02+00:00",
            "title": "Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words",
            "authors": [
                "Danushka Bollegala",
                "Kohei Hayashi",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Distributed word embeddings have shown superior performances in numerous\nNatural Language Processing (NLP) tasks. However, their performances vary\nsignificantly across different tasks, implying that the word embeddings learnt\nby those methods capture complementary aspects of lexical semantics. Therefore,\nwe believe that it is important to combine the existing word embeddings to\nproduce more accurate and complete \\emph{meta-embeddings} of words. For this\npurpose, we propose an unsupervised locally linear meta-embedding learning\nmethod that takes pre-trained word embeddings as the input, and produces more\naccurate meta embeddings. Unlike previously proposed meta-embedding learning\nmethods that learn a global projection over all words in a vocabulary, our\nproposed method is sensitive to the differences in local neighbourhoods of the\nindividual source word embeddings. Moreover, we show that vector concatenation,\na previously proposed highly competitive baseline approach for integrating word\nembeddings, can be derived as a special case of the proposed method.\nExperimental results on semantic similarity, word analogy, relation\nclassification, and short-text classification tasks show that our\nmeta-embeddings to significantly outperform prior methods in several benchmark\ndatasets, establishing a new state of the art for meta-embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06673v2",
            "date": "2017-09-19 23:09:15+00:00",
            "title": "Why PairDiff works? -- A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection",
            "authors": [
                "Huda Hakami",
                "Danushka Bollegala",
                "Hayashi Kohei"
            ],
            "abstract": "Representing the semantic relations that exist between two given words (or\nentities) is an important first step in a wide-range of NLP applications such\nas analogical reasoning, knowledge base completion and relational information\nretrieval. A simple, yet surprisingly accurate method for representing a\nrelation between two words is to compute the vector offset (\\PairDiff) between\ntheir corresponding word embeddings. Despite the empirical success, it remains\nunclear as to whether \\PairDiff is the best operator for obtaining a relational\nrepresentation from word embeddings. We conduct a theoretical analysis of\ngeneralised bilinear operators that can be used to measure the $\\ell_{2}$\nrelational distance between two word-pairs. We show that, if the word\nembeddings are standardised and uncorrelated, such an operator will be\nindependent of bilinear terms, and can be simplified to a linear form, where\n\\PairDiff is a special case. For numerous word embedding types, we empirically\nverify the uncorrelation assumption, demonstrating the general applicability of\nour theoretical result. Moreover, we experimentally discover \\PairDiff from the\nbilinear relation composition operator on several benchmark analogy datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06818v1",
            "date": "2017-09-20 11:28:40+00:00",
            "title": "Updating the silent speech challenge benchmark with deep learning",
            "authors": [
                "Yan Ji",
                "Licheng Liu",
                "Hongcui Wang",
                "Zhilei Liu",
                "Zhibin Niu",
                "Bruce Denby"
            ],
            "abstract": "The 2010 Silent Speech Challenge benchmark is updated with new results\nobtained in a Deep Learning strategy, using the same input features and\ndecoding strategy as in the original article. A Word Error Rate of 6.4% is\nobtained, compared to the published value of 17.4%. Additional results\ncomparing new auto-encoder-based features with the original features at reduced\ndimensionality, as well as decoding scenarios on two different language models,\nare also presented. The Silent Speech Challenge archive has been updated to\ncontain both the original and the new auto-encoder features, in addition to the\noriginal raw data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.06907v1",
            "date": "2017-09-20 14:43:08+00:00",
            "title": "Doctoral Advisor or Medical Condition: Towards Entity-specific Rankings of Knowledge Base Properties [Extended Version]",
            "authors": [
                "Simon Razniewski",
                "Vevake Balaraman",
                "Werner Nutt"
            ],
            "abstract": "In knowledge bases such as Wikidata, it is possible to assert a large set of\nproperties for entities, ranging from generic ones such as name and place of\nbirth to highly profession-specific or background-specific ones such as\ndoctoral advisor or medical condition. Determining a preference or ranking in\nthis large set is a challenge in tasks such as prioritisation of edits or\nnatural-language generation. Most previous approaches to ranking knowledge base\nproperties are purely data-driven, that is, as we show, mistake frequency for\ninterestingness.\n  In this work, we have developed a human-annotated dataset of 350 preference\njudgments among pairs of knowledge base properties for fixed entities. From\nthis set, we isolate a subset of pairs for which humans show a high level of\nagreement (87.5% on average). We show, however, that baseline and\nstate-of-the-art techniques achieve only 61.3% precision in predicting human\npreferences for this subset.\n  We then analyze what contributes to one property being rated as more\nimportant than another one, and identify that at least three factors play a\nrole, namely (i) general frequency, (ii) applicability to similar entities and\n(iii) semantic similarity between property and entity. We experimentally\nanalyze the contribution of each factor and show that a combination of\ntechniques addressing all the three factors achieves 74% precision on the task.\n  The dataset is available at\nwww.kaggle.com/srazniewski/wikidatapropertyranking.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1709.06990v1",
            "date": "2017-09-20 17:57:16+00:00",
            "title": "Text Compression for Sentiment Analysis via Evolutionary Algorithms",
            "authors": [
                "Emmanuel Dufourq",
                "Bruce A. Bassett"
            ],
            "abstract": "Can textual data be compressed intelligently without losing accuracy in\nevaluating sentiment? In this study, we propose a novel evolutionary\ncompression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression),\nwhich makes use of Parts-of-Speech tags to compress text in a way that\nsacrifices minimal classification accuracy when used in conjunction with\nsentiment analysis algorithms. An analysis of PARSEC with eight commercial and\nnon-commercial sentiment analysis algorithms on twelve English sentiment data\nsets reveals that accurate compression is possible with (0%, 1.3%, 3.3%) loss\nin sentiment classification accuracy for (20%, 50%, 75%) data compression with\nPARSEC using LingPipe, the most accurate of the sentiment algorithms. Other\nsentiment analysis algorithms are more severely affected by compression. We\nconclude that significant compression of text data is possible for sentiment\nanalysis depending on the accuracy demands of the specific application and the\nspecific sentiment analysis algorithm used.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1709.07109v3",
            "date": "2017-09-21 00:23:55+00:00",
            "title": "Deconvolutional Latent-Variable Model for Text Sequence Matching",
            "authors": [
                "Dinghan Shen",
                "Yizhe Zhang",
                "Ricardo Henao",
                "Qinliang Su",
                "Lawrence Carin"
            ],
            "abstract": "A latent-variable model is introduced for text matching, inferring sentence\nrepresentations by jointly optimizing generative and discriminative objectives.\nTo alleviate typical optimization challenges in latent-variable models for\ntext, we employ deconvolutional networks as the sequence decoder (generator),\nproviding learned latent codes with more semantic information and better\ngeneralization. Our model, trained in an unsupervised manner, yields stronger\nempirical predictive performance than a decoder based on Long Short-Term Memory\n(LSTM), with less parameters and considerably faster training. Further, we\napply it to text sequence-matching problems. The proposed model significantly\noutperforms several strong sentence-encoding baselines, especially in the\nsemi-supervised setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07434v1",
            "date": "2017-09-21 17:50:13+00:00",
            "title": "Analyzing users' sentiment towards popular consumer industries and brands on Twitter",
            "authors": [
                "Guoning Hu",
                "Preeti Bhargava",
                "Saul Fuhrmann",
                "Sarah Ellinger",
                "Nemanja Spasojevic"
            ],
            "abstract": "Social media serves as a unified platform for users to express their thoughts\non subjects ranging from their daily lives to their opinion on consumer brands\nand products. These users wield an enormous influence in shaping the opinions\nof other consumers and influence brand perception, brand loyalty and brand\nadvocacy. In this paper, we analyze the opinion of 19M Twitter users towards 62\npopular industries, encompassing 12,898 enterprise and consumer brands, as well\nas associated subject matter topics, via sentiment analysis of 330M tweets over\na period spanning a month. We find that users tend to be most positive towards\nmanufacturing and most negative towards service industries. In addition, they\ntend to be more positive or negative when interacting with brands than\ngenerally on Twitter. We also find that sentiment towards brands within an\nindustry varies greatly and we demonstrate this using two industries as use\ncases. In addition, we discover that there is no strong correlation between\ntopic sentiments of different industries, demonstrating that topic sentiments\nare highly dependent on the context of the industry that they are mentioned in.\nWe demonstrate the value of such an analysis in order to assess the impact of\nbrands on social media. We hope that this initial study will prove valuable for\nboth researchers and companies in understanding users' perception of\nindustries, brands and associated topics and encourage more research in this\nfield.",
            "journal ref": "2017 IEEE International Conference on Data Mining Workshops (ICDMW\n  2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07814v1",
            "date": "2017-09-22 15:38:09+00:00",
            "title": "Attention-based Wav2Text with Feature Transfer Learning",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Conventional automatic speech recognition (ASR) typically performs\nmulti-level pattern recognition tasks that map the acoustic speech waveform\ninto a hierarchy of speech units. But, it is widely known that information loss\nin the earlier stage can propagate through the later stages. After the\nresurgence of deep learning, interest has emerged in the possibility of\ndeveloping a purely end-to-end ASR system from the raw waveform to the\ntranscription without any predefined alignments and hand-engineered models.\nHowever, the successful attempts in end-to-end architecture still used\nspectral-based features, while the successful attempts in using raw waveform\nwere still based on the hybrid deep neural network - Hidden Markov model\n(DNN-HMM) framework. In this paper, we construct the first end-to-end\nattention-based encoder-decoder model to process directly from raw speech\nwaveform to the text transcription. We called the model as \"Attention-based\nWav2Text\". To assist the training process of the end-to-end model, we propose\nto utilize a feature transfer learning. Experimental results also reveal that\nthe proposed Attention-based Wav2Text model directly with raw waveform could\nachieve a better result in comparison with the attentional encoder-decoder\nmodel trained on standard front-end filterbank features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07871v2",
            "date": "2017-09-22 17:54:12+00:00",
            "title": "FiLM: Visual Reasoning with a General Conditioning Layer",
            "authors": [
                "Ethan Perez",
                "Florian Strub",
                "Harm de Vries",
                "Vincent Dumoulin",
                "Aaron Courville"
            ],
            "abstract": "We introduce a general-purpose conditioning method for neural networks called\nFiLM: Feature-wise Linear Modulation. FiLM layers influence neural network\ncomputation via a simple, feature-wise affine transformation based on\nconditioning information. We show that FiLM layers are highly effective for\nvisual reasoning - answering image-related questions which require a\nmulti-step, high-level process - a task which has proven difficult for standard\ndeep learning methods that do not explicitly model reasoning. Specifically, we\nshow on visual reasoning tasks that FiLM layers 1) halve state-of-the-art error\nfor the CLEVR benchmark, 2) modulate features in a coherent manner, 3) are\nrobust to ablations and architectural modifications, and 4) generalize well to\nchallenging, new data from few examples or even zero-shot.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1709.08294v3",
            "date": "2017-09-25 02:29:26+00:00",
            "title": "Learning Context-Sensitive Convolutional Filters for Text Processing",
            "authors": [
                "Dinghan Shen",
                "Martin Renqiang Min",
                "Yitong Li",
                "Lawrence Carin"
            ],
            "abstract": "Convolutional neural networks (CNNs) have recently emerged as a popular\nbuilding block for natural language processing (NLP). Despite their success,\nmost existing CNN models employed in NLP share the same learned (and static)\nset of filters for all input sentences. In this paper, we consider an approach\nof using a small meta network to learn context-sensitive convolutional filters\nfor text processing. The role of meta network is to abstract the contextual\ninformation of a sentence or document into a set of input-aware filters. We\nfurther generalize this framework to model sentence pairs, where a\nbidirectional filter generation mechanism is introduced to encapsulate\nco-dependent sentence representations. In our benchmarks on four different\ntasks, including ontology classification, sentiment analysis, answer sentence\nselection, and paraphrase identification, our proposed model, a modified CNN\nwith context-sensitive filters, consistently outperforms the standard CNN and\nattention-based CNN baselines. By visualizing the learned context-sensitive\nfilters, we further validate and rationalize the effectiveness of proposed\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08624v2",
            "date": "2017-09-24 13:35:08+00:00",
            "title": "Long Text Generation via Adversarial Training with Leaked Information",
            "authors": [
                "Jiaxian Guo",
                "Sidi Lu",
                "Han Cai",
                "Weinan Zhang",
                "Yong Yu",
                "Jun Wang"
            ],
            "abstract": "Automatically generating coherent and semantically meaningful text has many\napplications in machine translation, dialogue systems, image captioning, etc.\nRecently, by combining with policy gradient, Generative Adversarial Nets (GAN)\nthat use a discriminative model to guide the training of the generative model\nas a reinforcement learning policy has shown promising results in text\ngeneration. However, the scalar guiding signal is only available after the\nentire text has been generated and lacks intermediate information about text\nstructure during the generative process. As such, it limits its success when\nthe length of the generated text samples is long (more than 20 words). In this\npaper, we propose a new framework, called LeakGAN, to address the problem for\nlong text generation. We allow the discriminative net to leak its own\nhigh-level extracted features to the generative net to further help the\nguidance. The generator incorporates such informative signals into all\ngeneration steps through an additional Manager module, which takes the\nextracted features of current generated words and outputs a latent vector to\nguide the Worker module for next-word generation. Our extensive experiments on\nsynthetic data and various real-world tasks with Turing test demonstrate that\nLeakGAN is highly effective in long text generation and also improves the\nperformance in short text generation scenarios. More importantly, without any\nsupervision, LeakGAN would be able to implicitly learn sentence structures only\nthrough the interaction between Manager and Worker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.08853v6",
            "date": "2017-09-26 06:17:35+00:00",
            "title": "Object-oriented Neural Programming (OONP) for Document Understanding",
            "authors": [
                "Zhengdong Lu",
                "Xianggen Liu",
                "Haotian Cui",
                "Yukun Yan",
                "Daqi Zheng"
            ],
            "abstract": "We propose Object-oriented Neural Programming (OONP), a framework for\nsemantically parsing documents in specific domains. Basically, OONP reads a\ndocument and parses it into a predesigned object-oriented data structure\n(referred to as ontology in this paper) that reflects the domain-specific\nsemantics of the document. An OONP parser models semantic parsing as a decision\nprocess: a neural net-based Reader sequentially goes through the document, and\nduring the process it builds and updates an intermediate ontology to summarize\nits partial understanding of the text it covers. OONP supports a rich family of\noperations (both symbolic and differentiable) for composing the ontology, and a\nbig variety of forms (both symbolic and differentiable) for representing the\nstate and the document. An OONP parser can be trained with supervision of\ndifferent forms and strength, including supervised learning (SL) ,\nreinforcement learning (RL) and hybrid of the two. Our experiments on both\nsynthetic and real-world document parsing tasks have shown that OONP can learn\nto handle fairly complicated ontology with training data of modest sizes.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.09373v1",
            "date": "2017-09-27 07:49:03+00:00",
            "title": "A Bimodal Network Approach to Model Topic Dynamics",
            "authors": [
                "Luigi Di Caro",
                "Marco Guerzoni",
                "Massimiliano Nuccio",
                "Giovanni Siragusa"
            ],
            "abstract": "This paper presents an intertemporal bimodal network to analyze the evolution\nof the semantic content of a scientific field within the framework of topic\nmodeling, namely using the Latent Dirichlet Allocation (LDA). The main\ncontribution is the conceptualization of the topic dynamics and its\nformalization and codification into an algorithm. To benchmark the\neffectiveness of this approach, we propose three indexes which track the\ntransformation of topics over time, their rate of birth and death, and the\nnovelty of their content. Applying the LDA, we test the algorithm both on a\ncontrolled experiment and on a corpus of several thousands of scientific papers\nover a period of more than 100 years which account for the history of the\neconomic thought.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09741v1",
            "date": "2017-09-27 21:30:53+00:00",
            "title": "WHY: Natural Explanations from a Robot Navigator",
            "authors": [
                "Raj Korpan",
                "Susan L. Epstein",
                "Anoop Aroor",
                "Gil Dekel"
            ],
            "abstract": "Effective collaboration between a robot and a person requires natural\ncommunication. When a robot travels with a human companion, the robot should be\nable to explain its navigation behavior in natural language. This paper\nexplains how a cognitively-based, autonomous robot navigation system produces\ninformative, intuitive explanations for its decisions. Language generation here\nis based upon the robot's commonsense, its qualitative reasoning, and its\nlearned spatial model. This approach produces natural explanations in real time\nfor a robot as it navigates in a large, complex indoor environment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1709.09749v1",
            "date": "2017-09-27 22:05:59+00:00",
            "title": "KeyVec: Key-semantics Preserving Document Representations",
            "authors": [
                "Bin Bi",
                "Hao Ma"
            ],
            "abstract": "Previous studies have demonstrated the empirical success of word embeddings\nin various applications. In this paper, we investigate the problem of learning\ndistributed representations for text documents which many machine learning\nalgorithms take as input for a number of NLP tasks.\n  We propose a neural network model, KeyVec, which learns document\nrepresentations with the goal of preserving key semantics of the input text. It\nenables the learned low-dimensional vectors to retain the topics and important\ninformation from the documents that will flow to downstream tasks. Our\nempirical evaluations show the superior quality of KeyVec representations in\ntwo different document understanding tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.09927v3",
            "date": "2017-09-28 12:58:18+00:00",
            "title": "Inference of Personal Attributes from Tweets Using Machine Learning",
            "authors": [
                "Take Yo",
                "Kazutoshi Sasahara"
            ],
            "abstract": "Using machine learning algorithms, including deep learning, we studied the\nprediction of personal attributes from the text of tweets, such as gender,\noccupation, and age groups. We applied word2vec to construct word vectors,\nwhich were then used to vectorize tweet blocks. The resulting tweet vectors\nwere used as inputs for training models, and the prediction accuracy of those\nmodels was examined as a function of the dimension of the tweet vectors and the\nsize of the tweet blacks. The results showed that the machine learning\nalgorithms could predict the three personal attributes of interest with 60-70%\naccuracy.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1709.10204v2",
            "date": "2017-09-29 00:27:48+00:00",
            "title": "A Neural Comprehensive Ranker (NCR) for Open-Domain Question Answering",
            "authors": [
                "Bin Bi",
                "Hao Ma"
            ],
            "abstract": "This paper proposes a novel neural machine reading model for open-domain\nquestion answering at scale. Existing machine comprehension models typically\nassume that a short piece of relevant text containing answers is already\nidentified and given to the models, from which the models are designed to\nextract answers. This assumption, however, is not realistic for building a\nlarge-scale open-domain question answering system which requires both deep text\nunderstanding and identifying relevant text from corpus simultaneously.\n  In this paper, we introduce Neural Comprehensive Ranker (NCR) that integrates\nboth passage ranking and answer extraction in one single framework. A Q&A\nsystem based on this framework allows users to issue an open-domain question\nwithout needing to provide a piece of text that must contain the answer.\nExperiments show that the unified NCR model is able to outperform the\nstates-of-the-art in both retrieval of relevant text and answer extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10367v1",
            "date": "2017-09-28 14:14:58+00:00",
            "title": "Structured Embedding Models for Grouped Data",
            "authors": [
                "Maja Rudolph",
                "Francisco Ruiz",
                "Susan Athey",
                "David Blei"
            ],
            "abstract": "Word embeddings are a powerful approach for analyzing language, and\nexponential family embeddings (EFE) extend them to other types of data. Here we\ndevelop structured exponential family embeddings (S-EFE), a method for\ndiscovering embeddings that vary across related groups of data. We study how\nthe word usage of U.S. Congressional speeches varies across states and party\naffiliation, how words are used differently across sections of the ArXiv, and\nhow the co-purchase patterns of groceries can vary across seasons. Key to the\nsuccess of our method is that the groups share statistical information. We\ndevelop two sharing strategies: hierarchical modeling and amortization. We\ndemonstrate the benefits of this approach in empirical studies of speeches,\nabstracts, and shopping baskets. We show how S-EFE enables group-specific\ninterpretation of word usage, and outperforms EFE in predicting held-out data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10423v1",
            "date": "2017-09-29 14:21:31+00:00",
            "title": "Learning how to learn: an adaptive dialogue agent for incrementally learning visually grounded word meanings",
            "authors": [
                "Yanchao Yu",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "We present an optimised multi-modal dialogue agent for interactive learning\nof visually grounded word meanings from a human tutor, trained on real\nhuman-human tutoring data. Within a life-long interactive learning period, the\nagent, trained using Reinforcement Learning (RL), must be able to handle\nnatural conversations with human users and achieve good learning performance\n(accuracy) while minimising human effort in the learning process. We train and\nevaluate this system in interaction with a simulated human tutor, which is\nbuilt on the BURCHAK corpus -- a Human-Human Dialogue dataset for the visual\nlearning task. The results show that: 1) The learned policy can coherently\ninteract with the simulated user to achieve the goal of the task (i.e. learning\nvisual attributes of objects, e.g. colour and shape); and 2) it finds a better\ntrade-off between classifier accuracy and tutoring costs than hand-crafted\nrule-based policies, including ones with dynamic policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10426v1",
            "date": "2017-09-29 14:28:31+00:00",
            "title": "Training an adaptive dialogue policy for interactive learning of visually grounded word meanings",
            "authors": [
                "Yanchao Yu",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "We present a multi-modal dialogue system for interactive learning of\nperceptually grounded word meanings from a human tutor. The system integrates\nan incremental, semantic parsing/generation framework - Dynamic Syntax and Type\nTheory with Records (DS-TTR) - with a set of visual classifiers that are\nlearned throughout the interaction and which ground the meaning representations\nthat it produces. We use this system in interaction with a simulated human\ntutor to study the effects of different dialogue policies and capabilities on\nthe accuracy of learned meanings, learning rates, and efforts/costs to the\ntutor. We show that the overall performance of the learning agent is affected\nby (1) who takes initiative in the dialogues; (2) the ability to express/use\ntheir confidence level about visual attributes; and (3) the ability to process\nelliptical and incrementally constructed dialogue turns. Ultimately, we train\nan adaptive dialogue policy which optimises the trade-off between classifier\naccuracy and tutoring costs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.10431v1",
            "date": "2017-09-29 14:43:06+00:00",
            "title": "The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings",
            "authors": [
                "Yanchao Yu",
                "Arash Eshghi",
                "Gregory Mills",
                "Oliver Joseph Lemon"
            ],
            "abstract": "We motivate and describe a new freely available human-human dialogue dataset\nfor interactive learning of visually grounded word meanings through ostensive\ndefinition by a tutor to a learner. The data has been collected using a novel,\ncharacter-by-character variant of the DiET chat tool (Healey et al., 2003;\nMills and Healey, submitted) with a novel task, where a Learner needs to learn\ninvented visual attribute words (such as \" burchak \" for square) from a tutor.\nAs such, the text-based interactions closely resemble face-to-face conversation\nand thus contain many of the linguistic phenomena encountered in natural,\nspontaneous dialogue. These include self-and other-correction, mid-sentence\ncontinuations, interruptions, overlaps, fillers, and hedges. We also present a\ngeneric n-gram framework for building user (i.e. tutor) simulations from this\ntype of incremental data, which is freely available to researchers. We show\nthat the simulations produce outputs that are similar to the original data\n(e.g. 78% turn match similarity). Finally, we train and evaluate a\nReinforcement Learning dialogue control agent for learning visually grounded\nword meanings, trained from the BURCHAK corpus. The learned policy shows\ncomparable performance to a rule-based system built previously.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00168v1",
            "date": "2017-12-30 17:59:41+00:00",
            "title": "The origins of Zipf's meaning-frequency law",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Michael S. Vitevitch"
            ],
            "abstract": "In his pioneering research, G. K. Zipf observed that more frequent words tend\nto have more meanings, and showed that the number of meanings of a word grows\nas the square root of its frequency. He derived this relationship from two\nassumptions: that words follow Zipf's law for word frequencies (a power law\ndependency between frequency and rank) and Zipf's law of meaning distribution\n(a power law dependency between number of meanings and rank). Here we show that\na single assumption on the joint probability of a word and a meaning suffices\nto infer Zipf's meaning-frequency law or relaxed versions. Interestingly, this\nassumption can be justified as the outcome of a biased random walk in the\nprocess of mental exploration.",
            "journal ref": "Journal of the American Society for Information Science and\n  Technology 69 (11), 1369-1379 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00215v1",
            "date": "2017-12-31 00:51:56+00:00",
            "title": "Learning Continuous User Representations through Hybrid Filtering with doc2vec",
            "authors": [
                "Simon Stiebellehner",
                "Jun Wang",
                "Shuai Yuan"
            ],
            "abstract": "Players in the online ad ecosystem are struggling to acquire the user data\nrequired for precise targeting. Audience look-alike modeling has the potential\nto alleviate this issue, but models' performance strongly depends on quantity\nand quality of available data. In order to maximize the predictive performance\nof our look-alike modeling algorithms, we propose two novel hybrid filtering\ntechniques that utilize the recent neural probabilistic language model\nalgorithm doc2vec. We apply these methods to data from a large mobile ad\nexchange and additional app metadata acquired from the Apple App store and\nGoogle Play store. First, we model mobile app users through their app usage\nhistories and app descriptions (user2vec). Second, we introduce context\nawareness to that model by incorporating additional user and app-related\nmetadata in model training (context2vec). Our findings are threefold: (1) the\nquality of recommendations provided by user2vec is notably higher than current\nstate-of-the-art techniques. (2) User representations generated through hybrid\nfiltering using doc2vec prove to be highly valuable features in supervised\nmachine learning models for look-alike modeling. This represents the first\napplication of hybrid filtering user models using neural probabilistic language\nmodels, specifically doc2vec, in look-alike modeling. (3) Incorporating context\nmetadata in the doc2vec model training process to introduce context awareness\nhas positive effects on performance and is superior to directly including the\ndata as features in the downstream supervised models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.00119v1",
            "date": "2017-04-01 04:31:37+00:00",
            "title": "Psychological and Personality Profiles of Political Extremists",
            "authors": [
                "Meysam Alizadeh",
                "Ingmar Weber",
                "Claudio Cioffi-Revilla",
                "Santo Fortunato",
                "Michael Macy"
            ],
            "abstract": "Global recruitment into radical Islamic movements has spurred renewed\ninterest in the appeal of political extremism. Is the appeal a rational\nresponse to material conditions or is it the expression of psychological and\npersonality disorders associated with aggressive behavior, intolerance,\nconspiratorial imagination, and paranoia? Empirical answers using surveys have\nbeen limited by lack of access to extremist groups, while field studies have\nlacked psychological measures and failed to compare extremists with contrast\ngroups. We revisit the debate over the appeal of extremism in the U.S. context\nby comparing publicly available Twitter messages written by over 355,000\npolitical extremist followers with messages written by non-extremist U.S.\nusers. Analysis of text-based psychological indicators supports the moral\nfoundation theory which identifies emotion as a critical factor in determining\npolitical orientation of individuals. Extremist followers also differ from\nothers in four of the Big Five personality traits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00217v1",
            "date": "2017-04-01 19:29:21+00:00",
            "title": "Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification",
            "authors": [
                "Lianhui Qin",
                "Zhisong Zhang",
                "Hai Zhao",
                "Zhiting Hu",
                "Eric P. Xing"
            ],
            "abstract": "Implicit discourse relation classification is of great challenge due to the\nlack of connectives as strong linguistic cues, which motivates the use of\nannotated implicit connectives to improve the recognition. We propose a feature\nimitation framework in which an implicit relation network is driven to learn\nfrom another neural network with access to connectives, and thus encouraged to\nextract similarly salient features for accurate classification. We develop an\nadversarial model to enable an adaptive imitation scheme through competition\nbetween the implicit network and a rival feature discriminator. Our method\neffectively transfers discriminability of connectives to the implicit features,\nand achieves state-of-the-art performance on the PDTB benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00514v2",
            "date": "2017-04-03 10:25:22+00:00",
            "title": "Multi-Task Learning of Keyphrase Boundary Classification",
            "authors": [
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Keyphrase boundary classification (KBC) is the task of detecting keyphrases\nin scientific articles and labelling them with respect to predefined types.\nAlthough important in practice, this task is so far underexplored, partly due\nto the lack of labelled data. To overcome this, we explore several auxiliary\ntasks, including semantic super-sense tagging and identification of multi-word\nexpressions, and cast the task as a multi-task learning problem with deep\nrecurrent neural networks. Our multi-task models perform significantly better\nthan previous state of the art approaches on two scientific KBC datasets,\nparticularly for long keyphrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00656v3",
            "date": "2017-04-03 15:57:44+00:00",
            "title": "Detection and Resolution of Rumours in Social Media: A Survey",
            "authors": [
                "Arkaitz Zubiaga",
                "Ahmet Aker",
                "Kalina Bontcheva",
                "Maria Liakata",
                "Rob Procter"
            ],
            "abstract": "Despite the increasing use of social media platforms for information and news\ngathering, its unmoderated nature often leads to the emergence and spread of\nrumours, i.e. pieces of information that are unverified at the time of posting.\nAt the same time, the openness of social media platforms provides opportunities\nto study how users share and discuss rumours, and to explore how natural\nlanguage processing and data mining techniques may be used to find ways of\ndetermining their veracity. In this survey we introduce and discuss two types\nof rumours that circulate on social media; long-standing rumours that circulate\nfor long periods of time, and newly-emerging rumours spawned during fast-paced\nevents such as breaking news, where reports are released piecemeal and often\nwith an unverified status in their early stages. We provide an overview of\nresearch into social media rumours with the ultimate goal of developing a\nrumour classification system that consists of four components: rumour\ndetection, rumour tracking, rumour stance classification and rumour veracity\nclassification. We delve into the approaches presented in the scientific\nliterature for the development of each of these four components. We summarise\nthe efforts and achievements so far towards the development of rumour\nclassification systems and conclude with suggestions for avenues for future\nresearch in social media mining for detection and resolution of rumours.",
            "journal ref": "ACM Computing Surveys 51, 2, Article 32 (February 2018), 36 pages",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.00717v2",
            "date": "2017-04-03 17:58:07+00:00",
            "title": "It Takes Two to Tango: Towards Theory of AI's Mind",
            "authors": [
                "Arjun Chandrasekaran",
                "Deshraj Yadav",
                "Prithvijit Chattopadhyay",
                "Viraj Prabhu",
                "Devi Parikh"
            ],
            "abstract": "Theory of Mind is the ability to attribute mental states (beliefs, intents,\nknowledge, perspectives, etc.) to others and recognize that these mental states\nmay differ from one's own. Theory of Mind is critical to effective\ncommunication and to teams demonstrating higher collective performance. To\neffectively leverage the progress in Artificial Intelligence (AI) to make our\nlives more productive, it is important for humans and AI to work well together\nin a team. Traditionally, there has been much emphasis on research to make AI\nmore accurate, and (to a lesser extent) on having it better understand human\nintentions, tendencies, beliefs, and contexts. The latter involves making AI\nmore human-like and having it develop a theory of our minds. In this work, we\nargue that for human-AI teams to be effective, humans must also develop a\ntheory of AI's mind (ToAIM) - get to know its strengths, weaknesses, beliefs,\nand quirks. We instantiate these ideas within the domain of Visual Question\nAnswering (VQA). We find that using just a few examples (50), lay people can be\ntrained to better predict responses and oncoming failures of a complex VQA\nmodel. We further evaluate the role existing explanation (or interpretability)\nmodalities play in helping humans build ToAIM. Explainable AI has received\nconsiderable scientific and popular attention in recent times. Surprisingly, we\nfind that having access to the model's internal states - its confidence in its\ntop-k predictions, explicit or implicit attention maps which highlight regions\nin the image (and words in the question) the model is looking at (and listening\nto) while answering a question about an image - do not help people better\npredict its behavior.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1704.01444v2",
            "date": "2017-04-05 14:20:28+00:00",
            "title": "Learning to Generate Reviews and Discovering Sentiment",
            "authors": [
                "Alec Radford",
                "Rafal Jozefowicz",
                "Ilya Sutskever"
            ],
            "abstract": "We explore the properties of byte-level recurrent language models. When given\nsufficient amounts of capacity, training data, and compute time, the\nrepresentations learned by these models include disentangled features\ncorresponding to high-level concepts. Specifically, we find a single unit which\nperforms sentiment analysis. These representations, learned in an unsupervised\nmanner, achieve state of the art on the binary subset of the Stanford Sentiment\nTreebank. They are also very data efficient. When using only a handful of\nlabeled examples, our approach matches the performance of strong baselines\ntrained on full datasets. We also demonstrate the sentiment unit has a direct\ninfluence on the generative process of the model. Simply fixing its value to be\npositive or negative generates samples with the corresponding positive or\nnegative sentiment.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1704.01523v1",
            "date": "2017-04-05 16:54:20+00:00",
            "title": "MIT at SemEval-2017 Task 10: Relation Extraction with Convolutional Neural Networks",
            "authors": [
                "Ji Young Lee",
                "Franck Dernoncourt",
                "Peter Szolovits"
            ],
            "abstract": "Over 50 million scholarly articles have been published: they constitute a\nunique repository of knowledge. In particular, one may infer from them\nrelations between scientific concepts, such as synonyms and hyponyms.\nArtificial neural networks have been recently explored for relation extraction.\nIn this work, we continue this line of work and present a system based on a\nconvolutional neural network to extract relations. Our model ranked first in\nthe SemEval-2017 task 10 (ScienceIE) for relation extraction in scientific\narticles (subtask C).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01696v1",
            "date": "2017-04-06 03:13:46+00:00",
            "title": "A Syntactic Neural Model for General-Purpose Code Generation",
            "authors": [
                "Pengcheng Yin",
                "Graham Neubig"
            ],
            "abstract": "We consider the problem of parsing natural language descriptions into source\ncode written in a general-purpose programming language like Python. Existing\ndata-driven methods treat this problem as a language generation task without\nconsidering the underlying syntax of the target programming language. Informed\nby previous work in semantic parsing, in this paper we propose a novel neural\narchitecture powered by a grammar model to explicitly capture the target syntax\nas prior knowledge. Experiments find this an effective way to scale up to\ngeneration of complex programs from natural language descriptions, achieving\nstate-of-the-art results that well outperform previous code generation and\nsemantic parsing approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.01975v2",
            "date": "2017-04-06 18:01:22+00:00",
            "title": "An Automated Text Categorization Framework based on Hyperparameter Optimization",
            "authors": [
                "Eric S. Tellez",
                "Daniela Moctezuma",
                "Sabino Miranda-J\u00edmenez",
                "Mario Graff"
            ],
            "abstract": "A great variety of text tasks such as topic or spam identification, user\nprofiling, and sentiment analysis can be posed as a supervised learning problem\nand tackle using a text classifier. A text classifier consists of several\nsubprocesses, some of them are general enough to be applied to any supervised\nlearning problem, whereas others are specifically designed to tackle a\nparticular task, using complex and computational expensive processes such as\nlemmatization, syntactic analysis, etc. Contrary to traditional approaches, we\npropose a minimalistic and wide system able to tackle text classification tasks\nindependent of domain and language, namely microTC. It is composed by some easy\nto implement text transformations, text representations, and a supervised\nlearning algorithm. These pieces produce a competitive classifier even in the\ndomain of informally written text. We provide a detailed description of microTC\nalong with an extensive experimental comparison with relevant state-of-the-art\nmethods. mircoTC was compared on 30 different datasets. Regarding accuracy,\nmicroTC obtained the best performance in 20 datasets while achieves competitive\nresults in the remaining 10. The compared datasets include several problems\nlike topic and polarity classification, spam detection, user profiling and\nauthorship attribution. Furthermore, it is important to state that our approach\nallows the usage of the technology even without knowledge of machine learning\nand natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02298v2",
            "date": "2017-04-07 17:13:03+00:00",
            "title": "TransNets: Learning to Transform for Recommendation",
            "authors": [
                "Rose Catherine",
                "William Cohen"
            ],
            "abstract": "Recently, deep learning methods have been shown to improve the performance of\nrecommender systems over traditional methods, especially when review text is\navailable. For example, a recent model, DeepCoNN, uses neural nets to learn one\nlatent representation for the text of all reviews written by a target user, and\na second latent representation for the text of all reviews for a target item,\nand then combines these latent representations to obtain state-of-the-art\nperformance on recommendation tasks. We show that (unsurprisingly) much of the\npredictive value of review text comes from reviews of the target user for the\ntarget item. We then introduce a way in which this information can be used in\nrecommendation, even when the target user's review for the target item is not\navailable. Our model, called TransNets, extends the DeepCoNN model by\nintroducing an additional latent layer representing the target user-target item\npair. We then regularize this layer, at training time, to be similar to another\nlatent representation of the target user's review of the target item. We show\nthat TransNets and extensions of it improve substantially over the previous\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.02312v1",
            "date": "2017-04-07 17:53:24+00:00",
            "title": "A Constrained Sequence-to-Sequence Neural Model for Sentence Simplification",
            "authors": [
                "Yaoyuan Zhang",
                "Zhenxu Ye",
                "Yansong Feng",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Sentence simplification reduces semantic complexity to benefit people with\nlanguage impairments. Previous simplification studies on the sentence level and\nword level have achieved promising results but also meet great challenges. For\nsentence-level studies, sentences after simplification are fluent but sometimes\nare not really simplified. For word-level studies, words are simplified but\nalso have potential grammar errors due to different usages of words before and\nafter simplification. In this paper, we propose a two-step simplification\nframework by combining both the word-level and the sentence-level\nsimplifications, making use of their corresponding advantages. Based on the\ntwo-step framework, we implement a novel constrained neural generation model to\nsimplify sentences given simplified words. The final results on Wikipedia and\nSimple Wikipedia aligned datasets indicate that our method yields better\nperformance than various baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02360v4",
            "date": "2017-04-10 12:35:33+00:00",
            "title": "Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities",
            "authors": [
                "Hiroyuki Miyoshi",
                "Yuki Saito",
                "Shinnosuke Takamichi",
                "Hiroshi Saruwatari"
            ],
            "abstract": "Voice conversion (VC) using sequence-to-sequence learning of context\nposterior probabilities is proposed. Conventional VC using shared context\nposterior probabilities predicts target speech parameters from the context\nposterior probabilities estimated from the source speech parameters. Although\nconventional VC can be built from non-parallel data, it is difficult to convert\nspeaker individuality such as phonetic property and speaking rate contained in\nthe posterior probabilities because the source posterior probabilities are\ndirectly used for predicting target speech parameters. In this work, we assume\nthat the training data partly include parallel speech data and propose\nsequence-to-sequence learning between the source and target posterior\nprobabilities. The conversion models perform non-linear and variable-length\ntransformation from the source probability sequence to the target one. Further,\nwe propose a joint training algorithm for the modules. In contrast to\nconventional VC, which separately trains the speech recognition that estimates\nposterior probabilities and the speech synthesis that predicts target speech\nparameters, our proposed method jointly trains these modules along with the\nproposed probability conversion modules. Experimental results demonstrate that\nour approach outperforms the conventional VC.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1704.02686v2",
            "date": "2017-04-10 02:24:37+00:00",
            "title": "Word Embeddings via Tensor Factorization",
            "authors": [
                "Eric Bailey",
                "Shuchin Aeron"
            ],
            "abstract": "Most popular word embedding techniques involve implicit or explicit\nfactorization of a word co-occurrence based matrix into low rank factors. In\nthis paper, we aim to generalize this trend by using numerical methods to\nfactor higher-order word co-occurrence based arrays, or \\textit{tensors}. We\npresent four word embeddings using tensor factorization and analyze their\nadvantages and disadvantages. One of our main contributions is a novel joint\nsymmetric tensor factorization technique related to the idea of coupled tensor\nfactorization. We show that embeddings based on tensor factorization can be\nused to discern the various meanings of polysemous words without being\nexplicitly trained to do so, and motivate the intuition behind why this works\nin a way that doesn't with existing methods. We also modify an existing word\nembedding evaluation metric known as Outlier Detection [Camacho-Collados and\nNavigli, 2016] to evaluate the quality of the order-$N$ relations that a word\nembedding captures, and show that tensor-based methods outperform existing\nmatrix-based methods at this task. Experimentally, we show that all of our word\nembeddings either outperform or are competitive with state-of-the-art baselines\ncommonly used today on a variety of recent datasets. Suggested applications of\ntensor factorization-based word embeddings are given, and all source code and\npre-trained vectors are publicly available online.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1704.02853v3",
            "date": "2017-04-10 13:43:40+00:00",
            "title": "SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications",
            "authors": [
                "Isabelle Augenstein",
                "Mrinal Das",
                "Sebastian Riedel",
                "Lakshmi Vikraman",
                "Andrew McCallum"
            ],
            "abstract": "We describe the SemEval task of extracting keyphrases and relations between\nthem from scientific documents, which is crucial for understanding which\npublications describe which processes, tasks and materials. Although this was a\nnew task, we had a total of 26 submissions across 3 evaluation scenarios. We\nexpect the task and the findings reported in this paper to be relevant for\nresearchers working on understanding scientific content, as well as the broader\nknowledge base population and information extraction communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.02923v1",
            "date": "2017-04-10 16:03:31+00:00",
            "title": "Pay Attention to Those Sets! Learning Quantification from Images",
            "authors": [
                "Ionut Sorodoc",
                "Sandro Pezzelle",
                "Aur\u00e9lie Herbelot",
                "Mariella Dimiccoli",
                "Raffaella Bernardi"
            ],
            "abstract": "Major advances have recently been made in merging language and vision\nrepresentations. But most tasks considered so far have confined themselves to\nthe processing of objects and lexicalised relations amongst objects (content\nwords). We know, however, that humans (even pre-school children) can abstract\nover raw data to perform certain types of higher-level reasoning, expressed in\nnatural language by function words. A case in point is given by their ability\nto learn quantifiers, i.e. expressions like 'few', 'some' and 'all'. From\nformal semantics and cognitive linguistics, we know that quantifiers are\nrelations over sets which, as a simplification, we can see as proportions. For\ninstance, in 'most fish are red', most encodes the proportion of fish which are\nred fish. In this paper, we study how well current language and vision\nstrategies model such relations. We show that state-of-the-art attention\nmechanisms coupled with a traditional linguistic formalisation of quantifiers\ngives best performance on the task. Additionally, we provide insights on the\nrole of 'gist' representations in quantification. A 'logical' strategy to\ntackle the task would be to first obtain a numerosity estimation for the two\ninvolved sets and then compare their cardinalities. We however argue that\nprecisely identifying the composition of the sets is not only beyond current\nstate-of-the-art models but perhaps even detrimental to a task that is most\nefficiently performed by refining the approximate numerosity estimator of the\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03084v3",
            "date": "2017-04-10 23:24:46+00:00",
            "title": "Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning",
            "authors": [
                "Baolin Peng",
                "Xiujun Li",
                "Lihong Li",
                "Jianfeng Gao",
                "Asli Celikyilmaz",
                "Sungjin Lee",
                "Kam-Fai Wong"
            ],
            "abstract": "Building a dialogue agent to fulfill complex tasks, such as travel planning,\nis challenging because the agent has to learn to collectively complete multiple\nsubtasks. For example, the agent needs to reserve a hotel and book a flight so\nthat there leaves enough time for commute between arrival and hotel check-in.\nThis paper addresses this challenge by formulating the task in the mathematical\nframework of options over Markov Decision Processes (MDPs), and proposing a\nhierarchical deep reinforcement learning approach to learning a dialogue\nmanager that operates at different temporal scales. The dialogue manager\nconsists of: (1) a top-level dialogue policy that selects among subtasks or\noptions, (2) a low-level dialogue policy that selects primitive actions to\ncomplete the subtask given by the top-level policy, and (3) a global state\ntracker that helps ensure all cross-subtask constraints be satisfied.\nExperiments on a travel planning task with simulated and real users show that\nour approach leads to significant improvements over three baselines, two based\non handcrafted rules and the other based on flat deep reinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03223v1",
            "date": "2017-04-11 09:47:28+00:00",
            "title": "Persian Wordnet Construction using Supervised Learning",
            "authors": [
                "Zahra Mousavi",
                "Heshaam Faili"
            ],
            "abstract": "This paper presents an automated supervised method for Persian wordnet\nconstruction. Using a Persian corpus and a bi-lingual dictionary, the initial\nlinks between Persian words and Princeton WordNet synsets have been generated.\nThese links will be discriminated later as correct or incorrect by employing\nseven features in a trained classification system. The whole method is just a\nclassification system, which has been trained on a train set containing FarsNet\nas a set of correct instances. State of the art results on the automatically\nderived Persian wordnet is achieved. The resulted wordnet with a precision of\n91.18% includes more than 16,000 words and 22,000 synsets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.03520v2",
            "date": "2017-04-11 20:08:14+00:00",
            "title": "Unsupervised Event Abstraction using Pattern Abstraction and Local Process Models",
            "authors": [
                "Felix Mannhardt",
                "Niek Tax"
            ],
            "abstract": "Process mining analyzes business processes based on events stored in event\nlogs. However, some recorded events may correspond to activities on a very low\nlevel of abstraction. When events are recorded on a too low level of\ngranularity, process discovery methods tend to generate overgeneralizing\nprocess models. Grouping low-level events to higher level activities, i.e.,\nevent abstraction, can be used to discover better process models. Existing\nevent abstraction methods are mainly based on common sub-sequences and\nclustering techniques. In this paper, we propose to first discover local\nprocess models and then use those models to lift the event log to a higher\nlevel of abstraction. Our conjecture is that process models discovered on the\nobtained high-level event log return process models of higher quality: their\nfitness and precision scores are more balanced. We show this with preliminary\nresults on several real-life event logs.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1704.03543v1",
            "date": "2017-04-11 21:21:39+00:00",
            "title": "Leveraging Term Banks for Answering Complex Questions: A Case for Sparse Vectors",
            "authors": [
                "Peter D. Turney"
            ],
            "abstract": "While open-domain question answering (QA) systems have proven effective for\nanswering simple questions, they struggle with more complex questions. Our goal\nis to answer more complex questions reliably, without incurring a significant\ncost in knowledge resource construction to support the QA. One readily\navailable knowledge resource is a term bank, enumerating the key concepts in a\ndomain. We have developed an unsupervised learning approach that leverages a\nterm bank to guide a QA system, by representing the terminological knowledge\nwith thousands of specialized vector spaces. In experiments with complex\nscience questions, we show that this approach significantly outperforms several\nstate-of-the-art QA systems, demonstrating that significant leverage can be\ngained from continuous vector representations of domain terminology.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.03627v2",
            "date": "2017-04-12 05:48:18+00:00",
            "title": "Real-time On-Demand Crowd-powered Entity Extraction",
            "authors": [
                "Ting-Hao 'Kenneth' Huang",
                "Yun-Nung Chen",
                "Jeffrey P. Bigham"
            ],
            "abstract": "Output-agreement mechanisms such as ESP Game have been widely used in human\ncomputation to obtain reliable human-generated labels. In this paper, we argue\nthat a \"time-limited\" output-agreement mechanism can be used to create a fast\nand robust crowd-powered component in interactive systems, particularly\ndialogue systems, to extract key information from user utterances on the fly.\nOur experiments on Amazon Mechanical Turk using the Airline Travel Information\nSystem (ATIS) dataset showed that the proposed approach achieves high-quality\nresults with an average response time shorter than 9 seconds.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1704.03809v3",
            "date": "2017-04-12 15:57:08+00:00",
            "title": "A Neural Parametric Singing Synthesizer",
            "authors": [
                "Merlijn Blaauw",
                "Jordi Bonada"
            ],
            "abstract": "We present a new model for singing synthesis based on a modified version of\nthe WaveNet architecture. Instead of modeling raw waveform, we model features\nproduced by a parametric vocoder that separates the influence of pitch and\ntimbre. This allows conveniently modifying pitch to match any target melody,\nfacilitates training on more modest dataset sizes, and significantly reduces\ntraining and generation times. Our model makes frame-wise predictions using\nmixture density outputs rather than categorical outputs in order to reduce the\nrequired parameter count. As we found overfitting to be an issue with the\nrelatively small datasets used in our experiments, we propose a method to\nregularize the model and make the autoregressive generation process more robust\nto prediction errors. Using a simple multi-stream architecture, harmonic,\naperiodic and voiced/unvoiced components can all be predicted in a coherent\nmanner. We compare our method to existing parametric statistical and\nstate-of-the-art concatenative methods using quantitative metrics and a\nlistening test. While naive implementations of the autoregressive generation\nalgorithm tend to be inefficient, using a smart algorithm we can greatly speed\nup the process and obtain a system that's competitive in both speed and\nquality.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1704.04222v2",
            "date": "2017-04-13 17:41:11+00:00",
            "title": "Learning Latent Representations for Speech Generation and Transformation",
            "authors": [
                "Wei-Ning Hsu",
                "Yu Zhang",
                "James Glass"
            ],
            "abstract": "An ability to model a generative process and learn a latent representation\nfor speech in an unsupervised fashion will be crucial to process vast\nquantities of unlabelled speech data. Recently, deep probabilistic generative\nmodels such as Variational Autoencoders (VAEs) have achieved tremendous success\nin modeling natural images. In this paper, we apply a convolutional VAE to\nmodel the generative process of natural speech. We derive latent space\narithmetic operations to disentangle learned latent representations. We\ndemonstrate the capability of our model to modify the phonetic content or the\nspeaker identity for speech segments using the derived operations, without the\nneed for parallel supervisory data.",
            "journal ref": "Interspeech 2017, pp 1273-1277",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04451v3",
            "date": "2017-04-14 15:22:51+00:00",
            "title": "Optimizing Differentiable Relaxations of Coreference Evaluation Metrics",
            "authors": [
                "Phong Le",
                "Ivan Titov"
            ],
            "abstract": "Coreference evaluation metrics are hard to optimize directly as they are\nnon-differentiable functions, not easily decomposable into elementary\ndecisions. Consequently, most approaches optimize objectives only indirectly\nrelated to the end goal, resulting in suboptimal performance. Instead, we\npropose a differentiable relaxation that lends itself to gradient-based\noptimisation, thus bypassing the need for reinforcement learning or heuristic\nmodification of cross-entropy. We show that by modifying the training objective\nof a competitive neural coreference system, we obtain a substantial gain in\nperformance. This suggests that our approach can be regarded as a viable\nalternative to using reinforcement learning or more computationally expensive\nimitation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04517v1",
            "date": "2017-04-14 19:01:51+00:00",
            "title": "ShapeWorld - A new test methodology for multimodal language understanding",
            "authors": [
                "Alexander Kuhnle",
                "Ann Copestake"
            ],
            "abstract": "We introduce a novel framework for evaluating multimodal deep learning models\nwith respect to their language understanding and generalization abilities. In\nthis approach, artificial data is automatically generated according to the\nexperimenter's specifications. The content of the data, both during training\nand evaluation, can be controlled in detail, which enables tasks to be created\nthat require true generalization abilities, in particular the combination of\npreviously introduced concepts in novel ways. We demonstrate the potential of\nour methodology by evaluating various visual question answering models on four\ndifferent tasks, and show how our framework gives us detailed insights into\ntheir capabilities and limitations. By open-sourcing our framework, we hope to\nstimulate progress in the field of multimodal language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.04664v2",
            "date": "2017-04-15 17:18:11+00:00",
            "title": "Online Spatial Concept and Lexical Acquisition with Simultaneous Localization and Mapping",
            "authors": [
                "Akira Taniguchi",
                "Yoshinobu Hagiwara",
                "Tadahiro Taniguchi",
                "Tetsunari Inamura"
            ],
            "abstract": "In this paper, we propose an online learning algorithm based on a\nRao-Blackwellized particle filter for spatial concept acquisition and mapping.\nWe have proposed a nonparametric Bayesian spatial concept acquisition model\n(SpCoA). We propose a novel method (SpCoSLAM) integrating SpCoA and FastSLAM in\nthe theoretical framework of the Bayesian generative model. The proposed method\ncan simultaneously learn place categories and lexicons while incrementally\ngenerating an environmental map. Furthermore, the proposed method has scene\nimage features and a language model added to SpCoA. In the experiments, we\ntested online learning of spatial concepts and environmental maps in a novel\nenvironment of which the robot did not have a map. Then, we evaluated the\nresults of online learning of spatial concepts and lexical acquisition. The\nexperimental results demonstrated that the robot was able to more accurately\nlearn the relationships between words and the place in the environmental map\nincrementally by using the proposed method.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1704.04683v5",
            "date": "2017-04-15 19:31:41+00:00",
            "title": "RACE: Large-scale ReAding Comprehension Dataset From Examinations",
            "authors": [
                "Guokun Lai",
                "Qizhe Xie",
                "Hanxiao Liu",
                "Yiming Yang",
                "Eduard Hovy"
            ],
            "abstract": "We present RACE, a new dataset for benchmark evaluation of methods in the\nreading comprehension task. Collected from the English exams for middle and\nhigh school Chinese students in the age range between 12 to 18, RACE consists\nof near 28,000 passages and near 100,000 questions generated by human experts\n(English instructors), and covers a variety of topics which are carefully\ndesigned for evaluating the students' ability in understanding and reasoning.\nIn particular, the proportion of questions that requires reasoning is much\nlarger in RACE than that in other benchmark datasets for reading comprehension,\nand there is a significant gap between the performance of the state-of-the-art\nmodels (43%) and the ceiling human performance (95%). We hope this new dataset\ncan serve as a valuable resource for research and evaluation in machine\ncomprehension. The dataset is freely available at\nhttp://www.cs.cmu.edu/~glai1/data/race/ and the code is available at\nhttps://github.com/qizhex/RACE_AR_baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05021v2",
            "date": "2017-04-17 16:32:02+00:00",
            "title": "Sparse Communication for Distributed Gradient Descent",
            "authors": [
                "Alham Fikri Aji",
                "Kenneth Heafield"
            ],
            "abstract": "We make distributed stochastic gradient descent faster by exchanging sparse\nupdates instead of dense updates. Gradient updates are positively skewed as\nmost updates are near zero, so we map the 99% smallest updates (by absolute\nvalue) to zero then exchange sparse matrices. This method can be combined with\nquantization to further improve the compression. We explore different\nconfigurations and apply them to neural machine translation and MNIST image\nclassification tasks. Most configurations work on MNIST, whereas different\nconfigurations reduce convergence rate on the more complex translation task.\nOur experiments show that we can achieve up to 49% speed up on MNIST and 22% on\nNMT without damaging the final accuracy or BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05135v1",
            "date": "2017-04-17 21:42:19+00:00",
            "title": "Does Neural Machine Translation Benefit from Larger Context?",
            "authors": [
                "Sebastien Jean",
                "Stanislas Lauly",
                "Orhan Firat",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose a neural machine translation architecture that models the\nsurrounding text in addition to the source sentence. These models lead to\nbetter performance, both in terms of general translation quality and pronoun\nprediction, when trained on small corpora, although this improvement largely\ndisappears when trained with a larger corpus. We also discover that\nattention-based neural machine translation is well suited for pronoun\nprediction and compares favorably with other approaches that were specifically\ndesigned for this task.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1704.05393v1",
            "date": "2017-04-18 15:20:25+00:00",
            "title": "Mining Worse and Better Opinions. Unsupervised and Agnostic Aggregation of Online Reviews",
            "authors": [
                "Michela Fazzolari",
                "Marinella Petrocchi",
                "Alessandro Tommasi",
                "Cesare Zavattari"
            ],
            "abstract": "In this paper, we propose a novel approach for aggregating online reviews,\naccording to the opinions they express. Our methodology is unsupervised - due\nto the fact that it does not rely on pre-labeled reviews - and it is agnostic -\nsince it does not make any assumption about the domain or the language of the\nreview content. We measure the adherence of a review content to the domain\nterminology extracted from a review set. First, we demonstrate the\ninformativeness of the adherence metric with respect to the score associated\nwith a review. Then, we exploit the metric values to group reviews, according\nto the opinions they express. Our experimental campaign has been carried out on\ntwo large datasets collected from Booking and Amazon, respectively.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1704.05513v1",
            "date": "2017-04-18 20:16:31+00:00",
            "title": "25 Tweets to Know You: A New Model to Predict Personality with Social Media",
            "authors": [
                "Pierre-Hadrien Arnoux",
                "Anbang Xu",
                "Neil Boyette",
                "Jalal Mahmud",
                "Rama Akkiraju",
                "Vibha Sinha"
            ],
            "abstract": "Predicting personality is essential for social applications supporting\nhuman-centered activities, yet prior modeling methods with users written text\nrequire too much input data to be realistically used in the context of social\nmedia. In this work, we aim to drastically reduce the data requirement for\npersonality modeling and develop a model that is applicable to most users on\nTwitter. Our model integrates Word Embedding features with Gaussian Processes\nregression. Based on the evaluation of over 1.3K users on Twitter, we find that\nour model achieves comparable or better accuracy than state of the art\ntechniques with 8 times fewer data.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1704.05543v1",
            "date": "2017-04-18 21:57:10+00:00",
            "title": "Coordinating Collaborative Chat in Massive Open Online Courses",
            "authors": [
                "Gaurav Singh Tomar",
                "Sreecharan Sankaranarayanan",
                "Xu Wang",
                "Carolyn Penstein Ros\u00e9"
            ],
            "abstract": "An earlier study of a collaborative chat intervention in a Massive Open\nOnline Course (MOOC) identified negative effects on attrition stemming from a\nrequirement for students to be matched with exactly one partner prior to\nbeginning the activity. That study raised questions about how to orchestrate a\ncollaborative chat intervention in a MOOC context in order to provide the\nbenefit of synchronous social engagement without the coordination difficulties.\nIn this paper we present a careful analysis of an intervention designed to\novercome coordination difficulties by welcoming students into the chat on a\nrolling basis as they arrive rather than requiring them to be matched with a\npartner before beginning. The results suggest the most positive impact when\nexperiencing a chat with exactly one partner rather than more or less. A\nqualitative analysis of the chat data reveals differential experiences between\nthese configurations that suggests a potential explanation for the effect and\nraises questions for future research.",
            "journal ref": "Proceedings of the International Conference of the Learning\n  Sciences 2016, Volume 1, pp 607-614",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1704.05579v4",
            "date": "2017-04-19 02:01:39+00:00",
            "title": "A Large Self-Annotated Corpus for Sarcasm",
            "authors": [
                "Mikhail Khodak",
                "Nikunj Saunshi",
                "Kiran Vodrahalli"
            ],
            "abstract": "We introduce the Self-Annotated Reddit Corpus (SARC), a large corpus for\nsarcasm research and for training and evaluating systems for sarcasm detection.\nThe corpus has 1.3 million sarcastic statements -- 10 times more than any\nprevious dataset -- and many times more instances of non-sarcastic statements,\nallowing for learning in both balanced and unbalanced label regimes. Each\nstatement is furthermore self-annotated -- sarcasm is labeled by the author,\nnot an independent annotator -- and provided with user, topic, and conversation\ncontext. We evaluate the corpus for accuracy, construct benchmarks for sarcasm\ndetection, and evaluate baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05907v1",
            "date": "2017-04-19 19:33:38+00:00",
            "title": "End-to-End Multi-View Networks for Text Classification",
            "authors": [
                "Hongyu Guo",
                "Colin Cherry",
                "Jiang Su"
            ],
            "abstract": "We propose a multi-view network for text classification. Our method\nautomatically creates various views of its input text, each taking the form of\nsoft attention weights that distribute the classifier's focus among a set of\nbase features. For a bag-of-words representation, each view focuses on a\ndifferent subset of the text's words. Aggregating many such views results in a\nmore discriminative and robust representation. Through a novel architecture\nthat both stacks and concatenates views, we produce a network that emphasizes\nboth depth and width, allowing training to converge quickly. Using our\nmulti-view architecture, we establish new state-of-the-art accuracies on two\nbenchmark tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.05908v2",
            "date": "2017-04-19 19:35:54+00:00",
            "title": "An Interpretable Knowledge Transfer Model for Knowledge Base Completion",
            "authors": [
                "Qizhe Xie",
                "Xuezhe Ma",
                "Zihang Dai",
                "Eduard Hovy"
            ],
            "abstract": "Knowledge bases are important resources for a variety of natural language\nprocessing tasks but suffer from incompleteness. We propose a novel embedding\nmodel, \\emph{ITransF}, to perform knowledge base completion. Equipped with a\nsparse attention mechanism, ITransF discovers hidden concepts of relations and\ntransfer statistical strength through the sharing of concepts. Moreover, the\nlearned associations between relations and concepts, which are represented by\nsparse attention vectors, can be interpreted easily. We evaluate ITransF on two\nbenchmark datasets---WN18 and FB15k for knowledge base completion and obtains\nimprovements on both the mean rank and Hits@10 metrics, over all baselines that\ndo not use additional information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06194v2",
            "date": "2017-04-20 15:48:05+00:00",
            "title": "Improved Neural Relation Detection for Knowledge Base Question Answering",
            "authors": [
                "Mo Yu",
                "Wenpeng Yin",
                "Kazi Saidul Hasan",
                "Cicero dos Santos",
                "Bing Xiang",
                "Bowen Zhou"
            ],
            "abstract": "Relation detection is a core component for many NLP applications including\nKnowledge Base Question Answering (KBQA). In this paper, we propose a\nhierarchical recurrent neural network enhanced by residual learning that\ndetects KB relations given an input question. Our method uses deep residual\nbidirectional LSTMs to compare questions and relation names via different\nhierarchies of abstraction. Additionally, we propose a simple KBQA system that\nintegrates entity linking and our proposed relation detector to enable one\nenhance another. Experimental results evidence that our approach achieves not\nonly outstanding relation detection performance, but more importantly, it helps\nour KBQA system to achieve state-of-the-art accuracy for both single-relation\n(SimpleQuestions) and multi-relation (WebQSP) QA benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06497v2",
            "date": "2017-04-21 11:56:00+00:00",
            "title": "Bandit Structured Prediction for Neural Sequence-to-Sequence Learning",
            "authors": [
                "Julia Kreutzer",
                "Artem Sokolov",
                "Stefan Riezler"
            ],
            "abstract": "Bandit structured prediction describes a stochastic optimization framework\nwhere learning is performed from partial feedback. This feedback is received in\nthe form of a task loss evaluation to a predicted output structure, without\nhaving access to gold standard structures. We advance this framework by lifting\nlinear bandit learning to neural sequence-to-sequence learning problems using\nattention-based recurrent neural networks. Furthermore, we show how to\nincorporate control variates into our learning algorithms for variance\nreduction and improved generalization. We present an evaluation on a neural\nmachine translation task that shows improvements of up to 5.89 BLEU points for\ndomain adaptation from simulated bandit feedback.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1704.06933v4",
            "date": "2017-04-20 05:08:47+00:00",
            "title": "Adversarial Neural Machine Translation",
            "authors": [
                "Lijun Wu",
                "Yingce Xia",
                "Li Zhao",
                "Fei Tian",
                "Tao Qin",
                "Jianhuang Lai",
                "Tie-Yan Liu"
            ],
            "abstract": "In this paper, we study a new learning paradigm for Neural Machine\nTranslation (NMT). Instead of maximizing the likelihood of the human\ntranslation as in previous works, we minimize the distinction between human\ntranslation and the translation given by an NMT model. To achieve this goal,\ninspired by the recent success of generative adversarial networks (GANs), we\nemploy an adversarial training architecture and name it as Adversarial-NMT. In\nAdversarial-NMT, the training of the NMT model is assisted by an adversary,\nwhich is an elaborately designed Convolutional Neural Network (CNN). The goal\nof the adversary is to differentiate the translation result generated by the\nNMT model from that by human. The goal of the NMT model is to produce high\nquality translations so as to cheat the adversary. A policy gradient method is\nleveraged to co-train the NMT model and the adversary. Experimental results on\nEnglish$\\rightarrow$French and German$\\rightarrow$English translation tasks\nshow that Adversarial-NMT can achieve significantly better translation quality\nthan several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06956v1",
            "date": "2017-04-23 18:13:10+00:00",
            "title": "Naturalizing a Programming Language via Interactive Learning",
            "authors": [
                "Sida I. Wang",
                "Samuel Ginn",
                "Percy Liang",
                "Christoper D. Manning"
            ],
            "abstract": "Our goal is to create a convenient natural language interface for performing\nwell-specified but complex actions such as analyzing data, manipulating text,\nand querying databases. However, existing natural language interfaces for such\ntasks are quite primitive compared to the power one wields with a programming\nlanguage. To bridge this gap, we start with a core programming language and\nallow users to \"naturalize\" the core language incrementally by defining\nalternative, more natural syntax and increasingly complex concepts in terms of\ncompositions of simpler ones. In a voxel world, we show that a community of\nusers can simultaneously teach a common system a diverse language and use it to\nbuild hundreds of complex voxel structures. Over the course of three days,\nthese users went from using only the core language to using the naturalized\nlanguage in 85.9\\% of the last 10K utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.06970v1",
            "date": "2017-04-23 20:05:36+00:00",
            "title": "Differentiable Scheduled Sampling for Credit Assignment",
            "authors": [
                "Kartik Goyal",
                "Chris Dyer",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We demonstrate that a continuous relaxation of the argmax operation can be\nused to create a differentiable approximation to greedy decoding for\nsequence-to-sequence (seq2seq) models. By incorporating this approximation into\nthe scheduled sampling training procedure (Bengio et al., 2015)--a well-known\ntechnique for correcting exposure bias--we introduce a new training objective\nthat is continuous and differentiable everywhere and that can provide\ninformative gradients near points where previous decoding decisions change\ntheir value. In addition, by using a related approximation, we demonstrate a\nsimilar approach to sampled-based training. Finally, we show that our approach\noutperforms cross-entropy training and scheduled sampling procedures in two\nsequence prediction tasks: named entity recognition and machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07050v2",
            "date": "2017-04-24 06:04:50+00:00",
            "title": "Using Global Constraints and Reranking to Improve Cognates Detection",
            "authors": [
                "Michael Bloodgood",
                "Benjamin Strauss"
            ],
            "abstract": "Global constraints and reranking have not been used in cognates detection\nresearch to date. We propose methods for using global constraints by performing\nrescoring of the score matrices produced by state of the art cognates detection\nsystems. Using global constraints to perform rescoring is complementary to\nstate of the art methods for performing cognates detection and results in\nsignificant performance improvements beyond current state of the art\nperformance on publicly available datasets with different language pairs and\nvarious conditions such as different levels of baseline state of the art\nperformance and different data size conditions, including with more realistic\nlarge data size conditions than have been evaluated with in the past.",
            "journal ref": "In Proceedings of the 55th Annual Meeting of the Association for\n  Computational Linguistics, pages 1983-1992, Vancouver, Canada, July 2017.\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07121v2",
            "date": "2017-04-24 10:05:19+00:00",
            "title": "Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets",
            "authors": [
                "Wei-Lun Chao",
                "Hexiang Hu",
                "Fei Sha"
            ],
            "abstract": "Visual question answering (Visual QA) has attracted a lot of attention\nlately, seen essentially as a form of (visual) Turing test that artificial\nintelligence should strive to achieve. In this paper, we study a crucial\ncomponent of this task: how can we design good datasets for the task? We focus\non the design of multiple-choice based datasets where the learner has to select\nthe right answer from a set of candidate ones including the target (\\ie the\ncorrect one) and the decoys (\\ie the incorrect ones). Through careful analysis\nof the results attained by state-of-the-art learning models and human\nannotators on existing datasets, we show that the design of the decoy answers\nhas a significant impact on how and what the learning models learn from the\ndatasets. In particular, the resulting learner can ignore the visual\ninformation, the question, or both while still doing well on the task. Inspired\nby this, we propose automatic procedures to remedy such design deficiencies. We\napply the procedures to re-construct decoy answers for two popular Visual QA\ndatasets as well as to create a new Visual QA dataset from the Visual Genome\nproject, resulting in the largest dataset for this task. Extensive empirical\nstudies show that the design deficiencies have been alleviated in the remedied\ndatasets and the performance on them is likely a more faithful indicator of the\ndifference among learning models. The datasets are released and publicly\navailable via http://www.teds.usc.edu/website_vqa/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07156v1",
            "date": "2017-04-24 11:47:06+00:00",
            "title": "Semi-supervised Multitask Learning for Sequence Labeling",
            "authors": [
                "Marek Rei"
            ],
            "abstract": "We propose a sequence labeling framework with a secondary training objective,\nlearning to predict surrounding words for every word in the dataset. This\nlanguage modeling objective incentivises the system to learn general-purpose\npatterns of semantic and syntactic composition, which are also useful for\nimproving accuracy on different sequence labeling tasks. The architecture was\nevaluated on a range of datasets, covering the tasks of error detection in\nlearner texts, named entity recognition, chunking and POS-tagging. The novel\nlanguage modeling objective provided consistent performance improvements on\nevery benchmark, without requiring any additional annotated or unannotated\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07287v2",
            "date": "2017-04-24 15:33:26+00:00",
            "title": "Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information",
            "authors": [
                "Trang Tran",
                "Shubham Toshniwal",
                "Mohit Bansal",
                "Kevin Gimpel",
                "Karen Livescu",
                "Mari Ostendorf"
            ],
            "abstract": "In conversational speech, the acoustic signal provides cues that help\nlisteners disambiguate difficult parses. For automatically parsing spoken\nutterances, we introduce a model that integrates transcribed text and\nacoustic-prosodic features using a convolutional neural network over energy and\npitch trajectories coupled with an attention-based recurrent neural network\nthat accepts text and prosodic features. We find that different types of\nacoustic-prosodic features are individually helpful, and together give\nstatistically significant improvements in parse and disfluency detection F1\nscores over a strong text-only baseline. For this study with known sentence\nboundaries, error analyses show that the main benefit of acoustic-prosodic\nfeatures is in sentences with disfluencies, attachment decisions are most\nimproved, and transcription errors obscure gains from prosody.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07489v2",
            "date": "2017-04-24 23:07:32+00:00",
            "title": "Multi-Task Video Captioning with Video and Entailment Generation",
            "authors": [
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Video captioning, the task of describing the content of a video, has seen\nsome promising improvements in recent years with sequence-to-sequence models,\nbut accurately learning the temporal and logical dynamics involved in the task\nstill remains a challenge, especially given the lack of sufficient annotated\ndata. We improve video captioning by sharing knowledge with two related\ndirected-generation tasks: a temporally-directed unsupervised video prediction\ntask to learn richer context-aware video encoder representations, and a\nlogically-directed language entailment generation task to learn better\nvideo-entailed caption decoder representations. For this, we present a\nmany-to-many multi-task learning model that shares parameters across the\nencoders and decoders of the three tasks. We achieve significant improvements\nand the new state-of-the-art on several standard video captioning datasets\nusing diverse automatic and human evaluations. We also show mutual multi-task\nimprovements on the entailment generation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07535v1",
            "date": "2017-04-25 04:37:35+00:00",
            "title": "Abstract Syntax Networks for Code Generation and Semantic Parsing",
            "authors": [
                "Maxim Rabinovich",
                "Mitchell Stern",
                "Dan Klein"
            ],
            "abstract": "Tasks like code generation and semantic parsing require mapping unstructured\n(or partially structured) inputs to well-formed, executable outputs. We\nintroduce abstract syntax networks, a modeling framework for these problems.\nThe outputs are represented as abstract syntax trees (ASTs) and constructed by\na decoder with a dynamically-determined modular structure paralleling the\nstructure of the output tree. On the benchmark Hearthstone dataset for code\ngeneration, our model obtains 79.2 BLEU and 22.7% exact match accuracy,\ncompared to previous state-of-the-art values of 67.1 and 6.1%. Furthermore, we\nperform competitively on the Atis, Jobs, and Geo semantic parsing datasets with\nno task-specific engineering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07624v2",
            "date": "2017-04-25 10:45:43+00:00",
            "title": "280 Birds with One Stone: Inducing Multilingual Taxonomies from Wikipedia using Character-level Classification",
            "authors": [
                "Amit Gupta",
                "R\u00e9mi Lebret",
                "Hamza Harkous",
                "Karl Aberer"
            ],
            "abstract": "We propose a simple, yet effective, approach towards inducing multilingual\ntaxonomies from Wikipedia. Given an English taxonomy, our approach leverages\nthe interlanguage links of Wikipedia followed by character-level classifiers to\ninduce high-precision, high-coverage taxonomies in other languages. Through\nexperiments, we demonstrate that our approach significantly outperforms the\nstate-of-the-art, heuristics-heavy approaches for six languages. As a\nconsequence of our work, we release presumably the largest and the most\naccurate multilingual taxonomic resource spanning over 280 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07626v4",
            "date": "2017-04-25 10:49:53+00:00",
            "title": "Taxonomy Induction using Hypernym Subsequences",
            "authors": [
                "Amit Gupta",
                "R\u00e9mi Lebret",
                "Hamza Harkous",
                "Karl Aberer"
            ],
            "abstract": "We propose a novel, semi-supervised approach towards domain taxonomy\ninduction from an input vocabulary of seed terms. Unlike all previous\napproaches, which typically extract direct hypernym edges for terms, our\napproach utilizes a novel probabilistic framework to extract hypernym\nsubsequences. Taxonomy induction from extracted subsequences is cast as an\ninstance of the minimumcost flow problem on a carefully designed directed\ngraph. Through experiments, we demonstrate that our approach outperforms\nstateof- the-art taxonomy induction approaches across four languages.\nImportantly, we also show that our approach is robust to the presence of noise\nin the input vocabulary. To the best of our knowledge, no previous approaches\nhave been empirically proven to manifest noise-robustness in the input\nvocabulary.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1704.07734v1",
            "date": "2017-04-25 15:09:51+00:00",
            "title": "DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning",
            "authors": [
                "Xiaodong Gu",
                "Hongyu Zhang",
                "Dongmei Zhang",
                "Sunghun Kim"
            ],
            "abstract": "Computer programs written in one language are often required to be ported to\nother languages to support multiple devices and environments. When programs use\nlanguage specific APIs (Application Programming Interfaces), it is very\nchallenging to migrate these APIs to the corresponding APIs written in other\nlanguages. Existing approaches mine API mappings from projects that have\ncorresponding versions in two languages. They rely on the sparse availability\nof bilingual projects, thus producing a limited number of API mappings. In this\npaper, we propose an intelligent system called DeepAM for automatically mining\nAPI mappings from a large-scale code corpus without bilingual projects. The key\ncomponent of DeepAM is based on the multimodal sequence to sequence learning\narchitecture that aims to learn joint semantic representations of bilingual API\nsequences from big source code data. Experimental results indicate that DeepAM\nsignificantly increases the accuracy of API mappings as well as the number of\nAPI mappings, when compared with the state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1704.07828v2",
            "date": "2017-04-25 18:00:01+00:00",
            "title": "Friendships, Rivalries, and Trysts: Characterizing Relations between Ideas in Texts",
            "authors": [
                "Chenhao Tan",
                "Dallas Card",
                "Noah A. Smith"
            ],
            "abstract": "Understanding how ideas relate to each other is a fundamental question in\nmany domains, ranging from intellectual history to public communication.\nBecause ideas are naturally embedded in texts, we propose the first framework\nto systematically characterize the relations between ideas based on their\noccurrence in a corpus of documents, independent of how these ideas are\nrepresented. Combining two statistics --- cooccurrence within documents and\nprevalence correlation over time --- our approach reveals a number of different\nways in which ideas can cooperate and compete. For instance, two ideas can\nclosely track each other's prevalence over time, and yet rarely cooccur, almost\nlike a \"cold war\" scenario. We observe that pairwise cooccurrence and\nprevalence correlation exhibit different distributions. We further demonstrate\nthat our approach is able to uncover intriguing relations between ideas through\nin-depth case studies on news articles and research papers.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1704.08092v1",
            "date": "2017-04-26 13:10:12+00:00",
            "title": "A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations",
            "authors": [
                "Samuel R\u00f6nnqvist",
                "Niko Schenk",
                "Christian Chiarcos"
            ],
            "abstract": "We introduce an attention-based Bi-LSTM for Chinese implicit discourse\nrelations and demonstrate that modeling argument pairs as a joint sequence can\noutperform word order-agnostic approaches. Our model benefits from a partial\nsampling scheme and is conceptually simple, yet achieves state-of-the-art\nperformance on the Chinese Discourse Treebank. We also visualize its attention\nactivity to illustrate the model's ability to selectively focus on the relevant\nparts of an input sequence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08224v2",
            "date": "2017-04-26 17:22:53+00:00",
            "title": "Punny Captions: Witty Wordplay in Image Descriptions",
            "authors": [
                "Arjun Chandrasekaran",
                "Devi Parikh",
                "Mohit Bansal"
            ],
            "abstract": "Wit is a form of rich interaction that is often grounded in a specific\nsituation (e.g., a comment in response to an event). In this work, we attempt\nto build computational models that can produce witty descriptions for a given\nimage. Inspired by a cognitive account of humor appreciation, we employ\nlinguistic wordplay, specifically puns, in image descriptions. We develop two\napproaches which involve retrieving witty descriptions for a given image from a\nlarge corpus of sentences, or generating them via an encoder-decoder neural\nnetwork architecture. We compare our approach against meaningful baseline\napproaches via human studies and show substantial improvements. We find that\nwhen a human is subject to similar constraints as the model regarding word\nusage and style, people vote the image descriptions generated by our model to\nbe slightly wittier than human-written witty descriptions. Unsurprisingly,\nhumans are almost always wittier than the model when they are free to choose\nthe vocabulary, style, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.08243v1",
            "date": "2017-04-26 17:57:59+00:00",
            "title": "C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset",
            "authors": [
                "Aishwarya Agrawal",
                "Aniruddha Kembhavi",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "Visual Question Answering (VQA) has received a lot of attention over the past\ncouple of years. A number of deep learning models have been proposed for this\ntask. However, it has been shown that these models are heavily driven by\nsuperficial correlations in the training data and lack compositionality -- the\nability to answer questions about unseen compositions of seen concepts. This\ncompositionality is desirable and central to intelligence. In this paper, we\npropose a new setting for Visual Question Answering where the test\nquestion-answer pairs are compositionally novel compared to training\nquestion-answer pairs. To facilitate developing models under this setting, we\npresent a new compositional split of the VQA v1.0 dataset, which we call\nCompositional VQA (C-VQA). We analyze the distribution of questions and answers\nin the C-VQA splits. Finally, we evaluate several existing VQA models under\nthis new setting and show that the performances of these models degrade by a\nsignificant amount compared to the original VQA setting.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1704.08424v2",
            "date": "2017-04-27 03:59:54+00:00",
            "title": "Multimodal Word Distributions",
            "authors": [
                "Ben Athiwaratkun",
                "Andrew Gordon Wilson"
            ],
            "abstract": "Word embeddings provide point representations of words containing useful\nsemantic information. We introduce multimodal word distributions formed from\nGaussian mixtures, for multiple word meanings, entailment, and rich uncertainty\ninformation. To learn these distributions, we propose an energy-based\nmax-margin objective. We show that the resulting approach captures uniquely\nexpressive semantic information, and outperforms alternatives, such as word2vec\nskip-grams, and Gaussian embeddings, on benchmark datasets such as word\nsimilarity and entailment.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1704.08803v2",
            "date": "2017-04-28 04:08:47+00:00",
            "title": "Neural Ranking Models with Weak Supervision",
            "authors": [
                "Mostafa Dehghani",
                "Hamed Zamani",
                "Aliaksei Severyn",
                "Jaap Kamps",
                "W. Bruce Croft"
            ],
            "abstract": "Despite the impressive improvements achieved by unsupervised deep neural\nnetworks in computer vision and NLP tasks, such improvements have not yet been\nobserved in ranking for information retrieval. The reason may be the complexity\nof the ranking problem, as it is not obvious how to learn from queries and\ndocuments when no supervised signal is available. Hence, in this paper, we\npropose to train a neural ranking model using weak supervision, where labels\nare obtained automatically without human annotators or any external resources\n(e.g., click data). To this aim, we use the output of an unsupervised ranking\nmodel, such as BM25, as a weak supervision signal. We further train a set of\nsimple yet effective ranking models based on feed-forward neural networks. We\nstudy their effectiveness under various learning scenarios (point-wise and\npair-wise models) and using different input representations (i.e., from\nencoding query-document pairs into dense/sparse vectors to using word embedding\nrepresentation). We train our networks using tens of millions of training\ninstances and evaluate it on two standard collections: a homogeneous news\ncollection(Robust) and a heterogeneous large-scale web collection (ClueWeb).\nOur experiments indicate that employing proper objective functions and letting\nthe networks to learn the input representation based on weakly supervised data\nleads to impressive performance, with over 13% and 35% MAP improvements over\nthe BM25 model on the Robust and the ClueWeb collections. Our findings also\nsuggest that supervised neural ranking models can greatly benefit from\npre-training on large amounts of weakly labeled data that can be easily\nobtained from unsupervised IR models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1704.08914v2",
            "date": "2017-04-28 13:11:09+00:00",
            "title": "Past, Present, Future: A Computational Investigation of the Typology of Tense in 1000 Languages",
            "authors": [
                "Ehsaneddin Asgari",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present SuperPivot, an analysis method for low-resource languages that\noccur in a superparallel corpus, i.e., in a corpus that contains an order of\nmagnitude more languages than parallel corpora currently in use. We show that\nSuperPivot performs well for the crosslingual analysis of the linguistic\nphenomenon of tense. We produce analysis results for more than 1000 languages,\nconducting - to the best of our knowledge - the largest crosslingual\ncomputational study performed to date. We extend existing methodology for\nleveraging parallel corpora for typological analysis by overcoming a limiting\nassumption of earlier work: We only require that a linguistic feature is\novertly marked in a few of thousands of languages as opposed to requiring that\nit be marked in all languages under investigation.",
            "journal ref": "Extended version of EMNLP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00130v2",
            "date": "2017-06-01 00:24:55+00:00",
            "title": "Teaching Machines to Describe Images via Natural Language Feedback",
            "authors": [
                "Huan Ling",
                "Sanja Fidler"
            ],
            "abstract": "Robots will eventually be part of every household. It is thus critical to\nenable algorithms to learn from and be guided by non-expert users. In this\npaper, we bring a human in the loop, and enable a human teacher to give\nfeedback to a learning agent in the form of natural language. We argue that a\ndescriptive sentence can provide a much stronger learning signal than a numeric\nreward in that it can easily point to where the mistakes are and how to correct\nthem. We focus on the problem of image captioning in which the quality of the\noutput can easily be judged by non-experts. We propose a hierarchical\nphrase-based captioning model trained with policy gradients, and design a\nfeedback network that provides reward to the learner by conditioning on the\nhuman-provided feedback. We show that by exploiting descriptive feedback our\nmodel learns to perform better than when given independently written human\ncaptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00290v1",
            "date": "2017-06-01 13:33:54+00:00",
            "title": "Transfer Learning for Speech Recognition on a Budget",
            "authors": [
                "Julius Kunze",
                "Louis Kirsch",
                "Ilia Kurenkov",
                "Andreas Krug",
                "Jens Johannsmeier",
                "Sebastian Stober"
            ],
            "abstract": "End-to-end training of automated speech recognition (ASR) systems requires\nmassive data and compute resources. We explore transfer learning based on model\nadaptation as an approach for training ASR models under constrained GPU memory,\nthroughput and training data. We conduct several systematic experiments\nadapting a Wav2Letter convolutional neural network originally trained for\nEnglish ASR to the German language. We show that this technique allows faster\ntraining on consumer-grade resources while requiring less training data in\norder to achieve the same accuracy, thereby lowering the cost of training ASR\nmodels in other languages. Model introspection revealed that small adaptations\nto the network's weights were sufficient for good performance, especially for\ninner layers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.00359v2",
            "date": "2017-06-01 15:55:42+00:00",
            "title": "Discovering Discrete Latent Topics with Neural Variational Inference",
            "authors": [
                "Yishu Miao",
                "Edward Grefenstette",
                "Phil Blunsom"
            ],
            "abstract": "Topic models have been widely explored as probabilistic generative models of\ndocuments. Traditional inference methods have sought closed-form derivations\nfor updating the models, however as the expressiveness of these models grows,\nso does the difficulty of performing fast and accurate inference over their\nparameters. This paper presents alternative neural approaches to topic\nmodelling by providing parameterisable distributions over topics which permit\ntraining by backpropagation in the framework of neural variational inference.\nIn addition, with the help of a stick-breaking construction, we propose a\nrecurrent network that is able to discover a notionally unbounded number of\ntopics, analogous to Bayesian non-parametric topic models. Experimental results\non the MXM Song Lyrics, 20NewsGroups and Reuters News datasets demonstrate the\neffectiveness and efficiency of these neural topic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00374v1",
            "date": "2017-06-01 16:29:47+00:00",
            "title": "Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints",
            "authors": [
                "Nikola Mrk\u0161i\u0107",
                "Ivan Vuli\u0107",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Ira Leviant",
                "Roi Reichart",
                "Milica Ga\u0161i\u0107",
                "Anna Korhonen",
                "Steve Young"
            ],
            "abstract": "We present Attract-Repel, an algorithm for improving the semantic quality of\nword vectors by injecting constraints extracted from lexical resources.\nAttract-Repel facilitates the use of constraints from mono- and cross-lingual\nresources, yielding semantically specialised cross-lingual vector spaces. Our\nevaluation shows that the method can make use of existing cross-lingual\nlexicons to construct high-quality vector spaces for a plethora of different\nlanguages, facilitating semantic transfer from high- to lower-resource ones.\nThe effectiveness of our approach is demonstrated with state-of-the-art results\non semantic similarity datasets in six languages. We next show that\nAttract-Repel-specialised vectors boost performance in the downstream task of\ndialogue state tracking (DST) across multiple languages. Finally, we show that\ncross-lingual vector spaces produced by our algorithm facilitate the training\nof multilingual DST models, which brings further performance improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00593v1",
            "date": "2017-06-02 08:52:47+00:00",
            "title": "Joint Modeling of Topics, Citations, and Topical Authority in Academic Corpora",
            "authors": [
                "Jooyeon Kim",
                "Dongwoo Kim",
                "Alice Oh"
            ],
            "abstract": "Much of scientific progress stems from previously published findings, but\nsearching through the vast sea of scientific publications is difficult. We\noften rely on metrics of scholarly authority to find the prominent authors but\nthese authority indices do not differentiate authority based on research\ntopics. We present Latent Topical-Authority Indexing (LTAI) for jointly\nmodeling the topics, citations, and topical authority in a corpus of academic\npapers. Compared to previous models, LTAI differs in two main aspects. First,\nit explicitly models the generative process of the citations, rather than\ntreating the citations as given. Second, it models each author's influence on\ncitations of a paper based on the topics of the cited papers, as well as the\nciting papers. We fit LTAI to four academic corpora: CORA, Arxiv Physics, PNAS,\nand Citeseer. We compare the performance of LTAI against various baselines,\nstarting with the latent Dirichlet allocation, to the more advanced models\nincluding author-link topic model and dynamic author citation topic model. The\nresults show that LTAI achieves improved accuracy over other similar models\nwhen predicting words, citations and authors of publications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00884v1",
            "date": "2017-06-03 02:15:44+00:00",
            "title": "Task-specific Word Identification from Short Texts Using a Convolutional Neural Network",
            "authors": [
                "Shuhan Yuan",
                "Xintao Wu",
                "Yang Xiang"
            ],
            "abstract": "Task-specific word identification aims to choose the task-related words that\nbest describe a short text. Existing approaches require well-defined seed words\nor lexical dictionaries (e.g., WordNet), which are often unavailable for many\napplications such as social discrimination detection and fake review detection.\nHowever, we often have a set of labeled short texts where each short text has a\ntask-related class label, e.g., discriminatory or non-discriminatory, specified\nby users or learned by classification algorithms. In this paper, we focus on\nidentifying task-specific words and phrases from short texts by exploiting\ntheir class labels rather than using seed words or lexical dictionaries. We\nconsider the task-specific word and phrase identification as feature learning.\nWe train a convolutional neural network over a set of labeled texts and use\nscore vectors to localize the task-specific words and phrases. Experimental\nresults on sentiment word identification show that our approach significantly\noutperforms existing methods. We further conduct two case studies to show the\neffectiveness of our approach. One case study on a crawled tweets dataset\ndemonstrates that our approach can successfully capture the\ndiscrimination-related words/phrases. The other case study on fake review\ndetection shows that our approach can identify the fake-review words/phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.00887v1",
            "date": "2017-06-03 02:42:40+00:00",
            "title": "Wikipedia Vandal Early Detection: from User Behavior to User Embedding",
            "authors": [
                "Shuhan Yuan",
                "Panpan Zheng",
                "Xintao Wu",
                "Yang Xiang"
            ],
            "abstract": "Wikipedia is the largest online encyclopedia that allows anyone to edit\narticles. In this paper, we propose the use of deep learning to detect vandals\nbased on their edit history. In particular, we develop a multi-source\nlong-short term memory network (M-LSTM) to model user behaviors by using a\nvariety of user edit aspects as inputs, including the history of edit reversion\ninformation, edit page titles and categories. With M-LSTM, we can encode each\nuser into a low dimensional real vector, called user embedding. Meanwhile, as a\nsequential model, M-LSTM updates the user embedding each time after the user\ncommits a new edit. Thus, we can predict whether a user is benign or vandal\ndynamically based on the up-to-date user embedding. Furthermore, those user\nembeddings are crucial to discover collaborative vandals.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1706.01084v2",
            "date": "2017-06-04 14:48:28+00:00",
            "title": "Joint Text Embedding for Personalized Content-based Recommendation",
            "authors": [
                "Ting Chen",
                "Liangjie Hong",
                "Yue Shi",
                "Yizhou Sun"
            ],
            "abstract": "Learning a good representation of text is key to many recommendation\napplications. Examples include news recommendation where texts to be\nrecommended are constantly published everyday. However, most existing\nrecommendation techniques, such as matrix factorization based methods, mainly\nrely on interaction histories to learn representations of items. While latent\nfactors of items can be learned effectively from user interaction data, in many\ncases, such data is not available, especially for newly emerged items.\n  In this work, we aim to address the problem of personalized recommendation\nfor completely new items with text information available. We cast the problem\nas a personalized text ranking problem and propose a general framework that\ncombines text embedding with personalized recommendation. Users and textual\ncontent are embedded into latent feature space. The text embedding function can\nbe learned end-to-end by predicting user interactions with items. To alleviate\nsparsity in interaction data, and leverage large amount of text data with\nlittle or no user interactions, we further propose a joint text embedding model\nthat incorporates unsupervised text embedding with a combination module.\nExperimental results show that our model can significantly improve the\neffectiveness of recommendation systems on real-world datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1706.01322v2",
            "date": "2017-06-05 13:53:56+00:00",
            "title": "Deep learning evaluation using deep linguistic processing",
            "authors": [
                "Alexander Kuhnle",
                "Ann Copestake"
            ],
            "abstract": "We discuss problems with the standard approaches to evaluation for tasks like\nvisual question answering, and argue that artificial data can be used to\naddress these as a complement to current practice. We demonstrate that with the\nhelp of existing 'deep' linguistic processing technology we are able to create\nchallenging abstract datasets, which enable us to investigate the language\nunderstanding abilities of multimodal deep learning models in detail, as\ncompared to a single performance value on a static and monolithic dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01331v3",
            "date": "2017-06-05 14:04:48+00:00",
            "title": "Event Representations for Automated Story Generation with Deep Neural Nets",
            "authors": [
                "Lara J. Martin",
                "Prithviraj Ammanabrolu",
                "Xinyu Wang",
                "William Hancock",
                "Shruti Singh",
                "Brent Harrison",
                "Mark O. Riedl"
            ],
            "abstract": "Automated story generation is the problem of automatically selecting a\nsequence of events, actions, or words that can be told as a story. We seek to\ndevelop a system that can generate stories by learning everything it needs to\nknow from textual story corpora. To date, recurrent neural networks that learn\nlanguage models at character, word, or sentence levels have had little success\ngenerating coherent stories. We explore the question of event representations\nthat provide a mid-level of abstraction between words and sentences in order to\nretain the semantic information of the original data while minimizing event\nsparsity. We present a technique for preprocessing textual story data into\nevent sequences. We then present a technique for automated story generation\nwhereby we decompose the problem into the generation of successive events\n(event2event) and the generation of natural language sentences from events\n(event2sentence). We give empirical results comparing different event\nrepresentations and their effects on event successor generation and the\ntranslation of events to natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01450v1",
            "date": "2017-06-05 17:58:52+00:00",
            "title": "A Joint Model for Question Answering and Question Generation",
            "authors": [
                "Tong Wang",
                "Xingdi Yuan",
                "Adam Trischler"
            ],
            "abstract": "We propose a generative machine comprehension model that learns jointly to\nask and answer questions based on documents. The proposed model uses a\nsequence-to-sequence framework that encodes the document and generates a\nquestion (answer) given an answer (question). Significant improvement in model\nperformance is observed empirically on the SQuAD corpus, confirming our\nhypothesis that the model benefits from jointly learning to perform both tasks.\nWe believe the joint model's novelty offers a new perspective on machine\ncomprehension beyond architectural engineering, and serves as a first step\ntowards autonomous information seeking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01554v2",
            "date": "2017-06-05 22:50:37+00:00",
            "title": "Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model",
            "authors": [
                "Jiasen Lu",
                "Anitha Kannan",
                "Jianwei Yang",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We present a novel training framework for neural sequence models,\nparticularly for grounded dialog generation. The standard training paradigm for\nthese models is maximum likelihood estimation (MLE), or minimizing the\ncross-entropy of the human responses. Across a variety of domains, a recurring\nproblem with MLE trained generative neural dialog models (G) is that they tend\nto produce 'safe' and generic responses (\"I don't know\", \"I can't tell\"). In\ncontrast, discriminative dialog models (D) that are trained to rank a list of\ncandidate human responses outperform their generative counterparts; in terms of\nautomatic metrics, diversity, and informativeness of the responses. However, D\nis not useful in practice since it cannot be deployed to have real\nconversations with users.\n  Our work aims to achieve the best of both worlds -- the practical usefulness\nof G and the strong performance of D -- via knowledge transfer from D to G. Our\nprimary contribution is an end-to-end trainable generative visual dialog model,\nwhere G receives gradients from D as a perceptual (not adversarial) loss of the\nsequence sampled from G. We leverage the recently proposed Gumbel-Softmax (GS)\napproximation to the discrete distribution -- specifically, an RNN augmented\nwith a sequence of GS samplers, coupled with the straight-through gradient\nestimator to enable end-to-end differentiability. We also introduce a stronger\nencoder for visual dialog, and employ a self-attention mechanism for answer\nencoding along with a metric learning loss to aid D in better capturing\nsemantic similarities in answer responses. Overall, our proposed model\noutperforms state-of-the-art on the VisDial dataset by a significant margin\n(2.67% on recall@10). The source code can be downloaded from\nhttps://github.com/jiasenlu/visDial.pytorch.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1706.01556v2",
            "date": "2017-06-05 23:09:06+00:00",
            "title": "Deep learning for extracting protein-protein interactions from biomedical literature",
            "authors": [
                "Yifan Peng",
                "Zhiyong Lu"
            ],
            "abstract": "State-of-the-art methods for protein-protein interaction (PPI) extraction are\nprimarily feature-based or kernel-based by leveraging lexical and syntactic\ninformation. But how to incorporate such knowledge in the recent deep learning\nmethods remains an open question. In this paper, we propose a multichannel\ndependency-based convolutional neural network model (McDepCNN). It applies one\nchannel to the embedding vector of each word in the sentence, and another\nchannel to the embedding vector of the head of the corresponding word.\nTherefore, the model can use richer information obtained from different\nchannels. Experiments on two public benchmarking datasets, AIMed and BioInfer,\ndemonstrate that McDepCNN compares favorably to the state-of-the-art\nrich-feature and single-kernel based methods. In addition, McDepCNN achieves\n24.4% relative improvement in F1-score over the state-of-the-art methods on\ncross-corpus evaluation and 12% improvement in F1-score over kernel-based\nmethods on \"difficult\" instances. These results suggest that McDepCNN\ngeneralizes more easily over different corpora, and is capable of capturing\nlong distance features in the sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01875v2",
            "date": "2017-06-06 21:29:23+00:00",
            "title": "Measuring Offensive Speech in Online Political Discourse",
            "authors": [
                "Rishab Nithyanand",
                "Brian Schaffner",
                "Phillipa Gill"
            ],
            "abstract": "The Internet and online forums such as Reddit have become an increasingly\npopular medium for citizens to engage in political conversations. However, the\nonline disinhibition effect resulting from the ability to use pseudonymous\nidentities may manifest in the form of offensive speech, consequently making\npolitical discussions more aggressive and polarizing than they already are.\nSuch environments may result in harassment and self-censorship from its\ntargets. In this paper, we present preliminary results from a large-scale\ntemporal measurement aimed at quantifying offensiveness in online political\ndiscussions.\n  To enable our measurements, we develop and evaluate an offensive speech\nclassifier. We then use this classifier to quantify and compare offensiveness\nin the political and general contexts. We perform our study using a database of\nover 168M Reddit comments made by over 7M pseudonyms between January 2015 and\nJanuary 2017 -- a period covering several divisive political events including\nthe 2016 US presidential elections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02124v2",
            "date": "2017-06-07 10:50:47+00:00",
            "title": "Semi-Supervised Phoneme Recognition with Recurrent Ladder Networks",
            "authors": [
                "Marian Tietz",
                "Tayfun Alpay",
                "Johannes Twiefel",
                "Stefan Wermter"
            ],
            "abstract": "Ladder networks are a notable new concept in the field of semi-supervised\nlearning by showing state-of-the-art results in image recognition tasks while\nbeing compatible with many existing neural architectures. We present the\nrecurrent ladder network, a novel modification of the ladder network, for\nsemi-supervised learning of recurrent neural networks which we evaluate with a\nphoneme recognition task on the TIMIT corpus. Our results show that the model\nis able to consistently outperform the baseline and achieve fully-supervised\nbaseline performance with only 75% of all labels which demonstrates that the\nmodel is capable of using unsupervised data as an effective regulariser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02222v1",
            "date": "2017-06-07 15:05:39+00:00",
            "title": "Gated Recurrent Neural Tensor Network",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Ruli Manurung",
                "Mirna Adriani",
                "Satoshi Nakamura"
            ],
            "abstract": "Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling\ntemporal and sequential data need to capture long-term dependencies on datasets\nand represent them in hidden layers with a powerful model to capture more\ninformation from inputs. For modeling long-term dependencies in a dataset, the\ngating mechanism concept can help RNNs remember and forget previous\ninformation. Representing the hidden layers of an RNN with more expressive\noperations (i.e., tensor products) helps it learn a more complex relationship\nbetween the current input and the previous hidden layer information. These\nideas can generally improve RNN performances. In this paper, we proposed a\nnovel RNN architecture that combine the concepts of gating mechanism and the\ntensor product into a single model. By combining these two concepts into a\nsingle RNN, our proposed models learn long-term dependencies by modeling with\ngating units and obtain more expressive and direct interaction between input\nand hidden layers using a tensor product on 3-dimensional array (tensor) weight\nparameters. We use Long Short Term Memory (LSTM) RNN and Gated Recurrent Unit\n(GRU) RNN and combine them with a tensor product inside their formulations. Our\nproposed RNNs, which are called a Long-Short Term Memory Recurrent Neural\nTensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor\nNetwork (GRURNTN), are made by combining the LSTM and GRU RNN models with the\ntensor product. We conducted experiments with our proposed models on word-level\nand character-level language modeling tasks and revealed that our proposed\nmodels significantly improved their performance compared to our baseline\nmodels.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.02496v1",
            "date": "2017-06-08 09:56:11+00:00",
            "title": "Context encoders as a simple but powerful extension of word2vec",
            "authors": [
                "Franziska Horn"
            ],
            "abstract": "With a simple architecture and the ability to learn meaningful word\nembeddings efficiently from texts containing billions of words, word2vec\nremains one of the most popular neural language models used today. However, as\nonly a single embedding is learned for every word in the vocabulary, the model\nfails to optimally represent words with multiple meanings. Additionally, it is\nnot possible to create embeddings for new (out-of-vocabulary) words on the\nspot. Based on an intuitive interpretation of the continuous bag-of-words\n(CBOW) word2vec model's negative sampling training objective in terms of\npredicting context based similarities, we motivate an extension of the model we\ncall context encoders (ConEc). By multiplying the matrix of trained word2vec\nembeddings with a word's average context vector, out-of-vocabulary (OOV)\nembeddings and representations for a word with multiple meanings can be created\nbased on the word's local contexts. The benefits of this approach are\nillustrated by using these word embeddings as features in the CoNLL 2003 named\nentity recognition (NER) task.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1706.02596v3",
            "date": "2017-06-08 14:10:22+00:00",
            "title": "Dynamic Integration of Background Knowledge in Neural NLU Systems",
            "authors": [
                "Dirk Weissenborn",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Chris Dyer"
            ],
            "abstract": "Common-sense and background knowledge is required to understand natural\nlanguage, but in most neural natural language understanding (NLU) systems, this\nknowledge must be acquired from training corpora during learning, and then it\nis static at test time. We introduce a new architecture for the dynamic\nintegration of explicit background knowledge in NLU models. A general-purpose\nreading module reads background knowledge in the form of free-text statements\n(together with task-specific text inputs) and yields refined word\nrepresentations to a task-specific NLU architecture that reprocesses the task\ninputs with these representations. Experiments on document question answering\n(DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness\nand flexibility of the approach. Analysis shows that our model learns to\nexploit knowledge in a semantically appropriate way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02757v1",
            "date": "2017-06-08 20:33:00+00:00",
            "title": "Sympathy Begins with a Smile, Intelligence Begins with a Word: Use of Multimodal Features in Spoken Human-Robot Interaction",
            "authors": [
                "Jekaterina Novikova",
                "Christian Dondrup",
                "Ioannis Papaioannou",
                "Oliver Lemon"
            ],
            "abstract": "Recognition of social signals, from human facial expressions or prosody of\nspeech, is a popular research topic in human-robot interaction studies. There\nis also a long line of research in the spoken dialogue community that\ninvestigates user satisfaction in relation to dialogue characteristics.\nHowever, very little research relates a combination of multimodal social\nsignals and language features detected during spoken face-to-face human-robot\ninteraction to the resulting user perception of a robot. In this paper we show\nhow different emotional facial expressions of human users, in combination with\nprosodic characteristics of human speech and features of human-robot dialogue,\ncorrelate with users' impressions of the robot after a conversation. We find\nthat happiness in the user's recognised facial expression strongly correlates\nwith likeability of a robot, while dialogue-related features (such as number of\nhuman turns or number of sentences per robot utterance) correlate with\nperceiving a robot as intelligent. In addition, we show that facial expression,\nemotional features, and prosody are better predictors of human ratings related\nto perceived robot likeability and anthropomorphism, while linguistic and\nnon-linguistic features more often predict perceived robot intelligence and\ninterpretability. As such, these characteristics may in future be used as an\nonline reward signal for in-situ Reinforcement Learning based adaptive\nhuman-robot dialogue systems.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1706.02776v1",
            "date": "2017-06-08 21:14:48+00:00",
            "title": "Optimizing expected word error rate via sampling for speech recognition",
            "authors": [
                "Matt Shannon"
            ],
            "abstract": "State-level minimum Bayes risk (sMBR) training has become the de facto\nstandard for sequence-level training of speech recognition acoustic models. It\nhas an elegant formulation using the expectation semiring, and gives large\nimprovements in word error rate (WER) over models trained solely using\ncross-entropy (CE) or connectionist temporal classification (CTC). sMBR\ntraining optimizes the expected number of frames at which the reference and\nhypothesized acoustic states differ. It may be preferable to optimize the\nexpected WER, but WER does not interact well with the expectation semiring, and\nprevious approaches based on computing expected WER exactly involve expanding\nthe lattices used during training. In this paper we show how to perform\noptimization of the expected WER by sampling paths from the lattices used\nduring conventional sMBR training. The gradient of the expected WER is itself\nan expectation, and so may be approximated using Monte Carlo sampling. We show\nexperimentally that optimizing WER during acoustic model training gives 5%\nrelative improvement in WER over a well-tuned sMBR baseline on a 2-channel\nquery recognition task (Google Home).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02901v2",
            "date": "2017-06-07 15:17:21+00:00",
            "title": "Characterizing Types of Convolution in Deep Convolutional Recurrent Neural Networks for Robust Speech Emotion Recognition",
            "authors": [
                "Che-Wei Huang",
                "Shrikanth. S. Narayanan"
            ],
            "abstract": "Deep convolutional neural networks are being actively investigated in a wide\nrange of speech and audio processing applications including speech recognition,\naudio event detection and computational paralinguistics, owing to their ability\nto reduce factors of variations, for learning from speech. However, studies\nhave suggested to favor a certain type of convolutional operations when\nbuilding a deep convolutional neural network for speech applications although\nthere has been promising results using different types of convolutional\noperations. In this work, we study four types of convolutional operations on\ndifferent input features for speech emotion recognition under noisy and clean\nconditions in order to derive a comprehensive understanding. Since affective\nbehavioral information has been shown to reflect temporally varying of mental\nstate and convolutional operation are applied locally in time, all deep neural\nnetworks share a deep recurrent sub-network architecture for further temporal\nmodeling. We present detailed quantitative module-wise performance analysis to\ngain insights into information flows within the proposed architectures. In\nparticular, we demonstrate the interplay of affective information and the other\nirrelevant information during the progression from one module to another.\nFinally we show that all of our deep neural networks provide state-of-the-art\nperformance on the eNTERFACE'05 corpus.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.03146v1",
            "date": "2017-06-09 22:39:31+00:00",
            "title": "Rethinking Skip-thought: A Neighborhood based Approach",
            "authors": [
                "Shuai Tang",
                "Hailin Jin",
                "Chen Fang",
                "Zhaowen Wang",
                "Virginia R. de Sa"
            ],
            "abstract": "We study the skip-thought model with neighborhood information as weak\nsupervision. More specifically, we propose a skip-thought neighbor model to\nconsider the adjacent sentences as a neighborhood. We train our skip-thought\nneighbor model on a large corpus with continuous sentences, and then evaluate\nthe trained model on 7 tasks, which include semantic relatedness, paraphrase\ndetection, and classification benchmarks. Both quantitative comparison and\nqualitative investigation are conducted. We empirically show that, our\nskip-thought neighbor model performs as well as the skip-thought model on\nevaluation tasks. In addition, we found that, incorporating an autoencoder path\nin our model didn't aid our model to perform better, while it hurts the\nperformance of the skip-thought model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03610v2",
            "date": "2017-06-12 13:08:21+00:00",
            "title": "Neural Domain Adaptation for Biomedical Question Answering",
            "authors": [
                "Georg Wiese",
                "Dirk Weissenborn",
                "Mariana Neves"
            ],
            "abstract": "Factoid question answering (QA) has recently benefited from the development\nof deep learning (DL) systems. Neural network models outperform traditional\napproaches in domains where large datasets exist, such as SQuAD (ca. 100,000\nquestions) for Wikipedia articles. However, these systems have not yet been\napplied to QA in more specific domains, such as biomedicine, because datasets\nare generally too small to train a DL system from scratch. For example, the\nBioASQ dataset for biomedical QA comprises less then 900 factoid (single\nanswer) and list (multiple answers) QA instances. In this work, we adapt a\nneural QA system trained on a large open-domain dataset (SQuAD, source) to a\nbiomedical dataset (BioASQ, target) by employing various transfer learning\ntechniques. Our network architecture is based on a state-of-the-art QA system,\nextended with biomedical word embeddings and a novel mechanism to answer list\nquestions. In contrast to existing biomedical QA systems, our system does not\nrely on domain-specific ontologies, parsers or entity taggers, which are\nexpensive to create. Despite this fact, our systems achieve state-of-the-art\nresults on factoid questions and competitive results on list questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03757v2",
            "date": "2017-06-12 17:51:05+00:00",
            "title": "Semantic Entity Retrieval Toolkit",
            "authors": [
                "Christophe Van Gysel",
                "Maarten de Rijke",
                "Evangelos Kanoulas"
            ],
            "abstract": "Unsupervised learning of low-dimensional, semantic representations of words\nand entities has recently gained attention. In this paper we describe the\nSemantic Entity Retrieval Toolkit (SERT) that provides implementations of our\npreviously published entity representation models. The toolkit provides a\nunified interface to different representation learning algorithms, fine-grained\nparsing configuration and can be used transparently with GPUs. In addition,\nusers can easily modify existing models or implement their own models in the\nframework. After model training, SERT can be used to rank entities according to\na textual query and extract the learned entity/word representation for use in\ndownstream algorithms, such as clustering or recommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03815v2",
            "date": "2017-06-12 19:07:02+00:00",
            "title": "Encoding of phonology in a recurrent neural model of grounded speech",
            "authors": [
                "Afra Alishahi",
                "Marie Barking",
                "Grzegorz Chrupa\u0142a"
            ],
            "abstract": "We study the representation and encoding of phonemes in a recurrent neural\nnetwork model of grounded speech. We use a model which processes images and\ntheir spoken descriptions, and projects the visual and auditory representations\ninto the same semantic space. We perform a number of analyses on how\ninformation about individual phonemes is encoded in the MFCC features extracted\nfrom the speech signal, and the activations of the layers of the model. Via\nexperiments with phoneme decoding and phoneme discrimination we show that\nphoneme representations are most salient in the lower layers of the model,\nwhere low-level signals are processed at a fine-grained level, although a large\namount of phonological information is retain at the top recurrent layer. We\nfurther find out that the attention mechanism following the top recurrent layer\nsignificantly attenuates encoding of phonology and makes the utterance\nembeddings much more invariant to synonymy. Moreover, a hierarchical clustering\nof phoneme representations learned by the network shows an organizational\nstructure of phonemes similar to those proposed in linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.03850v3",
            "date": "2017-06-12 20:55:51+00:00",
            "title": "Adversarial Feature Matching for Text Generation",
            "authors": [
                "Yizhe Zhang",
                "Zhe Gan",
                "Kai Fan",
                "Zhi Chen",
                "Ricardo Henao",
                "Dinghan Shen",
                "Lawrence Carin"
            ],
            "abstract": "The Generative Adversarial Network (GAN) has achieved great success in\ngenerating realistic (real-valued) synthetic data. However, convergence issues\nand difficulties dealing with discrete data hinder the applicability of GAN to\ntext. We propose a framework for generating realistic text via adversarial\ntraining. We employ a long short-term memory network as generator, and a\nconvolutional network as discriminator. Instead of using the standard objective\nof GAN, we propose matching the high-dimensional latent feature distributions\nof real and synthetic sentences, via a kernelized discrepancy metric. This\neases adversarial training by alleviating the mode-collapsing problem. Our\nexperiments show superior performance in quantitative evaluation, and\ndemonstrate that our model can generate realistic-looking sentences.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1706.04115v1",
            "date": "2017-06-13 15:17:42+00:00",
            "title": "Zero-Shot Relation Extraction via Reading Comprehension",
            "authors": [
                "Omer Levy",
                "Minjoon Seo",
                "Eunsol Choi",
                "Luke Zettlemoyer"
            ],
            "abstract": "We show that relation extraction can be reduced to answering simple reading\ncomprehension questions, by associating one or more natural-language questions\nwith each relation slot. This reduction has several advantages: we can (1)\nlearn relation-extraction models by extending recent neural\nreading-comprehension techniques, (2) build very large training sets for those\nmodels by combining relation-specific crowd-sourced questions with distant\nsupervision, and even (3) do zero-shot learning by extracting new relation\ntypes that are only specified at test-time, for which we have no labeled\ntraining examples. Experiments on a Wikipedia slot-filling task demonstrate\nthat the approach can generalize to new questions for known relation types with\nhigh accuracy, and that zero-shot generalization to unseen relation types is\npossible, at lower accuracy levels, setting the bar for future work on this\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.04223v3",
            "date": "2017-06-13 19:00:53+00:00",
            "title": "Adversarially Regularized Autoencoders",
            "authors": [
                "Jake Zhao",
                "Yoon Kim",
                "Kelly Zhang",
                "Alexander M. Rush",
                "Yann LeCun"
            ],
            "abstract": "Deep latent variable models, trained using variational autoencoders or\ngenerative adversarial networks, are now a key technique for representation\nlearning of continuous structures. However, applying similar methods to\ndiscrete structures, such as text sequences or discretized images, has proven\nto be more challenging. In this work, we propose a flexible method for training\ndeep latent variable models of discrete structures. Our approach is based on\nthe recently-proposed Wasserstein autoencoder (WAE) which formalizes the\nadversarial autoencoder (AAE) as an optimal transport problem. We first extend\nthis framework to model discrete sequences, and then further explore different\nlearned priors targeting a controllable representation. This adversarially\nregularized autoencoder (ARAE) allows us to generate natural textual outputs as\nwell as perform manipulations in the latent space to induce change in the\noutput space. Finally we show that the latent representation can be trained to\nperform unaligned textual style transfer, giving improvements both in\nautomatic/human evaluation compared to existing methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.04432v2",
            "date": "2017-06-14 12:22:38+00:00",
            "title": "Is Natural Language a Perigraphic Process? The Theorem about Facts and Words Revisited",
            "authors": [
                "\u0141ukasz D\u0119bowski"
            ],
            "abstract": "As we discuss, a stationary stochastic process is nonergodic when a random\npersistent topic can be detected in the infinite random text sampled from the\nprocess, whereas we call the process strongly nonergodic when an infinite\nsequence of independent random bits, called probabilistic facts, is needed to\ndescribe this topic completely. Replacing probabilistic facts with an\nalgorithmically random sequence of bits, called algorithmic facts, we adapt\nthis property back to ergodic processes. Subsequently, we call a process\nperigraphic if the number of algorithmic facts which can be inferred from a\nfinite text sampled from the process grows like a power of the text length. We\npresent a simple example of such a process. Moreover, we demonstrate an\nassertion which we call the theorem about facts and words. This proposition\nstates that the number of probabilistic or algorithmic facts which can be\ninferred from a text drawn from a process must be roughly smaller than the\nnumber of distinct word-like strings detected in this text by means of the PPM\ncompression algorithm. We also observe that the number of the word-like strings\nfor a sample of plays by Shakespeare follows an empirical stepwise power law,\nin a stark contrast to Markov processes. Hence we suppose that natural language\nconsidered as a process is not only non-Markov but also perigraphic.",
            "journal ref": "Entropy 20(2):85, 2018",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1706.04560v3",
            "date": "2017-06-14 16:06:18+00:00",
            "title": "Neural Models for Key Phrase Detection and Question Generation",
            "authors": [
                "Sandeep Subramanian",
                "Tong Wang",
                "Xingdi Yuan",
                "Saizheng Zhang",
                "Yoshua Bengio",
                "Adam Trischler"
            ],
            "abstract": "We propose a two-stage neural model to tackle question generation from\ndocuments. First, our model estimates the probability that word sequences in a\ndocument are ones that a human would pick when selecting candidate answers by\ntraining a neural key-phrase extractor on the answers in a question-answering\ncorpus. Predicted key phrases then act as target answers and condition a\nsequence-to-sequence question-generation model with a copy mechanism.\nEmpirically, our key-phrase extraction model significantly outperforms an\nentity-tagging baseline and existing rule-based approaches. We further\ndemonstrate that our question generation system formulates fluent, answerable\nquestions from key phrases. This two-stage system could be used to augment or\ngenerate reading comprehension datasets, which may be leveraged to improve\nmachine reading systems or in educational settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05075v1",
            "date": "2017-06-07 03:14:23+00:00",
            "title": "Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme",
            "authors": [
                "Suncong Zheng",
                "Feng Wang",
                "Hongyun Bao",
                "Yuexing Hao",
                "Peng Zhou",
                "Bo Xu"
            ],
            "abstract": "Joint extraction of entities and relations is an important task in\ninformation extraction. To tackle this problem, we firstly propose a novel\ntagging scheme that can convert the joint extraction task to a tagging problem.\nThen, based on our tagging scheme, we study different end-to-end models to\nextract entities and their relations directly, without identifying entities and\nrelations separately. We conduct experiments on a public dataset produced by\ndistant supervision method and the experimental results show that the tagging\nbased methods are better than most of the existing pipelined and joint learning\nmethods. What's more, the end-to-end model proposed in this paper, achieves the\nbest results on the public dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05084v2",
            "date": "2017-06-12 04:20:04+00:00",
            "title": "Topic supervised non-negative matrix factorization",
            "authors": [
                "Kelsey MacMillan",
                "James D. Wilson"
            ],
            "abstract": "Topic models have been extensively used to organize and interpret the\ncontents of large, unstructured corpora of text documents. Although topic\nmodels often perform well on traditional training vs. test set evaluations, it\nis often the case that the results of a topic model do not align with human\ninterpretation. This interpretability fallacy is largely due to the\nunsupervised nature of topic models, which prohibits any user guidance on the\nresults of a model. In this paper, we introduce a semi-supervised method called\ntopic supervised non-negative matrix factorization (TS-NMF) that enables the\nuser to provide labeled example documents to promote the discovery of more\nmeaningful semantic structure of a corpus. In this way, the results of TS-NMF\nbetter match the intuition and desired labeling of the user. The core of TS-NMF\nrelies on solving a non-convex optimization problem for which we derive an\niterative algorithm that is shown to be monotonic and convergent to a local\noptimum. We demonstrate the practical utility of TS-NMF on the Reuters and\nPubMed corpora, and find that TS-NMF is especially useful for conceptual or\nbroad topics, where topic key terms are not well understood. Although\nidentifying an optimal latent structure for the data is not a primary objective\nof the proposed approach, we find that TS-NMF achieves higher weighted Jaccard\nsimilarity scores than the contemporary methods, (unsupervised) NMF and latent\nDirichlet allocation, at supervision rates as low as 10% to 20%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05122v3",
            "date": "2017-06-16 00:53:28+00:00",
            "title": "Bib2vec: An Embedding-based Search System for Bibliographic Information",
            "authors": [
                "Takuma Yoneda",
                "Koki Mori",
                "Makoto Miwa",
                "Yutaka Sasaki"
            ],
            "abstract": "We propose a novel embedding model that represents relationships among\nseveral elements in bibliographic information with high representation ability\nand flexibility. Based on this model, we present a novel search system that\nshows the relationships among the elements in the ACL Anthology Reference\nCorpus. The evaluation results show that our model can achieve a high\nprediction ability and produce reasonable search results.",
            "journal ref": "Proceedings of the EACL 2017 Software Demonstrations, Valencia,\n  Spain, April 3-7 2017, pages 112-115",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.05585v1",
            "date": "2017-06-17 22:29:37+00:00",
            "title": "Accelerating Innovation Through Analogy Mining",
            "authors": [
                "Tom Hope",
                "Joel Chan",
                "Aniket Kittur",
                "Dafna Shahaf"
            ],
            "abstract": "The availability of large idea repositories (e.g., the U.S. patent database)\ncould significantly accelerate innovation and discovery by providing people\nwith inspiration from solutions to analogous problems. However, finding useful\nanalogies in these large, messy, real-world repositories remains a persistent\nchallenge for either human or automated methods. Previous approaches include\ncostly hand-created databases that have high relational structure (e.g.,\npredicate calculus representations) but are very sparse. Simpler\nmachine-learning/information-retrieval similarity metrics can scale to large,\nnatural-language datasets, but struggle to account for structural similarity,\nwhich is central to analogy. In this paper we explore the viability and value\nof learning simpler structural representations, specifically, \"problem\nschemas\", which specify the purpose of a product and the mechanisms by which it\nachieves that purpose. Our approach combines crowdsourcing and recurrent neural\nnetworks to extract purpose and mechanism vector representations from product\ndescriptions. We demonstrate that these learned vectors allow us to find\nanalogies with higher precision and recall than traditional\ninformation-retrieval methods. In an ideation experiment, analogies retrieved\nby our models significantly increased people's likelihood of generating\ncreative ideas compared to analogies retrieved by traditional methods. Our\nresults suggest a promising approach to enabling computational analogy at scale\nis to learn and leverage weaker structural representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06197v5",
            "date": "2017-06-19 22:36:33+00:00",
            "title": "meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting",
            "authors": [
                "Xu Sun",
                "Xuancheng Ren",
                "Shuming Ma",
                "Houfeng Wang"
            ],
            "abstract": "We propose a simple yet effective technique for neural network learning. The\nforward propagation is computed as usual. In back propagation, only a small\nsubset of the full gradient is computed to update the model parameters. The\ngradient vectors are sparsified in such a way that only the top-$k$ elements\n(in terms of magnitude) are kept. As a result, only $k$ rows or columns\n(depending on the layout) of the weight matrix are modified, leading to a\nlinear reduction ($k$ divided by the vector dimension) in the computational\ncost. Surprisingly, experimental results demonstrate that we can update only\n1-4% of the weights at each back propagation pass. This does not result in a\nlarger number of training iterations. More interestingly, the accuracy of the\nresulting models is actually improved rather than degraded, and a detailed\nanalysis is given. The code is available at https://github.com/lancopku/meProp",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.06428v1",
            "date": "2017-06-16 20:58:43+00:00",
            "title": "An online sequence-to-sequence model for noisy speech recognition",
            "authors": [
                "Chung-Cheng Chiu",
                "Dieterich Lawson",
                "Yuping Luo",
                "George Tucker",
                "Kevin Swersky",
                "Ilya Sutskever",
                "Navdeep Jaitly"
            ],
            "abstract": "Generative models have long been the dominant approach for speech\nrecognition. The success of these models however relies on the use of\nsophisticated recipes and complicated machinery that is not easily accessible\nto non-practitioners. Recent innovations in Deep Learning have given rise to an\nalternative - discriminative models called Sequence-to-Sequence models, that\ncan almost match the accuracy of state of the art generative models. While\nthese models are easy to train as they can be trained end-to-end in a single\nstep, they have a practical limitation that they can only be used for offline\nrecognition. This is because the models require that the entirety of the input\nsequence be available at the beginning of inference, an assumption that is not\nvalid for instantaneous speech recognition. To address this problem, online\nsequence-to-sequence models were recently introduced. These models are able to\nstart producing outputs as data arrives, and the model feels confident enough\nto output partial transcripts. These models, like sequence-to-sequence are\ncausal - the output produced by the model until any time, $t$, affects the\nfeatures that are computed subsequently. This makes the model inherently more\npowerful than generative models that are unable to change features that are\ncomputed from the data. This paper highlights two main contributions - an\nimprovement to online sequence-to-sequence model training, and its application\nto noisy settings with mixed speech from two speakers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.06551v2",
            "date": "2017-06-20 17:09:29+00:00",
            "title": "Grounded Language Learning in a Simulated 3D World",
            "authors": [
                "Karl Moritz Hermann",
                "Felix Hill",
                "Simon Green",
                "Fumin Wang",
                "Ryan Faulkner",
                "Hubert Soyer",
                "David Szepesvari",
                "Wojciech Marian Czarnecki",
                "Max Jaderberg",
                "Denis Teplyashin",
                "Marcus Wainwright",
                "Chris Apps",
                "Demis Hassabis",
                "Phil Blunsom"
            ],
            "abstract": "We are increasingly surrounded by artificially intelligent technology that\ntakes decisions and executes actions on our behalf. This creates a pressing\nneed for general means to communicate with, instruct and guide artificial\nagents, with human language the most compelling means for such communication.\nTo achieve this in a scalable fashion, agents must be able to relate language\nto the world and to actions; that is, their understanding of language must be\ngrounded and embodied. However, learning grounded language is a notoriously\nchallenging problem in artificial intelligence research. Here we present an\nagent that learns to interpret language in a simulated 3D environment where it\nis rewarded for the successful execution of written instructions. Trained via a\ncombination of reinforcement and unsupervised learning, and beginning with\nminimal prior knowledge, the agent learns to relate linguistic symbols to\nemergent perceptual representations of its physical surroundings and to\npertinent sequences of actions. The agent's comprehension of language extends\nbeyond its prior experience, enabling it to apply familiar language to\nunfamiliar situations and to interpret entirely novel instructions. Moreover,\nthe speed with which this agent learns new words increases as its semantic\nknowledge grows. This facility for generalising and bootstrapping semantic\nknowledge indicates the potential of the present approach for reconciling\nambiguous natural language with the complexity of the physical world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07206v2",
            "date": "2017-06-22 08:24:59+00:00",
            "title": "Explaining Recurrent Neural Network Predictions in Sentiment Analysis",
            "authors": [
                "Leila Arras",
                "Gr\u00e9goire Montavon",
                "Klaus-Robert M\u00fcller",
                "Wojciech Samek"
            ],
            "abstract": "Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown\nto deliver insightful explanations in the form of input space relevances for\nunderstanding feed-forward neural network classification decisions. In the\npresent work, we extend the usage of LRP to recurrent neural networks. We\npropose a specific propagation rule applicable to multiplicative connections as\nthey arise in recurrent network architectures such as LSTMs and GRUs. We apply\nour technique to a word-based bi-directional LSTM model on a five-class\nsentiment prediction task, and evaluate the resulting LRP relevances both\nqualitatively and quantitatively, obtaining better results than a\ngradient-based related method which was used in previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07230v2",
            "date": "2017-06-22 09:39:17+00:00",
            "title": "Gated-Attention Architectures for Task-Oriented Language Grounding",
            "authors": [
                "Devendra Singh Chaplot",
                "Kanthashree Mysore Sathyendra",
                "Rama Kumar Pasumarthi",
                "Dheeraj Rajagopal",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "To perform tasks specified by natural language instructions, autonomous\nagents need to extract semantically meaningful representations of language and\nmap it to visual elements and actions in the environment. This problem is\ncalled task-oriented language grounding. We propose an end-to-end trainable\nneural architecture for task-oriented language grounding in 3D environments\nwhich assumes no prior linguistic or perceptual knowledge and requires only raw\npixels from the environment and the natural language instruction as input. The\nproposed model combines the image and text representations using a\nGated-Attention mechanism and learns a policy to execute the natural language\ninstruction using standard reinforcement and imitation learning methods. We\nshow the effectiveness of the proposed model on unseen instructions as well as\nunseen maps, both quantitatively and qualitatively. We also introduce a novel\nenvironment based on a 3D game engine to simulate the challenges of\ntask-oriented language grounding over a rich set of instructions and\nenvironment states.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.07276v1",
            "date": "2017-06-21 06:19:24+00:00",
            "title": "Jointly Learning Word Embeddings and Latent Topics",
            "authors": [
                "Bei Shi",
                "Wai Lam",
                "Shoaib Jameel",
                "Steven Schockaert",
                "Kwun Ping Lai"
            ],
            "abstract": "Word embedding models such as Skip-gram learn a vector-space representation\nfor each word, based on the local word collocation patterns that are observed\nin a text corpus. Latent topic models, on the other hand, take a more global\nview, looking at the word distributions across the corpus to assign a topic to\neach word occurrence. These two paradigms are complementary in how they\nrepresent the meaning of word occurrences. While some previous works have\nalready looked at using word embeddings for improving the quality of latent\ntopics, and conversely, at using latent topics for improving word embeddings,\nsuch \"two-step\" methods cannot capture the mutual interaction between the two\nparadigms. In this paper, we propose STE, a framework which can learn word\nembeddings and latent topics in a unified manner. STE naturally obtains\ntopic-specific word embeddings, and thus addresses the issue of polysemy. At\nthe same time, it also learns the term distributions of the topics, and the\ntopic distributions of the documents. Our experimental results demonstrate that\nthe STE model can indeed generate useful topic-specific word embeddings and\ncoherent latent topics in an effective and efficient way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.07913v1",
            "date": "2017-06-24 06:08:27+00:00",
            "title": "Semi-supervised Text Categorization Using Recursive K-means Clustering",
            "authors": [
                "Harsha S. Gowda",
                "Mahamad Suhil",
                "D. S. Guru",
                "Lavanya Narayana Raju"
            ],
            "abstract": "In this paper, we present a semi-supervised learning algorithm for\nclassification of text documents. A method of labeling unlabeled text documents\nis presented. The presented method is based on the principle of divide and\nconquer strategy. It uses recursive K-means algorithm for partitioning both\nlabeled and unlabeled data collection. The K-means algorithm is applied\nrecursively on each partition till a desired level partition is achieved such\nthat each partition contains labeled documents of a single class. Once the\ndesired clusters are obtained, the respective cluster centroids are considered\nas representatives of the clusters and the nearest neighbor rule is used for\nclassifying an unknown text document. Series of experiments have been conducted\nto bring out the superiority of the proposed model over other recent state of\nthe art models on 20Newsgroups dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1706.08502v3",
            "date": "2017-06-26 17:47:46+00:00",
            "title": "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog",
            "authors": [
                "Satwik Kottur",
                "Jos\u00e9 M. F. Moura",
                "Stefan Lee",
                "Dhruv Batra"
            ],
            "abstract": "A number of recent works have proposed techniques for end-to-end learning of\ncommunication protocols among cooperative multi-agent populations, and have\nsimultaneously found the emergence of grounded human-interpretable language in\nthe protocols developed by the agents, all learned without any human\nsupervision!\n  In this paper, using a Task and Tell reference game between two agents as a\ntestbed, we present a sequence of 'negative' results culminating in a\n'positive' one -- showing that while most agent-invented languages are\neffective (i.e. achieve near-perfect task rewards), they are decidedly not\ninterpretable or compositional.\n  In essence, we find that natural language does not emerge 'naturally',\ndespite the semblance of ease of natural-language-emergence that one may gather\nfrom recent literature. We discuss how it is possible to coax the invented\nlanguages to become more and more human-like and compositional by increasing\nrestrictions on how two agents may communicate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.08568v1",
            "date": "2017-06-26 19:14:10+00:00",
            "title": "Neural Question Answering at BioASQ 5B",
            "authors": [
                "Georg Wiese",
                "Dirk Weissenborn",
                "Mariana Neves"
            ],
            "abstract": "This paper describes our submission to the 2017 BioASQ challenge. We\nparticipated in Task B, Phase B which is concerned with biomedical question\nanswering (QA). We focus on factoid and list question, using an extractive QA\nmodel, that is, we restrict our system to output substrings of the provided\ntext snippets. At the core of our system, we use FastQA, a state-of-the-art\nneural QA system. We extended it with biomedical word embeddings and changed\nits answer layer to be able to answer list questions in addition to factoid\nquestions. We pre-trained the model on a large-scale open-domain QA dataset,\nSQuAD, and then fine-tuned the parameters on the BioASQ training set. With our\napproach, we achieve state-of-the-art results on factoid questions and\ncompetitive results on list questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.09588v1",
            "date": "2017-06-29 05:56:06+00:00",
            "title": "Multi-scale Multi-band DenseNets for Audio Source Separation",
            "authors": [
                "Naoya Takahashi",
                "Yuki Mitsufuji"
            ],
            "abstract": "This paper deals with the problem of audio source separation. To handle the\ncomplex and ill-posed nature of the problems of audio source separation, the\ncurrent state-of-the-art approaches employ deep neural networks to obtain\ninstrumental spectra from a mixture. In this study, we propose a novel network\narchitecture that extends the recently developed densely connected\nconvolutional network (DenseNet), which has shown excellent results on image\nclassification tasks. To deal with the specific problem of audio source\nseparation, an up-sampling layer, block skip connection and band-dedicated\ndense blocks are incorporated on top of DenseNet. The proposed approach takes\nadvantage of long contextual information and outperforms state-of-the-art\nresults on SiSEC 2016 competition by a large margin in terms of\nsignal-to-distortion ratio. Moreover, the proposed architecture requires\nsignificantly fewer parameters and considerably less training time compared\nwith other methods.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1706.10006v2",
            "date": "2017-06-30 02:55:55+00:00",
            "title": "Automated Audio Captioning with Recurrent Neural Networks",
            "authors": [
                "Konstantinos Drossos",
                "Sharath Adavanne",
                "Tuomas Virtanen"
            ],
            "abstract": "We present the first approach to automated audio captioning. We employ an\nencoder-decoder scheme with an alignment model in between. The input to the\nencoder is a sequence of log mel-band energies calculated from an audio file,\nwhile the output is a sequence of words, i.e. a caption. The encoder is a\nmulti-layered, bi-directional gated recurrent unit (GRU) and the decoder a\nmulti-layered GRU with a classification layer connected to the last GRU of the\ndecoder. The classification layer and the alignment model are fully connected\nlayers with shared weights between timesteps. The proposed method is evaluated\nusing data drawn from a commercial sound effects library, ProSound Effects. The\nresulting captions were rated through metrics utilized in machine translation\nand image captioning fields. Results from metrics show that the proposed method\ncan predict words appearing in the original caption, but not always correctly\nordered.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1708.00077v1",
            "date": "2017-07-31 21:33:42+00:00",
            "title": "Bayesian Sparsification of Recurrent Neural Networks",
            "authors": [
                "Ekaterina Lobacheva",
                "Nadezhda Chirkova",
                "Dmitry Vetrov"
            ],
            "abstract": "Recurrent neural networks show state-of-the-art results in many text analysis\ntasks but often require a lot of memory to store their weights. Recently\nproposed Sparse Variational Dropout eliminates the majority of the weights in a\nfeed-forward neural network without significant loss of quality. We apply this\ntechnique to sparsify recurrent neural networks. To account for recurrent\nspecifics we also rely on Binary Variational Dropout for RNN. We report 99.5%\nsparsity level on sentiment analysis task without a quality drop and up to 87%\nsparsity level on language modeling task with slight loss of accuracy.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1708.00107v2",
            "date": "2017-08-01 00:05:34+00:00",
            "title": "Learned in Translation: Contextualized Word Vectors",
            "authors": [
                "Bryan McCann",
                "James Bradbury",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Computer vision has benefited from initializing multiple deep layers with\nweights pretrained on large supervised training sets like ImageNet. Natural\nlanguage processing (NLP) typically sees initialization of only the lowest\nlayer of deep models with pretrained word vectors. In this paper, we use a deep\nLSTM encoder from an attentional sequence-to-sequence model trained for machine\ntranslation (MT) to contextualize word vectors. We show that adding these\ncontext vectors (CoVe) improves performance over using only unsupervised word\nand character vectors on a wide variety of common NLP tasks: sentiment analysis\n(SST, IMDb), question classification (TREC), entailment (SNLI), and question\nanswering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe\nimproves performance of our baseline models to the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00111v2",
            "date": "2017-08-01 00:16:08+00:00",
            "title": "A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models",
            "authors": [
                "Kartik Goyal",
                "Graham Neubig",
                "Chris Dyer",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Beam search is a desirable choice of test-time decoding algorithm for neural\nsequence models because it potentially avoids search errors made by simpler\ngreedy methods. However, typical cross entropy training procedures for these\nmodels do not directly consider the behaviour of the final decoding method. As\na result, for cross-entropy trained models, beam decoding can sometimes yield\nreduced test performance when compared with greedy decoding. In order to train\nmodels that can more effectively make use of beam search, we propose a new\ntraining procedure that focuses on the final loss metric (e.g. Hamming loss)\nevaluated on the output of beam search. While well-defined, this \"direct loss\"\nobjective is itself discontinuous and thus difficult to optimize. Hence, in our\napproach, we form a sub-differentiable surrogate objective by introducing a\nnovel continuous approximation of the beam search decoding procedure. In\nexperiments, we show that optimizing this new training objective yields\nsubstantially better results on two sequence tasks (Named Entity Recognition\nand CCG Supertagging) when compared with both cross entropy trained greedy\ndecoding and cross entropy trained beam decoding baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.00112v3",
            "date": "2017-08-01 00:23:03+00:00",
            "title": "Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relations",
            "authors": [
                "Benjamin J. Lengerich",
                "Andrew L. Maas",
                "Christopher Potts"
            ],
            "abstract": "Knowledge graphs are a versatile framework to encode richly structured data\nrelationships, but it can be challenging to combine these graphs with\nunstructured data. Methods for retrofitting pre-trained entity representations\nto the structure of a knowledge graph typically assume that entities are\nembedded in a connected space and that relations imply similarity. However,\nuseful knowledge graphs often contain diverse entities and relations (with\npotentially disjoint underlying corpora) which do not accord with these\nassumptions. To overcome these limitations, we present Functional Retrofitting,\na framework that generalizes current retrofitting methods by explicitly\nmodeling pairwise relations. Our framework can directly incorporate a variety\nof pairwise penalty functions previously developed for knowledge graph\ncompletion. Further, it allows users to encode, learn, and extract information\nabout relation semantics. We present both linear and neural instantiations of\nthe framework. Functional Retrofitting significantly outperforms existing\nretrofitting methods on complex knowledge graphs and loses no accuracy on\nsimpler graphs (in which relations do imply similarity). Finally, we\ndemonstrate the utility of the framework by predicting new drug--disease\ntreatment pairs in a large, complex health knowledge graph.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1708.00133v2",
            "date": "2017-08-01 02:20:00+00:00",
            "title": "Grounding Language for Transfer in Deep Reinforcement Learning",
            "authors": [
                "Karthik Narasimhan",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "In this paper, we explore the utilization of natural language to drive\ntransfer for reinforcement learning (RL). Despite the wide-spread application\nof deep RL techniques, learning generalized policy representations that work\nacross domains remains a challenging problem. We demonstrate that textual\ndescriptions of environments provide a compact intermediate channel to\nfacilitate effective policy transfer. Specifically, by learning to ground the\nmeaning of text to the dynamics of the environment such as transitions and\nrewards, an autonomous agent can effectively bootstrap policy learning on a new\ndomain given its description. We employ a model-based RL approach consisting of\na differentiable planning module, a model-free component and a factorized state\nrepresentation to effectively use entity descriptions. Our model outperforms\nprior work on both transfer and multi-task scenarios in a variety of different\nenvironments. For instance, we achieve up to 14% and 11.5% absolute improvement\nover previously existing models in terms of average and initial rewards,\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00154v1",
            "date": "2017-08-01 04:25:17+00:00",
            "title": "Neural Rating Regression with Abstractive Tips Generation for Recommendation",
            "authors": [
                "Piji Li",
                "Zihao Wang",
                "Zhaochun Ren",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "Recently, some E-commerce sites launch a new interaction box called Tips on\ntheir mobile apps. Users can express their experience and feelings or provide\nsuggestions using short texts typically several words or one sentence. In\nessence, writing some tips and giving a numerical rating are two facets of a\nuser's product assessment action, expressing the user experience and feelings.\nJointly modeling these two facets is helpful for designing a better\nrecommendation system. While some existing models integrate text information\nsuch as item specifications or user reviews into user and item latent factors\nfor improving the rating prediction, no existing works consider tips for\nimproving recommendation quality. We propose a deep learning based framework\nnamed NRT which can simultaneously predict precise ratings and generate\nabstractive tips with good linguistic quality simulating user experience and\nfeelings. For abstractive tips generation, gated recurrent neural networks are\nemployed to \"translate\" user and item latent representations into a concise\nsentence. Extensive experiments on benchmark datasets from different domains\nshow that NRT achieves significant improvements over the state-of-the-art\nmethods. Moreover, the generated tips can vividly predict the user experience\nand feelings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00308v1",
            "date": "2017-08-01 13:31:24+00:00",
            "title": "SenGen: Sentence Generating Neural Variational Topic Model",
            "authors": [
                "Ramesh Nallapati",
                "Igor Melnyk",
                "Abhishek Kumar",
                "Bowen Zhou"
            ],
            "abstract": "We present a new topic model that generates documents by sampling a topic for\none whole sentence at a time, and generating the words in the sentence using an\nRNN decoder that is conditioned on the topic of the sentence. We argue that\nthis novel formalism will help us not only visualize and model the topical\ndiscourse structure in a document better, but also potentially lead to more\ninterpretable topics since we can now illustrate topics by sampling\nrepresentative sentences instead of bag of words or phrases. We present a\nvariational auto-encoder approach for learning in which we use a factorized\nvariational encoder that independently models the posterior over topical\nmixture vectors of documents using a feed-forward network, and the posterior\nover topic assignments to sentences using an RNN. Our preliminary experiments\non two different datasets indicate early promise, but also expose many\nchallenges that remain to be addressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.00531v2",
            "date": "2017-08-01 21:53:56+00:00",
            "title": "End-to-End Neural Segmental Models for Speech Recognition",
            "authors": [
                "Hao Tang",
                "Liang Lu",
                "Lingpeng Kong",
                "Kevin Gimpel",
                "Karen Livescu",
                "Chris Dyer",
                "Noah A. Smith",
                "Steve Renals"
            ],
            "abstract": "Segmental models are an alternative to frame-based models for sequence\nprediction, where hypothesized path weights are based on entire segment scores\nrather than a single frame at a time. Neural segmental models are segmental\nmodels that use neural network-based weight functions. Neural segmental models\nhave achieved competitive results for speech recognition, and their end-to-end\ntraining has been explored in several studies. In this work, we review neural\nsegmental models, which can be viewed as consisting of a neural network-based\nacoustic encoder and a finite-state transducer decoder. We study end-to-end\nsegmental models with different weight functions, including ones based on\nframe-level neural classifiers and on segmental recurrent neural networks. We\nstudy how reducing the search space size impacts performance under different\nweight functions. We also compare several loss functions for end-to-end\ntraining. Finally, we explore training approaches, including multi-stage vs.\nend-to-end training and multitask training that combines segmental and\nframe-level losses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01318v2",
            "date": "2017-08-03 21:42:46+00:00",
            "title": "The UMD Neural Machine Translation Systems at WMT17 Bandit Learning Task",
            "authors": [
                "Amr Sharaf",
                "Shi Feng",
                "Khanh Nguyen",
                "Kiant\u00e9 Brantley",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We describe the University of Maryland machine translation systems submitted\nto the WMT17 German-English Bandit Learning Task. The task is to adapt a\ntranslation system to a new domain, using only bandit feedback: the system\nreceives a German sentence to translate, produces an English sentence, and only\ngets a scalar score as feedback. Targeting these two challenges (adaptation and\nbandit learning), we built a standard neural machine translation system and\nextended it in two ways: (1) robust reinforcement learning techniques to learn\neffectively from the bandit feedback, and (2) domain adaptation using data\nselection from a large corpus of parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01525v4",
            "date": "2017-08-04 14:35:38+00:00",
            "title": "Language Design as Information Renormalization",
            "authors": [
                "Angel J. Gallego",
                "Roman Orus"
            ],
            "abstract": "Here we consider some well-known facts in syntax from a physics perspective,\nallowing us to establish equivalences between both fields with many\nconsequences. Mainly, we observe that the operation MERGE, put forward by N.\nChomsky in 1995, can be interpreted as a physical information coarse-graining.\nThus, MERGE in linguistics entails information renormalization in physics,\naccording to different time scales. We make this point mathematically formal in\nterms of language models. In this setting, MERGE amounts to a probability\ntensor implementing a coarse-graining, akin to a probabilistic context-free\ngrammar. The probability vectors of meaningful sentences are given by\nstochastic tensor networks (TN) built from diagonal tensors and which are\nmostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus\nbeing computationally very efficient to manipulate. We show that this implies\nthe polynomially-decaying (long-range) correlations experimentally observed in\nlanguage, and also provides arguments in favour of certain types of neural\nnetworks for language processing. Moreover, we show how to obtain such language\nmodels from quantum states that can be efficiently prepared on a quantum\ncomputer, and use this to find bounds on the perplexity of the probability\ndistribution of words in a sentence. Implications of our results are discussed\nacross several ambits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.01677v2",
            "date": "2017-08-04 22:35:50+00:00",
            "title": "A network approach to topic models",
            "authors": [
                "Martin Gerlach",
                "Tiago P. Peixoto",
                "Eduardo G. Altmann"
            ],
            "abstract": "One of the main computational and scientific challenges in the modern age is\nto extract useful information from unstructured texts. Topic models are one\npopular machine-learning approach which infers the latent topical structure of\na collection of documents. Despite their success --- in particular of its most\nwidely used variant called Latent Dirichlet Allocation (LDA) --- and numerous\napplications in sociology, history, and linguistics, topic models are known to\nsuffer from severe conceptual and practical problems, e.g. a lack of\njustification for the Bayesian priors, discrepancies with statistical\nproperties of real texts, and the inability to properly choose the number of\ntopics. Here we obtain a fresh view on the problem of identifying topical\nstructures by relating it to the problem of finding communities in complex\nnetworks. This is achieved by representing text corpora as bipartite networks\nof documents and words. By adapting existing community-detection methods --\nusing a stochastic block model (SBM) with non-parametric priors -- we obtain a\nmore versatile and principled framework for topic modeling (e.g., it\nautomatically detects the number of topics and hierarchically clusters both the\nwords and documents). The analysis of artificial and real corpora demonstrates\nthat our SBM approach leads to better topic models than LDA in terms of\nstatistical model selection. More importantly, our work shows how to formally\nrelate methods from community detection and topic modeling, opening the\npossibility of cross-fertilization between these two fields.",
            "journal ref": "Science Advances 4, eaaq1360 (2018)",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1708.01776v1",
            "date": "2017-08-05 15:06:56+00:00",
            "title": "e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations",
            "authors": [
                "Clemens Rosenbaum",
                "Tian Gao",
                "Tim Klinger"
            ],
            "abstract": "In this paper we present a new dataset and user simulator e-QRAQ (explainable\nQuery, Reason, and Answer Question) which tests an Agent's ability to read an\nambiguous text; ask questions until it can answer a challenge question; and\nexplain the reasoning behind its questions and answer. The User simulator\nprovides the Agent with a short, ambiguous story and a challenge question about\nthe story. The story is ambiguous because some of the entities have been\nreplaced by variables. At each turn the Agent may ask for the value of a\nvariable or try to answer the challenge question. In response the User\nsimulator provides a natural language explanation of why the Agent's query or\nanswer was useful in narrowing down the set of possible answers, or not. To\ndemonstrate one potential application of the e-QRAQ dataset, we train a new\nneural architecture based on End-to-End Memory Networks to successfully\ngenerate both predictions and partial explanations of its current understanding\nof the problem. We observe a strong correlation between the quality of the\nprediction and explanation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.02043v2",
            "date": "2017-08-07 09:01:35+00:00",
            "title": "What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator?",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Kenneth P. Camilleri"
            ],
            "abstract": "In neural image captioning systems, a recurrent neural network (RNN) is\ntypically viewed as the primary `generation' component. This view suggests that\nthe image features should be `injected' into the RNN. This is in fact the\ndominant view in the literature. Alternatively, the RNN can instead be viewed\nas only encoding the previously generated words. This view suggests that the\nRNN should only be used to encode linguistic features and that only the final\nrepresentation should be `merged' with the image features at a later stage.\nThis paper compares these two architectures. We find that, in general, late\nmerging outperforms injection, suggesting that RNNs are better viewed as\nencoders, rather than generators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02099v1",
            "date": "2017-08-07 12:50:09+00:00",
            "title": "Multimodal Classification for Analysing Social Media",
            "authors": [
                "Chi Thang Duong",
                "Remi Lebret",
                "Karl Aberer"
            ],
            "abstract": "Classification of social media data is an important approach in understanding\nuser behavior on the Web. Although information on social media can be of\ndifferent modalities such as texts, images, audio or videos, traditional\napproaches in classification usually leverage only one prominent modality.\nTechniques that are able to leverage multiple modalities are often complex and\nsusceptible to the absence of some modalities. In this paper, we present simple\nmodels that combine information from different modalities to classify social\nmedia content and are able to handle the above problems with existing\ntechniques. Our models combine information from different modalities using a\npooling layer and an auxiliary learning task is used to learn a common feature\nspace. We demonstrate the performance of our models and their robustness to the\nmissing of some modalities in the emotion classification domain. Our\napproaches, although being simple, can not only achieve significantly higher\naccuracies than traditional fusion approaches but also have comparable results\nwhen only one modality is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02182v1",
            "date": "2017-08-07 16:03:44+00:00",
            "title": "Regularizing and Optimizing LSTM Language Models",
            "authors": [
                "Stephen Merity",
                "Nitish Shirish Keskar",
                "Richard Socher"
            ],
            "abstract": "Recurrent neural networks (RNNs), such as long short-term memory networks\n(LSTMs), serve as a fundamental building block for many sequence learning\ntasks, including machine translation, language modeling, and question\nanswering. In this paper, we consider the specific problem of word-level\nlanguage modeling and investigate strategies for regularizing and optimizing\nLSTM-based models. We propose the weight-dropped LSTM which uses DropConnect on\nhidden-to-hidden weights as a form of recurrent regularization. Further, we\nintroduce NT-ASGD, a variant of the averaged stochastic gradient method,\nwherein the averaging trigger is determined using a non-monotonic condition as\nopposed to being tuned by the user. Using these and other regularization\nstrategies, we achieve state-of-the-art word level perplexities on two data\nsets: 57.3 on Penn Treebank and 65.8 on WikiText-2. In exploring the\neffectiveness of a neural cache in conjunction with our proposed model, we\nachieve an even lower state-of-the-art perplexity of 52.8 on Penn Treebank and\n52.0 on WikiText-2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02214v3",
            "date": "2017-08-07 17:32:56+00:00",
            "title": "LitStoryTeller: An Interactive System for Visual Exploration of Scientific Papers Leveraging Named entities and Comparative Sentences",
            "authors": [
                "Qing Ping",
                "Chaomei Chen"
            ],
            "abstract": "The present study proposes LitStoryTeller, an interactive system for visually\nexploring the semantic structure of a scientific article. We demonstrate how\nLitStoryTeller could be used to answer some of the most fundamental research\nquestions, such as how a new method was built on top of existing methods, based\non what theoretical proof and experimental evidences. More importantly,\nLitStoryTeller can assist users to understand the full and interesting story a\nscientific paper, with a concise outline and important details. The proposed\nsystem borrows a metaphor from screen play, and visualizes the storyline of a\nscientific paper by arranging its characters (scientific concepts or\nterminologies) and scenes (paragraphs/sentences) into a progressive and\ninteractive storyline. Such storylines help to preserve the semantic structure\nand logical thinking process of a scientific paper. Semantic structures, such\nas scientific concepts and comparative sentences, are extracted using existing\nnamed entity recognition APIs and supervised classifiers, from a scientific\npaper automatically. Two supplementary views, ranked entity frequency view and\nentity co-occurrence network view, are provided to help users identify the\n\"main plot\" of such scientific storylines. When collective documents are ready,\nLitStoryTeller also provides a temporal entity evolution view and entity\ncommunity view for collection digestion.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1708.02255v3",
            "date": "2017-08-07 18:00:42+00:00",
            "title": "Generative Statistical Models with Self-Emergent Grammar of Chord Sequences",
            "authors": [
                "Hiroaki Tsushima",
                "Eita Nakamura",
                "Katsutoshi Itoyama",
                "Kazuyoshi Yoshii"
            ],
            "abstract": "Generative statistical models of chord sequences play crucial roles in music\nprocessing. To capture syntactic similarities among certain chords (e.g. in C\nmajor key, between G and G7 and between F and Dm), we study hidden Markov\nmodels and probabilistic context-free grammar models with latent variables\ndescribing syntactic categories of chord symbols and their unsupervised\nlearning techniques for inducing the latent grammar from data. Surprisingly, we\nfind that these models often outperform conventional Markov models in\npredictive power, and the self-emergent categories often correspond to\ntraditional harmonic functions. This implies the need for chord categories in\nharmony models from the informatics perspective.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1708.02300v1",
            "date": "2017-08-07 20:50:24+00:00",
            "title": "Reinforced Video Captioning with Entailment Rewards",
            "authors": [
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Sequence-to-sequence models have shown promising improvements on the temporal\ntask of video captioning, but they optimize word-level cross-entropy loss\nduring training. First, using policy gradient and mixed-loss methods for\nreinforcement learning, we directly optimize sentence-level task-based metrics\n(as rewards), achieving significant improvements over the baseline, based on\nboth automatic metrics and human evaluation on multiple datasets. Next, we\npropose a novel entailment-enhanced reward (CIDEnt) that corrects\nphrase-matching based metrics (such as CIDEr) to only allow for\nlogically-implied partial matches and avoid contradictions, achieving further\nsignificant improvements over the CIDEr-reward model. Overall, our\nCIDEnt-reward model achieves the new state-of-the-art on the MSR-VTT dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02312v2",
            "date": "2017-08-07 21:33:11+00:00",
            "title": "Shortcut-Stacked Sentence Encoders for Multi-Domain Inference",
            "authors": [
                "Yixin Nie",
                "Mohit Bansal"
            ],
            "abstract": "We present a simple sequential sentence encoder for multi-domain natural\nlanguage inference. Our encoder is based on stacked bidirectional LSTM-RNNs\nwith shortcut connections and fine-tuning of word embeddings. The overall\nsupervised model uses the above encoder to encode two input sentences into two\nvectors, and then uses a classifier over the vector combination to label the\nrelationship between these two sentences as that of entailment, contradiction,\nor neural. Our Shortcut-Stacked sentence encoders achieve strong improvements\nover existing encoders on matched and mismatched multi-domain natural language\ninference (top non-ensemble single-model result in the EMNLP RepEval 2017\nShared Task (Nangia et al., 2017)). Moreover, they achieve the new\nstate-of-the-art encoding result on the original SNLI dataset (Bowman et al.,\n2015).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02383v1",
            "date": "2017-08-08 07:06:48+00:00",
            "title": "Learning how to Active Learn: A Deep Reinforcement Learning Approach",
            "authors": [
                "Meng Fang",
                "Yuan Li",
                "Trevor Cohn"
            ],
            "abstract": "Active learning aims to select a small subset of data for annotation such\nthat a classifier learned on the data is highly accurate. This is usually done\nusing heuristic selection methods, however the effectiveness of such methods is\nlimited and moreover, the performance of heuristics varies between datasets. To\naddress these shortcomings, we introduce a novel formulation by reframing the\nactive learning as a reinforcement learning problem and explicitly learning a\ndata selection policy, where the policy takes the role of the active learning\nheuristic. Importantly, our method allows the selection policy learned using\nsimulation on one language to be transferred to other languages. We demonstrate\nour method using cross-lingual named entity recognition, observing uniform\nimprovements over traditional active learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02977v1",
            "date": "2017-08-09 19:26:47+00:00",
            "title": "Hierarchically-Attentive RNN for Album Summarization and Storytelling",
            "authors": [
                "Licheng Yu",
                "Mohit Bansal",
                "Tamara L. Berg"
            ],
            "abstract": "We address the problem of end-to-end visual storytelling. Given a photo\nalbum, our model first selects the most representative (summary) photos, and\nthen composes a natural language story for the album. For this task, we make\nuse of the Visual Storytelling dataset and a model composed of three\nhierarchically-attentive Recurrent Neural Nets (RNNs) to: encode the album\nphotos, select representative (summary) photos, and compose the story.\nAutomatic and human evaluations show our model achieves better performance on\nselection, generation, and retrieval than baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03044v1",
            "date": "2017-08-10 01:40:49+00:00",
            "title": "\"Is there anything else I can help you with?\": Challenges in Deploying an On-Demand Crowd-Powered Conversational Agent",
            "authors": [
                "Ting-Hao Kenneth Huang",
                "Walter S. Lasecki",
                "Amos Azaria",
                "Jeffrey P. Bigham"
            ],
            "abstract": "Intelligent conversational assistants, such as Apple's Siri, Microsoft's\nCortana, and Amazon's Echo, have quickly become a part of our digital life.\nHowever, these assistants have major limitations, which prevents users from\nconversing with them as they would with human dialog partners. This limits our\nability to observe how users really want to interact with the underlying\nsystem. To address this problem, we developed a crowd-powered conversational\nassistant, Chorus, and deployed it to see how users and workers would interact\ntogether when mediated by the system. Chorus sophisticatedly converses with end\nusers over time by recruiting workers on demand, which in turn decide what\nmight be the best response for each user sentence. Up to the first month of our\ndeployment, 59 users have held conversations with Chorus during 320\nconversational sessions. In this paper, we present an account of Chorus'\ndeployment, with a focus on four challenges: (i) identifying when conversations\nare over, (ii) malicious users and workers, (iii) on-demand recruiting, and\n(iv) settings in which consensus is not enough. Our observations could assist\nthe deployment of crowd-powered conversation systems and crowd-powered systems\nin general.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1708.03052v1",
            "date": "2017-08-10 02:03:52+00:00",
            "title": "Communication-Free Parallel Supervised Topic Models",
            "authors": [
                "Lee Gao",
                "Ronghuo Zheng"
            ],
            "abstract": "Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC)\nmethods are commonly used in learning graphical models. However, MCMC cannot be\ndirectly applied in learning topic models because of the quasi-ergodicity\nproblem caused by multimodal distribution of topics. In this paper, we develop\nan embarrassingly parallel MCMC algorithm for sLDA. Our algorithm works by\nswitching the order of sampled topics combination and labeling variable\nprediction in sLDA, in which it overcomes the quasi-ergodicity problem because\nhigh-dimension topics that follow a multimodal distribution are projected into\none-dimension document labels that follow a unimodal distribution. Our\nempirical experiments confirm that the out-of-sample prediction performance\nusing our embarrassingly parallel algorithm is comparable to non-parallel sLDA\nwhile the computation time is significantly reduced.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.03418v4",
            "date": "2017-08-11 00:55:57+00:00",
            "title": "Learning to Attend, Copy, and Generate for Session-Based Query Suggestion",
            "authors": [
                "Mostafa Dehghani",
                "Sascha Rothe",
                "Enrique Alfonseca",
                "Pascal Fleury"
            ],
            "abstract": "Users try to articulate their complex information needs during search\nsessions by reformulating their queries. To make this process more effective,\nsearch engines provide related queries to help users in specifying the\ninformation need in their search process. In this paper, we propose a\ncustomized sequence-to-sequence model for session-based query suggestion. In\nour model, we employ a query-aware attention mechanism to capture the structure\nof the session context. is enables us to control the scope of the session from\nwhich we infer the suggested next query, which helps not only handle the noisy\ndata but also automatically detect session boundaries. Furthermore, we observe\nthat, based on the user query reformulation behavior, within a single session a\nlarge portion of query terms is retained from the previously submitted queries\nand consists of mostly infrequent or unseen terms that are usually not included\nin the vocabulary. We therefore empower the decoder of our model to access the\nsource words from the session context during decoding by incorporating a copy\nmechanism. Moreover, we propose evaluation metrics to assess the quality of the\ngenerative models for query suggestion. We conduct an extensive set of\nexperiments and analysis. e results suggest that our model outperforms the\nbaselines both in terms of the generating queries and scoring candidate queries\nfor the task of query suggestion.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.03910v1",
            "date": "2017-08-13 14:09:22+00:00",
            "title": "Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings",
            "authors": [
                "Mario Giulianelli"
            ],
            "abstract": "There exist two main approaches to automatically extract affective\norientation: lexicon-based and corpus-based. In this work, we argue that these\ntwo methods are compatible and show that combining them can improve the\naccuracy of emotion classifiers. In particular, we introduce a novel variant of\nthe Label Propagation algorithm that is tailored to distributed word\nrepresentations, we apply batch gradient descent to accelerate the optimization\nof label propagation and to make the optimization feasible for large graphs,\nand we propose a reproducible method for emotion lexicon expansion. We conclude\nthat label propagation can expand an emotion lexicon in a meaningful way and\nthat the expanded emotion lexicon can be leveraged to improve the accuracy of\nan emotion classifier.",
            "journal ref": "Computational Linguistics in the Netherlands Journal, 8, 99-121\n  (2018). Retrieved from https://clinjournal.org/clinj/article/view/82",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03940v1",
            "date": "2017-08-13 17:38:17+00:00",
            "title": "Leveraging Sparse and Dense Feature Combinations for Sentiment Classification",
            "authors": [
                "Tao Yu",
                "Christopher Hidey",
                "Owen Rambow",
                "Kathleen McKeown"
            ],
            "abstract": "Neural networks are one of the most popular approaches for many natural\nlanguage processing tasks such as sentiment analysis. They often outperform\ntraditional machine learning models and achieve the state-of-art results on\nmost tasks. However, many existing deep learning models are complex, difficult\nto train and provide a limited improvement over simpler methods. We propose a\nsimple, robust and powerful model for sentiment classification. This model\noutperforms many deep learning models and achieves comparable results to other\ndeep learning models with complex architectures on sentiment analysis datasets.\nWe publish the code online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.03995v1",
            "date": "2017-08-14 02:40:20+00:00",
            "title": "Sentiment Analysis by Joint Learning of Word Embeddings and Classifier",
            "authors": [
                "Prathusha Kameswara Sarma",
                "Bill Sethares"
            ],
            "abstract": "Word embeddings are representations of individual words of a text document in\na vector space and they are often use- ful for performing natural language pro-\ncessing tasks. Current state of the art al- gorithms for learning word\nembeddings learn vector representations from large corpora of text documents in\nan unsu- pervised fashion. This paper introduces SWESA (Supervised Word\nEmbeddings for Sentiment Analysis), an algorithm for sentiment analysis via\nword embeddings. SWESA leverages document label infor- mation to learn vector\nrepresentations of words from a modest corpus of text doc- uments by solving an\noptimization prob- lem that minimizes a cost function with respect to both word\nembeddings as well as classification accuracy. Analysis re- veals that SWESA\nprovides an efficient way of estimating the dimension of the word embeddings\nthat are to be learned. Experiments on several real world data sets show that\nSWESA has superior per- formance when compared to previously suggested\napproaches to word embeddings and sentiment analysis tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04358v1",
            "date": "2017-08-14 23:52:02+00:00",
            "title": "Continuous Representation of Location for Geolocation and Lexical Dialectology using Mixture Density Networks",
            "authors": [
                "Afshin Rahimi",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "We propose a method for embedding two-dimensional locations in a continuous\nvector space using a neural network-based model incorporating mixtures of\nGaussian distributions, presenting two model variants for text-based\ngeolocation and lexical dialectology. Evaluated over Twitter data, the proposed\nmodel outperforms conventional regression-based geolocation and provides a\nbetter estimate of uncertainty. We also show the effectiveness of the\nrepresentation for predicting words from location in lexical dialectology, and\nevaluate it using the DARE dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04439v2",
            "date": "2017-08-15 09:08:50+00:00",
            "title": "Extractive Summarization using Deep Learning",
            "authors": [
                "Sukriti Verma",
                "Vagisha Nidhi"
            ],
            "abstract": "This paper proposes a text summarization approach for factual reports using a\ndeep learning model. This approach consists of three phases: feature\nextraction, feature enhancement, and summary generation, which work together to\nassimilate core information and generate a coherent, understandable summary. We\nare exploring various features to improve the set of sentences selected for the\nsummary, and are using a Restricted Boltzmann Machine to enhance and abstract\nthose features to improve resultant accuracy without losing any important\ninformation. The sentences are scored based on those enhanced features and an\nextractive summary is constructed. Experimentation carried out on several\narticles demonstrates the effectiveness of the proposed approach. Source code\navailable at: https://github.com/vagisha-nidhi/TextSummarizer",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04557v1",
            "date": "2017-08-15 15:34:33+00:00",
            "title": "Database of Parliamentary Speeches in Ireland, 1919-2013",
            "authors": [
                "Alexander Herzog",
                "Slava J. Mikhaylov"
            ],
            "abstract": "We present a database of parliamentary debates that contains the complete\nrecord of parliamentary speeches from D\\'ail \\'Eireann, the lower house and\nprincipal chamber of the Irish parliament, from 1919 to 2013. In addition, the\ndatabase contains background information on all TDs (Teachta D\\'ala, members of\nparliament), such as their party affiliations, constituencies and office\npositions. The current version of the database includes close to 4.5 million\nspeeches from 1,178 TDs. The speeches were downloaded from the official\nparliament website and further processed and parsed with a Python script.\nBackground information on TDs was collected from the member database of the\nparliament website. Data on cabinet positions (ministers and junior ministers)\nwas collected from the official website of the government. A record linkage\nalgorithm and human coders were used to match TDs and ministers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04587v1",
            "date": "2017-08-15 16:44:28+00:00",
            "title": "Automatic Summarization of Online Debates",
            "authors": [
                "Nattapong Sanchan",
                "Ahmet Aker",
                "Kalina Bontcheva"
            ],
            "abstract": "Debate summarization is one of the novel and challenging research areas in\nautomatic text summarization which has been largely unexplored. In this paper,\nwe develop a debate summarization pipeline to summarize key topics which are\ndiscussed or argued in the two opposing sides of online debates. We view that\nthe generation of debate summaries can be achieved by clustering, cluster\nlabeling, and visualization. In our work, we investigate two different\nclustering approaches for the generation of the summaries. In the first\napproach, we generate the summaries by applying purely term-based clustering\nand cluster labeling. The second approach makes use of X-means for clustering\nand Mutual Information for labeling the clusters. Both approaches are driven by\nontologies. We visualize the results using bar charts. We think that our\nresults are a smooth entry for users aiming to receive the first impression\nabout what is discussed within a debate topic containing waste number of\nargumentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04592v1",
            "date": "2017-08-15 16:52:22+00:00",
            "title": "Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data",
            "authors": [
                "Nattapong Sanchan",
                "Ahmet Aker",
                "Kalina Bontcheva"
            ],
            "abstract": "Usage of online textual media is steadily increasing. Daily, more and more\nnews stories, blog posts and scientific articles are added to the online\nvolumes. These are all freely accessible and have been employed extensively in\nmultiple research areas, e.g. automatic text summarization, information\nretrieval, information extraction, etc. Meanwhile, online debate forums have\nrecently become popular, but have remained largely unexplored. For this reason,\nthere are no sufficient resources of annotated debate data available for\nconducting research in this genre. In this paper, we collected and annotated\ndebate data for an automatic summarization task. Similar to extractive gold\nstandard summary generation our data contains sentences worthy to include into\na summary. Five human annotators performed this task. Inter-annotator\nagreement, based on semantic similarity, is 36% for Cohen's kappa and 48% for\nKrippendorff's alpha. Moreover, we also implement an extractive summarization\nsystem for online debates and discuss prominent features for the task of\nsummarizing online debate data automatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04686v1",
            "date": "2017-08-15 20:47:02+00:00",
            "title": "VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation",
            "authors": [
                "Chuang Gan",
                "Yandong Li",
                "Haoxiang Li",
                "Chen Sun",
                "Boqing Gong"
            ],
            "abstract": "Rich and dense human labeled datasets are among the main enabling factors for\nthe recent advance on vision-language understanding. Many seemingly distant\nannotations (e.g., semantic segmentation and visual question answering (VQA))\nare inherently connected in that they reveal different levels and perspectives\nof human understandings about the same visual scenes --- and even the same set\nof images (e.g., of COCO). The popularity of COCO correlates those annotations\nand tasks. Explicitly linking them up may significantly benefit both individual\ntasks and the unified vision and language modeling. We present the preliminary\nwork of linking the instance segmentations provided by COCO to the questions\nand answers (QAs) in the VQA dataset, and name the collected links visual\nquestions and segmentation answers (VQS). They transfer human supervision\nbetween the previously separate tasks, offer more effective leverage to\nexisting problems, and also open the door for new research problems and models.\nWe study two applications of the VQS data in this paper: supervised attention\nfor VQA and a novel question-focused semantic segmentation task. For the\nformer, we obtain state-of-the-art results on the VQA real multiple-choice task\nby simply augmenting the multilayer perceptrons with some attention features\nthat are learned using the segmentation-QA links as explicit supervision. To\nput the latter in perspective, we study two plausible methods and compare them\nto an oracle method assuming that the instance segmentations are given at the\ntest stage.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.04729v3",
            "date": "2017-08-16 00:52:32+00:00",
            "title": "Deconvolutional Paragraph Representation Learning",
            "authors": [
                "Yizhe Zhang",
                "Dinghan Shen",
                "Guoyin Wang",
                "Zhe Gan",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "Learning latent representations from long text sequences is an important\nfirst step in many natural language processing applications. Recurrent Neural\nNetworks (RNNs) have become a cornerstone for this challenging task. However,\nthe quality of sentences during RNN-based decoding (reconstruction) decreases\nwith the length of the text. We propose a sequence-to-sequence, purely\nconvolutional and deconvolutional autoencoding framework that is free of the\nabove issue, while also being computationally efficient. The proposed method is\nsimple, easy to implement and can be leveraged as a building block for many\napplications. We show empirically that compared to RNNs, our framework is\nbetter at reconstructing and correcting long paragraphs. Quantitative\nevaluation on semi-supervised text classification and summarization tasks\ndemonstrate the potential for better utilization of long unlabeled text data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.04776v1",
            "date": "2017-08-16 05:43:54+00:00",
            "title": "Modality-specific Cross-modal Similarity Measurement with Recurrent Attention Network",
            "authors": [
                "Yuxin Peng",
                "Jinwei Qi",
                "Yuxin Yuan"
            ],
            "abstract": "Nowadays, cross-modal retrieval plays an indispensable role to flexibly find\ninformation across different modalities of data. Effectively measuring the\nsimilarity between different modalities of data is the key of cross-modal\nretrieval. Different modalities such as image and text have imbalanced and\ncomplementary relationships, which contain unequal amount of information when\ndescribing the same semantics. For example, images often contain more details\nthat cannot be demonstrated by textual descriptions and vice versa. Existing\nworks based on Deep Neural Network (DNN) mostly construct one common space for\ndifferent modalities to find the latent alignments between them, which lose\ntheir exclusive modality-specific characteristics. Different from the existing\nworks, we propose modality-specific cross-modal similarity measurement (MCSM)\napproach by constructing independent semantic space for each modality, which\nadopts end-to-end framework to directly generate modality-specific cross-modal\nsimilarity without explicit common representation. For each semantic space,\nmodality-specific characteristics within one modality are fully exploited by\nrecurrent attention network, while the data of another modality is projected\ninto this space with attention based joint embedding to utilize the learned\nattention weights for guiding the fine-grained cross-modal correlation\nlearning, which can capture the imbalanced and complementary relationships\nbetween different modalities. Finally, the complementarity between the semantic\nspaces for different modalities is explored by adaptive fusion of the\nmodality-specific cross-modal similarities to perform cross-modal retrieval.\nExperiments on the widely-used Wikipedia and Pascal Sentence datasets as well\nas our constructed large-scale XMediaNet dataset verify the effectiveness of\nour proposed approach, outperforming 9 state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1708.05045v2",
            "date": "2017-08-16 19:30:17+00:00",
            "title": "Cross-lingual Entity Alignment via Joint Attribute-Preserving Embedding",
            "authors": [
                "Zequn Sun",
                "Wei Hu",
                "Chengkai Li"
            ],
            "abstract": "Entity alignment is the task of finding entities in two knowledge bases (KBs)\nthat represent the same real-world object. When facing KBs in different natural\nlanguages, conventional cross-lingual entity alignment methods rely on machine\ntranslation to eliminate the language barriers. These approaches often suffer\nfrom the uneven quality of translations between languages. While recent\nembedding-based techniques encode entities and relationships in KBs and do not\nneed machine translation for cross-lingual entity alignment, a significant\nnumber of attributes remain largely unexplored. In this paper, we propose a\njoint attribute-preserving embedding model for cross-lingual entity alignment.\nIt jointly embeds the structures of two KBs into a unified vector space and\nfurther refines it by leveraging attribute correlations in the KBs. Our\nexperimental results on real-world datasets show that this approach\nsignificantly outperforms the state-of-the-art embedding approaches for\ncross-lingual entity alignment and could be complemented with methods based on\nmachine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05122v1",
            "date": "2017-08-17 03:27:53+00:00",
            "title": "Evaluating Visual Conversational Agents via Cooperative Human-AI Games",
            "authors": [
                "Prithvijit Chattopadhyay",
                "Deshraj Yadav",
                "Viraj Prabhu",
                "Arjun Chandrasekaran",
                "Abhishek Das",
                "Stefan Lee",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "As AI continues to advance, human-AI teams are inevitable. However, progress\nin AI is routinely measured in isolation, without a human in the loop. It is\ncrucial to benchmark progress in AI, not just in isolation, but also in terms\nof how it translates to helping humans perform certain tasks, i.e., the\nperformance of human-AI teams.\n  In this work, we design a cooperative game - GuessWhich - to measure human-AI\nteam performance in the specific context of the AI being a visual\nconversational agent. GuessWhich involves live interaction between the human\nand the AI. The AI, which we call ALICE, is provided an image which is unseen\nby the human. Following a brief description of the image, the human questions\nALICE about this secret image to identify it from a fixed pool of images.\n  We measure performance of the human-ALICE team by the number of guesses it\ntakes the human to correctly identify the secret image after a fixed number of\ndialog rounds with ALICE. We compare performance of the human-ALICE teams for\ntwo versions of ALICE. Our human studies suggest a counterintuitive trend -\nthat while AI literature shows that one version outperforms the other when\npaired with an AI questioner bot, we find that this improvement in AI-AI\nperformance does not translate to improved human-AI performance. This suggests\na mismatch between benchmarking of AI in isolation and in the context of\nhuman-AI teams.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1708.05565v2",
            "date": "2017-08-18 11:25:30+00:00",
            "title": "LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions",
            "authors": [
                "Yu Wang",
                "Jiayi Liu",
                "Yuxiang Liu",
                "Jun Hao",
                "Yang He",
                "Jinghe Hu",
                "Weipeng P. Yan",
                "Mantian Li"
            ],
            "abstract": "We present LADDER, the first deep reinforcement learning agent that can\nsuccessfully learn control policies for large-scale real-world problems\ndirectly from raw inputs composed of high-level semantic information. The agent\nis based on an asynchronous stochastic variant of DQN (Deep Q Network) named\nDASQN. The inputs of the agent are plain-text descriptions of states of a game\nof incomplete information, i.e. real-time large scale online auctions, and the\nrewards are auction profits of very large scale. We apply the agent to an\nessential portion of JD's online RTB (real-time bidding) advertising business\nand find that it easily beats the former state-of-the-art bidding policy that\nhad been carefully engineered and calibrated by human experts: during JD.com's\nJune 18th anniversary sale, the agent increased the company's ads revenue from\nthe portion by more than 50%, while the advertisers' ROI (return on investment)\nalso improved significantly.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.05682v1",
            "date": "2017-08-17 01:37:21+00:00",
            "title": "An Improved Residual LSTM Architecture for Acoustic Modeling",
            "authors": [
                "Lu Huang",
                "Jiasong Sun",
                "Ji Xu",
                "Yi Yang"
            ],
            "abstract": "Long Short-Term Memory (LSTM) is the primary recurrent neural networks\narchitecture for acoustic modeling in automatic speech recognition systems.\nResidual learning is an efficient method to help neural networks converge\neasier and faster. In this paper, we propose several types of residual LSTM\nmethods for our acoustic modeling. Our experiments indicate that, compared with\nclassic LSTM, our architecture shows more than 8% relative reduction in Phone\nError Rate (PER) on TIMIT tasks. At the same time, our residual fast LSTM\napproach shows 4% relative reduction in PER on the same task. Besides, we find\nthat all this architecture could have good results on THCHS-30, Librispeech and\nSwitchboard corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.05963v1",
            "date": "2017-08-20 13:37:06+00:00",
            "title": "Neural Networks Compression for Language Modeling",
            "authors": [
                "Artem M. Grachev",
                "Dmitry I. Ignatov",
                "Andrey V. Savchenko"
            ],
            "abstract": "In this paper, we consider several compression techniques for the language\nmodeling problem based on recurrent neural networks (RNNs). It is known that\nconventional RNNs, e.g, LSTM-based networks in language modeling, are\ncharacterized with either high space complexity or substantial inference time.\nThis problem is especially crucial for mobile applications, in which the\nconstant interaction with the remote server is inappropriate. By using the Penn\nTreebank (PTB) dataset we compare pruning, quantization, low-rank\nfactorization, tensor train decomposition for LSTM networks in terms of model\nsize and suitability for fast inference.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1708.06000v1",
            "date": "2017-08-20 18:17:27+00:00",
            "title": "Efficient Online Inference for Infinite Evolutionary Cluster models with Applications to Latent Social Event Discovery",
            "authors": [
                "Wei Wei",
                "Kennth Joseph",
                "Kathleen Carley"
            ],
            "abstract": "The Recurrent Chinese Restaurant Process (RCRP) is a powerful statistical\nmethod for modeling evolving clusters in large scale social media data. With\nthe RCRP, one can allow both the number of clusters and the cluster parameters\nin a model to change over time. However, application of the RCRP has largely\nbeen limited due to the non-conjugacy between the cluster evolutionary priors\nand the Multinomial likelihood. This non-conjugacy makes inference di cult and\nrestricts the scalability of models which use the RCRP, leading to the RCRP\nbeing applied only in simple problems, such as those that can be approximated\nby a single Gaussian emission. In this paper, we provide a novel solution for\nthe non-conjugacy issues for the RCRP and an example of how to leverage our\nsolution for one speci c problem - the social event discovery problem. By\nutilizing Sequential Monte Carlo methods in inference, our approach can be\nmassively paralleled and is highly scalable, to the extent it can work on tens\nof millions of documents. We are able to generate high quality topical and\nlocation distributions of the clusters that can be directly interpreted as real\nsocial events, and our experimental results suggest that the approaches\nproposed achieve much better predictive performance than techniques reported in\nprior work. We also demonstrate how the techniques we develop can be used in a\nmuch more general ways toward similar problems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1708.06068v1",
            "date": "2017-08-21 03:06:07+00:00",
            "title": "Vector Space Model as Cognitive Space for Text Classification",
            "authors": [
                "Barathi Ganesh HB",
                "Anand Kumar M",
                "Soman KP"
            ],
            "abstract": "In this era of digitization, knowing the user's sociolect aspects have become\nessential features to build the user specific recommendation systems. These\nsociolect aspects could be found by mining the user's language sharing in the\nform of text in social media and reviews. This paper describes about the\nexperiment that was performed in PAN Author Profiling 2017 shared task. The\nobjective of the task is to find the sociolect aspects of the users from their\ntweets. The sociolect aspects considered in this experiment are user's gender\nand native language information. Here user's tweets written in a different\nlanguage from their native language are represented as Document - Term Matrix\nwith document frequency as the constraint. Further classification is done using\nthe Support Vector Machine by taking gender and native language as target\nclasses. This experiment attains the average accuracy of 73.42% in gender\nprediction and 76.26% in the native language identification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06828v1",
            "date": "2017-08-22 21:30:23+00:00",
            "title": "Classification of Radiology Reports Using Neural Attention Models",
            "authors": [
                "Bonggun Shin",
                "Falgun H. Chokshi",
                "Timothy Lee",
                "Jinho D. Choi"
            ],
            "abstract": "The electronic health record (EHR) contains a large amount of\nmulti-dimensional and unstructured clinical data of significant operational and\nresearch value. Distinguished from previous studies, our approach embraces a\ndouble-annotated dataset and strays away from obscure \"black-box\" models to\ncomprehensive deep learning models. In this paper, we present a novel neural\nattention mechanism that not only classifies clinically important findings.\nSpecifically, convolutional neural networks (CNN) with attention analysis are\nused to classify radiology head computed tomography reports based on five\ncategories that radiologists would account for in assessing acute and\ncommunicable findings in daily practice. The experiments show that our CNN\nattention models outperform non-neural models, especially when trained on a\nlarger dataset. Our attention analysis demonstrates the intuition behind the\nclassifier's decision by generating a heatmap that highlights attended terms\nused by the CNN model; this is valuable when potential downstream medical\ndecisions are to be performed by human experts or the classifier information is\nto be used in cohort construction such as for epidemiological studies.",
            "journal ref": "In Proceedings of the International Joint Conference on Neural\n  Networks, of IJCNN'17, pages 4363--4370, Anchorage, AK, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.06872v3",
            "date": "2017-08-23 02:42:59+00:00",
            "title": "Discovering Political Topics in Facebook Discussion threads with Graph Contextualization",
            "authors": [
                "Yilin Zhang",
                "Marie Poux-Berthe",
                "Chris Wells",
                "Karolina Koc-Michalska",
                "Karl Rohe"
            ],
            "abstract": "We propose a graph contextualization method, pairGraphText, to study\npolitical engagement on Facebook during the 2012 French presidential election.\nIt is a spectral algorithm that contextualizes graph data with text data for\nonline discussion thread. In particular, we examine the Facebook posts of the\neight leading candidates and the comments beneath these posts. We find evidence\nof both (i) candidate-centered structure, where citizens primarily comment on\nthe wall of one candidate and (ii) issue-centered structure (i.e. on political\ntopics), where citizens' attention and expression is primarily directed towards\na specific set of issues (e.g. economics, immigration, etc). To identify\nissue-centered structure, we develop pairGraphText, to analyze a network with\nhigh-dimensional features on the interactions (i.e. text). This technique\nscales to hundreds of thousands of nodes and thousands of unique words. In the\nFacebook data, spectral clustering without the contextualizing text information\nfinds a mixture of (i) candidate and (ii) issue clusters. The contextualized\ninformation with text data helps to separate these two structures. We conclude\nby showing that the novel methodology is consistent under a statistical model.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1708.07149v2",
            "date": "2017-08-23 18:56:00+00:00",
            "title": "Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses",
            "authors": [
                "Ryan Lowe",
                "Michael Noseworthy",
                "Iulian V. Serban",
                "Nicolas Angelard-Gontier",
                "Yoshua Bengio",
                "Joelle Pineau"
            ],
            "abstract": "Automatically evaluating the quality of dialogue responses for unstructured\ndomains is a challenging problem. Unfortunately, existing automatic evaluation\nmetrics are biased and correlate very poorly with human judgements of response\nquality. Yet having an accurate automatic evaluation procedure is crucial for\ndialogue research, as it allows rapid prototyping and testing of new models\nwith fewer expensive human evaluations. In response to this challenge, we\nformulate automatic dialogue evaluation as a learning problem. We present an\nevaluation model (ADEM) that learns to predict human-like scores to input\nresponses, using a new dataset of human response scores. We show that the ADEM\nmodel's predictions correlate significantly, and at a level much higher than\nword-overlap metrics such as BLEU, with human judgements at both the utterance\nand system-level. We also show that ADEM can generalize to evaluating dialogue\nmodels unseen during training, an important step for automatic dialogue\nevaluation.",
            "journal ref": "Proceedings of the 55th annual meeting on Association for\n  Computational Linguistics (2017), pp. 1116-1126",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07524v2",
            "date": "2017-08-24 18:51:50+00:00",
            "title": "Supervised Speech Separation Based on Deep Learning: An Overview",
            "authors": [
                "DeLiang Wang",
                "Jitong Chen"
            ],
            "abstract": "Speech separation is the task of separating target speech from background\ninterference. Traditionally, speech separation is studied as a signal\nprocessing problem. A more recent approach formulates speech separation as a\nsupervised learning problem, where the discriminative patterns of speech,\nspeakers, and background noise are learned from training data. Over the past\ndecade, many supervised separation algorithms have been put forward. In\nparticular, the recent introduction of deep learning to supervised speech\nseparation has dramatically accelerated progress and boosted separation\nperformance. This article provides a comprehensive overview of the research on\ndeep learning based supervised speech separation in the last several years. We\nfirst introduce the background of speech separation and the formulation of\nsupervised separation. Then we discuss three main components of supervised\nseparation: learning machines, training targets, and acoustic features. Much of\nthe overview is on separation algorithms where we review monaural methods,\nincluding speech enhancement (speech-nonspeech separation), speaker separation\n(multi-talker separation), and speech dereverberation, as well as\nmulti-microphone techniques. The important issue of generalization, unique to\nsupervised learning, is discussed. This overview provides a historical\nperspective on how advances are made. In addition, we discuss a number of\nconceptual issues, including what constitutes the target source.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07722v3",
            "date": "2017-08-24 04:59:53+00:00",
            "title": "A dependency look at the reality of constituency",
            "authors": [
                "Xinying Chen",
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "A comment on \"Neurophysiological dynamics of phrase-structure building during\nsentence processing\" by Nelson et al (2017), Proceedings of the National\nAcademy of Sciences USA 114(18), E3669-E3678.",
            "journal ref": "Glottometrics 40, 104-106 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.07918v2",
            "date": "2017-08-26 02:29:50+00:00",
            "title": "Robust Task Clustering for Deep Many-Task Learning",
            "authors": [
                "Mo Yu",
                "Xiaoxiao Guo",
                "Jinfeng Yi",
                "Shiyu Chang",
                "Saloni Potdar",
                "Gerald Tesauro",
                "Haoyu Wang",
                "Bowen Zhou"
            ],
            "abstract": "We investigate task clustering for deep-learning based multi-task and\nfew-shot learning in a many-task setting. We propose a new method to measure\ntask similarities with cross-task transfer performance matrix for the deep\nlearning scenario. Although this matrix provides us critical information\nregarding similarity between tasks, its asymmetric property and unreliable\nperformance scores can affect conventional clustering methods adversely.\nAdditionally, the uncertain task-pairs, i.e., the ones with extremely\nasymmetric transfer scores, may collectively mislead clustering algorithms to\noutput an inaccurate task-partition. To overcome these limitations, we propose\na novel task-clustering algorithm by using the matrix completion technique. The\nproposed algorithm constructs a partially-observed similarity matrix based on\nthe certainty of cluster membership of the task-pairs. We then use a matrix\ncompletion algorithm to complete the similarity matrix. Our theoretical\nanalysis shows that under mild constraints, the proposed algorithm will\nperfectly recover the underlying \"true\" similarity matrix with a high\nprobability. Our results show that the new task clustering method can discover\ntask clusters for training flexible and superior neural network models in a\nmulti-task learning setup for sentiment classification and dialog intent\nclassification tasks. Our task clustering approach also extends metric-based\nfew-shot learning methods to adapt multiple metrics, which demonstrates\nempirical advantages when the tasks are diverse.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1708.08289v1",
            "date": "2017-08-28 12:44:14+00:00",
            "title": "Generating Query Suggestions to Support Task-Based Search",
            "authors": [
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "We address the problem of generating query suggestions to support users in\ncompleting their underlying tasks (which motivated them to search in the first\nplace). Given an initial query, these query suggestions should provide a\ncoverage of possible subtasks the user might be looking for. We propose a\nprobabilistic modeling framework that obtains keyphrases from multiple sources\nand generates query suggestions from these keyphrases. Using the test suites of\nthe TREC Tasks track, we evaluate and analyze each component of our model.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.08291v1",
            "date": "2017-08-28 12:47:04+00:00",
            "title": "On Type-Aware Entity Retrieval",
            "authors": [
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "Today, the practice of returning entities from a knowledge base in response\nto search queries has become widespread. One of the distinctive characteristics\nof entities is that they are typed, i.e., assigned to some hierarchically\norganized type system (type taxonomy). The primary objective of this paper is\nto gain a better understanding of how entity type information can be utilized\nin entity retrieval. We perform this investigation in an idealized \"oracle\"\nsetting, assuming that we know the distribution of target types of the relevant\nentities for a given query. We perform a thorough analysis of three main\naspects: (i) the choice of type taxonomy, (ii) the representation of\nhierarchical type information, and (iii) the combination of type-based and\nterm-based similarity in the retrieval model. Using a standard entity search\ntest collection based on DBpedia, we find that type information proves most\nuseful when using large type taxonomies that provide very specific types. We\nprovide further insights on the extensional coverage of entities and on the\nutility of target types.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1708.09025v1",
            "date": "2017-08-29 21:04:11+00:00",
            "title": "Unsupervised Terminological Ontology Learning based on Hierarchical Topic Modeling",
            "authors": [
                "Xiaofeng Zhu",
                "Diego Klabjan",
                "Patrick Bless"
            ],
            "abstract": "In this paper, we present hierarchical relationbased latent Dirichlet\nallocation (hrLDA), a data-driven hierarchical topic model for extracting\nterminological ontologies from a large number of heterogeneous documents. In\ncontrast to traditional topic models, hrLDA relies on noun phrases instead of\nunigrams, considers syntax and document structures, and enriches topic\nhierarchies with topic relations. Through a series of experiments, we\ndemonstrate the superiority of hrLDA over existing topic models, especially for\nbuilding hierarchies. Furthermore, we illustrate the robustness of hrLDA in the\nsettings of noisy data sets, which are likely to occur in many practical\nscenarios. Our ontology evaluation results show that ontologies extracted from\nhrLDA are very competitive with the ontologies created by domain experts.",
            "journal ref": "IRI 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.09040v1",
            "date": "2017-08-29 21:40:22+00:00",
            "title": "Modelling Protagonist Goals and Desires in First-Person Narrative",
            "authors": [
                "Elahe Rahimtoroghi",
                "Jiaqi Wu",
                "Ruimin Wang",
                "Pranav Anand",
                "Marilyn A Walker"
            ],
            "abstract": "Many genres of natural language text are narratively structured, a testament\nto our predilection for organizing our experiences as narratives. There is\nbroad consensus that understanding a narrative requires identifying and\ntracking the goals and desires of the characters and their narrative outcomes.\nHowever, to date, there has been limited work on computational models for this\nproblem. We introduce a new dataset, DesireDB, which includes gold-standard\nlabels for identifying statements of desire, textual evidence for desire\nfulfillment, and annotations for whether the stated desire is fulfilled given\nthe evidence in the narrative context. We report experiments on tracking desire\nfulfillment using different methods, and show that LSTM Skip-Thought model\nachieves F-measure of 0.7 on our corpus.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1708.09516v1",
            "date": "2017-08-31 01:00:19+00:00",
            "title": "Leveraging Deep Neural Network Activation Entropy to cope with Unseen Data in Speech Recognition",
            "authors": [
                "Vikramjit Mitra",
                "Horacio Franco"
            ],
            "abstract": "Unseen data conditions can inflict serious performance degradation on systems\nrelying on supervised machine learning algorithms. Because data can often be\nunseen, and because traditional machine learning algorithms are trained in a\nsupervised manner, unsupervised adaptation techniques must be used to adapt the\nmodel to the unseen data conditions. However, unsupervised adaptation is often\nchallenging, as one must generate some hypothesis given a model and then use\nthat hypothesis to bootstrap the model to the unseen data conditions.\nUnfortunately, reliability of such hypotheses is often poor, given the mismatch\nbetween the training and testing datasets. In such cases, a model hypothesis\nconfidence measure enables performing data selection for the model adaptation.\nUnderlying this approach is the fact that for unseen data conditions, data\nvariability is introduced to the model, which the model propagates to its\noutput decision, impacting decision reliability. In a fully connected network,\nthis data variability is propagated as distortions from one layer to the next.\nThis work aims to estimate the propagation of such distortion in the form of\nnetwork activation entropy, which is measured over a short- time running window\non the activation from each neuron of a given hidden layer, and these\nmeasurements are then used to compute summary entropy. This work demonstrates\nthat such an entropy measure can help to select data for unsupervised model\nadaptation, resulting in performance gains in speech recognition tasks. Results\nfrom standard benchmark speech recognition tasks show that the proposed\napproach can alleviate the performance degradation experienced under unseen\ndata conditions by iteratively adapting the model to the unseen datas acoustic\ncondition.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.00641v1",
            "date": "2017-09-29 12:40:50+00:00",
            "title": "Improving speech recognition by revising gated recurrent units",
            "authors": [
                "Mirco Ravanelli",
                "Philemon Brakel",
                "Maurizio Omologo",
                "Yoshua Bengio"
            ],
            "abstract": "Speech recognition is largely taking advantage of deep learning, showing that\nsubstantial benefits can be obtained by modern Recurrent Neural Networks\n(RNNs). The most popular RNNs are Long Short-Term Memory (LSTMs), which\ntypically reach state-of-the-art performance in many tasks thanks to their\nability to learn long-term dependencies and robustness to vanishing gradients.\nNevertheless, LSTMs have a rather complex design with three multiplicative\ngates, that might impair their efficient implementation. An attempt to simplify\nLSTMs has recently led to Gated Recurrent Units (GRUs), which are based on just\ntwo multiplicative gates.\n  This paper builds on these efforts by further revising GRUs and proposing a\nsimplified architecture potentially more suitable for speech recognition. The\ncontribution of this work is two-fold. First, we suggest to remove the reset\ngate in the GRU design, resulting in a more efficient single-gate architecture.\nSecond, we propose to replace tanh with ReLU activations in the state update\nequations. Results show that, in our implementation, the revised architecture\nreduces the per-epoch training time with more than 30% and consistently\nimproves recognition performance across different tasks, input features, and\nnoisy conditions when compared to a standard GRU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00683v1",
            "date": "2017-09-28 00:39:25+00:00",
            "title": "The Dependence of Frequency Distributions on Multiple Meanings of Words, Codes and Signs",
            "authors": [
                "Xiaoyong Yan",
                "Petter Minnhagen"
            ],
            "abstract": "The dependence of the frequency distributions due to multiple meanings of\nwords in a text is investigated by deleting letters. By coding the words with\nfewer letters the number of meanings per coded word increases. This increase is\nmeasured and used as an input in a predictive theory. For a text written in\nEnglish, the word-frequency distribution is broad and fat-tailed, whereas if\nthe words are only represented by their first letter the distribution becomes\nexponential. Both distribution are well predicted by the theory, as is the\nwhole sequence obtained by consecutively representing the words by the first\nL=6,5,4,3,2,1 letters. Comparisons of texts written by Chinese characters and\nthe same texts written by letter-codes are made and the similarity of the\ncorresponding frequency-distributions are interpreted as a consequence of the\nmultiple meanings of Chinese characters. This further implies that the\ndifference of the shape for word-frequencies for an English text written by\nletters and a Chinese text written by Chinese characters is due to the coding\nand not to the language per se.",
            "journal ref": "Physica A 490, 554-564 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.00888v2",
            "date": "2017-10-02 20:07:18+00:00",
            "title": "Sentiment Perception of Readers and Writers in Emoji use",
            "authors": [
                "Jose Berengueres",
                "Dani Castro"
            ],
            "abstract": "Previous research has traditionally analyzed emoji sentiment from the point\nof view of the reader of the content not the author. Here, we analyze emoji\nsentiment from the point of view of the author and present a emoji sentiment\nbenchmark that was built from an employee happiness dataset where emoji happen\nto be annotated with daily happiness of the author of the comment. The data\nspans over 3 years, and 4k employees of 56 companies based in Barcelona. We\ncompare sentiment of writers to readers. Results indicate that, there is an 82%\nagreement in how emoji sentiment is perceived by readers and writers. Finally,\nwe report that when authors use emoji they report higher levels of happiness.\nEmoji use was not found to be correlated with differences in author moodiness.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.01093v1",
            "date": "2017-10-03 11:44:40+00:00",
            "title": "Which phoneme-to-viseme maps best improve visual-only computer lip-reading?",
            "authors": [
                "Helen L. Bear",
                "Richard W. Harvey",
                "Barry-John Theobald",
                "Yuxuan Lan"
            ],
            "abstract": "A critical assumption of all current visual speech recognition systems is\nthat there are visual speech units called visemes which can be mapped to units\nof acoustic speech, the phonemes. Despite there being a number of published\nmaps it is infrequent to see the effectiveness of these tested, particularly on\nvisual-only lip-reading (many works use audio-visual speech). Here we examine\n120 mappings and consider if any are stable across talkers. We show a method\nfor devising maps based on phoneme confusions from an automated lip-reading\nsystem, and we present new mappings that show improvements for individual\ntalkers.",
            "journal ref": "Helen L. Bear, Richard W. Harvey, Barry-John Theobald, and Yuxuan\n  Lan. Which phoneme-to-viseme maps best improve visual-only computer\n  lip-reading? Advances in Visual Computing 2014. p230-239",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1710.01142v1",
            "date": "2017-10-03 13:32:40+00:00",
            "title": "Finding phonemes: improving machine lip-reading",
            "authors": [
                "Helen L. Bear",
                "Richard W. Harvey",
                "Yuxuan Lan"
            ],
            "abstract": "In machine lip-reading there is continued debate and research around the\ncorrect classes to be used for recognition. In this paper we use a structured\napproach for devising speaker-dependent viseme classes, which enables the\ncreation of a set of phoneme-to-viseme maps where each has a different quantity\nof visemes ranging from two to 45. Viseme classes are based upon the mapping of\narticulated phonemes, which have been confused during phoneme recognition, into\nviseme groups. Using these maps, with the LiLIR dataset, we show the effect of\nchanging the viseme map size in speaker-dependent machine lip-reading, measured\nby word recognition correctness and so demonstrate that word recognition with\nphoneme classifiers is not just possible, but often better than word\nrecognition with viseme classifiers. Furthermore, there are intermediate units\nbetween visemes and phonemes which are better still.",
            "journal ref": "Helen L. Bear, Richard W. Harvey, Yuxuan Lan. Finding phonemes:\n  improving machine lip-reading. Audio-Visual Speech Processing (AVSP), 2015\n  p115-120",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1710.01507v4",
            "date": "2017-10-04 08:53:12+00:00",
            "title": "Identifying Clickbait: A Multi-Strategy Approach Using Neural Networks",
            "authors": [
                "Vaibhav Kumar",
                "Dhruv Khattar",
                "Siddhartha Gairola",
                "Yash Kumar Lal",
                "Vasudeva Varma"
            ],
            "abstract": "Online media outlets, in a bid to expand their reach and subsequently\nincrease revenue through ad monetisation, have begun adopting clickbait\ntechniques to lure readers to click on articles. The article fails to fulfill\nthe promise made by the headline. Traditional methods for clickbait detection\nhave relied heavily on feature engineering which, in turn, is dependent on the\ndataset it is built for. The application of neural networks for this task has\nonly been explored partially. We propose a novel approach considering all\ninformation found in a social media post. We train a bidirectional LSTM with an\nattention mechanism to learn the extent to which a word contributes to the\npost's clickbait score in a differential manner. We also employ a Siamese net\nto capture the similarity between source and target information. Information\ngleaned from images has not been considered in previous approaches. We learn\nimage embeddings from large amounts of data using Convolutional Neural Networks\nto add another layer of complexity to our model. Finally, we concatenate the\noutputs from the three separate components, serving it as input to a fully\nconnected layer. We conduct experiments over a test corpus of 19538 social\nmedia posts, attaining an F1 score of 65.37% on the dataset bettering the\nprevious state-of-the-art, as well as other proposed approaches, feature\nengineering or otherwise.",
            "journal ref": "\"Identifying Clickbait: A Multi-Strategy Approach Using Neural\n  Networks\". In Proceedings of the 41st International ACM SIGIR Conference on\n  Research and Development in Information Retrieval 2018. Pages: 1225-1228",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.01949v2",
            "date": "2017-10-05 10:24:46+00:00",
            "title": "Semantic speech retrieval with a visually grounded model of untranscribed speech",
            "authors": [
                "Herman Kamper",
                "Gregory Shakhnarovich",
                "Karen Livescu"
            ],
            "abstract": "There is growing interest in models that can learn from unlabelled speech\npaired with visual context. This setting is relevant for low-resource speech\nprocessing, robotics, and human language acquisition research. Here we study\nhow a visually grounded speech model, trained on images of scenes paired with\nspoken captions, captures aspects of semantics. We use an external image tagger\nto generate soft text labels from images, which serve as targets for a neural\nmodel that maps untranscribed speech to (semantic) keyword labels. We introduce\na newly collected data set of human semantic relevance judgements and an\nassociated task, semantic speech retrieval, where the goal is to search for\nspoken utterances that are semantically relevant to a given text query. Without\nseeing any text, the model trained on parallel speech and images achieves a\nprecision of almost 60% on its top ten semantic retrievals. Compared to a\nsupervised model trained on transcriptions, our model matches human judgements\nbetter by some measures, especially in retrieving non-verbatim semantic\nmatches. We perform an extensive analysis of the model and its resulting\nrepresentations.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech and Language Processing 27\n  (2019) 89-98",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.02560v1",
            "date": "2017-10-06 19:20:38+00:00",
            "title": "The DIRHA-English corpus and related tasks for distant-speech recognition in domestic environments",
            "authors": [
                "Mirco Ravanelli",
                "Maurizio Omologo"
            ],
            "abstract": "This paper introduces the contents and the possible usage of the\nDIRHA-ENGLISH multi-microphone corpus, recently realized under the EC DIRHA\nproject. The reference scenario is a domestic environment equipped with a large\nnumber of microphones and microphone arrays distributed in space.\n  The corpus is composed of both real and simulated material, and it includes\n12 US and 12 UK English native speakers. Each speaker uttered different sets of\nphonetically-rich sentences, newspaper articles, conversational speech,\nkeywords, and commands. From this material, a large set of 1-minute sequences\nwas generated, which also includes typical domestic background noise as well as\ninter/intra-room reverberation effects. Dev and test sets were derived, which\nrepresent a very precious material for different studies on multi-microphone\nspeech processing and distant-speech recognition. Various tasks and\ncorresponding Kaldi recipes have already been developed.\n  The paper reports a first set of baseline results obtained using different\ntechniques, including Deep Neural Networks (DNN), aligned with the\nstate-of-the-art at international level.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1710.02650v2",
            "date": "2017-10-07 08:18:12+00:00",
            "title": "Topic Modeling based on Keywords and Context",
            "authors": [
                "Johannes Schneider"
            ],
            "abstract": "Current topic models often suffer from discovering topics not matching human\nintuition, unnatural switching of topics within documents and high\ncomputational demands. We address these concerns by proposing a topic model and\nan inference algorithm based on automatically identifying characteristic\nkeywords for topics. Keywords influence topic-assignments of nearby words. Our\nalgorithm learns (key)word-topic scores and it self-regulates the number of\ntopics. Inference is simple and easily parallelizable. Qualitative analysis\nyields comparable results to state-of-the-art models (eg. LDA), but with\ndifferent strengths and weaknesses. Quantitative analysis using 9 datasets\nshows gains in terms of classification accuracy, PMI score, computational\nperformance and consistency of topic assignments within documents, while most\noften using less topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.03538v1",
            "date": "2017-10-10 12:36:45+00:00",
            "title": "Contaminated speech training methods for robust DNN-HMM distant speech recognition",
            "authors": [
                "Mirco Ravanelli",
                "Maurizio Omologo"
            ],
            "abstract": "Despite the significant progress made in the last years, state-of-the-art\nspeech recognition technologies provide a satisfactory performance only in the\nclose-talking condition. Robustness of distant speech recognition in adverse\nacoustic conditions, on the other hand, remains a crucial open issue for future\napplications of human-machine interaction. To this end, several advances in\nspeech enhancement, acoustic scene analysis as well as acoustic modeling, have\nrecently contributed to improve the state-of-the-art in the field. One of the\nmost effective approaches to derive a robust acoustic modeling is based on\nusing contaminated speech, which proved helpful in reducing the acoustic\nmismatch between training and testing conditions.\n  In this paper, we revise this classical approach in the context of modern\nDNN-HMM systems, and propose the adoption of three methods, namely, asymmetric\ncontext windowing, close-talk based supervision, and close-talk based\npre-training. The experimental results, obtained using both real and simulated\ndata, show a significant advantage in using these three methods, overall\nproviding a 15% error rate reduction compared to the baseline systems. The same\ntrend in performance is confirmed either using a high-quality training set of\nsmall size, and a large one.",
            "journal ref": "INTERSPEECH 2015",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1710.04099v1",
            "date": "2017-10-11 14:56:27+00:00",
            "title": "Wembedder: Wikidata entity embedding web service",
            "authors": [
                "Finn \u00c5rup Nielsen"
            ],
            "abstract": "I present a web service for querying an embedding of entities in the Wikidata\nknowledge graph. The embedding is trained on the Wikidata dump using Gensim's\nWord2Vec implementation and a simple graph walk. A REST API is implemented.\nTogether with the Wikidata API the web service exposes a multilingual resource\nfor over 600'000 Wikidata items and properties.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1710.04312v1",
            "date": "2017-10-11 21:37:07+00:00",
            "title": "Measurement Context Extraction from Text: Discovering Opportunities and Gaps in Earth Science",
            "authors": [
                "Kyle Hundman",
                "Chris A. Mattmann"
            ],
            "abstract": "We propose Marve, a system for extracting measurement values, units, and\nrelated words from natural language text. Marve uses conditional random fields\n(CRF) to identify measurement values and units, followed by a rule-based system\nto find related entities, descriptors and modifiers within a sentence. Sentence\ntokens are represented by an undirected graphical model, and rules are based on\npart-of-speech and word dependency patterns connecting values and units to\ncontextual words. Marve is unique in its focus on measurement context and early\nexperimentation demonstrates Marve's ability to generate high-precision\nextractions with strong recall. We also discuss Marve's role in refining\nmeasurement requirements for NASA's proposed HyspIRI mission, a hyperspectral\ninfrared imaging satellite that will study the world's ecosystems. In general,\nour work with HyspIRI demonstrates the value of semantic measurement\nextractions in characterizing quantitative discussion contained in large\ncorpuses of natural language text. These extractions accelerate broad,\ncross-cutting research and expose scientists new algorithmic approaches and\nexperimental nuances. They also facilitate identification of scientific\nopportunities enabled by HyspIRI leading to more efficient scientific\ninvestment and research.",
            "journal ref": "23rd ACM SIGKDD International Conference on Knowledge Discovery\n  and Data Mining, Data-Driven Discovery Workshop, Halifax, Canada, August 2017",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1710.05298v2",
            "date": "2017-10-15 07:51:01+00:00",
            "title": "Text2Action: Generative Adversarial Synthesis from Language to Action",
            "authors": [
                "Hyemin Ahn",
                "Timothy Ha",
                "Yunho Choi",
                "Hwiyeon Yoo",
                "Songhwai Oh"
            ],
            "abstract": "In this paper, we propose a generative model which learns the relationship\nbetween language and human action in order to generate a human action sequence\ngiven a sentence describing human behavior. The proposed generative model is a\ngenerative adversarial network (GAN), which is based on the sequence to\nsequence (SEQ2SEQ) model. Using the proposed generative network, we can\nsynthesize various actions for a robot or a virtual agent using a text encoder\nrecurrent neural network (RNN) and an action decoder RNN. The proposed\ngenerative network is trained from 29,770 pairs of actions and sentence\nannotations extracted from MSR-Video-to-Text (MSR-VTT), a large-scale video\ndataset. We demonstrate that the network can generate human-like actions which\ncan be transferred to a Baxter robot, such that the robot performs an action\nbased on a provided sentence. Results show that the proposed generative network\ncorrectly models the relationship between language and action and can generate\na diverse set of actions from the same sentence.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.05780v3",
            "date": "2017-10-16 15:23:56+00:00",
            "title": "A retrieval-based dialogue system utilizing utterance and context embeddings",
            "authors": [
                "Alexander Bartl",
                "Gerasimos Spanakis"
            ],
            "abstract": "Finding semantically rich and computer-understandable representations for\ntextual dialogues, utterances and words is crucial for dialogue systems (or\nconversational agents), as their performance mostly depends on understanding\nthe context of conversations. Recent research aims at finding distributed\nvector representations (embeddings) for words, such that semantically similar\nwords are relatively close within the vector-space. Encoding the \"meaning\" of\ntext into vectors is a current trend, and text can range from words, phrases\nand documents to actual human-to-human conversations. In recent research\napproaches, responses have been generated utilizing a decoder architecture,\ngiven the vector representation of the current conversation. In this paper, the\nutilization of embeddings for answer retrieval is explored by using\nLocality-Sensitive Hashing Forest (LSH Forest), an Approximate Nearest Neighbor\n(ANN) model, to find similar conversations in a corpus and rank possible\ncandidates. Experimental results on the well-known Ubuntu Corpus (in English)\nand a customer service chat dataset (in Dutch) show that, in combination with a\ncandidate selection method, retrieval-based approaches outperform generative\nones and reveal promising future research directions towards the usability of\nsuch a system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06071v1",
            "date": "2017-10-17 03:22:00+00:00",
            "title": "PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts",
            "authors": [
                "Franck Dernoncourt",
                "Ji Young Lee"
            ],
            "abstract": "We present PubMed 200k RCT, a new dataset based on PubMed for sequential\nsentence classification. The dataset consists of approximately 200,000\nabstracts of randomized controlled trials, totaling 2.3 million sentences. Each\nsentence of each abstract is labeled with their role in the abstract using one\nof the following classes: background, objective, method, result, or conclusion.\nThe purpose of releasing this dataset is twofold. First, the majority of\ndatasets for sequential short-text classification (i.e., classification of\nshort texts that appear in sequences) are small: we hope that releasing a new\nlarge dataset will help develop more accurate algorithms for this task. Second,\nfrom an application perspective, researchers need better tools to efficiently\nskim through the literature. Automatically classifying each sentence in an\nabstract would help researchers read abstracts more efficiently, especially in\nfields where abstracts may be long, such as the medical field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.06390v1",
            "date": "2017-10-17 17:00:59+00:00",
            "title": "Fishing for Clickbaits in Social Images and Texts with Linguistically-Infused Neural Network Models",
            "authors": [
                "Maria Glenski",
                "Ellyn Ayton",
                "Dustin Arendt",
                "Svitlana Volkova"
            ],
            "abstract": "This paper presents the results and conclusions of our participation in the\nClickbait Challenge 2017 on automatic clickbait detection in social media. We\nfirst describe linguistically-infused neural network models and identify\ninformative representations to predict the level of clickbaiting present in\nTwitter posts. Our models allow to answer the question not only whether a post\nis a clickbait or not, but to what extent it is a clickbait post e.g., not at\nall, slightly, considerably, or heavily clickbaity using a score ranging from 0\nto 1. We evaluate the predictive power of models trained on varied text and\nimage representations extracted from tweets. Our best performing model that\nrelies on the tweet text and linguistic markers of biased language extracted\nfrom the tweet and the corresponding page yields mean squared error (MSE) of\n0.04, mean absolute error (MAE) of 0.16 and R2 of 0.43 on the held-out test\ndata. For the binary classification setup (clickbait vs. non-clickbait), our\nmodel achieved F1 score of 0.69. We have not found that image representations\ncombined with text yield significant performance improvement yet. Nevertheless,\nthis work is the first to present preliminary analysis of objects extracted\nusing Google Tensorflow object detection API from images in clickbait vs.\nnon-clickbait Twitter posts. Finally, we outline several steps to improve model\nperformance as a part of the future work.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.06406v1",
            "date": "2017-10-17 17:34:31+00:00",
            "title": "Laying Down the Yellow Brick Road: Development of a Wizard-of-Oz Interface for Collecting Human-Robot Dialogue",
            "authors": [
                "Claire Bonial",
                "Matthew Marge",
                "Ron artstein",
                "Ashley Foots",
                "Felix Gervits",
                "Cory J. Hayes",
                "Cassidy Henry",
                "Susan G. Hill",
                "Anton Leuski",
                "Stephanie M. Lukin",
                "Pooja Moolchandani",
                "Kimberly A. Pollard",
                "David Traum",
                "Clare R. Voss"
            ],
            "abstract": "We describe the adaptation and refinement of a graphical user interface\ndesigned to facilitate a Wizard-of-Oz (WoZ) approach to collecting human-robot\ndialogue data. The data collected will be used to develop a dialogue system for\nrobot navigation. Building on an interface previously used in the development\nof dialogue systems for virtual agents and video playback, we add templates\nwith open parameters which allow the wizard to quickly produce a wide variety\nof utterances. Our research demonstrates that this approach to data collection\nis viable as an intermediate step in developing a dialogue system for physical\nrobots in remote locations from their users - a domain in which the human and\nrobot need to regularly verify and update a shared understanding of the\nphysical environment. We show that our WoZ interface and the fixed set of\nutterances and templates therein provide for a natural pace of dialogue with\ngood coverage of the navigation domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07551v1",
            "date": "2017-10-20 14:41:43+00:00",
            "title": "Spoken Language Biomarkers for Detecting Cognitive Impairment",
            "authors": [
                "Tuka Alhanai",
                "Rhoda Au",
                "James Glass"
            ],
            "abstract": "In this study we developed an automated system that evaluates speech and\nlanguage features from audio recordings of neuropsychological examinations of\n92 subjects in the Framingham Heart Study. A total of 265 features were used in\nan elastic-net regularized binomial logistic regression model to classify the\npresence of cognitive impairment, and to select the most predictive features.\nWe compared performance with a demographic model from 6,258 subjects in the\ngreater study cohort (0.79 AUC), and found that a system that incorporated both\naudio and text features performed the best (0.92 AUC), with a True Positive\nRate of 29% (at 0% False Positive Rate) and a good model fit (Hosmer-Lemeshow\ntest > 0.05). We also found that decreasing pitch and jitter, shorter segments\nof speech, and responses phrased as questions were positively associated with\ncognitive impairment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1710.07728v2",
            "date": "2017-10-20 23:42:22+00:00",
            "title": "A Computational Framework for Multi-Modal Social Action Identification",
            "authors": [
                "Jason Anastasopoulos",
                "Jake Ryland Williams"
            ],
            "abstract": "We create a computational framework for understanding social action and\ndemonstrate how this framework can be used to build an open-source event\ndetection tool with scalable statistical machine learning algorithms and a\nsubsampled database of over 600 million geo-tagged Tweets from around the\nworld. These Tweets were collected between April 1st, 2014 and April 30th,\n2015, most notably when the Black Lives Matter movement began. We demonstrate\nhow these methods can be used diagnostically-by researchers, government\nofficials and the public-to understand peaceful and violent collective action\nat very fine-grained levels of time and geography.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1710.07868v2",
            "date": "2017-10-22 01:06:23+00:00",
            "title": "Deep Triphone Embedding Improves Phoneme Recognition",
            "authors": [
                "Mohit Yadav",
                "Vivek Tyagi"
            ],
            "abstract": "In this paper, we present a novel Deep Triphone Embedding (DTE)\nrepresentation derived from Deep Neural Network (DNN) to encapsulate the\ndiscriminative information present in the adjoining speech frames. DTEs are\ngenerated using a four hidden layer DNN with 3000 nodes in each hidden layer at\nthe first-stage. This DNN is trained with the tied-triphone classification\naccuracy as an optimization criterion. Thereafter, we retain the activation\nvectors (3000) of the last hidden layer, for each speech MFCC frame, and\nperform dimension reduction to further obtain a 300 dimensional representation,\nwhich we termed as DTE. DTEs along with MFCC features are fed into a\nsecond-stage four hidden layer DNN, which is subsequently trained for the task\nof tied-triphone classification. Both DNNs are trained using tri-phone labels\ngenerated from a tied-state triphone HMM-GMM system, by performing a\nforced-alignment between the transcriptions and MFCC feature frames. We conduct\nthe experiments on publicly available TED-LIUM speech corpus. The results show\nthat the proposed DTE method provides an improvement of absolute 2.11% in\nphoneme recognition, when compared with a competitive hybrid tied-state\ntriphone HMM-DNN system.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1710.08963v1",
            "date": "2017-10-24 19:38:20+00:00",
            "title": "Scaling Text with the Class Affinity Model",
            "authors": [
                "Patrick O. Perry",
                "Kenneth Benoit"
            ],
            "abstract": "Probabilistic methods for classifying text form a rich tradition in machine\nlearning and natural language processing. For many important problems, however,\nclass prediction is uninteresting because the class is known, and instead the\nfocus shifts to estimating latent quantities related to the text, such as\naffect or ideology. We focus on one such problem of interest, estimating the\nideological positions of 55 Irish legislators in the 1991 D\\'ail confidence\nvote. To solve the D\\'ail scaling problem and others like it, we develop a text\nmodeling framework that allows actors to take latent positions on a \"gray\"\nspectrum between \"black\" and \"white\" polar opposites. We are able to validate\nresults from this model by measuring the influences exhibited by individual\nwords, and we are able to quantify the uncertainty in the scaling estimates by\nusing a sentence-level block bootstrap. Applying our method to the D\\'ail\ndebate, we are able to scale the legislators between extreme pro-government and\npro-opposition in a way that reveals nuances in their speeches not captured by\ntheir votes or party affiliations.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1710.09026v2",
            "date": "2017-10-25 00:20:55+00:00",
            "title": "Trace norm regularization and faster inference for embedded speech recognition RNNs",
            "authors": [
                "Markus Kliegl",
                "Siddharth Goyal",
                "Kexin Zhao",
                "Kavya Srinet",
                "Mohammad Shoeybi"
            ],
            "abstract": "We propose and evaluate new techniques for compressing and speeding up dense\nmatrix multiplications as found in the fully connected and recurrent layers of\nneural networks for embedded large vocabulary continuous speech recognition\n(LVCSR). For compression, we introduce and study a trace norm regularization\ntechnique for training low rank factored versions of matrix multiplications.\nCompared to standard low rank training, we show that our method leads to good\naccuracy versus number of parameter trade-offs and can be used to speed up\ntraining of large models. For speedup, we enable faster inference on ARM\nprocessors through new open sourced kernels optimized for small batch sizes,\nresulting in 3x to 7x speed ups over the widely used gemmlowp library. Beyond\nLVCSR, we expect our techniques and kernels to be more generally applicable to\nembedded neural networks with large fully connected or recurrent layers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.09805v3",
            "date": "2017-10-26 16:54:13+00:00",
            "title": "Improving Negative Sampling for Word Representation using Self-embedded Features",
            "authors": [
                "Long Chen",
                "Fajie Yuan",
                "Joemon M. Jose",
                "Weinan Zhang"
            ],
            "abstract": "Although the word-popularity based negative sampler has shown superb\nperformance in the skip-gram model, the theoretical motivation behind\noversampling popular (non-observed) words as negative samples is still not well\nunderstood. In this paper, we start from an investigation of the gradient\nvanishing issue in the skipgram model without a proper negative sampler. By\nperforming an insightful analysis from the stochastic gradient descent (SGD)\nlearning perspective, we demonstrate that, both theoretically and intuitively,\nnegative samples with larger inner product scores are more informative than\nthose with lower scores for the SGD learner in terms of both convergence rate\nand accuracy. Understanding this, we propose an alternative sampling algorithm\nthat dynamically selects informative negative samples during each SGD update.\nMore importantly, the proposed sampler accounts for multi-dimensional\nself-embedded features during the sampling process, which essentially makes it\nmore effective than the original popularity-based (one-dimensional) sampler.\nEmpirical experiments further verify our observations, and show that our\nfine-grained samplers gain significant improvement over the existing ones\nwithout increasing computational complexity.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.09867v2",
            "date": "2017-10-26 18:48:20+00:00",
            "title": "Understanding Early Word Learning in Situated Artificial Agents",
            "authors": [
                "Felix Hill",
                "Stephen Clark",
                "Karl Moritz Hermann",
                "Phil Blunsom"
            ],
            "abstract": "Neural network-based systems can now learn to locate the referents of words\nand phrases in images, answer questions about visual scenes, and execute\nsymbolic instructions as first-person actors in partially-observable worlds. To\nachieve this so-called grounded language learning, models must overcome\nchallenges that infants face when learning their first words. While it is\nnotable that models with no meaningful prior knowledge overcome these\nobstacles, researchers currently lack a clear understanding of how they do so,\na problem that we attempt to address in this paper. For maximum control and\ngenerality, we focus on a simple neural network-based language learning agent,\ntrained via policy-gradient methods, which can interpret single-word\ninstructions in a simulated 3D world. Whilst the goal is not to explicitly\nmodel infant word learning, we take inspiration from experimental paradigms in\ndevelopmental psychology and apply some of these to the artificial agent,\nexploring the conditions under which established human biases and learning\neffects emerge. We further propose a novel method for visualising semantic\nrepresentations in the agent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10224v3",
            "date": "2017-10-27 16:16:05+00:00",
            "title": "BridgeNets: Student-Teacher Transfer Learning Based on Recursive Neural Networks and its Application to Distant Speech Recognition",
            "authors": [
                "Jaeyoung Kim",
                "Mostafa El-Khamy",
                "Jungwon Lee"
            ],
            "abstract": "Despite the remarkable progress achieved on automatic speech recognition,\nrecognizing far-field speeches mixed with various noise sources is still a\nchallenging task. In this paper, we introduce novel student-teacher transfer\nlearning, BridgeNet which can provide a solution to improve distant speech\nrecognition. There are two key features in BridgeNet. First, BridgeNet extends\ntraditional student-teacher frameworks by providing multiple hints from a\nteacher network. Hints are not limited to the soft labels from a teacher\nnetwork. Teacher's intermediate feature representations can better guide a\nstudent network to learn how to denoise or dereverberate noisy input. Second,\nthe proposed recursive architecture in the BridgeNet can iteratively improve\ndenoising and recognition performance. The experimental results of BridgeNet\nshowed significant improvements in tackling the distant speech recognition\nproblem, where it achieved up to 13.24% relative WER reductions on AMI corpus\ncompared to a baseline neural network without teacher's hints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10280v2",
            "date": "2017-10-27 18:05:22+00:00",
            "title": "One-shot and few-shot learning of word embeddings",
            "authors": [
                "Andrew K. Lampinen",
                "James L. McClelland"
            ],
            "abstract": "Standard deep learning systems require thousands or millions of examples to\nlearn a concept, and cannot integrate new concepts easily. By contrast, humans\nhave an incredible ability to do one-shot or few-shot learning. For instance,\nfrom just hearing a word used in a sentence, humans can infer a great deal\nabout it, by leveraging what the syntax and semantics of the surrounding words\ntells us. Here, we draw inspiration from this to highlight a simple technique\nby which deep recurrent networks can similarly exploit their prior knowledge to\nlearn a useful representation for a new word from little data. This could make\nnatural language processing systems much more flexible, by allowing them to\nlearn continually from the new words they encounter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.10380v3",
            "date": "2017-10-28 03:18:12+00:00",
            "title": "Speeding up Context-based Sentence Representation Learning with Non-autoregressive Convolutional Decoding",
            "authors": [
                "Shuai Tang",
                "Hailin Jin",
                "Chen Fang",
                "Zhaowen Wang",
                "Virginia R. de Sa"
            ],
            "abstract": "Context plays an important role in human language understanding, thus it may\nalso be useful for machines learning vector representations of language. In\nthis paper, we explore an asymmetric encoder-decoder structure for unsupervised\ncontext-based sentence representation learning. We carefully designed\nexperiments to show that neither an autoregressive decoder nor an RNN decoder\nis required. After that, we designed a model which still keeps an RNN as the\nencoder, while using a non-autoregressive convolutional decoder. We further\ncombine a suite of effective designs to significantly improve model efficiency\nwhile also achieving better performance. Our model is trained on two different\nlarge unlabelled corpora, and in both cases the transferability is evaluated on\na set of downstream NLP tasks. We empirically show that our model is simple and\nfast while producing rich sentence representations that excel in downstream\ntasks.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1710.10393v1",
            "date": "2017-10-28 05:42:19+00:00",
            "title": "Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks",
            "authors": [
                "Xu Sun",
                "Bingzhen Wei",
                "Xuancheng Ren",
                "Shuming Ma"
            ],
            "abstract": "We propose a method, called Label Embedding Network, which can learn label\nrepresentation (label embedding) during the training process of deep networks.\nWith the proposed method, the label embedding is adaptively and automatically\nlearned through back propagation. The original one-hot represented loss\nfunction is converted into a new loss function with soft distributions, such\nthat the originally unrelated labels have continuous interactions with each\nother during the training process. As a result, the trained model can achieve\nsubstantially higher accuracy and with faster convergence speed. Experimental\nresults based on competitive tasks demonstrate the effectiveness of the\nproposed method, and the learned label embedding is reasonable and\ninterpretable. The proposed method achieves comparable or even better results\nthan the state-of-the-art systems. The source code is available at\n\\url{https://github.com/lancopku/LabelEmb}.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.10467v5",
            "date": "2017-10-28 13:51:51+00:00",
            "title": "Generalized End-to-End Loss for Speaker Verification",
            "authors": [
                "Li Wan",
                "Quan Wang",
                "Alan Papir",
                "Ignacio Lopez Moreno"
            ],
            "abstract": "In this paper, we propose a new loss function called generalized end-to-end\n(GE2E) loss, which makes the training of speaker verification models more\nefficient than our previous tuple-based end-to-end (TE2E) loss function. Unlike\nTE2E, the GE2E loss function updates the network in a way that emphasizes\nexamples that are difficult to verify at each step of the training process.\nAdditionally, the GE2E loss does not require an initial stage of example\nselection. With these properties, our model with the new loss function\ndecreases speaker verification EER by more than 10%, while reducing the\ntraining time by 60% at the same time. We also introduce the MultiReader\ntechnique, which allows us to do domain adaptation - training a more accurate\nmodel that supports multiple keywords (i.e. \"OK Google\" and \"Hey Google\") as\nwell as multiple dialects.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1710.10774v2",
            "date": "2017-10-30 05:09:36+00:00",
            "title": "Sequence-to-Sequence ASR Optimization via Reinforcement Learning",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Despite the success of sequence-to-sequence approaches in automatic speech\nrecognition (ASR) systems, the models still suffer from several problems,\nmainly due to the mismatch between the training and inference conditions. In\nthe sequence-to-sequence architecture, the model is trained to predict the\ngrapheme of the current time-step given the input of speech signal and the\nground-truth grapheme history of the previous time-steps. However, it remains\nunclear how well the model approximates real-world speech during inference.\nThus, generating the whole transcription from scratch based on previous\npredictions is complicated and errors can propagate over time. Furthermore, the\nmodel is optimized to maximize the likelihood of training data instead of error\nrate evaluation metrics that actually quantify recognition quality. This paper\npresents an alternative strategy for training sequence-to-sequence ASR models\nby adopting the idea of reinforcement learning (RL). Unlike the standard\ntraining scheme with maximum likelihood estimation, our proposed approach\nutilizes the policy gradient algorithm. We can (1) sample the whole\ntranscription based on the model's prediction in the training process and (2)\ndirectly optimize the model with negative Levenshtein distance as the reward.\nExperimental results demonstrate that we significantly improved the performance\ncompared to a model trained only with maximum likelihood estimation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11041v2",
            "date": "2017-10-30 16:17:34+00:00",
            "title": "Unsupervised Neural Machine Translation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre",
                "Kyunghyun Cho"
            ],
            "abstract": "In spite of the recent success of neural machine translation (NMT) in\nstandard benchmarks, the lack of large parallel corpora poses a major practical\nproblem for many language pairs. There have been several proposals to alleviate\nthis issue with, for instance, triangulation and semi-supervised learning\ntechniques, but they still require a strong cross-lingual signal. In this work,\nwe completely remove the need of parallel data and propose a novel method to\ntrain an NMT system in a completely unsupervised manner, relying on nothing but\nmonolingual corpora. Our model builds upon the recent work on unsupervised\nembedding mappings, and consists of a slightly modified attentional\nencoder-decoder model that can be trained on monolingual corpora alone using a\ncombination of denoising and backtranslation. Despite the simplicity of the\napproach, our system obtains 15.56 and 10.21 BLEU points in WMT 2014\nFrench-to-English and German-to-English translation. The model can also profit\nfrom small parallel corpora, and attains 21.81 and 15.24 points when combined\nwith 100,000 parallel sentences, respectively. Our implementation is released\nas an open source project.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11277v2",
            "date": "2017-10-31 00:25:03+00:00",
            "title": "Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning",
            "authors": [
                "Baolin Peng",
                "Xiujun Li",
                "Jianfeng Gao",
                "Jingjing Liu",
                "Yun-Nung Chen",
                "Kam-Fai Wong"
            ],
            "abstract": "This paper presents a new method --- adversarial advantage actor-critic\n(Adversarial A2C), which significantly improves the efficiency of dialogue\npolicy learning in task-completion dialogue systems. Inspired by generative\nadversarial networks (GAN), we train a discriminator to differentiate\nresponses/actions generated by dialogue agents from responses/actions by\nexperts. Then, we incorporate the discriminator as another critic into the\nadvantage actor-critic (A2C) framework, to encourage the dialogue agent to\nexplore state-action within the regions where the agent takes actions similar\nto those of the experts. Experimental results in a movie-ticket booking domain\nshow that the proposed Adversarial A2C can accelerate policy exploration\nefficiently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.11342v2",
            "date": "2017-10-31 06:22:26+00:00",
            "title": "Generating Natural Adversarial Examples",
            "authors": [
                "Zhengli Zhao",
                "Dheeru Dua",
                "Sameer Singh"
            ],
            "abstract": "Due to their complex nature, it is hard to characterize the ways in which\nmachine learning models can misbehave or be exploited when deployed. Recent\nwork on adversarial examples, i.e. inputs with minor perturbations that result\nin substantially different model predictions, is helpful in evaluating the\nrobustness of these models by exposing the adversarial scenarios where they\nfail. However, these malicious perturbations are often unnatural, not\nsemantically meaningful, and not applicable to complicated domains such as\nlanguage. In this paper, we propose a framework to generate natural and legible\nadversarial examples that lie on the data manifold, by searching in semantic\nspace of dense and continuous data representation, utilizing the recent\nadvances in generative adversarial networks. We present generated adversaries\nto demonstrate the potential of the proposed approach for black-box classifiers\nfor a wide range of applications such as image classification, textual\nentailment, and machine translation. We include experiments to show that the\ngenerated adversaries are natural, legible to humans, and useful in evaluating\nand analyzing black-box classifiers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1710.11601v1",
            "date": "2017-10-31 17:27:44+00:00",
            "title": "Whodunnit? Crime Drama as a Case for Natural Language Understanding",
            "authors": [
                "Lea Frermann",
                "Shay B. Cohen",
                "Mirella Lapata"
            ],
            "abstract": "In this paper we argue that crime drama exemplified in television programs\nsuch as CSI:Crime Scene Investigation is an ideal testbed for approximating\nreal-world natural language understanding and the complex inferences associated\nwith it. We propose to treat crime drama as a new inference task, capitalizing\non the fact that each episode poses the same basic question (i.e., who\ncommitted the crime) and naturally provides the answer when the perpetrator is\nrevealed. We develop a new dataset based on CSI episodes, formalize perpetrator\nidentification as a sequence labeling problem, and develop an LSTM-based model\nwhich learns from multi-modal data. Experimental results show that an\nincremental inference strategy is key to making accurate guesses as well as\nlearning from representations fusing textual, visual, and acoustic input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.00289v2",
            "date": "2017-01-01 20:54:52+00:00",
            "title": "Integrating sentiment and social structure to determine preference alignments: The Irish Marriage Referendum",
            "authors": [
                "David J. P. O'Sullivan",
                "Guillermo Gardu\u00f1o-Hern\u00e1ndez",
                "James P. Gleeson",
                "Mariano Beguerisse-D\u00edaz"
            ],
            "abstract": "We examine the relationship between social structure and sentiment through\nthe analysis of a large collection of tweets about the Irish Marriage\nReferendum of 2015. We obtain the sentiment of every tweet with the hashtags\n#marref and #marriageref that was posted in the days leading to the referendum,\nand construct networks to aggregate sentiment and use it to study the\ninteractions among users. Our results show that the sentiment of mention tweets\nposted by users is correlated with the sentiment of received mentions, and\nthere are significantly more connections between users with similar sentiment\nscores than among users with opposite scores in the mention and follower\nnetworks. We combine the community structure of the two networks with the\nactivity level of the users and sentiment scores to find groups of users who\nsupport voting `yes' or `no' in the referendum. There were numerous\nconversations between users on opposing sides of the debate in the absence of\nfollower connections, which suggests that there were efforts by some users to\nestablish dialogue and debate across ideological divisions. Our analysis shows\nthat social structure can be integrated successfully with sentiment to analyse\nand understand the disposition of social media users. These results have\npotential applications in the integration of data and meta-data to study\nopinion dynamics, public opinion modelling, and polling.",
            "journal ref": "R. Soc. open sci., 4, 170154 (2017)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1701.00660v1",
            "date": "2017-01-03 11:16:07+00:00",
            "title": "Ambiguity and Incomplete Information in Categorical Models of Language",
            "authors": [
                "Dan Marsden"
            ],
            "abstract": "We investigate notions of ambiguity and partial information in categorical\ndistributional models of natural language. Probabilistic ambiguity has\npreviously been studied using Selinger's CPM construction. This construction\nworks well for models built upon vector spaces, as has been shown in quantum\ncomputational applications. Unfortunately, it doesn't seem to provide a\nsatisfactory method for introducing mixing in other compact closed categories\nsuch as the category of sets and binary relations. We therefore lack a uniform\nstrategy for extending a category to model imprecise linguistic information.\n  In this work we adopt a different approach. We analyze different forms of\nambiguous and incomplete information, both with and without quantitative\nprobabilistic data. Each scheme then corresponds to a suitable enrichment of\nthe category in which we model language. We view different monads as\nencapsulating the informational behaviour of interest, by analogy with their\nuse in modelling side effects in computation. Previous results of Jacobs then\nallow us to systematically construct suitable bases for enrichment.\n  We show that we can freely enrich arbitrary dagger compact closed categories\nin order to capture all the phenomena of interest, whilst retaining the\nimportant dagger compact closed structure. This allows us to construct a model\nwith real convex combination of binary relations that makes non-trivial use of\nthe scalars. Finally we relate our various different enrichments, showing that\nfinite subconvex algebra enrichment covers all the effects under consideration.",
            "journal ref": "EPTCS 236, 2017, pp. 95-107",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1701.00874v4",
            "date": "2017-01-04 00:10:17+00:00",
            "title": "Neural Probabilistic Model for Non-projective MST Parsing",
            "authors": [
                "Xuezhe Ma",
                "Eduard Hovy"
            ],
            "abstract": "In this paper, we propose a probabilistic parsing model, which defines a\nproper conditional probability distribution over non-projective dependency\ntrees for a given sentence, using neural representations as inputs. The neural\nnetwork architecture is based on bi-directional LSTM-CNNs which benefits from\nboth word- and character-level representations automatically, by using\ncombination of bidirectional LSTM and CNN. On top of the neural network, we\nintroduce a probabilistic structured layer, defining a conditional log-linear\nmodel over non-projective trees. We evaluate our model on 17 different\ndatasets, across 14 different languages. By exploiting Kirchhoff's Matrix-Tree\nTheorem (Tutte, 1984), the partition functions and marginals can be computed\nefficiently, leading to a straight-forward end-to-end model training procedure\nvia back-propagation. Our parser achieves state-of-the-art parsing performance\non nine datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02163v1",
            "date": "2017-01-09 13:02:35+00:00",
            "title": "Just an Update on PMING Distance for Web-based Semantic Similarity in Artificial Intelligence and Data Mining",
            "authors": [
                "Valentina Franzoni"
            ],
            "abstract": "One of the main problems that emerges in the classic approach to semantics is\nthe difficulty in acquisition and maintenance of ontologies and semantic\nannotations. On the other hand, the Internet explosion and the massive\ndiffusion of mobile smart devices lead to the creation of a worldwide system,\nwhich information is daily checked and fueled by the contribution of millions\nof users who interacts in a collaborative way. Search engines, continually\nexploring the Web, are a natural source of information on which to base a\nmodern approach to semantic annotation. A promising idea is that it is possible\nto generalize the semantic similarity, under the assumption that semantically\nsimilar terms behave similarly, and define collaborative proximity measures\nbased on the indexing information returned by search engines. The PMING\nDistance is a proximity measure used in data mining and information retrieval,\nwhich collaborative information express the degree of relationship between two\nterms, using only the number of documents returned as result for a query on a\nsearch engine. In this work, the PMINIG Distance is updated, providing a novel\nformal algebraic definition, which corrects previous works. The novel point of\nview underlines the features of the PMING to be a locally normalized linear\ncombination of the Pointwise Mutual Information and Normalized Google Distance.\nThe analyzed measure dynamically reflects the collaborative change made on the\nweb resources.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1701.02477v1",
            "date": "2017-01-10 08:47:56+00:00",
            "title": "Multi-task Learning Of Deep Neural Networks For Audio Visual Automatic Speech Recognition",
            "authors": [
                "Abhinav Thanda",
                "Shankar M Venkatesan"
            ],
            "abstract": "Multi-task learning (MTL) involves the simultaneous training of two or more\nrelated tasks over shared representations. In this work, we apply MTL to\naudio-visual automatic speech recognition(AV-ASR). Our primary task is to learn\na mapping between audio-visual fused features and frame labels obtained from\nacoustic GMM/HMM model. This is combined with an auxiliary task which maps\nvisual features to frame labels obtained from a separate visual GMM/HMM model.\nThe MTL model is tested at various levels of babble noise and the results are\ncompared with a base-line hybrid DNN-HMM AV-ASR model. Our results indicate\nthat MTL is especially useful at higher level of noise. Compared to base-line,\nupto 7\\% relative improvement in WER is reported at -3 SNR dB",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02720v1",
            "date": "2017-01-10 18:30:11+00:00",
            "title": "Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks",
            "authors": [
                "Ying Zhang",
                "Mohammad Pezeshki",
                "Philemon Brakel",
                "Saizheng Zhang",
                "Cesar Laurent Yoshua Bengio",
                "Aaron Courville"
            ],
            "abstract": "Convolutional Neural Networks (CNNs) are effective models for reducing\nspectral variations and modeling spectral correlations in acoustic features for\nautomatic speech recognition (ASR). Hybrid speech recognition systems\nincorporating CNNs with Hidden Markov Models/Gaussian Mixture Models\n(HMMs/GMMs) have achieved the state-of-the-art in various benchmarks.\nMeanwhile, Connectionist Temporal Classification (CTC) with Recurrent Neural\nNetworks (RNNs), which is proposed for labeling unsegmented sequences, makes it\nfeasible to train an end-to-end speech recognition system instead of hybrid\nsettings. However, RNNs are computationally expensive and sometimes difficult\nto train. In this paper, inspired by the advantages of both CNNs and the CTC\napproach, we propose an end-to-end speech framework for sequence labeling, by\ncombining hierarchical CNNs with CTC directly without recurrent connections. By\nevaluating the approach on the TIMIT phoneme recognition task, we show that the\nproposed model is not only computationally efficient, but also competitive with\nthe existing baseline systems. Moreover, we argue that CNNs have the capability\nto model temporal correlations with appropriate context information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.02810v2",
            "date": "2017-01-10 23:32:43+00:00",
            "title": "OpenNMT: Open-Source Toolkit for Neural Machine Translation",
            "authors": [
                "Guillaume Klein",
                "Yoon Kim",
                "Yuntian Deng",
                "Jean Senellart",
                "Alexander M. Rush"
            ],
            "abstract": "We describe an open-source toolkit for neural machine translation (NMT). The\ntoolkit prioritizes efficiency, modularity, and extensibility with the goal of\nsupporting NMT research into model architectures, feature representations, and\nsource modalities, while maintaining competitive performance and reasonable\ntraining requirements. The toolkit consists of modeling and translation\nsupport, as well as detailed pedagogical documentation about the underlying\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03051v1",
            "date": "2017-01-11 16:39:04+00:00",
            "title": "Efficient Twitter Sentiment Classification using Subjective Distant Supervision",
            "authors": [
                "Tapan Sahni",
                "Chinmay Chandak",
                "Naveen Reddy Chedeti",
                "Manish Singh"
            ],
            "abstract": "As microblogging services like Twitter are becoming more and more influential\nin today's globalised world, its facets like sentiment analysis are being\nextensively studied. We are no longer constrained by our own opinion. Others\nopinions and sentiments play a huge role in shaping our perspective. In this\npaper, we build on previous works on Twitter sentiment analysis using Distant\nSupervision. The existing approach requires huge computation resource for\nanalysing large number of tweets. In this paper, we propose techniques to speed\nup the computation process for sentiment analysis. We use tweet subjectivity to\nselect the right training samples. We also introduce the concept of EFWS\n(Effective Word Score) of a tweet that is derived from polarity scores of\nfrequently used words, which is an additional heuristic that can be used to\nspeed up the sentiment classification with standard machine learning\nalgorithms. We performed our experiments using 1.6 million tweets. Experimental\nevaluations show that our proposed technique is more efficient and has higher\naccuracy compared to previously proposed methods. We achieve overall accuracies\nof around 80% (EFWS heuristic gives an accuracy around 85%) on a training\ndataset of 100K tweets, which is half the size of the dataset used for the\nbaseline model. The accuracy of our proposed model is 2-3% higher than the\nbaseline model, and the model effectively trains at twice the speed of the\nbaseline model.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1701.03079v2",
            "date": "2017-01-11 17:43:57+00:00",
            "title": "RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems",
            "authors": [
                "Chongyang Tao",
                "Lili Mou",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Open-domain human-computer conversation has been attracting increasing\nattention over the past few years. However, there does not exist a standard\nautomatic evaluation metric for open-domain dialog systems; researchers usually\nresort to human annotation for model evaluation, which is time- and\nlabor-intensive. In this paper, we propose RUBER, a Referenced metric and\nUnreferenced metric Blended Evaluation Routine, which evaluates a reply by\ntaking into consideration both a groundtruth reply and a query (previous\nuser-issued utterance). Our metric is learnable, but its training does not\nrequire labels of human satisfaction. Hence, RUBER is flexible and extensible\nto different datasets and languages. Experiments on both retrieval and\ngenerative dialog systems show that RUBER has a high correlation with human\nannotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03126v2",
            "date": "2017-01-11 19:16:42+00:00",
            "title": "Attention-Based Multimodal Fusion for Video Description",
            "authors": [
                "Chiori Hori",
                "Takaaki Hori",
                "Teng-Yok Lee",
                "Kazuhiro Sumi",
                "John R. Hershey",
                "Tim K. Marks"
            ],
            "abstract": "Currently successful methods for video description are based on\nencoder-decoder sentence generation using recur-rent neural networks (RNNs).\nRecent work has shown the advantage of integrating temporal and/or spatial\nattention mechanisms into these models, in which the decoder net-work predicts\neach word in the description by selectively giving more weight to encoded\nfeatures from specific time frames (temporal attention) or to features from\nspecific spatial regions (spatial attention). In this paper, we propose to\nexpand the attention model to selectively attend not just to specific times or\nspatial regions, but to specific modalities of input such as image features,\nmotion features, and audio features. Our new modality-dependent attention\nmechanism, which we call multimodal attention, provides a natural way to fuse\nmultimodal information for video description. We evaluate our method on the\nYoutube2Text dataset, achieving results that are competitive with current state\nof the art. More importantly, we demonstrate that our model incorporating\nmultimodal attention as well as temporal attention significantly outperforms\nthe model that uses temporal attention alone.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1701.03227v3",
            "date": "2017-01-12 04:26:00+00:00",
            "title": "Prior matters: simple and general methods for evaluating and improving topic quality in topic modeling",
            "authors": [
                "Angela Fan",
                "Finale Doshi-Velez",
                "Luke Miratrix"
            ],
            "abstract": "Latent Dirichlet Allocation (LDA) models trained without stopword removal\noften produce topics with high posterior probabilities on uninformative words,\nobscuring the underlying corpus content. Even when canonical stopwords are\nmanually removed, uninformative words common in that corpus will still dominate\nthe most probable words in a topic. In this work, we first show how the\nstandard topic quality measures of coherence and pointwise mutual information\nact counter-intuitively in the presence of common but irrelevant words, making\nit difficult to even quantitatively identify situations in which topics may be\ndominated by stopwords. We propose an additional topic quality metric that\ntargets the stopword problem, and show that it, unlike the standard measures,\ncorrectly correlates with human judgements of quality. We also propose a\nsimple-to-implement strategy for generating topics that are evaluated to be of\nmuch higher quality by both human assessment and our new metric. This approach,\na collection of informative priors easily introduced into most LDA-style\ninference methods, automatically promotes terms with domain relevance and\ndemotes domain-specific stop words. We demonstrate this approach's\neffectiveness in three very different domains: Department of Labor accident\nreports, online health forum posts, and NIPS abstracts. Overall we find that\ncurrent practices thought to solve this problem do not do so adequately, and\nthat our proposal offers a substantial improvement for those interested in\ninterpreting their topics as objects in their own right.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.03577v1",
            "date": "2017-01-13 07:24:18+00:00",
            "title": "Kernel Approximation Methods for Speech Recognition",
            "authors": [
                "Avner May",
                "Alireza Bagheri Garakani",
                "Zhiyun Lu",
                "Dong Guo",
                "Kuan Liu",
                "Aur\u00e9lien Bellet",
                "Linxi Fan",
                "Michael Collins",
                "Daniel Hsu",
                "Brian Kingsbury",
                "Michael Picheny",
                "Fei Sha"
            ],
            "abstract": "We study large-scale kernel methods for acoustic modeling in speech\nrecognition and compare their performance to deep neural networks (DNNs). We\nperform experiments on four speech recognition datasets, including the TIMIT\nand Broadcast News benchmark tasks, and compare these two types of models on\nframe-level performance metrics (accuracy, cross-entropy), as well as on\nrecognition metrics (word/character error rate). In order to scale kernel\nmethods to these large datasets, we use the random Fourier feature method of\nRahimi and Recht (2007). We propose two novel techniques for improving the\nperformance of kernel acoustic models. First, in order to reduce the number of\nrandom features required by kernel models, we propose a simple but effective\nmethod for feature selection. The method is able to explore a large number of\nnon-linear features while maintaining a compact model more efficiently than\nexisting approaches. Second, we present a number of frame-level metrics which\ncorrelate very strongly with recognition performance when computed on the\nheldout set; we take advantage of these correlations by monitoring these\nmetrics during training in order to decide when to stop learning. This\ntechnique can noticeably improve the recognition performance of both DNN and\nkernel models, while narrowing the gap between them. Additionally, we show that\nthe linear bottleneck method of Sainath et al. (2013) improves the performance\nof our kernel models significantly, in addition to speeding up training and\nmaking the models more compact. Together, these three methods dramatically\nimprove the performance of kernel acoustic models, making their performance\ncomparable to DNNs on the tasks we explored.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1701.03980v1",
            "date": "2017-01-15 01:53:23+00:00",
            "title": "DyNet: The Dynamic Neural Network Toolkit",
            "authors": [
                "Graham Neubig",
                "Chris Dyer",
                "Yoav Goldberg",
                "Austin Matthews",
                "Waleed Ammar",
                "Antonios Anastasopoulos",
                "Miguel Ballesteros",
                "David Chiang",
                "Daniel Clothiaux",
                "Trevor Cohn",
                "Kevin Duh",
                "Manaal Faruqui",
                "Cynthia Gan",
                "Dan Garrette",
                "Yangfeng Ji",
                "Lingpeng Kong",
                "Adhiguna Kuncoro",
                "Gaurav Kumar",
                "Chaitanya Malaviya",
                "Paul Michel",
                "Yusuke Oda",
                "Matthew Richardson",
                "Naomi Saphra",
                "Swabha Swayamdipta",
                "Pengcheng Yin"
            ],
            "abstract": "We describe DyNet, a toolkit for implementing neural network models based on\ndynamic declaration of network structure. In the static declaration strategy\nthat is used in toolkits like Theano, CNTK, and TensorFlow, the user first\ndefines a computation graph (a symbolic representation of the computation), and\nthen examples are fed into an engine that executes this computation and\ncomputes its derivatives. In DyNet's dynamic declaration strategy, computation\ngraph construction is mostly transparent, being implicitly constructed by\nexecuting procedural code that computes the network outputs, and the user is\nfree to use different network structures for each input. Dynamic declaration\nthus facilitates the implementation of more complicated network architectures,\nand DyNet is specifically designed to allow users to implement their models in\na way that is idiomatic in their preferred programming language (C++ or\nPython). One challenge with dynamic declaration is that because the symbolic\ncomputation graph is defined anew for every training example, its construction\nmust have low overhead. To achieve this, DyNet has an optimized C++ backend and\nlightweight graph representation. Experiments show that DyNet's speeds are\nfaster than or comparable with static declaration toolkits, and significantly\nfaster than Chainer, another dynamic declaration toolkit. DyNet is released\nopen-source under the Apache 2.0 license and available at\nhttp://github.com/clab/dynet.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1701.04313v1",
            "date": "2017-01-13 15:05:39+00:00",
            "title": "End-to-End ASR-free Keyword Search from Speech",
            "authors": [
                "Kartik Audhkhasi",
                "Andrew Rosenberg",
                "Abhinav Sethy",
                "Bhuvana Ramabhadran",
                "Brian Kingsbury"
            ],
            "abstract": "End-to-end (E2E) systems have achieved competitive results compared to\nconventional hybrid hidden Markov model (HMM)-deep neural network based\nautomatic speech recognition (ASR) systems. Such E2E systems are attractive due\nto the lack of dependence on alignments between input acoustic and output\ngrapheme or HMM state sequence during training. This paper explores the design\nof an ASR-free end-to-end system for text query-based keyword search (KWS) from\nspeech trained with minimal supervision. Our E2E KWS system consists of three\nsub-systems. The first sub-system is a recurrent neural network (RNN)-based\nacoustic auto-encoder trained to reconstruct the audio through a\nfinite-dimensional representation. The second sub-system is a character-level\nRNN language model using embeddings learned from a convolutional neural\nnetwork. Since the acoustic and text query embeddings occupy different\nrepresentation spaces, they are input to a third feed-forward neural network\nthat predicts whether the query occurs in the acoustic utterance or not. This\nE2E ASR-free KWS system performs respectably despite lacking a conventional ASR\nsystem and trains much faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.05311v1",
            "date": "2017-01-19 06:38:33+00:00",
            "title": "Semantic Evolutionary Concept Distances for Effective Information Retrieval in Query Expansion",
            "authors": [
                "Valentina Franzoni",
                "Yuanxi Li",
                "Clement H. C. Leung",
                "Alfredo Milani"
            ],
            "abstract": "In this work several semantic approaches to concept-based query expansion and\nreranking schemes are studied and compared with different ontology-based\nexpansion methods in web document search and retrieval. In particular, we focus\non concept-based query expansion schemes, where, in order to effectively\nincrease the precision of web document retrieval and to decrease the users\nbrowsing time, the main goal is to quickly provide users with the most suitable\nquery expansion. Two key tasks for query expansion in web document retrieval\nare to find the expansion candidates, as the closest concepts in web document\ndomain, and to rank the expanded queries properly. The approach we propose aims\nat improving the expansion phase for better web document retrieval and\nprecision. The basic idea is to measure the distance between candidate concepts\nusing the PMING distance, a collaborative semantic proximity measure, i.e. a\nmeasure which can be computed by using statistical results from web search\nengine. Experiments show that the proposed technique can provide users with\nmore satisfying expansion results and improve the quality of web document\nretrieval.",
            "journal ref": "Chapter Computational Science and Its Applications, ICCSA 2013,\n  Volume 7974 of the series Lecture Notes in Computer Science, pp 657-672",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.06233v1",
            "date": "2017-01-22 23:03:11+00:00",
            "title": "What the Language You Tweet Says About Your Occupation",
            "authors": [
                "Tianran Hu",
                "Haoyuan Xiao",
                "Thuy-vy Thi Nguyen",
                "Jiebo Luo"
            ],
            "abstract": "Many aspects of people's lives are proven to be deeply connected to their\njobs. In this paper, we first investigate the distinct characteristics of major\noccupation categories based on tweets. From multiple social media platforms, we\ngather several types of user information. From users' LinkedIn webpages, we\nlearn their proficiencies. To overcome the ambiguity of self-reported\ninformation, a soft clustering approach is applied to extract occupations from\ncrowd-sourced data. Eight job categories are extracted, including Marketing,\nAdministrator, Start-up, Editor, Software Engineer, Public Relation, Office\nClerk, and Designer. Meanwhile, users' posts on Twitter provide cues for\nunderstanding their linguistic styles, interests, and personalities. Our\nresults suggest that people of different jobs have unique tendencies in certain\nlanguage styles and interests. Our results also clearly reveal distinctive\nlevels in terms of Big Five Traits for different jobs. Finally, a classifier is\nbuilt to predict job types based on the features extracted from tweets. A high\naccuracy indicates a strong discrimination power of language features for job\nprediction task.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1701.06247v1",
            "date": "2017-01-23 01:36:10+00:00",
            "title": "A Multichannel Convolutional Neural Network For Cross-language Dialog State Tracking",
            "authors": [
                "Hongjie Shi",
                "Takashi Ushio",
                "Mitsuru Endo",
                "Katsuyoshi Yamagami",
                "Noriaki Horii"
            ],
            "abstract": "The fifth Dialog State Tracking Challenge (DSTC5) introduces a new\ncross-language dialog state tracking scenario, where the participants are asked\nto build their trackers based on the English training corpus, while evaluating\nthem with the unlabeled Chinese corpus. Although the computer-generated\ntranslations for both English and Chinese corpus are provided in the dataset,\nthese translations contain errors and careless use of them can easily hurt the\nperformance of the built trackers. To address this problem, we propose a\nmultichannel Convolutional Neural Networks (CNN) architecture, in which we\ntreat English and Chinese language as different input channels of one single\nCNN model. In the evaluation of DSTC5, we found that such multichannel\narchitecture can effectively improve the robustness against translation errors.\nAdditionally, our method for DSTC5 is purely machine learning based and\nrequires no prior knowledge about the target language. We consider this a\ndesirable property for building a tracker in the cross-language context, as not\nevery developer will be familiar with both languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.06279v1",
            "date": "2017-01-23 07:21:43+00:00",
            "title": "dna2vec: Consistent vector representations of variable-length k-mers",
            "authors": [
                "Patrick Ng"
            ],
            "abstract": "One of the ubiquitous representation of long DNA sequence is dividing it into\nshorter k-mer components. Unfortunately, the straightforward vector encoding of\nk-mer as a one-hot vector is vulnerable to the curse of dimensionality. Worse\nyet, the distance between any pair of one-hot vectors is equidistant. This is\nparticularly problematic when applying the latest machine learning algorithms\nto solve problems in biological sequence analysis. In this paper, we propose a\nnovel method to train distributed representations of variable-length k-mers.\nOur method is based on the popular word embedding model word2vec, which is\ntrained on a shallow two-layer neural network. Our experiments provide evidence\nthat the summing of dna2vec vectors is akin to nucleotides concatenation. We\nalso demonstrate that there is correlation between Needleman-Wunsch similarity\nscore and cosine similarity of dna2vec vectors.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1701.06538v1",
            "date": "2017-01-23 18:10:00+00:00",
            "title": "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer",
            "authors": [
                "Noam Shazeer",
                "Azalia Mirhoseini",
                "Krzysztof Maziarz",
                "Andy Davis",
                "Quoc Le",
                "Geoffrey Hinton",
                "Jeff Dean"
            ],
            "abstract": "The capacity of a neural network to absorb information is limited by its\nnumber of parameters. Conditional computation, where parts of the network are\nactive on a per-example basis, has been proposed in theory as a way of\ndramatically increasing model capacity without a proportional increase in\ncomputation. In practice, however, there are significant algorithmic and\nperformance challenges. In this work, we address these challenges and finally\nrealize the promise of conditional computation, achieving greater than 1000x\nimprovements in model capacity with only minor losses in computational\nefficiency on modern GPU clusters. We introduce a Sparsely-Gated\nMixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward\nsub-networks. A trainable gating network determines a sparse combination of\nthese experts to use for each example. We apply the MoE to the tasks of\nlanguage modeling and machine translation, where model capacity is critical for\nabsorbing the vast quantities of knowledge available in the training corpora.\nWe present model architectures in which a MoE with up to 137 billion parameters\nis applied convolutionally between stacked LSTM layers. On large language\nmodeling and machine translation benchmarks, these models achieve significantly\nbetter results than state-of-the-art at lower computational cost.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1701.08229v1",
            "date": "2017-01-28 00:32:40+00:00",
            "title": "Feature Studies to Inform the Classification of Depressive Symptoms from Twitter Data for Population Health",
            "authors": [
                "Danielle Mowery",
                "Craig Bryan",
                "Mike Conway"
            ],
            "abstract": "The utility of Twitter data as a medium to support population-level mental\nhealth monitoring is not well understood. In an effort to better understand the\npredictive power of supervised machine learning classifiers and the influence\nof feature sets for efficiently classifying depression-related tweets on a\nlarge-scale, we conducted two feature study experiments. In the first\nexperiment, we assessed the contribution of feature groups such as lexical\ninformation (e.g., unigrams) and emotions (e.g., strongly negative) using a\nfeature ablation study. In the second experiment, we determined the percentile\nof top ranked features that produced the optimal classification performance by\napplying a three-step feature elimination approach. In the first experiment, we\nobserved that lexical features are critical for identifying depressive\nsymptoms, specifically for depressed mood (-35 points) and for disturbed sleep\n(-43 points). In the second experiment, we observed that the optimal F1-score\nperformance of top ranked features in percentiles variably ranged across\nclasses e.g., fatigue or loss of energy (5th percentile, 288 features) to\ndepressed mood (55th percentile, 3,168 features) suggesting there is no\nconsistent count of features for predicting depressive-related tweets. We\nconclude that simple lexical features and reduced feature sets can produce\ncomparable results to larger feature sets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.08251v2",
            "date": "2017-01-28 05:06:11+00:00",
            "title": "Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation",
            "authors": [
                "Nasrin Mostafazadeh",
                "Chris Brockett",
                "Bill Dolan",
                "Michel Galley",
                "Jianfeng Gao",
                "Georgios P. Spithourakis",
                "Lucy Vanderwende"
            ],
            "abstract": "The popularity of image sharing on social media and the engagement it creates\nbetween users reflects the important role that visual context plays in everyday\nconversations. We present a novel task, Image-Grounded Conversations (IGC), in\nwhich natural-sounding conversations are generated about a shared image. To\nbenchmark progress, we introduce a new multiple-reference dataset of\ncrowd-sourced, event-centric conversations on images. IGC falls on the\ncontinuum between chit-chat and goal-directed conversation models, where visual\ngrounding constrains the topic of conversation to event-driven utterances.\nExperiments with models trained on social media data show that the combination\nof visual and textual context enhances the quality of generated conversational\nturns. In human evaluation, the gap between human performance and that of both\nneural and retrieval architectures suggests that multi-modal IGC presents an\ninteresting challenge for dialogue research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08269v2",
            "date": "2017-01-28 08:32:35+00:00",
            "title": "Systems of natural-language-facilitated human-robot cooperation: A review",
            "authors": [
                "Rui Liu",
                "Xiaoli Zhang"
            ],
            "abstract": "Natural-language-facilitated human-robot cooperation (NLC), in which natural\nlanguage (NL) is used to share knowledge between a human and a robot for\nconducting intuitive human-robot cooperation (HRC), is continuously developing\nin the recent decade. Currently, NLC is used in several robotic domains such as\nmanufacturing, daily assistance and health caregiving. It is necessary to\nsummarize current NLC-based robotic systems and discuss the future developing\ntrends, providing helpful information for future NLC research. In this review,\nwe first analyzed the driving forces behind the NLC research. Regarding to a\nrobot s cognition level during the cooperation, the NLC implementations then\nwere categorized into four types {NL-based control, NL-based robot training,\nNL-based task execution, NL-based social companion} for comparison and\ndiscussion. Last based on our perspective and comprehensive paper review, the\nfuture research trends were discussed.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1701.08711v5",
            "date": "2017-01-30 17:14:25+00:00",
            "title": "Predicting Auction Price of Vehicle License Plate with Deep Recurrent Neural Network",
            "authors": [
                "Vinci Chow"
            ],
            "abstract": "In Chinese societies, superstition is of paramount importance, and vehicle\nlicense plates with desirable numbers can fetch very high prices in auctions.\nUnlike other valuable items, license plates are not allocated an estimated\nprice before auction. I propose that the task of predicting plate prices can be\nviewed as a natural language processing (NLP) task, as the value depends on the\nmeaning of each individual character on the plate and its semantics. I\nconstruct a deep recurrent neural network (RNN) to predict the prices of\nvehicle license plates in Hong Kong, based on the characters on a plate. I\ndemonstrate the importance of having a deep network and of retraining.\nEvaluated on 13 years of historical auction prices, the deep RNN's predictions\ncan explain over 80 percent of price variations, outperforming previous models\nby a significant margin. I also demonstrate how the model can be extended to\nbecome a search engine for plates and to provide estimates of the expected\nprice distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1701.08756v3",
            "date": "2017-01-30 18:59:04+00:00",
            "title": "A Review of Methodologies for Natural-Language-Facilitated Human-Robot Cooperation",
            "authors": [
                "Rui Liu",
                "Xiaoli Zhang"
            ],
            "abstract": "Natural-language-facilitated human-robot cooperation (NLC) refers to using\nnatural language (NL) to facilitate interactive information sharing and task\nexecutions with a common goal constraint between robots and humans. Recently,\nNLC research has received increasing attention. Typical NLC scenarios include\nrobotic daily assistance, robotic health caregiving, intelligent manufacturing,\nautonomous navigation, and robot social accompany. However, a thorough review,\nthat can reveal latest methodologies to use NL to facilitate human-robot\ncooperation, is missing. In this review, a comprehensive summary about\nmethodologies for NLC is presented. NLC research includes three main research\nfocuses: NL instruction understanding, NL-based execution plan generation, and\nknowledge-world mapping. In-depth analyses on theoretical methods,\napplications, and model advantages and disadvantages are made. Based on our\npaper review and perspective, potential research directions of NLC are\nsummarized.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1701.08888v2",
            "date": "2017-01-31 02:13:57+00:00",
            "title": "Integrating Reviews into Personalized Ranking for Cold Start Recommendation",
            "authors": [
                "Guang-Neng Hu",
                "Xin-Yu Dai"
            ],
            "abstract": "Item recommendation task predicts a personalized ranking over a set of items\nfor each individual user. One paradigm is the rating-based methods that\nconcentrate on explicit feedbacks and hence face the difficulties in collecting\nthem. Meanwhile, the ranking-based methods are presented with rated items and\nthen rank the rated above the unrated. This paradigm takes advantage of widely\navailable implicit feedback. It, however, usually ignores a kind of important\ninformation: item reviews. Item reviews not only justify the preferences of\nusers, but also help alleviate the cold-start problem that fails the\ncollaborative filtering. In this paper, we propose two novel and simple models\nto integrate item reviews into Bayesian personalized ranking. In each model, we\nmake use of text features extracted from item reviews using word embeddings. On\ntop of text features we uncover the review dimensions that explain the\nvariation in users' feedback and these review factors represent a prior\npreference of users. Experiments on six real-world data sets show the benefits\nof leveraging item reviews on ranking prediction. We also conduct analyses to\nunderstand the proposed models.",
            "journal ref": "PAKDD 2017",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1701.08954v2",
            "date": "2017-01-31 09:20:17+00:00",
            "title": "CommAI: Evaluating the first steps towards a useful general AI",
            "authors": [
                "Marco Baroni",
                "Armand Joulin",
                "Allan Jabri",
                "Germ\u00e0n Kruszewski",
                "Angeliki Lazaridou",
                "Klemen Simonic",
                "Tomas Mikolov"
            ],
            "abstract": "With machine learning successfully applied to new daunting problems almost\nevery day, general AI starts looking like an attainable goal. However, most\ncurrent research focuses instead on important but narrow applications, such as\nimage classification or machine translation. We believe this to be largely due\nto the lack of objective ways to measure progress towards broad machine\nintelligence. In order to fill this gap, we propose here a set of concrete\ndesiderata for general AI, together with a platform to test machines on how\nwell they satisfy such desiderata, while keeping all further complexities to a\nminimum.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1707.00117v3",
            "date": "2017-07-01 09:00:28+00:00",
            "title": "SAM: Semantic Attribute Modulation for Language Modeling and Style Variation",
            "authors": [
                "Wenbo Hu",
                "Lifeng Hua",
                "Lei Li",
                "Hang Su",
                "Tian Wang",
                "Ning Chen",
                "Bo Zhang"
            ],
            "abstract": "This paper presents a Semantic Attribute Modulation (SAM) for language\nmodeling and style variation. The semantic attribute modulation includes\nvarious document attributes, such as titles, authors, and document categories.\nWe consider two types of attributes, (title attributes and category\nattributes), and a flexible attribute selection scheme by automatically scoring\nthem via an attribute attention mechanism. The semantic attributes are embedded\ninto the hidden semantic space as the generation inputs. With the attributes\nproperly harnessed, our proposed SAM can generate interpretable texts with\nregard to the input attributes. Qualitative analysis, including word semantic\nanalysis and attention values, shows the interpretability of SAM. On several\ntypical text datasets, we empirically demonstrate the superiority of the\nSemantic Attribute Modulated language model with different combinations of\ndocument attributes. Moreover, we present a style variation for the lyric\ngeneration using SAM, which shows a strong connection between the style\nvariation and the semantic attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00130v2",
            "date": "2017-07-01 09:56:31+00:00",
            "title": "Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management",
            "authors": [
                "Pei-Hao Su",
                "Pawel Budzianowski",
                "Stefan Ultes",
                "Milica Gasic",
                "Steve Young"
            ],
            "abstract": "Deep reinforcement learning (RL) methods have significant potential for\ndialogue policy optimisation. However, they suffer from a poor performance in\nthe early stages of learning. This is especially problematic for on-line\nlearning with real users. Two approaches are introduced to tackle this problem.\nFirstly, to speed up the learning process, two sample-efficient neural networks\nalgorithms: trust region actor-critic with experience replay (TRACER) and\nepisodic natural actor-critic with experience replay (eNACER) are presented.\nFor TRACER, the trust region helps to control the learning step size and avoid\ncatastrophic model changes. For eNACER, the natural gradient identifies the\nsteepest ascent direction in policy space to speed up the convergence. Both\nmodels employ off-policy learning with experience replay to improve\nsample-efficiency. Secondly, to mitigate the cold start issue, a corpus of\ndemonstration data is utilised to pre-train the models prior to on-line\nreinforcement learning. Combining these two approaches, we demonstrate a\npractical approach to learn deep RL-based dialogue policies and demonstrate\ntheir effectiveness in a task-oriented information seeking domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00206v1",
            "date": "2017-07-01 21:10:15+00:00",
            "title": "Efficient Correlated Topic Modeling with Topic Embedding",
            "authors": [
                "Junxian He",
                "Zhiting Hu",
                "Taylor Berg-Kirkpatrick",
                "Ying Huang",
                "Eric P. Xing"
            ],
            "abstract": "Correlated topic modeling has been limited to small model and problem sizes\ndue to their high computational cost and poor scaling. In this paper, we\npropose a new model which learns compact topic embeddings and captures topic\ncorrelations through the closeness between the topic vectors. Our method\nenables efficient inference in the low-dimensional embedding space, reducing\nprevious cubic or quadratic time complexity to linear w.r.t the topic size. We\nfurther speedup variational inference with a fast sampler to exploit sparsity\nof topic occurrence. Extensive experiments show that our approach is capable of\nhandling model and data scales which are several orders of magnitude larger\nthan existing correlation results, without sacrificing modeling quality by\nproviding competitive or superior performance in document classification and\nretrieval.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1707.00683v3",
            "date": "2017-07-02 04:06:01+00:00",
            "title": "Modulating early visual processing by language",
            "authors": [
                "Harm de Vries",
                "Florian Strub",
                "J\u00e9r\u00e9mie Mary",
                "Hugo Larochelle",
                "Olivier Pietquin",
                "Aaron Courville"
            ],
            "abstract": "It is commonly assumed that language refers to high-level visual concepts\nwhile leaving low-level visual processing unaffected. This view dominates the\ncurrent literature in computational models for language-vision tasks, where\nvisual and linguistic input are mostly processed independently before being\nfused into a single representation. In this paper, we deviate from this classic\npipeline and propose to modulate the \\emph{entire visual processing} by\nlinguistic input. Specifically, we condition the batch normalization parameters\nof a pretrained residual network (ResNet) on a language embedding. This\napproach, which we call MOdulated RESnet (\\MRN), significantly improves strong\nbaselines on two visual question answering tasks. Our ablation study shows that\nmodulating from the early stages of the visual processing is beneficial.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.00836v1",
            "date": "2017-07-04 07:42:05+00:00",
            "title": "DeepStory: Video Story QA by Deep Embedded Memory Networks",
            "authors": [
                "Kyung-Min Kim",
                "Min-Oh Heo",
                "Seong-Ho Choi",
                "Byoung-Tak Zhang"
            ],
            "abstract": "Question-answering (QA) on video contents is a significant challenge for\nachieving human-level intelligence as it involves both vision and language in\nreal-world settings. Here we demonstrate the possibility of an AI agent\nperforming video story QA by learning from a large amount of cartoon videos. We\ndevelop a video-story learning model, i.e. Deep Embedded Memory Networks\n(DEMN), to reconstruct stories from a joint scene-dialogue video stream using a\nlatent embedding space of observed data. The video stories are stored in a\nlong-term memory component. For a given question, an LSTM-based attention model\nuses the long-term memory to recall the best question-story-answer triplet by\nfocusing on specific words containing key information. We trained the DEMN on a\nnovel QA dataset of children's cartoon video series, Pororo. The dataset\ncontains 16,066 scene-dialogue pairs of 20.5-hour videos, 27,328 fine-grained\nsentences for scene description, and 8,913 story-related QA pairs. Our\nexperimental results show that the DEMN outperforms other QA models. This is\nmainly due to 1) the reconstruction of video stories in a scene-dialogue\ncombined form that utilize the latent embedding and 2) attention. DEMN also\nachieved state-of-the-art results on the MovieQA benchmark.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.01184v1",
            "date": "2017-07-04 23:29:44+00:00",
            "title": "Sentiment Identification in Code-Mixed Social Media Text",
            "authors": [
                "Souvick Ghosh",
                "Satanu Ghosh",
                "Dipankar Das"
            ],
            "abstract": "Sentiment analysis is the Natural Language Processing (NLP) task dealing with\nthe detection and classification of sentiments in texts. While some tasks deal\nwith identifying the presence of sentiment in the text (Subjectivity analysis),\nother tasks aim at determining the polarity of the text categorizing them as\npositive, negative and neutral. Whenever there is a presence of sentiment in\nthe text, it has a source (people, group of people or any entity) and the\nsentiment is directed towards some entity, object, event or person. Sentiment\nanalysis tasks aim to determine the subject, the target and the polarity or\nvalence of the sentiment. In our work, we try to automatically extract\nsentiment (positive or negative) from Facebook posts using a machine learning\napproach.While some works have been done in code-mixed social media data and in\nsentiment analysis separately, our work is the first attempt (as of now) which\naims at performing sentiment analysis of code-mixed social media text. We have\nused extensive pre-processing to remove noise from raw text. Multilayer\nPerceptron model has been used to determine the polarity of the sentiment. We\nhave also developed the corpus for this task by manually labeling Facebook\nposts with their associated sentiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01425v1",
            "date": "2017-07-05 15:03:30+00:00",
            "title": "Determining sentiment in citation text and analyzing its impact on the proposed ranking index",
            "authors": [
                "Souvick Ghosh",
                "Dipankar Das",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Whenever human beings interact with each other, they exchange or express\nopinions, emotions, and sentiments. These opinions can be expressed in text,\nspeech or images. Analysis of these sentiments is one of the popular research\nareas of present day researchers. Sentiment analysis, also known as opinion\nmining tries to identify or classify these sentiments or opinions into two\nbroad categories - positive and negative. In recent years, the scientific\ncommunity has taken a lot of interest in analyzing sentiment in textual data\navailable in various social media platforms. Much work has been done on social\nmedia conversations, blog posts, newspaper articles and various narrative\ntexts. However, when it comes to identifying emotions from scientific papers,\nresearchers have faced some difficulties due to the implicit and hidden nature\nof opinion. By default, citation instances are considered inherently positive\nin emotion. Popular ranking and indexing paradigms often neglect the opinion\npresent while citing. In this paper, we have tried to achieve three objectives.\nFirst, we try to identify the major sentiment in the citation text and assign a\nscore to the instance. We have used a statistical classifier for this purpose.\nSecondly, we have proposed a new index (we shall refer to it hereafter as\nM-index) which takes into account both the quantitative and qualitative factors\nwhile scoring a paper. Thirdly, we developed a ranking of research papers based\non the M-index. We also try to explain how the M-index impacts the ranking of\nscientific papers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.01477v1",
            "date": "2017-07-05 17:19:45+00:00",
            "title": "Like trainer, like bot? Inheritance of bias in algorithmic content moderation",
            "authors": [
                "Reuben Binns",
                "Michael Veale",
                "Max Van Kleek",
                "Nigel Shadbolt"
            ],
            "abstract": "The internet has become a central medium through which `networked publics'\nexpress their opinions and engage in debate. Offensive comments and personal\nattacks can inhibit participation in these spaces. Automated content moderation\naims to overcome this problem using machine learning classifiers trained on\nlarge corpora of texts manually annotated for offence. While such systems could\nhelp encourage more civil debate, they must navigate inherently normatively\ncontestable boundaries, and are subject to the idiosyncratic norms of the human\nraters who provide the training data. An important objective for platforms\nimplementing such measures might be to ensure that they are not unduly biased\ntowards or against particular norms of offence. This paper provides some\nexploratory methods by which the normative biases of algorithmic content\nmoderation systems can be measured, by way of a case study using an existing\ndataset of comments labelled for offence. We train classifiers on comments\nlabelled by different demographic subsets (men and women) to understand how\ndifferences in conceptions of offence between these groups might affect the\nperformance of the resulting models on various test sets. We conclude by\ndiscussing some of the ethical choices facing the implementers of algorithmic\nmoderation systems, given various desired levels of diversity of viewpoints\namongst discussion participants.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1707.01555v1",
            "date": "2017-07-05 19:37:23+00:00",
            "title": "A Deep Network with Visual Text Composition Behavior",
            "authors": [
                "Hongyu Guo"
            ],
            "abstract": "While natural languages are compositional, how state-of-the-art neural models\nachieve compositionality is still unclear. We propose a deep network, which not\nonly achieves competitive accuracy for text classification, but also exhibits\ncompositional behavior. That is, while creating hierarchical representations of\na piece of text, such as a sentence, the lower layers of the network distribute\ntheir layer-specific attention weights to individual words. In contrast, the\nhigher layers compose meaningful phrases and clauses, whose lengths increase as\nthe networks get deeper until fully composing the sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01736v2",
            "date": "2017-07-06 11:53:41+00:00",
            "title": "Cross-linguistic differences and similarities in image descriptions",
            "authors": [
                "Emiel van Miltenburg",
                "Desmond Elliott",
                "Piek Vossen"
            ],
            "abstract": "Automatic image description systems are commonly trained and evaluated on\nlarge image description datasets. Recently, researchers have started to collect\nsuch datasets for languages other than English. An unexplored question is how\ndifferent these datasets are from English and, if there are any differences,\nwhat causes them to differ. This paper provides a cross-linguistic comparison\nof Dutch, English, and German image descriptions. We find that these\ndescriptions are similar in many respects, but the familiarity of crowd workers\nwith the subjects of the images has a noticeable influence on description\nspecificity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.01890v2",
            "date": "2017-07-06 17:44:15+00:00",
            "title": "An Interactive Tool for Natural Language Processing on Clinical Text",
            "authors": [
                "Gaurav Trivedi",
                "Phuong Pham",
                "Wendy Chapman",
                "Rebecca Hwa",
                "Janyce Wiebe",
                "Harry Hochheiser"
            ],
            "abstract": "Natural Language Processing (NLP) systems often make use of machine learning\ntechniques that are unfamiliar to end-users who are interested in analyzing\nclinical records. Although NLP has been widely used in extracting information\nfrom clinical text, current systems generally do not support model revision\nbased on feedback from domain experts.\n  We present a prototype tool that allows end users to visualize and review the\noutputs of an NLP system that extracts binary variables from clinical text. Our\ntool combines multiple visualizations to help the users understand these\nresults and make any necessary corrections, thus forming a feedback loop and\nhelping improve the accuracy of the NLP models. We have tested our prototype in\na formative think-aloud user study with clinicians and researchers involved in\ncolonoscopy research. Results from semi-structured interviews and a System\nUsability Scale (SUS) analysis show that the users are able to quickly start\nrefining NLP models, despite having very little or no experience with machine\nlearning. Observations from these sessions suggest revisions to the interface\nto better support review workflow and interpretation of results.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1707.02459v1",
            "date": "2017-07-08 16:17:04+00:00",
            "title": "Improving Multilingual Named Entity Recognition with Wikipedia Entity Type Mapping",
            "authors": [
                "Jian Ni",
                "Radu Florian"
            ],
            "abstract": "The state-of-the-art named entity recognition (NER) systems are statistical\nmachine learning models that have strong generalization capability (i.e., can\nrecognize unseen entities that do not appear in training data) based on lexical\nand contextual information. However, such a model could still make mistakes if\nits features favor a wrong entity type. In this paper, we utilize Wikipedia as\nan open knowledge base to improve multilingual NER systems. Central to our\napproach is the construction of high-accuracy, high-coverage multilingual\nWikipedia entity type mappings. These mappings are built from weakly annotated\ndata and can be extended to new languages with no human annotation or\nlanguage-dependent knowledge involved. Based on these mappings, we develop\nseveral approaches to improve an NER system. We evaluate the performance of the\napproaches via experiments on NER systems trained for 6 languages. Experimental\nresults show that the proposed approaches are effective in improving the\naccuracy of such systems on unseen entities, especially when a system is\napplied to a new domain or it is trained with little training data (up to 18.3\nF1 score improvement).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02657v1",
            "date": "2017-07-09 23:13:58+00:00",
            "title": "PELESent: Cross-domain polarity classification using distant supervision",
            "authors": [
                "Edilson A. Corr\u00eaa Jr",
                "Vanessa Q. Marinho",
                "Leandro B. dos Santos",
                "Thales F. C. Bertaglia",
                "Marcos V. Treviso",
                "Henrico B. Brum"
            ],
            "abstract": "The enormous amount of texts published daily by Internet users has fostered\nthe development of methods to analyze this content in several natural language\nprocessing areas, such as sentiment analysis. The main goal of this task is to\nclassify the polarity of a message. Even though many approaches have been\nproposed for sentiment analysis, some of the most successful ones rely on the\navailability of large annotated corpus, which is an expensive and\ntime-consuming process. In recent years, distant supervision has been used to\nobtain larger datasets. So, inspired by these techniques, in this paper we\nextend such approaches to incorporate popular graphic symbols used in\nelectronic messages, the emojis, in order to create a large sentiment corpus\nfor Portuguese. Trained on almost one million tweets, several models were\ntested in both same domain and cross-domain corpora. Our methods obtained very\ncompetitive results in five annotated corpora from mixed domains (Twitter and\nproduct reviews), which proves the domain-independent property of such\napproach. In addition, our results suggest that the combination of emoticons\nand emojis is able to properly capture the sentiment of a message.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02774v1",
            "date": "2017-07-10 09:40:12+00:00",
            "title": "Understanding State Preferences With Text As Data: Introducing the UN General Debate Corpus",
            "authors": [
                "Alexander Baturo",
                "Niheer Dasandi",
                "Slava J. Mikhaylov"
            ],
            "abstract": "Every year at the United Nations, member states deliver statements during the\nGeneral Debate discussing major issues in world politics. These speeches\nprovide invaluable information on governments' perspectives and preferences on\na wide range of issues, but have largely been overlooked in the study of\ninternational politics. This paper introduces a new dataset consisting of over\n7,701 English-language country statements from 1970-2016. We demonstrate how\nthe UN General Debate Corpus (UNGDC) can be used to derive country positions on\ndifferent policy dimensions using text analytic methods. The paper provides\napplications of these estimates, demonstrating the contribution the UNGDC can\nmake to the study of international politics.",
            "journal ref": "Research & Politics, Volume 4, Issue 2, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.02812v1",
            "date": "2017-07-10 11:58:08+00:00",
            "title": "Towards Crafting Text Adversarial Samples",
            "authors": [
                "Suranjana Samanta",
                "Sameep Mehta"
            ],
            "abstract": "Adversarial samples are strategically modified samples, which are crafted\nwith the purpose of fooling a classifier at hand. An attacker introduces\nspecially crafted adversarial samples to a deployed classifier, which are being\nmis-classified by the classifier. However, the samples are perceived to be\ndrawn from entirely different classes and thus it becomes hard to detect the\nadversarial samples. Most of the prior works have been focused on synthesizing\nadversarial samples in the image domain. In this paper, we propose a new method\nof crafting adversarial text samples by modification of the original samples.\nModifications of the original text samples are done by deleting or replacing\nthe important or salient words in the text or by introducing new words in the\ntext sample. Our algorithm works best for the datasets which have\nsub-categories within each of the classes of examples. While crafting\nadversarial samples, one of the key constraint is to generate meaningful\nsentences which can at pass off as legitimate from language (English)\nviewpoint. Experimental results on IMDB movie review dataset for sentiment\nanalysis and Twitter dataset for gender detection show the efficiency of our\nproposed method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1707.02919v2",
            "date": "2017-07-10 16:02:44+00:00",
            "title": "A Brief Survey of Text Mining: Classification, Clustering and Extraction Techniques",
            "authors": [
                "Mehdi Allahyari",
                "Seyedamin Pouriyeh",
                "Mehdi Assefi",
                "Saied Safaei",
                "Elizabeth D. Trippe",
                "Juan B. Gutierrez",
                "Krys Kochut"
            ],
            "abstract": "The amount of text that is generated every day is increasing dramatically.\nThis tremendous volume of mostly unstructured text cannot be simply processed\nand perceived by computers. Therefore, efficient and effective techniques and\nalgorithms are required to discover useful patterns. Text mining is the task of\nextracting meaningful information from text, which has gained significant\nattentions in recent years. In this paper, we describe several of the most\nfundamental text mining tasks and techniques including text pre-processing,\nclassification and clustering. Additionally, we briefly explain text mining in\nbiomedical and health care domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03017v5",
            "date": "2017-07-10 18:49:28+00:00",
            "title": "Learning Visual Reasoning Without Strong Priors",
            "authors": [
                "Ethan Perez",
                "Harm de Vries",
                "Florian Strub",
                "Vincent Dumoulin",
                "Aaron Courville"
            ],
            "abstract": "Achieving artificial visual reasoning - the ability to answer image-related\nquestions which require a multi-step, high-level process - is an important step\ntowards artificial general intelligence. This multi-modal task requires\nlearning a question-dependent, structured reasoning process over images from\nlanguage. Standard deep learning approaches tend to exploit biases in the data\nrather than learn this underlying structure, while leading methods learn to\nvisually reason successfully but are hand-crafted for reasoning. We show that a\ngeneral-purpose, Conditional Batch Normalization approach achieves\nstate-of-the-art results on the CLEVR Visual Reasoning benchmark with a 2.4%\nerror rate. We outperform the next best end-to-end method (4.5%) and even\nmethods that use extra supervision (3.1%). We probe our model to shed light on\nhow it reasons, showing it has learned a question-dependent, multi-step\nprocess. Previous work has operated under the assumption that visual reasoning\ncalls for a specialized architecture, but we show that a general architecture\nwith proper conditioning can learn to visually reason effectively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.03095v3",
            "date": "2017-07-11 01:25:31+00:00",
            "title": "Look Who's Talking: Bipartite Networks as Representations of a Topic Model of New Zealand Parliamentary Speeches",
            "authors": [
                "Ben Curran",
                "Kyle Higham",
                "Elisenda Ortiz",
                "Demival Vasques Filho"
            ],
            "abstract": "Quantitative methods to measure the participation to parliamentary debate and\ndiscourse of elected Members of Parliament (MPs) and the parties they belong to\nare lacking. This is an exploratory study in which we propose the development\nof a new approach for a quantitative analysis of such participation. We utilize\nthe New Zealand government's digital Hansard database to construct a topic\nmodel of parliamentary speeches consisting of nearly 40 million words in the\nperiod 2003-2016. A Latent Dirichlet Allocation topic model is implemented in\norder to reveal the thematic structure of our set of documents. This generative\nstatistical model enables the detection of major themes or topics that are\npublicly discussed in the New Zealand parliament, as well as permitting their\nclassification by MP. Information on topic proportions is subsequently analyzed\nusing a combination of statistical methods. We observe patterns arising from\ntime-series analysis of topic frequencies which can be related to specific\nsocial, economic and legislative events. We then construct a bipartite network\nrepresentation, linking MPs to topics, for each of four parliamentary terms in\nthis time frame. We build projected networks (onto the set of nodes represented\nby MPs) and proceed to the study of the dynamical changes of their topology,\nincluding community structure. By performing this longitudinal network\nanalysis, we can observe the evolution of the New Zealand parliamentary topic\nnetwork and its main parties in the period studied.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03490v1",
            "date": "2017-07-11 23:16:20+00:00",
            "title": "Detecting Policy Preferences and Dynamics in the UN General Debate with Neural Word Embeddings",
            "authors": [
                "Stefano Gurciullo",
                "Slava Mikhaylov"
            ],
            "abstract": "Foreign policy analysis has been struggling to find ways to measure policy\npreferences and paradigm shifts in international political systems. This paper\npresents a novel, potential solution to this challenge, through the application\nof a neural word embedding (Word2vec) model on a dataset featuring speeches by\nheads of state or government in the United Nations General Debate. The paper\nprovides three key contributions based on the output of the Word2vec model.\nFirst, it presents a set of policy attention indices, synthesizing the semantic\nproximity of political speeches to specific policy themes. Second, it\nintroduces country-specific semantic centrality indices, based on topological\nanalyses of countries' semantic positions with respect to each other. Third, it\ntests the hypothesis that there exists a statistical relation between the\nsemantic content of political speeches and UN voting behavior, falsifying it\nand suggesting that political speeches contain information of different nature\nthen the one behind voting outcomes. The paper concludes with a discussion of\nthe practical use of its results and consequences for foreign policy analysis,\npublic accountability, and transparency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03569v1",
            "date": "2017-07-12 07:17:50+00:00",
            "title": "Multitask Learning for Fine-Grained Twitter Sentiment Analysis",
            "authors": [
                "Georgios Balikas",
                "Simon Moura",
                "Massih-Reza Amini"
            ],
            "abstract": "Traditional sentiment analysis approaches tackle problems like ternary\n(3-category) and fine-grained (5-category) classification by learning the tasks\nseparately. We argue that such classification tasks are correlated and we\npropose a multitask approach based on a recurrent neural network that benefits\nby jointly learning them. Our study demonstrates the potential of multitask\nmodels on this type of problems and improves the state-of-the-art results in\nthe fine-grained sentiment classification problem.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.03804v2",
            "date": "2017-07-12 17:15:57+00:00",
            "title": "Source-Target Inference Models for Spatial Instruction Understanding",
            "authors": [
                "Hao Tan",
                "Mohit Bansal"
            ],
            "abstract": "Models that can execute natural language instructions for situated robotic\ntasks such as assembly and navigation have several useful applications in\nhomes, offices, and remote scenarios. We study the semantics of\nspatially-referred configuration and arrangement instructions, based on the\nchallenging Bisk-2016 blank-labeled block dataset. This task involves finding a\nsource block and moving it to the target position (mentioned via a reference\nblock and offset), where the blocks have no names or colors and are just\nreferred to via spatial location features. We present novel models for the\nsubtasks of source block classification and target position regression, based\non joint-loss language and spatial-world representation learning, as well as\nCNN-based and dual attention models to compute the alignment between the world\nblocks and the instruction phrases. For target position prediction, we compare\ntwo inference approaches: annealed sampling via policy gradient versus\nexpectation inference via supervised regression. Our models achieve the new\nstate-of-the-art on this task, with an improvement of 47% on source block\naccuracy and 22% on target position distance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03904v2",
            "date": "2017-07-12 20:53:26+00:00",
            "title": "Quasar: Datasets for Question Answering by Search and Reading",
            "authors": [
                "Bhuwan Dhingra",
                "Kathryn Mazaitis",
                "William W. Cohen"
            ],
            "abstract": "We present two new large-scale datasets aimed at evaluating systems designed\nto comprehend a natural language query and extract its answer from a large\ncorpus of text. The Quasar-S dataset consists of 37000 cloze-style\n(fill-in-the-gap) queries constructed from definitions of software entity tags\non the popular website Stack Overflow. The posts and comments on the website\nserve as the background corpus for answering the cloze questions. The Quasar-T\ndataset consists of 43000 open-domain trivia questions and their answers\nobtained from various internet sources. ClueWeb09 serves as the background\ncorpus for extracting these answers. We pose these datasets as a challenge for\ntwo related subtasks of factoid Question Answering: (1) searching for relevant\npieces of text that include the correct answer to a query, and (2) reading the\nretrieved text to answer the query. We also describe a retrieval system for\nextracting relevant sentences and documents from the corpus given a query, and\ninclude these in the release for researchers wishing to only focus on (2). We\nevaluate several baselines on both datasets, ranging from simple heuristics to\npowerful neural models, and show that these lag behind human performance by\n16.4% and 32.1% for Quasar-S and -T respectively. The datasets are available at\nhttps://github.com/bdhingra/quasar .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.03938v2",
            "date": "2017-07-13 00:17:45+00:00",
            "title": "Representation Learning for Grounded Spatial Reasoning",
            "authors": [
                "Michael Janner",
                "Karthik Narasimhan",
                "Regina Barzilay"
            ],
            "abstract": "The interpretation of spatial references is highly contextual, requiring\njoint inference over both language and the environment. We consider the task of\nspatial reasoning in a simulated environment, where an agent can act and\nreceive rewards. The proposed model learns a representation of the world\nsteered by instruction text. This design allows for precise alignment of local\nneighborhoods with corresponding verbalizations, while also handling global\nreferences in the instructions. We train our model with reinforcement learning\nusing a variant of generalized value iteration. The model outperforms\nstate-of-the-art approaches on several metrics, yielding a 45% reduction in\ngoal localization error.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04242v1",
            "date": "2017-07-13 17:46:59+00:00",
            "title": "Neural Networks for Information Retrieval",
            "authors": [
                "Tom Kenter",
                "Alexey Borisov",
                "Christophe Van Gysel",
                "Mostafa Dehghani",
                "Maarten de Rijke",
                "Bhaskar Mitra"
            ],
            "abstract": "Machine learning plays a role in many aspects of modern IR systems, and deep\nlearning is applied in all of them. The fast pace of modern-day research has\ngiven rise to many different approaches for many different IR problems. The\namount of information available can be overwhelming both for junior students\nand for experienced researchers looking for new research topics and directions.\nAdditionally, it is interesting to see what key insights into IR problems the\nnew technologies are able to give us. The aim of this full-day tutorial is to\ngive a clear overview of current tried-and-trusted neural methods in IR and how\nthey benefit IR research. It covers key architectures, as well as the most\npromising future directions.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.04244v1",
            "date": "2017-07-13 17:52:51+00:00",
            "title": "Lithium NLP: A System for Rich Information Extraction from Noisy User Generated Text on Social Media",
            "authors": [
                "Preeti Bhargava",
                "Nemanja Spasojevic",
                "Guoning Hu"
            ],
            "abstract": "In this paper, we describe the Lithium Natural Language Processing (NLP)\nsystem - a resource-constrained, high- throughput and language-agnostic system\nfor information extraction from noisy user generated text on social media.\nLithium NLP extracts a rich set of information including entities, topics,\nhashtags and sentiment from text. We discuss several real world applications of\nthe system currently incorporated in Lithium products. We also compare our\nsystem with existing commercial and academic NLP systems in terms of\nperformance, information extracted and languages supported. We show that\nLithium NLP is at par with and in some cases, outperforms state- of-the-art\ncommercial NLP systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.04678v1",
            "date": "2017-07-15 02:22:41+00:00",
            "title": "Lyrics-Based Music Genre Classification Using a Hierarchical Attention Network",
            "authors": [
                "Alexandros Tsaptsinos"
            ],
            "abstract": "Music genre classification, especially using lyrics alone, remains a\nchallenging topic in Music Information Retrieval. In this study we apply\nrecurrent neural network models to classify a large dataset of intact song\nlyrics. As lyrics exhibit a hierarchical layer structure - in which words\ncombine to form lines, lines form segments, and segments form a complete song -\nwe adapt a hierarchical attention network (HAN) to exploit these layers and in\naddition learn the importance of the words, lines, and segments. We test the\nmodel over a 117-genre dataset and a reduced 20-genre dataset. Experimental\nresults show that the HAN outperforms both non-neural models and simpler neural\nmodels, whilst also classifying over a higher number of genres than previous\nresearch. Through the learning process we can also visualise which words or\nlines in a song the model believes are important to classifying the genre. As a\nresult the HAN provides insights, from a computational perspective, into\nlyrical structure and language features that differentiate musical genres.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.04724v1",
            "date": "2017-07-15 11:03:49+00:00",
            "title": "Memoisation: Purely, Left-recursively, and with (Continuation Passing) Style",
            "authors": [
                "Samer Abdallah"
            ],
            "abstract": "Memoisation, or tabling, is a well-known technique that yields large\nimprovements in the performance of some recursive computations. Tabled\nresolution in Prologs such as XSB and B-Prolog can transform so called\nleft-recursive predicates from non-terminating computations into finite and\nwell-behaved ones. In the functional programming literature, memoisation has\nusually been implemented in a way that does not handle left-recursion,\nrequiring supplementary mechanisms to prevent non-termination. A notable\nexception is Johnson's (1995) continuation passing approach in Scheme. This,\nhowever, relies on mutation of a memo table data structure and coding in\nexplicit continuation passing style. We show how Johnson's approach can be\nimplemented purely functionally in a modern, strongly typed functional language\n(OCaml), presented via a monadic interface that hides the implementation\ndetails, yet providing a way to return a compact represention of the memo\ntables at the end of the computation.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1707.04879v1",
            "date": "2017-07-16 13:27:56+00:00",
            "title": "Listening while Speaking: Speech Chain by Deep Learning",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Despite the close relationship between speech perception and production,\nresearch in automatic speech recognition (ASR) and text-to-speech synthesis\n(TTS) has progressed more or less independently without exerting much mutual\ninfluence on each other. In human communication, on the other hand, a\nclosed-loop speech chain mechanism with auditory feedback from the speaker's\nmouth to her ear is crucial. In this paper, we take a step further and develop\na closed-loop speech chain model based on deep learning. The\nsequence-to-sequence model in close-loop architecture allows us to train our\nmodel on the concatenation of both labeled and unlabeled data. While ASR\ntranscribes the unlabeled speech features, TTS attempts to reconstruct the\noriginal speech waveform based on the text from ASR. In the opposite direction,\nASR also attempts to reconstruct the original text transcription given the\nsynthesized speech. To the best of our knowledge, this is the first deep\nlearning model that integrates human speech perception and production\nbehaviors. Our experimental results show that the proposed approach\nsignificantly improved the performance more than separate systems that were\nonly trained with labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04935v1",
            "date": "2017-07-16 19:40:26+00:00",
            "title": "Automatized Generation of Alphabets of Symbols",
            "authors": [
                "Serhii Hamotskyi",
                "Anis Rojbi",
                "Sergii Stirenko",
                "Yuri Gordienko"
            ],
            "abstract": "In this paper, we discuss the generation of symbols (and alphabets) based on\nspecific user requirements (medium, priorities, type of information that needs\nto be conveyed). A framework for the generation of alphabets is proposed, and\nits use for the generation of a shorthand writing system is explored. We\ndiscuss the possible use of machine learning and genetic algorithms to gather\ninputs for generation of such alphabets and for optimization of already\ngenerated ones. The alphabets generated using such methods may be used in very\ndifferent fields, from the creation of synthetic languages and constructed\nscripts to the creation of sensible commands for multimodal interaction through\nHuman-Computer Interfaces, such as mouse gestures, touchpads, body gestures,\neye-tracking cameras, and brain-computing Interfaces, especially in\napplications for elderly care and people with disabilities.",
            "journal ref": "Proceedings of the 2017 Federated Conference on Computer Science\n  and Information Systems (FedCSIS-2017), p.639-642, Prague, Czech Republic,\n  September 3-6, 2017",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1707.05115v1",
            "date": "2017-07-17 12:20:15+00:00",
            "title": "The Power of Constraint Grammars Revisited",
            "authors": [
                "Anssi Yli-Jyr\u00e4"
            ],
            "abstract": "Sequential Constraint Grammar (SCG) (Karlsson, 1990) and its extensions have\nlacked clear connections to formal language theory. The purpose of this article\nis to lay a foundation for these connections by simplifying the definition of\nstrings processed by the grammar and by showing that Nonmonotonic SCG is\nundecidable and that derivations similar to the Generative Phonology exist. The\ncurrent investigations propose resource bounds that restrict the generative\npower of SCG to a subset of context sensitive languages and present a strong\nfinite-state condition for grammars as wholes. We show that a grammar is\nequivalent to a finite-state transducer if it is implemented with a Turing\nmachine that runs in o(n log n) time. This condition opens new finite-state\nhypotheses and avenues for deeper analysis of SCG instances in the way inspired\nby Finite-State Phonology.",
            "journal ref": "Proceedings of the NoDaLiDa 2017 Workshop on Constraint Grammar -\n  Methods, Tools and Applications, 22 May 2017, Gothenburg, Sweden. Pages 23-31",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05227v1",
            "date": "2017-07-17 15:24:09+00:00",
            "title": "Auxiliary Objectives for Neural Error Detection Models",
            "authors": [
                "Marek Rei",
                "Helen Yannakoudakis"
            ],
            "abstract": "We investigate the utility of different auxiliary objectives and training\nstrategies within a neural sequence labeling approach to error detection in\nlearner writing. Auxiliary costs provide the model with additional linguistic\ninformation, allowing it to learn general-purpose compositional features that\ncan then be exploited for other objectives. Our experiments show that a joint\nlearning approach trained with parallel labels on in-domain data improves\nperformance over the previous best error detection system. While the resulting\nmodel has the same number of parameters, the additional objectives allow it to\nbe optimised more efficiently and achieve better performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05233v1",
            "date": "2017-07-17 15:31:20+00:00",
            "title": "Detecting Off-topic Responses to Visual Prompts",
            "authors": [
                "Marek Rei"
            ],
            "abstract": "Automated methods for essay scoring have made great progress in recent years,\nachieving accuracies very close to human annotators. However, a known weakness\nof such automated scorers is not taking into account the semantic relevance of\nthe submitted text. While there is existing work on detecting answer relevance\ngiven a textual prompt, very little previous research has been done to\nincorporate visual writing prompts. We propose a neural architecture and\nseveral extensions for detecting off-topic responses to visual prompts and\nevaluate it on a dataset of texts written by language learners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05481v2",
            "date": "2017-07-18 05:49:30+00:00",
            "title": "A Comparative Analysis of Social Network Pages by Interests of Their Followers",
            "authors": [
                "Elena Mikhalkova",
                "Nadezhda Ganzherli",
                "Yuri Karyakin"
            ],
            "abstract": "Being a matter of cognition, user interests should be apt to classification\nindependent of the language of users, social network and content of interest\nitself. To prove it, we analyze a collection of English and Russian Twitter and\nVkontakte community pages by interests of their followers. First, we create a\nmodel of Major Interests (MaIs) with the help of expert analysis and then\nclassify a set of pages using machine learning algorithms (SVM, Neural Network,\nNaive Bayes, and some other). We take three interest domains that are typical\nof both English and Russian-speaking communities: football, rock music,\nvegetarianism. The results of classification show a greater correlation between\nRussian-Vkontakte and Russian-Twitter pages while English-Twitterpages appear\nto provide the highest score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05612v4",
            "date": "2017-07-18 13:51:32+00:00",
            "title": "VSE++: Improving Visual-Semantic Embeddings with Hard Negatives",
            "authors": [
                "Fartash Faghri",
                "David J. Fleet",
                "Jamie Ryan Kiros",
                "Sanja Fidler"
            ],
            "abstract": "We present a new technique for learning visual-semantic embeddings for\ncross-modal retrieval. Inspired by hard negative mining, the use of hard\nnegatives in structured prediction, and ranking loss functions, we introduce a\nsimple change to common loss functions used for multi-modal embeddings. That,\ncombined with fine-tuning and use of augmented data, yields significant gains\nin retrieval performance. We showcase our approach, VSE++, on MS-COCO and\nFlickr30K datasets, using ablation studies and comparisons with existing\nmethods. On MS-COCO our approach outperforms state-of-the-art methods by 8.8%\nin caption retrieval and 11.3% in image retrieval (at R@1).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1707.05720v1",
            "date": "2017-07-18 16:02:05+00:00",
            "title": "Grounding Spatio-Semantic Referring Expressions for Human-Robot Interaction",
            "authors": [
                "Mohit Shridhar",
                "David Hsu"
            ],
            "abstract": "The human language is one of the most natural interfaces for humans to\ninteract with robots. This paper presents a robot system that retrieves\neveryday objects with unconstrained natural language descriptions. A core issue\nfor the system is semantic and spatial grounding, which is to infer objects and\ntheir spatial relationships from images and natural language expressions. We\nintroduce a two-stage neural-network grounding pipeline that maps natural\nlanguage referring expressions directly to objects in the images. The first\nstage uses visual descriptions in the referring expressions to generate a\ncandidate set of relevant objects. The second stage examines all pairwise\nrelationships between the candidates and predicts the most likely referred\nobject according to the spatial descriptions in the referring expressions. A\nkey feature of our system is that by leveraging a large dataset of images\nlabeled with text descriptions, it allows unrestricted object types and natural\nlanguage referring expressions. Preliminary results indicate that our system\noutperforms a near state-of-the-art object comprehension system on standard\nbenchmark datasets. We also present a robot system that follows voice commands\nto pick and place previously unseen objects.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1707.06163v1",
            "date": "2017-07-19 15:39:09+00:00",
            "title": "Metrical-accent Aware Vocal Onset Detection in Polyphonic Audio",
            "authors": [
                "Georgi Dzhambazov",
                "Andre Holzapfel",
                "Ajay Srinivasamurthy",
                "Xavier Serra"
            ],
            "abstract": "The goal of this study is the automatic detection of onsets of the singing\nvoice in polyphonic audio recordings. Starting with a hypothesis that the\nknowledge of the current position in a metrical cycle (i.e. metrical accent)\ncan improve the accuracy of vocal note onset detection, we propose a novel\nprobabilistic model to jointly track beats and vocal note onsets. The proposed\nmodel extends a state of the art model for beat and meter tracking, in which\na-priori probability of a note at a specific metrical accent interacts with the\nprobability of observing a vocal note onset. We carry out an evaluation on a\nvaried collection of multi-instrument datasets from two music traditions\n(English popular music and Turkish makam) with different types of metrical\ncycles and singing styles. Results confirm that the proposed model reasonably\nimproves vocal note onset detection accuracy compared to a baseline model that\ndoes not take metrical position into account.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1707.06209v1",
            "date": "2017-07-19 17:28:46+00:00",
            "title": "Crowdsourcing Multiple Choice Science Questions",
            "authors": [
                "Johannes Welbl",
                "Nelson F. Liu",
                "Matt Gardner"
            ],
            "abstract": "We present a novel method for obtaining high-quality, domain-targeted\nmultiple choice questions from crowd workers. Generating these questions can be\ndifficult without trading away originality, relevance or diversity in the\nanswer options. Our method addresses these problems by leveraging a large\ncorpus of domain-specific text and a small set of existing questions. It\nproduces model suggestions for document selection and answer distractor choice\nwhich aid the human question generation process. With this method we have\nassembled SciQ, a dataset of 13.7K multiple choice science exam questions\n(Dataset available at http://allenai.org/data.html). We demonstrate that the\nmethod produces in-domain questions by providing an analysis of this new\ndataset and by showing that humans cannot distinguish the crowdsourced\nquestions from original questions. When using SciQ as additional training data\nto existing questions, we observe accuracy improvements on real science exams.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1707.06226v1",
            "date": "2017-07-19 01:21:26+00:00",
            "title": "The Role of Conversation Context for Sarcasm Detection in Online Interactions",
            "authors": [
                "Debanjan Ghosh",
                "Alexander Richard Fabbri",
                "Smaranda Muresan"
            ],
            "abstract": "Computational models for sarcasm detection have often relied on the content\nof utterances in isolation. However, speaker's sarcastic intent is not always\nobvious without additional context. Focusing on social media discussions, we\ninvestigate two issues: (1) does modeling of conversation context help in\nsarcasm detection and (2) can we understand what part of conversation context\ntriggered the sarcastic reply. To address the first issue, we investigate\nseveral types of Long Short-Term Memory (LSTM) networks that can model both the\nconversation context and the sarcastic response. We show that the conditional\nLSTM network (Rocktaschel et al., 2015) and LSTM networks with sentence level\nattention on context and response outperform the LSTM model that reads only the\nresponse. To address the second issue, we present a qualitative analysis of\nattention weights produced by the LSTM models with attention and discuss the\nresults compared with human performance on the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06355v1",
            "date": "2017-07-20 03:12:29+00:00",
            "title": "Video Question Answering via Attribute-Augmented Attention Network Learning",
            "authors": [
                "Yunan Ye",
                "Zhou Zhao",
                "Yimeng Li",
                "Long Chen",
                "Jun Xiao",
                "Yueting Zhuang"
            ],
            "abstract": "Video Question Answering is a challenging problem in visual information\nretrieval, which provides the answer to the referenced video content according\nto the question. However, the existing visual question answering approaches\nmainly tackle the problem of static image question, which may be ineffectively\nfor video question answering due to the insufficiency of modeling the temporal\ndynamics of video contents. In this paper, we study the problem of video\nquestion answering by modeling its temporal dynamics with frame-level attention\nmechanism. We propose the attribute-augmented attention network learning\nframework that enables the joint frame-level attribute detection and unified\nvideo representation learning for video question answering. We then incorporate\nthe multi-step reasoning process for our proposed attention network to further\nimprove the performance. We construct a large-scale video question answering\ndataset. We conduct the experiments on both multiple-choice and open-ended\nvideo question answering tasks to show the effectiveness of the proposed\nmethod.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1707.06480v1",
            "date": "2017-07-20 12:46:09+00:00",
            "title": "Syllable-aware Neural Language Models: A Failure to Beat Character-aware Ones",
            "authors": [
                "Zhenisbek Assylbekov",
                "Rustem Takhanov",
                "Bagdat Myrzakhmetov",
                "Jonathan N. Washington"
            ],
            "abstract": "Syllabification does not seem to improve word-level RNN language modeling\nquality when compared to character-based segmentation. However, our best\nsyllable-aware language model, achieving performance comparable to the\ncompetitive character-aware model, has 18%-33% fewer parameters and is trained\n1.2-2.2 times faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06527v1",
            "date": "2017-07-19 03:48:54+00:00",
            "title": "Single-Channel Multi-talker Speech Recognition with Permutation Invariant Training",
            "authors": [
                "Yanmin Qian",
                "Xuankai Chang",
                "Dong Yu"
            ],
            "abstract": "Although great progresses have been made in automatic speech recognition\n(ASR), significant performance degradation is still observed when recognizing\nmulti-talker mixed speech. In this paper, we propose and evaluate several\narchitectures to address this problem under the assumption that only a single\nchannel of mixed signal is available. Our technique extends permutation\ninvariant training (PIT) by introducing the front-end feature separation module\nwith the minimum mean square error (MSE) criterion and the back-end recognition\nmodule with the minimum cross entropy (CE) criterion. More specifically, during\ntraining we compute the average MSE or CE over the whole utterance for each\npossible utterance-level output-target assignment, pick the one with the\nminimum MSE or CE, and optimize for that assignment. This strategy elegantly\nsolves the label permutation problem observed in the deep learning based\nmulti-talker mixed speech separation and recognition systems. The proposed\narchitectures are evaluated and compared on an artificially mixed AMI dataset\nwith both two- and three-talker mixed speech. The experimental results indicate\nthat our proposed architectures can cut the word error rate (WER) by 45.0% and\n25.0% relatively against the state-of-the-art single-talker speech recognition\nsystem across all speakers when their energies are comparable, for two- and\nthree-talker mixed speech, respectively. To our knowledge, this is the first\nwork on the multi-talker mixed speech recognition on the challenging\nspeaker-independent spontaneous large vocabulary continuous speech task.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1707.06588v3",
            "date": "2017-07-20 16:18:00+00:00",
            "title": "VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop",
            "authors": [
                "Yaniv Taigman",
                "Lior Wolf",
                "Adam Polyak",
                "Eliya Nachmani"
            ],
            "abstract": "We present a new neural text to speech (TTS) method that is able to transform\ntext to speech in voices that are sampled in the wild. Unlike other systems,\nour solution is able to deal with unconstrained voice samples and without\nrequiring aligned phonemes or linguistic features. The network architecture is\nsimpler than those in the existing literature and is based on a novel shifting\nbuffer working memory. The same buffer is used for estimating the attention,\ncomputing the output audio, and for updating the buffer itself. The input\nsentence is encoded using a context-free lookup table that contains one entry\nper character or phoneme. The speakers are similarly represented by a short\nvector that can also be fitted to new identities, even with only a few samples.\nVariability in the generated speech is achieved by priming the buffer prior to\ngenerating the audio. Experimental results on several datasets demonstrate\nconvincing capabilities, making TTS accessible to a wider range of\napplications. In order to promote reproducibility, we release our source code\nand models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1707.06841v1",
            "date": "2017-07-21 11:06:12+00:00",
            "title": "An Error-Oriented Approach to Word Embedding Pre-Training",
            "authors": [
                "Youmna Farag",
                "Marek Rei",
                "Ted Briscoe"
            ],
            "abstract": "We propose a novel word embedding pre-training approach that exploits writing\nerrors in learners' scripts. We compare our method to previous models that tune\nthe embeddings based on script scores and the discrimination between correct\nand corrupt word contexts in addition to the generic commonly-used embeddings\npre-trained on large corpora. The comparison is achieved by using the\naforementioned models to bootstrap a neural network that learns to predict a\nholistic score for scripts. Furthermore, we investigate augmenting our model\nwith error corrections and monitor the impact on performance. Our results show\nthat our error-oriented approach outperforms other comparable ones which is\nfurther demonstrated when training on more data. Additionally, extending the\nmodel with corrections provides further performance gains when data sparsity is\nan issue.",
            "journal ref": "The 12th Workshop on Innovative Use of NLP for Building\n  Educational Applications (BEA 2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06932v1",
            "date": "2017-07-21 15:19:09+00:00",
            "title": "A study on text-score disagreement in online reviews",
            "authors": [
                "Michela Fazzolari",
                "Vittoria Cozza",
                "Marinella Petrocchi",
                "Angelo Spognardi"
            ],
            "abstract": "In this paper, we focus on online reviews and employ artificial intelligence\ntools, taken from the cognitive computing field, to help understanding the\nrelationships between the textual part of the review and the assigned numerical\nscore. We move from the intuitions that 1) a set of textual reviews expressing\ndifferent sentiments may feature the same score (and vice-versa); and 2)\ndetecting and analyzing the mismatches between the review content and the\nactual score may benefit both service providers and consumers, by highlighting\nspecific factors of satisfaction (and dissatisfaction) in texts.\n  To prove the intuitions, we adopt sentiment analysis techniques and we\nconcentrate on hotel reviews, to find polarity mismatches therein. In\nparticular, we first train a text classifier with a set of annotated hotel\nreviews, taken from the Booking website. Then, we analyze a large dataset, with\naround 160k hotel reviews collected from Tripadvisor, with the aim of detecting\na polarity mismatch, indicating if the textual content of the review is in\nline, or not, with the associated score.\n  Using well established artificial intelligence techniques and analyzing in\ndepth the reviews featuring a mismatch between the text polarity and the score,\nwe find that -on a scale of five stars- those reviews ranked with middle scores\ninclude a mixture of positive and negative aspects.\n  The approach proposed here, beside acting as a polarity detector, provides an\neffective selection of reviews -on an initial very large dataset- that may\nallow both consumers and providers to focus directly on the review subset\nfeaturing a text/score disagreement, which conveniently convey to the user a\nsummary of positive and negative features of the review target.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.06939v1",
            "date": "2017-07-21 15:41:38+00:00",
            "title": "Autocompletion interfaces make crowd workers slower, but their use promotes response diversity",
            "authors": [
                "Xipei Liu",
                "James P. Bagrow"
            ],
            "abstract": "Creative tasks such as ideation or question proposal are powerful\napplications of crowdsourcing, yet the quantity of workers available for\naddressing practical problems is often insufficient. To enable scalable\ncrowdsourcing thus requires gaining all possible efficiency and information\nfrom available workers. One option for text-focused tasks is to allow assistive\ntechnology, such as an autocompletion user interface (AUI), to help workers\ninput text responses. But support for the efficacy of AUIs is mixed. Here we\ndesigned and conducted a randomized experiment where workers were asked to\nprovide short text responses to given questions. Our experimental goal was to\ndetermine if an AUI helps workers respond more quickly and with improved\nconsistency by mitigating typos and misspellings. Surprisingly, we found that\nneither occurred: workers assigned to the AUI treatment were slower than those\nassigned to the non-AUI control and their responses were more diverse, not\nless, than those of the control. Both the lexical and semantic diversities of\nresponses were higher, with the latter measured using word2vec. A crowdsourcer\ninterested in worker speed may want to avoid using an AUI, but using an AUI to\nboost response diversity may be valuable to crowdsourcers interested in\nreceiving as much novel information from workers as possible.",
            "journal ref": "Human Computation 6:1:42-55 (2019)",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1707.07066v3",
            "date": "2017-07-21 23:13:50+00:00",
            "title": "Ultraslow diffusion in language: Dynamics of appearance of already popular adjectives on Japanese blogs",
            "authors": [
                "Hayafumi Watanabe"
            ],
            "abstract": "What dynamics govern a time series representing the appearance of words in\nsocial media data? In this paper, we investigate an elementary dynamics, from\nwhich word-dependent special effects are segregated, such as breaking news,\nincreasing (or decreasing) concerns, or seasonality. To elucidate this problem,\nwe investigated approximately three billion Japanese blog articles over a\nperiod of six years, and analysed some corresponding solvable mathematical\nmodels. From the analysis, we found that a word appearance can be explained by\nthe random diffusion model based on the power-law forgetting process, which is\na type of long memory point process related to ARFIMA(0,0.5,0). In particular,\nwe confirmed that ultraslow diffusion (where the mean squared displacement\ngrows logarithmically), which the model predicts in an approximate manner,\nreproduces the actual data. In addition, we also show that the model can\nreproduce other statistical properties of a time series: (i) the fluctuation\nscaling, (ii) spectrum density, and (iii) shapes of the probability density\nfunctions.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1707.07240v3",
            "date": "2017-07-23 03:06:47+00:00",
            "title": "Language modeling with Neural trans-dimensional random fields",
            "authors": [
                "Bin Wang",
                "Zhijian Ou"
            ],
            "abstract": "Trans-dimensional random field language models (TRF LMs) have recently been\nintroduced, where sentences are modeled as a collection of random fields. The\nTRF approach has been shown to have the advantages of being computationally\nmore efficient in inference than LSTM LMs with close performance and being able\nto flexibly integrating rich features. In this paper we propose neural TRFs,\nbeyond of the previous discrete TRFs that only use linear potentials with\ndiscrete features. The idea is to use nonlinear potentials with continuous\nfeatures, implemented by neural networks (NNs), in the TRF framework. Neural\nTRFs combine the advantages of both NNs and TRFs. The benefits of word\nembedding, nonlinear feature learning and larger context modeling are inherited\nfrom the use of NNs. At the same time, the strength of efficient inference by\navoiding expensive softmax is preserved. A number of technical contributions,\nincluding employing deep convolutional neural networks (CNNs) to define the\npotentials and incorporating the joint stochastic approximation (JSA) strategy\nin the training algorithm, are developed in this work, which enable us to\nsuccessfully train neural TRF LMs. Various LMs are evaluated in terms of speech\nrecognition WERs by rescoring the 1000-best lists of WSJ'92 test data. The\nresults show that neural TRF LMs not only improve over discrete TRF LMs, but\nalso perform slightly better than LSTM LMs with only one fifth of parameters\nand 16x faster inference efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07402v4",
            "date": "2017-07-24 04:35:19+00:00",
            "title": "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback",
            "authors": [
                "Khanh Nguyen",
                "Hal Daum\u00e9 III",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Machine translation is a natural candidate problem for reinforcement learning\nfrom human feedback: users provide quick, dirty ratings on candidate\ntranslations to guide a system to improve. Yet, current neural machine\ntranslation training focuses on expensive human-generated reference\ntranslations. We describe a reinforcement learning algorithm that improves\nneural machine translation systems from simulated human feedback. Our algorithm\ncombines the advantage actor-critic algorithm (Mnih et al., 2016) with the\nattention-based neural encoder-decoder architecture (Luong et al., 2015). This\nalgorithm (a) is well-designed for problems with a large action space and\ndelayed rewards, (b) effectively optimizes traditional corpus-level machine\ntranslation metrics, and (c) is robust to skewed, high-variance, granular\nfeedback modeled after actual human behaviors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.07585v1",
            "date": "2017-07-19 13:40:06+00:00",
            "title": "Stock Prediction: a method based on extraction of news features and recurrent neural networks",
            "authors": [
                "Zeya Zhang",
                "Weizheng Chen",
                "Hongfei Yan"
            ],
            "abstract": "This paper proposed a method for stock prediction. In terms of feature\nextraction, we extract the features of stock-related news besides stock prices.\nWe first select some seed words based on experience which are the symbols of\ngood news and bad news. Then we propose an optimization method and calculate\nthe positive polar of all words. After that, we construct the features of news\nbased on the positive polar of their words. In consideration of sequential\nstock prices and continuous news effects, we propose a recurrent neural network\nmodel to help predict stock prices. Compared to SVM classifier with price\nfeatures, we find our proposed method has an over 5% improvement on stock\nprediction accuracy in experiments.",
            "journal ref": "None",
            "category": "q-fin.ST"
        },
        {
            "url": "http://arxiv.org/abs/1707.07605v1",
            "date": "2017-07-24 15:23:41+00:00",
            "title": "Share your Model instead of your Data: Privacy Preserving Mimic Learning for Ranking",
            "authors": [
                "Mostafa Dehghani",
                "Hosein Azarbonyad",
                "Jaap Kamps",
                "Maarten de Rijke"
            ],
            "abstract": "Deep neural networks have become a primary tool for solving problems in many\nfields. They are also used for addressing information retrieval problems and\nshow strong performance in several tasks. Training these models requires large,\nrepresentative datasets and for most IR tasks, such data contains sensitive\ninformation from users. Privacy and confidentiality concerns prevent many data\nowners from sharing the data, thus today the research community can only\nbenefit from research on large-scale datasets in a limited manner. In this\npaper, we discuss privacy preserving mimic learning, i.e., using predictions\nfrom a privacy preserving trained model instead of labels from the original\nsensitive training data as a supervision signal. We present the results of\npreliminary experiments in which we apply the idea of mimic learning and\nprivacy preserving mimic learning for the task of document re-ranking as one of\nthe core IR tasks. This research is a step toward laying the ground for\nenabling researchers from data-rich environments to share knowledge learned\nfrom actual users' data, which should facilitate research collaborations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07678v1",
            "date": "2017-07-24 15:10:40+00:00",
            "title": "Extracting Core Claims from Scientific Articles",
            "authors": [
                "Tom Jansen",
                "Tobias Kuhn"
            ],
            "abstract": "The number of scientific articles has grown rapidly over the years and there\nare no signs that this growth will slow down in the near future. Because of\nthis, it becomes increasingly difficult to keep up with the latest developments\nin a scientific field. To address this problem, we present here an approach to\nhelp researchers learn about the latest developments and findings by extracting\nin a normalized form core claims from scientific articles. This normalized\nrepresentation is a controlled natural language of English sentences called\nAIDA, which has been proposed in previous work as a method to formally\nstructure and organize scientific findings and discourse. We show how such AIDA\nsentences can be automatically extracted by detecting the core claim of an\narticle, checking for AIDA compliance, and - if necessary - transforming it\ninto a compliant sentence. While our algorithm is still far from perfect, our\nresults indicate that the different steps are feasible and they support the\nclaim that AIDA sentences might be a promising approach to improve scientific\ncommunication in the future.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.07930v1",
            "date": "2017-07-25 11:54:19+00:00",
            "title": "Structural Regularities in Text-based Entity Vector Spaces",
            "authors": [
                "Christophe Van Gysel",
                "Maarten de Rijke",
                "Evangelos Kanoulas"
            ],
            "abstract": "Entity retrieval is the task of finding entities such as people or products\nin response to a query, based solely on the textual documents they are\nassociated with. Recent semantic entity retrieval algorithms represent queries\nand experts in finite-dimensional vector spaces, where both are constructed\nfrom text sequences.\n  We investigate entity vector spaces and the degree to which they capture\nstructural regularities. Such vector spaces are constructed in an unsupervised\nmanner without explicit information about structural aspects. For concreteness,\nwe address these questions for a specific type of entity: experts in the\ncontext of expert finding. We discover how clusterings of experts correspond to\ncommittees in organizations, the ability of expert representations to encode\nthe co-author graph, and the degree to which they encode academic rank. We\ncompare latent, continuous representations created using methods based on\ndistributional semantics (LSI), topic models (LDA) and neural networks\n(word2vec, doc2vec, SERT). Vector spaces created using neural methods, such as\ndoc2vec and SERT, systematically perform better at clustering than LSI, LDA and\nword2vec. When it comes to encoding entity relations, SERT performs best.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1707.08209v1",
            "date": "2017-07-11 19:52:56+00:00",
            "title": "On the letter frequencies and entropy of written Marathi",
            "authors": [
                "Jaydeep Chipalkatti",
                "Mihir Kulkarni"
            ],
            "abstract": "We carry out a comprehensive analysis of letter frequencies in contemporary\nwritten Marathi. We determine sets of letters which statistically predominate\nany large generic Marathi text, and use these sets to estimate the entropy of\nMarathi.",
            "journal ref": "None",
            "category": "cs.IT"
        },
        {
            "url": "http://arxiv.org/abs/1707.08214v2",
            "date": "2017-07-25 20:52:32+00:00",
            "title": "Dual Rectified Linear Units (DReLUs): A Replacement for Tanh Activation Functions in Quasi-Recurrent Neural Networks",
            "authors": [
                "Fr\u00e9deric Godin",
                "Jonas Degrave",
                "Joni Dambre",
                "Wesley De Neve"
            ],
            "abstract": "In this paper, we introduce a novel type of Rectified Linear Unit (ReLU),\ncalled a Dual Rectified Linear Unit (DReLU). A DReLU, which comes with an\nunbounded positive and negative image, can be used as a drop-in replacement for\na tanh activation function in the recurrent step of Quasi-Recurrent Neural\nNetworks (QRNNs) (Bradbury et al. (2017)). Similar to ReLUs, DReLUs are less\nprone to the vanishing gradient problem, they are noise robust, and they induce\nsparse activations.\n  We independently reproduce the QRNN experiments of Bradbury et al. (2017) and\ncompare our DReLU-based QRNNs with the original tanh-based QRNNs and Long\nShort-Term Memory networks (LSTMs) on sentiment classification and word-level\nlanguage modeling. Additionally, we evaluate on character-level language\nmodeling, showing that we are able to stack up to eight QRNN layers with\nDReLUs, thus making it possible to improve the current state-of-the-art in\ncharacter-level language modeling over shallow architectures based on LSTMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.08616v2",
            "date": "2017-07-26 19:23:54+00:00",
            "title": "Guiding Reinforcement Learning Exploration Using Natural Language",
            "authors": [
                "Brent Harrison",
                "Upol Ehsan",
                "Mark O. Riedl"
            ],
            "abstract": "In this work we present a technique to use natural language to help\nreinforcement learning generalize to unseen environments. This technique uses\nneural machine translation, specifically the use of encoder-decoder networks,\nto learn associations between natural language behavior descriptions and\nstate-action information. We then use this learned model to guide agent\nexploration using a modified version of policy shaping to make it more\neffective at learning in unseen environments. We evaluate this technique using\nthe popular arcade game, Frogger, under ideal and non-ideal conditions. This\nevaluation shows that our modified policy shaping algorithm improves over a\nQ-learning agent as well as a baseline version of policy shaping.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.08852v1",
            "date": "2017-07-27 13:14:57+00:00",
            "title": "Detecting and Explaining Causes From Text For a Time Series Event",
            "authors": [
                "Dongyeop Kang",
                "Varun Gangal",
                "Ang Lu",
                "Zheng Chen",
                "Eduard Hovy"
            ],
            "abstract": "Explaining underlying causes or effects about events is a challenging but\nvaluable task. We define a novel problem of generating explanations of a time\nseries event by (1) searching cause and effect relationships of the time series\nwith textual data and (2) constructing a connecting chain between them to\ngenerate an explanation. To detect causal features from text, we propose a\nnovel method based on the Granger causality of time series between features\nextracted from text such as N-grams, topics, sentiments, and their composition.\nThe generation of the sequence of causal entities requires a commonsense\ncausative knowledge base with efficient reasoning. To ensure good\ninterpretability and appropriate lexical usage we combine symbolic and neural\nrepresentations, using a neural reasoning algorithm trained on commonsense\ncausal tuples to predict the next cause step. Our quantitative and human\nanalysis show empirical evidence that our method successfully extracts\nmeaningful causality relationships between time series with textual features\nand generates appropriate explanation between them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09118v3",
            "date": "2017-07-28 06:32:47+00:00",
            "title": "Counterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translation",
            "authors": [
                "Carolin Lawrence",
                "Artem Sokolov",
                "Stefan Riezler"
            ],
            "abstract": "The goal of counterfactual learning for statistical machine translation (SMT)\nis to optimize a target SMT system from logged data that consist of user\nfeedback to translations that were predicted by another, historic SMT system. A\nchallenge arises by the fact that risk-averse commercial SMT systems\ndeterministically log the most probable translation. The lack of sufficient\nexploration of the SMT output space seemingly contradicts the theoretical\nrequirements for counterfactual learning. We show that counterfactual learning\nfrom deterministic bandit logs is possible nevertheless by smoothing out\ndeterministic components in learning. This can be achieved by additive and\nmultiplicative control variates that avoid degenerate behavior in empirical\nrisk minimization. Our simulation experiments show improvements of up to 2 BLEU\npoints by counterfactual learning from deterministic bandit feedback.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1707.09457v1",
            "date": "2017-07-29 03:38:32+00:00",
            "title": "Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints",
            "authors": [
                "Jieyu Zhao",
                "Tianlu Wang",
                "Mark Yatskar",
                "Vicente Ordonez",
                "Kai-Wei Chang"
            ],
            "abstract": "Language is increasingly being used to define rich visual recognition\nproblems with supporting image collections sourced from the web. Structured\nprediction models are used in these tasks to take advantage of correlations\nbetween co-occurring labels and visual input but risk inadvertently encoding\nsocial biases found in web corpora. In this work, we study data and models\nassociated with multilabel object classification and visual semantic role\nlabeling. We find that (a) datasets for these tasks contain significant gender\nbias and (b) models trained on these datasets further amplify existing bias.\nFor example, the activity cooking is over 33% more likely to involve females\nthan males in a training set, and a trained model further amplifies the\ndisparity to 68% at test time. We propose to inject corpus-level constraints\nfor calibrating existing structured prediction models and design an algorithm\nbased on Lagrangian relaxation for collective inference. Our method results in\nalmost no performance loss for the underlying recognition task but decreases\nthe magnitude of bias amplification by 47.5% and 40.5% for multilabel\nclassification and visual semantic role labeling, respectively.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.09491v1",
            "date": "2017-07-29 10:09:04+00:00",
            "title": "Topology Analysis of International Networks Based on Debates in the United Nations",
            "authors": [
                "Stefano Gurciullo",
                "Slava Mikhaylov"
            ],
            "abstract": "In complex, high dimensional and unstructured data it is often difficult to\nextract meaningful patterns. This is especially the case when dealing with\ntextual data. Recent studies in machine learning, information theory and\nnetwork science have developed several novel instruments to extract the\nsemantics of unstructured data, and harness it to build a network of relations.\nSuch approaches serve as an efficient tool for dimensionality reduction and\npattern detection. This paper applies semantic network science to extract\nideological proximity in the international arena, by focusing on the data from\nGeneral Debates in the UN General Assembly on the topics of high salience to\ninternational community. UN General Debate corpus (UNGDC) covers all high-level\ndebates in the UN General Assembly from 1970 to 2014, covering all UN member\nstates. The research proceeds in three main steps. First, Latent Dirichlet\nAllocation (LDA) is used to extract the topics of the UN speeches, and\ntherefore semantic information. Each country is then assigned a vector\nspecifying the exposure to each of the topics identified. This intermediate\noutput is then used in to construct a network of countries based on information\ntheoretical metrics where the links capture similar vectorial patterns in the\ntopic distributions. Topology of the networks is then analyzed through network\nproperties like density, path length and clustering. Finally, we identify\nspecific topological features of our networks using the map equation framework\nto detect communities in our networks of countries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.09872v2",
            "date": "2017-07-24 10:27:33+00:00",
            "title": "Full-Network Embedding in a Multimodal Embedding Pipeline",
            "authors": [
                "Armand Vilalta",
                "Dario Garcia-Gasulla",
                "Ferran Par\u00e9s",
                "Eduard Ayguad\u00e9",
                "Jesus Labarta",
                "Ulises Cort\u00e9s",
                "Toyotaro Suzumura"
            ],
            "abstract": "The current state-of-the-art for image annotation and image retrieval tasks\nis obtained through deep neural networks, which combine an image representation\nand a text representation into a shared embedding space. In this paper we\nevaluate the impact of using the Full-Network embedding in this setting,\nreplacing the original image representation in a competitive multimodal\nembedding generation scheme. Unlike the one-layer image embeddings typically\nused by most approaches, the Full-Network embedding provides a multi-scale\nrepresentation of images, which results in richer characterizations. To measure\nthe influence of the Full-Network embedding, we evaluate its performance on\nthree different datasets, and compare the results with the original multimodal\nembedding generation scheme when using a one-layer image embedding, and with\nthe rest of the state-of-the-art. Results for image annotation and image\nretrieval tasks indicate that the Full-Network embedding is consistently\nsuperior to the one-layer embedding. These results motivate the integration of\nthe Full-Network embedding on any multimodal embedding generation scheme,\nsomething feasible thanks to the flexibility of the approach.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.00313v2",
            "date": "2017-11-01 12:38:59+00:00",
            "title": "Avoiding Your Teacher's Mistakes: Training Neural Networks with Controlled Weak Supervision",
            "authors": [
                "Mostafa Dehghani",
                "Aliaksei Severyn",
                "Sascha Rothe",
                "Jaap Kamps"
            ],
            "abstract": "Training deep neural networks requires massive amounts of training data, but\nfor many tasks only limited labeled data is available. This makes weak\nsupervision attractive, using weak or noisy signals like the output of\nheuristic methods or user click-through data for training. In a semi-supervised\nsetting, we can use a large set of data with weak labels to pretrain a neural\nnetwork and then fine-tune the parameters with a small amount of data with true\nlabels. This feels intuitively sub-optimal as these two independent stages\nleave the model unaware about the varying label quality. What if we could\nsomehow inform the model about the label quality? In this paper, we propose a\nsemi-supervised learning method where we train two neural networks in a\nmulti-task fashion: a \"target network\" and a \"confidence network\". The target\nnetwork is optimized to perform a given task and is trained using a large set\nof unlabeled data that are weakly annotated. We propose to weight the gradient\nupdates to the target network using the scores provided by the second\nconfidence network, which is trained on a small amount of supervised data. Thus\nwe avoid that the weight updates computed from noisy labels harm the quality of\nthe target network model. We evaluate our learning strategy on two different\ntasks: document ranking and sentiment classification. The results demonstrate\nthat our approach not only enhances the performance compared to the baselines\nbut also speeds up the learning process from weak labels.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.00350v3",
            "date": "2017-10-31 01:50:02+00:00",
            "title": "Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks",
            "authors": [
                "Brenden M. Lake",
                "Marco Baroni"
            ],
            "abstract": "Humans can understand and produce new utterances effortlessly, thanks to\ntheir compositional skills. Once a person learns the meaning of a new verb\n\"dax,\" he or she can immediately understand the meaning of \"dax twice\" or \"sing\nand dax.\" In this paper, we introduce the SCAN domain, consisting of a set of\nsimple compositional navigation commands paired with the corresponding action\nsequences. We then test the zero-shot generalization capabilities of a variety\nof recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence\nmethods. We find that RNNs can make successful zero-shot generalizations when\nthe differences between training and test commands are small, so that they can\napply \"mix-and-match\" strategies to solve the task. However, when\ngeneralization requires systematic compositional skills (as in the \"dax\"\nexample above), RNNs fail spectacularly. We conclude with a proof-of-concept\nexperiment in neural machine translation, suggesting that lack of systematicity\nmight be partially responsible for neural networks' notorious training data\nthirst.",
            "journal ref": "Lake, B. M. and Baroni, M. (2018). Generalization without\n  systematicity: On the compositional skills of sequence-to-sequence recurrent\n  networks. International Conference on Machine Learning (ICML)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.00549v4",
            "date": "2017-11-01 22:10:11+00:00",
            "title": "Just ASK: Building an Architecture for Extensible Self-Service Spoken Language Understanding",
            "authors": [
                "Anjishnu Kumar",
                "Arpit Gupta",
                "Julian Chan",
                "Sam Tucker",
                "Bjorn Hoffmeister",
                "Markus Dreyer",
                "Stanislav Peshterliev",
                "Ankur Gandhe",
                "Denis Filiminov",
                "Ariya Rastrow",
                "Christian Monson",
                "Agnika Kumar"
            ],
            "abstract": "This paper presents the design of the machine learning architecture that\nunderlies the Alexa Skills Kit (ASK) a large scale Spoken Language\nUnderstanding (SLU) Software Development Kit (SDK) that enables developers to\nextend the capabilities of Amazon's virtual assistant, Alexa. At Amazon, the\ninfrastructure powers over 25,000 skills deployed through the ASK, as well as\nAWS's Amazon Lex SLU Service. The ASK emphasizes flexibility, predictability\nand a rapid iteration cycle for third party developers. It imposes inductive\nbiases that allow it to learn robust SLU models from extremely small and sparse\ndatasets and, in doing so, removes significant barriers to entry for software\ndevelopers and dialogue systems researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01694v2",
            "date": "2017-11-06 01:55:45+00:00",
            "title": "Multilingual Speech Recognition With A Single End-To-End Model",
            "authors": [
                "Shubham Toshniwal",
                "Tara N. Sainath",
                "Ron J. Weiss",
                "Bo Li",
                "Pedro Moreno",
                "Eugene Weinstein",
                "Kanishka Rao"
            ],
            "abstract": "Training a conventional automatic speech recognition (ASR) system to support\nmultiple languages is challenging because the sub-word unit, lexicon and word\ninventories are typically language specific. In contrast, sequence-to-sequence\nmodels are well suited for multilingual ASR because they encapsulate an\nacoustic, pronunciation and language model jointly in a single network. In this\nwork we present a single sequence-to-sequence ASR model trained on 9 different\nIndian languages, which have very little overlap in their scripts.\nSpecifically, we take a union of language-specific grapheme sets and train a\ngrapheme-based sequence-to-sequence model jointly on data from all languages.\nWe find that this model, which is not explicitly given any information about\nlanguage identity, improves recognition performance by 21% relative compared to\nanalogous sequence-to-sequence models trained on each language individually. By\nmodifying the model to accept a language identifier as an additional input\nfeature, we further improve performance by an additional 7% relative and\neliminate confusion between different languages.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1711.02295v1",
            "date": "2017-11-07 05:43:34+00:00",
            "title": "Quality-Efficiency Trade-offs in Machine Learning for Text Processing",
            "authors": [
                "Ricardo Baeza-Yates",
                "Zeinab Liaghat"
            ],
            "abstract": "Data mining, machine learning, and natural language processing are powerful\ntechniques that can be used together to extract information from large texts.\nDepending on the task or problem at hand, there are many different approaches\nthat can be used. The methods available are continuously being optimized, but\nnot all these methods have been tested and compared in a set of problems that\ncan be solved using supervised machine learning algorithms. The question is\nwhat happens to the quality of the methods if we increase the training data\nsize from, say, 100 MB to over 1 GB? Moreover, are quality gains worth it when\nthe rate of data processing diminishes? Can we trade quality for time\nefficiency and recover the quality loss by just being able to process more\ndata? We attempt to answer these questions in a general way for text processing\ntasks, considering the trade-offs involving training data size, learning time,\nand quality obtained. We propose a performance trade-off framework and apply it\nto three important text processing problems: Named Entity Recognition,\nSentiment Analysis and Document Classification. These problems were also chosen\nbecause they have different levels of object granularity: words, paragraphs,\nand documents. For each problem, we selected several supervised machine\nlearning algorithms and we evaluated the trade-offs of them on large publicly\navailable data sets (news, reviews, patents). To explore these trade-offs, we\nuse different data subsets of increasing size ranging from 50 MB to several GB.\nWe also consider the impact of the data set and the evaluation technique. We\nfind that the results do not change significantly and that most of the time the\nbest algorithms is the fastest. However, we also show that the results for\nsmall data (say less than 100 MB) are different from the results for big data\nand in those cases the best algorithm is much harder to determine.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.02799v2",
            "date": "2017-11-08 02:05:11+00:00",
            "title": "Fidelity-Weighted Learning",
            "authors": [
                "Mostafa Dehghani",
                "Arash Mehrjou",
                "Stephan Gouws",
                "Jaap Kamps",
                "Bernhard Sch\u00f6lkopf"
            ],
            "abstract": "Training deep neural networks requires many training samples, but in practice\ntraining labels are expensive to obtain and may be of varying quality, as some\nmay be from trusted expert labelers while others might be from heuristics or\nother sources of weak supervision such as crowd-sourcing. This creates a\nfundamental quality versus-quantity trade-off in the learning process. Do we\nlearn from the small amount of high-quality data or the potentially large\namount of weakly-labeled data? We argue that if the learner could somehow know\nand take the label-quality into account when learning the data representation,\nwe could get the best of both worlds. To this end, we propose\n\"fidelity-weighted learning\" (FWL), a semi-supervised student-teacher approach\nfor training deep neural networks using weakly-labeled data. FWL modulates the\nparameter updates to a student network (trained on the task we care about) on a\nper-sample basis according to the posterior confidence of its label-quality\nestimated by a teacher (who has access to the high-quality labels). Both\nstudent and teacher are learned from the data. We evaluate FWL on two tasks in\ninformation retrieval and natural language processing where we outperform\nstate-of-the-art alternative semi-supervised methods, indicating that our\napproach makes better use of strong and weak labels, and leads to better\ntask-dependent data representations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.03147v1",
            "date": "2017-11-08 20:25:43+00:00",
            "title": "On the incorporation of interval-valued fuzzy sets into the Bousi-Prolog system: declarative semantics, implementation and applications",
            "authors": [
                "Clemente Rubio-Manzano",
                "Martin Pereira-Fari\u00f1a"
            ],
            "abstract": "In this paper we analyse the benefits of incorporating interval-valued fuzzy\nsets into the Bousi-Prolog system. A syntax, declarative semantics and im-\nplementation for this extension is presented and formalised. We show, by using\npotential applications, that fuzzy logic programming frameworks enhanced with\nthem can correctly work together with lexical resources and ontologies in order\nto improve their capabilities for knowledge representation and reasoning.",
            "journal ref": "Interactions Between Computational Intelligence and Mathematics\n  Studies in Computational Intelligence, vol 794. Springer 2018",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.03483v1",
            "date": "2017-11-09 17:28:07+00:00",
            "title": "Learning Multi-Modal Word Representation Grounded in Visual Context",
            "authors": [
                "\u00c9loi Zablocki",
                "Benjamin Piwowarski",
                "Laure Soulier",
                "Patrick Gallinari"
            ],
            "abstract": "Representing the semantics of words is a long-standing problem for the\nnatural language processing community. Most methods compute word semantics\ngiven their textual context in large corpora. More recently, researchers\nattempted to integrate perceptual and visual features. Most of these works\nconsider the visual appearance of objects to enhance word representations but\nthey ignore the visual environment and context in which objects appear. We\npropose to unify text-based techniques with vision-based techniques by\nsimultaneously leveraging textual and visual context to learn multimodal word\nembeddings. We explore various choices for what can serve as a visual context\nand present an end-to-end method to integrate visual context elements in a\nmultimodal skip-gram model. We provide experiments and extensive analysis of\nthe obtained results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03689v1",
            "date": "2017-11-10 04:42:44+00:00",
            "title": "Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection",
            "authors": [
                "Taku Kato",
                "Takahiro Shinozaki"
            ],
            "abstract": "Speech recognition systems have achieved high recognition performance for\nseveral tasks. However, the performance of such systems is dependent on the\ntremendously costly development work of preparing vast amounts of task-matched\ntranscribed speech data for supervised training. The key problem here is the\ncost of transcribing speech data. The cost is repeatedly required to support\nnew languages and new tasks. Assuming broad network services for transcribing\nspeech data for many users, a system would become more self-sufficient and more\nuseful if it possessed the ability to learn from very light feedback from the\nusers without annoying them. In this paper, we propose a general reinforcement\nlearning framework for speech recognition systems based on the policy gradient\nmethod. As a particular instance of the framework, we also propose a hypothesis\nselection-based reinforcement learning method. The proposed framework provides\na new view for several existing training and adaptation methods. The\nexperimental results show that the proposed method improves the recognition\nperformance compared to unsupervised adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03736v1",
            "date": "2017-11-10 09:17:02+00:00",
            "title": "Joint Sentiment/Topic Modeling on Text Data Using Boosted Restricted Boltzmann Machine",
            "authors": [
                "Masoud Fatemi",
                "Mehran Safayani"
            ],
            "abstract": "Recently by the development of the Internet and the Web, different types of\nsocial media such as web blogs become an immense source of text data. Through\nthe processing of these data, it is possible to discover practical information\nabout different topics, individuals opinions and a thorough understanding of\nthe society. Therefore, applying models which can automatically extract the\nsubjective information from the documents would be efficient and helpful. Topic\nmodeling methods, also sentiment analysis are the most raised topics in the\nnatural language processing and text mining fields. In this paper a new\nstructure for joint sentiment-topic modeling based on Restricted Boltzmann\nMachine (RBM) which is a type of neural networks is proposed. By modifying the\nstructure of RBM as well as appending a layer which is analogous to sentiment\nof text data to it, we propose a generative structure for joint sentiment topic\nmodeling based on neutral networks. The proposed method is supervised and\ntrained by the Contrastive Divergence algorithm. The new attached layer in the\nproposed model is a layer with the multinomial probability distribution which\ncan be used in text data sentiment classification or any other supervised\napplication. The proposed model is compared with existing models in the\nexperiments such as evaluating as a generative model, sentiment classification,\ninformation retrieval and the corresponding results demonstrate the efficiency\nof the method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.03800v2",
            "date": "2017-11-10 13:04:55+00:00",
            "title": "Object Referring in Visual Scene with Spoken Language",
            "authors": [
                "Arun Balajee Vasudevan",
                "Dengxin Dai",
                "Luc Van Gool"
            ],
            "abstract": "Object referring has important applications, especially for human-machine\ninteraction. While having received great attention, the task is mainly attacked\nwith written language (text) as input rather than spoken language (speech),\nwhich is more natural. This paper investigates Object Referring with Spoken\nLanguage (ORSpoken) by presenting two datasets and one novel approach. Objects\nare annotated with their locations in images, text descriptions and speech\ndescriptions. This makes the datasets ideal for multi-modality learning. The\napproach is developed by carefully taking down ORSpoken problem into three\nsub-problems and introducing task-specific vision-language interactions at the\ncorresponding levels. Experiments show that our method outperforms competing\nmethods consistently and significantly. The approach is also evaluated in the\npresence of audio noise, showing the efficacy of the proposed vision-language\ninteraction methods in counteracting background noise.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.03946v2",
            "date": "2017-11-10 18:09:15+00:00",
            "title": "Bayesian Paragraph Vectors",
            "authors": [
                "Geng Ji",
                "Robert Bamler",
                "Erik B. Sudderth",
                "Stephan Mandt"
            ],
            "abstract": "Word2vec (Mikolov et al., 2013) has proven to be successful in natural\nlanguage processing by capturing the semantic relationships between different\nwords. Built on top of single-word embeddings, paragraph vectors (Le and\nMikolov, 2014) find fixed-length representations for pieces of text with\narbitrary lengths, such as documents, paragraphs, and sentences. In this work,\nwe propose a novel interpretation for neural-network-based paragraph vectors by\ndeveloping an unsupervised generative model whose maximum likelihood solution\ncorresponds to traditional paragraph vectors. This probabilistic formulation\nallows us to go beyond point estimates of parameters and to perform Bayesian\nposterior inference. We find that the entropy of paragraph vectors decreases\nwith the length of documents, and that information about posterior uncertainty\nimproves performance in supervised learning tasks such as sentiment analysis\nand paraphrase detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04044v7",
            "date": "2017-11-10 23:42:42+00:00",
            "title": "Kernelized Hashcode Representations for Relation Extraction",
            "authors": [
                "Sahil Garg",
                "Aram Galstyan",
                "Greg Ver Steeg",
                "Irina Rish",
                "Guillermo Cecchi",
                "Shuyang Gao"
            ],
            "abstract": "Kernel methods have produced state-of-the-art results for a number of NLP\ntasks such as relation extraction, but suffer from poor scalability due to the\nhigh cost of computing kernel similarities between natural language structures.\nA recently proposed technique, kernelized locality-sensitive hashing (KLSH),\ncan significantly reduce the computational cost, but is only applicable to\nclassifiers operating on kNN graphs. Here we propose to use random subspaces of\nKLSH codes for efficiently constructing an explicit representation of NLP\nstructures suitable for general classification methods. Further, we propose an\napproach for optimizing the KLSH model for classification problems by\nmaximizing an approximation of mutual information between the KLSH codes\n(feature vectors) and the class labels. We evaluate the proposed approach on\nbiomedical relation extraction datasets, and observe significant and robust\nimprovements in accuracy w.r.t. state-of-the-art classifiers, along with\ndrastic (orders-of-magnitude) speedup compared to conventional kernel methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04168v3",
            "date": "2017-11-11 16:43:38+00:00",
            "title": "Unsupervised Document Embedding With CNNs",
            "authors": [
                "Chundi Liu",
                "Shunan Zhao",
                "Maksims Volkovs"
            ],
            "abstract": "We propose a new model for unsupervised document embedding. Leading existing\napproaches either require complex inference or use recurrent neural networks\n(RNN) that are difficult to parallelize. We take a different route and develop\na convolutional neural network (CNN) embedding model. Our CNN architecture is\nfully parallelizable resulting in over 10x speedup in inference time over RNN\nmodels. Parallelizable architecture enables to train deeper models where each\nsuccessive layer has increasingly larger receptive field and models longer\nrange semantic structure within the document. We additionally propose a fully\nunsupervised learning algorithm to train this model based on stochastic forward\nprediction. Empirical results on two public benchmarks show that our approach\nproduces comparable to state-of-the-art accuracy at a fraction of computational\ncost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04436v1",
            "date": "2017-11-13 06:41:29+00:00",
            "title": "SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning",
            "authors": [
                "Xiaojun Xu",
                "Chang Liu",
                "Dawn Song"
            ],
            "abstract": "Synthesizing SQL queries from natural language is a long-standing open\nproblem and has been attracting considerable interest recently. Toward solving\nthe problem, the de facto approach is to employ a sequence-to-sequence-style\nmodel. Such an approach will necessarily require the SQL queries to be\nserialized. Since the same SQL query may have multiple equivalent\nserializations, training a sequence-to-sequence-style model is sensitive to the\nchoice from one of them. This phenomenon is documented as the \"order-matters\"\nproblem. Existing state-of-the-art approaches rely on reinforcement learning to\nreward the decoder when it generates any of the equivalent serializations.\nHowever, we observe that the improvement from reinforcement learning is\nlimited.\n  In this paper, we propose a novel approach, i.e., SQLNet, to fundamentally\nsolve this problem by avoiding the sequence-to-sequence structure when the\norder does not matter. In particular, we employ a sketch-based approach where\nthe sketch contains a dependency graph so that one prediction can be done by\ntaking into consideration only the previous predictions that it depends on. In\naddition, we propose a sequence-to-set model as well as the column attention\nmechanism to synthesize the query based on the sketch. By combining all these\nnovel techniques, we show that SQLNet can outperform the prior art by 9% to 13%\non the WikiSQL task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04452v1",
            "date": "2017-11-13 07:31:24+00:00",
            "title": "Digitising Cultural Complexity: Representing Rich Cultural Data in a Big Data environment",
            "authors": [
                "Jennifer Edmond",
                "Georgina Nugent Folan"
            ],
            "abstract": "One of the major terminological forces driving ICT integration in research\ntoday is that of \"big data.\" While the phrase sounds inclusive and integrative,\n\"big data\" approaches are highly selective, excluding input that cannot be\neffectively structured, represented, or digitised. Data of this complex sort is\nprecisely the kind that human activity produces, but the technological\nimperative to enhance signal through the reduction of noise does not\naccommodate this richness. Data and the computational approaches that\nfacilitate \"big data\" have acquired a perceived objectivity that belies their\ncurated, malleable, reactive, and performative nature. In an input environment\nwhere anything can \"be data\" once it is entered into the system as \"data,\" data\ncleaning and processing, together with the metadata and information\narchitectures that structure and facilitate our cultural archives acquire a\ncapacity to delimit what data are. This engenders a process of simplification\nthat has major implications for the potential for future innovation within\nresearch environments that depend on rich material yet are increasingly\nmediated by digital technologies. This paper presents the preliminary findings\nof the European-funded KPLEX (Knowledge Complexity) project which investigates\nthe delimiting effect digital mediation and datafication has on rich, complex\ncultural data. The paper presents a systematic review of existing implicit\ndefinitions of data, elaborating on the implications of these definitions and\nhighlighting the ways in which metadata and computational technologies can\nrestrict the interpretative potential of data. It sheds light on the gap\nbetween analogue or augmented digital practices and fully computational ones,\nand the strategies researchers have developed to deal with this gap. The paper\nproposes a reconceptualisation of data as it is functionally employed within\ndigitally-mediated research so as to incorporate and acknowledge the richness\nand complexity of our source materials.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.04498v1",
            "date": "2017-11-13 10:06:22+00:00",
            "title": "Targeted Advertising Based on Browsing History",
            "authors": [
                "Yong Zhang",
                "Hongming Zhou",
                "Nganmeng Tan",
                "Saeed Bagheri",
                "Meng Joo Er"
            ],
            "abstract": "Audience interest, demography, purchase behavior and other possible\nclassifications are ex- tremely important factors to be carefully studied in a\ntargeting campaign. This information can help advertisers and publishers\ndeliver advertisements to the right audience group. How- ever, it is not easy\nto collect such information, especially for the online audience with whom we\nhave limited interaction and minimum deterministic knowledge. In this paper, we\npro- pose a predictive framework that can estimate online audience demographic\nattributes based on their browsing histories. Under the proposed framework,\nfirst, we retrieve the content of the websites visited by audience, and\nrepresent the content as website feature vectors; second, we aggregate the\nvectors of websites that audience have visited and arrive at feature vectors\nrepresenting the users; finally, the support vector machine is exploited to\npredict the audience demographic attributes. The key to achieving good\nprediction performance is preparing representative features of the audience.\nWord Embedding, a widely used tech- nique in natural language processing tasks,\ntogether with term frequency-inverse document frequency weighting scheme is\nused in the proposed method. This new representation ap- proach is unsupervised\nand very easy to implement. The experimental results demonstrate that the new\naudience feature representation method is more powerful than existing baseline\nmethods, leading to a great improvement in prediction accuracy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.04564v1",
            "date": "2017-11-13 13:13:40+00:00",
            "title": "Phonemic and Graphemic Multilingual CTC Based Speech Recognition",
            "authors": [
                "Markus M\u00fcller",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "Training automatic speech recognition (ASR) systems requires large amounts of\ndata in the target language in order to achieve good performance. Whereas large\ntraining corpora are readily available for languages like English, there exists\na long tail of languages which do suffer from a lack of resources. One method\nto handle data sparsity is to use data from additional source languages and\nbuild a multilingual system. Recently, ASR systems based on recurrent neural\nnetworks (RNNs) trained with connectionist temporal classification (CTC) have\ngained substantial research interest. In this work, we extended our previous\napproach towards training CTC-based systems multilingually. Our systems feature\na global phone set, based on the joint phone sets of each source language. We\nevaluated the use of different language combinations as well as the addition of\nLanguage Feature Vectors (LFVs). As contrastive experiment, we built systems\nbased on graphemes as well. Systems having a multilingual phone set are known\nto suffer in performance compared to their monolingual counterparts. With our\nproposed approach, we could reduce the gap between these mono- and multilingual\nsetups, using either graphemes or phonemes.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1711.04569v2",
            "date": "2017-11-13 13:22:54+00:00",
            "title": "Multilingual Adaptation of RNN Based ASR Systems",
            "authors": [
                "Markus M\u00fcller",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "In this work, we focus on multilingual systems based on recurrent neural\nnetworks (RNNs), trained using the Connectionist Temporal Classification (CTC)\nloss function. Using a multilingual set of acoustic units poses difficulties.\nTo address this issue, we proposed Language Feature Vectors (LFVs) to train\nlanguage adaptive multilingual systems. Language adaptation, in contrast to\nspeaker adaptation, needs to be applied not only on the feature level, but also\nto deeper layers of the network. In this work, we therefore extended our\nprevious approach by introducing a novel technique which we call \"modulation\".\nBased on this method, we modulated the hidden layers of RNNs using LFVs. We\nevaluated this approach in both full and low resource conditions, as well as\nfor grapheme and phone based systems. Lower error rates throughout the\ndifferent conditions could be achieved by the use of the modulation.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1711.05170v1",
            "date": "2017-11-14 16:19:34+00:00",
            "title": "On Extending Neural Networks with Loss Ensembles for Text Classification",
            "authors": [
                "Hamideh Hajiabadi",
                "Diego Molla-Aliod",
                "Reza Monsefi"
            ],
            "abstract": "Ensemble techniques are powerful approaches that combine several weak\nlearners to build a stronger one. As a meta learning framework, ensemble\ntechniques can easily be applied to many machine learning techniques. In this\npaper we propose a neural network extended with an ensemble loss function for\ntext classification. The weight of each weak loss function is tuned within the\ntraining phase through the gradient propagation optimization method of the\nneural network. The approach is evaluated on several text classification\ndatasets. We also evaluate its performance in various environments with several\ndegrees of label noise. Experimental results indicate an improvement of the\nresults and strong resilience against label noise in comparison with other\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05240v5",
            "date": "2017-11-14 18:29:05+00:00",
            "title": "Weakly-supervised Semantic Parsing with Abstract Examples",
            "authors": [
                "Omer Goldman",
                "Veronica Latcinnik",
                "Udi Naveh",
                "Amir Globerson",
                "Jonathan Berant"
            ],
            "abstract": "Training semantic parsers from weak supervision (denotations) rather than\nstrong supervision (programs) complicates training in two ways. First, a large\nsearch space of potential programs needs to be explored at training time to\nfind a correct program. Second, spurious programs that accidentally lead to a\ncorrect denotation add noise to training. In this work we propose that in\nclosed worlds with clear semantic types, one can substantially alleviate these\nproblems by utilizing an abstract representation, where tokens in both the\nlanguage utterance and program are lifted to an abstract form. We show that\nthese abstractions can be defined with a handful of lexical rules and that they\nresult in sharing between different examples that alleviates the difficulties\nin training. To test our approach, we develop the first semantic parser for\nCNLVR, a challenging visual reasoning dataset, where the search space is large\nand overcoming spuriousness is critical, because denotations are either TRUE or\nFALSE, and thus random programs are likely to lead to a correct denotation. Our\nmethod substantially improves performance, and reaches 82.5% accuracy, a 14.7%\nabsolute accuracy improvement compared to the best reported accuracy so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05408v2",
            "date": "2017-11-15 04:54:08+00:00",
            "title": "Recurrent Neural Networks as Weighted Language Recognizers",
            "authors": [
                "Yining Chen",
                "Sorcha Gilroy",
                "Andreas Maletti",
                "Jonathan May",
                "Kevin Knight"
            ],
            "abstract": "We investigate the computational complexity of various problems for simple\nrecurrent neural networks (RNNs) as formal models for recognizing weighted\nlanguages. We focus on the single-layer, ReLU-activation, rational-weight RNNs\nwith softmax, which are commonly used in natural language processing\napplications. We show that most problems for such RNNs are undecidable,\nincluding consistency, equivalence, minimization, and the determination of the\nhighest-weighted string. However, for consistent RNNs the last problem becomes\ndecidable, although the solution length can surpass all computable bounds. If\nadditionally the string is limited to polynomial length, the problem becomes\nNP-complete and APX-hard. In summary, this shows that approximations and\nheuristic algorithms are necessary in practical applications of those RNNs.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05443v3",
            "date": "2017-11-15 08:21:20+00:00",
            "title": "Human and Machine Speaker Recognition Based on Short Trivial Events",
            "authors": [
                "Miao Zhang",
                "Xiaofei Kang",
                "Yanqing Wang",
                "Lantian Li",
                "Zhiyuan Tang",
                "Haisheng Dai",
                "Dong Wang"
            ],
            "abstract": "Trivial events are ubiquitous in human to human conversations, e.g., cough,\nlaugh and sniff. Compared to regular speech, these trivial events are usually\nshort and unclear, thus generally regarded as not speaker discriminative and so\nare largely ignored by present speaker recognition research. However, these\ntrivial events are highly valuable in some particular circumstances such as\nforensic examination, as they are less subjected to intentional change, so can\nbe used to discover the genuine speaker from disguised speech. In this paper,\nwe collect a trivial event speech database that involves 75 speakers and 6\ntypes of events, and report preliminary speaker recognition results on this\ndatabase, by both human listeners and machines. Particularly, the deep feature\nlearning technique recently proposed by our group is utilized to analyze and\nrecognize the trivial events, which leads to acceptable equal error rates\n(EERs) despite the extremely short durations (0.2-0.5 seconds) of these events.\nComparing different types of events, 'hmm' seems more speaker discriminative.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1711.05447v2",
            "date": "2017-11-15 08:27:35+00:00",
            "title": "Emotional End-to-End Neural Speech Synthesizer",
            "authors": [
                "Younggun Lee",
                "Azam Rabiee",
                "Soo-Young Lee"
            ],
            "abstract": "In this paper, we introduce an emotional speech synthesizer based on the\nrecent end-to-end neural model, named Tacotron. Despite its benefits, we found\nthat the original Tacotron suffers from the exposure bias problem and\nirregularity of the attention alignment. Later, we address the problem by\nutilization of context vector and residual connection at recurrent neural\nnetworks (RNNs). Our experiments showed that the model could successfully train\nand generate speech for given emotion labels.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1711.05448v1",
            "date": "2017-11-15 08:30:56+00:00",
            "title": "Lattice Rescoring Strategies for Long Short Term Memory Language Models in Speech Recognition",
            "authors": [
                "Shankar Kumar",
                "Michael Nirschl",
                "Daniel Holtmann-Rice",
                "Hank Liao",
                "Ananda Theertha Suresh",
                "Felix Yu"
            ],
            "abstract": "Recurrent neural network (RNN) language models (LMs) and Long Short Term\nMemory (LSTM) LMs, a variant of RNN LMs, have been shown to outperform\ntraditional N-gram LMs on speech recognition tasks. However, these models are\ncomputationally more expensive than N-gram LMs for decoding, and thus,\nchallenging to integrate into speech recognizers. Recent research has proposed\nthe use of lattice-rescoring algorithms using RNNLMs and LSTMLMs as an\nefficient strategy to integrate these models into a speech recognition system.\nIn this paper, we evaluate existing lattice rescoring algorithms along with new\nvariants on a YouTube speech recognition task. Lattice rescoring using LSTMLMs\nreduces the word error rate (WER) for this task by 8\\% relative to the WER\nobtained using an N-gram LM.",
            "journal ref": "Proceedings of ASRU 2017",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.05557v1",
            "date": "2017-11-11 10:48:59+00:00",
            "title": "Phrase-based Image Captioning with Hierarchical LSTM Model",
            "authors": [
                "Ying Hua Tan",
                "Chee Seng Chan"
            ],
            "abstract": "Automatic generation of caption to describe the content of an image has been\ngaining a lot of research interests recently, where most of the existing works\ntreat the image caption as pure sequential data. Natural language, however\npossess a temporal hierarchy structure, with complex dependencies between each\nsubsequence. In this paper, we propose a phrase-based hierarchical Long\nShort-Term Memory (phi-LSTM) model to generate image description. In contrast\nto the conventional solutions that generate caption in a pure sequential\nmanner, our proposed model decodes image caption from phrase to sentence. It\nconsists of a phrase decoder at the bottom hierarchy to decode noun phrases of\nvariable length, and an abbreviated sentence decoder at the upper hierarchy to\ndecode an abbreviated form of the image description. A complete image caption\nis formed by combining the generated phrases with sentence during the inference\nstage. Empirically, our proposed model shows a better or competitive result on\nthe Flickr8k, Flickr30k and MS-COCO datasets in comparison to the state-of-the\nart models. We also show that our proposed model is able to generate more novel\ncaptions (not seen in the training data) which are richer in word contents in\nall these three datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.05626v2",
            "date": "2017-11-15 15:33:59+00:00",
            "title": "Deep Temporal-Recurrent-Replicated-Softmax for Topical Trends over Time",
            "authors": [
                "Pankaj Gupta",
                "Subburam Rajaram",
                "Hinrich Sch\u00fctze",
                "Bernt Andrassy"
            ],
            "abstract": "Dynamic topic modeling facilitates the identification of topical trends over\ntime in temporal collections of unstructured documents. We introduce a novel\nunsupervised neural dynamic topic model named as Recurrent Neural\nNetwork-Replicated Softmax Model (RNNRSM), where the discovered topics at each\ntime influence the topic discovery in the subsequent time steps. We account for\nthe temporal ordering of documents by explicitly modeling a joint distribution\nof latent topical dependencies over time, using distributional estimators with\ntemporal recurrent connections. Applying RNN-RSM to 19 years of articles on NLP\nresearch, we demonstrate that compared to state-of-the art topic models, RNNRSM\nshows better generalization, topic interpretation, evolution and trends. We\nalso introduce a metric (named as SPAN) to quantify the capability of dynamic\ntopic model to capture word evolution in topics over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.05715v2",
            "date": "2017-11-15 18:23:48+00:00",
            "title": "BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems",
            "authors": [
                "Zachary Lipton",
                "Xiujun Li",
                "Jianfeng Gao",
                "Lihong Li",
                "Faisal Ahmed",
                "Li Deng"
            ],
            "abstract": "We present a new algorithm that significantly improves the efficiency of\nexploration for deep Q-learning agents in dialogue systems. Our agents explore\nvia Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop\nneural network. Our algorithm learns much faster than common exploration\nstrategies such as \\epsilon-greedy, Boltzmann, bootstrapping, and\nintrinsic-reward-based ones. Additionally, we show that spiking the replay\nbuffer with experiences from just a few successful episodes can make Q-learning\nfeasible when it might otherwise fail.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.06004v1",
            "date": "2017-11-16 09:50:52+00:00",
            "title": "Remedies against the Vocabulary Gap in Information Retrieval",
            "authors": [
                "Christophe Van Gysel"
            ],
            "abstract": "Search engines rely heavily on term-based approaches that represent queries\nand documents as bags of words. Text---a document or a query---is represented\nby a bag of its words that ignores grammar and word order, but retains word\nfrequency counts. When presented with a search query, the engine then ranks\ndocuments according to their relevance scores by computing, among other things,\nthe matching degrees between query and document terms. While term-based\napproaches are intuitive and effective in practice, they are based on the\nhypothesis that documents that exactly contain the query terms are highly\nrelevant regardless of query semantics. Inversely, term-based approaches assume\ndocuments that do not contain query terms as irrelevant. However, it is known\nthat a high matching degree at the term level does not necessarily mean high\nrelevance and, vice versa, documents that match null query terms may still be\nrelevant. Consequently, there exists a vocabulary gap between queries and\ndocuments that occurs when both use different words to describe the same\nconcepts. It is the alleviation of the effect brought forward by this\nvocabulary gap that is the topic of this dissertation. More specifically, we\npropose (1) methods to formulate an effective query from complex textual\nstructures and (2) latent vector space models that circumvent the vocabulary\ngap in information retrieval.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.06288v2",
            "date": "2017-11-16 19:10:21+00:00",
            "title": "Language-Based Image Editing with Recurrent Attentive Models",
            "authors": [
                "Jianbo Chen",
                "Yelong Shen",
                "Jianfeng Gao",
                "Jingjing Liu",
                "Xiaodong Liu"
            ],
            "abstract": "We investigate the problem of Language-Based Image Editing (LBIE). Given a\nsource image and a natural language description, we want to generate a target\nimage by editing the source image based on the description. We propose a\ngeneric modeling framework for two sub-tasks of LBIE: language-based image\nsegmentation and image colorization. The framework uses recurrent attentive\nmodels to fuse image and language features. Instead of using a fixed step size,\nwe introduce for each region of the image a termination gate to dynamically\ndetermine after each inference step whether to continue extrapolating\nadditional information from the textual description. The effectiveness of the\nframework is validated on three datasets. First, we introduce a synthetic\ndataset, called CoSaL, to evaluate the end-to-end performance of our LBIE\nsystem. Second, we show that the framework leads to state-of-the-art\nperformance on image segmentation on the ReferIt dataset. Third, we present the\nfirst language-based colorization result on the Oxford-102 Flowers dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.06351v1",
            "date": "2017-11-16 23:27:04+00:00",
            "title": "Question Asking as Program Generation",
            "authors": [
                "Anselm Rothe",
                "Brenden M. Lake",
                "Todd M. Gureckis"
            ],
            "abstract": "A hallmark of human intelligence is the ability to ask rich, creative, and\nrevealing questions. Here we introduce a cognitive model capable of\nconstructing human-like questions. Our approach treats questions as formal\nprograms that, when executed on the state of the world, output an answer. The\nmodel specifies a probability distribution over a complex, compositional space\nof programs, favoring concise programs that help the agent learn in the current\ncontext. We evaluate our approach by modeling the types of open-ended questions\ngenerated by humans who were attempting to learn about an ambiguous situation\nin a game. We find that our model predicts what questions people will ask, and\ncan creatively produce novel questions that were not present in the training\nset. In addition, we compare a number of model variants, finding that both\nquestion informativeness and complexity are important for producing human-like\nquestions.",
            "journal ref": "Rothe, A., Lake, B. M., and Gureckis, T. M. (2017). Question\n  asking as program generation. Advances in Neural Information Processing\n  Systems 30",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.06794v2",
            "date": "2017-11-18 02:07:34+00:00",
            "title": "Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering",
            "authors": [
                "Pan Lu",
                "Hongsheng Li",
                "Wei Zhang",
                "Jianyong Wang",
                "Xiaogang Wang"
            ],
            "abstract": "Recently, the Visual Question Answering (VQA) task has gained increasing\nattention in artificial intelligence. Existing VQA methods mainly adopt the\nvisual attention mechanism to associate the input question with corresponding\nimage regions for effective question answering. The free-form region based and\nthe detection-based visual attention mechanisms are mostly investigated, with\nthe former ones attending free-form image regions and the latter ones attending\npre-specified detection-box regions. We argue that the two attention mechanisms\nare able to provide complementary information and should be effectively\nintegrated to better solve the VQA problem. In this paper, we propose a novel\ndeep neural network for VQA that integrates both attention mechanisms. Our\nproposed framework effectively fuses features from free-form image regions,\ndetection boxes, and question representations via a multi-modal multiplicative\nfeature embedding scheme to jointly attend question-related free-form image\nregions and detection boxes for more accurate question answering. The proposed\nmethod is extensively evaluated on two publicly available datasets, COCO-QA and\nVQA, and outperforms state-of-the-art approaches. Source code is available at\nhttps://github.com/lupantech/dual-mfa-vqa.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.06821v3",
            "date": "2017-11-18 07:00:44+00:00",
            "title": "Acquiring Common Sense Spatial Knowledge through Implicit Spatial Templates",
            "authors": [
                "Guillem Collell",
                "Luc Van Gool",
                "Marie-Francine Moens"
            ],
            "abstract": "Spatial understanding is a fundamental problem with wide-reaching real-world\napplications. The representation of spatial knowledge is often modeled with\nspatial templates, i.e., regions of acceptability of two objects under an\nexplicit spatial relationship (e.g., \"on\", \"below\", etc.). In contrast with\nprior work that restricts spatial templates to explicit spatial prepositions\n(e.g., \"glass on table\"), here we extend this concept to implicit spatial\nlanguage, i.e., those relationships (generally actions) for which the spatial\narrangement of the objects is only implicitly implied (e.g., \"man riding\nhorse\"). In contrast with explicit relationships, predicting spatial\narrangements from implicit spatial language requires significant common sense\nspatial understanding. Here, we introduce the task of predicting spatial\ntemplates for two objects under a relationship, which can be seen as a spatial\nquestion-answering task with a (2D) continuous output (\"where is the man w.r.t.\na horse when the man is walking the horse?\"). We present two simple\nneural-based models that leverage annotated images and structured text to learn\nthis task. The good performance of these models reveals that spatial locations\nare to a large extent predictable from implicit spatial language. Crucially,\nthe models attain similar performance in a challenging generalized setting,\nwhere the object-relation-object combinations (e.g.,\"man walking dog\") have\nnever been seen before. Next, we go one step further by presenting the models\nwith unseen objects (e.g., \"dog\"). In this scenario, we show that leveraging\nword embeddings enables the models to output accurate spatial predictions,\nproving that the models acquire solid common sense spatial knowledge allowing\nfor such generalization.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1711.06899v1",
            "date": "2017-11-18 17:10:25+00:00",
            "title": "The Cultural Evolution of National Constitutions",
            "authors": [
                "Daniel N. Rockmore",
                "Chen Fang",
                "Nicholas J. Foti",
                "Tom Ginsburg",
                "David C. Krakauer"
            ],
            "abstract": "We explore how ideas from infectious disease and genetics can be used to\nuncover patterns of cultural inheritance and innovation in a corpus of 591\nnational constitutions spanning 1789 - 2008. Legal \"Ideas\" are encoded as\n\"topics\" - words statistically linked in documents - derived from topic\nmodeling the corpus of constitutions. Using these topics we derive a diffusion\nnetwork for borrowing from ancestral constitutions back to the US Constitution\nof 1789 and reveal that constitutions are complex cultural recombinants. We\nfind systematic variation in patterns of borrowing from ancestral texts and\n\"biological\"-like behavior in patterns of inheritance with the distribution of\n\"offspring\" arising through a bounded preferential-attachment process. This\nprocess leads to a small number of highly innovative (influential)\nconstitutions some of which have yet to have been identified as so in the\ncurrent literature. Our findings thus shed new light on the critical nodes of\nthe constitution-making network. The constitutional network structure reflects\nperiods of intense constitution creation, and systematic patterns of variation\nin constitutional life-span and temporal influence.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1711.07065v1",
            "date": "2017-11-19 19:56:23+00:00",
            "title": "Prior-aware Dual Decomposition: Document-specific Topic Inference for Spectral Topic Models",
            "authors": [
                "Moontae Lee",
                "David Bindel",
                "David Mimno"
            ],
            "abstract": "Spectral topic modeling algorithms operate on matrices/tensors of word\nco-occurrence statistics to learn topic-specific word distributions. This\napproach removes the dependence on the original documents and produces\nsubstantial gains in efficiency and provable topic inference, but at a cost:\nthe model can no longer provide information about the topic composition of\nindividual documents. Recently Thresholded Linear Inverse (TLI) is proposed to\nmap the observed words of each document back to its topic composition. However,\nits linear characteristics limit the inference quality without considering the\nimportant prior information over topics. In this paper, we evaluate Simple\nProbabilistic Inverse (SPI) method and novel Prior-aware Dual Decomposition\n(PADD) that is capable of learning document-specific topic compositions in\nparallel. Experiments show that PADD successfully leverages topic correlations\nas a prior, notably outperforming TLI and learning quality topic compositions\ncomparable to Gibbs sampling on various data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07274v2",
            "date": "2017-11-20 12:07:22+00:00",
            "title": "Speech recognition for medical conversations",
            "authors": [
                "Chung-Cheng Chiu",
                "Anshuman Tripathi",
                "Katherine Chou",
                "Chris Co",
                "Navdeep Jaitly",
                "Diana Jaunzeikare",
                "Anjuli Kannan",
                "Patrick Nguyen",
                "Hasim Sak",
                "Ananth Sankar",
                "Justin Tansuwan",
                "Nathan Wan",
                "Yonghui Wu",
                "Xuedong Zhang"
            ],
            "abstract": "In this work we explored building automatic speech recognition models for\ntranscribing doctor patient conversation. We collected a large scale dataset of\nclinical conversations ($14,000$ hr), designed the task to represent the real\nword scenario, and explored several alignment approaches to iteratively improve\ndata quality. We explored both CTC and LAS systems for building speech\nrecognition models. The LAS was more resilient to noisy data and CTC required\nmore data clean up. A detailed analysis is provided for understanding the\nperformance for clinical tasks. Our analysis showed the speech recognition\nmodels performed well on important medical utterances, while errors occurred in\ncausal conversations. Overall we believe the resulting models can provide\nreasonable quality in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07280v3",
            "date": "2017-11-20 12:17:47+00:00",
            "title": "Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments",
            "authors": [
                "Peter Anderson",
                "Qi Wu",
                "Damien Teney",
                "Jake Bruce",
                "Mark Johnson",
                "Niko S\u00fcnderhauf",
                "Ian Reid",
                "Stephen Gould",
                "Anton van den Hengel"
            ],
            "abstract": "A robot that can carry out a natural-language instruction has been a dream\nsince before the Jetsons cartoon series imagined a life of leisure mediated by\na fleet of attentive robot helpers. It is a dream that remains stubbornly\ndistant. However, recent advances in vision and language methods have made\nincredible progress in closely related areas. This is significant because a\nrobot interpreting a natural-language navigation instruction on the basis of\nwhat it sees is carrying out a vision and language process that is similar to\nVisual Question Answering. Both tasks can be interpreted as visually grounded\nsequence-to-sequence translation problems, and many of the same methods are\napplicable. To enable and encourage the application of vision and language\nmethods to the problem of interpreting visually-grounded navigation\ninstructions, we present the Matterport3D Simulator -- a large-scale\nreinforcement learning environment based on real imagery. Using this simulator,\nwhich can in future support a range of embodied vision and language tasks, we\nprovide the first benchmark dataset for visually-grounded natural language\nnavigation in real buildings -- the Room-to-Room (R2R) dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.07613v1",
            "date": "2017-11-21 03:11:49+00:00",
            "title": "Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning",
            "authors": [
                "Qi Wu",
                "Peng Wang",
                "Chunhua Shen",
                "Ian Reid",
                "Anton van den Hengel"
            ],
            "abstract": "The Visual Dialogue task requires an agent to engage in a conversation about\nan image with a human. It represents an extension of the Visual Question\nAnswering task in that the agent needs to answer a question about an image, but\nit needs to do so in light of the previous dialogue that has taken place. The\nkey challenge in Visual Dialogue is thus maintaining a consistent, and natural\ndialogue while continuing to answer questions correctly. We present a novel\napproach that combines Reinforcement Learning and Generative Adversarial\nNetworks (GANs) to generate more human-like responses to questions. The GAN\nhelps overcome the relative paucity of training data, and the tendency of the\ntypical MLE-based approach to generate overly terse answers. Critically, the\nGAN is tightly integrated into the attention mechanism that generates\nhuman-interpretable reasons for each answer. This means that the discriminative\nmodel of the GAN has the task of assessing whether a candidate answer is\ngenerated by a human or not, given the provided reason. This is significant\nbecause it drives the generative model to produce high quality answers that are\nwell supported by the associated reasoning. The method also generates the\nstate-of-the-art results on the primary benchmark.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.07614v1",
            "date": "2017-11-21 03:15:30+00:00",
            "title": "Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards",
            "authors": [
                "Junjie Zhang",
                "Qi Wu",
                "Chunhua Shen",
                "Jian Zhang",
                "Jianfeng Lu",
                "Anton van den Hengel"
            ],
            "abstract": "Despite significant progress in a variety of vision-and-language problems,\ndeveloping a method capable of asking intelligent, goal-oriented questions\nabout images is proven to be an inscrutable challenge. Towards this end, we\npropose a Deep Reinforcement Learning framework based on three new intermediate\nrewards, namely goal-achieved, progressive and informativeness that encourage\nthe generation of succinct questions, which in turn uncover valuable\ninformation towards the overall goal. By directly optimizing for questions that\nwork quickly towards fulfilling the overall goal, we avoid the tendency of\nexisting methods to generate long series of insane queries that add little\nvalue. We evaluate our model on the GuessWhat?! dataset and show that the\nresulting questions can help a standard Guesser identify a specific object in\nan image at a much higher success rate.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.07632v4",
            "date": "2017-11-21 04:40:38+00:00",
            "title": "Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders",
            "authors": [
                "Xiaopeng Yang",
                "Xiaowen Lin",
                "Shunda Suo",
                "Ming Li"
            ],
            "abstract": "Computer poetry generation is our first step towards computer writing.\nWriting must have a theme. The current approaches of using sequence-to-sequence\nmodels with attention often produce non-thematic poems. We present a novel\nconditional variational autoencoder with a hybrid decoder adding the\ndeconvolutional neural networks to the general recurrent neural networks to\nfully learn topic information via latent variables. This approach significantly\nimproves the relevance of the generated poems by representing each line of the\npoem not only in a context-sensitive manner but also in a holistic way that is\nhighly related to the given keyword and the learned topic. A proposed augmented\nword2vec model further improves the rhythm and symmetry. Tests show that the\ngenerated poems by our approach are mostly satisfying with regulated rules and\nconsistent themes, and 73.42% of them receive an Overall score no less than 3\n(the highest score is 5).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07656v1",
            "date": "2017-11-21 07:26:39+00:00",
            "title": "Cross Temporal Recurrent Networks for Ranking Question Answer Pairs",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Temporal gates play a significant role in modern recurrent-based neural\nencoders, enabling fine-grained control over recursive compositional operations\nover time. In recurrent models such as the long short-term memory (LSTM),\ntemporal gates control the amount of information retained or discarded over\ntime, not only playing an important role in influencing the learned\nrepresentations but also serving as a protection against vanishing gradients.\nThis paper explores the idea of learning temporal gates for sequence pairs\n(question and answer), jointly influencing the learned representations in a\npairwise manner. In our approach, temporal gates are learned via 1D\nconvolutional layers and then subsequently cross applied across question and\nanswer for joint learning. Empirically, we show that this conceptually simple\nsharing of temporal gates can lead to competitive performance across multiple\nbenchmarks. Intuitively, what our network achieves can be interpreted as\nlearning representations of question and answer pairs that are aware of what\neach other is remembering or forgetting, i.e., pairwise temporal gating. Via\nextensive experiments, we show that our proposed model achieves\nstate-of-the-art performance on two community-based QA datasets and competitive\nperformance on one factoid-based QA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.07798v1",
            "date": "2017-11-21 14:19:48+00:00",
            "title": "Visual and Textual Sentiment Analysis Using Deep Fusion Convolutional Neural Networks",
            "authors": [
                "Xingyue Chen",
                "Yunhong Wang",
                "Qingjie Liu"
            ],
            "abstract": "Sentiment analysis is attracting more and more attentions and has become a\nvery hot research topic due to its potential applications in personalized\nrecommendation, opinion mining, etc. Most of the existing methods are based on\neither textual or visual data and can not achieve satisfactory results, as it\nis very hard to extract sufficient information from only one single modality\ndata. Inspired by the observation that there exists strong semantic correlation\nbetween visual and textual data in social medias, we propose an end-to-end deep\nfusion convolutional neural network to jointly learn textual and visual\nsentiment representations from training examples. The two modality information\nare fused together in a pooling layer and fed into fully-connected layers to\npredict the sentiment polarity. We evaluate the proposed approach on two widely\nused data sets. Results show that our method achieves promising result compared\nwith the state-of-the-art methods which clearly demonstrate its competency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08010v2",
            "date": "2017-11-21 19:44:37+00:00",
            "title": "Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition",
            "authors": [
                "Zhong Meng",
                "Zhuo Chen",
                "Vadim Mazalov",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "Unsupervised domain adaptation of speech signal aims at adapting a\nwell-trained source-domain acoustic model to the unlabeled data from target\ndomain. This can be achieved by adversarial training of deep neural network\n(DNN) acoustic models to learn an intermediate deep representation that is both\nsenone-discriminative and domain-invariant. Specifically, the DNN is trained to\njointly optimize the primary task of senone classification and the secondary\ntask of domain classification with adversarial objective functions. In this\nwork, instead of only focusing on learning a domain-invariant feature (i.e. the\nshared component between domains), we also characterize the difference between\nthe source and target domain distributions by explicitly modeling the private\ncomponent of each domain through a private component extractor DNN. The private\ncomponent is trained to be orthogonal with the shared component and thus\nimplicitly increases the degree of domain-invariance of the shared component. A\nreconstructor DNN is used to reconstruct the original speech feature from the\nprivate and shared components as a regularization. This domain separation\nframework is applied to the unsupervised environment adaptation task and\nachieved 11.08% relative WER reduction from the gradient reversal layer\ntraining, a representative adversarial training method, for automatic speech\nrecognition on CHiME-3 dataset.",
            "journal ref": "2017 IEEE Automatic Speech Recognition and Understanding Workshop\n  (ASRU), Okinawa, 2017, pp. 214-221",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.08016v1",
            "date": "2017-11-21 20:03:03+00:00",
            "title": "Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition",
            "authors": [
                "Zhong Meng",
                "Shinji Watanabe",
                "John R. Hershey",
                "Hakan Erdogan"
            ],
            "abstract": "Far-field speech recognition in noisy and reverberant conditions remains a\nchallenging problem despite recent deep learning breakthroughs. This problem is\ncommonly addressed by acquiring a speech signal from multiple microphones and\nperforming beamforming over them. In this paper, we propose to use a recurrent\nneural network with long short-term memory (LSTM) architecture to adaptively\nestimate real-time beamforming filter coefficients to cope with non-stationary\nenvironmental noise and dynamic nature of source and microphones positions\nwhich results in a set of timevarying room impulse responses. The LSTM adaptive\nbeamformer is jointly trained with a deep LSTM acoustic model to predict senone\nlabels. Further, we use hidden units in the deep LSTM acoustic model to assist\nin predicting the beamforming filter coefficients. The proposed system achieves\n7.97% absolute gain over baseline systems with no beamforming on CHiME-3 real\nevaluation set.",
            "journal ref": "2017 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), New Orleans, LA, 2017, pp. 271-275",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1711.08058v1",
            "date": "2017-11-21 21:42:17+00:00",
            "title": "Multiple-Instance, Cascaded Classification for Keyword Spotting in Narrow-Band Audio",
            "authors": [
                "Ahmad AbdulKader",
                "Kareem Nassar",
                "Mohamed Mahmoud",
                "Daniel Galvez",
                "Chetan Patil"
            ],
            "abstract": "We propose using cascaded classifiers for a keyword spotting (KWS) task on\nnarrow-band (NB), 8kHz audio acquired in non-IID environments --- a more\nchallenging task than most state-of-the-art KWS systems face. We present a\nmodel that incorporates Deep Neural Networks (DNNs), cascading,\nmultiple-feature representations, and multiple-instance learning. The cascaded\nclassifiers handle the task's class imbalance and reduce power consumption on\ncomputationally-constrained devices via early termination. The KWS system\nachieves a false negative rate of 6% at an hourly false positive rate of 0.75",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.08621v3",
            "date": "2017-11-23 08:54:05+00:00",
            "title": "Counterfactual Learning for Machine Translation: Degeneracies and Solutions",
            "authors": [
                "Carolin Lawrence",
                "Pratik Gajane",
                "Stefan Riezler"
            ],
            "abstract": "Counterfactual learning is a natural scenario to improve web-based machine\ntranslation services by offline learning from feedback logged during user\ninteractions. In order to avoid the risk of showing inferior translations to\nusers, in such scenarios mostly exploration-free deterministic logging policies\nare in place. We analyze possible degeneracies of inverse and reweighted\npropensity scoring estimators, in stochastic and deterministic settings, and\nrelate them to recently proposed techniques for counterfactual learning under\ndeterministic logging.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.08870v1",
            "date": "2017-11-24 05:37:35+00:00",
            "title": "Continuous Semantic Topic Embedding Model Using Variational Autoencoder",
            "authors": [
                "Namkyu Jung",
                "Hyeong In Choi"
            ],
            "abstract": "This paper proposes the continuous semantic topic embedding model (CSTEM)\nwhich finds latent topic variables in documents using continuous semantic\ndistance function between the topics and the words by means of the variational\nautoencoder(VAE). The semantic distance could be represented by any symmetric\nbell-shaped geometric distance function on the Euclidean space, for which the\nMahalanobis distance is used in this paper. In order for the semantic distance\nto perform more properly, we newly introduce an additional model parameter for\neach word to take out the global factor from this distance indicating how\nlikely it occurs regardless of its topic. It certainly improves the problem\nthat the Gaussian distribution which is used in previous topic model with\ncontinuous word embedding could not explain the semantic relation correctly and\nhelps to obtain the higher topic coherence. Through the experiments with the\ndataset of 20 Newsgroup, NIPS papers and CNN/Dailymail corpus, the performance\nof the recent state-of-the-art models is accomplished by our model as well as\ngenerating topic embedding vectors which makes possible to observe where the\ntopic vectors are embedded with the word vectors in the real Euclidean space\nand how the topics are related each other semantically.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.09395v2",
            "date": "2017-11-26 14:42:52+00:00",
            "title": "Improved Neural Text Attribute Transfer with Non-parallel Data",
            "authors": [
                "Igor Melnyk",
                "Cicero Nogueira dos Santos",
                "Kahini Wadhawan",
                "Inkit Padhi",
                "Abhishek Kumar"
            ],
            "abstract": "Text attribute transfer using non-parallel data requires methods that can\nperform disentanglement of content and linguistic attributes. In this work, we\npropose multiple improvements over the existing approaches that enable the\nencoder-decoder framework to cope with the text attribute transfer from\nnon-parallel data. We perform experiments on the sentiment transfer task using\ntwo datasets. For both datasets, our proposed method outperforms a strong\nbaseline in two of the three employed evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09534v1",
            "date": "2017-11-27 04:50:15+00:00",
            "title": "Neural Text Generation: A Practical Guide",
            "authors": [
                "Ziang Xie"
            ],
            "abstract": "Deep learning methods have recently achieved great empirical success on\nmachine translation, dialogue response generation, summarization, and other\ntext generation tasks. At a high level, the technique has been to train\nend-to-end neural network models consisting of an encoder model to produce a\nhidden representation of the source text, followed by a decoder model to\ngenerate the target. While such models have significantly fewer pieces than\nearlier systems, significant tuning is still required to achieve good\nperformance. For text generation models in particular, the decoder can behave\nin undesired ways, such as by generating truncated or repetitive outputs,\noutputting bland and generic responses, or in some cases producing\nungrammatical gibberish. This paper is intended as a practical guide for\nresolving such undesired behavior in text generation models, with the aim of\nhelping enable real-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09645v2",
            "date": "2017-11-27 12:21:22+00:00",
            "title": "Multiple Instance Learning Networks for Fine-Grained Sentiment Analysis",
            "authors": [
                "Stefanos Angelidis",
                "Mirella Lapata"
            ],
            "abstract": "We consider the task of fine-grained sentiment analysis from the perspective\nof multiple instance learning (MIL). Our neural model is trained on document\nsentiment labels, and learns to predict the sentiment of text segments, i.e.\nsentences or elementary discourse units (EDUs), without segment-level\nsupervision. We introduce an attention-based polarity scoring method for\nidentifying positive and negative text snippets and a new dataset which we call\nSPOT (as shorthand for Segment-level POlariTy annotations) for evaluating\nMIL-style sentiment models like ours. Experimental results demonstrate superior\nperformance against multiple baselines, whereas a judgement elicitation study\nshows that EDU-level opinion extraction produces more informative summaries\nthan sentence-based alternatives.",
            "journal ref": "Transactions of the Association for Computational Linguistics\n  (TACL), 2018, Volume 6, pages 17-31",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.09714v1",
            "date": "2017-11-27 14:42:26+00:00",
            "title": "Language Bootstrapping: Learning Word Meanings From Perception-Action Association",
            "authors": [
                "Giampiero Salvi",
                "Luis Montesano",
                "Alexandre Bernardino",
                "Jos\u00e9 Santos-Victor"
            ],
            "abstract": "We address the problem of bootstrapping language acquisition for an\nartificial system similarly to what is observed in experiments with human\ninfants. Our method works by associating meanings to words in manipulation\ntasks, as a robot interacts with objects and listens to verbal descriptions of\nthe interactions. The model is based on an affordance network, i.e., a mapping\nbetween robot actions, robot perceptions, and the perceived effects of these\nactions upon objects. We extend the affordance model to incorporate spoken\nwords, which allows us to ground the verbal symbols to the execution of actions\nand the perception of the environment. The model takes verbal descriptions of a\ntask as the input and uses temporal co-occurrence to create links between\nspeech utterances and the involved objects, actions, and effects. We show that\nthe robot is able form useful word-to-meaning associations, even without\nconsidering grammatical structure in the learning process and in the presence\nof recognition errors. These word-to-meaning associations are embedded in the\nrobot's own understanding of its actions. Thus, they can be directly used to\ninstruct the robot to perform tasks and also allow to incorporate context in\nthe speech recognition task. We believe that the encouraging results with our\napproach may afford robots with a capacity to acquire language descriptors in\ntheir operation's environment as well as to shed some light as to how this\nchallenging process develops with human infants.",
            "journal ref": "in IEEE Transactions on Systems, Man, and Cybernetics, Part B\n  (Cybernetics), Volume: 42 Issue: 3, year 2012, pages 660-671",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1711.10327v1",
            "date": "2017-11-28 15:00:24+00:00",
            "title": "Generative Interest Estimation for Document Recommendations",
            "authors": [
                "Danijar Hafner",
                "Alexander Immer",
                "Willi Raschkowski",
                "Fabian Windheuser"
            ],
            "abstract": "Learning distributed representations of documents has pushed the\nstate-of-the-art in several natural language processing tasks and was\nsuccessfully applied to the field of recommender systems recently. In this\npaper, we propose a novel content-based recommender system based on learned\nrepresentations and a generative model of user interest. Our method works as\nfollows: First, we learn representations on a corpus of text documents. Then,\nwe capture a user's interest as a generative model in the space of the document\nrepresentations. In particular, we model the distribution of interest for each\nuser as a Gaussian mixture model (GMM). Recommendations can be obtained\ndirectly by sampling from a user's generative model. Using Latent semantic\nanalysis (LSA) as comparison, we compute and explore document representations\non the Delicious bookmarks dataset, a standard benchmark for recommender\nsystems. We then perform density estimation in both spaces and show that\nlearned representations outperform LSA in terms of predictive performance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1711.10331v1",
            "date": "2017-11-25 07:47:02+00:00",
            "title": "Complex Structure Leads to Overfitting: A Structure Regularization Decoding Method for Natural Language Processing",
            "authors": [
                "Xu Sun",
                "Weiwei Sun",
                "Shuming Ma",
                "Xuancheng Ren",
                "Yi Zhang",
                "Wenjie Li",
                "Houfeng Wang"
            ],
            "abstract": "Recent systems on structured prediction focus on increasing the level of\nstructural dependencies within the model. However, our study suggests that\ncomplex structures entail high overfitting risks. To control the\nstructure-based overfitting, we propose to conduct structure regularization\ndecoding (SR decoding). The decoding of the complex structure model is\nregularized by the additionally trained simple structure model. We\ntheoretically analyze the quantitative relations between the structural\ncomplexity and the overfitting risk. The analysis shows that complex structure\nmodels are prone to the structure-based overfitting. Empirical evaluations show\nthat the proposed method improves the performance of the complex structure\nmodels by reducing the structure-based overfitting. On the sequence labeling\ntasks, the proposed method substantially improves the performance of the\ncomplex neural network models. The maximum F1 error rate reduction is 36.4% for\nthe third-order model. The proposed method also works for the parsing task. The\nmaximum UAS improvement is 5.5% for the tri-sibling model. The results are\ncompetitive with or better than the state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.11023v2",
            "date": "2017-11-29 18:51:14+00:00",
            "title": "A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Management",
            "authors": [
                "I\u00f1igo Casanueva",
                "Pawe\u0142 Budzianowski",
                "Pei-Hao Su",
                "Nikola Mrk\u0161i\u0107",
                "Tsung-Hsien Wen",
                "Stefan Ultes",
                "Lina Rojas-Barahona",
                "Steve Young",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid\nthe significant effort needed to hand-craft the required dialogue flow, the\nDialogue Management (DM) module can be cast as a continuous Markov Decision\nProcess (MDP) and trained through Reinforcement Learning (RL). Several RL\nmodels have been investigated over recent years. However, the lack of a common\nbenchmarking framework makes it difficult to perform a fair comparison between\ndifferent models and their capability to generalise to different environments.\nTherefore, this paper proposes a set of challenging simulated environments for\ndialogue model development and evaluation. To provide some baselines, we\ninvestigate a number of representative parametric algorithms, namely deep\nreinforcement learning algorithms - DQN, A2C and Natural Actor-Critic and\ncompare them to a non-parametric model, GP-SARSA. Both the environments and\npolicy models are implemented using the publicly available PyDial toolkit and\nreleased on-line, in order to establish a testbed framework for further\nexperiments and to facilitate experimental reproducibility.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.11027v2",
            "date": "2017-11-29 18:55:48+00:00",
            "title": "Embedding Words as Distributions with a Bayesian Skip-gram Model",
            "authors": [
                "Arthur Bra\u017einskas",
                "Serhii Havrylov",
                "Ivan Titov"
            ],
            "abstract": "We introduce a method for embedding words as probability densities in a\nlow-dimensional space. Rather than assuming that a word embedding is fixed\nacross the entire text collection, as in standard word embedding methods, in\nour Bayesian model we generate it from a word-specific prior density for each\noccurrence of a given word. Intuitively, for each word, the prior density\nencodes the distribution of its potential 'meanings'. These prior densities are\nconceptually similar to Gaussian embeddings. Interestingly, unlike the Gaussian\nembeddings, we can also obtain context-specific densities: they encode\nuncertainty about the sense of a word given its context and correspond to\nposterior distributions within our model. The context-dependent densities have\nmany potential applications: for example, we show that they can be directly\nused in the lexical substitution task. We describe an effective estimation\nmethod based on the variational autoencoding framework. We also demonstrate\nthat our embeddings achieve competitive results on standard benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11135v3",
            "date": "2017-11-29 22:23:59+00:00",
            "title": "Video Captioning via Hierarchical Reinforcement Learning",
            "authors": [
                "Xin Wang",
                "Wenhu Chen",
                "Jiawei Wu",
                "Yuan-Fang Wang",
                "William Yang Wang"
            ],
            "abstract": "Video captioning is the task of automatically generating a textual\ndescription of the actions in a video. Although previous work (e.g.\nsequence-to-sequence model) has shown promising results in abstracting a coarse\ndescription of a short video, it is still very challenging to caption a video\ncontaining multiple fine-grained actions with a detailed description. This\npaper aims to address the challenge by proposing a novel hierarchical\nreinforcement learning framework for video captioning, where a high-level\nManager module learns to design sub-goals and a low-level Worker module\nrecognizes the primitive actions to fulfill the sub-goal. With this\ncompositional framework to reinforce video captioning at different levels, our\napproach significantly outperforms all the baseline methods on a newly\nintroduced large-scale dataset for fine-grained video captioning. Furthermore,\nour non-ensemble model has already achieved the state-of-the-art results on the\nwidely-used MSR-VTT dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.11383v1",
            "date": "2017-11-30 13:32:45+00:00",
            "title": "Learning to Learn from Weak Supervision by Full Supervision",
            "authors": [
                "Mostafa Dehghani",
                "Aliaksei Severyn",
                "Sascha Rothe",
                "Jaap Kamps"
            ],
            "abstract": "In this paper, we propose a method for training neural networks when we have\na large set of data with weak labels and a small amount of data with true\nlabels. In our proposed model, we train two neural networks: a target network,\nthe learner and a confidence network, the meta-learner. The target network is\noptimized to perform a given task and is trained using a large set of unlabeled\ndata that are weakly annotated. We propose to control the magnitude of the\ngradient updates to the target network using the scores provided by the second\nconfidence network, which is trained on a small amount of supervised data. Thus\nwe avoid that the weight updates computed from noisy labels harm the quality of\nthe target network model.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.11486v1",
            "date": "2017-11-30 16:09:02+00:00",
            "title": "Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation",
            "authors": [
                "Christopher Tegho",
                "Pawe\u0142 Budzianowski",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "In statistical dialogue management, the dialogue manager learns a policy that\nmaps a belief state to an action for the system to perform. Efficient\nexploration is key to successful policy optimisation. Current deep\nreinforcement learning methods are very promising but rely on epsilon-greedy\nexploration, thus subjecting the user to a random choice of action during\nlearning. Alternative approaches such as Gaussian Process SARSA (GPSARSA)\nestimate uncertainties and are sample efficient, leading to better user\nexperience, but on the expense of a greater computational complexity. This\npaper examines approaches to extract uncertainty estimates from deep Q-networks\n(DQN) in the context of dialogue management. We perform an extensive benchmark\nof deep Bayesian methods to extract uncertainty estimates, namely\nBayes-By-Backprop, dropout, its concrete variation, bootstrapped ensemble and\nalpha-divergences, combining it with DQN algorithm.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1711.11508v1",
            "date": "2017-11-30 16:58:46+00:00",
            "title": "Calculating Semantic Similarity between Academic Articles using Topic Event and Ontology",
            "authors": [
                "Ming Liu",
                "Bo Lang",
                "Zepeng Gu"
            ],
            "abstract": "Determining semantic similarity between academic documents is crucial to many\ntasks such as plagiarism detection, automatic technical survey and semantic\nsearch. Current studies mostly focus on semantic similarity between concepts,\nsentences and short text fragments. However, document-level semantic matching\nis still based on statistical information in surface level, neglecting article\nstructures and global semantic meanings, which may cause the deviation in\ndocument understanding. In this paper, we focus on the document-level semantic\nsimilarity issue for academic literatures with a novel method. We represent\nacademic articles with topic events that utilize multiple information profiles,\nsuch as research purposes, methodologies and domains to integrally describe the\nresearch work, and calculate the similarity between topic events based on the\ndomain ontology to acquire the semantic similarity between articles.\nExperiments show that our approach achieves significant performance compared to\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.11543v2",
            "date": "2017-11-30 18:06:47+00:00",
            "title": "Embodied Question Answering",
            "authors": [
                "Abhishek Das",
                "Samyak Datta",
                "Georgia Gkioxari",
                "Stefan Lee",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- where\nan agent is spawned at a random location in a 3D environment and asked a\nquestion (\"What color is the car?\"). In order to answer, the agent must first\nintelligently navigate to explore the environment, gather information through\nfirst-person (egocentric) vision, and then answer the question (\"orange\").\n  This challenging task requires a range of AI skills -- active perception,\nlanguage understanding, goal-driven navigation, commonsense reasoning, and\ngrounding of language into actions. In this work, we develop the environments,\nend-to-end-trained reinforcement learning agents, and evaluation protocols for\nEmbodiedQA.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.00044v1",
            "date": "2017-11-30 19:39:23+00:00",
            "title": "Graph Centrality Measures for Boosting Popularity-Based Entity Linking",
            "authors": [
                "Hussam Hamdan",
                "Jean-Gabriel Ganascia"
            ],
            "abstract": "Many Entity Linking systems use collective graph-based methods to\ndisambiguate the entity mentions within a document. Most of them have focused\non graph construction and initial weighting of the candidate entities, less\nattention has been devoted to compare the graph ranking algorithms. In this\nwork, we focus on the graph-based ranking algorithms, therefore we propose to\napply five centrality measures: Degree, HITS, PageRank, Betweenness and\nCloseness. A disambiguation graph of candidate entities is constructed for each\ndocument using the popularity method, then centrality measures are applied to\nchoose the most relevant candidate to boost the results of entity popularity\nmethod. We investigate the effectiveness of each centrality measure on the\nperformance across different domains and datasets. Our experiments show that a\nsimple and fast centrality measure such as Degree centrality can outperform\nother more time-consuming measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.00377v2",
            "date": "2017-12-01 15:48:50+00:00",
            "title": "Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering",
            "authors": [
                "Aishwarya Agrawal",
                "Dhruv Batra",
                "Devi Parikh",
                "Aniruddha Kembhavi"
            ],
            "abstract": "A number of studies have found that today's Visual Question Answering (VQA)\nmodels are heavily driven by superficial correlations in the training data and\nlack sufficient image grounding. To encourage development of models geared\ntowards the latter, we propose a new setting for VQA where for every question\ntype, train and test sets have different prior distributions of answers.\nSpecifically, we present new splits of the VQA v1 and VQA v2 datasets, which we\ncall Visual Question Answering under Changing Priors (VQA-CP v1 and VQA-CP v2\nrespectively). First, we evaluate several existing VQA models under this new\nsetting and show that their performance degrades significantly compared to the\noriginal VQA setting. Second, we propose a novel Grounded Visual Question\nAnswering model (GVQA) that contains inductive biases and restrictions in the\narchitecture specifically designed to prevent the model from 'cheating' by\nprimarily relying on priors in the training data. Specifically, GVQA explicitly\ndisentangles the recognition of visual concepts present in the image from the\nidentification of plausible answer space for a given question, enabling the\nmodel to more robustly generalize across different distributions of answers.\nGVQA is built off an existing VQA model -- Stacked Attention Networks (SAN).\nOur experiments demonstrate that GVQA significantly outperforms SAN on both\nVQA-CP v1 and VQA-CP v2 datasets. Interestingly, it also outperforms more\npowerful VQA models such as Multimodal Compact Bilinear Pooling (MCB) in\nseveral cases. GVQA offers strengths complementary to SAN when trained and\nevaluated on the original VQA v1 and VQA v2 datasets. Finally, GVQA is more\ntransparent and interpretable than existing VQA models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.01238v1",
            "date": "2017-12-04 18:23:19+00:00",
            "title": "Learning by Asking Questions",
            "authors": [
                "Ishan Misra",
                "Ross Girshick",
                "Rob Fergus",
                "Martial Hebert",
                "Abhinav Gupta",
                "Laurens van der Maaten"
            ],
            "abstract": "We introduce an interactive learning framework for the development and\ntesting of intelligent visual systems, called learning-by-asking (LBA). We\nexplore LBA in context of the Visual Question Answering (VQA) task. LBA differs\nfrom standard VQA training in that most questions are not observed during\ntraining time, and the learner must ask questions it wants answers to. Thus,\nLBA more closely mimics natural learning and has the potential to be more\ndata-efficient than the traditional VQA setting. We present a model that\nperforms LBA on the CLEVR dataset, and show that it automatically discovers an\neasy-to-hard curriculum when learning interactively from an oracle. Our LBA\ngenerated data consistently matches or outperforms the CLEVR train data and is\nmore sample efficient. We also show that our model asks questions that\ngeneralize to state-of-the-art VQA models and to novel test time distributions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.01455v1",
            "date": "2017-12-05 02:51:35+00:00",
            "title": "Multimodal Storytelling via Generative Adversarial Imitation Learning",
            "authors": [
                "Zhiqian Chen",
                "Xuchao Zhang",
                "Arnold P. Boedihardjo",
                "Jing Dai",
                "Chang-Tien Lu"
            ],
            "abstract": "Deriving event storylines is an effective summarization method to succinctly\norganize extensive information, which can significantly alleviate the pain of\ninformation overload. The critical challenge is the lack of widely recognized\ndefinition of storyline metric. Prior studies have developed various approaches\nbased on different assumptions about users' interests. These works can extract\ninteresting patterns, but their assumptions do not guarantee that the derived\npatterns will match users' preference. On the other hand, their exclusiveness\nof single modality source misses cross-modality information. This paper\nproposes a method, multimodal imitation learning via generative adversarial\nnetworks(MIL-GAN), to directly model users' interests as reflected by various\ndata. In particular, the proposed model addresses the critical challenge by\nimitating users' demonstrated storylines. Our proposed model is designed to\nlearn the reward patterns given user-provided storylines and then applies the\nlearned policy to unseen data. The proposed approach is demonstrated to be\ncapable of acquiring the user's implicit intent and outperforming competing\nmethods by a substantial margin with a user study.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1712.01562v1",
            "date": "2017-12-05 10:29:14+00:00",
            "title": "EmTaggeR: A Word Embedding Based Novel Method for Hashtag Recommendation on Twitter",
            "authors": [
                "Kuntal Dey",
                "Ritvik Shrivastava",
                "Saroj Kaushik",
                "L. Venkata Subramaniam"
            ],
            "abstract": "The hashtag recommendation problem addresses recommending (suggesting) one or\nmore hashtags to explicitly tag a post made on a given social network platform,\nbased upon the content and context of the post. In this work, we propose a\nnovel methodology for hashtag recommendation for microblog posts, specifically\nTwitter. The methodology, EmTaggeR, is built upon a training-testing framework\nthat builds on the top of the concept of word embedding. The training phase\ncomprises of learning word vectors associated with each hashtag, and deriving a\nword embedding for each hashtag. We provide two training procedures, one in\nwhich each hashtag is trained with a separate word embedding model applicable\nin the context of that hashtag, and another in which each hashtag obtains its\nembedding from a global context. The testing phase constitutes computing the\naverage word embedding of the test post, and finding the similarity of this\nembedding with the known embeddings of the hashtags. The tweets that contain\nthe most-similar hashtag are extracted, and all the hashtags that appear in\nthese tweets are ranked in terms of embedding similarity scores. The top-K\nhashtags that appear in this ranked list, are recommended for the given test\npost. Our system produces F1 score of 50.83%, improving over the LDA baseline\nby around 6.53 times, outperforming the best-performing system known in the\nliterature that provides a lift of 6.42 times. EmTaggeR is a fast, scalable and\nlightweight system, which makes it practical to deploy in real-life\napplications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01769v6",
            "date": "2017-12-05 17:24:05+00:00",
            "title": "State-of-the-art Speech Recognition With Sequence-to-Sequence Models",
            "authors": [
                "Chung-Cheng Chiu",
                "Tara N. Sainath",
                "Yonghui Wu",
                "Rohit Prabhavalkar",
                "Patrick Nguyen",
                "Zhifeng Chen",
                "Anjuli Kannan",
                "Ron J. Weiss",
                "Kanishka Rao",
                "Ekaterina Gonina",
                "Navdeep Jaitly",
                "Bo Li",
                "Jan Chorowski",
                "Michiel Bacchiani"
            ],
            "abstract": "Attention-based encoder-decoder architectures such as Listen, Attend, and\nSpell (LAS), subsume the acoustic, pronunciation and language model components\nof a traditional automatic speech recognition (ASR) system into a single neural\nnetwork. In previous work, we have shown that such architectures are comparable\nto state-of-theart ASR systems on dictation tasks, but it was not clear if such\narchitectures would be practical for more challenging tasks such as voice\nsearch. In this work, we explore a variety of structural and optimization\nimprovements to our LAS model which significantly improve performance. On the\nstructural side, we show that word piece models can be used instead of\ngraphemes. We also introduce a multi-head attention architecture, which offers\nimprovements over the commonly-used single-head attention. On the optimization\nside, we explore synchronous training, scheduled sampling, label smoothing, and\nminimum word error rate optimization, which are all shown to improve accuracy.\nWe present results with a unidirectional LSTM encoder for streaming\nrecognition. On a 12, 500 hour voice search task, we find that the proposed\nchanges improve the WER from 9.2% to 5.6%, while the best conventional system\nachieves 6.7%; on a dictation task our model achieves a WER of 4.1% compared to\n5% for the conventional system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01807v1",
            "date": "2017-12-05 18:34:56+00:00",
            "title": "Improving the Performance of Online Neural Transducer Models",
            "authors": [
                "Tara N. Sainath",
                "Chung-Cheng Chiu",
                "Rohit Prabhavalkar",
                "Anjuli Kannan",
                "Yonghui Wu",
                "Patrick Nguyen",
                "Zhifeng Chen"
            ],
            "abstract": "Having a sequence-to-sequence model which can operate in an online fashion is\nimportant for streaming applications such as Voice Search. Neural transducer is\na streaming sequence-to-sequence model, but has shown a significant degradation\nin performance compared to non-streaming models such as Listen, Attend and\nSpell (LAS). In this paper, we present various improvements to NT.\nSpecifically, we look at increasing the window over which NT computes\nattention, mainly by looking backwards in time so the model still remains\nonline. In addition, we explore initializing a NT model from a LAS-trained\nmodel so that it is guided with a better alignment. Finally, we explore\nincluding stronger language models such as using wordpiece models, and applying\nan external LM during the beam search. On a Voice Search task, we find with\nthese improvements we can get NT to match the performance of LAS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01818v1",
            "date": "2017-12-05 18:52:18+00:00",
            "title": "Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models",
            "authors": [
                "Rohit Prabhavalkar",
                "Tara N. Sainath",
                "Yonghui Wu",
                "Patrick Nguyen",
                "Zhifeng Chen",
                "Chung-Cheng Chiu",
                "Anjuli Kannan"
            ],
            "abstract": "Sequence-to-sequence models, such as attention-based models in automatic\nspeech recognition (ASR), are typically trained to optimize the cross-entropy\ncriterion which corresponds to improving the log-likelihood of the data.\nHowever, system performance is usually measured in terms of word error rate\n(WER), not log-likelihood. Traditional ASR systems benefit from discriminative\nsequence training which optimizes criteria such as the state-level minimum\nBayes risk (sMBR) which are more closely related to WER. In the present work,\nwe explore techniques to train attention-based models to directly minimize\nexpected word error rate. We consider two loss functions which approximate the\nexpected number of word errors: either by sampling from the model, or by using\nN-best lists of decoded hypotheses, which we find to be more effective than the\nsampling-based method. In experimental evaluations, we find that the proposed\ntraining procedure improves performance by up to 8.2% relative to the baseline\nsystem. This allows us to train grapheme-based, uni-directional attention-based\nmodels which match the performance of a traditional, state-of-the-art,\ndiscriminative sequence-trained system on a mobile voice-search task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01864v1",
            "date": "2017-12-05 19:02:28+00:00",
            "title": "No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models",
            "authors": [
                "Tara N. Sainath",
                "Rohit Prabhavalkar",
                "Shankar Kumar",
                "Seungji Lee",
                "Anjuli Kannan",
                "David Rybach",
                "Vlad Schogol",
                "Patrick Nguyen",
                "Bo Li",
                "Yonghui Wu",
                "Zhifeng Chen",
                "Chung-Cheng Chiu"
            ],
            "abstract": "For decades, context-dependent phonemes have been the dominant sub-word unit\nfor conventional acoustic modeling systems. This status quo has begun to be\nchallenged recently by end-to-end models which seek to combine acoustic,\npronunciation, and language model components into a single neural network. Such\nsystems, which typically predict graphemes or words, simplify the recognition\nprocess since they remove the need for a separate expert-curated pronunciation\nlexicon to map from phoneme-based units to words. However, there has been\nlittle previous work comparing phoneme-based versus grapheme-based sub-word\nunits in the end-to-end modeling framework, to determine whether the gains from\nsuch approaches are primarily due to the new probabilistic model, or from the\njoint learning of the various components with grapheme-based units.\n  In this work, we conduct detailed experiments which are aimed at quantifying\nthe value of phoneme-based pronunciation lexica in the context of end-to-end\nmodels. We examine phoneme-based end-to-end models, which are contrasted\nagainst grapheme-based ones on a large vocabulary English Voice-search task,\nwhere we find that graphemes do indeed outperform phonemes. We also compare\ngrapheme and phoneme-based approaches on a multi-dialect English task, which\nonce again confirm the superiority of graphemes, greatly simplifying the system\nfor recognizing multiple dialects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01996v1",
            "date": "2017-12-06 01:30:54+00:00",
            "title": "An analysis of incorporating an external language model into a sequence-to-sequence model",
            "authors": [
                "Anjuli Kannan",
                "Yonghui Wu",
                "Patrick Nguyen",
                "Tara N. Sainath",
                "Zhifeng Chen",
                "Rohit Prabhavalkar"
            ],
            "abstract": "Attention-based sequence-to-sequence models for automatic speech recognition\njointly train an acoustic model, language model, and alignment mechanism. Thus,\nthe language model component is only trained on transcribed audio-text pairs.\nThis leads to the use of shallow fusion with an external language model at\ninference time. Shallow fusion refers to log-linear interpolation with a\nseparately trained language model at each step of the beam search. In this\nwork, we investigate the behavior of shallow fusion across a range of\nconditions: different types of language models, different decoding units, and\ndifferent tasks. On Google Voice Search, we demonstrate that the use of shallow\nfusion with a neural LM with wordpieces yields a 9.1% relative word error rate\nreduction (WERR) over our competitive attention-based sequence-to-sequence\nmodel, obviating the need for second-pass rescoring.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1712.02034v2",
            "date": "2017-12-06 04:29:28+00:00",
            "title": "SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties",
            "authors": [
                "Garrett B. Goh",
                "Nathan O. Hodas",
                "Charles Siegel",
                "Abhinav Vishnu"
            ],
            "abstract": "Chemical databases store information in text representations, and the SMILES\nformat is a universal standard used in many cheminformatics software. Encoded\nin each SMILES string is structural information that can be used to predict\ncomplex chemical properties. In this work, we develop SMILES2vec, a deep RNN\nthat automatically learns features from SMILES to predict chemical properties,\nwithout the need for additional explicit feature engineering. Using Bayesian\noptimization methods to tune the network architecture, we show that an\noptimized SMILES2vec model can serve as a general-purpose neural network for\npredicting distinct chemical properties including toxicity, activity,\nsolubility and solvation energy, while also outperforming contemporary MLP\nneural networks that uses engineered features. Furthermore, we demonstrate\nproof-of-concept of interpretability by developing an explanation mask that\nlocalizes on the most important characters used in making a prediction. When\ntested on the solubility dataset, it identified specific parts of a chemical\nthat is consistent with established first-principles knowledge with an accuracy\nof 88%. Our work demonstrates that neural networks can learn technically\naccurate chemical concept and provide state-of-the-art accuracy, making\ninterpretable deep neural networks a useful tool of relevance to the chemical\nindustry.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1712.02820v1",
            "date": "2017-12-07 19:10:45+00:00",
            "title": "A Deep Network Model for Paraphrase Detection in Short Text Messages",
            "authors": [
                "Basant Agarwal",
                "Heri Ramampiaro",
                "Helge Langseth",
                "Massimiliano Ruocco"
            ],
            "abstract": "This paper is concerned with paraphrase detection. The ability to detect\nsimilar sentences written in natural language is crucial for several\napplications, such as text mining, text summarization, plagiarism detection,\nauthorship authentication and question answering. Given two sentences, the\nobjective is to detect whether they are semantically identical. An important\ninsight from this work is that existing paraphrase systems perform well when\napplied on clean texts, but they do not necessarily deliver good performance\nagainst noisy texts. Challenges with paraphrase detection on user generated\nshort texts, such as Twitter, include language irregularity and noise. To cope\nwith these challenges, we propose a novel deep neural network-based approach\nthat relies on coarse-grained sentence modeling using a convolutional neural\nnetwork and a long short-term memory model, combined with a specific\nfine-grained word-level similarity matching model. Our experimental results\nshow that the proposed approach outperforms existing state-of-the-art\napproaches on user-generated noisy social media data, such as Twitter texts,\nand achieves highly competitive performance on a cleaner corpus.",
            "journal ref": "B Agarwal, H. Ramampiaro, H Langseth, M Ruocco, (2018), \"A Deep\n  Network Model for Paraphrase Detection in Short Text Messages\". In\n  Information Processing & Management Journal (IPM), 54(6), pp. 922-937.\n  Elsevier",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1712.02838v1",
            "date": "2017-12-07 19:52:50+00:00",
            "title": "End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient",
            "authors": [
                "Li Zhou",
                "Kevin Small",
                "Oleg Rokhlenko",
                "Charles Elkan"
            ],
            "abstract": "Learning a goal-oriented dialog policy is generally performed offline with\nsupervised learning algorithms or online with reinforcement learning (RL).\nAdditionally, as companies accumulate massive quantities of dialog transcripts\nbetween customers and trained human agents, encoder-decoder methods have gained\npopularity as agent utterances can be directly treated as supervision without\nthe need for utterance-level annotations. However, one potential drawback of\nsuch approaches is that they myopically generate the next agent utterance\nwithout regard for dialog-level considerations. To resolve this concern, this\npaper describes an offline RL method for learning from unannotated corpora that\ncan optimize a goal-oriented policy at both the utterance and dialog level. We\nintroduce a novel reward function and use both on-policy and off-policy policy\ngradient to learn a policy offline without requiring online user interaction or\nan explicit state space definition.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1712.03086v1",
            "date": "2017-12-05 21:15:48+00:00",
            "title": "FlagIt: A System for Minimally Supervised Human Trafficking Indicator Mining",
            "authors": [
                "Mayank Kejriwal",
                "Jiayuan Ding",
                "Runqi Shao",
                "Anoop Kumar",
                "Pedro Szekely"
            ],
            "abstract": "In this paper, we describe and study the indicator mining problem in the\nonline sex advertising domain. We present an in-development system, FlagIt\n(Flexible and adaptive generation of Indicators from text), which combines the\nbenefits of both a lightweight expert system and classical semi-supervision\n(heuristic re-labeling) with recently released state-of-the-art unsupervised\ntext embeddings to tag millions of sentences with indicators that are highly\ncorrelated with human trafficking. The FlagIt technology stack is open source.\nOn preliminary evaluations involving five indicators, FlagIt illustrates\npromising performance compared to several alternatives. The system is being\nactively developed, refined and integrated into a domain-specific search system\nused by over 200 law enforcement agencies to combat human trafficking, and is\nbeing aggressively extended to mine at least six more indicators with minimal\nprogramming effort. FlagIt is a good example of a system that operates in\nlimited label settings, and that requires creative combinations of established\nmachine learning techniques to produce outputs that could be used by real-world\nnon-technical analysts.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1712.03133v1",
            "date": "2017-12-08 15:43:21+00:00",
            "title": "Building competitive direct acoustics-to-word models for English conversational speech recognition",
            "authors": [
                "Kartik Audhkhasi",
                "Brian Kingsbury",
                "Bhuvana Ramabhadran",
                "George Saon",
                "Michael Picheny"
            ],
            "abstract": "Direct acoustics-to-word (A2W) models in the end-to-end paradigm have\nreceived increasing attention compared to conventional sub-word based automatic\nspeech recognition models using phones, characters, or context-dependent hidden\nMarkov model states. This is because A2W models recognize words from speech\nwithout any decoder, pronunciation lexicon, or externally-trained language\nmodel, making training and decoding with such models simple. Prior work has\nshown that A2W models require orders of magnitude more training data in order\nto perform comparably to conventional models. Our work also showed this\naccuracy gap when using the English Switchboard-Fisher data set. This paper\ndescribes a recipe to train an A2W model that closes this gap and is at-par\nwith state-of-the-art sub-word based models. We achieve a word error rate of\n8.8%/13.9% on the Hub5-2000 Switchboard/CallHome test sets without any decoder\nor language model. We find that model initialization, training data order, and\nregularization have the most impact on the A2W model performance. Next, we\npresent a joint word-character A2W model that learns to first spell the word\nand then recognize it. This model provides a rich output to the user instead of\nsimple word hypotheses, making it especially useful in the case of words unseen\nor rarely-seen during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.03249v1",
            "date": "2017-12-08 19:05:50+00:00",
            "title": "Social Emotion Mining Techniques for Facebook Posts Reaction Prediction",
            "authors": [
                "Florian Krebs",
                "Bruno Lubascher",
                "Tobias Moers",
                "Pieter Schaap",
                "Gerasimos Spanakis"
            ],
            "abstract": "As of February 2016 Facebook allows users to express their experienced\nemotions about a post by using five so-called `reactions'. This research paper\nproposes and evaluates alternative methods for predicting these reactions to\nuser posts on public pages of firms/companies (like supermarket chains). For\nthis purpose, we collected posts (and their reactions) from Facebook pages of\nlarge supermarket chains and constructed a dataset which is available for other\nresearches. In order to predict the distribution of reactions of a new post,\nneural network architectures (convolutional and recurrent neural networks) were\ntested using pretrained word embeddings. Results of the neural networks were\nimproved by introducing a bootstrapping approach for sentiment and emotion\nmining on the comments for each post. The final model (a combination of neural\nnetwork and a baseline emotion miner) is able to predict the reaction\ndistribution on Facebook posts with a mean squared error (or misclassification\nrate) of 0.135.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1712.03512v1",
            "date": "2017-12-10 12:04:19+00:00",
            "title": "Comparative analysis of criteria for filtering time series of word usage frequencies",
            "authors": [
                "Inna A. Belashova",
                "Vladimir V. Bochkarev"
            ],
            "abstract": "This paper describes a method of nonlinear wavelet thresholding of time\nseries. The Ramachandran-Ranganathan runs test is used to assess the quality of\napproximation. To minimize the objective function, it is proposed to use\ngenetic algorithms - one of the stochastic optimization methods. The suggested\nmethod is tested both on the model series and on the word frequency series\nusing the Google Books Ngram data. It is shown that method of filtering which\nuses the runs criterion shows significantly better results compared with the\nstandard wavelet thresholding. The method can be used when quality of filtering\nis of primary importance but not the speed of calculations.",
            "journal ref": "None",
            "category": "stat.ME"
        },
        {
            "url": "http://arxiv.org/abs/1712.03897v1",
            "date": "2017-12-11 17:18:34+00:00",
            "title": "Learning Modality-Invariant Representations for Speech and Images",
            "authors": [
                "Kenneth Leidal",
                "David Harwath",
                "James Glass"
            ],
            "abstract": "In this paper, we explore the unsupervised learning of a semantic embedding\nspace for co-occurring sensory inputs. Specifically, we focus on the task of\nlearning a semantic vector space for both spoken and handwritten digits using\nthe TIDIGITs and MNIST datasets. Current techniques encode image and\naudio/textual inputs directly to semantic embeddings. In contrast, our\ntechnique maps an input to the mean and log variance vectors of a diagonal\nGaussian from which sample semantic embeddings are drawn. In addition to\nencouraging semantic similarity between co-occurring inputs,our loss function\nincludes a regularization term borrowed from variational autoencoders (VAEs)\nwhich drives the posterior distributions over embeddings to be unit Gaussian.\nWe can use this regularization term to filter out modality information while\npreserving semantic information. We speculate this technique may be more\nbroadly applicable to other areas of cross-modality/domain information\nretrieval and transfer learning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1712.04046v3",
            "date": "2017-12-11 21:57:03+00:00",
            "title": "Character-Based Handwritten Text Transcription with Attention Networks",
            "authors": [
                "Jason Poulos",
                "Rafael Valle"
            ],
            "abstract": "The paper approaches the task of handwritten text recognition (HTR) with\nattentional encoder-decoder networks trained on sequences of characters, rather\nthan words. We experiment on lines of text from popular handwriting datasets\nand compare different activation functions for the attention mechanism used for\naligning image pixels and target characters. We find that softmax attention\nfocuses heavily on individual characters, while sigmoid attention focuses on\nmultiple characters at each step of the decoding. When the sequence alignment\nis one-to-one, softmax attention is able to learn a more precise alignment at\neach step of the decoding, whereas the alignment generated by sigmoid attention\nis much less precise. When a linear function is used to obtain attention\nweights, the model predicts a character by looking at the entire sequence of\ncharacters and performs poorly because it lacks a precise alignment between the\nsource and target. Future research may explore HTR in natural scene images,\nsince the model is capable of transcribing handwritten text without the need\nfor producing segmentations or bounding boxes of text in images.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.04048v1",
            "date": "2017-12-11 22:04:39+00:00",
            "title": "Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks",
            "authors": [
                "Hao Zhang",
                "Shizhen Xu",
                "Graham Neubig",
                "Wei Dai",
                "Qirong Ho",
                "Guangwen Yang",
                "Eric P. Xing"
            ],
            "abstract": "Recent deep learning (DL) models have moved beyond static network\narchitectures to dynamic ones, handling data where the network structure\nchanges every example, such as sequences of variable lengths, trees, and\ngraphs. Existing dataflow-based programming models for DL---both static and\ndynamic declaration---either cannot readily express these dynamic models, or\nare inefficient due to repeated dataflow graph construction and processing, and\ndifficulties in batched execution. We present Cavs, a vertex-centric\nprogramming interface and optimized system implementation for dynamic DL\nmodels. Cavs represents dynamic network structure as a static vertex function\n$\\mathcal{F}$ and a dynamic instance-specific graph $\\mathcal{G}$, and performs\nbackpropagation by scheduling the execution of $\\mathcal{F}$ following the\ndependencies in $\\mathcal{G}$. Cavs bypasses expensive graph construction and\npreprocessing overhead, allows for the use of static graph optimization\ntechniques on pre-defined operations in $\\mathcal{F}$, and naturally exposes\nbatched execution opportunities over different graphs. Experiments comparing\nCavs to two state-of-the-art frameworks for dynamic NNs (TensorFlow Fold and\nDyNet) demonstrate the efficacy of this approach: Cavs achieves a near one\norder of magnitude speedup on training of various dynamic NN architectures, and\nablations demonstrate the contribution of our proposed batching and memory\nmanagement strategies.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1712.04116v3",
            "date": "2017-12-12 04:07:10+00:00",
            "title": "A Novel Document Generation Process for Topic Detection based on Hierarchical Latent Tree Models",
            "authors": [
                "Peixian Chen",
                "Zhourong Chen",
                "Nevin L. Zhang"
            ],
            "abstract": "We propose a novel document generation process based on hierarchical latent\ntree models (HLTMs) learned from data. An HLTM has a layer of observed word\nvariables at the bottom and multiple layers of latent variables on top. For\neach document, we first sample values for the latent variables layer by layer\nvia logic sampling, then draw relative frequencies for the words conditioned on\nthe values of the latent variables, and finally generate words for the document\nusing the relative word frequencies. The motivation for the work is to take\nword counts into consideration with HLTMs. In comparison with LDA-based\nhierarchical document generation processes, the new process achieves\ndrastically better model fit with much fewer parameters. It also yields more\nmeaningful topics and topic hierarchies. It is the new state-of-the-art for the\nhierarchical topic detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.04314v1",
            "date": "2017-12-12 14:38:34+00:00",
            "title": "Generating and Estimating Nonverbal Alphabets for Situated and Multimodal Communications",
            "authors": [
                "Serhii Hamotskyi",
                "Sergii Stirenko",
                "Yuri Gordienko",
                "Anis Rojbi"
            ],
            "abstract": "In this paper, we discuss the formalized approach for generating and\nestimating symbols (and alphabets), which can be communicated by the wide range\nof non-verbal means based on specific user requirements (medium, priorities,\ntype of information that needs to be conveyed). The short characterization of\nbasic terms and parameters of such symbols (and alphabets) with approaches to\ngenerate them are given. Then the framework, experimental setup, and some\nmachine learning methods to estimate usefulness and effectiveness of the\nnonverbal alphabets and systems are presented. The previous results demonstrate\nthat usage of multimodal data sources (like wearable accelerometer, heart\nmonitor, muscle movements sensors, braincomputer interface) along with machine\nlearning approaches can provide the deeper understanding of the usefulness and\neffectiveness of such alphabets and systems for nonverbal and situated\ncommunication. The symbols (and alphabets) generated and estimated by such\nmethods may be useful in various applications: from synthetic languages and\nconstructed scripts to multimodal nonverbal and situated interaction between\npeople and artificial intelligence systems through Human-Computer Interfaces,\nsuch as mouse gestures, touchpads, body gestures, eyetracking cameras,\nwearables, and brain-computing interfaces, especially in applications for\nelderly care and people with disabilities.",
            "journal ref": "International Journal of Systems Applications Engineering and\n  Development, 11, 232-236 (2017)",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1712.04753v3",
            "date": "2017-12-12 14:30:27+00:00",
            "title": "Learning Spontaneity to Improve Emotion Recognition In Speech",
            "authors": [
                "Karttikeya Mangalam",
                "Tanaya Guha"
            ],
            "abstract": "We investigate the effect and usefulness of spontaneity (i.e. whether a given\nspeech is spontaneous or not) in speech in the context of emotion recognition.\nWe hypothesize that emotional content in speech is interrelated with its\nspontaneity, and use spontaneity classification as an auxiliary task to the\nproblem of emotion recognition. We propose two supervised learning settings\nthat utilize spontaneity to improve speech emotion recognition: a hierarchical\nmodel that performs spontaneity detection before performing emotion\nrecognition, and a multitask learning model that jointly learns to recognize\nboth spontaneity and emotion. Through various experiments on the well known\nIEMOCAP database, we show that by using spontaneity detection as an additional\ntask, significant improvement can be achieved over emotion recognition systems\nthat are unaware of spontaneity. We achieve state-of-the-art emotion\nrecognition accuracy (4-class, 69.1%) on the IEMOCAP database outperforming\nseveral relevant and competitive baselines.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1712.05181v2",
            "date": "2017-12-14 11:37:18+00:00",
            "title": "Rasa: Open Source Language Understanding and Dialogue Management",
            "authors": [
                "Tom Bocklisch",
                "Joey Faulkner",
                "Nick Pawlowski",
                "Alan Nichol"
            ],
            "abstract": "We introduce a pair of tools, Rasa NLU and Rasa Core, which are open source\npython libraries for building conversational software. Their purpose is to make\nmachine-learning based dialogue management and language understanding\naccessible to non-specialist software developers. In terms of design\nphilosophy, we aim for ease of use, and bootstrapping from minimal (or no)\ninitial training data. Both packages are extensively documented and ship with a\ncomprehensive suite of tests. The code is available at\nhttps://github.com/RasaHQ/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05191v1",
            "date": "2017-12-14 12:04:10+00:00",
            "title": "Relation Extraction : A Survey",
            "authors": [
                "Sachin Pawar",
                "Girish K. Palshikar",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "With the advent of the Internet, large amount of digital text is generated\neveryday in the form of news articles, research publications, blogs, question\nanswering forums and social media. It is important to develop techniques for\nextracting information automatically from these documents, as lot of important\ninformation is hidden within them. This extracted information can be used to\nimprove access and management of knowledge hidden in large text corpora.\nSeveral applications such as Question Answering, Information Retrieval would\nbenefit from this information. Entities like persons and organizations, form\nthe most basic unit of the information. Occurrences of entities in a sentence\nare often linked through well-defined relations; e.g., occurrences of person\nand organization in a sentence may be linked through relations such as employed\nat. The task of Relation Extraction (RE) is to identify such relations\nautomatically. In this paper, we survey several important supervised,\nsemi-supervised and unsupervised RE techniques. We also cover the paradigms of\nOpen Information Extraction (OIE) and Distant Supervision. Finally, we describe\nsome of the recent trends in the RE techniques and possible future research\ndirections. This survey would be useful for three kinds of readers - i)\nNewcomers in the field who want to quickly learn about RE; ii) Researchers who\nwant to know how the various RE techniques evolved over time and what are\npossible future research directions and iii) Practitioners who just need to\nknow which RE technique works best in various settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05403v1",
            "date": "2017-12-14 12:46:44+00:00",
            "title": "Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis",
            "authors": [
                "Yi Tay",
                "Anh Tuan Luu",
                "Siu Cheung Hui"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) tries to predict the polarity of a\ngiven document with respect to a given aspect entity. While neural network\narchitectures have been successful in predicting the overall polarity of\nsentences, aspect-specific sentiment analysis still remains as an open problem.\nIn this paper, we propose a novel method for integrating aspect information\ninto the neural model. More specifically, we incorporate aspect information\ninto the neural model by modeling word-aspect relationships. Our novel model,\n\\textit{Aspect Fusion LSTM} (AF-LSTM) learns to attend based on associative\nrelationships between sentence words and aspect which allows our model to\nadaptively focus on the correct words given an aspect term. This ameliorates\nthe flaws of other state-of-the-art models that utilize naive concatenations to\nmodel word-aspect similarity. Instead, our model adopts circular convolution\nand circular correlation to model the similarity between aspect and words and\nelegantly incorporates this within a differentiable neural attention framework.\nFinally, our model is end-to-end differentiable and highly related to\nconvolution-correlation (holographic like) memories. Our proposed neural model\nachieves state-of-the-art performance on benchmark datasets, outperforming\nATAE-LSTM by $4\\%-5\\%$ on average across multiple datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05558v3",
            "date": "2017-12-15 06:38:15+00:00",
            "title": "CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication",
            "authors": [
                "Jin-Hwa Kim",
                "Nikita Kitaev",
                "Xinlei Chen",
                "Marcus Rohrbach",
                "Byoung-Tak Zhang",
                "Yuandong Tian",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "In this work, we propose a goal-driven collaborative task that combines\nlanguage, perception, and action. Specifically, we develop a Collaborative\nimage-Drawing game between two agents, called CoDraw. Our game is grounded in a\nvirtual world that contains movable clip art objects. The game involves two\nplayers: a Teller and a Drawer. The Teller sees an abstract scene containing\nmultiple clip art pieces in a semantically meaningful configuration, while the\nDrawer tries to reconstruct the scene on an empty canvas using available clip\nart pieces. The two players communicate with each other using natural language.\nWe collect the CoDraw dataset of ~10K dialogs consisting of ~138K messages\nexchanged between human players. We define protocols and metrics to evaluate\nlearned agents in this testbed, highlighting the need for a novel \"crosstalk\"\nevaluation condition which pairs agents trained independently on disjoint\nsubsets of the training data. We present models for our task and benchmark them\nusing both fully automated evaluation and by having them play the game live\nwith humans.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.05608v1",
            "date": "2017-12-15 10:33:57+00:00",
            "title": "A Novel Approach for Effective Learning in Low Resourced Scenarios",
            "authors": [
                "Sri Harsha Dumpala",
                "Rupayan Chakraborty",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Deep learning based discriminative methods, being the state-of-the-art\nmachine learning techniques, are ill-suited for learning from lower amounts of\ndata. In this paper, we propose a novel framework, called simultaneous two\nsample learning (s2sL), to effectively learn the class discriminative\ncharacteristics, even from very low amount of data. In s2sL, more than one\nsample (here, two samples) are simultaneously considered to both, train and\ntest the classifier. We demonstrate our approach for speech/music\ndiscrimination and emotion classification through experiments. Further, we also\nshow the effectiveness of s2sL approach for classification in low-resource\nscenario, and for imbalanced data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05690v2",
            "date": "2017-12-15 14:44:28+00:00",
            "title": "Sockeye: A Toolkit for Neural Machine Translation",
            "authors": [
                "Felix Hieber",
                "Tobias Domhan",
                "Michael Denkowski",
                "David Vilar",
                "Artem Sokolov",
                "Ann Clifton",
                "Matt Post"
            ],
            "abstract": "We describe Sockeye (version 1.12), an open-source sequence-to-sequence\ntoolkit for Neural Machine Translation (NMT). Sockeye is a production-ready\nframework for training and applying models as well as an experimental platform\nfor researchers. Written in Python and built on MXNet, the toolkit offers\nscalable training and inference for the three most prominent encoder-decoder\narchitectures: attentional recurrent neural networks, self-attentional\ntransformers, and fully convolutional networks. Sockeye also supports a wide\nrange of optimizers, normalization and regularization techniques, and inference\nimprovements from current NMT literature. Users can easily run standard\ntraining recipes, explore different model settings, and incorporate new ideas.\nIn this paper, we highlight Sockeye's features and benchmark it against other\nNMT toolkits on two language arcs from the 2017 Conference on Machine\nTranslation (WMT): English-German and Latvian-English. We report competitive\nBLEU scores across all three architectures, including an overall best score for\nSockeye's transformer implementation. To facilitate further comparison, we\nrelease all system outputs and training scripts used in our experiments. The\nSockeye toolkit is free software released under the Apache 2.0 license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05785v2",
            "date": "2017-12-15 18:41:53+00:00",
            "title": "Sentiment Predictability for Stocks",
            "authors": [
                "Jordan Prosky",
                "Xingyou Song",
                "Andrew Tan",
                "Michael Zhao"
            ],
            "abstract": "In this work, we present our findings and experiments for stock-market\nprediction using various textual sentiment analysis tools, such as mood\nanalysis and event extraction, as well as prediction models, such as LSTMs and\nspecific convolutional architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.05999v1",
            "date": "2017-12-16 18:07:58+00:00",
            "title": "Characterizing Political Fake News in Twitter by its Meta-Data",
            "authors": [
                "Julio Amador",
                "Axel Oehmichen",
                "Miguel Molina-Solana"
            ],
            "abstract": "This article presents a preliminary approach towards characterizing political\nfake news on Twitter through the analysis of their meta-data. In particular, we\nfocus on more than 1.5M tweets collected on the day of the election of Donald\nTrump as 45th president of the United States of America. We use the meta-data\nembedded within those tweets in order to look for differences between tweets\ncontaining fake news and tweets not containing them. Specifically, we perform\nour analysis only on tweets that went viral, by studying proxies for users'\nexposure to the tweets, by characterizing accounts spreading fake news, and by\nlooking at their polarization. We found significant differences on the\ndistribution of followers, the number of URLs on tweets, and the verification\nof the users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06086v1",
            "date": "2017-12-17 10:29:15+00:00",
            "title": "Deep Learning for Distant Speech Recognition",
            "authors": [
                "Mirco Ravanelli"
            ],
            "abstract": "Deep learning is an emerging technology that is considered one of the most\npromising directions for reaching higher levels of artificial intelligence.\nAmong the other achievements, building computers that understand speech\nrepresents a crucial leap towards intelligent machines. Despite the great\nefforts of the past decades, however, a natural and robust human-machine speech\ninteraction still appears to be out of reach, especially when users interact\nwith a distant microphone in noisy and reverberant environments. The latter\ndisturbances severely hamper the intelligibility of a speech signal, making\nDistant Speech Recognition (DSR) one of the major open challenges in the field.\n  This thesis addresses the latter scenario and proposes some novel techniques,\narchitectures, and algorithms to improve the robustness of distant-talking\nacoustic models. We first elaborate on methodologies for realistic data\ncontamination, with a particular emphasis on DNN training with simulated data.\nWe then investigate on approaches for better exploiting speech contexts,\nproposing some original methodologies for both feed-forward and recurrent\nneural networks. Lastly, inspired by the idea that cooperation across different\nDNNs could be the key for counteracting the harmful effects of noise and\nreverberation, we propose a novel deep learning paradigm called network of deep\nneural networks. The analysis of the original concepts were based on extensive\nexperimental validations conducted on both real and simulated data, considering\ndifferent corpora, microphone configurations, environments, noisy conditions,\nand ASR tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.06682v1",
            "date": "2017-12-18 21:25:37+00:00",
            "title": "Synthesizing Novel Pairs of Image and Text",
            "authors": [
                "Jason Xie",
                "Tingwen Bao"
            ],
            "abstract": "Generating novel pairs of image and text is a problem that combines computer\nvision and natural language processing. In this paper, we present strategies\nfor generating novel image and caption pairs based on existing captioning\ndatasets. The model takes advantage of recent advances in generative\nadversarial networks and sequence-to-sequence modeling. We make generalizations\nto generate paired samples from multiple domains. Furthermore, we study cycles\n-- generating from image to text then back to image and vise versa, as well as\nits connection with autoencoders.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.06704v1",
            "date": "2017-12-18 22:45:20+00:00",
            "title": "Multilingual Topic Models",
            "authors": [
                "Kriste Krstovski",
                "Michael J. Kurtz",
                "David A. Smith",
                "Alberto Accomazzi"
            ],
            "abstract": "Scientific publications have evolved several features for mitigating\nvocabulary mismatch when indexing, retrieving, and computing similarity between\narticles. These mitigation strategies range from simply focusing on high-value\narticle sections, such as titles and abstracts, to assigning keywords, often\nfrom controlled vocabularies, either manually or through automatic annotation.\nVarious document representation schemes possess different cost-benefit\ntradeoffs. In this paper, we propose to model different representations of the\nsame article as translations of each other, all generated from a common latent\nrepresentation in a multilingual topic model. We start with a methodological\noverview on latent variable models for parallel document representations that\ncould be used across many information science tasks. We then show how solving\nthe inference problem of mapping diverse representations into a shared topic\nspace allows us to evaluate representations based on how topically similar they\nare to the original article. In addition, our proposed approach provides means\nto discover where different concept vocabularies require improvement.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1712.07004v1",
            "date": "2017-12-19 15:47:00+00:00",
            "title": "Any-gram Kernels for Sentence Classification: A Sentiment Analysis Case Study",
            "authors": [
                "Rasoul Kaljahi",
                "Jennifer Foster"
            ],
            "abstract": "Any-gram kernels are a flexible and efficient way to employ bag-of-n-gram\nfeatures when learning from textual data. They are also compatible with the use\nof word embeddings so that word similarities can be accounted for. While the\noriginal any-gram kernels are implemented on top of tree kernels, we propose a\nnew approach which is independent of tree kernels and is more efficient. We\nalso propose a more effective way to make use of word embeddings than the\noriginal any-gram formulation. When applied to the task of sentiment\nclassification, our new formulation achieves significantly better performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07040v1",
            "date": "2017-12-19 16:48:05+00:00",
            "title": "The NarrativeQA Reading Comprehension Challenge",
            "authors": [
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Jonathan Schwarz",
                "Phil Blunsom",
                "Chris Dyer",
                "Karl Moritz Hermann",
                "G\u00e1bor Melis",
                "Edward Grefenstette"
            ],
            "abstract": "Reading comprehension (RC)---in contrast to information retrieval---requires\nintegrating information and reasoning about events, entities, and their\nrelations across a full document. Question answering is conventionally used to\nassess RC ability, in both artificial agents and children learning to read.\nHowever, existing RC datasets and tasks are dominated by questions that can be\nsolved by selecting answers using superficial information (e.g., local context\nsimilarity or global term frequency); they thus fail to test for the essential\nintegrative aspect of RC. To encourage progress on deeper comprehension of\nlanguage, we present a new dataset and set of tasks in which the reader must\nanswer questions about stories by reading entire books or movie scripts. These\ntasks are designed so that successfully answering their questions requires\nunderstanding the underlying narrative rather than relying on shallow pattern\nmatching or salience. We show that although humans solve the tasks easily,\nstandard RC models struggle on the tasks presented here. We provide an analysis\nof the dataset and the challenges it presents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07101v1",
            "date": "2017-12-19 18:39:50+00:00",
            "title": "Improving End-to-End Speech Recognition with Policy Learning",
            "authors": [
                "Yingbo Zhou",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Connectionist temporal classification (CTC) is widely used for maximum\nlikelihood learning in end-to-end speech recognition models. However, there is\nusually a disparity between the negative maximum likelihood and the performance\nmetric used in speech recognition, e.g., word error rate (WER). This results in\na mismatch between the objective function and metric during training. We show\nthat the above problem can be mitigated by jointly training with maximum\nlikelihood and policy gradient. In particular, with policy learning we are able\nto directly optimize on the (otherwise non-differentiable) performance metric.\nWe show that joint training improves relative performance by 4% to 13% for our\nend-to-end model as compared to the same model learned through maximum\nlikelihood. The model achieves 5.53% WER on Wall Street Journal dataset, and\n5.42% and 14.70% on Librispeech test-clean and test-other set, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07108v1",
            "date": "2017-12-19 18:45:25+00:00",
            "title": "Improved Regularization Techniques for End-to-End Speech Recognition",
            "authors": [
                "Yingbo Zhou",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Regularization is important for end-to-end speech models, since the models\nare highly flexible and easy to overfit. Data augmentation and dropout has been\nimportant for improving end-to-end models in other domains. However, they are\nrelatively under explored for end-to-end speech models. Therefore, we\ninvestigate the effectiveness of both methods for end-to-end trainable, deep\nspeech recognition models. We augment audio data through random perturbations\nof tempo, pitch, volume, temporal alignment, and adding random noise.We further\ninvestigate the effect of dropout when applied to the inputs of all layers of\nthe network. We show that the combination of data augmentation and dropout give\na relative performance improvement on both Wall Street Journal (WSJ) and\nLibriSpeech dataset of over 20%. Our model performance is also competitive with\nother end-to-end speech models on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07199v1",
            "date": "2017-12-19 20:49:26+00:00",
            "title": "Cognitive Database: A Step towards Endowing Relational Databases with Artificial Intelligence Capabilities",
            "authors": [
                "Rajesh Bordawekar",
                "Bortik Bandyopadhyay",
                "Oded Shmueli"
            ],
            "abstract": "We propose Cognitive Databases, an approach for transparently enabling\nArtificial Intelligence (AI) capabilities in relational databases. A novel\naspect of our design is to first view the structured data source as meaningful\nunstructured text, and then use the text to build an unsupervised neural\nnetwork model using a Natural Language Processing (NLP) technique called word\nembedding. This model captures the hidden inter-/intra-column relationships\nbetween database tokens of different types. For each database token, the model\nincludes a vector that encodes contextual semantic relationships. We seamlessly\nintegrate the word embedding model into existing SQL query infrastructure and\nuse it to enable a new class of SQL-based analytics queries called cognitive\nintelligence (CI) queries. CI queries use the model vectors to enable complex\nqueries such as semantic matching, inductive reasoning queries such as\nanalogies, predictive queries using entities not present in a database, and,\nmore generally, using knowledge from external sources. We demonstrate unique\ncapabilities of Cognitive Databases using an Apache Spark based prototype to\nexecute inductive reasoning CI queries over a multi-modal database containing\ntext and images. We believe our first-of-a-kind system exemplifies using AI\nfunctionality to endow relational databases with capabilities that were\npreviously very hard to realize in practice.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1712.07316v1",
            "date": "2017-12-20 04:20:40+00:00",
            "title": "A Flexible Approach to Automated RNN Architecture Generation",
            "authors": [
                "Martin Schrimpf",
                "Stephen Merity",
                "James Bradbury",
                "Richard Socher"
            ],
            "abstract": "The process of designing neural architectures requires expert knowledge and\nextensive trial and error. While automated architecture search may simplify\nthese requirements, the recurrent neural network (RNN) architectures generated\nby existing methods are limited in both flexibility and components. We propose\na domain-specific language (DSL) for use in automated architecture search which\ncan produce novel RNNs of arbitrary depth and width. The DSL is flexible enough\nto define standard architectures such as the Gated Recurrent Unit and Long\nShort Term Memory and allows the introduction of non-standard RNN components\nsuch as trigonometric curves and layer normalization. Using two different\ncandidate generation techniques, random search with a ranking function and\nreinforcement learning, we explore the novel architectures produced by the RNN\nDSL for language modeling and machine translation domains. The resulting\narchitectures do not follow human intuition yet perform well on their targeted\ntasks, suggesting the space of usable RNN architectures is far larger than\npreviously assumed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.07473v3",
            "date": "2017-12-20 13:28:13+00:00",
            "title": "Differentially Private Distributed Learning for Language Modeling Tasks",
            "authors": [
                "Vadim Popov",
                "Mikhail Kudinov",
                "Irina Piontkovskaya",
                "Petr Vytovtov",
                "Alex Nevidomsky"
            ],
            "abstract": "One of the big challenges in machine learning applications is that training\ndata can be different from the real-world data faced by the algorithm. In\nlanguage modeling, users' language (e.g. in private messaging) could change in\na year and be completely different from what we observe in publicly available\ndata. At the same time, public data can be used for obtaining general knowledge\n(i.e. general model of English). We study approaches to distributed fine-tuning\nof a general model on user private data with the additional requirements of\nmaintaining the quality on the general data and minimization of communication\ncosts. We propose a novel technique that significantly improves prediction\nquality on users' language compared to a general model and outperforms gradient\ncompression methods in terms of communication efficiency. The proposed\nprocedure is fast and leads to an almost 70% perplexity reduction and 8.7\npercentage point improvement in keystroke saving rate on informal English\ntexts. We also show that the range of tasks our approach is applicable to is\nnot limited by language modeling only. Finally, we propose an experimental\nframework for evaluating differential privacy of distributed training of\nlanguage models and show that our approach has good privacy guarantees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08647v2",
            "date": "2017-12-22 20:27:11+00:00",
            "title": "Emo, Love, and God: Making Sense of Urban Dictionary, a Crowd-Sourced Online Dictionary",
            "authors": [
                "Dong Nguyen",
                "Barbara McGillivray",
                "Taha Yasseri"
            ],
            "abstract": "The Internet facilitates large-scale collaborative projects and the emergence\nof Web 2.0 platforms, where producers and consumers of content unify, has\ndrastically changed the information market. On the one hand, the promise of the\n\"wisdom of the crowd\" has inspired successful projects such as Wikipedia, which\nhas become the primary source of crowd-based information in many languages. On\nthe other hand, the decentralized and often un-monitored environment of such\nprojects may make them susceptible to low quality content. In this work, we\nfocus on Urban Dictionary, a crowd-sourced online dictionary. We combine\ncomputational methods with qualitative annotation and shed light on the overall\nfeatures of Urban Dictionary in terms of growth, coverage and types of content.\nWe measure a high presence of opinion-focused entries, as opposed to the\nmeaning-focused entries that we expect from traditional dictionaries.\nFurthermore, Urban Dictionary covers many informal, unfamiliar words as well as\nproper nouns. Urban Dictionary also contains offensive content, but highly\noffensive content tends to receive lower scores through the dictionary's voting\nsystem. The low threshold to include new material in Urban Dictionary enables\nquick recording of new words and new meanings, but the resulting heterogeneous\ncontent can pose challenges in using Urban Dictionary as a source to study\nlanguage innovation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.08697v2",
            "date": "2017-12-23 01:44:45+00:00",
            "title": "Interpretable Counting for Visual Question Answering",
            "authors": [
                "Alexander Trott",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Questions that require counting a variety of objects in images remain a major\nchallenge in visual question answering (VQA). The most common approaches to VQA\ninvolve either classifying answers based on fixed length representations of\nboth the image and question or summing fractional counts estimated from each\nsection of the image. In contrast, we treat counting as a sequential decision\nprocess and force our model to make discrete choices of what to count.\nSpecifically, the model sequentially selects from detected objects and learns\ninteractions between objects that influence subsequent selections. A\ndistinction of our approach is its intuitive and interpretable output, as\ndiscrete counts are automatically grounded in the image. Furthermore, our\nmethod outperforms the state of the art architecture for VQA on multiple\nmetrics that evaluate counting.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1712.08992v2",
            "date": "2017-12-25 02:28:32+00:00",
            "title": "Leveraging Native Language Speech for Accent Identification using Deep Siamese Networks",
            "authors": [
                "Aditya Siddhant",
                "Preethi Jyothi",
                "Sriram Ganapathy"
            ],
            "abstract": "The problem of automatic accent identification is important for several\napplications like speaker profiling and recognition as well as for improving\nspeech recognition systems. The accented nature of speech can be primarily\nattributed to the influence of the speaker's native language on the given\nspeech recording. In this paper, we propose a novel accent identification\nsystem whose training exploits speech in native languages along with the\naccented speech. Specifically, we develop a deep Siamese network-based model\nwhich learns the association between accented speech recordings and the native\nlanguage speech recordings. The Siamese networks are trained with i-vector\nfeatures extracted from the speech recordings using either an unsupervised\nGaussian mixture model (GMM) or a supervised deep neural network (DNN) model.\nWe perform several accent identification experiments using the CSLU Foreign\nAccented English (FAE) corpus. In these experiments, our proposed approach\nusing deep Siamese networks yield significant relative performance improvements\nof 15.4 percent on a 10-class accent identification task, over a baseline\nDNN-based classification system that uses GMM i-vectors. Furthermore, we\npresent a detailed error analysis of the proposed accent identification system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09662v1",
            "date": "2017-12-27 19:49:09+00:00",
            "title": "CNN Is All You Need",
            "authors": [
                "Qiming Chen",
                "Ren Wu"
            ],
            "abstract": "The Convolution Neural Network (CNN) has demonstrated the unique advantage in\naudio, image and text learning; recently it has also challenged Recurrent\nNeural Networks (RNNs) with long short-term memory cells (LSTM) in\nsequence-to-sequence learning, since the computations involved in CNN are\neasily parallelizable whereas those involved in RNN are mostly sequential,\nleading to a performance bottleneck. However, unlike RNN, the native CNN lacks\nthe history sensitivity required for sequence transformation; therefore\nenhancing the sequential order awareness, or position-sensitivity, becomes the\nkey to make CNN the general deep learning model. In this work we introduce an\nextended CNN model with strengthen position-sensitivity, called PoseNet. A\nnotable feature of PoseNet is the asymmetric treatment of position information\nin the encoder and the decoder. Experiments shows that PoseNet allows us to\nimprove the accuracy of CNN based sequence-to-sequence learning significantly,\nachieving around 33-36 BLEU scores on the WMT 2014 English-to-German\ntranslation task, and around 44-46 BLEU scores on the English-to-French\ntranslation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.09687v1",
            "date": "2017-12-27 21:09:36+00:00",
            "title": "Combining Representation Learning with Logic for Language Processing",
            "authors": [
                "Tim Rockt\u00e4schel"
            ],
            "abstract": "The current state-of-the-art in many natural language processing and\nautomated knowledge base completion tasks is held by representation learning\nmethods which learn distributed vector representations of symbols via\ngradient-based optimization. They require little or no hand-crafted features,\nthus avoiding the need for most preprocessing steps and task-specific\nassumptions. However, in many cases representation learning requires a large\namount of annotated training data to generalize well to unseen data. Such\nlabeled training data is provided by human annotators who often use formal\nlogic as the language for specifying annotations. This thesis investigates\ndifferent combinations of representation learning methods with logic for\nreducing the need for annotated training data, and for improving\ngeneralization.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1712.09943v3",
            "date": "2017-12-28 17:21:42+00:00",
            "title": "Toward Continual Learning for Conversational Agents",
            "authors": [
                "Sungjin Lee"
            ],
            "abstract": "While end-to-end neural conversation models have led to promising advances in\nreducing hand-crafted features and errors induced by the traditional complex\nsystem architecture, they typically require an enormous amount of data due to\nthe lack of modularity. Previous studies adopted a hybrid approach with\nknowledge-based components either to abstract out domain-specific information\nor to augment data to cover more diverse patterns. On the contrary, we propose\nto directly address the problem using recent developments in the space of\ncontinual learning for neural models. Specifically, we adopt a\ndomain-independent neural conversational model and introduce a novel neural\ncontinual learning algorithm that allows a conversational agent to accumulate\nskills across different tasks in a data-efficient way. To the best of our\nknowledge, this is the first work that applies continual learning to\nconversation systems. We verified the efficacy of our method through a\nconversational skill transfer from either synthetic dialogs or human-human\ndialogs to human-computer conversations in a customer support domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.00860v1",
            "date": "2017-02-02 22:51:04+00:00",
            "title": "Topic Modeling the H\u00e0n di\u0103n Ancient Classics",
            "authors": [
                "Colin Allen",
                "Hongliang Luo",
                "Jaimie Murdock",
                "Jianghuai Pu",
                "Xiaohong Wang",
                "Yanjie Zhai",
                "Kun Zhao"
            ],
            "abstract": "Ancient Chinese texts present an area of enormous challenge and opportunity\nfor humanities scholars interested in exploiting computational methods to\nassist in the development of new insights and interpretations of culturally\nsignificant materials. In this paper we describe a collaborative effort between\nIndiana University and Xi'an Jiaotong University to support exploration and\ninterpretation of a digital corpus of over 18,000 ancient Chinese documents,\nwhich we refer to as the \"Handian\" ancient classics corpus (H\\`an di\\u{a}n\ng\\u{u} j\\'i, i.e, the \"Han canon\" or \"Chinese classics\"). It contains classics\nof ancient Chinese philosophy, documents of historical and biographical\nsignificance, and literary works. We begin by describing the Digital Humanities\ncontext of this joint project, and the advances in humanities computing that\nmade this project feasible. We describe the corpus and introduce our\napplication of probabilistic topic modeling to this corpus, with attention to\nthe particular challenges posed by modeling ancient Chinese documents. We give\na specific example of how the software we have developed can be used to aid\ndiscovery and interpretation of themes in the corpus. We outline more advanced\nforms of computer-aided interpretation that are also made possible by the\nprogramming interface provided by our system, and the general implications of\nthese methods for understanding the nature of meaning in these texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1702.07717v2",
            "date": "2017-02-24 19:00:01+00:00",
            "title": "When confidence and competence collide: Effects on online decision-making discussions",
            "authors": [
                "Liye Fu",
                "Lillian Lee",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Group discussions are a way for individuals to exchange ideas and arguments\nin order to reach better decisions than they could on their own. One of the\npremises of productive discussions is that better solutions will prevail, and\nthat the idea selection process is mediated by the (relative) competence of the\nindividuals involved. However, since people may not know their actual\ncompetence on a new task, their behavior is influenced by their self-estimated\ncompetence --- that is, their confidence --- which can be misaligned with their\nactual competence.\n  Our goal in this work is to understand the effects of confidence-competence\nmisalignment on the dynamics and outcomes of discussions. To this end, we\ndesign a large-scale natural setting, in the form of an online team-based\ngeography game, that allows us to disentangle confidence from competence and\nthus separate their effects.\n  We find that in task-oriented discussions, the more-confident individuals\nhave a larger impact on the group's decisions even when these individuals are\nat the same level of competence as their teammates. Furthermore, this\nunjustified role of confidence in the decision-making process often leads teams\nto under-perform. We explore this phenomenon by investigating the effects of\nconfidence on conversational dynamics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1703.02819v1",
            "date": "2017-03-08 12:53:21+00:00",
            "title": "Introduction to Formal Concept Analysis and Its Applications in Information Retrieval and Related Fields",
            "authors": [
                "Dmitry I. Ignatov"
            ],
            "abstract": "This paper is a tutorial on Formal Concept Analysis (FCA) and its\napplications. FCA is an applied branch of Lattice Theory, a mathematical\ndiscipline which enables formalisation of concepts as basic units of human\nthinking and analysing data in the object-attribute form. Originated in early\n80s, during the last three decades, it became a popular human-centred tool for\nknowledge representation and data analysis with numerous applications. Since\nthe tutorial was specially prepared for RuSSIR 2014, the covered FCA topics\ninclude Information Retrieval with a focus on visualisation aspects, Machine\nLearning, Data Mining and Knowledge Discovery, Text Mining and several others.",
            "journal ref": "RuSSIR 2014, Nizhniy Novgorod, Russia, CCIS vol. 505, Springer\n  42-141",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1705.02518v1",
            "date": "2017-05-06 19:21:48+00:00",
            "title": "Exploring Latent Semantic Factors to Find Useful Product Reviews",
            "authors": [
                "Subhabrata Mukherjee",
                "Kashyap Popat",
                "Gerhard Weikum"
            ],
            "abstract": "Online reviews provided by consumers are a valuable asset for e-Commerce\nplatforms, influencing potential consumers in making purchasing decisions.\nHowever, these reviews are of varying quality, with the useful ones buried deep\nwithin a heap of non-informative reviews. In this work, we attempt to\nautomatically identify review quality in terms of its helpfulness to the end\nconsumers. In contrast to previous works in this domain exploiting a variety of\nsyntactic and community-level features, we delve deep into the semantics of\nreviews as to what makes them useful, providing interpretable explanation for\nthe same. We identify a set of consistency and semantic factors, all from the\ntext, ratings, and timestamps of user-generated reviews, making our approach\ngeneralizable across all communities and domains. We explore review semantics\nin terms of several latent factors like the expertise of its author, his\njudgment about the fine-grained facets of the underlying product, and his\nwriting style. These are cast into a Hidden Markov Model -- Latent Dirichlet\nAllocation (HMM-LDA) based model to jointly infer: (i) reviewer expertise, (ii)\nitem facets, and (iii) review helpfulness. Large-scale experiments on five\nreal-world datasets from Amazon show significant improvement over\nstate-of-the-art baselines in predicting and ranking useful reviews.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.02519v1",
            "date": "2017-05-06 19:22:41+00:00",
            "title": "Item Recommendation with Evolving User Preferences and Experience",
            "authors": [
                "Subhabrata Mukherjee",
                "Hemank Lamba",
                "Gerhard Weikum"
            ],
            "abstract": "Current recommender systems exploit user and item similarities by\ncollaborative filtering. Some advanced methods also consider the temporal\nevolution of item ratings as a global background process. However, all prior\nmethods disregard the individual evolution of a user's experience level and how\nthis is expressed in the user's writing in a review community. In this paper,\nwe model the joint evolution of user experience, interest in specific item\nfacets, writing style, and rating behavior. This way we can generate individual\nrecommendations that take into account the user's maturity level (e.g.,\nrecommending art movies rather than blockbusters for a cinematography expert).\nAs only item ratings and review texts are observables, we capture the user's\nexperience and interests in a latent model learned from her reviews, vocabulary\nand writing style. We develop a generative HMM-LDA model to trace user\nevolution, where the Hidden Markov Model (HMM) traces her latent experience\nprogressing over time -- with solely user reviews and ratings as observables\nover time. The facets of a user's interest are drawn from a Latent Dirichlet\nAllocation (LDA) model derived from her reviews, as a function of her (again\nlatent) experience level. In experiments with five real-world datasets, we show\nthat our model improves the rating prediction over state-of-the-art baselines,\nby a substantial margin. We also show, in a use-case study, that our model\nperforms well in the assessment of user experience levels.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.02522v1",
            "date": "2017-05-06 19:38:33+00:00",
            "title": "People on Drugs: Credibility of User Statements in Health Communities",
            "authors": [
                "Subhabrata Mukherjee",
                "Gerhard Weikum",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Online health communities are a valuable source of information for patients\nand physicians. However, such user-generated resources are often plagued by\ninaccuracies and misinformation. In this work we propose a method for\nautomatically establishing the credibility of user-generated medical statements\nand the trustworthiness of their authors by exploiting linguistic cues and\ndistant supervision from expert sources. To this end we introduce a\nprobabilistic graphical model that jointly learns user trustworthiness,\nstatement credibility, and language objectivity. We apply this methodology to\nthe task of extracting rare or unknown side-effects of medical drugs --- this\nbeing one of the problems where large scale non-expert data has the potential\nto complement expert medical knowledge. We show that our method can reliably\nextract side-effects and filter out false statements, while identifying\ntrustworthy users that are likely to contribute valuable medical information.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.02667v2",
            "date": "2017-05-07 17:41:31+00:00",
            "title": "People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communities",
            "authors": [
                "Subhabrata Mukherjee",
                "Gerhard Weikum"
            ],
            "abstract": "Media seems to have become more partisan, often providing a biased coverage\nof news catering to the interest of specific groups. It is therefore essential\nto identify credible information content that provides an objective narrative\nof an event. News communities such as digg, reddit, or newstrust offer\nrecommendations, reviews, quality ratings, and further insights on journalistic\nworks. However, there is a complex interaction between different factors in\nsuch online communities: fairness and style of reporting, language clarity and\nobjectivity, topical perspectives (like political viewpoint), expertise and\nbias of community members, and more. This paper presents a model to\nsystematically analyze the different interactions in a news community between\nusers, news, and sources. We develop a probabilistic graphical model that\nleverages this joint interaction to identify 1) highly credible news articles,\n2) trustworthy news sources, and 3) expert users who perform the role of\n\"citizen journalists\" in the community. Our method extends CRF models to\nincorporate real-valued ratings, as some communities have very fine-grained\nscales that cannot be easily discretized without losing information. To the\nbest of our knowledge, this paper is the first full-fledged analysis of\ncredibility, trust, and expertise in news communities.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.02668v1",
            "date": "2017-05-07 17:43:01+00:00",
            "title": "Credible Review Detection with Limited Information using Consistency Analysis",
            "authors": [
                "Subhabrata Mukherjee",
                "Sourav Dutta",
                "Gerhard Weikum"
            ],
            "abstract": "Online reviews provide viewpoints on the strengths and shortcomings of\nproducts/services, influencing potential customers' purchasing decisions.\nHowever, the proliferation of non-credible reviews -- either fake (promoting/\ndemoting an item), incompetent (involving irrelevant aspects), or biased --\nentails the problem of identifying credible reviews. Prior works involve\nclassifiers harnessing rich information about items/users -- which might not be\nreadily available in several domains -- that provide only limited\ninterpretability as to why a review is deemed non-credible. This paper presents\na novel approach to address the above issues. We utilize latent topic models\nleveraging review texts, item ratings, and timestamps to derive consistency\nfeatures without relying on item/user histories, unavailable for \"long-tail\"\nitems/users. We develop models, for computing review credibility scores to\nprovide interpretable evidence for non-credible reviews, that are also\ntransferable to other domains -- addressing the scarcity of labeled data.\nExperiments on real-world datasets demonstrate improvements over\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.02669v3",
            "date": "2017-05-07 17:46:43+00:00",
            "title": "Item Recommendation with Continuous Experience Evolution of Users using Brownian Motion",
            "authors": [
                "Subhabrata Mukherjee",
                "Stephan Guennemann",
                "Gerhard Weikum"
            ],
            "abstract": "Online review communities are dynamic as users join and leave, adopt new\nvocabulary, and adapt to evolving trends. Recent work has shown that\nrecommender systems benefit from explicit consideration of user experience.\nHowever, prior work assumes a fixed number of discrete experience levels,\nwhereas in reality users gain experience and mature continuously over time.\nThis paper presents a new model that captures the continuous evolution of user\nexperience, and the resulting language model in reviews and other posts. Our\nmodel is unsupervised and combines principles of Geometric Brownian Motion,\nBrownian Motion, and Latent Dirichlet Allocation to trace a smooth temporal\nprogression of user experience and language model respectively. We develop\npractical algorithms for estimating the model parameters from data and for\ninference with our model (e.g., to recommend items). Extensive experiments with\nfive real-world datasets show that our model not only fits data better than\ndiscrete-model baselines, but also outperforms state-of-the-art methods for\npredicting item ratings.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1705.07962v2",
            "date": "2017-05-22 19:32:20+00:00",
            "title": "pix2code: Generating Code from a Graphical User Interface Screenshot",
            "authors": [
                "Tony Beltramelli"
            ],
            "abstract": "Transforming a graphical user interface screenshot created by a designer into\ncomputer code is a typical task conducted by a developer in order to build\ncustomized software, websites, and mobile applications. In this paper, we show\nthat deep learning methods can be leveraged to train a model end-to-end to\nautomatically generate code from a single input image with over 77% of accuracy\nfor three different platforms (i.e. iOS, Android and web-based technologies).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1705.08142v3",
            "date": "2017-05-23 08:58:09+00:00",
            "title": "Latent Multi-task Architecture Learning",
            "authors": [
                "Sebastian Ruder",
                "Joachim Bingel",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Multi-task learning (MTL) allows deep neural networks to learn from related\ntasks by sharing parameters with other networks. In practice, however, MTL\ninvolves searching an enormous space of possible parameter sharing\narchitectures to find (a) the layers or subspaces that benefit from sharing,\n(b) the appropriate amount of sharing, and (c) the appropriate relative weights\nof the different task losses. Recent work has addressed each of the above\nproblems in isolation. In this work we present an approach that learns a latent\nmulti-task architecture that jointly addresses (a)--(c). We present experiments\non synthetic data and data from OntoNotes 5.0, including four different tasks\nand seven different domains. Our extension consistently outperforms previous\napproaches to learning latent architectures for multi-task problems and\nachieves up to 15% average error reductions over common approaches to MTL.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1709.02349v2",
            "date": "2017-09-07 16:51:09+00:00",
            "title": "A Deep Reinforcement Learning Chatbot",
            "authors": [
                "Iulian V. Serban",
                "Chinnadhurai Sankar",
                "Mathieu Germain",
                "Saizheng Zhang",
                "Zhouhan Lin",
                "Sandeep Subramanian",
                "Taesup Kim",
                "Michael Pieper",
                "Sarath Chandar",
                "Nan Rosemary Ke",
                "Sai Rajeshwar",
                "Alexandre de Brebisson",
                "Jose M. R. Sotelo",
                "Dendi Suhubdy",
                "Vincent Michalski",
                "Alexandre Nguyen",
                "Joelle Pineau",
                "Yoshua Bengio"
            ],
            "abstract": "We present MILABOT: a deep reinforcement learning chatbot developed by the\nMontreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize\ncompetition. MILABOT is capable of conversing with humans on popular small talk\ntopics through both speech and text. The system consists of an ensemble of\nnatural language generation and retrieval models, including template-based\nmodels, bag-of-words models, sequence-to-sequence neural network and latent\nvariable neural network models. By applying reinforcement learning to\ncrowdsourced data and real-world user interactions, the system has been trained\nto select an appropriate response from the models in its ensemble. The system\nhas been evaluated through A/B testing with real-world users, where it\nperformed significantly better than many competing systems. Due to its machine\nlearning architecture, the system is likely to improve with additional data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.03968v1",
            "date": "2017-09-12 17:41:30+00:00",
            "title": "Affective Neural Response Generation",
            "authors": [
                "Nabiha Asghar",
                "Pascal Poupart",
                "Jesse Hoey",
                "Xin Jiang",
                "Lili Mou"
            ],
            "abstract": "Existing neural conversational models process natural language primarily on a\nlexico-syntactic level, thereby ignoring one of the most crucial components of\nhuman-to-human dialogue: its affective content. We take a step in this\ndirection by proposing three novel ways to incorporate affective/emotional\naspects into long short term memory (LSTM) encoder-decoder neural conversation\nmodels: (1) affective word embeddings, which are cognitively engineered, (2)\naffect-based objective functions that augment the standard cross-entropy loss,\nand (3) affectively diverse beam search for decoding. Experiments show that\nthese techniques improve the open-domain conversational prowess of\nencoder-decoder networks by enabling them to produce emotionally rich responses\nthat are more interesting and natural.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1709.07902v1",
            "date": "2017-09-22 18:36:50+00:00",
            "title": "Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data",
            "authors": [
                "Wei-Ning Hsu",
                "Yu Zhang",
                "James Glass"
            ],
            "abstract": "We present a factorized hierarchical variational autoencoder, which learns\ndisentangled and interpretable representations from sequential data without\nsupervision. Specifically, we exploit the multi-scale nature of information in\nsequential data by formulating it explicitly within a factorized hierarchical\ngraphical model that imposes sequence-dependent priors and sequence-independent\npriors to different sets of latent variables. The model is evaluated on two\nspeech corpora to demonstrate, qualitatively, its ability to transform speakers\nor linguistic content by manipulating different sets of latent variables; and\nquantitatively, its ability to outperform an i-vector baseline for speaker\nverification and reduce the word error rate by as much as 35% in mismatched\ntrain/test scenarios for automatic speech recognition tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.07915v1",
            "date": "2017-09-22 19:18:42+00:00",
            "title": "Computational Content Analysis of Negative Tweets for Obesity, Diet, Diabetes, and Exercise",
            "authors": [
                "George Shaw Jr.",
                "Amir Karami"
            ],
            "abstract": "Social media based digital epidemiology has the potential to support faster\nresponse and deeper understanding of public health related threats. This study\nproposes a new framework to analyze unstructured health related textual data\nvia Twitter users' post (tweets) to characterize the negative health sentiments\nand non-health related concerns in relations to the corpus of negative\nsentiments, regarding Diet Diabetes Exercise, and Obesity (DDEO). Through the\ncollection of 6 million Tweets for one month, this study identified the\nprominent topics of users as it relates to the negative sentiments. Our\nproposed framework uses two text mining methods, sentiment analysis and topic\nmodeling, to discover negative topics. The negative sentiments of Twitter users\nsupport the literature narratives and the many morbidity issues that are\nassociated with DDEO and the linkage between obesity and diabetes. The\nframework offers a potential method to understand the publics' opinions and\nsentiments regarding DDEO. More importantly, this research provides new\nopportunities for computational social scientists, medical experts, and public\nhealth professionals to collectively address DDEO-related issues.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1709.07916v1",
            "date": "2017-09-22 19:19:49+00:00",
            "title": "Characterizing Diabetes, Diet, Exercise, and Obesity Comments on Twitter",
            "authors": [
                "Amir Karami",
                "Alicia A. Dahl",
                "Gabrielle Turner-McGrievy",
                "Hadi Kharrazi, Jr.",
                "George Shaw"
            ],
            "abstract": "Social media provide a platform for users to express their opinions and share\ninformation. Understanding public health opinions on social media, such as\nTwitter, offers a unique approach to characterizing common health issues such\nas diabetes, diet, exercise, and obesity (DDEO), however, collecting and\nanalyzing a large scale conversational public health data set is a challenging\nresearch task. The goal of this research is to analyze the characteristics of\nthe general public's opinions in regard to diabetes, diet, exercise and obesity\n(DDEO) as expressed on Twitter. A multi-component semantic and linguistic\nframework was developed to collect Twitter data, discover topics of interest\nabout DDEO, and analyze the topics. From the extracted 4.5 million tweets, 8%\nof tweets discussed diabetes, 23.7% diet, 16.6% exercise, and 51.7% obesity.\nThe strongest correlation among the topics was determined between exercise and\nobesity. Other notable correlations were: diabetes and obesity, and diet and\nobesity DDEO terms were also identified as subtopics of each of the DDEO\ntopics. The frequent subtopics discussed along with Diabetes, excluding the\nDDEO terms themselves, were blood pressure, heart attack, yoga, and Alzheimer.\nThe non-DDEO subtopics for Diet included vegetarian, pregnancy, celebrities,\nweight loss, religious, and mental health, while subtopics for Exercise\nincluded computer games, brain, fitness, and daily plan. Non-DDEO subtopics for\nObesity included Alzheimer, cancer, and children. With 2.67 billion social\nmedia users in 2016, publicly available data such as Twitter posts can be\nutilized to support clinical providers, public health experts, and social\nscientists in better understanding common public opinions in regard to\ndiabetes, diet, exercise, and obesity.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1709.08267v2",
            "date": "2017-09-24 21:58:12+00:00",
            "title": "HDLTex: Hierarchical Deep Learning for Text Classification",
            "authors": [
                "Kamran Kowsari",
                "Donald E. Brown",
                "Mojtaba Heidarysafa",
                "Kiana Jafari Meimandi",
                "Matthew S. Gerber",
                "Laura E. Barnes"
            ],
            "abstract": "The continually increasing number of documents produced each year\nnecessitates ever improving information processing methods for searching,\nretrieving, and organizing text. Central to these information processing\nmethods is document classification, which has become an important application\nfor supervised learning. Recently the performance of these traditional\nclassifiers has degraded as the number of documents has increased. This is\nbecause along with this growth in the number of documents has come an increase\nin the number of categories. This paper approaches this problem differently\nfrom current document classification methods that view the problem as\nmulti-class classification. Instead we perform hierarchical classification\nusing an approach we call Hierarchical Deep Learning for Text classification\n(HDLTex). HDLTex employs stacks of deep learning architectures to provide\nspecialized understanding at each level of the document hierarchy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1709.08878v2",
            "date": "2017-09-26 08:11:33+00:00",
            "title": "Generating Sentences by Editing Prototypes",
            "authors": [
                "Kelvin Guu",
                "Tatsunori B. Hashimoto",
                "Yonatan Oren",
                "Percy Liang"
            ],
            "abstract": "We propose a new generative model of sentences that first samples a prototype\nsentence from the training corpus and then edits it into a new sentence.\nCompared to traditional models that generate from scratch either left-to-right\nor by first sampling a latent sentence vector, our prototype-then-edit model\nimproves perplexity on language modeling and generates higher quality outputs\naccording to human evaluation. Furthermore, the model gives rise to a latent\nedit vector that captures interpretable semantics such as sentence similarity\nand sentence-level analogies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1704.07468v3",
            "date": "2017-04-24 21:43:21+00:00",
            "title": "GaKCo: a Fast GApped k-mer string Kernel using COunting",
            "authors": [
                "Ritambhara Singh",
                "Arshdeep Sekhon",
                "Kamran Kowsari",
                "Jack Lanchantin",
                "Beilun Wang",
                "Yanjun Qi"
            ],
            "abstract": "String Kernel (SK) techniques, especially those using gapped $k$-mers as\nfeatures (gk), have obtained great success in classifying sequences like DNA,\nprotein, and text. However, the state-of-the-art gk-SK runs extremely slow when\nwe increase the dictionary size ($\\Sigma$) or allow more mismatches ($M$). This\nis because current gk-SK uses a trie-based algorithm to calculate co-occurrence\nof mismatched substrings resulting in a time cost proportional to\n$O(\\Sigma^{M})$. We propose a \\textbf{fast} algorithm for calculating\n\\underline{Ga}pped $k$-mer \\underline{K}ernel using \\underline{Co}unting\n(GaKCo). GaKCo uses associative arrays to calculate the co-occurrence of\nsubstrings using cumulative counting. This algorithm is fast, scalable to\nlarger $\\Sigma$ and $M$, and naturally parallelizable. We provide a rigorous\nasymptotic analysis that compares GaKCo with the state-of-the-art gk-SK.\nTheoretically, the time cost of GaKCo is independent of the $\\Sigma^{M}$ term\nthat slows down the trie-based approach. Experimentally, we observe that GaKCo\nachieves the same accuracy as the state-of-the-art and outperforms its speed by\nfactors of 2, 100, and 4, on classifying sequences of DNA (5 datasets), protein\n(12 datasets), and character-based English text (2 datasets), respectively.\n  GaKCo is shared as an open source tool at\n\\url{https://github.com/QData/GaKCo-SVM}",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1704.07751v1",
            "date": "2017-04-25 15:52:52+00:00",
            "title": "Fine-Grained Entity Typing with High-Multiplicity Assignments",
            "authors": [
                "Maxim Rabinovich",
                "Dan Klein"
            ],
            "abstract": "As entity type systems become richer and more fine-grained, we expect the\nnumber of types assigned to a given entity to increase. However, most\nfine-grained typing work has focused on datasets that exhibit a low degree of\ntype multiplicity. In this paper, we consider the high-multiplicity regime\ninherent in data sources such as Wikipedia that have semi-open type systems. We\nintroduce a set-prediction approach to this problem and show that our model\noutperforms unstructured baselines on a new Wikipedia-based fine-grained typing\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.01340v1",
            "date": "2017-06-02 17:05:26+00:00",
            "title": "Yeah, Right, Uh-Huh: A Deep Learning Backchannel Predictor",
            "authors": [
                "Robin Ruede",
                "Markus M\u00fcller",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "Using supporting backchannel (BC) cues can make human-computer interaction\nmore social. BCs provide a feedback from the listener to the speaker indicating\nto the speaker that he is still listened to. BCs can be expressed in different\nways, depending on the modality of the interaction, for example as gestures or\nacoustic cues. In this work, we only considered acoustic cues. We are proposing\nan approach towards detecting BC opportunities based on acoustic input features\nlike power and pitch. While other works in the field rely on the use of a\nhand-written rule set or specialized features, we made use of artificial neural\nnetworks. They are capable of deriving higher order features from input\nfeatures themselves. In our setup, we first used a fully connected feed-forward\nnetwork to establish an updated baseline in comparison to our previously\nproposed setup. We also extended this setup by the use of Long Short-Term\nMemory (LSTM) networks which have shown to outperform feed-forward based setups\non various tasks. Our best system achieved an F1-Score of 0.37 using power and\npitch features. Adding linguistic information using word2vec, the score\nincreased to 0.39.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1706.02490v1",
            "date": "2017-06-08 09:31:42+00:00",
            "title": "Where is my forearm? Clustering of body parts from simultaneous tactile and linguistic input using sequential mapping",
            "authors": [
                "Karla Stepanova",
                "Matej Hoffmann",
                "Zdenek Straka",
                "Frederico B. Klein",
                "Angelo Cangelosi",
                "Michal Vavrecka"
            ],
            "abstract": "Humans and animals are constantly exposed to a continuous stream of sensory\ninformation from different modalities. At the same time, they form more\ncompressed representations like concepts or symbols. In species that use\nlanguage, this process is further structured by this interaction, where a\nmapping between the sensorimotor concepts and linguistic elements needs to be\nestablished. There is evidence that children might be learning language by\nsimply disambiguating potential meanings based on multiple exposures to\nutterances in different contexts (cross-situational learning). In existing\nmodels, the mapping between modalities is usually found in a single step by\ndirectly using frequencies of referent and meaning co-occurrences. In this\npaper, we present an extension of this one-step mapping and introduce a newly\nproposed sequential mapping algorithm together with a publicly available Matlab\nimplementation. For demonstration, we have chosen a less typical scenario:\ninstead of learning to associate objects with their names, we focus on body\nrepresentations. A humanoid robot is receiving tactile stimulations on its\nbody, while at the same time listening to utterances of the body part names\n(e.g., hand, forearm and torso). With the goal at arriving at the correct \"body\ncategories\", we demonstrate how a sequential mapping algorithm outperforms\none-step mapping. In addition, the effect of data set size and noise in the\nlinguistic input are studied.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1706.03946v1",
            "date": "2017-06-13 08:15:25+00:00",
            "title": "A Supervised Approach to Extractive Summarisation of Scientific Papers",
            "authors": [
                "Ed Collins",
                "Isabelle Augenstein",
                "Sebastian Riedel"
            ],
            "abstract": "Automatic summarisation is a popular approach to reduce a document to its\nmain arguments. Recent research in the area has focused on neural approaches to\nsummarisation, which can be very data-hungry. However, few large datasets exist\nand none for the traditionally popular domain of scientific publications, which\nopens up challenging research avenues centered on encoding large, complex\ndocuments. In this paper, we introduce a new dataset for summarisation of\ncomputer science publications by exploiting a large resource of author provided\nsummaries and show straightforward ways of extending it further. We develop\nmodels on the dataset making use of both neural sentence encoding and\ntraditionally used summarisation features and show that models which encode\nsentences as well as their local and global context perform best, significantly\noutperforming well-established baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1708.02254v1",
            "date": "2017-08-07 18:00:32+00:00",
            "title": "Asking Too Much? The Rhetorical Role of Questions in Political Discourse",
            "authors": [
                "Justine Zhang",
                "Arthur Spirling",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Questions play a prominent role in social interactions, performing rhetorical\nfunctions that go beyond that of simple informational exchange. The surface\nform of a question can signal the intention and background of the person asking\nit, as well as the nature of their relation with the interlocutor. While the\ninformational nature of questions has been extensively examined in the context\nof question-answering applications, their rhetorical aspects have been largely\nunderstudied.\n  In this work we introduce an unsupervised methodology for extracting surface\nmotifs that recur in questions, and for grouping them according to their latent\nrhetorical role. By applying this framework to the setting of question sessions\nin the UK parliament, we show that the resulting typology encodes key aspects\nof the political discourse---such as the bifurcation in questioning behavior\nbetween government and opposition parties---and reveals new insights into the\neffects of a legislator's tenure and political career ambitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1710.07654v3",
            "date": "2017-10-20 18:17:23+00:00",
            "title": "Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning",
            "authors": [
                "Wei Ping",
                "Kainan Peng",
                "Andrew Gibiansky",
                "Sercan O. Arik",
                "Ajay Kannan",
                "Sharan Narang",
                "Jonathan Raiman",
                "John Miller"
            ],
            "abstract": "We present Deep Voice 3, a fully-convolutional attention-based neural\ntext-to-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural\nspeech synthesis systems in naturalness while training ten times faster. We\nscale Deep Voice 3 to data set sizes unprecedented for TTS, training on more\nthan eight hundred hours of audio from over two thousand speakers. In addition,\nwe identify common error modes of attention-based speech synthesis networks,\ndemonstrate how to mitigate them, and compare several different waveform\nsynthesis methods. We also describe how to scale inference to ten million\nqueries per day on one single-GPU server.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1710.10248v2",
            "date": "2017-10-27 17:26:57+00:00",
            "title": "Tensor network language model",
            "authors": [
                "Vasily Pestun",
                "Yiannis Vlassopoulos"
            ],
            "abstract": "We propose a new statistical model suitable for machine learning of systems\nwith long distance correlations such as natural languages. The model is based\non directed acyclic graph decorated by multi-linear tensor maps in the vertices\nand vector spaces in the edges, called tensor network. Such tensor networks\nhave been previously employed for effective numerical computation of the\nrenormalization group flow on the space of effective quantum field theories and\nlattice models of statistical mechanics. We provide explicit algebro-geometric\nanalysis of the parameter moduli space for tree graphs, discuss model\nproperties and applications such as statistical translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.00781v2",
            "date": "2017-07-03 23:32:55+00:00",
            "title": "Mapping the Americanization of English in Space and Time",
            "authors": [
                "Bruno Gon\u00e7alves",
                "Luc\u00eda Loureiro-Porto",
                "Jos\u00e9 J. Ramasco",
                "David S\u00e1nchez"
            ],
            "abstract": "As global political preeminence gradually shifted from the United Kingdom to\nthe United States, so did the capacity to culturally influence the rest of the\nworld. In this work, we analyze how the world-wide varieties of written English\nare evolving. We study both the spatial and temporal variations of vocabulary\nand spelling of English using a large corpus of geolocated tweets and the\nGoogle Books datasets corresponding to books published in the US and the UK.\nThe advantage of our approach is that we can address both standard written\nlanguage (Google Books) and the more colloquial forms of microblogging messages\n(Twitter). We find that American English is the dominant form of English\noutside the UK and that its influence is felt even within the UK borders.\nFinally, we analyze how this trend has evolved over time and the impact that\nsome cultural events have had in shaping it.",
            "journal ref": "PLoS ONE 13: e0197741 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.04218v1",
            "date": "2017-07-13 16:46:50+00:00",
            "title": "Learning Features from Co-occurrences: A Theoretical Analysis",
            "authors": [
                "Yanpeng Li"
            ],
            "abstract": "Representing a word by its co-occurrences with other words in context is an\neffective way to capture the meaning of the word. However, the theory behind\nremains a challenge. In this work, taking the example of a word classification\ntask, we give a theoretical analysis of the approaches that represent a word X\nby a function f(P(C|X)), where C is a context feature, P(C|X) is the\nconditional probability estimated from a text corpus, and the function f maps\nthe co-occurrence measure to a prediction score. We investigate the impact of\ncontext feature C and the function f. We also explain the reasons why using the\nco-occurrences with multiple context features may be better than just using a\nsingle one. In addition, some of the results shed light on the theory of\nfeature learning and machine learning in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1707.05005v1",
            "date": "2017-07-17 05:09:03+00:00",
            "title": "graph2vec: Learning Distributed Representations of Graphs",
            "authors": [
                "Annamalai Narayanan",
                "Mahinthan Chandramohan",
                "Rajasekar Venkatesan",
                "Lihui Chen",
                "Yang Liu",
                "Shantanu Jaiswal"
            ],
            "abstract": "Recent works on representation learning for graph structured data\npredominantly focus on learning distributed representations of graph\nsubstructures such as nodes and subgraphs. However, many graph analytics tasks\nsuch as graph classification and clustering require representing entire graphs\nas fixed length feature vectors. While the aforementioned approaches are\nnaturally unequipped to learn such representations, graph kernels remain as the\nmost effective way of obtaining them. However, these graph kernels use\nhandcrafted features (e.g., shortest paths, graphlets, etc.) and hence are\nhampered by problems such as poor generalization. To address this limitation,\nin this work, we propose a neural embedding framework named graph2vec to learn\ndata-driven distributed representations of arbitrary sized graphs. graph2vec's\nembeddings are learnt in an unsupervised manner and are task agnostic. Hence,\nthey could be used for any downstream task such as graph classification,\nclustering and even seeding supervised representation learning approaches. Our\nexperiments on several benchmark and large real-world datasets show that\ngraph2vec achieves significant improvements in classification and clustering\naccuracies over substructure representation learning approaches and are\ncompetitive with state-of-the-art graph kernels.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1707.08309v1",
            "date": "2017-07-26 07:41:27+00:00",
            "title": "Probabilistic Graphical Models for Credibility Analysis in Evolving Online Communities",
            "authors": [
                "Subhabrata Mukherjee"
            ],
            "abstract": "One of the major hurdles preventing the full exploitation of information from\nonline communities is the widespread concern regarding the quality and\ncredibility of user-contributed content. Prior works in this domain operate on\na static snapshot of the community, making strong assumptions about the\nstructure of the data (e.g., relational tables), or consider only shallow\nfeatures for text classification.\n  To address the above limitations, we propose probabilistic graphical models\nthat can leverage the joint interplay between multiple factors in online\ncommunities --- like user interactions, community dynamics, and textual content\n--- to automatically assess the credibility of user-contributed online content,\nand the expertise of users and their evolution with user-interpretable\nexplanation. To this end, we devise new models based on Conditional Random\nFields for different settings like incorporating partial expert knowledge for\nsemi-supervised learning, and handling discrete labels as well as numeric\nratings for fine-grained analysis. This enables applications such as extracting\nreliable side-effects of drugs from user-contributed posts in healthforums, and\nidentifying credible content in news communities.\n  Online communities are dynamic, as users join and leave, adapt to evolving\ntrends, and mature over time. To capture this dynamics, we propose generative\nmodels based on Hidden Markov Model, Latent Dirichlet Allocation, and Brownian\nMotion to trace the continuous evolution of user expertise and their language\nmodel over time. This allows us to identify expert users and credible content\njointly over time, improving state-of-the-art recommender systems by explicitly\nconsidering the maturity of users. This also enables applications such as\nidentifying helpful product reviews, and detecting fake and anomalous reviews\nwith limited information.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1707.08559v1",
            "date": "2017-07-26 17:44:38+00:00",
            "title": "Video Highlight Prediction Using Audience Chat Reactions",
            "authors": [
                "Cheng-Yang Fu",
                "Joon Lee",
                "Mohit Bansal",
                "Alexander C. Berg"
            ],
            "abstract": "Sports channel video portals offer an exciting domain for research on\nmultimodal, multilingual analysis. We present methods addressing the problem of\nautomatic video highlight prediction based on joint visual features and textual\nanalysis of the real-world audience discourse with complex slang, in both\nEnglish and traditional Chinese. We present a novel dataset based on League of\nLegends championships recorded from North American and Taiwanese Twitch.tv\nchannels (will be released for further research), and demonstrate strong\nresults on these using multimodal, character-level CNN-RNN model architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01416v1",
            "date": "2017-11-04 09:11:18+00:00",
            "title": "Language as a matrix product state",
            "authors": [
                "Vasily Pestun",
                "John Terilla",
                "Yiannis Vlassopoulos"
            ],
            "abstract": "We propose a statistical model for natural language that begins by\nconsidering language as a monoid, then representing it in complex matrices with\na compatible translation invariant probability measure. We interpret the\nprobability measure as arising via the Born rule from a translation invariant\nmatrix product state.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1711.01921v3",
            "date": "2017-11-06 14:54:56+00:00",
            "title": "$A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation",
            "authors": [
                "Rakshith Shetty",
                "Bernt Schiele",
                "Mario Fritz"
            ],
            "abstract": "Text-based analysis methods allow to reveal privacy relevant author\nattributes such as gender, age and identify of the text's author. Such methods\ncan compromise the privacy of an anonymous author even when the author tries to\nremove privacy sensitive content. In this paper, we propose an automatic\nmethod, called Adversarial Author Attribute Anonymity Neural Translation\n($A^4NT$), to combat such text-based adversaries. We combine\nsequence-to-sequence language models used in machine translation and generative\nadversarial networks to obfuscate author attributes. Unlike machine translation\ntechniques which need paired data, our method can be trained on unpaired\ncorpora of text containing different authors. Importantly, we propose and\nevaluate techniques to impose constraints on our $A^4NT$ to preserve the\nsemantics of the input text. $A^4NT$ learns to make minimal changes to the\ninput text to successfully fool author attribute classifiers, while aiming to\nmaintain the meaning of the input. We show through experiments on two different\ndatasets and three settings that our proposed method is effective in fooling\nthe author attribute classifiers and thereby improving the anonymity of\nauthors.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1711.07128v3",
            "date": "2017-11-20 03:19:03+00:00",
            "title": "Hello Edge: Keyword Spotting on Microcontrollers",
            "authors": [
                "Yundong Zhang",
                "Naveen Suda",
                "Liangzhen Lai",
                "Vikas Chandra"
            ],
            "abstract": "Keyword spotting (KWS) is a critical component for enabling speech based user\ninteractions on smart devices. It requires real-time response and high accuracy\nfor good user experience. Recently, neural networks have become an attractive\nchoice for KWS architecture because of their superior accuracy compared to\ntraditional speech processing algorithms. Due to its always-on nature, KWS\napplication has highly constrained power budget and typically runs on tiny\nmicrocontrollers with limited memory and compute capability. The design of\nneural network architecture for KWS must consider these constraints. In this\nwork, we perform neural network architecture evaluation and exploration for\nrunning KWS on resource-constrained microcontrollers. We train various neural\nnetwork architectures for keyword spotting published in literature to compare\ntheir accuracy and memory/compute requirements. We show that it is possible to\noptimize these neural network architectures to fit within the memory and\ncompute constraints of microcontrollers without sacrificing accuracy. We\nfurther explore the depthwise separable convolutional neural network (DS-CNN)\nand compare it against other neural network architectures. DS-CNN achieves an\naccuracy of 95.4%, which is ~10% higher than the DNN model with similar number\nof parameters.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1711.08992v2",
            "date": "2017-11-24 14:45:06+00:00",
            "title": "Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition",
            "authors": [
                "Kalin Stefanov",
                "Jonas Beskow",
                "Giampiero Salvi"
            ],
            "abstract": "This paper presents a self-supervised method for visual detection of the\nactive speaker in a multi-person spoken interaction scenario. Active speaker\ndetection is a fundamental prerequisite for any artificial cognitive system\nattempting to acquire language in social settings. The proposed method is\nintended to complement the acoustic detection of the active speaker, thus\nimproving the system robustness in noisy conditions. The method can detect an\narbitrary number of possibly overlapping active speakers based exclusively on\nvisual information about their face. Furthermore, the method does not rely on\nexternal annotations, thus complying with cognitive development. Instead, the\nmethod uses information from the auditory modality to support learning in the\nvisual domain. This paper reports an extensive evaluation of the proposed\nmethod using a large multi-person face-to-face interaction dataset. The results\nshow good performance in a speaker dependent setting. However, in a speaker\nindependent setting the proposed method yields a significantly lower\nperformance. We believe that the proposed method represents an essential\ncomponent of any artificial cognitive system or robotic platform engaging in\nsocial interactions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1711.09055v1",
            "date": "2017-11-24 17:34:32+00:00",
            "title": "Interactive Robot Learning of Gestures, Language and Affordances",
            "authors": [
                "Giovanni Saponaro",
                "Lorenzo Jamone",
                "Alexandre Bernardino",
                "Giampiero Salvi"
            ],
            "abstract": "A growing field in robotics and Artificial Intelligence (AI) research is\nhuman-robot collaboration, whose target is to enable effective teamwork between\nhumans and robots. However, in many situations human teams are still superior\nto human-robot teams, primarily because human teams can easily agree on a\ncommon goal with language, and the individual members observe each other\neffectively, leveraging their shared motor repertoire and sensorimotor\nresources. This paper shows that for cognitive robots it is possible, and\nindeed fruitful, to combine knowledge acquired from interacting with elements\nof the environment (affordance exploration) with the probabilistic observation\nof another agent's actions.\n  We propose a model that unites (i) learning robot affordances and word\ndescriptions with (ii) statistical recognition of human gestures with vision\nsensors. We discuss theoretical motivations, possible implementations, and we\nshow initial results which highlight that, after having acquired knowledge of\nits surrounding environment, a humanoid robot can generalize this knowledge to\nthe case when it observes another agent (human partner) performing the same\nmotor actions previously executed during training.",
            "journal ref": "International Workshop on Grounding Language Understanding (GLU),\n  Satellite of Interspeech 2017",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1712.00334v1",
            "date": "2017-11-30 08:27:08+00:00",
            "title": "Enabling Embodied Analogies in Intelligent Music Systems",
            "authors": [
                "Fabio Paolizzo"
            ],
            "abstract": "The present methodology is aimed at cross-modal machine learning and uses\nmultidisciplinary tools and methods drawn from a broad range of areas and\ndisciplines, including music, systematic musicology, dance, motion capture,\nhuman-computer interaction, computational linguistics and audio signal\nprocessing. Main tasks include: (1) adapting wisdom-of-the-crowd approaches to\nembodiment in music and dance performance to create a dataset of music and\nmusic lyrics that covers a variety of emotions, (2) applying\naudio/language-informed machine learning techniques to that dataset to identify\nautomatically the emotional content of the music and the lyrics, and (3)\nintegrating motion capture data from a Vicon system and dancers performing on\nthat music.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1712.00489v1",
            "date": "2017-12-01 20:56:31+00:00",
            "title": "Visual Features for Context-Aware Speech Recognition",
            "authors": [
                "Abhinav Gupta",
                "Yajie Miao",
                "Leonardo Neves",
                "Florian Metze"
            ],
            "abstract": "Automatic transcriptions of consumer-generated multi-media content such as\n\"Youtube\" videos still exhibit high word error rates. Such data typically\noccupies a very broad domain, has been recorded in challenging conditions, with\ncheap hardware and a focus on the visual modality, and may have been\npost-processed or edited. In this paper, we extend our earlier work on adapting\nthe acoustic model of a DNN-based speech recognition system to an RNN language\nmodel and show how both can be adapted to the objects and scenes that can be\nautomatically detected in the video. We are working on a corpus of \"how-to\"\nvideos from the web, and the idea is that an object that can be seen (\"car\"),\nor a scene that is being detected (\"kitchen\") can be used to condition both\nmodels on the \"context\" of the recording, thereby reducing perplexity and\nimproving transcription. We achieve good improvements in both cases and compare\nand analyze the respective reductions in word error rate. We expect that our\nresults can be used for any type of speech processing in which \"context\"\ninformation is available, for example in robotics, man-machine interaction, or\nwhen indexing large audio-visual archives, and should ultimately help to bring\ntogether the \"video-to-text\" and \"speech-to-text\" communities.",
            "journal ref": "IEEE Xplore (ICASSP) (2017) 5020-5024",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.00725v1",
            "date": "2017-12-03 07:20:15+00:00",
            "title": "Sentiment Classification using Images and Label Embeddings",
            "authors": [
                "Laura Graesser",
                "Abhinav Gupta",
                "Lakshay Sharma",
                "Evelina Bakhturina"
            ],
            "abstract": "In this project we analysed how much semantic information images carry, and\nhow much value image data can add to sentiment analysis of the text associated\nwith the images. To better understand the contribution from images, we compared\nmodels which only made use of image data, models which only made use of text\ndata, and models which combined both data types. We also analysed if this\napproach could help sentiment classifiers generalize to unknown sentiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1712.01329v1",
            "date": "2017-12-04 20:16:52+00:00",
            "title": "Examining Cooperation in Visual Dialog Models",
            "authors": [
                "Mircea Mironenco",
                "Dana Kianfar",
                "Ke Tran",
                "Evangelos Kanoulas",
                "Efstratios Gavves"
            ],
            "abstract": "In this work we propose a blackbox intervention method for visual dialog\nmodels, with the aim of assessing the contribution of individual linguistic or\nvisual components. Concretely, we conduct structured or randomized\ninterventions that aim to impair an individual component of the model, and\nobserve changes in task performance. We reproduce a state-of-the-art visual\ndialog model and demonstrate that our methodology yields surprising insights,\nnamely that both dialog and image information have minimal contributions to\ntask performance. The intervention method presented here can be applied as a\nsanity check for the strength and robustness of each component in visual dialog\nsystems.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1712.05997v1",
            "date": "2017-12-16 17:57:57+00:00",
            "title": "Taming Wild High Dimensional Text Data with a Fuzzy Lash",
            "authors": [
                "Amir Karami"
            ],
            "abstract": "The bag of words (BOW) represents a corpus in a matrix whose elements are the\nfrequency of words. However, each row in the matrix is a very high-dimensional\nsparse vector. Dimension reduction (DR) is a popular method to address sparsity\nand high-dimensionality issues. Among different strategies to develop DR\nmethod, Unsupervised Feature Transformation (UFT) is a popular strategy to map\nall words on a new basis to represent BOW. The recent increase of text data and\nits challenges imply that DR area still needs new perspectives. Although a wide\nrange of methods based on the UFT strategy has been developed, the fuzzy\napproach has not been considered for DR based on this strategy. This research\ninvestigates the application of fuzzy clustering as a DR method based on the\nUFT strategy to collapse BOW matrix to provide a lower-dimensional\nrepresentation of documents instead of the words in a corpus. The quantitative\nevaluation shows that fuzzy clustering produces superior performance and\nfeatures to Principal Components Analysis (PCA) and Singular Value\nDecomposition (SVD), two popular DR methods based on the UFT strategy.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1712.06414v1",
            "date": "2017-11-29 21:40:29+00:00",
            "title": "The Wisdom of Polarized Crowds",
            "authors": [
                "Feng Shi",
                "Misha Teplitskiy",
                "Eamon Duede",
                "James Evans"
            ],
            "abstract": "As political polarization in the United States continues to rise, the\nquestion of whether polarized individuals can fruitfully cooperate becomes\npressing. Although diversity of individual perspectives typically leads to\nsuperior team performance on complex tasks, strong political perspectives have\nbeen associated with conflict, misinformation and a reluctance to engage with\npeople and perspectives beyond one's echo chamber. It is unclear whether\nself-selected teams of politically diverse individuals will create higher or\nlower quality outcomes. In this paper, we explore the effect of team political\ncomposition on performance through analysis of millions of edits to Wikipedia's\nPolitical, Social Issues, and Science articles. We measure editors' political\nalignments by their contributions to conservative versus liberal articles. A\nsurvey of editors validates that those who primarily edit liberal articles\nidentify more strongly with the Democratic party and those who edit\nconservative ones with the Republican party. Our analysis then reveals that\npolarized teams---those consisting of a balanced set of politically diverse\neditors---create articles of higher quality than politically homogeneous teams.\nThe effect appears most strongly in Wikipedia's Political articles, but is also\nobserved in Social Issues and even Science articles. Analysis of article \"talk\npages\" reveals that politically polarized teams engage in longer, more\nconstructive, competitive, and substantively focused but linguistically diverse\ndebates than political moderates. More intense use of Wikipedia policies by\npolitically diverse teams suggests institutional design principles to help\nunleash the power of politically polarized teams.",
            "journal ref": "Nature Human Behavior. 2019",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1705.10369v4",
            "date": "2017-05-29 19:25:49+00:00",
            "title": "Emergent Communication in a Multi-Modal, Multi-Step Referential Game",
            "authors": [
                "Katrina Evtimova",
                "Andrew Drozdov",
                "Douwe Kiela",
                "Kyunghyun Cho"
            ],
            "abstract": "Inspired by previous work on emergent communication in referential games, we\npropose a novel multi-modal, multi-step referential game, where the sender and\nreceiver have access to distinct modalities of an object, and their information\nexchange is bidirectional and of arbitrary duration. The multi-modal multi-step\nsetting allows agents to develop an internal communication significantly closer\nto natural language, in that they share a single set of messages, and that the\nlength of the conversation may vary according to the difficulty of the task. We\nexamine these properties empirically using a dataset consisting of images and\ntextual descriptions of mammals, where the agents are tasked with identifying\nthe correct object. Our experiments indicate that a robust and efficient\ncommunication protocol emerges, where gradual information exchange informs\nbetter predictions and higher communication bandwidth improves generalization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1711.11017v1",
            "date": "2017-11-29 18:45:59+00:00",
            "title": "HoME: a Household Multimodal Environment",
            "authors": [
                "Simon Brodeur",
                "Ethan Perez",
                "Ankesh Anand",
                "Florian Golemo",
                "Luca Celotti",
                "Florian Strub",
                "Jean Rouat",
                "Hugo Larochelle",
                "Aaron Courville"
            ],
            "abstract": "We introduce HoME: a Household Multimodal Environment for artificial agents\nto learn from vision, audio, semantics, physics, and interaction with objects\nand other agents, all within a realistic context. HoME integrates over 45,000\ndiverse 3D house layouts based on the SUNCG dataset, a scale which may\nfacilitate learning, generalization, and transfer. HoME is an open-source,\nOpenAI Gym-compatible platform extensible to tasks in reinforcement learning,\nlanguage grounding, sound-based navigation, robotics, multi-agent learning, and\nmore. We hope HoME better enables artificial agents to learn as humans do: in\nan interactive, multimodal, and richly contextualized setting.",
            "journal ref": "None",
            "category": "cs.AI"
        }
    ],
    "2018": [
        {
            "url": "http://arxiv.org/abs/1801.00409v2",
            "date": "2018-01-01 07:54:09+00:00",
            "title": "PronouncUR: An Urdu Pronunciation Lexicon Generator",
            "authors": [
                "Haris Bin Zia",
                "Agha Ali Raza",
                "Awais Athar"
            ],
            "abstract": "State-of-the-art speech recognition systems rely heavily on three basic\ncomponents: an acoustic model, a pronunciation lexicon and a language model. To\nbuild these components, a researcher needs linguistic as well as technical\nexpertise, which is a barrier in low-resource domains. Techniques to construct\nthese three components without having expert domain knowledge are in great\ndemand. Urdu, despite having millions of speakers all over the world, is a\nlow-resource language in terms of standard publically available linguistic\nresources. In this paper, we present a grapheme-to-phoneme conversion tool for\nUrdu that generates a pronunciation lexicon in a form suitable for use with\nspeech recognition systems from a list of Urdu words. The tool predicts the\npronunciation of words using a LSTM-based model trained on a handcrafted expert\nlexicon of around 39,000 words and shows an accuracy of 64% upon internal\nevaluation. For external evaluation on a speech recognition task, we obtain a\nword error rate comparable to one achieved using a fully handcrafted expert\nlexicon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00428v4",
            "date": "2018-01-01 11:27:05+00:00",
            "title": "Sanskrit Sandhi Splitting using seq2(seq)^2",
            "authors": [
                "Rahul Aralikatte",
                "Neelamadhav Gantayat",
                "Naveen Panwar",
                "Anush Sankaran",
                "Senthil Mani"
            ],
            "abstract": "In Sanskrit, small words (morphemes) are combined to form compound words\nthrough a process known as Sandhi. Sandhi splitting is the process of splitting\na given compound word into its constituent morphemes. Although rules governing\nword splitting exists in the language, it is highly challenging to identify the\nlocation of the splits in a compound word. Though existing Sandhi splitting\nsystems incorporate these pre-defined splitting rules, they have a low accuracy\nas the same compound word might be broken down in multiple ways to provide\nsyntactically correct splits.\n  In this research, we propose a novel deep learning architecture called Double\nDecoder RNN (DD-RNN), which (i) predicts the location of the split(s) with 95%\naccuracy, and (ii) predicts the constituent words (learning the Sandhi\nsplitting rules) with 79.5% accuracy, outperforming the state-of-art by 20%.\nAdditionally, we show the generalization capability of our deep learning model,\nby showing competitive results in the problem of Chinese word segmentation, as\nwell.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00532v1",
            "date": "2018-01-02 00:32:29+00:00",
            "title": "Learning Multimodal Word Representation via Dynamic Fusion Methods",
            "authors": [
                "Shaonan Wang",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Multimodal models have been proven to outperform text-based models on\nlearning semantic word representations. Almost all previous multimodal models\ntypically treat the representations from different modalities equally. However,\nit is obvious that information from different modalities contributes\ndifferently to the meaning of words. This motivates us to build a multimodal\nmodel that can dynamically fuse the semantic representations from different\nmodalities according to different types of words. To that end, we propose three\nnovel dynamic fusion methods to assign importance weights to each modality, in\nwhich weights are learned under the weak supervision of word association pairs.\nThe extensive experiments have demonstrated that the proposed methods\noutperform strong unimodal baselines and state-of-the-art multimodal models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00625v1",
            "date": "2018-01-02 12:19:08+00:00",
            "title": "An Attentive Sequence Model for Adverse Drug Event Extraction from Biomedical Text",
            "authors": [
                "Suriyadeepan Ramamoorthy",
                "Selvakumar Murugan"
            ],
            "abstract": "Adverse reaction caused by drugs is a potentially dangerous problem which may\nlead to mortality and morbidity in patients. Adverse Drug Event (ADE)\nextraction is a significant problem in biomedical research. We model ADE\nextraction as a Question-Answering problem and take inspiration from Machine\nReading Comprehension (MRC) literature, to design our model. Our objective in\ndesigning such a model, is to exploit the local linguistic context in clinical\ntext and enable intra-sequence interaction, in order to jointly learn to\nclassify drug and disease entities, and to extract adverse reactions caused by\na given drug. Our model makes use of a self-attention mechanism to facilitate\nintra-sequence interaction in a text sequence. This enables us to visualize and\nunderstand how the network makes use of the local and wider context for\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00801v2",
            "date": "2018-01-02 19:09:47+00:00",
            "title": "Identifying emergency stages in Facebook posts of police departments with convolutional and recurrent neural networks and support vector machines",
            "authors": [
                "Nicolai Pogrebnyakov",
                "Edgar Maldonado"
            ],
            "abstract": "Classification of social media posts in emergency response is an important\npractical problem: accurate classification can help automate processing of such\nmessages and help other responders and the public react to emergencies in a\ntimely fashion. This research focused on classifying Facebook messages of US\npolice departments. Randomly selected 5,000 messages were used to train\nclassifiers that distinguished between four categories of messages: emergency\npreparedness, response and recovery, as well as general engagement messages.\nFeatures were represented with bag-of-words and word2vec, and models were\nconstructed using support vector machines (SVMs) and convolutional (CNNs) and\nrecurrent neural networks (RNNs). The best performing classifier was an RNN\nwith a custom-trained word2vec model to represent features, which achieved the\nF1 measure of 0.839.",
            "journal ref": "Proceedings of the 5th IEEE International Conference on Big Data,\n  Boston, MA, USA, December 11-14, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01331v2",
            "date": "2018-01-04 12:52:43+00:00",
            "title": "VnCoreNLP: A Vietnamese Natural Language Processing Toolkit",
            "authors": [
                "Thanh Vu",
                "Dat Quoc Nguyen",
                "Dai Quoc Nguyen",
                "Mark Dras",
                "Mark Johnson"
            ],
            "abstract": "We present an easy-to-use and fast toolkit, namely VnCoreNLP---a Java NLP\nannotation pipeline for Vietnamese. Our VnCoreNLP supports key natural language\nprocessing (NLP) tasks including word segmentation, part-of-speech (POS)\ntagging, named entity recognition (NER) and dependency parsing, and obtains\nstate-of-the-art (SOTA) results for these tasks. We release VnCoreNLP to\nprovide rich linguistic annotations to facilitate research work on Vietnamese\nNLP. Our VnCoreNLP is open-source and available at:\nhttps://github.com/vncorenlp/VnCoreNLP",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01725v1",
            "date": "2018-01-05 11:52:44+00:00",
            "title": "A Multi-task Learning Approach for Improving Product Title Compression with User Search Log Data",
            "authors": [
                "Jingang Wang",
                "Junfeng Tian",
                "Long Qiu",
                "Sheng Li",
                "Jun Lang",
                "Luo Si",
                "Man Lan"
            ],
            "abstract": "It is a challenging and practical research problem to obtain effective\ncompression of lengthy product titles for E-commerce. This is particularly\nimportant as more and more users browse mobile E-commerce apps and more\nmerchants make the original product titles redundant and lengthy for Search\nEngine Optimization. Traditional text summarization approaches often require a\nlarge amount of preprocessing costs and do not capture the important issue of\nconversion rate in E-commerce. This paper proposes a novel multi-task learning\napproach for improving product title compression with user search log data. In\nparticular, a pointer network-based sequence-to-sequence approach is utilized\nfor title compression with an attentive mechanism as an extractive method and\nan attentive encoder-decoder approach is utilized for generating user search\nqueries. The encoding parameters (i.e., semantic embedding of original titles)\nare shared among the two tasks and the attention distributions are jointly\noptimized. An extensive set of experiments with both human annotated data and\nonline deployment demonstrate the advantage of the proposed research for both\ncompression qualities and online business values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01999v1",
            "date": "2018-01-06 10:38:51+00:00",
            "title": "Using reinforcement learning to learn how to play text-based games",
            "authors": [
                "Mikul\u00e1\u0161 Zelinka"
            ],
            "abstract": "The ability to learn optimal control policies in systems where action space\nis defined by sentences in natural language would allow many interesting\nreal-world applications such as automatic optimisation of dialogue systems.\nText-based games with multiple endings and rewards are a promising platform for\nthis task, since their feedback allows us to employ reinforcement learning\ntechniques to jointly learn text representations and control policies. We\npresent a general text game playing agent, testing its generalisation and\ntransfer learning performance and showing its ability to play multiple games at\nonce. We also present pyfiction, an open-source library for universal access to\ndifferent text games that could, together with our agent that implements its\ninterface, serve as a baseline for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02054v1",
            "date": "2018-01-06 17:28:21+00:00",
            "title": "Explorations in an English Poetry Corpus: A Neurocognitive Poetics Perspective",
            "authors": [
                "Arthur M. Jacobs"
            ],
            "abstract": "This paper describes a corpus of about 3000 English literary texts with about\n250 million words extracted from the Gutenberg project that span a range of\ngenres from both fiction and non-fiction written by more than 130 authors\n(e.g., Darwin, Dickens, Shakespeare). Quantitative Narrative Analysis (QNA) is\nused to explore a cleaned subcorpus, the Gutenberg English Poetry Corpus (GEPC)\nwhich comprises over 100 poetic texts with around 2 million words from about 50\nauthors (e.g., Keats, Joyce, Wordsworth). Some exemplary QNA studies show\nauthor similarities based on latent semantic analysis, significant topics for\neach author or various text-analytic metrics for George Eliot's poem 'How Lisa\nLoved the King' and James Joyce's 'Chamber Music', concerning e.g. lexical\ndiversity or sentiment analysis. The GEPC is particularly suited for research\nin Digital Humanities, Natural Language Processing or Neurocognitive Poetics,\ne.g. as training and test corpus, or for stimulus development and control.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02073v2",
            "date": "2018-01-06 19:28:15+00:00",
            "title": "Analysis of Wikipedia-based Corpora for Question Answering",
            "authors": [
                "Tomasz Jurczyk",
                "Amit Deshmane",
                "Jinho D. Choi"
            ],
            "abstract": "This paper gives comprehensive analyses of corpora based on Wikipedia for\nseveral tasks in question answering. Four recent corpora are collected,WikiQA,\nSelQA, SQuAD, and InfoQA, and first analyzed intrinsically by contextual\nsimilarities, question types, and answer categories. These corpora are then\nanalyzed extrinsically by three question answering tasks, answer retrieval,\nselection, and triggering. An indexing-based method for the creation of a\nsilver-standard dataset for answer retrieval using the entire Wikipedia is also\npresented. Our analysis shows the uniqueness of these corpora and suggests a\nbetter use of them for statistical question answering learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02107v3",
            "date": "2018-01-07 00:48:43+00:00",
            "title": "MIZAN: A Large Persian-English Parallel Corpus",
            "authors": [
                "Omid Kashefi"
            ],
            "abstract": "One of the most major and essential tasks in natural language processing is\nmachine translation that is now highly dependent upon multilingual parallel\ncorpora. Through this paper, we introduce the biggest Persian-English parallel\ncorpus with more than one million sentence pairs collected from masterpieces of\nliterature. We also present acquisition process and statistics of the corpus,\nand experiment a base-line statistical machine translation system using the\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02581v2",
            "date": "2018-01-08 17:43:12+00:00",
            "title": "Analyzing Roles of Classifiers and Code-Mixed factors for Sentiment Identification",
            "authors": [
                "Soumil Mandal",
                "Dipankar Das"
            ],
            "abstract": "Multilingual speakers often switch between languages to express themselves on\nsocial communication platforms. Sometimes, the original script of the language\nis preserved, while using a common script for all the languages is quite\npopular as well due to convenience. On such occasions, multiple languages are\nbeing mixed with different rules of grammar, using the same script which makes\nit a challenging task for natural language processing even in case of accurate\nsentiment identification. In this paper, we report results of various\nexperiments carried out on movie reviews dataset having this code-mixing\nproperty of two languages, English and Bengali, both typed in Roman script. We\nhave tested various machine learning algorithms trained only on English\nfeatures on our code-mixed data and have achieved the maximum accuracy of\n59.00% using Naive Bayes (NB) model. We have also tested various models trained\non code-mixed data, as well as English features and the highest accuracy of\n72.50% was obtained by a Support Vector Machine (SVM) model. Finally, we have\nanalyzed the misclassified snippets and have discussed the challenges needed to\nbe resolved for better accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02916v1",
            "date": "2018-01-09 12:30:45+00:00",
            "title": "Denotation Extraction for Interactive Learning in Dialogue Systems",
            "authors": [
                "Miroslav Vodol\u00e1n",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "This paper presents a novel task using real user data obtained in\nhuman-machine conversation. The task concerns with denotation extraction from\nanswer hints collected interactively in a dialogue. The task is motivated by\nthe need for large amounts of training data for question answering dialogue\nsystem development, where the data is often expensive and hard to collect.\nBeing able to collect denotation interactively and directly from users, one\ncould improve, for example, natural understanding components on-line and ease\nthe collection of the training data. This paper also presents introductory\nresults of evaluation of several denotation extraction models including\nattention-based neural network approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03257v1",
            "date": "2018-01-10 07:53:22+00:00",
            "title": "Translating Pro-Drop Languages with Reconstruction Models",
            "authors": [
                "Longyue Wang",
                "Zhaopeng Tu",
                "Shuming Shi",
                "Tong Zhang",
                "Yvette Graham",
                "Qun Liu"
            ],
            "abstract": "Pronouns are frequently omitted in pro-drop languages, such as Chinese,\ngenerally leading to significant challenges with respect to the production of\ncomplete translations. To date, very little attention has been paid to the\ndropped pronoun (DP) problem within neural machine translation (NMT). In this\nwork, we propose a novel reconstruction-based approach to alleviating DP\ntranslation problems for NMT models. Firstly, DPs within all source sentences\nare automatically annotated with parallel information extracted from the\nbilingual training corpus. Next, the annotated source sentence is reconstructed\nfrom hidden representations in the NMT model. With auxiliary training\nobjectives, in terms of reconstruction scores, the parameters associated with\nthe NMT model are guided to produce enhanced hidden representations that are\nencouraged as much as possible to embed annotated DP information. Experimental\nresults on both Chinese-English and Japanese-English dialogue translation tasks\nshow that the proposed approach significantly and consistently improves\ntranslation performance over a strong NMT baseline, which is directly built on\nthe training data annotated with DPs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03460v1",
            "date": "2018-01-10 17:16:36+00:00",
            "title": "MilkQA: a Dataset of Consumer Questions for the Task of Answer Selection",
            "authors": [
                "Marcelo Criscuolo",
                "Erick Rocha Fonseca",
                "Sandra Maria Alu\u00edsio",
                "Ana Carolina Speran\u00e7a-Criscuolo"
            ],
            "abstract": "We introduce MilkQA, a question answering dataset from the dairy domain\ndedicated to the study of consumer questions. The dataset contains 2,657 pairs\nof questions and answers, written in the Portuguese language and originally\ncollected by the Brazilian Agricultural Research Corporation (Embrapa). All\nquestions were motivated by real situations and written by thousands of authors\nwith very different backgrounds and levels of literacy, while answers were\nelaborated by specialists from Embrapa's customer service. Our dataset was\nfiltered and anonymized by three human annotators. Consumer questions are a\nchallenging kind of question that is usually employed as a form of seeking\ninformation. Although several question answering datasets are available, most\nof such resources are not suitable for research on answer selection models for\nconsumer questions. We aim to fill this gap by making MilkQA publicly\navailable. We study the behavior of four answer selection models on MilkQA: two\nbaseline models and two convolutional neural network archictetures. Our results\nshow that MilkQA poses real challenges to computational models, particularly\ndue to linguistic characteristics of its questions and to their unusually\nlonger lengths. Only one of the experimented models gives reasonable results,\nat the cost of high computational requirements.",
            "journal ref": "Intelligent Systems (BRACIS), 2017 Brazilian Conference on",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03562v1",
            "date": "2018-01-04 21:30:05+00:00",
            "title": "Discrete symbolic optimization and Boltzmann sampling by continuous neural dynamics: Gradient Symbolic Computation",
            "authors": [
                "Paul Tupper",
                "Paul Smolensky",
                "Pyeong Whan Cho"
            ],
            "abstract": "Gradient Symbolic Computation is proposed as a means of solving discrete\nglobal optimization problems using a neurally plausible continuous stochastic\ndynamical system. Gradient symbolic dynamics involves two free parameters that\nmust be adjusted as a function of time to obtain the global maximizer at the\nend of the computation. We provide a summary of what is known about the GSC\ndynamics for special cases of settings of the parameters, and also establish\nthat there is a schedule for the two parameters for which convergence to the\ncorrect answer occurs with high probability. These results put the empirical\nresults already obtained for GSC on a sound theoretical footing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03563v1",
            "date": "2018-01-07 15:18:21+00:00",
            "title": "Group Communication Analysis: A Computational Linguistics Approach for Detecting Sociocognitive Roles in Multi-Party Interactions",
            "authors": [
                "Nia Dowell",
                "Tristian Nixon",
                "Arthur Graesser"
            ],
            "abstract": "Roles are one of the most important concepts in understanding human\nsociocognitive behavior. During group interactions, members take on different\nroles within the discussion. Roles have distinct patterns of behavioral\nengagement (i.e., active or passive, leading or following), contribution\ncharacteristics (i.e., providing new information or echoing given material),\nand social orientation (i.e., individual or group). Different combinations of\nthese roles can produce characteristically different group outcomes, being\neither less or more productive towards collective goals. In online\ncollaborative learning environments, this can lead to better or worse learning\noutcomes for the individual participants. In this study, we propose and\nvalidate a novel approach for detecting emergent roles from the participants'\ncontributions and patterns of interaction. Specifically, we developed a group\ncommunication analysis (GCA) by combining automated computational linguistic\ntechniques with analyses of the sequential interactions of online group\ncommunication. The GCA was applied to three large collaborative interaction\ndatasets (participant N = 2,429; group N = 3,598). Cluster analyses and linear\nmixed-effects modeling were used to assess the validity of the GCA approach and\nthe influence of learner roles on student and group performance. The results\nindicate that participants' patterns in linguistic coordination and cohesion\nare representative of the roles that individuals play in collaborative\ndiscussions. More broadly, GCA provides a framework for researchers to explore\nthe micro intra- and interpersonal patterns associated with the participants'\nroles and the sociocognitive processes related to successful collaboration.",
            "journal ref": "Behavior Research Methods 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03564v1",
            "date": "2018-01-10 21:47:26+00:00",
            "title": "Unsupervised Part-of-Speech Induction",
            "authors": [
                "Omid Kashefi"
            ],
            "abstract": "Part-of-Speech (POS) tagging is an old and fundamental task in natural\nlanguage processing. While supervised POS taggers have shown promising\naccuracy, it is not always feasible to use supervised methods due to lack of\nlabeled data. In this project, we attempt to unsurprisingly induce POS tags by\niteratively looking for a recurring pattern of words through a hierarchical\nagglomerative clustering process. Our approach shows promising results when\ncompared to the tagging results of the state-of-the-art unsupervised POS\ntaggers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03603v1",
            "date": "2018-01-11 01:16:13+00:00",
            "title": "SEE: Syntax-aware Entity Embedding for Neural Relation Extraction",
            "authors": [
                "Zhengqiu He",
                "Wenliang Chen",
                "Zhenghua Li",
                "Meishan Zhang",
                "Wei Zhang",
                "Min Zhang"
            ],
            "abstract": "Distant supervised relation extraction is an efficient approach to scale\nrelation extraction to very large corpora, and has been widely used to find\nnovel relational facts from plain text. Recent studies on neural relation\nextraction have shown great progress on this task via modeling the sentences in\nlow-dimensional spaces, but seldom considered syntax information to model the\nentities. In this paper, we propose to learn syntax-aware entity embedding for\nneural relation extraction. First, we encode the context of entities on a\ndependency tree as sentence-level entity embedding based on tree-GRU. Then, we\nutilize both intra-sentence and inter-sentence attentions to obtain sentence\nset-level entity embedding over all sentences containing the focus entity pair.\nFinally, we combine both sentence embedding and entity embedding for relation\nclassification. We conduct experiments on a widely used real-world dataset and\nthe experimental results show that our model can make full use of all\ninformative instances and achieve state-of-the-art performance of relation\nextraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03615v1",
            "date": "2018-01-11 02:18:06+00:00",
            "title": "Improved English to Russian Translation by Neural Suffix Prediction",
            "authors": [
                "Kai Song",
                "Yue Zhang",
                "Min Zhang",
                "Weihua Luo"
            ],
            "abstract": "Neural machine translation (NMT) suffers a performance deficiency when a\nlimited vocabulary fails to cover the source or target side adequately, which\nhappens frequently when dealing with morphologically rich languages. To address\nthis problem, previous work focused on adjusting translation granularity or\nexpanding the vocabulary size. However, morphological information is relatively\nunder-considered in NMT architectures, which may further improve translation\nquality. We propose a novel method, which can not only reduce data sparsity but\nalso model morphology through a simple but effective mechanism. By predicting\nthe stem and suffix separately during decoding, our system achieves an\nimprovement of up to 1.98 BLEU compared with previous work on English to\nRussian translation. Our method is orthogonal to different NMT architectures\nand stably gains improvements on various domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04017v1",
            "date": "2018-01-11 23:40:39+00:00",
            "title": "Did William Shakespeare and Thomas Kyd Write Edward III?",
            "authors": [
                "David Kernot",
                "Terry Bossomaier",
                "Roger Bradbury"
            ],
            "abstract": "William Shakespeare is believed to be a significant author in the anonymous\nplay, The Reign of King Edward III, published in 1596. However, recently,\nThomas Kyd, has been suggested as the primary author. Using a neurolinguistics\napproach to authorship identification we use a four-feature technique, RPAS, to\nconvert the 19 scenes in Edward III into a multi-dimensional vector. Three\ncomplementary analytical techniques are applied to cluster the data and reduce\nsingle technique bias before an alternate method, seriation, is used to measure\nthe distances between clusters and test the strength of the connections. We\nfind the multivariate techniques robust and are able to allocate up to 14\nscenes to Thomas Kyd, and further question if scenes long believed to be\nShakespeare's are not his.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.6,\n  No.6, December 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04470v3",
            "date": "2018-01-13 17:57:33+00:00",
            "title": "Simple Unsupervised Keyphrase Extraction using Sentence Embeddings",
            "authors": [
                "Kamil Bennani-Smires",
                "Claudiu Musat",
                "Andreea Hossmann",
                "Michael Baeriswyl",
                "Martin Jaggi"
            ],
            "abstract": "Keyphrase extraction is the task of automatically selecting a small set of\nphrases that best describe a given free text document. Supervised keyphrase\nextraction requires large amounts of labeled training data and generalizes very\npoorly outside the domain of the training data. At the same time, unsupervised\nsystems have poor accuracy, and often do not generalize well, as they require\nthe input document to belong to a larger corpus also given as input. Addressing\nthese drawbacks, in this paper, we tackle keyphrase extraction from single\ndocuments with EmbedRank: a novel unsupervised method, that leverages sentence\nembeddings. EmbedRank achieves higher F-scores than graph-based state of the\nart systems on standard datasets and is suitable for real-time processing of\nlarge amounts of Web data. With EmbedRank, we also explicitly increase coverage\nand diversity among the selected keyphrases by introducing an embedding-based\nmaximal marginal relevance (MMR) for new phrases. A user study including over\n200 votes showed that, although reducing the phrases' semantic overlap leads to\nno gains in F-score, our high diversity selection is preferred by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04726v3",
            "date": "2018-01-15 10:31:01+00:00",
            "title": "An Interpretable Reasoning Network for Multi-Relation Question Answering",
            "authors": [
                "Mantong Zhou",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Multi-relation Question Answering is a challenging task, due to the\nrequirement of elaborated analysis on questions and reasoning over multiple\nfact triples in knowledge base. In this paper, we present a novel model called\nInterpretable Reasoning Network that employs an interpretable, hop-by-hop\nreasoning process for question answering. The model dynamically decides which\npart of an input question should be analyzed at each hop; predicts a relation\nthat corresponds to the current parsed results; utilizes the predicted relation\nto update the question representation and the state of the reasoning process;\nand then drives the next-hop reasoning. Experiments show that our model yields\nstate-of-the-art results on two datasets. More interestingly, the model can\noffer traceable and observable intermediate predictions for reasoning analysis\nand failure diagnosis, thereby allowing manual manipulation in predicting the\nfinal answer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04962v1",
            "date": "2018-01-15 19:39:19+00:00",
            "title": "What Level of Quality can Neural Machine Translation Attain on Literary Text?",
            "authors": [
                "Antonio Toral",
                "Andy Way"
            ],
            "abstract": "Given the rise of a new approach to MT, Neural MT (NMT), and its promising\nperformance on different text types, we assess the translation quality it can\nattain on what is perceived to be the greatest challenge for MT: literary text.\nSpecifically, we target novels, arguably the most popular type of literary\ntext. We build a literary-adapted NMT system for the English-to-Catalan\ntranslation direction and evaluate it against a system pertaining to the\nprevious dominant paradigm in MT: statistical phrase-based MT (PBSMT). To this\nend, for the first time we train MT systems, both NMT and PBSMT, on large\namounts of literary text (over 100 million words) and evaluate them on a set of\ntwelve widely known novels spanning from the the 1920s to the present day.\nAccording to the BLEU automatic evaluation metric, NMT is significantly better\nthan PBSMT (p < 0.01) on all the novels considered. Overall, NMT results in a\n11% relative improvement (3 points absolute) over PBSMT. A complementary human\nevaluation on three of the books shows that between 17% and 34% of the\ntranslations, depending on the book, produced by NMT (versus 8% and 20% with\nPBSMT) are perceived by native speakers of the target language to be of\nequivalent quality to translations produced by a professional human translator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05119v1",
            "date": "2018-01-16 05:18:06+00:00",
            "title": "Variational Recurrent Neural Machine Translation",
            "authors": [
                "Jinsong Su",
                "Shan Wu",
                "Deyi Xiong",
                "Yaojie Lu",
                "Xianpei Han",
                "Biao Zhang"
            ],
            "abstract": "Partially inspired by successful applications of variational recurrent neural\nnetworks, we propose a novel variational recurrent neural machine translation\n(VRNMT) model in this paper. Different from the variational NMT, VRNMT\nintroduces a series of latent random variables to model the translation\nprocedure of a sentence in a generative way, instead of a single latent\nvariable. Specifically, the latent random variables are included into the\nhidden states of the NMT decoder with elements from the variational\nautoencoder. In this way, these variables are recurrently generated, which\nenables them to further capture strong and complex dependencies among the\noutput translations at different timesteps. In order to deal with the\nchallenges in performing efficient posterior inference and large-scale training\nduring the incorporation of latent variables, we build a neural posterior\napproximator, and equip it with a reparameterization technique to estimate the\nvariational lower bound. Experiments on Chinese-English and English-German\ntranslation tasks demonstrate that the proposed model achieves significant\nimprovements over both the conventional and variational NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05122v2",
            "date": "2018-01-16 05:21:43+00:00",
            "title": "Asynchronous Bidirectional Decoding for Neural Machine Translation",
            "authors": [
                "Xiangwen Zhang",
                "Jinsong Su",
                "Yue Qin",
                "Yang Liu",
                "Rongrong Ji",
                "Hongji Wang"
            ],
            "abstract": "The dominant neural machine translation (NMT) models apply unified\nattentional encoder-decoder neural networks for translation. Traditionally, the\nNMT decoders adopt recurrent neural networks (RNNs) to perform translation in a\nleft-toright manner, leaving the target-side contexts generated from right to\nleft unexploited during translation. In this paper, we equip the conventional\nattentional encoder-decoder NMT framework with a backward decoder, in order to\nexplore bidirectional decoding for NMT. Attending to the hidden state sequence\nproduced by the encoder, our backward decoder first learns to generate the\ntarget-side hidden state sequence from right to left. Then, the forward decoder\nperforms translation in the forward direction, while in each translation\nprediction timestep, it simultaneously applies two attention models to consider\nthe source-side and reverse target-side hidden states, respectively. With this\nnew architecture, our model is able to fully exploit source- and target-side\ncontexts to improve translation quality altogether. Experimental results on\nNIST Chinese-English and WMT English-German translation tasks demonstrate that\nour model achieves substantial improvements over the conventional NMT by 3.14\nand 1.38 BLEU points, respectively. The source code of this work can be\nobtained from https://github.com/DeepLearnXMU/ABDNMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05147v1",
            "date": "2018-01-16 08:11:01+00:00",
            "title": "Adversarial Learning for Chinese NER from Crowd Annotations",
            "authors": [
                "YaoSheng Yang",
                "Meishan Zhang",
                "Wenliang Chen",
                "Wei Zhang",
                "Haofen Wang",
                "Min Zhang"
            ],
            "abstract": "To quickly obtain new labeled data, we can choose crowdsourcing as an\nalternative way at lower cost in a short time. But as an exchange, crowd\nannotations from non-experts may be of lower quality than those from experts.\nIn this paper, we propose an approach to performing crowd annotation learning\nfor Chinese Named Entity Recognition (NER) to make full use of the noisy\nsequence labels from multiple annotators. Inspired by adversarial learning, our\napproach uses a common Bi-LSTM and a private Bi-LSTM for representing\nannotator-generic and -specific information. The annotator-generic information\nis the common knowledge for entities easily mastered by the crowd. Finally, we\nbuild our Chinese NE tagger based on the LSTM-CRF model. In our experiments, we\ncreate two data sets for Chinese NER tasks from two domains. The experimental\nresults show that our system achieves better scores than strong baseline\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05149v1",
            "date": "2018-01-16 08:28:53+00:00",
            "title": "OneNet: Joint Domain, Intent, Slot Prediction for Spoken Language Understanding",
            "authors": [
                "Young-Bum Kim",
                "Sungjin Lee",
                "Karl Stratos"
            ],
            "abstract": "In practice, most spoken language understanding systems process user input in\na pipelined manner; first domain is predicted, then intent and semantic slots\nare inferred according to the semantic frames of the predicted domain. The\npipeline approach, however, has some disadvantages: error propagation and lack\nof information sharing. To address these issues, we present a unified neural\nnetwork that jointly performs domain, intent, and slot predictions. Our\napproach adopts a principled architecture for multitask learning to fold in the\nstate-of-the-art models for each task. With a few more ingredients, e.g.\northography-sensitive input encoding and curriculum training, our model\ndelivered significant improvements in all three tasks across all domains over\nstrong baselines, including one using oracle prediction for domain detection,\non real user data of a commercial personal assistant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06172v1",
            "date": "2018-01-18 18:51:58+00:00",
            "title": "Contextual and Position-Aware Factorization Machines for Sentiment Classification",
            "authors": [
                "Shuai Wang",
                "Mianwei Zhou",
                "Geli Fei",
                "Yi Chang",
                "Bing Liu"
            ],
            "abstract": "While existing machine learning models have achieved great success for\nsentiment classification, they typically do not explicitly capture\nsentiment-oriented word interaction, which can lead to poor results for\nfine-grained analysis at the snippet level (a phrase or sentence).\nFactorization Machine provides a possible approach to learning element-wise\ninteraction for recommender systems, but they are not directly applicable to\nour task due to the inability to model contexts and word sequences. In this\nwork, we develop two Position-aware Factorization Machines which consider word\ninteraction, context and position information. Such information is jointly\nencoded in a set of sentiment-oriented word interaction vectors. Compared to\ntraditional word embeddings, SWI vectors explicitly capture sentiment-oriented\nword interaction and simplify the parameter learning. Experimental results show\nthat while they have comparable performance with state-of-the-art methods for\ndocument-level classification, they benefit the snippet/sentence-level\nsentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06261v2",
            "date": "2018-01-19 00:29:55+00:00",
            "title": "Investigating the Working of Text Classifiers",
            "authors": [
                "Devendra Singh Sachan",
                "Manzil Zaheer",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Text classification is one of the most widely studied tasks in natural\nlanguage processing. Motivated by the principle of compositionality, large\nmultilayer neural network models have been employed for this task in an attempt\nto effectively utilize the constituent expressions. Almost all of the reported\nwork train large networks using discriminative approaches, which come with a\ncaveat of no proper capacity control, as they tend to latch on to any signal\nthat may not generalize. Using various recent state-of-the-art approaches for\ntext classification, we explore whether these models actually learn to compose\nthe meaning of the sentences or still just focus on some keywords or lexicons\nfor classifying the document. To test our hypothesis, we carefully construct\ndatasets where the training and test splits have no direct overlap of such\nlexicons, but overall language structure would be similar. We study various\ntext classifiers and observe that there is a big performance drop on these\ndatasets. Finally, we show that even simple models with our proposed\nregularization techniques, which disincentivize focusing on key lexicons, can\nsubstantially improve classification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06407v1",
            "date": "2018-01-19 14:11:16+00:00",
            "title": "Size vs. Structure in Training Corpora for Word Embedding Models: Araneum Russicum Maximum and Russian National Corpus",
            "authors": [
                "Andrey Kutuzov",
                "Maria Kunilovskaya"
            ],
            "abstract": "In this paper, we present a distributional word embedding model trained on\none of the largest available Russian corpora: Araneum Russicum Maximum (over 10\nbillion words crawled from the web). We compare this model to the model trained\non the Russian National Corpus (RNC). The two corpora are much different in\ntheir size and compilation procedures. We test these differences by evaluating\nthe trained models against the Russian part of the Multilingual SimLex999\nsemantic similarity dataset. We detect and describe numerous issues in this\ndataset and publish a new corrected version. Aside from the already known fact\nthat the RNC is generally a better training corpus than web corpora, we\nenumerate and explain fine differences in how the models process semantic\nsimilarity task, what parts of the evaluation set are difficult for particular\nmodels and why. Additionally, the learning curves for both models are\ndescribed, showing that the RNC is generally more robust as training material\nfor this task.",
            "journal ref": "In: van der Aalst W. et al. (eds) Analysis of Images, Social\n  Networks and Texts. AIST 2017. Lecture Notes in Computer Science, vol 10716.\n  Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06422v3",
            "date": "2018-01-19 14:41:45+00:00",
            "title": "Evaluating neural network explanation methods using hybrid documents and morphological agreement",
            "authors": [
                "Nina Poerner",
                "Benjamin Roth",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "The behavior of deep neural networks (DNNs) is hard to understand. This makes\nit necessary to explore post hoc explanation methods. We conduct the first\ncomprehensive evaluation of explanation methods for NLP. To this end, we design\ntwo novel evaluation paradigms that cover two important classes of NLP\nproblems: small context and large context problems. Both paradigms require no\nmanual annotation and are therefore broadly applicable. We also introduce\nLIMSSE, an explanation method inspired by LIME that is designed for NLP. We\nshow empirically that LIMSSE, LRP and DeepLIFT are the most effective\nexplanation methods and recommend them for explaining DNNs in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06436v1",
            "date": "2018-01-19 15:00:33+00:00",
            "title": "A Resource-Light Method for Cross-Lingual Semantic Textual Similarity",
            "authors": [
                "Goran Glava\u0161",
                "Marc Franco-Salvador",
                "Simone Paolo Ponzetto",
                "Paolo Rosso"
            ],
            "abstract": "Recognizing semantically similar sentences or paragraphs across languages is\nbeneficial for many tasks, ranging from cross-lingual information retrieval and\nplagiarism detection to machine translation. Recently proposed methods for\npredicting cross-lingual semantic similarity of short texts, however, make use\nof tools and resources (e.g., machine translation systems, syntactic parsers or\nnamed entity recognition) that for many languages (or language pairs) do not\nexist. In contrast, we propose an unsupervised and a very resource-light\napproach for measuring semantic similarity between texts in different\nlanguages. To operate in the bilingual (or multilingual) space, we project\ncontinuous word vectors (i.e., word embeddings) from one language to the vector\nspace of the other language via the linear translation model. We then align\nwords according to the similarity of their vectors in the bilingual embedding\nspace and investigate different unsupervised measures of semantic similarity\nexploiting bilingual embeddings and word alignments. Requiring only a\nlimited-size set of word translation pairs between the languages, the proposed\napproach is applicable to virtually any pair of languages for which there\nexists a sufficiently large corpus, required to learn monolingual word\nembeddings. Experimental results on three different datasets for measuring\nsemantic textual similarity show that our simple resource-light approach\nreaches performance close to that of supervised and resource intensive methods,\ndisplaying stability across different language pairs. Furthermore, we evaluate\nthe proposed method on two extrinsic tasks, namely extraction of parallel\nsentences from comparable corpora and cross lingual plagiarism detection, and\nshow that it yields performance comparable to those of complex\nresource-intensive state-of-the-art models for the respective tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06480v1",
            "date": "2018-01-19 16:24:47+00:00",
            "title": "A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks",
            "authors": [
                "Tushar Semwal",
                "Gaurav Mathur",
                "Promod Yenigalla",
                "Shivashankar B. Nair"
            ],
            "abstract": "Transfer Learning (TL) plays a crucial role when a given dataset has\ninsufficient labeled examples to train an accurate model. In such scenarios,\nthe knowledge accumulated within a model pre-trained on a source dataset can be\ntransferred to a target dataset, resulting in the improvement of the target\nmodel. Though TL is found to be successful in the realm of image-based\napplications, its impact and practical use in Natural Language Processing (NLP)\napplications is still a subject of research. Due to their hierarchical\narchitecture, Deep Neural Networks (DNN) provide flexibility and customization\nin adjusting their parameters and depth of layers, thereby forming an apt area\nfor exploiting the use of TL. In this paper, we report the results and\nconclusions obtained from extensive empirical experiments using a Convolutional\nNeural Network (CNN) and try to uncover thumb rules to ensure a successful\npositive transfer. In addition, we also highlight the flawed means that could\nlead to a negative transfer. We explore the transferability of various layers\nand describe the effect of varying hyper-parameters on the transfer\nperformance. Also, we present a comparison of accuracy value and model size\nagainst state-of-the-art methods. Finally, we derive inferences from the\nempirical results and provide best practices to achieve a successful positive\ntransfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06607v2",
            "date": "2018-01-20 00:15:09+00:00",
            "title": "Efficient Text Classification Using Tree-structured Multi-linear Principal Component Analysis",
            "authors": [
                "Yuanhang Su",
                "Yuzhong Huang",
                "C. -C. Jay Kuo"
            ],
            "abstract": "A novel text data dimension reduction technique, called the tree-structured\nmulti-linear principal component anal- ysis (TMPCA), is proposed in this work.\nBeing different from traditional text dimension reduction methods that deal\nwith the word-level representation, the TMPCA technique reduces the dimension\nof input sequences and sentences to simplify the following text classification\ntasks. It is shown mathematically and experimentally that the TMPCA tool\ndemands much lower complexity (and, hence, less computing power) than the\nordinary principal component analysis (PCA). Furthermore, it is demon- strated\nby experimental results that the support vector machine (SVM) method applied to\nthe TMPCA-processed data achieves commensurable or better performance than the\nstate-of-the-art recurrent neural network (RNN) approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06613v2",
            "date": "2018-01-20 01:59:21+00:00",
            "title": "Building an Ellipsis-aware Chinese Dependency Treebank for Web Text",
            "authors": [
                "Xuancheng Ren",
                "Xu Sun",
                "Ji Wen",
                "Bingzhen Wei",
                "Weidong Zhan",
                "Zhiyuan Zhang"
            ],
            "abstract": "Web 2.0 has brought with it numerous user-produced data revealing one's\nthoughts, experiences, and knowledge, which are a great source for many tasks,\nsuch as information extraction, and knowledge base construction. However, the\ncolloquial nature of the texts poses new challenges for current natural\nlanguage processing techniques, which are more adapt to the formal form of the\nlanguage. Ellipsis is a common linguistic phenomenon that some words are left\nout as they are understood from the context, especially in oral utterance,\nhindering the improvement of dependency parsing, which is of great importance\nfor tasks relied on the meaning of the sentence. In order to promote research\nin this area, we are releasing a Chinese dependency treebank of 319 weibos,\ncontaining 572 sentences with omissions restored and contexts reserved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06792v1",
            "date": "2018-01-21 09:01:46+00:00",
            "title": "Attentive Recurrent Tensor Model for Community Question Answering",
            "authors": [
                "Gaurav Bhatt",
                "Shivam Sharma",
                "Balasubramanian Raman"
            ],
            "abstract": "A major challenge to the problem of community question answering is the\nlexical and semantic gap between the sentence representations. Some solutions\nto minimize this gap includes the introduction of extra parameters to deep\nmodels or augmenting the external handcrafted features. In this paper, we\npropose a novel attentive recurrent tensor network for solving the lexical and\nsemantic gap in community question answering. We introduce token-level and\nphrase-level attention strategy that maps input sequences to the output using\ntrainable parameters. Further, we use the tensor parameters to introduce a\n3-way interaction between question, answer and external features in vector\nspace. We introduce simplified tensor matrices with L2 regularization that\nresults in smooth optimization during training. The proposed model achieves\nstate-of-the-art performance on the task of answer sentence selection (TrecQA\nand WikiQA datasets) while outperforming the current state-of-the-art on the\ntasks of best answer selection (Yahoo! L4) and answer triggering task (WikiQA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06807v3",
            "date": "2018-01-21 11:19:10+00:00",
            "title": "Embedding Learning Through Multilingual Concept Induction",
            "authors": [
                "Philipp Dufter",
                "Mengjie Zhao",
                "Martin Schmitt",
                "Alexander Fraser",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present a new method for estimating vector space representations of words:\nembedding learning by concept induction. We test this method on a highly\nparallel corpus and learn semantic representations of words in 1259 different\nlanguages in a single common space. An extensive experimental evaluation on\ncrosslingual word similarity and sentiment analysis indicates that\nconcept-based multilingual embedding learning performs better than previous\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06830v1",
            "date": "2018-01-21 14:38:23+00:00",
            "title": "Neural Multi-task Learning in Automated Assessment",
            "authors": [
                "Ronan Cummins",
                "Marek Rei"
            ],
            "abstract": "Grammatical error detection and automated essay scoring are two tasks in the\narea of automated assessment. Traditionally these tasks have been treated\nindependently with different machine learning models and features used for each\ntask. In this paper, we develop a multi-task neural network model that jointly\noptimises for both tasks, and in particular we show that neural automated essay\nscoring can be significantly improved. We show that while the essay score\nprovides little evidence to inform grammatical error detection, the essay score\nis highly influenced by error detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07073v2",
            "date": "2018-01-22 13:00:02+00:00",
            "title": "BiographyNet: Extracting Relations Between People and Events",
            "authors": [
                "Antske Fokkens",
                "Serge ter Braake",
                "Niels Ockeloen",
                "Piek Vossen",
                "Susan Leg\u00eane",
                "Guus Schreiber",
                "Victor de Boer"
            ],
            "abstract": "This paper describes BiographyNet, a digital humanities project (2012-2016)\nthat brings together researchers from history, computational linguistics and\ncomputer science. The project uses data from the Biography Portal of the\nNetherlands (BPN), which contains approximately 125,000 biographies from a\nvariety of Dutch biographical dictionaries from the eighteenth century until\nnow, describing around 76,000 individuals. BiographyNet's aim is to strengthen\nthe value of the portal and comparable biographical datasets for historical\nresearch, by improving the search options and the presentation of its outcome,\nwith a historically justified NLP pipeline that works through a user evaluated\ndemonstrator. The project's main target group are professional historians. The\nproject therefore worked with two key concepts: \"provenance\" -understood as a\nterm allowing for both historical source criticism and for references to\ndata-management and programming interventions in digitized sources; and\n\"perspective\" interpreted as inherent uncertainty concerning the interpretation\nof historical results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07174v1",
            "date": "2018-01-22 16:19:12+00:00",
            "title": "Unsupervised Open Relation Extraction",
            "authors": [
                "Hady Elsahar",
                "Elena Demidova",
                "Simon Gottschalk",
                "Christophe Gravier",
                "Frederique Laforest"
            ],
            "abstract": "We explore methods to extract relations between named entities from free text\nin an unsupervised setting. In addition to standard feature extraction, we\ndevelop a novel method to re-weight word embeddings. We alleviate the problem\nof features sparsity using an individual feature reduction. Our approach\nexhibits a significant improvement by 5.8% over the state-of-the-art relation\nclustering scoring a F1-score of 0.416 on the NYT-FB dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07288v3",
            "date": "2018-01-22 19:27:31+00:00",
            "title": "Siamese Neural Networks with Random Forest for detecting duplicate question pairs",
            "authors": [
                "Ameya Godbole",
                "Aman Dalmia",
                "Sunil Kumar Sahu"
            ],
            "abstract": "Determining whether two given questions are semantically similar is a fairly\nchallenging task given the different structures and forms that the questions\ncan take. In this paper, we use Gated Recurrent Units(GRU) in combination with\nother highly used machine learning algorithms like Random Forest, Adaboost and\nSVM for the similarity prediction task on a dataset released by Quora,\nconsisting of about 400k labeled question pairs. We got the best result by\nusing the Siamese adaptation of a Bidirectional GRU with a Random Forest\nclassifier, which landed us among the top 24% in the competition Quora Question\nPairs hosted on Kaggle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07414v1",
            "date": "2018-01-23 07:22:34+00:00",
            "title": "Assertion-based QA with Question-Aware Open Information Extraction",
            "authors": [
                "Zhao Yan",
                "Duyu Tang",
                "Nan Duan",
                "Shujie Liu",
                "Wendi Wang",
                "Daxin Jiang",
                "Ming Zhou",
                "Zhoujun Li"
            ],
            "abstract": "We present assertion based question answering (ABQA), an open domain question\nanswering task that takes a question and a passage as inputs, and outputs a\nsemi-structured assertion consisting of a subject, a predicate and a list of\narguments. An assertion conveys more evidences than a short answer span in\nreading comprehension, and it is more concise than a tedious passage in\npassage-based QA. These advantages make ABQA more suitable for human-computer\ninteraction scenarios such as voice-controlled speakers. Further progress\ntowards improving ABQA requires richer supervised dataset and powerful models\nof text understanding. To remedy this, we introduce a new dataset called\nWebAssertions, which includes hand-annotated QA labels for 358,427 assertions\nin 55,960 web passages. To address ABQA, we develop both generative and\nextractive approaches. The backbone of our generative approach is sequence to\nsequence learning. In order to capture the structure of the output assertion,\nwe introduce a hierarchical decoder that first generates the structure of the\nassertion and then generates the words of each field. The extractive approach\nis based on learning to rank. Features at different levels of granularity are\ndesigned to measure the semantic relevance between a question and an assertion.\nExperimental results show that our approaches have the ability to infer\nquestion-aware assertions from a passage. We further evaluate our approaches by\nincorporating the ABQA results as additional features in passage-based QA.\nResults on two datasets show that ABQA features significantly improve the\naccuracy on passage-based~QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07507v3",
            "date": "2018-01-23 12:22:52+00:00",
            "title": "What did you Mention? A Large Scale Mention Detection Benchmark for Spoken and Written Text",
            "authors": [
                "Yosi Mass",
                "Lili Kotlerman",
                "Shachar Mirkin",
                "Elad Venezian",
                "Gera Witzling",
                "Noam Slonim"
            ],
            "abstract": "We describe a large, high-quality benchmark for the evaluation of Mention\nDetection tools. The benchmark contains annotations of both named entities as\nwell as other types of entities, annotated on different types of text, ranging\nfrom clean text taken from Wikipedia, to noisy spoken data. The benchmark was\nbuilt through a highly controlled crowd sourcing process to ensure its quality.\nWe describe the benchmark, the process and the guidelines that were used to\nbuild it. We then demonstrate the results of a state-of-the-art system running\non that benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07704v2",
            "date": "2018-01-23 18:47:03+00:00",
            "title": "Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models",
            "authors": [
                "Tal Baumel",
                "Matan Eyal",
                "Michael Elhadad"
            ],
            "abstract": "Query Focused Summarization (QFS) has been addressed mostly using extractive\nmethods. Such methods, however, produce text which suffers from low coherence.\nWe investigate how abstractive methods can be applied to QFS, to overcome such\nlimitations. Recent developments in neural-attention based sequence-to-sequence\nmodels have led to state-of-the-art results on the task of abstractive generic\nsingle document summarization. Such models are trained in an end to end method\non large amounts of training data. We address three aspects to make abstractive\nsummarization applicable to QFS: (a)since there is no training data, we\nincorporate query relevance into a pre-trained abstractive model; (b) since\nexisting abstractive models are trained in a single-document setting, we design\nan iterated method to embed abstractive models within the multi-document\nrequirement of QFS; (c) the abstractive models we adapt are trained to generate\ntext of specific length (about 100 words), while we aim at generating output of\na different size (about 250 words); we design a way to adapt the target size of\nthe generated summaries to a given size ratio. We compare our method (Relevance\nSensitive Attention for QFS) to extractive baselines and with various ways to\ncombine abstractive models on the DUC QFS datasets and demonstrate solid\nimprovements on ROUGE performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07737v2",
            "date": "2018-01-23 19:24:38+00:00",
            "title": "SentiPers: A Sentiment Analysis Corpus for Persian",
            "authors": [
                "Pedram Hosseini",
                "Ali Ahmadian Ramaki",
                "Hassan Maleki",
                "Mansoureh Anvari",
                "Seyed Abolghasem Mirroshandel"
            ],
            "abstract": "Sentiment Analysis (SA) is a major field of study in natural language\nprocessing, computational linguistics and information retrieval. Interest in SA\nhas been constantly growing in both academia and industry over the recent\nyears. Moreover, there is an increasing need for generating appropriate\nresources and datasets in particular for low resource languages including\nPersian. These datasets play an important role in designing and developing\nappropriate opinion mining platforms using supervised, semi-supervised or\nunsupervised methods. In this paper, we outline the entire process of\ndeveloping a manually annotated sentiment corpus, SentiPers, which covers\nformal and informal written contemporary Persian. To the best of our knowledge,\nSentiPers is a unique sentiment corpus with such a rich annotation in three\ndifferent levels including document-level, sentence-level, and\nentity/aspect-level for Persian. The corpus contains more than 26000 sentences\nof users opinions from digital product domain and benefits from special\ncharacteristics such as quantifying the positiveness or negativity of an\nopinion through assigning a number within a specific range to any given\nsentence. Furthermore, we present statistics on various components of our\ncorpus as well as studying the inter-annotator agreement among the annotators.\nFinally, some of the challenges that we faced during the annotation process\nwill be discussed as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07746v2",
            "date": "2018-01-23 19:49:58+00:00",
            "title": "HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments",
            "authors": [
                "Akari Asai",
                "Sara Evensen",
                "Behzad Golshan",
                "Alon Halevy",
                "Vivian Li",
                "Andrei Lopatenko",
                "Daniela Stepanov",
                "Yoshihiko Suhara",
                "Wang-Chiew Tan",
                "Yinzhan Xu"
            ],
            "abstract": "The science of happiness is an area of positive psychology concerned with\nunderstanding what behaviors make people happy in a sustainable fashion.\nRecently, there has been interest in developing technologies that help\nincorporate the findings of the science of happiness into users' daily lives by\nsteering them towards behaviors that increase happiness. With the goal of\nbuilding technology that can understand how people express their happy moments\nin text, we crowd-sourced HappyDB, a corpus of 100,000 happy moments that we\nmake publicly available. This paper describes HappyDB and its properties, and\noutlines several important NLP problems that can be studied with the help of\nthe corpus. We also apply several state-of-the-art analysis techniques to\nanalyze HappyDB. Our results demonstrate the need for deeper NLP techniques to\nbe developed which makes HappyDB an exciting resource for follow-on research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07772v1",
            "date": "2018-01-23 20:59:55+00:00",
            "title": "Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks",
            "authors": [
                "Yonatan Belinkov",
                "Llu\u00eds M\u00e0rquez",
                "Hassan Sajjad",
                "Nadir Durrani",
                "Fahim Dalvi",
                "James Glass"
            ],
            "abstract": "While neural machine translation (NMT) models provide improved translation\nquality in an elegant, end-to-end framework, it is less clear what they learn\nabout language. Recent work has started evaluating the quality of vector\nrepresentations learned by NMT models on morphological and syntactic tasks. In\nthis paper, we investigate the representations learned at different layers of\nNMT encoders. We train NMT systems on parallel data and use the trained models\nto extract features for training a classifier on two tasks: part-of-speech and\nsemantic tagging. We then measure the performance of the classifier as a proxy\nto the quality of the original NMT model for the given task. Our quantitative\nanalysis yields interesting insights regarding representation learning in NMT\nmodels. For instance, we find that higher layers are better at learning\nsemantics while lower layers tend to be better for part-of-speech tagging. We\nalso observe little effect of the target language on source-side\nrepresentations, especially with higher quality NMT models.",
            "journal ref": "IJCNLP 8 (2017), volume 1, 1-10",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07804v1",
            "date": "2018-01-23 23:03:23+00:00",
            "title": "Vietnamese Open Information Extraction",
            "authors": [
                "Diem Truong",
                "Duc-Thuan Vo",
                "U. T Nguyen"
            ],
            "abstract": "Open information extraction (OIE) is the process to extract relations and\ntheir arguments automatically from textual documents without the need to\nrestrict the search to predefined relations. In recent years, several OIE\nsystems for the English language have been created but there is not any system\nfor the Vietnamese language. In this paper, we propose a method of OIE for\nVietnamese using a clause-based approach. Accordingly, we exploit Vietnamese\ndependency parsing using grammar clauses that strives to consider all possible\nrelations in a sentence. The corresponding clause types are identified by their\npropositions as extractable relations based on their grammatical functions of\nconstituents. As a result, our system is the first OIE system named vnOIE for\nthe Vietnamese language that can generate open relations and their arguments\nfrom Vietnamese text with highly scalable extraction while being domain\nindependent. Experimental results show that our OIE system achieves promising\nresults with a precision of 83.71%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.08290v1",
            "date": "2018-01-25 07:08:04+00:00",
            "title": "A Question-Focused Multi-Factor Attention Network for Question Answering",
            "authors": [
                "Souvik Kundu",
                "Hwee Tou Ng"
            ],
            "abstract": "Neural network models recently proposed for question answering (QA) primarily\nfocus on capturing the passage-question relation. However, they have minimal\ncapability to link relevant facts distributed across multiple sentences which\nis crucial in achieving deeper understanding, such as performing multi-sentence\nreasoning, co-reference resolution, etc. They also do not explicitly focus on\nthe question and answer type which often plays a critical role in QA. In this\npaper, we propose a novel end-to-end question-focused multi-factor attention\nnetwork for answer extraction. Multi-factor attentive encoding using\ntensor-based transformation aggregates meaningful facts even when they are\nlocated in multiple sentences. To implicitly infer the answer type, we also\npropose a max-attentional question aggregation mechanism to encode a question\nvector based on the important words in a question. During prediction, we\nincorporate sequence-level encoding of the first wh-word and its immediately\nfollowing word as an additional source of question type information. Our\nproposed model achieves significant improvements over the best prior\nstate-of-the-art results on three large-scale challenging QA datasets, namely\nNewsQA, TriviaQA, and SearchQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.08337v2",
            "date": "2018-01-25 10:17:32+00:00",
            "title": "Continuous Space Reordering Models for Phrase-based MT",
            "authors": [
                "Nadir Durrani",
                "Fahim Dalvi"
            ],
            "abstract": "Bilingual sequence models improve phrase-based translation and reordering by\novercoming phrasal independence assumption and handling long range reordering.\nHowever, due to data sparsity, these models often fall back to very small\ncontext sizes. This problem has been previously addressed by learning sequences\nover generalized representations such as POS tags or word clusters. In this\npaper, we explore an alternative based on neural network models. More\nconcretely we train neuralized versions of lexicalized reordering and the\noperation sequence models using feed-forward neural network. Our results show\nimprovements of up to 0.6 and 0.5 BLEU points on top of the baseline\nGerman->English and English->German systems. We also observed improvements\ncompared to the systems that used POS tags and word clusters to train these\nmodels. Because we modify the bilingual corpus to integrate reordering\noperations, this allows us to also train a sequence-to-sequence neural MT model\nhaving explicit reordering triggers. Our motivation was to directly enable\nreordering information in the encoder-decoder framework, which otherwise relies\nsolely on the attention model to handle long range reordering. We tried both\ncoarser and fine-grained reordering operations. However, these experiments did\nnot yield any improvements over the baseline Neural MT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.08831v1",
            "date": "2018-01-26 14:45:24+00:00",
            "title": "A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction",
            "authors": [
                "Shamil Chollampatt",
                "Hwee Tou Ng"
            ],
            "abstract": "We improve automatic correction of grammatical, orthographic, and collocation\nerrors in text using a multilayer convolutional encoder-decoder neural network.\nThe network is initialized with embeddings that make use of character N-gram\ninformation to better suit this task. When evaluated on common benchmark test\ndata sets (CoNLL-2014 and JFLEG), our model substantially outperforms all prior\nneural approaches on this task as well as strong statistical machine\ntranslation-based systems with neural and task-specific features trained on the\nsame data. Our analysis shows the superiority of convolutional neural networks\nover recurrent neural networks such as long short-term memory (LSTM) networks\nin capturing the local context via attention, and thereby improving the\ncoverage in correcting grammatical errors. By ensembling multiple models, and\nincorporating an N-gram language model and edit features via rescoring, our\nnovel method becomes the first neural approach to outperform the current\nstate-of-the-art statistical machine translation-based approach, both in terms\nof grammaticality and fluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.08991v2",
            "date": "2018-01-26 22:11:10+00:00",
            "title": "A Simple Theoretical Model of Importance for Summarization",
            "authors": [
                "Maxime Peyrard"
            ],
            "abstract": "Research on summarization has mainly been driven by empirical approaches,\ncrafting systems to perform well on standard datasets with the notion of\ninformation Importance remaining latent. We argue that establishing theoretical\nmodels of Importance will advance our understanding of the task and help to\nfurther improve summarization systems. To this end, we propose simple but\nrigorous definitions of several concepts that were previously used only\nintuitively in summarization: Redundancy, Relevance, and Informativeness.\nImportance arises as a single quantity naturally unifying these concepts.\nAdditionally, we provide intuitions to interpret the proposed quantities and\nexperiments to demonstrate the potential of the framework to inform and guide\nsubsequent works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09030v2",
            "date": "2018-01-27 04:01:44+00:00",
            "title": "Exploration on Generating Traditional Chinese Medicine Prescription from Symptoms with an End-to-End method",
            "authors": [
                "Wei Li",
                "Zheng Yang",
                "Xu Sun"
            ],
            "abstract": "Traditional Chinese Medicine (TCM) is an influential form of medical\ntreatment in China and surrounding areas. In this paper, we propose a TCM\nprescription generation task that aims to automatically generate a herbal\nmedicine prescription based on textual symptom descriptions.\nSequence-to-sequence (seq2seq) model has been successful in dealing with\nsequence generation tasks. We explore a potential end-to-end solution to the\nTCM prescription generation task using seq2seq models. However, experiments\nshow that directly applying seq2seq model leads to unfruitful results due to\nthe repetition problem. To solve the problem, we propose a novel decoder with\ncoverage mechanism and a novel soft loss function. The experimental results\ndemonstrate the effectiveness of the proposed approach. Judged by professors\nwho excel in TCM, the generated prescriptions are rated 7.3 out of 10. It shows\nthat the model can indeed help with the prescribing procedure in real life.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09031v1",
            "date": "2018-01-27 04:13:52+00:00",
            "title": "Improving Word Vector with Prior Knowledge in Semantic Dictionary",
            "authors": [
                "Wei Li",
                "Yunfang Wu",
                "Xueqiang Lv"
            ],
            "abstract": "Using low dimensional vector space to represent words has been very effective\nin many NLP tasks. However, it doesn't work well when faced with the problem of\nrare and unseen words. In this paper, we propose to leverage the knowledge in\nsemantic dictionary in combination with some morphological information to build\nan enhanced vector space. We get an improvement of 2.3% over the\nstate-of-the-art Heidel Time system in temporal expression recognition, and\nobtain a large gain in other name entity recognition (NER) tasks. The semantic\ndictionary Hownet alone also shows promising results in computing lexical\nsimilarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09036v1",
            "date": "2018-01-27 05:13:55+00:00",
            "title": "A Sheaf Model of Contradictions and Disagreements. Preliminary Report and Discussion",
            "authors": [
                "Wlodek Zadrozny",
                "Luciana Garbayo"
            ],
            "abstract": "We introduce a new formal model -- based on the mathematical construct of\nsheaves -- for representing contradictory information in textual sources. This\nmodel has the advantage of letting us (a) identify the causes of the\ninconsistency; (b) measure how strong it is; (c) and do something about it,\ne.g. suggest ways to reconcile inconsistent advice. This model naturally\nrepresents the distinction between contradictions and disagreements. It is\nbased on the idea of representing natural language sentences as formulas with\nparameters sitting on lattices, creating partial orders based on predicates\nshared by theories, and building sheaves on these partial orders with products\nof lattices as stalks. Degrees of disagreement are measured by the existence of\nglobal and local sections.\n  Limitations of the sheaf approach and connections to recent work in natural\nlanguage processing, as well as the topics of contextuality in physics, data\nfusion, topological data analysis and epistemology are also discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09053v1",
            "date": "2018-01-27 08:33:22+00:00",
            "title": "Combining Convolution and Recursive Neural Networks for Sentiment Analysis",
            "authors": [
                "Vinh D. Van",
                "Thien Thai",
                "Minh-Quoc Nghiem"
            ],
            "abstract": "This paper addresses the problem of sentence-level sentiment analysis. In\nrecent years, Convolution and Recursive Neural Networks have been proven to be\neffective network architecture for sentence-level sentiment analysis.\nNevertheless, each of them has their own potential drawbacks. For alleviating\ntheir weaknesses, we combined Convolution and Recursive Neural Networks into a\nnew network architecture. In addition, we employed transfer learning from a\nlarge document-level labeled sentiment dataset to improve the word embedding in\nour models. The resulting models outperform all recent Convolution and\nRecursive Neural Networks. Beyond that, our models achieve comparable\nperformance with state-of-the-art systems on Stanford Sentiment Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09536v1",
            "date": "2018-01-21 13:23:10+00:00",
            "title": "A Survey of Word Embeddings Evaluation Methods",
            "authors": [
                "Amir Bakarov"
            ],
            "abstract": "Word embeddings are real-valued word representations able to capture lexical\nsemantics and trained on natural language corpora. Models proposing these\nrepresentations have gained popularity in the recent years, but the issue of\nthe most adequate evaluation method still remains open. This paper presents an\nextensive overview of the field of word embeddings evaluation, highlighting\nmain problems and proposing a typology of approaches to evaluation, summarizing\n16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and\nexperimental methods, systematize information about evaluation datasets and\ndiscuss some key challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09633v1",
            "date": "2018-01-29 17:18:40+00:00",
            "title": "Helping Crisis Responders Find the Informative Needle in the Tweet Haystack",
            "authors": [
                "Leon Derczynski",
                "Kenny Meesters",
                "Kalina Bontcheva",
                "Diana Maynard"
            ],
            "abstract": "Crisis responders are increasingly using social media, data and other digital\nsources of information to build a situational understanding of a crisis\nsituation in order to design an effective response. However with the increased\navailability of such data, the challenge of identifying relevant information\nfrom it also increases. This paper presents a successful automatic approach to\nhandling this problem. Messages are filtered for informativeness based on a\ndefinition of the concept drawn from prior research and crisis response\nexperts. Informative messages are tagged for actionable data -- for example,\npeople in need, threats to rescue efforts, changes in environment, and so on.\nIn all, eight categories of actionability are identified. The two components --\ninformativeness and actionability classification -- are packaged together as an\nopenly-available tool called Emina (Emergent Informativeness and\nActionability).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09746v3",
            "date": "2018-01-29 20:37:10+00:00",
            "title": "A Corpus for Modeling Word Importance in Spoken Dialogue Transcripts",
            "authors": [
                "Sushant Kafle",
                "Matt Huenerfauth"
            ],
            "abstract": "Motivated by a project to create a system for people who are deaf or\nhard-of-hearing that would use automatic speech recognition (ASR) to produce\nreal-time text captions of spoken English during in-person meetings with\nhearing individuals, we have augmented a transcript of the Switchboard\nconversational dialogue corpus with an overlay of word-importance annotations,\nwith a numeric score for each word, to indicate its importance to the meaning\nof each dialogue turn. Further, we demonstrate the utility of this corpus by\ntraining an automatic word importance labeling model; our best performing model\nhas an F-score of 0.60 in an ordinal 6-class word-importance classification\ntask with an agreement (concordance correlation coefficient) of 0.839 with the\nhuman annotators (agreement score between annotators is 0.89). Finally, we\ndiscuss our intended future applications of this resource, particularly for the\ntask of evaluating ASR performance, i.e. creating metrics that predict\nASR-output caption text usability for DHH users better thanWord Error Rate\n(WER).",
            "journal ref": "Proceedings of the Eleventh International Conference on Language\n  Resources and Evaluation (LREC 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09872v1",
            "date": "2018-01-30 07:17:40+00:00",
            "title": "A State-of-the-Art of Semantic Change Computation",
            "authors": [
                "Xuri Tang"
            ],
            "abstract": "This paper reviews the state-of-the-art of semantic change computation, one\nemerging research field in computational linguistics, proposing a framework\nthat summarizes the literature by identifying and expounding five essential\ncomponents in the field: diachronic corpus, diachronic word sense\ncharacterization, change modelling, evaluation data and data visualization.\nDespite the potential of the field, the review shows that current studies are\nmainly focused on testifying hypotheses proposed in theoretical linguistics and\nthat several core issues remain to be solved: the need for diachronic corpora\nof languages other than English, the need for comprehensive evaluation data for\nevaluation, the comparison and construction of approaches to diachronic word\nsense characterization and change modelling, and further exploration of data\nvisualization techniques for hypothesis justification.",
            "journal ref": "2018, Natural Language Engineering",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09893v1",
            "date": "2018-01-30 08:44:19+00:00",
            "title": "An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering",
            "authors": [
                "Hongzhi Zhang",
                "Guandong Xu",
                "Xiao Liang",
                "Tinglei Huang",
                "Kun fu"
            ],
            "abstract": "Relation detection plays a crucial role in Knowledge Base Question Answering\n(KBQA) because of the high variance of relation expression in the question.\nTraditional deep learning methods follow an encoding-comparing paradigm, where\nthe question and the candidate relation are represented as vectors to compare\ntheir semantic similarity. Max- or average- pooling operation, which compresses\nthe sequence of words into fixed-dimensional vectors, becomes the bottleneck of\ninformation. In this paper, we propose to learn attention-based word-level\ninteractions between questions and relations to alleviate the bottleneck issue.\nSimilar to the traditional models, the question and relation are firstly\nrepresented as sequences of vectors. Then, instead of merging the sequence into\na single vector with pooling operation, soft alignments between words from the\nquestion and the relation are learned. The aligned words are subsequently\ncompared with the convolutional neural network (CNN) and the comparison results\nare merged finally. Through performing the comparison on low-level\nrepresentations, the attention-based word-level interaction model (ABWIM)\nrelieves the information loss issue caused by merging the sequence into a\nfixed-dimensional vector before the comparison. The experimental results of\nrelation detection on both SimpleQuestions and WebQuestions datasets show that\nABWIM achieves state-of-the-art accuracy, demonstrating its effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09896v1",
            "date": "2018-01-30 08:50:26+00:00",
            "title": "Pilot study for the COST Action \"Reassembling the Republic of Letters\": language-driven network analysis of letters from the Hartlib's Papers",
            "authors": [
                "Barbara McGillivray",
                "Federico Sangati"
            ],
            "abstract": "The present report summarizes an exploratory study which we carried out in\nthe context of the COST Action IS1310 \"Reassembling the Republic of Letters,\n1500-1800\", and which is relevant to the activities of Working Group 3 \"Texts\nand Topics\" and Working Group 2 \"People and Networks\". In this study we\ninvestigated the use of Natural Language Processing (NLP) and Network Text\nAnalysis on a small sample of seventeenth-century letters selected from Hartlib\nPapers, whose records are in one of the catalogues of Early Modern Letters\nOnline (EMLO) and whose online edition is available on the website of the\nHumanities Research Institute at the University of Sheffield\n(http://www.hrionline.ac.uk/hartlib/). We outline the NLP pipeline used to\nautomatically process the texts into a network representation, in order to\nidentify the texts' \"narrative centrality\", i.e. the most central entities in\nthe texts, and the relations between them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09936v1",
            "date": "2018-01-30 11:30:38+00:00",
            "title": "PEYMA: A Tagged Corpus for Persian Named Entities",
            "authors": [
                "Mahsa Sadat Shahshahani",
                "Mahdi Mohseni",
                "Azadeh Shakery",
                "Heshaam Faili"
            ],
            "abstract": "The goal in the NER task is to classify proper nouns of a text into classes\nsuch as person, location, and organization. This is an important preprocessing\nstep in many NLP tasks such as question-answering and summarization. Although\nmany research studies have been conducted in this area in English and the\nstate-of-the-art NER systems have reached performances of higher than 90\npercent in terms of F1 measure, there are very few research studies for this\ntask in Persian. One of the main important causes of this may be the lack of a\nstandard Persian NER dataset to train and test NER systems. In this research we\ncreate a standard, big-enough tagged Persian NER dataset which will be\ndistributed for free for research purposes. In order to construct such a\nstandard dataset, we studied standard NER datasets which are constructed for\nEnglish researches and found out that almost all of these datasets are\nconstructed using news texts. So we collected documents from ten news websites.\nLater, in order to provide annotators with some guidelines to tag these\ndocuments, after studying guidelines used for constructing CoNLL and MUC\nstandard English datasets, we set our own guidelines considering the Persian\nlinguistic rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10198v1",
            "date": "2018-01-30 20:07:01+00:00",
            "title": "Generating Wikipedia by Summarizing Long Sequences",
            "authors": [
                "Peter J. Liu",
                "Mohammad Saleh",
                "Etienne Pot",
                "Ben Goodrich",
                "Ryan Sepassi",
                "Lukasz Kaiser",
                "Noam Shazeer"
            ],
            "abstract": "We show that generating English Wikipedia articles can be approached as a\nmulti- document summarization of source documents. We use extractive\nsummarization to coarsely identify salient information and a neural abstractive\nmodel to generate the article. For the abstractive model, we introduce a\ndecoder-only architecture that can scalably attend to very long sequences, much\nlonger than typical encoder- decoder architectures used in sequence\ntransduction. We show that this model can generate fluent, coherent\nmulti-sentence paragraphs and even whole Wikipedia articles. When given\nreference documents, we show it can extract relevant factual information as\nreflected in perplexity, ROUGE scores and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10293v1",
            "date": "2018-01-31 04:14:11+00:00",
            "title": "Paraphrase-Supervised Models of Compositionality",
            "authors": [
                "Avneesh Saluja",
                "Chris Dyer",
                "Jean-David Ruvini"
            ],
            "abstract": "Compositional vector space models of meaning promise new solutions to\nstubborn language understanding problems. This paper makes two contributions\ntoward this end: (i) it uses automatically-extracted paraphrase examples as a\nsource of supervision for training compositional models, replacing previous\nwork which relied on manual annotations used for the same purpose, and (ii)\ndevelops a context-aware model for scoring phrasal compositionality.\nExperimental results indicate that these multiple sources of information can be\nused to learn partial semantic supervision that matches previous techniques in\nintrinsic evaluation tasks. Our approaches are also evaluated for their impact\non a machine translation system where we show improvements in translation\nquality, demonstrating that compositionality in interpretation correlates with\ncompositionality in translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10296v2",
            "date": "2018-01-31 04:30:32+00:00",
            "title": "Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling",
            "authors": [
                "Tao Shen",
                "Tianyi Zhou",
                "Guodong Long",
                "Jing Jiang",
                "Sen Wang",
                "Chengqi Zhang"
            ],
            "abstract": "Many natural language processing tasks solely rely on sparse dependencies\nbetween a few tokens in a sentence. Soft attention mechanisms show promising\nperformance in modeling local/global dependencies by soft probabilities between\nevery two tokens, but they are not effective and efficient when applied to long\nsentences. By contrast, hard attention mechanisms directly select a subset of\ntokens but are difficult and inefficient to train due to their combinatorial\nnature. In this paper, we integrate both soft and hard attention into one\ncontext fusion model, \"reinforced self-attention (ReSA)\", for the mutual\nbenefit of each other. In ReSA, a hard attention trims a sequence for a soft\nself-attention to process, while the soft attention feeds reward signals back\nto facilitate the training of the hard one. For this purpose, we develop a\nnovel hard attention called \"reinforced sequence sampling (RSS)\", selecting\ntokens in parallel and trained via policy gradient. Using two RSS modules, ReSA\nefficiently extracts the sparse dependencies between each pair of selected\ntokens. We finally propose an RNN/CNN-free sentence-encoding model, \"reinforced\nself-attention network (ReSAN)\", solely based on ReSA. It achieves\nstate-of-the-art performance on both Stanford Natural Language Inference (SNLI)\nand Sentences Involving Compositional Knowledge (SICK) datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10314v2",
            "date": "2018-01-31 06:40:40+00:00",
            "title": "Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph",
            "authors": [
                "Amrita Saha",
                "Vardaan Pahuja",
                "Mitesh M. Khapra",
                "Karthik Sankaranarayanan",
                "Sarath Chandar"
            ],
            "abstract": "While conversing with chatbots, humans typically tend to ask many questions,\na significant portion of which can be answered by referring to large-scale\nknowledge graphs (KG). While Question Answering (QA) and dialog systems have\nbeen studied independently, there is a need to study them closely to evaluate\nsuch real-world scenarios faced by bots involving both these tasks. Towards\nthis end, we introduce the task of Complex Sequential QA which combines the two\ntasks of (i) answering factual questions through complex inferencing over a\nrealistic-sized KG of millions of entities, and (ii) learning to converse\nthrough a series of coherently linked QA pairs. Through a labor intensive\nsemi-automatic process, involving in-house and crowdsourced workers, we created\na dataset containing around 200K dialogs with a total of 1.6M turns. Further,\nunlike existing large scale QA datasets which contain simple questions that can\nbe answered from a single tuple, the questions in our dialogs require a larger\nsubgraph of the KG. Specifically, our dataset has questions which require\nlogical, quantitative, and comparative reasoning as well as their combinations.\nThis calls for models which can: (i) parse complex natural language questions,\n(ii) use conversation context to resolve coreferences and ellipsis in\nutterances, (iii) ask for clarifications for ambiguous queries, and finally\n(iv) retrieve relevant subgraphs of the KG to answer such questions. However,\nour experiments with a combination of state of the art dialog and QA models\nshow that they clearly do not achieve the above objectives and are inadequate\nfor dealing with such complex real world settings. We believe that this new\ndataset coupled with the limitations of existing models as reported in this\npaper should encourage further research in Complex Sequential QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00015v1",
            "date": "2018-03-30 18:09:39+00:00",
            "title": "ESPnet: End-to-End Speech Processing Toolkit",
            "authors": [
                "Shinji Watanabe",
                "Takaaki Hori",
                "Shigeki Karita",
                "Tomoki Hayashi",
                "Jiro Nishitoba",
                "Yuya Unno",
                "Nelson Enrique Yalta Soplin",
                "Jahn Heymann",
                "Matthew Wiesner",
                "Nanxin Chen",
                "Adithya Renduchintala",
                "Tsubasa Ochiai"
            ],
            "abstract": "This paper introduces a new open source platform for end-to-end speech\nprocessing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech\nrecognition (ASR), and adopts widely-used dynamic neural network toolkits,\nChainer and PyTorch, as a main deep learning engine. ESPnet also follows the\nKaldi ASR toolkit style for data processing, feature extraction/format, and\nrecipes to provide a complete setup for speech recognition and other speech\nprocessing experiments. This paper explains a major architecture of this\nsoftware platform, several important functionalities, which differentiate\nESPnet from other open source ASR toolkits, and experimental results with major\nASR benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00079v1",
            "date": "2018-03-30 23:05:15+00:00",
            "title": "Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning",
            "authors": [
                "Sandeep Subramanian",
                "Adam Trischler",
                "Yoshua Bengio",
                "Christopher J Pal"
            ],
            "abstract": "A lot of the recent success in natural language processing (NLP) has been\ndriven by distributed vector representations of words trained on large amounts\nof text in an unsupervised manner. These representations are typically used as\ngeneral purpose features for words across a range of NLP problems. However,\nextending this success to learning representations of sequences of words, such\nas sentences, remains an open problem. Recent work has explored unsupervised as\nwell as supervised learning techniques with different training objectives to\nlearn general purpose fixed-length sentence representations. In this work, we\npresent a simple, effective multi-task learning framework for sentence\nrepresentations that combines the inductive biases of diverse training\nobjectives in a single model. We train this model on several data sources with\nmultiple training objectives on over 100 million sentences. Extensive\nexperiments demonstrate that sharing a single recurrent sentence encoder across\nweakly related tasks leads to consistent improvements over previous methods. We\npresent substantial improvements in the context of transfer learning and\nlow-resource settings using our learned general-purpose representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00146v1",
            "date": "2018-03-31 10:15:44+00:00",
            "title": "Towards Learning Transferable Conversational Skills using Multi-dimensional Dialogue Modelling",
            "authors": [
                "Simon Keizer",
                "Verena Rieser"
            ],
            "abstract": "Recent statistical approaches have improved the robustness and scalability of\nspoken dialogue systems. However, despite recent progress in domain adaptation,\ntheir reliance on in-domain data still limits their cross-domain scalability.\nIn this paper, we argue that this problem can be addressed by extending current\nmodels to reflect and exploit the multi-dimensional nature of human dialogue.\nWe present our multi-dimensional, statistical dialogue management framework, in\nwhich transferable conversational skills can be learnt by separating out\ndomain-independent dimensions of communication and using multi-agent\nreinforcement learning. Our initial experiments with a simulated user show that\nwe can speed up the learning process by transferring learnt policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00247v2",
            "date": "2018-04-01 01:59:52+00:00",
            "title": "Training Tips for the Transformer Model",
            "authors": [
                "Martin Popel",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "This article describes our experiments in neural machine translation using\nthe recent Tensor2Tensor framework and the Transformer sequence-to-sequence\nmodel (Vaswani et al., 2017). We examine some of the critical parameters that\naffect the final translation quality, memory usage, training stability and\ntraining time, concluding each experiment with a set of recommendations for\nfellow researchers. In addition to confirming the general mantra \"more data and\nlarger models\", we address scaling to multiple GPUs and provide practical tips\nfor improved training regarding batch size, learning rate, warmup steps,\nmaximum sentence length and checkpoint averaging. We hope that our observations\nwill allow others to get better results given their particular hardware and\ndata constraints.",
            "journal ref": "The Prague Bulletin of Mathematical Linguistics 110, April 2018,\n  pp. 43-70",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00316v1",
            "date": "2018-04-01 16:35:21+00:00",
            "title": "Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings",
            "authors": [
                "Da-Rong Liu",
                "Kuan-Yu Chen",
                "Hung-Yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Unsupervised discovery of acoustic tokens from audio corpora without\nannotation and learning vector representations for these tokens have been\nwidely studied. Although these techniques have been shown successful in some\napplications such as query-by-example Spoken Term Detection (STD), the lack of\nmapping relationships between these discovered tokens and real phonemes have\nlimited the down-stream applications. This paper represents probably the first\nattempt towards the goal of completely unsupervised phoneme recognition, or\nmapping audio signals to phoneme sequences without phoneme-labeled audio data.\nThe basic idea is to cluster the embedded acoustic tokens and learn the mapping\nbetween the cluster sequences and the unknown phoneme sequences with a\nGenerative Adversarial Network (GAN). An unsupervised phoneme recognition\naccuracy of 36% was achieved in the preliminary experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00318v1",
            "date": "2018-04-01 16:46:53+00:00",
            "title": "Joint Learning of Interactive Spoken Content Retrieval and Trainable User Simulator",
            "authors": [
                "Pei-Hung Chung",
                "Kuan Tung",
                "Ching-Lun Tai",
                "Hung-Yi Lee"
            ],
            "abstract": "User-machine interaction is crucial for information retrieval, especially for\nspoken content retrieval, because spoken content is difficult to browse, and\nspeech recognition has a high degree of uncertainty. In interactive retrieval,\nthe machine takes different actions to interact with the user to obtain better\nretrieval results; here it is critical to select the most efficient action. In\nprevious work, deep Q-learning techniques were proposed to train an interactive\nretrieval system but rely on a hand-crafted user simulator; building a reliable\nuser simulator is difficult. In this paper, we further improve the interactive\nspoken content retrieval framework by proposing a learnable user simulator\nwhich is jointly trained with interactive retrieval system, making the\nhand-crafted user simulator unnecessary. The experimental results show that the\nlearned simulated users not only achieve larger rewards than the hand-crafted\nones but act more like real users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00320v1",
            "date": "2018-04-01 17:12:47+00:00",
            "title": "Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension",
            "authors": [
                "Chia-Hsuan Li",
                "Szu-Lin Wu",
                "Chi-Liang Liu",
                "Hung-yi Lee"
            ],
            "abstract": "Reading comprehension has been widely studied. One of the most representative\nreading comprehension tasks is Stanford Question Answering Dataset (SQuAD), on\nwhich machine is already comparable with human. On the other hand, accessing\nlarge collections of multimedia or spoken content is much more difficult and\ntime-consuming than plain text content for humans. It's therefore highly\nattractive to develop machines which can automatically understand spoken\ncontent. In this paper, we propose a new listening comprehension task - Spoken\nSQuAD. On the new task, we found that speech recognition errors have\ncatastrophic impact on machine comprehension, and several approaches are\nproposed to mitigate the impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00344v3",
            "date": "2018-04-01 20:50:57+00:00",
            "title": "Marian: Fast Neural Machine Translation in C++",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz",
                "Tomasz Dwojak",
                "Hieu Hoang",
                "Kenneth Heafield",
                "Tom Neckermann",
                "Frank Seide",
                "Ulrich Germann",
                "Alham Fikri Aji",
                "Nikolay Bogoychev",
                "Andr\u00e9 F. T. Martins",
                "Alexandra Birch"
            ],
            "abstract": "We present Marian, an efficient and self-contained Neural Machine Translation\nframework with an integrated automatic differentiation engine based on dynamic\ncomputation graphs. Marian is written entirely in C++. We describe the design\nof the encoder-decoder framework and demonstrate that a research-friendly\ntoolkit can achieve high training and translation speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00482v1",
            "date": "2018-04-02 13:31:11+00:00",
            "title": "Real Time Sentiment Change Detection of Twitter Data Streams",
            "authors": [
                "Sotiris K. Tasoulis",
                "Aristidis G. Vrahatis",
                "Spiros V. Georgakopoulos",
                "Vassilis P. Plagianakos"
            ],
            "abstract": "In the past few years, there has been a huge growth in Twitter sentiment\nanalysis having already provided a fair amount of research on sentiment\ndetection of public opinion among Twitter users. Given the fact that Twitter\nmessages are generated constantly with dizzying rates, a huge volume of\nstreaming data is created, thus there is an imperative need for accurate\nmethods for knowledge discovery and mining of this information. Although there\nexists a plethora of twitter sentiment analysis methods in the recent\nliterature, the researchers have shifted to real-time sentiment identification\non twitter streaming data, as expected. A major challenge is to deal with the\nBig Data challenges arising in Twitter streaming applications concerning both\nVolume and Velocity. Under this perspective, in this paper, a methodological\napproach based on open source tools is provided for real-time detection of\nchanges in sentiment that is ultra efficient with respect to both memory\nconsumption and computational cost. This is achieved by iteratively collecting\ntweets in real time and discarding them immediately after their process. For\nthis purpose, we employ the Lexicon approach for sentiment characterizations,\nwhile change detection is achieved through appropriate control charts that do\nnot require historical information. We believe that the proposed methodology\nprovides the trigger for a potential large-scale monitoring of threads in an\nattempt to discover fake news spread or propaganda efforts in their early\nstages. Our experimental real-time analysis based on a recent hashtag provides\nevidence that the proposed approach can detect meaningful sentiment changes\nacross a hashtags lifetime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00520v2",
            "date": "2018-04-02 14:09:01+00:00",
            "title": "NIHRIO at SemEval-2018 Task 3: A Simple and Accurate Neural Network Model for Irony Detection in Twitter",
            "authors": [
                "Thanh Vu",
                "Dat Quoc Nguyen",
                "Xuan-Son Vu",
                "Dai Quoc Nguyen",
                "Michael Catt",
                "Michael Trenell"
            ],
            "abstract": "This paper describes our NIHRIO system for SemEval-2018 Task 3 \"Irony\ndetection in English tweets\". We propose to use a simple neural network\narchitecture of Multilayer Perceptron with various types of input features\nincluding: lexical, syntactic, semantic and polarity features. Our system\nachieves very high performance in both subtasks of binary and multi-class irony\ndetection in tweets. In particular, we rank third using the accuracy metric and\nfifth using the F1 metric. Our code is available at\nhttps://github.com/NIHRIO/IronyDetectionInTwitter",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00540v1",
            "date": "2018-03-29 10:42:03+00:00",
            "title": "A Systematic Review of Automated Grammar Checking in English Language",
            "authors": [
                "Madhvi Soni",
                "Jitendra Singh Thakur"
            ],
            "abstract": "Grammar checking is the task of detection and correction of grammatical\nerrors in the text. English is the dominating language in the field of science\nand technology. Therefore, the non-native English speakers must be able to use\ncorrect English grammar while reading, writing or speaking. This generates the\nneed of automatic grammar checking tools. So far many approaches have been\nproposed and implemented. But less efforts have been made in surveying the\nliterature in the past decade. The objective of this systematic review is to\nexamine the existing literature, highlighting the current issues and suggesting\nthe potential directions of future research. This systematic review is a result\nof analysis of 12 primary studies obtained after designing a search strategy\nfor selecting papers found on the web. We also present a possible scheme for\nthe classification of grammar errors. Among the main observations, we found\nthat there is a lack of efficient and robust grammar checking tools for real\ntime applications. We present several useful illustrations- most prominent are\nthe schematic diagrams that we provide for each approach and a table that\nsummarizes these approaches along different dimensions such as target error\ntypes, linguistic dataset used, strengths and limitations of the approach. This\nfacilitates better understandability, comparison and evaluation of previous\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00619v3",
            "date": "2018-04-02 16:32:53+00:00",
            "title": "Modeling Semantic Plausibility by Injecting World Knowledge",
            "authors": [
                "Su Wang",
                "Greg Durrett",
                "Katrin Erk"
            ],
            "abstract": "Distributional data tells us that a man can swallow candy, but not that a man\ncan swallow a paintball, since this is never attested. However both are\nphysically plausible events. This paper introduces the task of semantic\nplausibility: recognizing plausible but possibly novel events. We present a new\ncrowdsourced dataset of semantic plausibility judgments of single events such\nas \"man swallow paintball\". Simple models based on distributional\nrepresentations perform poorly on this task, despite doing well on selection\npreference, but injecting manually elicited knowledge about entity properties\nprovides a substantial performance boost. Our error analysis shows that our new\ndataset is a great testbed for semantic plausibility models: more sophisticated\nknowledge representation and propagation could address many of the remaining\nerrors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00804v1",
            "date": "2018-04-03 03:19:31+00:00",
            "title": "Automatic Normalization of Word Variations in Code-Mixed Social Media Text",
            "authors": [
                "Rajat Singh",
                "Nurendra Choudhary",
                "Manish Shrivastava"
            ],
            "abstract": "Social media platforms such as Twitter and Facebook are becoming popular in\nmultilingual societies. This trend induces portmanteau of South Asian languages\nwith English. The blend of multiple languages as code-mixed data has recently\nbecome popular in research communities for various NLP tasks. Code-mixed data\nconsist of anomalies such as grammatical errors and spelling variations. In\nthis paper, we leverage the contextual property of words where the different\nspelling variation of words share similar context in a large noisy social media\ntext. We capture different variations of words belonging to same context in an\nunsupervised manner using distributed representations of words. Our experiments\nreveal that preprocessing of the code-mixed dataset based on our approach\nimproves the performance in state-of-the-art part-of-speech tagging\n(POS-tagging) and sentiment analysis tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00805v1",
            "date": "2018-04-03 03:19:36+00:00",
            "title": "Emotions are Universal: Learning Sentiment Based Representations of Resource-Poor Languages using Siamese Networks",
            "authors": [
                "Nurendra Choudhary",
                "Rajat Singh",
                "Ishita Bindlish",
                "Manish Shrivastava"
            ],
            "abstract": "Machine learning approaches in sentiment analysis principally rely on the\nabundance of resources. To limit this dependence, we propose a novel method\ncalled Siamese Network Architecture for Sentiment Analysis (SNASA) to learn\nrepresentations of resource-poor languages by jointly training them with\nresource-rich languages using a siamese network.\n  SNASA model consists of twin Bi-directional Long Short-Term Memory Recurrent\nNeural Networks (Bi-LSTM RNN) with shared parameters joined by a contrastive\nloss function, based on a similarity metric. The model learns the sentence\nrepresentations of resource-poor and resource-rich language in a common\nsentiment space by using a similarity metric based on their individual\nsentiments. The model, hence, projects sentences with similar sentiment closer\nto each other and the sentences with different sentiment farther from each\nother. Experiments on large-scale datasets of resource-rich languages - English\nand Spanish and resource-poor languages - Hindi and Telugu reveal that SNASA\noutperforms the state-of-the-art sentiment analysis approaches based on\ndistributional semantics, semantic rules, lexicon lists and deep neural network\nrepresentations without sh",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00806v1",
            "date": "2018-04-03 03:19:41+00:00",
            "title": "Sentiment Analysis of Code-Mixed Languages leveraging Resource Rich Languages",
            "authors": [
                "Nurendra Choudhary",
                "Rajat Singh",
                "Ishita Bindlish",
                "Manish Shrivastava"
            ],
            "abstract": "Code-mixed data is an important challenge of natural language processing\nbecause its characteristics completely vary from the traditional structures of\nstandard languages.\n  In this paper, we propose a novel approach called Sentiment Analysis of\nCode-Mixed Text (SACMT) to classify sentences into their corresponding\nsentiment - positive, negative or neutral, using contrastive learning. We\nutilize the shared parameters of siamese networks to map the sentences of\ncode-mixed and standard languages to a common sentiment space. Also, we\nintroduce a basic clustering based preprocessing method to capture variations\nof code-mixed transliterated words. Our experiments reveal that SACMT\noutperforms the state-of-the-art approaches in sentiment analysis for\ncode-mixed text by 7.6% in accuracy and 10.1% in F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00828v1",
            "date": "2018-04-03 05:09:32+00:00",
            "title": "Incorporating Word Embeddings into Open Directory Project based Large-scale Classification",
            "authors": [
                "Kang-Min Kim",
                "Aliyeva Dinara",
                "Byung-Ju Choi",
                "SangKeun Lee"
            ],
            "abstract": "Recently, implicit representation models, such as embedding or deep learning,\nhave been successfully adopted to text classification task due to their\noutstanding performance. However, these approaches are limited to small- or\nmoderate-scale text classification. Explicit representation models are often\nused in a large-scale text classification, like the Open Directory Project\n(ODP)-based text classification. However, the performance of these models is\nlimited to the associated knowledge bases. In this paper, we incorporate word\nembeddings into the ODP-based large-scale classification. To this end, we first\ngenerate category vectors, which represent the semantics of ODP categories by\njointly modeling word embeddings and the ODP-based text classification. We then\npropose a novel semantic similarity measure, which utilizes the category and\nword vectors obtained from the joint model and word embeddings, respectively.\nThe evaluation results clearly show the efficacy of our methodology in\nlarge-scale text classification. The proposed scheme exhibits significant\nimprovements of 10% and 28% in terms of macro-averaging F1-score and precision\nat k, respectively, over state-of-the-art techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00832v2",
            "date": "2018-04-03 05:33:38+00:00",
            "title": "Attentive Sequence-to-Sequence Learning for Diacritic Restoration of Yor\u00f9b\u00e1 Language Text",
            "authors": [
                "Iroro Orife"
            ],
            "abstract": "Yor\\`ub\\'a is a widely spoken West African language with a writing system\nrich in tonal and orthographic diacritics. With very few exceptions, diacritics\nare omitted from electronic texts, due to limited device and application\nsupport. Diacritics provide morphological information, are crucial for lexical\ndisambiguation, pronunciation and are vital for any Yor\\`ub\\'a text-to-speech\n(TTS), automatic speech recognition (ASR) and natural language processing (NLP)\ntasks. Reframing Automatic Diacritic Restoration (ADR) as a machine translation\ntask, we experiment with two different attentive Sequence-to-Sequence neural\nmodels to process undiacritized text. On our evaluation dataset, this approach\nproduces diacritization error rates of less than 5%. We have released\npre-trained models, datasets and source-code as an open-source project to\nadvance efforts on Yor\\`ub\\'a language technology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00968v1",
            "date": "2018-03-31 19:52:26+00:00",
            "title": "In-depth Question classification using Convolutional Neural Networks",
            "authors": [
                "Prudhvi Raj Dachapally",
                "Srikanth Ramanam"
            ],
            "abstract": "Convolutional neural networks for computer vision are fairly intuitive. In a\ntypical CNN used in image classification, the first layers learn edges, and the\nfollowing layers learn some filters that can identify an object. But CNNs for\nNatural Language Processing are not used often and are not completely\nintuitive. We have a good idea about what the convolution filters learn for the\ntask of text classification, and to that, we propose a neural network structure\nthat will be able to give good results in less time. We will be using\nconvolutional neural networks to predict the primary or broader topic of a\nquestion, and then use separate networks for each of these predicted topics to\naccurately classify their sub-topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01041v1",
            "date": "2018-04-03 15:41:44+00:00",
            "title": "Multi-lingual neural title generation for e-Commerce browse pages",
            "authors": [
                "Prashant Mathur",
                "Nicola Ueffing",
                "Gregor Leusch"
            ],
            "abstract": "To provide better access of the inventory to buyers and better search engine\noptimization, e-Commerce websites are automatically generating millions of\neasily searchable browse pages. A browse page consists of a set of slot\nname/value pairs within a given category, grouping multiple items which share\nsome characteristics. These browse pages require a title describing the content\nof the page. Since the number of browse pages are huge, manual creation of\nthese titles is infeasible. Previous statistical and neural approaches depend\nheavily on the availability of large amounts of data in a language. In this\nresearch, we apply sequence-to-sequence models to generate titles for high- &\nlow-resourced languages by leveraging transfer learning. We train these models\non multi-lingual data, thereby creating one joint model which can generate\ntitles in various different languages. Performance of the title generation\nsystem is evaluated on three different languages; English, German, and French,\nwith a particular focus on low-resourced French language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01760v1",
            "date": "2018-04-05 10:24:15+00:00",
            "title": "Domain Adaptation for Statistical Machine Translation",
            "authors": [
                "Longyue Wang"
            ],
            "abstract": "Statistical machine translation (SMT) systems perform poorly when it is\napplied to new target domains. Our goal is to explore domain adaptation\napproaches and techniques for improving the translation quality of\ndomain-specific SMT systems. However, translating texts from a specific domain\n(e.g., medicine) is full of challenges. The first challenge is ambiguity. Words\nor phrases contain different meanings in different contexts. The second one is\nlanguage style due to the fact that texts from different genres are always\npresented in different syntax, length and structural organization. The third\none is the out-of-vocabulary words (OOVs) problem. In-domain training data are\noften scarce with low terminology coverage. In this thesis, we explore the\nstate-of-the-art domain adaptation approaches and propose effective solutions\nto address those problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01768v1",
            "date": "2018-04-05 10:35:51+00:00",
            "title": "Chinese-Portuguese Machine Translation: A Study on Building Parallel Corpora from Comparable Texts",
            "authors": [
                "Siyou Liu",
                "Longyue Wang",
                "Chao-Hong Liu"
            ],
            "abstract": "Although there are increasing and significant ties between China and\nPortuguese-speaking countries, there is not much parallel corpora in the\nChinese-Portuguese language pair. Both languages are very populous, with 1.2\nbillion native Chinese speakers and 279 million native Portuguese speakers, the\nlanguage pair, however, could be considered as low-resource in terms of\navailable parallel corpora. In this paper, we describe our methods to curate\nChinese-Portuguese parallel corpora and evaluate their quality. We extracted\nbilingual data from Macao government websites and proposed a hierarchical\nstrategy to build a large parallel corpus. Experiments are conducted on\nexisting and our corpora using both Phrased-Based Machine Translation (PBMT)\nand the state-of-the-art Neural Machine Translation (NMT) models. The results\nof this work can be used as a benchmark for future Chinese-Portuguese MT\nsystems. The approach we used in this paper also shows a good example on how to\nboost performance of MT systems for low-resource language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01772v1",
            "date": "2018-04-05 10:48:26+00:00",
            "title": "Not just about size - A Study on the Role of Distributed Word Representations in the Analysis of Scientific Publications",
            "authors": [
                "Andres Garcia",
                "Jose Manuel Gomez-Perez"
            ],
            "abstract": "The emergence of knowledge graphs in the scholarly communication domain and\nrecent advances in artificial intelligence and natural language processing\nbring us closer to a scenario where intelligent systems can assist scientists\nover a range of knowledge-intensive tasks. In this paper we present\nexperimental results about the generation of word embeddings from scholarly\npublications for the intelligent processing of scientific texts extracted from\nSciGraph. We compare the performance of domain-specific embeddings with\nexisting pre-trained vectors generated from very large and general purpose\ncorpora. Our results suggest that there is a trade-off between corpus\nspecificity and volume. Embeddings from domain-specific scientific corpora\neffectively capture the semantics of the domain. On the other hand, obtaining\ncomparable results through general corpora can also be achieved, but only in\nthe presence of very large corpora of well formed text. Furthermore, We also\nshow that the degree of overlapping between knowledge areas is directly related\nto the performance of embeddings in domain evaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01778v1",
            "date": "2018-04-05 11:02:38+00:00",
            "title": "Word Segmentation as Graph Partition",
            "authors": [
                "Yuanhao Liu",
                "Sheng Yu"
            ],
            "abstract": "We propose a new approach to the Chinese word segmentation problem that\nconsiders the sentence as an undirected graph, whose nodes are the characters.\nOne can use various techniques to compute the edge weights that measure the\nconnection strength between characters. Spectral graph partition algorithms are\nused to group the characters and achieve word segmentation. We follow the graph\npartition approach and design several unsupervised algorithms, and we show\ntheir inspiring segmentation results on two corpora: (1) electronic health\nrecords in Chinese, and (2) benchmark data from the Second International\nChinese Word Segmentation Bakeoff.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01855v1",
            "date": "2018-04-03 03:19:45+00:00",
            "title": "Contrastive Learning of Emoji-based Representations for Resource-Poor Languages",
            "authors": [
                "Nurendra Choudhary",
                "Rajat Singh",
                "Ishita Bindlish",
                "Manish Shrivastava"
            ],
            "abstract": "The introduction of emojis (or emoticons) in social media platforms has given\nthe users an increased potential for expression. We propose a novel method\ncalled Classification of Emojis using Siamese Network Architecture (CESNA) to\nlearn emoji-based representations of resource-poor languages by jointly\ntraining them with resource-rich languages using a siamese network.\n  CESNA model consists of twin Bi-directional Long Short-Term Memory Recurrent\nNeural Networks (Bi-LSTM RNN) with shared parameters joined by a contrastive\nloss function based on a similarity metric. The model learns the\nrepresentations of resource-poor and resource-rich language in a common emoji\nspace by using a similarity metric based on the emojis present in sentences\nfrom both languages. The model, hence, projects sentences with similar emojis\ncloser to each other and the sentences with different emojis farther from one\nanother. Experiments on large-scale Twitter datasets of resource-rich languages\n- English and Spanish and resource-poor languages - Hindi and Telugu reveal\nthat CESNA outperforms the state-of-the-art emoji prediction approaches based\non distributional semantics, semantic rules, lexicon lists and deep neural\nnetwork representations without shared parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02042v1",
            "date": "2018-04-05 20:01:48+00:00",
            "title": "ETH-DS3Lab at SemEval-2018 Task 7: Effectively Combining Recurrent and Convolutional Neural Networks for Relation Classification and Extraction",
            "authors": [
                "Jonathan Rotsztejn",
                "Nora Hollenstein",
                "Ce Zhang"
            ],
            "abstract": "Reliably detecting relevant relations between entities in unstructured text\nis a valuable resource for knowledge extraction, which is why it has awaken\nsignificant interest in the field of Natural Language Processing. In this\npaper, we present a system for relation classification and extraction based on\nan ensemble of convolutional and recurrent neural networks that ranked first in\n3 out of the 4 subtasks at SemEval 2018 Task 7. We provide detailed\nexplanations and grounds for the design choices behind the most relevant\nfeatures and analyze their importance.",
            "journal ref": "Proceedings of The 12th International Workshop on Semantic\n  Evaluation, 2018, Association for Computational Linguistics, p. 689-696,\n  http://aclweb.org/anthology/S18-1112",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02063v1",
            "date": "2018-04-05 21:28:54+00:00",
            "title": "Few-Shot Text Classification with Pre-Trained Word Embeddings and a Human in the Loop",
            "authors": [
                "Katherine Bailey",
                "Sunny Chopra"
            ],
            "abstract": "Most of the literature around text classification treats it as a supervised\nlearning problem: given a corpus of labeled documents, train a classifier such\nthat it can accurately predict the classes of unseen documents. In industry,\nhowever, it is not uncommon for a business to have entire corpora of documents\nwhere few or none have been classified, or where existing classifications have\nbecome meaningless. With web content, for example, poor taxonomy management can\nresult in labels being applied indiscriminately, making filtering by these\nlabels unhelpful. Our work aims to make it possible to classify an entire\ncorpus of unlabeled documents using a human-in-the-loop approach, where the\ncontent owner manually classifies just one or two documents per category and\nthe rest can be automatically classified. This \"few-shot\" learning approach\nrequires rich representations of the documents such that those that have been\nmanually labeled can be treated as prototypes, and automatic classification of\nthe rest is a simple case of measuring the distance to prototypes. This\napproach uses pre-trained word embeddings, where documents are represented\nusing a simple weighted average of constituent word embeddings. We have tested\nthe accuracy of the approach on existing labeled datasets and provide the\nresults here. We have also made code available for reproducing the results we\ngot on the 20 Newsgroups dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02186v2",
            "date": "2018-04-06 10:17:16+00:00",
            "title": "Enrichment of OntoSenseNet: Adding a Sense-annotated Telugu lexicon",
            "authors": [
                "Sreekavitha Parupalli",
                "Navjyoti Singh"
            ],
            "abstract": "The paper describes the enrichment of OntoSenseNet - a verb-centric lexical\nresource for Indian Languages. This resource contains a newly developed\nTelugu-Telugu dictionary. It is important because native speakers can better\nannotate the senses when both the word and its meaning are in Telugu. Hence\nefforts are made to develop a soft copy of Telugu dictionary. Our resource also\nhas manually annotated gold standard corpus consisting 8483 verbs, 253 adverbs\nand 1673 adjectives. Annotations are done by native speakers according to\ndefined annotation guidelines. In this paper, we provide an overview of the\nannotation procedure and present the validation of our resource through\ninter-annotator agreement. Concepts of sense-class and sense-type are\ndiscussed. Additionally, we discuss the potential of lexical sense-annotated\ncorpora in improving word sense disambiguation (WSD) tasks. Telugu WordNet is\ncrowd-sourced for annotation of individual words in synsets and is compared\nwith the developed sense-annotated lexicon (OntoSenseNet) to examine the\nimprovement. Also, we present a special categorization (spatio-temporal\nclassification) of adjectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02286v1",
            "date": "2018-04-06 14:11:33+00:00",
            "title": "Chart Parsing Multimodal Grammars",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "The short note describes the chart parser for multimodal type-logical\ngrammars which has been developed in conjunction with the type-logical treebank\nfor French. The chart parser presents an incomplete but fast implementation of\nproof search for multimodal type-logical grammars using the \"deductive parsing\"\nframework. Proofs found can be transformed to natural deduction proofs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02472v1",
            "date": "2018-04-06 22:11:17+00:00",
            "title": "Neural models of factuality",
            "authors": [
                "Rachel Rudinger",
                "Aaron Steven White",
                "Benjamin Van Durme"
            ],
            "abstract": "We present two neural models for event factuality prediction, which yield\nsignificant performance gains over previous models on three event factuality\ndatasets: FactBank, UW, and MEANTIME. We also present a substantial expansion\nof the It Happened portion of the Universal Decompositional Semantics dataset,\nyielding the largest event factuality dataset to date. We report model results\non this extended factuality dataset as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02504v1",
            "date": "2018-04-07 03:56:55+00:00",
            "title": "Scalable Sentiment for Sequence-to-sequence Chatbot Response with Performance Analysis",
            "authors": [
                "Chih-Wei Lee",
                "Yau-Shian Wang",
                "Tsung-Yuan Hsu",
                "Kuan-Yu Chen",
                "Hung-Yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Conventional seq2seq chatbot models only try to find the sentences with the\nhighest probabilities conditioned on the input sequences, without considering\nthe sentiment of the output sentences. Some research works trying to modify the\nsentiment of the output sequences were reported. In this paper, we propose five\nmodels to scale or adjust the sentiment of the chatbot response: persona-based\nmodel, reinforcement learning, plug and play model, sentiment transformation\nnetwork and cycleGAN, all based on the conventional seq2seq model. We also\ndevelop two evaluation metrics to estimate if the responses are reasonable\ngiven the input. These metrics together with other two popularly used metrics\nwere used to analyze the performance of the five proposed models on different\naspects, and reinforcement learning and cycleGAN were shown to be very\nattractive. The evaluation metrics were also found to be well correlated with\nhuman evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02545v2",
            "date": "2018-04-07 11:06:26+00:00",
            "title": "Evaluating historical text normalization systems: How well do they generalize?",
            "authors": [
                "Alexander Robertson",
                "Sharon Goldwater"
            ],
            "abstract": "We highlight several issues in the evaluation of historical text\nnormalization systems that make it hard to tell how well these systems would\nactually work in practice---i.e., for new datasets or languages; in comparison\nto more na\\\"ive systems; or as a preprocessing step for downstream NLP tools.\nWe illustrate these issues and exemplify our proposed evaluation practices by\ncomparing two neural models against a na\\\"ive baseline system. We show that the\nneural models generalize well to unseen words in tests on five languages;\nnevertheless, they provide no clear benefit over the na\\\"ive baseline for\ndownstream POS tagging of an English historical collection. We conclude that\nfuture work should include more rigorous evaluation, including both intrinsic\nand extrinsic measures where possible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02559v1",
            "date": "2018-04-07 13:20:24+00:00",
            "title": "Guiding Neural Machine Translation with Retrieved Translation Pieces",
            "authors": [
                "Jingyi Zhang",
                "Masao Utiyama",
                "Eiichro Sumita",
                "Graham Neubig",
                "Satoshi Nakamura"
            ],
            "abstract": "One of the difficulties of neural machine translation (NMT) is the recall and\nappropriate translation of low-frequency words or phrases. In this paper, we\npropose a simple, fast, and effective method for recalling previously seen\ntranslation examples and incorporating them into the NMT decoding process.\nSpecifically, for an input sentence, we use a search engine to retrieve\nsentence pairs whose source sides are similar with the input sentence, and then\ncollect $n$-grams that are both in the retrieved target sentences and aligned\nwith words that match in the source sentences, which we call \"translation\npieces\". We compute pseudo-probabilities for each retrieved sentence based on\nsimilarities between the input sentence and the retrieved source sentences, and\nuse these to weight the retrieved translation pieces. Finally, an existing NMT\nmodel is used to translate the input sentence, with an additional bonus given\nto outputs that contain the collected translation pieces. We show our method\nimproves NMT translation results up to 6 BLEU points on three narrow domain\ntranslation tasks where repetitiveness of the target sentences is particularly\nsalient. It also causes little increase in the translation time, and compares\nfavorably to another alternative retrieval-based method with respect to\naccuracy, speed, and simplicity of implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03243v3",
            "date": "2018-04-09 21:23:26+00:00",
            "title": "A GPU-based WFST Decoder with Exact Lattice Generation",
            "authors": [
                "Zhehuai Chen",
                "Justin Luitjens",
                "Hainan Xu",
                "Yiming Wang",
                "Daniel Povey",
                "Sanjeev Khudanpur"
            ],
            "abstract": "We describe initial work on an extension of the Kaldi toolkit that supports\nweighted finite-state transducer (WFST) decoding on Graphics Processing Units\n(GPUs). We implement token recombination as an atomic GPU operation in order to\nfully parallelize the Viterbi beam search, and propose a dynamic load balancing\nstrategy for more efficient token passing scheduling among GPU threads. We also\nredesign the exact lattice generation and lattice pruning algorithms for better\nutilization of the GPUs. Experiments on the Switchboard corpus show that the\nproposed method achieves identical 1-best results and lattice quality in\nrecognition and confidence measure tasks, while running 3 to 15 times faster\nthan the single process Kaldi decoder. The above results are reported on\ndifferent GPU architectures. Additionally we obtain a 46-fold speedup with\nsequence parallelism and multi-process service (MPS) in GPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03257v2",
            "date": "2018-04-09 22:10:57+00:00",
            "title": "Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings",
            "authors": [
                "Haw-Shiuan Chang",
                "Amol Agrawal",
                "Ananya Ganesh",
                "Anirudha Desai",
                "Vinayak Mathur",
                "Alfred Hough",
                "Andrew McCallum"
            ],
            "abstract": "Word sense induction (WSI), which addresses polysemy by unsupervised\ndiscovery of multiple word senses, resolves ambiguities for downstream NLP\ntasks and also makes word representations more interpretable. This paper\nproposes an accurate and efficient graph-based method for WSI that builds a\nglobal non-negative vector embedding basis (which are interpretable like\ntopics) and clusters the basis indexes in the ego network of each polysemous\nword. By adopting distributional inclusion vector embeddings as our basis\nformation model, we avoid the expensive step of nearest neighbor search that\nplagues other graph-based methods without sacrificing the quality of sense\nclusters. Experiments on three datasets show that our proposed method produces\nsimilar or better sense clusters and embeddings compared with previous\nstate-of-the-art methods while being significantly more efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03433v1",
            "date": "2018-04-10 10:22:19+00:00",
            "title": "Who framed Roger Reindeer? De-censorship of Facebook posts by snippet classification",
            "authors": [
                "Fabio Del Vigna",
                "Marinella Petrocchi",
                "Alessandro Tommasi",
                "Cesare Zavattari",
                "Maurizio Tesconi"
            ],
            "abstract": "This paper considers online news censorship and it concentrates on censorship\nof identities. Obfuscating identities may occur for disparate reasons, from\nmilitary to judiciary ones. In the majority of cases, this happens to protect\nindividuals from being identified and persecuted by hostile people. However,\nbeing the collaborative web characterised by a redundancy of information, it is\nnot unusual that the same fact is reported by multiple sources, which may not\napply the same restriction policies in terms of censorship. Also, the proven\naptitude of social network users to disclose personal information leads to the\nphenomenon that comments to news can reveal the data withheld in the news\nitself. This gives us a mean to figure out who the subject of the censored news\nis. We propose an adaptation of a text analysis approach to unveil censored\nidentities. The approach is tested on a synthesised scenario, which however\nresembles a real use case. Leveraging a text analysis based on a context\nclassifier trained over snippets from posts and comments of Facebook pages, we\nachieve promising results. Despite the quite constrained settings in which we\noperate -- such as considering only snippets of very short length -- our system\nsuccessfully detects the censored name, choosing among 10 different candidate\nnames, in more than 50\\% of the investigated cases. This outperforms the\nresults of two reference baselines. The findings reported in this paper, other\nthan being supported by a thorough experimental methodology and interesting on\ntheir own, also pave the way for further investigation on the insidious issues\nof censorship on the web.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03673v1",
            "date": "2018-04-10 18:10:33+00:00",
            "title": "Deep Learning for Digital Text Analytics: Sentiment Analysis",
            "authors": [
                "Reshma U",
                "Barathi Ganesh H B",
                "Mandar Kale",
                "Prachi Mankame",
                "Gouri Kulkarni"
            ],
            "abstract": "In today's scenario, imagining a world without negativity is something very\nunrealistic, as bad NEWS spreads more virally than good ones. Though it seems\nimpractical in real life, this could be implemented by building a system using\nMachine Learning and Natural Language Processing techniques in identifying the\nnews datum with negative shade and filter them by taking only the news with\npositive shade (good news) to the end user. In this work, around two lakhs\ndatum have been trained and tested using a combination of rule-based and data\ndriven approaches. VADER along with a filtration method has been used as an\nannotating tool followed by statistical Machine Learning approach that have\nused Document Term Matrix (representation) and Support Vector Machine\n(classification). Deep Learning algorithms then came into picture to make this\nsystem reliable (Doc2Vec) which finally ended up with Convolutional Neural\nNetwork(CNN) that yielded better results than the other experimented modules.\nIt showed up a training accuracy of 96%, while a test accuracy of (internal and\nexternal news datum) above 85% was obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03923v1",
            "date": "2018-04-11 11:07:16+00:00",
            "title": "Generating Multilingual Parallel Corpus Using Subtitles",
            "authors": [
                "Farshad Jafari"
            ],
            "abstract": "Neural Machine Translation with its significant results, still has a great\nproblem: lack or absence of parallel corpus for many languages. This article\nsuggests a method for generating considerable amount of parallel corpus for any\nlanguage pairs, extracted from open source materials existing on the Internet.\nParallel corpus contents will be derived from video subtitles. It needs a set\nof video titles, with some attributes like release date, rating, duration and\netc. Process of finding and downloading subtitle pairs for desired language\npairs is automated by using a crawler. Finally sentence pairs will be extracted\nfrom synchronous dialogues in subtitles. The main problem of this method is\nunsynchronized subtitle pairs. Therefore subtitles will be verified before\ndownloading. If two subtitle were not synchronized, then another subtitle of\nthat video will be processed till it finds the matching subtitle. Using this\napproach gives ability to make context based parallel corpus through filtering\nvideos by genre. Context based corpus can be used in complex translators which\ndecode sentences by different networks after determining contents subject.\nLanguages have many differences in their formal and informal styles, including\nwords and syntax. Other advantage of this method is to make corpus of informal\nstyle of languages. Because most of movies dialogues are parts of a\nconversation. So they had informal style. This feature of generated corpus can\nbe used in real-time translators to have more accurate conversation\ntranslations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04003v1",
            "date": "2018-04-10 16:28:33+00:00",
            "title": "Sentiment Transfer using Seq2Seq Adversarial Autoencoders",
            "authors": [
                "Ayush Singh",
                "Ritu Palod"
            ],
            "abstract": "Expressing in language is subjective. Everyone has a different style of\nreading and writing, apparently it all boil downs to the way their mind\nunderstands things (in a specific format). Language style transfer is a way to\npreserve the meaning of a text and change the way it is expressed. Progress in\nlanguage style transfer is lagged behind other domains, such as computer\nvision, mainly because of the lack of parallel data, use cases, and reliable\nevaluation metrics. In response to the challenge of lacking parallel data, we\nexplore learning style transfer from non-parallel data. We propose a model\ncombining seq2seq, autoencoders, and adversarial loss to achieve this goal. The\nkey idea behind the proposed models is to learn separate content\nrepresentations and style representations using adversarial networks.\nConsidering the problem of evaluating style transfer tasks, we frame the\nproblem as sentiment transfer and evaluation using a sentiment classifier to\ncalculate how many sentiments was the model able to transfer. We report our\nresults on several kinds of models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04083v3",
            "date": "2018-04-11 16:43:13+00:00",
            "title": "Multi-Task Learning for Argumentation Mining in Low-Resource Settings",
            "authors": [
                "Claudia Schulz",
                "Steffen Eger",
                "Johannes Daxenberger",
                "Tobias Kahse",
                "Iryna Gurevych"
            ],
            "abstract": "We investigate whether and where multi-task learning (MTL) can improve\nperformance on NLP problems related to argumentation mining (AM), in particular\nargument component identification. Our results show that MTL performs\nparticularly well (and better than single-task learning) when little training\ndata is available for the main task, a common scenario in AM. Our findings\nchallenge previous assumptions that conceptualizations across AM datasets are\ndivergent and that MTL is difficult for semantic or higher-level tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04093v1",
            "date": "2018-04-11 16:56:04+00:00",
            "title": "SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation",
            "authors": [
                "Ye Zhang",
                "Nan Ding",
                "Radu Soricut"
            ],
            "abstract": "Supervised training of abstractive language generation models results in\nlearning conditional probabilities over language sequences based on the\nsupervised training signal. When the training signal contains a variety of\nwriting styles, such models may end up learning an 'average' style that is\ndirectly influenced by the training data make-up and cannot be controlled by\nthe needs of an application. We describe a family of model architectures\ncapable of capturing both generic language characteristics via shared model\nparameters, as well as particular style characteristics via private model\nparameters. Such models are able to generate language according to a specific\nlearned style, while still taking advantage of their power to model generic\nlanguage phenomena. Furthermore, we describe an extension that uses a mixture\nof output distributions from all learned styles to perform on-the fly style\nadaptation based on the textual input alone. Experimentally, we find that the\nproposed models consistently outperform models that encapsulate single-style or\naverage-style language generation capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04211v1",
            "date": "2018-04-11 20:35:36+00:00",
            "title": "Evaluating Word Embedding Hyper-Parameters for Similarity and Analogy Tasks",
            "authors": [
                "Maryam Fanaeepour",
                "Adam Makarucha",
                "Jey Han Lau"
            ],
            "abstract": "The versatility of word embeddings for various applications is attracting\nresearchers from various fields. However, the impact of hyper-parameters when\ntraining embedding model is often poorly understood. How much do\nhyper-parameters such as vector dimensions and corpus size affect the quality\nof embeddings, and how do these results translate to downstream applications?\nUsing standard embedding evaluation metrics and datasets, we conduct a study to\nempirically measure the impact of these hyper-parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04242v3",
            "date": "2018-04-11 22:04:07+00:00",
            "title": "English Out-of-Vocabulary Lexical Evaluation Task",
            "authors": [
                "Han Wang",
                "Ye Wang",
                "Xinxiang Zhang",
                "Mi Lu",
                "Yoonsuck Choe",
                "Jingjing Cao"
            ],
            "abstract": "Unlike previous unknown nouns tagging task, this is the first attempt to\nfocus on out-of-vocabulary (OOV) lexical evaluation tasks that do not require\nany prior knowledge. The OOV words are words that only appear in test samples.\nThe goal of tasks is to provide solutions for OOV lexical classification and\nprediction. The tasks require annotators to conclude the attributes of the OOV\nwords based on their related contexts. Then, we utilize unsupervised word\nembedding methods such as Word2Vec and Word2GM to perform the baseline\nexperiments on the categorical classification task and OOV words attribute\nprediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04589v1",
            "date": "2018-03-19 18:49:16+00:00",
            "title": "A Survey on Neural Network-Based Summarization Methods",
            "authors": [
                "Yue Dong"
            ],
            "abstract": "Automatic text summarization, the automated process of shortening a text\nwhile reserving the main ideas of the document(s), is a critical research area\nin natural language processing. The aim of this literature review is to survey\nthe recent work on neural-based models in automatic text summarization. We\nexamine in detail ten state-of-the-art neural-based summarizers: five\nabstractive models and five extractive models. In addition, we discuss the\nrelated techniques that can be applied to the summarization tasks and present\npromising paths for future research in neural-based summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04838v1",
            "date": "2018-04-13 08:40:57+00:00",
            "title": "An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systems",
            "authors": [
                "Duygu Altinok"
            ],
            "abstract": "Keeping the dialogue state in dialogue systems is a notoriously difficult\ntask. We introduce an ontology-based dialogue manage(OntoDM), a dialogue\nmanager that keeps the state of the conversation, provides a basis for anaphora\nresolution and drives the conversation via domain ontologies. The banking and\nfinance area promises great potential for disambiguating the context via a rich\nset of products and specificity of proper nouns, named entities and verbs. We\nused ontologies both as a knowledge base and a basis for the dialogue manager;\nthe knowledge base component and dialogue manager components coalesce in a\nsense. Domain knowledge is used to track Entities of Interest, i.e. nodes\n(classes) of the ontology which happen to be products and services. In this way\nwe also introduced conversation memory and attention in a sense. We finely\nblended linguistic methods, domain-driven keyword ranking and domain ontologies\nto create ways of domain-driven conversation. Proposed framework is used in our\nin-house German language banking and finance chatbots. General challenges of\nGerman language processing and finance-banking domain chatbot language models\nand lexicons are also introduced. This work is still in progress, hence no\nsuccess metrics have been introduced yet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05017v1",
            "date": "2018-04-13 16:36:44+00:00",
            "title": "Incorporating Dictionaries into Deep Neural Networks for the Chinese Clinical Named Entity Recognition",
            "authors": [
                "Qi Wang",
                "Yuhang Xia",
                "Yangming Zhou",
                "Tong Ruan",
                "Daqi Gao",
                "Ping He"
            ],
            "abstract": "Clinical Named Entity Recognition (CNER) aims to identify and classify\nclinical terms such as diseases, symptoms, treatments, exams, and body parts in\nelectronic health records, which is a fundamental and crucial task for clinical\nand translational research. In recent years, deep neural networks have achieved\nsignificant success in named entity recognition and many other Natural Language\nProcessing (NLP) tasks. Most of these algorithms are trained end to end, and\ncan automatically learn features from large scale labeled datasets. However,\nthese data-driven methods typically lack the capability of processing rare or\nunseen entities. Previous statistical methods and feature engineering practice\nhave demonstrated that human knowledge can provide valuable information for\nhandling rare and unseen cases. In this paper, we address the problem by\nincorporating dictionaries into deep neural networks for the Chinese CNER task.\nTwo different architectures that extend the Bi-directional Long Short-Term\nMemory (Bi-LSTM) neural network and five different feature representation\nschemes are proposed to handle the task. Computational results on the CCKS-2017\nTask 2 benchmark dataset show that the proposed method achieves the highly\ncompetitive performance compared with the state-of-the-art deep learning\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05038v1",
            "date": "2018-04-13 17:10:12+00:00",
            "title": "Pieces of Eight: 8-bit Neural Machine Translation",
            "authors": [
                "Jerry Quinn",
                "Miguel Ballesteros"
            ],
            "abstract": "Neural machine translation has achieved levels of fluency and adequacy that\nwould have been surprising a short time ago. Output quality is extremely\nrelevant for industry purposes, however it is equally important to produce\nresults in the shortest time possible, mainly for latency-sensitive\napplications and to control cloud hosting costs. In this paper we show the\neffectiveness of translating with 8-bit quantization for models that have been\ntrained using 32-bit floating point values. Results show that 8-bit translation\nmakes a non-negligible impact in terms of speed with no degradation in accuracy\nand adequacy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05095v1",
            "date": "2018-04-13 19:38:52+00:00",
            "title": "Automatic Language Identification System for Hindi and Magahi",
            "authors": [
                "Priya Rani",
                "Atul Kr. Ojha",
                "Girish Nath Jha"
            ],
            "abstract": "Language identification has become a prerequisite for all kinds of automated\ntext processing systems. In this paper, we present a rule-based language\nidentifier tool for two closely related Indo-Aryan languages: Hindi and Magahi.\nThis system has currently achieved an accuracy of approx 86.34%. We hope to\nimprove this in the future. Automatic identification of languages will be\nsignificant in the accuracy of output of Web Crawlers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05166v1",
            "date": "2018-04-14 04:52:23+00:00",
            "title": "Developing Far-Field Speaker System Via Teacher-Student Learning",
            "authors": [
                "Jinyu Li",
                "Rui Zhao",
                "Zhuo Chen",
                "Changliang Liu",
                "Xiong Xiao",
                "Guoli Ye",
                "Yifan Gong"
            ],
            "abstract": "In this study, we develop the keyword spotting (KWS) and acoustic model (AM)\ncomponents in a far-field speaker system. Specifically, we use teacher-student\n(T/S) learning to adapt a close-talk well-trained production AM to far-field by\nusing parallel close-talk and simulated far-field data. We also use T/S\nlearning to compress a large-size KWS model into a small-size one to fit the\ndevice computational cost. Without the need of transcription, T/S learning well\nutilizes untranscribed data to boost the model performance in both the AM\nadaptation and KWS model compression. We further optimize the models with\nsequence discriminative training and live data to reach the best performance of\nsystems. The adapted AM improved from the baseline by 72.60% and 57.16%\nrelative word error rate reduction on play-back and live test data,\nrespectively. The final KWS model size was reduced by 27 times from a\nlarge-size KWS model without losing accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05253v1",
            "date": "2018-04-14 17:39:45+00:00",
            "title": "\"With 1 follower I must be AWESOME :P\". Exploring the role of irony markers in irony recognition",
            "authors": [
                "Debanjan Ghosh",
                "Smaranda Muresan"
            ],
            "abstract": "Conversations in social media often contain the use of irony or sarcasm, when\nthe users say the opposite of what they really mean. Irony markers are the\nmeta-communicative clues that inform the reader that an utterance is ironic. We\npropose a thorough analysis of theoretically grounded irony markers in two\nsocial media platforms: $Twitter$ and $Reddit$. Classification and frequency\nanalysis show that for $Twitter$, typographic markers such as emoticons and\nemojis are the most discriminative markers to recognize ironic utterances,\nwhile for $Reddit$ the morphological markers (e.g., interjections, tag\nquestions) are the most discriminative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05262v1",
            "date": "2018-04-14 18:30:01+00:00",
            "title": "Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings",
            "authors": [
                "Joshua Coates",
                "Danushka Bollegala"
            ],
            "abstract": "Creating accurate meta-embeddings from pre-trained source embeddings has\nreceived attention lately. Methods based on global and locally-linear\ntransformation and concatenation have shown to produce accurate\nmeta-embeddings. In this paper, we show that the arithmetic mean of two\ndistinct word embedding sets yields a performant meta-embedding that is\ncomparable or better than more complex meta-embedding learning methods. The\nresult seems counter-intuitive given that vector spaces in different source\nembeddings are not comparable and cannot be simply averaged. We give insight\ninto why averaging can still produce accurate meta-embedding despite the\nincomparability of the source vector spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05276v1",
            "date": "2018-04-14 20:56:58+00:00",
            "title": "Predicting Cyber Events by Leveraging Hacker Sentiment",
            "authors": [
                "Ashok Deb",
                "Kristina Lerman",
                "Emilio Ferrara"
            ],
            "abstract": "Recent high-profile cyber attacks exemplify why organizations need better\ncyber defenses. Cyber threats are hard to accurately predict because attackers\nusually try to mask their traces. However, they often discuss exploits and\ntechniques on hacking forums. The community behavior of the hackers may provide\ninsights into groups' collective malicious activity. We propose a novel\napproach to predict cyber events using sentiment analysis. We test our approach\nusing cyber attack data from 2 major business organizations. We consider 3\ntypes of events: malicious software installation, malicious destination visits,\nand malicious emails that surpassed the target organizations' defenses. We\nconstruct predictive signals by applying sentiment analysis on hacker forum\nposts to better understand hacker behavior. We analyze over 400K posts\ngenerated between January 2016 and January 2018 on over 100 hacking forums both\non surface and Dark Web. We find that some forums have significantly more\npredictive power than others. Sentiment-based models that leverage specific\nforums can outperform state-of-the-art deep learning and time-series models on\nforecasting cyber attacks weeks ahead of the events.",
            "journal ref": "Information 2018, 9(11), 280",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05294v1",
            "date": "2018-04-15 02:21:28+00:00",
            "title": "The EcoLexicon Semantic Sketch Grammar: from Knowledge Patterns to Word Sketches",
            "authors": [
                "P. Le\u00f3n-Ara\u00faz",
                "A. San Mart\u00edn"
            ],
            "abstract": "Many projects have applied knowledge patterns (KPs) to the retrieval of\nspecialized information. Yet terminologists still rely on manual analysis of\nconcordance lines to extract semantic information, since there are no\nuser-friendly publicly available applications enabling them to find knowledge\nrich contexts (KRCs). To fill this void, we have created the KP-based\nEcoLexicon Semantic SketchGrammar (ESSG) in the well-known corpus query system\nSketch Engine. For the first time, the ESSG is now publicly available inSketch\nEngine to query the EcoLexicon English Corpus. Additionally, reusing the ESSG\nin any English corpus uploaded by the user enables Sketch Engine to extract\nKRCs codifying generic-specific, part-whole, location, cause and function\nrelations, because most of the KPs are domain-independent. The information is\ndisplayed in the form of summary lists (word sketches) containing the pairs of\nterms linked by a given semantic relation. This paper describes the process of\nbuilding a KP-based sketch grammar with special focus on the last stage,\nnamely, the evaluation with refinement purposes. We conducted an initial\nshallow precision and recall evaluation of the 64 English sketch grammar rules\ncreated so far for hyponymy, meronymy and causality. Precision was measured\nbased on a random sample of concordances extracted from each word sketch type.\nRecall was assessed based on a random sample of concordances where known term\npairs are found. The results are necessary for the improvement and refinement\nof the ESSG. The noise of false positives helped to further specify the rules,\nwhereas the silence of false negatives allows us to find useful new patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05388v2",
            "date": "2018-04-15 17:38:16+00:00",
            "title": "Introducing two Vietnamese Datasets for Evaluating Semantic Models of (Dis-)Similarity and Relatedness",
            "authors": [
                "Kim Anh Nguyen",
                "Sabine Schulte im Walde",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present two novel datasets for the low-resource language Vietnamese to\nassess models of semantic similarity: ViCon comprises pairs of synonyms and\nantonyms across word classes, thus offering data to distinguish between\nsimilarity and dissimilarity. ViSim-400 provides degrees of similarity across\nfive semantic relations, as rated by human judges. The two datasets are\nverified through standard co-occurrence and neural network models, showing\nresults comparable to the respective English datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05392v1",
            "date": "2018-04-15 17:47:26+00:00",
            "title": "Higher-order Coreference Resolution with Coarse-to-fine Inference",
            "authors": [
                "Kenton Lee",
                "Luheng He",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce a fully differentiable approximation to higher-order inference\nfor coreference resolution. Our approach uses the antecedent distribution from\na span-ranking architecture as an attention mechanism to iteratively refine\nspan representations. This enables the model to softly consider multiple hops\nin the predicted clusters. To alleviate the computational cost of this\niterative process, we introduce a coarse-to-fine approach that incorporates a\nless accurate but more efficient bilinear factor, enabling more aggressive\npruning without hurting accuracy. Compared to the existing state-of-the-art\nspan-ranking approach, our model significantly improves accuracy on the English\nOntoNotes benchmark, while being far more computationally efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05398v1",
            "date": "2018-04-15 18:00:53+00:00",
            "title": "Context and Humor: Understanding Amul advertisements of India",
            "authors": [
                "Radhika Mamidi"
            ],
            "abstract": "Contextual knowledge is the most important element in understanding language.\nBy contextual knowledge we mean both general knowledge and discourse knowledge\ni.e. knowledge of the situational context, background knowledge and the\nco-textual context [10]. In this paper, we will discuss the importance of\ncontextual knowledge in understanding the humor present in the cartoon based\nAmul advertisements in India.In the process, we will analyze these\nadvertisements and also see if humor is an effective tool for advertising and\nthereby, for marketing.These bilingual advertisements also expect the audience\nto have the appropriate linguistic knowledge which includes knowledge of\nEnglish and Hindi vocabulary, morphology and syntax. Different techniques like\npunning, portmanteaus and parodies of popular proverbs, expressions, acronyms,\nfamous dialogues, songs etc are employed to convey the message in a humorous\nway. The present study will concentrate on these linguistic cues and the\nrequired context for understanding wit and humor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05408v1",
            "date": "2018-04-15 19:02:13+00:00",
            "title": "GU IRLAB at SemEval-2018 Task 7: Tree-LSTMs for Scientific Relation Classification",
            "authors": [
                "Sean MacAvaney",
                "Luca Soldaini",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "SemEval 2018 Task 7 focuses on relation ex- traction and classification in\nscientific literature. In this work, we present our tree-based LSTM network for\nthis shared task. Our approach placed 9th (of 28) for subtask 1.1 (relation\nclassification), and 5th (of 20) for subtask 1.2 (relation classification with\nnoisy entities). We also provide an ablation study of features included as\ninput to the network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05416v1",
            "date": "2018-04-15 19:45:07+00:00",
            "title": "Are Automatic Methods for Cognate Detection Good Enough for Phylogenetic Reconstruction in Historical Linguistics?",
            "authors": [
                "Taraka Rama",
                "Johann-Mattis List",
                "Johannes Wahle",
                "Gerhard J\u00e4ger"
            ],
            "abstract": "We evaluate the performance of state-of-the-art algorithms for automatic\ncognate detection by comparing how useful automatically inferred cognates are\nfor the task of phylogenetic inference compared to classical manually annotated\ncognate sets. Our findings suggest that phylogenies inferred from automated\ncognate sets come close to phylogenies inferred from expert-annotated ones,\nalthough on average, the latter are still superior. We conclude that future\nwork on phylogenetic reconstruction can profit much from automatic cognate\ndetection. Especially where scholars are merely interested in exploring the\nbigger picture of a language family's phylogeny, algorithms for automatic\ncognate detection are a useful complement for current research on language\nphylogenies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05417v2",
            "date": "2018-04-15 19:55:13+00:00",
            "title": "Pragmatically Informative Image Captioning with Character-Level Inference",
            "authors": [
                "Reuben Cohn-Gordon",
                "Noah Goodman",
                "Christopher Potts"
            ],
            "abstract": "We combine a neural image captioner with a Rational Speech Acts (RSA) model\nto make a system that is pragmatically informative: its objective is to produce\ncaptions that are not merely true but also distinguish their inputs from\nsimilar images. Previous attempts to combine RSA with neural image captioning\nrequire an inference which normalizes over the entire set of possible\nutterances. This poses a serious problem of efficiency, previously solved by\nsampling a small subset of possible utterances. We instead solve this problem\nby implementing a version of RSA which operates at the level of characters\n(\"a\",\"b\",\"c\"...) during the unrolling of the caption. We find that the\nutterance-level effect of referential captions can be obtained with only\ncharacter-level decisions. Finally, we introduce an automatic method for\ntesting the performance of pragmatic speaker models, and show that our model\noutperforms a non-pragmatic baseline as well as a word-level RSA captioner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05499v1",
            "date": "2018-04-16 04:03:52+00:00",
            "title": "Community Member Retrieval on Social Media using Textual Information",
            "authors": [
                "Aaron Jaech",
                "Shobhit Hathi",
                "Mari Ostendorf"
            ],
            "abstract": "This paper addresses the problem of community membership detection using only\ntext features in a scenario where a small number of positive labeled examples\ndefines the community. The solution introduces an unsupervised proxy task for\nlearning user embeddings: user re-identification. Experiments with 16 different\ncommunities show that the resulting embeddings are more effective for community\nmembership identification than common unsupervised representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05630v1",
            "date": "2018-04-16 12:08:13+00:00",
            "title": "Arabic Named Entity Recognition using Word Representations",
            "authors": [
                "Ismail El Bazi",
                "Nabil Laachfoubi"
            ],
            "abstract": "Recent work has shown the effectiveness of the word representations features\nin significantly improving supervised NER for the English language. In this\nstudy we investigate whether word representations can also boost supervised NER\nin Arabic. We use word representations as additional features in a Conditional\nRandom Field (CRF) model and we systematically compare three popular neural\nword embedding algorithms (SKIP-gram, CBOW and GloVe) and six different\napproaches for integrating word representations into NER system. Experimental\nresults show that Brown Clustering achieves the best performance among the six\napproaches. Concerning the word embedding features, the clustering embedding\nfeatures outperform other embedding features and the distributional prototypes\nproduce the second best result. Moreover, the combination of Brown clusters and\nword embedding features provides additional improvement of nearly 10% in\nF1-score over the baseline.",
            "journal ref": "International Journal of Computer Science and Information Security\n  (IJCSIS), Vol. 14, No. 8, August 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05685v2",
            "date": "2018-04-16 13:55:20+00:00",
            "title": "A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents",
            "authors": [
                "Arman Cohan",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Trung Bui",
                "Seokhwan Kim",
                "Walter Chang",
                "Nazli Goharian"
            ],
            "abstract": "Neural abstractive summarization models have led to promising results in\nsummarizing relatively short documents. We propose the first model for\nabstractive summarization of single, longer-form documents (e.g., research\npapers). Our approach consists of a new hierarchical encoder that models the\ndiscourse structure of a document, and an attentive discourse-aware decoder to\ngenerate the summary. Empirical results on two large-scale datasets of\nscientific papers show that our model significantly outperforms\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05686v1",
            "date": "2018-04-16 13:59:08+00:00",
            "title": "Organization and Independence or Interdependence? Study of the Neurophysiological Dynamics of Syntactic and Semantic Processing",
            "authors": [
                "Sabine Ploux",
                "Viviane D\u00e9prez"
            ],
            "abstract": "In this article we present a multivariate model for determining the different\nsyntactic, semantic, and form (surface-structure) processes underlying the\ncomprehension of simple phrases. This model is applied to EEG signals recorded\nduring a reading task. The results show a hierarchical precedence of the\nneurolinguistic processes : form, then syntactic and lastly semantic processes.\nWe also found (a) that verbs are at the heart of phrase syntax processing, (b)\nan interaction between syntactic movement within the phrase, and semantic\nprocesses derived from a person-centered reference frame. Eigenvectors of the\nmultivariate model provide electrode-times profiles that separate the\ndistinctive linguistic processes and/or highlight their interaction. The\naccordance of these findings with different linguistic theories are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05689v1",
            "date": "2018-04-16 14:03:45+00:00",
            "title": "The Relevance of Text and Speech Features in Automatic Non-native English Accent Identification",
            "authors": [
                "Sowmya Vajjala",
                "Ziwei Zhou"
            ],
            "abstract": "This paper describes our experiments with automatically identifying native\naccents from speech samples of non-native English speakers using low level\naudio features, and n-gram features from manual transcriptions. Using a\npublicly available non-native speech corpus and simple audio feature\nrepresentations that do not perform word/phoneme recognition, we show that it\nis possible to achieve close to 90% classification accuracy for this task.\nWhile character n-grams perform similar to speech features, we show that speech\nfeatures are not affected by prompt variation, whereas ngrams are. Since the\napproach followed can be easily adapted to any language provided we have enough\ntraining data, we believe these results will provide useful insights for the\ndevelopment of accent recognition systems and for the study of accents in the\ncontext of language learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05734v1",
            "date": "2018-04-16 15:25:53+00:00",
            "title": "Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning",
            "authors": [
                "Chenhua Chen",
                "Yue Zhang"
            ],
            "abstract": "Self-training is a useful strategy for semi-supervised learning, leveraging\nraw texts for enhancing model performances. Traditional self-training methods\ndepend on heuristics such as model confidence for instance selection, the\nmanual adjustment of which can be expensive. To address these challenges, we\npropose a deep reinforcement learning method to learn the self-training\nstrategy automatically. Based on neural network representation of sentences,\nour model automatically learns an optimal policy for instance selection.\nExperimental results show that our approach outperforms the baseline solutions\nin terms of better tagging performances and stability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05825v3",
            "date": "2018-04-16 17:52:36+00:00",
            "title": "ClaiRE at SemEval-2018 Task 7 - Extended Version",
            "authors": [
                "Lena Hettinger",
                "Alexander Dallmann",
                "Albin Zehe",
                "Thomas Niebler",
                "Andreas Hotho"
            ],
            "abstract": "In this paper we describe our post-evaluation results for SemEval-2018 Task 7\non clas- sification of semantic relations in scientific literature for clean\n(subtask 1.1) and noisy data (subtask 1.2). This is an extended ver- sion of\nour workshop paper (Hettinger et al., 2018) including further technical details\n(Sec- tions 3.2 and 4.3) and changes made to the preprocessing step in the\npost-evaluation phase (Section 2.1). Due to these changes Classification of\nRelations using Embeddings (ClaiRE) achieved an improved F1 score of 75.11% for\nthe first subtask and 81.44% for the second.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05831v1",
            "date": "2018-04-13 16:57:59+00:00",
            "title": "Neologisms on Facebook",
            "authors": [
                "Nikita Muravyev",
                "Alexander Panchenko",
                "Sergei Obiedkov"
            ],
            "abstract": "In this paper, we present a study of neologisms and loan words frequently\noccurring in Facebook user posts. We have analyzed a dataset of several million\npublically available posts written during 2006-2013 by Russian-speaking\nFacebook users. From these, we have built a vocabulary of most frequent\nlemmatized words missing from the OpenCorpora dictionary the assumption being\nthat many such words have entered common use only recently. This assumption is\ncertainly not true for all the words extracted in this way; for that reason, we\nmanually filtered the automatically obtained list in order to exclude\nnon-Russian or incorrectly lemmatized words, as well as words recorded by other\ndictionaries or those occurring in texts from the Russian National Corpus. The\nresult is a list of 168 words that can potentially be considered neologisms. We\npresent an attempt at an etymological classification of these neologisms\n(unsurprisingly, most of them have recently been borrowed from English, but\nthere are also quite a few new words composed of previously borrowed stems) and\nidentify various derivational patterns. We also classify words into several\nlarge thematic areas, \"internet\", \"marketing\", and \"multimedia\" being among\nthose with the largest number of words. We believe that, together with the word\nbase collected in the process, they can serve as a starting point in further\nstudies of neologisms and lexical processes that lead to their acceptance into\nthe mainstream language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05868v3",
            "date": "2018-04-16 18:05:52+00:00",
            "title": "Universal Dependency Parsing for Hindi-English Code-switching",
            "authors": [
                "Irshad Ahmad Bhat",
                "Riyaz Ahmad Bhat",
                "Manish Shrivastava",
                "Dipti Misra Sharma"
            ],
            "abstract": "Code-switching is a phenomenon of mixing grammatical structures of two or\nmore languages under varied social constraints. The code-switching data differ\nso radically from the benchmark corpora used in NLP community that the\napplication of standard technologies to these data degrades their performance\nsharply. Unlike standard corpora, these data often need to go through\nadditional processes such as language identification, normalization and/or\nback-transliteration for their efficient processing. In this paper, we\ninvestigate these indispensable processes and other problems associated with\nsyntactic parsing of code-switching data and propose methods to mitigate their\neffects. In particular, we study dependency parsing of code-switching data of\nHindi and English multilingual speakers from Twitter. We present a treebank of\nHindi-English code-switching tweets under Universal Dependencies scheme and\npropose a neural stacking model for parsing that efficiently leverages\npart-of-speech tag and syntactic tree annotations in the code-switching\ntreebank and the preexisting Hindi and English treebanks. We also present\nnormalization and back-transliteration models with a decoding process tailored\nfor code-switching data. Results show that our neural stacking parser is 1.5%\nLAS points better than the augmented parsing model and our decoding process\nimproves results by 3.8% LAS points over the first-best normalization and/or\nback-transliteration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05918v1",
            "date": "2018-04-16 20:01:06+00:00",
            "title": "Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph",
            "authors": [
                "Zeyu Dai",
                "Ruihong Huang"
            ],
            "abstract": "We argue that semantic meanings of a sentence or clause can not be\ninterpreted independently from the rest of a paragraph, or independently from\nall discourse relations and the overall paragraph-level discourse structure.\nWith the goal of improving implicit discourse relation classification, we\nintroduce a paragraph-level neural networks that model inter-dependencies\nbetween discourse units as well as discourse relation continuity and patterns,\nand predict a sequence of discourse relations in a paragraph. Experimental\nresults show that our model outperforms the previous state-of-the-art systems\non the benchmark corpus of PDTB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05940v1",
            "date": "2018-04-16 21:06:14+00:00",
            "title": "Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz",
                "Shubha Guha",
                "Kenneth Heafield"
            ],
            "abstract": "Previously, neural methods in grammatical error correction (GEC) did not\nreach state-of-the-art results compared to phrase-based statistical machine\ntranslation (SMT) baselines. We demonstrate parallels between neural GEC and\nlow-resource neural MT and successfully adapt several methods from low-resource\nMT to neural GEC. We further establish guidelines for trustable results in\nneural GEC and propose a set of model-independent methods for neural GEC that\ncan be easily applied in most GEC settings. Proposed methods include adding\nsource-side noise, domain-adaptation techniques, a GEC-specific\ntraining-objective, transfer learning with monolingual data, and ensembling of\nindependently trained GEC models and language models. The combined effects of\nthese methods result in better than state-of-the-art neural GEC models that\noutperform previously best neural GEC systems by more than 10% M$^2$ on the\nCoNLL-2014 benchmark and 5.9% on the JFLEG test set. Non-neural\nstate-of-the-art systems are outperformed by more than 2% on the CoNLL-2014\nbenchmark and by 4% on JFLEG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05945v1",
            "date": "2018-04-16 21:13:31+00:00",
            "title": "Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation",
            "authors": [
                "Roman Grundkiewicz",
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "We combine two of the most popular approaches to automated Grammatical Error\nCorrection (GEC): GEC based on Statistical Machine Translation (SMT) and GEC\nbased on Neural Machine Translation (NMT). The hybrid system achieves new\nstate-of-the-art results on the CoNLL-2014 and JFLEG benchmarks. This GEC\nsystem preserves the accuracy of SMT output and, at the same time, generates\nmore fluent sentences as it typical for NMT. Our analysis shows that the\ncreated systems are closer to reaching human-level performance than any other\nGEC system reported so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05972v1",
            "date": "2018-04-16 23:12:07+00:00",
            "title": "A Deeper Look into Dependency-Based Word Embeddings",
            "authors": [
                "Sean MacAvaney",
                "Amir Zeldes"
            ],
            "abstract": "We investigate the effect of various dependency-based word embeddings on\ndistinguishing between functional and domain similarity, word similarity\nrankings, and two downstream tasks in English. Variations include word\nembeddings trained using context windows from Stanford and Universal\ndependencies at several levels of enhancement (ranging from unlabeled, to\nEnhanced++ dependencies). Results are compared to basic linear contexts and\nevaluated on several datasets. We found that embeddings trained with Universal\nand Stanford dependency contexts excel at different tasks, and that enhanced\ndependencies often improve performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05990v1",
            "date": "2018-04-17 00:14:32+00:00",
            "title": "Learning Joint Semantic Parsers from Disjoint Data",
            "authors": [
                "Hao Peng",
                "Sam Thomson",
                "Swabha Swayamdipta",
                "Noah A. Smith"
            ],
            "abstract": "We present a new approach to learning semantic parsers from multiple\ndatasets, even when the target semantic formalisms are drastically different,\nand the underlying corpora do not overlap. We handle such \"disjoint\" data by\ntreating annotations for unobserved formalisms as latent structured variables.\nBuilding on state-of-the-art baselines, we show improvements both in\nframe-semantic parsing and semantic dependency parsing by modeling them\njointly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06004v1",
            "date": "2018-04-17 01:22:41+00:00",
            "title": "Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses",
            "authors": [
                "Katherine A. Keith",
                "Su Lin Blodgett",
                "Brendan O'Connor"
            ],
            "abstract": "Dependency parsing research, which has made significant gains in recent\nyears, typically focuses on improving the accuracy of single-tree predictions.\nHowever, ambiguity is inherent to natural language syntax, and communicating\nsuch ambiguity is important for error analysis and better-informed downstream\napplications. In this work, we propose a transition sampling algorithm to\nsample from the full joint distribution of parse trees defined by a\ntransition-based parsing model, and demonstrate the use of the samples in\nprobabilistic dependency analysis. First, we define the new task of dependency\npath prediction, inferring syntactic substructures over part of a sentence, and\nprovide the first analysis of performance on this task. Second, we demonstrate\nthe usefulness of our Monte Carlo syntax marginal method for parser error\nanalysis and calibration. Finally, we use this method to propagate parse\nuncertainty to two downstream information extraction applications: identifying\npersons killed by police and semantic role assignment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06024v1",
            "date": "2018-04-17 03:10:51+00:00",
            "title": "Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages",
            "authors": [
                "Katharina Kann",
                "Manuel Mager",
                "Ivan Meza-Ruiz",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Morphological segmentation for polysynthetic languages is challenging,\nbecause a word may consist of many individual morphemes and training data can\nbe extremely scarce. Since neural sequence-to-sequence (seq2seq) models define\nthe state of the art for morphological segmentation in high-resource settings\nand for (mostly) European languages, we first show that they also obtain\ncompetitive performance for Mexican polysynthetic languages in minimal-resource\nsettings. We then propose two novel multi-task training approaches -one with,\none without need for external unlabeled resources-, and two corresponding data\naugmentation methods, improving over the neural baseline for all languages.\nFinally, we explore cross-lingual transfer as a third way to fortify our neural\nmodel and show that we can train one single multi-lingual model for related\nlanguages while maintaining comparable or even improved performance, thus\nreducing the amount of parameters by close to 75%. We provide our morphological\nsegmentation datasets for Mexicanero, Nahuatl, Wixarika and Yorem Nokki for\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06028v1",
            "date": "2018-04-17 03:26:28+00:00",
            "title": "ListOps: A Diagnostic Dataset for Latent Tree Learning",
            "authors": [
                "Nikita Nangia",
                "Samuel R. Bowman"
            ],
            "abstract": "Latent tree learning models learn to parse a sentence without syntactic\nsupervision, and use that parse to build the sentence representation. Existing\nwork on such models has shown that, while they perform well on tasks like\nsentence classification, they do not learn grammars that conform to any\nplausible semantic or syntactic formalism (Williams et al., 2018a). Studying\nthe parsing ability of such models in natural language can be challenging due\nto the inherent complexities of natural language, like having several valid\nparses for a single sentence. In this paper we introduce ListOps, a toy dataset\ncreated to study the parsing ability of latent tree models. ListOps sequences\nare in the style of prefix arithmetic. The dataset is designed to have a single\ncorrect parsing strategy that a system needs to learn to succeed at the task.\nWe show that the current leading latent tree models are unable to learn to\nparse and succeed at ListOps. These models achieve accuracies worse than purely\nsequential RNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06035v1",
            "date": "2018-04-17 03:41:55+00:00",
            "title": "Reinforced Co-Training",
            "authors": [
                "Jiawei Wu",
                "Lei Li",
                "William Yang Wang"
            ],
            "abstract": "Co-training is a popular semi-supervised learning framework to utilize a\nlarge amount of unlabeled data in addition to a small labeled set. Co-training\nmethods exploit predicted labels on the unlabeled data and select samples based\non prediction confidence to augment the training. However, the selection of\nsamples in existing co-training methods is based on a predetermined policy,\nwhich ignores the sampling bias between the unlabeled and the labeled subsets,\nand fails to explore the data space. In this paper, we propose a novel method,\nReinforced Co-Training, to select high-quality unlabeled samples to better\nco-train on. More specifically, our approach uses Q-learning to learn a data\nselection policy with a small labeled dataset, and then exploits this policy to\ntrain the co-training classifiers automatically. Experimental results on\nclickbait detection and generic text classification tasks demonstrate that our\nproposed method can obtain more accurate text classification results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06059v1",
            "date": "2018-04-17 06:12:36+00:00",
            "title": "Adversarial Example Generation with Syntactically Controlled Paraphrase Networks",
            "authors": [
                "Mohit Iyyer",
                "John Wieting",
                "Kevin Gimpel",
                "Luke Zettlemoyer"
            ],
            "abstract": "We propose syntactically controlled paraphrase networks (SCPNs) and use them\nto generate adversarial examples. Given a sentence and a target syntactic form\n(e.g., a constituency parse), SCPNs are trained to produce a paraphrase of the\nsentence with the desired syntax. We show it is possible to create training\ndata for this task by first doing backtranslation at a very large scale, and\nthen using a parser to label the syntactic transformations that naturally occur\nduring this process. Such data allows us to train a neural encoder-decoder\nmodel with extra inputs to specify the target syntax. A combination of\nautomated and human evaluations show that SCPNs generate paraphrases that\nfollow their target specifications without decreasing paraphrase quality when\ncompared to baseline (uncontrolled) paraphrase systems. Furthermore, they are\nmore capable of generating syntactically adversarial examples that both (1)\n\"fool\" pretrained models and (2) improve the robustness of these models to\nsyntactic variation when used to augment their training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06137v1",
            "date": "2018-04-17 09:50:01+00:00",
            "title": "SeerNet at SemEval-2018 Task 1: Domain Adaptation for Affect in Tweets",
            "authors": [
                "Venkatesh Duppada",
                "Royal Jain",
                "Sushant Hiray"
            ],
            "abstract": "The paper describes the best performing system for the SemEval-2018 Affect in\nTweets (English) sub-tasks. The system focuses on the ordinal classification\nand regression sub-tasks for valence and emotion. For ordinal classification\nvalence is classified into 7 different classes ranging from -3 to 3 whereas\nemotion is classified into 4 different classes 0 to 3 separately for each\nemotion namely anger, fear, joy and sadness. The regression sub-tasks estimate\nthe intensity of valence and each emotion. The system performs domain\nadaptation of 4 different models and creates an ensemble to give the final\nprediction. The proposed system achieved 1st position out of 75 teams which\nparticipated in the fore-mentioned sub-tasks. We outperform the baseline model\nby margins ranging from 49.2% to 76.4%, thus, pushing the state-of-the-art\nsignificantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06189v1",
            "date": "2018-04-17 12:16:25+00:00",
            "title": "Investigating Backtranslation in Neural Machine Translation",
            "authors": [
                "Alberto Poncelas",
                "Dimitar Shterionov",
                "Andy Way",
                "Gideon Maillette de Buy Wenniger",
                "Peyman Passban"
            ],
            "abstract": "A prerequisite for training corpus-based machine translation (MT) systems --\neither Statistical MT (SMT) or Neural MT (NMT) -- is the availability of\nhigh-quality parallel data. This is arguably more important today than ever\nbefore, as NMT has been shown in many studies to outperform SMT, but mostly\nwhen large parallel corpora are available; in cases where data is limited, SMT\ncan still outperform NMT.\n  Recently researchers have shown that back-translating monolingual data can be\nused to create synthetic parallel corpora, which in turn can be used in\ncombination with authentic parallel data to train a high-quality NMT system.\nGiven that large collections of new parallel text become available only quite\nrarely, backtranslation has become the norm when building state-of-the-art NMT\nsystems, especially in resource-poor scenarios.\n  However, we assert that there are many unknown factors regarding the actual\neffects of back-translated data on the translation capabilities of an NMT\nmodel. Accordingly, in this work we investigate how using back-translated data\nas a training corpus -- both as a separate standalone dataset as well as\ncombined with human-generated parallel data -- affects the performance of an\nNMT model. We use incrementally larger amounts of back-translated data to train\na range of NMT systems for German-to-English, and analyse the resulting\ntranslation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06323v2",
            "date": "2018-04-17 15:34:07+00:00",
            "title": "When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?",
            "authors": [
                "Ye Qi",
                "Devendra Singh Sachan",
                "Matthieu Felix",
                "Sarguna Janani Padmanabhan",
                "Graham Neubig"
            ],
            "abstract": "The performance of Neural Machine Translation (NMT) systems often suffers in\nlow-resource scenarios where sufficiently large-scale parallel corpora cannot\nbe obtained. Pre-trained word embeddings have proven to be invaluable for\nimproving performance in natural language analysis tasks, which often suffer\nfrom paucity of data. However, their utility for NMT has not been extensively\nexplored. In this work, we perform five sets of experiments that analyze when\nwe can expect pre-trained word embeddings to help in NMT tasks. We show that\nsuch embeddings can be surprisingly effective in some cases -- providing gains\nof up to 20 BLEU points in the most favorable setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06333v1",
            "date": "2018-04-17 15:54:25+00:00",
            "title": "Similarity between Learning Outcomes from Course Objectives using Semantic Analysis, Blooms taxonomy and Corpus statistics",
            "authors": [
                "Atish Pawar",
                "Vijay Mago"
            ],
            "abstract": "The course description provided by instructors is an essential piece of\ninformation as it defines what is expected from the instructor and what he/she\nis going to deliver during a particular course. One of the key components of a\ncourse description is the Learning Objectives section. The contents of this\nsection are used by program managers who are tasked to compare and match two\ndifferent courses during the development of Transfer Agreements between various\ninstitutions. This research introduces the development of semantic similarity\nalgorithms to calculate the similarity between two learning objectives of the\nsame domain. We present a novel methodology which deals with the semantic\nsimilarity by using a previously established algorithm and integrating it with\nthe domain corpus utilizing domain statistics. The disambiguated domain serves\nas a supervised learning data for the algorithm. We also introduce Bloom Index\nto calculate the similarity between action verbs in the Learning Objectives\nreferring to the Blooms taxonomy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06385v4",
            "date": "2018-04-17 17:30:02+00:00",
            "title": "Bootstrapping Generators from Noisy Data",
            "authors": [
                "Laura Perez-Beltrachini",
                "Mirella Lapata"
            ],
            "abstract": "A core step in statistical data-to-text generation concerns learning\ncorrespondences between structured data representations (e.g., facts in a\ndatabase) and associated texts. In this paper we aim to bootstrap generators\nfrom large scale datasets where the data (e.g., DBPedia facts) and related\ntexts (e.g., Wikipedia abstracts) are loosely aligned. We tackle this\nchallenging task by introducing a special-purpose content selection mechanism.\nWe use multi-instance learning to automatically discover correspondences\nbetween data and text pairs and show how these can be used to enhance the\ncontent signal while training an encoder-decoder architecture. Experimental\nresults demonstrate that models trained with content-specific objectives\nimprove upon a vanilla encoder-decoder which solely relies on soft attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06437v1",
            "date": "2018-04-17 18:59:51+00:00",
            "title": "Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer",
            "authors": [
                "Juncen Li",
                "Robin Jia",
                "He He",
                "Percy Liang"
            ],
            "abstract": "We consider the task of text attribute transfer: transforming a sentence to\nalter a specific attribute (e.g., sentiment) while preserving its\nattribute-independent content (e.g., changing \"screen is just the right size\"\nto \"screen is too small\"). Our training data includes only sentences labeled\nwith their attribute (e.g., positive or negative), but not pairs of sentences\nthat differ only in their attributes, so we must learn to disentangle\nattributes from attribute-independent content in an unsupervised way. Previous\nwork using adversarial methods has struggled to produce high-quality outputs.\nIn this paper, we propose simpler methods motivated by the observation that\ntext attributes are often marked by distinctive phrases (e.g., \"too small\").\nOur strongest method extracts content words by deleting phrases associated with\nthe sentence's original attribute value, retrieves new phrases associated with\nthe target attribute, and uses a neural model to fluently combine these into a\nfinal output. On human evaluation, our best method generates grammatical and\nappropriate responses on 22% more inputs than the best previous system,\naveraged over three attribute transfer datasets: altering sentiment of reviews\non Yelp, altering sentiment of reviews on Amazon, and altering image captions\nto be more romantic or humorous.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06440v1",
            "date": "2018-04-17 19:17:05+00:00",
            "title": "Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models",
            "authors": [
                "Sweta Karlekar",
                "Tong Niu",
                "Mohit Bansal"
            ],
            "abstract": "Alzheimer's disease (AD) is an irreversible and progressive brain disease\nthat can be stopped or slowed down with medical treatment. Language changes\nserve as a sign that a patient's cognitive functions have been impacted,\npotentially leading to early diagnosis. In this work, we use NLP techniques to\nclassify and analyze the linguistic characteristics of AD patients using the\nDementiaBank dataset. We apply three neural models based on CNNs, LSTM-RNNs,\nand their combination, to distinguish between language samples from AD and\ncontrol patients. We achieve a new independent benchmark accuracy for the AD\nclassification task. More importantly, we next interpret what these neural\nmodels have learned about the linguistic characteristics of AD patients, via\nanalysis based on activation clustering and first-derivative saliency\ntechniques. We then perform novel automatic pattern discovery inside activation\nclusters, and consolidate AD patients' distinctive grammar patterns.\nAdditionally, we show that first derivative saliency can not only rediscover\nprevious language patterns of AD patients, but also shed light on the\nlimitations of neural models. Lastly, we also include analysis of\ngender-separated AD data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06473v1",
            "date": "2018-04-17 21:24:20+00:00",
            "title": "Robust Machine Comprehension Models via Adversarial Training",
            "authors": [
                "Yicheng Wang",
                "Mohit Bansal"
            ],
            "abstract": "It is shown that many published models for the Stanford Question Answering\nDataset (Rajpurkar et al., 2016) lack robustness, suffering an over 50%\ndecrease in F1 score during adversarial evaluation based on the AddSent (Jia\nand Liang, 2017) algorithm. It has also been shown that retraining models on\ndata generated by AddSent has limited effect on their robustness. We propose a\nnovel alternative adversary-generation algorithm, AddSentDiverse, that\nsignificantly increases the variance within the adversarial training data by\nproviding effective examples that punish the model for making certain\nsuperficial assumptions. Further, in order to improve robustness to AddSent's\nsemantic perturbations (e.g., antonyms), we jointly improve the model's\nsemantic-relationship learning capabilities in addition to our\nAddSentDiverse-based adversarial training data augmentation. With these\nadditions, we show that we can make a state-of-the-art model significantly more\nrobust, achieving a 36.5% increase in F1 score under many different types of\nadversarial evaluation while maintaining performance on the regular SQuAD task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06506v1",
            "date": "2018-04-17 23:54:26+00:00",
            "title": "Improving Character-based Decoding Using Target-Side Morphological Information for Neural Machine Translation",
            "authors": [
                "Peyman Passban",
                "Qun Liu",
                "Andy Way"
            ],
            "abstract": "Recently, neural machine translation (NMT) has emerged as a powerful\nalternative to conventional statistical approaches. However, its performance\ndrops considerably in the presence of morphologically rich languages (MRLs).\nNeural engines usually fail to tackle the large vocabulary and high\nout-of-vocabulary (OOV) word rate of MRLs. Therefore, it is not suitable to\nexploit existing word-based models to translate this set of languages. In this\npaper, we propose an extension to the state-of-the-art model of Chung et al.\n(2016), which works at the character level and boosts the decoder with\ntarget-side morphological information. In our architecture, an additional\nmorphology table is plugged into the model. Each time the decoder samples from\na target vocabulary, the table sends auxiliary signals from the most relevant\naffixes in order to enrich the decoder's current state and constrain it to\nprovide better predictions. We evaluated our model to translate English into\nGerman, Russian, and Turkish as three MRLs and observed significant\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06512v1",
            "date": "2018-04-18 00:26:38+00:00",
            "title": "Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems",
            "authors": [
                "Bing Liu",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Pararth Shah",
                "Larry Heck"
            ],
            "abstract": "In this work, we present a hybrid learning method for training task-oriented\ndialogue systems through online user interactions. Popular methods for learning\ntask-oriented dialogues include applying reinforcement learning with user\nfeedback on supervised pre-training models. Efficiency of such learning method\nmay suffer from the mismatch of dialogue state distribution between offline\ntraining and online interactive learning stages. To address this challenge, we\npropose a hybrid imitation and reinforcement learning method, with which a\ndialogue agent can effectively learn from its interaction with users by\nlearning from human teaching and feedback. We design a neural network based\ntask-oriented dialogue agent that can be optimized end-to-end with the proposed\nlearning method. Experimental results show that our end-to-end dialogue agent\ncan learn effectively from the mistake it makes via imitation learning from\nuser teaching. Applying reinforcement learning with user feedback after the\nimitation learning stage further improves the agent's capability in\nsuccessfully completing a task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06517v1",
            "date": "2018-04-18 00:50:56+00:00",
            "title": "Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Change",
            "authors": [
                "Dominik Schlechtweg",
                "Sabine Schulte im Walde",
                "Stefanie Eckmann"
            ],
            "abstract": "We propose a framework that extends synchronic polysemy annotation to\ndiachronic changes in lexical meaning, to counteract the lack of resources for\nevaluating computational models of lexical semantic change. Our framework\nexploits an intuitive notion of semantic relatedness, and distinguishes between\ninnovative and reductive meaning changes with high inter-annotator agreement.\nThe resulting test set for German comprises ratings from five annotators for\nthe relatedness of 1,320 use pairs across 22 target words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06536v1",
            "date": "2018-04-18 03:15:01+00:00",
            "title": "Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks",
            "authors": [
                "Binxuan Huang",
                "Yanglan Ou",
                "Kathleen M. Carley"
            ],
            "abstract": "Aspect-level sentiment classification aims to identify the sentiment\nexpressed towards some aspects given context sentences. In this paper, we\nintroduce an attention-over-attention (AOA) neural network for aspect level\nsentiment classification. Our approach models aspects and sentences in a joint\nway and explicitly captures the interaction between aspects and context\nsentences. With the AOA module, our model jointly learns the representations\nfor aspects and sentences, and automatically focuses on the important parts in\nsentences. Our experiments on laptop and restaurant datasets demonstrate our\napproach outperforms previous LSTM-based architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06609v2",
            "date": "2018-04-18 09:06:11+00:00",
            "title": "Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation",
            "authors": [
                "Matt Post",
                "David Vilar"
            ],
            "abstract": "The end-to-end nature of neural machine translation (NMT) removes many ways\nof manually guiding the translation process that were available in older\nparadigms. Recent work, however, has introduced a new capability: lexically\nconstrained or guided decoding, a modification to beam search that forces the\ninclusion of pre-specified words and phrases in the output. However, while\ntheoretically sound, existing approaches have computational complexities that\nare either linear (Hokamp and Liu, 2017) or exponential (Anderson et al., 2017)\nin the number of constraints. We present a algorithm for lexically constrained\ndecoding with a complexity of O(1) in the number of constraints. We demonstrate\nthe algorithms remarkable ability to properly place these constraints, and use\nit to explore the shaky relationship between model and BLEU scores. Our\nimplementation is available as part of Sockeye.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06610v3",
            "date": "2018-04-18 09:07:16+00:00",
            "title": "End-to-end Graph-based TAG Parsing with Neural Networks",
            "authors": [
                "Jungo Kasai",
                "Robert Frank",
                "Pauli Xu",
                "William Merrill",
                "Owen Rambow"
            ],
            "abstract": "We present a graph-based Tree Adjoining Grammar (TAG) parser that uses\nBiLSTMs, highway connections, and character-level CNNs. Our best end-to-end\nparser, which jointly performs supertagging, POS tagging, and parsing,\noutperforms the previously reported best results by more than 2.2 LAS and UAS\npoints. The graph-based parsing architecture allows for global inference and\nrich feature representations for TAG parsing, alleviating the fundamental\ntrade-off between transition-based and graph-based parsing systems. We also\ndemonstrate that the proposed parser achieves state-of-the-art performance in\nthe downstream tasks of Parsing Evaluation using Textual Entailments (PETE) and\nUnbounded Dependency Recovery. This provides further support for the claim that\nTAG is a viable formalism for problems that require rich structural analysis of\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06636v1",
            "date": "2018-04-18 10:33:47+00:00",
            "title": "Experiments with Universal CEFR Classification",
            "authors": [
                "Sowmya Vajjala",
                "Taraka Rama"
            ],
            "abstract": "The Common European Framework of Reference (CEFR) guidelines describe\nlanguage proficiency of learners on a scale of 6 levels. While the description\nof CEFR guidelines is generic across languages, the development of automated\nproficiency classification systems for different languages follow different\napproaches. In this paper, we explore universal CEFR classification using\ndomain-specific and domain-agnostic, theory-guided as well as data-driven\nfeatures. We report the results of our preliminary experiments in monolingual,\ncross-lingual, and multilingual classification with three languages: German,\nCzech, and Italian. Our results show that both monolingual and multilingual\nmodels achieve similar performance, and cross-lingual classification yields\nlower, but comparable results to monolingual classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06657v1",
            "date": "2018-04-18 11:30:57+00:00",
            "title": "NTUA-SLP at SemEval-2018 Task 2: Predicting Emojis using RNNs with Context-aware Attention",
            "authors": [
                "Christos Baziotis",
                "Nikos Athanasiou",
                "Georgios Paraskevopoulos",
                "Nikolaos Ellinas",
                "Athanasia Kolovou",
                "Alexandros Potamianos"
            ],
            "abstract": "In this paper we present a deep-learning model that competed at SemEval-2018\nTask 2 \"Multilingual Emoji Prediction\". We participated in subtask A, in which\nwe are called to predict the most likely associated emoji in English tweets.\nThe proposed architecture relies on a Long Short-Term Memory network, augmented\nwith an attention mechanism, that conditions the weight of each word, on a\n\"context vector\" which is taken as the aggregation of a tweet's meaning.\nMoreover, we initialize the embedding layer of our model, with word2vec word\nembeddings, pretrained on a dataset of 550 million English tweets. Finally, our\nmodel does not rely on hand-crafted features or lexicons and is trained\nend-to-end with back-propagation. We ranked 2nd out of 48 teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06658v1",
            "date": "2018-04-18 11:31:06+00:00",
            "title": "NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning",
            "authors": [
                "Christos Baziotis",
                "Nikos Athanasiou",
                "Alexandra Chronopoulou",
                "Athanasia Kolovou",
                "Georgios Paraskevopoulos",
                "Nikolaos Ellinas",
                "Shrikanth Narayanan",
                "Alexandros Potamianos"
            ],
            "abstract": "In this paper we present deep-learning models that submitted to the\nSemEval-2018 Task~1 competition: \"Affect in Tweets\". We participated in all\nsubtasks for English tweets. We propose a Bi-LSTM architecture equipped with a\nmulti-layer self attention mechanism. The attention mechanism improves the\nmodel performance and allows us to identify salient words in tweets, as well as\ngain insight into the models making them more interpretable. Our model utilizes\na set of word2vec word embeddings trained on a large collection of 550 million\nTwitter messages, augmented by a set of word affective features. Due to the\nlimited amount of task-specific training data, we opted for a transfer learning\napproach by pretraining the Bi-LSTMs on the dataset of Semeval 2017, Task 4A.\nThe proposed approach ranked 1st in Subtask E \"Multi-Label Emotion\nClassification\", 2nd in Subtask A \"Emotion Intensity Regression\" and achieved\ncompetitive results in other subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06659v1",
            "date": "2018-04-18 11:35:56+00:00",
            "title": "NTUA-SLP at SemEval-2018 Task 3: Tracking Ironic Tweets using Ensembles of Word and Character Level Attentive RNNs",
            "authors": [
                "Christos Baziotis",
                "Nikos Athanasiou",
                "Pinelopi Papalampidi",
                "Athanasia Kolovou",
                "Georgios Paraskevopoulos",
                "Nikolaos Ellinas",
                "Alexandros Potamianos"
            ],
            "abstract": "In this paper we present two deep-learning systems that competed at\nSemEval-2018 Task 3 \"Irony detection in English tweets\". We design and ensemble\ntwo independent models, based on recurrent neural networks (Bi-LSTM), which\noperate at the word and character level, in order to capture both the semantic\nand syntactic information in tweets. Our models are augmented with a\nself-attention mechanism, in order to identify the most informative words. The\nembedding layer of our word-level model is initialized with word2vec word\nembeddings, pretrained on a collection of 550 million English tweets. We did\nnot utilize any handcrafted features, lexicons or external datasets as prior\ninformation and our models are trained end-to-end using back propagation on\nconstrained data. Furthermore, we provide visualizations of tweets with\nannotations for the salient tokens of the attention layer that can help to\ninterpret the inner workings of the proposed models. We ranked 2nd out of 42\nteams in Subtask A and 2nd out of 31 teams in Subtask B. However,\npost-task-completion enhancements of our models achieve state-of-the-art\nresults ranking 1st for both subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06705v1",
            "date": "2018-04-18 13:22:06+00:00",
            "title": "Alquist: The Alexa Prize Socialbot",
            "authors": [
                "Jan Pichl",
                "Petr Marek",
                "Jakub Konr\u00e1d",
                "Martin Matul\u00edk",
                "Hoang Long Nguyen",
                "Jan \u0160ediv\u00fd"
            ],
            "abstract": "This paper describes a new open domain dialogue system Alquist developed as\npart of the Alexa Prize competition for the Amazon Echo line of products. The\nAlquist dialogue system is designed to conduct a coherent and engaging\nconversation on popular topics. We are presenting a hybrid system combining\nseveral machine learning and rule based approaches. We discuss and describe the\nAlquist pipeline, data acquisition, and processing, dialogue manager, NLG,\nknowledge aggregation and hierarchy of sub-dialogs. We present some of the\nexperimental results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06716v1",
            "date": "2018-04-13 19:44:56+00:00",
            "title": "Demo of Sanskrit-Hindi SMT System",
            "authors": [
                "Rajneesh Pandey",
                "Atul Kr. Ojha",
                "Girish Nath Jha"
            ],
            "abstract": "The demo proposal presents a Phrase-based Sanskrit-Hindi (SaHiT) Statistical\nMachine Translation system. The system has been developed on Moses. 43k\nsentences of Sanskrit-Hindi parallel corpus and 56k sentences of a monolingual\ncorpus in the target language (Hindi) have been used. This system gives 57 BLEU\nscore.",
            "journal ref": "http://lrec-conf.org/workshops/lrec2018/W11/summaries/20_W11.html",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06719v1",
            "date": "2018-04-14 18:19:38+00:00",
            "title": "Distribution-based Prediction of the Degree of Grammaticalization for German Prepositions",
            "authors": [
                "Dominik Schlechtweg",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We test the hypothesis that the degree of grammaticalization of German\nprepositions correlates with their corpus-based contextual dispersion measured\nby word entropy. We find that there is indeed a moderate correlation for\nentropy, but a stronger correlation for frequency and number of context types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06868v2",
            "date": "2018-04-18 18:34:46+00:00",
            "title": "Learning to Map Context-Dependent Sentences to Executable Formal Queries",
            "authors": [
                "Alane Suhr",
                "Srinivasan Iyer",
                "Yoav Artzi"
            ],
            "abstract": "We propose a context-dependent model to map utterances within an interaction\nto executable formal queries. To incorporate interaction history, the model\nmaintains an interaction-level encoder that updates after each turn, and can\ncopy sub-sequences of previously predicted queries during generation. Our\napproach combines implicit and explicit modeling of references between\nutterances. We evaluate our model on the ATIS flight planning interactions, and\ndemonstrate the benefits of modeling context and explicit references.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06922v1",
            "date": "2018-04-18 21:32:17+00:00",
            "title": "Sentences with Gapping: Parsing and Reconstructing Elided Predicates",
            "authors": [
                "Sebastian Schuster",
                "Joakim Nivre",
                "Christopher D. Manning"
            ],
            "abstract": "Sentences with gapping, such as Paul likes coffee and Mary tea, lack an overt\npredicate to indicate the relation between two or more arguments. Surface\nsyntax representations of such sentences are often produced poorly by parsers,\nand even if correct, not well suited to downstream natural language\nunderstanding tasks such as relation extraction that are typically designed to\nextract information from sentences with canonical clause structure. In this\npaper, we present two methods for parsing to a Universal Dependencies graph\nrepresentation that explicitly encodes the elided material with additional\nnodes and edges. We find that both methods can reconstruct elided material from\ndependency trees with high accuracy when the parser correctly predicts the\nexistence of a gap. We further demonstrate that one of our methods can be\napplied to other languages based on a case study on Swedish.",
            "journal ref": "Proceedings of the 16th Annual Conference of the North American\n  Chapter of the Association for Computational Linguistics: Human Language\n  Technologies (NAACL 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06987v1",
            "date": "2018-04-19 03:38:43+00:00",
            "title": "Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention",
            "authors": [
                "Sharmistha Jat",
                "Siddhesh Khandelwal",
                "Partha Talukdar"
            ],
            "abstract": "Relation extraction is the problem of classifying the relationship between\ntwo entities in a given sentence. Distant Supervision (DS) is a popular\ntechnique for developing relation extractors starting with limited supervision.\nWe note that most of the sentences in the distant supervision relation\nextraction setting are very long and may benefit from word attention for better\nsentence representation. Our contributions in this paper are threefold.\nFirstly, we propose two novel word attention models for distantly- supervised\nrelation extraction: (1) a Bi-directional Gated Recurrent Unit (Bi-GRU) based\nword attention model (BGWA), (2) an entity-centric attention model (EA), and\n(3) a combination model which combines multiple complementary models using\nweighted voting method for improved relation extraction. Secondly, we introduce\nGDS, a new distant supervision dataset for relation extraction. GDS removes\ntest data noise present in all previous distant- supervision benchmark\ndatasets, making credible automatic evaluation possible. Thirdly, through\nextensive experiments on multiple real-world datasets, we demonstrate the\neffectiveness of the proposed methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07007v3",
            "date": "2018-04-19 06:13:48+00:00",
            "title": "QuaSE: Accurate Text Style Transfer under Quantifiable Guidance",
            "authors": [
                "Yi Liao",
                "Lidong Bing",
                "Piji Li",
                "Shuming Shi",
                "Wai Lam",
                "Tong Zhang"
            ],
            "abstract": "We propose the task of Quantifiable Sequence Editing (QuaSE): editing an\ninput sequence to generate an output sequence that satisfies a given numerical\noutcome value measuring a certain property of the sequence, with the\nrequirement of keeping the main content of the input sequence. For example, an\ninput sequence could be a word sequence, such as review sentence and\nadvertisement text. For a review sentence, the outcome could be the review\nrating; for an advertisement, the outcome could be the click-through rate. The\nmajor challenge in performing QuaSE is how to perceive the outcome-related\nwordings, and only edit them to change the outcome. In this paper, the proposed\nframework contains two latent factors, namely, outcome factor and content\nfactor, disentangled from the input sentence to allow convenient editing to\nchange the outcome and keep the content. Our framework explores the\npseudo-parallel sentences by modeling their content similarity and outcome\ndifferences to enable a better disentanglement of the latent factors, which\nallows generating an output to better satisfy the desired outcome and keep the\ncontent. The dual reconstruction structure further enhances the capability of\ngenerating expected output by exploiting the couplings of latent factors of\npseudo-parallel sentences. For evaluation, we prepared a dataset of Yelp review\nsentences with the ratings as outcome. Extensive experimental results are\nreported and discussed to elaborate the peculiarities of our framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07036v1",
            "date": "2018-04-19 08:42:32+00:00",
            "title": "Learning to Extract Coherent Summary via Deep Reinforcement Learning",
            "authors": [
                "Yuxiang Wu",
                "Baotian Hu"
            ],
            "abstract": "Coherence plays a critical role in producing a high-quality summary from a\ndocument. In recent years, neural extractive summarization is becoming\nincreasingly attractive. However, most of them ignore the coherence of\nsummaries when extracting sentences. As an effort towards extracting coherent\nsummaries, we propose a neural coherence model to capture the cross-sentence\nsemantic and syntactic coherence patterns. The proposed neural coherence model\nobviates the need for feature engineering and can be trained in an end-to-end\nfashion using unlabeled data. Empirical results show that the proposed neural\ncoherence model can efficiently capture the cross-sentence coherence patterns.\nUsing the combined output of the neural coherence model and ROUGE package as\nthe reward, we design a reinforcement learning method to train a proposed\nneural extractive summarizer which is named Reinforced Neural Extractive\nSummarization (RNES) model. The RNES model learns to optimize coherence and\ninformative importance of the summary simultaneously. Experimental results show\nthat the proposed RNES outperforms existing baselines and achieves\nstate-of-the-art performance in term of ROUGE on CNN/Daily Mail dataset. The\nqualitative evaluation indicates that summaries produced by RNES are more\ncoherent and readable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07068v1",
            "date": "2018-04-19 10:17:47+00:00",
            "title": "Consistent CCG Parsing over Multiple Sentences for Improved Logical Reasoning",
            "authors": [
                "Masashi Yoshikawa",
                "Koji Mineshima",
                "Hiroshi Noji",
                "Daisuke Bekki"
            ],
            "abstract": "In formal logic-based approaches to Recognizing Textual Entailment (RTE), a\nCombinatory Categorial Grammar (CCG) parser is used to parse input premises and\nhypotheses to obtain their logical formulas. Here, it is important that the\nparser processes the sentences consistently; failing to recognize a similar\nsyntactic structure results in inconsistent predicate argument structures among\nthem, in which case the succeeding theorem proving is doomed to failure. In\nthis work, we present a simple method to extend an existing CCG parser to parse\na set of sentences consistently, which is achieved with an inter-sentence\nmodeling with Markov Random Fields (MRF). When combined with existing\nlogic-based systems, our method always shows improvement in the RTE experiments\non English and Japanese languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07097v2",
            "date": "2018-04-19 11:40:08+00:00",
            "title": "Putting Question-Answering Systems into Practice: Transfer Learning for Efficient Domain Customization",
            "authors": [
                "Bernhard Kratzwald",
                "Stefan Feuerriegel"
            ],
            "abstract": "Traditional information retrieval (such as that offered by web search\nengines) impedes users with information overload from extensive result pages\nand the need to manually locate the desired information therein. Conversely,\nquestion-answering systems change how humans interact with information systems:\nusers can now ask specific questions and obtain a tailored answer - both\nconveniently in natural language. Despite obvious benefits, their use is often\nlimited to an academic context, largely because of expensive domain\ncustomizations, which means that the performance in domain-specific\napplications often fails to meet expectations. This paper proposes\ncost-efficient remedies: (i) we leverage metadata through a filtering\nmechanism, which increases the precision of document retrieval, and (ii) we\ndevelop a novel fuse-and-oversample approach for transfer learning in order to\nimprove the performance of answer extraction. Here knowledge is inductively\ntransferred from a related, yet different, tasks to the domain-specific\napplication, while accounting for potential differences in the sample sizes\nacross both tasks. The resulting performance is demonstrated with actual use\ncases from a finance company and the film industry, where fewer than 400\nquestion-answer pairs had to be annotated in order to yield significant\nperformance gains. As a direct implication to management, this presents a\npromising path to better leveraging of knowledge stored in information systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07212v2",
            "date": "2018-04-19 15:09:14+00:00",
            "title": "Learning Disentangled Representations of Texts with Application to Biomedical Abstracts",
            "authors": [
                "Sarthak Jain",
                "Edward Banner",
                "Jan-Willem van de Meent",
                "Iain J. Marshall",
                "Byron C. Wallace"
            ],
            "abstract": "We propose a method for learning disentangled representations of texts that\ncode for distinct and complementary aspects, with the aim of affording\nefficient model transfer and interpretability. To induce disentangled\nembeddings, we propose an adversarial objective based on the (dis)similarity\nbetween triplets of documents with respect to specific aspects. Our motivating\napplication is embedding biomedical abstracts describing clinical trials in a\nmanner that disentangles the populations, interventions, and outcomes in a\ngiven trial. We show that our method learns representations that encode these\nclinically salient aspects, and that these can be effectively used to perform\naspect-specific retrieval. We demonstrate that the approach generalizes beyond\nour motivating application in experiments on two multi-aspect review corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07253v1",
            "date": "2018-04-19 16:18:51+00:00",
            "title": "Helping or Hurting? Predicting Changes in Users' Risk of Self-Harm Through Online Community Interactions",
            "authors": [
                "Luca Soldaini",
                "Timothy Walsh",
                "Arman Cohan",
                "Julien Han",
                "Nazli Goharian"
            ],
            "abstract": "In recent years, online communities have formed around suicide and self-harm\nprevention. While these communities offer support in moment of crisis, they can\nalso normalize harmful behavior, discourage professional treatment, and\ninstigate suicidal ideation. In this work, we focus on how interaction with\nothers in such a community affects the mental state of users who are seeking\nsupport. We first build a dataset of conversation threads between users in a\ndistressed state and community members offering support. We then show how to\nconstruct a classifier to predict whether distressed users are helped or harmed\nby the interactions in the thread, and we achieve a macro-F1 score of up to\n0.69.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07329v2",
            "date": "2018-04-19 18:32:11+00:00",
            "title": "Assessing Language Proficiency from Eye Movements in Reading",
            "authors": [
                "Yevgeni Berzak",
                "Boris Katz",
                "Roger Levy"
            ],
            "abstract": "We present a novel approach for determining learners' second language\nproficiency which utilizes behavioral traces of eye movements during reading.\nOur approach provides stand-alone eyetracking based English proficiency scores\nwhich reflect the extent to which the learner's gaze patterns in reading are\nsimilar to those of native English speakers. We show that our scores correlate\nstrongly with standardized English proficiency tests. We also demonstrate that\ngaze information can be used to accurately predict the outcomes of such tests.\nOur approach yields the strongest performance when the test taker is presented\nwith a suite of sentences for which we have eyetracking data from other\nreaders. However, it remains effective even using eyetracking with sentences\nfor which eye movement data have not been previously collected. By deriving\nproficiency as an automatic byproduct of eye movements during ordinary reading,\nour approach offers a potentially valuable new tool for second language\nproficiency assessment. More broadly, our results open the door to future\nmethods for inferring reader characteristics from the behavioral traces of\nreading.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07375v1",
            "date": "2018-04-19 20:48:53+00:00",
            "title": "A Predictive Model for Notional Anaphora in English",
            "authors": [
                "Amir Zeldes"
            ],
            "abstract": "Notional anaphors are pronouns which disagree with their antecedents'\ngrammatical categories for notional reasons, such as plural to singular\nagreement in: 'the government ... they'. Since such cases are rare and conflict\nwith evidence from strictly agreeing cases ('the government ... it'), they\npresent a substantial challenge to both coreference resolution and referring\nexpression generation. Using the OntoNotes corpus, this paper takes an ensemble\napproach to predicting English notional anaphora in context on the basis of the\nlargest empirical data to date. In addition to state of the art prediction\naccuracy, the results suggest that theoretical approaches positing a plural\nconstrual at the antecedent's utterance are insufficient, and that\ncircumstances at the anaphor's utterance location, as well as global factors\nsuch as genre, have a strong effect on the choice of referring expression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07445v1",
            "date": "2018-04-20 03:52:20+00:00",
            "title": "Sentence Simplification with Memory-Augmented Neural Networks",
            "authors": [
                "Tu Vu",
                "Baotian Hu",
                "Tsendsuren Munkhdalai",
                "Hong Yu"
            ],
            "abstract": "Sentence simplification aims to simplify the content and structure of complex\nsentences, and thus make them easier to interpret for human readers, and easier\nto process for downstream NLP applications. Recent advances in neural machine\ntranslation have paved the way for novel approaches to the task. In this paper,\nwe adapt an architecture with augmented memory capacities called Neural\nSemantic Encoders (Munkhdalai and Yu, 2017) for sentence simplification. Our\nexperiments demonstrate the effectiveness of our approach on different\nsimplification datasets, both in terms of automatic evaluation measures and\nhuman judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07461v3",
            "date": "2018-04-20 06:35:04+00:00",
            "title": "GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding",
            "authors": [
                "Alex Wang",
                "Amanpreet Singh",
                "Julian Michael",
                "Felix Hill",
                "Omer Levy",
                "Samuel R. Bowman"
            ],
            "abstract": "For natural language understanding (NLU) technology to be maximally useful,\nboth practically and as a scientific object of study, it must be general: it\nmust be able to process language in a way that is not exclusively tailored to\nany one specific task or dataset. In pursuit of this objective, we introduce\nthe General Language Understanding Evaluation benchmark (GLUE), a tool for\nevaluating and analyzing the performance of models across a diverse range of\nexisting NLU tasks. GLUE is model-agnostic, but it incentivizes sharing\nknowledge across tasks because certain tasks have very limited training data.\nWe further provide a hand-crafted diagnostic test suite that enables detailed\nlinguistic analysis of NLU models. We evaluate baselines based on current\nmethods for multi-task and transfer learning and find that they do not\nimmediately give substantial improvements over the aggregate performance of\ntraining a separate model per task, indicating room for improvement in\ndeveloping general and robust NLU systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07581v1",
            "date": "2018-04-20 12:48:10+00:00",
            "title": "Automatic Stance Detection Using End-to-End Memory Networks",
            "authors": [
                "Mitra Mohtarami",
                "Ramy Baly",
                "James Glass",
                "Preslav Nakov",
                "Lluis Marquez",
                "Alessandro Moschitti"
            ],
            "abstract": "We present a novel end-to-end memory network for stance detection, which\njointly (i) predicts whether a document agrees, disagrees, discusses or is\nunrelated with respect to a given target claim, and also (ii) extracts snippets\nof evidence for that prediction. The network operates at the paragraph level\nand integrates convolutional and recurrent neural networks, as well as a\nsimilarity matrix as part of the overall architecture. The experimental\nevaluation on the Fake News Challenge dataset shows state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07587v1",
            "date": "2018-04-20 13:00:58+00:00",
            "title": "ClaimRank: Detecting Check-Worthy Claims in Arabic and English",
            "authors": [
                "Israa Jaradat",
                "Pepa Gencheva",
                "Alberto Barron-Cedeno",
                "Lluis Marquez",
                "Preslav Nakov"
            ],
            "abstract": "We present ClaimRank, an online system for detecting check-worthy claims.\nWhile originally trained on political debates, the system can work for any kind\nof text, e.g., interviews or regular news articles. Its aim is to facilitate\nmanual fact-checking efforts by prioritizing the claims that fact-checkers\nshould consider first. ClaimRank supports both Arabic and English, it is\ntrained on actual annotations from nine reputable fact-checking organizations\n(PolitiFact, FactCheck, ABC, CNN, NPR, NYT, Chicago Tribune, The Guardian, and\nWashington Post), and thus it can mimic the claim selection strategies for each\nand any of them, as well as for the union of them all.",
            "journal ref": "NAACL-2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07656v1",
            "date": "2018-04-20 15:02:25+00:00",
            "title": "Acquisition of Phrase Correspondences using Natural Deduction Proofs",
            "authors": [
                "Hitomi Yanaka",
                "Koji Mineshima",
                "Pascual Martinez-Gomez",
                "Daisuke Bekki"
            ],
            "abstract": "How to identify, extract, and use phrasal knowledge is a crucial problem for\nthe task of Recognizing Textual Entailment (RTE). To solve this problem, we\npropose a method for detecting paraphrases via natural deduction proofs of\nsemantic relations between sentence pairs. Our solution relies on a graph\nreformulation of partial variable unifications and an algorithm that induces\nsubgraph alignments between meaning representations. Experiments show that our\nmethod can automatically detect various paraphrases that are absent from\nexisting paraphrase databases. In addition, the detection of paraphrases using\nproof information improves the accuracy of RTE tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07705v2",
            "date": "2018-04-20 16:18:35+00:00",
            "title": "Lightweight Adaptive Mixture of Neural and N-gram Language Models",
            "authors": [
                "Anton Bakhtin",
                "Arthur Szlam",
                "Marc'Aurelio Ranzato",
                "Edouard Grave"
            ],
            "abstract": "It is often the case that the best performing language model is an ensemble\nof a neural language model with n-grams. In this work, we propose a method to\nimprove how these two models are combined. By using a small network which\npredicts the mixture weight between the two models, we adapt their relative\nimportance at each time step. Because the gating network is small, it trains\nquickly on small amounts of held out data, and does not add overhead at scoring\ntime. Our experiments carried out on the One Billion Word benchmark show a\nsignificant improvement over the state of the art ensemble without retraining\nof the basic modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07707v2",
            "date": "2018-04-20 16:24:12+00:00",
            "title": "Factorising AMR generation through syntax",
            "authors": [
                "Kris Cao",
                "Stephen Clark"
            ],
            "abstract": "Generating from Abstract Meaning Representation (AMR) is an underspecified\nproblem, as many syntactic decisions are not constrained by the semantic graph.\nTo explicitly account for this underspecification, we break down generating\nfrom AMR into two steps: first generate a syntactic structure, and then\ngenerate the surface form. We show that decomposing the generation process this\nway leads to state-of-the-art single model performance generating from AMR\nwithout additional unlabelled data. We also demonstrate that we can generate\nmeaning-preserving syntactic paraphrases of the same AMR graph, as judged by\nhumans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07726v2",
            "date": "2018-04-20 17:05:03+00:00",
            "title": "Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension",
            "authors": [
                "Minjoon Seo",
                "Tom Kwiatkowski",
                "Ankur P. Parikh",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We formalize a new modular variant of current question answering tasks by\nenforcing complete independence of the document encoder from the question\nencoder. This formulation addresses a key challenge in machine comprehension by\nrequiring a standalone representation of the document discourse. It\nadditionally leads to a significant scalability advantage since the encoding of\nthe answer candidate phrases in the document can be pre-computed and indexed\noffline for efficient retrieval. We experiment with baseline models for the new\ntask, which achieve a reasonable accuracy but significantly underperform\nunconstrained QA models. We invite the QA research community to engage in\nPhrase-Indexed Question Answering (PIQA, pika) for closing the gap. The\nleaderboard is at: nlp.cs.washington.edu/piqa",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07754v1",
            "date": "2018-04-20 17:58:45+00:00",
            "title": "Learning Semantic Textual Similarity from Conversations",
            "authors": [
                "Yinfei Yang",
                "Steve Yuan",
                "Daniel Cer",
                "Sheng-yi Kong",
                "Noah Constant",
                "Petr Pilar",
                "Heming Ge",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "We present a novel approach to learn representations for sentence-level\nsemantic similarity using conversational data. Our method trains an\nunsupervised model to predict conversational input-response pairs. The\nresulting sentence embeddings perform well on the semantic textual similarity\n(STS) benchmark and SemEval 2017's Community Question Answering (CQA) question\nsimilarity subtask. Performance is further improved by introducing multitask\ntraining combining the conversational input-response prediction task and a\nnatural language inference task. Extensive experiments show the proposed model\nachieves the best performance among all neural models on the STS benchmark and\nis competitive with the state-of-the-art feature engineered and mixed systems\nin both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07755v2",
            "date": "2018-04-20 17:59:13+00:00",
            "title": "Phrase-Based & Neural Unsupervised Machine Translation",
            "authors": [
                "Guillaume Lample",
                "Myle Ott",
                "Alexis Conneau",
                "Ludovic Denoyer",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Machine translation systems achieve near human-level performance on some\nlanguages, yet their effectiveness strongly relies on the availability of large\namounts of parallel sentences, which hinders their applicability to the\nmajority of language pairs. This work investigates how to learn to translate\nwhen having access to only large monolingual corpora in each language. We\npropose two model variants, a neural and a phrase-based model. Both versions\nleverage a careful initialization of the parameters, the denoising effect of\nlanguage models and automatic generation of parallel data by iterative\nback-translation. These models are significantly better than methods from the\nliterature, while being simpler and having fewer hyper-parameters. On the\nwidely used WMT'14 English-French and WMT'16 German-English benchmarks, our\nmodels respectively obtain 28.1 and 25.2 BLEU points without using a single\nparallel sentence, outperforming the state of the art by more than 11 BLEU\npoints. On low-resource languages like English-Urdu and English-Romanian, our\nmethods achieve even better results than semi-supervised and supervised\napproaches leveraging the paucity of available bitexts. Our code for NMT and\nPBSMT is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07781v3",
            "date": "2018-04-20 18:18:06+00:00",
            "title": "Pathologies of Neural Models Make Interpretations Difficult",
            "authors": [
                "Shi Feng",
                "Eric Wallace",
                "Alvin Grissom II",
                "Mohit Iyyer",
                "Pedro Rodriguez",
                "Jordan Boyd-Graber"
            ],
            "abstract": "One way to interpret neural model predictions is to highlight the most\nimportant input features---for example, a heatmap visualization over the words\nin an input sentence. In existing interpretation methods for NLP, a word's\nimportance is determined by either input perturbation---measuring the decrease\nin model confidence when that word is removed---or by the gradient with respect\nto that word. To understand the limitations of these methods, we use input\nreduction, which iteratively removes the least important word from the input.\nThis exposes pathological behaviors of neural models: the remaining words\nappear nonsensical to humans and are not the ones determined as important by\ninterpretation methods. As we confirm with human experiments, the reduced\nexamples lack information to support the prediction of any label, but models\nstill make the same predictions with high confidence. To explain these\ncounterintuitive results, we draw connections to adversarial examples and\nconfidence calibration: pathological behaviors reveal difficulties in\ninterpreting neural models trained with maximum likelihood. To mitigate their\ndeficiencies, we fine-tune the models by encouraging high entropy outputs on\nreduced examples. Fine-tuned models become more interpretable under input\nreduction without accuracy loss on regular examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07827v2",
            "date": "2018-04-20 21:10:17+00:00",
            "title": "Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling",
            "authors": [
                "Liyuan Liu",
                "Xiang Ren",
                "Jingbo Shang",
                "Jian Peng",
                "Jiawei Han"
            ],
            "abstract": "Many efforts have been made to facilitate natural language processing tasks\nwith pre-trained language models (LMs), and brought significant improvements to\nvarious applications. To fully leverage the nearly unlimited corpora and\ncapture linguistic information of multifarious levels, large-size LMs are\nrequired; but for a specific task, only parts of these information are useful.\nSuch large-sized LMs, even in the inference stage, may cause heavy computation\nworkloads, making them too time-consuming for large-scale applications. Here we\npropose to compress bulky LMs while preserving useful information with regard\nto a specific task. As different layers of the model keep different\ninformation, we develop a layer selection method for model pruning using\nsparsity-inducing regularization. By introducing the dense connectivity, we can\ndetach any layer without affecting others, and stretch shallow and wide LMs to\nbe deep and narrow. In model training, LMs are learned with layer-wise dropouts\nfor better robustness. Experiments on two benchmark datasets demonstrate the\neffectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07828v2",
            "date": "2018-04-20 21:10:21+00:00",
            "title": "A Multi-Axis Annotation Scheme for Event Temporal Relations",
            "authors": [
                "Qiang Ning",
                "Hao Wu",
                "Dan Roth"
            ],
            "abstract": "Existing temporal relation (TempRel) annotation schemes often have low\ninter-annotator agreements (IAA) even between experts, suggesting that the\ncurrent annotation task needs a better definition. This paper proposes a new\nmulti-axis modeling to better capture the temporal structure of events. In\naddition, we identify that event end-points are a major source of confusion in\nannotation, so we also propose to annotate TempRels based on start-points only.\nA pilot expert annotation using the proposed scheme shows significant\nimprovement in IAA from the conventional 60's to 80's (Cohen's Kappa). This\nbetter-defined annotation scheme further enables the use of crowdsourcing to\nalleviate the labor intensity for each annotator. We hope that this work can\nfoster more interesting studies towards event understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07835v2",
            "date": "2018-04-20 21:40:28+00:00",
            "title": "Direct Network Transfer: Transfer Learning of Sentence Embeddings for Semantic Similarity",
            "authors": [
                "Li Zhang",
                "Steven R. Wilson",
                "Rada Mihalcea"
            ],
            "abstract": "Sentence encoders, which produce sentence embeddings using neural networks,\nare typically evaluated by how well they transfer to downstream tasks. This\nincludes semantic similarity, an important task in natural language\nunderstanding. Although there has been much work dedicated to building sentence\nencoders, the accompanying transfer learning techniques have received\nrelatively little attention. In this paper, we propose a transfer learning\nsetting specialized for semantic similarity, which we refer to as direct\nnetwork transfer. Through experiments on several standard text similarity\ndatasets, we show that applying direct network transfer to existing encoders\ncan lead to state-of-the-art performance. Additionally, we compare several\napproaches to transfer sentence encoders to semantic similarity tasks, showing\nthat the choice of transfer learning setting greatly affects the performance in\nmany cases, and differs by encoder and dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07847v3",
            "date": "2018-04-20 22:16:40+00:00",
            "title": "Joint entity recognition and relation extraction as a multi-head selection problem",
            "authors": [
                "Giannis Bekoulis",
                "Johannes Deleu",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "State-of-the-art models for joint entity recognition and relation extraction\nstrongly rely on external natural language processing (NLP) tools such as POS\n(part-of-speech) taggers and dependency parsers. Thus, the performance of such\njoint models depends on the quality of the features obtained from these NLP\ntools. However, these features are not always accurate for various languages\nand contexts. In this paper, we propose a joint neural model which performs\nentity recognition and relation extraction simultaneously, without the need of\nany manually extracted features or the use of any external tool. Specifically,\nwe model the entity recognition task using a CRF (Conditional Random Fields)\nlayer and the relation extraction task as a multi-head selection problem (i.e.,\npotentially identify multiple relations for each entity). We present an\nextensive experimental setup, to demonstrate the effectiveness of our method\nusing datasets from various contexts (i.e., news, biomedical, real estate) and\nlanguages (i.e., English, Dutch). Our model outperforms the previous neural\nmodels that use automatically extracted features, while it performs within a\nreasonable margin of feature-based neural models, or even beats them.",
            "journal ref": "Expert Systems with Applications, Volume 114, 30 December 2018,\n  Pages 34-45, ISSN 0957-4174",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07849v4",
            "date": "2018-04-20 22:34:29+00:00",
            "title": "Mutual Information Maximization for Simple and Accurate Part-Of-Speech Induction",
            "authors": [
                "Karl Stratos"
            ],
            "abstract": "We address part-of-speech (POS) induction by maximizing the mutual\ninformation between the induced label and its context. We focus on two training\nobjectives that are amenable to stochastic gradient descent (SGD): a novel\ngeneralization of the classical Brown clustering objective and a recently\nproposed variational lower bound. While both objectives are subject to noise in\ngradient updates, we show through analysis and experiments that the variational\nlower bound is robust whereas the generalized Brown objective is vulnerable. We\nobtain competitive performance on a multitude of datasets and languages with a\nsimple architecture that encodes morphology and context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07853v1",
            "date": "2018-04-20 23:00:11+00:00",
            "title": "What's Going On in Neural Constituency Parsers? An Analysis",
            "authors": [
                "David Gaddy",
                "Mitchell Stern",
                "Dan Klein"
            ],
            "abstract": "A number of differences have emerged between modern and classic approaches to\nconstituency parsing in recent years, with structural components like grammars\nand feature-rich lexicons becoming less central while recurrent neural network\nrepresentations rise in popularity. The goal of this work is to analyze the\nextent to which information provided directly by the model structure in\nclassical systems is still being captured by neural methods. To this end, we\npropose a high-performance neural model (92.08 F1 on PTB) that is\nrepresentative of recent work and perform a series of investigative\nexperiments. We find that our model implicitly learns to encode much of the\nsame information that was explicitly provided by grammars and lexicons in the\npast, indicating that this scaffolding can largely be subsumed by powerful\ngeneral-purpose neural machinery.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07878v2",
            "date": "2018-04-21 02:18:36+00:00",
            "title": "Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation",
            "authors": [
                "Zhong Zhou",
                "Matthias Sperber",
                "Alex Waibel"
            ],
            "abstract": "We work on translation from rich-resource languages to low-resource\nlanguages. The main challenges we identify are the lack of low-resource\nlanguage data, effective methods for cross-lingual transfer, and the\nvariable-binding problem that is common in neural systems. We build a\ntranslation system that addresses these challenges using eight European\nlanguage families as our test ground. Firstly, we add the source and the target\nfamily labels and study intra-family and inter-family influences for effective\ncross-lingual transfer. We achieve an improvement of +9.9 in BLEU score for\nEnglish-Swedish translation using eight families compared to the single-family\nmulti-source multi-target baseline. Moreover, we find that training on two\nneighboring families closest to the low-resource language is often enough.\nSecondly, we construct an ablation study and find that reasonably good results\ncan be achieved even with considerably less target data. Thirdly, we address\nthe variable-binding problem by building an order-preserving named entity\ntranslation model. We obtain 60.6% accuracy in qualitative evaluation where our\ntranslations are akin to human translations in a preliminary study.",
            "journal ref": "Proceedings of the 3rd conference on Machine Translation Worshop\n  of the 23rd Conference on Empirical Methods in Natural Language Processing.\n  2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07881v1",
            "date": "2018-04-21 02:43:00+00:00",
            "title": "Event Extraction with Generative Adversarial Imitation Learning",
            "authors": [
                "Tongtao Zhang",
                "Heng Ji"
            ],
            "abstract": "We propose a new method for event extraction (EE) task based on an imitation\nlearning framework, specifically, inverse reinforcement learning (IRL) via\ngenerative adversarial network (GAN). The GAN estimates proper rewards\naccording to the difference between the actions committed by the expert (or\nground truth) and the agent among complicated states in the environment. EE\ntask benefits from these dynamic rewards because instances and labels yield to\nvarious extents of difficulty and the gains are expected to be diverse -- e.g.,\nan ambiguous but correctly detected trigger or argument should receive high\ngains -- while the traditional RL models usually neglect such differences and\npay equal attention on all instances. Moreover, our experiments also\ndemonstrate that the proposed framework outperforms state-of-the-art methods,\nwithout explicit feature engineering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07888v2",
            "date": "2018-04-21 04:31:59+00:00",
            "title": "Stochastic Answer Networks for Natural Language Inference",
            "authors": [
                "Xiaodong Liu",
                "Kevin Duh",
                "Jianfeng Gao"
            ],
            "abstract": "We propose a stochastic answer network (SAN) to explore multi-step inference\nstrategies in Natural Language Inference. Rather than directly predicting the\nresults given the inputs, the model maintains a state and iteratively refines\nits predictions. Our experiments show that SAN achieves the state-of-the-art\nresults on three benchmarks: Stanford Natural Language Inference (SNLI)\ndataset, MultiGenre Natural Language Inference (MultiNLI) dataset and Quora\nQuestion Pairs dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07889v2",
            "date": "2018-04-21 04:40:10+00:00",
            "title": "Entity-aware Image Caption Generation",
            "authors": [
                "Di Lu",
                "Spencer Whitehead",
                "Lifu Huang",
                "Heng Ji",
                "Shih-Fu Chang"
            ],
            "abstract": "Current image captioning approaches generate descriptions which lack specific\ninformation, such as named entities that are involved in the images. In this\npaper we propose a new task which aims to generate informative image captions,\ngiven images and hashtags as input. We propose a simple but effective approach\nto tackle this problem. We first train a convolutional neural networks - long\nshort term memory networks (CNN-LSTM) model to generate a template caption\nbased on the input image. Then we use a knowledge graph based collective\ninference algorithm to fill in the template with specific named entities\nretrieved via the hashtags. Experiments on a new benchmark dataset collected\nfrom Flickr show that our model generates news-style image descriptions with\nmuch richer information. Our model outperforms unimodal baselines significantly\nwith various evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07893v2",
            "date": "2018-04-21 05:24:10+00:00",
            "title": "Taylor's law for Human Linguistic Sequences",
            "authors": [
                "Tatsuru Kobayashi",
                "Kumiko Tanaka-Ishii"
            ],
            "abstract": "Taylor's law describes the fluctuation characteristics underlying a system in\nwhich the variance of an event within a time span grows by a power law with\nrespect to the mean. Although Taylor's law has been applied in many natural and\nsocial systems, its application for language has been scarce. This article\ndescribes a new quantification of Taylor's law in natural language and reports\nan analysis of over 1100 texts across 14 languages. The Taylor exponents of\nwritten natural language texts were found to exhibit almost the same value. The\nexponent was also compared for other language-related data, such as the\nchild-directed speech, music, and programming language code. The results show\nhow the Taylor exponent serves to quantify the fundamental structural\ncomplexity underlying linguistic time series. The article also shows the\napplicability of these findings in evaluating language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07899v2",
            "date": "2018-04-21 06:16:57+00:00",
            "title": "Unsupervised Natural Language Generation with Denoising Autoencoders",
            "authors": [
                "Markus Freitag",
                "Scott Roy"
            ],
            "abstract": "Generating text from structured data is important for various tasks such as\nquestion answering and dialog systems. We show that in at least one domain,\nwithout any supervision and only based on unlabeled text, we are able to build\na Natural Language Generation (NLG) system with higher performance than\nsupervised approaches. In our approach, we interpret the structured data as a\ncorrupt representation of the desired output and use a denoising auto-encoder\nto reconstruct the sentence. We show how to introduce noise into training\nexamples that do not contain structured data, and that the resulting denoising\nauto-encoder generalizes to generate correct sentences when given structured\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07911v2",
            "date": "2018-04-21 07:53:07+00:00",
            "title": "Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Tasks",
            "authors": [
                "Wasi Uddin Ahmad",
                "Xueying Bai",
                "Zhechao Huang",
                "Chao Jiang",
                "Nanyun Peng",
                "Kai-Wei Chang"
            ],
            "abstract": "Learning distributed sentence representations is one of the key challenges in\nnatural language processing. Previous work demonstrated that a recurrent neural\nnetwork (RNNs) based sentence encoder trained on a large collection of\nannotated natural language inference data, is efficient in the transfer\nlearning to facilitate other related tasks. In this paper, we show that joint\nlearning of multiple tasks results in better generalizable sentence\nrepresentations by conducting extensive experiments and analysis comparing the\nmulti-task and single-task learned sentence encoders. The quantitative analysis\nusing auxiliary tasks show that multi-task learning helps to embed better\nsemantic information in the sentence representations compared to single-task\nlearning. In addition, we compare multi-task sentence encoders with\ncontextualized word representations and show that combining both of them can\nfurther boost the performance of transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07915v2",
            "date": "2018-04-21 08:14:58+00:00",
            "title": "A Stable and Effective Learning Strategy for Trainable Greedy Decoding",
            "authors": [
                "Yun Chen",
                "Victor O. K. Li",
                "Kyunghyun Cho",
                "Samuel R. Bowman"
            ],
            "abstract": "Beam search is a widely used approximate search strategy for neural network\ndecoders, and it generally outperforms simple greedy decoding on tasks like\nmachine translation. However, this improvement comes at substantial\ncomputational cost. In this paper, we propose a flexible new method that allows\nus to reap nearly the full benefits of beam search with nearly no additional\ncomputational cost. The method revolves around a small neural network actor\nthat is trained to observe and manipulate the hidden state of a\npreviously-trained decoder. To train this actor network, we introduce the use\nof a pseudo-parallel corpus built using the output of beam search on a base\nmodel, ranked by a target quality metric like BLEU. Our method is inspired by\nearlier work on this problem, but requires no reinforcement learning, and can\nbe trained reliably on a range of models. Experiments on three parallel corpora\nand three architectures show that the method yields substantial improvements in\ntranslation quality and speed over each base system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07927v4",
            "date": "2018-04-21 09:43:06+00:00",
            "title": "DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension",
            "authors": [
                "Amrita Saha",
                "Rahul Aralikatte",
                "Mitesh M. Khapra",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "We propose DuoRC, a novel dataset for Reading Comprehension (RC) that\nmotivates several new challenges for neural approaches in language\nunderstanding beyond those offered by existing RC datasets. DuoRC contains\n186,089 unique question-answer pairs created from a collection of 7680 pairs of\nmovie plots where each pair in the collection reflects two versions of the same\nmovie - one from Wikipedia and the other from IMDb - written by two different\nauthors. We asked crowdsourced workers to create questions from one version of\nthe plot and a different set of workers to extract or synthesize answers from\nthe other version. This unique characteristic of DuoRC where questions and\nanswers are created from different versions of a document narrating the same\nunderlying story, ensures by design, that there is very little lexical overlap\nbetween the questions created from one version and the segments containing the\nanswer in the other version. Further, since the two versions have different\nlevels of plot detail, narration style, vocabulary, etc., answering questions\nfrom the second version requires deeper language understanding and\nincorporating external background knowledge. Additionally, the narrative style\nof passages arising from movie plots (as opposed to typical descriptive\npassages in existing datasets) exhibits the need to perform complex reasoning\nover events across multiple sentences. Indeed, we observe that state-of-the-art\nneural RC models which have achieved near human performance on the SQuAD\ndataset, even when coupled with traditional NLP techniques to address the\nchallenges presented in DuoRC exhibit very poor performance (F1 score of 37.42%\non DuoRC v/s 86% on SQuAD dataset). This opens up several interesting research\navenues wherein DuoRC could complement other RC datasets to explore novel\nneural approaches for studying language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07942v2",
            "date": "2018-04-21 10:59:06+00:00",
            "title": "Generative Stock Question Answering",
            "authors": [
                "Zhaopeng Tu",
                "Yong Jiang",
                "Xiaojiang Liu",
                "Lei Shu",
                "Shuming Shi"
            ],
            "abstract": "We study the problem of stock related question answering (StockQA):\nautomatically generating answers to stock related questions, just like\nprofessional stock analysts providing action recommendations to stocks upon\nuser's requests. StockQA is quite different from previous QA tasks since (1)\nthe answers in StockQA are natural language sentences (rather than entities or\nvalues) and due to the dynamic nature of StockQA, it is scarcely possible to\nget reasonable answers in an extractive way from the training data; and (2)\nStockQA requires properly analyzing the relationship between keywords in QA\npair and the numerical features of a stock. We propose to address the problem\nwith a memory-augmented encoder-decoder architecture, and integrate different\nmechanisms of number understanding and generation, which is a critical\ncomponent of StockQA.\n  We build a large-scale dataset containing over 180K StockQA instances, based\non which various technique combinations are extensively studied and compared.\nExperimental results show that a hybrid word-character model with separate\ncharacter components for number processing, achieves the best performance. By\nanalyzing the results, we found that 44.8% of answers generated by our best\nmodel still suffer from the generic answer problem, which can be alleviated by\na straightforward hybrid retrieval-generation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07954v2",
            "date": "2018-04-21 12:26:29+00:00",
            "title": "Automated essay scoring with string kernels and word embeddings",
            "authors": [
                "M\u0103d\u0103lina Cozma",
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this work, we present an approach based on combining string kernels and\nword embeddings for automatic essay scoring. String kernels capture the\nsimilarity among strings based on counting common character n-grams, which are\na low-level yet powerful type of feature, demonstrating state-of-the-art\nresults in various text classification tasks such as Arabic dialect\nidentification or native language identification. To our best knowledge, we are\nthe first to apply string kernels to automatically score essays. We are also\nthe first to combine them with a high-level semantic feature representation,\nnamely the bag-of-super-word-embeddings. We report the best performance on the\nAutomated Student Assessment Prize data set, in both in-domain and cross-domain\nsettings, surpassing recent state-of-the-art deep learning approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07961v3",
            "date": "2018-04-21 13:01:55+00:00",
            "title": "Faster Shift-Reduce Constituent Parsing with a Non-Binary, Bottom-Up Strategy",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "An increasingly wide range of artificial intelligence applications rely on\nsyntactic information to process and extract meaning from natural language text\nor speech, with constituent trees being one of the most widely used syntactic\nformalisms. To produce these phrase-structure representations from sentences in\nnatural language, shift-reduce constituent parsers have become one of the most\nefficient approaches. Increasing their accuracy and speed is still one of the\nmain objectives pursued by the research community so that artificial\nintelligence applications that make use of parsing outputs, such as machine\ntranslation or voice assistant services, can improve their performance. With\nthis goal in mind, we propose in this article a novel non-binary shift-reduce\nalgorithm for constituent parsing. Our parser follows a classical bottom-up\nstrategy but, unlike others, it straightforwardly creates non-binary branchings\nwith just one Reduce transition, instead of requiring prior binarization or a\nsequence of binary transitions, allowing its direct application to any language\nwithout the need of further resources such as percolation tables. As a result,\nit uses fewer transitions per sentence than existing transition-based\nconstituent parsers, becoming the fastest such system and, as a consequence,\nspeeding up downstream applications. Using static oracle training and greedy\nsearch, the accuracy of this novel approach is on par with state-of-the-art\ntransition-based constituent parsers and outperforms all top-down and bottom-up\ngreedy shift-reduce systems on the Wall Street Journal section from the English\nPenn Treebank and the Penn Chinese Treebank. Additionally, we develop a dynamic\noracle for training the proposed transition-based algorithm, achieving further\nimprovements in both benchmarks and obtaining the best accuracy to date on the\nPenn Chinese Treebank among greedy shift-reduce parsers.",
            "journal ref": "Artificial Intelligence, Volume 275, 2019, Pages 559-574, ISSN\n  0004-3702",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07972v2",
            "date": "2018-04-21 14:29:39+00:00",
            "title": "Eval all, trust a few, do wrong to none: Comparing sentence generation models",
            "authors": [
                "Ond\u0159ej C\u00edfka",
                "Aliaksei Severyn",
                "Enrique Alfonseca",
                "Katja Filippova"
            ],
            "abstract": "In this paper, we study recent neural generative models for text generation\nrelated to variational autoencoders. Previous works have employed various\ntechniques to control the prior distribution of the latent codes in these\nmodels, which is important for sampling performance, but little attention has\nbeen paid to reconstruction error. In our study, we follow a rigorous\nevaluation protocol using a large set of previously used and novel automatic\nand human evaluation metrics, applied to both generated samples and\nreconstructions. We hope that it will become the new evaluation standard when\ncomparing neural generative models for text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07976v3",
            "date": "2018-04-21 14:48:56+00:00",
            "title": "Neural-Davidsonian Semantic Proto-role Labeling",
            "authors": [
                "Rachel Rudinger",
                "Adam Teichert",
                "Ryan Culkin",
                "Sheng Zhang",
                "Benjamin Van Durme"
            ],
            "abstract": "We present a model for semantic proto-role labeling (SPRL) using an adapted\nbidirectional LSTM encoding strategy that we call \"Neural-Davidsonian\":\npredicate-argument structure is represented as pairs of hidden states\ncorresponding to predicate and argument head tokens of the input sequence. We\ndemonstrate: (1) state-of-the-art results in SPRL, and (2) that our network\nnaturally shares parameters between attributes, allowing for learning new\nattribute types with limited added supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07983v2",
            "date": "2018-04-21 15:32:32+00:00",
            "title": "Dynamic Meta-Embeddings for Improved Sentence Representations",
            "authors": [
                "Douwe Kiela",
                "Changhan Wang",
                "Kyunghyun Cho"
            ],
            "abstract": "While one of the first steps in many NLP systems is selecting what\npre-trained word embeddings to use, we argue that such a step is better left\nfor neural networks to figure out by themselves. To that end, we introduce\ndynamic meta-embeddings, a simple yet effective method for the supervised\nlearning of embedding ensembles, which leads to state-of-the-art performance\nwithin the same model class on a variety of tasks. We subsequently show how the\ntechnique can be used to shed new light on the usage of word embeddings in NLP\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07998v2",
            "date": "2018-04-21 17:02:20+00:00",
            "title": "Generating Natural Language Adversarial Examples",
            "authors": [
                "Moustafa Alzantot",
                "Yash Sharma",
                "Ahmed Elgohary",
                "Bo-Jhang Ho",
                "Mani Srivastava",
                "Kai-Wei Chang"
            ],
            "abstract": "Deep neural networks (DNNs) are vulnerable to adversarial examples,\nperturbations to correctly classified examples which can cause the model to\nmisclassify. In the image domain, these perturbations are often virtually\nindistinguishable to human perception, causing humans and state-of-the-art\nmodels to disagree. However, in the natural language domain, small\nperturbations are clearly perceptible, and the replacement of a single word can\ndrastically alter the semantics of the document. Given these challenges, we use\na black-box population-based optimization algorithm to generate semantically\nand syntactically similar adversarial examples that fool well-trained sentiment\nanalysis and textual entailment models with success rates of 97% and 70%,\nrespectively. We additionally demonstrate that 92.3% of the successful\nsentiment analysis adversarial examples are classified to their original label\nby 20 human annotators, and that the examples are perceptibly quite similar.\nFinally, we discuss an attempt to use adversarial training as a defense, but\nfail to yield improvement, demonstrating the strength and diversity of our\nadversarial examples. We hope our findings encourage researchers to pursue\nimproving the robustness of DNNs in the natural language domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08000v1",
            "date": "2018-04-21 17:21:28+00:00",
            "title": "Fine-grained Entity Typing through Increased Discourse Context and Adaptive Classification Thresholds",
            "authors": [
                "Sheng Zhang",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "Fine-grained entity typing is the task of assigning fine-grained semantic\ntypes to entity mentions. We propose a neural architecture which learns a\ndistributional semantic representation that leverages a greater amount of\nsemantic context -- both document and sentence level information -- than prior\nwork. We find that additional context improves performance, with further\nimprovements gained by utilizing adaptive classification thresholds.\nExperiments show that our approach without reliance on hand-crafted features\nachieves the state-of-the-art results on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08012v1",
            "date": "2018-04-21 19:18:22+00:00",
            "title": "Integrating Stance Detection and Fact Checking in a Unified Corpus",
            "authors": [
                "Ramy Baly",
                "Mitra Mohtarami",
                "James Glass",
                "Lluis Marquez",
                "Alessandro Moschitti",
                "Preslav Nakov"
            ],
            "abstract": "A reasonable approach for fact checking a claim involves retrieving\npotentially relevant documents from different sources (e.g., news websites,\nsocial media, etc.), determining the stance of each document with respect to\nthe claim, and finally making a prediction about the claim's factuality by\naggregating the strength of the stances, while taking the reliability of the\nsource into account. Moreover, a fact checking system should be able to explain\nits decision by providing relevant extracts (rationales) from the documents.\nYet, this setup is not directly supported by existing datasets, which treat\nfact checking, document retrieval, source credibility, stance detection and\nrationale extraction as independent tasks. In this paper, we support the\ninterdependencies between these tasks as annotations in the same corpus. We\nimplement this setup on an Arabic fact checking corpus, the first of its kind.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08037v1",
            "date": "2018-04-21 22:15:36+00:00",
            "title": "Cross-lingual Semantic Parsing",
            "authors": [
                "Sheng Zhang",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "We introduce the task of cross-lingual semantic parsing: mapping content\nprovided in a source language into a meaning representation based on a target\nlanguage. We present: (1) a meaning representation designed to allow systems to\ntarget varying levels of structural complexity (shallow to deep analysis), (2)\nan evaluation metric to measure the similarity between system output and\nreference meaning representations, (3) an end-to-end model with a novel copy\nmechanism that supports intrasentential coreference, and (4) an evaluation\ndataset where experiments show our model outperforms strong baselines by at\nleast 1.18 F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08049v4",
            "date": "2018-04-22 00:23:58+00:00",
            "title": "Semi-supervised User Geolocation via Graph Convolutional Networks",
            "authors": [
                "Afshin Rahimi",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Social media user geolocation is vital to many applications such as event\ndetection. In this paper, we propose GCN, a multiview geolocation model based\non Graph Convolutional Networks, that uses both text and network context. We\ncompare GCN to the state-of-the-art, and to two baselines we propose, and show\nthat our model achieves or is competitive with the state- of-the-art over three\nbenchmark geolocation datasets when sufficient supervision is available. We\nalso evaluate GCN under a minimal supervision scenario, and show it outperforms\nbaselines. We find that highway network gates are essential for controlling the\namount of useful neighbourhood expansion in GCN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08050v2",
            "date": "2018-04-22 00:28:23+00:00",
            "title": "Multi-Head Decoder for End-to-End Speech Recognition",
            "authors": [
                "Tomoki Hayashi",
                "Shinji Watanabe",
                "Tomoki Toda",
                "Kazuya Takeda"
            ],
            "abstract": "This paper presents a new network architecture called multi-head decoder for\nend-to-end speech recognition as an extension of a multi-head attention model.\nIn the multi-head attention model, multiple attentions are calculated, and\nthen, they are integrated into a single attention. On the other hand, instead\nof the integration in the attention level, our proposed method uses multiple\ndecoders for each attention and integrates their outputs to generate a final\noutput. Furthermore, in order to make each head to capture the different\nmodalities, different attention functions are used for each head, leading to\nthe improvement of the recognition performance with an ensemble effect. To\nevaluate the effectiveness of our proposed method, we conduct an experimental\nevaluation using Corpus of Spontaneous Japanese. Experimental results\ndemonstrate that our proposed method outperforms the conventional methods such\nas location-based and multi-head attention models, and that it can capture\ndifferent speech/linguistic contexts within the attention-based encoder-decoder\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08053v2",
            "date": "2018-04-22 01:02:23+00:00",
            "title": "Learning Sentence Embeddings for Coherence Modelling and Beyond",
            "authors": [
                "Tanner Bohn",
                "Yining Hu",
                "Jinhang Zhang",
                "Charles X. Ling"
            ],
            "abstract": "We present a novel and effective technique for performing text coherence\ntasks while facilitating deeper insights into the data. Despite obtaining\never-increasing task performance, modern deep-learning approaches to NLP tasks\noften only provide users with the final network decision and no additional\nunderstanding of the data. In this work, we show that a new type of sentence\nembedding learned through self-supervision can be applied effectively to text\ncoherence tasks while serving as a window through which deeper understanding of\nthe data can be obtained. To produce these sentence embeddings, we train a\nrecurrent neural network to take individual sentences and predict their\nlocation in a document in the form of a distribution over locations. We\ndemonstrate that these embeddings, combined with simple visual heuristics, can\nbe used to achieve performance competitive with state-of-the-art on multiple\ntext coherence tasks, outperforming more complex and specialized approaches.\nAdditionally, we demonstrate that these embeddings can provide insights useful\nto writers for improving writing quality and informing document structuring,\nand assisting readers in summarizing and locating information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08058v2",
            "date": "2018-04-22 02:24:54+00:00",
            "title": "Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching",
            "authors": [
                "Xiao Yang",
                "Madian Khabsa",
                "Miaosen Wang",
                "Wei Wang",
                "Madian Khabsa",
                "Ahmed Awadallah",
                "Daniel Kifer",
                "C. Lee Giles"
            ],
            "abstract": "Community-based question answering (CQA) websites represent an important\nsource of information. As a result, the problem of matching the most valuable\nanswers to their corresponding questions has become an increasingly popular\nresearch topic. We frame this task as a binary (relevant/irrelevant)\nclassification problem, and present an adversarial training framework to\nalleviate label imbalance issue. We employ a generative model to iteratively\nsample a subset of challenging negative samples to fool our classification\nmodel. Both models are alternatively optimized using REINFORCE algorithm. The\nproposed method is completely different from previous ones, where negative\nsamples in training set are directly used or uniformly down-sampled. Further,\nwe propose using Multi-scale Matching which explicitly inspects the correlation\nbetween words and ngrams of different levels of granularity. We evaluate the\nproposed method on SemEval 2016 and SemEval 2017 datasets and achieves\nstate-of-the-art or similar performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08064v1",
            "date": "2018-04-22 03:56:39+00:00",
            "title": "A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding",
            "authors": [
                "Young-Bum Kim",
                "Dongchan Kim",
                "Joo-Kyung Kim",
                "Ruhi Sarikaya"
            ],
            "abstract": "Intelligent personal digital assistants (IPDAs), a popular real-life\napplication with spoken language understanding capabilities, can cover\npotentially thousands of overlapping domains for natural language\nunderstanding, and the task of finding the best domain to handle an utterance\nbecomes a challenging problem on a large scale. In this paper, we propose a set\nof efficient and scalable neural shortlisting-reranking models for large-scale\ndomain classification in IPDAs. The shortlisting stage focuses on efficiently\ntrimming all domains down to a list of k-best candidate domains, and the\nreranking stage performs a list-wise reranking of the initial k-best domains\nwith additional contextual information. We show the effectiveness of our\napproach with extensive experiments on 1,500 IPDA domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08065v1",
            "date": "2018-04-22 04:00:36+00:00",
            "title": "Efficient Large-Scale Domain Classification with Personalized Attention",
            "authors": [
                "Young-Bum Kim",
                "Dongchan Kim",
                "Anjishnu Kumar",
                "Ruhi Sarikaya"
            ],
            "abstract": "In this paper, we explore the task of mapping spoken language utterances to\none of thousands of natural language understanding domains in intelligent\npersonal digital assistants (IPDAs). This scenario is observed for many\nmainstream IPDAs in industry that allow third parties to develop thousands of\nnew domains to augment built-in ones to rapidly increase domain coverage and\noverall IPDA capabilities. We propose a scalable neural model architecture with\na shared encoder, a novel attention mechanism that incorporates personalization\ninformation and domain-specific classifiers that solves the problem\nefficiently. Our architecture is designed to efficiently accommodate new\ndomains that appear in-between full model retraining cycles with a rapid\nbootstrapping mechanism two orders of magnitude faster than retraining. We\naccount for practical constraints in real-time production systems, and design\nto minimize memory footprint and runtime latency. We demonstrate that\nincorporating personalization results in significantly more accurate domain\nclassification in the setting with thousands of overlapping domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08077v2",
            "date": "2018-04-22 07:12:05+00:00",
            "title": "Inducing and Embedding Senses with Scaled Gumbel Softmax",
            "authors": [
                "Fenfei Guo",
                "Mohit Iyyer",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Methods for learning word sense embeddings represent a single word with\nmultiple sense-specific vectors. These methods should not only produce\ninterpretable sense embeddings, but should also learn how to select which sense\nto use in a given context. We propose an unsupervised model that learns sense\nembeddings using a modified Gumbel softmax function, which allows for\ndifferentiable discrete sense selection. Our model produces sense embeddings\nthat are competitive (and sometimes state of the art) on multiple similarity\nbased downstream evaluations. However, performance on these downstream\nevaluations tasks does not correlate with interpretability of sense embeddings,\nas we discover through an interpretability comparison with competing\nmulti-sense embeddings. While many previous approaches perform well on\ndownstream evaluations, they do not produce interpretable embeddings and learn\nduplicated sense groups; our method achieves the best of both worlds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08094v1",
            "date": "2018-04-22 11:01:08+00:00",
            "title": "IIIDYT at SemEval-2018 Task 3: Irony detection in English tweets",
            "authors": [
                "Edison Marrese-Taylor",
                "Suzana Ilic",
                "Jorge A. Balazs",
                "Yutaka Matsuo",
                "Helmut Prendinger"
            ],
            "abstract": "In this paper we introduce our system for the task of Irony detection in\nEnglish tweets, a part of SemEval 2018. We propose representation learning\napproach that relies on a multi-layered bidirectional LSTM, without using\nexternal features that provide additional semantic information. Although our\nmodel is able to outperform the baseline in the validation set, our results\nshow limited generalization power over the test set. Given the limited size of\nthe dataset, we think the usage of more pre-training schemes would greatly\nimprove the obtained results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08125v2",
            "date": "2018-04-22 15:44:17+00:00",
            "title": "Reduce, Reuse, Recycle: New uses for old QA resources",
            "authors": [
                "Jeff Mitchell",
                "Sebastian Riedel"
            ],
            "abstract": "We investigate applying repurposed generic QA data and models to a recently\nproposed relation extraction task. We find that training on SQuAD produces\nbetter zero-shot performance and more robust generalisation compared to the\ntask specific training set. We also show that standard QA architectures (e.g.\nFastQA or BiDAF) can be applied to the slot filling queries without the need\nfor model modification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08166v1",
            "date": "2018-04-22 20:42:13+00:00",
            "title": "Word Embedding Perturbation for Sentence Classification",
            "authors": [
                "Dongxu Zhang",
                "Zhichao Yang"
            ],
            "abstract": "In this technique report, we aim to mitigate the overfitting problem of\nnatural language by applying data augmentation methods. Specifically, we\nattempt several types of noise to perturb the input word embedding, such as\nGaussian noise, Bernoulli noise, and adversarial noise, etc. We also apply\nseveral constraints on different types of noise. By implementing these proposed\ndata augmentation methods, the baseline models can gain improvements on several\nsentence classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08186v2",
            "date": "2018-04-22 22:30:30+00:00",
            "title": "Automatic Language Identification in Texts: A Survey",
            "authors": [
                "Tommi Jauhiainen",
                "Marco Lui",
                "Marcos Zampieri",
                "Timothy Baldwin",
                "Krister Lind\u00e9n"
            ],
            "abstract": "Language identification (LI) is the problem of determining the natural\nlanguage that a document or part thereof is written in. Automatic LI has been\nextensively researched for over fifty years. Today, LI is a key part of many\ntext processing pipelines, as text processing techniques generally assume that\nthe language of the input text is known. Research in this area has recently\nbeen especially active. This article provides a brief history of LI research,\nand an extensive survey of the features and methods used so far in the LI\nliterature. For describing the features and methods we introduce a unified\nnotation. We discuss evaluation methods, applications of LI, as well as\noff-the-shelf LI systems that do not require training by the end user. Finally,\nwe identify open issues, survey the work to date on each issue, and propose\nfuture directions for research in LI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08198v3",
            "date": "2018-04-23 00:21:37+00:00",
            "title": "A neural interlingua for multilingual machine translation",
            "authors": [
                "Yichao Lu",
                "Phillip Keung",
                "Faisal Ladhak",
                "Vikas Bhardwaj",
                "Shaonan Zhang",
                "Jason Sun"
            ],
            "abstract": "We incorporate an explicit neural interlingua into a multilingual\nencoder-decoder neural machine translation (NMT) architecture. We demonstrate\nthat our model learns a language-independent representation by performing\ndirect zero-shot translation (without using pivot translation), and by using\nthe source sentence embeddings to create an English Yelp review classifier\nthat, through the mediation of the neural interlingua, can also classify French\nand German reviews. Furthermore, we show that, despite using a smaller number\nof parameters than a pairwise collection of bilingual NMT models, our approach\nproduces comparable BLEU scores for each language pair in WMT15.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08199v3",
            "date": "2018-04-23 00:21:49+00:00",
            "title": "Linguistically-Informed Self-Attention for Semantic Role Labeling",
            "authors": [
                "Emma Strubell",
                "Patrick Verga",
                "Daniel Andor",
                "David Weiss",
                "Andrew McCallum"
            ],
            "abstract": "Current state-of-the-art semantic role labeling (SRL) uses a deep neural\nnetwork with no explicit linguistic features. However, prior work has shown\nthat gold syntax trees can dramatically improve SRL decoding, suggesting the\npossibility of increased accuracy from explicit modeling of syntax. In this\nwork, we present linguistically-informed self-attention (LISA): a neural\nnetwork model that combines multi-head self-attention with multi-task learning\nacross dependency parsing, part-of-speech tagging, predicate detection and SRL.\nUnlike previous models which require significant pre-processing to prepare\nlinguistic features, LISA can incorporate syntax using merely raw tokens as\ninput, encoding the sequence only once to simultaneously perform parsing,\npredicate detection and role labeling for all predicates. Syntax is\nincorporated by training one attention head to attend to syntactic parents for\neach token. Moreover, if a high-quality syntactic parse is already available,\nit can be beneficially injected at test time without re-training our SRL model.\nIn experiments on CoNLL-2005 SRL, LISA achieves new state-of-the-art\nperformance for a model using predicted predicates and standard word\nembeddings, attaining 2.5 F1 absolute higher than the previous state-of-the-art\non newswire and more than 3.5 F1 on out-of-domain data, nearly 10% reduction in\nerror. On ConLL-2012 English SRL we also show an improvement of more than 2.5\nF1. LISA also out-performs the state-of-the-art with contextually-encoded\n(ELMo) word representations, by nearly 1.0 F1 on news and more than 2.0 F1 on\nout-of-domain text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08205v4",
            "date": "2018-04-23 00:56:23+00:00",
            "title": "Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model",
            "authors": [
                "Sabrina J. Mielke",
                "Jason Eisner"
            ],
            "abstract": "We show how the spellings of known words can help us deal with unknown words\nin open-vocabulary NLP tasks. The method we propose can be used to extend any\nclosed-vocabulary generative model, but in this paper we specifically consider\nthe case of neural language modeling. Our Bayesian generative story combines a\nstandard RNN language model (generating the word tokens in each sentence) with\nan RNN-based spelling model (generating the letters in each word type). These\ntwo RNNs respectively capture sentence structure and word structure, and are\nkept separate as in linguistics. By invoking the second RNN to generate\nspellings for novel words in context, we obtain an open-vocabulary language\nmodel. For known words, embeddings are naturally inferred by combining evidence\nfrom type spelling and token context. Comparing to baselines (including a novel\nstrong baseline), we beat previous work and establish state-of-the-art results\non multiple datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08207v2",
            "date": "2018-04-23 01:03:46+00:00",
            "title": "Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation",
            "authors": [
                "Adam Poliak",
                "Aparajita Haldar",
                "Rachel Rudinger",
                "J. Edward Hu",
                "Ellie Pavlick",
                "Aaron Steven White",
                "Benjamin Van Durme"
            ],
            "abstract": "We present a large-scale collection of diverse natural language inference\n(NLI) datasets that help provide insight into how well a sentence\nrepresentation captures distinct types of reasoning. The collection results\nfrom recasting 13 existing datasets from 7 semantic phenomena into a common NLI\nstructure, resulting in over half a million labeled context-hypothesis pairs in\ntotal. We refer to our collection as the DNC: Diverse Natural Language\nInference Collection. The DNC is available online at https://www.decomp.net,\nand will grow over time as additional resources are recast and added from novel\nsources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08217v3",
            "date": "2018-04-23 01:46:13+00:00",
            "title": "Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems",
            "authors": [
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "End-to-end task-oriented dialog systems usually suffer from the challenge of\nincorporating knowledge bases. In this paper, we propose a novel yet simple\nend-to-end differentiable model called memory-to-sequence (Mem2Seq) to address\nthis issue. Mem2Seq is the first neural generative model that combines the\nmulti-hop attention over memories with the idea of pointer network. We\nempirically show how Mem2Seq controls each generation step, and how its\nmulti-hop attention mechanism helps in learning correlations between memories.\nIn addition, our model is quite general without complicated task-specific\ndesigns. As a result, we show that Mem2Seq can be trained faster and attain the\nstate-of-the-art performance on three different task-oriented dialog datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08228v1",
            "date": "2018-04-23 02:38:20+00:00",
            "title": "Parsing Tweets into Universal Dependencies",
            "authors": [
                "Yijia Liu",
                "Yi Zhu",
                "Wanxiang Che",
                "Bing Qin",
                "Nathan Schneider",
                "Noah A. Smith"
            ],
            "abstract": "We study the problem of analyzing tweets with Universal Dependencies. We\nextend the UD guidelines to cover special constructions in tweets that affect\ntokenization, part-of-speech tagging, and labeled dependencies. Using the\nextended guidelines, we create a new tweet treebank for English (Tweebank v2)\nthat is four times larger than the (unlabeled) Tweebank v1 introduced by Kong\net al. (2014). We characterize the disagreements between our annotators and\nshow that it is challenging to deliver consistent annotation due to ambiguity\nin understanding and explaining tweets. Nonetheless, using the new treebank, we\nbuild a pipeline system to parse raw tweets into UD. To overcome annotation\nnoise without sacrificing computational efficiency, we propose a new method to\ndistill an ensemble of 20 transition-based parsers into a single one. Our\nparser achieves an improvement of 2.2 in LAS over the un-ensembled baseline and\noutperforms parsers that are state-of-the-art on other treebanks in both\naccuracy and speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08262v1",
            "date": "2018-04-23 07:01:52+00:00",
            "title": "On the Diachronic Stability of Irregularity in Inflectional Morphology",
            "authors": [
                "Ryan Cotterell",
                "Christo Kirov",
                "Mans Hulden",
                "Jason Eisner"
            ],
            "abstract": "Many languages' inflectional morphological systems are replete with\nirregulars, i.e., words that do not seem to follow standard inflectional rules.\nIn this work, we quantitatively investigate the conditions under which\nirregulars can survive in a language over the course of time. Using recurrent\nneural networks to simulate language learners, we test the diachronic relation\nbetween frequency of words and their irregularity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08266v1",
            "date": "2018-04-23 07:21:21+00:00",
            "title": "NLITrans at SemEval-2018 Task 12: Transfer of Semantic Knowledge for Argument Comprehension",
            "authors": [
                "Tim Niven",
                "Hung-Yu Kao"
            ],
            "abstract": "The Argument Reasoning Comprehension Task requires significant language\nunderstanding and complex reasoning over world knowledge. We focus on transfer\nof a sentence encoder to bootstrap more complicated models given the small size\nof the dataset. Our best model uses a pre-trained BiLSTM to encode input\nsentences, learns task-specific features for the argument and warrants, then\nperforms independent argument-warrant matching. This model achieves mean test\nset accuracy of 64.43%. Encoder transfer yields a significant gain to our best\nmodel over random initialization. Independent warrant matching effectively\ndoubles the size of the dataset and provides additional regularization. We\ndemonstrate that regularization comes from ignoring statistical correlations\nbetween warrant features and position. We also report an experiment with our\nbest model that only matches warrants to reasons, ignoring claims. Relatively\nlow performance degradation suggests that our model is not necessarily learning\nthe intended task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08280v1",
            "date": "2018-04-23 08:30:46+00:00",
            "title": "PlusEmo2Vec at SemEval-2018 Task 1: Exploiting emotion knowledge from emoji and #hashtags",
            "authors": [
                "Ji Ho Park",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "This paper describes our system that has been submitted to SemEval-2018 Task\n1: Affect in Tweets (AIT) to solve five subtasks. We focus on modeling both\nsentence and word level representations of emotion inside texts through large\ndistantly labeled corpora with emojis and hashtags. We transfer the emotional\nknowledge by exploiting neural network models as feature extractors and use\nthese representations for traditional machine learning models such as support\nvector regression (SVR) and logistic regression to solve the competition tasks.\nOur system is placed among the Top3 for all subtasks we participated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08313v2",
            "date": "2018-04-23 09:54:29+00:00",
            "title": "Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks",
            "authors": [
                "Diego Marcheggiani",
                "Jasmijn Bastings",
                "Ivan Titov"
            ],
            "abstract": "Semantic representations have long been argued as potentially useful for\nenforcing meaning preservation and improving generalization performance of\nmachine translation methods. In this work, we are the first to incorporate\ninformation about predicate-argument structure of source sentences (namely,\nsemantic-role representations) into neural machine translation. We use Graph\nConvolutional Networks (GCNs) to inject a semantic bias into sentence encoders\nand achieve improvements in BLEU scores over the linguistic-agnostic and\nsyntax-aware versions on the English--German language pair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08338v1",
            "date": "2018-04-23 11:18:47+00:00",
            "title": "Semantic Parsing with Syntax- and Table-Aware SQL Generation",
            "authors": [
                "Yibo Sun",
                "Duyu Tang",
                "Nan Duan",
                "Jianshu Ji",
                "Guihong Cao",
                "Xiaocheng Feng",
                "Bing Qin",
                "Ting Liu",
                "Ming Zhou"
            ],
            "abstract": "We present a generative model to map natural language questions into SQL\nqueries. Existing neural network based approaches typically generate a SQL\nquery word-by-word, however, a large portion of the generated results are\nincorrect or not executable due to the mismatch between question words and\ntable contents. Our approach addresses this problem by considering the\nstructure of table and the syntax of SQL language. The quality of the generated\nSQL query is significantly improved through (1) learning to replicate content\nfrom column names, cells or SQL keywords; and (2) improving the generation of\nWHERE clause by leveraging the column-cell relation. Experiments are conducted\non WikiSQL, a recently released dataset with the largest question-SQL pairs.\nOur approach significantly improves the state-of-the-art execution accuracy\nfrom 69.0% to 74.4%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08460v1",
            "date": "2018-04-23 14:22:21+00:00",
            "title": "Mixing Context Granularities for Improved Entity Linking on Question Answering Data across Entity Categories",
            "authors": [
                "Daniil Sorokin",
                "Iryna Gurevych"
            ],
            "abstract": "The first stage of every knowledge base question answering approach is to\nlink entities in the input question. We investigate entity linking in the\ncontext of a question answering task and present a jointly optimized neural\narchitecture for entity mention detection and entity disambiguation that models\nthe surrounding context on different levels of granularity. We use the Wikidata\nknowledge base and available question answering datasets to create benchmarks\nfor entity linking on question answering data. Our approach outperforms the\nprevious state-of-the-art system on this data, resulting in an average 8%\nimprovement of the final score. We further demonstrate that our model delivers\na strong performance across different entity categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08477v1",
            "date": "2018-04-23 14:42:12+00:00",
            "title": "ASR Performance Prediction on Unseen Broadcast Programs using Convolutional Neural Networks",
            "authors": [
                "Zied Elloumi",
                "Laurent Besacier",
                "Olivier Galibert",
                "Juliette Kahn",
                "Benjamin Lecouteux"
            ],
            "abstract": "In this paper, we address a relatively new task: prediction of ASR\nperformance on unseen broadcast programs. We first propose an heterogenous\nFrench corpus dedicated to this task. Two prediction approaches are compared: a\nstate-of-the-art performance prediction based on regression (engineered\nfeatures) and a new strategy based on convolutional neural networks (learnt\nfeatures). We particularly focus on the combination of both textual (ASR\ntranscription) and signal inputs. While the joint use of textual and signal\nfeatures did not work for the regression baseline, the combination of inputs\nfor CNNs leads to the best WER prediction performance. We also show that our\nCNN prediction remarkably predicts the WER distribution on a collection of\nspeech recordings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08666v2",
            "date": "2018-04-23 18:52:50+00:00",
            "title": "Using Aspect Extraction Approaches to Generate Review Summaries and User Profiles",
            "authors": [
                "Christopher Mitcheltree",
                "Skyler Wharton",
                "Avneesh Saluja"
            ],
            "abstract": "Reviews of products or services on Internet marketplace websites contain a\nrich amount of information. Users often wish to survey reviews or review\nsnippets from the perspective of a certain aspect, which has resulted in a\nlarge body of work on aspect identification and extraction from such corpora.\nIn this work, we evaluate a newly-proposed neural model for aspect extraction\non two practical tasks. The first is to extract canonical sentences of various\naspects from reviews, and is judged by human evaluators against alternatives. A\n$k$-means baseline does remarkably well in this setting. The second experiment\nfocuses on the suitability of the recovered aspect distributions to represent\nusers by the reviews they have written. Through a set of review reranking\nexperiments, we find that aspect-based profiles can largely capture notions of\nuser preferences, by showing that divergent users generate markedly different\nreview rankings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08675v1",
            "date": "2018-04-23 19:15:45+00:00",
            "title": "Data-Driven Investigative Journalism For Connectas Dataset",
            "authors": [
                "Aniket Jain",
                "Bhavya Sharma",
                "Paridhi Choudhary",
                "Rohan Sangave",
                "William Yang"
            ],
            "abstract": "The following paper explores the possibility of using Machine Learning\nalgorithms to detect the cases of corruption and malpractice by governments.\nThe dataset used by the authors contains information about several government\ncontracts in Colombia from year 2007 to 2012. The authors begin with exploring\nand cleaning the data, followed by which they perform feature engineering\nbefore finally implementing Machine Learning models to detect anomalies in the\ngiven dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08749v1",
            "date": "2018-04-23 21:29:30+00:00",
            "title": "Can Eye Movement Data Be Used As Ground Truth For Word Embeddings Evaluation?",
            "authors": [
                "Amir Bakarov"
            ],
            "abstract": "In recent years a certain success in the task of modeling lexical semantics\nwas obtained with distributional semantic models. Nevertheless, the scientific\ncommunity is still unaware what is the most reliable evaluation method for\nthese models. Some researchers argue that the only possible gold standard could\nbe obtained from neuro-cognitive resources that store information about human\ncognition. One of such resources is eye movement data on silent reading. The\ngoal of this work is to test the hypothesis of whether such data could be used\nto evaluate distributional semantic models on different languages. We propose\nexperiments with English and Russian eye movement datasets (Provo Corpus, GECO\nand Russian Sentence Corpus), word vectors (Skip-Gram models trained on\nnational corpora and Web corpora) and word similarity datasets of Russian and\nEnglish assessed by humans in order to find the existence of correlation\nbetween embeddings and eye movement data and test the hypothesis that this\ncorrelation is language independent. As a result, we found that the validity of\nthe hypothesis being tested could be questioned.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08756v1",
            "date": "2018-04-23 21:56:11+00:00",
            "title": "Detecting Syntactic Features of Translated Chinese",
            "authors": [
                "Hai Hu",
                "Wen Li",
                "Sandra K\u00fcbler"
            ],
            "abstract": "We present a machine learning approach to distinguish texts translated to\nChinese (by humans) from texts originally written in Chinese, with a focus on a\nwide range of syntactic features. Using Support Vector Machines (SVMs) as\nclassifier on a genre-balanced corpus in translation studies of Chinese, we\nfind that constituent parse trees and dependency triples as features without\nlexical information perform very well on the task, with an F-measure above 90%,\nclose to the results of lexical n-gram features, without the risk of learning\ntopic information rather than translation features. Thus, we claim syntactic\nfeatures alone can accurately distinguish translated from original Chinese.\nTranslated Chinese exhibits an increased use of determiners, subject position\npronouns, NP + 'de' as NP modifiers, multiple NPs or VPs conjoined by a Chinese\nspecific punctuation, among other structures. We also interpret the syntactic\nfeatures with reference to previous translation studies in Chinese,\nparticularly the usage of pronouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08771v2",
            "date": "2018-04-23 22:54:55+00:00",
            "title": "A Call for Clarity in Reporting BLEU Scores",
            "authors": [
                "Matt Post"
            ],
            "abstract": "The field of machine translation faces an under-recognized problem because of\ninconsistency in the reporting of scores from its dominant metric. Although\npeople refer to \"the\" BLEU score, BLEU is in fact a parameterized metric whose\nvalues can vary wildly with changes to these parameters. These parameters are\noften not reported or are hard to find, and consequently, BLEU scores between\npapers cannot be directly compared. I quantify this variation, finding\ndifferences as high as 1.8 between commonly used configurations. The main\nculprit is different tokenization and normalization schemes applied to the\nreference. Pointing to the success of the parsing community, I suggest machine\ntranslation researchers settle upon the BLEU scheme used by the annual\nConference on Machine Translation (WMT), which does not allow for user-supplied\nreference processing, and provide a new tool, SacreBLEU, to facilitate this.",
            "journal ref": "Proceedings of the Third Conference on Machine Translation\n  (WMT18). 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08813v3",
            "date": "2018-04-24 02:29:14+00:00",
            "title": "End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions",
            "authors": [
                "Wenpeng Yin",
                "Hinrich Sch\u00fctze",
                "Dan Roth"
            ],
            "abstract": "This work deals with SciTail, a natural entailment challenge derived from a\nmulti-choice question answering problem. The premises and hypotheses in SciTail\nwere generated with no awareness of each other, and did not specifically aim at\nthe entailment task. This makes it more challenging than other entailment data\nsets and more directly useful to the end-task -- question answering. We propose\nDEISTE (deep explorations of inter-sentence interactions for textual\nentailment) for this entailment task. Given word-to-word interactions between\nthe premise-hypothesis pair ($P$, $H$), DEISTE consists of: (i) a\nparameter-dynamic convolution to make important words in $P$ and $H$ play a\ndominant role in learnt representations; and (ii) a position-aware attentive\nconvolution to encode the representation and position information of the\naligned word pairs. Experiments show that DEISTE gets $\\approx$5\\% improvement\nover prior state of the art and that the pretrained DEISTE on SciTail\ngeneralizes well on RTE-5.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08845v1",
            "date": "2018-04-24 05:34:59+00:00",
            "title": "Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment",
            "authors": [
                "Tu Vu",
                "Vered Shwartz"
            ],
            "abstract": "Supervised distributional methods are applied successfully in lexical\nentailment, but recent work questioned whether these methods actually learn a\nrelation between two words. Specifically, Levy et al. (2015) claimed that\nlinear classifiers learn only separate properties of each word. We suggest a\ncheap and easy way to boost the performance of these methods by integrating\nmultiplicative features into commonly used representations. We provide an\nextensive evaluation with different classifiers and evaluation setups, and\nsuggest a suitable evaluation setup for the task, eliminating biases existing\nin previous ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08875v1",
            "date": "2018-04-24 07:40:31+00:00",
            "title": "Data-driven Summarization of Scientific Articles",
            "authors": [
                "Nikola I. Nikolov",
                "Michael Pfeiffer",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "Data-driven approaches to sequence-to-sequence modelling have been\nsuccessfully applied to short text summarization of news articles. Such models\nare typically trained on input-summary pairs consisting of only a single or a\nfew sentences, partially due to limited availability of multi-sentence training\ndata. Here, we propose to use scientific articles as a new milestone for text\nsummarization: large-scale training data come almost for free with two types of\nhigh-quality summaries at different levels - the title and the abstract. We\ngenerate two novel multi-sentence summarization datasets from scientific\narticles and test the suitability of a wide range of existing extractive and\nabstractive neural network-based summarization approaches. Our analysis\ndemonstrates that scientific papers are suitable for data-driven text\nsummarization. Our results could serve as valuable benchmarks for scaling\nsequence-to-sequence models to very long sequences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08881v1",
            "date": "2018-04-24 07:58:20+00:00",
            "title": "Assessing Language Models with Scaling Properties",
            "authors": [
                "Shuntaro Takahashi",
                "Kumiko Tanaka-Ishii"
            ],
            "abstract": "Language models have primarily been evaluated with perplexity. While\nperplexity quantifies the most comprehensible prediction performance, it does\nnot provide qualitative information on the success or failure of models.\nAnother approach for evaluating language models is thus proposed, using the\nscaling properties of natural language. Five such tests are considered, with\nthe first two accounting for the vocabulary population and the other three for\nthe long memory of natural language. The following models were evaluated with\nthese tests: n-grams, probabilistic context-free grammar (PCFG), Simon and\nPitman-Yor (PY) processes, hierarchical PY, and neural language models. Only\nthe neural language models exhibit the long memory properties of natural\nlanguage, but to a limited degree. The effectiveness of every test of these\nmodels is also discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08887v1",
            "date": "2018-04-24 08:10:07+00:00",
            "title": "SIRIUS-LTG-UiO at SemEval-2018 Task 7: Convolutional Neural Networks with Shortest Dependency Paths for Semantic Relation Extraction and Classification in Scientific Papers",
            "authors": [
                "Farhad Nooralahzadeh",
                "Lilja \u00d8vrelid",
                "Jan Tore L\u00f8nning"
            ],
            "abstract": "This article presents the SIRIUS-LTG-UiO system for the SemEval 2018 Task 7\non Semantic Relation Extraction and Classification in Scientific Papers. First\nwe extract the shortest dependency path (sdp) between two entities, then we\nintroduce a convolutional neural network (CNN) which takes the shortest\ndependency path embeddings as input and performs relation classification with\ndiffering objectives for each subtask of the shared task. This approach\nachieved overall F1 scores of 76.7 and 83.2 for relation classification on\nclean and noisy data, respectively. Furthermore, for combined relation\nextraction and classification on clean data, it obtained F1 scores of 37.4 and\n33.6 for each phase. Our system ranks 3rd in all three sub-tasks of the shared\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08915v1",
            "date": "2018-04-24 09:18:13+00:00",
            "title": "Scheduled Multi-Task Learning: From Syntax to Translation",
            "authors": [
                "Eliyahu Kiperwasser",
                "Miguel Ballesteros"
            ],
            "abstract": "Neural encoder-decoder models of machine translation have achieved impressive\nresults, while learning linguistic knowledge of both the source and target\nlanguages in an implicit end-to-end manner. We propose a framework in which our\nmodel begins learning syntax and translation interleaved, gradually putting\nmore focus on translation. Using this approach, we achieve considerable\nimprovements in terms of BLEU score on relatively large parallel corpus (WMT14\nEnglish to German) and a low-resource (WIT German to English) setup.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  6:225-240 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09000v3",
            "date": "2018-04-24 12:58:45+00:00",
            "title": "Style Transfer Through Back-Translation",
            "authors": [
                "Shrimai Prabhumoye",
                "Yulia Tsvetkov",
                "Ruslan Salakhutdinov",
                "Alan W Black"
            ],
            "abstract": "Style transfer is the task of rephrasing the text to contain specific\nstylistic properties without changing the intent or affect within the context.\nThis paper introduces a new method for automatic style transfer. We first learn\na latent representation of the input sentence which is grounded in a language\ntranslation model in order to better preserve the meaning of the sentence while\nreducing stylistic properties. Then adversarial generation techniques are used\nto make the output match the desired style. We evaluate this technique on three\ndifferent style transformations: sentiment, gender and political slant.\nCompared to two state-of-the-art style transfer modeling techniques we show\nimprovements both in automatic evaluation of style transfer and in manual\nevaluation of meaning preservation and fluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09010v1",
            "date": "2018-04-24 13:24:08+00:00",
            "title": "Towards a Neural Network Approach to Abstractive Multi-Document Summarization",
            "authors": [
                "Jianmin Zhang",
                "Jiwei Tan",
                "Xiaojun Wan"
            ],
            "abstract": "Till now, neural abstractive summarization methods have achieved great\nsuccess for single document summarization (SDS). However, due to the lack of\nlarge scale multi-document summaries, such methods can be hardly applied to\nmulti-document summarization (MDS). In this paper, we investigate neural\nabstractive methods for MDS by adapting a state-of-the-art neural abstractive\nsummarization model for SDS. We propose an approach to extend the neural\nabstractive model trained on large scale SDS data to the MDS task. Our approach\nonly makes use of a small number of multi-document summaries for fine tuning.\nExperimental results on two benchmark DUC datasets demonstrate that our\napproach can outperform a variety of baseline neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09057v1",
            "date": "2018-04-24 14:11:28+00:00",
            "title": "Unsupervised Neural Machine Translation with Weight Sharing",
            "authors": [
                "Zhen Yang",
                "Wei Chen",
                "Feng Wang",
                "Bo Xu"
            ],
            "abstract": "Unsupervised neural machine translation (NMT) is a recently proposed approach\nfor machine translation which aims to train the model without using any labeled\ndata. The models proposed for unsupervised NMT often use only one shared\nencoder to map the pairs of sentences from different languages to a\nshared-latent space, which is weak in keeping the unique and internal\ncharacteristics of each language, such as the style, terminology, and sentence\nstructure. To address this issue, we introduce an extension by utilizing two\nindependent encoders but sharing some partial weights which are responsible for\nextracting high-level representations of the input sentences. Besides, two\ndifferent generative adversarial networks (GANs), namely the local GAN and\nglobal GAN, are proposed to enhance the cross-language translation. With this\nnew approach, we achieve significant improvements on English-German,\nEnglish-French and Chinese-to-English translation tasks.",
            "journal ref": "ACL2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09132v1",
            "date": "2018-04-24 16:49:30+00:00",
            "title": "A Report on the Complex Word Identification Shared Task 2018",
            "authors": [
                "Seid Muhie Yimam",
                "Chris Biemann",
                "Shervin Malmasi",
                "Gustavo H. Paetzold",
                "Lucia Specia",
                "Sanja \u0160tajner",
                "Ana\u00efs Tack",
                "Marcos Zampieri"
            ],
            "abstract": "We report the findings of the second Complex Word Identification (CWI) shared\ntask organized as part of the BEA workshop co-located with NAACL-HLT'2018. The\nsecond CWI shared task featured multilingual and multi-genre datasets divided\ninto four tracks: English monolingual, German monolingual, Spanish monolingual,\nand a multilingual track with a French test set, and two tasks: binary\nclassification and probabilistic classification. A total of 12 teams submitted\ntheir results in different task/track combinations and 11 of them wrote system\ndescription papers that are referred to in this report and appear in the BEA\nworkshop proceedings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09259v1",
            "date": "2018-04-24 21:07:04+00:00",
            "title": "Commonsense mining as knowledge base completion? A study on the impact of novelty",
            "authors": [
                "Stanis\u0142aw Jastrz\u0119bski",
                "Dzmitry Bahdanau",
                "Seyedarian Hosseini",
                "Michael Noukhovitch",
                "Yoshua Bengio",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Commonsense knowledge bases such as ConceptNet represent knowledge in the\nform of relational triples. Inspired by the recent work by Li et al., we\nanalyse if knowledge base completion models can be used to mine commonsense\nknowledge from raw text. We propose novelty of predicted triples with respect\nto the training set as an important factor in interpreting results. We\ncritically analyse the difficulty of mining novel commonsense knowledge, and\nshow that a simple baseline method outperforms the previous state of the art on\npredicting more novel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09301v1",
            "date": "2018-04-25 00:46:14+00:00",
            "title": "Gender Bias in Coreference Resolution",
            "authors": [
                "Rachel Rudinger",
                "Jason Naradowsky",
                "Brian Leonard",
                "Benjamin Van Durme"
            ],
            "abstract": "We present an empirical study of gender bias in coreference resolution\nsystems. We first introduce a novel, Winograd schema-style set of minimal pair\nsentences that differ only by pronoun gender. With these \"Winogender schemas,\"\nwe evaluate and confirm systematic gender bias in three publicly-available\ncoreference resolution systems, and correlate this bias with real-world and\ntextual gender statistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09321v1",
            "date": "2018-04-25 02:18:51+00:00",
            "title": "Hierarchical RNN for Information Extraction from Lawsuit Documents",
            "authors": [
                "Xi Rao",
                "Zhenxing Ke"
            ],
            "abstract": "Every lawsuit document contains the information about the party's claim,\ncourt's analysis, decision and others, and all of this information are helpful\nto understand the case better and predict the judge's decision on similar case\nin the future. However, the extraction of these information from the document\nis difficult because the language is too complicated and sentences varied at\nlength. We treat this problem as a task of sequence labeling, and this paper\npresents the first research to extract relevant information from the civil\nlawsuit document in China with the hierarchical RNN framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09692v1",
            "date": "2018-04-25 17:40:20+00:00",
            "title": "Factors Influencing the Surprising Instability of Word Embeddings",
            "authors": [
                "Laura Wendlandt",
                "Jonathan K. Kummerfeld",
                "Rada Mihalcea"
            ],
            "abstract": "Despite the recent popularity of word embedding methods, there is only a\nsmall body of work exploring the limitations of these representations. In this\npaper, we consider one aspect of embedding spaces, namely their stability. We\nshow that even relatively high frequency words (100-200 occurrences) are often\nunstable. We provide empirical evidence for how various factors contribute to\nthe stability of word embeddings, and we analyze the effects of stability on\ndownstream tasks.",
            "journal ref": "NAACL-HLT (2018) 2092-2102",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09769v1",
            "date": "2018-04-25 19:35:56+00:00",
            "title": "TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation",
            "authors": [
                "Tao Yu",
                "Zifan Li",
                "Zilin Zhang",
                "Rui Zhang",
                "Dragomir Radev"
            ],
            "abstract": "Interacting with relational databases through natural language helps users of\nany background easily query and analyze a vast amount of data. This requires a\nsystem that understands users' questions and converts them to SQL queries\nautomatically. In this paper we present a novel approach, TypeSQL, which views\nthis problem as a slot filling task. Additionally, TypeSQL utilizes type\ninformation to better understand rare entities and numbers in natural language\nquestions. We test this idea on the WikiSQL dataset and outperform the prior\nstate-of-the-art by 5.5% in much less time. We also show that accessing the\ncontent of databases can significantly improve the performance when users'\nqueries are not well-formed. TypeSQL gets 82.6% accuracy, a 17.5% absolute\nimprovement compared to the previous content-sensitive model.",
            "journal ref": "The 16th Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics, New Orleans, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09779v2",
            "date": "2018-04-25 20:03:09+00:00",
            "title": "On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference",
            "authors": [
                "Adam Poliak",
                "Yonatan Belinkov",
                "James Glass",
                "Benjamin Van Durme"
            ],
            "abstract": "We propose a process for investigating the extent to which sentence\nrepresentations arising from neural machine translation (NMT) systems encode\ndistinct semantic phenomena. We use these representations as features to train\na natural language inference (NLI) classifier based on datasets recast from\nexisting semantic annotations. In applying this process to a representative NMT\nsystem, we find its encoder appears most suited to supporting inferences at the\nsyntax-semantics interface, as compared to anaphora resolution requiring\nworld-knowledge. We conclude with a discussion on the merits and potential\ndeficiencies of the existing process, and how it may be improved and extended\nas a broader framework for evaluating semantic coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09931v4",
            "date": "2018-04-26 08:10:58+00:00",
            "title": "Integrating Local Context and Global Cohesiveness for Open Information Extraction",
            "authors": [
                "Qi Zhu",
                "Xiang Ren",
                "Jingbo Shang",
                "Yu Zhang",
                "Ahmed El-Kishky",
                "Jiawei Han"
            ],
            "abstract": "Extracting entities and their relations from text is an important task for\nunderstanding massive text corpora. Open information extraction (IE) systems\nmine relation tuples (i.e., entity arguments and a predicate string to describe\ntheir relation) from sentences. These relation tuples are not confined to a\npredefined schema for the relations of interests. However, current Open IE\nsystems focus on modeling local context information in a sentence to extract\nrelation tuples, while ignoring the fact that global statistics in a large\ncorpus can be collectively leveraged to identify high-quality sentence-level\nextractions. In this paper, we propose a novel Open IE system, called ReMine,\nwhich integrates local context signals and global structural signals in a\nunified, distant-supervision framework. Leveraging facts from external\nknowledge bases as supervision, the new system can be applied to many different\ndomains to facilitate sentence-level tuple extractions using corpus-level\nstatistics. Our system operates by solving a joint optimization problem to\nunify (1) segmenting entity/relation phrases in individual sentences based on\nlocal context; and (2) measuring the quality of tuples extracted from\nindividual sentences with a translating-based objective. Learning the two\nsubtasks jointly helps correct errors produced in each subtask so that they can\nmutually enhance each other. Experiments on two real-world corpora from\ndifferent domains demonstrate the effectiveness, generality, and robustness of\nReMine when compared to state-of-the-art open IE systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10184v1",
            "date": "2018-04-26 17:35:15+00:00",
            "title": "Lessons from the Bible on Modern Topics: Low-Resource Multilingual Topic Model Evaluation",
            "authors": [
                "Shudong Hao",
                "Jordan Boyd-Graber",
                "Michael J. Paul"
            ],
            "abstract": "Multilingual topic models enable document analysis across languages through\ncoherent multilingual summaries of the data. However, there is no standard and\neffective metric to evaluate the quality of multilingual topics. We introduce a\nnew intrinsic evaluation of multilingual topic models that correlates well with\nhuman judgments of multilingual topic coherence as well as performance in\ndownstream applications. Importantly, we also study evaluation for low-resource\nlanguages. Because standard metrics fail to accurately measure topic quality\nwhen robust external resources are unavailable, we propose an adaptation model\nthat improves the accuracy and reliability of these metrics in low-resource\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10413v1",
            "date": "2018-04-27 09:33:49+00:00",
            "title": "Extracting Parallel Paragraphs from Common Crawl",
            "authors": [
                "Jakub K\u00fadela",
                "Irena Holubov\u00e1",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Most of the current methods for mining parallel texts from the web assume\nthat web pages of web sites share same structure across languages. We believe\nthat there still exists a non-negligible amount of parallel data spread across\nsources not satisfying this assumption. We propose an approach based on a\ncombination of bivec (a bilingual extension of word2vec) and locality-sensitive\nhashing which allows us to efficiently identify pairs of parallel segments\nlocated anywhere on pages of a given web domain, regardless their structure. We\nvalidate our method on realigning segments from a large parallel corpus.\nAnother experiment with real-world data provided by Common Crawl Foundation\nconfirms that our solution scales to hundreds of terabytes large set of\nweb-crawled data.",
            "journal ref": "The Prague Bulletin of Mathematical Linguistics, Volume 107, Issue\n  1, Pages 39-56, ISSN (Online) 1804-0462 (2017)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10490v1",
            "date": "2018-04-27 13:24:11+00:00",
            "title": "Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading",
            "authors": [
                "Martin Raison",
                "Pierre-Emmanuel Mazar\u00e9",
                "Rajarshi Das",
                "Antoine Bordes"
            ],
            "abstract": "This paper aims at improving how machines can answer questions directly from\ntext, with the focus of having models that can answer correctly multiple types\nof questions and from various types of texts, documents or even from large\ncollections of them. To that end, we introduce the Weaver model that uses a new\nway to relate a question to a textual context by weaving layers of recurrent\nnetworks, with the goal of making as few assumptions as possible as to how the\ninformation from both question and context should be combined to form the\nanswer. We show empirically on six datasets that Weaver performs well in\nmultiple conditions. For instance, it produces solid results on the very\npopular SQuAD dataset (Rajpurkar et al., 2016), solves almost all bAbI tasks\n(Weston et al., 2015) and greatly outperforms state-of-the-art methods for open\ndomain question answering from text (Chen et al., 2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10615v2",
            "date": "2018-04-27 17:59:15+00:00",
            "title": "Improving Coverage and Runtime Complexity for Exact Inference in Non-Projective Transition-Based Dependency Parsers",
            "authors": [
                "Tianze Shi",
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Lillian Lee"
            ],
            "abstract": "We generalize Cohen, G\\'omez-Rodr\\'iguez, and Satta's (2011) parser to a\nfamily of non-projective transition-based dependency parsers allowing\npolynomial-time exact inference. This includes novel parsers with better\ncoverage than Cohen et al. (2011), and even a variant that reduces time\ncomplexity to $O(n^6)$, improving over the known bounds in exact inference for\nnon-projective transition-based parsing. We hope that this piece of theoretical\nwork inspires design of novel transition systems with better coverage and\nbetter run-time guarantees.\n  Code available at https://github.com/tzshi/nonproj-dp-variants-naacl2018",
            "journal ref": "Proceedings of NAACL-HLT 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10637v1",
            "date": "2018-04-27 18:28:24+00:00",
            "title": "Improving Entity Linking by Modeling Latent Relations between Mentions",
            "authors": [
                "Phong Le",
                "Ivan Titov"
            ],
            "abstract": "Entity linking involves aligning textual mentions of named entities to their\ncorresponding entries in a knowledge base. Entity linking systems often exploit\nrelations between textual mentions in a document (e.g., coreference) to decide\nif the linking decisions are compatible. Unlike previous approaches, which\nrelied on supervised systems or heuristics to predict these relations, we treat\nrelations as latent variables in our neural entity-linking model. We induce the\nrelations without any supervision while optimizing the entity-linking system in\nan end-to-end fashion. Our multi-relational model achieves the best reported\nscores on the standard benchmark (AIDA-CoNLL) and substantially outperforms its\nrelation-agnostic version. Its training also converges much faster, suggesting\nthat the injected structural bias helps to explain regularities in the training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10686v1",
            "date": "2018-04-27 20:44:49+00:00",
            "title": "An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages",
            "authors": [
                "Dmitry Ustalov",
                "Denis Teslenko",
                "Alexander Panchenko",
                "Mikhail Chernoskutov",
                "Chris Biemann",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "In this paper, we present Watasense, an unsupervised system for word sense\ndisambiguation. Given a sentence, the system chooses the most relevant sense of\neach input word with respect to the semantic similarity between the given\nsentence and the synset constituting the sense of the target word. Watasense\nhas two modes of operation. The sparse mode uses the traditional vector space\nmodel to estimate the most similar word sense corresponding to its context. The\ndense mode, instead, uses synset embeddings to cope with the sparsity problem.\nWe describe the architecture of the present system and also conduct its\nevaluation on three different lexical semantic resources for Russian. We found\nthat the dense mode substantially outperforms the sparse one on all datasets\naccording to the adjusted Rand index.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10731v3",
            "date": "2018-04-28 03:29:22+00:00",
            "title": "Sentiment Adaptive End-to-End Dialog Systems",
            "authors": [
                "Weiyan Shi",
                "Zhou Yu"
            ],
            "abstract": "End-to-end learning framework is useful for building dialog systems for its\nsimplicity in training and efficiency in model updating. However, current\nend-to-end approaches only consider user semantic inputs in learning and\nunder-utilize other user information. Therefore, we propose to include user\nsentiment obtained through multimodal information (acoustic, dialogic and\ntextual), in the end-to-end learning framework to make systems more\nuser-adaptive and effective. We incorporated user sentiment information in both\nsupervised and reinforcement learning settings. In both settings, adding\nsentiment information reduced the dialog length and improved the task success\nrate on a bus information search task. This work is the first attempt to\nincorporate multimodal user information in the adaptive end-to-end dialog\nsystem training framework and attained state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10747v1",
            "date": "2018-04-28 06:10:45+00:00",
            "title": "Neural Particle Smoothing for Sampling from Conditional Sequence Models",
            "authors": [
                "Chu-Cheng Lin",
                "Jason Eisner"
            ],
            "abstract": "We introduce neural particle smoothing, a sequential Monte Carlo method for\nsampling annotations of an input string from a given probability model. In\ncontrast to conventional particle filtering algorithms, we train a proposal\ndistribution that looks ahead to the end of the input string by means of a\nright-to-left LSTM. We demonstrate that this innovation can improve the quality\nof the sample. To motivate our formal choices, we explain how our neural model\nand neural sampler can be viewed as low-dimensional but nonlinear\napproximations to working with HMMs over very large state spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10959v1",
            "date": "2018-04-29 16:13:44+00:00",
            "title": "Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates",
            "authors": [
                "Taku Kudo"
            ],
            "abstract": "Subword units are an effective way to alleviate the open vocabulary problems\nin neural machine translation (NMT). While sentences are usually converted into\nunique subword sequences, subword segmentation is potentially ambiguous and\nmultiple segmentations are possible even with the same vocabulary. The question\naddressed in this paper is whether it is possible to harness the segmentation\nambiguity as a noise to improve the robustness of NMT. We present a simple\nregularization method, subword regularization, which trains the model with\nmultiple subword segmentations probabilistically sampled during training. In\naddition, for better subword sampling, we propose a new subword segmentation\nalgorithm based on a unigram language model. We experiment with multiple\ncorpora and report consistent improvements especially on low resource and\nout-of-domain settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11019v1",
            "date": "2018-04-30 01:57:31+00:00",
            "title": "Recurrent Entity Networks with Delayed Memory Update for Targeted Aspect-based Sentiment Analysis",
            "authors": [
                "Fei Liu",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "While neural networks have been shown to achieve impressive results for\nsentence-level sentiment analysis, targeted aspect-based sentiment analysis\n(TABSA) --- extraction of fine-grained opinion polarity w.r.t. a pre-defined\nset of aspects --- remains a difficult task. Motivated by recent advances in\nmemory-augmented models for machine reading, we propose a novel architecture,\nutilising external \"memory chains\" with a delayed memory update mechanism to\ntrack entities. On a TABSA task, the proposed model demonstrates substantial\nimprovements over state-of-the-art approaches, including those using external\nknowledge bases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11225v2",
            "date": "2018-04-30 14:17:36+00:00",
            "title": "Automatic Metric Validation for Grammatical Error Correction",
            "authors": [
                "Leshem Choshen",
                "Omri Abend"
            ],
            "abstract": "Metric validation in Grammatical Error Correction (GEC) is currently done by\nobserving the correlation between human and metric-induced rankings. However,\nsuch correlation studies are costly, methodologically troublesome, and suffer\nfrom low inter-rater agreement. We propose MAEGE, an automatic methodology for\nGEC metric validation, that overcomes many of the difficulties with existing\npractices. Experiments with \\maege\\ shed a new light on metric quality, showing\nfor example that the standard $M^2$ metric fares poorly on corpus-level\nranking. Moreover, we use MAEGE to perform a detailed analysis of metric\nbehavior, showing that correcting some types of errors is consistently\npenalized by existing metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11251v1",
            "date": "2018-04-30 14:58:22+00:00",
            "title": "BomJi at SemEval-2018 Task 10: Combining Vector-, Pattern- and Graph-based Information to Identify Discriminative Attributes",
            "authors": [
                "Enrico Santus",
                "Chris Biemann",
                "Emmanuele Chersoni"
            ],
            "abstract": "This paper describes BomJi, a supervised system for capturing discriminative\nattributes in word pairs (e.g. yellow as discriminative for banana over\nwatermelon). The system relies on an XGB classifier trained on carefully\nengineered graph-, pattern- and word embedding based features. It participated\nin the SemEval- 2018 Task 10 on Capturing Discriminative Attributes, achieving\nan F1 score of 0:73 and ranking 2nd out of 26 participant systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11254v3",
            "date": "2018-04-30 14:59:56+00:00",
            "title": "Inherent Biases in Reference based Evaluation for Grammatical Error Correction and Text Simplification",
            "authors": [
                "Leshem Choshen",
                "Omri Abend"
            ],
            "abstract": "The prevalent use of too few references for evaluating text-to-text\ngeneration is known to bias estimates of their quality ({\\it low coverage bias}\nor LCB). This paper shows that overcoming LCB in Grammatical Error Correction\n(GEC) evaluation cannot be attained by re-scaling or by increasing the number\nof references in any feasible range, contrary to previous suggestions. This is\ndue to the long-tailed distribution of valid corrections for a sentence.\nConcretely, we show that LCB incentivizes GEC systems to avoid correcting even\nwhen they can generate a valid correction. Consequently, existing systems\nobtain comparable or superior performance compared to humans, by making few but\ntargeted changes to the input. Similar effects on Text Simplification further\nsupport our claims.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11283v2",
            "date": "2018-04-30 15:53:05+00:00",
            "title": "Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies",
            "authors": [
                "Max Grusky",
                "Mor Naaman",
                "Yoav Artzi"
            ],
            "abstract": "We present NEWSROOM, a summarization dataset of 1.3 million articles and\nsummaries written by authors and editors in newsrooms of 38 major news\npublications. Extracted from search and social media metadata between 1998 and\n2017, these high-quality summaries demonstrate high diversity of summarization\nstyles. In particular, the summaries combine abstractive and extractive\nstrategies, borrowing words and phrases from articles at varying rates. We\nanalyze the extraction strategies used in NEWSROOM summaries against other\ndatasets to quantify the diversity and difficulty of our new data, and train\nexisting methods on the data to evaluate its utility and challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11324v1",
            "date": "2018-04-30 17:03:07+00:00",
            "title": "Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment",
            "authors": [
                "Gonzalo Iglesias",
                "William Tambellini",
                "Adri\u00e0 De Gispert",
                "Eva Hasler",
                "Bill Byrne"
            ],
            "abstract": "We describe a batched beam decoding algorithm for NMT with LMBR n-gram\nposteriors, showing that LMBR techniques still yield gains on top of the best\nrecently reported results with Transformers. We also discuss acceleration\nstrategies for deployment, and the effect of the beam size and batching on\nmemory and speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11346v1",
            "date": "2018-04-30 17:52:28+00:00",
            "title": "A Portuguese Native Language Identification Dataset",
            "authors": [
                "Iria del R\u00edo",
                "Marcos Zampieri",
                "Shervin Malmasi"
            ],
            "abstract": "In this paper we present NLI-PT, the first Portuguese dataset compiled for\nNative Language Identification (NLI), the task of identifying an author's first\nlanguage based on their second language writing. The dataset includes 1,868\nstudent essays written by learners of European Portuguese, native speakers of\nthe following L1s: Chinese, English, Spanish, German, Russian, French,\nJapanese, Italian, Dutch, Tetum, Arabic, Polish, Korean, Romanian, and Swedish.\nNLI-PT includes the original student text and four different types of\nannotation: POS, fine-grained POS, constituency parses, and dependency parses.\nNLI-PT can be used not only in NLI but also in research on several topics in\nthe field of Second Language Acquisition and educational NLP. We discuss\npossible applications of this dataset and present the results obtained for the\nfirst lexical baseline system for Portuguese NLI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00054v2",
            "date": "2018-07-31 20:02:08+00:00",
            "title": "Modeling Task Effects in Human Reading with Neural Attention",
            "authors": [
                "Michael Hahn",
                "Frank Keller"
            ],
            "abstract": "Humans read by making a sequence of fixations and saccades. They often skip\nwords, without apparent detriment to understanding. We offer a novel\nexplanation for skipping: readers optimize a tradeoff between performing a\nlanguage-related task and fixating as few words as possible. We propose a\nneural architecture that combines an attention module (deciding whether to skip\nwords) and a task module (memorizing the input). We show that our model\npredicts human skipping behavior, while also modeling reading times well, even\nthough it skips 40% of the input. A key prediction of our model is that\ndifferent reading tasks should result in different skipping behaviors. We\nconfirm this prediction in an eye-tracking experiment in which participants\nanswers questions about a text. We are able to capture these experimental\nresults using the our model, replacing the memorization module with a task\nmodule that performs neural question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00179v1",
            "date": "2018-08-01 06:12:49+00:00",
            "title": "Monolingual and Cross-lingual Zero-shot Style Transfer",
            "authors": [
                "Elizaveta Korotkova",
                "Maksym Del",
                "Mark Fishel"
            ],
            "abstract": "We introduce the task of zero-shot style transfer between different\nlanguages. Our training data includes multilingual parallel corpora, but does\nnot contain any parallel sentences between styles, similarly to the recent\nprevious work. We propose a unified multilingual multi-style machine\ntranslation system design, that allows to perform zero-shot style conversions\nduring inference; moreover, it does so both monolingually and cross-lingually.\nOur model allows to increase the presence of dissimilar styles in corpus by up\nto 3 times, easily learns to operate with various contractions, and provides\nreasonable lexicon swaps as we see from manual evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00491v1",
            "date": "2018-08-01 18:17:05+00:00",
            "title": "Low-Latency Neural Speech Translation",
            "authors": [
                "Jan Niehues",
                "Ngoc-Quan Pham",
                "Thanh-Le Ha",
                "Matthias Sperber",
                "Alex Waibel"
            ],
            "abstract": "Through the development of neural machine translation, the quality of machine\ntranslation systems has been improved significantly. By exploiting advancements\nin deep learning, systems are now able to better approximate the complex\nmapping from source sentences to target sentences. But with this ability, new\nchallenges also arise. An example is the translation of partial sentences in\nlow-latency speech translation. Since the model has only seen complete\nsentences in training, it will always try to generate a complete sentence,\nthough the input may only be a partial sentence. We show that NMT systems can\nbe adapted to scenarios where no task-specific training data is available.\nFurthermore, this is possible without losing performance on the original\ntraining data. We achieve this by creating artificial data and by using\nmulti-task learning. After adaptation, we are able to reduce the number of\ncorrections displayed during incremental output construction by 45%, without a\ndecrease in translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00521v1",
            "date": "2018-07-28 15:16:33+00:00",
            "title": "Code-Switching Detection with Data-Augmented Acoustic and Language Models",
            "authors": [
                "Emre Y\u0131lmaz",
                "Henk van den Heuvel",
                "David A. van Leeuwen"
            ],
            "abstract": "In this paper, we investigate the code-switching detection performance of a\ncode-switching (CS) automatic speech recognition (ASR) system with\ndata-augmented acoustic and language models. We focus on the recognition of\nFrisian-Dutch radio broadcasts where one of the mixed languages, namely\nFrisian, is under-resourced. Recently, we have explored how the acoustic\nmodeling (AM) can benefit from monolingual speech data belonging to the\nhigh-resourced mixed language. For this purpose, we have trained\nstate-of-the-art AMs on a significantly increased amount of CS speech by\napplying automatic transcription and monolingual Dutch speech. Moreover, we\nhave improved the language model (LM) by creating CS text in various ways\nincluding text generation using recurrent LMs trained on existing CS text.\nMotivated by the significantly improved CS ASR performance, we delve into the\nCS detection performance of the same ASR system in this work by reporting CS\ndetection accuracies together with a detailed detection error analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00639v1",
            "date": "2018-08-02 02:26:53+00:00",
            "title": "Sequence Discriminative Training for Deep Learning based Acoustic Keyword Spotting",
            "authors": [
                "Zhehuai Chen",
                "Yanmin Qian",
                "Kai Yu"
            ],
            "abstract": "Speech recognition is a sequence prediction problem. Besides employing\nvarious deep learning approaches for framelevel classification, sequence-level\ndiscriminative training has been proved to be indispensable to achieve the\nstate-of-the-art performance in large vocabulary continuous speech recognition\n(LVCSR). However, keyword spotting (KWS), as one of the most common speech\nrecognition tasks, almost only benefits from frame-level deep learning due to\nthe difficulty of getting competing sequence hypotheses. The few studies on\nsequence discriminative training for KWS are limited for fixed vocabulary or\nLVCSR based methods and have not been compared to the state-of-the-art deep\nlearning based KWS approaches. In this paper, a sequence discriminative\ntraining framework is proposed for both fixed vocabulary and unrestricted\nacoustic KWS. Sequence discriminative training for both sequence-level\ngenerative and discriminative models are systematically investigated. By\nintroducing word-independent phone lattices or non-keyword blank symbols to\nconstruct competing hypotheses, feasible and efficient sequence discriminative\ntraining approaches are proposed for acoustic KWS. Experiments showed that the\nproposed approaches obtained consistent and significant improvement in both\nfixed vocabulary and unrestricted KWS tasks, compared to previous frame-level\ndeep learning based acoustic KWS methods.",
            "journal ref": "Speech Communication, vol. 102, 100-111, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00687v1",
            "date": "2018-08-02 06:47:23+00:00",
            "title": "Linguistic Search Optimization for Deep Learning Based LVCSR",
            "authors": [
                "Zhehuai Chen"
            ],
            "abstract": "Recent advances in deep learning based large vocabulary con- tinuous speech\nrecognition (LVCSR) invoke growing demands in large scale speech transcription.\nThe inference process of a speech recognizer is to find a sequence of labels\nwhose corresponding acoustic and language models best match the input feature\n[1]. The main computation includes two stages: acoustic model (AM) inference\nand linguistic search (weighted finite-state transducer, WFST). Large\ncomputational overheads of both stages hamper the wide application of LVCSR.\nBenefit from stronger classifiers, deep learning, and more powerful computing\ndevices, we propose general ideas and some initial trials to solve these\nfundamental problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00694v1",
            "date": "2018-08-02 07:18:55+00:00",
            "title": "OntoSenseNet: A Verb-Centric Ontological Resource for Indian Languages",
            "authors": [
                "Jyoti Jha",
                "Sreekavitha Parupalli",
                "Navjyoti Singh"
            ],
            "abstract": "Following approaches for understanding lexical meaning developed by Yaska,\nPatanjali and Bhartrihari from Indian linguistic traditions and extending\napproaches developed by Leibniz and Brentano in the modern times, a framework\nof formal ontology of language was developed. This framework proposes that\nmeaning of words are in-formed by intrinsic and extrinsic ontological\nstructures. The paper aims to capture such intrinsic and extrinsic meanings of\nwords for two major Indian languages, namely, Hindi and Telugu. Parts-of-speech\nhave been rendered into sense-types and sense-classes. Using them we have\ndeveloped a gold- standard annotated lexical resource to support semantic\nunderstanding of a language. The resource has collection of Hindi and Telugu\nlexicons, which has been manually annotated by native speakers of the languages\nfollowing our annotation guidelines. Further, the resource was utilised to\nderive adverbial sense-class distribution of verbs and karaka-verb sense- type\ndistribution. Different corpora (news, novels) were compared using verb\nsense-types distribution. Word Embedding was used as an aid for the enrichment\nof the resource. This is a work in progress that aims at lexical coverage of\nlanguage extensively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00926v1",
            "date": "2018-08-02 17:22:06+00:00",
            "title": "Cyberbullying Detection -- Technical Report 2/2018, Department of Computer Science AGH, University of Science and Technology",
            "authors": [
                "Micha\u0142 Ptaszy\u0144ski",
                "Gniewosz Leliwa",
                "Mateusz Piech",
                "Aleksander Smywi\u0144ski-Pohl"
            ],
            "abstract": "The research described in this paper concerns automatic cyberbullying\ndetection in social media. There are two goals to achieve: building a gold\nstandard cyberbullying detection dataset and measuring the performance of the\nSamurai cyberbullying detection system. The Formspring dataset provided in a\nKaggle competition was re-annotated as a part of the research. The annotation\nprocedure is described in detail and, unlike many other recent data annotation\ninitiatives, does not use Mechanical Turk for finding people willing to perform\nthe annotation. The new annotation compared to the old one seems to be more\ncoherent since all tested cyberbullying detection system performed better on\nthe former. The performance of the Samurai system is compared with 5 commercial\nsystems and one well-known machine learning algorithm, used for classifying\ntextual content, namely Fasttext. It turns out that Samurai scores the best in\nall measures (accuracy, precision and recall), while Fasttext is the\nsecond-best performing algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01160v1",
            "date": "2018-08-03 11:31:26+00:00",
            "title": "Efficient Purely Convolutional Text Encoding",
            "authors": [
                "Szymon Malik",
                "Adrian Lancucki",
                "Jan Chorowski"
            ],
            "abstract": "In this work, we focus on a lightweight convolutional architecture that\ncreates fixed-size vector embeddings of sentences. Such representations are\nuseful for building NLP systems, including conversational agents. Our work\nderives from a recently proposed recursive convolutional architecture for\nauto-encoding text paragraphs at byte level. We propose alternations that\nsignificantly reduce training time, the number of parameters, and improve\nauto-encoding accuracy. Finally, we evaluate the representations created by our\nmodel on tasks from SentEval benchmark suite, and show that it can serve as a\nbetter, yet fairly low-resource alternative to popular bag-of-words embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01216v2",
            "date": "2018-08-03 14:58:55+00:00",
            "title": "A Multi-task Ensemble Framework for Emotion, Sentiment and Intensity Prediction",
            "authors": [
                "Md Shad Akhtar",
                "Deepanway Ghosal",
                "Asif Ekbal",
                "Pushpak Bhattacharyya",
                "Sadao Kurohashi"
            ],
            "abstract": "In this paper, through multi-task ensemble framework we address three\nproblems of emotion and sentiment analysis i.e. \"emotion classification &\nintensity\", \"valence, arousal & dominance for emotion\" and \"valence & arousal}\nfor sentiment\". The underlying problems cover two granularities (i.e.\ncoarse-grained and fine-grained) and a diverse range of domains (i.e. tweets,\nFacebook posts, news headlines, blogs, letters etc.). The ensemble model aims\nto leverage the learned representations of three deep learning models (i.e.\nCNN, LSTM and GRU) and a hand-crafted feature representation for the\npredictions. Experimental results on the benchmark datasets show the efficacy\nof our proposed multi-task ensemble frameworks. We obtain the performance\nimprovement of 2-3 points on an average over single-task systems for most of\nthe problems and domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01426v1",
            "date": "2018-08-04 05:03:43+00:00",
            "title": "Abstractive Summarization Improved by WordNet-based Extractive Sentences",
            "authors": [
                "Niantao Xie",
                "Sujian Li",
                "Huiling Ren",
                "Qibin Zhai"
            ],
            "abstract": "Recently, the seq2seq abstractive summarization models have achieved good\nresults on the CNN/Daily Mail dataset. Still, how to improve abstractive\nmethods with extractive methods is a good research direction, since extractive\nmethods have their potentials of exploiting various efficient features for\nextracting important sentences in one text. In this paper, in order to improve\nthe semantic relevance of abstractive summaries, we adopt the WordNet based\nsentence ranking algorithm to extract the sentences which are most semantically\nto one text. Then, we design a dual attentional seq2seq framework to generate\nsummaries with consideration of the extracted information. At the same time, we\ncombine pointer-generator and coverage mechanisms to solve the problems of\nout-of-vocabulary (OOV) words and duplicate words which exist in the\nabstractive models. Experiments on the CNN/Daily Mail dataset show that our\nmodels achieve competitive performance with the state-of-the-art ROUGE scores.\nHuman evaluations also show that the summaries generated by our models have\nhigh semantic relevance to the original text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01662v1",
            "date": "2018-08-05 17:47:18+00:00",
            "title": "Instantiation",
            "authors": [
                "Abhijeet Gupta",
                "Gemma Boleda",
                "Sebastian Pado"
            ],
            "abstract": "In computational linguistics, a large body of work exists on distributed\nmodeling of lexical relations, focussing largely on lexical relations such as\nhypernymy (scientist -- person) that hold between two categories, as expressed\nby common nouns. In contrast, computational linguistics has paid little\nattention to entities denoted by proper nouns (Marie Curie, Mumbai, ...). These\nhave investigated in detail by the Knowledge Representation and Semantic Web\ncommunities, but generally not with regard to their linguistic properties.\n  Our paper closes this gap by investigating and modeling the lexical relation\nof instantiation, which holds between an entity-denoting and a\ncategory-denoting expression (Marie Curie -- scientist or Mumbai -- city). We\npresent a new, principled dataset for the task of instantiation detection as\nwell as experiments and analyses on this dataset. We obtain the following\nresults: (a), entities belonging to one category form a region in\ndistributional space, but the embedding for the category word is typically\nlocated outside this subspace; (b) it is easy to learn to distinguish entities\nfrom categories from distributional evidence, but due to (a), instantiation\nproper is much harder to learn when using common nouns as representations of\ncategories; (c) this problem can be alleviated by using category\nrepresentations based on entity rather than category word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02082v1",
            "date": "2018-08-03 02:39:38+00:00",
            "title": "Did you take the pill? - Detecting Personal Intake of Medicine from Twitter",
            "authors": [
                "Debanjan Mahata",
                "Jasper Friedrichs",
                "Rajiv Ratn Shah",
                "Jing Jiang"
            ],
            "abstract": "Mining social media messages such as tweets, articles, and Facebook posts for\nhealth and drug related information has received significant interest in\npharmacovigilance research. Social media sites (e.g., Twitter), have been used\nfor monitoring drug abuse, adverse reactions of drug usage and analyzing\nexpression of sentiments related to drugs. Most of these studies are based on\naggregated results from a large population rather than specific sets of\nindividuals. In order to conduct studies at an individual level or specific\ncohorts, identifying posts mentioning intake of medicine by the user is\nnecessary. Towards this objective we develop a classifier for identifying\nmentions of personal intake of medicine in tweets. We train a stacked ensemble\nof shallow convolutional neural network (CNN) models on an annotated dataset.\nWe use random search for tuning the hyper-parameters of the CNN models and\npresent an ensemble of best models for the prediction task. Our system produces\nstate-of-the-art result, with a micro-averaged F-score of 0.693. We believe\nthat the developed classifier has direct uses in the areas of psychology,\nhealth informatics, pharmacovigilance and affective computing for tracking\nmoods, emotions and sentiments of patients expressing intake of medicine in\nsocial media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02171v1",
            "date": "2018-08-07 01:04:39+00:00",
            "title": "Dialog-context aware end-to-end speech recognition",
            "authors": [
                "Suyoun Kim",
                "Florian Metze"
            ],
            "abstract": "Existing speech recognition systems are typically built at the sentence\nlevel, although it is known that dialog context, e.g. higher-level knowledge\nthat spans across sentences or speakers, can help the processing of long\nconversations. The recent progress in end-to-end speech recognition systems\npromises to integrate all available information (e.g. acoustic, language\nresources) into a single model, which is then jointly optimized. It seems\nnatural that such dialog context information should thus also be integrated\ninto the end-to-end models to improve further recognition accuracy. In this\nwork, we present a dialog-context aware speech recognition model, which\nexplicitly uses context information beyond sentence-level information, in an\nend-to-end fashion. Our dialog-context model captures a history of\nsentence-level context so that the whole system can be trained with\ndialog-context information in an end-to-end manner. We evaluate our proposed\napproach on the Switchboard conversational speech corpus and show that our\nsystem outperforms a comparable sentence-level end-to-end speech recognition\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02228v1",
            "date": "2018-08-07 06:47:50+00:00",
            "title": "Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detection",
            "authors": [
                "Yu-Hsuan Wang",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "While Word2Vec represents words (in text) as vectors carrying semantic\ninformation, audio Word2Vec was shown to be able to represent signal segments\nof spoken words as vectors carrying phonetic structure information. Audio\nWord2Vec can be trained in an unsupervised way from an unlabeled corpus, except\nthe word boundaries are needed. In this paper, we extend audio Word2Vec from\nword-level to utterance-level by proposing a new segmental audio Word2Vec, in\nwhich unsupervised spoken word boundary segmentation and audio Word2Vec are\njointly learned and mutually enhanced, so an utterance can be directly\nrepresented as a sequence of vectors carrying phonetic structure information.\nThis is achieved by a segmental sequence-to-sequence autoencoder (SSAE), in\nwhich a segmentation gate trained with reinforcement learning is inserted in\nthe encoder. Experiments on English, Czech, French and German show very good\nperformance in both unsupervised spoken word segmentation and spoken term\ndetection applications (significantly better than frame-based DTW).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02280v1",
            "date": "2018-08-07 09:47:00+00:00",
            "title": "ODSQA: Open-domain Spoken Question Answering Dataset",
            "authors": [
                "Chia-Hsuan Lee",
                "Shang-Ming Wang",
                "Huan-Cheng Chang",
                "Hung-Yi Lee"
            ],
            "abstract": "Reading comprehension by machine has been widely studied, but machine\ncomprehension of spoken content is still a less investigated problem. In this\npaper, we release Open-Domain Spoken Question Answering Dataset (ODSQA) with\nmore than three thousand questions. To the best of our knowledge, this is the\nlargest real SQA dataset. On this dataset, we found that ASR errors have\ncatastrophic impact on SQA. To mitigate the effect of ASR errors, subword units\nare involved, which brings consistent improvements over all the models. We\nfurther found that data augmentation on text-based QA training examples can\nimprove SQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02374v1",
            "date": "2018-08-07 14:00:11+00:00",
            "title": "Word-Level Loss Extensions for Neural Temporal Relation Classification",
            "authors": [
                "Artuur Leeuwenberg",
                "Marie-Francine Moens"
            ],
            "abstract": "Unsupervised pre-trained word embeddings are used effectively for many tasks\nin natural language processing to leverage unlabeled textual data. Often these\nembeddings are either used as initializations or as fixed word representations\nfor task-specific classification models. In this work, we extend our\nclassification model's task loss with an unsupervised auxiliary loss on the\nword-embedding level of the model. This is to ensure that the learned word\nrepresentations contain both task-specific features, learned from the\nsupervised loss component, and more general features learned from the\nunsupervised loss component. We evaluate our approach on the task of temporal\nrelation extraction, in particular, narrative containment relation extraction\nfrom clinical records, and show that continued training of the embeddings on\nthe unsupervised objective together with the task objective gives better\ntask-specific embeddings, and results in an improvement over the state of the\nart on the THYME dataset, using only a general-domain part-of-speech tagger as\nlinguistic resource.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02563v1",
            "date": "2018-08-07 22:01:37+00:00",
            "title": "Design Challenges in Named Entity Transliteration",
            "authors": [
                "Yuval Merhav",
                "Stephen Ash"
            ],
            "abstract": "We analyze some of the fundamental design challenges that impact the\ndevelopment of a multilingual state-of-the-art named entity transliteration\nsystem, including curating bilingual named entity datasets and evaluation of\nmultiple transliteration methods. We empirically evaluate the transliteration\ntask using traditional weighted finite state transducer (WFST) approach against\ntwo neural approaches: the encoder-decoder recurrent neural network method and\nthe recent, non-sequential Transformer method. In order to improve availability\nof bilingual named entity transliteration datasets, we release personal name\nbilingual dictionaries minded from Wikidata for English to Russian, Hebrew,\nArabic and Japanese Katakana. Our code and dictionaries are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02586v1",
            "date": "2018-08-08 00:02:18+00:00",
            "title": "Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems",
            "authors": [
                "Van-Khanh Tran",
                "Le-Minh Nguyen"
            ],
            "abstract": "Domain Adaptation arises when we aim at learning from source domain a model\nthat can per- form acceptably well on a different target domain. It is\nespecially crucial for Natural Language Generation (NLG) in Spoken Dialogue\nSystems when there are sufficient annotated data in the source domain, but\nthere is a limited labeled data in the target domain. How to effectively\nutilize as much of existing abilities from source domains is a crucial issue in\ndomain adaptation. In this paper, we propose an adversarial training procedure\nto train a Variational encoder-decoder based language generator via multiple\nadaptation steps. In this procedure, a model is first trained on a source\ndomain data and then fine-tuned on a small set of target domain utterances\nunder the guidance of two proposed critics. Experimental results show that the\nproposed method can effec- tively leverage the existing knowledge in the source\ndomain to adapt to another related domain by using only a small amount of\nin-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02622v1",
            "date": "2018-08-08 04:49:41+00:00",
            "title": "Learning to Write Notes in Electronic Health Records",
            "authors": [
                "Peter J. Liu"
            ],
            "abstract": "Clinicians spend a significant amount of time inputting free-form textual\nnotes into Electronic Health Records (EHR) systems. Much of this documentation\nwork is seen as a burden, reducing time spent with patients and contributing to\nclinician burnout. With the aspiration of AI-assisted note-writing, we propose\na new language modeling task predicting the content of notes conditioned on\npast data from a patient's medical record, including patient demographics,\nlabs, medications, and past notes. We train generative models using the public,\nde-identified MIMIC-III dataset and compare generated notes with those in the\ndataset on multiple measures. We find that much of the content can be\npredicted, and that many common templates found in notes can be learned. We\ndiscuss how such models can be useful in supporting assistive note-writing\nfeatures such as error-detection and auto-complete.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02724v1",
            "date": "2018-08-08 11:06:15+00:00",
            "title": "Learning to Focus when Ranking Answers",
            "authors": [
                "Dana Sagi",
                "Tzoof Avny",
                "Kira Radinsky",
                "Eugene Agichtein"
            ],
            "abstract": "One of the main challenges in ranking is embedding the query and document\npairs into a joint feature space, which can then be fed to a learning-to-rank\nalgorithm. To achieve this representation, the conventional state of the art\napproaches perform extensive feature engineering that encode the similarity of\nthe query-answer pair. Recently, deep-learning solutions have shown that it is\npossible to achieve comparable performance, in some settings, by learning the\nsimilarity representation directly from data. Unfortunately, previous models\nperform poorly on longer texts, or on texts with significant portion of\nirrelevant information, or which are grammatically incorrect. To overcome these\nlimitations, we propose a novel ranking algorithm for question answering,\nQARAT, which uses an attention mechanism to learn on which words and phrases to\nfocus when building the mutual representation. We demonstrate superior ranking\nperformance on several real-world question-answer ranking datasets, and provide\nvisualization of the attention mechanism to otter more insights into how our\nmodels of attention could benefit ranking for difficult question answering\nchallenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02733v1",
            "date": "2018-08-08 11:55:36+00:00",
            "title": "Debugging Neural Machine Translations",
            "authors": [
                "Mat\u012bss Rikters"
            ],
            "abstract": "In this paper, we describe a tool for debugging the output and attention\nweights of neural machine translation (NMT) systems and for improved\nestimations of confidence about the output based on the attention. The purpose\nof the tool is to help researchers and developers find weak and faulty example\ntranslations that their NMT systems produce without the need for reference\ntranslations. Our tool also includes an option to directly compare translation\noutputs from two different NMT engines or experiments. In addition, we present\na demo website of our tool with examples of good and bad translations:\nhttp://attention.lielakeda.lv",
            "journal ref": "Baltic DB&IS 2018 Joint Proceedings of the Conference Forum,\n  Trakai, Lithuania, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02747v2",
            "date": "2018-08-08 13:12:10+00:00",
            "title": "Natural Language Generation by Hierarchical Decoding with Linguistic Patterns",
            "authors": [
                "Shang-Yu Su",
                "Kai-Ling Lo",
                "Yi-Ting Yeh",
                "Yun-Nung Chen"
            ],
            "abstract": "Natural language generation (NLG) is a critical component in spoken dialogue\nsystems. Classic NLG can be divided into two phases: (1) sentence planning:\ndeciding on the overall sentence structure, (2) surface realization:\ndetermining specific word forms and flattening the sentence structure into a\nstring. Many simple NLG models are based on recurrent neural networks (RNN) and\nsequence-to-sequence (seq2seq) model, which basically contains an\nencoder-decoder structure; these NLG models generate sentences from scratch by\njointly optimizing sentence planning and surface realization using a simple\ncross entropy loss training criterion. However, the simple encoder-decoder\narchitecture usually suffers from generating complex and long sentences,\nbecause the decoder has to learn all grammar and diction knowledge. This paper\nintroduces a hierarchical decoding NLG model based on linguistic patterns in\ndifferent levels, and shows that the proposed method outperforms the\ntraditional one with a smaller model size. Furthermore, the design of the\nhierarchical decoding is flexible and easily-extensible in various NLG systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02772v2",
            "date": "2018-08-07 15:37:20+00:00",
            "title": "Effective Character-augmented Word Embedding for Machine Reading Comprehension",
            "authors": [
                "Zhuosheng Zhang",
                "Yafang Huang",
                "Pengfei Zhu",
                "Hai Zhao"
            ],
            "abstract": "Machine reading comprehension is a task to model relationship between passage\nand query. In terms of deep learning framework, most of state-of-the-art models\nsimply concatenate word and character level representations, which has been\nshown suboptimal for the concerned task. In this paper, we empirically explore\ndifferent integration strategies of word and character embeddings and propose a\ncharacter-augmented reader which attends character-level representation to\naugment word embedding with a short list to improve word representations,\nespecially for rare words. Experimental results show that the proposed approach\nhelps the baseline model significantly outperform state-of-the-art baselines on\nvarious public benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02831v1",
            "date": "2018-08-08 15:45:06+00:00",
            "title": "Debunking Fake News One Feature at a Time",
            "authors": [
                "Melanie Tosik",
                "Antonio Mallia",
                "Kedar Gangopadhyay"
            ],
            "abstract": "Identifying the stance of a news article body with respect to a certain\nheadline is the first step to automated fake news detection. In this paper, we\nintroduce a 2-stage ensemble model to solve the stance detection task. By using\nonly hand-crafted features as input to a gradient boosting classifier, we are\nable to achieve a score of 9161.5 out of 11651.25 (78.63%) on the official Fake\nNews Challenge (Stage 1) dataset. We identify the most useful features for\ndetecting fake news and discuss how sampling techniques can be used to improve\nrecall accuracy on a highly imbalanced dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02961v1",
            "date": "2018-08-08 23:06:12+00:00",
            "title": "Exploiting Effective Representations for Chinese Sentiment Analysis Using a Multi-Channel Convolutional Neural Network",
            "authors": [
                "Pengfei Liu",
                "Ji Zhang",
                "Cane Wing-Ki Leung",
                "Chao He",
                "Thomas L. Griffiths"
            ],
            "abstract": "Effective representation of a text is critical for various natural language\nprocessing tasks. For the particular task of Chinese sentiment analysis, it is\nimportant to understand and choose an effective representation of a text from\ndifferent forms of Chinese representations such as word, character and pinyin.\nThis paper presents a systematic study of the effect of these representations\nfor Chinese sentiment analysis by proposing a multi-channel convolutional\nneural network (MCCNN), where each channel corresponds to a representation.\nExperimental results show that: (1) Word wins on the dataset of low OOV rate\nwhile character wins otherwise; (2) Using these representations in combination\ngenerally improves the performance; (3) The representations based on MCCNN\noutperform conventional ngram features using SVM; (4) The proposed MCCNN model\nachieves the competitive performance against the state-of-the-art model\nfastText for Chinese sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03028v1",
            "date": "2018-08-09 05:57:13+00:00",
            "title": "Arithmetic Word Problem Solver using Frame Identification",
            "authors": [
                "Pruthwik Mishra",
                "Litton J Kurisinkel",
                "Dipti Misra Sharma"
            ],
            "abstract": "Automatic Word problem solving has always posed a great challenge for the NLP\ncommunity. Usually a word problem is a narrative comprising of a few sentences\nand a question is asked about a quantity referred in the sentences. Solving\nword problem involves reasoning across sentences, identification of operations,\ntheir order, relevant quantities and discarding irrelevant quantities. In this\npaper, we present a novel approach for automatic arithmetic word problem\nsolving. Our approach starts with frame identification. Each frame can either\nbe classified as a state or an action frame. The frame identification is\ndependent on the verb in a sentence. Every frame is unique and is identified by\nits slots. The slots are filled using dependency parsed output of a sentence.\nThe slots are entity holder, entity, quantity of the entity, recipient,\nadditional information like place, time. The slots and frames helps to identify\nthe type of question asked and the entity referred. Action frames act on state\nframe(s) which causes a change in quantities of the state frames. The frames\nare then used to build a graph where any change in quantities can be propagated\nto the neighboring nodes. Most of the current solvers can only answer questions\nrelated to the quantity, while our system can answer different kinds of\nquestions like `who', `what' other than the quantity related questions `how\nmany'.\n  There are three major contributions of this paper. 1. Frame Annotated Corpus\n(with a frame annotation tool) 2. Frame Identification Module 3. A new easily\nunderstandable Framework for word problem solving",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03137v3",
            "date": "2018-08-09 13:12:07+00:00",
            "title": "A Survey on Sentiment and Emotion Analysis for Computational Literary Studies",
            "authors": [
                "Evgeny Kim",
                "Roman Klinger"
            ],
            "abstract": "Emotions are a crucial part of compelling narratives: literature tells us\nabout people with goals, desires, passions, and intentions. Emotion analysis is\npart of the broader and larger field of sentiment analysis, and receives\nincreasing attention in literary studies. In the past, the affective dimension\nof literature was mainly studied in the context of literary hermeneutics.\nHowever, with the emergence of the research field known as Digital Humanities\n(DH), some studies of emotions in a literary context have taken a computational\nturn. Given the fact that DH is still being formed as a field, this direction\nof research can be rendered relatively new. In this survey, we offer an\noverview of the existing body of research on emotion analysis as applied to\nliterature. The research under review deals with a variety of topics including\ntracking dramatic changes of a plot development, network analysis of a literary\ntext, and understanding the emotionality of texts, among other topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03175v1",
            "date": "2018-08-09 14:16:30+00:00",
            "title": "Building a Kannada POS Tagger Using Machine Learning and Neural Network Models",
            "authors": [
                "Ketan Kumar Todi",
                "Pruthwik Mishra",
                "Dipti Misra Sharma"
            ],
            "abstract": "POS Tagging serves as a preliminary task for many NLP applications. Kannada\nis a relatively poor Indian language with very limited number of quality NLP\ntools available for use. An accurate and reliable POS Tagger is essential for\nmany NLP tasks like shallow parsing, dependency parsing, sentiment analysis,\nnamed entity recognition. We present a statistical POS tagger for Kannada using\ndifferent machine learning and neural network models. Our Kannada POS tagger\noutperforms the state-of-the-art Kannada POS tagger by 6%. Our contribution in\nthis paper is three folds - building a generic POS Tagger, comparing the\nperformances of different modeling techniques, exploring the use of character\nand word embeddings together for Kannada POS Tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03299v1",
            "date": "2018-08-09 18:38:16+00:00",
            "title": "Code-Mixed Sentiment Analysis Using Machine Learning and Neural Network Approaches",
            "authors": [
                "Pruthwik Mishra",
                "Prathyusha Danda",
                "Pranav Dhakras"
            ],
            "abstract": "Sentiment Analysis for Indian Languages (SAIL)-Code Mixed tools contest aimed\nat identifying the sentence level sentiment polarity of the code-mixed dataset\nof Indian languages pairs (Hi-En, Ben-Hi-En). Hi-En dataset is henceforth\nreferred to as HI-EN and Ben-Hi-En dataset as BN-EN respectively. For this, we\nsubmitted four models for sentiment analysis of code-mixed HI-EN and BN-EN\ndatasets. The first model was an ensemble voting classifier consisting of three\nclassifiers - linear SVM, logistic regression and random forests while the\nsecond one was a linear SVM. Both the models used TF-IDF feature vectors of\ncharacter n-grams where n ranged from 2 to 6. We used scikit-learn (sklearn)\nmachine learning library for implementing both the approaches. Run1 was\nobtained from the voting classifier and Run2 used the linear SVM model for\nproducing the results. Out of the four submitted outputs Run2 outperformed Run1\nin both the datasets. We finished first in the contest for both HI-EN with an\nF-score of 0.569 and BN-EN with an F-score of 0.526.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03353v1",
            "date": "2018-08-09 21:44:58+00:00",
            "title": "Efficient human-like semantic representations via the Information Bottleneck principle",
            "authors": [
                "Noga Zaslavsky",
                "Charles Kemp",
                "Terry Regier",
                "Naftali Tishby"
            ],
            "abstract": "Maintaining efficient semantic representations of the environment is a major\nchallenge both for humans and for machines. While human languages represent\nuseful solutions to this problem, it is not yet clear what computational\nprinciple could give rise to similar solutions in machines. In this work we\npropose an answer to this open question. We suggest that languages compress\npercepts into words by optimizing the Information Bottleneck (IB) tradeoff\nbetween the complexity and accuracy of their lexicons. We present empirical\nevidence that this principle may give rise to human-like semantic\nrepresentations, by exploring how human languages categorize colors. We show\nthat color naming systems across languages are near-optimal in the IB sense,\nand that these natural systems are similar to artificial IB color naming\nsystems with a single tradeoff parameter controlling the cross-language\nvariability. In addition, the IB systems evolve through a sequence of\nstructural phase transitions, demonstrating a possible adaptation process. This\nwork thus identifies a computational principle that characterizes human\nsemantic systems, and that could usefully inform semantic representations in\nmachines.",
            "journal ref": "Cognitively Informed Artificial Intelligence Workshop at NIPS 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03430v1",
            "date": "2018-08-10 06:58:33+00:00",
            "title": "Lingke: A Fine-grained Multi-turn Chatbot for Customer Service",
            "authors": [
                "Pengfei Zhu",
                "Zhuosheng Zhang",
                "Jiangtong Li",
                "Yafang Huang",
                "Hai Zhao"
            ],
            "abstract": "Traditional chatbots usually need a mass of human dialogue data, especially\nwhen using supervised machine learning method. Though they can easily deal with\nsingle-turn question answering, for multi-turn the performance is usually\nunsatisfactory. In this paper, we present Lingke, an information retrieval\naugmented chatbot which is able to answer questions based on given product\nintroduction document and deal with multi-turn conversations. We will introduce\na fine-grained pipeline processing to distill responses based on unstructured\ndocuments, and attentive sequential context-response matching for multi-turn\nconversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03437v1",
            "date": "2018-08-10 07:46:48+00:00",
            "title": "Hybrid approach for transliteration of Algerian arabizi: a primary study",
            "authors": [
                "Imane Guellil",
                "Faical Azouaou",
                "Fodil Benali",
                "Ala-Eddine Hachani",
                "Houda Saadane"
            ],
            "abstract": "A hybrid approach for the transliteration of Algerian Arabizi: A primary\nstudy In this paper, we present a hybrid approach for the transliteration of\nthe Algerian Arabizi. We define a set of rules enable us the passage from\nArabizi to Arabic. Through these rules, we generate a set of candidates for the\ntransliteration of each Arabizi word into arabic. Then, we extract the best\ncandidate. This approach was evaluated by using three test corpora, and the\nobtained results show an improvement of the precision score which is equal to\n75.11% for the best result. These results allow us to verify that our approach\nis very competitive comparing to others works that treat Arabizi\ntransliteration in general.\n  Keywords: Arabizi, Dialecte Alg\\'erien, Arabizi Alg\\'erien,\nTranslit\\'eration.",
            "journal ref": "25e conf\\'erence sur le Traitement Automatique des Langues\n  Naturelles (TALN), May 2018, Rennes, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03465v2",
            "date": "2018-08-10 09:27:44+00:00",
            "title": "TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification",
            "authors": [
                "Wenpeng Yin",
                "Dan Roth"
            ],
            "abstract": "Determining whether a given claim is supported by evidence is a fundamental\nNLP problem that is best modeled as Textual Entailment. However, given a large\ncollection of text, finding evidence that could support or refute a given claim\nis a challenge in itself, amplified by the fact that different evidence might\nbe needed to support or refute a claim. Nevertheless, most prior work decouples\nevidence identification from determining the truth value of the claim given the\nevidence.\n  We propose to consider these two aspects jointly. We develop TwoWingOS\n(two-wing optimization strategy), a system that, while identifying appropriate\nevidence for a claim, also determines whether or not the claim is supported by\nthe evidence. Given the claim, TwoWingOS attempts to identify a subset of the\nevidence candidates; given the predicted evidence, it then attempts to\ndetermine the truth value of the corresponding claim. We treat this challenge\nas coupled optimization problems, training a joint model for it. TwoWingOS\noffers two advantages: (i) Unlike pipeline systems, it facilitates\nflexible-size evidence set, and (ii) Joint training improves both the claim\nentailment and the evidence identification. Experiments on a benchmark dataset\nshow state-of-the-art performance. Code: https://github.com/yinwenpeng/FEVER",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03570v1",
            "date": "2018-08-10 14:54:10+00:00",
            "title": "Densely Connected Convolutional Networks for Speech Recognition",
            "authors": [
                "Chia Yu Li",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper presents our latest investigation on Densely Connected\nConvolutional Networks (DenseNets) for acoustic modelling (AM) in automatic\nspeech recognition. DenseN-ets are very deep, compact convolutional neural\nnetworks, which have demonstrated incredible improvements over the\nstate-of-the-art results on several data sets in computer vision. Our\nexperimental results show that DenseNet can be used for AM significantly\noutperforming other neural-based models such as DNNs, CNNs, VGGs. Furthermore,\nresults on Wall Street Journal revealed that with only a half of the training\ndata DenseNet was able to outperform other models trained with the full data\nset by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03712v3",
            "date": "2018-08-10 21:50:59+00:00",
            "title": "Unsupervised Keyphrase Extraction from Scientific Publications",
            "authors": [
                "Eirini Papagiannopoulou",
                "Grigorios Tsoumakas"
            ],
            "abstract": "We propose a novel unsupervised keyphrase extraction approach that filters\ncandidate keywords using outlier detection. It starts by training word\nembeddings on the target document to capture semantic regularities among the\nwords. It then uses the minimum covariance determinant estimator to model the\ndistribution of non-keyphrase word vectors, under the assumption that these\nvectors come from the same distribution, indicative of their irrelevance to the\nsemantics expressed by the dimensions of the learned vector representation.\nCandidate keyphrases only consist of words that are detected as outliers of\nthis dominant distribution. Empirical results show that our approach\noutperforms state-of-the-art and recent unsupervised keyphrase extraction\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03728v1",
            "date": "2018-08-10 23:28:33+00:00",
            "title": "Hierarchical Attention: What Really Counts in Various NLP Tasks",
            "authors": [
                "Zehao Dou",
                "Zhihua Zhang"
            ],
            "abstract": "Attention mechanisms in sequence to sequence models have shown great ability\nand wonderful performance in various natural language processing (NLP) tasks,\nsuch as sentence embedding, text generation, machine translation, machine\nreading comprehension, etc. Unfortunately, existing attention mechanisms only\nlearn either high-level or low-level features. In this paper, we think that the\nlack of hierarchical mechanisms is a bottleneck in improving the performance of\nthe attention mechanisms, and propose a novel Hierarchical Attention Mechanism\n(Ham) based on the weighted sum of different layers of a multi-level attention.\nHam achieves a state-of-the-art BLEU score of 0.26 on Chinese poem generation\ntask and a nearly 6.5% averaged improvement compared with the existing machine\nreading comprehension models such as BIDAF and Match-LSTM. Furthermore, our\nexperiments and theorems reveal that Ham has greater generalization and\nrepresentation ability than existing attention mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03731v2",
            "date": "2018-08-11 01:03:45+00:00",
            "title": "From POS tagging to dependency parsing for biomedical event extraction",
            "authors": [
                "Dat Quoc Nguyen",
                "Karin Verspoor"
            ],
            "abstract": "Background: Given the importance of relation or event extraction from\nbiomedical research publications to support knowledge capture and synthesis,\nand the strong dependency of approaches to this information extraction task on\nsyntactic information, it is valuable to understand which approaches to\nsyntactic processing of biomedical text have the highest performance. Results:\nWe perform an empirical study comparing state-of-the-art traditional\nfeature-based and neural network-based models for two core natural language\nprocessing tasks of part-of-speech (POS) tagging and dependency parsing on two\nbenchmark biomedical corpora, GENIA and CRAFT. To the best of our knowledge,\nthere is no recent work making such comparisons in the biomedical context;\nspecifically no detailed analysis of neural models on this data is available.\nExperimental results show that in general, the neural models outperform the\nfeature-based models on two benchmark biomedical corpora GENIA and CRAFT. We\nalso perform a task-oriented evaluation to investigate the influences of these\nmodels in a downstream application on biomedical event extraction, and show\nthat better intrinsic parsing performance does not always imply better\nextrinsic event extraction performance. Conclusion: We have presented a\ndetailed empirical study comparing traditional feature-based and neural\nnetwork-based models for POS tagging and dependency parsing in the biomedical\ncontext, and also investigated the influence of parser selection for a\nbiomedical event extraction downstream task. Availability of data and material:\nWe make the retrained models available at\nhttps://github.com/datquocnguyen/BioPosDep",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03738v2",
            "date": "2018-08-11 02:06:25+00:00",
            "title": "Ancient-Modern Chinese Translation with a Large Training Dataset",
            "authors": [
                "Dayiheng Liu",
                "Jiancheng Lv",
                "Kexin Yang",
                "Qian Qu"
            ],
            "abstract": "Ancient Chinese brings the wisdom and spirit culture of the Chinese nation.\nAutomatic translation from ancient Chinese to modern Chinese helps to inherit\nand carry forward the quintessence of the ancients. However, the lack of\nlarge-scale parallel corpus limits the study of machine translation in\nAncient-Modern Chinese. In this paper, we propose an Ancient-Modern Chinese\nclause alignment approach based on the characteristics of these two languages.\nThis method combines both lexical-based information and statistical-based\ninformation, which achieves 94.2 F1-score on our manual annotation Test set. We\nuse this method to create a new large-scale Ancient-Modern Chinese parallel\ncorpus which contains 1.24M bilingual pairs. To our best knowledge, this is the\nfirst large high-quality Ancient-Modern Chinese dataset. Furthermore, we\nanalyzed and compared the performance of the SMT and various NMT models on this\ndataset and provided a strong baseline for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03747v1",
            "date": "2018-08-11 03:50:27+00:00",
            "title": "Dropout during inference as a model for neurological degeneration in an image captioning network",
            "authors": [
                "Bai Li",
                "Ran Zhang",
                "Frank Rudzicz"
            ],
            "abstract": "We replicate a variation of the image captioning architecture by Vinyals et\nal. (2015), then introduce dropout during inference mode to simulate the\neffects of neurodegenerative diseases like Alzheimer's disease (AD) and\nWernicke's aphasia (WA). We evaluate the effects of dropout on language\nproduction by measuring the KL-divergence of word frequency distributions and\nother linguistic metrics as dropout is added. We find that the generated\nsentences most closely approximate the word frequency distribution of the\ntraining corpus when using a moderate dropout of 0.4 during inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03806v1",
            "date": "2018-08-11 13:55:27+00:00",
            "title": "The Impact of Automatic Pre-annotation in Clinical Note Data Element Extraction - the CLEAN Tool",
            "authors": [
                "Tsung-Ting Kuo",
                "Jina Huh",
                "Jihoon Kim",
                "Robert El-Kareh",
                "Siddharth Singh",
                "Stephanie Feudjio Feupe",
                "Vincent Kuri",
                "Gordon Lin",
                "Michele E. Day",
                "Lucila Ohno-Machado",
                "Chun-Nan Hsu"
            ],
            "abstract": "Objective. Annotation is expensive but essential for clinical note review and\nclinical natural language processing (cNLP). However, the extent to which\ncomputer-generated pre-annotation is beneficial to human annotation is still an\nopen question. Our study introduces CLEAN (CLinical note rEview and\nANnotation), a pre-annotation-based cNLP annotation system to improve clinical\nnote annotation of data elements, and comprehensively compares CLEAN with the\nwidely-used annotation system Brat Rapid Annotation Tool (BRAT).\n  Materials and Methods. CLEAN includes an ensemble pipeline (CLEAN-EP) with a\nnewly developed annotation tool (CLEAN-AT). A domain expert and a novice\nuser/annotator participated in a comparative usability test by tagging 87 data\nelements related to Congestive Heart Failure (CHF) and Kawasaki Disease (KD)\ncohorts in 84 public notes.\n  Results. CLEAN achieved higher note-level F1-score (0.896) over BRAT (0.820),\nwith significant difference in correctness (P-value < 0.001), and the mostly\nrelated factor being system/software (P-value < 0.001). No significant\ndifference (P-value 0.188) in annotation time was observed between CLEAN (7.262\nminutes/note) and BRAT (8.286 minutes/note). The difference was mostly\nassociated with note length (P-value < 0.001) and system/software (P-value\n0.013). The expert reported CLEAN to be useful/satisfactory, while the novice\nreported slight improvements.\n  Discussion. CLEAN improves the correctness of annotation and increases\nusefulness/satisfaction with the same level of efficiency. Limitations include\nuntested impact of pre-annotation correctness rate, small sample size, small\nuser size, and restrictedly validated gold standard.\n  Conclusion. CLEAN with pre-annotation can be beneficial for an expert to deal\nwith complex annotation tasks involving numerous and diverse target data\nelements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03815v2",
            "date": "2018-08-11 14:59:07+00:00",
            "title": "A Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?",
            "authors": [
                "Jiaxun Cai",
                "Shexia He",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "Semantic role labeling (SRL) is to recognize the predicate-argument structure\nof a sentence, including subtasks of predicate disambiguation and argument\nlabeling. Previous studies usually formulate the entire SRL problem into two or\nmore subtasks. For the first time, this paper introduces an end-to-end neural\nmodel which unifiedly tackles the predicate disambiguation and the argument\nlabeling in one shot. Using a biaffine scorer, our model directly predicts all\nsemantic role labels for all given word pairs in the sentence without relying\non any syntactic parse information. Specifically, we augment the BiLSTM encoder\nwith a non-linear transformation to further distinguish the predicate and the\nargument in a given sentence, and model the semantic role labeling process as a\nword pair classification task by employing the biaffine attentional mechanism.\nThough the proposed model is syntax-agnostic with local decoder, it outperforms\nthe state-of-the-art syntax-aware SRL systems on the CoNLL-2008, 2009\nbenchmarks for both English and Chinese. To our best knowledge, we report the\nfirst syntax-agnostic SRL model that surpasses all known syntax-aware models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03840v4",
            "date": "2018-08-11 17:31:15+00:00",
            "title": "Fake Sentence Detection as a Training Task for Sentence Encoding",
            "authors": [
                "Viresh Ranjan",
                "Heeyoung Kwon",
                "Niranjan Balasubramanian",
                "Minh Hoai"
            ],
            "abstract": "Sentence encoders are typically trained on language modeling tasks with large\nunlabeled datasets. While these encoders achieve state-of-the-art results on\nmany sentence-level tasks, they are difficult to train with long training\ncycles. We introduce fake sentence detection as a new training task for\nlearning sentence encoders. We automatically generate fake sentences by\ncorrupting original sentences from a source collection and train the encoders\nto produce representations that are effective at detecting fake sentences. This\nbinary classification task turns to be quite efficient for training sentence\nencoders. We compare a basic BiLSTM encoder trained on this task with a strong\nsentence encoding models (Skipthought and FastSent) trained on a language\nmodeling task. We find that the BiLSTM trains much faster on fake sentence\ndetection (20 hours instead of weeks) using smaller amounts of data (1M instead\nof 64M sentences). Further analysis shows the learned representations capture\nmany syntactic and semantic properties expected from good sentence\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03867v3",
            "date": "2018-08-11 21:23:24+00:00",
            "title": "Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction",
            "authors": [
                "Maha Elbayad",
                "Laurent Besacier",
                "Jakob Verbeek"
            ],
            "abstract": "Current state-of-the-art machine translation systems are based on\nencoder-decoder architectures, that first encode the input sequence, and then\ngenerate an output sequence based on the input encoding. Both are interfaced\nwith an attention mechanism that recombines a fixed encoding of the source\ntokens based on the decoder state. We propose an alternative approach which\ninstead relies on a single 2D convolutional neural network across both\nsequences. Each layer of our network re-codes source tokens on the basis of the\noutput sequence produced so far. Attention-like properties are therefore\npervasive throughout the network. Our model yields excellent results,\noutperforming state-of-the-art encoder-decoder systems, while being\nconceptually simpler and having fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03915v1",
            "date": "2018-08-12 09:37:25+00:00",
            "title": "Addressee and Response Selection for Multilingual Conversation",
            "authors": [
                "Motoki Sato",
                "Hiroki Ouch",
                "Yuta Tsuboi"
            ],
            "abstract": "Developing conversational systems that can converse in many languages is an\ninteresting challenge for natural language processing. In this paper, we\nintroduce multilingual addressee and response selection. In this task, a\nconversational system predicts an appropriate addressee and response for an\ninput message in multiple languages. A key to developing such multilingual\nresponding systems is how to utilize high-resource language data to compensate\nfor low-resource language data. We present several knowledge transfer methods\nfor conversational systems. To evaluate our methods, we create a new\nmultilingual conversation dataset. Experiments on the dataset demonstrate the\neffectiveness of our methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03976v2",
            "date": "2018-08-12 18:08:38+00:00",
            "title": "Text Classification using Capsules",
            "authors": [
                "Jaeyoung Kim",
                "Sion Jang",
                "Sungchul Choi",
                "Eunjeong Park"
            ],
            "abstract": "This paper presents an empirical exploration of the use of capsule networks\nfor text classification. While it has been shown that capsule networks are\neffective for image classification, their validity in the domain of text has\nnot been explored. In this paper, we show that capsule networks indeed have the\npotential for text classification and that they have several advantages over\nconvolutional neural networks. We further suggest a simple routing method that\neffectively reduces the computational complexity of dynamic routing. We\nutilized seven benchmark datasets to demonstrate that capsule networks, along\nwith the proposed routing method provide comparable results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04029v2",
            "date": "2018-08-13 00:16:55+00:00",
            "title": "Confidence penalty, annealing Gaussian noise and zoneout for biLSTM-CRF networks for named entity recognition",
            "authors": [
                "Antonio Jimeno Yepes"
            ],
            "abstract": "Named entity recognition (NER) is used to identify relevant entities in text.\nA bidirectional LSTM (long short term memory) encoder with a neural conditional\nrandom fields (CRF) decoder (biLSTM-CRF) is the state of the art methodology.\nIn this work, we have done an analysis of several methods that intend to\noptimize the performance of networks based on this architecture, which in some\ncases encourage overfitting avoidance. These methods target exploration of\nparameter space, regularization of LSTMs and penalization of confident output\ndistributions. Results show that the optimization methods improve the\nperformance of the biLSTM-CRF NER baseline system, setting a new state of the\nart performance for the CoNLL-2003 Spanish set with an F1 of 87.18.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04064v2",
            "date": "2018-08-13 05:03:42+00:00",
            "title": "Regularizing Neural Machine Translation by Target-bidirectional Agreement",
            "authors": [
                "Zhirui Zhang",
                "Shuangzhi Wu",
                "Shujie Liu",
                "Mu Li",
                "Ming Zhou",
                "Tong Xu"
            ],
            "abstract": "Although Neural Machine Translation (NMT) has achieved remarkable progress in\nthe past several years, most NMT systems still suffer from a fundamental\nshortcoming as in other sequence generation tasks: errors made early in\ngeneration process are fed as inputs to the model and can be quickly amplified,\nharming subsequent sequence generation. To address this issue, we propose a\nnovel model regularization method for NMT training, which aims to improve the\nagreement between translations generated by left-to-right (L2R) and\nright-to-left (R2L) NMT decoders. This goal is achieved by introducing two\nKullback-Leibler divergence regularization terms into the NMT training\nobjective to reduce the mismatch between output probabilities of L2R and R2L\nmodels. In addition, we also employ a joint training strategy to allow L2R and\nR2L models to improve each other in an interactive update process. Experimental\nresults show that our proposed method significantly outperforms\nstate-of-the-art baselines on Chinese-English and English-German translation\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04071v1",
            "date": "2018-08-13 06:08:45+00:00",
            "title": "Language Style Transfer from Sentences with Arbitrary Unknown Styles",
            "authors": [
                "Yanpeng Zhao",
                "Wei Bi",
                "Deng Cai",
                "Xiaojiang Liu",
                "Kewei Tu",
                "Shuming Shi"
            ],
            "abstract": "Language style transfer is the problem of migrating the content of a source\nsentence to a target style. In many of its applications, parallel training data\nare not available and source sentences to be transferred may have arbitrary and\nunknown styles. First, each sentence is encoded into its content and style\nlatent representations. Then, by recombining the content with the target style,\nwe decode a sentence aligned in the target domain. To adequately constrain the\nencoding and decoding functions, we couple them with two loss functions. The\nfirst is a style discrepancy loss, enforcing that the style representation\naccurately encodes the style information guided by the discrepancy between the\nsentence style and the target style. The second is a cycle consistency loss,\nwhich ensures that the transferred sentence should preserve the content of the\noriginal sentence disentangled from its style. We validate the effectiveness of\nour model in three tasks: sentiment modification of restaurant reviews, dialog\nresponse revision with a romantic style, and sentence rewriting with a\nShakespearean style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04091v1",
            "date": "2018-08-13 07:52:49+00:00",
            "title": "Live Video Comment Generation Based on Surrounding Frames and Live Comments",
            "authors": [
                "Damai Dai"
            ],
            "abstract": "In this paper, we propose the task of live comment generation. Live comments\nare a new form of comments on videos, which can be regarded as a mixture of\ncomments and chats. A high-quality live comment should be not only relevant to\nthe video, but also interactive with other users. In this work, we first\nconstruct a new dataset for live comment generation. Then, we propose a novel\nend-to-end model to generate the human-like live comments by referring to the\nvideo and the other users' comments. Finally, we evaluate our model on the\nconstructed dataset. Experimental results show that our method can\nsignificantly outperform the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04126v1",
            "date": "2018-08-13 09:50:43+00:00",
            "title": "Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering",
            "authors": [
                "Daniil Sorokin",
                "Iryna Gurevych"
            ],
            "abstract": "The most approaches to Knowledge Base Question Answering are based on\nsemantic parsing. In this paper, we address the problem of learning vector\nrepresentations for complex semantic parses that consist of multiple entities\nand relations. Previous work largely focused on selecting the correct semantic\nrelations for a question and disregarded the structure of the semantic parse:\nthe connections between entities and the directions of the relations. We\npropose to use Gated Graph Neural Networks to encode the graph structure of the\nsemantic parse. We show on two data sets that the graph networks outperform all\nbaseline models that do not explicitly model the structure. The error analysis\nconfirms that our approach can successfully process complex semantic parses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04151v1",
            "date": "2018-08-13 11:15:23+00:00",
            "title": "Multi-Task Learning for Sequence Tagging: An Empirical Study",
            "authors": [
                "Soravit Changpinyo",
                "Hexiang Hu",
                "Fei Sha"
            ],
            "abstract": "We study three general multi-task learning (MTL) approaches on 11 sequence\ntagging tasks. Our extensive empirical results show that in about 50% of the\ncases, jointly learning all 11 tasks improves upon either independent or\npairwise learning of the tasks. We also show that pairwise MTL can inform us\nwhat tasks can benefit others or what tasks can be benefited if they are\nlearned jointly. In particular, we identify tasks that can always benefit\nothers as well as tasks that can always be harmed by others. Interestingly, one\nof our MTL approaches yields embeddings of the tasks that reveal the natural\nclustering of semantic and syntactic tasks. Our inquiries have opened the doors\nto further utilization of MTL in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04164v1",
            "date": "2018-08-13 12:04:44+00:00",
            "title": "Automatic Reference-Based Evaluation of Pronoun Translation Misses the Point",
            "authors": [
                "Liane Guillou",
                "Christian Hardmeier"
            ],
            "abstract": "We compare the performance of the APT and AutoPRF metrics for pronoun\ntranslation against a manually annotated dataset comprising human judgements as\nto the correctness of translations of the PROTEST test suite. Although there is\nsome correlation with the human judgements, a range of issues limit the\nperformance of the automated metrics. Instead, we recommend the use of\nsemi-automatic metrics and test suites in place of fully automatic metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04189v1",
            "date": "2018-08-13 13:06:24+00:00",
            "title": "Rapid Adaptation of Neural Machine Translation to New Languages",
            "authors": [
                "Graham Neubig",
                "Junjie Hu"
            ],
            "abstract": "This paper examines the problem of adapting neural machine translation\nsystems to new, low-resourced languages (LRLs) as effectively and rapidly as\npossible. We propose methods based on starting with massively multilingual\n\"seed models\", which can be trained ahead-of-time, and then continuing training\non data related to the LRL. We contrast a number of strategies, leading to a\nnovel, simple, yet effective method of \"similar-language regularization\", where\nwe jointly train on both a LRL of interest and a similar high-resourced\nlanguage to prevent over-fitting to small LRL data. Experiments demonstrate\nthat massively multilingual models, even without any explicit adaptation, are\nsurprisingly effective, achieving BLEU scores of up to 15.5 with no data from\nthe LRL, and that the proposed similar-language regularization method improves\nover other adaptation methods by 1.7 BLEU points average over 4 LRL settings.\nCode to reproduce experiments at https://github.com/neubig/rapid-adaptation",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04208v3",
            "date": "2018-08-13 13:44:22+00:00",
            "title": "Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging",
            "authors": [
                "Apostolos Kemos",
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Character-level models of tokens have been shown to be effective at dealing\nwith within-token noise and out-of-vocabulary words. But these models still\nrely on correct token boundaries. In this paper, we propose a novel end-to-end\ncharacter-level model and demonstrate its effectiveness in multilingual\nsettings and when token boundaries are noisy. Our model is a semi-Markov\nconditional random field with neural networks for character and segment\nrepresentation. It requires no tokenizer. The model matches state-of-the-art\nbaselines for various languages and significantly outperforms them on a noisy\nEnglish version of a part-of-speech tagging benchmark dataset. Our code and the\nnoisy dataset are publicly available at http://cistern.cis.lmu.de/semiCRF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04314v1",
            "date": "2018-08-13 16:16:11+00:00",
            "title": "Comparing morphological complexity of Spanish, Otomi and Nahuatl",
            "authors": [
                "Ximena Gutierrez-Vasques",
                "Victor Mijangos"
            ],
            "abstract": "We use two small parallel corpora for comparing the morphological complexity\nof Spanish, Otomi and Nahuatl. These are languages that belong to different\nlinguistic families, the latter are low-resourced. We take into account two\nquantitative criteria, on one hand the distribution of types over tokens in a\ncorpus, on the other, perplexity and entropy as indicators of word structure\npredictability. We show that a language can be complex in terms of how many\ndifferent morphological word forms can produce, however, it may be less complex\nin terms of predictability of its internal structure of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04339v2",
            "date": "2018-08-13 17:26:49+00:00",
            "title": "Disentangled Representation Learning for Non-Parallel Text Style Transfer",
            "authors": [
                "Vineet John",
                "Lili Mou",
                "Hareesh Bahuleyan",
                "Olga Vechtomova"
            ],
            "abstract": "This paper tackles the problem of disentangling the latent variables of style\nand content in language models. We propose a simple yet effective approach,\nwhich incorporates auxiliary multi-task and adversarial objectives, for label\nprediction and bag-of-words prediction, respectively. We show, both\nqualitatively and quantitatively, that the style and content are indeed\ndisentangled in the latent space. This disentangled latent representation\nlearning method is applied to style transfer on non-parallel corpora. We\nachieve substantially better results in terms of transfer accuracy, content\npreservation and language fluency, in comparison to previous state-of-the-art\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04459v1",
            "date": "2018-08-09 14:21:35+00:00",
            "title": "Deep Learning Based Natural Language Processing for End to End Speech Translation",
            "authors": [
                "Sarvesh Patil"
            ],
            "abstract": "Deep Learning methods employ multiple processing layers to learn hierarchial\nrepresentations of data. They have already been deployed in a humongous number\nof applications and have produced state-of-the-art results. Recently with the\ngrowth in processing power of computers to be able to do high dimensional\ntensor calculations, Natural Language Processing (NLP) applications have been\ngiven a significant boost in terms of efficiency as well as accuracy. In this\npaper, we will take a look at various signal processing techniques and then\napplication of them to produce a speech-to-text system using Deep Recurrent\nNeural Networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04525v1",
            "date": "2018-08-14 05:13:23+00:00",
            "title": "Discrete Structural Planning for Neural Machine Translation",
            "authors": [
                "Raphael Shu",
                "Hideki Nakayama"
            ],
            "abstract": "Structural planning is important for producing long sentences, which is a\nmissing part in current language generation models. In this work, we add a\nplanning phase in neural machine translation to control the coarse structure of\noutput sentences. The model first generates some planner codes, then predicts\nreal output words conditioned on them. The codes are learned to capture the\ncoarse structure of the target sentence. In order to obtain the codes, we\ndesign an end-to-end neural network with a discretization bottleneck, which\npredicts the simplified part-of-speech tags of target sentences. Experiments\nshow that the translation performance are generally improved by planning ahead.\nWe also find that translations with different structures can be obtained by\nmanipulating the planner codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04660v2",
            "date": "2018-08-14 12:37:31+00:00",
            "title": "Primal Meaning Recommendation via On-line Encyclopedia",
            "authors": [
                "Zhiyuan Zhang",
                "Wei Li",
                "Jingjing Xu",
                "Xu Sun"
            ],
            "abstract": "Polysemy is a very common phenomenon in modern languages. Under many\ncircumstances, there exists a primal meaning for the expression. We define the\nprimal meaning of an expression to be a frequently used sense of that\nexpression from which its other frequent senses can be deduced. Many of the new\nappearing meanings of the expressions are either originated from a primal\nmeaning, or are merely literal references to the original expression, e.g.,\napple (fruit), Apple (Inc), and Apple (movie). When constructing a knowledge\nbase from on-line encyclopedia data, it would be more efficient to be aware of\nthe information about the importance of the senses. In this paper, we would\nlike to explore a way to automatically recommend the primal meaning of an\nexpression based on the textual descriptions of the multiple senses of an\nexpression from on-line encyclopedia websites. We propose a hybrid model that\ncaptures both the pattern of the description and the relationship between\ndifferent descriptions with both weakly supervised and unsupervised models. The\nexperiment results show that our method yields a good result with a P@1\n(precision) score of 83.3 per cent, and a MAP (mean average precision) of 90.5\nper cent, surpassing the UMFS-WE baseline by a big margin (P@1 is 61.1 per cent\nand MAP is 76.3 per cent).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04736v1",
            "date": "2018-08-14 15:12:55+00:00",
            "title": "Adversarial Neural Networks for Cross-lingual Sequence Tagging",
            "authors": [
                "Heike Adel",
                "Anton Bryl",
                "David Weiss",
                "Aliaksei Severyn"
            ],
            "abstract": "We study cross-lingual sequence tagging with little or no labeled data in the\ntarget language. Adversarial training has previously been shown to be effective\nfor training cross-lingual sentence classifiers. However, it is not clear if\nlanguage-agnostic representations enforced by an adversarial language\ndiscriminator will also enable effective transfer for token-level prediction\ntasks. Therefore, we experiment with different types of adversarial training on\ntwo tasks: dependency parsing and sentence compression. We show that\nadversarial training consistently leads to improved cross-lingual performance\non each task compared to a conventionally trained baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04776v2",
            "date": "2018-08-14 16:13:44+00:00",
            "title": "Retrieve and Refine: Improved Sequence Generation Models For Dialogue",
            "authors": [
                "Jason Weston",
                "Emily Dinan",
                "Alexander H. Miller"
            ],
            "abstract": "Sequence generation models for dialogue are known to have several problems:\nthey tend to produce short, generic sentences that are uninformative and\nunengaging. Retrieval models on the other hand can surface interesting\nresponses, but are restricted to the given retrieval set leading to erroneous\nreplies that cannot be tuned to the specific context. In this work we develop a\nmodel that combines the two approaches to avoid both their deficiencies: first\nretrieve a response and then refine it -- the final sequence generator treating\nthe retrieval as additional context. We show on the recent CONVAI2 challenge\ntask our approach produces responses superior to both standard retrieval and\ngeneration models in human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04800v1",
            "date": "2018-08-14 17:22:25+00:00",
            "title": "Classifier Ensembles for Dialect and Language Variety Identification",
            "authors": [
                "Liviu P. Dinu",
                "Alina Maria Ciobanu",
                "Marcos Zampieri",
                "Shervin Malmasi"
            ],
            "abstract": "In this paper we present ensemble-based systems for dialect and language\nvariety identification using the datasets made available by the organizers of\nthe VarDial Evaluation Campaign 2018. We present a system developed to\ndiscriminate between Flemish and Dutch in subtitles and a system trained to\ndiscriminate between four Arabic dialects: Egyptian, Levantine, Gulf, North\nAfrican, and Modern Standard Arabic in speech broadcasts. Finally, we compare\nthe performance of these two systems with the other systems submitted to the\nDiscriminating between Dutch and Flemish in Subtitles (DFS) and the Arabic\nDialect Identification (ADI) shared tasks at VarDial 2018.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04816v2",
            "date": "2018-08-14 17:55:13+00:00",
            "title": "KGCleaner : Identifying and Correcting Errors Produced by Information Extraction Systems",
            "authors": [
                "Ankur Padia",
                "Frank Ferraro",
                "Tim Finin"
            ],
            "abstract": "KGCleaner is a framework to identify and correct errors in data produced and\ndelivered by an information extraction system. These tasks have been\nunderstudied and KGCleaner is the first to address both. We introduce a\nmulti-task model that jointly learns to predict if an extracted relation is\ncredible and repair it if not. We evaluate our approach and other models as\ninstance of our framework on two collections: a Wikidata corpus of nearly 700K\nfacts and 5M fact-relevant sentences and a collection of 30K facts from the\n2015 TAC Knowledge Base Population task. For credibility classification,\nparameter efficient simple shallow neural network can achieve an absolute\nperformance gain of 30 $F_1$ points on Wikidata and comparable performance on\nTAC. For the repair task, significant performance (at more than twice) gain can\nbe obtained depending on the nature of the dataset and the models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04850v2",
            "date": "2018-08-14 18:34:17+00:00",
            "title": "Two Local Models for Neural Constituent Parsing",
            "authors": [
                "Zhiyang Teng",
                "Yue Zhang"
            ],
            "abstract": "Non-local features have been exploited by syntactic parsers for capturing\ndependencies between sub output structures. Such features have been a key to\nthe success of state-of-the-art statistical parsers. With the rise of deep\nlearning, however, it has been shown that local output decisions can give\nhighly competitive accuracies, thanks to the power of dense neural input\nrepresentations that embody global syntactic information. We investigate two\nconceptually simple local neural models for constituent parsing, which make\nlocal decisions to constituent spans and CFG rules, respectively. Consistent\nwith previous findings along the line, our best model gives highly competitive\nresults, achieving the labeled bracketing F1 scores of 92.4% on PTB and 87.3%\non CTB 5.1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04865v1",
            "date": "2018-08-14 19:12:44+00:00",
            "title": "Top-Down Tree Structured Text Generation",
            "authors": [
                "Qipeng Guo",
                "Xipeng Qiu",
                "Xiangyang Xue",
                "Zheng Zhang"
            ],
            "abstract": "Text generation is a fundamental building block in natural language\nprocessing tasks. Existing sequential models performs autoregression directly\nover the text sequence and have difficulty generating long sentences of complex\nstructures. This paper advocates a simple approach that treats sentence\ngeneration as a tree-generation task. By explicitly modelling syntactic\nstructures in a constituent syntactic tree and performing top-down,\nbreadth-first tree generation, our model fixes dependencies appropriately and\nperforms implicit global planning. This is in contrast to transition-based\ndepth-first generation process, which has difficulty dealing with incomplete\ntexts when parsing and also does not incorporate future contexts in planning.\nOur preliminary results on two generation tasks and one parsing task\ndemonstrate that this is an effective strategy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04891v1",
            "date": "2018-08-14 20:52:43+00:00",
            "title": "Embedding Grammars",
            "authors": [
                "David Wingate",
                "William Myers",
                "Nancy Fulda",
                "Tyler Etchart"
            ],
            "abstract": "Classic grammars and regular expressions can be used for a variety of\npurposes, including parsing, intent detection, and matching. However, the\ncomparisons are performed at a structural level, with constituent elements\n(words or characters) matched exactly. Recent advances in word embeddings show\nthat semantically related words share common features in a vector-space\nrepresentation, suggesting the possibility of a hybrid grammar and word\nembedding. In this paper, we blend the structure of standard context-free\ngrammars with the semantic generalization capabilities of word embeddings to\ncreate hybrid semantic grammars. These semantic grammars generalize the\nspecific terminals used by the programmer to other words and phrases with\nrelated meanings, allowing the construction of compact grammars that match an\nentire region of the vector space rather than matching specific elements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04943v1",
            "date": "2018-08-15 01:50:56+00:00",
            "title": "Folksonomication: Predicting Tags for Movies from Plot Synopses Using Emotion Flow Encoded Neural Network",
            "authors": [
                "Sudipta Kar",
                "Suraj Maharjan",
                "Thamar Solorio"
            ],
            "abstract": "Folksonomy of movies covers a wide range of heterogeneous information about\nmovies, like the genre, plot structure, visual experiences, soundtracks,\nmetadata, and emotional experiences from watching a movie. Being able to\nautomatically generate or predict tags for movies can help recommendation\nengines improve retrieval of similar movies, and help viewers know what to\nexpect from a movie in advance. In this work, we explore the problem of\ncreating tags for movies from plot synopses. We propose a novel neural network\nmodel that merges information from synopses and emotion flows throughout the\nplots to predict a set of tags for movies. We compare our system with multiple\nbaselines and found that the addition of emotion flows boosts the performance\nof the network by learning ~18\\% more tags than a traditional machine learning\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04961v5",
            "date": "2018-08-15 04:05:02+00:00",
            "title": "Putting the Horse Before the Cart:A Generator-Evaluator Framework for Question Generation from Text",
            "authors": [
                "Vishwajeet Kumar",
                "Ganesh Ramakrishnan",
                "Yuan-Fang Li"
            ],
            "abstract": "Automatic question generation (QG) is a useful yet challenging task in NLP.\nRecent neural network-based approaches represent the state-of-the-art in this\ntask. In this work, we attempt to strengthen them significantly by adopting a\nholistic and novel generator-evaluator framework that directly optimizes\nobjectives that reward semantics and structure. The {\\it generator} is a\nsequence-to-sequence model that incorporates the {\\it structure} and {\\it\nsemantics} of the question being generated. The generator predicts an answer in\nthe passage that the question can pivot on. Employing the copy and coverage\nmechanisms, it also acknowledges other contextually important (and possibly\nrare) keywords in the passage that the question needs to conform to, while not\nredundantly repeating words. The {\\it evaluator} model evaluates and assigns a\nreward to each predicted question based on its conformity to the {\\it\nstructure} of ground-truth questions. We propose two novel QG-specific reward\nfunctions for text conformity and answer conformity of the generated question.\nThe evaluator also employs structure-sensitive rewards based on evaluation\nmeasures such as BLEU, GLEU, and ROUGE-L, which are suitable for QG. In\ncontrast, most of the previous works only optimize the cross-entropy loss,\nwhich can induce inconsistencies between training (objective) and testing\n(evaluation) measures. Our evaluation shows that our approach significantly\noutperforms state-of-the-art systems on the widely-used SQuAD benchmark as per\nboth automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04963v3",
            "date": "2018-08-15 04:10:35+00:00",
            "title": "Multiple Character Embeddings for Chinese Word Segmentation",
            "authors": [
                "Jingkang Wang",
                "Jianing Zhou",
                "Jie Zhou",
                "Gongshen Liu"
            ],
            "abstract": "Chinese word segmentation (CWS) is often regarded as a character-based\nsequence labeling task in most current works which have achieved great success\nwith the help of powerful neural networks. However, these works neglect an\nimportant clue: Chinese characters incorporate both semantic and phonetic\nmeanings. In this paper, we introduce multiple character embeddings including\nPinyin Romanization and Wubi Input, both of which are easily accessible and\neffective in depicting semantics of characters. We propose a novel shared\nBi-LSTM-CRF model to fuse linguistic features efficiently by sharing the LSTM\nnetwork during the training procedure. Extensive experiments on five corpora\nshow that extra embeddings help obtain a significant improvement in labeling\naccuracy. Specifically, we achieve the state-of-the-art performance in AS and\nCityU corpora with F1 scores of 96.9 and 97.3, respectively without leveraging\nany external lexical resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05077v1",
            "date": "2018-08-15 13:46:54+00:00",
            "title": "Exploiting Deep Learning for Persian Sentiment Analysis",
            "authors": [
                "Kia Dashtipour",
                "Mandar Gogate",
                "Ahsan Adeel",
                "Cosimo Ieracitano",
                "Hadi Larijani",
                "Amir Hussain"
            ],
            "abstract": "The rise of social media is enabling people to freely express their opinions\nabout products and services. The aim of sentiment analysis is to automatically\ndetermine subject's sentiment (e.g., positive, negative, or neutral) towards a\nparticular aspect such as topic, product, movie, news etc. Deep learning has\nrecently emerged as a powerful machine learning technique to tackle a growing\ndemand of accurate sentiment analysis. However, limited work has been conducted\nto apply deep learning algorithms to languages other than English, such as\nPersian. In this work, two deep learning models (deep autoencoders and deep\nconvolutional neural networks (CNNs)) are developed and applied to a novel\nPersian movie reviews dataset. The proposed deep learning models are analyzed\nand compared with the state-of-the-art shallow multilayer perceptron (MLP)\nbased machine learning model. Simulation results demonstrate the enhanced\nperformance of deep learning over state-of-the-art MLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05079v1",
            "date": "2018-08-15 13:48:16+00:00",
            "title": "SentiALG: Automated Corpus Annotation for Algerian Sentiment Analysis",
            "authors": [
                "Imane Guellil",
                "Ahsan Adeel",
                "Faical Azouaou",
                "Amir Hussain"
            ],
            "abstract": "Data annotation is an important but time-consuming and costly procedure. To\nsort a text into two classes, the very first thing we need is a good annotation\nguideline, establishing what is required to qualify for each class. In the\nliterature, the difficulties associated with an appropriate data annotation has\nbeen underestimated. In this paper, we present a novel approach to\nautomatically construct an annotated sentiment corpus for Algerian dialect (a\nMaghrebi Arabic dialect). The construction of this corpus is based on an\nAlgerian sentiment lexicon that is also constructed automatically. The\npresented work deals with the two widely used scripts on Arabic social media:\nArabic and Arabizi. The proposed approach automatically constructs a sentiment\ncorpus containing 8000 messages (where 4000 are dedicated to Arabic and 4000 to\nArabizi). The achieved F1-score is up to 72% and 78% for an Arabic and Arabizi\ntest sets, respectively. Ongoing work is aimed at integrating transliteration\nprocess for Arabizi messages to further improve the obtained results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05306v2",
            "date": "2018-08-15 23:23:20+00:00",
            "title": "Incorporating Consistency Verification into Neural Data-to-Document Generation",
            "authors": [
                "Feng Nie",
                "Hailin Chen",
                "Jinpeng Wang",
                "Jin-Ge Yao",
                "Chin-Yew Lin",
                "Rong Pan"
            ],
            "abstract": "Recent neural models for data-to-document generation have achieved remarkable\nprogress in producing fluent and informative texts. However, large proportions\nof generated texts do not actually conform to the input data. To address this\nissue, we propose a new training framework which attempts to verify the\nconsistency between the generated texts and the input data to guide the\ntraining process. To measure the consistency, a relation extraction model is\napplied to check information overlaps between the input data and the generated\ntexts. The non-differentiable consistency signal is optimized via reinforcement\nlearning. Experimental results on a recently released challenging dataset\nROTOWIRE show improvements from our framework in various metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05326v1",
            "date": "2018-08-16 02:21:01+00:00",
            "title": "SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference",
            "authors": [
                "Rowan Zellers",
                "Yonatan Bisk",
                "Roy Schwartz",
                "Yejin Choi"
            ],
            "abstract": "Given a partial description like \"she opened the hood of the car,\" humans can\nreason about the situation and anticipate what might come next (\"then, she\nexamined the engine\"). In this paper, we introduce the task of grounded\ncommonsense inference, unifying natural language inference and commonsense\nreasoning.\n  We present SWAG, a new dataset with 113k multiple choice questions about a\nrich spectrum of grounded situations. To address the recurring challenges of\nthe annotation artifacts and human biases found in many existing datasets, we\npropose Adversarial Filtering (AF), a novel procedure that constructs a\nde-biased dataset by iteratively training an ensemble of stylistic classifiers,\nand using them to filter the data. To account for the aggressive adversarial\nfiltering, we use state-of-the-art language models to massively oversample a\ndiverse set of potential counterfactuals. Empirical results demonstrate that\nwhile humans can solve the resulting inference problems with high accuracy\n(88%), various competitive models struggle on our task. We provide\ncomprehensive analysis that indicates significant opportunities for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05374v1",
            "date": "2018-08-16 08:11:24+00:00",
            "title": "Computing Word Classes Using Spectral Clustering",
            "authors": [
                "Effi Levi",
                "Saggy Herman",
                "Ari Rappoport"
            ],
            "abstract": "Clustering a lexicon of words is a well-studied problem in natural language\nprocessing (NLP). Word clusters are used to deal with sparse data in\nstatistical language processing, as well as features for solving various NLP\ntasks (text categorization, question answering, named entity recognition and\nothers).\n  Spectral clustering is a widely used technique in the field of image\nprocessing and speech recognition. However, it has scarcely been explored in\nthe context of NLP; specifically, the method used in this (Meila and Shi, 2001)\nhas never been used to cluster a general word lexicon.\n  We apply spectral clustering to a lexicon of words, evaluating the resulting\nclusters by using them as features for solving two classical NLP tasks:\nsemantic role labeling and dependency parsing. We compare performance with\nBrown clustering, a widely-used technique for word clustering, as well as with\nother clustering methods. We show that spectral clusters produce similar\nresults to Brown clusters, and outperform other clustering methods. In\naddition, we quantify the overlap between spectral and Brown clusters, showing\nthat each model captures some information which is uncaptured by the other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05437v1",
            "date": "2018-08-16 12:13:16+00:00",
            "title": "Sememe Prediction: Learning Semantic Knowledge from Unstructured Textual Wiki Descriptions",
            "authors": [
                "Wei Li",
                "Xuancheng Ren",
                "Damai Dai",
                "Yunfang Wu",
                "Houfeng Wang",
                "Xu Sun"
            ],
            "abstract": "Huge numbers of new words emerge every day, leading to a great need for\nrepresenting them with semantic meaning that is understandable to NLP systems.\nSememes are defined as the minimum semantic units of human languages, the\ncombination of which can represent the meaning of a word. Manual construction\nof sememe based knowledge bases is time-consuming and labor-intensive.\nFortunately, communities are devoted to composing the descriptions of words in\nthe wiki websites. In this paper, we explore to automatically predict lexical\nsememes based on the descriptions of the words in the wiki websites. We view\nthis problem as a weakly ordered multi-label task and propose a Label\nDistributed seq2seq model (LD-seq2seq) with a novel soft loss function to solve\nthe problem. In the experiments, we take a real-world sememe knowledge base\nHowNet and the corresponding descriptions of the words in Baidu Wiki for\ntraining and evaluation. The results show that our LD-seq2seq model not only\nbeats all the baselines significantly on the test set, but also outperforms\namateur human annotators in a random subset of the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05505v3",
            "date": "2018-08-16 14:20:50+00:00",
            "title": "Paraphrase Thought: Sentence Embedding Module Imitating Human Language Recognition",
            "authors": [
                "Myeongjun Jang",
                "Pilsung Kang"
            ],
            "abstract": "Sentence embedding is an important research topic in natural language\nprocessing. It is essential to generate a good embedding vector that fully\nreflects the semantic meaning of a sentence in order to achieve an enhanced\nperformance for various natural language processing tasks, such as machine\ntranslation and document classification. Thus far, various sentence embedding\nmodels have been proposed, and their feasibility has been demonstrated through\ngood performances on tasks following embedding, such as sentiment analysis and\nsentence classification. However, because the performances of sentence\nclassification and sentiment analysis can be enhanced by using a simple\nsentence representation method, it is not sufficient to claim that these models\nfully reflect the meanings of sentences based on good performances for such\ntasks. In this paper, inspired by human language recognition, we propose the\nfollowing concept of semantic coherence, which should be satisfied for a good\nsentence embedding method: similar sentences should be located close to each\nother in the embedding space. Then, we propose the Paraphrase-Thought\n(P-thought) model to pursue semantic coherence as much as possible.\nExperimental results on two paraphrase identification datasets (MS COCO and STS\nbenchmark) show that the P-thought models outperform the benchmarked sentence\nembedding methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05542v1",
            "date": "2018-08-08 12:51:21+00:00",
            "title": "Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness",
            "authors": [
                "Pepa Atanasova",
                "Alberto Barron-Cedeno",
                "Tamer Elsayed",
                "Reem Suwaileh",
                "Wajdi Zaghouani",
                "Spas Kyuchukov",
                "Giovanni Da San Martino",
                "Preslav Nakov"
            ],
            "abstract": "We present an overview of the CLEF-2018 CheckThat! Lab on Automatic\nIdentification and Verification of Political Claims, with focus on Task 1:\nCheck-Worthiness. The task asks to predict which claims in a political debate\nshould be prioritized for fact-checking. In particular, given a debate or a\npolitical speech, the goal was to produce a ranked list of its sentences based\non their worthiness for fact checking. We offered the task in both English and\nArabic, based on debates from the 2016 US Presidential Campaign, as well as on\nsome speeches during and after the campaign. A total of 30 teams registered to\nparticipate in the Lab and seven teams actually submitted systems for Task~1.\nThe most successful approaches used by the participants relied on recurrent and\nmulti-layer neural networks, as well as on combinations of distributional\nrepresentations, on matchings claims' vocabulary against lexicons, and on\nmeasures of syntactic dependency. The best systems achieved mean average\nprecision of 0.18 and 0.15 on the English and on the Arabic test datasets,\nrespectively. This leaves large room for further improvement, and thus we\nrelease all datasets and the scoring scripts, which should enable further\nresearch in check-worthiness estimation.",
            "journal ref": "CLEF-2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05611v4",
            "date": "2018-08-16 17:59:14+00:00",
            "title": "Learning Graph Embeddings from WordNet-based Similarity Measures",
            "authors": [
                "Andrey Kutuzov",
                "Mohammad Dorgham",
                "Oleksiy Oliynyk",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We present path2vec, a new approach for learning graph embeddings that relies\non structural measures of pairwise node similarities. The model learns\nrepresentations for nodes in a dense space that approximate a given\nuser-defined graph distance measure, such as e.g. the shortest path distance or\ndistance measures that take information beyond the graph structure into\naccount. Evaluation of the proposed model on semantic similarity and word sense\ndisambiguation tasks, using various WordNet-based similarity measures, show\nthat our approach yields competitive results, outperforming strong graph\nembedding baselines. The model is computationally efficient, being orders of\nmagnitude faster than the direct computation of graph-based distances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05668v1",
            "date": "2018-08-16 20:22:18+00:00",
            "title": "Predicting Human Trustfulness from Facebook Language",
            "authors": [
                "Mohammadzaman Zamani",
                "Anneke Buffone",
                "H. Andrew Schwartz"
            ],
            "abstract": "Trustfulness -- one's general tendency to have confidence in unknown people\nor situations -- predicts many important real-world outcomes such as mental\nhealth and likelihood to cooperate with others such as clinicians. While\ndata-driven measures of interpersonal trust have previously been introduced,\nhere, we develop the first language-based assessment of the personality trait\nof trustfulness by fitting one's language to an accepted questionnaire-based\ntrust score. Further, using trustfulness as a type of case study, we explore\nthe role of questionnaire size as well as word count in developing\nlanguage-based predictive models of users' psychological traits. We find that\nleveraging a longer questionnaire can yield greater test set accuracy, while,\nfor training, we find it beneficial to include users who took smaller\nquestionnaires which offers more observations for training. Similarly, after\nnoting a decrease in individual prediction error as word count increased, we\nfound a word count-weighted training scheme was helpful when there were very\nfew users in the first place.",
            "journal ref": "In Proceedings of the Fifth Workshop on Computational Linguistics\n  and Clinical Psychology: From Keyboard to Clinic, pages 174-181, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05700v1",
            "date": "2018-08-16 23:05:17+00:00",
            "title": "Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words",
            "authors": [
                "Nelson F. Liu",
                "Jonathan May",
                "Michael Pust",
                "Kevin Knight"
            ],
            "abstract": "Most statistical machine translation systems cannot translate words that are\nunseen in the training data. However, humans can translate many classes of\nout-of-vocabulary (OOV) words (e.g., novel morphological variants,\nmisspellings, and compounds) without context by using orthographic clues.\nFollowing this observation, we describe and evaluate several general methods\nfor OOV translation that use only subword information. We pose the OOV\ntranslation problem as a standalone task and intrinsically evaluate our\napproaches on fourteen typologically diverse languages across varying resource\nlevels. Adding OOV translators to a statistical machine translation system\nyields consistent BLEU gains (0.5 points on average, and up to 2.0) for all\nfourteen languages, especially in low-resource scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05759v5",
            "date": "2018-08-17 05:18:52+00:00",
            "title": "Read + Verify: Machine Reading Comprehension with Unanswerable Questions",
            "authors": [
                "Minghao Hu",
                "Furu Wei",
                "Yuxing Peng",
                "Zhen Huang",
                "Nan Yang",
                "Dongsheng Li"
            ],
            "abstract": "Machine reading comprehension with unanswerable questions aims to abstain\nfrom answering when no answer can be inferred. In addition to extract answers,\nprevious works usually predict an additional \"no-answer\" probability to detect\nunanswerable cases. However, they fail to validate the answerability of the\nquestion by verifying the legitimacy of the predicted answer. To address this\nproblem, we propose a novel read-then-verify system, which not only utilizes a\nneural reader to extract candidate answers and produce no-answer probabilities,\nbut also leverages an answer verifier to decide whether the predicted answer is\nentailed by the input snippets. Moreover, we introduce two auxiliary losses to\nhelp the reader better handle answer extraction as well as no-answer detection,\nand investigate three different architectures for the answer verifier. Our\nexperiments on the SQuAD 2.0 dataset show that our system achieves a score of\n74.2 F1 on the test set, achieving state-of-the-art results at the time of\nsubmission (Aug. 28th, 2018).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05907v1",
            "date": "2018-08-14 14:40:56+00:00",
            "title": "Syntree2Vec - An algorithm to augment syntactic hierarchy into word embeddings",
            "authors": [
                "Shubham Bhardwaj"
            ],
            "abstract": "Word embeddings aims to map sense of the words into a lower dimensional\nvector space in order to reason over them. Training embeddings on domain\nspecific data helps express concepts more relevant to their use case but comes\nat a cost of accuracy when data is less. Our effort is to minimise this by\ninfusing syntactic knowledge into the embeddings. We propose a graph based\nembedding algorithm inspired from node2vec. Experimental results have shown\nthat our algorithm improves the syntactic strength and gives robust performance\non meagre data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06068v1",
            "date": "2018-08-18 10:23:09+00:00",
            "title": "SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors",
            "authors": [
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "We present SeVeN (Semantic Vector Networks), a hybrid resource that encodes\nrelationships between words in the form of a graph. Different from traditional\nsemantic networks, these relations are represented as vectors in a continuous\nvector space. We propose a simple pipeline for learning such relation vectors,\nwhich is based on word vector averaging in combination with an ad hoc\nautoencoder. We show that by explicitly encoding relational information in a\ndedicated vector space we can capture aspects of word meaning that are\ncomplementary to what is captured by word embeddings. For example, by examining\nclusters of relation vectors, we observe that relational similarities can be\nidentified at a more abstract level than with traditional word vector\ndifferences. Finally, we test the effectiveness of semantic vector networks in\ntwo tasks: measuring word similarity and neural text categorization. SeVeN is\navailable at bitbucket.org/luisespinosa/seven.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06075v2",
            "date": "2018-08-18 11:53:24+00:00",
            "title": "Learning to Compose over Tree Structures via POS Tags",
            "authors": [
                "Gehui Shen",
                "Zhi-Hong Deng",
                "Ting Huang",
                "Xi Chen"
            ],
            "abstract": "Recursive Neural Network (RecNN), a type of models which compose words or\nphrases recursively over syntactic tree structures, has been proven to have\nsuperior ability to obtain sentence representation for a variety of NLP tasks.\nHowever, RecNN is born with a thorny problem that a shared compositional\nfunction for each node of trees can't capture the complex semantic\ncompositionality so that the expressive power of model is limited. In this\npaper, in order to address this problem, we propose Tag-Guided\nHyperRecNN/TreeLSTM (TG-HRecNN/TreeLSTM), which introduces hypernetwork into\nRecNNs to take as inputs Part-of-Speech (POS) tags of word/phrase and generate\nthe semantic composition parameters dynamically. Experimental results on five\ndatasets for two typical NLP tasks show proposed models both obtain significant\nimprovement compared with RecNN and TreeLSTM consistently. Our TG-HTreeLSTM\noutperforms all existing RecNN-based models and achieves or is competitive with\nstate-of-the-art on four sentence classification benchmarks. The effectiveness\nof our models is also demonstrated by qualitative analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06110v2",
            "date": "2018-08-18 18:37:05+00:00",
            "title": "Emoji Sentiment Scores of Writers using Odds Ratio and Fisher Exact Test",
            "authors": [
                "Jose Berengueres"
            ],
            "abstract": "The sentiment of a given emoji is traditionally calculated by averaging the\nratings {-1, 0 or +1} given by various users to a given context where the emoji\nappears. However, using such formula complicates the statistical significance\nanalysis particularly for low sample sizes. Here, we provide sentiment scores\nusing odds and a sentiment mapping to a 4-icon scale. We show how odds ratio\nstatistics leads to simpler sentiment analysis. Finally, we provide a list of\nsentiment scores with the often-missing exact p-values and CI for the most\ncommon emoji.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06116v1",
            "date": "2018-08-18 19:04:54+00:00",
            "title": "A Recipe for Arabic-English Neural Machine Translation",
            "authors": [
                "Abdullah Alrajeh"
            ],
            "abstract": "In this paper, we present a recipe for building a good Arabic-English neural\nmachine translation. We compare neural systems with traditional phrase-based\nsystems using various parallel corpora including UN, ISI and Ummah. We also\ninvestigate the importance of special preprocessing of the Arabic script. The\npresented results are based on test sets from NIST MT 2005 and 2012. The best\nneural system produces a gain of +13 BLEU points compared to an equivalent\nsimple phrase-based system in NIST MT12 test set. Unexpectedly, we find that\ntuning a model trained on the whole data using a small high quality corpus like\nUmmah gives a substantial improvement (+3 BLEU points). We also find that\ntraining a neural system with a small Arabic-English corpus is competitive to a\ntraditional phrase-based system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06161v1",
            "date": "2018-08-19 05:06:28+00:00",
            "title": "Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts",
            "authors": [
                "Di Jin",
                "Peter Szolovits"
            ],
            "abstract": "Prevalent models based on artificial neural network (ANN) for sentence\nclassification often classify sentences in isolation without considering the\ncontext in which sentences appear. This hampers the traditional sentence\nclassification approaches to the problem of sequential sentence classification,\nwhere structured prediction is needed for better overall classification\nperformance. In this work, we present a hierarchical sequential labeling\nnetwork to make use of the contextual information within surrounding sentences\nto help classify the current sentence. Our model outperforms the\nstate-of-the-art results by 2%-3% on two benchmarking datasets for sequential\nsentence classification in medical scientific abstracts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06167v2",
            "date": "2018-08-19 05:49:46+00:00",
            "title": "Source-Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language",
            "authors": [
                "He Bai",
                "Yu Zhou",
                "Jiajun Zhang",
                "Liang Zhao",
                "Mei-Yuh Hwang",
                "Chengqing Zong"
            ],
            "abstract": "To deploy a spoken language understanding (SLU) model to a new language,\nlanguage transferring is desired to avoid the trouble of acquiring and labeling\na new big SLU corpus. Translating the original SLU corpus into the target\nlanguage is an attractive strategy. However, SLU corpora consist of plenty of\nsemantic labels (slots), which general-purpose translators cannot handle well,\nnot to mention additional culture differences. This paper focuses on the\nlanguage transferring task given a tiny in-domain parallel SLU corpus. The\nin-domain parallel corpus can be used as the first adaptation on the general\ntranslator. But more importantly, we show how to use reinforcement learning\n(RL) to further finetune the adapted translator, where translated sentences\nwith more proper slot tags receive higher rewards. We evaluate our approach on\nChinese to English language transferring for SLU systems. The experimental\nresults show that the generated English SLU corpus via adaptation and\nreinforcement learning gives us over 97% in the slot F1 score and over 84%\naccuracy in domain classification. It demonstrates the effectiveness of the\nproposed language transferring method. Compared with naive translation, our\nproposed method improves domain classification accuracy by relatively 22%, and\nthe slot filling F1 score by relatively more than 71%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06218v2",
            "date": "2018-08-19 14:43:09+00:00",
            "title": "Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization",
            "authors": [
                "Logan Lebanoff",
                "Kaiqiang Song",
                "Fei Liu"
            ],
            "abstract": "Generating a text abstract from a set of documents remains a challenging\ntask. The neural encoder-decoder framework has recently been exploited to\nsummarize single documents, but its success can in part be attributed to the\navailability of large parallel data automatically acquired from the Web. In\ncontrast, parallel data for multi-document summarization are scarce and costly\nto obtain. There is a pressing need to adapt an encoder-decoder model trained\non single-document summarization data to work with multiple-document input. In\nthis paper, we present an initial investigation into a novel adaptation method.\nIt exploits the maximal marginal relevance method to select representative\nsentences from multi-document input, and leverages an abstractive\nencoder-decoder model to fuse disparate sentences to an abstractive summary.\nThe adaptation method is robust and itself requires no training data. Our\nsystem compares favorably to state-of-the-art extractive and abstractive\napproaches judged by automatic metrics and human assessors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06219v2",
            "date": "2018-08-19 15:12:19+00:00",
            "title": "Automatic Detection of Vague Words and Sentences in Privacy Policies",
            "authors": [
                "Logan Lebanoff",
                "Fei Liu"
            ],
            "abstract": "Website privacy policies represent the single most important source of\ninformation for users to gauge how their personal data are collected, used and\nshared by companies. However, privacy policies are often vague and people\nstruggle to understand the content. Their opaqueness poses a significant\nchallenge to both users and policy regulators. In this paper, we seek to\nidentify vague content in privacy policies. We construct the first corpus of\nhuman-annotated vague words and sentences and present empirical studies on\nautomatic vagueness detection. In particular, we investigate context-aware and\ncontext-agnostic models for predicting vague words, and explore\nauxiliary-classifier generative adversarial networks for characterizing\nsentence vagueness. Our experimental results demonstrate the effectiveness of\nproposed approaches. Finally, we provide suggestions for resolving vagueness\nand improving the usability of privacy policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06226v1",
            "date": "2018-08-19 16:49:06+00:00",
            "title": "SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing",
            "authors": [
                "Taku Kudo",
                "John Richardson"
            ],
            "abstract": "This paper describes SentencePiece, a language-independent subword tokenizer\nand detokenizer designed for Neural-based text processing, including Neural\nMachine Translation. It provides open-source C++ and Python implementations for\nsubword units. While existing subword segmentation tools assume that the input\nis pre-tokenized into word sequences, SentencePiece can train subword models\ndirectly from raw sentences, which allows us to make a purely end-to-end and\nlanguage independent system. We perform a validation experiment of NMT on\nEnglish-Japanese machine translation, and find that it is possible to achieve\ncomparable accuracy to direct subword training from raw sentences. We also\ncompare the performance of subword training and segmentation with various\nconfigurations. SentencePiece is available under the Apache 2 license at\nhttps://github.com/google/sentencepiece.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06232v1",
            "date": "2018-08-19 17:45:51+00:00",
            "title": "Lexicosyntactic Inference in Neural Models",
            "authors": [
                "Aaron Steven White",
                "Rachel Rudinger",
                "Kyle Rawlins",
                "Benjamin Van Durme"
            ],
            "abstract": "We investigate neural models' ability to capture lexicosyntactic inferences:\ninferences triggered by the interaction of lexical and syntactic information.\nWe take the task of event factuality prediction as a case study and build a\nfactuality judgment dataset for all English clause-embedding verbs in various\nsyntactic contexts. We use this dataset, which we make publicly available, to\nprobe the behavior of current state-of-the-art neural systems, showing that\nthese systems make certain systematic errors that are clearly visible through\nthe lens of factuality prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06267v2",
            "date": "2018-08-19 22:42:06+00:00",
            "title": "Neural Machine Translation of Text from Non-Native Speakers",
            "authors": [
                "Antonios Anastasopoulos",
                "Alison Lui",
                "Toan Nguyen",
                "David Chiang"
            ],
            "abstract": "Neural Machine Translation (NMT) systems are known to degrade when confronted\nwith noisy data, especially when the system is trained only on clean data. In\nthis paper, we show that augmenting training data with sentences containing\nartificially-introduced grammatical errors can make the system more robust to\nsuch errors. In combination with an automatic grammar error correction system,\nwe can recover 1.5 BLEU out of 2.4 BLEU lost due to grammatical errors. We also\npresent a set of Spanish translations of the JFLEG grammar error correction\ncorpus, which allows for testing NMT robustness to real grammatical errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06289v1",
            "date": "2018-08-20 02:36:55+00:00",
            "title": "Multi-Perspective Context Aggregation for Semi-supervised Cloze-style Reading Comprehension",
            "authors": [
                "Liang Wang",
                "Sujian Li",
                "Wei Zhao",
                "Kewei Shen",
                "Meng Sun",
                "Ruoyu Jia",
                "Jingming Liu"
            ],
            "abstract": "Cloze-style reading comprehension has been a popular task for measuring the\nprogress of natural language understanding in recent years. In this paper, we\ndesign a novel multi-perspective framework, which can be seen as the joint\ntraining of heterogeneous experts and aggregate context information from\ndifferent perspectives. Each perspective is modeled by a simple aggregation\nmodule. The outputs of multiple aggregation modules are fed into a one-timestep\npointer network to get the final answer. At the same time, to tackle the\nproblem of insufficient labeled data, we propose an efficient sampling\nmechanism to automatically generate more training examples by matching the\ndistribution of candidates between labeled and unlabeled data. We conduct our\nexperiments on a recently released cloze-test dataset CLOTH (Xie et al., 2017),\nwhich consists of nearly 100k questions designed by professional teachers.\nResults show that our method achieves new state-of-the-art performance over\nprevious strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06304v2",
            "date": "2018-08-20 04:39:58+00:00",
            "title": "Question Generation from SQL Queries Improves Neural Semantic Parsing",
            "authors": [
                "Daya Guo",
                "Yibo Sun",
                "Duyu Tang",
                "Nan Duan",
                "Jian Yin",
                "Hong Chi",
                "James Cao",
                "Peng Chen",
                "Ming Zhou"
            ],
            "abstract": "We study how to learn a semantic parser of state-of-the-art accuracy with\nless supervised training data. We conduct our study on WikiSQL, the largest\nhand-annotated semantic parsing dataset to date. First, we demonstrate that\nquestion generation is an effective method that empowers us to learn a\nstate-of-the-art neural network based semantic parser with thirty percent of\nthe supervised training data. Second, we show that applying question generation\nto the full supervised training data further improves the state-of-the-art\nmodel. In addition, we observe that there is a logarithmic relationship between\nthe accuracy of a semantic parser and the amount of training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06305v3",
            "date": "2018-08-20 04:51:33+00:00",
            "title": "Post-Processing of Word Representations via Variance Normalization and Dynamic Embedding",
            "authors": [
                "Bin Wang",
                "Fenxiao Chen",
                "Angela Wang",
                "C. -C. Jay Kuo"
            ],
            "abstract": "Although embedded vector representations of words offer impressive\nperformance on many natural language processing (NLP) applications, the\ninformation of ordered input sequences is lost to some extent if only\ncontext-based samples are used in the training. For further performance\nimprovement, two new post-processing techniques, called post-processing via\nvariance normalization (PVN) and post-processing via dynamic embedding (PDE),\nare proposed in this work. The PVN method normalizes the variance of principal\ncomponents of word vectors while the PDE method learns orthogonal latent\nvariables from ordered input sequences. The PVN and the PDE methods can be\nintegrated to achieve better performance. We apply these post-processing\ntechniques to two popular word embedding methods (i.e., word2vec and GloVe) to\nyield their post-processed representations. Extensive experiments are conducted\nto demonstrate the effectiveness of the proposed post-processing techniques.",
            "journal ref": "2019 International Conference on Multimedia and Expo",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06528v1",
            "date": "2018-08-20 15:53:32+00:00",
            "title": "Adaptive Document Retrieval for Deep Question Answering",
            "authors": [
                "Bernhard Kratzwald",
                "Stefan Feuerriegel"
            ],
            "abstract": "State-of-the-art systems in deep question answering proceed as follows: (1)\nan initial document retrieval selects relevant documents, which (2) are then\nprocessed by a neural network in order to extract the final answer. Yet the\nexact interplay between both components is poorly understood, especially\nconcerning the number of candidate documents that should be retrieved. We show\nthat choosing a static number of documents -- as used in prior research --\nsuffers from a noise-information trade-off and yields suboptimal results. As a\nremedy, we propose an adaptive document retrieval model. This learns the\noptimal candidate number for document retrieval, conditional on the size of the\ncorpus and the query. We report extensive experimental results showing that our\nadaptive approach outperforms state-of-the-art methods on multiple benchmark\ndatasets, as well as in the context of corpora with variable sizes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06696v4",
            "date": "2018-08-20 21:06:01+00:00",
            "title": "Watset: Local-Global Graph Clustering with Applications in Sense and Frame Induction",
            "authors": [
                "Dmitry Ustalov",
                "Alexander Panchenko",
                "Chris Biemann",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "We present a detailed theoretical and computational analysis of the Watset\nmeta-algorithm for fuzzy graph clustering, which has been found to be widely\napplicable in a variety of domains. This algorithm creates an intermediate\nrepresentation of the input graph that reflects the \"ambiguity\" of its nodes.\nThen, it uses hard clustering to discover clusters in this \"disambiguated\"\nintermediate graph. After outlining the approach and analyzing its\ncomputational complexity, we demonstrate that Watset shows competitive results\nin three applications: unsupervised synset induction from a synonymy graph,\nunsupervised semantic frame induction from dependency triples, and unsupervised\nsemantic class induction from a distributional thesaurus. Our algorithm is\ngeneric and can be also applied to other networks of linguistic data.",
            "journal ref": "Computational Linguistics 45:3 (2019) 423-479",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06729v3",
            "date": "2018-08-21 01:18:37+00:00",
            "title": "You Shall Know the Most Frequent Sense by the Company it Keeps",
            "authors": [
                "Bradley Hauer",
                "Yixing Luan",
                "Grzegorz Kondrak"
            ],
            "abstract": "Identification of the most frequent sense of a polysemous word is an\nimportant semantic task. We introduce two concepts that can benefit MFS\ndetection: companions, which are the most frequently co-occurring words, and\nthe most frequent translation in a bitext. We present two novel methods that\nincorporate these new concepts, and show that they advance the state of the art\non MFS detection.",
            "journal ref": "Proceedings of IEEE ICSC 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06738v2",
            "date": "2018-08-21 02:15:05+00:00",
            "title": "Neural Relation Extraction via Inner-Sentence Noise Reduction and Transfer Learning",
            "authors": [
                "Tianyi Liu",
                "Xinsong Zhang",
                "Wanhao Zhou",
                "Weijia Jia"
            ],
            "abstract": "Extracting relations is critical for knowledge base completion and\nconstruction in which distant supervised methods are widely used to extract\nrelational facts automatically with the existing knowledge bases. However, the\nautomatically constructed datasets comprise amounts of low-quality sentences\ncontaining noisy words, which is neglected by current distant supervised\nmethods resulting in unacceptable precisions. To mitigate this problem, we\npropose a novel word-level distant supervised approach for relation extraction.\nWe first build Sub-Tree Parse(STP) to remove noisy words that are irrelevant to\nrelations. Then we construct a neural network inputting the sub-tree while\napplying the entity-wise attention to identify the important semantic features\nof relational words in each instance. To make our model more robust against\nnoisy words, we initialize our network with a priori knowledge learned from the\nrelevant task of entity classification by transfer learning. We conduct\nextensive experiments using the corpora of New York Times(NYT) and Freebase.\nExperiments show that our approach is effective and improves the area of\nPrecision/Recall(PR) from 0.35 to 0.39 over the state-of-the-art work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06752v2",
            "date": "2018-08-21 04:00:23+00:00",
            "title": "Lessons from Natural Language Inference in the Clinical Domain",
            "authors": [
                "Alexey Romanov",
                "Chaitanya Shivade"
            ],
            "abstract": "State of the art models using deep neural networks have become very good in\nlearning an accurate mapping from inputs to outputs. However, they still lack\ngeneralization capabilities in conditions that differ from the ones encountered\nduring training. This is even more challenging in specialized, and knowledge\nintensive domains, where training data is limited. To address this gap, we\nintroduce MedNLI - a dataset annotated by doctors, performing a natural\nlanguage inference task (NLI), grounded in the medical history of patients. We\npresent strategies to: 1) leverage transfer learning using datasets from the\nopen domain, (e.g. SNLI) and 2) incorporate domain knowledge from external data\nand lexical sources (e.g. medical terminologies). Our results demonstrate\nperformance gains using both strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06773v2",
            "date": "2018-08-21 05:38:32+00:00",
            "title": "Semi-Supervised Learning for Neural Keyphrase Generation",
            "authors": [
                "Hai Ye",
                "Lu Wang"
            ],
            "abstract": "We study the problem of generating keyphrases that summarize the key points\nfor a given document. While sequence-to-sequence (seq2seq) models have achieved\nremarkable performance on this task (Meng et al., 2017), model training often\nrelies on large amounts of labeled data, which is only applicable to\nresource-rich domains. In this paper, we propose semi-supervised keyphrase\ngeneration methods by leveraging both labeled data and large-scale unlabeled\nsamples for learning. Two strategies are proposed. First, unlabeled documents\nare first tagged with synthetic keyphrases obtained from unsupervised keyphrase\nextraction methods or a selflearning algorithm, and then combined with labeled\nsamples for training. Furthermore, we investigate a multi-task learning\nframework to jointly learn to generate keyphrases as well as the titles of the\narticles. Experimental results show that our semi-supervised learning-based\nmethods outperform a state-of-the-art model trained with labeled data only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06810v2",
            "date": "2018-08-21 09:11:11+00:00",
            "title": "The Influence of Down-Sampling Strategies on SVD Word Embedding Stability",
            "authors": [
                "Johannes Hellrich",
                "Bernd Kampe",
                "Udo Hahn"
            ],
            "abstract": "The stability of word embedding algorithms, i.e., the consistency of the word\nrepresentations they reveal when trained repeatedly on the same data set, has\nrecently raised concerns. We here compare word embedding algorithms on three\ncorpora of different sizes, and evaluate both their stability and accuracy. We\nfind strong evidence that down-sampling strategies (used as part of their\ntraining procedures) are particularly influential for the stability of\nSVDPPMI-type embeddings. This finding seems to explain diverging reports on\ntheir stability and lead us to a simple modification which provides superior\nstability as well as accuracy on par with skip-gram embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06826v2",
            "date": "2018-08-21 10:07:18+00:00",
            "title": "Measuring Semantic Abstraction of Multilingual NMT with Paraphrase Recognition and Generation Tasks",
            "authors": [
                "J\u00f6rg Tiedemann",
                "Yves Scherrer"
            ],
            "abstract": "In this paper, we investigate whether multilingual neural translation models\nlearn stronger semantic abstractions of sentences than bilingual ones. We test\nthis hypotheses by measuring the perplexity of such models when applied to\nparaphrases of the source language. The intuition is that an encoder produces\nbetter representations if a decoder is capable of recognizing synonymous\nsentences in the same language even though the model is never trained for that\ntask. In our setup, we add 16 different auxiliary languages to a bidirectional\nbilingual baseline model (English-French) and test it with in-domain and\nout-of-domain paraphrases in English. The results show that the perplexity is\nsignificantly reduced in each of the cases, indicating that meaning can be\ngrounded in translation. This is further supported by a study on paraphrase\ngeneration that we also include at the end of the paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06834v1",
            "date": "2018-08-21 10:25:30+00:00",
            "title": "Analysis of Speeches in Indian Parliamentary Debates",
            "authors": [
                "Sakala Venkata Krishna Rohit",
                "Navjyoti Singh"
            ],
            "abstract": "With the increasing usage of the internet, more and more data is being\ndigitized including parliamentary debates but they are in an unstructured\nformat. There is a need to convert them into a structured format for linguistic\nanalysis. Much work has been done on parliamentary data such as Hansard,\nAmerican congressional floor-debate data on various aspects but less on\npragmatics. In this paper, we provide a dataset for the synopsis of Indian\nparliamentary debates and perform stance classification of speeches i.e\nidentifying if the speaker is supporting the bill/issue or against it. We also\nanalyze the intention of the speeches beyond mere sentences i.e pragmatics in\nthe parliament. Based on thorough manual analysis of the debates, we developed\nan annotation scheme of 4 mutually exclusive categories to analyze the purpose\nof the speeches: to find out ISSUES, to BLAME, to APPRECIATE and for CALL FOR\nACTION. We have annotated the dataset provided, with these 4 categories and\nconducted preliminary experiments for automatic detection of the categories.\nOur automated classification approach gave us promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06853v1",
            "date": "2018-08-21 11:37:57+00:00",
            "title": "Demonstrating PAR4SEM - A Semantic Writing Aid with Adaptive Paraphrasing",
            "authors": [
                "Seid Muhie Yimam",
                "Chris Biemann"
            ],
            "abstract": "In this paper, we present Par4Sem, a semantic writing aid tool based on\nadaptive paraphrasing. Unlike many annotation tools that are primarily used to\ncollect training examples, Par4Sem is integrated into a real word application,\nin this case a writing aid tool, in order to collect training examples from\nusage data. Par4Sem is a tool, which supports an adaptive, iterative, and\ninteractive process where the underlying machine learning models are updated\nfor each iteration using new training examples from usage data. After\nmotivating the use of ever-learning tools in NLP applications, we evaluate\nPar4Sem by adopting it to a text simplification task through mere usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06876v3",
            "date": "2018-08-21 12:52:03+00:00",
            "title": "Adversarial training for multi-context joint entity and relation extraction",
            "authors": [
                "Giannis Bekoulis",
                "Johannes Deleu",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "Adversarial training (AT) is a regularization method that can be used to\nimprove the robustness of neural network methods by adding small perturbations\nin the training data. We show how to use AT for the tasks of entity recognition\nand relation extraction. In particular, we demonstrate that applying AT to a\ngeneral purpose baseline model for jointly extracting entities and relations,\nallows improving the state-of-the-art effectiveness on several datasets in\ndifferent contexts (i.e., news, biomedical, and real estate data) and for\ndifferent languages (English and Dutch).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06945v2",
            "date": "2018-08-21 14:55:34+00:00",
            "title": "A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation",
            "authors": [
                "Jingjing Xu",
                "Xuancheng Ren",
                "Yi Zhang",
                "Qi Zeng",
                "Xiaoyan Cai",
                "Xu Sun"
            ],
            "abstract": "Narrative story generation is a challenging problem because it demands the\ngenerated sentences with tight semantic connections, which has not been well\nstudied by most existing generative models. To address this problem, we propose\na skeleton-based model to promote the coherence of generated stories. Different\nfrom traditional models that generate a complete sentence at a stroke, the\nproposed model first generates the most critical phrases, called skeleton, and\nthen expands the skeleton to a complete and fluent sentence. The skeleton is\nnot manually defined, but learned by a reinforcement learning method. Compared\nto the state-of-the-art models, our skeleton-based model can generate\nsignificantly more coherent text according to human evaluation and automatic\nevaluation. The G-score is improved by 20.1% in the human evaluation. The code\nis available at https://github.com/lancopku/Skeleton-Based-Generation-Model",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07016v7",
            "date": "2018-08-21 16:59:39+00:00",
            "title": "Gaussian Word Embedding with a Wasserstein Distance Loss",
            "authors": [
                "Chi Sun",
                "Hang Yan",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Compared with word embedding based on point representation,\ndistribution-based word embedding shows more flexibility in expressing\nuncertainty and therefore embeds richer semantic information when representing\nwords. The Wasserstein distance provides a natural notion of dissimilarity with\nprobability measures and has a closed-form solution when measuring the distance\nbetween two Gaussian distributions. Therefore, with the aim of representing\nwords in a highly efficient way, we propose to operate a Gaussian word\nembedding model with a loss function based on the Wasserstein distance. Also,\nexternal information from ConceptNet will be used to semi-supervise the results\nof the Gaussian word embedding. Thirteen datasets from the word similarity\ntask, together with one from the word entailment task, and six datasets from\nthe downstream document classification task will be evaluated in this paper to\ntest our hypothesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07046v1",
            "date": "2018-08-21 17:58:05+00:00",
            "title": "ISNA-Set: A novel English Corpus of Iran NEWS",
            "authors": [
                "Mohammad Kamel",
                "Hadi Sadoghi-Yazdi"
            ],
            "abstract": "News agencies publish news on their websites all over the world. Moreover,\ncreating novel corpuses is necessary to bring natural processing to new\ndomains. Textual processing of online news is challenging in terms of the\nstrategy of collecting data, the complex structure of news websites, and\nselecting or designing suitable algorithms for processing these types of data.\nDespite the previous works which focus on creating corpuses for Iran news in\nPersian, in this paper, we introduce a new corpus for English news of a\nnational news agency. ISNA-Set is a new dataset of English news of Iranian\nStudents News Agency (ISNA), as one of the most famous news agencies in Iran.\nWe statistically analyze the data and the sentiments of news, and also extract\nentities and part-of-speech tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07048v1",
            "date": "2018-08-21 17:58:21+00:00",
            "title": "Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation",
            "authors": [
                "Samuel L\u00e4ubli",
                "Rico Sennrich",
                "Martin Volk"
            ],
            "abstract": "Recent research suggests that neural machine translation achieves parity with\nprofessional human translation on the WMT Chinese--English news translation\ntask. We empirically test this claim with alternative evaluation protocols,\ncontrasting the evaluation of single sentences and entire documents. In a\npairwise ranking experiment, human raters assessing adequacy and fluency show a\nstronger preference for human over machine translation when evaluating\ndocuments as compared to isolated sentences. Our findings emphasise the need to\nshift towards document-level evaluation as machine translation improves to the\ndegree that errors which are hard or impossible to spot at the sentence-level\nbecome decisive in discriminating quality of different translation outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07118v1",
            "date": "2018-08-21 20:22:09+00:00",
            "title": "Language Identification in Code-Mixed Data using Multichannel Neural Networks and Context Capture",
            "authors": [
                "Soumil Mandal",
                "Anil Kumar Singh"
            ],
            "abstract": "An accurate language identification tool is an absolute necessity for\nbuilding complex NLP systems to be used on code-mixed data. Lot of work has\nbeen recently done on the same, but there's still room for improvement.\nInspired from the recent advancements in neural network architectures for\ncomputer vision tasks, we have implemented multichannel neural networks\ncombining CNN and LSTM for word level language identification of code-mixed\ndata. Combining this with a Bi-LSTM-CRF context capture module, accuracies of\n93.28% and 93.32% is achieved on our two testing sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07185v1",
            "date": "2018-08-22 02:06:08+00:00",
            "title": "Keyphrase Generation with Correlation Constraints",
            "authors": [
                "Jun Chen",
                "Xiaoming Zhang",
                "Yu Wu",
                "Zhao Yan",
                "Zhoujun Li"
            ],
            "abstract": "In this paper, we study automatic keyphrase generation. Although conventional\napproaches to this task show promising results, they neglect correlation among\nkeyphrases, resulting in duplication and coverage issues. To solve these\nproblems, we propose a new sequence-to-sequence architecture for keyphrase\ngeneration named CorrRNN, which captures correlation among multiple keyphrases\nin two ways. First, we employ a coverage vector to indicate whether the word in\nthe source document has been summarized by previous phrases to improve the\ncoverage for keyphrases. Second, preceding phrases are taken into account to\neliminate duplicate phrases and improve result coherence. Experiment results\nshow that our model significantly outperforms the state-of-the-art method on\nbenchmark datasets in terms of both accuracy and diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07214v2",
            "date": "2018-08-22 04:15:38+00:00",
            "title": "A Characterwise Windowed Approach to Hebrew Morphological Segmentation",
            "authors": [
                "Amir Zeldes"
            ],
            "abstract": "This paper presents a novel approach to the segmentation of orthographic word\nforms in contemporary Hebrew, focusing purely on splitting without carrying out\nmorphological analysis or disambiguation. Casting the analysis task as\ncharacter-wise binary classification and using adjacent character and\nword-based lexicon-lookup features, this approach achieves over 98% accuracy on\nthe benchmark SPMRL shared task data for Hebrew, and 97% accuracy on a new out\nof domain Wikipedia dataset, an improvement of ~4% and 5% over previous state\nof the art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07231v1",
            "date": "2018-08-22 06:00:56+00:00",
            "title": "Reducing Gender Bias in Abusive Language Detection",
            "authors": [
                "Ji Ho Park",
                "Jamin Shin",
                "Pascale Fung"
            ],
            "abstract": "Abusive language detection models tend to have a problem of being biased\ntoward identity words of a certain group of people because of imbalanced\ntraining datasets. For example, \"You are a good woman\" was considered \"sexist\"\nwhen trained on an existing dataset. Such model bias is an obstacle for models\nto be robust enough for practical use. In this work, we measure gender biases\non models trained with different abusive language datasets, while analyzing the\neffect of different pre-trained word embeddings and model architectures. We\nalso experiment with three bias mitigation methods: (1) debiased word\nembeddings, (2) gender swap data augmentation, and (3) fine-tuning with a\nlarger corpus. These methods can effectively reduce gender bias by 90-98% and\ncan be extended to correct model bias in other scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07235v1",
            "date": "2018-08-22 06:07:24+00:00",
            "title": "Finding Good Representations of Emotions for Text Classification",
            "authors": [
                "Ji Ho Park"
            ],
            "abstract": "It is important for machines to interpret human emotions properly for better\nhuman-machine communications, as emotion is an essential part of human-to-human\ncommunications. One aspect of emotion is reflected in the language we use. How\nto represent emotions in texts is a challenge in natural language processing\n(NLP). Although continuous vector representations like word2vec have become the\nnew norm for NLP problems, their limitations are that they do not take emotions\ninto consideration and can unintentionally contain bias toward certain\nidentities like different genders.\n  This thesis focuses on improving existing representations in both word and\nsentence levels by explicitly taking emotions inside text and model bias into\naccount in their training process. Our improved representations can help to\nbuild more robust machine learning models for affect-related text\nclassification like sentiment/emotion analysis and abusive language detection.\n  We first propose representations called emotional word vectors (EVEC), which\nis learned from a convolutional neural network model with an emotion-labeled\ncorpus, which is constructed using hashtags. Secondly, we extend to learning\nsentence-level representations with a huge corpus of texts with the pseudo task\nof recognizing emojis. Our results show that, with the representations trained\nfrom millions of tweets with weakly supervised labels such as hashtags and\nemojis, we can solve sentiment/emotion analysis tasks more effectively.\n  Lastly, as examples of model bias in representations of existing approaches,\nwe explore a specific problem of automatic detection of abusive language. We\naddress the issue of gender bias in various neural network models by conducting\nexperiments to measure and reduce those biases in the representations in order\nto build more robust classification models.",
            "journal ref": "HKUST MPhil Thesis, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07244v2",
            "date": "2018-08-22 06:58:01+00:00",
            "title": "Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection",
            "authors": [
                "Chongyang Tao",
                "Wei Wu",
                "Can Xu",
                "Yansong Feng",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "In this paper, we study context-response matching with pre-trained\ncontextualized representations for multi-turn response selection in\nretrieval-based chatbots. Existing models, such as Cove and ELMo, are trained\nwith limited context (often a single sentence or paragraph), and may not work\nwell on multi-turn conversations, due to the hierarchical nature, informal\nlanguage, and domain-specific words. To address the challenges, we propose\npre-training hierarchical contextualized representations, including contextual\nword-level and sentence-level representations, by learning a dialogue\ngeneration model from large-scale conversations with a hierarchical\nencoder-decoder architecture. Then the two levels of representations are\nblended into the input and output layer of a matching model respectively.\nExperimental results on two benchmark conversation datasets indicate that the\nproposed hierarchical contextualized representations can bring significantly\nand consistently improvement to existing matching models for response\nselection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07290v2",
            "date": "2018-08-22 09:18:38+00:00",
            "title": "The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers",
            "authors": [
                "Dongxiang Zhang",
                "Lei Wang",
                "Luming Zhang",
                "Bing Tian Dai",
                "Heng Tao Shen"
            ],
            "abstract": "Solving mathematical word problems (MWPs) automatically is challenging,\nprimarily due to the semantic gap between human-readable words and\nmachine-understandable logics. Despite the long history dated back to the1960s,\nMWPs have regained intensive attention in the past few years with the\nadvancement of Artificial Intelligence (AI). Solving MWPs successfully is\nconsidered as a milestone towards general AI. Many systems have claimed\npromising results in self-crafted and small-scale datasets. However, when\napplied on large and diverse datasets, none of the proposed methods in the\nliterature achieves high precision, revealing that current MWP solvers still\nhave much room for improvement. This motivated us to present a comprehensive\nsurvey to deliver a clear and complete picture of automatic math problem\nsolvers. In this survey, we emphasize on algebraic word problems, summarize\ntheir extracted features and proposed techniques to bridge the semantic gap and\ncompare their performance in the publicly accessible datasets. We also cover\nautomatic solvers for other types of math problems such as geometric problems\nthat require the understanding of diagrams. Finally, we identify several\nemerging research directions for the readers with interests in MWPs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07311v1",
            "date": "2018-08-22 10:57:04+00:00",
            "title": "Learning Sentiment Memories for Sentiment Modification without Parallel Data",
            "authors": [
                "Yi Zhang",
                "Jingjing Xu",
                "Pengcheng Yang",
                "Xu Sun"
            ],
            "abstract": "The task of sentiment modification requires reversing the sentiment of the\ninput and preserving the sentiment-independent content. However, aligned\nsentences with the same content but different sentiments are usually\nunavailable. Due to the lack of such parallel data, it is hard to extract\nsentiment independent content and reverse the sentiment in an unsupervised way.\nPrevious work usually can not reconcile sentiment transformation and content\npreservation. In this paper, motivated by the fact the non-emotional context\n(e.g., \"staff\") provides strong cues for the occurrence of emotional words\n(e.g., \"friendly\"), we propose a novel method that automatically extracts\nappropriate sentiment information from learned sentiment memories according to\nspecific context. Experiments show that our method substantially improves the\ncontent preservation degree and achieves the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07337v3",
            "date": "2018-08-22 12:49:46+00:00",
            "title": "Expansional Retrofitting for Word Vector Enrichment",
            "authors": [
                "Hwiyeol Jo"
            ],
            "abstract": "Retrofitting techniques, which inject external resources into word\nrepresentations, have compensated the weakness of distributed representations\nin semantic and relational knowledge between words. Implicitly retrofitting\nword vectors by expansional technique outperforms retrofitting in word\nsimilarity tasks with word vector generalization. In this paper, we propose\nunsupervised extrofitting: expansional retrofitting (extrofitting) without\nexternal semantic lexicons. We also propose deep extrofitting: in-depth\nstacking of extrofitting and further combinations of extrofitting with\nretrofitting. When experimenting with GloVe, we show that our methods\noutperform the previous methods on most of word similarity tasks while\nrequiring only synonyms as an external resource. Lastly, we show the effect of\nword vector enrichment on text classification task, as a downstream task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07364v3",
            "date": "2018-08-22 13:56:51+00:00",
            "title": "Neural Named Entity Recognition from Subword Units",
            "authors": [
                "Abdalghani Abujabal",
                "Judith Gaspers"
            ],
            "abstract": "Named entity recognition (NER) is a vital task in spoken language\nunderstanding, which aims to identify mentions of named entities in text e.g.,\nfrom transcribed speech. Existing neural models for NER rely mostly on\ndedicated word-level representations, which suffer from two main shortcomings.\nFirst, the vocabulary size is large, yielding large memory requirements and\ntraining time. Second, these models are not able to learn morphological or\nphonological representations. To remedy the above shortcomings, we adopt a\nneural solution based on bidirectional LSTMs and conditional random fields,\nwhere we rely on subword units, namely characters, phonemes, and bytes. For\neach word in an utterance, our model learns a representation from each of the\nsubword units. We conducted experiments in a real-world large-scale setting for\nthe use case of a voice-controlled device covering four languages with up to\n5.5M utterances per language. Our experiments show that (1) with increasing\ntraining data, performance of models trained solely on subword units becomes\ncloser to that of models with dedicated word-level embeddings (91.35 vs 93.92\nF1 for English), while using a much smaller vocabulary size (332 vs 74K), (2)\nsubword units enhance models with dedicated word-level embeddings, and (3)\ncombining different subword units improves performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07512v2",
            "date": "2018-08-22 18:26:43+00:00",
            "title": "SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation",
            "authors": [
                "Xinyi Wang",
                "Hieu Pham",
                "Zihang Dai",
                "Graham Neubig"
            ],
            "abstract": "In this work, we examine methods for data augmentation for text-based tasks\nsuch as neural machine translation (NMT). We formulate the design of a data\naugmentation policy with desirable properties as an optimization problem, and\nderive a generic analytic solution. This solution not only subsumes some\nexisting augmentation schemes, but also leads to an extremely simple data\naugmentation strategy for NMT: randomly replacing words in both the source\nsentence and the target sentence with other random words from their\ncorresponding vocabularies. We name this method SwitchOut. Experiments on three\ntranslation datasets of different scales show that SwitchOut yields consistent\nimprovements of about 0.5 BLEU, achieving better or comparable performances to\nstrong alternatives such as word dropout (Sennrich et al., 2016a). Code to\nimplement this method is included in the appendix.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07531v2",
            "date": "2018-08-22 19:21:04+00:00",
            "title": "Sarcasm Analysis using Conversation Context",
            "authors": [
                "Debanjan Ghosh",
                "Alexander R. Fabbri",
                "Smaranda Muresan"
            ],
            "abstract": "Computational models for sarcasm detection have often relied on the content\nof utterances in isolation. However, the speaker's sarcastic intent is not\nalways apparent without additional context. Focusing on social media\ndiscussions, we investigate three issues: (1) does modeling conversation\ncontext help in sarcasm detection; (2) can we identify what part of\nconversation context triggered the sarcastic reply; and (3) given a sarcastic\npost that contains multiple sentences, can we identify the specific sentence\nthat is sarcastic. To address the first issue, we investigate several types of\nLong Short-Term Memory (LSTM) networks that can model both the conversation\ncontext and the current turn. We show that LSTM networks with sentence-level\nattention on context and current turn, as well as the conditional LSTM network\n(Rocktaschel et al. 2016), outperform the LSTM model that reads only the\ncurrent turn. As conversation context, we consider the prior turn, the\nsucceeding turn or both. Our computational models are tested on two types of\nsocial media platforms: Twitter and discussion forums. We discuss several\ndifferences between these datasets ranging from their size to the nature of the\ngold-label annotations. To address the last two issues, we present a\nqualitative analysis of attention weights produced by the LSTM models (with\nattention) and discuss the results compared with human performance on the two\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07599v1",
            "date": "2018-08-23 00:37:58+00:00",
            "title": "Structured Interpretation of Temporal Relations",
            "authors": [
                "Yuchen Zhang",
                "Nianwen Xue"
            ],
            "abstract": "Temporal relations between events and time expressions in a document are\noften modeled in an unstructured manner where relations between individual\npairs of time expressions and events are considered in isolation. This often\nresults in inconsistent and incomplete annotation and computational modeling.\nWe propose a novel annotation approach where events and time expressions in a\ndocument form a dependency tree in which each dependency relation corresponds\nto an instance of temporal anaphora where the antecedent is the parent and the\nanaphor is the child. We annotate a corpus of 235 documents using this approach\nin the two genres of news and narratives, with 48 documents doubly annotated.\nWe report a stable and high inter-annotator agreement on the doubly annotated\nsubset, validating our approach, and perform a quantitative comparison between\nthe two genres of the entire corpus. We make this corpus publicly available.",
            "journal ref": "Yuchen Zhang and Nianwen Xue. 2018. Structured Interpretation of\n  Temporal Relations. In Proceedings of the 11th Language Resources and\n  Evaluation Conference (LREC-2018), Miyazaki, Japan",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07604v1",
            "date": "2018-08-23 02:02:32+00:00",
            "title": "Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations",
            "authors": [
                "Guangxiang Zhao",
                "Jingjing Xu",
                "Qi Zeng",
                "Xuancheng Ren"
            ],
            "abstract": "This paper explores a new natural language processing task, review-driven\nmulti-label music style classification. This task requires the system to\nidentify multiple styles of music based on its reviews on websites. The biggest\nchallenge lies in the complicated relations of music styles. It has brought\nfailure to many multi-label classification methods. To tackle this problem, we\npropose a novel deep learning approach to automatically learn and exploit style\ncorrelations. The proposed method consists of two parts: a label-graph based\nneural network, and a soft training mechanism with correlation-based continuous\nlabel representation. Experimental results show that our approach achieves\nlarge improvements over the baselines on the proposed dataset. Especially, the\nmicro F1 is improved from 53.9 to 64.5, and the one-error is reduced from 30.5\nto 22.6. Furthermore, the visualized analysis shows that our approach performs\nwell in capturing style correlations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07625v1",
            "date": "2018-08-23 04:03:58+00:00",
            "title": "Weakly-supervised Neural Semantic Parsing with a Generative Ranker",
            "authors": [
                "Jianpeng Cheng",
                "Mirella Lapata"
            ],
            "abstract": "Weakly-supervised semantic parsers are trained on utterance-denotation pairs,\ntreating logical forms as latent. The task is challenging due to the large\nsearch space and spuriousness of logical forms. In this paper we introduce a\nneural parser-ranker system for weakly-supervised semantic parsing. The parser\ngenerates candidate tree-structured logical forms from utterances using clues\nof denotations. These candidates are then ranked based on two criterion: their\nlikelihood of executing to the correct denotation, and their agreement with the\nutterance semantics. We present a scheduled training procedure to balance the\ncontribution of the two objectives. Furthermore, we propose to use a neurally\nencoded lexicon to inject prior domain knowledge to the model. Experiments on\nthree Freebase datasets demonstrate the effectiveness of our semantic parser,\nachieving results within the state-of-the-art range.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07644v4",
            "date": "2018-08-23 06:27:58+00:00",
            "title": "Attention-Guided Answer Distillation for Machine Reading Comprehension",
            "authors": [
                "Minghao Hu",
                "Yuxing Peng",
                "Furu Wei",
                "Zhen Huang",
                "Dongsheng Li",
                "Nan Yang",
                "Ming Zhou"
            ],
            "abstract": "Despite that current reading comprehension systems have achieved significant\nadvancements, their promising performances are often obtained at the cost of\nmaking an ensemble of numerous models. Besides, existing approaches are also\nvulnerable to adversarial attacks. This paper tackles these problems by\nleveraging knowledge distillation, which aims to transfer knowledge from an\nensemble model to a single model. We first demonstrate that vanilla knowledge\ndistillation applied to answer span prediction is effective for reading\ncomprehension systems. We then propose two novel approaches that not only\npenalize the prediction on confusing answers but also guide the training with\nalignment information distilled from the ensemble. Experiments show that our\nbest student model has only a slight drop of 0.4% F1 on the SQuAD test set\ncompared to the ensemble teacher, while running 12x faster during inference. It\neven outperforms the teacher on adversarial SQuAD datasets and NarrativeQA\nbenchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07674v1",
            "date": "2018-08-23 09:41:17+00:00",
            "title": "Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification",
            "authors": [
                "Wajdi Zaghouani",
                "Anis Charfi"
            ],
            "abstract": "In this paper, we present Arap-Tweet, which is a large-scale and\nmulti-dialectal corpus of Tweets from 11 regions and 16 countries in the Arab\nworld representing the major Arabic dialectal varieties. To build this corpus,\nwe collected data from Twitter and we provided a team of experienced annotators\nwith annotation guidelines that they used to annotate the corpus for age\ncategories, gender, and dialectal variety. During the data collection effort,\nwe based our search on distinctive keywords that are specific to the different\nArabic dialects and we also validated the location using Twitter API. In this\npaper, we report on the corpus data collection and annotation efforts. We also\npresent some issues that we encountered during these phases. Then, we present\nthe results of the evaluation performed to ensure the consistency of the\nannotation. The provided corpus will enrich the limited set of available\nlanguage resources for Arabic and will be an invaluable enabler for developing\nauthor profiling tools and NLP tools for Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07678v1",
            "date": "2018-08-23 09:52:26+00:00",
            "title": "Guidelines and Annotation Framework for Arabic Author Profiling",
            "authors": [
                "Wajdi Zaghouani",
                "Anis Charfi"
            ],
            "abstract": "In this paper, we present the annotation pipeline and the guidelines we wrote\nas part of an effort to create a large manually annotated Arabic author\nprofiling dataset from various social media sources covering 16 Arabic\ncountries and 11 dialectal regions. The target size of the annotated ARAP-Tweet\ncorpus is more than 2.4 million words. We illustrate and summarize our general\nand dialect-specific guidelines for each of the dialectal regions selected. We\nalso present the annotation framework and logistics. We control the annotation\nquality frequently by computing the inter-annotator agreement during the\nannotation process. Finally, we describe the issues encountered during the\nannotation phase, especially those related to the peculiarities of Arabic\ndialectal varieties as used in social media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07688v2",
            "date": "2018-08-23 10:13:52+00:00",
            "title": "Role of Intonation in Scoring Spoken English",
            "authors": [
                "Amber Nigam",
                "Arpan Saxena",
                "Ishan Sodhi"
            ],
            "abstract": "In this paper, we have introduced and evaluated intonation based feature for\nscoring the English speech of nonnative English speakers in Indian context. For\nthis, we created an automated spoken English scoring engine to learn from the\nmanual evaluation of spoken English. This involved using an existing Automatic\nSpeech Recognition (ASR) engine to convert the speech to text. Thereafter,\nmacro features like accuracy, fluency and prosodic features were used to build\na scoring model. In the process, we introduced SimIntonation, short for\nsimilarity between spoken intonation pattern and \"ideal\" i.e. training\nintonation pattern. Our results show that it is a highly predictive feature\nunder controlled environment. We also categorized interword pauses into 4\ndistinct types for a granular evaluation of pauses and their impact on speech\nevaluation. Moreover, we took steps to moderate test difficulty through its\nevaluation across parameters like difficult word count, average sentence\nreadability and lexical density. Our results show that macro features like\naccuracy and intonation, and micro features like pause-topography are strongly\npredictive. The scoring of spoken English is not within the purview of this\npaper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07699v2",
            "date": "2018-08-23 11:16:57+00:00",
            "title": "End-to-End Neural Entity Linking",
            "authors": [
                "Nikolaos Kolitsas",
                "Octavian-Eugen Ganea",
                "Thomas Hofmann"
            ],
            "abstract": "Entity Linking (EL) is an essential task for semantic text understanding and\ninformation extraction. Popular methods separately address the Mention\nDetection (MD) and Entity Disambiguation (ED) stages of EL, without leveraging\ntheir mutual dependency. We here propose the first neural end-to-end EL system\nthat jointly discovers and links entities in a text document. The main idea is\nto consider all possible spans as potential mentions and learn contextual\nsimilarity scores over their entity candidates that are useful for both MD and\nED decisions. Key components are context-aware mention embeddings, entity\nembeddings and a probabilistic mention - entity map, without demanding other\nengineered features. Empirically, we show that our end-to-end method\nsignificantly outperforms popular systems on the Gerbil platform when enough\ntraining data is available. Conversely, if testing datasets follow different\nannotation conventions compared to the training set (e.g. queries/ tweets vs\nnews documents), our ED model coupled with a traditional NER system offers the\nbest or second best EL accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07733v1",
            "date": "2018-08-23 13:03:55+00:00",
            "title": "Revisiting the Importance of Encoding Logic Rules in Sentiment Classification",
            "authors": [
                "Kalpesh Krishna",
                "Preethi Jyothi",
                "Mohit Iyyer"
            ],
            "abstract": "We analyze the performance of different sentiment classification models on\nsyntactically complex inputs like A-but-B sentences. The first contribution of\nthis analysis addresses reproducible research: to meaningfully compare\ndifferent models, their accuracies must be averaged over far more random seeds\nthan what has traditionally been reported. With proper averaging in place, we\nnotice that the distillation model described in arXiv:1603.06318v4 [cs.LG],\nwhich incorporates explicit logic rules for sentiment classification, is\nineffective. In contrast, using contextualized ELMo embeddings\n(arXiv:1802.05365v2 [cs.CL]) instead of logic rules yields significantly better\nperformance. Additionally, we provide analysis and visualizations that\ndemonstrate ELMo's ability to implicitly learn logic rules. Finally, a\ncrowdsourced analysis reveals how ELMo outperforms baseline models even on\nsentences with ambiguous sentiment labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07851v1",
            "date": "2018-08-23 17:24:42+00:00",
            "title": "Sentiment Index of the Russian Speaking Facebook",
            "authors": [
                "Alexander Panchenko"
            ],
            "abstract": "A sentiment index measures the average emotional level in a corpus. We\nintroduce four such indexes and use them to gauge average \"positiveness\" of a\npopulation during some period based on posts in a social network. This article\nfor the first time presents a text-, rather than word-based sentiment index.\nFurthermore, this study presents the first large-scale study of the sentiment\nindex of the Russian-speaking Facebook. Our results are consistent with the\nprior experiments for the English language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07894v1",
            "date": "2018-08-23 18:18:32+00:00",
            "title": "Style Transfer as Unsupervised Machine Translation",
            "authors": [
                "Zhirui Zhang",
                "Shuo Ren",
                "Shujie Liu",
                "Jianyong Wang",
                "Peng Chen",
                "Mu Li",
                "Ming Zhou",
                "Enhong Chen"
            ],
            "abstract": "Language style transferring rephrases text with specific stylistic attributes\nwhile preserving the original attribute-independent content. One main challenge\nin learning a style transfer system is a lack of parallel data where the source\nsentence is in one style and the target sentence in another style. With this\nconstraint, in this paper, we adapt unsupervised machine translation methods\nfor the task of automatic style transfer. We first take advantage of\nstyle-preference information and word embedding similarity to produce\npseudo-parallel data with a statistical machine translation (SMT) framework.\nThen the iterative back-translation approach is employed to jointly train two\nneural machine translation (NMT) based transfer systems. To control the noise\ngenerated during joint training, a style classifier is introduced to guarantee\nthe accuracy of style transfer and penalize bad candidates in the generated\npseudo data. Experiments on benchmark datasets show that our proposed method\noutperforms previous state-of-the-art models in terms of both accuracy of style\ntransfer and quality of input-output correspondence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07913v1",
            "date": "2018-08-23 19:19:27+00:00",
            "title": "Improving Abstraction in Text Summarization",
            "authors": [
                "Wojciech Kry\u015bci\u0144ski",
                "Romain Paulus",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Abstractive text summarization aims to shorten long text documents into a\nhuman readable form that contains the most important facts from the original\ndocument. However, the level of actual abstraction as measured by novel phrases\nthat do not appear in the source document remains low in existing approaches.\nWe propose two techniques to improve the level of abstraction of generated\nsummaries. First, we decompose the decoder into a contextual network that\nretrieves relevant parts of the source document, and a pretrained language\nmodel that incorporates prior knowledge about language generation. Second, we\npropose a novelty metric that is optimized directly through policy learning to\nencourage the generation of novel phrases. Our model achieves results\ncomparable to state-of-the-art models, as determined by ROUGE scores and human\nevaluations, while achieving a significantly higher level of abstraction as\nmeasured by n-gram overlap with the source document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07931v1",
            "date": "2018-08-23 20:23:36+00:00",
            "title": "Financial Aspect-Based Sentiment Analysis using Deep Representations",
            "authors": [
                "Steve Yang",
                "Jason Rosenfeld",
                "Jacques Makutonin"
            ],
            "abstract": "The topic of aspect-based sentiment analysis (ABSA) has been explored for a\nvariety of industries, but it still remains much unexplored in finance. The\nrecent release of data for an open challenge (FiQA) from the companion\nproceedings of WWW '18 has provided valuable finance-specific annotations. FiQA\ncontains high quality labels, but it still lacks data quantity to apply\ntraditional ABSA deep learning architecture. In this paper, we employ\nhigh-level semantic representations and methods of inductive transfer learning\nfor NLP. We experiment with extensions of recently developed domain adaptation\nmethods and target task fine-tuning that significantly improve performance on a\nsmall dataset. Our results show an 8.7% improvement in the F1 score for\nclassification and an 11% improvement over the MSE for regression on current\nstate-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07999v1",
            "date": "2018-08-24 03:46:45+00:00",
            "title": "Features of word similarity",
            "authors": [
                "Arthur M. Jacobs",
                "Annette Kinder"
            ],
            "abstract": "In this theoretical note we compare different types of computational models\nof word similarity and association in their ability to predict a set of about\n900 rating data. Using regression and predictive modeling tools (neural net,\ndecision tree) the performance of a total of 28 models using different\ncombinations of both surface and semantic word features is evaluated. The\nresults present evidence for the hypothesis that word similarity ratings are\nbased on more than only semantic relatedness. The limited cross-validated\nperformance of the models asks for the development of psychological process\nmodels of the word similarity rating task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08047v1",
            "date": "2018-08-24 08:32:38+00:00",
            "title": "Role Semantics for Better Models of Implicit Discourse Relations",
            "authors": [
                "Michael Roth"
            ],
            "abstract": "Predicting the structure of a discourse is challenging because relations\nbetween discourse segments are often implicit and thus hard to distinguish\ncomputationally. I extend previous work to classify implicit discourse\nrelations by introducing a novel set of features on the level of semantic\nroles. My results demonstrate that such features are helpful, yielding results\ncompetitive with other feature-rich approaches on the PDTB. My main\ncontribution is an analysis of improvements that can be traced back to\nrole-based features, providing insights into why and when role semantics is\nhelpful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08098v1",
            "date": "2018-08-24 11:37:40+00:00",
            "title": "Measuring LDA Topic Stability from Clusters of Replicated Runs",
            "authors": [
                "Mika M\u00e4ntyl\u00e4",
                "Ma\u00eblick Claes",
                "Umar Farooq"
            ],
            "abstract": "Background: Unstructured and textual data is increasing rapidly and Latent\nDirichlet Allocation (LDA) topic modeling is a popular data analysis methods\nfor it. Past work suggests that instability of LDA topics may lead to\nsystematic errors. Aim: We propose a method that relies on replicated LDA runs,\nclustering, and providing a stability metric for the topics. Method: We\ngenerate k LDA topics and replicate this process n times resulting in n*k\ntopics. Then we use K-medioids to cluster the n*k topics to k clusters. The k\nclusters now represent the original LDA topics and we present them like normal\nLDA topics showing the ten most probable words. For the clusters, we try\nmultiple stability metrics, out of which we recommend Rank-Biased Overlap,\nshowing the stability of the topics inside the clusters. Results: We provide an\ninitial validation where our method is used for 270,000 Mozilla Firefox commit\nmessages with k=20 and n=20. We show how our topic stability metrics are\nrelated to the contents of the topics. Conclusions: Advances in text mining\nenable us to analyze large masses of text in software engineering but\nnon-deterministic algorithms, such as LDA, may lead to unreplicable\nconclusions. Our approach makes LDA stability transparent and is also\ncomplementary rather than alternative to many prior works that focus on LDA\nparameter tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08266v2",
            "date": "2018-08-24 18:47:29+00:00",
            "title": "A Visual Attention Grounding Neural Model for Multimodal Machine Translation",
            "authors": [
                "Mingyang Zhou",
                "Runxiang Cheng",
                "Yong Jae Lee",
                "Zhou Yu"
            ],
            "abstract": "We introduce a novel multimodal machine translation model that utilizes\nparallel visual and textual information. Our model jointly optimizes the\nlearning of a shared visual-language embedding and a translator. The model\nleverages a visual attention grounding mechanism that links the visual\nsemantics with the corresponding textual semantics. Our approach achieves\ncompetitive state-of-the-art results on the Multi30K and the Ambiguous COCO\ndatasets. We also collected a new multilingual multimodal product description\ndataset to simulate a real-world international online shopping scenario. On\nthis dataset, our visual attention grounding model outperforms other methods by\na large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08392v1",
            "date": "2018-08-25 09:32:30+00:00",
            "title": "MADARi: A Web Interface for Joint Arabic Morphological Annotation and Spelling Correction",
            "authors": [
                "Ossama Obeid",
                "Salam Khalifa",
                "Nizar Habash",
                "Houda Bouamor",
                "Wajdi Zaghouani",
                "Kemal Oflazer"
            ],
            "abstract": "In this paper, we introduce MADARi, a joint morphological annotation and\nspelling correction system for texts in Standard and Dialectal Arabic. The\nMADARi framework provides intuitive interfaces for annotating text and managing\nthe annotation process of a large number of sizable documents. Morphological\nannotation includes indicating, for a word, in context, its baseword, clitics,\npart-of-speech, lemma, gloss, and dialect identification. MADARi has a suite of\nutilities to help with annotator productivity. For example, annotators are\nprovided with pre-computed analyses to assist them in their task and reduce the\namount of work needed to complete it. MADARi also allows annotators to query a\nmorphological analyzer for a list of possible analyses in multiple dialects or\nlook up previously submitted analyses. The MADARi management interface enables\na lead annotator to easily manage and organize the whole annotation process\nremotely and concurrently. We describe the motivation, design and\nimplementation of this interface; and we present details from a user study\nworking with this system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08409v2",
            "date": "2018-08-25 11:08:28+00:00",
            "title": "Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set",
            "authors": [
                "Radu Tudor Ionescu",
                "Andrei M. Butnaru"
            ],
            "abstract": "Recently, string kernels have obtained state-of-the-art results in various\ntext classification tasks such as Arabic dialect identification or native\nlanguage identification. In this paper, we apply two simple yet effective\ntransductive learning approaches to further improve the results of string\nkernels. The first approach is based on interpreting the pairwise string kernel\nsimilarities between samples in the training set and samples in the test set as\nfeatures. Our second approach is a simple self-training method based on two\nlearning iterations. In the first iteration, a classifier is trained on the\ntraining set and tested on the test set, as usual. In the second iteration, a\nnumber of test samples (to which the classifier associated higher confidence\nscores) are added to the training set for another round of training. However,\nthe ground-truth labels of the added test samples are not necessary. Instead,\nwe use the labels predicted by the classifier in the first training iteration.\nBy adapting string kernels to the test set, we report significantly better\naccuracy rates in English polarity classification and Arabic dialect\nidentification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08432v1",
            "date": "2018-08-25 14:24:23+00:00",
            "title": "Churn Intent Detection in Multilingual Chatbot Conversations and Social Media",
            "authors": [
                "Christian Abbet",
                "Meryem M'hamdi",
                "Athanasios Giannakopoulos",
                "Robert West",
                "Andreea Hossmann",
                "Michael Baeriswyl",
                "Claudiu Musat"
            ],
            "abstract": "We propose a new method to detect when users express the intent to leave a\nservice, also known as churn. While previous work focuses solely on social\nmedia, we show that this intent can be detected in chatbot conversations. As\ncompanies increasingly rely on chatbots they need an overview of potentially\nchurny users. To this end, we crowdsource and publish a dataset of churn intent\nexpressions in chatbot interactions in German and English. We show that\nclassifiers trained on social media data can detect the same intent in the\ncontext of chatbots.\n  We introduce a classification architecture that outperforms existing work on\nchurn intent detection in social media. Moreover, we show that, using bilingual\nword embeddings, a system trained on combined English and German data\noutperforms monolingual approaches. As the only existing dataset is in English,\nwe crowdsource and publish a novel dataset of German tweets. We thus underline\nthe universal aspect of the problem, as examples of churn intent in English\nhelp us identify churn in German tweets and chatbot conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08438v2",
            "date": "2018-08-25 15:20:30+00:00",
            "title": "Paraphrases as Foreign Languages in Multilingual Neural Machine Translation",
            "authors": [
                "Zhong Zhou",
                "Matthias Sperber",
                "Alex Waibel"
            ],
            "abstract": "Paraphrases, the rewordings of the same semantic meaning, are useful for\nimproving generalization and translation. However, prior works only explore\nparaphrases at the word or phrase level, not at the sentence or corpus level.\nUnlike previous works that only explore paraphrases at the word or phrase\nlevel, we use different translations of the whole training data that are\nconsistent in structure as paraphrases at the corpus level. We train on\nparallel paraphrases in multiple languages from various sources. We treat\nparaphrases as foreign languages, tag source sentences with paraphrase labels,\nand train on parallel paraphrases in the style of multilingual Neural Machine\nTranslation (NMT). Our multi-paraphrase NMT that trains only on two languages\noutperforms the multilingual baselines. Adding paraphrases improves the rare\nword translation and increases entropy and diversity in lexical choice. Adding\nthe source paraphrases boosts performance better than adding the target ones.\nCombining both the source and the target paraphrases lifts performance further;\ncombining paraphrases with multilingual data helps but has mixed performance.\nWe achieve a BLEU score of 57.2 for French-to-English translation using 24\ncorpus-level paraphrases of the Bible, which outperforms the multilingual\nbaselines and is +34.7 above the single-source single-target NMT baseline.",
            "journal ref": "Proceedings of 57th Annual Meeting of the Association for\n  Computational Linguistics Student Research Workshop, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08450v1",
            "date": "2018-08-25 17:02:29+00:00",
            "title": "Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition",
            "authors": [
                "Zenan Zhai",
                "Dat Quoc Nguyen",
                "Karin Verspoor"
            ],
            "abstract": "We compare the use of LSTM-based and CNN-based character-level word\nembeddings in BiLSTM-CRF models to approach chemical and disease named entity\nrecognition (NER) tasks. Empirical results over the BioCreative V CDR corpus\nshow that the use of either type of character-level word embeddings in\nconjunction with the BiLSTM-CRF models leads to comparable state-of-the-art\nperformance. However, the models using CNN-based character-level word\nembeddings have a computational performance advantage, increasing training time\nover word-based models by 25% while the LSTM-based character-level word\nembeddings more than double the required training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08482v2",
            "date": "2018-08-25 23:26:10+00:00",
            "title": "Exploring Recombination for Efficient Decoding of Neural Machine Translation",
            "authors": [
                "Zhisong Zhang",
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Hai Zhao"
            ],
            "abstract": "In Neural Machine Translation (NMT), the decoder can capture the features of\nthe entire prediction history with neural connections and representations. This\nmeans that partial hypotheses with different prefixes will be regarded\ndifferently no matter how similar they are. However, this might be inefficient\nsince some partial hypotheses can contain only local differences that will not\ninfluence future predictions. In this work, we introduce recombination in NMT\ndecoding based on the concept of the \"equivalence\" of partial hypotheses.\nHeuristically, we use a simple $n$-gram suffix based equivalence function and\nadapt it into beam search decoding. Through experiments on large-scale\nChinese-to-English and English-to-Germen translation tasks, we show that the\nproposed method can obtain similar translation quality with a smaller beam\nsize, making NMT decoding more efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08485v1",
            "date": "2018-08-26 00:02:36+00:00",
            "title": "Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision",
            "authors": [
                "Hai Wang",
                "Hoifung Poon"
            ],
            "abstract": "Deep learning has emerged as a versatile tool for a wide range of NLP tasks,\ndue to its superior capacity in representation learning. But its applicability\nis limited by the reliance on annotated examples, which are difficult to\nproduce at scale. Indirect supervision has emerged as a promising direction to\naddress this bottleneck, either by introducing labeling functions to\nautomatically generate noisy examples from unlabeled text, or by imposing\nconstraints over interdependent label decisions. A plethora of methods have\nbeen proposed, each with respective strengths and limitations. Probabilistic\nlogic offers a unifying language to represent indirect supervision, but\nend-to-end modeling with probabilistic logic is often infeasible due to\nintractable inference and learning. In this paper, we propose deep\nprobabilistic logic (DPL) as a general framework for indirect supervision, by\ncomposing probabilistic logic with deep learning. DPL models label decisions as\nlatent variables, represents prior knowledge on their relations using weighted\nfirst-order logical formulas, and alternates between learning a deep neural\nnetwork for the end task and refining uncertain formula weights for indirect\nsupervision, using variational EM. This framework subsumes prior indirect\nsupervision methods as special cases, and enables novel combination via\ninfusion of rich domain and linguistic knowledge. Experiments on biomedical\nmachine reading demonstrate the promise of this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08504v1",
            "date": "2018-08-26 04:04:39+00:00",
            "title": "Event Detection with Neural Networks: A Rigorous Empirical Evaluation",
            "authors": [
                "J. Walker Orr",
                "Prasad Tadepalli",
                "Xiaoli Fern"
            ],
            "abstract": "Detecting events and classifying them into predefined types is an important\nstep in knowledge extraction from natural language texts. While the neural\nnetwork models have generally led the state-of-the-art, the differences in\nperformance between different architectures have not been rigorously studied.\nIn this paper we present a novel GRU-based model that combines syntactic\ninformation along with temporal structure through an attention mechanism. We\nshow that it is competitive with other neural network architectures through\nempirical evaluations under different random initializations and\ntraining-validation-test splits of ACE2005 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08518v2",
            "date": "2018-08-26 08:36:15+00:00",
            "title": "Word Sense Induction with Neural biLM and Symmetric Patterns",
            "authors": [
                "Asaf Amrami",
                "Yoav Goldberg"
            ],
            "abstract": "An established method for Word Sense Induction (WSI) uses a language model to\npredict probable substitutes for target words, and induces senses by clustering\nthese resulting substitute vectors.\n  We replace the ngram-based language model (LM) with a recurrent one. Beyond\nbeing more accurate, the use of the recurrent LM allows us to effectively query\nit in a creative way, using what we call dynamic symmetric patterns.\n  The combination of the RNN-LM and the dynamic symmetric patterns results in\nstrong substitute vectors for WSI, allowing to surpass the current\nstate-of-the-art on the SemEval 2013 WSI shared task by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08561v2",
            "date": "2018-08-26 14:36:22+00:00",
            "title": "Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification",
            "authors": [
                "Junyang Lin",
                "Qi Su",
                "Pengcheng Yang",
                "Shuming Ma",
                "Xu Sun"
            ],
            "abstract": "We propose a novel model for multi-label text classification, which is based\non sequence-to-sequence learning. The model generates higher-level semantic\nunit representations with multi-level dilated convolution as well as a\ncorresponding hybrid attention mechanism that extracts both the information at\nthe word-level and the level of the semantic unit. Our designed dilated\nconvolution effectively reduces dimension and supports an exponential expansion\nof receptive fields without loss of local information, and the\nattention-over-attention mechanism is able to capture more summary relevant\ninformation from the source context. Results of our experiments show that the\nproposed model has significant advantages over the baseline models on the\ndataset RCV1-V2 and Ren-CECps, and our analysis demonstrates that our model is\ncompetitive to the deterministic hierarchical models and it is more robust to\nclassifying low-frequency labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08573v2",
            "date": "2018-08-26 15:10:47+00:00",
            "title": "Analyzing Learned Representations of a Deep ASR Performance Prediction Model",
            "authors": [
                "Zied Elloumi",
                "Laurent Besacier",
                "Olivier Galibert",
                "Benjamin Lecouteux"
            ],
            "abstract": "This paper addresses a relatively new task: prediction of ASR performance on\nunseen broadcast programs. In a previous paper, we presented an ASR performance\nprediction system using CNNs that encode both text (ASR transcript) and speech,\nin order to predict word error rate. This work is dedicated to the analysis of\nspeech signal embeddings and text embeddings learnt by the CNN while training\nour prediction model. We try to better understand which information is captured\nby the deep model and its relation with different conditioning factors. It is\nshown that hidden layers convey a clear signal about speech style, accent and\nbroadcast type. We then try to leverage these 3 types of information at\ntraining time through multi-task learning. Our experiments show that this\nallows to train slightly more efficient ASR performance prediction systems that\n- in addition - simultaneously tag the analyzed utterances according to their\nspeech style, accent and broadcast program origin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08575v5",
            "date": "2018-08-26 15:28:11+00:00",
            "title": "Title-Guided Encoding for Keyphrase Generation",
            "authors": [
                "Wang Chen",
                "Yifan Gao",
                "Jiani Zhang",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "Keyphrase generation (KG) aims to generate a set of keyphrases given a\ndocument, which is a fundamental task in natural language processing (NLP).\nMost previous methods solve this problem in an extractive manner, while\nrecently, several attempts are made under the generative setting using deep\nneural networks. However, the state-of-the-art generative methods simply treat\nthe document title and the document main body equally, ignoring the leading\nrole of the title to the overall document. To solve this problem, we introduce\na new model called Title-Guided Network (TG-Net) for automatic keyphrase\ngeneration task based on the encoder-decoder architecture with two new\nfeatures: (i) the title is additionally employed as a query-like input, and\n(ii) a title-guided encoder gathers the relevant information from the title to\neach word in the document. Experiments on a range of KG datasets demonstrate\nthat our model outperforms the state-of-the-art models with a large margin,\nespecially for documents with either very low or very high title length ratios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08583v2",
            "date": "2018-08-26 16:22:30+00:00",
            "title": "Semi-Autoregressive Neural Machine Translation",
            "authors": [
                "Chunqi Wang",
                "Ji Zhang",
                "Haiqing Chen"
            ],
            "abstract": "Existing approaches to neural machine translation are typically\nautoregressive models. While these models attain state-of-the-art translation\nquality, they are suffering from low parallelizability and thus slow at\ndecoding long sequences. In this paper, we propose a novel model for fast\nsequence generation --- the semi-autoregressive Transformer (SAT). The SAT\nkeeps the autoregressive property in global but relieves in local and thus is\nable to produce multiple successive words in parallel at each time step.\nExperiments conducted on English-German and Chinese-English translation tasks\nshow that the SAT achieves a good balance between translation quality and\ndecoding speed. On WMT'14 English-German translation, the SAT achieves\n5.58$\\times$ speedup while maintains 88\\% translation quality, significantly\nbetter than the previous non-autoregressive methods. When produces two words at\neach time step, the SAT is almost lossless (only 1\\% degeneration in BLEU\nscore).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08622v1",
            "date": "2018-08-26 21:12:43+00:00",
            "title": "Semi-Supervised Event Extraction with Paraphrase Clusters",
            "authors": [
                "James Ferguson",
                "Colin Lockard",
                "Daniel S. Weld",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Supervised event extraction systems are limited in their accuracy due to the\nlack of available training data. We present a method for self-training event\nextraction systems by bootstrapping additional training data. This is done by\ntaking advantage of the occurrence of multiple mentions of the same event\ninstances across newswire articles from multiple sources. If our system can\nmake a highconfidence extraction of some mentions in such a cluster, it can\nthen acquire diverse training examples by adding the other mentions as well.\nOur experiments show significant performance improvements on multiple event\nextractors over ACE 2005 and TAC-KBP 2015 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08626v1",
            "date": "2018-08-26 21:33:32+00:00",
            "title": "Identifying Domain Adjacent Instances for Semantic Parsers",
            "authors": [
                "James Ferguson",
                "Janara Christensen",
                "Edward Li",
                "Edgar Gonz\u00e0lez"
            ],
            "abstract": "When the semantics of a sentence are not representable in a semantic parser's\noutput schema, parsing will inevitably fail. Detection of these instances is\ncommonly treated as an out-of-domain classification problem. However, there is\nalso a more subtle scenario in which the test data is drawn from the same\ndomain. In addition to formalizing this problem of domain-adjacency, we present\na comparison of various baselines that could be used to solve it. We also\npropose a new simple sentence representation that emphasizes words which are\nunexpected. This approach improves the performance of a downstream semantic\nparser run on in-domain and domain-adjacent instances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08644v1",
            "date": "2018-08-27 00:01:11+00:00",
            "title": "Predicting Semantic Relations using Global Graph Properties",
            "authors": [
                "Yuval Pinter",
                "Jacob Eisenstein"
            ],
            "abstract": "Semantic graphs, such as WordNet, are resources which curate natural language\non two distinguishable layers. On the local level, individual relations between\nsynsets (semantic building blocks) such as hypernymy and meronymy enhance our\nunderstanding of the words used to express their meanings. Globally, analysis\nof graph-theoretic properties of the entire net sheds light on the structure of\nhuman language as a whole. In this paper, we combine global and local\nproperties of semantic graphs through the framework of Max-Margin Markov Graph\nModels (M3GM), a novel extension of Exponential Random Graph Model (ERGM) that\nscales to large multi-relational graphs. We demonstrate how such global\nmodeling improves performance on the local task of predicting semantic\nrelations between synsets, yielding new state-of-the-art results on the WN18RR\ndataset, a challenging version of WordNet link prediction in which \"easy\"\nreciprocal cases are removed. In addition, the M3GM model identifies\nmultirelational motifs that are characteristic of well-formed lexical semantic\nontologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08669v3",
            "date": "2018-08-27 02:42:48+00:00",
            "title": "Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions",
            "authors": [
                "Jiahui Qiu",
                "Qi Wang",
                "Yangming Zhou",
                "Tong Ruan",
                "Ju Gao"
            ],
            "abstract": "Clinical Named Entity Recognition (CNER) aims to identify and classify\nclinical terms such as diseases, symptoms, treatments, exams, and body parts in\nelectronic health records, which is a fundamental and crucial task for clinical\nand translation research. In recent years, deep learning methods have achieved\nsignificant success in CNER tasks. However, these methods depend greatly on\nRecurrent Neural Networks (RNNs), which maintain a vector of hidden activations\nthat are propagated through time, thus causing too much time to train models.\nIn this paper, we propose a Residual Dilated Convolutional Neural Network with\nConditional Random Field (RD-CNN-CRF) to solve it. Specifically, Chinese\ncharacters and dictionary features are first projected into dense vector\nrepresentations, then they are fed into the residual dilated convolutional\nneural network to capture contextual features. Finally, a conditional random\nfield is employed to capture dependencies between neighboring tags.\nComputational results on the CCKS-2017 Task 2 benchmark dataset show that our\nproposed RD-CNN-CRF method competes favorably with state-of-the-art RNN-based\nmethods both in terms of computational performance and training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08672v2",
            "date": "2018-08-27 02:57:42+00:00",
            "title": "IIIDYT at IEST 2018: Implicit Emotion Classification With Deep Contextualized Word Representations",
            "authors": [
                "Jorge A. Balazs",
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "In this paper we describe our system designed for the WASSA 2018 Implicit\nEmotion Shared Task (IEST), which obtained 2$^{\\text{nd}}$ place out of 26\nteams with a test macro F1 score of $0.710$. The system is composed of a single\npre-trained ELMo layer for encoding words, a Bidirectional Long-Short Memory\nNetwork BiLSTM for enriching word representations with context, a max-pooling\noperation for creating sentence representations from said word vectors, and a\nDense Layer for projecting the sentence representations into label space. Our\nofficial submission was obtained by ensembling 6 of these models initialized\nwith different random seeds. The code for replicating this paper is available\nat https://github.com/jabalazs/implicit_emotion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08732v1",
            "date": "2018-08-27 08:37:42+00:00",
            "title": "simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captions",
            "authors": [
                "Fenglin Liu",
                "Xuancheng Ren",
                "Yuanxin Liu",
                "Houfeng Wang",
                "Xu Sun"
            ],
            "abstract": "The encode-decoder framework has shown recent success in image captioning.\nVisual attention, which is good at detailedness, and semantic attention, which\nis good at comprehensiveness, have been separately proposed to ground the\ncaption on the image. In this paper, we propose the Stepwise Image-Topic\nMerging Network (simNet) that makes use of the two kinds of attention at the\nsame time. At each time step when generating the caption, the decoder\nadaptively merges the attentive information in the extracted topics and the\nimage according to the generated context, so that the visual information and\nthe semantic information can be effectively combined. The proposed approach is\nevaluated on two benchmark datasets and reaches the state-of-the-art\nperformances.(The code is available at https://github.com/lancopku/simNet)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08744v1",
            "date": "2018-08-27 09:04:22+00:00",
            "title": "Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehension",
            "authors": [
                "Matthias Blohm",
                "Glorianna Jagfeld",
                "Ekta Sood",
                "Xiang Yu",
                "Ngoc Thang Vu"
            ],
            "abstract": "We propose a machine reading comprehension model based on the\ncompare-aggregate framework with two-staged attention that achieves\nstate-of-the-art results on the MovieQA question answering dataset. To\ninvestigate the limitations of our model as well as the behavioral difference\nbetween convolutional and recurrent neural networks, we generate adversarial\nexamples to confuse the model and compare to human performance. Furthermore, we\nassess the generalizability of our model by analyzing its differences to human\ninference,",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08745v1",
            "date": "2018-08-27 09:08:18+00:00",
            "title": "Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization",
            "authors": [
                "Shashi Narayan",
                "Shay B. Cohen",
                "Mirella Lapata"
            ],
            "abstract": "We introduce extreme summarization, a new single-document summarization task\nwhich does not favor extractive strategies and calls for an abstractive\nmodeling approach. The idea is to create a short, one-sentence news summary\nanswering the question \"What is the article about?\". We collect a real-world,\nlarge-scale dataset for this task by harvesting online articles from the\nBritish Broadcasting Corporation (BBC). We propose a novel abstractive model\nwhich is conditioned on the article's topics and based entirely on\nconvolutional neural networks. We demonstrate experimentally that this\narchitecture captures long-range dependencies in a document and recognizes\npertinent content, outperforming an oracle extractive system and\nstate-of-the-art abstractive approaches when evaluated automatically and by\nhumans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08780v1",
            "date": "2018-08-27 10:54:37+00:00",
            "title": "Improving Cross-Lingual Word Embeddings by Meeting in the Middle",
            "authors": [
                "Yerai Doval",
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "Cross-lingual word embeddings are becoming increasingly important in\nmultilingual NLP. Recently, it has been shown that these embeddings can be\neffectively learned by aligning two disjoint monolingual vector spaces through\nlinear transformations, using no more than a small bilingual dictionary as\nsupervision. In this work, we propose to apply an additional transformation\nafter the initial alignment step, which moves cross-lingual synonyms towards a\nmiddle point between them. By applying this transformation our aim is to obtain\na better cross-lingual integration of the vector spaces. In addition, and\nperhaps surprisingly, the monolingual spaces also improve by this\ntransformation. This is in contrast to the original alignment, which is\ntypically learned such that the structure of the monolingual spaces is\npreserved. Our experiments confirm that the resulting cross-lingual embeddings\noutperform state-of-the-art models in both monolingual and cross-lingual\nevaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08795v1",
            "date": "2018-08-27 11:46:13+00:00",
            "title": "An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation",
            "authors": [
                "Liangchen Luo",
                "Jingjing Xu",
                "Junyang Lin",
                "Qi Zeng",
                "Xu Sun"
            ],
            "abstract": "Generating semantically coherent responses is still a major challenge in\ndialogue generation. Different from conventional text generation tasks, the\nmapping between inputs and responses in conversations is more complicated,\nwhich highly demands the understanding of utterance-level semantic dependency,\na relation between the whole meanings of inputs and outputs. To address this\nproblem, we propose an Auto-Encoder Matching (AEM) model to learn such\ndependency. The model contains two auto-encoders and one mapping module. The\nauto-encoders learn the semantic representations of inputs and responses, and\nthe mapping module learns to connect the utterance-level representations.\nExperimental results from automatic and human evaluations demonstrate that our\nmodel is capable of generating responses of high coherence and fluency compared\nto baseline models. The code is available at https://github.com/lancopku/AMM",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08850v1",
            "date": "2018-08-27 14:02:58+00:00",
            "title": "WiSeBE: Window-based Sentence Boundary Evaluation",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Sentence Boundary Detection (SBD) has been a major research topic since\nAutomatic Speech Recognition transcripts have been used for further Natural\nLanguage Processing tasks like Part of Speech Tagging, Question Answering or\nAutomatic Summarization. But what about evaluation? Do standard evaluation\nmetrics like precision, recall, F-score or classification error; and more\nimportant, evaluating an automatic system against a unique reference is enough\nto conclude how well a SBD system is performing given the final application of\nthe transcript? In this paper we propose Window-based Sentence Boundary\nEvaluation (WiSeBE), a semi-supervised metric for evaluating Sentence Boundary\nDetection systems based on multi-reference (dis)agreement. We evaluate and\ncompare the performance of different SBD systems over a set of Youtube\ntranscripts using WiSeBE and standard metrics. This double evaluation gives an\nunderstanding of how WiSeBE is a more reliable metric for the SBD task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08859v2",
            "date": "2018-08-27 14:19:18+00:00",
            "title": "Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation",
            "authors": [
                "Nikolay Bogoychev",
                "Marcin Junczys-Dowmunt",
                "Kenneth Heafield",
                "Alham Fikri Aji"
            ],
            "abstract": "In order to extract the best possible performance from asynchronous\nstochastic gradient descent one must increase the mini-batch size and scale the\nlearning rate accordingly. In order to achieve further speedup we introduce a\ntechnique that delays gradient updates effectively increasing the mini-batch\nsize. Unfortunately with the increase of mini-batch size we worsen the stale\ngradient problem in asynchronous stochastic gradient descent (SGD) which makes\nthe model convergence poor. We introduce local optimizers which mitigate the\nstale gradient problem and together with fine tuning our momentum we are able\nto train a shallow machine translation system 27% faster than an optimized\nbaseline with negligible penalty in BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08932v1",
            "date": "2018-08-27 17:15:54+00:00",
            "title": "Extracting Sentiment Attitudes From Analytical Texts",
            "authors": [
                "Natalia Loukachevitch",
                "Nicolay Rusnachenko"
            ],
            "abstract": "In this paper we present the RuSentRel corpus including analytical texts in\nthe sphere of international relations. For each document we annotated\nsentiments from the author to mentioned named entities, and sentiments of\nrelations between mentioned entities. In the current experiments, we considered\nthe problem of extracting sentiment relations between entities for the whole\ndocuments as a three-class machine learning task. We experimented with\nconventional machine-learning methods (Naive Bayes, SVM, Random Forest).",
            "journal ref": "Proceedings of International Conference on Computational\n  Linguistics and Intellectual Technologies Dialoque-2018, 2018, p.459-468",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08933v2",
            "date": "2018-08-27 17:22:15+00:00",
            "title": "Unsupervised Multilingual Word Embeddings",
            "authors": [
                "Xilun Chen",
                "Claire Cardie"
            ],
            "abstract": "Multilingual Word Embeddings (MWEs) represent words from multiple languages\nin a single distributional vector space. Unsupervised MWE (UMWE) methods\nacquire multilingual embeddings without cross-lingual supervision, which is a\nsignificant advantage over traditional supervised approaches and opens many new\npossibilities for low-resource languages. Prior art for learning UMWEs,\nhowever, merely relies on a number of independently trained Unsupervised\nBilingual Word Embeddings (UBWEs) to obtain multilingual embeddings. These\nmethods fail to leverage the interdependencies that exist among many languages.\nTo address this shortcoming, we propose a fully unsupervised framework for\nlearning MWEs that directly exploits the relations between all language pairs.\nOur model substantially outperforms previous approaches in the experiments on\nmultilingual word translation and cross-lingual word similarity. In addition,\nour model even beats supervised approaches trained with cross-lingual\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08946v3",
            "date": "2018-08-27 17:51:27+00:00",
            "title": "Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures",
            "authors": [
                "Gongbo Tang",
                "Mathias M\u00fcller",
                "Annette Rios",
                "Rico Sennrich"
            ],
            "abstract": "Recently, non-recurrent architectures (convolutional, self-attentional) have\noutperformed RNNs in neural machine translation. CNNs and self-attentional\nnetworks can connect distant words via shorter network paths than RNNs, and it\nhas been speculated that this improves their ability to model long-range\ndependencies. However, this theoretical argument has not been tested\nempirically, nor have alternative explanations for their strong performance\nbeen explored in-depth. We hypothesize that the strong performance of CNNs and\nself-attentional networks could also be due to their ability to extract\nsemantic features from the source text, and we evaluate RNNs, CNNs and\nself-attention networks on two tasks: subject-verb agreement (where capturing\nlong-range dependencies is required) and word sense disambiguation (where\nsemantic feature extraction is required). Our experimental results show that:\n1) self-attentional networks and CNNs do not outperform RNNs in modeling\nsubject-verb agreement over long distances; 2) self-attentional networks\nperform distinctly better than RNNs and CNNs on word sense disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08949v2",
            "date": "2018-08-27 17:54:29+00:00",
            "title": "Dissecting Contextual Word Embeddings: Architecture and Representation",
            "authors": [
                "Matthew E. Peters",
                "Mark Neumann",
                "Luke Zettlemoyer",
                "Wen-tau Yih"
            ],
            "abstract": "Contextual word representations derived from pre-trained bidirectional\nlanguage models (biLMs) have recently been shown to provide significant\nimprovements to the state of the art for a wide range of NLP tasks. However,\nmany questions remain as to how and why these models are so effective. In this\npaper, we present a detailed empirical study of how the choice of neural\narchitecture (e.g. LSTM, CNN, or self attention) influences both end task\naccuracy and qualitative properties of the representations that are learned. We\nshow there is a tradeoff between speed and accuracy, but all architectures\nlearn high quality contextual representations that outperform word embeddings\nfor four challenging NLP tasks. Additionally, all architectures learn\nrepresentations that vary with network depth, from exclusively morphological\nbased at the word embedding layer through local syntax based in the lower\ncontextual layers to longer range semantics such coreference at the upper\nlayers. Together, these results suggest that unsupervised biLMs, independent of\narchitecture, are learning much more about the structure of language than\npreviously appreciated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08987v1",
            "date": "2018-08-27 18:31:33+00:00",
            "title": "Large Margin Neural Language Model",
            "authors": [
                "Jiaji Huang",
                "Yi Li",
                "Wei Ping",
                "Liang Huang"
            ],
            "abstract": "We propose a large margin criterion for training neural language models.\nConventionally, neural language models are trained by minimizing perplexity\n(PPL) on grammatical sentences. However, we demonstrate that PPL may not be the\nbest metric to optimize in some tasks, and further propose a large margin\nformulation. The proposed method aims to enlarge the margin between the \"good\"\nand \"bad\" sentences in a task-specific sense. It is trained end-to-end and can\nbe widely applied to tasks that involve re-scoring of generated text. Compared\nwith minimum-PPL training, our method gains up to 1.1 WER reduction for speech\nrecognition and 1.0 BLEU increase for machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09006v2",
            "date": "2018-08-27 19:27:01+00:00",
            "title": "Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation",
            "authors": [
                "Marzieh Fadaee",
                "Christof Monz"
            ],
            "abstract": "Neural Machine Translation has achieved state-of-the-art performance for\nseveral language pairs using a combination of parallel and synthetic data.\nSynthetic data is often generated by back-translating sentences randomly\nsampled from monolingual data using a reverse translation model. While\nback-translation has been shown to be very effective in many cases, it is not\nentirely clear why. In this work, we explore different aspects of\nback-translation, and show that words with high prediction loss during training\nbenefit most from the addition of synthetic data. We introduce several\nvariations of sampling strategies targeting difficult-to-predict words using\nprediction losses and frequencies of words. In addition, we also target the\ncontexts of difficult words and sample sentences that are similar in context.\nExperimental results for the WMT news translation task show that our method\nimproves translation quality by up to 1.7 and 1.2 Bleu points over\nback-translation using random sampling for German-English and English-German,\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09012v1",
            "date": "2018-08-27 19:40:53+00:00",
            "title": "Natural Language Generation with Neural Variational Models",
            "authors": [
                "Hareesh Bahuleyan"
            ],
            "abstract": "In this thesis, we explore the use of deep neural networks for generation of\nnatural language. Specifically, we implement two sequence-to-sequence neural\nvariational models - variational autoencoders (VAE) and variational\nencoder-decoders (VED). VAEs for text generation are difficult to train due to\nissues associated with the Kullback-Leibler (KL) divergence term of the loss\nfunction vanishing to zero. We successfully train VAEs by implementing\noptimization heuristics such as KL weight annealing and word dropout. We also\ndemonstrate the effectiveness of this continuous latent space through\nexperiments such as random sampling, linear interpolation and sampling from the\nneighborhood of the input. We argue that if VAEs are not designed\nappropriately, it may lead to bypassing connections which results in the latent\nspace being ignored during training. We show experimentally with the example of\ndecoder hidden state initialization that such bypassing connections degrade the\nVAE into a deterministic model, thereby reducing the diversity of generated\nsentences. We discover that the traditional attention mechanism used in\nsequence-to-sequence VED models serves as a bypassing connection, thereby\ndeteriorating the model's latent space. In order to circumvent this issue, we\npropose the variational attention mechanism where the attention context vector\nis modeled as a random variable that can be sampled from a distribution. We\nshow empirically using automatic evaluation metrics, namely entropy and\ndistinct measures, that our variational attention model generates more diverse\noutput sentences than the deterministic attention model. A qualitative analysis\nwith human evaluation study proves that our model simultaneously produces\nsentences that are of high quality and equally fluent as the ones generated by\nthe deterministic attention counterpart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09029v1",
            "date": "2018-08-27 20:31:27+00:00",
            "title": "Pyramidal Recurrent Unit for Language Modeling",
            "authors": [
                "Sachin Mehta",
                "Rik Koncel-Kedziorski",
                "Mohammad Rastegari",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "LSTMs are powerful tools for modeling contextual information, as evidenced by\ntheir success at the task of language modeling. However, modeling contexts in\nvery high dimensional space can lead to poor generalizability. We introduce the\nPyramidal Recurrent Unit (PRU), which enables learning representations in high\ndimensional space with more generalization power and fewer parameters. PRUs\nreplace the linear transformation in LSTMs with more sophisticated interactions\nincluding pyramidal and grouped linear transformations. This architecture gives\nstrong results on word-level language modeling while reducing the number of\nparameters significantly. In particular, PRU improves the perplexity of a\nrecent state-of-the-art language model Merity et al. (2018) by up to 1.3 points\nwhile learning 15-20% fewer parameters. For similar number of model parameters,\nPRU outperforms all previous RNN models that exploit different gating\nmechanisms and transformations. We provide a detailed examination of the PRU\nand its behavior on the language modeling tasks. Our code is open-source and\navailable at https://sacmehta.github.io/PRU/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09031v1",
            "date": "2018-08-27 20:42:51+00:00",
            "title": "Targeted Syntactic Evaluation of Language Models",
            "authors": [
                "Rebecca Marvin",
                "Tal Linzen"
            ],
            "abstract": "We present a dataset for evaluating the grammaticality of the predictions of\na language model. We automatically construct a large number of minimally\ndifferent pairs of English sentences, each consisting of a grammatical and an\nungrammatical sentence. The sentence pairs represent different variations of\nstructure-sensitive phenomena: subject-verb agreement, reflexive anaphora and\nnegative polarity items. We expect a language model to assign a higher\nprobability to the grammatical sentence than the ungrammatical one. In an\nexperiment using this data set, an LSTM language model performed poorly on many\nof the constructions. Multi-task training with a syntactic objective (CCG\nsupertagging) improved the LSTM's accuracy, but a large gap remained between\nits performance and the accuracy of human participants recruited online. This\nsuggests that there is considerable room for improvement over LSTMs in\ncapturing syntax in a language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09040v1",
            "date": "2018-08-27 21:42:56+00:00",
            "title": "One-Shot Relational Learning for Knowledge Graphs",
            "authors": [
                "Wenhan Xiong",
                "Mo Yu",
                "Shiyu Chang",
                "Xiaoxiao Guo",
                "William Yang Wang"
            ],
            "abstract": "Knowledge graphs (KGs) are the key components of various natural language\nprocessing applications. To further expand KGs' coverage, previous studies on\nknowledge graph completion usually require a large number of training instances\nfor each relation. However, we observe that long-tail relations are actually\nmore common in KGs and those newly added relations often do not have many known\ntriples for training. In this work, we aim at predicting new facts under a\nchallenging setting where only one training instance is available. We propose a\none-shot relational learning framework, which utilizes the knowledge extracted\nby embedding models and learns a matching metric by considering both the\nlearned embeddings and one-hop graph structures. Empirically, our model yields\nconsiderable performance improvements over existing embedding models, and also\neliminates the need of re-training the embedding models when dealing with newly\nadded relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09042v2",
            "date": "2018-08-27 21:49:12+00:00",
            "title": "Adversarial Decomposition of Text Representation",
            "authors": [
                "Alexey Romanov",
                "Anna Rumshisky",
                "Anna Rogers",
                "David Donahue"
            ],
            "abstract": "In this paper, we present a method for adversarial decomposition of text\nrepresentation. This method can be used to decompose a representation of an\ninput sentence into several independent vectors, each of them responsible for a\nspecific aspect of the input sentence. We evaluate the proposed method on two\ncase studies: the conversion between different social registers and diachronic\nlanguage change. We show that the proposed method is capable of fine-grained\ncontrolled change of these aspects of the input sentence. It is also learning a\ncontinuous (rather than categorical) representation of the style of the\nsentence, which is more linguistically realistic. The model uses\nadversarial-motivational training and includes a special motivational loss,\nwhich acts opposite to the discriminator and encourages a better decomposition.\nFurthermore, we evaluate the obtained meaning embeddings on a downstream task\nof paraphrase detection and show that they significantly outperform the\nembeddings of a regular autoencoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09055v2",
            "date": "2018-08-27 22:47:59+00:00",
            "title": "Parameter sharing between dependency parsers for related languages",
            "authors": [
                "Miryam de Lhoneux",
                "Johannes Bjerva",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Previous work has suggested that parameter sharing between transition-based\nneural dependency parsers for related languages can lead to better performance,\nbut there is no consensus on what parameters to share. We present an evaluation\nof 27 different parameter sharing strategies across 10 languages, representing\nfive pairs of related languages, each pair from a different language family. We\nfind that sharing transition classifier parameters always helps, whereas the\nusefulness of sharing word and/or character LSTM parameters varies. Based on\nthis result, we propose an architecture where the transition classifier is\nshared, and the sharing of word and character parameters is controlled by a\nparameter that can be tuned on validation data. This model is linguistically\nmotivated and obtains significant improvements over a monolingually trained\nbaseline. We also find that sharing transition classifier parameters helps when\ntraining a parser on unrelated language pairs, but we find that, in the case of\nunrelated languages, sharing too many parameters does not help.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09060v1",
            "date": "2018-08-27 23:11:47+00:00",
            "title": "An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing",
            "authors": [
                "Aaron Smith",
                "Miryam de Lhoneux",
                "Sara Stymne",
                "Joakim Nivre"
            ],
            "abstract": "We provide a comprehensive analysis of the interactions between pre-trained\nword embeddings, character models and POS tags in a transition-based dependency\nparser. While previous studies have shown POS information to be less important\nin the presence of character models, we show that in fact there are complex\ninteractions between all three techniques. In isolation each produces large\nimprovements over a baseline system using randomly initialised word embeddings\nonly, but combining them quickly leads to diminishing returns. We categorise\nwords by frequency, POS tag and language in order to systematically investigate\nhow each of the techniques affects parsing quality. For many word categories,\napplying any two of the three techniques is almost as good as the full combined\nsystem. Character models tend to be more important for low-frequency open-class\nwords, especially in morphologically rich languages, while POS tags can help\ndisambiguate high-frequency function words. We also show that large character\nembedding sizes help even for languages with small character sets, especially\nin morphologically rich languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09075v1",
            "date": "2018-08-28 00:53:06+00:00",
            "title": "Evaluating the Utility of Hand-crafted Features in Sequence Labelling",
            "authors": [
                "Minghao Wu",
                "Fei Liu",
                "Trevor Cohn"
            ],
            "abstract": "Conventional wisdom is that hand-crafted features are redundant for deep\nlearning models, as they already learn adequate representations of text\nautomatically from corpora. In this work, we test this claim by proposing a new\nmethod for exploiting handcrafted features as part of a novel hybrid learning\napproach, incorporating a feature auto-encoder loss component. We evaluate on\nthe task of named entity recognition (NER), where we show that including manual\nfeatures for part-of-speech, word shapes and gazetteers can improve the\nperformance of a neural CRF model. We obtain a $F_1$ of 91.89 for the\nCoNLL-2003 English shared task, which significantly outperforms a collection of\nhighly competitive baseline models. We also present an ablation study showing\nthe importance of auto-encoding, over using features as either inputs or\noutputs alone, and moreover, show including the autoencoder components reduces\ntraining requirements to 60\\%, while retaining the same predictive accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09091v1",
            "date": "2018-08-28 02:23:56+00:00",
            "title": "Disfluency Detection using a Noisy Channel Model and a Deep Neural Language Model",
            "authors": [
                "Paria Jamshid Lou",
                "Mark Johnson"
            ],
            "abstract": "This paper presents a model for disfluency detection in spontaneous speech\ntranscripts called LSTM Noisy Channel Model. The model uses a Noisy Channel\nModel (NCM) to generate n-best candidate disfluency analyses and a Long\nShort-Term Memory (LSTM) language model to score the underlying fluent\nsentences of each analysis. The LSTM language model scores, along with other\nfeatures, are used in a MaxEnt reranker to identify the most plausible\nanalysis. We show that using an LSTM language model in the reranking process of\nnoisy channel disfluency model improves the state-of-the-art in disfluency\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09092v3",
            "date": "2018-08-28 02:28:12+00:00",
            "title": "Disfluency Detection using Auto-Correlational Neural Networks",
            "authors": [
                "Paria Jamshid Lou",
                "Peter Anderson",
                "Mark Johnson"
            ],
            "abstract": "In recent years, the natural language processing community has moved away\nfrom task-specific feature engineering, i.e., researchers discovering ad-hoc\nfeature representations for various tasks, in favor of general-purpose methods\nthat learn the input representation by themselves. However, state-of-the-art\napproaches to disfluency detection in spontaneous speech transcripts currently\nstill depend on an array of hand-crafted features, and other representations\nderived from the output of pre-existing systems such as language models or\ndependency parsers. As an alternative, this paper proposes a simple yet\neffective model for automatic disfluency detection, called an\nauto-correlational neural network (ACNN). The model uses a convolutional neural\nnetwork (CNN) and augments it with a new auto-correlation operator at the\nlowest layer that can capture the kinds of \"rough copy\" dependencies that are\ncharacteristic of repair disfluencies in speech. In experiments, the ACNN model\noutperforms the baseline CNN on a disfluency detection task with a 5% increase\nin f-score, which is close to the previous best result on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09101v1",
            "date": "2018-08-28 03:37:39+00:00",
            "title": "N-ary Relation Extraction using Graph State LSTM",
            "authors": [
                "Linfeng Song",
                "Yue Zhang",
                "Zhiguo Wang",
                "Daniel Gildea"
            ],
            "abstract": "Cross-sentence $n$-ary relation extraction detects relations among $n$\nentities across multiple sentences. Typical methods formulate an input as a\n\\textit{document graph}, integrating various intra-sentential and\ninter-sentential dependencies. The current state-of-the-art method splits the\ninput graph into two DAGs, adopting a DAG-structured LSTM for each. Though\nbeing able to model rich linguistic knowledge by leveraging graph edges,\nimportant information can be lost in the splitting procedure. We propose a\ngraph-state LSTM model, which uses a parallel state to model each word,\nrecurrently enriching state values via message passing. Compared with DAG\nLSTMs, our graph LSTM keeps the original graph structure, and speeds up\ncomputation by allowing more parallelization. On a standard benchmark, our\nmodel shows the best result in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09115v3",
            "date": "2018-08-28 04:49:54+00:00",
            "title": "All You Need is \"Love\": Evading Hate-speech Detection",
            "authors": [
                "Tommi Gr\u00f6ndahl",
                "Luca Pajola",
                "Mika Juuti",
                "Mauro Conti",
                "N. Asokan"
            ],
            "abstract": "With the spread of social networks and their unfortunate use for hate speech,\nautomatic detection of the latter has become a pressing problem. In this paper,\nwe reproduce seven state-of-the-art hate speech detection models from prior\nwork, and show that they perform well only when tested on the same type of data\nthey were trained on. Based on these results, we argue that for successful hate\nspeech detection, model architecture is less important than the type of data\nand labeling criteria. We further show that all proposed detection techniques\nare brittle against adversaries who can (automatically) insert typos, change\nword boundaries or add innocuous words to the original hate speech. A\ncombination of these methods is also effective against Google Perspective -- a\ncutting-edge solution from industry. Our experiments demonstrate that\nadversarial training does not completely mitigate the attacks, and using\ncharacter-level features makes the models systematically more attack-resistant\nthan using word-level features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09121v3",
            "date": "2018-08-28 05:16:35+00:00",
            "title": "WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations",
            "authors": [
                "Mohammad Taher Pilehvar",
                "Jose Camacho-Collados"
            ],
            "abstract": "By design, word embeddings are unable to model the dynamic nature of words'\nsemantics, i.e., the property of words to correspond to potentially different\nmeanings. To address this limitation, dozens of specialized meaning\nrepresentation techniques such as sense or contextualized embeddings have been\nproposed. However, despite the popularity of research on this topic, very few\nevaluation benchmarks exist that specifically focus on the dynamic semantics of\nwords. In this paper we show that existing models have surpassed the\nperformance ceiling of the standard evaluation dataset for the purpose, i.e.,\nStanford Contextual Word Similarity, and highlight its shortcomings. To address\nthe lack of a suitable benchmark, we put forward a large-scale Word in Context\ndataset, called WiC, based on annotations curated by experts, for generic\nevaluation of context-sensitive representations. WiC is released in\nhttps://pilehvar.github.io/wic/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09132v2",
            "date": "2018-08-28 06:09:39+00:00",
            "title": "Mapping Natural Language Commands to Web Elements",
            "authors": [
                "Panupong Pasupat",
                "Tian-Shun Jiang",
                "Evan Zheran Liu",
                "Kelvin Guu",
                "Percy Liang"
            ],
            "abstract": "The web provides a rich, open-domain environment with textual, structural,\nand spatial properties. We propose a new task for grounding language in this\nenvironment: given a natural language command (e.g., \"click on the second\narticle\"), choose the correct element on the web page (e.g., a hyperlink or\ntext box). We collected a dataset of over 50,000 commands that capture various\nphenomena such as functional references (e.g. \"find who made this site\"),\nrelational reasoning (e.g. \"article by john\"), and visual reasoning (e.g.\n\"top-most article\"). We also implemented and analyzed three baseline models\nthat capture different phenomena present in the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09147v1",
            "date": "2018-08-28 07:27:45+00:00",
            "title": "Toward Fast and Accurate Neural Discourse Segmentation",
            "authors": [
                "Yizhong Wang",
                "Sujian Li",
                "Jingfeng Yang"
            ],
            "abstract": "Discourse segmentation, which segments texts into Elementary Discourse Units,\nis a fundamental step in discourse analysis. Previous discourse segmenters rely\non complicated hand-crafted features and are not practical in actual use. In\nthis paper, we propose an end-to-end neural segmenter based on BiLSTM-CRF\nframework. To improve its accuracy, we address the problem of data\ninsufficiency by transferring a word representation model that is trained on a\nlarge corpus. We also propose a restricted self-attention mechanism in order to\ncapture useful information within a neighborhood. Experiments on the RST-DT\ncorpus show that our model is significantly faster than previous methods, while\nachieving new state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09160v1",
            "date": "2018-08-28 08:04:21+00:00",
            "title": "Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation",
            "authors": [
                "Hardy",
                "Andreas Vlachos"
            ],
            "abstract": "Recent work on abstractive summarization has made progress with neural\nencoder-decoder architectures. However, such models are often challenged due to\ntheir lack of explicit semantic modeling of the source document and its\nsummary. In this paper, we extend previous work on abstractive summarization\nusing Abstract Meaning Representation (AMR) with a neural language generation\nstage which we guide using the source document. We demonstrate that this\nguidance improves summarization results by 7.4 and 10.5 points in ROUGE-2 using\ngold standard AMR parses and parses obtained from an off-the-shelf parser\nrespectively. We also find that the summarization performance using the latter\nis 2 ROUGE-2 points higher than that of a well-established neural\nencoder-decoder approach trained on a larger dataset. Code is available at\n\\url{https://github.com/sheffieldnlp/AMR2Text-summ}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09178v1",
            "date": "2018-08-28 09:00:37+00:00",
            "title": "Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialogue",
            "authors": [
                "Dieuwke Hupkes",
                "Sanne Bouwmeester",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "We investigate how encoder-decoder models trained on a synthetic dataset of\ntask-oriented dialogues process disfluencies, such as hesitations and\nself-corrections. We find that, contrary to earlier results, disfluencies have\nvery little impact on the task success of seq-to-seq models with attention.\nUsing visualisation and diagnostic classifiers, we analyse the representations\nthat are incrementally built by the model, and discover that models develop\nlittle to no awareness of the structure of disfluencies. However, adding\ndisfluencies to the data appears to help the model create clearer\nrepresentations overall, as evidenced by the attention patterns the different\nmodels exhibit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09180v1",
            "date": "2018-08-28 09:02:48+00:00",
            "title": "What do character-level models learn about morphology? The case of dependency parsing",
            "authors": [
                "Clara Vania",
                "Andreas Grivas",
                "Adam Lopez"
            ],
            "abstract": "When parsing morphologically-rich languages with neural models, it is\nbeneficial to model input at the character level, and it has been claimed that\nthis is because character-level models learn morphology. We test these claims\nby comparing character-level models to an oracle with access to explicit\nmorphological analysis on twelve languages with varying morphological\ntypologies. Our results highlight many strengths of character-level models, but\nalso show that they are poor at disambiguating some words, particularly in the\nface of case syncretism. We then demonstrate that explicitly modeling\nmorphological case improves our best model, showing that character-level models\ncan benefit from targeted forms of explicit morphological modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09187v2",
            "date": "2018-08-28 09:11:49+00:00",
            "title": "Why Do Neural Response Generation Models Prefer Universal Replies?",
            "authors": [
                "Bowen Wu",
                "Nan Jiang",
                "Zhifeng Gao",
                "Mengyuan Li",
                "Zongsheng Wang",
                "Suke Li",
                "Qihang Feng",
                "Wenge Rong",
                "Baoxun Wang"
            ],
            "abstract": "Recent advances in sequence-to-sequence learning reveal a purely data-driven\napproach to the response generation task. Despite its diverse applications,\nexisting neural models are prone to producing short and generic replies, making\nit infeasible to tackle open-domain challenges. In this research, we analyze\nthis critical issue in light of the model's optimization goal and the specific\ncharacteristics of the human-to-human dialog corpus. By decomposing the black\nbox into parts, a detailed analysis of the probability limit was conducted to\nreveal the reason behind these universal replies. Based on these analyses, we\npropose a max-margin ranking regularization term to avoid the models leaning to\nthese replies. Finally, empirical experiments on case studies and benchmarks\nwith several metrics validate this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09238v1",
            "date": "2018-08-28 11:53:05+00:00",
            "title": "Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks",
            "authors": [
                "Martin Schmitt",
                "Simon Steinheber",
                "Konrad Schreiber",
                "Benjamin Roth"
            ],
            "abstract": "In this work, we propose a new model for aspect-based sentiment analysis. In\ncontrast to previous approaches, we jointly model the detection of aspects and\nthe classification of their polarity in an end-to-end trainable neural network.\nWe conduct experiments with different neural architectures and word\nrepresentations on the recent GermEval 2017 dataset. We were able to show\nconsiderable performance gains by using the joint modeling approach in all\nsettings compared to pipeline approaches. The combination of a convolutional\nneural network and fasttext embeddings outperformed the best submission of the\nshared task in 2017, establishing a new state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09308v1",
            "date": "2018-08-28 14:01:07+00:00",
            "title": "Card-660: Cambridge Rare Word Dataset - a Reliable Benchmark for Infrequent Word Representation Models",
            "authors": [
                "Mohammad Taher Pilehvar",
                "Dimitri Kartsaklis",
                "Victor Prokhorov",
                "Nigel Collier"
            ],
            "abstract": "Rare word representation has recently enjoyed a surge of interest, owing to\nthe crucial role that effective handling of infrequent words can play in\naccurate semantic understanding. However, there is a paucity of reliable\nbenchmarks for evaluation and comparison of these techniques. We show in this\npaper that the only existing benchmark (the Stanford Rare Word dataset) suffers\nfrom low-confidence annotations and limited vocabulary; hence, it does not\nconstitute a solid comparison framework. In order to fill this evaluation gap,\nwe propose CAmbridge Rare word Dataset (Card-660), an expert-annotated word\nsimilarity dataset which provides a highly reliable, yet challenging, benchmark\nfor rare word representation techniques. Through a set of experiments we show\nthat even the best mainstream word embeddings, with millions of words in their\nvocabularies, are unable to achieve performances higher than 0.43 (Pearson\ncorrelation) on the dataset, compared to a human-level upperbound of 0.90. We\nrelease the dataset and the annotation materials at\nhttps://pilehvar.github.io/card-660/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09315v1",
            "date": "2018-08-28 14:13:26+00:00",
            "title": "Convolutional Neural Networks with Recurrent Neural Filters",
            "authors": [
                "Yi Yang"
            ],
            "abstract": "We introduce a class of convolutional neural networks (CNNs) that utilize\nrecurrent neural networks (RNNs) as convolution filters. A convolution filter\nis typically implemented as a linear affine transformation followed by a\nnon-linear function, which fails to account for language compositionality. As a\nresult, it limits the use of high-order filters that are often warranted for\nnatural language processing tasks. In this work, we model convolution filters\nwith RNNs that naturally capture compositionality and long-term dependencies in\nlanguage. We show that simple CNN architectures equipped with recurrent neural\nfilters (RNFs) achieve results that are on par with the best published ones on\nthe Stanford Sentiment Treebank and two answer sentence selection datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09352v1",
            "date": "2018-08-28 15:16:17+00:00",
            "title": "Evaluating Theory of Mind in Question Answering",
            "authors": [
                "Aida Nematzadeh",
                "Kaylee Burns",
                "Erin Grant",
                "Alison Gopnik",
                "Thomas L. Griffiths"
            ],
            "abstract": "We propose a new dataset for evaluating question answering models with\nrespect to their capacity to reason about beliefs. Our tasks are inspired by\ntheory-of-mind experiments that examine whether children are able to reason\nabout the beliefs of others, in particular when those beliefs differ from\nreality. We evaluate a number of recent neural models with memory augmentation.\nWe find that all fail on our tasks, which require keeping track of inconsistent\nstates of the world; moreover, the models' accuracy decreases notably when\nrandom sentences are introduced to the tasks at test.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09354v1",
            "date": "2018-08-28 15:19:33+00:00",
            "title": "Universal Dependency Parsing with a General Transition-Based DAG Parser",
            "authors": [
                "Daniel Hershcovich",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "This paper presents our experiments with applying TUPA to the CoNLL 2018 UD\nshared task. TUPA is a general neural transition-based DAG parser, which we use\nto present the first experiments on recovering enhanced dependencies as part of\nthe general parsing task. TUPA was designed for parsing UCCA, a\ncross-linguistic semantic annotation scheme, exhibiting reentrancy,\ndiscontinuity and non-terminal nodes. By converting UD trees and graphs to a\nUCCA-like DAG format, we train TUPA almost without modification on the UD\nparsing task. The generic nature of our approach lends itself naturally to\nmultitask learning. Our code is available at\nhttps://github.com/CoNLL-UD-2018/HUJI",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09357v1",
            "date": "2018-08-28 15:28:25+00:00",
            "title": "Rational Recurrences",
            "authors": [
                "Hao Peng",
                "Roy Schwartz",
                "Sam Thomson",
                "Noah A. Smith"
            ],
            "abstract": "Despite the tremendous empirical success of neural models in natural language\nprocessing, many of them lack the strong intuitions that accompany classical\nmachine learning approaches. Recently, connections have been shown between\nconvolutional neural networks (CNNs) and weighted finite state automata\n(WFSAs), leading to new interpretations and insights. In this work, we show\nthat some recurrent neural networks also share this connection to WFSAs. We\ncharacterize this connection formally, defining rational recurrences to be\nrecurrent hidden state update functions that can be written as the Forward\ncalculation of a finite set of WFSAs. We show that several recent neural models\nuse rational recurrences. Our analysis provides a fresh view of these models\nand facilitates devising new neural architectures that draw inspiration from\nWFSAs. We present one such model, which performs better than two recent\nbaselines on language modeling and text classification. Our results demonstrate\nthat transferring intuitions from classical models like WFSAs can be an\neffective approach to designing and understanding neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09367v1",
            "date": "2018-08-28 15:38:41+00:00",
            "title": "Deriving Machine Attention from Human Rationales",
            "authors": [
                "Yujia Bao",
                "Shiyu Chang",
                "Mo Yu",
                "Regina Barzilay"
            ],
            "abstract": "Attention-based models are successful when trained on large amounts of data.\nIn this paper, we demonstrate that even in the low-resource scenario, attention\ncan be learned effectively. To this end, we start with discrete human-annotated\nrationales and map them into continuous attention. Our central hypothesis is\nthat this mapping is general across domains, and thus can be transferred from\nresource-rich domains to low-resource ones. Our model jointly learns a\ndomain-invariant representation and induces the desired mapping between\nrationales and attention. Our empirical results validate this hypothesis and\nshow that our approach delivers significant gains over state-of-the-art\nbaselines, yielding over 15% average error reduction on benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09374v1",
            "date": "2018-08-28 15:52:19+00:00",
            "title": "A Tree-based Decoder for Neural Machine Translation",
            "authors": [
                "Xinyi Wang",
                "Hieu Pham",
                "Pengcheng Yin",
                "Graham Neubig"
            ],
            "abstract": "Recent advances in Neural Machine Translation (NMT) show that adding\nsyntactic information to NMT systems can improve the quality of their\ntranslations. Most existing work utilizes some specific types of\nlinguistically-inspired tree structures, like constituency and dependency parse\ntrees. This is often done via a standard RNN decoder that operates on a\nlinearized target tree structure. However, it is an open question of what\nspecific linguistic formalism, if any, is the best structural representation\nfor NMT. In this paper, we (1) propose an NMT model that can naturally generate\nthe topology of an arbitrary tree structure on the target side, and (2)\nexperiment with various target tree structures. Our experiments show the\nsurprising result that our model delivers the best improvements with balanced\nbinary trees constructed without any linguistic knowledge; this model\noutperforms standard seq2seq models by up to 2.1 BLEU points, and other methods\nfor incorporating target-side syntax by up to 0.7 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09381v2",
            "date": "2018-08-28 16:05:40+00:00",
            "title": "Understanding Back-Translation at Scale",
            "authors": [
                "Sergey Edunov",
                "Myle Ott",
                "Michael Auli",
                "David Grangier"
            ],
            "abstract": "An effective method to improve neural machine translation with monolingual\ndata is to augment the parallel training corpus with back-translations of\ntarget language sentences. This work broadens the understanding of\nback-translation and investigates a number of methods to generate synthetic\nsource sentences. We find that in all but resource poor settings\nback-translations obtained via sampling or noised beam outputs are most\neffective. Our analysis shows that sampling or noisy synthetic data gives a\nmuch stronger training signal than data generated by beam or greedy search. We\nalso compare how synthetic data compares to genuine bitext and study various\ndomain effects. Finally, we scale to hundreds of millions of monolingual\nsentences and achieve a new state of the art of 35 BLEU on the WMT'14\nEnglish-German test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09386v2",
            "date": "2018-08-28 16:20:20+00:00",
            "title": "Framing and Agenda-setting in Russian News: a Computational Analysis of Intricate Political Strategies",
            "authors": [
                "Anjalie Field",
                "Doron Kliger",
                "Shuly Wintner",
                "Jennifer Pan",
                "Dan Jurafsky",
                "Yulia Tsvetkov"
            ],
            "abstract": "Amidst growing concern over media manipulation, NLP attention has focused on\novert strategies like censorship and \"fake news'\". Here, we draw on two\nconcepts from the political science literature to explore subtler strategies\nfor government media manipulation: agenda-setting (selecting what topics to\ncover) and framing (deciding how topics are covered). We analyze 13 years (100K\narticles) of the Russian newspaper Izvestia and identify a strategy of\ndistraction: articles mention the U.S. more frequently in the month directly\nfollowing an economic downturn in Russia. We introduce embedding-based methods\nfor cross-lingually projecting English frames to Russian, and discover that\nthese articles emphasize U.S. moral failings and threats to the U.S. Our work\noffers new ways to identify subtle media manipulation strategies at the\nintersection of agenda-setting and framing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09401v1",
            "date": "2018-08-28 16:46:30+00:00",
            "title": "Temporal Information Extraction by Predicting Relative Time-lines",
            "authors": [
                "Artuur Leeuwenberg",
                "Marie-Francine Moens"
            ],
            "abstract": "The current leading paradigm for temporal information extraction from text\nconsists of three phases: (1) recognition of events and temporal expressions,\n(2) recognition of temporal relations among them, and (3) time-line\nconstruction from the temporal relations. In contrast to the first two phases,\nthe last phase, time-line construction, received little attention and is the\nfocus of this work. In this paper, we propose a new method to construct a\nlinear time-line from a set of (extracted) temporal relations. But more\nimportantly, we propose a novel paradigm in which we directly predict start and\nend-points for events from the text, constituting a time-line without going\nthrough the intermediate step of prediction of temporal relations as in earlier\nwork. Within this paradigm, we propose two models that predict in linear\ncomplexity, and a new training loss using TimeML-style annotations, yielding\npromising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09408v1",
            "date": "2018-08-28 16:57:37+00:00",
            "title": "Privacy-preserving Neural Representations of Text",
            "authors": [
                "Maximin Coavoux",
                "Shashi Narayan",
                "Shay B. Cohen"
            ],
            "abstract": "This article deals with adversarial attacks towards deep learning systems for\nNatural Language Processing (NLP), in the context of privacy protection. We\nstudy a specific type of attack: an attacker eavesdrops on the hidden\nrepresentations of a neural text classifier and tries to recover information\nabout the input text. Such scenario may arise in situations when the\ncomputation of a neural network is shared across multiple devices, e.g. some\nhidden representation is computed by a user's device and sent to a cloud-based\nmodel. We measure the privacy of a hidden representation by the ability of an\nattacker to predict accurately specific private information from it and\ncharacterize the tradeoff between the privacy and the utility of neural\nrepresentations. Finally, we propose several defense methods based on modified\ntraining objectives and show that they improve the privacy of neural\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09409v2",
            "date": "2018-08-28 16:57:47+00:00",
            "title": "Semantic Role Labeling for Learner Chinese: the Importance of Syntactic Parsing and L2-L1 Parallel Data",
            "authors": [
                "Zi Lin",
                "Yuguang Duan",
                "Yuanyuan Zhao",
                "Weiwei Sun",
                "Xiaojun Wan"
            ],
            "abstract": "This paper studies semantic parsing for interlanguage (L2), taking semantic\nrole labeling (SRL) as a case task and learner Chinese as a case language. We\nfirst manually annotate the semantic roles for a set of learner texts to derive\na gold standard for automatic SRL. Based on the new data, we then evaluate\nthree off-the-shelf SRL systems, i.e., the PCFGLA-parser-based,\nneural-parser-based and neural-syntax-agnostic systems, to gauge how successful\nSRL for learner Chinese can be. We find two non-obvious facts: 1) the\nL1-sentence-trained systems performs rather badly on the L2 data; 2) the\nperformance drop from the L1 data to the L2 data of the two parser-based\nsystems is much smaller, indicating the importance of syntactic parsing in SRL\nfor interlanguages. Finally, the paper introduces a new agreement-based model\nto explore the semantic coherency information in the large-scale L2-L1 parallel\ndata. We then show such information is very effective to enhance SRL for\nlearner texts. Our model achieves an F-score of 72.06, which is a 2.02 point\nimprovement over the best baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09419v1",
            "date": "2018-08-28 17:20:51+00:00",
            "title": "Identifying Well-formed Natural Language Questions",
            "authors": [
                "Manaal Faruqui",
                "Dipanjan Das"
            ],
            "abstract": "Understanding search queries is a hard problem as it involves dealing with\n\"word salad\" text ubiquitously issued by users. However, if a query resembles a\nwell-formed question, a natural language processing pipeline is able to perform\nmore accurate interpretation, thus reducing downstream compounding errors.\nHence, identifying whether or not a query is well formed can enhance query\nunderstanding. Here, we introduce a new task of identifying a well-formed\nnatural language question. We construct and release a dataset of 25,100\npublicly available questions classified into well-formed and non-wellformed\ncategories and report an accuracy of 70.7% on the test set. We also show that\nour classifier can be used to improve the performance of neural\nsequence-to-sequence models for generating questions for reading comprehension.",
            "journal ref": "Proc. of EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09422v1",
            "date": "2018-08-28 17:25:27+00:00",
            "title": "WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse",
            "authors": [
                "Manaal Faruqui",
                "Ellie Pavlick",
                "Ian Tenney",
                "Dipanjan Das"
            ],
            "abstract": "We release a corpus of 43 million atomic edits across 8 languages. These\nedits are mined from Wikipedia edit history and consist of instances in which a\nhuman editor has inserted a single contiguous phrase into, or deleted a single\ncontiguous phrase from, an existing sentence. We use the collected data to show\nthat the language generated during editing differs from the language that we\nobserve in standard corpora, and that models trained on edits encode different\naspects of semantics and discourse than models trained on raw, unstructured\ntext. We release the full corpus as a resource to aid ongoing research in\nsemantics, discourse, and representation learning.",
            "journal ref": "Proc. of EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09463v1",
            "date": "2018-08-28 18:00:44+00:00",
            "title": "Graphene: A Context-Preserving Open Information Extraction System",
            "authors": [
                "Matthias Cetto",
                "Christina Niklaus",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We introduce Graphene, an Open IE system whose goal is to generate accurate,\nmeaningful and complete propositions that may facilitate a variety of\ndownstream semantic applications. For this purpose, we transform syntactically\ncomplex input sentences into clean, compact structures in the form of core\nfacts and accompanying contexts, while identifying the rhetorical relations\nthat hold between them in order to maintain their semantic relationship. In\nthat way, we preserve the context of the relational tuples extracted from a\nsource sentence, generating a novel lightweight semantic representation for\nOpen IE that enhances the expressiveness of the extracted propositions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09465v1",
            "date": "2018-08-28 18:02:31+00:00",
            "title": "The University of Cambridge's Machine Translation Systems for WMT18",
            "authors": [
                "Felix Stahlberg",
                "Adria de Gispert",
                "Bill Byrne"
            ],
            "abstract": "The University of Cambridge submission to the WMT18 news translation task\nfocuses on the combination of diverse models of translation. We compare\nrecurrent, convolutional, and self-attention-based neural models on\nGerman-English, English-German, and Chinese-English. Our final system combines\nall neural models together with a phrase-based SMT system in an MBR-based\nscheme. We report small but consistent gains on top of strong Transformer\nensembles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09468v1",
            "date": "2018-08-28 18:03:32+00:00",
            "title": "Learning To Split and Rephrase From Wikipedia Edit History",
            "authors": [
                "Jan A. Botha",
                "Manaal Faruqui",
                "John Alex",
                "Jason Baldridge",
                "Dipanjan Das"
            ],
            "abstract": "Split and rephrase is the task of breaking down a sentence into shorter ones\nthat together convey the same meaning. We extract a rich new dataset for this\ntask by mining Wikipedia's edit history: WikiSplit contains one million\nnaturally occurring sentence rewrites, providing sixty times more distinct\nsplit examples and a ninety times larger vocabulary than the WebSplit corpus\nintroduced by Narayan et al. (2017) as a benchmark for this task. Incorporating\nWikiSplit as training data produces a model with qualitatively better\npredictions that score 32 BLEU points above the prior best result on the\nWebSplit benchmark.",
            "journal ref": "Proc. of EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09479v1",
            "date": "2018-08-28 18:23:56+00:00",
            "title": "Residualized Factor Adaptation for Community Social Media Prediction Tasks",
            "authors": [
                "Mohammadzaman Zamani",
                "H. Andrew Schwartz",
                "Veronica E. Lynn",
                "Salvatore Giorgi",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Predictive models over social media language have shown promise in capturing\ncommunity outcomes, but approaches thus far largely neglect the\nsocio-demographic context (e.g. age, education rates, race) of the community\nfrom which the language originates. For example, it may be inaccurate to assume\npeople in Mobile, Alabama, where the population is relatively older, will use\nwords the same way as those from San Francisco, where the median age is younger\nwith a higher rate of college education. In this paper, we present residualized\nfactor adaptation, a novel approach to community prediction tasks which both\n(a) effectively integrates community attributes, as well as (b) adapts\nlinguistic features to community attributes (factors). We use eleven\ndemographic and socioeconomic attributes, and evaluate our approach over five\ndifferent community-level predictive tasks, spanning health (heart disease\nmortality, percent fair/poor health), psychology (life satisfaction), and\neconomics (percent housing price increase, foreclosure rate). Our evaluation\nshows that residualized factor adaptation significantly improves 4 out of 5\ncommunity-level outcome predictions over prior state-of-the-art for\nincorporating socio-demographic contexts.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing, pages 3560-3569, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09492v5",
            "date": "2018-08-28 18:48:30+00:00",
            "title": "Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering",
            "authors": [
                "Jianmo Ni",
                "Chenguang Zhu",
                "Weizhu Chen",
                "Julian McAuley"
            ],
            "abstract": "Open-domain question answering remains a challenging task as it requires\nmodels that are capable of understanding questions and answers, collecting\nuseful information, and reasoning over evidence. Previous work typically\nformulates this task as a reading comprehension or entailment problem given\nevidence retrieved from search engines. However, existing techniques struggle\nto retrieve indirectly related evidence when no directly related evidence is\nprovided, especially for complex questions where it is hard to parse precisely\nwhat the question asks. In this paper we propose a retriever-reader model that\nlearns to attend on essential terms during the question answering process. We\nbuild (1) an essential term selector which first identifies the most important\nwords in a question, then reformulates the query and searches for related\nevidence; and (2) an enhanced reader that distinguishes between essential terms\nand distracting words to predict the answer. We evaluate our model on multiple\nopen-domain multiple-choice QA datasets, notably performing at the level of the\nstate-of-the-art on the AI2 Reasoning Challenge (ARC) dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09500v1",
            "date": "2018-08-28 19:11:20+00:00",
            "title": "Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations",
            "authors": [
                "Aditi Chaudhary",
                "Chunting Zhou",
                "Lori Levin",
                "Graham Neubig",
                "David R. Mortensen",
                "Jaime G. Carbonell"
            ],
            "abstract": "Much work in Natural Language Processing (NLP) has been for resource-rich\nlanguages, making generalization to new, less-resourced languages challenging.\nWe present two approaches for improving generalization to low-resourced\nlanguages by adapting continuous word representations using linguistically\nmotivated subword units: phonemes, morphemes and graphemes. Our method requires\nneither parallel corpora nor bilingual dictionaries and provides a significant\ngain in performance over previous methods relying on these resources. We\ndemonstrate the effectiveness of our approaches on Named Entity Recognition for\nfour languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and\nBengali are low resource languages, and also perform experiments on Machine\nTranslation. Exploiting subwords with transfer learning gives us a boost of\n+15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in\nthe monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09502v1",
            "date": "2018-08-28 19:15:57+00:00",
            "title": "Semantic Matching Against a Corpus: New Applications and Methods",
            "authors": [
                "Lucy H. Lin",
                "Scott Miles",
                "Noah A. Smith"
            ],
            "abstract": "We consider the case of a domain expert who wishes to explore the extent to\nwhich a particular idea is expressed in a text collection. We propose the task\nof semantically matching the idea, expressed as a natural language proposition,\nagainst a corpus. We create two preliminary tasks derived from existing\ndatasets, and then introduce a more realistic one on disaster recovery designed\nfor emergency managers, whom we engaged in a user study. On the latter, we find\nthat a new model built from natural language entailment data produces\nhigher-quality matches than simple word-vector averaging, both on\nexpert-crafted queries and on ones produced by the subjects themselves. This\nwork provides a proof-of-concept for such applications of semantic matching and\nillustrates key challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09522v1",
            "date": "2018-08-28 20:13:18+00:00",
            "title": "Layer Trajectory LSTM",
            "authors": [
                "Jinyu Li",
                "Changliang Liu",
                "Yifan Gong"
            ],
            "abstract": "It is popular to stack LSTM layers to get better modeling power, especially\nwhen large amount of training data is available. However, an LSTM-RNN with too\nmany vanilla LSTM layers is very hard to train and there still exists the\ngradient vanishing issue if the network goes too deep. This issue can be\npartially solved by adding skip connections between layers, such as residual\nLSTM. In this paper, we propose a layer trajectory LSTM (ltLSTM) which builds a\nlayer-LSTM using all the layer outputs from a standard multi-layer time-LSTM.\nThis layer-LSTM scans the outputs from time-LSTMs, and uses the summarized\nlayer trajectory information for final senone classification. The\nforward-propagation of time-LSTM and layer-LSTM can be handled in two separate\nthreads in parallel so that the network computation time is the same as the\nstandard time-LSTM. With a layer-LSTM running through layers, a gated path is\nprovided from the output layer to the bottom layer, alleviating the gradient\nvanishing issue. Trained with 30 thousand hours of EN-US Microsoft internal\ndata, the proposed ltLSTM performed significantly better than the standard\nmulti-layer LSTM and residual LSTM, with up to 9.0% relative word error rate\nreduction across different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09542v1",
            "date": "2018-08-28 20:53:56+00:00",
            "title": "Hierarchical Quantized Representations for Script Generation",
            "authors": [
                "Noah Weber",
                "Leena Shekhar",
                "Niranjan Balasubramanian",
                "Nathanael Chambers"
            ],
            "abstract": "Scripts define knowledge about how everyday scenarios (such as going to a\nrestaurant) are expected to unfold. One of the challenges to learning scripts\nis the hierarchical nature of the knowledge. For example, a suspect arrested\nmight plead innocent or guilty, and a very different track of events is then\nexpected to happen. To capture this type of information, we propose an\nautoencoder model with a latent space defined by a hierarchy of categorical\nvariables. We utilize a recently proposed vector quantization based approach,\nwhich allows continuous embeddings to be associated with each latent variable\nvalue. This permits the decoder to softly decide what portions of the latent\nhierarchy to condition on by attending over the value embeddings for a given\nsetting. Our model effectively encodes and generates scripts, outperforming a\nrecent language modeling-based method on several standard tasks, and allowing\nthe autoencoder model to achieve substantially lower perplexity scores compared\nto the previous language modeling-based method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09543v1",
            "date": "2018-08-28 20:55:41+00:00",
            "title": "Towards Semi-Supervised Learning for Deep Semantic Role Labeling",
            "authors": [
                "Sanket Vaibhav Mehta",
                "Jay Yoon Lee",
                "Jaime Carbonell"
            ],
            "abstract": "Neural models have shown several state-of-the-art performances on Semantic\nRole Labeling (SRL). However, the neural models require an immense amount of\nsemantic-role corpora and are thus not well suited for low-resource languages\nor domains. The paper proposes a semi-supervised semantic role labeling method\nthat outperforms the state-of-the-art in limited SRL training corpora. The\nmethod is based on explicitly enforcing syntactic constraints by augmenting the\ntraining objective with a syntactic-inconsistency loss component and uses\nSRL-unlabeled instances to train a joint-objective LSTM. On CoNLL-2012 English\nsection, the proposed semi-supervised training with 1%, 10% SRL-labeled data\nand varying amounts of SRL-unlabeled data achieves +1.58, +0.78 F1,\nrespectively, over the pre-trained models that were trained on SOTA\narchitecture with ELMo on the same SRL-labeled data. Additionally, by using the\nsyntactic-inconsistency loss on inference time, the proposed model achieves\n+3.67, +2.1 F1 over pre-trained model on 1%, 10% SRL-labeled data,\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09564v1",
            "date": "2018-08-28 22:30:11+00:00",
            "title": "Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation",
            "authors": [
                "Renjie Zheng",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "Neural text generation, including neural machine translation, image\ncaptioning, and summarization, has been quite successful recently. However,\nduring training time, typically only one reference is considered for each\nexample, even though there are often multiple references available, e.g., 4\nreferences in NIST MT evaluations, and 5 references in image captioning data.\nWe first investigate several different ways of utilizing multiple human\nreferences during training. But more importantly, we then propose an algorithm\nto generate exponentially many pseudo-references by first compressing existing\nhuman references into lattices and then traversing them to generate new\npseudo-references. These approaches lead to substantial improvements over\nstrong baselines in both machine translation (+1.5 BLEU) and image captioning\n(+3.1 BLEU / +11.7 CIDEr).",
            "journal ref": "Published in EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09582v3",
            "date": "2018-08-28 23:50:22+00:00",
            "title": "Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation",
            "authors": [
                "Yilin Yang",
                "Liang Huang",
                "Mingbo Ma"
            ],
            "abstract": "Beam search is widely used in neural machine translation, and usually\nimproves translation quality compared to greedy search. It has been widely\nobserved that, however, beam sizes larger than 5 hurt translation quality. We\nexplain why this happens, and propose several methods to address this problem.\nFurthermore, we discuss the optimal stopping criteria for these methods.\nResults show that our hyperparameter-free methods outperform the widely-used\nhyperparameter-free heuristic of length normalization by +2.0 BLEU, and achieve\nthe best results among all methods on Chinese-to-English translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09588v1",
            "date": "2018-08-29 00:08:25+00:00",
            "title": "Mapping Language to Code in Programmatic Context",
            "authors": [
                "Srinivasan Iyer",
                "Ioannis Konstas",
                "Alvin Cheung",
                "Luke Zettlemoyer"
            ],
            "abstract": "Source code is rarely written in isolation. It depends significantly on the\nprogrammatic context, such as the class that the code would reside in. To study\nthis phenomenon, we introduce the task of generating class member functions\ngiven English documentation and the programmatic context provided by the rest\nof the class. This task is challenging because the desired code can vary\ngreatly depending on the functionality the class provides (e.g., a sort\nfunction may or may not be available when we are asked to \"return the smallest\nelement\" in a particular member variable list). We introduce CONCODE, a new\nlarge dataset with over 100,000 examples consisting of Java classes from online\ncode repositories, and develop a new encoder-decoder architecture that models\nthe interaction between the method documentation and the class environment. We\nalso present a detailed error analysis suggesting that there is significant\nroom for future work on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09602v1",
            "date": "2018-08-29 01:53:12+00:00",
            "title": "Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction",
            "authors": [
                "Yi Luan",
                "Luheng He",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce a multi-task setup of identifying and classifying entities,\nrelations, and coreference clusters in scientific articles. We create SciERC, a\ndataset that includes annotations for all three tasks and develop a unified\nframework called Scientific Information Extractor (SciIE) for with shared span\nrepresentations. The multi-task setup reduces cascading errors between tasks\nand leverages cross-sentence relations through coreference links. Experiments\nshow that our multi-task model outperforms previous models in scientific\ninformation extraction without using any domain-specific features. We further\nshow that the framework supports construction of a scientific knowledge graph,\nwhich we use to analyze information in scientific literature.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09637v1",
            "date": "2018-08-29 04:59:15+00:00",
            "title": "Decoupling Strategy and Generation in Negotiation Dialogues",
            "authors": [
                "He He",
                "Derek Chen",
                "Anusha Balakrishnan",
                "Percy Liang"
            ],
            "abstract": "We consider negotiation settings in which two agents use natural language to\nbargain on goods. Agents need to decide on both high-level strategy (e.g.,\nproposing \\$50) and the execution of that strategy (e.g., generating \"The bike\nis brand new. Selling for just \\$50.\"). Recent work on negotiation trains\nneural models, but their end-to-end nature makes it hard to control their\nstrategy, and reinforcement learning tends to lead to degenerate solutions. In\nthis paper, we propose a modular approach based on coarse di- alogue acts\n(e.g., propose(price=50)) that decouples strategy and generation. We show that\nwe can flexibly set the strategy using supervised learning, reinforcement\nlearning, or domain-specific knowledge without degeneracy, while our\nretrieval-based generation can maintain context-awareness and produce diverse\nutterances. We test our approach on the recently proposed DEALORNODEAL game,\nand we also collect a richer dataset based on real items on Craigslist. Human\nevaluation shows that our systems achieve higher task success rate and more\nhuman-like negotiation behavior than previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09644v1",
            "date": "2018-08-29 05:32:17+00:00",
            "title": "On Tree-Based Neural Sentence Modeling",
            "authors": [
                "Haoyue Shi",
                "Hao Zhou",
                "Jiaze Chen",
                "Lei Li"
            ],
            "abstract": "Neural networks with tree-based sentence encoders have shown better results\non many downstream tasks. Most of existing tree-based encoders adopt syntactic\nparsing trees as the explicit structure prior. To study the effectiveness of\ndifferent tree structures, we replace the parsing trees with trivial trees\n(i.e., binary balanced tree, left-branching tree and right-branching tree) in\nthe encoders. Though trivial trees contain no syntactic information, those\nencoders get competitive or even better results on all of the ten downstream\ntasks we investigated. This surprising result indicates that explicit syntax\nguidance may not be the main contributor to the superior performances of\ntree-based neural sentence modeling. Further analysis show that tree modeling\ngives better results when crucial words are closer to the final representation.\nAdditional experiments give more clues on how to design an effective tree-based\nencoder. Our code is open-source and available at\nhttps://github.com/ExplorerFreda/TreeEnc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09653v1",
            "date": "2018-08-29 06:32:47+00:00",
            "title": "Neural Metaphor Detection in Context",
            "authors": [
                "Ge Gao",
                "Eunsol Choi",
                "Yejin Choi",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present end-to-end neural models for detecting metaphorical word use in\ncontext. We show that relatively standard BiLSTM models which operate on\ncomplete sentences work well in this setting, in comparison to previous work\nthat used more restricted forms of linguistic context. These models establish a\nnew state-of-the-art on existing verb metaphor detection benchmarks, and show\nstrong performance on jointly predicting the metaphoricity of all words in a\nrunning text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09688v1",
            "date": "2018-08-29 08:56:42+00:00",
            "title": "An Operation Sequence Model for Explainable Neural Machine Translation",
            "authors": [
                "Felix Stahlberg",
                "Danielle Saunders",
                "Bill Byrne"
            ],
            "abstract": "We propose to achieve explainable neural machine translation (NMT) by\nchanging the output representation to explain itself. We present a novel\napproach to NMT which generates the target sentence by monotonically walking\nthrough the source sentence. Word reordering is modeled by operations which\nallow setting markers in the target sentence and move a target-side write head\nbetween those markers. In contrast to many modern neural models, our system\nemits explicit word alignment information which is often crucial to practical\nmachine translation as it improves explainability. Our technique can outperform\na plain text system in terms of BLEU score under the recent Transformer\narchitecture on Japanese-English and Portuguese-English, and is within 0.5 BLEU\ndifference on Spanish-English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09716v1",
            "date": "2018-08-29 10:30:03+00:00",
            "title": "What can we learn from Semantic Tagging?",
            "authors": [
                "Mostafa Abdou",
                "Artur Kulmizev",
                "Vinit Ravishankar",
                "Lasha Abzianidze",
                "Johan Bos"
            ],
            "abstract": "We investigate the effects of multi-task learning using the recently\nintroduced task of semantic tagging. We employ semantic tagging as an auxiliary\ntask for three different NLP tasks: part-of-speech tagging, Universal\nDependency parsing, and Natural Language Inference. We compare full neural\nnetwork sharing, partial neural network sharing, and what we term the learning\nwhat to share setting where negative transfer between tasks is less likely. Our\nfindings show considerable improvements for all tasks, particularly in the\nlearning what to share setting, which shows consistent gains across all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09718v1",
            "date": "2018-08-29 10:32:24+00:00",
            "title": "Characterizing the Influence of Features on Reading Difficulty Estimation for Non-native Readers",
            "authors": [
                "Yi-Ting Huang",
                "Meng Chang Chen",
                "Yeali S. Sun"
            ],
            "abstract": "In recent years, the number of people studying English as a second language\n(ESL) has surpassed the number of native speakers. Recent work have\ndemonstrated the success of providing personalized content based on reading\ndifficulty, such as information retrieval and summarization. However, almost\nall prior studies of reading difficulty are designed for native speakers,\nrather than non-native readers. In this study, we investigate various features\nfor ESL readers, by conducting a linear regression to estimate the reading\nlevel of English language sources. This estimation is based not only on the\ncomplexity of lexical and syntactic features, but also several novel concepts,\nincluding the age of word and grammar acquisition from several sources, word\nsense from WordNet, and the implicit relation between sentences. By employing\nBayesian Information Criterion (BIC) to select the optimal model, we find that\nthe combination of the number of words, the age of word acquisition and the\nheight of the parsing tree generate better results than alternative competing\nmodels. Thus, our results show that proposed second language reading difficulty\nestimation outperforms other first language reading difficulty estimations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09722v1",
            "date": "2018-08-29 10:45:45+00:00",
            "title": "Identifying the sentiment styles of YouTube's vloggers",
            "authors": [
                "Bennett Kleinberg",
                "Maximilian Mozes",
                "Isabelle van der Vegt"
            ],
            "abstract": "Vlogs provide a rich public source of data in a novel setting. This paper\nexamined the continuous sentiment styles employed in 27,333 vlogs using a\ndynamic intra-textual approach to sentiment analysis. Using unsupervised\nclustering, we identified seven distinct continuous sentiment trajectories\ncharacterized by fluctuations of sentiment throughout a vlog's narrative time.\nWe provide a taxonomy of these seven continuous sentiment styles and found that\nvlogs whose sentiment builds up towards a positive ending are the most\nprevalent in our sample. Gender was associated with preferences for different\ncontinuous sentiment trajectories. This paper discusses the findings with\nrespect to previous work and concludes with an outlook towards possible uses of\nthe corpus, method and findings of this paper for related areas of research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09733v1",
            "date": "2018-08-29 11:30:22+00:00",
            "title": "Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging",
            "authors": [
                "Barbara Plank",
                "\u017deljko Agi\u0107"
            ],
            "abstract": "We introduce DsDs: a cross-lingual neural part-of-speech tagger that learns\nfrom disparate sources of distant supervision, and realistically scales to\nhundreds of low-resource languages. The model exploits annotation projection,\ninstance selection, tag dictionaries, morphological lexicons, and distributed\nrepresentations, all in a uniform framework. The approach is simple, yet\nsurprisingly effective, resulting in a new state of the art without access to\nany gold annotated data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09772v2",
            "date": "2018-08-29 12:58:45+00:00",
            "title": "Notes on Deep Learning for NLP",
            "authors": [
                "Antoine J. -P. Tixier"
            ],
            "abstract": "My notes on Deep Learning for NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09861v2",
            "date": "2018-08-29 14:55:31+00:00",
            "title": "Neural Cross-Lingual Named Entity Recognition with Minimal Resources",
            "authors": [
                "Jiateng Xie",
                "Zhilin Yang",
                "Graham Neubig",
                "Noah A. Smith",
                "Jaime Carbonell"
            ],
            "abstract": "For languages with no annotated resources, unsupervised transfer of natural\nlanguage processing models such as named-entity recognition (NER) from\nresource-rich languages would be an appealing capability. However, differences\nin words and word order across languages make it a challenging problem. To\nimprove mapping of lexical items across languages, we propose a method that\nfinds translations based on bilingual word embeddings. To improve robustness to\nword order differences, we propose to use self-attention, which allows for a\ndegree of flexibility with respect to word order. We demonstrate that these\nmethods achieve state-of-the-art or competitive NER performance on commonly\ntested languages under a cross-lingual setting, with much lower resource\nrequirements than past approaches. We also evaluate the challenges of applying\nthese methods to Uyghur, a low-resource language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09891v3",
            "date": "2018-08-28 13:39:44+00:00",
            "title": "A Quantum Many-body Wave Function Inspired Language Modeling Approach",
            "authors": [
                "Peng Zhang",
                "Zhan Su",
                "Lipeng Zhang",
                "Benyou Wang",
                "Dawei Song"
            ],
            "abstract": "The recently proposed quantum language model (QLM) aimed at a principled\napproach to modeling term dependency by applying the quantum probability\ntheory. The latest development for a more effective QLM has adopted word\nembeddings as a kind of global dependency information and integrated the\nquantum-inspired idea in a neural network architecture. While these\nquantum-inspired LMs are theoretically more general and also practically\neffective, they have two major limitations. First, they have not taken into\naccount the interaction among words with multiple meanings, which is common and\nimportant in understanding natural language text. Second, the integration of\nthe quantum-inspired LM with the neural network was mainly for effective\ntraining of parameters, yet lacking a theoretical foundation accounting for\nsuch integration. To address these two issues, in this paper, we propose a\nQuantum Many-body Wave Function (QMWF) inspired language modeling approach. The\nQMWF inspired LM can adopt the tensor product to model the aforesaid\ninteraction among words. It also enables us to reveal the inherent necessity of\nusing Convolutional Neural Network (CNN) in QMWF language modeling.\nFurthermore, our approach delivers a simple algorithm to represent and match\ntext/sentence pairs. Systematic evaluation shows the effectiveness of the\nproposed QMWF-LM algorithm, in comparison with the state of the art\nquantum-inspired LMs and a couple of CNN-based methods, on three typical\nQuestion Answering (QA) datasets.",
            "journal ref": "The 27th ACM International Conference on Information and Knowledge\n  Management, October 22--26, 2018, Torino, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09896v1",
            "date": "2018-08-29 15:52:06+00:00",
            "title": "Review Helpfulness Prediction with Embedding-Gated CNN",
            "authors": [
                "Cen Chen",
                "Minghui Qiu",
                "Yinfei Yang",
                "Jun Zhou",
                "Jun Huang",
                "Xiaolong Li",
                "Forrest Bao"
            ],
            "abstract": "Product reviews, in the form of texts dominantly, significantly help\nconsumers finalize their purchasing decisions. Thus, it is important for\ne-commerce companies to predict review helpfulness to present and recommend\nreviews in a more informative manner. In this work, we introduce a\nconvolutional neural network model that is able to extract abstract features\nfrom multi-granularity representations. Inspired by the fact that different\nwords contribute to the meaning of a sentence differently, we consider to learn\nword-level embedding-gates for all the representations. Furthermore, as it is\ncommon that some product domains/categories have rich user reviews, other\ndomains not. To help domains with less sufficient data, we integrate our model\ninto a cross-domain relationship learning framework for effectively\ntransferring knowledge from other domains. Extensive experiments show that our\nmodel yields better performance than the existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09930v2",
            "date": "2018-08-29 17:10:47+00:00",
            "title": "A Neural Model of Adaptation in Reading",
            "authors": [
                "Marten van Schijndel",
                "Tal Linzen"
            ],
            "abstract": "It has been argued that humans rapidly adapt their lexical and syntactic\nexpectations to match the statistics of the current linguistic context. We\nprovide further support to this claim by showing that the addition of a simple\nadaptation mechanism to a neural language model improves our predictions of\nhuman reading times compared to a non-adaptive model. We analyze the\nperformance of the model on controlled materials from psycholinguistic\nexperiments and show that it adapts not only to lexical items but also to\nabstract syntactic structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09943v1",
            "date": "2018-08-29 17:46:50+00:00",
            "title": "Revisiting Character-Based Neural Machine Translation with Capacity and Compression",
            "authors": [
                "Colin Cherry",
                "George Foster",
                "Ankur Bapna",
                "Orhan Firat",
                "Wolfgang Macherey"
            ],
            "abstract": "Translating characters instead of words or word-fragments has the potential\nto simplify the processing pipeline for neural machine translation (NMT), and\nimprove results by eliminating hyper-parameters and manual feature engineering.\nHowever, it results in longer sequences in which each symbol contains less\ninformation, creating both modeling and computational challenges. In this\npaper, we show that the modeling problem can be solved by standard\nsequence-to-sequence architectures of sufficient depth, and that deep models\noperating at the character level outperform identical models operating over\nword fragments. This result implies that alternative architectures for handling\ncharacter input are better viewed as methods for reducing computation time than\nas improved ways of modeling longer sequences. From this perspective, we\nevaluate several techniques for character-level NMT, verify that they do not\nmatch the performance of our deep character baseline model, and evaluate the\nperformance versus computation time tradeoffs they offer. Within this\nframework, we also perform the first evaluation for NMT of conditional\ncomputation over time, in which the model learns which timesteps can be\nskipped, rather than having them be dictated by a fixed schedule specified\nbefore training begins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10000v1",
            "date": "2018-08-29 18:21:50+00:00",
            "title": "Grammar Induction with Neural Language Models: An Unusual Replication",
            "authors": [
                "Phu Mon Htut",
                "Kyunghyun Cho",
                "Samuel R. Bowman"
            ],
            "abstract": "A substantial thread of recent work on latent tree learning has attempted to\ndevelop neural network models with parse-valued latent variables and train them\non non-parsing tasks, in the hope of having them discover interpretable tree\nstructure. In a recent paper, Shen et al. (2018) introduce such a model and\nreport near-state-of-the-art results on the target task of language modeling,\nand the first strong latent tree learning result on constituency parsing. In an\nattempt to reproduce these results, we discover issues that make the original\nresults hard to trust, including tuning and even training on what is\neffectively the test set. Here, we attempt to reproduce these results in a fair\nexperiment and to extend them to two new datasets. We find that the results of\nthis work are robust: All variants of the model under study outperform all\nlatent tree learning baselines, and perform competitively with symbolic grammar\ninduction systems. We find that this model represents the first empirical\nsuccess for latent tree learning, and that neural network language modeling\nwarrants further study as a setting for grammar induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10006v2",
            "date": "2018-08-29 18:33:11+00:00",
            "title": "Correcting Length Bias in Neural Machine Translation",
            "authors": [
                "Kenton Murray",
                "David Chiang"
            ],
            "abstract": "We study two problems in neural machine translation (NMT). First, in beam\nsearch, whereas a wider beam should in principle help translation, it often\nhurts NMT. Second, NMT has a tendency to produce translations that are too\nshort. Here, we argue that these problems are closely related and both rooted\nin label bias. We show that correcting the brevity problem almost eliminates\nthe beam problem; we compare some commonly-used methods for doing this, finding\nthat a simple per-word reward works well; and we introduce a simple and quick\nway to tune this reward using the perceptron algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10024v2",
            "date": "2018-08-29 20:00:20+00:00",
            "title": "Hard Non-Monotonic Attention for Character-Level Transduction",
            "authors": [
                "Shijie Wu",
                "Pamela Shapiro",
                "Ryan Cotterell"
            ],
            "abstract": "Character-level string-to-string transduction is an important component of\nvarious NLP tasks. The goal is to map an input string to an output string,\nwhere the strings may be of different lengths and have characters taken from\ndifferent alphabets. Recent approaches have used sequence-to-sequence models\nwith an attention mechanism to learn which parts of the input string the model\nshould focus on during the generation of the output string. Both soft attention\nand hard monotonic attention have been used, but hard non-monotonic attention\nhas only been used in other sequence modeling tasks such as image captioning\nand has required a stochastic approximation to compute the gradient. In this\nwork, we introduce an exact, polynomial-time algorithm for marginalizing over\nthe exponential number of non-monotonic alignments between two strings, showing\nthat hard attention models can be viewed as neural reparameterizations of the\nclassical IBM Model 1. We compare soft and hard non-monotonic attention\nexperimentally and find that the exact algorithm significantly improves\nperformance over the stochastic approximation and outperforms soft attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10025v1",
            "date": "2018-08-29 20:01:21+00:00",
            "title": "Retrieval-Based Neural Code Generation",
            "authors": [
                "Shirley Anugrah Hayati",
                "Raphael Olivier",
                "Pravalika Avvaru",
                "Pengcheng Yin",
                "Anthony Tomasic",
                "Graham Neubig"
            ],
            "abstract": "In models to generate program source code from natural language, representing\nthis code in a tree structure has been a common approach. However, existing\nmethods often fail to generate complex code correctly due to a lack of ability\nto memorize large and complex structures. We introduce ReCode, a method based\non subtree retrieval that makes it possible to explicitly reference existing\ncode examples within a neural code generation model. First, we retrieve\nsentences that are similar to input sentences using a dynamic-programming-based\nsentence similarity scoring method. Next, we extract n-grams of action\nsequences that build the associated abstract syntax tree. Finally, we increase\nthe probability of actions that cause the retrieved n-gram action subtree to be\nin the predicted code. We show that our approach improves the performance on\ntwo code generation tasks by up to +2.6 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10059v1",
            "date": "2018-08-29 22:57:55+00:00",
            "title": "Zero-Shot Adaptive Transfer for Conversational Language Understanding",
            "authors": [
                "Sungjin Lee",
                "Rahul Jha"
            ],
            "abstract": "Conversational agents such as Alexa and Google Assistant constantly need to\nincrease their language understanding capabilities by adding new domains. A\nmassive amount of labeled data is required for training each new domain. While\ndomain adaptation approaches alleviate the annotation cost, prior approaches\nsuffer from increased training time and suboptimal concept alignments. To\ntackle this, we introduce a novel Zero-Shot Adaptive Transfer method for slot\ntagging that utilizes the slot description for transferring reusable concepts\nacross domains, and enjoys efficient training without any explicit concept\nalignments. Extensive experimentation over a dataset of 10 domains relevant to\nour commercial personal digital assistant shows that our model outperforms\nprevious state-of-the-art systems by a large margin, and achieves an even\nhigher improvement in the low data regime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10113v3",
            "date": "2018-08-30 04:30:33+00:00",
            "title": "Story Ending Generation with Incremental Encoding and Commonsense Knowledge",
            "authors": [
                "Jian Guan",
                "Yansen Wang",
                "Minlie Huang"
            ],
            "abstract": "Generating a reasonable ending for a given story context, i.e., story ending\ngeneration, is a strong indication of story comprehension. This task requires\nnot only to understand the context clues which play an important role in\nplanning the plot but also to handle implicit knowledge to make a reasonable,\ncoherent story.\n  In this paper, we devise a novel model for story ending generation. The model\nadopts an incremental encoding scheme to represent context clues which are\nspanning in the story context. In addition, commonsense knowledge is applied\nthrough multi-source attention to facilitate story comprehension, and thus to\nhelp generate coherent and reasonable endings. Through building context clues\nand using implicit knowledge, the model is able to produce reasonable story\nendings. context clues implied in the post and make the inference based on it.\n  Automatic and manual evaluation shows that our model can generate more\nreasonable story endings than state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10143v2",
            "date": "2018-08-30 07:03:51+00:00",
            "title": "Direct Output Connection for a High-Rank Language Model",
            "authors": [
                "Sho Takase",
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "This paper proposes a state-of-the-art recurrent neural network (RNN)\nlanguage model that combines probability distributions computed not only from a\nfinal RNN layer but also from middle layers. Our proposed method raises the\nexpressive power of a language model based on the matrix factorization\ninterpretation of language modeling introduced by Yang et al. (2018). The\nproposed method improves the current state-of-the-art language model and\nachieves the best score on the Penn Treebank and WikiText-2, which are the\nstandard benchmark datasets. Moreover, we indicate our proposed method\ncontributes to two application tasks: machine translation and headline\ngeneration. Our code is publicly available at:\nhttps://github.com/nttcslab-nlp/doc_lm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10192v2",
            "date": "2018-08-30 09:12:19+00:00",
            "title": "Towards a Better Metric for Evaluating Question Generation Systems",
            "authors": [
                "Preksha Nema",
                "Mitesh M. Khapra"
            ],
            "abstract": "There has always been criticism for using $n$-gram based similarity metrics,\nsuch as BLEU, NIST, etc, for evaluating the performance of NLG systems.\nHowever, these metrics continue to remain popular and are recently being used\nfor evaluating the performance of systems which automatically generate\nquestions from documents, knowledge graphs, images, etc. Given the rising\ninterest in such automatic question generation (AQG) systems, it is important\nto objectively examine whether these metrics are suitable for this task. In\nparticular, it is important to verify whether such metrics used for evaluating\nAQG systems focus on answerability of the generated question by preferring\nquestions which contain all relevant information such as question type\n(Wh-types), entities, relations, etc. In this work, we show that current\nautomatic evaluation metrics based on $n$-gram similarity do not always\ncorrelate well with human judgments about answerability of a question. To\nalleviate this problem and as a first step towards better evaluation metrics\nfor AQG, we introduce a scoring function to capture answerability and show that\nwhen this scoring function is integrated with existing metrics, they correlate\nsignificantly better with human judgments. The scripts and data developed as a\npart of this work are made publicly available at\nhttps://github.com/PrekshaNema25/Answerability-Metric",
            "journal ref": "EMNLP, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10196v1",
            "date": "2018-08-30 09:30:14+00:00",
            "title": "Pronoun Translation in English-French Machine Translation: An Analysis of Error Types",
            "authors": [
                "Christian Hardmeier",
                "Liane Guillou"
            ],
            "abstract": "Pronouns are a long-standing challenge in machine translation. We present a\nstudy of the performance of a range of rule-based, statistical and neural MT\nsystems on pronoun translation based on an extensive manual evaluation using\nthe PROTEST test suite, which enables a fine-grained analysis of different\npronoun types and sheds light on the difficulties of the task. We find that the\nrule-based approaches in our corpus perform poorly as a result of\noversimplification, whereas SMT and early NMT systems exhibit significant\nshortcomings due to a lack of awareness of the functional and referential\nproperties of pronouns. A recent Transformer-based NMT system with\ncross-sentence context shows very promising results on non-anaphoric pronouns\nand intra-sentential anaphora, but there is still considerable room for\nimprovement in examples with cross-sentence dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10239v1",
            "date": "2018-08-30 11:47:07+00:00",
            "title": "Learning to adapt: a meta-learning approach for speaker adaptation",
            "authors": [
                "Ond\u0159ej Klejch",
                "Joachim Fainberg",
                "Peter Bell"
            ],
            "abstract": "The performance of automatic speech recognition systems can be improved by\nadapting an acoustic model to compensate for the mismatch between training and\ntesting conditions, for example by adapting to unseen speakers. The success of\nspeaker adaptation methods relies on selecting weights that are suitable for\nadaptation and using good adaptation schedules to update these weights in order\nnot to overfit to the adaptation data. In this paper we investigate a\nprincipled way of adapting all the weights of the acoustic model using a\nmeta-learning. We show that the meta-learner can learn to perform supervised\nand unsupervised speaker adaptation and that it outperforms a strong baseline\nadapting LHUC parameters when adapting a DNN AM with 1.5M parameters. We also\nreport initial experiments on adapting TDNN AMs, where the meta-learner\nachieves comparable performance with LHUC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10245v1",
            "date": "2018-08-30 12:15:31+00:00",
            "title": "Comparative Studies of Detecting Abusive Language on Twitter",
            "authors": [
                "Younghun Lee",
                "Seunghyun Yoon",
                "Kyomin Jung"
            ],
            "abstract": "The context-dependent nature of online aggression makes annotating large\ncollections of data extremely difficult. Previously studied datasets in abusive\nlanguage detection have been insufficient in size to efficiently train deep\nlearning models. Recently, Hate and Abusive Speech on Twitter, a dataset much\ngreater in size and reliability, has been released. However, this dataset has\nnot been comprehensively studied to its potential. In this paper, we conduct\nthe first comparative study of various learning models on Hate and Abusive\nSpeech on Twitter, and discuss the possibility of using additional features and\ncontext data for improvements. Experimental results show that bidirectional GRU\nnetworks trained on word-level features, with Latent Topic Clustering modules,\nis the most accurate model scoring 0.805 F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10267v1",
            "date": "2018-08-30 13:18:57+00:00",
            "title": "Multi-Source Syntactic Neural Machine Translation",
            "authors": [
                "Anna Currey",
                "Kenneth Heafield"
            ],
            "abstract": "We introduce a novel multi-source technique for incorporating source syntax\ninto neural machine translation using linearized parses. This is achieved by\nemploying separate encoders for the sequential and parsed versions of the same\nsource sentence; the resulting representations are then combined using a\nhierarchical attention mechanism. The proposed model improves over both seq2seq\nand parsed baselines by over 1 BLEU on the WMT17 English-German task. Further\nanalysis shows that our multi-source syntactic model is able to translate\nsuccessfully without any parsed input, unlike standard parsed methods. In\naddition, performance does not deteriorate as much on long sentences as for the\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10290v2",
            "date": "2018-08-30 13:36:04+00:00",
            "title": "Acquiring Annotated Data with Cross-lingual Explicitation for Implicit Discourse Relation Classification",
            "authors": [
                "Wei Shi",
                "Frances Yung",
                "Vera Demberg"
            ],
            "abstract": "Implicit discourse relation classification is one of the most challenging and\nimportant tasks in discourse parsing, due to the lack of connective as strong\nlinguistic cues. A principle bottleneck to further improvement is the shortage\nof training data (ca.~16k instances in the PDTB). Shi et al. (2017) proposed to\nacquire additional data by exploiting connectives in translation: human\ntranslators mark discourse relations which are implicit in the source language\nexplicitly in the translation. Using back-translations of such explicitated\nconnectives improves discourse relation parsing performance. This paper\naddresses the open question of whether the choice of the translation language\nmatters, and whether multiple translations into different languages can be\neffectively used to improve the quality of the additional data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10326v3",
            "date": "2018-08-30 14:51:43+00:00",
            "title": "Generalize Symbolic Knowledge With Neural Rule Engine",
            "authors": [
                "Shen Li",
                "Hengru Xu",
                "Zhengdong Lu"
            ],
            "abstract": "As neural networks have dominated the state-of-the-art results in a wide\nrange of NLP tasks, it attracts considerable attention to improve the\nperformance of neural models by integrating symbolic knowledge. Different from\nexisting works, this paper investigates the combination of these two powerful\nparadigms from the knowledge-driven side. We propose Neural Rule Engine (NRE),\nwhich can learn knowledge explicitly from logic rules and then generalize them\nimplicitly with neural networks. NRE is implemented with neural module networks\nin which each module represents an action of a logic rule. The experiments show\nthat NRE could greatly improve the generalization abilities of logic rules with\na significant increase in recall. Meanwhile, the precision is still maintained\nat a high level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10399v1",
            "date": "2018-08-30 17:07:47+00:00",
            "title": "Modeling Empathy and Distress in Reaction to News Stories",
            "authors": [
                "Sven Buechel",
                "Anneke Buffone",
                "Barry Slaff",
                "Lyle Ungar",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Computational detection and understanding of empathy is an important factor\nin advancing human-computer interaction. Yet to date, text-based empathy\nprediction has the following major limitations: It underestimates the\npsychological complexity of the phenomenon, adheres to a weak notion of ground\ntruth where empathic states are ascribed by third parties, and lacks a shared\ncorpus. In contrast, this contribution presents the first publicly available\ngold standard for empathy prediction. It is constructed using a novel\nannotation methodology which reliably captures empathy assessments by the\nwriter of a statement using multi-item scales. This is also the first\ncomputational work distinguishing between multiple forms of empathy, empathic\nconcern, and personal distress, as recognized throughout psychology. Finally,\nwe present experimental results for three different predictive models, of which\na CNN performs the best.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10432v1",
            "date": "2018-08-30 17:49:42+00:00",
            "title": "Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation",
            "authors": [
                "Antonio Toral",
                "Sheila Castilho",
                "Ke Hu",
                "Andy Way"
            ],
            "abstract": "We reassess a recent study (Hassan et al., 2018) that claimed that machine\ntranslation (MT) has reached human parity for the translation of news from\nChinese into English, using pairwise ranking and considering three variables\nthat were not taken into account in that previous study: the language in which\nthe source side of the test set was originally written, the translation\nproficiency of the evaluators, and the provision of inter-sentential context.\nIf we consider only original source text (i.e. not translated from another\nlanguage, or translationese), then we find evidence showing that human parity\nhas not been achieved. We compare the judgments of professional translators\nagainst those of non-experts and discover that those of the experts result in\nhigher inter-annotator agreement and better discrimination between human and\nmachine translations. In addition, we analyse the human translations of the\ntest set and identify important translation issues. Finally, based on these\nfindings, we provide a set of recommendations for future human evaluations of\nMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10485v1",
            "date": "2018-08-30 19:01:20+00:00",
            "title": "Syntactic Scaffolds for Semantic Structures",
            "authors": [
                "Swabha Swayamdipta",
                "Sam Thomson",
                "Kenton Lee",
                "Luke Zettlemoyer",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "We introduce the syntactic scaffold, an approach to incorporating syntactic\ninformation into semantic tasks. Syntactic scaffolds avoid expensive syntactic\nprocessing at runtime, only making use of a treebank during training, through a\nmultitask objective. We improve over strong baselines on PropBank semantics,\nframe semantics, and coreference resolution, achieving competitive performance\non all three tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10503v1",
            "date": "2018-08-30 20:19:02+00:00",
            "title": "Iterative Recursive Attention Model for Interpretable Sequence Classification",
            "authors": [
                "Martin Tutek",
                "Jan \u0160najder"
            ],
            "abstract": "Natural language processing has greatly benefited from the introduction of\nthe attention mechanism. However, standard attention models are of limited\ninterpretability for tasks that involve a series of inference steps. We\ndescribe an iterative recursive attention model, which constructs incremental\nrepresentations of input data through reusing results of previously computed\nqueries. We train our model on sentiment classification datasets and\ndemonstrate its capacity to identify and combine different aspects of the input\nin an easily interpretable manner, while obtaining performance close to the\nstate of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10583v2",
            "date": "2018-08-31 03:11:08+00:00",
            "title": "AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale",
            "authors": [
                "Jiayu Du",
                "Xingyu Na",
                "Xuechen Liu",
                "Hui Bu"
            ],
            "abstract": "AISHELL-1 is by far the largest open-source speech corpus available for\nMandarin speech recognition research. It was released with a baseline system\ncontaining solid training and testing pipelines for Mandarin ASR. In AISHELL-2,\n1000 hours of clean read-speech data from iOS is published, which is free for\nacademic usage. On top of AISHELL-2 corpus, an improved recipe is developed and\nreleased, containing key components for industrial applications, such as\nChinese word segmentation, flexible vocabulary expension and phone set\ntransformation etc. Pipelines support various state-of-the-art techniques, such\nas time-delayed neural networks and Lattic-Free MMI objective funciton. In\naddition, we also release dev and test data from other channels(Android and\nMic). For research community, we hope that AISHELL-2 corpus can be a solid\nresource for topics like transfer learning and robust ASR. For industry, we\nhope AISHELL-2 recipe can be a helpful reference for building meaningful\nindustrial systems and products.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10592v1",
            "date": "2018-08-31 04:14:40+00:00",
            "title": "Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report",
            "authors": [
                "Renjie Zheng",
                "Yilin Yang",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "This paper describes multimodal machine translation systems developed jointly\nby Oregon State University and Baidu Research for WMT 2018 Shared Task on\nmultimodal translation. In this paper, we introduce a simple approach to\nincorporate image information by feeding image features to the decoder side. We\nalso explore different sequence level training methods including scheduled\nsampling and reinforcement learning which lead to substantial improvements. Our\nsystems ensemble several models using different architectures and training\nmethods and achieve the best performance for three subtasks: En-De and En-Cs in\ntask 1 and (En+De+Fr)-Cs task 1B.",
            "journal ref": "Published in WMT 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10596v1",
            "date": "2018-08-31 04:27:41+00:00",
            "title": "Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation",
            "authors": [
                "Xisen Jin",
                "Wenqiang Lei",
                "Zhaochun Ren",
                "Hongshen Chen",
                "Shangsong Liang",
                "Yihong Zhao",
                "Dawei Yin"
            ],
            "abstract": "The task of dialogue generation aims to automatically provide responses given\nprevious utterances. Tracking dialogue states is an important ingredient in\ndialogue generation for estimating users' intention. However, the\n\\emph{expensive nature of state labeling} and the \\emph{weak interpretability}\nmake the dialogue state tracking a challenging problem for both task-oriented\nand non-task-oriented dialogue generation: For generating responses in\ntask-oriented dialogues, state tracking is usually learned from manually\nannotated corpora, where the human annotation is expensive for training; for\ngenerating responses in non-task-oriented dialogues, most of existing work\nneglects the explicit state tracking due to the unlimited number of dialogue\nstates.\n  In this paper, we propose the \\emph{semi-supervised explicit dialogue state\ntracker} (SEDST) for neural dialogue generation. To this end, our approach has\ntwo core ingredients: \\emph{CopyFlowNet} and \\emph{posterior regularization}.\nSpecifically, we propose an encoder-decoder architecture, named\n\\emph{CopyFlowNet}, to represent an explicit dialogue state with a\nprobabilistic distribution over the vocabulary space. To optimize the training\nprocedure, we apply a posterior regularization strategy to integrate indirect\nsupervision. Extensive experiments conducted on both task-oriented and\nnon-task-oriented dialogue corpora demonstrate the effectiveness of our\nproposed model. Moreover, we find that our proposed semi-supervised dialogue\nstate tracker achieves a comparable performance as state-of-the-art supervised\nlearning baselines in state tracking procedure.",
            "journal ref": "The 27th ACM International Conference on Information and Knowledge\n  Management, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10627v1",
            "date": "2018-08-31 08:21:45+00:00",
            "title": "Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items",
            "authors": [
                "Jaap Jumelet",
                "Dieuwke Hupkes"
            ],
            "abstract": "In this paper, we attempt to link the inner workings of a neural language\nmodel to linguistic theory, focusing on a complex phenomenon well discussed in\nformal linguis- tics: (negative) polarity items. We briefly discuss the leading\nhypotheses about the licensing contexts that allow negative polarity items and\nevaluate to what extent a neural language model has the ability to correctly\nprocess a subset of such constructions. We show that the model finds a relation\nbetween the licensing context and the negative polarity item and appears to be\naware of the scope of this context, which we extract from a parse tree of the\nsentence. With this research, we hope to pave the way for other studies linking\nformal linguistics to deep learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10653v2",
            "date": "2018-08-31 09:55:06+00:00",
            "title": "An Empirical Analysis of the Role of Amplifiers, Downtoners, and Negations in Emotion Classification in Microblogs",
            "authors": [
                "Florian Strohm",
                "Roman Klinger"
            ],
            "abstract": "The effect of amplifiers, downtoners, and negations has been studied in\ngeneral and particularly in the context of sentiment analysis. However, there\nis only limited work which aims at transferring the results and methods to\ndiscrete classes of emotions, e. g., joy, anger, fear, sadness, surprise, and\ndisgust. For instance, it is not straight-forward to interpret which emotion\nthe phrase \"not happy\" expresses. With this paper, we aim at obtaining a better\nunderstanding of such modifiers in the context of emotion-bearing words and\ntheir impact on document-level emotion classification, namely, microposts on\nTwitter. We select an appropriate scope detection method for modifiers of\nemotion words, incorporate it in a document-level emotion classification model\nas additional bag of words and show that this approach improves the performance\nof emotion classification. In addition, we build a term weighting approach\nbased on the different modifiers into a lexical model for the analysis of the\nsemantics of modifiers and their impact on emotion meaning. We show that\namplifiers separate emotions expressed with an emotion- bearing word more\nclearly from other secondary connotations. Downtoners have the opposite effect.\nIn addition, we discuss the meaning of negations of emotion-bearing words. For\ninstance we show empirically that \"not happy\" is closer to sadness than to\nanger and that fear-expressing words in the scope of downtoners often express\nsurprise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10681v1",
            "date": "2018-08-31 11:14:36+00:00",
            "title": "Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation",
            "authors": [
                "Nikolaos Pappas",
                "Lesly Miculicich Werlen",
                "James Henderson"
            ],
            "abstract": "Tying the weights of the target word embeddings with the target word\nclassifiers of neural machine translation models leads to faster training and\noften to better translation quality. Given the success of this parameter\nsharing, we investigate other forms of sharing in between no sharing and hard\nequality of parameters. In particular, we propose a structure-aware output\nlayer which captures the semantic structure of the output space of words within\na joint input-output embedding. The model is a generalized form of weight tying\nwhich shares parameters but allows learning a more flexible relationship with\ninput word embeddings and allows the effective capacity of the output layer to\nbe controlled. In addition, the model shares weights across output classifiers\nand translation contexts which allows it to better leverage prior knowledge\nabout them. Our evaluation on English-to-Finnish and English-to-German datasets\nshows the effectiveness of the method against strong encoder-decoder baselines\ntrained with or without weight tying.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10685v3",
            "date": "2018-08-31 11:20:53+00:00",
            "title": "Extracting Keywords from Open-Ended Business Survey Questions",
            "authors": [
                "Barbara McGillivray",
                "Gard Jenset",
                "Dominik Heil"
            ],
            "abstract": "Open-ended survey data constitute an important basis in research as well as\nfor making business decisions. Collecting and manually analysing free-text\nsurvey data is generally more costly than collecting and analysing survey data\nconsisting of answers to multiple-choice questions. Yet free-text data allow\nfor new content to be expressed beyond predefined categories and are a very\nvaluable source of new insights into people's opinions. At the same time,\nsurveys always make ontological assumptions about the nature of the entities\nthat are researched, and this has vital ethical consequences. Human\ninterpretations and opinions can only be properly ascertained in their richness\nusing textual data sources; if these sources are analyzed appropriately, the\nessential linguistic nature of humans and social entities is safeguarded.\nNatural Language Processing (NLP) offers possibilities for meeting this ethical\nbusiness challenge by automating the analysis of natural language and thus\nallowing for insightful investigations of human judgements. We present a\ncomputational pipeline for analysing large amounts of responses to open-ended\nquestions in surveys and extract keywords that appropriately represent people's\nopinions. This pipeline addresses the need to perform such tasks outside the\nscope of both commercial software and bespoke analysis, exceeds the performance\nto state-of-the-art systems, and performs this task in a transparent way that\nallows for scrutinising and exposing potential biases in the analysis.\nFollowing the principle of Open Data Science, our code is open-source and\ngeneralizable to other datasets.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2020, Project (March\n  17, 2020) jdmdh:5398",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10701v1",
            "date": "2018-08-31 12:08:55+00:00",
            "title": "Imitation Learning for Neural Morphological String Transduction",
            "authors": [
                "Peter Makarov",
                "Simon Clematide"
            ],
            "abstract": "We employ imitation learning to train a neural transition-based string\ntransducer for morphological tasks such as inflection generation and\nlemmatization. Previous approaches to training this type of model either rely\non an external character aligner for the production of gold action sequences,\nwhich results in a suboptimal model due to the unwarranted dependence on a\nsingle gold action sequence despite spurious ambiguity, or require warm\nstarting with an MLE model. Our approach only requires a simple expert policy,\neliminating the need for a character aligner or warm start. It also addresses\nfamiliar MLE training biases and leads to strong and state-of-the-art\nperformance on several benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10785v1",
            "date": "2018-08-31 14:38:50+00:00",
            "title": "Multimodal Continuous Turn-Taking Prediction Using Multiscale RNNs",
            "authors": [
                "Matthew Roddy",
                "Gabriel Skantze",
                "Naomi Harte"
            ],
            "abstract": "In human conversational interactions, turn-taking exchanges can be\ncoordinated using cues from multiple modalities. To design spoken dialog\nsystems that can conduct fluid interactions it is desirable to incorporate cues\nfrom separate modalities into turn-taking models. We propose that there is an\nappropriate temporal granularity at which modalities should be modeled. We\ndesign a multiscale RNN architecture to model modalities at separate timescales\nin a continuous manner. Our results show that modeling linguistic and acoustic\nfeatures at separate temporal rates can be beneficial for turn-taking modeling.\nWe also show that our approach can be used to incorporate gaze features into\nturn-taking models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10791v1",
            "date": "2018-08-31 14:54:49+00:00",
            "title": "Cognate-aware morphological segmentation for multilingual neural translation",
            "authors": [
                "Stig-Arne Gr\u00f6nroos",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "This article describes the Aalto University entry to the WMT18 News\nTranslation Shared Task. We participate in the multilingual subtrack with a\nsystem trained under the constrained condition to translate from English to\nboth Finnish and Estonian. The system is based on the Transformer model. We\nfocus on improving the consistency of morphological segmentation for words that\nare similar orthographically, semantically, and distributionally; such words\ninclude etymological cognates, loan words, and proper names. For this, we\nintroduce Cognate Morfessor, a multilingual variant of the Morfessor method. We\nshow that our approach improves the translation quality particularly for\nEstonian, which has less resources for training the translation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10802v2",
            "date": "2018-08-31 15:14:59+00:00",
            "title": "The MeMAD Submission to the WMT18 Multimodal Translation Task",
            "authors": [
                "Stig-Arne Gr\u00f6nroos",
                "Benoit Huet",
                "Mikko Kurimo",
                "Jorma Laaksonen",
                "Bernard Merialdo",
                "Phu Pham",
                "Mats Sj\u00f6berg",
                "Umut Sulubacak",
                "J\u00f6rg Tiedemann",
                "Raphael Troncy",
                "Ra\u00fal V\u00e1zquez"
            ],
            "abstract": "This paper describes the MeMAD project entry to the WMT Multimodal Machine\nTranslation Shared Task.\n  We propose adapting the Transformer neural machine translation (NMT)\narchitecture to a multi-modal setting. In this paper, we also describe the\npreliminary experiments with text-only translation systems leading us up to\nthis choice.\n  We have the top scoring system for both English-to-German and\nEnglish-to-French, according to the automatic metrics for flickr18.\n  Our experiments show that the effect of the visual features in our system is\nsmall. Our largest gains come from the quality of the underlying text-only NMT\nsystem. We find that appropriate use of additional data is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10805v2",
            "date": "2018-08-31 15:21:05+00:00",
            "title": "Spherical Latent Spaces for Stable Variational Autoencoders",
            "authors": [
                "Jiacheng Xu",
                "Greg Durrett"
            ],
            "abstract": "A hallmark of variational autoencoders (VAEs) for text processing is their\ncombination of powerful encoder-decoder models, such as LSTMs, with simple\nlatent distributions, typically multivariate Gaussians. These models pose a\ndifficult optimization problem: there is an especially bad local optimum where\nthe variational posterior always equals the prior and the model does not use\nthe latent variable at all, a kind of \"collapse\" which is encouraged by the KL\ndivergence term of the objective. In this work, we experiment with another\nchoice of latent distribution, namely the von Mises-Fisher (vMF) distribution,\nwhich places mass on the surface of the unit hypersphere. With this choice of\nprior and posterior, the KL divergence term now only depends on the variance of\nthe vMF distribution, giving us the ability to treat it as a fixed\nhyperparameter. We show that doing so not only averts the KL collapse, but\nconsistently gives better likelihoods than Gaussians across a range of modeling\nconditions, including recurrent language modeling and bag-of-words document\nmodeling. An analysis of the properties of our vMF representations shows that\nthey learn richer and more nuanced structures in their latent representations\nthan their Gaussian counterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00231v1",
            "date": "2018-02-01 10:35:17+00:00",
            "title": "Adapting predominant and novel sense discovery algorithms for identifying corpus-specific sense differences",
            "authors": [
                "Binny Mathew",
                "Suman Kalyan Maity",
                "Pratip Sarkar",
                "Animesh Mukherjee",
                "Pawan Goyal"
            ],
            "abstract": "Word senses are not static and may have temporal, spatial or corpus-specific\nscopes. Identifying such scopes might benefit the existing WSD systems largely.\nIn this paper, while studying corpus specific word senses, we adapt three\nexisting predominant and novel-sense discovery algorithms to identify these\ncorpus-specific senses. We make use of text data available in the form of\nmillions of digitized books and newspaper archives as two different sources of\ncorpora and propose automated methods to identify corpus-specific word senses\nat various time points. We conduct an extensive and thorough human judgment\nexperiment to rigorously evaluate and compare the performance of these\napproaches. Post adaptation, the output of the three algorithms are in the same\nformat and the accuracy results are also comparable, with roughly 45-60% of the\nreported corpus-specific senses being judged as genuine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00273v1",
            "date": "2018-02-01 12:58:16+00:00",
            "title": "Emerging Language Spaces Learned From Massively Multilingual Corpora",
            "authors": [
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Translations capture important information about languages that can be used\nas implicit supervision in learning linguistic properties and semantic\nrepresentations. In an information-centric view, translated texts may be\nconsidered as semantic mirrors of the original text and the significant\nvariations that we can observe across various languages can be used to\ndisambiguate a given expression using the linguistic signal that is grounded in\ntranslation. Parallel corpora consisting of massive amounts of human\ntranslations with a large linguistic variation can be applied to increase\nabstractions and we propose the use of highly multilingual machine translation\nmodels to find language-independent meaning representations. Our initial\nexperiments show that neural machine translation models can indeed learn in\nsuch a setup and we can show that the learning algorithm picks up information\nabout the relation between languages in order to optimize transfer leaning with\nshared parameters. The model creates a continuous language space that\nrepresents relationships in terms of geometric distances, which we can\nvisualize to illustrate how languages cluster according to language families\nand groups. Does this open the door for new ideas of data-driven language\ntypology with promising models and techniques in empirical cross-linguistic\nresearch?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00500v2",
            "date": "2018-02-01 21:50:40+00:00",
            "title": "Goal-Oriented Chatbot Dialog Management Bootstrapping with Transfer Learning",
            "authors": [
                "Vladimir Ilievski",
                "Claudiu Musat",
                "Andreea Hossmann",
                "Michael Baeriswyl"
            ],
            "abstract": "Goal-Oriented (GO) Dialogue Systems, colloquially known as goal oriented\nchatbots, help users achieve a predefined goal (e.g. book a movie ticket)\nwithin a closed domain. A first step is to understand the user's goal by using\nnatural language understanding techniques. Once the goal is known, the bot must\nmanage a dialogue to achieve that goal, which is conducted with respect to a\nlearnt policy. The success of the dialogue system depends on the quality of the\npolicy, which is in turn reliant on the availability of high-quality training\ndata for the policy learning method, for instance Deep Reinforcement Learning.\n  Due to the domain specificity, the amount of available data is typically too\nlow to allow the training of good dialogue policies. In this paper we introduce\na transfer learning method to mitigate the effects of the low in-domain data\navailability. Our transfer learning based approach improves the bot's success\nrate by 20% in relative terms for distant domains and we more than double it\nfor close domains, compared to the model without transfer learning. Moreover,\nthe transfer learning chatbots learn the policy up to 5 to 10 times faster.\nFinally, as the transfer learning approach is complementary to additional\nprocessing such as warm-starting, we show that their joint application gives\nthe best outcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00757v2",
            "date": "2018-02-02 16:26:39+00:00",
            "title": "Submodularity-Inspired Data Selection for Goal-Oriented Chatbot Training Based on Sentence Embeddings",
            "authors": [
                "Mladen Dimovski",
                "Claudiu Musat",
                "Vladimir Ilievski",
                "Andreea Hossmann",
                "Michael Baeriswyl"
            ],
            "abstract": "Spoken language understanding (SLU) systems, such as goal-oriented chatbots\nor personal assistants, rely on an initial natural language understanding (NLU)\nmodule to determine the intent and to extract the relevant information from the\nuser queries they take as input. SLU systems usually help users to solve\nproblems in relatively narrow domains and require a large amount of in-domain\ntraining data. This leads to significant data availability issues that inhibit\nthe development of successful systems. To alleviate this problem, we propose a\ntechnique of data selection in the low-data regime that enables us to train\nwith fewer labeled sentences, thus smaller labelling costs.\n  We propose a submodularity-inspired data ranking function, the ratio-penalty\nmarginal gain, for selecting data points to label based only on the information\nextracted from the textual embedding space. We show that the distances in the\nembedding space are a viable source of information that can be used for data\nselection. Our method outperforms two known active learning techniques and\nenables cost-efficient training of the NLU unit. Moreover, our proposed\nselection technique does not need the model to be retrained in between the\nselection steps, making it time efficient as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00768v1",
            "date": "2018-02-02 16:57:18+00:00",
            "title": "Order matters: Distributional properties of speech to young children bootstraps learning of semantic representations",
            "authors": [
                "Philip A Huebner",
                "Jon A Willits"
            ],
            "abstract": "Some researchers claim that language acquisition is critically dependent on\nexperiencing linguistic input in order of increasing complexity. We set out to\ntest this hypothesis using a simple recurrent neural network (SRN) trained to\npredict word sequences in CHILDES, a 5-million-word corpus of speech directed\nto children. First, we demonstrated that age-ordered CHILDES exhibits a gradual\nincrease in linguistic complexity. Next, we compared the performance of two\ngroups of SRNs trained on CHILDES which had either been age-ordered or not.\nSpecifically, we assessed learning of grammatical and semantic structure and\nshowed that training on age-ordered input facilitates learning of semantic, but\nnot of sequential structure. We found that this advantage is eliminated when\nthe models were trained on input with utterance boundary information removed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00889v1",
            "date": "2018-02-03 01:40:20+00:00",
            "title": "Densely Connected Bidirectional LSTM with Applications to Sentence Classification",
            "authors": [
                "Zixiang Ding",
                "Rui Xia",
                "Jianfei Yu",
                "Xiang Li",
                "Jian Yang"
            ],
            "abstract": "Deep neural networks have recently been shown to achieve highly competitive\nperformance in many computer vision tasks due to their abilities of exploring\nin a much larger hypothesis space. However, since most deep architectures like\nstacked RNNs tend to suffer from the vanishing-gradient and overfitting\nproblems, their effects are still understudied in many NLP tasks. Inspired by\nthis, we propose a novel multi-layer RNN model called densely connected\nbidirectional long short-term memory (DC-Bi-LSTM) in this paper, which\nessentially represents each layer by the concatenation of its hidden state and\nall preceding layers' hidden states, followed by recursively passing each\nlayer's representation to all subsequent layers. We evaluate our proposed model\non five benchmark datasets of sentence classification. DC-Bi-LSTM with depth up\nto 20 can be successfully trained and obtain significant improvements over the\ntraditional Bi-LSTM with the same or even less parameters. Moreover, our model\nhas promising performance compared with the state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00892v1",
            "date": "2018-02-03 01:43:37+00:00",
            "title": "Left-Center-Right Separated Neural Network for Aspect-based Sentiment Analysis with Rotatory Attention",
            "authors": [
                "Shiliang Zheng",
                "Rui Xia"
            ],
            "abstract": "Deep learning techniques have achieved success in aspect-based sentiment\nanalysis in recent years. However, there are two important issues that still\nremain to be further studied, i.e., 1) how to efficiently represent the target\nespecially when the target contains multiple words; 2) how to utilize the\ninteraction between target and left/right contexts to capture the most\nimportant words in them. In this paper, we propose an approach, called\nleft-center-right separated neural network with rotatory attention (LCR-Rot),\nto better address the two problems. Our approach has two characteristics: 1) it\nhas three separated LSTMs, i.e., left, center and right LSTMs, corresponding to\nthree parts of a review (left context, target phrase and right context); 2) it\nhas a rotatory attention mechanism which models the relation between target and\nleft/right contexts. The target2context attention is used to capture the most\nindicative sentiment words in left/right contexts. Subsequently, the\ncontext2target attention is used to capture the most important word in the\ntarget. This leads to a two-side representation of the target: left-aware\ntarget and right-aware target. We compare our approach on three benchmark\ndatasets with ten related methods proposed recently. The results show that our\napproach significantly outperforms the state-of-the-art techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01021v1",
            "date": "2018-02-03 20:13:42+00:00",
            "title": "DeepType: Multilingual Entity Linking by Neural Type System Evolution",
            "authors": [
                "Jonathan Raiman",
                "Olivier Raiman"
            ],
            "abstract": "The wealth of structured (e.g. Wikidata) and unstructured data about the\nworld available today presents an incredible opportunity for tomorrow's\nArtificial Intelligence. So far, integration of these two different modalities\nis a difficult process, involving many decisions concerning how best to\nrepresent the information so that it will be captured or useful, and\nhand-labeling large amounts of data. DeepType overcomes this challenge by\nexplicitly integrating symbolic information into the reasoning process of a\nneural network with a type system. First we construct a type system, and\nsecond, we use it to constrain the outputs of a neural network to respect the\nsymbolic structure. We achieve this by reformulating the design problem into a\nmixed integer problem: create a type system and subsequently train a neural\nnetwork with it. In this reformulation discrete variables select which\nparent-child relations from an ontology are types within the type system, while\ncontinuous variables control a classifier fit to the type system. The original\nproblem cannot be solved exactly, so we propose a 2-step algorithm: 1)\nheuristic search or stochastic optimization over discrete variables that define\na type system informed by an Oracle and a Learnability heuristic, 2) gradient\ndescent to fit classifier parameters. We apply DeepType to the problem of\nEntity Linking on three standard datasets (i.e. WikiDisamb30, CoNLL (YAGO), TAC\nKBP 2010) and find that it outperforms all existing solutions by a wide margin,\nincluding approaches that rely on a human-designed type system or recent deep\nlearning-based entity embeddings, while explicitly using symbolic information\nlets it integrate new entities without retraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01191v1",
            "date": "2018-02-04 20:45:20+00:00",
            "title": "Heuristic Feature Selection for Clickbait Detection",
            "authors": [
                "Matti Wiegmann",
                "Michael V\u00f6lske",
                "Benno Stein",
                "Matthias Hagen",
                "Martin Potthast"
            ],
            "abstract": "We study feature selection as a means to optimize the baseline clickbait\ndetector employed at the Clickbait Challenge 2017. The challenge's task is to\nscore the \"clickbaitiness\" of a given Twitter tweet on a scale from 0 (no\nclickbait) to 1 (strong clickbait). Unlike most other approaches submitted to\nthe challenge, the baseline approach is based on manual feature engineering and\ndoes not compete out of the box with many of the deep learning-based\napproaches. We show that scaling up feature selection efforts to heuristically\nidentify better-performing feature subsets catapults the performance of the\nbaseline classifier to second rank overall, beating 12 other competing\napproaches and improving over the baseline performance by 20%. This\ndemonstrates that traditional classification approaches can still keep up with\ndeep learning on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01241v2",
            "date": "2018-02-05 02:42:40+00:00",
            "title": "Semantic projection: recovering human knowledge of multiple, distinct object features from word embeddings",
            "authors": [
                "Gabriel Grand",
                "Idan Asher Blank",
                "Francisco Pereira",
                "Evelina Fedorenko"
            ],
            "abstract": "The words of a language reflect the structure of the human mind, allowing us\nto transmit thoughts between individuals. However, language can represent only\na subset of our rich and detailed cognitive architecture. Here, we ask what\nkinds of common knowledge (semantic memory) are captured by word meanings\n(lexical semantics). We examine a prominent computational model that represents\nwords as vectors in a multidimensional space, such that proximity between\nword-vectors approximates semantic relatedness. Because related words appear in\nsimilar contexts, such spaces - called \"word embeddings\" - can be learned from\npatterns of lexical co-occurrences in natural language. Despite their\npopularity, a fundamental concern about word embeddings is that they appear to\nbe semantically \"rigid\": inter-word proximity captures only overall similarity,\nyet human judgments about object similarities are highly context-dependent and\ninvolve multiple, distinct semantic features. For example, dolphins and\nalligators appear similar in size, but differ in intelligence and\naggressiveness. Could such context-dependent relationships be recovered from\nword embeddings? To address this issue, we introduce a powerful, domain-general\nsolution: \"semantic projection\" of word-vectors onto lines that represent\nvarious object features, like size (the line extending from the word \"small\" to\n\"big\"), intelligence (from \"dumb\" to \"smart\"), or danger (from \"safe\" to\n\"dangerous\"). This method, which is intuitively analogous to placing objects\n\"on a mental scale\" between two extremes, recovers human judgments across a\nrange of object categories and properties. We thus show that word embeddings\ninherit a wealth of common knowledge from word co-occurrence statistics and can\nbe flexibly manipulated to express context-dependent meanings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01255v1",
            "date": "2018-02-05 03:42:36+00:00",
            "title": "Chemical-protein relation extraction with ensembles of SVM, CNN, and RNN models",
            "authors": [
                "Yifan Peng",
                "Anthony Rios",
                "Ramakanth Kavuluru",
                "Zhiyong Lu"
            ],
            "abstract": "Text mining the relations between chemicals and proteins is an increasingly\nimportant task. The CHEMPROT track at BioCreative VI aims to promote the\ndevelopment and evaluation of systems that can automatically detect the\nchemical-protein relations in running text (PubMed abstracts). This manuscript\ndescribes our submission, which is an ensemble of three systems, including a\nSupport Vector Machine, a Convolutional Neural Network, and a Recurrent Neural\nNetwork. Their output is combined using a decision based on majority voting or\nstacking. Our CHEMPROT system obtained 0.7266 in precision and 0.5735 in recall\nfor an f-score of 0.6410, demonstrating the effectiveness of machine\nlearning-based approaches for automatic relation extraction from biomedical\nliterature. Our submission achieved the highest performance in the task during\nthe 2017 challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01345v3",
            "date": "2018-02-05 10:54:29+00:00",
            "title": "DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text",
            "authors": [
                "Jingjing Xu",
                "Xuancheng Ren",
                "Junyang Lin",
                "Xu Sun"
            ],
            "abstract": "Existing text generation methods tend to produce repeated and \"boring\"\nexpressions. To tackle this problem, we propose a new text generation model,\ncalled Diversity-Promoting Generative Adversarial Network (DP-GAN). The\nproposed model assigns low reward for repeatedly generated text and high reward\nfor \"novel\" and fluent text, encouraging the generator to produce diverse and\ninformative text. Moreover, we propose a novel language-model based\ndiscriminator, which can better distinguish novel text from repeated text\nwithout the saturation problem compared with existing classifier-based\ndiscriminators. The experimental results on review generation and dialogue\ngeneration tasks demonstrate that our model can generate substantially more\ndiverse and informative text than existing baselines. The code is available at\nhttps://github.com/lancopku/DPGAN",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01457v1",
            "date": "2018-02-05 15:17:01+00:00",
            "title": "Diverse Beam Search for Increased Novelty in Abstractive Summarization",
            "authors": [
                "Andr\u00e9 Cibils",
                "Claudiu Musat",
                "Andreea Hossman",
                "Michael Baeriswyl"
            ],
            "abstract": "Text summarization condenses a text to a shorter version while retaining the\nimportant informations. Abstractive summarization is a recent development that\ngenerates new phrases, rather than simply copying or rephrasing sentences\nwithin the original text. Recently neural sequence-to-sequence models have\nachieved good results in the field of abstractive summarization, which opens\nnew possibilities and applications for industrial purposes. However, most\npractitioners observe that these models still use large parts of the original\ntext in the output summaries, making them often similar to extractive\nframeworks. To address this drawback, we first introduce a new metric to\nmeasure how much of a summary is extracted from the input text. Secondly, we\npresent a novel method, that relies on a diversity factor in computing the\nneural network loss, to improve the diversity of the summaries generated by any\nneural abstractive model implementing beam search. Finally, we show that this\nmethod not only makes the system less extractive, but also improves the overall\nrouge score of state-of-the-art methods by at least 2 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01766v1",
            "date": "2018-02-06 02:39:54+00:00",
            "title": "Question-Answer Selection in User to User Marketplace Conversations",
            "authors": [
                "Girish Kumar",
                "Matthew Henderson",
                "Shannon Chan",
                "Hoang Nguyen",
                "Lucas Ngoo"
            ],
            "abstract": "Sellers in user to user marketplaces can be inundated with questions from\npotential buyers. Answers are often already available in the product\ndescription. We collected a dataset of around 590K such questions and answers\nfrom conversations in an online marketplace. We propose a question answering\nsystem that selects a sentence from the product description using a\nneural-network ranking model. We explore multiple encoding strategies, with\nrecurrent neural networks and feed-forward attention layers yielding good\nresults. This paper presents a demo to interactively pose buyer questions and\nvisualize the ranking scores of product description sentences from live online\nlistings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01817v1",
            "date": "2018-02-06 06:47:09+00:00",
            "title": "Byte-Level Recursive Convolutional Auto-Encoder for Text",
            "authors": [
                "Xiang Zhang",
                "Yann LeCun"
            ],
            "abstract": "This article proposes to auto-encode text at byte-level using convolutional\nnetworks with a recursive architecture. The motivation is to explore whether it\nis possible to have scalable and homogeneous text generation at byte-level in a\nnon-sequential fashion through the simple task of auto-encoding. We show that\nnon-sequential text generation from a fixed-length representation is not only\npossible, but also achieved much better auto-encoding results than recurrent\nnetworks. The proposed model is a multi-stage deep convolutional\nencoder-decoder framework using residual connections, containing up to 160\nparameterized layers. Each encoder or decoder contains a shared group of\nmodules that consists of either pooling or upsampling layers, making the\nnetwork recursive in terms of abstraction levels in representation. Results for\n6 large-scale paragraph datasets are reported, in 3 languages including Arabic,\nChinese and English. Analyses are conducted to study several properties of the\nproposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02114v1",
            "date": "2018-02-06 18:20:17+00:00",
            "title": "Investigations on Knowledge Base Embedding for Relation Prediction and Extraction",
            "authors": [
                "Peng Xu",
                "Denilson Barbosa"
            ],
            "abstract": "We report an evaluation of the effectiveness of the existing knowledge base\nembedding models for relation prediction and for relation extraction on a wide\nrange of benchmarks. We also describe a new benchmark, which is much larger and\ncomplex than previous ones, which we introduce to help validate the\neffectiveness of both tasks. The results demonstrate that knowledge base\nembedding models are generally effective for relation prediction but unable to\ngive improvements for the state-of-art neural relation extraction model with\nthe existing strategies, while pointing limitations of existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02116v1",
            "date": "2018-02-06 18:28:45+00:00",
            "title": "Non-Projective Dependency Parsing via Latent Heads Representation (LHR)",
            "authors": [
                "Matteo Grella",
                "Simone Cangialosi"
            ],
            "abstract": "In this paper, we introduce a novel approach based on a bidirectional\nrecurrent autoencoder to perform globally optimized non-projective dependency\nparsing via semi-supervised learning. The syntactic analysis is completed at\nthe end of the neural process that generates a Latent Heads Representation\n(LHR), without any algorithmic constraint and with a linear complexity. The\nresulting \"latent syntactic structure\" can be used directly in other semantic\ntasks. The LHR is transformed into the usual dependency tree computing a simple\nvectors similarity. We believe that our model has the potential to compete with\nmuch more complex state-of-the-art parsing architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02311v2",
            "date": "2018-02-07 05:23:21+00:00",
            "title": "An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notes",
            "authors": [
                "Jinmiao Huang",
                "Cesar Osorio",
                "Luke Wicent Sy"
            ],
            "abstract": "Background and Objective: Code assignment is of paramount importance in many\nlevels in modern hospitals, from ensuring accurate billing process to creating\na valid record of patient care history. However, the coding process is tedious\nand subjective, and it requires medical coders with extensive training. This\nstudy aims to evaluate the performance of deep-learning-based systems to\nautomatically map clinical notes to ICD-9 medical codes. Methods: The\nevaluations of this research are focused on end-to-end learning methods without\nmanually defined rules. Traditional machine learning algorithms, as well as\nstate-of-the-art deep learning methods such as Recurrent Neural Networks and\nConvolution Neural Networks, were applied to the Medical Information Mart for\nIntensive Care (MIMIC-III) dataset. An extensive number of experiments was\napplied to different settings of the tested algorithm. Results: Findings showed\nthat the deep learning-based methods outperformed other conventional machine\nlearning methods. From our assessment, the best models could predict the top 10\nICD-9 codes with 0.6957 F1 and 0.8967 accuracy and could estimate the top 10\nICD-9 categories with 0.7233 F1 and 0.8588 accuracy. Our implementation also\noutperformed existing work under certain evaluation metrics. Conclusion: A set\nof standard metrics was utilized in assessing the performance of ICD-9 code\nassignment on MIMIC-III dataset. All the developed evaluation tools and\nresources are available online, which can be used as a baseline for further\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02605v2",
            "date": "2018-02-07 19:27:27+00:00",
            "title": "Unsupervised word sense disambiguation in dynamic semantic spaces",
            "authors": [
                "Jean-Fran\u00e7ois Delpech"
            ],
            "abstract": "In this paper, we are mainly concerned with the ability to quickly and\nautomatically distinguish word senses in dynamic semantic spaces in which new\nterms and new senses appear frequently. Such spaces are built '\"on the fly\"\nfrom constantly evolving data sets such as Wikipedia, repositories of patent\ngrants and applications, or large sets of legal documents for Technology\nAssisted Review and e-discovery. This immediacy rules out supervision as well\nas the use of a priori training sets. We show that the various senses of a term\ncan be automatically made apparent with a simple clustering algorithm, each\nsense being a vector in the semantic space. While we only consider here\nsemantic spaces built by using random vectors, this algorithm should work with\nany kind of embedding, provided meaningful similarities between terms can be\ncomputed and do fulfill at least the two basic conditions that terms which\nclose meanings have high similarities and terms with unrelated meanings have\nnear-zero similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02614v2",
            "date": "2018-02-07 19:41:02+00:00",
            "title": "Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus",
            "authors": [
                "Jianxiong Dong",
                "Jim Huang"
            ],
            "abstract": "Ubuntu dialogue corpus is the largest public available dialogue corpus to\nmake it feasible to build end-to-end deep neural network models directly from\nthe conversation data. One challenge of Ubuntu dialogue corpus is the large\nnumber of out-of-vocabulary words. In this paper we proposed a method which\ncombines the general pre-trained word embedding vectors with those generated on\nthe task-specific training set to address this issue. We integrated character\nembedding into Chen et al's Enhanced LSTM method (ESIM) and used it to evaluate\nthe effectiveness of our proposed method. For the task of next utterance\nselection, the proposed method has demonstrated a significant performance\nimprovement against original ESIM and the new model has achieved\nstate-of-the-art results on both Ubuntu dialogue corpus and Douban conversation\ncorpus. In addition, we investigated the performance impact of end-of-utterance\nand end-of-turn token tags.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02870v2",
            "date": "2018-02-08 14:16:18+00:00",
            "title": "Biomedical term normalization of EHRs with UMLS",
            "authors": [
                "Naiara Perez",
                "Montse Cuadros",
                "German Rigau"
            ],
            "abstract": "This paper presents a novel prototype for biomedical term normalization of\nelectronic health record excerpts with the Unified Medical Language System\n(UMLS) Metathesaurus. Despite being multilingual and cross-lingual by design,\nwe first focus on processing clinical text in Spanish because there is no\nexisting tool for this language and for this specific purpose. The tool is\nbased on Apache Lucene to index the Metathesaurus and generate mapping\ncandidates from input text. It uses the IXA pipeline for basic language\nprocessing and resolves ambiguities with the UKB toolkit. It has been evaluated\nby measuring its agreement with MetaMap in two English-Spanish parallel\ncorpora. In addition, we present a web-based interface for the tool.",
            "journal ref": "Perez, N., Cuadros, M., & Rigau, G. (2018). Biomedical term\n  normalization of EHRs with UMLS. In Proceedings of the Eleventh International\n  Conference on Language Resources and Evaluation (LREC 2018). ELRA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02926v1",
            "date": "2018-02-08 15:38:54+00:00",
            "title": "DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech",
            "authors": [
                "George Christodoulides",
                "Mathieu Avanzi",
                "Jean-Philippe Goldman"
            ],
            "abstract": "We present DisMo, a multi-level annotator for spoken language corpora that\nintegrates part-of-speech tagging with basic disfluency detection and\nannotation, and multi-word unit recognition. DisMo is a hybrid system that uses\na combination of lexical resources, rules, and statistical models based on\nConditional Random Fields (CRF). In this paper, we present the first public\nversion of DisMo for French. The system is trained and its performance\nevaluated on a 57k-token corpus, including different varieties of French spoken\nin three countries (Belgium, France and Switzerland). DisMo supports a\nmulti-level annotation scheme, in which the tokenisation to minimal word units\nis complemented with multi-word unit groupings (each having associated POS\ntags), as well as separate levels for annotating disfluencies and discourse\nphenomena. We present the system's architecture, linguistic resources and its\nhierarchical tag-set. Results show that DisMo achieves a precision of 95%\n(finest tag-set) to 96.8% (coarse tag-set) in POS-tagging non-punctuated,\nsound-aligned transcriptions of spoken French, while also offering substantial\npossibilities for automated multi-level annotation.",
            "journal ref": "Proceedings of the 9th International Conference on Language\n  Resources and Evaluation (LREC), May 2014, Reykjavik, Iceland",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03116v1",
            "date": "2018-02-09 03:53:13+00:00",
            "title": "Zero-Resource Neural Machine Translation with Multi-Agent Communication Game",
            "authors": [
                "Yun Chen",
                "Yang Liu",
                "Victor O. K. Li"
            ],
            "abstract": "While end-to-end neural machine translation (NMT) has achieved notable\nsuccess in the past years in translating a handful of resource-rich language\npairs, it still suffers from the data scarcity problem for low-resource\nlanguage pairs and domains. To tackle this problem, we propose an interactive\nmultimodal framework for zero-resource neural machine translation. Instead of\nbeing passively exposed to large amounts of parallel corpora, our learners\n(implemented as encoder-decoder architecture) engage in cooperative image\ndescription games, and thus develop their own image captioning or neural\nmachine translation model from the need to communicate in order to succeed at\nthe game. Experimental results on the IAPR-TC12 and Multi30K datasets show that\nthe proposed learning mechanism significantly improves over the\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03142v1",
            "date": "2018-02-09 06:29:43+00:00",
            "title": "Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation",
            "authors": [
                "Ali Can Kocabiyikoglu",
                "Laurent Besacier",
                "Olivier Kraif"
            ],
            "abstract": "Recent works in spoken language translation (SLT) have attempted to build\nend-to-end speech-to-text translation without using source language\ntranscription during learning or decoding. However, while large quantities of\nparallel texts (such as Europarl, OpenSubtitles) are available for training\nmachine translation systems, there are no large (100h) and open source parallel\ncorpora that include speech in a source language aligned to text in a target\nlanguage. This paper tries to fill this gap by augmenting an existing\n(monolingual) corpus: LibriSpeech. This corpus, used for automatic speech\nrecognition, is derived from read audiobooks from the LibriVox project, and has\nbeen carefully segmented and aligned. After gathering French e-books\ncorresponding to the English audio-books from LibriSpeech, we align speech\nsegments at the sentence level with their respective translations and obtain\n236h of usable parallel data. This paper presents the details of the processing\nas well as a manual evaluation conducted on a small subset of the corpus. This\nevaluation shows that the automatic alignments scores are reasonably correlated\nwith the human judgments of the bilingual alignment quality. We believe that\nthis corpus (which is made available online) is useful for replicable\nexperiments in direct speech translation or more general spoken language\ntranslation experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03198v1",
            "date": "2018-02-09 10:49:47+00:00",
            "title": "Natural Language Inference over Interaction Space: ICLR 2018 Reproducibility Report",
            "authors": [
                "Martin Mirakyan",
                "Karen Hambardzumyan",
                "Hrant Khachatrian"
            ],
            "abstract": "We have tried to reproduce the results of the paper \"Natural Language\nInference over Interaction Space\" submitted to ICLR 2018 conference as part of\nthe ICLR 2018 Reproducibility Challenge. Initially, we were not aware that the\ncode was available, so we started to implement the network from scratch. We\nhave evaluated our version of the model on Stanford NLI dataset and reached\n86.38% accuracy on the test set, while the paper claims 88.0% accuracy. The\nmain difference, as we understand it, comes from the optimizers and the way\nmodel selection is performed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03238v2",
            "date": "2018-02-09 12:58:29+00:00",
            "title": "Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning",
            "authors": [
                "Myeongjun Jang",
                "Seungwan Seo",
                "Pilsung Kang"
            ],
            "abstract": "Sequence-to-sequence (Seq2seq) models have played an important role in the\nrecent success of various natural language processing methods, such as machine\ntranslation, text summarization, and speech recognition. However, current\nSeq2seq models have trouble preserving global latent information from a long\nsequence of words. Variational autoencoder (VAE) alleviates this problem by\nlearning a continuous semantic space of the input sentence. However, it does\nnot solve the problem completely. In this paper, we propose a new recurrent\nneural network (RNN)-based Seq2seq model, RNN semantic variational autoencoder\n(RNN--SVAE), to better capture the global latent information of a sequence of\nwords. To reflect the meaning of words in a sentence properly, without regard\nto its position within the sentence, we construct a document information vector\nusing the attention information between the final state of the encoder and\nevery prior hidden state. Then, the mean and standard deviation of the\ncontinuous semantic space are learned by using this vector to take advantage of\nthe variational method. By using the document information vector to find the\nsemantic space of the sentence, it becomes possible to better capture the\nglobal latent feature of the sentence. Experimental results of three natural\nlanguage tasks (i.e., language modeling, missing word imputation, paraphrase\nidentification) confirm that the proposed RNN--SVAE yields higher performance\nthan two benchmark models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03594v2",
            "date": "2018-02-10 14:07:58+00:00",
            "title": "Online Learning for Effort Reduction in Interactive Neural Machine Translation",
            "authors": [
                "\u00c1lvaro Peris",
                "Francisco Casacuberta"
            ],
            "abstract": "Neural machine translation systems require large amounts of training data and\nresources. Even with this, the quality of the translations may be insufficient\nfor some users or domains. In such cases, the output of the system must be\nrevised by a human agent. This can be done in a post-editing stage or following\nan interactive machine translation protocol.\n  We explore the incremental update of neural machine translation systems\nduring the post-editing or interactive translation processes. Such\nmodifications aim to incorporate the new knowledge, from the edited sentences,\ninto the translation system. Updates to the model are performed on-the-fly, as\nsentences are corrected, via online learning techniques. In addition, we\nimplement a novel interactive, adaptive system, able to react to\nsingle-character interactions. This system greatly reduces the human effort\nrequired for obtaining high-quality translations.\n  In order to stress our proposals, we conduct exhaustive experiments varying\nthe amount and type of data available for training. Results show that online\nlearning effectively achieves the objective of reducing the human effort\nrequired during the post-editing or the interactive machine translation stages.\nMoreover, these adaptive systems also perform well in scenarios with scarce\nresources. We show that a neural machine translation system can be rapidly\nadapted to a specific domain, exclusively by means of online learning\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03656v2",
            "date": "2018-02-10 22:34:38+00:00",
            "title": "TextZoo, a New Benchmark for Reconsidering Text Classification",
            "authors": [
                "Benyou Wang",
                "Li Wang",
                "Qikang Wei",
                "Lichun Liu"
            ],
            "abstract": "Text representation is a fundamental concern in Natural Language Processing,\nespecially in text classification. Recently, many neural network approaches\nwith delicate representation model (e.g. FASTTEXT, CNN, RNN and many hybrid\nmodels with attention mechanisms) claimed that they achieved state-of-art in\nspecific text classification datasets. However, it lacks an unified benchmark\nto compare these models and reveals the advantage of each sub-components for\nvarious settings. We re-implement more than 20 popular text representation\nmodels for classification in more than 10 datasets. In this paper, we\nreconsider the text classification task in the perspective of neural network\nand get serval effects with analysis of the above results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03712v2",
            "date": "2018-02-11 08:54:58+00:00",
            "title": "Syntax and Semantics of Italian Poetry in the First Half of the 20th Century",
            "authors": [
                "Rodolfo Delmonte"
            ],
            "abstract": "In this paper we study, analyse and comment rhetorical figures present in\nsome of most interesting poetry of the first half of the twentieth century.\nThese figures are at first traced back to some famous poet of the past and then\ncompared to classical Latin prose. Linguistic theory is then called in to show\nhow they can be represented in syntactic structures and classified as\nnoncanonical structures, by positioning discontinuous or displaced linguistic\nelements in Spec XP projections at various levels of constituency. Then we\nintroduce LFG (Lexical Functional Grammar) as the theory that allows us to\nconnect syntactic noncanonical structures with informational structure and\npsycholinguistic theories for complexity evaluation. We end up with two\ncomputational linguistics experiments and then evaluate the results. The first\none uses best online parsers of Italian to parse poetic structures; the second\none uses Getarun, the system created at Ca Foscari Computational Linguistics\nLaboratory. As will be shown, the first approach is unable to cope with these\nstructures due to the use of only statistical probabilistic information. On the\ncontrary, the second one, being a symbolic rule based system, is by far\nsuperior and allows also to complete both semantic an pragmatic analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03816v1",
            "date": "2018-02-11 20:59:56+00:00",
            "title": "Understanding Recurrent Neural State Using Memory Signatures",
            "authors": [
                "Skanda Koppula",
                "Khe Chai Sim",
                "Kean Chin"
            ],
            "abstract": "We demonstrate a network visualization technique to analyze the recurrent\nstate inside the LSTMs/GRUs used commonly in language and acoustic models.\nInterpreting intermediate state and network activations inside end-to-end\nmodels remains an open challenge. Our method allows users to understand exactly\nhow much and what history is encoded inside recurrent state in grapheme\nsequence models. Our procedure trains multiple decoders that predict prior\ninput history. Compiling results from these decoders, a user can obtain a\nsignature of the recurrent kernel that characterizes its memory behavior. We\ndemonstrate this method's usefulness in revealing information divergence in the\nbases of recurrent factorized kernels, visualizing the character-level\ndifferences between the memory of n-gram and recurrent language models, and\nextracting knowledge of history encoded in the layers of grapheme-based\nend-to-end ASR networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04028v1",
            "date": "2018-02-12 13:20:57+00:00",
            "title": "Automatic Generation of Language-Independent Features for Cross-Lingual Classification",
            "authors": [
                "Sarai Duek",
                "Shaul Markovitch"
            ],
            "abstract": "Many applications require categorization of text documents using predefined\ncategories. The main approach to performing text categorization is learning\nfrom labeled examples. For many tasks, it may be difficult to find examples in\none language but easy in others. The problem of learning from examples in one\nor more languages and classifying (categorizing) in another is called\ncross-lingual learning. In this work, we present a novel approach that solves\nthe general cross-lingual text categorization problem. Our method generates,\nfor each training document, a set of language-independent features. Using these\nfeatures for training yields a language-independent classifier. At the\nclassification stage, we generate language-independent features for the\nunlabeled document, and apply the classifier on the new representation.\n  To build the feature generator, we utilize a hierarchical\nlanguage-independent ontology, where each concept has a set of support\ndocuments for each language involved. In the preprocessing stage, we use the\nsupport documents to build a set of language-independent feature generators,\none for each language. The collection of these generators is used to map any\ndocument into the language-independent feature space.\n  Our methodology works on the most general cross-lingual text categorization\nproblems, being able to learn from any mix of languages and classify documents\nin any other language. We also present a method for exploiting the hierarchical\nstructure of the ontology to create virtual supporting documents for languages\nthat do not have them. We tested our method, using Wikipedia as our ontology,\non the most commonly used test collections in cross-lingual text\ncategorization, and found that it outperforms existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04140v2",
            "date": "2018-02-09 15:32:01+00:00",
            "title": "Making \"fetch\" happen: The influence of social and linguistic context on nonstandard word growth and decline",
            "authors": [
                "Ian Stewart",
                "Jacob Eisenstein"
            ],
            "abstract": "In an online community, new words come and go: today's \"haha\" may be replaced\nby tomorrow's \"lol.\" Changes in online writing are usually studied as a social\nprocess, with innovations diffusing through a network of individuals in a\nspeech community. But unlike other types of innovation, language change is\nshaped and constrained by the system in which it takes part. To investigate the\nlinks between social and structural factors in language change, we undertake a\nlarge-scale analysis of nonstandard word growth in the online community Reddit.\nWe find that dissemination across many linguistic contexts is a sign of growth:\nwords that appear in more linguistic contexts grow faster and survive longer.\nWe also find that social dissemination likely plays a less important role in\nexplaining word growth and decline than previously hypothesized.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04200v1",
            "date": "2018-02-12 17:37:11+00:00",
            "title": "End-to-End Automatic Speech Translation of Audiobooks",
            "authors": [
                "Alexandre B\u00e9rard",
                "Laurent Besacier",
                "Ali Can Kocabiyikoglu",
                "Olivier Pietquin"
            ],
            "abstract": "We investigate end-to-end speech-to-text translation on a corpus of\naudiobooks specifically augmented for this task. Previous works investigated\nthe extreme case where source language transcription is not available during\nlearning nor decoding, but we also study a midway case where source language\ntranscription is available at training time only. In this case, a single model\nis trained to decode source speech into target text in a single pass.\nExperimental results show that it is possible to train compact and efficient\nend-to-end speech translation models in this setup. We also distribute the\ncorpus and hope that our speech translation baseline on this corpus will be\nchallenged in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04358v1",
            "date": "2018-02-12 20:53:50+00:00",
            "title": "A Unified Implicit Dialog Framework for Conversational Search",
            "authors": [
                "Song Feng",
                "R. Chulaka Gunasekara",
                "Sunil Shashidhara",
                "Kshitij P. Fadnis",
                "Lazaros C. Polymenakos"
            ],
            "abstract": "We propose a unified Implicit Dialog framework for goal-oriented, information\nseeking tasks of Conversational Search applications. It aims to enable dialog\ninteractions with domain data without replying on explicitly encoded the rules\nbut utilizing the underlying data representation to build the components\nrequired for dialog interaction, which we refer as Implicit Dialog in this\nwork. The proposed framework consists of a pipeline of End-to-End trainable\nmodules. A centralized knowledge representation is used to semantically ground\nmultiple dialog modules. An associated set of tools are integrated with the\nframework to gather end users' input for continuous improvement of the system.\nThe goal is to facilitate development of conversational systems by identifying\nthe components and the data that can be adapted and reused across many end-user\napplications. We demonstrate our approach by creating conversational agents for\nseveral independent domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04559v1",
            "date": "2018-02-13 11:04:07+00:00",
            "title": "Sentence Boundary Detection for French with Subword-Level Information Vectors and Convolutional Neural Networks",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "In this work we tackle the problem of sentence boundary detection applied to\nFrench as a binary classification task (\"sentence boundary\" or \"not sentence\nboundary\"). We combine convolutional neural networks with subword-level\ninformation vectors, which are word embedding representations learned from\nWikipedia that take advantage of the words morphology; so each word is\nrepresented as a bag of their character n-grams.\n  We decide to use a big written dataset (French Gigaword) instead of standard\nsize transcriptions to train and evaluate the proposed architectures with the\nintention of using the trained models in posterior real life ASR\ntranscriptions.\n  Three different architectures are tested showing similar results; general\naccuracy for all models overpasses 0.96. All three models have good F1 scores\nreaching values over 0.97 regarding the \"not sentence boundary\" class. However,\nthe \"sentence boundary\" class reflects lower scores decreasing the F1 metric to\n0.778 for one of the models.\n  Using subword-level information vectors seem to be very effective leading to\nconclude that the morphology of words encoded in the embeddings representations\nbehave like pixels in an image making feasible the use of convolutional neural\nnetwork architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04609v1",
            "date": "2018-02-13 13:22:19+00:00",
            "title": "Network Features Based Co-hyponymy Detection",
            "authors": [
                "Abhik Jana",
                "Pawan Goyal"
            ],
            "abstract": "Distinguishing lexical relations has been a long term pursuit in natural\nlanguage processing (NLP) domain. Recently, in order to detect lexical\nrelations like hypernymy, meronymy, co-hyponymy etc., distributional semantic\nmodels are being used extensively in some form or the other. Even though a lot\nof efforts have been made for detecting hypernymy relation, the problem of\nco-hyponymy detection has been rarely investigated. In this paper, we are\nproposing a novel supervised model where various network measures have been\nutilized to identify co-hyponymy relation with high accuracy performing better\nor at par with the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04681v1",
            "date": "2018-02-13 15:25:21+00:00",
            "title": "Examining the Tip of the Iceberg: A Data Set for Idiom Translation",
            "authors": [
                "Marzieh Fadaee",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "Neural Machine Translation (NMT) has been widely used in recent years with\nsignificant improvements for many language pairs. Although state-of-the-art NMT\nsystems are generating progressively better translations, idiom translation\nremains one of the open challenges in this field. Idioms, a category of\nmultiword expressions, are an interesting language phenomenon where the overall\nmeaning of the expression cannot be composed from the meanings of its parts. A\nfirst important challenge is the lack of dedicated data sets for learning and\nevaluating idiom translation. In this paper we address this problem by creating\nthe first large-scale data set for idiom translation. Our data set is\nautomatically extracted from a widely used German-English translation corpus\nand includes, for each language direction, a targeted evaluation set where all\nsentences contain idioms and a regular training corpus where sentences\nincluding idioms are marked. We release this data set and use it to perform\npreliminary NMT experiments as the first step towards better idiom translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04744v4",
            "date": "2018-02-13 17:10:54+00:00",
            "title": "A Short Survey on Sense-Annotated Corpora",
            "authors": [
                "Tommaso Pasini",
                "Jose Camacho-Collados"
            ],
            "abstract": "Large sense-annotated datasets are increasingly necessary for training deep\nsupervised systems in Word Sense Disambiguation. However, gathering\nhigh-quality sense-annotated data for as many instances as possible is a\nlaborious and expensive task. This has led to the proliferation of automatic\nand semi-automatic methods for overcoming the so-called knowledge-acquisition\nbottleneck. In this short survey we present an overview of sense-annotated\ncorpora, annotated either manually- or (semi)automatically, that are currently\navailable for different languages and featuring distinct lexical resources as\ninventory of senses, i.e. WordNet, Wikipedia, BabelNet. Furthermore, we provide\nthe reader with general statistics of each dataset and an analysis of their\nspecific features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05014v1",
            "date": "2018-02-14 10:04:48+00:00",
            "title": "Distributional Term Set Expansion",
            "authors": [
                "Amaru Cuba Gyllensten",
                "Magnus Sahlgren"
            ],
            "abstract": "This paper is a short empirical study of the performance of centrality and\nclassification based iterative term set expansion methods for distributional\nsemantic models. Iterative term set expansion is an interactive process using\ndistributional semantics models where a user labels terms as belonging to some\nsought after term set, and a system uses this labeling to supply the user with\nnew, candidate, terms to label, trying to maximize the number of positive\nexamples found. While centrality based methods have a long history in term set\nexpansion, we compare them to classification methods based on the the Simple\nMargin method, an Active Learning approach to classification using Support\nVector Machines. Examining the performance of various centrality and\nclassification based methods for a variety of distributional models over five\ndifferent term sets, we can show that active learning based methods\nconsistently outperform centrality based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05092v1",
            "date": "2018-02-14 13:46:03+00:00",
            "title": "Linguistic unit discovery from multi-modal inputs in unwritten languages: Summary of the \"Speaking Rosetta\" JSALT 2017 Workshop",
            "authors": [
                "Odette Scharenborg",
                "Laurent Besacier",
                "Alan Black",
                "Mark Hasegawa-Johnson",
                "Florian Metze",
                "Graham Neubig",
                "Sebastian Stueker",
                "Pierre Godard",
                "Markus Mueller",
                "Lucas Ondel",
                "Shruti Palaskar",
                "Philip Arthur",
                "Francesco Ciannella",
                "Mingxing Du",
                "Elin Larsen",
                "Danny Merkx",
                "Rachid Riad",
                "Liming Wang",
                "Emmanuel Dupoux"
            ],
            "abstract": "We summarize the accomplishments of a multi-disciplinary workshop exploring\nthe computational and scientific issues surrounding the discovery of linguistic\nunits (subwords and words) in a language without orthography. We study the\nreplacement of orthographic transcriptions by images and/or translated text in\na well-resourced language to help unsupervised discovery from raw speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05322v1",
            "date": "2018-02-14 21:04:15+00:00",
            "title": "Classifying movie genres by analyzing text reviews",
            "authors": [
                "Adam Nyberg"
            ],
            "abstract": "This paper proposes a method for classifying movie genres by only looking at\ntext reviews. The data used are from Large Movie Review Dataset v1.0 and IMDb.\nThis paper compared a K-nearest neighbors (KNN) model and a multilayer\nperceptron (MLP) that uses tf-idf as input features. The paper also discusses\ndifferent evaluation metrics used when doing multi-label classification. For\nthe data used in this research, the KNN model performed the best with an\naccuracy of 55.4\\% and a Hamming loss of 0.047.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05365v2",
            "date": "2018-02-15 00:05:11+00:00",
            "title": "Deep contextualized word representations",
            "authors": [
                "Matthew E. Peters",
                "Mark Neumann",
                "Mohit Iyyer",
                "Matt Gardner",
                "Christopher Clark",
                "Kenton Lee",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce a new type of deep contextualized word representation that\nmodels both (1) complex characteristics of word use (e.g., syntax and\nsemantics), and (2) how these uses vary across linguistic contexts (i.e., to\nmodel polysemy). Our word vectors are learned functions of the internal states\nof a deep bidirectional language model (biLM), which is pre-trained on a large\ntext corpus. We show that these representations can be easily added to existing\nmodels and significantly improve the state of the art across six challenging\nNLP problems, including question answering, textual entailment and sentiment\nanalysis. We also present an analysis showing that exposing the deep internals\nof the pre-trained network is crucial, allowing downstream models to mix\ndifferent types of semi-supervision signals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05368v2",
            "date": "2018-02-15 00:35:08+00:00",
            "title": "Universal Neural Machine Translation for Extremely Low Resource Languages",
            "authors": [
                "Jiatao Gu",
                "Hany Hassan",
                "Jacob Devlin",
                "Victor O. K. Li"
            ],
            "abstract": "In this paper, we propose a new universal machine translation approach\nfocusing on languages with a limited amount of parallel data. Our proposed\napproach utilizes a transfer-learning approach to share lexical and sentence\nlevel representations across multiple source languages into one target\nlanguage. The lexical part is shared through a Universal Lexical Representation\nto support multilingual word-level sharing. The sentence-level sharing is\nrepresented by a model of experts from all source languages that share the\nsource encoders with all other languages. This enables the low-resource\nlanguage to utilize the lexical and sentence representations of the higher\nresource languages. Our approach is able to achieve 23 BLEU on Romanian-English\nWMT2016 using a tiny parallel corpus of 6k sentences, compared to the 18 BLEU\nof strong baseline system which uses multilingual training and\nback-translation. Furthermore, we show that the proposed approach can achieve\nalmost 20 BLEU on the same dataset through fine-tuning a pre-trained\nmulti-lingual system in a zero-shot setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05373v2",
            "date": "2018-02-15 00:49:52+00:00",
            "title": "Improving Retrieval Modeling Using Cross Convolution Networks And Multi Frequency Word Embedding",
            "authors": [
                "Guozhen An",
                "Mehrnoosh Shafiee",
                "Davood Shamsi"
            ],
            "abstract": "To build a satisfying chatbot that has the ability of managing a\ngoal-oriented multi-turn dialogue, accurate modeling of human conversation is\ncrucial. In this paper we concentrate on the task of response selection for\nmulti-turn human-computer conversation with a given context. Previous\napproaches show weakness in capturing information of rare keywords that appear\nin either or both context and correct response, and struggle with long input\nsequences. We propose Cross Convolution Network (CCN) and Multi Frequency word\nembedding to address both problems. We train several models using the Ubuntu\nDialogue dataset which is the largest freely available multi-turn based\ndialogue corpus. We further build an ensemble model by averaging predictions of\nmultiple models. We achieve a new state-of-the-art on this dataset with\nconsiderable improvements compared to previous best results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05574v2",
            "date": "2018-02-15 14:38:46+00:00",
            "title": "Open Information Extraction on Scientific Text: An Evaluation",
            "authors": [
                "Paul Groth",
                "Michael Lauruhn",
                "Antony Scerri",
                "Ron Daniel Jr"
            ],
            "abstract": "Open Information Extraction (OIE) is the task of the unsupervised creation of\nstructured information from text. OIE is often used as a starting point for a\nnumber of downstream tasks including knowledge base construction, relation\nextraction, and question answering. While OIE methods are targeted at being\ndomain independent, they have been evaluated primarily on newspaper,\nencyclopedic or general web text. In this article, we evaluate the performance\nof OIE on scientific texts originating from 10 different disciplines. To do so,\nwe use two state-of-the-art OIE systems applying a crowd-sourcing approach. We\nfind that OIE systems perform significantly worse on scientific text than\nencyclopedic text. We also provide an error analysis and suggest areas of work\nto reduce errors. Our corpus of sentences and judgments are made available.",
            "journal ref": "The 27th International Conference on Computational Linguistics\n  (COLING 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05577v2",
            "date": "2018-02-15 14:39:47+00:00",
            "title": "DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference",
            "authors": [
                "Reza Ghaeini",
                "Sadid A. Hasan",
                "Vivek Datla",
                "Joey Liu",
                "Kathy Lee",
                "Ashequl Qadir",
                "Yuan Ling",
                "Aaditya Prakash",
                "Xiaoli Z. Fern",
                "Oladimeji Farri"
            ],
            "abstract": "We present a novel deep learning architecture to address the natural language\ninference (NLI) task. Existing approaches mostly rely on simple reading\nmechanisms for independent encoding of the premise and hypothesis. Instead, we\npropose a novel dependent reading bidirectional LSTM network (DR-BiLSTM) to\nefficiently model the relationship between a premise and a hypothesis during\nencoding and inference. We also introduce a sophisticated ensemble strategy to\ncombine our proposed models, which noticeably improves final predictions.\nFinally, we demonstrate how the results can be improved further with an\nadditional preprocessing step. Our evaluation shows that DR-BiLSTM obtains the\nbest single model and ensemble model results achieving the new state-of-the-art\nscores on the Stanford NLI dataset.",
            "journal ref": "NAACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05583v1",
            "date": "2018-02-15 14:50:54+00:00",
            "title": "Tools and resources for Romanian text-to-speech and speech-to-text applications",
            "authors": [
                "Tiberiu Boros",
                "Stefan Daniel Dumitrescu",
                "Vasile Pais"
            ],
            "abstract": "In this paper we introduce a set of resources and tools aimed at providing\nsupport for natural language processing, text-to-speech synthesis and speech\nrecognition for Romanian. While the tools are general purpose and can be used\nfor any language (we successfully trained our system for more than 50 languages\nand participated in the Universal Dependencies Shared Task), the resources are\nonly relevant for Romanian language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05667v2",
            "date": "2018-02-15 17:15:25+00:00",
            "title": "Calculating the similarity between words and sentences using a lexical database and corpus statistics",
            "authors": [
                "Atish Pawar",
                "Vijay Mago"
            ],
            "abstract": "Calculating the semantic similarity between sentences is a long dealt problem\nin the area of natural language processing. The semantic analysis field has a\ncrucial role to play in the research related to the text analytics. The\nsemantic similarity differs as the domain of operation differs. In this paper,\nwe present a methodology which deals with this issue by incorporating semantic\nsimilarity and corpus statistics. To calculate the semantic similarity between\nwords and sentences, the proposed method follows an edge-based approach using a\nlexical database. The methodology can be applied in a variety of domains. The\nmethodology has been tested on both benchmark standards and mean human\nsimilarity dataset. When tested on these two datasets, it gives highest\ncorrelation value for both word and sentence similarity outperforming other\nsimilar models. For word similarity, we obtained Pearson correlation\ncoefficient of 0.8753 and for sentence similarity, the correlation obtained is\n0.8794.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05672v1",
            "date": "2018-02-15 17:28:46+00:00",
            "title": "Event Nugget Detection with Forward-Backward Recurrent Neural Networks",
            "authors": [
                "Reza Ghaeini",
                "Xiaoli Z. Fern",
                "Liang Huang",
                "Prasad Tadepalli"
            ],
            "abstract": "Traditional event detection methods heavily rely on manually engineered rich\nfeatures. Recent deep learning approaches alleviate this problem by automatic\nfeature engineering. But such efforts, like tradition methods, have so far only\nfocused on single-token event mentions, whereas in practice events can also be\na phrase. We instead use forward-backward recurrent neural networks (FBRNNs) to\ndetect events that can be either words or phrases. To the best our knowledge,\nthis is one of the first efforts to handle multi-word events and also the first\nattempt to use RNNs for event detection. Experimental results demonstrate that\nFBRNN is competitive with the state-of-the-art methods on the ACE 2005 and the\nRich ERE 2015 event detection tasks.",
            "journal ref": "ACL 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05737v1",
            "date": "2018-02-15 20:02:43+00:00",
            "title": "JU_KS@SAIL_CodeMixed-2017: Sentiment Analysis for Indian Code Mixed Social Media Texts",
            "authors": [
                "Kamal Sarkar"
            ],
            "abstract": "This paper reports about our work in the NLP Tool Contest @ICON-2017, shared\ntask on Sentiment Analysis for Indian Languages (SAIL) (code mixed). To\nimplement our system, we have used a machine learning algo-rithm called\nMultinomial Na\\\"ive Bayes trained using n-gram and SentiWordnet features. We\nhave also used a small SentiWordnet for English and a small SentiWordnet for\nBengali. But we have not used any SentiWordnet for Hindi language. We have\ntested our system on Hindi-English and Bengali-English code mixed social media\ndata sets released for the contest. The performance of our system is very close\nto the best system participated in the contest. For both Bengali-English and\nHindi-English runs, our system was ranked at the 3rd position out of all\nsubmitted runs and awarded the 3rd prize in the contest.",
            "journal ref": "Kamal Sarkar, JU_KS@SAIL_CodeMixed-2017: Sentiment Analysis for\n  Indian Code Mixed Social Media Texts, NLP Tool Contest@the 14th International\n  Conference on Natural Language Processing, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05758v1",
            "date": "2018-02-15 20:49:19+00:00",
            "title": "Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks",
            "authors": [
                "Christian Stab",
                "Tristan Miller",
                "Iryna Gurevych"
            ],
            "abstract": "Argument mining is a core technology for automating argument search in large\ndocument collections. Despite its usefulness for this task, most current\napproaches to argument mining are designed for use only with specific text\ntypes and fall short when applied to heterogeneous texts. In this paper, we\npropose a new sentential annotation scheme that is reliably applicable by crowd\nworkers to arbitrary Web texts. We source annotations for over 25,000 instances\ncovering eight controversial topics. The results of cross-topic experiments\nshow that our attention-based neural network generalizes best to unseen topics\nand outperforms vanilla BiLSTM models by 6% in accuracy and 11% in F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05930v2",
            "date": "2018-02-16 13:38:00+00:00",
            "title": "Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing",
            "authors": [
                "K M Annervaz",
                "Somnath Basu Roy Chowdhury",
                "Ambedkar Dukkipati"
            ],
            "abstract": "Machine Learning has been the quintessential solution for many AI problems,\nbut learning is still heavily dependent on the specific training data. Some\nlearning models can be incorporated with a prior knowledge in the Bayesian set\nup, but these learning models do not have the ability to access any organised\nworld knowledge on demand. In this work, we propose to enhance learning models\nwith world knowledge in the form of Knowledge Graph (KG) fact triples for\nNatural Language Processing (NLP) tasks. Our aim is to develop a deep learning\nmodel that can extract relevant prior support facts from knowledge graphs\ndepending on the task using attention mechanism. We introduce a\nconvolution-based model for learning representations of knowledge graph entity\nand relation clusters in order to reduce the attention space. We show that the\nproposed method is highly scalable to the amount of prior information that has\nto be processed and can be applied to any generic NLP task. Using this method\nwe show significant improvement in performance for text classification with\nNews20, DBPedia datasets and natural language inference with Stanford Natural\nLanguage Inference (SNLI) dataset. We also demonstrate that a deep learning\nmodel can be trained well with substantially less amount of labeled training\ndata, when it has access to organised world knowledge in the form of knowledge\ngraph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05934v1",
            "date": "2018-02-16 13:59:15+00:00",
            "title": "Instance-based Inductive Deep Transfer Learning by Cross-Dataset Querying with Locality Sensitive Hashing",
            "authors": [
                "Somnath Basu Roy Chowdhury",
                "K M Annervaz",
                "Ambedkar Dukkipati"
            ],
            "abstract": "Supervised learning models are typically trained on a single dataset and the\nperformance of these models rely heavily on the size of the dataset, i.e.,\namount of data available with the ground truth. Learning algorithms try to\ngeneralize solely based on the data that is presented with during the training.\nIn this work, we propose an inductive transfer learning method that can augment\nlearning models by infusing similar instances from different learning tasks in\nthe Natural Language Processing (NLP) domain. We propose to use instance\nrepresentations from a source dataset, \\textit{without inheriting anything}\nfrom the source learning model. Representations of the instances of\n\\textit{source} \\& \\textit{target} datasets are learned, retrieval of relevant\nsource instances is performed using soft-attention mechanism and\n\\textit{locality sensitive hashing}, and then, augmented into the model during\ntraining on the target dataset. Our approach simultaneously exploits the local\n\\textit{instance level information} as well as the macro statistical viewpoint\nof the dataset. Using this approach we have shown significant improvements for\nthree major news classification datasets over the baseline. Experimental\nevaluations also show that the proposed approach reduces dependency on labeled\ndata by a significant margin for comparable performance. With our proposed\ncross dataset learning procedure we show that one can achieve\ncompetitive/better performance than learning from a single dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06041v1",
            "date": "2018-02-16 17:34:32+00:00",
            "title": "Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT",
            "authors": [
                "Marianna J. Martindale",
                "Marine Carpuat"
            ],
            "abstract": "Although measuring intrinsic quality has been a key factor in the advancement\nof Machine Translation (MT), successfully deploying MT requires considering not\njust intrinsic quality but also the user experience, including aspects such as\ntrust. This work introduces a method of studying how users modulate their trust\nin an MT system after seeing errorful (disfluent or inadequate) output amidst\ngood (fluent and adequate) output. We conduct a survey to determine how users\nrespond to good translations compared to translations that are either adequate\nbut not fluent, or fluent but not adequate. In this pilot study, users\nresponded strongly to disfluent translations, but were, surprisingly, much less\nconcerned with adequacy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06053v2",
            "date": "2018-02-16 17:58:43+00:00",
            "title": "Bayesian Models for Unit Discovery on a Very Low Resource Language",
            "authors": [
                "Lucas Ondel",
                "Pierre Godard",
                "Laurent Besacier",
                "Elin Larsen",
                "Mark Hasegawa-Johnson",
                "Odette Scharenborg",
                "Emmanuel Dupoux",
                "Lukas Burget",
                "Fran\u00e7ois Yvon",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Developing speech technologies for low-resource languages has become a very\nactive research field over the last decade. Among others, Bayesian models have\nshown some promising results on artificial examples but still lack of in situ\nexperiments. Our work applies state-of-the-art Bayesian models to unsupervised\nAcoustic Unit Discovery (AUD) in a real low-resource language scenario. We also\nshow that Bayesian models can naturally integrate information from other\nresourceful languages by means of informative prior leading to more consistent\ndiscovered units. Finally, discovered acoustic units are used, either as the\n1-best sequence or as a lattice, to perform word segmentation. Word\nsegmentation results show that this Bayesian approach clearly outperforms a\nSegmental-DTW baseline on the same corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06196v1",
            "date": "2018-02-17 05:52:23+00:00",
            "title": "Can Network Embedding of Distributional Thesaurus be Combined with Word Vectors for Better Representation?",
            "authors": [
                "Abhik Jana",
                "Pawan Goyal"
            ],
            "abstract": "Distributed representations of words learned from text have proved to be\nsuccessful in various natural language processing tasks in recent times. While\nsome methods represent words as vectors computed from text using predictive\nmodel (Word2vec) or dense count based model (GloVe), others attempt to\nrepresent these in a distributional thesaurus network structure where the\nneighborhood of a word is a set of words having adequate context overlap. Being\nmotivated by recent surge of research in network embedding techniques\n(DeepWalk, LINE, node2vec etc.), we turn a distributional thesaurus network\ninto dense word vectors and investigate the usefulness of distributional\nthesaurus embedding in improving overall word representation. This is the first\nattempt where we show that combining the proposed word representation obtained\nby distributional thesaurus embedding with the state-of-the-art word\nrepresentations helps in improving the performance by a significant margin when\nevaluated against NLP tasks like word similarity and relatedness, synonym\ndetection, analogy detection. Additionally, we show that even without using any\nhandcrafted lexical resources we can come up with representations having\ncomparable performance in the word similarity and relatedness tasks compared to\nthe representations where a lexical resource has been used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06613v2",
            "date": "2018-02-19 12:57:56+00:00",
            "title": "Before Name-calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation",
            "authors": [
                "Ivan Habernal",
                "Henning Wachsmuth",
                "Iryna Gurevych",
                "Benno Stein"
            ],
            "abstract": "Arguing without committing a fallacy is one of the main requirements of an\nideal debate. But even when debating rules are strictly enforced and fallacious\narguments punished, arguers often lapse into attacking the opponent by an ad\nhominem argument. As existing research lacks solid empirical investigation of\nthe typology of ad hominem arguments as well as their potential causes, this\npaper fills this gap by (1) performing several large-scale annotation studies,\n(2) experimenting with various neural architectures and validating our working\nhypotheses, such as controversy or reasonableness, and (3) providing linguistic\ninsights into triggers of ad hominem using explainable neural network\narchitectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06655v2",
            "date": "2018-02-19 14:49:42+00:00",
            "title": "Tied Multitask Learning for Neural Speech Translation",
            "authors": [
                "Antonios Anastasopoulos",
                "David Chiang"
            ],
            "abstract": "We explore multitask models for neural translation of speech, augmenting them\nin order to reflect two intuitive notions. First, we introduce a model where\nthe second task decoder receives information from the decoder of the first\ntask, since higher-level intermediate representations should provide useful\ninformation. Second, we apply regularization that encourages transitivity and\ninvertibility. We show that the application of these notions on jointly trained\nmodels improves performance on the tasks of low-resource speech transcription\nand translation. It also leads to better performance when using attention\ninformation for word discovery over unsegmented input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06842v1",
            "date": "2018-02-19 20:43:53+00:00",
            "title": "Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types",
            "authors": [
                "Hady Elsahar",
                "Christophe Gravier",
                "Frederique Laforest"
            ],
            "abstract": "We present a neural model for question generation from knowledge base triples\nin a \"Zero-Shot\" setup, that is generating questions for triples containing\npredicates, subject types or object types that were not seen at training time.\nOur model leverages triples occurrences in the natural language corpus in an\nencoder-decoder architecture, paired with an original part-of-speech copy\naction mechanism to generate questions. Benchmark and human evaluation show\nthat our model sets a new state-of-the-art for zero-shot QG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06950v1",
            "date": "2018-02-20 03:42:11+00:00",
            "title": "TAP-DLND 1.0 : A Corpus for Document Level Novelty Detection",
            "authors": [
                "Tirthankar Ghosal",
                "Amitra Salam",
                "Swati Tiwari",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Detecting novelty of an entire document is an Artificial Intelligence (AI)\nfrontier problem that has widespread NLP applications, such as extractive\ndocument summarization, tracking development of news events, predicting impact\nof scholarly articles, etc. Important though the problem is, we are unaware of\nany benchmark document level data that correctly addresses the evaluation of\nautomatic novelty detection techniques in a classification framework. To bridge\nthis gap, we present here a resource for benchmarking the techniques for\ndocument level novelty detection. We create the resource via event-specific\ncrawling of news documents across several domains in a periodic manner. We\nrelease the annotated corpus with necessary statistics and show its use with a\ndeveloped system for the problem in concern.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07117v1",
            "date": "2018-02-20 14:05:48+00:00",
            "title": "Combining Textual Content and Structure to Improve Dialog Similarity",
            "authors": [
                "Ana Paula Appel",
                "Paulo Rodrigo Cavalin",
                "Marisa Affonso Vasconcelos",
                "Claudio Santos Pinhanez"
            ],
            "abstract": "Chatbots, taking advantage of the success of the messaging apps and recent\nadvances in Artificial Intelligence, have become very popular, from helping\nbusiness to improve customer services to chatting to users for the sake of\nconversation and engagement (celebrity or personal bots). However, developing\nand improving a chatbot requires understanding their data generated by its\nusers. Dialog data has a different nature of a simple question and answering\ninteraction, in which context and temporal properties (turn order) creates a\ndifferent understanding of such data. In this paper, we propose a novelty\nmetric to compute dialogs' similarity based not only on the text content but\nalso on the information related to the dialog structure. Our experimental\nresults performed over the Switchboard dataset show that using evidence from\nboth textual content and the dialog structure leads to more accurate results\nthan using each measure in isolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07170v2",
            "date": "2018-02-17 03:11:50+00:00",
            "title": "CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++",
            "authors": [
                "Xiaolin Wang",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "This paper presents an open-source neural machine translation toolkit named\nCytonMT (https://github.com/arthurxlw/cytonMt). The toolkit is built from\nscratch only using C++ and NVIDIA's GPU-accelerated libraries. The toolkit\nfeatures training efficiency, code simplicity and translation quality.\nBenchmarks show that CytonMT accelerates the training speed by 64.5% to 110.8%\non neural networks of various sizes, and achieves competitive translation\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07226v2",
            "date": "2018-02-20 18:04:46+00:00",
            "title": "Implicit Argument Prediction with Event Knowledge",
            "authors": [
                "Pengxiang Cheng",
                "Katrin Erk"
            ],
            "abstract": "Implicit arguments are not syntactically connected to their predicates, and\nare therefore hard to extract. Previous work has used models with large numbers\nof features, evaluated on very small datasets. We propose to train models for\nimplicit argument prediction on a simple cloze task, for which data can be\ngenerated automatically at scale. This allows us to use a neural model, which\ndraws on narrative coherence and entity salience for predictions. We show that\nour model has superior performance on both synthetic and natural data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07839v2",
            "date": "2018-02-21 22:52:35+00:00",
            "title": "CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions",
            "authors": [
                "Kevin Tian",
                "Teng Zhang",
                "James Zou"
            ],
            "abstract": "Word embedding is a useful approach to capture co-occurrence structures in\nlarge text corpora. However, in addition to the text data itself, we often have\nadditional covariates associated with individual corpus documents---e.g. the\ndemographic of the author, time and venue of publication---and we would like\nthe embedding to naturally capture this information. We propose CoVeR, a new\ntensor decomposition model for vector embeddings with covariates. CoVeR jointly\nlearns a \\emph{base} embedding for all the words as well as a weighted diagonal\nmatrix to model how each covariate affects the base embedding. To obtain author\nor venue-specific embedding, for example, we can then simply multiply the base\nembedding by the associated transformation matrix. The main advantages of our\napproach are data efficiency and interpretability of the covariate\ntransformation. Our experiments demonstrate that our joint model learns\nsubstantially better covariate-specific embeddings compared to the standard\napproach of learning a separate embedding for each covariate using only the\nrelevant subset of data, as well as other related methods. Furthermore, CoVeR\nencourages the embeddings to be \"topic-aligned\" in that the dimensions have\nspecific independent meanings. This allows our covariate-specific embeddings to\nbe compared by topic, enabling downstream differential analysis. We empirically\nevaluate the benefits of our algorithm on datasets, and demonstrate how it can\nbe used to address many natural questions about covariate effects.\n  Accompanying code to this paper can be found at\nhttp://github.com/kjtian/CoVeR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07858v2",
            "date": "2018-02-22 00:27:54+00:00",
            "title": "MPST: A Corpus of Movie Plot Synopses with Tags",
            "authors": [
                "Sudipta Kar",
                "Suraj Maharjan",
                "A. Pastor L\u00f3pez-Monroy",
                "Thamar Solorio"
            ],
            "abstract": "Social tagging of movies reveals a wide range of heterogeneous information\nabout movies, like the genre, plot structure, soundtracks, metadata, visual and\nemotional experiences. Such information can be valuable in building automatic\nsystems to create tags for movies. Automatic tagging systems can help\nrecommendation engines to improve the retrieval of similar movies as well as\nhelp viewers to know what to expect from a movie in advance. In this paper, we\nset out to the task of collecting a corpus of movie plot synopses and tags. We\ndescribe a methodology that enabled us to build a fine-grained set of around 70\ntags exposing heterogeneous characteristics of movie plots and the multi-label\nassociations of these tags with some 14K movie plot synopses. We investigate\nhow these tags correlate with movies and the flow of emotions throughout\ndifferent types of movies. Finally, we use this corpus to explore the\nfeasibility of inferring tags from plot synopses. We expect the corpus will be\nuseful in other tasks where analysis of narratives is relevant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07862v1",
            "date": "2018-02-22 00:54:47+00:00",
            "title": "Multimodal Named Entity Recognition for Short Social Media Posts",
            "authors": [
                "Seungwhan Moon",
                "Leonardo Neves",
                "Vitor Carvalho"
            ],
            "abstract": "We introduce a new task called Multimodal Named Entity Recognition (MNER) for\nnoisy user-generated data such as tweets or Snapchat captions, which comprise\nshort text with accompanying images. These social media posts often come in\ninconsistent or incomplete syntax and lexical notations with very limited\nsurrounding textual contexts, bringing significant challenges for NER. To this\nend, we create a new dataset for MNER called SnapCaptions (Snapchat\nimage-caption pairs submitted to public and crowd-sourced stories with fully\nannotated named entities). We then build upon the state-of-the-art Bi-LSTM\nword/character based NER models with 1) a deep image network which incorporates\nrelevant visual context to augment textual information, and 2) a generic\nmodality-attention module which learns to attenuate irrelevant modalities while\namplifying the most informative ones to extract contexts from, adaptive to each\nsample and token. The proposed MNER model with modality attention significantly\noutperforms the state-of-the-art text-only NER models by successfully\nleveraging provided visual contexts, opening up potential applications of MNER\non myriads of social media platforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08148v1",
            "date": "2018-02-22 16:38:40+00:00",
            "title": "LIDIOMS: A Multilingual Linked Idioms Data Set",
            "authors": [
                "Diego Moussallem",
                "Mohamed Ahmed Sherif",
                "Diego Esteves",
                "Marcos Zampieri",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "In this paper, we describe the LIDIOMS data set, a multilingual RDF\nrepresentation of idioms currently containing five languages: English, German,\nItalian, Portuguese, and Russian. The data set is intended to support natural\nlanguage processing applications by providing links between idioms across\nlanguages. The underlying data was crawled and integrated from various sources.\nTo ensure the quality of the crawled data, all idioms were evaluated by at\nleast two native speakers. Herein, we present the model devised for structuring\nthe data. We also provide the details of linking LIDIOMS to well-known\nmultilingual data sets such as BabelNet. The resulting data set complies with\nbest practices according to Linguistic Linked Open Data Community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08150v1",
            "date": "2018-02-22 16:41:56+00:00",
            "title": "RDF2PT: Generating Brazilian Portuguese Texts from RDF Data",
            "authors": [
                "Diego Moussallem",
                "Thiago Castro Ferreira",
                "Marcos Zampieri",
                "Maria Claudia Cavalcanti",
                "Geraldo Xex\u00e9o",
                "Mariana Neves",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "The generation of natural language from Resource Description Framework (RDF)\ndata has recently gained significant attention due to the continuous growth of\nLinked Data. A number of these approaches generate natural language in\nlanguages other than English, however, no work has been proposed to generate\nBrazilian Portuguese texts out of RDF. We address this research gap by\npresenting RDF2PT, an approach that verbalizes RDF data to Brazilian Portuguese\nlanguage. We evaluated RDF2PT in an open questionnaire with 44 native speakers\ndivided into experts and non-experts. Our results suggest that RDF2PT is able\nto generate text which is similar to that generated by humans and can hence be\neasily understood.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08332v1",
            "date": "2018-02-22 22:34:24+00:00",
            "title": "Deep Multimodal Learning for Emotion Recognition in Spoken Language",
            "authors": [
                "Yue Gu",
                "Shuhong Chen",
                "Ivan Marsic"
            ],
            "abstract": "In this paper, we present a novel deep multimodal framework to predict human\nemotions based on sentence-level spoken language. Our architecture has two\ndistinctive characteristics. First, it extracts the high-level features from\nboth text and audio via a hybrid deep multimodal structure, which considers the\nspatial information from text, temporal information from audio, and high-level\nassociations from low-level handcrafted features. Second, we fuse all features\nby using a three-layer deep neural network to learn the correlations across\nmodalities and train the feature extraction and fusion modules together,\nallowing optimal global fine-tuning of the entire structure. We evaluated the\nproposed framework on the IEMOCAP dataset. Our result shows promising\nperformance, achieving 60.4% in weighted accuracy for five emotion categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08379v2",
            "date": "2018-02-23 04:06:38+00:00",
            "title": "EmotionLines: An Emotion Corpus of Multi-Party Conversations",
            "authors": [
                "Sheng-Yeh Chen",
                "Chao-Chun Hsu",
                "Chuan-Chun Kuo",
                "Ting-Hao",
                "Huang",
                "Lun-Wei Ku"
            ],
            "abstract": "Feeling emotion is a critical characteristic to distinguish people from\nmachines. Among all the multi-modal resources for emotion detection, textual\ndatasets are those containing the least additional information in addition to\nsemantics, and hence are adopted widely for testing the developed systems.\nHowever, most of the textual emotional datasets consist of emotion labels of\nonly individual words, sentences or documents, which makes it challenging to\ndiscuss the contextual flow of emotions. In this paper, we introduce\nEmotionLines, the first dataset with emotions labeling on all utterances in\neach dialogue only based on their textual content. Dialogues in EmotionLines\nare collected from Friends TV scripts and private Facebook messenger dialogues.\nThen one of seven emotions, six Ekman's basic emotions plus the neutral\nemotion, is labeled on each utterance by 5 Amazon MTurkers. A total of 29,245\nutterances from 2,000 dialogues are labeled in EmotionLines. We also provide\nseveral strong baselines for emotion detection models on EmotionLines in this\npaper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08395v1",
            "date": "2018-02-23 05:39:46+00:00",
            "title": "Towards end-to-end spoken language understanding",
            "authors": [
                "Dmitriy Serdyuk",
                "Yongqiang Wang",
                "Christian Fuegen",
                "Anuj Kumar",
                "Baiyang Liu",
                "Yoshua Bengio"
            ],
            "abstract": "Spoken language understanding system is traditionally designed as a pipeline\nof a number of components. First, the audio signal is processed by an automatic\nspeech recognizer for transcription or n-best hypotheses. With the recognition\nresults, a natural language understanding system classifies the text to\nstructured data as domain, intent and slots for down-streaming consumers, such\nas dialog system, hands-free applications. These components are usually\ndeveloped and optimized independently. In this paper, we present our study on\nan end-to-end learning system for spoken language understanding. With this\nunified approach, we can infer the semantic meaning directly from audio\nfeatures without the intermediate text representation. This study showed that\nthe trained model can achieve reasonable good result and demonstrated that the\nmodel can capture the semantic attention directly from the audio features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08504v1",
            "date": "2018-02-23 12:33:29+00:00",
            "title": "Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions",
            "authors": [
                "Hai Ye",
                "Xin Jiang",
                "Zhunchen Luo",
                "Wenhan Chao"
            ],
            "abstract": "In this paper, we propose to study the problem of COURT VIEW GENeration from\nthe fact description in a criminal case. The task aims to improve the\ninterpretability of charge prediction systems and help automatic legal document\ngeneration. We formulate this task as a text-to-text natural language\ngeneration (NLG) problem. Sequenceto-sequence model has achieved cutting-edge\nperformances in many NLG tasks. However, due to the non-distinctions of fact\ndescriptions, it is hard for Seq2Seq model to generate charge-discriminative\ncourt views. In this work, we explore charge labels to tackle this issue. We\npropose a label-conditioned Seq2Seq model with attention for this problem, to\ndecode court views conditioned on encoded charge labels. Experimental results\nshow the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08599v2",
            "date": "2018-02-23 15:31:19+00:00",
            "title": "Evaluating Scoped Meaning Representations",
            "authors": [
                "Rik van Noord",
                "Lasha Abzianidze",
                "Hessel Haagsma",
                "Johan Bos"
            ],
            "abstract": "Semantic parsing offers many opportunities to improve natural language\nunderstanding. We present a semantically annotated parallel corpus for English,\nGerman, Italian, and Dutch where sentences are aligned with scoped meaning\nrepresentations in order to capture the semantics of negation, modals,\nquantification, and presupposition triggers. The semantic formalism is based on\nDiscourse Representation Theory, but concepts are represented by WordNet\nsynsets and thematic roles by VerbNet relations. Translating scoped meaning\nrepresentations to sets of clauses enables us to compare them for the purpose\nof semantic parser evaluation and checking translations. This is done by\ncomputing precision and recall on matching clauses, in a similar way as is done\nfor Abstract Meaning Representations. We show that our matching tool for\nevaluating scoped meaning representations is both accurate and efficient.\nApplying this matching tool to three baseline semantic parsers yields F-scores\nbetween 43% and 54%. A pilot study is performed to automatically find changes\nin meaning by comparing meaning representations of translations. This\ncomparison turns out to be an additional way of (i) finding annotation mistakes\nand (ii) finding instances where our semantic analysis needs to be improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08636v2",
            "date": "2018-02-23 16:55:31+00:00",
            "title": "Ranking Sentences for Extractive Summarization with Reinforcement Learning",
            "authors": [
                "Shashi Narayan",
                "Shay B. Cohen",
                "Mirella Lapata"
            ],
            "abstract": "Single document summarization is the task of producing a shorter version of a\ndocument while preserving its principal information content. In this paper we\nconceptualize extractive summarization as a sentence ranking task and propose a\nnovel training algorithm which globally optimizes the ROUGE evaluation metric\nthrough a reinforcement learning objective. We use our algorithm to train a\nneural summarization model on the CNN and DailyMail datasets and demonstrate\nexperimentally that it outperforms state-of-the-art extractive and abstractive\nsystems when evaluated automatically and by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08731v2",
            "date": "2018-02-23 20:47:51+00:00",
            "title": "Automatic Speech Recognition and Topic Identification for Almost-Zero-Resource Languages",
            "authors": [
                "Matthew Wiesner",
                "Chunxi Liu",
                "Lucas Ondel",
                "Craig Harman",
                "Vimal Manohar",
                "Jan Trmal",
                "Zhongqiang Huang",
                "Najim Dehak",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Automatic speech recognition (ASR) systems often need to be developed for\nextremely low-resource languages to serve end-uses such as audio content\ncategorization and search. While universal phone recognition is natural to\nconsider when no transcribed speech is available to train an ASR system in a\nlanguage, adapting universal phone models using very small amounts (minutes\nrather than hours) of transcribed speech also needs to be studied, particularly\nwith state-of-the-art DNN-based acoustic models. The DARPA LORELEI program\nprovides a framework for such very-low-resource ASR studies, and provides an\nextrinsic metric for evaluating ASR performance in a humanitarian assistance,\ndisaster relief setting. This paper presents our Kaldi-based systems for the\nprogram, which employ a universal phone modeling approach to ASR, and describes\nrecipes for very rapid adaptation of this universal ASR system. The results we\nobtain significantly outperform results obtained by many competing approaches\non the NIST LoReHLT 2017 Evaluation datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08949v2",
            "date": "2018-02-25 03:49:21+00:00",
            "title": "OhioState at SemEval-2018 Task 7: Exploiting Data Augmentation for Relation Classification in Scientific Papers using Piecewise Convolutional Neural Networks",
            "authors": [
                "Dushyanta Dhyani"
            ],
            "abstract": "We describe our system for SemEval-2018 Shared Task on Semantic Relation\nExtraction and Classification in Scientific Papers where we focus on the\nClassification task. Our simple piecewise convolution neural encoder performs\ndecently in an end to end manner. A simple inter-task data augmentation\nsignifi- cantly boosts the performance of the model. Our best-performing\nsystems stood 8th out of 20 teams on the classification task on noisy data and\n12th out of 28 teams on the classification task on clean data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08970v1",
            "date": "2018-02-25 09:01:55+00:00",
            "title": "Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method",
            "authors": [
                "Jinyue Su",
                "Jiacheng Xu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Generating plausible and fluent sentence with desired properties has long\nbeen a challenge. Most of the recent works use recurrent neural networks (RNNs)\nand their variants to predict following words given previous sequence and\ntarget label. In this paper, we propose a novel framework to generate\nconstrained sentences via Gibbs Sampling. The candidate sentences are revised\nand updated iteratively, with sampled new words replacing old ones. Our\nexperiments show the effectiveness of the proposed method to generate plausible\nand diverse sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09091v3",
            "date": "2018-02-25 21:52:37+00:00",
            "title": "Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks",
            "authors": [
                "R. Thomas McCoy",
                "Robert Frank",
                "Tal Linzen"
            ],
            "abstract": "Syntactic rules in natural language typically need to make reference to\nhierarchical sentence structure. However, the simple examples that language\nlearners receive are often equally compatible with linear rules. Children\nconsistently ignore these linear explanations and settle instead on the correct\nhierarchical one. This fact has motivated the proposal that the learner's\nhypothesis space is constrained to include only hierarchical rules. We examine\nthis proposal using recurrent neural networks (RNNs), which are not constrained\nin such a way. We simulate the acquisition of question formation, a\nhierarchical transformation, in a fragment of English. We find that some RNN\narchitectures tend to learn the hierarchical rule, suggesting that hierarchical\ncues within the language, combined with the implicit architectural biases\ninherent in certain RNNs, may be sufficient to induce hierarchical\ngeneralizations. The likelihood of acquiring the hierarchical generalization\nincreased when the language included an additional cue to hierarchy in the form\nof subject-verb agreement, underscoring the role of cues to hierarchy in the\nlearner's input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09130v2",
            "date": "2018-02-26 02:08:28+00:00",
            "title": "Did You Really Just Have a Heart Attack? Towards Robust Detection of Personal Health Mentions in Social Media",
            "authors": [
                "Payam Karisani",
                "Eugene Agichtein"
            ],
            "abstract": "Millions of users share their experiences on social media sites, such as\nTwitter, which in turn generate valuable data for public health monitoring,\ndigital epidemiology, and other analyses of population health at global scale.\nThe first, critical, task for these applications is classifying whether a\npersonal health event was mentioned, which we call the (PHM) problem. This task\nis challenging for many reasons, including typically short length of social\nmedia posts, inventive spelling and lexicons, and figurative language,\nincluding hyperbole using diseases like \"heart attack\" or \"cancer\" for\nemphasis, and not as a health self-report. This problem is even more\nchallenging for rarely reported, or frequent but ambiguously expressed\nconditions, such as \"stroke\". To address this problem, we propose a general,\nrobust method for detecting PHMs in social media, which we call WESPAD, that\ncombines lexical, syntactic, word embedding-based, and context-based features.\nWESPAD is able to generalize from few examples by automatically distorting the\nword embedding space to most effectively detect the true health mentions.\nUnlike previously proposed state-of-the-art supervised and deep-learning\ntechniques, WESPAD requires relatively little training data, which makes it\npossible to adapt, with minimal effort, to each new disease and condition. We\nevaluate WESPAD on both an established publicly available Flu detection\nbenchmark, and on a new dataset that we have constructed with mentions of\nmultiple health conditions. Our experiments show that WESPAD outperforms the\nbaselines and state-of-the-art methods, especially in cases when the number and\nproportion of true health mentions in the training data is small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09189v1",
            "date": "2018-02-26 07:52:30+00:00",
            "title": "Language Distribution Prediction based on Batch Markov Monte Carlo Simulation with Migration",
            "authors": [
                "XingYu Fu",
                "ZiYi Yang",
                "XiuWen Duan"
            ],
            "abstract": "Language spreading is a complex mechanism that involves issues like culture,\neconomics, migration, population etc. In this paper, we propose a set of\nmethods to model the dynamics of the spreading system. To model the randomness\nof language spreading, we propose the Batch Markov Monte Carlo Simulation with\nMigration(BMMCSM) algorithm, in which each agent is treated as a language\nstack. The agent learns languages and migrates based on the proposed Batch\nMarkov Property according to the transition matrix T and migration matrix M.\nSince population plays a crucial role in language spreading, we also introduce\nthe Mortality and Fertility Mechanism, which controls the birth and death of\nthe simulated agents, into the BMMCSM algorithm. The simulation results of\nBMMCSM show that the numerical and geographic distribution of languages varies\nacross the time. The change of distribution fits the world cultural and\neconomic development trend. Next, when we construct Matrix T, there are some\nentries of T can be directly calculated from historical statistics while some\nentries of T is unknown. Thus, the key to the success of the BMMCSM lies in the\naccurate estimation of transition matrix T by estimating the unknown entries of\nT under the supervision of the known entries. To achieve this, we first\nconstruct a 20 by 20 by 5 factor tensor X to characterize each entry of T. Then\nwe train a Random Forest Regressor on the known entries of T and use the\ntrained regressor to predict the unknown entries. The reason why we choose\nRandom Forest(RF) is that, compared to Single Decision Tree, it conquers the\nproblem of over fitting and the Shapiro test also suggests that the residual of\nRF subjects to the Normal distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09194v1",
            "date": "2018-02-26 08:21:26+00:00",
            "title": "Deep Feed-forward Sequential Memory Networks for Speech Synthesis",
            "authors": [
                "Mengxiao Bi",
                "Heng Lu",
                "Shiliang Zhang",
                "Ming Lei",
                "Zhijie Yan"
            ],
            "abstract": "The Bidirectional LSTM (BLSTM) RNN based speech synthesis system is among the\nbest parametric Text-to-Speech (TTS) systems in terms of the naturalness of\ngenerated speech, especially the naturalness in prosody. However, the model\ncomplexity and inference cost of BLSTM prevents its usage in many runtime\napplications. Meanwhile, Deep Feed-forward Sequential Memory Networks (DFSMN)\nhas shown its consistent out-performance over BLSTM in both word error rate\n(WER) and the runtime computation cost in speech recognition tasks. Since\nspeech synthesis also requires to model long-term dependencies compared to\nspeech recognition, in this paper, we investigate the Deep-FSMN (DFSMN) in\nspeech synthesis. Both objective and subjective experiments show that, compared\nwith BLSTM TTS method, the DFSMN system can generate synthesized speech with\ncomparable speech quality while drastically reduce model complexity and speech\ngeneration time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09233v1",
            "date": "2018-02-26 10:20:09+00:00",
            "title": "EiTAKA at SemEval-2018 Task 1: An Ensemble of N-Channels ConvNet and XGboost Regressors for Emotion Analysis of Tweets",
            "authors": [
                "Mohammed Jabreel",
                "Antonio Moreno"
            ],
            "abstract": "This paper describes our system that has been used in Task1 Affect in Tweets.\nWe combine two different approaches. The first one called N-Stream ConvNets,\nwhich is a deep learning approach where the second one is XGboost regresseor\nbased on a set of embedding and lexicons based features. Our system was\nevaluated on the testing sets of the tasks outperforming all other approaches\nfor the Arabic version of valence intensity regression task and valence ordinal\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09287v1",
            "date": "2018-02-26 13:26:43+00:00",
            "title": "Gender Aware Spoken Language Translation Applied to English-Arabic",
            "authors": [
                "Mostafa Elaraby",
                "Ahmed Y. Tawfik",
                "Mahmoud Khaled",
                "Hany Hassan",
                "Aly Osama"
            ],
            "abstract": "Spoken Language Translation (SLT) is becoming more widely used and becoming a\ncommunication tool that helps in crossing language barriers. One of the\nchallenges of SLT is the translation from a language without gender agreement\nto a language with gender agreement such as English to Arabic. In this paper,\nwe introduce an approach to tackle such limitation by enabling a Neural Machine\nTranslation system to produce gender-aware translation. We show that NMT system\ncan model the speaker/listener gender information to produce gender-aware\ntranslation. We propose a method to generate data used in adapting a NMT system\nto produce gender-aware. The proposed approach can achieve significant\nimprovement of the translation quality by 2 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09375v1",
            "date": "2018-02-23 11:55:44+00:00",
            "title": "From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings",
            "authors": [
                "Johannes Bjerva",
                "Isabelle Augenstein"
            ],
            "abstract": "A core part of linguistic typology is the classification of languages\naccording to linguistic properties, such as those detailed in the World Atlas\nof Language Structure (WALS). Doing this manually is prohibitively\ntime-consuming, which is in part evidenced by the fact that only 100 out of\nover 7,000 languages spoken in the world are fully covered in WALS.\n  We learn distributed language representations, which can be used to predict\ntypological properties on a massively multilingual scale. Additionally,\nquantitative and qualitative analyses of these language embeddings can tell us\nhow language similarities are encoded in NLP models for tasks at different\ntypological levels. The representations are learned in an unsupervised manner\nalongside tasks at three typological levels: phonology (grapheme-to-phoneme\nprediction, and phoneme reconstruction), morphology (morphological inflection),\nand syntax (part-of-speech tagging).\n  We consider more than 800 languages and find significant differences in the\nlanguage representations encoded, depending on the target task. For instance,\nalthough Norwegian Bokm{\\aa}l and Danish are typologically close to one\nanother, they are phonologically distant, which is reflected in their language\nembeddings growing relatively distant in a phonological task. We are also able\nto predict typological features in WALS with high accuracies, even for unseen\nlanguage families.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09416v2",
            "date": "2018-02-26 15:59:22+00:00",
            "title": "A Quality Type-aware Annotated Corpus and Lexicon for Harassment Research",
            "authors": [
                "Mohammadreza Rezvan",
                "Saeedeh Shekarpour",
                "Lakshika Balasuriya",
                "Krishnaprasad Thirunarayan",
                "Valerie Shalin",
                "Amit Sheth"
            ],
            "abstract": "Having a quality annotated corpus is essential especially for applied\nresearch. Despite the recent focus of Web science community on researching\nabout cyberbullying, the community dose not still have standard benchmarks. In\nthis paper, we publish first, a quality annotated corpus and second, an\noffensive words lexicon capturing different types type of harassment as (i)\nsexual harassment, (ii) racial harassment, (iii) appearance-related harassment,\n(iv) intellectual harassment, and (v) political harassment.We crawled data from\nTwitter using our offensive lexicon. Then relied on the human judge to annotate\nthe collected tweets w.r.t. the contextual types because using offensive words\nis not sufficient to reliably detect harassment. Our corpus consists of 25,000\nannotated tweets in five contextual types. We are pleased to share this novel\nannotated corpus and the lexicon with the research community. The instruction\nto acquire the corpus has been published on the Git repository.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09884v1",
            "date": "2018-02-27 13:51:30+00:00",
            "title": "Live Blog Corpus for Summarization",
            "authors": [
                "Avinesh P. V. S.",
                "Maxime Peyrard",
                "Christian M. Meyer"
            ],
            "abstract": "Live blogs are an increasingly popular news format to cover breaking news and\nlive events in online journalism. Online news websites around the world are\nusing this medium to give their readers a minute by minute update on an event.\nGood summaries enhance the value of the live blogs for a reader but are often\nnot available. In this paper, we study a way of collecting corpora for\nautomatic live blog summarization. In an empirical evaluation using well-known\nstate-of-the-art summarization systems, we show that live blogs corpus poses\nnew challenges in the field of summarization. We make our tools publicly\navailable to reconstruct the corpus to encourage the research community and\nreplicate our results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09961v1",
            "date": "2018-02-27 15:20:43+00:00",
            "title": "Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotions",
            "authors": [
                "Jing Peng",
                "Anna Feldman",
                "Ekaterina Vylomova"
            ],
            "abstract": "We describe an algorithm for automatic classification of idiomatic and\nliteral expressions. Our starting point is that words in a given text segment,\nsuch as a paragraph, that are highranking representatives of a common topic of\ndiscussion are less likely to be a part of an idiomatic expression. Our\nadditional hypothesis is that contexts in which idioms occur, typically, are\nmore affective and therefore, we incorporate a simple analysis of the intensity\nof the emotions expressed by the contexts. We investigate the bag of words\ntopic representation of one to three paragraphs containing an expression that\nshould be classified as idiomatic or literal (a target phrase). We extract\ntopics from paragraphs containing idioms and from paragraphs containing\nliterals using an unsupervised clustering method, Latent Dirichlet Allocation\n(LDA) (Blei et al., 2003). Since idiomatic expressions exhibit the property of\nnon-compositionality, we assume that they usually present different semantics\nthan the words used in the local topic. We treat idioms as semantic outliers,\nand the identification of a semantic shift as outlier detection. Thus, this\ntopic representation allows us to differentiate idioms from literals using\nlocal semantic contexts. Our results are encouraging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09968v2",
            "date": "2018-02-27 15:31:11+00:00",
            "title": "A Hybrid Word-Character Approach to Abstractive Summarization",
            "authors": [
                "Chieh-Teng Chang",
                "Chi-Chia Huang",
                "Chih-Yuan Yang",
                "Jane Yung-Jen Hsu"
            ],
            "abstract": "Automatic abstractive text summarization is an important and challenging\nresearch topic of natural language processing. Among many widely used\nlanguages, the Chinese language has a special property that a Chinese character\ncontains rich information comparable to a word. Existing Chinese text\nsummarization methods, either adopt totally character-based or word-based\nrepresentations, fail to fully exploit the information carried by both\nrepresentations. To accurately capture the essence of articles, we propose a\nhybrid word-character approach (HWC) which preserves the advantages of both\nword-based and character-based representations. We evaluate the advantage of\nthe proposed HWC approach by applying it to two existing methods, and discover\nthat it generates state-of-the-art performance with a margin of 24 ROUGE points\non a widely used dataset LCSTS. In addition, we find an issue contained in the\nLCSTS dataset and offer a script to remove overlapping pairs (a summary and a\nshort text) to create a clean dataset for the community. The proposed HWC\napproach also generates the best performance on the new, clean LCSTS dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10137v1",
            "date": "2018-02-27 19:48:51+00:00",
            "title": "Extractive Text Summarization using Neural Networks",
            "authors": [
                "Aakash Sinha",
                "Abhishek Yadav",
                "Akshay Gahlot"
            ],
            "abstract": "Text Summarization has been an extensively studied problem. Traditional\napproaches to text summarization rely heavily on feature engineering. In\ncontrast to this, we propose a fully data-driven approach using feedforward\nneural networks for single document summarization. We train and evaluate the\nmodel on standard DUC 2002 dataset which shows results comparable to the state\nof the art models. The proposed model is scalable and is able to produce the\nsummary of arbitrarily sized documents by breaking the original document into\nfixed sized parts and then feeding it recursively to the network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10229v2",
            "date": "2018-02-28 02:01:30+00:00",
            "title": "Collective Entity Disambiguation with Structured Gradient Tree Boosting",
            "authors": [
                "Yi Yang",
                "Ozan Irsoy",
                "Kazi Shefaet Rahman"
            ],
            "abstract": "We present a gradient-tree-boosting-based structured learning model for\njointly disambiguating named entities in a document. Gradient tree boosting is\na widely used machine learning algorithm that underlies many top-performing\nnatural language processing systems. Surprisingly, most works limit the use of\ngradient tree boosting as a tool for regular classification or regression\nproblems, despite the structured nature of language. To the best of our\nknowledge, our work is the first one that employs the structured gradient tree\nboosting (SGTB) algorithm for collective entity disambiguation. By defining\nglobal features over previous disambiguation decisions and jointly modeling\nthem with local features, our system is able to produce globally optimized\nentity assignments for mentions in a document. Exact inference is prohibitively\nexpensive for our globally normalized model. To solve this problem, we propose\nBidirectional Beam Search with Gold path (BiBSG), an approximate inference\nalgorithm that is a variant of the standard beam search algorithm. BiBSG makes\nuse of global information from both past and future to perform better local\nsearch. Experiments on standard benchmark datasets show that SGTB significantly\nimproves upon published results. Specifically, SGTB outperforms the previous\nstate-of-the-art neural system by near 1\\% absolute accuracy on the popular\nAIDA-CoNLL dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10279v1",
            "date": "2018-02-28 06:27:37+00:00",
            "title": "Medical Exam Question Answering with Large-scale Reading Comprehension",
            "authors": [
                "Xiao Zhang",
                "Ji Wu",
                "Zhiyang He",
                "Xien Liu",
                "Ying Su"
            ],
            "abstract": "Reading and understanding text is one important component in computer aided\ndiagnosis in clinical medicine, also being a major research problem in the\nfield of NLP. In this work, we introduce a question-answering task called MedQA\nto study answering questions in clinical medicine using knowledge in a\nlarge-scale document collection. The aim of MedQA is to answer real-world\nquestions with large-scale reading comprehension. We propose our solution\nSeaReader--a modular end-to-end reading comprehension model based on LSTM\nnetworks and dual-path attention architecture. The novel dual-path attention\nmodels information flow from two perspectives and has the ability to\nsimultaneously read individual documents and integrate information across\nmultiple documents. In experiments our SeaReader achieved a large increase in\naccuracy on MedQA over competing models. Additionally, we develop a series of\nnovel techniques to demonstrate the interpretation of the question answering\nprocess in SeaReader.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10569v1",
            "date": "2018-02-28 18:17:40+00:00",
            "title": "Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction",
            "authors": [
                "Patrick Verga",
                "Emma Strubell",
                "Andrew McCallum"
            ],
            "abstract": "Most work in relation extraction forms a prediction by looking at a short\nspan of text within a single sentence containing a single entity pair mention.\nThis approach often does not consider interactions across mentions, requires\nredundant computation for each mention pair, and ignores relationships\nexpressed across sentence boundaries. These problems are exacerbated by the\ndocument- (rather than sentence-) level annotation common in biological text.\nIn response, we propose a model which simultaneously predicts relationships\nbetween all mention pairs in a document. We form pairwise predictions over\nentire paper abstracts using an efficient self-attention encoder. All-pairs\nmention scores allow us to perform multi-instance learning by aggregating over\nmentions to form entity pair representations. We further adapt to settings\nwithout mention-level annotation by jointly training to predict named entities\nand adding a corpus of weakly labeled data. In experiments on two Biocreative\nbenchmark datasets, we achieve state of the art performance on the Biocreative\nV Chemical Disease Relation dataset for models without external KB resources.\nWe also introduce a new dataset an order of magnitude larger than existing\nhuman-annotated biological information extraction datasets and more accurate\nthan distantly supervised alternatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00047v4",
            "date": "2018-02-28 19:33:24+00:00",
            "title": "Analyzing Uncertainty in Neural Machine Translation",
            "authors": [
                "Myle Ott",
                "Michael Auli",
                "David Grangier",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Machine translation is a popular test bed for research in neural\nsequence-to-sequence models but despite much recent research, there is still a\nlack of understanding of these models. Practitioners report performance\ndegradation with large beams, the under-estimation of rare words and a lack of\ndiversity in the final translations. Our study relates some of these issues to\nthe inherent uncertainty of the task, due to the existence of multiple valid\ntranslations for a single source sentence, and to the extrinsic uncertainty\ncaused by noisy training data. We propose tools and metrics to assess how\nuncertainty in the data is captured by the model distribution and how it\naffects search strategies that generate translations. Our results show that\nsearch works remarkably well but that models tend to spread too much\nprobability mass over the hypothesis space. Next, we propose tools to assess\nmodel calibration and show how to easily fix some shortcomings of current\nmodels. As part of this study, we release multiple human reference translations\nfor two popular benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00124v2",
            "date": "2018-02-28 22:46:19+00:00",
            "title": "Improving Sentiment Analysis in Arabic Using Word Representation",
            "authors": [
                "Abdulaziz M. Alayba",
                "Vasile Palade",
                "Matthew England",
                "Rahat Iqbal"
            ],
            "abstract": "The complexities of Arabic language in morphology, orthography and dialects\nmakes sentiment analysis for Arabic more challenging. Also, text feature\nextraction from short messages like tweets, in order to gauge the sentiment,\nmakes this task even more difficult. In recent years, deep neural networks were\noften employed and showed very good results in sentiment classification and\nnatural language processing applications. Word embedding, or word distributing\napproach, is a current and powerful tool to capture together the closest words\nfrom a contextual text. In this paper, we describe how we construct Word2Vec\nmodels from a large Arabic corpus obtained from ten newspapers in different\nArab countries. By applying different machine learning algorithms and\nconvolutional neural networks with different text feature selections, we report\nimproved accuracy of sentiment classification (91%-95%) on our publicly\navailable Arabic language health sentiment dataset [1]",
            "journal ref": "Proc. 2nd International Workshop on Arabic and Derived Script\n  Analysis and Recognition (ASAR '18), pp. 13-18. IEEE, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00179v1",
            "date": "2018-03-01 02:48:47+00:00",
            "title": "Matching Natural Language Sentences with Hierarchical Sentence Factorization",
            "authors": [
                "Bang Liu",
                "Ting Zhang",
                "Fred X. Han",
                "Di Niu",
                "Kunfeng Lai",
                "Yu Xu"
            ],
            "abstract": "Semantic matching of natural language sentences or identifying the\nrelationship between two sentences is a core research problem underlying many\nnatural language tasks. Depending on whether training data is available, prior\nresearch has proposed both unsupervised distance-based schemes and supervised\ndeep learning schemes for sentence matching. However, previous approaches\neither omit or fail to fully utilize the ordered, hierarchical, and flexible\nstructures of language objects, as well as the interactions between them. In\nthis paper, we propose Hierarchical Sentence Factorization---a technique to\nfactorize a sentence into a hierarchical representation, with the components at\neach different scale reordered into a \"predicate-argument\" form. The proposed\nsentence factorization technique leads to the invention of: 1) a new\nunsupervised distance metric which calculates the semantic distance between a\npair of text snippets by solving a penalized optimal transport problem while\npreserving the logical relationship of words in the reordered sentences, and 2)\nnew multi-scale deep learning models for supervised semantic training, based on\nfactorized sentence hierarchies. We apply our techniques to text-pair\nsimilarity estimation and text-pair relationship classification tasks, based on\nmultiple datasets such as STSbenchmark, the Microsoft Research paraphrase\nidentification (MSRP) dataset, the SICK dataset, etc. Extensive experiments\nshow that the proposed hierarchical sentence factorization can be used to\nsignificantly improve the performance of existing unsupervised distance-based\nmetrics as well as multiple supervised deep learning models based on the\nconvolutional neural network (CNN) and long short-term memory (LSTM).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00188v1",
            "date": "2018-03-01 03:14:54+00:00",
            "title": "XNMT: The eXtensible Neural Machine Translation Toolkit",
            "authors": [
                "Graham Neubig",
                "Matthias Sperber",
                "Xinyi Wang",
                "Matthieu Felix",
                "Austin Matthews",
                "Sarguna Padmanabhan",
                "Ye Qi",
                "Devendra Singh Sachan",
                "Philip Arthur",
                "Pierre Godard",
                "John Hewitt",
                "Rachid Riad",
                "Liming Wang"
            ],
            "abstract": "This paper describes XNMT, the eXtensible Neural Machine Translation toolkit.\nXNMT distin- guishes itself from other open-source NMT toolkits by its focus on\nmodular code design, with the purpose of enabling fast iteration in research\nand replicable, reliable results. In this paper we describe the design of XNMT\nand its experiment configuration system, and demonstrate its utility on the\ntasks of machine translation, speech recognition, and multi-tasked machine\ntranslation/parsing. XNMT is available open-source at\nhttps://github.com/neulab/xnmt",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00191v5",
            "date": "2018-03-01 03:23:55+00:00",
            "title": "Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension",
            "authors": [
                "Liang Wang",
                "Meng Sun",
                "Wei Zhao",
                "Kewei Shen",
                "Jingming Liu"
            ],
            "abstract": "This paper describes our system for SemEval-2018 Task 11: Machine\nComprehension using Commonsense Knowledge. We use Three-way Attentive Networks\n(TriAN) to model interactions between the passage, question and answers. To\nincorporate commonsense knowledge, we augment the input with relation embedding\nfrom the graph of general knowledge ConceptNet (Speer et al., 2017). As a\nresult, our system achieves state-of-the-art performance with 83.95% accuracy\non the official test data. Code is publicly available at\nhttps://github.com/intfloat/commonsense-rc",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00353v1",
            "date": "2018-03-01 13:14:35+00:00",
            "title": "Joint Training for Neural Machine Translation Models with Monolingual Data",
            "authors": [
                "Zhirui Zhang",
                "Shujie Liu",
                "Mu Li",
                "Ming Zhou",
                "Enhong Chen"
            ],
            "abstract": "Monolingual data have been demonstrated to be helpful in improving\ntranslation quality of both statistical machine translation (SMT) systems and\nneural machine translation (NMT) systems, especially in resource-poor or domain\nadaptation tasks where parallel data are not rich enough. In this paper, we\npropose a novel approach to better leveraging monolingual data for neural\nmachine translation by jointly learning source-to-target and target-to-source\nNMT models for a language pair with a joint EM optimization method. The\ntraining process starts with two initial NMT models pre-trained on parallel\ndata for each direction, and these two models are iteratively updated by\nincrementally decreasing translation losses on training data. In each iteration\nstep, both NMT models are first used to translate monolingual data from one\nlanguage to the other, forming pseudo-training data of the other NMT model.\nThen two new NMT models are learnt from parallel data together with the pseudo\ntraining data. Both NMT models are expected to be improved and better\npseudo-training data can be generated in next step. Experiment results on\nChinese-English and English-German translation tasks show that our approach can\nsimultaneously improve translation quality of source-to-target and\ntarget-to-source models, significantly outperforming strong baseline systems\nwhich are enhanced with monolingual data for model training including\nback-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00357v1",
            "date": "2018-03-01 13:40:04+00:00",
            "title": "Cross-lingual and Multilingual Speech Emotion Recognition on English and French",
            "authors": [
                "Michael Neumann",
                "Ngoc Thang Vu"
            ],
            "abstract": "Research on multilingual speech emotion recognition faces the problem that\nmost available speech corpora differ from each other in important ways, such as\nannotation methods or interaction scenarios. These inconsistencies complicate\nbuilding a multilingual system. We present results for cross-lingual and\nmultilingual emotion recognition on English and French speech data with similar\ncharacteristics in terms of interaction (human-human conversations). Further,\nwe explore the possibility of fine-tuning a pre-trained cross-lingual model\nwith only a small number of samples from the target language, which is of great\ninterest for low-resource languages. To gain more insights in what is learned\nby the deployed convolutional neural network, we perform an analysis on the\nattention mechanism inside the network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00712v3",
            "date": "2018-03-02 04:10:24+00:00",
            "title": "A Factoid Question Answering System for Vietnamese",
            "authors": [
                "Phuong Le-Hong",
                "Duc-Thien Bui"
            ],
            "abstract": "In this paper, we describe the development of an end-to-end factoid question\nanswering system for the Vietnamese language. This system combines both\nstatistical models and ontology-based methods in a chain of processing modules\nto provide high-quality mappings from natural language text to entities. We\npresent the challenges in the development of such an intelligent user interface\nfor an isolating language like Vietnamese and show that techniques developed\nfor inflectional languages cannot be applied \"as is\". Our question answering\nsystem can answer a wide range of general knowledge questions with promising\naccuracy on a test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00831v1",
            "date": "2018-03-02 12:59:32+00:00",
            "title": "Lexico-acoustic Neural-based Models for Dialog Act Classification",
            "authors": [
                "Daniel Ortega",
                "Ngoc Thang Vu"
            ],
            "abstract": "Recent works have proposed neural models for dialog act classification in\nspoken dialogs. However, they have not explored the role and the usefulness of\nacoustic information. We propose a neural model that processes both lexical and\nacoustic features for classification. Our results on two benchmark datasets\nreveal that acoustic features are helpful in improving the overall accuracy.\nFinally, a deeper analysis shows that acoustic features are valuable in three\ncases: when a dialog act has sufficient data, when lexical information is\nlimited and when strong lexical cues are not present.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00902v1",
            "date": "2018-03-02 15:41:33+00:00",
            "title": "DEMorphy, German Language Morphological Analyzer",
            "authors": [
                "Duygu Altinok"
            ],
            "abstract": "DEMorphy is a morphological analyzer for German. It is built onto large,\ncompactified lexicons from German Morphological Dictionary. A guesser based on\nGerman declension suffixed is also provided. For German, we provided a\nstate-of-art morphological analyzer. DEMorphy is implemented in Python with\nease of usability and accompanying documentation. The package is suitable for\nboth academic and commercial purposes wit a permissive licence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00985v1",
            "date": "2018-03-02 18:34:08+00:00",
            "title": "Hybrid Model For Word Prediction Using Naive Bayes and Latent Information",
            "authors": [
                "Henrique X. Goulart",
                "Mauro D. L. Tosi",
                "Daniel Soares Gon\u00e7alves",
                "Rodrigo F. Maia",
                "Guilherme A. Wachs-Lopes"
            ],
            "abstract": "Historically, the Natural Language Processing area has been given too much\nattention by many researchers. One of the main motivation beyond this interest\nis related to the word prediction problem, which states that given a set words\nin a sentence, one can recommend the next word. In literature, this problem is\nsolved by methods based on syntactic or semantic analysis. Solely, each of\nthese analysis cannot achieve practical results for end-user applications. For\ninstance, the Latent Semantic Analysis can handle semantic features of text,\nbut cannot suggest words considering syntactical rules. On the other hand,\nthere are models that treat both methods together and achieve state-of-the-art\nresults, e.g. Deep Learning. These models can demand high computational effort,\nwhich can make the model infeasible for certain types of applications. With the\nadvance of the technology and mathematical models, it is possible to develop\nfaster systems with more accuracy. This work proposes a hybrid word suggestion\nmodel, based on Naive Bayes and Latent Semantic Analysis, considering\nneighbouring words around unfilled gaps. Results show that this model could\nachieve 44.2% of accuracy in the MSR Sentence Completion Challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01090v1",
            "date": "2018-03-03 02:08:46+00:00",
            "title": "On Modular Training of Neural Acoustics-to-Word Model for LVCSR",
            "authors": [
                "Zhehuai Chen",
                "Qi Liu",
                "Hao Li",
                "Kai Yu"
            ],
            "abstract": "End-to-end (E2E) automatic speech recognition (ASR) systems directly map\nacoustics to words using a unified model. Previous works mostly focus on E2E\ntraining a single model which integrates acoustic and language model into a\nwhole. Although E2E training benefits from sequence modeling and simplified\ndecoding pipelines, large amount of transcribed acoustic data is usually\nrequired, and traditional acoustic and language modelling techniques cannot be\nutilized. In this paper, a novel modular training framework of E2E ASR is\nproposed to separately train neural acoustic and language models during\ntraining stage, while still performing end-to-end inference in decoding stage.\nHere, an acoustics-to-phoneme model (A2P) and a phoneme-to-word model (P2W) are\ntrained using acoustic data and text data respectively. A phone synchronous\ndecoding (PSD) module is inserted between A2P and P2W to reduce sequence\nlengths without precision loss. Finally, modules are integrated into an\nacousticsto-word model (A2W) and jointly optimized using acoustic data to\nretain the advantage of sequence modeling. Experiments on a 300- hour\nSwitchboard task show significant improvement over the direct A2W model. The\nefficiency in both training and decoding also benefits from the proposed\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01165v1",
            "date": "2018-03-03 13:57:37+00:00",
            "title": "Tag-Enhanced Tree-Structured Neural Networks for Implicit Discourse Relation Classification",
            "authors": [
                "Yizhong Wang",
                "Sujian Li",
                "Jingfeng Yang",
                "Xu Sun",
                "Houfeng Wang"
            ],
            "abstract": "Identifying implicit discourse relations between text spans is a challenging\ntask because it requires understanding the meaning of the text. To tackle this\ntask, recent studies have tried several deep learning methods but few of them\nexploited the syntactic information. In this work, we explore the idea of\nincorporating syntactic parse tree into neural networks. Specifically, we\nemploy the Tree-LSTM model and Tree-GRU model, which are based on the tree\nstructure, to encode the arguments in a relation. Moreover, we further leverage\nthe constituent tags to control the semantic composition process in these\ntree-structured neural networks. Experimental results show that our method\nachieves state-of-the-art performance on PDTB corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01255v1",
            "date": "2018-03-03 22:39:02+00:00",
            "title": "Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis",
            "authors": [
                "Haoyue Shi",
                "Yuqi Sun",
                "Junfeng Hu"
            ],
            "abstract": "Unsupervised learned representations of polysemous words generate a large of\npseudo multi senses since unsupervised methods are overly sensitive to\ncontextual variations. In this paper, we address the pseudo multi-sense\ndetection for word embeddings by dimensionality reduction of sense pairs. We\npropose a novel principal analysis method, termed Ex-RPCA, designed to detect\nboth pseudo multi senses and real multi senses. With Ex-RPCA, we empirically\nshow that pseudo multi senses are generated systematically in unsupervised\nmethod. Moreover, the multi-sense word embeddings can by improved by a simple\nlinear transformation based on Ex-RPCA. Our improved word embedding outperform\nthe original one by 5.6 points on Stanford contextual word similarity (SCWS)\ndataset. We hope our simple yet effective approach will help the linguistic\nanalysis of multi-sense word embeddings in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01335v1",
            "date": "2018-03-04 11:07:55+00:00",
            "title": "CAESAR: Context Awareness Enabled Summary-Attentive Reader",
            "authors": [
                "Long-Huei Chen",
                "Kshitiz Tripathi"
            ],
            "abstract": "Comprehending meaning from natural language is a primary objective of Natural\nLanguage Processing (NLP), and text comprehension is the cornerstone for\nachieving this objective upon which all other problems like chat bots, language\ntranslation and others can be achieved. We report a Summary-Attentive Reader we\ndesigned to better emulate the human reading process, along with a\ndictiontary-based solution regarding out-of-vocabulary (OOV) words in the data,\nto generate answer based on machine comprehension of reading passages and\nquestion from the SQuAD benchmark. Our implementation of these features with\ntwo popular models (Match LSTM and Dynamic Coattention) was able to reach close\nto matching the results obtained from humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01400v2",
            "date": "2018-03-04 18:42:05+00:00",
            "title": "Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations",
            "authors": [
                "Andreas R\u00fcckl\u00e9",
                "Steffen Eger",
                "Maxime Peyrard",
                "Iryna Gurevych"
            ],
            "abstract": "Average word embeddings are a common baseline for more sophisticated sentence\nembedding techniques. However, they typically fall short of the performances of\nmore complex models such as InferSent. Here, we generalize the concept of\naverage word embeddings to power mean word embeddings. We show that the\nconcatenation of different types of power mean word embeddings considerably\ncloses the gap to state-of-the-art methods monolingually and substantially\noutperforms these more complex techniques cross-lingually. In addition, our\nproposed method outperforms different recently proposed baselines such as SIF\nand Sent2Vec by a solid margin, thus constituting a much harder-to-beat\nmonolingual baseline. Our data and code are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01557v3",
            "date": "2018-03-05 08:37:47+00:00",
            "title": "Automatic Translating Between Ancient Chinese and Contemporary Chinese with Limited Aligned Corpora",
            "authors": [
                "Zhiyuan Zhang",
                "Wei Li",
                "Qi Su"
            ],
            "abstract": "The Chinese language has evolved a lot during the long-term development.\nTherefore, native speakers now have trouble in reading sentences written in\nancient Chinese. In this paper, we propose to build an end-to-end neural model\nto automatically translate between ancient and contemporary Chinese. However,\nthe existing ancient-contemporary Chinese parallel corpora are not aligned at\nthe sentence level and sentence-aligned corpora are limited, which makes it\ndifficult to train the model. To build the sentence level parallel training\ndata for the model, we propose an unsupervised algorithm that constructs\nsentence-aligned ancient-contemporary pairs by using the fact that the aligned\nsentence pair shares many of the tokens. Based on the aligned corpus, we\npropose an end-to-end neural model with copying mechanism and local attention\nto translate between ancient and contemporary Chinese. Experiments show that\nthe proposed unsupervised algorithm achieves 99.4% F1 score for sentence\nalignment, and the translation model achieves 26.95 BLEU from ancient to\ncontemporary, and 36.34 BLEU from contemporary to ancient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01707v2",
            "date": "2018-03-05 15:09:49+00:00",
            "title": "Neural Architectures for Open-Type Relation Argument Extraction",
            "authors": [
                "Benjamin Roth",
                "Costanza Conforti",
                "Nina Poerner",
                "Sanjeev Karn",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In this work, we introduce the task of Open-Type Relation Argument Extraction\n(ORAE): Given a corpus, a query entity Q and a knowledge base relation (e.g.,\"Q\nauthored notable work with title X\"), the model has to extract an argument of\nnon-standard entity type (entities that cannot be extracted by a standard named\nentity tagger, e.g. X: the title of a book or a work of art) from the corpus. A\ndistantly supervised dataset based on WikiData relations is obtained and\nreleased to address the task.\n  We develop and compare a wide range of neural models for this task yielding\nlarge improvements over a strong baseline obtained with a neural question\nanswering system. The impact of different sentence encoding architectures and\nanswer extraction methods is systematically compared. An encoder based on gated\nrecurrent units combined with a conditional random fields tagger gives the best\nresults.",
            "journal ref": "Nat. Lang. Eng. 25 (2019) 219-238",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02155v2",
            "date": "2018-03-06 13:13:11+00:00",
            "title": "Self-Attention with Relative Position Representations",
            "authors": [
                "Peter Shaw",
                "Jakob Uszkoreit",
                "Ashish Vaswani"
            ],
            "abstract": "Relying entirely on an attention mechanism, the Transformer introduced by\nVaswani et al. (2017) achieves state-of-the-art results for machine\ntranslation. In contrast to recurrent and convolutional neural networks, it\ndoes not explicitly model relative or absolute position information in its\nstructure. Instead, it requires adding representations of absolute positions to\nits inputs. In this work we present an alternative approach, extending the\nself-attention mechanism to efficiently consider representations of the\nrelative positions, or distances between sequence elements. On the WMT 2014\nEnglish-to-German and English-to-French translation tasks, this approach yields\nimprovements of 1.3 BLEU and 0.3 BLEU over absolute position representations,\nrespectively. Notably, we observe that combining relative and absolute position\nrepresentations yields no further improvement in translation quality. We\ndescribe an efficient implementation of our method and cast it as an instance\nof relation-aware self-attention mechanisms that can generalize to arbitrary\ngraph-labeled inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02245v1",
            "date": "2018-03-06 15:17:40+00:00",
            "title": "CliNER 2.0: Accessible and Accurate Clinical Concept Extraction",
            "authors": [
                "Willie Boag",
                "Elena Sergeeva",
                "Saurabh Kulshreshtha",
                "Peter Szolovits",
                "Anna Rumshisky",
                "Tristan Naumann"
            ],
            "abstract": "Clinical notes often describe important aspects of a patient's stay and are\ntherefore critical to medical research. Clinical concept extraction (CCE) of\nnamed entities - such as problems, tests, and treatments - aids in forming an\nunderstanding of notes and provides a foundation for many downstream clinical\ndecision-making tasks. Historically, this task has been posed as a standard\nnamed entity recognition (NER) sequence tagging problem, and solved with\nfeature-based methods using handengineered domain knowledge. Recent advances,\nhowever, have demonstrated the efficacy of LSTM-based models for NER tasks,\nincluding CCE. This work presents CliNER 2.0, a simple-to-install, open-source\ntool for extracting concepts from clinical text. CliNER 2.0 uses a word- and\ncharacter- level LSTM model, and achieves state-of-the-art performance. For\nease of use, the tool also includes pre-trained models available for public\nuse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02279v2",
            "date": "2018-03-06 16:17:18+00:00",
            "title": "An End-to-End Goal-Oriented Dialog System with a Generative Natural Language Response Generation",
            "authors": [
                "Stefan Constantin",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "Recently advancements in deep learning allowed the development of end-to-end\ntrained goal-oriented dialog systems. Although these systems already achieve\ngood performance, some simplifications limit their usage in real-life\nscenarios.\n  In this work, we address two of these limitations: ignoring positional\ninformation and a fixed number of possible response candidates. We propose to\nuse positional encodings in the input to model the word order of the user\nutterances. Furthermore, by using a feedforward neural network, we are able to\ngenerate the output word by word and are no longer restricted to a fixed number\nof possible response candidates. Using the positional encoding, we were able to\nachieve better accuracies in the Dialog bAbI Tasks and using the feedforward\nneural network for generating the response, we were able to save computation\ntime and space consumption.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02392v2",
            "date": "2018-03-06 19:23:24+00:00",
            "title": "Multimodal Emoji Prediction",
            "authors": [
                "Francesco Barbieri",
                "Miguel Ballesteros",
                "Francesco Ronzano",
                "Horacio Saggion"
            ],
            "abstract": "Emojis are small images that are commonly included in social media text\nmessages. The combination of visual and textual content in the same message\nbuilds up a modern way of communication, that automatic systems are not used to\ndeal with. In this paper we extend recent advances in emoji prediction by\nputting forward a multimodal approach that is able to predict emojis in\nInstagram posts. Instagram posts are composed of pictures together with texts\nwhich sometimes include emojis. We show that these emojis can be predicted by\nusing the text, but also using the picture. Our main finding is that\nincorporating the two synergistic modalities, in a combined model, improves\naccuracy in an emoji prediction task. This result demonstrates that these two\nmodalities (text and images) encode different information on the use of emojis\nand therefore can complement each other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02914v1",
            "date": "2018-03-07 23:29:31+00:00",
            "title": "Translating Questions into Answers using DBPedia n-triples",
            "authors": [
                "Mihael Arcan"
            ],
            "abstract": "In this paper we present a question answering system using a neural network\nto interpret questions learned from the DBpedia repository. We train a\nsequence-to-sequence neural network model with n-triples extracted from the\nDBpedia Infobox Properties. Since these properties do not represent the natural\nlanguage, we further used question-answer dialogues from movie subtitles.\nAlthough the automatic evaluation shows a low overlap of the generated answers\ncompared to the gold standard set, a manual inspection of the showed promising\noutcomes from the experiment for further work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02994v1",
            "date": "2018-03-08 08:07:31+00:00",
            "title": "How Images Inspire Poems: Generating Classical Chinese Poetry from Images with Memory Networks",
            "authors": [
                "Linli Xu",
                "Liang Jiang",
                "Chuan Qin",
                "Zhe Wang",
                "Dongfang Du"
            ],
            "abstract": "With the recent advances of neural models and natural language processing,\nautomatic generation of classical Chinese poetry has drawn significant\nattention due to its artistic and cultural value. Previous works mainly focus\non generating poetry given keywords or other text information, while visual\ninspirations for poetry have been rarely explored. Generating poetry from\nimages is much more challenging than generating poetry from text, since images\ncontain very rich visual information which cannot be described completely using\nseveral keywords, and a good poem should convey the image accurately. In this\npaper, we propose a memory based neural model which exploits images to generate\npoems. Specifically, an Encoder-Decoder model with a topic memory network is\nproposed to generate classical Chinese poetry from images. To the best of our\nknowledge, this is the first work attempting to generate classical Chinese\npoetry from images with neural networks. A comprehensive experimental\ninvestigation with both human evaluation and quantitative analysis demonstrates\nthat the proposed model can generate poems which convey images accurately.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03178v1",
            "date": "2018-03-08 16:06:54+00:00",
            "title": "Fact Checking in Community Forums",
            "authors": [
                "Tsvetomila Mihaylova",
                "Preslav Nakov",
                "Lluis Marquez",
                "Alberto Barron-Cedeno",
                "Mitra Mohtarami",
                "Georgi Karadzhov",
                "James Glass"
            ],
            "abstract": "Community Question Answering (cQA) forums are very popular nowadays, as they\nrepresent effective means for communities around particular topics to share\ninformation. Unfortunately, this information is not always factual. Thus, here\nwe explore a new dimension in the context of cQA, which has been ignored so\nfar: checking the veracity of answers to particular questions in cQA forums. As\nthis is a new problem, we create a specialized dataset for it. We further\npropose a novel multi-faceted model, which captures information from the answer\ncontent (what is said and how), from the author profile (who says it), from the\nrest of the community forum (where it is said), and from external authoritative\nsources of information (external support). Evaluation results show a MAP value\nof 86.54, which is 21 points absolute above the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03378v2",
            "date": "2018-03-09 04:15:22+00:00",
            "title": "Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss",
            "authors": [
                "Peng Xu",
                "Denilson Barbosa"
            ],
            "abstract": "The task of Fine-grained Entity Type Classification (FETC) consists of\nassigning types from a hierarchy to entity mentions in text. Existing methods\nrely on distant supervision and are thus susceptible to noisy labels that can\nbe out-of-context or overly-specific for the training sentence. Previous\nmethods that attempt to address these issues do so with heuristics or with the\nhelp of hand-crafted features. Instead, we propose an end-to-end solution with\na neural network model that uses a variant of cross- entropy loss function to\nhandle out-of-context labels, and hierarchical loss normalization to cope with\noverly-specific ones. Also, previous work solve FETC a multi-label\nclassification followed by ad-hoc post-processing. In contrast, our solution is\nmore elegant: we use public word embeddings to train a single-label that\njointly learns representations for entity mentions and their context. We show\nexperimentally that our approach is robust against noise and consistently\noutperforms the state-of-the-art on established benchmarks for the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03476v1",
            "date": "2018-03-09 11:44:39+00:00",
            "title": "An Unsupervised Model with Attention Autoencoders for Question Retrieval",
            "authors": [
                "Minghua Zhang",
                "Yunfang Wu"
            ],
            "abstract": "Question retrieval is a crucial subtask for community question answering.\nPrevious research focus on supervised models which depend heavily on training\ndata and manual feature engineering. In this paper, we propose a novel\nunsupervised framework, namely reduced attentive matching network (RAMN), to\ncompute semantic matching between two questions. Our RAMN integrates together\nthe deep semantic representations, the shallow lexical mismatching information\nand the initial rank produced by an external search engine. For the first time,\nwe propose attention autoencoders to generate semantic representations of\nquestions. In addition, we employ lexical mismatching to capture surface\nmatching between two questions, which is derived from the importance of each\nword in a question. We conduct experiments on the open CQA datasets of\nSemEval-2016 and SemEval-2017. The experimental results show that our\nunsupervised model obtains comparable performance with the state-of-the-art\nsupervised methods in SemEval-2016 Task 3, and outperforms the best system in\nSemEval-2017 Task 3 by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03585v2",
            "date": "2018-03-09 16:13:02+00:00",
            "title": "The Importance of Being Recurrent for Modeling Hierarchical Structure",
            "authors": [
                "Ke Tran",
                "Arianna Bisazza",
                "Christof Monz"
            ],
            "abstract": "Recent work has shown that recurrent neural networks (RNNs) can implicitly\ncapture and exploit hierarchical information when trained to solve common\nnatural language processing tasks such as language modeling (Linzen et al.,\n2016) and neural machine translation (Shi et al., 2016). In contrast, the\nability to model structured data with non-recurrent neural networks has\nreceived little attention despite their success in many NLP tasks (Gehring et\nal., 2017; Vaswani et al., 2017). In this work, we compare the two\narchitectures---recurrent versus non-recurrent---with respect to their ability\nto model hierarchical structure and find that recurrency is indeed important\nfor this purpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03662v2",
            "date": "2018-02-27 20:49:51+00:00",
            "title": "Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter",
            "authors": [
                "Ziqi Zhang",
                "Lei Luo"
            ],
            "abstract": "In recent years, the increasing propagation of hate speech on social media\nand the urgent need for effective counter-measures have drawn significant\ninvestment from governments, companies, and researchers. A large number of\nmethods have been developed for automated hate speech detection online. This\naims to classify textual content into non-hate or hate speech, in which case\nthe method may also identify the targeting characteristics (i.e., types of\nhate, such as race, and religion) in the hate speech. However, we notice\nsignificant difference between the performance of the two (i.e., non-hate v.s.\nhate). In this work, we argue for a focus on the latter problem for practical\nreasons. We show that it is a much more challenging task, as our analysis of\nthe language in the typical datasets shows that hate speech lacks unique,\ndiscriminative features and therefore is found in the 'long tail' in a dataset\nthat is difficult to discover. We then propose Deep Neural Network structures\nserving as feature extractors that are particularly effective for capturing the\nsemantics of hate speech. Our methods are evaluated on the largest collection\nof hate speech datasets based on Twitter, and are shown to be able to\noutperform the best performing method by up to 5 percentage points in\nmacro-average F1, or 8 percentage points in the more challenging case of\nidentifying hateful content.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03670v1",
            "date": "2018-03-08 09:14:37+00:00",
            "title": "IcoRating: A Deep-Learning System for Scam ICO Identification",
            "authors": [
                "Shuqing Bian",
                "Zhenpeng Deng",
                "Fei Li",
                "Will Monroe",
                "Peng Shi",
                "Zijun Sun",
                "Wei Wu",
                "Sikuang Wang",
                "William Yang Wang",
                "Arianna Yuan",
                "Tianwei Zhang",
                "Jiwei Li"
            ],
            "abstract": "Cryptocurrencies (or digital tokens, digital currencies, e.g., BTC, ETH, XRP,\nNEO) have been rapidly gaining ground in use, value, and understanding among\nthe public, bringing astonishing profits to investors. Unlike other money and\nbanking systems, most digital tokens do not require central authorities. Being\ndecentralized poses significant challenges for credit rating. Most ICOs are\ncurrently not subject to government regulations, which makes a reliable credit\nrating system for ICO projects necessary and urgent.\n  In this paper, we introduce IcoRating, the first learning--based\ncryptocurrency rating system. We exploit natural-language processing techniques\nto analyze various aspects of 2,251 digital currencies to date, such as white\npaper content, founding teams, Github repositories, websites, etc. Supervised\nlearning models are used to correlate the life span and the price change of\ncryptocurrencies with these features. For the best setting, the proposed system\nis able to identify scam ICO projects with 0.83 precision.\n  We hope this work will help investors identify scam ICOs and attract more\nefforts in automatically evaluating and analyzing ICO projects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03786v1",
            "date": "2018-03-10 10:09:13+00:00",
            "title": "We Built a Fake News & Click-bait Filter: What Happened Next Will Blow Your Mind!",
            "authors": [
                "Georgi Karadzhov",
                "Pepa Gencheva",
                "Preslav Nakov",
                "Ivan Koychev"
            ],
            "abstract": "It is completely amazing! Fake news and click-baits have totally invaded the\ncyber space. Let us face it: everybody hates them for three simple reasons.\nReason #2 will absolutely amaze you. What these can achieve at the time of\nelection will completely blow your mind! Now, we all agree, this cannot go on,\nyou know, somebody has to stop it. So, we did this research on fake\nnews/click-bait detection and trust us, it is totally great research, it really\nis! Make no mistake. This is the best research ever! Seriously, come have a\nlook, we have it all: neural networks, attention mechanism, sentiment lexicons,\nauthor profiling, you name it. Lexical features, semantic features, we\nabsolutely have it all. And we have totally tested it, trust us! We have\nresults, and numbers, really big numbers. The best numbers ever! Oh, and\nanalysis, absolutely top notch analysis. Interested? Come read the shocking\ntruth about fake news and click-bait in the Bulgarian cyber space. You won't\nbelieve what we have found!",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03859v2",
            "date": "2018-03-10 20:38:55+00:00",
            "title": "Language Identification of Bengali-English Code-Mixed data using Character & Phonetic based LSTM Models",
            "authors": [
                "Soumil Mandal",
                "Sourya Dipta Das",
                "Dipankar Das"
            ],
            "abstract": "Language identification of social media text still remains a challenging task\ndue to properties like code-mixing and inconsistent phonetic transliterations.\nIn this paper, we present a supervised learning approach for language\nidentification at the word level of low resource Bengali-English code-mixed\ndata taken from social media. We employ two methods of word encoding, namely\ncharacter based and root phone based to train our deep LSTM models. Utilizing\nthese two models we created two ensemble models using stacking and threshold\ntechnique which gave 91.78% and 92.35% accuracies respectively on our testing\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03887v1",
            "date": "2018-03-11 02:58:40+00:00",
            "title": "Path of Vowel Raising in Chengdu Dialect of Mandarin",
            "authors": [
                "Hai Hu",
                "Yiwen Zhang"
            ],
            "abstract": "He and Rao (2013) reported a raising phenomenon of /a/ in /Xan/ (X being a\nconsonant or a vowel) in Chengdu dialect of Mandarin, i.e. /a/ is realized as\n[epsilon] for young speakers but [ae] for older speakers, but they offered no\nacoustic analysis. We designed an acoustic study that examined the realization\nof /Xan/ in speakers of different age (old vs. young) and gender (male vs.\nfemale) groups, where X represents three conditions: 1) unaspirated consonants:\nC ([p], [t], [k]), 2) aspirated consonants: Ch ([ph], [th], [kh]), and 3) high\nvowels: V ([i], [y], [u]). 17 native speakers were asked to read /Xan/\ncharacters and the F1 values are extracted for comparison. Our results\nconfirmed the raising effect in He and Rao (2013), i.e., young speakers realize\n/a/ as [epsilon] in /an/, whereas older speakers in the most part realize it as\n[ae]. Also, female speakers raise more than male speakers within the same age\ngroup. Interestingly, within the /Van/ condition, older speakers do raise /a/\nin /ian/ and /yan/. We interpret this as /a/ first assimilates to its preceding\nfront high vowels /i/ and /y/ for older speakers, which then becomes\nphonologized in younger speakers in all conditions, including /Chan/ and /Can/.\nThis shows a possible trajectory of the ongoing sound change in the Chengdu\ndialect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03917v2",
            "date": "2018-03-11 07:05:50+00:00",
            "title": "Generating Bilingual Pragmatic Color References",
            "authors": [
                "Will Monroe",
                "Jennifer Hu",
                "Andrew Jong",
                "Christopher Potts"
            ],
            "abstract": "Contextual influences on language often exhibit substantial cross-lingual\nregularities; for example, we are more verbose in situations that require finer\ndistinctions. However, these regularities are sometimes obscured by semantic\nand syntactic differences. Using a newly-collected dataset of color reference\ngames in Mandarin Chinese (which we release to the public), we confirm that a\nvariety of constructions display the same sensitivity to contextual difficulty\nin Chinese and English. We then show that a neural speaker agent trained on\nbilingual data with a simple multitask learning approach displays more\nhuman-like patterns of context dependence and is more pragmatically informative\nthan its monolingual Chinese counterpart. Moreover, this is not at the expense\nof language-specific semantic understanding: the resulting speaker model learns\nthe different basic color term systems of English and Chinese (with noteworthy\ncross-lingual influences), and it can identify synonyms between the two\nlanguages using vector analogy operations on its output layer, despite having\nno exposure to parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04000v1",
            "date": "2018-03-11 18:13:01+00:00",
            "title": "Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages",
            "authors": [
                "Soumil Mandal",
                "Sainik Kumar Mahata",
                "Dipankar Das"
            ],
            "abstract": "Analysis of informative contents and sentiments of social users has been\nattempted quite intensively in the recent past. Most of the systems are usable\nonly for monolingual data and fails or gives poor results when used on data\nwith code-mixing property. To gather attention and encourage researchers to\nwork on this crisis, we prepared gold standard Bengali-English code-mixed data\nwith language and polarity tag for sentiment analysis purposes. In this paper,\nwe discuss the systems we prepared to collect and filter raw Twitter data. In\norder to reduce manual work while annotation, hybrid systems combining rule\nbased and supervised models were developed for both language and sentiment\ntagging. The final corpus was annotated by a group of annotators following a\nfew guidelines. The gold standard corpus thus obtained has impressive\ninter-annotator agreement obtained in terms of Kappa values. Various metrics\nlike Code-Mixed Index (CMI), Code-Mixed Factor (CF) along with various aspects\n(language and emotion) also qualitatively polled the code-mixed and sentiment\nproperties of the corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04291v2",
            "date": "2018-03-12 14:47:43+00:00",
            "title": "Entity-Aware Language Model as an Unsupervised Reranker",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Sarangarajan Parthasarathy"
            ],
            "abstract": "In language modeling, it is difficult to incorporate entity relationships\nfrom a knowledge-base. One solution is to use a reranker trained with global\nfeatures, in which global features are derived from n-best lists. However,\ntraining such a reranker requires manually annotated n-best lists, which is\nexpensive to obtain. We propose a method based on the contrastive estimation\nmethod that alleviates the need for such data. Experiments in the music domain\ndemonstrate that global features, as well as features extracted from an\nexternal knowledge-base, can be incorporated into our reranker. Our final\nmodel, a simple ensemble of a language model and reranker, achieves a 0.44\\%\nabsolute word error rate improvement over an LSTM language model on the blind\ntest data.",
            "journal ref": "Interspeech 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04329v1",
            "date": "2018-03-12 15:59:10+00:00",
            "title": "Semantic Parsing Natural Language into SPARQL: Improving Target Language Representation with Neural Attention",
            "authors": [
                "Fabiano Ferreira Luz",
                "Marcelo Finger"
            ],
            "abstract": "Semantic parsing is the process of mapping a natural language sentence into a\nformal representation of its meaning. In this work we use the neural network\napproach to transform natural language sentence into a query to an ontology\ndatabase in the SPARQL language. This method does not rely on handcraft-rules,\nhigh-quality lexicons, manually-built templates or other handmade complex\nstructures. Our approach is based on vector space model and neural networks.\nThe proposed model is based in two learning steps. The first step generates a\nvector representation for the sentence in natural language and SPARQL query.\nThe second step uses this vector representation as input to a neural network\n(LSTM with attention mechanism) to generate a model able to encode natural\nlanguage and decode SPARQL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04375v1",
            "date": "2018-03-12 17:07:40+00:00",
            "title": "A Feature-Rich Vietnamese Named-Entity Recognition Model",
            "authors": [
                "Pham Quang Nhat Minh"
            ],
            "abstract": "In this paper, we present a feature-based named-entity recognition (NER)\nmodel that achieves the start-of-the-art accuracy for Vietnamese language. We\ncombine word, word-shape features, PoS, chunk, Brown-cluster-based features,\nand word-embedding-based features in the Conditional Random Fields (CRF) model.\nWe also explore the effects of word segmentation, PoS tagging, and chunking\nresults of many popular Vietnamese NLP toolkits on the accuracy of the proposed\nfeature-based NER model. Up to now, our work is the first work that\nsystematically performs an extrinsic evaluation of basic Vietnamese NLP\ntoolkits on the downstream NER task. Experimental results show that while\nautomatically-generated word segmentation is useful, PoS and chunking\ninformation generated by Vietnamese NLP tools does not show their benefits for\nthe proposed feature-based NER model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04757v1",
            "date": "2018-03-13 12:56:27+00:00",
            "title": "Monitoring Targeted Hate in Online Environments",
            "authors": [
                "Tim Isbister",
                "Magnus Sahlgren",
                "Lisa Kaati",
                "Milan Obaidi",
                "Nazar Akrami"
            ],
            "abstract": "Hateful comments, swearwords and sometimes even death threats are becoming a\nreality for many people today in online environments. This is especially true\nfor journalists, politicians, artists, and other public figures. This paper\ndescribes how hate directed towards individuals can be measured in online\nenvironments using a simple dictionary-based approach. We present a case study\non Swedish politicians, and use examples from this study to discuss\nshortcomings of the proposed dictionary-based approach. We also outline\npossibilities for potential refinements of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04790v2",
            "date": "2018-03-13 13:33:06+00:00",
            "title": "Enhanced Word Representations for Bridging Anaphora Resolution",
            "authors": [
                "Yufang Hou"
            ],
            "abstract": "Most current models of word representations(e.g.,GloVe) have successfully\ncaptured fine-grained semantics. However, semantic similarity exhibited in\nthese word embeddings is not suitable for resolving bridging anaphora, which\nrequires the knowledge of associative similarity (i.e., relatedness) instead of\nsemantic similarity information between synonyms or hypernyms. We create word\nembeddings (embeddings_PP) to capture such relatedness by exploring the\nsyntactic structure of noun phrases. We demonstrate that using embeddings_PP\nalone achieves around 30% of accuracy for bridging anaphora resolution on the\nISNotes corpus. Furthermore, we achieve a substantial gain over the\nstate-of-the-art system (Hou et al., 2013) for bridging antecedent selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05071v1",
            "date": "2018-03-13 23:13:58+00:00",
            "title": "Neural Lattice Language Models",
            "authors": [
                "Jacob Buckman",
                "Graham Neubig"
            ],
            "abstract": "In this work, we propose a new language modeling paradigm that has the\nability to perform both prediction and moderation of information flow at\nmultiple granularities: neural lattice language models. These models construct\na lattice of possible paths through a sentence and marginalize across this\nlattice to calculate sequence probabilities or optimize parameters. This\napproach allows us to seamlessly incorporate linguistic intuitions - including\npolysemy and existence of multi-word lexical items - into our language model.\nExperiments on multiple language modeling tasks show that English neural\nlattice language models that utilize polysemous embeddings are able to improve\nperplexity by 9.95% relative to a word-level baseline, and that a Chinese model\nthat handles multi-character tokens is able to improve perplexity by 20.94%\nrelative to a character-level baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05223v1",
            "date": "2018-03-14 11:59:13+00:00",
            "title": "MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge",
            "authors": [
                "Simon Ostermann",
                "Ashutosh Modi",
                "Michael Roth",
                "Stefan Thater",
                "Manfred Pinkal"
            ],
            "abstract": "We introduce a large dataset of narrative texts and questions about these\ntexts, intended to be used in a machine comprehension task that requires\nreasoning using commonsense knowledge. Our dataset complements similar datasets\nin that we focus on stories about everyday activities, such as going to the\nmovies or working in the garden, and that the questions require commonsense\nknowledge, or more specifically, script knowledge, to be answered. We show that\nour mode of data collection via crowdsourcing results in a substantial amount\nof such inference questions. The dataset forms the basis of a shared task on\ncommonsense and script knowledge organized at SemEval 2018 and provides\nchallenging test cases for the broader natural language understanding\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05355v3",
            "date": "2018-03-14 15:30:37+00:00",
            "title": "FEVER: a large-scale dataset for Fact Extraction and VERification",
            "authors": [
                "James Thorne",
                "Andreas Vlachos",
                "Christos Christodoulopoulos",
                "Arpit Mittal"
            ],
            "abstract": "In this paper we introduce a new publicly available dataset for verification\nagainst textual sources, FEVER: Fact Extraction and VERification. It consists\nof 185,445 claims generated by altering sentences extracted from Wikipedia and\nsubsequently verified without knowledge of the sentence they were derived from.\nThe claims are classified as Supported, Refuted or NotEnoughInfo by annotators\nachieving 0.6841 in Fleiss $\\kappa$. For the first two classes, the annotators\nalso recorded the sentence(s) forming the necessary evidence for their\njudgment. To characterize the challenge of the dataset presented, we develop a\npipeline approach and compare it to suitably designed oracles. The best\naccuracy we achieve on labeling a claim accompanied by the correct evidence is\n31.87%, while if we ignore the evidence we achieve 50.91%. Thus we believe that\nFEVER is a challenging testbed that will help stimulate progress on claim\nverification against textual sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05449v1",
            "date": "2018-03-14 18:01:15+00:00",
            "title": "SentEval: An Evaluation Toolkit for Universal Sentence Representations",
            "authors": [
                "Alexis Conneau",
                "Douwe Kiela"
            ],
            "abstract": "We introduce SentEval, a toolkit for evaluating the quality of universal\nsentence representations. SentEval encompasses a variety of tasks, including\nbinary and multi-class classification, natural language inference and sentence\nsimilarity. The set of tasks was selected based on what appears to be the\ncommunity consensus regarding the appropriate evaluations for universal\nsentence representations. The toolkit comes with scripts to download and\npreprocess datasets, and an easy interface to evaluate sentence encoders. The\naim is to provide a fairer, less cumbersome and more centralized way for\nevaluating sentence representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05495v1",
            "date": "2018-03-14 20:00:08+00:00",
            "title": "Challenges in Discriminating Profanity from Hate Speech",
            "authors": [
                "Shervin Malmasi",
                "Marcos Zampieri"
            ],
            "abstract": "In this study we approach the problem of distinguishing general profanity\nfrom hate speech in social media, something which has not been widely\nconsidered. Using a new dataset annotated specifically for this task, we employ\nsupervised classification along with a set of features that includes n-grams,\nskip-grams and clustering-based word representations. We apply approaches based\non single classifiers as well as more advanced ensemble classifiers and stacked\ngeneralization, achieving the best result of 80% accuracy for this 3-class\nclassification task. Analysis of the results reveals that discriminating hate\nspeech and profanity is not a simple task, which may require features that\ncapture a deeper understanding of the text not always possible with surface\nn-grams. The variability of gold labels in the annotated data, due to\ndifferences in the subjective adjudications of the annotators, is also an\nissue. Other directions for future work are discussed.",
            "journal ref": "Shervin Malmasi, Marcos Zampieri (2018) Challenges in\n  Discriminating Profanity from Hate Speech. Journal of Experimental &\n  Theoretical Artificial Intelligence. Volume 30, Issue 2, pp. 187-202. Taylor\n  & Francis",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05547v1",
            "date": "2018-03-15 00:16:43+00:00",
            "title": "A Simple and Effective Approach to the Story Cloze Test",
            "authors": [
                "Siddarth Srinivasan",
                "Richa Arora",
                "Mark Riedl"
            ],
            "abstract": "In the Story Cloze Test, a system is presented with a 4-sentence prompt to a\nstory, and must determine which one of two potential endings is the 'right'\nending to the story. Previous work has shown that ignoring the training set and\ntraining a model on the validation set can achieve high accuracy on this task\ndue to stylistic differences between the story endings in the training set and\nvalidation and test sets. Following this approach, we present a simpler\nfully-neural approach to the Story Cloze Test using skip-thought embeddings of\nthe stories in a feed-forward network that achieves close to state-of-the-art\nperformance on this task without any feature engineering. We also find that\nconsidering just the last sentence of the prompt instead of the whole prompt\nyields higher accuracy with our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05563v1",
            "date": "2018-03-15 01:19:21+00:00",
            "title": "Advancing Connectionist Temporal Classification With Attention Modeling",
            "authors": [
                "Amit Das",
                "Jinyu Li",
                "Rui Zhao",
                "Yifan Gong"
            ],
            "abstract": "In this study, we propose advancing all-neural speech recognition by directly\nincorporating attention modeling within the Connectionist Temporal\nClassification (CTC) framework. In particular, we derive new context vectors\nusing time convolution features to model attention as part of the CTC network.\nTo further improve attention modeling, we utilize content information extracted\nfrom a network representing an implicit language model. Finally, we introduce\nvector based attention weights that are applied on context vectors across both\ntime and their individual components. We evaluate our system on a 3400 hours\nMicrosoft Cortana voice assistant task and demonstrate that our proposed model\nconsistently outperforms the baseline model achieving about 20% relative\nreduction in word error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05566v1",
            "date": "2018-03-15 01:25:17+00:00",
            "title": "Advancing Acoustic-to-Word CTC Model",
            "authors": [
                "Jinyu Li",
                "Guoli Ye",
                "Amit Das",
                "Rui Zhao",
                "Yifan Gong"
            ],
            "abstract": "The acoustic-to-word model based on the connectionist temporal classification\n(CTC) criterion was shown as a natural end-to-end (E2E) model directly\ntargeting words as output units. However, the word-based CTC model suffers from\nthe out-of-vocabulary (OOV) issue as it can only model limited number of words\nin the output layer and maps all the remaining words into an OOV output node.\nHence, such a word-based CTC model can only recognize the frequent words\nmodeled by the network output nodes. Our first attempt to improve the\nacoustic-to-word model is a hybrid CTC model which consults a letter-based CTC\nwhen the word-based CTC model emits OOV tokens during testing time. Then, we\npropose a much better solution by training a mixed-unit CTC model which\ndecomposes all the OOV words into sequences of frequent words and multi-letter\nunits. Evaluated on a 3400 hours Microsoft Cortana voice assistant task, the\nfinal acoustic-to-word solution improves the baseline word-based CTC by\nrelative 12.09% word error rate (WER) reduction when combined with our proposed\nattention CTC. Such an E2E model without using any language model (LM) or\ncomplex decoder outperforms the traditional context-dependent phoneme CTC which\nhas strong LM and decoder by relative 6.79%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05567v2",
            "date": "2018-03-15 01:30:58+00:00",
            "title": "Achieving Human Parity on Automatic Chinese to English News Translation",
            "authors": [
                "Hany Hassan",
                "Anthony Aue",
                "Chang Chen",
                "Vishal Chowdhary",
                "Jonathan Clark",
                "Christian Federmann",
                "Xuedong Huang",
                "Marcin Junczys-Dowmunt",
                "William Lewis",
                "Mu Li",
                "Shujie Liu",
                "Tie-Yan Liu",
                "Renqian Luo",
                "Arul Menezes",
                "Tao Qin",
                "Frank Seide",
                "Xu Tan",
                "Fei Tian",
                "Lijun Wu",
                "Shuangzhi Wu",
                "Yingce Xia",
                "Dongdong Zhang",
                "Zhirui Zhang",
                "Ming Zhou"
            ],
            "abstract": "Machine translation has made rapid advances in recent years. Millions of\npeople are using it today in online translation systems and mobile applications\nin order to communicate across language barriers. The question naturally arises\nwhether such systems can approach or achieve parity with human translations. In\nthis paper, we first address the problem of how to define and accurately\nmeasure human parity in translation. We then describe Microsoft's machine\ntranslation system and measure the quality of its translations on the widely\nused WMT 2017 news translation task from Chinese to English. We find that our\nlatest neural machine translation system has reached a new state-of-the-art,\nand that the translation quality is at human parity when compared to\nprofessional human translations. We also find that it significantly exceeds the\nquality of crowd-sourced non-professional translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05651v3",
            "date": "2018-03-15 09:21:34+00:00",
            "title": "Word2Bits - Quantized Word Vectors",
            "authors": [
                "Maximilian Lam"
            ],
            "abstract": "Word vectors require significant amounts of memory and storage, posing issues\nto resource limited devices like mobile phones and GPUs. We show that high\nquality quantized word vectors using 1-2 bits per parameter can be learned by\nintroducing a quantization function into Word2Vec. We furthermore show that\ntraining with the quantization function acts as a regularizer. We train word\nvectors on English Wikipedia (2017) and evaluate them on standard word\nsimilarity and analogy tasks and on question answering (SQuAD). Our quantized\nword vectors not only take 8-16x less space than full precision (32 bit) word\nvectors but also outperform them on word similarity tasks and question\nanswering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05655v1",
            "date": "2018-03-15 09:30:12+00:00",
            "title": "HFL-RC System at SemEval-2018 Task 11: Hybrid Multi-Aspects Model for Commonsense Reading Comprehension",
            "authors": [
                "Zhipeng Chen",
                "Yiming Cui",
                "Wentao Ma",
                "Shijin Wang",
                "Ting Liu",
                "Guoping Hu"
            ],
            "abstract": "This paper describes the system which got the state-of-the-art results at\nSemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. In\nthis paper, we present a neural network called Hybrid Multi-Aspects (HMA)\nmodel, which mimic the human's intuitions on dealing with the multiple-choice\nreading comprehension. In this model, we aim to produce the predictions in\nmultiple aspects by calculating attention among the text, question and choices,\nand combine these results for final predictions. Experimental results show that\nour HMA model could give substantial improvements over the baseline system and\ngot the first place on the final test set leaderboard with the accuracy of\n84.13%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05662v1",
            "date": "2018-03-15 09:45:58+00:00",
            "title": "Structure Regularized Neural Network for Entity Relation Classification for Chinese Literature Text",
            "authors": [
                "Ji Wen",
                "Xu Sun",
                "Xuancheng Ren",
                "Qi Su"
            ],
            "abstract": "Relation classification is an important semantic processing task in the field\nof natural language processing. In this paper, we propose the task of relation\nclassification for Chinese literature text. A new dataset of Chinese literature\ntext is constructed to facilitate the study in this task. We present a novel\nmodel, named Structure Regularized Bidirectional Recurrent Convolutional Neural\nNetwork (SR-BRCNN), to identify the relation between entities. The proposed\nmodel learns relation representations along the shortest dependency path (SDP)\nextracted from the structure regularized dependency tree, which has the\nbenefits of reducing the complexity of the whole model. Experimental results\nshow that the proposed method significantly improves the F1 score by 10.3, and\noutperforms the state-of-the-art approaches on Chinese literature text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05795v3",
            "date": "2018-03-15 15:08:36+00:00",
            "title": "RUSSE'2018: A Shared Task on Word Sense Induction for the Russian Language",
            "authors": [
                "Alexander Panchenko",
                "Anastasiya Lopukhina",
                "Dmitry Ustalov",
                "Konstantin Lopukhin",
                "Nikolay Arefyev",
                "Alexey Leontyev",
                "Natalia Loukachevitch"
            ],
            "abstract": "The paper describes the results of the first shared task on word sense\ninduction (WSI) for the Russian language. While similar shared tasks were\nconducted in the past for some Romance and Germanic languages, we explore the\nperformance of sense induction and disambiguation methods for a Slavic language\nthat shares many features with other Slavic languages, such as rich morphology\nand virtually free word order. The participants were asked to group contexts of\na given word in accordance with its senses that were not provided beforehand.\nFor instance, given a word \"bank\" and a set of contexts for this word, e.g.\n\"bank is a financial institution that accepts deposits\" and \"river bank is a\nslope beside a body of water\", a participant was asked to cluster such contexts\nin the unknown in advance number of clusters corresponding to, in this case,\nthe \"company\" and the \"area\" senses of the word \"bank\". For the purpose of this\nevaluation campaign, we developed three new evaluation datasets based on sense\ninventories that have different sense granularity. The contexts in these\ndatasets were sampled from texts of Wikipedia, the academic corpus of Russian,\nand an explanatory dictionary of Russian. Overall, 18 teams participated in the\ncompetition submitting 383 models. Multiple teams managed to substantially\noutperform competitive state-of-the-art baselines from the previous years based\non sense embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05820v1",
            "date": "2018-03-15 15:50:58+00:00",
            "title": "RUSSE: The First Workshop on Russian Semantic Similarity",
            "authors": [
                "Alexander Panchenko",
                "Natalia Loukachevitch",
                "Dmitry Ustalov",
                "Denis Paperno",
                "Christian Meyer",
                "Natalia Konstantinova"
            ],
            "abstract": "The paper gives an overview of the Russian Semantic Similarity Evaluation\n(RUSSE) shared task held in conjunction with the Dialogue 2015 conference.\nThere exist a lot of comparative studies on semantic similarity, yet no\nanalysis of such measures was ever performed for the Russian language.\nExploring this problem for the Russian language is even more interesting,\nbecause this language has features, such as rich morphology and free word\norder, which make it significantly different from English, German, and other\nwell-studied languages. We attempt to bridge this gap by proposing a shared\ntask on the semantic similarity of Russian nouns. Our key contribution is an\nevaluation methodology based on four novel benchmark datasets for the Russian\nlanguage. Our analysis of the 105 submissions from 19 teams reveals that\nsuccessful approaches for English, such as distributional and skip-gram models,\nare directly applicable to Russian as well. On the one hand, the best results\nin the contest were obtained by sophisticated supervised models that combine\nevidence from different sources. On the other hand, completely unsupervised\napproaches, such as a skip-gram model estimated on a large-scale corpus, were\nable score among the top 5 systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05829v1",
            "date": "2018-03-15 16:06:33+00:00",
            "title": "Enriching Frame Representations with Distributionally Induced Senses",
            "authors": [
                "Stefano Faralli",
                "Alexander Panchenko",
                "Chris Biemann",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "We introduce a new lexical resource that enriches the Framester knowledge\ngraph, which links Framnet, WordNet, VerbNet and other resources, with semantic\nfeatures from text corpora. These features are extracted from distributionally\ninduced sense inventories and subsequently linked to the manually-constructed\nframe representations to boost the performance of frame disambiguation in\ncontext. Since Framester is a frame-based knowledge graph, which enables\nfull-fledged OWL querying and reasoning, our resource paves the way for the\ndevelopment of novel, deeper semantic-aware applications that could benefit\nfrom the combination of knowledge from text and complex symbolic\nrepresentations of events and participants. Together with the resource we also\nprovide the software we developed for the evaluation in the task of Word Frame\nDisambiguation (WFD).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05928v1",
            "date": "2018-03-15 18:10:45+00:00",
            "title": "RankME: Reliable Human Ratings for Natural Language Generation",
            "authors": [
                "Jekaterina Novikova",
                "Ond\u0159ej Du\u0161ek",
                "Verena Rieser"
            ],
            "abstract": "Human evaluation for natural language generation (NLG) often suffers from\ninconsistent user ratings. While previous research tends to attribute this\nproblem to individual user preferences, we show that the quality of human\njudgements can also be improved by experimental design. We present a novel\nrank-based magnitude estimation method (RankME), which combines the use of\ncontinuous scales and relative assessments. We show that RankME significantly\nimproves the reliability and consistency of human ratings compared to\ntraditional evaluation methods. In addition, we show that it is possible to\nevaluate NLG systems according to multiple, distinct criteria, which is\nimportant for error analysis. Finally, we demonstrate that RankME, in\ncombination with Bayesian estimation of system quality, is a cost-effective\nalternative for ranking multiple NLG systems.",
            "journal ref": "Proceedings of NAACL-HLT 2018, pages 72-78, New Orleans,\n  Louisiana, June 1-6, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06397v6",
            "date": "2018-03-16 21:05:13+00:00",
            "title": "Deep learning for affective computing: text-based emotion recognition in decision support",
            "authors": [
                "Bernhard Kratzwald",
                "Suzana Ilic",
                "Mathias Kraus",
                "Stefan Feuerriegel",
                "Helmut Prendinger"
            ],
            "abstract": "Emotions widely affect human decision-making. This fact is taken into account\nby affective computing with the goal of tailoring decision support to the\nemotional states of individuals. However, the accurate recognition of emotions\nwithin narrative documents presents a challenging undertaking due to the\ncomplexity and ambiguity of language. Performance improvements can be achieved\nthrough deep learning; yet, as demonstrated in this paper, the specific nature\nof this task requires the customization of recurrent neural networks with\nregard to bidirectional processing, dropout layers as a means of\nregularization, and weighted loss functions. In addition, we propose\nsent2affect, a tailored form of transfer learning for affective computing: here\nthe network is pre-trained for a different task (i.e. sentiment analysis),\nwhile the output layer is subsequently tuned to the task of emotion\nrecognition. The resulting performance is evaluated in a holistic setting\nacross 6 benchmark datasets, where we find that both recurrent neural networks\nand transfer learning consistently outperform traditional machine learning.\nAltogether, the findings have considerable implications for the use of\naffective computing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06456v1",
            "date": "2018-03-17 04:11:22+00:00",
            "title": "Experiments with Neural Networks for Small and Large Scale Authorship Verification",
            "authors": [
                "Marjan Hosseinia",
                "Arjun Mukherjee"
            ],
            "abstract": "We propose two models for a special case of authorship verification problem.\nThe task is to investigate whether the two documents of a given pair are\nwritten by the same author. We consider the authorship verification problem for\nboth small and large scale datasets. The underlying small-scale problem has two\nmain challenges: First, the authors of the documents are unknown to us because\nno previous writing samples are available. Second, the two documents are short\n(a few hundred to a few thousand words) and may differ considerably in the\ngenre and/or topic. To solve it we propose transformation encoder to transform\none document of the pair into the other. This document transformation generates\na loss which is used as a recognizable feature to verify if the authors of the\npair are identical. For the large scale problem where various authors are\nengaged and more examples are available with larger length, a parallel\nrecurrent neural network is proposed. It compares the language models of the\ntwo documents. We evaluate our methods on various types of datasets including\nAuthorship Identification datasets of PAN competition, Amazon reviews, and\nmachine learning articles. Experiments show that both methods achieve stable\nand competitive performance compared to the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06535v2",
            "date": "2018-03-17 16:35:04+00:00",
            "title": "Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer",
            "authors": [
                "Sudha Rao",
                "Joel Tetreault"
            ],
            "abstract": "Style transfer is the task of automatically transforming a piece of text in\none particular style into another. A major barrier to progress in this field\nhas been a lack of training and evaluation datasets, as well as benchmarks and\nautomatic metrics. In this work, we create the largest corpus for a particular\nstylistic transfer (formality) and show that techniques from the machine\ntranslation community can serve as strong baselines for future work. We also\ndiscuss challenges of using automatic metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06745v1",
            "date": "2018-03-18 21:32:07+00:00",
            "title": "Sentiment Analysis of Code-Mixed Indian Languages: An Overview of SAIL_Code-Mixed Shared Task @ICON-2017",
            "authors": [
                "Braja Gopal Patra",
                "Dipankar Das",
                "Amitava Das"
            ],
            "abstract": "Sentiment analysis is essential in many real-world applications such as\nstance detection, review analysis, recommendation system, and so on. Sentiment\nanalysis becomes more difficult when the data is noisy and collected from\nsocial media. India is a multilingual country; people use more than one\nlanguages to communicate within themselves. The switching in between the\nlanguages is called code-switching or code-mixing, depending upon the type of\nmixing. This paper presents overview of the shared task on sentiment analysis\nof code-mixed data pairs of Hindi-English and Bengali-English collected from\nthe different social media platform. The paper describes the task, dataset,\nevaluation, baseline and participant's systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06805v2",
            "date": "2018-03-19 05:13:09+00:00",
            "title": "Acoustic feature learning using cross-domain articulatory measurements",
            "authors": [
                "Qingming Tang",
                "Weiran Wang",
                "Karen Livescu"
            ],
            "abstract": "Previous work has shown that it is possible to improve speech recognition by\nlearning acoustic features from paired acoustic-articulatory data, for example\nby using canonical correlation analysis (CCA) or its deep extensions. One\nlimitation of this prior work is that the learned feature models are difficult\nto port to new datasets or domains, and articulatory data is not available for\nmost speech corpora. In this work we study the problem of acoustic feature\nlearning in the setting where we have access to an external, domain-mismatched\ndataset of paired speech and articulatory measurements, either with or without\nlabels. We develop methods for acoustic feature learning in these settings,\nbased on deep variational CCA and extensions that use both source and target\ndomain data and labels. Using this approach, we improve phonetic recognition\naccuracies on both TIMIT and Wall Street Journal and analyze a number of design\nchoices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06966v2",
            "date": "2018-03-19 14:55:12+00:00",
            "title": "Polyglot Semantic Parsing in APIs",
            "authors": [
                "Kyle Richardson",
                "Jonathan Berant",
                "Jonas Kuhn"
            ],
            "abstract": "Traditional approaches to semantic parsing (SP) work by training individual\nmodels for each available parallel dataset of text-meaning pairs. In this\npaper, we explore the idea of polyglot semantic translation, or learning\nsemantic parsing models that are trained on multiple datasets and natural\nlanguages. In particular, we focus on translating text to code signature\nrepresentations using the software component datasets of Richardson and Kuhn\n(2017a,b). The advantage of such models is that they can be used for parsing a\nwide variety of input natural languages and output programming languages, or\nmixed input languages, using a single unified model. To facilitate modeling of\nthis type, we develop a novel graph-based decoding framework that achieves\nstate-of-the-art performance on the above datasets, and apply this method to\ntwo other benchmark SP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07038v2",
            "date": "2018-03-19 17:02:23+00:00",
            "title": "Controlling Decoding for More Abstractive Summaries with Copy-Based Networks",
            "authors": [
                "Noah Weber",
                "Leena Shekhar",
                "Niranjan Balasubramanian",
                "Kyunghyun Cho"
            ],
            "abstract": "Attention-based neural abstractive summarization systems equipped with copy\nmechanisms have shown promising results. Despite this success, it has been\nnoticed that such a system generates a summary by mostly, if not entirely,\ncopying over phrases, sentences, and sometimes multiple consecutive sentences\nfrom an input paragraph, effectively performing extractive summarization. In\nthis paper, we verify this behavior using the latest neural abstractive\nsummarization system - a pointer-generator network. We propose a simple\nbaseline method that allows us to control the amount of copying without\nretraining. Experiments indicate that the method provides a strong baseline for\nabstractive systems looking to obtain high ROUGE scores while minimizing\noverlap with the source article, substantially reducing the n-gram overlap with\nthe original article while keeping within 2 points of the original model's\nROUGE score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07116v2",
            "date": "2018-03-19 18:53:17+00:00",
            "title": "Learning to Generate Wikipedia Summaries for Underserved Languages from Wikidata",
            "authors": [
                "Lucie-Aim\u00e9e Kaffee",
                "Hady Elsahar",
                "Pavlos Vougiouklis",
                "Christophe Gravier",
                "Fr\u00e9d\u00e9rique Laforest",
                "Jonathon Hare",
                "Elena Simperl"
            ],
            "abstract": "While Wikipedia exists in 287 languages, its content is unevenly distributed\namong them. In this work, we investigate the generation of open domain\nWikipedia summaries in underserved languages using structured data from\nWikidata. To this end, we propose a neural network architecture equipped with\ncopy actions that learns to generate single-sentence and comprehensible textual\nsummaries from Wikidata triples. We demonstrate the effectiveness of the\nproposed approach by evaluating it against a set of baselines on two languages\nof different natures: Arabic, a morphological rich language with a larger\nvocabulary than English, and Esperanto, a constructed language known for its\neasy acquisition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07136v1",
            "date": "2018-03-19 19:45:38+00:00",
            "title": "Dynamic Natural Language Processing with Recurrence Quantification Analysis",
            "authors": [
                "Rick Dale",
                "Nicholas D. Duran",
                "Moreno Coco"
            ],
            "abstract": "Writing and reading are dynamic processes. As an author composes a text, a\nsequence of words is produced. This sequence is one that, the author hopes,\ncauses a revisitation of certain thoughts and ideas in others. These processes\nof composition and revisitation by readers are ordered in time. This means that\ntext itself can be investigated under the lens of dynamical systems. A common\ntechnique for analyzing the behavior of dynamical systems, known as recurrence\nquantification analysis (RQA), can be used as a method for analyzing sequential\nstructure of text. RQA treats text as a sequential measurement, much like a\ntime series, and can thus be seen as a kind of dynamic natural language\nprocessing (NLP). The extension has several benefits. Because it is part of a\nsuite of time series analysis tools, many measures can be extracted in one\ncommon framework. Secondly, the measures have a close relationship with some\ncommonly used measures from natural language processing. Finally, using\nrecurrence analysis offers an opportunity expand analysis of text by developing\ntheoretical descriptions derived from complex dynamic systems. We showcase an\nexample analysis on 8,000 texts from the Gutenberg Project, compare it to\nwell-known NLP approaches, and describe an R package (crqanlp) that can be used\nin conjunction with R library crqa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07204v1",
            "date": "2018-03-20 00:44:18+00:00",
            "title": "Why not be Versatile? Applications of the SGNMT Decoder for Machine Translation",
            "authors": [
                "Felix Stahlberg",
                "Danielle Saunders",
                "Gonzalo Iglesias",
                "Bill Byrne"
            ],
            "abstract": "SGNMT is a decoding platform for machine translation which allows paring\nvarious modern neural models of translation with different kinds of constraints\nand symbolic models. In this paper, we describe three use cases in which SGNMT\nis currently playing an active role: (1) teaching as SGNMT is being used for\ncourse work and student theses in the MPhil in Machine Learning, Speech and\nLanguage Technology at the University of Cambridge, (2) research as most of the\nresearch work of the Cambridge MT group is based on SGNMT, and (3) technology\ntransfer as we show how SGNMT is helping to transfer research findings from the\nlaboratory to the industry, eg. into a product of SDL plc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07274v1",
            "date": "2018-03-20 06:59:27+00:00",
            "title": "eSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing",
            "authors": [
                "Matteo Negri",
                "Marco Turchi",
                "Rajen Chatterjee",
                "Nicola Bertoldi"
            ],
            "abstract": "Training models for the automatic correction of machine-translated text\nusually relies on data consisting of (source, MT, human post- edit) triplets\nproviding, for each source sentence, examples of translation errors with the\ncorresponding corrections made by a human post-editor. Ideally, a large amount\nof data of this kind should allow the model to learn reliable correction\npatterns and effectively apply them at test stage on unseen (source, MT) pairs.\nIn practice, however, their limited availability calls for solutions that also\nintegrate in the training process other sources of knowledge. Along this\ndirection, state-of-the-art results have been recently achieved by systems\nthat, in addition to a limited amount of available training data, exploit\nartificial corpora that approximate elements of the \"gold\" training instances\nwith automatic translations. Following this idea, we present eSCAPE, the\nlargest freely-available Synthetic Corpus for Automatic Post-Editing released\nso far. eSCAPE consists of millions of entries in which the MT element of the\ntraining triplets has been obtained by translating the source side of\npublicly-available parallel corpora, and using the target side as an artificial\nhuman post-edit. Translations are obtained both with phrase-based and neural\nmodels. For each MT paradigm, eSCAPE contains 7.2 million triplets for\nEnglish-German and 3.3 millions for English-Italian, resulting in a total of\n14,4 and 6,6 million instances respectively. The usefulness of eSCAPE is proved\nthrough experiments in a general-domain scenario, the most challenging one for\nautomatic post-editing. For both language directions, the models trained on our\nartificial data always improve MT quality with statistically significant gains.\nThe current version of eSCAPE can be freely downloaded from:\nhttp://hltshare.fbk.eu/QT21/eSCAPE.html.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07295v1",
            "date": "2018-03-20 08:35:16+00:00",
            "title": "Expressivity in TTS from Semantics and Pragmatics",
            "authors": [
                "Rodolfo Delmonte"
            ],
            "abstract": "In this paper we present ongoing work to produce an expressive TTS reader\nthat can be used both in text and dialogue applications. The system called\nSPARSAR has been used to read (English) poetry so far but it can now be applied\nto any text. The text is fully analyzed both at phonetic and phonological\nlevel, and at syntactic and semantic level. In addition, the system has access\nto a restricted list of typical pragmatically marked phrases and expressions\nthat are used to convey specific discourse function and speech acts and need\nspecialized intonational contours. The text is transformed into a poem-like\nstructures, where each line corresponds to a Breath Group, semantically and\nsyntactically consistent. Stanzas correspond to paragraph boundaries.\nAnalogical parameters are related to ToBI theoretical indices but their number\nis doubled. In this paper, we concentrate on short stories and fables.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07602v4",
            "date": "2018-03-20 18:47:54+00:00",
            "title": "UnibucKernel: A kernel-based learning method for complex word identification",
            "authors": [
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this paper, we present a kernel-based learning approach for the 2018\nComplex Word Identification (CWI) Shared Task. Our approach is based on\ncombining multiple low-level features, such as character n-grams, with\nhigh-level semantic features that are either automatically learned using word\nembeddings or extracted from a lexical knowledge base, namely WordNet. After\nfeature extraction, we employ a kernel method for the learning phase. The\nfeature matrix is first transformed into a normalized kernel matrix. For the\nbinary classification task (simple versus complex), we employ Support Vector\nMachines. For the regression task, in which we have to predict the complexity\nlevel of a word (a word is more complex if it is labeled as complex by more\nannotators), we employ v-Support Vector Regression. We applied our approach\nonly on the three English data sets containing documents from Wikipedia,\nWikiNews and News domains. Our best result during the competition was the third\nplace on the English Wikipedia data set. However, in this paper, we also report\nbetter post-competition results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07640v2",
            "date": "2018-03-20 20:32:07+00:00",
            "title": "AllenNLP: A Deep Semantic Natural Language Processing Platform",
            "authors": [
                "Matt Gardner",
                "Joel Grus",
                "Mark Neumann",
                "Oyvind Tafjord",
                "Pradeep Dasigi",
                "Nelson Liu",
                "Matthew Peters",
                "Michael Schmitz",
                "Luke Zettlemoyer"
            ],
            "abstract": "This paper describes AllenNLP, a platform for research on deep learning\nmethods in natural language understanding. AllenNLP is designed to support\nresearchers who want to build novel language understanding models quickly and\neasily. It is built on top of PyTorch, allowing for dynamic computation graphs,\nand provides (1) a flexible data API that handles intelligent batching and\npadding, (2) high-level abstractions for common operations in working with\ntext, and (3) a modular and extensible experiment framework that makes doing\ngood science easy. It also includes reference implementations of high quality\napproaches for both core semantic problems (e.g. semantic role labeling (Palmer\net al., 2005)) and language understanding applications (e.g. machine\ncomprehension (Rajpurkar et al., 2016)). AllenNLP is an ongoing open-source\neffort maintained by engineers and researchers at the Allen Institute for\nArtificial Intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07718v1",
            "date": "2018-03-21 02:14:16+00:00",
            "title": "InfyNLP at SMM4H Task 2: Stacked Ensemble of Shallow Convolutional Neural Networks for Identifying Personal Medication Intake from Twitter",
            "authors": [
                "Jasper Friedrichs",
                "Debanjan Mahata",
                "Shubham Gupta"
            ],
            "abstract": "This paper describes Infosys's participation in the \"2nd Social Media Mining\nfor Health Applications Shared Task at AMIA, 2017, Task 2\". Mining social media\nmessages for health and drug related information has received significant\ninterest in pharmacovigilance research. This task targets at developing\nautomated classification models for identifying tweets containing descriptions\nof personal intake of medicines. Towards this objective we train a stacked\nensemble of shallow convolutional neural network (CNN) models on an annotated\ndataset provided by the organizers. We use random search for tuning the\nhyper-parameters of the CNN and submit an ensemble of best models for the\nprediction task. Our system secured first place among 9 teams, with a\nmicro-averaged F-score of 0.693.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07771v1",
            "date": "2018-03-21 07:13:16+00:00",
            "title": "$\u03c1$-hot Lexicon Embedding-based Two-level LSTM for Sentiment Analysis",
            "authors": [
                "Ou Wu",
                "Tao Yang",
                "Mengyang Li",
                "Ming Li"
            ],
            "abstract": "Sentiment analysis is a key component in various text mining applications.\nNumerous sentiment classification techniques, including conventional and deep\nlearning-based methods, have been proposed in the literature. In most existing\nmethods, a high-quality training set is assumed to be given. Nevertheless,\nconstructing a high-quality training set that consists of highly accurate\nlabels is challenging in real applications. This difficulty stems from the fact\nthat text samples usually contain complex sentiment representations, and their\nannotation is subjective. We address this challenge in this study by leveraging\na new labeling strategy and utilizing a two-level long short-term memory\nnetwork to construct a sentiment classifier. Lexical cues are useful for\nsentiment analysis, and they have been utilized in conventional studies. For\nexample, polar and privative words play important roles in sentiment analysis.\nA new encoding strategy, that is, $\\rho$-hot encoding, is proposed to alleviate\nthe drawbacks of one-hot encoding and thus effectively incorporate useful\nlexical cues. We compile three Chinese data sets on the basis of our label\nstrategy and proposed methodology. Experiments on the three data sets\ndemonstrate that the proposed method outperforms state-of-the-art algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08073v1",
            "date": "2018-03-21 18:16:23+00:00",
            "title": "Olive Oil is Made of Olives, Baby Oil is Made for Babies: Interpreting Noun Compounds using Paraphrases in a Neural Model",
            "authors": [
                "Vered Shwartz",
                "Chris Waterson"
            ],
            "abstract": "Automatic interpretation of the relation between the constituents of a noun\ncompound, e.g. olive oil (source) and baby oil (purpose) is an important task\nfor many NLP applications. Recent approaches are typically based on either\nnoun-compound representations or paraphrases. While the former has initially\nshown promising results, recent work suggests that the success stems from\nmemorizing single prototypical words for each relation. We explore a neural\nparaphrasing approach that demonstrates superior performance when such\nmemorization is not possible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08409v1",
            "date": "2018-03-22 15:31:39+00:00",
            "title": "Quality expectations of machine translation",
            "authors": [
                "Andy Way"
            ],
            "abstract": "Machine Translation (MT) is being deployed for a range of use-cases by\nmillions of people on a daily basis. There should, therefore, be no doubt as to\nthe utility of MT. However, not everyone is convinced that MT can be useful,\nespecially as a productivity enhancer for human translators. In this chapter, I\naddress this issue, describing how MT is currently deployed, how its output is\nevaluated and how this could be enhanced, especially as MT quality itself\nimproves. Central to these issues is the acceptance that there is no longer a\nsingle 'gold standard' measure of quality, such that the situation in which MT\nis deployed needs to be borne in mind, especially with respect to the expected\n'shelf-life' of the translation itself.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08463v1",
            "date": "2018-03-22 17:04:32+00:00",
            "title": "A Feature-Based Model for Nested Named-Entity Recognition at VLSP-2018 NER Evaluation Campaign",
            "authors": [
                "Pham Quang Nhat Minh"
            ],
            "abstract": "In this report, we describe our participant named-entity recognition system\nat VLSP 2018 evaluation campaign. We formalized the task as a sequence labeling\nproblem using BIO encoding scheme. We applied a feature-based model which\ncombines word, word-shape features, Brown-cluster-based features, and\nword-embedding-based features. We compare several methods to deal with nested\nentities in the dataset. We showed that combining tags of entities at all\nlevels for training a sequence labeling model (joint-tag model) improved the\naccuracy of nested named-entity recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08493v6",
            "date": "2018-03-22 17:54:21+00:00",
            "title": "Contextual Salience for Fast and Accurate Sentence Vectors",
            "authors": [
                "Eric Zelikman",
                "Richard Socher"
            ],
            "abstract": "Unsupervised vector representations of sentences or documents are a major\nbuilding block for many language tasks such as sentiment classification.\nHowever, current methods are uninterpretable and slow or require large training\ndatasets. Recent word vector-based proposals implicitly assume that distances\nin a word embedding space are equally important, regardless of context. We\nintroduce contextual salience (CoSal), a measure of word importance that uses\nthe distribution of context vectors to normalize distances and weights. CoSal\nrelies on the insight that unusual word vectors disproportionately affect\nphrase vectors. A bag-of-words model with CoSal-based weights produces accurate\nunsupervised sentence or document representations for classification, requiring\nlittle computation to evaluate and only a single covariance calculation to\n``train.\" CoSal supports small contexts, out-of context words and outperforms\nSkipThought on most benchmarks, beats tf-idf on all benchmarks, and is\ncompetitive with the unsupervised state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08614v1",
            "date": "2018-03-22 23:46:22+00:00",
            "title": "MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classification",
            "authors": [
                "Jeremy Barnes",
                "Patrik Lambert",
                "Toni Badia"
            ],
            "abstract": "While sentiment analysis has become an established field in the NLP\ncommunity, research into languages other than English has been hindered by the\nlack of resources. Although much research in multi-lingual and cross-lingual\nsentiment analysis has focused on unsupervised or semi-supervised approaches,\nthese still require a large number of resources and do not reach the\nperformance of supervised approaches. With this in mind, we introduce two\ndatasets for supervised aspect-level sentiment analysis in Basque and Catalan,\nboth of which are under-resourced languages. We provide high-quality\nannotations and benchmarks with the hope that they will be useful to the\ngrowing community of researchers working on these languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08652v1",
            "date": "2018-03-23 04:15:07+00:00",
            "title": "Studio Ousia's Quiz Bowl Question Answering System",
            "authors": [
                "Ikuya Yamada",
                "Ryuji Tamaki",
                "Hiroyuki Shindo",
                "Yoshiyasu Takefuji"
            ],
            "abstract": "In this chapter, we describe our question answering system, which was the\nwinning system at the Human-Computer Question Answering (HCQA) Competition at\nthe Thirty-first Annual Conference on Neural Information Processing Systems\n(NIPS). The competition requires participants to address a factoid question\nanswering task referred to as quiz bowl. To address this task, we use two novel\nneural network models and combine these models with conventional information\nretrieval models using a supervised machine learning model. Our system achieved\nthe best performance among the systems submitted in the competition and won a\nmatch against six top human quiz experts by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08910v1",
            "date": "2018-03-23 17:49:04+00:00",
            "title": "Stance Detection on Tweets: An SVM-based Approach",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Fazli Can"
            ],
            "abstract": "Stance detection is a subproblem of sentiment analysis where the stance of\nthe author of a piece of natural language text for a particular target (either\nexplicitly stated in the text or not) is explored. The stance output is usually\ngiven as Favor, Against, or Neither. In this paper, we target at stance\ndetection on sports-related tweets and present the performance results of our\nSVM-based stance classifiers on such tweets. First, we describe three versions\nof our proprietary tweet data set annotated with stance information, all of\nwhich are made publicly available for research purposes. Next, we evaluate SVM\nclassifiers using different feature sets for stance detection on this data set.\nThe employed features are based on unigrams, bigrams, hashtags, external links,\nemoticons, and lastly, named entities. The results indicate that joint use of\nthe features based on unigrams, hashtags, and named entities by SVM classifiers\nis a plausible approach for stance detection problem on sports-related tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08976v2",
            "date": "2018-03-23 20:59:09+00:00",
            "title": "Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech",
            "authors": [
                "Yu-An Chung",
                "James Glass"
            ],
            "abstract": "In this paper, we propose a novel deep neural network architecture,\nSpeech2Vec, for learning fixed-length vector representations of audio segments\nexcised from a speech corpus, where the vectors contain semantic information\npertaining to the underlying spoken words, and are close to other vectors in\nthe embedding space if their corresponding underlying spoken words are\nsemantically similar. The proposed model can be viewed as a speech version of\nWord2Vec. Its design is based on a RNN Encoder-Decoder framework, and borrows\nthe methodology of skipgrams or continuous bag-of-words for training. Learning\nword embeddings directly from speech enables Speech2Vec to make use of the\nsemantic information carried by speech that does not exist in plain text. The\nlearned word embeddings are evaluated and analyzed on 13 widely used word\nsimilarity benchmarks, and outperform word embeddings learned by Word2Vec from\nthe transcriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08991v2",
            "date": "2018-03-23 21:56:54+00:00",
            "title": "Leveraging translations for speech transcription in low-resource settings",
            "authors": [
                "Antonis Anastasopoulos",
                "David Chiang"
            ],
            "abstract": "Recently proposed data collection frameworks for endangered language\ndocumentation aim not only to collect speech in the language of interest, but\nalso to collect translations into a high-resource language that will render the\ncollected resource interpretable. We focus on this scenario and explore whether\nwe can improve transcription quality under these extremely low-resource\nsettings with the assistance of text translations. We present a neural\nmulti-source model and evaluate several variations of it on three low-resource\ndatasets. We find that our multi-source model with shared attention outperforms\nthe baselines, reducing transcription character error rate by up to 12.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09065v3",
            "date": "2018-03-24 06:21:56+00:00",
            "title": "Near-lossless Binarization of Word Embeddings",
            "authors": [
                "Julien Tissier",
                "Christophe Gravier",
                "Amaury Habrard"
            ],
            "abstract": "Word embeddings are commonly used as a starting point in many NLP models to\nachieve state-of-the-art performances. However, with a large vocabulary and\nmany dimensions, these floating-point representations are expensive both in\nterms of memory and calculations which makes them unsuitable for use on\nlow-resource devices. The method proposed in this paper transforms real-valued\nembeddings into binary embeddings while preserving semantic information,\nrequiring only 128 or 256 bits for each vector. This leads to a small memory\nfootprint and fast vector operations. The model is based on an autoencoder\narchitecture, which also allows to reconstruct original vectors from the binary\nones. Experimental results on semantic similarity, text classification and\nsentiment analysis tasks show that the binarization of word embeddings only\nleads to a loss of ~2% in accuracy while vector size is reduced by 97%.\nFurthermore, a top-k benchmark demonstrates that using these binary vectors is\n30 times faster than using real-valued vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09091v1",
            "date": "2018-03-24 10:57:41+00:00",
            "title": "Simple Large-scale Relation Extraction from Unstructured Text",
            "authors": [
                "Christos Christodoulopoulos",
                "Arpit Mittal"
            ],
            "abstract": "Knowledge-based question answering relies on the availability of facts, the\nmajority of which cannot be found in structured sources (e.g. Wikipedia\ninfo-boxes, Wikidata). One of the major components of extracting facts from\nunstructured text is Relation Extraction (RE). In this paper we propose a novel\nmethod for creating distant (weak) supervision labels for training a\nlarge-scale RE system. We also provide new evidence about the effectiveness of\nneural network approaches by decoupling the model architecture from the feature\ndesign of a state-of-the-art neural network system. Surprisingly, a much\nsimpler classifier trained on similar features performs on par with the highly\ncomplex neural network system (at 75x reduction to the training time),\nsuggesting that the features are a bigger contributor to the final performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09103v1",
            "date": "2018-03-24 13:08:56+00:00",
            "title": "Machine Learning and Applied Linguistics",
            "authors": [
                "Sowmya Vajjala"
            ],
            "abstract": "This entry introduces the topic of machine learning and provides an overview\nof its relevance for applied linguistics and language learning. The discussion\nwill focus on giving an introduction to the methods and applications of machine\nlearning in applied linguistics, and will provide references for further study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09164v2",
            "date": "2018-03-24 21:17:52+00:00",
            "title": "Low-Resource Speech-to-Text Translation",
            "authors": [
                "Sameer Bansal",
                "Herman Kamper",
                "Karen Livescu",
                "Adam Lopez",
                "Sharon Goldwater"
            ],
            "abstract": "Speech-to-text translation has many potential applications for low-resource\nlanguages, but the typical approach of cascading speech recognition with\nmachine translation is often impossible, since the transcripts needed to train\na speech recognizer are usually not available for low-resource languages.\nRecent work has found that neural encoder-decoder models can learn to directly\ntranslate foreign speech in high-resource scenarios, without the need for\nintermediate transcription. We investigate whether this approach also works in\nsettings where both data and computation are limited. To make the approach\nefficient, we make several architectural changes, including a change from\ncharacter-level to word-level decoding. We find that this choice yields crucial\nspeed improvements that allow us to train with fewer computational resources,\nyet still performs well on frequent words. We explore models trained on between\n20 and 160 hours of data, and find that although models trained on less data\nhave considerably lower BLEU scores, they can still predict words with\nrelatively high precision and recall---around 50% for a model trained on 50\nhours of data, versus around 60% for the full 160 hour model. Thus, they may\nstill be useful for some low-resource scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09230v1",
            "date": "2018-03-25 10:58:42+00:00",
            "title": "Pay More Attention - Neural Architectures for Question-Answering",
            "authors": [
                "Zia Hasan",
                "Sebastian Fischer"
            ],
            "abstract": "Machine comprehension is a representative task of natural language\nunderstanding. Typically, we are given context paragraph and the objective is\nto answer a question that depends on the context. Such a problem requires to\nmodel the complex interactions between the context paragraph and the question.\nLately, attention mechanisms have been found to be quite successful at these\ntasks and in particular, attention mechanisms with attention flow from both\ncontext-to-question and question-to-context have been proven to be quite\nuseful. In this paper, we study two state-of-the-art attention mechanisms\ncalled Bi-Directional Attention Flow (BiDAF) and Dynamic Co-Attention Network\n(DCN) and propose a hybrid scheme combining these two architectures that gives\nbetter overall performance. Moreover, we also suggest a new simpler attention\nmechanism that we call Double Cross Attention (DCA) that provides better\nresults compared to both BiDAF and Co-Attention mechanisms while providing\nsimilar performance as the hybrid scheme. The objective of our paper is to\nfocus particularly on the attention layer and to suggest improvements on that.\nOur experimental evaluations show that both our proposed models achieve\nsuperior results on the Stanford Question Answering Dataset (SQuAD) compared to\nBiDAF and DCN attention mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09288v1",
            "date": "2018-03-25 16:08:06+00:00",
            "title": "The Geometry of Culture: Analyzing Meaning through Word Embeddings",
            "authors": [
                "Austin C. Kozlowski",
                "Matt Taddy",
                "James A. Evans"
            ],
            "abstract": "We demonstrate the utility of a new methodological tool, neural-network word\nembedding models, for large-scale text analysis, revealing how these models\nproduce richer insights into cultural associations and categories than possible\nwith prior methods. Word embeddings represent semantic relations between words\nas geometric relationships between vectors in a high-dimensional space,\noperationalizing a relational model of meaning consistent with contemporary\ntheories of identity and culture. We show that dimensions induced by word\ndifferences (e.g. man - woman, rich - poor, black - white, liberal -\nconservative) in these vector spaces closely correspond to dimensions of\ncultural meaning, and the projection of words onto these dimensions reflects\nwidely shared cultural connotations when compared to surveyed responses and\nlabeled historical data. We pilot a method for testing the stability of these\nassociations, then demonstrate applications of word embeddings for\nmacro-cultural investigation with a longitudinal analysis of the coevolution of\ngender and class associations in the United States over the 20th century and a\ncomparative analysis of historic distinctions between markers of gender and\nclass in the U.S. and Britain. We argue that the success of these\nhigh-dimensional models motivates a move towards \"high-dimensional theorizing\"\nof meanings, identities and cultural processes.",
            "journal ref": "American Sociological Review 2019, Vol. 84(5) 905-949",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09337v1",
            "date": "2018-03-25 20:53:40+00:00",
            "title": "Text Segmentation as a Supervised Learning Task",
            "authors": [
                "Omri Koshorek",
                "Adir Cohen",
                "Noam Mor",
                "Michael Rotman",
                "Jonathan Berant"
            ],
            "abstract": "Text segmentation, the task of dividing a document into contiguous segments\nbased on its semantic structure, is a longstanding challenge in language\nunderstanding. Previous work on text segmentation focused on unsupervised\nmethods such as clustering or graph search, due to the paucity in labeled data.\nIn this work, we formulate text segmentation as a supervised learning problem,\nand present a large new dataset for text segmentation that is automatically\nextracted and labeled from Wikipedia. Moreover, we develop a segmentation model\nbased on this dataset and show that it generalizes well to unseen natural text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09371v1",
            "date": "2018-03-26 00:06:57+00:00",
            "title": "StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow",
            "authors": [
                "Ziyu Yao",
                "Daniel S. Weld",
                "Wei-Peng Chen",
                "Huan Sun"
            ],
            "abstract": "Stack Overflow (SO) has been a great source of natural language questions and\ntheir code solutions (i.e., question-code pairs), which are critical for many\ntasks including code retrieval and annotation. In most existing research,\nquestion-code pairs were collected heuristically and tend to have low quality.\nIn this paper, we investigate a new problem of systematically mining\nquestion-code pairs from Stack Overflow (in contrast to heuristically\ncollecting them). It is formulated as predicting whether or not a code snippet\nis a standalone solution to a question. We propose a novel Bi-View Hierarchical\nNeural Network which can capture both the programming content and the textual\ncontext of a code snippet (i.e., two views) to make a prediction. On two\nmanually annotated datasets in Python and SQL domain, our framework\nsubstantially outperforms heuristic methods with at least 15% higher F1 and\naccuracy. Furthermore, we present StaQC (Stack Overflow Question-Code pairs),\nthe largest dataset to date of ~148K Python and ~120K SQL question-code pairs,\nautomatically mined from SO using our framework. Under various case studies, we\ndemonstrate that StaQC can greatly help develop data-hungry models for\nassociating natural language with programming language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09402v1",
            "date": "2018-03-26 03:54:34+00:00",
            "title": "Aggression-annotated Corpus of Hindi-English Code-mixed Data",
            "authors": [
                "Ritesh Kumar",
                "Aishwarya N. Reganti",
                "Akshit Bhatia",
                "Tushar Maheshwari"
            ],
            "abstract": "As the interaction over the web has increased, incidents of aggression and\nrelated events like trolling, cyberbullying, flaming, hate speech, etc. too\nhave increased manifold across the globe. While most of these behaviour like\nbullying or hate speech have predated the Internet, the reach and extent of the\nInternet has given these an unprecedented power and influence to affect the\nlives of billions of people. So it is of utmost significance and importance\nthat some preventive measures be taken to provide safeguard to the people using\nthe web such that the web remains a viable medium of communication and\nconnection, in general. In this paper, we discuss the development of an\naggression tagset and an annotated corpus of Hindi-English code-mixed data from\ntwo of the most popular social networking and social media platforms in India,\nTwitter and Facebook. The corpus is annotated using a hierarchical tagset of 3\ntop-level tags and 10 level 2 tags. The final dataset contains approximately\n18k tweets and 21k facebook comments and is being released for further research\nin the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09405v1",
            "date": "2018-03-26 04:02:00+00:00",
            "title": "Automatic Identification of Closely-related Indian Languages: Resources and Experiments",
            "authors": [
                "Ritesh Kumar",
                "Bornini Lahiri",
                "Deepak Alok",
                "Atul Kr. Ojha",
                "Mayank Jain",
                "Abdul Basit",
                "Yogesh Dawer"
            ],
            "abstract": "In this paper, we discuss an attempt to develop an automatic language\nidentification system for 5 closely-related Indo-Aryan languages of India,\nAwadhi, Bhojpuri, Braj, Hindi and Magahi. We have compiled a comparable corpora\nof varying length for these languages from various resources. We discuss the\nmethod of creation of these corpora in detail. Using these corpora, a language\nidentification system was developed, which currently gives state of the art\naccuracy of 96.48\\%. We also used these corpora to study the similarity between\nthe 5 languages at the lexical level, which is the first data-based study of\nthe extent of closeness of these languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09519v2",
            "date": "2018-03-26 11:36:36+00:00",
            "title": "Self-Attentional Acoustic Models",
            "authors": [
                "Matthias Sperber",
                "Jan Niehues",
                "Graham Neubig",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "Self-attention is a method of encoding sequences of vectors by relating these\nvectors to each-other based on pairwise similarities. These models have\nrecently shown promising results for modeling discrete sequences, but they are\nnon-trivial to apply to acoustic modeling due to computational and modeling\nissues. In this paper, we apply self-attention to acoustic modeling, proposing\nseveral improvements to mitigate these issues: First, self-attention memory\ngrows quadratically in the sequence length, which we address through a\ndownsampling technique. Second, we find that previous approaches to incorporate\nposition information into the model are unsuitable and explore other\nrepresentations and hybrid models to this end. Third, to stress the importance\nof local context in the acoustic signal, we propose a Gaussian biasing approach\nthat allows explicit control over the context range. Experiments find that our\nmodel approaches a strong baseline based on LSTMs with network-in-network\nconnections while being much faster to compute. Besides speed, we find that\ninterpretability is a strength of self-attentional acoustic models, and\ndemonstrate that self-attention heads learn a linguistically plausible division\nof labor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09641v1",
            "date": "2018-03-26 15:01:52+00:00",
            "title": "Unsupervised Separation of Transliterable and Native Words for Malayalam",
            "authors": [
                "Deepak P"
            ],
            "abstract": "Differentiating intrinsic language words from transliterable words is a key\nstep aiding text processing tasks involving different natural languages. We\nconsider the problem of unsupervised separation of transliterable words from\nnative words for text in Malayalam language. Outlining a key observation on the\ndiversity of characters beyond the word stem, we develop an optimization method\nto score words based on their nativeness. Our method relies on the usage of\nprobability distributions over character n-grams that are refined in step with\nthe nativeness scorings in an iterative optimization formulation. Using an\nempirical evaluation, we illustrate that our method, DTIM, provides significant\nimprovements in nativeness scoring for Malayalam, establishing DTIM as the\npreferred method for the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09720v1",
            "date": "2018-03-26 17:20:23+00:00",
            "title": "CliCR: A Dataset of Clinical Case Reports for Machine Reading Comprehension",
            "authors": [
                "Simon \u0160uster",
                "Walter Daelemans"
            ],
            "abstract": "We present a new dataset for machine comprehension in the medical domain. Our\ndataset uses clinical case reports with around 100,000 gap-filling queries\nabout these cases. We apply several baselines and state-of-the-art neural\nreaders to the dataset, and observe a considerable gap in performance (20% F1)\nbetween the best human and machine readers. We analyze the skills required for\nsuccessful answering and show how reader performance varies depending on the\napplicable skills. We find that inferences using domain knowledge and object\ntracking are the most frequently required skills, and that recognizing omitted\ninformation and spatio-temporal reasoning are the most difficult for the\nmachines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09832v1",
            "date": "2018-03-26 20:38:31+00:00",
            "title": "Heat Kernel analysis of Syntactic Structures",
            "authors": [
                "Andrew Ortegaray",
                "Robert C. Berwick",
                "Matilde Marcolli"
            ],
            "abstract": "We consider two different data sets of syntactic parameters and we discuss\nhow to detect relations between parameters through a heat kernel method\ndeveloped by Belkin-Niyogi, which produces low dimensional representations of\nthe data, based on Laplace eigenfunctions, that preserve neighborhood\ninformation. We analyze the different connectivity and clustering structures\nthat arise in the two datasets, and the regions of maximal variance in the\ntwo-parameter space of the Belkin-Niyogi construction, which identify\npreferable choices of independent variables. We compute clustering coefficients\nand their variance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09901v1",
            "date": "2018-03-27 05:23:01+00:00",
            "title": "Mittens: An Extension of GloVe for Learning Domain-Specialized Representations",
            "authors": [
                "Nicholas Dingwall",
                "Christopher Potts"
            ],
            "abstract": "We present a simple extension of the GloVe representation learning model that\nbegins with general-purpose representations and updates them based on data from\na specialized domain. We show that the resulting representations can lead to\nfaster learning and better results on a variety of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10357v3",
            "date": "2018-03-27 23:29:23+00:00",
            "title": "Deep Communicating Agents for Abstractive Summarization",
            "authors": [
                "Asli Celikyilmaz",
                "Antoine Bosselut",
                "Xiaodong He",
                "Yejin Choi"
            ],
            "abstract": "We present deep communicating agents in an encoder-decoder architecture to\naddress the challenges of representing a long document for abstractive\nsummarization. With deep communicating agents, the task of encoding a long text\nis divided across multiple collaborating agents, each in charge of a subsection\nof the input text. These encoders are connected to a single decoder, trained\nend-to-end using reinforcement learning to generate a focused and coherent\nsummary. Empirical results demonstrate that multiple communicating encoders\nlead to a higher quality summary compared to several strong baselines,\nincluding those based on a single encoder or multiple non-communicating\nencoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10421v1",
            "date": "2018-03-28 05:47:31+00:00",
            "title": "Handling Verb Phrase Anaphora with Dependent Types and Events",
            "authors": [
                "Daniyar Itegulov",
                "Ekaterina Lebedeva"
            ],
            "abstract": "This paper studies how dependent typed events can be used to treat verb\nphrase anaphora. We introduce a framework that extends Dependent Type Semantics\n(DTS) with a new atomic type for neo-Davidsonian events and an extended\n@-operator that can return new events that share properties of events\nreferenced by verb phrase anaphora. The proposed framework, along with\nillustrative examples of its use, are presented after a brief overview of the\nnecessary background and of the major challenges posed by verb phrase anaphora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10547v3",
            "date": "2018-03-28 11:50:32+00:00",
            "title": "Neural Network Architecture for Credibility Assessment of Textual Claims",
            "authors": [
                "Nurendra Choudhary",
                "Rajat Singh",
                "Ishita Bindlish",
                "Manish Shrivastava"
            ],
            "abstract": "Text articles with false claims, especially news, have recently become\naggravating for the Internet users. These articles are in wide circulation and\nreaders face difficulty discerning fact from fiction. Previous work on\ncredibility assessment has focused on factual analysis and linguistic features.\nThe task's main challenge is the distinction between the features of true and\nfalse articles. In this paper, we propose a novel approach called Credibility\nOutcome (CREDO) which aims at scoring the credibility of an article in an open\ndomain setting.\n  CREDO consists of different modules for capturing various features\nresponsible for the credibility of an article. These features includes\ncredibility of the article's source and author, semantic similarity between the\narticle and related credible articles retrieved from a knowledge base, and\nsentiments conveyed by the article. A neural network architecture learns the\ncontribution of each of these modules to the overall credibility of an article.\nExperiments on Snopes dataset reveals that CREDO outperforms the\nstate-of-the-art approaches based on linguistic features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10631v1",
            "date": "2018-03-28 14:08:12+00:00",
            "title": "Meta-Learning a Dynamical Language Model",
            "authors": [
                "Thomas Wolf",
                "Julien Chaumond",
                "Clement Delangue"
            ],
            "abstract": "We consider the task of word-level language modeling and study the\npossibility of combining hidden-states-based short-term representations with\nmedium-term representations encoded in dynamical weights of a language model.\nOur work extends recent experiments on language models with dynamically\nevolving weights by casting the language modeling problem into an online\nlearning-to-learn framework in which a meta-learner is trained by\ngradient-descent to continuously update a language model weights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10952v3",
            "date": "2018-03-29 08:03:45+00:00",
            "title": "Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only",
            "authors": [
                "Yi-Chen Chen",
                "Chia-Hao Shen",
                "Sung-Feng Huang",
                "Hung-yi Lee"
            ],
            "abstract": "Automatic speech recognition (ASR) has been widely researched with supervised\napproaches, while many low-resourced languages lack audio-text aligned data,\nand supervised methods cannot be applied on them.\n  In this work, we propose a framework to achieve unsupervised ASR on a read\nEnglish speech dataset, where audio and text are unaligned. In the first stage,\neach word-level audio segment in the utterances is represented by a vector\nrepresentation extracted by a sequence-of-sequence autoencoder, in which\nphonetic information and speaker information are disentangled.\n  Secondly, semantic embeddings of audio segments are trained from the vector\nrepresentations using a skip-gram model. Last but not the least, an\nunsupervised method is utilized to transform semantic embeddings of audio\nsegments to text embedding space, and finally the transformed embeddings are\nmapped to words.\n  With the above framework, we are towards unsupervised ASR trained by\nunaligned text and speech only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11045v2",
            "date": "2018-03-29 13:11:32+00:00",
            "title": "Computer-Assisted Text Analysis for Social Science: Topic Models and Beyond",
            "authors": [
                "Ryan Wesslen"
            ],
            "abstract": "Topic models are a family of statistical-based algorithms to summarize,\nexplore and index large collections of text documents. After a decade of\nresearch led by computer scientists, topic models have spread to social science\nas a new generation of data-driven social scientists have searched for tools to\nexplore large collections of unstructured text. Recently, social scientists\nhave contributed to topic model literature with developments in causal\ninference and tools for handling the problem of multi-modality. In this paper,\nI provide a literature review on the evolution of topic modeling including\nextensions for document covariates, methods for evaluation and interpretation,\nand advances in interactive visualizations along with each aspect's relevance\nand application for social science research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11112v1",
            "date": "2018-03-29 15:18:09+00:00",
            "title": "Identifying Semantic Divergences in Parallel Text without Annotations",
            "authors": [
                "Yogarshi Vyas",
                "Xing Niu",
                "Marine Carpuat"
            ],
            "abstract": "Recognizing that even correct translations are not always semantically\nequivalent, we automatically detect meaning divergences in parallel sentence\npairs with a deep neural model of bilingual semantic similarity which can be\ntrained for any parallel corpus without any manual annotation. We show that our\nsemantic model detects divergences more accurately than models based on surface\nfeatures derived from word alignments, and that these divergences matter for\nneural machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11138v1",
            "date": "2018-03-29 16:27:36+00:00",
            "title": "Colorless green recurrent networks dream hierarchically",
            "authors": [
                "Kristina Gulordava",
                "Piotr Bojanowski",
                "Edouard Grave",
                "Tal Linzen",
                "Marco Baroni"
            ],
            "abstract": "Recurrent neural networks (RNNs) have achieved impressive results in a\nvariety of linguistic processing tasks, suggesting that they can induce\nnon-trivial properties of language. We investigate here to what extent RNNs\nlearn to track abstract hierarchical syntactic structure. We test whether RNNs\ntrained with a generic language modeling objective in four languages (Italian,\nEnglish, Hebrew, Russian) can predict long-distance number agreement in various\nconstructions. We include in our evaluation nonsensical sentences where RNNs\ncannot rely on semantic or lexical cues (\"The colorless green ideas I ate with\nthe chair sleep furiously\"), and, for Italian, we compare model performance to\nhuman intuitions. Our language-model-trained RNNs make reliable predictions\nabout long-distance agreement, and do not lag much behind human performance. We\nthus bring support to the hypothesis that RNNs are not just shallow-pattern\nextractors, but they also acquire deeper grammatical competence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11175v2",
            "date": "2018-03-29 17:43:03+00:00",
            "title": "Universal Sentence Encoder",
            "authors": [
                "Daniel Cer",
                "Yinfei Yang",
                "Sheng-yi Kong",
                "Nan Hua",
                "Nicole Limtiaco",
                "Rhomni St. John",
                "Noah Constant",
                "Mario Guajardo-Cespedes",
                "Steve Yuan",
                "Chris Tar",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "We present models for encoding sentences into embedding vectors that\nspecifically target transfer learning to other NLP tasks. The models are\nefficient and result in accurate performance on diverse transfer tasks. Two\nvariants of the encoding models allow for trade-offs between accuracy and\ncompute resources. For both variants, we investigate and report the\nrelationship between model complexity, resource consumption, the availability\nof transfer task training data, and task performance. Comparisons are made with\nbaselines that use word level transfer learning via pretrained word embeddings\nas well as baselines do not use any transfer learning. We find that transfer\nlearning using sentence embeddings tends to outperform word level transfer.\nWith transfer learning via sentence embeddings, we observe surprisingly good\nperformance with minimal amounts of supervised training data for a transfer\ntask. We obtain encouraging results on Word Embedding Association Tests (WEAT)\ntargeted at detecting model bias. Our pre-trained sentence encoding models are\nmade freely available for download and on TF Hub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11291v1",
            "date": "2018-03-30 00:01:08+00:00",
            "title": "Robust Cross-lingual Hypernymy Detection using Dependency Context",
            "authors": [
                "Shyam Upadhyay",
                "Yogarshi Vyas",
                "Marine Carpuat",
                "Dan Roth"
            ],
            "abstract": "Cross-lingual Hypernymy Detection involves determining if a word in one\nlanguage (\"fruit\") is a hypernym of a word in another language (\"pomme\" i.e.\napple in French). The ability to detect hypernymy cross-lingually can aid in\nsolving cross-lingual versions of tasks such as textual entailment and event\ncoreference. We propose BISPARSE-DEP, a family of unsupervised approaches for\ncross-lingual hypernymy detection, which learns sparse, bilingual word\nembeddings based on dependency contexts. We show that BISPARSE-DEP can\nsignificantly improve performance on this task, compared to approaches based\nonly on lexical context. Our approach is also robust, showing promise for\nlow-resource settings: our dependency-based embeddings can be learned using a\nparser trained on related languages, with negligible loss in performance. We\nalso crowd-source a challenging dataset for this task on four languages --\nRussian, French, Arabic, and Chinese. Our embeddings and datasets are publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11326v4",
            "date": "2018-03-30 03:49:18+00:00",
            "title": "Deep Cascade Multi-task Learning for Slot Filling in Online Shopping Assistant",
            "authors": [
                "Yu Gong",
                "Xusheng Luo",
                "Yu Zhu",
                "Wenwu Ou",
                "Zhao Li",
                "Muhua Zhu",
                "Kenny Q. Zhu",
                "Lu Duan",
                "Xi Chen"
            ],
            "abstract": "Slot filling is a critical task in natural language understanding (NLU) for\ndialog systems. State-of-the-art approaches treat it as a sequence labeling\nproblem and adopt such models as BiLSTM-CRF. While these models work relatively\nwell on standard benchmark datasets, they face challenges in the context of\nE-commerce where the slot labels are more informative and carry richer\nexpressions. In this work, inspired by the unique structure of E-commerce\nknowledge base, we propose a novel multi-task model with cascade and residual\nconnections, which jointly learns segment tagging, named entity tagging and\nslot filling. Experiments show the effectiveness of the proposed cascade and\nresidual structures. Our model has a 14.6% advantage in F1 score over the\nstrong baseline methods on a new Chinese E-commerce shopping assistant dataset,\nwhile achieving competitive accuracies on a standard dataset. Furthermore,\nonline test deployed on such dominant E-commerce platform shows 130%\nimprovement on accuracy of understanding user utterances. Our model has already\ngone into production in the E-commerce platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11359v4",
            "date": "2018-03-30 06:34:17+00:00",
            "title": "Automatic Generation of Chinese Short Product Titles for Mobile Display",
            "authors": [
                "Yu Gong",
                "Xusheng Luo",
                "Kenny Q. Zhu",
                "Wenwu Ou",
                "Zhao Li",
                "Lu Duan"
            ],
            "abstract": "This paper studies the problem of automatically extracting a short title from\na manually written longer description of E-commerce products for display on\nmobile devices. It is a new extractive summarization problem on short text\ninputs, for which we propose a feature-enriched network model, combining three\ndifferent categories of features in parallel. Experimental results show that\nour framework significantly outperforms several baselines by a substantial gain\nof 4.5%. Moreover, we produce an extractive summarization dataset for\nE-commerce short texts and will release it to the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11407v2",
            "date": "2018-03-30 10:38:33+00:00",
            "title": "Fine-Grained Attention Mechanism for Neural Machine Translation",
            "authors": [
                "Heeyoul Choi",
                "Kyunghyun Cho",
                "Yoshua Bengio"
            ],
            "abstract": "Neural machine translation (NMT) has been a new paradigm in machine\ntranslation, and the attention mechanism has become the dominant approach with\nthe state-of-the-art records in many language pairs. While there are variants\nof the attention mechanism, all of them use only temporal attention where one\nscalar value is assigned to one context vector corresponding to a source word.\nIn this paper, we propose a fine-grained (or 2D) attention mechanism where each\ndimension of a context vector will receive a separate attention score. In\nexperiments with the task of En-De and En-Fi translation, the fine-grained\nattention method improves the translation quality in terms of BLEU score. In\naddition, our alignment analysis reveals how the fine-grained attention\nmechanism exploits the internal structure of context vectors.",
            "journal ref": "Neurocomputing 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11506v1",
            "date": "2018-03-30 15:14:51+00:00",
            "title": "Automatically augmenting an emotion dataset improves classification using audio",
            "authors": [
                "Egor Lakomkin",
                "Cornelius Weber",
                "Stefan Wermter"
            ],
            "abstract": "In this work, we tackle a problem of speech emotion classification. One of\nthe issues in the area of affective computation is that the amount of annotated\ndata is very limited. On the other hand, the number of ways that the same\nemotion can be expressed verbally is enormous due to variability between\nspeakers. This is one of the factors that limits performance and\ngeneralization. We propose a simple method that extracts audio samples from\nmovies using textual sentiment analysis. As a result, it is possible to\nautomatically construct a larger dataset of audio samples with positive,\nnegative emotional and neutral speech. We show that pretraining recurrent\nneural network on such a dataset yields better results on the challenging\nEmotiW corpus. This experiment shows a potential benefit of combining textual\nsentiment analysis with vocal information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11508v1",
            "date": "2018-03-30 15:18:20+00:00",
            "title": "Reusing Neural Speech Representations for Auditory Emotion Recognition",
            "authors": [
                "Egor Lakomkin",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Acoustic emotion recognition aims to categorize the affective state of the\nspeaker and is still a difficult task for machine learning models. The\ndifficulties come from the scarcity of training data, general subjectivity in\nemotion perception resulting in low annotator agreement, and the uncertainty\nabout which features are the most relevant and robust ones for classification.\nIn this paper, we will tackle the latter problem. Inspired by the recent\nsuccess of transfer learning methods we propose a set of architectures which\nutilize neural representations inferred by training on large speech databases\nfor the acoustic emotion recognition task. Our experiments on the IEMOCAP\ndataset show ~10% relative improvements in the accuracy and F1-score over the\nbaseline recurrent neural network which is trained end-to-end for emotion\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11509v1",
            "date": "2018-03-30 15:21:00+00:00",
            "title": "GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection",
            "authors": [
                "Egor Lakomkin",
                "Chandrakant Bothe",
                "Stefan Wermter"
            ],
            "abstract": "The WASSA 2017 EmoInt shared task has the goal to predict emotion intensity\nvalues of tweet messages. Given the text of a tweet and its emotion category\n(anger, joy, fear, and sadness), the participants were asked to build a system\nthat assigns emotion intensity values. Emotion intensity estimation is a\nchallenging problem given the short length of the tweets, the noisy structure\nof the text and the lack of annotated data. To solve this problem, we developed\nan ensemble of two neural models, processing input on the character. and\nword-level with a lexicon-driven system. The correlation scores across all four\nemotions are averaged to determine the bottom-line competition metric, and our\nsystem ranks place forth in full intensity range and third in 0.5-1 range of\nintensity among 23 systems at the time of writing (June 2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00097v1",
            "date": "2018-04-30 21:00:03+00:00",
            "title": "Syntactic Patterns Improve Information Extraction for Medical Search",
            "authors": [
                "Roma Patel",
                "Yinfei Yang",
                "Iain Marshall",
                "Ani Nenkova",
                "Byron Wallace"
            ],
            "abstract": "Medical professionals search the published literature by specifying the type\nof patients, the medical intervention(s) and the outcome measure(s) of\ninterest. In this paper we demonstrate how features encoding syntactic patterns\nimprove the performance of state-of-the-art sequence tagging models (both\nlinear and neural) for information extraction of these medically relevant\ncategories. We present an analysis of the type of patterns exploited, and the\nsemantic space induced for these, i.e., the distributed representations learned\nfor identified multi-token patterns. We show that these learned representations\ndiffer substantially from those of the constituent unigrams, suggesting that\nthe patterns capture contextual information that is otherwise lost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00178v3",
            "date": "2018-05-01 04:09:09+00:00",
            "title": "Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation",
            "authors": [
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "Traditional Neural machine translation (NMT) involves a fixed training\nprocedure where each sentence is sampled once during each epoch. In reality,\nsome sentences are well-learned during the initial few epochs; however, using\nthis approach, the well-learned sentences would continue to be trained along\nwith those sentences that were not well learned for 10-30 epochs, which results\nin a wastage of time. Here, we propose an efficient method to dynamically\nsample the sentences in order to accelerate the NMT training. In this approach,\na weight is assigned to each sentence based on the measured difference between\nthe training costs of two iterations. Further, in each epoch, a certain\npercentage of sentences are dynamically sampled according to their weights.\nEmpirical results based on the NIST Chinese-to-English and the WMT\nEnglish-to-German tasks depict that the proposed method can significantly\naccelerate the NMT training and improve the NMT performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00249v1",
            "date": "2018-05-01 09:16:53+00:00",
            "title": "Nugget Proposal Networks for Chinese Event Detection",
            "authors": [
                "Hongyu Lin",
                "Yaojie Lu",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "Neural network based models commonly regard event detection as a word-wise\nclassification task, which suffer from the mismatch problem between words and\nevent triggers, especially in languages without natural word delimiters such as\nChinese. In this paper, we propose Nugget Proposal Networks (NPNs), which can\nsolve the word-trigger mismatch problem by directly proposing entire trigger\nnuggets centered at each character regardless of word boundaries. Specifically,\nNPNs perform event detection in a character-wise paradigm, where a hybrid\nrepresentation for each character is first learned to capture both structural\nand semantic information from both characters and words. Then based on learned\nrepresentations, trigger nuggets are proposed and categorized by exploiting\ncharacter compositional structures of Chinese event triggers. Experiments on\nboth ACE2005 and TAC KBP 2017 datasets show that NPNs significantly outperform\nthe state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00270v1",
            "date": "2018-05-01 11:08:00+00:00",
            "title": "Capturing Ambiguity in Crowdsourcing Frame Disambiguation",
            "authors": [
                "Anca Dumitrache",
                "Lora Aroyo",
                "Chris Welty"
            ],
            "abstract": "FrameNet is a computational linguistics resource composed of semantic frames,\nhigh-level concepts that represent the meanings of words. In this paper, we\npresent an approach to gather frame disambiguation annotations in sentences\nusing a crowdsourcing approach with multiple workers per sentence to capture\ninter-annotator disagreement. We perform an experiment over a set of 433\nsentences annotated with frames from the FrameNet corpus, and show that the\naggregated crowd annotations achieve an F1 score greater than 0.67 as compared\nto expert linguists. We highlight cases where the crowd annotation was correct\neven though the expert is in disagreement, arguing for the need to have\nmultiple annotators per sentence. Most importantly, we examine cases in which\ncrowd workers could not agree, and demonstrate that these cases exhibit\nambiguity, either in the sentence, frame, or the task itself, and argue that\ncollapsing such cases to a single, discrete truth value (i.e. correct or\nincorrect) is inappropriate, creating arbitrary targets for machine learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00287v1",
            "date": "2018-05-01 12:21:50+00:00",
            "title": "Multitask Parsing Across Semantic Representations",
            "authors": [
                "Daniel Hershcovich",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "The ability to consolidate information of different types is at the core of\nintelligence, and has tremendous practical value in allowing learning for one\ntask to benefit from generalizations learned for others. In this paper we\ntackle the challenging task of improving semantic parsing performance, taking\nUCCA parsing as a test case, and AMR, SDP and Universal Dependencies (UD)\nparsing as auxiliary tasks. We experiment on three languages, using a uniform\ntransition-based system and learning architecture for all parsing tasks.\nDespite notable conceptual, formal and domain differences, we show that\nmultitask learning significantly improves UCCA parsing in both in-domain and\nout-of-domain settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00456v3",
            "date": "2018-05-01 17:48:10+00:00",
            "title": "Multi-representation Ensembles and Delayed SGD Updates Improve Syntax-based NMT",
            "authors": [
                "Danielle Saunders",
                "Felix Stahlberg",
                "Adria de Gispert",
                "Bill Byrne"
            ],
            "abstract": "We explore strategies for incorporating target syntax into Neural Machine\nTranslation. We specifically focus on syntax in ensembles containing multiple\nsentence representations. We formulate beam search over such ensembles using\nWFSTs, and describe a delayed SGD update training procedure that is especially\neffective for long representations like linearized syntax. Our approach gives\nstate-of-the-art performance on a difficult Japanese-English task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00471v1",
            "date": "2018-05-01 05:24:40+00:00",
            "title": "\"I ain't tellin' white folks nuthin\": A quantitative exploration of the race-related problem of candour in the WPA slave narratives",
            "authors": [
                "Soumya Kambhampati"
            ],
            "abstract": "From 1936-38, the Works Progress Administration interviewed thousands of\nformer slaves about their life experiences. While these interviews are crucial\nto understanding the \"peculiar institution\" from the standpoint of the slave\nhimself, issues relating to bias cloud analyses of these interviews. The\nproblem I investigate is the problem of candour in the WPA slave narratives: it\nis widely held in the historical community that the strict racial caste system\nof the Deep South compelled black ex-slaves to tell white interviewers what\nthey thought they wanted to hear, suggesting that there was a significant\ndifference candour depending on whether their interviewer was white or black.\nIn this work, I attempt to quantitatively characterise this race-related\nproblem of candour. Prior work has either been of an impressionistic,\nqualitative nature, or utilised exceedingly simple quantitative methodology. In\ncontrast, I use more sophisticated statistical methods: in particular word\nfrequency and sentiment analysis and comparative topic modelling with LDA to\ntry and identify differences in the content and sentiment expressed by\nex-slaves in front of white interviewers versus black interviewers. While my\nsentiment analysis methodology was ultimately unsuccessful due to the\ncomplexity of the task, my word frequency analysis and comparative topic\nmodelling methods both showed strong evidence that the content expressed in\nfront of white interviewers was different from that of black interviewers. In\nparticular, I found that the ex-slaves spoke much more about unfavourable\naspects of slavery like whipping and slave patrollers in front of interviewers\nof their own race. I hope that my more-sophisticated statistical methodology\nhelps improve the robustness of the argument for the existence of this problem\nof candour in the slave narratives, which some would seek to deny for\nrevisionist purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00551v1",
            "date": "2018-05-01 21:05:34+00:00",
            "title": "Exploring Conversational Language Generation for Rich Content about Hotels",
            "authors": [
                "Marilyn A. Walker",
                "Albry Smither",
                "Shereen Oraby",
                "Vrindavan Harrison",
                "Hadar Shemtov"
            ],
            "abstract": "Dialogue systems for hotel and tourist information have typically simplified\nthe richness of the domain, focusing system utterances on only a few selected\nattributes such as price, location and type of rooms. However, much more\ncontent is typically available for hotels, often as many as 50 distinct\ninstantiated attributes for an individual entity. New methods are needed to use\nthis content to generate natural dialogues for hotel information, and in\ngeneral for any domain with such rich complex content. We describe three\nexperiments aimed at collecting data that can inform an NLG for hotels\ndialogues, and show, not surprisingly, that the sentences in the original\nwritten hotel descriptions provided on webpages for each hotel are\nstylistically not a very good match for conversational interaction. We quantify\nthe stylistic features that characterize the differences between the original\ntextual data and the collected dialogic data. We plan to use these in stylistic\nmodels for generation, and for scoring retrieved utterances for use in hotel\ndialogues",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00631v3",
            "date": "2018-05-02 05:25:43+00:00",
            "title": "Accelerating Neural Transformer via an Average Attention Network",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su"
            ],
            "abstract": "With parallelizable attention networks, the neural Transformer is very fast\nto train. However, due to the auto-regressive architecture and self-attention\nin the decoder, the decoding procedure becomes slow. To alleviate this issue,\nwe propose an average attention network as an alternative to the self-attention\nnetwork in the decoder of the neural Transformer. The average attention network\nconsists of two layers, with an average layer that models dependencies on\nprevious positions and a gating layer that is stacked over the average layer to\nenhance the expressiveness of the proposed attention network. We apply this\nnetwork on the decoder part of the neural Transformer to replace the original\ntarget-side self-attention model. With masking tricks and dynamic programming,\nour model enables the neural Transformer to decode sentences over four times\nfaster than its original version with almost no loss in training time and\ntranslation performance. We conduct a series of experiments on WMT17\ntranslation tasks, where on 6 different language pairs, we obtain robust and\nconsistent speed-ups in decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00731v1",
            "date": "2018-05-02 11:03:52+00:00",
            "title": "Exploring Emoji Usage and Prediction Through a Temporal Variation Lens",
            "authors": [
                "Francesco Barbieri",
                "Luis Marujo",
                "Pradeep Karuturi",
                "William Brendel",
                "Horacio Saggion"
            ],
            "abstract": "The frequent use of Emojis on social media platforms has created a new form\nof multimodal social interaction. Developing methods for the study and\nrepresentation of emoji semantics helps to improve future multimodal\ncommunication systems. In this paper, we explore the usage and semantics of\nemojis over time. We compare emoji embeddings trained on a corpus of different\nseasons and show that some emojis are used differently depending on the time of\nthe year. Moreover, we propose a method to take into account the time\ninformation for emoji prediction systems, outperforming state-of-the-art\nsystems. We show that, using the time information, the accuracy of some emojis\ncan be significantly improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00760v1",
            "date": "2018-05-02 12:14:11+00:00",
            "title": "Aspect Term Extraction with History Attention and Selective Transformation",
            "authors": [
                "Xin Li",
                "Lidong Bing",
                "Piji Li",
                "Wai Lam",
                "Zhimou Yang"
            ],
            "abstract": "Aspect Term Extraction (ATE), a key sub-task in Aspect-Based Sentiment\nAnalysis, aims to extract explicit aspect expressions from online user reviews.\nWe present a new framework for tackling ATE. It can exploit two useful clues,\nnamely opinion summary and aspect detection history. Opinion summary is\ndistilled from the whole input sentence, conditioned on each current token for\naspect prediction, and thus the tailor-made summary can help aspect prediction\non this token. Another clue is the information of aspect detection history, and\nit is distilled from the previous aspect predictions so as to leverage the\ncoordinate structure and tagging schema constraints to upgrade the aspect\nprediction. Experimental results over four benchmark datasets clearly\ndemonstrate that our framework can outperform all state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00879v1",
            "date": "2018-05-02 15:52:48+00:00",
            "title": "Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only",
            "authors": [
                "Robert Litschko",
                "Goran Glava\u0161",
                "Simone Paolo Ponzetto",
                "Ivan Vuli\u0107"
            ],
            "abstract": "We propose a fully unsupervised framework for ad-hoc cross-lingual\ninformation retrieval (CLIR) which requires no bilingual data at all. The\nframework leverages shared cross-lingual word embedding spaces in which terms,\nqueries, and documents can be represented, irrespective of their actual\nlanguage. The shared embedding spaces are induced solely on the basis of\nmonolingual corpora in two languages through an iterative process based on\nadversarial neural networks. Our experiments on the standard CLEF CLIR\ncollections for three language pairs of varying degrees of language similarity\n(English-Dutch/Italian/Finnish) demonstrate the usefulness of the proposed\nfully unsupervised approach. Our CLIR models with unsupervised cross-lingual\nembeddings outperform baselines that utilize cross-lingual embeddings induced\nrelying on word-level and document-level alignments. We then demonstrate that\nfurther improvements can be achieved by unsupervised ensemble CLIR models. We\nbelieve that the proposed framework is the first step towards development of\neffective CLIR models for language pairs and domains where parallel data are\nscarce or non-existent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01035v1",
            "date": "2018-05-02 21:36:38+00:00",
            "title": "Split and Rephrase: Better Evaluation and a Stronger Baseline",
            "authors": [
                "Roee Aharoni",
                "Yoav Goldberg"
            ],
            "abstract": "Splitting and rephrasing a complex sentence into several shorter sentences\nthat convey the same meaning is a challenging problem in NLP. We show that\nwhile vanilla seq2seq models can reach high scores on the proposed benchmark\n(Narayan et al., 2017), they suffer from memorization of the training set which\ncontains more than 89% of the unique simple sentences from the validation and\ntest sets. To aid this, we present a new train-development-test data split and\nneural models augmented with a copy-mechanism, outperforming the best reported\nbaseline by 8.68 BLEU and fostering further progress on the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01042v1",
            "date": "2018-05-02 22:16:21+00:00",
            "title": "Hypothesis Only Baselines in Natural Language Inference",
            "authors": [
                "Adam Poliak",
                "Jason Naradowsky",
                "Aparajita Haldar",
                "Rachel Rudinger",
                "Benjamin Van Durme"
            ],
            "abstract": "We propose a hypothesis only baseline for diagnosing Natural Language\nInference (NLI). Especially when an NLI dataset assumes inference is occurring\nbased purely on the relationship between a context and a hypothesis, it follows\nthat assessing entailment relations while ignoring the provided context is a\ndegenerate solution. Yet, through experiments on ten distinct NLI datasets, we\nfind that this approach, which we refer to as a hypothesis-only model, is able\nto significantly outperform a majority class baseline across a number of NLI\ndatasets. Our analysis suggests that statistical irregularities may allow a\nmodel to perform NLI in some datasets beyond what should be achievable without\naccess to the context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01052v1",
            "date": "2018-05-02 23:21:12+00:00",
            "title": "Constituency Parsing with a Self-Attentive Encoder",
            "authors": [
                "Nikita Kitaev",
                "Dan Klein"
            ],
            "abstract": "We demonstrate that replacing an LSTM encoder with a self-attentive\narchitecture can lead to improvements to a state-of-the-art discriminative\nconstituency parser. The use of attention makes explicit the manner in which\ninformation is propagated between different locations in the sentence, which we\nuse to both analyze our model and propose potential improvements. For example,\nwe find that separating positional and content information in the encoder can\nlead to improved parsing accuracy. Additionally, we evaluate different\napproaches for lexical representation. Our parser achieves new state-of-the-art\nresults for single models trained on the Penn Treebank: 93.55 F1 without the\nuse of any external data, and 95.13 F1 when using pre-trained word\nrepresentations. Our parser also outperforms the previous best-published\naccuracy figures on 8 of the 9 languages in the SPMRL dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01054v1",
            "date": "2018-05-02 23:29:19+00:00",
            "title": "Automatic Coding for Neonatal Jaundice From Free Text Data Using Ensemble Methods",
            "authors": [
                "Scott Werwath"
            ],
            "abstract": "This study explores the creation of a machine learning model to automatically\nidentify whether a Neonatal Intensive Care Unit (NICU) patient was diagnosed\nwith neonatal jaundice during a particular hospitalization based on their\nassociated clinical notes. We develop a number of techniques for text\npreprocessing and feature selection and compare the effectiveness of different\nclassification models. We show that using ensemble decision tree\nclassification, both with AdaBoost and with bagging, outperforms support vector\nmachines (SVM), the current state-of-the-art technique for neonatal jaundice\ncoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01070v2",
            "date": "2018-05-03 00:46:56+00:00",
            "title": "What you can cram into a single vector: Probing sentence embeddings for linguistic properties",
            "authors": [
                "Alexis Conneau",
                "German Kruszewski",
                "Guillaume Lample",
                "Lo\u00efc Barrault",
                "Marco Baroni"
            ],
            "abstract": "Although much effort has recently been devoted to training high-quality\nsentence embeddings, we still have a poor understanding of what they are\ncapturing. \"Downstream\" tasks, often based on sentence classification, are\ncommonly used to evaluate the quality of sentence representations. The\ncomplexity of the tasks makes it however difficult to infer what kind of\ninformation is present in the representations. We introduce here 10 probing\ntasks designed to capture simple linguistic features of sentences, and we use\nthem to study embeddings generated by three different encoders trained in eight\ndistinct ways, uncovering intriguing properties of both encoders and training\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01086v1",
            "date": "2018-05-03 02:16:27+00:00",
            "title": "Transformation Networks for Target-Oriented Sentiment Classification",
            "authors": [
                "Xin Li",
                "Lidong Bing",
                "Wai Lam",
                "Bei Shi"
            ],
            "abstract": "Target-oriented sentiment classification aims at classifying sentiment\npolarities over individual opinion targets in a sentence. RNN with attention\nseems a good fit for the characteristics of this task, and indeed it achieves\nthe state-of-the-art performance. After re-examining the drawbacks of attention\nmechanism and the obstacles that block CNN to perform well in this\nclassification task, we propose a new model to overcome these issues. Instead\nof attention, our model employs a CNN layer to extract salient features from\nthe transformed word representations originated from a bi-directional RNN\nlayer. Between the two layers, we propose a component to generate\ntarget-specific representations of words in the sentence, meanwhile incorporate\na mechanism for preserving the original contextual information from the RNN\nlayer. Experiments show that our model achieves a new state-of-the-art\nperformance on a few benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01112v1",
            "date": "2018-05-03 04:53:06+00:00",
            "title": "Binarizer at SemEval-2018 Task 3: Parsing dependency and deep learning for irony detection",
            "authors": [
                "Nishant Nikhil",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "In this paper, we describe the system submitted for the SemEval 2018 Task 3\n(Irony detection in English tweets) Subtask A by the team Binarizer. Irony\ndetection is a key task for many natural language processing works. Our method\ntreats ironical tweets to consist of smaller parts containing different\nemotions. We break down tweets into separate phrases using a dependency parser.\nWe then embed those phrases using an LSTM-based neural network model which is\npre-trained to predict emoticons for tweets. Finally, we train a\nfully-connected network to achieve classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01445v2",
            "date": "2018-05-03 17:45:33+00:00",
            "title": "The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models",
            "authors": [
                "Noah Weber",
                "Leena Shekhar",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Seq2Seq based neural architectures have become the go-to architecture to\napply to sequence to sequence language tasks. Despite their excellent\nperformance on these tasks, recent work has noted that these models usually do\nnot fully capture the linguistic structure required to generalize beyond the\ndense sections of the data distribution \\cite{ettinger2017towards}, and as\nsuch, are likely to fail on samples from the tail end of the distribution (such\nas inputs that are noisy \\citep{belkinovnmtbreak} or of different lengths\n\\citep{bentivoglinmtlength}). In this paper, we look at a model's ability to\ngeneralize on a simple symbol rewriting task with a clearly defined structure.\nWe find that the model's ability to generalize this structure beyond the\ntraining distribution depends greatly on the chosen random seed, even when\nperformance on the standard test set remains the same. This suggests that a\nmodel's ability to capture generalizable structure is highly sensitive.\nMoreover, this sensitivity may not be apparent when evaluating it on standard\ntest sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01542v1",
            "date": "2018-05-03 21:21:16+00:00",
            "title": "Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents",
            "authors": [
                "Anuj Goyal",
                "Angeliki Metallinou",
                "Spyros Matsoukas"
            ],
            "abstract": "Fast expansion of natural language functionality of intelligent virtual\nagents is critical for achieving engaging and informative interactions.\nHowever, developing accurate models for new natural language domains is a time\nand data intensive process. We propose efficient deep neural network\narchitectures that maximally re-use available resources through transfer\nlearning. Our methods are applied for expanding the understanding capabilities\nof a popular commercial agent and are evaluated on hundreds of new domains,\ndesigned by internal or external developers. We demonstrate that our proposed\nmethods significantly increase accuracy in low resource settings and enable\nrapid development of accurate models with less data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01555v1",
            "date": "2018-05-03 21:59:13+00:00",
            "title": "An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking",
            "authors": [
                "Puyang Xu",
                "Qi Hu"
            ],
            "abstract": "We highlight a practical yet rarely discussed problem in dialogue state\ntracking (DST), namely handling unknown slot values. Previous approaches\ngenerally assume predefined candidate lists and thus are not designed to output\nunknown values, especially when the spoken language understanding (SLU) module\nis absent as in many end-to-end (E2E) systems. We describe in this paper an E2E\narchitecture based on the pointer network (PtrNet) that can effectively extract\nunknown slot values while still obtains state-of-the-art accuracy on the\nstandard DSTC2 benchmark. We also provide extensive empirical evidence to show\nthat tracking unknown values can be challenging and our approach can bring\nsignificant improvement with the help of an effective feature dropout\ntechnique.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01565v3",
            "date": "2018-05-03 22:58:54+00:00",
            "title": "Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level",
            "authors": [
                "Lifeng Han",
                "Shaohui Kuang"
            ],
            "abstract": "In neural machine translation (NMT), researchers face the challenge of\nun-seen (or out-of-vocabulary OOV) words translation. To solve this, some\nresearchers propose the splitting of western languages such as English and\nGerman into sub-words or compounds. In this paper, we try to address this OOV\nissue and improve the NMT adequacy with a harder language Chinese whose\ncharacters are even more sophisticated in composition. We integrate the Chinese\nradicals into the NMT model with different settings to address the unseen words\nchallenge in Chinese to English translation. On the other hand, this also can\nbe considered as semantic part of the MT system since the Chinese radicals\nusually carry the essential meaning of the words they are constructed in.\nMeaningful radicals and new characters can be integrated into the NMT systems\nwith our models. We use an attention-based NMT system as a strong baseline\nsystem. The experiments on standard Chinese-to-English NIST translation shared\ntask data 2006 and 2008 show that our designed models outperform the baseline\nmodel in a wide range of state-of-the-art evaluation metrics including LEPOR,\nBEER, and CharacTER, in addition to BLEU and NIST scores, especially on the\nadequacy-level translation. We also have some interesting findings from the\nresults of our various experiment settings about the performance of words and\ncharacters in Chinese NMT, which is different with other languages. For\ninstance, the fully character level NMT may perform well or the state of the\nart in some other languages as researchers demonstrated recently, however, in\nthe Chinese NMT model, word boundary knowledge is important for the model\nlearning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01646v1",
            "date": "2018-05-04 08:11:09+00:00",
            "title": "Cross-lingual Candidate Search for Biomedical Concept Normalization",
            "authors": [
                "Roland Roller",
                "Madeleine Kittner",
                "Dirk Weissenborn",
                "Ulf Leser"
            ],
            "abstract": "Biomedical concept normalization links concept mentions in texts to a\nsemantically equivalent concept in a biomedical knowledge base. This task is\nchallenging as concepts can have different expressions in natural languages,\ne.g. paraphrases, which are not necessarily all present in the knowledge base.\nConcept normalization of non-English biomedical text is even more challenging\nas non-English resources tend to be much smaller and contain less synonyms. To\novercome the limitations of non-English terminologies we propose a\ncross-lingual candidate search for concept normalization using a\ncharacter-based neural translation model trained on a multilingual biomedical\nterminology. Our model is trained with Spanish, French, Dutch and German\nversions of UMLS. The evaluation of our model is carried out on the French\nQuaero corpus, showing that it outperforms most teams of CLEF eHealth 2015 and\n2016. Additionally, we compare performance to commercial translators on\nSpanish, French, Dutch and German versions of Mantra. Our model performs\nsimilarly well, but is free of charge and can be run locally. This is\nparticularly important for clinical NLP applications as medical documents\nunderlay strict privacy restrictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01676v1",
            "date": "2018-05-04 09:22:13+00:00",
            "title": "Upping the Ante: Towards a Better Benchmark for Chinese-to-English Machine Translation",
            "authors": [
                "Christian Hadiwinoto",
                "Hwee Tou Ng"
            ],
            "abstract": "There are many machine translation (MT) papers that propose novel approaches\nand show improvements over their self-defined baselines. The experimental\nsetting in each paper often differs from one another. As such, it is hard to\ndetermine if a proposed approach is really useful and advances the state of the\nart. Chinese-to-English translation is a common translation direction in MT\npapers, although there is not one widely accepted experimental setting in\nChinese-to-English MT. Our goal in this paper is to propose a benchmark in\nevaluation setup for Chinese-to-English machine translation, such that the\neffectiveness of a new proposed MT approach can be directly compared to\nprevious approaches. Towards this end, we also built a highly competitive\nstate-of-the-art MT system trained on a large-scale training set. Our system\noutperforms reported results on NIST OpenMT test sets in almost all papers\npublished in major conferences and journals in computational linguistics and\nartificial intelligence in the past 11 years. We argue that a standardized\nbenchmark on data and performance is important for meaningful comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01817v1",
            "date": "2018-05-04 15:07:25+00:00",
            "title": "Extreme Adaptation for Personalized Neural Machine Translation",
            "authors": [
                "Paul Michel",
                "Graham Neubig"
            ],
            "abstract": "Every person speaks or writes their own flavor of their native language,\ninfluenced by a number of factors: the content they tend to talk about, their\ngender, their social status, or their geographical origin.\n  When attempting to perform Machine Translation (MT), these variations have a\nsignificant effect on how the system should perform translation, but this is\nnot captured well by standard one-size-fits-all models.\n  In this paper, we propose a simple and parameter-efficient adaptation\ntechnique that only requires adapting the bias of the output softmax to each\nparticular user of the MT system, either directly or through a factored\napproximation.\n  Experiments on TED talks in three languages demonstrate improvements in\ntranslation accuracy, and better reflection of speaker traits in the target\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01923v1",
            "date": "2018-05-04 19:48:36+00:00",
            "title": "A Rank-Based Similarity Metric for Word Embeddings",
            "authors": [
                "Enrico Santus",
                "Hongmin Wang",
                "Emmanuele Chersoni",
                "Yue Zhang"
            ],
            "abstract": "Word Embeddings have recently imposed themselves as a standard for\nrepresenting word meaning in NLP. Semantic similarity between word pairs has\nbecome the most common evaluation benchmark for these representations, with\nvector cosine being typically used as the only similarity metric. In this\npaper, we report experiments with a rank-based metric for WE, which performs\ncomparably to vector cosine in similarity estimation and outperforms it in the\nrecently-introduced and challenging task of outlier detection, thus suggesting\nthat rank-based measures can improve clustering quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01984v1",
            "date": "2018-05-05 02:44:51+00:00",
            "title": "Various Approaches to Aspect-based Sentiment Analysis",
            "authors": [
                "Amlaan Bhoi",
                "Sandeep Joshi"
            ],
            "abstract": "The problem of aspect-based sentiment analysis deals with classifying\nsentiments (negative, neutral, positive) for a given aspect in a sentence. A\ntraditional sentiment classification task involves treating the entire sentence\nas a text document and classifying sentiments based on all the words. Let us\nassume, we have a sentence such as \"the acceleration of this car is fast, but\nthe reliability is horrible\". This can be a difficult sentence because it has\ntwo aspects with conflicting sentiments about the same entity. Considering\nmachine learning techniques (or deep learning), how do we encode the\ninformation that we are interested in one aspect and its sentiment but not the\nother? Let us explore various pre-processing steps, features, and methods used\nto facilitate in solving this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02023v4",
            "date": "2018-05-05 08:48:32+00:00",
            "title": "Chinese NER Using Lattice LSTM",
            "authors": [
                "Yue Zhang",
                "Jie Yang"
            ],
            "abstract": "We investigate a lattice-structured LSTM model for Chinese NER, which encodes\na sequence of input characters as well as all potential words that match a\nlexicon. Compared with character-based methods, our model explicitly leverages\nword and word sequence information. Compared with word-based methods, lattice\nLSTM does not suffer from segmentation errors. Gated recurrent cells allow our\nmodel to choose the most relevant characters and words from a sentence for\nbetter NER results. Experiments on various datasets show that lattice LSTM\noutperforms both word-based and character-based LSTM baselines, achieving the\nbest results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02036v1",
            "date": "2018-05-05 10:27:32+00:00",
            "title": "Compositional Representation of Morphologically-Rich Input for Neural Machine Translation",
            "authors": [
                "Duygu Ataman",
                "Marcello Federico"
            ],
            "abstract": "Neural machine translation (NMT) models are typically trained with fixed-size\ninput and output vocabularies, which creates an important bottleneck on their\naccuracy and generalization capability. As a solution, various studies proposed\nsegmenting words into sub-word units and performing translation at the\nsub-lexical level. However, statistical word segmentation methods have recently\nshown to be prone to morphological errors, which can lead to inaccurate\ntranslations. In this paper, we propose to overcome this problem by replacing\nthe source-language embedding layer of NMT with a bi-directional recurrent\nneural network that generates compositional representations of the input at any\ndesired level of granularity. We test our approach in a low-resource setting\nwith five languages from different morphological typologies, and under\ndifferent composition assumptions. By training NMT to compose word\nrepresentations from character n-grams, our approach consistently outperforms\n(from 1.71 to 2.48 BLEU points) NMT learning embeddings of statistically\ngenerated sub-word units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02094v1",
            "date": "2018-05-05 18:13:41+00:00",
            "title": "Exploring Hyper-Parameter Optimization for Neural Machine Translation on GPU Architectures",
            "authors": [
                "Robert Lim",
                "Kenneth Heafield",
                "Hieu Hoang",
                "Mark Briers",
                "Allen Malony"
            ],
            "abstract": "Neural machine translation (NMT) has been accelerated by deep learning neural\nnetworks over statistical-based approaches, due to the plethora and\nprogrammability of commodity heterogeneous computing architectures such as\nFPGAs and GPUs and the massive amount of training corpuses generated from news\noutlets, government agencies and social media. Training a learning classifier\nfor neural networks entails tuning hyper-parameters that would yield the best\nperformance. Unfortunately, the number of parameters for machine translation\ninclude discrete categories as well as continuous options, which makes for a\ncombinatorial explosive problem. This research explores optimizing\nhyper-parameters when training deep learning neural networks for machine\ntranslation. Specifically, our work investigates training a language model with\nMarian NMT. Results compare NMT under various hyper-parameter settings across a\nvariety of modern GPU architecture generations in single node and multi-node\nsettings, revealing insights on which hyper-parameters matter most in terms of\nperformance, such as words processed per second, convergence rates, and\ntranslation accuracy, and provides insights on how to best achieve\nhigh-performing NMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02096v1",
            "date": "2018-05-05 18:20:48+00:00",
            "title": "Learning Patient Representations from Text",
            "authors": [
                "Dmitriy Dligach",
                "Timothy Miller"
            ],
            "abstract": "Mining electronic health records for patients who satisfy a set of predefined\ncriteria is known in medical informatics as phenotyping. Phenotyping has\nnumerous applications such as outcome prediction, clinical trial recruitment,\nand retrospective studies. Supervised machine learning for phenotyping\ntypically relies on sparse patient representations such as bag-of-words. We\nconsider an alternative that involves learning patient representations. We\ndevelop a neural network model for learning patient representations and show\nthat the learned representations are general enough to obtain state-of-the-art\nperformance on a standard comorbidity detection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02203v1",
            "date": "2018-05-06 12:41:47+00:00",
            "title": "Dynamic and Static Topic Model for Analyzing Time-Series Document Collections",
            "authors": [
                "Rem Hida",
                "Naoya Takeishi",
                "Takehisa Yairi",
                "Koichi Hori"
            ],
            "abstract": "For extracting meaningful topics from texts, their structures should be\nconsidered properly. In this paper, we aim to analyze structured time-series\ndocuments such as a collection of news articles and a series of scientific\npapers, wherein topics evolve along time depending on multiple topics in the\npast and are also related to each other at each time. To this end, we propose a\ndynamic and static topic model, which simultaneously considers the dynamic\nstructures of the temporal topic evolution and the static structures of the\ntopic hierarchy at each time. We show the results of experiments on collections\nof scientific papers, in which the proposed method outperformed conventional\nmodels. Moreover, we show an example of extracted topic structures, which we\nfound helpful for analyzing research activities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02220v2",
            "date": "2018-05-06 14:26:35+00:00",
            "title": "Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification",
            "authors": [
                "Yizhong Wang",
                "Kai Liu",
                "Jing Liu",
                "Wei He",
                "Yajuan Lyu",
                "Hua Wu",
                "Sujian Li",
                "Haifeng Wang"
            ],
            "abstract": "Machine reading comprehension (MRC) on real web data usually requires the\nmachine to answer a question by analyzing multiple passages retrieved by search\nengine. Compared with MRC on a single passage, multi-passage MRC is more\nchallenging, since we are likely to get multiple confusing answer candidates\nfrom different passages. To address this problem, we propose an end-to-end\nneural model that enables those answer candidates from different passages to\nverify each other based on their content representations. Specifically, we\njointly train three modules that can predict the final answer based on three\nfactors: the answer boundary, the answer content and the cross-passage answer\nverification. The experimental results show that our method outperforms the\nbaseline by a large margin and achieves the state-of-the-art performance on the\nEnglish MS-MARCO dataset and the Chinese DuReader dataset, both of which are\ndesigned for MRC in real-world settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02258v1",
            "date": "2018-05-06 18:25:12+00:00",
            "title": "Russian word sense induction by clustering averaged word embeddings",
            "authors": [
                "Andrey Kutuzov"
            ],
            "abstract": "The paper reports our participation in the shared task on word sense\ninduction and disambiguation for the Russian language (RUSSE-2018). Our team\nwas ranked 2nd for the wiki-wiki dataset (containing mostly homonyms) and 5th\nfor the bts-rnc and active-dict datasets (containing mostly polysemous words)\namong all 19 participants.\n  The method we employed was extremely naive. It implied representing contexts\nof ambiguous words as averaged word embedding vectors, using off-the-shelf\npre-trained distributional models. Then, these vector representations were\nclustered with mainstream clustering techniques, thus producing the groups\ncorresponding to the ambiguous word senses. As a side result, we show that word\nembedding models trained on small but balanced corpora can be superior to those\ntrained on large but noisy data - not only in intrinsic evaluation, but also in\ndownstream tasks like word sense induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02262v1",
            "date": "2018-05-06 18:35:48+00:00",
            "title": "Construction of the Literature Graph in Semantic Scholar",
            "authors": [
                "Waleed Ammar",
                "Dirk Groeneveld",
                "Chandra Bhagavatula",
                "Iz Beltagy",
                "Miles Crawford",
                "Doug Downey",
                "Jason Dunkelberger",
                "Ahmed Elgohary",
                "Sergey Feldman",
                "Vu Ha",
                "Rodney Kinney",
                "Sebastian Kohlmeier",
                "Kyle Lo",
                "Tyler Murray",
                "Hsu-Han Ooi",
                "Matthew Peters",
                "Joanna Power",
                "Sam Skjonsberg",
                "Lucy Lu Wang",
                "Chris Wilhelm",
                "Zheng Yuan",
                "Madeleine van Zuylen",
                "Oren Etzioni"
            ],
            "abstract": "We describe a deployed scalable system for organizing published scientific\nliterature into a heterogeneous graph to facilitate algorithmic manipulation\nand discovery. The resulting literature graph consists of more than 280M nodes,\nrepresenting papers, authors, entities and various interactions between them\n(e.g., authorships, citations, entity mentions). We reduce literature graph\nconstruction into familiar NLP tasks (e.g., entity extraction and linking),\npoint out research challenges due to differences from standard formulations of\nthese tasks, and report empirical results for each task. The methods described\nin this paper are used to enable semantic features in www.semanticscholar.org",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02266v1",
            "date": "2018-05-06 18:49:48+00:00",
            "title": "Breaking NLI Systems with Sentences that Require Simple Lexical Inferences",
            "authors": [
                "Max Glockner",
                "Vered Shwartz",
                "Yoav Goldberg"
            ],
            "abstract": "We create a new NLI test set that shows the deficiency of state-of-the-art\nmodels in inferences that require lexical and world knowledge. The new examples\nare simpler than the SNLI test set, containing sentences that differ by at most\none word from sentences in the training set. Yet, the performance on the new\ntest set is substantially worse across systems trained on SNLI, demonstrating\nthat these systems are limited in their generalization ability, failing to\ncapture many simple inferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02275v1",
            "date": "2018-05-06 20:40:19+00:00",
            "title": "Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach",
            "authors": [
                "Tasnim Mohiuddin",
                "Shafiq Joty",
                "Dat Tien Nguyen"
            ],
            "abstract": "We propose a novel coherence model for written asynchronous conversations\n(e.g., forums, emails), and show its applications in coherence assessment and\nthread reconstruction tasks. We conduct our research in two steps. First, we\npropose improvements to the recently proposed neural entity grid model by\nlexicalizing its entity transitions. Then, we extend the model to asynchronous\nconversations by incorporating the underlying conversational structure in the\nentity grid representation and feature computation. Our model achieves state of\nthe art results on standard coherence assessment tasks in monologue and\nconversations outperforming existing models. We also demonstrate its\neffectiveness in reconstructing thread structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02282v1",
            "date": "2018-05-06 22:14:47+00:00",
            "title": "Multi-Domain Neural Machine Translation",
            "authors": [
                "Sander Tars",
                "Mark Fishel"
            ],
            "abstract": "We present an approach to neural machine translation (NMT) that supports\nmultiple domains in a single model and allows switching between the domains\nwhen translating. The core idea is to treat text domains as distinct languages\nand use multilingual NMT methods to create multi-domain translation systems, we\nshow that this approach results in significant translation quality gains over\nfine-tuning. We also explore whether the knowledge of pre-specified text\ndomains is necessary, turns out that it is after all, but also that when it is\nnot known quite high translation quality can be reached.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02333v2",
            "date": "2018-05-07 03:31:00+00:00",
            "title": "Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Yu Wu",
                "Wei Wu",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "We propose a method that can leverage unlabeled data to learn a matching\nmodel for response selection in retrieval-based chatbots. The method employs a\nsequence-to-sequence architecture (Seq2Seq) model as a weak annotator to judge\nthe matching degree of unlabeled pairs, and then performs learning with both\nthe weak signals and the unlabeled data. Experimental results on two public\ndata sets indicate that matching models get significant improvements when they\nare learned with the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02442v1",
            "date": "2018-05-07 11:14:07+00:00",
            "title": "Paraphrase to Explicate: Revealing Implicit Noun-Compound Relations",
            "authors": [
                "Vered Shwartz",
                "Ido Dagan"
            ],
            "abstract": "Revealing the implicit semantic relation between the constituents of a\nnoun-compound is important for many NLP applications. It has been addressed in\nthe literature either as a classification task to a set of pre-defined\nrelations or by producing free text paraphrases explicating the relations. Most\nexisting paraphrasing methods lack the ability to generalize, and have a hard\ntime interpreting infrequent or new noun-compounds. We propose a neural model\nthat generalizes better by representing paraphrases in a continuous space,\ngeneralizing for both unseen noun-compounds and rare paraphrases. Our model\nhelps improving performance on both the noun-compound paraphrasing and\nclassification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02473v3",
            "date": "2018-05-07 12:31:27+00:00",
            "title": "A Graph-to-Sequence Model for AMR-to-Text Generation",
            "authors": [
                "Linfeng Song",
                "Yue Zhang",
                "Zhiguo Wang",
                "Daniel Gildea"
            ],
            "abstract": "The problem of AMR-to-text generation is to recover a text representing the\nsame meaning as an input AMR graph. The current state-of-the-art method uses a\nsequence-to-sequence model, leveraging LSTM for encoding a linearized AMR\nstructure. Although being able to model non-local semantic information, a\nsequence LSTM can lose information from the AMR graph structure, and thus faces\nchallenges with large graphs, which result in long sequences. We introduce a\nneural graph-to-sequence model, using a novel LSTM structure for directly\nencoding graph-level semantics. On a standard benchmark, our model shows\nsuperior results to existing methods in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02823v1",
            "date": "2018-05-08 04:05:54+00:00",
            "title": "Hierarchical Structured Model for Fine-to-coarse Manifesto Text Analysis",
            "authors": [
                "Shivashankar Subramanian",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Election manifestos document the intentions, motives, and views of political\nparties. They are often used for analysing a party's fine-grained position on a\nparticular issue, as well as for coarse-grained positioning of a party on the\nleft--right spectrum. In this paper we propose a two-stage model for\nautomatically performing both levels of analysis over manifestos. In the first\nstep we employ a hierarchical multi-task structured deep model to predict fine-\nand coarse-grained positions, and in the second step we perform post-hoc\ncalibration of coarse-grained positions using probabilistic soft logic. We\nempirically show that the proposed model outperforms state-of-art approaches at\nboth granularities using manifestos from twelve countries, written in ten\ndifferent languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02914v1",
            "date": "2018-05-08 09:24:32+00:00",
            "title": "One \"Ruler\" for All Languages: Multi-Lingual Dialogue Evaluation with Adversarial Multi-Task Learning",
            "authors": [
                "Xiaowei Tong",
                "Zhenxin Fu",
                "Mingyue Shang",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Automatic evaluating the performance of Open-domain dialogue system is a\nchallenging problem. Recent work in neural network-based metrics has shown\npromising opportunities for automatic dialogue evaluation. However, existing\nmethods mainly focus on monolingual evaluation, in which the trained metric is\nnot flexible enough to transfer across different languages. To address this\nissue, we propose an adversarial multi-task neural metric (ADVMT) for\nmulti-lingual dialogue evaluation, with shared feature extraction across\nlanguages. We evaluate the proposed model in two different languages.\nExperiments show that the adversarial multi-task neural metric achieves a high\ncorrelation with human annotation, which yields better performance than\nmonolingual ones and various existing metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02937v1",
            "date": "2018-05-08 10:39:45+00:00",
            "title": "Improving Character-level Japanese-Chinese Neural Machine Translation with Radicals as an Additional Input Feature",
            "authors": [
                "Jinyi Zhang",
                "Tadahiro Matsumoto"
            ],
            "abstract": "In recent years, Neural Machine Translation (NMT) has been proven to get\nimpressive results. While some additional linguistic features of input words\nimprove word-level NMT, any additional character features have not been used to\nimprove character-level NMT so far. In this paper, we show that the radicals of\nChinese characters (or kanji), as a character feature information, can be\neasily provide further improvements in the character-level NMT. In experiments\non WAT2016 Japanese-Chinese scientific paper excerpt corpus (ASPEC-JP), we find\nthat the proposed method improves the translation quality according to two\naspects: perplexity and BLEU. The character-level NMT with the radical input\nfeature's model got a state-of-the-art result of 40.61 BLEU points in the test\nset, which is an improvement of about 8.6 BLEU points over the best system on\nthe WAT2016 Japanese-to-Chinese translation subtask with ASPEC-JP. The\nimprovements over the character-level NMT with no additional input feature are\nup to about 1.5 and 1.4 BLEU points in the development-test set and the test\nset of the corpus, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03122v1",
            "date": "2018-05-08 15:50:32+00:00",
            "title": "Bleaching Text: Abstract Features for Cross-lingual Gender Prediction",
            "authors": [
                "Rob van der Goot",
                "Nikola Ljube\u0161i\u0107",
                "Ian Matroos",
                "Malvina Nissim",
                "Barbara Plank"
            ],
            "abstract": "Gender prediction has typically focused on lexical and social network\nfeatures, yielding good performance, but making systems highly language-,\ntopic-, and platform-dependent. Cross-lingual embeddings circumvent some of\nthese limitations, but capture gender-specific style less. We propose an\nalternative: bleaching text, i.e., transforming lexical strings into more\nabstract features. This study provides evidence that such features allow for\nbetter transfer across languages. Moreover, we present a first study on the\nability of humans to perform cross-lingual gender prediction. We find that\nhuman predictive power proves similar to that of our bleached models, and both\nperform better than lexical models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03228v1",
            "date": "2018-05-08 18:46:24+00:00",
            "title": "Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources",
            "authors": [
                "Ivan Vuli\u0107",
                "Goran Glava\u0161",
                "Nikola Mrk\u0161i\u0107",
                "Anna Korhonen"
            ],
            "abstract": "Word vector specialisation (also known as retrofitting) is a portable,\nlight-weight approach to fine-tuning arbitrary distributional word vector\nspaces by injecting external knowledge from rich lexical resources such as\nWordNet. By design, these post-processing methods only update the vectors of\nwords occurring in external lexicons, leaving the representations of all unseen\nwords intact. In this paper, we show that constraint-driven vector space\nspecialisation can be extended to unseen words. We propose a novel\npost-specialisation method that: a) preserves the useful linguistic knowledge\nfor seen words; while b) propagating this external signal to unseen words in\norder to improve their vector representations as well. Our post-specialisation\napproach explicits a non-linear specialisation function in the form of a deep\nneural network by learning to predict specialised vectors from their original\ndistributional counterparts. The learned function is then used to specialise\nvectors of unseen words. This approach, applicable to any post-processing\nmodel, yields considerable gains over the initial specialisation models both in\nintrinsic word similarity tasks, and in two downstream tasks: dialogue state\ntracking and lexical text simplification. The positive effects persist across\nthree languages, demonstrating the importance of specialising the full\nvocabulary of distributional word vector spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03257v1",
            "date": "2018-05-08 19:54:47+00:00",
            "title": "Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog",
            "authors": [
                "Jiaping Zhang",
                "Tiancheng Zhao",
                "Zhou Yu"
            ],
            "abstract": "Creating an intelligent conversational system that understands vision and\nlanguage is one of the ultimate goals in Artificial Intelligence\n(AI)~\\cite{winograd1972understanding}. Extensive research has focused on\nvision-to-language generation, however, limited research has touched on\ncombining these two modalities in a goal-driven dialog context. We propose a\nmultimodal hierarchical reinforcement learning framework that dynamically\nintegrates vision and language for task-oriented visual dialog. The framework\njointly learns the multimodal dialog state representation and the hierarchical\ndialog policy to improve both dialog task success and efficiency. We also\npropose a new technique, state adaptation, to integrate context awareness in\nthe dialog state representation. We evaluate the proposed framework and the\nstate adaptation technique in an image guessing game and achieve promising\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03330v2",
            "date": "2018-05-09 00:44:59+00:00",
            "title": "Character-level Chinese-English Translation through ASCII Encoding",
            "authors": [
                "Nikola I. Nikolov",
                "Yuhuang Hu",
                "Mi Xue Tan",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "Character-level Neural Machine Translation (NMT) models have recently\nachieved impressive results on many language pairs. They mainly do well for\nIndo-European language pairs, where the languages share the same writing\nsystem. However, for translating between Chinese and English, the gap between\nthe two different writing systems poses a major challenge because of a lack of\nsystematic correspondence between the individual linguistic units. In this\npaper, we enable character-level NMT for Chinese, by breaking down Chinese\ncharacters into linguistic units similar to that of Indo-European languages. We\nuse the Wubi encoding scheme, which preserves the original shape and semantic\ninformation of the characters, while also being reversible. We show promising\nresults from training Wubi-based models on the character- and subword-level\nwith recurrent as well as convolutional models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03366v1",
            "date": "2018-05-09 04:05:37+00:00",
            "title": "LearningWord Embeddings for Low-resource Languages by PU Learning",
            "authors": [
                "Chao Jiang",
                "Hsiang-Fu Yu",
                "Cho-Jui Hsieh",
                "Kai-Wei Chang"
            ],
            "abstract": "Word embedding is a key component in many downstream applications in\nprocessing natural languages. Existing approaches often assume the existence of\na large collection of text for learning effective word embedding. However, such\na corpus may not be available for some low-resource languages. In this paper,\nwe study how to effectively learn a word embedding model on a corpus with only\na few million tokens. In such a situation, the co-occurrence matrix is sparse\nas the co-occurrences of many word pairs are unobserved. In contrast to\nexisting approaches often only sample a few unobserved word pairs as negative\nsamples, we argue that the zero entries in the co-occurrence matrix also\nprovide valuable information. We then design a Positive-Unlabeled Learning\n(PU-Learning) approach to factorize the co-occurrence matrix and validate the\nproposed approaches in four different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03645v1",
            "date": "2018-05-09 07:56:06+00:00",
            "title": "Three tree priors and five datasets: A study of the effect of tree priors in Indo-European phylogenetics",
            "authors": [
                "Taraka Rama"
            ],
            "abstract": "The age of the root of the Indo-European language family has received much\nattention since the application of Bayesian phylogenetic methods by Gray and\nAtkinson(2003). The root age of the Indo-European family has tended to decrease\nfrom an age that supported the Anatolian origin hypothesis to an age that\nsupports the Steppe origin hypothesis with the application of new models (Chang\net al., 2015). However, none of the published work in the Indo-European\nphylogenetics studied the effect of tree priors on phylogenetic analyses of the\nIndo-European family. In this paper, I intend to fill this gap by exploring the\neffect of tree priors on different aspects of the Indo-European family's\nphylogenetic inference. I apply three tree priors---Uniform, Fossilized\nBirth-Death (FBD), and Coalescent---to five publicly available datasets of the\nIndo-European language family. I evaluate the posterior distribution of the\ntrees from the Bayesian analysis using Bayes Factor, and find that there is\nsupport for the Steppe origin hypothesis in the case of two tree priors. I\nreport the median and 95% highest posterior density (HPD) interval of the root\nages for all the three tree priors. A model comparison suggested that either\nUniform prior or FBD prior is more suitable than the Coalescent prior to the\ndatasets belonging to the Indo-European language family.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03668v2",
            "date": "2018-05-09 18:00:15+00:00",
            "title": "Automatic Article Commenting: the Task and Dataset",
            "authors": [
                "Lianhui Qin",
                "Lemao Liu",
                "Victoria Bi",
                "Yan Wang",
                "Xiaojiang Liu",
                "Zhiting Hu",
                "Hai Zhao",
                "Shuming Shi"
            ],
            "abstract": "Comments of online articles provide extended views and improve user\nengagement. Automatically making comments thus become a valuable functionality\nfor online forums, intelligent chatbots, etc. This paper proposes the new task\nof automatic article commenting, and introduces a large-scale Chinese dataset\nwith millions of real comments and a human-annotated subset characterizing the\ncomments' varying quality. Incorporating the human bias of comment quality, we\nfurther develop automatic metrics that generalize a broad set of popular\nreference-based metrics and exhibit greatly improved correlations with human\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03710v1",
            "date": "2018-05-09 19:54:58+00:00",
            "title": "Incorporating Subword Information into Matrix Factorization Word Embeddings",
            "authors": [
                "Alexandre Salle",
                "Aline Villavicencio"
            ],
            "abstract": "The positive effect of adding subword information to word embeddings has been\ndemonstrated for predictive models. In this paper we investigate whether\nsimilar benefits can also be derived from incorporating subwords into counting\nmodels. We evaluate the impact of different types of subwords (n-grams and\nunsupervised morphemes), with results confirming the importance of subword\ninformation in learning representations of rare and out-of-vocabulary words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03750v1",
            "date": "2018-05-09 22:47:30+00:00",
            "title": "Neural Machine Translation Decoding with Terminology Constraints",
            "authors": [
                "Eva Hasler",
                "Adri\u00e0 De Gispert",
                "Gonzalo Iglesias",
                "Bill Byrne"
            ],
            "abstract": "Despite the impressive quality improvements yielded by neural machine\ntranslation (NMT) systems, controlling their translation output to adhere to\nuser-provided terminology constraints remains an open problem. We describe our\napproach to constrained neural decoding based on finite-state machines and\nmulti-stack decoding which supports target-side constraints as well as\nconstraints with corresponding aligned input text spans. We demonstrate the\nperformance of our framework on multiple translation tasks and motivate the\nneed for constrained decoding with attentions as a means of reducing\nmisplacement and duplication when translating user constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03766v1",
            "date": "2018-05-10 00:51:06+00:00",
            "title": "Discourse-Aware Neural Rewards for Coherent Text Generation",
            "authors": [
                "Antoine Bosselut",
                "Asli Celikyilmaz",
                "Xiaodong He",
                "Jianfeng Gao",
                "Po-Sen Huang",
                "Yejin Choi"
            ],
            "abstract": "In this paper, we investigate the use of discourse-aware rewards with\nreinforcement learning to guide a model to generate long, coherent text. In\nparticular, we propose to learn neural rewards to model cross-sentence ordering\nas a means to approximate desired discourse structure. Empirical results\ndemonstrate that a generator trained with the learned reward produces more\ncoherent and less repetitive text than models trained with cross-entropy or\nwith reinforcement learning with commonly used scores as rewards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03774v1",
            "date": "2018-05-10 01:27:58+00:00",
            "title": "The Evolution of Popularity and Images of Characters in Marvel Cinematic Universe Fanfictions",
            "authors": [
                "Fan Bu"
            ],
            "abstract": "This analysis proposes a new topic model to study the yearly trends in Marvel\nCinematic Universe fanfictions on three levels: character popularity, character\nimages/topics, and vocabulary pattern of topics. It is found that character\nappearances in fanfictions have become more diverse over the years thanks to\nconstant introduction of new characters in feature films, and in the case of\nCaptain America, multi-dimensional character development is well-received by\nthe fanfiction world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03784v1",
            "date": "2018-05-10 02:07:02+00:00",
            "title": "SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue Systems",
            "authors": [
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Shereen Oraby",
                "Amita Misra",
                "Marilyn Walker"
            ],
            "abstract": "In dialogue systems, the tasks of named entity recognition (NER) and named\nentity linking (NEL) are vital preprocessing steps for understanding user\nintent, especially in open domain interaction where we cannot rely on\ndomain-specific inference. UCSC's effort as one of the funded teams in the 2017\nAmazon Alexa Prize Contest has yielded Slugbot, an open domain social bot,\naimed at casual conversation. We discovered several challenges specifically\nassociated with both NER and NEL when building Slugbot, such as that the NE\nlabels are too coarse-grained or the entity types are not linked to a useful\nontology. Moreover, we have discovered that traditional approaches do not\nperform well in our context: even systems designed to operate on tweets or\nother social media data do not work well in dialogue systems. In this paper, we\nintroduce Slugbot's Named Entity Recognition for dialogue Systems (SlugNERDS),\na NER and NEL tool which is optimized to address these issues. We describe two\nnew resources that we are building as part of this work: SlugEntityDB and\nSchemaActuator. We believe these resources will be useful for the research\ncommunity.",
            "journal ref": "Kevin K. Bowden, Jiaqi Wu, Shereen Oraby, Amita Misra, and Marilyn\n  Walker. SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue\n  Systems. Language Resources and Evaluation Conference (LREC), Miyazaki,\n  Japan, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03818v4",
            "date": "2018-05-10 04:59:59+00:00",
            "title": "Training Classifiers with Natural Language Explanations",
            "authors": [
                "Braden Hancock",
                "Paroma Varma",
                "Stephanie Wang",
                "Martin Bringmann",
                "Percy Liang",
                "Christopher R\u00e9"
            ],
            "abstract": "Training accurate classifiers requires many labels, but each label provides\nonly limited information (one bit for binary classification). In this work, we\npropose BabbleLabble, a framework for training classifiers in which an\nannotator provides a natural language explanation for each labeling decision. A\nsemantic parser converts these explanations into programmatic labeling\nfunctions that generate noisy labels for an arbitrary amount of unlabeled data,\nwhich is used to train a classifier. On three relation extraction tasks, we\nfind that users are able to train classifiers with comparable F1 scores from\n5-100$\\times$ faster by providing explanations instead of just labels.\nFurthermore, given the inherent imperfection of labeling functions, we find\nthat a simple rule-based semantic parser suffices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03838v1",
            "date": "2018-05-10 06:14:36+00:00",
            "title": "Hybrid semi-Markov CRF for Neural Sequence Labeling",
            "authors": [
                "Zhi-Xiu Ye",
                "Zhen-Hua Ling"
            ],
            "abstract": "This paper proposes hybrid semi-Markov conditional random fields (SCRFs) for\nneural sequence labeling in natural language processing. Based on conventional\nconditional random fields (CRFs), SCRFs have been designed for the tasks of\nassigning labels to segments by extracting features from and describing\ntransitions between segments instead of words. In this paper, we improve the\nexisting SCRF methods by employing word-level and segment-level information\nsimultaneously. First, word-level labels are utilized to derive the segment\nscores in SCRFs. Second, a CRF output layer and an SCRF output layer are\nintegrated into an unified neural network and trained jointly. Experimental\nresults on CoNLL 2003 named entity recognition (NER) shared task show that our\nmodel achieves state-of-the-art performance when no external knowledge is used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03871v1",
            "date": "2018-05-10 08:04:17+00:00",
            "title": "Obligation and Prohibition Extraction Using Hierarchical RNNs",
            "authors": [
                "Ilias Chalkidis",
                "Ion Androutsopoulos",
                "Achilleas Michos"
            ],
            "abstract": "We consider the task of detecting contractual obligations and prohibitions.\nWe show that a self-attention mechanism improves the performance of a BILSTM\nclassifier, the previous state of the art for this task, by allowing it to\nfocus on indicative tokens. We also introduce a hierarchical BILSTM, which\nconverts each sentence to an embedding, and processes the sentence embeddings\nto classify each sentence. Apart from being faster to train, the hierarchical\nBILSTM outperforms the flat one, even when the latter considers surrounding\nsentences, because the hierarchical model has a broader discourse view.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03900v1",
            "date": "2018-05-10 09:22:56+00:00",
            "title": "Improv Chat: Second Response Generation for Chatbot",
            "authors": [
                "Furu Wei"
            ],
            "abstract": "Existing research on response generation for chatbot focuses on \\textbf{First\nResponse Generation} which aims to teach the chatbot to say the first response\n(e.g. a sentence) appropriate to the conversation context (e.g. the user's\nquery). In this paper, we introduce a new task \\textbf{Second Response\nGeneration}, termed as Improv chat, which aims to teach the chatbot to say the\nsecond response after saying the first response with respect the conversation\ncontext, so as to lighten the burden on the user to keep the conversation\ngoing. Specifically, we propose a general learning based framework and develop\na retrieval based system which can generate the second responses with the\nusers' query and the chatbot's first response as input. We present the approach\nto building the conversation corpus for Improv chat from public forums and\nsocial networks, as well as the neural networks based models for response\nmatching and ranking. We include the preliminary experiments and results in\nthis paper. This work could be further advanced with better deep matching\nmodels for retrieval base systems or generative models for generation based\nsystems as well as extensive evaluations in real-life applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03977v1",
            "date": "2018-05-10 13:42:29+00:00",
            "title": "Automatic Academic Paper Rating Based on Modularized Hierarchical Convolutional Neural Network",
            "authors": [
                "Pengcheng Yang",
                "Xu Sun",
                "Wei Li",
                "Shuming Ma"
            ],
            "abstract": "As more and more academic papers are being submitted to conferences and\njournals, evaluating all these papers by professionals is time-consuming and\ncan cause inequality due to the personal factors of the reviewers. In this\npaper, in order to assist professionals in evaluating academic papers, we\npropose a novel task: automatic academic paper rating (AAPR), which\nautomatically determine whether to accept academic papers. We build a new\ndataset for this task and propose a novel modularized hierarchical\nconvolutional neural network to achieve automatic academic paper rating.\nEvaluation results show that the proposed model outperforms the baselines by a\nlarge margin. The dataset and code are available at\n\\url{https://github.com/lancopku/AAPR}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04016v2",
            "date": "2018-05-10 15:07:22+00:00",
            "title": "Automatic Estimation of Simultaneous Interpreter Performance",
            "authors": [
                "Craig Stewart",
                "Nikolai Vogler",
                "Junjie Hu",
                "Jordan Boyd-Graber",
                "Graham Neubig"
            ],
            "abstract": "Simultaneous interpretation, translation of the spoken word in real-time, is\nboth highly challenging and physically demanding. Methods to predict\ninterpreter confidence and the adequacy of the interpreted message have a\nnumber of potential applications, such as in computer-assisted interpretation\ninterfaces or pedagogical tools. We propose the task of predicting simultaneous\ninterpreter performance by building on existing methodology for quality\nestimation (QE) of machine translation output. In experiments over five\nsettings in three language pairs, we extend a QE pipeline to estimate\ninterpreter performance (as approximated by the METEOR evaluation metric) and\npropose novel features reflecting interpretation strategy and evaluation\nmeasures that further improve prediction accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04033v1",
            "date": "2018-05-10 15:58:09+00:00",
            "title": "Regularizing Output Distribution of Abstractive Chinese Social Media Text Summarization for Improved Semantic Consistency",
            "authors": [
                "Bingzhen Wei",
                "Xuancheng Ren",
                "Xu Sun",
                "Yi Zhang",
                "Xiaoyan Cai",
                "Qi Su"
            ],
            "abstract": "Abstractive text summarization is a highly difficult problem, and the\nsequence-to-sequence model has shown success in improving the performance on\nthe task. However, the generated summaries are often inconsistent with the\nsource content in semantics. In such cases, when generating summaries, the\nmodel selects semantically unrelated words with respect to the source content\nas the most probable output. The problem can be attributed to heuristically\nconstructed training data, where summaries can be unrelated to the source\ncontent, thus containing semantically unrelated words and spurious word\ncorrespondence. In this paper, we propose a regularization approach for the\nsequence-to-sequence model and make use of what the model has learned to\nregularize the learning objective to alleviate the effect of the problem. In\naddition, we propose a practical human evaluation method to address the problem\nthat the existing automatic evaluation method does not evaluate the semantic\nconsistency with the source content properly. Experimental results demonstrate\nthe effectiveness of the proposed approach, which outperforms almost all the\nexisting models. Especially, the proposed approach improves the semantic\nconsistency by 4\\% in terms of human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04044v1",
            "date": "2018-05-10 16:19:14+00:00",
            "title": "End-to-End Reinforcement Learning for Automatic Taxonomy Induction",
            "authors": [
                "Yuning Mao",
                "Xiang Ren",
                "Jiaming Shen",
                "Xiaotao Gu",
                "Jiawei Han"
            ],
            "abstract": "We present a novel end-to-end reinforcement learning approach to automatic\ntaxonomy induction from a set of terms. While prior methods treat the problem\nas a two-phase task (i.e., detecting hypernymy pairs followed by organizing\nthese pairs into a tree-structured hierarchy), we argue that such two-phase\nmethods may suffer from error propagation, and cannot effectively optimize\nmetrics that capture the holistic structure of a taxonomy. In our approach, the\nrepresentations of term pairs are learned using multiple sources of information\nand used to determine \\textit{which} term to select and \\textit{where} to place\nit on the taxonomy via a policy network. All components are trained in an\nend-to-end manner with cumulative rewards, measured by a holistic tree metric\nover the training taxonomies. Experiments on two public datasets of different\ndomains show that our approach outperforms prior state-of-the-art taxonomy\ninduction methods up to 19.6\\% on ancestor F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04185v1",
            "date": "2018-05-10 21:55:32+00:00",
            "title": "Deep Neural Machine Translation with Weakly-Recurrent Units",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Marcello Federico"
            ],
            "abstract": "Recurrent neural networks (RNNs) have represented for years the state of the\nart in neural machine translation. Recently, new architectures have been\nproposed, which can leverage parallel computation on GPUs better than classical\nRNNs. Faster training and inference combined with different\nsequence-to-sequence modeling also lead to performance improvements. While the\nnew models completely depart from the original recurrent architecture, we\ndecided to investigate how to make RNNs more efficient. In this work, we\npropose a new recurrent NMT architecture, called Simple Recurrent NMT, built on\na class of fast and weakly-recurrent units that use layer normalization and\nmultiple attentions. Our experiments on the WMT14 English-to-German and WMT16\nEnglish-Romanian benchmarks show that our model represents a valid alternative\nto LSTMs, as it can achieve better results at a significantly lower\ncomputational cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04218v1",
            "date": "2018-05-11 01:34:52+00:00",
            "title": "Deep RNNs Encode Soft Hierarchical Syntax",
            "authors": [
                "Terra Blevins",
                "Omer Levy",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present a set of experiments to demonstrate that deep recurrent neural\nnetworks (RNNs) learn internal representations that capture soft hierarchical\nnotions of syntax from highly varied supervision. We consider four syntax tasks\nat different depths of the parse tree; for each word, we predict its part of\nspeech as well as the first (parent), second (grandparent) and third level\n(great-grandparent) constituent labels that appear above it. These predictions\nare made from representations produced at different depths in networks that are\npretrained with one of four objectives: dependency parsing, semantic role\nlabeling, machine translation, or language modeling. In every case, we find a\ncorrespondence between network depth and syntactic depth, suggesting that a\nsoft syntactic hierarchy emerges. This effect is robust across all conditions,\nindicating that the models encode significant amounts of syntax even in the\nabsence of an explicit syntactic training supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04237v1",
            "date": "2018-05-11 03:36:32+00:00",
            "title": "Neural Machine Translation for Bilingually Scarce Scenarios: A Deep Multi-task Learning Approach",
            "authors": [
                "Poorya Zaremoodi",
                "Gholamreza Haffari"
            ],
            "abstract": "Neural machine translation requires large amounts of parallel training text\nto learn a reasonable-quality translation model. This is particularly\ninconvenient for language pairs for which enough parallel text is not\navailable. In this paper, we use monolingual linguistic resources in the source\nside to address this challenging problem based on a multi-task learning\napproach. More specifically, we scaffold the machine translation task on\nauxiliary tasks including semantic parsing, syntactic parsing, and named-entity\nrecognition. This effectively injects semantic and/or syntactic knowledge into\nthe translation model, which would otherwise require a large amount of training\nbitext. We empirically evaluate and show the effectiveness of our multi-task\nlearning approach on three translation tasks: English-to-French,\nEnglish-to-Farsi, and English-to-Vietnamese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04270v1",
            "date": "2018-05-11 08:17:49+00:00",
            "title": "Neural Open Information Extraction",
            "authors": [
                "Lei Cui",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Conventional Open Information Extraction (Open IE) systems are usually built\non hand-crafted patterns from other NLP tools such as syntactic parsing, yet\nthey face problems of error propagation. In this paper, we propose a neural\nOpen IE approach with an encoder-decoder framework. Distinct from existing\nmethods, the neural Open IE approach learns highly confident arguments and\nrelation tuples bootstrapped from a state-of-the-art Open IE system. An\nempirical study on a large benchmark dataset shows that the neural Open IE\nsystem significantly outperforms several baselines, while maintaining\ncomparable computational efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04277v1",
            "date": "2018-05-11 08:46:15+00:00",
            "title": "The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD",
            "authors": [
                "Eneko Agirre",
                "Oier L\u00f3pez de Lacalle",
                "Aitor Soroa"
            ],
            "abstract": "UKB is an open source collection of programs for performing, among other\ntasks, knowledge-based Word Sense Disambiguation (WSD). Since it was released\nin 2009 it has been often used out-of-the-box in sub-optimal settings. We show\nthat nine years later it is the state-of-the-art on knowledge-based WSD. This\ncase shows the pitfalls of releasing open source NLP software without optimal\ndefault settings and precise instructions for reproducibility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04453v1",
            "date": "2018-05-11 15:42:27+00:00",
            "title": "Bootstrapping Multilingual Intent Models via Machine Translation for Dialog Automation",
            "authors": [
                "Nicholas Ruiz",
                "Srinivas Bangalore",
                "John Chen"
            ],
            "abstract": "With the resurgence of chat-based dialog systems in consumer and enterprise\napplications, there has been much success in developing data-driven and\nrule-based natural language models to understand human intent. Since these\nmodels require large amounts of data and in-domain knowledge, expanding an\nequivalent service into new markets is disrupted by language barriers that\ninhibit dialog automation.\n  This paper presents a user study to evaluate the utility of out-of-the-box\nmachine translation technology to (1) rapidly bootstrap multilingual spoken\ndialog systems and (2) enable existing human analysts to understand foreign\nlanguage utterances. We additionally evaluate the utility of machine\ntranslation in human assisted environments, where a portion of the traffic is\nprocessed by analysts. In English->Spanish experiments, we observe a high\npotential for dialog automation, as well as the potential for human analysts to\nprocess foreign language utterances with high accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04508v1",
            "date": "2018-05-11 17:57:40+00:00",
            "title": "Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad"
            ],
            "abstract": "Automatic machine learning systems can inadvertently accentuate and\nperpetuate inappropriate human biases. Past work on examining inappropriate\nbiases has largely focused on just individual systems. Further, there is no\nbenchmark dataset for examining inappropriate biases in systems. Here for the\nfirst time, we present the Equity Evaluation Corpus (EEC), which consists of\n8,640 English sentences carefully chosen to tease out biases towards certain\nraces and genders. We use the dataset to examine 219 automatic sentiment\nanalysis systems that took part in a recent shared task, SemEval-2018 Task 1\n'Affect in Tweets'. We find that several of the systems show statistically\nsignificant bias; that is, they consistently provide slightly higher sentiment\nintensity predictions for one race or one gender. We make the EEC freely\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04542v1",
            "date": "2018-05-11 18:16:54+00:00",
            "title": "Sentiment Composition of Words with Opposing Polarities",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad"
            ],
            "abstract": "In this paper, we explore sentiment composition in phrases that have at least\none positive and at least one negative word---phrases like 'happy accident' and\n'best winter break'. We compiled a dataset of such opposing polarity phrases\nand manually annotated them with real-valued scores of sentiment association.\nUsing this dataset, we analyze the linguistic patterns present in opposing\npolarity phrases. Finally, we apply several unsupervised and supervised\ntechniques of sentiment composition to determine their efficacy on this\ndataset. Our best system, which incorporates information from the phrase's\nconstituents, their parts of speech, their sentiment association scores, and\ntheir embedding vectors, obtains an accuracy of over 80% on the opposing\npolarity phrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04558v1",
            "date": "2018-05-11 19:01:49+00:00",
            "title": "NRC-Canada at SMM4H Shared Task: Classifying Tweets Mentioning Adverse Drug Reactions and Medication Intake",
            "authors": [
                "Svetlana Kiritchenko",
                "Saif M. Mohammad",
                "Jason Morin",
                "Berry de Bruijn"
            ],
            "abstract": "Our team, NRC-Canada, participated in two shared tasks at the AMIA-2017\nWorkshop on Social Media Mining for Health Applications (SMM4H): Task 1 -\nclassification of tweets mentioning adverse drug reactions, and Task 2 -\nclassification of tweets describing personal medication intake. For both tasks,\nwe trained Support Vector Machine classifiers using a variety of surface-form,\nsentiment, and domain-specific features. With nine teams participating in each\ntask, our submissions ranked first on Task 1 and third on Task 2. Handling\nconsiderable class imbalance proved crucial for Task 1. We applied an\nunder-sampling technique to reduce class imbalance (from about 1:10 to 1:2).\nStandard n-gram features, n-grams generalized over domain terms, as well as\ngeneral-domain and domain-specific word embeddings had a substantial impact on\nthe overall performance in both tasks. On the other hand, including sentiment\nlexicon features did not result in any improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04570v3",
            "date": "2018-05-11 19:27:07+00:00",
            "title": "Neural Factor Graph Models for Cross-lingual Morphological Tagging",
            "authors": [
                "Chaitanya Malaviya",
                "Matthew R. Gormley",
                "Graham Neubig"
            ],
            "abstract": "Morphological analysis involves predicting the syntactic traits of a word\n(e.g. {POS: Noun, Case: Acc, Gender: Fem}). Previous work in morphological\ntagging improves performance for low-resource languages (LRLs) through\ncross-lingual training with a high-resource language (HRL) from the same\nfamily, but is limited by the strict, often false, assumption that tag sets\nexactly overlap between the HRL and LRL. In this paper we propose a method for\ncross-lingual morphological tagging that aims to improve information sharing\nbetween languages by relaxing this assumption. The proposed model uses\nfactorial conditional random fields with neural network potentials, making it\npossible to (1) utilize the expressive power of neural network representations\nto smooth over superficial differences in the surface forms, (2) model pairwise\nand transitive relationships between tags, and (3) accurately generate tag sets\nthat are unseen or rare in the training data. Experiments on four languages\nfrom the Universal Dependencies Treebank demonstrate superior tagging\naccuracies over existing cross-lingual approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04576v1",
            "date": "2018-05-11 19:58:59+00:00",
            "title": "Domain Adapted Word Embeddings for Improved Sentiment Classification",
            "authors": [
                "Prathusha K Sarma",
                "YIngyu Liang",
                "William A Sethares"
            ],
            "abstract": "Generic word embeddings are trained on large-scale generic corpora; Domain\nSpecific (DS) word embeddings are trained only on data from a domain of\ninterest. This paper proposes a method to combine the breadth of generic\nembeddings with the specificity of domain specific embeddings. The resulting\nembeddings, called Domain Adapted (DA) word embeddings, are formed by aligning\ncorresponding word vectors using Canonical Correlation Analysis (CCA) or the\nrelated nonlinear Kernel CCA. Evaluation results on sentiment classification\ntasks show that the DA embeddings substantially outperform both generic and DS\nembeddings when used as input features to standard or state-of-the-art sentence\nencoding algorithms for classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04579v2",
            "date": "2018-05-11 20:15:52+00:00",
            "title": "Using Statistical and Semantic Models for Multi-Document Summarization",
            "authors": [
                "Divyanshu Daiya",
                "Anukarsh Singh",
                "Mukesh Jadon"
            ],
            "abstract": "We report a series of experiments with different semantic models on top of\nvarious statistical models for extractive text summarization. Though\nstatistical models may better capture word co-occurrences and distribution\naround the text, they fail to detect the context and the sense of sentences\n/words as a whole. Semantic models help us gain better insight into the context\nof sentences. We show that how tuning weights between different models can help\nus achieve significant results on various benchmarks. Learning pre-trained\nvectors used in semantic models further, on given corpus, can give addition\nspike in performance. Using weighing techniques in between different\nstatistical models too further refines our result. For Statistical models, we\nhave used TF/IDF, TextRAnk, Jaccard/Cosine Similarities. For Semantic Models,\nwe have used WordNet-based Model and proposed two models based on Glove Vectors\nand Facebook's InferSent. We tested our approach on DUC 2004 dataset,\ngenerating 100-word summaries. We have discussed the system, algorithms,\nanalysis and also proposed and tested possible improvements. ROUGE scores were\nused to compare to other summarizers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04601v1",
            "date": "2018-05-11 21:53:23+00:00",
            "title": "Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "One key task of fine-grained sentiment analysis of product reviews is to\nextract product aspects or features that users have expressed opinions on. This\npaper focuses on supervised aspect extraction using deep learning. Unlike other\nhighly sophisticated supervised deep learning models, this paper proposes a\nnovel and yet simple CNN model employing two types of pre-trained embeddings\nfor aspect extraction: general-purpose embeddings and domain-specific\nembeddings. Without using any additional supervision, this model achieves\nsurprisingly good results, outperforming state-of-the-art sophisticated\nexisting methods. To our knowledge, this paper is the first to report such\ndouble embeddings based CNN model for aspect extraction and achieve very good\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04604v1",
            "date": "2018-05-11 22:09:37+00:00",
            "title": "Confidence Modeling for Neural Semantic Parsing",
            "authors": [
                "Li Dong",
                "Chris Quirk",
                "Mirella Lapata"
            ],
            "abstract": "In this work we focus on confidence modeling for neural semantic parsers\nwhich are built upon sequence-to-sequence models. We outline three major causes\nof uncertainty, and design various metrics to quantify these factors. These\nmetrics are then used to estimate confidence scores that indicate whether model\npredictions are likely to be correct. Beyond confidence estimation, we identify\nwhich parts of the input contribute to uncertain predictions allowing users to\ninterpret their model, and verify or refine its input. Experimental results\nshow that our confidence model significantly outperforms a widely used method\nthat relies on posterior probability, and improves the quality of\ninterpretation compared to simply relying on attention scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04617v1",
            "date": "2018-05-11 23:13:34+00:00",
            "title": "TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation",
            "authors": [
                "Alexander R. Fabbri",
                "Irene Li",
                "Prawat Trairatvorakul",
                "Yijiao He",
                "Wei Tai Ting",
                "Robert Tung",
                "Caitlin Westerfield",
                "Dragomir R. Radev"
            ],
            "abstract": "The field of Natural Language Processing (NLP) is growing rapidly, with new\nresearch published daily along with an abundance of tutorials, codebases and\nother online resources. In order to learn this dynamic field or stay up-to-date\non the latest research, students as well as educators and researchers must\nconstantly sift through multiple sources to find valuable, relevant\ninformation. To address this situation, we introduce TutorialBank, a new,\npublicly available dataset which aims to facilitate NLP education and research.\nWe have manually collected and categorized over 6,300 resources on NLP as well\nas the related fields of Artificial Intelligence (AI), Machine Learning (ML)\nand Information Retrieval (IR). Our dataset is notably the largest\nmanually-picked corpus of resources intended for NLP education which does not\ninclude only academic papers. Additionally, we have created both a search\nengine and a command-line tool for the resources and have annotated the corpus\nto include lists of research topics, relevant resources for each topic,\nprerequisite relations among topics, relevant sub-parts of individual\nresources, among other annotations. We are releasing the dataset and present\nseveral avenues for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04623v1",
            "date": "2018-05-12 00:26:29+00:00",
            "title": "Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context",
            "authors": [
                "Urvashi Khandelwal",
                "He He",
                "Peng Qi",
                "Dan Jurafsky"
            ],
            "abstract": "We know very little about how neural language models (LM) use prior\nlinguistic context. In this paper, we investigate the role of context in an\nLSTM LM, through ablation studies. Specifically, we analyze the increase in\nperplexity when prior context words are shuffled, replaced, or dropped. On two\nstandard datasets, Penn Treebank and WikiText-2, we find that the model is\ncapable of using about 200 tokens of context on average, but sharply\ndistinguishes nearby context (recent 50 tokens) from the distant history. The\nmodel is highly sensitive to the order of words within the most recent\nsentence, but ignores word order in the long-range context (beyond 50 tokens),\nsuggesting the distant past is modeled only as a rough semantic field or topic.\nWe further find that the neural caching model (Grave et al., 2017b) especially\nhelps the LSTM to copy words from within this distant context. Overall, our\nanalysis not only provides a better understanding of how neural LMs use their\ncontext, but also sheds light on recent success from cache-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04655v2",
            "date": "2018-05-12 05:11:07+00:00",
            "title": "Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information",
            "authors": [
                "Sudha Rao",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "Inquiry is fundamental to communication, and machines cannot effectively\ncollaborate with humans unless they can ask questions. In this work, we build a\nneural network model for the task of ranking clarification questions. Our model\nis inspired by the idea of expected value of perfect information: a good\nquestion is one whose expected answer will be useful. We study this problem\nusing data from StackExchange, a plentiful online resource in which people\nroutinely ask clarifying questions to posts so that they can better offer\nassistance to the original poster. We create a dataset of clarification\nquestions consisting of ~77K posts paired with a clarification question (and\nanswer) from three domains of StackExchange: askubuntu, unix and superuser. We\nevaluate our model on 500 samples of this dataset against expert human\njudgments and demonstrate significant improvements over controlled baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04658v1",
            "date": "2018-05-12 05:27:45+00:00",
            "title": "Backpropagating through Structured Argmax using a SPIGOT",
            "authors": [
                "Hao Peng",
                "Sam Thomson",
                "Noah A. Smith"
            ],
            "abstract": "We introduce the structured projection of intermediate gradients optimization\ntechnique (SPIGOT), a new method for backpropagating through neural networks\nthat include hard-decision structured predictions (e.g., parsing) in\nintermediate layers. SPIGOT requires no marginal inference, unlike structured\nattention networks (Kim et al., 2017) and some reinforcement learning-inspired\nsolutions (Yogatama et al., 2017). Like so-called straight-through estimators\n(Hinton, 2012), SPIGOT defines gradient-like quantities associated with\nintermediate nondifferentiable operations, allowing backpropagation before and\nafter them; SPIGOT's proxy aims to ensure that, after a parameter update, the\nintermediate structure will remain well-formed.\n  We experiment on two structured NLP pipelines: syntactic-then-semantic\ndependency parsing, and semantic parsing followed by sentiment classification.\nWe show that training with SPIGOT leads to a larger improvement on the\ndownstream task than a modularly-trained pipeline, the straight-through\nestimator, and structured attention, reaching a new state of the art on\nsemantic dependency parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04685v1",
            "date": "2018-05-12 08:25:33+00:00",
            "title": "Huge Automatically Extracted Training Sets for Multilingual Word Sense Disambiguation",
            "authors": [
                "Tommaso Pasini",
                "Francesco Maria Elia",
                "Roberto Navigli"
            ],
            "abstract": "We release to the community six large-scale sense-annotated datasets in\nmultiple language to pave the way for supervised multilingual Word Sense\nDisambiguation. Our datasets cover all the nouns in the English WordNet and\ntheir translations in other languages for a total of millions of sense-tagged\nsentences. Experiments prove that these corpora can be effectively used as\ntraining sets for supervised WSD systems, surpassing the state of the art for\nlow-resourced languages and providing competitive results for English, where\nmanually annotated training sets are accessible. The data is available at\ntrainomatic.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04699v4",
            "date": "2018-05-12 11:06:47+00:00",
            "title": "TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation",
            "authors": [
                "Fran\u00e7ois Hernandez",
                "Vincent Nguyen",
                "Sahar Ghannay",
                "Natalia Tomashenko",
                "Yannick Est\u00e8ve"
            ],
            "abstract": "In this paper, we present TED-LIUM release 3 corpus dedicated to speech\nrecognition in English, that multiplies by more than two the available data to\ntrain acoustic models in comparison with TED-LIUM 2. We present the recent\ndevelopment on Automatic Speech Recognition (ASR) systems in comparison with\nthe two previous releases of the TED-LIUM Corpus from 2012 and 2014. We\ndemonstrate that, passing from 207 to 452 hours of transcribed speech training\ndata is really more useful for end-to-end ASR systems than for HMM-based\nstate-of-the-art ones, even if the HMM-based ASR system still outperforms\nend-to-end ASR system when the size of audio training data is 452 hours, with\nrespectively a Word Error Rate (WER) of 6.6% and 13.7%. Last, we propose two\nrepartitions of the TED-LIUM release 3 corpus: the legacy one that is the same\nas the one existing in release 2, and a new one, calibrated and designed to\nmake experiments on speaker adaptation. Like the two first releases, TED-LIUM 3\ncorpus will be freely available for the research community.",
            "journal ref": "SPECOM 2018. Lecture Notes in Computer Science, vol 11096, pp\n  198-208",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04715v2",
            "date": "2018-05-12 12:55:31+00:00",
            "title": "Unsupervised Semantic Frame Induction using Triclustering",
            "authors": [
                "Dmitry Ustalov",
                "Alexander Panchenko",
                "Andrei Kutuzov",
                "Chris Biemann",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "We use dependency triples automatically extracted from a Web-scale corpus to\nperform unsupervised semantic frame induction. We cast the frame induction\nproblem as a triclustering problem that is a generalization of clustering for\ntriadic data. Our replicable benchmarks demonstrate that the proposed\ngraph-based approach, Triframes, shows state-of-the art results on this task on\na FrameNet-derived dataset and performing on par with competitive methods on a\nverb class clustering task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04787v2",
            "date": "2018-05-12 21:32:50+00:00",
            "title": "Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling",
            "authors": [
                "Luheng He",
                "Kenton Lee",
                "Omer Levy",
                "Luke Zettlemoyer"
            ],
            "abstract": "Recent BIO-tagging-based neural semantic role labeling models are very high\nperforming, but assume gold predicates as part of the input and cannot\nincorporate span-level features. We propose an end-to-end approach for jointly\npredicting all predicates, arguments spans, and the relations between them. The\nmodel makes independent decisions about what relationship, if any, holds\nbetween every possible word-span pair, and learns contextualized span\nrepresentations that provide rich, shared input features for each decision.\nExperiments demonstrate that this approach sets a new state of the art on\nPropBank SRL without gold predicates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04793v1",
            "date": "2018-05-12 22:06:17+00:00",
            "title": "Coarse-to-Fine Decoding for Neural Semantic Parsing",
            "authors": [
                "Li Dong",
                "Mirella Lapata"
            ],
            "abstract": "Semantic parsing aims at mapping natural language utterances into structured\nmeaning representations. In this work, we propose a structure-aware neural\narchitecture which decomposes the semantic parsing process into two stages.\nGiven an input utterance, we first generate a rough sketch of its meaning,\nwhere low-level information (such as variable names and arguments) is glossed\nover. Then, we fill in missing details by taking into account the natural\nlanguage input and the sketch itself. Experimental results on four datasets\ncharacteristic of different domains and meaning representations show that our\napproach consistently improves performance, achieving competitive results\ndespite the use of relatively simple decoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04827v3",
            "date": "2018-05-13 06:09:40+00:00",
            "title": "An attention-based Bi-GRU-CapsNet model for hypernymy detection between compound entities",
            "authors": [
                "Qi Wang",
                "Chenming Xu",
                "Yangming Zhou",
                "Tong Ruan",
                "Daqi Gao",
                "Ping He"
            ],
            "abstract": "Named entities are usually composable and extensible. Typical examples are\nnames of symptoms and diseases in medical areas. To distinguish these entities\nfrom general entities, we name them \\textit{compound entities}. In this paper,\nwe present an attention-based Bi-GRU-CapsNet model to detect hypernymy\nrelationship between compound entities. Our model consists of several important\ncomponents. To avoid the out-of-vocabulary problem, English words or Chinese\ncharacters in compound entities are fed into the bidirectional gated recurrent\nunits. An attention mechanism is designed to focus on the differences between\nthe two compound entities. Since there are some different cases in hypernymy\nrelationship between compound entities, capsule network is finally employed to\ndecide whether the hypernymy relationship exists or not. Experimental results\ndemonstrate",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04833v1",
            "date": "2018-05-13 07:07:08+00:00",
            "title": "Hierarchical Neural Story Generation",
            "authors": [
                "Angela Fan",
                "Mike Lewis",
                "Yann Dauphin"
            ],
            "abstract": "We explore story generation: creative systems that can build coherent and\nfluent passages of text about a topic. We collect a large dataset of 300K\nhuman-written stories paired with writing prompts from an online forum. Our\ndataset enables hierarchical story generation, where the model first generates\na premise, and then transforms it into a passage of text. We gain further\nimprovements with a novel form of model fusion that improves the relevance of\nthe story to the prompt, and adding a new gated multi-scale self-attention\nmechanism to model long-range context. Experiments show large improvements over\nstrong baselines on both automated and human evaluations. Human judges prefer\nstories generated by our approach to those from a strong non-hierarchical model\nby a factor of two to one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04836v1",
            "date": "2018-05-13 07:46:12+00:00",
            "title": "Building Language Models for Text with Named Entities",
            "authors": [
                "Md Rizwan Parvez",
                "Saikat Chakraborty",
                "Baishakhi Ray",
                "Kai-Wei Chang"
            ],
            "abstract": "Text in many domains involves a significant amount of named entities.\nPredict- ing the entity names is often challenging for a language model as they\nappear less frequent on the training corpus. In this paper, we propose a novel\nand effective approach to building a discriminative language model which can\nlearn the entity names by leveraging their entity type information. We also\nintroduce two benchmark datasets based on recipes and Java programming codes,\non which we evalu- ate the proposed model. Experimental re- sults show that our\nmodel achieves 52.2% better perplexity in recipe generation and 22.06% on code\ngeneration than the state-of-the-art language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04843v1",
            "date": "2018-05-13 08:38:20+00:00",
            "title": "Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders",
            "authors": [
                "Yansen Wang",
                "Chenyi Liu",
                "Minlie Huang",
                "Liqiang Nie"
            ],
            "abstract": "Asking good questions in large-scale, open-domain conversational systems is\nquite significant yet rather untouched. This task, substantially different from\ntraditional question generation, requires to question not only with various\npatterns but also on diverse and relevant topics. We observe that a good\nquestion is a natural composition of {\\it interrogatives}, {\\it topic words},\nand {\\it ordinary words}. Interrogatives lexicalize the pattern of questioning,\ntopic words address the key information for topic transition in dialogue, and\nordinary words play syntactical and grammatical roles in making a natural\nsentence. We devise two typed decoders (\\textit{soft typed decoder} and\n\\textit{hard typed decoder}) in which a type distribution over the three types\nis estimated and used to modulate the final generation distribution. Extensive\nexperiments show that the typed decoders outperform state-of-the-art baselines\nand can generate more meaningful questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04869v1",
            "date": "2018-05-13 12:23:44+00:00",
            "title": "Autoencoder as Assistant Supervisor: Improving Text Representation for Chinese Social Media Text Summarization",
            "authors": [
                "Shuming Ma",
                "Xu Sun",
                "Junyang Lin",
                "Houfeng Wang"
            ],
            "abstract": "Most of the current abstractive text summarization models are based on the\nsequence-to-sequence model (Seq2Seq). The source content of social media is\nlong and noisy, so it is difficult for Seq2Seq to learn an accurate semantic\nrepresentation. Compared with the source content, the annotated summary is\nshort and well written. Moreover, it shares the same meaning as the source\ncontent. In this work, we supervise the learning of the representation of the\nsource content with that of the summary. In implementation, we regard a summary\nautoencoder as an assistant supervisor of Seq2Seq. Following previous work, we\nevaluate our model on a popular Chinese social media dataset. Experimental\nresults show that our model achieves the state-of-the-art performances on the\nbenchmark dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04871v1",
            "date": "2018-05-13 12:24:20+00:00",
            "title": "Bag-of-Words as Target for Neural Machine Translation",
            "authors": [
                "Shuming Ma",
                "Xu Sun",
                "Yizhong Wang",
                "Junyang Lin"
            ],
            "abstract": "A sentence can be translated into more than one correct sentences. However,\nmost of the existing neural machine translation models only use one of the\ncorrect translations as the targets, and the other correct sentences are\npunished as the incorrect sentences in the training stage. Since most of the\ncorrect translations for one sentence share the similar bag-of-words, it is\npossible to distinguish the correct translations from the incorrect ones by the\nbag-of-words. In this paper, we propose an approach that uses both the\nsentences and the bag-of-words as targets in the training stage, in order to\nencourage the model to generate the potentially correct sentences that are not\nappeared in the training set. We evaluate our model on a Chinese-English\ntranslation dataset, and experiments show our model outperforms the strong\nbaselines by the BLEU score of 4.55.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04876v4",
            "date": "2018-05-13 12:53:47+00:00",
            "title": "UnibucKernel Reloaded: First Place in Arabic Dialect Identification for the Second Year in a Row",
            "authors": [
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu"
            ],
            "abstract": "We present a machine learning approach that ranked on the first place in the\nArabic Dialect Identification (ADI) Closed Shared Tasks of the 2018 VarDial\nEvaluation Campaign. The proposed approach combines several kernels using\nmultiple kernel learning. While most of our kernels are based on character\np-grams (also known as n-grams) extracted from speech or phonetic transcripts,\nwe also use a kernel based on dialectal embeddings generated from audio\nrecordings by the organizers. In the learning stage, we independently employ\nKernel Discriminant Analysis (KDA) and Kernel Ridge Regression (KRR).\nPreliminary experiments indicate that KRR provides better classification\nresults. Our approach is shallow and simple, but the empirical results obtained\nin the 2018 ADI Closed Shared Task prove that it achieves the best performance.\nFurthermore, our top macro-F1 score (58.92%) is significantly better than the\nsecond best score (57.59%) in the 2018 ADI Shared Task, according to the\nstatistical significance test performed by the organizers. Nevertheless, we\nobtain even better post-competition results (a macro-F1 score of 62.28%) using\nthe audio embeddings released by the organizers after the competition. With a\nvery similar approach (that did not include phonetic features), we also ranked\nfirst in the ADI Closed Shared Tasks of the 2017 VarDial Evaluation Campaign,\nsurpassing the second best method by 4.62%. We therefore conclude that our\nmultiple kernel learning method is the best approach to date for Arabic dialect\nidentification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04893v1",
            "date": "2018-05-13 14:24:31+00:00",
            "title": "Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering",
            "authors": [
                "Rui Zhang",
                "Cicero Nogueira dos Santos",
                "Michihiro Yasunaga",
                "Bing Xiang",
                "Dragomir Radev"
            ],
            "abstract": "Coreference resolution aims to identify in a text all mentions that refer to\nthe same real-world entity. The state-of-the-art end-to-end neural coreference\nmodel considers all text spans in a document as potential mentions and learns\nto link an antecedent for each possible mention. In this paper, we propose to\nimprove the end-to-end coreference resolution system by (1) using a biaffine\nattention model to get antecedent scores for each possible mention, and (2)\njointly optimizing the mention detection accuracy and the mention clustering\nlog-likelihood given the mention cluster labels. Our model achieves the\nstate-of-the-art performance on the CoNLL-2012 Shared Task English test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04905v1",
            "date": "2018-05-13 16:08:57+00:00",
            "title": "Comprehensive Supersense Disambiguation of English Prepositions and Possessives",
            "authors": [
                "Nathan Schneider",
                "Jena D. Hwang",
                "Vivek Srikumar",
                "Jakob Prange",
                "Austin Blodgett",
                "Sarah R. Moeller",
                "Aviram Stern",
                "Adi Bitan",
                "Omri Abend"
            ],
            "abstract": "Semantic relations are often signaled with prepositional or possessive\nmarking--but extreme polysemy bedevils their analysis and automatic\ninterpretation. We introduce a new annotation scheme, corpus, and task for the\ndisambiguation of prepositions and possessives in English. Unlike previous\napproaches, our annotations are comprehensive with respect to types and tokens\nof these markers; use broadly applicable supersense classes rather than\nfine-grained dictionary definitions; unite prepositions and possessives under\nthe same class inventory; and distinguish between a marker's lexical\ncontribution and the role it marks in the context of a predicate or scene.\nStrong interannotator agreement rates, as well as encouraging disambiguation\nresults with established supervised methods, speak to the viability of the\nscheme and task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04988v1",
            "date": "2018-05-14 02:20:07+00:00",
            "title": "Word learning and the acquisition of syntactic--semantic overhypotheses",
            "authors": [
                "Jon Gauthier",
                "Roger Levy",
                "Joshua B. Tenenbaum"
            ],
            "abstract": "Children learning their first language face multiple problems of induction:\nhow to learn the meanings of words, and how to build meaningful phrases from\nthose words according to syntactic rules. We consider how children might solve\nthese problems efficiently by solving them jointly, via a computational model\nthat learns the syntax and semantics of multi-word utterances in a grounded\nreference game. We select a well-studied empirical case in which children are\naware of patterns linking the syntactic and semantic properties of words ---\nthat the properties picked out by base nouns tend to be related to shape, while\nprenominal adjectives tend to refer to other properties such as color. We show\nthat children applying such inductive biases are accurately reflecting the\nstatistics of child-directed speech, and that inducing similar biases in our\ncomputational model captures children's behavior in a classic adjective\nlearning experiment. Our model incorporating such biases also demonstrates a\nclear data efficiency in learning, relative to a baseline model that learns\nwithout forming syntax-sensitive overhypotheses of word meaning. Thus solving a\nmore complex joint inference problem may make the full problem of language\nacquisition easier, not harder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04993v1",
            "date": "2018-05-14 02:48:29+00:00",
            "title": "Discourse Coherence in the Wild: A Dataset, Evaluation and Methods",
            "authors": [
                "Alice Lai",
                "Joel Tetreault"
            ],
            "abstract": "To date there has been very little work on assessing discourse coherence\nmethods on real-world data. To address this, we present a new corpus of\nreal-world texts (GCDC) as well as the first large-scale evaluation of leading\ndiscourse coherence algorithms. We show that neural models, including two that\nwe introduce here (SentAvg and ParSeq), tend to perform best. We analyze these\nperformance differences and discuss patterns we observed in low coherence texts\nin four domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05089v1",
            "date": "2018-05-14 09:52:27+00:00",
            "title": "Parser Training with Heterogeneous Treebanks",
            "authors": [
                "Sara Stymne",
                "Miryam de Lhoneux",
                "Aaron Smith",
                "Joakim Nivre"
            ],
            "abstract": "How to make the most of multiple heterogeneous treebanks when training a\nmonolingual dependency parser is an open question. We start by investigating\npreviously suggested, but little evaluated, strategies for exploiting multiple\ntreebanks based on concatenating training sets, with or without fine-tuning. We\ngo on to propose a new method based on treebank embeddings. We perform\nexperiments for several languages and show that in many cases fine-tuning and\ntreebank embeddings lead to substantial improvements over single treebanks or\nconcatenation, with average gains of 2.0--3.5 LAS points. We argue that\ntreebank embeddings should be preferred due to their conceptual simplicity,\nflexibility and extensibility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05091v1",
            "date": "2018-05-14 09:57:47+00:00",
            "title": "The Spot the Difference corpus: a multi-modal corpus of spontaneous task oriented spoken interactions",
            "authors": [
                "Jos\u00e9 Lopes",
                "Nils Hemmingsson",
                "Oliver \u00c5strand"
            ],
            "abstract": "This paper describes the Spot the Difference Corpus which contains 54\ninteractions between pairs of subjects interacting to find differences in two\nvery similar scenes. The setup used, the participants' metadata and details\nabout collection are described. We are releasing this corpus of task-oriented\nspontaneous dialogues. This release includes rich transcriptions, annotations,\naudio and video. We believe that this dataset constitutes a valuable resource\nto study several dimensions of human communication that go from turn-taking to\nthe study of referring expressions. In our preliminary analyses we have looked\nat task success (how many differences were found out of the total number of\ndifferences) and how it evolves over time. In addition we have looked at scene\ncomplexity provided by the RGB components' entropy and how it could relate to\nspeech overlaps, interruptions and the expression of uncertainty. We found\nthere is a tendency that more complex scenes have more competitive\ninterruptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05095v1",
            "date": "2018-05-14 10:07:51+00:00",
            "title": "Bianet: A Parallel News Corpus in Turkish, Kurdish and English",
            "authors": [
                "Duygu Ataman"
            ],
            "abstract": "We present a new open-source parallel corpus consisting of news articles\ncollected from the Bianet magazine, an online newspaper that publishes Turkish\nnews, often along with their translations in English and Kurdish. In this\npaper, we describe the collection process of the corpus and its statistical\nproperties. We validate the benefit of using the Bianet corpus by evaluating\nbilingual and multilingual neural machine translation models in English-Turkish\nand English-Kurdish directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05181v2",
            "date": "2018-05-14 14:17:03+00:00",
            "title": "Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach",
            "authors": [
                "Jingjing Xu",
                "Xu Sun",
                "Qi Zeng",
                "Xuancheng Ren",
                "Xiaodong Zhang",
                "Houfeng Wang",
                "Wenjie Li"
            ],
            "abstract": "The goal of sentiment-to-sentiment \"translation\" is to change the underlying\nsentiment of a sentence while keeping its content. The main challenge is the\nlack of parallel data. To solve this problem, we propose a cycled reinforcement\nlearning method that enables training on unpaired data by collaboration between\na neutralization module and an emotionalization module. We evaluate our\napproach on two review datasets, Yelp and Amazon. Experimental results show\nthat our approach significantly outperforms the state-of-the-art systems.\nEspecially, the proposed method substantially improves the content preservation\nperformance. The BLEU score is improved from 1.64 to 22.46 and from 0.56 to\n14.06 on the two datasets, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05202v2",
            "date": "2018-05-14 14:46:18+00:00",
            "title": "A Dynamic Oracle for Linear-Time 2-Planar Dependency Parsing",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We propose an efficient dynamic oracle for training the 2-Planar\ntransition-based parser, a linear-time parser with over 99% coverage on\nnon-projective syntactic corpora. This novel approach outperforms the static\ntraining strategy in the vast majority of languages tested and scored better on\nmost datasets than the arc-hybrid parser enhanced with the SWAP transition,\nwhich can handle unrestricted non-projectivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05237v2",
            "date": "2018-05-14 15:40:31+00:00",
            "title": "Effects of Word Embeddings on Neural Network-based Pitch Accent Detection",
            "authors": [
                "Sabrina Stehwien",
                "Ngoc Thang Vu",
                "Antje Schweitzer"
            ],
            "abstract": "Pitch accent detection often makes use of both acoustic and lexical features\nbased on the fact that pitch accents tend to correlate with certain words. In\nthis paper, we extend a pitch accent detector that involves a convolutional\nneural network to include word embeddings, which are state-of-the-art vector\nrepresentations of words. We examine the effect these features have on\nwithin-corpus and cross-corpus experiments on three English datasets. The\nresults show that while word embeddings can improve the performance in\ncorpus-dependent experiments, they also have the potential to make\ngeneralization to unseen data more challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05271v2",
            "date": "2018-05-14 16:36:11+00:00",
            "title": "Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization",
            "authors": [
                "Guokan Shang",
                "Wensi Ding",
                "Zekun Zhang",
                "Antoine Jean-Pierre Tixier",
                "Polykarpos Meladianos",
                "Michalis Vazirgiannis",
                "Jean-Pierre Lorr\u00e9"
            ],
            "abstract": "We introduce a novel graph-based framework for abstractive meeting speech\nsummarization that is fully unsupervised and does not rely on any annotations.\nOur work combines the strengths of multiple recent approaches while addressing\ntheir weaknesses. Moreover, we leverage recent advances in word embeddings and\ngraph degeneracy applied to NLP to take exterior semantic knowledge into\naccount, and to design custom diversity and informativeness measures.\nExperiments on the AMI and ICSI corpus show that our system improves on the\nstate-of-the-art. Code and data are publicly available, and our system can be\ninteractively tested.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05286v1",
            "date": "2018-05-14 16:56:36+00:00",
            "title": "AMR Parsing as Graph Prediction with Latent Alignment",
            "authors": [
                "Chunchuan Lyu",
                "Ivan Titov"
            ],
            "abstract": "Abstract meaning representations (AMRs) are broad-coverage sentence-level\nsemantic representations. AMRs represent sentences as rooted labeled directed\nacyclic graphs. AMR parsing is challenging partly due to the lack of annotated\nalignments between nodes in the graphs and words in the corresponding\nsentences. We introduce a neural parser which treats alignments as latent\nvariables within a joint probabilistic model of concepts, relations and\nalignments. As exact inference requires marginalizing over alignments and is\ninfeasible, we use the variational auto-encoding framework and a continuous\nrelaxation of the discrete alignments. We show that joint modeling is\npreferable to using a pipeline of align and parse. The parser achieves the best\nreported results on the standard benchmark (74.4% on LDC2016E25).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05370v1",
            "date": "2018-05-14 18:17:12+00:00",
            "title": "AMORE-UPF at SemEval-2018 Task 4: BiLSTM with Entity Library",
            "authors": [
                "Laura Aina",
                "Carina Silberer",
                "Ionut-Teodor Sorodoc",
                "Matthijs Westera",
                "Gemma Boleda"
            ],
            "abstract": "This paper describes our winning contribution to SemEval 2018 Task 4:\nCharacter Identification on Multiparty Dialogues. It is a simple, standard\nmodel with one key innovation, an entity library. Our results show that this\ninnovation greatly facilitates the identification of infrequent characters.\nBecause of the generic nature of our model, this finding is potentially\nrelevant to any task that requires effective learning from sparse or unbalanced\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05542v1",
            "date": "2018-05-15 03:06:04+00:00",
            "title": "A Manually Annotated Chinese Corpus for Non-task-oriented Dialogue Systems",
            "authors": [
                "Jing Li",
                "Yan Song",
                "Haisong Zhang",
                "Shuming Shi"
            ],
            "abstract": "This paper presents a large-scale corpus for non-task-oriented dialogue\nresponse selection, which contains over 27K distinct prompts more than 82K\nresponses collected from social media. To annotate this corpus, we define a\n5-grade rating scheme: bad, mediocre, acceptable, good, and excellent,\naccording to the relevance, coherence, informativeness, interestingness, and\nthe potential to move a conversation forward. To test the validity and\nusefulness of the produced corpus, we compare various unsupervised and\nsupervised models for response selection. Experimental results confirm that the\nproposed corpus is helpful in training response selection models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05557v1",
            "date": "2018-05-15 04:49:55+00:00",
            "title": "Simplifying Sentences with Sequence to Sequence Models",
            "authors": [
                "Alexander Mathews",
                "Lexing Xie",
                "Xuming He"
            ],
            "abstract": "We simplify sentences with an attentive neural network sequence to sequence\nmodel, dubbed S4. The model includes a novel word-copy mechanism and loss\nfunction to exploit linguistic similarities between the original and simplified\nsentences. It also jointly uses pre-trained and fine-tuned word embeddings to\ncapture the semantics of complex sentences and to mitigate the effects of\nlimited data. When trained and evaluated on pairs of sentences from thousands\nof news articles, we observe a 8.8 point improvement in BLEU score over a\nsequence to sequence baseline; however, learning word substitutions remains\ndifficult. Such sequence to sequence models are promising for other text\ngeneration tasks such as style transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05581v1",
            "date": "2018-05-15 06:19:12+00:00",
            "title": "Unsupervised Learning of Style-sensitive Word Vectors",
            "authors": [
                "Reina Akama",
                "Kento Watanabe",
                "Sho Yokoi",
                "Sosuke Kobayashi",
                "Kentaro Inui"
            ],
            "abstract": "This paper presents the first study aimed at capturing stylistic similarity\nbetween words in an unsupervised manner. We propose extending the continuous\nbag of words (CBOW) model (Mikolov et al., 2013) to learn style-sensitive word\nvectors using a wider context window under the assumption that the style of all\nthe words in an utterance is consistent. In addition, we introduce a novel task\nto predict lexical stylistic similarity and to create a benchmark dataset for\nthis task. Our experiment with this dataset supports our assumption and\ndemonstrates that the proposed extensions contribute to the acquisition of\nstyle-sensitive word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05588v1",
            "date": "2018-05-15 06:40:44+00:00",
            "title": "Marrying up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding",
            "authors": [
                "Bingfeng Luo",
                "Yansong Feng",
                "Zheng Wang",
                "Songfang Huang",
                "Rui Yan",
                "Dongyan Zhao"
            ],
            "abstract": "The success of many natural language processing (NLP) tasks is bound by the\nnumber and quality of annotated data, but there is often a shortage of such\ntraining data. In this paper, we ask the question: \"Can we combine a neural\nnetwork (NN) with regular expressions (RE) to improve supervised learning for\nNLP?\". In answer, we develop novel methods to exploit the rich expressiveness\nof REs at different levels within a NN, showing that the combination\nsignificantly enhances the learning effectiveness when a small number of\ntraining examples are available. We evaluate our approach by applying it to\nspoken language understanding for intent detection and slot filling.\nExperimental results show that our approach is highly effective in exploiting\nthe available training data, giving a clear boost to the RE-unaware NN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05593v1",
            "date": "2018-05-15 07:03:21+00:00",
            "title": "Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information",
            "authors": [
                "Masaki Asada",
                "Makoto Miwa",
                "Yutaka Sasaki"
            ],
            "abstract": "We propose a novel neural method to extract drug-drug interactions (DDIs)\nfrom texts using external drug molecular structure information. We encode\ntextual drug pairs with convolutional neural networks and their molecular pairs\nwith graph convolutional networks (GCNs), and then we concatenate the outputs\nof these two networks. In the experiments, we show that GCNs can predict DDIs\nfrom the molecular structures of drugs in high accuracy and the molecular\ninformation can enhance text-based DDI extraction by 2.39 percent points in the\nF-score on the DDIExtraction 2013 shared task data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05691v1",
            "date": "2018-05-15 10:32:53+00:00",
            "title": "Generating Continuous Representations of Medical Texts",
            "authors": [
                "Graham Spinks",
                "Marie-Francine Moens"
            ],
            "abstract": "We present an architecture that generates medical texts while learning an\ninformative, continuous representation with discriminative features. During\ntraining the input to the system is a dataset of captions for medical X-Rays.\nThe acquired continuous representations are of particular interest for use in\nmany machine learning techniques where the discrete and high-dimensional nature\nof textual input is an obstacle. We use an Adversarially Regularized\nAutoencoder to create realistic text in both an unconditional and conditional\nsetting. We show that this technique is applicable to medical texts which often\ncontain syntactic and domain-specific shorthands. A quantitative evaluation\nshows that we achieve a lower model perplexity than a traditional LSTM\ngenerator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05758v1",
            "date": "2018-05-15 13:37:33+00:00",
            "title": "Continuous Learning in a Hierarchical Multiscale Neural Network",
            "authors": [
                "Thomas Wolf",
                "Julien Chaumond",
                "Clement Delangue"
            ],
            "abstract": "We reformulate the problem of encoding a multi-scale representation of a\nsequence in a language model by casting it in a continuous learning framework.\nWe propose a hierarchical multi-scale language model in which short time-scale\ndependencies are encoded in the hidden state of a lower-level recurrent neural\nnetwork while longer time-scale dependencies are encoded in the dynamic of the\nlower-level network by having a meta-learner update the weights of the\nlower-level neural network in an online meta-learning fashion. We use elastic\nweights consolidation as a higher-level to prevent catastrophic forgetting in\nour continuous learning framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05927v1",
            "date": "2018-05-15 17:45:25+00:00",
            "title": "CLINIQA: A Machine Intelligence Based Clinical Question Answering System",
            "authors": [
                "M A H Zahid",
                "Ankush Mittal",
                "R. C. Joshi",
                "G. Atluri"
            ],
            "abstract": "The recent developments in the field of biomedicine have made large volumes\nof biomedical literature available to the medical practitioners. Due to the\nlarge size and lack of efficient searching strategies, medical practitioners\nstruggle to obtain necessary information available in the biomedical\nliterature. Moreover, the most sophisticated search engines of age are not\nintelligent enough to interpret the clinicians' questions. These facts reflect\nthe urgent need of an information retrieval system that accepts the queries\nfrom medical practitioners' in natural language and returns the answers quickly\nand efficiently. In this paper, we present an implementation of a machine\nintelligence based CLINIcal Question Answering system (CLINIQA) to answer\nmedical practitioner's questions. The system was rigorously evaluated on\ndifferent text mining algorithms and the best components for the system were\nselected. The system makes use of Unified Medical Language System for semantic\nanalysis of both questions and medical documents. In addition, the system\nemploys supervised machine learning algorithms for classification of the\ndocuments, identifying the focus of the question and answer selection.\nEffective domain-specific heuristics are designed for answer ranking. The\nperformance evaluation on hundred clinical questions shows the effectiveness of\nour approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05942v1",
            "date": "2018-05-15 17:58:25+00:00",
            "title": "Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia",
            "authors": [
                "Xinya Du",
                "Claire Cardie"
            ],
            "abstract": "We study the task of generating from Wikipedia articles question-answer pairs\nthat cover content beyond a single sentence. We propose a neural network\napproach that incorporates coreference knowledge via a novel gating mechanism.\nCompared to models that only take into account sentence-level information\n(Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017), we find that the\nlinguistic knowledge introduced by the coreference representation aids question\ngeneration significantly, producing models that outperform the current\nstate-of-the-art. We apply our system (composed of an answer span extraction\nsystem and the passage-level QG system) to the 10,000 top-ranking Wikipedia\narticles and create a corpus of over one million question-answer pairs. We also\nprovide a qualitative analysis for this large-scale generated corpus from\nWikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06016v1",
            "date": "2018-05-15 20:05:58+00:00",
            "title": "Author Commitment and Social Power: Automatic Belief Tagging to Infer the Social Context of Interactions",
            "authors": [
                "Vinodkumar Prabhakaran",
                "Premkumar Ganeshkumar",
                "Owen Rambow"
            ],
            "abstract": "Understanding how social power structures affect the way we interact with one\nanother is of great interest to social scientists who want to answer\nfundamental questions about human behavior, as well as to computer scientists\nwho want to build automatic methods to infer the social contexts of\ninteractions. In this paper, we employ advancements in extra-propositional\nsemantics extraction within NLP to study how author commitment reflects the\nsocial context of an interaction. Specifically, we investigate whether the\nlevel of commitment expressed by individuals in an organizational interaction\nreflects the hierarchical power structures they are part of. We find that\nsubordinates use significantly more instances of non-commitment than superiors.\nMore importantly, we also find that subordinates attribute propositions to\nother agents more often than superiors do --- an aspect that has not been\nstudied before. Finally, we show that enriching lexical features with\ncommitment labels captures important distinctions in social meanings.",
            "journal ref": "North American Chapter of the Association for Computational\n  Linguistics: Human Language Technologies (NAACL-HLT). 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06087v1",
            "date": "2018-05-16 01:27:58+00:00",
            "title": "Learning to Write with Cooperative Discriminators",
            "authors": [
                "Ari Holtzman",
                "Jan Buys",
                "Maxwell Forbes",
                "Antoine Bosselut",
                "David Golub",
                "Yejin Choi"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models,\nbut when used to generate natural language their output tends to be overly\ngeneric, repetitive, and self-contradictory. We postulate that the objective\nfunction optimized by RNN language models, which amounts to the overall\nperplexity of a text, is not expressive enough to capture the notion of\ncommunicative goals described by linguistic principles such as Grice's Maxims.\nWe propose learning a mixture of multiple discriminative models that can be\nused to complement the RNN generator and guide the decoding process. Human\nevaluation demonstrates that text generated by our system is preferred over\nthat of baselines by a large margin and significantly enhances the overall\ncoherence, style, and information content of the generated text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06088v1",
            "date": "2018-05-16 01:29:14+00:00",
            "title": "What's in a Domain? Learning Domain-Robust Text Representations using Adversarial Training",
            "authors": [
                "Yitong Li",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Most real world language problems require learning from heterogenous corpora,\nraising the problem of learning robust models which generalise well to both\nsimilar (in domain) and dissimilar (out of domain) instances to those seen in\ntraining. This requires learning an underlying task, while not learning\nirrelevant signals and biases specific to individual domains. We propose a\nnovel method to optimise both in- and out-of-domain accuracy based on joint\nlearning of a structured neural model with domain-specific and domain-general\ncomponents, coupled with adversarial training for domain. Evaluating on\nmulti-domain language identification and multi-domain sentiment analysis, we\nshow substantial improvements over standard domain adaptation techniques, and\ndomain-adversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06093v1",
            "date": "2018-05-16 01:57:47+00:00",
            "title": "Towards Robust and Privacy-preserving Text Representations",
            "authors": [
                "Yitong Li",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Written text often provides sufficient clues to identify the author, their\ngender, age, and other important attributes. Consequently, the authorship of\ntraining and evaluation corpora can have unforeseen impacts, including\ndiffering model performance for different user groups, as well as privacy\nimplications. In this paper, we propose an approach to explicitly obscure\nimportant author characteristics at training time, such that representations\nlearned are invariant to these attributes. Evaluating on two tasks, we show\nthat this leads to increased privacy in the learned representations, as well as\nmore robust models to varying evaluation conditions, including out-of-domain\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06122v1",
            "date": "2018-05-16 04:17:18+00:00",
            "title": "Narrative Modeling with Memory Chains and Semantic Supervision",
            "authors": [
                "Fei Liu",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Story comprehension requires a deep semantic understanding of the narrative,\nmaking it a challenging task. Inspired by previous studies on ROC Story Cloze\nTest, we propose a novel method, tracking various semantic aspects with\nexternal neural memory chains while encouraging each to focus on a particular\nsemantic aspect. Evaluated on the task of story ending prediction, our model\ndemonstrates superior performance to a collection of competitive baselines,\nsetting a new state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06130v1",
            "date": "2018-05-16 04:51:29+00:00",
            "title": "Towards Robust Neural Machine Translation",
            "authors": [
                "Yong Cheng",
                "Zhaopeng Tu",
                "Fandong Meng",
                "Junjie Zhai",
                "Yang Liu"
            ],
            "abstract": "Small perturbations in the input can severely distort intermediate\nrepresentations and thus impact translation quality of neural machine\ntranslation (NMT) models. In this paper, we propose to improve the robustness\nof NMT models with adversarial stability training. The basic idea is to make\nboth the encoder and decoder in NMT models robust against input perturbations\nby enabling them to behave similarly for the original input and its perturbed\ncounterpart. Experimental results on Chinese-English, English-German and\nEnglish-French translation tasks show that our approaches can not only achieve\nsignificant improvements over strong NMT systems but also improve the\nrobustness of NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06145v1",
            "date": "2018-05-16 06:14:31+00:00",
            "title": "Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension",
            "authors": [
                "Zhen Wang",
                "Jiachen Liu",
                "Xinyan Xiao",
                "Yajuan Lyu",
                "Tian Wu"
            ],
            "abstract": "While sophisticated neural-based techniques have been developed in reading\ncomprehension, most approaches model the answer in an independent manner,\nignoring its relations with other answer candidates. This problem can be even\nworse in open-domain scenarios, where candidates from multiple passages should\nbe combined to answer a single question. In this paper, we formulate reading\ncomprehension as an extract-then-select two-stage procedure. We first extract\nanswer candidates from passages, then select the final answer by combining\ninformation from all the candidates. Furthermore, we regard candidate\nextraction as a latent variable and train the two-stage process jointly with\nreinforcement learning. As a result, our approach has improved the\nstate-of-the-art performance significantly on two challenging open-domain\nreading comprehension datasets. Further analysis demonstrates the effectiveness\nof our model components, especially the information fusion of all the\ncandidates and the joint training of the extract-then-select procedure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06165v3",
            "date": "2018-05-16 07:34:00+00:00",
            "title": "Color naming reflects both perceptual structure and communicative need",
            "authors": [
                "Noga Zaslavsky",
                "Charles Kemp",
                "Naftali Tishby",
                "Terry Regier"
            ],
            "abstract": "Gibson et al. (2017) argued that color naming is shaped by patterns of\ncommunicative need. In support of this claim, they showed that color naming\nsystems across languages support more precise communication about warm colors\nthan cool colors, and that the objects we talk about tend to be warm-colored\nrather than cool-colored. Here, we present new analyses that alter this\npicture. We show that greater communicative precision for warm than for cool\ncolors, and greater communicative need, may both be explained by perceptual\nstructure. However, using an information-theoretic analysis, we also show that\ncolor naming across languages bears signs of communicative need beyond what\nwould be predicted by perceptual structure alone. We conclude that color naming\nis shaped both by perceptual structure, as has traditionally been argued, and\nby patterns of communicative need, as argued by Gibson et al. - although for\nreasons other than those they advanced.",
            "journal ref": "Proceedings of the 40th Annual Conference of the Cognitive Science\n  Society (pp. 1250 - 1255), 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06266v2",
            "date": "2018-05-16 12:17:09+00:00",
            "title": "A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss",
            "authors": [
                "Wan-Ting Hsu",
                "Chieh-Kai Lin",
                "Ming-Ying Lee",
                "Kerui Min",
                "Jing Tang",
                "Min Sun"
            ],
            "abstract": "We propose a unified model combining the strength of extractive and\nabstractive summarization. On the one hand, a simple extractive model can\nobtain sentence-level attention with high ROUGE scores but less readable. On\nthe other hand, a more complicated abstractive model can obtain word-level\ndynamic attention to generate a more readable paragraph. In our model,\nsentence-level attention is used to modulate the word-level attention such that\nwords in less attended sentences are less likely to be generated. Moreover, a\nnovel inconsistency loss function is introduced to penalize the inconsistency\nbetween two levels of attentions. By end-to-end training our model with the\ninconsistency loss and original losses of extractive and abstractive models, we\nachieve state-of-the-art ROUGE scores while being the most informative and\nreadable summarization on the CNN/Daily Mail dataset in a solid human\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06344v2",
            "date": "2018-05-16 14:26:32+00:00",
            "title": "Automatic Annotation of Locative and Directional Expressions in Arabic",
            "authors": [
                "Rita Hijazi",
                "Amani Sabra",
                "Moustafa Al-Hajj"
            ],
            "abstract": "In this paper, we introduce a rule-based approach to annotate Locative and\nDirectional Expressions in Arabic natural language text. The annotation is\nbased on a constructed semantic map of the spatiality domain. Challenges are\ntwofold: first, we need to study how locative and directional expressions are\nexpressed linguistically in these texts; and second, we need to automatically\nannotate the relevant textual segments accordingly. The research method we will\nuse in this article is analytic-descriptive. We will validate this approach on\nspecific novel rich with these expressions and show that it has very promising\nresults. We will be using NOOJ as a software tool to implement finite-state\ntransducers to annotate linguistic elements according to Locative and\nDirectional Expressions. In conclusion, NOOJ allowed us to write linguistic\nrules for the automatic annotation in Arabic text of Locative and Directional\nExpressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06375v1",
            "date": "2018-05-16 15:43:21+00:00",
            "title": "#phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter",
            "authors": [
                "Debanjan Mahata",
                "Jasper Friedrichs",
                "Hitkul",
                "Rajiv Ratn Shah"
            ],
            "abstract": "Mining social media messages for health and drug related information has\nreceived significant interest in pharmacovigilance research. Social media sites\n(e.g., Twitter), have been used for monitoring drug abuse, adverse reactions of\ndrug usage and analyzing expression of sentiments related to drugs. Most of\nthese studies are based on aggregated results from a large population rather\nthan specific sets of individuals. In order to conduct studies at an individual\nlevel or specific cohorts, identifying posts mentioning intake of medicine by\nthe user is necessary. Towards this objective, we train different deep neural\nnetwork classification models on a publicly available annotated dataset and\nstudy their performances on identifying mentions of personal intake of medicine\nin tweets. We also design and train a new architecture of a stacked ensemble of\nshallow convolutional neural network (CNN) ensembles. We use random search for\ntuning the hyperparameters of the models and share the details of the values\ntaken by the hyperparameters for the best learnt model in different deep neural\nnetwork architectures. Our system produces state-of-the-art results, with a\nmicro- averaged F-score of 0.693.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06413v1",
            "date": "2018-05-16 16:38:38+00:00",
            "title": "CASCADE: Contextual Sarcasm Detection in Online Discussion Forums",
            "authors": [
                "Devamanyu Hazarika",
                "Soujanya Poria",
                "Sruthi Gorantla",
                "Erik Cambria",
                "Roger Zimmermann",
                "Rada Mihalcea"
            ],
            "abstract": "The literature in automated sarcasm detection has mainly focused on lexical,\nsyntactic and semantic-level analysis of text. However, a sarcastic sentence\ncan be expressed with contextual presumptions, background and commonsense\nknowledge. In this paper, we propose CASCADE (a ContextuAl SarCasm DEtector)\nthat adopts a hybrid approach of both content and context-driven modeling for\nsarcasm detection in online social media discussions. For the latter, CASCADE\naims at extracting contextual information from the discourse of a discussion\nthread. Also, since the sarcastic nature and form of expression can vary from\nperson to person, CASCADE utilizes user embeddings that encode stylometric and\npersonality features of the users. When used along with content-based feature\nextractors such as Convolutional Neural Networks (CNNs), we see a significant\nboost in the classification performance on a large Reddit corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06521v1",
            "date": "2018-05-16 20:41:33+00:00",
            "title": "Composite Semantic Relation Classification",
            "authors": [
                "Siamak Barzegar",
                "Andre Freitas",
                "Siegfried Handschuh",
                "Brian Davis"
            ],
            "abstract": "Different semantic interpretation tasks such as text entailment and question\nanswering require the classification of semantic relations between terms or\nentities within text. However, in most cases it is not possible to assign a\ndirect semantic relation between entities/terms. This paper proposes an\napproach for composite semantic relation classification, extending the\ntraditional semantic relation classification task. Different from existing\napproaches, which use machine learning models built over lexical and\ndistributional word vector features, the proposed model uses the combination of\na large commonsense knowledge base of binary relations, a distributional\nnavigational algorithm and sequence classification to provide a solution for\nthe composite semantic relation classification problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06522v1",
            "date": "2018-05-16 20:43:45+00:00",
            "title": "Semantic Relatedness for All (Languages): A Comparative Analysis of Multilingual Semantic Relatedness Using Machine Translation",
            "authors": [
                "Andre Freitas",
                "Siamak Barzegar",
                "Juliano Efson Sales",
                "Siegfried Handschuh",
                "Brian Davis"
            ],
            "abstract": "This paper provides a comparative analysis of the performance of four\nstate-of-the-art distributional semantic models (DSMs) over 11 languages,\ncontrasting the native language-specific models with the use of machine\ntranslation over English-based DSMs. The experimental results show that there\nis a significant improvement (average of 16.7% for the Spearman correlation) by\nusing state-of-the-art machine translation approaches. The results also show\nthat the benefit of using the most informative corpus outweighs the possible\nerrors introduced by the machine translation. For all languages, the\ncombination of machine translation over the Word2Vec English distributional\nmodel provided the best results consistently (average Spearman correlation of\n0.68).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06533v1",
            "date": "2018-05-16 21:39:02+00:00",
            "title": "Modeling Naive Psychology of Characters in Simple Commonsense Stories",
            "authors": [
                "Hannah Rashkin",
                "Antoine Bosselut",
                "Maarten Sap",
                "Kevin Knight",
                "Yejin Choi"
            ],
            "abstract": "Understanding a narrative requires reading between the lines and reasoning\nabout the unspoken but obvious implications about events and people's mental\nstates - a capability that is trivial for humans but remarkably hard for\nmachines. To facilitate research addressing this challenge, we introduce a new\nannotation framework to explain naive psychology of story characters as\nfully-specified chains of mental states with respect to motivations and\nemotional reactions. Our work presents a new large-scale dataset with rich\nlow-level annotations and establishes baseline performance on several new\ntasks, suggesting avenues for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06536v1",
            "date": "2018-05-16 21:42:21+00:00",
            "title": "Are BLEU and Meaning Representation in Opposition?",
            "authors": [
                "Ond\u0159ej C\u00edfka",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "One of possible ways of obtaining continuous-space sentence representations\nis by training neural machine translation (NMT) systems. The recent attention\nmechanism however removes the single point in the neural network from which the\nsource sentence representation can be extracted. We propose several variations\nof the attentive NMT architecture bringing this meeting point back. Empirical\nevaluation suggests that the better the translation quality, the worse the\nlearned sentence representations serve in a wide range of classification and\nsimilarity tasks.",
            "journal ref": "Proceedings of the 56th Annual Meeting of the Association for\n  Computational Linguistics (Volume 1: Long Papers), 2018, pages 1362--1371",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06553v1",
            "date": "2018-05-16 23:30:01+00:00",
            "title": "A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation",
            "authors": [
                "Juraj Juraska",
                "Panagiotis Karagiannis",
                "Kevin K. Bowden",
                "Marilyn A. Walker"
            ],
            "abstract": "Natural language generation lies at the core of generative dialogue systems\nand conversational agents. We describe an ensemble neural language generator,\nand present several novel methods for data representation and augmentation that\nyield improved results in our model. We test the model on three datasets in the\nrestaurant, TV and laptop domains, and report both objective and subjective\nevaluations of our best model. Using a range of automatic metrics, as well as\nhuman evaluators, we show that our approach achieves better results than\nstate-of-the-art models on the same datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06556v1",
            "date": "2018-05-16 23:42:04+00:00",
            "title": "Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples",
            "authors": [
                "Vidur Joshi",
                "Matthew Peters",
                "Mark Hopkins"
            ],
            "abstract": "We revisit domain adaptation for parsers in the neural era. First we show\nthat recent advances in word representations greatly diminish the need for\ndomain adaptation when the target domain is syntactically similar to the source\ndomain. As evidence, we train a parser on the Wall Street Jour- nal alone that\nachieves over 90% F1 on the Brown corpus. For more syntactically dis- tant\ndomains, we provide a simple way to adapt a parser using only dozens of partial\nannotations. For instance, we increase the percentage of error-free\ngeometry-domain parses in a held-out set from 45% to 73% using approximately\nfive dozen training examples. In the process, we demon- strate a new\nstate-of-the-art single model result on the Wall Street Journal test set of\n94.3%. This is an absolute increase of 1.7% over the previous state-of-the-art\nof 92.6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06566v1",
            "date": "2018-05-17 00:59:22+00:00",
            "title": "Content-based Popularity Prediction of Online Petitions Using a Deep Regression Model",
            "authors": [
                "Shivashankar Subramanian",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Online petitions are a cost-effective way for citizens to collectively engage\nwith policy-makers in a democracy. Predicting the popularity of a petition ---\ncommonly measured by its signature count --- based on its textual content has\nutility for policy-makers as well as those posting the petition. In this work,\nwe model this task using CNN regression with an auxiliary ordinal regression\nobjective. We demonstrate the effectiveness of our proposed approach using UK\nand US government petition datasets.",
            "journal ref": "ACL 2018 (camera ready pre-print)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06648v1",
            "date": "2018-05-17 08:29:09+00:00",
            "title": "Extrapolation in NLP",
            "authors": [
                "Jeff Mitchell",
                "Pasquale Minervini",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "We argue that extrapolation to examples outside the training space will often\nbe easier for models that capture global structures, rather than just maximise\ntheir local fit to the training data. We show that this is true for two popular\nmodels: the Decomposable Attention Model and word2vec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06665v1",
            "date": "2018-05-17 09:20:52+00:00",
            "title": "Classifying medical relations in clinical text via convolutional neural networks",
            "authors": [
                "Bin He",
                "Yi Guan",
                "Rui Dai"
            ],
            "abstract": "Deep learning research on relation classification has achieved solid\nperformance in the general domain. This study proposes a convolutional neural\nnetwork (CNN) architecture with a multi-pooling operation for medical relation\nclassification on clinical records and explores a loss function with a\ncategory-level constraint matrix. Experiments using the 2010 i2b2/VA relation\ncorpus demonstrate these models, which do not depend on any external features,\noutperform previous single-model methods and our best model is competitive with\nthe existing ensemble-based method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06939v2",
            "date": "2018-05-17 19:18:57+00:00",
            "title": "Event2Mind: Commonsense Inference on Events, Intents, and Reactions",
            "authors": [
                "Hannah Rashkin",
                "Maarten Sap",
                "Emily Allaway",
                "Noah A. Smith",
                "Yejin Choi"
            ],
            "abstract": "We investigate a new commonsense inference task: given an event described in\na short free-form text (\"X drinks coffee in the morning\"), a system reasons\nabout the likely intents (\"X wants to stay awake\") and reactions (\"X feels\nalert\") of the event's participants. To support this study, we construct a new\ncrowdsourced corpus of 25,000 event phrases covering a diverse range of\neveryday events and situations. We report baseline performance on this task,\ndemonstrating that neural encoder-decoder models can successfully compose\nembedding representations of previously unseen events and reason about the\nlikely intents and reactions of the event participants. In addition, we\ndemonstrate how commonsense inference on people's intents and reactions can\nhelp unveil the implicit gender inequality prevalent in modern movie scripts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06975v1",
            "date": "2018-05-17 21:42:04+00:00",
            "title": "Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension",
            "authors": [
                "Bhavana Dalvi Mishra",
                "Lifu Huang",
                "Niket Tandon",
                "Wen-tau Yih",
                "Peter Clark"
            ],
            "abstract": "We present a new dataset and models for comprehending paragraphs about\nprocesses (e.g., photosynthesis), an important genre of text describing a\ndynamic world. The new dataset, ProPara, is the first to contain natural\n(rather than machine-generated) text about a changing world along with a full\nannotation of entity states (location and existence) during those changes (81k\ndatapoints). The end-task, tracking the location and existence of entities\nthrough the text, is challenging because the causal effects of actions are\noften implicit and need to be inferred. We find that previous models that have\nworked well on synthetic data achieve only mediocre performance on ProPara, and\nintroduce two new neural models that exploit alternative mechanisms for state\nprediction, in particular using LSTM input encoding and span prediction. The\nnew models improve accuracy by up to 19%. The dataset and models are available\nto the community at http://data.allenai.org/propara.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06995v2",
            "date": "2018-05-17 23:40:06+00:00",
            "title": "Linear-Time Constituency Parsing with RNNs and Dynamic Programming",
            "authors": [
                "Juneki Hong",
                "Liang Huang"
            ],
            "abstract": "Recently, span-based constituency parsing has achieved competitive accuracies\nwith extremely simple models by using bidirectional RNNs to model \"spans\".\nHowever, the minimal span parser of Stern et al (2017a) which holds the current\nstate of the art accuracy is a chart parser running in cubic time, $O(n^3)$,\nwhich is too slow for longer sentences and for applications beyond sentence\nboundaries such as end-to-end discourse parsing and joint sentence boundary\ndetection and parsing. We propose a linear-time constituency parser with RNNs\nand dynamic programming using graph-structured stack and beam search, which\nruns in time $O(n b^2)$ where $b$ is the beam size. We further speed this up to\n$O(n b\\log b)$ by integrating cube pruning. Compared with chart parsing\nbaselines, this linear-time parser is substantially faster for long sentences\non the Penn Treebank and orders of magnitude faster for discourse parsing, and\nachieves the highest F1 accuracy on the Penn Treebank among single model\nend-to-end systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07043v1",
            "date": "2018-05-18 04:24:52+00:00",
            "title": "Aspect Based Sentiment Analysis with Gated Convolutional Networks",
            "authors": [
                "Wei Xue",
                "Tao Li"
            ],
            "abstract": "Aspect based sentiment analysis (ABSA) can provide more detailed information\nthan general sentiment analysis, because it aims to predict the sentiment\npolarities of the given aspects or entities in text. We summarize previous\napproaches into two subtasks: aspect-category sentiment analysis (ACSA) and\naspect-term sentiment analysis (ATSA). Most previous approaches employ long\nshort-term memory and attention mechanisms to predict the sentiment polarity of\nthe concerned targets, which are often complicated and need more training time.\nWe propose a model based on convolutional neural networks and gating\nmechanisms, which is more accurate and efficient. First, the novel Gated\nTanh-ReLU Units can selectively output the sentiment features according to the\ngiven aspect or entity. The architecture is much simpler than attention layer\nused in the existing models. Second, the computations of our model could be\neasily parallelized during training, because convolutional layers do not have\ntime dependency as in LSTM layers, and gating units also work independently.\nThe experiments on SemEval datasets demonstrate the efficiency and\neffectiveness of our models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07049v1",
            "date": "2018-05-18 05:19:49+00:00",
            "title": "SNU_IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension",
            "authors": [
                "Taeuk Kim",
                "Jihun Choi",
                "Sang-goo Lee"
            ],
            "abstract": "We present a novel neural architecture for the Argument Reasoning\nComprehension task of SemEval 2018. It is a simple neural network consisting of\nthree parts, collectively judging whether the logic built on a set of given\nsentences (a claim, reason, and warrant) is plausible or not. The model\nutilizes contextualized word vectors pre-trained on large machine translation\n(MT) datasets as a form of transfer learning, which can help to mitigate the\nlack of training data. Quantitative analysis shows that simply leveraging LSTMs\ntrained on MT datasets outperforms several baselines and non-transferred\nmodels, achieving accuracies of about 70% on the development set and about 60%\non the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07133v1",
            "date": "2018-05-18 10:36:37+00:00",
            "title": "Combining Advanced Methods in Japanese-Vietnamese Neural Machine Translation",
            "authors": [
                "Thi-Vinh Ngo",
                "Thanh-Le Ha",
                "Phuong-Thai Nguyen",
                "Le-Minh Nguyen"
            ],
            "abstract": "Neural machine translation (NMT) systems have recently obtained state-of-the\nart in many machine translation systems between popular language pairs because\nof the availability of data. For low-resourced language pairs, there are few\nresearches in this field due to the lack of bilingual data. In this paper, we\nattempt to build the first NMT systems for a low-resourced language\npairs:Japanese-Vietnamese. We have also shown significant improvements when\ncombining advanced methods to reduce the adverse impacts of data sparsity and\nimprove the quality of NMT systems. In addition, we proposed a variant of\nByte-Pair Encoding algorithm to perform effective word segmentation for\nVietnamese texts and alleviate the rare-word problem that persists in NMT\nsystems.",
            "journal ref": "2018 10th International Conference on Knowledge and Systems\n  Engineering (KSE)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07143v1",
            "date": "2018-05-18 11:09:28+00:00",
            "title": "Style Obfuscation by Invariance",
            "authors": [
                "Chris Emmery",
                "Enrique Manjavacas",
                "Grzegorz Chrupa\u0142a"
            ],
            "abstract": "The task of obfuscating writing style using sequence models has previously\nbeen investigated under the framework of obfuscation-by-transfer, where the\ninput text is explicitly rewritten in another style. These approaches also\noften lead to major alterations to the semantic content of the input. In this\nwork, we propose obfuscation-by-invariance, and investigate to what extent\nmodels trained to be explicitly style-invariant preserve semantics. We evaluate\nour architectures on parallel and non-parallel corpora, and compare automatic\nand human evaluations on the obfuscated sentences. Our experiments show that\nstyle classifier performance can be reduced to chance level, whilst the\nautomatic evaluation of the output is seemingly equal to models applying\nstyle-transfer. However, based on human evaluation we demonstrate a trade-off\nbetween the level of obfuscation and the observed quality of the output in\nterms of meaning preservation and grammaticality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07231v2",
            "date": "2018-05-18 14:17:07+00:00",
            "title": "A Study on Dialog Act Recognition using Character-Level Tokenization",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "Dialog act recognition is an important step for dialog systems since it\nreveals the intention behind the uttered words. Most approaches on the task use\nword-level tokenization. In contrast, this paper explores the use of\ncharacter-level tokenization. This is relevant since there is information at\nthe sub-word level that is related to the function of the words and, thus,\ntheir intention. We also explore the use of different context windows around\neach token, which are able to capture important elements, such as affixes.\nFurthermore, we assess the importance of punctuation and capitalization. We\nperformed experiments on both the Switchboard Dialog Act Corpus and the DIHANA\nCorpus. In both cases, the experiments not only show that character-level\ntokenization leads to better performance than the typical word-level\napproaches, but also that both approaches are able to capture complementary\ninformation. Thus, the best results are achieved by combining tokenization at\nboth levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07469v1",
            "date": "2018-05-18 23:05:18+00:00",
            "title": "Metric for Automatic Machine Translation Evaluation based on Universal Sentence Representations",
            "authors": [
                "Hiroki Shimanaka",
                "Tomoyuki Kajiwara",
                "Mamoru Komachi"
            ],
            "abstract": "Sentence representations can capture a wide range of information that cannot\nbe captured by local features based on character or word N-grams. This paper\nexamines the usefulness of universal sentence representations for evaluating\nthe quality of machine translation. Although it is difficult to train sentence\nrepresentations using small-scale translation datasets with manual evaluation,\nsentence representations trained from large-scale data in other tasks can\nimprove the automatic evaluation of machine translation. Experimental results\nof the WMT-2016 dataset show that the proposed method achieves state-of-the-art\nperformance with sentence representation features only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07697v1",
            "date": "2018-05-20 03:42:50+00:00",
            "title": "The UN Parallel Corpus Annotated for Translation Direction",
            "authors": [
                "Elad Tolochinsky",
                "Ohad Mosafi",
                "Ella Rabinovich",
                "Shuly Wintner"
            ],
            "abstract": "This work distinguishes between translated and original text in the UN\nprotocol corpus. By modeling the problem as classification problem, we can\nachieve up to 95% classification accuracy. We begin by deriving a parallel\ncorpus for different language-pairs annotated for translation direction, and\nthen classify the data by using various feature extraction methods. We compare\nthe different methods as well as the ability to distinguish between translated\nand original texts in the different languages. The annotated corpus is publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07731v1",
            "date": "2018-05-20 08:40:06+00:00",
            "title": "Generating High-Quality Surface Realizations Using Data Augmentation and Factored Sequence Models",
            "authors": [
                "Henry Elder",
                "Chris Hokamp"
            ],
            "abstract": "This work presents a new state of the art in reconstruction of surface\nrealizations from obfuscated text. We identify the lack of sufficient training\ndata as the major obstacle to training high-performing models, and solve this\nissue by generating large amounts of synthetic training data. We also propose\npreprocessing techniques which make the structure contained in the input\nfeatures more accessible to sequence models. Our models were ranked first on\nall evaluation metrics in the English portion of the 2018 Surface Realization\nshared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07745v6",
            "date": "2018-05-20 09:34:20+00:00",
            "title": "Abstractive Text Classification Using Sequence-to-convolution Neural Networks",
            "authors": [
                "Taehoon Kim",
                "Jihoon Yang"
            ],
            "abstract": "We propose a new deep neural network model and its training scheme for text\nclassification. Our model Sequence-to-convolution Neural Networks(Seq2CNN)\nconsists of two blocks: Sequential Block that summarizes input texts and\nConvolution Block that receives summary of input and classifies it to a label.\nSeq2CNN is trained end-to-end to classify various-length texts without\npreprocessing inputs into fixed length. We also present Gradual Weight\nShift(GWS) method that stabilizes training. GWS is applied to our model's loss\nfunction. We compared our model with word-based TextCNN trained with different\ndata preprocessing methods. We obtained significant improvement in\nclassification accuracy over word-based TextCNN without any ensemble or data\naugmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07799v1",
            "date": "2018-05-20 17:16:49+00:00",
            "title": "A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)",
            "authors": [
                "Kamal Al-Sabahi",
                "Zhang Zuping",
                "Mohammed Nadher"
            ],
            "abstract": "The recent advance in neural network architecture and training algorithms\nhave shown the effectiveness of representation learning. The neural\nnetwork-based models generate better representation than the traditional ones.\nThey have the ability to automatically learn the distributed representation for\nsentences and documents. To this end, we proposed a novel model that addresses\nseveral issues that are not adequately modeled by the previously proposed\nmodels, such as the memory problem and incorporating the knowledge of document\nstructure. Our model uses a hierarchical structured self-attention mechanism to\ncreate the sentence and document embeddings. This architecture mirrors the\nhierarchical structure of the document and in turn enables us to obtain better\nfeature representation. The attention mechanism provides extra source of\ninformation to guide the summary extraction. The new model treated the\nsummarization task as a classification problem in which the model computes the\nrespective probabilities of sentence-summary membership. The model predictions\nare broken up by several features such as information content, salience,\nnovelty and positional representation. The proposed model was evaluated on two\nwell-known datasets, the CNN / Daily Mail, and DUC 2002. The experimental\nresults show that our model outperforms the current extractive state-of-the-art\nby a considerable margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07819v4",
            "date": "2018-05-20 20:00:27+00:00",
            "title": "Knowledge-enriched Two-layered Attention Network for Sentiment Analysis",
            "authors": [
                "Abhishek Kumar",
                "Daisuke Kawahara",
                "Sadao Kurohashi"
            ],
            "abstract": "We propose a novel two-layered attention network based on Bidirectional Long\nShort-Term Memory for sentiment analysis. The novel two-layered attention\nnetwork takes advantage of the external knowledge bases to improve the\nsentiment prediction. It uses the Knowledge Graph Embedding generated using the\nWordNet. We build our model by combining the two-layered attention network with\nthe supervised model based on Support Vector Regression using a Multilayer\nPerceptron network for sentiment analysis. We evaluate our model on the\nbenchmark dataset of SemEval 2017 Task 5. Experimental results show that the\nproposed model surpasses the top system of SemEval 2017 Task 5. The model\nperforms significantly better by improving the state-of-the-art system at\nSemEval 2017 Task 5 by 1.7 and 3.7 points for sub-tracks 1 and 2 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07824v1",
            "date": "2018-05-20 20:50:17+00:00",
            "title": "Validating WordNet Meronymy Relations using Adimen-SUMO",
            "authors": [
                "Javier \u00c1lvez",
                "Itziar Gonzalez-Dios",
                "German Rigau"
            ],
            "abstract": "In this paper, we report on the practical application of a novel approach for\nvalidating the knowledge of WordNet using Adimen-SUMO. In particular, this\npaper focuses on cross-checking the WordNet meronymy relations against the\nknowledge encoded in Adimen-SUMO. Our validation approach tests a large set of\ncompetency questions (CQs), which are derived (semi)-automatically from the\nknowledge encoded in WordNet, SUMO and their mapping, by applying efficient\nfirst-order logic automated theorem provers. Unfortunately, despite of being\ncreated manually, these knowledge resources are not free of errors and\ndiscrepancies. In consequence, some of the resulting CQs are not plausible\naccording to the knowledge included in Adimen-SUMO. Thus, first we focus on\n(semi)-automatically improving the alignment between these knowledge resources,\nand second, we perform a minimal set of corrections in the ontology. Our aim is\nto minimize the manual effort required for an extensive validation process. We\nreport on the strategies followed, the changes made, the effort needed and its\nimpact when validating the WordNet meronymy relations using improved versions\nof the mapping and the ontology. Based on the new results, we discuss the\nimplications of the appropriate corrections and the need of future\nenhancements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07858v1",
            "date": "2018-05-21 01:13:42+00:00",
            "title": "Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge",
            "authors": [
                "Todor Mihaylov",
                "Anette Frank"
            ],
            "abstract": "We introduce a neural reading comprehension model that integrates external\ncommonsense knowledge, encoded as a key-value memory, in a cloze-style setting.\nInstead of relying only on document-to-question interaction or discrete\nfeatures as in prior work, our model attends to relevant external knowledge and\ncombines this knowledge with the context representation before inferring the\nanswer. This allows the model to attract and imply knowledge from an external\nknowledge source that is not explicitly stated in the text, but that is\nrelevant for inferring the answer. Our model improves results over a very\nstrong baseline on a hard Common Nouns dataset, making it a strong competitor\nof much more complex models. By including knowledge explicitly, our model can\nalso provide evidence about the background knowledge used in the RC process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07882v1",
            "date": "2018-05-21 03:54:39+00:00",
            "title": "Sentence Modeling via Multiple Word Embeddings and Multi-level Comparison for Semantic Textual Similarity",
            "authors": [
                "Huy Nguyen Tien",
                "Minh Nguyen Le",
                "Yamasaki Tomohiro",
                "Izuha Tatsuya"
            ],
            "abstract": "Different word embedding models capture different aspects of linguistic\nproperties. This inspired us to propose a model (M-MaxLSTM-CNN) for employing\nmultiple sets of word embeddings for evaluating sentence similarity/relation.\nRepresenting each word by multiple word embeddings, the MaxLSTM-CNN encoder\ngenerates a novel sentence embedding. We then learn the similarity/relation\nbetween our sentence embeddings via Multi-level comparison. Our method\nM-MaxLSTM-CNN consistently shows strong performances in several tasks (i.e.,\nmeasure textual similarity, identify paraphrase, recognize textual entailment).\nAccording to the experimental results on STS Benchmark dataset and SICK dataset\nfrom SemEval, M-MaxLSTM-CNN outperforms the state-of-the-art methods for\ntextual similarity tasks. Our model does not use hand-crafted features (e.g.,\nalignment features, Ngram overlaps, dependency features) as well as does not\nrequire pre-trained word embeddings to have the same dimension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07889v2",
            "date": "2018-05-21 04:49:53+00:00",
            "title": "Improving Aspect Term Extraction with Bidirectional Dependency Tree Representation",
            "authors": [
                "Huaishao Luo",
                "Tianrui Li",
                "Bing Liu",
                "Bin Wang",
                "Herwig Unger"
            ],
            "abstract": "Aspect term extraction is one of the important subtasks in aspect-based\nsentiment analysis. Previous studies have shown that using dependency tree\nstructure representation is promising for this task. However, most dependency\ntree structures involve only one directional propagation on the dependency\ntree. In this paper, we first propose a novel bidirectional dependency tree\nnetwork to extract dependency structure features from the given sentences. The\nkey idea is to explicitly incorporate both representations gained separately\nfrom the bottom-up and top-down propagation on the given dependency syntactic\ntree. An end-to-end framework is then developed to integrate the embedded\nrepresentations and BiLSTM plus CRF to learn both tree-structured and\nsequential features to solve the aspect term extraction problem. Experimental\nresults demonstrate that the proposed model outperforms state-of-the-art\nbaseline models on four benchmark SemEval datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07946v2",
            "date": "2018-05-21 08:49:32+00:00",
            "title": "Morphological analysis using a sequence decoder",
            "authors": [
                "Ekin Aky\u00fcrek",
                "Erenay Dayan\u0131k",
                "Deniz Yuret"
            ],
            "abstract": "We introduce Morse, a recurrent encoder-decoder model that produces\nmorphological analyses of each word in a sentence. The encoder turns the\nrelevant information about the word and its context into a fixed size vector\nrepresentation and the decoder generates the sequence of characters for the\nlemma followed by a sequence of individual morphological features. We show that\ngenerating morphological features individually rather than as a combined tag\nallows the model to handle rare or unseen tags and outperform whole-tag models.\nIn addition, generating morphological features as a sequence rather than e.g.\\\nan unordered set allows our model to produce an arbitrary number of features\nthat represent multiple inflectional groups in morphologically complex\nlanguages. We obtain state-of-the art results in nine languages of different\nmorphological complexity under low-resource, high-resource and transfer\nlearning settings. We also introduce TrMor2018, a new high accuracy Turkish\nmorphology dataset. Our Morse implementation and the TrMor2018 dataset are\navailable online to support future research\\footnote{See\n\\url{https://github.com/ai-ku/Morse.jl} for a Morse implementation in\nJulia/Knet \\cite{knet2016mlsys} and \\url{https://github.com/ai-ku/TrMor2018}\nfor the new Turkish dataset.}.",
            "journal ref": "Transactions Of The Association For Computational Linguistics, 7,\n  567-579 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07952v1",
            "date": "2018-05-21 09:01:31+00:00",
            "title": "A new dataset and model for learning to understand navigational instructions",
            "authors": [
                "Ozan Arkan Can",
                "Deniz Yuret"
            ],
            "abstract": "In this paper, we present a state-of-the-art model and introduce a new\ndataset for grounded language learning. Our goal is to develop a model that can\nlearn to follow new instructions given prior instruction-perception-action\nexamples. We based our work on the SAIL dataset which consists of navigational\ninstructions and actions in a maze-like environment. The new model we propose\nachieves the best results to date on the SAIL dataset by using an improved\nperceptual component that can represent relative positions of objects. We also\nanalyze the problems with the SAIL dataset regarding its size and balance. We\nargue that performance on a small, fixed-size dataset is no longer a good\nmeasure to differentiate state-of-the-art models. We introduce SAILx, a\nsynthetic dataset generator, and perform experiments where the size and balance\nof the dataset are controlled.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08028v2",
            "date": "2018-05-21 12:59:17+00:00",
            "title": "Incorporating Glosses into Neural Word Sense Disambiguation",
            "authors": [
                "Fuli Luo",
                "Tianyu Liu",
                "Qiaolin Xia",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "Word Sense Disambiguation (WSD) aims to identify the correct meaning of\npolysemous words in the particular context. Lexical resources like WordNet\nwhich are proved to be of great help for WSD in the knowledge-based methods.\nHowever, previous neural networks for WSD always rely on massive labeled data\n(context), ignoring lexical resources like glosses (sense definitions). In this\npaper, we integrate the context and glosses of the target word into a unified\nframework in order to make full use of both labeled data and lexical knowledge.\nTherefore, we propose GAS: a gloss-augmented WSD neural network which jointly\nencodes the context and glosses of the target word. GAS models the semantic\nrelationship between the context and the gloss in an improved memory network\nframework, which breaks the barriers of the previous supervised methods and\nknowledge-based methods. We further extend the original gloss of word sense via\nits semantic relations in WordNet to enrich the gloss information. The\nexperimental results show that our model outperforms the state-of-theart\nsystems on several English all-words WSD datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08032v1",
            "date": "2018-05-21 13:07:31+00:00",
            "title": "A Talker Ensemble: the University of Wroc\u0142aw's Entry to the NIPS 2017 Conversational Intelligence Challenge",
            "authors": [
                "Jan Chorowski",
                "Adrian \u0141a\u0144cucki",
                "Szymon Malik",
                "Maciej Pawlikowski",
                "Pawe\u0142 Rychlikowski",
                "Pawe\u0142 Zykowski"
            ],
            "abstract": "We present Poetwannabe, a chatbot submitted by the University of Wroc{\\l}aw\nto the NIPS 2017 Conversational Intelligence Challenge, in which it ranked\nfirst ex-aequo. It is able to conduct a conversation with a user in a natural\nlanguage. The primary functionality of our dialogue system is context-aware\nquestion answering (QA), while its secondary function is maintaining user\nengagement. The chatbot is composed of a number of sub-modules, which\nindependently prepare replies to user's prompts and assess their own\nconfidence. To answer questions, our dialogue system relies heavily on factual\ndata, sourced mostly from Wikipedia and DBpedia, data of real user interactions\nin public forums, as well as data concerning general literature. Where\napplicable, modules are trained on large datasets using GPUs. However, to\ncomply with the competition's requirements, the final system is compact and\nruns on commodity hardware.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08092v1",
            "date": "2018-05-21 14:48:08+00:00",
            "title": "Efficient and Robust Question Answering from Minimal Context over Documents",
            "authors": [
                "Sewon Min",
                "Victor Zhong",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Neural models for question answering (QA) over documents have achieved\nsignificant performance improvements. Although effective, these models do not\nscale to large corpora due to their complex modeling of interactions between\nthe document and the question. Moreover, recent work has shown that such models\nare sensitive to adversarial inputs. In this paper, we study the minimal\ncontext required to answer the question, and find that most questions in\nexisting datasets can be answered with a small set of sentences. Inspired by\nthis observation, we propose a simple sentence selector to select the minimal\nset of sentences to feed into the QA model. Our overall system achieves\nsignificant reductions in training (up to 15 times) and inference times (up to\n13 times), with accuracy comparable to or better than the state-of-the-art on\nSQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results\nand analyses show that our approach is more robust to adversarial inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08093v1",
            "date": "2018-05-21 14:48:14+00:00",
            "title": "NeuralREG: An end-to-end approach to referring expression generation",
            "authors": [
                "Thiago Castro Ferreira",
                "Diego Moussallem",
                "\u00c1kos K\u00e1d\u00e1r",
                "Sander Wubben",
                "Emiel Krahmer"
            ],
            "abstract": "Traditionally, Referring Expression Generation (REG) models first decide on\nthe form and then on the content of references to discourse entities in text,\ntypically relying on features such as salience and grammatical function. In\nthis paper, we present a new approach (NeuralREG), relying on deep neural\nnetworks, which makes decisions about form and content in one go without\nexplicit feature extraction. Using a delexicalized version of the WebNLG\ncorpus, we show that the neural model substantially improves over two strong\nbaselines. Data and models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08099v1",
            "date": "2018-05-21 14:57:38+00:00",
            "title": "Computational Historical Linguistics",
            "authors": [
                "Gerhard J\u00e4ger"
            ],
            "abstract": "Computational approaches to historical linguistics have been proposed since\nhalf a century. Within the last decade, this line of research has received a\nmajor boost, owing both to the transfer of ideas and software from\ncomputational biology and to the release of several large electronic data\nresources suitable for systematic comparative work.\n  In this article, some of the central research topic of this new wave of\ncomputational historical linguistics are introduced and discussed. These are\nautomatic assessment of genetic relatedness, automatic cognate detection,\nphylogenetic inference and ancestral state reconstruction. They will be\ndemonstrated by means of a case study of automatically reconstructing a\nProto-Romance word list from lexical data of 50 modern Romance languages and\ndialects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08237v1",
            "date": "2018-05-21 18:09:23+00:00",
            "title": "Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings",
            "authors": [
                "Bernd Bohnet",
                "Ryan McDonald",
                "Goncalo Simoes",
                "Daniel Andor",
                "Emily Pitler",
                "Joshua Maynez"
            ],
            "abstract": "The rise of neural networks, and particularly recurrent neural networks, has\nproduced significant advances in part-of-speech tagging accuracy. One\ncharacteristic common among these models is the presence of rich initial word\nencodings. These encodings typically are composed of a recurrent\ncharacter-based representation with learned and pre-trained word embeddings.\nHowever, these encodings do not consider a context wider than a single word and\nit is only through subsequent recurrent layers that word or sub-word\ninformation interacts. In this paper, we investigate models that use recurrent\nneural networks with sentence-level context for initial character and\nword-based representations. In particular we show that optimal results are\nobtained by integrating these context sensitive representations through\nsynchronized training with a meta-model that learns to combine their states. We\npresent results on part-of-speech and morphological tagging with\nstate-of-the-art performance on a number of languages.",
            "journal ref": "ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08241v1",
            "date": "2018-05-21 18:14:35+00:00",
            "title": "Sparse and Constrained Attention for Neural Machine Translation",
            "authors": [
                "Chaitanya Malaviya",
                "Pedro Ferreira",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "In NMT, words are sometimes dropped from the source or generated repeatedly\nin the translation. We explore novel strategies to address the coverage problem\nthat change only the attention transformation. Our approach allocates\nfertilities to source words, used to bound the attention each word can receive.\nWe experiment with various sparse and constrained attention transformations and\npropose a new one, constrained sparsemax, shown to be differentiable and\nsparse. Empirical evaluation is provided in three languages pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08271v1",
            "date": "2018-05-21 19:57:23+00:00",
            "title": "Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction",
            "authors": [
                "Hongyuan Mei",
                "Sheng Zhang",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "Cross-lingual information extraction (CLIE) is an important and challenging\ntask, especially in low resource scenarios. To tackle this challenge, we\npropose a training method, called Halo, which enforces the local region of each\nhidden state of a neural model to only generate target tokens with the same\nsemantic structure tag. This simple but powerful technique enables a neural\nmodel to learn semantics-aware representations that are robust to noise,\nwithout introducing any extra parameter, thus yielding better generalization in\nboth high and low resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08297v1",
            "date": "2018-05-21 21:36:09+00:00",
            "title": "Character-based Neural Networks for Sentence Pair Modeling",
            "authors": [
                "Wuwei Lan",
                "Wei Xu"
            ],
            "abstract": "Sentence pair modeling is critical for many NLP tasks, such as paraphrase\nidentification, semantic textual similarity, and natural language inference.\nMost state-of-the-art neural models for these tasks rely on pretrained word\nembedding and compose sentence-level semantics in varied ways; however, few\nworks have attempted to verify whether we really need pretrained embeddings in\nthese tasks. In this paper, we study how effective subword-level (character and\ncharacter n-gram) representations are in sentence pair modeling. Though it is\nwell-known that subword models are effective in tasks with single sentence\ninput, including language modeling and machine translation, they have not been\nsystematically studied in sentence pair modeling tasks where the semantic and\nstring similarities between texts matter. Our experiments show that subword\nmodels without any pretrained word embedding can achieve new state-of-the-art\nresults on two social media datasets and competitive results on news data for\nparaphrase identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08352v1",
            "date": "2018-05-22 02:07:32+00:00",
            "title": "Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators",
            "authors": [
                "Shereen Oraby",
                "Lena Reed",
                "Shubhangi Tandon",
                "T. S. Sharath",
                "Stephanie Lukin",
                "Marilyn Walker"
            ],
            "abstract": "Natural language generators for task-oriented dialogue must effectively\nrealize system dialogue actions and their associated semantics. In many\napplications, it is also desirable for generators to control the style of an\nutterance. To date, work on task-oriented neural generation has primarily\nfocused on semantic fidelity rather than achieving stylistic goals, while work\non style has been done in contexts where it is difficult to measure content\npreservation. Here we present three different sequence-to-sequence models and\ncarefully test how well they disentangle content and style. We use a\nstatistical generator, Personage, to synthesize a new corpus of over 88,000\nrestaurant domain utterances whose style varies according to models of\npersonality, giving us total control over both the semantic content and the\nstylistic variation in the training data. We then vary the amount of explicit\nstylistic supervision given to the three models. We show that our most explicit\nmodel can simultaneously achieve high fidelity to both semantic and stylistic\ngoals: this model adds a context vector of 36 stylistic parameters as input to\nthe hidden state of the encoder at each time step, showing the benefits of\nexplicit stylistic supervision, even when the amount of training data is large.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08353v1",
            "date": "2018-05-22 02:09:02+00:00",
            "title": "Learning sentence embeddings using Recursive Networks",
            "authors": [
                "Anson Bastos"
            ],
            "abstract": "Learning sentence vectors that generalise well is a challenging task. In this\npaper we compare three methods of learning phrase embeddings: 1) Using LSTMs,\n2) using recursive nets, 3) A variant of the method 2 using the POS information\nof the phrase. We train our models on dictionary definitions of words to obtain\na reverse dictionary application similar to Felix et al. [1]. To see if our\nembeddings can be transferred to a new task we also train and test on the\nrotten tomatoes dataset [2]. We train keeping the sentence embeddings fixed as\nwell as with fine tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08415v1",
            "date": "2018-05-22 06:09:39+00:00",
            "title": "Estimating the Rating of Reviewers Based on the Text",
            "authors": [
                "Mohammadamir Kavousi",
                "Sepehr Saadatmand"
            ],
            "abstract": "User-generated texts such as reviews and social media are valuable sources of\ninformation. Online reviews are important assets for users to buy a product,\nsee a movie, or make a decision. Therefore, rating of a review is one of the\nreliable factors for all users to read and trust the reviews. This paper\nanalyzes the texts of the reviews to evaluate and predict the ratings.\nMoreover, we study the effect of lexical features generated from text as well\nas sentimental words on the accuracy of rating prediction. Our analysis show\nthat words with high information gain score are more efficient compared to\nwords with high TF-IDF value. In addition, we explore the best number of\nfeatures for predicting the ratings of the reviews.",
            "journal ref": "First International Conference on Data Analytics & Learning 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08438v1",
            "date": "2018-05-22 07:59:53+00:00",
            "title": "Paracompositionality, MWEs and Argument Substitution",
            "authors": [
                "Cem Bozsahin",
                "Arzu Burcu Guven"
            ],
            "abstract": "Multi-word expressions, verb-particle constructions, idiomatically combining\nphrases, and phrasal idioms have something in common: not all of their elements\ncontribute to the argument structure of the predicate implicated by the\nexpression.\n  Radically lexicalized theories of grammar that avoid string-, term-, logical\nform-, and tree-writing, and categorial grammars that avoid wrap operation,\nmake predictions about the categories involved in verb-particles and phrasal\nidioms. They may require singleton types, which can only substitute for one\nvalue, not just for one kind of value. These types are asymmetric: they can be\narguments only. They also narrowly constrain the kind of semantic value that\ncan correspond to such syntactic categories. Idiomatically combining phrases do\nnot subcategorize for singleton types, and they exploit another locally\ncomputable and compositional property of a correspondence, that every syntactic\nexpression can project its head word. Such MWEs can be seen as empirically\nrealized categorial possibilities, rather than lacuna in a theory of\nlexicalizable syntactic categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08533v1",
            "date": "2018-05-22 12:08:22+00:00",
            "title": "Sentiment Analysis of Arabic Tweets: Feature Engineering and A Hybrid Approach",
            "authors": [
                "Nora Al-Twairesh",
                "Hend Al-Khalifa",
                "AbdulMalik Alsalman",
                "Yousef Al-Ohali"
            ],
            "abstract": "Sentiment Analysis in Arabic is a challenging task due to the rich morphology\nof the language. Moreover, the task is further complicated when applied to\nTwitter data that is known to be highly informal and noisy. In this paper, we\ndevelop a hybrid method for sentiment analysis for Arabic tweets for a specific\nArabic dialect which is the Saudi Dialect. Several features were engineered and\nevaluated using a feature backward selection method. Then a hybrid method that\ncombines a corpus-based and lexicon-based method was developed for several\nclassification models (two-way, three-way, four-way). The best F1-score for\neach of these models was (69.9,61.63,55.07) respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08660v1",
            "date": "2018-05-22 15:25:29+00:00",
            "title": "Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment",
            "authors": [
                "Yue Gu",
                "Kangning Yang",
                "Shiyu Fu",
                "Shuhong Chen",
                "Xinyu Li",
                "Ivan Marsic"
            ],
            "abstract": "Multimodal affective computing, learning to recognize and interpret human\naffects and subjective information from multiple data sources, is still\nchallenging because: (i) it is hard to extract informative features to\nrepresent human affects from heterogeneous inputs; (ii) current fusion\nstrategies only fuse different modalities at abstract level, ignoring\ntime-dependent interactions between modalities. Addressing such issues, we\nintroduce a hierarchical multimodal architecture with attention and word-level\nfusion to classify utter-ance-level sentiment and emotion from text and audio\ndata. Our introduced model outperforms the state-of-the-art approaches on\npublished datasets and we demonstrated that our model is able to visualize and\ninterpret the synchronized attention over modalities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08701v1",
            "date": "2018-05-22 16:11:12+00:00",
            "title": "Normalization of Transliterated Words in Code-Mixed Data Using Seq2Seq Model & Levenshtein Distance",
            "authors": [
                "Soumil Mandal",
                "Karthick Nanmaran"
            ],
            "abstract": "Building tools for code-mixed data is rapidly gaining popularity in the NLP\nresearch community as such data is exponentially rising on social media.\nWorking with code-mixed data contains several challenges, especially due to\ngrammatical inconsistencies and spelling variations in addition to all the\nprevious known challenges for social media scenarios. In this article, we\npresent a novel architecture focusing on normalizing phonetic typing\nvariations, which is commonly seen in code-mixed data. One of the main features\nof our architecture is that in addition to normalizing, it can also be utilized\nfor back-transliteration and word identification in some cases. Our model\nachieved an accuracy of 90.27% on the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08914v1",
            "date": "2018-05-23 00:18:42+00:00",
            "title": "Enhancing Chinese Intent Classification by Dynamically Integrating Character Features into Word Embeddings with Ensemble Techniques",
            "authors": [
                "Ruixi Lin",
                "Charles Costello",
                "Charles Jankowski"
            ],
            "abstract": "Intent classification has been widely researched on English data with deep\nlearning approaches that are based on neural networks and word embeddings. The\nchallenge for Chinese intent classification stems from the fact that, unlike\nEnglish where most words are made up of 26 phonologic alphabet letters, Chinese\nis logographic, where a Chinese character is a more basic semantic unit that\ncan be informative and its meaning does not vary too much in contexts. Chinese\nword embeddings alone can be inadequate for representing words, and pre-trained\nembeddings can suffer from not aligning well with the task at hand. To account\nfor the inadequacy and leverage Chinese character information, we propose a\nlow-effort and generic way to dynamically integrate character embedding based\nfeature maps with word embedding based inputs, whose resulting word-character\nembeddings are stacked with a contextual information extraction module to\nfurther incorporate context information for predictions. On top of the proposed\nmodel, we employ an ensemble method to combine single models and obtain the\nfinal result. The approach is data-independent without relying on external\nsources like pre-trained word embeddings. The proposed model outperforms\nbaseline models and existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08983v1",
            "date": "2018-05-23 07:14:21+00:00",
            "title": "Self-Attention-Based Message-Relevant Response Generation for Neural Conversation Model",
            "authors": [
                "Jonggu Kim",
                "Doyeon Kong",
                "Jong-Hyeok Lee"
            ],
            "abstract": "Using a sequence-to-sequence framework, many neural conversation models for\nchit-chat succeed in naturalness of the response. Nevertheless, the neural\nconversation models tend to give generic responses which are not specific to\ngiven messages, and it still remains as a challenge. To alleviate the tendency,\nwe propose a method to promote message-relevant and diverse responses for\nneural conversation model by using self-attention, which is time-efficient as\nwell as effective. Furthermore, we present an investigation of why and how\neffective self-attention is in deep comparison with the standard dialogue\ngeneration. The experiment results show that the proposed method improves the\nstandard dialogue generation in various evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09007v1",
            "date": "2018-05-23 08:20:06+00:00",
            "title": "A Transition-based Algorithm for Unrestricted AMR Parsing",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Non-projective parsing can be useful to handle cycles and reentrancy in AMR\ngraphs. We explore this idea and introduce a greedy left-to-right\nnon-projective transition-based parser. At each parsing configuration, an\noracle decides whether to create a concept or whether to connect a pair of\nexisting concepts. The algorithm handles reentrancy and arbitrary cycles\nnatively, i.e. within the transition system itself. The model is evaluated on\nthe LDC2015E86 corpus, obtaining results close to the state of the art,\nincluding a Smatch of 64%, and showing good behavior on reentrant edges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09016v1",
            "date": "2018-05-23 08:56:15+00:00",
            "title": "Bilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languages",
            "authors": [
                "Jeremy Barnes",
                "Roman Klinger",
                "Sabine Schulte im Walde"
            ],
            "abstract": "Sentiment analysis in low-resource languages suffers from a lack of annotated\ncorpora to estimate high-performing models. Machine translation and bilingual\nword embeddings provide some relief through cross-lingual sentiment approaches.\nHowever, they either require large amounts of parallel data or do not\nsufficiently capture sentiment information. We introduce Bilingual Sentiment\nEmbeddings (BLSE), which jointly represent sentiment information in a source\nand target language. This model only requires a small bilingual lexicon, a\nsource-language corpus annotated for sentiment, and monolingual word embeddings\nfor each language. We perform experiments on three language combinations\n(Spanish, Catalan, Basque) for sentence-level cross-lingual sentiment\nclassification and find that our model significantly outperforms\nstate-of-the-art methods on four out of six experimental setups, as well as\ncapturing complementary information to machine translation. Our analysis of the\nresulting embedding space provides evidence that it represents sentiment\ninformation in the resource-poor target language without any annotated data in\nthat language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09055v1",
            "date": "2018-05-23 11:04:46+00:00",
            "title": "Grounding the Semantics of Part-of-Day Nouns Worldwide using Twitter",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "The usage of part-of-day nouns, such as 'night', and their time-specific\ngreetings ('good night'), varies across languages and cultures. We show the\npossibilities that Twitter offers for studying the semantics of these terms and\nits variability between countries. We mine a worldwide sample of multilingual\ntweets with temporal greetings, and study how their frequencies vary in\nrelation with local time. The results provide insights into the semantics of\nthese temporal expressions and the cultural and sociological factors\ninfluencing their usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09119v1",
            "date": "2018-05-23 13:20:16+00:00",
            "title": "Selecting Machine-Translated Data for Quick Bootstrapping of a Natural Language Understanding System",
            "authors": [
                "Judith Gaspers",
                "Penny Karanasou",
                "Rajen Chatterjee"
            ],
            "abstract": "This paper investigates the use of Machine Translation (MT) to bootstrap a\nNatural Language Understanding (NLU) system for a new language for the use case\nof a large-scale voice-controlled device. The goal is to decrease the cost and\ntime needed to get an annotated corpus for the new language, while still having\na large enough coverage of user requests. Different methods of filtering MT\ndata in order to keep utterances that improve NLU performance and\nlanguage-specific post-processing methods are investigated. These methods are\ntested in a large-scale NLU task with translating around 10 millions training\nutterances from English to German. The results show a large improvement for\nusing MT data over a grammar-based and over an in-house data collection\nbaseline, while reducing the manual effort greatly. Both filtering and\npost-processing approaches improve results further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09209v2",
            "date": "2018-05-23 14:58:13+00:00",
            "title": "How much does a word weigh? Weighting word embeddings for word sense induction",
            "authors": [
                "Nikolay Arefyev",
                "Pavel Ermolaev",
                "Alexander Panchenko"
            ],
            "abstract": "The paper describes our participation in the first shared task on word sense\ninduction and disambiguation for the Russian language RUSSE'2018 (Panchenko et\nal., 2018). For each of several dozens of ambiguous words, the participants\nwere asked to group text fragments containing it according to the senses of\nthis word, which were not provided beforehand, therefore the \"induction\" part\nof the task. For instance, a word \"bank\" and a set of text fragments (also\nknown as \"contexts\") in which this word occurs, e.g. \"bank is a financial\ninstitution that accepts deposits\" and \"river bank is a slope beside a body of\nwater\" were given. A participant was asked to cluster such contexts in the\nunknown in advance number of clusters corresponding to, in this case, the\n\"company\" and the \"area\" senses of the word \"bank\". The organizers proposed\nthree evaluation datasets of varying complexity and text genres based\nrespectively on texts of Wikipedia, Web pages, and a dictionary of the Russian\nlanguage. We present two experiments: a positive and a negative one, based\nrespectively on clustering of contexts represented as a weighted average of\nword embeddings and on machine translation using two state-of-the-art\nproduction neural machine translation systems. Our team showed the second best\nresult on two datasets and the third best result on the remaining one dataset\namong 18 participating teams. We managed to substantially outperform\ncompetitive state-of-the-art baselines from the previous years based on sense\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09354v1",
            "date": "2018-05-23 18:03:08+00:00",
            "title": "Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module",
            "authors": [
                "Juan Pavez",
                "H\u00e9ctor Allende",
                "H\u00e9ctor Allende-Cid"
            ],
            "abstract": "During the last years, there has been a lot of interest in achieving some\nkind of complex reasoning using deep neural networks. To do that, models like\nMemory Networks (MemNNs) have combined external memory storages and attention\nmechanisms. These architectures, however, lack of more complex reasoning\nmechanisms that could allow, for instance, relational reasoning. Relation\nNetworks (RNs), on the other hand, have shown outstanding results in relational\nreasoning tasks. Unfortunately, their computational cost grows quadratically\nwith the number of memories, something prohibitive for larger problems. To\nsolve these issues, we introduce the Working Memory Network, a MemNN\narchitecture with a novel working memory storage and reasoning module. Our\nmodel retains the relational reasoning abilities of the RN while reducing its\ncomputational complexity from quadratic to linear. We tested our model on the\ntext QA dataset bAbI and the visual QA dataset NLVR. In the jointly trained\nbAbI-10k, we set a new state-of-the-art, achieving a mean error of less than\n0.5%. Moreover, a simple ensemble of two of our models solves all 20 tasks in\nthe joint version of the benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09389v1",
            "date": "2018-05-23 19:20:43+00:00",
            "title": "Embedding Syntax and Semantics of Prepositions via Tensor Decomposition",
            "authors": [
                "Hongyu Gong",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Prepositions are among the most frequent words in English and play complex\nroles in the syntax and semantics of sentences. Not surprisingly, they pose\nwell-known difficulties in automatic processing of sentences (prepositional\nattachment ambiguities and idiosyncratic uses in phrases). Existing methods on\npreposition representation treat prepositions no different from content words\n(e.g., word2vec and GloVe). In addition, recent studies aiming at solving\nprepositional attachment and preposition selection problems depend heavily on\nexternal linguistic resources and use dataset-specific word representations. In\nthis paper we use word-triple counts (one of the triples being a preposition)\nto capture a preposition's interaction with its attachment and complement. We\nthen derive preposition embeddings via tensor decomposition on a large\nunlabeled corpus. We reveal a new geometry involving Hadamard products and\nempirically demonstrate its utility in paraphrasing phrasal verbs. Furthermore,\nour preposition embeddings are used as simple features in two challenging\ndownstream tasks: preposition selection and prepositional attachment\ndisambiguation. We achieve results comparable to or better than the\nstate-of-the-art on multiple standardized datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09436v1",
            "date": "2018-05-23 21:38:45+00:00",
            "title": "Modeling Interpersonal Influence of Verbal Behavior in Couples Therapy Dyadic Interactions",
            "authors": [
                "Sandeep Nallan Chakravarthula",
                "Brian Baucom",
                "Panayiotis Georgiou"
            ],
            "abstract": "Dyadic interactions among humans are marked by speakers continuously\ninfluencing and reacting to each other in terms of responses and behaviors,\namong others. Understanding how interpersonal dynamics affect behavior is\nimportant for successful treatment in psychotherapy domains. Traditional\nschemes that automatically identify behavior for this purpose have often looked\nat only the target speaker. In this work, we propose a Markov model of how a\ntarget speaker's behavior is influenced by their own past behavior as well as\ntheir perception of their partner's behavior, based on lexical features. Apart\nfrom incorporating additional potentially useful information, our model can\nalso control the degree to which the partner affects the target speaker. We\nevaluate our proposed model on the task of classifying Negative behavior in\nCouples Therapy and show that it is more accurate than the single-speaker\nmodel. Furthermore, we investigate the degree to which the optimal influence\nrelates to how well a couple does on the long-term, via relating to\nrelationship outcomes",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09590v1",
            "date": "2018-05-24 10:24:47+00:00",
            "title": "Native Language Cognate Effects on Second Language Lexical Choice",
            "authors": [
                "Ella Rabinovich",
                "Yulia Tsvetkov",
                "Shuly Wintner"
            ],
            "abstract": "We present a computational analysis of cognate effects on the spontaneous\nlinguistic productions of advanced non-native speakers. Introducing a large\ncorpus of highly competent non-native English speakers, and using a set of\ncarefully selected lexical items, we show that the lexical choices of\nnon-natives are affected by cognates in their native language. This effect is\nso powerful that we are able to reconstruct the phylogenetic language tree of\nthe Indo-European language family solely from the frequencies of specific\nlexical items in the English of authors with various native languages. We\nquantitatively analyze non-native lexical choice, highlighting cognate\nfacilitation as one of the important phenomena shaping the language of\nnon-native speakers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09648v1",
            "date": "2018-04-05 16:07:08+00:00",
            "title": "Crowd-Labeling Fashion Reviews with Quality Control",
            "authors": [
                "Iurii Chernushenko",
                "Felix A. Gers",
                "Alexander L\u00f6ser",
                "Alessandro Checco"
            ],
            "abstract": "We present a new methodology for high-quality labeling in the fashion domain\nwith crowd workers instead of experts. We focus on the Aspect-Based Sentiment\nAnalysis task. Our methods filter out inaccurate input from crowd workers but\nwe preserve different worker labeling to capture the inherent high variability\nof the opinions. We demonstrate the quality of labeled data based on Facebook's\nFastText framework as a baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09746v2",
            "date": "2018-05-24 15:56:08+00:00",
            "title": "Letting Emotions Flow: Success Prediction by Modeling the Flow of Emotions in Books",
            "authors": [
                "Suraj Maharjan",
                "Sudipta Kar",
                "Manuel Montes-y-Gomez",
                "Fabio A. Gonzalez",
                "Thamar Solorio"
            ],
            "abstract": "Books have the power to make us feel happiness, sadness, pain, surprise, or\nsorrow. An author's dexterity in the use of these emotions captivates readers\nand makes it difficult for them to put the book down. In this paper, we model\nthe flow of emotions over a book using recurrent neural networks and quantify\nits usefulness in predicting success in books. We obtained the best weighted\nF1-score of 69% for predicting books' success in a multitask setting\n(simultaneously predicting success and genre of books).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09821v1",
            "date": "2018-05-24 10:36:20+00:00",
            "title": "A Corpus for Multilingual Document Classification in Eight Languages",
            "authors": [
                "Holger Schwenk",
                "Xian Li"
            ],
            "abstract": "Cross-lingual document classification aims at training a document classifier\non resources in one language and transferring it to a different language\nwithout any additional resources. Several approaches have been proposed in the\nliterature and the current best practice is to evaluate them on a subset of the\nReuters Corpus Volume 2. However, this subset covers only few languages\n(English, German, French and Spanish) and almost all published works focus on\nthe the transfer between English and German. In addition, we have observed that\nthe class prior distributions differ significantly between the languages. We\nargue that this complicates the evaluation of the multilinguality. In this\npaper, we propose a new subset of the Reuters corpus with balanced class priors\nfor eight languages. By adding Italian, Russian, Japanese and Chinese, we cover\nlanguages which are very different with respect to syntax, morphology, etc. We\nprovide strong baselines for all language transfer directions using\nmultilingual word and sentence embeddings respectively. Our goal is to offer a\nfreely available framework to evaluate cross-lingual document classification,\nand we hope to foster by these means, research in this important area.",
            "journal ref": "LREC, May 2018, Miyazaki, Japan",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09863v3",
            "date": "2018-05-24 19:33:03+00:00",
            "title": "Fast Neural Machine Translation Implementation",
            "authors": [
                "Hieu Hoang",
                "Tomasz Dwojak",
                "Rihards Krislauks",
                "Daniel Torregrosa",
                "Kenneth Heafield"
            ],
            "abstract": "This paper describes the submissions to the efficiency track for GPUs at the\nWorkshop for Neural Machine Translation and Generation by members of the\nUniversity of Edinburgh, Adam Mickiewicz University, Tilde and University of\nAlicante. We focus on efficient implementation of the recurrent deep-learning\nmodel as implemented in Amun, the fast inference engine for neural machine\ntranslation. We improve the performance with an efficient mini-batching\nalgorithm, and by fusing the softmax operation with the k-best extraction\nalgorithm. Submissions using Amun were first, second and third fastest in the\nGPU efficiency track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09927v1",
            "date": "2018-05-24 22:32:55+00:00",
            "title": "Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning",
            "authors": [
                "Pengda Qin",
                "Weiran Xu",
                "William Yang Wang"
            ],
            "abstract": "Distant supervision has become the standard method for relation extraction.\nHowever, even though it is an efficient method, it does not come at no\ncost---The resulted distantly-supervised training samples are often very noisy.\nTo combat the noise, most of the recent state-of-the-art approaches focus on\nselecting one-best sentence or calculating soft attention weights over the set\nof the sentences of one specific entity pair. However, these methods are\nsuboptimal, and the false positive problem is still a key stumbling bottleneck\nfor the performance. We argue that those incorrectly-labeled candidate\nsentences must be treated with a hard decision, rather than being dealt with\nsoft attention weights. To do this, our paper describes a radical solution---We\nexplore a deep reinforcement learning strategy to generate the false-positive\nindicator, where we automatically recognize false positives for each relation\ntype without any supervised information. Unlike the removal operation in the\nprevious studies, we redistribute them into the negative examples. The\nexperimental results show that the proposed strategy significantly improves the\nperformance of distant supervision comparing to state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09929v1",
            "date": "2018-05-24 22:37:59+00:00",
            "title": "DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction",
            "authors": [
                "Pengda Qin",
                "Weiran Xu",
                "William Yang Wang"
            ],
            "abstract": "Distant supervision can effectively label data for relation extraction, but\nsuffers from the noise labeling problem. Recent works mainly perform soft\nbag-level noise reduction strategies to find the relatively better samples in a\nsentence bag, which is suboptimal compared with making a hard decision of false\npositive samples in sentence level. In this paper, we introduce an adversarial\nlearning framework, which we named DSGAN, to learn a sentence-level\ntrue-positive generator. Inspired by Generative Adversarial Networks, we regard\nthe positive samples generated by the generator as the negative samples to\ntrain the discriminator. The optimal generator is obtained until the\ndiscrimination ability of the discriminator has the greatest decline. We adopt\nthe generator to filter distant supervision training dataset and redistribute\nthe false positive instances into the negative set, in which way to provide a\ncleaned dataset for relation classification. The experimental results show that\nthe proposed strategy significantly improves the performance of distant\nsupervision relation extraction comparing to state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09960v1",
            "date": "2018-05-25 03:14:27+00:00",
            "title": "Phrase Table as Recommendation Memory for Neural Machine Translation",
            "authors": [
                "Yang Zhao",
                "Yining Wang",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Neural Machine Translation (NMT) has drawn much attention due to its\npromising translation performance recently. However, several studies indicate\nthat NMT often generates fluent but unfaithful translations. In this paper, we\npropose a method to alleviate this problem by using a phrase table as\nrecommendation memory. The main idea is to add bonus to words worthy of\nrecommendation, so that NMT can make correct predictions. Specifically, we\nfirst derive a prefix tree to accommodate all the candidate target phrases by\nsearching the phrase translation table according to the source sentence. Then,\nwe construct a recommendation word set by matching between candidate target\nphrases and previously translated target words by NMT. After that, we determine\nthe specific bonus value for each recommendable word by using the attention\nvector and phrase translation probability. Finally, we integrate this bonus\nvalue into NMT to improve the translation results. The extensive experiments\ndemonstrate that the proposed methods obtain remarkable improvements over the\nstrong attentionbased NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09991v1",
            "date": "2018-05-25 06:10:54+00:00",
            "title": "Lifelong Domain Word Embedding via Meta-Learning",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Learning high-quality domain word embeddings is important for achieving good\nperformance in many NLP tasks. General-purpose embeddings trained on\nlarge-scale corpora are often sub-optimal for domain-specific applications.\nHowever, domain-specific tasks often do not have large in-domain corpora for\ntraining high-quality domain embeddings. In this paper, we propose a novel\nlifelong learning setting for domain embedding. That is, when performing the\nnew domain embedding, the system has seen many past domains, and it tries to\nexpand the new in-domain corpus by exploiting the corpora from the past domains\nvia meta-learning. The proposed meta-learner characterizes the similarities of\nthe contexts of the same word in many domain corpora, which helps retrieve\nrelevant data from the past domains to expand the new domain corpus.\nExperimental results show that domain embeddings produced from such a process\nimprove the performance of the downstream tasks.",
            "journal ref": "IJCAI 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10047v1",
            "date": "2018-05-25 08:56:43+00:00",
            "title": "Japanese Predicate Conjugation for Neural Machine Translation",
            "authors": [
                "Michiki Kurosawa",
                "Yukio Matsumura",
                "Hayahide Yamagishi",
                "Mamoru Komachi"
            ],
            "abstract": "Neural machine translation (NMT) has a drawback in that can generate only\nhigh-frequency words owing to the computational costs of the softmax function\nin the output layer.\n  In Japanese-English NMT, Japanese predicate conjugation causes an increase in\nvocabulary size. For example, one verb can have as many as 19 surface\nvarieties. In this research, we focus on predicate conjugation for compressing\nthe vocabulary size in Japanese. The vocabulary list is filled with the various\nforms of verbs. We propose methods using predicate conjugation information\nwithout discarding linguistic information. The proposed methods can generate\nlow-frequency words and deal with unknown words. Two methods were considered to\nintroduce conjugation information: the first considers it as a token\n(conjugation token) and the second considers it as an embedded vector\n(conjugation feature).\n  The results using these methods demonstrate that the vocabulary size can be\ncompressed by approximately 86.1% (Tanaka corpus) and the NMT models can output\nthe words not in the training data set. Furthermore, BLEU scores improved by\n0.91 points in Japanese-to-English translation, and 0.32 points in\nEnglish-to-Japanese translation with ASPEC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10163v1",
            "date": "2018-05-25 14:03:27+00:00",
            "title": "Context-Aware Neural Machine Translation Learns Anaphora Resolution",
            "authors": [
                "Elena Voita",
                "Pavel Serdyukov",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "Standard machine translation systems process sentences in isolation and hence\nignore extra-sentential information, even though extended context can both\nprevent mistakes in ambiguous cases and improve translation coherence. We\nintroduce a context-aware neural machine translation model designed in such way\nthat the flow of information from the extended context to the translation model\ncan be controlled and analyzed. We experiment with an English-Russian subtitles\ndataset, and observe that much of what is captured by our model deals with\nimproving pronoun translation. We measure correspondences between induced\nattention distributions and coreference relations and observe that the model\nimplicitly captures anaphora. It is consistent with gains for sentences where\npronouns need to be gendered in translation. Beside improvements in anaphoric\ncases, the model also improves in overall BLEU, both over its context-agnostic\nversion (+0.7) and over simple concatenation of the context and source\nsentences (+0.6).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10187v1",
            "date": "2018-05-25 15:00:30+00:00",
            "title": "Recursive Neural Network Based Preordering for English-to-Japanese Machine Translation",
            "authors": [
                "Yuki Kawara",
                "Chenhui Chu",
                "Yuki Arase"
            ],
            "abstract": "The word order between source and target languages significantly influences\nthe translation quality in machine translation. Preordering can effectively\naddress this problem. Previous preordering methods require a manual feature\ndesign, making language dependent design costly. In this paper, we propose a\npreordering method with a recursive neural network that learns features from\nraw inputs. Experiments show that the proposed method achieves comparable gain\nin translation quality to the state-of-the-art method but without a manual\nfeature design.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10209v2",
            "date": "2018-05-25 15:47:38+00:00",
            "title": "Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation",
            "authors": [
                "Alane Suhr",
                "Yoav Artzi"
            ],
            "abstract": "We propose a learning approach for mapping context-dependent sequential\ninstructions to actions. We address the problem of discourse and state\ndependencies with an attention-based model that considers both the history of\nthe interaction and the state of the world. To train from start and goal states\nwithout access to demonstrations, we propose SESTRA, a learning algorithm that\ntakes advantage of single-step reward observations and immediate expected\nreward maximization. We evaluate on the SCONE domains, and show absolute\naccuracy improvements of 9.8%-25.3% across the domains over approaches that use\nhigh-level logical representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10254v1",
            "date": "2018-05-25 17:12:03+00:00",
            "title": "Neural Argument Generation Augmented with Externally Retrieved Evidence",
            "authors": [
                "Xinyu Hua",
                "Lu Wang"
            ],
            "abstract": "High quality arguments are essential elements for human reasoning and\ndecision-making processes. However, effective argument construction is a\nchallenging task for both human and machines. In this work, we study a novel\ntask on automatically generating arguments of a different stance for a given\nstatement. We propose an encoder-decoder style neural network-based argument\ngeneration model enriched with externally retrieved evidence from Wikipedia.\nOur model first generates a set of talking point phrases as intermediate\nrepresentation, followed by a separate decoder producing the final argument\nbased on both input and the keyphrases. Experiments on a large-scale dataset\ncollected from Reddit show that our model constructs arguments with more\ntopic-relevant content than a popular sequence-to-sequence generation model\naccording to both automatic evaluation and human assessments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10267v1",
            "date": "2018-05-25 17:36:51+00:00",
            "title": "Duluth UROP at SemEval-2018 Task 2: Multilingual Emoji Prediction with Ensemble Learning and Oversampling",
            "authors": [
                "Shuning Jin",
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth UROP systems that participated in\nSemEval--2018 Task 2, Multilingual Emoji Prediction. We relied on a variety of\nensembles made up of classifiers using Naive Bayes, Logistic Regression, and\nRandom Forests. We used unigram and bigram features and tried to offset the\nskewness of the data through the use of oversampling. Our task evaluation\nresults place us 19th of 48 systems in the English evaluation, and 5th of 21 in\nthe Spanish. After the evaluation we realized that some simple changes to\npreprocessing could significantly improve our results. After making these\nchanges we attained results that would have placed us sixth in the English\nevaluation, and second in the Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10271v1",
            "date": "2018-05-25 17:44:03+00:00",
            "title": "UMDuluth-CS8761 at SemEval-2018 Task 9: Hypernym Discovery using Hearst Patterns, Co-occurrence frequencies and Word Embeddings",
            "authors": [
                "Arshia Z. Hassan",
                "Manikya S. Vallabhajosyula",
                "Ted Pedersen"
            ],
            "abstract": "Hypernym Discovery is the task of identifying potential hypernyms for a given\nterm. A hypernym is a more generalized word that is super-ordinate to more\nspecific words. This paper explores several approaches that rely on\nco-occurrence frequencies of word pairs, Hearst Patterns based on regular\nexpressions, and word embeddings created from the UMBC corpus. Our system\nBabbage participated in Subtask 1A for English and placed 6th of 19 systems\nwhen identifying concept hypernyms, and 12th of 18 systems for entity\nhypernyms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10274v1",
            "date": "2018-05-25 17:48:20+00:00",
            "title": "UMDSub at SemEval-2018 Task 2: Multilingual Emoji Prediction Multi-channel Convolutional Neural Network on Subword Embedding",
            "authors": [
                "Zhenduo Wang",
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the UMDSub system that participated in Task 2 of\nSemEval-2018. We developed a system that predicts an emoji given the raw text\nin a English tweet. The system is a Multi-channel Convolutional Neural Network\nbased on subword embeddings for the representation of tweets. This model\nimproves on character or word based methods by about 2\\%. Our system placed\n21st of 48 participating systems in the official evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10387v2",
            "date": "2018-05-25 22:54:38+00:00",
            "title": "Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq",
            "authors": [
                "Oleksii Kuchaiev",
                "Boris Ginsburg",
                "Igor Gitman",
                "Vitaly Lavrukhin",
                "Jason Li",
                "Huyen Nguyen",
                "Carl Case",
                "Paulius Micikevicius"
            ],
            "abstract": "We present OpenSeq2Seq - a TensorFlow-based toolkit for training\nsequence-to-sequence models that features distributed and mixed-precision\ntraining. Benchmarks on machine translation and speech recognition tasks show\nthat models built using OpenSeq2Seq give state-of-the-art performance at 1.5-3x\nless training time. OpenSeq2Seq currently provides building blocks for models\nthat solve a wide range of tasks including neural machine translation,\nautomatic speech recognition, and speech synthesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10389v1",
            "date": "2018-05-25 23:00:03+00:00",
            "title": "A Study of Question Effectiveness Using Reddit \"Ask Me Anything\" Threads",
            "authors": [
                "Kristjan Arumae",
                "Guo-Jun Qi",
                "Fei Liu"
            ],
            "abstract": "Asking effective questions is a powerful social skill. In this paper we seek\nto build computational models that learn to discriminate effective questions\nfrom ineffective ones. Armed with such a capability, future advanced systems\ncan evaluate the quality of questions and provide suggestions for effective\nquestion wording. We create a large-scale, real-world dataset that contains\nover 400,000 questions collected from Reddit \"Ask Me Anything\" threads. Each\nthread resembles an online press conference where questions compete with each\nother for attention from the host. This dataset enables the development of a\nclass of computational models for predicting whether a question will be\nanswered. We develop a new convolutional neural network architecture with\nvariable-length context and demonstrate the efficacy of the model by comparing\nit with state-of-the-art baselines and human judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10390v2",
            "date": "2018-05-25 23:01:01+00:00",
            "title": "Toward Extractive Summarization of Online Forum Discussions via Hierarchical Attention Networks",
            "authors": [
                "Sansiri Tarnpradab",
                "Fei Liu",
                "Kien A. Hua"
            ],
            "abstract": "Forum threads are lengthy and rich in content. Concise thread summaries will\nbenefit both newcomers seeking information and those who participate in the\ndiscussion. Few studies, however, have examined the task of forum thread\nsummarization. In this work we make the first attempt to adapt the hierarchical\nattention networks for thread summarization. The model draws on the recent\ndevelopment of neural attention mechanisms to build sentence and thread\nrepresentations and use them for summarization. Our results indicate that the\nproposed approach can outperform a range of competitive baselines. Further, a\nredundancy removal step is crucial for achieving outstanding results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10392v2",
            "date": "2018-05-25 23:05:48+00:00",
            "title": "Reinforced Extractive Summarization with Question-Focused Rewards",
            "authors": [
                "Kristjan Arumae",
                "Fei Liu"
            ],
            "abstract": "We investigate a new training paradigm for extractive summarization.\nTraditionally, human abstracts are used to derive goldstandard labels for\nextraction units. However, the labels are often inaccurate, because human\nabstracts and source documents cannot be easily aligned at the word level. In\nthis paper we convert human abstracts to a set of Cloze-style comprehension\nquestions. System summaries are encouraged to preserve salient source content\nuseful for answering questions and share common words with the abstracts. We\nuse reinforcement learning to explore the space of possible extractive\nsummaries and introduce a question-focused reward function to promote concise,\nfluent, and informative summaries. Our experiments show that the proposed\nmethod is effective. It surpasses state-of-the-art systems on the standard\nsummarization dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10393v1",
            "date": "2018-05-25 23:13:41+00:00",
            "title": "Modeling Language Vagueness in Privacy Policies using Deep Neural Networks",
            "authors": [
                "Fei Liu",
                "Nicole Lee Fella",
                "Kexin Liao"
            ],
            "abstract": "Website privacy policies are too long to read and difficult to understand.\nThe over-sophisticated language makes privacy notices to be less effective than\nthey should be. People become even less willing to share their personal\ninformation when they perceive the privacy policy as vague. This paper focuses\non decoding vagueness from a natural language processing perspective. While\nthoroughly identifying the vague terms and their linguistic scope remains an\nelusive challenge, in this work we seek to learn vector representations of\nwords in privacy policies using deep neural networks. The vector\nrepresentations are fed to an interactive visualization tool (LSTMVis) to test\non their ability to discover syntactically and semantically related vague\nterms. The approach holds promise for modeling and understanding language\nvagueness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10395v1",
            "date": "2018-05-25 23:36:33+00:00",
            "title": "Automatic Summarization of Student Course Feedback",
            "authors": [
                "Wencan Luo",
                "Fei Liu",
                "Zitao Liu",
                "Diane Litman"
            ],
            "abstract": "Student course feedback is generated daily in both classrooms and online\ncourse discussion forums. Traditionally, instructors manually analyze these\nresponses in a costly manner. In this work, we propose a new approach to\nsummarizing student course feedback based on the integer linear programming\n(ILP) framework. Our approach allows different student responses to share\nco-occurrence statistics and alleviates sparsity issues. Experimental results\non a student feedback corpus show that our approach outperforms a range of\nbaselines in terms of both ROUGE scores and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10396v1",
            "date": "2018-05-25 23:38:36+00:00",
            "title": "An Improved Phrase-based Approach to Annotating and Summarizing Student Course Responses",
            "authors": [
                "Wencan Luo",
                "Fei Liu",
                "Diane Litman"
            ],
            "abstract": "Teaching large classes remains a great challenge, primarily because it is\ndifficult to attend to all the student needs in a timely manner. Automatic text\nsummarization systems can be leveraged to summarize the student feedback,\nsubmitted immediately after each lecture, but it is left to be discovered what\nmakes a good summary for student responses. In this work we explore a new\nmethodology that effectively extracts summary phrases from the student\nresponses. Each phrase is tagged with the number of students who raise the\nissue. The phrases are evaluated along two dimensions: with respect to text\ncontent, they should be informative and well-formed, measured by the ROUGE\nmetric; additionally, they shall attend to the most pressing student needs,\nmeasured by a newly proposed metric. This work is enabled by a phrase-based\nannotation and highlighting scheme, which is new to the summarization task. The\nphrase-based framework allows us to summarize the student responses into a set\nof bullet points and present to the instructor promptly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10399v1",
            "date": "2018-05-25 23:46:11+00:00",
            "title": "Toward Abstractive Summarization Using Semantic Representations",
            "authors": [
                "Fei Liu",
                "Jeffrey Flanigan",
                "Sam Thomson",
                "Norman Sadeh",
                "Noah A. Smith"
            ],
            "abstract": "We present a novel abstractive summarization framework that draws on the\nrecent development of a treebank for the Abstract Meaning Representation (AMR).\nIn this framework, the source text is parsed to a set of AMR graphs, the graphs\nare transformed into a summary graph, and then text is generated from the\nsummary graph. We focus on the graph-to-graph transformation that reduces the\nsource semantic graph into a summary graph, making use of an existing AMR\nparser and assuming the eventual availability of an AMR-to-text generator. The\nframework is data-driven, trainable, and not specifically designed for a\nparticular domain. Experiments on gold-standard AMR annotations and system\nparses show promising results. Code is available at:\nhttps://github.com/summarization",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10414v1",
            "date": "2018-05-26 02:39:10+00:00",
            "title": "Connecting Distant Entities with Induction through Conditional Random Fields for Named Entity Recognition: Precursor-Induced CRF",
            "authors": [
                "Wangjin Lee",
                "Jinwook Choi"
            ],
            "abstract": "This paper presents a method of designing specific high-order dependency\nfactor on the linear chain conditional random fields (CRFs) for named entity\nrecognition (NER). Named entities tend to be separated from each other by\nmultiple outside tokens in a text, and thus the first-order CRF, as well as the\nsecond-order CRF, may innately lose transition information between distant\nnamed entities. The proposed design uses outside label in NER as a transmission\nmedium of precedent entity information on the CRF. Then, empirical results\napparently demonstrate that it is possible to exploit long-distance label\ndependency in the original first-order linear chain CRF structure upon NER\nwhile reducing computational loss rather than in the second-order CRF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10465v1",
            "date": "2018-05-26 11:55:59+00:00",
            "title": "SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings",
            "authors": [
                "Zhuosheng Zhang",
                "Jiangtong Li",
                "Hai Zhao",
                "Bingjie Tang"
            ],
            "abstract": "This paper describes a hypernym discovery system for our participation in the\nSemEval-2018 Task 9, which aims to discover the best (set of) candidate\nhypernyms for input concepts or entities, given the search space of a\npre-defined vocabulary. We introduce a neural network architecture for the\nconcerned task and empirically study various neural network models to build the\nrepresentations in latent space for words and phrases. The evaluated models\ninclude convolutional neural network, long-short term memory network, gated\nrecurrent unit and recurrent convolutional neural network. We also explore\ndifferent embedding methods, including word embedding and sense embedding for\nbetter performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10564v1",
            "date": "2018-05-27 01:58:39+00:00",
            "title": "Generating Fine-Grained Open Vocabulary Entity Type Descriptions",
            "authors": [
                "Rajarshi Bhowmik",
                "Gerard de Melo"
            ],
            "abstract": "While large-scale knowledge graphs provide vast amounts of structured facts\nabout entities, a short textual description can often be useful to succinctly\ncharacterize an entity and its type. Unfortunately, many knowledge graph\nentities lack such textual descriptions. In this paper, we introduce a dynamic\nmemory-based network that generates a short open vocabulary description of an\nentity by jointly leveraging induced fact embeddings as well as the dynamic\ncontext of the generated sequence of words. We demonstrate the ability of our\narchitecture to discern relevant information for more accurate generation of\ntype description by pitting the system against several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10586v1",
            "date": "2018-05-27 06:25:11+00:00",
            "title": "Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings",
            "authors": [
                "Dat Quoc Nguyen",
                "Karin Verspoor"
            ],
            "abstract": "We investigate the incorporation of character-based word representations into\na standard CNN-based relation extraction model. We experiment with two common\nneural architectures, CNN and LSTM, to learn word vector representations from\ncharacter embeddings. Through a task on the BioCreative-V CDR corpus,\nextracting relationships between chemicals and diseases, we show that models\nexploiting the character-based word representations improve on models that do\nnot use this information, obtaining state-of-the-art result relative to\nprevious neural approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10824v1",
            "date": "2018-05-28 09:02:18+00:00",
            "title": "UG18 at SemEval-2018 Task 1: Generating Additional Training Data for Predicting Emotion Intensity in Spanish",
            "authors": [
                "Marloes Kuijper",
                "Mike van Lenthe",
                "Rik van Noord"
            ],
            "abstract": "The present study describes our submission to SemEval 2018 Task 1: Affect in\nTweets. Our Spanish-only approach aimed to demonstrate that it is beneficial to\nautomatically generate additional training data by (i) translating training\ndata from other languages and (ii) applying a semi-supervised learning method.\nWe find strong support for both approaches, with those models outperforming our\nregular models in all subtasks. However, creating a stepwise ensemble of\ndifferent models as opposed to simply averaging did not result in an increase\nin performance. We placed second (EI-Reg), second (EI-Oc), fourth (V-Reg) and\nfifth (V-Oc) in the four Spanish subtasks we participated in.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10850v1",
            "date": "2018-05-28 10:19:38+00:00",
            "title": "Inducing Grammars with and for Neural Machine Translation",
            "authors": [
                "Ke Tran",
                "Yonatan Bisk"
            ],
            "abstract": "Machine translation systems require semantic knowledge and grammatical\nunderstanding. Neural machine translation (NMT) systems often assume this\ninformation is captured by an attention mechanism and a decoder that ensures\nfluency. Recent work has shown that incorporating explicit syntax alleviates\nthe burden of modeling both types of knowledge. However, requiring parses is\nexpensive and does not explore the question of what syntax a model needs during\ntranslation. To address both of these issues we introduce a model that\nsimultaneously translates while inducing dependency trees. In this way, we\nleverage the benefits of structure while investigating what syntax NMT must\ninduce to maximize performance. We show that our dependency trees are 1.\nlanguage pair dependent and 2. improve translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10959v1",
            "date": "2018-05-28 14:56:35+00:00",
            "title": "Denoising Distant Supervision for Relation Extraction via Instance-Level Adversarial Training",
            "authors": [
                "Xu Han",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Existing neural relation extraction (NRE) models rely on distant supervision\nand suffer from wrong labeling problems. In this paper, we propose a novel\nadversarial training mechanism over instances for relation extraction to\nalleviate the noise issue. As compared with previous denoising methods, our\nproposed method can better discriminate those informative instances from noisy\nones. Our method is also efficient and flexible to be applied to various NRE\narchitectures. As shown in the experiments on a large-scale benchmark dataset\nin relation extraction, our denoising method can effectively filter out noisy\ninstances and achieve significant improvements as compared with the\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10985v1",
            "date": "2018-05-28 15:48:39+00:00",
            "title": "Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization",
            "authors": [
                "Kian Kenyon-Dean",
                "Jackie Chi Kit Cheung",
                "Doina Precup"
            ],
            "abstract": "We present an approach to event coreference resolution by developing a\ngeneral framework for clustering that uses supervised representation learning.\nWe propose a neural network architecture with novel Clustering-Oriented\nRegularization (CORE) terms in the objective function. These terms encourage\nthe model to create embeddings of event mentions that are amenable to\nclustering. We then use agglomerative clustering on these embeddings to build\nevent coreference chains. For both within- and cross-document coreference on\nthe ECB+ corpus, our model obtains better results than models that require\nsignificantly more pre-annotated information. This work provides insight and\nmotivating results for a new general approach to solving coreference and\nclustering problems with representation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11140v1",
            "date": "2018-05-28 19:21:59+00:00",
            "title": "Core Conflictual Relationship: Text Mining to Discover What and When",
            "authors": [
                "Fionn Murtagh",
                "Giuseppe Iurato"
            ],
            "abstract": "Following detailed presentation of the Core Conflictual Relationship Theme\n(CCRT), there is the objective of relevant methods for what has been described\nas verbalization and visualization of data. Such is also termed data mining and\ntext mining, and knowledge discovery in data. The Correspondence Analysis\nmethodology, also termed Geometric Data Analysis, is shown in a case study to\nbe comprehensive and revealing. Computational efficiency depends on how the\nanalysis process is structured. For both illustrative and revealing aspects of\nthe case study here, relatively extensive dream reports are used. This\nGeometric Data Analysis confirms the validity of CCRT method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11189v1",
            "date": "2018-05-28 22:09:49+00:00",
            "title": "Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models",
            "authors": [
                "Satoru Katsumata",
                "Yukio Matsumura",
                "Hayahide Yamagishi",
                "Mamoru Komachi"
            ],
            "abstract": "Encoder-decoder models typically only employ words that are frequently used\nin the training corpus to reduce the computational costs and exclude noise.\nHowever, this vocabulary set may still include words that interfere with\nlearning in encoder-decoder models. This paper proposes a method for selecting\nmore suitable words for learning encoders by utilizing not only frequency, but\nalso co-occurrence information, which we capture using the HITS algorithm. We\napply our proposed method to two tasks: machine translation and grammatical\nerror correction. For Japanese-to-English translation, this method achieves a\nBLEU score that is 0.56 points more than that of a baseline. It also\noutperforms the baseline method for English grammatical error correction, with\nan F0.5-measure that is 1.48 points higher.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11213v2",
            "date": "2018-05-29 01:45:22+00:00",
            "title": "Bi-Directional Neural Machine Translation with Synthetic Parallel Data",
            "authors": [
                "Xing Niu",
                "Michael Denkowski",
                "Marine Carpuat"
            ],
            "abstract": "Despite impressive progress in high-resource settings, Neural Machine\nTranslation (NMT) still struggles in low-resource and out-of-domain scenarios,\noften failing to match the quality of phrase-based translation. We propose a\nnovel technique that combines back-translation and multilingual NMT to improve\nperformance in these difficult cases. Our technique trains a single model for\nboth directions of a language pair, allowing us to back-translate source or\ntarget monolingual data without requiring an auxiliary model. We then continue\ntraining on the augmented parallel data, enabling a cycle of improvement for a\nsingle model that can incorporate any source, target, or parallel data to\nimprove both translation directions. As a byproduct, these models can reduce\ntraining and deployment costs significantly compared to uni-directional models.\nExtensive experiments show that our technique outperforms standard\nback-translation in low-resource scenarios, improves quality on cross-domain\ntasks, and effectively reduces costs across the board.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11224v1",
            "date": "2018-05-29 02:39:43+00:00",
            "title": "Distilling Knowledge for Search-based Structured Prediction",
            "authors": [
                "Yijia Liu",
                "Wanxiang Che",
                "Huaipeng Zhao",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Many natural language processing tasks can be modeled into structured\nprediction and solved as a search problem. In this paper, we distill an\nensemble of multiple models trained with different initialization into a single\nmodel. In addition to learning to match the ensemble's probability output on\nthe reference states, we also use the ensemble to explore the search space and\nlearn from the encountered states in the exploration. Experimental results on\ntwo typical search-based structured prediction tasks -- transition-based\ndependency parsing and neural machine translation show that distillation can\neffectively improve the single model's performance and the final model achieves\nimprovements of 1.32 in LAS and 2.65 in BLEU score on these two tasks\nrespectively over strong baselines and it outperforms the greedy structured\nprediction models in previous literatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11267v1",
            "date": "2018-05-29 06:52:29+00:00",
            "title": "Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?",
            "authors": [
                "Peter Jansen"
            ],
            "abstract": "Question Answering for complex questions is often modeled as a graph\nconstruction or traversal task, where a solver must build or traverse a graph\nof facts that answer and explain a given question. This \"multi-hop\" inference\nhas been shown to be extremely challenging, with few models able to aggregate\nmore than two facts before being overwhelmed by \"semantic drift\", or the\ntendency for long chains of facts to quickly drift off topic. This is a major\nbarrier to current inference models, as even elementary science questions\nrequire an average of 4 to 6 facts to answer and explain. In this work we\nempirically characterize the difficulty of building or traversing a graph of\nsentences connected by lexical overlap, by evaluating chance sentence\naggregation quality through 9,784 manually-annotated judgments across knowledge\ngraphs built from three free-text corpora (including study guides and Simple\nWikipedia). We demonstrate semantic drift tends to be high and aggregation\nquality low, at between 0.04% and 3%, and highlight scenarios that maximize the\nlikelihood of meaningfully combining information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11295v2",
            "date": "2018-05-29 08:22:50+00:00",
            "title": "Unsupervised detection of diachronic word sense evolution",
            "authors": [
                "Jean-Fran\u00e7ois Delpech"
            ],
            "abstract": "Most words have several senses and connotations which evolve in time due to\nsemantic shift, so that closely related words may gain different or even\nopposite meanings over the years. This evolution is very relevant to the study\nof language and of cultural changes, but the tools currently available for\ndiachronic semantic analysis have significant, inherent limitations and are not\nsuitable for real-time analysis. In this article, we demonstrate how the\nlinearity of random vectors techniques enables building time series of\ncongruent word embeddings (or semantic spaces) which can then be compared and\ncombined linearly without loss of precision over any time period to detect\ndiachronic semantic shifts. We show how this approach yields time trajectories\nof polysemous words such as amazon or apple, enables following semantic drifts\nand gender bias across time, reveals the shifting instantiations of stable\nconcepts such as hurricane or president. This very fast, linear approach can\neasily be distributed over many processors to follow in real time streams of\nsocial media such as Twitter or Facebook; the resulting, time-dependent\nsemantic spaces can then be combined at will by simple additions or\nsubtractions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11351v1",
            "date": "2018-05-29 10:46:30+00:00",
            "title": "Quantum-inspired Complex Word Embedding",
            "authors": [
                "Qiuchi Li",
                "Sagar Uprety",
                "Benyou Wang",
                "Dawei Song"
            ],
            "abstract": "A challenging task for word embeddings is to capture the emergent meaning or\npolarity of a combination of individual words. For example, existing approaches\nin word embeddings will assign high probabilities to the words \"Penguin\" and\n\"Fly\" if they frequently co-occur, but it fails to capture the fact that they\noccur in an opposite sense - Penguins do not fly. We hypothesize that humans do\nnot associate a single polarity or sentiment to each word. The word contributes\nto the overall polarity of a combination of words depending upon which other\nwords it is combined with. This is analogous to the behavior of microscopic\nparticles which exist in all possible states at the same time and interfere\nwith each other to give rise to new states depending upon their relative\nphases. We make use of the Hilbert Space representation of such particles in\nQuantum Mechanics where we subscribe a relative phase to each word, which is a\ncomplex number, and investigate two such quantum inspired models to derive the\nmeaning of a combination of words. The proposed models achieve better\nperformances than state-of-the-art non-quantum models on the binary sentence\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11360v2",
            "date": "2018-05-29 11:29:56+00:00",
            "title": "Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information",
            "authors": [
                "Seonhoon Kim",
                "Inho Kang",
                "Nojun Kwak"
            ],
            "abstract": "Sentence matching is widely used in various natural language tasks such as\nnatural language inference, paraphrase identification, and question answering.\nFor these tasks, understanding logical and semantic relationship between two\nsentences is required but it is yet challenging. Although attention mechanism\nis useful to capture the semantic relationship and to properly align the\nelements of two sentences, previous methods of attention mechanism simply use a\nsummation operation which does not retain original features enough. Inspired by\nDenseNet, a densely connected convolutional network, we propose a\ndensely-connected co-attentive recurrent neural network, each layer of which\nuses concatenated information of attentive features as well as hidden features\nof all the preceding recurrent layers. It enables preserving the original and\nthe co-attentive feature information from the bottommost word embedding layer\nto the uppermost recurrent layer. To alleviate the problem of an\never-increasing size of feature vectors due to dense concatenation operations,\nwe also propose to use an autoencoder after dense concatenation. We evaluate\nour proposed architecture on highly competitive benchmark datasets related to\nsentence matching. Experimental results show that our architecture, which\nretains recurrent and attentive features, achieves state-of-the-art\nperformances for most of the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11461v1",
            "date": "2018-05-28 16:17:27+00:00",
            "title": "Syntactic Dependency Representations in Neural Relation Classification",
            "authors": [
                "Farhad Nooralahzadeh",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "We investigate the use of different syntactic dependency representations in a\nneural relation classification task and compare the CoNLL, Stanford Basic and\nUniversal Dependencies schemes. We further compare with a syntax-agnostic\napproach and perform an error analysis in order to gain a better understanding\nof the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11462v1",
            "date": "2018-05-28 07:58:46+00:00",
            "title": "OpenNMT: Neural Machine Translation Toolkit",
            "authors": [
                "Guillaume Klein",
                "Yoon Kim",
                "Yuntian Deng",
                "Vincent Nguyen",
                "Jean Senellart",
                "Alexander M. Rush"
            ],
            "abstract": "OpenNMT is an open-source toolkit for neural machine translation (NMT). The\nsystem prioritizes efficiency, modularity, and extensibility with the goal of\nsupporting NMT research into model architectures, feature representations, and\nsource modalities, while maintaining competitive performance and reasonable\ntraining requirements. The toolkit consists of modeling and translation\nsupport, as well as detailed pedagogical documentation about the underlying\ntechniques. OpenNMT has been used in several production MT systems, modified\nfor numerous research papers, and is implemented across several deep learning\nframeworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11465v1",
            "date": "2018-05-29 13:44:19+00:00",
            "title": "AMR Dependency Parsing with a Typed Semantic Algebra",
            "authors": [
                "Jonas Groschwitz",
                "Matthias Lindemann",
                "Meaghan Fowlie",
                "Mark Johnson",
                "Alexander Koller"
            ],
            "abstract": "We present a semantic parser for Abstract Meaning Representations which\nlearns to parse strings into tree representations of the compositional\nstructure of an AMR graph. This allows us to use standard neural techniques for\nsupertagging and dependency tree parsing, constrained by a linguistically\nprincipled type system. We present two approximative decoding algorithms, which\nachieve state-of-the-art accuracy and outperform strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11467v1",
            "date": "2018-05-29 13:46:27+00:00",
            "title": "Entity Linking in 40 Languages using MAG",
            "authors": [
                "Diego Moussallem",
                "Ricardo Usbeck",
                "Michael R\u00f6der",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "A plethora of Entity Linking (EL) approaches has recently been developed.\nWhile many claim to be multilingual, the MAG (Multilingual AGDISTIS) approach\nhas been shown recently to outperform the state of the art in multilingual EL\non 7 languages. With this demo, we extend MAG to support EL in 40 different\nlanguages, including especially low-resources languages such as Ukrainian,\nGreek, Hungarian, Croatian, Portuguese, Japanese and Korean. Our demo relies on\nonline web services which allow for an easy access to our entity linking\napproaches and can disambiguate against DBpedia and Wikidata. During the demo,\nwe will show how to use MAG by means of POST requests as well as using its\nuser-friendly web interface. All data used in the demo is available at\nhttps://hobbitdata.informatik.uni-leipzig.de/agdistis/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11474v3",
            "date": "2018-05-29 13:53:16+00:00",
            "title": "Human vs Automatic Metrics: on the Importance of Correlation Design",
            "authors": [
                "Anastasia Shimorina"
            ],
            "abstract": "This paper discusses two existing approaches to the correlation analysis\nbetween automatic evaluation metrics and human scores in the area of natural\nlanguage generation. Our experiments show that depending on the usage of a\nsystem- or sentence-level correlation analysis, correlation results between\nautomatic scores and human judgments are inconsistent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11545v1",
            "date": "2018-05-29 15:49:11+00:00",
            "title": "Lightly-supervised Representation Learning with Global Interpretability",
            "authors": [
                "Marco A. Valenzuela-Esc\u00e1rcega",
                "Ajay Nagesh",
                "Mihai Surdeanu"
            ],
            "abstract": "We propose a lightly-supervised approach for information extraction, in\nparticular named entity classification, which combines the benefits of\ntraditional bootstrapping, i.e., use of limited annotations and\ninterpretability of extraction patterns, with the robust learning approaches\nproposed in representation learning. Our algorithm iteratively learns custom\nembeddings for both the multi-word entities to be extracted and the patterns\nthat match them from a few example entities per category. We demonstrate that\nthis representation-based approach outperforms three other state-of-the-art\nbootstrapping approaches on two datasets: CoNLL-2003 and OntoNotes.\nAdditionally, using these embeddings, our approach outputs a\nglobally-interpretable model consisting of a decision list, by ranking patterns\nbased on their proximity to the average entity embedding in a given class. We\nshow that this interpretable model performs close to our complete bootstrapping\nmodel, proving that representation learning can be used to produce\ninterpretable models with small loss in performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11564v1",
            "date": "2018-05-29 16:22:42+00:00",
            "title": "Entrainment profiles: Comparison by gender, role, and feature set",
            "authors": [
                "Uwe D. Reichel",
                "\u0160tefan Be\u0148u\u0161",
                "Katalin M\u00e1dy"
            ],
            "abstract": "We examine prosodic entrainment in cooperative game dialogs for new feature\nsets describing register, pitch accent shape, and rhythmic aspects of\nutterances. For these as well as for established features we present\nentrainment profiles to detect within- and across-dialog entrainment by the\nspeakers' gender and role in the game. It turned out, that feature sets undergo\nentrainment in different quantitative and qualitative ways, which can partly be\nattributed to their different functions. Furthermore, interactions between\nspeaker gender and role (describer vs. follower) suggest gender-dependent\nstrategies in cooperative solution-oriented interactions: female describers\nentrain most, male describers least. Our data suggests a slight advantage of\nthe latter strategy on task success.",
            "journal ref": "U.D. Reichel, \\v{S}. Be\\v{n}u\\v{s}, K. M\\'ady. Entrainment\n  profiles: Comparison by gender, role, and feature set. Speech Communication,\n  100:46-57, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11598v1",
            "date": "2018-05-29 17:29:55+00:00",
            "title": "Polyglot Semantic Role Labeling",
            "authors": [
                "Phoebe Mulcaire",
                "Swabha Swayamdipta",
                "Noah Smith"
            ],
            "abstract": "Previous approaches to multilingual semantic dependency parsing treat\nlanguages independently, without exploiting the similarities between semantic\nstructures across languages. We experiment with a new approach where we combine\nresources from a pair of languages in the CoNLL 2009 shared task to build a\npolyglot semantic role labeler. Notwithstanding the absence of parallel data,\nand the dissimilarity in annotations between languages, our approach results in\nan improvement in SRL performance on multiple languages over a monolingual\nbaseline. Analysis of the polyglot model shows it to be advantageous in\nlower-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11603v1",
            "date": "2018-05-29 17:41:52+00:00",
            "title": "Automatic Identification of Arabic expressions related to future events in Lebanon's economy",
            "authors": [
                "Moustafa Al-Hajj",
                "Amani Sabra"
            ],
            "abstract": "In this paper, we propose a method to automatically identify future events in\nLebanon's economy from Arabic texts. Challenges are threefold: first, we need\nto build a corpus of Arabic texts that covers Lebanon's economy; second, we\nneed to study how future events are expressed linguistically in these texts;\nand third, we need to automatically identify the relevant textual segments\naccordingly. We will validate this method on a constructed corpus form the web\nand show that it has very promising results. To do so, we will be using SLCSAS,\na system for semantic analysis, based on the Contextual Explorer method, and\n\"AlKhalil Morpho Sys\" system for morpho-syntactic analysis.",
            "journal ref": "International Journal of Science and Research (IJSR),\n  https://www.ijsr.net/archive/v7i4/ART20182042.pdf, Volume 7 Issue 4, April\n  2018, 1656 - 1660",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11611v1",
            "date": "2018-05-29 17:54:52+00:00",
            "title": "Semantically-informed distance and similarity measures for paraphrase plagiarism identification",
            "authors": [
                "Miguel A. \u00c1lvarez-Carmona",
                "Marc Franco-Salvador",
                "Esa\u00fa Villatoro-Tello",
                "Manuel Montes-y-G\u00f3mez",
                "Paolo Rosso",
                "Luis Villase\u00f1or-Pineda"
            ],
            "abstract": "Paraphrase plagiarism identification represents a very complex task given\nthat plagiarized texts are intentionally modified through several rewording\ntechniques. Accordingly, this paper introduces two new measures for evaluating\nthe relatedness of two given texts: a semantically-informed similarity measure\nand a semantically-informed edit distance. Both measures are able to extract\nsemantic information from either an external resource or a distributed\nrepresentation of words, resulting in informative features for training a\nsupervised classifier for detecting paraphrase plagiarism. Obtained results\nindicate that the proposed metrics are consistently good in detecting different\ntypes of paraphrase plagiarism. In addition, results are very competitive\nagainst state-of-the art methods having the advantage of representing a much\nmore simple but equally effective solution.",
            "journal ref": "Journal of Intelligent & Fuzzy Systems, vol. 34, no. 5, pp.\n  2983-2990, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11653v2",
            "date": "2018-05-29 18:44:31+00:00",
            "title": "LSTMs Exploit Linguistic Attributes of Data",
            "authors": [
                "Nelson F. Liu",
                "Omer Levy",
                "Roy Schwartz",
                "Chenhao Tan",
                "Noah A. Smith"
            ],
            "abstract": "While recurrent neural networks have found success in a variety of natural\nlanguage processing applications, they are general models of sequential data.\nWe investigate how the properties of natural language data affect an LSTM's\nability to learn a nonlinguistic task: recalling elements from its input. We\nfind that models trained on natural language data are able to recall tokens\nfrom much longer sequences than models trained on non-language sequential data.\nFurthermore, we show that the LSTM learns to solve the memorization task by\nexplicitly using a subset of its neurons to count timesteps in the input. We\nhypothesize that the patterns and structure in natural language data enable\nLSTMs to learn by providing approximate ways of reducing loss, but\nunderstanding the effect of different training data on the learnability of\nLSTMs remains an open question.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11749v3",
            "date": "2018-05-30 00:02:59+00:00",
            "title": "Unsupervised Text Style Transfer using Language Models as Discriminators",
            "authors": [
                "Zichao Yang",
                "Zhiting Hu",
                "Chris Dyer",
                "Eric P. Xing",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Binary classifiers are often employed as discriminators in GAN-based\nunsupervised style transfer systems to ensure that transferred sentences are\nsimilar to sentences in the target domain. One difficulty with this approach is\nthat the error signal provided by the discriminator can be unstable and is\nsometimes insufficient to train the generator to produce fluent language. In\nthis paper, we propose a new technique that uses a target domain language model\nas the discriminator, providing richer and more stable token-level feedback\nduring the learning process. We train the generator to minimize the negative\nlog likelihood (NLL) of generated sentences, evaluated by the language model.\nBy using a continuous approximation of discrete sampling under the generator,\nour model can be trained using back-propagation in an end- to-end fashion.\nMoreover, our empirical results show that when using a language model as a\nstructured discriminator, it is possible to forgo adversarial steps during\ntraining, making the process more stable. We compare our model with previous\nwork using convolutional neural networks (CNNs) as discriminators and show that\nour approach leads to improved performance on three tasks: word substitution\ndecipherment, sentiment modification, and related language translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11762v1",
            "date": "2018-05-30 00:48:44+00:00",
            "title": "Adversarial Learning of Task-Oriented Neural Dialog Models",
            "authors": [
                "Bing Liu",
                "Ian Lane"
            ],
            "abstract": "In this work, we propose an adversarial learning method for reward estimation\nin reinforcement learning (RL) based task-oriented dialog models. Most of the\ncurrent RL based task-oriented dialog systems require the access to a reward\nsignal from either user feedback or user ratings. Such user ratings, however,\nmay not always be consistent or available in practice. Furthermore, online\ndialog policy learning with RL typically requires a large number of queries to\nusers, suffering from sample efficiency problem. To address these challenges,\nwe propose an adversarial learning method to learn dialog rewards directly from\ndialog samples. Such rewards are further used to optimize the dialog policy\nwith policy gradient based RL. In the evaluation in a restaurant search domain,\nwe show that the proposed adversarial dialog learning method achieves advanced\ndialog success rate comparing to strong baseline methods. We further discuss\nthe covariate shift problem in online adversarial dialog learning and show how\nwe can address that with partial access to user feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11774v1",
            "date": "2018-05-30 02:04:30+00:00",
            "title": "Planning, Inference and Pragmatics in Sequential Language Games",
            "authors": [
                "Fereshte Khani",
                "Noah D. Goodman",
                "Percy Liang"
            ],
            "abstract": "We study sequential language games in which two players, each with private\ninformation, communicate to achieve a common goal. In such games, a successful\nplayer must (i) infer the partner's private information from the partner's\nmessages, (ii) generate messages that are most likely to help with the goal,\nand (iii) reason pragmatically about the partner's strategy. We propose a model\nthat captures all three characteristics and demonstrate their importance in\ncapturing human behavior on a new goal-oriented dataset we collected using\ncrowdsourcing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11824v1",
            "date": "2018-05-30 06:49:15+00:00",
            "title": "Anaphora and Coreference Resolution: A Review",
            "authors": [
                "Rhea Sukthanker",
                "Soujanya Poria",
                "Erik Cambria",
                "Ramkumar Thirunavukarasu"
            ],
            "abstract": "Entity resolution aims at resolving repeated references to an entity in a\ndocument and forms a core component of natural language processing (NLP)\nresearch. This field possesses immense potential to improve the performance of\nother NLP fields like machine translation, sentiment analysis, paraphrase\ndetection, summarization, etc. The area of entity resolution in NLP has seen\nproliferation of research in two separate sub-areas namely: anaphora resolution\nand coreference resolution. Through this review article, we aim at clarifying\nthe scope of these two tasks in entity resolution. We also carry out a detailed\nanalysis of the datasets, evaluation metrics and research methods that have\nbeen adopted to tackle this NLP problem. This survey is motivated with the aim\nof providing the reader with a clear understanding of what constitutes this NLP\nproblem and the issues that require attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11868v1",
            "date": "2018-05-30 09:03:50+00:00",
            "title": "An English-Hindi Code-Mixed Corpus: Stance Annotation and Baseline System",
            "authors": [
                "Sahil Swami",
                "Ankush Khandelwal",
                "Vinay Singh",
                "Syed Sarfaraz Akhtar",
                "Manish Shrivastava"
            ],
            "abstract": "Social media has become one of the main channels for peo- ple to communicate\nand share their views with the society. We can often detect from these views\nwhether the person is in favor, against or neu- tral towards a given topic.\nThese opinions from social media are very useful for various companies. We\npresent a new dataset that consists of 3545 English-Hindi code-mixed tweets\nwith opinion towards Demoneti- sation that was implemented in India in 2016\nwhich was followed by a large countrywide debate. We present a baseline\nsupervised classification system for stance detection developed using the same\ndataset that uses various machine learning techniques to achieve an accuracy of\n58.7% on 10-fold cross validation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11869v1",
            "date": "2018-05-30 09:08:54+00:00",
            "title": "A Corpus of English-Hindi Code-Mixed Tweets for Sarcasm Detection",
            "authors": [
                "Sahil Swami",
                "Ankush Khandelwal",
                "Vinay Singh",
                "Syed Sarfaraz Akhtar",
                "Manish Shrivastava"
            ],
            "abstract": "Social media platforms like twitter and facebook have be- come two of the\nlargest mediums used by people to express their views to- wards different\ntopics. Generation of such large user data has made NLP tasks like sentiment\nanalysis and opinion mining much more important. Using sarcasm in texts on\nsocial media has become a popular trend lately. Using sarcasm reverses the\nmeaning and polarity of what is implied by the text which poses challenge for\nmany NLP tasks. The task of sarcasm detection in text is gaining more and more\nimportance for both commer- cial and security services. We present the first\nEnglish-Hindi code-mixed dataset of tweets marked for presence of sarcasm and\nirony where each token is also annotated with a language tag. We present a\nbaseline su- pervised classification system developed using the same dataset\nwhich achieves an average F-score of 78.4 after using random forest classifier\nand performing 10-fold cross validation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11937v1",
            "date": "2018-05-30 13:22:02+00:00",
            "title": "Character-Level Models versus Morphology in Semantic Role Labeling",
            "authors": [
                "G\u00f6zde G\u00fcl \u015eahin",
                "Mark Steedman"
            ],
            "abstract": "Character-level models have become a popular approach specially for their\naccessibility and ability to handle unseen data. However, little is known on\ntheir ability to reveal the underlying morphological structure of a word, which\nis a crucial skill for high-level semantic analysis tasks, such as semantic\nrole labeling (SRL). In this work, we train various types of SRL models that\nuse word, character and morphology level information and analyze how\nperformance of characters compare to words and morphology for several\nlanguages. We conduct an in-depth error analysis for each morphological\ntypology and analyze the strengths and limitations of character-level models\nthat relate to out-of-domain data, training data size, long range dependencies\nand model complexity. Our exhaustive analyses shed light on important\ncharacteristics of character-level models and their semantic capability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12032v1",
            "date": "2018-05-30 15:20:14+00:00",
            "title": "Identifying and Understanding User Reactions to Deceptive and Trusted Social News Sources",
            "authors": [
                "Maria Glenski",
                "Tim Weninger",
                "Svitlana Volkova"
            ],
            "abstract": "In the age of social news, it is important to understand the types of\nreactions that are evoked from news sources with various levels of credibility.\nIn the present work we seek to better understand how users react to trusted and\ndeceptive news sources across two popular, and very different, social media\nplatforms. To that end, (1) we develop a model to classify user reactions into\none of nine types, such as answer, elaboration, and question, etc, and (2) we\nmeasure the speed and the type of reaction for trusted and deceptive news\nsources for 10.8M Twitter posts and 6.2M Reddit comments. We show that there\nare significant differences in the speed and the type of reactions between\ntrusted and deceptive news sources on Twitter, but far smaller differences on\nReddit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12045v1",
            "date": "2018-05-30 15:56:22+00:00",
            "title": "End-to-end named entity extraction from speech",
            "authors": [
                "Sahar Ghannay",
                "Antoine Caubri\u00e8re",
                "Yannick Est\u00e8ve",
                "Antoine Laurent",
                "Emmanuel Morin"
            ],
            "abstract": "Named entity recognition (NER) is among SLU tasks that usually extract\nsemantic information from textual documents. Until now, NER from speech is made\nthrough a pipeline process that consists in processing first an automatic\nspeech recognition (ASR) on the audio and then processing a NER on the ASR\noutputs. Such approach has some disadvantages (error propagation, metric to\ntune ASR systems sub-optimal in regards to the final task, reduced space search\nat the ASR output level...) and it is known that more integrated approaches\noutperform sequential ones, when they can be applied. In this paper, we present\na first study of end-to-end approach that directly extracts named entities from\nspeech, though a unique neural architecture. On a such way, a joint\noptimization is able for both ASR and NER. Experiments are carried on French\ndata easily accessible, composed of data distributed in several evaluation\ncampaign. Experimental results show that this end-to-end approach provides\nbetter results (F-measure=0.69 on test data) than a classical pipeline approach\nto detect named entity categories (F-measure=0.65).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12061v2",
            "date": "2018-05-30 16:29:32+00:00",
            "title": "Bilingual Character Representation for Efficiently Addressing Out-of-Vocabulary Words in Code-Switching Named Entity Recognition",
            "authors": [
                "Genta Indra Winata",
                "Chien-Sheng Wu",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "We propose an LSTM-based model with hierarchical architecture on named entity\nrecognition from code-switching Twitter data. Our model uses bilingual\ncharacter representation and transfer learning to address out-of-vocabulary\nwords. In order to mitigate data noise, we propose to use token replacement and\nnormalization. In the 3rd Workshop on Computational Approaches to Linguistic\nCode-Switching Shared Task, we achieved second place with 62.76% harmonic mean\nF1-score for English-Spanish language pair without using any gazetteer and\nknowledge-based information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12070v2",
            "date": "2018-05-30 16:37:43+00:00",
            "title": "Code-Switching Language Modeling using Syntax-Aware Multi-Task Learning",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "Lack of text data has been the major issue on code-switching language\nmodeling. In this paper, we introduce multi-task learning based language model\nwhich shares syntax representation of languages to leverage linguistic\ninformation and tackle the low resource data issue. Our model jointly learns\nboth language modeling and Part-of-Speech tagging on code-switched utterances.\nIn this way, the model is able to identify the location of code-switching\npoints and improves the prediction of next word. Our approach outperforms\nstandard LSTM based language model, with an improvement of 9.7% and 7.4% in\nperplexity on SEAME Phase I and Phase II dataset respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12096v1",
            "date": "2018-05-30 17:23:24+00:00",
            "title": "Marian: Cost-effective High-Quality Neural Machine Translation in C++",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Kenneth Heafield",
                "Hieu Hoang",
                "Roman Grundkiewicz",
                "Anthony Aue"
            ],
            "abstract": "This paper describes the submissions of the \"Marian\" team to the WNMT 2018\nshared task. We investigate combinations of teacher-student training,\nlow-precision matrix products, auto-tuning and other methods to optimize the\nTransformer model on GPU and CPU. By further integrating these methods with the\nnew averaging attention networks, a recently introduced faster Transformer\nvariant, we create a number of high-quality, high-performance models on the GPU\nand CPU, dominating the Pareto frontier for this shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12282v1",
            "date": "2018-05-31 01:33:19+00:00",
            "title": "On the Impact of Various Types of Noise on Neural Machine Translation",
            "authors": [
                "Huda Khayrallah",
                "Philipp Koehn"
            ],
            "abstract": "We examine how various types of noise in the parallel training data impact\nthe quality of neural machine translation systems. We create five types of\nartificial noise and analyze how they degrade performance in neural and\nstatistical machine translation. We find that neural models are generally more\nharmed by noise than statistical models. For one especially egregious type of\nnoise they learn to just copy the input sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12291v3",
            "date": "2018-05-31 02:21:39+00:00",
            "title": "Empirical Evaluation of Character-Based Model on Neural Named-Entity Recognition in Indonesian Conversational Texts",
            "authors": [
                "Kemal Kurniawan",
                "Samuel Louvan"
            ],
            "abstract": "Despite the long history of named-entity recognition (NER) task in the\nnatural language processing community, previous work rarely studied the task on\nconversational texts. Such texts are challenging because they contain a lot of\nword variations which increase the number of out-of-vocabulary (OOV) words. The\nhigh number of OOV words poses a difficulty for word-based neural models.\nMeanwhile, there is plenty of evidence to the effectiveness of character-based\nneural models in mitigating this OOV problem. We report an empirical evaluation\nof neural sequence labeling models with character embedding to tackle NER task\nin Indonesian conversational texts. Our experiments show that (1) character\nmodels outperform word embedding-only models by up to 4 $F_1$ points, (2)\ncharacter models perform better in OOV cases with an improvement of as high as\n15 $F_1$ points, and (3) character models are robust against a very high OOV\nrate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12307v1",
            "date": "2018-05-31 03:27:53+00:00",
            "title": "Attention-Based LSTM for Psychological Stress Detection from Spoken Language Using Distant Supervision",
            "authors": [
                "Genta Indra Winata",
                "Onno Pepijn Kampman",
                "Pascale Fung"
            ],
            "abstract": "We propose a Long Short-Term Memory (LSTM) with attention mechanism to\nclassify psychological stress from self-conducted interview transcriptions. We\napply distant supervision by automatically labeling tweets based on their\nhashtag content, which complements and expands the size of our corpus. This\nadditional data is used to initialize the model parameters, and which it is\nfine-tuned using the interview data. This improves the model's robustness,\nespecially by expanding the vocabulary size. The bidirectional LSTM model with\nattention is found to be the best model in terms of accuracy (74.1%) and\nf-score (74.3%). Furthermore, we show that distant supervision fine-tuning\nenhances the model's performance by 1.6% accuracy and 2.1% f-score. The\nattention mechanism helps the model to select informative words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12386v4",
            "date": "2018-05-31 09:11:16+00:00",
            "title": "SemEval 2019 Shared Task: Cross-lingual Semantic Parsing with UCCA - Call for Participation",
            "authors": [
                "Daniel Hershcovich",
                "Leshem Choshen",
                "Elior Sulem",
                "Zohar Aizenbud",
                "Ari Rappoport",
                "Omri Abend"
            ],
            "abstract": "We announce a shared task on UCCA parsing in English, German and French, and\ncall for participants to submit their systems. UCCA is a cross-linguistically\napplicable framework for semantic representation, which builds on extensive\ntypological work and supports rapid annotation. UCCA poses a challenge for\nexisting parsing techniques, as it exhibits reentrancy (resulting in DAG\nstructures), discontinuous structures and non-terminal nodes corresponding to\ncomplex semantic units. Given the success of recent semantic parsing shared\ntasks (on SDP and AMR), we expect the task to have a significant contribution\nto the advancement of UCCA parsing in particular, and semantic parsing in\ngeneral. Furthermore, existing applications for semantic evaluation that are\nbased on UCCA will greatly benefit from better automatic methods for UCCA\nparsing. The competition website is\nhttps://competitions.codalab.org/competitions/19160",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12471v3",
            "date": "2018-05-31 13:52:06+00:00",
            "title": "Neural Network Acceptability Judgments",
            "authors": [
                "Alex Warstadt",
                "Amanpreet Singh",
                "Samuel R. Bowman"
            ],
            "abstract": "This paper investigates the ability of artificial neural networks to judge\nthe grammatical acceptability of a sentence, with the goal of testing their\nlinguistic competence. We introduce the Corpus of Linguistic Acceptability\n(CoLA), a set of 10,657 English sentences labeled as grammatical or\nungrammatical from published linguistics literature. As baselines, we train\nseveral recurrent neural network models on acceptability classification, and\nfind that our models outperform unsupervised models by Lau et al (2016) on\nCoLA. Error-analysis on specific grammatical phenomena reveals that both Lau et\nal.'s models and ours learn systematic generalizations like subject-verb-object\norder. However, all models we test perform far below human level on a wide\nrange of grammatical constructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12501v2",
            "date": "2018-05-31 14:54:33+00:00",
            "title": "Multi-Label Transfer Learning for Multi-Relational Semantic Similarity",
            "authors": [
                "Li Zhang",
                "Steven R. Wilson",
                "Rada Mihalcea"
            ],
            "abstract": "Multi-relational semantic similarity datasets define the semantic relations\nbetween two short texts in multiple ways, e.g., similarity, relatedness, and so\non. Yet, all the systems to date designed to capture such relations target one\nrelation at a time. We propose a multi-label transfer learning approach based\non LSTM to make predictions for several relations simultaneously and aggregate\nthe losses to update the parameters. This multi-label regression approach\njointly learns the information provided by the multiple relations, rather than\ntreating them as separate tasks. Not only does this approach outperform the\nsingle-task approach and the traditional multi-task learning approach, but it\nalso achieves state-of-the-art performance on all but one relation of the Human\nActivity Phrase dataset.",
            "journal ref": "Proceedings of the Eighth Joint Conference on Lexical and\n  Computational Semantics (SEM 2019) (2019) 44-50",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12518v2",
            "date": "2018-05-31 15:29:32+00:00",
            "title": "Incremental Natural Language Processing: Challenges, Strategies, and Evaluation",
            "authors": [
                "Arne K\u00f6hn"
            ],
            "abstract": "Incrementality is ubiquitous in human-human interaction and beneficial for\nhuman-computer interaction. It has been a topic of research in different parts\nof the NLP community, mostly with focus on the specific topic at hand even\nthough incremental systems have to deal with similar challenges regardless of\ndomain. In this survey, I consolidate and categorize the approaches,\nidentifying similarities and differences in the computation and data, and show\ntrade-offs that have to be considered. A focus lies on evaluating incremental\nsystems because the standard metrics often fail to capture the incremental\nproperties of a system and coming up with a suitable evaluation scheme is\nnon-trivial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00044v3",
            "date": "2018-05-31 18:37:37+00:00",
            "title": "Text normalization using memory augmented neural networks",
            "authors": [
                "Subhojeet Pramanik",
                "Aman Hussain"
            ],
            "abstract": "We perform text normalization, i.e. the transformation of words from the\nwritten to the spoken form, using a memory augmented neural network. With the\naddition of dynamic memory access and storage mechanism, we present a neural\narchitecture that will serve as a language-agnostic text normalization system\nwhile avoiding the kind of unacceptable errors made by the LSTM-based recurrent\nneural networks. By successfully reducing the frequency of such mistakes, we\nshow that this novel architecture is indeed a better alternative. Our proposed\nsystem requires significantly lesser amounts of data, training time and compute\nresources. Additionally, we perform data up-sampling, circumventing the data\nsparsity problem in some semiotic classes, to show that sufficient examples in\nany particular class can improve the performance of our text normalization\nsystem. Although a few occurrences of these errors still remain in certain\nsemiotic classes, we demonstrate that memory augmented networks with\nmeta-learning capabilities can open many doors to a superior text normalization\nsystem.",
            "journal ref": "Speech Communication, Volume 109, 2019, Pages 15-23, ISSN\n  0167-6393",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00187v3",
            "date": "2018-06-01 04:33:16+00:00",
            "title": "Scaling Neural Machine Translation",
            "authors": [
                "Myle Ott",
                "Sergey Edunov",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "Sequence to sequence learning models still require several days to reach\nstate of the art performance on large benchmark datasets using a single\nmachine. This paper shows that reduced precision and large batch training can\nspeedup training by nearly 5x on a single 8-GPU machine with careful tuning and\nimplementation. On WMT'14 English-German translation, we match the accuracy of\nVaswani et al. (2017) in under 5 hours when training on 8 GPUs and we obtain a\nnew state of the art of 29.3 BLEU after training for 85 minutes on 128 GPUs. We\nfurther improve these results to 29.8 BLEU by training on the much larger\nParacrawl dataset. On the WMT'14 English-French task, we obtain a\nstate-of-the-art BLEU of 43.2 in 8.5 hours on 128 GPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00522v1",
            "date": "2018-05-30 22:27:15+00:00",
            "title": "Improving Dialogue Act Classification for Spontaneous Arabic Speech and Instant Messages at Utterance Level",
            "authors": [
                "AbdelRahim Elmadany",
                "Sherif Abdou",
                "Mervat Gheith"
            ],
            "abstract": "The ability to model and automatically detect dialogue act is an important\nstep toward understanding spontaneous speech and Instant Messages. However, it\nhas been difficult to infer a dialogue act from a surface utterance because it\nhighly depends on the context of the utterance and speaker linguistic\nknowledge; especially in Arabic dialects. This paper proposes a statistical\ndialogue analysis model to recognize utterance's dialogue acts using a\nmulti-classes hierarchical structure. The model can automatically acquire\nprobabilistic discourse knowledge from a dialogue corpus were collected and\nannotated manually from multi-genre Egyptian call-centers. Extensive\nexperiments were conducted using Support Vector Machines classifier to evaluate\nthe system performance. The results attained in the term of average F-measure\nscores of 0.912; showed that the proposed approach has moderately improved\nF-measure by approximately 20%.",
            "journal ref": "11th edition of the Language Resources and Evaluation Conference,\n  7-12 May 2018, Miyazaki (Japan)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00591v1",
            "date": "2018-06-02 06:33:47+00:00",
            "title": "Does the brain represent words? An evaluation of brain decoding studies of language understanding",
            "authors": [
                "Jon Gauthier",
                "Anna Ivanova"
            ],
            "abstract": "Language decoding studies have identified word representations which can be\nused to predict brain activity in response to novel words and sentences\n(Anderson et al., 2016; Pereira et al., 2018). The unspoken assumption of these\nstudies is that, during processing, linguistic information is transformed into\nsome shared semantic space, and those semantic representations are then used\nfor a variety of linguistic and non-linguistic tasks. We claim that current\nstudies vastly underdetermine the content of these representations, the\nalgorithms which the brain deploys to produce and consume them, and the\ncomputational tasks which they are designed to solve. We illustrate this\nindeterminacy with an extension of the sentence-decoding experiment of Pereira\net al. (2018), showing how standard evaluations fail to distinguish between\nlanguage processing models which deploy different mechanisms and which are\noptimized to solve very different tasks. We conclude by suggesting changes to\nthe brain decoding paradigm which can support stronger claims of neural\nrepresentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00616v1",
            "date": "2018-06-02 10:07:10+00:00",
            "title": "AP18-OLR Challenge: Three Tasks and Their Baselines",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Qing Chen"
            ],
            "abstract": "The third oriental language recognition (OLR) challenge AP18-OLR is\nintroduced in this paper, including the data profile, the tasks and the\nevaluation principles. Following the events in the last two years, namely\nAP16-OLR and AP17-OLR, the challenge this year focuses on more challenging\ntasks, including (1) short-duration utterances, (2) confusing languages, and\n(3) open-set recognition. The same as the previous events, the data of AP18-OLR\nis also provided by SpeechOcean and the NSFC M2ASR project. Baselines based on\nboth the i-vector model and neural networks are constructed for the\nparticipants' reference. We report the baseline results on the three tasks and\ndemonstrate that the three tasks are truly challenging. All the data is free\nfor participants, and the Kaldi recipes for the baselines have been published\nonline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00674v1",
            "date": "2018-06-02 17:18:06+00:00",
            "title": "Emotion Detection in Text: a Review",
            "authors": [
                "Armin Seyeditabari",
                "Narges Tabari",
                "Wlodek Zadrozny"
            ],
            "abstract": "In recent years, emotion detection in text has become more popular due to its\nvast potential applications in marketing, political science, psychology,\nhuman-computer interaction, artificial intelligence, etc. Access to a huge\namount of textual data, especially opinionated and self-expression text also\nplayed a special role to bring attention to this field. In this paper, we\nreview the work that has been done in identifying emotion expressions in text\nand argue that although many techniques, methodologies, and models have been\ncreated to detect emotion in text, there are various reasons that make these\nmethods insufficient. Although, there is an essential need to improve the\ndesign and architecture of current systems, factors such as the complexity of\nhuman emotions, and the use of implicit and metaphorical language in expressing\nit, lead us to think that just re-purposing standard methodologies will not be\nenough to capture these complexities, and it is important to pay attention to\nthe linguistic intricacies of emotion expression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00692v3",
            "date": "2018-06-02 19:14:39+00:00",
            "title": "Stress Test Evaluation for Natural Language Inference",
            "authors": [
                "Aakanksha Naik",
                "Abhilasha Ravichander",
                "Norman Sadeh",
                "Carolyn Rose",
                "Graham Neubig"
            ],
            "abstract": "Natural language inference (NLI) is the task of determining if a natural\nlanguage hypothesis can be inferred from a given premise in a justifiable\nmanner. NLI was proposed as a benchmark task for natural language\nunderstanding. Existing models perform well at standard datasets for NLI,\nachieving impressive results across different genres of text. However, the\nextent to which these models understand the semantic content of sentences is\nunclear. In this work, we propose an evaluation methodology consisting of\nautomatically constructed \"stress tests\" that allow us to examine whether\nsystems have the ability to make real inferential decisions. Our evaluation of\nsix sentence-encoder models on these stress tests reveals strengths and\nweaknesses of these models with respect to challenging linguistic phenomena,\nand suggests important directions for future work in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00699v3",
            "date": "2018-06-02 20:14:17+00:00",
            "title": "Quantifying the dynamics of topical fluctuations in language",
            "authors": [
                "Andres Karjus",
                "Richard A. Blythe",
                "Simon Kirby",
                "Kenny Smith"
            ],
            "abstract": "The availability of large diachronic corpora has provided the impetus for a\ngrowing body of quantitative research on language evolution and meaning change.\nThe central quantities in this research are token frequencies of linguistic\nelements in texts, with changes in frequency taken to reflect the popularity or\nselective fitness of an element. However, corpus frequencies may change for a\nwide variety of reasons, including purely random sampling effects, or because\ncorpora are composed of contemporary media and fiction texts within which the\nunderlying topics ebb and flow with cultural and socio-political trends. In\nthis work, we introduce a simple model for controlling for topical fluctuations\nin corpora - the topical-cultural advection model - and demonstrate how it\nprovides a robust baseline of variability in word frequency changes over time.\nWe validate the model on a diachronic corpus spanning two centuries, and a\ncarefully-controlled artificial language change scenario, and then use it to\ncorrect for topical fluctuations in historical time series. Finally, we use the\nmodel to show that the emergence of new words typically corresponds with the\nrise of a trending topic. This suggests that some lexical innovations occur due\nto growing communicative need in a subspace of the lexicon, and that the\ntopical-cultural advection model can be used to quantify this.",
            "journal ref": "Language Dynamics and Change, 10(1) 2020, 86-125",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00722v2",
            "date": "2018-06-03 01:29:27+00:00",
            "title": "Dense Information Flow for Neural Machine Translation",
            "authors": [
                "Yanyao Shen",
                "Xu Tan",
                "Di He",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Recently, neural machine translation has achieved remarkable progress by\nintroducing well-designed deep neural networks into its encoder-decoder\nframework. From the optimization perspective, residual connections are adopted\nto improve learning performance for both encoder and decoder in most of these\ndeep architectures, and advanced attention connections are applied as well.\nInspired by the success of the DenseNet model in computer vision problems, in\nthis paper, we propose a densely connected NMT architecture (DenseNMT) that is\nable to train more efficiently for NMT. The proposed DenseNMT not only allows\ndense connection in creating new features for both encoder and decoder, but\nalso uses the dense attention structure to improve attention quality. Our\nexperiments on multiple datasets show that DenseNMT structure is more\ncompetitive and efficient.",
            "journal ref": "in Proceedings of NAACL-HLT 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00780v1",
            "date": "2018-06-03 12:36:06+00:00",
            "title": "Building Advanced Dialogue Managers for Goal-Oriented Dialogue Systems",
            "authors": [
                "Vladimir Ilievski"
            ],
            "abstract": "Goal-Oriented (GO) Dialogue Systems, colloquially known as goal oriented\nchatbots, help users achieve a predefined goal (e.g. book a movie ticket)\nwithin a closed domain. A first step is to understand the user's goal by using\nnatural language understanding techniques. Once the goal is known, the bot must\nmanage a dialogue to achieve that goal, which is conducted with respect to a\nlearnt policy. The success of the dialogue system depends on the quality of the\npolicy, which is in turn reliant on the availability of high-quality training\ndata for the policy learning method, for instance Deep Reinforcement Learning.\n  Due to the domain specificity, the amount of available data is typically too\nlow to allow the training of good dialogue policies. In this master thesis we\nintroduce a transfer learning method to mitigate the effects of the low\nin-domain data availability. Our transfer learning based approach improves the\nbot's success rate by $20\\%$ in relative terms for distant domains and we more\nthan double it for close domains, compared to the model without transfer\nlearning. Moreover, the transfer learning chatbots learn the policy up to 5 to\n10 times faster. Finally, as the transfer learning approach is complementary to\nadditional processing such as warm-starting, we show that their joint\napplication gives the best outcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00840v1",
            "date": "2018-06-03 17:34:41+00:00",
            "title": "Latent Tree Learning with Differentiable Parsers: Shift-Reduce Parsing and Chart Parsing",
            "authors": [
                "Jean Maillard",
                "Stephen Clark"
            ],
            "abstract": "Latent tree learning models represent sentences by composing their words\naccording to an induced parse tree, all based on a downstream task. These\nmodels often outperform baselines which use (externally provided) syntax trees\nto drive the composition order. This work contributes (a) a new latent tree\nlearning model based on shift-reduce parsing, with competitive downstream\nperformance and non-trivial induced trees, and (b) an analysis of the trees\nlearned by our shift-reduce model and by a chart-based model.",
            "journal ref": "Proceedings of the Workshop on the Relevance of Linguistic\n  Structure in Neural Architectures for NLP, ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00910v1",
            "date": "2018-06-04 01:07:37+00:00",
            "title": "An unsupervised and customizable misspelling generator for mining noisy health-related text sources",
            "authors": [
                "Abeed Sarker",
                "Graciela Gonzalez-Hernandez"
            ],
            "abstract": "In this paper, we present a customizable datacentric system that\nautomatically generates common misspellings for complex health-related terms.\nThe spelling variant generator relies on a dense vector model learned from\nlarge unlabeled text, which is used to find semantically close terms to the\noriginal/seed keyword, followed by the filtering of terms that are lexically\ndissimilar beyond a given threshold. The process is executed recursively,\nconverging when no new terms similar (lexically and semantically) to the seed\nkeyword are found. Weighting of intra-word character sequence similarities\nallows further problem-specific customization of the system. On a dataset\nprepared for this study, our system outperforms the current state-of-the-art\nfor medication name variant generation with best F1-score of 0.69 and\nF1/4-score of 0.78. Extrinsic evaluation of the system on a set of\ncancer-related terms showed an increase of over 67% in retrieval rate from\nTwitter posts when the generated variants are included. Our proposed spelling\nvariant generator has several advantages over the current state-of-the-art and\nother types of variant generators-(i) it is capable of filtering out lexically\nsimilar but semantically dissimilar terms, (ii) the number of variants\ngenerated is low as many low-frequency and ambiguous misspellings are filtered\nout, and (iii) the system is fully automatic, customizable and easily\nexecutable. While the base system is fully unsupervised, we show how\nsupervision maybe employed to adjust weights for task-specific customization.\nThe performance and significant relative simplicity of our proposed approach\nmakes it a much needed misspelling generation resource for health-related text\nmining from noisy sources. The source code for the system has been made\npublicly available for research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00913v1",
            "date": "2018-06-04 01:20:59+00:00",
            "title": "Self-Normalization Properties of Language Modeling",
            "authors": [
                "Jacob Goldberger",
                "Oren Melamud"
            ],
            "abstract": "Self-normalizing discriminative models approximate the normalized probability\nof a class without having to compute the partition function. In the context of\nlanguage modeling, this property is particularly appealing as it may\nsignificantly reduce run-times due to large word vocabularies. In this study,\nwe provide a comprehensive investigation of language modeling\nself-normalization. First, we theoretically analyze the inherent\nself-normalization properties of Noise Contrastive Estimation (NCE) language\nmodels. Then, we compare them empirically to softmax-based approaches, which\nare self-normalized using explicit regularization, and suggest a hybrid model\nwith compelling properties. Finally, we uncover a surprising negative\ncorrelation between self-normalization and perplexity across the board, as well\nas some regularity in the observed errors, which may potentially be used for\nimproving self-normalization algorithms in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00920v3",
            "date": "2018-06-04 01:50:21+00:00",
            "title": "DRCD: a Chinese Machine Reading Comprehension Dataset",
            "authors": [
                "Chih Chieh Shao",
                "Trois Liu",
                "Yuting Lai",
                "Yiying Tseng",
                "Sam Tsai"
            ],
            "abstract": "In this paper, we introduce DRCD (Delta Reading Comprehension Dataset), an\nopen domain traditional Chinese machine reading comprehension (MRC) dataset.\nThis dataset aimed to be a standard Chinese machine reading comprehension\ndataset, which can be a source dataset in transfer learning. The dataset\ncontains 10,014 paragraphs from 2,108 Wikipedia articles and 30,000+ questions\ngenerated by annotators. We build a baseline model that achieves an F1 score of\n89.59%. F1 score of Human performance is 93.30%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00971v2",
            "date": "2018-06-04 06:26:19+00:00",
            "title": "Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis",
            "authors": [
                "Shuhei Kurita",
                "Daisuke Kawahara",
                "Sadao Kurohashi"
            ],
            "abstract": "Japanese predicate-argument structure (PAS) analysis involves zero anaphora\nresolution, which is notoriously difficult. To improve the performance of\nJapanese PAS analysis, it is straightforward to increase the size of corpora\nannotated with PAS. However, since it is prohibitively expensive, it is\npromising to take advantage of a large amount of raw corpora. In this paper, we\npropose a novel Japanese PAS analysis model based on semi-supervised\nadversarial training with a raw corpus. In our experiments, our model\noutperforms existing state-of-the-art models for Japanese PAS analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01045v1",
            "date": "2018-06-04 11:03:11+00:00",
            "title": "Topic Modelling of Empirical Text Corpora: Validity, Reliability, and Reproducibility in Comparison to Semantic Maps",
            "authors": [
                "Tobias Hecking",
                "Loet Leydesdorff"
            ],
            "abstract": "Using the 6,638 case descriptions of societal impact submitted for evaluation\nin the Research Excellence Framework (REF 2014), we replicate the topic model\n(Latent Dirichlet Allocation or LDA) made in this context and compare the\nresults with factor-analytic results using a traditional word-document matrix\n(Principal Component Analysis or PCA). Removing a small fraction of documents\nfrom the sample, for example, has on average a much larger impact on LDA than\non PCA-based models to the extent that the largest distortion in the case of\nPCA has less effect than the smallest distortion of LDA-based models. In terms\nof semantic coherence, however, LDA models outperform PCA-based models. The\ntopic models inform us about the statistical properties of the document sets\nunder study, but the results are statistical and should not be used for a\nsemantic interpretation - for example, in grant selections and micro-decision\nmaking, or scholarly work-without follow-up using domain-specific semantic\nmaps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01170v1",
            "date": "2018-06-04 16:10:19+00:00",
            "title": "Efficient Online Scalar Annotation with Bounded Support",
            "authors": [
                "Keisuke Sakaguchi",
                "Benjamin Van Durme"
            ],
            "abstract": "We describe a novel method for efficiently eliciting scalar annotations for\ndataset construction and system quality estimation by human judgments. We\ncontrast direct assessment (annotators assign scores to items directly), online\npairwise ranking aggregation (scores derive from annotator comparison of\nitems), and a hybrid approach (EASL: Efficient Annotation of Scalar Labels)\nproposed here. Our proposal leads to increased correlation with ground truth,\nat far greater annotator efficiency, suggesting this strategy as an improved\nmechanism for dataset creation and manual system evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01185v1",
            "date": "2018-06-04 16:30:03+00:00",
            "title": "History Playground: A Tool for Discovering Temporal Trends in Massive Textual Corpora",
            "authors": [
                "Thomas Lansdall-Welfare",
                "Nello Cristianini"
            ],
            "abstract": "Recent studies have shown that macroscopic patterns of continuity and change\nover the course of centuries can be detected through the analysis of time\nseries extracted from massive textual corpora. Similar data-driven approaches\nhave already revolutionised the natural sciences, and are widely believed to\nhold similar potential for the humanities and social sciences, driven by the\nmass-digitisation projects that are currently under way, and coupled with the\never-increasing number of documents which are \"born digital\". As such, new\ninteractive tools are required to discover and extract macroscopic patterns\nfrom these vast quantities of textual data. Here we present History Playground,\nan interactive web-based tool for discovering trends in massive textual\ncorpora. The tool makes use of scalable algorithms to first extract trends from\ntextual corpora, before making them available for real-time search and\ndiscovery, presenting users with an interface to explore the data. Included in\nthe tool are algorithms for standardization, regression, change-point detection\nin the relative frequencies of ngrams, multi-term indices and comparison of\ntrends across different corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01483v2",
            "date": "2018-06-05 03:50:50+00:00",
            "title": "JTAV: Jointly Learning Social Media Content Representation by Fusing Textual, Acoustic, and Visual Features",
            "authors": [
                "Hongru Liang",
                "Haozheng Wang",
                "Jun Wang",
                "Shaodi You",
                "Zhe Sun",
                "Jin-Mao Wei",
                "Zhenglu Yang"
            ],
            "abstract": "Learning social media content is the basis of many real-world applications,\nincluding information retrieval and recommendation systems, among others. In\ncontrast with previous works that focus mainly on single modal or bi-modal\nlearning, we propose to learn social media content by fusing jointly textual,\nacoustic, and visual information (JTAV). Effective strategies are proposed to\nextract fine-grained features of each modality, that is, attBiGRU and DCRNN. We\nalso introduce cross-modal fusion and attentive pooling techniques to integrate\nmulti-modal information comprehensively. Extensive experimental evaluation\nconducted on real-world datasets demonstrates our proposed model outperforms\nthe state-of-the-art approaches by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01501v1",
            "date": "2018-06-05 05:34:15+00:00",
            "title": "Information Aggregation via Dynamic Routing for Sequence Encoding",
            "authors": [
                "Jingjing Gong",
                "Xipeng Qiu",
                "Shaojing Wang",
                "Xuanjing Huang"
            ],
            "abstract": "While much progress has been made in how to encode a text sequence into a\nsequence of vectors, less attention has been paid to how to aggregate these\npreceding vectors (outputs of RNN/CNN) into fixed-size encoding vector.\nUsually, a simple max or average pooling is used, which is a bottom-up and\npassive way of aggregation and lack of guidance by task information. In this\npaper, we propose an aggregation mechanism to obtain a fixed-size encoding with\na dynamic routing policy. The dynamic routing policy is dynamically deciding\nthat what and how much information need be transferred from each word to the\nfinal encoding of the text sequence. Following the work of Capsule Network, we\ndesign two dynamic routing policies to aggregate the outputs of RNN/CNN\nencoding layer into a final encoding vector. Compared to the other aggregation\nmethods, dynamic routing can refine the messages according to the state of\nfinal encoding vector. Experimental results on five text classification tasks\nshow that our method outperforms other aggregating models by a significant\nmargin. Related source code is released on our github page.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01515v2",
            "date": "2018-06-05 06:45:23+00:00",
            "title": "How Do Source-side Monolingual Word Embeddings Impact Neural Machine Translation?",
            "authors": [
                "Shuoyang Ding",
                "Kevin Duh"
            ],
            "abstract": "Using pre-trained word embeddings as input layer is a common practice in many\nnatural language processing (NLP) tasks, but it is largely neglected for neural\nmachine translation (NMT). In this paper, we conducted a systematic analysis on\nthe effect of using pre-trained source-side monolingual word embedding in NMT.\nWe compared several strategies, such as fixing or updating the embeddings\nduring NMT training on varying amounts of data, and we also proposed a novel\nstrategy called dual-embedding that blends the fixing and updating strategies.\nOur results suggest that pre-trained embeddings can be helpful if properly\nincorporated into NMT, especially when parallel data is limited or additional\nin-domain monolingual data is readily available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01523v1",
            "date": "2018-06-05 07:34:02+00:00",
            "title": "Multi-Task Active Learning for Neural Semantic Role Labeling on Low Resource Conversational Corpus",
            "authors": [
                "Fariz Ikhwantri",
                "Samuel Louvan",
                "Kemal Kurniawan",
                "Bagas Abisena",
                "Valdi Rachman",
                "Alfan Farizki Wicaksono",
                "Rahmad Mahendra"
            ],
            "abstract": "Most Semantic Role Labeling (SRL) approaches are supervised methods which\nrequire a significant amount of annotated corpus, and the annotation requires\nlinguistic expertise. In this paper, we propose a Multi-Task Active Learning\nframework for Semantic Role Labeling with Entity Recognition (ER) as the\nauxiliary task to alleviate the need for extensive data and use additional\ninformation from ER to help SRL. We evaluate our approach on Indonesian\nconversational dataset. Our experiments show that multi-task active learning\ncan outperform single-task active learning method and standard multi-task\nlearning. According to our results, active learning is more efficient by using\n12% less of training data compared to passive learning in both single-task and\nmulti-task setting. We also introduce a new dataset for SRL in Indonesian\nconversational domain to encourage further research in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01620v1",
            "date": "2018-06-05 12:02:11+00:00",
            "title": "Explaining Away Syntactic Structure in Semantic Document Representations",
            "authors": [
                "Erik Holmer",
                "Andreas Marfurt"
            ],
            "abstract": "Most generative document models act on bag-of-words input in an attempt to\nfocus on the semantic content and thereby partially forego syntactic\ninformation. We argue that it is preferable to keep the original word order\nintact and explicitly account for the syntactic structure instead. We propose\nan extension to the Neural Variational Document Model (Miao et al., 2016) that\ndoes exactly that to separate local (syntactic) context from the global\n(semantic) representation of the document. Our model builds on the variational\nautoencoder framework to define a generative document model based on next-word\nprediction. We name our approach Sequence-Aware Variational Autoencoder since\nin contrast to its predecessor, it operates on the true input sequence. In a\nseries of experiments we observe stronger topicality of the learned\nrepresentations as well as increased robustness to syntactic noise in our\ntraining data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01694v1",
            "date": "2018-06-05 13:59:32+00:00",
            "title": "Understanding Meanings in Multilingual Customer Feedback",
            "authors": [
                "Chao-Hong Liu",
                "Declan Groves",
                "Akira Hayakawa",
                "Alberto Poncelas",
                "Qun Liu"
            ],
            "abstract": "Understanding and being able to react to customer feedback is the most\nfundamental task in providing good customer service. However, there are two\nmajor obstacles for international companies to automatically detect the meaning\nof customer feedback in a global multilingual environment. Firstly, there is no\nwidely acknowledged categorisation (classes) of meaning for customer feedback.\nSecondly, the applicability of one meaning categorisation, if it exists, to\ncustomer feedback in multiple languages is questionable. In this paper, we\nextracted representative real world samples of customer feedback from Microsoft\nOffice customers in multiple languages, English, Spanish and Japanese,and\nconcluded a five-class categorisation(comment, request, bug, complaint and\nmeaningless) for meaning classification that could be used across languages in\nthe realm of customer feedback analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01733v2",
            "date": "2018-06-05 15:02:13+00:00",
            "title": "Luminoso at SemEval-2018 Task 10: Distinguishing Attributes Using Text Corpora and Relational Knowledge",
            "authors": [
                "Robyn Speer",
                "Joanna Lowry-Duda"
            ],
            "abstract": "Luminoso participated in the SemEval 2018 task on \"Capturing Discriminative\nAttributes\" with a system based on ConceptNet, an open knowledge graph focused\non general knowledge. In this paper, we describe how we trained a linear\nclassifier on a small number of semantically-informed features to achieve an\n$F_1$ score of 0.7368 on the task, close to the task's high score of 0.75.",
            "journal ref": "Proceedings of The 12th International Workshop on Semantic\n  Evaluation (2018), p. 985-989",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01773v1",
            "date": "2018-06-05 16:15:23+00:00",
            "title": "Contextual Slot Carryover for Disparate Schemas",
            "authors": [
                "Chetan Naik",
                "Arpit Gupta",
                "Hancheng Ge",
                "Lambert Mathias",
                "Ruhi Sarikaya"
            ],
            "abstract": "In the slot-filling paradigm, where a user can refer back to slots in the\ncontext during a conversation, the goal of the contextual understanding system\nis to resolve the referring expressions to the appropriate slots in the\ncontext. In large-scale multi-domain systems, this presents two challenges -\nscaling to a very large and potentially unbounded set of slot values, and\ndealing with diverse schemas. We present a neural network architecture that\naddresses the slot value scalability challenge by reformulating the contextual\ninterpretation as a decision to carryover a slot from a set of possible\ncandidates. To deal with heterogenous schemas, we introduce a simple\ndata-driven method for trans- forming the candidate slots. Our experiments show\nthat our approach can scale to multiple domains and provides competitive\nresults over a strong baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02179v2",
            "date": "2018-06-06 13:38:57+00:00",
            "title": "Open Domain Suggestion Mining: Problem Definition and Datasets",
            "authors": [
                "Sapna Negi",
                "Maarten de Rijke",
                "Paul Buitelaar"
            ],
            "abstract": "We propose a formal definition for the task of suggestion mining in the\ncontext of a wide range of open domain applications. Human perception of the\nterm \\emph{suggestion} is subjective and this effects the preparation of hand\nlabeled datasets for the task of suggestion mining. Existing work either lacks\na formal problem definition and annotation procedure, or provides domain and\napplication specific definitions. Moreover, many previously used manually\nlabeled datasets remain proprietary. We first present an annotation study, and\nbased on our observations propose a formal task definition and annotation\nprocedure for creating benchmark datasets for suggestion mining. With this\nstudy, we also provide publicly available labeled datasets for suggestion\nmining in multiple domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02253v1",
            "date": "2018-06-06 15:42:22+00:00",
            "title": "The Limitations of Cross-language Word Embeddings Evaluation",
            "authors": [
                "Amir Bakarov",
                "Roman Suvorov",
                "Ilya Sochenkov"
            ],
            "abstract": "The aim of this work is to explore the possible limitations of existing\nmethods of cross-language word embeddings evaluation, addressing the lack of\ncorrelation between intrinsic and extrinsic cross-language evaluation methods.\nTo prove this hypothesis, we construct English-Russian datasets for extrinsic\nand intrinsic evaluation tasks and compare performances of 5 different\ncross-language models on them. The results say that the scores even on\ndifferent intrinsic benchmarks do not correlate to each other. We can conclude\nthat the use of human references as ground truth for cross-language word\nembeddings is not proper unless one does not understand how do native speakers\nprocess semantics in their cognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02418v1",
            "date": "2018-06-06 20:47:47+00:00",
            "title": "Finding Convincing Arguments Using Scalable Bayesian Preference Learning",
            "authors": [
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "We introduce a scalable Bayesian preference learning method for identifying\nconvincing arguments in the absence of gold-standard rat- ings or rankings. In\ncontrast to previous work, we avoid the need for separate methods to perform\nquality control on training data, predict rankings and perform pairwise\nclassification. Bayesian approaches are an effective solution when faced with\nsparse or noisy training data, but have not previously been used to identify\nconvincing arguments. One issue is scalability, which we address by developing\na stochastic variational inference method for Gaussian process (GP) preference\nlearning. We show how our method can be applied to predict argument\nconvincingness from crowdsourced data, outperforming the previous\nstate-of-the-art, particularly when trained with small amounts of unreliable\ndata. We demonstrate how the Bayesian approach enables more effective active\nlearning, thereby reducing the amount of data required to identify convincing\narguments for new users and domains. While word embeddings are principally used\nwith neural networks, our results show that word embeddings in combination with\nlinguistic features also benefit GPs when predicting argument convincingness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02437v1",
            "date": "2018-06-06 22:00:32+00:00",
            "title": "Studying the Difference Between Natural and Programming Language Corpora",
            "authors": [
                "Casey Casalnuovo",
                "Kenji Sagae",
                "Prem Devanbu"
            ],
            "abstract": "Code corpora, as observed in large software systems, are now known to be far\nmore repetitive and predictable than natural language corpora. But why? Does\nthe difference simply arise from the syntactic limitations of programming\nlanguages? Or does it arise from the differences in authoring decisions made by\nthe writers of these natural and programming language texts? We conjecture that\nthe differences are not entirely due to syntax, but also from the fact that\nreading and writing code is un-natural for humans, and requires substantial\nmental effort; so, people prefer to write code in ways that are familiar to\nboth reader and writer. To support this argument, we present results from two\nsets of studies: 1) a first set aimed at attenuating the effects of syntax, and\n2) a second, aimed at measuring repetitiveness of text written in other\nsettings (e.g. second language, technical/specialized jargon), which are also\neffortful to write. We find find that this repetition in source code is not\nentirely the result of grammar constraints, and thus some repetition must\nresult from human choice. While the evidence we find of similar repetitive\nbehavior in technical and learner corpora does not conclusively show that such\nlanguage is used by humans to mitigate difficulty, it is consistent with that\ntheory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02525v2",
            "date": "2018-06-07 06:11:34+00:00",
            "title": "Multi-Source Neural Machine Translation with Missing Data",
            "authors": [
                "Yuta Nishimura",
                "Katsuhito Sudoh",
                "Graham Neubig",
                "Satoshi Nakamura"
            ],
            "abstract": "Multi-source translation is an approach to exploit multiple inputs (e.g. in\ntwo different languages) to increase translation accuracy. In this paper, we\nexamine approaches for multi-source neural machine translation (NMT) using an\nincomplete multilingual corpus in which some translations are missing. In\npractice, many multilingual corpora are not complete due to the difficulty to\nprovide translations in all of the relevant languages (for example, in TED\ntalks, most English talks only have subtitles for a small portion of the\nlanguages that TED supports). Existing studies on multi-source translation did\nnot explicitly handle such situations. This study focuses on the use of\nincomplete multilingual corpora in multi-encoder NMT and mixture of NMT experts\nand examines a very simple implementation where missing source translations are\nreplaced by a special symbol <NULL>. These methods allow us to use incomplete\ncorpora both at training time and test time. In experiments with real\nincomplete multilingual corpora of TED Talks, the multi-source NMT with the\n<NULL> tokens achieved higher translation accuracies measured by BLEU than\nthose by any one-to-one NMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02725v2",
            "date": "2018-06-07 15:16:02+00:00",
            "title": "A Challenge Set for French --> English Machine Translation",
            "authors": [
                "Pierre Isabelle",
                "Roland Kuhn"
            ],
            "abstract": "We present a challenge set for French --> English machine translation based\non the approach introduced in Isabelle, Cherry and Foster (EMNLP 2017). Such\nchallenge sets are made up of sentences that are expected to be relatively\ndifficult for machines to translate correctly because their most\nstraightforward translations tend to be linguistically divergent. We present\nhere a set of 506 manually constructed French sentences, 307 of which are\ntargeted to the same kinds of structural divergences as in the paper mentioned\nabove. The remaining 199 sentences are designed to test the ability of the\nsystems to correctly translate difficult grammatical words such as\nprepositions. We report on the results of using this challenge set for testing\ntwo different systems, namely Google Translate and DEEPL, each on two different\ndates (October 2017 and January 2018). All the resulting data are made publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02786v1",
            "date": "2018-06-07 17:02:54+00:00",
            "title": "Domain Adversarial Training for Accented Speech Recognition",
            "authors": [
                "Sining Sun",
                "Ching-Feng Yeh",
                "Mei-Yuh Hwang",
                "Mari Ostendorf",
                "Lei Xie"
            ],
            "abstract": "In this paper, we propose a domain adversarial training (DAT) algorithm to\nalleviate the accented speech recognition problem. In order to reduce the\nmismatch between labeled source domain data (\"standard\" accent) and unlabeled\ntarget domain data (with heavy accents), we augment the learning objective for\na Kaldi TDNN network with a domain adversarial training (DAT) objective to\nencourage the model to learn accent-invariant features. In experiments with\nthree Mandarin accents, we show that DAT yields up to 7.45% relative character\nerror rate reduction when we do not have transcriptions of the accented speech,\ncompared with the baseline trained on standard accent data only. We also find a\nbenefit from DAT when used in combination with training from automatic\ntranscriptions on the accented data. Furthermore, we find that DAT is superior\nto multi-task learning for accented speech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02875v1",
            "date": "2018-06-07 19:28:35+00:00",
            "title": "An Exploration of Unreliable News Classification in Brazil and The U.S",
            "authors": [
                "Mauricio Gruppi",
                "Benjamin D. Horne",
                "Sibel Adali"
            ],
            "abstract": "The propagation of unreliable information is on the rise in many places\naround the world. This expansion is facilitated by the rapid spread of\ninformation and anonymity granted by the Internet. The spread of unreliable\ninformation is a wellstudied issue and it is associated with negative social\nimpacts. In a previous work, we have identified significant differences in the\nstructure of news articles from reliable and unreliable sources in the US\nmedia. Our goal in this work was to explore such differences in the Brazilian\nmedia. We found significant features in two data sets: one with Brazilian news\nin Portuguese and another one with US news in English. Our results show that\nfeatures related to the writing style were prominent in both data sets and,\ndespite the language difference, some features have a universal behavior, being\nsignificant to both US and Brazilian news articles. Finally, we combined both\ndata sets and used the universal features to build a machine learning\nclassifier to predict the source type of a news article as reliable or\nunreliable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02923v1",
            "date": "2018-06-07 23:21:51+00:00",
            "title": "Multimodal Relational Tensor Network for Sentiment and Emotion Classification",
            "authors": [
                "Saurav Sahay",
                "Shachi H Kumar",
                "Rui Xia",
                "Jonathan Huang",
                "Lama Nachman"
            ],
            "abstract": "Understanding Affect from video segments has brought researchers from the\nlanguage, audio and video domains together. Most of the current multimodal\nresearch in this area deals with various techniques to fuse the modalities, and\nmostly treat the segments of a video independently. Motivated by the work of\n(Zadeh et al., 2017) and (Poria et al., 2017), we present our architecture,\nRelational Tensor Network, where we use the inter-modal interactions within a\nsegment (intra-segment) and also consider the sequence of segments in a video\nto model the inter-segment inter-modal interactions. We also generate rich\nrepresentations of text and audio modalities by leveraging richer audio and\nlinguistic context alongwith fusing fine-grained knowledge based polarity\nscores from text. We present the results of our model on CMU-MOSEI dataset and\nshow that our model outperforms many baselines and state of the art methods for\nsentiment classification and emotion recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02940v3",
            "date": "2018-06-08 01:41:20+00:00",
            "title": "Findings of the Second Workshop on Neural Machine Translation and Generation",
            "authors": [
                "Alexandra Birch",
                "Andrew Finch",
                "Minh-Thang Luong",
                "Graham Neubig",
                "Yusuke Oda"
            ],
            "abstract": "This document describes the findings of the Second Workshop on Neural Machine\nTranslation and Generation, held in concert with the annual conference of the\nAssociation for Computational Linguistics (ACL 2018). First, we summarize the\nresearch trends of papers presented in the proceedings, and note that there is\nparticular interest in linguistic structure, domain adaptation, data\naugmentation, handling inadequate resources, and analysis of models. Second, we\ndescribe the results of the workshop's shared task on efficient neural machine\ntranslation, where participants were tasked with creating MT systems that are\nboth accurate and efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03191v1",
            "date": "2018-06-08 14:34:29+00:00",
            "title": "Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora",
            "authors": [
                "Stephen Roller",
                "Douwe Kiela",
                "Maximilian Nickel"
            ],
            "abstract": "Methods for unsupervised hypernym detection may broadly be categorized\naccording to two paradigms: pattern-based and distributional methods. In this\npaper, we study the performance of both approaches on several hypernymy tasks\nand find that simple pattern-based methods consistently outperform\ndistributional methods on common benchmark datasets. Our results show that\npattern-based models provide important contextual constraints which are not yet\ncaptured in distributional methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03223v1",
            "date": "2018-06-08 15:38:04+00:00",
            "title": "ChangeMyView Through Concessions: Do Concessions Increase Persuasion?",
            "authors": [
                "Elena Musi",
                "Debanjan Ghosh",
                "Smaranda Muresan"
            ],
            "abstract": "In discourse studies concessions are considered among those argumentative\nstrategies that increase persuasion. We aim to empirically test this hypothesis\nby calculating the distribution of argumentative concessions in persuasive vs.\nnon-persuasive comments from the ChangeMyView subreddit. This constitutes a\nchallenging task since concessions are not always part of an argument. Drawing\nfrom a theoretically-informed typology of concessions, we conduct an annotation\ntask to label a set of polysemous lexical markers as introducing an\nargumentative concession or not and we observe their distribution in threads\nthat achieved and did not achieve persuasion. For the annotation, we used both\nexpert and novice annotators. With the ultimate goal of conducting the study on\nlarge datasets, we present a self-training method to automatically identify\nargumentative concessions using linguistically motivated features. We achieve a\nmoderate F1 of 57.4% on the development set and 46.0% on the test set via the\nself-training method. These results are comparable to state of the art results\non similar tasks of identifying explicit discourse connective types from the\nPenn Discourse Treebank. Our findings from the manual labeling and the\nclassification experiments indicate that the type of argumentative concessions\nwe investigated is almost equally likely to be used in winning and losing\narguments from the ChangeMyView dataset. While this result seems to contradict\ntheoretical assumptions, we provide some reasons for this discrepancy related\nto the ChangeMyView subreddit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03280v1",
            "date": "2018-06-08 17:18:26+00:00",
            "title": "Multilingual Neural Machine Translation with Task-Specific Attention",
            "authors": [
                "Graeme Blackwood",
                "Miguel Ballesteros",
                "Todd Ward"
            ],
            "abstract": "Multilingual machine translation addresses the task of translating between\nmultiple source and target languages. We propose task-specific attention\nmodels, a simple but effective technique for improving the quality of\nsequence-to-sequence neural multilingual translation. Our approach seeks to\nretain as much of the parameter sharing generalization of NMT models as\npossible, while still allowing for language-specific specialization of the\nattention model to a particular language-pair or task. Our experiments on four\nlanguages of the Europarl corpus show that using a target-specific model of\nattention provides consistent gains in translation quality for all possible\ntranslation directions, compared to a model in which all parameters are shared.\nWe observe improved translation quality even in the (extreme) low-resource\nzero-shot translation directions for which the model never saw explicitly\npaired parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03290v1",
            "date": "2018-06-08 17:45:13+00:00",
            "title": "Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing",
            "authors": [
                "Daniel Fried",
                "Dan Klein"
            ],
            "abstract": "Dynamic oracles provide strong supervision for training constituency parsers\nwith exploration, but must be custom defined for a given parser's transition\nsystem. We explore using a policy gradient method as a parser-agnostic\nalternative. In addition to directly optimizing for a tree-level metric such as\nF1, policy gradient has the potential to reduce exposure bias by allowing\nexploration during training; moreover, it does not require a dynamic oracle for\nsupervision. On four constituency parsers in three languages, the method\nsubstantially outperforms static oracle likelihood training in almost all\nsettings. For parsers where a dynamic oracle is available (including a novel\noracle which we define for the transition system of Dyer et al. 2016), policy\ngradient typically recaptures a substantial fraction of the performance gain\nafforded by the dynamic oracle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03369v1",
            "date": "2018-06-08 22:47:10+00:00",
            "title": "#SarcasmDetection is soooo general! Towards a Domain-Independent Approach for Detecting Sarcasm",
            "authors": [
                "Natalie Parde",
                "Rodney D. Nielsen"
            ],
            "abstract": "Automatic sarcasm detection methods have traditionally been designed for\nmaximum performance on a specific domain. This poses challenges for those\nwishing to transfer those approaches to other existing or novel domains, which\nmay be typified by very different language characteristics. We develop a\ngeneral set of features and evaluate it under different training scenarios\nutilizing in-domain and/or out-of-domain training data. The best-performing\nscenario, training on both while employing a domain adaptation step, achieves\nan F1 of 0.780, which is well above baseline F1-measures of 0.515 and 0.345. We\nalso show that the approach outperforms the best results from prior work on the\nsame target domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03431v1",
            "date": "2018-06-09 07:40:38+00:00",
            "title": "Word Familiarity and Frequency",
            "authors": [
                "Kumiko Tanaka-Ishii",
                "Hiroshi Terada"
            ],
            "abstract": "Word frequency is assumed to correlate with word familiarity, but the\nstrength of this correlation has not been thoroughly investigated. In this\npaper, we report on our analysis of the correlation between a word familiarity\nrating list obtained through a psycholinguistic experiment and the\nlog-frequency obtained from various corpora of different kinds and sizes (up to\nthe terabyte scale) for English and Japanese. Major findings are threefold:\nFirst, for a given corpus, familiarity is necessary for a word to achieve high\nfrequency, but familiar words are not necessarily frequent. Second, correlation\nincreases with the corpus data size. Third, a corpus of spoken language\ncorrelates better than one of written language. These findings suggest that\ncognitive familiarity ratings are correlated to frequency, but more highly to\nthat of spoken rather than written language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03489v1",
            "date": "2018-06-09 15:40:09+00:00",
            "title": "Robust Lexical Features for Improved Neural Network Named-Entity Recognition",
            "authors": [
                "Abbas Ghaddar",
                "Philippe Langlais"
            ],
            "abstract": "Neural network approaches to Named-Entity Recognition reduce the need for\ncarefully hand-crafted features. While some features do remain in\nstate-of-the-art systems, lexical features have been mostly discarded, with the\nexception of gazetteers. In this work, we show that this is unfair: lexical\nfeatures are actually quite useful. We propose to embed words and entity types\ninto a low-dimensional vector space we train from annotated data produced by\ndistant supervision thanks to Wikipedia. From this, we compute - offline - a\nfeature vector representing each word. When used with a vanilla recurrent\nneural network model, this representation yields substantial improvements. We\nestablish a new state-of-the-art F1 score of 87.95 on ONTONOTES 5.0, while\nmatching state-of-the-art performance with a F1 score of 91.73 on the\nover-studied CONLL-2003 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03537v2",
            "date": "2018-06-09 20:23:27+00:00",
            "title": "Diachronic word embeddings and semantic shifts: a survey",
            "authors": [
                "Andrey Kutuzov",
                "Lilja \u00d8vrelid",
                "Terrence Szymanski",
                "Erik Velldal"
            ],
            "abstract": "Recent years have witnessed a surge of publications aimed at tracing temporal\nchanges in lexical semantics using distributional methods, particularly\nprediction-based word embedding models. However, this vein of research lacks\nthe cohesion, common terminology and shared practices of more established areas\nof natural language processing. In this paper, we survey the current state of\nacademic research related to diachronic word embeddings and semantic shifts\ndetection. We start with discussing the notion of semantic shifts, and then\ncontinue with an overview of the existing methods for tracing such time-related\nshifts with word embedding models. We propose several axes along which these\nmethods can be compared, and outline the main challenges before this emerging\nsubfield of NLP, as well as prospects and possible applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03578v1",
            "date": "2018-06-10 03:50:10+00:00",
            "title": "Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task",
            "authors": [
                "An Yang",
                "Kai Liu",
                "Jing Liu",
                "Yajuan Lyu",
                "Sujian Li"
            ],
            "abstract": "Current evaluation metrics to question answering based machine reading\ncomprehension (MRC) systems generally focus on the lexical overlap between the\ncandidate and reference answers, such as ROUGE and BLEU. However, bias may\nappear when these metrics are used for specific question types, especially\nquestions inquiring yes-no opinions and entity lists. In this paper, we make\nadaptations on the metrics to better correlate n-gram overlap with the human\njudgment for answers to these two question types. Statistical analysis proves\nthe effectiveness of our approach. Our adaptations may provide positive\nguidance for the development of real-scene MRC systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03590v1",
            "date": "2018-06-10 06:09:57+00:00",
            "title": "Cross-Lingual Task-Specific Representation Learning for Text Classification in Resource Poor Languages",
            "authors": [
                "Nurendra Choudhary",
                "Rajat Singh",
                "Manish Shrivastava"
            ],
            "abstract": "Neural network models have shown promising results for text classification.\nHowever, these solutions are limited by their dependence on the availability of\nannotated data.\n  The prospect of leveraging resource-rich languages to enhance the text\nclassification of resource-poor languages is fascinating. The performance on\nresource-poor languages can significantly improve if the resource availability\nconstraints can be offset. To this end, we present a twin Bidirectional Long\nShort Term Memory (Bi-LSTM) network with shared parameters consolidated by a\ncontrastive loss function (based on a similarity metric). The model learns the\nrepresentation of resource-poor and resource-rich sentences in a common space\nby using the similarity between their assigned annotation tags. Hence, the\nmodel projects sentences with similar tags closer and those with different tags\nfarther from each other. We evaluated our model on the classification tasks of\nsentiment analysis and emoji prediction for resource-poor languages - Hindi and\nTelugu and resource-rich languages - English and Spanish. Our model\nsignificantly outperforms the state-of-the-art approaches in both the tasks\nacross all metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03621v2",
            "date": "2018-06-10 09:40:08+00:00",
            "title": "Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search",
            "authors": [
                "Yougen Yuan",
                "Cheung-Chi Leung",
                "Lei Xie",
                "Hongjie Chen",
                "Bin Ma",
                "Haizhou Li"
            ],
            "abstract": "We propose to learn acoustic word embeddings with temporal context for\nquery-by-example (QbE) speech search. The temporal context includes the leading\nand trailing word sequences of a word. We assume that there exist spoken word\npairs in the training database. We pad the word pairs with their original\ntemporal context to form fixed-length speech segment pairs. We obtain the\nacoustic word embeddings through a deep convolutional neural network (CNN)\nwhich is trained on the speech segment pairs with a triplet loss. Shifting a\nfixed-length analysis window through the search content, we obtain a running\nsequence of embeddings. In this way, searching for the spoken query is\nequivalent to the matching of acoustic word embeddings. The experiments show\nthat our proposed acoustic word embeddings learned with temporal context are\neffective in QbE speech search. They outperform the state-of-the-art\nframe-level feature representations and reduce run-time computation since no\ndynamic time warping is required in QbE speech search. We also find that it is\nimportant to have sufficient speech segment pairs to train the deep CNN for\neffective acoustic word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03653v1",
            "date": "2018-06-10 13:00:15+00:00",
            "title": "SciDTB: Discourse Dependency TreeBank for Scientific Abstracts",
            "authors": [
                "An Yang",
                "Sujian Li"
            ],
            "abstract": "Annotation corpus for discourse relations benefits NLP tasks such as machine\ntranslation and question answering. In this paper, we present SciDTB, a\ndomain-specific discourse treebank annotated on scientific articles. Different\nfrom widely-used RST-DT and PDTB, SciDTB uses dependency trees to represent\ndiscourse structure, which is flexible and simplified to some extent but do not\nsacrifice structural integrity. We discuss the labeling framework, annotation\nworkflow and some statistics about SciDTB. Furthermore, our treebank is made as\na benchmark for evaluating discourse dependency parsers, on which we provide\nseveral baselines as fundamental work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03661v1",
            "date": "2018-06-10 13:50:17+00:00",
            "title": "Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation",
            "authors": [
                "Fahim Dalvi",
                "Nadir Durrani",
                "Hassan Sajjad",
                "Stephan Vogel"
            ],
            "abstract": "We address the problem of simultaneous translation by modifying the Neural MT\ndecoder to operate with dynamically built encoder and attention. We propose a\ntunable agent which decides the best segmentation strategy for a user-defined\nBLEU loss and Average Proportion (AP) constraint. Our agent outperforms\npreviously proposed Wait-if-diff and Wait-if-worse agents (Cho and Esipova,\n2016) on BLEU with a lower latency. Secondly we proposed data-driven changes to\nNeural MT training to better match the incremental decoding framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03711v2",
            "date": "2018-06-10 19:29:03+00:00",
            "title": "Deep Reinforcement Learning for Chinese Zero pronoun Resolution",
            "authors": [
                "Qingyu Yin",
                "Yu Zhang",
                "Weinan Zhang",
                "Ting Liu",
                "William Yang Wang"
            ],
            "abstract": "Deep neural network models for Chinese zero pronoun resolution learn semantic\ninformation for zero pronoun and candidate antecedents, but tend to be\nshort-sighted---they often make local decisions. They typically predict\ncoreference chains between the zero pronoun and one single candidate antecedent\none link at a time, while overlooking their long-term influence on future\ndecisions. Ideally, modeling useful information of preceding potential\nantecedents is critical when later predicting zero pronoun-candidate antecedent\npairs. In this study, we show how to integrate local and global decision-making\nby exploiting deep reinforcement learning models. With the help of the\nreinforcement learning agent, our model learns the policy of selecting\nantecedents in a sequential manner, where useful information provided by\nearlier predicted antecedents could be utilized for making later coreference\ndecisions. Experimental results on OntoNotes 5.0 dataset show that our\ntechnique surpasses the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03713v1",
            "date": "2018-06-10 19:46:17+00:00",
            "title": "All-in-one: Multi-task Learning for Rumour Verification",
            "authors": [
                "Elena Kochkina",
                "Maria Liakata",
                "Arkaitz Zubiaga"
            ],
            "abstract": "Automatic resolution of rumours is a challenging task that can be broken down\ninto smaller components that make up a pipeline, including rumour detection,\nrumour tracking and stance classification, leading to the final outcome of\ndetermining the veracity of a rumour. In previous work, these steps in the\nprocess of rumour verification have been developed as separate components where\nthe output of one feeds into the next. We propose a multi-task learning\napproach that allows joint training of the main and auxiliary tasks, improving\nthe performance of rumour verification. We examine the connection between the\ndataset properties and the outcomes of the multi-task learning models used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03740v2",
            "date": "2018-06-10 23:19:07+00:00",
            "title": "Unsupervised Disambiguation of Syncretism in Inflected Lexicons",
            "authors": [
                "Ryan Cotterell",
                "Christo Kirov",
                "Sabrina J. Mielke",
                "Jason Eisner"
            ],
            "abstract": "Lexical ambiguity makes it difficult to compute various useful statistics of\na corpus. A given word form might represent any of several morphological\nfeature bundles. One can, however, use unsupervised learning (as in EM) to fit\na model that probabilistically disambiguates word forms. We present such an\napproach, which employs a neural network to smoothly model a prior distribution\nover feature bundles (even rare ones). Although this basic model does not\nconsider a token's context, that very property allows it to operate on a simple\nlist of unigram type counts, partitioning each count among different analyses\nof that unigram. We discuss evaluation metrics for this novel task and report\nresults on 5 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03743v2",
            "date": "2018-06-10 23:24:33+00:00",
            "title": "Are All Languages Equally Hard to Language-Model?",
            "authors": [
                "Ryan Cotterell",
                "Sabrina J. Mielke",
                "Jason Eisner",
                "Brian Roark"
            ],
            "abstract": "For general modeling methods applied to diverse languages, a natural question\nis: how well should we expect our models to work on languages with differing\ntypological profiles? In this work, we develop an evaluation framework for fair\ncross-linguistic comparison of language models, using translated text so that\nall models are asked to predict approximately the same information. We then\nconduct a study on 21 languages, demonstrating that in some languages, the\ntextual expression of the information is harder to predict with both $n$-gram\nand LSTM language models. We show complex inflectional morphology to be a cause\nof performance differences among languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03746v2",
            "date": "2018-06-10 23:47:53+00:00",
            "title": "A Structured Variational Autoencoder for Contextual Morphological Inflection",
            "authors": [
                "Lawrence Wolf-Sonkin",
                "Jason Naradowsky",
                "Sabrina J. Mielke",
                "Ryan Cotterell"
            ],
            "abstract": "Statistical morphological inflectors are typically trained on fully\nsupervised, type-level data. One remaining open research question is the\nfollowing: How can we effectively exploit raw, token-level data to improve\ntheir performance? To this end, we introduce a novel generative latent-variable\nmodel for the semi-supervised learning of inflection generation. To enable\nposterior inference over the latent variables, we derive an efficient\nvariational inference procedure based on the wake-sleep algorithm. We\nexperiment on 23 languages, using the Universal Dependencies corpora in a\nsimulated low-resource setting, and find improvements of over 10% absolute\naccuracy in some cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03757v1",
            "date": "2018-06-11 01:08:52+00:00",
            "title": "Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource",
            "authors": [
                "Antonis Anastasopoulos",
                "Marika Lekakou",
                "Josep Quer",
                "Eleni Zimianiti",
                "Justin DeBenedetto",
                "David Chiang"
            ],
            "abstract": "Most work on part-of-speech (POS) tagging is focused on high resource\nlanguages, or examines low-resource and active learning settings through\nsimulated studies. We evaluate POS tagging techniques on an actual endangered\nlanguage, Griko. We present a resource that contains 114 narratives in Griko,\nalong with sentence-level translations in Italian, and provides gold\nannotations for the test set. Based on a previously collected small corpus, we\ninvestigate several traditional methods, as well as methods that take advantage\nof monolingual data or project cross-lingual POS tags. We show that the\ncombination of a semi-supervised method with cross-lingual transfer is more\nappropriate for this extremely challenging setting, with the best tagger\nachieving an accuracy of 72.9%. With an applied active learning scheme, which\nwe use to collect sentence-level annotations over the test set, we achieve\nimprovements of more than 21 percentage points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03821v1",
            "date": "2018-06-11 06:08:40+00:00",
            "title": "Addition of Code Mixed Features to Enhance the Sentiment Prediction of Song Lyrics",
            "authors": [
                "Gangula Rama Rohit Reddy",
                "Radhika Mamidi"
            ],
            "abstract": "Sentiment analysis, also called opinion mining, is the field of study that\nanalyzes people's opinions,sentiments, attitudes and emotions. Songs are\nimportant to sentiment analysis since the songs and mood are mutually dependent\non each other. Based on the selected song it becomes easy to find the mood of\nthe listener, in future it can be used for recommendation. The song lyric is a\nrich source of datasets containing words that are helpful in analysis and\nclassification of sentiments generated from it. Now a days we observe a lot of\ninter-sentential and intra-sentential code-mixing in songs which has a varying\nimpact on audience. To study this impact we created a Telugu songs dataset\nwhich contained both Telugu-English code-mixed and pure Telugu songs. In this\npaper, we classify the songs based on its arousal as exciting or non-exciting.\nWe develop a language identification tool and introduce code-mixing features\nobtained from it as additional features. Our system with these additional\nfeatures attains 4-5% accuracy greater than traditional approaches on our\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03822v1",
            "date": "2018-06-11 06:10:11+00:00",
            "title": "Know What You Don't Know: Unanswerable Questions for SQuAD",
            "authors": [
                "Pranav Rajpurkar",
                "Robin Jia",
                "Percy Liang"
            ],
            "abstract": "Extractive reading comprehension systems can often locate the correct answer\nto a question in a context document, but they also tend to make unreliable\nguesses on questions for which the correct answer is not stated in the context.\nExisting datasets either focus exclusively on answerable questions, or use\nautomatically generated unanswerable questions that are easy to identify. To\naddress these weaknesses, we present SQuAD 2.0, the latest version of the\nStanford Question Answering Dataset (SQuAD). SQuAD 2.0 combines existing SQuAD\ndata with over 50,000 unanswerable questions written adversarially by\ncrowdworkers to look similar to answerable ones. To do well on SQuAD 2.0,\nsystems must not only answer questions when possible, but also determine when\nno answer is supported by the paragraph and abstain from answering. SQuAD 2.0\nis a challenging natural language understanding task for existing models: a\nstrong neural system that gets 86% F1 on SQuAD 1.1 achieves only 66% F1 on\nSQuAD 2.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03869v2",
            "date": "2018-06-11 09:18:52+00:00",
            "title": "Distance-Free Modeling of Multi-Predicate Interactions in End-to-End Japanese Predicate-Argument Structure Analysis",
            "authors": [
                "Yuichiroh Matsubayashi",
                "Kentaro Inui"
            ],
            "abstract": "Capturing interactions among multiple predicate-argument structures (PASs) is\na crucial issue in the task of analyzing PAS in Japanese. In this paper, we\npropose new Japanese PAS analysis models that integrate the label prediction\ninformation of arguments in multiple PASs by extending the input and last\nlayers of a standard deep bidirectional recurrent neural network (bi-RNN)\nmodel. In these models, using the mechanisms of pooling and attention, we aim\nto directly capture the potential interactions among multiple PASs, without\nbeing disturbed by the word order and distance. Our experiments show that the\nproposed models improve the prediction accuracy specifically for cases where\nthe predicate and argument are in an indirect dependency relation and achieve a\nnew state of the art in the overall $F_1$ on a standard benchmark corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04068v1",
            "date": "2018-06-11 15:50:13+00:00",
            "title": "A Co-Matching Model for Multi-choice Reading Comprehension",
            "authors": [
                "Shuohang Wang",
                "Mo Yu",
                "Shiyu Chang",
                "Jing Jiang"
            ],
            "abstract": "Multi-choice reading comprehension is a challenging task, which involves the\nmatching between a passage and a question-answer pair. This paper proposes a\nnew co-matching approach to this problem, which jointly models whether a\npassage can match both a question and a candidate answer. Experimental results\non the RACE dataset demonstrate that our approach achieves state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04092v2",
            "date": "2018-06-11 16:26:56+00:00",
            "title": "WikiRef: Wikilinks as a route to recommending appropriate references for scientific Wikipedia pages",
            "authors": [
                "Abhik Jana",
                "Pranjal Kanojiya",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "The exponential increase in the usage of Wikipedia as a key source of\nscientific knowledge among the researchers is making it absolutely necessary to\nmetamorphose this knowledge repository into an integral and self-contained\nsource of information for direct utilization. Unfortunately, the references\nwhich support the content of each Wikipedia entity page, are far from complete.\nWhy are the reference section ill-formed for most Wikipedia pages? Is this\nsection edited as frequently as the other sections of a page? Can there be\nappropriate surrogates that can automatically enhance the reference section? In\nthis paper, we propose a novel two step approach -- WikiRef -- that (i)\nleverages the wikilinks present in a scientific Wikipedia target page and,\nthereby, (ii) recommends highly relevant references to be included in that\ntarget page appropriately and automatically borrowed from the reference section\nof the wikilinks. In the first step, we build a classifier to ascertain whether\na wikilink is a potential source of reference or not. In the following step, we\nrecommend references to the target page from the reference section of the\nwikilinks that are classified as potential sources of references in the first\nstep. We perform an extensive evaluation of our approach on datasets from two\ndifferent domains -- Computer Science and Physics. For Computer Science we\nachieve a notably good performance with a precision@1 of 0.44 for reference\nrecommendation as opposed to 0.38 obtained from the most competitive baseline.\nFor the Physics dataset, we obtain a similar performance boost of 10% with\nrespect to the most competitive baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04127v1",
            "date": "2018-06-11 17:51:23+00:00",
            "title": "Finding Syntax in Human Encephalography with Beam Search",
            "authors": [
                "John Hale",
                "Chris Dyer",
                "Adhiguna Kuncoro",
                "Jonathan R. Brennan"
            ],
            "abstract": "Recurrent neural network grammars (RNNGs) are generative models of\n(tree,string) pairs that rely on neural networks to evaluate derivational\nchoices. Parsing with them using beam search yields a variety of incremental\ncomplexity metrics such as word surprisal and parser action count. When used as\nregressors against human electrophysiological responses to naturalistic text,\nthey derive two amplitude effects: an early peak and a P600-like later peak. By\ncontrast, a non-syntactic neural language model yields no reliable effects.\nModel comparisons attribute the early peak to syntactic composition within the\nRNNG. This pattern of results recommends the RNNG+beam search combination as a\nmechanistic model of the syntactic processing that occurs during normal human\nlanguage comprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04185v1",
            "date": "2018-06-11 18:52:01+00:00",
            "title": "A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature",
            "authors": [
                "Benjamin Nye",
                "Junyi Jessy Li",
                "Roma Patel",
                "Yinfei Yang",
                "Iain J. Marshall",
                "Ani Nenkova",
                "Byron C. Wallace"
            ],
            "abstract": "We present a corpus of 5,000 richly annotated abstracts of medical articles\ndescribing clinical randomized controlled trials. Annotations include\ndemarcations of text spans that describe the Patient population enrolled, the\nInterventions studied and to what they were Compared, and the Outcomes measured\n(the `PICO' elements). These spans are further annotated at a more granular\nlevel, e.g., individual interventions within them are marked and mapped onto a\nstructured medical vocabulary. We acquired annotations from a diverse set of\nworkers with varying levels of expertise and cost. We describe our data\ncollection process and the corpus itself in detail. We then outline a set of\nchallenging NLP tasks that would aid searching of the medical literature and\nthe practice of evidence-based medicine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04197v1",
            "date": "2018-06-11 19:07:40+00:00",
            "title": "Degree based Classification of Harmful Speech using Twitter Data",
            "authors": [
                "Sanjana Sharma",
                "Saksham Agrawal",
                "Manish Shrivastava"
            ],
            "abstract": "Harmful speech has various forms and it has been plaguing the social media in\ndifferent ways. If we need to crackdown different degrees of hate speech and\nabusive behavior amongst it, the classification needs to be based on complex\nramifications which needs to be defined and hold accountable for, other than\nracist, sexist or against some particular group and community. This paper\nprimarily describes how we created an ontological classification of harmful\nspeech based on degree of hateful intent, and used it to annotate twitter data\naccordingly. The key contribution of this paper is the new dataset of tweets we\ncreated based on ontological classes and degrees of harmful speech found in the\ntext. We also propose supervised classification system for recognizing these\nrespective harmful speech classes in the texts hence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04262v1",
            "date": "2018-06-11 22:44:38+00:00",
            "title": "Let's do it \"again\": A First Computational Approach to Detecting Adverbial Presupposition Triggers",
            "authors": [
                "Andre Cianflone",
                "Yulan Feng",
                "Jad Kabbara",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "We introduce the task of predicting adverbial presupposition triggers such as\nalso and again. Solving such a task requires detecting recurring or similar\nevents in the discourse context, and has applications in natural language\ngeneration tasks such as summarization and dialogue systems. We create two new\ndatasets for the task, derived from the Penn Treebank and the Annotated English\nGigaword corpora, as well as a novel attention mechanism tailored to this task.\nOur attention mechanism augments a baseline recurrent neural network without\nthe need for additional trainable parameters, minimizing the added\ncomputational cost of our mechanism. We demonstrate that our model\nstatistically outperforms a number of baselines, including an LSTM-based\nlanguage model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04270v1",
            "date": "2018-06-11 23:51:18+00:00",
            "title": "Learning Multilingual Topics from Incomparable Corpus",
            "authors": [
                "Shudong Hao",
                "Michael J. Paul"
            ],
            "abstract": "Multilingual topic models enable crosslingual tasks by extracting consistent\ntopics from multilingual corpora. Most models require parallel or comparable\ntraining corpora, which limits their ability to generalize. In this paper, we\nfirst demystify the knowledge transfer mechanism behind multilingual topic\nmodels by defining an alternative but equivalent formulation. Based on this\nanalysis, we then relax the assumption of training data required by most\nexisting models, creating a model that only requires a dictionary for training.\nExperiments show that our new method effectively learns coherent multilingual\ntopics from partially and fully incomparable corpora with limited amounts of\ndictionary resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04291v1",
            "date": "2018-06-12 01:26:55+00:00",
            "title": "Challenges of language technologies for the indigenous languages of the Americas",
            "authors": [
                "Manuel Mager",
                "Ximena Gutierrez-Vasques",
                "Gerardo Sierra",
                "Ivan Meza"
            ],
            "abstract": "Indigenous languages of the American continent are highly diverse. However,\nthey have received little attention from the technological perspective. In this\npaper, we review the research, the digital resources and the available NLP\nsystems that focus on these languages. We present the main challenges and\nresearch questions that arise when distant languages and low-resource scenarios\nare faced. We would like to encourage NLP research in linguistically rich and\ndiverse areas like the Americas.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04327v1",
            "date": "2018-06-12 04:22:11+00:00",
            "title": "ISO-Standard Domain-Independent Dialogue Act Tagging for Conversational Agents",
            "authors": [
                "Stefano Mezza",
                "Alessandra Cervone",
                "Giuliano Tortoreto",
                "Evgeny A. Stepanov",
                "Giuseppe Riccardi"
            ],
            "abstract": "Dialogue Act (DA) tagging is crucial for spoken language understanding\nsystems, as it provides a general representation of speakers' intents, not\nbound to a particular dialogue system. Unfortunately, publicly available data\nsets with DA annotation are all based on different annotation schemes and thus\nincompatible with each other. Moreover, their schemes often do not cover all\naspects necessary for open-domain human-machine interaction. In this paper, we\npropose a methodology to map several publicly available corpora to a subset of\nthe ISO standard, in order to create a large task-independent training corpus\nfor DA classification. We show the feasibility of using this corpus to train a\ndomain-independent DA tagger testing it on out-of-domain conversational data,\nand argue the importance of training on multiple corpora to achieve robustness\nacross different DA categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04330v2",
            "date": "2018-06-12 04:48:06+00:00",
            "title": "Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering",
            "authors": [
                "Wuwei Lan",
                "Wei Xu"
            ],
            "abstract": "In this paper, we analyze several neural network designs (and their\nvariations) for sentence pair modeling and compare their performance\nextensively across eight datasets, including paraphrase identification,\nsemantic textual similarity, natural language inference, and question answering\ntasks. Although most of these models have claimed state-of-the-art performance,\nthe original papers often reported on only one or two selected datasets. We\nprovide a systematic study and show that (i) encoding contextual information by\nLSTM and inter-sentence interactions are critical, (ii) Tree-LSTM does not help\nas much as previously claimed but surprisingly improves performance on Twitter\ndatasets, (iii) the Enhanced Sequential Inference Model is the best so far for\nlarger datasets, while the Pairwise Word Interaction Model achieves the best\nperformance when less data is available. We release our implementations as an\nopen-source toolkit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04346v1",
            "date": "2018-06-12 06:04:11+00:00",
            "title": "Exploiting Document Knowledge for Aspect-level Sentiment Classification",
            "authors": [
                "Ruidan He",
                "Wee Sun Lee",
                "Hwee Tou Ng",
                "Daniel Dahlmeier"
            ],
            "abstract": "Attention-based long short-term memory (LSTM) networks have proven to be\nuseful in aspect-level sentiment classification. However, due to the\ndifficulties in annotating aspect-level data, existing public datasets for this\ntask are all relatively small, which largely limits the effectiveness of those\nneural models. In this paper, we explore two approaches that transfer knowledge\nfrom document- level data, which is much less expensive to obtain, to improve\nthe performance of aspect-level sentiment classification. We demonstrate the\neffectiveness of our approaches on 4 public datasets from SemEval 2014, 2015,\nand 2016, and we show that attention-based LSTM benefits from document-level\nknowledge in multiple ways.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04357v1",
            "date": "2018-06-12 06:59:43+00:00",
            "title": "Multi-Task Neural Models for Translating Between Styles Within and Across Languages",
            "authors": [
                "Xing Niu",
                "Sudha Rao",
                "Marine Carpuat"
            ],
            "abstract": "Generating natural language requires conveying content in an appropriate\nstyle. We explore two related tasks on generating text of varying formality:\nmonolingual formality transfer and formality-sensitive machine translation. We\npropose to solve these tasks jointly using multi-task learning, and show that\nour models achieve state-of-the-art performance for formality transfer and are\nable to perform formality-sensitive translation without being explicitly\ntrained on style-annotated translation examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04381v2",
            "date": "2018-06-12 08:16:28+00:00",
            "title": "Projecting Embeddings for Domain Adaptation: Joint Modeling of Sentiment Analysis in Diverse Domains",
            "authors": [
                "Jeremy Barnes",
                "Roman Klinger",
                "Sabine Schulte im Walde"
            ],
            "abstract": "Domain adaptation for sentiment analysis is challenging due to the fact that\nsupervised classifiers are very sensitive to changes in domain. The two most\nprominent approaches to this problem are structural correspondence learning and\nautoencoders. However, they either require long training times or suffer\ngreatly on highly divergent domains. Inspired by recent advances in\ncross-lingual sentiment analysis, we provide a novel perspective and cast the\ndomain adaptation problem as an embedding projection task. Our model takes as\ninput two mono-domain embedding spaces and learns to project them to a\nbi-domain space, which is jointly optimized to (1) project across domains and\nto (2) predict sentiment. We perform domain adaptation experiments on 20\nsource-target domain pairs for sentiment classification and report novel\nstate-of-the-art results on 11 domain pairs, including the Amazon domain\nadaptation datasets and SemEval 2013 and 2016 datasets. Our analysis shows that\nour model performs comparably to state-of-the-art approaches on domains that\nare similar, while performing significantly better on highly divergent domains.\nOur code is available at https://github.com/jbarnesspain/domain_blse",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04402v1",
            "date": "2018-06-12 09:17:40+00:00",
            "title": "Explaining and Generalizing Back-Translation through Wake-Sleep",
            "authors": [
                "Ryan Cotterell",
                "Julia Kreutzer"
            ],
            "abstract": "Back-translation has become a commonly employed heuristic for semi-supervised\nneural machine translation. The technique is both straightforward to apply and\nhas led to state-of-the-art results. In this work, we offer a principled\ninterpretation of back-translation as approximate inference in a generative\nmodel of bitext and show how the standard implementation of back-translation\ncorresponds to a single iteration of the wake-sleep algorithm in our proposed\nmodel. Moreover, this interpretation suggests a natural iterative\ngeneralization, which we demonstrate leads to further improvement of up to 1.6\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04441v1",
            "date": "2018-06-12 11:21:16+00:00",
            "title": "Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation",
            "authors": [
                "Haoyang Wen",
                "Yijia Liu",
                "Wanxiang Che",
                "Libo Qin",
                "Ting Liu"
            ],
            "abstract": "Classic pipeline models for task-oriented dialogue system require explicit\nmodeling the dialogue states and hand-crafted action spaces to query a\ndomain-specific knowledge base. Conversely, sequence-to-sequence models learn\nto map dialogue history to the response in current turn without explicit\nknowledge base querying. In this work, we propose a novel framework that\nleverages the advantages of classic pipeline and sequence-to-sequence models.\nOur framework models a dialogue state as a fixed-size distributed\nrepresentation and use this representation to query a knowledge base via an\nattention mechanism. Experiment on Stanford Multi-turn Multi-domain\nTask-oriented Dialogue Dataset shows that our framework significantly\noutperforms other sequence-to-sequence based baseline models on both automatic\nand human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04450v1",
            "date": "2018-06-12 11:46:51+00:00",
            "title": "An Ensemble Model for Sentiment Analysis of Hindi-English Code-Mixed Data",
            "authors": [
                "Madan Gopal Jhanwar",
                "Arpita Das"
            ],
            "abstract": "In multilingual societies like India, code-mixed social media texts comprise\nthe majority of the Internet. Detecting the sentiment of the code-mixed user\nopinions plays a crucial role in understanding social, economic and political\ntrends. In this paper, we propose an ensemble of character-trigrams based LSTM\nmodel and word-ngrams based Multinomial Naive Bayes (MNB) model to identify the\nsentiments of Hindi-English (Hi-En) code-mixed data. The ensemble model\ncombines the strengths of rich sequential patterns from the LSTM model and\npolarity of keywords from the probabilistic ngram model to identify sentiments\nin sparse and inconsistent code-mixed data. Experiments on reallife user\ncode-mixed data reveals that our approach yields state-of-the-art results as\ncompared to several baselines and other deep learning based proposed methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04456v1",
            "date": "2018-06-12 12:08:30+00:00",
            "title": "Impersonation: Modeling Persona in Smart Responses to Email",
            "authors": [
                "Rajeev Gupta",
                "Ranganath Kondapally",
                "Chakrapani Ravi Kiran"
            ],
            "abstract": "In this paper, we present design, implementation, and effectiveness of\ngenerating personalized suggestions for email replies. To personalize email\nresponses based on users style and personality, we model the users persona\nbased on her past responses to emails. This model is added to the\nlanguage-based model created across users using past responses of the all user\nemails.\n  A users model captures the typical responses of the user given a particular\ncontext. The context includes the email received, recipient of the email, and\nother external signals such as calendar activities, preferences, etc. The\ncontext along with users personality (e.g., extrovert, formal, reserved, etc.)\nis used to suggest responses. These responses can be a mixture of multiple\nmodes: email replies (textual), audio clips, etc. This helps in making\nresponses mimic the user as much as possible and helps the user to be more\nproductive while retaining her mark in the responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04466v1",
            "date": "2018-06-12 12:37:20+00:00",
            "title": "Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model",
            "authors": [
                "Shaohui Kuang",
                "Deyi Xiong"
            ],
            "abstract": "Neural machine translation (NMT) systems are usually trained on a large\namount of bilingual sentence pairs and translate one sentence at a time,\nignoring inter-sentence information. This may make the translation of a\nsentence ambiguous or even inconsistent with the translations of neighboring\nsentences. In order to handle this issue, we propose an inter-sentence gate\nmodel that uses the same encoder to encode two adjacent sentences and controls\nthe amount of information flowing from the preceding sentence to the\ntranslation of the current sentence with an inter-sentence gate. In this way,\nour proposed model can capture the connection between sentences and fuse\nrecency from neighboring sentences into neural machine translation. On several\nNIST Chinese-English translation tasks, our experiments demonstrate that the\nproposed inter-sentence gate model achieves substantial improvements over the\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04470v2",
            "date": "2018-06-12 12:43:42+00:00",
            "title": "Design Challenges and Misconceptions in Neural Sequence Labeling",
            "authors": [
                "Jie Yang",
                "Shuailong Liang",
                "Yue Zhang"
            ],
            "abstract": "We investigate the design challenges of constructing effective and efficient\nneural sequence labeling systems, by reproducing twelve neural sequence\nlabeling models, which include most of the state-of-the-art structures, and\nconduct a systematic model comparison on three benchmarks (i.e. NER, Chunking,\nand POS tagging). Misconceptions and inconsistent conclusions in existing\nliterature are examined and clarified under statistical experiments. In the\ncomparison and analysis process, we reach several practical conclusions which\ncan be useful to practitioners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04508v2",
            "date": "2018-06-07 23:04:19+00:00",
            "title": "Characterizing Departures from Linearity in Word Translation",
            "authors": [
                "Ndapa Nakashole",
                "Raphael Flauger"
            ],
            "abstract": "We investigate the behavior of maps learned by machine translation methods.\nThe maps translate words by projecting between word embedding spaces of\ndifferent languages. We locally approximate these maps using linear maps, and\nfind that they vary across the word embedding space. This demonstrates that the\nunderlying maps are non-linear. Importantly, we show that the locally linear\nmaps vary by an amount that is tightly correlated with the distance between the\nneighborhoods on which they are trained. Our results can be used to test\nnon-linear methods, and to drive the design of more accurate maps for word\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04523v1",
            "date": "2018-06-12 13:52:15+00:00",
            "title": "Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs",
            "authors": [
                "Wenpeng Yin",
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Large scale knowledge graphs (KGs) such as Freebase are generally incomplete.\nReasoning over multi-hop (mh) KG paths is thus an important capability that is\nneeded for question answering or other NLP tasks that require knowledge about\nthe world. mh-KG reasoning includes diverse scenarios, e.g., given a head\nentity and a relation path, predict the tail entity; or given two entities\nconnected by some relation paths, predict the unknown relation between them. We\npresent ROPs, recurrent one-hop predictors, that predict entities at each step\nof mh-KB paths by using recurrent neural networks and vector representations of\nentities and relations, with two benefits: (i) modeling mh-paths of arbitrary\nlengths while updating the entity and relation representations by the training\nsignal at each step; (ii) handling different types of mh-KG reasoning in a\nunified framework. Our models show state-of-the-art for two important multi-hop\nKG reasoning tasks: Knowledge Base Completion and Path Query Answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04524v1",
            "date": "2018-06-12 13:53:22+00:00",
            "title": "Learning to Automatically Generate Fill-In-The-Blank Quizzes",
            "authors": [
                "Edison Marrese-Taylor",
                "Ai Nakajima",
                "Yutaka Matsuo",
                "Ono Yuichi"
            ],
            "abstract": "In this paper we formalize the problem automatic fill-in-the-blank question\ngeneration using two standard NLP machine learning schemes, proposing concrete\ndeep learning models for each. We present an empirical study based on data\nobtained from a language learning platform showing that both of our proposed\nsettings offer promising results.",
            "journal ref": "5th Workshop on Natural Language Processing Techniques for\n  Educational Applications (NLPTEA), collocated with ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04525v1",
            "date": "2018-06-09 18:09:37+00:00",
            "title": "Second Language Acquisition Modeling: An Ensemble Approach",
            "authors": [
                "Anton Osika",
                "Susanna Nilsson",
                "Andrii Sydorchuk",
                "Faruk Sahin",
                "Anders Huss"
            ],
            "abstract": "Accurate prediction of students knowledge is a fundamental building block of\npersonalized learning systems. Here, we propose a novel ensemble model to\npredict student knowledge gaps. Applying our approach to student trace data\nfrom the online educational platform Duolingo we achieved highest score on both\nevaluation metrics for all three datasets in the 2018 Shared Task on Second\nLanguage Acquisition Modeling. We describe our model and discuss relevance of\nthe task compared to how it would be setup in a production environment for\npersonalized education.",
            "journal ref": "Proceedings of the Thirteenth Workshop, Building Educational\n  Applications, NAACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04532v1",
            "date": "2018-06-12 14:00:18+00:00",
            "title": "Term Definitions Help Hypernymy Detection",
            "authors": [
                "Wenpeng Yin",
                "Dan Roth"
            ],
            "abstract": "Existing methods of hypernymy detection mainly rely on statistics over a big\ncorpus, either mining some co-occurring patterns like \"animals such as cats\" or\nembedding words of interest into context-aware vectors. These approaches are\ntherefore limited by the availability of a large enough corpus that can cover\nall terms of interest and provide sufficient contextual information to\nrepresent their meaning. In this work, we propose a new paradigm, HyperDef, for\nhypernymy detection -- expressing word meaning by encoding word definitions,\nalong with context driven representation. This has two main benefits: (i)\nDefinitional sentences express (sense-specific) corpus-independent meanings of\nwords, hence definition-driven approaches enable strong generalization -- once\ntrained, the model is expected to work well in open-domain testbeds; (ii)\nGlobal context from a large corpus and definitions provide complementary\ninformation for words. Consequently, our model, HyperDef, once trained on\ntask-agnostic data, gets state-of-the-art results in multiple benchmarks",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04713v2",
            "date": "2018-06-12 18:53:14+00:00",
            "title": "Evaluation of Unsupervised Compositional Representations",
            "authors": [
                "Hanan Aldarmaki",
                "Mona Diab"
            ],
            "abstract": "We evaluated various compositional models, from bag-of-words representations\nto compositional RNN-based models, on several extrinsic supervised and\nunsupervised evaluation benchmarks. Our results confirm that weighted vector\naveraging can outperform context-sensitive models in most benchmarks, but\nstructural features encoded in RNN models can also be useful in certain\nclassification tasks. We analyzed some of the evaluation datasets to identify\nthe aspects of meaning they measure and the characteristics of the various\nmodels that explain their performance variance.",
            "journal ref": "Proceedings of the 27th International Conference on Computational\n  Linguistics (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04818v2",
            "date": "2018-06-13 01:58:22+00:00",
            "title": "Using Clinical Narratives and Structured Data to Identify Distant Recurrences in Breast Cancer",
            "authors": [
                "Zexian Zeng",
                "Ankita Roy",
                "Xiaoyu Li",
                "Sasa Espino",
                "Susan Clare",
                "Seema Khan",
                "Yuan Luo"
            ],
            "abstract": "Accurately identifying distant recurrences in breast cancer from the\nElectronic Health Records (EHR) is important for both clinical care and\nsecondary analysis. Although multiple applications have been developed for\ncomputational phenotyping in breast cancer, distant recurrence identification\nstill relies heavily on manual chart review. In this study, we aim to develop a\nmodel that identifies distant recurrences in breast cancer using clinical\nnarratives and structured data from EHR. We apply MetaMap to extract features\nfrom clinical narratives and also retrieve structured clinical data from EHR.\nUsing these features, we train a support vector machine model to identify\ndistant recurrences in breast cancer patients. We train the model using 1,396\ndouble-annotated subjects and validate the model using 599 double-annotated\nsubjects. In addition, we validate the model on a set of 4,904 single-annotated\nsubjects as a generalization test. We obtained a high area under curve (AUC)\nscore of 0.92 (SD=0.01) in the cross-validation using the training dataset,\nthen obtained AUC scores of 0.95 and 0.93 in the held-out test and\ngeneralization test using 599 and 4,904 samples respectively. Our model can\naccurately and efficiently identify distant recurrences in breast cancer by\ncombining features extracted from unstructured clinical narratives and\nstructured clinical data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04820v2",
            "date": "2018-06-13 02:14:19+00:00",
            "title": "Natural Language Processing for EHR-Based Computational Phenotyping",
            "authors": [
                "Zexian Zeng",
                "Yu Deng",
                "Xiaoyu Li",
                "Tristan Naumann",
                "Yuan Luo"
            ],
            "abstract": "This article reviews recent advances in applying natural language processing\n(NLP) to Electronic Health Records (EHRs) for computational phenotyping.\nNLP-based computational phenotyping has numerous applications including\ndiagnosis categorization, novel phenotype discovery, clinical trial screening,\npharmacogenomics, drug-drug interaction (DDI) and adverse drug event (ADE)\ndetection, as well as genome-wide and phenome-wide association studies.\nSignificant progress has been made in algorithm development and resource\nconstruction for computational phenotyping. Among the surveyed methods,\nwell-designed keyword search and rule-based systems often achieve good\nperformance. However, the construction of keyword and rule lists requires\nsignificant manual effort, which is difficult to scale. Supervised machine\nlearning models have been favored because they are capable of acquiring both\nclassification patterns and structures from data. Recently, deep learning and\nunsupervised learning have received growing attention, with the former favored\nfor its performance and the latter for its ability to find novel phenotypes.\nIntegrating heterogeneous data sources have become increasingly important and\nhave shown promise in improving model performance. Often better performance is\nachieved by combining multiple modalities of information. Despite these many\nadvances, challenges and opportunities remain for NLP-based computational\nphenotyping, including better model interpretability and generalizability, and\nproper characterization of feature relations in clinical narratives",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04822v3",
            "date": "2018-06-13 02:16:01+00:00",
            "title": "SGM: Sequence Generation Model for Multi-label Classification",
            "authors": [
                "Pengcheng Yang",
                "Xu Sun",
                "Wei Li",
                "Shuming Ma",
                "Wei Wu",
                "Houfeng Wang"
            ],
            "abstract": "Multi-label classification is an important yet challenging task in natural\nlanguage processing. It is more complex than single-label classification in\nthat the labels tend to be correlated. Existing methods tend to ignore the\ncorrelations between labels. Besides, different parts of the text can\ncontribute differently for predicting different labels, which is not considered\nby existing models. In this paper, we propose to view the multi-label\nclassification task as a sequence generation problem, and apply a sequence\ngeneration model with a novel decoder structure to solve it. Extensive\nexperimental results show that our proposed methods outperform previous work by\na substantial margin. Further analysis of experimental results demonstrates\nthat the proposed methods not only capture the correlations between labels, but\nalso select the most informative words automatically when predicting different\nlabels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04856v2",
            "date": "2018-06-13 05:51:10+00:00",
            "title": "Double Path Networks for Sequence to Sequence Learning",
            "authors": [
                "Kaitao Song",
                "Xu Tan",
                "Di He",
                "Jianfeng Lu",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Encoder-decoder based Sequence to Sequence learning (S2S) has made remarkable\nprogress in recent years. Different network architectures have been used in the\nencoder/decoder. Among them, Convolutional Neural Networks (CNN) and Self\nAttention Networks (SAN) are the prominent ones. The two architectures achieve\nsimilar performances but use very different ways to encode and decode context:\nCNN use convolutional layers to focus on the local connectivity of the\nsequence, while SAN uses self-attention layers to focus on global semantics. In\nthis work we propose Double Path Networks for Sequence to Sequence learning\n(DPN-S2S), which leverage the advantages of both models by using double path\ninformation fusion. During the encoding step, we develop a double path\narchitecture to maintain the information coming from different paths with\nconvolutional layers and self-attention layers separately. To effectively use\nthe encoded context, we develop a cross attention module with gating and use it\nto automatically pick up the information needed during the decoding step. By\ndeeply integrating the two paths with cross attention, both types of\ninformation are combined and well exploited. Experiments show that our proposed\nmethod can significantly improve the performance of sequence to sequence\nlearning over state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04936v3",
            "date": "2018-06-13 10:35:45+00:00",
            "title": "On Accurate Evaluation of GANs for Language Generation",
            "authors": [
                "Stanislau Semeniuta",
                "Aliaksei Severyn",
                "Sylvain Gelly"
            ],
            "abstract": "Generative Adversarial Networks (GANs) are a promising approach to language\ngeneration. The latest works introducing novel GAN models for language\ngeneration use n-gram based metrics for evaluation and only report single\nscores of the best run. In this paper, we argue that this often misrepresents\nthe true picture and does not tell the full story, as GAN models can be\nextremely sensitive to the random initialization and small deviations from the\nbest hyperparameter choice. In particular, we demonstrate that the previously\nused BLEU score is not sensitive to semantic deterioration of generated texts\nand propose alternative metrics that better capture the quality and diversity\nof the generated samples. We also conduct a set of experiments comparing a\nnumber of GAN models for text with a conventional Language Model (LM) and find\nthat neither of the considered models performs convincingly better than the LM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05099v1",
            "date": "2018-06-13 15:05:39+00:00",
            "title": "Graph-Based Decoding for Event Sequencing and Coreference Resolution",
            "authors": [
                "Zhengzhong Liu",
                "Teruko Mitamura",
                "Eduard Hovy"
            ],
            "abstract": "Events in text documents are interrelated in complex ways. In this paper, we\nstudy two types of relation: Event Coreference and Event Sequencing. We show\nthat the popular tree-like decoding structure for automated Event Coreference\nis not suitable for Event Sequencing. To this end, we propose a graph-based\ndecoding algorithm that is applicable to both tasks. The new decoding algorithm\nsupports flexible feature sets for both tasks. Empirically, our event\ncoreference system has achieved state-of-the-art performance on the TAC-KBP\n2015 event coreference task and our event sequencing system beats a strong\ntemporal-based, oracle-informed baseline. We discuss the challenges of studying\nthese event relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05210v2",
            "date": "2018-06-13 18:29:09+00:00",
            "title": "An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization",
            "authors": [
                "Gongbo Tang",
                "Fabienne Cap",
                "Eva Pettersson",
                "Joakim Nivre"
            ],
            "abstract": "In this paper, we apply different NMT models to the problem of historical\nspelling normalization for five languages: English, German, Hungarian,\nIcelandic, and Swedish. The NMT models are at different levels, have different\nattention mechanisms, and different neural network architectures. Our results\nshow that NMT models are much better than SMT models in terms of character\nerror rate. The vanilla RNNs are competitive to GRUs/LSTMs in historical\nspelling normalization. Transformer models perform better only when provided\nwith more training data. We also find that subword-level models with a small\nsubword vocabulary are better than character-level models for low-resource\nlanguages. In addition, we propose a hybrid method which further improves the\nperformance of historical spelling normalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05219v2",
            "date": "2018-06-13 18:51:05+00:00",
            "title": "Bringing replication and reproduction together with generalisability in NLP: Three reproduction studies for Target Dependent Sentiment Analysis",
            "authors": [
                "Andrew Moore",
                "Paul Rayson"
            ],
            "abstract": "Lack of repeatability and generalisability are two significant threats to\ncontinuing scientific development in Natural Language Processing. Language\nmodels and learning methods are so complex that scientific conference papers no\nlonger contain enough space for the technical depth required for replication or\nreproduction. Taking Target Dependent Sentiment Analysis as a case study, we\nshow how recent work in the field has not consistently released code, or\ndescribed settings for learning methods in enough detail, and lacks\ncomparability and generalisability in train, test or validation data. To\ninvestigate generalisability and to enable state of the art comparative\nevaluations, we carry out the first reproduction studies of three groups of\ncomplementary methods and perform the first large-scale mass evaluation on six\ndifferent English datasets. Reflecting on our experiences, we recommend that\nfuture replication or reproduction experiments should always consider a variety\nof datasets alongside documenting and releasing their methods and published\ncode in order to minimise the barriers to both repeatability and\ngeneralisability. We have released our code with a model zoo on GitHub with\nJupyter Notebooks to aid understanding and full documentation, and we recommend\nthat others do the same with their papers at submission time through an\nanonymised GitHub account.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05231v1",
            "date": "2018-06-13 19:14:14+00:00",
            "title": "Beyond Bags of Words: Inferring Systemic Nets",
            "authors": [
                "D. B. Skillicorn",
                "N. Alsadhan"
            ],
            "abstract": "Textual analytics based on representations of documents as bags of words have\nbeen reasonably successful. However, analysis that requires deeper insight into\nlanguage, into author properties, or into the contexts in which documents were\ncreated requires a richer representation. Systemic nets are one such\nrepresentation. They have not been extensively used because they required human\neffort to construct. We show that systemic nets can be algorithmically inferred\nfrom corpora, that the resulting nets are plausible, and that they can provide\npractical benefits for knowledge discovery problems. This opens up a new class\nof practical analysis techniques for textual analytics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05258v2",
            "date": "2018-06-13 20:29:25+00:00",
            "title": "SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions",
            "authors": [
                "Arman Cohan",
                "Bart Desmet",
                "Andrew Yates",
                "Luca Soldaini",
                "Sean MacAvaney",
                "Nazli Goharian"
            ],
            "abstract": "Mental health is a significant and growing public health concern. As language\nusage can be leveraged to obtain crucial insights into mental health\nconditions, there is a need for large-scale, labeled, mental health-related\ndatasets of users who have been diagnosed with one or more of such conditions.\nIn this paper, we investigate the creation of high-precision patterns to\nidentify self-reported diagnoses of nine different mental health conditions,\nand obtain high-quality labeled data without the need for manual labelling. We\nintroduce the SMHD (Self-reported Mental Health Diagnoses) dataset and make it\navailable. SMHD is a novel large dataset of social media posts from users with\none or multiple mental health conditions along with matched control users. We\nexamine distinctions in users' language, as measured by linguistic and\npsychological variables. We further explore text classification methods to\nidentify individuals with mental conditions through their language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05284v1",
            "date": "2018-06-13 22:05:10+00:00",
            "title": "How Predictable is Your State? Leveraging Lexical and Contextual Information for Predicting Legislative Floor Action at the State Level",
            "authors": [
                "Vlad Eidelman",
                "Anastassia Kornilova",
                "Daniel Argyle"
            ],
            "abstract": "Modeling U.S. Congressional legislation and roll-call votes has received\nsignificant attention in previous literature. However, while legislators across\n50 state governments and D.C. propose over 100,000 bills each year, and on\naverage enact over 30% of them, state level analysis has received relatively\nless attention due in part to the difficulty in obtaining the necessary data.\nSince each state legislature is guided by their own procedures, politics and\nissues, however, it is difficult to qualitatively asses the factors that affect\nthe likelihood of a legislative initiative succeeding. Herein, we present\nseveral methods for modeling the likelihood of a bill receiving floor action\nacross all 50 states and D.C. We utilize the lexical content of over 1 million\nbills, along with contextual legislature and legislator derived features to\nbuild our predictive models, allowing a comparison of the factors that are\nimportant to the lawmaking process. Furthermore, we show that these signals\nhold complementary predictive power, together achieving an average improvement\nin accuracy of 18% over state specific baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05432v1",
            "date": "2018-06-14 09:39:32+00:00",
            "title": "Urdu Word Segmentation using Conditional Random Fields (CRFs)",
            "authors": [
                "Haris Bin Zia",
                "Agha Ali Raza",
                "Awais Athar"
            ],
            "abstract": "State-of-the-art Natural Language Processing algorithms rely heavily on\nefficient word segmentation. Urdu is amongst languages for which word\nsegmentation is a complex task as it exhibits space omission as well as space\ninsertion issues. This is partly due to the Arabic script which although\ncursive in nature, consists of characters that have inherent joining and\nnon-joining attributes regardless of word boundary. This paper presents a word\nsegmentation system for Urdu which uses a Conditional Random Field sequence\nmodeler with orthographic, linguistic and morphological features. Our proposed\nmodel automatically learns to predict white space as word boundary as well as\nZero Width Non-Joiner (ZWNJ) as sub-word boundary. Using a manually annotated\ncorpus, our model achieves F1 score of 0.97 for word boundary identification\nand 0.85 for sub-word boundary identification tasks. We have made our code and\ncorpus publicly available to make our results reproducible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05434v1",
            "date": "2018-06-14 09:44:59+00:00",
            "title": "Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce",
            "authors": [
                "Minghui Qiu",
                "Liu Yang",
                "Feng Ji",
                "Weipeng Zhao",
                "Wei Zhou",
                "Jun Huang",
                "Haiqing Chen",
                "W. Bruce Croft",
                "Wei Lin"
            ],
            "abstract": "Building multi-turn information-seeking conversation systems is an important\nand challenging research topic. Although several advanced neural text matching\nmodels have been proposed for this task, they are generally not efficient for\nindustrial applications. Furthermore, they rely on a large amount of labeled\ndata, which may not be available in real-world applications. To alleviate these\nproblems, we study transfer learning for multi-turn information seeking\nconversations in this paper. We first propose an efficient and effective\nmulti-turn conversation model based on convolutional neural networks. After\nthat, we extend our model to adapt the knowledge learned from a resource-rich\ndomain to enhance the performance. Finally, we deployed our model in an\nindustrial chatbot called AliMe Assist\n(https://consumerservice.taobao.com/online-help) and observed a significant\nimprovement over the existing online model.",
            "journal ref": "ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05461v1",
            "date": "2018-06-14 10:59:22+00:00",
            "title": "Learning Cross-lingual Distributed Logical Representations for Semantic Parsing",
            "authors": [
                "Yanyan Zou",
                "Wei Lu"
            ],
            "abstract": "With the development of several multilingual datasets used for semantic\nparsing, recent research efforts have looked into the problem of learning\nsemantic parsers in a multilingual setup. However, how to improve the\nperformance of a monolingual semantic parser for a specific language by\nleveraging data annotated in different languages remains a research question\nthat is under-explored. In this work, we present a study to show how learning\ndistributed representations of the logical forms from data annotated in\ndifferent languages can be used for improving the performance of a monolingual\nsemantic parser. We extend two existing monolingual semantic parsers to\nincorporate such cross-lingual distributed logical representations as features.\nExperiments show that our proposed approach is able to yield improved semantic\nparsing results on the standard multilingual GeoQuery dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05482v1",
            "date": "2018-06-14 11:44:48+00:00",
            "title": "Morphological and Language-Agnostic Word Segmentation for NMT",
            "authors": [
                "Dominik Mach\u00e1\u010dek",
                "Jon\u00e1\u0161 Vidra",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "The state of the art of handling rich morphology in neural machine\ntranslation (NMT) is to break word forms into subword units, so that the\noverall vocabulary size of these units fits the practical limits given by the\nNMT model and GPU memory capacity. In this paper, we compare two common but\nlinguistically uninformed methods of subword construction (BPE and STE, the\nmethod implemented in Tensor2Tensor toolkit) and two linguistically-motivated\nmethods: Morfessor and one novel method, based on a derivational dictionary.\nOur experiments with German-to-Czech translation, both morphologically rich,\ndocument that so far, the non-motivated methods perform better. Furthermore, we\niden- tify a critical difference between BPE and STE and show a simple pre-\nprocessing step for BPE that considerably increases translation quality as\nevaluated by automatic measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05499v1",
            "date": "2018-06-14 12:28:43+00:00",
            "title": "Aspect Sentiment Model for Micro Reviews",
            "authors": [
                "Reinald Kim Amplayo",
                "Seung-won Hwang"
            ],
            "abstract": "This paper aims at an aspect sentiment model for aspect-based sentiment\nanalysis (ABSA) focused on micro reviews. This task is important in order to\nunderstand short reviews majority of the users write, while existing topic\nmodels are targeted for expert-level long reviews with sufficient co-occurrence\npatterns to observe. Current methods on aggregating micro reviews using\nmetadata information may not be effective as well due to metadata absence,\ntopical heterogeneity, and cold start problems. To this end, we propose a model\ncalled Micro Aspect Sentiment Model (MicroASM). MicroASM is based on the\nobservation that short reviews 1) are viewed with sentiment-aspect word pairs\nas building blocks of information, and 2) can be clustered into larger reviews.\nWhen compared to the current state-of-the-art aspect sentiment models,\nexperiments show that our model provides better performance on aspect-level\ntasks such as aspect term extraction and document-level tasks such as sentiment\nclassification.",
            "journal ref": "Data Mining (ICDM), 2017 IEEE International Conference on",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05504v1",
            "date": "2018-06-14 12:41:50+00:00",
            "title": "Entity Commonsense Representation for Neural Abstractive Summarization",
            "authors": [
                "Reinald Kim Amplayo",
                "Seonjae Lim",
                "Seung-won Hwang"
            ],
            "abstract": "A major proportion of a text summary includes important entities found in the\noriginal text. These entities build up the topic of the summary. Moreover, they\nhold commonsense information once they are linked to a knowledge base. Based on\nthese observations, this paper investigates the usage of linked entities to\nguide the decoder of a neural text summarizer to generate concise and better\nsummaries. To this end, we leverage on an off-the-shelf entity linking system\n(ELS) to extract linked entities and propose Entity2Topic (E2T), a module\neasily attachable to a sequence-to-sequence model that transforms a list of\nentities into a vector representation of the topic of the summary. Current\navailable ELS's are still not sufficiently effective, possibly introducing\nunresolved ambiguities and irrelevant entities. We resolve the imperfections of\nthe ELS by (a) encoding entities with selective disambiguation, and (b) pooling\nentity vectors using firm attention. By applying E2T to a simple\nsequence-to-sequence model with attention mechanism as base model, we see\nsignificant improvements of the performance in the Gigaword (sentence to title)\nand CNN (long document to multi-sentence highlights) summarization datasets by\nat least 2 ROUGE points.",
            "journal ref": "Proceedings of the 2018 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, Volume 1 (Long Papers)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05507v1",
            "date": "2018-06-14 12:48:31+00:00",
            "title": "Cold-Start Aware User and Product Attention for Sentiment Classification",
            "authors": [
                "Reinald Kim Amplayo",
                "Jihyeok Kim",
                "Sua Sung",
                "Seung-won Hwang"
            ],
            "abstract": "The use of user/product information in sentiment analysis is important,\nespecially for cold-start users/products, whose number of reviews are very\nlimited. However, current models do not deal with the cold-start problem which\nis typical in review websites. In this paper, we present Hybrid Contextualized\nSentiment Classifier (HCSC), which contains two modules: (1) a fast word\nencoder that returns word vectors embedded with short and long range dependency\nfeatures; and (2) Cold-Start Aware Attention (CSAA), an attention mechanism\nthat considers the existence of cold-start problem when attentively pooling the\nencoded word vectors. HCSC introduces shared vectors that are constructed from\nsimilar users/products, and are used when the original distinct vectors do not\nhave sufficient information (i.e. cold-start). This is decided by a\nfrequency-guided selective gate vector. Our experiments show that in terms of\nRMSE, HCSC performs significantly better when compared with on famous datasets,\ndespite having less complexity, and thus can be trained much faster. More\nimportantly, our model performs significantly better than previous models when\nthe training data is sparse and has cold-start problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05513v1",
            "date": "2018-06-14 12:57:13+00:00",
            "title": "Humor Detection in English-Hindi Code-Mixed Social Media Content : Corpus and Baseline System",
            "authors": [
                "Ankush Khandelwal",
                "Sahil Swami",
                "Syed S. Akhtar",
                "Manish Shrivastava"
            ],
            "abstract": "The tremendous amount of user generated data through social networking sites\nled to the gaining popularity of automatic text classification in the field of\ncomputational linguistics over the past decade. Within this domain, one problem\nthat has drawn the attention of many researchers is automatic humor detection\nin texts. In depth semantic understanding of the text is required to detect\nhumor which makes the problem difficult to automate. With increase in the\nnumber of social media users, many multilingual speakers often interchange\nbetween languages while posting on social media which is called code-mixing. It\nintroduces some challenges in the field of linguistic analysis of social media\ncontent (Barman et al., 2014), like spelling variations and non-grammatical\nstructures in a sentence. Past researches include detecting puns in texts (Kao\net al., 2016) and humor in one-lines (Mihalcea et al., 2010) in a single\nlanguage, but with the tremendous amount of code-mixed data available online,\nthere is a need to develop techniques which detects humor in code-mixed tweets.\nIn this paper, we analyze the task of humor detection in texts and describe a\nfreely available corpus containing English-Hindi code-mixed tweets annotated\nwith humorous(H) or non-humorous(N) tags. We also tagged the words in the\ntweets with Language tags (English/Hindi/Others). Moreover, we describe the\nexperiments carried out on the corpus and provide a baseline classification\nsystem which distinguishes between humorous and non-humorous texts.",
            "journal ref": "Khandelwa, Ankush, et. al , \"Humor Detection in English-Hindi\n  Code-Mixed Social Media Content : Corpus and Baseline System\". Proceedings of\n  the Eleventh International Conference on Language Resources and Evaluation\n  (LREC 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05516v1",
            "date": "2018-06-14 13:01:04+00:00",
            "title": "Translations as Additional Contexts for Sentence Classification",
            "authors": [
                "Reinald Kim Amplayo",
                "Kyungjae Lee",
                "Jinyeong Yeo",
                "Seung-won Hwang"
            ],
            "abstract": "In sentence classification tasks, additional contexts, such as the\nneighboring sentences, may improve the accuracy of the classifier. However,\nsuch contexts are domain-dependent and thus cannot be used for another\nclassification task with an inappropriate domain. In contrast, we propose the\nuse of translated sentences as context that is always available regardless of\nthe domain. We find that naive feature expansion of translations gains only\nmarginal improvements and may decrease the performance of the classifier, due\nto possible inaccurate translations thus producing noisy sentence vectors. To\nthis end, we present multiple context fixing attachment (MCFA), a series of\nmodules attached to multiple sentence vectors to fix the noise in the vectors\nusing the other sentence vectors as context. We show that our method performs\ncompetitively compared to previous models, achieving best classification\nperformance on multiple data sets. We are the first to use translations as\ndomain-free contexts for sentence classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05599v1",
            "date": "2018-06-14 15:07:46+00:00",
            "title": "A Survey on Open Information Extraction",
            "authors": [
                "Christina Niklaus",
                "Matthias Cetto",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We provide a detailed overview of the various approaches that were proposed\nto date to solve the task of Open Information Extraction. We present the major\nchallenges that such systems face, show the evolution of the suggested\napproaches over time and depict the specific issues they address. In addition,\nwe provide a critique of the commonly applied evaluation procedures for\nassessing the performance of Open IE systems and highlight some directions for\nfuture work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05600v1",
            "date": "2018-06-14 15:08:22+00:00",
            "title": "Gender Prediction in English-Hindi Code-Mixed Social Media Content : Corpus and Baseline System",
            "authors": [
                "Ankush Khandelwal",
                "Sahil Swami",
                "Syed Sarfaraz Akhtar",
                "Manish Shrivastava"
            ],
            "abstract": "The rapid expansion in the usage of social media networking sites leads to a\nhuge amount of unprocessed user generated data which can be used for text\nmining. Author profiling is the problem of automatically determining profiling\naspects like the author's gender and age group through a text is gaining much\npopularity in computational linguistics. Most of the past research in author\nprofiling is concentrated on English texts \\cite{1,2}. However many users often\nchange the language while posting on social media which is called code-mixing,\nand it develops some challenges in the field of text classification and author\nprofiling like variations in spelling, non-grammatical structure and\ntransliteration \\cite{3}. There are very few English-Hindi code-mixed annotated\ndatasets of social media content present online \\cite{4}. In this paper, we\nanalyze the task of author's gender prediction in code-mixed content and\npresent a corpus of English-Hindi texts collected from Twitter which is\nannotated with author's gender. We also explore language identification of\nevery word in this corpus. We present a supervised classification baseline\nsystem which uses various machine learning algorithms to identify the gender of\nan author using a text, based on character and word level features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05626v2",
            "date": "2018-06-14 16:10:57+00:00",
            "title": "NCRF++: An Open-source Neural Sequence Labeling Toolkit",
            "authors": [
                "Jie Yang",
                "Yue Zhang"
            ],
            "abstract": "This paper describes NCRF++, a toolkit for neural sequence labeling. NCRF++\nis designed for quick implementation of different neural sequence labeling\nmodels with a CRF inference layer. It provides users with an inference for\nbuilding the custom model structure through configuration file with flexible\nneural feature design and utilization. Built on PyTorch, the core operations\nare calculated in batch, making the toolkit efficient with the acceleration of\nGPU. It also includes the implementations of most state-of-the-art neural\nsequence labeling models such as LSTM-CRF, facilitating reproducing and\nrefinement on those methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05655v1",
            "date": "2018-06-14 17:25:43+00:00",
            "title": "Abstract Meaning Representation for Multi-Document Summarization",
            "authors": [
                "Kexin Liao",
                "Logan Lebanoff",
                "Fei Liu"
            ],
            "abstract": "Generating an abstract from a collection of documents is a desirable\ncapability for many real-world applications. However, abstractive approaches to\nmulti-document summarization have not been thoroughly investigated. This paper\nstudies the feasibility of using Abstract Meaning Representation (AMR), a\nsemantic representation of natural language grounded in linguistic theory, as a\nform of content representation. Our approach condenses source documents to a\nset of summary graphs following the AMR formalism. The summary graphs are then\ntransformed to a set of summary sentences in a surface realization step. The\nframework is fully data-driven and flexible. Each component can be optimized\nindependently using small-scale, in-domain training data. We perform\nexperiments on benchmark summarization datasets and report promising results.\nWe also describe opportunities and challenges for advancing this line of\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05658v2",
            "date": "2018-06-14 17:31:18+00:00",
            "title": "Structure-Infused Copy Mechanisms for Abstractive Summarization",
            "authors": [
                "Kaiqiang Song",
                "Lin Zhao",
                "Fei Liu"
            ],
            "abstract": "Seq2seq learning has produced promising results on summarization. However, in\nmany cases, system summaries still struggle to keep the meaning of the original\nintact. They may miss out important words or relations that play critical roles\nin the syntactic structure of source sentences. In this paper, we present\nstructure-infused copy mechanisms to facilitate copying important words and\nrelations from the source sentence to summary sentence. The approach naturally\ncombines source dependency structure with the copy mechanism of an abstractive\nsentence summarizer. Experimental results demonstrate the effectiveness of\nincorporating source-side syntactic information in the system, and our proposed\napproach compares favorably to state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05847v1",
            "date": "2018-06-15 08:10:12+00:00",
            "title": "Semantic Variation in Online Communities of Practice",
            "authors": [
                "Marco Del Tredici",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "We introduce a framework for quantifying semantic variation of common words\nin Communities of Practice and in sets of topic-related communities. We show\nthat while some meaning shifts are shared across related communities, others\nare community-specific, and therefore independent from the discussed topic. We\npropose such findings as evidence in favour of sociolinguistic theories of\nsocially-driven semantic variation. Results are evaluated using an independent\nlanguage modelling task. Furthermore, we investigate extralinguistic features\nand show that factors such as prominence and dissemination of words are related\nto semantic variation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05900v1",
            "date": "2018-06-15 10:55:39+00:00",
            "title": "An Empirical Analysis of the Correlation of Syntax and Prosody",
            "authors": [
                "Arne K\u00f6hn",
                "Timo Baumann",
                "Oskar D\u00f6rfler"
            ],
            "abstract": "The relation of syntax and prosody (the syntax--prosody interface) has been\nan active area of research, mostly in linguistics and typically studied under\ncontrolled conditions. More recently, prosody has also been successfully used\nin the data-based training of syntax parsers. However, there is a gap between\nthe controlled and detailed study of the individual effects between syntax and\nprosody and the large-scale application of prosody in syntactic parsing with\nonly a shallow analysis of the respective influences. In this paper, we close\nthe gap by investigating the significance of correlations of prosodic\nrealization with specific syntactic functions using linear mixed effects models\nin a very large corpus of read-out German encyclopedic texts. Using this\ncorpus, we are able to analyze prosodic structuring performed by a diverse set\nof speakers while they try to optimize factual content delivery. After\nnormalization by speaker, we obtain significant effects, e.g. confirming that\nthe subject function, as compared to the object function, has a positive effect\non pitch and duration of a word, but a negative effect on loudness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05947v1",
            "date": "2018-06-15 13:27:32+00:00",
            "title": "Discovering User Groups for Natural Language Generation",
            "authors": [
                "Nikos Engonopoulos",
                "Christoph Teichmann",
                "Alexander Koller"
            ],
            "abstract": "We present a model which predicts how individual users of a dialog system\nunderstand and produce utterances based on user groups. In contrast to previous\nwork, these user groups are not specified beforehand, but learned in training.\nWe evaluate on two referring expression (RE) generation tasks; our experiments\nshow that our model can identify user groups and learn how to most effectively\ntalk to them, and can dynamically assign unseen users to the correct groups as\nthey interact with the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05997v1",
            "date": "2018-06-15 14:28:50+00:00",
            "title": "A Dataset for Building Code-Mixed Goal Oriented Conversation Systems",
            "authors": [
                "Suman Banerjee",
                "Nikita Moghe",
                "Siddhartha Arora",
                "Mitesh M. Khapra"
            ],
            "abstract": "There is an increasing demand for goal-oriented conversation systems which\ncan assist users in various day-to-day activities such as booking tickets,\nrestaurant reservations, shopping, etc. Most of the existing datasets for\nbuilding such conversation systems focus on monolingual conversations and there\nis hardly any work on multilingual and/or code-mixed conversations. Such\ndatasets and systems thus do not cater to the multilingual regions of the\nworld, such as India, where it is very common for people to speak more than one\nlanguage and seamlessly switch between them resulting in code-mixed\nconversations. For example, a Hindi speaking user looking to book a restaurant\nwould typically ask, \"Kya tum is restaurant mein ek table book karne mein meri\nhelp karoge?\" (\"Can you help me in booking a table at this restaurant?\"). To\nfacilitate the development of such code-mixed conversation models, we build a\ngoal-oriented dialog dataset containing code-mixed conversations. Specifically,\nwe take the text from the DSTC2 restaurant reservation dataset and create\ncode-mixed versions of it in Hindi-English, Bengali-English, Gujarati-English\nand Tamil-English. We also establish initial baselines on this dataset using\nexisting state of the art models. This dataset along with our baseline\nimplementations is made publicly available for research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06187v2",
            "date": "2018-06-16 05:17:32+00:00",
            "title": "Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents",
            "authors": [
                "Wenhan Xiong",
                "Xiaoxiao Guo",
                "Mo Yu",
                "Shiyu Chang",
                "Bowen Zhou",
                "William Yang Wang"
            ],
            "abstract": "We investigate the task of learning to follow natural language instructions\nby jointly reasoning with visual observations and language inputs. In contrast\nto existing methods which start with learning from demonstrations (LfD) and\nthen use reinforcement learning (RL) to fine-tune the model parameters, we\npropose a novel policy optimization algorithm which dynamically schedules\ndemonstration learning and RL. The proposed training paradigm provides\nefficient exploration and better generalization beyond existing methods.\nComparing to existing ensemble models, the best single model based on our\nproposed method tremendously decreases the execution error by over 50% on a\nblock-world environment. To further illustrate the exploration strategy of our\nRL algorithm, We also include systematic studies on the evolution of policy\nentropy during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06200v1",
            "date": "2018-06-16 07:18:22+00:00",
            "title": "Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition",
            "authors": [
                "Pengcheng Guo",
                "Haihua Xu",
                "Lei Xie",
                "Eng Siong Chng"
            ],
            "abstract": "In this paper, we present our overall efforts to improve the performance of a\ncode-switching speech recognition system using semi-supervised training methods\nfrom lexicon learning to acoustic modeling, on the South East Asian\nMandarin-English (SEAME) data. We first investigate semi-supervised lexicon\nlearning approach to adapt the canonical lexicon, which is meant to alleviate\nthe heavily accented pronunciation issue within the code-switching conversation\nof the local area. As a result, the learned lexicon yields improved\nperformance. Furthermore, we attempt to use semi-supervised training to deal\nwith those transcriptions that are highly mismatched between human transcribers\nand ASR system. Specifically, we conduct semi-supervised training assuming\nthose poorly transcribed data as unsupervised data. We found the\nsemi-supervised acoustic modeling can lead to improved results. Finally, to\nmake up for the limitation of the conventional n-gram language models due to\ndata sparsity issue, we perform lattice rescoring using neural network language\nmodels, and significant WER reduction is obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06219v3",
            "date": "2018-06-16 10:47:41+00:00",
            "title": "GILE: A Generalized Input-Label Embedding for Text Classification",
            "authors": [
                "Nikolaos Pappas",
                "James Henderson"
            ],
            "abstract": "Neural text classification models typically treat output labels as\ncategorical variables which lack description and semantics. This forces their\nparametrization to be dependent on the label set size, and, hence, they are\nunable to scale to large label sets and generalize to unseen ones. Existing\njoint input-label text models overcome these issues by exploiting label\ndescriptions, but they are unable to capture complex label relationships, have\nrigid parametrization, and their gains on unseen labels happen often at the\nexpense of weak performance on the labels seen during training. In this paper,\nwe propose a new input-label model which generalizes over previous such models,\naddresses their limitations and does not compromise performance on seen labels.\nThe model consists of a joint non-linear input-label embedding with\ncontrollable capacity and a joint-space-dependent classification unit which is\ntrained with cross-entropy loss to optimize classification performance. We\nevaluate models on full-resource and low- or zero-resource text classification\nof multilingual news and biomedical text with a large label set. Our model\noutperforms monolingual and multilingual models which do not leverage label\nsemantics and previous joint input-label space models in both scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06259v1",
            "date": "2018-06-16 16:07:49+00:00",
            "title": "Evaluation of sentence embeddings in downstream and linguistic probing tasks",
            "authors": [
                "Christian S. Perone",
                "Roberto Silveira",
                "Thomas S. Paula"
            ],
            "abstract": "Despite the fast developmental pace of new sentence embedding methods, it is\nstill challenging to find comprehensive evaluations of these different\ntechniques. In the past years, we saw significant improvements in the field of\nsentence embeddings and especially towards the development of universal\nsentence encoders that could provide inductive transfer to a wide variety of\ndownstream tasks. In this work, we perform a comprehensive evaluation of recent\nmethods using a wide variety of downstream and linguistic feature probing\ntasks. We show that a simple approach using bag-of-words with a recently\nintroduced language model for deep context-dependent word embeddings proved to\nyield better results in many tasks when compared to sentence encoders trained\non entailment datasets. We also show, however, that we are still far away from\na universal encoder that can perform consistently across several downstream\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06571v1",
            "date": "2018-06-18 09:31:38+00:00",
            "title": "SubGram: Extending Skip-gram Word Representation with Substrings",
            "authors": [
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Skip-gram (word2vec) is a recent method for creating vector representations\nof words (\"distributed word representations\") using a neural network. The\nrepresentation gained popularity in various areas of natural language\nprocessing, because it seems to capture syntactic and semantic information\nabout words without any explicit supervision in this respect. We propose\nSubGram, a refinement of the Skip-gram model to consider also the word\nstructure during the training process, achieving large gains on the Skip-gram\noriginal test set.",
            "journal ref": "Text, Speech, and Dialogue: 19th International Conference, TSD\n  2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06626v1",
            "date": "2018-06-18 12:21:18+00:00",
            "title": "On Enhancing Speech Emotion Recognition using Generative Adversarial Networks",
            "authors": [
                "Saurabh Sahu",
                "Rahul Gupta",
                "Carol Espy-Wilson"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have gained a lot of attention from\nmachine learning community due to their ability to learn and mimic an input\ndata distribution. GANs consist of a discriminator and a generator working in\ntandem playing a min-max game to learn a target underlying data distribution;\nwhen fed with data-points sampled from a simpler distribution (like uniform or\nGaussian distribution). Once trained, they allow synthetic generation of\nexamples sampled from the target distribution. We investigate the application\nof GANs to generate synthetic feature vectors used for speech emotion\nrecognition. Specifically, we investigate two set ups: (i) a vanilla GAN that\nlearns the distribution of a lower dimensional representation of the actual\nhigher dimensional feature vector and, (ii) a conditional GAN that learns the\ndistribution of the higher dimensional feature vectors conditioned on the\nlabels or the emotional class to which it belongs. As a potential practical\napplication of these synthetically generated samples, we measure any\nimprovement in a classifier's performance when the synthetic data is used along\nwith real data for training. We perform cross-validation analyses followed by a\ncross-corpus study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06874v1",
            "date": "2018-06-18 18:13:21+00:00",
            "title": "Combining Word Feature Vector Method with the Convolutional Neural Network for Slot Filling in Spoken Language Understanding",
            "authors": [
                "Ruixi Lin"
            ],
            "abstract": "Slot filling is an important problem in Spoken Language Understanding (SLU)\nand Natural Language Processing (NLP), which involves identifying a user's\nintent and assigning a semantic concept to each word in a sentence. This paper\npresents a word feature vector method and combines it into the convolutional\nneural network (CNN). We consider 18 word features and each word feature is\nconstructed by merging similar word labels. By introducing the concept of\nexternal library, we propose a feature set approach that is beneficial for\nbuilding the relationship between a word from the training dataset and the\nfeature. Computational results are reported using the ATIS dataset and\ncomparisons with traditional CNN as well as bi-directional sequential CNN are\nalso presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06957v2",
            "date": "2018-06-18 21:18:18+00:00",
            "title": "A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation",
            "authors": [
                "Surafel M. Lakew",
                "Mauro Cettolo",
                "Marcello Federico"
            ],
            "abstract": "Recently, neural machine translation (NMT) has been extended to\nmultilinguality, that is to handle more than one translation direction with a\nsingle system. Multilingual NMT showed competitive performance against pure\nbilingual systems. Notably, in low-resource settings, it proved to work\neffectively and efficiently, thanks to shared representation space that is\nforced across languages and induces a sort of transfer-learning. Furthermore,\nmultilingual NMT enables so-called zero-shot inference across language pairs\nnever seen at training time. Despite the increasing interest in this framework,\nan in-depth analysis of what a multilingual NMT model is capable of and what it\nis not is still missing. Motivated by this, our work (i) provides a\nquantitative and comparative analysis of the translations produced by\nbilingual, multilingual and zero-shot systems; (ii) investigates the\ntranslation quality of two of the currently dominant neural architectures in\nMT, which are the Recurrent and the Transformer ones; and (iii) quantitatively\nexplores how the closeness between languages influences the zero-shot\ntranslation. Our analysis leverages multiple professional post-edits of\nautomatic translations by several different systems and focuses both on\nautomatic standard metrics (BLEU and TER) and on widely used error categories,\nwhich are lexical, morphology, and word order errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07039v2",
            "date": "2018-06-19 05:03:23+00:00",
            "title": "EmotionX-DLC: Self-Attentive BiLSTM for Detecting Sequential Emotions in Dialogue",
            "authors": [
                "Linkai Luo",
                "Haiqing Yang",
                "Francis Y. L. Chin"
            ],
            "abstract": "In this paper, we propose a self-attentive bidirectional long short-term\nmemory (SA-BiLSTM) network to predict multiple emotions for the EmotionX\nchallenge. The BiLSTM exhibits the power of modeling the word dependencies, and\nextracting the most relevant features for emotion classification. Building on\ntop of BiLSTM, the self-attentive network can model the contextual dependencies\nbetween utterances which are helpful for classifying the ambiguous emotions. We\nachieve 59.6 and 55.0 unweighted accuracy scores in the \\textit{Friends} and\nthe \\textit{EmotionPush} test sets, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07042v4",
            "date": "2018-06-19 05:13:34+00:00",
            "title": "Response Generation by Context-aware Prototype Editing",
            "authors": [
                "Yu Wu",
                "Furu Wei",
                "Shaohan Huang",
                "Yunli Wang",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "Open domain response generation has achieved remarkable progress in recent\nyears, but sometimes yields short and uninformative responses. We propose a new\nparadigm for response generation, that is response generation by editing, which\nsignificantly increases the diversity and informativeness of the generation\nresults. Our assumption is that a plausible response can be generated by\nslightly revising an existing response prototype. The prototype is retrieved\nfrom a pre-defined index and provides a good start-point for generation because\nit is grammatical and informative. We design a response editing model, where an\nedit vector is formed by considering differences between a prototype context\nand a current context, and then the edit vector is fed to a decoder to revise\nthe prototype response for the current context. Experiment results on a large\nscale dataset demonstrate that the response editing model outperforms\ngenerative and retrieval-based models on various aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07169v1",
            "date": "2018-06-19 11:56:22+00:00",
            "title": "Learning from Chunk-based Feedback in Neural Machine Translation",
            "authors": [
                "Pavel Petrushkov",
                "Shahram Khadivi",
                "Evgeny Matusov"
            ],
            "abstract": "We empirically investigate learning from partial feedback in neural machine\ntranslation (NMT), when partial feedback is collected by asking users to\nhighlight a correct chunk of a translation. We propose a simple and effective\nway of utilizing such feedback in NMT training. We demonstrate how the common\nmachine translation problem of domain mismatch between training and deployment\ncan be reduced solely based on chunk-level user feedback. We conduct a series\nof simulation experiments to test the effectiveness of the proposed method. Our\nresults show that chunk-level feedback outperforms sentence based feedback by\nup to 2.61% BLEU absolute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07186v1",
            "date": "2018-06-19 12:39:22+00:00",
            "title": "Recurrent DNNs and its Ensembles on the TIMIT Phone Recognition Task",
            "authors": [
                "Jan Vanek",
                "Josef Michalek",
                "Josef Psutka"
            ],
            "abstract": "In this paper, we have investigated recurrent deep neural networks (DNNs) in\ncombination with regularization techniques as dropout, zoneout, and\nregularization post-layer. As a benchmark, we chose the TIMIT phone recognition\ntask due to its popularity and broad availability in the community. It also\nsimulates a low-resource scenario that is helpful in minor languages. Also, we\nprefer the phone recognition task because it is much more sensitive to an\nacoustic model quality than a large vocabulary continuous speech recognition\ntask. In recent years, recurrent DNNs pushed the error rates in automatic\nspeech recognition down. But, there was no clear winner in proposed\narchitectures. The dropout was used as the regularization technique in most\ncases, but combination with other regularization techniques together with model\nensembles was omitted. However, just an ensemble of recurrent DNNs performed\nbest and achieved an average phone error rate from 10 experiments 14.84 %\n(minimum 14.69 %) on core test set that is slightly lower then the\nbest-published PER to date, according to our knowledge. Finally, in contrast of\nthe most papers, we published the open-source scripts to easily replicate the\nresults and to help continue the development.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07687v2",
            "date": "2018-06-20 12:13:53+00:00",
            "title": "Automated Fact Checking: Task formulations, methods and future directions",
            "authors": [
                "James Thorne",
                "Andreas Vlachos"
            ],
            "abstract": "The recently increased focus on misinformation has stimulated research in\nfact checking, the task of assessing the truthfulness of a claim. Research in\nautomating this task has been conducted in a variety of disciplines including\nnatural language processing, machine learning, knowledge representation,\ndatabases, and journalism. While there has been substantial progress, relevant\npapers and articles have been published in research communities that are often\nunaware of each other and use inconsistent terminology, thus impeding\nunderstanding and further progress. In this paper we survey automated fact\nchecking research stemming from natural language processing and related\ndisciplines, unifying the task formulations and methodologies across papers and\nauthors. Furthermore, we highlight the use of evidence as an important\ndistinguishing factor among them cutting across task formulations and methods.\nWe conclude with proposing avenues for future NLP research on automated fact\nchecking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07699v1",
            "date": "2018-06-20 12:56:54+00:00",
            "title": "Word Tagging with Foundational Ontology Classes: Extending the WordNet-DOLCE Mapping to Verbs",
            "authors": [
                "Vivian S. Silva",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "Semantic annotation is fundamental to deal with large-scale lexical\ninformation, mapping the information to an enumerable set of categories over\nwhich rules and algorithms can be applied, and foundational ontology classes\ncan be used as a formal set of categories for such tasks. A previous alignment\nbetween WordNet noun synsets and DOLCE provided a starting point for\nontology-based annotation, but in NLP tasks verbs are also of substantial\nimportance. This work presents an extension to the WordNet-DOLCE noun mapping,\naligning verbs according to their links to nouns denoting perdurants,\ntransferring to the verb the DOLCE class assigned to the noun that best\nrepresents that verb's occurrence. To evaluate the usefulness of this resource,\nwe implemented a foundational ontology-based semantic annotation framework,\nthat assigns a high-level foundational category to each word or phrase in a\ntext, and compared it to a similar annotation tool, obtaining an increase of\n9.05% in accuracy.",
            "journal ref": "Proceedings of the 20th International Conference on Knowledge\n  Engineering and Knowledge Management, Bologna, Italy, 2016, pp 593-605",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07711v1",
            "date": "2018-06-20 13:14:12+00:00",
            "title": "Categorization of Semantic Roles for Dictionary Definitions",
            "authors": [
                "Vivian S. Silva",
                "Siegfried Handschuh",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "Understanding the semantic relationships between terms is a fundamental task\nin natural language processing applications. While structured resources that\ncan express those relationships in a formal way, such as ontologies, are still\nscarce, a large number of linguistic resources gathering dictionary definitions\nis becoming available, but understanding the semantic structure of natural\nlanguage definitions is fundamental to make them useful in semantic\ninterpretation tasks. Based on an analysis of a subset of WordNet's glosses, we\npropose a set of semantic roles that compose the semantic structure of a\ndictionary definition, and show how they are related to the definition's\nsyntactic configuration, identifying patterns that can be used in the\ndevelopment of information extraction frameworks and semantic models.",
            "journal ref": "Proceedings of the 5th Workshop on Cognitive Aspects of the\n  Lexicon (CogALex-V), Osaka, Japan, 2016, pp 176-184",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07721v1",
            "date": "2018-06-20 13:30:51+00:00",
            "title": "Semantic Relation Classification: Task Formalisation and Refinement",
            "authors": [
                "Vivian S. Silva",
                "Manuela H\u00fcrliman",
                "Brian Davis",
                "Siegfried Handschuh",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "The identification of semantic relations between terms within texts is a\nfundamental task in Natural Language Processing which can support applications\nrequiring a lightweight semantic interpretation model. Currently, semantic\nrelation classification concentrates on relations which are evaluated over\nopen-domain data. This work provides a critique on the set of abstract\nrelations used for semantic relation classification with regard to their\nability to express relationships between terms which are found in a\ndomain-specific corpora. Based on this analysis, this work proposes an\nalternative semantic relation model based on reusing and extending the set of\nabstract relations present in the DOLCE ontology. The resulting set of\nrelations is well grounded, allows to capture a wide range of relations and\ncould thus be used as a foundation for automatic classification of semantic\nrelations.",
            "journal ref": "Proceedings of the 5th Workshop on Cognitive Aspects of the\n  Lexicon, Osaka, Japan, 2016, pp 30-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07731v1",
            "date": "2018-06-20 13:50:46+00:00",
            "title": "Building a Knowledge Graph from Natural Language Definitions for Interpretable Text Entailment Recognition",
            "authors": [
                "Vivian S. Silva",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "Natural language definitions of terms can serve as a rich source of\nknowledge, but structuring them into a comprehensible semantic model is\nessential to enable them to be used in semantic interpretation tasks. We\npropose a method and provide a set of tools for automatically building a graph\nworld knowledge base from natural language definitions. Adopting a conceptual\nmodel composed of a set of semantic roles for dictionary definitions, we\ntrained a classifier for automatically labeling definitions, preparing the data\nto be later converted to a graph representation. WordNetGraph, a knowledge\ngraph built out of noun and verb WordNet definitions according to this\nmethodology, was successfully used in an interpretable text entailment\nrecognition approach which uses paths in this graph to provide clear\njustifications for entailment decisions.",
            "journal ref": "Proceedings of the Eleventh International Conference on Language\n  Resources and Evaluation, Miyazaki, Japan, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07787v1",
            "date": "2018-06-20 15:14:10+00:00",
            "title": "Opinion Dynamics Modeling for Movie Review Transcripts Classification with Hidden Conditional Random Fields",
            "authors": [
                "Valentin Barriere",
                "Chlo\u00e9 Clavel",
                "Slim Essid"
            ],
            "abstract": "In this paper, the main goal is to detect a movie reviewer's opinion using\nhidden conditional random fields. This model allows us to capture the dynamics\nof the reviewer's opinion in the transcripts of long unsegmented audio reviews\nthat are analyzed by our system. High level linguistic features are computed at\nthe level of inter-pausal segments. The features include syntactic features, a\nstatistical word embedding model and subjectivity lexicons. The proposed system\nis evaluated on the ICT-MMMO corpus. We obtain a F1-score of 82\\%, which is\nbetter than logistic regression and recurrent neural network approaches. We\nalso offer a discussion that sheds some light on the capacity of our system to\nadapt the word embedding model learned from general written texts data to\nspoken movie reviews and thus model the dynamics of the opinion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07914v1",
            "date": "2018-06-20 18:17:40+00:00",
            "title": "Multi-Layer Ensembling Techniques for Multilingual Intent Classification",
            "authors": [
                "Charles Costello",
                "Ruixi Lin",
                "Vishwas Mruthyunjaya",
                "Bettina Bolla",
                "Charles Jankowski"
            ],
            "abstract": "In this paper we determine how multi-layer ensembling improves performance on\nmultilingual intent classification. We develop a novel multi-layer ensembling\napproach that ensembles both different model initializations and different\nmodel architectures. We also introduce a new banking domain dataset and compare\nresults against the standard ATIS dataset and the Chinese SMP2017 dataset to\ndetermine ensembling performance in multilingual and multi-domain contexts. We\nrun ensemble experiments across all three datasets, and conclude that\nensembling provides significant performance increases, and that multi-layer\nensembling is a no-risk way to improve performance on intent classification. We\nalso find that a diverse ensemble of simple models can reach perform comparable\nto much more sophisticated state-of-the-art models. Our best F 1 scores on\nATIS, Banking, and SMP are 97.54%, 91.79%, and 93.55% respectively, which\ncompare well with the state-of-the-art on ATIS and best submission to the\nSMP2017 competition. The total ensembling performance increases we achieve are\n0.23%, 1.96%, and 4.04% F 1 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07916v1",
            "date": "2018-06-20 18:18:52+00:00",
            "title": "RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses",
            "authors": [
                "Sean MacAvaney",
                "Bart Desmet",
                "Arman Cohan",
                "Luca Soldaini",
                "Andrew Yates",
                "Ayah Zirikly",
                "Nazli Goharian"
            ],
            "abstract": "Self-reported diagnosis statements have been widely employed in studying\nlanguage related to mental health in social media. However, existing research\nhas largely ignored the temporality of mental health diagnoses. In this work,\nwe introduce RSDD-Time: a new dataset of 598 manually annotated self-reported\ndepression diagnosis posts from Reddit that include temporal information about\nthe diagnosis. Annotations include whether a mental health condition is present\nand how recently the diagnosis happened. Furthermore, we include exact temporal\nspans that relate to the date of diagnosis. This information is valuable for\nvarious computational methods to examine mental health through social media\nbecause one's mental health state is not static. We also test several baseline\nclassification and extraction approaches, which suggest that extracting\ntemporal information from self-reported diagnosis statements is challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07976v1",
            "date": "2018-06-20 20:29:24+00:00",
            "title": "Ontology Alignment in the Biomedical Domain Using Entity Definitions and Context",
            "authors": [
                "Lucy Lu Wang",
                "Chandra Bhagavatula",
                "Mark Neumann",
                "Kyle Lo",
                "Chris Wilhelm",
                "Waleed Ammar"
            ],
            "abstract": "Ontology alignment is the task of identifying semantically equivalent\nentities from two given ontologies. Different ontologies have different\nrepresentations of the same entity, resulting in a need to de-duplicate\nentities when merging ontologies. We propose a method for enriching entities in\nan ontology with external definition and context information, and use this\nadditional information for ontology alignment. We develop a neural architecture\ncapable of encoding the additional information when available, and show that\nthe addition of external data results in an F1-score of 0.69 on the Ontology\nAlignment Evaluation Initiative (OAEI) largebio SNOMED-NCI subtask, comparable\nwith the entity-level matchers in a SOTA system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07977v1",
            "date": "2018-06-20 20:31:47+00:00",
            "title": "TxPI-u: A Resource for Personality Identification of Undergraduates",
            "authors": [
                "Gabriela Ram\u00edrez-de-la-Rosa",
                "Esa\u00fa Villatoro-Tello",
                "H\u00e9ctor Jim\u00e9nez-Salazar"
            ],
            "abstract": "Resources such as labeled corpora are necessary to train automatic models\nwithin the natural language processing (NLP) field. Historically, a large\nnumber of resources regarding a broad number of problems are available mostly\nin English. One of such problems is known as Personality Identification where\nbased on a psychological model (e.g. The Big Five Model), the goal is to find\nthe traits of a subject's personality given, for instance, a text written by\nthe same subject. In this paper we introduce a new corpus in Spanish called\nTexts for Personality Identification (TxPI). This corpus will help to develop\nmodels to automatically assign a personality trait to an author of a text\ndocument. Our corpus, TxPI-u, contains information of 416 Mexican undergraduate\nstudents with some demographics information such as, age, gender, and the\nacademic program they are enrolled. Finally, as an additional contribution, we\npresent a set of baselines to provide a comparison scheme for further research.",
            "journal ref": "Journal of Intelligent & Fuzzy Systems, vol. 34, no. 5, pp.\n  2991-3001, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07999v1",
            "date": "2018-06-20 21:46:49+00:00",
            "title": "A Supervised Approach To The Interpretation Of Imperative To-Do Lists",
            "authors": [
                "Paul Landes",
                "Barbara Di Eugenio"
            ],
            "abstract": "To-do lists are a popular medium for personal information management. As\nto-do tasks are increasingly tracked in electronic form with mobile and desktop\norganizers, so does the potential for software support for the corresponding\ntasks by means of intelligent agents. While there has been work in the area of\npersonal assistants for to-do tasks, no work has focused on classifying user\nintention and information extraction as we do. We show that our methods perform\nwell across two corpora that span sub-domains, one of which we released.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08009v1",
            "date": "2018-06-20 22:09:50+00:00",
            "title": "Injecting Relational Structural Representation in Neural Networks for Question Similarity",
            "authors": [
                "Antonio Uva",
                "Daniele Bonadiman",
                "Alessandro Moschitti"
            ],
            "abstract": "Effectively using full syntactic parsing information in Neural Networks (NNs)\nto solve relational tasks, e.g., question similarity, is still an open problem.\nIn this paper, we propose to inject structural representations in NNs by (i)\nlearning an SVM model using Tree Kernels (TKs) on relatively few pairs of\nquestions (few thousands) as gold standard (GS) training data is typically\nscarce, (ii) predicting labels on a very large corpus of question pairs, and\n(iii) pre-training NNs on such large corpus. The results on Quora and SemEval\nquestion similarity datasets show that NNs trained with our approach can learn\nmore accurate models, especially after fine tuning on GS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08040v1",
            "date": "2018-06-21 01:33:30+00:00",
            "title": "An empirical study on the names of points of interest and their changes with geographic distance",
            "authors": [
                "Yingjie Hu",
                "Krzysztof Janowicz"
            ],
            "abstract": "While Points Of Interest (POIs), such as restaurants, hotels, and barber\nshops, are part of urban areas irrespective of their specific locations, the\nnames of these POIs often reveal valuable information related to local culture,\nlandmarks, influential families, figures, events, and so on. Place names have\nlong been studied by geographers, e.g., to understand their origins and\nrelations to family names. However, there is a lack of large-scale empirical\nstudies that examine the localness of place names and their changes with\ngeographic distance. In addition to enhancing our understanding of the\ncoherence of geographic regions, such empirical studies are also significant\nfor geographic information retrieval where they can inform computational models\nand improve the accuracy of place name disambiguation. In this work, we conduct\nan empirical study based on 112,071 POIs in seven US metropolitan areas\nextracted from an open Yelp dataset. We propose to adopt term frequency and\ninverse document frequency in geographic contexts to identify local terms used\nin POI names and to analyze their usages across different POI types. Our\nresults show an uneven usage of local terms across POI types, which is highly\nconsistent among different geographic regions. We also examine the decaying\neffect of POI name similarity with the increase of distance among POIs. While\nour analysis focuses on urban POI names, the presented methods can be\ngeneralized to other place types as well, such as mountain peaks and streets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08044v1",
            "date": "2018-06-21 02:14:24+00:00",
            "title": "Coherence Models for Dialogue",
            "authors": [
                "Alessandra Cervone",
                "Evgeny Stepanov",
                "Giuseppe Riccardi"
            ],
            "abstract": "Coherence across multiple turns is a major challenge for state-of-the-art\ndialogue models. Arguably the most successful approach to automatically\nlearning text coherence is the entity grid, which relies on modelling patterns\nof distribution of entities across multiple sentences of a text. Originally\napplied to the evaluation of automatic summaries and the news genre, among its\nmany extensions, this model has also been successfully used to assess dialogue\ncoherence. Nevertheless, both the original grid and its extensions do not model\nintents, a crucial aspect that has been studied widely in the literature in\nconnection to dialogue structure. We propose to augment the original grid\ndocument representation for dialogue with the intentional structure of the\nconversation. Our models outperform the original grid representation on both\ntext discrimination and insertion, the two main standard tasks for coherence\nassessment across three different dialogue datasets, confirming that intents\nplay a key role in modelling dialogue coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08077v1",
            "date": "2018-06-21 06:21:14+00:00",
            "title": "Dictionary-Guided Editing Networks for Paraphrase Generation",
            "authors": [
                "Shaohan Huang",
                "Yu Wu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "An intuitive way for a human to write paraphrase sentences is to replace\nwords or phrases in the original sentence with their corresponding synonyms and\nmake necessary changes to ensure the new sentences are fluent and grammatically\ncorrect. We propose a novel approach to modeling the process with\ndictionary-guided editing networks which effectively conduct rewriting on the\nsource sentence to generate paraphrase sentences. It jointly learns the\nselection of the appropriate word level and phrase level paraphrase pairs in\nthe context of the original sentence from an off-the-shelf dictionary as well\nas the generation of fluent natural language sentences. Specifically, the\nsystem retrieves a set of word level and phrase level araphrased pairs derived\nfrom the Paraphrase Database (PPDB) for the original sentence, which is used to\nguide the decision of which the words might be deleted or inserted with the\nsoft attention mechanism under the sequence-to-sequence framework. We conduct\nexperiments on two benchmark datasets for paraphrase generation, namely the\nMSCOCO and Quora dataset. The evaluation results demonstrate that our\ndictionary-guided editing networks outperforms the baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08097v2",
            "date": "2018-06-21 07:44:32+00:00",
            "title": "BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation",
            "authors": [
                "Dayiheng Liu",
                "Jie Fu",
                "Qian Qu",
                "Jiancheng Lv"
            ],
            "abstract": "Incorporating prior knowledge like lexical constraints into the model's\noutput to generate meaningful and coherent sentences has many applications in\ndialogue system, machine translation, image captioning, etc. However, existing\nRNN-based models incrementally generate sentences from left to right via beam\nsearch, which makes it difficult to directly introduce lexical constraints into\nthe generated sentences. In this paper, we propose a new algorithmic framework,\ndubbed BFGAN, to address this challenge. Specifically, we employ a backward\ngenerator and a forward generator to generate lexically constrained sentences\ntogether, and use a discriminator to guide the joint training of two generators\nby assigning them reward signals. Due to the difficulty of BFGAN training, we\npropose several training techniques to make the training process more stable\nand efficient. Our extensive experiments on two large-scale datasets with human\nevaluation demonstrate that BFGAN has significant improvements over previous\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08115v2",
            "date": "2018-06-21 08:53:06+00:00",
            "title": "Modeling Word Emotion in Historical Language: Quantity Beats Supposed Stability in Seed Word Selection",
            "authors": [
                "Johannes Hellrich",
                "Sven Buechel",
                "Udo Hahn"
            ],
            "abstract": "To understand historical texts, we must be aware that language -- including\nthe emotional connotation attached to words -- changes over time. In this\npaper, we aim at estimating the emotion which is associated with a given word\nin former language stages of English and German. Emotion is represented\nfollowing the popular Valence-Arousal-Dominance (VAD) annotation scheme. While\nbeing more expressive than polarity alone, existing word emotion induction\nmethods are typically not suited for addressing it. To overcome this\nlimitation, we present adaptations of two popular algorithms to VAD. To measure\ntheir effectiveness in diachronic settings, we present the first gold standard\nfor historical word emotions, which was created by scholars with proficiency in\nthe respective language stages and covers both English and German. In contrast\nto claims in previous work, our findings indicate that hand-selecting small\nsets of seed words with supposedly stable emotional meaning is actually harmful\nrather than helpful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08309v1",
            "date": "2018-06-21 16:24:31+00:00",
            "title": "Par4Sim -- Adaptive Paraphrasing for Text Simplification",
            "authors": [
                "Seid Muhie Yimam",
                "Chris Biemann"
            ],
            "abstract": "Learning from a real-world data stream and continuously updating the model\nwithout explicit supervision is a new challenge for NLP applications with\nmachine learning components. In this work, we have developed an adaptive\nlearning system for text simplification, which improves the underlying\nlearning-to-rank model from usage data, i.e. how users have employed the system\nfor the task of simplification. Our experimental result shows that, over a\nperiod of time, the performance of the embedded paraphrase ranking model\nincreases steadily improving from a score of 62.88% up to 75.70% based on the\nNDCG@10 evaluation metrics. To our knowledge, this is the first study where an\nNLP component is adaptively improved through usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08890v1",
            "date": "2018-06-23 02:00:13+00:00",
            "title": "Emotion Representation Mapping for Automatic Lexicon Construction (Mostly) Performs on Human Level",
            "authors": [
                "Sven Buechel",
                "Udo Hahn"
            ],
            "abstract": "Emotion Representation Mapping (ERM) has the goal to convert existing emotion\nratings from one representation format into another one, e.g., mapping\nValence-Arousal-Dominance annotations for words or sentences into Ekman's Basic\nEmotions and vice versa. ERM can thus not only be considered as an alternative\nto Word Emotion Induction (WEI) techniques for automatic emotion lexicon\nconstruction but may also help mitigate problems that come from the\nproliferation of emotion representation formats in recent years. We propose a\nnew neural network approach to ERM that not only outperforms the previous\nstate-of-the-art. Equally important, we present a refined evaluation\nmethodology and gather strong evidence that our model yields results which are\n(almost) as reliable as human annotations, even in cross-lingual settings.\nBased on these results we generate new emotion ratings for 13 typologically\ndiverse languages and claim that they have near-gold quality, at least.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09089v2",
            "date": "2018-06-24 05:54:45+00:00",
            "title": "Character-Level Feature Extraction with Densely Connected Networks",
            "authors": [
                "Chanhee Lee",
                "Young-Bum Kim",
                "Dongyub Lee",
                "HeuiSeok Lim"
            ],
            "abstract": "Generating character-level features is an important step for achieving good\nresults in various natural language processing tasks. To alleviate the need for\nhuman labor in generating hand-crafted features, methods that utilize neural\narchitectures such as Convolutional Neural Network (CNN) or Recurrent Neural\nNetwork (RNN) to automatically extract such features have been proposed and\nhave shown great results. However, CNN generates position-independent features,\nand RNN is slow since it needs to process the characters sequentially. In this\npaper, we propose a novel method of using a densely connected network to\nautomatically extract character-level features. The proposed method does not\nrequire any language or task specific assumptions, and shows robustness and\neffectiveness while being faster than CNN- or RNN-based methods. Evaluating\nthis method on three sequence labeling tasks - slot tagging, Part-of-Speech\n(POS) tagging, and Named-Entity Recognition (NER) - we obtain state-of-the-art\nperformance with a 96.62 F1-score and 97.73% accuracy on slot tagging and POS\ntagging, respectively, and comparable performance to the state-of-the-art 91.13\nF1-score on NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09102v2",
            "date": "2018-06-24 08:15:54+00:00",
            "title": "Modeling Multi-turn Conversation with Deep Utterance Aggregation",
            "authors": [
                "Zhuosheng Zhang",
                "Jiangtong Li",
                "Pengfei Zhu",
                "Hai Zhao",
                "Gongshen Liu"
            ],
            "abstract": "Multi-turn conversation understanding is a major challenge for building\nintelligent dialogue systems. This work focuses on retrieval-based response\nmatching for multi-turn conversation whose related work simply concatenates the\nconversation utterances, ignoring the interactions among previous utterances\nfor context modeling. In this paper, we formulate previous utterances into\ncontext using a proposed deep utterance aggregation model to form a\nfine-grained context representation. In detail, a self-matching attention is\nfirst introduced to route the vital information in each utterance. Then the\nmodel matches a response with each refined utterance and the final matching\nscore is obtained after attentive turns aggregation. Experimental results show\nour model outperforms the state-of-the-art methods on three multi-turn\nconversation benchmarks, including a newly introduced e-commerce dialogue\ncorpus.",
            "journal ref": "COLING 2018, pages 3740-3752",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09103v1",
            "date": "2018-06-24 08:18:12+00:00",
            "title": "Subword-augmented Embedding for Cloze Reading Comprehension",
            "authors": [
                "Zhuosheng Zhang",
                "Yafang Huang",
                "Hai Zhao"
            ],
            "abstract": "Representation learning is the foundation of machine reading comprehension.\nIn state-of-the-art models, deep learning methods broadly use word and\ncharacter level representations. However, character is not naturally the\nminimal linguistic unit. In addition, with a simple concatenation of character\nand word embedding, previous models actually give suboptimal solution. In this\npaper, we propose to use subword rather than character for word embedding\nenhancement. We also empirically explore different augmentation strategies on\nsubword-augmented embedding to enhance the cloze-style reading comprehension\nmodel reader. In detail, we present a reader that uses subword-level\nrepresentation to augment word embedding with a short list to handle rare words\neffectively. A thorough examination is conducted to evaluate the comprehensive\nperformance and generalization ability of the proposed reader. Experimental\nresults show that the proposed approach helps the reader significantly\noutperform the state-of-the-art baselines on various public datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09105v1",
            "date": "2018-06-24 08:36:23+00:00",
            "title": "One-shot Learning for Question-Answering in Gaokao History Challenge",
            "authors": [
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "Answering questions from university admission exams (Gaokao in Chinese) is a\nchallenging AI task since it requires effective representation to capture\ncomplicated semantic relations between questions and answers. In this work, we\npropose a hybrid neural model for deep question-answering task from history\nexaminations. Our model employs a cooperative gated neural network to retrieve\nanswers with the assistance of extra labels given by a neural turing machine\nlabeler. Empirical study shows that the labeler works well with only a small\ntraining dataset and the gated mechanism is good at fetching the semantic\nrepresentation of lengthy answers. Experiments on question answering\ndemonstrate the proposed model obtains substantial performance gains over\nvarious neural model baselines in terms of multiple evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09374v1",
            "date": "2018-06-25 10:41:29+00:00",
            "title": "Fast ASR-free and almost zero-resource keyword spotting using DTW and CNNs for humanitarian monitoring",
            "authors": [
                "Raghav Menon",
                "Herman Kamper",
                "John Quinn",
                "Thomas Niesler"
            ],
            "abstract": "We use dynamic time warping (DTW) as supervision for training a convolutional\nneural network (CNN) based keyword spotting system using a small set of spoken\nisolated keywords. The aim is to allow rapid deployment of a keyword spotting\nsystem in a new language to support urgent United Nations (UN) relief\nprogrammes in parts of Africa where languages are extremely under-resourced and\nthe development of annotated speech resources is infeasible. First, we use 1920\nrecorded keywords (40 keyword types, 34 minutes of speech) as exemplars in a\nDTW-based template matching system and apply it to untranscribed broadcast\nspeech. Then, we use the resulting DTW scores as targets to train a CNN on the\nsame unlabelled speech. In this way we use just 34 minutes of labelled speech,\nbut leverage a large amount of unlabelled data for training. While the\nresulting CNN keyword spotter cannot match the performance of the DTW-based\nsystem, it substantially outperforms a CNN classifier trained only on the\nkeywords, improving the area under the ROC curve from 0.54 to 0.64. Because our\nCNN system is several orders of magnitude faster at runtime than the DTW\nsystem, it represents the most viable keyword spotter on this extremely limited\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09439v1",
            "date": "2018-06-25 13:17:24+00:00",
            "title": "Prior Attention for Style-aware Sequence-to-Sequence Models",
            "authors": [
                "Lucas Sterckx",
                "Johannes Deleu",
                "Chris Develder",
                "Thomas Demeester"
            ],
            "abstract": "We extend sequence-to-sequence models with the possibility to control the\ncharacteristics or style of the generated output, via attention that is\ngenerated a priori (before decoding) from a latent code vector. After training\nan initial attention-based sequence-to-sequence model, we use a variational\nauto-encoder conditioned on representations of input sequences and a latent\ncode vector space to generate attention matrices. By sampling the code vector\nfrom specific regions of this latent space during decoding and imposing prior\nattention generated from it in the seq2seq model, output can be steered towards\nhaving certain attributes. This is demonstrated for the task of sentence\nsimplification, where the latent code vector allows control over output length\nand lexical simplification, and enables fine-tuning to optimize for different\nevaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09652v1",
            "date": "2018-06-25 18:22:27+00:00",
            "title": "Neural Machine Translation for Low Resource Languages using Bilingual Lexicon Induced from Comparable Corpora",
            "authors": [
                "Sree Harsha Ramesh",
                "Krishna Prasad Sankaranarayanan"
            ],
            "abstract": "Resources for the non-English languages are scarce and this paper addresses\nthis problem in the context of machine translation, by automatically extracting\nparallel sentence pairs from the multilingual articles available on the\nInternet. In this paper, we have used an end-to-end Siamese bidirectional\nrecurrent neural network to generate parallel sentences from comparable\nmultilingual articles in Wikipedia. Subsequently, we have showed that using the\nharvested dataset improved BLEU scores on both NMT and phrase-based SMT systems\nfor the low-resource language pairs: English--Hindi and English--Tamil, when\ncompared to training exclusively on the limited bilingual corpora collected for\nthese language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09751v2",
            "date": "2018-06-26 01:36:44+00:00",
            "title": "A Practical Incremental Learning Framework For Sparse Entity Extraction",
            "authors": [
                "Hussein S. Al-Olimat",
                "Steven Gustafson",
                "Jason Mackay",
                "Krishnaprasad Thirunarayan",
                "Amit Sheth"
            ],
            "abstract": "This work addresses challenges arising from extracting entities from textual\ndata, including the high cost of data annotation, model accuracy, selecting\nappropriate evaluation criteria, and the overall quality of annotation. We\npresent a framework that integrates Entity Set Expansion (ESE) and Active\nLearning (AL) to reduce the annotation cost of sparse data and provide an\nonline evaluation method as feedback. This incremental and interactive learning\nframework allows for rapid annotation and subsequent extraction of sparse data\nwhile maintaining high accuracy. We evaluate our framework on three publicly\navailable datasets and show that it drastically reduces the cost of sparse\nentity annotation by an average of 85% and 45% to reach 0.9 and 1.0 F-Scores\nrespectively. Moreover, the method exhibited robust performance across all\ndatasets.",
            "journal ref": "In Proceedings of COLING 2018, the 27th International Conference\n  on Computational Linguistics: Technical Papers",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09792v1",
            "date": "2018-06-26 05:01:51+00:00",
            "title": "A Multi-Modal Chinese Poetry Generation Model",
            "authors": [
                "Dayiheng Liu",
                "Quan Guo",
                "Wubo Li",
                "Jiancheng Lv"
            ],
            "abstract": "Recent studies in sequence-to-sequence learning demonstrate that RNN\nencoder-decoder structure can successfully generate Chinese poetry. However,\nexisting methods can only generate poetry with a given first line or user's\nintent theme. In this paper, we proposed a three-stage multi-modal Chinese\npoetry generation approach. Given a picture, the first line, the title and the\nother lines of the poem are successively generated in three stages. According\nto the characteristics of Chinese poems, we propose a hierarchy-attention\nseq2seq model which can effectively capture character, phrase, and sentence\ninformation between contexts and improve the symmetry delivered in poems. In\naddition, the Latent Dirichlet allocation (LDA) model is utilized for title\ngeneration and improve the relevance of the whole poem and the title. Compared\nwith strong baseline, the experimental results demonstrate the effectiveness of\nour approach, using machine evaluations as well as human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09828v1",
            "date": "2018-06-26 07:50:46+00:00",
            "title": "Enhancing Sentence Embedding with Generalized Pooling",
            "authors": [
                "Qian Chen",
                "Zhen-Hua Ling",
                "Xiaodan Zhu"
            ],
            "abstract": "Pooling is an essential component of a wide variety of sentence\nrepresentation and embedding models. This paper explores generalized pooling\nmethods to enhance sentence embedding. We propose vector-based multi-head\nattention that includes the widely used max pooling, mean pooling, and scalar\nself-attention as special cases. The model benefits from properly designed\npenalization terms to reduce redundancy in multi-head attention. We evaluate\nthe proposed model on three different tasks: natural language inference (NLI),\nauthor profiling, and sentiment classification. The experiments show that the\nproposed model achieves significant improvement over strong\nsentence-encoding-based methods, resulting in state-of-the-art performances on\nfour datasets. The proposed approach can be easily implemented for more\nproblems than we discuss in this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10090v1",
            "date": "2018-06-26 16:07:50+00:00",
            "title": "Conditional Generators of Words Definitions",
            "authors": [
                "Artyom Gadetsky",
                "Ilya Yakubovskiy",
                "Dmitry Vetrov"
            ],
            "abstract": "We explore recently introduced definition modeling technique that provided\nthe tool for evaluation of different distributed vector representations of\nwords through modeling dictionary definitions of words. In this work, we study\nthe problem of word ambiguities in definition modeling and propose a possible\nsolution by employing latent variable modeling and soft attention mechanisms.\nOur quantitative and qualitative evaluation and analysis of the model shows\nthat taking into account words ambiguity and polysemy leads to performance\nimprovement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10201v1",
            "date": "2018-06-26 20:24:52+00:00",
            "title": "Neural Cross-Lingual Coreference Resolution and its Application to Entity Linking",
            "authors": [
                "Gourab Kundu",
                "Avirup Sil",
                "Radu Florian",
                "Wael Hamza"
            ],
            "abstract": "We propose an entity-centric neural cross-lingual coreference model that\nbuilds on multi-lingual embeddings and language-independent features. We\nperform both intrinsic and extrinsic evaluations of our model. In the intrinsic\nevaluation, we show that our model, when trained on English and tested on\nChinese and Spanish, achieves competitive results to the models trained\ndirectly on Chinese and Spanish respectively. In the extrinsic evaluation, we\nshow that our English model helps achieve superior entity linking accuracy on\nChinese and Spanish test sets than the top 2015 TAC system without using any\nannotated data from Chinese or Spanish.",
            "journal ref": "ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10215v4",
            "date": "2018-06-26 21:17:09+00:00",
            "title": "Contextual Language Model Adaptation for Conversational Agents",
            "authors": [
                "Anirudh Raju",
                "Behnam Hedayatnia",
                "Linda Liu",
                "Ankur Gandhe",
                "Chandra Khatri",
                "Angeliki Metallinou",
                "Anu Venkatesh",
                "Ariya Rastrow"
            ],
            "abstract": "Statistical language models (LM) play a key role in Automatic Speech\nRecognition (ASR) systems used by conversational agents. These ASR systems\nshould provide a high accuracy under a variety of speaking styles, domains,\nvocabulary and argots. In this paper, we present a DNN-based method to adapt\nthe LM to each user-agent interaction based on generalized contextual\ninformation, by predicting an optimal, context-dependent set of LM\ninterpolation weights. We show that this framework for contextual adaptation\nprovides accuracy improvements under different possible mixture LM partitions\nthat are relevant for both (1) Goal-oriented conversational agents where it's\nnatural to partition the data by the requested application and for (2) Non-goal\noriented conversational agents where the data can be partitioned using topic\nlabels that come from predictions of a topic classifier. We obtain a relative\nWER improvement of 3% with a 1-pass decoding strategy and 6% in a 2-pass\ndecoding framework, over an unadapted model. We also show up to a 15% relative\nimprovement in recognizing named entities which is of significant value for\nconversational ASR systems.",
            "journal ref": "Proc. Interspeech 2018, 3333-3337",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10654v1",
            "date": "2018-06-27 19:36:05+00:00",
            "title": "Generalized chart constraints for efficient PCFG and TAG parsing",
            "authors": [
                "Stefan Gr\u00fcnewald",
                "Sophie Henning",
                "Alexander Koller"
            ],
            "abstract": "Chart constraints, which specify at which string positions a constituent may\nbegin or end, have been shown to speed up chart parsers for PCFGs. We\ngeneralize chart constraints to more expressive grammar formalisms and describe\na neural tagger which predicts chart constraints at very high precision. Our\nconstraints accelerate both PCFG and TAG parsing, and combine effectively with\nother pruning techniques (coarse-to-fine and supertagging) for an overall\nspeedup of two orders of magnitude, while improving accuracy.",
            "journal ref": "Proceedings of ACL 2018 (Short Papers)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10722v2",
            "date": "2018-06-28 00:45:04+00:00",
            "title": "DeepTag: inferring all-cause diagnoses from clinical notes in under-resourced medical domain",
            "authors": [
                "Allen Nie",
                "Ashley Zehnder",
                "Rodney L. Page",
                "Arturo L. Pineda",
                "Manuel A. Rivas",
                "Carlos D. Bustamante",
                "James Zou"
            ],
            "abstract": "Large scale veterinary clinical records can become a powerful resource for\npatient care and research. However, clinicians lack the time and resource to\nannotate patient records with standard medical diagnostic codes and most\nveterinary visits are captured in free text notes. The lack of standard coding\nmakes it challenging to use the clinical data to improve patient care. It is\nalso a major impediment to cross-species translational research, which relies\non the ability to accurately identify patient cohorts with specific diagnostic\ncriteria in humans and animals. In order to reduce the coding burden for\nveterinary clinical practice and aid translational research, we have developed\na deep learning algorithm, DeepTag, which automatically infers diagnostic codes\nfrom veterinary free text notes. DeepTag is trained on a newly curated dataset\nof 112,558 veterinary notes manually annotated by experts. DeepTag extends\nmulti-task LSTM with an improved hierarchical objective that captures the\nsemantic structures between diseases. To foster human-machine collaboration,\nDeepTag also learns to abstain in examples when it is uncertain and defers them\nto human experts, resulting in improved performance. DeepTag accurately infers\ndisease codes from free text even in challenging cross-hospital settings where\nthe text comes from different clinical settings than the ones used for\ntraining. It enables automated disease annotation across a broad range of\nclinical diagnoses with minimal pre-processing. The technical framework in this\nwork can be applied in other medical domains that currently lack medical coding\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10771v1",
            "date": "2018-06-28 05:09:38+00:00",
            "title": "Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging",
            "authors": [
                "Andrew Matteson",
                "Chanhee Lee",
                "Young-Bum Kim",
                "Heuiseok Lim"
            ],
            "abstract": "Due to the fact that Korean is a highly agglutinative, character-rich\nlanguage, previous work on Korean morphological analysis typically employs the\nuse of sub-character features known as graphemes or otherwise utilizes\ncomprehensive prior linguistic knowledge (i.e., a dictionary of known\nmorphological transformation forms, or actions). These models have been created\nwith the assumption that character-level, dictionary-less morphological\nanalysis was intractable due to the number of actions required. We present, in\nthis study, a multi-stage action-based model that can perform morphological\ntransformation and part-of-speech tagging using arbitrary units of input and\napply it to the case of character-level Korean morphological analysis. Among\nmodels that do not employ prior linguistic knowledge, we achieve\nstate-of-the-art word and sentence-level tagging accuracy with the Sejong\nKorean corpus using our proposed data-driven Bi-LSTM model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11099v1",
            "date": "2018-06-28 17:42:54+00:00",
            "title": "Predicting CEFRL levels in learner English on the basis of metrics and full texts",
            "authors": [
                "Taylor Arnold",
                "Nicolas Ballier",
                "Thomas Gaillat",
                "Paula Liss\u00f2n"
            ],
            "abstract": "This paper analyses the contribution of language metrics and, potentially, of\nlinguistic structures, to classify French learners of English according to\nlevels of the Common European Framework of Reference for Languages (CEFRL). The\npurpose is to build a model for the prediction of learner levels as a function\nof language complexity features. We used the EFCAMDAT corpus, a database of one\nmillion written assignments by learners. After applying language complexity\nmetrics on the texts, we built a representation matching the language metrics\nof the texts to their assigned CEFRL levels. Lexical and syntactic metrics were\ncomputed with LCA, LSA, and koRpus. Several supervised learning models were\nbuilt by using Gradient Boosted Trees and Keras Neural Network methods and by\ncontrasting pairs of CEFRL levels. Results show that it is possible to\nimplement pairwise distinctions, especially for levels ranging from A1 to B1\n(A1=>A2: 0.916 AUC and A2=>B1: 0.904 AUC). Model explanation reveals\nsignificant linguistic features for the predictiveness in the corpus. Word\ntokens and word types appear to play a significant role in determining levels.\nThis shows that levels are highly dependent on specific semantic profiles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11183v1",
            "date": "2018-06-28 20:55:18+00:00",
            "title": "Cross-Discourse and Multilingual Exploration of Textual Corpora with the DualNeighbors Algorithm",
            "authors": [
                "Taylor Arnold",
                "Lauren Tilton"
            ],
            "abstract": "Word choice is dependent on the cultural context of writers and their\nsubjects. Different words are used to describe similar actions, objects, and\nfeatures based on factors such as class, race, gender, geography and political\naffinity. Exploratory techniques based on locating and counting words may,\ntherefore, lead to conclusions that reinforce culturally inflected boundaries.\nWe offer a new method, the DualNeighbors algorithm, for linking thematically\nsimilar documents both within and across discursive and linguistic barriers to\nreveal cross-cultural connections. Qualitative and quantitative evaluations of\nthis technique are shown as applied to two cultural datasets of interest to\nresearchers across the humanities and social sciences. An open-source\nimplementation of the DualNeighbors algorithm is provided to assist in its\napplication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11249v1",
            "date": "2018-06-29 02:06:00+00:00",
            "title": "Neural Machine Translation with Key-Value Memory-Augmented Attention",
            "authors": [
                "Fandong Meng",
                "Zhaopeng Tu",
                "Yong Cheng",
                "Haiyang Wu",
                "Junjie Zhai",
                "Yuekui Yang",
                "Di Wang"
            ],
            "abstract": "Although attention-based Neural Machine Translation (NMT) has achieved\nremarkable progress in recent years, it still suffers from issues of repeating\nand dropping translations. To alleviate these issues, we propose a novel\nkey-value memory-augmented attention model for NMT, called KVMEMATT.\nSpecifically, we maintain a timely updated keymemory to keep track of attention\nhistory and a fixed value-memory to store the representation of source sentence\nthroughout the whole translation process. Via nontrivial transformations and\niterative interactions between the two memories, the decoder focuses on more\nappropriate source word(s) for predicting the next target word at each decoding\nstep, therefore can improve the adequacy of translations. Experimental results\non Chinese=>English and WMT17 German<=>English translation tasks demonstrate\nthe superiority of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11322v1",
            "date": "2018-06-29 09:51:26+00:00",
            "title": "Bias in Semantic and Discourse Interpretation",
            "authors": [
                "Nicholas Asher",
                "Soumya Paul"
            ],
            "abstract": "In this paper, we show how game-theoretic work on conversation combined with\na theory of discourse structure provides a framework for studying interpretive\nbias. Interpretive bias is an essential feature of learning and understanding\nbut also something that can be used to pervert or subvert the truth. The\nframework we develop here provides tools for understanding and analyzing the\nrange of interpretive biases and the factors that contribute to them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11432v1",
            "date": "2018-06-29 14:19:20+00:00",
            "title": "Using General Adversarial Networks for Marketing: A Case Study of Airbnb",
            "authors": [
                "Richard Diehl Martinez",
                "John Kaleialoha Kamalu"
            ],
            "abstract": "In this paper, we examine the use case of general adversarial networks (GANs)\nin the field of marketing. In particular, we analyze how GAN models can\nreplicate text patterns from successful product listings on Airbnb, a\npeer-to-peer online market for short-term apartment rentals. To do so, we\ndefine the Diehl-Martinez-Kamalu (DMK) loss function as a new class of\nfunctions that forces the model's generated output to include a set of\nuser-defined keywords. This allows the general adversarial network to recommend\na way of rewording the phrasing of a listing description to increase the\nlikelihood that it is booked. Although we tailor our analysis to Airbnb data,\nwe believe this framework establishes a more general model for how generative\nalgorithms can be used to produce text samples for the purposes of marketing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11461v1",
            "date": "2018-06-29 15:07:17+00:00",
            "title": "Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs",
            "authors": [
                "Matthew Roddy",
                "Gabriel Skantze",
                "Naomi Harte"
            ],
            "abstract": "For spoken dialog systems to conduct fluid conversational interactions with\nusers, the systems must be sensitive to turn-taking cues produced by a user.\nModels should be designed so that effective decisions can be made as to when it\nis appropriate, or not, for the system to speak. Traditional end-of-turn\nmodels, where decisions are made at utterance end-points, are limited in their\nability to model fast turn-switches and overlap. A more flexible approach is to\nmodel turn-taking in a continuous manner using RNNs, where the system predicts\nspeech probability scores for discrete frames within a future window. The\ncontinuous predictions represent generalized turn-taking behaviors observed in\nthe training data and can be applied to make decisions that are not just\nlimited to end-of-turn detection. In this paper, we investigate optimal\nspeech-related feature sets for making predictions at pauses and overlaps in\nconversation. We find that while traditional acoustic features perform well,\npart-of-speech features generally perform worse than word features. We show\nthat our current models outperform previously reported baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00072v1",
            "date": "2018-06-29 21:27:05+00:00",
            "title": "Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Rates",
            "authors": [
                "Joo-Kyung Kim",
                "Young-Bum Kim"
            ],
            "abstract": "In domain classification for spoken dialog systems, correct detection of\nout-of-domain (OOD) utterances is crucial because it reduces confusion and\nunnecessary interaction costs between users and the systems. Previous work\nusually utilizes OOD detectors that are trained separately from in-domain (IND)\nclassifiers, and confidence thresholding for OOD detection given target\nevaluation scores. In this paper, we introduce a neural joint learning model\nfor domain classification and OOD detection, where dynamic class weighting is\nused during the model training to satisfice a given OOD false acceptance rate\n(FAR) while maximizing the domain classification accuracy. Evaluating on two\ndomain classification tasks for the utterances from a large spoken dialogue\nsystem, we show that our approach significantly improves the domain\nclassification performance with satisficing given target FARs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00248v1",
            "date": "2018-07-01 00:31:27+00:00",
            "title": "A Shared Attention Mechanism for Interpretation of Neural Automatic Post-Editing Systems",
            "authors": [
                "Inigo Jauregi Unanue",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "Automatic post-editing (APE) systems aim to correct the systematic errors\nmade by machine translators. In this paper, we propose a neural APE system that\nencodes the source (src) and machine translated (mt) sentences with two\nseparate encoders, but leverages a shared attention mechanism to better\nunderstand how the two inputs contribute to the generation of the post-edited\n(pe) sentences. Our empirical observations have showed that when the mt is\nincorrect, the attention shifts weight toward tokens in the src sentence to\nproperly edit the incorrect translation. The model has been trained and\nevaluated on the official data from the WMT16 and WMT17 APE IT domain\nEnglish-German shared tasks. Additionally, we have used the extra 500K\nartificial data provided by the shared task. Our system has been able to\nreproduce the accuracies of systems trained with the same data, while at the\nsame time providing better interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00267v1",
            "date": "2018-07-01 04:11:18+00:00",
            "title": "An Efficient Approach to Encoding Context for Spoken Language Understanding",
            "authors": [
                "Raghav Gupta",
                "Abhinav Rastogi",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "In task-oriented dialogue systems, spoken language understanding, or SLU,\nrefers to the task of parsing natural language user utterances into semantic\nframes. Making use of context from prior dialogue history holds the key to more\neffective SLU. State of the art approaches to SLU use memory networks to encode\ncontext by processing multiple utterances from the dialogue at each turn,\nresulting in significant trade-offs between accuracy and computational\nefficiency. On the other hand, downstream components like the dialogue state\ntracker (DST) already keep track of the dialogue state, which can serve as a\nsummary of the dialogue history. In this work, we propose an efficient approach\nto encoding context from prior utterances for SLU. More specifically, our\narchitecture includes a separate recurrent neural network (RNN) based encoding\nmodule that accumulates dialogue context to guide the frame parsing sub-tasks\nand can be shared between SLU and DST. In our experiments, we demonstrate the\neffectiveness of our approach on dialogues from two domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00286v1",
            "date": "2018-07-01 07:17:36+00:00",
            "title": "Lost in Translation: Analysis of Information Loss During Machine Translation Between Polysynthetic and Fusional Languages",
            "authors": [
                "Manuel Mager",
                "Elisabeth Mager",
                "Alfonso Medina-Urrea",
                "Ivan Meza",
                "Katharina Kann"
            ],
            "abstract": "Machine translation from polysynthetic to fusional languages is a challenging\ntask, which gets further complicated by the limited amount of parallel text\navailable. Thus, translation performance is far from the state of the art for\nhigh-resource and more intensively studied language pairs. To shed light on the\nphenomena which hamper automatic translation to and from polysynthetic\nlanguages, we study translations from three low-resource, polysynthetic\nlanguages (Nahuatl, Wixarika and Yorem Nokki) into Spanish and vice versa.\nDoing so, we find that in a morpheme-to-morpheme alignment an important amount\nof information contained in polysynthetic morphemes has no Spanish counterpart,\nand its translation is often omitted. We further conduct a qualitative analysis\nand, thus, identify morpheme types that are commonly hard to align or ignored\nin the translation process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00488v1",
            "date": "2018-07-02 06:48:46+00:00",
            "title": "A Simple but Effective Classification Model for Grammatical Error Correction",
            "authors": [
                "Zhu Kaili",
                "Chuan Wang",
                "Ruobing Li",
                "Yang Liu",
                "Tianlei Hu",
                "Hui Lin"
            ],
            "abstract": "We treat grammatical error correction (GEC) as a classification problem in\nthis study, where for different types of errors, a target word is identified,\nand the classifier predicts the correct word form from a set of possible\nchoices. We propose a novel neural network based feature representation and\nclassification model, trained using large text corpora without human\nannotations. Specifically we use RNNs with attention to represent both the left\nand right context of a target word. All feature embeddings are learned jointly\nin an end-to-end fashion. Experimental results show that our novel approach\noutperforms other classifier methods on the CoNLL-2014 test set (F0.5 45.05%).\nOur model is simple but effective, and is suitable for industrial production.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00543v1",
            "date": "2018-07-02 09:06:27+00:00",
            "title": "Punctuation Prediction Model for Conversational Speech",
            "authors": [
                "Piotr \u017belasko",
                "Piotr Szyma\u0144ski",
                "Jan Mizgajski",
                "Adrian Szymczak",
                "Yishay Carmiel",
                "Najim Dehak"
            ],
            "abstract": "An ASR system usually does not predict any punctuation or capitalization.\nLack of punctuation causes problems in result presentation and confuses both\nthe human reader andoff-the-shelf natural language processing algorithms. To\novercome these limitations, we train two variants of Deep Neural Network (DNN)\nsequence labelling models - a Bidirectional Long Short-Term Memory (BLSTM) and\na Convolutional Neural Network (CNN), to predict the punctuation. The models\nare trained on the Fisher corpus which includes punctuation annotation. In our\nexperiments, we combine time-aligned and punctuated Fisher corpus transcripts\nusing a sequence alignment algorithm. The neural networks are trained on Common\nWeb Crawl GloVe embedding of the words in Fisher transcripts aligned with\nconversation side indicators and word time infomation. The CNNs yield a better\nprecision and BLSTMs tend to have better recall. While BLSTMs make fewer\nmistakes overall, the punctuation predicted by the CNN is more accurate -\nespecially in the case of question marks. Our results constitute significant\nevidence that the distribution of words in time, as well as pre-trained\nembeddings, can be useful in the punctuation prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00571v1",
            "date": "2018-07-02 09:53:50+00:00",
            "title": "The Interplay between Lexical Resources and Natural Language Processing",
            "authors": [
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Mohammad Taher Pilehvar"
            ],
            "abstract": "Incorporating linguistic, world and common sense knowledge into AI/NLP\nsystems is currently an important research area, with several open problems and\nchallenges. At the same time, processing and storing this knowledge in lexical\nresources is not a straightforward task. This tutorial proposes to address\nthese complementary goals from two methodological perspectives: the use of NLP\nmethods to help the process of constructing and enriching lexical resources and\nthe use of lexical resources for improving NLP applications. Two main types of\naudience can benefit from this tutorial: those working on language resources\nwho are interested in becoming acquainted with automatic NLP techniques, with\nthe end goal of speeding and/or easing up the process of resource curation; and\non the other hand, researchers in NLP who would like to benefit from the\nknowledge of lexical resources to improve their systems and models. The slides\nof the tutorial are available at https://bitbucket.org/luisespinosa/lr-nlp/",
            "journal ref": "Proceedings of the 2018 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Tutorial Abstracts",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00651v1",
            "date": "2018-07-02 13:29:46+00:00",
            "title": "A Neural Approach to Language Variety Translation",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Marcos Zampieri",
                "Santanu Pal"
            ],
            "abstract": "In this paper we present the first neural-based machine translation system\ntrained to translate between standard national varieties of the same language.\nWe take the pair Brazilian - European Portuguese as an example and compare the\nperformance of this method to a phrase-based statistical machine translation\nsystem. We report a performance improvement of 0.9 BLEU points in translating\nfrom European to Brazilian Portuguese and 0.2 BLEU points when translating in\nthe opposite direction. We also carried out a human evaluation experiment with\nnative speakers of Brazilian Portuguese which indicates that humans prefer the\noutput produced by the neural-based system in comparison to the statistical\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00717v1",
            "date": "2018-07-02 14:47:30+00:00",
            "title": "Transparent, Efficient, and Robust Word Embedding Access with WOMBAT",
            "authors": [
                "Mark-Christoph M\u00fcller",
                "Michael Strube"
            ],
            "abstract": "We present WOMBAT, a Python tool which supports NLP practitioners in\naccessing word embeddings from code. WOMBAT addresses common research problems,\nincluding unified access, scaling, and robust and reproducible preprocessing.\nCode that uses WOMBAT for accessing word embeddings is not only cleaner, more\nreadable, and easier to reuse, but also much more efficient than code using\nstandard in-memory methods: a Python script using WOMBAT for evaluating seven\nlarge word embedding collections (8.7M embedding vectors in total) on a simple\nSemEval sentence similarity task involving 250 raw sentence pairs completes in\nunder ten seconds end-to-end on a standard notebook computer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00775v1",
            "date": "2018-07-02 16:29:34+00:00",
            "title": "Representation Mapping: A Novel Approach to Generate High-Quality Multi-Lingual Emotion Lexicons",
            "authors": [
                "Sven Buechel",
                "Udo Hahn"
            ],
            "abstract": "In the past years, sentiment analysis has increasingly shifted attention to\nrepresentational frameworks more expressive than semantic polarity (being\npositive, negative or neutral). However, these richer formats (like Basic\nEmotions or Valence-Arousal-Dominance, and variants therefrom), rooted in\npsychological research, tend to proliferate the number of representation\nschemes for emotion encoding. Thus, a large amount of representationally\nincompatible emotion lexicons has been developed by various research groups\nadopting one or the other emotion representation format. As a consequence, the\nreusability of these resources decreases as does the comparability of systems\nusing them. In this paper, we propose to solve this dilemma by methods and\ntools which map different representation formats onto each other for the sake\nof mutual compatibility and interoperability of language resources. We present\nthe first large-scale investigation of such representation mappings for four\ntypologically diverse languages and find evidence that our approach produces\n(near-)gold quality emotion lexicons, even in cross-lingual settings. Finally,\nwe use our models to create new lexicons for eight typologically diverse\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00791v1",
            "date": "2018-07-02 17:24:22+00:00",
            "title": "Pragmatic approach to structured data querying via natural language interface",
            "authors": [
                "Aliaksei Vertsel",
                "Mikhail Rumiantsau"
            ],
            "abstract": "As the use of technology increases and data analysis becomes integral in many\nbusinesses, the ability to quickly access and interpret data has become more\nimportant than ever. Information retrieval technologies are being utilized by\norganizations and companies to manage their information systems and processes.\nDespite information retrieval of a large amount of data being efficient\norganized in relational databases, a user still needs to master the DB\nlanguage/schema to completely formulate the queries. This puts a burden on\norganizations and companies to hire employees that are proficient in DB\nlanguages/schemas to formulate queries. To reduce some of the burden on already\noverstretched data teams, many organizations are looking for tools that allow\nnon-developers to query their databases. Unfortunately, writing a valid SQL\nquery that answers the question a user is trying to ask isn't always easy. Even\nseemingly simple questions, like \"Which start-up companies received more than\n$200M in funding?\" can actually be very hard to answer, let alone convert into\na SQL query. How do you define start-up companies? By size, location, duration\nof time they have been incorporated? This may be fine if a user is working with\na database they're already familiar with, but what if users are not familiar\nwith the database. What is needed is a centralized system that can effectively\ntranslate natural language queries into specific database queries for different\ncustomer database types. There is a number of factors that can dramatically\naffect the system architecture and the set of algorithms used to translate NL\nqueries into a structured query representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00914v3",
            "date": "2018-07-02 22:09:59+00:00",
            "title": "Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing",
            "authors": [
                "Edoardo Maria Ponti",
                "Helen O'Horan",
                "Yevgeni Berzak",
                "Ivan Vuli\u0107",
                "Roi Reichart",
                "Thierry Poibeau",
                "Ekaterina Shutova",
                "Anna Korhonen"
            ],
            "abstract": "Linguistic typology aims to capture structural and semantic variation across\nthe world's languages. A large-scale typology could provide excellent guidance\nfor multilingual Natural Language Processing (NLP), particularly for languages\nthat suffer from the lack of human labeled resources. We present an extensive\nliterature survey on the use of typological information in the development of\nNLP techniques. Our survey demonstrates that to date, the use of information in\nexisting typological databases has resulted in consistent but modest\nimprovements in system performance. We show that this is due to both intrinsic\nlimitations of databases (in terms of coverage and feature granularity) and\nunder-employment of the typological features included in them. We advocate for\na new approach that adapts the broad and discrete nature of typological\ncategories to the contextual and continuous nature of machine learning\nalgorithms used in contemporary NLP. In particular, we suggest that such\napproach could be facilitated by recent developments in data-driven induction\nof typological knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00938v2",
            "date": "2018-07-03 00:52:50+00:00",
            "title": "Topic Discovery in Massive Text Corpora Based on Min-Hashing",
            "authors": [
                "Gibran Fuentes-Pineda",
                "Ivan Vladimir Meza-Ruiz"
            ],
            "abstract": "The task of discovering topics in text corpora has been dominated by Latent\nDirichlet Allocation and other Topic Models for over a decade. In order to\napply these approaches to massive text corpora, the vocabulary needs to be\nreduced considerably and large computer clusters and/or GPUs are typically\nrequired. Moreover, the number of topics must be provided beforehand but this\ndepends on the corpus characteristics and it is often difficult to estimate,\nespecially for massive text corpora. Unfortunately, both topic quality and time\ncomplexity are sensitive to this choice. This paper describes an alternative\napproach to discover topics based on Min-Hashing, which can handle massive text\ncorpora and large vocabularies using modest computer hardware and does not\nrequire to fix the number of topics in advance. The basic idea is to generate\nmultiple random partitions of the corpus vocabulary to find sets of highly\nco-occurring words, which are then clustered to produce the final topics. In\ncontrast to probabilistic topic models where topics are distributions over the\ncomplete vocabulary, the topics discovered by the proposed approach are sets of\nhighly co-occurring words. Interestingly, these topics underlie various\nthematics with different levels of granularity. An extensive qualitative and\nquantitative evaluation using the 20 Newsgroups (18K), Reuters (800K), Spanish\nWikipedia (1M), and English Wikipedia (5M) corpora shows that the proposed\napproach is able to consistently discover meaningful and coherent topics.\nRemarkably, the time complexity of the proposed approach is linear with respect\nto corpus and vocabulary size; a non-parallel implementation was able to\ndiscover topics from the entire English edition of Wikipedia with over 5\nmillion documents and 1 million words in less than 7 hours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01292v1",
            "date": "2018-07-03 17:15:49+00:00",
            "title": "Intent Generation for Goal-Oriented Dialogue Systems based on Schema.org Annotations",
            "authors": [
                "Umutcan \u015eim\u015fek",
                "Dieter Fensel"
            ],
            "abstract": "Goal-oriented dialogue systems typically communicate with a backend (e.g.\ndatabase, Web API) to complete certain tasks to reach a goal. The intents that\na dialogue system can recognize are mostly included to the system by the\ndeveloper statically. For an open dialogue system that can work on more than a\nsmall set of well curated data and APIs, this manual intent creation will not\nscalable. In this paper, we introduce a straightforward methodology for intent\ncreation based on semantic annotation of data and services on the web. With\nthis method, the Natural Language Understanding (NLU) module of a goal-oriented\ndialogue system can adapt to newly introduced APIs without requiring heavy\ndeveloper involvement. We were able to extract intents and necessary slots to\nbe filled from schema.org annotations. We were also able to create a set of\ninitial training sentences for classifying user utterances into the generated\nintents. We demonstrate our approach on the NLU module of a state-of-the art\ndialogue system development framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01396v1",
            "date": "2018-07-03 23:29:49+00:00",
            "title": "Simpler but More Accurate Semantic Dependency Parsing",
            "authors": [
                "Timothy Dozat",
                "Christopher D. Manning"
            ],
            "abstract": "While syntactic dependency annotations concentrate on the surface or\nfunctional structure of a sentence, semantic dependency annotations aim to\ncapture between-word relationships that are more closely related to the meaning\nof a sentence, using graph-structured representations. We extend the LSTM-based\nsyntactic parser of Dozat and Manning (2017) to train on and generate these\ngraph structures. The resulting system on its own achieves state-of-the-art\nperformance, beating the previous, substantially more complex state-of-the-art\nsystem by 0.6% labeled F1. Adding linguistically richer input representations\npushes the margin even higher, allowing us to beat it by 1.9% labeled F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01466v1",
            "date": "2018-07-04 07:18:36+00:00",
            "title": "Polarity and Intensity: the Two Aspects of Sentiment Analysis",
            "authors": [
                "Leimin Tian",
                "Catherine Lai",
                "Johanna D. Moore"
            ],
            "abstract": "Current multimodal sentiment analysis frames sentiment score prediction as a\ngeneral Machine Learning task. However, what the sentiment score actually\nrepresents has often been overlooked. As a measurement of opinions and\naffective states, a sentiment score generally consists of two aspects: polarity\nand intensity. We decompose sentiment scores into these two aspects and study\nhow they are conveyed through individual modalities and combined multimodal\nmodels in a naturalistic monologue setting. In particular, we build unimodal\nand multimodal multi-task learning models with sentiment score prediction as\nthe main task and polarity and/or intensity classification as the auxiliary\ntasks. Our experiments show that sentiment analysis benefits from multi-task\nlearning, and individual modalities differ when conveying the polarity and\nintensity aspects of sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01677v1",
            "date": "2018-07-04 16:54:05+00:00",
            "title": "Towards Automation of Sense-type Identification of Verbs in OntoSenseNet(Telugu)",
            "authors": [
                "Sreekavitha Parupalli",
                "Vijjini Anvesh Rao",
                "Radhika Mamidi"
            ],
            "abstract": "In this paper, we discuss the enrichment of a manually developed resource of\nTelugu lexicon, OntoSenseNet. OntoSenseNet is a ontological sense annotated\nlexicon that marks each verb of Telugu with a primary and a secondary sense.\nThe area of research is relatively recent but has a large scope of development.\nWe provide an introductory work to enrich the OntoSenseNet to promote further\nresearch in Telugu. Classifiers are adopted to learn the sense relevant\nfeatures of the words in the resource and also to automate the tagging of\nsense-types for verbs. We perform a comparative analysis of different\nclassifiers applied on OntoSenseNet. The results of the experiment prove that\nautomated enrichment of the resource is effective using SVM classifiers and\nAdaboost ensemble.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01679v1",
            "date": "2018-07-04 16:56:50+00:00",
            "title": "BCSAT : A Benchmark Corpus for Sentiment Analysis in Telugu Using Word-level Annotations",
            "authors": [
                "Sreekavitha Parupalli",
                "Vijjini Anvesh Rao",
                "Radhika Mamidi"
            ],
            "abstract": "The presented work aims at generating a systematically annotated corpus that\ncan support the enhancement of sentiment analysis tasks in Telugu using\nword-level sentiment annotations. From OntoSenseNet, we extracted 11,000\nadjectives, 253 adverbs, 8483 verbs and sentiment annotation is being done by\nlanguage experts. We discuss the methodology followed for the polarity\nannotations and validate the developed resource. This work aims at developing a\nbenchmark corpus, as an extension to SentiWordNet, and baseline accuracy for a\nmodel where lexeme annotations are applied for sentiment predictions. The\nfundamental aim of this paper is to validate and study the possibility of\nutilizing machine learning algorithms, word-level sentiment annotations in the\ntask of automated sentiment identification. Furthermore, accuracy is improved\nby annotating the bi-grams extracted from the target corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01682v1",
            "date": "2018-07-04 17:04:14+00:00",
            "title": "Generating Mandarin and Cantonese F0 Contours with Decision Trees and BLSTMs",
            "authors": [
                "Weidong Yuan",
                "Alan W Black"
            ],
            "abstract": "This paper models the fundamental frequency contours on both Mandarin and\nCantonese speech with decision trees and DNNs (deep neural networks). Different\nkinds of f0 representations and model architectures are tested for decision\ntrees and DNNs. A new model called Additive-BLSTM (additive bidirectional long\nshort term memory) that predicts a base f0 contour and a residual f0 contour\nwith two BLSTMs is proposed. With respect to objective measures of RMSE and\ncorrelation, applying tone-dependent trees together with sample normalization\nand delta feature regularization within decision tree framework performs best.\nWhile the new Additive-BLSTM model with delta feature regularization performs\neven better. Subjective listening tests on both Mandarin and Cantonese\ncomparing Random Forest model (multiple decision trees) and the Additive-BLSTM\nmodel were also held and confirmed the advantage of the new model according to\nthe listeners' preference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01704v1",
            "date": "2018-07-04 09:07:34+00:00",
            "title": "A Convolutional Neural Network for Aspect Sentiment Classification",
            "authors": [
                "Yongping Xing",
                "Chuangbai Xiao",
                "Yifei Wu",
                "Ziming Ding"
            ],
            "abstract": "With the development of the Internet, natural language processing (NLP), in\nwhich sentiment analysis is an important task, became vital in information\nprocessing.Sentiment analysis includes aspect sentiment classification. Aspect\nsentiment can provide complete and in-depth results with increased attention on\naspect-level. Different context words in a sentence influence the sentiment\npolarity of a sentence variably, and polarity varies based on the different\naspects in a sentence. Take the sentence, 'I bought a new camera. The picture\nquality is amazing but the battery life is too short.'as an example. If the\naspect is picture quality, then the expected sentiment polarity is 'positive',\nif the battery life aspect is considered, then the sentiment polarity should be\n'negative'; therefore, aspect is important to consider when we explore aspect\nsentiment in the sentence. Recurrent neural network (RNN) is regarded as a good\nmodel to deal with natural language processing, and RNNs has get good\nperformance on aspect sentiment classification including Target-Dependent LSTM\n(TD-LSTM) ,Target-Connection LSTM (TC-LSTM) (Tang, 2015a, b), AE-LSTM, AT-LSTM,\nAEAT-LSTM (Wang et al., 2016).There are also extensive literatures on sentiment\nclassification utilizing convolutional neural network, but there is little\nliterature on aspect sentiment classification using convolutional neural\nnetwork. In our paper, we develop attention-based input layers in which aspect\ninformation is considered by input layer. We then incorporate attention-based\ninput layers into convolutional neural network (CNN) to introduce context words\ninformation. In our experiment, incorporating aspect information into CNN\nimproves the latter's aspect sentiment classification performance without using\nsyntactic parser or external sentiment lexicons in a benchmark dataset from\nTwitter but get better performance compared with other models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01745v1",
            "date": "2018-07-04 19:09:40+00:00",
            "title": "Global Transition-based Non-projective Dependency Parsing",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Tianze Shi",
                "Lillian Lee"
            ],
            "abstract": "Shi, Huang, and Lee (2017) obtained state-of-the-art results for English and\nChinese dependency parsing by combining dynamic-programming implementations of\ntransition-based dependency parsers with a minimal set of bidirectional LSTM\nfeatures. However, their results were limited to projective parsing. In this\npaper, we extend their approach to support non-projectivity by providing the\nfirst practical implementation of the MH_4 algorithm, an $O(n^4)$ mildly\nnonprojective dynamic-programming parser with very high coverage on\nnon-projective treebanks. To make MH_4 compatible with minimal transition-based\nfeature sets, we introduce a transition-based interpretation of it in which\nparser items are mapped to sequences of transitions. We thus obtain the first\nimplementation of global decoding for non-projective transition-based parsing,\nand demonstrate empirically that it is more effective than its projective\ncounterpart in parsing a number of highly non-projective languages",
            "journal ref": "Proceedings of ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01855v1",
            "date": "2018-07-05 06:03:39+00:00",
            "title": "Zipf's law in 50 languages: its structural pattern, linguistic interpretation, and cognitive motivation",
            "authors": [
                "Shuiyuan Yu",
                "Chunshan Xu",
                "Haitao Liu"
            ],
            "abstract": "Zipf's law has been found in many human-related fields, including language,\nwhere the frequency of a word is persistently found as a power law function of\nits frequency rank, known as Zipf's law. However, there is much dispute whether\nit is a universal law or a statistical artifact, and little is known about what\nmechanisms may have shaped it. To answer these questions, this study conducted\na large scale cross language investigation into Zipf's law. The statistical\nresults show that Zipf's laws in 50 languages all share a 3-segment structural\npattern, with each segment demonstrating distinctive linguistic properties and\nthe lower segment invariably bending downwards to deviate from theoretical\nexpectation. This finding indicates that this deviation is a fundamental and\nuniversal feature of word frequency distributions in natural languages, not the\nstatistical error of low frequency words. A computer simulation based on the\ndual-process theory yields Zipf's law with the same structural pattern,\nsuggesting that Zipf's law of natural languages are motivated by common\ncognitive mechanisms. These results show that Zipf's law in languages is\nmotivated by cognitive mechanisms like dual-processing that govern human verbal\nbehaviors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01882v1",
            "date": "2018-07-05 07:45:25+00:00",
            "title": "Chinese Lexical Analysis with Deep Bi-GRU-CRF Network",
            "authors": [
                "Zhenyu Jiao",
                "Shuqi Sun",
                "Ke Sun"
            ],
            "abstract": "Lexical analysis is believed to be a crucial step towards natural language\nunderstanding and has been widely studied. Recent years, end-to-end lexical\nanalysis models with recurrent neural networks have gained increasing\nattention. In this report, we introduce a deep Bi-GRU-CRF network that jointly\nmodels word segmentation, part-of-speech tagging and named entity recognition\ntasks. We trained the model using several massive corpus pre-tagged by our best\nChinese lexical analysis tool, together with a small, yet high-quality human\nannotated corpus. We conducted balanced sampling between different corpora to\nguarantee the influence of human annotations, and fine-tune the CRF decoding\nlayer regularly during the training progress. As evaluated by linguistic\nexperts, the model achieved a 95.5% accuracy on the test set, roughly 13%\nrelative error reduction over our (previously) best Chinese lexical analysis\ntool. The model is computationally efficient, achieving the speed of 2.3K\ncharacters per second with one thread.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01996v1",
            "date": "2018-07-04 17:05:09+00:00",
            "title": "A Formal Ontology-Based Classification of Lexemes and its Applications",
            "authors": [
                "Sreekavitha Parupalli",
                "Navjyoti Singh"
            ],
            "abstract": "The paper describes the enrichment of OntoSenseNet - a verb-centric lexical\nresource for Indian Languages. A major contribution of this work is\npreservation of an authentic Telugu dictionary by developing a computational\nversion of the same. It is important because native speakers can better\nannotate the sense-types when both the word and its meaning are in Telugu.\nHence efforts are made to develop the aforementioned Telugu dictionary and\nannotations are done manually. The manually annotated gold standard corpus\nconsists 8483 verbs, 253 adverbs and 1673 adjectives. Annotations are done by\nnative speakers according to defined annotation guidelines. In this paper, we\nprovide an overview of the annotation procedure and present the validation of\nthe developed resource through inter-annotator agreement. Additional words from\nTelugu WordNet are added to our resource and are crowd-sourced for annotation.\nThe statistics are compared with the sense-annotated lexicon, our resource for\nmore insights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02200v1",
            "date": "2018-07-06 00:07:27+00:00",
            "title": "Natural Language Processing for Music Knowledge Discovery",
            "authors": [
                "Sergio Oramas",
                "Luis Espinosa-Anke",
                "Francisco G\u00f3mez",
                "Xavier Serra"
            ],
            "abstract": "Today, a massive amount of musical knowledge is stored in written form, with\ntestimonies dated as far back as several centuries ago. In this work, we\npresent different Natural Language Processing (NLP) approaches to harness the\npotential of these text collections for automatic music knowledge discovery,\ncovering different phases in a prototypical NLP pipeline, namely corpus\ncompilation, text-mining, information extraction, knowledge graph generation\nand sentiment analysis. Each of these approaches is presented alongside\ndifferent use cases (i.e., flamenco, Renaissance and popular music) where large\ncollections of documents are processed, and conclusions stemming from\ndata-driven analyses are presented and discussed.",
            "journal ref": "Journal of New Music Research (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02202v1",
            "date": "2018-07-06 00:11:27+00:00",
            "title": "The price of debiasing automatic metrics in natural language evaluation",
            "authors": [
                "Arun Tejasvi Chaganty",
                "Stephen Mussman",
                "Percy Liang"
            ],
            "abstract": "For evaluating generation systems, automatic metrics such as BLEU cost\nnothing to run but have been shown to correlate poorly with human judgment,\nleading to systematic bias against certain model improvements. On the other\nhand, averaging human judgments, the unbiased gold standard, is often too\nexpensive. In this paper, we use control variates to combine automatic metrics\nwith human evaluation to obtain an unbiased estimator with lower cost than\nhuman evaluation alone. In practice, however, we obtain only a 7-13% cost\nreduction on evaluating summarization and open-response question answering\nsystems. We then prove that our estimator is optimal: there is no unbiased\nestimator with lower cost. Our theory further highlights the two fundamental\nbottlenecks---the automatic metric and the prompt shown to human\nevaluators---both of which need to be improved to obtain greater cost savings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02226v3",
            "date": "2018-07-06 02:53:32+00:00",
            "title": "A Concept Specification and Abstraction-based Semantic Representation: Addressing the Barriers to Rule-based Machine Translation",
            "authors": [
                "Patrick Connor"
            ],
            "abstract": "Rule-based machine translation is more data efficient than the big data-based\nmachine translation approaches, making it appropriate for languages with low\nbilingual corpus resources -- i.e., minority languages. However, the rule-based\napproach has declined in popularity relative to its big data cousins primarily\nbecause of the extensive training and labour required to define the language\nrules. To address this, we present a semantic representation that 1) treats all\nbits of meaning as individual concepts that 2) modify or further specify one\nanother to build a network that relates entities in space and time. Also, the\nrepresentation can 3) encapsulate propositions and thereby define concepts in\nterms of other concepts, supporting the abstraction of underlying linguistic\nand ontological details. These features afford an exact, yet intuitive semantic\nrepresentation aimed at handling the great variety in language and reducing\nlabour and training time. The proposed natural language generation, parsing,\nand translation strategies are also amenable to probabilistic modeling and thus\nto learning the necessary rules from example data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02291v1",
            "date": "2018-07-06 07:31:13+00:00",
            "title": "Sliced Recurrent Neural Networks",
            "authors": [
                "Zeping Yu",
                "Gongshen Liu"
            ],
            "abstract": "Recurrent neural networks have achieved great success in many NLP tasks.\nHowever, they have difficulty in parallelization because of the recurrent\nstructure, so it takes much time to train RNNs. In this paper, we introduce\nsliced recurrent neural networks (SRNNs), which could be parallelized by\nslicing the sequences into many subsequences. SRNNs have the ability to obtain\nhigh-level information through multiple layers with few extra parameters. We\nprove that the standard RNN is a special case of the SRNN when we use linear\nactivation functions. Without changing the recurrent units, SRNNs are 136 times\nas fast as standard RNNs and could be even faster when we train longer\nsequences. Experiments on six largescale sentiment analysis datasets show that\nSRNNs achieve better performance than standard RNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02301v1",
            "date": "2018-07-06 08:09:37+00:00",
            "title": "Sequential Copying Networks",
            "authors": [
                "Qingyu Zhou",
                "Nan Yang",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Copying mechanism shows effectiveness in sequence-to-sequence based neural\nnetwork models for text generation tasks, such as abstractive sentence\nsummarization and question generation. However, existing works on modeling\ncopying or pointing mechanism only considers single word copying from the\nsource sentences. In this paper, we propose a novel copying framework, named\nSequential Copying Networks (SeqCopyNet), which not only learns to copy single\nwords, but also copies sequences from the input sentence. It leverages the\npointer networks to explicitly select a sub-span from the source side to target\nside, and integrates this sequential copying mechanism to the generation\nprocess in the encoder-decoder paradigm. Experiments on abstractive sentence\nsummarization and question generation tasks show that the proposed SeqCopyNet\ncan copy meaningful spans and outperforms the baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02305v1",
            "date": "2018-07-06 08:15:15+00:00",
            "title": "Neural Document Summarization by Jointly Learning to Score and Select Sentences",
            "authors": [
                "Qingyu Zhou",
                "Nan Yang",
                "Furu Wei",
                "Shaohan Huang",
                "Ming Zhou",
                "Tiejun Zhao"
            ],
            "abstract": "Sentence scoring and sentence selection are two main steps in extractive\ndocument summarization systems. However, previous works treat them as two\nseparated subtasks. In this paper, we present a novel end-to-end neural network\nframework for extractive document summarization by jointly learning to score\nand select sentences. It first reads the document sentences with a hierarchical\nencoder to obtain the representation of sentences. Then it builds the output\nsummary by extracting sentences one by one. Different from previous methods,\nour approach integrates the selection strategy into the scoring model, which\ndirectly predicts the relative importance given previously selected sentences.\nExperiments on the CNN/Daily Mail dataset show that the proposed framework\nsignificantly outperforms the state-of-the-art extractive summarization models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02478v1",
            "date": "2018-07-04 02:09:06+00:00",
            "title": "CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction",
            "authors": [
                "Chaojun Xiao",
                "Haoxi Zhong",
                "Zhipeng Guo",
                "Cunchao Tu",
                "Zhiyuan Liu",
                "Maosong Sun",
                "Yansong Feng",
                "Xianpei Han",
                "Zhen Hu",
                "Heng Wang",
                "Jianfeng Xu"
            ],
            "abstract": "In this paper, we introduce the \\textbf{C}hinese \\textbf{AI} and \\textbf{L}aw\nchallenge dataset (CAIL2018), the first large-scale Chinese legal dataset for\njudgment prediction. \\dataset contains more than $2.6$ million criminal cases\npublished by the Supreme People's Court of China, which are several times\nlarger than other datasets in existing works on judgment prediction. Moreover,\nthe annotations of judgment results are more detailed and rich. It consists of\napplicable law articles, charges, and prison terms, which are expected to be\ninferred according to the fact descriptions of cases. For comparison, we\nimplement several conventional text classification baselines for judgment\nprediction and experimental results show that it is still a challenge for\ncurrent models to predict the judgment results of legal cases, especially on\nprison terms. To help the researchers make improvements on legal judgment\nprediction, both \\dataset and baselines will be released after the CAIL\ncompetition\\footnote{http://cail.cipsc.org.cn/}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02745v1",
            "date": "2018-07-08 03:26:01+00:00",
            "title": "A Deep Generative Model of Vowel Formant Typology",
            "authors": [
                "Ryan Cotterell",
                "Jason Eisner"
            ],
            "abstract": "What makes some types of languages more probable than others? For instance,\nwe know that almost all spoken languages contain the vowel phoneme /i/; why\nshould that be? The field of linguistic typology seeks to answer these\nquestions and, thereby, divine the mechanisms that underlie human language. In\nour work, we tackle the problem of vowel system typology, i.e., we propose a\ngenerative probability model of which vowels a language contains. In contrast\nto previous work, we work directly with the acoustic information -- the first\ntwo formant values -- rather than modeling discrete sets of phonemic symbols\n(IPA). We develop a novel generative probability model and report results based\non a corpus of 233 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02747v1",
            "date": "2018-07-08 03:32:45+00:00",
            "title": "On the Complexity and Typology of Inflectional Morphological Systems",
            "authors": [
                "Ryan Cotterell",
                "Christo Kirov",
                "Mans Hulden",
                "Jason Eisner"
            ],
            "abstract": "We quantify the linguistic complexity of different languages' morphological\nsystems. We verify that there is an empirical trade-off between paradigm size\nand irregularity: a language's inflectional paradigms may be either large in\nsize or highly irregular, but never both. Our methodology measures paradigm\nirregularity as the entropy of the surface realization of a paradigm -- how\nhard it is to jointly predict all the surface forms of a paradigm. We estimate\nthis by a variational approximation. Our measurements are taken on large\nmorphological paradigms from 31 typologically diverse languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02748v2",
            "date": "2018-07-08 03:39:37+00:00",
            "title": "Latent Semantic Analysis Approach for Document Summarization Based on Word Embeddings",
            "authors": [
                "Kamal Al-Sabahi",
                "Zhang Zuping",
                "Yang Kang"
            ],
            "abstract": "Since the amount of information on the internet is growing rapidly, it is not\neasy for a user to find relevant information for his/her query. To tackle this\nissue, much attention has been paid to Automatic Document Summarization. The\nkey point in any successful document summarizer is a good document\nrepresentation. The traditional approaches based on word overlapping mostly\nfail to produce that kind of representation. Word embedding, distributed\nrepresentation of words, has shown an excellent performance that allows words\nto match on semantic level. Naively concatenating word embeddings makes the\ncommon word dominant which in turn diminish the representation quality. In this\npaper, we employ word embeddings to improve the weighting schemes for\ncalculating the input matrix of Latent Semantic Analysis method. Two\nembedding-based weighting schemes are proposed and then combined to calculate\nthe values of this matrix. The new weighting schemes are modified versions of\nthe augment weight and the entropy frequency. The new schemes combine the\nstrength of the traditional weighting schemes and word embedding. The proposed\napproach is experimentally evaluated on three well-known English datasets, DUC\n2002, DUC 2004 and Multilingual 2015 Single-document Summarization for English.\nThe proposed model performs comprehensively better compared to the\nstate-of-the-art methods, by at least 1% ROUGE points, leading to a conclusion\nthat it provides a better document representation and a better document summary\nas a result.",
            "journal ref": "KSII Transactions on Internet and Information Systems, 2019, Vol.\n  13, No.1",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02903v1",
            "date": "2018-07-09 00:44:47+00:00",
            "title": "Predicting Concreteness and Imageability of Words Within and Across Languages via Word Embeddings",
            "authors": [
                "Nikola Ljube\u0161i\u0107",
                "Darja Fi\u0161er",
                "Anita Peti-Stanti\u0107"
            ],
            "abstract": "The notions of concreteness and imageability, traditionally important in\npsycholinguistics, are gaining significance in semantic-oriented natural\nlanguage processing tasks. In this paper we investigate the predictability of\nthese two concepts via supervised learning, using word embeddings as\nexplanatory variables. We perform predictions both within and across languages\nby exploiting collections of cross-lingual embeddings aligned to a single\nvector space. We show that the notions of concreteness and imageability are\nhighly predictable both within and across languages, with a moderate loss of up\nto 20% in correlation when predicting across languages. We further show that\nthe cross-lingual transfer via word embeddings is more efficient than the\nsimple transfer via bilingual dictionaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02911v3",
            "date": "2018-07-09 01:41:20+00:00",
            "title": "A Combined CNN and LSTM Model for Arabic Sentiment Analysis",
            "authors": [
                "Abdulaziz M. Alayba",
                "Vasile Palade",
                "Matthew England",
                "Rahat Iqbal"
            ],
            "abstract": "Deep neural networks have shown good data modelling capabilities when dealing\nwith challenging and large datasets from a wide range of application areas.\nConvolutional Neural Networks (CNNs) offer advantages in selecting good\nfeatures and Long Short-Term Memory (LSTM) networks have proven good abilities\nof learning sequential data. Both approaches have been reported to provide\nimproved results in areas such image processing, voice recognition, language\ntranslation and other Natural Language Processing (NLP) tasks. Sentiment\nclassification for short text messages from Twitter is a challenging task, and\nthe complexity increases for Arabic language sentiment classification tasks\nbecause Arabic is a rich language in morphology. In addition, the availability\nof accurate pre-processing tools for Arabic is another current limitation,\nalong with limited research available in this area. In this paper, we\ninvestigate the benefits of integrating CNNs and LSTMs and report obtained\nimproved accuracy for Arabic sentiment analysis on different datasets.\nAdditionally, we seek to consider the morphological diversity of particular\nArabic words by using different sentiment classification levels.",
            "journal ref": "Proc. International Cross-Domain Conference for Machine Learning\n  and Knowledge Extraction. CD-MAKE 2018. Lecture Notes in Computer Science,\n  vol 11015, pp. 179-191. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02974v1",
            "date": "2018-07-09 07:51:51+00:00",
            "title": "Universal Word Segmentation: Implementation and Interpretation",
            "authors": [
                "Yan Shao",
                "Christian Hardmeier",
                "Joakim Nivre"
            ],
            "abstract": "Word segmentation is a low-level NLP task that is non-trivial for a\nconsiderable number of languages. In this paper, we present a sequence tagging\nframework and apply it to word segmentation for a wide range of languages with\ndifferent writing systems and typological characteristics. Additionally, we\ninvestigate the correlations between various typological factors and word\nsegmentation accuracy. The experimental results indicate that segmentation\naccuracy is positively related to word boundary markers and negatively to the\nnumber of unique non-segmental terms. Based on the analysis, we design a small\nset of language-specific settings and extensively evaluate the segmentation\nsystem on the Universal Dependencies datasets. Our model obtains\nstate-of-the-art accuracies on all the UD languages. It performs substantially\nbetter on languages that are non-trivial to segment, such as Chinese, Japanese,\nArabic and Hebrew, when compared to previous work.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  vol. 6, pp. 421--435, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03004v2",
            "date": "2018-07-09 09:27:25+00:00",
            "title": "Towards Enhancing Lexical Resource and Using Sense-annotations of OntoSenseNet for Sentiment Analysis",
            "authors": [
                "Sreekavitha Parupalli",
                "Vijjini Anvesh Rao",
                "Radhika Mamidi"
            ],
            "abstract": "This paper illustrates the interface of the tool we developed for crowd\nsourcing and we explain the annotation procedure in detail. Our tool is named\nas 'Parupalli Padajaalam' which means web of words by Parupalli. The aim of\nthis tool is to populate the OntoSenseNet, sentiment polarity annotated Telugu\nresource. Recent works have shown the importance of word-level annotations on\nsentiment analysis. With this as basis, we aim to analyze the importance of\nsense-annotations obtained from OntoSenseNet in performing the task of\nsentiment analysis. We explain the fea- tures extracted from OntoSenseNet\n(Telugu). Furthermore we compute and explain the adverbial class distribution\nof verbs in OntoSenseNet. This task is known to aid in disambiguating\nword-senses which helps in enhancing the performance of word-sense\ndisambiguation (WSD) task(s).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03006v1",
            "date": "2018-07-09 09:37:43+00:00",
            "title": "A Sequence-to-Sequence Model for Semantic Role Labeling",
            "authors": [
                "Angel Daza",
                "Anette Frank"
            ],
            "abstract": "We explore a novel approach for Semantic Role Labeling (SRL) by casting it as\na sequence-to-sequence process. We employ an attention-based model enriched\nwith a copying mechanism to ensure faithful regeneration of the input sequence,\nwhile enabling interleaved generation of argument role labels. Here, we apply\nthis model in a monolingual setting, performing PropBank SRL on English\nlanguage data. The constrained sequence generation set-up enforced with the\ncopying mechanism allows us to analyze the performance and special properties\nof the model on manually labeled data and benchmarking against state-of-the-art\nsequence labeling models. We show that our model is able to solve the SRL\nargument labeling task on English data, yet further structural decoding\nconstraints will need to be added to make the model truly competitive. Our work\nrepresents a first step towards more advanced, generative SRL labeling setups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03096v3",
            "date": "2018-07-09 13:14:00+00:00",
            "title": "NMT-Keras: a Very Flexible Toolkit with a Focus on Interactive NMT and Online Learning",
            "authors": [
                "\u00c1lvaro Peris",
                "Francisco Casacuberta"
            ],
            "abstract": "We present NMT-Keras, a flexible toolkit for training deep learning models,\nwhich puts a particular emphasis on the development of advanced applications of\nneural machine translation systems, such as interactive-predictive translation\nprotocols and long-term adaptation of the translation system via continuous\nlearning. NMT-Keras is based on an extended version of the popular Keras\nlibrary, and it runs on Theano and Tensorflow. State-of-the-art neural machine\ntranslation models are deployed and used following the high-level framework\nprovided by Keras. Given its high modularity and flexibility, it also has been\nextended to tackle different problems, such as image and video captioning,\nsentence classification and visual question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03108v1",
            "date": "2018-07-09 13:26:44+00:00",
            "title": "Discriminating between Indo-Aryan Languages Using SVM Ensembles",
            "authors": [
                "Alina Maria Ciobanu",
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Santanu Pal",
                "Liviu P. Dinu"
            ],
            "abstract": "In this paper we present a system based on SVM ensembles trained on\ncharacters and words to discriminate between five similar languages of the\nIndo-Aryan family: Hindi, Braj Bhasha, Awadhi, Bhojpuri, and Magahi. We\ninvestigate the performance of individual features and combine the output of\nsingle classifiers to maximize performance. The system competed in the\nIndo-Aryan Language Identification (ILI) shared task organized within the\nVarDial Evaluation Campaign 2018. Our best entry in the competition, named\nILIdentification, scored 88:95% F1 score and it was ranked 3rd out of 8 teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03121v3",
            "date": "2018-07-09 13:34:16+00:00",
            "title": "Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation",
            "authors": [
                "Wanxiang Che",
                "Yijia Liu",
                "Yuxuan Wang",
                "Bo Zheng",
                "Ting Liu"
            ],
            "abstract": "This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared\ntask on Multilingual Parsing from Raw Text to Universal Dependencies. We base\nour submission on Stanford's winning system for the CoNLL 2017 shared task and\nmake two effective extensions: 1) incorporating deep contextualized word\nembeddings into both the part of speech tagger and parser; 2) ensembling\nparsers trained with different initialization. We also explore different ways\nof concatenating treebanks for further improvements. Experimental results on\nthe development data show the effectiveness of our methods. In the final\nevaluation, our system was ranked first according to LAS (75.84%) and\noutperformed the other systems by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03397v1",
            "date": "2018-07-09 21:32:47+00:00",
            "title": "Detecting Levels of Depression in Text Based on Metrics",
            "authors": [
                "Ashwath Kumar Salimath",
                "Robin K Thomas",
                "Sethuram Ramalinga Reddy",
                "Yuhao Qiao"
            ],
            "abstract": "Depression is one of the most common and a major concern for society. Proper\nmonitoring using devices that can aid in its detection could be helpful to\nprevent it all together. The Distress Analysis Interview Corpus (DAIC) is used\nto build a metric-based depression detection. We have designed a metric to\ndescribe the level of depression using negative sentences and classify the\nparticipant accordingly. The score generated from the algorithm is then\nlevelled up to denote the intensity of depression. The results show that\nmeasuring depression is very complex to using text alone as other factors are\nnot taken into consideration. Further, In the paper, the limitations of\nmeasuring depression using text are described, and future suggestions are made.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03409v1",
            "date": "2018-07-09 22:28:09+00:00",
            "title": "Who is Killed by Police: Introducing Supervised Attention for Hierarchical LSTMs",
            "authors": [
                "Minh Nguyen",
                "Thien Huu Nguyen"
            ],
            "abstract": "Finding names of people killed by police has become increasingly important as\npolice shootings get more and more public attention (police killing detection).\nUnfortunately, there has been not much work in the literature addressing this\nproblem. The early work in this field \\cite{keith2017identifying} proposed a\ndistant supervision framework based on Expectation Maximization (EM) to deal\nwith the multiple appearances of the names in documents. However, such EM-based\nframework cannot take full advantages of deep learning models, necessitating\nthe use of hand-designed features to improve the detection performance. In this\nwork, we present a novel deep learning method to solve the problem of police\nkilling recognition. The proposed method relies on hierarchical LSTMs to model\nthe multiple sentences that contain the person names of interests, and\nintroduce supervised attention mechanisms based on semantical word lists and\ndependency trees to upweight the important contextual words. Our experiments\ndemonstrate the benefits of the proposed model and yield the state-of-the-art\nperformance for police killing detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03491v1",
            "date": "2018-07-10 06:26:24+00:00",
            "title": "Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme",
            "authors": [
                "Jey Han Lau",
                "Trevor Cohn",
                "Timothy Baldwin",
                "Julian Brooke",
                "Adam Hammond"
            ],
            "abstract": "In this paper, we propose a joint architecture that captures language, rhyme\nand meter for sonnet modelling. We assess the quality of generated poems using\ncrowd and expert judgements. The stress and rhyme models perform very well, as\ngenerated poems are largely indistinguishable from human-written poems. Expert\nevaluation, however, reveals that a vanilla language model captures meter\nimplicitly, and that machine-generated poems still underperform in terms of\nreadability and emotion. Our research shows the importance expert evaluation\nfor poetry generation, and that future research should look beyond rhyme/meter\nand focus on poetic language.",
            "journal ref": "Proceedings of the 56th Annual Meeting of the Association for\n  Computational Linguistics (ACL 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03583v1",
            "date": "2018-07-10 12:04:54+00:00",
            "title": "Multi-D Kneser-Ney Smoothing Preserving the Original Marginal Distributions",
            "authors": [
                "Andr\u00e1s Dob\u00f3"
            ],
            "abstract": "Smoothing is an essential tool in many NLP tasks, therefore numerous\ntechniques have been developed for this purpose in the past. One of the most\nwidely used smoothing methods are the Kneser-Ney smoothing (KNS) and its\nvariants, including the Modified Kneser-Ney smoothing (MKNS), which are widely\nconsidered to be among the best smoothing methods available. Although when\ncreating the original KNS the intention of the authors was to develop such a\nsmoothing method that preserves the marginal distributions of the original\nmodel, this property was not maintained when developing the MKNS.\n  In this article I would like to overcome this and propose such a refined\nversion of the MKNS that preserves these marginal distributions while keeping\nthe advantages of both previous versions. Beside its advantageous properties,\nthis novel smoothing method is shown to achieve about the same results as the\nMKNS in a standard language modelling task.",
            "journal ref": "Research in Computing Science, 147 (6), 11-25",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03591v1",
            "date": "2018-07-10 12:25:59+00:00",
            "title": "Paired Comparison Sentiment Scores",
            "authors": [
                "Christoph Dalitz",
                "Jens Wilberg",
                "Katrin E. Bednarek"
            ],
            "abstract": "The method of paired comparisons is an established method in psychology. In\nthis article, it is applied to obtain continuous sentiment scores for words\nfrom comparisons made by test persons. We created an initial lexicon with\n$n=199$ German words from a two-fold all-pair comparison experiment with ten\ndifferent test persons. From the probabilistic models taken into account, the\nlogistic model showed the best agreement with the results of the comparison\nexperiment. The initial lexicon can then be used in different ways. One is to\ncreate special purpose sentiment lexica through the addition of arbitrary words\nthat are compared with some of the initial words by test persons. A\ncross-validation experiment suggests that only about 18 two-fold comparisons\nare necessary to estimate the score of a new, yet unknown word, provided these\nwords are selected by a modification of a method by Silverstein & Farrell.\nAnother application of the initial lexicon is the evaluation of automatically\ncreated corpus-based lexica. By such an evaluation, we compared the\ncorpus-based lexica SentiWS, SenticNet, and SentiWordNet, of which SenticNet 4\nperformed best. This technical report is a corrected and extended version of a\npresentation made at the ICDM Sentire workshop in 2016.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03595v1",
            "date": "2018-07-10 12:46:30+00:00",
            "title": "Revisiting the Hierarchical Multiscale LSTM",
            "authors": [
                "\u00c1kos K\u00e1d\u00e1r",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi"
            ],
            "abstract": "Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art\nlanguage model that learns interpretable structure from character-level input.\nSuch models can provide fertile ground for (cognitive) computational\nlinguistics studies. However, the high complexity of the architecture, training\nprocedure and implementations might hinder its applicability. We provide a\ndetailed reproduction and ablation study of the architecture, shedding light on\nsome of the potential caveats of re-purposing complex deep-learning\narchitectures. We further show that simplifying certain aspects of the\narchitecture can in fact improve its performance. We also investigate the\nlinguistic units (segments) learned by various levels of the model, and argue\nthat their quality does not correlate with the overall performance of the model\non language modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03654v1",
            "date": "2018-07-10 14:02:35+00:00",
            "title": "Linguistic Characteristics of Censorable Language on SinaWeibo",
            "authors": [
                "Kei Yin Ng",
                "Anna Feldman",
                "Jing Peng",
                "Chris Leberknight"
            ],
            "abstract": "This paper investigates censorship from a linguistic perspective. We collect\na corpus of censored and uncensored posts on a number of topics, build a\nclassifier that predicts censorship decisions independent of discussion topics.\nOur investigation reveals that the strongest linguistic indicator of censored\ncontent of our corpus is its readability.",
            "journal ref": "1st Workshop on NLP for Internet Freedom (NLP4IF-2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03656v1",
            "date": "2018-07-10 14:03:23+00:00",
            "title": "Enriching Knowledge Bases with Counting Quantifiers",
            "authors": [
                "Paramita Mirza",
                "Simon Razniewski",
                "Fariz Darari",
                "Gerhard Weikum"
            ],
            "abstract": "Information extraction traditionally focuses on extracting relations between\nidentifiable entities, such as <Monterey, locatedIn, California>. Yet, texts\noften also contain Counting information, stating that a subject is in a\nspecific relation with a number of objects, without mentioning the objects\nthemselves, for example, \"California is divided into 58 counties\". Such\ncounting quantifiers can help in a variety of tasks such as query answering or\nknowledge base curation, but are neglected by prior work. This paper develops\nthe first full-fledged system for extracting counting information from text,\ncalled CINEX. We employ distant supervision using fact counts from a knowledge\nbase as training seeds, and develop novel techniques for dealing with several\nchallenges: (i) non-maximal training seeds due to the incompleteness of\nknowledge bases, (ii) sparse and skewed observations in text sources, and (iii)\nhigh diversity of linguistic patterns. Experiments with five human-evaluated\nrelations show that CINEX can achieve 60% average precision for extracting\ncounting information. In a large-scale experiment, we demonstrate the potential\nfor knowledge base enrichment by applying CINEX to 2,474 frequent relations in\nWikidata. CINEX can assert the existence of 2.5M facts for 110 distinct\nrelations, which is 28% more than the existing Wikidata facts for these\nrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03674v1",
            "date": "2018-07-10 14:31:25+00:00",
            "title": "IAM at CLEF eHealth 2018: Concept Annotation and Coding in French Death Certificates",
            "authors": [
                "S\u00e9bastien Cossin",
                "Vianney Jouhet",
                "Fleur Mougin",
                "Gayo Diallo",
                "Frantz Thiessard"
            ],
            "abstract": "In this paper, we describe the approach and results for our participation in\nthe task 1 (multilingual information extraction) of the CLEF eHealth 2018\nchallenge. We addressed the task of automatically assigning ICD-10 codes to\nFrench death certificates. We used a dictionary-based approach using materials\nprovided by the task organizers. The terms of the ICD-10 terminology were\nnormalized, tokenized and stored in a tree data structure. The Levenshtein\ndistance was used to detect typos. Frequent abbreviations were detected by\nmanually creating a small set of them. Our system achieved an F-score of 0.786\n(precision: 0.794, recall: 0.779). These scores were substantially higher than\nthe average score of the systems that participated in the challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03948v1",
            "date": "2018-07-11 04:45:59+00:00",
            "title": "A Dialogue Annotation Scheme for Weight Management Chat using the Trans-Theoretical Model of Health Behavior Change",
            "authors": [
                "Ramesh Manuvinakurike",
                "Sumanth Bharadwaj",
                "Kallirroi Georgila"
            ],
            "abstract": "In this study we collect and annotate human-human role-play dialogues in the\ndomain of weight management. There are two roles in the conversation: the\n\"seeker\" who is looking for ways to lose weight and the \"helper\" who provides\nsuggestions to help the \"seeker\" in their weight loss journey. The chat\ndialogues collected are then annotated with a novel annotation scheme inspired\nby a popular health behavior change theory called \"trans-theoretical model of\nhealth behavior change\". We also build classifiers to automatically predict the\nannotation labels used in our corpus. We find that classification accuracy\nimproves when oracle segmentations of the interlocutors' sentences are provided\ncompared to directly classifying unsegmented sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03950v1",
            "date": "2018-07-11 05:06:53+00:00",
            "title": "Towards Understanding End-of-trip Instructions in a Taxi Ride Scenario",
            "authors": [
                "Deepthi Karkada",
                "Ramesh Manuvinakurike",
                "Kallirroi Georgila"
            ],
            "abstract": "We introduce a dataset containing human-authored descriptions of target\nlocations in an \"end-of-trip in a taxi ride\" scenario. We describe our data\ncollection method and a novel annotation scheme that supports understanding of\nsuch descriptions of target locations. Our dataset contains target location\ndescriptions for both synthetic and real-world images as well as visual\nannotations (ground truth labels, dimensions of vehicles and objects,\ncoordinates of the target location,distance and direction of the target\nlocation from vehicles and objects) that can be used in various visual and\nlanguage tasks. We also perform a pilot experiment on how the corpus could be\napplied to visual reference resolution in this domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03955v2",
            "date": "2018-07-11 05:47:33+00:00",
            "title": "An improved neural network model for joint POS tagging and dependency parsing",
            "authors": [
                "Dat Quoc Nguyen",
                "Karin Verspoor"
            ],
            "abstract": "We propose a novel neural network model for joint part-of-speech (POS)\ntagging and dependency parsing. Our model extends the well-known BIST\ngraph-based dependency parser (Kiperwasser and Goldberg, 2016) by incorporating\na BiLSTM-based tagging component to produce automatically predicted POS tags\nfor the parser. On the benchmark English Penn treebank, our model obtains\nstrong UAS and LAS scores at 94.51% and 92.87%, respectively, producing 1.5+%\nabsolute improvements to the BIST graph-based parser, and also obtaining a\nstate-of-the-art POS tagging accuracy at 97.97%. Furthermore, experimental\nresults on parsing 61 \"big\" Universal Dependencies treebanks from raw texts\nshow that our model outperforms the baseline UDPipe (Straka and Strakov\\'a,\n2017) with 0.8% higher average POS tagging score and 3.6% higher average LAS\nscore. In addition, with our model, we also obtain state-of-the-art downstream\ntask scores for biomedical event extraction and opinion analysis applications.\nOur code is available together with all pre-trained models at:\nhttps://github.com/datquocnguyen/jPTDP",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04053v1",
            "date": "2018-07-11 10:14:48+00:00",
            "title": "UniParse: A universal graph-based parsing toolkit",
            "authors": [
                "Daniel Varab",
                "Natalie Schluter"
            ],
            "abstract": "This paper describes the design and use of the graph-based parsing framework\nand toolkit UniParse, released as an open-source python software package.\nUniParse as a framework novelly streamlines research prototyping, development\nand evaluation of graph-based dependency parsing architectures. UniParse does\nthis by enabling highly efficient, sufficiently independent, easily readable,\nand easily extensible implementations for all dependency parser components. We\ndistribute the toolkit with ready-made configurations as re-implementations of\nall current state-of-the-art first-order graph-based parsers, including even\nmore efficient Cython implementations of both encoders and decoders, as well as\nthe required specialised loss functions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04148v2",
            "date": "2018-07-11 14:12:58+00:00",
            "title": "JeSemE: A Website for Exploring Diachronic Changes in Word Meaning and Emotion",
            "authors": [
                "Johannes Hellrich",
                "Sven Buechel",
                "Udo Hahn"
            ],
            "abstract": "We here introduce a substantially extended version of JeSemE, an interactive\nwebsite for visually exploring computationally derived time-variant information\non word meanings and lexical emotions assembled from five large diachronic text\ncorpora. JeSemE is designed for scholars in the (digital) humanities as an\nalternative to consulting manually compiled, printed dictionaries for such\ninformation (if available at all). This tool uniquely combines state-of-the-art\ndistributional semantics with a nuanced model of human emotions, two\ninformation streams we deem beneficial for a data-driven interpretation of\ntexts in the humanities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04172v1",
            "date": "2018-07-11 14:48:02+00:00",
            "title": "Linear Transformations for Cross-lingual Semantic Textual Similarity",
            "authors": [
                "Tom\u00e1\u0161 Brychc\u00edn"
            ],
            "abstract": "Cross-lingual semantic textual similarity systems estimate the degree of the\nmeaning similarity between two sentences, each in a different language.\nState-of-the-art algorithms usually employ machine translation and combine vast\namount of features, making the approach strongly supervised, resource rich, and\ndifficult to use for poorly-resourced languages.\n  In this paper, we study linear transformations, which project monolingual\nsemantic spaces into a shared space using bilingual dictionaries. We propose a\nnovel transformation, which builds on the best ideas from prior works. We\nexperiment with unsupervised techniques for sentence similarity based only on\nsemantic spaces and we show they can be significantly improved by the word\nweighting. Our transformation outperforms other methods and together with word\nweighting leads to very promising results on several datasets in different\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04175v1",
            "date": "2018-07-11 14:51:35+00:00",
            "title": "Cross-lingual Word Analogies using Linear Transformations between Semantic Spaces",
            "authors": [
                "Tom\u00e1\u0161 Brychc\u00edn",
                "Stephen Eugene Taylor",
                "Luk\u00e1\u0161 Svoboda"
            ],
            "abstract": "We generalize the word analogy task across languages, to provide a new\nintrinsic evaluation method for cross-lingual semantic spaces. We experiment\nwith six languages within different language families, including English,\nGerman, Spanish, Italian, Czech, and Croatian. State-of-the-art monolingual\nsemantic spaces are transformed into a shared space using dictionaries of word\ntranslations. We compare several linear transformations and rank them for\nexperiments with monolingual (no transformation), bilingual (one semantic space\nis transformed to another), and multilingual (all semantic spaces are\ntransformed onto English space) versions of semantic spaces. We show that\ntested linear transformations preserve relationships between words (word\nanalogies) and lead to impressive results. We achieve average accuracy of\n51.1%, 43.1%, and 38.2% for monolingual, bilingual, and multilingual semantic\nspaces, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04783v2",
            "date": "2018-07-12 18:44:34+00:00",
            "title": "Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate",
            "authors": [
                "Christo Kirov",
                "Ryan Cotterell"
            ],
            "abstract": "Can advances in NLP help advance cognitive modeling? We examine the role of\nartificial neural networks, the current state of the art in many common NLP\ntasks, by returning to a classic case study. In 1986, Rumelhart and McClelland\nfamously introduced a neural architecture that learned to transduce English\nverb stems to their past tense forms. Shortly thereafter, Pinker & Prince\n(1988) presented a comprehensive rebuttal of many of Rumelhart and McClelland's\nclaims. Much of the force of their attack centered on the empirical inadequacy\nof the Rumelhart and McClelland (1986) model. Today, however, that model is\nseverely outmoded. We show that the Encoder-Decoder network architectures used\nin modern NLP systems obviate most of Pinker and Prince's criticisms without\nrequiring any simplication of the past tense mapping problem. We suggest that\nthe empirical performance of modern networks warrants a re-examination of their\nutility in linguistic and cognitive modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04990v1",
            "date": "2018-07-13 10:01:19+00:00",
            "title": "A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification",
            "authors": [
                "Zeyang Lei",
                "Yujiu Yang",
                "Min Yang",
                "Yi Liu"
            ],
            "abstract": "Deep learning approaches for sentiment classification do not fully exploit\nsentiment linguistic knowledge. In this paper, we propose a\nMulti-sentiment-resource Enhanced Attention Network (MEAN) to alleviate the\nproblem by integrating three kinds of sentiment linguistic knowledge (e.g.,\nsentiment lexicon, negation words, intensity words) into the deep neural\nnetwork via attention mechanisms. By using various types of sentiment\nresources, MEAN utilizes sentiment-relevant information from different\nrepresentation subspaces, which makes it more effective to capture the overall\nsemantics of the sentiment, negation and intensity words for sentiment\nprediction. The experimental results demonstrate that MEAN has robust\nsuperiority over strong competitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05013v1",
            "date": "2018-07-13 11:26:42+00:00",
            "title": "Multi-task dialog act and sentiment recognition on Mastodon",
            "authors": [
                "Christophe Cerisara",
                "Somayeh Jafaritazehjani",
                "Adedayo Oluokun",
                "Hoa Le"
            ],
            "abstract": "Because of license restrictions, it often becomes impossible to strictly\nreproduce most research results on Twitter data already a few months after the\ncreation of the corpus. This situation worsened gradually as time passes and\ntweets become inaccessible. This is a critical issue for reproducible and\naccountable research on social media. We partly solve this challenge by\nannotating a new Twitter-like corpus from an alternative large social medium\nwith licenses that are compatible with reproducible experiments: Mastodon. We\nmanually annotate both dialogues and sentiments on this corpus, and train a\nmulti-task hierarchical recurrent network on joint sentiment and dialog act\nrecognition. We experimentally demonstrate that transfer learning may be\nefficiently achieved between both tasks, and further analyze some specific\ncorrelations between sentiments and dialogues on social media. Both the\nannotated corpus and deep network are released with an open-source license.",
            "journal ref": "COLING, Aug 2018, Santa Fe, United States",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05127v1",
            "date": "2018-07-13 15:15:41+00:00",
            "title": "Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking",
            "authors": [
                "Shikhar Murty*",
                "Patrick Verga*",
                "Luke Vilnis",
                "Irena Radovanovic",
                "Andrew McCallum"
            ],
            "abstract": "Extraction from raw text to a knowledge base of entities and fine-grained\ntypes is often cast as prediction into a flat set of entity and type labels,\nneglecting the rich hierarchies over types and entities contained in curated\nontologies. Previous attempts to incorporate hierarchical structure have\nyielded little benefit and are restricted to shallow ontologies. This paper\npresents new methods using real and complex bilinear mappings for integrating\nhierarchical information, yielding substantial improvement over flat\npredictions in entity linking and fine-grained entity typing, and achieving new\nstate-of-the-art results for end-to-end models on the benchmark FIGER dataset.\nWe also present two new human-annotated datasets containing wide and deep\nhierarchies which we will release to the community to encourage further\nresearch in this direction: MedMentions, a collection of PubMed abstracts in\nwhich 246k mentions have been mapped to the massive UMLS ontology; and TypeNet,\nwhich aligns Freebase types with the WordNet hierarchy to obtain nearly 2k\nentity types. In experiments on all three datasets we show substantial gains\nfrom hierarchy-aware training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05195v2",
            "date": "2018-07-13 17:30:32+00:00",
            "title": "Low-Resource Text Classification using Domain-Adversarial Learning",
            "authors": [
                "Daniel Grie\u00dfhaber",
                "Ngoc Thang Vu",
                "Johannes Maucher"
            ],
            "abstract": "Deep learning techniques have recently shown to be successful in many natural\nlanguage processing tasks forming state-of-the-art systems. They require,\nhowever, a large amount of annotated data which is often missing. This paper\nexplores the use of domain-adversarial learning as a regularizer to avoid\noverfitting when training domain invariant features for deep, complex neural\nnetworks in low-resource and zero-resource settings in new target domains or\nlanguages. In case of new languages, we show that monolingual word vectors can\nbe directly used for training without prealignment. Their projection into a\ncommon space can be learnt ad-hoc at training time reaching the final\nperformance of pretrained multilingual word vectors.",
            "journal ref": "Computer Speech & Language Volume 62, July 2020, 101056 Computer\n  Speech & Language Volume 62, July 2020, 101056",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05353v2",
            "date": "2018-07-14 08:18:45+00:00",
            "title": "Recurrent Stacking of Layers for Compact Neural Machine Translation Models",
            "authors": [
                "Raj Dabre",
                "Atsushi Fujita"
            ],
            "abstract": "In neural machine translation (NMT), the most common practice is to stack a\nnumber of recurrent or feed-forward layers in the encoder and the decoder. As a\nresult, the addition of each new layer improves the translation quality\nsignificantly. However, this also leads to a significant increase in the number\nof parameters. In this paper, we propose to share parameters across all the\nlayers thereby leading to a recurrently stacked NMT model. We empirically show\nthat the translation quality of a model that recurrently stacks a single layer\n6 times is comparable to the translation quality of a model that stacks 6\nseparate layers. We also show that using pseudo-parallel corpora by\nback-translation leads to further significant improvements in translation\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05518v1",
            "date": "2018-07-15 09:23:49+00:00",
            "title": "Syllabification by Phone Categorization",
            "authors": [
                "Jacob Krantz",
                "Maxwell Dulin",
                "Paul De Palma",
                "Mark VanDam"
            ],
            "abstract": "Syllables play an important role in speech synthesis, speech recognition, and\nspoken document retrieval. A novel, low cost, and language agnostic approach to\ndividing words into their corresponding syllables is presented. A hybrid\ngenetic algorithm constructs a categorization of phones optimized for\nsyllabification. This categorization is used on top of a hidden Markov model\nsequence classifier to find syllable boundaries. The technique shows promising\npreliminary results when trained and tested on English words.",
            "journal ref": "Jacob Krantz, Maxwell Dulin, Paul De Palma, and Mark VanDam. 2018.\n  Syllabification by Phone Categorization. In Proceedings of the Genetic and\n  Evolutionary Computation Conference Companion (GECCO '18) 47-48",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05519v1",
            "date": "2018-07-15 09:36:39+00:00",
            "title": "Concept-Based Embeddings for Natural Language Processing",
            "authors": [
                "Yukun Ma",
                "Erik Cambria"
            ],
            "abstract": "In this work, we focus on effectively leveraging and integrating information\nfrom concept-level as well as word-level via projecting concepts and words into\na lower dimensional space while retaining most critical semantics. In a broad\ncontext of opinion understanding system, we investigate the use of the fused\nembedding for several core NLP tasks: named entity detection and\nclassification, automatic speech recognition reranking, and targeted sentiment\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05797v1",
            "date": "2018-07-16 11:32:40+00:00",
            "title": "The EcoLexicon English Corpus as an open corpus in Sketch Engine",
            "authors": [
                "Pilar Leon-Arauz",
                "Antonio San Martin",
                "Arianne Reimerink"
            ],
            "abstract": "The EcoLexicon English Corpus (EEC) is a 23.1-million-word corpus of\ncontemporary environmental texts. It was compiled by the LexiCon research group\nfor the development of EcoLexicon (Faber, Leon-Arauz & Reimerink 2016; San\nMartin et al. 2017), a terminological knowledge base on the environment. It is\navailable as an open corpus in the well-known corpus query system Sketch Engine\n(Kilgarriff et al. 2014), which means that any user, even without a\nsubscription, can freely access and query the corpus. In this paper, the EEC is\nintroduced by de- scribing how it was built and compiled and how it can be\nqueried and exploited, based both on the functionalities provided by Sketch\nEngine and on the parameters in which the texts in the EEC are classified.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05962v1",
            "date": "2018-07-16 16:39:11+00:00",
            "title": "Theme-weighted Ranking of Keywords from Text Documents using Phrase Embeddings",
            "authors": [
                "Debanjan Mahata",
                "John Kuriakose",
                "Rajiv Ratn Shah",
                "Roger Zimmermann",
                "John R. Talburt"
            ],
            "abstract": "Keyword extraction is a fundamental task in natural language processing that\nfacilitates mapping of documents to a concise set of representative single and\nmulti-word phrases. Keywords from text documents are primarily extracted using\nsupervised and unsupervised approaches. In this paper, we present an\nunsupervised technique that uses a combination of theme-weighted personalized\nPageRank algorithm and neural phrase embeddings for extracting and ranking\nkeywords. We also introduce an efficient way of processing text documents and\ntraining phrase embeddings using existing techniques. We share an evaluation\ndataset derived from an existing dataset that is used for choosing the\nunderlying embedding model. The evaluations for ranked keyword extraction are\nperformed on two benchmark datasets comprising of short abstracts (Inspec), and\nlong scientific papers (SemEval 2010), and is shown to produce results better\nthan the state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06008v1",
            "date": "2018-07-16 19:40:23+00:00",
            "title": "Using Textual Summaries to Describe a Set of Products",
            "authors": [
                "Kittipitch Kuptavanich"
            ],
            "abstract": "When customers are faced with the task of making a purchase in an unfamiliar\nproduct domain, it might be useful to provide them with an overview of the\nproduct set to help them understand what they can expect. In this paper we\npresent and evaluate a method to summarise sets of products in natural\nlanguage, focusing on the price range, common product features across the set,\nand product features that impact on price. In our study, participants reported\nthat they found our summaries useful, but we found no evidence that the\nsummaries influenced the selections made by participants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06151v1",
            "date": "2018-07-16 23:35:57+00:00",
            "title": "LSTMs with Attention for Aggression Detection",
            "authors": [
                "Nishant Nikhil",
                "Ramit Pahwa",
                "Mehul Kumar Nirala",
                "Rohan Khilnani"
            ],
            "abstract": "In this paper, we describe the system submitted for the shared task on\nAggression Identification in Facebook posts and comments by the team Nishnik.\nPrevious works demonstrate that LSTMs have achieved remarkable performance in\nnatural language processing tasks. We deploy an LSTM model with an attention\nunit over it. Our system ranks 6th and 4th in the Hindi subtask for Facebook\ncomments and subtask for generalized social media data respectively. And it\nranks 17th and 10th in the corresponding English subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06204v2",
            "date": "2018-07-17 04:01:06+00:00",
            "title": "Low-Resource Contextual Topic Identification on Speech",
            "authors": [
                "Chunxi Liu",
                "Matthew Wiesner",
                "Shinji Watanabe",
                "Craig Harman",
                "Jan Trmal",
                "Najim Dehak",
                "Sanjeev Khudanpur"
            ],
            "abstract": "In topic identification (topic ID) on real-world unstructured audio, an audio\ninstance of variable topic shifts is first broken into sequential segments, and\neach segment is independently classified. We first present a general purpose\nmethod for topic ID on spoken segments in low-resource languages, using a\ncascade of universal acoustic modeling, translation lexicons to English, and\nEnglish-language topic classification. Next, instead of classifying each\nsegment independently, we demonstrate that exploring the contextual\ndependencies across sequential segments can provide large improvements. In\nparticular, we propose an attention-based contextual model which is able to\nleverage the contexts in a selective manner. We test both our contextual and\nnon-contextual models on four LORELEI languages, and on all but one our\nattention-based contextual model significantly outperforms the\ncontext-independent models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06234v2",
            "date": "2018-07-17 06:05:00+00:00",
            "title": "Hierarchical Multitask Learning for CTC-based Speech Recognition",
            "authors": [
                "Kalpesh Krishna",
                "Shubham Toshniwal",
                "Karen Livescu"
            ],
            "abstract": "Previous work has shown that neural encoder-decoder speech recognition can be\nimproved with hierarchical multitask learning, where auxiliary tasks are added\nat intermediate layers of a deep encoder. We explore the effect of hierarchical\nmultitask learning in the context of connectionist temporal classification\n(CTC)-based speech recognition, and investigate several aspects of this\napproach. Consistent with previous work, we observe performance improvements on\ntelephone conversational speech recognition (specifically the Eval2000 test\nsets) when training a subword-level CTC model with an auxiliary phone loss at\nan intermediate layer. We analyze the effects of a number of experimental\nvariables (like interpolation constant and position of the auxiliary loss\nfunction), performance in lower-resource settings, and the relationship between\npretraining and multitask learning. We observe that the hierarchical multitask\napproach improves over standard multitask training in our higher-data\nexperiments, while in the low-resource settings standard multitask training\nworks well. The best results are obtained by combining hierarchical multitask\nlearning and pretraining, which improves word error rates by 3.4% absolute on\nthe Eval2000 test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06500v2",
            "date": "2018-07-17 15:26:04+00:00",
            "title": "Chinese Poetry Generation with Flexible Styles",
            "authors": [
                "Jiyuan Zhang",
                "Dong Wang"
            ],
            "abstract": "Research has shown that sequence-to-sequence neural models, particularly\nthose with the attention mechanism, can successfully generate classical Chinese\npoems. However, neural models are not capable of generating poems that match\nspecific styles, such as the impulsive style of Li Bai, a famous poet in the\nTang Dynasty. This work proposes a memory-augmented neural model to enable the\ngeneration of style-specific poetry. The key idea is a memory structure that\nstores how poems with a desired style were generated by humans, and uses\nsimilar fragments to adjust the generation. We demonstrate that the proposed\nalgorithm generates poems with flexible styles, including styles of a\nparticular era and an individual poet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06517v1",
            "date": "2018-07-17 16:00:35+00:00",
            "title": "Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing",
            "authors": [
                "Osman Ramadan",
                "Pawe\u0142 Budzianowski",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Robust dialogue belief tracking is a key component in maintaining good\nquality dialogue systems. The tasks that dialogue systems are trying to solve\nare becoming increasingly complex, requiring scalability to multi domain,\nsemantically rich dialogues. However, most current approaches have difficulty\nscaling up with domains because of the dependency of the model parameters on\nthe dialogue ontology. In this paper, a novel approach is introduced that fully\nutilizes semantic similarity between dialogue utterances and the ontology\nterms, allowing the information to be shared across domains. The evaluation is\nperformed on a recently collected multi-domain dialogues dataset, one order of\nmagnitude larger than currently available corpora. Our model demonstrates great\ncapability in handling multi-domain dialogues, simultaneously outperforming\nexisting state-of-the-art models in single-domain dialogue tracking tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06557v1",
            "date": "2018-07-17 17:04:52+00:00",
            "title": "Power Networks: A Novel Neural Architecture to Predict Power Relations",
            "authors": [
                "Michelle Lam",
                "Catherina Xu",
                "Angela Kong",
                "Vinodkumar Prabhakaran"
            ],
            "abstract": "Can language analysis reveal the underlying social power relations that exist\nbetween participants of an interaction? Prior work within NLP has shown promise\nin this area, but the performance of automatically predicting power relations\nusing NLP analysis of social interactions remains wanting. In this paper, we\npresent a novel neural architecture that captures manifestations of power\nwithin individual emails which are then aggregated in an order-preserving way\nin order to infer the direction of power between pairs of participants in an\nemail thread. We obtain an accuracy of 80.4%, a 10.1% improvement over\nstate-of-the-art methods, in this task. We further apply our model to the task\nof predicting power relations between individuals based on the entire set of\nmessages exchanged between them; here also, our model significantly outperforms\nthe70.0% accuracy using prior state-of-the-art techniques, obtaining an\naccuracy of 83.0%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06588v1",
            "date": "2018-07-17 17:59:39+00:00",
            "title": "Using semantic clustering to support situation awareness on Twitter: The case of World Views",
            "authors": [
                "Charlie Kingston",
                "Jason R. C. Nurse",
                "Ioannis Agrafiotis",
                "Andrew Milich"
            ],
            "abstract": "In recent years, situation awareness has been recognised as a critical part\nof effective decision making, in particular for crisis management. One way to\nextract value and allow for better situation awareness is to develop a system\ncapable of analysing a dataset of multiple posts, and clustering consistent\nposts into different views or stories (or, world views). However, this can be\nchallenging as it requires an understanding of the data, including determining\nwhat is consistent data, and what data corroborates other data. Attempting to\naddress these problems, this article proposes Subject-Verb-Object Semantic\nSuffix Tree Clustering (SVOSSTC) and a system to support it, with a special\nfocus on Twitter content. The novelty and value of SVOSSTC is its emphasis on\nutilising the Subject-Verb-Object (SVO) typology in order to construct\nsemantically consistent world views, in which individuals---particularly those\ninvolved in crisis response---might achieve an enhanced picture of a situation\nfrom social media data. To evaluate our system and its ability to provide\nenhanced situation awareness, we tested it against existing approaches,\nincluding human data analysis, using a variety of real-world scenarios. The\nresults indicated a noteworthy degree of evidence (e.g., in cluster granularity\nand meaningfulness) to affirm the suitability and rigour of our approach.\nMoreover, these results highlight this article's proposals as innovative and\npractical system contributions to the research field.",
            "journal ref": "Human-centric Computing and Information Sciences (HCIS) 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06683v1",
            "date": "2018-07-17 21:46:02+00:00",
            "title": "Improving Named Entity Recognition by Jointly Learning to Disambiguate Morphological Tags",
            "authors": [
                "Onur G\u00fcng\u00f6r",
                "Suzan \u00dcsk\u00fcdarl\u0131",
                "Tunga G\u00fcng\u00f6r"
            ],
            "abstract": "Previous studies have shown that linguistic features of a word such as\npossession, genitive or other grammatical cases can be employed in word\nrepresentations of a named entity recognition (NER) tagger to improve the\nperformance for morphologically rich languages. However, these taggers require\nexternal morphological disambiguation (MD) tools to function which are hard to\nobtain or non-existent for many languages. In this work, we propose a model\nwhich alleviates the need for such disambiguators by jointly learning NER and\nMD taggers in languages for which one can provide a list of candidate\nmorphological analyses. We show that this can be done independent of the\nmorphological annotation schemes, which differ among languages. Our experiments\nemploying three different model architectures that join these two tasks show\nthat joint learning improves NER performance. Furthermore, the morphological\ndisambiguator's performance is shown to be competitive.",
            "journal ref": "Proceedings of the 27th International Conference on Computational\n  Linguistics (COLING 2018). pp. 2082-2092",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06718v2",
            "date": "2018-07-18 00:38:47+00:00",
            "title": "Automatic Severity Classification of Coronary Artery Disease via Recurrent Capsule Network",
            "authors": [
                "Qi Wang",
                "Jiahui Qiu",
                "Yangming Zhou",
                "Tong Ruan",
                "Daqi Gao",
                "Ju Gao"
            ],
            "abstract": "Coronary artery disease (CAD) is one of the leading causes of cardiovascular\ndisease deaths. CAD condition progresses rapidly, if not diagnosed and treated\nat an early stage may eventually lead to an irreversible state of the heart\nmuscle death. Invasive coronary arteriography is the gold standard technique\nfor CAD diagnosis. Coronary arteriography texts describe which part has\nstenosis and how much stenosis is in details. It is crucial to conduct the\nseverity classification of CAD. In this paper, we employ a recurrent capsule\nnetwork (RCN) to extract semantic relations between clinical named entities in\nChinese coronary arteriography texts, through which we can automatically find\nout the maximal stenosis for each lumen to inference how severe CAD is\naccording to the improved method of Gensini. Experimental results on the corpus\ncollected from Shanghai Shuguang Hospital show that our proposed method\nachieves an accuracy of 97.0\\% in the severity classification of CAD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06792v2",
            "date": "2018-07-18 06:39:07+00:00",
            "title": "Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings",
            "authors": [
                "Shao-Yen Tseng",
                "Brian Baucom",
                "Panayiotis Georgiou"
            ],
            "abstract": "Unsupervised learning has been an attractive method for easily deriving\nmeaningful data representations from vast amounts of unlabeled data. These\nrepresentations, or embeddings, often yield superior results in many tasks,\nwhether used directly or as features in subsequent training stages. However,\nthe quality of the embeddings is highly dependent on the assumed knowledge in\nthe unlabeled data and how the system extracts information without supervision.\nDomain portability is also very limited in unsupervised learning, often\nrequiring re-training on other in-domain corpora to achieve robustness. In this\nwork we present a multitask paradigm for unsupervised contextual learning of\nbehavioral interactions which addresses unsupervised domain adaption. We\nintroduce an online multitask objective into unsupervised learning and show\nthat sentence embeddings generated through this process increases performance\nof affective tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06882v1",
            "date": "2018-07-18 11:58:59+00:00",
            "title": "Distinct patterns of syntactic agreement errors in recurrent networks and humans",
            "authors": [
                "Tal Linzen",
                "Brian Leonard"
            ],
            "abstract": "Determining the correct form of a verb in context requires an understanding\nof the syntactic structure of the sentence. Recurrent neural networks have been\nshown to perform this task with an error rate comparable to humans, despite the\nfact that they are not designed with explicit syntactic representations. To\nexamine the extent to which the syntactic representations of these networks are\nsimilar to those used by humans when processing sentences, we compare the\ndetailed pattern of errors that RNNs and humans make on this task. Despite\nsignificant similarities (attraction errors, asymmetry between singular and\nplural subjects), the error patterns differed in important ways. In particular,\nin complex sentences with relative clauses error rates increased in RNNs but\ndecreased in humans. Furthermore, RNNs showed a cumulative effect of attractors\nbut humans did not. We conclude that at least in some respects the syntactic\nrepresentations acquired by RNNs are fundamentally different from those used by\nhumans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06998v1",
            "date": "2018-07-18 15:37:58+00:00",
            "title": "Is it worth it? Budget-related evaluation metrics for model selection",
            "authors": [
                "Filip Klubi\u010dka",
                "Giancarlo D. Salton",
                "John D. Kelleher"
            ],
            "abstract": "Creating a linguistic resource is often done by using a machine learning\nmodel that filters the content that goes through to a human annotator, before\ngoing into the final resource. However, budgets are often limited, and the\namount of available data exceeds the amount of affordable annotation. In order\nto optimize the benefit from the invested human work, we argue that deciding on\nwhich model one should employ depends not only on generalized evaluation\nmetrics such as F-score, but also on the gain metric. Because the model with\nthe highest F-score may not necessarily have the best sequencing of predicted\nclasses, this may lead to wasting funds on annotating false positives, yielding\nzero improvement of the linguistic resource. We exemplify our point with a case\nstudy, using real data from a task of building a verb-noun idiom dictionary. We\nshow that, given the choice of three systems with varying F-scores, the system\nwith the highest F-score does not yield the highest profits. In other words, in\nour case the cost-benefit trade off is more favorable for a system with a lower\nF-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07104v5",
            "date": "2018-07-18 18:57:37+00:00",
            "title": "Hierarchical Multi Task Learning With CTC",
            "authors": [
                "Ramon Sanabria",
                "Florian Metze"
            ],
            "abstract": "In Automatic Speech Recognition it is still challenging to learn useful\nintermediate representations when using high-level (or abstract) target units\nsuch as words. For that reason, character or phoneme based systems tend to\noutperform word-based systems when just few hundreds of hours of training data\nare being used. In this paper, we first show how hierarchical multi-task\ntraining can encourage the formation of useful intermediate representations. We\nachieve this by performing Connectionist Temporal Classification at different\nlevels of the network with targets of different granularity. Our model thus\nperforms predictions in multiple scales for the same input. On the standard\n300h Switchboard training setup, our hierarchical multi-task architecture\nexhibits improvements over single-task architectures with the same number of\nparameters. Our model obtains 14.0% Word Error Rate on the Eval2000 Switchboard\nsubset without any decoder or language model, outperforming the current\nstate-of-the-art on acoustic-to-word models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07108v1",
            "date": "2018-07-18 19:10:45+00:00",
            "title": "Semantic Parsing: Syntactic assurance to target sentence using LSTM Encoder CFG-Decoder",
            "authors": [
                "Fabiano Ferreira Luz",
                "Marcelo Finger"
            ],
            "abstract": "Semantic parsing can be defined as the process of mapping natural language\nsentences into a machine interpretable, formal representation of its meaning.\nSemantic parsing using LSTM encoder-decoder neural networks have become\npromising approach. However, human automated translation of natural language\ndoes not provide grammaticality guarantees for the sentences generate such a\nguarantee is particularly important for practical cases where a data base query\ncan cause critical errors if the sentence is ungrammatical. In this work, we\npropose an neural architecture called Encoder CFG-Decoder, whose output\nconforms to a given context-free grammar. Results are show for any\nimplementation of such architecture display its correctness and providing\nbenchmark accuracy levels better than the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07279v4",
            "date": "2018-07-19 08:14:59+00:00",
            "title": "Imparting Interpretability to Word Embeddings while Preserving Semantic Structure",
            "authors": [
                "Lutfi Kerem Senel",
                "Ihsan Utlu",
                "Furkan \u015eahinu\u00e7",
                "Haldun M. Ozaktas",
                "Aykut Ko\u00e7"
            ],
            "abstract": "As an ubiquitous method in natural language processing, word embeddings are\nextensively employed to map semantic properties of words into a dense vector\nrepresentation. They capture semantic and syntactic relations among words but\nthe vectors corresponding to the words are only meaningful relative to each\nother. Neither the vector nor its dimensions have any absolute, interpretable\nmeaning. We introduce an additive modification to the objective function of the\nembedding learning algorithm that encourages the embedding vectors of words\nthat are semantically related to a predefined concept to take larger values\nalong a specified dimension, while leaving the original semantic learning\nmechanism mostly unaffected. In other words, we align words that are already\ndetermined to be related, along predefined concepts. Therefore, we impart\ninterpretability to the word embedding by assigning meaning to its vector\ndimensions. The predefined concepts are derived from an external lexical\nresource, which in this paper is chosen as Roget's Thesaurus. We observe that\nalignment along the chosen concepts is not limited to words in the Thesaurus\nand extends to other related words as well. We quantify the extent of\ninterpretability and assignment of meaning from our experimental results.\nManual human evaluation results have also been presented to further verify that\nthe proposed method increases interpretability. We also demonstrate the\npreservation of semantic coherence of the resulting vector space by using\nword-analogy and word-similarity tests. These tests show that the\ninterpretability-imparted word embeddings that are obtained by the proposed\nframework do not sacrifice performances in common benchmark tests.",
            "journal ref": "Natural Language Engineering, 1-26, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07425v2",
            "date": "2018-07-17 20:00:41+00:00",
            "title": "Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networks",
            "authors": [
                "Liang Yao",
                "Chengsheng Mao",
                "Yuan Luo"
            ],
            "abstract": "Clinical text classification is an important problem in medical natural\nlanguage processing. Existing studies have conventionally focused on rules or\nknowledge sources-based feature engineering, but only a few have exploited\neffective feature learning capability of deep learning methods. In this study,\nwe propose a novel approach which combines rule-based features and\nknowledge-guided deep learning techniques for effective disease classification.\nCritical Steps of our method include identifying trigger phrases, predicting\nclasses with very few examples using trigger phrases and training a\nconvolutional neural network with word embeddings and Unified Medical Language\nSystem (UMLS) entity embeddings. We evaluated our method on the 2008\nIntegrating Informatics with Biology and the Bedside (i2b2) obesity challenge.\nThe results show that our method outperforms the state of the art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07520v1",
            "date": "2018-07-19 16:23:35+00:00",
            "title": "Statistical Model Compression for Small-Footprint Natural Language Understanding",
            "authors": [
                "Grant P. Strimel",
                "Kanthashree Mysore Sathyendra",
                "Stanislav Peshterliev"
            ],
            "abstract": "In this paper we investigate statistical model compression applied to natural\nlanguage understanding (NLU) models. Small-footprint NLU models are important\nfor enabling offline systems on hardware restricted devices, and for decreasing\non-demand model loading latency in cloud-based systems. To compress NLU models,\nwe present two main techniques, parameter quantization and perfect feature\nhashing. These techniques are complementary to existing model pruning\nstrategies such as L1 regularization. We performed experiments on a large scale\nNLU system. The results show that our approach achieves 14-fold reduction in\nmemory usage compared to the original models with minimal predictive\nperformance impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08000v2",
            "date": "2018-07-20 18:56:32+00:00",
            "title": "Abstractive and Extractive Text Summarization using Document Context Vector and Recurrent Neural Networks",
            "authors": [
                "Chandra Khatri",
                "Gyanit Singh",
                "Nish Parikh"
            ],
            "abstract": "Sequence to sequence (Seq2Seq) learning has recently been used for\nabstractive and extractive summarization. In current study, Seq2Seq models have\nbeen used for eBay product description summarization. We propose a novel\nDocument-Context based Seq2Seq models using RNNs for abstractive and extractive\nsummarizations. Intuitively, this is similar to humans reading the title,\nabstract or any other contextual information before reading the document. This\ngives humans a high-level idea of what the document is about. We use this idea\nand propose that Seq2Seq models should be started with contextual information\nat the first time-step of the input to obtain better summaries. In this manner,\nthe output summaries are more document centric, than being generic, overcoming\none of the major hurdles of using generative models. We generate\ndocument-context from user-behavior and seller provided information. We train\nand evaluate our models on human-extracted-golden-summaries. The\ndocument-contextual Seq2Seq models outperform standard Seq2Seq models.\nMoreover, generating human extracted summaries is prohibitively expensive to\nscale, we therefore propose a semi-supervised technique for extracting\napproximate summaries and using it for training Seq2Seq models at scale.\nSemi-supervised models are evaluated against human extracted summaries and are\nfound to be of similar efficacy. We provide side by side comparison for\nabstractive and extractive summarizers (contextual and non-contextual) on same\nevaluation dataset. Overall, we provide methodologies to use and evaluate the\nproposed techniques for large document summarization. Furthermore, we found\nthese techniques to be highly effective, which is not the case with existing\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08077v1",
            "date": "2018-07-21 03:33:50+00:00",
            "title": "A Pipeline for Creative Visual Storytelling",
            "authors": [
                "Stephanie M. Lukin",
                "Reginald Hobbs",
                "Clare R. Voss"
            ],
            "abstract": "Computational visual storytelling produces a textual description of events\nand interpretations depicted in a sequence of images. These texts are made\npossible by advances and cross-disciplinary approaches in natural language\nprocessing, generation, and computer vision. We define a computational creative\nvisual storytelling as one with the ability to alter the telling of a story\nalong three aspects: to speak about different environments, to produce\nvariations based on narrative goals, and to adapt the narrative to the\naudience. These aspects of creative storytelling and their effect on the\nnarrative have yet to be explored in visual storytelling. This paper presents a\npipeline of task-modules, Object Identification, Single-Image Inferencing, and\nMulti-Image Narration, that serve as a preliminary design for building a\ncreative visual storyteller. We have piloted this design for a sequence of\nimages in an annotation task. We present and analyze the collected corpus and\ndescribe plans towards automation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08228v2",
            "date": "2018-07-22 03:15:44+00:00",
            "title": "Tree-structured multi-stage principal component analysis (TMPCA): theory and applications",
            "authors": [
                "Yuanhang Su",
                "Ruiyuan Lin",
                "C. -C. Jay Kuo"
            ],
            "abstract": "A PCA based sequence-to-vector (seq2vec) dimension reduction method for the\ntext classification problem, called the tree-structured multi-stage principal\ncomponent analysis (TMPCA) is presented in this paper. Theoretical analysis and\napplicability of TMPCA are demonstrated as an extension to our previous work\n(Su, Huang & Kuo). Unlike conventional word-to-vector embedding methods, the\nTMPCA method conducts dimension reduction at the sequence level without labeled\ntraining data. Furthermore, it can preserve the sequential structure of input\nsequences. We show that TMPCA is computationally efficient and able to\nfacilitate sequence-based text classification tasks by preserving strong mutual\ninformation between its input and output mathematically. It is also\ndemonstrated by experimental results that a dense (fully connected) network\ntrained on the TMPCA preprocessed data achieves better performance than\nstate-of-the-art fastText and other neural-network-based solutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08230v1",
            "date": "2018-07-22 03:20:19+00:00",
            "title": "German Dialect Identification Using Classifier Ensembles",
            "authors": [
                "Alina Maria Ciobanu",
                "Shervin Malmasi",
                "Liviu P. Dinu"
            ],
            "abstract": "In this paper we present the GDI_classification entry to the second German\nDialect Identification (GDI) shared task organized within the scope of the\nVarDial Evaluation Campaign 2018. We present a system based on SVM classifier\nensembles trained on characters and words. The system was trained on a\ncollection of speech transcripts of five Swiss-German dialects provided by the\norganizers. The transcripts included in the dataset contained speakers from\nBasel, Bern, Lucerne, and Zurich. Our entry in the challenge reached 62.03%\nF1-score and was ranked third out of eight teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08374v2",
            "date": "2018-07-22 21:35:01+00:00",
            "title": "Examining Scientific Writing Styles from the Perspective of Linguistic Complexity",
            "authors": [
                "Chao Lu",
                "Yi Bu",
                "Jie Wang",
                "Ying Ding",
                "Vetle Torvik",
                "Matthew Schnaars",
                "Chengzhi Zhang"
            ],
            "abstract": "Publishing articles in high-impact English journals is difficult for scholars\naround the world, especially for non-native English-speaking scholars (NNESs),\nmost of whom struggle with proficiency in English. In order to uncover the\ndifferences in English scientific writing between native English-speaking\nscholars (NESs) and NNESs, we collected a large-scale data set containing more\nthan 150,000 full-text articles published in PLoS between 2006 and 2015. We\ndivided these articles into three groups according to the ethnic backgrounds of\nthe first and corresponding authors, obtained by Ethnea, and examined the\nscientific writing styles in English from a two-fold perspective of linguistic\ncomplexity: (1) syntactic complexity, including measurements of sentence length\nand sentence complexity; and (2) lexical complexity, including measurements of\nlexical diversity, lexical density, and lexical sophistication. The\nobservations suggest marginal differences between groups in syntactical and\nlexical complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08587v2",
            "date": "2018-07-23 13:12:28+00:00",
            "title": "Deep Dialog Act Recognition using Multiple Token, Segment, and Context Information Representations",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "Dialog act (DA) recognition is a task that has been widely explored over the\nyears. Recently, most approaches to the task explored different DNN\narchitectures to combine the representations of the words in a segment and\ngenerate a segment representation that provides cues for intention. In this\nstudy, we explore means to generate more informative segment representations,\nnot only by exploring different network architectures, but also by considering\ndifferent token representations, not only at the word level, but also at the\ncharacter and functional levels. At the word level, in addition to the commonly\nused uncontextualized embeddings, we explore the use of contextualized\nrepresentations, which provide information concerning word sense and segment\nstructure. Character-level tokenization is important to capture\nintention-related morphological aspects that cannot be captured at the word\nlevel. Finally, the functional level provides an abstraction from words, which\nshifts the focus to the structure of the segment. We also explore approaches to\nenrich the segment representation with context information from the history of\nthe dialog, both in terms of the classifications of the surrounding segments\nand the turn-taking history. This kind of information has already been proved\nimportant for the disambiguation of DAs in previous studies. Nevertheless, we\nare able to capture additional information by considering a summary of the\ndialog history and a wider turn-taking context. By combining the best\napproaches at each step, we achieve results that surpass the previous\nstate-of-the-art on generic DA recognition on both SwDA and MRDA, two of the\nmost widely explored corpora for the task. Furthermore, by considering both\npast and future context, simulating annotation scenario, our approach achieves\na performance similar to that of a human annotator on SwDA and surpasses it on\nMRDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08945v1",
            "date": "2018-07-24 08:09:22+00:00",
            "title": "Otem&Utem: Over- and Under-Translation Evaluation Metric for NMT",
            "authors": [
                "Jing Yang",
                "Biao Zhang",
                "Yue Qin",
                "Xiangwen Zhang",
                "Qian Lin",
                "Jinsong Su"
            ],
            "abstract": "Although neural machine translation(NMT) yields promising translation\nperformance, it unfortunately suffers from over- and under-translation is- sues\n[Tu et al., 2016], of which studies have become research hotspots in NMT. At\npresent, these studies mainly apply the dominant automatic evaluation metrics,\nsuch as BLEU, to evaluate the overall translation quality with respect to both\nadequacy and uency. However, they are unable to accurately measure the ability\nof NMT systems in dealing with the above-mentioned issues. In this paper, we\npropose two quantitative metrics, the Otem and Utem, to automatically evaluate\nthe system perfor- mance in terms of over- and under-translation respectively.\nBoth metrics are based on the proportion of mismatched n-grams between gold\nref- erence and system translation. We evaluate both metrics by comparing their\nscores with human evaluations, where the values of Pearson Cor- relation\nCoefficient reveal their strong correlation. Moreover, in-depth analyses on\nvarious translation systems indicate some inconsistency be- tween BLEU and our\nproposed metrics, highlighting the necessity and significance of our metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08998v1",
            "date": "2018-07-24 09:48:43+00:00",
            "title": "Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need!",
            "authors": [
                "Steffen Eger",
                "Johannes Daxenberger",
                "Christian Stab",
                "Iryna Gurevych"
            ],
            "abstract": "Argumentation mining (AM) requires the identification of complex discourse\nstructures and has lately been applied with success monolingually. In this\nwork, we show that the existing resources are, however, not adequate for\nassessing cross-lingual AM, due to their heterogeneity or lack of complexity.\nWe therefore create suitable parallel corpora by (human and machine)\ntranslating a popular AM dataset consisting of persuasive student essays into\nGerman, French, Spanish, and Chinese. We then compare (i) annotation projection\nand (ii) bilingual word embeddings based direct transfer strategies for\ncross-lingual AM, finding that the former performs considerably better and\nalmost eliminates the loss from cross-lingual transfer. Moreover, we find that\nannotation projection works equally well when using either costly human or\ncheap machine translations. Our code and data are available at\n\\url{http://github.com/UKPLab/coling2018-xling_argument_mining}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09000v4",
            "date": "2018-07-24 09:56:53+00:00",
            "title": "The division of labor in communication: Speakers help listeners account for asymmetries in visual perspective",
            "authors": [
                "Robert D. Hawkins",
                "Hyowon Gweon",
                "Noah D. Goodman"
            ],
            "abstract": "Recent debates over adults' theory of mind use have been fueled by surprising\nfailures of perspective-taking in communication, suggesting that\nperspective-taking can be relatively effortful. How, then, should speakers and\nlisteners allocate their resources to achieve successful communication? We\nbegin with the observation that this shared goal induces a natural division of\nlabor: the resources one agent chooses to allocate toward perspective-taking\nshould depend on their expectations about the other's allocation. We formalize\nthis idea in a resource-rational model augmenting recent probabilistic\nweighting accounts with a mechanism for (costly) control over the degree of\nperspective-taking. In a series of simulations, we first derive an intermediate\ndegree of perspective weighting as an optimal tradeoff between expected costs\nand benefits of perspective-taking. We then present two behavioral experiments\ntesting novel predictions of our model. In Experiment 1, we manipulated the\npresence or absence of occlusions in a director-matcher task and found that\nspeakers spontaneously produced more informative descriptions to account for\n\"known unknowns\" in their partner's private view. In Experiment 2, we compared\nthe scripted utterances used by confederates in prior work with those produced\nin interactions with unscripted directors. We found that confederates were\nsystematically less informative than listeners would initially expect given the\npresence of occlusions, but listeners used violations to adaptively make fewer\nerrors over time. Taken together, our work suggests that people are not simply\n\"mindblind\"; they use contextually appropriate expectations to navigate the\ndivision of labor with their partner. We discuss how a resource rational\nframework may provide a more deeply explanatory foundation for understanding\nflexible perspective-taking under processing constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09433v3",
            "date": "2018-07-25 04:31:21+00:00",
            "title": "\"Bilingual Expert\" Can Find Translation Errors",
            "authors": [
                "Kai Fan",
                "Jiayi Wang",
                "Bo Li",
                "Fengming Zhou",
                "Boxing Chen",
                "Luo Si"
            ],
            "abstract": "Recent advances in statistical machine translation via the adoption of neural\nsequence-to-sequence models empower the end-to-end system to achieve\nstate-of-the-art in many WMT benchmarks. The performance of such machine\ntranslation (MT) system is usually evaluated by automatic metric BLEU when the\ngolden references are provided for validation. However, for model inference or\nproduction deployment, the golden references are prohibitively available or\nrequire expensive human annotation with bilingual expertise. In order to\naddress the issue of quality evaluation (QE) without reference, we propose a\ngeneral framework for automatic evaluation of translation output for most WMT\nquality evaluation tasks. We first build a conditional target language model\nwith a novel bidirectional transformer, named neural bilingual expert model,\nwhich is pre-trained on large parallel corpora for feature extraction. For QE\ninference, the bilingual expert model can simultaneously produce the joint\nlatent representation between the source and the translation, and real-valued\nmeasurements of possible erroneous tokens based on the prior knowledge learned\nfrom parallel data. Subsequently, the features will further be fed into a\nsimple Bi-LSTM predictive model for quality evaluation. The experimental\nresults show that our approach achieves the state-of-the-art performance in the\nquality estimation track of WMT 2017/2018.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09602v1",
            "date": "2018-07-23 13:58:38+00:00",
            "title": "Text Classification based on Multiple Block Convolutional Highways",
            "authors": [
                "Seyed Mahdi Rezaeinia",
                "Ali Ghodsi",
                "Rouhollah Rahmani"
            ],
            "abstract": "In the Text Classification areas of Sentiment Analysis,\nSubjectivity/Objectivity Analysis, and Opinion Polarity, Convolutional Neural\nNetworks have gained special attention because of their performance and\naccuracy. In this work, we applied recent advances in CNNs and propose a novel\narchitecture, Multiple Block Convolutional Highways (MBCH), which achieves\nimproved accuracy on multiple popular benchmark datasets, compared to previous\narchitectures. The MBCH is based on new techniques and architectures including\nhighway networks, DenseNet, batch normalization and bottleneck layers. In\naddition, to cope with the limitations of existing pre-trained word vectors\nwhich are used as inputs for the CNN, we propose a novel method, Improved Word\nVectors (IWV). The IWV improves the accuracy of CNNs which are used for text\nclassification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09671v1",
            "date": "2018-07-25 15:42:01+00:00",
            "title": "A Novel ILP Framework for Summarizing Content with High Lexical Variety",
            "authors": [
                "Wencan Luo",
                "Fei Liu",
                "Zitao Liu",
                "Diane Litman"
            ],
            "abstract": "Summarizing content contributed by individuals can be challenging, because\npeople make different lexical choices even when describing the same events.\nHowever, there remains a significant need to summarize such content. Examples\ninclude the student responses to post-class reflective questions, product\nreviews, and news articles published by different news agencies related to the\nsame events. High lexical diversity of these documents hinders the system's\nability to effectively identify salient content and reduce summary redundancy.\nIn this paper, we overcome this issue by introducing an integer linear\nprogramming-based summarization framework. It incorporates a low-rank\napproximation to the sentence-word co-occurrence matrix to intrinsically group\nsemantically-similar lexical items. We conduct extensive experiments on\ndatasets of student responses, product reviews, and news documents. Our\napproach compares favorably to a number of extractive baselines as well as a\nneural abstractive summarization system. The paper finally sheds light on when\nand why the proposed framework is effective at summarizing content with high\nlexical variety.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09950v2",
            "date": "2018-07-26 04:41:30+00:00",
            "title": "Variational Memory Encoder-Decoder",
            "authors": [
                "Hung Le",
                "Truyen Tran",
                "Thin Nguyen",
                "Svetha Venkatesh"
            ],
            "abstract": "Introducing variability while maintaining coherence is a core task in\nlearning to generate utterances in conversation. Standard neural\nencoder-decoder models and their extensions using conditional variational\nautoencoder often result in either trivial or digressive responses. To overcome\nthis, we explore a novel approach that injects variability into neural\nencoder-decoder via the use of external memory as a mixture model, namely\nVariational Memory Encoder-Decoder (VMED). By associating each memory read with\na mode in the latent mixture distribution at each timestep, our model can\ncapture the variability observed in sequential data such as natural\nconversations. We empirically compare the proposed model against other recent\napproaches on various conversational datasets. The results show that VMED\nconsistently achieves significant improvement over others in both metric-based\nand qualitative evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10076v3",
            "date": "2018-07-26 11:44:19+00:00",
            "title": "Concurrent Learning of Semantic Relations",
            "authors": [
                "Georgios Balikas",
                "Ga\u00ebl Dias",
                "Rumen Moraliyski",
                "Massih-Reza Amini"
            ],
            "abstract": "Discovering whether words are semantically related and identifying the\nspecific semantic relation that holds between them is of crucial importance for\nNLP as it is essential for tasks like query expansion in IR. Within this\ncontext, different methodologies have been proposed that either exclusively\nfocus on a single lexical relation (e.g. hypernymy vs. random) or learn\nspecific classifiers capable of identifying multiple semantic relations (e.g.\nhypernymy vs. synonymy vs. random). In this paper, we propose another way to\nlook at the problem that relies on the multi-task learning paradigm. In\nparticular, we want to study whether the learning process of a given semantic\nrelation (e.g. hypernymy) can be improved by the concurrent learning of another\nsemantic relation (e.g. co-hyponymy). Within this context, we particularly\nexamine the benefits of semi-supervised learning where the training of a\nprediction function is performed over few labeled data jointly with many\nunlabeled ones. Preliminary results based on simple learning strategies and\nstate-of-the-art distributional feature representations show that concurrent\nlearning can lead to improvements in a vast majority of tested situations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10311v1",
            "date": "2018-07-26 18:31:08+00:00",
            "title": "Open Source Automatic Speech Recognition for German",
            "authors": [
                "Benjamin Milde",
                "Arne K\u00f6hn"
            ],
            "abstract": "High quality Automatic Speech Recognition (ASR) is a prerequisite for\nspeech-based applications and research. While state-of-the-art ASR software is\nfreely available, the language dependent acoustic models are lacking for\nlanguages other than English, due to the limited amount of freely available\ntraining data. We train acoustic models for German with Kaldi on two datasets,\nwhich are both distributed under a Creative Commons license. The resulting\nmodel is freely redistributable, lowering the cost of entry for German ASR. The\nmodels are trained on a total of 412 hours of German read speech data and we\nachieve a relative word error reduction of 26% by adding data from the Spoken\nWikipedia Corpus to the previously best freely available German acoustic model\nrecipe and dataset. Our best model achieves a word error rate of 14.38 on the\nTuda-De test set. Due to the large amount of speakers and the diversity of\ntopics included in the training data, our model is robust against speaker\nvariation and topic shift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10543v3",
            "date": "2018-07-27 12:00:21+00:00",
            "title": "Automatic Short Answer Grading and Feedback Using Text Mining Methods",
            "authors": [
                "Neslihan Suzen",
                "Alexander Gorban",
                "Jeremy Levesley",
                "Evgeny Mirkes"
            ],
            "abstract": "Automatic grading is not a new approach but the need to adapt the latest\ntechnology to automatic grading has become very important. As the technology\nhas rapidly became more powerful on scoring exams and essays, especially from\nthe 1990s onwards, partially or wholly automated grading systems using\ncomputational methods have evolved and have become a major area of research. In\nparticular, the demand of scoring of natural language responses has created a\nneed for tools that can be applied to automatically grade these responses. In\nthis paper, we focus on the concept of automatic grading of short answer\nquestions such as are typical in the UK GCSE system, and providing useful\nfeedback on their answers to students. We present experimental results on a\ndataset provided from the introductory computer science class in the University\nof North Texas. We first apply standard data mining techniques to the corpus of\nstudent answers for the purpose of measuring similarity between the student\nanswers and the model answer. This is based on the number of common words. We\nthen evaluate the relation between these similarities and marks awarded by\nscorers. We then consider an approach that groups student answers into\nclusters. Each cluster would be awarded the same mark, and the same feedback\ngiven to each answer in a cluster. In this manner, we demonstrate that clusters\nindicate the groups of students who are awarded the same or the similar scores.\nWords in each cluster are compared to show that clusters are constructed based\non how many and which words of the model answer have been used. The main\nnovelty in this paper is that we design a model to predict marks based on the\nsimilarities between the student answers and the model answer.",
            "journal ref": "Procedia Computer Science 169 (2020), 726-743",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10564v4",
            "date": "2018-07-27 13:03:06+00:00",
            "title": "Auto-Encoding Variational Neural Machine Translation",
            "authors": [
                "Bryan Eikema",
                "Wilker Aziz"
            ],
            "abstract": "We present a deep generative model of bilingual sentence pairs for machine\ntranslation. The model generates source and target sentences jointly from a\nshared latent representation and is parameterised by neural networks. We\nperform efficient training using amortised variational inference and\nreparameterised gradients. Additionally, we discuss the statistical\nimplications of joint modelling and propose an efficient approximation to\nmaximum a posteriori decoding for fast test-time predictions. We demonstrate\nthe effectiveness of our model in three machine translation scenarios:\nin-domain training, mixed-domain training, and learning from a mix of\ngold-standard and synthetic data. Our experiments show consistently that our\njoint formulation outperforms conditional modelling (i.e. standard neural\nmachine translation) in all such scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10661v1",
            "date": "2018-07-27 14:40:32+00:00",
            "title": "Concept Tagging for Natural Language Understanding: Two Decadelong Algorithm Development",
            "authors": [
                "Jacopo Gobbi",
                "Evgeny Stepanov",
                "Giuseppe Riccardi"
            ],
            "abstract": "Concept tagging is a type of structured learning needed for natural language\nunderstanding (NLU) systems. In this task, meaning labels from a domain\nontology are assigned to word sequences. In this paper, we review the\nalgorithms developed over the last twenty five years. We perform a comparative\nevaluation of generative, discriminative and deep learning methods on two\npublic datasets. We report on the statistical variability performance\nmeasurements. The third contribution is the release of a repository of the\nalgorithms, datasets and recipes for NLU evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10740v1",
            "date": "2018-07-27 17:29:20+00:00",
            "title": "A small Griko-Italian speech translation corpus",
            "authors": [
                "Marcely Zanon Boito",
                "Antonios Anastasopoulos",
                "Marika Lekakou",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "This paper presents an extension to a very low-resource parallel corpus\ncollected in an endangered language, Griko, making it useful for computational\nresearch. The corpus consists of 330 utterances (about 20 minutes of speech)\nwhich have been transcribed and translated in Italian, with annotations for\nword-level speech-to-transcription and speech-to-translation alignments. The\ncorpus also includes morphosyntactic tags and word-level glosses. Applying an\nautomatic unit discovery method, pseudo-phones were also generated. We detail\nhow the corpus was collected, cleaned and processed, and we illustrate its use\non zero-resource tasks by presenting some baseline results for the task of\nspeech-to-translation alignment and unsupervised word discovery. The dataset is\navailable online, aiming to encourage replicability and diversity in\ncomputational language documentation experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10800v2",
            "date": "2018-07-27 19:00:01+00:00",
            "title": "Clustering Prominent People and Organizations in Topic-Specific Text Corpora",
            "authors": [
                "Abdulkareem Alsudais",
                "Hovig Tchalian"
            ],
            "abstract": "Named entities in text documents are the names of people, organization,\nlocation or other types of objects in the documents that exist in the real\nworld. A persisting research challenge is to use computational techniques to\nidentify such entities in text documents. Once identified, several text mining\ntools and algorithms can be utilized to leverage these discovered named\nentities and improve NLP applications. In this paper, a method that clusters\nprominent names of people and organizations based on their semantic similarity\nin a text corpus is proposed. The method relies on common named entity\nrecognition techniques and on recent word embeddings models. The semantic\nsimilarity scores generated using the word embeddings models for the named\nentities are used to cluster similar entities of the people and organizations\ntypes. Two human judges evaluated ten variations of the method after it was run\non a corpus that consists of 4,821 articles on a specific topic. The\nperformance of the method was measured using three quantitative measures. The\nresults of these three metrics demonstrate that the method is effective in\nclustering semantically similar named entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10854v3",
            "date": "2018-07-27 23:11:39+00:00",
            "title": "A Survey of the Usages of Deep Learning in Natural Language Processing",
            "authors": [
                "Daniel W. Otter",
                "Julian R. Medina",
                "Jugal K. Kalita"
            ],
            "abstract": "Over the last several years, the field of natural language processing has\nbeen propelled forward by an explosion in the use of deep learning models. This\nsurvey provides a brief introduction to the field and a quick overview of deep\nlearning architectures and methods. It then sifts through the plethora of\nrecent studies and summarizes a large assortment of relevant contributions.\nAnalyzed research areas include several core linguistic processing issues in\naddition to a number of applications of computational linguistics. A discussion\nof the current state of the art is then provided along with recommendations for\nfuture research in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10893v1",
            "date": "2018-07-28 05:32:11+00:00",
            "title": "Back-Translation-Style Data Augmentation for End-to-End ASR",
            "authors": [
                "Tomoki Hayashi",
                "Shinji Watanabe",
                "Yu Zhang",
                "Tomoki Toda",
                "Takaaki Hori",
                "Ramon Astudillo",
                "Kazuya Takeda"
            ],
            "abstract": "In this paper we propose a novel data augmentation method for attention-based\nend-to-end automatic speech recognition (E2E-ASR), utilizing a large amount of\ntext which is not paired with speech signals. Inspired by the back-translation\ntechnique proposed in the field of machine translation, we build a neural\ntext-to-encoder model which predicts a sequence of hidden states extracted by a\npre-trained E2E-ASR encoder from a sequence of characters. By using hidden\nstates as a target instead of acoustic features, it is possible to achieve\nfaster attention learning and reduce computational cost, thanks to sub-sampling\nin E2E-ASR encoder, also the use of the hidden states can avoid to model\nspeaker dependencies unlike acoustic features. After training, the\ntext-to-encoder model generates the hidden states from a large amount of\nunpaired text, then E2E-ASR decoder is retrained using the generated hidden\nstates as additional training data. Experimental evaluation using LibriSpeech\ndataset demonstrates that our proposed method achieves improvement of ASR\nperformance and reduces the number of unknown words without the need for paired\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10945v1",
            "date": "2018-07-28 14:59:52+00:00",
            "title": "Acoustic and Textual Data Augmentation for Improved ASR of Code-Switching Speech",
            "authors": [
                "Emre Y\u0131lmaz",
                "Henk van den Heuvel",
                "David A. van Leeuwen"
            ],
            "abstract": "In this paper, we describe several techniques for improving the acoustic and\nlanguage model of an automatic speech recognition (ASR) system operating on\ncode-switching (CS) speech. We focus on the recognition of Frisian-Dutch radio\nbroadcasts where one of the mixed languages, namely Frisian, is an\nunder-resourced language. In previous work, we have proposed several automatic\ntranscription strategies for CS speech to increase the amount of available\ntraining speech data. In this work, we explore how the acoustic modeling (AM)\ncan benefit from monolingual speech data belonging to the high-resourced mixed\nlanguage. For this purpose, we train state-of-the-art AMs, which were\nineffective due to lack of training data, on a significantly increased amount\nof CS speech and monolingual Dutch speech. Moreover, we improve the language\nmodel (LM) by creating code-switching text, which is in practice almost\nnon-existent, by (1) generating text using recurrent LMs trained on the\ntranscriptions of the training CS speech data, (2) adding the transcriptions of\nthe automatically transcribed CS speech data and (3) translating Dutch text\nextracted from the transcriptions of a large Dutch speech corpora. We report\nsignificantly improved CS ASR performance due to the increase in the acoustic\nand textual training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10948v1",
            "date": "2018-07-28 15:04:53+00:00",
            "title": "Articulatory Features for ASR of Pathological Speech",
            "authors": [
                "Emre Y\u0131lmaz",
                "Vikramjit Mitra",
                "Chris Bartels",
                "Horacio Franco"
            ],
            "abstract": "In this work, we investigate the joint use of articulatory and acoustic\nfeatures for automatic speech recognition (ASR) of pathological speech. Despite\nlong-lasting efforts to build speaker- and text-independent ASR systems for\npeople with dysarthria, the performance of state-of-the-art systems is still\nconsiderably lower on this type of speech than on normal speech. The most\nprominent reason for the inferior performance is the high variability in\npathological speech that is characterized by the spectrotemporal deviations\ncaused by articulatory impairments due to various etiologies. To cope with this\nhigh variation, we propose to use speech representations which utilize\narticulatory information together with the acoustic properties. A designated\nacoustic model, namely a fused-feature-map convolutional neural network (fCNN),\nwhich performs frequency convolution on acoustic features and time convolution\non articulatory features is trained and tested on a Dutch and a Flemish\npathological speech corpus. The ASR performance of fCNN-based ASR system using\njoint features is compared to other neural network architectures such\nconventional CNNs and time-frequency convolutional networks (TFCNNs) in several\ntraining scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10949v1",
            "date": "2018-07-28 15:09:55+00:00",
            "title": "Building a Unified Code-Switching ASR System for South African Languages",
            "authors": [
                "Emre Y\u0131lmaz",
                "Astik Biswas",
                "Ewald van der Westhuizen",
                "Febe de Wet",
                "Thomas Niesler"
            ],
            "abstract": "We present our first efforts towards building a single multilingual automatic\nspeech recognition (ASR) system that can process code-switching (CS) speech in\nfive languages spoken within the same population. This contrasts with related\nprior work which focuses on the recognition of CS speech in bilingual\nscenarios. Recently, we have compiled a small five-language corpus of South\nAfrican soap opera speech which contains examples of CS between 5 languages\noccurring in various contexts such as using English as the matrix language and\nswitching to other indigenous languages. The ASR system presented in this work\nis trained on 4 corpora containing English-isiZulu, English-isiXhosa,\nEnglish-Setswana and English-Sesotho CS speech. The interpolation of multiple\nlanguage models trained on these language pairs enables the ASR system to\nhypothesize mixed word sequences from these 5 languages. We evaluate various\nstate-of-the-art acoustic models trained on this 5-lingual training data and\nreport ASR accuracy and language recognition performance on the development and\ntest sets of the South African multilingual soap opera corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11057v3",
            "date": "2018-07-29 13:49:00+00:00",
            "title": "NMT-based Cross-lingual Document Embeddings",
            "authors": [
                "Wei Li",
                "Brian Mak"
            ],
            "abstract": "This paper investigates a cross-lingual document embedding method that\nimproves the current Neural machine Translation framework based Document Vector\n(NTDV or simply NV). NV is developed with a self-attention mechanism under the\nneural machine translation (NMT) framework. In NV, each pair of parallel\ndocuments in different languages are projected to the same shared layer in the\nmodel. However, the pair of NV embeddings are not guaranteed to be similar.\nThis paper further adds a distance constraint to the training objective\nfunction of NV so that the two embeddings of a parallel document are required\nto be as close as possible. The new method will be called constrained NV (cNV).\nIn a cross-lingual document classification task, the new cNV performs as well\nas NV and outperforms other published studies that require forward-pass\ndecoding. Compared with the previous NV, cNV does not need a translator during\ntesting, and so the method is lighter and more flexible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11082v1",
            "date": "2018-07-29 16:43:06+00:00",
            "title": "Convolutional Gated Recurrent Units for Medical Relation Classification",
            "authors": [
                "Bin He",
                "Yi Guan",
                "Rui Dai"
            ],
            "abstract": "Convolutional neural network (CNN) and recurrent neural network (RNN) models\nhave become the mainstream methods for relation classification. We propose a\nunified architecture, which exploits the advantages of CNN and RNN\nsimultaneously, to identify medical relations in clinical records, with only\nword embedding features. Our model learns phrase-level features through a CNN\nlayer, and these feature representations are directly fed into a bidirectional\ngated recurrent unit (GRU) layer to capture long-term feature dependencies. We\nevaluate our model on two clinical datasets, and experiments demonstrate that\nour model performs significantly better than previous single-model methods on\nboth datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11172v1",
            "date": "2018-07-30 04:59:43+00:00",
            "title": "Leveraging Medical Sentiment to Understand Patients Health on Social Media",
            "authors": [
                "Shweta Yadav",
                "Joy Sain",
                "Amit Sheth",
                "Asif Ekbal",
                "Sriparna Saha",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "The unprecedented growth of Internet users in recent years has resulted in an\nabundance of unstructured information in the form of social media text. A large\npercentage of this population is actively engaged in health social networks to\nshare health-related information. In this paper, we address an important and\ntimely topic by analyzing the users' sentiments and emotions w.r.t their\nmedical conditions. Towards this, we examine users on popular medical forums\n(Patient.info,dailystrength.org), where they post on important topics such as\nasthma, allergy, depression, and anxiety. First, we provide a benchmark setup\nfor the task by crawling the data, and further define the sentiment specific\nfine-grained medical conditions (Recovered, Exist, Deteriorate, and Other). We\npropose an effective architecture that uses a Convolutional Neural Network\n(CNN) as a data-driven feature extractor and a Support Vector Machine (SVM) as\na classifier. We further develop a sentiment feature which is sensitive to the\nmedical context. Here, we show that the use of medical sentiment feature along\nwith extracted features from CNN improves the model performance. In addition to\nour dataset, we also evaluate our approach on the benchmark \"CLEF eHealth 2014\"\ncorpora and show that our model outperforms the state-of-the-art techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11219v1",
            "date": "2018-07-30 08:11:52+00:00",
            "title": "Training Neural Machine Translation using Word Embedding-based Loss",
            "authors": [
                "Katsuki Chousa",
                "Katsuhito Sudoh",
                "Satoshi Nakamura"
            ],
            "abstract": "In neural machine translation (NMT), the computational cost at the output\nlayer increases with the size of the target-side vocabulary. Using a\nlimited-size vocabulary instead may cause a significant decrease in translation\nquality. This trade-off is derived from a softmax-based loss function that\nhandles in-dictionary words independently, in which word similarity is not\nconsidered. In this paper, we propose a novel NMT loss function that includes\nword similarity in forms of distances in a word embedding space. The proposed\nloss function encourages an NMT decoder to generate words close to their\nreferences in the embedding space; this helps the decoder to choose similar\nacceptable words when the actual best candidates are not included in the\nvocabulary due to its size limitation. In experiments using ASPEC\nJapanese-to-English and IWSLT17 English-to-French data sets, the proposed\nmethod showed improvements against a standard NMT baseline in both datasets;\nespecially with IWSLT17 En-Fr, it achieved up to +1.72 in BLEU and +1.99 in\nMETEOR. When the target-side vocabulary was very limited to 1,000 words, the\nproposed method demonstrated a substantial gain, +1.72 in METEOR with ASPEC\nJa-En.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11243v2",
            "date": "2018-07-30 09:11:26+00:00",
            "title": "Active Learning for Interactive Neural Machine Translation of Data Streams",
            "authors": [
                "\u00c1lvaro Peris",
                "Francisco Casacuberta"
            ],
            "abstract": "We study the application of active learning techniques to the translation of\nunbounded data streams via interactive neural machine translation. The main\nidea is to select, from an unbounded stream of source sentences, those worth to\nbe supervised by a human agent. The user will interactively translate those\nsamples. Once validated, these data is useful for adapting the neural machine\ntranslation model.\n  We propose two novel methods for selecting the samples to be validated. We\nexploit the information from the attention mechanism of a neural machine\ntranslation system. Our experiments show that the inclusion of active learning\ntechniques into this pipeline allows to reduce the effort required during the\nprocess, while increasing the quality of the translation system. Moreover, it\nenables to balance the human effort required for achieving a certain\ntranslation quality. Moreover, our neural system outperforms classical\napproaches by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11276v1",
            "date": "2018-07-30 10:31:52+00:00",
            "title": "Graphene: Semantically-Linked Propositions in Open Information Extraction",
            "authors": [
                "Matthias Cetto",
                "Christina Niklaus",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We present an Open Information Extraction (IE) approach that uses a\ntwo-layered transformation stage consisting of a clausal disembedding layer and\na phrasal disembedding layer, together with rhetorical relation identification.\nIn that way, we convert sentences that present a complex linguistic structure\ninto simplified, syntactically sound sentences, from which we can extract\npropositions that are represented in a two-layered hierarchy in the form of\ncore relational tuples and accompanying contextual information which are\nsemantically linked via rhetorical relations. In a comparative evaluation, we\ndemonstrate that our reference implementation Graphene outperforms\nstate-of-the-art Open IE systems in the construction of correct n-ary\npredicate-argument structures. Moreover, we show that existing Open IE\napproaches can benefit from the transformation process of our framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11535v2",
            "date": "2018-07-30 19:16:09+00:00",
            "title": "News Article Teaser Tweets and How to Generate Them",
            "authors": [
                "Sanjeev Kumar Karn",
                "Mark Buckley",
                "Ulli Waltinger",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In this work, we define the task of teaser generation and provide an\nevaluation benchmark and baseline systems for the process of generating\nteasers. A teaser is a short reading suggestion for an article that is\nillustrative and includes curiosity-arousing elements to entice potential\nreaders to read particular news items. Teasers are one of the main vehicles for\ntransmitting news to social media users. We compile a novel dataset of teasers\nby systematically accumulating tweets and selecting those that conform to the\nteaser definition. We have compared a number of neural abstractive\narchitectures on the task of teaser generation and the overall best performing\nsystem is See et al.(2017)'s seq2seq with pointer network.",
            "journal ref": "2019 Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics (NAACL 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11584v1",
            "date": "2018-07-30 21:14:25+00:00",
            "title": "UH-PRHLT at SemEval-2016 Task 3: Combining Lexical and Semantic-based Features for Community Question Answering",
            "authors": [
                "Marc Franco-Salvador",
                "Sudipta Kar",
                "Thamar Solorio",
                "Paolo Rosso"
            ],
            "abstract": "In this work we describe the system built for the three English subtasks of\nthe SemEval 2016 Task 3 by the Department of Computer Science of the University\nof Houston (UH) and the Pattern Recognition and Human Language Technology\n(PRHLT) research center - Universitat Polit`ecnica de Val`encia: UH-PRHLT. Our\nsystem represents instances by using both lexical and semantic-based similarity\nmeasures between text pairs. Our semantic features include the use of\ndistributed representations of words, knowledge graphs generated with the\nBabelNet multilingual semantic network, and the FrameNet lexical database.\nExperimental results outperform the random and Google search engine baselines\nin the three English subtasks. Our approach obtained the highest results of\nsubtask B compared to the other task participants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11605v1",
            "date": "2018-07-30 23:13:55+00:00",
            "title": "Doubly Attentive Transformer Machine Translation",
            "authors": [
                "Hasan Sait Arslan",
                "Mark Fishel",
                "Gholamreza Anbarjafari"
            ],
            "abstract": "In this paper a doubly attentive transformer machine translation model\n(DATNMT) is presented in which a doubly-attentive transformer decoder normally\njoins spatial visual features obtained via pretrained convolutional neural\nnetworks, conquering any gap between image captioning and translation. In this\nframework, the transformer decoder figures out how to take care of\nsource-language words and parts of an image freely by methods for two separate\nattention components in an Enhanced Multi-Head Attention Layer of doubly\nattentive transformer, as it generates words in the target language. We find\nthat the proposed model can effectively exploit not just the scarce multimodal\nmachine translation data, but also large general-domain text-only machine\ntranslation corpora, or image-text image captioning corpora. The experimental\nresults show that the proposed doubly-attentive transformer-decoder performs\nbetter than a single-decoder transformer model, and gives the state-of-the-art\nresults in the English-German multimodal machine translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11618v1",
            "date": "2018-07-31 00:50:15+00:00",
            "title": "An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization",
            "authors": [
                "Kamal Al-Sabahi",
                "Zuping Zhang",
                "Jun Long",
                "Khaled Alwesabi"
            ],
            "abstract": "The fast-growing amount of information on the Internet makes the research in\nautomatic document summarization very urgent. It is an effective solution for\ninformation overload. Many approaches have been proposed based on different\nstrategies, such as latent semantic analysis (LSA). However, LSA, when applied\nto document summarization, has some limitations which diminish its performance.\nIn this work, we try to overcome these limitations by applying statistic and\nlinear algebraic approaches combined with syntactic and semantic processing of\ntext. First, the part of speech tagger is utilized to reduce the dimension of\nLSA. Then, the weight of the term in four adjacent sentences is added to the\nweighting schemes while calculating the input matrix to take into account the\nword order and the syntactic relations. In addition, a new LSA-based sentence\nselection algorithm is proposed, in which the term description is combined with\nsentence description for each topic which in turn makes the generated summary\nmore informative and diverse. To ensure the effectiveness of the proposed\nLSA-based sentence selection algorithm, extensive experiment on Arabic and\nEnglish are done. Four datasets are used to evaluate the new model, Linguistic\nData Consortium (LDC) Arabic Newswire-a corpus, Essex Arabic Summaries Corpus\n(EASC), DUC2002, and Multilingual MSS 2015 dataset. Experimental results on the\nfour datasets show the effectiveness of the proposed model on Arabic and\nEnglish datasets. It performs comprehensively better compared to the\nstate-of-the-art methods.",
            "journal ref": "K. Al-Sabahi, Z. Zhang, J. Long, and K. Alwesabi, \"An Enhanced\n  Latent Semantic Analysis Approach for Arabic Document Summarization,\" Arabian\n  Journal for Science and Engineering, journal article May 05 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11712v1",
            "date": "2018-07-31 09:21:22+00:00",
            "title": "RiTUAL-UH at TRAC 2018 Shared Task: Aggression Identification",
            "authors": [
                "Niloofar Safi Samghabadi",
                "Deepthi Mave",
                "Sudipta Kar",
                "Thamar Solorio"
            ],
            "abstract": "This paper presents our system for \"TRAC 2018 Shared Task on Aggression\nIdentification\". Our best systems for the English dataset use a combination of\nlexical and semantic features. However, for Hindi data using only lexical\nfeatures gave us the best results. We obtained weighted F1- measures of 0.5921\nfor the English Facebook task (ranked 12th), 0.5663 for the English Social\nMedia task (ranked 6th), 0.6292 for the Hindi Facebook task (ranked 1st), and\n0.4853 for the Hindi Social Media task (ranked 2nd).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11714v2",
            "date": "2018-07-31 09:27:27+00:00",
            "title": "Gender Bias in Neural Natural Language Processing",
            "authors": [
                "Kaiji Lu",
                "Piotr Mardziel",
                "Fangjing Wu",
                "Preetam Amancharla",
                "Anupam Datta"
            ],
            "abstract": "We examine whether neural natural language processing (NLP) systems reflect\nhistorical biases in training data. We define a general benchmark to quantify\ngender bias in a variety of neural NLP tasks. Our empirical evaluation with\nstate-of-the-art neural coreference resolution and textbook RNN-based language\nmodels trained on benchmark datasets finds significant gender bias in how\nmodels view occupations. We then mitigate bias with CDA: a generic methodology\nfor corpus augmentation via causal interventions that breaks associations\nbetween gendered and gender-neutral words. We empirically show that CDA\neffectively decreases gender bias while preserving accuracy. We also explore\nthe space of mitigation strategies with CDA, a prior approach to word embedding\ndebiasing (WED), and their compositions. We show that CDA outperforms WED,\ndrastically so when word embeddings are trained. For pre-trained embeddings,\nthe two methods can be effectively composed. We also find that as training\nproceeds on the original data set with gradient descent the gender bias grows\nas the loss reduces, indicating that the optimization encourages bias; CDA\nmitigates this behavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11906v2",
            "date": "2018-07-31 16:32:50+00:00",
            "title": "Effective Parallel Corpus Mining using Bilingual Sentence Embeddings",
            "authors": [
                "Mandy Guo",
                "Qinlan Shen",
                "Yinfei Yang",
                "Heming Ge",
                "Daniel Cer",
                "Gustavo Hernandez Abrego",
                "Keith Stevens",
                "Noah Constant",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "This paper presents an effective approach for parallel corpus mining using\nbilingual sentence embeddings. Our embedding models are trained to produce\nsimilar representations exclusively for bilingual sentence pairs that are\ntranslations of each other. This is achieved using a novel training method that\nintroduces hard negatives consisting of sentences that are not translations but\nthat have some degree of semantic similarity. The quality of the resulting\nembeddings are evaluated on parallel corpus reconstruction and by assessing\nmachine translation systems trained on gold vs. mined sentence pairs. We find\nthat the sentence embeddings can be used to reconstruct the United Nations\nParallel Corpus at the sentence level with a precision of 48.9% for en-fr and\n54.9% for en-es. When adapted to document level matching, we achieve a parallel\ndocument matching accuracy that is comparable to the significantly more\ncomputationally intensive approach of [Jakob 2010]. Using reconstructed\nparallel data, we are able to train NMT models that perform nearly as well as\nmodels trained on the original data (within 1-2 BLEU).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00013v1",
            "date": "2018-08-31 18:00:27+00:00",
            "title": "Gromov-Wasserstein Alignment of Word Embedding Spaces",
            "authors": [
                "David Alvarez-Melis",
                "Tommi S. Jaakkola"
            ],
            "abstract": "Cross-lingual or cross-domain correspondences play key roles in tasks ranging\nfrom machine translation to transfer learning. Recently, purely unsupervised\nmethods operating on monolingual embeddings have become effective alignment\ntools. Current state-of-the-art methods, however, involve multiple steps,\nincluding heuristic post-hoc refinement strategies. In this paper, we cast the\ncorrespondence problem directly as an optimal transport (OT) problem, building\non the idea that word embeddings arise from metric recovery algorithms. Indeed,\nwe exploit the Gromov-Wasserstein distance that measures how similarities\nbetween pairs of words relate across languages. We show that our OT objective\ncan be estimated efficiently, requires little or no tuning, and results in\nperformance comparable with the state-of-the-art in various unsupervised word\ntranslation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00042v1",
            "date": "2018-08-31 20:04:42+00:00",
            "title": "What do RNN Language Models Learn about Filler-Gap Dependencies?",
            "authors": [
                "Ethan Wilcox",
                "Roger Levy",
                "Takashi Morita",
                "Richard Futrell"
            ],
            "abstract": "RNN language models have achieved state-of-the-art perplexity results and\nhave proven useful in a suite of NLP tasks, but it is as yet unclear what\nsyntactic generalizations they learn. Here we investigate whether\nstate-of-the-art RNN language models represent long-distance filler-gap\ndependencies and constraints on them. Examining RNN behavior on experimentally\ncontrolled sentences designed to expose filler-gap dependencies, we show that\nRNNs can represent the relationship in multiple syntactic positions and over\nlarge spans of text. Furthermore, we show that RNNs learn a subset of the known\nrestrictions on filler-gap dependencies, known as island constraints: RNNs show\nevidence for wh-islands, adjunct islands, and complex NP islands. These studies\ndemonstrates that state-of-the-art RNN models are able to learn and generalize\nabout empty syntactic positions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00064v3",
            "date": "2018-08-31 21:20:00+00:00",
            "title": "Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction",
            "authors": [
                "Yova Kementchedjhieva",
                "Sebastian Ruder",
                "Ryan Cotterell",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Most recent approaches to bilingual dictionary induction find a linear\nalignment between the word vector spaces of two languages. We show that\nprojecting the two languages onto a third, latent space, rather than directly\nonto each other, while equivalent in terms of expressivity, makes it easier to\nlearn approximate alignments. Our modified approach also allows for supporting\nlanguages to be included in the alignment process, to obtain an even better\nperformance in low resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00066v1",
            "date": "2018-08-31 21:27:54+00:00",
            "title": "Indicatements that character language models learn English morpho-syntactic units and regularities",
            "authors": [
                "Yova Kementchedjhieva",
                "Adam Lopez"
            ],
            "abstract": "Character language models have access to surface morphological patterns, but\nit is not clear whether or how they learn abstract morphological regularities.\nWe instrument a character language model with several probes, finding that it\ncan develop a specific unit to identify word boundaries and, by extension,\nmorpheme boundaries, which allows it to capture linguistic properties and\nregularities of these units. Our language model proves surprisingly good at\nidentifying the selectional restrictions of English derivational morphemes, a\ntask that requires both morphological and syntactic awareness. Thus we conclude\nthat, when morphemes overlap extensively with the words of a language, a\ncharacter language model can perform morphological abstraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00069v1",
            "date": "2018-08-31 22:01:48+00:00",
            "title": "When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)",
            "authors": [
                "Liang Huang",
                "Kai Zhao",
                "Mingbo Ma"
            ],
            "abstract": "In neural text generation such as neural machine translation, summarization,\nand image captioning, beam search is widely used to improve the output text\nquality. However, in the neural generation setting, hypotheses can finish in\ndifferent steps, which makes it difficult to decide when to end beam search to\nensure optimality. We propose a provably optimal beam search algorithm that\nwill always return the optimal-score complete hypothesis (modulo beam size),\nand finish as soon as the optimality is established (finishing no later than\nthe baseline). To counter neural generation's tendency for shorter hypotheses,\nwe also introduce a bounded length reward mechanism which allows a modified\nversion of our beam search algorithm to remain optimal. Experiments on neural\nmachine translation demonstrate that our principled beam search algorithm leads\nto improvement in BLEU score over previously proposed alternatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00070v1",
            "date": "2018-08-31 22:07:19+00:00",
            "title": "Nightmare at test time: How punctuation prevents parsers from generalizing",
            "authors": [
                "Anders S\u00f8gaard",
                "Miryam de Lhoneux",
                "Isabelle Augenstein"
            ],
            "abstract": "Punctuation is a strong indicator of syntactic structure, and parsers trained\non text with punctuation often rely heavily on this signal. Punctuation is a\ndiversion, however, since human language processing does not rely on\npunctuation to the same extent, and in informal texts, we therefore often leave\nout punctuation. We also use punctuation ungrammatically for emphatic or\ncreative purposes, or simply by mistake. We show that (a) dependency parsers\nare sensitive to both absence of punctuation and to alternative uses; (b)\nneural parsers tend to be more sensitive than vintage parsers; (c) training\nneural parsers without punctuation outperforms all out-of-the-box parsers\nacross all scenarios where punctuation departs from standard punctuation. Our\nmain experiments are on synthetically corrupted data to study the effect of\npunctuation in isolation and avoid potential confounds, but we also show\neffects on out-of-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00107v1",
            "date": "2018-09-01 03:07:17+00:00",
            "title": "Dependency-based Hybrid Trees for Semantic Parsing",
            "authors": [
                "Zhanming Jie",
                "Wei Lu"
            ],
            "abstract": "We propose a novel dependency-based hybrid tree model for semantic parsing,\nwhich converts natural language utterance into machine interpretable meaning\nrepresentations. Unlike previous state-of-the-art models, the semantic\ninformation is interpreted as the latent dependency between the natural\nlanguage words in our joint representation. Such dependency information can\ncapture the interactions between the semantics and natural language words. We\nintegrate a neural component into our model and propose an efficient\ndynamic-programming algorithm to perform tractable inference. Through extensive\nexperiments on the standard multilingual GeoQuery dataset with eight languages,\nwe demonstrate that our proposed approach is able to achieve state-of-the-art\nperformance across several languages. Analysis also justifies the effectiveness\nof using our new dependency-based representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00120v2",
            "date": "2018-09-01 06:06:20+00:00",
            "title": "Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter",
            "authors": [
                "Lijun Wu",
                "Xu Tan",
                "Di He",
                "Fei Tian",
                "Tao Qin",
                "Jianhuang Lai",
                "Tie-Yan Liu"
            ],
            "abstract": "Neural machine translation usually adopts autoregressive models and suffers\nfrom exposure bias as well as the consequent error propagation problem. Many\nprevious works have discussed the relationship between error propagation and\nthe \\emph{accuracy drop} (i.e., the left part of the translated sentence is\noften better than its right part in left-to-right decoding models) problem. In\nthis paper, we conduct a series of analyses to deeply understand this problem\nand get several interesting findings. (1) The role of error propagation on\naccuracy drop is overstated in the literature, although it indeed contributes\nto the accuracy drop problem. (2) Characteristics of a language play a more\nimportant role in causing the accuracy drop: the left part of the translation\nresult in a right-branching language (e.g., English) is more likely to be more\naccurate than its right part, while the right part is more accurate for a\nleft-branching language (e.g., Japanese). Our discoveries are confirmed on\ndifferent model structures including Transformer and RNN, and in other sequence\ngeneration tasks such as text summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00125v2",
            "date": "2018-09-01 06:39:56+00:00",
            "title": "Simple Fusion: Return of the Language Model",
            "authors": [
                "Felix Stahlberg",
                "James Cross",
                "Veselin Stoyanov"
            ],
            "abstract": "Neural Machine Translation (NMT) typically leverages monolingual data in\ntraining through backtranslation. We investigate an alternative simple method\nto use monolingual data for NMT training: We combine the scores of a\npre-trained and fixed language model (LM) with the scores of a translation\nmodel (TM) while the TM is trained from scratch. To achieve that, we train the\ntranslation model to predict the residual probability of the training data\nadded to the prediction of the LM. This enables the TM to focus its capacity on\nmodeling the source sentence since it can rely on the LM for fluency. We show\nthat our method outperforms previous approaches to integrate LMs into NMT while\nthe architecture is simpler as it does not require gating networks to balance\nTM and LM. We observe gains of between +0.24 and +2.36 BLEU on all four test\nsets (English-Turkish, Turkish-English, Estonian-English, Xhosa-English) on top\nof ensembles without LM. We compare our method with alternative ways to utilize\nmonolingual data such as backtranslation, shallow fusion, and cold fusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00129v1",
            "date": "2018-09-01 08:01:29+00:00",
            "title": "Contextual Encoding for Translation Quality Estimation",
            "authors": [
                "Junjie Hu",
                "Wei-Cheng Chang",
                "Yuexin Wu",
                "Graham Neubig"
            ],
            "abstract": "The task of word-level quality estimation (QE) consists of taking a source\nsentence and machine-generated translation, and predicting which words in the\noutput are correct and which are wrong.\n  In this paper, propose a method to effectively encode the local and global\ncontextual information for each target word using a three-part neural network\napproach.\n  The first part uses an embedding layer to represent words and their\npart-of-speech tags in both languages. The second part leverages a\none-dimensional convolution layer to integrate local context information for\neach target word. The third part applies a stack of feed-forward and recurrent\nneural networks to further encode the global context in the sentence before\nmaking the predictions. This model was submitted as the CMU entry to the\nWMT2018 shared task on QE, and achieves strong results, ranking first in three\nof the six tracks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00150v1",
            "date": "2018-09-01 10:31:57+00:00",
            "title": "Why is unsupervised alignment of English embeddings from different algorithms so hard?",
            "authors": [
                "Mareike Hartmann",
                "Yova Kementchedjhieva",
                "Anders S\u00f8gaard"
            ],
            "abstract": "This paper presents a challenge to the community: Generative adversarial\nnetworks (GANs) can perfectly align independent English word embeddings induced\nusing the same algorithm, based on distributional information alone; but fails\nto do so, for two different embeddings algorithms. Why is that? We believe\nunderstanding why, is key to understand both modern word embedding algorithms\nand the limitations and instability dynamics of GANs. This paper shows that (a)\nin all these cases, where alignment fails, there exists a linear transform\nbetween the two embeddings (so algorithm biases do not lead to non-linear\ndifferences), and (b) similar effects can not easily be obtained by varying\nhyper-parameters. One plausible suggestion based on our initial experiments is\nthat the differences in the inductive biases of the embedding algorithms lead\nto an optimization landscape that is riddled with local optima, leading to a\nvery small basin of convergence, but we present this more as a challenge paper\nthan a technical contribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00151v1",
            "date": "2018-09-01 10:54:33+00:00",
            "title": "LIUM-CVC Submissions for WMT18 Multimodal Translation Task",
            "authors": [
                "Ozan Caglayan",
                "Adrien Bardet",
                "Fethi Bougares",
                "Lo\u00efc Barrault",
                "Kai Wang",
                "Marc Masana",
                "Luis Herranz",
                "Joost van de Weijer"
            ],
            "abstract": "This paper describes the multimodal Neural Machine Translation systems\ndeveloped by LIUM and CVC for WMT18 Shared Task on Multimodal Translation. This\nyear we propose several modifications to our previous multimodal attention\narchitecture in order to better integrate convolutional features and refine\nthem using encoder-side information. Our final constrained submissions ranked\nfirst for English-French and second for English-German language pairs among the\nconstrained submissions according to the automatic evaluation metric METEOR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00188v1",
            "date": "2018-09-01 14:10:50+00:00",
            "title": "MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing",
            "authors": [
                "Marcin Junczys-Dowmunt",
                "Roman Grundkiewicz"
            ],
            "abstract": "This paper describes the Microsoft and University of Edinburgh submission to\nthe Automatic Post-editing shared task at WMT2018. Based on training data and\nsystems from the WMT2017 shared task, we re-implement our own models from the\nlast shared task and introduce improvements based on extensive parameter\nsharing. Next we experiment with our implementation of dual-source transformer\nmodels and data selection for the IT domain. Our submissions decisively wins\nthe SMT post-editing sub-task establishing the new state-of-the-art and is a\nvery close second (or equal, 16.46 vs 16.50 TER) in the NMT sub-task. Based on\nthe rather weak results in the NMT sub-task, we hypothesize that\nneural-on-neural APE might not be actually useful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00196v1",
            "date": "2018-09-01 14:33:11+00:00",
            "title": "Microsoft's Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data",
            "authors": [
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "This paper describes the Microsoft submission to the WMT2018 news translation\nshared task. We participated in one language direction -- English-German. Our\nsystem follows current best-practice and combines state-of-the-art models with\nnew data filtering (dual conditional cross-entropy filtering) and sentence\nweighting methods. We trained fairly standard Transformer-big models with an\nupdated version of Edinburgh's training scheme for WMT2017 and experimented\nwith different filtering schemes for Paracrawl. According to automatic metrics\n(BLEU) we reached the highest score for this subtask with a nearly 2 BLEU point\nmargin over the next strongest system. Based on human evaluation we ranked\nfirst among constrained systems. We believe this is mostly caused by our data\nfiltering/weighting regime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00197v2",
            "date": "2018-09-01 14:38:16+00:00",
            "title": "Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora",
            "authors": [
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "In this work we introduce dual conditional cross-entropy filtering for noisy\nparallel data. For each sentence pair of the noisy parallel corpus we compute\ncross-entropy scores according to two inverse translation models trained on\nclean data. We penalize divergent cross-entropies and weigh the penalty by the\ncross-entropy average of both models. Sorting or thresholding according to\nthese scores results in better subsets of parallel data. We achieve higher BLEU\nscores with models trained on parallel data filtered only from Paracrawl than\nwith models trained on clean WMT data. We further evaluate our method in the\ncontext of the WMT2018 shared task on parallel corpus filtering and achieve the\noverall highest ranking scores of the shared task, scoring top in three out of\nfour subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00221v1",
            "date": "2018-09-01 16:54:15+00:00",
            "title": "A Multilingual Information Extraction Pipeline for Investigative Journalism",
            "authors": [
                "Gregor Wiedemann",
                "Seid Muhie Yimam",
                "Chris Biemann"
            ],
            "abstract": "We introduce an advanced information extraction pipeline to automatically\nprocess very large collections of unstructured textual data for the purpose of\ninvestigative journalism. The pipeline serves as a new input processor for the\nupcoming major release of our New/s/leak 2.0 software, which we develop in\ncooperation with a large German news organization. The use case is that\njournalists receive a large collection of files up to several Gigabytes\ncontaining unknown contents. Collections may originate either from official\ndisclosures of documents, e.g. Freedom of Information Act requests, or\nunofficial data leaks. Our software prepares a visually-aided exploration of\nthe collection to quickly learn about potential stories contained in the data.\nIt is based on the automatic extraction of entities and their co-occurrence in\ndocuments. In contrast to comparable projects, we focus on the following three\nmajor requirements particularly serving the use case of investigative\njournalism in cross-border collaborations: 1) composition of multiple\nstate-of-the-art NLP tools for entity extraction, 2) support of multi-lingual\ndocument sets up to 40 languages, 3) fast and easy-to-use extraction of\nfull-text, metadata and entities from various file formats.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00303v1",
            "date": "2018-09-02 06:22:39+00:00",
            "title": "Towards Automated Customer Support",
            "authors": [
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Recent years have seen growing interest in conversational agents, such as\nchatbots, which are a very good fit for automated customer support because the\ndomain in which they need to operate is narrow. This interest was in part\ninspired by recent advances in neural machine translation, esp. the rise of\nsequence-to-sequence (seq2seq) and attention-based models such as the\nTransformer, which have been applied to various other tasks and have opened new\nresearch directions in question answering, chatbots, and conversational\nsystems. Still, in many cases, it might be feasible and even preferable to use\nsimple information retrieval techniques. Thus, here we compare three different\nmodels:(i) a retrieval model, (ii) a sequence-to-sequence model with attention,\nand (iii) Transformer. Our experiments with the Twitter Customer Support\nDataset, which contains over two million posts from customer support services\nof twenty major brands, show that the seq2seq model outperforms the other two\nin terms of semantics and word overlap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00315v1",
            "date": "2018-09-02 09:16:18+00:00",
            "title": "Exploring Gap Filling as a Cheaper Alternative to Reading Comprehension Questionnaires when Evaluating Machine Translation for Gisting",
            "authors": [
                "Mikel L. Forcada",
                "Carolina Scarton",
                "Lucia Specia",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "A popular application of machine translation (MT) is gisting: MT is consumed\nas is to make sense of text in a foreign language. Evaluation of the usefulness\nof MT for gisting is surprisingly uncommon. The classical method uses reading\ncomprehension questionnaires (RCQ), in which informants are asked to answer\nprofessionally-written questions in their language about a foreign text that\nhas been machine-translated into their language. Recently, gap-filling (GF), a\nform of cloze testing, has been proposed as a cheaper alternative to RCQ. In\nGF, certain words are removed from reference translations and readers are asked\nto fill the gaps left using the machine-translated text as a hint. This paper\nreports, for thefirst time, a comparative evaluation, using both RCQ and GF, of\ntranslations from multiple MT systems for the same foreign texts, and a\nsystematic study on the effect of variables such as gap density, gap-selection\nstrategies, and document context in GF. The main findings of the study are: (a)\nboth RCQ and GF clearly identify MT to be useful, (b) global RCQ and GF\nrankings for the MT systems are mostly in agreement, (c) GF scores vary very\nwidely across informants, making comparisons among MT systems hard, and (d)\nunlike RCQ, which is framed around documents, GF evaluation can be framed at\nthe sentence level. These findings support the use of GF as a cheaper\nalternative to RCQ.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00336v1",
            "date": "2018-09-02 13:47:03+00:00",
            "title": "Future-Prediction-Based Model for Neural Machine Translation",
            "authors": [
                "Bingzhen Wei",
                "Junyang Lin"
            ],
            "abstract": "We propose a novel model for Neural Machine Translation (NMT). Different from\nthe conventional method, our model can predict the future text length and words\nat each decoding time step so that the generation can be helped with the\ninformation from the future prediction. With such information, the model does\nnot stop generation without having translated enough content. Experimental\nresults demonstrate that our model can significantly outperform the baseline\nmodels. Besides, our analysis reflects that our model is effective in the\nprediction of the length and words of the untranslated content.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00344v1",
            "date": "2018-09-02 14:26:44+00:00",
            "title": "Contextual Neural Model for Translating Bilingual Multi-Speaker Conversations",
            "authors": [
                "Sameen Maruf",
                "Andr\u00e9 F. T. Martins",
                "Gholamreza Haffari"
            ],
            "abstract": "Recent works in neural machine translation have begun to explore document\ntranslation. However, translating online multi-speaker conversations is still\nan open problem. In this work, we propose the task of translating Bilingual\nMulti-Speaker Conversations, and explore neural architectures which exploit\nboth source and target-side conversation histories for this task. To initiate\nan evaluation for this task, we introduce datasets extracted from Europarl v7\nand OpenSubtitles2016. Our experiments on four language-pairs confirm the\nsignificance of leveraging conversation history, both in terms of BLEU and\nmanual evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00357v1",
            "date": "2018-09-02 15:24:15+00:00",
            "title": "Trivial Transfer Learning for Low-Resource Neural Machine Translation",
            "authors": [
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Transfer learning has been proven as an effective technique for neural\nmachine translation under low-resource conditions. Existing methods require a\ncommon target language, language relatedness, or specific training tricks and\nregimes. We present a simple transfer learning method, where we first train a\n\"parent\" model for a high-resource language pair and then continue the training\non a lowresource pair only by replacing the training corpus. This \"child\" model\nperforms significantly better than the baseline trained for lowresource pair\nonly. We are the first to show this for targeting different languages, and we\nobserve the improvements even for unrelated languages with different alphabets.",
            "journal ref": "Proceedings of the Third Conference on Machine Translation:\n  Research Papers 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00370v1",
            "date": "2018-09-02 17:36:51+00:00",
            "title": "Neural Ranking Models for Temporal Dependency Structure Parsing",
            "authors": [
                "Yuchen Zhang",
                "Nianwen Xue"
            ],
            "abstract": "We design and build the first neural temporal dependency parser. It utilizes\na neural ranking model with minimal feature engineering, and parses time\nexpressions and events in a text into a temporal dependency tree structure. We\nevaluate our parser on two domains: news reports and narrative stories. In a\nparsing-only evaluation setup where gold time expressions and events are\nprovided, our parser reaches 0.81 and 0.70 f-score on unlabeled and labeled\nparsing respectively, a result that is very competitive against alternative\napproaches. In an end-to-end evaluation setup where time expressions and events\nare automatically recognized, our parser beats two strong baselines on both\ndata domains. Our experimental results and discussions shed light on the nature\nof temporal dependency structures in different domains and provide insights\nthat we believe will be valuable to future research in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00378v1",
            "date": "2018-09-02 19:36:03+00:00",
            "title": "Neural Character-based Composition Models for Abuse Detection",
            "authors": [
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "The advent of social media in recent years has fed into some highly\nundesirable phenomena such as proliferation of offensive language, hate speech,\nsexist remarks, etc. on the Internet. In light of this, there have been several\nefforts to automate the detection and moderation of such abusive content.\nHowever, deliberate obfuscation of words by users to evade detection poses a\nserious challenge to the effectiveness of these efforts. The current state of\nthe art approaches to abusive language detection, based on recurrent neural\nnetworks, do not explicitly address this problem and resort to a generic OOV\n(out of vocabulary) embedding for unseen words. However, in using a single\nembedding for all unseen words we lose the ability to distinguish between\nobfuscated and non-obfuscated or rare words. In this paper, we address this\nproblem by designing a model that can compose embeddings for unseen words. We\nexperimentally demonstrate that our approach significantly advances the current\nstate of the art in abuse detection on datasets from two different domains,\nnamely Twitter and Wikipedia talk page.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00388v1",
            "date": "2018-09-02 20:43:09+00:00",
            "title": "MTNT: A Testbed for Machine Translation of Noisy Text",
            "authors": [
                "Paul Michel",
                "Graham Neubig"
            ],
            "abstract": "Noisy or non-standard input text can cause disastrous mistranslations in most\nmodern Machine Translation (MT) systems, and there has been growing research\ninterest in creating noise-robust MT systems. However, as of yet there are no\npublicly available parallel corpora of with naturally occurring noisy inputs\nand translations, and thus previous work has resorted to evaluating on\nsynthetically created datasets. In this paper, we propose a benchmark dataset\nfor Machine Translation of Noisy Text (MTNT), consisting of noisy comments on\nReddit (www.reddit.com) and professionally sourced translations. We\ncommissioned translations of English comments into French and Japanese, as well\nas French and Japanese comments into English, on the order of 7k-37k sentences\nper language pair. We qualitatively and quantitatively examine the types of\nnoise included in this dataset, then demonstrate that existing MT models fail\nbadly on a number of noise-related phenomena, even after performing adaptation\non a small training set of in-domain data. This indicates that this dataset can\nprovide an attractive testbed for methods tailored to handling noisy text in\nMT. The data is publicly available at www.cs.cmu.edu/~pmichel1/mtnt/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00428v1",
            "date": "2018-09-03 02:12:56+00:00",
            "title": "Data Augmentation for Neural Online Chat Response Selection",
            "authors": [
                "Wenchao Du",
                "Alan W Black"
            ],
            "abstract": "Data augmentation seeks to manipulate the available data for training to\nimprove the generalization ability of models. We investigate two data\naugmentation proxies, permutation and flipping, for neural dialog response\nselection task on various models over multiple datasets, including both Chinese\nand English languages. Different from standard data augmentation techniques,\nour method combines the original and synthesized data for prediction. Empirical\nresults show that our approach can gain 1 to 3 recall-at-1 points over baseline\nmodels in both full-scale and small-scale settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00530v1",
            "date": "2018-09-03 10:15:04+00:00",
            "title": "Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification",
            "authors": [
                "Ruidan He",
                "Wee Sun Lee",
                "Hwee Tou Ng",
                "Daniel Dahlmeier"
            ],
            "abstract": "We consider the cross-domain sentiment classification problem, where a\nsentiment classifier is to be learned from a source domain and to be\ngeneralized to a target domain. Our approach explicitly minimizes the distance\nbetween the source and the target instances in an embedded feature space. With\nthe difference between source and target minimized, we then exploit additional\ninformation from the target domain by consolidating the idea of semi-supervised\nlearning, for which, we jointly employ two regularizations -- entropy\nminimization and self-ensemble bootstrapping -- to incorporate the unlabeled\ntarget data for classifier refinement. Our experimental results demonstrate\nthat the proposed approach can better leverage unlabeled data from the target\ndomain and achieve substantial improvements over baseline methods in various\nexperimental settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00537v1",
            "date": "2018-09-03 10:29:09+00:00",
            "title": "Crowdsourcing Semantic Label Propagation in Relation Classification",
            "authors": [
                "Anca Dumitrache",
                "Lora Aroyo",
                "Chris Welty"
            ],
            "abstract": "Distant supervision is a popular method for performing relation extraction\nfrom text that is known to produce noisy labels. Most progress in relation\nextraction and classification has been made with crowdsourced corrections to\ndistant-supervised labels, and there is evidence that indicates still more\nwould be better. In this paper, we explore the problem of propagating human\nannotation signals gathered for open-domain relation classification through the\nCrowdTruth methodology for crowdsourcing, that captures ambiguity in\nannotations by measuring inter-annotator disagreement. Our approach propagates\nannotations to sentences that are similar in a low dimensional embedding space,\nexpanding the number of labels by two orders of magnitude. Our experiments show\nsignificant improvement in a sentence-level multi-class relation classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00563v1",
            "date": "2018-09-03 11:48:28+00:00",
            "title": "End-to-End Argument Mining for Discussion Threads Based on Parallel Constrained Pointer Architecture",
            "authors": [
                "Gaku Morio",
                "Katsuhide Fujita"
            ],
            "abstract": "Argument Mining (AM) is a relatively recent discipline, which concentrates on\nextracting claims or premises from discourses, and inferring their structures.\nHowever, many existing works do not consider micro-level AM studies on\ndiscussion threads sufficiently. In this paper, we tackle AM for discussion\nthreads. Our main contributions are follows: (1) A novel combination scheme\nfocusing on micro-level inner- and inter- post schemes for a discussion thread.\n(2) Annotation of large-scale civic discussion threads with the scheme. (3)\nParallel constrained pointer architecture (PCPA), a novel end-to-end technique\nto discriminate sentence types, inner-post relations, and inter-post\ninteractions simultaneously. The experimental results demonstrate that our\nproposed model shows better accuracy in terms of relations extraction, in\ncomparison to existing state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00582v2",
            "date": "2018-09-03 12:41:44+00:00",
            "title": "Data-to-Text Generation with Content Selection and Planning",
            "authors": [
                "Ratish Puduppully",
                "Li Dong",
                "Mirella Lapata"
            ],
            "abstract": "Recent advances in data-to-text generation have led to the use of large-scale\ndatasets and neural network models which are trained end-to-end, without\nexplicitly modeling what to say and in what order. In this work, we present a\nneural network architecture which incorporates content selection and planning\nwithout sacrificing end-to-end training. We decompose the generation task into\ntwo stages. Given a corpus of data records (paired with descriptive documents),\nwe first generate a content plan highlighting which information should be\nmentioned and in which order and then generate the document while taking the\ncontent plan into account. Automatic and human-based evaluation experiments\nshow that our model outperforms strong baselines improving the state-of-the-art\non the recently released RotoWire dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00589v1",
            "date": "2018-09-03 13:11:07+00:00",
            "title": "Affordance Extraction and Inference based on Semantic Role Labeling",
            "authors": [
                "Daniel Loureiro",
                "Al\u00edpio M\u00e1rio Jorge"
            ],
            "abstract": "Common-sense reasoning is becoming increasingly important for the advancement\nof Natural Language Processing. While word embeddings have been very\nsuccessful, they cannot explain which aspects of 'coffee' and 'tea' make them\nsimilar, or how they could be related to 'shop'. In this paper, we propose an\nexplicit word representation that builds upon the Distributional Hypothesis to\nrepresent meaning from semantic roles, and allow inference of relations from\ntheir meshing, as supported by the affordance-based Indexical Hypothesis. We\nfind that our model improves the state-of-the-art on unsupervised word\nsimilarity tasks while allowing for direct inference of new relations from the\nsame vector space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00640v1",
            "date": "2018-09-03 16:17:11+00:00",
            "title": "Deep learning for language understanding of mental health concepts derived from Cognitive Behavioural Therapy",
            "authors": [
                "Lina Rojas-Barahona",
                "Bo-Hsiang Tseng",
                "Yinpei Dai",
                "Clare Mansfield",
                "Osman Ramadan",
                "Stefan Ultes",
                "Michael Crawford",
                "Milica Gasic"
            ],
            "abstract": "In recent years, we have seen deep learning and distributed representations\nof words and sentences make impact on a number of natural language processing\ntasks, such as similarity, entailment and sentiment analysis. Here we introduce\na new task: understanding of mental health concepts derived from Cognitive\nBehavioural Therapy (CBT). We define a mental health ontology based on the CBT\nprinciples, annotate a large corpus where this phenomena is exhibited and\nperform understanding using deep learning and distributed representations. Our\nresults show that the performance of deep learning models combined with word\nembeddings or sentence embeddings significantly outperform non-deep-learning\nmodels in this difficult task. This understanding module will be an essential\ncomponent of a statistical dialogue system delivering therapy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00676v1",
            "date": "2018-09-03 18:03:46+00:00",
            "title": "A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension",
            "authors": [
                "Jiuniu Wang",
                "Xingyu Fu",
                "Guangluan Xu",
                "Yirong Wu",
                "Ziyan Chen",
                "Yang Wei",
                "Li Jin"
            ],
            "abstract": "In this paper, we introduce Adversarial-and-attention Network (A3Net) for\nMachine Reading Comprehension. This model extends existing approaches from two\nperspectives. First, adversarial training is applied to several target\nvariables within the model, rather than only to the inputs or embeddings. We\ncontrol the norm of adversarial perturbations according to the norm of original\ntarget variables, so that we can jointly add perturbations to several target\nvariables during training. As an effective regularization method, adversarial\ntraining improves robustness and generalization of our model. Second, we\npropose a multi-layer attention network utilizing three kinds of\nhigh-efficiency attention mechanisms. Multi-layer attention conducts\ninteraction between question and passage within each layer, which contributes\nto reasonable representation and understanding of the model. Combining these\ntwo contributions, we enhance the diversity of dataset and the information\nextracting ability of the model at the same time. Meanwhile, we construct A3Net\nfor the WebQA dataset. Results show that our model outperforms the\nstate-of-the-art models (improving Fuzzy Score from 73.50% to 77.0%).",
            "journal ref": "NLPCC 2018: Natural Language Processing and Chinese Computing pp\n  64-75",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00699v1",
            "date": "2018-09-03 19:39:20+00:00",
            "title": "Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction",
            "authors": [
                "Jinhua Du",
                "Jingguang Han",
                "Andy Way",
                "Dadong Wan"
            ],
            "abstract": "Attention mechanisms are often used in deep neural networks for distantly\nsupervised relation extraction (DS-RE) to distinguish valid from noisy\ninstances. However, traditional 1-D vector attention models are insufficient\nfor the learning of different contexts in the selection of valid instances to\npredict the relationship for an entity pair. To alleviate this issue, we\npropose a novel multi-level structured (2-D matrix) self-attention mechanism\nfor DS-RE in a multi-instance learning (MIL) framework using bidirectional\nrecurrent neural networks. In the proposed method, a structured word-level\nself-attention mechanism learns a 2-D matrix where each row vector represents a\nweight distribution for different aspects of an instance regarding two\nentities. Targeting the MIL issue, the structured sentence-level attention\nlearns a 2-D matrix where each row vector represents a weight distribution on\nselection of different valid in-stances. Experiments conducted on two publicly\navailable DS-RE datasets show that the proposed framework with a multi-level\nstructured self-attention mechanism significantly outperform state-of-the-art\nbaselines in terms of PR curves, P@N and F1 measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00717v1",
            "date": "2018-09-03 21:00:10+00:00",
            "title": "NTUA-SLP at IEST 2018: Ensemble of Neural Transfer Methods for Implicit Emotion Classification",
            "authors": [
                "Alexandra Chronopoulou",
                "Aikaterini Margatina",
                "Christos Baziotis",
                "Alexandros Potamianos"
            ],
            "abstract": "In this paper we present our approach to tackle the Implicit Emotion Shared\nTask (IEST) organized as part of WASSA 2018 at EMNLP 2018. Given a tweet, from\nwhich a certain word has been removed, we are asked to predict the emotion of\nthe missing word. In this work, we experiment with neural Transfer Learning\n(TL) methods. Our models are based on LSTM networks, augmented with a\nself-attention mechanism. We use the weights of various pretrained models, for\ninitializing specific layers of our networks. We leverage a big collection of\nunlabeled Twitter messages, for pretraining word2vec word embeddings and a set\nof diverse language models. Moreover, we utilize a sentiment analysis dataset\nfor pretraining a model, which encodes emotion related information. The\nsubmitted model consists of an ensemble of the aforementioned TL models. Our\nteam ranked 3rd out of 30 participants, achieving an F1 score of 0.703.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00732v1",
            "date": "2018-09-03 21:56:47+00:00",
            "title": "emrQA: A Large Corpus for Question Answering on Electronic Medical Records",
            "authors": [
                "Anusri Pampari",
                "Preethi Raghavan",
                "Jennifer Liang",
                "Jian Peng"
            ],
            "abstract": "We propose a novel methodology to generate domain-specific large-scale\nquestion answering (QA) datasets by re-purposing existing annotations for other\nNLP tasks. We demonstrate an instance of this methodology in generating a\nlarge-scale QA dataset for electronic medical records by leveraging existing\nexpert annotations on clinical notes for various NLP tasks from the community\nshared i2b2 datasets. The resulting corpus (emrQA) has 1 million\nquestion-logical form and 400,000+ question-answer evidence pairs. We\ncharacterize the dataset and explore its learning potential by training\nbaseline models for question to logical form and question to answer mapping.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00773v1",
            "date": "2018-09-04 02:30:57+00:00",
            "title": "Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsing",
            "authors": [
                "Bo Chen",
                "Le Sun",
                "Xianpei Han"
            ],
            "abstract": "This paper proposes a neural semantic parsing approach -- Sequence-to-Action,\nwhich models semantic parsing as an end-to-end semantic graph generation\nprocess. Our method simultaneously leverages the advantages from two recent\npromising directions of semantic parsing. Firstly, our model uses a semantic\ngraph to represent the meaning of a sentence, which has a tight-coupling with\nknowledge bases. Secondly, by leveraging the powerful representation learning\nand prediction ability of neural network models, we propose a RNN model which\ncan effectively map sentences to action sequences for semantic graph\ngeneration. Experiments show that our method achieves state-of-the-art\nperformance on OVERNIGHT dataset and gets competitive performance on GEO and\nATIS datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00786v2",
            "date": "2018-09-04 03:36:21+00:00",
            "title": "Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction",
            "authors": [
                "Dipendra Misra",
                "Andrew Bennett",
                "Valts Blukis",
                "Eyvind Niklasson",
                "Max Shatkhin",
                "Yoav Artzi"
            ],
            "abstract": "We propose to decompose instruction execution to goal prediction and action\ngeneration. We design a model that maps raw visual observations to goals using\nLINGUNET, a language-conditioned image generation network, and then generates\nthe actions required to complete them. Our model is trained from demonstration\nonly without external resources. To evaluate our approach, we introduce two\nbenchmarks for instruction following: LANI, a navigation task; and CHAI, where\nan agent executes household instructions. Our evaluation demonstrates the\nadvantages of our model decomposition, and illustrates the challenges posed by\nour new benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00938v2",
            "date": "2018-09-04 13:19:16+00:00",
            "title": "Improving generalization of vocal tract feature reconstruction: from augmented acoustic inversion to articulatory feature reconstruction without articulatory data",
            "authors": [
                "Rosanna Turrisi",
                "Raffaele Tavarone",
                "Leonardo Badino"
            ],
            "abstract": "We address the problem of reconstructing articulatory movements, given audio\nand/or phonetic labels. The scarce availability of multi-speaker articulatory\ndata makes it difficult to learn a reconstruction that generalizes to new\nspeakers and across datasets. We first consider the XRMB dataset where audio,\narticulatory measurements and phonetic transcriptions are available. We show\nthat phonetic labels, used as input to deep recurrent neural networks that\nreconstruct articulatory features, are in general more helpful than acoustic\nfeatures in both matched and mismatched training-testing conditions. In a\nsecond experiment, we test a novel approach that attempts to build articulatory\nfeatures from prior articulatory information extracted from phonetic labels.\nSuch approach recovers vocal tract movements directly from an acoustic-only\ndataset without using any articulatory measurement. Results show that\narticulatory features generated by this approach can correlate up to 0.59\nPearson product-moment correlation with measured articulatory features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00994v1",
            "date": "2018-09-04 14:07:40+00:00",
            "title": "\u00c9tude de l'informativit\u00e9 des transcriptions : une approche bas\u00e9e sur le r\u00e9sum\u00e9 automatique",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Malek Hajjem",
                "Eric SanJuan",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "In this paper we propose a new approach to evaluate the informativeness of\ntranscriptions coming from Automatic Speech Recognition systems. This approach,\nbased in the notion of informativeness, is focused on the framework of\nAutomatic Text Summarization performed over these transcriptions. At a first\nglance we estimate the informative content of the various automatic\ntranscriptions, then we explore the capacity of Automatic Text Summarization to\novercome the informative loss. To do this we use an automatic summary\nevaluation protocol without reference (based on the informative content), which\ncomputes the divergence between probability distributions of different textual\nrepresentations: manual and automatic transcriptions and their summaries. After\na set of evaluations this analysis allowed us to judge both the quality of the\ntranscriptions in terms of informativeness and to assess the ability of\nautomatic text summarization to compensate the problems raised during the\ntranscription phase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01060v1",
            "date": "2018-09-04 16:03:06+00:00",
            "title": "The Effect of Context on Metaphor Paraphrase Aptness Judgments",
            "authors": [
                "Yuri Bizzoni",
                "Shalom Lappin"
            ],
            "abstract": "We conduct two experiments to study the effect of context on metaphor\nparaphrase aptness judgments. The first is an AMT crowd source task in which\nspeakers rank metaphor paraphrase candidate sentence pairs in short document\ncontexts for paraphrase aptness. In the second we train a composite DNN to\npredict these human judgments, first in binary classifier mode, and then as\ngradient ratings. We found that for both mean human judgments and our DNN's\npredictions, adding document context compresses the aptness scores towards the\ncenter of the scale, raising low out of context ratings and decreasing high out\nof context scores. We offer a provisional explanation for this compression\neffect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01083v2",
            "date": "2018-09-04 16:44:16+00:00",
            "title": "IEST: WASSA-2018 Implicit Emotions Shared Task",
            "authors": [
                "Roman Klinger",
                "Orph\u00e9e De Clercq",
                "Saif M. Mohammad",
                "Alexandra Balahur"
            ],
            "abstract": "Past shared tasks on emotions use data with both overt expressions of\nemotions (I am so happy to see you!) as well as subtle expressions where the\nemotions have to be inferred, for instance from event descriptions. Further,\nmost datasets do not focus on the cause or the stimulus of the emotion. Here,\nfor the first time, we propose a shared task where systems have to predict the\nemotions in a large automatically labeled dataset of tweets without access to\nwords denoting emotions. Based on this intention, we call this the Implicit\nEmotion Shared Task (IEST) because the systems have to infer the emotion mostly\nfrom the context. Every tweet has an occurrence of an explicit emotion word\nthat is masked. The tweets are collected in a manner such that they are likely\nto include a description of the cause of the emotion - the stimulus.\nAltogether, 30 teams submitted results which range from macro F1 scores of 21 %\nto 71 %. The baseline (MaxEnt bag of words and bigrams) obtains an F1 score of\n60 % which was available to the participants during the development phase. A\nstudy with human annotators suggests that automatic methods outperform human\npredictions, possibly by honing into subtle textual clues not used by humans.\nCorpora, resources, and results are available at the shared task website at\nhttp://implicitemotions.wassa2018.com.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01202v2",
            "date": "2018-09-04 19:06:34+00:00",
            "title": "Causal Explanation Analysis on Social Media",
            "authors": [
                "Youngseo Son",
                "Nipun Bayas",
                "H. Andrew Schwartz"
            ],
            "abstract": "Understanding causal explanations - reasons given for happenings in one's\nlife - has been found to be an important psychological factor linked to\nphysical and mental health. Causal explanations are often studied through\nmanual identification of phrases over limited samples of personal writing.\nAutomatic identification of causal explanations in social media, while\nchallenging in relying on contextual and sequential cues, offers a larger-scale\nalternative to expensive manual ratings and opens the door for new applications\n(e.g. studying prevailing beliefs about causes, such as climate change). Here,\nwe explore automating causal explanation analysis, building on discourse\nparsing, and presenting two novel subtasks: causality detection (determining\nwhether a causal explanation exists at all) and causal explanation\nidentification (identifying the specific phrase that is the explanation). We\nachieve strong accuracies for both tasks but find different approaches best: an\nSVM for causality prediction (F1 = 0.791) and a hierarchy of Bidirectional\nLSTMs for causal explanation identification (F1 = 0.853). Finally, we explore\napplications of our complete pipeline (F1 = 0.868), showing demographic\ndifferences in mentions of causal explanation and that the association between\na word and sentiment can change when it is used within a causal explanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01215v1",
            "date": "2018-09-04 19:29:03+00:00",
            "title": "Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints",
            "authors": [
                "Ashutosh Baheti",
                "Alan Ritter",
                "Jiwei Li",
                "Bill Dolan"
            ],
            "abstract": "Neural conversation models tend to generate safe, generic responses for most\ninputs. This is due to the limitations of likelihood-based decoding objectives\nin generation tasks with diverse outputs, such as conversation. To address this\nchallenge, we propose a simple yet effective approach for incorporating side\ninformation in the form of distributional constraints over the generated\nresponses. We propose two constraints that help generate more content rich\nresponses that are based on a model of syntax and topics (Griffiths et al.,\n2005) and semantic similarity (Arora et al., 2016). We evaluate our approach\nagainst a variety of competitive baselines, using both automatic metrics and\nhuman judgments, showing that our proposed approach generates responses that\nare much less generic without sacrificing plausibility. A working demo of our\ncode can be found at https://github.com/abaheti95/DC-NeuralConversation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01219v1",
            "date": "2018-09-04 19:39:24+00:00",
            "title": "Graph-based Deep-Tree Recursive Neural Network (DTRNN) for Text Classification",
            "authors": [
                "Fenxiao Chen",
                "Bin Wang",
                "C. -C. Jay Kuo"
            ],
            "abstract": "A novel graph-to-tree conversion mechanism called the deep-tree generation\n(DTG) algorithm is first proposed to predict text data represented by graphs.\nThe DTG method can generate a richer and more accurate representation for nodes\n(or vertices) in graphs. It adds flexibility in exploring the vertex\nneighborhood information to better reflect the second order proximity and\nhomophily equivalence in a graph. Then, a Deep-Tree Recursive Neural Network\n(DTRNN) method is presented and used to classify vertices that contains text\ndata in graphs. To demonstrate the effectiveness of the DTRNN method, we apply\nit to three real-world graph datasets and show that the DTRNN method\noutperforms several state-of-the-art benchmarking methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01285v1",
            "date": "2018-09-05 01:08:23+00:00",
            "title": "Learning Concept Abstractness Using Weak Supervision",
            "authors": [
                "Ella Rabinovich",
                "Benjamin Sznajder",
                "Artem Spector",
                "Ilya Shnayderman",
                "Ranit Aharonov",
                "David Konopnicki",
                "Noam Slonim"
            ],
            "abstract": "We introduce a weakly supervised approach for inferring the property of\nabstractness of words and expressions in the complete absence of labeled data.\nExploiting only minimal linguistic clues and the contextual usage of a concept\nas manifested in textual data, we train sufficiently powerful classifiers,\nobtaining high correlation with human labels. The results imply the\napplicability of this approach to additional properties of concepts, additional\nlanguages, and resource-scarce scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01299v1",
            "date": "2018-09-05 02:15:02+00:00",
            "title": "Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations",
            "authors": [
                "Dipendra Misra",
                "Ming-Wei Chang",
                "Xiaodong He",
                "Wen-tau Yih"
            ],
            "abstract": "Semantic parsing from denotations faces two key challenges in model training:\n(1) given only the denotations (e.g., answers), search for good candidate\nsemantic parses, and (2) choose the best model update algorithm. We propose\neffective and general solutions to each of them. Using policy shaping, we bias\nthe search procedure towards semantic parses that are more compatible to the\ntext, which provide better supervision signals for training. In addition, we\npropose an update equation that generalizes three different families of\nlearning algorithms, which enables fast model exploration. When experimented on\na recently proposed sequential question answering dataset, our framework leads\nto a new state-of-the-art model that outperforms previous work by 5.0% absolute\non exact match accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01301v2",
            "date": "2018-09-05 02:26:09+00:00",
            "title": "BPE and CharCNNs for Translation of Morphology: A Cross-Lingual Comparison and Analysis",
            "authors": [
                "Pamela Shapiro",
                "Kevin Duh"
            ],
            "abstract": "Neural Machine Translation (NMT) in low-resource settings and of\nmorphologically rich languages is made difficult in part by data sparsity of\nvocabulary words. Several methods have been used to help reduce this sparsity,\nnotably Byte-Pair Encoding (BPE) and a character-based CNN layer (charCNN).\nHowever, the charCNN has largely been neglected, possibly because it has only\nbeen compared to BPE rather than combined with it. We argue for a\nreconsideration of the charCNN, based on cross-lingual improvements on\nlow-resource data. We translate from 8 languages into English, using a\nmulti-way parallel collection of TED transcripts. We find that in most cases,\nusing both BPE and a charCNN performs best, while in Hebrew, using a charCNN\nover words is best.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01329v1",
            "date": "2018-09-05 05:12:34+00:00",
            "title": "RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency",
            "authors": [
                "Richard Futrell",
                "Ethan Wilcox",
                "Takashi Morita",
                "Roger Levy"
            ],
            "abstract": "Recurrent neural networks (RNNs) are the state of the art in sequence\nmodeling for natural language. However, it remains poorly understood what\ngrammatical characteristics of natural language they implicitly learn and\nrepresent as a consequence of optimizing the language modeling objective. Here\nwe deploy the methods of controlled psycholinguistic experimentation to shed\nlight on to what extent RNN behavior reflects incremental syntactic state and\ngrammatical dependency representations known to characterize human linguistic\nbehavior. We broadly test two publicly available long short-term memory (LSTM)\nEnglish sequence models, and learn and test a new Japanese LSTM. We demonstrate\nthat these models represent and maintain incremental syntactic state, but that\nthey do not always generalize in the same way as humans. Furthermore, none of\nour models learn the appropriate grammatical dependency configurations\nlicensing reflexive pronouns or negative polarity items.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01331v1",
            "date": "2018-09-05 05:24:00+00:00",
            "title": "Neural MultiVoice Models for Expressing Novel Personalities in Dialog",
            "authors": [
                "Shereen Oraby",
                "Lena Reed",
                "Sharath TS",
                "Shubhangi Tandon",
                "Marilyn Walker"
            ],
            "abstract": "Natural language generators for task-oriented dialog should be able to vary\nthe style of the output utterance while still effectively realizing the system\ndialog actions and their associated semantics. While the use of neural\ngeneration for training the response generation component of conversational\nagents promises to simplify the process of producing high quality responses in\nnew domains, to our knowledge, there has been very little investigation of\nneural generators for task-oriented dialog that can vary their response style,\nand we know of no experiments on models that can generate responses that are\ndifferent in style from those seen during training, while still maintain- ing\nsemantic fidelity to the input meaning representation. Here, we show that a\nmodel that is trained to achieve a single stylis- tic personality target can\nproduce outputs that combine stylistic targets. We carefully evaluate the\nmultivoice outputs for both semantic fidelity and for similarities to and\ndifferences from the linguistic features that characterize the original\ntraining style. We show that contrary to our predictions, the learned models do\nnot always simply interpolate model parameters, but rather produce styles that\nare distinct, and novel from the personalities they were trained on.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01375v1",
            "date": "2018-09-05 08:14:16+00:00",
            "title": "Firearms and Tigers are Dangerous, Kitchen Knives and Zebras are Not: Testing whether Word Embeddings Can Tell",
            "authors": [
                "Pia Sommerauer",
                "Antske Fokkens"
            ],
            "abstract": "This paper presents an approach for investigating the nature of semantic\ninformation captured by word embeddings. We propose a method that extends an\nexisting human-elicited semantic property dataset with gold negative examples\nusing crowd judgments. Our experimental approach tests the ability of\nsupervised classifiers to identify semantic features in word embedding vectors\nand com- pares this to a feature-identification method based on full vector\ncosine similarity. The idea behind this method is that properties identified by\nclassifiers, but not through full vector comparison are captured by embeddings.\nProperties that cannot be identified by either method are not. Our results\nprovide an initial indication that semantic properties relevant for the way\nentities interact (e.g. dangerous) are captured, while perceptual information\n(e.g. colors) is not represented. We conclude that, though preliminary, these\nresults show that our method is suitable for identifying which properties are\ncaptured by embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01431v2",
            "date": "2018-09-05 10:56:30+00:00",
            "title": "Pre-training on high-resource speech recognition improves low-resource speech-to-text translation",
            "authors": [
                "Sameer Bansal",
                "Herman Kamper",
                "Karen Livescu",
                "Adam Lopez",
                "Sharon Goldwater"
            ],
            "abstract": "We present a simple approach to improve direct speech-to-text translation\n(ST) when the source language is low-resource: we pre-train the model on a\nhigh-resource automatic speech recognition (ASR) task, and then fine-tune its\nparameters for ST. We demonstrate that our approach is effective by\npre-training on 300 hours of English ASR data to improve Spanish-English ST\nfrom 10.8 to 20.2 BLEU when only 20 hours of Spanish-English ST training data\nare available. Through an ablation study, we find that the pre-trained encoder\n(acoustic model) accounts for most of the improvement, despite the fact that\nthe shared language in these tasks is the target language text, not the source\nlanguage audio. Applying this insight, we show that pre-training on ASR helps\nST even when the ASR language differs from both source and target ST languages:\npre-training on French ASR also improves Spanish-English ST. Finally, we show\nthat the approach improves performance on a true low-resource task:\npre-training on a combination of English ASR and French ASR improves\nMboshi-French ST, where only 4 hours of data are available, from 3.5 to 7.1\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01446v2",
            "date": "2018-09-05 11:44:13+00:00",
            "title": "Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit",
            "authors": [
                "Amrith Krishna",
                "Bishal Santra",
                "Sasi Prasanth Bandaru",
                "Gaurav Sahu",
                "Vishnu Dutt Sharma",
                "Pavankumar Satuluri",
                "Pawan Goyal"
            ],
            "abstract": "The configurational information in sentences of a free word order language\nsuch as Sanskrit is of limited use. Thus, the context of the entire sentence\nwill be desirable even for basic processing tasks such as word segmentation. We\npropose a structured prediction framework that jointly solves the word\nsegmentation and morphological tagging tasks in Sanskrit. We build an energy\nbased model where we adopt approaches generally employed in graph based parsing\ntechniques (McDonald et al., 2005a; Carreras, 2007). Our model outperforms the\nstate of the art with an F-Score of 96.92 (percentage improvement of 7.06%)\nwhile using less than one-tenth of the task-specific training data. We find\nthat the use of a graph based ap- proach instead of a traditional lattice-based\nsequential labelling approach leads to a percentage gain of 12.6% in F-Score\nfor the segmentation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01448v1",
            "date": "2018-09-05 11:55:05+00:00",
            "title": "Appendix - Recommended Statistical Significance Tests for NLP Tasks",
            "authors": [
                "Rotem Dror",
                "Roi Reichart"
            ],
            "abstract": "Statistical significance testing plays an important role when drawing\nconclusions from experimental results in NLP papers. Particularly, it is a\nvaluable tool when one would like to establish the superiority of one algorithm\nover another. This appendix complements the guide for testing statistical\nsignificance in NLP presented in \\cite{dror2018hitchhiker} by proposing valid\nstatistical tests for the common tasks and evaluation measures in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01500v1",
            "date": "2018-08-31 10:16:19+00:00",
            "title": "Neural DrugNet",
            "authors": [
                "Nishant Nikhil",
                "Shivansh Mundra"
            ],
            "abstract": "In this paper, we describe the system submitted for the shared task on Social\nMedia Mining for Health Applications by the team Light. Previous works\ndemonstrate that LSTMs have achieved remarkable performance in natural language\nprocessing tasks. We deploy an ensemble of two LSTM models. The first one is a\npretrained language model appended with a classifier and takes words as input,\nwhile the second one is a LSTM model with an attention unit over it which takes\ncharacter tri-gram as input. We call the ensemble of these two models:\nNeural-DrugNet. Our system ranks 2nd in the second shared task: Automatic\nclassification of posts describing medication intake.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01541v1",
            "date": "2018-09-05 14:31:04+00:00",
            "title": "Copenhagen at CoNLL--SIGMORPHON 2018: Multilingual Inflection in Context with Explicit Morphosyntactic Decoding",
            "authors": [
                "Yova Kementchedjhieva",
                "Johannes Bjerva",
                "Isabelle Augenstein"
            ],
            "abstract": "This paper documents the Team Copenhagen system which placed first in the\nCoNLL--SIGMORPHON 2018 shared task on universal morphological reinflection,\nTask 2 with an overall accuracy of 49.87. Task 2 focuses on morphological\ninflection in context: generating an inflected word form, given the lemma of\nthe word and the context it occurs in. Previous SIGMORPHON shared tasks have\nfocused on context-agnostic inflection---the \"inflection in context\" task was\nintroduced this year. We approach this with an encoder-decoder architecture\nover character sequences with three core innovations, all contributing to an\nimprovement in performance: (1) a wide context window; (2) a multi-task\nlearning approach with the auxiliary task of MSD prediction; (3) training\nmodels in a multilingual fashion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01557v2",
            "date": "2018-09-05 14:53:14+00:00",
            "title": "Dynamically Context-Sensitive Time-Decay Attention for Dialogue Modeling",
            "authors": [
                "Shang-Yu Su",
                "Pei-Chieh Yuan",
                "Yun-Nung Chen"
            ],
            "abstract": "Spoken language understanding (SLU) is an essential component in\nconversational systems. Considering that contexts provide informative cues for\nbetter understanding, history can be leveraged for contextual SLU. However,\nmost prior work only paid attention to the related content in history\nutterances and ignored the temporal information. In dialogues, it is intuitive\nthat the most recent utterances are more important than the least recent ones,\nand time-aware attention should be in a decaying manner. Therefore, this paper\nallows the model to automatically learn a time-decay attention function where\nthe attentional weights can be dynamically decided based on the content of each\nrole's contexts, which effectively integrates both content-aware and time-aware\nperspectives and demonstrates remarkable flexibility to complex dialogue\ncontexts. The experiments on the benchmark Dialogue State Tracking Challenge\n(DSTC4) dataset show that the proposed dynamically context-sensitive time-decay\nattention mechanisms significantly improve the state-of-the-art model for\ncontextual understanding performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01574v2",
            "date": "2018-09-05 15:20:41+00:00",
            "title": "Stance Prediction for Russian: Data and Analysis",
            "authors": [
                "Nikita Lozhnikov",
                "Leon Derczynski",
                "Manuel Mazzara"
            ],
            "abstract": "Stance detection is a critical component of rumour and fake news\nidentification. It involves the extraction of the stance a particular author\ntakes related to a given claim, both expressed in text. This paper investigates\nstance classification for Russian. It introduces a new dataset, RuStance, of\nRussian tweets and news comments from multiple sources, covering multiple\nstories, as well as text classification approaches to stance detection as\nbenchmarks over this data in this language. As well as presenting this\nopenly-available dataset, the first of its kind for Russian, the paper presents\na baseline for stance prediction in the language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01576v2",
            "date": "2018-09-05 15:27:16+00:00",
            "title": "Document-Level Neural Machine Translation with Hierarchical Attention Networks",
            "authors": [
                "Lesly Miculicich",
                "Dhananjay Ram",
                "Nikolaos Pappas",
                "James Henderson"
            ],
            "abstract": "Neural Machine Translation (NMT) can be improved by including document-level\ncontextual information. For this purpose, we propose a hierarchical attention\nmodel to capture the context in a structured and dynamic manner. The model is\nintegrated in the original NMT architecture as another level of abstraction,\nconditioning on the NMT model's own previous hidden states. Experiments show\nthat hierarchical attention significantly improves the BLEU score over a strong\nNMT baseline with the state-of-the-art in context-aware methods, and that both\nthe encoder and decoder benefit from context in complementary ways.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01694v2",
            "date": "2018-09-05 19:11:14+00:00",
            "title": "Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction",
            "authors": [
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "A major obstacle in reinforcement learning-based sentence generation is the\nlarge action space whose size is equal to the vocabulary size of the\ntarget-side language. To improve the efficiency of reinforcement learning, we\npresent a novel approach for reducing the action space based on dynamic\nvocabulary prediction. Our method first predicts a fixed-size small vocabulary\nfor each input to generate its target sentence. The input-specific vocabularies\nare then used at supervised and reinforcement learning steps, and also at test\ntime. In our experiments on six machine translation and two image captioning\ndatasets, our method achieves faster reinforcement learning ($\\sim$2.7x faster)\nwith less GPU memory ($\\sim$2.3x less) than the full-vocabulary counterpart.\nThe reinforcement learning with our method consistently leads to significant\nimprovement of BLEU scores, and the scores are equal to or better than those of\nbaselines using the full vocabularies, with faster decoding time ($\\sim$3x\nfaster) on CPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01854v1",
            "date": "2018-09-06 07:33:48+00:00",
            "title": "Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing",
            "authors": [
                "Jetic G\u016b",
                "Hassan S. Shavarani",
                "Anoop Sarkar"
            ],
            "abstract": "The addition of syntax-aware decoding in Neural Machine Translation (NMT)\nsystems requires an effective tree-structured neural network, a syntax-aware\nattention model and a language generation model that is sensitive to sentence\nstructure. We exploit a top-down tree-structured model called DRNN\n(Doubly-Recurrent Neural Networks) first proposed by Alvarez-Melis and Jaakola\n(2017) to create an NMT model called Seq2DRNN that combines a sequential\nencoder with tree-structured decoding augmented with a syntax-aware attention\nmodel. Unlike previous approaches to syntax-based NMT which use dependency\nparsing models our method uses constituency parsing which we argue provides\nuseful information for translation. In addition, we use the syntactic structure\nof the sentence to add new connections to the tree-structured decoder neural\nnetwork (Seq2DRNN+SynC). We compare our NMT model with sequential and state of\nthe art syntax-based NMT models and show that our model produces more fluent\ntranslations with better reordering. Since our model is capable of doing\ntranslation and constituency parsing at the same time we also compare our\nparsing accuracy against other neural parsing models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01984v1",
            "date": "2018-09-06 13:36:40+00:00",
            "title": "Training Millions of Personalized Dialogue Agents",
            "authors": [
                "Pierre-Emmanuel Mazar\u00e9",
                "Samuel Humeau",
                "Martin Raison",
                "Antoine Bordes"
            ],
            "abstract": "Current dialogue systems are not very engaging for users, especially when\ntrained end-to-end without relying on proactive reengaging scripted strategies.\nZhang et al. (2018) showed that the engagement level of end-to-end dialogue\nmodels increases when conditioning them on text personas providing some\npersonalized back-story to the model. However, the dataset used in Zhang et al.\n(2018) is synthetic and of limited size as it contains around 1k different\npersonas. In this paper we introduce a new dataset providing 5 million personas\nand 700 million persona-based dialogues. Our experiments show that, at this\nscale, training using personas still improves the performance of end-to-end\nsystems. In addition, we show that other tasks benefit from the wide coverage\nof our dataset by fine-tuning our model on the data from Zhang et al. (2018)\nand achieving state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02035v1",
            "date": "2018-09-06 15:09:46+00:00",
            "title": "Evaluating Syntactic Properties of Seq2seq Output with a Broad Coverage HPSG: A Case Study on Machine Translation",
            "authors": [
                "Johnny Tian-Zheng Wei",
                "Khiem Pham",
                "Brian Dillon",
                "Brendan O'Connor"
            ],
            "abstract": "Sequence to sequence (seq2seq) models are often employed in settings where\nthe target output is natural language. However, the syntactic properties of the\nlanguage generated from these models are not well understood. We explore\nwhether such output belongs to a formal and realistic grammar, by employing the\nEnglish Resource Grammar (ERG), a broad coverage, linguistically precise\nHPSG-based grammar of English. From a French to English parallel corpus, we\nanalyze the parseability and grammatical constructions occurring in output from\na seq2seq translation model. Over 93\\% of the model translations are parseable,\nsuggesting that it learns to generate conforming to a grammar. The model has\ntrouble learning the distribution of rarer syntactic rules, and we pinpoint\nseveral constructions that differentiate translations between the references\nand our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02147v1",
            "date": "2018-09-06 18:02:55+00:00",
            "title": "Upcycle Your OCR: Reusing OCRs for Post-OCR Text Correction in Romanised Sanskrit",
            "authors": [
                "Amrith Krishna",
                "Bodhisattwa Prasad Majumder",
                "Rajesh Shreedhar Bhat",
                "Pawan Goyal"
            ],
            "abstract": "We propose a post-OCR text correction approach for digitising texts in\nRomanised Sanskrit. Owing to the lack of resources our approach uses OCR models\ntrained for other languages written in Roman. Currently, there exists no\ndataset available for Romanised Sanskrit OCR. So, we bootstrap a dataset of 430\nimages, scanned in two different settings and their corresponding ground truth.\nFor training, we synthetically generate training images for both the settings.\nWe find that the use of copying mechanism (Gu et al., 2016) yields a percentage\nincrease of 7.69 in Character Recognition Rate (CRR) than the current state of\nthe art model in solving monotone sequence-to-sequence tasks (Schnober et al.,\n2016). We find that our system is robust in combating OCR-prone errors, as it\nobtains a CRR of 87.01% from an OCR output with CRR of 35.76% for one of the\ndataset settings. A human judgment survey performed on the models shows that\nour proposed model results in predictions which are faster to comprehend and\nfaster to improve for a human than the other systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02223v5",
            "date": "2018-09-06 21:26:31+00:00",
            "title": "Character-Aware Decoder for Translation into Morphologically Rich Languages",
            "authors": [
                "Adithya Renduchintala",
                "Pamela Shapiro",
                "Kevin Duh",
                "Philipp Koehn"
            ],
            "abstract": "Neural machine translation (NMT) systems operate primarily on words (or\nsub-words), ignoring lower-level patterns of morphology. We present a\ncharacter-aware decoder designed to capture such patterns when translating into\nmorphologically rich languages. We achieve character-awareness by augmenting\nboth the softmax and embedding layers of an attention-based encoder-decoder\nmodel with convolutional neural networks that operate on the spelling of a\nword. To investigate performance on a wide variety of morphological phenomena,\nwe translate English into 14 typologically diverse target languages using the\nTED multi-target dataset. In this low-resource setting, the character-aware\ndecoder provides consistent improvements with BLEU score gains of up to\n$+3.05$. In addition, we analyze the relationship between the gains obtained\nand properties of the target language and find evidence that our model does\nindeed exploit morphological patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02237v1",
            "date": "2018-09-06 22:10:38+00:00",
            "title": "82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Models",
            "authors": [
                "Aaron Smith",
                "Bernd Bohnet",
                "Miryam de Lhoneux",
                "Joakim Nivre",
                "Yan Shao",
                "Sara Stymne"
            ],
            "abstract": "We present the Uppsala system for the CoNLL 2018 Shared Task on universal\ndependency parsing. Our system is a pipeline consisting of three components:\nthe first performs joint word and sentence segmentation; the second predicts\npart-of- speech tags and morphological features; the third predicts dependency\ntrees from words and tags. Instead of training a single parsing model for each\ntreebank, we trained models with multiple treebanks for one language or closely\nrelated languages, greatly reducing the number of models. On the official test\nrun, we ranked 7th of 27 teams for the LAS and MLAS metrics. Our system\nobtained the best scores overall for word segmentation, universal POS tagging,\nand morphological features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02255v1",
            "date": "2018-09-07 00:00:39+00:00",
            "title": "Adversarial Domain Adaptation for Duplicate Question Detection",
            "authors": [
                "Darsh J Shah",
                "Tao Lei",
                "Alessandro Moschitti",
                "Salvatore Romeo",
                "Preslav Nakov"
            ],
            "abstract": "We address the problem of detecting duplicate questions in forums, which is\nan important step towards automating the process of answering new questions. As\nfinding and annotating such potential duplicates manually is very tedious and\ncostly, automatic methods based on machine learning are a viable alternative.\nHowever, many forums do not have annotated data, i.e., questions labeled by\nexperts as duplicates, and thus a promising solution is to use domain\nadaptation from another forum that has such annotations. Here we focus on\nadversarial domain adaptation, deriving important findings about when it\nperforms well and what properties of the domains are important in this regard.\nOur experiments with StackExchange data show an average improvement of 5.6%\nover the best baseline across multiple pairs of domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02256v2",
            "date": "2018-09-07 00:01:42+00:00",
            "title": "Multi-Source Domain Adaptation with Mixture of Experts",
            "authors": [
                "Jiang Guo",
                "Darsh J Shah",
                "Regina Barzilay"
            ],
            "abstract": "We propose a mixture-of-experts approach for unsupervised domain adaptation\nfrom multiple sources. The key idea is to explicitly capture the relationship\nbetween a target example and different source domains. This relationship,\nexpressed by a point-to-set metric, determines how to combine predictors\ntrained on various domains. The metric is learned in an unsupervised fashion\nusing meta-training. Experimental results on sentiment analysis and\npart-of-speech tagging demonstrate that our approach consistently outperforms\nmultiple baselines and can robustly handle negative transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02279v2",
            "date": "2018-09-07 02:17:23+00:00",
            "title": "Cell-aware Stacked LSTMs for Modeling Sentences",
            "authors": [
                "Jihun Choi",
                "Taeuk Kim",
                "Sang-goo Lee"
            ],
            "abstract": "We propose a method of stacking multiple long short-term memory (LSTM) layers\nfor modeling sentences. In contrast to the conventional stacked LSTMs where\nonly hidden states are fed as input to the next layer, the suggested\narchitecture accepts both hidden and memory cell states of the preceding layer\nand fuses information from the left and the lower context using the soft gating\nmechanism of LSTMs. Thus the architecture modulates the amount of information\nto be delivered not only in horizontal recurrence but also in vertical\nconnections, from which useful features extracted from lower layers are\neffectively conveyed to upper layers. We dub this architecture Cell-aware\nStacked LSTM (CAS-LSTM) and show from experiments that our models bring\nsignificant performance gain over the standard LSTMs on benchmark datasets for\nnatural language inference, paraphrase detection, sentiment classification, and\nmachine translation. We also conduct extensive qualitative analysis to\nunderstand the internal behavior of the suggested approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02286v2",
            "date": "2018-09-07 02:59:42+00:00",
            "title": "Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations",
            "authors": [
                "Taeuk Kim",
                "Jihun Choi",
                "Daniel Edmiston",
                "Sanghwan Bae",
                "Sang-goo Lee"
            ],
            "abstract": "Most existing recursive neural network (RvNN) architectures utilize only the\nstructure of parse trees, ignoring syntactic tags which are provided as\nby-products of parsing. We present a novel RvNN architecture that can provide\ndynamic compositionality by considering comprehensive syntactic information\nderived from both the structure and linguistic tags. Specifically, we introduce\na structure-aware tag representation constructed by a separate tag-level\ntree-LSTM. With this, we can control the composition function of the existing\nword-level tree-LSTM by augmenting the representation as a supplementary input\nto the gate functions of the tree-LSTM. In extensive experiments, we show that\nmodels built upon the proposed architecture obtain superior or competitive\nperformance on several sentence-level tasks such as sentiment analysis and\nnatural language inference when compared against previous tree-structured\nmodels and other sophisticated neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02305v2",
            "date": "2018-09-07 04:17:06+00:00",
            "title": "Data Augmentation for Spoken Language Understanding via Joint Variational Generation",
            "authors": [
                "Kang Min Yoo",
                "Youhyun Shin",
                "Sang-goo Lee"
            ],
            "abstract": "Data scarcity is one of the main obstacles of domain adaptation in spoken\nlanguage understanding (SLU) due to the high cost of creating manually tagged\nSLU datasets. Recent works in neural text generative models, particularly\nlatent variable models such as variational autoencoder (VAE), have shown\npromising results in regards to generating plausible and natural sentences. In\nthis paper, we propose a novel generative architecture which leverages the\ngenerative power of latent variable models to jointly synthesize fully\nannotated utterances. Our experiments show that existing SLU models trained on\nthe additional synthetic examples achieve performance gains. Our approach not\nonly helps alleviate the data scarcity issue in the SLU task for many datasets\nbut also indiscriminately improves language understanding performances for\nvarious SLU models, supported by extensive experiments and rigorous statistical\ntesting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02428v1",
            "date": "2018-09-07 12:07:59+00:00",
            "title": "Multitask and Multilingual Modelling for Lexical Analysis",
            "authors": [
                "Johannes Bjerva"
            ],
            "abstract": "In Natural Language Processing (NLP), one traditionally considers a single\ntask (e.g. part-of-speech tagging) for a single language (e.g. English) at a\ntime. However, recent work has shown that it can be beneficial to take\nadvantage of relatedness between tasks, as well as between languages. In this\nwork I examine the concept of relatedness and explore how it can be utilised to\nbuild NLP models that require less manually annotated data. A large selection\nof NLP tasks is investigated for a substantial language sample comprising 60\nlanguages. The results show potential for joint multitask and multilingual\nmodelling, and hints at linguistic insights which can be gained from such\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02534v3",
            "date": "2018-09-07 15:22:04+00:00",
            "title": "Using Sparse Semantic Embeddings Learned from Multimodal Text and Image Data to Model Human Conceptual Knowledge",
            "authors": [
                "Steven Derby",
                "Paul Miller",
                "Brian Murphy",
                "Barry Devereux"
            ],
            "abstract": "Distributional models provide a convenient way to model semantics using dense\nembedding spaces derived from unsupervised learning algorithms. However, the\ndimensions of dense embedding spaces are not designed to resemble human\nsemantic knowledge. Moreover, embeddings are often built from a single source\nof information (typically text data), even though neurocognitive research\nsuggests that semantics is deeply linked to both language and perception. In\nthis paper, we combine multimodal information from both text and image-based\nrepresentations derived from state-of-the-art distributional models to produce\nsparse, interpretable vectors using Joint Non-Negative Sparse Embedding.\nThrough in-depth analyses comparing these sparse models to human-derived\nbehavioural and neuroimaging data, we demonstrate their ability to predict\ninterpretable linguistic descriptions of human ground-truth semantic knowledge.",
            "journal ref": "Proceedings of the 22nd Conference on Computational Natural\n  Language Learning (CoNLL 2018), pages 260-270. Brussels, Belgium, October 31\n  - November 1, 2018. Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02669v1",
            "date": "2018-09-07 20:56:33+00:00",
            "title": "Unsupervised Sentence Compression using Denoising Auto-Encoders",
            "authors": [
                "Thibault F\u00e9vry",
                "Jason Phang"
            ],
            "abstract": "In sentence compression, the task of shortening sentences while retaining the\noriginal meaning, models tend to be trained on large corpora containing pairs\nof verbose and compressed sentences. To remove the need for paired corpora, we\nemulate a summarization task and add noise to extend sentences and train a\ndenoising auto-encoder to recover the original, constructing an end-to-end\ntraining regime without the need for any examples of compressed sentences. We\nconduct a human evaluation of our model on a standard text summarization\ndataset and show that it performs comparably to a supervised baseline based on\ngrammatical correctness and retention of meaning. Despite being exposed to no\ntarget data, our unsupervised models learn to generate imperfect but reasonably\nreadable sentence summaries. Although we underperform supervised models based\non ROUGE scores, our models are competitive with a supervised baseline based on\nhuman evaluation for grammatical correctness and retention of meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02694v1",
            "date": "2018-09-07 22:02:43+00:00",
            "title": "Neural Machine Translation of Logographic Languages Using Sub-character Level Information",
            "authors": [
                "Longtu Zhang",
                "Mamoru Komachi"
            ],
            "abstract": "Recent neural machine translation (NMT) systems have been greatly improved by\nencoder-decoder models with attention mechanisms and sub-word units. However,\nimportant differences between languages with logographic and alphabetic writing\nsystems have long been overlooked. This study focuses on these differences and\nuses a simple approach to improve the performance of NMT systems utilizing\ndecomposed sub-character level information for logographic languages. Our\nresults indicate that our approach not only improves the translation\ncapabilities of NMT systems between Chinese and English, but also further\nimproves NMT systems between Chinese and Japanese, because it utilizes the\nshared information brought by similar sub-character units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02700v1",
            "date": "2018-09-07 22:22:26+00:00",
            "title": "Textual Analogy Parsing: What's Shared and What's Compared among Analogous Facts",
            "authors": [
                "Matthew Lamm",
                "Arun Tejasvi Chaganty",
                "Christopher D. Manning",
                "Dan Jurafsky",
                "Percy Liang"
            ],
            "abstract": "To understand a sentence like \"whereas only 10% of White Americans live at or\nbelow the poverty line, 28% of African Americans do\" it is important not only\nto identify individual facts, e.g., poverty rates of distinct demographic\ngroups, but also the higher-order relations between them, e.g., the disparity\nbetween them. In this paper, we propose the task of Textual Analogy Parsing\n(TAP) to model this higher-order meaning. The output of TAP is a frame-style\nmeaning representation which explicitly specifies what is shared (e.g., poverty\nrates) and what is compared (e.g., White Americans vs. African Americans, 10%\nvs. 28%) between its component facts. Such a meaning representation can enable\nnew applications that rely on discourse understanding such as automated chart\ngeneration from quantitative text. We present a new dataset for TAP, baselines,\nand a model that successfully uses an ILP to enforce the structural constraints\nof the problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02701v4",
            "date": "2018-09-07 22:39:33+00:00",
            "title": "Trick Me If You Can: Human-in-the-loop Generation of Adversarial Examples for Question Answering",
            "authors": [
                "Eric Wallace",
                "Pedro Rodriguez",
                "Shi Feng",
                "Ikuya Yamada",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Adversarial evaluation stress tests a model's understanding of natural\nlanguage. While past approaches expose superficial patterns, the resulting\nadversarial examples are limited in complexity and diversity. We propose\nhuman-in-the-loop adversarial generation, where human authors are guided to\nbreak models. We aid the authors with interpretations of model predictions\nthrough an interactive user interface. We apply this generation framework to a\nquestion answering task called Quizbowl, where trivia enthusiasts craft\nadversarial questions. The resulting questions are validated via live\nhuman--computer matches: although the questions appear ordinary to humans, they\nsystematically stump neural and information retrieval models. The adversarial\nquestions cover diverse phenomena from multi-hop reasoning to entity type\ndistractors, exposing open challenges in robust question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02765v1",
            "date": "2018-09-08 06:43:01+00:00",
            "title": "Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model",
            "authors": [
                "Ruixuan Luo"
            ],
            "abstract": "Word embedding is designed to represent the semantic meaning of a word with\nlow dimensional vectors. The state-of-the-art methods of learning word\nembeddings (word2vec and GloVe) only use the word co-occurrence information.\nThe learned embeddings are real number vectors, which are obscure to human. In\nthis paper, we propose an Image-Enhanced Skip-Gram Model to learn grounded word\nembeddings by representing the word vectors in the same hyper-plane with image\nvectors. Experiments show that the image vectors and word embeddings learned by\nour model are highly correlated, which indicates that our model is able to\nprovide a vivid image-based explanation to the word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02768v2",
            "date": "2018-09-08 07:11:15+00:00",
            "title": "Generating Distractors for Reading Comprehension Questions from Real Examinations",
            "authors": [
                "Yifan Gao",
                "Lidong Bing",
                "Piji Li",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "We investigate the task of distractor generation for multiple choice reading\ncomprehension questions from examinations. In contrast to all previous works,\nwe do not aim at preparing words or short phrases distractors, instead, we\nendeavor to generate longer and semantic-rich distractors which are closer to\ndistractors in real reading comprehension from examinations. Taking a reading\ncomprehension article, a pair of question and its correct option as input, our\ngoal is to generate several distractors which are somehow related to the\nanswer, consistent with the semantic context of the question and have some\ntrace in the article. We propose a hierarchical encoder-decoder framework with\nstatic and dynamic attention mechanisms to tackle this task. Specifically, the\ndynamic attention can combine sentence-level and word-level attention varying\nat each recurrent time step to generate a more readable sequence. The static\nattention is to modulate the dynamic attention not to focus on question\nirrelevant sentences or sentences which contribute to the correct option. Our\nproposed framework outperforms several strong baselines on the first prepared\ndistractor generation dataset of real reading comprehension questions. For\nhuman evaluation, compared with those distractors generated by baselines, our\ngenerated distractors are more functional to confuse the annotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02782v2",
            "date": "2018-09-08 09:47:01+00:00",
            "title": "Sentiment analysis for Arabic language: A brief survey of approaches and techniques",
            "authors": [
                "Mo'ath Alrefai",
                "Hossam Faris",
                "Ibrahim Aljarah"
            ],
            "abstract": "With the emergence of Web 2.0 technology and the expansion of on-line social\nnetworks, current Internet users have the ability to add their reviews, ratings\nand opinions on social media and on commercial and news web sites. Sentiment\nanalysis aims to classify these reviews reviews in an automatic way. In the\nliterature, there are numerous approaches proposed for automatic sentiment\nanalysis for different language contexts. Each language has its own properties\nthat makes the sentiment analysis more challenging. In this regard, this work\npresents a comprehensive survey of existing Arabic sentiment analysis studies,\nand covers the various approaches and techniques proposed in the literature.\nMoreover, we highlight the main difficulties and challenges of Arabic sentiment\nanalysis, and the proposed techniques in literature to overcome these barriers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02789v1",
            "date": "2018-09-08 11:47:16+00:00",
            "title": "Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering",
            "authors": [
                "Todor Mihaylov",
                "Peter Clark",
                "Tushar Khot",
                "Ashish Sabharwal"
            ],
            "abstract": "We present a new kind of question answering dataset, OpenBookQA, modeled\nafter open book exams for assessing human understanding of a subject. The open\nbook that comes with our questions is a set of 1329 elementary level science\nfacts. Roughly 6000 questions probe an understanding of these facts and their\napplication to novel situations. This requires combining an open book fact\n(e.g., metals conduct electricity) with broad common knowledge (e.g., a suit of\narmor is made of metal) obtained from other sources. While existing QA datasets\nover documents or knowledge bases, being generally self-contained, focus on\nlinguistic understanding, OpenBookQA probes a deeper understanding of both the\ntopic---in the context of common knowledge---and the language it is expressed\nin. Human performance on OpenBookQA is close to 92%, but many state-of-the-art\npre-trained QA methods perform surprisingly poorly, worse than several simple\nneural baselines we develop. Our oracle experiments designed to circumvent the\nknowledge retrieval bottleneck demonstrate the value of both the open book and\nadditional facts. We leave it as a challenge to solve the retrieval problem in\nthis multi-hop setting and to close the large gap to human performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02794v3",
            "date": "2018-09-08 12:34:59+00:00",
            "title": "Explicit Contextual Semantics for Text Comprehension",
            "authors": [
                "Zhuosheng Zhang",
                "Yuwei Wu",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "Who did what to whom is a major focus in natural language understanding,\nwhich is right the aim of semantic role labeling (SRL) task. Despite of sharing\na lot of processing characteristics and even task purpose, it is surprisingly\nthat jointly considering these two related tasks was never formally reported in\nprevious work. Thus this paper makes the first attempt to let SRL enhance text\ncomprehension and inference through specifying verbal predicates and their\ncorresponding semantic roles. In terms of deep learning models, our embeddings\nare enhanced by explicit contextual semantic role labels for more fine-grained\nsemantics. We show that the salient labels can be conveniently added to\nexisting models and significantly improve deep learning models in challenging\ntext comprehension tasks. Extensive experiments on benchmark machine reading\ncomprehension and inference datasets verify that the proposed semantic learning\nhelps our system reach new state-of-the-art over strong baselines which have\nbeen enhanced by well pretrained language models from the latest progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02796v1",
            "date": "2018-09-08 12:49:18+00:00",
            "title": "Attentive Semantic Role Labeling with Boundary Indicator",
            "authors": [
                "Zhuosheng Zhang",
                "Shexia He",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "The goal of semantic role labeling (SRL) is to discover the\npredicate-argument structure of a sentence, which plays a critical role in deep\nprocessing of natural language. This paper introduces simple yet effective\nauxiliary tags for dependency-based SRL to enhance a syntax-agnostic model with\nmulti-hop self-attention. Our syntax-agnostic model achieves competitive\nperformance with state-of-the-art models on the CoNLL-2009 benchmarks both for\nEnglish and Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02847v2",
            "date": "2018-09-08 18:03:56+00:00",
            "title": "Interpreting Neural Networks With Nearest Neighbors",
            "authors": [
                "Eric Wallace",
                "Shi Feng",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Local model interpretation methods explain individual predictions by\nassigning an importance value to each input feature. This value is often\ndetermined by measuring the change in confidence when a feature is removed.\nHowever, the confidence of neural networks is not a robust measure of model\nuncertainty. This issue makes reliably judging the importance of the input\nfeatures difficult. We address this by changing the test-time behavior of\nneural networks using Deep k-Nearest Neighbors. Without harming text\nclassification accuracy, this algorithm provides a more robust uncertainty\nmetric which we use to generate feature importance values. The resulting\ninterpretations better align with human perception than baseline methods.\nFinally, we use our interpretation method to analyze model predictions on\ndataset annotation artifacts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02922v2",
            "date": "2018-09-09 05:03:34+00:00",
            "title": "Transforming Question Answering Datasets Into Natural Language Inference Datasets",
            "authors": [
                "Dorottya Demszky",
                "Kelvin Guu",
                "Percy Liang"
            ],
            "abstract": "Existing datasets for natural language inference (NLI) have propelled\nresearch on language understanding. We propose a new method for automatically\nderiving NLI datasets from the growing abundance of large-scale question\nanswering datasets. Our approach hinges on learning a sentence transformation\nmodel which converts question-answer pairs into their declarative forms.\nDespite being primarily trained on a single QA dataset, we show that it can be\nsuccessfully applied to a variety of other QA resources. Using this system, we\nautomatically derive a new freely available dataset of over 500k NLI examples\n(QA-NLI), and show that it exhibits a wide range of inference phenomena rarely\nseen in previous NLI datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03015v2",
            "date": "2018-09-09 18:01:33+00:00",
            "title": "Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?",
            "authors": [
                "Lena Reed",
                "Shereen Oraby",
                "Marilyn Walker"
            ],
            "abstract": "Responses in task-oriented dialogue systems often realize multiple\npropositions whose ultimate form depends on the use of sentence planning and\ndiscourse structuring operations. For example a recommendation may consist of\nan explicitly evaluative utterance e.g. Chanpen Thai is the best option, along\nwith content related by the justification discourse relation, e.g. It has great\nfood and service, that combines multiple propositions into a single phrase.\nWhile neural generation methods integrate sentence planning and surface\nrealization in one end-to-end learning framework, previous work has not shown\nthat neural generators can: (1) perform common sentence planning and discourse\nstructuring operations; (2) make decisions as to whether to realize content in\na single sentence or over multiple sentences; (3) generalize sentence planning\nand discourse relation operations beyond what was seen in training. We\nsystematically create large training corpora that exhibit particular sentence\nplanning operations and then test neural models to see what they learn. We\ncompare models without explicit latent variables for sentence planning with\nones that provide explicit supervision during training. We show that only the\nmodels with additional supervision can reproduce sentence planing and discourse\noperations and generalize to situations unseen in training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03056v1",
            "date": "2018-09-09 22:46:51+00:00",
            "title": "SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation",
            "authors": [
                "Shiva Taslimipoor",
                "Omid Rohanian"
            ],
            "abstract": "This paper presents a language-independent deep learning architecture adapted\nto the task of multiword expression (MWE) identification. We employ a neural\narchitecture comprising of convolutional and recurrent layers with the addition\nof an optional CRF layer at the top. This system participated in the open track\nof the Parseme shared task on automatic identification of verbal MWEs due to\nthe use of pre-trained wikipedia word embeddings. It outperformed all\nparticipating systems in both open and closed tracks with the overall\nmacro-average MWE-based F1 score of 58.09 averaged among all languages. A\nparticular strength of the system is its superior performance on unseen data\nentries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03068v1",
            "date": "2018-09-10 00:34:34+00:00",
            "title": "A case for deep learning in semantics",
            "authors": [
                "Christopher Potts"
            ],
            "abstract": "Pater's target article builds a persuasive case for establishing stronger\nties between theoretical linguistics and connectionism (deep learning). This\ncommentary extends his arguments to semantics, focusing in particular on issues\nof learning, compositionality, and lexical meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03112v1",
            "date": "2018-09-10 03:02:46+00:00",
            "title": "Depth-bounding is effective: Improvements and evaluation of unsupervised PCFG induction",
            "authors": [
                "Lifeng Jin",
                "Finale Doshi-Velez",
                "Timothy Miller",
                "William Schuler",
                "Lane Schwartz"
            ],
            "abstract": "There have been several recent attempts to improve the accuracy of grammar\ninduction systems by bounding the recursive complexity of the induction model\n(Ponvert et al., 2011; Noji and Johnson, 2016; Shain et al., 2016; Jin et al.,\n2018). Modern depth-bounded grammar inducers have been shown to be more\naccurate than early unbounded PCFG inducers, but this technique has never been\ncompared against unbounded induction within the same system, in part because\nmost previous depth-bounding models are built around sequence models, the\ncomplexity of which grows exponentially with the maximum allowed depth. The\npresent work instead applies depth bounds within a chart-based Bayesian PCFG\ninducer (Johnson et al., 2007b), where bounding can be switched on and off, and\nthen samples trees with and without bounding. Results show that depth-bounding\nis indeed significantly effective in limiting the search space of the inducer\nand thereby increasing the accuracy of the resulting parsing model. Moreover,\nparsing results on English, Chinese and German show that this bounded model\nwith a new inference technique is able to produce parse trees more accurately\nthan or competitively with state-of-the-art constituency-based grammar\ninduction models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03118v1",
            "date": "2018-09-10 03:33:48+00:00",
            "title": "A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification",
            "authors": [
                "Pengcheng Yang",
                "Shuming Ma",
                "Yi Zhang",
                "Junyang Lin",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Multi-label text classification (MLTC) aims to assign multiple labels to each\nsample in the dataset. The labels usually have internal correlations. However,\ntraditional methods tend to ignore the correlations between labels. In order to\ncapture the correlations between labels, the sequence-to-sequence (Seq2Seq)\nmodel views the MLTC task as a sequence generation problem, which achieves\nexcellent performance on this task. However, the Seq2Seq model is not suitable\nfor the MLTC task in essence. The reason is that it requires humans to\npredefine the order of the output labels, while some of the output labels in\nthe MLTC task are essentially an unordered set rather than an ordered sequence.\nThis conflicts with the strict requirement of the Seq2Seq model for the label\norder. In this paper, we propose a novel sequence-to-set framework utilizing\ndeep reinforcement learning, which not only captures the correlations between\nlabels, but also reduces the dependence on the label order. Extensive\nexperimental results show that our proposed method outperforms the competitive\nbaselines by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03132v1",
            "date": "2018-09-10 04:41:44+00:00",
            "title": "Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation",
            "authors": [
                "Chenze Shao",
                "Yang Feng",
                "Xilin Chen"
            ],
            "abstract": "Neural machine translation (NMT) models are usually trained with the\nword-level loss using the teacher forcing algorithm, which not only evaluates\nthe translation improperly but also suffers from exposure bias. Sequence-level\ntraining under the reinforcement framework can mitigate the problems of the\nword-level loss, but its performance is unstable due to the high variance of\nthe gradient estimation. On these grounds, we present a method with a\ndifferentiable sequence-level training objective based on probabilistic n-gram\nmatching which can avoid the reinforcement framework. In addition, this method\nperforms greedy search in the training which uses the predicted words as\ncontext just as at inference to alleviate the problem of exposure bias.\nExperiment results on the NIST Chinese-to-English translation tasks show that\nour method significantly outperforms the reinforcement-based algorithms and\nachieves an improvement of 1.5 BLEU points on average over a strong baseline\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03169v3",
            "date": "2018-09-10 08:05:56+00:00",
            "title": "Short-Term Meaning Shift: A Distributional Exploration",
            "authors": [
                "Marco Del Tredici",
                "Raquel Fern\u00e1ndez",
                "Gemma Boleda"
            ],
            "abstract": "We present the first exploration of meaning shift over short periods of time\nin online communities using distributional representations. We create a small\nannotated dataset and use it to assess the performance of a standard model for\nmeaning shift detection on short-term meaning shift. We find that the model has\nproblems distinguishing meaning shift from referential phenomena, and propose a\nmeasure of contextual variability to remedy this.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03182v1",
            "date": "2018-09-10 08:40:04+00:00",
            "title": "Towards one-shot learning for rare-word translation with external experts",
            "authors": [
                "Ngoc-Quan Pham",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "Neural machine translation (NMT) has significantly improved the quality of\nautomatic translation models. One of the main challenges in current systems is\nthe translation of rare words. We present a generic approach to address this\nweakness by having external models annotate the training data as Experts, and\ncontrol the model-expert interaction with a pointer network and reinforcement\nlearning. Our experiments using phrase-based models to simulate Experts to\ncomplement neural machine translation models show that the model can be trained\nto copy the annotations into the output consistently. We demonstrate the\nbenefit of our proposed framework in outof-domain translation scenarios with\nonly lexical resources, improving more than 1.0 BLEU point in both translation\ndirections English to Spanish and German to English",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03195v1",
            "date": "2018-09-10 09:10:49+00:00",
            "title": "Learning to Generate Structured Queries from Natural Language with Indirect Supervision",
            "authors": [
                "Ziwei Bai",
                "Bo Yu",
                "Bowen Wu",
                "Zhuoran Wang",
                "Baoxun Wang"
            ],
            "abstract": "Generating structured query language (SQL) from natural language is an\nemerging research topic. This paper presents a new learning paradigm from\nindirect supervision of the answers to natural language questions, instead of\nSQL queries. This paradigm facilitates the acquisition of training data due to\nthe abundant resources of question-answer pairs for various domains in the\nInternet, and expels the difficult SQL annotation job. An end-to-end neural\nmodel integrating with reinforcement learning is proposed to learn SQL\ngeneration policy within the answer-driven learning paradigm. The model is\nevaluated on datasets of different domains, including movie and academic\npublication. Experimental results show that our model outperforms the baseline\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03211v1",
            "date": "2018-09-10 09:31:24+00:00",
            "title": "Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks",
            "authors": [
                "Gor Arakelyan",
                "Karen Hambardzumyan",
                "Hrant Khachatrian"
            ],
            "abstract": "This paper describes our submission to CoNLL 2018 UD Shared Task. We have\nextended an LSTM-based neural network designed for sequence tagging to\nadditionally generate character-level sequences. The network was jointly\ntrained to produce lemmas, part-of-speech tags and morphological features.\nSentence segmentation, tokenization and dependency parsing were handled by\nUDPipe 1.2 baseline. The results demonstrate the viability of the proposed\nmultitask architecture, although its performance still remains far from\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03275v2",
            "date": "2018-09-10 12:41:21+00:00",
            "title": "Multilingual Extractive Reading Comprehension by Runtime Machine Translation",
            "authors": [
                "Akari Asai",
                "Akiko Eriguchi",
                "Kazuma Hashimoto",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "Despite recent work in Reading Comprehension (RC), progress has been mostly\nlimited to English due to the lack of large-scale datasets in other languages.\nIn this work, we introduce the first RC system for languages without RC\ntraining data. Given a target language without RC training data and a pivot\nlanguage with RC training data (e.g. English), our method leverages existing RC\nresources in the pivot language by combining a competitive RC model in the\npivot language with an attentive Neural Machine Translation (NMT) model. We\nfirst translate the data from the target to the pivot language, and then obtain\nan answer using the RC model in the pivot language. Finally, we recover the\ncorresponding answer in the original language using soft-alignment attention\nscores from the NMT model. We create evaluation sets of RC data in two\nnon-English languages, namely Japanese and French, to evaluate our method.\nExperimental results on these datasets show that our method significantly\noutperforms a back-translation baseline of a state-of-the-art product-level\nmachine translation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03348v1",
            "date": "2018-09-10 14:27:08+00:00",
            "title": "xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks",
            "authors": [
                "Ting-Yun Chang",
                "Ta-Chung Chi",
                "Shang-Chi Tsai",
                "Yun-Nung Chen"
            ],
            "abstract": "Despite the success achieved on various natural language processing tasks,\nword embeddings are difficult to interpret due to the dense vector\nrepresentations. This paper focuses on interpreting the embeddings for various\naspects, including sense separation in the vector dimensions and definition\ngeneration. Specifically, given a context together with a target word, our\nalgorithm first projects the target word embedding to a high-dimensional sparse\nvector and picks the specific dimensions that can best explain the semantic\nmeaning of the target word by the encoded contextual information, where the\nsense of the target word can be indirectly inferred. Finally, our algorithm\napplies an RNN to generate the textual definition of the target word in the\nhuman readable form, which enables direct interpretation of the corresponding\nword embedding. This paper also introduces a large and high-quality\ncontext-definition dataset that consists of sense definitions together with\nmultiple example sentences per polysemous word, which is a valuable resource\nfor definition modeling and word sense disambiguation. The conducted\nexperiments show the superior performance in BLEU score and the human\nevaluation test.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03391v3",
            "date": "2018-09-10 15:23:48+00:00",
            "title": "Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging",
            "authors": [
                "Kemal Kurniawan",
                "Alham Fikri Aji"
            ],
            "abstract": "Previous work in Indonesian part-of-speech (POS) tagging are hard to compare\nas they are not evaluated on a common dataset. Furthermore, in spite of the\nsuccess of neural network models for English POS tagging, they are rarely\nexplored for Indonesian. In this paper, we explored various techniques for\nIndonesian POS tagging, including rule-based, CRF, and neural network-based\nmodels. We evaluated our models on the IDN Tagged Corpus. A new\nstate-of-the-art of 97.47 F1 score is achieved with a recurrent neural network.\nTo provide a standard for future work, we release the dataset split that we\nused publicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03401v1",
            "date": "2018-09-10 15:37:30+00:00",
            "title": "Neural Latent Relational Analysis to Capture Lexical Semantic Relations in a Vector Space",
            "authors": [
                "Koki Washio",
                "Tsuneaki Kato"
            ],
            "abstract": "Capturing the semantic relations of words in a vector space contributes to\nmany natural language processing tasks. One promising approach exploits\nlexico-syntactic patterns as features of word pairs. In this paper, we propose\na novel model of this pattern-based approach, neural latent relational analysis\n(NLRA). NLRA can generalize co-occurrences of word pairs and lexico-syntactic\npatterns, and obtain embeddings of the word pairs that do not co-occur. This\novercomes the critical data sparseness problem encountered in previous\npattern-based models. Our experimental results on measuring relational\nsimilarity demonstrate that NLRA outperforms the previous pattern-based models.\nIn addition, when combined with a vector offset model, NLRA achieves a\nperformance comparable to that of the state-of-the-art model that exploits\nadditional semantic relational data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03411v1",
            "date": "2018-09-10 15:47:37+00:00",
            "title": "Filling Missing Paths: Modeling Co-occurrences of Word Pairs and Dependency Paths for Recognizing Lexical Semantic Relations",
            "authors": [
                "Koki Washio",
                "Tsuneaki Kato"
            ],
            "abstract": "Recognizing lexical semantic relations between word pairs is an important\ntask for many applications of natural language processing. One of the\nmainstream approaches to this task is to exploit the lexico-syntactic paths\nconnecting two target words, which reflect the semantic relations of word\npairs. However, this method requires that the considered words co-occur in a\nsentence. This requirement is hardly satisfied because of Zipf's law, which\nstates that most content words occur very rarely. In this paper, we propose\nnovel methods with a neural model of $P(path|w_1, w_2)$ to solve this problem.\nOur proposed model of $P(path|w_1, w_2)$ can be learned in an unsupervised\nmanner and can generalize the co-occurrences of word pairs and dependency\npaths. This model can be used to augment the path data of word pairs that do\nnot co-occur in the corpus, and extract features capturing relational\ninformation from word pairs. Our experimental results demonstrate that our\nmethods improve on previous neural approaches based on dependency paths and\nsuccessfully solve the focused problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03485v1",
            "date": "2018-09-10 17:57:10+00:00",
            "title": "Multi-view Models for Political Ideology Detection of News Articles",
            "authors": [
                "Vivek Kulkarni",
                "Junting Ye",
                "Steven Skiena",
                "William Yang Wang"
            ],
            "abstract": "A news article's title, content and link structure often reveal its political\nideology. However, most existing works on automatic political ideology\ndetection only leverage textual cues. Drawing inspiration from recent advances\nin neural inference, we propose a novel attention based multi-view model to\nleverage cues from all of the above views to identify the ideology evinced by a\nnews article. Our model draws on advances in representation learning in natural\nlanguage processing and network science to capture cues from both textual\ncontent and the network structure of news articles. We empirically evaluate our\nmodel against a battery of baselines and show that our model outperforms state\nof the art by 10 percentage points F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03568v3",
            "date": "2018-09-05 13:04:56+00:00",
            "title": "Improving Question Answering by Commonsense-Based Pre-Training",
            "authors": [
                "Wanjun Zhong",
                "Duyu Tang",
                "Nan Duan",
                "Ming Zhou",
                "Jiahai Wang",
                "Jian Yin"
            ],
            "abstract": "Although neural network approaches achieve remarkable success on a variety of\nNLP tasks, many of them struggle to answer questions that require commonsense\nknowledge. We believe the main reason is the lack of commonsense\n\\mbox{connections} between concepts. To remedy this, we provide a simple and\neffective method that leverages external commonsense knowledge base such as\nConceptNet. We pre-train direct and indirect relational functions between\nconcepts, and show that these pre-trained functions could be easily added to\nexisting neural network models. Results show that incorporating\ncommonsense-based function improves the baseline on three question answering\ntasks that require commonsense reasoning. Further analysis shows that our\nsystem \\mbox{discovers} and leverages useful evidence from an external\ncommonsense knowledge base, which is missing in existing neural network models\nand help derive the correct answer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03599v1",
            "date": "2018-09-10 21:15:30+00:00",
            "title": "Learning Named Entity Tagger using Domain-Specific Dictionary",
            "authors": [
                "Jingbo Shang",
                "Liyuan Liu",
                "Xiang Ren",
                "Xiaotao Gu",
                "Teng Ren",
                "Jiawei Han"
            ],
            "abstract": "Recent advances in deep neural models allow us to build reliable named entity\nrecognition (NER) systems without handcrafting features. However, such methods\nrequire large amounts of manually-labeled training data. There have been\nefforts on replacing human annotations with distant supervision (in conjunction\nwith external dictionaries), but the generated noisy labels pose significant\nchallenges on learning effective neural models. Here we propose two neural\nmodels to suit noisy distant supervision from the dictionary. First, under the\ntraditional sequence labeling framework, we propose a revised fuzzy CRF layer\nto handle tokens with multiple possible labels. After identifying the nature of\nnoisy labels in distant supervision, we go beyond the traditional framework and\npropose a novel, more effective neural model AutoNER with a new Tie or Break\nscheme. In addition, we discuss how to refine distant supervision for better\nNER performance. Extensive experiments on three benchmark datasets demonstrate\nthat AutoNER achieves the best performance when only using dictionaries with no\nadditional human effort, and delivers competitive results with state-of-the-art\nsupervised benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03632v1",
            "date": "2018-09-10 23:20:00+00:00",
            "title": "Detecting Gang-Involved Escalation on Social Media Using Context",
            "authors": [
                "Serina Chang",
                "Ruiqi Zhong",
                "Ethan Adams",
                "Fei-Tzin Lee",
                "Siddharth Varia",
                "Desmond Patton",
                "William Frey",
                "Chris Kedzie",
                "Kathleen McKeown"
            ],
            "abstract": "Gang-involved youth in cities such as Chicago have increasingly turned to\nsocial media to post about their experiences and intents online. In some\nsituations, when they experience the loss of a loved one, their online\nexpression of emotion may evolve into aggression towards rival gangs and\nultimately into real-world violence. In this paper, we present a novel system\nfor detecting Aggression and Loss in social media. Our system features the use\nof domain-specific resources automatically derived from a large unlabeled\ncorpus, and contextual representations of the emotional and semantic content of\nthe user's recent tweets as well as their interactions with other users.\nIncorporating context in our Convolutional Neural Network (CNN) leads to a\nsignificant improvement.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03633v1",
            "date": "2018-09-10 23:22:43+00:00",
            "title": "Unsupervised Cross-lingual Transfer of Word Embedding Spaces",
            "authors": [
                "Ruochen Xu",
                "Yiming Yang",
                "Naoki Otani",
                "Yuexin Wu"
            ],
            "abstract": "Cross-lingual transfer of word embeddings aims to establish the semantic\nmappings among words in different languages by learning the transformation\nfunctions over the corresponding word embedding spaces. Successfully solving\nthis problem would benefit many downstream tasks such as to translate text\nclassification models from resource-rich languages (e.g. English) to\nlow-resource languages. Supervised methods for this problem rely on the\navailability of cross-lingual supervision, either using parallel corpora or\nbilingual lexicons as the labeled data for training, which may not be available\nfor many low resource languages. This paper proposes an unsupervised learning\napproach that does not require any cross-lingual labeled data. Given two\nmonolingual word embedding spaces for any language pair, our algorithm\noptimizes the transformation functions in both directions simultaneously based\non distributional matching as well as minimizing the back-translation losses.\nWe use a neural network implementation to calculate the Sinkhorn distance, a\nwell-defined distributional similarity measure, and optimize our objective\nthrough back-propagation. Our evaluation on benchmark datasets for bilingual\nlexicon induction and cross-lingual word similarity prediction shows stronger\nor competitive performance of the proposed method compared to other\nstate-of-the-art supervised and unsupervised baseline methods over many\nlanguage pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03664v1",
            "date": "2018-09-11 03:03:37+00:00",
            "title": "Topic Memory Networks for Short Text Classification",
            "authors": [
                "Jichuan Zeng",
                "Jing Li",
                "Yan Song",
                "Cuiyun Gao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Many classification models work poorly on short texts due to data sparsity.\nTo address this issue, we propose topic memory networks for short text\nclassification with a novel topic memory mechanism to encode latent topic\nrepresentations indicative of class labels. Different from most prior work that\nfocuses on extending features with external knowledge or pre-trained topics,\nour model jointly explores topic inference and text classification with memory\nnetworks in an end-to-end manner. Experimental results on four benchmark\ndatasets show that our model outperforms state-of-the-art models on short text\nclassification, meanwhile generates coherent topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03680v1",
            "date": "2018-09-11 05:02:28+00:00",
            "title": "Learning Scripts as Hidden Markov Models",
            "authors": [
                "J. Walker Orr",
                "Prasad Tadepalli",
                "Janardhan Rao Doppa",
                "Xiaoli Fern",
                "Thomas G. Dietterich"
            ],
            "abstract": "Scripts have been proposed to model the stereotypical event sequences found\nin narratives. They can be applied to make a variety of inferences including\nfilling gaps in the narratives and resolving ambiguous references. This paper\nproposes the first formal framework for scripts based on Hidden Markov Models\n(HMMs). Our framework supports robust inference and learning algorithms, which\nare lacking in previous clustering models. We develop an algorithm for\nstructure and parameter learning based on Expectation Maximization and evaluate\nit on a number of natural datasets. The results show that our algorithm is\nsuperior to several informed baselines for predicting missing events in partial\nobservation sequences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03690v1",
            "date": "2018-09-11 06:13:37+00:00",
            "title": "A Joint Model of Conversational Discourse and Latent Topics on Microblogs",
            "authors": [
                "Jing Li",
                "Yan Song",
                "Zhongyu Wei",
                "Kam-Fai Wong"
            ],
            "abstract": "Conventional topic models are ineffective for topic extraction from microblog\nmessages, because the data sparseness exhibited in short messages lacking\nstructure and contexts results in poor message-level word co-occurrence\npatterns. To address this issue, we organize microblog messages as conversation\ntrees based on their reposting and replying relations, and propose an\nunsupervised model that jointly learns word distributions to represent: 1)\ndifferent roles of conversational discourse, 2) various latent topics in\nreflecting content information. By explicitly distinguishing the probabilities\nof messages with varying discourse roles in containing topical words, our model\nis able to discover clusters of discourse words that are indicative of topical\ncontent. In an automatic evaluation on large-scale microblog corpora, our joint\nmodel yields topics with better coherence scores than competitive topic models\nfrom previous studies. Qualitative analysis on model outputs indicates that our\nmodel induces meaningful representations for both discourse and topics. We\nfurther present an empirical study on microblog summarization based on the\noutputs of our joint model. The results show that the jointly modeled discourse\nand topic representations can effectively indicate summary-worthy content in\nmicroblog conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03734v1",
            "date": "2018-09-11 08:26:36+00:00",
            "title": "How much should you ask? On the question structure in QA systems",
            "authors": [
                "Dominika Basaj",
                "Barbara Rychalska",
                "Przemyslaw Biecek",
                "Anna Wroblewska"
            ],
            "abstract": "Datasets that boosted state-of-the-art solutions for Question Answering (QA)\nsystems prove that it is possible to ask questions in natural language manner.\nHowever, users are still used to query-like systems where they type in keywords\nto search for answer. In this study we validate which parts of questions are\nessential for obtaining valid answer. In order to conclude that, we take\nadvantage of LIME - a framework that explains prediction by local\napproximation. We find that grammar and natural language is disregarded by QA.\nState-of-the-art model can answer properly even if 'asked' only with a few\nwords with high coefficients calculated with LIME. According to our knowledge,\nit is the first time that QA model is being explained by LIME.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03740v1",
            "date": "2018-09-11 08:37:07+00:00",
            "title": "Does it care what you asked? Understanding Importance of Verbs in Deep Learning QA System",
            "authors": [
                "Barbara Rychalska",
                "Dominika Basaj",
                "Przemyslaw Biecek",
                "Anna Wroblewska"
            ],
            "abstract": "In this paper we present the results of an investigation of the importance of\nverbs in a deep learning QA system trained on SQuAD dataset. We show that main\nverbs in questions carry little influence on the decisions made by the system -\nin over 90% of researched cases swapping verbs for their antonyms did not\nchange system decision. We track this phenomenon down to the insides of the\nnet, analyzing the mechanism of self-attention and values contained in hidden\nlayers of RNN. Finally, we recognize the characteristics of the SQuAD dataset\nas the source of the problem. Our work refers to the recently popular topic of\nadversarial examples in NLP, combined with investigating deep net structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03891v1",
            "date": "2018-09-11 13:44:48+00:00",
            "title": "Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus",
            "authors": [
                "Yonatan Belinkov",
                "Alexander Magidow",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Avi Shmidman",
                "Maxim Romanov"
            ],
            "abstract": "Arabic is a widely-spoken language with a long and rich history, but existing\ncorpora and language technology focus mostly on modern Arabic and its\nvarieties. Therefore, studying the history of the language has so far been\nmostly limited to manual analyses on a small scale. In this work, we present a\nlarge-scale historical corpus of the written Arabic language, spanning 1400\nyears. We describe our efforts to clean and process this corpus using Arabic\nNLP tools, including the identification of reused text. We study the history of\nthe Arabic language using a novel automatic periodization algorithm, as well as\nother techniques. Our findings confirm the established division of written\nArabic into Modern Standard and Classical Arabic, and confirm other established\nperiodizations, while suggesting that written Arabic may be divisible into\nstill further periods of development.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03944v1",
            "date": "2018-09-11 14:49:05+00:00",
            "title": "Multilingual Cross-domain Perspectives on Online Hate Speech",
            "authors": [
                "Tom De Smedt",
                "Sylvia Jaki",
                "Eduan Kotz\u00e9",
                "Le\u00efla Saoud",
                "Maja Gw\u00f3\u017ad\u017a",
                "Guy De Pauw",
                "Walter Daelemans"
            ],
            "abstract": "In this report, we present a study of eight corpora of online hate speech, by\ndemonstrating the NLP techniques that we used to collect and analyze the\njihadist, extremist, racist, and sexist content. Analysis of the multilingual\ncorpora shows that the different contexts share certain characteristics in\ntheir hateful rhetoric. To expose the main features, we have focused on text\nclassification, text profiling, keyword and collocation extraction, along with\nmanual annotation and qualitative study.",
            "journal ref": "CLiPS Technical Report Series 8 (2018) 1-24",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03985v1",
            "date": "2018-09-11 15:41:12+00:00",
            "title": "On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation",
            "authors": [
                "Tamer Alkhouli",
                "Gabriel Bretschner",
                "Hermann Ney"
            ],
            "abstract": "This work investigates the alignment problem in state-of-the-art multi-head\nattention models based on the transformer architecture. We demonstrate that\nalignment extraction in transformer models can be improved by augmenting an\nadditional alignment head to the multi-head source-to-target attention\ncomponent. This is used to compute sharper attention weights. We describe how\nto use the alignment head to achieve competitive performance. To study the\neffect of adding the alignment head, we simulate a dictionary-guided\ntranslation task, where the user wants to guide translation using pre-defined\ndictionary entries. Using the proposed approach, we achieve up to $3.8$ % BLEU\nimprovement when using the dictionary, in comparison to $2.4$ % BLEU in the\nbaseline case. We also propose alignment pruning to speed up decoding in\nalignment-based neural machine translation (ANMT), which speeds up translation\nby a factor of $1.8$ without loss in translation performance. We carry out\nexperiments on the shared WMT 2016 English$\\to$Romanian news task and the BOLT\nChinese$\\to$English discussion forum task.",
            "journal ref": "WMT 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03992v1",
            "date": "2018-09-11 15:56:46+00:00",
            "title": "Assessing Composition in Sentence Vector Representations",
            "authors": [
                "Allyson Ettinger",
                "Ahmed Elgohary",
                "Colin Phillips",
                "Philip Resnik"
            ],
            "abstract": "An important component of achieving language understanding is mastering the\ncomposition of sentence meaning, but an immediate challenge to solving this\nproblem is the opacity of sentence vector representations produced by current\nneural sentence composition models. We present a method to address this\nchallenge, developing tasks that directly target compositional meaning\ninformation in sentence vector representations with a high degree of precision\nand control. To enable the creation of these controlled tasks, we introduce a\nspecialized sentence generation system that produces large, annotated sentence\nsets meeting specified syntactic, semantic and lexical constraints. We describe\nthe details of the method and generation system, and then present results of\nexperiments applying our method to probe for compositional information in\nembeddings from a number of existing sentence composition models. We find that\nthe method is able to extract useful information about the differing capacities\nof these models, and we discuss the implications of our results with respect to\nthese systems' capturing of sentence information. We make available for public\nuse the datasets used for these experiments, as well as the generation system.",
            "journal ref": "In Proceedings of the 27th International Conference on\n  Computational Linguistics (pp. 1790-1801)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03999v1",
            "date": "2018-09-11 16:09:53+00:00",
            "title": "Evaluating Semantic Rationality of a Sentence: A Sememe-Word-Matching Neural Network based on HowNet",
            "authors": [
                "Shu Liu",
                "Jingjing Xu",
                "Xuancheng Ren",
                "Xu Sun"
            ],
            "abstract": "Automatic evaluation of semantic rationality is an important yet challenging\ntask, and current automatic techniques cannot well identify whether a sentence\nis semantically rational. The methods based on the language model do not\nmeasure the sentence by rationality but by commonness. The methods based on the\nsimilarity with human written sentences will fail if human-written references\nare not available. In this paper, we propose a novel model called\nSememe-Word-Matching Neural Network (SWM-NN) to tackle semantic rationality\nevaluation by taking advantage of sememe knowledge base HowNet. The advantage\nis that our model can utilize a proper combination of sememes to represent the\nfine-grained semantic meanings of a word within the specific contexts. We use\nthe fine-grained semantic representation to help the model learn the semantic\ndependency among words. To evaluate the effectiveness of the proposed model, we\nbuild a large-scale rationality evaluation dataset. Experimental results on\nthis dataset show that the proposed model outperforms the competitive baselines\nwith a 5.4\\% improvement in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04022v4",
            "date": "2018-09-11 16:44:02+00:00",
            "title": "Can LSTM Learn to Capture Agreement? The Case of Basque",
            "authors": [
                "Shauli Ravfogel",
                "Francis M. Tyers",
                "Yoav Goldberg"
            ],
            "abstract": "Sequential neural networks models are powerful tools in a variety of Natural\nLanguage Processing (NLP) tasks. The sequential nature of these models raises\nthe questions: to what extent can these models implicitly learn hierarchical\nstructures typical to human language, and what kind of grammatical phenomena\ncan they acquire?\n  We focus on the task of agreement prediction in Basque, as a case study for a\ntask that requires implicit understanding of sentence structure and the\nacquisition of a complex but consistent morphological system. Analyzing\nexperimental results from two syntactic prediction tasks -- verb number\nprediction and suffix recovery -- we find that sequential models perform worse\non agreement prediction in Basque than one might expect on the basis of a\nprevious agreement prediction work in English. Tentative findings based on\ndiagnostic classifiers suggest the network makes use of local heuristics as a\nproxy for the hierarchical structure of the sentence. We propose the Basque\nagreement prediction task as challenging benchmark for models that attempt to\nlearn regularities in human language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04047v2",
            "date": "2018-09-11 17:30:12+00:00",
            "title": "AWE: Asymmetric Word Embedding for Textual Entailment",
            "authors": [
                "Tengfei Ma",
                "Chiamin Wu",
                "Cao Xiao",
                "Jimeng Sun"
            ],
            "abstract": "Textual entailment is a fundamental task in natural language processing. It\nrefers to the directional relation between text fragments such that the\n\"premise\" can infer \"hypothesis\". In recent years deep learning methods have\nachieved great success in this task. Many of them have considered the\ninter-sentence word-word interactions between the premise-hypothesis pairs,\nhowever, few of them considered the \"asymmetry\" of these interactions.\nDifferent from paraphrase identification or sentence similarity evaluation,\ntextual entailment is essentially determining a directional (asymmetric)\nrelation between the premise and the hypothesis. In this paper, we propose a\nsimple but effective way to enhance existing textual entailment algorithms by\nusing asymmetric word embeddings. Experimental results on SciTail and SNLI\ndatasets show that the learned asymmetric word embeddings could significantly\nimprove the word-word interaction based textual entailment models. It is\nnoteworthy that the proposed AWE-DeIsTe model can get 2.1% accuracy improvement\nover prior state-of-the-art on SciTail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04128v1",
            "date": "2018-09-11 19:52:44+00:00",
            "title": "Limitations in learning an interpreted language with recurrent models",
            "authors": [
                "Denis Paperno"
            ],
            "abstract": "In this submission I report work in progress on learning simplified\ninterpreted languages by means of recurrent models. The data is constructed to\nreflect core properties of natural language as modeled in formal syntax and\nsemantics: recursive syntactic structure and compositionality. Preliminary\nresults suggest that LSTM networks do generalise to compositional\ninterpretation, albeit only in the most favorable learning setting, with a\nwell-paced curriculum, extensive training data, and left-to-right (but not\nright-to-left) composition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04163v1",
            "date": "2018-09-11 21:08:00+00:00",
            "title": "Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization",
            "authors": [
                "Edoardo Maria Ponti",
                "Ivan Vuli\u0107",
                "Goran Glava\u0161",
                "Nikola Mrk\u0161i\u0107",
                "Anna Korhonen"
            ],
            "abstract": "Semantic specialization is the process of fine-tuning pre-trained\ndistributional word vectors using external lexical knowledge (e.g., WordNet) to\naccentuate a particular semantic relation in the specialized vector space.\nWhile post-processing specialization methods are applicable to arbitrary\ndistributional vectors, they are limited to updating only the vectors of words\noccurring in external lexicons (i.e., seen words), leaving the vectors of all\nother words unchanged. We propose a novel approach to specializing the full\ndistributional vocabulary. Our adversarial post-specialization method\npropagates the external lexical knowledge to the full distributional space. We\nexploit words seen in the resources as training examples for learning a global\nspecialization function. This function is learned by combining a standard\nL2-distance loss with an adversarial loss: the adversarial component produces\nmore realistic output vectors. We show the effectiveness and robustness of the\nproposed method across three languages and on three tasks: word similarity,\ndialog state tracking, and lexical simplification. We report consistent\nimprovements over distributional word vectors and vectors specialized by other\nstate-of-the-art specialization frameworks. Finally, we also propose a\ncross-lingual transfer method for zero-shot specialization which successfully\nspecializes a full target distributional space without any lexical knowledge in\nthe target language and without any bilingual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04179v2",
            "date": "2018-09-11 21:55:11+00:00",
            "title": "What can linguistics and deep learning contribute to each other?",
            "authors": [
                "Tal Linzen"
            ],
            "abstract": "Joe Pater's target article calls for greater interaction between neural\nnetwork research and linguistics. I expand on this call and show how such\ninteraction can benefit both fields. Linguists can contribute to research on\nneural networks for language technologies by clearly delineating the linguistic\ncapabilities that can be expected of such systems, and by constructing\ncontrolled experimental paradigms that can determine whether those desiderata\nhave been met. In the other direction, neural networks can benefit the\nscientific study of language by providing infrastructure for modeling human\nsentence processing and for evaluating the necessity of particular innate\nconstraints on language acquisition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04203v1",
            "date": "2018-09-12 00:09:20+00:00",
            "title": "Multimodal neural pronunciation modeling for spoken languages with logographic origin",
            "authors": [
                "Minh Nguyen",
                "Gia H. Ngo",
                "Nancy F. Chen"
            ],
            "abstract": "Graphemes of most languages encode pronunciation, though some are more\nexplicit than others. Languages like Spanish have a straightforward mapping\nbetween its graphemes and phonemes, while this mapping is more convoluted for\nlanguages like English. Spoken languages such as Cantonese present even more\nchallenges in pronunciation modeling: (1) they do not have a standard written\nform, (2) the closest graphemic origins are logographic Han characters, of\nwhich only a subset of these logographic characters implicitly encodes\npronunciation. In this work, we propose a multimodal approach to predict the\npronunciation of Cantonese logographic characters, using neural networks with a\ngeometric representation of logographs and pronunciation of cognates in\nhistorically related languages. The proposed framework improves performance by\n18.1% and 25.0% respective to unimodal and multimodal baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04267v1",
            "date": "2018-09-12 06:21:32+00:00",
            "title": "Knowledge Based Machine Reading Comprehension",
            "authors": [
                "Yibo Sun",
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Zhao Yan",
                "Xiaocheng Feng",
                "Bing Qin"
            ],
            "abstract": "Machine reading comprehension (MRC) requires reasoning about both the\nknowledge involved in a document and knowledge about the world. However,\nexisting datasets are typically dominated by questions that can be well solved\nby context matching, which fail to test this capability. To encourage the\nprogress on knowledge-based reasoning in MRC, we present knowledge-based MRC in\nthis paper, and build a new dataset consisting of 40,047 question-answer pairs.\nThe annotation of this dataset is designed so that successfully answering the\nquestions requires understanding and the knowledge involved in a document. We\nimplement a framework consisting of both a question answering model and a\nquestion generation model, both of which take the knowledge extracted from the\ndocument as well as relevant facts from an external knowledge base such as\nFreebase/ProBase/Reverb/NELL. Results show that incorporating side information\nfrom external KB improves the accuracy of the baseline question answer system.\nWe compare it with a standard MRC model BiDAF, and also provide the difficulty\nof the dataset and lay out remaining challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04271v1",
            "date": "2018-09-12 06:37:51+00:00",
            "title": "Knowledge-Aware Conversational Semantic Parsing Over Web Tables",
            "authors": [
                "Yibo Sun",
                "Duyu Tang",
                "Nan Duan",
                "Jingjing Xu",
                "Xiaocheng Feng",
                "Bing Qin"
            ],
            "abstract": "Conversational semantic parsing over tables requires knowledge acquiring and\nreasoning abilities, which have not been well explored by current\nstate-of-the-art approaches. Motivated by this fact, we propose a\nknowledge-aware semantic parser to improve parsing performance by integrating\nvarious types of knowledge. In this paper, we consider three types of\nknowledge, including grammar knowledge, expert knowledge, and external resource\nknowledge. First, grammar knowledge empowers the model to effectively replicate\npreviously generated logical form, which effectively handles the co-reference\nand ellipsis phenomena in conversation Second, based on expert knowledge, we\npropose a decomposable model, which is more controllable compared with\ntraditional end-to-end models that put all the burdens of learning on\ntrial-and-error in an end-to-end way. Third, external resource knowledge, i.e.,\nprovided by a pre-trained language model or an entity typing model, is used to\nimprove the representation of question and table for a better semantic\nunderstanding. We conduct experiments on the SequentialQA dataset. Results show\nthat our knowledge-aware model outperforms the state-of-the-art approaches.\nIncremental experimental results also prove the usefulness of various\nknowledge. Further analysis shows that our approach has the ability to derive\nthe meaning representation of a context-dependent utterance by leveraging\npreviously generated outcomes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04276v2",
            "date": "2018-09-12 06:47:21+00:00",
            "title": "Retrieval-Enhanced Adversarial Training for Neural Response Generation",
            "authors": [
                "Qingfu Zhu",
                "Lei Cui",
                "Weinan Zhang",
                "Furu Wei",
                "Ting Liu"
            ],
            "abstract": "Dialogue systems are usually built on either generation-based or\nretrieval-based approaches, yet they do not benefit from the advantages of\ndifferent models. In this paper, we propose a Retrieval-Enhanced Adversarial\nTraining (REAT) method for neural response generation. Distinct from existing\napproaches, the REAT method leverages an encoder-decoder framework in terms of\nan adversarial training paradigm, while taking advantage of N-best response\ncandidates from a retrieval-based system to construct the discriminator. An\nempirical study on a large scale public available benchmark dataset shows that\nthe REAT method significantly outperforms the vanilla Seq2Seq model as well as\nthe conventional adversarial training approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04444v1",
            "date": "2018-09-12 13:51:02+00:00",
            "title": "Hate Speech Dataset from a White Supremacy Forum",
            "authors": [
                "Ona de Gibert",
                "Naiara Perez",
                "Aitor Garc\u00eda-Pablos",
                "Montse Cuadros"
            ],
            "abstract": "Hate speech is commonly defined as any communication that disparages a target\ngroup of people based on some characteristic such as race, colour, ethnicity,\ngender, sexual orientation, nationality, religion, or other characteristic. Due\nto the massive rise of user-generated web content on social media, the amount\nof hate speech is also steadily increasing. Over the past years, interest in\nonline hate speech detection and, particularly, the automation of this task has\ncontinuously grown, along with the societal impact of the phenomenon. This\npaper describes a hate speech dataset composed of thousands of sentences\nmanually labelled as containing hate speech or not. The sentences have been\nextracted from Stormfront, a white supremacist forum. A custom annotation tool\nhas been developed to carry out the manual labelling task which, among other\nthings, allows the annotators to choose whether to read the context of a\nsentence before labelling it. The paper also provides a thoughtful qualitative\nand quantitative study of the resulting dataset and several baseline\nexperiments with different classification models. The dataset is publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04505v1",
            "date": "2018-09-12 15:08:00+00:00",
            "title": "Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training",
            "authors": [
                "Peng Xu",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Ji Ho Park",
                "Pascale Fung"
            ],
            "abstract": "In this paper, we propose Emo2Vec which encodes emotional semantics into\nvectors. We train Emo2Vec by multi-task learning six different emotion-related\ntasks, including emotion/sentiment analysis, sarcasm classification, stress\ndetection, abusive language classification, insult detection, and personality\nrecognition. Our evaluation of Emo2Vec shows that it outperforms existing\naffect-related representations, such as Sentiment-Specific Word Embedding and\nDeepMoji embeddings with much smaller training corpora. When concatenated with\nGloVe, Emo2Vec achieves competitive performances to state-of-the-art results on\nseveral tasks using a simple logistic regression classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04557v1",
            "date": "2018-09-11 13:29:08+00:00",
            "title": "Solving Sinhala Language Arithmetic Problems using Neural Networks",
            "authors": [
                "W. M. T Chathurika",
                "K. C. E De Silva",
                "A. M. Raddella",
                "E. M. R. S. Ekanayake",
                "A. Nugaliyadde",
                "Y. Mallawarachchi"
            ],
            "abstract": "A methodology is presented to solve Arithmetic problems in Sinhala Language\nusing a Neural Network. The system comprises of (a) keyword identification, (b)\nquestion identification, (c) mathematical operation identification and is\ncombined using a neural network. Naive Bayes Classification is used in order to\nidentify keywords and Conditional Random Field to identify the question and the\noperation which should be performed on the identified keywords to achieve the\nexpected result. \"One vs. all Classification\" is done using a neural network\nfor sentences. All functions are combined through the neural network which\nbuilds an equation to solve the problem. The paper compares each methodology in\nARIS and Mahoshadha to the method presented in the paper. Mahoshadha2 learns to\nsolve arithmetic problems with the accuracy of 76%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04640v2",
            "date": "2018-09-12 19:18:16+00:00",
            "title": "Jump to better conclusions: SCAN both left and right",
            "authors": [
                "Jasmijn Bastings",
                "Marco Baroni",
                "Jason Weston",
                "Kyunghyun Cho",
                "Douwe Kiela"
            ],
            "abstract": "Lake and Baroni (2018) recently introduced the SCAN data set, which consists\nof simple commands paired with action sequences and is intended to test the\nstrong generalization abilities of recurrent sequence-to-sequence models. Their\ninitial experiments suggested that such models may fail because they lack the\nability to extract systematic rules. Here, we take a closer look at SCAN and\nshow that it does not always capture the kind of generalization that it was\ndesigned for. To mitigate this we propose a complementary dataset, which\nrequires mapping actions back to the original commands, called NACS. We show\nthat models that do well on SCAN do not necessarily do well on NACS, and that\nNACS exhibits properties more closely aligned with realistic use-cases for\nsequence-to-sequence models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04649v1",
            "date": "2018-09-12 19:53:56+00:00",
            "title": "Semantic WordRank: Generating Finer Single-Document Summarizations",
            "authors": [
                "Hao Zhang",
                "Jie Wang"
            ],
            "abstract": "We present Semantic WordRank (SWR), an unsupervised method for generating an\nextractive summary of a single document. Built on a weighted word graph with\nsemantic and co-occurrence edges, SWR scores sentences using an\narticle-structure-biased PageRank algorithm with a Softplus function\nadjustment, and promotes topic diversity using spectral subtopic clustering\nunder the Word-Movers-Distance metric. We evaluate SWR on the DUC-02 and\nSummBank datasets and show that SWR produces better summaries than the\nstate-of-the-art algorithms over DUC-02 under common ROUGE measures. We then\nshow that, under the same measures over SummBank, SWR outperforms each of the\nthree human annotators (aka. judges) and compares favorably with the combined\nperformance of all judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04698v2",
            "date": "2018-09-12 22:41:47+00:00",
            "title": "Learning to Summarize Radiology Findings",
            "authors": [
                "Yuhao Zhang",
                "Daisy Yi Ding",
                "Tianpei Qian",
                "Christopher D. Manning",
                "Curtis P. Langlotz"
            ],
            "abstract": "The Impression section of a radiology report summarizes crucial radiology\nfindings in natural language and plays a central role in communicating these\nfindings to physicians. However, the process of generating impressions by\nsummarizing findings is time-consuming for radiologists and prone to errors. We\npropose to automate the generation of radiology impressions with neural\nsequence-to-sequence learning. We further propose a customized neural model for\nthis task which learns to encode the study background information and use this\ninformation to guide the decoding process. On a large dataset of radiology\nreports collected from actual hospital studies, our model outperforms existing\nnon-neural and neural baselines under the ROUGE metrics. In a blind experiment,\na board-certified radiologist indicated that 67% of sampled system summaries\nare at least as good as the corresponding human-written summaries, suggesting\nsignificant clinical validity. To our knowledge our work represents the first\nattempt in this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04838v1",
            "date": "2018-09-13 08:53:14+00:00",
            "title": "T\u00fcbingen-Oslo system: Linear regression works the best at Predicting Current and Future Psychological Health from Childhood Essays in the CLPsych 2018 Shared Task",
            "authors": [
                "\u00c7a\u011fr\u0131 \u00c7\u00f6ltekin",
                "Taraka Rama"
            ],
            "abstract": "This paper describes our efforts in predicting current and future\npsychological health from childhood essays within the scope of the CLPsych-2018\nShared Task. We experimented with a number of different models, including\nrecurrent and convolutional networks, Poisson regression, support vector\nregression, and L1 and L2 regularized linear regression. We obtained the best\nresults on the training/development data with L2 regularized linear regression\n(ridge regression) which also got the best scores on main metrics in the\nofficial testing for task A (predicting psychological health from essays\nwritten at the age of 11 years) and task B (predicting later psychological\nhealth from essays written at the age of 11).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04938v2",
            "date": "2018-09-13 13:27:52+00:00",
            "title": "LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts",
            "authors": [
                "Shuming Ma",
                "Lei Cui",
                "Damai Dai",
                "Furu Wei",
                "Xu Sun"
            ],
            "abstract": "We introduce the task of automatic live commenting. Live commenting, which is\nalso called `video barrage', is an emerging feature on online video sites that\nallows real-time comments from viewers to fly across the screen like bullets or\nroll at the right side of the screen. The live comments are a mixture of\nopinions for the video and the chit chats with other comments. Automatic live\ncommenting requires AI agents to comprehend the videos and interact with human\nviewers who also make the comments, so it is a good testbed of an AI agent's\nability of dealing with both dynamic vision and language. In this work, we\nconstruct a large-scale live comment dataset with 2,361 videos and 895,929 live\ncomments. Then, we introduce two neural models to generate live comments based\non the visual and textual contexts, which achieve better performance than\nprevious neural baselines such as the sequence-to-sequence model. Finally, we\nprovide a retrieval-based evaluation protocol for automatic live commenting\nwhere the model is asked to sort a set of candidate comments based on the\nlog-likelihood score, and evaluated on metrics such as mean-reciprocal-rank.\nPutting it all together, we demonstrate the first `LiveBot'.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04960v1",
            "date": "2018-09-13 13:48:42+00:00",
            "title": "Unsupervised Machine Commenting with Neural Variational Topic Model",
            "authors": [
                "Shuming Ma",
                "Lei Cui",
                "Furu Wei",
                "Xu Sun"
            ],
            "abstract": "Article comments can provide supplementary opinions and facts for readers,\nthereby increase the attraction and engagement of articles. Therefore,\nautomatically commenting is helpful in improving the activeness of the\ncommunity, such as online forums and news websites. Previous work shows that\ntraining an automatic commenting system requires large parallel corpora.\nAlthough part of articles are naturally paired with the comments on some\nwebsites, most articles and comments are unpaired on the Internet. To fully\nexploit the unpaired data, we completely remove the need for parallel data and\npropose a novel unsupervised approach to train an automatic article commenting\nmodel, relying on nothing but unpaired articles and comments. Our model is\nbased on a retrieval-based commenting framework, which uses news to retrieve\ncomments based on the similarity of their topics. The topic representation is\nobtained from a neural variational topic model, which is trained in an\nunsupervised manner. We evaluate our model on a news comment dataset.\nExperiments show that our proposed topic-based approach significantly\noutperforms previous lexicon-based models. The model also profits from paired\ncorpora and achieves state-of-the-art performance under semi-supervised\nscenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05218v4",
            "date": "2018-09-14 01:42:21+00:00",
            "title": "Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation",
            "authors": [
                "Brian Thompson",
                "Huda Khayrallah",
                "Antonios Anastasopoulos",
                "Arya D. McCarthy",
                "Kevin Duh",
                "Rebecca Marvin",
                "Paul McNamee",
                "Jeremy Gwinnup",
                "Tim Anderson",
                "Philipp Koehn"
            ],
            "abstract": "To better understand the effectiveness of continued training, we analyze the\nmajor components of a neural machine translation system (the encoder, decoder,\nand each embedding space) and consider each component's contribution to, and\ncapacity for, domain adaptation. We find that freezing any single component\nduring continued training has minimal impact on performance, and that\nperformance is surprisingly good when a single component is adapted while\nholding the rest of the model fixed. We also find that continued training does\nnot move the model very far from the out-of-domain model, compared to a\nsensitivity analysis metric, suggesting that the out-of-domain model can\nprovide a good generic initialization for the new domain.",
            "journal ref": "Proceedings of the Third Conference on Machine Translation:\n  Research Papers (2018) 124-132",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05219v1",
            "date": "2018-09-14 01:48:46+00:00",
            "title": "Automatic Catchphrase Extraction from Legal Case Documents via Scoring using Deep Neural Networks",
            "authors": [
                "Vu Tran",
                "Minh Le Nguyen",
                "Ken Satoh"
            ],
            "abstract": "In this paper, we present a method of automatic catchphrase extracting from\nlegal case documents. We utilize deep neural networks for constructing scoring\nmodel of our extraction system. We achieve comparable performance with systems\nusing corpus-wide and citation information which we do not use in our system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05233v2",
            "date": "2018-09-14 02:52:02+00:00",
            "title": "Unsupervised Abstractive Sentence Summarization using Length Controlled Variational Autoencoder",
            "authors": [
                "Raphael Schumann"
            ],
            "abstract": "In this work we present an unsupervised approach to summarize sentences in\nabstractive way using Variational Autoencoder (VAE). VAE are known to learn a\nsemantically rich latent variable, representing high dimensional input. VAEs\nare trained by learning to reconstruct the input from the probabilistic latent\nvariable. Explicitly providing the information about output length during\ntraining influences the VAE to not encode this information and thus can be\nmanipulated during inference. Instructing the decoder to produce a shorter\noutput sequence leads to expressing the input sentence with fewer words. We\nshow on different summarization data sets, that these shorter sentences can not\nbeat a simple baseline but yield higher ROUGE scores than trying to reconstruct\nthe whole sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05268v2",
            "date": "2018-09-14 06:27:38+00:00",
            "title": "Supervised Machine Learning for Extractive Query Based Summarisation of Biomedical Data",
            "authors": [
                "Mandeep Kaur",
                "Diego Moll\u00e1"
            ],
            "abstract": "The automation of text summarisation of biomedical publications is a pressing\nneed due to the plethora of information available on-line. This paper explores\nthe impact of several supervised machine learning approaches for extracting\nmulti-document summaries for given queries. In particular, we compare\nclassification and regression approaches for query-based extractive\nsummarisation using data provided by the BioASQ Challenge. We tackled the\nproblem of annotating sentences for training classification systems and show\nthat a simple annotation approach outperforms regression-based summarisation.",
            "journal ref": "Proceedings of the Ninth International Workshop on Health Text\n  Mining and Information Analysis, Louhi 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05283v2",
            "date": "2018-09-14 07:26:31+00:00",
            "title": "Macquarie University at BioASQ 6b: Deep learning and deep reinforcement learning for query-based multi-document summarisation",
            "authors": [
                "Diego Moll\u00e1"
            ],
            "abstract": "This paper describes Macquarie University's contribution to the BioASQ\nChallenge (BioASQ 6b, Phase B). We focused on the extraction of the ideal\nanswers, and the task was approached as an instance of query-based\nmulti-document summarisation. In particular, this paper focuses on the\nexperiments related to the deep learning and reinforcement learning approaches\nused in the submitted runs. The best run used a deep learning model under a\nregression-based framework. The deep learning architecture used features\nderived from the output of LSTM chains on word embeddings, plus features based\non similarity with the query, and sentence position. The reinforcement learning\napproach was a proof-of-concept prototype that trained a global policy using\nREINFORCE. The global policy was implemented as a neural network that used\n$tf.idf$ features encoding the candidate sentence, question, and context.",
            "journal ref": "Proceedings of the 6th BioASQ Workshop A challenge on large-scale\n  biomedical semantic indexing and question answering, EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05288v1",
            "date": "2018-09-14 07:51:19+00:00",
            "title": "Characterizing Variation in Crowd-Sourced Data for Training Neural Language Generators to Produce Stylistically Varied Outputs",
            "authors": [
                "Juraj Juraska",
                "Marilyn Walker"
            ],
            "abstract": "One of the biggest challenges of end-to-end language generation from meaning\nrepresentations in dialogue systems is making the outputs more natural and\nvaried. Here we take a large corpus of 50K crowd-sourced utterances in the\nrestaurant domain and develop text analysis methods that systematically\ncharacterize types of sentences in the training data. We then automatically\nlabel the training data to allow us to conduct two kinds of experiments with a\nneural generator. First, we test the effect of training the system with\ndifferent stylistic partitions and quantify the effect of smaller, but more\nstylistically controlled training data. Second, we propose a method of labeling\nthe style variants during training, and show that we can modify the style of\nthe generated utterances using our stylistic labels. We contrast and compare\nthese methods that can be used with any existing large corpus, showing how they\nvary in terms of semantic quality and stylistic control.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05296v5",
            "date": "2018-09-14 08:07:54+00:00",
            "title": "Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory",
            "authors": [
                "Deng Cai",
                "Yan Wang",
                "Victoria Bi",
                "Zhaopeng Tu",
                "Xiaojiang Liu",
                "Wai Lam",
                "Shuming Shi"
            ],
            "abstract": "For dialogue response generation, traditional generative models generate\nresponses solely from input queries. Such models rely on insufficient\ninformation for generating a specific response since a certain query could be\nanswered in multiple ways. Consequentially, those models tend to output generic\nand dull responses, impeding the generation of informative utterances.\nRecently, researchers have attempted to fill the information gap by exploiting\ninformation retrieval techniques. When generating a response for a current\nquery, similar dialogues retrieved from the entire training data are considered\nas an additional knowledge source. While this may harvest massive information,\nthe generative models could be overwhelmed, leading to undesirable performance.\nIn this paper, we propose a new framework which exploits retrieval results via\na skeleton-then-response paradigm. At first, a skeleton is generated by\nrevising the retrieved responses. Then, a novel generative model uses both the\ngenerated skeleton and the original query for response generation. Experimental\nresults show that our approaches significantly improve the diversity and\ninformativeness of the generated responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05356v2",
            "date": "2018-09-14 11:11:37+00:00",
            "title": "Numeral Understanding in Financial Tweets for Fine-grained Crowd-based Forecasting",
            "authors": [
                "Chung-Chi Chen",
                "Hen-Hsen Huang",
                "Yow-Ting Shiue",
                "Hsin-Hsi Chen"
            ],
            "abstract": "Numerals that contain much information in financial documents are crucial for\nfinancial decision making. They play different roles in financial analysis\nprocesses. This paper is aimed at understanding the meanings of numerals in\nfinancial tweets for fine-grained crowd-based forecasting. We propose a\ntaxonomy that classifies the numerals in financial tweets into 7 categories,\nand further extend some of these categories into several subcategories. Neural\nnetwork-based models with word and character-level encoders are proposed for\n7-way classification and 17-way classification. We perform backtest to confirm\nthe effectiveness of the numeric opinions made by the crowd. This work is the\nfirst attempt to understand numerals in financial social media data, and we\nprovide the first comparison of fine-grained opinion of individual investors\nand analysts based on their forecast price. The numeral corpus used in our\nexperiments, called FinNum 1.0 , is available for research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05501v1",
            "date": "2018-09-14 16:52:12+00:00",
            "title": "Ground Truth for training OCR engines on historical documents in German Fraktur and Early Modern Latin",
            "authors": [
                "Uwe Springmann",
                "Christian Reul",
                "Stefanie Dipper",
                "Johannes Baiter"
            ],
            "abstract": "In this paper we describe a dataset of German and Latin \\textit{ground truth}\n(GT) for historical OCR in the form of printed text line images paired with\ntheir transcription. This dataset, called \\textit{GT4HistOCR}, consists of\n313,173 line pairs covering a wide period of printing dates from incunabula\nfrom the 15th century to 19th century books printed in Fraktur types and is\nopenly available under a CC-BY 4.0 license. The special form of GT as line\nimage/transcription pairs makes it directly usable to train state-of-the-art\nrecognition models for OCR software employing recurring neural networks in LSTM\narchitecture such as Tesseract 4 or OCRopus. We also provide some pretrained\nOCRopus models for subcorpora of our dataset yielding between 95\\% (early\nprintings) and 98\\% (19th century Fraktur printings) character accuracy rates\non unseen test cases, a Perl script to harmonize GT produced by different\ntranscription rules, and give hints on how to construct GT for OCR purposes\nwhich has requirements that may differ from linguistically motivated\ntranscriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05576v2",
            "date": "2018-09-14 20:37:38+00:00",
            "title": "Events Beyond ACE: Curated Training for Events",
            "authors": [
                "Ryan Gabbard",
                "Jay DeYoung",
                "Marjorie Freedman"
            ],
            "abstract": "We explore a human-driven approach to annotation, curated training (CT), in\nwhich annotation is framed as teaching the system by using interactive search\nto identify informative snippets of text to annotate, unlike traditional\napproaches which either annotate preselected text or use active learning. A\ntrained annotator performed 80 hours of CT for the thirty event types of the\nNIST TAC KBP Event Argument Extraction evaluation. Combining this annotation\nwith ACE results in a 6% reduction in error and the learning curve of CT\nplateaus more slowly than for full-document annotation. 3 NLP researchers\nperformed CT for one event type and showed much sharper learning curves with\nall three exceeding ACE performance in less than ninety minutes, suggesting\nthat CT can provide further benefits when the annotator deeply understands the\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05694v2",
            "date": "2018-09-15 10:42:41+00:00",
            "title": "CLUSE: Cross-Lingual Unsupervised Sense Embeddings",
            "authors": [
                "Ta-Chung Chi",
                "Yun-Nung Chen"
            ],
            "abstract": "This paper proposes a modularized sense induction and representation learning\nmodel that jointly learns bilingual sense embeddings that align well in the\nvector space, where the cross-lingual signal in the English-Chinese parallel\ncorpus is exploited to capture the collocation and distributed characteristics\nin the language pair. The model is evaluated on the Stanford Contextual Word\nSimilarity (SCWS) dataset to ensure the quality of monolingual sense\nembeddings. In addition, we introduce Bilingual Contextual Word Similarity\n(BCWS), a large and high-quality dataset for evaluating cross-lingual sense\nembeddings, which is the first attempt of measuring whether the learned\nembeddings are indeed aligned well in the vector space. The proposed approach\nshows the superior quality of sense embeddings evaluated in both monolingual\nand bilingual spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05715v2",
            "date": "2018-09-15 13:30:03+00:00",
            "title": "Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts",
            "authors": [
                "Chih-Wen Goo",
                "Yun-Nung Chen"
            ],
            "abstract": "Neural abstractive summarization has been increasingly studied, where the\nprior work mainly focused on summarizing single-speaker documents (news,\nscientific publications, etc). In dialogues, there are different interactions\nbetween speakers, which are usually defined as dialogue acts. The interactive\nsignals may provide informative cues for better summarizing dialogues. This\npaper proposes to explicitly leverage dialogue acts in a neural summarization\nmodel, where a sentence-gated mechanism is designed for modeling the\nrelationship between dialogue acts and the summary. The experiments show that\nour proposed model significantly improves the abstractive summarization\nperformance compared to the state-of-the-art baselines on AMI meeting corpus,\ndemonstrating the usefulness of the interactive signal provided by dialogue\nacts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05733v1",
            "date": "2018-09-15 15:44:54+00:00",
            "title": "Neural Networks and Quantifier Conservativity: Does Data Distribution Affect Learnability?",
            "authors": [
                "Vishwali Mhasawade",
                "Ildik\u00f3 Emese Szab\u00f3",
                "Melanie Tosik",
                "Sheng-Fu Wang"
            ],
            "abstract": "All known natural language determiners are conservative. Psycholinguistic\nexperiments indicate that children exhibit a corresponding learnability bias\nwhen faced with the task of learning new determiners. However, recent work\nindicates that this bias towards conservativity is not observed during the\ntraining stage of artificial neural networks. In this work, we investigate\nwhether the learnability bias exhibited by children is in part due to the\ndistribution of quantifiers in natural language. We share results of five\nexperiments, contrasted by the distribution of conservative vs.\nnon-conservative determiners in the training data. We demonstrate that the\naquisitional issues with non-conservative quantifiers can not be explained by\nthe distribution of natural language data, which favors conservative\nquantifiers. This finding indicates that the bias in language acquisition data\nmight be innate or representational.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05742v1",
            "date": "2018-09-15 16:42:24+00:00",
            "title": "Finding the way from \u00e4 to a: Sub-character morphological inflection for the SIGMORPHON 2018 Shared Task",
            "authors": [
                "Fynn Schr\u00f6der",
                "Marcel Kamlot",
                "Gregor Billing",
                "Arne K\u00f6hn"
            ],
            "abstract": "In this paper we describe the system submitted by UHH to the\nCoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection. We\npropose a neural architecture based on the concepts of UZH (Makarov et al.,\n2017), adding new ideas and techniques to their key concept and evaluating\ndifferent combinations of parameters. The resulting system is a\nlanguage-agnostic network model that aims to reduce the number of learned edit\noperations by introducing equivalence classes over graphical features of\nindividual characters. We try to pinpoint advantages and drawbacks of this\napproach by comparing different network configurations and evaluating our\nresults over a wide range of languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05752v1",
            "date": "2018-09-15 18:27:59+00:00",
            "title": "Analysis of Risk Factor Domains in Psychosis Patient Health Records",
            "authors": [
                "Eben Holderness",
                "Nicholas Miller",
                "Philip Cawkwell",
                "Kirsten Bolton",
                "James Pustejovsky",
                "Marie Meteer",
                "Mei-Hua Hall"
            ],
            "abstract": "Readmission after discharge from a hospital is disruptive and costly,\nregardless of the reason. However, it can be particularly problematic for\npsychiatric patients, so predicting which patients may be readmitted is\ncritically important but also very difficult. Clinical narratives in\npsychiatric electronic health records (EHRs) span a wide range of topics and\nvocabulary; therefore, a psychiatric readmission prediction model must begin\nwith a robust and interpretable topic extraction component. We created a data\npipeline for using document vector similarity metrics to perform topic\nextraction on psychiatric EHR data in service of our long-term goal of creating\na readmission risk classifier. We show initial results for our topic extraction\nmodel and identify additional features we will be incorporating in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05820v1",
            "date": "2018-09-16 06:02:37+00:00",
            "title": "Cross-Domain Labeled LDA for Cross-Domain Text Classification",
            "authors": [
                "Baoyu Jing",
                "Chenwei Lu",
                "Deqing Wang",
                "Fuzhen Zhuang",
                "Cheng Niu"
            ],
            "abstract": "Cross-domain text classification aims at building a classifier for a target\ndomain which leverages data from both source and target domain. One promising\nidea is to minimize the feature distribution differences of the two domains.\nMost existing studies explicitly minimize such differences by an exact\nalignment mechanism (aligning features by one-to-one feature alignment,\nprojection matrix etc.). Such exact alignment, however, will restrict models'\nlearning ability and will further impair models' performance on classification\ntasks when the semantic distributions of different domains are very different.\nTo address this problem, we propose a novel group alignment which aligns the\nsemantics at group level. In addition, to help the model learn better semantic\ngroups and semantics within these groups, we also propose a partial supervision\nfor model's learning in source domain. To this end, we embed the group\nalignment and a partial supervision into a cross-domain topic model, and\npropose a Cross-Domain Labeled LDA (CDL-LDA). On the standard 20Newsgroup and\nReuters dataset, extensive quantitative (classification, perplexity etc.) and\nqualitative (topic detection) experiments are conducted to show the\neffectiveness of the proposed group alignment and partial supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06083v1",
            "date": "2018-09-17 09:09:48+00:00",
            "title": "Similarity measure for Public Persons",
            "authors": [
                "Andreas St\u00f6ckl"
            ],
            "abstract": "For the webportal \"Who is in the News!\" with statistics about the appearence\nof persons in written news we developed an extension, which measures the\nrelationship of public persons depending on a time parameter, as the\nrelationship may vary over time. On a training corpus of English and German\nnews articles we built a measure by extracting the persons occurrence in the\ntext via pretrained named entity extraction and then construct time series of\ncounts for each person. Pearson correlation over a sliding window is then used\nto measure the relation of two persons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06142v1",
            "date": "2018-09-17 11:49:19+00:00",
            "title": "Open Subtitles Paraphrase Corpus for Six Languages",
            "authors": [
                "Mathias Creutz"
            ],
            "abstract": "This paper accompanies the release of Opusparcus, a new paraphrase corpus for\nsix European languages: German, English, Finnish, French, Russian, and Swedish.\nThe corpus consists of paraphrases, that is, pairs of sentences in the same\nlanguage that mean approximately the same thing. The paraphrases are extracted\nfrom the OpenSubtitles2016 corpus, which contains subtitles from movies and TV\nshows. The informal and colloquial genre that occurs in subtitles makes such\ndata a very interesting language resource, for instance, from the perspective\nof computer assisted language learning. For each target language, the\nOpusparcus data have been partitioned into three types of data sets: training,\ndevelopment and test sets. The training sets are large, consisting of millions\nof sentence pairs, and have been compiled automatically, with the help of\nprobabilistic ranking functions. The development and test sets consist of\nsentence pairs that have been checked manually; each set contains approximately\n1000 sentence pairs that have been verified to be acceptable paraphrases by two\nannotators.",
            "journal ref": "Proceedings of the Eleventh International Conference on Language\n  Resources and Evaluation (LREC 2018), pp. 1364-1369, Miyazaki, Japan, 10 May\n  2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06152v2",
            "date": "2018-09-17 12:04:24+00:00",
            "title": "Categorizing Comparative Sentences",
            "authors": [
                "Alexander Panchenko",
                "Alexander Bondarenko",
                "Mirco Franzek",
                "Matthias Hagen",
                "Chris Biemann"
            ],
            "abstract": "We tackle the tasks of automatically identifying comparative sentences and\ncategorizing the intended preference (e.g., \"Python has better NLP libraries\nthan MATLAB\" => (Python, better, MATLAB). To this end, we manually annotate\n7,199 sentences for 217 distinct target item pairs from several domains (27% of\nthe sentences contain an oriented comparison in the sense of \"better\" or\n\"worse\"). A gradient boosting model based on pre-trained sentence embeddings\nreaches an F1 score of 85% in our experimental evaluation. The model can be\nused to extract comparative sentences for pro/con argumentation in comparative\n/ argument search engines or debating technologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06194v2",
            "date": "2018-09-17 13:34:49+00:00",
            "title": "The Fast and the Flexible: training neural networks to learn to follow instructions from small data",
            "authors": [
                "Rezka Leonandya",
                "Elia Bruni",
                "Dieuwke Hupkes",
                "Germ\u00e1n Kruszewski"
            ],
            "abstract": "Learning to follow human instructions is a long-pursued goal in artificial\nintelligence. The task becomes particularly challenging if no prior knowledge\nof the employed language is assumed while relying only on a handful of examples\nto learn from. Work in the past has relied on hand-coded components or manually\nengineered features to provide strong inductive biases that make learning in\nsuch situations possible. In contrast, here we seek to establish whether this\nknowledge can be acquired automatically by a neural network system through a\ntwo phase training procedure: A (slow) offline learning stage where the network\nlearns about the general structure of the task and a (fast) online adaptation\nphase where the network learns the language of a new given speaker. Controlled\nexperiments show that when the network is exposed to familiar instructions but\ncontaining novel words, the model adapts very efficiently to the new\nvocabulary. Moreover, even for human speakers whose language usage can depart\nsignificantly from our artificial training language, our network can still make\nuse of its automatically acquired inductive bias to learn to follow\ninstructions more effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06223v1",
            "date": "2018-09-17 14:16:49+00:00",
            "title": "Unsupervised Sense-Aware Hypernymy Extraction",
            "authors": [
                "Dmitry Ustalov",
                "Alexander Panchenko",
                "Chris Biemann",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "In this paper, we show how unsupervised sense representations can be used to\nimprove hypernymy extraction. We present a method for extracting disambiguated\nhypernymy relationships that propagates hypernyms to sets of synonyms\n(synsets), constructs embeddings for these sets, and establishes sense-aware\nrelationships between matching synsets. Evaluation on two gold standard\ndatasets for English and Russian shows that the method successfully recognizes\nhypernymy relationships that cannot be found with standard Hearst patterns and\nWiktionary datasets for the respective languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06284v1",
            "date": "2018-09-17 15:47:06+00:00",
            "title": "Style Transfer Through Multilingual and Feedback-Based Back-Translation",
            "authors": [
                "Shrimai Prabhumoye",
                "Yulia Tsvetkov",
                "Alan W Black",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Style transfer is the task of transferring an attribute of a sentence (e.g.,\nformality) while maintaining its semantic content. The key challenge in style\ntransfer is to strike a balance between the competing goals, one to preserve\nmeaning and the other to improve the style transfer accuracy. Prior research\nhas identified that the task of meaning preservation is generally harder to\nattain and evaluate. This paper proposes two extensions of the state-of-the-art\nstyle transfer models aiming at improving the meaning preservation in style\ntransfer. Our evaluation shows that these extensions help to ground meaning\nbetter while improving the transfer accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06297v2",
            "date": "2018-09-17 16:03:13+00:00",
            "title": "Adversarial Text Generation via Feature-Mover's Distance",
            "authors": [
                "Liqun Chen",
                "Shuyang Dai",
                "Chenyang Tao",
                "Dinghan Shen",
                "Zhe Gan",
                "Haichao Zhang",
                "Yizhe Zhang",
                "Lawrence Carin"
            ],
            "abstract": "Generative adversarial networks (GANs) have achieved significant success in\ngenerating real-valued data. However, the discrete nature of text hinders the\napplication of GAN to text-generation tasks. Instead of using the standard GAN\nobjective, we propose to improve text-generation GAN via a novel approach\ninspired by optimal transport. Specifically, we consider matching the latent\nfeature distributions of real and synthetic sentences using a novel metric,\ntermed the feature-mover's distance (FMD). This formulation leads to a highly\ndiscriminative critic and easy-to-optimize objective, overcoming the\nmode-collapsing and brittle-training problems in existing methods. Extensive\nexperiments are conducted on a variety of tasks to evaluate the proposed model\nempirically, including unconditional text generation, style transfer from\nnon-parallel text, and unsupervised cipher cracking. The proposed model yields\nsuperior performance, demonstrating wide applicability and effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06444v1",
            "date": "2018-09-17 21:01:35+00:00",
            "title": "Robust Spoken Language Understanding via Paraphrasing",
            "authors": [
                "Avik Ray",
                "Yilin Shen",
                "Hongxia Jin"
            ],
            "abstract": "Learning intents and slot labels from user utterances is a fundamental step\nin all spoken language understanding (SLU) and dialog systems. State-of-the-art\nneural network based methods, after deployment, often suffer from performance\ndegradation on encountering paraphrased utterances, and out-of-vocabulary\nwords, rarely observed in their training set. We address this challenging\nproblem by introducing a novel paraphrasing based SLU model which can be\nintegrated with any existing SLU model in order to improve their overall\nperformance. We propose two new paraphrase generators using RNN and\nsequence-to-sequence based neural networks, which are suitable for our\napplication. Our experiments on existing benchmark and in house datasets\ndemonstrate the robustness of our models to rare and complex paraphrased\nutterances, even under adversarial test distributions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06502v1",
            "date": "2018-09-18 02:09:56+00:00",
            "title": "Analysis of Bag-of-n-grams Representation's Properties Based on Textual Reconstruction",
            "authors": [
                "Qi Huang",
                "Zhanghao Chen",
                "Zijie Lu",
                "Yuan Ye"
            ],
            "abstract": "Despite its simplicity, bag-of-n-grams sen- tence representation has been\nfound to excel in some NLP tasks. However, it has not re- ceived much attention\nin recent years and fur- ther analysis on its properties is necessary. We\npropose a framework to investigate the amount and type of information captured\nin a general- purposed bag-of-n-grams sentence represen- tation. We first use\nsentence reconstruction as a tool to obtain bag-of-n-grams representa- tion\nthat contains general information of the sentence. We then run prediction tasks\n(sen- tence length, word content, phrase content and word order) using the\nobtained representation to look into the specific type of information captured\nin the representation. Our analysis demonstrates that bag-of-n-grams\nrepresenta- tion does contain sentence structure level in- formation. However,\nincorporating n-grams with higher order n empirically helps little with\nencoding more information in general, except for phrase content information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06590v1",
            "date": "2018-09-18 08:34:12+00:00",
            "title": "Learning Universal Sentence Representations with Mean-Max Attention Autoencoder",
            "authors": [
                "Minghua Zhang",
                "Yunfang Wu",
                "Weikang Li",
                "Wei Li"
            ],
            "abstract": "In order to learn universal sentence representations, previous methods focus\non complex recurrent neural networks or supervised learning. In this paper, we\npropose a mean-max attention autoencoder (mean-max AAE) within the\nencoder-decoder framework. Our autoencoder rely entirely on the MultiHead\nself-attention mechanism to reconstruct the input sequence. In the encoding we\npropose a mean-max strategy that applies both mean and max pooling operations\nover the hidden vectors to capture diverse information of the input. To enable\nthe information to steer the reconstruction process dynamically, the decoder\nperforms attention over the mean-max representation. By training our model on a\nlarge collection of unlabelled data, we obtain high-quality representations of\nsentences. Experimental results on a broad range of 10 transfer tasks\ndemonstrate that our model outperforms the state-of-the-art unsupervised single\nmethods, including the classical skip-thoughts and the advanced\nskip-thoughts+LN model. Furthermore, compared with the traditional recurrent\nneural network, our mean-max AAE greatly reduce the training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06662v1",
            "date": "2018-09-18 12:18:22+00:00",
            "title": "Bidirectional Attentional Encoder-Decoder Model and Bidirectional Beam Search for Abstractive Summarization",
            "authors": [
                "Kamal Al-Sabahi",
                "Zhang Zuping",
                "Yang Kang"
            ],
            "abstract": "Sequence generative models with RNN variants, such as LSTM, GRU, show\npromising performance on abstractive document summarization. However, they\nstill have some issues that limit their performance, especially while deal-ing\nwith long sequences. One of the issues is that, to the best of our knowledge,\nall current models employ a unidirectional decoder, which reasons only about\nthe past and still limited to retain future context while giving a prediction.\nThis makes these models suffer on their own by generating unbalanced outputs.\nMoreover, unidirec-tional attention-based document summarization can only\ncapture partial aspects of attentional regularities due to the inherited\nchallenges in document summarization. To this end, we propose an end-to-end\ntrainable bidirectional RNN model to tackle the aforementioned issues. The\nmodel has a bidirectional encoder-decoder architecture; in which the encoder\nand the decoder are bidirectional LSTMs. The forward decoder is initialized\nwith the last hidden state of the backward encoder while the backward decoder\nis initialized with the last hidden state of the for-ward encoder. In addition,\na bidirectional beam search mechanism is proposed as an approximate inference\nalgo-rithm for generating the output summaries from the bidi-rectional model.\nThis enables the model to reason about the past and future and to generate\nbalanced outputs as a result. Experimental results on CNN / Daily Mail dataset\nshow that the proposed model outperforms the current abstractive\nstate-of-the-art models by a considerable mar-gin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06683v1",
            "date": "2018-09-18 13:12:21+00:00",
            "title": "RumourEval 2019: Determining Rumour Veracity and Support for Rumours",
            "authors": [
                "Genevieve Gorrell",
                "Kalina Bontcheva",
                "Leon Derczynski",
                "Elena Kochkina",
                "Maria Liakata",
                "Arkaitz Zubiaga"
            ],
            "abstract": "This is the proposal for RumourEval-2019, which will run in early 2019 as\npart of that year's SemEval event. Since the first RumourEval shared task in\n2017, interest in automated claim validation has greatly increased, as the\ndangers of \"fake news\" have become a mainstream concern. Yet automated support\nfor rumour checking remains in its infancy. For this reason, it is important\nthat a shared task in this area continues to provide a focus for effort, which\nis likely to increase. We therefore propose a continuation in which the\nveracity of further rumours is determined, and as previously, supportive of\nthis goal, tweets discussing them are classified according to the stance they\ntake regarding the rumour. Scope is extended compared with the first\nRumourEval, in that the dataset is substantially expanded to include Reddit as\nwell as Twitter data, and additional languages are also included.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06748v1",
            "date": "2018-09-18 14:01:22+00:00",
            "title": "Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation",
            "authors": [
                "Murhaf Fares",
                "Stephan Oepen",
                "Erik Velldal"
            ],
            "abstract": "In this paper, we empirically evaluate the utility of transfer and multi-task\nlearning on a challenging semantic classification task: semantic interpretation\nof noun--noun compounds. Through a comprehensive series of experiments and\nin-depth error analysis, we show that transfer learning via parameter\ninitialization and multi-task learning via parameter sharing can help a neural\nclassification model generalize over a highly skewed distribution of relations.\nFurther, we demonstrate how dual annotation with two distinct sets of relations\nover the same set of compounds can be exploited to improve the overall accuracy\nof a neural classifier and its F1 scores on the less frequent, but more\ndifficult relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06873v1",
            "date": "2018-09-18 18:08:19+00:00",
            "title": "Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity",
            "authors": [
                "Xinnuo Xu",
                "Ond\u0159ej Du\u0161ek",
                "Ioannis Konstas",
                "Verena Rieser"
            ],
            "abstract": "We present three enhancements to existing encoder-decoder models for\nopen-domain conversational agents, aimed at effectively modeling coherence and\npromoting output diversity: (1) We introduce a measure of coherence as the\nGloVe embedding similarity between the dialogue context and the generated\nresponse, (2) we filter our training corpora based on the measure of coherence\nto obtain topically coherent and lexically diverse context-response pairs, (3)\nwe then train a response generator using a conditional variational autoencoder\nmodel that incorporates the measure of coherence as a latent variable and uses\na context gate to guarantee topical consistency with the context and promote\nlexical diversity. Experiments on the OpenSubtitles corpus show a substantial\nimprovement over competitive neural models in terms of BLEU score as well as\nmetrics of coherence and diversity.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing, pages 3981-3991, Brussels, Belgium, November 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06906v1",
            "date": "2018-09-18 19:45:25+00:00",
            "title": "Improving Moderation of Online Discussions via Interpretable Neural Models",
            "authors": [
                "Andrej \u0160vec",
                "Mat\u00fa\u0161 Pikuliak",
                "Mari\u00e1n \u0160imko",
                "M\u00e1ria Bielikov\u00e1"
            ],
            "abstract": "Growing amount of comments make online discussions difficult to moderate by\nhuman moderators only. Antisocial behavior is a common occurrence that often\ndiscourages other users from participating in discussion. We propose a neural\nnetwork based method that partially automates the moderation process. It\nconsists of two steps. First, we detect inappropriate comments for moderators\nto see. Second, we highlight inappropriate parts within these comments to make\nthe moderation faster. We evaluated our method on data from a major Slovak news\ndiscussion platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07037v2",
            "date": "2018-09-19 07:41:55+00:00",
            "title": "NICT's Neural and Statistical Machine Translation Systems for the WMT18 News Translation Task",
            "authors": [
                "Benjamin Marie",
                "Rui Wang",
                "Atsushi Fujita",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "This paper presents the NICT's participation to the WMT18 shared news\ntranslation task. We participated in the eight translation directions of four\nlanguage pairs: Estonian-English, Finnish-English, Turkish-English and\nChinese-English. For each translation direction, we prepared state-of-the-art\nstatistical (SMT) and neural (NMT) machine translation systems. Our NMT systems\nwere trained with the transformer architecture using the provided parallel data\nenlarged with a large quantity of back-translated monolingual data that we\ngenerated with a new incremental training framework. Our primary submissions to\nthe task are the result of a simple combination of our SMT and NMT systems. Our\nsystems are ranked first for the Estonian-English and Finnish-English language\npairs (constraint) according to BLEU-cased.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07043v2",
            "date": "2018-09-19 07:52:16+00:00",
            "title": "NICT's Corpus Filtering Systems for the WMT18 Parallel Corpus Filtering Task",
            "authors": [
                "Rui Wang",
                "Benjamin Marie",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "This paper presents the NICT's participation in the WMT18 shared parallel\ncorpus filtering task. The organizers provided 1 billion words German-English\ncorpus crawled from the web as part of the Paracrawl project. This corpus is\ntoo noisy to build an acceptable neural machine translation (NMT) system. Using\nthe clean data of the WMT18 shared news translation task, we designed several\nfeatures and trained a classifier to score each sentence pairs in the noisy\ndata. Finally, we sampled 100 million and 10 million words and built\ncorresponding NMT systems. Empirical results show that our NMT systems trained\non sampled data achieve promising performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07070v1",
            "date": "2018-09-19 08:58:23+00:00",
            "title": "Latent Topic Conversational Models",
            "authors": [
                "Tsung-Hsien Wen",
                "Minh-Thang Luong"
            ],
            "abstract": "Latent variable models have been a preferred choice in conversational\nmodeling compared to sequence-to-sequence (seq2seq) models which tend to\ngenerate generic and repetitive responses. Despite so, training latent variable\nmodels remains to be difficult. In this paper, we propose Latent Topic\nConversational Model (LTCM) which augments seq2seq with a neural latent topic\ncomponent to better guide response generation and make training easier. The\nneural topic component encodes information from the source sentence to build a\nglobal \"topic\" distribution over words, which is then consulted by the seq2seq\nmodel at each generation step. We study in details how the latent\nrepresentation is learnt in both the vanilla model and LTCM. Our extensive\nexperiments contribute to better understanding and training of conditional\nlatent models for languages. Our results show that by sampling from the learnt\nlatent representations, LTCM can generate diverse and interesting responses. In\na subjective human evaluation, the judges also confirm that LTCM is the overall\npreferred option.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07182v1",
            "date": "2018-09-19 13:39:40+00:00",
            "title": "String Transduction with Target Language Models and Insertion Handling",
            "authors": [
                "Garrett Nicolai",
                "Saeed Najafi",
                "Grzegorz Kondrak"
            ],
            "abstract": "Many character-level tasks can be framed as sequence-to-sequence\ntransduction, where the target is a word from a natural language. We show that\nleveraging target language models derived from unannotated target corpora,\ncombined with a precise alignment of the training data, yields state-of-the art\nresults on cognate projection, inflection generation, and phoneme-to-grapheme\nconversion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07234v1",
            "date": "2018-09-19 15:07:45+00:00",
            "title": "Unsupervised cross-lingual matching of product classifications",
            "authors": [
                "Denis Gordeev",
                "Alexey Rey",
                "Dmitry Shagarov"
            ],
            "abstract": "Unsupervised cross-lingual embeddings mapping has provided a unique tool for\ncompletely unsupervised translation even for languages with different scripts.\nIn this work we use this method for the task of unsupervised cross-lingual\nmatching of product classifications. Our work also investigates limitations of\nunsupervised vector alignment and we also suggest two other techniques for\naligning product classifications based on their descriptions: using\nhierarchical information and translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07291v1",
            "date": "2018-09-19 16:32:47+00:00",
            "title": "Interpretable Textual Neuron Representations for NLP",
            "authors": [
                "Nina Poerner",
                "Benjamin Roth",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Input optimization methods, such as Google Deep Dream, create interpretable\nrepresentations of neurons for computer vision DNNs. We propose and evaluate\nways of transferring this technology to NLP. Our results suggest that gradient\nascent with a gumbel softmax layer produces n-gram representations that\noutperform naive corpus search in terms of target neuron activation. The\nrepresentations highlight differences in syntax awareness between the language\nand visual models of the Imaginet architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07358v1",
            "date": "2018-09-19 18:22:44+00:00",
            "title": "A Dataset for Document Grounded Conversations",
            "authors": [
                "Kangyan Zhou",
                "Shrimai Prabhumoye",
                "Alan W Black"
            ],
            "abstract": "This paper introduces a document grounded dataset for text conversations. We\ndefine \"Document Grounded Conversations\" as conversations that are about the\ncontents of a specified document. In this dataset the specified documents were\nWikipedia articles about popular movies. The dataset contains 4112\nconversations with an average of 21.43 turns per conversation. This positions\nthis dataset to not only provide a relevant chat history while generating\nresponses but also provide a source of information that the models could use.\nWe describe two neural architectures that provide benchmark performance on the\ntask of generating the next response. We also evaluate our models for\nengagement and fluency, and find that the information from the document helps\nin generating more engaging and fluent responses.",
            "journal ref": "Conference on Empirical Methods in Natural Language Processing\n  (EMNLP), 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07483v1",
            "date": "2018-09-20 05:40:46+00:00",
            "title": "Building Context-aware Clause Representations for Situation Entity Type Classification",
            "authors": [
                "Zeyu Dai",
                "Ruihong Huang"
            ],
            "abstract": "Capabilities to categorize a clause based on the type of situation entity\n(e.g., events, states and generic statements) the clause introduces to the\ndiscourse can benefit many NLP applications. Observing that the situation\nentity type of a clause depends on discourse functions the clause plays in a\nparagraph and the interpretation of discourse functions depends heavily on\nparagraph-wide contexts, we propose to build context-aware clause\nrepresentations for predicting situation entity types of clauses. Specifically,\nwe propose a hierarchical recurrent neural network model to read a whole\nparagraph at a time and jointly learn representations for all the clauses in\nthe paragraph by extensively modeling context influences and inter-dependencies\nof clauses. Experimental results show that our model achieves the\nstate-of-the-art performance for clause-level situation entity classification\non the genre-rich MASC+Wiki corpus, which approaches human-level performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07485v1",
            "date": "2018-09-20 05:49:40+00:00",
            "title": "A Quantitative Evaluation of Natural Language Question Interpretation for Question Answering Systems",
            "authors": [
                "Takuto Asakura",
                "Jin-Dong Kim",
                "Yasunori Yamamoto",
                "Yuka Tateisi",
                "Toshihisa Takagi"
            ],
            "abstract": "Systematic benchmark evaluation plays an important role in the process of\nimproving technologies for Question Answering (QA) systems. While currently\nthere are a number of existing evaluation methods for natural language (NL) QA\nsystems, most of them consider only the final answers, limiting their utility\nwithin a black box style evaluation. Herein, we propose a subdivided evaluation\napproach to enable finer-grained evaluation of QA systems, and present an\nevaluation tool which targets the NL question (NLQ) interpretation step, an\ninitial step of a QA pipeline. The results of experiments using two public\nbenchmark datasets suggest that we can get a deeper insight about the\nperformance of a QA system using the proposed approach, which should provide a\nbetter guidance for improving the systems, than using black box style\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07572v1",
            "date": "2018-09-20 11:11:42+00:00",
            "title": "Challenges for Toxic Comment Classification: An In-Depth Error Analysis",
            "authors": [
                "Betty van Aken",
                "Julian Risch",
                "Ralf Krestel",
                "Alexander L\u00f6ser"
            ],
            "abstract": "Toxic comment classification has become an active research field with many\nrecently proposed approaches. However, while these approaches address some of\nthe task's challenges others still remain unsolved and directions for further\nresearch are needed. To this end, we compare different deep learning and\nshallow approaches on a new, large comment dataset and propose an ensemble that\noutperforms all individual models. Further, we validate our findings on a\nsecond dataset. The results of the ensemble enable us to perform an extensive\nerror analysis, which reveals open challenges for state-of-the-art methods and\ndirections towards pending future research. These challenges include missing\nparadigmatic context and inconsistent dataset labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07615v1",
            "date": "2018-09-20 13:39:10+00:00",
            "title": "Lessons learned in multilingual grounded language learning",
            "authors": [
                "\u00c1kos K\u00e1d\u00e1r",
                "Desmond Elliott",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi"
            ],
            "abstract": "Recent work has shown how to learn better visual-semantic embeddings by\nleveraging image descriptions in more than one language. Here, we investigate\nin detail which conditions affect the performance of this type of grounded\nlanguage learning model. We show that multilingual training improves over\nbilingual training, and that low-resource languages benefit from training with\nhigher-resource languages. We demonstrate that a multilingual model can be\ntrained equally well on either translations or comparable sentence pairs, and\nthat annotating the same set of images in multiple language enables further\nimprovements via an additional caption-caption ranking objective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07629v1",
            "date": "2018-09-19 05:40:35+00:00",
            "title": "Investigating Linguistic Pattern Ordering in Hierarchical Natural Language Generation",
            "authors": [
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Natural language generation (NLG) is a critical component in spoken dialogue\nsystem, which can be divided into two phases: (1) sentence planning: deciding\nthe overall sentence structure, (2) surface realization: determining specific\nword forms and flattening the sentence structure into a string. With the rise\nof deep learning, most modern NLG models are based on a sequence-to-sequence\n(seq2seq) model, which basically contains an encoder-decoder structure; these\nNLG models generate sentences from scratch by jointly optimizing sentence\nplanning and surface realization. However, such simple encoder-decoder\narchitecture usually fail to generate complex and long sentences, because the\ndecoder has difficulty learning all grammar and diction knowledge well. This\npaper introduces an NLG model with a hierarchical attentional decoder, where\nthe hierarchy focuses on leveraging linguistic knowledge in a specific order.\nThe experiments show that the proposed method significantly outperforms the\ntraditional seq2seq model with a smaller model size, and the design of the\nhierarchical attentional decoder can be applied to various NLG systems.\nFurthermore, different generation strategies based on linguistic patterns are\ninvestigated and analyzed in order to guide future NLG research work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07657v1",
            "date": "2018-09-20 14:53:47+00:00",
            "title": "Joint Multilingual Supervision for Cross-lingual Entity Linking",
            "authors": [
                "Shyam Upadhyay",
                "Nitish Gupta",
                "Dan Roth"
            ],
            "abstract": "Cross-lingual Entity Linking (XEL) aims to ground entity mentions written in\nany language to an English Knowledge Base (KB), such as Wikipedia. XEL for most\nlanguages is challenging, owing to limited availability of resources as\nsupervision. We address this challenge by developing the first XEL approach\nthat combines supervision from multiple languages jointly. This enables our\napproach to: (a) augment the limited supervision in the target language with\nadditional supervision from a high-resource language (like English), and (b)\ntrain a single entity linking model for multiple languages, improving upon\nindividually trained models for each language. Extensive evaluation on three\nbenchmark datasets across 8 languages shows that our approach significantly\nimproves over the current state-of-the-art. We also provide analyses in two\nlimited resource settings: (a) zero-shot setting, when no supervision in the\ntarget language is available, and in (b) low-resource setting, when some\nsupervision in the target language is available. Our analysis provides insights\ninto the limitations of zero-shot XEL approaches in realistic scenarios, and\nshows the value of joint supervision in low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07721v1",
            "date": "2018-09-20 16:21:29+00:00",
            "title": "Symbolic Priors for RNN-based Semantic Parsing",
            "authors": [
                "Chunyang Xiao",
                "Marc Dymetman",
                "Claire Gardent"
            ],
            "abstract": "Seq2seq models based on Recurrent Neural Networks (RNNs) have recently\nreceived a lot of attention in the domain of Semantic Parsing for Question\nAnswering. While in principle they can be trained directly on pairs (natural\nlanguage utterances, logical forms), their performance is limited by the amount\nof available data. To alleviate this problem, we propose to exploit various\nsources of prior knowledge: the well-formedness of the logical forms is modeled\nby a weighted context-free grammar; the likelihood that certain entities\npresent in the input utterance are also present in the logical form is modeled\nby weighted finite-state automata. The grammar and automata are combined\ntogether through an efficient intersection algorithm to form a soft guide\n(\"background\") to the RNN. We test our method on an extension of the Overnight\ndataset and show that it not only strongly improves over an RNN baseline, but\nalso outperforms non-RNN models based on rich sets of hand-crafted features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07783v1",
            "date": "2018-09-20 18:02:49+00:00",
            "title": "Rapid Customization for Event Extraction",
            "authors": [
                "Yee Seng Chan",
                "Joshua Fasching",
                "Haoling Qiu",
                "Bonan Min"
            ],
            "abstract": "We present a system for rapidly customizing event extraction capability to\nfind new event types and their arguments. The system allows a user to find,\nexpand and filter event triggers for a new event type by exploring an\nunannotated corpus. The system will then automatically generate mention-level\nevent annotation automatically, and train a Neural Network model for finding\nthe corresponding event. Additionally, the system uses the ACE corpus to train\nan argument model for extracting Actor, Place, and Time arguments for any event\ntypes, including ones not seen in its training data. Experiments show that with\nless than 10 minutes of human effort per event type, the system achieves good\nperformance for 67 novel event types. The code, documentation, and a\ndemonstration video will be released as open source on github.com.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07807v1",
            "date": "2018-09-20 19:05:29+00:00",
            "title": "Bootstrapping Transliteration with Constrained Discovery for Low-Resource Languages",
            "authors": [
                "Shyam Upadhyay",
                "Jordan Kodner",
                "Dan Roth"
            ],
            "abstract": "Generating the English transliteration of a name written in a foreign script\nis an important and challenging step in multilingual knowledge acquisition and\ninformation extraction. Existing approaches to transliteration generation\nrequire a large (>5000) number of training examples. This difficulty contrasts\nwith transliteration discovery, a somewhat easier task that involves picking a\nplausible transliteration from a given list. In this work, we present a\nbootstrapping algorithm that uses constrained discovery to improve generation,\nand can be used with as few as 500 training examples, which we show can be\nsourced from annotators in a matter of hours. This opens the task to languages\nfor which large number of training examples are unavailable. We evaluate\ntransliteration generation performance itself, as well the improvement it\nbrings to cross-lingual candidate generation for entity linking, a typical\ndownstream task. We present a comprehensive evaluation of our approach on nine\nlanguages, each written in a unique script.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07832v2",
            "date": "2018-09-20 20:04:07+00:00",
            "title": "LSTM-based Whisper Detection",
            "authors": [
                "Zeynab Raeesy",
                "Kellen Gillespie",
                "Zhenpei Yang",
                "Chengyuan Ma",
                "Thomas Drugman",
                "Jiacheng Gu",
                "Roland Maas",
                "Ariya Rastrow",
                "Bj\u00f6rn Hoffmeister"
            ],
            "abstract": "This article presents a whisper speech detector in the far-field domain. The\nproposed system consists of a long-short term memory (LSTM) neural network\ntrained on log-filterbank energy (LFBE) acoustic features. This model is\ntrained and evaluated on recordings of human interactions with\nvoice-controlled, far-field devices in whisper and normal phonation modes. We\ncompare multiple inference approaches for utterance-level classification by\nexamining trajectories of the LSTM posteriors. In addition, we engineer a set\nof features based on the signal characteristics inherent to whisper speech, and\nevaluate their effectiveness in further separating whisper from normal speech.\nA benchmarking of these features using multilayer perceptrons (MLP) and LSTMs\nsuggests that the proposed features, in combination with LFBE features, can\nhelp us further improve our classifiers. We prove that, with enough data, the\nLSTM model is indeed as capable of learning whisper characteristics from LFBE\nfeatures alone compared to a simpler MLP model that uses both LFBE and features\nengineered for separating whisper and normal speech. In addition, we prove that\nthe LSTM classifiers accuracy can be further improved with the incorporation of\nthe proposed engineered features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07889v4",
            "date": "2018-09-20 23:21:39+00:00",
            "title": "Predicting the Argumenthood of English Prepositional Phrases",
            "authors": [
                "Najoung Kim",
                "Kyle Rawlins",
                "Benjamin Van Durme",
                "Paul Smolensky"
            ],
            "abstract": "Distinguishing between arguments and adjuncts of a verb is a longstanding,\nnontrivial problem. In natural language processing, argumenthood information is\nimportant in tasks such as semantic role labeling (SRL) and prepositional\nphrase (PP) attachment disambiguation. In theoretical linguistics, many\ndiagnostic tests for argumenthood exist but they often yield conflicting and\npotentially gradient results. This is especially the case for syntactically\noblique items such as PPs. We propose two PP argumenthood prediction tasks\nbranching from these two motivations: (1) binary argument-adjunct\nclassification of PPs in VerbNet, and (2) gradient argumenthood prediction\nusing human judgments as gold standard, and report results from prediction\nmodels that use pretrained word embeddings and other linguistically informed\nfeatures. Our best results on each task are (1) $acc.=0.955$, $F_1=0.954$\n(ELMo+BiLSTM) and (2) Pearson's $r=0.624$ (word2vec+MLP). Furthermore, we\ndemonstrate the utility of argumenthood prediction in improving sentence\nrepresentations via performance gains on SRL when a sentence encoder is\npretrained with our tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07978v1",
            "date": "2018-09-21 08:18:12+00:00",
            "title": "Paraphrase Detection on Noisy Subtitles in Six Languages",
            "authors": [
                "Eetu Sj\u00f6blom",
                "Mathias Creutz",
                "Mikko Aulamo"
            ],
            "abstract": "We perform automatic paraphrase detection on subtitle data from the\nOpusparcus corpus comprising six European languages: German, English, Finnish,\nFrench, Russian, and Swedish. We train two types of supervised sentence\nembedding models: a word-averaging (WA) model and a gated recurrent averaging\nnetwork (GRAN) model. We find out that GRAN outperforms WA and is more robust\nto noisy training data. Better results are obtained with more and noisier data\nthan less and cleaner data. Additionally, we experiment on other datasets,\nwithout reaching the same level of performance, because of domain mismatch\nbetween training and test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08037v3",
            "date": "2018-09-21 11:03:48+00:00",
            "title": "Understanding Convolutional Neural Networks for Text Classification",
            "authors": [
                "Alon Jacovi",
                "Oren Sar Shalom",
                "Yoav Goldberg"
            ],
            "abstract": "We present an analysis into the inner workings of Convolutional Neural\nNetworks (CNNs) for processing text. CNNs used for computer vision can be\ninterpreted by projecting filters into image space, but for discrete sequence\ninputs CNNs remain a mystery. We aim to understand the method by which the\nnetworks process and classify text. We examine common hypotheses to this\nproblem: that filters, accompanied by global max-pooling, serve as ngram\ndetectors. We show that filters may capture several different semantic classes\nof ngrams by using different activation patterns, and that global max-pooling\ninduces behavior which separates important ngrams from the rest. Finally, we\nshow practical use cases derived from our findings in the form of model\ninterpretability (explaining a trained model by deriving a concrete identity\nfor each filter, bridging the gap between visualization tools in vision tasks\nand NLP) and prediction interpretability (explaining predictions). Code\nimplementation is available online at\ngithub.com/sayaendo/interpreting-cnn-for-text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08145v1",
            "date": "2018-09-21 14:31:57+00:00",
            "title": "Predicting the Usefulness of Amazon Reviews Using Off-The-Shelf Argumentation Mining",
            "authors": [
                "Marco Passon",
                "Marco Lippi",
                "Giuseppe Serra",
                "Carlo Tasso"
            ],
            "abstract": "Internet users generate content at unprecedented rates. Building intelligent\nsystems capable of discriminating useful content within this ocean of\ninformation is thus becoming a urgent need. In this paper, we aim to predict\nthe usefulness of Amazon reviews, and to do this we exploit features coming\nfrom an off-the-shelf argumentation mining system. We argue that the usefulness\nof a review, in fact, is strictly related to its argumentative content, whereas\nthe use of an already trained system avoids the costly need of relabeling a\nnovel dataset. Results obtained on a large publicly available corpus support\nthis hypothesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08193v2",
            "date": "2018-09-21 16:24:37+00:00",
            "title": "Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection",
            "authors": [
                "Lev Konstantinovskiy",
                "Oliver Price",
                "Mevan Babakar",
                "Arkaitz Zubiaga"
            ],
            "abstract": "In an effort to assist factcheckers in the process of factchecking, we tackle\nthe claim detection task, one of the necessary stages prior to determining the\nveracity of a claim. It consists of identifying the set of sentences, out of a\nlong text, deemed capable of being factchecked. This paper is a collaborative\nwork between Full Fact, an independent factchecking charity, and academic\npartners. Leveraging the expertise of professional factcheckers, we develop an\nannotation schema and a benchmark for automated claim detection that is more\nconsistent across time, topics and annotators than previous approaches. Our\nannotation schema has been used to crowdsource the annotation of a dataset with\nsentences from UK political TV shows. We introduce an approach based on\nuniversal sentence representations to perform the classification, achieving an\nF1 score of 0.83, with over 5% relative improvement over the state-of-the-art\nmethods ClaimBuster and ClaimRank. The system was deployed in production and\nreceived positive user feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08205v1",
            "date": "2018-09-21 16:50:17+00:00",
            "title": "Towards Exploiting Background Knowledge for Building Conversation Systems",
            "authors": [
                "Nikita Moghe",
                "Siddhartha Arora",
                "Suman Banerjee",
                "Mitesh M. Khapra"
            ],
            "abstract": "Existing dialog datasets contain a sequence of utterances and responses\nwithout any explicit background knowledge associated with them. This has\nresulted in the development of models which treat conversation as a\nsequence-to-sequence generation task i.e, given a sequence of utterances\ngenerate the response sequence). This is not only an overly simplistic view of\nconversation but it is also emphatically different from the way humans converse\nby heavily relying on their background knowledge about the topic (as opposed to\nsimply relying on the previous sequence of utterances). For example, it is\ncommon for humans to (involuntarily) produce utterances which are copied or\nsuitably modified from background articles they have read about the topic. To\nfacilitate the development of such natural conversation models which mimic the\nhuman process of conversing, we create a new dataset containing movie chats\nwherein each response is explicitly generated by copying and/or modifying\nsentences from unstructured background knowledge such as plots, comments and\nreviews about the movie. We establish baseline results on this dataset (90K\nutterances from 9K conversations) using three different models: (i) pure\ngeneration based models which ignore the background knowledge (ii) generation\nbased models which learn to copy information from the background knowledge when\nrequired and (iii) span prediction based models which predict the appropriate\nresponse span in the background knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08267v3",
            "date": "2018-09-21 18:42:24+00:00",
            "title": "Neural Approaches to Conversational AI",
            "authors": [
                "Jianfeng Gao",
                "Michel Galley",
                "Lihong Li"
            ],
            "abstract": "The present paper surveys neural approaches to conversational AI that have\nbeen developed in the last few years. We group conversational systems into\nthree categories: (1) question answering agents, (2) task-oriented dialogue\nagents, and (3) chatbots. For each category, we present a review of\nstate-of-the-art neural approaches, draw the connection between them and\ntraditional approaches, and discuss the progress that has been made and\nchallenges still being faced, using specific systems and models as case\nstudies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08298v1",
            "date": "2018-09-21 20:13:41+00:00",
            "title": "How do you correct run-on sentences it's not as easy as it seems",
            "authors": [
                "Junchao Zheng",
                "Courtney Napoles",
                "Joel Tetreault",
                "Kostiantyn Omelianchuk"
            ],
            "abstract": "Run-on sentences are common grammatical mistakes but little research has\ntackled this problem to date. This work introduces two machine learning models\nto correct run-on sentences that outperform leading methods for related tasks,\npunctuation restoration and whole-sentence grammatical error correction. Due to\nthe limited annotated data for this error, we experiment with artificially\ngenerating training data from clean newswire text. Our findings suggest\nartificial training data is viable for this task. We discuss implications for\ncorrecting run-ons and other types of mistakes that have low coverage in\nerror-annotated corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08370v1",
            "date": "2018-09-22 02:39:32+00:00",
            "title": "Semi-Supervised Sequence Modeling with Cross-View Training",
            "authors": [
                "Kevin Clark",
                "Minh-Thang Luong",
                "Christopher D. Manning",
                "Quoc V. Le"
            ],
            "abstract": "Unsupervised representation learning algorithms such as word2vec and ELMo\nimprove the accuracy of many supervised NLP models, mainly because they can\ntake advantage of large amounts of unlabeled text. However, the supervised\nmodels only learn from task-specific labeled data during the main training\nphase. We therefore propose Cross-View Training (CVT), a semi-supervised\nlearning algorithm that improves the representations of a Bi-LSTM sentence\nencoder using a mix of labeled and unlabeled data. On labeled examples,\nstandard supervised learning is used. On unlabeled examples, CVT teaches\nauxiliary prediction modules that see restricted views of the input (e.g., only\npart of a sentence) to match the predictions of the full model seeing the whole\ninput. Since the auxiliary modules and the full model share intermediate\nrepresentations, this in turn improves the full model. Moreover, we show that\nCVT is particularly effective when combined with multi-task learning. We\nevaluate CVT on five sequence tagging tasks, machine translation, and\ndependency parsing, achieving state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08386v1",
            "date": "2018-09-22 06:03:29+00:00",
            "title": "A Byte-sized Approach to Named Entity Recognition",
            "authors": [
                "Emily Sheng",
                "Prem Natarajan"
            ],
            "abstract": "In biomedical literature, it is common for entity boundaries to not align\nwith word boundaries. Therefore, effective identification of entity spans\nrequires approaches capable of considering tokens that are smaller than words.\nWe introduce a novel, subword approach for named entity recognition (NER) that\nuses byte-pair encodings (BPE) in combination with convolutional and recurrent\nneural networks to produce byte-level tags of entities. We present experimental\nresults on several standard biomedical datasets, namely the BioCreative VI\nBio-ID, JNLPBA, and GENETAG datasets. We demonstrate competitive performance\nwhile bypassing the specialized domain expertise needed to create biomedical\ntext tokenization rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08621v2",
            "date": "2018-09-23 16:02:03+00:00",
            "title": "Learning and Evaluating Sparse Interpretable Sentence Embeddings",
            "authors": [
                "Valentin Trifonov",
                "Octavian-Eugen Ganea",
                "Anna Potapenko",
                "Thomas Hofmann"
            ],
            "abstract": "Previous research on word embeddings has shown that sparse representations,\nwhich can be either learned on top of existing dense embeddings or obtained\nthrough model constraints during training time, have the benefit of increased\ninterpretability properties: to some degree, each dimension can be understood\nby a human and associated with a recognizable feature in the data. In this\npaper, we transfer this idea to sentence embeddings and explore several\napproaches to obtain a sparse representation. We further introduce a novel,\nquantitative and automated evaluation metric for sentence embedding\ninterpretability, based on topic coherence methods. We observe an increase in\ninterpretability compared to dense models, on a dataset of movie dialogs and on\nthe scene descriptions from the MS COCO dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08651v1",
            "date": "2018-09-23 18:19:03+00:00",
            "title": "Detecting Hate Speech and Offensive Language on Twitter using Machine Learning: An N-gram and TFIDF based Approach",
            "authors": [
                "Aditya Gaydhani",
                "Vikrant Doma",
                "Shrikant Kendre",
                "Laxmi Bhagwat"
            ],
            "abstract": "Toxic online content has become a major issue in today's world due to an\nexponential increase in the use of internet by people of different cultures and\neducational background. Differentiating hate speech and offensive language is a\nkey challenge in automatic detection of toxic text content. In this paper, we\npropose an approach to automatically classify tweets on Twitter into three\nclasses: hateful, offensive and clean. Using Twitter dataset, we perform\nexperiments considering n-grams as features and passing their term\nfrequency-inverse document frequency (TFIDF) values to multiple machine\nlearning models. We perform comparative analysis of the models considering\nseveral values of n in n-grams and TFIDF normalization methods. After tuning\nthe model giving the best results, we achieve 95.6% accuracy upon evaluating it\non test data. We also create a module which serves as an intermediate between\nuser and Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08652v1",
            "date": "2018-09-23 18:19:46+00:00",
            "title": "Mind Your Language: Abuse and Offense Detection for Code-Switched Languages",
            "authors": [
                "Raghav Kapoor",
                "Yaman Kumar",
                "Kshitij Rajput",
                "Rajiv Ratn Shah",
                "Ponnurangam Kumaraguru",
                "Roger Zimmermann"
            ],
            "abstract": "In multilingual societies like the Indian subcontinent, use of code-switched\nlanguages is much popular and convenient for the users. In this paper, we study\noffense and abuse detection in the code-switched pair of Hindi and English\n(i.e. Hinglish), the pair that is the most spoken. The task is made difficult\ndue to non-fixed grammar, vocabulary, semantics and spellings of Hinglish\nlanguage. We apply transfer learning and make a LSTM based model for hate\nspeech classification. This model surpasses the performance shown by the\ncurrent best models to establish itself as the state-of-the-art in the\nunexplored domain of Hinglish offensive text classification.We also release our\nmodel and the embeddings trained for research purposes",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08703v1",
            "date": "2018-09-19 04:15:36+00:00",
            "title": "Monolingual sentence matching for text simplification",
            "authors": [
                "Yonghui Huang",
                "Yunhui Li",
                "Yi Luan"
            ],
            "abstract": "This work improves monolingual sentence alignment for text simplification,\nspecifically for text in standard and simple Wikipedia. We introduce a\nconvolutional neural network structure to model similarity between two\nsentences. Due to the limitation of available parallel corpora, the model is\ntrained in a semi-supervised way, by using the output of a knowledge-based high\nperformance aligning system. We apply the resulting similarity score to rescore\nthe knowledge-based output, and adapt the model by a small hand-aligned\ndataset. Experiments show that both rescoring and adaptation improve the\nperformance of knowledge-based method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08726v2",
            "date": "2018-09-24 02:18:01+00:00",
            "title": "Context-Aware Attention for Understanding Twitter Abuse",
            "authors": [
                "Tuhin Chakrabarty",
                "Kilol Gupta"
            ],
            "abstract": "The original goal of any social media platform is to facilitate users to\nindulge in healthy and meaningful conversations. But more often than not, it\nhas been found that it becomes an avenue for wanton attacks. We want to\nalleviate this issue and hence we try to provide a detailed analysis of how\nabusive behavior can be monitored in Twitter. The complexity of the natural\nlanguage constructs makes this task challenging. We show how applying\ncontextual attention to Long Short Term Memory networks help us give near state\nof art results on multiple benchmarks abuse detection data sets from Twitter.",
            "journal ref": "Proc. 3rd Workshop on Abusive Language Online, pp. 70-79, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08730v2",
            "date": "2018-09-24 02:42:40+00:00",
            "title": "Deformable Stacked Structure for Named Entity Recognition",
            "authors": [
                "Shuyang Cao",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural architecture for named entity recognition has achieved great success\nin the field of natural language processing. Currently, the dominating\narchitecture consists of a bi-directional recurrent neural network (RNN) as the\nencoder and a conditional random field (CRF) as the decoder. In this paper, we\npropose a deformable stacked structure for named entity recognition, in which\nthe connections between two adjacent layers are dynamically established. We\nevaluate the deformable stacked structure by adapting it to different layers.\nOur model achieves the state-of-the-art performances on the OntoNotes dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08731v1",
            "date": "2018-09-24 02:42:42+00:00",
            "title": "Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!",
            "authors": [
                "Katharina Kann",
                "Sascha Rothe",
                "Katja Filippova"
            ],
            "abstract": "Motivated by recent findings on the probabilistic modeling of acceptability\njudgments, we propose syntactic log-odds ratio (SLOR), a normalized language\nmodel score, as a metric for referenceless fluency evaluation of natural\nlanguage generation output at the sentence level. We further introduce WPSLOR,\na novel WordPiece-based version, which harnesses a more compact language model.\nEven though word-overlap metrics like ROUGE are computed with the help of\nhand-written references, our referenceless methods obtain a significantly\nhigher correlation with human fluency scores on a benchmark dataset of\ncompressed sentences. Finally, we present ROUGE-LM, a reference-based metric\nwhich is a natural extension of WPSLOR to the case of available references. We\nshow that ROUGE-LM yields a significantly higher correlation with human\njudgments than all baseline metrics, including WPSLOR on its own.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08733v2",
            "date": "2018-09-24 02:56:31+00:00",
            "title": "Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting",
            "authors": [
                "Katharina Kann",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Neural state-of-the-art sequence-to-sequence (seq2seq) models often do not\nperform well for small training sets. We address paradigm completion, the\nmorphological task of, given a partial paradigm, generating all missing forms.\nWe propose two new methods for the minimal-resource setting: (i) Paradigm\ntransduction: Since we assume only few paradigms available for training, neural\nseq2seq models are able to capture relationships between paradigm cells, but\nare tied to the idiosyncracies of the training set. Paradigm transduction\nmitigates this problem by exploiting the input subset of inflected forms at\ntest time. (ii) Source selection with high precision (SHIP): Multi-source\nmodels which learn to automatically select one or multiple sources to predict a\ntarget inflection do not perform well in the minimal-resource setting. SHIP is\nan alternative to identify a reliable source if training data is limited. On a\n52-language benchmark dataset, we outperform the previous state of the art by\nup to 9.71% absolute accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08895v3",
            "date": "2018-09-19 07:41:17+00:00",
            "title": "Neural Speech Synthesis with Transformer Network",
            "authors": [
                "Naihan Li",
                "Shujie Liu",
                "Yanqing Liu",
                "Sheng Zhao",
                "Ming Liu",
                "Ming Zhou"
            ],
            "abstract": "Although end-to-end neural text-to-speech (TTS) methods (such as Tacotron2)\nare proposed and achieve state-of-the-art performance, they still suffer from\ntwo problems: 1) low efficiency during training and inference; 2) hard to model\nlong dependency using current recurrent neural networks (RNNs). Inspired by the\nsuccess of Transformer network in neural machine translation (NMT), in this\npaper, we introduce and adapt the multi-head attention mechanism to replace the\nRNN structures and also the original attention mechanism in Tacotron2. With the\nhelp of multi-head self-attention, the hidden states in the encoder and decoder\nare constructed in parallel, which improves the training efficiency. Meanwhile,\nany two inputs at different times are connected directly by self-attention\nmechanism, which solves the long range dependency problem effectively. Using\nphoneme sequences as input, our Transformer TTS network generates mel\nspectrograms, followed by a WaveNet vocoder to output the final audio results.\nExperiments are conducted to test the efficiency and performance of our new\nnetwork. For the efficiency, our Transformer TTS network can speed up the\ntraining about 4.25 times faster compared with Tacotron2. For the performance,\nrigorous human tests show that our proposed model achieves state-of-the-art\nperformance (outperforms Tacotron2 with a gap of 0.048) and is very close to\nhuman quality (4.39 vs 4.44 in MOS).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08928v1",
            "date": "2018-09-24 13:49:14+00:00",
            "title": "Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings",
            "authors": [
                "Shafiq Joty",
                "Lluis Marquez",
                "Preslav Nakov"
            ],
            "abstract": "We address jointly two important tasks for Question Answering in community\nforums: given a new question, (i) find related existing questions, and (ii)\nfind relevant answers to this new question. We further use an auxiliary task to\ncomplement the previous two, i.e., (iii) find good answers with respect to the\nthread question in a question-comment thread. We use deep neural networks\n(DNNs) to learn meaningful task-specific embeddings, which we then incorporate\ninto a conditional random field (CRF) model for the multitask setting,\nperforming joint learning over a complex graph structure. While DNNs alone\nachieve competitive results when trained to produce the embeddings, the CRF,\nwhich makes use of the embeddings and the dependencies between the tasks,\nimproves the results significantly and consistently across a variety of\nevaluation metrics, thus showing the complementarity of DNNs and structured\nlearning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09078v2",
            "date": "2018-09-24 17:46:27+00:00",
            "title": "Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation",
            "authors": [
                "Xiao Liu",
                "Zhunchen Luo",
                "Heyan Huang"
            ],
            "abstract": "Event extraction is of practical utility in natural language processing. In\nthe real world, it is a common phenomenon that multiple events existing in the\nsame sentence, where extracting them are more difficult than extracting a\nsingle event. Previous works on modeling the associations between events by\nsequential modeling methods suffer a lot from the low efficiency in capturing\nvery long-range dependencies. In this paper, we propose a novel Jointly\nMultiple Events Extraction (JMEE) framework to jointly extract multiple event\ntriggers and arguments by introducing syntactic shortcut arcs to enhance\ninformation flow and attention-based graph convolution networks to model graph\ninformation. The experiment results demonstrate that our proposed framework\nachieves competitive results compared with state-of-the-art methods.",
            "journal ref": "EMNLP. 1 (2018) 1247-1256",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09194v1",
            "date": "2018-09-24 19:58:07+00:00",
            "title": "Stochastic Answer Networks for SQuAD 2.0",
            "authors": [
                "Xiaodong Liu",
                "Wei Li",
                "Yuwei Fang",
                "Aerin Kim",
                "Kevin Duh",
                "Jianfeng Gao"
            ],
            "abstract": "This paper presents an extension of the Stochastic Answer Network (SAN), one\nof the state-of-the-art machine reading comprehension models, to be able to\njudge whether a question is unanswerable or not. The extended SAN contains two\ncomponents: a span detector and a binary classifier for judging whether the\nquestion is unanswerable, and both components are jointly optimized.\nExperiments show that SAN achieves the results competitive to the\nstate-of-the-art on Stanford Question Answering Dataset (SQuAD) 2.0. To\nfacilitate the research on this field, we release our code:\nhttps://github.com/kevinduh/san_mrc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09296v2",
            "date": "2018-09-25 03:02:38+00:00",
            "title": "Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation",
            "authors": [
                "Xiang Kong",
                "Qizhe Xie",
                "Zihang Dai",
                "Eduard Hovy"
            ],
            "abstract": "Mixture of Softmaxes (MoS) has been shown to be effective at addressing the\nexpressiveness limitation of Softmax-based models. Despite the known advantage,\nMoS is practically sealed by its large consumption of memory and computational\ntime due to the need of computing multiple Softmaxes. In this work, we set out\nto unleash the power of MoS in practical applications by investigating improved\nword coding schemes, which could effectively reduce the vocabulary size and\nhence relieve the memory and computation burden. We show both BPE and our\nproposed Hybrid-LightRNN lead to improved encoding mechanisms that can halve\nthe time and memory consumption of MoS without performance losses. With MoS, we\nachieve an improvement of 1.5 BLEU scores on IWSLT 2014 German-to-English\ncorpus and an improvement of 0.76 CIDEr score on image captioning. Moreover, on\nthe larger WMT 2014 machine translation dataset, our MoS-boosted Transformer\nyields 29.5 BLEU score for English-to-German and 42.1 BLEU score for\nEnglish-to-French, outperforming the single-Softmax Transformer by 0.8 and 0.4\nBLEU scores respectively and achieving the state-of-the-art result on WMT 2014\nEnglish-to-German task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09528v2",
            "date": "2018-09-25 14:54:26+00:00",
            "title": "ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters",
            "authors": [
                "Abdalghani Abujabal",
                "Rishiraj Saha Roy",
                "Mohamed Yahya",
                "Gerhard Weikum"
            ],
            "abstract": "To bridge the gap between the capabilities of the state-of-the-art in factoid\nquestion answering (QA) and what users ask, we need large datasets of real user\nquestions that capture the various question phenomena users are interested in,\nand the diverse ways in which these questions are formulated. We introduce\nComQA, a large dataset of real user questions that exhibit different\nchallenging aspects such as compositionality, temporal reasoning, and\ncomparisons. ComQA questions come from the WikiAnswers community QA platform,\nwhich typically contains questions that are not satisfactorily answerable by\nexisting search engine technology. Through a large crowdsourcing effort, we\nclean the question dataset, group questions into paraphrase clusters, and\nannotate clusters with their answers. ComQA contains 11,214 questions grouped\ninto 4,834 paraphrase clusters. We detail the process of constructing ComQA,\nincluding the measures taken to ensure its high quality while making effective\nuse of crowdsourcing. We also present an extensive analysis of the dataset and\nthe results achieved by state-of-the-art systems on ComQA, demonstrating that\nour dataset can be a driver of future research on QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09600v1",
            "date": "2018-09-25 17:28:20+00:00",
            "title": "HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering",
            "authors": [
                "Zhilin Yang",
                "Peng Qi",
                "Saizheng Zhang",
                "Yoshua Bengio",
                "William W. Cohen",
                "Ruslan Salakhutdinov",
                "Christopher D. Manning"
            ],
            "abstract": "Existing question answering (QA) datasets fail to train QA systems to perform\ncomplex reasoning and provide explanations for answers. We introduce HotpotQA,\na new dataset with 113k Wikipedia-based question-answer pairs with four key\nfeatures: (1) the questions require finding and reasoning over multiple\nsupporting documents to answer; (2) the questions are diverse and not\nconstrained to any pre-existing knowledge bases or knowledge schemas; (3) we\nprovide sentence-level supporting facts required for reasoning, allowing QA\nsystems to reason with strong supervision and explain the predictions; (4) we\noffer a new type of factoid comparison questions to test QA systems' ability to\nextract relevant facts and perform necessary comparison. We show that HotpotQA\nis challenging for the latest QA systems, and the supporting facts enable\nmodels to improve performance and make explainable predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09605v1",
            "date": "2018-09-25 17:35:57+00:00",
            "title": "A Re-ranker Scheme for Integrating Large Scale NLU models",
            "authors": [
                "Chengwei Su",
                "Rahul Gupta",
                "Shankar Ananthakrishnan",
                "Spyros Matsoukas"
            ],
            "abstract": "Large scale Natural Language Understanding (NLU) systems are typically\ntrained on large quantities of data, requiring a fast and scalable training\nstrategy. A typical design for NLU systems consists of domain-level NLU modules\n(domain classifier, intent classifier and named entity recognizer). Hypotheses\n(NLU interpretations consisting of various intent+slot combinations) from these\ndomain specific modules are typically aggregated with another downstream\ncomponent. The re-ranker integrates outputs from domain-level recognizers,\nreturning a scored list of cross domain hypotheses. An ideal re-ranker will\nexhibit the following two properties: (a) it should prefer the most relevant\nhypothesis for the given input as the top hypothesis and, (b) the\ninterpretation scores corresponding to each hypothesis produced by the\nre-ranker should be calibrated. Calibration allows the final NLU interpretation\nscore to be comparable across domains. We propose a novel re-ranker strategy\nthat addresses these aspects, while also maintaining domain specific\nmodularity. We design optimization loss functions for such a modularized\nre-ranker and present results on decreasing the top hypothesis error rate as\nwell as maintaining the model calibration. We also experiment with an extension\ninvolving training the domain specific re-rankers on datasets curated\nindependently by each domain to allow further asynchronization. %The proposed\nre-ranker design showcases the following: (i) improved NLU performance over an\nunweighted aggregation strategy, (ii) cross-domain calibrated performance and,\n(iii) support for use cases involving training each re-ranker on datasets\ncurated by each domain independently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09658v1",
            "date": "2018-09-25 18:50:39+00:00",
            "title": "Non-native children speech recognition through transfer learning",
            "authors": [
                "Marco Matassoni",
                "Roberto Gretter",
                "Daniele Falavigna",
                "Diego Giuliani"
            ],
            "abstract": "This work deals with non-native children's speech and investigates both\nmulti-task and transfer learning approaches to adapt a multi-language Deep\nNeural Network (DNN) to speakers, specifically children, learning a foreign\nlanguage. The application scenario is characterized by young students learning\nEnglish and German and reading sentences in these second-languages, as well as\nin their mother language. The paper analyzes and discusses techniques for\ntraining effective DNN-based acoustic models starting from children native\nspeech and performing adaptation with limited non-native audio material. A\nmulti-lingual model is adopted as baseline, where a common phonetic lexicon,\ndefined in terms of the units of the International Phonetic Alphabet (IPA), is\nshared across the three languages at hand (Italian, German and English); DNN\nadaptation methods based on transfer learning are evaluated on significant\nnon-native evaluation sets. Results show that the resulting non-native models\nallow a significant improvement with respect to a mono-lingual system adapted\nto speakers of the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09672v3",
            "date": "2018-09-25 19:18:52+00:00",
            "title": "BanditSum: Extractive Summarization as a Contextual Bandit",
            "authors": [
                "Yue Dong",
                "Yikang Shen",
                "Eric Crawford",
                "Herke van Hoof",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "In this work, we propose a novel method for training neural networks to\nperform single-document extractive summarization without\nheuristically-generated extractive labels. We call our approach BanditSum as it\ntreats extractive summarization as a contextual bandit (CB) problem, where the\nmodel receives a document to summarize (the context), and chooses a sequence of\nsentences to include in the summary (the action). A policy gradient\nreinforcement learning algorithm is used to train the model to select sequences\nof sentences that maximize ROUGE score. We perform a series of experiments\ndemonstrating that BanditSum is able to achieve ROUGE scores that are better\nthan or comparable to the state-of-the-art for extractive summarization, and\nconverges using significantly fewer update steps than competing approaches. In\naddition, we show empirically that BanditSum performs significantly better than\ncompeting approaches when good summary sentences appear late in the source\ndocument.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09795v1",
            "date": "2018-09-26 03:54:22+00:00",
            "title": "Deep contextualized word representations for detecting sarcasm and irony",
            "authors": [
                "Suzana Ili\u0107",
                "Edison Marrese-Taylor",
                "Jorge A. Balazs",
                "Yutaka Matsuo"
            ],
            "abstract": "Predicting context-dependent and non-literal utterances like sarcastic and\nironic expressions still remains a challenging task in NLP, as it goes beyond\nlinguistic patterns, encompassing common sense and shared knowledge as crucial\ncomponents. To capture complex morpho-syntactic features that can usually serve\nas indicators for irony or sarcasm across dynamic contexts, we propose a model\nthat uses character-level vector representations of words, based on ELMo. We\ntest our model on 7 different datasets derived from 3 different data sources,\nproviding state-of-the-art performance in 6 of them, and otherwise offering\ncompetitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10040v2",
            "date": "2018-09-26 14:58:59+00:00",
            "title": "Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis",
            "authors": [
                "Kelly W. Zhang",
                "Samuel R. Bowman"
            ],
            "abstract": "Recent work using auxiliary prediction task classifiers to investigate the\nproperties of LSTM representations has begun to shed light on why pretrained\nrepresentations, like ELMo (Peters et al., 2018) and CoVe (McCann et al.,\n2017), are so beneficial for neural language understanding models. We still,\nthough, do not yet have a clear understanding of how the choice of pretraining\nobjective affects the type of linguistic information that models learn. With\nthis in mind, we compare four objectives---language modeling, translation,\nskip-thought, and autoencoding---on their ability to induce syntactic and\npart-of-speech information. We make a fair comparison between the tasks by\nholding constant the quantity and genre of the training data, as well as the\nLSTM architecture. We find that representations from language models\nconsistently perform best on our syntactic auxiliary prediction tasks, even\nwhen trained on relatively small amounts of data. These results suggest that\nlanguage modeling may be the best data-rich pretraining task for transfer\nlearning applications requiring syntactic information. We also find that the\nrepresentations from randomly-initialized, frozen LSTMs perform strikingly well\non our syntactic auxiliary tasks, but this effect disappears when the amount of\ntraining data for the auxiliary tasks is reduced.",
            "journal ref": "Blackbox NLP Workshop, EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10185v1",
            "date": "2018-09-26 18:49:07+00:00",
            "title": "Graph Convolution over Pruned Dependency Trees Improves Relation Extraction",
            "authors": [
                "Yuhao Zhang",
                "Peng Qi",
                "Christopher D. Manning"
            ],
            "abstract": "Dependency trees help relation extraction models capture long-range relations\nbetween words. However, existing dependency-based models either neglect crucial\ninformation (e.g., negation) by pruning the dependency trees too aggressively,\nor are computationally inefficient because it is difficult to parallelize over\ndifferent tree structures. We propose an extension of graph convolutional\nnetworks that is tailored for relation extraction, which pools information over\narbitrary dependency structures efficiently in parallel. To incorporate\nrelevant information while maximally removing irrelevant content, we further\napply a novel pruning strategy to the input trees by keeping words immediately\naround the shortest path between the two entities among which a relation might\nhold. The resulting model achieves state-of-the-art performance on the\nlarge-scale TACRED dataset, outperforming existing sequence and\ndependency-based neural models. We also show through detailed analysis that\nthis model has complementary strengths to sequence models, and combining them\nfurther improves the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10324v2",
            "date": "2018-09-27 03:11:35+00:00",
            "title": "Iterative Document Representation Learning Towards Summarization with Polishing",
            "authors": [
                "Xiuying Chen",
                "Shen Gao",
                "Chongyang Tao",
                "Yan Song",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "In this paper, we introduce Iterative Text Summarization (ITS), an\niteration-based model for supervised extractive text summarization, inspired by\nthe observation that it is often necessary for a human to read an article\nmultiple times in order to fully understand and summarize its contents. Current\nsummarization approaches read through a document only once to generate a\ndocument representation, resulting in a sub-optimal representation. To address\nthis issue we introduce a model which iteratively polishes the document\nrepresentation on many passes through the document. As part of our model, we\nalso introduce a selective reading mechanism that decides more accurately the\nextent to which each sentence in the model should be updated. Experimental\nresults on the CNN/DailyMail and DUC2002 datasets demonstrate that our model\nsignificantly outperforms state-of-the-art extractive systems when evaluated by\nmachines and by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10644v1",
            "date": "2018-09-27 17:12:24+00:00",
            "title": "Predictive Embeddings for Hate Speech Detection on Twitter",
            "authors": [
                "Rohan Kshirsagar",
                "Tyus Cukuvac",
                "Kathleen McKeown",
                "Susan McGregor"
            ],
            "abstract": "We present a neural-network based approach to classifying online hate speech\nin general, as well as racist and sexist speech in particular. Using\npre-trained word embeddings and max/mean pooling from simple, fully-connected\ntransformations of these embeddings, we are able to predict the occurrence of\nhate speech on three commonly used publicly available datasets. Our models\nmatch or outperform state of the art F1 performance on all three datasets using\nsignificantly fewer parameters and minimal feature preprocessing compared to\nprevious methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10735v2",
            "date": "2018-09-27 19:33:03+00:00",
            "title": "A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC",
            "authors": [
                "Mark Yatskar"
            ],
            "abstract": "We compare three new datasets for question answering: SQuAD 2.0, QuAC, and\nCoQA, along several of their new features: (1) unanswerable questions, (2)\nmulti-turn interactions, and (3) abstractive answers. We show that the datasets\nprovide complementary coverage of the first two aspects, but weak coverage of\nthe third. Because of the datasets' structural similarity, a single extractive\nmodel can be easily adapted to any of the datasets and we show improved\nbaseline results on both SQuAD 2.0 and CoQA. Despite the similarity, models\ntrained on one dataset are ineffective on another dataset, but we find moderate\nperformance improvement through pretraining. To encourage cross-evaluation, we\nrelease code for conversion between datasets at\nhttps://github.com/my89/co-squac .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10736v3",
            "date": "2018-09-27 19:33:54+00:00",
            "title": "Controllable Neural Story Plot Generation via Reinforcement Learning",
            "authors": [
                "Pradyumna Tambwekar",
                "Murtaza Dhuliawala",
                "Lara J. Martin",
                "Animesh Mehta",
                "Brent Harrison",
                "Mark O. Riedl"
            ],
            "abstract": "Language-modeling--based approaches to story plot generation attempt to\nconstruct a plot by sampling from a language model (LM) to predict the next\ncharacter, word, or sentence to add to the story. LM techniques lack the\nability to receive guidance from the user to achieve a specific goal, resulting\nin stories that don't have a clear sense of progression and lack coherence. We\npresent a reward-shaping technique that analyzes a story corpus and produces\nintermediate rewards that are backpropagated into a pre-trained LM in order to\nguide the model towards a given goal. Automated evaluations show our technique\ncan create a model that generates story plots which consistently achieve a\nspecified goal. Human-subject studies show that the generated stories have more\nplausible event ordering than baseline plot generation techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10763v1",
            "date": "2018-09-27 21:00:36+00:00",
            "title": "Building a Lemmatizer and a Spell-checker for Sorani Kurdish",
            "authors": [
                "Shahin Salavati",
                "Sina Ahmadi"
            ],
            "abstract": "The present paper aims at presenting a lemmatization and a word-level error\ncorrection system for Sorani Kurdish. We propose a hybrid approach based on the\nmorphological rules and a n-gram language model. We have called our\nlemmatization and error correction systems Peyv and R\\^en\\^us respectively,\nwhich are the first tools presented for Sorani Kurdish to the best of our\nknowledge. The Peyv lemmatizer has shown 86.7% accuracy. As for R\\^en\\^us,\nusing a lexicon, we have obtained 96.4% accuracy while without a lexicon, the\ncorrection system has 87% accuracy. As two fundamental text processing tools,\nthese tools can pave the way for further researches on more natural language\nprocessing applications for Sorani Kurdish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10853v3",
            "date": "2018-09-28 04:30:11+00:00",
            "title": "Adaptive Input Representations for Neural Language Modeling",
            "authors": [
                "Alexei Baevski",
                "Michael Auli"
            ],
            "abstract": "We introduce adaptive input representations for neural language modeling\nwhich extend the adaptive softmax of Grave et al. (2017) to input\nrepresentations of variable capacity. There are several choices on how to\nfactorize the input and output layers, and whether to model words, characters\nor sub-word units. We perform a systematic comparison of popular choices for a\nself-attentional architecture. Our experiments show that models equipped with\nadaptive embeddings are more than twice as fast to train than the popular\ncharacter input CNN while having a lower number of parameters. On the\nWikiText-103 benchmark we achieve 18.7 perplexity, an improvement of 10.5\nperplexity compared to the previously best published result and on the Billion\nWord benchmark, we achieve 23.02 perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10867v1",
            "date": "2018-09-28 06:04:32+00:00",
            "title": "The Rule of Three: Abstractive Text Summarization in Three Bullet Points",
            "authors": [
                "Tomonori Kodaira",
                "Mamoru Komachi"
            ],
            "abstract": "Neural network-based approaches have become widespread for abstractive text\nsummarization. Though previously proposed models for abstractive text\nsummarization addressed the problem of repetition of the same contents in the\nsummary, they did not explicitly consider its information structure. One of the\nreasons these previous models failed to account for information structure in\nthe generated summary is that standard datasets include summaries of variable\nlengths, resulting in problems in analyzing information flow, specifically, the\nmanner in which the first sentence is related to the following sentences.\nTherefore, we use a dataset containing summaries with only three bullet points,\nand propose a neural network-based abstractive summarization model that\nconsiders the information structures of the generated summaries. Our\nexperimental results show that the information structure of a summary can be\ncontrolled, thus improving the performance of the overall summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00278v3",
            "date": "2018-09-29 23:44:39+00:00",
            "title": "MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling",
            "authors": [
                "Pawe\u0142 Budzianowski",
                "Tsung-Hsien Wen",
                "Bo-Hsiang Tseng",
                "I\u00f1igo Casanueva",
                "Stefan Ultes",
                "Osman Ramadan",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Even though machine learning has become the major scene in dialogue research\ncommunity, the real breakthrough has been blocked by the scale of data\navailable. To address this fundamental obstacle, we introduce the Multi-Domain\nWizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human\nwritten conversations spanning over multiple domains and topics. At a size of\n$10$k dialogues, it is at least one order of magnitude larger than all previous\nannotated task-oriented corpora. The contribution of this work apart from the\nopen-sourced dataset labelled with dialogue belief states and dialogue actions\nis two-fold: firstly, a detailed description of the data collection procedure\nalong with a summary of data structure and analysis is provided. The proposed\ndata-collection pipeline is entirely based on crowd-sourcing without the need\nof hiring professional annotators; secondly, a set of benchmark results of\nbelief tracking, dialogue act and response generation is reported, which shows\nthe usability of the data and sets a baseline for future studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00333v1",
            "date": "2018-09-30 07:49:22+00:00",
            "title": "Specificity measures and reference",
            "authors": [
                "Albert Gatt",
                "Nicol\u00e1s Mar\u00edn",
                "Gustavo Rivas-Gervilla",
                "Daniel S\u00e1nchez"
            ],
            "abstract": "In this paper we study empirically the validity of measures of referential\nsuccess for referring expressions involving gradual properties. More\nspecifically, we study the ability of several measures of referential success\nto predict the success of a user in choosing the right object, given a\nreferring expression. Experimental results indicate that certain fuzzy measures\nof success are able to predict human accuracy in reference resolution. Such\nmeasures are therefore suitable for the estimation of the success or otherwise\nof a referring expression produced by a generation algorithm, especially in\ncase the properties in a domain cannot be assumed to have crisp denotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00341v1",
            "date": "2018-09-30 09:04:54+00:00",
            "title": "Text Morphing",
            "authors": [
                "Shaohan Huang",
                "Yu Wu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we introduce a novel natural language generation task, termed\nas text morphing, which targets at generating the intermediate sentences that\nare fluency and smooth with the two input sentences. We propose the Morphing\nNetworks consisting of the editing vector generation networks and the sentence\nediting networks which are trained jointly. Specifically, the editing vectors\nare generated with a recurrent neural networks model from the lexical gap\nbetween the source sentence and the target sentence. Then the sentence editing\nnetworks iteratively generate new sentences with the current editing vector and\nthe sentence generated in the previous step. We conduct experiments with 10\nmillion text morphing sequences which are extracted from the Yelp review\ndataset. Experiment results show that the proposed method outperforms baselines\non the text morphing task. We also discuss directions and opportunities for\nfuture research of text morphing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00367v2",
            "date": "2018-09-30 12:52:03+00:00",
            "title": "An Incremental Iterated Response Model of Pragmatics",
            "authors": [
                "Reuben Cohn-Gordon",
                "Noah D. Goodman",
                "Christopher Potts"
            ],
            "abstract": "Recent Iterated Response (IR) models of pragmatics conceptualize language use\nas a recursive process in which agents reason about each other to increase\ncommunicative efficiency. These models are generally defined over complete\nutterances. However, there is substantial evidence that pragmatic reasoning\ntakes place incrementally during production and comprehension. We address this\nwith an incremental IR model. We compare the incremental and global versions\nusing computational simulations, and we assess the incremental model against\nexisting experimental data and in the TUNA corpus for referring expression\ngeneration, showing that the model can capture phenomena out of reach of global\nversions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00438v2",
            "date": "2018-09-30 18:26:30+00:00",
            "title": "Parameter-free Sentence Embedding via Orthogonal Basis",
            "authors": [
                "Ziyi Yang",
                "Chenguang Zhu",
                "Weizhu Chen"
            ],
            "abstract": "We propose a simple and robust non-parameterized approach for building\nsentence representations. Inspired by the Gram-Schmidt Process in geometric\ntheory, we build an orthogonal basis of the subspace spanned by a word and its\nsurrounding context in a sentence. We model the semantic meaning of a word in a\nsentence based on two aspects. One is its relatedness to the word vector\nsubspace already spanned by its contextual words. The other is the word's novel\nsemantic meaning which shall be introduced as a new basis vector perpendicular\nto this existing subspace. Following this motivation, we develop an innovative\nmethod based on orthogonal basis to combine pre-trained word embeddings into\nsentence representations. This approach requires zero parameters, along with\nefficient inference performance. We evaluate our approach on 11 downstream NLP\ntasks. Our model shows superior performance compared with non-parameterized\nalternatives and it is competitive to other approaches relying on either large\namounts of labelled data or prolonged training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00472v1",
            "date": "2018-09-30 21:53:47+00:00",
            "title": "Automatic Evaluation of Neural Personality-based Chatbots",
            "authors": [
                "Yujie Xing",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "Stylistic variation is critical to render the utterances generated by\nconversational agents natural and engaging. In this paper, we focus on\nsequence-to-sequence models for open-domain dialogue response generation and\npropose a new method to evaluate the extent to which such models are able to\ngenerate responses that reflect different personality traits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00593v1",
            "date": "2018-10-01 09:26:43+00:00",
            "title": "Detecting Satire in the News with Machine Learning",
            "authors": [
                "Andreas St\u00f6ckl"
            ],
            "abstract": "We built models with Logistic Regression and linear Support Vector Machines\non a large dataset consisting of regular news articles and news from satirical\nwebsites, and showed that such linear classifiers on a corpus with about 60,000\narticles can perform with a precision of 98.7% and a recall of 95.2% on a\nrandom test set of the news. On the other hand, when testing the classifier on\n\"publication sources\" which are completely unknown during training, only an\naccuracy of 88.2% and an F1-score of 76.3% are achieved. As another result, we\nshowed that the same algorithm can distinguish between news written by the news\nagency itself and paid articles from customers. Here the results had an\naccuracy of 99%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00662v1",
            "date": "2018-09-24 03:39:47+00:00",
            "title": "Hindi-English Code-Switching Speech Corpus",
            "authors": [
                "Ganji Sreeram",
                "Kunal Dhawan",
                "Rohit Sinha"
            ],
            "abstract": "Code-switching refers to the usage of two languages within a sentence or\ndiscourse. It is a global phenomenon among multilingual communities and has\nemerged as an independent area of research. With the increasing demand for the\ncode-switching automatic speech recognition (ASR) systems, the development of a\ncode-switching speech corpus has become highly desirable. However, for training\nsuch systems, very limited code-switched resources are available as yet. In\nthis work, we present our first efforts in building a code-switching ASR system\nin the Indian context. For that purpose, we have created a Hindi-English\ncode-switching speech database. The database not only contains the speech\nutterances with code-switching properties but also covers the session and the\nspeaker variations like pronunciation, accent, age, gender, etc. This database\ncan be applied in several speech signal processing applications, such as\ncode-switching ASR, language identification, language modeling, speech\nsynthesis etc. This paper mainly presents an analysis of the statistics of the\ncollected code-switching speech corpus. Later, the performance results for the\nASR task have been reported for the created database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00670v1",
            "date": "2018-09-27 18:49:29+00:00",
            "title": "Automatic Data Expansion for Customer-care Spoken Language Understanding",
            "authors": [
                "Shahab Jalalvand",
                "Andrej Ljolje",
                "Srinivas Bangalore"
            ],
            "abstract": "Spoken language understanding (SLU) systems are widely used in handling of\ncustomer-care calls.A traditional SLU system consists of an acoustic model (AM)\nand a language model (LM) that areused to decode the utterance and a natural\nlanguage understanding (NLU) model that predicts theintent. While AM can be\nshared across different domains, LM and NLU models need to be\ntrainedspecifically for every new task. However, preparing enough data to train\nthese models is prohibitivelyexpensive. In this paper, we introduce an\nefficient method to expand the limited in-domain data. Theprocess starts with\ntraining a preliminary NLU model based on logistic regression on the\nin-domaindata. Since the features are based onn= 1,2-grams, we can detect the\nmost informative n-gramsfor each intent class. Using these n-grams, we find the\nsamples in the out-of-domain corpus that1) contain the desired n-gram and/or 2)\nhave similar intent label. The ones which meet the firstconstraint are used to\ntrain a new LM model and the ones that meet both constraints are used to train\nanew NLU model. Our results on two divergent experimental setups show that the\nproposed approachreduces by 30% the absolute classification error rate (CER)\ncomparing to the preliminary modelsand it significantly outperforms the\ntraditional data expansion algorithms such as the ones based onsemi-supervised\nlearning, TF-IDF and embedding vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00681v1",
            "date": "2018-09-28 05:40:20+00:00",
            "title": "Learning Robust, Transferable Sentence Representations for Text Classification",
            "authors": [
                "Wasi Uddin Ahmad",
                "Xueying Bai",
                "Nanyun Peng",
                "Kai-Wei Chang"
            ],
            "abstract": "Despite deep recurrent neural networks (RNNs) demonstrate strong performance\nin text classification, training RNN models are often expensive and requires an\nextensive collection of annotated data which may not be available. To overcome\nthe data limitation issue, existing approaches leverage either pre-trained word\nembedding or sentence representation to lift the burden of training RNNs from\nscratch. In this paper, we show that jointly learning sentence representations\nfrom multiple text classification tasks and combining them with pre-trained\nword-level and sentence level encoders result in robust sentence\nrepresentations that are useful for transfer learning. Extensive experiments\nand analyses using a wide range of transfer and linguistic tasks endorse the\neffectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00967v1",
            "date": "2018-10-01 20:40:15+00:00",
            "title": "Efficient and Accurate Abnormality Mining from Radiology Reports with Customized False Positive Reduction",
            "authors": [
                "Nithya Attaluri",
                "Ahmed Nasir",
                "Carolynne Powe",
                "Harold Racz",
                "Ben Covington",
                "Li Yao",
                "Jordan Prosky",
                "Eric Poblenz",
                "Tobi Olatunji",
                "Kevin Lyman"
            ],
            "abstract": "Obtaining datasets labeled to facilitate model development is a challenge for\nmost machine learning tasks. The difficulty is heightened for medical imaging,\nwhere data itself is limited in accessibility and labeling requires costly time\nand effort by trained medical specialists. Medical imaging studies, however,\nare often accompanied by a medical report produced by a radiologist,\nidentifying important features on the corresponding scan for other physicians\nnot specifically trained in radiology. We propose a methodology for\napproximating image-level labels for radiology studies from associated reports\nusing a general purpose language processing tool for medical concept extraction\nand sentiment analysis, and simple manually crafted heuristics for false\npositive reduction. Using this approach, we label more than 175,000 Head CT\nstudies for the presence of 33 features indicative of 11 clinically relevant\nconditions. For 27 of the 30 keywords that yielded positive results (3 had no\noccurrences), the lower bound of the confidence intervals created to estimate\nthe percentage of accurately labeled reports was above 85%, with the average\nbeing above 95%. Though noisier then manual labeling, these results suggest\nthis method to be a viable means of labeling medical images at scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01170v1",
            "date": "2018-10-02 11:06:35+00:00",
            "title": "Findings of the E2E NLG Challenge",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Jekaterina Novikova",
                "Verena Rieser"
            ],
            "abstract": "This paper summarises the experimental setup and results of the first shared\ntask on end-to-end (E2E) natural language generation (NLG) in spoken dialogue\nsystems. Recent end-to-end generation systems are promising since they reduce\nthe need for data annotation. However, they are currently limited to small,\ndelexicalised datasets. The E2E NLG shared task aims to assess whether these\nnovel approaches can generate better-quality output by learning from a dataset\ncontaining higher lexical richness, syntactic complexity and diverse discourse\nphenomena. We compare 62 systems submitted by 17 institutions, covering a wide\nrange of approaches, including machine learning architectures -- with the\nmajority implementing sequence-to-sequence models (seq2seq) -- as well as\nsystems based on grammatical rules and templates.",
            "journal ref": "Proceedings of the 11th International Conference on Natural\n  Language Generation, pages 322-328, Tilburg, The Netherlands, November 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01375v1",
            "date": "2018-10-02 17:12:44+00:00",
            "title": "A Knowledge Hunting Framework for Common Sense Reasoning",
            "authors": [
                "Ali Emami",
                "Noelia De La Cruz",
                "Adam Trischler",
                "Kaheer Suleman",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "We introduce an automatic system that achieves state-of-the-art results on\nthe Winograd Schema Challenge (WSC), a common sense reasoning task that\nrequires diverse, complex forms of inference and knowledge. Our method uses a\nknowledge hunting module to gather text from the web, which serves as evidence\nfor candidate problem resolutions. Given an input problem, our system generates\nrelevant queries to send to a search engine, then extracts and classifies\nknowledge from the returned results and weighs them to make a resolution. Our\napproach improves F1 performance on the full WSC by 0.21 over the previous best\nand represents the first system to exceed 0.5 F1. We further demonstrate that\nthe approach is competitive on the Choice of Plausible Alternatives (COPA)\ntask, which suggests that it is generally applicable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01570v1",
            "date": "2018-10-03 02:53:04+00:00",
            "title": "A Deep Learning Architecture for De-identification of Patient Notes: Implementation and Evaluation",
            "authors": [
                "Kaung Khin",
                "Philipp Burckhardt",
                "Rema Padman"
            ],
            "abstract": "De-identification is the process of removing 18 protected health information\n(PHI) from clinical notes in order for the text to be considered not\nindividually identifiable. Recent advances in natural language processing (NLP)\nhas allowed for the use of deep learning techniques for the task of\nde-identification. In this paper, we present a deep learning architecture that\nbuilds on the latest NLP advances by incorporating deep contextualized word\nembeddings and variational drop out Bi-LSTMs. We test this architecture on two\ngold standard datasets and show that the architecture achieves state-of-the-art\nperformance on both data sets while also converging faster than other systems\nwithout the use of dictionaries or other knowledge sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01656v1",
            "date": "2018-10-03 09:36:20+00:00",
            "title": "A Comparative Study of Neural Network Models for Sentence Classification",
            "authors": [
                "Phuong Le-Hong",
                "Anh-Cuong Le"
            ],
            "abstract": "This paper presents an extensive comparative study of four neural network\nmodels, including feed-forward networks, convolutional networks, recurrent\nnetworks and long short-term memory networks, on two sentence classification\ndatasets of English and Vietnamese text. We show that on the English dataset,\nthe convolutional network models without any feature engineering outperform\nsome competitive sentence classifiers with rich hand-crafted linguistic\nfeatures. We demonstrate that the GloVe word embeddings are consistently better\nthan both Skip-gram word embeddings and word count vectors. We also show the\nsuperiority of convolutional neural network models on a Vietnamese newspaper\nsentence dataset over strong baseline models. Our experimental results suggest\nsome good practices for applying neural network models in sentence\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01808v1",
            "date": "2018-10-03 15:53:37+00:00",
            "title": "A Neural Transition-based Model for Nested Mention Recognition",
            "authors": [
                "Bailin Wang",
                "Wei Lu",
                "Yu Wang",
                "Hongxia Jin"
            ],
            "abstract": "It is common that entity mentions can contain other mentions recursively.\nThis paper introduces a scalable transition-based method to model the nested\nstructure of mentions. We first map a sentence with nested mentions to a\ndesignated forest where each mention corresponds to a constituent of the\nforest. Our shift-reduce based system then learns to construct the forest\nstructure in a bottom-up manner through an action sequence whose maximal length\nis guaranteed to be three times of the sentence length. Based on Stack-LSTM\nwhich is employed to efficiently and effectively represent the states of the\nsystem in a continuous space, our system is further incorporated with a\ncharacter-based component to capture letter-level patterns. Our model achieves\nthe state-of-the-art results on ACE datasets, showing its effectiveness in\ndetecting nested mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01817v1",
            "date": "2018-10-03 16:13:26+00:00",
            "title": "Neural Segmental Hypergraphs for Overlapping Mention Recognition",
            "authors": [
                "Bailin Wang",
                "Wei Lu"
            ],
            "abstract": "In this work, we propose a novel segmental hypergraph representation to model\noverlapping entity mentions that are prevalent in many practical datasets. We\nshow that our model built on top of such a new representation is able to\ncapture features and interactions that cannot be captured by previous models\nwhile maintaining a low time complexity for inference. We also present a\ntheoretical analysis to formally assess how our representation is better than\nalternative representations reported in the literature in terms of\nrepresentational power. Coupled with neural networks for feature learning, our\nmodel achieves the state-of-the-art performance in three benchmark datasets\nannotated with overlapping mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01912v1",
            "date": "2018-10-03 18:45:40+00:00",
            "title": "Fast Approach to Build an Automatic Sentiment Annotator for Legal Domain using Transfer Learning",
            "authors": [
                "Viraj Gamage",
                "Menuka Warushavithana",
                "Nisansa de Silva",
                "Amal Shehan Perera",
                "Gathika Ratnayaka",
                "Thejan Rupasinghe"
            ],
            "abstract": "This study proposes a novel way of identifying the sentiment of the phrases\nused in the legal domain. The added complexity of the language used in law, and\nthe inability of the existing systems to accurately predict the sentiments of\nwords in law are the main motivations behind this study. This is a transfer\nlearning approach, which can be used for other domain adaptation tasks as well.\nThe proposed methodology achieves an improvement of over 6\\% compared to the\nsource model's accuracy in the legal domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02100v1",
            "date": "2018-10-04 08:41:50+00:00",
            "title": "Semi-Supervised Methods for Out-of-Domain Dependency Parsing",
            "authors": [
                "Juntao Yu"
            ],
            "abstract": "Dependency parsing is one of the important natural language processing tasks\nthat assigns syntactic trees to texts. Due to the wider availability of\ndependency corpora and improved parsing and machine learning techniques,\nparsing accuracies of supervised learning-based systems have been significantly\nimproved. However, due to the nature of supervised learning, those parsing\nsystems highly rely on the manually annotated training corpora. They work\nreasonably good on the in-domain data but the performance drops significantly\nwhen tested on out-of-domain texts. To bridge the performance gap between\nin-domain and out-of-domain, this thesis investigates three semi-supervised\ntechniques for out-of-domain dependency parsing, namely co-training,\nself-training and dependency language models. Our approaches use easily\nobtainable unlabelled data to improve out-of-domain parsing accuracies without\nthe need of expensive corpora annotation. The evaluations on several English\ndomains and multi-lingual data show quite good improvements on parsing\naccuracy. Overall this work conducted a survey of semi-supervised methods for\nout-of-domain dependency parsing, where I extended and compared a number of\nimportant semi-supervised methods in a unified framework. The comparison\nbetween those techniques shows that self-training works equally well as\nco-training on out-of-domain parsing, while dependency language models can\nimprove both in- and out-of-domain accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02114v1",
            "date": "2018-10-04 09:20:32+00:00",
            "title": "Zooming Network",
            "authors": [
                "Yukun Yan",
                "Daqi Zheng",
                "Zhengdong Lu",
                "Sen Song"
            ],
            "abstract": "Structural information is important in natural language understanding.\nAlthough some current neural net-based models have a limited ability to take\nlocal syntactic information, they fail to use high-level and large-scale\nstructures of documents. This information is valuable for text understanding\nsince it contains the author's strategy to express information, in building an\neffective representation and forming appropriate output modes. We propose a\nneural net-based model, Zooming Network, capable of representing and leveraging\ntext structure of long document and developing its own analyzing rhythm to\nextract critical information. Generally, ZN consists of an encoding neural net\nthat can build a hierarchical representation of a document, and an interpreting\nneural model that can read the information at multi-levels and issuing labeling\nactions through a policy-net. Our model is trained with a hybrid paradigm of\nsupervised learning (distinguishing right and wrong decision) and reinforcement\nlearning (determining the goodness among multiple right paths). We applied the\nproposed model to long text sequence labeling tasks, with performance exceeding\nbaseline model (biLSTM-crf) by 10 F1-measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02156v1",
            "date": "2018-10-04 11:51:47+00:00",
            "title": "Neural Networks for Cross-lingual Negation Scope Detection",
            "authors": [
                "Federico Fancellu",
                "Adam Lopez",
                "Bonnie Webber"
            ],
            "abstract": "Negation scope has been annotated in several English and Chinese corpora, and\nhighly accurate models for this task in these languages have been learned from\nthese annotations. Unfortunately, annotations are not available in other\nlanguages. Could a model that detects negation scope be applied to a language\nthat it hasn't been trained on? We develop neural models that learn from\ncross-lingual word embeddings or universal dependencies in English, and test\nthem on Chinese, showing that they work surprisingly well. We find that\nmodelling syntax is helpful even in monolingual settings and that cross-lingual\nword embeddings help relatively little, and we analyse cases that are still\ndifficult for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02245v1",
            "date": "2018-10-04 14:34:02+00:00",
            "title": "A Span Selection Model for Semantic Role Labeling",
            "authors": [
                "Hiroki Ouchi",
                "Hiroyuki Shindo",
                "Yuji Matsumoto"
            ],
            "abstract": "We present a simple and accurate span-based model for semantic role labeling\n(SRL). Our model directly takes into account all possible argument spans and\nscores them for each label. At decoding time, we greedily select higher scoring\nlabeled spans. One advantage of our model is to allow us to design and use\nspan-level features, that are difficult to use in token-based BIO tagging\napproaches. Experimental results demonstrate that our ensemble model achieves\nthe state-of-the-art results, 87.4 F1 and 87.0 F1 on the CoNLL-2005 and 2012\ndatasets, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02268v3",
            "date": "2018-10-04 15:06:27+00:00",
            "title": "A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation",
            "authors": [
                "Mathias M\u00fcller",
                "Annette Rios",
                "Elena Voita",
                "Rico Sennrich"
            ],
            "abstract": "The translation of pronouns presents a special challenge to machine\ntranslation to this day, since it often requires context outside the current\nsentence. Recent work on models that have access to information across sentence\nboundaries has seen only moderate improvements in terms of automatic evaluation\nmetrics such as BLEU. However, metrics that quantify the overall translation\nquality are ill-equipped to measure gains from additional context. We argue\nthat a different kind of evaluation is needed to assess how well models\ntranslate inter-sentential phenomena such as pronouns. This paper therefore\npresents a test suite of contrastive translations focused specifically on the\ntranslation of pronouns. Furthermore, we perform experiments with several\ncontext-aware models. We show that, while gains in BLEU are moderate for those\nsystems, they outperform baselines by a large margin in terms of accuracy on\nour contrastive test set. Our experiments also show the effectiveness of\nparameter tying for multi-encoder architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02508v6",
            "date": "2018-10-05 03:50:24+00:00",
            "title": "MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations",
            "authors": [
                "Soujanya Poria",
                "Devamanyu Hazarika",
                "Navonil Majumder",
                "Gautam Naik",
                "Erik Cambria",
                "Rada Mihalcea"
            ],
            "abstract": "Emotion recognition in conversations is a challenging task that has recently\ngained popularity due to its potential applications. Until now, however, a\nlarge-scale multimodal multi-party emotional conversational database containing\nmore than two speakers per dialogue was missing. Thus, we propose the\nMultimodal EmotionLines Dataset (MELD), an extension and enhancement of\nEmotionLines. MELD contains about 13,000 utterances from 1,433 dialogues from\nthe TV-series Friends. Each utterance is annotated with emotion and sentiment\nlabels, and encompasses audio, visual and textual modalities. We propose\nseveral strong multimodal baselines and show the importance of contextual and\nmultimodal information for emotion recognition in conversations. The full\ndataset is available for use at http:// affective-meld.github.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02614v1",
            "date": "2018-10-05 11:20:39+00:00",
            "title": "Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation",
            "authors": [
                "Xiao Pu",
                "Nikolaos Pappas",
                "James Henderson",
                "Andrei Popescu-Belis"
            ],
            "abstract": "This paper demonstrates that word sense disambiguation (WSD) can improve\nneural machine translation (NMT) by widening the source context considered when\nmodeling the senses of potentially ambiguous words. We first introduce three\nadaptive clustering algorithms for WSD, based on k-means, Chinese restaurant\nprocesses, and random walks, which are then applied to large word contexts\nrepresented in a low-rank space and evaluated on SemEval shared-task data. We\nthen learn word vectors jointly with sense vectors defined by our best WSD\nmethod, within a state-of-the-art NMT system. We show that the concatenation of\nthese vectors, and the use of a sense selection mechanism based on the weighted\naverage of sense vectors, outperforms several baselines including sense-aware\nones. This is demonstrated by translation on five language pairs. The\nimprovements are above one BLEU point over strong NMT baselines, +4% accuracy\nover all ambiguous nouns and verbs, or +20% when scored manually over several\nchallenging words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02741v1",
            "date": "2018-10-05 15:11:32+00:00",
            "title": "From direct tagging to Tagging with sentences compression",
            "authors": [
                "Peihui Chen"
            ],
            "abstract": "In essence, the two tagging methods (direct tagging and tagging with\nsentences compression) are to tag the information we need by using regular\nexpression which basing on the inherent language patterns of the natural\nlanguage. Though it has many advantages in extracting regular data, Direct\ntagging is not applicable to some situations. if the data we need extract is\nnot regular and its surrounding words are regular is relatively regular, then\nwe can use information compression to cut the information we do not need before\nwe tagging the data we need. In this way we can increase the precision of the\ndata while not undermine the recall of the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02851v1",
            "date": "2018-10-05 18:58:35+00:00",
            "title": "Learning to Encode Text as Human-Readable Summaries using Generative Adversarial Networks",
            "authors": [
                "Yau-Shian Wang",
                "Hung-Yi Lee"
            ],
            "abstract": "Auto-encoders compress input data into a latent-space representation and\nreconstruct the original data from the representation. This latent\nrepresentation is not easily interpreted by humans. In this paper, we propose\ntraining an auto-encoder that encodes input text into human-readable sentences,\nand unpaired abstractive summarization is thereby achieved. The auto-encoder is\ncomposed of a generator and a reconstructor. The generator encodes the input\ntext into a shorter word sequence, and the reconstructor recovers the generator\ninput from the generator output. To make the generator output human-readable, a\ndiscriminator restricts the output of the generator to resemble human-written\nsentences. By taking the generator output as the summary of the input text,\nabstractive summarization is achieved without document-summary pairs as\ntraining data. Promising results are shown on both English and Chinese corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02889v3",
            "date": "2018-10-05 21:07:11+00:00",
            "title": "Scalable Micro-planned Generation of Discourse from Structured Data",
            "authors": [
                "Anirban Laha",
                "Parag Jain",
                "Abhijit Mishra",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "We present a framework for generating natural language description from\nstructured data such as tables; the problem comes under the category of\ndata-to-text natural language generation (NLG). Modern data-to-text NLG systems\ntypically employ end-to-end statistical and neural architectures that learn\nfrom a limited amount of task-specific labeled data, and therefore, exhibit\nlimited scalability, domain-adaptability, and interpretability. Unlike these\nsystems, ours is a modular, pipeline-based approach, and does not require\ntask-specific parallel data. It rather relies on monolingual corpora and basic\noff-the-shelf NLP tools. This makes our system more scalable and easily\nadaptable to newer domains.\n  Our system employs a 3-staged pipeline that: (i) converts entries in the\nstructured data to canonical form, (ii) generates simple sentences for each\natomic entry in the canonicalized representation, and (iii) combines the\nsentences to produce a coherent, fluent and adequate paragraph description\nthrough sentence compounding and co-reference replacement modules. Experiments\non a benchmark mixed-domain dataset curated for paragraph description from\ntables reveals the superiority of our system over existing data-to-text\napproaches. We also demonstrate the robustness of our system in accepting other\npopular datasets covering diverse data types such as Knowledge Graphs and\nKey-Value maps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02891v1",
            "date": "2018-10-05 21:20:25+00:00",
            "title": "Entity Tracking Improves Cloze-style Reading Comprehension",
            "authors": [
                "Luong Hoang",
                "Sam Wiseman",
                "Alexander M. Rush"
            ],
            "abstract": "Reading comprehension tasks test the ability of models to process long-term\ncontext and remember salient information. Recent work has shown that relatively\nsimple neural methods such as the Attention Sum-Reader can perform well on\nthese tasks; however, these systems still significantly trail human\nperformance. Analysis suggests that many of the remaining hard instances are\nrelated to the inability to track entity-references throughout documents. This\nwork focuses on these hard entity tracking cases with two extensions: (1)\nadditional entity features, and (2) training with a multi-task tracking\nobjective. We show that these simple modifications improve performance both\nindependently and in combination, and we outperform the previous state of the\nart on the LAMBADA dataset, particularly on difficult entity examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03148v1",
            "date": "2018-10-07 14:14:27+00:00",
            "title": "Assessing Crosslingual Discourse Relations in Machine Translation",
            "authors": [
                "Karin Sim Smith",
                "Lucia Specia"
            ],
            "abstract": "In an attempt to improve overall translation quality, there has been an\nincreasing focus on integrating more linguistic elements into Machine\nTranslation (MT). While significant progress has been achieved, especially\nrecently with neural models, automatically evaluating the output of such\nsystems is still an open problem. Current practice in MT evaluation relies on a\nsingle reference translation, even though there are many ways of translating a\nparticular text, and it tends to disregard higher level information such as\ndiscourse. We propose a novel approach that assesses the translated output\nbased on the source text rather than the reference translation, and measures\nthe extent to which the semantics of the discourse elements (discourse\nrelations, in particular) in the source text are preserved in the MT output.\nThe challenge is to detect the discourse relations in the source text and\ndetermine whether these relations are correctly transferred crosslingually to\nthe target language -- without a reference translation. This methodology could\nbe used independently for discourse-level evaluation, or as a component in\nother metrics, at a time where substantial amounts of MT are online and would\nbenefit from evaluation where the source text serves as a benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03184v1",
            "date": "2018-10-07 17:32:11+00:00",
            "title": "Phonology-Augmented Statistical Framework for Machine Transliteration using Limited Linguistic Resources",
            "authors": [
                "Gia H. Ngo",
                "Minh Nguyen",
                "Nancy F. Chen"
            ],
            "abstract": "Transliteration converts words in a source language (e.g., English) into\nwords in a target language (e.g., Vietnamese). This conversion considers the\nphonological structure of the target language, as the transliterated output\nneeds to be pronounceable in the target language. For example, a word in\nVietnamese that begins with a consonant cluster is phonologically invalid and\nthus would be an incorrect output of a transliteration system. Most statistical\ntransliteration approaches, albeit being widely adopted, do not explicitly\nmodel the target language's phonology, which often results in invalid outputs.\nThe problem is compounded by the limited linguistic resources available when\nconverting foreign words to transliterated words in the target language. In\nthis work, we present a phonology-augmented statistical framework suitable for\ntransliteration, especially when only limited linguistic resources are\navailable. We propose the concept of pseudo-syllables as structures\nrepresenting how segments of a foreign word are organized according to the\nsyllables of the target language's phonology. We performed transliteration\nexperiments on Vietnamese and Cantonese. We show that the proposed framework\noutperforms the statistical baseline by up to 44.68% relative, when there are\nlimited training examples (587 entries).",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech and Language Processing.\n  27(2019) 199-211",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03274v1",
            "date": "2018-10-08 05:27:44+00:00",
            "title": "Query Tracking for E-commerce Conversational Search: A Machine Comprehension Perspective",
            "authors": [
                "Yunlun Yang",
                "Yu Gong",
                "Xi Chen"
            ],
            "abstract": "With the development of dialog techniques, conversational search has\nattracted more and more attention as it enables users to interact with the\nsearch engine in a natural and efficient manner. However, comparing with the\nnatural language understanding in traditional task-oriented dialog which\nfocuses on slot filling and tracking, the query understanding in E-commerce\nconversational search is quite different and more challenging due to more\ndiverse user expressions and complex intentions. In this work, we define the\nreal-world problem of query tracking in E-commerce conversational search, in\nwhich the goal is to update the internal query after each round of interaction.\nWe also propose a self attention based neural network to handle the task in a\nmachine comprehension perspective. Further more we build a novel E-commerce\nquery tracking dataset from an operational E-commerce Search Engine, and\nexperimental results on this dataset suggest that our proposed model\noutperforms several baseline methods by a substantial gain for Exact Match\naccuracy and F1 score, showing the potential of machine comprehension like\nmodel for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03352v1",
            "date": "2018-10-08 09:57:44+00:00",
            "title": "Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems",
            "authors": [
                "Igor Shalyminov",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "Spontaneous spoken dialogue is often disfluent, containing pauses,\nhesitations, self-corrections and false starts. Processing such phenomena is\nessential in understanding a speaker's intended meaning and controlling the\nflow of the conversation. Furthermore, this processing needs to be word-by-word\nincremental to allow further downstream processing to begin as early as\npossible in order to handle real spontaneous human conversational behaviour.\n  In addition, from a developer's point of view, it is highly desirable to be\nable to develop systems which can be trained from `clean' examples while also\nable to generalise to the very diverse disfluent variations on the same data --\nthereby enhancing both data-efficiency and robustness. In this paper, we\npresent a multi-task LSTM-based model for incremental detection of disfluency\nstructure, which can be hooked up to any component for incremental\ninterpretation (e.g. an incremental semantic parser), or else simply used to\n`clean up' the current utterance as it is being produced.\n  We train the system on the Switchboard Dialogue Acts (SWDA) corpus and\npresent its accuracy on this dataset. Our model outperforms prior neural\nnetwork-based incremental approaches by about 10 percentage points on SWDA\nwhile employing a simpler architecture. To test the model's generalisation\npotential, we evaluate the same model on the bAbI+ dataset, without any\nadditional training. bAbI+ is a dataset of synthesised goal-oriented dialogues\nwhere we control the distribution of disfluencies and their types. This shows\nthat our approach has good generalisation potential, and sheds more light on\nwhich types of disfluency might be amenable to domain-general processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03444v1",
            "date": "2018-09-30 16:28:40+00:00",
            "title": "Phrase-Based Attentions",
            "authors": [
                "Phi Xuan Nguyen",
                "Shafiq Joty"
            ],
            "abstract": "Most state-of-the-art neural machine translation systems, despite being\ndifferent in architectural skeletons (e.g. recurrence, convolutional), share an\nindispensable feature: the Attention. However, most existing attention methods\nare token-based and ignore the importance of phrasal alignments, the key\ningredient for the success of phrase-based statistical machine translation. In\nthis paper, we propose novel phrase-based attention methods to model n-grams of\ntokens as attention entities. We incorporate our phrase-based attentions into\nthe recently proposed Transformer network, and demonstrate that our approach\nyields improvements of 1.3 BLEU for English-to-German and 0.5 BLEU for\nGerman-to-English translation tasks on WMT newstest2014 using WMT'16 training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03541v1",
            "date": "2018-10-08 16:00:50+00:00",
            "title": "An AMR Aligner Tuned by Transition-based Parser",
            "authors": [
                "Yijia Liu",
                "Wanxiang Che",
                "Bo Zheng",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "In this paper, we propose a new rich resource enhanced AMR aligner which\nproduces multiple alignments and a new transition system for AMR parsing along\nwith its oracle parser. Our aligner is further tuned by our oracle parser via\npicking the alignment that leads to the highest-scored achievable AMR graph.\nExperimental results show that our aligner outperforms the rule-based aligner\nin previous work by achieving higher alignment F1 score and consistently\nimproving two open-sourced AMR parsers. Based on our aligner and transition\nsystem, we develop a transition-based AMR parser that parses a sentence into\nits AMR graph directly. An ensemble of our parsers with only words and POS tags\nas input leads to 68.4 Smatch F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03581v1",
            "date": "2018-10-08 17:09:10+00:00",
            "title": "Improving the Transformer Translation Model with Document-Level Context",
            "authors": [
                "Jiacheng Zhang",
                "Huanbo Luan",
                "Maosong Sun",
                "FeiFei Zhai",
                "Jingfang Xu",
                "Min Zhang",
                "Yang Liu"
            ],
            "abstract": "Although the Transformer translation model (Vaswani et al., 2017) has\nachieved state-of-the-art performance in a variety of translation tasks, how to\nuse document-level context to deal with discourse phenomena problematic for\nTransformer still remains a challenge. In this work, we extend the Transformer\nmodel with a new context encoder to represent document-level context, which is\nthen incorporated into the original encoder and decoder. As large-scale\ndocument-level parallel corpora are usually not available, we introduce a\ntwo-step training method to take full advantage of abundant sentence-level\nparallel corpora and limited document-level parallel corpora. Experiments on\nthe NIST Chinese-English datasets and the IWSLT French-English datasets show\nthat our approach improves over Transformer significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03595v2",
            "date": "2018-10-08 17:44:34+00:00",
            "title": "End-to-End Text Classification via Image-based Embedding using Character-level Networks",
            "authors": [
                "Shunsuke Kitada",
                "Ryunosuke Kotani",
                "Hitoshi Iyatomi"
            ],
            "abstract": "For analysing and/or understanding languages having no word boundaries based\non morphological analysis such as Japanese, Chinese, and Thai, it is desirable\nto perform appropriate word segmentation before word embeddings. But it is\ninherently difficult in these languages. In recent years, various language\nmodels based on deep learning have made remarkable progress, and some of these\nmethodologies utilizing character-level features have successfully avoided such\na difficult problem. However, when a model is fed character-level features of\nthe above languages, it often causes overfitting due to a large number of\ncharacter types. In this paper, we propose a CE-CLCNN, character-level\nconvolutional neural networks using a character encoder to tackle these\nproblems. The proposed CE-CLCNN is an end-to-end learning model and has an\nimage-based character encoder, i.e. the CE-CLCNN handles each character in the\ntarget document as an image. Through various experiments, we found and\nconfirmed that our CE-CLCNN captured closely embedded features for visually and\nsemantically similar characters and achieves state-of-the-art results on\nseveral open document classification tasks. In this paper we report the\nperformance of our CE-CLCNN with the Wikipedia title estimation task and\nanalyse the internal behaviour.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03717v1",
            "date": "2018-10-08 21:51:44+00:00",
            "title": "Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games",
            "authors": [
                "Judy Hanwen Shen",
                "Matthias Hofer",
                "Bjarke Felbo",
                "Roger Levy"
            ],
            "abstract": "Simple reference games are of central theoretical and empirical importance in\nthe study of situated language use. Although language provides rich,\ncompositional truth-conditional semantics to facilitate reference, speakers and\nlisteners may sometimes lack the overall lexical and cognitive resources to\nguarantee successful reference through these means alone. However, language\nalso has rich associational structures that can serve as a further resource for\nachieving successful reference. Here we investigate this use of associational\ninformation in a setting where only associational information is available: a\nsimplified version of the popular game Codenames. Using optimal experiment\ndesign techniques, we compare a range of models varying in the type of\nassociative information deployed and in level of pragmatic sophistication\nagainst human behavior. In this setting, we find that listeners' behavior\nreflects direct bigram collocational associations more strongly than\nword-embedding or semantic knowledge graph-based associations and that there is\nlittle evidence for pragmatically sophisticated behavior by either speakers or\nlisteners of the type that might be predicted by recursive-reasoning models\nsuch as the Rational Speech Acts theory. These results shed light on the nature\nof the lexical resources that speakers and listeners can bring to bear in\nachieving reference through associative meaning alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03875v1",
            "date": "2018-10-09 09:28:27+00:00",
            "title": "Towards Verifying Semantic Roles Co-occurrence",
            "authors": [
                "Aliaksandr Huminski",
                "Hao Zhang",
                "Gangeshwar Krishnamurthy"
            ],
            "abstract": "Semantic role theory considers roles as a small universal set of unanalyzed\nentities. It means that formally there are no restrictions on role\ncombinations. We argue that the semantic roles co-occur in verb\nrepresentations. It means that there are hidden restrictions on role\ncombinations. To demonstrate that a practical and evidence-based approach has\nbeen built on in-depth analysis of the largest verb database VerbNet. The\nconsequences of this approach are considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03879v1",
            "date": "2018-10-09 09:37:31+00:00",
            "title": "Event Representation through Semantic Roles: Evaluation of Coverage",
            "authors": [
                "Aliaksandr Huminski",
                "Hao Zhang"
            ],
            "abstract": "Semantic role theory is a widely used approach for event representation. Yet,\nthere are multiple indications that semantic role paradigm is necessary but not\nsufficient to cover all elements of event structure. We conducted an analysis\nof semantic role representation for events to provide an empirical evidence of\ninsufficiency. The consequence of that is a hybrid role-scalar approach. The\nresults are considered as preliminary in investigation of semantic roles\ncoverage for event representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03975v1",
            "date": "2018-10-09 13:42:58+00:00",
            "title": "Towards Two-Dimensional Sequence to Sequence Model in Neural Machine Translation",
            "authors": [
                "Parnia Bahar",
                "Christopher Brix",
                "Hermann Ney"
            ],
            "abstract": "This work investigates an alternative model for neural machine translation\n(NMT) and proposes a novel architecture, where we employ a multi-dimensional\nlong short-term memory (MDLSTM) for translation modeling. In the\nstate-of-the-art methods, source and target sentences are treated as\none-dimensional sequences over time, while we view translation as a\ntwo-dimensional (2D) mapping using an MDLSTM layer to define the correspondence\nbetween source and target words. We extend beyond the current sequence to\nsequence backbone NMT models to a 2D structure in which the source and target\nsentences are aligned with each other in a 2D grid. Our proposed topology shows\nconsistent improvements over attention-based sequence to sequence model on two\nWMT 2017 tasks, German$\\leftrightarrow$English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03996v1",
            "date": "2018-10-09 14:01:41+00:00",
            "title": "Learning Noun Cases Using Sequential Neural Networks",
            "authors": [
                "Sina Ahmadi"
            ],
            "abstract": "Morphological declension, which aims to inflect nouns to indicate number,\ncase and gender, is an important task in natural language processing (NLP).\nThis research proposal seeks to address the degree to which Recurrent Neural\nNetworks (RNNs) are efficient in learning to decline noun cases. Given the\nchallenge of data sparsity in processing morphologically rich languages and\nalso, the flexibility of sentence structures in such languages, we believe that\nmodeling morphological dependencies can improve the performance of neural\nnetwork models. It is suggested to carry out various experiments to understand\nthe interpretable features that may lead to a better generalization of the\nlearned models on cross-lingual tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04142v1",
            "date": "2018-10-09 17:21:41+00:00",
            "title": "A Fast, Compact, Accurate Model for Language Identification of Codemixed Text",
            "authors": [
                "Yuan Zhang",
                "Jason Riesa",
                "Daniel Gillick",
                "Anton Bakalov",
                "Jason Baldridge",
                "David Weiss"
            ],
            "abstract": "We address fine-grained multilingual language identification: providing a\nlanguage code for every token in a sentence, including codemixed text\ncontaining multiple languages. Such text is prevalent online, in documents,\nsocial media, and message boards. We show that a feed-forward network with a\nsimple globally constrained decoder can accurately and rapidly label both\ncodemixed and monolingual text in 100 languages and 100 language pairs. This\nmodel outperforms previously published multilingual approaches in terms of both\naccuracy and speed, yielding an 800x speed-up and a 19.5% averaged absolute\ngain on three codemixed datasets. It furthermore outperforms several benchmark\nsystems on monolingual language identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04216v1",
            "date": "2018-10-09 19:00:52+00:00",
            "title": "Event Coreference Resolution Using Neural Network Classifiers",
            "authors": [
                "Arun Pandian",
                "Lamana Mulaffer",
                "Kemal Oflazer",
                "Amna AlZeyara"
            ],
            "abstract": "This paper presents a neural network classifier approach to detecting both\nwithin- and cross- document event coreference effectively using only event\nmention based features. Our approach does not (yet) rely on any event argument\nfeatures such as semantic roles or spatiotemporal arguments. Experimental\nresults on the ECB+ dataset show that our approach produces F1 scores that\nsignificantly outperform the state-of-the-art methods for both within-document\nand cross-document event coreference resolution when we use B3 and CEAFe\nevaluation measures, but gets worse F1 score with the MUC measure. However,\nwhen we use the CoNLL measure, which is the average of these three scores, our\napproach has slightly better F1 for within- document event coreference\nresolution but is significantly better for cross-document event coreference\nresolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04297v3",
            "date": "2018-10-09 23:21:18+00:00",
            "title": "Decipherment of Historical Manuscript Images",
            "authors": [
                "Xusen Yin",
                "Nada Aldarrab",
                "Be\u00e1ta Megyesi",
                "Kevin Knight"
            ],
            "abstract": "European libraries and archives are filled with enciphered manuscripts from\nthe early modern period. These include military and diplomatic correspondence,\nrecords of secret societies, private letters, and so on. Although they are\nenciphered with classical cryptographic algorithms, their contents are\nunavailable to working historians. We therefore attack the problem of\nautomatically converting cipher manuscript images into plaintext. We develop\nunsupervised models for character segmentation, character-image clustering, and\ndecipherment of cluster sequences. We experiment with both pipelined and joint\nmodels, and we give empirical results for multiple ciphers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04428v1",
            "date": "2018-10-10 09:14:06+00:00",
            "title": "Improving Neural Text Simplification Model with Simplified Corpora",
            "authors": [
                "Jipeng Qiang"
            ],
            "abstract": "Text simplification (TS) can be viewed as monolingual translation task,\ntranslating between text variations within a single language. Recent neural TS\nmodels draw on insights from neural machine translation to learn lexical\nsimplification and content reduction using encoder-decoder model. But different\nfrom neural machine translation, we cannot obtain enough ordinary and\nsimplified sentence pairs for TS, which are expensive and time-consuming to\nbuild. Target-side simplified sentences plays an important role in boosting\nfluency for statistical TS, and we investigate the use of simplified sentences\nto train, with no changes to the network architecture. We propose to pair\nsimple training sentence with a synthetic ordinary sentence via\nback-translation, and treating this synthetic data as additional training data.\nWe train encoder-decoder model using synthetic sentence pairs and original\nsentence pairs, which can obtain substantial improvements on the available\nWikiLarge data and WikiSmall data compared with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04440v1",
            "date": "2018-10-10 10:00:17+00:00",
            "title": "New Vistas to study Bhartrhari: Cognitive NLP",
            "authors": [
                "Jayashree Gajjam",
                "Diptesh Kanojia",
                "Malhar Kulkarni"
            ],
            "abstract": "The Sanskrit grammatical tradition which has commenced with Panini's\nAstadhyayi mostly as a Padasastra has culminated as a Vakyasastra, at the hands\nof Bhartrhari. The grammarian-philosopher Bhartrhari and his authoritative work\n'Vakyapadiya' have been a matter of study for modern scholars, at least for\nmore than 50 years, since Ashok Aklujkar submitted his Ph.D. dissertation at\nHarvard University. The notions of a sentence and a word as a meaningful\nlinguistic unit in the language have been a subject matter for the discussion\nin many works that followed later on. While some scholars have applied\nphilological techniques to critically establish the text of the works of\nBhartrhari, some others have devoted themselves to exploring philosophical\ninsights from them. Some others have studied his works from the point of view\nof modern linguistics, and psychology. Few others have tried to justify the\nviews by logical discussions.\n  In this paper, we present a fresh view to study Bhartrhari, and his works,\nespecially the 'Vakyapadiya'. This view is from the field of Natural Language\nProcessing (NLP), more specifically, what is called as Cognitive NLP. We have\nstudied the definitions of a sentence given by Bhartrhari at the beginning of\nthe second chapter of 'Vakyapadiya'. We have researched one of these\ndefinitions by conducting an experiment and following the methodology of\nsilent-reading of Sanskrit paragraphs. We collect the Gaze-behavior data of\nparticipants and analyze it to understand the underlying comprehension\nprocedure in the human mind and present our results. We evaluate the\nstatistical significance of our results using T-test, and discuss the caveats\nof our work. We also present some general remarks on this experiment and\nusefulness of this method for gaining more insights in the work of Bhartrhari.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04631v3",
            "date": "2018-10-10 16:46:41+00:00",
            "title": "Extracting Arguments from Korean Question and Command: An Annotated Corpus for Structured Paraphrasing",
            "authors": [
                "Won Ik Cho",
                "Young Ki Moon",
                "Woo Hyun Kang",
                "Nam Soo Kim"
            ],
            "abstract": "Intention identification is a core issue in dialog management. However, due\nto the non-canonicality of the spoken language, it is difficult to extract the\ncontent automatically from the conversation-style utterances. This is much more\nchallenging for languages like Korean and Japanese since the agglutination\nbetween morphemes make it difficult for the machines to parse the sentence and\nunderstand the intention. To suggest a guideline for this problem, and to merge\nthe issue flexibly with the neural paraphrasing systems introduced recently, we\npropose a structured annotation scheme for Korean question/commands and the\nresulting corpus which are widely applicable to the field of argument mining.\nThe scheme and dataset are expected to help machines understand the intention\nof natural language and grasp the core meaning of conversation-style\ninstructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04635v1",
            "date": "2018-10-10 16:51:58+00:00",
            "title": "Multimodal Speech Emotion Recognition Using Audio and Text",
            "authors": [
                "Seunghyun Yoon",
                "Seokhyun Byun",
                "Kyomin Jung"
            ],
            "abstract": "Speech emotion recognition is a challenging task, and extensive reliance has\nbeen placed on models that use audio features in building well-performing\nclassifiers. In this paper, we propose a novel deep dual recurrent encoder\nmodel that utilizes text data and audio signals simultaneously to obtain a\nbetter understanding of speech data. As emotional dialogue is composed of sound\nand spoken content, our model encodes the information from audio and text\nsequences using dual recurrent neural networks (RNNs) and then combines the\ninformation from these sources to predict the emotion class. This architecture\nanalyzes speech data from the signal level to the language level, and it thus\nutilizes the information within the data more comprehensively than models that\nfocus on audio features. Extensive experiments are conducted to investigate the\nefficacy and properties of the proposed model. Our proposed model outperforms\nprevious state-of-the-art methods in assigning data to one of four emotion\ncategories (i.e., angry, happy, sad and neutral) when the model is applied to\nthe IEMOCAP dataset, as reflected by accuracies ranging from 68.8% to 71.8%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04805v2",
            "date": "2018-10-11 00:50:01+00:00",
            "title": "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding",
            "authors": [
                "Jacob Devlin",
                "Ming-Wei Chang",
                "Kenton Lee",
                "Kristina Toutanova"
            ],
            "abstract": "We introduce a new language representation model called BERT, which stands\nfor Bidirectional Encoder Representations from Transformers. Unlike recent\nlanguage representation models, BERT is designed to pre-train deep\nbidirectional representations from unlabeled text by jointly conditioning on\nboth left and right context in all layers. As a result, the pre-trained BERT\nmodel can be fine-tuned with just one additional output layer to create\nstate-of-the-art models for a wide range of tasks, such as question answering\nand language inference, without substantial task-specific architecture\nmodifications.\n  BERT is conceptually simple and empirically powerful. It obtains new\nstate-of-the-art results on eleven natural language processing tasks, including\npushing the GLUE score to 80.5% (7.7% point absolute improvement), MultiNLI\naccuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering\nTest F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1\n(5.1 point absolute improvement).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04839v1",
            "date": "2018-10-11 04:34:31+00:00",
            "title": "Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour",
            "authors": [
                "Sandeep Mathias",
                "Diptesh Kanojia",
                "Kevin Patel",
                "Samarth Agarwal",
                "Abhijit Mishra",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Predicting a reader's rating of text quality is a challenging task that\ninvolves estimating different subjective aspects of the text, like structure,\nclarity, etc. Such subjective aspects are better handled using cognitive\ninformation. One such source of cognitive information is gaze behaviour. In\nthis paper, we show that gaze behaviour does indeed help in effectively\npredicting the rating of text quality. To do this, we first model text quality\nas a function of three properties - organization, coherence and cohesion. Then,\nwe demonstrate how capturing gaze behaviour helps in predicting each of these\nproperties, and hence the overall quality, by reporting improvements obtained\nby adding gaze features to traditional textual features for score prediction.\nWe also hypothesize that if a reader has fully understood the text, the\ncorresponding gaze behaviour would give a better indication of the assigned\nrating, as opposed to partial understanding. Our experiments validate this\nhypothesis by showing greater agreement between the given rating and the\npredicted rating when the reader has a full understanding of the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04864v1",
            "date": "2018-10-11 06:43:28+00:00",
            "title": "Sequence-to-Sequence Models for Data-to-Text Natural Language Generation: Word- vs. Character-based Processing and Output Diversity",
            "authors": [
                "Glorianna Jagfeld",
                "Sabrina Jenne",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present a comparison of word-based and character-based\nsequence-to-sequence models for data-to-text natural language generation, which\ngenerate natural language descriptions for structured inputs. On the datasets\nof two recent generation challenges, our models achieve comparable or better\nautomatic evaluation results than the best challenge submissions. Subsequent\ndetailed statistical and human analyses shed light on the differences between\nthe two input representations and the diversity of the generated texts. In a\ncontrolled experiment with synthetic training data generated from templates, we\ndemonstrate the ability of neural models to learn novel combinations of the\ntemplates and thereby generalize beyond the linguistic structures they were\ntrained on.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04882v7",
            "date": "2018-10-11 08:08:40+00:00",
            "title": "Towards Understanding Linear Word Analogies",
            "authors": [
                "Kawin Ethayarajh",
                "David Duvenaud",
                "Graeme Hirst"
            ],
            "abstract": "A surprising property of word vectors is that word analogies can often be\nsolved with vector arithmetic. However, it is unclear why arithmetic operators\ncorrespond to non-linear embedding models such as skip-gram with negative\nsampling (SGNS). We provide a formal explanation of this phenomenon without\nmaking the strong assumptions that past theories have made about the vector\nspace and word distribution. Our theory has several implications. Past work has\nconjectured that linear substructures exist in vector spaces because relations\ncan be represented as ratios; we prove that this holds for SGNS. We provide\nnovel justification for the addition of SGNS word vectors by showing that it\nautomatically down-weights the more frequent word, as weighting schemes do ad\nhoc. Lastly, we offer an information theoretic interpretation of Euclidean\ndistance in vector spaces, justifying its use in capturing word dissimilarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05022v1",
            "date": "2018-10-11 13:54:50+00:00",
            "title": "Semantic Structural Evaluation for Text Simplification",
            "authors": [
                "Elior Sulem",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "Current measures for evaluating text simplification systems focus on\nevaluating lexical text aspects, neglecting its structural aspects. In this\npaper we propose the first measure to address structural aspects of text\nsimplification, called SAMSA. It leverages recent advances in semantic parsing\nto assess simplification quality by decomposing the input based on its semantic\nstructure and comparing it to the output. SAMSA provides a reference-less\nautomatic evaluation procedure, avoiding the problems that reference-based\nmethods face due to the vast space of valid simplifications for a given\nsentence. Our human evaluation experiments show both SAMSA's substantial\ncorrelation with human judgments, as well as the deficiency of existing\nreference-based measures in evaluating structural simplification.",
            "journal ref": "Proc. of NAACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05104v1",
            "date": "2018-10-11 16:14:24+00:00",
            "title": "Simple and Effective Text Simplification Using Semantic and Neural Methods",
            "authors": [
                "Elior Sulem",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "Sentence splitting is a major simplification operator. Here we present a\nsimple and efficient splitting algorithm based on an automatic semantic parser.\nAfter splitting, the text is amenable for further fine-tuned simplification\noperations. In particular, we show that neural Machine Translation can be\neffectively used in this situation. Previous application of Machine Translation\nfor simplification suffers from a considerable disadvantage in that they are\nover-conservative, often failing to modify the source in any way. Splitting\nbased on semantic parsing, as proposed here, alleviates this issue. Extensive\nautomatic and human evaluation shows that the proposed method compares\nfavorably to the state-of-the-art in combined lexical and structural\nsimplification.",
            "journal ref": "Proc. of ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05201v1",
            "date": "2018-10-11 18:52:42+00:00",
            "title": "Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns",
            "authors": [
                "Kellie Webster",
                "Marta Recasens",
                "Vera Axelrod",
                "Jason Baldridge"
            ],
            "abstract": "Coreference resolution is an important task for natural language\nunderstanding, and the resolution of ambiguous pronouns a longstanding\nchallenge. Nonetheless, existing corpora do not capture ambiguous pronouns in\nsufficient volume or diversity to accurately indicate the practical utility of\nmodels. Furthermore, we find gender bias in existing corpora and systems\nfavoring masculine entities. To address this, we present and release GAP, a\ngender-balanced labeled corpus of 8,908 ambiguous pronoun-name pairs sampled to\nprovide diverse coverage of challenges posed by real-world text. We explore a\nrange of baselines which demonstrate the complexity of the challenge, the best\nachieving just 66.9% F1. We show that syntactic structure and continuous neural\nmodels provide promising, complementary cues for approaching the challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05320v1",
            "date": "2018-10-12 02:19:42+00:00",
            "title": "Important Attribute Identification in Knowledge Graph",
            "authors": [
                "Shengjie Sun",
                "Dong Yang",
                "Hongchun Zhang",
                "Yanxu Chen",
                "Chao Wei",
                "Xiaonan Meng",
                "Yi Hu"
            ],
            "abstract": "The knowledge graph(KG) composed of entities with their descriptions and\nattributes, and relationship between entities, is finding more and more\napplication scenarios in various natural language processing tasks. In a\ntypical knowledge graph like Wikidata, entities usually have a large number of\nattributes, but it is difficult to know which ones are important. The\nimportance of attributes can be a valuable piece of information in various\napplications spanning from information retrieval to natural language\ngeneration. In this paper, we propose a general method of using external user\ngenerated text data to evaluate the relative importance of an entity's\nattributes. To be more specific, we use the word/sub-word embedding techniques\nto match the external textual data back to entities' attribute name and values\nand rank the attributes by their matching cohesiveness. To our best knowledge,\nthis is the first work of applying vector based semantic matching to important\nattribute identification, and our method outperforms the previous traditional\nmethods. We also apply the outcome of the detected important attributes to a\nlanguage generation task; compared with previous generated text, the new method\ngenerates much more customized and informative messages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05334v5",
            "date": "2018-10-12 03:07:21+00:00",
            "title": "IndoSum: A New Benchmark Dataset for Indonesian Text Summarization",
            "authors": [
                "Kemal Kurniawan",
                "Samuel Louvan"
            ],
            "abstract": "Automatic text summarization is generally considered as a challenging task in\nthe NLP community. One of the challenges is the publicly available and large\ndataset that is relatively rare and difficult to construct. The problem is even\nworse for low-resource languages such as Indonesian. In this paper, we present\nIndoSum, a new benchmark dataset for Indonesian text summarization. The dataset\nconsists of news articles and manually constructed summaries. Notably, the\ndataset is almost 200x larger than the previous Indonesian summarization\ndataset of the same domain. We evaluated various extractive summarization\napproaches and obtained encouraging results which demonstrate the usefulness of\nthe dataset and provide baselines for future research. The code and the dataset\nare available online under permissive licenses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05682v1",
            "date": "2018-10-12 19:01:32+00:00",
            "title": "Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension",
            "authors": [
                "Rajarshi Das",
                "Tsendsuren Munkhdalai",
                "Xingdi Yuan",
                "Adam Trischler",
                "Andrew McCallum"
            ],
            "abstract": "We propose a neural machine-reading model that constructs dynamic knowledge\ngraphs from procedural text. It builds these graphs recurrently for each step\nof the described procedure, and uses them to track the evolving states of\nparticipant entities. We harness and extend a recently proposed machine reading\ncomprehension (MRC) model to query for entity states, since these states are\ngenerally communicated in spans of text and MRC models perform well in\nextracting entity-centric spans. The explicit, structured, and evolving\nknowledge graph representations that our model constructs can be used in\ndownstream question answering tasks to improve machine comprehension of text,\nas we demonstrate empirically. On two comprehension tasks from the recently\nproposed PROPARA dataset (Dalvi et al., 2018), our model achieves\nstate-of-the-art results. We further show that our model is competitive on the\nRECIPES dataset (Kiddon et al., 2015), suggesting it may be generally\napplicable. We present some evidence that the model's knowledge graphs help it\nto impose commonsense constraints on its predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05739v4",
            "date": "2018-10-12 21:24:33+00:00",
            "title": "MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization",
            "authors": [
                "Eric Chu",
                "Peter J. Liu"
            ],
            "abstract": "Abstractive summarization has been studied using neural sequence transduction\nmethods with datasets of large, paired document-summary examples. However, such\ndatasets are rare and the models trained from them do not generalize to other\ndomains. Recently, some progress has been made in learning sequence-to-sequence\nmappings with only unpaired examples. In our work, we consider the setting\nwhere there are only documents (product or business reviews) with no summaries\nprovided, and propose an end-to-end, neural model architecture to perform\nunsupervised abstractive summarization. Our proposed model consists of an\nauto-encoder where the mean of the representations of the input reviews decodes\nto a reasonable summary-review while not relying on any review-specific\nfeatures. We consider variants of the proposed architecture and perform an\nablation study to show the importance of specific components. We show through\nautomated metrics and human evaluation that the generated summaries are highly\nabstractive, fluent, relevant, and representative of the average sentiment of\nthe input reviews. Finally, we collect a reference evaluation dataset and show\nthat our model outperforms a strong extractive baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05754v1",
            "date": "2018-10-12 23:05:01+00:00",
            "title": "A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification",
            "authors": [
                "Mounica Maddela",
                "Wei Xu"
            ],
            "abstract": "Current lexical simplification approaches rely heavily on heuristics and\ncorpus level features that do not always align with human judgment. We create a\nhuman-rated word-complexity lexicon of 15,000 English words and propose a novel\nneural readability ranking model with a Gaussian-based feature vectorization\nlayer that utilizes these human ratings to measure the complexity of any given\nword or phrase. Our model performs better than the state-of-the-art systems for\ndifferent lexical simplification tasks and evaluation datasets. Additionally,\nwe also produce SimplePPDB++, a lexical resource of over 10 million simplifying\nparaphrase rules, by applying our model to the Paraphrase Database (PPDB).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05788v2",
            "date": "2018-10-13 02:39:39+00:00",
            "title": "Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework",
            "authors": [
                "Shun Kiyono",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "The current success of deep neural networks (DNNs) in an increasingly broad\nrange of tasks involving artificial intelligence strongly depends on the\nquality and quantity of labeled training data. In general, the scarcity of\nlabeled data, which is often observed in many natural language processing\ntasks, is one of the most important issues to be addressed. Semi-supervised\nlearning (SSL) is a promising approach to overcoming this issue by\nincorporating a large amount of unlabeled data. In this paper, we propose a\nnovel scalable method of SSL for text classification tasks. The unique property\nof our method, Mixture of Expert/Imitator Networks, is that imitator networks\nlearn to \"imitate\" the estimated label distribution of the expert network over\nthe unlabeled data, which potentially contributes a set of features for the\nclassification. Our experiments demonstrate that the proposed method\nconsistently improves the performance of several types of baseline DNNs. We\nalso demonstrate that our method has the more data, better performance property\nwith promising scalability to the amount of unlabeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05867v2",
            "date": "2018-10-13 14:35:08+00:00",
            "title": "An Empirical Study on Crosslingual Transfer in Probabilistic Topic Models",
            "authors": [
                "Shudong Hao",
                "Michael J. Paul"
            ],
            "abstract": "Probabilistic topic modeling is a popular choice as the first step of\ncrosslingual tasks to enable knowledge transfer and extract multilingual\nfeatures. While many multilingual topic models have been developed, their\nassumptions on the training corpus are quite varied, and it is not clear how\nwell the models can be applied under various training conditions. In this\npaper, we systematically study the knowledge transfer mechanisms behind\ndifferent multilingual topic models, and through a broad set of experiments\nwith four models on ten languages, we provide empirical insights that can\ninform the selection and future development of multilingual topic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05995v1",
            "date": "2018-10-14 08:32:35+00:00",
            "title": "BLEU is Not Suitable for the Evaluation of Text Simplification",
            "authors": [
                "Elior Sulem",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "BLEU is widely considered to be an informative metric for text-to-text\ngeneration, including Text Simplification (TS). TS includes both lexical and\nstructural aspects. In this paper we show that BLEU is not suitable for the\nevaluation of sentence splitting, the major structural simplification\noperation. We manually compiled a sentence splitting gold standard corpus\ncontaining multiple structural paraphrases, and performed a correlation\nanalysis with human judgments. We find low or no correlation between BLEU and\nthe grammaticality and meaning preservation parameters where sentence splitting\nis involved. Moreover, BLEU often negatively correlates with simplicity,\nessentially penalizing simpler sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06065v1",
            "date": "2018-10-14 17:08:54+00:00",
            "title": "Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information",
            "authors": [
                "Lisa Fan",
                "Dong Yu",
                "Lu Wang"
            ],
            "abstract": "Sequence-to-sequence (seq2seq) neural models have been actively investigated\nfor abstractive summarization. Nevertheless, existing neural abstractive\nsystems frequently generate factually incorrect summaries and are vulnerable to\nadversarial information, suggesting a crucial lack of semantic understanding.\nIn this paper, we propose a novel semantic-aware neural abstractive\nsummarization model that learns to generate high quality summaries through\nsemantic interpretation over salient content. A novel evaluation scheme with\nadversarial samples is introduced to measure how well a model identifies\noff-topic information, where our model yields significantly better performance\nthan the popular pointer-generator summarizer. Human evaluation also confirms\nthat our system summaries are uniformly more informative and faithful as well\nas less redundant than the seq2seq model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06195v1",
            "date": "2018-10-15 06:30:04+00:00",
            "title": "Learning to Jointly Translate and Predict Dropped Pronouns with a Shared Reconstruction Mechanism",
            "authors": [
                "Longyue Wang",
                "Zhaopeng Tu",
                "Andy Way",
                "Qun Liu"
            ],
            "abstract": "Pronouns are frequently omitted in pro-drop languages, such as Chinese,\ngenerally leading to significant challenges with respect to the production of\ncomplete translations. Recently, Wang et al. (2018) proposed a novel\nreconstruction-based approach to alleviating dropped pronoun (DP) translation\nproblems for neural machine translation models. In this work, we improve the\noriginal model from two perspectives. First, we employ a shared reconstructor\nto better exploit encoder and decoder representations. Second, we jointly learn\nto translate and predict DPs in an end-to-end manner, to avoid the errors\npropagated from an external DP prediction model. Experimental results show that\nour approach significantly improves both translation performance and DP\nprediction accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06233v1",
            "date": "2018-10-15 09:05:21+00:00",
            "title": "UMONS Submission for WMT18 Multimodal Translation Task",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont"
            ],
            "abstract": "This paper describes the UMONS solution for the Multimodal Machine\nTranslation Task presented at the third conference on machine translation\n(WMT18). We explore a novel architecture, called deepGRU, based on recent\nfindings in the related task of Neural Image Captioning (NIC). The models\npresented in the following sections lead to the best METEOR translation score\nfor both constrained (English, image) -> German and (English, image) -> French\nsub-tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06245v1",
            "date": "2018-10-15 09:42:55+00:00",
            "title": "Bringing back simplicity and lightliness into neural image captioning",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont"
            ],
            "abstract": "Neural Image Captioning (NIC) or neural caption generation has attracted a\nlot of attention over the last few years. Describing an image with a natural\nlanguage has been an emerging challenge in both fields of computer vision and\nlanguage processing. Therefore a lot of research has focused on driving this\ntask forward with new creative ideas. So far, the goal has been to maximize\nscores on automated metric and to do so, one has to come up with a plurality of\nnew modules and techniques. Once these add up, the models become complex and\nresource-hungry. In this paper, we take a small step backwards in order to\nstudy an architecture with interesting trade-off between performance and\ncomputational complexity. To do so, we tackle every component of a neural\ncaptioning model and propose one or more solution that lightens the model\noverall. Our ideas are inspired by two related tasks: Multimodal and Monomodal\nNeural Machine Translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06351v1",
            "date": "2018-10-15 13:52:35+00:00",
            "title": "(Self-Attentive) Autoencoder-based Universal Language Representation for Machine Translation",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Universal language representation is the holy grail in machine translation\n(MT). Thanks to the new neural MT approach, it seems that there are good\nperspectives towards this goal. In this paper, we propose a new architecture\nbased on combining variational autoencoders with encoder-decoders and\nintroducing an interlingual loss as an additional training objective. By adding\nand forcing this interlingual loss, we are able to train multiple encoders and\ndecoders for each language, sharing a common universal representation. Since\nthe final objective of this universal representation is producing close results\nfor similar input sentences (in any language), we propose to evaluate it by\nencoding the same sentence in two different languages, decoding both latent\nrepresentations into the same language and comparing both outputs. Preliminary\nresults on the WMT 2017 Turkish/English task shows that the proposed\narchitecture is capable of learning a universal language representation and\nsimultaneously training both translation directions with state-of-the-art\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06368v1",
            "date": "2018-10-15 14:24:36+00:00",
            "title": "Neural Adaptation Layers for Cross-domain Named Entity Recognition",
            "authors": [
                "Bill Yuchen Lin",
                "Wei Lu"
            ],
            "abstract": "Recent research efforts have shown that neural architectures can be effective\nin conventional information extraction tasks such as named entity recognition,\nyielding state-of-the-art results on standard newswire datasets. However,\ndespite significant resources required for training such models, the\nperformance of a model trained on one domain typically degrades dramatically\nwhen applied to a different domain, yet extracting entities from new emerging\ndomains such as social media can be of significant interest. In this paper, we\nempirically investigate effective methods for conveniently adapting an\nexisting, well-trained neural NER model for a new domain. Unlike existing\napproaches, we propose lightweight yet effective methods for performing domain\nadaptation for neural models. Specifically, we introduce adaptation layers on\ntop of existing neural architectures, where no re-training using the source\ndomain data is required. We conduct extensive empirical studies and show that\nour approach significantly outperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06546v2",
            "date": "2018-10-15 17:54:36+00:00",
            "title": "Poincar\u00e9 GloVe: Hyperbolic Word Embeddings",
            "authors": [
                "Alexandru Tifrea",
                "Gary B\u00e9cigneul",
                "Octavian-Eugen Ganea"
            ],
            "abstract": "Words are not created equal. In fact, they form an aristocratic graph with a\nlatent hierarchical structure that the next generation of unsupervised learned\nword embeddings should reveal. In this paper, justified by the notion of\ndelta-hyperbolicity or tree-likeliness of a space, we propose to embed words in\na Cartesian product of hyperbolic spaces which we theoretically connect to the\nGaussian word embeddings and their Fisher geometry. This connection allows us\nto introduce a novel principled hypernymy score for word embeddings. Moreover,\nwe adapt the well-known Glove algorithm to learn unsupervised word embeddings\nin this type of Riemannian manifolds. We further explain how to solve the\nanalogy task using the Riemannian parallel transport that generalizes vector\narithmetics to this new type of geometry. Empirically, based on extensive\nexperiments, we prove that our embeddings, trained unsupervised, are the first\nto simultaneously outperform strong and popular baselines on the tasks of\nsimilarity, analogy and hypernymy detection. In particular, for word hypernymy,\nwe obtain new state-of-the-art on fully unsupervised WBLESS classification\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06619v3",
            "date": "2018-10-15 19:08:03+00:00",
            "title": "Diacritization of Maghrebi Arabic Sub-Dialects",
            "authors": [
                "Ahmed Abdelali",
                "Mohammed Attia",
                "Younes Samih",
                "Kareem Darwish",
                "Hamdy Mubarak"
            ],
            "abstract": "Diacritization process attempt to restore the short vowels in Arabic written\ntext; which typically are omitted. This process is essential for applications\nsuch as Text-to-Speech (TTS). While diacritization of Modern Standard Arabic\n(MSA) still holds the lion share, research on dialectal Arabic (DA)\ndiacritization is very limited. In this paper, we present our contribution and\nresults on the automatic diacritization of two sub-dialects of Maghrebi Arabic,\nnamely Tunisian and Moroccan, using a character-level deep neural network\narchitecture that stacks two bi-LSTM layers over a CRF output layer. The model\nachieves word error rate of 2.7% and 3.6% for Moroccan and Tunisian\nrespectively and is capable of implicitly identifying the sub-dialect of the\ninput.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06639v4",
            "date": "2018-10-07 19:07:59+00:00",
            "title": "A Machine Learning Approach to Persian Text Readability Assessment Using a Crowdsourced Dataset",
            "authors": [
                "Hamid Mohammadi",
                "Seyed Hossein Khasteh"
            ],
            "abstract": "An automated approach to text readability assessment is essential to a\nlanguage and can be a powerful tool for improving the understandability of\ntexts written and published in that language. However, the Persian language,\nwhich is spoken by over 110 million speakers, lacks such a system. Unlike other\nlanguages such as English, French, and Chinese, very limited research studies\nhave been carried out to build an accurate and reliable text readability\nassessment system for the Persian language. In the present research, the first\nPersian dataset for text readability assessment was gathered and the first\nmodel for Persian text readability assessment using machine learning was\nintroduced. The experiments showed that this model was accurate and could\nassess the readability of Persian texts with a high degree of confidence. The\nresults of this study can be used in a number of applications such as medical\nand educational text readability evaluation and have the potential to be the\ncornerstone of future studies in Persian text readability assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06694v3",
            "date": "2018-10-08 17:19:07+00:00",
            "title": "Word Embeddings from Large-Scale Greek Web Content",
            "authors": [
                "Stamatis Outsios",
                "Konstantinos Skianis",
                "Polykarpos Meladianos",
                "Christos Xypolopoulos",
                "Michalis Vazirgiannis"
            ],
            "abstract": "Word embeddings are undoubtedly very useful components in many NLP tasks. In\nthis paper, we present word embeddings and other linguistic resources trained\non the largest to date digital Greek language corpus. We also present a live\nweb tool for testing the Greek word embeddings, by offering \"analogy\",\n\"similarity score\" and \"most similar words\" functions. Through our explorer,\none could interact with the Greek word vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06729v2",
            "date": "2018-10-15 22:23:28+00:00",
            "title": "Robust Neural Machine Translation with Joint Textual and Phonetic Embedding",
            "authors": [
                "Hairong Liu",
                "Mingbo Ma",
                "Liang Huang",
                "Hao Xiong",
                "Zhongjun He"
            ],
            "abstract": "Neural machine translation (NMT) is notoriously sensitive to noises, but\nnoises are almost inevitable in practice. One special kind of noise is the\nhomophone noise, where words are replaced by other words with similar\npronunciations. We propose to improve the robustness of NMT to homophone noises\nby 1) jointly embedding both textual and phonetic information of source\nsentences, and 2) augmenting the training dataset with homophone noises.\nInterestingly, to achieve better translation quality and more robustness, we\nfound that most (though not all) weights should be put on the phonetic rather\nthan textual information. Experiments show that our method not only\nsignificantly improves the robustness of NMT to homophone noises, but also\nsurprisingly improves the translation quality on some clean test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06743v1",
            "date": "2018-10-15 23:00:13+00:00",
            "title": "Marrying Universal Dependencies and Universal Morphology",
            "authors": [
                "Arya D. McCarthy",
                "Miikka Silfverberg",
                "Ryan Cotterell",
                "Mans Hulden",
                "David Yarowsky"
            ],
            "abstract": "The Universal Dependencies (UD) and Universal Morphology (UniMorph) projects\neach present schemata for annotating the morphosyntactic details of language.\nEach project also provides corpora of annotated text in many languages - UD at\nthe token level and UniMorph at the type level. As each corpus is built by\ndifferent annotators, language-specific decisions hinder the goal of universal\nschemata. With compatibility of tags, each project's annotations could be used\nto validate the other's. Additionally, the availability of both type- and\ntoken-level resources would be a boon to tasks such as parsing and homograph\ndisambiguation. To ease this interoperability, we present a deterministic\nmapping from Universal Dependencies v2 features into the UniMorph schema. We\nvalidate our approach by lookup in the UniMorph corpora and find a\nmacro-average of 64.13% recall. We also note incompatibilities due to paucity\nof data on either side. Finally, we present a critical evaluation of the\nfoundations, strengths, and weaknesses of the two annotation projects.",
            "journal ref": "Proceedings of the Second Workshop on Universal Dependencies\n  (2018) 91-101",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06826v2",
            "date": "2018-10-16 06:00:37+00:00",
            "title": "Multi-Source Neural Machine Translation with Data Augmentation",
            "authors": [
                "Yuta Nishimura",
                "Katsuhito Sudoh",
                "Graham Neubig",
                "Satoshi Nakamura"
            ],
            "abstract": "Multi-source translation systems translate from multiple languages to a\nsingle target language. By using information from these multiple sources, these\nsystems achieve large gains in accuracy. To train these systems, it is\nnecessary to have corpora with parallel text in multiple sources and the target\nlanguage. However, these corpora are rarely complete in practice due to the\ndifficulty of providing human translations in all of the relevant languages. In\nthis paper, we propose a data augmentation approach to fill such incomplete\nparts using multi-source neural machine translation (NMT). In our experiments,\nresults varied over different language combinations but significant gains were\nobserved when using a source language similar to the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06898v1",
            "date": "2018-10-16 09:37:24+00:00",
            "title": "Creating a New Persian Poet Based on Machine Learning",
            "authors": [
                "Mehdi Hosseini Moghadam",
                "Bardia Panahbehagh"
            ],
            "abstract": "In this article we describe an application of Machine Learning (ML) and\nLinguistic Modeling to generate persian poems. In fact we teach machine by\nreading and learning persian poems to generate fake poems in the same style of\nthe original poems. As two well known poets we used Hafez (1310-1390) and Saadi\n(1210-1292) poems. First we feed the machine with Hafez poems to generate fake\npoems with the same style and then we feed the machine with the both Hafez and\nSaadi poems to generate a new style poems which is combination of these two\npoets styles with emotional (Hafez) and rational (Saadi) elements. This idea of\ncombination of different styles with ML opens new gates for extending the\ntreasure of past literature of different cultures. Results show with enough\nmemory, processing power and time it is possible to generate reasonable good\npoems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06908v1",
            "date": "2018-10-16 10:00:06+00:00",
            "title": "Neural Morphological Tagging for Estonian",
            "authors": [
                "Alexander Tkachenko",
                "Kairit Sirts"
            ],
            "abstract": "We develop neural morphological tagging and disambiguation models for\nEstonian. First, we experiment with two neural architectures for morphological\ntagging - a standard multiclass classifier which treats each morphological tag\nas a single unit, and a sequence model which handles the morphological tags as\nsequences of morphological category values. Secondly, we complement these\nmodels with the analyses generated by a rule-based Estonian morphological\nanalyser (MA) VABAMORF , thus performing a soft morphological disambiguation.\nWe compare two ways of supplementing a neural morphological tagger with the MA\noutputs: firstly, by adding the combined analyses embeddings to the word\nrepresentation input to the neural tagging model, and secondly, by adopting an\nattention mechanism to focus on the most relevant analyses generated by the MA.\nExperiments on three Estonian datasets show that our neural architectures\nconsistently outperform the non-neural baselines, including HMM-disambiguated\nVABAMORF, while augmenting models with MA outputs results in a further\nperformance boost for both models.",
            "journal ref": "Proceedings of the Eighth International Conference Baltic HLT 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07091v1",
            "date": "2018-10-16 15:44:53+00:00",
            "title": "INFODENS: An Open-source Framework for Learning Text Representations",
            "authors": [
                "Ahmad Taie",
                "Raphael Rubino",
                "Josef van Genabith"
            ],
            "abstract": "The advent of representation learning methods enabled large performance gains\non various language tasks, alleviating the need for manual feature engineering.\nWhile engineered representations are usually based on some linguistic\nunderstanding and are therefore more interpretable, learned representations are\nharder to interpret. Empirically studying the complementarity of both\napproaches can provide more linguistic insights that would help reach a better\ncompromise between interpretability and performance. We present INFODENS, a\nframework for studying learned and engineered representations of text in the\ncontext of text classification tasks. It is designed to simplify the tasks of\nfeature engineering as well as provide the groundwork for extracting learned\nfeatures and combining both approaches. INFODENS is flexible, extensible, with\na short learning curve, and is easy to integrate with many of the available and\nwidely used natural language processing tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07125v3",
            "date": "2018-10-16 16:39:49+00:00",
            "title": "The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection",
            "authors": [
                "Ryan Cotterell",
                "Christo Kirov",
                "John Sylak-Glassman",
                "G\u00e9raldine Walther",
                "Ekaterina Vylomova",
                "Arya D. McCarthy",
                "Katharina Kann",
                "Sabrina J. Mielke",
                "Garrett Nicolai",
                "Miikka Silfverberg",
                "David Yarowsky",
                "Jason Eisner",
                "Mans Hulden"
            ],
            "abstract": "The CoNLL--SIGMORPHON 2018 shared task on supervised learning of\nmorphological generation featured data sets from 103 typologically diverse\nlanguages. Apart from extending the number of languages involved in earlier\nsupervised tasks of generating inflected forms, this year the shared task also\nfeatured a new second task which asked participants to inflect words in\nsentential context, similar to a cloze task. This second task featured seven\nlanguages. Task 1 received 27 submissions and task 2 received 6 submissions.\nBoth tasks featured a low, medium, and high data condition. Nearly all\nsubmissions featured a neural component and built on highly-ranked systems from\nthe earlier 2017 shared task. In the inflection task (task 1), 41 of the 52\nlanguages present in last year's inflection task showed improvement by the best\nsystems in the low-resource setting. The cloze task (task 2) proved to be\ndifficult, and few submissions managed to consistently improve upon both a\nsimple neural baseline system and a lemma-repeating baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07150v3",
            "date": "2018-10-16 17:25:22+00:00",
            "title": "Subword Semantic Hashing for Intent Classification on Small Datasets",
            "authors": [
                "Kumar Shridhar",
                "Ayushman Dash",
                "Amit Sahu",
                "Gustav Grund Pihlgren",
                "Pedro Alonso",
                "Vinaychandran Pondenkandath",
                "Gyorgy Kovacs",
                "Foteini Simistira",
                "Marcus Liwicki"
            ],
            "abstract": "In this paper, we introduce the use of Semantic Hashing as embedding for the\ntask of Intent Classification and achieve state-of-the-art performance on three\nfrequently used benchmarks. Intent Classification on a small dataset is a\nchallenging task for data-hungry state-of-the-art Deep Learning based systems.\nSemantic Hashing is an attempt to overcome such a challenge and learn robust\ntext classification. Current word embedding based are dependent on\nvocabularies. One of the major drawbacks of such methods is out-of-vocabulary\nterms, especially when having small training datasets and using a wider\nvocabulary. This is the case in Intent Classification for chatbots, where\ntypically small datasets are extracted from internet communication. Two\nproblems arise by the use of internet communication. First, such datasets miss\na lot of terms in the vocabulary to use word embeddings efficiently. Second,\nusers frequently make spelling errors. Typically, the models for intent\nclassification are not trained with spelling errors and it is difficult to\nthink about ways in which users will make mistakes. Models depending on a word\nvocabulary will always face such issues. An ideal classifier should handle\nspelling errors inherently. With Semantic Hashing, we overcome these challenges\nand achieve state-of-the-art results on three datasets: AskUbuntu, Chatbot, and\nWeb Application. Our benchmarks are available online:\nhttps://github.com/kumar-shridhar/Know-Your-Intent",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07156v2",
            "date": "2018-10-16 17:35:31+00:00",
            "title": "Strategies for Language Identification in Code-Mixed Low Resource Languages",
            "authors": [
                "Soumil Mandal",
                "Sankalp Sanand"
            ],
            "abstract": "In recent years, substantial work has been done on language tagging of\ncode-mixed data, but most of them use large amounts of data to build their\nmodels. In this article, we present three strategies to build a word level\nlanguage tagger for code-mixed data using very low resources. Each of them\nsecured an accuracy higher than our baseline model, and the best performing\nsystem got an accuracy around 91%. Combining all, the ensemble system achieved\nan accuracy of around 92.6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07366v2",
            "date": "2018-10-17 03:02:26+00:00",
            "title": "A Span-Extraction Dataset for Chinese Machine Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Ting Liu",
                "Wanxiang Che",
                "Li Xiao",
                "Zhipeng Chen",
                "Wentao Ma",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Machine Reading Comprehension (MRC) has become enormously popular recently\nand has attracted a lot of attention. However, the existing reading\ncomprehension datasets are mostly in English. In this paper, we introduce a\nSpan-Extraction dataset for Chinese machine reading comprehension to add\nlanguage diversities in this area. The dataset is composed by near 20,000 real\nquestions annotated on Wikipedia paragraphs by human experts. We also annotated\na challenge set which contains the questions that need comprehensive\nunderstanding and multi-sentence inference throughout the context. We present\nseveral baseline systems as well as anonymous submissions for demonstrating the\ndifficulties in this dataset. With the release of the dataset, we hosted the\nSecond Evaluation Workshop on Chinese Machine Reading Comprehension (CMRC\n2018). We hope the release of the dataset could further accelerate the Chinese\nmachine reading comprehension research. Resources are available:\nhttps://github.com/ymcui/cmrc2018",
            "journal ref": "EMNLP 2019 5886-5891",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07391v1",
            "date": "2018-10-17 05:42:49+00:00",
            "title": "Sequence to Sequence Mixture Model for Diverse Machine Translation",
            "authors": [
                "Xuanli He",
                "Gholamreza Haffari",
                "Mohammad Norouzi"
            ],
            "abstract": "Sequence to sequence (SEQ2SEQ) models often lack diversity in their generated\ntranslations. This can be attributed to the limitation of SEQ2SEQ models in\ncapturing lexical and syntactic variations in a parallel corpus resulting from\ndifferent styles, genres, topics, or ambiguity of the translation process. In\nthis paper, we develop a novel sequence to sequence mixture (S2SMIX) model that\nimproves both translation diversity and quality by adopting a committee of\nspecialized translation models rather than a single translation model. Each\nmixture component selects its own training dataset via optimization of the\nmarginal loglikelihood, which leads to a soft clustering of the parallel\ncorpus. Experiments on four language pairs demonstrate the superiority of our\nmixture model compared to a SEQ2SEQ baseline with standard or diversity-boosted\nbeam search. Our mixture model uses negligible additional parameters and incurs\nno extra computation cost during decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07455v1",
            "date": "2018-10-17 09:53:03+00:00",
            "title": "Exploring Textual and Speech information in Dialogue Act Classification with Speaker Domain Adaptation",
            "authors": [
                "Xuanli He",
                "Quan Hung Tran",
                "William Havard",
                "Laurent Besacier",
                "Ingrid Zukerman",
                "Gholamreza Haffari"
            ],
            "abstract": "In spite of the recent success of Dialogue Act (DA) classification, the\nmajority of prior works focus on text-based classification with oracle\ntranscriptions, i.e. human transcriptions, instead of Automatic Speech\nRecognition (ASR)'s transcriptions. In spoken dialog systems, however, the\nagent would only have access to noisy ASR transcriptions, which may further\nsuffer performance degradation due to domain shift. In this paper, we explore\nthe effectiveness of using both acoustic and textual signals, either oracle or\nASR transcriptions, and investigate speaker domain adaptation for DA\nclassification. Our multimodal model proves to be superior to the unimodal\nmodels, particularly when the oracle transcriptions are not available. We also\npropose an effective method for speaker domain adaptation, which achieves\ncompetitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07595v1",
            "date": "2018-10-17 14:58:54+00:00",
            "title": "An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation",
            "authors": [
                "Gongbo Tang",
                "Rico Sennrich",
                "Joakim Nivre"
            ],
            "abstract": "Recent work has shown that the encoder-decoder attention mechanisms in neural\nmachine translation (NMT) are different from the word alignment in statistical\nmachine translation. In this paper, we focus on analyzing encoder-decoder\nattention mechanisms, in the case of word sense disambiguation (WSD) in NMT\nmodels. We hypothesize that attention mechanisms pay more attention to context\ntokens when translating ambiguous words. We explore the attention distribution\npatterns when translating ambiguous nouns. Counter-intuitively, we find that\nattention mechanisms are likely to distribute more attention to the ambiguous\nnoun itself rather than context tokens, in comparison to other nouns. We\nconclude that attention mechanism is not the main mechanism used by NMT models\nto incorporate contextual information for WSD. The experimental results suggest\nthat NMT models learn to encode contextual information necessary for WSD in the\nencoder hidden states. For the attention mechanism in Transformer models, we\nreveal that the first few layers gradually learn to \"align\" source and target\ntokens and the last few layers learn to extract features from the related but\nunaligned context tokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07653v1",
            "date": "2018-10-15 22:49:48+00:00",
            "title": "Super Characters: A Conversion from Sentiment Classification to Image Classification",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Patrick Dong",
                "Wenhan Zhang",
                "Jason Dong",
                "Charles Young"
            ],
            "abstract": "We propose a method named Super Characters for sentiment classification. This\nmethod converts the sentiment classification problem into image classification\nproblem by projecting texts into images and then applying CNN models for\nclassification. Text features are extracted automatically from the generated\nSuper Characters images, hence there is no need of any explicit step of\nembedding the words or characters into numerical vector representations.\nExperimental results on large social media corpus show that the Super\nCharacters method consistently outperforms other methods for sentiment\nclassification and topic classification tasks on ten large social media\ndatasets of millions of contents in four different languages, including\nChinese, Japanese, Korean and English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07942v1",
            "date": "2018-10-18 08:22:49+00:00",
            "title": "Semantic Parsing for Task Oriented Dialog using Hierarchical Representations",
            "authors": [
                "Sonal Gupta",
                "Rushin Shah",
                "Mrinal Mohit",
                "Anuj Kumar",
                "Mike Lewis"
            ],
            "abstract": "Task oriented dialog systems typically first parse user utterances to\nsemantic frames comprised of intents and slots. Previous work on task oriented\nintent and slot-filling work has been restricted to one intent per query and\none slot label per token, and thus cannot model complex compositional requests.\nAlternative semantic parsing systems have represented queries as logical forms,\nbut these are challenging to annotate and parse. We propose a hierarchical\nannotation scheme for semantic parsing that allows the representation of\ncompositional queries, and can be efficiently and accurately parsed by standard\nconstituency parsing models. We release a dataset of 44k annotated queries\n(fb.me/semanticparsingdialog), and show that parsing models outperform\nsequence-to-sequence approaches on this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07949v1",
            "date": "2018-10-18 08:46:40+00:00",
            "title": "A Temporally Sensitive Submodularity Framework for Timeline Summarization",
            "authors": [
                "Sebastian Martschat",
                "Katja Markert"
            ],
            "abstract": "Timeline summarization (TLS) creates an overview of long-running events via\ndated daily summaries for the most important dates. TLS differs from standard\nmulti-document summarization (MDS) in the importance of date selection,\ninterdependencies between summaries of different dates and by having very short\nsummaries compared to the number of corpus documents. However, we show that MDS\noptimization models using submodular functions can be adapted to yield\nwell-performing TLS models by designing objective functions and constraints\nthat model the temporal dimension inherent in TLS. Importantly, these\nadaptations retain the elegance and advantages of the original MDS models\n(clear separation of features and inference, performance guarantees and\nscalability, little need for supervision) that current TLS-specific models\nlack. An open-source implementation of the framework and all models described\nin this paper is available online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08113v2",
            "date": "2018-10-18 15:35:07+00:00",
            "title": "Adversarial TableQA: Attention Supervision for Question Answering on Tables",
            "authors": [
                "Minseok Cho",
                "Reinald Kim Amplayo",
                "Seung-won Hwang",
                "Jonghyuck Park"
            ],
            "abstract": "The task of answering a question given a text passage has shown great\ndevelopments on model performance thanks to community efforts in building\nuseful datasets. Recently, there have been doubts whether such rapid progress\nhas been based on truly understanding language. The same question has not been\nasked in the table question answering (TableQA) task, where we are tasked to\nanswer a query given a table. We show that existing efforts, of using \"answers\"\nfor both evaluation and supervision for TableQA, show deteriorating\nperformances in adversarial settings of perturbations that do not affect the\nanswer. This insight naturally motivates to develop new models that understand\nquestion and table more precisely. For this goal, we propose Neural Operator\n(NeOp), a multi-layer sequential network with attention supervision to answer\nthe query given a table. NeOp uses multiple Selective Recurrent Units (SelRUs)\nto further help the interpretability of the answers of the model. Experiments\nshow that the use of operand information to train the model significantly\nimproves the performance and interpretability of TableQA models. NeOp\noutperforms all the previous models by a big margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08135v2",
            "date": "2018-10-18 16:19:16+00:00",
            "title": "Contextual Topic Modeling For Dialog Systems",
            "authors": [
                "Chandra Khatri",
                "Rahul Goel",
                "Behnam Hedayatnia",
                "Angeliki Metanillou",
                "Anushree Venkatesh",
                "Raefer Gabriel",
                "Arindam Mandal"
            ],
            "abstract": "Accurate prediction of conversation topics can be a valuable signal for\ncreating coherent and engaging dialog systems. In this work, we focus on\ncontext-aware topic classification methods for identifying topics in free-form\nhuman-chatbot dialogs. We extend previous work on neural topic classification\nand unsupervised topic keyword detection by incorporating conversational\ncontext and dialog act features. On annotated data, we show that incorporating\ncontext and dialog acts leads to relative gains in topic classification\naccuracy by 35% and on unsupervised keyword detection recall by 11% for\nconversational interactions where topics frequently span multiple utterances.\nWe show that topical metrics such as topical depth is highly correlated with\ndialog evaluation metrics such as coherence and engagement implying that\nconversational topic models can predict user satisfaction. Our work for\ndetecting conversation topics and keywords can be used to guide chatbots\ntowards coherent dialog.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08237v2",
            "date": "2018-10-18 18:51:43+00:00",
            "title": "Large-scale Hierarchical Alignment for Data-driven Text Rewriting",
            "authors": [
                "Nikola I. Nikolov",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "We propose a simple unsupervised method for extracting pseudo-parallel\nmonolingual sentence pairs from comparable corpora representative of two\ndifferent text styles, such as news articles and scientific papers. Our\napproach does not require a seed parallel corpus, but instead relies solely on\nhierarchical search over pre-trained embeddings of documents and sentences. We\ndemonstrate the effectiveness of our method through automatic and extrinsic\nevaluation on text simplification from the normal to the Simple Wikipedia. We\nshow that pseudo-parallel sentences extracted with our method not only\nsupplement existing parallel data, but can even lead to competitive performance\non their own.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08307v1",
            "date": "2018-10-18 23:40:47+00:00",
            "title": "Reduction of Parameter Redundancy in Biaffine Classifiers with Symmetric and Circulant Weight Matrices",
            "authors": [
                "Tomoki Matsuno",
                "Katsuhiko Hayashi",
                "Takahiro Ishihara",
                "Hitoshi Manabe",
                "Yuji Matsumoto"
            ],
            "abstract": "Currently, the biaffine classifier has been attracting attention as a method\nto introduce an attention mechanism into the modeling of binary relations. For\ninstance, in the field of dependency parsing, the Deep Biaffine Parser by Dozat\nand Manning has achieved state-of-the-art performance as a graph-based\ndependency parser on the English Penn Treebank and CoNLL 2017 shared task. On\nthe other hand, it is reported that parameter redundancy in the weight matrix\nin biaffine classifiers, which has O(n^2) parameters, results in overfitting (n\nis the number of dimensions). In this paper, we attempted to reduce the\nparameter redundancy by assuming either symmetry or circularity of weight\nmatrices. In our experiments on the CoNLL 2017 shared task dataset, our model\nachieved better or comparable accuracy on most of the treebanks with more than\n16% parameter reduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08392v1",
            "date": "2018-10-19 08:28:07+00:00",
            "title": "Impact of Corpora Quality on Neural Machine Translation",
            "authors": [
                "Mat\u012bss Rikters"
            ],
            "abstract": "Large parallel corpora that are automatically obtained from the web,\ndocuments or elsewhere often exhibit many corrupted parts that are bound to\nnegatively affect the quality of the systems and models that learn from these\ncorpora. This paper describes frequent problems found in data and such data\naffects neural machine translation systems, as well as how to identify and deal\nwith them. The solutions are summarised in a set of scripts that remove\nproblematic sentences from input corpora.",
            "journal ref": "Published in the proceedings of the 8th International Baltic Human\n  Language Technologies Conference (Baltic HLT 2018), held in Tartu, Estonia,\n  on 27-29 September 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08398v5",
            "date": "2018-10-19 08:37:40+00:00",
            "title": "STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework",
            "authors": [
                "Mingbo Ma",
                "Liang Huang",
                "Hao Xiong",
                "Renjie Zheng",
                "Kaibo Liu",
                "Baigong Zheng",
                "Chuanqiang Zhang",
                "Zhongjun He",
                "Hairong Liu",
                "Xing Li",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Simultaneous translation, which translates sentences before they are\nfinished, is useful in many scenarios but is notoriously difficult due to\nword-order differences. While the conventional seq-to-seq framework is only\nsuitable for full-sentence translation, we propose a novel prefix-to-prefix\nframework for simultaneous translation that implicitly learns to anticipate in\na single translation model. Within this framework, we present a very simple yet\nsurprisingly effective wait-k policy trained to generate the target sentence\nconcurrently with the source sentence, but always k words behind. Experiments\nshow our strategy achieves low latency and reasonable quality (compared to\nfull-sentence translation) on 4 directions: zh<->en and de<->en.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08436v2",
            "date": "2018-10-19 10:47:12+00:00",
            "title": "Efficient Dependency-Guided Named Entity Recognition",
            "authors": [
                "Zhanming Jie",
                "Aldrian Obaja Muis",
                "Wei Lu"
            ],
            "abstract": "Named entity recognition (NER), which focuses on the extraction of\nsemantically meaningful named entities and their semantic classes from text,\nserves as an indispensable component for several down-stream natural language\nprocessing (NLP) tasks such as relation extraction and event extraction.\nDependency trees, on the other hand, also convey crucial semantic-level\ninformation. It has been shown previously that such information can be used to\nimprove the performance of NER (Sasano and Kurohashi 2008, Ling and Weld 2012).\nIn this work, we investigate on how to better utilize the structured\ninformation conveyed by dependency trees to improve the performance of NER.\nSpecifically, unlike existing approaches which only exploit dependency\ninformation for designing local features, we show that certain global\nstructured information of the dependency trees can be exploited when building\nNER models where such information can provide guided learning and inference.\nThrough extensive experiments, we show that our proposed novel\ndependency-guided NER model performs competitively with models based on\nconventional semi-Markov conditional random fields, while requiring\nsignificantly less running time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08567v1",
            "date": "2018-10-19 16:09:35+00:00",
            "title": "Weak Semi-Markov CRFs for NP Chunking in Informal Text",
            "authors": [
                "Aldrian Obaja Muis",
                "Wei Lu"
            ],
            "abstract": "This paper introduces a new annotated corpus based on an existing informal\ntext corpus: the NUS SMS Corpus (Chen and Kan, 2013). The new corpus includes\n76,490 noun phrases from 26,500 SMS messages, annotated by university students.\nWe then explored several graphical models, including a novel variant of the\nsemi-Markov conditional random fields (semi-CRF) for the task of noun phrase\nchunking. We demonstrated through empirical evaluations on the new dataset that\nthe new variant yielded similar accuracy but ran in significantly lower running\ntime compared to the conventional semi-CRF.",
            "journal ref": "Aldrian Obaja Muis and Wei Lu. 2016. Weak Semi-Markov CRFs for\n  Noun Phrase Chunking in Informal Text. In Proceedings of HLT-NAACL 2016,\n  pages 714-719",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08579v3",
            "date": "2018-10-19 16:48:25+00:00",
            "title": "Learning to Recognize Discontiguous Entities",
            "authors": [
                "Aldrian Obaja Muis",
                "Wei Lu"
            ],
            "abstract": "This paper focuses on the study of recognizing discontiguous entities.\nMotivated by a previous work, we propose to use a novel hypergraph\nrepresentation to jointly encode discontiguous entities of unbounded length,\nwhich can overlap with one another. To compare with existing approaches, we\nfirst formally introduce the notion of model ambiguity, which defines the\ndifficulty level of interpreting the outputs of a model, and then formally\nanalyze the theoretical advantages of our model over previous existing\napproaches based on linear-chain CRFs. Our empirical results also show that our\nmodel is able to achieve significantly better results when evaluated on\nstandard data with many discontiguous entities.",
            "journal ref": "In Proc. of EMNLP, pages 75-84, Stroudsburg, PA, USA. Association\n  for Computational Linguistics (2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08603v1",
            "date": "2018-10-19 17:55:46+00:00",
            "title": "Mainumby: un Ayudante para la Traducci\u00f3n Castellano-Guaran\u00ed",
            "authors": [
                "Michael Gasser"
            ],
            "abstract": "A wide range of applications play an important role in the daily work of the\nmodern human translator. However, the computational tools designed to aid in\nthe process of translation only benefit translation from or to a small minority\nof the 7,000 languages of the world, those that we may call \"privileged\nlanguages\". As for those translators who work with the remaining languages, the\nmarginalized languages in the digital world, they cannot benefit from the tools\nthat are speeding up the production of translation in the privileged languages.\nWe may ask whether it is possible to bridge the gap between what is available\nfor these languages and for the marginalized ones. This paper proposes a\nframework for computer-assisted translation into marginalized languages and its\nimplementation in a web application for Spanish-Guarani translation. The\nproposed system is based on a new theory for phrase-level translation in\ncontexts where adequate bilingual corpora are not available: Translation by\nGeneralized Segments (referred to as Minimal Dependency Translation in previous\nwork).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08641v1",
            "date": "2018-10-19 18:47:02+00:00",
            "title": "Optimizing Segmentation Granularity for Neural Machine Translation",
            "authors": [
                "Elizabeth Salesky",
                "Andrew Runge",
                "Alex Coda",
                "Jan Niehues",
                "Graham Neubig"
            ],
            "abstract": "In neural machine translation (NMT), it is has become standard to translate\nusing subword units to allow for an open vocabulary and improve accuracy on\ninfrequent words. Byte-pair encoding (BPE) and its variants are the predominant\napproach to generating these subwords, as they are unsupervised, resource-free,\nand empirically effective. However, the granularity of these subword units is a\nhyperparameter to be tuned for each language and task, using methods such as\ngrid search. Tuning may be done inexhaustively or skipped entirely due to\nresource constraints, leading to sub-optimal performance. In this paper, we\npropose a method to automatically tune this parameter using only one training\npass. We incrementally introduce new vocabulary online based on the held-out\nvalidation loss, beginning with smaller, general subwords and adding larger,\nmore specific units over the course of training. Our method matches the results\nfound with grid search, optimizing segmentation granularity without any\nadditional training time. We also show benefits in training efficiency and\nperformance improvements for rare words due to the way embeddings for larger\nunits are incrementally constructed by combining those from smaller units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08699v1",
            "date": "2018-10-19 22:01:48+00:00",
            "title": "pioNER: Datasets and Baselines for Armenian Named Entity Recognition",
            "authors": [
                "Tsolak Ghukasyan",
                "Garnik Davtyan",
                "Karen Avetisyan",
                "Ivan Andrianov"
            ],
            "abstract": "In this work, we tackle the problem of Armenian named entity recognition,\nproviding silver- and gold-standard datasets as well as establishing baseline\nresults on popular models. We present a 163000-token named entity corpus\nautomatically generated and annotated from Wikipedia, and another 53400-token\ncorpus of news sentences with manual annotation of people, organization and\nlocation named entities. The corpora were used to train and evaluate several\npopular named entity recognition models. Alongside the datasets, we release\n50-, 100-, 200-, 300-dimensional GloVe word embeddings trained on a collection\nof Armenian texts from Wikipedia, news, blogs, and encyclopedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08717v1",
            "date": "2018-10-19 23:53:25+00:00",
            "title": "Learning Personas from Dialogue with Attentive Memory Networks",
            "authors": [
                "Eric Chu",
                "Prashanth Vijayaraghavan",
                "Deb Roy"
            ],
            "abstract": "The ability to infer persona from dialogue can have applications in areas\nranging from computational narrative analysis to personalized dialogue\ngeneration. We introduce neural models to learn persona embeddings in a\nsupervised character trope classification task. The models encode dialogue\nsnippets from IMDB into representations that can capture the various categories\nof film characters. The best-performing models use a multi-level attention\nmechanism over a set of utterances. We also utilize prior knowledge in the form\nof textual descriptions of the different tropes. We apply the learned\nembeddings to find similar characters across different movies, and cluster\nmovies according to the distribution of the embeddings. The use of short\nconversational text as input, and the ability to learn from prior knowledge\nusing memory, suggests these methods could be applied to other domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08740v2",
            "date": "2018-10-20 03:00:53+00:00",
            "title": "Improving Multilingual Semantic Textual Similarity with Shared Sentence Encoder for Low-resource Languages",
            "authors": [
                "Xin Tang",
                "Shanbo Cheng",
                "Loc Do",
                "Zhiyu Min",
                "Feng Ji",
                "Heng Yu",
                "Ji Zhang",
                "Haiqin Chen"
            ],
            "abstract": "Measuring the semantic similarity between two sentences (or Semantic Textual\nSimilarity - STS) is fundamental in many NLP applications. Despite the\nremarkable results in supervised settings with adequate labeling, little\nattention has been paid to this task in low-resource languages with\ninsufficient labeling. Existing approaches mostly leverage machine translation\ntechniques to translate sentences into rich-resource language. These approaches\neither beget language biases, or be impractical in industrial applications\nwhere spoken language scenario is more often and rigorous efficiency is\nrequired. In this work, we propose a multilingual framework to tackle the STS\ntask in a low-resource language e.g. Spanish, Arabic , Indonesian and Thai, by\nutilizing the rich annotation data in a rich resource language, e.g. English.\nOur approach is extended from a basic monolingual STS framework to a shared\nmultilingual encoder pretrained with translation task to incorporate\nrich-resource language data. By exploiting the nature of a shared multilingual\nencoder, one sentence can have multiple representations for different target\ntranslation language, which are used in an ensemble model to improve similarity\nevaluation. We demonstrate the superiority of our method over other state of\nthe art approaches on SemEval STS task by its significant improvement on non-MT\nmethod, as well as an online industrial product where MT method fails to beat\nbaseline while our approach still has consistently improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08815v1",
            "date": "2018-10-20 15:00:23+00:00",
            "title": "Modeling Composite Labels for Neural Morphological Tagging",
            "authors": [
                "Alexander Tkachenko",
                "Kairit Sirts"
            ],
            "abstract": "Neural morphological tagging has been regarded as an extension to POS tagging\ntask, treating each morphological tag as a monolithic label and ignoring its\ninternal structure. We propose to view morphological tags as composite labels\nand explicitly model their internal structure in a neural sequence tagger. For\nthis, we explore three different neural architectures and compare their\nperformance with both CRF and simple neural multiclass baselines. We evaluate\nour models on 49 languages and show that the neural architecture that models\nthe morphological labels as sequences of morphological category values performs\nsignificantly better than both baselines establishing state-of-the-art results\nin morphological tagging for most languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08838v1",
            "date": "2018-10-20 18:24:48+00:00",
            "title": "Abstractive Summarization Using Attentive Neural Techniques",
            "authors": [
                "Jacob Krantz",
                "Jugal Kalita"
            ],
            "abstract": "In a world of proliferating data, the ability to rapidly summarize text is\ngrowing in importance. Automatic summarization of text can be thought of as a\nsequence to sequence problem. Another area of natural language processing that\nsolves a sequence to sequence problem is machine translation, which is rapidly\nevolving due to the development of attention-based encoder-decoder networks.\nThis work applies these modern techniques to abstractive summarization. We\nperform analysis on various attention mechanisms for summarization with the\ngoal of developing an approach and architecture aimed at improving the state of\nthe art. In particular, we modify and optimize a translation model with\nself-attention for generating abstractive sentence summaries. The effectiveness\nof this base model along with attention variants is compared and analyzed in\nthe context of standardized evaluation sets and test metrics. However, we show\nthat these metrics are limited in their ability to effectively score\nabstractive summaries, and propose a new approach based on the intuition that\nan abstractive model requires an abstractive evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08854v2",
            "date": "2018-10-20 21:37:08+00:00",
            "title": "pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference",
            "authors": [
                "Mandar Joshi",
                "Eunsol Choi",
                "Omer Levy",
                "Daniel S. Weld",
                "Luke Zettlemoyer"
            ],
            "abstract": "Reasoning about implied relationships (e.g., paraphrastic, common sense,\nencyclopedic) between pairs of words is crucial for many cross-sentence\ninference problems. This paper proposes new methods for learning and using\nembeddings of word pairs that implicitly represent background knowledge about\nsuch relationships. Our pairwise embeddings are computed as a compositional\nfunction on word representations, which is learned by maximizing the pointwise\nmutual information (PMI) with the contexts in which the two words co-occur. We\nadd these representations to the cross-sentence attention layer of existing\ninference models (e.g. BiDAF for QA, ESIM for NLI), instead of extending or\nreplacing existing word embeddings. Experiments show a gain of 2.7% on the\nrecently released SQuAD2.0 and 1.3% on MultiNLI. Our representations also aid\nin better generalization with gains of around 6-7% on adversarial SQuAD\ndatasets, and 8.8% on the adversarial entailment test set by Glockner et al.\n(2018).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08951v1",
            "date": "2018-10-21 13:57:17+00:00",
            "title": "BCWS: Bilingual Contextual Word Similarity",
            "authors": [
                "Ta-Chung Chi",
                "Ching-Yen Shih",
                "Yun-Nung Chen"
            ],
            "abstract": "This paper introduces the first dataset for evaluating English-Chinese\nBilingual Contextual Word Similarity, namely BCWS\n(https://github.com/MiuLab/BCWS). The dataset consists of 2,091 English-Chinese\nword pairs with the corresponding sentential contexts and their similarity\nscores annotated by the human. Our annotated dataset has higher consistency\ncompared to other similar datasets. We establish several baselines for the\nbilingual embedding task to benchmark the experiments. Modeling cross-lingual\nsense representations as provided in this dataset has the potential of moving\nartificial intelligence from monolingual understanding towards multilingual\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08994v2",
            "date": "2018-10-21 17:59:52+00:00",
            "title": "Constituent Parsing as Sequence Labeling",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "David Vilares"
            ],
            "abstract": "We introduce a method to reduce constituent parsing to sequence labeling. For\neach word w_t, it generates a label that encodes: (1) the number of ancestors\nin the tree that the words w_t and w_{t+1} have in common, and (2) the\nnonterminal symbol at the lowest common ancestor. We first prove that the\nproposed encoding function is injective for any tree without unary branches. In\npractice, the approach is made extensible to all constituency trees by\ncollapsing unary branches. We then use the PTB and CTB treebanks as testbeds\nand propose a set of fast baselines. We achieve 90.7% F-score on the PTB test\nset, outperforming the Vinyals et al. (2015) sequence-to-sequence parser. In\naddition, sacrificing some accuracy, our approach achieves the fastest\nconstituent parsing speeds reported to date on PTB by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08997v1",
            "date": "2018-10-21 18:17:37+00:00",
            "title": "Transition-based Parsing with Lighter Feed-Forward Networks",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We explore whether it is possible to build lighter parsers, that are\nstatistically equivalent to their corresponding standard version, for a wide\nset of languages showing different structures and morphologies. As testbed, we\nuse the Universal Dependencies and transition-based dependency parsers trained\non feed-forward networks. For these, most existing research assumes de facto\nstandard embedded features and relies on pre-computation tricks to obtain\nspeed-ups. We explore how these features and their size can be reduced and\nwhether this translates into speed-ups with a negligible impact on accuracy.\nThe experiments show that grand-daughter features can be removed for the\nmajority of treebanks without a significant (negative or positive) LAS\ndifference. They also show how the size of the embeddings can be notably\nreduced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09073v1",
            "date": "2018-10-22 03:38:23+00:00",
            "title": "Labeling Gaps Between Words: Recognizing Overlapping Mentions with Mention Separators",
            "authors": [
                "Aldrian Obaja Muis",
                "Wei Lu"
            ],
            "abstract": "In this paper, we propose a new model that is capable of recognizing\noverlapping mentions. We introduce a novel notion of mention separators that\ncan be effectively used to capture how mentions overlap with one another. On\ntop of a novel multigraph representation that we introduce, we show that\nefficient and exact inference can still be performed. We present some\ntheoretical analysis on the differences between our model and a recently\nproposed model for recognizing overlapping mentions, and discuss the possible\nimplications of the differences. Through extensive empirical analysis on\nstandard datasets, we demonstrate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09154v3",
            "date": "2018-10-22 09:45:52+00:00",
            "title": "A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification",
            "authors": [
                "Ruizhe Li",
                "Chenghua Lin",
                "Matthew Collinson",
                "Xiao Li",
                "Guanyi Chen"
            ],
            "abstract": "Recognising dialogue acts (DA) is important for many natural language\nprocessing tasks such as dialogue generation and intention recognition. In this\npaper, we propose a dual-attention hierarchical recurrent neural network for DA\nclassification. Our model is partially inspired by the observation that\nconversational utterances are normally associated with both a DA and a topic,\nwhere the former captures the social act and the latter describes the subject\nmatter. However, such a dependency between DAs and topics has not been utilised\nby most existing systems for DA classification. With a novel dual task-specific\nattention mechanism, our model is able, for utterances, to capture information\nabout both DAs and topics, as well as information about the interactions\nbetween them. Experimental results show that by modelling topic as an auxiliary\ntask, our model can significantly improve DA classification, yielding better or\ncomparable performance to the state-of-the-art method on three public datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09164v1",
            "date": "2018-10-22 10:16:07+00:00",
            "title": "Named Entity Disambiguation using Deep Learning on Graphs",
            "authors": [
                "Alberto Cetoli",
                "Mohammad Akbari",
                "Stefano Bragaglia",
                "Andrew D. O'Harney",
                "Marc Sloan"
            ],
            "abstract": "We tackle \\ac{NED} by comparing entities in short sentences with \\wikidata{}\ngraphs. Creating a context vector from graphs through deep learning is a\nchallenging problem that has never been applied to \\ac{NED}. Our main\ncontribution is to present an experimental study of recent neural techniques,\nas well as a discussion about which graph features are most important for the\ndisambiguation task. In addition, a new dataset (\\wikidatadisamb{}) is created\nto allow a clean and scalable evaluation of \\ac{NED} with \\wikidata{} entries,\nand to be used as a reference in future research. In the end our results show\nthat a \\ac{Bi-LSTM} encoding of the graph triplets performs best, improving\nupon the baseline models and scoring an \\rm{F1} value of $91.6\\%$ on the\n\\wikidatadisamb{} test set",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09377v1",
            "date": "2018-10-22 15:48:52+00:00",
            "title": "Predictive Linguistic Features of Schizophrenia",
            "authors": [
                "Efsun Sarioglu Kayi",
                "Mona Diab",
                "Luca Pauselli",
                "Michael Compton",
                "Glen Coppersmith"
            ],
            "abstract": "Schizophrenia is one of the most disabling and difficult to treat of all\nhuman medical/health conditions, ranking in the top ten causes of disability\nworldwide. It has been a puzzle in part due to difficulty in identifying its\nbasic, fundamental components. Several studies have shown that some\nmanifestations of schizophrenia (e.g., the negative symptoms that include\nblunting of speech prosody, as well as the disorganization symptoms that lead\nto disordered language) can be understood from the perspective of linguistics.\nHowever, schizophrenia research has not kept pace with technologies in\ncomputational linguistics, especially in semantics and pragmatics. As such, we\nexamine the writings of schizophrenia patients analyzing their syntax,\nsemantics and pragmatics. In addition, we analyze tweets of (self pro-claimed)\nschizophrenia patients who publicly discuss their diagnoses. For writing\nsamples dataset, syntactic features are found to be the most successful in\nclassification whereas for the less structured Twitter dataset, a combination\nof features performed the best.",
            "journal ref": "Proceedings of the 6th Joint Conference on Lexical and\n  Computational Semantics (*SEM 2017), Association for Computational\n  Linguistics, 2017, pp. 241-250",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09379v1",
            "date": "2018-10-22 15:58:57+00:00",
            "title": "Linguistic Legal Concept Extraction in Portuguese",
            "authors": [
                "Alessandra Cid",
                "Alexandre Rademaker",
                "Bruno Cuconato",
                "Valeria de Paiva"
            ],
            "abstract": "This work investigates legal concepts and their expression in Portuguese,\nconcentrating on the \"Order of Attorneys of Brazil\" Bar exam. Using a corpus\nformed by a collection of multiple-choice questions, three norms related to the\nEthics part of the OAB exam, language resources (Princeton WordNet and\nOpenWordNet-PT) and tools (AntConc and Freeling), we began to investigate the\nconcepts and words missing from our repertory of concepts and words in\nPortuguese, the knowledge base OpenWordNet-PT. We add these concepts and words\nto OpenWordNet-PT and hence obtain a representation of these texts that is\n\"contained\" in the lexical knowledge base.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09506v1",
            "date": "2018-10-22 19:00:46+00:00",
            "title": "Automatically Detecting Self-Reported Birth Defect Outcomes on Twitter for Large-scale Epidemiological Research",
            "authors": [
                "Ari Z. Klein",
                "Abeed Sarker",
                "Davy Weissenbacher",
                "Graciela Gonzalez-Hernandez"
            ],
            "abstract": "In recent work, we identified and studied a small cohort of Twitter users\nwhose pregnancies with birth defect outcomes could be observed via their\npublicly available tweets. Exploiting social media's large-scale potential to\ncomplement the limited methods for studying birth defects, the leading cause of\ninfant mortality, depends on the further development of automatic methods. The\nprimary objective of this study was to take the first step towards scaling the\nuse of social media for observing pregnancies with birth defect outcomes,\nnamely, developing methods for automatically detecting tweets by users\nreporting their birth defect outcomes. We annotated and pre-processed\napproximately 23,000 tweets that mention birth defects in order to train and\nevaluate supervised machine learning algorithms, including feature-engineered\nand deep learning-based classifiers. We also experimented with various\nunder-sampling and over-sampling approaches to address the class imbalance. A\nSupport Vector Machine (SVM) classifier trained on the original, imbalanced\ndata set, with n-grams, word clusters, and structural features, achieved the\nbest baseline performance for the positive classes: an F1-score of 0.65 for the\n\"defect\" class and 0.51 for the \"possible defect\" class. Our contributions\ninclude (i) natural language processing (NLP) and supervised machine learning\nmethods for automatically detecting tweets by users reporting their birth\ndefect outcomes, (ii) a comparison of feature-engineered and deep\nlearning-based classifiers trained on imbalanced, under-sampled, and\nover-sampled data, and (iii) an error analysis that could inform classification\nimprovements using our publicly available corpus. Future work will focus on\nautomating user-level analyses for cohort inclusion.",
            "journal ref": "npj Digital Medicine. 2019;2:96",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09587v1",
            "date": "2018-10-22 22:47:48+00:00",
            "title": "Towards Universal Dialogue State Tracking",
            "authors": [
                "Liliang Ren",
                "Kaige Xie",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Dialogue state tracking is the core part of a spoken dialogue system. It\nestimates the beliefs of possible user's goals at every dialogue turn. However,\nfor most current approaches, it's difficult to scale to large dialogue domains.\nThey have one or more of following limitations: (a) Some models don't work in\nthe situation where slot values in ontology changes dynamically; (b) The number\nof model parameters is proportional to the number of slots; (c) Some models\nextract features based on hand-crafted lexicons. To tackle these challenges, we\npropose StateNet, a universal dialogue state tracker. It is independent of the\nnumber of values, shares parameters across all slots, and uses pre-trained word\nvectors instead of explicit semantic dictionaries. Our experiments on two\ndatasets show that our approach not only overcomes the limitations, but also\nsignificantly outperforms the performance of state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09609v1",
            "date": "2018-10-23 00:47:24+00:00",
            "title": "Neural Transition-based Syntactic Linearization",
            "authors": [
                "Linfeng Song",
                "Yue Zhang",
                "Daniel Gildea"
            ],
            "abstract": "The task of linearization is to find a grammatical order given a set of\nwords. Traditional models use statistical methods. Syntactic linearization\nsystems, which generate a sentence along with its syntactic tree, have shown\nstate-of-the-art performance. Recent work shows that a multi-layer LSTM\nlanguage model outperforms competitive statistical syntactic linearization\nsystems without using syntax. In this paper, we study neural syntactic\nlinearization, building a transition-based syntactic linearizer leveraging a\nfeed-forward neural network, observing significantly better results compared to\nLSTM language models on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09699v1",
            "date": "2018-10-23 07:33:06+00:00",
            "title": "Semi-supervised acoustic model training for speech with code-switching",
            "authors": [
                "Emre Y\u0131lmaz",
                "Mitchell McLaren",
                "Henk van den Heuvel",
                "David A. van Leeuwen"
            ],
            "abstract": "In the FAME! project, we aim to develop an automatic speech recognition (ASR)\nsystem for Frisian-Dutch code-switching (CS) speech extracted from the archives\nof a local broadcaster with the ultimate goal of building a spoken document\nretrieval system. Unlike Dutch, Frisian is a low-resourced language with a very\nlimited amount of manually annotated speech data. In this paper, we describe\nseveral automatic annotation approaches to enable using of a large amount of\nraw bilingual broadcast data for acoustic model training in a semi-supervised\nsetting. Previously, it has been shown that the best-performing ASR system is\nobtained by two-stage multilingual deep neural network (DNN) training using 11\nhours of manually annotated CS speech (reference) data together with speech\ndata from other high-resourced languages. We compare the quality of\ntranscriptions provided by this bilingual ASR system with several other\napproaches that use a language recognition system for assigning language labels\nto raw speech segments at the front-end and using monolingual ASR resources for\ntranscription. We further investigate automatic annotation of the speakers\nappearing in the raw broadcast data by first labeling with (pseudo) speaker\ntags using a speaker diarization system and then linking to the known speakers\nappearing in the reference data using a speaker recognition system. These\nspeaker labels are essential for speaker-adaptive training in the proposed\nsetting. We train acoustic models using the manually and automatically\nannotated data and run recognition experiments on the development and test data\nof the FAME! speech corpus to quantify the quality of the automatic\nannotations. The ASR and CS detection results demonstrate the potential of\nusing automatic language and speaker tagging in semi-supervised bilingual\nacoustic model training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09774v3",
            "date": "2018-10-23 11:06:55+00:00",
            "title": "Testing the Generalization Power of Neural Network Models Across NLI Benchmarks",
            "authors": [
                "Aarne Talman",
                "Stergios Chatzikyriakidis"
            ],
            "abstract": "Neural network models have been very successful in natural language\ninference, with the best models reaching 90% accuracy in some benchmarks.\nHowever, the success of these models turns out to be largely benchmark\nspecific. We show that models trained on a natural language inference dataset\ndrawn from one benchmark fail to perform well in others, even if the notion of\ninference assumed in these benchmarks is the same or similar. We train six high\nperforming neural network models on different datasets and show that each one\nof these has problems of generalizing when we replace the original test set\nwith a test set taken from another corpus designed for the same task. In light\nof these results, we argue that most of the current neural network models are\nnot able to generalize well in the task of natural language inference. We find\nthat using large pre-trained language models helps with transfer learning when\nthe datasets are similar enough. Our results also highlight that the current\nNLI datasets do not cover the different nuances of inference extensively\nenough.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09947v1",
            "date": "2018-10-23 16:27:07+00:00",
            "title": "Object-oriented lexical encoding of multiword expressions: Short and sweet",
            "authors": [
                "Agata Savary",
                "Simon Petitjean",
                "Timm Lichte",
                "Laura Kallmeyer",
                "Jakub Waszczuk"
            ],
            "abstract": "Multiword expressions (MWEs) exhibit both regular and idiosyncratic\nproperties. Their idiosyncrasy requires lexical encoding in parallel with their\ncomponent words. Their (at times intricate) regularity, on the other hand,\ncalls for means of flexible factorization to avoid redundant descriptions of\nshared properties. However, so far, non-redundant general-purpose lexical\nencoding of MWEs has not received a satisfactory solution. We offer a proof of\nconcept that this challenge might be effectively addressed within eXtensible\nMetaGrammar (XMG), an object-oriented metagrammar framework. We first make an\nexisting metagrammatical resource, the FrenchTAG grammar, MWE-aware. We then\nevaluate the factorization gain during incremental implementation with XMG on a\ndataset extracted from an MWE-annotated reference corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09995v1",
            "date": "2018-10-23 17:56:23+00:00",
            "title": "Deep Graph Convolutional Encoders for Structured Data to Text Generation",
            "authors": [
                "Diego Marcheggiani",
                "Laura Perez-Beltrachini"
            ],
            "abstract": "Most previous work on neural text generation from graph-structured data\nrelies on standard sequence-to-sequence methods. These approaches linearise the\ninput graph to be fed to a recurrent neural network. In this paper, we propose\nan alternative encoder based on graph convolutional networks that directly\nexploits the input structure. We report results on two graph-to-sequence\ndatasets that empirically show the benefits of explicitly encoding the input\ngraph structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10045v1",
            "date": "2018-10-23 18:44:55+00:00",
            "title": "Language Modeling at Scale",
            "authors": [
                "Mostofa Patwary",
                "Milind Chabbi",
                "Heewoo Jun",
                "Jiaji Huang",
                "Gregory Diamos",
                "Kenneth Church"
            ],
            "abstract": "We show how Zipf's Law can be used to scale up language modeling (LM) to take\nadvantage of more training data and more GPUs. LM plays a key role in many\nimportant natural language applications such as speech recognition and machine\ntranslation. Scaling up LM is important since it is widely accepted by the\ncommunity that there is no data like more data. Eventually, we would like to\ntrain on terabytes (TBs) of text (trillions of words). Modern training methods\nare far from this goal, because of various bottlenecks, especially memory\n(within GPUs) and communication (across GPUs). This paper shows how Zipf's Law\ncan address these bottlenecks by grouping parameters for common words and\ncharacter sequences, because $U \\ll N$, where $U$ is the number of unique words\n(types) and $N$ is the size of the training set (tokens). For a local batch\nsize $K$ with $G$ GPUs and a $D$-dimension embedding matrix, we reduce the\noriginal per-GPU memory and communication asymptotic complexity from\n$\\Theta(GKD)$ to $\\Theta(GK + UD)$. Empirically, we find $U \\propto\n(GK)^{0.64}$ on four publicly available large datasets. When we scale up the\nnumber of GPUs to 64, a factor of 8, training time speeds up by factors up to\n6.7$\\times$ (for character LMs) and 6.3$\\times$ (for word LMs) with negligible\nloss of accuracy. Our weak scaling on 192 GPUs on the Tieba dataset shows a\n35\\% improvement in LM prediction accuracy by training on 93 GB of data\n(2.5$\\times$ larger than publicly available SOTA dataset), but taking only\n1.25$\\times$ increase in training time, compared to 3 GB of the same dataset\nrunning on 6 GPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10254v2",
            "date": "2018-10-24 09:02:17+00:00",
            "title": "Learn to Code-Switch: Data Augmentation using Copy Mechanism on Language Modeling",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "Building large-scale datasets for training code-switching language models is\nchallenging and very expensive. To alleviate this problem using parallel corpus\nhas been a major workaround. However, existing solutions use linguistic\nconstraints which may not capture the real data distribution. In this work, we\npropose a novel method for learning how to generate code-switching sentences\nfrom parallel corpora. Our model uses a Seq2Seq model in combination with\npointer networks to align and choose words from the monolingual sentences and\nform a grammatical code-switching sentence. In our experiment, we show that by\ntraining a language model using the augmented sentences we improve the\nperplexity score by 10% compared to the LSTM baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10317v3",
            "date": "2018-10-24 12:16:55+00:00",
            "title": "Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation",
            "authors": [
                "Zaixiang Zheng",
                "Shujian Huang",
                "Zewei Sun",
                "Rongxiang Weng",
                "Xin-Yu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Previous studies show that incorporating external information could improve\nthe translation quality of Neural Machine Translation (NMT) systems. However,\nthere are inevitably noises in the external information, severely reducing the\nbenefit that the existing methods could receive from the incorporation. To\ntackle the problem, this study pays special attention to the discrimination of\nthe noises during the incorporation. We argue that there exist two kinds of\nnoise in this external information, i.e. global noise and local noise, which\naffect the translations for the whole sentence and for some specific words,\nrespectively. Accordingly, we propose a general framework that learns to\njointly discriminate both the global and local noises, so that the external\ninformation could be better leveraged. Our model is trained on the dataset\nderived from the original parallel corpus without any external labeled data or\nannotation. Experimental results in various real-world scenarios, language\npairs, and neural architectures indicate that discriminating noises contributes\nto significant improvements in translation quality by being able to better\nincorporate the external information, even in very noisy conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10320v1",
            "date": "2018-10-24 12:18:44+00:00",
            "title": "The MeMAD Submission to the IWSLT 2018 Speech Translation Task",
            "authors": [
                "Umut Sulubacak",
                "J\u00f6rg Tiedemann",
                "Aku Rouhe",
                "Stig-Arne Gr\u00f6nroos",
                "Mikko Kurimo"
            ],
            "abstract": "This paper describes the MeMAD project entry to the IWSLT Speech Translation\nShared Task, addressing the translation of English audio into German text.\nBetween the pipeline and end-to-end model tracks, we participated only in the\nformer, with three contrastive systems. We tried also the latter, but were not\nable to finish our end-to-end model in time.\n  All of our systems start by transcribing the audio into text through an\nautomatic speech recognition (ASR) model trained on the TED-LIUM English Speech\nRecognition Corpus (TED-LIUM). Afterwards, we feed the transcripts into\nEnglish-German text-based neural machine translation (NMT) models. Our systems\nemploy three different translation models trained on separate training sets\ncompiled from the English-German part of the TED Speech Translation Corpus\n(TED-Trans) and the OpenSubtitles2018 section of the OPUS collection.\n  In this paper, we also describe the experiments leading up to our final\nsystems. Our experiments indicate that using OpenSubtitles2018 in training\nsignificantly improves translation performance. We also experimented with\nvarious pre- and postprocessing routines for the NMT module, but we did not\nhave much success with these.\n  Our best-scoring system attains a BLEU score of 16.45 on the test set for\nthis year's task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10401v2",
            "date": "2018-10-24 13:46:58+00:00",
            "title": "Image-based Natural Language Understanding Using 2D Convolutional Neural Networks",
            "authors": [
                "Erinc Merdivan",
                "Anastasios Vafeiadis",
                "Dimitrios Kalatzis",
                "Sten Hanke",
                "Johannes Kropf",
                "Konstantinos Votis",
                "Dimitrios Giakoumis",
                "Dimitrios Tzovaras",
                "Liming Chen",
                "Raouf Hamzaoui",
                "Matthieu Geist"
            ],
            "abstract": "We propose a new approach to natural language understanding in which we\nconsider the input text as an image and apply 2D Convolutional Neural Networks\nto learn the local and global semantics of the sentences from the variations\nofthe visual patterns of words. Our approach demonstrates that it is possible\nto get semantically meaningful features from images with text without using\noptical character recognition and sequential processing pipelines, techniques\nthat traditional Natural Language Understanding algorithms require. To validate\nour approach, we present results for two applications: text classification and\ndialog modeling. Using a 2D Convolutional Neural Network, we were able to\noutperform the state-of-art accuracy results of non-Latin alphabet-based text\nclassification and achieved promising results for eight text classification\ndatasets. Furthermore, our approach outperformed the memory networks when using\nout of vocabulary entities fromtask 4 of the bAbI dialog dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10437v3",
            "date": "2018-10-24 15:07:19+00:00",
            "title": "Variational Semi-supervised Aspect-term Sentiment Analysis via Transformer",
            "authors": [
                "Xingyi Cheng",
                "Weidi Xu",
                "Taifeng Wang",
                "Wei Chu"
            ],
            "abstract": "Aspect-term sentiment analysis (ATSA) is a longstanding challenge in natural\nlanguage understanding. It requires fine-grained semantical reasoning about a\ntarget entity appeared in the text. As manual annotation over the aspects is\nlaborious and time-consuming, the amount of labeled data is limited for\nsupervised learning. This paper proposes a semi-supervised method for the ATSA\nproblem by using the Variational Autoencoder based on Transformer (VAET), which\nmodels the latent distribution via variational inference. By disentangling the\nlatent representation into the aspect-specific sentiment and the lexical\ncontext, our method induces the underlying sentiment prediction for the\nunlabeled data, which then benefits the ATSA classifier. Our method is\nclassifier agnostic, i.e., the classifier is an independent module and various\nadvanced supervised models can be integrated. Experimental results are obtained\non the SemEval 2014 task 4 and show that our method is effective with four\nclassical classifiers. The proposed method outperforms two general\nsemisupervised methods and achieves state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10566v2",
            "date": "2018-10-24 18:18:19+00:00",
            "title": "Clinical Concept Extraction with Contextual Word Embedding",
            "authors": [
                "Henghui Zhu",
                "Ioannis Ch. Paschalidis",
                "Amir Tahmasebi"
            ],
            "abstract": "Automatic extraction of clinical concepts is an essential step for turning\nthe unstructured data within a clinical note into structured and actionable\ninformation. In this work, we propose a clinical concept extraction model for\nautomatic annotation of clinical problems, treatments, and tests in clinical\nnotes utilizing domain-specific contextual word embedding. A contextual word\nembedding model is first trained on a corpus with a mixture of clinical reports\nand relevant Wikipedia pages in the clinical domain. Next, a bidirectional\nLSTM-CRF model is trained for clinical concept extraction using the contextual\nword embedding model. We tested our proposed model on the I2B2 2010 challenge\ndataset. Our proposed model achieved the best performance among reported\nbaseline models and outperformed the state-of-the-art models by 3.4% in terms\nof F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10639v1",
            "date": "2018-10-24 21:58:48+00:00",
            "title": "A Multilingual Study of Compressive Cross-Language Text Summarization",
            "authors": [
                "Elvys Linhares Pontes",
                "St\u00e9phane Huet",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Cross-Language Text Summarization (CLTS) generates summaries in a language\ndifferent from the language of the source documents. Recent methods use\ninformation from both languages to generate summaries with the most informative\nsentences. However, these methods have performance that can vary according to\nlanguages, which can reduce the quality of summaries. In this paper, we propose\na compressive framework to generate cross-language summaries. In order to\nanalyze performance and especially stability, we tested our system and\nextractive baselines on a dataset available in four languages (English, French,\nPortuguese, and Spanish) to generate English and French summaries. An automatic\nevaluation showed that our method outperformed extractive state-of-art CLTS\nmethods with better and more stable ROUGE scores for all languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10641v1",
            "date": "2018-10-24 22:08:26+00:00",
            "title": "Predicting the Semantic Textual Similarity with Siamese CNN and LSTM",
            "authors": [
                "Elvys Linhares Pontes",
                "St\u00e9phane Huet",
                "Andr\u00e9a Carneiro Linhares",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Semantic Textual Similarity (STS) is the basis of many applications in\nNatural Language Processing (NLP). Our system combines convolution and\nrecurrent neural networks to measure the semantic similarity of sentences. It\nuses a convolution network to take account of the local context of words and an\nLSTM to consider the global context of sentences. This combination of networks\nhelps to preserve the relevant information of sentences and improves the\ncalculation of the similarity between sentences. Our model has achieved good\nresults and is competitive with the best state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10647v2",
            "date": "2018-10-24 22:49:32+00:00",
            "title": "Multi-level Memory for Task Oriented Dialogs",
            "authors": [
                "Revanth Reddy",
                "Danish Contractor",
                "Dinesh Raghu",
                "Sachindra Joshi"
            ],
            "abstract": "Recent end-to-end task oriented dialog systems use memory architectures to\nincorporate external knowledge in their dialogs. Current work makes simplifying\nassumptions about the structure of the knowledge base, such as the use of\ntriples to represent knowledge, and combines dialog utterances (context) as\nwell as knowledge base (KB) results as part of the same memory. This causes an\nexplosion in the memory size, and makes the reasoning over memory harder. In\naddition, such a memory design forces hierarchical properties of the data to be\nfit into a triple structure of memory. This requires the memory reader to infer\nrelationships across otherwise connected attributes. In this paper we relax the\nstrong assumptions made by existing architectures and separate memories used\nfor modeling dialog context and KB results. Instead of using triples to store\nKB results, we introduce a novel multi-level memory architecture consisting of\ncells for each query and their corresponding results. The multi-level memory\nfirst addresses queries, followed by results and finally each key-value pair\nwithin a result. We conduct detailed experiments on three publicly available\ntask oriented dialog data sets and we find that our method conclusively\noutperforms current state-of-the-art models. We report a 15-25% increase in\nboth entity F1 and BLEU scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10752v1",
            "date": "2018-10-25 07:50:17+00:00",
            "title": "Word Embedding based Edit Distance",
            "authors": [
                "Yilin Niu",
                "Chao Qiao",
                "Hang Li",
                "Minlie Huang"
            ],
            "abstract": "Text similarity calculation is a fundamental problem in natural language\nprocessing and related fields. In recent years, deep neural networks have been\ndeveloped to perform the task and high performances have been achieved. The\nneural networks are usually trained with labeled data in supervised learning,\nand creation of labeled data is usually very costly. In this short paper, we\naddress unsupervised learning for text similarity calculation. We propose a new\nmethod called Word Embedding based Edit Distance (WED), which incorporates word\nembedding into edit distance. Experiments on three benchmark datasets show WED\noutperforms state-of-the-art unsupervised methods including edit distance,\nTF-IDF based cosine, word embedding based cosine, Jaccard index, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10797v1",
            "date": "2018-10-25 09:16:52+00:00",
            "title": "The Logoscope: a Semi-Automatic Tool for Detecting and Documenting French New Words",
            "authors": [
                "Ingrid Falk",
                "Delphine Bernhard",
                "Christophe G\u00e9rard"
            ],
            "abstract": "In this article we present the design and implementation of the Logoscope,\nthe first tool especially developed to detect new words of the French language,\nto document them and allow a public access through a web interface.\n  This semi-automatic tool collects new words daily by browsing the online\nversions of French well known newspapers such as Le Monde, Le Figaro, L'Equipe,\nLib\\'eration, La Croix, Les \\'Echos. In contrast to other existing tools\nessentially dedicated to dictionary development, the Logoscope attempts to give\na more complete account of the context in which the new words occur. In\naddition to the commonly given morpho-syntactic information it also provides\ninformation about the textual and discursive contexts of the word creation; in\nparticular, it automatically determines the (journalistic) topics of the text\ncontaining the new word.\n  In this article we first give a general overview of the developed tool. We\nthen describe the approach taken, we discuss the linguistic background which\nguided our design decisions and present the computational methods we used to\nimplement it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10882v1",
            "date": "2018-10-25 13:58:20+00:00",
            "title": "Dynamic Oracles for Top-Down and In-Order Shift-Reduce Constituent Parsing",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We introduce novel dynamic oracles for training two of the most accurate\nknown shift-reduce algorithms for constituent parsing: the top-down and\nin-order transition-based parsers. In both cases, the dynamic oracles manage to\nnotably increase their accuracy, in comparison to that obtained by performing\nclassic static training. In addition, by improving the performance of the\nstate-of-the-art in-order shift-reduce parser, we achieve the best accuracy to\ndate (92.0 F1) obtained by a fully-supervised single-model greedy shift-reduce\nconstituent parser on the WSJ benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10949v3",
            "date": "2018-10-25 16:08:18+00:00",
            "title": "Learning Emotion from 100 Observations: Unexpected Robustness of Deep Learning under Strong Data Limitations",
            "authors": [
                "Sven Buechel",
                "Jo\u00e3o Sedoc",
                "H. Andrew Schwartz",
                "Lyle Ungar"
            ],
            "abstract": "One of the major downsides of Deep Learning is its supposed need for vast\namounts of training data. As such, these techniques appear ill-suited for NLP\nareas where annotated data is limited, such as less-resourced languages or\nemotion analysis, with its many nuanced and hard-to-acquire annotation formats.\nWe conduct a questionnaire study indicating that indeed the vast majority of\nresearchers in emotion analysis deems neural models inferior to traditional\nmachine learning when training data is limited. In stark contrast to those\nsurvey results, we provide empirical evidence for English, Polish, and\nPortuguese that commonly used neural architectures can be trained on\nsurprisingly few observations, outperforming $n$-gram based ridge regression on\nonly 100 data points. Our analysis suggests that high-quality, pre-trained word\nembeddings are a main factor for achieving those results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11067v1",
            "date": "2018-10-25 18:54:49+00:00",
            "title": "Teaching Syntax by Adversarial Distraction",
            "authors": [
                "Juho Kim",
                "Christopher Malon",
                "Asim Kadav"
            ],
            "abstract": "Existing entailment datasets mainly pose problems which can be answered\nwithout attention to grammar or word order. Learning syntax requires comparing\nexamples where different grammar and word order change the desired\nclassification. We introduce several datasets based on synthetic\ntransformations of natural entailment examples in SNLI or FEVER, to teach\naspects of grammar and word order. We show that without retraining, popular\nentailment models are unaware that these syntactic differences change meaning.\nWith retraining, some but not all popular entailment models can learn to\ncompare the syntax properly.",
            "journal ref": "Juho Kim, Christopher Malon, and Asim Kadav. 2018. \"Teaching\n  Syntax by Adversarial Distraction.\" Proceedings of the EMNLP First Workshop\n  on Fact Extraction and Verification",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11101v2",
            "date": "2018-10-25 20:42:40+00:00",
            "title": "UniMorph 2.0: Universal Morphology",
            "authors": [
                "Christo Kirov",
                "Ryan Cotterell",
                "John Sylak-Glassman",
                "G\u00e9raldine Walther",
                "Ekaterina Vylomova",
                "Patrick Xia",
                "Manaal Faruqui",
                "Sabrina J. Mielke",
                "Arya D. McCarthy",
                "Sandra K\u00fcbler",
                "David Yarowsky",
                "Jason Eisner",
                "Mans Hulden"
            ],
            "abstract": "The Universal Morphology UniMorph project is a collaborative effort to\nimprove how NLP handles complex morphology across the world's languages. The\nproject releases annotated morphological data using a universal tagset, the\nUniMorph schema. Each inflected form is associated with a lemma, which\ntypically carries its underlying lexical meaning, and a bundle of morphological\nfeatures from our schema. Additional supporting data and tools are also\nreleased on a per-language basis when available. UniMorph is based at the\nCenter for Language and Speech Processing (CLSP) at Johns Hopkins University in\nBaltimore, Maryland and is sponsored by the DARPA LORELEI program. This paper\ndetails advances made to the collection, annotation, and dissemination of\nproject resources since the initial UniMorph release described at LREC 2016.\nlexical resources} }",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11118v2",
            "date": "2018-10-25 21:44:14+00:00",
            "title": "A Large-Scale Corpus for Conversation Disentanglement",
            "authors": [
                "Jonathan K. Kummerfeld",
                "Sai R. Gouravajhala",
                "Joseph Peper",
                "Vignesh Athreya",
                "Chulaka Gunasekara",
                "Jatin Ganhotra",
                "Siva Sankalp Patel",
                "Lazaros Polymenakos",
                "Walter S. Lasecki"
            ],
            "abstract": "Disentangling conversations mixed together in a single stream of messages is\na difficult task, made harder by the lack of large manually annotated datasets.\nWe created a new dataset of 77,563 messages manually annotated with\nreply-structure graphs that both disentangle conversations and define internal\nconversation structure. Our dataset is 16 times larger than all previously\nreleased datasets combined, the first to include adjudication of annotation\ndisagreements, and the first to include context. We use our data to re-examine\nprior work, in particular, finding that 80% of conversations in a widely used\ndialogue corpus are either missing messages or contain extra messages. Our\nmanually-annotated data presents an opportunity to develop robust data-driven\nmethods for conversation disentanglement, which will help advance dialogue\nresearch.",
            "journal ref": "ACL (2019) 3846-3856",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11351v1",
            "date": "2018-10-26 14:41:37+00:00",
            "title": "Static and Dynamic Vector Semantics for Lambda Calculus Models of Natural Language",
            "authors": [
                "Mehrnoosh Sadrzadeh",
                "Reinhard Muskens"
            ],
            "abstract": "Vector models of language are based on the contextual aspects of language,\nthe distributions of words and how they co-occur in text. Truth conditional\nmodels focus on the logical aspects of language, compositional properties of\nwords and how they compose to form sentences. In the truth conditional\napproach, the denotation of a sentence determines its truth conditions, which\ncan be taken to be a truth value, a set of possible worlds, a context change\npotential, or similar. In the vector models, the degree of co-occurrence of\nwords in context determines how similar the meanings of words are. In this\npaper, we put these two models together and develop a vector semantics for\nlanguage based on the simply typed lambda calculus models of natural language.\nWe provide two types of vector semantics: a static one that uses techniques\nfamiliar from the truth conditional tradition and a dynamic one based on a form\nof dynamic interpretation inspired by Heim's context change potentials. We show\nhow the dynamic model can be applied to entailment between a corpus and a\nsentence and we provide examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11476v2",
            "date": "2018-10-26 18:10:03+00:00",
            "title": "Named Person Coreference in English News",
            "authors": [
                "Oshin Agarwal",
                "Sanjay Subramanian",
                "Ani Nenkova",
                "Dan Roth"
            ],
            "abstract": "People are often entities of interest in tasks such as search and information\nextraction. In these tasks, the goal is to find as much information as possible\nabout people specified by their name. However in text, some of the references\nto people are by pronouns (she, his) or generic descriptions (the professor,\nthe German chancellor). It is therefore important that coreference resolution\nsystems are able to link these different types of mentions to the correct\nperson name. Here, we evaluate two state of the art coreference resolution\nsystems on the subtask of Named Person Coreference, in which we are interested\nin identifying a person mentioned by name, along with all other mentions of the\nperson, by pronoun or generic noun phrase. Our analysis reveals that standard\ncoreference metrics do not reflect adequately the requirements in this task:\nthey do not penalize systems for not identifying any mentions by name and they\nreward systems even if systems find correctly mentions to the same entity but\nfail to link these to a proper name (she--the student---no name). We introduce\nnew metrics for evaluating named person coreference that address these\ndiscrepancies. We present a simple rule-based named entity recognition driven\nsystem, which outperforms the current state-of-the-art systems on these\ntask-specific metrics and performs on par with them on traditional coreference\nevaluations. Finally, we present similar evaluation for coreference resolution\nof other named entities and show that the rule-based approach is effective only\nfor person named coreference, not other named entity types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11481v1",
            "date": "2018-10-26 18:19:29+00:00",
            "title": "Can Entropy Explain Successor Surprisal Effects in Reading?",
            "authors": [
                "Marten van Schijndel",
                "Tal Linzen"
            ],
            "abstract": "Human reading behavior is sensitive to surprisal: more predictable words tend\nto be read faster. Unexpectedly, this applies not only to the surprisal of the\nword that is currently being read, but also to the surprisal of upcoming\n(successor) words that have not been fixated yet. This finding has been\ninterpreted as evidence that readers can extract lexical information\nparafoveally. Calling this interpretation into question, Angele et al. (2015)\nshowed that successor effects appear even in contexts in which those successor\nwords are not yet visible. They hypothesized that successor surprisal predicts\nreading time because it approximates the reader's uncertainty about upcoming\nwords. We test this hypothesis on a reading time corpus using an LSTM language\nmodel, and find that successor surprisal and entropy are independent predictors\nof reading time. This independence suggests that entropy alone is unlikely to\nbe the full explanation for successor surprisal effects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11663v1",
            "date": "2018-10-27 15:24:15+00:00",
            "title": "Suspicious News Detection Using Micro Blog Text",
            "authors": [
                "Tsubasa Tagami",
                "Hiroki Ouchi",
                "Hiroki Asano",
                "Kazuaki Hanawa",
                "Kaori Uchiyama",
                "Kaito Suzuki",
                "Kentaro Inui",
                "Atsushi Komiya",
                "Atsuo Fujimura",
                "Hitofumi Yanai",
                "Ryo Yamashita",
                "Akinori Machino"
            ],
            "abstract": "We present a new task, suspicious news detection using micro blog text. This\ntask aims to support human experts to detect suspicious news articles to be\nverified, which is costly but a crucial step before verifying the truthfulness\nof the articles. Specifically, in this task, given a set of posts on SNS\nreferring to a news article, the goal is to judge whether the article is to be\nverified or not. For this task, we create a publicly available dataset in\nJapanese and provide benchmark results by using several basic machine learning\ntechniques. Experimental results show that our models can reduce the cost of\nmanual fact-checking process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11735v1",
            "date": "2018-10-28 00:19:26+00:00",
            "title": "Middle-Out Decoding",
            "authors": [
                "Shikib Mehri",
                "Leonid Sigal"
            ],
            "abstract": "Despite being virtually ubiquitous, sequence-to-sequence models are\nchallenged by their lack of diversity and inability to be externally\ncontrolled. In this paper, we speculate that a fundamental shortcoming of\nsequence generation models is that the decoding is done strictly from\nleft-to-right, meaning that outputs values generated earlier have a profound\neffect on those generated later. To address this issue, we propose a novel\nmiddle-out decoder architecture that begins from an initial middle-word and\nsimultaneously expands the sequence in both directions. To facilitate\ninformation flow and maintain consistent decoding, we introduce a dual\nself-attention mechanism that allows us to model complex dependencies between\nthe outputs. We illustrate the performance of our model on the task of video\ncaptioning, as well as a synthetic sequence de-noising task. Our middle-out\ndecoder achieves significant improvements on de-noising and competitive\nperformance in the task of video captioning, while quantifiably improving the\ncaption diversity. Furthermore, we perform a qualitative analysis that\ndemonstrates our ability to effectively control the generation process of our\ndecoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11895v3",
            "date": "2018-10-28 22:15:32+00:00",
            "title": "Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training",
            "authors": [
                "Hila Gonen",
                "Yoav Goldberg"
            ],
            "abstract": "We focus on the problem of language modeling for code-switched language, in\nthe context of automatic speech recognition (ASR). Language modeling for\ncode-switched language is challenging for (at least) three reasons: (1) lack of\navailable large-scale code-switched data for training; (2) lack of a replicable\nevaluation setup that is ASR directed yet isolates language modeling\nperformance from the other intricacies of the ASR system; and (3) the reliance\non generative modeling. We tackle these three issues: we propose an\nASR-motivated evaluation setup which is decoupled from an ASR system and the\nchoice of vocabulary, and provide an evaluation dataset for English-Spanish\ncode-switching. This setup lends itself to a discriminative training approach,\nwhich we demonstrate to work better than generative language modeling. Finally,\nwe explore a variety of training protocols and verify the effectiveness of\ntraining with large amounts of monolingual data followed by fine-tuning with\nsmall amounts of code-switched data, for both the generative and discriminative\ncases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11955v1",
            "date": "2018-10-20 17:07:42+00:00",
            "title": "Improving Context Modelling in Multimodal Dialogue Generation",
            "authors": [
                "Shubham Agarwal",
                "Ondrej Dusek",
                "Ioannis Konstas",
                "Verena Rieser"
            ],
            "abstract": "In this work, we investigate the task of textual response generation in a\nmultimodal task-oriented dialogue system. Our work is based on the recently\nreleased Multimodal Dialogue (MMD) dataset (Saha et al., 2017) in the fashion\ndomain. We introduce a multimodal extension to the Hierarchical Recurrent\nEncoder-Decoder (HRED) model and show that this extension outperforms strong\nbaselines in terms of text-based similarity metrics. We also showcase the\nshortcomings of current vision and language models by performing an error\nanalysis on our system's output.",
            "journal ref": "Proceedings of the 11th International Conference on Natural\n  Language Generation, pages 129-134, Tilburg, The Netherlands, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12097v1",
            "date": "2018-10-22 14:26:13+00:00",
            "title": "Ruuh: A Deep Learning Based Conversational Social Agent",
            "authors": [
                "Sonam Damani",
                "Nitya Raviprakash",
                "Umang Gupta",
                "Ankush Chatterjee",
                "Meghana Joshi",
                "Khyatti Gupta",
                "Kedhar Nath Narahari",
                "Puneet Agrawal",
                "Manoj Kumar Chinnakotla",
                "Sneha Magapu",
                "Abhishek Mathur"
            ],
            "abstract": "Dialogue systems and conversational agents are becoming increasingly popular\nin the modern society but building an agent capable of holding intelligent\nconversation with its users is a challenging problem for artificial\nintelligence. In this demo, we demonstrate a deep learning based conversational\nsocial agent called \"Ruuh\" (facebook.com/Ruuh) designed by a team at Microsoft\nIndia to converse on a wide range of topics. Ruuh needs to think beyond the\nutilitarian notion of merely generating \"relevant\" responses and meet a wider\nrange of user social needs, like expressing happiness when user's favorite team\nwins, sharing a cute comment on showing the pictures of the user's pet and so\non. The agent also needs to detect and respond to abusive language, sensitive\ntopics and trolling behavior of the users. Many of these problems pose\nsignificant research challenges which will be demonstrated in our demo. Our\nagent has interacted with over 2 million real world users till date which has\ngenerated over 150 million user conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12196v1",
            "date": "2018-10-29 15:35:18+00:00",
            "title": "ReviewQA: a relational aspect-based opinion reading dataset",
            "authors": [
                "Quentin Grail",
                "Julien Perez"
            ],
            "abstract": "Deep reading models for question-answering have demonstrated promising\nperformance over the last couple of years. However current systems tend to\nlearn how to cleverly extract a span of the source document, based on its\nsimilarity with the question, instead of seeking for the appropriate answer.\nIndeed, a reading machine should be able to detect relevant passages in a\ndocument regarding a question, but more importantly, it should be able to\nreason over the important pieces of the document in order to produce an answer\nwhen it is required. To motivate this purpose, we present ReviewQA, a\nquestion-answering dataset based on hotel reviews. The questions of this\ndataset are linked to a set of relational understanding competencies that we\nexpect a model to master. Indeed, each question comes with an associated type\nthat characterizes the required competency. With this framework, it is possible\nto benchmark the main families of models and to get an overview of what are the\nstrengths and the weaknesses of a given model on the set of tasks evaluated in\nthis dataset. Our corpus contains more than 500.000 questions in natural\nlanguage over 100.000 hotel reviews. Our setup is projective, the answer of a\nquestion does not need to be extracted from a document, like in most of the\nrecent datasets, but selected among a set of candidates that contains all the\npossible answers to the questions of the dataset. Finally, we present several\nbaselines over this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12264v2",
            "date": "2018-10-29 17:23:33+00:00",
            "title": "Learning Comment Generation by Leveraging User-Generated Data",
            "authors": [
                "Zhaojiang Lin",
                "Genta Indra Winata",
                "Pascale Fung"
            ],
            "abstract": "Existing models on open-domain comment generation are difficult to train, and\nthey produce repetitive and uninteresting responses. The problem is due to\nmultiple and contradictory responses from a single article, and by the rigidity\nof retrieval methods. To solve this problem, we propose a combined approach to\nretrieval and generation methods. We propose an attentive scorer to retrieve\ninformative and relevant comments by leveraging user-generated data. Then, we\nuse such comments, together with the article, as input for a\nsequence-to-sequence model with copy mechanism. We show the robustness of our\nmodel and how it can alleviate the aforementioned issue by using a large scale\ncomment generation dataset. The result shows that the proposed generative model\nsignificantly outperforms strong baseline such as Seq2Seq with attention and\nInformation Retrieval models by around 27 and 30 BLEU-1 points respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12343v2",
            "date": "2018-10-29 18:42:46+00:00",
            "title": "Content Selection in Deep Learning Models of Summarization",
            "authors": [
                "Chris Kedzie",
                "Kathleen McKeown",
                "Hal Daume III"
            ],
            "abstract": "We carry out experiments with deep learning models of summarization across\nthe domains of news, personal stories, meetings, and medical articles in order\nto understand how content selection is performed. We find that many\nsophisticated features of state of the art extractive summarizers do not\nimprove performance over simpler models. These results suggest that it is\neasier to create a summarizer for a new domain than previous work suggests and\nbring into question the benefit of deep learning models for summarization for\nthose domains that do have massive datasets (i.e., news). At the same time,\nthey suggest important questions for new research in summarization; namely, new\nforms of sentence representations or external knowledge sources are needed that\nare better suited to the summarization task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12349v2",
            "date": "2018-10-29 18:57:30+00:00",
            "title": "Multi-label Multi-task Deep Learning for Behavioral Coding",
            "authors": [
                "James Gibson",
                "David C. Atkins",
                "Torrey Creed",
                "Zac Imel",
                "Panayiotis Georgiou",
                "Shrikanth Narayanan"
            ],
            "abstract": "We propose a methodology for estimating human behaviors in psychotherapy\nsessions using mutli-label and multi-task learning paradigms. We discuss the\nproblem of behavioral coding in which data of human interactions is the\nannotated with labels to describe relevant human behaviors of interest. We\ndescribe two related, yet distinct, corpora consisting of therapist client\ninteractions in psychotherapy sessions. We experimentally compare the proposed\nlearning approaches for estimating behaviors of interest in these datasets.\nSpecifically, we compare single and multiple label learning approaches, single\nand multiple task learning approaches, and evaluate the performance of these\napproaches when incorporating turn context. We demonstrate the prediction\nperformance gains which can be achieved by using the proposed paradigms and\ndiscuss the insights these models provide into these complex interactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12368v5",
            "date": "2018-10-29 19:22:12+00:00",
            "title": "A Pragmatic Guide to Geoparsing Evaluation",
            "authors": [
                "Milan Gritta",
                "Mohammad Taher Pilehvar",
                "Nigel Collier"
            ],
            "abstract": "Empirical methods in geoparsing have thus far lacked a standard evaluation\nframework describing the task, metrics and data used to compare\nstate-of-the-art systems. Evaluation is further made inconsistent, even\nunrepresentative of real-world usage by the lack of distinction between the\ndifferent types of toponyms, which necessitates new guidelines, a consolidation\nof metrics and a detailed toponym taxonomy with implications for Named Entity\nRecognition (NER) and beyond. To address these deficiencies, our manuscript\nintroduces a new framework in three parts. Part 1) Task Definition: clarified\nvia corpus linguistic analysis proposing a fine-grained Pragmatic Taxonomy of\nToponyms. Part 2) Metrics: discussed and reviewed for a rigorous evaluation\nincluding recommendations for NER/Geoparsing practitioners. Part 3) Evaluation\nData: shared via a new dataset called GeoWebNews to provide test/train examples\nand enable immediate use of our contributions. In addition to fine-grained\nGeotagging and Toponym Resolution (Geocoding), this dataset is also suitable\nfor prototyping and evaluating machine learning NLP models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12443v1",
            "date": "2018-10-29 22:39:49+00:00",
            "title": "Learning Better Internal Structure of Words for Sequence Labeling",
            "authors": [
                "Yingwei Xin",
                "Ethan Hart",
                "Vibhuti Mahajan",
                "Jean-David Ruvini"
            ],
            "abstract": "Character-based neural models have recently proven very useful for many NLP\ntasks. However, there is a gap of sophistication between methods for learning\nrepresentations of sentences and words. While most character models for\nlearning representations of sentences are deep and complex, models for learning\nrepresentations of words are shallow and simple. Also, in spite of considerable\nresearch on learning character embeddings, it is still not clear which kind of\narchitecture is the best for capturing character-to-word representations. To\naddress these questions, we first investigate the gaps between methods for\nlearning word and sentence representations. We conduct detailed experiments and\ncomparisons of different state-of-the-art convolutional models, and also\ninvestigate the advantages and disadvantages of their constituents.\nFurthermore, we propose IntNet, a funnel-shaped wide convolutional neural\narchitecture with no down-sampling for learning representations of the internal\nstructure of words by composing their characters from limited, supervised\ntraining corpora. We evaluate our proposed model on six sequence labeling\ndatasets, including named entity recognition, part-of-speech tagging, and\nsyntactic chunking. Our in-depth analysis shows that IntNet significantly\noutperforms other character embedding models and obtains new state-of-the-art\nperformance without relying on any external knowledge or resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12546v1",
            "date": "2018-10-30 06:55:23+00:00",
            "title": "Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks",
            "authors": [
                "Biao Zhang",
                "Deyi Xiong",
                "Jinsong Su",
                "Qian Lin",
                "Huiji Zhang"
            ],
            "abstract": "In this paper, we propose an additionsubtraction twin-gated recurrent network\n(ATR) to simplify neural machine translation. The recurrent units of ATR are\nheavily simplified to have the smallest number of weight matrices among units\nof all existing gated RNNs. With the simple addition and subtraction operation,\nwe introduce a twin-gated mechanism to build input and forget gates which are\nhighly correlated. Despite this simplification, the essential non-linearities\nand capability of modeling long-distance dependencies are preserved.\nAdditionally, the proposed ATR is more transparent than LSTM/GRU due to the\nsimplification. Forward self-attention can be easily established in ATR, which\nmakes the proposed network interpretable. Experiments on WMT14 translation\ntasks demonstrate that ATR-based neural machine translation can yield\ncompetitive performance on English- German and English-French language pairs in\nterms of both translation quality and speed. Further experiments on NIST\nChinese-English translation, natural language inference and Chinese word\nsegmentation verify the generality and applicability of ATR on different\nnatural language processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12557v1",
            "date": "2018-10-30 07:33:52+00:00",
            "title": "Machine Translation between Vietnamese and English: an Empirical Study",
            "authors": [
                "Hong-Hai Phan-Vu",
                "Viet-Trung Tran",
                "Van-Nam Nguyen",
                "Hoang-Vu Dang",
                "Phan-Thuan Do"
            ],
            "abstract": "Machine translation is shifting to an end-to-end approach based on deep\nneural networks. The state of the art achieves impressive results for popular\nlanguage pairs such as English - French or English - Chinese. However for\nEnglish - Vietnamese the shortage of parallel corpora and expensive\nhyper-parameter search present practical challenges to neural-based approaches.\nThis paper highlights our efforts on improving English-Vietnamese translations\nin two directions: (1) Building the largest open Vietnamese - English corpus to\ndate, and (2) Extensive experiments with the latest neural models to achieve\nthe highest BLEU scores. Our experiments provide practical examples of\neffectively employing different neural machine translation models with\nlow-resource language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12579v1",
            "date": "2018-10-30 08:42:52+00:00",
            "title": "Exploring Neural Methods for Parsing Discourse Representation Structures",
            "authors": [
                "Rik van Noord",
                "Lasha Abzianidze",
                "Antonio Toral",
                "Johan Bos"
            ],
            "abstract": "Neural methods have had several recent successes in semantic parsing, though\nthey have yet to face the challenge of producing meaning representations based\non formal semantics. We present a sequence-to-sequence neural semantic parser\nthat is able to produce Discourse Representation Structures (DRSs) for English\nsentences with high accuracy, outperforming traditional DRS parsers. To\nfacilitate the learning of the output, we represent DRSs as a sequence of flat\nclauses and introduce a method to verify that produced DRSs are well-formed and\ninterpretable. We compare models using characters and words as input and see\n(somewhat surprisingly) that the former performs better than the latter. We\nshow that eliminating variable names from the output using De Bruijn-indices\nincreases parser performance. Adding silver training data boosts performance\neven further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12594v1",
            "date": "2018-10-30 09:14:37+00:00",
            "title": "Subword Encoding in Lattice LSTM for Chinese Word Segmentation",
            "authors": [
                "Jie Yang",
                "Yue Zhang",
                "Shuailong Liang"
            ],
            "abstract": "We investigate a lattice LSTM network for Chinese word segmentation (CWS) to\nutilize words or subwords. It integrates the character sequence features with\nall subsequences information matched from a lexicon. The matched subsequences\nserve as information shortcut tunnels which link their start and end characters\ndirectly. Gated units are used to control the contribution of multiple input\nlinks. Through formula derivation and comparison, we show that the lattice LSTM\nis an extension of the standard LSTM with the ability to take multiple inputs.\nPrevious lattice LSTM model takes word embeddings as the lexicon input, we\nprove that subword encoding can give the comparable performance and has the\nbenefit of not relying on any external segmentor. The contribution of lattice\nLSTM comes from both lexicon and pretrained embeddings information, we find\nthat the lexicon information contributes more than the pretrained embeddings\ninformation through controlled experiments. Our experiments show that the\nlattice structure with subword encoding gives competitive or better results\nwith previous state-of-the-art methods on four segmentation benchmarks.\nDetailed analyses are conducted to compare the performance of word encoding and\nsubword encoding in lattice LSTM. We also investigate the performance of\nlattice LSTM structure under different circumstances and when this model works\nor fails.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12620v1",
            "date": "2018-10-30 10:11:35+00:00",
            "title": "Towards End-to-end Automatic Code-Switching Speech Recognition",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "Speech recognition in mixed language has difficulties to adapt end-to-end\nframework due to the lack of data and overlapping phone sets, for example in\nwords such as \"one\" in English and \"w\\`an\" in Chinese. We propose a CTC-based\nend-to-end automatic speech recognition model for intra-sentential\nEnglish-Mandarin code-switching. The model is trained by joint training on\nmonolingual datasets, and fine-tuning with the mixed-language corpus. During\nthe decoding process, we apply a beam search and combine CTC predictions and\nlanguage model score. The proposed method is effective in leveraging\nmonolingual corpus and detecting language transitions and it improves the CER\nby 5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12646v1",
            "date": "2018-10-30 10:53:42+00:00",
            "title": "Prosodic entrainment in dialog acts",
            "authors": [
                "Uwe D. Reichel",
                "Katalin M\u00e1dy",
                "Jennifer Cole"
            ],
            "abstract": "We examined prosodic entrainment in spoken dialogs separately for several\ndialog acts in cooperative and competitive games. Entrainment was measured for\nintonation features derived from a superpositional intonation stylization as\nwell as for rhythm features. The found differences can be related to the\ncooperative or competitive nature of the game, as well as to dialog act\nproperties as its intrinsic authority, supportiveness and distributional\ncharacteristics. In cooperative games dialog acts with a high authority given\nby knowledge and with a high frequency showed the most entrainment. The results\nare discussed amongst others with respect to the degree of active entrainment\ncontrol in cooperative behavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12686v2",
            "date": "2018-10-30 11:58:16+00:00",
            "title": "Evaluating Text GANs as Language Models",
            "authors": [
                "Guy Tevet",
                "Gavriel Habib",
                "Vered Shwartz",
                "Jonathan Berant"
            ],
            "abstract": "Generative Adversarial Networks (GANs) are a promising approach for text\ngeneration that, unlike traditional language models (LM), does not suffer from\nthe problem of ``exposure bias''. However, A major hurdle for understanding the\npotential of GANs for text generation is the lack of a clear evaluation metric.\nIn this work, we propose to approximate the distribution of text generated by a\nGAN, which permits evaluating them with traditional probability-based LM\nmetrics. We apply our approximation procedure on several GAN-based models and\nshow that they currently perform substantially worse than state-of-the-art LMs.\nOur evaluation procedure promotes better understanding of the relation between\nGANs and LMs, and can accelerate progress in GAN-based text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12703v1",
            "date": "2018-10-30 12:33:03+00:00",
            "title": "Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation",
            "authors": [
                "Benjamin Marie",
                "Atsushi Fujita"
            ],
            "abstract": "Recent work achieved remarkable results in training neural machine\ntranslation (NMT) systems in a fully unsupervised way, with new and dedicated\narchitectures that rely on monolingual corpora only. In this work, we propose\nto define unsupervised NMT (UNMT) as NMT trained with the supervision of\nsynthetic bilingual data. Our approach straightforwardly enables the use of\nstate-of-the-art architectures proposed for supervised NMT by replacing\nhuman-made bilingual data with synthetic bilingual data for training. We\npropose to initialize the training of UNMT with synthetic bilingual data\ngenerated by unsupervised statistical machine translation (USMT). The UNMT\nsystem is then incrementally improved using back-translation. Our preliminary\nexperiments show that our approach achieves a new state-of-the-art for\nunsupervised machine translation on the WMT16 German--English news translation\ntask, for both translation directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12836v4",
            "date": "2018-10-30 16:18:05+00:00",
            "title": "Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model",
            "authors": [
                "Muthuraman Chidambaram",
                "Yinfei Yang",
                "Daniel Cer",
                "Steve Yuan",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "A significant roadblock in multilingual neural language modeling is the lack\nof labeled non-English data. One potential method for overcoming this issue is\nlearning cross-lingual text representations that can be used to transfer the\nperformance from training on English tasks to non-English tasks, despite little\nto no task-specific non-English data. In this paper, we explore a natural setup\nfor learning cross-lingual sentence representations: the dual-encoder. We\nprovide a comprehensive evaluation of our cross-lingual representations on a\nnumber of monolingual, cross-lingual, and zero-shot/few-shot learning tasks,\nand also give an analysis of different learned cross-lingual embedding spaces.",
            "journal ref": "In Proceedings of the 4th Workshop on Representation Learning for\n  NLP (RepL4NLP-2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12885v1",
            "date": "2018-10-30 17:32:16+00:00",
            "title": "ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension",
            "authors": [
                "Sheng Zhang",
                "Xiaodong Liu",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "We present a large-scale dataset, ReCoRD, for machine reading comprehension\nrequiring commonsense reasoning. Experiments on this dataset demonstrate that\nthe performance of state-of-the-art MRC systems fall far behind human\nperformance. ReCoRD represents a challenge for future research to bridge the\ngap between human and machine commonsense reading comprehension. ReCoRD is\navailable at http://nlp.jhu.edu/record.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12956v2",
            "date": "2018-10-30 18:31:16+00:00",
            "title": "Combining Distant and Direct Supervision for Neural Relation Extraction",
            "authors": [
                "Iz Beltagy",
                "Kyle Lo",
                "Waleed Ammar"
            ],
            "abstract": "In relation extraction with distant supervision, noisy labels make it\ndifficult to train quality models. Previous neural models addressed this\nproblem using an attention mechanism that attends to sentences that are likely\nto express the relations. We improve such models by combining the distant\nsupervision data with an additional directly-supervised data, which we use as\nsupervision for the attention weights. We find that joint training on both\ntypes of supervision leads to a better model because it improves the model's\nability to identify noisy sentences. In addition, we find that sigmoidal\nattention weights with max pooling achieves better performance over the\ncommonly used weighted average attention in this setup. Our proposed method\nachieves a new state-of-the-art result on the widely used FB-NYT dataset.",
            "journal ref": "2019 Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics (NAACL 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13033v1",
            "date": "2018-10-30 23:31:50+00:00",
            "title": "Stress-Testing Neural Models of Natural Language Inference with Multiply-Quantified Sentences",
            "authors": [
                "Atticus Geiger",
                "Ignacio Cases",
                "Lauri Karttunen",
                "Christopher Potts"
            ],
            "abstract": "Standard evaluations of deep learning models for semantics using naturalistic\ncorpora are limited in what they can tell us about the fidelity of the learned\nrepresentations, because the corpora rarely come with good measures of semantic\ncomplexity. To overcome this limitation, we present a method for generating\ndata sets of multiply-quantified natural language inference (NLI) examples in\nwhich semantic complexity can be precisely characterized, and we use this\nmethod to show that a variety of common architectures for NLI inevitably fail\nto encode crucial information; only a model with forced lexical alignments\navoids this damaging information loss.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13083v3",
            "date": "2018-10-31 02:52:21+00:00",
            "title": "GraphIE: A Graph-Based Framework for Information Extraction",
            "authors": [
                "Yujie Qian",
                "Enrico Santus",
                "Zhijing Jin",
                "Jiang Guo",
                "Regina Barzilay"
            ],
            "abstract": "Most modern Information Extraction (IE) systems are implemented as sequential\ntaggers and only model local dependencies. Non-local and non-sequential context\nis, however, a valuable source of information to improve predictions. In this\npaper, we introduce GraphIE, a framework that operates over a graph\nrepresenting a broad set of dependencies between textual units (i.e. words or\nsentences). The algorithm propagates information between connected nodes\nthrough graph convolutions, generating a richer representation that can be\nexploited to improve word-level predictions. Evaluation on three different\ntasks --- namely textual, social media and visual information extraction ---\nshows that GraphIE consistently outperforms the state-of-the-art sequence\ntagging model by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13097v2",
            "date": "2018-10-31 04:05:05+00:00",
            "title": "Attentive Neural Network for Named Entity Recognition in Vietnamese",
            "authors": [
                "Kim Anh Nguyen",
                "Ngan Dong",
                "Cam-Tu Nguyen"
            ],
            "abstract": "We propose an attentive neural network for the task of named entity\nrecognition in Vietnamese. The proposed attentive neural model makes use of\ncharacter-based language models and word embeddings to encode words as vector\nrepresentations. A neural network architecture of encoder, attention, and\ndecoder layers is then utilized to encode knowledge of input sentences and to\nlabel entity tags. The experimental results show that the proposed attentive\nneural network achieves the state-of-the-art results on the benchmark named\nentity recognition datasets in Vietnamese in comparison to both hand-crafted\nfeatures based models and neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13113v3",
            "date": "2018-10-31 05:21:17+00:00",
            "title": "Giving Space to Your Message: Assistive Word Segmentation for the Electronic Typing of Digital Minorities",
            "authors": [
                "Won Ik Cho",
                "Sung Jun Cheon",
                "Woo Hyun Kang",
                "Ji Won Kim",
                "Nam Soo Kim"
            ],
            "abstract": "For readability and disambiguation of the written text, appropriate word\nsegmentation is recommended for documentation, and it also holds for the\ndigitized texts. If the language is agglutinative while far from scriptio\ncontinua, for instance in the Korean language, the problem becomes more\nsignificant. However, some device users these days find it challenging to\ncommunicate via key stroking, not only for handicap but also for being\nunskilled. In this study, we propose a real-time assistive technology that\nutilizes an automatic word segmentation, designed for digital minorities who\nare not familiar with electronic typing. We propose a data-driven system\ntrained upon a spoken Korean language corpus with various non-canonical\nexpressions and dialects, guaranteeing the comprehension of contextual\ninformation. Through quantitative and qualitative comparison with other text\nprocessing toolkits, we show the reliability of the proposed system and its fit\nwith colloquial and non-normalized texts, which fulfills the aim of supportive\ntechnology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13181v1",
            "date": "2018-10-31 09:46:11+00:00",
            "title": "WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community",
            "authors": [
                "Yiqing Hua",
                "Cristian Danescu-Niculescu-Mizil",
                "Dario Taraborelli",
                "Nithum Thain",
                "Jeffery Sorensen",
                "Lucas Dixon"
            ],
            "abstract": "We present a corpus that encompasses the complete history of conversations\nbetween contributors to Wikipedia, one of the largest online collaborative\ncommunities. By recording the intermediate states of conversations---including\nnot only comments and replies, but also their modifications, deletions and\nrestorations---this data offers an unprecedented view of online conversation.\nThis level of detail supports new research questions pertaining to the process\n(and challenges) of large-scale online collaboration. We illustrate the corpus'\npotential with two case studies that highlight new perspectives on earlier\nwork. First, we explore how a person's conversational behavior depends on how\nthey relate to the discussion's venue. Second, we show that community\nmoderation of toxic behavior happens at a higher rate than previously\nestimated. Finally the reconstruction framework is designed to be language\nagnostic, and we show that it can extract high quality conversational data in\nboth Chinese and English.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13223v1",
            "date": "2018-10-31 11:38:40+00:00",
            "title": "SURFACE: Semantically Rich Fact Validation with Explanations",
            "authors": [
                "Ankur Padia",
                "Francis Ferraro",
                "Tim Finin"
            ],
            "abstract": "Judging the veracity of a sentence making one or more claims is an important\nand challenging problem with many dimensions. The recent FEVER task asked\nparticipants to classify input sentences as either SUPPORTED, REFUTED or\nNotEnoughInfo using Wikipedia as a source of true facts. SURFACE does this task\nand explains its decision through a selection of sentences from the trusted\nsource. Our multi-task neural approach uses semantic lexical frames from\nFrameNet to jointly (i) find relevant evidential sentences in the trusted\nsource and (ii) use them to classify the input sentence's veracity. An\nevaluation of our efficient three-parameter model on the FEVER dataset showed\nan improvement of 90% over the state-of-the-art baseline on retrieving relevant\nsentences and a 70% relative improvement in classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13320v2",
            "date": "2018-10-31 14:58:30+00:00",
            "title": "Convolutional Self-Attention Network",
            "authors": [
                "Baosong Yang",
                "Longyue Wang",
                "Derek F. Wong",
                "Lidia S. Chao",
                "Zhaopeng Tu"
            ],
            "abstract": "Self-attention network (SAN) has recently attracted increasing interest due\nto its fully parallelized computation and flexibility in modeling dependencies.\nIt can be further enhanced with multi-headed attention mechanism by allowing\nthe model to jointly attend to information from different representation\nsubspaces at different positions (Vaswani et al., 2017). In this work, we\npropose a novel convolutional self-attention network (CSAN), which offers SAN\nthe abilities to 1) capture neighboring dependencies, and 2) model the\ninteraction between multiple attention heads. Experimental results on WMT14\nEnglish-to-German translation task demonstrate that the proposed approach\noutperforms both the strong Transformer baseline and other existing works on\nenhancing the locality of SAN. Comparing with previous work, our model does not\nintroduce any new parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13327v2",
            "date": "2018-10-31 15:15:00+00:00",
            "title": "Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog",
            "authors": [
                "Sebastian Schuster",
                "Sonal Gupta",
                "Rushin Shah",
                "Mike Lewis"
            ],
            "abstract": "One of the first steps in the utterance interpretation pipeline of many\ntask-oriented conversational AI systems is to identify user intents and the\ncorresponding slots. Since data collection for machine learning models for this\ntask is time-consuming, it is desirable to make use of existing data in a\nhigh-resource language to train models in low-resource languages. However,\ndevelopment of such models has largely been hindered by the lack of\nmultilingual training data. In this paper, we present a new data set of 57k\nannotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the\ndomains weather, alarm, and reminder. We use this data set to evaluate three\ndifferent cross-lingual transfer methods: (1) translating the training data,\n(2) using cross-lingual pre-trained embeddings, and (3) a novel method of using\na multilingual machine translation encoder as contextual word representations.\nWe find that given several hundred training examples in the the target\nlanguage, the latter two methods outperform translating the training data.\nFurther, in very low-resource settings, multilingual contextual word\nrepresentations give better results than using cross-lingual static embeddings.\nWe also compare the cross-lingual methods to using monolingual resources in the\nform of contextual ELMo representations and find that given just small amounts\nof target language data, this method outperforms all cross-lingual methods,\nwhich highlights the need for more sophisticated cross-lingual methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13391v1",
            "date": "2018-10-31 16:38:55+00:00",
            "title": "Picking Apart Story Salads",
            "authors": [
                "Su Wang",
                "Eric Holgate",
                "Greg Durrett",
                "Katrin Erk"
            ],
            "abstract": "During natural disasters and conflicts, information about what happened is\noften confusing, messy, and distributed across many sources. We would like to\nbe able to automatically identify relevant information and assemble it into\ncoherent narratives of what happened. To make this task accessible to neural\nmodels, we introduce Story Salads, mixtures of multiple documents that can be\ngenerated at scale. By exploiting the Wikipedia hierarchy, we can generate\nsalads that exhibit challenging inference problems. Story salads give rise to a\nnovel, challenging clustering task, where the objective is to group sentences\nfrom the same narratives. We demonstrate that simple bag-of-words similarity\nclustering falls short on this task and that it is necessary to take into\naccount global context and coherence.",
            "journal ref": "Empirical Methods in Natural Language Processing 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13409v1",
            "date": "2018-10-31 17:09:37+00:00",
            "title": "You May Not Need Attention",
            "authors": [
                "Ofir Press",
                "Noah A. Smith"
            ],
            "abstract": "In NMT, how far can we get without attention and without separate encoding\nand decoding? To answer that question, we introduce a recurrent neural\ntranslation model that does not use attention and does not have a separate\nencoder and decoder. Our eager translation model is low-latency, writing target\ntokens as soon as it reads the first source token, and uses constant memory\nduring decoding. It performs on par with the standard attention-based model of\nBahdanau et al. (2014), and better on long sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13414v1",
            "date": "2018-10-31 17:23:05+00:00",
            "title": "Extracting Linguistic Resources from the Web for Concept-to-Text Generation",
            "authors": [
                "Gerasimos Lampouras",
                "Ion Androutsopoulos"
            ],
            "abstract": "Many concept-to-text generation systems require domain-specific linguistic\nresources to produce high quality texts, but manually constructing these\nresources can be tedious and costly. Focusing on NaturalOWL, a publicly\navailable state of the art natural language generator for OWL ontologies, we\npropose methods to extract from the Web sentence plans and natural language\nnames, two of the most important types of domain-specific linguistic resources\nused by the generator. Experiments show that texts generated using linguistic\nresources extracted by our methods in a semi-automatic manner, with minimal\nhuman involvement, are perceived as being almost as good as texts generated\nusing manually authored linguistic resources, and much better than texts\nproduced by using linguistic resources extracted from the relation and entity\nidentifiers of the ontology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13441v2",
            "date": "2018-10-31 17:54:37+00:00",
            "title": "Improving Machine Reading Comprehension with General Reading Strategies",
            "authors": [
                "Kai Sun",
                "Dian Yu",
                "Dong Yu",
                "Claire Cardie"
            ],
            "abstract": "Reading strategies have been shown to improve comprehension levels,\nespecially for readers lacking adequate prior knowledge. Just as the process of\nknowledge accumulation is time-consuming for human readers, it is\nresource-demanding to impart rich general domain knowledge into a deep language\nmodel via pre-training. Inspired by reading strategies identified in cognitive\nscience, and given limited computational resources -- just a pre-trained model\nand a fixed number of training instances -- we propose three general strategies\naimed to improve non-extractive machine reading comprehension (MRC): (i) BACK\nAND FORTH READING that considers both the original and reverse order of an\ninput sequence, (ii) HIGHLIGHTING, which adds a trainable embedding to the text\nembedding of tokens that are relevant to the question and candidate answers,\nand (iii) SELF-ASSESSMENT that generates practice questions and candidate\nanswers directly from the text in an unsupervised manner.\n  By fine-tuning a pre-trained language model (Radford et al., 2018) with our\nproposed strategies on the largest general domain multiple-choice MRC dataset\nRACE, we obtain a 5.8% absolute increase in accuracy over the previous best\nresult achieved by the same pre-trained model fine-tuned on RACE without the\nuse of strategies. We further fine-tune the resulting model on a target MRC\ntask, leading to an absolute improvement of 6.2% in average accuracy over\nprevious state-of-the-art approaches on six representative non-extractive MRC\ndatasets from different domains (i.e., ARC, OpenBookQA, MCTest, SemEval-2018\nTask 11, ROCStories, and MultiRC). These results demonstrate the effectiveness\nof our proposed strategies and the versatility and general applicability of our\nfine-tuned models that incorporate these strategies. Core code is available at\nhttps://github.com/nlpdata/strategy/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00051v1",
            "date": "2018-10-31 18:24:32+00:00",
            "title": "Generating Texts with Integer Linear Programming",
            "authors": [
                "Gerasimos Lampouras",
                "Ion Androutsopoulos"
            ],
            "abstract": "Concept-to-text generation typically employs a pipeline architecture, which\noften leads to suboptimal texts. Content selection, for example, may greedily\nselect the most important facts, which may require, however, too many words to\nexpress, and this may be undesirable when space is limited or expensive.\nSelecting other facts, possibly only slightly less important, may allow the\nlexicalization stage to use much fewer words, or to report more facts in the\nsame space. Decisions made during content selection and lexicalization may also\nlead to more or fewer sentence aggregation opportunities, affecting the length\nand readability of the resulting texts. Building upon on a publicly available\nstate of the art natural language generator for Semantic Web ontologies, this\narticle presents an Integer Linear Programming model that, unlike pipeline\narchitectures, jointly considers choices available in content selection,\nlexicalization, and sentence aggregation to avoid greedy local decisions and\nproduce more compact texts, i.e., texts that report more facts per word.\nCompact texts are desirable, for example, when generating advertisements to be\nincluded in Web search results, or when summarizing structured information in\nlimited space. An extended version of the proposed model also considers a\nlimited form of referring expression generation and avoids redundant sentences.\nAn approximation of the two models can be used when longer texts need to be\ngenerated. Experiments with three ontologies confirm that the proposed models\nlead to more compact texts, compared to pipeline systems, with no deterioration\nor with improvements in the perceived quality of the generated texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00066v1",
            "date": "2018-10-31 18:58:22+00:00",
            "title": "Aligning Very Small Parallel Corpora Using Cross-Lingual Word Embeddings and a Monogamy Objective",
            "authors": [
                "Nina Poerner",
                "Masoud Jalili Sabet",
                "Benjamin Roth",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Count-based word alignment methods, such as the IBM models or fast-align,\nstruggle on very small parallel corpora. We therefore present an alternative\napproach based on cross-lingual word embeddings (CLWEs), which are trained on\npurely monolingual data. Our main contribution is an unsupervised objective to\nadapt CLWEs to parallel corpora. In experiments on between 25 and 500\nsentences, our method outperforms fast-align. We also show that our fine-tuning\nobjective consistently improves a CLWE-only baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00070v2",
            "date": "2018-10-31 19:06:50+00:00",
            "title": "Effective Feature Representation for Clinical Text Concept Extraction",
            "authors": [
                "Yifeng Tao",
                "Bruno Godefroy",
                "Guillaume Genthial",
                "Christopher Potts"
            ],
            "abstract": "Crucial information about the practice of healthcare is recorded only in\nfree-form text, which creates an enormous opportunity for high-impact NLP.\nHowever, annotated healthcare datasets tend to be small and expensive to\nobtain, which raises the question of how to make maximally efficient uses of\nthe available data. To this end, we develop an LSTM-CRF model for combining\nunsupervised word representations and hand-built feature representations\nderived from publicly available healthcare ontologies. We show that this\ncombined model yields superior performance on five datasets of diverse kinds of\nhealthcare text (clinical, social, scientific, commercial). Each involves the\nlabeling of complex, multi-word spans that pick out different healthcare\nconcepts. We also introduce a new labeled dataset for identifying the treatment\nrelations between drugs and diseases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00119v2",
            "date": "2018-10-31 21:04:04+00:00",
            "title": "A task in a suit and a tie: paraphrase generation with semantic augmentation",
            "authors": [
                "Su Wang",
                "Rahul Gupta",
                "Nancy Chang",
                "Jason Baldridge"
            ],
            "abstract": "Paraphrasing is rooted in semantics. We show the effectiveness of\ntransformers (Vaswani et al. 2017) for paraphrase generation and further\nimprovements by incorporating PropBank labels via a multi-encoder. Evaluating\non MSCOCO and WikiAnswers, we find that transformers are fast and effective,\nand that semantic augmentation for both transformers and LSTMs leads to sizable\n2-3 point gains in BLEU, METEOR and TER. More importantly, we find surprisingly\nlarge gains on human evaluations compared to previous models. Nevertheless,\nmanual inspection of generated paraphrases reveals ample room for improvement:\neven our best model produces human-acceptable paraphrases for only 28% of\ncaptions from the CHIA dataset (Sharma et al. 2018), and it fails spectacularly\non sentences from Wikipedia. Overall, these results point to the potential for\nincorporating semantics in the task while highlighting the need for stronger\nevaluation.",
            "journal ref": "Association for the Advancement of Artificial Intelligence (AAAI)\n  2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00127v1",
            "date": "2018-10-31 21:31:08+00:00",
            "title": "Measuring Issue Ownership using Word Embeddings",
            "authors": [
                "Amaru Cuba Gyllensten",
                "Magnus Sahlgren"
            ],
            "abstract": "Sentiment and topic analysis are common methods used for social media\nmonitoring. Essentially, these methods answers questions such as, \"what is\nbeing talked about, regarding X\", and \"what do people feel, regarding X\". In\nthis paper, we investigate another venue for social media monitoring, namely\nissue ownership and agenda setting, which are concepts from political science\nthat have been used to explain voter choice and electoral outcomes. We argue\nthat issue alignment and agenda setting can be seen as a kind of semantic\nsource similarity of the kind \"how similar is source A to issue owner P, when\ntalking about issue X\", and as such can be measured using word/document\nembedding techniques. We present work in progress towards measuring that kind\nof conditioned similarity, and introduce a new notion of similarity for\npredictive embeddings. We then test this method by measuring the similarity\nbetween politically aligned media and political parties, conditioned on\nbloc-specific issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00146v3",
            "date": "2018-10-31 22:57:51+00:00",
            "title": "ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning",
            "authors": [
                "Maarten Sap",
                "Ronan LeBras",
                "Emily Allaway",
                "Chandra Bhagavatula",
                "Nicholas Lourie",
                "Hannah Rashkin",
                "Brendan Roof",
                "Noah A. Smith",
                "Yejin Choi"
            ],
            "abstract": "We present ATOMIC, an atlas of everyday commonsense reasoning, organized\nthrough 877k textual descriptions of inferential knowledge. Compared to\nexisting resources that center around taxonomic knowledge, ATOMIC focuses on\ninferential knowledge organized as typed if-then relations with variables\n(e.g., \"if X pays Y a compliment, then Y will likely return the compliment\").\nWe propose nine if-then relation types to distinguish causes vs. effects,\nagents vs. themes, voluntary vs. involuntary events, and actions vs. mental\nstates. By generatively training on the rich inferential knowledge described in\nATOMIC, we show that neural models can acquire simple commonsense capabilities\nand reason about previously unseen events. Experimental results demonstrate\nthat multitask models that incorporate the hierarchical structure of if-then\nrelation types lead to more accurate inference compared to models trained in\nisolation, as measured by both automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00185v1",
            "date": "2018-11-01 02:10:50+00:00",
            "title": "Dial2Desc: End-to-end Dialogue Description Generation",
            "authors": [
                "Haojie Pan",
                "Junpei Zhou",
                "Zhou Zhao",
                "Yan Liu",
                "Deng Cai",
                "Min Yang"
            ],
            "abstract": "We first propose a new task named Dialogue Description (Dial2Desc). Unlike\nother existing dialogue summarization tasks such as meeting summarization, we\ndo not maintain the natural flow of a conversation but describe an object or an\naction of what people are talking about. The Dial2Desc system takes a dialogue\ntext as input, then outputs a concise description of the object or the action\ninvolved in this conversation. After reading this short description, one can\nquickly extract the main topic of a conversation and build a clear picture in\nhis mind, without reading or listening to the whole conversation. Based on the\nexisting dialogue dataset, we build a new dataset, which has more than one\nhundred thousand dialogue-description pairs. As a step forward, we demonstrate\nthat one can get more accurate and descriptive results using a new neural\nattentive model that exploits the interaction between utterances from different\nspeakers, compared with other baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00207v5",
            "date": "2018-11-01 03:43:10+00:00",
            "title": "Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset",
            "authors": [
                "Hannah Rashkin",
                "Eric Michael Smith",
                "Margaret Li",
                "Y-Lan Boureau"
            ],
            "abstract": "One challenge for dialogue agents is recognizing feelings in the conversation\npartner and replying accordingly, a key communicative skill. While it is\nstraightforward for humans to recognize and acknowledge others' feelings in a\nconversation, this is a significant challenge for AI systems due to the paucity\nof suitable publicly-available datasets for training and evaluation. This work\nproposes a new benchmark for empathetic dialogue generation and\nEmpatheticDialogues, a novel dataset of 25k conversations grounded in emotional\nsituations. Our experiments indicate that dialogue models that use our dataset\nare perceived to be more empathetic by human evaluators, compared to models\nmerely trained on large-scale Internet conversation data. We also present\nempirical comparisons of dialogue model adaptations for empathetic responding,\nleveraging existing models or datasets without requiring lengthy re-training of\nthe full model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00232v2",
            "date": "2018-11-01 05:10:44+00:00",
            "title": "Textbook Question Answering with Multi-modal Context Graph Understanding and Self-supervised Open-set Comprehension",
            "authors": [
                "Daesik Kim",
                "Seonhoon Kim",
                "Nojun Kwak"
            ],
            "abstract": "In this work, we introduce a novel algorithm for solving the textbook\nquestion answering (TQA) task which describes more realistic QA problems\ncompared to other recent tasks. We mainly focus on two related issues with\nanalysis of the TQA dataset. First, solving the TQA problems requires to\ncomprehend multi-modal contexts in complicated input data. To tackle this issue\nof extracting knowledge features from long text lessons and merging them with\nvisual features, we establish a context graph from texts and images, and\npropose a new module f-GCN based on graph convolutional networks (GCN). Second,\nscientific terms are not spread over the chapters and subjects are split in the\nTQA dataset. To overcome this so called \"out-of-domain\" issue, before learning\nQA problems, we introduce a novel self-supervised open-set learning process\nwithout any annotations. The experimental results show that our model\nsignificantly outperforms prior state-of-the-art methods. Moreover, ablation\nstudies validate that both methods of incorporating f-GCN for extracting\nknowledge from multi-modal contexts and our newly proposed self-supervised\nlearning process are effective for TQA problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00238v1",
            "date": "2018-11-01 05:21:47+00:00",
            "title": "Spelling Error Correction Using a Nested RNN Model and Pseudo Training Data",
            "authors": [
                "Hao Li",
                "Yang Wang",
                "Xinyu Liu",
                "Zhichao Sheng",
                "Si Wei"
            ],
            "abstract": "We propose a nested recurrent neural network (nested RNN) model for English\nspelling error correction and generate pseudo data based on phonetic similarity\nto train it. The model fuses orthographic information and context as a whole\nand is trained in an end-to-end fashion. This avoids feature engineering and\ndoes not rely on a noisy channel model as in traditional methods. Experiments\nshow that the proposed method is superior to existing systems in correcting\nspelling errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00240v2",
            "date": "2018-11-01 05:26:27+00:00",
            "title": "GlobalTrait: Personality Alignment of Multilingual Word Embeddings",
            "authors": [
                "Farhad Bin Siddique",
                "Dario Bertero",
                "Pascale Fung"
            ],
            "abstract": "We propose a multilingual model to recognize Big Five Personality traits from\ntext data in four different languages: English, Spanish, Dutch and Italian. Our\nanalysis shows that words having a similar semantic meaning in different\nlanguages do not necessarily correspond to the same personality traits.\nTherefore, we propose a personality alignment method, GlobalTrait, which has a\nmapping for each trait from the source language to the target language\n(English), such that words that correlate positively to each trait are close\ntogether in the multilingual vector space. Using these aligned embeddings for\ntraining, we can transfer personality related training features from\nhigh-resource languages such as English to other low-resource languages, and\nget better multilingual results, when compared to using simple monolingual and\nunaligned multilingual embeddings. We achieve an average F-score increase\n(across all three languages except English) from 65 to 73.4 (+8.4), when\ncomparing our monolingual model to multilingual using CNN with personality\naligned embeddings. We also show relatively good performance in the regression\ntasks, and better classification results when evaluating our model on a\nseparate Chinese dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00241v2",
            "date": "2018-11-01 05:29:02+00:00",
            "title": "On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition",
            "authors": [
                "Zhiping Zeng",
                "Yerbolat Khassanov",
                "Van Tung Pham",
                "Haihua Xu",
                "Eng Siong Chng",
                "Haizhou Li"
            ],
            "abstract": "Code-switching (CS) refers to a linguistic phenomenon where a speaker uses\ndifferent languages in an utterance or between alternating utterances. In this\nwork, we study end-to-end (E2E) approaches to the Mandarin-English\ncode-switching speech recognition (CSSR) task. We first examine the\neffectiveness of using data augmentation and byte-pair encoding (BPE) subword\nunits. More importantly, we propose a multitask learning recipe, where a\nlanguage identification task is explicitly learned in addition to the E2E\nspeech recognition task. Furthermore, we introduce an efficient word vocabulary\nexpansion method for language modeling to alleviate data sparsity issues under\nthe code-switching scenario. Experimental results on the SEAME data, a\nMandarin-English CS corpus, demonstrate the effectiveness of the proposed\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00253v3",
            "date": "2018-11-01 06:35:21+00:00",
            "title": "Hybrid Self-Attention Network for Machine Translation",
            "authors": [
                "Kaitao Song",
                "Xu Tan",
                "Furong Peng",
                "Jianfeng Lu"
            ],
            "abstract": "The encoder-decoder is the typical framework for Neural Machine Translation\n(NMT), and different structures have been developed for improving the\ntranslation performance. Transformer is one of the most promising structures,\nwhich can leverage the self-attention mechanism to capture the semantic\ndependency from global view. However, it cannot distinguish the relative\nposition of different tokens very well, such as the tokens located at the left\nor right of the current token, and cannot focus on the local information around\nthe current token either. To alleviate these problems, we propose a novel\nattention mechanism named Hybrid Self-Attention Network (HySAN) which\naccommodates some specific-designed masks for self-attention network to extract\nvarious semantic, such as the global/local information, the left/right part\ncontext. Finally, a squeeze gate is introduced to combine different kinds of\nSANs for fusion. Experimental results on three machine translation tasks show\nthat our proposed framework outperforms the Transformer baseline significantly\nand achieves superior results over state-of-the-art NMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00258v1",
            "date": "2018-11-01 06:47:23+00:00",
            "title": "Language-Independent Representor for Neural Machine Translation",
            "authors": [
                "Long Zhou",
                "Yuchen Liu",
                "Jiajun Zhang",
                "Chengqing Zong",
                "Guoping Huang"
            ],
            "abstract": "Current Neural Machine Translation (NMT) employs a language-specific encoder\nto represent the source sentence and adopts a language-specific decoder to\ngenerate target translation. This language-dependent design leads to\nlarge-scale network parameters and makes the duality of the parallel data\nunderutilized. To address the problem, we propose in this paper a\nlanguage-independent representor to replace the encoder and decoder by using\nweight sharing. This shared representor can not only reduce large portion of\nnetwork parameters, but also facilitate us to fully explore the language\nduality by jointly training source-to-target, target-to-source, left-to-right\nand right-to-left translations within a multi-task learning framework.\nExperiments show that our proposed framework can obtain significant\nimprovements over conventional NMT models on resource-rich and low-resource\ntranslation tasks with only a quarter of parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00266v2",
            "date": "2018-11-01 07:18:33+00:00",
            "title": "Learning to Describe Phrases with Local and Global Contexts",
            "authors": [
                "Shonosuke Ishiwatari",
                "Hiroaki Hayashi",
                "Naoki Yoshinaga",
                "Graham Neubig",
                "Shoetsu Sato",
                "Masashi Toyoda",
                "Masaru Kitsuregawa"
            ],
            "abstract": "When reading a text, it is common to become stuck on unfamiliar words and\nphrases, such as polysemous words with novel senses, rarely used idioms,\ninternet slang, or emerging entities. If we humans cannot figure out the\nmeaning of those expressions from the immediate local context, we consult\ndictionaries for definitions or search documents or the web to find other\nglobal context to help in interpretation. Can machines help us do this work?\nWhich type of context is more important for machines to solve the problem? To\nanswer these questions, we undertake a task of describing a given phrase in\nnatural language based on its local and global contexts. To solve this task, we\npropose a neural description model that consists of two context encoders and a\ndescription decoder. In contrast to the existing methods for non-standard\nEnglish explanation [Ni+ 2017] and definition generation [Noraset+ 2017;\nGadetsky+ 2018], our model appropriately takes important clues from both local\nand global contexts. Experimental results on three existing datasets (including\nWordNet, Oxford and Urban Dictionaries) and a dataset newly created from\nWikipedia demonstrate the effectiveness of our method over previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00275v1",
            "date": "2018-11-01 07:54:31+00:00",
            "title": "Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy",
            "authors": [
                "Pengcheng Yang",
                "Fuli Luo",
                "Shuangzhi Wu",
                "Jingjing Xu",
                "Dongdong Zhang",
                "Xu Sun"
            ],
            "abstract": "Cross-lingual word embeddings aim to capture common linguistic regularities\nof different languages, which benefit various downstream tasks ranging from\nmachine translation to transfer learning. Recently, it has been shown that\nthese embeddings can be effectively learned by aligning two disjoint\nmonolingual vector spaces through a linear transformation (word mapping). In\nthis work, we focus on learning such a word mapping without any supervision\nsignal. Most previous work of this task adopts parametric metrics to measure\ndistribution differences, which typically requires a sophisticated alternate\noptimization process, either in the form of \\emph{minmax game} or intermediate\n\\emph{density estimation}. This alternate optimization process is relatively\nhard and unstable. In order to avoid such sophisticated alternate optimization,\nwe propose to learn unsupervised word mapping by directly maximizing the mean\ndiscrepancy between the distribution of transferred embedding and target\nembedding. Extensive experimental results show that our proposed model\noutperforms competitive baselines by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00287v4",
            "date": "2018-11-01 09:37:48+00:00",
            "title": "Towards Linear Time Neural Machine Translation with Capsule Networks",
            "authors": [
                "Mingxuan Wang",
                "Jun Xie",
                "Zhixing Tan",
                "Jinsong Su",
                "Deyi Xiong",
                "Lei Li"
            ],
            "abstract": "In this study, we first investigate a novel capsule network with dynamic\nrouting for linear time Neural Machine Translation (NMT), referred as\n\\textsc{CapsNMT}. \\textsc{CapsNMT} uses an aggregation mechanism to map the\nsource sentence into a matrix with pre-determined size, and then applys a deep\nLSTM network to decode the target sequence from the source representation.\nUnlike the previous work \\cite{sutskever2014sequence} to store the source\nsentence with a passive and bottom-up way, the dynamic routing policy encodes\nthe source sentence with an iterative process to decide the credit attribution\nbetween nodes from lower and higher layers. \\textsc{CapsNMT} has two core\nproperties: it runs in time that is linear in the length of the sequences and\nprovides a more flexible way to select, represent and aggregates the part-whole\ninformation of the source sentence. On WMT14 English-German task and a larger\nWMT14 English-French task, \\textsc{CapsNMT} achieves comparable results with\nthe state-of-the-art NMT systems. To the best of our knowledge, this is the\nfirst work that capsule networks have been empirically investigated for\nsequence to sequence problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00347v2",
            "date": "2018-11-01 12:47:11+00:00",
            "title": "How2: A Large-scale Dataset for Multimodal Language Understanding",
            "authors": [
                "Ramon Sanabria",
                "Ozan Caglayan",
                "Shruti Palaskar",
                "Desmond Elliott",
                "Lo\u00efc Barrault",
                "Lucia Specia",
                "Florian Metze"
            ],
            "abstract": "In this paper, we introduce How2, a multimodal collection of instructional\nvideos with English subtitles and crowdsourced Portuguese translations. We also\npresent integrated sequence-to-sequence baselines for machine translation,\nautomatic speech recognition, spoken language translation, and multimodal\nsummarization. By making available data and code for several multimodal natural\nlanguage tasks, we hope to stimulate more research on these and similar\nchallenges, to obtain a deeper understanding of multimodality in language\nprocessing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00357v2",
            "date": "2018-11-01 13:19:27+00:00",
            "title": "Latent Variable Model for Multi-modal Translation",
            "authors": [
                "Iacer Calixto",
                "Miguel Rios",
                "Wilker Aziz"
            ],
            "abstract": "In this work, we propose to model the interaction between visual and textual\nfeatures for multi-modal neural machine translation (MMT) through a latent\nvariable model. This latent variable can be seen as a multi-modal stochastic\nembedding of an image and its description in a foreign language. It is used in\na target-language decoder and also to predict image features. Importantly, our\nmodel formulation utilises visual and textual inputs during training but does\nnot require that images be available at test time. We show that our latent\nvariable MMT formulation improves considerably over strong baselines, including\na multi-task learning approach (Elliott and K\\'ad\\'ar, 2017) and a conditional\nvariational auto-encoder approach (Toyama et al., 2016). Finally, we show\nimprovements due to (i) predicting image features in addition to only\nconditioning on them, (ii) imposing a constraint on the minimum amount of\ninformation encoded in the latent variable, and (iii) by training on additional\ntarget-language image descriptions (i.e. synthetic data).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00379v1",
            "date": "2018-11-01 13:49:58+00:00",
            "title": "Helping each Other: A Framework for Customer-to-Customer Suggestion Mining using a Semi-supervised Deep Neural Network",
            "authors": [
                "Hitesh Golchha",
                "Deepak Gupta",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Suggestion mining is increasingly becoming an important task along with\nsentiment analysis. In today's cyberspace world, people not only express their\nsentiments and dispositions towards some entities or services, but they also\nspend considerable time sharing their experiences and advice to fellow\ncustomers and the product/service providers with two-fold agenda: helping\nfellow customers who are likely to share a similar experience, and motivating\nthe producer to bring specific changes in their offerings which would be more\nappreciated by the customers. In our current work, we propose a hybrid deep\nlearning model to identify whether a review text contains any suggestion. The\nmodel employs semi-supervised learning to leverage the useful information from\nthe large amount of unlabeled data. We evaluate the performance of our proposed\nmodel on a benchmark customer review dataset, comprising of the reviews of\nHotel and Electronics domains. Our proposed approach shows the F-scores of\n65.6% and 65.5% for the Hotel and Electronics review datasets, respectively.\nThese performances are significantly better compared to the existing\nstate-of-the-art system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00383v2",
            "date": "2018-11-01 13:53:27+00:00",
            "title": "Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages",
            "authors": [
                "Rudra Murthy V",
                "Anoop Kunchukuttan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Transfer learning approaches for Neural Machine Translation (NMT) train a NMT\nmodel on the assisting-target language pair (parent model) which is later\nfine-tuned for the source-target language pair of interest (child model), with\nthe target language being the same. In many cases, the assisting language has a\ndifferent word order from the source language. We show that divergent word\norder adversely limits the benefits from transfer learning when little to no\nparallel corpus between the source and target language is available. To bridge\nthis divergence, We propose to pre-order the assisting language sentence to\nmatch the word order of the source language and train the parent model. Our\nexperiments on many language pairs show that bridging the word order gap leads\nto significant improvement in the translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00405v4",
            "date": "2018-11-01 14:27:19+00:00",
            "title": "DialogueRNN: An Attentive RNN for Emotion Detection in Conversations",
            "authors": [
                "Navonil Majumder",
                "Soujanya Poria",
                "Devamanyu Hazarika",
                "Rada Mihalcea",
                "Alexander Gelbukh",
                "Erik Cambria"
            ],
            "abstract": "Emotion detection in conversations is a necessary step for a number of\napplications, including opinion mining over chat history, social media threads,\ndebates, argumentation mining, understanding consumer feedback in live\nconversations, etc. Currently, systems do not treat the parties in the\nconversation individually by adapting to the speaker of each utterance. In this\npaper, we describe a new method based on recurrent neural networks that keeps\ntrack of the individual party states throughout the conversation and uses this\ninformation for emotion classification. Our model outperforms the state of the\nart by a significant margin on two different datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00436v1",
            "date": "2018-11-01 15:32:38+00:00",
            "title": "Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-based Extractive Summarization",
            "authors": [
                "Haggai Roitman",
                "Guy Feigenblat",
                "David Konopnicki",
                "Doron Cohen",
                "Odellia Boni"
            ],
            "abstract": "We propose Dual-CES -- a novel unsupervised, query-focused, multi-document\nextractive summarizer. Dual-CES is designed to better handle the tradeoff\nbetween saliency and focus in summarization. To this end, Dual-CES employs a\ntwo-step dual-cascade optimization approach with saliency-based pseudo-feedback\ndistillation. Overall, Dual-CES significantly outperforms all other\nstate-of-the-art unsupervised alternatives. Dual-CES is even shown to be able\nto outperform strong supervised summarizers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00498v1",
            "date": "2018-11-01 17:06:09+00:00",
            "title": "Multilingual NMT with a language-independent attention bridge",
            "authors": [
                "Ra\u00fal V\u00e1zquez",
                "Alessandro Raganato",
                "J\u00f6rg Tiedemann",
                "Mathias Creutz"
            ],
            "abstract": "In this paper, we propose a multilingual encoder-decoder architecture capable\nof obtaining multilingual sentence representations by means of incorporating an\nintermediate {\\em attention bridge} that is shared across all languages. That\nis, we train the model with language-specific encoders and decoders that are\nconnected via self-attention with a shared layer that we call attention bridge.\nThis layer exploits the semantics from each language for performing translation\nand develops into a language-independent meaning representation that can\nefficiently be used for transfer learning. We present a new framework for the\nefficient development of multilingual NMT using this model and scheduled\ntraining. We have tested the approach in a systematic way with a multi-parallel\ndata set. We show that the model achieves substantial improvements over strong\nbilingual models and that it also works well for zero-shot translation, which\ndemonstrates its ability of abstraction and transfer learning.",
            "journal ref": "Proceedings of the 4th Workshop on Representation Learning for NLP\n  (RepL4NLP-2019) Pages 33-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00511v2",
            "date": "2018-11-01 17:30:50+00:00",
            "title": "Towards Coherent and Cohesive Long-form Text Generation",
            "authors": [
                "Woon Sang Cho",
                "Pengchuan Zhang",
                "Yizhe Zhang",
                "Xiujun Li",
                "Michel Galley",
                "Chris Brockett",
                "Mengdi Wang",
                "Jianfeng Gao"
            ],
            "abstract": "Generating coherent and cohesive long-form texts is a challenging task.\nPrevious works relied on large amounts of human-generated texts to train neural\nlanguage models. However, few attempted to explicitly improve neural language\nmodels from the perspectives of coherence and cohesion. In this work, we\npropose a new neural language model that is equipped with two neural\ndiscriminators which provide feedback signals at the levels of sentence\n(cohesion) and paragraph (coherence). Our model is trained using a simple yet\nefficient variant of policy gradient, called negative-critical sequence\ntraining, which is proposed to eliminate the need of training a separate critic\nfor estimating baseline. Results demonstrate the effectiveness of our approach,\nshowing improvements over the strong baseline -- recurrent attention-based\nbidirectional MLE-trained neural language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00586v2",
            "date": "2018-11-01 18:48:57+00:00",
            "title": "Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable",
            "authors": [
                "Philipp Dufter",
                "Mengjie Zhao",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word embeddings induced from local context are prevalent in NLP. A simple and\neffective context-based multilingual embedding learner is Levy et al. (2017)'s\nS-ID (sentence ID) method. Another line of work induces high-performing\nmultilingual embeddings from concepts (Dufter et al., 2018). In this paper, we\npropose Co+Co, a simple and scalable method that combines context-based and\nconcept-based learning. From a sentence aligned corpus, concepts are extracted\nvia sampling; words are then associated with their concept ID and sentence ID\nin embedding learning. This is the first work that successfully combines\ncontext-based and concept-based embedding learning. We show that Co+Co performs\nwell for two different application scenarios: the Parallel Bible Corpus (1000+\nlanguages, low-resource) and EuroParl (12 languages, high-resource). Among\nmethods applicable to both corpora, Co+Co performs best in our evaluation setup\nof six tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00613v3",
            "date": "2018-11-01 19:54:58+00:00",
            "title": "Shifting the Baseline: Single Modality Performance on Visual Navigation & QA",
            "authors": [
                "Jesse Thomason",
                "Daniel Gordon",
                "Yonatan Bisk"
            ],
            "abstract": "We demonstrate the surprising strength of unimodal baselines in multimodal\ndomains, and make concrete recommendations for best practices in future\nresearch. Where existing work often compares against random or majority class\nbaselines, we argue that unimodal approaches better capture and reflect dataset\nbiases and therefore provide an important comparison when assessing the\nperformance of multimodal techniques. We present unimodal ablations on three\nrecent datasets in visual navigation and QA, seeing an up to 29% absolute gain\nin performance over published baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00625v1",
            "date": "2018-11-01 20:30:25+00:00",
            "title": "Incorporating Structured Commonsense Knowledge in Story Completion",
            "authors": [
                "Jiaao Chen",
                "Jianshu Chen",
                "Zhou Yu"
            ],
            "abstract": "The ability to select an appropriate story ending is the first step towards\nperfect narrative comprehension. Story ending prediction requires not only the\nexplicit clues within the context, but also the implicit knowledge (such as\ncommonsense) to construct a reasonable and consistent story. However, most\nprevious approaches do not explicitly use background commonsense knowledge. We\npresent a neural story ending selection model that integrates three types of\ninformation: narrative sequence, sentiment evolution and commonsense knowledge.\nExperiments show that our model outperforms state-of-the-art approaches on a\npublic dataset, ROCStory Cloze Task , and the performance gain from adding the\nadditional commonsense knowledge is significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00633v1",
            "date": "2018-11-01 21:03:41+00:00",
            "title": "Embedding Individual Table Columns for Resilient SQL Chatbots",
            "authors": [
                "Bojan Petrovski",
                "Ignacio Aguado",
                "Andreea Hossmann",
                "Michael Baeriswyl",
                "Claudiu Musat"
            ],
            "abstract": "Most of the world's data is stored in relational databases. Accessing these\nrequires specialized knowledge of the Structured Query Language (SQL), putting\nthem out of the reach of many people. A recent research thread in Natural\nLanguage Processing (NLP) aims to alleviate this problem by automatically\ntranslating natural language questions into SQL queries. While the proposed\nsolutions are a great start, they lack robustness and do not easily generalize:\nthe methods require high quality descriptions of the database table columns,\nand the most widely used training dataset, WikiSQL, is heavily biased towards\nusing those descriptions as part of the questions.\n  In this work, we propose solutions to both problems: we entirely eliminate\nthe need for column descriptions, by relying solely on their contents, and we\naugment the WikiSQL dataset by paraphrasing column names to reduce bias. We\nshow that the accuracy of existing methods drops when trained on our augmented,\ncolumn-agnostic dataset, and that our own method reaches state of the art\naccuracy, while relying on column contents only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00644v2",
            "date": "2018-11-01 21:40:26+00:00",
            "title": "Analyzing and learning the language for different types of harassment",
            "authors": [
                "Mohammadreza Rezvan",
                "Saeedeh Shekarpour",
                "Faisal Alshargi",
                "Krishnaprasad Thirunarayan",
                "Valerie L. Shalin",
                "Amit Sheth"
            ],
            "abstract": "Disclaimer: This paper is concerned with violent online harassment. To\ndescribe the subject at an adequate level of realism, examples of our collected\ntweets involve violent, threatening, vulgar and hateful speech language in the\ncontext of racial, sexual, political, appearance and intellectual harassment.\n  The presence of a significant amount of harassment in user-generated content\nand its negative impact calls for robust automatic detection approaches. This\nrequires that we can identify different forms or types of harassment. Earlier\nwork has classified harassing language in terms of hurtfulness, abusiveness,\nsentiment, and profanity. However, to identify and understand harassment more\naccurately, it is essential to determine the context that represents the\ninterrelated conditions in which they occur. In this paper, we introduce the\nnotion of contextual type to harassment involving five categories: (i) sexual,\n(ii) racial, (iii) appearance-related, (iv) intellectual and (v) political. We\nutilize an annotated corpus from Twitter distinguishing these types of\nharassment. To study the context for each type that sheds light on the\nlinguistic meaning, interpretation, and distribution, we conduct two lines of\ninvestigation: an extensive linguistic analysis, and a statistical distribution\nof unigrams. We then build type-ware classifiers to automate the identification\nof type-specific harassment. Our experiments demonstrate that these classifiers\nprovide competitive accuracy for identifying and analyzing harassment on social\nmedia. We present extensive discussion and major observations about the\neffectiveness of type-aware classifiers using a detailed comparison setup\nproviding insight into the role of type-dependent features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00693v1",
            "date": "2018-11-02 01:06:14+00:00",
            "title": "Meta-path Augmented Response Generation",
            "authors": [
                "Yanran Li",
                "Wenjie Li"
            ],
            "abstract": "We propose a chatbot, namely Mocha to make good use of relevant entities when\ngenerating responses. Augmented with meta-path information, Mocha is able to\nmention proper entities following the conversation flow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00696v1",
            "date": "2018-11-02 01:21:17+00:00",
            "title": "Sequence Generation with Guider Network",
            "authors": [
                "Ruiyi Zhang",
                "Changyou Chen",
                "Zhe Gan",
                "Wenlin Wang",
                "Liqun Chen",
                "Dinghan Shen",
                "Guoyin Wang",
                "Lawrence Carin"
            ],
            "abstract": "Sequence generation with reinforcement learning (RL) has received significant\nattention recently. However, a challenge with such methods is the sparse-reward\nproblem in the RL training process, in which a scalar guiding signal is often\nonly available after an entire sequence has been generated. This type of sparse\nreward tends to ignore the global structural information of a sequence, causing\ngeneration of sequences that are semantically inconsistent. In this paper, we\npresent a model-based RL approach to overcome this issue. Specifically, we\npropose a novel guider network to model the sequence-generation environment,\nwhich can assist next-word prediction and provide intermediate rewards for\ngenerator optimization. Extensive experiments show that the proposed method\nleads to improved performance for both unconditional and conditional\nsequence-generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00720v2",
            "date": "2018-11-02 03:04:21+00:00",
            "title": "Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems",
            "authors": [
                "Ting-Rui Chiang",
                "Yun-Nung Chen"
            ],
            "abstract": "Solving math word problems is a challenging task that requires accurate\nnatural language understanding to bridge natural language texts and math\nexpressions. Motivated by the intuition about how human generates the equations\ngiven the problem texts, this paper presents a neural approach to automatically\nsolve math word problems by operating symbols according to their semantic\nmeanings in texts. This paper views the process of generating equation as a\nbridge between the semantic world and the symbolic world, where the proposed\nneural math solver is based on an encoder-decoder framework. In the proposed\nmodel, the encoder is designed to understand the semantics of problems, and the\ndecoder focuses on tracking semantic meanings of the generated symbols and then\ndeciding which symbol to generate next. The preliminary experiments are\nconducted in a dataset Math23K, and our model significantly outperforms both\nthe state-of-the-art single model and the best non-retrieval-based model over\nabout 10% accuracy, demonstrating the effectiveness of bridging the symbolic\nand semantic worlds from math word problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00728v1",
            "date": "2018-11-02 03:56:42+00:00",
            "title": "Improving the Robustness of Speech Translation",
            "authors": [
                "Xiang Li",
                "Haiyang Xue",
                "Wei Chen",
                "Yang Liu",
                "Yang Feng",
                "Qun Liu"
            ],
            "abstract": "Although neural machine translation (NMT) has achieved impressive progress\nrecently, it is usually trained on the clean parallel data set and hence cannot\nwork well when the input sentence is the production of the automatic speech\nrecognition (ASR) system due to the enormous errors in the source. To solve\nthis problem, we propose a simple but effective method to improve the\nrobustness of NMT in the case of speech translation. We simulate the noise\nexisting in the realistic output of the ASR system and inject them into the\nclean parallel data so that NMT can work under similar word distributions\nduring training and testing. Besides, we also incorporate the Chinese Pinyin\nfeature which is easy to get in speech translation to further improve the\ntranslation performance. Experiment results show that our method has a more\nstable performance and outperforms the baseline by an average of 3.12 BLEU on\nmultiple noisy test sets, even while achieves a generalization improvement on\nthe WMT'17 Chinese-English test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00783v2",
            "date": "2018-11-02 08:59:19+00:00",
            "title": "Abstractive Summarization of Reddit Posts with Multi-level Memory Networks",
            "authors": [
                "Byeongchang Kim",
                "Hyunwoo Kim",
                "Gunhee Kim"
            ],
            "abstract": "We address the problem of abstractive summarization in two directions:\nproposing a novel dataset and a new model. First, we collect Reddit TIFU\ndataset, consisting of 120K posts from the online discussion forum Reddit. We\nuse such informal crowd-generated posts as text source, in contrast with\nexisting datasets that mostly use formal documents as source such as news\narticles. Thus, our dataset could less suffer from some biases that key\nsentences usually locate at the beginning of the text and favorable summary\ncandidates are already inside the text in similar forms. Second, we propose a\nnovel abstractive summarization model named multi-level memory networks (MMN),\nequipped with multi-level memory to store the information of text from\ndifferent levels of abstraction. With quantitative evaluation and user studies\nvia Amazon Mechanical Turk, we show the Reddit TIFU dataset is highly\nabstractive and the MMN outperforms the state-of-the-art summarization models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00787v1",
            "date": "2018-11-02 09:10:24+00:00",
            "title": "Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model",
            "authors": [
                "Alexander H. Liu",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "In this paper we proposed a novel Adversarial Training (AT) approach for\nend-to-end speech recognition using a Criticizing Language Model (CLM). In this\nway the CLM and the automatic speech recognition (ASR) model can challenge and\nlearn from each other iteratively to improve the performance. Since the CLM\nonly takes the text as input, huge quantities of unpaired text data can be\nutilized in this approach within end-to-end training. Moreover, AT can be\napplied to any end-to-end ASR model using any deep-learning-based language\nmodeling frameworks, and compatible with any existing end-to-end decoding\nmethod. Initial results with an example experimental setup demonstrated the\nproposed approach is able to gain consistent improvements efficiently from\nauxiliary text data under different scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00942v1",
            "date": "2018-11-02 15:46:52+00:00",
            "title": "Progress and Tradeoffs in Neural Language Models",
            "authors": [
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "In recent years, we have witnessed a dramatic shift towards techniques driven\nby neural networks for a variety of NLP tasks. Undoubtedly, neural language\nmodels (NLMs) have reduced perplexity by impressive amounts. This progress,\nhowever, comes at a substantial cost in performance, in terms of inference\nlatency and energy consumption, which is particularly of concern in deployments\non mobile devices. This paper, which examines the quality-performance tradeoff\nof various language modeling techniques, represents to our knowledge the first\nto make this observation. We compare state-of-the-art NLMs with \"classic\"\nKneser-Ney (KN) LMs in terms of energy usage, latency, perplexity, and\nprediction accuracy using two standard benchmarks. On a Raspberry Pi, we find\nthat orders of increase in latency and energy usage correspond to less change\nin perplexity, while the difference is much less pronounced on a desktop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00945v2",
            "date": "2018-11-02 15:54:15+00:00",
            "title": "Image Chat: Engaging Grounded Conversations",
            "authors": [
                "Kurt Shuster",
                "Samuel Humeau",
                "Antoine Bordes",
                "Jason Weston"
            ],
            "abstract": "To achieve the long-term goal of machines being able to engage humans in\nconversation, our models should captivate the interest of their speaking\npartners. Communication grounded in images, whereby a dialogue is conducted\nbased on a given photo, is a setup naturally appealing to humans (Hu et al.,\n2014). In this work we study large-scale architectures and datasets for this\ngoal. We test a set of neural architectures using state-of-the-art image and\ntext representations, considering various ways to fuse the components. To test\nsuch models, we collect a dataset of grounded human-human conversations, where\nspeakers are asked to play roles given a provided emotional mood or style, as\nthe use of such traits is also a key factor in engagingness (Guo et al., 2019).\nOur dataset, Image-Chat, consists of 202k dialogues over 202k images using 215\npossible style traits. Automatic metrics and human evaluations of engagingness\nshow the efficacy of our approach; in particular, we obtain state-of-the-art\nperformance on the existing IGC task, and our best performing model is almost\non par with humans on the Image-Chat test set (preferred 47.7% of the time).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00960v1",
            "date": "2018-11-02 16:20:13+00:00",
            "title": "Improving the Coverage and the Generalization Ability of Neural Word Sense Disambiguation through Hypernymy and Hyponymy Relationships",
            "authors": [
                "Lo\u00efc Vial",
                "Benjamin Lecouteux",
                "Didier Schwab"
            ],
            "abstract": "In Word Sense Disambiguation (WSD), the predominant approach generally\ninvolves a supervised system trained on sense annotated corpora. The limited\nquantity of such corpora however restricts the coverage and the performance of\nthese systems. In this article, we propose a new method that solves these\nissues by taking advantage of the knowledge present in WordNet, and especially\nthe hypernymy and hyponymy relationships between synsets, in order to reduce\nthe number of different sense tags that are necessary to disambiguate all words\nof the lexical database. Our method leads to state of the art results on most\nWSD evaluation tasks, while improving the coverage of supervised systems,\nreducing the training time and the size of the models, without additional\ntraining data. In addition, we exhibit results that significantly outperform\nthe state of the art when our method is combined with an ensembling technique\nand the addition of the WordNet Gloss Tagged as training corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00967v1",
            "date": "2018-11-02 16:31:22+00:00",
            "title": "Neural Response Ranking for Social Conversation: A Data-Efficient Approach",
            "authors": [
                "Igor Shalyminov",
                "Ond\u0159ej Du\u0161ek",
                "Oliver Lemon"
            ],
            "abstract": "The overall objective of 'social' dialogue systems is to support engaging,\nentertaining, and lengthy conversations on a wide variety of topics, including\nsocial chit-chat. Apart from raw dialogue data, user-provided ratings are the\nmost common signal used to train such systems to produce engaging responses. In\nthis paper we show that social dialogue systems can be trained effectively from\nraw unannotated data. Using a dataset of real conversations collected in the\n2017 Alexa Prize challenge, we developed a neural ranker for selecting 'good'\nsystem responses to user utterances, i.e. responses which are likely to lead to\nlong and engaging conversations. We show that (1) our neural ranker\nconsistently outperforms several strong baselines when trained to optimise for\nuser ratings; (2) when trained on larger amounts of data and only using\nconversation length as the objective, the ranker performs better than the one\ntrained using ratings -- ultimately reaching a Precision@1 of 0.87. This\nadvance will make data collection for social conversational agents simpler and\nless expensive in the future.",
            "journal ref": "Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International\n  Workshop on Search-Oriented Conversational AI, pages 1-8. ISBN\n  978-1-948087-75-9",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01013v2",
            "date": "2018-11-02 17:57:42+00:00",
            "title": "Simple Attention-Based Representation Learning for Ranking Short Social Media Posts",
            "authors": [
                "Peng Shi",
                "Jinfeng Rao",
                "Jimmy Lin"
            ],
            "abstract": "This paper explores the problem of ranking short social media posts with\nrespect to user queries using neural networks. Instead of starting with a\ncomplex architecture, we proceed from the bottom up and examine the\neffectiveness of a simple, word-level Siamese architecture augmented with\nattention-based mechanisms for capturing semantic \"soft\" matches between query\nand post tokens. Extensive experiments on datasets from the TREC Microblog\nTracks show that our simple models not only achieve better effectiveness than\nexisting approaches that are far more complex or exploit a more diverse set of\nrelevance signals, but are also much faster. Implementations of our samCNN\n(Simple Attention-based Matching CNN) models are shared with the community to\nsupport future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01062v2",
            "date": "2018-11-02 19:20:53+00:00",
            "title": "Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations",
            "authors": [
                "Matthias Lalisse",
                "Paul Smolensky"
            ],
            "abstract": "Neural models of Knowledge Base data have typically employed compositional\nrepresentations of graph objects: entity and relation embeddings are\nsystematically combined to evaluate the truth of a candidate Knowedge Base\nentry. Using a model inspired by Harmonic Grammar, we propose to tokenize\ntriplet embeddings by subjecting them to a process of optimization with respect\nto learned well-formedness conditions on Knowledge Base triplets. The resulting\nmodel, known as Gradient Graphs, leads to sizable improvements when implemented\nas a companion to compositional models. Also, we show that the\n\"supracompositional\" triplet token embeddings it produces have interpretable\nproperties that prove helpful in performing inference on the resulting triplet\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01063v2",
            "date": "2018-11-02 19:38:18+00:00",
            "title": "Augmenting Neural Response Generation with Context-Aware Topical Attention",
            "authors": [
                "Nouha Dziri",
                "Ehsan Kamalloo",
                "Kory W. Mathewson",
                "Osmar Zaiane"
            ],
            "abstract": "Sequence-to-Sequence (Seq2Seq) models have witnessed a notable success in\ngenerating natural conversational exchanges. Notwithstanding the syntactically\nwell-formed responses generated by these neural network models, they are prone\nto be acontextual, short and generic. In this work, we introduce a Topical\nHierarchical Recurrent Encoder Decoder (THRED), a novel, fully data-driven,\nmulti-turn response generation system intended to produce contextual and\ntopic-aware responses. Our model is built upon the basic Seq2Seq model by\naugmenting it with a hierarchical joint attention mechanism that incorporates\ntopical concepts and previous interactions into the response generation. To\ntrain our model, we provide a clean and high-quality conversational dataset\nmined from Reddit comments. We evaluate THRED on two novel automated metrics,\ndubbed Semantic Similarity and Response Echo Index, as well as with human\nevaluation. Our experiments demonstrate that the proposed model is able to\ngenerate more diverse and contextually relevant responses compared to the\nstrong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01064v1",
            "date": "2018-11-02 19:41:44+00:00",
            "title": "Neural Machine Translation into Language Varieties",
            "authors": [
                "Surafel M. Lakew",
                "Aliia Erofeeva",
                "Marcello Federico"
            ],
            "abstract": "Both research and commercial machine translation have so far neglected the\nimportance of properly handling the spelling, lexical and grammar divergences\noccurring among language varieties. Notable cases are standard national\nvarieties such as Brazilian and European Portuguese, and Canadian and European\nFrench, which popular online machine translation services are not keeping\ndistinct. We show that an evident side effect of modeling such varieties as\nunique classes is the generation of inconsistent translations. In this work, we\ninvestigate the problem of training neural machine translation from English to\nspecific pairs of language varieties, assuming both labeled and unlabeled\nparallel texts, and low-resource conditions. We report experiments from English\nto two pairs of dialects, EuropeanBrazilian Portuguese and European-Canadian\nFrench, and two pairs of standardized varieties, Croatian-Serbian and\nIndonesian-Malay. We show significant BLEU score improvements over baseline\nsystems when translation into similar languages is learned as a multilingual\ntask with shared representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01088v2",
            "date": "2018-11-02 21:04:24+00:00",
            "title": "Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks",
            "authors": [
                "Jason Phang",
                "Thibault F\u00e9vry",
                "Samuel R. Bowman"
            ],
            "abstract": "Pretraining sentence encoders with language modeling and related unsupervised\ntasks has recently been shown to be very effective for language understanding\ntasks. By supplementing language model-style pretraining with further training\non data-rich supervised tasks, such as natural language inference, we obtain\nadditional performance improvements on the GLUE benchmark. Applying\nsupplementary training on BERT (Devlin et al., 2018), we attain a GLUE score of\n81.8---the state of the art (as of 02/24/2019) and a 1.4 point improvement over\nBERT. We also observe reduced variance across random restarts in this setting.\nOur approach yields similar improvements when applied to ELMo (Peters et al.,\n2018a) and Radford et al. (2018)'s model. In addition, the benefits of\nsupplementary training are particularly pronounced in data-constrained regimes,\nas we show in experiments with artificially limited training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01090v2",
            "date": "2018-11-02 21:14:46+00:00",
            "title": "Value-based Search in Execution Space for Mapping Instructions to Programs",
            "authors": [
                "Dor Muhlgay",
                "Jonathan Herzig",
                "Jonathan Berant"
            ],
            "abstract": "Training models to map natural language instructions to programs given target\nworld supervision only requires searching for good programs at training time.\nSearch is commonly done using beam search in the space of partial programs or\nprogram trees, but as the length of the instructions grows finding a good\nprogram becomes difficult. In this work, we propose a search algorithm that\nuses the target world state, known at training time, to train a critic network\nthat predicts the expected reward of every search state. We then score search\nstates on the beam by interpolating their expected reward with the likelihood\nof programs represented by the search state. Moreover, we search not in the\nspace of programs but in a more compressed state of program executions,\naugmented with recent entities and actions. On the SCONE dataset, we show that\nour algorithm dramatically improves performance on all three domains compared\nto standard beam search and other baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01100v1",
            "date": "2018-11-02 21:33:40+00:00",
            "title": "Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization",
            "authors": [
                "Jiacheng Zhang",
                "Yang Liu",
                "Huanbo Luan",
                "Jingfang Xu",
                "Maosong Sun"
            ],
            "abstract": "Although neural machine translation has made significant progress recently,\nhow to integrate multiple overlapping, arbitrary prior knowledge sources\nremains a challenge. In this work, we propose to use posterior regularization\nto provide a general framework for integrating prior knowledge into neural\nmachine translation. We represent prior knowledge sources as features in a\nlog-linear model, which guides the learning process of the neural translation\nmodel. Experiments on Chinese-English translation show that our approach leads\nto significant improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01115v1",
            "date": "2018-11-02 22:52:22+00:00",
            "title": "Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer",
            "authors": [
                "Sujay Kumar Jauhar",
                "Michael Gamon",
                "Patrick Pantel"
            ],
            "abstract": "Natural language processing is heavily Anglo-centric, while the demand for\nmodels that work in languages other than English is greater than ever. Yet, the\ntask of transferring a model from one language to another can be expensive in\nterms of annotation costs, engineering time and effort. In this paper, we\npresent a general framework for easily and effectively transferring neural\nmodels from English to other languages. The framework, which relies on task\nrepresentations as a form of weak supervision, is model and task agnostic,\nmeaning that many existing neural architectures can be ported to other\nlanguages with minimal effort. The only requirement is unlabeled parallel data,\nand a loss defined over task representations. We evaluate our framework by\ntransferring an English sentiment classifier to three different languages. On a\nbattery of tests, we show that our models outperform a number of strong\nbaselines and rival state-of-the-art results, which rely on more complex\napproaches and significantly more resources and data. Additionally, we find\nthat the framework proposed in this paper is able to capture semantically rich\nand meaningful representations across languages, despite the lack of direct\nsupervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01116v2",
            "date": "2018-11-02 22:55:04+00:00",
            "title": "Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation",
            "authors": [
                "Xing Niu",
                "Weijia Xu",
                "Marine Carpuat"
            ],
            "abstract": "We aim to better exploit the limited amounts of parallel text available in\nlow-resource settings by introducing a differentiable reconstruction loss for\nneural machine translation (NMT). This loss compares original inputs to\nreconstructed inputs, obtained by back-translating translation hypotheses into\nthe input language. We leverage differentiable sampling and bi-directional NMT\nto train models end-to-end, without introducing additional parameters. This\napproach achieves small but consistent BLEU improvements on four language pairs\nin both translation directions, and outperforms an alternative differentiable\nreconstruction strategy based on hidden states.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01137v1",
            "date": "2018-11-03 00:38:45+00:00",
            "title": "Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary",
            "authors": [
                "Surafel M. Lakew",
                "Aliia Erofeeva",
                "Matteo Negri",
                "Marcello Federico",
                "Marco Turchi"
            ],
            "abstract": "We propose a method to transfer knowledge across neural machine translation\n(NMT) models by means of a shared dynamic vocabulary. Our approach allows to\nextend an initial model for a given language pair to cover new languages by\nadapting its vocabulary as long as new data become available (i.e., introducing\nnew vocabulary items if they are not included in the initial model). The\nparameter transfer mechanism is evaluated in two scenarios: i) to adapt a\ntrained single language NMT system to work with a new language pair and ii) to\ncontinuously add new language pairs to grow to a multilingual NMT system. In\nboth the scenarios our goal is to improve the translation performance, while\nminimizing the training convergence time. Preliminary experiments spanning five\nlanguages with different training data sizes (i.e., 5k and 50k parallel\nsentences) show a significant performance gain ranging from +3.85 up to +13.63\nBLEU in different language directions. Moreover, when compared with training an\nNMT model from scratch, our transfer-learning approach allows us to reach\nhigher performance after training up to 4% of the total training steps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01157v1",
            "date": "2018-11-03 04:22:52+00:00",
            "title": "Identifying and Controlling Important Neurons in Neural Machine Translation",
            "authors": [
                "Anthony Bau",
                "Yonatan Belinkov",
                "Hassan Sajjad",
                "Nadir Durrani",
                "Fahim Dalvi",
                "James Glass"
            ],
            "abstract": "Neural machine translation (NMT) models learn representations containing\nsubstantial linguistic information. However, it is not clear if such\ninformation is fully distributed or if some of it can be attributed to\nindividual neurons. We develop unsupervised methods for discovering important\nneurons in NMT models. Our methods rely on the intuition that different models\nlearn similar properties, and do not require any costly external supervision.\nWe show experimentally that translation quality depends on the discovered\nneurons, and find that many of them capture common linguistic phenomena.\nFinally, we show how to control NMT translations in predictable ways, by\nmodifying activations of individual neurons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01241v2",
            "date": "2018-11-03 16:11:29+00:00",
            "title": "Wizard of Wikipedia: Knowledge-Powered Conversational agents",
            "authors": [
                "Emily Dinan",
                "Stephen Roller",
                "Kurt Shuster",
                "Angela Fan",
                "Michael Auli",
                "Jason Weston"
            ],
            "abstract": "In open-domain dialogue intelligent agents should exhibit the use of\nknowledge, however there are few convincing demonstrations of this to date. The\nmost popular sequence to sequence models typically \"generate and hope\" generic\nutterances that can be memorized in the weights of the model when mapping from\ninput utterance(s) to output, rather than employing recalled knowledge as\ncontext. Use of knowledge has so far proved difficult, in part because of the\nlack of a supervised learning benchmark task which exhibits knowledgeable open\ndialogue with clear grounding. To that end we collect and release a large\ndataset with conversations directly grounded with knowledge retrieved from\nWikipedia. We then design architectures capable of retrieving knowledge,\nreading and conditioning on it, and finally generating natural responses. Our\nbest performing dialogue models are able to conduct knowledgeable discussions\non open-domain topics as evaluated by automatic metrics and human evaluations,\nwhile our new benchmark allows for measuring further improvements in this\nimportant research direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01275v2",
            "date": "2018-11-03 20:02:17+00:00",
            "title": "Challenges in detecting evolutionary forces in language change using diachronic corpora",
            "authors": [
                "Andres Karjus",
                "Richard A. Blythe",
                "Simon Kirby",
                "Kenny Smith"
            ],
            "abstract": "Newberry et al. (Detecting evolutionary forces in language change, Nature\n551, 2017) tackle an important but difficult problem in linguistics, the\ntesting of selective theories of language change against a null model of drift.\nHaving applied a test from population genetics (the Frequency Increment Test)\nto a number of relevant examples, they suggest stochasticity has a previously\nunder-appreciated role in language evolution. We replicate their results and\nfind that while the overall observation holds, results produced by this\napproach on individual time series can be sensitive to how the corpus is\norganized into temporal segments (binning). Furthermore, we use a large set of\nsimulations in conjunction with binning to systematically explore the range of\napplicability of the Frequency Increment Test. We conclude that care should be\nexercised with interpreting results of tests like the Frequency Increment Test\non individual series, given the researcher degrees of freedom available when\napplying the test to corpus data, and fundamental differences between genetic\nand linguistic data. Our findings have implications for selection testing and\ntemporal binning in general, as well as demonstrating the usefulness of\nsimulations for evaluating methods newly introduced to the field.",
            "journal ref": "Glossa: a journal of general linguistics, 5(1) (2020), p.45",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01331v1",
            "date": "2018-11-04 07:38:17+00:00",
            "title": "Elastic CRFs for Open-ontology Slot Filling",
            "authors": [
                "Yinpei Dai",
                "Yichi Zhang",
                "Zhijian Ou",
                "Yanmeng Wang",
                "Junlan Feng"
            ],
            "abstract": "Slot filling is a crucial component in task-oriented dialog systems, which is\nto parse (user) utterances into semantic concepts called slots. An ontology is\ndefined by the collection of slots and the values that each slot can take. The\nwidely-used practice of treating slot filling as a sequence labeling task\nsuffers from two drawbacks. First, the ontology is usually pre-defined and\nfixed. Most current methods are unable to predict new labels for unseen slots.\nSecond, the one-hot encoding of slot labels ignores the semantic meanings and\nrelations for slots, which are implicit in their natural language descriptions.\nThese observations motivate us to propose a novel model called elastic\nconditional random field (eCRF), for open-ontology slot filling. eCRFs can\nleverage the neural features of both the utterance and the slot descriptions,\nand are able to model the interactions between different slots. Experimental\nresults show that eCRFs outperforms existing models on both the in-domain and\nthe cross-doamin tasks, especially in predictions of unseen slots and values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01355v1",
            "date": "2018-11-04 12:00:37+00:00",
            "title": "Semi-Supervised Confidence Network aided Gated Attention based Recurrent Neural Network for Clickbait Detection",
            "authors": [
                "Amrith Rajagopal Setlur"
            ],
            "abstract": "Clickbaits are catchy headlines that are frequently used by social media\noutlets in order to allure its viewers into clicking them and thus leading them\nto dubious content. Such venal schemes thrive on exploiting the curiosity of\nnaive social media users, directing traffic to web pages that won't be visited\notherwise. In this paper, we propose a novel, semi-supervised classification\nbased approach, that employs attentions sampled from a Gumbel-Softmax\ndistribution to distill contexts that are fairly important in clickbait\ndetection. An additional loss over the attention weights is used to encode\nprior knowledge. Furthermore, we propose a confidence network that enables\nlearning over weak labels and improves robustness to noisy labels. We show that\nwith merely 30% of strongly labeled samples we can achieve over 97% of the\naccuracy, of current state of the art methods in clickbait detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01389v1",
            "date": "2018-11-04 15:41:42+00:00",
            "title": "Improving Zero-Shot Translation of Low-Resource Languages",
            "authors": [
                "Surafel M. Lakew",
                "Quintino F. Lotito",
                "Matteo Negri",
                "Marco Turchi",
                "Marcello Federico"
            ],
            "abstract": "Recent work on multilingual neural machine translation reported competitive\nperformance with respect to bilingual models and surprisingly good performance\neven on (zeroshot) translation directions not observed at training time. We\ninvestigate here a zero-shot translation in a particularly lowresource\nmultilingual setting. We propose a simple iterative training procedure that\nleverages a duality of translations directly generated by the system for the\nzero-shot directions. The translations produced by the system (sub-optimal\nsince they contain mixed language from the shared vocabulary), are then used\ntogether with the original parallel data to feed and iteratively re-train the\nmultilingual network. Over time, this allows the system to learn from its own\ngenerated and increasingly better output. Our approach shows to be effective in\nimproving the two zero-shot directions of our multilingual model. In\nparticular, we observed gains of about 9 BLEU points over a baseline\nmultilingual model and up to 2.08 BLEU over a pivoting mechanism using two\nbilingual models. Further analysis shows that there is also a slight\nimprovement in the non-zero-shot language directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01697v2",
            "date": "2018-11-05 14:09:53+00:00",
            "title": "Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification",
            "authors": [
                "Wei Shi",
                "Vera Demberg"
            ],
            "abstract": "Implicit discourse relation classification is one of the most difficult steps\nin discourse parsing. The difficulty stems from the fact that the coherence\nrelation must be inferred based on the content of the discourse relational\narguments. Therefore, an effective encoding of the relational arguments is of\ncrucial importance. We here propose a new model for implicit discourse relation\nclassification, which consists of a classifier, and a sequence-to-sequence\nmodel which is trained to generate a representation of the discourse relational\narguments by trying to predict the relational arguments including a suitable\nimplicit connective. Training is possible because such implicit connectives\nhave been annotated as part of the PDTB corpus. Along with a memory network,\nour model could generate more refined representations for the task. And on the\nnow standard 11-way classification, our method outperforms previous state of\nthe art systems on the PDTB benchmark on multiple settings including cross\nvalidation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01866v1",
            "date": "2018-11-05 17:32:14+00:00",
            "title": "Do RNNs learn human-like abstract word order preferences?",
            "authors": [
                "Richard Futrell",
                "Roger P. Levy"
            ],
            "abstract": "RNN language models have achieved state-of-the-art results on various tasks,\nbut what exactly they are representing about syntax is as yet unclear. Here we\ninvestigate whether RNN language models learn humanlike word order preferences\nin syntactic alternations. We collect language model surprisal scores for\ncontrolled sentence stimuli exhibiting major syntactic alternations in English:\nheavy NP shift, particle shift, the dative alternation, and the genitive\nalternation. We show that RNN language models reproduce human preferences in\nthese alternations based on NP length, animacy, and definiteness. We collect\nhuman acceptability ratings for our stimuli, in the first acceptability\njudgment experiment directly manipulating the predictors of syntactic\nalternations. We show that the RNNs' performance is similar to the human\nacceptability ratings and is not matched by an n-gram baseline model. Our\nresults show that RNNs learn the abstract features of weight, animacy, and\ndefiniteness which underlie soft constraints on syntactic alternations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01990v1",
            "date": "2018-11-05 19:29:56+00:00",
            "title": "Compact Personalized Models for Neural Machine Translation",
            "authors": [
                "Joern Wuebker",
                "Patrick Simianer",
                "John DeNero"
            ],
            "abstract": "We propose and compare methods for gradient-based domain adaptation of\nself-attentive neural machine translation models. We demonstrate that a large\nproportion of model parameters can be frozen during adaptation with minimal or\nno reduction in translation quality by encouraging structured sparsity in the\nset of offset tensors during learning via group lasso regularization. We\nevaluate this technique for both batch and incremental adaptation across\nmultiple data sets and language pairs. Our system architecture - combining a\nstate-of-the-art self-attentive model with compact domain adaptation - provides\nhigh quality personalized machine translation that is both space and time\nefficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02058v2",
            "date": "2018-11-05 22:18:31+00:00",
            "title": "The Marchex 2018 English Conversational Telephone Speech Recognition System",
            "authors": [
                "Seongjun Hahm",
                "Iroro Orife",
                "Shane Walker",
                "Jason Flaks"
            ],
            "abstract": "In this paper, we describe recent performance improvements to the production\nMarchex speech recognition system for our spontaneous customer-to-business\ntelephone conversations. In our previous work, we focused on in-domain language\nand acoustic model training. In this work we employ state-of-the-art\nsemi-supervised lattice-free maximum mutual information (LF-MMI) training\nprocess which can supervise over full lattices from unlabeled audio. On Marchex\nEnglish (ME), a modern evaluation set of conversational North American English,\nwe observed a 3.3% (3.2% for agent, 3.6% for caller) reduction in absolute word\nerror rate (WER) with 3x faster decoding speed over the performance of the 2017\nproduction system. We expect this improvement boost Marchex Call Analytics\nsystem performance especially for natural language processing pipeline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02076v1",
            "date": "2018-11-05 23:03:02+00:00",
            "title": "Improving Span-based Question Answering Systems with Coarsely Labeled Data",
            "authors": [
                "Hao Cheng",
                "Ming-Wei Chang",
                "Kenton Lee",
                "Ankur Parikh",
                "Michael Collins",
                "Kristina Toutanova"
            ],
            "abstract": "We study approaches to improve fine-grained short answer Question Answering\nmodels by integrating coarse-grained data annotated for paragraph-level\nrelevance and show that coarsely annotated data can bring significant\nperformance gains. Experiments demonstrate that the standard multi-task\nlearning approach of sharing representations is not the most effective way to\nleverage coarse-grained annotations. Instead, we can explicitly model the\nlatent fine-grained short answer variables and optimize the marginal\nlog-likelihood directly or use a newly proposed \\emph{posterior distillation}\nlearning objective. Since these latent-variable methods have explicit access to\nthe relationship between the fine and coarse tasks, they result in\nsignificantly larger improvements from coarse supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02134v2",
            "date": "2018-11-06 02:46:23+00:00",
            "title": "Transfer learning of language-independent end-to-end ASR with language model fusion",
            "authors": [
                "Hirofumi Inaguma",
                "Jaejin Cho",
                "Murali Karthick Baskar",
                "Tatsuya Kawahara",
                "Shinji Watanabe"
            ],
            "abstract": "This work explores better adaptation methods to low-resource languages using\nan external language model (LM) under the framework of transfer learning. We\nfirst build a language-independent ASR system in a unified sequence-to-sequence\n(S2S) architecture with a shared vocabulary among all languages. During\nadaptation, we perform LM fusion transfer, where an external LM is integrated\ninto the decoder network of the attention-based S2S model in the whole\nadaptation stage, to effectively incorporate linguistic context of the target\nlanguage. We also investigate various seed models for transfer learning.\nExperimental evaluations using the IARPA BABEL data set show that LM fusion\ntransfer improves performances on all target five languages compared with\nsimple transfer learning when the external text data is available. Our final\nsystem drastically reduces the performance gap from the hybrid systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02230v1",
            "date": "2018-11-06 08:57:53+00:00",
            "title": "CIS at TAC Cold Start 2015: Neural Networks and Coreference Resolution for Slot Filling",
            "authors": [
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper describes the CIS slot filling system for the TAC Cold Start\nevaluations 2015. It extends and improves the system we have built for the\nevaluation last year. This paper mainly describes the changes to our last\nyear's system. Especially, it focuses on the coreference and classification\ncomponent. For coreference, we have performed several analysis and prepared a\nresource to simplify our end-to-end system and improve its runtime. For\nclassification, we propose to use neural networks. We have trained\nconvolutional and recurrent neural networks and combined them with traditional\nevaluation methods, namely patterns and support vector machines. Our runs for\nthe 2015 evaluation have been designed to directly assess the effect of each\nnetwork on the end-to-end performance of the system. The CIS system achieved\nrank 3 of all slot filling systems participating in the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02278v1",
            "date": "2018-11-06 10:50:04+00:00",
            "title": "Off-the-Shelf Unsupervised NMT",
            "authors": [
                "Chris Hokamp",
                "Sebastian Ruder",
                "John Glover"
            ],
            "abstract": "We frame unsupervised machine translation (MT) in the context of multi-task\nlearning (MTL), combining insights from both directions. We leverage\noff-the-shelf neural MT architectures to train unsupervised MT models with no\nparallel data and show that such models can achieve reasonably good\nperformance, competitive with models purpose-built for unsupervised MT.\nFinally, we propose improvements that allow us to apply our models to\nEnglish-Turkish, a truly low-resource language pair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02338v2",
            "date": "2018-11-06 13:12:22+00:00",
            "title": "Learning to Embed Sentences Using Attentive Recursive Trees",
            "authors": [
                "Jiaxin Shi",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu",
                "Hanwang Zhang"
            ],
            "abstract": "Sentence embedding is an effective feature representation for most deep\nlearning-based NLP tasks. One prevailing line of methods is using recursive\nlatent tree-structured networks to embed sentences with task-specific\nstructures. However, existing models have no explicit mechanism to emphasize\ntask-informative words in the tree structure. To this end, we propose an\nAttentive Recursive Tree model (AR-Tree), where the words are dynamically\nlocated according to their importance in the task. Specifically, we construct\nthe latent tree for a sentence in a proposed important-first strategy, and\nplace more attentive words nearer to the root; thus, AR-Tree can inherently\nemphasize important words during the bottom-up composition of the sentence\nembedding. We propose an end-to-end reinforced training strategy for AR-Tree,\nwhich is demonstrated to consistently outperform, or be at least comparable to,\nthe state-of-the-art sentence embedding methods on three sentence understanding\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02356v4",
            "date": "2018-11-06 14:07:15+00:00",
            "title": "Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation",
            "authors": [
                "Ching-Ting Chang",
                "Shun-Po Chuang",
                "Hung-Yi Lee"
            ],
            "abstract": "Code-switching is about dealing with alternative languages in speech or text.\nIt is partially speaker-depend and domain-related, so completely explaining the\nphenomenon by linguistic rules is challenging. Compared to most monolingual\ntasks, insufficient data is an issue for code-switching. To mitigate the issue\nwithout expensive human annotation, we proposed an unsupervised method for\ncode-switching data augmentation. By utilizing a generative adversarial\nnetwork, we can generate intra-sentential code-switching sentences from\nmonolingual sentences. We applied proposed method on two corpora, and the\nresult shows that the generated code-switching sentences improve the\nperformance of code-switching language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02364v2",
            "date": "2018-11-06 14:24:37+00:00",
            "title": "Effective Subword Segmentation for Text Comprehension",
            "authors": [
                "Zhuosheng Zhang",
                "Hai Zhao",
                "Kangwei Ling",
                "Jiangtong Li",
                "Zuchao Li",
                "Shexia He",
                "Guohong Fu"
            ],
            "abstract": "Representation learning is the foundation of machine reading comprehension\nand inference. In state-of-the-art models, character-level representations have\nbeen broadly adopted to alleviate the problem of effectively representing rare\nor complex words. However, character itself is not a natural minimal linguistic\nunit for representation or word embedding composing due to ignoring the\nlinguistic coherence of consecutive characters inside word. This paper presents\na general subword-augmented embedding framework for learning and composing\ncomputationally-derived subword-level representations. We survey a series of\nunsupervised segmentation methods for subword acquisition and different\nsubword-augmented strategies for text understanding, showing that\nsubword-augmented embedding significantly improves our baselines in various\ntypes of text understanding tasks on both English and Chinese benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02394v2",
            "date": "2018-11-06 15:06:44+00:00",
            "title": "DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization",
            "authors": [
                "Jiaxin Shi",
                "Chen Liang",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu",
                "Hanwang Zhang"
            ],
            "abstract": "We propose DeepChannel, a robust, data-efficient, and interpretable neural\nmodel for extractive document summarization. Given any document-summary pair,\nwe estimate a salience score, which is modeled using an attention-based deep\nneural network, to represent the salience degree of the summary for yielding\nthe document. We devise a contrastive training strategy to learn the salience\nestimation network, and then use the learned salience score as a guide and\niteratively extract the most salient sentences from the document as our\ngenerated summary. In experiments, our model not only achieves state-of-the-art\nROUGE scores on CNN/Daily Mail dataset, but also shows strong robustness in the\nout-of-domain test on DUC2007 test set. Moreover, our model reaches a ROUGE-1\nF-1 score of 39.41 on CNN/Daily Mail test set with merely $1 / 100$ training\nset, demonstrating a tremendous data efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02435v1",
            "date": "2018-11-06 15:42:17+00:00",
            "title": "WordNet-feelings: A linguistic categorisation of human feelings",
            "authors": [
                "Advaith Siddharthan",
                "Nicolas Cherbuin",
                "Paul J. Eslinger",
                "Kasia Kozlowska",
                "Nora A. Murphy",
                "Leroy Lowe"
            ],
            "abstract": "In this article, we present the first in depth linguistic study of human\nfeelings. While there has been substantial research on incorporating some\naffective categories into linguistic analysis (e.g. sentiment, and to a lesser\nextent, emotion), the more diverse category of human feelings has thus far not\nbeen investigated. We surveyed the extensive interdisciplinary literature\naround feelings to construct a working definition of what constitutes a feeling\nand propose 9 broad categories of feeling. We identified potential feeling\nwords based on their pointwise mutual information with morphological variants\nof the word `feel' in the Google n-gram corpus, and present a manual annotation\nexercise where 317 WordNet senses of one hundred of these words were\ncategorised as `not a feeling' or as one of the 9 proposed categories of\nfeeling. We then proceeded to annotate 11386 WordNet senses of all these words\nto create WordNet-feelings, a new affective dataset that identifies 3664 word\nsenses as feelings, and associates each of these with one of the 9 categories\nof feeling. WordNet-feelings can be used in conjunction with other datasets\nsuch as SentiWordNet that annotate word senses with complementary affective\nproperties such as valence and intensity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02510v1",
            "date": "2018-11-06 17:25:21+00:00",
            "title": "UAlacant machine translation quality estimation at WMT 2018: a simple approach using phrase tables and feed-forward neural networks",
            "authors": [
                "Miquel Espl\u00e0-Gomis",
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Mikel L. Forcada"
            ],
            "abstract": "We describe the Universitat d'Alacant submissions to the word- and\nsentence-level machine translation (MT) quality estimation (QE) shared task at\nWMT 2018. Our approach to word-level MT QE builds on previous work to mark the\nwords in the machine-translated sentence as \\textit{OK} or \\textit{BAD}, and is\nextended to determine if a word or sequence of words need to be inserted in the\ngap after each word. Our sentence-level submission simply uses the edit\noperations predicted by the word-level approach to approximate TER. The method\npresented ranked first in the sub-task of identifying insertions in gaps for\nthree out of the six datasets, and second in the rest of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02602v2",
            "date": "2018-11-06 19:19:49+00:00",
            "title": "Fast Neural Chinese Word Segmentation for Long Sentences",
            "authors": [
                "Sufeng Duan",
                "Jiangtong Li",
                "Hai Zhao"
            ],
            "abstract": "Rapidly developed neural models have achieved competitive performance in\nChinese word segmentation (CWS) as their traditional counterparts. However,\nmost of methods encounter the computational inefficiency especially for long\nsentences because of the increasing model complexity and slower decoders. This\npaper presents a simple neural segmenter which directly labels the gap\nexistence between adjacent characters to alleviate the existing drawback. Our\nsegmenter is fully end-to-end and capable of performing segmentation very fast.\nWe also show a performance difference with different tag sets. The experiments\nshow that our segmenter can provide comparable performance with\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02611v1",
            "date": "2018-11-06 20:07:47+00:00",
            "title": "Evaluating the Ability of LSTMs to Learn Context-Free Grammars",
            "authors": [
                "Luzi Sennhauser",
                "Robert C. Berwick"
            ],
            "abstract": "While long short-term memory (LSTM) neural net architectures are designed to\ncapture sequence information, human language is generally composed of\nhierarchical structures. This raises the question as to whether LSTMs can learn\nhierarchical structures. We explore this question with a well-formed bracket\nprediction task using two types of brackets modeled by an LSTM. Demonstrating\nthat such a system is learnable by an LSTM is the first step in demonstrating\nthat the entire class of CFLs is also learnable. We observe that the model\nrequires exponential memory in terms of the number of characters and embedded\ndepth, where a sub-linear memory should suffice. Still, the model does more\nthan memorize the training input. It learns how to distinguish between relevant\nand irrelevant information. On the other hand, we also observe that the model\ndoes not generalize well. We conclude that LSTMs do not learn the relevant\nunderlying context-free rules, suggesting the good overall performance is\nattained rather by an efficient way of evaluating nuisance variables. LSTMs are\na way to quickly reach good results for many natural language tasks, but to\nunderstand and generate natural language one has to investigate other concepts\nthat can make more direct use of natural language's structural nature.",
            "journal ref": "Proceedings of the EMNLP Workshop BlackboxNLP (2018) 115-124",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02641v1",
            "date": "2018-11-06 20:59:54+00:00",
            "title": "Building Corpora for Single-Channel Speech Separation Across Multiple Domains",
            "authors": [
                "Matthew Maciejewski",
                "Gregory Sell",
                "Leibny Paola Garcia-Perera",
                "Shinji Watanabe",
                "Sanjeev Khudanpur"
            ],
            "abstract": "To date, the bulk of research on single-channel speech separation has been\nconducted using clean, near-field, read speech, which is not representative of\nmany modern applications. In this work, we develop a procedure for constructing\nhigh-quality synthetic overlap datasets, necessary for most deep learning-based\nseparation frameworks. We produced datasets that are more representative of\nrealistic applications using the CHiME-5 and Mixer 6 corpora and evaluate\nstandard methods on this data to demonstrate the shortcomings of current\nsource-separation performance. We also demonstrate the value of a wide variety\nof data in training robust models that generalize well to multiple conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02714v2",
            "date": "2018-11-07 01:19:05+00:00",
            "title": "The RLLChatbot: a solution to the ConvAI challenge",
            "authors": [
                "Nicolas Gontier",
                "Koustuv Sinha",
                "Peter Henderson",
                "Iulian Serban",
                "Michael Noseworthy",
                "Prasanna Parthasarathi",
                "Joelle Pineau"
            ],
            "abstract": "Current conversational systems can follow simple commands and answer basic\nquestions, but they have difficulty maintaining coherent and open-ended\nconversations about specific topics. Competitions like the Conversational\nIntelligence (ConvAI) challenge are being organized to push the research\ndevelopment towards that goal. This article presents in detail the RLLChatbot\nthat participated in the 2017 ConvAI challenge. The goal of this research is to\nbetter understand how current deep learning and reinforcement learning tools\ncan be used to build a robust yet flexible open domain conversational agent. We\nprovide a thorough description of how a dialog system can be built and trained\nfrom mostly public-domain datasets using an ensemble model. The first\ncontribution of this work is a detailed description and analysis of different\ntext generation models in addition to novel message ranking and selection\nmethods. Moreover, a new open-source conversational dataset is presented.\nTraining on this data significantly improves the Recall@k score of the ranking\nand selection mechanisms compared to our baseline model responsible for\nselecting the message returned at each interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02775v1",
            "date": "2018-11-07 06:18:14+00:00",
            "title": "Improved Audio Embeddings by Adjacency-Based Clustering with Applications in Spoken Term Detection",
            "authors": [
                "Sung-Feng Huang",
                "Yi-Chen Chen",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Embedding audio signal segments into vectors with fixed dimensionality is\nattractive because all following processing will be easier and more efficient,\nfor example modeling, classifying or indexing. Audio Word2Vec previously\nproposed was shown to be able to represent audio segments for spoken words as\nsuch vectors carrying information about the phonetic structures of the signal\nsegments. However, each linguistic unit (word, syllable, phoneme in text form)\ncorresponds to unlimited number of audio segments with vector representations\ninevitably spread over the embedding space, which causes some confusion. It is\ntherefore desired to better cluster the audio embeddings such that those\ncorresponding to the same linguistic unit can be more compactly distributed. In\nthis paper, inspired by Siamese networks, we propose some approaches to achieve\nthe above goal. This includes identifying positive and negative pairs from\nunlabeled data for Siamese style training, disentangling acoustic factors such\nas speaker characteristics from the audio embedding, handling unbalanced data\ndistribution, and having the embedding processes learn from the adjacency\nrelationships among data points. All these can be done in an unsupervised way.\nImproved performance was obtained in preliminary experiments on the LibriSpeech\ndata set, including clustering characteristics analysis and applications of\nspoken term detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02902v1",
            "date": "2018-11-07 14:31:13+00:00",
            "title": "microNER: A Micro-Service for German Named Entity Recognition based on BiLSTM-CRF",
            "authors": [
                "Gregor Wiedemann",
                "Raghav Jindal",
                "Chris Biemann"
            ],
            "abstract": "For named entity recognition (NER), bidirectional recurrent neural networks\nbecame the state-of-the-art technology in recent years. Competing approaches\nvary with respect to pre-trained word embeddings as well as models for\ncharacter embeddings to represent sequence information most effectively. For\nNER in German language texts, these model variations have not been studied\nextensively. We evaluate the performance of different word and character\nembeddings on two standard German datasets and with a special focus on\nout-of-vocabulary words. With F-Scores above 82% for the GermEval'14 dataset\nand above 85% for the CoNLL'03 dataset, we achieve (near) state-of-the-art\nperformance for this task. We publish several pre-trained models wrapped into a\nmicro-service based on Docker to allow for easy integration of German NER into\nother applications via a JSON API.",
            "journal ref": "Proceedings of the 14th Conference on Natural Language Processing\n  / Konferenz zur Verarbeitung nat\\\"urlicher Sprache (KONVENS 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02906v1",
            "date": "2018-11-07 14:40:10+00:00",
            "title": "Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter",
            "authors": [
                "Gregor Wiedemann",
                "Eugen Ruppert",
                "Raghav Jindal",
                "Chris Biemann"
            ],
            "abstract": "We investigate different strategies for automatic offensive language\nclassification on German Twitter data. For this, we employ a sequentially\ncombined BiLSTM-CNN neural network. Based on this model, three transfer\nlearning tasks to improve the classification performance with background\nknowledge are tested. We compare 1. Supervised category transfer: social media\ndata annotated with near-offensive language categories, 2. Weakly-supervised\ncategory transfer: tweets annotated with emojis they contain, 3. Unsupervised\ncategory transfer: tweets annotated with topic clusters obtained by Latent\nDirichlet Allocation (LDA). Further, we investigate the effect of three\ndifferent strategies to mitigate negative effects of 'catastrophic forgetting'\nduring transfer learning. Our results indicate that transfer learning in\ngeneral improves offensive language detection. Best results are achieved from\npre-training our model on the unsupervised topic clustering of tweets in\ncombination with thematic user cluster information.",
            "journal ref": "Proceedings of GermEval 2018, 14th Conference on Natural Language\n  Processing (KONVENS 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03036v1",
            "date": "2018-11-07 17:40:03+00:00",
            "title": "IMS at the PolEval 2018: A Bulky Ensemble Depedency Parser meets 12 Simple Rules for Predicting Enhanced Dependencies in Polish",
            "authors": [
                "Agnieszka Falenska",
                "Anders Bj\u00f6rkelund",
                "Xiang Yu",
                "Jonas Kuhn"
            ],
            "abstract": "This paper presents the IMS contribution to the PolEval 2018 Shared Task. We\nsubmitted systems for both of the Subtasks of Task 1. In Subtask (A), which was\nabout dependency parsing, we used our ensemble system from the CoNLL 2017 UD\nShared Task. The system first preprocesses the sentences with a CRF\nPOS/morphological tagger and predicts supertags with a neural tagger. Then, it\nemploys multiple instances of three different parsers and merges their outputs\nby applying blending. The system achieved the second place out of four\nparticipating teams. In this paper we show which components of the system were\nthe most responsible for its final performance.\n  The goal of Subtask (B) was to predict enhanced graphs. Our approach\nconsisted of two steps: parsing the sentences with our ensemble system from\nSubtask (A), and applying 12 simple rules to obtain the final dependency\ngraphs. The rules introduce additional enhanced arcs only for tokens with\n\"conj\" heads (conjuncts). They do not predict semantic relations at all. The\nsystem ranked first out of three participating teams. In this paper we show\nexamples of rules we designed and analyze the relation between the quality of\nautomatically parsed trees and the accuracy of the enhanced graphs.",
            "journal ref": "Proceedings of the PolEval 2018 Workshop, 2018, 25-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03039v1",
            "date": "2018-11-07 17:50:05+00:00",
            "title": "Data Selection with Feature Decay Algorithms Using an Approximated Target Side",
            "authors": [
                "Alberto Poncelas",
                "Gideon Maillette de Buy Wenniger",
                "Andy Way"
            ],
            "abstract": "Data selection techniques applied to neural machine translation (NMT) aim to\nincrease the performance of a model by retrieving a subset of sentences for use\nas training data.\n  One of the possible data selection techniques are transductive learning\nmethods, which select the data based on the test set, i.e. the document to be\ntranslated. A limitation of these methods to date is that using the source-side\ntest set does not by itself guarantee that sentences are selected with correct\ntranslations, or translations that are suitable given the test-set domain. Some\ncorpora, such as subtitle corpora, may contain parallel sentences with\ninaccurate translations caused by localization or length restrictions.\n  In order to try to fix this problem, in this paper we propose to use an\napproximated target-side in addition to the source-side when selecting suitable\nsentence-pairs for training a model. This approximated target-side is built by\npre-translating the source-side.\n  In this work, we explore the performance of this general idea for one\nspecific data selection approach called Feature Decay Algorithms (FDA).\n  We train German-English NMT models on data selected by using the test set\n(source), the approximated target side, and a mixture of both. Our findings\nreveal that models built using a combination of outputs of FDA (using the test\nset and an approximated target side) perform better than those solely using the\ntest set. We obtain a statistically significant improvement of more than 1.5\nBLEU points over a model trained with all data, and more than 0.5 BLEU points\nover a strong FDA baseline that uses source-side information only.",
            "journal ref": "Proceedings of the 15th International Workshop on Spoken Language\n  Translation (2018) 173-180",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03189v1",
            "date": "2018-11-07 23:47:01+00:00",
            "title": "Towards Fluent Translations from Disfluent Speech",
            "authors": [
                "Elizabeth Salesky",
                "Susanne Burger",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "When translating from speech, special consideration for conversational speech\nphenomena such as disfluencies is necessary. Most machine translation training\ndata consists of well-formed written texts, causing issues when translating\nspontaneous speech. Previous work has introduced an intermediate step between\nspeech recognition (ASR) and machine translation (MT) to remove disfluencies,\nmaking the data better-matched to typical translation text and significantly\nimproving performance. However, with the rise of end-to-end speech translation\nsystems, this intermediate step must be incorporated into the\nsequence-to-sequence architecture. Further, though translated speech datasets\nexist, they are typically news or rehearsed speech without many disfluencies\n(e.g. TED), or the disfluencies are translated into the references (e.g.\nFisher). To generate clean translations from disfluent speech, cleaned\nreferences are necessary for evaluation. We introduce a corpus of cleaned\ntarget data for the Fisher Spanish-English dataset for this task. We compare\nhow different architectures handle disfluencies and provide a baseline for\nremoving disfluencies in end-to-end translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03199v2",
            "date": "2018-11-08 00:40:25+00:00",
            "title": "Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Panayiotis Georgiou"
            ],
            "abstract": "Word vector representations are a crucial part of Natural Language Processing\n(NLP) and Human Computer Interaction. In this paper, we propose a novel word\nvector representation, Confusion2Vec, motivated from the human speech\nproduction and perception that encodes representational ambiguity. Humans\nemploy both acoustic similarity cues and contextual cues to decode information\nand we focus on a model that incorporates both sources of information. The\nrepresentational ambiguity of acoustics, which manifests itself in word\nconfusions, is often resolved by both humans and machines through contextual\ncues. A range of representational ambiguities can emerge in various domains\nfurther to acoustic perception, such as morphological transformations,\nparaphrasing for NLP tasks like machine translation etc. In this work, we\npresent a case study in application to Automatic Speech Recognition (ASR),\nwhere the word confusions are related to acoustic similarity. We present\nseveral techniques to train an acoustic perceptual similarity representation\nambiguity. We term this Confusion2Vec and learn on unsupervised-generated data\nfrom ASR confusion networks or lattice-like structures. Appropriate evaluations\nfor the Confusion2Vec are formulated for gauging acoustic similarity in\naddition to semantic-syntactic and word similarity evaluations. The\nConfusion2Vec is able to model word confusions efficiently, without\ncompromising on the semantic-syntactic word relations, thus effectively\nenriching the word vector space with extra task relevant ambiguity information.\nWe provide an intuitive exploration of the 2-dimensional Confusion2Vec space\nusing Principal Component Analysis of the embedding and relate to semantic,\nsyntactic and acoustic relationships. The potential of Confusion2Vec in the\nutilization of uncertainty present in lattices is demonstrated through small\nexamples relating to ASR error correction.",
            "journal ref": "PeerJ Computer Science 5:e195, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03245v1",
            "date": "2018-11-08 03:19:59+00:00",
            "title": "Evaluating the Complementarity of Taxonomic Relation Extraction Methods Across Different Languages",
            "authors": [
                "Roger Granada",
                "Renata Vieira",
                "Cassia Trojahn",
                "Nathalie Aussenac-Gilles"
            ],
            "abstract": "Modern information systems are changing the idea of \"data processing\" to the\nidea of \"concept processing\", meaning that instead of processing words, such\nsystems process semantic concepts which carry meaning and share contexts with\nother concepts. Ontology is commonly used as a structure that captures the\nknowledge about a certain area via providing concepts and relations between\nthem. Traditionally, concept hierarchies have been built manually by knowledge\nengineers or domain experts. However, the manual construction of a concept\nhierarchy suffers from several limitations such as its coverage and the\nenormous costs of its extension and maintenance. Ontology learning, usually\nreferred to the (semi-)automatic support in ontology development, is usually\ndivided into steps, going from concepts identification, passing through\nhierarchy and non-hierarchy relations detection and, seldom, axiom extraction.\nIt is reasonable to say that among these steps the current frontier is in the\nestablishment of concept hierarchies, since this is the backbone of ontologies\nand, therefore, a good concept hierarchy is already a valuable resource for\nmany ontology applications. The automatic construction of concept hierarchies\nfrom texts is a complex task and much work have been proposing approaches to\nbetter extract relations between concepts. These different proposals have never\nbeen contrasted against each other on the same set of data and across different\nlanguages. Such comparison is important to see whether they are complementary\nor incremental. Also, we can see whether they present different tendencies\ntowards recall and precision. This paper evaluates these different methods on\nthe basis of hierarchy metrics such as density and depth, and evaluation\nmetrics such as Recall and Precision. Results shed light over the comprehensive\nset of methods according to the literature in the area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03274v1",
            "date": "2018-11-08 05:10:51+00:00",
            "title": "Applying Distributional Compositional Categorical Models of Meaning to Language Translation",
            "authors": [
                "Brian Tyrrell"
            ],
            "abstract": "The aim of this paper is twofold: first we will use vector space\ndistributional compositional categorical models of meaning to compare the\nmeaning of sentences in Irish and in English (and thus ascertain when a\nsentence is the translation of another sentence) using the cosine similarity\nscore. Then we shall outline a procedure which translates nouns by\nunderstanding their context, using a conceptual space model of cognition. We\nshall use metrics on the category ConvexRel to determine the distance between\nconcepts (and determine when a noun is the translation of another noun). This\npaper will focus on applications to Irish, a member of the Gaelic family of\nlanguages.",
            "journal ref": "EPTCS 283, 2018, pp. 28-49",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03399v1",
            "date": "2018-11-08 13:28:01+00:00",
            "title": "Untangling the GDPR Using ConRelMiner",
            "authors": [
                "Karolin Winter",
                "Stefanie Rinderle-Ma"
            ],
            "abstract": "The General Data Protection Regulation (GDPR) poses enormous challenges on\ncompanies and organizations with respect to understanding, implementing, and\nmaintaining the contained constraints. We report on how the ConRelMiner method\ncan be used for untangling the GDPR. For this, the GDPR is filtered and grouped\nalong the roles mentioned by the GDPR and the reduction of sentences to be read\nby analysts is shown. Moreover, the output of the ConRelMiner - a cluster graph\nwith relations between the sentences - is displayed and interpreted. Overall\nthe goal is to illustrate how the effort for implementing the GDPR can be\nreduced and a structured and meaningful representation of the relevant GDPR\nsentences can be found.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03511v3",
            "date": "2018-11-08 15:59:11+00:00",
            "title": "Effective Representation for Easy-First Dependency Parsing",
            "authors": [
                "Zuchao Li",
                "Jiaxun Cai",
                "Hai Zhao"
            ],
            "abstract": "Easy-first parsing relies on subtree re-ranking to build the complete parse\ntree. Whereas the intermediate state of parsing processing is represented by\nvarious subtrees, whose internal structural information is the key lead for\nlater parsing action decisions, we explore a better representation for such\nsubtrees. In detail, this work introduces a bottom-up subtree encoding method\nbased on the child-sum tree-LSTM. Starting from an easy-first dependency parser\nwithout other handcraft features, we show that the effective subtree encoder\ndoes promote the parsing process, and can make a greedy search easy-first\nparser achieve promising results on benchmark treebanks compared to\nstate-of-the-art baselines. Furthermore, with the help of the current\npre-training language model, we further improve the state-of-the-art results of\nthe easy-first approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03519v2",
            "date": "2018-11-08 16:05:50+00:00",
            "title": "Few-shot learning with attention-based sequence-to-sequence models",
            "authors": [
                "Bertrand Higy",
                "Peter Bell"
            ],
            "abstract": "End-to-end approaches have recently become popular as a means of simplifying\nthe training and deployment of speech recognition systems. However, they often\nrequire large amounts of data to perform well on large vocabulary tasks. With\nthe aim of making end-to-end approaches usable by a broader range of\nresearchers, we explore the potential to use end-to-end methods in small\nvocabulary contexts where smaller datasets may be used. A significant drawback\nof small-vocabulary systems is the difficulty of expanding the vocabulary\nbeyond the original training samples -- therefore we also study strategies to\nextend the vocabulary with only few examples per new class (few-shot learning).\n  Our results show that an attention-based encoder-decoder can be competitive\nagainst a strong baseline on a small vocabulary keyword classification task,\nreaching 97.5% of accuracy on Tensorflow's Speech Commands dataset. It also\nshows promising results on the few-shot learning problem where a simple\nstrategy achieved 68.8\\% of accuracy on new keywords with only 10 examples for\neach new class. This score goes up to 88.4\\% with a larger set of 100 examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03554v1",
            "date": "2018-11-08 17:13:46+00:00",
            "title": "Implicit Argument Prediction as Reading Comprehension",
            "authors": [
                "Pengxiang Cheng",
                "Katrin Erk"
            ],
            "abstract": "Implicit arguments, which cannot be detected solely through syntactic cues,\nmake it harder to extract predicate-argument tuples. We present a new model for\nimplicit argument prediction that draws on reading comprehension, casting the\npredicate-argument tuple with the missing argument as a query. We also draw on\npointer networks and multi-hop computation. Our model shows good performance on\nan argument cloze task as well as on a nominal implicit argument prediction\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03604v2",
            "date": "2018-11-08 18:37:03+00:00",
            "title": "Federated Learning for Mobile Keyboard Prediction",
            "authors": [
                "Andrew Hard",
                "Kanishka Rao",
                "Rajiv Mathews",
                "Swaroop Ramaswamy",
                "Fran\u00e7oise Beaufays",
                "Sean Augenstein",
                "Hubert Eichner",
                "Chlo\u00e9 Kiddon",
                "Daniel Ramage"
            ],
            "abstract": "We train a recurrent neural network language model using a distributed,\non-device learning framework called federated learning for the purpose of\nnext-word prediction in a virtual keyboard for smartphones. Server-based\ntraining using stochastic gradient descent is compared with training on client\ndevices using the Federated Averaging algorithm. The federated algorithm, which\nenables training on a higher-quality dataset for this use case, is shown to\nachieve better prediction recall. This work demonstrates the feasibility and\nbenefit of training language models on client devices without exporting\nsensitive user data to servers. The federated learning environment gives users\ngreater control over the use of their data and simplifies the task of\nincorporating privacy by default with distributed training and aggregation\nacross a population of client devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03729v1",
            "date": "2018-11-09 01:18:27+00:00",
            "title": "Incorporating Relevant Knowledge in Context Modeling and Response Generation",
            "authors": [
                "Yanran Li",
                "Wenjie Li",
                "Ziqiang Cao",
                "Chengyao Chen"
            ],
            "abstract": "To sustain engaging conversation, it is critical for chatbots to make good\nuse of relevant knowledge. Equipped with a knowledge base, chatbots are able to\nextract conversation-related attributes and entities to facilitate context\nmodeling and response generation. In this work, we distinguish the uses of\nattribute and entity and incorporate them into the encoder-decoder architecture\nin different manners. Based on the augmented architecture, our chatbot, namely\nMike, is able to generate responses by referring to proper entities from the\ncollected knowledge. To validate the proposed approach, we build a movie\nconversation corpus on which the proposed approach significantly outperforms\nother four knowledge-grounded models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03754v2",
            "date": "2018-11-09 03:15:23+00:00",
            "title": "Neural sequence labeling for Vietnamese POS Tagging and NER",
            "authors": [
                "Duong Nguyen Anh",
                "Hieu Nguyen Kiem",
                "Vi Ngo Van"
            ],
            "abstract": "This paper presents a neural architecture for Vietnamese sequence labeling\ntasks including part-of-speech (POS) tagging and named entity recognition\n(NER). We applied the model described in \\cite{lample-EtAl:2016:N16-1} that is\na combination of bidirectional Long-Short Term Memory and Conditional Random\nFields, which rely on two sources of information about words: character-based\nword representations learned from the supervised corpus and pre-trained word\nembeddings learned from other unannotated corpora. Experiments on benchmark\ndatasets show that this work achieves state-of-the-art performances on both\ntasks - 93.52\\% accuracy for POS tagging and 94.88\\% F1 for NER. Our sourcecode\nis available at here.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03796v1",
            "date": "2018-11-09 07:06:46+00:00",
            "title": "Encoding Implicit Relation Requirements for Relation Extraction: A Joint Inference Approach",
            "authors": [
                "Liwei Chen",
                "Yansong Feng",
                "Songfang Huang",
                "Bingfeng Luo",
                "Dongyan Zhao"
            ],
            "abstract": "Relation extraction is the task of identifying predefined relationship\nbetween entities, and plays an essential role in information extraction,\nknowledge base construction, question answering and so on. Most existing\nrelation extractors make predictions for each entity pair locally and\nindividually, while ignoring implicit global clues available across different\nentity pairs and in the knowledge base, which often leads to conflicts among\nlocal predictions from different entity pairs. This paper proposes a joint\ninference framework that employs such global clues to resolve disagreements\namong local predictions. We exploit two kinds of clues to generate constraints\nwhich can capture the implicit type and cardinality requirements of a relation.\nThose constraints can be examined in either hard style or soft style, both of\nwhich can be effectively explored in an integer linear program formulation.\nExperimental results on both English and Chinese datasets show that our\nproposed framework can effectively utilize those two categories of global clues\nand resolve the disagreements among local predictions, thus improve various\nrelation extractors when such clues are applicable to the datasets. Our\nexperiments also indicate that the clues learnt automatically from existing\nknowledge bases perform comparably to or better than those refined by human.",
            "journal ref": "https://doi.org/10.1016/j.artint.2018.08.004",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03865v2",
            "date": "2018-11-09 11:30:11+00:00",
            "title": "Multimodal Grounding for Sequence-to-Sequence Speech Recognition",
            "authors": [
                "Ozan Caglayan",
                "Ramon Sanabria",
                "Shruti Palaskar",
                "Lo\u00efc Barrault",
                "Florian Metze"
            ],
            "abstract": "Humans are capable of processing speech by making use of multiple sensory\nmodalities. For example, the environment where a conversation takes place\ngenerally provides semantic and/or acoustic context that helps us to resolve\nambiguities or to recall named entities. Motivated by this, there have been\nmany works studying the integration of visual information into the speech\nrecognition pipeline. Specifically, in our previous work, we propose a\nmultistep visual adaptive training approach which improves the accuracy of an\naudio-based Automatic Speech Recognition (ASR) system. This approach, however,\nis not end-to-end as it requires fine-tuning the whole model with an adaptation\nlayer. In this paper, we propose novel end-to-end multimodal ASR systems and\ncompare them to the adaptive approach by using a range of visual\nrepresentations obtained from state-of-the-art convolutional neural networks.\nWe show that adaptive training is effective for S2S models leading to an\nabsolute improvement of 1.4% in word error rate. As for the end-to-end systems,\nalthough they perform better than baseline, the improvements are slightly less\nthan adaptive training, 0.8 absolute WER reduction in single-best models. Using\nensemble decoding, end-to-end models reach a WER of 15% which is the lowest\nscore among all systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03873v1",
            "date": "2018-11-09 12:11:22+00:00",
            "title": "Long Short-Term Memory with Dynamic Skip Connections",
            "authors": [
                "Tao Gui",
                "Qi Zhang",
                "Lujun Zhao",
                "Yaosong Lin",
                "Minlong Peng",
                "Jingjing Gong",
                "Xuanjing Huang"
            ],
            "abstract": "In recent years, long short-term memory (LSTM) has been successfully used to\nmodel sequential data of variable length. However, LSTM can still experience\ndifficulty in capturing long-term dependencies. In this work, we tried to\nalleviate this problem by introducing a dynamic skip connection, which can\nlearn to directly connect two dependent words. Since there is no dependency\ninformation in the training data, we propose a novel reinforcement\nlearning-based method to model the dependency relationship and connect\ndependent words. The proposed model computes the recurrent transition functions\nbased on the skip connections, which provides a dynamic skipping advantage over\nRNNs that always tackle entire sentences sequentially. Our experimental results\non three natural language processing tasks demonstrate that the proposed method\ncan achieve better performance than existing methods. In the number prediction\nexperiment, the proposed model outperformed LSTM with respect to accuracy by\nnearly 20%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04154v1",
            "date": "2018-11-09 22:50:50+00:00",
            "title": "Zero-shot Neural Transfer for Cross-lingual Entity Linking",
            "authors": [
                "Shruti Rijhwani",
                "Jiateng Xie",
                "Graham Neubig",
                "Jaime Carbonell"
            ],
            "abstract": "Cross-lingual entity linking maps an entity mention in a source language to\nits corresponding entry in a structured knowledge base that is in a different\n(target) language. While previous work relies heavily on bilingual lexical\nresources to bridge the gap between the source and the target languages, these\nresources are scarce or unavailable for many low-resource languages. To address\nthis problem, we investigate zero-shot cross-lingual entity linking, in which\nwe assume no bilingual lexical resources are available in the source\nlow-resource language. Specifically, we propose pivot-based entity linking,\nwhich leverages information from a high-resource \"pivot\" language to train\ncharacter-level neural entity linking models that are transferred to the source\nlow-resource language in a zero-shot manner. With experiments on 9 low-resource\nlanguages and transfer through a total of 54 languages, we show that our\nproposed pivot-based framework improves entity linking accuracy 17% (absolute)\non average over the baseline systems, for the zero-shot scenario. Further, we\nalso investigate the use of language-universal phonological representations\nwhich improves average accuracy (absolute) by 36% when transferring between\nlanguages that use different scripts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04164v1",
            "date": "2018-11-10 00:12:56+00:00",
            "title": "Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems",
            "authors": [
                "Van-Khanh Tran",
                "Le-Minh Nguyen"
            ],
            "abstract": "Recent deep learning models have shown improving results to natural language\ngeneration (NLG) irrespective of providing sufficient annotated data. However,\na modest training data may harm such models performance. Thus, how to build a\ngenerator that can utilize as much of knowledge from a low-resource setting\ndata is a crucial issue in NLG. This paper presents a variational neural-based\ngeneration model to tackle the NLG problem of having limited labeled dataset,\nin which we integrate a variational inference into an encoder-decoder generator\nand introduce a novel auxiliary autoencoding with an effective training\nprocedure. Experiments showed that the proposed methods not only outperform the\nprevious models when having sufficient training dataset but also show strong\nability to work acceptably well when the training data is scarce.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04231v2",
            "date": "2018-11-10 10:41:27+00:00",
            "title": "Speech Intention Understanding in a Head-final Language: A Disambiguation Utilizing Intonation-dependency",
            "authors": [
                "Won Ik Cho",
                "Hyeon Seung Lee",
                "Ji Won Yoon",
                "Seok Min Kim",
                "Nam Soo Kim"
            ],
            "abstract": "For a large portion of real-life utterances, the intention cannot be solely\ndecided by either their semantic or syntactic characteristics. Although not all\nthe sociolinguistic and pragmatic information can be digitized, at least\nphonetic features are indispensable in understanding the spoken language.\nEspecially in head-final languages such as Korean, sentence-final prosody has\ngreat importance in identifying the speaker's intention. This paper suggests a\nsystem which identifies the inherent intention of a spoken utterance given its\ntranscript, in some cases using auxiliary acoustic features. The main point\nhere is a separate distinction for cases where discrimination of intention\nrequires an acoustic cue. Thus, the proposed classification system decides\nwhether the given utterance is a fragment, statement, question, command, or a\nrhetorical question/command, utilizing the intonation-dependency coming from\nthe head-finality. Based on an intuitive understanding of the Korean language\nthat is engaged in the data annotation, we construct a network which identifies\nthe intention of a speech, and validate its utility with the test sentences.\nThe system, if combined with up-to-date speech recognizers, is expected to be\nflexibly inserted into various language understanding modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04284v2",
            "date": "2018-11-10 17:07:44+00:00",
            "title": "Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling",
            "authors": [
                "Hainan Xu",
                "Shuoyang Ding",
                "Shinji Watanabe"
            ],
            "abstract": "Most end-to-end speech recognition systems model text directly as a sequence\nof characters or sub-words. Current approaches to sub-word extraction only\nconsider character sequence frequencies, which at times produce inferior\nsub-word segmentation that might lead to erroneous speech recognition output.\nWe propose pronunciation-assisted sub-word modeling (PASM), a sub-word\nextraction method that leverages the pronunciation information of a word.\nExperiments show that the proposed method can greatly improve upon the\ncharacter-based baseline, and also outperform commonly used byte-pair encoding\nmethods.",
            "journal ref": "ICASSP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04354v1",
            "date": "2018-11-11 05:29:17+00:00",
            "title": "Multi-labeled Relation Extraction with Attentive Capsule Network",
            "authors": [
                "Xinsong Zhang",
                "Pengshuai Li",
                "Weijia Jia",
                "Hai Zhao"
            ],
            "abstract": "To disclose overlapped multiple relations from a sentence still keeps\nchallenging. Most current works in terms of neural models inconveniently\nassuming that each sentence is explicitly mapped to a relation label, cannot\nhandle multiple relations properly as the overlapped features of the relations\nare either ignored or very difficult to identify. To tackle with the new issue,\nwe propose a novel approach for multi-labeled relation extraction with capsule\nnetwork which acts considerably better than current convolutional or recurrent\nnet in identifying the highly overlapped relations within an individual\nsentence. To better cluster the features and precisely extract the relations,\nwe further devise attention-based routing algorithm and sliding-margin loss\nfunction, and embed them into our capsule network. The experimental results\nshow that the proposed approach can indeed extract the highly overlapped\nfeatures and achieve significant performance improvement for relation\nextraction comparing to the state-of-the-art works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04498v1",
            "date": "2018-11-11 22:37:38+00:00",
            "title": "Product Title Refinement via Multi-Modal Generative Adversarial Learning",
            "authors": [
                "Jianguo Zhang",
                "Pengcheng Zou",
                "Zhao Li",
                "Yao Wan",
                "Ye Liu",
                "Xiuming Pan",
                "Yu Gong",
                "Philip S. Yu"
            ],
            "abstract": "Nowadays, an increasing number of customers are in favor of using E-commerce\nApps to browse and purchase products. Since merchants are usually inclined to\nemploy redundant and over-informative product titles to attract customers'\nattention, it is of great importance to concisely display short product titles\non limited screen of cell phones. Previous researchers mainly consider textual\ninformation of long product titles and lack of human-like view during training\nand evaluation procedure. In this paper, we propose a Multi-Modal Generative\nAdversarial Network (MM-GAN) for short product title generation, which\ninnovatively incorporates image information, attribute tags from the product\nand the textual information from original long titles. MM-GAN treats short\ntitles generation as a reinforcement learning process, where the generated\ntitles are evaluated by the discriminator in a human-like view.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04531v1",
            "date": "2018-11-12 02:55:55+00:00",
            "title": "Sequence-Level Knowledge Distillation for Model Compression of Attention-based Sequence-to-Sequence Speech Recognition",
            "authors": [
                "Raden Mu'az Mun'im",
                "Nakamasa Inoue",
                "Koichi Shinoda"
            ],
            "abstract": "We investigate the feasibility of sequence-level knowledge distillation of\nSequence-to-Sequence (Seq2Seq) models for Large Vocabulary Continuous Speech\nRecognition (LVSCR). We first use a pre-trained larger teacher model to\ngenerate multiple hypotheses per utterance with beam search. With the same\ninput, we then train the student model using these hypotheses generated from\nthe teacher as pseudo labels in place of the original ground truth labels. We\nevaluate our proposed method using Wall Street Journal (WSJ) corpus. It\nachieved up to $ 9.8 \\times$ parameter reduction with accuracy loss of up to\n7.0\\% word-error rate (WER) increase",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04577v2",
            "date": "2018-11-12 06:37:20+00:00",
            "title": "Forecasting People's Needs in Hurricane Events from Social Network",
            "authors": [
                "Long Nguyen",
                "Zhou Yang",
                "Jia Li",
                "Guofeng Cao",
                "Fang Jin"
            ],
            "abstract": "Social networks can serve as a valuable communication channel for calls for\nhelp, offering assistance, and coordinating rescue activities in disaster.\nSocial networks such as Twitter allow users to continuously update relevant\ninformation, which is especially useful during a crisis, where the rapidly\nchanging conditions make it crucial to be able to access accurate information\npromptly. Social media helps those directly affected to inform others of\nconditions on the ground in real time and thus enables rescue workers to\ncoordinate their efforts more effectively, better meeting the survivors' need.\nThis paper presents a new sequence to sequence based framework for forecasting\npeople's needs during disasters using social media and weather data. It\nconsists of two Long Short-Term Memory (LSTM) models, one of which encodes\ninput sequences of weather information and the other plays as a conditional\ndecoder that decodes the encoded vector and forecasts the survivors' needs.\nCase studies utilizing data collected during Hurricane Sandy in 2012, Hurricane\nHarvey and Hurricane Irma in 2017 were analyzed and the results compared with\nthose obtained using a statistical language model n-gram and an LSTM generative\nmodel. Our proposed sequence to sequence method forecast people's needs more\nsuccessfully than either of the other models. This new approach shows great\npromise for enhancing disaster management activities such as evacuation\nplanning and commodity flow management.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04599v3",
            "date": "2018-11-12 08:26:57+00:00",
            "title": "The Cinderella Complex: Word Embeddings Reveal Gender Stereotypes in Movies and Books",
            "authors": [
                "Huimin Xu",
                "Zhang Zhang",
                "Lingfei Wu",
                "Cheng-Jun Wang"
            ],
            "abstract": "Our analysis of thousands of movies and books reveals how these cultural\nproducts weave stereotypical gender roles into morality tales and perpetuate\ngender inequality through storytelling. Using the word embedding techniques, we\nreveal the constructed emotional dependency of female characters on male\ncharacters in stories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04604v1",
            "date": "2018-11-12 08:45:25+00:00",
            "title": "Learning Personalized End-to-End Goal-Oriented Dialog",
            "authors": [
                "Liangchen Luo",
                "Wenhao Huang",
                "Qi Zeng",
                "Zaiqing Nie",
                "Xu Sun"
            ],
            "abstract": "Most existing works on dialog systems only consider conversation content\nwhile neglecting the personality of the user the bot is interacting with, which\nbegets several unsolved issues. In this paper, we present a personalized\nend-to-end model in an attempt to leverage personalization in goal-oriented\ndialogs. We first introduce a Profile Model which encodes user profiles into\ndistributed embeddings and refers to conversation history from other similar\nusers. Then a Preference Model captures user preferences over knowledge base\nentities to handle the ambiguity in user requests. The two models are combined\ninto the Personalized MemN2N. Experiments show that the proposed model achieves\nqualitative performance improvements over state-of-the-art methods. As for\nhuman evaluation, it also outperforms other approaches in terms of task\ncompletion rate and user satisfaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04623v2",
            "date": "2018-11-12 09:43:24+00:00",
            "title": "Fine-tuning of Language Models with Discriminator",
            "authors": [
                "Vadim Popov",
                "Mikhail Kudinov"
            ],
            "abstract": "Cross-entropy loss is a common choice when it comes to multiclass\nclassification tasks and language modeling in particular. Minimizing this loss\nresults in language models of very good quality. We show that it is possible to\nfine-tune these models and make them perform even better if they are fine-tuned\nwith sum of cross-entropy loss and reverse Kullback-Leibler divergence. The\nlatter is estimated using discriminator network that we train in advance.\nDuring fine-tuning probabilities of rare words that are usually underestimated\nby language models become bigger. The novel approach that we propose allows us\nto reach state-of-the-art quality on Penn Treebank: perplexity decreases from\n52.4 to 52.1. Our fine-tuning algorithm is rather fast, scales well to\ndifferent architectures and datasets and requires almost no hyperparameter\ntuning: the only hyperparameter that needs to be tuned is learning rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04670v1",
            "date": "2018-11-12 11:40:09+00:00",
            "title": "A Deep Ensemble Framework for Fake News Detection and Classification",
            "authors": [
                "Arjun Roy",
                "Kingshuk Basak",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Fake news, rumor, incorrect information, and misinformation detection are\nnowadays crucial issues as these might have serious consequences for our social\nfabrics. The rate of such information is increasing rapidly due to the\navailability of enormous web information sources including social media feeds,\nnews blogs, online newspapers etc.\n  In this paper, we develop various deep learning models for detecting fake\nnews and classifying them into the pre-defined fine-grained categories.\n  At first, we develop models based on Convolutional Neural Network (CNN) and\nBi-directional Long Short Term Memory (Bi-LSTM) networks. The representations\nobtained from these two models are fed into a Multi-layer Perceptron Model\n(MLP) for the final classification. Our experiments on a benchmark dataset show\npromising results with an overall accuracy of 44.87\\%, which outperforms the\ncurrent state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04695v1",
            "date": "2018-11-12 12:49:42+00:00",
            "title": "Classifying Patent Applications with Ensemble Methods",
            "authors": [
                "Fernando Benites",
                "Shervin Malmasi",
                "Marcos Zampieri"
            ],
            "abstract": "We present methods for the automatic classification of patent applications\nusing an annotated dataset provided by the organizers of the ALTA 2018 shared\ntask - Classifying Patent Applications. The goal of the task is to use\ncomputational methods to categorize patent applications according to a\ncoarse-grained taxonomy of eight classes based on the International Patent\nClassification (IPC). We tested a variety of approaches for this task and the\nbest results, 0.778 micro-averaged F1-Score, were achieved by SVM ensembles\nusing a combination of words and characters as features. Our team, BMZ, was\nranked first among 14 teams in the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04697v1",
            "date": "2018-11-12 12:52:03+00:00",
            "title": "CUNI System for the WMT18 Multimodal Translation Task",
            "authors": [
                "Jind\u0159ich Helcl",
                "Jind\u0159ich Libovick\u00fd",
                "Du\u0161an Vari\u0161"
            ],
            "abstract": "We present our submission to the WMT18 Multimodal Translation Task. The main\nfeature of our submission is applying a self-attentive network instead of a\nrecurrent neural network. We evaluate two methods of incorporating the visual\nfeatures in the model: first, we include the image representation as another\ninput to the network; second, we train the model to predict the visual features\nand use it as an auxiliary objective. For our submission, we acquired both\ntextual and multimodal additional data. Both of the proposed methods yield\nsignificant improvements over recurrent networks and self-attentive textual\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04708v1",
            "date": "2018-11-12 13:10:25+00:00",
            "title": "Analyzing deep CNN-based utterance embeddings for acoustic model adaptation",
            "authors": [
                "Joanna Rownicka",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "We explore why deep convolutional neural networks (CNNs) with small\ntwo-dimensional kernels, primarily used for modeling spatial relations in\nimages, are also effective in speech recognition. We analyze the\nrepresentations learned by deep CNNs and compare them with deep neural network\n(DNN) representations and i-vectors, in the context of acoustic model\nadaptation. To explore whether interpretable information can be decoded from\nthe learned representations we evaluate their ability to discriminate between\nspeakers, acoustic conditions, noise type, and gender using the Aurora-4\ndataset. We extract both whole model embeddings (to capture the information\nlearned across the whole network) and layer-specific embeddings which enable\nunderstanding of the flow of information across the network. We also use\nlearned representations as the additional input for a time-delay neural network\n(TDNN) for the Aurora-4 and MGB-3 English datasets. We find that deep CNN\nembeddings outperform DNN embeddings for acoustic model adaptation and\nauxiliary features based on deep CNN embeddings result in similar word error\nrates to i-vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04716v1",
            "date": "2018-11-12 13:33:35+00:00",
            "title": "Input Combination Strategies for Multi-Source Transformer Decoder",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Jind\u0159ich Helcl",
                "David Mare\u010dek"
            ],
            "abstract": "In multi-source sequence-to-sequence tasks, the attention mechanism can be\nmodeled in several ways. This topic has been thoroughly studied on recurrent\narchitectures. In this paper, we extend the previous work to the\nencoder-decoder attention in the Transformer architecture. We propose four\ndifferent input combination strategies for the encoder-decoder attention:\nserial, parallel, flat, and hierarchical. We evaluate our methods on tasks of\nmultimodal translation and translation with multiple source languages. The\nexperiments show that the models are able to use multiple sources and improve\nover single source baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04719v1",
            "date": "2018-11-12 13:40:04+00:00",
            "title": "End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Jind\u0159ich Helcl"
            ],
            "abstract": "Autoregressive decoding is the only part of sequence-to-sequence models that\nprevents them from massive parallelization at inference time.\nNon-autoregressive models enable the decoder to generate all output symbols\nindependently in parallel. We present a novel non-autoregressive architecture\nbased on connectionist temporal classification and evaluate it on the task of\nneural machine translation. Unlike other non-autoregressive methods which\noperate in several steps, our model can be trained end-to-end. We conduct\nexperiments on the WMT English-Romanian and English-German datasets. Our models\nachieve a significant speedup over the autoregressive models, keeping the\ntranslation quality comparable to other non-autoregressive models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04773v1",
            "date": "2018-11-12 15:16:12+00:00",
            "title": "Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?",
            "authors": [
                "Emma Strubell",
                "Andrew McCallum"
            ],
            "abstract": "Do unsupervised methods for learning rich, contextualized token\nrepresentations obviate the need for explicit modeling of linguistic structure\nin neural network models for semantic role labeling (SRL)? We address this\nquestion by incorporating the massively successful ELMo embeddings (Peters et\nal., 2018) into LISA (Strubell et al., 2018), a strong, linguistically-informed\nneural network architecture for SRL. In experiments on the CoNLL-2005 shared\ntask we find that though ELMo out-performs typical word embeddings, beginning\nto close the gap in F1 between LISA with predicted and gold syntactic parses,\nsyntactically-informed models still out-perform syntax-free models when both\nuse ELMo, especially on out-of-domain data. Our results suggest that linguistic\nstructures are indeed still relevant in this golden age of deep learning for\nNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04884v1",
            "date": "2018-11-12 18:03:45+00:00",
            "title": "CQASUMM: Building References for Community Question Answering Summarization Corpora",
            "authors": [
                "Tanya Chowdhury",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Community Question Answering forums such as Quora, Stackoverflow are rich\nknowledge resources, often catering to information on topics overlooked by\nmajor search engines. Answers submitted to these forums are often elaborated,\ncontain spam, are marred by slurs and business promotions. It is difficult for\na reader to go through numerous such answers to gauge community opinion. As a\nresult summarization becomes a prioritized task for CQA forums. While a number\nof efforts have been made to summarize factoid CQA, little work exists in\nsummarizing non-factoid CQA. We believe this is due to the lack of a\nconsiderably large, annotated dataset for CQA summarization. We create CQASUMM,\nthe first huge annotated CQA summarization dataset by filtering the 4.4 million\nYahoo! Answers L6 dataset. We sample threads where the best answer can double\nup as a reference summary and build hundred word summaries from them. We treat\nother answers as candidates documents for summarization. We provide a script to\ngenerate the dataset and introduce the new task of Community Question Answering\nSummarization. Multi document summarization has been widely studied with news\narticle datasets, especially in the DUC and TAC challenges using news corpora.\nHowever documents in CQA have higher variance, contradicting opinion and lesser\namount of overlap. We compare the popular multi document summarization\ntechniques and evaluate their performance on our CQA corpora. We look into the\nstate-of-the-art and understand the cases where existing multi document\nsummarizers (MDS) fail. We find that most MDS workflows are built for the\nentirely factual news corpora, whereas our corpus has a fair share of opinion\nbased instances too. We therefore introduce OpinioSumm, a new MDS which\noutperforms the best baseline by 4.6% w.r.t ROUGE-1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04897v1",
            "date": "2018-11-12 18:33:21+00:00",
            "title": "Multi-encoder multi-resolution framework for end-to-end speech recognition",
            "authors": [
                "Ruizhi Li",
                "Xiaofei Wang",
                "Sri Harish Mallidi",
                "Takaaki Hori",
                "Shinji Watanabe",
                "Hynek Hermansky"
            ],
            "abstract": "Attention-based methods and Connectionist Temporal Classification (CTC)\nnetwork have been promising research directions for end-to-end Automatic Speech\nRecognition (ASR). The joint CTC/Attention model has achieved great success by\nutilizing both architectures during multi-task training and joint decoding. In\nthis work, we present a novel Multi-Encoder Multi-Resolution (MEMR) framework\nbased on the joint CTC/Attention model. Two heterogeneous encoders with\ndifferent architectures, temporal resolutions and separate CTC networks work in\nparallel to extract complimentary acoustic information. A hierarchical\nattention mechanism is then used to combine the encoder-level information. To\ndemonstrate the effectiveness of the proposed model, experiments are conducted\non Wall Street Journal (WSJ) and CHiME-4, resulting in relative Word Error Rate\n(WER) reduction of 18.0-32.1%. Moreover, the proposed MEMR model achieves 3.6%\nWER in the WSJ eval92 test set, which is the best WER reported for an\nend-to-end system on this benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05021v1",
            "date": "2018-11-12 22:05:25+00:00",
            "title": "Improved Dynamic Memory Network for Dialogue Act Classification with Adversarial Training",
            "authors": [
                "Yao Wan",
                "Wenqiang Yan",
                "Jianwei Gao",
                "Zhou Zhao",
                "Jian Wu",
                "Philip S. Yu"
            ],
            "abstract": "Dialogue Act (DA) classification is a challenging problem in dialogue\ninterpretation, which aims to attach semantic labels to utterances and\ncharacterize the speaker's intention. Currently, many existing approaches\nformulate the DA classification problem ranging from multi-classification to\nstructured prediction, which suffer from two limitations: a) these methods are\neither handcrafted feature-based or have limited memories. b) adversarial\nexamples can't be correctly classified by traditional training methods. To\naddress these issues, in this paper we first cast the problem into a question\nand answering problem and proposed an improved dynamic memory networks with\nhierarchical pyramidal utterance encoder. Moreover, we apply adversarial\ntraining to train our proposed model. We evaluate our model on two public\ndatasets, i.e., Switchboard dialogue act corpus and the MapTask corpus.\nExtensive experiments show that our proposed model is not only robust, but also\nachieves better performance when compared with some state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05082v2",
            "date": "2018-11-13 03:04:41+00:00",
            "title": "A Unified Model for Opinion Target Extraction and Target Sentiment Prediction",
            "authors": [
                "Xin Li",
                "Lidong Bing",
                "Piji Li",
                "Wai Lam"
            ],
            "abstract": "Target-based sentiment analysis involves opinion target extraction and target\nsentiment classification. However, most of the existing works usually studied\none of these two sub-tasks alone, which hinders their practical use. This paper\naims to solve the complete task of target-based sentiment analysis in an\nend-to-end fashion, and presents a novel unified model which applies a unified\ntagging scheme. Our framework involves two stacked recurrent neural networks:\nThe upper one predicts the unified tags to produce the final output results of\nthe primary target-based sentiment analysis; The lower one performs an\nauxiliary target boundary prediction aiming at guiding the upper network to\nimprove the performance of the primary task. To explore the inter-task\ndependency, we propose to explicitly model the constrained transitions from\ntarget boundaries to target sentiment polarities. We also propose to maintain\nthe sentiment consistency within an opinion target via a gate mechanism which\nmodels the relation between the features for the current word and the previous\nword. We conduct extensive experiments on three benchmark datasets and our\nframework achieves consistently superior results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05085v2",
            "date": "2018-11-13 03:16:29+00:00",
            "title": "Domain Agnostic Real-Valued Specificity Prediction",
            "authors": [
                "Wei-Jen Ko",
                "Greg Durrett",
                "Junyi Jessy Li"
            ],
            "abstract": "Sentence specificity quantifies the level of detail in a sentence,\ncharacterizing the organization of information in discourse. While this\ninformation is useful for many downstream applications, specificity prediction\nsystems predict very coarse labels (binary or ternary) and are trained on and\ntailored toward specific domains (e.g., news). The goal of this work is to\ngeneralize specificity prediction to domains where no labeled data is available\nand output more nuanced real-valued specificity ratings.\n  We present an unsupervised domain adaptation system for sentence specificity\nprediction, specifically designed to output real-valued estimates from binary\ntraining labels. To calibrate the values of these predictions appropriately, we\nregularize the posterior distribution of the labels towards a reference\ndistribution. We show that our framework generalizes well to three different\ndomains with 50%~68% mean absolute error reduction than the current\nstate-of-the-art system trained for news sentence specificity. We also\ndemonstrate the potential of our work in improving the quality and\ninformativeness of dialogue generation systems.",
            "journal ref": "The AAAI Conference on Artificial Intelligence 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05145v1",
            "date": "2018-11-13 07:49:02+00:00",
            "title": "Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models",
            "authors": [
                "Satyajit Kamble",
                "Aditya Joshi"
            ],
            "abstract": "This paper reports an increment to the state-of-the-art in hate speech\ndetection for English-Hindi code-mixed tweets. We compare three typical deep\nlearning models using domain-specific embeddings. On experimenting with a\nbenchmark dataset of English-Hindi code-mixed tweets, we observe that using\ndomain-specific embeddings results in an improved representation of target\ngroups, and an improved F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05408v1",
            "date": "2018-11-13 17:01:50+00:00",
            "title": "Multi-task learning for Joint Language Understanding and Dialogue State Tracking",
            "authors": [
                "Abhinav Rastogi",
                "Raghav Gupta",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "This paper presents a novel approach for multi-task learning of language\nunderstanding (LU) and dialogue state tracking (DST) in task-oriented dialogue\nsystems. Multi-task training enables the sharing of the neural network layers\nresponsible for encoding the user utterance for both LU and DST and improves\nperformance while reducing the number of network parameters. In our proposed\nframework, DST operates on a set of candidate values for each slot that has\nbeen mentioned so far. These candidate sets are generated using LU slot\nannotations for the current user utterance, dialogue acts corresponding to the\npreceding system utterance and the dialogue state estimated for the previous\nturn, enabling DST to handle slots with a large or unbounded set of possible\nvalues and deal with slot values not seen during training. Furthermore, to\nbridge the gap between training and inference, we investigate the use of\nscheduled sampling on LU output for the current user utterance as well as the\nDST output for the preceding turn.",
            "journal ref": "Proceedings of the 19th Annual SIGdial Meeting on Discourse and\n  Dialogue 2018 (pp. 376-384)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05546v1",
            "date": "2018-11-13 22:08:39+00:00",
            "title": "Discourse in Multimedia: A Case Study in Information Extraction",
            "authors": [
                "Mrinmaya Sachan",
                "Kumar Avinava Dubey",
                "Eduard H. Hovy",
                "Tom M. Mitchell",
                "Dan Roth",
                "Eric P. Xing"
            ],
            "abstract": "To ensure readability, text is often written and presented with due\nformatting. These text formatting devices help the writer to effectively convey\nthe narrative. At the same time, these help the readers pick up the structure\nof the discourse and comprehend the conveyed information. There have been a\nnumber of linguistic theories on discourse structure of text. However, these\ntheories only consider unformatted text. Multimedia text contains rich\nformatting features which can be leveraged for various NLP tasks. In this\npaper, we study some of these discourse features in multimedia text and what\ncommunicative function they fulfil in the context. We examine how these\nmultimedia discourse features can be used to improve an information extraction\nsystem. We show that the discourse and text layout features provide information\nthat is complementary to lexical semantic information commonly used for\ninformation extraction. As a case study, we use these features to harvest\nstructured subject knowledge of geometry from textbooks. We show that the\nharvested structured knowledge can be used to improve an existing solver for\ngeometry problems, making it more accurate as well as more explainable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05553v1",
            "date": "2018-11-13 22:42:10+00:00",
            "title": "Corpus Phonetics Tutorial",
            "authors": [
                "Eleanor Chodroff"
            ],
            "abstract": "Corpus phonetics has become an increasingly popular method of research in\nlinguistic analysis. With advances in speech technology and computational\npower, large scale processing of speech data has become a viable technique.\nThis tutorial introduces the speech scientist and engineer to various automatic\nspeech processing tools. These include acoustic model creation and forced\nalignment using the Kaldi Automatic Speech Recognition Toolkit (Povey et al.,\n2011), forced alignment using FAVE-align (Rosenfelder et al., 2014), the\nMontreal Forced Aligner (McAuliffe et al., 2017), and the Penn Phonetics Lab\nForced Aligner (Yuan & Liberman, 2008), as well as stop consonant burst\nalignment using AutoVOT (Keshet et al., 2014). The tutorial provides a general\noverview of each program, step-by-step instructions for running the program, as\nwell as several tips and tricks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05569v1",
            "date": "2018-11-13 23:27:06+00:00",
            "title": "Cross-lingual Short-text Matching with Deep Learning",
            "authors": [
                "Asmelash Teka Hadgu"
            ],
            "abstract": "The problem of short text matching is formulated as follows: given a pair of\nsentences or questions, a matching model determines whether the input pair mean\nthe same or not. Models that can automatically identify questions with the same\nmeaning have a wide range of applications in question answering sites and\nmodern chatbots. In this article, we describe the approach by team hahu to\nsolve this problem in the context of the \"CIKM AnalytiCup 2018 - Cross-lingual\nShort-text Matching of Question Pairs\" that is sponsored by Alibaba. Our\nsolution is an end-to-end system based on current advances in deep learning\nwhich avoids heavy feature-engineering and achieves improved performance over\ntraditional machine-learning approaches. The log-loss scores for the first and\nsecond rounds of the contest are 0.35 and 0.39 respectively. The team was\nranked 7th from 1027 teams in the overall ranking scheme by the organizers that\nconsisted of the two contest scores as well as: innovation and system\nintegrity, understanding data as well as practicality of the solution for\nbusiness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05632v2",
            "date": "2018-11-14 04:18:00+00:00",
            "title": "Translating a Math Word Problem to an Expression Tree",
            "authors": [
                "Lei Wang",
                "Yan Wang",
                "Deng Cai",
                "Dongxiang Zhang",
                "Xiaojiang Liu"
            ],
            "abstract": "Sequence-to-sequence (SEQ2SEQ) models have been successfully applied to\nautomatic math word problem solving. Despite its simplicity, a drawback still\nremains: a math word problem can be correctly solved by more than one\nequations. This non-deterministic transduction harms the performance of maximum\nlikelihood estimation. In this paper, by considering the uniqueness of\nexpression tree, we propose an equation normalization method to normalize the\nduplicated equations. Moreover, we analyze the performance of three popular\nSEQ2SEQ models on the math word problem solving. We find that each model has\nits own specialty in solving problems, consequently an ensemble model is then\nproposed to combine their advantages. Experiments on dataset Math23K show that\nthe ensemble model with equation normalization significantly outperforms the\nprevious state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05683v1",
            "date": "2018-11-14 08:22:27+00:00",
            "title": "Modeling Coherence for Discourse Neural Machine Translation",
            "authors": [
                "Hao Xiong",
                "Zhongjun He",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Discourse coherence plays an important role in the translation of one text.\nHowever, the previous reported models most focus on improving performance over\nindividual sentence while ignoring cross-sentence links and dependencies, which\naffects the coherence of the text. In this paper, we propose to use discourse\ncontext and reward to refine the translation quality from the discourse\nperspective. In particular, we generate the translation of individual sentences\nat first. Next, we deliberate the preliminary produced translations, and train\nthe model to learn the policy that produces discourse coherent text by a reward\nteacher. Practical results on multiple discourse test datasets indicate that\nour model significantly improves the translation quality over the\nstate-of-the-art baseline system by +1.23 BLEU score. Moreover, our model\ngenerates more discourse coherent text and obtains +2.2 BLEU improvements when\nevaluated by discourse metrics.",
            "journal ref": "AAAI2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05689v1",
            "date": "2018-11-14 08:43:07+00:00",
            "title": "Leveraging Aspect Phrase Embeddings for Cross-Domain Review Rating Prediction",
            "authors": [
                "Aiqi Jiang",
                "Arkaitz Zubiaga"
            ],
            "abstract": "Online review platforms are a popular way for users to post reviews by\nexpressing their opinions towards a product or service, as well as they are\nvaluable for other users and companies to find out the overall opinions of\ncustomers. These reviews tend to be accompanied by a rating, where the star\nrating has become the most common approach for users to give their feedback in\na quantitative way, generally as a likert scale of 1-5 stars. In other social\nmedia platforms like Facebook or Twitter, an automated review rating prediction\nsystem can be useful to determine the rating that a user would have given to\nthe product or service. Existing work on review rating prediction focuses on\nspecific domains, such as restaurants or hotels. This, however, ignores the\nfact that some review domains which are less frequently rated, such as\ndentists, lack sufficient data to build a reliable prediction model. In this\npaper, we experiment on 12 datasets pertaining to 12 different review domains\nof varying level of popularity to assess the performance of predictions across\ndifferent domains. We introduce a model that leverages aspect phrase embeddings\nextracted from the reviews, which enables the development of both in-domain and\ncross-domain review rating prediction systems. Our experiments show that both\nof our review rating prediction systems outperform all other baselines. The\ncross-domain review rating prediction system is particularly significant for\nthe least popular review domains, where leveraging training data from other\ndomains leads to remarkable improvements in performance. The in-domain review\nrating prediction system is instead more suitable for popular review domains,\nprovided that a model built from training data pertaining to the target domain\nis more suitable when this data is abundant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05696v3",
            "date": "2018-11-14 09:20:46+00:00",
            "title": "Generating Multiple Diverse Responses for Short-Text Conversation",
            "authors": [
                "Jun Gao",
                "Wei Bi",
                "Xiaojiang Liu",
                "Junhui Li",
                "Shuming Shi"
            ],
            "abstract": "Neural generative models have become popular and achieved promising\nperformance on short-text conversation tasks. They are generally trained to\nbuild a 1-to-1 mapping from the input post to its output response. However, a\ngiven post is often associated with multiple replies simultaneously in real\napplications. Previous research on this task mainly focuses on improving the\nrelevance and informativeness of the top one generated response for each post.\nVery few works study generating multiple accurate and diverse responses for the\nsame post. In this paper, we propose a novel response generation model, which\nconsiders a set of responses jointly and generates multiple diverse responses\nsimultaneously. A reinforcement learning algorithm is designed to solve our\nmodel. Experiments on two short-text conversation tasks validate that the\nmultiple responses generated by our model obtain higher quality and larger\ndiversity compared with various state-of-the-art generative models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05701v3",
            "date": "2018-11-14 09:37:40+00:00",
            "title": "Plan-And-Write: Towards Better Automatic Storytelling",
            "authors": [
                "Lili Yao",
                "Nanyun Peng",
                "Ralph Weischedel",
                "Kevin Knight",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Automatic storytelling is challenging since it requires generating long,\ncoherent natural language to describes a sensible sequence of events. Despite\nconsiderable efforts on automatic story generation in the past, prior work\neither is restricted in plot planning, or can only generate stories in a narrow\ndomain. In this paper, we explore open-domain story generation that writes\nstories given a title (topic) as input. We propose a plan-and-write\nhierarchical generation framework that first plans a storyline, and then\ngenerates a story based on the storyline. We compare two planning strategies.\nThe dynamic schema interweaves story planning and its surface realization in\ntext, while the static schema plans out the entire storyline before generating\nstories. Experiments show that with explicit storyline planning, the generated\nstories are more diverse, coherent, and on topic than those generated without\ncreating a full plan, according to both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05721v1",
            "date": "2018-11-14 10:49:39+00:00",
            "title": "A Deterministic Algorithm for Bridging Anaphora Resolution",
            "authors": [
                "Yufang Hou"
            ],
            "abstract": "Previous work on bridging anaphora resolution (Poesio et al., 2004; Hou et\nal., 2013b) use syntactic preposition patterns to calculate word relatedness.\nHowever, such patterns only consider NPs' head nouns and hence do not fully\ncapture the semantics of NPs. Recently, Hou (2018) created word embeddings\n(embeddings_PP) to capture associative similarity (ie, relatedness) between\nnouns by exploring the syntactic structure of noun phrases. But embeddings_PP\nonly contains word representations for nouns. In this paper, we create new word\nvectors by combining embeddings_PP with GloVe. This new word embeddings\n(embeddings_bridging) are a more general lexical knowledge resource for\nbridging and allow us to represent the meaning of an NP beyond its head easily.\nWe therefore develop a deterministic approach for bridging anaphora resolution,\nwhich represents the semantics of an NP based on its head noun and\nmodifications. We show that this simple approach achieves the competitive\nresults compared to the best system in Hou et al.(2013b) which explores Markov\nLogic Networks to model the problem. Additionally, we further improve the\nresults for bridging anaphora resolution reported in Hou (2018) by combining\nour simple deterministic approach with Hou et al.(2013b)'s best system MLN II.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05740v1",
            "date": "2018-11-14 11:55:51+00:00",
            "title": "Neural Based Statement Classification for Biased Language",
            "authors": [
                "Christoph Hube",
                "Besnik Fetahu"
            ],
            "abstract": "Biased language commonly occurs around topics which are of controversial\nnature, thus, stirring disagreement between the different involved parties of a\ndiscussion. This is due to the fact that for language and its use,\nspecifically, the understanding and use of phrases, the stances are cohesive\nwithin the particular groups. However, such cohesiveness does not hold across\ngroups.\n  In collaborative environments or environments where impartial language is\ndesired (e.g. Wikipedia, news media), statements and the language therein\nshould represent equally the involved parties and be neutrally phrased. Biased\nlanguage is introduced through the presence of inflammatory words or phrases,\nor statements that may be incorrect or one-sided, thus violating such\nconsensus.\n  In this work, we focus on the specific case of phrasing bias, which may be\nintroduced through specific inflammatory words or phrases in a statement. For\nthis purpose, we propose an approach that relies on a recurrent neural networks\nin order to capture the inter-dependencies between words in a phrase that\nintroduced bias.\n  We perform a thorough experimental evaluation, where we show the advantages\nof a neural based approach over competitors that rely on word lexicons and\nother hand-crafted features in detecting biased language. We are able to\ndistinguish biased statements with a precision of P=0.92, thus significantly\noutperforming baseline models with an improvement of over 30%. Finally, we\nrelease the largest corpus of statements annotated for biased language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05827v1",
            "date": "2018-11-13 09:27:24+00:00",
            "title": "Jointly identifying opinion mining elements and fuzzy measurement of opinion intensity to analyze product features",
            "authors": [
                "Haiqing Zhang",
                "Aicha Sekhari",
                "Yacine Ouzrout",
                "Abdelaziz Bouras"
            ],
            "abstract": "Opinion mining mainly involves three elements: feature and feature-of\nrelations, opinion expressions and the related opinion attributes (e.g.\nPolarity), and feature-opinion relations. Although many works have emerged to\nachieve its aim of gaining information, the previous researches typically\nhandled each of the three elements in isolation, which cannot give sufficient\ninformation extraction results; hence, the complexity and the running time of\ninformation extraction is increased. In this paper, we propose an opinion\nmining extraction algorithm to jointly discover the main opinion mining\nelements. Specifically, the algorithm automatically builds kernels to combine\nclosely related words into new terms from word level to phrase level based on\ndependency relations; and we ensure the accuracy of opinion expressions and\npolarity based on: fuzzy measurements, opinion degree intensifiers, and opinion\npatterns. The 3458 analyzed reviews show that the proposed algorithm can\neffectively identify the main elements simultaneously and outperform the\nbaseline methods. The proposed algorithm is used to analyze the features among\nheterogeneous products in the same category. The feature-by-feature comparison\ncan help to select the weaker features and recommend the correct specifications\nfrom the beginning life of a product. From this comparison, some interesting\nobservations are revealed. For example, the negative polarity of video\ndimension is higher than the product usability dimension for a product. Yet,\nenhancing the dimension of product usability can more effectively improve the\nproduct (C) 2015 Elsevier Ltd. All rights reserved.",
            "journal ref": "Engineering Applications of Artificial Intelligence, Elsevier,\n  2016, 47, pp.122--139",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05889v1",
            "date": "2018-11-14 16:30:22+00:00",
            "title": "Dependency Grammar Induction with a Neural Variational Transition-based Parser",
            "authors": [
                "Bowen Li",
                "Jianpeng Cheng",
                "Yang Liu",
                "Frank Keller"
            ],
            "abstract": "Dependency grammar induction is the task of learning dependency syntax\nwithout annotated training data. Traditional graph-based models with global\ninference achieve state-of-the-art results on this task but they require\n$O(n^3)$ run time. Transition-based models enable faster inference with $O(n)$\ntime complexity, but their performance still lags behind. In this work, we\npropose a neural transition-based parser for dependency grammar induction,\nwhose inference procedure utilizes rich neural features with $O(n)$ time\ncomplexity. We train the parser with an integration of variational inference,\nposterior regularization and variance reduction techniques. The resulting\nframework outperforms previous unsupervised transition-based dependency parsers\nand achieves performance comparable to graph-based models, both on the English\nPenn Treebank and on the Universal Dependency Treebank. In an empirical\ncomparison, we show that our approach substantially increases parsing speed\nover graph-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05909v1",
            "date": "2018-11-14 17:06:33+00:00",
            "title": "The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task",
            "authors": [
                "Alberto Poncelas",
                "Andy Way",
                "Kepa Sarasola"
            ],
            "abstract": "In this paper we present the ADAPT system built for the Basque to English Low\nResource MT Evaluation Campaign. Basque is a low-resourced,\nmorphologically-rich language. This poses a challenge for Neural Machine\nTranslation models which usually achieve better performance when trained with\nlarge sets of data.\n  Accordingly, we used synthetic data to improve the translation quality\nproduced by a model built using only authentic data. Our proposal uses\nback-translated data to: (a) create new sentences, so the system can be trained\nwith more data; and (b) translate sentences that are close to the test set, so\nthe model can be fine-tuned to the document to be translated.",
            "journal ref": "Proceedings of the 15th International Workshop on Spoken Language\n  Translation (2018) 76-82",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06031v2",
            "date": "2018-11-14 19:42:03+00:00",
            "title": "A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks",
            "authors": [
                "Victor Sanh",
                "Thomas Wolf",
                "Sebastian Ruder"
            ],
            "abstract": "Much effort has been devoted to evaluate whether multi-task learning can be\nleveraged to learn rich representations that can be used in various Natural\nLanguage Processing (NLP) down-stream applications. However, there is still a\nlack of understanding of the settings in which multi-task learning has a\nsignificant effect. In this work, we introduce a hierarchical model trained in\na multi-task learning setup on a set of carefully selected semantic tasks. The\nmodel is trained in a hierarchical fashion to introduce an inductive bias by\nsupervising a set of low level tasks at the bottom layers of the model and more\ncomplex tasks at the top layers of the model. This model achieves\nstate-of-the-art results on a number of tasks, namely Named Entity Recognition,\nEntity Mention Detection and Relation Extraction without hand-engineered\nfeatures or external NLP tools like syntactic parsers. The hierarchical\ntraining supervision induces a set of shared semantic representations at lower\nlayers of the model. We show that as we move from the bottom to the top layers\nof the model, the hidden states of the layers tend to represent more complex\nsemantic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06278v2",
            "date": "2018-11-15 10:26:16+00:00",
            "title": "Survey of Computational Approaches to Lexical Semantic Change",
            "authors": [
                "Nina Tahmasebi",
                "Lars Borin",
                "Adam Jatowt"
            ],
            "abstract": "Our languages are in constant flux driven by external factors such as\ncultural, societal and technological changes, as well as by only partially\nunderstood internal motivations. Words acquire new meanings and lose old\nsenses, new words are coined or borrowed from other languages and obsolete\nwords slide into obscurity. Understanding the characteristics of shifts in the\nmeaning and in the use of words is useful for those who work with the content\nof historical texts, the interested general public, but also in and of itself.\nThe findings from automatic lexical semantic change detection, and the models\nof diachronic conceptual change are currently being incorporated in approaches\nfor measuring document across-time similarity, information retrieval from\nlong-term document archives, the design of OCR algorithms, and so on. In recent\nyears we have seen a surge in interest in the academic community in\ncomputational methods and tools supporting inquiry into diachronic conceptual\nchange and lexical replacement. This article is an extract of a survey of\nrecent computational techniques to tackle lexical semantic change currently\nunder review. In this article we focus on diachronic conceptual change as an\nextension of semantic change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06303v2",
            "date": "2018-11-15 11:46:14+00:00",
            "title": "End-to-End Learning for Answering Structured Queries Directly over Text",
            "authors": [
                "Paul Groth",
                "Antony Scerri",
                "Ron Daniel, Jr.",
                "Bradley P. Allen"
            ],
            "abstract": "Structured queries expressed in languages (such as SQL, SPARQL, or XQuery)\noffer a convenient and explicit way for users to express their information\nneeds for a number of tasks. In this work, we present an approach to answer\nthese directly over text data without storing results in a database. We\nspecifically look at the case of knowledge bases where queries are over\nentities and the relations between them. Our approach combines distributed\nquery answering (e.g. Triple Pattern Fragments) with models built for\nextractive question answering. Importantly, by applying distributed querying\nanswering we are able to simplify the model learning problem. We train models\nfor a large portion (572) of the relations within Wikidata and achieve an\naverage 0.70 F1 measure across all models. We also present a systematic method\nto construct the necessary training data for this task from knowledge graphs\nand describe a prototype implementation.",
            "journal ref": "Proceedings of the Workshop on Deep Learning for Knowledge Graphs\n  (DL4KG2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06621v1",
            "date": "2018-11-15 23:09:44+00:00",
            "title": "Streaming End-to-end Speech Recognition For Mobile Devices",
            "authors": [
                "Yanzhang He",
                "Tara N. Sainath",
                "Rohit Prabhavalkar",
                "Ian McGraw",
                "Raziel Alvarez",
                "Ding Zhao",
                "David Rybach",
                "Anjuli Kannan",
                "Yonghui Wu",
                "Ruoming Pang",
                "Qiao Liang",
                "Deepti Bhatia",
                "Yuan Shangguan",
                "Bo Li",
                "Golan Pundak",
                "Khe Chai Sim",
                "Tom Bagby",
                "Shuo-yiin Chang",
                "Kanishka Rao",
                "Alexander Gruenstein"
            ],
            "abstract": "End-to-end (E2E) models, which directly predict output character sequences\ngiven input speech, are good candidates for on-device speech recognition. E2E\nmodels, however, present numerous challenges: In order to be truly useful, such\nmodels must decode speech utterances in a streaming fashion, in real time; they\nmust be robust to the long tail of use cases; they must be able to leverage\nuser-specific context (e.g., contact lists); and above all, they must be\nextremely accurate. In this work, we describe our efforts at building an E2E\nspeech recognizer using a recurrent neural network transducer. In experimental\nevaluations, we find that the proposed approach can outperform a conventional\nCTC-based model in terms of both latency and accuracy in a number of evaluation\ncategories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07032v1",
            "date": "2018-11-16 21:08:50+00:00",
            "title": "Mining Entity Synonyms with Efficient Neural Set Generation",
            "authors": [
                "Jiaming Shen",
                "Ruiliang Lyu",
                "Xiang Ren",
                "Michelle Vanni",
                "Brian Sadler",
                "Jiawei Han"
            ],
            "abstract": "Mining entity synonym sets (i.e., sets of terms referring to the same entity)\nis an important task for many entity-leveraging applications. Previous work\neither rank terms based on their similarity to a given query term, or treats\nthe problem as a two-phase task (i.e., detecting synonymy pairs, followed by\norganizing these pairs into synonym sets). However, these approaches fail to\nmodel the holistic semantics of a set and suffer from the error propagation\nissue. Here we propose a new framework, named SynSetMine, that efficiently\ngenerates entity synonym sets from a given vocabulary, using example sets from\nexternal knowledge bases as distant supervision. SynSetMine consists of two\nnovel modules: (1) a set-instance classifier that jointly learns how to\nrepresent a permutation invariant synonym set and whether to include a new\ninstance (i.e., a term) into the set, and (2) a set generation algorithm that\nenumerates the vocabulary only once and applies the learned set-instance\nclassifier to detect all entity synonym sets in it. Experiments on three real\ndatasets from different domains demonstrate both effectiveness and efficiency\nof SynSetMine for mining entity synonym sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07061v1",
            "date": "2018-11-16 23:17:33+00:00",
            "title": "Using Sentiment Induction to Understand Variation in Gendered Online Communities",
            "authors": [
                "Li Lucy",
                "Julia Mendelsohn"
            ],
            "abstract": "We analyze gendered communities defined in three different ways: text, users,\nand sentiment. Differences across these representations reveal facets of\ncommunities' distinctive identities, such as social group, topic, and\nattitudes. Two communities may have high text similarity but not user\nsimilarity or vice versa, and word usage also does not vary according to a\nclearcut, binary perspective of gender. Community-specific sentiment lexicons\ndemonstrate that sentiment can be a useful indicator of words' social meaning\nand community values, especially in the context of discussion content and user\ndemographics. Our results show that social platforms such as Reddit are active\nsettings for different constructions of gender.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07066v2",
            "date": "2018-11-17 00:21:10+00:00",
            "title": "Detecting Incongruity Between News Headline and Body Text via a Deep Hierarchical Encoder",
            "authors": [
                "Seunghyun Yoon",
                "Kunwoo Park",
                "Joongbo Shin",
                "Hongjun Lim",
                "Seungpil Won",
                "Meeyoung Cha",
                "Kyomin Jung"
            ],
            "abstract": "Some news headlines mislead readers with overrated or false information, and\nidentifying them in advance will better assist readers in choosing proper news\nstories to consume. This research introduces million-scale pairs of news\nheadline and body text dataset with incongruity label, which can uniquely be\nutilized for detecting news stories with misleading headlines. On this dataset,\nwe develop two neural networks with hierarchical architectures that model a\ncomplex textual representation of news articles and measure the incongruity\nbetween the headline and the body text. We also present a data augmentation\nmethod that dramatically reduces the text input size a model handles by\nindependently investigating each paragraph of news stories, which further\nboosts the performance. Our experiments and qualitative evaluations demonstrate\nthat the proposed methods outperform existing approaches and efficiently detect\nnews stories with misleading headlines in the real world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07080v2",
            "date": "2018-11-17 02:36:07+00:00",
            "title": "Bilingual Dictionary Induction for Bantu Languages",
            "authors": [
                "Ndapa Nakashole"
            ],
            "abstract": "We present a method for learning bilingual translation dictionaries between\nEnglish and Bantu languages. We show that exploiting the grammatical structure\ncommon to Bantu languages enables bilingual dictionary induction for languages\nwhere training data is unavailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07092v2",
            "date": "2018-11-17 03:58:26+00:00",
            "title": "Unnamed Entity Recognition of Sense Mentions",
            "authors": [
                "Ndapa Nakashole"
            ],
            "abstract": "We consider the problem of recognizing mentions of human senses in text. Our\ncontribution is a method for acquiring labeled data, and a learning method that\nis trained on this data. Experiments show the effectiveness of our proposed\ndata labeling approach and our learning model on the task of sense recognition\nin text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07098v2",
            "date": "2018-11-17 04:24:14+00:00",
            "title": "Sense Perception Common Sense Relationships",
            "authors": [
                "Ndapa Nakashole"
            ],
            "abstract": "Often missing in existing knowledge bases of facts, are relationships that\nencode common sense knowledge about unnamed entities. In this paper, we propose\nto extract novel, common sense relationships pertaining to sense perception\nconcepts such as sound and smell.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07351v2",
            "date": "2018-11-18 16:55:25+00:00",
            "title": "Neural Multi-Task Learning for Citation Function and Provenance",
            "authors": [
                "Xuan Su",
                "Animesh Prasad",
                "Min-Yen Kan",
                "Kazunari Sugiyama"
            ],
            "abstract": "Citation function and provenance are two cornerstone tasks in citation\nanalysis. Given a citation, the former task determines its rhetorical role,\nwhile the latter locates the text in the cited paper that contains the relevant\ncited information. We hypothesize that these two tasks are synergistically\nrelated, and build a model that validates this claim. For both tasks, we show\nthat a single-layer convolutional neural network (CNN) outperforms existing\nstate-of-the-art baselines. More importantly, we show that the two tasks are\nindeed synergistic: by jointly training both of the tasks in a multi-task\nlearning setup, we demonstrate additional performance gains. Altogether, our\nmodels improve the current state-of-the-arts up to 2\\%, with statistical\nsignificance for both citation function and provenance prediction tasks.",
            "journal ref": "JCDL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07497v1",
            "date": "2018-11-19 04:42:54+00:00",
            "title": "A Comparative Analysis of Content-based Geolocation in Blogs and Tweets",
            "authors": [
                "Konstantinos Pappas",
                "Mahmoud Azab",
                "Rada Mihalcea"
            ],
            "abstract": "The geolocation of online information is an essential component in any\ngeospatial application. While most of the previous work on geolocation has\nfocused on Twitter, in this paper we quantify and compare the performance of\ntext-based geolocation methods on social media data drawn from both Blogger and\nTwitter. We introduce a novel set of location specific features that are both\nhighly informative and easily interpretable, and show that we can achieve error\nrate reductions of up to 12.5% with respect to the best previously proposed\ngeolocation features. We also show that despite posting longer text, Blogger\nusers are significantly harder to geolocate than Twitter users. Additionally,\nwe investigate the effect of training and testing on different media\n(cross-media predictions), or combining multiple social media sources\n(multi-media predictions). Finally, we explore the geolocability of social\nmedia in relation to three user dimensions: state, gender, and industry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07631v1",
            "date": "2018-11-19 11:54:25+00:00",
            "title": "Chat More If You Like: Dynamic Cue Words Planning to Flow Longer Conversations",
            "authors": [
                "Lili Yao",
                "Ruijian Xu",
                "Chao Li",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "To build an open-domain multi-turn conversation system is one of the most\ninteresting and challenging tasks in Artificial Intelligence. Many research\nefforts have been dedicated to building such dialogue systems, yet few shed\nlight on modeling the conversation flow in an ongoing dialogue. Besides, it is\ncommon for people to talk about highly relevant aspects during a conversation.\nAnd the topics are coherent and drift naturally, which demonstrates the\nnecessity of dialogue flow modeling. To this end, we present the multi-turn\ncue-words driven conversation system with reinforcement learning method (RLCw),\nwhich strives to select an adaptive cue word with the greatest future credit,\nand therefore improve the quality of generated responses. We introduce a new\nreward to measure the quality of cue words in terms of effectiveness and\nrelevance. To further optimize the model for long-term conversations, a\nreinforcement approach is adopted in this paper. Experiments on real-life\ndataset demonstrate that our model consistently outperforms a set of\ncompetitive baselines in terms of simulated turns, diversity and human\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07720v1",
            "date": "2018-11-16 04:40:03+00:00",
            "title": "Beam Search Decoding using Manner of Articulation Detection Knowledge Derived from Connectionist Temporal Classification",
            "authors": [
                "Pradeep Rangan",
                "Sreenivasa Rao K"
            ],
            "abstract": "Manner of articulation detection using deep neural networks require a priori\nknowledge of the attribute discriminative features or the decent phoneme\nalignments. However generating an appropriate phoneme alignment is complex and\nits performance depends on the choice of optimal number of senones, Gaussians,\netc. In the first part of our work, we exploit the manner of articulation\ndetection using connectionist temporal classification (CTC) which doesn't need\nany phoneme alignment. Later we modify the state-of-the-art character based\nposteriors generated by CTC using the manner of articulation CTC detector. Beam\nsearch decoding is performed on the modified posteriors and it's impact on open\nsource datasets such as AN4 and LibriSpeech is observed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07851v3",
            "date": "2018-11-19 18:11:09+00:00",
            "title": "The Mafiascum Dataset: A Large Text Corpus for Deception Detection",
            "authors": [
                "Bob de Ruiter",
                "George Kachergis"
            ],
            "abstract": "Detecting deception in natural language has a wide variety of applications,\nbut because of its hidden nature there are currently no public, large-scale\nsources of labeled deceptive text. This work introduces the Mafiascum dataset\n[1], a collection of over 700 games of Mafia, in which players are randomly\nassigned either deceptive or non-deceptive roles and then interact via forum\npostings. Over 9000 documents were compiled from the dataset, which each\ncontained all messages written by a single player in a single game. This corpus\nwas used to construct a set of hand-picked linguistic features based on prior\ndeception research, as well as a set of average word vectors enriched with\nsubword information. A logistic regression classifier fit on a combination of\nthese feature sets achieved an average precision of 0.39 (chance = 0.26) and an\nAUROC of 0.68 on 5000+ word documents. On 50+ word documents, an average\nprecision of 0.29 (chance = 0.23) and an AUROC of 0.59 was achieved.\n  [1] https://bitbucket.org/bopjesvla/thesis/src",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08040v3",
            "date": "2018-11-20 01:08:09+00:00",
            "title": "Unsupervised Pseudo-Labeling for Extractive Summarization on Electronic Health Records",
            "authors": [
                "Xiangan Liu",
                "Keyang Xu",
                "Pengtao Xie",
                "Eric Xing"
            ],
            "abstract": "Extractive summarization is very useful for physicians to better manage and\ndigest Electronic Health Records (EHRs). However, the training of a supervised\nmodel requires disease-specific medical background and is thus very expensive.\nWe studied how to utilize the intrinsic correlation between multiple EHRs to\ngenerate pseudo-labels and train a supervised model with no external\nannotation. Experiments on real-patient data validate that our model is\neffective in summarizing crucial disease-specific information for patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08048v1",
            "date": "2018-11-20 02:59:30+00:00",
            "title": "QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships",
            "authors": [
                "Oyvind Tafjord",
                "Peter Clark",
                "Matt Gardner",
                "Wen-tau Yih",
                "Ashish Sabharwal"
            ],
            "abstract": "Many natural language questions require recognizing and reasoning with\nqualitative relationships (e.g., in science, economics, and medicine), but are\nchallenging to answer with corpus-based methods. Qualitative modeling provides\ntools that support such reasoning, but the semantic parsing task of mapping\nquestions into those models has formidable challenges. We present QuaRel, a\ndataset of diverse story questions involving qualitative relationships that\ncharacterize these challenges, and techniques that begin to address them. The\ndataset has 2771 questions relating 19 different types of quantities. For\nexample, \"Jenny observes that the robot vacuum cleaner moves slower on the\nliving room carpet than on the bedroom carpet. Which carpet has more friction?\"\nWe contribute (1) a simple and flexible conceptual framework for representing\nthese kinds of questions; (2) the QuaRel dataset, including logical forms,\nexemplifying the parsing challenges; and (3) two novel models for this task,\nbuilt as extensions of type-constrained semantic parsing. The first of these\nmodels (called QuaSP+) significantly outperforms off-the-shelf tools on QuaRel.\nThe second (QuaSP+Zero) demonstrates zero-shot capability, i.e., the ability to\nhandle new qualitative relationships without requiring additional training\ndata, something not possible with previous models. This work thus makes inroads\ninto answering complex, qualitative questions that require reasoning, and\nscaling to new relationships at low cost. The dataset and models are available\nat http://data.allenai.org/quarel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08078v1",
            "date": "2018-11-20 05:12:31+00:00",
            "title": "An empirical evaluation of AMR parsing for legal documents",
            "authors": [
                "Sinh Vu Trong",
                "Minh Nguyen Le"
            ],
            "abstract": "Many approaches have been proposed to tackle the problem of Abstract Meaning\nRepresentation (AMR) parsing, helps solving various natural language processing\nissues recently. In our paper, we provide an overview of different methods in\nAMR parsing and their performances when analyzing legal documents. We conduct\nexperiments of different AMR parsers on our annotated dataset extracted from\nthe English version of Japanese Civil Code. Our results show the limitations as\nwell as open a room for improvements of current parsing techniques when\napplying in this complicated domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08100v2",
            "date": "2018-11-20 07:15:31+00:00",
            "title": "Another Diversity-Promoting Objective Function for Neural Dialogue Generation",
            "authors": [
                "Ryo Nakamura",
                "Katsuhito Sudoh",
                "Koichiro Yoshino",
                "Satoshi Nakamura"
            ],
            "abstract": "Although generation-based dialogue systems have been widely researched, the\nresponse generations by most existing systems have very low diversities. The\nmost likely reason for this problem is Maximum Likelihood Estimation (MLE) with\nSoftmax Cross-Entropy (SCE) loss. MLE trains models to generate the most\nfrequent responses from enormous generation candidates, although in actual\ndialogues there are various responses based on the context. In this paper, we\npropose a new objective function called Inverse Token Frequency (ITF) loss,\nwhich individually scales smaller loss for frequent token classes and larger\nloss for rare token classes. This function encourages the model to generate\nrare tokens rather than frequent tokens. It does not complicate the model and\nits training is stable because we only replace the objective function. On the\nOpenSubtitles dialogue dataset, our loss model establishes a state-of-the-art\nDIST-1 of 7.56, which is the unigram diversity score, while maintaining a good\nBLEU-1 score. On a Japanese Twitter replies dataset, our loss model achieves a\nDIST-1 score comparable to the ground truth.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08600v1",
            "date": "2018-11-21 05:14:37+00:00",
            "title": "Contextualized Non-local Neural Networks for Sequence Learning",
            "authors": [
                "Pengfei Liu",
                "Shuaichen Chang",
                "Xuanjing Huang",
                "Jian Tang",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Recently, a large number of neural mechanisms and models have been proposed\nfor sequence learning, of which self-attention, as exemplified by the\nTransformer model, and graph neural networks (GNNs) have attracted much\nattention. In this paper, we propose an approach that combines and draws on the\ncomplementary strengths of these two methods. Specifically, we propose\ncontextualized non-local neural networks (CN$^{\\textbf{3}}$), which can both\ndynamically construct a task-specific structure of a sentence and leverage rich\nlocal dependencies within a particular neighborhood.\n  Experimental results on ten NLP tasks in text classification, semantic\nmatching, and sequence labeling show that our proposed model outperforms\ncompetitive baselines and discovers task-specific dependency structures, thus\nproviding better interpretability to users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08603v1",
            "date": "2018-11-21 06:00:23+00:00",
            "title": "Neural Collective Entity Linking",
            "authors": [
                "Yixin Cao",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu"
            ],
            "abstract": "Entity Linking aims to link entity mentions in texts to knowledge bases, and\nneural models have achieved recent success in this task. However, most existing\nmethods rely on local contexts to resolve entities independently, which may\nusually fail due to the data sparsity of local information. To address this\nissue, we propose a novel neural model for collective entity linking, named as\nNCEL. NCEL applies Graph Convolutional Network to integrate both local\ncontextual features and global coherence information for entity linking. To\nimprove the computation efficiency, we approximately perform graph convolution\non a subgraph of adjacent entity mentions instead of those in the entire text.\nWe further introduce an attention scheme to improve the robustness of NCEL to\ndata noise and train the model on Wikipedia hyperlinks to avoid overfitting and\ndomain bias. In experiments, we evaluate NCEL on five publicly available\ndatasets to verify the linking performance as well as generalization ability.\nWe also conduct an extensive analysis of time complexity, the impact of key\nmodules, and qualitative results, which demonstrate the effectiveness and\nefficiency of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08610v1",
            "date": "2018-11-21 06:42:47+00:00",
            "title": "Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions",
            "authors": [
                "Zhipeng Chen",
                "Yiming Cui",
                "Wentao Ma",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Machine Reading Comprehension (MRC) with multiple-choice questions requires\nthe machine to read given passage and select the correct answer among several\ncandidates. In this paper, we propose a novel approach called Convolutional\nSpatial Attention (CSA) model which can better handle the MRC with\nmultiple-choice questions. The proposed model could fully extract the mutual\ninformation among the passage, question, and the candidates, to form the\nenriched representations. Furthermore, to merge various attention results, we\npropose to use convolutional operation to dynamically summarize the attention\nvalues within the different size of regions. Experimental results show that the\nproposed model could give substantial improvements over various\nstate-of-the-art systems on both RACE and SemEval-2018 Task11 datasets.",
            "journal ref": "AAAI 2019 6276-6283",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08619v2",
            "date": "2018-11-21 07:55:16+00:00",
            "title": "Multi Task Deep Morphological Analyzer: Context Aware Joint Morphological Tagging and Lemma Prediction",
            "authors": [
                "Saurav Jha",
                "Akhilesh Sudhakar",
                "Anil Kumar Singh"
            ],
            "abstract": "The ambiguities introduced by the recombination of morphemes constructing\nseveral possible inflections for a word makes the prediction of syntactic\ntraits in Morphologically Rich Languages (MRLs) a notoriously complicated task.\nWe propose the Multi Task Deep Morphological analyzer (MT-DMA), a\ncharacter-level neural morphological analyzer based on multitask learning of\nword-level tag markers for Hindi and Urdu. MT-DMA predicts a set of six\nmorphological tags for words of Indo-Aryan languages: Parts-of-speech (POS),\nGender (G), Number (N), Person (P), Case (C), Tense-Aspect-Modality (TAM)\nmarker as well as the Lemma (L) by jointly learning all these in one trainable\nframework. We show the effectiveness of training of such deep neural networks\nby the simultaneous optimization of multiple loss functions and sharing of\ninitial parameters for context-aware morphological analysis. Exploiting\ncharacter-level features in phonological space optimized for each tag using\nmulti-objective genetic algorithm, our model establishes a new state-of-the-art\naccuracy score upon all seven of the tasks for both the languages. MT-DMA is\npublicly accessible: code, models and data are available at\nhttps://github.com/Saurav0074/morph_analyzer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08757v1",
            "date": "2018-11-21 14:36:30+00:00",
            "title": "The Best of Both Worlds: Lexical Resources To Improve Low-Resource Part-of-Speech Tagging",
            "authors": [
                "Barbara Plank",
                "Sigrid Klerke",
                "Zeljko Agic"
            ],
            "abstract": "In natural language processing, the deep learning revolution has shifted the\nfocus from conventional hand-crafted symbolic representations to dense inputs,\nwhich are adequate representations learned automatically from corpora. However,\nparticularly when working with low-resource languages, small amounts of\nsymbolic lexical resources such as user-generated lexicons are often available\neven when gold-standard corpora are not. Such additional linguistic information\nis though often neglected, and recent neural approaches to cross-lingual\ntagging typically rely only on word and subword embeddings. While these\nrepresentations are effective, our recent work has shown clear benefits of\ncombining the best of both worlds: integrating conventional lexical information\nimproves neural cross-lingual part-of-speech (PoS) tagging. However, little is\nknown on how complementary such additional information is, and to what extent\nimprovements depend on the coverage and quality of these external resources.\nThis paper seeks to fill this gap by providing the first thorough analysis on\nthe contributions of lexical resources for cross-lingual PoS tagging in neural\ntimes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08816v2",
            "date": "2018-11-21 16:36:08+00:00",
            "title": "Learning cross-lingual phonological and orthagraphic adaptations: a case study in improving neural machine translation between low-resource languages",
            "authors": [
                "Saurav Jha",
                "Akhilesh Sudhakar",
                "Anil Kumar Singh"
            ],
            "abstract": "Out-of-vocabulary (OOV) words can pose serious challenges for machine\ntranslation (MT) tasks, and in particular, for low-resource language (LRL)\npairs, i.e., language pairs for which few or no parallel corpora exist. Our\nwork adapts variants of seq2seq models to perform transduction of such words\nfrom Hindi to Bhojpuri (an LRL instance), learning from a set of cognate pairs\nbuilt from a bilingual dictionary of Hindi--Bhojpuri words. We demonstrate that\nour models can be effectively used for language pairs that have limited\nparallel corpora; our models work at the character level to grasp phonetic and\northographic similarities across multiple types of word adaptations, whether\nsynchronic or diachronic, loan words or cognates. We describe the training\naspects of several character level NMT systems that we adapted to this task and\ncharacterize their typical errors. Our method improves BLEU score by 6.3 on the\nHindi-to-Bhojpuri translation task. Further, we show that such transductions\ncan generalize well to other languages by applying it successfully to Hindi --\nBangla cognate pairs. Our work can be seen as an important step in the process\nof: (i) resolving the OOV words problem arising in MT tasks, (ii) creating\neffective parallel corpora for resource-constrained languages, and (iii)\nleveraging the enhanced semantic knowledge captured by word-level embeddings to\nperform character-level tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09242v1",
            "date": "2018-11-22 17:19:31+00:00",
            "title": "AutoSense Model for Word Sense Induction",
            "authors": [
                "Reinald Kim Amplayo",
                "Seung-won Hwang",
                "Min Song"
            ],
            "abstract": "Word sense induction (WSI), or the task of automatically discovering multiple\nsenses or meanings of a word, has three main challenges: domain adaptability,\nnovel sense detection, and sense granularity flexibility. While current latent\nvariable models are known to solve the first two challenges, they are not\nflexible to different word sense granularities, which differ very much among\nwords, from aardvark with one sense, to play with over 50 senses. Current\nmodels either require hyperparameter tuning or nonparametric induction of the\nnumber of senses, which we find both to be ineffective. Thus, we aim to\neliminate these requirements and solve the sense granularity problem by\nproposing AutoSense, a latent variable model based on two observations: (1)\nsenses are represented as a distribution over topics, and (2) senses generate\npairings between the target word and its neighboring word. These observations\nalleviate the problem by (a) throwing garbage senses and (b) additionally\ninducing fine-grained word senses. Results show great improvements over the\nstate-of-the-art models on popular WSI datasets. We also show that AutoSense is\nable to learn the appropriate sense granularity of a word. Finally, we apply\nAutoSense to the unsupervised author name disambiguation task where the sense\ngranularity problem is more evident and show that AutoSense is evidently better\nthan competing models. We share our data and code here:\nhttps://github.com/rktamplayo/AutoSense.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09353v2",
            "date": "2018-11-23 03:39:27+00:00",
            "title": "Learning to Discover, Ground and Use Words with Segmental Neural Language Models",
            "authors": [
                "Kazuya Kawakami",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "We propose a segmental neural language model that combines the generalization\npower of neural networks with the ability to discover word-like units that are\nlatent in unsegmented character sequences. In contrast to previous segmentation\nmodels that treat word segmentation as an isolated task, our model unifies word\ndiscovery, learning how words fit together to form sentences, and, by\nconditioning the model on visual context, how words' meanings ground in\nrepresentations of non-linguistic modalities. Experiments show that the\nunconditional model learns predictive distributions better than character LSTM\nmodels, discovers words competitively with nonparametric Bayesian word\nsegmentation models, and that modeling language conditional on visual context\nimproves performance on both.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09417v2",
            "date": "2018-11-23 10:20:02+00:00",
            "title": "Natural language understanding for task oriented dialog in the biomedical domain in a low resources context",
            "authors": [
                "Antoine Neuraz",
                "Leonardo Campillos Llanos",
                "Anita Burgun",
                "Sophie Rosset"
            ],
            "abstract": "In the biomedical domain, the lack of sharable datasets often limit the\npossibility of developing natural language processing systems, especially\ndialogue applications and natural language understanding models. To overcome\nthis issue, we explore data generation using templates and terminologies and\ndata augmentation approaches. Namely, we report our experiments using\nparaphrasing and word representations learned on a large EHR corpus with\nFasttext and ELMo, to learn a NLU model without any available dataset. We\nevaluate on a NLU task of natural language queries in EHRs divided in\nslot-filling and intent classification sub-tasks. On the slot-filling task, we\nobtain a F-score of 0.76 with the ELMo representation; and on the\nclassification task, a mean F-score of 0.71. Our results show that this method\ncould be used to develop a baseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09575v1",
            "date": "2018-11-23 17:40:30+00:00",
            "title": "A Hierarchical Neural Network for Sequence-to-Sequences Learning",
            "authors": [
                "Si Zuo",
                "Zhimin Xu"
            ],
            "abstract": "In recent years, the sequence-to-sequence learning neural networks with\nattention mechanism have achieved great progress. However, there are still\nchallenges, especially for Neural Machine Translation (NMT), such as lower\ntranslation quality on long sentences. In this paper, we present a hierarchical\ndeep neural network architecture to improve the quality of long sentences\ntranslation. The proposed network embeds sequence-to-sequence neural networks\ninto a two-level category hierarchy by following the coarse-to-fine paradigm.\nLong sentences are input by splitting them into shorter sequences, which can be\nwell processed by the coarse category network as the long distance dependencies\nfor short sentences is able to be handled by network based on\nsequence-to-sequence neural network. Then they are concatenated and corrected\nby the fine category network. The experiments shows that our method can achieve\nsuperior results with higher BLEU(Bilingual Evaluation Understudy) scores,\nlower perplexity and better performance in imitating expression style and words\nusage than the traditional networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09785v1",
            "date": "2018-11-24 08:04:42+00:00",
            "title": "Strategy of the Negative Sampling for Training Retrieval-Based Dialogue Systems",
            "authors": [
                "Aigul Nugmanova",
                "Andrei Smirnov",
                "Galina Lavrentyeva",
                "Irina Chernykh"
            ],
            "abstract": "The article describes the new approach for quality improvement of automated\ndialogue systems for customer support service. Analysis produced in the paper\ndemonstrates the dependency of the quality of the retrieval-based dialogue\nsystem quality on the choice of negative responses. The proposed approach\nimplies choosing the negative samples according to the distribution of\nresponses in the train set. In this implementation the negative samples are\nrandomly chosen from the original response distribution and from the\n\"artificial\" distribution of negative responses, such as uniform distribution\nor the distribution obtained by transformation of the original one. The results\nobtained for the implemented systems and reported in this paper confirm the\nsignificant improvement of automated dialogue systems quality in case of using\nthe negative responses from transformed distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10167v1",
            "date": "2018-11-26 03:58:09+00:00",
            "title": "LSICC: A Large Scale Informal Chinese Corpus",
            "authors": [
                "Jianyu Zhao",
                "Zhuoran Ji"
            ],
            "abstract": "Deep learning based natural language processing model is proven powerful, but\nneed large-scale dataset. Due to the significant gap between the real-world\ntasks and existing Chinese corpus, in this paper, we introduce a large-scale\ncorpus of informal Chinese. This corpus contains around 37 million book reviews\nand 50 thousand netizen's comments to the news. We explore the informal words\nfrequencies of the corpus and show the difference between our corpus and the\nexisting ones. The corpus can be further used to train deep learning based\nnatural language processing tasks such as Chinese word segmentation, sentiment\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10188v1",
            "date": "2018-11-26 06:01:35+00:00",
            "title": "Implanting Rational Knowledge into Distributed Representation at Morpheme Level",
            "authors": [
                "Zi Lin",
                "Yang Liu"
            ],
            "abstract": "Previously, researchers paid no attention to the creation of unambiguous\nmorpheme embeddings independent from the corpus, while such information plays\nan important role in expressing the exact meanings of words for parataxis\nlanguages like Chinese. In this paper, after constructing the Chinese lexical\nand semantic ontology based on word-formation, we propose a novel approach to\nimplanting the structured rational knowledge into distributed representation at\nmorpheme level, naturally avoiding heavy disambiguation in the corpus. We\ndesign a template to create the instances as pseudo-sentences merely from the\npieces of knowledge of morphemes built in the lexicon. To exploit hierarchical\ninformation and tackle the data sparseness problem, the instance proliferation\ntechnique is applied based on similarity to expand the collection of\npseudo-sentences. The distributed representation for morphemes can then be\ntrained on these pseudo-sentences using word2vec. For evaluation, we validate\nthe paradigmatic and syntagmatic relations of morpheme embeddings, and apply\nthe obtained embeddings to word similarity measurement, achieving significant\nimprovements over the classical models by more than 5 Spearman scores or 8\npercentage points, which shows very promising prospects for adoption of the new\nsource of knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10211v1",
            "date": "2018-11-26 06:54:51+00:00",
            "title": "Multi-task Learning over Graph Structures",
            "authors": [
                "Pengfei Liu",
                "Jie Fu",
                "Yue Dong",
                "Xipeng Qiu",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "We present two architectures for multi-task learning with neural sequence\nmodels. Our approach allows the relationships between different tasks to be\nlearned dynamically, rather than using an ad-hoc pre-defined structure as in\nprevious work. We adopt the idea from message-passing graph neural networks and\npropose a general \\textbf{graph multi-task learning} framework in which\ndifferent tasks can communicate with each other in an effective and\ninterpretable way. We conduct extensive experiments in text classification and\nsequence labeling to evaluate our approach on multi-task learning and transfer\nlearning. The empirical results show that our models not only outperform\ncompetitive baselines but also learn interpretable and transferable patterns\nacross tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10278v1",
            "date": "2018-11-26 10:37:05+00:00",
            "title": "A Rule-based Kurdish Text Transliteration System",
            "authors": [
                "Sina Ahmadi"
            ],
            "abstract": "In this article, we present a rule-based approach for transliterating two\nmostly used orthographies in Sorani Kurdish. Our work consists of detecting a\ncharacter in a word by removing the possible ambiguities and mapping it into\nthe target orthography. We describe different challenges in Kurdish text mining\nand propose novel ideas concerning the transliteration task for Sorani Kurdish.\nOur transliteration system, named Wergor, achieves 82.79% overall precision and\nmore than 99% in detecting the double-usage characters. We also present a\nmanually transliterated corpus for Kurdish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10418v1",
            "date": "2018-11-26 14:52:06+00:00",
            "title": "Combining neural and knowledge-based approaches to Named Entity Recognition in Polish",
            "authors": [
                "S\u0142awomir Dadas"
            ],
            "abstract": "Named entity recognition (NER) is one of the tasks in natural language\nprocessing that can greatly benefit from the use of external knowledge sources.\nWe propose a named entity recognition framework composed of knowledge-based\nfeature extractors and a deep learning model including contextual word\nembeddings, long short-term memory (LSTM) layers and conditional random fields\n(CRF) inference layer. We use an entity linking module to integrate our system\nwith Wikipedia. The combination of effective neural architecture and external\nresources allows us to obtain state-of-the-art results on recognition of Polish\nproper names. We evaluate our model on data from PolEval 2018 NER challenge on\nwhich it outperforms other methods, reducing the error rate by 22.4% compared\nto the winning solution. Our work shows that combining neural NER model and\nentity linking model with a knowledge base is more effective in recognizing\nnamed entities than using NER model alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10761v2",
            "date": "2018-11-27 00:58:39+00:00",
            "title": "Speaker Diarization With Lexical Information",
            "authors": [
                "Tae Jin Park",
                "Kyu Han",
                "Ian Lane",
                "Panayiotis Georgiou"
            ],
            "abstract": "This work presents a novel approach to leverage lexical information for\nspeaker diarization. We introduce a speaker diarization system that can\ndirectly integrate lexical as well as acoustic information into a speaker\nclustering process. Thus, we propose an adjacency matrix integration technique\nto integrate word level speaker turn probabilities with speaker embeddings in a\ncomprehensive way. Our proposed method works without any reference transcript.\nWords, and word boundary information are provided by an ASR system. We show\nthat our proposed method improves a baseline speaker diarization system solely\nbased on speaker embeddings, achieving a meaningful improvement on the CALLHOME\nAmerican English Speech dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10773v1",
            "date": "2018-11-27 02:07:36+00:00",
            "title": "Verb Argument Structure Alternations in Word and Sentence Embeddings",
            "authors": [
                "Katharina Kann",
                "Alex Warstadt",
                "Adina Williams",
                "Samuel R. Bowman"
            ],
            "abstract": "Verbs occur in different syntactic environments, or frames. We investigate\nwhether artificial neural networks encode grammatical distinctions necessary\nfor inferring the idiosyncratic frame-selectional properties of verbs. We\nintroduce five datasets, collectively called FAVA, containing in aggregate\nnearly 10k sentences labeled for grammatical acceptability, illustrating\ndifferent verbal argument structure alternations. We then test whether models\ncan distinguish acceptable English verb-frame combinations from unacceptable\nones using a sentence embedding alone. For converging evidence, we further\nconstruct LaVA, a corresponding word-level dataset, and investigate whether the\nsame syntactic features can be extracted from word embeddings. Our models\nperform reliable classifications for some verbal alternations but not others,\nsuggesting that while these representations do encode fine-grained lexical\ninformation, it is incomplete or can be hard to extract. Further, differences\nbetween the word- and sentence-level models show that some information present\nin word embeddings is not passed on to the down-stream sentence embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10776v1",
            "date": "2018-11-27 02:24:37+00:00",
            "title": "Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision",
            "authors": [
                "Yixin Cao",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu",
                "Chengjiang Li",
                "Xu Chen",
                "Tiansi Dong"
            ],
            "abstract": "Joint representation learning of words and entities benefits many NLP tasks,\nbut has not been well explored in cross-lingual settings. In this paper, we\npropose a novel method for joint representation learning of cross-lingual words\nand entities. It captures mutually complementary knowledge, and enables\ncross-lingual inferences among knowledge bases and texts. Our method does not\nrequire parallel corpora, and automatically generates comparable data via\ndistant supervision using multi-lingual knowledge bases. We utilize two types\nof regularizers to align cross-lingual words and entities, and design knowledge\nattention and cross-lingual attention to further reduce noises. We conducted a\nseries of experiments on three tasks: word translation, entity relatedness, and\ncross-lingual entity linking. The results, both qualitatively and\nquantitatively, demonstrate the significance of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10971v2",
            "date": "2018-11-27 13:32:26+00:00",
            "title": "The Fact Extraction and VERification (FEVER) Shared Task",
            "authors": [
                "James Thorne",
                "Andreas Vlachos",
                "Oana Cocarascu",
                "Christos Christodoulopoulos",
                "Arpit Mittal"
            ],
            "abstract": "We present the results of the first Fact Extraction and VERification (FEVER)\nShared Task. The task challenged participants to classify whether human-written\nfactoid claims could be Supported or Refuted using evidence retrieved from\nWikipedia. We received entries from 23 competing teams, 19 of which scored\nhigher than the previously published baseline. The best performing system\nachieved a FEVER score of 64.21%. In this paper, we present the results of the\nshared task and a summary of the systems, highlighting commonalities and\ninnovations among participating systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11101v2",
            "date": "2018-11-27 16:51:39+00:00",
            "title": "Learning to detect dysarthria from raw speech",
            "authors": [
                "Juliette Millet",
                "Neil Zeghidour"
            ],
            "abstract": "Speech classifiers of paralinguistic traits traditionally learn from diverse\nhand-crafted low-level features, by selecting the relevant information for the\ntask at hand. We explore an alternative to this selection, by learning jointly\nthe classifier, and the feature extraction. Recent work on speech recognition\nhas shown improved performance over speech features by learning from the\nwaveform. We extend this approach to paralinguistic classification and propose\na neural network that can learn a filterbank, a normalization factor and a\ncompression power from the raw speech, jointly with the rest of the\narchitecture. We apply this model to dysarthria detection from sentence-level\naudio recordings. Starting from a strong attention-based baseline on which\nmel-filterbanks outperform standard low-level descriptors, we show that\nlearning the filters or the normalization and compression improves over fixed\nfeatures by 10% absolute accuracy. We also observe a gain over OpenSmile\nfeatures by learning jointly the feature extraction, the normalization, and the\ncompression factor with the architecture. This constitutes a first attempt at\nlearning jointly all these operations from raw audio for a speech\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11161v1",
            "date": "2018-11-27 18:52:58+00:00",
            "title": "Cross-Lingual Approaches to Reference Resolution in Dialogue Systems",
            "authors": [
                "Amr Sharaf",
                "Arpit Gupta",
                "Hancheng Ge",
                "Chetan Naik",
                "Lambert Mathias"
            ],
            "abstract": "In the slot-filling paradigm, where a user can refer back to slots in the\ncontext during the conversation, the goal of the contextual understanding\nsystem is to resolve the referring expressions to the appropriate slots in the\ncontext. In this paper, we build on the context carryover\nsystem~\\citep{Naik2018ContextualSC}, which provides a scalable multi-domain\nframework for resolving references. However, scaling this approach across\nlanguages is not a trivial task, due to the large demand on acquisition of\nannotated data in the target language. Our main focus is on cross-lingual\nmethods for reference resolution as a way to alleviate the need for annotated\ndata in the target language. In the cross-lingual setup, we assume there is\naccess to annotated resources as well as a well trained model in the source\nlanguage and little to no annotated data in the target language. In this paper,\nwe explore three different approaches for cross-lingual transfer \\textemdash~\\\ndelexicalization as data augmentation, multilingual embeddings and machine\ntranslation. We compare these approaches both on a low resource setting as well\nas a large resource setting. Our experiments show that multilingual embeddings\nand delexicalization via data augmentation have a significant impact in the low\nresource setting, but the gains diminish as the amount of available data in the\ntarget language increases. Furthermore, when combined with machine translation\nwe can get performance very close to actual live data in the target language,\nwith only 25\\% of the data projected into the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11374v1",
            "date": "2018-11-28 03:48:34+00:00",
            "title": "A Deep Cascade Model for Multi-Document Reading Comprehension",
            "authors": [
                "Ming Yan",
                "Jiangnan Xia",
                "Chen Wu",
                "Bin Bi",
                "Zhongzhou Zhao",
                "Ji Zhang",
                "Luo Si",
                "Rui Wang",
                "Wei Wang",
                "Haiqing Chen"
            ],
            "abstract": "A fundamental trade-off between effectiveness and efficiency needs to be\nbalanced when designing an online question answering system. Effectiveness\ncomes from sophisticated functions such as extractive machine reading\ncomprehension (MRC), while efficiency is obtained from improvements in\npreliminary retrieval components such as candidate document selection and\nparagraph ranking. Given the complexity of the real-world multi-document MRC\nscenario, it is difficult to jointly optimize both in an end-to-end system. To\naddress this problem, we develop a novel deep cascade learning model, which\nprogressively evolves from the document-level and paragraph-level ranking of\ncandidate texts to more precise answer extraction with machine reading\ncomprehension. Specifically, irrelevant documents and paragraphs are first\nfiltered out with simple functions for efficiency consideration. Then we\njointly train three modules on the remaining texts for better tracking the\nanswer: the document extraction, the paragraph extraction and the answer\nextraction. Experiment results show that the proposed method outperforms the\nprevious state-of-the-art methods on two large-scale multi-document benchmark\ndatasets, i.e., TriviaQA and DuReader. In addition, our online system can\nstably serve typical scenarios with millions of daily requests in less than\n50ms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11430v1",
            "date": "2018-11-28 07:58:16+00:00",
            "title": "Context-Aware Dialog Re-Ranking for Task-Oriented Dialog Systems",
            "authors": [
                "Junki Ohmura",
                "Maxine Eskenazi"
            ],
            "abstract": "Dialog response ranking is used to rank response candidates by considering\ntheir relation to the dialog history. Although researchers have addressed this\nconcept for open-domain dialogs, little attention has been focused on\ntask-oriented dialogs. Furthermore, no previous studies have analyzed whether\nresponse ranking can improve the performance of existing dialog systems in real\nhuman-computer dialogs with speech recognition errors. In this paper, we\npropose a context-aware dialog response re-ranking system. Our system reranks\nresponses in two steps: (1) it calculates matching scores for each candidate\nresponse and the current dialog context; (2) it combines the matching scores\nand a probability distribution of the candidates from an existing dialog system\nfor response re-ranking. By using neural word embedding-based models and\nhandcrafted or logistic regression-based ensemble models, we have improved the\nperformance of a recently proposed end-to-end task-oriented dialog system on\nreal dialogs with speech recognition errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11934v1",
            "date": "2018-11-29 02:44:07+00:00",
            "title": "Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering",
            "authors": [
                "Wei Wang",
                "Ming Yan",
                "Chen Wu"
            ],
            "abstract": "This paper describes a novel hierarchical attention network for reading\ncomprehension style question answering, which aims to answer questions for a\ngiven narrative paragraph. In the proposed method, attention and fusion are\nconducted horizontally and vertically across layers at different levels of\ngranularity between question and paragraph. Specifically, it first encode the\nquestion and paragraph with fine-grained language embeddings, to better capture\nthe respective representations at semantic level. Then it proposes a\nmulti-granularity fusion approach to fully fuse information from both global\nand attended representations. Finally, it introduces a hierarchical attention\nnetwork to focuses on the answer span progressively with multi-level\nsoftalignment. Extensive experiments on the large-scale SQuAD and TriviaQA\ndatasets validate the effectiveness of the proposed method. At the time of\nwriting the paper (Jan. 12th 2018), our model achieves the first position on\nthe SQuAD leaderboard for both single and ensemble models. We also achieves\nstate-of-the-art results on TriviaQA, AddSent and AddOne-Sent datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11958v1",
            "date": "2018-11-29 04:26:07+00:00",
            "title": "Large-scale Generative Modeling to Improve Automated Veterinary Disease Coding",
            "authors": [
                "Yuhui Zhang",
                "Allen Nie",
                "James Zou"
            ],
            "abstract": "Supervised learning is limited both by the quantity and quality of the\nlabeled data. In the field of medical record tagging, writing styles between\nhospitals vary drastically. The knowledge learned from one hospital might not\ntransfer well to another. This problem is amplified in veterinary medicine\ndomain because veterinary clinics rarely apply medical codes to their records.\nWe proposed and trained the first large-scale generative modeling algorithm in\nautomated disease coding. We demonstrate that generative modeling can learn\ndiscriminative features when additionally trained with supervised fine-tuning.\nWe systematically ablate and evaluate the effect of generative modeling on the\nfinal system's performance. We compare the performance of our model with\nseveral baselines in a challenging cross-hospital setting with substantial\ndomain shift. We outperform competitive baselines by a large margin. In\naddition, we provide interpretation for what is learned by our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12112v2",
            "date": "2018-11-29 13:05:18+00:00",
            "title": "Non-entailed subsequences as a challenge for natural language inference",
            "authors": [
                "R. Thomas McCoy",
                "Tal Linzen"
            ],
            "abstract": "Neural network models have shown great success at natural language inference\n(NLI), the task of determining whether a premise entails a hypothesis. However,\nrecent studies suggest that these models may rely on fallible heuristics rather\nthan deep language understanding. We introduce a challenge set to test whether\nNLI systems adopt one such heuristic: assuming that a sentence entails all of\nits subsequences, such as assuming that \"Alice believes Mary is lying\" entails\n\"Alice believes Mary.\" We evaluate several competitive NLI models on this\nchallenge set and find strong evidence that they do rely on the subsequence\nheuristic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12148v1",
            "date": "2018-11-29 14:03:00+00:00",
            "title": "Improving Robustness of Neural Dialog Systems in a Data-Efficient Way with Turn Dropout",
            "authors": [
                "Igor Shalyminov",
                "Sungjin Lee"
            ],
            "abstract": "Neural network-based dialog models often lack robustness to anomalous,\nout-of-domain (OOD) user input which leads to unexpected dialog behavior and\nthus considerably limits such models' usage in mission-critical production\nenvironments. The problem is especially relevant in the setting of dialog\nsystem bootstrapping with limited training data and no access to OOD examples.\nIn this paper, we explore the problem of robustness of such systems to\nanomalous input and the associated to it trade-off in accuracies on seen and\nunseen data. We present a new dataset for studying the robustness of dialog\nsystems to OOD input, which is bAbI Dialog Task 6 augmented with OOD content in\na controlled way. We then present turn dropout, a simple yet efficient negative\nsampling-based technique for improving robustness of neural dialog models. We\ndemonstrate its effectiveness applied to Hybrid Code Network-family models\n(HCNs) which reach state-of-the-art results on our OOD-augmented dataset as\nwell as the original one. Specifically, an HCN trained with turn dropout\nachieves state-of-the-art performance of more than 75% per-utterance accuracy\non the augmented dataset's OOD turns and 74% F1-score as an OOD detector.\nFurthermore, we introduce a Variational HCN enhanced with turn dropout which\nachieves more than 56.5% accuracy on the original bAbI Task 6 dataset, thus\noutperforming the initially reported HCN's result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12728v1",
            "date": "2018-11-30 11:10:59+00:00",
            "title": "Document Structure Measure for Hypernym discovery",
            "authors": [
                "Aswin Kannan",
                "Shanmukha C Guttula",
                "Balaji Ganesan",
                "Hima P Karanam",
                "Arun Kumar"
            ],
            "abstract": "Hypernym discovery is the problem of finding terms that have is-a\nrelationship with a given term. We introduce a new context type, and a\nrelatedness measure to differentiate hypernyms from other types of semantic\nrelationships. Our Document Structure measure is based on hierarchical position\nof terms in a document, and their presence or otherwise in definition text.\nThis measure quantifies the document structure using multiple attributes, and\nclasses of weighted distance functions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12793v2",
            "date": "2018-11-30 13:50:15+00:00",
            "title": "TIFTI: A Framework for Extracting Drug Intervals from Longitudinal Clinic Notes",
            "authors": [
                "Monica Agrawal",
                "Griffin Adams",
                "Nathan Nussbaum",
                "Benjamin Birnbaum"
            ],
            "abstract": "Oral drugs are becoming increasingly common in oncology care. In contrast to\nintravenous chemotherapy, which is administered in the clinic and carefully\ntracked via structure electronic health records (EHRs), oral drug treatment is\nself-administered and therefore not tracked as well. Often, the details of oral\ncancer treatment occur only in unstructured clinic notes. Extracting this\ninformation is critical to understanding a patient's treatment history. Yet,\nthis a challenging task because treatment intervals must be inferred\nlongitudinally from both explicit mentions in the text as well as from document\ntimestamps. In this work, we present TIFTI (Temporally Integrated Framework for\nTreatment Intervals), a robust framework for extracting oral drug treatment\nintervals from a patient's unstructured notes. TIFTI leverages distinct sources\nof temporal information by breaking the problem down into two separate\nsubtasks: document-level sequence labeling and date extraction. On a labeled\ndataset of metastatic renal-cell carcinoma (RCC) patients, it exactly matched\nthe labeled start date in 46% of the examples (86% of the examples within 30\ndays), and it exactly matched the labeled end date in 52% of the examples (78%\nof the examples within 30 days). Without retraining, the model achieved a\nsimilar level of performance on a labeled dataset of advanced non-small-cell\nlung cancer (NSCLC) patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12900v1",
            "date": "2018-11-30 17:14:30+00:00",
            "title": "Detecting Offensive Content in Open-domain Conversations using Two Stage Semi-supervision",
            "authors": [
                "Chandra Khatri",
                "Behnam Hedayatnia",
                "Rahul Goel",
                "Anushree Venkatesh",
                "Raefer Gabriel",
                "Arindam Mandal"
            ],
            "abstract": "As open-ended human-chatbot interaction becomes commonplace, sensitive\ncontent detection gains importance. In this work, we propose a two stage\nsemi-supervised approach to bootstrap large-scale data for automatic sensitive\nlanguage detection from publicly available web resources. We explore various\ndata selection methods including 1) using a blacklist to rank online discussion\nforums by the level of their sensitiveness followed by randomly sampling\nutterances and 2) training a weakly supervised model in conjunction with the\nblacklist for scoring sentences from online discussion forums to curate a\ndataset. Our data collection strategy is flexible and allows the models to\ndetect implicit sensitive content for which manual annotations may be\ndifficult. We train models using publicly available annotated datasets as well\nas using the proposed large-scale semi-supervised datasets. We evaluate the\nperformance of all the models on Twitter and Toxic Wikipedia comments testsets\nas well as on a manually annotated spoken language dataset collected during a\nlarge scale chatbot competition. Results show that a model trained on this\ncollected data outperforms the baseline models by a large margin on both\nin-domain and out-of-domain testsets, achieving an F1 score of 95.5% on an\nout-of-domain testset compared to a score of 75% for models trained on public\ndatasets. We also showcase that large scale two stage semi-supervision\ngeneralizes well across multiple classes of sensitivities such as hate speech,\nracism, sexual and pornographic content, etc. without even providing explicit\nlabels for these classes, leading to an average recall of 95.5% versus the\nmodels trained using annotated public datasets which achieve an average recall\nof 73.2% across seven sensitive classes on out-of-domain testsets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00161v1",
            "date": "2018-12-01 06:43:55+00:00",
            "title": "QADiver: Interactive Framework for Diagnosing QA Models",
            "authors": [
                "Gyeongbok Lee",
                "Sungdong Kim",
                "Seung-won Hwang"
            ],
            "abstract": "Question answering (QA) extracting answers from text to the given question in\nnatural language, has been actively studied and existing models have shown a\npromise of outperforming human performance when trained and evaluated with\nSQuAD dataset. However, such performance may not be replicated in the actual\nsetting, for which we need to diagnose the cause, which is non-trivial due to\nthe complexity of model. We thus propose a web-based UI that provides how each\nmodel contributes to QA performances, by integrating visualization and analysis\ntools for model explanation. We expect this framework can help QA model\nresearchers to refine and improve their models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00195v1",
            "date": "2018-12-01 12:13:55+00:00",
            "title": "One for All: Neural Joint Modeling of Entities and Events",
            "authors": [
                "Trung Minh Nguyen",
                "Thien Huu Nguyen"
            ],
            "abstract": "The previous work for event extraction has mainly focused on the predictions\nfor event triggers and argument roles, treating entity mentions as being\nprovided by human annotators. This is unrealistic as entity mentions are\nusually predicted by some existing toolkits whose errors might be propagated to\nthe event trigger and argument role recognition. Few of the recent work has\naddressed this problem by jointly predicting entity mentions, event triggers\nand arguments. However, such work is limited to using discrete engineering\nfeatures to represent contextual information for the individual tasks and their\ninteractions. In this work, we propose a novel model to jointly perform\npredictions for entity mentions, event triggers and arguments based on the\nshared hidden representations from deep learning. The experiments demonstrate\nthe benefits of the proposed method, leading to the state-of-the-art\nperformance for event extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00686v1",
            "date": "2018-12-03 11:46:43+00:00",
            "title": "Building Sequential Inference Models for End-to-End Response Selection",
            "authors": [
                "Jia-Chen Gu",
                "Zhen-Hua Ling",
                "Yu-Ping Ruan",
                "Quan Liu"
            ],
            "abstract": "This paper presents an end-to-end response selection model for Track 1 of the\n7th Dialogue System Technology Challenges (DSTC7). This task focuses on\nselecting the correct next utterance from a set of candidates given a partial\nconversation. We propose an end-to-end neural network based on enhanced\nsequential inference model (ESIM) for this task. Our proposed model differs\nfrom the original ESIM model in the following four aspects. First, a new word\nrepresentation method which combines the general pre-trained word embeddings\nwith those estimated on the task-specific training set is adopted in order to\naddress the challenge of out-of-vocabulary (OOV) words. Second, an attentive\nhierarchical recurrent encoder (AHRE) is designed which is capable to encode\nsentences hierarchically and generate more descriptive representations by\naggregation. Third, a new pooling method which combines multi-dimensional\npooling and last-state pooling is used instead of the simple combination of max\npooling and average pooling in the original ESIM. Last, a modification layer is\nadded before the softmax layer to emphasize the importance of the last\nutterance in the context for response selection. In the released evaluation\nresults of DSTC7, our proposed method ranked second on the Ubuntu dataset and\nthird on the Advising dataset in subtask 1 of Track 1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00798v1",
            "date": "2018-12-03 14:53:41+00:00",
            "title": "The RGNLP Machine Translation Systems for WAT 2018",
            "authors": [
                "Atul Kr. Ojha",
                "Koel Dutta Chowdhury",
                "Chao-Hong Liu",
                "Karan Saxena"
            ],
            "abstract": "This paper presents the system description of Machine Translation (MT)\nsystem(s) for Indic Languages Multilingual Task for the 2018 edition of the WAT\nShared Task. In our experiments, we (the RGNLP team) explore both statistical\nand neural methods across all language pairs. (We further present an extensive\ncomparison of language-related problems for both the approaches in the context\nof low-resourced settings.) Our PBSMT models were highest score on all\nautomatic evaluation metrics in the English into Telugu, Hindi, Bengali, Tamil\nportion of the shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00815v1",
            "date": "2018-12-03 15:04:23+00:00",
            "title": "Comparing Neural- and N-Gram-Based Language Models for Word Segmentation",
            "authors": [
                "Yerai Doval",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Word segmentation is the task of inserting or deleting word boundary\ncharacters in order to separate character sequences that correspond to words in\nsome language. In this article we propose an approach based on a beam search\nalgorithm and a language model working at the byte/character level, the latter\ncomponent implemented either as an n-gram model or a recurrent neural network.\nThe resulting system analyzes the text input with no word boundaries one token\nat a time, which can be a character or a byte, and uses the information\ngathered by the language model to determine if a boundary must be placed in the\ncurrent position or not. Our aim is to use this system in a preprocessing step\nfor a microtext normalization system. This means that it needs to effectively\ncope with the data sparsity present on this kind of texts. We also strove to\nsurpass the performance of two readily available word segmentation systems: The\nwell-known and accessible Word Breaker by Microsoft, and the Python module\nWordSegment by Grant Jenks. The results show that we have met our objectives,\nand we hope to continue to improve both the precision and the efficiency of our\nsystem in the future.",
            "journal ref": "Volume 69, Issue 11, 2018, 11 pages",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00899v1",
            "date": "2018-12-03 16:52:34+00:00",
            "title": "Toward Scalable Neural Dialogue State Tracking Model",
            "authors": [
                "Elnaz Nouri",
                "Ehsan Hosseini-Asl"
            ],
            "abstract": "The latency in the current neural based dialogue state tracking models\nprohibits them from being used efficiently for deployment in production\nsystems, albeit their highly accurate performance. This paper proposes a new\nscalable and accurate neural dialogue state tracking model, based on the\nrecently proposed Global-Local Self-Attention encoder (GLAD) model by Zhong et\nal. which uses global modules to share parameters between estimators for\ndifferent types (called slots) of dialogue states, and uses local modules to\nlearn slot-specific features. By using only one recurrent networks with global\nconditioning, compared to (1 + \\# slots) recurrent networks with global and\nlocal conditioning used in the GLAD model, our proposed model reduces the\nlatency in training and inference times by $35\\%$ on average, while preserving\nperformance of belief state tracking, by $97.38\\%$ on turn request and\n$88.51\\%$ on joint goal and accuracy. Evaluation on Multi-domain dataset\n(Multi-WoZ) also demonstrates that our model outperforms GLAD on turn inform\nand joint goal accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00978v3",
            "date": "2018-12-03 18:53:08+00:00",
            "title": "A Survey on Semantic Parsing",
            "authors": [
                "Aishwarya Kamath",
                "Rajarshi Das"
            ],
            "abstract": "A significant amount of information in today's world is stored in structured\nand semi-structured knowledge bases. Efficient and simple methods to query them\nare essential and must not be restricted to only those who have expertise in\nformal query languages. The field of semantic parsing deals with converting\nnatural language utterances to logical forms that can be easily executed on a\nknowledge base. In this survey, we examine the various components of a semantic\nparsing system and discuss prominent work ranging from the initial rule based\nmethods to the current neural approaches to program synthesis. We also discuss\nmethods that operate using varying levels of supervision and highlight the key\nchallenges involved in the learning of such systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01083v1",
            "date": "2018-12-03 21:12:31+00:00",
            "title": "A System for Automated Image Editing from Natural Language Commands",
            "authors": [
                "Jacqueline Brixey",
                "Ramesh Manuvinakurike",
                "Nham Le",
                "Tuan Lai",
                "Walter Chang",
                "Trung Bui"
            ],
            "abstract": "This work presents the task of modifying images in an image editing program\nusing natural language written commands. We utilize a corpus of over 6000 image\nedit text requests to alter real world images collected via crowdsourcing. A\nnovel framework composed of actions and entities to map a user's natural\nlanguage request to executable commands in an image editing program is\ndescribed. We resolve previously labeled annotator disagreement through a\nvoting process and complete annotation of the corpus. We experimented with\ndifferent machine learning models and found that the LSTM, the SVM, and the\nbidirectional LSTM-CRF joint models are the best to detect image editing\nactions and associated entities in a given utterance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01193v2",
            "date": "2018-12-04 03:15:38+00:00",
            "title": "e-SNLI: Natural Language Inference with Natural Language Explanations",
            "authors": [
                "Oana-Maria Camburu",
                "Tim Rockt\u00e4schel",
                "Thomas Lukasiewicz",
                "Phil Blunsom"
            ],
            "abstract": "In order for machine learning to garner widespread public adoption, models\nmust be able to provide interpretable and robust explanations for their\ndecisions, as well as learn from human-provided explanations at train time. In\nthis work, we extend the Stanford Natural Language Inference dataset with an\nadditional layer of human-annotated natural language explanations of the\nentailment relations. We further implement models that incorporate these\nexplanations into their training process and output them at test time. We show\nhow our corpus of explanations, which we call e-SNLI, can be used for various\ngoals, such as obtaining full sentence justifications of a model's decisions,\nimproving universal sentence representations and transferring to out-of-domain\nNLI datasets. Our dataset thus opens up a range of research directions for\nusing natural language explanations, both for improving models and for\nasserting their trust.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01207v1",
            "date": "2018-12-04 04:04:56+00:00",
            "title": "Practical Text Classification With Large Pre-Trained Language Models",
            "authors": [
                "Neel Kant",
                "Raul Puri",
                "Nikolai Yakovenko",
                "Bryan Catanzaro"
            ],
            "abstract": "Multi-emotion sentiment classification is a natural language processing (NLP)\nproblem with valuable use cases on real-world data. We demonstrate that\nlarge-scale unsupervised language modeling combined with finetuning offers a\npractical solution to this task on difficult datasets, including those with\nlabel class imbalance and domain-specific context. By training an\nattention-based Transformer network (Vaswani et al. 2017) on 40GB of text\n(Amazon reviews) (McAuley et al. 2015) and fine-tuning on the training set, our\nmodel achieves a 0.69 F1 score on the SemEval Task 1:E-c multi-dimensional\nemotion classification problem (Mohammad et al. 2018), based on the Plutchik\nwheel of emotions (Plutchik 1979). These results are competitive with state of\nthe art models, including strong F1 scores on difficult (emotion) categories\nsuch as Fear (0.73), Disgust (0.77) and Anger (0.78), as well as competitive\nresults on rare categories such as Anticipation (0.42) and Surprise (0.37).\nFurthermore, we demonstrate our application on a real world text classification\ntask. We create a narrowly collected text dataset of real tweets on several\ntopics, and show that our finetuned model outperforms general purpose\ncommercially available APIs for sentiment and multidimensional emotion\nclassification on this dataset by a significant margin. We also perform a\nvariety of additional studies, investigating properties of deep learning\narchitectures, datasets and algorithms for achieving practical multidimensional\nsentiment classification. Overall, we find that unsupervised language modeling\nand finetuning is a simple framework for achieving high quality results on\nreal-world sentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01527v2",
            "date": "2018-12-04 17:01:52+00:00",
            "title": "Leveraging Multi-grained Sentiment Lexicon Information for Neural Sequence Models",
            "authors": [
                "Yan Zeng",
                "Yangyang Lan",
                "Yazhou Hao",
                "Chen Li",
                "Qinhua Zheng"
            ],
            "abstract": "Neural sequence models have achieved great success in sentence-level\nsentiment classification. However, some models are exceptionally complex or\nbased on expensive features. Some other models recognize the value of existed\nlinguistic resource but utilize it insufficiently. This paper proposes a novel\nand general method to incorporate lexicon information, including sentiment\nlexicons(+/-), negation words and intensifiers. Words are annotated in\nfine-grained and coarse-grained labels. The proposed method first encodes the\nfine-grained labels into sentiment embedding and concatenates it with word\nembedding. Second, the coarse-grained labels are utilized to enhance the\nattention mechanism to give large weight on sentiment-related words.\nExperimental results show that our method can increase classification accuracy\nfor neural sequence models on both SST-5 and MR dataset. Specifically, the\nenhanced Bi-LSTM model can even compare with a Tree-LSTM which uses expensive\nphrase-level annotations. Further analysis shows that in most cases the lexicon\nresource can offer the right annotations. Besides, the proposed method is\ncapable of overcoming the effect from inevitably wrong annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01704v1",
            "date": "2018-12-04 21:44:36+00:00",
            "title": "Impact of Sentiment Detection to Recognize Toxic and Subversive Online Comments",
            "authors": [
                "\u00c9loi Brassard-Gourdeau",
                "Richard Khoury"
            ],
            "abstract": "The presence of toxic content has become a major problem for many online\ncommunities. Moderators try to limit this problem by implementing more and more\nrefined comment filters, but toxic users are constantly finding new ways to\ncircumvent them. Our hypothesis is that while modifying toxic content and\nkeywords to fool filters can be easy, hiding sentiment is harder. In this\npaper, we explore various aspects of sentiment detection and their correlation\nto toxicity, and use our results to implement a toxicity detection tool. We\nthen test how adding the sentiment information helps detect toxicity in three\ndifferent real-world datasets, and incorporate subversion to these datasets to\nsimulate a user trying to circumvent the system. Our results show sentiment\ninformation has a positive impact on toxicity detection against a subversive\nuser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01840v2",
            "date": "2018-12-05 07:24:51+00:00",
            "title": "Attention Boosted Sequential Inference Model",
            "authors": [
                "Guanyu Li",
                "Pengfei Zhang",
                "Caiyan Jia"
            ],
            "abstract": "Attention mechanism has been proven effective on natural language processing.\nThis paper proposes an attention boosted natural language inference model named\naESIM by adding word attention and adaptive direction-oriented attention\nmechanisms to the traditional Bi-LSTM layer of natural language inference\nmodels, e.g. ESIM. This makes the inference model aESIM has the ability to\neffectively learn the representation of words and model the local subsentential\ninference between pairs of premise and hypothesis. The empirical studies on the\nSNLI, MultiNLI and Quora benchmarks manifest that aESIM is superior to the\noriginal ESIM model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01885v1",
            "date": "2018-12-05 10:02:59+00:00",
            "title": "Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding",
            "authors": [
                "Ying Shen",
                "Qiang Zhang",
                "Jin Zhang",
                "Jiyue Huang",
                "Yuming Lu",
                "Kai Lei"
            ],
            "abstract": "Automatic text classification (TC) research can be used for real-world\nproblems such as the classification of in-patient discharge summaries and\nmedical text reports, which is beneficial to make medical documents more\nunderstandable to doctors. However, in electronic medical records (EMR), the\ntexts containing sentences are shorter than that in general domain, which leads\nto the lack of semantic features and the ambiguity of semantic. To tackle this\nchallenge, we propose to add word-cluster embedding to deep neural network for\nimproving short text classification. Concretely, we first use hierarchical\nagglomerative clustering to cluster the word vectors in the semantic space.\nThen we calculate the cluster center vector which represents the implicit topic\ninformation of words in the cluster. Finally, we expand word vector with\ncluster center vector, and implement classifiers using CNN and LSTM\nrespectively. To evaluate the performance of our proposed method, we conduct\nexperiments on public data sets TREC and the medical short sentences data sets\nwhich is constructed and released by us. The experimental results demonstrate\nthat our proposed method outperforms state-of-the-art baselines in short\nsentence classification on both medical domain and general domain.",
            "journal ref": "International Conference on Information Science and Applications\n  ICISA 2018: Information Science and Applications 2018 pp 401-411",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02119v1",
            "date": "2018-12-05 17:17:30+00:00",
            "title": "Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications",
            "authors": [
                "Christian Jilek",
                "Markus Schr\u00f6der",
                "Rudolf Novik",
                "Sven Schwarz",
                "Heiko Maus",
                "Andreas Dengel"
            ],
            "abstract": "A growing number of applications users daily interact with have to operate in\n(near) real-time: chatbots, digital companions, knowledge work support systems\n-- just to name a few. To perform the services desired by the user, these\nsystems have to analyze user activity logs or explicit user input extremely\nfast. In particular, text content (e.g. in form of text snippets) needs to be\nprocessed in an information extraction task. Regarding the aforementioned\ntemporal requirements, this has to be accomplished in just a few milliseconds,\nwhich limits the number of methods that can be applied. Practically, only very\nfast methods remain, which on the other hand deliver worse results than slower\nbut more sophisticated Natural Language Processing (NLP) pipelines. In this\npaper, we investigate and propose methods for real-time capable Named Entity\nRecognition (NER). As a first improvement step we address are word variations\ninduced by inflection, for example present in the German language. Our approach\nis ontology-based and makes use of several language information sources like\nWiktionary. We evaluated it using the German Wikipedia (about 9.4B characters),\nfor which the whole NER process took considerably less than an hour. Since\nprecision and recall are higher than with comparably fast methods, we conclude\nthat the quality gap between high speed methods and sophisticated NLP pipelines\ncan be narrowed a bit more without losing too much runtime performance.",
            "journal ref": "2nd Conference on Language, Data and Knowledge (LDK 2019),\n  OpenAccess Series in Informatics (OASIcs), Vol. 70, pp. 11:1-11:14",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02205v1",
            "date": "2018-12-05 19:55:24+00:00",
            "title": "Are you tough enough? Framework for Robustness Validation of Machine Comprehension Systems",
            "authors": [
                "Barbara Rychalska",
                "Dominika Basaj",
                "Przemyslaw Biecek"
            ],
            "abstract": "Deep Learning NLP domain lacks procedures for the analysis of model\nrobustness. In this paper we propose a framework which validates robustness of\nany Question Answering model through model explainers. We propose that a robust\nmodel should transgress the initial notion of semantic similarity induced by\nword embeddings to learn a more human-like understanding of meaning. We test\nthis property by manipulating questions in two ways: swapping important\nquestion word for 1) its semantically correct synonym and 2) for word vector\nthat is close in embedding space. We estimate importance of words in asked\nquestions with Locally Interpretable Model Agnostic Explanations method (LIME).\nWith these two steps we compare state-of-the-art Q&A models. We show that\nalthough accuracy of state-of-the-art models is high, they are very fragile to\nchanges in the input. Moreover, we propose 2 adversarial training scenarios\nwhich raise model sensitivity to true synonyms by up to 7% accuracy measure.\nOur findings help to understand which models are more stable and how they can\nbe improved. In addition, we have created and published a new dataset that may\nbe used for validation of robustness of a Q&A model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02253v1",
            "date": "2018-12-05 22:27:20+00:00",
            "title": "Weighted Global Normalization for Multiple Choice ReadingComprehension over Long Documents",
            "authors": [
                "Aditi Chaudhary",
                "Bhargavi Paranjape",
                "Michiel de Jong"
            ],
            "abstract": "Motivated by recent evidence pointing out the fragility of high-performing\nspan prediction models, we direct our attention to multiple choice reading\ncomprehension. In particular, this work introduces a novel method for improving\nanswer selection on long documents through weighted global normalization of\npredictions over portions of the documents. We show that applying our method to\na span prediction model adapted for answer selection helps model performance on\nlong summaries from NarrativeQA, a challenging reading comprehension dataset\nwith an answer selection task, and we strongly improve on the task baseline\nperformance by +36.2 Mean Reciprocal Rank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02317v1",
            "date": "2018-12-06 02:54:49+00:00",
            "title": "Adpositional Supersenses for Mandarin Chinese",
            "authors": [
                "Yilun Zhu",
                "Yang Liu",
                "Siyao Peng",
                "Austin Blodgett",
                "Yushi Zhao",
                "Nathan Schneider"
            ],
            "abstract": "This study adapts Semantic Network of Adposition and Case Supersenses (SNACS)\nannotation to Mandarin Chinese and demonstrates that the same supersense\ncategories are appropriate for Chinese adposition semantics. We annotated 15\nchapters of The Little Prince, with high interannotator agreement. The parallel\ncorpus gives insight into differences in construal between the two languages'\nadpositions, namely a number of construals that are frequent in Chinese but\nrare or unattested in the English corpus. The annotated corpus can further\nsupport automatic disambiguation of adpositions in Chinese, and the common\ninventory of supersenses between the two languages can potentially serve\ncross-linguistic tasks such as machine translation.",
            "journal ref": "Zhu, Yilun; Liu, Yang; Peng, Siyao; Blodgett, Austin; Zhao, Yushi;\n  and Schneider, Nathan (2019) \"Adpositional Supersenses for Mandarin Chinese,\"\n  Proceedings of the Society for Computation in Linguistics: Vol. 2 , Article\n  40",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02354v1",
            "date": "2018-12-06 05:20:59+00:00",
            "title": "Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering",
            "authors": [
                "Yang Deng",
                "Yuexiang Xie",
                "Yaliang Li",
                "Min Yang",
                "Nan Du",
                "Wei Fan",
                "Kai Lei",
                "Ying Shen"
            ],
            "abstract": "Answer selection and knowledge base question answering (KBQA) are two\nimportant tasks of question answering (QA) systems. Existing methods solve\nthese two tasks separately, which requires large number of repetitive work and\nneglects the rich correlation information between tasks. In this paper, we\ntackle answer selection and KBQA tasks simultaneously via multi-task learning\n(MTL), motivated by the following motivations. First, both answer selection and\nKBQA can be regarded as a ranking problem, with one at text-level while the\nother at knowledge-level. Second, these two tasks can benefit each other:\nanswer selection can incorporate the external knowledge from knowledge base\n(KB), while KBQA can be improved by learning contextual information from answer\nselection. To fulfill the goal of jointly learning these two tasks, we propose\na novel multi-task learning scheme that utilizes multi-view attention learned\nfrom various perspectives to enable these tasks to interact with each other as\nwell as learn more comprehensive sentence representations. The experiments\nconducted on several real-world datasets demonstrate the effectiveness of the\nproposed method, and the performance of answer selection and KBQA is improved.\nAlso, the multi-view attention scheme is proved to be effective in assembling\nattentive information from different representational perspectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02370v1",
            "date": "2018-12-06 06:53:37+00:00",
            "title": "Exploring the importance of context and embeddings in neural NER models for task-oriented dialogue systems",
            "authors": [
                "Pratik Jayarao",
                "Chirag Jain",
                "Aman Srivastava"
            ],
            "abstract": "Named Entity Recognition (NER), a classic sequence labelling task, is an\nessential component of natural language understanding (NLU) systems in\ntask-oriented dialog systems for slot filling. For well over a decade,\ndifferent methods from lookup using gazetteers and domain ontology, classifiers\nover handcrafted features to end-to-end systems involving neural network\narchitectures have been evaluated mostly in language-independent\nnon-conversational settings. In this paper, we evaluate a modified version of\nthe recent state of the art neural architecture in a conversational setting\nwhere messages are often short and noisy. We perform an array of experiments\nwith different combinations of including the previous utterance in the dialogue\nas a source of additional features and using word and character level\nembeddings trained on a larger external corpus. All methods are evaluated on a\ncombined dataset formed from two public English task-oriented conversational\ndatasets belonging to travel and restaurant domains respectively. For\nadditional evaluation, we also repeat some of our experiments after adding\nautomatically translated and transliterated (from translated) versions to the\nEnglish only dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02802v2",
            "date": "2018-12-06 21:00:58+00:00",
            "title": "End-to-End Streaming Keyword Spotting",
            "authors": [
                "Alvarez Raziel",
                "Park Hyun-Jin"
            ],
            "abstract": "We present a system for keyword spotting that, except for a frontend\ncomponent for feature generation, it is entirely contained in a deep neural\nnetwork (DNN) model trained \"end-to-end\" to predict the presence of the keyword\nin a stream of audio. The main contributions of this work are, first, an\nefficient memoized neural network topology that aims at making better use of\nthe parameters and associated computations in the DNN by holding a memory of\nprevious activations distributed over the depth of the DNN. The second\ncontribution is a method to train the DNN, end-to-end, to produce the keyword\nspotting score. This system significantly outperforms previous approaches both\nin terms of quality of detection as well as size and computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02914v1",
            "date": "2018-12-07 05:23:12+00:00",
            "title": "Intent Detection for code-mix utterances in task oriented dialogue systems",
            "authors": [
                "Pratik Jayarao",
                "Aman Srivastava"
            ],
            "abstract": "Intent detection is an essential component of task oriented dialogue systems.\nOver the years, extensive research has been conducted resulting in many state\nof the art models directed towards resolving user's intents in dialogue. A\nvariety of vector representations foruser utterances have been explored for the\nsame. However, these models and vectorization approaches have more so been\nevaluated in a single language environment. Dialogude systems generally have to\ndeal with queries in different languages. We thus conduct experiments across\ncombinations of models and various vectors representations for Code Mix as well\nas multi language utterances and evaluate how these models scale to a multi\nlanguage environment. Our aim is to find the best suitable combination of\nvector representation and models for the process of intent detection for Code\nMix utterances. we have evaluated the experiments on two different datasets\nconsisting of only Code Mix utterances and the other dataset consisting of\nEnglish, Hindi and Code Mix English Hindi utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02971v2",
            "date": "2018-12-07 10:44:14+00:00",
            "title": "Improving Retrieval-Based Question Answering with Deep Inference Models",
            "authors": [
                "George-Sebastian Pirtoaca",
                "Traian Rebedea",
                "Stefan Ruseti"
            ],
            "abstract": "Question answering is one of the most important and difficult applications at\nthe border of information retrieval and natural language processing, especially\nwhen we talk about complex science questions which require some form of\ninference to determine the correct answer. In this paper, we present a two-step\nmethod that combines information retrieval techniques optimized for question\nanswering with deep learning models for natural language inference in order to\ntackle the multi-choice question answering in the science domain. For each\nquestion-answer pair, we use standard retrieval-based models to find relevant\ncandidate contexts and decompose the main problem into two different\nsub-problems. First, assign correctness scores for each candidate answer based\non the context using retrieval models from Lucene. Second, we use deep learning\narchitectures to compute if a candidate answer can be inferred from some\nwell-chosen context consisting of sentences retrieved from the knowledge base.\nIn the end, all these solvers are combined using a simple neural network to\npredict the correct answer. This proposed two-step model outperforms the best\nretrieval-based solver by over 3% in absolute accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03361v2",
            "date": "2018-12-08 18:11:16+00:00",
            "title": "An Unsupervised Approach for Aspect Category Detection Using Soft Cosine Similarity Measure",
            "authors": [
                "Erfan Ghadery",
                "Sajad Movahedi",
                "Heshaam Faili",
                "Azadeh Shakery"
            ],
            "abstract": "Aspect category detection is one of the important and challenging subtasks of\naspect-based sentiment analysis. Given a set of pre-defined categories, this\ntask aims to detect categories which are indicated implicitly or explicitly in\na given review sentence. Supervised machine learning approaches perform well to\naccomplish this subtask. Note that, the performance of these methods depends on\nthe availability of labeled train data, which is often difficult and costly to\nobtain. Besides, most of these supervised methods require feature engineering\nto perform well. In this paper, we propose an unsupervised method to address\naspect category detection task without the need for any feature engineering.\nOur method utilizes clusters of unlabeled reviews and soft cosine similarity\nmeasure to accomplish aspect category detection task. Experimental results on\nSemEval-2014 restaurant dataset shows that proposed unsupervised approach\noutperforms several baselines by a substantial margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03593v5",
            "date": "2018-12-10 01:43:14+00:00",
            "title": "SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering",
            "authors": [
                "Chenguang Zhu",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "Conversational question answering (CQA) is a novel QA task that requires\nunderstanding of dialogue context. Different from traditional single-turn\nmachine reading comprehension (MRC) tasks, CQA includes passage comprehension,\ncoreference resolution, and contextual understanding. In this paper, we propose\nan innovated contextualized attention-based deep neural network, SDNet, to fuse\ncontext into traditional MRC models. Our model leverages both inter-attention\nand self-attention to comprehend conversation context and extract relevant\ninformation from passage. Furthermore, we demonstrated a novel method to\nintegrate the latest BERT contextual model. Empirical results show the\neffectiveness of our model, which sets the new state of the art result in CoQA\nleaderboard, outperforming the previous best model by 1.6% F1. Our ensemble\nmodel further improves the result by 2.7% F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04160v2",
            "date": "2018-12-11 00:19:32+00:00",
            "title": "Delta Embedding Learning",
            "authors": [
                "Xiao Zhang",
                "Ji Wu",
                "Dejing Dou"
            ],
            "abstract": "Unsupervised word embeddings have become a popular approach of word\nrepresentation in NLP tasks. However there are limitations to the semantics\nrepresented by unsupervised embeddings, and inadequate fine-tuning of\nembeddings can lead to suboptimal performance. We propose a novel learning\ntechnique called Delta Embedding Learning, which can be applied to general NLP\ntasks to improve performance by optimized tuning of the word embeddings. A\nstructured regularization is applied to the embeddings to ensure they are tuned\nin an incremental way. As a result, the tuned word embeddings become better\nword representations by absorbing semantic information from supervision without\n\"forgetting.\" We apply the method to various NLP tasks and see a consistent\nimprovement in performance. Evaluation also confirms the tuned word embeddings\nhave better semantic properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04361v2",
            "date": "2018-12-11 12:41:14+00:00",
            "title": "RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information",
            "authors": [
                "Shikhar Vashishth",
                "Rishabh Joshi",
                "Sai Suman Prayaga",
                "Chiranjib Bhattacharyya",
                "Partha Talukdar"
            ],
            "abstract": "Distantly-supervised Relation Extraction (RE) methods train an extractor by\nautomatically aligning relation instances in a Knowledge Base (KB) with\nunstructured text. In addition to relation instances, KBs often contain other\nrelevant side information, such as aliases of relations (e.g., founded and\nco-founded are aliases for the relation founderOfCompany). RE models usually\nignore such readily available side information. In this paper, we propose\nRESIDE, a distantly-supervised neural relation extraction method which utilizes\nadditional side information from KBs for improved relation extraction. It uses\nentity type and relation alias information for imposing soft constraints while\npredicting relations. RESIDE employs Graph Convolution Networks (GCN) to encode\nsyntactic information from text and improves performance even when limited side\ninformation is available. Through extensive experiments on benchmark datasets,\nwe demonstrate RESIDE's effectiveness. We have made RESIDE's source code\navailable to encourage reproducible research.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04405v1",
            "date": "2018-12-11 14:05:24+00:00",
            "title": "Conditional Variational Autoencoder for Neural Machine Translation",
            "authors": [
                "Artidoro Pagnoni",
                "Kevin Liu",
                "Shangyan Li"
            ],
            "abstract": "We explore the performance of latent variable models for conditional text\ngeneration in the context of neural machine translation (NMT). Similar to Zhang\net al., we augment the encoder-decoder NMT paradigm by introducing a continuous\nlatent variable to model features of the translation process. We extend this\nmodel with a co-attention mechanism motivated by Parikh et al. in the inference\nnetwork. Compared to the vision domain, latent variable models for text face\nadditional challenges due to the discrete nature of language, namely posterior\ncollapse. We experiment with different approaches to mitigate this issue. We\nshow that our conditional variational model improves upon both discriminative\nattention-based translation and the variational baseline presented in Zhang et\nal. Finally, we present some exploration of the learned latent space to\nillustrate what the latent variable is capable of capturing. This is the first\nreported conditional variational model for text that meaningfully utilizes the\nlatent variable without weakening the translation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04647v1",
            "date": "2018-12-11 19:02:05+00:00",
            "title": "Scalable language model adaptation for spoken dialogue systems",
            "authors": [
                "Ankur Gandhe",
                "Ariya Rastrow",
                "Bjorn Hoffmeister"
            ],
            "abstract": "Language models (LM) for interactive speech recognition systems are trained\non large amounts of data and the model parameters are optimized on past user\ndata. New application intents and interaction types are released for these\nsystems over time, imposing challenges to adapt the LMs since the existing\ntraining data is no longer sufficient to model the future user interactions. It\nis unclear how to adapt LMs to new application intents without degrading the\nperformance on existing applications. In this paper, we propose a solution to\n(a) estimate n-gram counts directly from the hand-written grammar for training\nLMs and (b) use constrained optimization to optimize the system parameters for\nfuture use cases, while not degrading the performance on past usage. We\nevaluated our approach on new applications intents for a personal assistant\nsystem and find that the adaptation improves the word error rate by up to 15%\non new applications even when there is no adaptation data available for an\napplication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04662v1",
            "date": "2018-12-11 19:34:49+00:00",
            "title": "Unsupervised domain-agnostic identification of product names in social media posts",
            "authors": [
                "Nicolai Pogrebnyakov"
            ],
            "abstract": "Product name recognition is a significant practical problem, spurred by the\ngreater availability of platforms for discussing products such as social media\nand product review functionalities of online marketplaces. Customers, product\nmanufacturers and online marketplaces may want to identify product names in\nunstructured text to extract important insights, such as sentiment, surrounding\na product. Much extant research on product name identification has been\ndomain-specific (e.g., identifying mobile phone models) and used supervised or\nsemi-supervised methods. With massive numbers of new products released to the\nmarket every year such methods may require retraining on updated labeled data\nto stay relevant, and may transfer poorly across domains. This research\naddresses this challenge and develops a domain-agnostic, unsupervised algorithm\nfor identifying product names based on Facebook posts. The algorithm consists\nof two general steps: (a) candidate product name identification using an\noff-the-shelf pretrained conditional random fields (CRF) model, part-of-speech\ntagging and a set of simple patterns; and (b) filtering of candidate names to\nremove spurious entries using clustering and word embeddings generated from the\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04784v1",
            "date": "2018-12-12 02:47:11+00:00",
            "title": "Sentence-wise Smooth Regularization for Sequence to Sequence Learning",
            "authors": [
                "Chengyue Gong",
                "Xu Tan",
                "Di He",
                "Tao Qin"
            ],
            "abstract": "Maximum-likelihood estimation (MLE) is widely used in sequence to sequence\ntasks for model training. It uniformly treats the generation/prediction of each\ntarget token as multi-class classification, and yields non-smooth prediction\nprobabilities: in a target sequence, some tokens are predicted with small\nprobabilities while other tokens are with large probabilities. According to our\nempirical study, we find that the non-smoothness of the probabilities results\nin low quality of generated sequences. In this paper, we propose a\nsentence-wise regularization method which aims to output smooth prediction\nprobabilities for all the tokens in the target sequence. Our proposed method\ncan automatically adjust the weights and gradients of each token in one\nsentence to ensure the predictions in a sequence uniformly well. Experiments on\nthree neural machine translation tasks and one text summarization task show\nthat our method outperforms conventional MLE loss on all these tasks and\nachieves promising BLEU scores on WMT14 English-German and WMT17\nChinese-English translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04891v2",
            "date": "2018-12-12 10:57:52+00:00",
            "title": "A Multimodal LSTM for Predicting Listener Empathic Responses Over Time",
            "authors": [
                "Zhi-Xuan Tan",
                "Arushi Goel",
                "Thanh-Son Nguyen",
                "Desmond C. Ong"
            ],
            "abstract": "People naturally understand the emotions of-and often also empathize\nwith-those around them. In this paper, we predict the emotional valence of an\nempathic listener over time as they listen to a speaker narrating a life story.\nWe use the dataset provided by the OMG-Empathy Prediction Challenge, a workshop\nheld in conjunction with IEEE FG 2019. We present a multimodal LSTM model with\nfeature-level fusion and local attention that predicts empathic responses from\naudio, text, and visual features. Our best-performing model, which used only\nthe audio and text features, achieved a concordance correlation coefficient\n(CCC) of 0.29 and 0.32 on the Validation set for the Generalized and\nPersonalized track respectively, and achieved a CCC of 0.14 and 0.14 on the\nheld-out Test set. We discuss the difficulties faced and the lessons learnt\ntackling this challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04898v1",
            "date": "2018-12-12 11:11:08+00:00",
            "title": "SMT vs NMT: A Comparison over Hindi & Bengali Simple Sentences",
            "authors": [
                "Sainik Kumar Mahata",
                "Soumil Mandal",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "In the present article, we identified the qualitative differences between\nStatistical Machine Translation (SMT) and Neural Machine Translation (NMT)\noutputs. We have tried to answer two important questions: 1. Does NMT perform\nequivalently well with respect to SMT and 2. Does it add extra flavor in\nimproving the quality of MT output by employing simple sentences as training\nunits. In order to obtain insights, we have developed three core models viz.,\nSMT model based on Moses toolkit, followed by character and word level NMT\nmodels. All of the systems use English-Hindi and English-Bengali language pairs\ncontaining simple sentences as well as sentences of other complexity. In order\nto preserve the translations semantics with respect to the target words of a\nsentence, we have employed soft-attention into our word level NMT model. We\nhave further evaluated all the systems with respect to the scenarios where they\nsucceed and fail. Finally, the quality of translation has been validated using\nBLEU and TER metrics along with manual parameters like fluency, adequacy etc.\nWe observed that NMT outperforms SMT in case of simple sentences whereas SMT\noutperforms in case of all types of sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05270v5",
            "date": "2018-12-13 05:32:23+00:00",
            "title": "Joint Entity Extraction and Assertion Detection for Clinical Text",
            "authors": [
                "Parminder Bhatia",
                "Busra Celikkaya",
                "Mohammed Khalilia"
            ],
            "abstract": "Negative medical findings are prevalent in clinical reports, yet\ndiscriminating them from positive findings remains a challenging task for\ninformation extraction. Most of the existing systems treat this task as a\npipeline of two separate tasks, i.e., named entity recognition (NER) and\nrule-based negation detection. We consider this as a multi-task problem and\npresent a novel end-to-end neural model to jointly extract entities and\nnegations. We extend a standard hierarchical encoder-decoder NER model and\nfirst adopt a shared encoder followed by separate decoders for the two tasks.\nThis architecture performs considerably better than the previous rule-based and\nmachine learning-based systems. To overcome the problem of increased parameter\nsize especially for low-resource settings, we propose the Conditional Softmax\nShared Decoder architecture which achieves state-of-art results for NER and\nnegation detection on the 2010 i2b2/VA challenge dataset and a proprietary\nde-identified clinical dataset.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics. Florence, Italy, 2019, pp. 954-959",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05272v1",
            "date": "2018-12-13 05:33:11+00:00",
            "title": "Towards a General-Purpose Linguistic Annotation Backend",
            "authors": [
                "Graham Neubig",
                "Patrick Littell",
                "Chian-Yu Chen",
                "Jean Lee",
                "Zirui Li",
                "Yu-Hsiang Lin",
                "Yuyan Zhang"
            ],
            "abstract": "Language documentation is inherently a time-intensive process; transcription,\nglossing, and corpus management consume a significant portion of documentary\nlinguists' work. Advances in natural language processing can help to accelerate\nthis work, using the linguists' past decisions as training material, but\nquestions remain about how to prioritize human involvement. In this extended\nabstract, we describe the beginnings of a new project that will attempt to ease\nthis language documentation process through the use of natural language\nprocessing (NLP) technology. It is based on (1) methods to adapt NLP tools to\nnew languages, based on recent advances in massively multilingual neural\nnetworks, and (2) backend APIs and interfaces that allow linguists to upload\ntheir data. We then describe our current progress on two fronts: automatic\nphoneme transcription, and glossing. Finally, we briefly describe our future\ndirections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05411v1",
            "date": "2018-12-13 13:20:40+00:00",
            "title": "Find a Reasonable Ending for Stories: Does Logic Relation Help the Story Cloze Test?",
            "authors": [
                "Mingyue Shang",
                "Zhenxin Fu",
                "Hongzhi Yin",
                "Bo Tang",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Natural language understanding is a challenging problem that covers a wide\nrange of tasks. While previous methods generally train each task separately, we\nconsider combining the cross-task features to enhance the task performance. In\nthis paper, we incorporate the logic information with the help of the Natural\nLanguage Inference (NLI) task to the Story Cloze Test (SCT). Previous work on\nSCT considered various semantic information, such as sentiment and topic, but\nlack the logic information between sentences which is an essential element of\nstories. Thus we propose to extract the logic information during the course of\nthe story to improve the understanding of the whole story. The logic\ninformation is modeled with the help of the NLI task. Experimental results\nprove the strength of the logic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05813v1",
            "date": "2018-12-14 08:09:29+00:00",
            "title": "A corpus of precise natural textual entailment problems",
            "authors": [
                "Jean-Philippe Bernardy",
                "Stergios Chatzikyriakidis"
            ],
            "abstract": "In this paper, we present a new corpus of entailment problems. This corpus\ncombines the following characteristics: 1. it is precise (does not leave out\nimplicit hypotheses) 2. it is based on \"real-world\" texts (i.e. most of the\npremises were written for purposes other than testing textual entailment). 3.\nits size is 150. The corpus was constructed by taking problems from the Real\nText Entailment and discovering missing hypotheses using a crowd of experts. We\nbelieve that this corpus constitutes a first step towards wide-coverage testing\nof precise natural-language inference systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05936v1",
            "date": "2018-12-14 13:57:43+00:00",
            "title": "Detecting Reliable Novel Word Senses: A Network-Centric Approach",
            "authors": [
                "Abhik Jana",
                "Animesh Mukherjee",
                "Pawan Goyal"
            ],
            "abstract": "In this era of Big Data, due to expeditious exchange of information on the\nweb, words are being used to denote newer meanings, causing linguistic shift.\nWith the recent availability of large amounts of digitized texts, an automated\nanalysis of the evolution of language has become possible. Our study mainly\nfocuses on improving the detection of new word senses. This paper presents a\nunique proposal based on network features to improve the precision of new word\nsense detection. For a candidate word where a new sense (birth) has been\ndetected by comparing the sense clusters induced at two different time points,\nwe further compare the network properties of the subgraphs induced from novel\nsense cluster across these two time points. Using the mean fractional change in\nedge density, structural similarity and average path length as features in an\nSVM classifier, manual evaluation gives precision values of 0.86 and 0.74 for\nthe task of new sense detection, when tested on 2 distinct time-point pairs, in\ncomparison to the precision values in the range of 0.23-0.32, when the proposed\nscheme is not used. The outlined method can therefore be used as a new post-hoc\nstep to improve the precision of novel word sense detection in a robust and\nreliable way where the underlying framework uses a graph structure. Another\nimportant observation is that even though our proposal is a post-hoc step, it\ncan be used in isolation and that itself results in a very decent performance\nachieving a precision of 0.54-0.62. Finally, we show that our method is able to\ndetect the well-known historical shifts in 80% cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05984v1",
            "date": "2018-12-14 15:39:34+00:00",
            "title": "Measuring Similarity: Computationally Reproducing the Scholar's Interests",
            "authors": [
                "Ashley Lee",
                "Jo Guldi",
                "Andras Zsom"
            ],
            "abstract": "Computerized document classification already orders the news articles that\nApple's \"News\" app or Google's \"personalized search\" feature groups together to\nmatch a reader's interests. The invisible and therefore illegible decisions\nthat go into these tailored searches have been the subject of a critique by\nscholars who emphasize that our intelligence about documents is only as good as\nour ability to understand the criteria of search. This article will attempt to\nunpack the procedures used in computational classification of texts,\ntranslating them into term legible to humanists, and examining opportunities to\nrender the computational text classification process subject to expert critique\nand improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06604v1",
            "date": "2018-12-17 03:52:11+00:00",
            "title": "Siamese Networks for Semantic Pattern Similarity",
            "authors": [
                "Yassine Benajiba",
                "Jin Sun",
                "Yong Zhang",
                "Longquan Jiang",
                "Zhiliang Weng",
                "Or Biran"
            ],
            "abstract": "Semantic Pattern Similarity is an interesting, though not often encountered\nNLP task where two sentences are compared not by their specific meaning, but by\ntheir more abstract semantic pattern (e.g., preposition or frame). We utilize\nSiamese Networks to model this task, and show its usefulness in determining SQL\npatterns for unseen questions in a database-backed question answering scenario.\nOur approach achieves high accuracy and contains a built-in proxy for\nconfidence, which can be used to keep precision arbitrarily high.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06864v2",
            "date": "2018-12-17 16:07:12+00:00",
            "title": "Fully Convolutional Speech Recognition",
            "authors": [
                "Neil Zeghidour",
                "Qiantong Xu",
                "Vitaliy Liptchinsky",
                "Nicolas Usunier",
                "Gabriel Synnaeve",
                "Ronan Collobert"
            ],
            "abstract": "Current state-of-the-art speech recognition systems build on recurrent neural\nnetworks for acoustic and/or language modeling, and rely on feature extraction\npipelines to extract mel-filterbanks or cepstral coefficients. In this paper we\npresent an alternative approach based solely on convolutional neural networks,\nleveraging recent advances in acoustic models from the raw waveform and\nlanguage modeling. This fully convolutional approach is trained end-to-end to\npredict characters from the raw waveform, removing the feature extraction step\naltogether. An external convolutional language model is used to decode words.\nOn Wall Street Journal, our model matches the current state-of-the-art. On\nLibrispeech, we report state-of-the-art performance among end-to-end models,\nincluding Deep Speech 2 trained with 12 times more acoustic data and\nsignificantly more linguistic data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06876v2",
            "date": "2018-12-17 16:32:05+00:00",
            "title": "Multi-task learning to improve natural language understanding",
            "authors": [
                "Stefan Constantin",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "Recently advancements in sequence-to-sequence neural network architectures\nhave led to an improved natural language understanding. When building a neural\nnetwork-based Natural Language Understanding component, one main challenge is\nto collect enough training data. The generation of a synthetic dataset is an\ninexpensive and quick way to collect data. Since this data often has less\nvariety than real natural language, neural networks often have problems to\ngeneralize to unseen utterances during testing. In this work, we address this\nchallenge by using multi-task learning. We train out-of-domain real data\nalongside in-domain synthetic data to improve natural language understanding.\nWe evaluate this approach in the domain of airline travel information with two\nsynthetic datasets. As out-of-domain real data, we test two datasets based on\nthe subtitles of movies and series. By using an attention-based encoder-decoder\nmodel, we were able to improve the F1-score over strong baselines from 80.76 %\nto 84.98 % in the smaller synthetic dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07207v2",
            "date": "2018-12-18 07:24:17+00:00",
            "title": "Multiple topic identification in human/human conversations",
            "authors": [
                "X. Bost",
                "G. Senay",
                "M. El-B\u00e8ze",
                "R. De Mori"
            ],
            "abstract": "The paper deals with the automatic analysis of real-life telephone\nconversations between an agent and a customer of a customer care service (ccs).\nThe application domain is the public transportation system in Paris and the\npurpose is to collect statistics about customer problems in order to monitor\nthe service and decide priorities on the intervention for improving user\nsatisfaction. Of primary importance for the analysis is the detection of themes\nthat are the object of customer problems. Themes are defined in the application\nrequirements and are part of the application ontology that is implicit in the\nccs documentation. Due to variety of customer population, the structure of\nconversations with an agent is unpredictable. A conversation may be about one\nor more themes. Theme mentions can be interleaved with mentions of facts that\nare irrelevant for the application purpose. Furthermore, in certain\nconversations theme mentions are localized in specific conversation segments\nwhile in other conversations mentions cannot be localized. As a consequence,\napproaches to feature extraction with and without mention localization are\nconsidered. Application domain relevant themes identified by an automatic\nprocedure are expressed by specific sentences whose words are hypothesized by\nan automatic speech recognition (asr) system. The asr system is error prone.\nThe word error rates can be very high for many reasons. Among them it is worth\nmentioning unpredictable background noise, speaker accent, and various types of\nspeech disfluencies. As the application task requires the composition of\nproportions of theme mentions, a sequential decision strategy is introduced in\nthis paper for performing a survey of the large amount of conversations made\navailable in a given time period. The strategy has to sample the conversations\nto form a survey containing enough data analyzed with high accuracy so that\nproportions can be estimated with sufficient accuracy. Due to the unpredictable\ntype of theme mentions, it is appropriate to consider methods for theme\nhypothesization based on global as well as local feature extraction. Two\nsystems based on each type of feature extraction will be considered by the\nstrategy. One of the four methods is novel. It is based on a new definition of\ndensity of theme mentions and on the localization of high density zones whose\nboundaries do not need to be precisely detected. The sequential decision\nstrategy starts by grouping theme hypotheses into sets of different expected\naccuracy and coverage levels. For those sets for which accuracy can be improved\nwith a consequent increase of coverage a new system with new features is\nintroduced. Its execution is triggered only when specific preconditions are met\non the hypotheses generated by the basic four systems. Experimental results are\nprovided on a corpus collected in the call center of the Paris transportation\nsystem known as ratp. The results show that surveys with high accuracy and\ncoverage can be composed with the proposed strategy and systems. This makes it\npossible to apply a previously published proportion estimation approach that\ntakes into account hypothesization errors .",
            "journal ref": "Computer Speech \\& Language, 2015, 34 (1), pp.18-42",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07324v1",
            "date": "2018-12-18 12:15:33+00:00",
            "title": "Predicting user intent from search queries using both CNNs and RNNs",
            "authors": [
                "Mihai Cristian P\u00eervu",
                "Alexandra Anghel",
                "Ciprian Borodescu",
                "Alexandru Constantin"
            ],
            "abstract": "Predicting user behaviour on a website is a difficult task, which requires\nthe integration of multiple sources of information, such as geo-location, user\nprofile or web surfing history. In this paper we tackle the problem of\npredicting the user intent, based on the queries that were used to access a\ncertain webpage. We make no additional assumptions, such as domain detection,\ndevice used or location, and only use the word information embedded in the\ngiven query. In order to build competitive classifiers, we label a small\nfraction of the EDI query intent prediction dataset\n\\cite{edi-challenge-dataset}, which is used as ground truth. Then, using\nvarious rule-based approaches, we automatically label the rest of the dataset,\ntrain the classifiers and evaluate the quality of the automatic labeling on the\nground truth dataset. We used both recurrent and convolutional networks as the\nmodels, while representing the words in the query with multiple embedding\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07546v1",
            "date": "2018-12-18 18:21:24+00:00",
            "title": "Supervised Domain Enablement Attention for Personalized Domain Classification",
            "authors": [
                "Joo-Kyung Kim",
                "Young-Bum Kim"
            ],
            "abstract": "In large-scale domain classification for natural language understanding,\nleveraging each user's domain enablement information, which refers to the\npreferred or authenticated domains by the user, with attention mechanism has\nbeen shown to improve the overall domain classification performance. In this\npaper, we propose a supervised enablement attention mechanism, which utilizes\nsigmoid activation for the attention weighting so that the attention can be\ncomputed with more expressive power without the weight sum constraint of\nsoftmax attention. The attention weights are explicitly encouraged to be\nsimilar to the corresponding elements of the ground-truth's one-hot vector by\nsupervised attention, and the attention information of the other enabled\ndomains is leveraged through self-distillation. By evaluating on the actual\nutterances from a large-scale IPDA, we show that our approach significantly\nimproves domain classification performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07625v1",
            "date": "2018-12-18 19:57:24+00:00",
            "title": "wav2letter++: The Fastest Open-source Speech Recognition System",
            "authors": [
                "Vineel Pratap",
                "Awni Hannun",
                "Qiantong Xu",
                "Jeff Cai",
                "Jacob Kahn",
                "Gabriel Synnaeve",
                "Vitaliy Liptchinsky",
                "Ronan Collobert"
            ],
            "abstract": "This paper introduces wav2letter++, the fastest open-source deep learning\nspeech recognition framework. wav2letter++ is written entirely in C++, and uses\nthe ArrayFire tensor library for maximum efficiency. Here we explain the\narchitecture and design of the wav2letter++ system and compare it to other\nmajor open-source speech recognition systems. In some cases wav2letter++ is\nmore than 2x faster than other optimized frameworks for training end-to-end\nneural networks for speech recognition. We also show that wav2letter++'s\ntraining times scale linearly to 64 GPUs, the highest we tested, for models\nwith 100 million parameters. High-performance frameworks enable fast iteration,\nwhich is often a crucial factor in successful research and model tuning on new\ndatasets and tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07754v1",
            "date": "2018-12-19 04:51:42+00:00",
            "title": "Streaming Voice Query Recognition using Causal Convolutional Recurrent Neural Networks",
            "authors": [
                "Raphael Tang",
                "Gefei Yang",
                "Hong Wei",
                "Yajie Mao",
                "Ferhan Ture",
                "Jimmy Lin"
            ],
            "abstract": "Voice-enabled commercial products are ubiquitous, typically enabled by\nlightweight on-device keyword spotting (KWS) and full automatic speech\nrecognition (ASR) in the cloud. ASR systems require significant computational\nresources in training and for inference, not to mention copious amounts of\nannotated speech data. KWS systems, on the other hand, are less\nresource-intensive but have limited capabilities. On the Comcast Xfinity X1\nentertainment platform, we explore a middle ground between ASR and KWS: We\nintroduce a novel, resource-efficient neural network for voice query\nrecognition that is much more accurate than state-of-the-art CNNs for KWS, yet\ncan be easily trained and deployed with limited resources. On an evaluation\ndataset representing the top 200 voice queries, we achieve a low false alarm\nrate of 1% and a query error rate of 6%. Our model performs inference 8.24x\nfaster than the current ASR system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07807v2",
            "date": "2018-12-19 08:35:38+00:00",
            "title": "DTMT: A Novel Deep Transition Architecture for Neural Machine Translation",
            "authors": [
                "Fandong Meng",
                "Jinchao Zhang"
            ],
            "abstract": "Past years have witnessed rapid developments in Neural Machine Translation\n(NMT). Most recently, with advanced modeling and training techniques, the\nRNN-based NMT (RNMT) has shown its potential strength, even compared with the\nwell-known Transformer (self-attentional) model. Although the RNMT model can\npossess very deep architectures through stacking layers, the transition depth\nbetween consecutive hidden states along the sequential axis is still shallow.\nIn this paper, we further enhance the RNN-based NMT through increasing the\ntransition depth between consecutive hidden states and build a novel Deep\nTransition RNN-based Architecture for Neural Machine Translation, named DTMT.\nThis model enhances the hidden-to-hidden transition with multiple non-linear\ntransformations, as well as maintains a linear transformation path throughout\nthis deep transition by the well-designed linear transformation mechanism to\nalleviate the gradient vanishing problem. Experiments show that with the\nspecially designed deep transition modules, our DTMT can achieve remarkable\nimprovements on translation quality. Experimental results on Chinese->English\ntranslation task show that DTMT can outperform the Transformer model by +2.09\nBLEU points and achieve the best results ever reported in the same dataset. On\nWMT14 English->German and English->French translation tasks, DTMT shows\nsuperior quality to the state-of-the-art NMT systems, including the Transformer\nand the RNMT+.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07860v1",
            "date": "2018-12-19 10:21:20+00:00",
            "title": "Self-Attention: A Better Building Block for Sentiment Analysis Neural Network Classifiers",
            "authors": [
                "Artaches Ambartsoumian",
                "Fred Popowich"
            ],
            "abstract": "Sentiment Analysis has seen much progress in the past two decades. For the\npast few years, neural network approaches, primarily RNNs and CNNs, have been\nthe most successful for this task. Recently, a new category of neural networks,\nself-attention networks (SANs), have been created which utilizes the attention\nmechanism as the basic building block. Self-attention networks have been shown\nto be effective for sequence modeling tasks, while having no recurrence or\nconvolutions. In this work we explore the effectiveness of the SANs for\nsentiment analysis. We demonstrate that SANs are superior in performance to\ntheir RNN and CNN counterparts by comparing their classification accuracy on\nsix datasets as well as their model characteristics such as training speed and\nmemory consumption. Finally, we explore the effects of various SAN\nmodifications such as multi-head attention as well as two methods of\nincorporating sequence position information into SANs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08033v1",
            "date": "2018-12-19 15:48:01+00:00",
            "title": "Switch-LSTMs for Multi-Criteria Chinese Word Segmentation",
            "authors": [
                "Jingjing Gong",
                "Xinchi Chen",
                "Tao Gui",
                "Xipeng Qiu"
            ],
            "abstract": "Multi-criteria Chinese word segmentation is a promising but challenging task,\nwhich exploits several different segmentation criteria and mines their common\nunderlying knowledge. In this paper, we propose a flexible multi-criteria\nlearning for Chinese word segmentation. Usually, a segmentation criterion could\nbe decomposed into multiple sub-criteria, which are shareable with other\nsegmentation criteria. The process of word segmentation is a routing among\nthese sub-criteria. From this perspective, we present Switch-LSTMs to segment\nwords, which consist of several long short-term memory neural networks (LSTM),\nand a switcher to automatically switch the routing among these LSTMs. With\nthese auto-switched LSTMs, our model provides a more flexible solution for\nmulti-criteria CWS, which is also easy to transfer the learned knowledge to new\ncriteria. Experiments show that our model obtains significant improvements on\neight corpora with heterogeneous segmentation criteria, compared to the\nprevious method and single-criterion learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08044v1",
            "date": "2018-12-19 15:59:31+00:00",
            "title": "FrameNet automatic analysis : a study on a French corpus of encyclopedic texts",
            "authors": [
                "Gabriel Marzinotto",
                "G\u00e9raldine Damnati",
                "Frederic Bechet"
            ],
            "abstract": "This article presents an automatic frame analysis system evaluated on a\ncorpus of French encyclopedic history texts annotated according to the FrameNet\nformalism. The chosen approach relies on an integrated sequence labeling model\nwhich jointly optimizes frame identification and semantic role segmentation and\nidentification. The purpose of this study is to analyze the task complexity\nfrom several dimensions. Hence we provide detailed evaluations from a feature\nselection point of view and from the data point of view.",
            "journal ref": "TALN 2017, 2017, Orl{\\'e}ans, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08407v1",
            "date": "2018-12-20 08:05:54+00:00",
            "title": "Context, Attention and Audio Feature Explorations for Audio Visual Scene-Aware Dialog",
            "authors": [
                "Shachi H Kumar",
                "Eda Okur",
                "Saurav Sahay",
                "Juan Jose Alvarado Leanos",
                "Jonathan Huang",
                "Lama Nachman"
            ],
            "abstract": "With the recent advancements in AI, Intelligent Virtual Assistants (IVA) have\nbecome a ubiquitous part of every home. Going forward, we are witnessing a\nconfluence of vision, speech and dialog system technologies that are enabling\nthe IVAs to learn audio-visual groundings of utterances and have conversations\nwith users about the objects, activities and events surrounding them. As a part\nof the 7th Dialog System Technology Challenges (DSTC7), for Audio Visual\nScene-Aware Dialog (AVSD) track, We explore `topics' of the dialog as an\nimportant contextual feature into the architecture along with explorations\naround multimodal Attention. We also incorporate an end-to-end audio\nclassification ConvNet, AclNet, into our models. We present detailed analysis\nof the experiments and show that some of our model variations outperform the\nbaseline system presented for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08621v4",
            "date": "2018-12-20 15:02:39+00:00",
            "title": "How Much Does Tokenization Affect Neural Machine Translation?",
            "authors": [
                "Miguel Domingo",
                "Mercedes Garc\u0131a-Mart\u0131nez",
                "Alexandre Helle",
                "Francisco Casacuberta",
                "Manuel Herranz"
            ],
            "abstract": "Tokenization or segmentation is a wide concept that covers simple processes\nsuch as separating punctuation from words, or more sophisticated processes such\nas applying morphological knowledge. Neural Machine Translation (NMT) requires\na limited-size vocabulary for computational cost and enough examples to\nestimate word embeddings. Separating punctuation and splitting tokens into\nwords or subwords has proven to be helpful to reduce vocabulary and increase\nthe number of examples of each word, improving the translation quality.\nTokenization is more challenging when dealing with languages with no separator\nbetween words. In order to assess the impact of the tokenization in the quality\nof the final translation on NMT, we experimented on five tokenizers over ten\nlanguage pairs. We reached the conclusion that the tokenization significantly\naffects the final translation quality and that the best tokenizer differs for\ndifferent language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08718v2",
            "date": "2018-12-20 17:44:05+00:00",
            "title": "RNNs Implicitly Implement Tensor Product Representations",
            "authors": [
                "R. Thomas McCoy",
                "Tal Linzen",
                "Ewan Dunbar",
                "Paul Smolensky"
            ],
            "abstract": "Recurrent neural networks (RNNs) can learn continuous vector representations\nof symbolic structures such as sequences and sentences; these representations\noften exhibit linear regularities (analogies). Such regularities motivate our\nhypothesis that RNNs that show such regularities implicitly compile symbolic\nstructures into tensor product representations (TPRs; Smolensky, 1990), which\nadditively combine tensor products of vectors representing roles (e.g.,\nsequence positions) and vectors representing fillers (e.g., particular words).\nTo test this hypothesis, we introduce Tensor Product Decomposition Networks\n(TPDNs), which use TPRs to approximate existing vector representations. We\ndemonstrate using synthetic data that TPDNs can successfully approximate linear\nand tree-based RNN autoencoder representations, suggesting that these\nrepresentations exhibit interpretable compositional structure; we explore the\nsettings that lead RNNs to induce such structure-sensitive representations. By\ncontrast, further TPDN experiments show that the representations of four models\ntrained to encode naturally-occurring sentences can be largely approximated\nwith a bag of words, with only marginal improvements from more sophisticated\nstructures. We conclude that TPDNs provide a powerful method for interpreting\nvector representations, and that standard RNNs can induce compositional\nsequence representations that are remarkably well approximated by TPRs; at the\nsame time, existing training tasks for sentence representation learning may not\nbe sufficient for inducing robust structural representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08729v1",
            "date": "2018-12-12 23:06:43+00:00",
            "title": "PyText: A Seamless Path from NLP research to production",
            "authors": [
                "Ahmed Aly",
                "Kushal Lakhotia",
                "Shicong Zhao",
                "Mrinal Mohit",
                "Barlas Oguz",
                "Abhinav Arora",
                "Sonal Gupta",
                "Christopher Dewan",
                "Stef Nelson-Lindall",
                "Rushin Shah"
            ],
            "abstract": "We introduce PyText - a deep learning based NLP modeling framework built on\nPyTorch. PyText addresses the often-conflicting requirements of enabling rapid\nexperimentation and of serving models at scale. It achieves this by providing\nsimple and extensible interfaces for model components, and by using PyTorch's\ncapabilities of exporting models for inference via the optimized Caffe2\nexecution engine. We report our own experience of migrating experimentation and\nproduction workflows to PyText, which enabled us to iterate faster on novel\nmodeling ideas and then seamlessly ship them at industrial scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09321v2",
            "date": "2018-12-21 15:31:17+00:00",
            "title": "Multiple topic identification in telephone conversations",
            "authors": [
                "Xavier Bost",
                "Marc El B\u00e8ze",
                "Renato De Mori"
            ],
            "abstract": "This paper deals with the automatic analysis of conversations between a\ncustomer and an agent in a call centre of a customer care service. The purpose\nof the analysis is to hypothesize themes about problems and complaints\ndiscussed in the conversation. Themes are defined by the application\ndocumentation topics. A conversation may contain mentions that are irrelevant\nfor the application purpose and multiple themes whose mentions may be\ninterleaved portions of a conversation that cannot be well defined. Two methods\nare proposed for multiple theme hypothesization. One of them is based on a\ncosine similarity measure using a bag of features extracted from the entire\nconversation. The other method introduces the concept of thematic density\ndistributed around specific word positions in a conversation. In addition to\nautomatically selected words, word bi-grams with possible gaps between\nsuccessive words are also considered and selected. Experimental results show\nthat the results obtained with the proposed methods outperform the results\nobtained with support vector machines on the same data. Furthermore, using the\ntheme skeleton of a conversation from which thematic densities are derived, it\nwill be possible to extract components of an automatic conversation report to\nbe used for improving the service performance. Index Terms: multi-topic audio\ndocument classification, hu-man/human conversation analysis, speech analytics,\ndistance bigrams",
            "journal ref": "Interspeech, Aug 2013, Lyon, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09355v1",
            "date": "2018-12-21 19:51:47+00:00",
            "title": "What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models",
            "authors": [
                "Fahim Dalvi",
                "Nadir Durrani",
                "Hassan Sajjad",
                "Yonatan Belinkov",
                "Anthony Bau",
                "James Glass"
            ],
            "abstract": "Despite the remarkable evolution of deep neural networks in natural language\nprocessing (NLP), their interpretability remains a challenge. Previous work\nlargely focused on what these models learn at the representation level. We\nbreak this analysis down further and study individual dimensions (neurons) in\nthe vector representation learned by end-to-end neural models in NLP tasks. We\npropose two methods: Linguistic Correlation Analysis, based on a supervised\nmethod to extract the most relevant neurons with respect to an extrinsic task,\nand Cross-model Correlation Analysis, an unsupervised method to extract salient\nneurons w.r.t. the model itself. We evaluate the effectiveness of our\ntechniques by ablating the identified neurons and reevaluating the network's\nperformance for two tasks: neural machine translation (NMT) and neural language\nmodeling (NLM). We further present a comprehensive analysis of neurons with the\naim to address the following questions: i) how localized or distributed are\ndifferent linguistic properties in the models? ii) are certain neurons\nexclusive to some properties and not others? iii) is the information more or\nless distributed in NMT vs. NLM? and iv) how important are the neurons\nidentified through the linguistic correlation method to the overall task? Our\ncode is publicly available as part of the NeuroX toolkit (Dalvi et al. 2019).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09359v1",
            "date": "2018-12-21 20:20:26+00:00",
            "title": "NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks",
            "authors": [
                "Fahim Dalvi",
                "Avery Nortonsmith",
                "D. Anthony Bau",
                "Yonatan Belinkov",
                "Hassan Sajjad",
                "Nadir Durrani",
                "James Glass"
            ],
            "abstract": "We present a toolkit to facilitate the interpretation and understanding of\nneural network models. The toolkit provides several methods to identify salient\nneurons with respect to the model itself or an external task. A user can\nvisualize selected neurons, ablate them to measure their effect on the model\naccuracy, and manipulate them to control the behavior of the model at the test\ntime. Such an analysis has a potential to serve as a springboard in various\nresearch directions, such as understanding the model, better architectural\nchoices, model distillation and controlling data biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09449v3",
            "date": "2018-12-22 04:54:13+00:00",
            "title": "A Survey on Deep Learning for Named Entity Recognition",
            "authors": [
                "Jing Li",
                "Aixin Sun",
                "Jianglei Han",
                "Chenliang Li"
            ],
            "abstract": "Named entity recognition (NER) is the task to identify mentions of rigid\ndesignators from text belonging to predefined semantic types such as person,\nlocation, organization etc. NER always serves as the foundation for many\nnatural language applications such as question answering, text summarization,\nand machine translation. Early NER systems got a huge success in achieving good\nperformance with the cost of human engineering in designing domain-specific\nfeatures and rules. In recent years, deep learning, empowered by continuous\nreal-valued vector representations and semantic composition through nonlinear\nprocessing, has been employed in NER systems, yielding stat-of-the-art\nperformance. In this paper, we provide a comprehensive review on existing deep\nlearning techniques for NER. We first introduce NER resources, including tagged\nNER corpora and off-the-shelf NER tools. Then, we systematically categorize\nexisting works based on a taxonomy along three axes: distributed\nrepresentations for input, context encoder, and tag decoder. Next, we survey\nthe most representative methods for recent applied techniques of deep learning\nin new NER problem settings and applications. Finally, we present readers with\nthe challenges faced by NER systems and outline future directions in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09471v2",
            "date": "2018-12-22 07:49:42+00:00",
            "title": "Joint Slot Filling and Intent Detection via Capsule Neural Networks",
            "authors": [
                "Chenwei Zhang",
                "Yaliang Li",
                "Nan Du",
                "Wei Fan",
                "Philip S. Yu"
            ],
            "abstract": "Being able to recognize words as slots and detect the intent of an utterance\nhas been a keen issue in natural language understanding. The existing works\neither treat slot filling and intent detection separately in a pipeline manner,\nor adopt joint models which sequentially label slots while summarizing the\nutterance-level intent without explicitly preserving the hierarchical\nrelationship among words, slots, and intents. To exploit the semantic hierarchy\nfor effective modeling, we propose a capsule-based neural network model which\naccomplishes slot filling and intent detection via a dynamic\nrouting-by-agreement schema. A re-routing schema is proposed to further\nsynergize the slot filling performance using the inferred intent\nrepresentation. Experiments on two real-world datasets show the effectiveness\nof our model when compared with other alternative model architectures, as well\nas existing natural language understanding services.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09516v1",
            "date": "2018-12-22 12:04:17+00:00",
            "title": "Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding",
            "authors": [
                "Changsen Yuan",
                "Heyan Huang",
                "Chong Feng",
                "Xiao Liu",
                "Xiaochi Wei"
            ],
            "abstract": "Distant supervision for relation extraction is an efficient method to reduce\nlabor costs and has been widely used to seek novel relational facts in large\ncorpora, which can be identified as a multi-instance multi-label problem.\nHowever, existing distant supervision methods suffer from selecting important\nwords in the sentence and extracting valid sentences in the bag. Towards this\nend, we propose a novel approach to address these problems in this paper.\nFirstly, we propose a linear attenuation simulation to reflect the importance\nof words in the sentence with respect to the distances between entities and\nwords. Secondly, we propose a non-independent and identically distributed\n(non-IID) relevance embedding to capture the relevance of sentences in the bag.\nOur method can not only capture complex information of words about hidden\nrelations, but also express the mutual information of instances in the bag.\nExtensive experiments on a benchmark dataset have well-validated the\neffectiveness of the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10037v1",
            "date": "2018-12-25 05:30:18+00:00",
            "title": "Building a Neural Semantic Parser from a Domain Ontology",
            "authors": [
                "Jianpeng Cheng",
                "Siva Reddy",
                "Mirella Lapata"
            ],
            "abstract": "Semantic parsing is the task of converting natural language utterances into\nmachine interpretable meaning representations which can be executed against a\nreal-world environment such as a database. Scaling semantic parsing to\narbitrary domains faces two interrelated challenges: obtaining broad coverage\ntraining data effectively and cheaply; and developing a model that generalizes\nto compositional utterances and complex intentions. We address these challenges\nwith a framework which allows to elicit training data from a domain ontology\nand bootstrap a neural parser which recursively builds derivations of logical\nforms. In our framework meaning representations are described by sequences of\nnatural language templates, where each template corresponds to a decomposed\nfragment of the underlying meaning representation. Although artificial,\ntemplates can be understood and paraphrased by humans to create natural\nutterances, resulting in parallel triples of utterances, meaning\nrepresentations, and their decompositions. These allow us to train a neural\nsemantic parser which learns to compose rules in deriving meaning\nrepresentations. We crowdsource training data on six domains, covering both\nsingle-turn utterances which exhibit rich compositionality, and sequential\nutterances where a complex task is procedurally performed in steps. We then\ndevelop neural semantic parsers which perform such compositional tasks. In\ngeneral, our approach allows to deploy neural semantic parsers quickly and\ncheaply from a given domain ontology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10281v1",
            "date": "2018-12-26 10:54:52+00:00",
            "title": "An Investigation of Supervised Learning Methods for Authorship Attribution in Short Hinglish Texts using Char & Word N-grams",
            "authors": [
                "Abhay Sharma",
                "Ananya Nandan",
                "Reetika Ralhan"
            ],
            "abstract": "The writing style of a person can be affirmed as a unique identity indicator;\nthe words used, and the structuring of the sentences are clear measures which\ncan identify the author of a specific work. Stylometry and its subset -\nAuthorship Attribution, have a long history beginning from the 19th century,\nand we can still find their use in modern times. The emergence of the Internet\nhas shifted the application of attribution studies towards non-standard texts\nthat are comparatively shorter to and different from the long texts on which\nmost research has been done. The aim of this paper focuses on the study of\nshort online texts, retrieved from messaging application called WhatsApp and\nstudying the distinctive features of a macaronic language (Hinglish), using\nsupervised learning methods and then comparing the models. Various features\nsuch as word n-gram and character n-gram are compared via methods viz., Naive\nBayes Classifier, Support Vector Machine, Conditional Tree, and Random Forest,\nto find the best discriminator for such corpora. Our results showed that SVM\nattained a test accuracy of up to 95.079% while similarly, Naive Bayes attained\nan accuracy of up to 94.455% for the dataset. Conditional Tree & Random Forest\nfailed to perform as well as expected. We also found that word unigram and\ncharacter 3-grams features were more likely to distinguish authors accurately\nthan other features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10356v1",
            "date": "2018-12-26 16:09:38+00:00",
            "title": "Quantized-Dialog Language Model for Goal-Oriented Conversational Systems",
            "authors": [
                "R. Chulaka Gunasekara",
                "David Nahamoo",
                "Lazaros C. Polymenakos",
                "Jatin Ganhotra",
                "Kshitij P. Fadnis"
            ],
            "abstract": "We propose a novel methodology to address dialog learning in the context of\ngoal-oriented conversational systems. The key idea is to quantize the dialog\nspace into clusters and create a language model across the clusters, thus\nallowing for an accurate choice of the next utterance in the conversation. The\nlanguage model relies on n-grams associated with clusters of utterances. This\nquantized-dialog language model methodology has been applied to the end-to-end\ngoal-oriented track of the latest Dialog System Technology Challenges (DSTC6).\nThe objective is to find the correct system utterance from a pool of candidates\nin order to complete a dialog between a user and an automated\nrestaurant-reservation system. Our results show that the technique proposed in\nthis paper achieves high accuracy regarding selection of the correct candidate\nutterance, and outperforms other state-of-the-art approaches based on neural\nnetworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10735v2",
            "date": "2018-12-27 14:49:02+00:00",
            "title": "CAN: Constrained Attention Networks for Multi-Aspect Sentiment Analysis",
            "authors": [
                "Mengting Hu",
                "Shiwan Zhao",
                "Li Zhang",
                "Keke Cai",
                "Zhong Su",
                "Renhong Cheng",
                "Xiaowei Shen"
            ],
            "abstract": "Aspect level sentiment classification is a fine-grained sentiment analysis\ntask. To detect the sentiment towards a particular aspect in a sentence,\nprevious studies have developed various attention-based methods for generating\naspect-specific sentence representations. However, the attention may inherently\nintroduce noise and downgrade the performance. In this paper, we propose\nconstrained attention networks (CAN), a simple yet effective solution, to\nregularize the attention for multi-aspect sentiment analysis, which alleviates\nthe drawback of the attention mechanism. Specifically, we introduce orthogonal\nregularization on multiple aspects and sparse regularization on each single\naspect. Experimental results on two public datasets demonstrate the\neffectiveness of our approach. We further extend our approach to multi-task\nsettings and outperform the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10860v5",
            "date": "2018-12-28 01:21:17+00:00",
            "title": "Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling",
            "authors": [
                "Alex Wang",
                "Jan Hula",
                "Patrick Xia",
                "Raghavendra Pappagari",
                "R. Thomas McCoy",
                "Roma Patel",
                "Najoung Kim",
                "Ian Tenney",
                "Yinghui Huang",
                "Katherin Yu",
                "Shuning Jin",
                "Berlin Chen",
                "Benjamin Van Durme",
                "Edouard Grave",
                "Ellie Pavlick",
                "Samuel R. Bowman"
            ],
            "abstract": "Natural language understanding has recently seen a surge of progress with the\nuse of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et\nal., 2019) which are pretrained on variants of language modeling. We conduct\nthe first large-scale systematic study of candidate pretraining tasks,\ncomparing 19 different tasks both as alternatives and complements to language\nmodeling. Our primary results support the use language modeling, especially\nwhen combined with pretraining on additional labeled-data tasks. However, our\nresults are mixed across pretraining tasks and show some concerning trends: In\nELMo's pretrain-then-freeze paradigm, random baselines are worryingly strong\nand results vary strikingly across target tasks. In addition, fine-tuning BERT\non an intermediate task often negatively impacts downstream transfer. In a more\npositive trend, we see modest gains from multitask training, suggesting the\ndevelopment of more sophisticated multitask and transfer learning techniques as\nan avenue for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10896v1",
            "date": "2018-12-28 05:35:31+00:00",
            "title": "Identifying Computer-Translated Paragraphs using Coherence Features",
            "authors": [
                "Hoang-Quoc Nguyen-Son",
                "Ngoc-Dung T. Tieu",
                "Huy H. Nguyen",
                "Junichi Yamagishi",
                "Isao Echizen"
            ],
            "abstract": "We have developed a method for extracting the coherence features from a\nparagraph by matching similar words in its sentences. We conducted an\nexperiment with a parallel German corpus containing 2000 human-created and 2000\nmachine-translated paragraphs. The result showed that our method achieved the\nbest performance (accuracy = 72.3%, equal error rate = 29.8%) when it is\ncompared with previous methods on various computer-generated text including\ntranslation and paper generation (best accuracy = 67.9%, equal error rate =\n32.0%). Experiments on Dutch, another rich resource language, and a low\nresource one (Japanese) attained similar performances. It demonstrated the\nefficiency of the coherence features at distinguishing computer-translated from\nhuman-created paragraphs on diverse languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10901v1",
            "date": "2018-12-28 06:15:53+00:00",
            "title": "Knowledge Representation Learning: A Quantitative Review",
            "authors": [
                "Yankai Lin",
                "Xu Han",
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Knowledge representation learning (KRL) aims to represent entities and\nrelations in knowledge graph in low-dimensional semantic space, which have been\nwidely used in massive knowledge-driven tasks. In this article, we introduce\nthe reader to the motivations for KRL, and overview existing approaches for\nKRL. Afterwards, we extensively conduct and quantitative comparison and\nanalysis of several typical KRL methods on three evaluation tasks of knowledge\nacquisition including knowledge graph completion, triple classification, and\nrelation extraction. We also review the real-world applications of KRL, such as\nlanguage modeling, question answering, information retrieval, and recommender\nsystems. Finally, we discuss the remaining challenges and outlook the future\ndirections for KRL. The codes and datasets used in the experiments can be found\nin https://github.com/thunlp/OpenKE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11459v3",
            "date": "2018-12-30 03:03:28+00:00",
            "title": "A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing",
            "authors": [
                "Dat Quoc Nguyen"
            ],
            "abstract": "We propose the first multi-task learning model for joint Vietnamese word\nsegmentation, part-of-speech (POS) tagging and dependency parsing. In\nparticular, our model extends the BIST graph-based dependency parser\n(Kiperwasser and Goldberg, 2016) with BiLSTM-CRF-based neural layers (Huang et\nal., 2015) for word segmentation and POS tagging. On Vietnamese benchmark\ndatasets, experimental results show that our joint model obtains\nstate-of-the-art or competitive performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11559v4",
            "date": "2018-12-30 15:36:11+00:00",
            "title": "Variational Self-attention Model for Sentence Representation",
            "authors": [
                "Qiang Zhang",
                "Shangsong Liang",
                "Emine Yilmaz"
            ],
            "abstract": "This paper proposes a variational self-attention model (VSAM) that employs\nvariational inference to derive self-attention. We model the self-attention\nvector as random variables by imposing a probabilistic distribution. The\nself-attention mechanism summarizes source information as an attention vector\nby weighted sum, where the weights are a learned probabilistic distribution.\nCompared with conventional deterministic counterpart, the stochastic units\nincorporated by VSAM allow multi-modal attention distributions. Furthermore, by\nmarginalizing over the latent variables, VSAM is more robust against\noverfitting. Experiments on the stance detection task demonstrate the\nsuperiority of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11587v1",
            "date": "2018-12-30 18:50:35+00:00",
            "title": "Sentiment Classification of Customer Reviews about Automobiles in Roman Urdu",
            "authors": [
                "Moin Khan",
                "Kamran Malik"
            ],
            "abstract": "Text mining is a broad field having sentiment mining as its important\nconstituent in which we try to deduce the behavior of people towards a specific\nitem, merchandise, politics, sports, social media comments, review sites etc.\nOut of many issues in sentiment mining, analysis and classification, one major\nissue is that the reviews and comments can be in different languages like\nEnglish, Arabic, Urdu etc. Handling each language according to its rules is a\ndifficult task. A lot of research work has been done in English Language for\nsentiment analysis and classification but limited sentiment analysis work is\nbeing carried out on other regional languages like Arabic, Urdu and Hindi. In\nthis paper, Waikato Environment for Knowledge Analysis (WEKA) is used as a\nplatform to execute different classification models for text classification of\nRoman Urdu text. Reviews dataset has been scrapped from different automobiles\nsites. These extracted Roman Urdu reviews, containing 1000 positive and 1000\nnegative reviews, are then saved in WEKA attribute-relation file format (arff)\nas labeled examples. Training is done on 80% of this data and rest of it is\nused for testing purpose which is done using different models and results are\nanalyzed in each case. The results show that Multinomial Naive Bayes\noutperformed Bagging, Deep Neural Network, Decision Tree, Random Forest,\nAdaBoost, k-NN and SVM Classifiers in terms of more accuracy, precision, recall\nand F-measure.",
            "journal ref": "Advances in Intelligent Systems and Computing, vol 887 (2018)\n  630-640",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11760v2",
            "date": "2018-12-31 11:01:02+00:00",
            "title": "Multilingual Constituency Parsing with Self-Attention and Pre-Training",
            "authors": [
                "Nikita Kitaev",
                "Steven Cao",
                "Dan Klein"
            ],
            "abstract": "We show that constituency parsing benefits from unsupervised pre-training\nacross a variety of languages and a range of pre-training conditions. We first\ncompare the benefits of no pre-training, fastText, ELMo, and BERT for English\nand find that BERT outperforms ELMo, in large part due to increased model\ncapacity, whereas ELMo in turn outperforms the non-contextual fastText\nembeddings. We also find that pre-training is beneficial across all 11\nlanguages tested; however, large model sizes (more than 100 million parameters)\nmake it computationally expensive to train separate models for each language.\nTo address this shortcoming, we show that joint multilingual pre-training and\nfine-tuning allows sharing all but a small number of parameters between ten\nlanguages in the final model. The 10x reduction in model size compared to\nfine-tuning one model per language causes only a 3.2% relative error increase\nin aggregate. We further explore the idea of joint fine-tuning and show that it\ngives low-resource languages a way to benefit from the larger datasets of other\nlanguages. Finally, we demonstrate new state-of-the-art results for 11\nlanguages, including English (95.8 F1) and Chinese (91.8 F1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11928v2",
            "date": "2018-12-31 18:10:42+00:00",
            "title": "Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units",
            "authors": [
                "Amit Das",
                "Jinyu Li",
                "Guoli Ye",
                "Rui Zhao",
                "Yifan Gong"
            ],
            "abstract": "The acoustic-to-word model based on the Connectionist Temporal Classification\n(CTC) criterion is a natural end-to-end (E2E) system directly targeting word as\noutput unit. Two issues exist in the system: first, the current output of the\nCTC model relies on the current input and does not account for context weighted\ninputs. This is the hard alignment issue. Second, the word-based CTC model\nsuffers from the out-of-vocabulary (OOV) issue. This means it can model only\nfrequently occurring words while tagging the remaining words as OOV. Hence,\nsuch a model is limited in its capacity in recognizing only a fixed set of\nfrequent words. In this study, we propose addressing these problems using a\ncombination of attention mechanism and mixed-units. In particular, we introduce\nAttention CTC, Self-Attention CTC, Hybrid CTC, and Mixed-unit CTC.\n  First, we blend attention modeling capabilities directly into the CTC network\nusing Attention CTC and Self-Attention CTC. Second, to alleviate the OOV issue,\nwe present Hybrid CTC which uses a word and letter CTC with shared hidden\nlayers. The Hybrid CTC consults the letter CTC when the word CTC emits an OOV.\nThen, we propose a much better solution by training a Mixed-unit CTC which\ndecomposes all the OOV words into sequences of frequent words and multi-letter\nunits. Evaluated on a 3400 hours Microsoft Cortana voice assistant task, our\nfinal acoustic-to-word solution using attention and mixed-units achieves a\nrelative reduction in word error rate (WER) over the vanilla word CTC by\n12.09\\%. Such an E2E model without using any language model (LM) or complex\ndecoder also outperforms a traditional context-dependent (CD) phoneme CTC with\nstrong LM and decoder by 6.79% relative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00521v1",
            "date": "2018-12-31 15:00:38+00:00",
            "title": "Types, Tokens, and Hapaxes: A New Heap's Law",
            "authors": [
                "Victor Davis"
            ],
            "abstract": "Heap's Law states that in a large enough text corpus, the number of types as\na function of tokens grows as $N=KM^\\beta$ for some free parameters $K,\\beta$.\nMuch has been written about how this result and various generalizations can be\nderived from Zipf's Law. Here we derive from first principles a completely\nnovel expression of the type-token curve and prove its superior accuracy on\nreal text. This expression naturally generalizes to equally accurate estimates\nfor counting hapaxes and higher $n$-legomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01122v1",
            "date": "2018-12-28 19:04:36+00:00",
            "title": "Machine Translation: A Literature Review",
            "authors": [
                "Ankush Garg",
                "Mayank Agarwal"
            ],
            "abstract": "Machine translation (MT) plays an important role in benefiting linguists,\nsociologists, computer scientists, etc. by processing natural language to\ntranslate it into some other natural language. And this demand has grown\nexponentially over past couple of years, considering the enormous exchange of\ninformation between different regions with different regional languages.\nMachine Translation poses numerous challenges, some of which are: a) Not all\nwords in one language has equivalent word in another language b) Two given\nlanguages may have completely different structures c) Words can have more than\none meaning. Owing to these challenges, along with many others, MT has been\nactive area of research for more than five decades. Numerous methods have been\nproposed in the past which either aim at improving the quality of the\ntranslations generated by them, or study the robustness of these systems by\nmeasuring their performance on many different languages. In this literature\nreview, we discuss statistical approaches (in particular word-based and\nphrase-based) and neural approaches which have gained widespread prominence\nowing to their state-of-the-art results across multiple major languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04899v1",
            "date": "2018-12-14 00:43:58+00:00",
            "title": "Conversational Intent Understanding for Passengers in Autonomous Vehicles",
            "authors": [
                "Eda Okur",
                "Shachi H Kumar",
                "Saurav Sahay",
                "Asli Arslan Esme",
                "Lama Nachman"
            ],
            "abstract": "Understanding passenger intents and extracting relevant slots are important\nbuilding blocks towards developing a contextual dialogue system responsible for\nhandling certain vehicle-passenger interactions in autonomous vehicles (AV).\nWhen the passengers give instructions to AMIE (Automated-vehicle Multimodal\nIn-cabin Experience), the agent should parse such commands properly and trigger\nthe appropriate functionality of the AV system. In our AMIE scenarios, we\ndescribe usages and support various natural commands for interacting with the\nvehicle. We collected a multimodal in-cabin data-set with multi-turn dialogues\nbetween the passengers and AMIE using a Wizard-of-Oz scheme. We explored\nvarious recent Recurrent Neural Networks (RNN) based techniques and built our\nown hierarchical models to recognize passenger intents along with relevant\nslots associated with the action to be performed in AV scenarios. Our\nexperimental results achieved F1-score of 0.91 on utterance-level intent\nrecognition and 0.96 on slot extraction models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00453v1",
            "date": "2018-01-01 14:56:56+00:00",
            "title": "Automated rating of recorded classroom presentations using speech analysis in kazakh",
            "authors": [
                "Akzharkyn Izbassarova",
                "Aidana Irmanova",
                "A. P. James"
            ],
            "abstract": "Effective presentation skills can help to succeed in business, career and\nacademy. This paper presents the design of speech assessment during the oral\npresentation and the algorithm for speech evaluation based on criteria of\noptimal intonation. As the pace of the speech and its optimal intonation varies\nfrom language to language, developing an automatic identification of language\nduring the presentation is required. Proposed algorithm was tested with\npresentations delivered in Kazakh language. For testing purposes the features\nof Kazakh phonemes were extracted using MFCC and PLP methods and created a\nHidden Markov Model (HMM) [5], [5] of Kazakh phonemes. Kazakh vowel formants\nwere defined and the correlation between the deviation rate in fundamental\nfrequency and the liveliness of the speech to evaluate intonation of the\npresentation was analyzed. It was established that the threshold value between\nmonotone and dynamic speech is 0.16 and the error for intonation evaluation is\n19%.",
            "journal ref": "2017 International Conference on Advances in Computing,\n  Communications and Informatics (ICACCI), Udupi, 2017, pp. 393-397",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00554v1",
            "date": "2018-01-02 05:24:30+00:00",
            "title": "Did you hear that? Adversarial Examples Against Automatic Speech Recognition",
            "authors": [
                "Moustafa Alzantot",
                "Bharathan Balaji",
                "Mani Srivastava"
            ],
            "abstract": "Speech is a common and effective way of communication between humans, and\nmodern consumer devices such as smartphones and home hubs are equipped with\ndeep learning based accurate automatic speech recognition to enable natural\ninteraction between humans and machines. Recently, researchers have\ndemonstrated powerful attacks against machine learning models that can fool\nthem to produceincorrect results. However, nearly all previous research in\nadversarial attacks has focused on image recognition and object detection\nmodels. In this short paper, we present a first of its kind demonstration of\nadversarial attacks against speech classification model. Our algorithm performs\ntargeted attacks with 87% success by adding small background noise without\nhaving to know the underlying model parameter and architecture. Our attack only\nchanges the least significant bits of a subset of audio clip samples, and the\nnoise does not change 89% the human listener's perception of the audio clip as\nevaluated in our human study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00644v7",
            "date": "2018-01-02 13:47:43+00:00",
            "title": "Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality",
            "authors": [
                "Reagan Mozer",
                "Luke Miratrix",
                "Aaron Russell Kaufman",
                "L. Jason Anastasopoulos"
            ],
            "abstract": "Matching for causal inference is a well-studied problem, but standard methods\nfail when the units to match are text documents: the high-dimensional and rich\nnature of the data renders exact matching infeasible, causes propensity scores\nto produce incomparable matches, and makes assessing match quality difficult.\nIn this paper, we characterize a framework for matching text documents that\ndecomposes existing methods into: (1) the choice of text representation, and\n(2) the choice of distance metric. We investigate how different choices within\nthis framework affect both the quantity and quality of matches identified\nthrough a systematic multifactor evaluation experiment using human subjects.\nAltogether we evaluate over 100 unique text matching methods along with 5\ncomparison methods taken from the literature. Our experimental results identify\nmethods that generate matches with higher subjective match quality than current\nstate-of-the-art techniques. We enhance the precision of these results by\ndeveloping a predictive model to estimate the match quality of pairs of text\ndocuments as a function of our various distance scores. This model, which we\nfind successfully mimics human judgment, also allows for approximate and\nunsupervised evaluation of new procedures. We then employ the identified best\nmethod to illustrate the utility of text matching in two applications. First,\nwe engage with a substantive debate in the study of media bias by using text\nmatching to control for topic selection when comparing news articles from\nthirteen news sources. We then show how conditioning on text data leads to more\nprecise causal inferences in an observational study examining the effects of a\nmedical intervention.",
            "journal ref": "None",
            "category": "stat.ME"
        },
        {
            "url": "http://arxiv.org/abs/1801.00984v2",
            "date": "2018-01-03 13:02:25+00:00",
            "title": "Sentence Object Notation: Multilingual sentence notation based on Wordnet",
            "authors": [
                "Abdelkrime Aries",
                "Djamel Eddine Zegour",
                "Walid Khaled Hidouci"
            ],
            "abstract": "The representation of sentences is a very important task. It can be used as a\nway to exchange data inter-applications. One main characteristic, that a\nnotation must have, is a minimal size and a representative form. This can\nreduce the transfer time, and hopefully the processing time as well.\n  Usually, sentence representation is associated to the processed language. The\ngrammar of this language affects how we represent the sentence. To avoid\nlanguage-dependent notations, we have to come up with a new representation\nwhich don't use words, but their meanings. This can be done using a lexicon\nlike wordnet, instead of words we use their synsets. As for syntactic\nrelations, they have to be universal as much as possible.\n  Our new notation is called STON \"SenTences Object Notation\", which somehow\nhas similarities to JSON. It is meant to be minimal, representative and\nlanguage-independent syntactic representation. Also, we want it to be readable\nand easy to be created. This simplifies developing simple automatic generators\nand creating test banks manually. Its benefit is to be used as a medium between\ndifferent parts of applications like: text summarization, language translation,\netc. The notation is based on 4 languages: Arabic, English, Franch and\nJapanese; and there are some cases where these languages don't agree on one\nrepresentation. Also, given the diversity of grammatical structure of different\nworld languages, this annotation may fail for some languages which allows more\nfuture improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01531v1",
            "date": "2018-01-04 19:58:46+00:00",
            "title": "Slugbot: An Application of a Novel and Scalable Open Domain Socialbot Framework",
            "authors": [
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Shereen Oraby",
                "Amita Misra",
                "Marilyn Walker"
            ],
            "abstract": "In this paper we introduce a novel, open domain socialbot for the Amazon\nAlexa Prize competition, aimed at carrying on friendly conversations with users\non a variety of topics. We present our modular system, highlighting our\ndifferent data sources and how we use the human mind as a model for data\nmanagement. Additionally we build and employ natural language understanding and\ninformation retrieval tools and APIs to expand our knowledge bases. We describe\nour semistructured, scalable framework for crafting topic-specific dialogue\nflows, and give details on our dialogue management schemes and scoring\nmechanisms. Finally we briefly evaluate the performance of our system and\nobserve the challenges that an open domain socialbot faces.",
            "journal ref": "Alexa Prize Proceedings 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01641v2",
            "date": "2018-01-05 06:06:17+00:00",
            "title": "aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model",
            "authors": [
                "Liu Yang",
                "Qingyao Ai",
                "Jiafeng Guo",
                "W. Bruce Croft"
            ],
            "abstract": "As an alternative to question answering methods based on feature engineering,\ndeep learning approaches such as convolutional neural networks (CNNs) and Long\nShort-Term Memory Models (LSTMs) have recently been proposed for semantic\nmatching of questions and answers. To achieve good results, however, these\nmodels have been combined with additional features such as word overlap or BM25\nscores. Without this combination, these models perform significantly worse than\nmethods based on linguistic feature engineering. In this paper, we propose an\nattention based neural matching model for ranking short answer text. We adopt\nvalue-shared weighting scheme instead of position-shared weighting scheme for\ncombining different matching signals and incorporate question term importance\nlearning using question attention network. Using the popular benchmark TREC QA\ndata, we show that the relatively simple aNMM model can significantly\noutperform other neural network models that have been used for the question\nanswering task, and is competitive with models that are combined with\nadditional features. When aNMM is combined with additional features, it\noutperforms all baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.01768v1",
            "date": "2018-01-05 14:36:31+00:00",
            "title": "Learning Feature Representations for Keyphrase Extraction",
            "authors": [
                "Corina Florescu",
                "Wei Jin"
            ],
            "abstract": "In supervised approaches for keyphrase extraction, a candidate phrase is\nencoded with a set of hand-crafted features and machine learning algorithms are\ntrained to discriminate keyphrases from non-keyphrases. Although the\nmanually-designed features have shown to work well in practice, feature\nengineering is a difficult process that requires expert knowledge and normally\ndoes not generalize well. In this paper, we present SurfKE, a feature learning\nframework that exploits the text itself to automatically discover patterns that\nkeyphrases exhibit. Our model represents the document as a graph and\nautomatically learns feature representation of phrases. The proposed model\nobtains remarkable improvements in performance over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01825v1",
            "date": "2018-01-05 16:38:05+00:00",
            "title": "Towards Understanding and Answering Multi-Sentence Recommendation Questions on Tourism",
            "authors": [
                "Danish Contractor",
                "Barun Patra",
                "Mausam Singla",
                "Parag Singla"
            ],
            "abstract": "We introduce the first system towards the novel task of answering complex\nmultisentence recommendation questions in the tourism domain. Our solution uses\na pipeline of two modules: question understanding and answering. For question\nunderstanding, we define an SQL-like query language that captures the semantic\nintent of a question; it supports operators like subset, negation, preference\nand similarity, which are often found in recommendation questions. We train and\ncompare traditional CRFs as well as bidirectional LSTM-based models for\nconverting a question to its semantic representation. We extend these models to\na semisupervised setting with partially labeled sequences gathered through\ncrowdsourcing. We find that our best model performs semi-supervised training of\nBiDiLSTM+CRF with hand-designed features and CCM(Chang et al., 2007)\nconstraints. Finally, in an end to end QA system, our answering component\nconverts our question representation into queries fired on underlying knowledge\nsources. Our experiments on two different answer corpora demonstrate that our\nsystem can significantly outperform baselines with up to 20 pt higher accuracy\nand 17 pt higher recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01828v1",
            "date": "2018-01-05 16:45:59+00:00",
            "title": "Shielding Google's language toxicity model against adversarial attacks",
            "authors": [
                "Nestor Rodriguez",
                "Sergio Rojas-Galeano"
            ],
            "abstract": "Lack of moderation in online communities enables participants to incur in\npersonal aggression, harassment or cyberbullying, issues that have been\naccentuated by extremist radicalisation in the contemporary post-truth politics\nscenario. This kind of hostility is usually expressed by means of toxic\nlanguage, profanity or abusive statements. Recently Google has developed a\nmachine-learning-based toxicity model in an attempt to assess the hostility of\na comment; unfortunately, it has been suggested that said model can be deceived\nby adversarial attacks that manipulate the text sequence of the comment. In\nthis paper we firstly characterise such adversarial attacks as using\nobfuscation and polarity transformations. The former deceives by corrupting\ntoxic trigger content with typographic edits, whereas the latter deceives by\ngrammatical negation of the toxic content. Then, we propose a two--stage\napproach to counter--attack these anomalies, bulding upon a recently proposed\ntext deobfuscation method and the toxicity scoring model. Lastly, we conducted\nan experiment with approximately 24000 distorted comments, showing how in this\nway it is feasible to restore toxicity of the adversarial variants, while\nincurring roughly on a twofold increase in processing time. Even though novel\nadversary challenges would keep coming up derived from the versatile nature of\nwritten language, we anticipate that techniques combining machine learning and\ntext pattern recognition methods, each one targeting different layers of\nlinguistic features, would be needed to achieve robust detection of toxic\nlanguage, thus fostering aggression--free digital interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01884v2",
            "date": "2018-01-05 19:00:04+00:00",
            "title": "Unsupervised Low-Dimensional Vector Representations for Words, Phrases and Text that are Transparent, Scalable, and produce Similarity Metrics that are Complementary to Neural Embeddings",
            "authors": [
                "Neil R. Smalheiser",
                "Gary Bonifield"
            ],
            "abstract": "Neural embeddings are a popular set of methods for representing words,\nphrases or text as a low dimensional vector (typically 50-500 dimensions).\nHowever, it is difficult to interpret these dimensions in a meaningful manner,\nand creating neural embeddings requires extensive training and tuning of\nmultiple parameters and hyperparameters. We present here a simple unsupervised\nmethod for representing words, phrases or text as a low dimensional vector, in\nwhich the meaning and relative importance of dimensions is transparent to\ninspection. We have created a near-comprehensive vector representation of\nwords, and selected bigrams, trigrams and abbreviations, using the set of\ntitles and abstracts in PubMed as a corpus. This vector is used to create\nseveral novel implicit word-word and text-text similarity metrics. The implicit\nword-word similarity metrics correlate well with human judgement of word pair\nsimilarity and relatedness, and outperform or equal all other reported methods\non a variety of biomedical benchmarks, including several implementations of\nneural embeddings trained on PubMed corpora. Our implicit word-word metrics\ncapture different aspects of word-word relatedness than word2vec-based metrics\nand are only partially correlated (rho = ~0.5-0.8 depending on task and\ncorpus). The vector representations of words, bigrams, trigrams, abbreviations,\nand PubMed title+abstracts are all publicly available from\nhttp://arrowsmith.psych.uic.edu for release under CC-BY-NC license. Several\npublic web query interfaces are also available at the same site, including one\nwhich allows the user to specify a given word and view its most closely related\nterms according to direct co-occurrence as well as different implicit\nsimilarity metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01900v1",
            "date": "2018-01-05 19:20:24+00:00",
            "title": "Knowledge-based Word Sense Disambiguation using Topic Models",
            "authors": [
                "Devendra Singh Chaplot",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Word Sense Disambiguation is an open problem in Natural Language Processing\nwhich is particularly challenging and useful in the unsupervised setting where\nall the words in any given text need to be disambiguated without using any\nlabeled data. Typically WSD systems use the sentence or a small window of words\naround the target word as the context for disambiguation because their\ncomputational complexity scales exponentially with the size of the context. In\nthis paper, we leverage the formalism of topic model to design a WSD system\nthat scales linearly with the number of words in the context. As a result, our\nsystem is able to utilize the whole document as the context for a word to be\ndisambiguated. The proposed method is a variant of Latent Dirichlet Allocation\nin which the topic proportions for a document are replaced by synset\nproportions. We further utilize the information in the WordNet by assigning a\nnon-uniform prior to synset distribution over words and a logistic-normal prior\nfor document distribution over synsets. We evaluate the proposed method on\nSenseval-2, Senseval-3, SemEval-2007, SemEval-2013 and SemEval-2015 English\nAll-Word WSD datasets and show that it outperforms the state-of-the-art\nunsupervised knowledge-based WSD system by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01967v3",
            "date": "2018-01-06 04:58:38+00:00",
            "title": "Visual Text Correction",
            "authors": [
                "Amir Mazaheri",
                "Mubarak Shah"
            ],
            "abstract": "Videos, images, and sentences are mediums that can express the same\nsemantics. One can imagine a picture by reading a sentence or can describe a\nscene with some words. However, even small changes in a sentence can cause a\nsignificant semantic inconsistency with the corresponding video/image. For\nexample, by changing the verb of a sentence, the meaning may drastically\nchange. There have been many efforts to encode a video/sentence and decode it\nas a sentence/video. In this research, we study a new scenario in which both\nthe sentence and the video are given, but the sentence is inaccurate. A\nsemantic inconsistency between the sentence and the video or between the words\nof a sentence can result in an inaccurate description. This paper introduces a\nnew problem, called Visual Text Correction (VTC), i.e., finding and replacing\nan inaccurate word in the textual description of a video. We propose a deep\nnetwork that can simultaneously detect an inaccuracy in a sentence, and fix it\nby replacing the inaccurate word(s). Our method leverages the semantic\ninterdependence of videos and words, as well as the short-term and long-term\nrelations of the words in a sentence. In our formulation, part of a visual\nfeature vector for every single word is dynamically selected through a gating\nprocess. Furthermore, to train and evaluate our model, we propose an approach\nto automatically construct a large dataset for VTC problem. Our experiments and\nperformance analysis demonstrates that the proposed method provides very good\nresults and also highlights the general challenges in solving the VTC problem.\nTo the best of our knowledge, this work is the first of its kind for the Visual\nText Correction task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1801.03339v2",
            "date": "2018-01-10 12:24:34+00:00",
            "title": "Fooling End-to-end Speaker Verification by Adversarial Examples",
            "authors": [
                "Felix Kreuk",
                "Yossi Adi",
                "Moustapha Cisse",
                "Joseph Keshet"
            ],
            "abstract": "Automatic speaker verification systems are increasingly used as the primary\nmeans to authenticate costumers. Recently, it has been proposed to train\nspeaker verification systems using end-to-end deep neural models. In this\npaper, we show that such systems are vulnerable to adversarial example attack.\nAdversarial examples are generated by adding a peculiar noise to original\nspeaker examples, in such a way that they are almost indistinguishable from the\noriginal examples by a human listener. Yet, the generated waveforms, which\nsound as speaker A can be used to fool such a system by claiming as if the\nwaveforms were uttered by speaker B. We present white-box attacks on an\nend-to-end deep network that was either trained on YOHO or NTIMIT. We also\npresent two black-box attacks: where the adversarial examples were generated\nwith a system that was trained on YOHO, but the attack is on a system that was\ntrained on NTIMIT; and when the adversarial examples were generated with a\nsystem that was trained on Mel-spectrum feature set, but the attack is on a\nsystem that was trained on MFCC. Results suggest that the accuracy of the\nattacked system was decreased and the false-positive rate was dramatically\nincreased.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.03825v4",
            "date": "2018-01-11 15:40:31+00:00",
            "title": "EARL: Joint Entity and Relation Linking for Question Answering over Knowledge Graphs",
            "authors": [
                "Mohnish Dubey",
                "Debayan Banerjee",
                "Debanjan Chaudhuri",
                "Jens Lehmann"
            ],
            "abstract": "Many question answering systems over knowledge graphs rely on entity and\nrelation linking components in order to connect the natural language input to\nthe underlying knowledge graph. Traditionally, entity linking and relation\nlinking have been performed either as dependent sequential tasks or as\nindependent parallel tasks. In this paper, we propose a framework called EARL,\nwhich performs entity linking and relation linking as a joint task. EARL\nimplements two different solution strategies for which we provide a comparative\nanalysis in this paper: The first strategy is a formalisation of the joint\nentity and relation linking tasks as an instance of the Generalised Travelling\nSalesman Problem (GTSP). In order to be computationally feasible, we employ\napproximate GTSP solvers. The second strategy uses machine learning in order to\nexploit the connection density between nodes in the knowledge graph. It relies\non three base features and re-ranking steps in order to predict entities and\nrelations. We compare the strategies and evaluate them on a dataset with 5000\nquestions. Both strategies significantly outperform the current\nstate-of-the-art approaches for entity and relation linking.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1801.04871v1",
            "date": "2018-01-15 16:45:56+00:00",
            "title": "Building a Conversational Agent Overnight with Dialogue Self-Play",
            "authors": [
                "Pararth Shah",
                "Dilek Hakkani-T\u00fcr",
                "Gokhan T\u00fcr",
                "Abhinav Rastogi",
                "Ankur Bapna",
                "Neha Nayak",
                "Larry Heck"
            ],
            "abstract": "We propose Machines Talking To Machines (M2M), a framework combining\nautomation and crowdsourcing to rapidly bootstrap end-to-end dialogue agents\nfor goal-oriented dialogues in arbitrary domains. M2M scales to new tasks with\njust a task schema and an API client from the dialogue system developer, but it\nis also customizable to cater to task-specific interactions. Compared to the\nWizard-of-Oz approach for data collection, M2M achieves greater diversity and\ncoverage of salient dialogue flows while maintaining the naturalness of\nindividual utterances. In the first phase, a simulated user bot and a\ndomain-agnostic system bot converse to exhaustively generate dialogue\n\"outlines\", i.e. sequences of template utterances and their semantic parses. In\nthe second phase, crowd workers provide contextual rewrites of the dialogues to\nmake the utterances more natural while preserving their meaning. The entire\nprocess can finish within a few hours. We propose a new corpus of 3,000\ndialogues spanning 2 domains collected with M2M, and present comparisons with\npopular dialogue datasets on the quality and diversity of the surface forms and\ndialogue flows.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1801.05032v1",
            "date": "2018-01-12 12:11:30+00:00",
            "title": "AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience",
            "authors": [
                "Feng-Lin Li",
                "Minghui Qiu",
                "Haiqing Chen",
                "Xiongwei Wang",
                "Xing Gao",
                "Jun Huang",
                "Juwei Ren",
                "Zhongzhou Zhao",
                "Weipeng Zhao",
                "Lei Wang",
                "Guwei Jin",
                "Wei Chu"
            ],
            "abstract": "We present AliMe Assist, an intelligent assistant designed for creating an\ninnovative online shopping experience in E-commerce. Based on question\nanswering (QA), AliMe Assist offers assistance service, customer service, and\nchatting service. It is able to take voice and text input, incorporate context\nto QA, and support multi-round interaction. Currently, it serves millions of\ncustomer questions per day and is able to address 85% of them. In this paper,\nwe demonstrate the system, present the underlying techniques, and share our\nexperience in dealing with real-world QA in the E-commerce field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05420v2",
            "date": "2018-01-16 03:19:37+00:00",
            "title": "A Comparative Study of Rule Extraction for Recurrent Neural Networks",
            "authors": [
                "Qinglong Wang",
                "Kaixuan Zhang",
                "Alexander G. Ororbia II",
                "Xinyu Xing",
                "Xue Liu",
                "C. Lee Giles"
            ],
            "abstract": "Understanding recurrent networks through rule extraction has a long history.\nThis has taken on new interests due to the need for interpreting or verifying\nneural networks. One basic form for representing stateful rules is\ndeterministic finite automata (DFA). Previous research shows that extracting\nDFAs from trained second-order recurrent networks is not only possible but also\nrelatively stable. Recently, several new types of recurrent networks with more\ncomplicated architectures have been introduced. These handle challenging\nlearning tasks usually involving sequential data. However, it remains an open\nproblem whether DFAs can be adequately extracted from these models.\nSpecifically, it is not clear how DFA extraction will be affected when applied\nto different recurrent networks trained on data sets with different levels of\ncomplexity. Here, we investigate DFA extraction on several widely adopted\nrecurrent networks that are trained to learn a set of seven regular Tomita\ngrammars. We first formally analyze the complexity of Tomita grammars and\ncategorize these grammars according to that complexity. Then we empirically\nevaluate different recurrent networks for their performance of DFA extraction\non all Tomita grammars. Our experiments show that for most recurrent networks,\ntheir extraction performance decreases as the complexity of the underlying\ngrammar increases. On grammars of lower complexity, most recurrent networks\nobtain desirable extraction performance. As for grammars with the highest level\nof complexity, while several complicated models fail with only certain\nrecurrent networks having satisfactory extraction performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.05613v2",
            "date": "2018-01-17 10:21:49+00:00",
            "title": "Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics",
            "authors": [
                "Shrainik Jain",
                "Bill Howe",
                "Jiaqi Yan",
                "Thierry Cruanes"
            ],
            "abstract": "We consider methods for learning vector representations of SQL queries to\nsupport generalized workload analytics tasks, including workload summarization\nfor index selection and predicting queries that will trigger memory errors. We\nconsider vector representations of both raw SQL text and optimized query plans,\nand evaluate these methods on synthetic and real SQL workloads. We find that\ngeneral algorithms based on vector representations can outperform existing\napproaches that rely on specialized features. For index recommendation, we\ncluster the vector representations to compress large workloads with no loss in\nperformance from the recommended index. For error prediction, we train a\nclassifier over learned vectors that can automatically relate subtle syntactic\npatterns with specific errors raised during query execution. Surprisingly, we\nalso find that these methods enable transfer learning, where a model trained on\none SQL corpus can be applied to an unrelated corpus and still enable good\nperformance. We find that these general approaches, when trained on a large\ncorpus of SQL queries, provides a robust foundation for a variety of workload\nanalysis tasks and database features, without requiring application-specific\nfeature engineering.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1801.06126v3",
            "date": "2018-01-18 16:59:19+00:00",
            "title": "Non-Adversarial Unsupervised Word Translation",
            "authors": [
                "Yedid Hoshen",
                "Lior Wolf"
            ],
            "abstract": "Unsupervised word translation from non-parallel inter-lingual corpora has\nattracted much research interest. Very recently, neural network methods trained\nwith adversarial loss functions achieved high accuracy on this task. Despite\nthe impressive success of the recent techniques, they suffer from the typical\ndrawbacks of generative adversarial models: sensitivity to hyper-parameters,\nlong training time and lack of interpretability. In this paper, we make the\nobservation that two sufficiently similar distributions can be aligned\ncorrectly with iterative matching methods. We present a novel method that first\naligns the second moment of the word distributions of the two languages and\nthen iteratively refines the alignment. Extensive experiments on word\ntranslation of European and Non-European languages show that our method\nachieves better performance than recent state-of-the-art deep adversarial\napproaches and is competitive with the supervised baseline. It is also\nefficient, easy to parallelize on CPU and interpretable.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.06353v4",
            "date": "2018-01-19 10:16:11+00:00",
            "title": "Transfer Learning for Improving Speech Emotion Classification Accuracy",
            "authors": [
                "Siddique Latif",
                "Rajib Rana",
                "Shahzad Younis",
                "Junaid Qadir",
                "Julien Epps"
            ],
            "abstract": "The majority of existing speech emotion recognition research focuses on\nautomatic emotion detection using training and testing data from same corpus\ncollected under the same conditions. The performance of such systems has been\nshown to drop significantly in cross-corpus and cross-language scenarios. To\naddress the problem, this paper exploits a transfer learning technique to\nimprove the performance of speech emotion recognition systems that is novel in\ncross-language and cross-corpus scenarios. Evaluations on five different\ncorpora in three different languages show that Deep Belief Networks (DBNs)\noffer better accuracy than previous approaches on cross-corpus emotion\nrecognition, relative to a Sparse Autoencoder and SVM baseline system. Results\nalso suggest that using a large number of languages for training and using a\nsmall fraction of the target data in training can significantly boost accuracy\ncompared with baseline also for the corpus with limited training examples.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1801.07243v5",
            "date": "2018-01-22 18:58:18+00:00",
            "title": "Personalizing Dialogue Agents: I have a dog, do you have pets too?",
            "authors": [
                "Saizheng Zhang",
                "Emily Dinan",
                "Jack Urbanek",
                "Arthur Szlam",
                "Douwe Kiela",
                "Jason Weston"
            ],
            "abstract": "Chit-chat models are known to have several problems: they lack specificity,\ndo not display a consistent personality and are often not very captivating. In\nthis work we present the task of making chit-chat more engaging by conditioning\non profile information. We collect data and train models to (i) condition on\ntheir given profile information; and (ii) information about the person they are\ntalking to, resulting in improved dialogues, as measured by next utterance\nprediction. Since (ii) is initially unknown our model is trained to engage its\npartner with personal topics, and we show the resulting dialogue can be used to\npredict profile information about the interlocutors.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1801.07311v3",
            "date": "2018-01-22 20:41:50+00:00",
            "title": "Early Detection of Social Media Hoaxes at Scale",
            "authors": [
                "Arkaitz Zubiaga",
                "Aiqi Jiang"
            ],
            "abstract": "The unmoderated nature of social media enables the diffusion of hoaxes, which\nin turn jeopardises the credibility of information gathered from social media\nplatforms. Existing research on automated detection of hoaxes has the\nlimitation of using relatively small datasets, owing to the difficulty of\ngetting labelled data. This in turn has limited research exploring early\ndetection of hoaxes as well as exploring other factors such as the effect of\nthe size of the training data or the use of sliding windows. To mitigate this\nproblem, we introduce a semi-automated method that leverages the Wikidata\nknowledge base to build large-scale datasets for veracity classification,\nfocusing on celebrity death reports. This enables us to create a dataset with\n4,007 reports including over 13 million tweets, 15% of which are fake.\nExperiments using class-specific representations of word embeddings show that\nwe can achieve F1 scores nearing 72% within 10 minutes of the first tweet being\nposted when we expand the size of the training data following our\nsemi-automated means. Our dataset represents a realistic scenario with a real\ndistribution of true, commemorative and false stories, which we release for\nfurther use as a benchmark in future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07495v3",
            "date": "2018-01-23 11:43:54+00:00",
            "title": "The Enemy Among Us: Detecting Hate Speech with Threats Based 'Othering' Language Embeddings",
            "authors": [
                "Wafa Alorainy",
                "Pete Burnap",
                "Han Liu",
                "Matthew Williams"
            ],
            "abstract": "Offensive or antagonistic language targeted at individuals and social groups\nbased on their personal characteristics (also known as cyber hate speech or\ncyberhate) has been frequently posted and widely circulated viathe World Wide\nWeb. This can be considered as a key risk factor for individual and societal\ntension linked toregional instability. Automated Web-based cyberhate detection\nis important for observing and understandingcommunity and regional societal\ntension - especially in online social networks where posts can be rapidlyand\nwidely viewed and disseminated. While previous work has involved using\nlexicons, bags-of-words orprobabilistic language parsing approaches, they often\nsuffer from a similar issue which is that cyberhate can besubtle and indirect -\nthus depending on the occurrence of individual words or phrases can lead to a\nsignificantnumber of false negatives, providing inaccurate representation of\nthe trends in cyberhate. This problemmotivated us to challenge thinking around\nthe representation of subtle language use, such as references toperceived\nthreats from \"the other\" including immigration or job prosperity in a hateful\ncontext. We propose anovel framework that utilises language use around the\nconcept of \"othering\" and intergroup threat theory toidentify these subtleties\nand we implement a novel classification method using embedding learning to\ncomputesemantic distances between parts of speech considered to be part of an\n\"othering\" narrative. To validate ourapproach we conduct several experiments on\ndifferent types of cyberhate, namely religion, disability, race andsexual\norientation, with F-measure scores for classifying hateful instances obtained\nthrough applying ourmodel of 0.93, 0.86, 0.97 and 0.98 respectively, providing\na significant improvement in classifier accuracy overthe state-of-the-art",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07537v1",
            "date": "2018-01-23 13:50:11+00:00",
            "title": "Analyzing Language Learned by an Active Question Answering Agent",
            "authors": [
                "Christian Buck",
                "Jannis Bulian",
                "Massimiliano Ciaramita",
                "Wojciech Gajewski",
                "Andrea Gesmundo",
                "Neil Houlsby",
                "Wei Wang"
            ],
            "abstract": "We analyze the language learned by an agent trained with reinforcement\nlearning as a component of the ActiveQA system [Buck et al., 2017]. In\nActiveQA, question answering is framed as a reinforcement learning task in\nwhich an agent sits between the user and a black box question-answering system.\nThe agent learns to reformulate the user's questions to elicit the optimal\nanswers. It probes the system with many versions of a question that are\ngenerated via a sequence-to-sequence question reformulation model, then\naggregates the returned evidence to find the best answer. This process is an\ninstance of \\emph{machine-machine} communication. The question reformulation\nmodel must adapt its language to increase the quality of the answers returned,\nmatching the language of the question answering system. We find that the agent\ndoes not learn transformations that align with semantic intuitions but\ndiscovers through learning classical information retrieval techniques such as\ntf-idf re-weighting and stemming.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07779v1",
            "date": "2018-01-23 21:40:53+00:00",
            "title": "The WiLI benchmark dataset for written language identification",
            "authors": [
                "Martin Thoma"
            ],
            "abstract": "This paper describes the WiLI-2018 benchmark dataset for monolingual written\nnatural language identification. WiLI-2018 is a publicly available, free of\ncharge dataset of short text extracts from Wikipedia. It contains 1000\nparagraphs of 235 languages, totaling in 23500 paragraphs. WiLI is a\nclassification dataset: Given an unknown paragraph written in one dominant\nlanguage, it has to be decided which language it is.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1801.08660v1",
            "date": "2018-01-26 02:50:03+00:00",
            "title": "Context Models for OOV Word Translation in Low-Resource Languages",
            "authors": [
                "Angli Liu",
                "Katrin Kirchhoff"
            ],
            "abstract": "Out-of-vocabulary word translation is a major problem for the translation of\nlow-resource languages that suffer from a lack of parallel training data. This\npaper evaluates the contributions of target-language context models towards the\ntranslation of OOV words, specifically in those cases where OOV translations\nare derived from external knowledge sources, such as dictionaries. We develop\nboth neural and non-neural context models and evaluate them within both\nphrase-based and self-attention based neural machine translation systems. Our\nresults show that neural language models that integrate additional context\nbeyond the current sentence are the most effective in disambiguating possible\nOOV word translations. We present an efficient second-pass lattice-rescoring\nmethod for wide-context neural language models and demonstrate performance\nimprovements over state-of-the-art self-attention based neural MT systems in\nfive out of six low-resource language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09079v2",
            "date": "2018-01-27 12:02:03+00:00",
            "title": "Using Additional Indexes for Fast Full-Text Search of Phrases That Contain Frequently Used Words",
            "authors": [
                "A. B. Veretennikov"
            ],
            "abstract": "Searches for phrases and word sets in large text arrays by means of\nadditional indexes are considered. Their use may reduce the query-processing\ntime by an order of magnitude in comparison with standard inverted files.",
            "journal ref": "Control Systems and Information Technologies. 2013. vol. 52, no.\n  2. pp. 61-66 (in Russian)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.09866v1",
            "date": "2018-01-30 06:58:50+00:00",
            "title": "Accelerating recurrent neural network language model based online speech recognition system",
            "authors": [
                "Kyungmin Lee",
                "Chiyoun Park",
                "Namhoon Kim",
                "Jaewon Lee"
            ],
            "abstract": "This paper presents methods to accelerate recurrent neural network based\nlanguage models (RNNLMs) for online speech recognition systems. Firstly, a\nlossy compression of the past hidden layer outputs (history vector) with\ncaching is introduced in order to reduce the number of LM queries. Next, RNNLM\ncomputations are deployed in a CPU-GPU hybrid manner, which computes each layer\nof the model on a more advantageous platform. The added overhead by data\nexchanges between CPU and GPU is compensated through a frame-wise batching\nstrategy. The performance of the proposed methods evaluated on LibriSpeech test\nsets indicates that the reduction in history vector precision improves the\naverage recognition speed by 1.23 times with minimum degradation in accuracy.\nOn the other hand, the CPU-GPU hybrid parallelization enables RNNLM based\nreal-time recognition with a four times improvement in speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10080v1",
            "date": "2018-01-30 16:13:55+00:00",
            "title": "A Machine Learning Approach to Quantitative Prosopography",
            "authors": [
                "Aayushee Gupta",
                "Haimonti Dutta",
                "Srikanta Bedathur",
                "Lipika Dey"
            ],
            "abstract": "Prosopography is an investigation of the common characteristics of a group of\npeople in history, by a collective study of their lives. It involves a study of\nbiographies to solve historical problems. If such biographies are unavailable,\nsurviving documents and secondary biographical data are used. Quantitative\nprosopography involves analysis of information from a wide variety of sources\nabout \"ordinary people\". In this paper, we present a machine learning framework\nfor automatically designing a people gazetteer which forms the basis of\nquantitative prosopographical research. The gazetteer is learnt from the noisy\ntext of newspapers using a Named Entity Recognizer (NER). It is capable of\nidentifying influential people from it by making use of a custom designed\nInfluential Person Index (IPI). Our corpus comprises of 14020 articles from a\nlocal newspaper, \"The Sun\", published from New York in 1896. Some influential\npeople identified by our algorithm include Captain Donald Hankey (an English\nsoldier), Dame Nellie Melba (an Australian operatic soprano), Hugh Allan (a\nCanadian shipping magnate) and Sir Hugh John McDonald (the first Prime Minister\nof Canada).",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10095v2",
            "date": "2018-01-30 17:01:01+00:00",
            "title": "TransRev: Modeling Reviews as Translations from Users to Items",
            "authors": [
                "Alberto Garcia-Duran",
                "Roberto Gonzalez",
                "Daniel Onoro-Rubio",
                "Mathias Niepert",
                "Hui Li"
            ],
            "abstract": "The text of a review expresses the sentiment a customer has towards a\nparticular product. This is exploited in sentiment analysis where machine\nlearning models are used to predict the review score from the text of the\nreview. Furthermore, the products costumers have purchased in the past are\nindicative of the products they will purchase in the future. This is what\nrecommender systems exploit by learning models from purchase information to\npredict the items a customer might be interested in. We propose TransRev, an\napproach to the product recommendation problem that integrates ideas from\nrecommender systems, sentiment analysis, and multi-relational learning into a\njoint learning objective. TransRev learns vector representations for users,\nitems, and reviews. The embedding of a review is learned such that (a) it\nperforms well as input feature of a regression model for sentiment prediction;\nand (b) it always translates the reviewer embedding to the embedding of the\nreviewed items. This allows TransRev to approximate a review embedding at test\ntime as the difference of the embedding of each item and the user embedding.\nThe approximated review embedding is then used with the regression model to\npredict the review score for each item. TransRev outperforms state of the art\nrecommender systems on a large number of benchmark data sets. Moreover, it is\nable to retrieve, for each user and item, the review text from the training set\nwhose embedding is most similar to the approximated review embedding.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.10308v1",
            "date": "2018-01-31 05:52:08+00:00",
            "title": "Nested LSTMs",
            "authors": [
                "Joel Ruben Antony Moniz",
                "David Krueger"
            ],
            "abstract": "We propose Nested LSTMs (NLSTM), a novel RNN architecture with multiple\nlevels of memory. Nested LSTMs add depth to LSTMs via nesting as opposed to\nstacking. The value of a memory cell in an NLSTM is computed by an LSTM cell,\nwhich has its own inner memory cell. Specifically, instead of computing the\nvalue of the (outer) memory cell as $c^{outer}_t = f_t \\odot c_{t-1} + i_t\n\\odot g_t$, NLSTM memory cells use the concatenation $(f_t \\odot c_{t-1}, i_t\n\\odot g_t)$ as input to an inner LSTM (or NLSTM) memory cell, and set\n$c^{outer}_t$ = $h^{inner}_t$. Nested LSTMs outperform both stacked and\nsingle-layer LSTMs with similar numbers of parameters in our experiments on\nvarious character-level language modeling tasks, and the inner memories of an\nLSTM learn longer term dependencies compared with the higher-level units of a\nstacked LSTM.",
            "journal ref": "Proceedings of the Ninth Asian Conference on Machine Learning,\n  PMLR 77:530-544, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00065v2",
            "date": "2018-03-30 21:57:40+00:00",
            "title": "Attentive Interaction Model: Modeling Changes in View in Argumentation",
            "authors": [
                "Yohan Jo",
                "Shivani Poddar",
                "Byungsoo Jeon",
                "Qinlan Shen",
                "Carolyn P. Rose",
                "Graham Neubig"
            ],
            "abstract": "We present a neural architecture for modeling argumentative dialogue that\nexplicitly models the interplay between an Opinion Holder's (OH's) reasoning\nand a challenger's argument, with the goal of predicting if the argument\nsuccessfully changes the OH's view. The model has two components: (1)\nvulnerable region detection, an attention model that identifies parts of the\nOH's reasoning that are amenable to change, and (2) interaction encoding, which\nidentifies the relationship between the content of the OH's reasoning and that\nof the challenger's argument. Based on evaluation on discussions from the\nChange My View forum on Reddit, the two components work together to predict an\nOH's change in view, outperforming several baselines. A posthoc analysis\nsuggests that sentences picked out by the attention model are addressed more\nfrequently by successful arguments than by unsuccessful ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00084v1",
            "date": "2018-03-30 23:31:43+00:00",
            "title": "Characterizing Interconnections and Linguistic Patterns in Twitter",
            "authors": [
                "Johnnatan Messias"
            ],
            "abstract": "Social media is considered a democratic space in which people connect and\ninteract with each other regardless of their gender, race, or any other\ndemographic aspect. Despite numerous efforts that explore demographic aspects\nin social media, it is still unclear whether social media perpetuates old\ninequalities from the offline world. In this dissertation, we attempt to\nidentify gender and race of Twitter users located in the United States using\nadvanced image processing algorithms from Face++. We investigate how different\ndemographic groups connect with each other and differentiate them regarding\nlinguistic styles and also their interests. We quantify to what extent one\ngroup follows and interacts with each other and the extent to which these\nconnections and interactions reflect in inequalities in Twitter. We also\nextract linguistic features from six categories (affective attributes,\ncognitive attributes, lexical density and awareness, temporal references,\nsocial and personal concerns, and interpersonal focus) in order to identify the\nsimilarities and the differences in the messages they share in Twitter.\nFurthermore, we extract the absolute ranking difference of top phrases between\ndemographic groups. As a dimension of diversity, we use the topics of interest\nthat we retrieve from each user. Our analysis shows that users identified as\nwhite and male tend to attain higher positions, in terms of the number of\nfollowers and number of times in another user's lists, in Twitter. There are\nclear differences in the way of writing across different demographic groups in\nboth gender and race domains as well as in the topic of interest. We hope our\neffort can stimulate the development of new theories of demographic information\nin the online space. Finally, we developed a Web-based system that leverages\nthe demographic aspects of users to provide transparency to the Twitter\ntrending topics system.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.00508v1",
            "date": "2018-03-22 05:40:28+00:00",
            "title": "Entrenamiento de una red neuronal para el reconocimiento de imagenes de lengua de senas capturadas con sensores de profundidad",
            "authors": [
                "Rivas P. Pedro E.",
                "Velarde-Anaya Omar",
                "Gonzalez-Lopez Samuel",
                "Rivas P. Pablo",
                "Alvarez-Torres Norma Angelica"
            ],
            "abstract": "Due to the growth of the population with hearing problems, devices have been\ndeveloped that facilitate the inclusion of deaf people in society, using\ntechnology as a communication tool, such as vision systems. Then, a solution to\nthis problem is presented using neural networks and autoencoders for the\nclassification of American Sign Language images. As a result, 99.5% accuracy\nand an error of 0.01684 were obtained for image classification",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.00522v4",
            "date": "2018-03-27 05:04:39+00:00",
            "title": "A Multi-Discriminator CycleGAN for Unsupervised Non-Parallel Speech Domain Adaptation",
            "authors": [
                "Ehsan Hosseini-Asl",
                "Yingbo Zhou",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Domain adaptation plays an important role for speech recognition models, in\nparticular, for domains that have low resources. We propose a novel generative\nmodel based on cyclic-consistent generative adversarial network (CycleGAN) for\nunsupervised non-parallel speech domain adaptation. The proposed model employs\nmultiple independent discriminators on the power spectrogram, each in charge of\ndifferent frequency bands. As a result we have 1) better discriminators that\nfocus on fine-grained details of the frequency features, and 2) a generator\nthat is capable of generating more realistic domain-adapted spectrogram. We\ndemonstrate the effectiveness of our method on speech recognition with gender\nadaptation, where the model only has access to supervised data from one gender\nduring training, but is evaluated on the other at test time. Our model is able\nto achieve an average of $7.41\\%$ on phoneme error rate, and $11.10\\%$ word\nerror rate relative performance improvement as compared to the baseline, on\nTIMIT and WSJ dataset, respectively. Qualitatively, our model also generates\nmore natural sounding speech, when conditioned on data from the other domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00538v4",
            "date": "2018-03-29 02:27:42+00:00",
            "title": "Investigating Capsule Networks with Dynamic Routing for Text Classification",
            "authors": [
                "Wei Zhao",
                "Jianbo Ye",
                "Min Yang",
                "Zeyang Lei",
                "Suofei Zhang",
                "Zhou Zhao"
            ],
            "abstract": "In this study, we explore capsule networks with dynamic routing for text\nclassification. We propose three strategies to stabilize the dynamic routing\nprocess to alleviate the disturbance of some noise capsules which may contain\n\"background\" information or have not been successfully trained. A series of\nexperiments are conducted with capsule networks on six text classification\nbenchmarks. Capsule networks achieve state of the art on 4 out of 6 datasets,\nwhich shows the effectiveness of capsule networks for text classification. We\nadditionally show that capsule networks exhibit significant improvement when\ntransfer single-label to multi-label text classification over strong baseline\nmethods. To the best of our knowledge, this is the first work that capsule\nnetworks have been empirically investigated for text modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00551v1",
            "date": "2018-03-30 08:32:42+00:00",
            "title": "The Training of Neuromodels for Machine Comprehension of Text. Brain2Text Algorithm",
            "authors": [
                "A. Artemov",
                "A. Sergeev",
                "A. Khasenevich",
                "A. Yuzhakov",
                "M. Chugunov"
            ],
            "abstract": "Nowadays, the Internet represents a vast informational space, growing\nexponentially and the problem of search for relevant data becomes essential as\nnever before. The algorithm proposed in the article allows to perform natural\nlanguage queries on content of the document and get comprehensive meaningful\nanswers. The problem is partially solved for English as SQuAD contains enough\ndata to learn on, but there is no such dataset in Russian, so the methods used\nby scientists now are not applicable to Russian. Brain2 framework allows to\ncope with the problem - it stands out for its ability to be applied on small\ndatasets and does not require impressive computing power. The algorithm is\nillustrated on Sberbank of Russia Strategy's text and assumes the use of a\nneuromodel consisting of 65 mln synapses. The trained model is able to\nconstruct word-by-word answers to questions based on a given text. The existing\nlimitations are its current inability to identify synonyms, pronoun relations\nand allegories. Nevertheless, the results of conducted experiments showed high\ncapacity and generalisation ability of the suggested approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00720v1",
            "date": "2018-04-02 20:29:21+00:00",
            "title": "Simple and Effective Semi-Supervised Question Answering",
            "authors": [
                "Bhuwan Dhingra",
                "Danish Pruthi",
                "Dheeraj Rajagopal"
            ],
            "abstract": "Recent success of deep learning models for the task of extractive Question\nAnswering (QA) is hinged on the availability of large annotated corpora.\nHowever, large domain specific annotated corpora are limited and expensive to\nconstruct. In this work, we envision a system where the end user specifies a\nset of base documents and only a few labelled examples. Our system exploits the\ndocument structure to create cloze-style questions from these base documents;\npre-trains a powerful neural network on the cloze style questions; and further\nfine-tunes the model on the labeled examples. We evaluate our proposed system\nacross three diverse datasets from different domains, and find it to be highly\neffective with very little labeled data. We attain more than 50% F1 score on\nSQuAD and TriviaQA with less than a thousand labelled examples. We are also\nreleasing a set of 3.2M cloze-style questions for practitioners to use while\nbuilding QA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00857v1",
            "date": "2018-04-03 07:41:10+00:00",
            "title": "Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling",
            "authors": [
                "Tao Shen",
                "Tianyi Zhou",
                "Guodong Long",
                "Jing Jiang",
                "Chengqi Zhang"
            ],
            "abstract": "Recurrent neural networks (RNN), convolutional neural networks (CNN) and\nself-attention networks (SAN) are commonly used to produce context-aware\nrepresentations. RNN can capture long-range dependency but is hard to\nparallelize and not time-efficient. CNN focuses on local dependency but does\nnot perform well on some tasks. SAN can model both such dependencies via highly\nparallelizable computation, but memory requirement grows rapidly in line with\nsequence length. In this paper, we propose a model, called \"bi-directional\nblock self-attention network (Bi-BloSAN)\", for RNN/CNN-free sequence encoding.\nIt requires as little memory as RNN but with all the merits of SAN. Bi-BloSAN\nsplits the entire sequence into blocks, and applies an intra-block SAN to each\nblock for modeling local context, then applies an inter-block SAN to the\noutputs for all blocks to capture long-range dependency. Thus, each SAN only\nneeds to process a short sequence, and only a small amount of memory is\nrequired. Additionally, we use feature-level attention to handle the variation\nof contexts around the same word, and use forward/backward masks to encode\ntemporal order information. On nine benchmark datasets for different NLP tasks,\nBi-BloSAN achieves or improves upon state-of-the-art accuracy, and shows better\nefficiency-memory trade-off than existing RNN/CNN/SAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01000v1",
            "date": "2018-04-01 13:02:57+00:00",
            "title": "CIKM AnalytiCup 2017 Lazada Product Title Quality Challenge An Ensemble of Deep and Shallow Learning to predict the Quality of Product Titles",
            "authors": [
                "Karamjit Singh",
                "Vishal Sunder"
            ],
            "abstract": "We present an approach where two different models (Deep and Shallow) are\ntrained separately on the data and a weighted average of the outputs is taken\nas the final result. For the Deep approach, we use different combinations of\nmodels like Convolution Neural Network, pretrained word2vec embeddings and\nLSTMs to get representations which are then used to train a Deep Neural\nNetwork. For Clarity prediction, we also use an Attentive Pooling approach for\nthe pooling operation so as to be aware of the Title-Category pair. For the\nshallow approach, we use boosting technique LightGBM on features generated\nusing title and categories. We find that an ensemble of these approaches does a\nbetter job than using them alone suggesting that the results of the deep and\nshallow approach are highly complementary",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01503v2",
            "date": "2018-04-04 16:45:04+00:00",
            "title": "Abstractive Tabular Dataset Summarization via Knowledge Base Semantic Embeddings",
            "authors": [
                "Paul Azunre",
                "Craig Corcoran",
                "David Sullivan",
                "Garrett Honke",
                "Rebecca Ruppel",
                "Sandeep Verma",
                "Jonathon Morgan"
            ],
            "abstract": "This paper describes an abstractive summarization method for tabular data\nwhich employs a knowledge base semantic embedding to generate the summary.\nAssuming the dataset contains descriptive text in headers, columns and/or some\naugmenting metadata, the system employs the embedding to recommend a\nsubject/type for each text segment. Recommendations are aggregated into a small\ncollection of super types considered to be descriptive of the dataset by\nexploiting the hierarchy of types in a pre-specified ontology. Using February\n2015 Wikipedia as the knowledge base, and a corresponding DBpedia ontology as\ntypes, we present experimental results on open data taken from several\nsources--OpenML, CKAN and data.world--to illustrate the effectiveness of the\napproach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.02596v1",
            "date": "2018-04-07 21:59:46+00:00",
            "title": "Simple Models for Word Formation in English Slang",
            "authors": [
                "Vivek Kulkarni",
                "William Yang Wang"
            ],
            "abstract": "We propose generative models for three types of extra-grammatical word\nformation phenomena abounding in English slang: Blends, Clippings, and\nReduplicatives. Adopting a data-driven approach coupled with linguistic\nknowledge, we propose simple models with state of the art performance on human\nannotated gold standard datasets. Overall, our models reveal insights into the\ngenerative processes of word formation in slang -- insights which are\nincreasingly relevant in the context of the rising prevalence of slang and\nnon-standard varieties on the Internet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03124v2",
            "date": "2018-04-09 17:46:33+00:00",
            "title": "Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection",
            "authors": [
                "Jing Qian",
                "Mai ElSherief",
                "Elizabeth M. Belding",
                "William Yang Wang"
            ],
            "abstract": "Hate speech detection is a critical, yet challenging problem in Natural\nLanguage Processing (NLP). Despite the existence of numerous studies dedicated\nto the development of NLP hate speech detection approaches, the accuracy is\nstill poor. The central problem is that social media posts are short and noisy,\nand most existing hate speech detection solutions take each post as an isolated\ninput instance, which is likely to yield high false positive and negative\nrates. In this paper, we radically improve automated hate speech detection by\npresenting a novel model that leverages intra-user and inter-user\nrepresentation learning for robust hate speech detection on Twitter. In\naddition to the target Tweet, we collect and analyze the user's historical\nposts to model intra-user Tweet representations. To suppress the noise in a\nsingle Tweet, we also model the similar Tweets posted by all other users with\nreinforced inter-user representation learning techniques. Experimentally, we\nshow that leveraging these two representations can significantly improve the\nf-score of a strong bidirectional LSTM baseline model by 10.1%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03209v1",
            "date": "2018-04-09 19:58:17+00:00",
            "title": "Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition",
            "authors": [
                "Pete Warden"
            ],
            "abstract": "Describes an audio dataset of spoken words designed to help train and\nevaluate keyword spotting systems. Discusses why this task is an interesting\nchallenge, and why it requires a specialized dataset that is different from\nconventional datasets used for automatic speech recognition of full sentences.\nSuggests a methodology for reproducible and comparable accuracy metrics for\nthis task. Describes how the data was collected and verified, what it contains,\nprevious versions and properties. Concludes by reporting baseline results of\nmodels trained on this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03799v1",
            "date": "2018-04-11 03:49:22+00:00",
            "title": "Achieving Fluency and Coherency in Task-oriented Dialog",
            "authors": [
                "Rashmi Gangadharaiah",
                "Balakrishnan Narayanaswamy",
                "Charles Elkan"
            ],
            "abstract": "We consider real world task-oriented dialog settings, where agents need to\ngenerate both fluent natural language responses and correct external actions\nlike database queries and updates. We demonstrate that, when applied to\ncustomer support chat transcripts, Sequence to Sequence (Seq2Seq) models often\ngenerate short, incoherent and ungrammatical natural language responses that\nare dominated by words that occur with high frequency in the training data.\nThese phenomena do not arise in synthetic datasets such as bAbI, where we show\nSeq2Seq models are nearly perfect. We develop techniques to learn embeddings\nthat succinctly capture relevant information from the dialog history, and\ndemonstrate that nearest neighbor based approaches in this learned neural\nembedding space generate more fluent responses. However, we see that these\nmethods are not able to accurately predict when to execute an external action.\nWe show how to combine nearest neighbor and Seq2Seq methods in a hybrid model,\nwhere nearest neighbor is used to generate fluent responses and Seq2Seq type\nmodels ensure dialog coherency and generate accurate external actions. We show\nthat this approach is well suited for customer support scenarios, where agents'\nresponses are typically script-driven, and correct external actions are\ncritically important. The hybrid model on the customer support data achieves a\n78% relative improvement in fluency scores, and a 130% improvement in accuracy\nof external calls.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03824v4",
            "date": "2018-04-11 06:10:48+00:00",
            "title": "Reference-less Measure of Faithfulness for Grammatical Error Correction",
            "authors": [
                "Leshem Choshen",
                "Omri Abend"
            ],
            "abstract": "We propose USim, a semantic measure for Grammatical Error Correction (GEC)\nthat measures the semantic faithfulness of the output to the source, thereby\ncomplementing existing reference-less measures (RLMs) for measuring the\noutput's grammaticality. USim operates by comparing the semantic symbolic\nstructure of the source and the correction, without relying on manually-curated\nreferences. Our experiments establish the validity of USim, by showing that (1)\nsemantic annotation can be consistently applied to ungrammatical text; (2)\nvalid corrections obtain a high USim similarity score to the source; and (3)\ninvalid corrections obtain a lower score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03839v1",
            "date": "2018-04-11 07:14:09+00:00",
            "title": "Generating Clues for Gender based Occupation De-biasing in Text",
            "authors": [
                "Nishtha Madaan",
                "Gautam Singh",
                "Sameep Mehta",
                "Aditya Chetan",
                "Brihi Joshi"
            ],
            "abstract": "Vast availability of text data has enabled widespread training and use of AI\nsystems that not only learn and predict attributes from the text but also\ngenerate text automatically. However, these AI models also learn gender, racial\nand ethnic biases present in the training data. In this paper, we present the\nfirst system that discovers the possibility that a given text portrays a gender\nstereotype associated with an occupation. If the possibility exists, the system\noffers counter-evidences of opposite gender also being associated with the same\noccupation in the context of user-provided geography and timespan. The system\nthus enables text de-biasing by assisting a human-in-the-loop. The system can\nnot only act as a text pre-processor before training any AI model but also help\nhuman story writers write stories free of occupation-level gender bias in the\ngeographical and temporal context of their choice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04059v2",
            "date": "2018-03-14 05:29:30+00:00",
            "title": "ISIS at its apogee: the Arabic discourse on Twitter and what we can learn from that about ISIS support and Foreign Fighters",
            "authors": [
                "A. Ceron",
                "L. Curini",
                "S. M. Iacus"
            ],
            "abstract": "We analyze 26.2 million comments published in Arabic language on Twitter,\nfrom July 2014 to January 2015, when ISIS' strength reached its peak and the\ngroup was prominently expanding the territorial area under its control. By\ndoing that, we are able to measure the share of support and aversion toward the\nIslamic State within the online Arab communities. We then investigate two\nspecific topics. First, by exploiting the time-granularity of the tweets, we\nlink the opinions with daily events to understand the main determinants of the\nchanging trend in support toward ISIS. Second, by taking advantage of the\ngeographical locations of tweets, we explore the relationship between online\nopinions across countries and the number of foreign fighters joining ISIS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04087v2",
            "date": "2018-04-10 13:17:36+00:00",
            "title": "Natural Language Statistical Features of LSTM-generated Texts",
            "authors": [
                "Marco Lippi",
                "Marcelo A Montemurro",
                "Mirko Degli Esposti",
                "Giampaolo Cristadoro"
            ],
            "abstract": "Long Short-Term Memory (LSTM) networks have recently shown remarkable\nperformance in several tasks dealing with natural language generation, such as\nimage captioning or poetry composition. Yet, only few works have analyzed text\ngenerated by LSTMs in order to quantitatively evaluate to which extent such\nartificial texts resemble those generated by humans. We compared the\nstatistical structure of LSTM-generated language to that of written natural\nlanguage, and to those produced by Markov models of various orders. In\nparticular, we characterized the statistical structure of language by assessing\nword-frequency statistics, long-range correlations, and entropy measures. Our\nmain finding is that while both LSTM and Markov-generated texts can exhibit\nfeatures similar to real ones in their word-frequency statistics and entropy\nmeasures, LSTM-texts are shown to reproduce long-range correlations at scales\ncomparable to those found in natural language. Moreover, for LSTM networks a\ntemperature-like parameter controlling the generation process shows an optimal\nvalue---for which the produced texts are closest to real language---consistent\nacross all the different statistical features investigated.",
            "journal ref": "IEEE Transactions on Neural Networks and Learning Systems, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04225v1",
            "date": "2018-04-11 21:16:39+00:00",
            "title": "Exploiting Task-Oriented Resources to Learn Word Embeddings for Clinical Abbreviation Expansion",
            "authors": [
                "Yue Liu",
                "Tao Ge",
                "Kusum S. Mathews",
                "Heng Ji",
                "Deborah L. McGuinness"
            ],
            "abstract": "In the medical domain, identifying and expanding abbreviations in clinical\ntexts is a vital task for both better human and machine understanding. It is a\nchallenging task because many abbreviations are ambiguous especially for\nintensive care medicine texts, in which phrase abbreviations are frequently\nused. Besides the fact that there is no universal dictionary of clinical\nabbreviations and no universal rules for abbreviation writing, such texts are\ndifficult to acquire, expensive to annotate and even sometimes, confusing to\ndomain experts. This paper proposes a novel and effective approach - exploiting\ntask-oriented resources to learn word embeddings for expanding abbreviations in\nclinical notes. We achieved 82.27% accuracy, close to expert human performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04257v1",
            "date": "2018-04-11 23:39:49+00:00",
            "title": "Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media",
            "authors": [
                "Mai ElSherief",
                "Vivek Kulkarni",
                "Dana Nguyen",
                "William Yang Wang",
                "Elizabeth Belding"
            ],
            "abstract": "While social media empowers freedom of expression and individual voices, it\nalso enables anti-social behavior, online harassment, cyberbullying, and hate\nspeech. In this paper, we deepen our understanding of online hate speech by\nfocusing on a largely neglected but crucial aspect of hate speech -- its\ntarget: either \"directed\" towards a specific person or entity, or \"generalized\"\ntowards a group of people sharing a common protected characteristic. We perform\nthe first linguistic and psycholinguistic analysis of these two forms of hate\nspeech and reveal the presence of interesting markers that distinguish these\ntypes of hate speech. Our analysis reveals that Directed hate speech, in\naddition to being more personal and directed, is more informal, angrier, and\noften explicitly attacks the target (via name calling) with fewer analytic\nwords and more words suggesting authority and influence. Generalized hate\nspeech, on the other hand, is dominated by religious hate, is characterized by\nthe use of lethal words such as murder, exterminate, and kill; and quantity\nwords such as million and many. Altogether, our work provides a data-driven\nanalysis of the nuances of online-hate speech that enables not only a deepened\nunderstanding of hate speech and its social implications but also its\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04264v1",
            "date": "2018-04-12 00:25:45+00:00",
            "title": "Training a Ranking Function for Open-Domain Question Answering",
            "authors": [
                "Phu Mon Htut",
                "Samuel R. Bowman",
                "Kyunghyun Cho"
            ],
            "abstract": "In recent years, there have been amazing advances in deep learning methods\nfor machine reading. In machine reading, the machine reader has to extract the\nanswer from the given ground truth paragraph. Recently, the state-of-the-art\nmachine reading models achieve human level performance in SQuAD which is a\nreading comprehension-style question answering (QA) task. The success of\nmachine reading has inspired researchers to combine information retrieval with\nmachine reading to tackle open-domain QA. However, these systems perform poorly\ncompared to reading comprehension-style QA because it is difficult to retrieve\nthe pieces of paragraphs that contain the answer to the question. In this\nstudy, we propose two neural network rankers that assign scores to different\npassages based on their likelihood of containing the answer to a given\nquestion. Additionally, we analyze the relative importance of semantic\nsimilarity and word level relevance matching in open-domain QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04266v2",
            "date": "2018-04-12 00:36:53+00:00",
            "title": "A Capsule Network-based Embedding Model for Search Personalization",
            "authors": [
                "Dai Quoc Nguyen",
                "Thanh Vu",
                "Tu Dinh Nguyen",
                "Dinh Phung"
            ],
            "abstract": "Search personalization aims to tailor search results to each specific user\nbased on the user's personal interests and preferences (i.e., the user\nprofile). Recent research approaches to search personalization by modelling the\npotential 3-way relationship between the submitted query, the user and the\nsearch results (i.e., documents). That relationship is then used to personalize\nthe search results to that user. In this paper, we introduce a novel embedding\nmodel based on capsule network, which recently is a breakthrough in deep\nlearning, to model the 3-way relationships for search personalization. In the\nmodel, each user (submitted query or returned document) is embedded by a vector\nin the same vector space. The 3-way relationship is described as a triple of\n(query, user, document) which is then modeled as a 3-column matrix containing\nthe three embedding vectors. After that, the 3-column matrix is fed into a deep\nlearning architecture to re-rank the search results returned by a basis ranker.\nExperimental results on query logs from a commercial web search engine show\nthat our model achieves better performances than the basis ranker as well as\nstrong search personalization baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04380v1",
            "date": "2018-04-12 09:04:50+00:00",
            "title": "Amobee at SemEval-2018 Task 1: GRU Neural Network with a CNN Attention Mechanism for Sentiment Classification",
            "authors": [
                "Alon Rozental",
                "Daniel Fleischer"
            ],
            "abstract": "This paper describes the participation of Amobee in the shared sentiment\nanalysis task at SemEval 2018. We participated in all the English sub-tasks and\nthe Spanish valence tasks. Our system consists of three parts: training\ntask-specific word embeddings, training a model consisting of\ngated-recurrent-units (GRU) with a convolution neural network (CNN) attention\nmechanism and training stacking-based ensembles for each of the sub-tasks. Our\nalgorithm reached 3rd and 1st places in the valence ordinal classification\nsub-tasks in English and Spanish, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04475v1",
            "date": "2018-04-12 12:46:08+00:00",
            "title": "Learning Multilingual Embeddings for Cross-Lingual Information Retrieval in the Presence of Topically Aligned Corpora",
            "authors": [
                "Mitodru Niyogi",
                "Kripabandhu Ghosh",
                "Arnab Bhattacharya"
            ],
            "abstract": "Cross-lingual information retrieval is a challenging task in the absence of\naligned parallel corpora. In this paper, we address this problem by considering\ntopically aligned corpora designed for evaluating an IR setup. To emphasize, we\nneither use any sentence-aligned corpora or document-aligned corpora, nor do we\nuse any language specific resources such as dictionary, thesaurus, or grammar\nrules. Instead, we use an embedding into a common space and learn word\ncorrespondences directly from there. We test our proposed approach for\nbilingual IR on standard FIRE datasets for Bangla, Hindi and English. The\nproposed method is superior to the state-of-the-art method not only for IR\nevaluation measures but also in terms of time requirements. We extend our\nmethod successfully to the trilingual setting.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.04526v1",
            "date": "2018-04-12 14:12:48+00:00",
            "title": "EventKG: A Multilingual Event-Centric Temporal Knowledge Graph",
            "authors": [
                "Simon Gottschalk",
                "Elena Demidova"
            ],
            "abstract": "One of the key requirements to facilitate semantic analytics of information\nregarding contemporary and historical events on the Web, in the news and in\nsocial media is the availability of reference knowledge repositories containing\ncomprehensive representations of events and temporal relations. Existing\nknowledge graphs, with popular examples including DBpedia, YAGO and Wikidata,\nfocus mostly on entity-centric information and are insufficient in terms of\ntheir coverage and completeness with respect to events and temporal relations.\nEventKG presented in this paper is a multilingual event-centric temporal\nknowledge graph that addresses this gap. EventKG incorporates over 690 thousand\ncontemporary and historical events and over 2.3 million temporal relations\nextracted from several large-scale knowledge graphs and semi-structured sources\nand makes them available through a canonical representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04749v3",
            "date": "2018-04-13 00:09:48+00:00",
            "title": "Predicting Good Configurations for GitHub and Stack Overflow Topic Models",
            "authors": [
                "Christoph Treude",
                "Markus Wagner"
            ],
            "abstract": "Software repositories contain large amounts of textual data, ranging from\nsource code comments and issue descriptions to questions, answers, and comments\non Stack Overflow. To make sense of this textual data, topic modelling is\nfrequently used as a text-mining tool for the discovery of hidden semantic\nstructures in text bodies. Latent Dirichlet allocation (LDA) is a commonly used\ntopic model that aims to explain the structure of a corpus by grouping texts.\nLDA requires multiple parameters to work well, and there are only rough and\nsometimes conflicting guidelines available on how these parameters should be\nset. In this paper, we contribute (i) a broad study of parameters to arrive at\ngood local optima for GitHub and Stack Overflow text corpora, (ii) an\na-posteriori characterisation of text corpora related to eight programming\nlanguages, and (iii) an analysis of corpus feature importance via per-corpus\nLDA configuration. We find that (1) popular rules of thumb for topic modelling\nparameter configuration are not applicable to the corpora used in our\nexperiments, (2) corpora sampled from GitHub and Stack Overflow have different\ncharacteristics and require different configurations to achieve good model fit,\nand (3) we can predict good configurations for unseen corpora reliably. These\nfindings support researchers and practitioners in efficiently determining\nsuitable configurations for topic modelling when analysing textual data\ncontained in software repositories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05088v1",
            "date": "2018-04-13 18:52:14+00:00",
            "title": "S\u00ed o no, qu\u00e8 penses? Catalonian Independence and Linguistic Identity on Social Media",
            "authors": [
                "Ian Stewart",
                "Yuval Pinter",
                "Jacob Eisenstein"
            ],
            "abstract": "Political identity is often manifested in language variation, but the\nrelationship between the two is still relatively unexplored from a quantitative\nperspective. This study examines the use of Catalan, a language local to the\nsemi-autonomous region of Catalonia in Spain, on Twitter in discourse related\nto the 2017 independence referendum. We corroborate prior findings that\npro-independence tweets are more likely to include the local language than\nanti-independence tweets. We also find that Catalan is used more often in\nreferendum-related discourse than in other contexts, contrary to prior findings\non language variation. This suggests a strong role for the Catalan language in\nthe expression of Catalonian political identity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05435v1",
            "date": "2018-04-15 21:48:28+00:00",
            "title": "What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text",
            "authors": [
                "Peter Clark",
                "Bhavana Dalvi",
                "Niket Tandon"
            ],
            "abstract": "Our goal is to answer questions about paragraphs describing processes (e.g.,\nphotosynthesis). Texts of this genre are challenging because the effects of\nactions are often implicit (unstated), requiring background knowledge and\ninference to reason about the changing world states. To supply this knowledge,\nwe leverage VerbNet to build a rulebase (called the Semantic Lexicon) of the\npreconditions and effects of actions, and use it along with commonsense\nknowledge of persistence to answer questions about change. Our evaluation shows\nthat our system, ProComp, significantly outperforms two strong reading\ncomprehension (RC) baselines. Our contributions are two-fold: the Semantic\nLexicon rulebase itself, and a demonstration of how a simulation-based approach\nto machine reading can outperform RC methods that rely on surface cues alone.\n  Since this work was performed, we have developed neural systems that\noutperform ProComp, described elsewhere (Dalvi et al., NAACL'18). However, the\nSemantic Lexicon remains a novel and potentially useful resource, and its\nintegration with neural systems remains a currently unexplored opportunity for\nfurther improvements in machine reading about processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05922v1",
            "date": "2018-04-16 20:07:44+00:00",
            "title": "Neural Models for Reasoning over Multiple Mentions using Coreference",
            "authors": [
                "Bhuwan Dhingra",
                "Qiao Jin",
                "Zhilin Yang",
                "William W. Cohen",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Many problems in NLP require aggregating information from multiple mentions\nof the same entity which may be far apart in the text. Existing Recurrent\nNeural Network (RNN) layers are biased towards short-term dependencies and\nhence not suited to such tasks. We present a recurrent layer which is instead\nbiased towards coreferent dependencies. The layer uses coreference annotations\nextracted from an external system to connect entity mentions belonging to the\nsame cluster. Incorporating this layer into a state-of-the-art reading\ncomprehension model improves performance on three datasets -- Wikihop, LAMBADA\nand the bAbi AI tasks -- with large gains when training data is scarce.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05958v1",
            "date": "2018-04-16 21:55:45+00:00",
            "title": "Can Neural Machine Translation be Improved with User Feedback?",
            "authors": [
                "Julia Kreutzer",
                "Shahram Khadivi",
                "Evgeny Matusov",
                "Stefan Riezler"
            ],
            "abstract": "We present the first real-world application of methods for improving neural\nmachine translation (NMT) with human reinforcement, based on explicit and\nimplicit user feedback collected on the eBay e-commerce platform. Previous work\nhas been confined to simulation experiments, whereas in this paper we work with\nreal logged feedback for offline bandit learning of NMT parameters. We conduct\na thorough analysis of the available explicit user judgments---five-star\nratings of translation quality---and show that they are not reliable enough to\nyield significant improvements in bandit learning. In contrast, we successfully\nutilize implicit task-based feedback collected in a cross-lingual search task\nto improve task-specific and machine translation quality metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06026v1",
            "date": "2018-04-17 03:22:00+00:00",
            "title": "Learning to Color from Language",
            "authors": [
                "Varun Manjunatha",
                "Mohit Iyyer",
                "Jordan Boyd-Graber",
                "Larry Davis"
            ],
            "abstract": "Automatic colorization is the process of adding color to greyscale images. We\ncondition this process on language, allowing end users to manipulate a\ncolorized image by feeding in different captions. We present two different\narchitectures for language-conditioned colorization, both of which produce more\naccurate and plausible colorizations than a language-agnostic version. Through\nthis language-based framework, we can dramatically alter colorizations by\nmanipulating descriptive color words in captions.",
            "journal ref": "North American Chapter of the Association for Computational\n  Linguistics (NAACL), 2018",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.06439v3",
            "date": "2018-04-17 19:11:14+00:00",
            "title": "Personalized neural language models for real-world query auto completion",
            "authors": [
                "Nicolas Fiorini",
                "Zhiyong Lu"
            ],
            "abstract": "Query auto completion (QAC) systems are a standard part of search engines in\nindustry, helping users formulate their query. Such systems update their\nsuggestions after the user types each character, predicting the user's intent\nusing various signals - one of the most common being popularity. Recently, deep\nlearning approaches have been proposed for the QAC task, to specifically\naddress the main limitation of previous popularity-based methods: the inability\nto predict unseen queries. In this work we improve previous methods based on\nneural language modeling, with the goal of building an end-to-end system. We\nparticularly focus on using real-world data by integrating user information for\npersonalized suggestions when possible. We also make use of time information\nand study how to increase diversity in the suggestions while studying the\nimpact on scalability. Our empirical results demonstrate a marked improvement\non two separate datasets over previous best methods in both accuracy and\nscalability, making a step towards neural query auto-completion in production\nsearch engines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06759v1",
            "date": "2018-04-18 14:32:34+00:00",
            "title": "Forecasting the presence and intensity of hostility on Instagram using linguistic and social features",
            "authors": [
                "Ping Liu",
                "Joshua Guberman",
                "Libby Hemphill",
                "Aron Culotta"
            ],
            "abstract": "Online antisocial behavior, such as cyberbullying, harassment, and trolling,\nis a widespread problem that threatens free discussion and has negative\nphysical and mental health consequences for victims and communities. While\nprior work has proposed automated methods to identify hostile comments in\nonline discussions, these methods work retrospectively on comments that have\nalready been posted, making it difficult to intervene before an interaction\nescalates. In this paper we instead consider the problem of forecasting future\nhostilities in online discussions, which we decompose into two tasks: (1) given\nan initial sequence of non-hostile comments in a discussion, predict whether\nsome future comment will contain hostility; and (2) given the first hostile\ncomment in a discussion, predict whether this will lead to an escalation of\nhostility in subsequent comments. Thus, we aim to forecast both the presence\nand intensity of hostile comments based on linguistic and social features from\nearlier comments. To evaluate our approach, we introduce a corpus of over 30K\nannotated Instagram comments from over 1,100 posts. Our approach is able to\npredict the appearance of a hostile comment on an Instagram post ten or more\nhours in the future with an AUC of .82 (task 1), and can furthermore\ndistinguish between high and low levels of future hostility with an AUC of .91\n(task 2).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06876v1",
            "date": "2018-04-18 18:51:00+00:00",
            "title": "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods",
            "authors": [
                "Jieyu Zhao",
                "Tianlu Wang",
                "Mark Yatskar",
                "Vicente Ordonez",
                "Kai-Wei Chang"
            ],
            "abstract": "We introduce a new benchmark, WinoBias, for coreference resolution focused on\ngender bias. Our corpus contains Winograd-schema style sentences with entities\ncorresponding to people referred by their occupation (e.g. the nurse, the\ndoctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a\nneural coreference system all link gendered pronouns to pro-stereotypical\nentities with higher accuracy than anti-stereotypical entities, by an average\ndifference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation\napproach that, in combination with existing word-embedding debiasing\ntechniques, removes the bias demonstrated by these systems in WinoBias without\nsignificantly affecting their performance on existing coreference benchmark\ndatasets. Our dataset and code are available at http://winobias.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06898v3",
            "date": "2018-04-18 19:55:18+00:00",
            "title": "Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input",
            "authors": [
                "Youmna Farag",
                "Helen Yannakoudakis",
                "Ted Briscoe"
            ],
            "abstract": "We demonstrate that current state-of-the-art approaches to Automated Essay\nScoring (AES) are not well-suited to capturing adversarially crafted input of\ngrammatical but incoherent sequences of sentences. We develop a neural model of\nlocal coherence that can effectively learn connectedness features between\nsentences, and propose a framework for integrating and jointly training the\nlocal coherence model with a state-of-the-art AES model. We evaluate our\napproach against a number of baselines and experimentally demonstrate its\neffectiveness on both the AES task and the task of flagging adversarial input,\nfurther contributing to the development of an approach that strengthens the\nvalidity of neural essay scoring models.",
            "journal ref": "The 16th Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics (NAACL 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07000v3",
            "date": "2018-04-19 05:25:51+00:00",
            "title": "Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences",
            "authors": [
                "Marcel Trotzek",
                "Sven Koitka",
                "Christoph M. Friedrich"
            ],
            "abstract": "Depression is ranked as the largest contributor to global disability and is\nalso a major reason for suicide. Still, many individuals suffering from forms\nof depression are not treated for various reasons. Previous studies have shown\nthat depression also has an effect on language usage and that many depressed\nindividuals use social media platforms or the internet in general to get\ninformation or discuss their problems. This paper addresses the early detection\nof depression using machine learning models based on messages on a social\nplatform. In particular, a convolutional neural network based on different word\nembeddings is evaluated and compared to a classification based on user-level\nlinguistic metadata. An ensemble of both approaches is shown to achieve\nstate-of-the-art results in a current early detection task. Furthermore, the\ncurrently popular ERDE score as metric for early detection systems is examined\nin detail and its drawbacks in the context of shared tasks are illustrated. A\nslightly modified metric is proposed and compared to the original score.\nFinally, a new word embedding was trained on a large corpus of the same domain\nas the described task and is evaluated as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07331v1",
            "date": "2018-04-19 18:37:40+00:00",
            "title": "Stylistic Variation in Social Media Part-of-Speech Tagging",
            "authors": [
                "Murali Raghu Babu Balusu",
                "Taha Merghani",
                "Jacob Eisenstein"
            ],
            "abstract": "Social media features substantial stylistic variation, raising new challenges\nfor syntactic analysis of online writing. However, this variation is often\naligned with author attributes such as age, gender, and geography, as well as\nmore readily-available social network metadata. In this paper, we report new\nevidence on the link between language and social networks in the task of\npart-of-speech tagging. We find that tagger error rates are correlated with\nnetwork structure, with high accuracy in some parts of the network, and lower\naccuracy elsewhere. As a result, tagger accuracy depends on training from a\nbalanced sample of the network, rather than training on texts from a narrow\nsubcommunity. We also describe our attempts to add robustness to stylistic\nvariation, by building a mixture-of-experts model in which each expert is\nassociated with a region of the social network. While prior work found that\nsimilar approaches yield performance improvements in sentiment analysis and\nentity linking, we were unable to obtain performance improvements in\npart-of-speech tagging, despite strong evidence for the link between\npart-of-speech error rates and social network structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07399v1",
            "date": "2018-04-19 23:06:00+00:00",
            "title": "Video based Contextual Question Answering",
            "authors": [
                "Akash Ganesan",
                "Divyansh Pal",
                "Karthik Muthuraman",
                "Shubham Dash"
            ],
            "abstract": "The primary aim of this project is to build a contextual Question-Answering\nmodel for videos. The current methodologies provide a robust model for image\nbased Question-Answering, but we are aim to generalize this approach to be\nvideos. We propose a graphical representation of video which is able to handle\nseveral types of queries across the whole video. For example, if a frame has an\nimage of a man and a cat sitting, it should be able to handle queries like,\nwhere is the cat sitting with respect to the man? or ,what is the man holding\nin his hand?. It should be able to answer queries relating to temporal\nrelationships also.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07583v2",
            "date": "2018-04-20 12:49:54+00:00",
            "title": "Approaches for Enriching and Improving Textual Knowledge Bases",
            "authors": [
                "Besnik Fetahu"
            ],
            "abstract": "Verifiability is one of the core editing principles in Wikipedia, where\neditors are encouraged to provide citations for the added statements.\nStatements can be any arbitrary piece of text, ranging from a sentence up to a\nparagraph. However, in many cases, citations are either outdated, missing, or\nlink to non-existing references (e.g. dead URL, moved content etc.). In total,\n20\\% of the cases such citations refer to news articles and represent the\nsecond most cited source. Even in cases where citations are provided, there are\nno explicit indicators for the span of a citation for a given piece of text. In\naddition to issues related with the verifiability principle, many Wikipedia\nentity pages are incomplete, with relevant information that is already\navailable in online news sources missing. Even for the already existing\ncitations, there is often a delay between the news publication time and the\nreference time.\n  In this thesis, we address the aforementioned issues and propose automated\napproaches that enforce the verifiability principle in Wikipedia, and suggest\nrelevant and missing news references for further enriching Wikipedia entity\npages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07691v1",
            "date": "2018-04-20 15:51:14+00:00",
            "title": "Cross-domain Dialogue Policy Transfer via Simultaneous Speech-act and Slot Alignment",
            "authors": [
                "Kaixiang Mo",
                "Yu Zhang",
                "Qiang Yang",
                "Pascale Fung"
            ],
            "abstract": "Dialogue policy transfer enables us to build dialogue policies in a target\ndomain with little data by leveraging knowledge from a source domain with\nplenty of data. Dialogue sentences are usually represented by speech-acts and\ndomain slots, and the dialogue policy transfer is usually achieved by assigning\na slot mapping matrix based on human heuristics. However, existing dialogue\npolicy transfer methods cannot transfer across dialogue domains with different\nspeech-acts, for example, between systems built by different companies. Also,\nthey depend on either common slots or slot entropy, which are not available\nwhen the source and target slots are totally disjoint and no database is\navailable to calculate the slot entropy. To solve this problem, we propose a\nPolicy tRansfer across dOMaIns and SpEech-acts (PROMISE) model, which is able\nto transfer dialogue policies across domains with different speech-acts and\ndisjoint slots. The PROMISE model can learn to align different speech-acts and\nslots simultaneously, and it does not require common slots or the calculation\nof the slot entropy. Experiments on both real-world dialogue data and\nsimulations demonstrate that PROMISE model can effectively transfer dialogue\npolicies across domains with different speech-acts and disjoint slots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07745v3",
            "date": "2018-04-20 17:41:15+00:00",
            "title": "Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion",
            "authors": [
                "Armand Joulin",
                "Piotr Bojanowski",
                "Tomas Mikolov",
                "Herve Jegou",
                "Edouard Grave"
            ],
            "abstract": "Continuous word representations learned separately on distinct languages can\nbe aligned so that their words become comparable in a common space. Existing\nworks typically solve a least-square regression problem to learn a rotation\naligning a small bilingual lexicon, and use a retrieval criterion for\ninference. In this paper, we propose an unified formulation that directly\noptimizes a retrieval criterion in an end-to-end fashion. Our experiments on\nstandard benchmarks show that our approach outperforms the state of the art on\nword translation, with the biggest improvements observed for distant language\npairs such as English-Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07790v1",
            "date": "2018-04-20 18:31:29+00:00",
            "title": "A Mixed Hierarchical Attention based Encoder-Decoder Approach for Standard Table Summarization",
            "authors": [
                "Parag Jain",
                "Anirban Laha",
                "Karthik Sankaranarayanan",
                "Preksha Nema",
                "Mitesh M. Khapra",
                "Shreyas Shetty"
            ],
            "abstract": "Structured data summarization involves generation of natural language\nsummaries from structured input data. In this work, we consider summarizing\nstructured data occurring in the form of tables as they are prevalent across a\nwide variety of domains. We formulate the standard table summarization problem,\nwhich deals with tables conforming to a single predefined schema. To this end,\nwe propose a mixed hierarchical attention based encoder-decoder model which is\nable to leverage the structure in addition to the content of the tables. Our\nexperiments on the publicly available WEATHERGOV dataset show around 18 BLEU (~\n30%) improvement over the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07875v1",
            "date": "2018-04-21 01:48:38+00:00",
            "title": "Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding",
            "authors": [
                "Lifu Huang",
                "Kyunghyun Cho",
                "Boliang Zhang",
                "Heng Ji",
                "Kevin Knight"
            ],
            "abstract": "We construct a multilingual common semantic space based on distributional\nsemantics, where words from multiple languages are projected into a shared\nspace to enable knowledge and resource transfer across languages. Beyond word\nalignment, we introduce multiple cluster-level alignments and enforce the word\nclusters to be consistently distributed across multiple languages. We exploit\nthree signals for clustering: (1) neighbor words in the monolingual word\nembedding space; (2) character-level information; and (3) linguistic properties\n(e.g., apposition, locative suffix) derived from linguistic structure knowledge\nbases available for thousands of languages. We introduce a new\ncluster-consistent correlational neural network to construct the common\nsemantic space by aligning words as well as clusters. Intrinsic evaluation on\nmonolingual and multilingual QVEC tasks shows our approach achieves\nsignificantly higher correlation with linguistic features than state-of-the-art\nmulti-lingual embedding learning methods do. Using low-resource language name\ntagging as a case study for extrinsic evaluation, our approach achieves up to\n24.5\\% absolute F-score gain over the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07918v2",
            "date": "2018-04-21 08:27:14+00:00",
            "title": "Decoupling Structure and Lexicon for Zero-Shot Semantic Parsing",
            "authors": [
                "Jonathan Herzig",
                "Jonathan Berant"
            ],
            "abstract": "Building a semantic parser quickly in a new domain is a fundamental challenge\nfor conversational interfaces, as current semantic parsers require expensive\nsupervision and lack the ability to generalize to new domains. In this paper,\nwe introduce a zero-shot approach to semantic parsing that can parse utterances\nin unseen domains while only being trained on examples in other source domains.\nFirst, we map an utterance to an abstract, domain-independent, logical form\nthat represents the structure of the logical form, but contains slots instead\nof KB constants. Then, we replace slots with KB constants via lexical alignment\nscores and global inference. Our model reaches an average accuracy of 53.4% on\n7 domains in the Overnight dataset, substantially better than other zero-shot\nbaselines, and performs as good as a parser trained on over 30% of the target\ndomain examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07946v2",
            "date": "2018-04-21 11:17:26+00:00",
            "title": "Extrofitting: Enriching Word Representation and its Vector Space with Semantic Lexicons",
            "authors": [
                "Hwiyeol Jo",
                "Stanley Jungkyu Choi"
            ],
            "abstract": "We propose post-processing method for enriching not only word representation\nbut also its vector space using semantic lexicons, which we call extrofitting.\nThe method consists of 3 steps as follows: (i) Expanding 1 or more dimension(s)\non all the word vectors, filling with their representative value. (ii)\nTransferring semantic knowledge by averaging each representative values of\nsynonyms and filling them in the expanded dimension(s). These two steps make\nrepresentations of the synonyms close together. (iii) Projecting the vector\nspace using Linear Discriminant Analysis, which eliminates the expanded\ndimension(s) with semantic knowledge. When experimenting with GloVe, we find\nthat our method outperforms Faruqui's retrofitting on some of word similarity\ntask. We also report further analysis on our method in respect to word vector\ndimensions, vocabulary size as well as other well-known pretrained word vectors\n(e.g., Word2Vec, Fasttext).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07958v1",
            "date": "2018-04-21 12:37:45+00:00",
            "title": "Expert Finding in Community Question Answering: A Review",
            "authors": [
                "Sha Yuan",
                "Yu Zhang",
                "Jie Tang",
                "Juan Bautista Cabot\u00e0"
            ],
            "abstract": "The rapid development recently of Community Question Answering (CQA)\nsatisfies users quest for professional and personal knowledge about anything.\nIn CQA, one central issue is to find users with expertise and willingness to\nanswer the given questions. Expert finding in CQA often exhibits very different\nchallenges compared to traditional methods. Sparse data and new features\nviolate fundamental assumptions of traditional recommendation systems. This\npaper focuses on reviewing and categorizing the current progress on expert\nfinding in CQA. We classify all the existing solutions into four different\ncategories: matrix factorization based models (MF-based models), gradient\nboosting tree based models (GBT-based models), deep learning based models\n(DL-based models) and ranking based models (R-based models). We find that\nMF-based models outperform other categories of models in the field of expert\nfinding in CQA. Moreover, we use innovative diagrams to clarify several\nimportant concepts of ensemble learning, and find that ensemble models with\nseveral specific single models can further boosting the performance. Further,\nwe compare the performance of different models on different types of matching\ntasks, including text vs. text, graph vs. text, audio vs. text and video vs.\ntext. The results can help the model selection of expert finding in practice.\nFinally, we explore some potential future issues in expert finding research in\nCQA.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.08057v4",
            "date": "2018-04-22 02:20:32+00:00",
            "title": "A Study on Passage Re-ranking in Embedding based Unsupervised Semantic Search",
            "authors": [
                "Md Faisal Mahbub Chowdhury",
                "Vijil Chenthamarakshan",
                "Rishav Chakravarti",
                "Alfio M. Gliozzo"
            ],
            "abstract": "State of the art approaches for (embedding based) unsupervised semantic\nsearch exploits either compositional similarity (of a query and a passage) or\npair-wise word (or term) similarity (from the query and the passage). By\ndesign, word based approaches do not incorporate similarity in the larger\ncontext (query/passage), while compositional similarity based approaches are\nusually unable to take advantage of the most important cues in the context. In\nthis paper we propose a new compositional similarity based approach, called\nvariable centroid vector (VCVB), that tries to address both of these\nlimitations. We also presents results using a different type of compositional\nsimilarity based approach by exploiting universal sentence embedding. We\nprovide empirical evaluation on two different benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08069v1",
            "date": "2018-04-22 05:08:52+00:00",
            "title": "Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation",
            "authors": [
                "Tiancheng Zhao",
                "Kyusong Lee",
                "Maxine Eskenazi"
            ],
            "abstract": "The encoder-decoder dialog model is one of the most prominent methods used to\nbuild dialog systems in complex domains. Yet it is limited because it cannot\noutput interpretable actions as in traditional systems, which hinders humans\nfrom understanding its generation process. We present an unsupervised discrete\nsentence representation learning method that can integrate with any existing\nencoder-decoder dialog models for interpretable response generation. Building\nupon variational autoencoders (VAEs), we present two novel models, DI-VAE and\nDI-VST that improve VAEs and can discover interpretable semantics via either\nauto encoding or context predicting. Our methods have been validated on\nreal-world dialog datasets to discover semantic representations and enhance\nencoder-decoder models with interpretable generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08117v1",
            "date": "2018-04-22 14:26:28+00:00",
            "title": "Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment",
            "authors": [
                "Masatoshi Tsuchiya"
            ],
            "abstract": "The quality of training data is one of the crucial problems when a\nlearning-centered approach is employed. This paper proposes a new method to\ninvestigate the quality of a large corpus designed for the recognizing textual\nentailment (RTE) task. The proposed method, which is inspired by a statistical\nhypothesis test, consists of two phases: the first phase is to introduce the\npredictability of textual entailment labels as a null hypothesis which is\nextremely unacceptable if a target corpus has no hidden bias, and the second\nphase is to test the null hypothesis using a Naive Bayes model. The\nexperimental result of the Stanford Natural Language Inference (SNLI) corpus\ndoes not reject the null hypothesis. Therefore, it indicates that the SNLI\ncorpus has a hidden bias which allows prediction of textual entailment labels\nfrom hypothesis sentences even if no context information is given by a premise\nsentence. This paper also presents the performance impact of NN models for RTE\ncaused by this hidden bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08139v1",
            "date": "2018-04-22 17:13:06+00:00",
            "title": "Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks",
            "authors": [
                "Renjie Zheng",
                "Junkun Chen",
                "Xipeng Qiu"
            ],
            "abstract": "Distributed representation plays an important role in deep learning based\nnatural language processing. However, the representation of a sentence often\nvaries in different tasks, which is usually learned from scratch and suffers\nfrom the limited amounts of training data. In this paper, we claim that a good\nsentence representation should be invariant and can benefit the various\nsubsequent tasks. To achieve this purpose, we propose a new scheme of\ninformation sharing for multi-task learning. More specifically, all tasks share\nthe same sentence representation and each task can select the task-specific\ninformation from the shared sentence representation with attention mechanism.\nThe query vector of each task's attention could be either static parameters or\ngenerated dynamically. We conduct extensive experiments on 16 different text\nclassification tasks, which demonstrate the benefits of our architecture.",
            "journal ref": "IJCAI 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08204v1",
            "date": "2018-04-23 00:47:48+00:00",
            "title": "Knowledge-based end-to-end memory networks",
            "authors": [
                "Jatin Ganhotra",
                "Lazaros Polymenakos"
            ],
            "abstract": "End-to-end dialog systems have become very popular because they hold the\npromise of learning directly from human to human dialog interaction. Retrieval\nand Generative methods have been explored in this area with mixed results. A\nkey element that is missing so far, is the incorporation of a-priori knowledge\nabout the task at hand. This knowledge may exist in the form of structured or\nunstructured information. As a first step towards this direction, we present a\nnovel approach, Knowledge based end-to-end memory networks (KB-memN2N), which\nallows special handling of named entities for goal-oriented dialog tasks. We\npresent results on two datasets, DSTC6 challenge dataset and dialog bAbI tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08316v1",
            "date": "2018-04-23 10:02:29+00:00",
            "title": "Bilingual Embeddings with Random Walks over Multilingual Wordnets",
            "authors": [
                "J. Goikoetxea",
                "A. Soroa",
                "E. Agirre"
            ],
            "abstract": "Bilingual word embeddings represent words of two languages in the same space,\nand allow to transfer knowledge from one language to the other without machine\ntranslation. The main approach is to train monolingual embeddings first and\nthen map them using bilingual dictionaries. In this work, we present a novel\nmethod to learn bilingual embeddings based on multilingual knowledge bases (KB)\nsuch as WordNet. Our method extracts bilingual information from multilingual\nwordnets via random walks and learns a joint embedding space in one go. We\nfurther reinforce cross-lingual equivalence adding bilingual con- straints in\nthe loss function of the popular skipgram model. Our experiments involve twelve\ncross-lingual word similarity and relatedness datasets in six lan- guage pairs\ncovering four languages, and show that: 1) random walks over mul- tilingual\nwordnets improve results over just using dictionaries; 2) multilingual wordnets\non their own improve over text-based systems in similarity datasets; 3) the\ngood results are consistent for large wordnets (e.g. English, Spanish), smaller\nwordnets (e.g. Basque) or loosely aligned wordnets (e.g. Italian); 4) the\ncombination of wordnets and text yields the best results, above mapping-based\napproaches. Our method can be applied to richer KBs like DBpedia or Babel- Net,\nand can be easily extended to multilingual embeddings. All software and\nresources are open source.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08426v1",
            "date": "2018-04-19 09:42:01+00:00",
            "title": "LightRel SemEval-2018 Task 7: Lightweight and Fast Relation Classification",
            "authors": [
                "Tyler Renslow",
                "G\u00fcnter Neumann"
            ],
            "abstract": "We present LightRel, a lightweight and fast relation classifier. Our goal is\nto develop a high baseline for different relation extraction tasks. By defining\nonly very few data-internal, word-level features and external knowledge sources\nin the form of word clusters and word embeddings, we train a fast and simple\nlinear classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08759v1",
            "date": "2018-04-23 22:07:28+00:00",
            "title": "Analyzing and Characterizing User Intent in Information-seeking Conversations",
            "authors": [
                "Chen Qu",
                "Liu Yang",
                "W. Bruce Croft",
                "Johanne R. Trippas",
                "Yongfeng Zhang",
                "Minghui Qiu"
            ],
            "abstract": "Understanding and characterizing how people interact in information-seeking\nconversations is crucial in developing conversational search systems. In this\npaper, we introduce a new dataset designed for this purpose and use it to\nanalyze information-seeking conversations by user intent distribution,\nco-occurrence, and flow patterns. The MSDialog dataset is a labeled dialog\ndataset of question answering (QA) interactions between information seekers and\nproviders from an online forum on Microsoft products. The dataset contains more\nthan 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with\nuser intent on the utterance level. Annotations were done using crowdsourcing.\nWith MSDialog, we find some highly recurring patterns in user intent during an\ninformation-seeking process. They could be useful for designing conversational\nsearch systems. We will make our dataset freely available to encourage\nexploration of information-seeking conversation models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.08798v1",
            "date": "2018-04-24 01:24:35+00:00",
            "title": "SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach",
            "authors": [
                "Michael Petrochuk",
                "Luke Zettlemoyer"
            ],
            "abstract": "The SimpleQuestions dataset is one of the most commonly used benchmarks for\nstudying single-relation factoid questions. In this paper, we present new\nevidence that this benchmark can be nearly solved by standard methods. First we\nshow that ambiguity in the data bounds performance on this benchmark at 83.4%;\nthere are often multiple answers that cannot be disambiguated from the\nlinguistic signal alone. Second we introduce a baseline that sets a new\nstate-of-the-art performance level at 78.1% accuracy, despite using standard\nmethods. Finally, we report an empirical analysis showing that the upperbound\nis loose; roughly a third of the remaining errors are also not resolvable from\nthe linguistic signal. Together, these results suggest that the SimpleQuestions\ndataset is nearly solved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08847v1",
            "date": "2018-04-24 06:00:28+00:00",
            "title": "DeepEmo: Learning and Enriching Pattern-Based Emotion Representations",
            "authors": [
                "Elvis Saravia",
                "Hsien-Chi Toby Liu",
                "Yi-Shin Chen"
            ],
            "abstract": "We propose a graph-based mechanism to extract rich-emotion bearing patterns,\nwhich fosters a deeper analysis of online emotional expressions, from a corpus.\nThe patterns are then enriched with word embeddings and evaluated through\nseveral emotion recognition tasks. Moreover, we conduct analysis on the\nemotion-oriented patterns to demonstrate its applicability and to explore its\nproperties. Our experimental results demonstrate that the proposed techniques\noutperform most state-of-the-art emotion recognition techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09021v2",
            "date": "2018-04-24 13:35:11+00:00",
            "title": "Label-aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition",
            "authors": [
                "Zhenghui Wang",
                "Yanru Qu",
                "Liheng Chen",
                "Jian Shen",
                "Weinan Zhang",
                "Shaodian Zhang",
                "Yimei Gao",
                "Gen Gu",
                "Ken Chen",
                "Yong Yu"
            ],
            "abstract": "We study the problem of named entity recognition (NER) from electronic\nmedical records, which is one of the most fundamental and critical problems for\nmedical text mining. Medical records which are written by clinicians from\ndifferent specialties usually contain quite different terminologies and writing\nstyles. The difference of specialties and the cost of human annotation makes it\nparticularly difficult to train a universal medical NER system. In this paper,\nwe propose a label-aware double transfer learning framework (La-DTL) for\ncross-specialty NER, so that a medical NER system designed for one specialty\ncould be conveniently applied to another one with minimal annotation efforts.\nThe transferability is guaranteed by two components: (i) we propose label-aware\nMMD for feature representation transfer, and (ii) we perform parameter transfer\nwith a theoretical upper bound which is also label aware. We conduct extensive\nexperiments on 12 cross-specialty NER tasks. The experimental results\ndemonstrate that La-DTL provides consistent accuracy improvement over strong\nbaselines. Besides, the promising experimental results on non-medical NER\nscenarios indicate that La-DTL is potential to be seamlessly adapted to a wide\nrange of NER tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09540v2",
            "date": "2018-04-22 20:09:13+00:00",
            "title": "NE-Table: A Neural key-value table for Named Entities",
            "authors": [
                "Janarthanan Rajendran",
                "Jatin Ganhotra",
                "Xiaoxiao Guo",
                "Mo Yu",
                "Satinder Singh",
                "Lazaros Polymenakos"
            ],
            "abstract": "Many Natural Language Processing (NLP) tasks depend on using Named Entities\n(NEs) that are contained in texts and in external knowledge sources. While this\nis easy for humans, the present neural methods that rely on learned word\nembeddings may not perform well for these NLP tasks, especially in the presence\nof Out-Of-Vocabulary (OOV) or rare NEs. In this paper, we propose a solution\nfor this problem, and present empirical evaluations on: a) a structured\nQuestion-Answering task, b) three related Goal-Oriented dialog tasks, and c) a\nReading-Comprehension task, which show that the proposed method can be\neffective in dealing with both in-vocabulary and OOV NEs. We create extended\nversions of dialog bAbI tasks 1,2 and 4 and OOV versions of the CBT test set\navailable at - https://github.com/IBM/ne-table-datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09543v4",
            "date": "2018-04-23 20:54:43+00:00",
            "title": "The Future of Prosody: It's about Time",
            "authors": [
                "Dafydd Gibbon"
            ],
            "abstract": "Prosody is usually defined in terms of the three distinct but interacting\ndomains of pitch, intensity and duration patterning, or, more generally, as\nphonological and phonetic properties of 'suprasegmentals', speech segments\nwhich are larger than consonants and vowels. Rather than taking this approach,\nthe concept of multiple time domains for prosody processing is taken up, and\nmethods of time domain analysis are discussed: annotation mining with timing\ndispersion measures, time tree induction, oscillator models in phonology and\nphonetics, and finally the use of the Amplitude Envelope Modulation Spectrum\n(AEMS). While frequency demodulation (in the form of pitch tracking) is a\ncentral issue in prosodic analysis, in the present context it is amplitude\nenvelope demodulation and frequency zones in the long time-domain spectra of\nthe demodulated envelope which are focused. A generalised view is taken of\noscillation as iteration in abstract prosodic models and as modulation and\ndemodulation of a variety of rhythms in the speech signal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09552v1",
            "date": "2018-04-24 10:28:57+00:00",
            "title": "Automatic speech recognition for launch control center communication using recurrent neural networks with data augmentation and custom language model",
            "authors": [
                "Kyongsik Yun",
                "Joseph Osborne",
                "Madison Lee",
                "Thomas Lu",
                "Edward Chow"
            ],
            "abstract": "Transcribing voice communications in NASA's launch control center is\nimportant for information utilization. However, automatic speech recognition in\nthis environment is particularly challenging due to the lack of training data,\nunfamiliar words in acronyms, multiple different speakers and accents, and\nconversational characteristics of speaking. We used bidirectional deep\nrecurrent neural networks to train and test speech recognition performance. We\nshowed that data augmentation and custom language models can improve speech\nrecognition accuracy. Transcribing communications from the launch control\ncenter will help the machine analyze information and accelerate knowledge\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09635v1",
            "date": "2018-04-25 15:41:15+00:00",
            "title": "A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications",
            "authors": [
                "Dongyeop Kang",
                "Waleed Ammar",
                "Bhavana Dalvi",
                "Madeleine van Zuylen",
                "Sebastian Kohlmeier",
                "Eduard Hovy",
                "Roy Schwartz"
            ],
            "abstract": "Peer reviewing is a central component in the scientific publishing process.\nWe present the first public dataset of scientific peer reviews available for\nresearch purposes (PeerRead v1) providing an opportunity to study this\nimportant artifact. The dataset consists of 14.7K paper drafts and the\ncorresponding accept/reject decisions in top-tier venues including ACL, NIPS\nand ICLR. The dataset also includes 10.7K textual peer reviews written by\nexperts for a subset of the papers. We describe the data collection process and\nreport interesting observed phenomena in the peer reviews. We also propose two\nnovel NLP tasks based on this dataset and provide simple baseline models. In\nthe first task, we show that simple models can predict whether a paper is\naccepted with up to 21% error reduction compared to the majority baseline. In\nthe second task, we predict the numerical scores of review aspects and show\nthat simple models can outperform the mean baseline for aspects with high\nvariance such as 'originality' and 'impact'.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09661v1",
            "date": "2018-04-25 16:26:39+00:00",
            "title": "Personalized Language Model for Query Auto-Completion",
            "authors": [
                "Aaron Jaech",
                "Mari Ostendorf"
            ],
            "abstract": "Query auto-completion is a search engine feature whereby the system suggests\ncompleted queries as the user types. Recently, the use of a recurrent neural\nnetwork language model was suggested as a method of generating query\ncompletions. We show how an adaptable language model can be used to generate\npersonalized completions and how the model can use online updating to make\npredictions for users not seen during training. The personalized predictions\nare significantly better than a baseline that uses no user information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09746v2",
            "date": "2018-04-25 18:36:17+00:00",
            "title": "Cheap Non-standard Analysis and Computability",
            "authors": [
                "Olivier Bournez",
                "Sabrina Ouazzani"
            ],
            "abstract": "Non standard analysis is an area of Mathematics dealing with notions of\ninfinitesimal and infinitely large numbers, in which many statements from\nclassical analysis can be expressed very naturally. Cheap non-standard analysis\nintroduced by Terence Tao in 2012 is based on the idea that considering that a\nproperty holds eventually is sufficient to give the essence of many of its\nstatements. This provides constructivity but at some (acceptable) price. We\nconsider computability in cheap non-standard analysis. We prove that many\nconcepts from computable analysis as well as several concepts from\ncomputability can be very elegantly and alternatively presented in this\nframework. It provides a dual view and dual proofs to several statements\nalready known in these fields.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1804.09849v2",
            "date": "2018-04-26 01:24:39+00:00",
            "title": "The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation",
            "authors": [
                "Mia Xu Chen",
                "Orhan Firat",
                "Ankur Bapna",
                "Melvin Johnson",
                "Wolfgang Macherey",
                "George Foster",
                "Llion Jones",
                "Niki Parmar",
                "Mike Schuster",
                "Zhifeng Chen",
                "Yonghui Wu",
                "Macduff Hughes"
            ],
            "abstract": "The past year has witnessed rapid advances in sequence-to-sequence (seq2seq)\nmodeling for Machine Translation (MT). The classic RNN-based approaches to MT\nwere first out-performed by the convolutional seq2seq model, which was then\nout-performed by the more recent Transformer model. Each of these new\napproaches consists of a fundamental architecture accompanied by a set of\nmodeling and training techniques that are in principle applicable to other\nseq2seq architectures. In this paper, we tease apart the new architectures and\ntheir accompanying techniques in two ways. First, we identify several key\nmodeling and training techniques, and apply them to the RNN architecture,\nyielding a new RNMT+ model that outperforms all of the three fundamental\narchitectures on the benchmark WMT'14 English to French and English to German\ntasks. Second, we analyze the properties of each fundamental seq2seq\narchitecture and devise new hybrid architectures intended to combine their\nstrengths. Our hybrid models obtain further improvements, outperforming the\nRNMT+ model on both benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09949v1",
            "date": "2018-04-26 09:06:06+00:00",
            "title": "Pay Attention to Virality: understanding popularity of social media videos with the attention mechanism",
            "authors": [
                "Adam Bielski",
                "Tomasz Trzcinski"
            ],
            "abstract": "Predicting popularity of social media videos before they are published is a\nchallenging task, mainly due to the complexity of content distribution network\nas well as the number of factors that play part in this process. As solving\nthis task provides tremendous help for media content creators, many successful\nmethods were proposed to solve this problem with machine learning. In this\nwork, we change the viewpoint and postulate that it is not only the predicted\npopularity that matters, but also, maybe even more importantly, understanding\nof how individual parts influence the final popularity score. To that end, we\npropose to combine the Grad-CAM visualization method with a soft attention\nmechanism. Our preliminary results show that this approach allows for more\nintuitive interpretation of the content impact on video popularity, while\nachieving competitive results in terms of prediction accuracy.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.10718v1",
            "date": "2018-04-28 01:19:51+00:00",
            "title": "Data-Driven Methods for Solving Algebra Word Problems",
            "authors": [
                "Benjamin Robaidek",
                "Rik Koncel-Kedziorski",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We explore contemporary, data-driven techniques for solving math word\nproblems over recent large-scale datasets. We show that well-tuned neural\nequation classifiers can outperform more sophisticated models such as sequence\nto sequence and self-attention across these datasets. Our error analysis\nindicates that, while fully data driven models show some promise, semantic and\nworld knowledge is necessary for further advances.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.10911v2",
            "date": "2018-04-29 11:57:15+00:00",
            "title": "A Tree Search Algorithm for Sequence Labeling",
            "authors": [
                "Yadi Lao",
                "Jun Xu",
                "Yanyan Lan",
                "Jiafeng Guo",
                "Sheng Gao",
                "Xueqi Cheng"
            ],
            "abstract": "In this paper we propose a novel reinforcement learning based model for\nsequence tagging, referred to as MM-Tag. Inspired by the success and\nmethodology of the AlphaGo Zero, MM-Tag formalizes the problem of sequence\ntagging with a Monte Carlo tree search (MCTS) enhanced Markov decision process\n(MDP) model, in which the time steps correspond to the positions of words in a\nsentence from left to right, and each action corresponds to assign a tag to a\nword. Two long short-term memory networks (LSTM) are used to summarize the past\ntag assignments and words in the sentence. Based on the outputs of LSTMs, the\npolicy for guiding the tag assignment and the value for predicting the whole\ntagging accuracy of the whole sentence are produced. The policy and value are\nthen strengthened with MCTS, which takes the produced raw policy and value as\ninputs, simulates and evaluates the possible tag assignments at the subsequent\npositions, and outputs a better search policy for assigning tags. A\nreinforcement learning algorithm is proposed to train the model parameters. Our\nwork is the first to apply the MCTS enhanced MDP model to the sequence tagging\ntask. We show that MM-Tag can accurately predict the tags thanks to the\nexploratory decision making mechanism introduced by MCTS. Experimental results\nshow based on a chunking benchmark showed that MM-Tag outperformed the\nstate-of-the-art sequence tagging baselines including CRF and CRF with LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11105v1",
            "date": "2018-04-30 09:54:12+00:00",
            "title": "Fast and scalable learning of neuro-symbolic representations of biomedical knowledge",
            "authors": [
                "Asan Agibetov",
                "Matthias Samwald"
            ],
            "abstract": "In this work we address the problem of fast and scalable learning of\nneuro-symbolic representations for general biological knowledge. Based on a\nrecently published comprehensive biological knowledge graph (Alshahrani, 2017)\nthat was used for demonstrating neuro-symbolic representation learning, we show\nhow to train fast (under 1 minute) log-linear neural embeddings of the\nentities. We utilize these representations as inputs for machine learning\nclassifiers to enable important tasks such as biological link prediction.\nClassifiers are trained by concatenating learned entity embeddings to represent\nentity relations, and training classifiers on the concatenated embeddings to\ndiscern true relations from automatically generated negative examples. Our\nsimple embedding methodology greatly improves on classification error compared\nto previously published state-of-the-art results, yielding a maximum increase\nof $+0.28$ F-measure and $+0.22$ ROC AUC scores for the most difficult\nbiological link prediction problem. Finally, our embedding approach is orders\nof magnitude faster to train ($\\leq$ 1 minute vs. hours), much more economical\nin terms of embedding dimensions ($d=50$ vs. $d=512$), and naturally encodes\nthe directionality of the asymmetric biological relations, that can be\ncontrolled by the order with which we concatenate the embeddings.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.11109v1",
            "date": "2018-04-30 10:06:46+00:00",
            "title": "Demand-Weighted Completeness Prediction for a Knowledge Base",
            "authors": [
                "Andrew Hopkinson",
                "Amit Gurdasani",
                "Dave Palfrey",
                "Arpit Mittal"
            ],
            "abstract": "In this paper we introduce the notion of Demand-Weighted Completeness,\nallowing estimation of the completeness of a knowledge base with respect to how\nit is used. Defining an entity by its classes, we employ usage data to predict\nthe distribution over relations for that entity. For example, instances of\nperson in a knowledge base may require a birth date, name and nationality to be\nconsidered complete. These predicted relation distributions enable detection of\nimportant gaps in the knowledge base, and define the required facts for unseen\nentities. Such characterisation of the knowledge base can also quantify how\nusage and completeness change over time. We demonstrate a method to measure\nDemand-Weighted Completeness, and show that a simple neural network model\nperforms well at this prediction task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.11149v2",
            "date": "2018-04-30 12:19:03+00:00",
            "title": "Q-Map: Clinical Concept Mining from Clinical Documents",
            "authors": [
                "Sheikh Shams Azam",
                "Manoj Raju",
                "Venkatesh Pagidimarri",
                "Vamsi Kasivajjala"
            ],
            "abstract": "Over the past decade, there has been a steep rise in the data-driven analysis\nin major areas of medicine, such as clinical decision support system, survival\nanalysis, patient similarity analysis, image analytics etc. Most of the data in\nthe field are well-structured and available in numerical or categorical formats\nwhich can be used for experiments directly. But on the opposite end of the\nspectrum, there exists a wide expanse of data that is intractable for direct\nanalysis owing to its unstructured nature which can be found in the form of\ndischarge summaries, clinical notes, procedural notes which are in human\nwritten narrative format and neither have any relational model nor any standard\ngrammatical structure. An important step in the utilization of these texts for\nsuch studies is to transform and process the data to retrieve structured\ninformation from the haystack of irrelevant data using information retrieval\nand data mining techniques. To address this problem, the authors present Q-Map\nin this paper, which is a simple yet robust system that can sift through\nmassive datasets with unregulated formats to retrieve structured information\naggressively and efficiently. It is backed by an effective mining technique\nwhich is based on a string matching algorithm that is indexed on curated\nknowledge sources, that is both fast and configurable. The authors also briefly\nexamine its comparative performance with MetaMap, one of the most reputed tools\nfor medical concepts retrieval and present the advantages the former displays\nover the latter.",
            "journal ref": "International Journal of Computer and Information Engineering,\n  12(9), 2018, 691 - 696",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.11297v2",
            "date": "2018-04-30 16:19:51+00:00",
            "title": "Sampling strategies in Siamese Networks for unsupervised speech representation learning",
            "authors": [
                "Rachid Riad",
                "Corentin Dancette",
                "Julien Karadayi",
                "Neil Zeghidour",
                "Thomas Schatz",
                "Emmanuel Dupoux"
            ],
            "abstract": "Recent studies have investigated siamese network architectures for learning\ninvariant speech representations using same-different side information at the\nword level. Here we investigate systematically an often ignored component of\nsiamese networks: the sampling procedure (how pairs of same vs. different\ntokens are selected). We show that sampling strategies taking into account\nZipf's Law, the distribution of speakers and the proportions of same and\ndifferent pairs of words significantly impact the performance of the network.\nIn particular, we show that word frequency compression improves learning across\na large range of variations in number of training pairs. This effect does not\napply to the same extent to the fully unsupervised setting, where the pairs of\nsame-different words are obtained by spoken term discovery. We apply these\nresults to pairs of words discovered using an unsupervised algorithm and show\nan improvement on state-of-the-art in unsupervised representation learning\nusing siamese networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00103v3",
            "date": "2018-07-31 22:53:30+00:00",
            "title": "An Ontology-Based Recommender System with an Application to the Star Trek Television Franchise",
            "authors": [
                "Paul Sheridan",
                "Mikael Onsj\u00f6",
                "Claudia Becerra",
                "Sergio Jimenez",
                "George Due\u00f1as"
            ],
            "abstract": "Collaborative filtering based recommender systems have proven to be extremely\nsuccessful in settings where user preference data on items is abundant.\nHowever, collaborative filtering algorithms are hindered by their weakness\nagainst the item cold-start problem and general lack of interpretability.\nOntology-based recommender systems exploit hierarchical organizations of users\nand items to enhance browsing, recommendation, and profile construction. While\nontology-based approaches address the shortcomings of their collaborative\nfiltering counterparts, ontological organizations of items can be difficult to\nobtain for items that mostly belong to the same category (e.g., television\nseries episodes). In this paper, we present an ontology-based recommender\nsystem that integrates the knowledge represented in a large ontology of\nliterary themes to produce fiction content recommendations. The main novelty of\nthis work is an ontology-based method for computing similarities between items\nand its integration with the classical Item-KNN (K-nearest neighbors)\nalgorithm. As a study case, we evaluated the proposed method against other\napproaches by performing the classical rating prediction task on a collection\nof Star Trek television series episodes in an item cold-start scenario. This\ntransverse evaluation provides insights into the utility of different\ninformation resources and methods for the initial stages of recommender system\ndevelopment. We found our proposed method to be a convenient alternative to\ncollaborative filtering approaches for collections of mostly similar items,\nparticularly when other content-based approaches are not applicable or\notherwise unavailable. Aside from the new methods, this paper contributes a\ntestbed for future research and an online framework to collaboratively extend\nthe ontology of literary themes to cover other narrative content.",
            "journal ref": "Future Internet 2019, 11(9), 182",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.00957v1",
            "date": "2018-08-02 09:02:00+00:00",
            "title": "SWDE : A Sub-Word And Document Embedding Based Engine for Clickbait Detection",
            "authors": [
                "Vaibhav Kumar",
                "Mrinal Dhar",
                "Dhruv Khattar",
                "Yash Kumar Lal",
                "Abhimanshu Mishra",
                "Manish Shrivastava",
                "Vasudeva Varma"
            ],
            "abstract": "In order to expand their reach and increase website ad revenue, media outlets\nhave started using clickbait techniques to lure readers to click on articles on\ntheir digital platform. Having successfully enticed the user to open the\narticle, the article fails to satiate his curiosity serving only to boost\nclick-through rates. Initial methods for this task were dependent on feature\nengineering, which varies with each dataset. Industry systems have relied on an\nexhaustive set of rules to get the job done. Neural networks have barely been\nexplored to perform this task. We propose a novel approach considering\ndifferent textual embeddings of a news headline and the related article. We\ngenerate sub-word level embeddings of the title using Convolutional Neural\nNetworks and use them to train a bidirectional LSTM architecture. An attention\nlayer allows for calculation of significance of each term towards the nature of\nthe post. We also generate Doc2Vec embeddings of the title and article text and\nmodel how they interact, following which it is concatenated with the output of\nthe previous component. Finally, this representation is passed through a neural\nnetwork to obtain a score for the headline. We test our model over 2538 posts\n(having trained it on 17000 records) and achieve an accuracy of 83.49%\noutscoring previous state-of-the-art approaches.",
            "journal ref": "\"SWDE : A Sub-Word And Document Embedding Based Engine for\n  Clickbait Detection\". In Proceedings of SIGIR 2018 Workshop on Computational\n  Surprise in Information Retrieval, Ann Arbor, MI, USA, July 8-12 (CompS'18,\n  SIGIR), 4 pages",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.01741v2",
            "date": "2018-08-06 06:20:41+00:00",
            "title": "Logical Semantics and Commonsense Knowledge: Where Did we Go Wrong, and How to Go Forward, Again",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "We argue that logical semantics might have faltered due to its failure in\ndistinguishing between two fundamentally very different types of concepts:\nontological concepts, that should be types in a strongly-typed ontology, and\nlogical concepts, that are predicates corresponding to properties of and\nrelations between objects of various ontological types. We will then show that\naccounting for these differences amounts to the integration of lexical and\ncompositional semantics in one coherent framework, and to an embedding in our\nlogical semantics of a strongly-typed ontology that reflects our commonsense\nview of the world and the way we talk about it in ordinary language. We will\nshow that in such a framework a number of challenges in natural language\nsemantics can be adequately and systematically treated.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.01742v1",
            "date": "2018-08-06 06:22:17+00:00",
            "title": "Using Linguistic Cues for Analyzing Social Movements",
            "authors": [
                "Rezvaneh Rezapour"
            ],
            "abstract": "With the growth of social media usage, social activists try to leverage this\nplatform to raise the awareness related to a social issue and engage the public\nworldwide. The broad use of social media platforms in recent years, made it\neasier for the people to stay up-to-date on the news related to regional and\nworldwide events. While social media, namely Twitter, assists social movements\nto connect with more people and mobilize the movement, traditional media such\nas news articles help in spreading the news related to the events in a broader\naspect. In this study, we analyze linguistic features and cues, such as\nindividualism vs. pluralism, sentiment and emotion to examine the relationship\nbetween the medium and discourse over time. We conduct this work in a specific\napplication context, the \"Black Lives Matter\" (BLM) movement, and compare\ndiscussions related to this event in social media vs. news articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02290v1",
            "date": "2018-08-07 10:14:38+00:00",
            "title": "How did the discussion go: Discourse act classification in social media conversations",
            "authors": [
                "Subhabrata Dutta",
                "Tanmoy Chakraborty",
                "Dipankar Das"
            ],
            "abstract": "We propose a novel attention based hierarchical LSTM model to classify\ndiscourse act sequences in social media conversations, aimed at mining data\nfrom online discussion using textual meanings beyond sentence level. The very\nuniqueness of the task is the complete categorization of possible pragmatic\nroles in informal textual discussions, contrary to extraction of\nquestion-answers, stance detection or sarcasm identification which are very\nmuch role specific tasks. Early attempt was made on a Reddit discussion\ndataset. We train our model on the same data, and present test results on two\ndifferent datasets, one from Reddit and one from Facebook. Our proposed model\noutperformed the previous one in terms of domain independence; without using\nplatform-dependent structural features, our hierarchical LSTM with word\nrelevance attention mechanism achieved F1-scores of 71\\% and 66\\% respectively\nto predict discourse roles of comments in Reddit and Facebook discussions.\nEfficiency of recurrent and convolutional architectures in order to learn\ndiscursive representation on the same task has been presented and analyzed,\nwith different word and comment embedding schemes. Our attention mechanism\nenables us to inquire into relevance ordering of text segments according to\ntheir roles in discourse. We present a human annotator experiment to unveil\nimportant observations about modeling and data annotation. Equipped with our\ntext-based discourse identification model, we inquire into how heterogeneous\nnon-textual features like location, time, leaning of information etc. play\ntheir roles in charaterizing online discussions on Facebook.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02504v1",
            "date": "2018-08-07 18:19:30+00:00",
            "title": "Device-directed Utterance Detection",
            "authors": [
                "Sri Harish Mallidi",
                "Roland Maas",
                "Kyle Goehner",
                "Ariya Rastrow",
                "Spyros Matsoukas",
                "Bj\u00f6rn Hoffmeister"
            ],
            "abstract": "In this work, we propose a classifier for distinguishing device-directed\nqueries from background speech in the context of interactions with voice\nassistants. Applications include rejection of false wake-ups or unintended\ninteractions as well as enabling wake-word free follow-up queries. Consider the\nexample interaction: $\"Computer,~play~music\", \"Computer,~reduce~the~volume\"$.\nIn this interaction, the user needs to repeat the wake-word ($Computer$) for\nthe second query. To allow for more natural interactions, the device could\nimmediately re-enter listening state after the first query (without wake-word\nrepetition) and accept or reject a potential follow-up as device-directed or\nbackground speech. The proposed model consists of two long short-term memory\n(LSTM) neural networks trained on acoustic features and automatic speech\nrecognition (ASR) 1-best hypotheses, respectively. A feed-forward deep neural\nnetwork (DNN) is then trained to combine the acoustic and 1-best embeddings,\nderived from the LSTMs, with features from the ASR decoder. Experimental\nresults show that ASR decoder, acoustic embeddings, and 1-best embeddings yield\nan equal-error-rate (EER) of $9.3~\\%$, $10.9~\\%$ and $20.1~\\%$, respectively.\nCombination of the features resulted in a $44~\\%$ relative improvement and a\nfinal EER of $5.2~\\%$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02608v1",
            "date": "2018-08-08 03:05:11+00:00",
            "title": "End-to-end Speech Recognition with Word-based RNN Language Models",
            "authors": [
                "Takaaki Hori",
                "Jaejin Cho",
                "Shinji Watanabe"
            ],
            "abstract": "This paper investigates the impact of word-based RNN language models\n(RNN-LMs) on the performance of end-to-end automatic speech recognition (ASR).\nIn our prior work, we have proposed a multi-level LM, in which character-based\nand word-based RNN-LMs are combined in hybrid CTC/attention-based ASR. Although\nthis multi-level approach achieves significant error reduction in the Wall\nStreet Journal (WSJ) task, two different LMs need to be trained and used for\ndecoding, which increase the computational cost and memory usage. In this\npaper, we further propose a novel word-based RNN-LM, which allows us to decode\nwith only the word-based LM, where it provides look-ahead word probabilities to\npredict next characters instead of the character-based LM, leading competitive\naccuracy with less computation compared to the multi-level LM. We demonstrate\nthe efficacy of the word-based RNN-LMs using a larger corpus, LibriSpeech, in\naddition to WSJ we used in the prior work. Furthermore, we show that the\nproposed model achieves 5.1 %WER for WSJ Eval'92 test set when the vocabulary\nsize is increased, which is the best WER reported for end-to-end ASR systems on\nthis benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02636v2",
            "date": "2018-08-08 06:04:35+00:00",
            "title": "Cognitive system to achieve human-level accuracy in automated assignment of helpdesk email tickets",
            "authors": [
                "Atri Mandal",
                "Nikhil Malhotra",
                "Shivali Agarwal",
                "Anupama Ray",
                "Giriprasad Sridhara"
            ],
            "abstract": "Ticket assignment/dispatch is a crucial part of service delivery business\nwith lot of scope for automation and optimization. In this paper, we present an\nend-to-end automated helpdesk email ticket assignment system, which is also\noffered as a service. The objective of the system is to determine the nature of\nthe problem mentioned in an incoming email ticket and then automatically\ndispatch it to an appropriate resolver group (or team) for resolution.\n  The proposed system uses an ensemble classifier augmented with a configurable\nrule engine. While design of classifier that is accurate is one of the main\nchallenges, we also need to address the need of designing a system that is\nrobust and adaptive to changing business needs. We discuss some of the main\ndesign challenges associated with email ticket assignment automation and how we\nsolve them. The design decisions for our system are driven by high accuracy,\ncoverage, business continuity, scalability and optimal usage of computational\nresources.\n  Our system has been deployed in production of three major service providers\nand currently assigning over 40,000 emails per month, on an average, with an\naccuracy close to 90% and covering at least 90% of email tickets. This\ntranslates to achieving human-level accuracy and results in a net saving of\nabout 23000 man-hours of effort per annum.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.03027v1",
            "date": "2018-08-09 05:42:49+00:00",
            "title": "Sentimental Content Analysis and Knowledge Extraction from News Articles",
            "authors": [
                "Mohammad Kamel",
                "Neda Keyvani",
                "Hadi Sadoghi Yazdi"
            ],
            "abstract": "In web era, since technology has revolutionized mankind life, plenty of data\nand information are published on the Internet each day. For instance, news\nagencies publish news on their websites all over the world. These raw data\ncould be an important resource for knowledge extraction. These shared data\ncontain emotions (i.e., positive, neutral or negative) toward various topics;\ntherefore, sentimental content extraction could be a beneficial task in many\naspects. Extracting the sentiment of news illustrates highly valuable\ninformation about the events over a period of time, the viewpoint of a media or\nnews agency to these events. In this paper an attempt is made to propose an\napproach for news analysis and extracting useful knowledge from them. Firstly,\nwe attempt to extract a noise robust sentiment of news documents; therefore,\nthe news associated to six countries: United State, United Kingdom, Germany,\nCanada, France and Australia in 5 different news categories: Politics, Sports,\nBusiness, Entertainment and Technology are downloaded. In this paper we compare\nthe condition of different countries in each 5 news topics based on the\nextracted sentiments and emotional contents in news documents. Moreover, we\npropose an approach to reduce the bulky news data to extract the hottest topics\nand news titles as a knowledge. Eventually, we generate a word model to map\neach word to a fixed-size vector by Word2Vec in order to understand the\nrelations between words in our collected news database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03090v1",
            "date": "2018-08-09 11:17:38+00:00",
            "title": "Image Inspired Poetry Generation in XiaoIce",
            "authors": [
                "Wen-Feng Cheng",
                "Chao-Chung Wu",
                "Ruihua Song",
                "Jianlong Fu",
                "Xing Xie",
                "Jian-Yun Nie"
            ],
            "abstract": "Vision is a common source of inspiration for poetry. The objects and the\nsentimental imprints that one perceives from an image may lead to various\nfeelings depending on the reader. In this paper, we present a system of poetry\ngeneration from images to mimic the process. Given an image, we first extract a\nfew keywords representing objects and sentiments perceived from the image.\nThese keywords are then expanded to related ones based on their associations in\nhuman written poems. Finally, verses are generated gradually from the keywords\nusing recurrent neural networks trained on existing poems. Our approach is\nevaluated by human assessors and compared to other generation baselines. The\nresults show that our method can generate poems that are more artistic than the\nbaseline methods. This is one of the few attempts to generate poetry from\nimages. By deploying our proposed approach, XiaoIce has already generated more\nthan 12 million poems for users since its release in July 2017. A book of its\npoems has been published by Cheers Publishing, which claimed that the book is\nthe first-ever poetry collection written by an AI in human history.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.03507v1",
            "date": "2018-08-10 12:33:45+00:00",
            "title": "Making effective use of healthcare data using data-to-text technology",
            "authors": [
                "Steffen Pauws",
                "Albert Gatt",
                "Emiel Krahmer",
                "Ehud Reiter"
            ],
            "abstract": "Healthcare organizations are in a continuous effort to improve health\noutcomes, reduce costs and enhance patient experience of care. Data is\nessential to measure and help achieving these improvements in healthcare\ndelivery. Consequently, a data influx from various clinical, financial and\noperational sources is now overtaking healthcare organizations and their\npatients. The effective use of this data, however, is a major challenge.\nClearly, text is an important medium to make data accessible. Financial reports\nare produced to assess healthcare organizations on some key performance\nindicators to steer their healthcare delivery. Similarly, at a clinical level,\ndata on patient status is conveyed by means of textual descriptions to\nfacilitate patient review, shift handover and care transitions. Likewise,\npatients are informed about data on their health status and treatments via\ntext, in the form of reports or via ehealth platforms by their doctors.\nUnfortunately, such text is the outcome of a highly labour-intensive process if\nit is done by healthcare professionals. It is also prone to incompleteness,\nsubjectivity and hard to scale up to different domains, wider audiences and\nvarying communication purposes. Data-to-text is a recent breakthrough\ntechnology in artificial intelligence which automatically generates natural\nlanguage in the form of text or speech from data. This chapter provides a\nsurvey of data-to-text technology, with a focus on how it can be deployed in a\nhealthcare setting. It will (1) give an up-to-date synthesis of data-to-text\napproaches, (2) give a categorized overview of use cases in healthcare, (3)\nseek to make a strong case for evaluating and implementing data-to-text in a\nhealthcare setting, and (4) highlight recent research challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03752v1",
            "date": "2018-08-11 05:05:06+00:00",
            "title": "Knowledge Graph Embedding with Entity Neighbors and Deep Memory Network",
            "authors": [
                "Kai Wang",
                "Yu Liu",
                "Xiujuan Xu",
                "Dan Lin"
            ],
            "abstract": "Knowledge Graph Embedding (KGE) aims to represent entities and relations of\nknowledge graph in a low-dimensional continuous vector space. Recent works\nfocus on incorporating structural knowledge with additional information, such\nas entity descriptions, relation paths and so on. However, common used\nadditional information usually contains plenty of noise, which makes it hard to\nlearn valuable representation. In this paper, we propose a new kind of\nadditional information, called entity neighbors, which contain both semantic\nand topological features about given entity. We then develop a deep memory\nnetwork model to encode information from neighbors. Employing a gating\nmechanism, representations of structure and neighbors are integrated into a\njoint representation. The experimental results show that our model outperforms\nexisting KGE methods utilizing entity descriptions and achieves\nstate-of-the-art metrics on 4 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03926v1",
            "date": "2018-08-12 11:53:04+00:00",
            "title": "Sequence Labeling: A Practical Approach",
            "authors": [
                "Adnan Akhundov",
                "Dietrich Trautmann",
                "Georg Groh"
            ],
            "abstract": "We take a practical approach to solving sequence labeling problem assuming\nunavailability of domain expertise and scarcity of informational and\ncomputational resources. To this end, we utilize a universal end-to-end\nBi-LSTM-based neural sequence labeling model applicable to a wide range of NLP\ntasks and languages. The model combines morphological, semantic, and structural\ncues extracted from data to arrive at informed predictions. The model's\nperformance is evaluated on eight benchmark datasets (covering three tasks:\nPOS-tagging, NER, and Chunking, and four languages: English, German, Dutch, and\nSpanish). We observe state-of-the-art results on four of them: CoNLL-2012\n(English NER), CoNLL-2002 (Dutch NER), GermEval 2014 (German NER), Tiger Corpus\n(German POS-tagging), and competitive performance on the rest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03967v1",
            "date": "2018-08-12 16:32:18+00:00",
            "title": "Augmenting word2vec with latent Dirichlet allocation within a clinical application",
            "authors": [
                "Akshay Budhkar",
                "Frank Rudzicz"
            ],
            "abstract": "This paper presents three hybrid models that directly combine latent\nDirichlet allocation and word embedding for distinguishing between speakers\nwith and without Alzheimer's disease from transcripts of picture descriptions.\nTwo of our models get F-scores over the current state-of-the-art using\nautomatic methods on the DementiaBank dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04122v3",
            "date": "2018-08-13 09:35:44+00:00",
            "title": "A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization",
            "authors": [
                "Dai Quoc Nguyen",
                "Thanh Vu",
                "Tu Dinh Nguyen",
                "Dat Quoc Nguyen",
                "Dinh Phung"
            ],
            "abstract": "In this paper, we introduce an embedding model, named CapsE, exploring a\ncapsule network to model relationship triples (subject, relation, object). Our\nCapsE represents each triple as a 3-column matrix where each column vector\nrepresents the embedding of an element in the triple. This 3-column matrix is\nthen fed to a convolution layer where multiple filters are operated to generate\ndifferent feature maps. These feature maps are reconstructed into corresponding\ncapsules which are then routed to another capsule to produce a continuous\nvector. The length of this vector is used to measure the plausibility score of\nthe triple. Our proposed CapsE obtains better performance than previous\nstate-of-the-art embedding models for knowledge graph completion on two\nbenchmark datasets WN18RR and FB15k-237, and outperforms strong search\npersonalization baselines on SEARCH17.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04127v1",
            "date": "2018-08-13 09:51:46+00:00",
            "title": "Learning Explanations from Language Data",
            "authors": [
                "David Harbecke",
                "Robert Schwarzenberg",
                "Christoph Alt"
            ],
            "abstract": "PatternAttribution is a recent method, introduced in the vision domain, that\nexplains classifications of deep neural networks. We demonstrate that it also\ngenerates meaningful interpretations in the language domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04217v4",
            "date": "2018-08-10 08:15:01+00:00",
            "title": "Unsupervised Learning of Sentence Representations Using Sequence Consistency",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "Computing universal distributed representations of sentences is a fundamental\ntask in natural language processing. We propose ConsSent, a simple yet\nsurprisingly powerful unsupervised method to learn such representations by\nenforcing consistency constraints on sequences of tokens. We consider two\nclasses of such constraints -- sequences that form a sentence and between two\nsequences that form a sentence when merged. We learn sentence encoders by\ntraining them to distinguish between consistent and inconsistent examples, the\nlatter being generated by randomly perturbing consistent examples in six\ndifferent ways. Extensive evaluation on several transfer learning and\nlinguistic probing tasks shows improved performance over strong unsupervised\nand supervised baselines, substantially surpassing them in several cases. Our\nbest results are achieved by training sentence encoders in a multitask setting\nand by an ensemble of encoders trained on the individual tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04254v1",
            "date": "2018-08-10 12:40:58+00:00",
            "title": "Homophonic Quotients of Linguistic Free Groups: German, Korean, and Turkish",
            "authors": [
                "Herbert Gangl",
                "Gizem Karaali",
                "Woohyung Lee"
            ],
            "abstract": "In 1993, the homophonic quotient groups for French and English (the quotient\nof the free group generated by the French (respectively English) alphabet\ndetermined by relations representing standard pronunciation rules) were\nexplicitly characterized [5]. In this paper we apply the same methodology to\nthree different language systems: German, Korean, and Turkish. We argue that\nour results point to some interesting differences between these three languages\n(or at least their current script systems).",
            "journal ref": "Revised and final version at: Involve 12:3 (2019) 463-474",
            "category": "math.GR"
        },
        {
            "url": "http://arxiv.org/abs/1808.04343v3",
            "date": "2018-08-13 17:38:54+00:00",
            "title": "REGMAPR - Text Matching Made Easy",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "Text matching is a fundamental problem in natural language processing. Neural\nmodels using bidirectional LSTMs for sentence encoding and inter-sentence\nattention mechanisms perform remarkably well on several benchmark datasets. We\npropose REGMAPR - a simple and general architecture for text matching that does\nnot use inter-sentence attention. Starting from a Siamese architecture, we\naugment the embeddings of the words with two features based on exact and para-\nphrase match between words in the two sentences. We train the model using three\ntypes of regularization on datasets for textual entailment, paraphrase\ndetection and semantic related- ness. REGMAPR performs comparably or better\nthan more complex neural models or models using a large number of handcrafted\nfeatures. REGMAPR achieves state-of-the-art results for paraphrase detection on\nthe SICK dataset and for textual entailment on the SNLI dataset among models\nthat do not use inter-sentence attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04364v1",
            "date": "2018-08-13 10:18:54+00:00",
            "title": "D-PAGE: Diverse Paraphrase Generation",
            "authors": [
                "Qiongkai Xu",
                "Juyan Zhang",
                "Lizhen Qu",
                "Lexing Xie",
                "Richard Nock"
            ],
            "abstract": "In this paper, we investigate the diversity aspect of paraphrase generation.\nPrior deep learning models employ either decoding methods or add random input\nnoise for varying outputs. We propose a simple method Diverse Paraphrase\nGeneration (D-PAGE), which extends neural machine translation (NMT) models to\nsupport the generation of diverse paraphrases with implicit rewriting patterns.\nOur experimental results on two real-world benchmark datasets demonstrate that\nour model generates at least one order of magnitude more diverse outputs than\nthe baselines in terms of a new evaluation metric Jeffrey's Divergence. We have\nalso conducted extensive experiments to understand various properties of our\nmodel with a focus on diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04365v1",
            "date": "2018-08-13 11:50:03+00:00",
            "title": "What is wrong with style transfer for texts?",
            "authors": [
                "Alexey Tikhonov",
                "Ivan P. Yamshchikov"
            ],
            "abstract": "A number of recent machine learning papers work with an automated style\ntransfer for texts and, counter to intuition, demonstrate that there is no\nconsensus formulation of this NLP task. Different researchers propose different\nalgorithms, datasets and target metrics to address it. This short opinion paper\naims to discuss possible formalization of this NLP task in anticipation of a\nfurther growing interest to it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04614v1",
            "date": "2018-08-14 10:23:32+00:00",
            "title": "Explaining Queries over Web Tables to Non-Experts",
            "authors": [
                "Jonathan Berant",
                "Daniel Deutch",
                "Amir Globerson",
                "Tova Milo",
                "Tomer Wolfson"
            ],
            "abstract": "Designing a reliable natural language (NL) interface for querying tables has\nbeen a longtime goal of researchers in both the data management and natural\nlanguage processing (NLP) communities. Such an interface receives as input an\nNL question, translates it into a formal query, executes the query and returns\nthe results. Errors in the translation process are not uncommon, and users\ntypically struggle to understand whether their query has been mapped correctly.\nWe address this problem by explaining the obtained formal queries to non-expert\nusers. Two methods for query explanations are presented: the first translates\nqueries into NL, while the second method provides a graphic representation of\nthe query cell-based provenance (in its execution on a given table). Our\nsolution augments a state-of-the-art NL interface over web tables, enhancing it\nin both its training and deployment phase. Experiments, including a user study\nconducted on Amazon Mechanical Turk, show our solution to improve both the\ncorrectness and reliability of an NL interface.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04670v2",
            "date": "2018-08-14 13:15:43+00:00",
            "title": "R-grams: Unsupervised Learning of Semantic Units in Natural Language",
            "authors": [
                "Ariel Ekgren",
                "Amaru Cuba Gyllensten",
                "Magnus Sahlgren"
            ],
            "abstract": "This paper investigates data-driven segmentation using Re-Pair or Byte Pair\nEncoding-techniques. In contrast to previous work which has primarily been\nfocused on subword units for machine translation, we are interested in the\ngeneral properties of such segments above the word level. We call these\nsegments r-grams, and discuss their properties and the effect they have on the\ntoken frequency distribution. The proposed approach is evaluated by\ndemonstrating its viability in embedding techniques, both in monolingual and\nmultilingual test settings. We also provide a number of qualitative examples of\nthe proposed methodology, demonstrating its viability as a language-invariant\nsegmentation procedure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04694v1",
            "date": "2018-08-10 04:24:34+00:00",
            "title": "A Hassle-Free Machine Learning Method for Cohort Selection of Clinical Trials",
            "authors": [
                "Liu Man"
            ],
            "abstract": "Traditional text classification techniques in clinical domain have heavily\nrelied on the manually extracted textual cues. This paper proposes a generally\nsupervised machine learning method that is equally hassle-free and does not use\nclinical knowledge. The employed methods were simple to implement, fast to run\nand yet effective. This paper proposes a novel named entity recognition (NER)\nbased an ensemble system capable of learning the keyword features in the\ndocument. Instead of merely considering the whole sentence/paragraph for\nanalysis, the NER based keyword features can stress the important clinic\nrelevant phases more. In addition, to capture the semantic information in the\ndocuments, the FastText features originating from the document level FastText\nclassification results are exploited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04911v2",
            "date": "2018-08-14 22:04:34+00:00",
            "title": "Cross-Lingual Cross-Platform Rumor Verification Pivoting on Multimedia Content",
            "authors": [
                "Weiming Wen",
                "Songwen Su",
                "Zhou Yu"
            ],
            "abstract": "With the increasing popularity of smart devices, rumors with multimedia\ncontent become more and more common on social networks. The multimedia\ninformation usually makes rumors look more convincing. Therefore, finding an\nautomatic approach to verify rumors with multimedia content is a pressing task.\nPrevious rumor verification research only utilizes multimedia as input\nfeatures. We propose not to use the multimedia content but to find external\ninformation in other news platforms pivoting on it. We introduce a new features\nset, cross-lingual cross-platform features that leverage the semantic\nsimilarity between the rumors and the external information. When implemented,\nmachine learning methods utilizing such features achieved the state-of-the-art\nrumor verification results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05312v1",
            "date": "2018-08-16 00:24:49+00:00",
            "title": "Toward domain-invariant speech recognition via large scale training",
            "authors": [
                "Arun Narayanan",
                "Ananya Misra",
                "Khe Chai Sim",
                "Golan Pundak",
                "Anshuman Tripathi",
                "Mohamed Elfeky",
                "Parisa Haghani",
                "Trevor Strohman",
                "Michiel Bacchiani"
            ],
            "abstract": "Current state-of-the-art automatic speech recognition systems are trained to\nwork in specific `domains', defined based on factors like application, sampling\nrate and codec. When such recognizers are used in conditions that do not match\nthe training domain, performance significantly drops. This work explores the\nidea of building a single domain-invariant model for varied use-cases by\ncombining large scale training data from multiple application domains. Our\nfinal system is trained using 162,000 hours of speech. Additionally, each\nutterance is artificially distorted during training to simulate effects like\nbackground noise, codec distortion, and sampling rates. Our results show that,\neven at such a scale, a model thus trained works almost as well as those\nfine-tuned to specific subsets: A single model can be robust to multiple\napplication domains, and variations like codecs and noise. More importantly,\nsuch models generalize better to unseen conditions and allow for rapid\nadaptation -- we show that by using as little as 10 hours of data from a new\ndomain, an adapted domain-invariant model can match performance of a\ndomain-specific model trained from scratch using 70 times as much data. We also\nhighlight some of the limitations of such models and areas that need addressing\nin future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05439v2",
            "date": "2018-08-16 12:14:07+00:00",
            "title": "Linguistic data mining with complex networks: a stylometric-oriented approach",
            "authors": [
                "Tomasz Stanisz",
                "Jaros\u0142aw Kwapie\u0144",
                "Stanis\u0142aw Dro\u017cd\u017c"
            ],
            "abstract": "By representing a text by a set of words and their co-occurrences, one\nobtains a word-adjacency network being a reduced representation of a given\nlanguage sample. In this paper, the possibility of using network representation\nto extract information about individual language styles of literary texts is\nstudied. By determining selected quantitative characteristics of the networks\nand applying machine learning algorithms, it is possible to distinguish between\ntexts of different authors. Within the studied set of texts, English and\nPolish, a properly rescaled weighted clustering coefficients and weighted\ndegrees of only a few nodes in the word-adjacency networks are sufficient to\nobtain the authorship attribution accuracy over 90%. A correspondence between\nthe text authorship and the word-adjacency network structure can therefore be\nfound. The network representation allows to distinguish individual language\nstyles by comparing the way the authors use particular words and punctuation\nmarks. The presented approach can be viewed as a generalization of the\nauthorship attribution methods based on simple lexical features.\n  Additionally, other network parameters are studied, both local and global\nones, for both the unweighted and weighted networks. Their potential to capture\nthe writing style diversity is discussed; some differences between languages\nare observed.",
            "journal ref": "Information Sciences 482, 301-320 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05599v2",
            "date": "2018-08-16 17:41:00+00:00",
            "title": "Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation",
            "authors": [
                "Yi-Lin Tuan",
                "Hung-Yi Lee"
            ],
            "abstract": "Sequence generative adversarial networks (SeqGAN) have been used to improve\nconditional sequence generation tasks, for example, chit-chat dialogue\ngeneration. To stabilize the training of SeqGAN, Monte Carlo tree search (MCTS)\nor reward at every generation step (REGS) is used to evaluate the goodness of a\ngenerated subsequence. MCTS is computationally intensive, but the performance\nof REGS is worse than MCTS. In this paper, we propose stepwise GAN (StepGAN),\nin which the discriminator is modified to automatically assign scores\nquantifying the goodness of each subsequence at every generation step. StepGAN\nhas significantly less computational costs than MCTS. We demonstrate that\nStepGAN outperforms previous GAN-based methods on both synthetic experiment and\nchit-chat dialogue generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05908v4",
            "date": "2018-08-14 18:44:58+00:00",
            "title": "Improved Language Modeling by Decoding the Past",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "Highly regularized LSTMs achieve impressive results on several benchmark\ndatasets in language modeling. We propose a new regularization method based on\ndecoding the last token in the context using the predicted distribution of the\nnext token. This biases the model towards retaining more contextual\ninformation, in turn improving its ability to predict the next token. With\nnegligible overhead in the number of parameters and training time, our Past\nDecode Regularization (PDR) method achieves a word level perplexity of 55.6 on\nthe Penn Treebank and 63.5 on the WikiText-2 datasets using a single softmax.\nWe also show gains by using PDR in combination with a mixture-of-softmaxes,\nachieving a word level perplexity of 53.8 and 60.5 on these datasets. In\naddition, our method achieves 1.169 bits-per-character on the Penn Treebank\nCharacter dataset for character level language modeling. These results\nconstitute a new state-of-the-art in their respective settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05946v1",
            "date": "2018-08-17 05:14:41+00:00",
            "title": "Disambiguating fine-grained place names from descriptions by clustering",
            "authors": [
                "Hao Chen",
                "Maria Vasardani",
                "Stephan Winter"
            ],
            "abstract": "Everyday place descriptions often contain place names of fine-grained\nfeatures, such as buildings or businesses, that are more difficult to\ndisambiguate than names referring to larger places, for example cities or\nnatural geographic features. Fine-grained places are often significantly more\nfrequent and more similar to each other, and disambiguation heuristics\ndeveloped for larger places, such as those based on population or containment\nrelationships, are often not applicable in these cases. In this research, we\naddress the disambiguation of fine-grained place names from everyday place\ndescriptions. For this purpose, we evaluate the performance of different\nexisting clustering-based approaches, since clustering approaches require no\nmore knowledge other than the locations of ambiguous place names. We consider\nnot only approaches developed specifically for place name disambiguation, but\nalso clustering algorithms developed for general data mining that could\npotentially be leveraged. We compare these methods with a novel algorithm, and\nshow that the novel algorithm outperforms the other algorithms in terms of\ndisambiguation precision and distance error over several tested datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.06244v2",
            "date": "2018-08-19 19:08:10+00:00",
            "title": "XL-NBT: A Cross-lingual Neural Belief Tracking Framework",
            "authors": [
                "Wenhu Chen",
                "Jianshu Chen",
                "Yu Su",
                "Xin Wang",
                "Dong Yu",
                "Xifeng Yan",
                "William Yang Wang"
            ],
            "abstract": "Task-oriented dialog systems are becoming pervasive, and many companies\nheavily rely on them to complement human agents for customer service in call\ncenters. With globalization, the need for providing cross-lingual customer\nsupport becomes more urgent than ever. However, cross-lingual support poses\ngreat challenges---it requires a large amount of additional annotated data from\nnative speakers. In order to bypass the expensive human annotation and achieve\nthe first step towards the ultimate goal of building a universal dialog system,\nwe set out to build a cross-lingual state tracking framework. Specifically, we\nassume that there exists a source language with dialog belief tracking\nannotations while the target languages have no annotated dialog data of any\nform. Then, we pre-train a state tracker for the source language as a teacher,\nwhich is able to exploit easy-to-access parallel data. We then distill and\ntransfer its own knowledge to the student state tracker in target languages. We\nspecifically discuss two types of common parallel resources: bilingual corpus\nand bilingual dictionary, and design different transfer learning strategies\naccordingly. Experimentally, we successfully use English state tracker as the\nteacher to transfer its knowledge to both Italian and German trackers and\nachieve promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06497v2",
            "date": "2018-08-20 15:04:30+00:00",
            "title": "Goal-oriented Dialogue Policy Learning from Failures",
            "authors": [
                "Keting Lu",
                "Shiqi Zhang",
                "Xiaoping Chen"
            ],
            "abstract": "Reinforcement learning methods have been used for learning dialogue policies.\nHowever, learning an effective dialogue policy frequently requires\nprohibitively many conversations. This is partly because of the sparse rewards\nin dialogues, and the very few successful dialogues in early learning phase.\nHindsight experience replay (HER) enables learning from failures, but the\nvanilla HER is inapplicable to dialogue learning due to the implicit goals. In\nthis work, we develop two complex HER methods providing different trade-offs\nbetween complexity and performance, and, for the first time, enabled HER-based\ndialogue policy learning. Experiments using a realistic user simulator show\nthat our HER methods perform better than existing experience replay methods (as\napplied to deep Q-networks) in learning rate.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.06511v2",
            "date": "2018-08-20 15:19:38+00:00",
            "title": "State-of-the-art Chinese Word Segmentation with Bi-LSTMs",
            "authors": [
                "Ji Ma",
                "Kuzman Ganchev",
                "David Weiss"
            ],
            "abstract": "A wide variety of neural-network architectures have been proposed for the\ntask of Chinese word segmentation.\n  Surprisingly, we find that a bidirectional LSTM model, when combined with\nstandard deep learning techniques and best practices, can achieve better\naccuracy on many of the popular datasets as compared to models based on more\ncomplex neural-network architectures.\n  Furthermore, our error analysis shows that out-of-vocabulary words remain\nchallenging for neural-network models, and many of the remaining errors are\nunlikely to be fixed through architecture changes.\n  Instead, more effort should be made on exploring resources for further\nimprovement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06570v3",
            "date": "2018-08-20 17:05:46+00:00",
            "title": "Detecting cognitive impairments by agreeing on interpretations of linguistic features",
            "authors": [
                "Zining Zhu",
                "Jekaterina Novikova",
                "Frank Rudzicz"
            ],
            "abstract": "Linguistic features have shown promising applications for detecting various\ncognitive impairments. To improve detection accuracies, increasing the amount\nof data or the number of linguistic features have been two applicable\napproaches. However, acquiring additional clinical data can be expensive, and\nhand-crafting features is burdensome. In this paper, we take a third approach,\nproposing Consensus Networks (CNs), a framework to classify after reaching\nagreements between modalities. We divide linguistic features into\nnon-overlapping subsets according to their modalities, and let neural networks\nlearn low-dimensional representations that agree with each other. These\nrepresentations are passed into a classifier network. All neural networks are\noptimized iteratively.\n  In this paper, we also present two methods that improve the performance of\nCNs. We then present ablation studies to illustrate the effectiveness of\nmodality division. To understand further what happens in CNs, we visualize the\nrepresentations during training. Overall, using all of the 413 linguistic\nfeatures, our models significantly outperform traditional classifiers, which\nare used by the state-of-the-art papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06740v2",
            "date": "2018-08-21 02:39:08+00:00",
            "title": "Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning",
            "authors": [
                "Ziyu Yao",
                "Xiujun Li",
                "Jianfeng Gao",
                "Brian Sadler",
                "Huan Sun"
            ],
            "abstract": "Given a text description, most existing semantic parsers synthesize a program\nin one shot. However, it is quite challenging to produce a correct program\nsolely based on the description, which in reality is often ambiguous or\nincomplete. In this paper, we investigate interactive semantic parsing, where\nthe agent can ask the user clarification questions to resolve ambiguities via a\nmulti-turn dialogue, on an important type of programs called \"If-Then recipes.\"\nWe develop a hierarchical reinforcement learning (HRL) based agent that\nsignificantly improves the parsing performance with minimal questions to the\nuser. Results under both simulation and human evaluation show that our agent\nsubstantially outperforms non-interactive semantic parsers and rule-based\nagents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06880v1",
            "date": "2018-08-21 12:53:52+00:00",
            "title": "Automatic Generation of Text Descriptive Comments for Code Blocks",
            "authors": [
                "Yuding Liang",
                "Kenny Q. Zhu"
            ],
            "abstract": "We propose a framework to automatically generate descriptive comments for\nsource code blocks. While this problem has been studied by many researchers\npreviously, their methods are mostly based on fixed template and achieves poor\nresults. Our framework does not rely on any template, but makes use of a new\nrecursive neural network called Code-RNN to extract features from the source\ncode and embed them into one vector. When this vector representation is input\nto a new recurrent neural network (Code-GRU), the overall framework generates\ntext descriptions of the code with accuracy (Rouge-2 value) significantly\nhigher than other learning-based approaches such as sequence-to-sequence model.\nThe Code-RNN model can also be used in other scenario where the representation\nof code is required.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.07104v1",
            "date": "2018-08-21 19:52:08+00:00",
            "title": "Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner",
            "authors": [
                "Yury Zemlyanskiy",
                "Fei Sha"
            ],
            "abstract": "There have been several attempts to define a plausible motivation for a\nchit-chat dialogue agent that can lead to engaging conversations. In this work,\nwe explore a new direction where the agent specifically focuses on discovering\ninformation about its interlocutor. We formalize this approach by defining a\nquantitative metric. We propose an algorithm for the agent to maximize it. We\nvalidate the idea with human evaluation where our system outperforms various\nbaselines. We demonstrate that the metric indeed correlates with the human\njudgments of engagingness.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.07166v1",
            "date": "2018-08-22 00:32:23+00:00",
            "title": "Deciding the status of controversial phonemes using frequency distributions; an application to semiconsonants in Spanish",
            "authors": [
                "Manuel Ortega-Rodr\u00edguez",
                "Hugo Sol\u00eds-S\u00e1nchez",
                "Ricardo Gamboa-Alfaro"
            ],
            "abstract": "Exploiting the fact that natural languages are complex systems, the present\nexploratory article proposes a direct method based on frequency distributions\nthat may be useful when making a decision on the status of problematic\nphonemes, an open problem in linguistics. The main notion is that natural\nlanguages, which can be considered from a complex outlook as information\nprocessing machines, and which somehow manage to set appropriate levels of\nredundancy, already \"made the choice\" whether a linguistic unit is a phoneme or\nnot, and this would be reflected in a greater smoothness in a frequency versus\nrank graph. For the particular case we chose to study, we conclude that it is\nreasonable to consider the Spanish semiconsonant /w/ as a separate phoneme from\nits vowel counterpart /u/, on the one hand, and possibly also the semiconsonant\n/j/ as a separate phoneme from its vowel counterpart /i/, on the other. As\nlanguage has been so central a topic in the study of complexity, this\ndiscussion grants us, in addition, an opportunity to gain insight into emerging\nproperties in the broader complex systems debate.",
            "journal ref": "Physica A 503 (2018) 1020-1029",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07191v1",
            "date": "2018-08-22 02:33:15+00:00",
            "title": "Identifying High-Quality Chinese News Comments Based on Multi-Target Text Matching Model",
            "authors": [
                "Deli Chen",
                "Shuming Ma",
                "Pengcheng Yang",
                "Xu Sun"
            ],
            "abstract": "With the development of information technology, there is an explosive growth\nin the number of online comment concerning news, blogs and so on. The massive\ncomments are overloaded, and often contain some misleading and unwelcome\ninformation. Therefore, it is necessary to identify high-quality comments and\nfilter out low-quality comments. In this work, we introduce a novel task:\nhigh-quality comment identification (HQCI), which aims to automatically assess\nthe quality of online comments. First, we construct a news comment corpus,\nwhich consists of news, comments, and the corresponding quality label. Second,\nwe analyze the dataset, and find the quality of comments can be measured in\nthree aspects: informativeness, consistency, and novelty. Finally, we propose a\nnovel multi-target text matching model, which can measure three aspects by\nreferring to the news and surrounding comments. Experimental results show that\nour method can outperform various baselines by a large margin on the news\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07325v3",
            "date": "2018-08-22 12:03:48+00:00",
            "title": "An Attention-Gated Convolutional Neural Network for Sentence Classification",
            "authors": [
                "Yang Liu",
                "Lixin Ji",
                "Ruiyang Huang",
                "Tuosiyu Ming",
                "Chao Gao",
                "Jianpeng Zhang"
            ],
            "abstract": "The classification of sentences is very challenging, since sentences contain\nthe limited contextual information. In this paper, we proposed an\nAttention-Gated Convolutional Neural Network (AGCNN) for sentence\nclassification, which generates attention weights from the feature's context\nwindows of different sizes by using specialized convolution encoders. It makes\nfull use of limited contextual information to extract and enhance the influence\nof important features in predicting the sentence's category. Experimental\nresults demonstrated that our model can achieve up to 3.1% higher accuracy than\nstandard CNN models, and gain competitive results over the baselines on four\nout of the six tasks. Besides, we designed an activation function, namely,\nNatural Logarithm rescaled Rectified Linear Unit (NLReLU). Experiments showed\nthat NLReLU can outperform ReLU and is comparable to other well-known\nactivation functions on AGCNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07374v2",
            "date": "2018-08-22 14:13:24+00:00",
            "title": "Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation",
            "authors": [
                "Junyang Lin",
                "Xu Sun",
                "Xuancheng Ren",
                "Muyu Li",
                "Qi Su"
            ],
            "abstract": "Most of the Neural Machine Translation (NMT) models are based on the\nsequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped\nwith the attention mechanism. However, the conventional attention mechanism\ntreats the decoding at each time step equally with the same matrix, which is\nproblematic since the softness of the attention for different types of words\n(e.g. content words and function words) should differ. Therefore, we propose a\nnew model with a mechanism called Self-Adaptive Control of Temperature (SACT)\nto control the softness of attention by means of an attention temperature.\nExperimental results on the Chinese-English translation and English-Vietnamese\ntranslation demonstrate that our model outperforms the baseline models, and the\nanalysis and the case study show that our model can attend to the most relevant\nelements in the source-side contexts and generate the translation of high\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07582v1",
            "date": "2018-08-22 22:32:34+00:00",
            "title": "TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks",
            "authors": [
                "Xinyue Liu",
                "Xiangnan Kong",
                "Lei Liu",
                "Kuorong Chiang"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have shown great capacity on image\ngeneration, in which a discriminative model guides the training of a generative\nmodel to construct images that resemble real images. Recently, GANs have been\nextended from generating images to generating sequences (e.g., poems, music and\ncodes). Existing GANs on sequence generation mainly focus on general sequences,\nwhich are grammar-free. In many real-world applications, however, we need to\ngenerate sequences in a formal language with the constraint of its\ncorresponding grammar. For example, to test the performance of a database, one\nmay want to generate a collection of SQL queries, which are not only similar to\nthe queries of real users, but also follow the SQL syntax of the target\ndatabase. Generating such sequences is highly challenging because both the\ngenerator and discriminator of GANs need to consider the structure of the\nsequences and the given grammar in the formal language. To address these\nissues, we study the problem of syntax-aware sequence generation with GANs, in\nwhich a collection of real sequences and a set of pre-defined grammatical rules\nare given to both discriminator and generator. We propose a novel GAN\nframework, namely TreeGAN, to incorporate a given Context-Free Grammar (CFG)\ninto the sequence generation process. In TreeGAN, the generator employs a\nrecurrent neural network (RNN) to construct a parse tree. Each generated parse\ntree can then be translated to a valid sequence of the given grammar. The\ndiscriminator uses a tree-structured RNN to distinguish the generated trees\nfrom real trees. We show that TreeGAN can generate sequences for any CFG and\nits generation fully conforms with the given syntax. Experiments on synthetic\nand real data sets demonstrated that TreeGAN significantly improves the quality\nof the sequence generation in context-free languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07624v1",
            "date": "2018-08-23 03:58:21+00:00",
            "title": "Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model",
            "authors": [
                "Kun Xu",
                "Lingfei Wu",
                "Zhiguo Wang",
                "Mo Yu",
                "Liwei Chen",
                "Vadim Sheinin"
            ],
            "abstract": "Existing neural semantic parsers mainly utilize a sequence encoder, i.e., a\nsequential LSTM, to extract word order features while neglecting other valuable\nsyntactic information such as dependency graph or constituent trees. In this\npaper, we first propose to use the \\textit{syntactic graph} to represent three\ntypes of syntactic information, i.e., word order, dependency and constituency\nfeatures. We further employ a graph-to-sequence model to encode the syntactic\ngraph and decode a logical form. Experimental results on benchmark datasets\nshow that our model is comparable to the state-of-the-art on Jobs640, ATIS and\nGeo880. Experimental results on adversarial examples demonstrate the robustness\nof the model is also improved by encoding more syntactic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07658v1",
            "date": "2018-08-23 08:07:44+00:00",
            "title": "Exploring Shared Structures and Hierarchies for Multiple NLP Tasks",
            "authors": [
                "Junkun Chen",
                "Kaiyu Chen",
                "Xinchi Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Designing shared neural architecture plays an important role in multi-task\nlearning. The challenge is that finding an optimal sharing scheme heavily\nrelies on the expert knowledge and is not scalable to a large number of diverse\ntasks. Inspired by the promising work of neural architecture search (NAS), we\napply reinforcement learning to automatically find possible shared architecture\nfor multi-task learning. Specifically, we use a controller to select from a set\nof shareable modules and assemble a task-specific architecture, and repeat the\nsame procedure for other tasks. The controller is trained with reinforcement\nlearning to maximize the expected accuracies for all tasks. We conduct\nextensive experiments on two types of tasks, text classification and sequence\nlabeling, which demonstrate the benefits of our approach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.08003v3",
            "date": "2018-08-24 05:00:17+00:00",
            "title": "Approximate Distribution Matching for Sequence-to-Sequence Learning",
            "authors": [
                "Wenhu Chen",
                "Guanlin Li",
                "Shujie Liu",
                "Zhirui Zhang",
                "Mu Li",
                "Ming Zhou"
            ],
            "abstract": "Sequence-to-Sequence models were introduced to tackle many real-life problems\nlike machine translation, summarization, image captioning, etc. The standard\noptimization algorithms are mainly based on example-to-example matching like\nmaximum likelihood estimation, which is known to suffer from data sparsity\nproblem. Here we present an alternate view to explain sequence-to-sequence\nlearning as a distribution matching problem, where each source or target\nexample is viewed to represent a local latent distribution in the source or\ntarget domain. Then, we interpret sequence-to-sequence learning as learning a\ntransductive model to transform the source local latent distributions to match\ntheir corresponding target distributions. In our framework, we approximate both\nthe source and target latent distributions with recurrent neural networks\n(augmenter). During training, the parallel augmenters learn to better\napproximate the local latent distributions, while the sequence prediction model\nlearns to minimize the KL-divergence of the transformed source distributions\nand the approximated target distributions. This algorithm can alleviate the\ndata sparsity issues in sequence learning by locally augmenting more unseen\ndata pairs and increasing the model's robustness. Experiments conducted on\nmachine translation and image captioning consistently demonstrate the\nsuperiority of our proposed algorithm over the other competing algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08079v2",
            "date": "2018-08-24 10:29:45+00:00",
            "title": "Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information",
            "authors": [
                "Mario Giulianelli",
                "Jack Harding",
                "Florian Mohnert",
                "Dieuwke Hupkes",
                "Willem Zuidema"
            ],
            "abstract": "How do neural language models keep track of number agreement between subject\nand verb? We show that `diagnostic classifiers', trained to predict number from\nthe internal states of a language model, provide a detailed understanding of\nhow, when, and where this information is represented. Moreover, they give us\ninsight into when and where number information is corrupted in cases where the\nlanguage model ends up making agreement errors. To demonstrate the causal role\nplayed by the representations we find, we then use agreement information to\ninfluence the course of the LSTM during the processing of difficult sentences.\nResults from such an intervention reveal a large increase in the language\nmodel's accuracy. Together, these results show that diagnostic classifiers give\nus an unrivalled detailed look into the representation of linguistic\ninformation in neural models, and demonstrate that this knowledge can be used\nto improve their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08357v1",
            "date": "2018-08-25 05:17:43+00:00",
            "title": "Dr. Tux: A Question Answering System for Ubuntu users",
            "authors": [
                "Bijil Abraham Philip",
                "Manas Jog",
                "Apurv Milind Upasani"
            ],
            "abstract": "Various forums and question answering (Q&A) sites are available online that\nallow Ubuntu users to find results similar to their queries. However, searching\nfor a result is often time consuming as it requires the user to find a specific\nproblem instance relevant to his/her query from a large set of questions. In\nthis paper, we present an automated question answering system for Ubuntu users\ncalled Dr. Tux that is designed to answer user's queries by selecting the most\nsimilar question from an online database. The prototype was implemented in\nPython and uses NLTK and CoreNLP tools for Natural Language Processing. The\ndata for the prototype was taken from the AskUbuntu website which contains\nabout 150k questions. The results obtained from the manual evaluation of the\nprototype were promising while also presenting some interesting opportunities\nfor improvement.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.08437v1",
            "date": "2018-08-25 15:10:59+00:00",
            "title": "Meta-Learning for Low-Resource Neural Machine Translation",
            "authors": [
                "Jiatao Gu",
                "Yong Wang",
                "Yun Chen",
                "Kyunghyun Cho",
                "Victor O. K. Li"
            ],
            "abstract": "In this paper, we propose to extend the recently introduced model-agnostic\nmeta-learning algorithm (MAML) for low-resource neural machine translation\n(NMT). We frame low-resource translation as a meta-learning problem, and we\nlearn to adapt to low-resource languages based on multilingual high-resource\nlanguage tasks. We use the universal lexical\nrepresentation~\\citep{gu2018universal} to overcome the input-output mismatch\nacross different languages. We evaluate the proposed meta-learning strategy\nusing eighteen European languages (Bg, Cs, Da, De, El, Es, Et, Fr, Hu, It, Lt,\nNl, Pl, Pt, Sk, Sl, Sv and Ru) as source tasks and five diverse languages (Ro,\nLv, Fi, Tr and Ko) as target tasks. We show that the proposed approach\nsignificantly outperforms the multilingual, transfer learning based\napproach~\\citep{zoph2016transfer} and enables us to train a competitive NMT\nsystem with only a fraction of training examples. For instance, the proposed\napproach can achieve as high as 22.04 BLEU on Romanian-English WMT'16 by seeing\nonly 16,000 translated words (~600 parallel sentences).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08470v1",
            "date": "2018-08-25 21:04:53+00:00",
            "title": "Representing Social Media Users for Sarcasm Detection",
            "authors": [
                "Y. Alex Kolchinski",
                "Christopher Potts"
            ],
            "abstract": "We explore two methods for representing authors in the context of textual\nsarcasm detection: a Bayesian approach that directly represents authors'\npropensities to be sarcastic, and a dense embedding approach that can learn\ninteractions between the author and the text. Using the SARC dataset of Reddit\ncomments, we show that augmenting a bidirectional RNN with these\nrepresentations improves performance; the Bayesian approach suffices in\nhomogeneous contexts, whereas the added power of the dense embeddings proves\nvaluable in more diverse ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08643v1",
            "date": "2018-08-26 23:31:02+00:00",
            "title": "Scientific Relation Extraction with Selectively Incorporated Concept Embeddings",
            "authors": [
                "Yi Luan",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "This paper describes our submission for the SemEval 2018 Task 7 shared task\non semantic relation extraction and classification in scientific papers. We\nextend the end-to-end relation extraction model of (Miwa and Bansal) with\nenhancements such as a character-level encoding attention mechanism on\nselecting pretrained concept candidate embeddings. Our official submission\nranked the second in relation classification task (Subtask 1.1 and Subtask 2\nSenerio 2), and the first in the relation extraction task (Subtask 2 Scenario\n1).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.08762v2",
            "date": "2018-08-27 09:50:56+00:00",
            "title": "Sentence Embeddings in NLI with Iterative Refinement Encoders",
            "authors": [
                "Aarne Talman",
                "Anssi Yli-Jyr\u00e4",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Sentence-level representations are necessary for various NLP tasks. Recurrent\nneural networks have proven to be very effective in learning distributed\nrepresentations and can be trained efficiently on natural language inference\ntasks. We build on top of one such model and propose a hierarchy of BiLSTM and\nmax pooling layers that implements an iterative refinement strategy and yields\nstate of the art results on the SciTail dataset as well as strong results for\nSNLI and MultiNLI. We can show that the sentence embeddings learned in this way\ncan be utilized in a wide variety of transfer learning tasks, outperforming\nInferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence\nembedding evaluation tasks. Furthermore, our model beats the InferSent model in\n8 out of 10 recently published SentEval probing tasks designed to evaluate\nsentence embeddings' ability to capture some of the important linguistic\nproperties of sentences.",
            "journal ref": "Nat. Lang. Eng. 25 (2019) 467-482",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08782v2",
            "date": "2018-08-27 11:04:55+00:00",
            "title": "Amobee at IEST 2018: Transfer Learning from Language Models",
            "authors": [
                "Alon Rozental",
                "Daniel Fleischer",
                "Zohar Kelrich"
            ],
            "abstract": "This paper describes the system developed at Amobee for the WASSA 2018\nimplicit emotions shared task (IEST). The goal of this task was to predict the\nemotion expressed by missing words in tweets without an explicit mention of\nthose words. We developed an ensemble system consisting of language models\ntogether with LSTM-based networks containing a CNN attention mechanism. Our\napproach represents a novel use of language models (specifically trained on a\nlarge Twitter dataset) to predict and classify emotions. Our system reached 1st\nplace with a macro $\\text{F}_1$ score of 0.7145.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08953v2",
            "date": "2018-08-27 12:19:07+00:00",
            "title": "Term Set Expansion based NLP Architect by Intel AI Lab",
            "authors": [
                "Jonathan Mamou",
                "Oren Pereg",
                "Moshe Wasserblat",
                "Alon Eirew",
                "Yael Green",
                "Shira Guskin",
                "Peter Izsak",
                "Daniel Korat"
            ],
            "abstract": "We present SetExpander, a corpus-based system for expanding a seed set of\nterms into amore complete set of terms that belong to the same semantic class.\nSetExpander implements an iterative end-to-end workflow. It enables users to\neasily select a seed set of terms, expand it, view the expanded set, validate\nit, re-expand the validated set and store it, thus simplifying the extraction\nof domain-specific fine-grained semantic classes.SetExpander has been used\nsuccessfully in real-life use cases including integration into an automated\nrecruitment system and an issues and defects resolution system. A video demo of\nSetExpander is available at\nhttps://drive.google.com/open?id=1e545bB87Autsch36DjnJHmq3HWfSd1Rv (some images\nwere blurred for privacy reasons)",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.09111v1",
            "date": "2018-08-28 04:33:25+00:00",
            "title": "Unsupervised Learning of Syntactic Structure with Invertible Neural Projections",
            "authors": [
                "Junxian He",
                "Graham Neubig",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Unsupervised learning of syntactic structure is typically performed using\ngenerative models with discrete latent variables and multinomial parameters. In\nmost cases, these models have not leveraged continuous word representations. In\nthis work, we propose a novel generative model that jointly learns discrete\nsyntactic structure and continuous word representations in an unsupervised\nfashion by cascading an invertible neural network with a structured generative\nprior. We show that the invertibility condition allows for efficient exact\ninference and marginal likelihood computation in our model so long as the prior\nis well-behaved. In experiments we instantiate our approach with both Markov\nand tree-structured priors, evaluating on two tasks: part-of-speech (POS)\ninduction, and unsupervised dependency parsing without gold POS annotation. On\nthe Penn Treebank, our Markov-structured model surpasses state-of-the-art\nresults on POS induction. Similarly, we find that our tree-structured model\nachieves state-of-the-art performance on unsupervised dependency parsing for\nthe difficult training condition where neither gold POS annotation nor\npunctuation-based constraints are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09333v2",
            "date": "2018-08-28 14:45:47+00:00",
            "title": "Bridging Knowledge Gaps in Neural Entailment via Symbolic Models",
            "authors": [
                "Dongyeop Kang",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Peter Clark"
            ],
            "abstract": "Most textual entailment models focus on lexical gaps between the premise text\nand the hypothesis, but rarely on knowledge gaps. We focus on filling these\nknowledge gaps in the Science Entailment task, by leveraging an external\nstructured knowledge base (KB) of science facts. Our new architecture combines\nstandard neural entailment models with a knowledge lookup module. To facilitate\nthis lookup, we propose a fact-level decomposition of the hypothesis, and\nverifying the resulting sub-facts against both the textual premise and the\nstructured KB. Our model, NSnet, learns to aggregate predictions from these\nheterogeneous data formats. On the SciTail dataset, NSnet outperforms a simpler\ncombination of the two predictions by 3% and the base entailment model by 5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09353v2",
            "date": "2018-08-28 15:18:14+00:00",
            "title": "Xu: An Automated Query Expansion and Optimization Tool",
            "authors": [
                "Morgan Gallant",
                "Haruna Isah",
                "Farhana Zulkernine",
                "Shahzad Khan"
            ],
            "abstract": "The exponential growth of information on the Internet is a big challenge for\ninformation retrieval systems towards generating relevant results. Novel\napproaches are required to reformat or expand user queries to generate a\nsatisfactory response and increase recall and precision. Query expansion (QE)\nis a technique to broaden users' queries by introducing additional tokens or\nphrases based on some semantic similarity metrics. The tradeoff is the added\ncomputational complexity to find semantically similar words and a possible\nincrease in noise in information retrieval. Despite several research efforts on\nthis topic, QE has not yet been explored enough and more work is needed on\nsimilarity matching and composition of query terms with an objective to\nretrieve a small set of most appropriate responses. QE should be scalable,\nfast, and robust in handling complex queries with a good response time and\nnoise ceiling. In this paper, we propose Xu, an automated QE technique, using\nhigh dimensional clustering of word vectors and Datamuse API, an open source\nquery engine to find semantically similar words. We implemented Xu as a command\nline tool and evaluated its performances using datasets containing news\narticles and human-generated QEs. The evaluation results show that Xu was\nbetter than Datamuse by achieving about 88% accuracy with reference to the\nhuman-generated QE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09384v1",
            "date": "2018-08-28 16:17:43+00:00",
            "title": "What Makes Reading Comprehension Questions Easier?",
            "authors": [
                "Saku Sugawara",
                "Kentaro Inui",
                "Satoshi Sekine",
                "Akiko Aizawa"
            ],
            "abstract": "A challenge in creating a dataset for machine reading comprehension (MRC) is\nto collect questions that require a sophisticated understanding of language to\nanswer beyond using superficial cues. In this work, we investigate what makes\nquestions easier across recent 12 MRC datasets with three question styles\n(answer extraction, description, and multiple choice). We propose to employ\nsimple heuristics to split each dataset into easy and hard subsets and examine\nthe performance of two baseline models for each of the subsets. We then\nmanually annotate questions sampled from each subset with both validity and\nrequisite reasoning skills to investigate which skills explain the difference\nbetween easy and hard questions. From this study, we observed that (i) the\nbaseline performances for the hard subsets remarkably degrade compared to those\nof entire datasets, (ii) hard questions require knowledge inference and\nmultiple-sentence reasoning in comparison with easy questions, and (iii)\nmultiple-choice questions tend to require a broader range of reasoning skills\nthan answer extraction and description questions. These results suggest that\none might overestimate recent advances in MRC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09732v1",
            "date": "2018-08-29 11:26:07+00:00",
            "title": "Development and Evaluation of a Personalized Computer-aided Question Generation for English Learners to Improve Proficiency and Correct Mistakes",
            "authors": [
                "Yi-Ting Huang",
                "Meng Chang Chen",
                "Yeali S. Sun"
            ],
            "abstract": "In the last several years, the field of computer assisted language learning\nhas increasingly focused on computer aided question generation. However, this\napproach often provides test takers with an exhaustive amount of questions that\nare not designed for any specific testing purpose. In this work, we present a\npersonalized computer aided question generation that generates multiple choice\nquestions at various difficulty levels and types, including vocabulary, grammar\nand reading comprehension. In order to improve the weaknesses of test takers,\nit selects questions depending on an estimated proficiency level and unclear\nconcepts behind incorrect responses. This results show that the students with\nthe personalized automatic quiz generation corrected their mistakes more\nfrequently than ones only with computer aided question generation. Moreover,\nstudents demonstrated the most progress between the pretest and post test and\ncorrectly answered more difficult questions. Finally, we investigated the\npersonalizing strategy and found that a student could make a significant\nprogress if the proposed system offered the vocabulary questions at the same\nlevel of his or her proficiency level, and if the grammar and reading\ncomprehension questions were at a level lower than his or her proficiency\nlevel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09888v4",
            "date": "2018-08-28 12:20:37+00:00",
            "title": "KDSL: a Knowledge-Driven Supervised Learning Framework for Word Sense Disambiguation",
            "authors": [
                "Shi Yin",
                "Yi Zhou",
                "Chenguang Li",
                "Shangfei Wang",
                "Jianmin Ji",
                "Xiaoping Chen",
                "Ruili Wang"
            ],
            "abstract": "We propose KDSL, a new word sense disambiguation (WSD) framework that\nutilizes knowledge to automatically generate sense-labeled data for supervised\nlearning. First, from WordNet, we automatically construct a semantic knowledge\nbase called DisDict, which provides refined feature words that highlight the\ndifferences among word senses, i.e., synsets. Second, we automatically generate\nnew sense-labeled data by DisDict from unlabeled corpora. Third, these\ngenerated data, together with manually labeled data and unlabeled data, are fed\nto a neural framework conducting supervised and unsupervised learning jointly\nto model the semantic relations among synsets, feature words and their\ncontexts. The experimental results show that KDSL outperforms several\nrepresentative state-of-the-art methods on various major benchmarks.\nInterestingly, it performs relatively well even when manually labeled data is\nunavailable, thus provides a potential solution for similar tasks in a lack of\nmanual annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09920v3",
            "date": "2018-08-29 16:44:51+00:00",
            "title": "Question Answering by Reasoning Across Documents with Graph Convolutional Networks",
            "authors": [
                "Nicola De Cao",
                "Wilker Aziz",
                "Ivan Titov"
            ],
            "abstract": "Most research in reading comprehension has focused on answering questions\nbased on individual documents or even single paragraphs. We introduce a neural\nmodel which integrates and reasons relying on information spread within\ndocuments and across multiple documents. We frame it as an inference problem on\na graph. Mentions of entities are nodes of this graph while edges encode\nrelations between different mentions (e.g., within- and cross-document\nco-reference). Graph convolutional networks (GCNs) are applied to these graphs\nand trained to perform multi-step reasoning. Our Entity-GCN method is scalable\nand compact, and it achieves state-of-the-art results on a multi-document\nquestion answering dataset, WikiHop (Welbl et al., 2018).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09996v1",
            "date": "2018-08-24 19:24:58+00:00",
            "title": "Learning End-to-End Goal-Oriented Dialog with Multiple Answers",
            "authors": [
                "Janarthanan Rajendran",
                "Jatin Ganhotra",
                "Satinder Singh",
                "Lazaros Polymenakos"
            ],
            "abstract": "In a dialog, there can be multiple valid next utterances at any point. The\npresent end-to-end neural methods for dialog do not take this into account.\nThey learn with the assumption that at any time there is only one correct next\nutterance. In this work, we focus on this problem in the goal-oriented dialog\nsetting where there are different paths to reach a goal. We propose a new\nmethod, that uses a combination of supervised learning and reinforcement\nlearning approaches to address this issue. We also propose a new and more\neffective testbed, permuted-bAbI dialog tasks, by introducing multiple valid\nnext utterances to the original-bAbI dialog tasks, which allows evaluation of\ngoal-oriented dialog systems in a more realistic setting. We show that there is\na significant drop in performance of existing end-to-end neural methods from\n81.5% per-dialog accuracy on original-bAbI dialog tasks to 30.3% on\npermuted-bAbI dialog tasks. We also show that our proposed method improves the\nperformance and achieves 47.3% per-dialog accuracy on permuted-bAbI dialog\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10122v3",
            "date": "2018-08-30 05:15:42+00:00",
            "title": "Learning Neural Templates for Text Generation",
            "authors": [
                "Sam Wiseman",
                "Stuart M. Shieber",
                "Alexander M. Rush"
            ],
            "abstract": "While neural, encoder-decoder models have had significant empirical success\nin text generation, there remain several unaddressed problems with this style\nof generation. Encoder-decoder models are largely (a) uninterpretable, and (b)\ndifficult to control in terms of their phrasing or content. This work proposes\na neural generation system using a hidden semi-markov model (HSMM) decoder,\nwhich learns latent, discrete templates jointly with learning to generate. We\nshow that this model learns useful templates, and that these templates make\ngeneration both more interpretable and controllable. Furthermore, we show that\nthis approach scales to real data sets and achieves strong performance nearing\nthat of encoder-decoder text generation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10584v1",
            "date": "2018-08-31 03:15:28+00:00",
            "title": "Learning to Describe Differences Between Pairs of Similar Images",
            "authors": [
                "Harsh Jhamtani",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "In this paper, we introduce the task of automatically generating text to\ndescribe the differences between two similar images. We collect a new dataset\nby crowd-sourcing difference descriptions for pairs of image frames extracted\nfrom video-surveillance footage. Annotators were asked to succinctly describe\nall the differences in a short paragraph. As a result, our novel dataset\nprovides an opportunity to explore models that align language and vision, and\ncapture visual salience. The dataset may also be a useful benchmark for\ncoherent multi-sentence generation. We perform a firstpass visual analysis that\nexposes clusters of differing pixels as a proxy for object-level differences.\nWe propose a model that captures visual salience by using a latent variable to\nalign clusters of differing pixels with output sentences. We find that, for\nboth single-sentence generation and as well as multi-sentence generation, the\nproposed model outperforms the models that use attention alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10628v1",
            "date": "2018-08-31 08:22:12+00:00",
            "title": "Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension",
            "authors": [
                "Kyosuke Nishida",
                "Itsumi Saito",
                "Atsushi Otsuka",
                "Hisako Asano",
                "Junji Tomita"
            ],
            "abstract": "This study considers the task of machine reading at scale (MRS) wherein,\ngiven a question, a system first performs the information retrieval (IR) task\nof finding relevant passages in a knowledge source and then carries out the\nreading comprehension (RC) task of extracting an answer span from the passages.\nPrevious MRS studies, in which the IR component was trained without considering\nanswer spans, struggled to accurately find a small number of relevant passages\nfrom a large set of passages. In this paper, we propose a simple and effective\napproach that incorporates the IR and RC tasks by using supervised multi-task\nlearning in order that the IR component can be trained by considering answer\nspans. Experimental results on the standard benchmark, answering SQuAD\nquestions using the full Wikipedia as the knowledge source, showed that our\nmodel achieved state-of-the-art performance. Moreover, we thoroughly evaluated\nthe individual contributions of our model components with our new Japanese\ndataset and SQuAD. The results showed significant improvements in the IR task\nand provided a new perspective on IR for RC: it is effective to teach which\npart of the passage answers the question rather than to give only a relevance\nscore to the whole passage.",
            "journal ref": "CIKM 2018, October 22-26, 2018, Torino, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10696v2",
            "date": "2018-08-31 11:56:10+00:00",
            "title": "How agents see things: On visual representations in an emergent language game",
            "authors": [
                "Diane Bouchacourt",
                "Marco Baroni"
            ],
            "abstract": "There is growing interest in the language developed by agents interacting in\nemergent-communication settings. Earlier studies have focused on the agents'\nsymbol usage, rather than on their representation of visual input. In this\npaper, we consider the referential games of Lazaridou et al. (2017) and\ninvestigate the representations the agents develop during their evolving\ninteraction. We find that the agents establish successful communication by\ninducing visual representations that almost perfectly align with each other,\nbut, surprisingly, do not capture the conceptual properties of the objects\ndepicted in the input images. We conclude that, if we are interested in\ndeveloping language-like communication systems, we must pay more attention to\nthe visual semantics agents associate to the symbols they use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00033v1",
            "date": "2018-01-31 19:41:19+00:00",
            "title": "Technical Report: Adjudication of Coreference Annotations via Answer Set Optimization",
            "authors": [
                "Peter Sch\u00fcller"
            ],
            "abstract": "We describe the first automatic approach for merging coreference annotations\nobtained from multiple annotators into a single gold standard. This merging is\nsubject to certain linguistic hard constraints and optimization criteria that\nprefer solutions with minimal divergence from annotators. The representation\ninvolves an equivalence relation over a large number of elements. We use Answer\nSet Programming to describe two representations of the problem and four\nobjective functions suitable for different datasets. We provide two\nstructurally different real-world benchmark datasets based on the METU-Sabanci\nTurkish Treebank and we report our experiences in using the Gringo, Clasp, and\nWasp tools for computing optimal adjudication results on these datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00385v2",
            "date": "2018-02-01 16:48:39+00:00",
            "title": "A Unified Deep Learning Architecture for Abuse Detection",
            "authors": [
                "Antigoni-Maria Founta",
                "Despoina Chatzakou",
                "Nicolas Kourtellis",
                "Jeremy Blackburn",
                "Athena Vakali",
                "Ilias Leontiadis"
            ],
            "abstract": "Hate speech, offensive language, sexism, racism and other types of abusive\nbehavior have become a common phenomenon in many online social media platforms.\nIn recent years, such diverse abusive behaviors have been manifesting with\nincreased frequency and levels of intensity. This is due to the openness and\nwillingness of popular media platforms, such as Twitter and Facebook, to host\ncontent of sensitive or controversial topics. However, these platforms have not\nadequately addressed the problem of online abusive behavior, and their\nresponsiveness to the effective detection and blocking of such inappropriate\nbehavior remains limited.\n  In the present paper, we study this complex problem by following a more\nholistic approach, which considers the various aspects of abusive behavior. To\nmake the approach tangible, we focus on Twitter data and analyze user and\ntextual properties from different angles of abusive posting behavior. We\npropose a deep learning architecture, which utilizes a wide variety of\navailable metadata, and combines it with automatically-extracted hidden\npatterns within the text of the tweets, to detect multiple abusive behavioral\nnorms which are highly inter-related. We apply this unified architecture in a\nseamless, transparent fashion to detect different types of abusive behavior\n(hate speech, sexism vs. racism, bullying, sarcasm, etc.) without the need for\nany tuning of the model architecture for each task. We test the proposed\napproach with multiple datasets addressing different and multiple abusive\nbehaviors on Twitter. Our results demonstrate that it largely outperforms the\nstate-of-art methods (between 21 and 45\\% improvement in AUC, depending on the\ndataset).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00510v1",
            "date": "2018-02-01 22:33:56+00:00",
            "title": "Adaptive Memory Networks",
            "authors": [
                "Daniel Li",
                "Asim Kadav"
            ],
            "abstract": "We present Adaptive Memory Networks (AMN) that processes input-question pairs\nto dynamically construct a network architecture optimized for lower inference\ntimes for Question Answering (QA) tasks. AMN processes the input story to\nextract entities and stores them in memory banks. Starting from a single bank,\nas the number of input entities increases, AMN learns to create new banks as\nthe entropy in a single bank becomes too high. Hence, after processing an\ninput-question(s) pair, the resulting network represents a hierarchical\nstructure where entities are stored in different banks, distanced by question\nrelevance. At inference, one or few banks are used, creating a tradeoff between\naccuracy and performance. AMN is enabled by dynamic networks that allow input\ndependent network creation and efficiency in dynamic mini-batching as well as\nour novel bank controller that allows learning discrete decision making with\nhigh accuracy. In our results, we demonstrate that AMN learns to create\nvariable depth networks depending on task complexity and reduces inference\ntimes for QA tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.00840v2",
            "date": "2018-02-02 20:35:36+00:00",
            "title": "Preserved Structure Across Vector Space Representations",
            "authors": [
                "Andrei Amatuni",
                "Estelle He",
                "Elika Bergelson"
            ],
            "abstract": "Certain concepts, words, and images are intuitively more similar than others\n(dog vs. cat, dog vs. spoon), though quantifying such similarity is notoriously\ndifficult. Indeed, this kind of computation is likely a critical part of\nlearning the category boundaries for words within a given language. Here, we\nuse a set of 27 items (e.g. 'dog') that are highly common in infants' input,\nand use both image- and word-based algorithms to independently compute\nsimilarity among them. We find three key results. First, the pairwise item\nsimilarities derived within image-space and word-space are correlated,\nsuggesting preserved structure among these extremely different representational\nformats. Second, the closest 'neighbors' for each item, within each space,\nshowed significant overlap (e.g. both found 'egg' as a neighbor of 'apple').\nThird, items with the most overlapping neighbors are later-learned by infants\nand toddlers. We conclude that this approach, which does not rely on human\nratings of similarity, may nevertheless reflect stable within-class structure\nacross these two spaces. We speculate that such invariance might aid lexical\nacquisition, by serving as an informative marker of category boundaries.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1802.00946v1",
            "date": "2018-02-03 09:56:54+00:00",
            "title": "Content based Weighted Consensus Summarization",
            "authors": [
                "Parth Mehta",
                "Prasenjit Majumder"
            ],
            "abstract": "Multi-document summarization has received a great deal of attention in the\npast couple of decades. Several approaches have been proposed, many of which\nperform equally well and it is becoming in- creasingly difficult to choose one\nparticular system over another. An ensemble of such systems that is able to\nleverage the strengths of each individual systems can build a better and more\nrobust summary. Despite this, few attempts have been made in this direction. In\nthis paper, we describe a category of ensemble systems which use consensus\nbetween the candidate systems to build a better meta-summary. We highlight two\nmajor shortcomings of such systems: the inability to take into account relative\nperformance of individual systems and overlooking content of candidate\nsummaries in favour of the sentence rankings. We propose an alternate method,\ncontent-based weighted consensus summarization, which address these concerns.\nWe use pseudo-relevant summaries to estimate the performance of individual\ncandidate systems, and then use this information to generate a better aggregate\nranking. Experiments on DUC 2003 and DUC 2004 datasets show that the proposed\nsystem outperforms existing consensus-based techniques by a large margin.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.01074v3",
            "date": "2018-02-04 05:50:39+00:00",
            "title": "Pair-Linking for Collective Entity Disambiguation: Two Could Be Better Than All",
            "authors": [
                "Minh C. Phan",
                "Aixin Sun",
                "Yi Tay",
                "Jialong Han",
                "Chenliang Li"
            ],
            "abstract": "Collective entity disambiguation aims to jointly resolve multiple mentions by\nlinking them to their associated entities in a knowledge base. Previous works\nare primarily based on the underlying assumption that entities within the same\ndocument are highly related. However, the extend to which these mentioned\nentities are actually connected in reality is rarely studied and therefore\nraises interesting research questions. For the first time, we show that the\nsemantic relationships between the mentioned entities are in fact less dense\nthan expected. This could be attributed to several reasons such as noise, data\nsparsity and knowledge base incompleteness. As a remedy, we introduce MINTREE,\na new tree-based objective for the entity disambiguation problem. The key\nintuition behind MINTREE is the concept of coherence relaxation which utilizes\nthe weight of a minimum spanning tree to measure the coherence between\nentities. Based on this new objective, we design a novel entity disambiguation\nalgorithms which we call Pair-Linking. Instead of considering all the given\nmentions, Pair-Linking iteratively selects a pair with the highest confidence\nat each step for decision making. Via extensive experiments, we show that our\napproach is not only more accurate but also surprisingly faster than many\nstate-of-the-art collective linking algorithms.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.01429v1",
            "date": "2018-01-30 09:07:05+00:00",
            "title": "Manuscripts in Time and Space: Experiments in Scriptometrics on an Old French Corpus",
            "authors": [
                "Jean-Baptiste Camps"
            ],
            "abstract": "Witnesses of medieval literary texts, preserved in manuscript, are layered\nobjects , being almost exclusively copies of copies. This results in multiple\nand hard to distinguish linguistic strata -- the author's scripta interacting\nwith the scriptae of the various scribes -- in a context where literary written\nlanguage is already a dialectal hybrid. Moreover, no single linguistic\nphenomenon allows to distinguish between different scriptae, and only the\ncombination of multiple characteristics is likely to be significant [9] -- but\nwhich ones? The most common approach is to search for these features in a set\nof previously selected texts, that are supposed to be representative of a given\nscripta. This can induce a circularity, in which texts are used to select\nfeatures that in turn characterise them as belonging to a linguistic area. To\ncounter this issue, this paper offers an unsupervised and corpus-based\napproach, in which clustering methods are applied to an Old French corpus to\nidentify main divisions and groups. Ultimately, scriptometric profiles are\nbuilt for each of them.",
            "journal ref": "Proceedings of the Second Workshop on Corpus-Based Research in the\n  Humanities CRH-2, 25-26 January 2018, Vienna, Austria, pp.55-64, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01451v1",
            "date": "2018-02-02 14:41:08+00:00",
            "title": "Quantitative Fine-Grained Human Evaluation of Machine Translation Systems: a Case Study on English to Croatian",
            "authors": [
                "Filip Klubi\u010dka",
                "Antonio Toral",
                "V\u00edctor M. S\u00e1nchez-Cartagena"
            ],
            "abstract": "This paper presents a quantitative fine-grained manual evaluation approach to\ncomparing the performance of different machine translation (MT) systems. We\nbuild upon the well-established Multidimensional Quality Metrics (MQM) error\ntaxonomy and implement a novel method that assesses whether the differences in\nperformance for MQM error types between different MT systems are statistically\nsignificant. We conduct a case study for English-to-Croatian, a language\ndirection that involves translating into a morphologically rich language, for\nwhich we compare three MT systems belonging to different paradigms: pure\nphrase-based, factored phrase-based and neural. First, we design an\nMQM-compliant error taxonomy tailored to the relevant linguistic phenomena of\nSlavic languages, which made the annotation process feasible and accurate.\nErrors in MT outputs were then annotated by two annotators following this\ntaxonomy. Subsequently, we carried out a statistical analysis which showed that\nthe best-performing system (neural) reduces the errors produced by the worst\nsystem (pure phrase-based) by more than half (54\\%). Moreover, we conducted an\nadditional analysis of agreement errors in which we distinguished between short\n(phrase-level) and long distance (sentence-level) errors. We discovered that\nphrase-based MT approaches are of limited use for long distance agreement\nphenomena, for which neural MT was found to be especially effective.",
            "journal ref": "Machine Translation, pp 1-21, (2018), http://rdcu.be/GIkb",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01830v1",
            "date": "2018-02-06 07:41:14+00:00",
            "title": "A Neurobiologically Motivated Analysis of Distributional Semantic Models",
            "authors": [
                "Akira Utsumi"
            ],
            "abstract": "The pervasive use of distributional semantic models or word embeddings in a\nvariety of research fields is due to their remarkable ability to represent the\nmeanings of words for both practical application and cognitive modeling.\nHowever, little has been known about what kind of information is encoded in\ntext-based word vectors. This lack of understanding is particularly problematic\nwhen word vectors are regarded as a model of semantic representation for\nabstract concepts. This paper attempts to reveal the internal information of\ndistributional word vectors by the analysis using Binder et al.'s (2016)\nbrain-based vectors, explicitly structured conceptual representations based on\nneurobiologically motivated attributes. In the analysis, the mapping from\ntext-based vectors to brain-based vectors is trained and prediction performance\nis evaluated by comparing the estimated and original brain-based vectors. The\nanalysis demonstrates that social and cognitive information is better encoded\nin text-based word vectors, but emotional information is not. This result is\ndiscussed in terms of embodied theories for abstract concepts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02053v1",
            "date": "2018-02-06 16:36:44+00:00",
            "title": "Syst\u00e8me de traduction automatique statistique Anglais-Arabe",
            "authors": [
                "Marwa Hadj Salah",
                "Didier Schwab",
                "Herv\u00e9 Blanchon",
                "Mounir Zrigui"
            ],
            "abstract": "Machine translation (MT) is the process of translating text written in a\nsource language into text in a target language. In this article, we present our\nEnglish-Arabic statistical machine translation system. First, we present the\ngeneral process for setting up a statistical machine translation system, then\nwe describe the tools as well as the different corpora we used to build our MT\nsystem. Our system was evaluated in terms of the BLUE score (24.51%)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02914v1",
            "date": "2018-02-08 15:15:51+00:00",
            "title": "Praaline: Integrating Tools for Speech Corpus Research",
            "authors": [
                "George Christodoulides"
            ],
            "abstract": "This paper presents Praaline, an open-source software system for managing,\nannotating, analysing and visualising speech corpora. Researchers working with\nspeech corpora are often faced with multiple tools and formats, and they need\nto work with ever-increasing amounts of data in a collaborative way. Praaline\nintegrates and extends existing time-proven tools for spoken corpora analysis\n(Praat, Sonic Visualiser and a bridge to the R statistical package) in a\nmodular system, facilitating automation and reuse. Users are exposed to an\nintegrated, user-friendly interface from which to access multiple tools. Corpus\nmetadata and annotations may be stored in a database, locally or remotely, and\nusers can define the metadata and annotation structure. Users may run a\ncustomisable cascade of analysis steps, based on plug-ins and scripts, and\nupdate the database with the results. The corpus database may be queried, to\nproduce aggregated data-sets. Praaline is extensible using Python or C++\nplug-ins, while Praat and R scripts may be executed against the corpus data. A\nseries of visualisations, editors and plug-ins are provided. Praaline is free\nsoftware, released under the GPL license.",
            "journal ref": "Proceedings of the 9th International Conference on Language\n  Resources and Evaluation (LREC), May 2014, Reykjavik, Iceland",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03606v1",
            "date": "2018-02-10 15:08:56+00:00",
            "title": "Distributed NLP",
            "authors": [
                "Galip Aydin",
                "Ibrahim Riza Hallac"
            ],
            "abstract": "In this paper we present the performance of parallel text processing with Map\nReduce on a cloud platform. Scientific papers in Turkish language are processed\nusing Zemberek NLP library. Experiments were run on a Hadoop cluster and\ncompared with the single machines performance.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1802.03701v1",
            "date": "2018-02-11 06:41:54+00:00",
            "title": "Formal Ontology Learning from English IS-A Sentences",
            "authors": [
                "Sourish Dasgupta",
                "Ankur Padia",
                "Gaurav Maheshwari",
                "Priyansh Trivedi",
                "Jens Lehmann"
            ],
            "abstract": "Ontology learning (OL) is the process of automatically generating an\nontological knowledge base from a plain text document. In this paper, we\npropose a new ontology learning approach and tool, called DLOL, which generates\na knowledge base in the description logic (DL) SHOQ(D) from a collection of\nfactual non-negative IS-A sentences in English. We provide extensive\nexperimental results on the accuracy of DLOL, giving experimental comparisons\nto three state-of-the-art existing OL tools, namely Text2Onto, FRED, and LExO.\nHere, we use the standard OL accuracy measure, called lexical accuracy, and a\nnovel OL accuracy measure, called instance-based inference model. In our\nexperimental results, DLOL turns out to be about 21% and 46%, respectively,\nbetter than the best of the other three approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.03793v4",
            "date": "2018-02-11 19:04:49+00:00",
            "title": "Large-Scale Validation of Hypothesis Generation Systems via Candidate Ranking",
            "authors": [
                "Justin Sybrandt",
                "Michael Shtutman",
                "Ilya Safro"
            ],
            "abstract": "The first step of many research projects is to define and rank a short list\nof candidates for study. In the modern rapidity of scientific progress, some\nturn to automated hypothesis generation (HG) systems to aid this process. These\nsystems can identify implicit or overlooked connections within a large\nscientific corpus, and while their importance grows alongside the pace of\nscience, they lack thorough validation. Without any standard numerical\nevaluation method, many validate general-purpose HG systems by rediscovering a\nhandful of historical findings, and some wishing to be more thorough may run\nlaboratory experiments based on automatic suggestions. These methods are\nexpensive, time consuming, and cannot scale. Thus, we present a numerical\nevaluation framework for the purpose of validating HG systems that leverages\nthousands of validation hypotheses. This method evaluates a HG system by its\nability to rank hypotheses by plausibility; a process reminiscent of human\ncandidate selection. Because HG systems do not produce a ranking criteria,\nspecifically those that produce topic models, we additionally present novel\nmetrics to quantify the plausibility of hypotheses given topic model system\noutput. Finally, we demonstrate that our proposed validation method aligns with\nreal-world research goals by deploying our method within Moliere, our recent\ntopic-driven HG system, in order to automatically generate a set of candidate\ngenes related to HIV-associated neurodegenerative disease (HAND). By performing\nlaboratory experiments based on this candidate set, we discover a new\nconnection between HAND and Dead Box RNA Helicase 3 (DDX3). Reproducibility:\ncode, validation data, and results can be found at\nsybrandt.com/2018/validation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.03821v1",
            "date": "2018-02-11 21:18:45+00:00",
            "title": "Distributed Readability Analysis Of Turkish Elementary School Textbooks",
            "authors": [
                "Betul Karakus",
                "Ibrahim Riza Hallac",
                "Galip Aydin"
            ],
            "abstract": "The readability assessment deals with estimating the level of difficulty in\nreading texts.Many readability tests, which do not indicate execution\nefficiency, have been applied on specific texts to measure the reading grade\nlevel in science textbooks. In this paper, we analyze the content covered in\nelementary school Turkish textbooks by employing a distributed parallel\nprocessing framework based on popular MapReduce paradigm. We outline the\narchitecture of a distributed Big Data processing system which uses Hadoop for\nfull-text readability analysis. The readability scores of the textbooks and\nsystem performance measurements are also given in the paper.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1802.04302v2",
            "date": "2018-02-12 19:02:52+00:00",
            "title": "Evaluating Compositionality in Sentence Embeddings",
            "authors": [
                "Ishita Dasgupta",
                "Demi Guo",
                "Andreas Stuhlm\u00fcller",
                "Samuel J. Gershman",
                "Noah D. Goodman"
            ],
            "abstract": "An important challenge for human-like AI is compositional semantics. Recent\nresearch has attempted to address this by using deep neural networks to learn\nvector space embeddings of sentences, which then serve as input to other tasks.\nWe present a new dataset for one such task, `natural language inference' (NLI),\nthat cannot be solved using only word-level knowledge and requires some\ncompositionality. We find that the performance of state of the art sentence\nembeddings (InferSent; Conneau et al., 2017) on our new dataset is poor. We\nanalyze the decision rules learned by InferSent and find that they are\nconsistent with simple heuristics that are ecologically valid in its training\ndataset. Further, we find that augmenting training with our dataset improves\ntest performance on our dataset without loss of performance on the original\ntraining dataset. This highlights the importance of structured datasets in\nbetter understanding and improving AI systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.04425v1",
            "date": "2018-02-13 01:43:44+00:00",
            "title": "\"How Was Your Weekend?\" A Generative Model of Phatic Conversation",
            "authors": [
                "Hannah Morrison",
                "Chris Martens"
            ],
            "abstract": "Unspoken social rules, such as those that govern choosing a proper discussion\ntopic and when to change discussion topics, guide conversational behaviors. We\npropose a computational model of conversation that can follow or break such\nrules, with participant agents that respond accordingly. Additionally, we\ndemonstrate an application of the model: the Experimental Social Tutor (EST), a\nfirst step toward a social skills training tool that generates human-readable\nconversation and a conversational guideline at each point in the dialogue.\nFinally, we discuss the design and results of a pilot study evaluating the EST.\nResults show that our model is capable of producing conversations that follow\nsocial norms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05121v1",
            "date": "2018-02-14 14:47:56+00:00",
            "title": "Co-training for Extraction of Adverse Drug Reaction Mentions from Tweets",
            "authors": [
                "Shashank Gupta",
                "Manish Gupta",
                "Vasudeva Varma",
                "Sachin Pawar",
                "Nitin Ramrakhiyani",
                "Girish K. Palshikar"
            ],
            "abstract": "Adverse drug reactions (ADRs) are one of the leading causes of mortality in\nhealth care. Current ADR surveillance systems are often associated with a\nsubstantial time lag before such events are officially published. On the other\nhand, online social media such as Twitter contain information about ADR events\nin real-time, much before any official reporting. Current state-of-the-art\nmethods in ADR mention extraction use Recurrent Neural Networks (RNN), which\ntypically need large labeled corpora. Towards this end, we propose a\nsemi-supervised method based on co-training which can exploit a large pool of\nunlabeled tweets to augment the limited supervised training data, and as a\nresult enhance the performance. Experiments with 0.1M tweets show that the\nproposed approach outperforms the state-of-the-art methods for the ADR mention\nextraction task by 5% in terms of F1 score.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.05130v1",
            "date": "2018-02-14 14:53:06+00:00",
            "title": "Multi-Task Learning for Extraction of Adverse Drug Reaction Mentions from Tweets",
            "authors": [
                "Shashank Gupta",
                "Manish Gupta",
                "Vasudeva Varma",
                "Sachin Pawar",
                "Nitin Ramrakhiyani",
                "Girish K. Palshikar"
            ],
            "abstract": "Adverse drug reactions (ADRs) are one of the leading causes of mortality in\nhealth care. Current ADR surveillance systems are often associated with a\nsubstantial time lag before such events are officially published. On the other\nhand, online social media such as Twitter contain information about ADR events\nin real-time, much before any official reporting. Current state-of-the-art in\nADR mention extraction uses Recurrent Neural Networks (RNN), which typically\nneed large labeled corpora. Towards this end, we propose a multi-task learning\nbased method which can utilize a similar auxiliary task (adverse drug event\ndetection) to enhance the performance of the main task, i.e., ADR extraction.\nFurthermore, in the absence of auxiliary task dataset, we propose a novel joint\nmulti-task learning method to automatically generate weak supervision dataset\nfor the auxiliary task when a large pool of unlabeled tweets is available.\nExperiments with 0.48M tweets show that the proposed approach outperforms the\nstate-of-the-art methods for the ADR mention extraction task by 7.2% in terms\nof F1 score.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.05412v2",
            "date": "2018-02-15 05:34:21+00:00",
            "title": "NtMalDetect: A Machine Learning Approach to Malware Detection Using Native API System Calls",
            "authors": [
                "Chan Woo Kim"
            ],
            "abstract": "As computing systems become increasingly advanced and as users increasingly\nengage themselves in technology, security has never been a greater concern. In\nmalware detection, static analysis, the method of analyzing potentially\nmalicious files, has been the prominent approach. This approach, however,\nquickly falls short as malicious programs become more advanced and adopt the\ncapabilities of obfuscating its binaries to execute the same malicious\nfunctions, making static analysis extremely difficult for newer variants. The\napproach assessed in this paper is a novel dynamic malware analysis method,\nwhich may generalize better than static analysis to newer variants. Inspired by\nrecent successes in Natural Language Processing (NLP), widely used document\nclassification techniques were assessed in detecting malware by doing such\nanalysis on system calls, which contain useful information about the operation\nof a program as requests that the program makes of the kernel. Features\nconsidered are extracted from system call traces of benign and malicious\nprograms, and the task to classify these traces is treated as a binary document\nclassification task of system call traces. The system call traces were\nprocessed to remove the parameters to only leave the system call function\nnames. The features were grouped into various n-grams and weighted with Term\nFrequency-Inverse Document Frequency. This paper shows that Linear Support\nVector Machines (SVM) optimized by Stochastic Gradient Descent and the\ntraditional Coordinate Descent on the Wolfe Dual form of the SVM are effective\nin this approach, achieving a highest of 96% accuracy with 95% recall score.\nAdditional contributions include the identification of significant system call\nsequences that could be avenues for further research.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1802.05766v1",
            "date": "2018-02-15 21:16:59+00:00",
            "title": "Learning to Count Objects in Natural Images for Visual Question Answering",
            "authors": [
                "Yan Zhang",
                "Jonathon Hare",
                "Adam Pr\u00fcgel-Bennett"
            ],
            "abstract": "Visual Question Answering (VQA) models have struggled with counting objects\nin natural images so far. We identify a fundamental problem due to soft\nattention in these models as a cause. To circumvent this problem, we propose a\nneural network component that allows robust counting from object proposals.\nExperiments on a toy task show the effectiveness of this component and we\nobtain state-of-the-art accuracy on the number category of the VQA v2 dataset\nwithout negatively affecting other categories, even outperforming ensemble\nmodels with our single model. On a difficult balanced pair metric, the\ncomponent gives a substantial improvement in counting over a strong baseline by\n6.6%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1802.05818v1",
            "date": "2018-02-16 02:00:10+00:00",
            "title": "Disentangling Aspect and Opinion Words in Target-based Sentiment Analysis using Lifelong Learning",
            "authors": [
                "Shuai Wang",
                "Mianwei Zhou",
                "Sahisnu Mazumder",
                "Bing Liu",
                "Yi Chang"
            ],
            "abstract": "Given a target name, which can be a product aspect or entity, identifying its\naspect words and opinion words in a given corpus is a fine-grained task in\ntarget-based sentiment analysis (TSA). This task is challenging, especially\nwhen we have no labeled data and we want to perform it for any given domain. To\naddress it, we propose a general two-stage approach. Stage one extracts/groups\nthe target-related words (call t-words) for a given target. This is relatively\neasy as we can apply an existing semantics-based learning technique. Stage two\nseparates the aspect and opinion words from the grouped t-words, which is\nchallenging because we often do not have enough word-level aspect and opinion\nlabels. In this work, we formulate this problem in a PU learning setting and\nincorporate the idea of lifelong learning to solve it. Experimental results\nshow the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05883v3",
            "date": "2018-02-16 10:11:39+00:00",
            "title": "Deep Generative Model for Joint Alignment and Word Representation",
            "authors": [
                "Miguel Rios",
                "Wilker Aziz",
                "Khalil Sima'an"
            ],
            "abstract": "This work exploits translation data as a source of semantically relevant\nlearning signal for models of word representation. In particular, we exploit\nequivalence through translation as a form of distributed context and jointly\nlearn how to embed and align with a deep generative model. Our EmbedAlign model\nembeds words in their complete observed context and learns by marginalisation\nof latent lexical alignments. Besides, it embeds words as posterior probability\ndensities, rather than point estimates, which allows us to compare words in\ncontext using a measure of overlap between distributions (e.g. KL divergence).\nWe investigate our model's performance on a range of lexical semantics tasks\nachieving competitive results on several standard benchmarks including natural\nlanguage inference, paraphrasing, and text similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06079v1",
            "date": "2018-02-17 13:51:59+00:00",
            "title": "Global-scale phylogenetic linguistic inference from lexical resources",
            "authors": [
                "Gerhard J\u00e4ger"
            ],
            "abstract": "Automatic phylogenetic inference plays an increasingly important role in\ncomputational historical linguistics. Most pertinent work is currently based on\nexpert cognate judgments. This limits the scope of this approach to a small\nnumber of well-studied language families. We used machine learning techniques\nto compile data suitable for phylogenetic inference from the ASJP database, a\ncollection of almost 7,000 phonetically transcribed word lists over 40\nconcepts, covering two third of the extant world-wide linguistic diversity.\nFirst, we estimated Pointwise Mutual Information scores between sound classes\nusing weighted sequence alignment and general-purpose optimization. From this\nwe computed a dissimilarity matrix over all ASJP word lists. This matrix is\nsuitable for distance-based phylogenetic inference. Second, we applied cognate\nclustering to the ASJP data, using supervised training of an SVM classifier on\nexpert cognacy judgments. Third, we defined two types of binary characters,\nbased on automatically inferred cognate classes and on sound-class occurrences.\nSeveral tests are reported demonstrating the suitability of these characters\nfor character-based phylogenetic inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06185v1",
            "date": "2018-02-17 04:05:36+00:00",
            "title": "Building a Word Segmenter for Sanskrit Overnight",
            "authors": [
                "Vikas Reddy",
                "Amrith Krishna",
                "Vishnu Dutt Sharma",
                "Prateek Gupta",
                "Vineeth M R",
                "Pawan Goyal"
            ],
            "abstract": "There is an abundance of digitised texts available in Sanskrit. However, the\nword segmentation task in such texts are challenging due to the issue of\n'Sandhi'. In Sandhi, words in a sentence often fuse together to form a single\nchunk of text, where the word delimiter vanishes and sounds at the word\nboundaries undergo transformations, which is also reflected in the written\ntext. Here, we propose an approach that uses a deep sequence to sequence\n(seq2seq) model that takes only the sandhied string as the input and predicts\nthe unsandhied string. The state of the art models are linguistically involved\nand have external dependencies for the lexical and morphological analysis of\nthe input. Our model can be trained \"overnight\" and be used for production. In\nspite of the knowledge lean approach, our system preforms better than the\ncurrent state of the art by gaining a percentage increase of 16.79 % than the\ncurrent state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06764v2",
            "date": "2018-02-20 17:45:53+00:00",
            "title": "Stability of meanings versus rate of replacement of words: an experimental test",
            "authors": [
                "Michele Pasquini",
                "Maurizio Serva"
            ],
            "abstract": "The words of a language are randomly replaced in time by new ones, but it has\nlong been known that words corresponding to some items (meanings) are less\nfrequently replaced than others. Usually, the rate of replacement for a given\nitem is not directly observable, but it is inferred by the estimated stability\nwhich, on the contrary, is observable. This idea goes back a long way in the\nlexicostatistical literature, nevertheless nothing ensures that it gives the\ncorrect answer. The family of Romance languages allows for a direct test of the\nestimated stabilities against the replacement rates since the proto-language\n(Latin) is known and the replacement rates can be explicitly computed. The\noutput of the test is threefold:first, we prove that the standard approach\nwhich tries to infer the replacement rates trough the estimated stabilities is\nsound; second, we are able to rewrite the fundamental formula of\nGlottochronology for a non universal replacement rate (a rate which depends on\nthe item); third, we give indisputable evidence that the stability ranking is\nfar from being the same for different families of languages. This last result\nis also supported by comparison with the Malagasy family of dialects. As a side\nresult we also provide some evidence that Vulgar Latin and not Late Classical\nLatin is at the root of modern Romance languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06893v2",
            "date": "2018-02-19 22:32:47+00:00",
            "title": "Learning Word Vectors for 157 Languages",
            "authors": [
                "Edouard Grave",
                "Piotr Bojanowski",
                "Prakhar Gupta",
                "Armand Joulin",
                "Tomas Mikolov"
            ],
            "abstract": "Distributed word representations, or word vectors, have recently been applied\nto many tasks in natural language processing, leading to state-of-the-art\nperformance. A key ingredient to the successful application of these\nrepresentations is to train them on very large corpora, and use these\npre-trained models in downstream tasks. In this paper, we describe how we\ntrained such high quality word representations for 157 languages. We used two\nsources of data to train these models: the free online encyclopedia Wikipedia\nand data from the common crawl project. We also introduce three new word\nanalogy datasets to evaluate these word vectors, for French, Hindi and Polish.\nFinally, we evaluate our pre-trained word vectors on 10 languages for which\nevaluation datasets exists, showing very strong performance compared to\nprevious models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07370v1",
            "date": "2018-02-20 23:08:19+00:00",
            "title": "SufiSent - Universal Sentence Representations Using Suffix Encodings",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "Computing universal distributed representations of sentences is a fundamental\ntask in natural language processing. We propose a method to learn such\nrepresentations by encoding the suffixes of word sequences in a sentence and\ntraining on the Stanford Natural Language Inference (SNLI) dataset. We\ndemonstrate the effectiveness of our approach by evaluating it on the SentEval\nbenchmark, improving on existing approaches on several transfer tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07374v1",
            "date": "2018-02-20 23:22:25+00:00",
            "title": "On the scaling of polynomial features for representation matching",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "In many neural models, new features as polynomial functions of existing ones\nare used to augment representations. Using the natural language inference task\nas an example, we investigate the use of scaled polynomials of degree 2 and\nabove as matching features. We find that scaling degree 2 features has the\nhighest impact on performance, reducing classification error by 5% in the best\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07459v2",
            "date": "2018-02-21 08:01:41+00:00",
            "title": "Matching Article Pairs with Graphical Decomposition and Convolutions",
            "authors": [
                "Bang Liu",
                "Di Niu",
                "Haojie Wei",
                "Jinghong Lin",
                "Yancheng He",
                "Kunfeng Lai",
                "Yu Xu"
            ],
            "abstract": "Identifying the relationship between two articles, e.g., whether two articles\npublished from different sources describe the same breaking news, is critical\nto many document understanding tasks. Existing approaches for modeling and\nmatching sentence pairs do not perform well in matching longer documents, which\nembody more complex interactions between the enclosed entities than a sentence\ndoes. To model article pairs, we propose the Concept Interaction Graph to\nrepresent an article as a graph of concepts. We then match a pair of articles\nby comparing the sentences that enclose the same concept vertex through a\nseries of encoding techniques, and aggregate the matching signals through a\ngraph convolutional network. To facilitate the evaluation of long article\nmatching, we have created two datasets, each consisting of about 30K pairs of\nbreaking news articles covering diverse topics in the open domain. Extensive\nevaluations of the proposed methods on the two datasets demonstrate significant\nimprovements over a wide range of state-of-the-art methods for natural language\nmatching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07859v2",
            "date": "2018-02-22 00:37:32+00:00",
            "title": "Modeling Spatiotemporal Factors Associated With Sentiment on Twitter: Synthesis and Suggestions for Improving the Identification of Localized Deviations",
            "authors": [
                "Zubair Shah",
                "Paige Martin",
                "Enrico Coiera",
                "Kenneth D. Mandl",
                "Adam G. Dunn"
            ],
            "abstract": "Background: Studies examining how sentiment on social media varies depending\non timing and location appear to produce inconsistent results, making it hard\nto design systems that use sentiment to detect localized events for public\nhealth applications.\n  Objective: The aim of this study was to measure how common timing and\nlocation confounders explain variation in sentiment on Twitter.\n  Methods: Using a dataset of 16.54 million English-language tweets from 100\ncities posted between July 13 and November 30, 2017, we estimated the positive\nand negative sentiment for each of the cities using a dictionary-based\nsentiment analysis and constructed models to explain the differences in\nsentiment using time of day, day of week, weather, city, and interaction type\n(conversations or broadcasting) as factors and found that all factors were\nindependently associated with sentiment.\n  Results: In the full multivariable model of positive (Pearson r in test data\n0.236; 95\\% CI 0.231-0.241) and negative (Pearson r in test data 0.306; 95\\% CI\n0.301-0.310) sentiment, the city and time of day explained more of the variance\nthan weather and day of week. Models that account for these confounders produce\na different distribution and ranking of important events compared with models\nthat do not account for these confounders.\n  Conclusions: In public health applications that aim to detect localized\nevents by aggregating sentiment across populations of Twitter users, it is\nworthwhile accounting for baseline differences before looking for unexpected\nchanges.",
            "journal ref": "(J Med Internet Res 2019;21(5):e12881",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1802.08545v2",
            "date": "2018-02-23 14:30:00+00:00",
            "title": "Unsupervised Grammar Induction with Depth-bounded PCFG",
            "authors": [
                "Lifeng Jin",
                "Finale Doshi-Velez",
                "Timothy Miller",
                "William Schuler",
                "Lane Schwartz"
            ],
            "abstract": "There has been recent interest in applying cognitively or empirically\nmotivated bounds on recursion depth to limit the search space of grammar\ninduction models (Ponvert et al., 2011; Noji and Johnson, 2016; Shain et al.,\n2016). This work extends this depth-bounding approach to probabilistic\ncontext-free grammar induction (DB-PCFG), which has a smaller parameter space\nthan hierarchical sequence models, and therefore more fully exploits the space\nreductions of depth-bounding. Results for this model on grammar acquisition\nfrom transcribed child-directed speech and newswire text exceed or are\ncompetitive with those of other models when evaluated on parse accuracy.\nMoreover, gram- mars acquired from this model demonstrate a consistent use of\ncategory labels, something which has not been demonstrated by other acquisition\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08786v1",
            "date": "2018-02-24 02:34:40+00:00",
            "title": "Syntax-Directed Variational Autoencoder for Structured Data",
            "authors": [
                "Hanjun Dai",
                "Yingtao Tian",
                "Bo Dai",
                "Steven Skiena",
                "Le Song"
            ],
            "abstract": "Deep generative models have been enjoying success in modeling continuous\ndata. However it remains challenging to capture the representations for\ndiscrete structures with formal grammars and semantics, e.g., computer programs\nand molecular structures. How to generate both syntactically and semantically\ncorrect data still remains largely an open problem. Inspired by the theory of\ncompiler where the syntax and semantics check is done via syntax-directed\ntranslation (SDT), we propose a novel syntax-directed variational autoencoder\n(SD-VAE) by introducing stochastic lazy attributes. This approach converts the\noffline SDT check into on-the-fly generated guidance for constraining the\ndecoder. Comparing to the state-of-the-art methods, our approach enforces\nconstraints on the output space so that the output will be not only\nsyntactically valid, but also semantically reasonable. We evaluate the proposed\nmodel with applications in programming language and molecules, including\nreconstruction and program/molecule optimization. The results demonstrate the\neffectiveness in incorporating syntactic and semantic constraints in discrete\ngenerative models, which is significantly better than current state-of-the-art\napproaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.08969v1",
            "date": "2018-02-25 09:01:25+00:00",
            "title": "Meta Multi-Task Learning for Sequence Modeling",
            "authors": [
                "Junkun Chen",
                "Xipeng Qiu",
                "Pengfei Liu",
                "Xuanjing Huang"
            ],
            "abstract": "Semantic composition functions have been playing a pivotal role in neural\nrepresentation learning of text sequences. In spite of their success, most\nexisting models suffer from the underfitting problem: they use the same shared\ncompositional function on all the positions in the sequence, thereby lacking\nexpressive power due to incapacity to capture the richness of compositionality.\nBesides, the composition functions of different tasks are independent and\nlearned from scratch. In this paper, we propose a new sharing scheme of\ncomposition function across multiple tasks. Specifically, we use a shared\nmeta-network to capture the meta-knowledge of semantic composition and generate\nthe parameters of the task-specific semantic composition models. We conduct\nextensive experiments on two types of tasks, text classification and sequence\ntagging, which demonstrate the benefits of our approach. Besides, we show that\nthe shared meta-knowledge learned by our proposed model can be regarded as\noff-the-shelf knowledge and easily transferred to new tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.08979v2",
            "date": "2018-02-25 09:52:24+00:00",
            "title": "NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System",
            "authors": [
                "Xi Victoria Lin",
                "Chenglong Wang",
                "Luke Zettlemoyer",
                "Michael D. Ernst"
            ],
            "abstract": "We present new data and semantic parsing methods for the problem of mapping\nEnglish sentences to Bash commands (NL2Bash). Our long-term goal is to enable\nany user to perform operations such as file manipulation, search, and\napplication-specific scripting by simply stating their goals in English. We\ntake a first step in this domain, by providing a new dataset of challenging but\ncommonly used Bash commands and expert-written English descriptions, along with\nbaseline methods to establish performance levels on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09296v1",
            "date": "2018-02-26 13:50:14+00:00",
            "title": "AMUSE: Multilingual Semantic Parsing for Question Answering over Linked Data",
            "authors": [
                "Sherzod Hakimov",
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "The task of answering natural language questions over RDF data has received\nwide interest in recent years, in particular in the context of the series of\nQALD benchmarks. The task consists of mapping a natural language question to an\nexecutable form, e.g. SPARQL, so that answers from a given KB can be extracted.\nSo far, most systems proposed are i) monolingual and ii) rely on a set of\nhard-coded rules to interpret questions and map them into a SPARQL query. We\npresent the first multilingual QALD pipeline that induces a model from training\ndata for mapping a natural language question into logical form as probabilistic\ninference. In particular, our approach learns to map universal syntactic\ndependency representations to a language-independent logical form based on\nDUDES (Dependency-based Underspecified Discourse Representation Structures)\nthat are then mapped to a SPARQL query as a deterministic second step. Our\nmodel builds on factor graphs that rely on features extracted from the\ndependency graph and corresponding semantic representations. We rely on\napproximate inference techniques, Markov Chain Monte Carlo methods in\nparticular, as well as Sample Rank to update parameters using a ranking\nobjective. Our focus lies on developing methods that overcome the lexical gap\nand present a novel combination of machine translation and word embedding\napproaches for this purpose. As a proof of concept for our approach, we\nevaluate our approach on the QALD-6 datasets for English, German & Spanish.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.09426v2",
            "date": "2018-02-26 16:12:40+00:00",
            "title": "Tone Biased MMR Text Summarization",
            "authors": [
                "Mayank Chaudhari",
                "Aakash Nelson Mattukoyya"
            ],
            "abstract": "Text summarization is an interesting area for researchers to develop new\ntechniques to provide human like summaries for vast amounts of information.\nSummarization techniques tend to focus on providing accurate representation of\ncontent, and often the tone of the content is ignored. Tone of the content sets\na baseline for how a reader perceives the content. As such being able to\ngenerate summary with tone that is appropriate for the reader is important. In\nour work we implement Maximal Marginal Relevance [MMR] based multi-document\ntext summarization and propose a naive model to change tone of the\nsummarization by setting a bias to specific set of words and restricting other\nwords in the summarization output. This bias towards a specified set of words\nproduces a summary whose tone is same as tone of specified words.",
            "journal ref": "Mayank Chaudhari, Aakash Nelson Mattukoyya,Tone Biased MMR Text\n  Summarization,IJCRT, Feb 2018",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.09944v1",
            "date": "2018-02-26 16:22:14+00:00",
            "title": "The Development of Darwin's Origin of Species",
            "authors": [
                "Jaimie Murdock",
                "Colin Allen",
                "Simon DeDeo"
            ],
            "abstract": "From 1837, when he returned to England aboard the $\\textit{HMS Beagle}$, to\n1860, just after publication of $\\textit{The Origin of Species}$, Charles\nDarwin kept detailed notes of each book he read or wanted to read. His notes\nand manuscripts provide information about decades of individual scientific\npractice. Previously, we trained topic models on the full texts of each\nreading, and applied information-theoretic measures to detect that changes in\nhis reading patterns coincided with the boundaries of his three major\nintellectual projects in the period 1837-1860. In this new work we apply the\nreading model to five additional documents, four of them by Darwin: the first\nedition of $\\textit{The Origin of Species}$, two private essays stating\nintermediate forms of his theory in 1842 and 1844, a third essay of disputed\ndating, and Alfred Russel Wallace's essay, which Darwin received in 1858. We\naddress three historical inquiries, previously treated qualitatively: 1) the\nmythology of \"Darwin's Delay,\" that despite completing an extensive draft in\n1844, Darwin waited until 1859 to publish $\\textit{The Origin of Species}$ due\nto external pressures; 2) the relationship between Darwin and Wallace's\ncontemporaneous theories, especially in light of their joint presentation; and\n3) dating of the \"Outline and Draft\" which was rediscovered in 1975 and\npostulated first as an 1839 draft preceding the Sketch of 1842, then as an\ninterstitial draft between the 1842 and 1844 essays.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09957v1",
            "date": "2018-02-27 15:11:28+00:00",
            "title": "Convolutional Neural Networks for Toxic Comment Classification",
            "authors": [
                "Spiros V. Georgakopoulos",
                "Sotiris K. Tasoulis",
                "Aristidis G. Vrahatis",
                "Vassilis P. Plagianakos"
            ],
            "abstract": "Flood of information is produced in a daily basis through the global Internet\nusage arising from the on-line interactive communications among users. While\nthis situation contributes significantly to the quality of human life,\nunfortunately it involves enormous dangers, since on-line texts with high\ntoxicity can cause personal attacks, on-line harassment and bullying behaviors.\nThis has triggered both industrial and research community in the last few years\nwhile there are several tries to identify an efficient model for on-line toxic\ncomment prediction. However, these steps are still in their infancy and new\napproaches and frameworks are required. On parallel, the data explosion that\nappears constantly, makes the construction of new machine learning\ncomputational tools for managing this information, an imperative need.\nThankfully advances in hardware, cloud computing and big data management allow\nthe development of Deep Learning approaches appearing very promising\nperformance so far. For text classification in particular the use of\nConvolutional Neural Networks (CNN) have recently been proposed approaching\ntext analytics in a modern manner emphasizing in the structure of words in a\ndocument. In this work, we employ this approach to discover toxic comments in a\nlarge pool of documents provided by a current Kaggle's competition regarding\nWikipedia's talk page edits. To justify this decision we choose to compare CNNs\nagainst the traditional bag-of-words approach for text analysis combined with a\nselection of algorithms proven to be very effective in text classification. The\nreported results provide enough evidence that CNN enhance toxic comment\nclassification reinforcing research interest towards this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10078v1",
            "date": "2018-02-26 21:43:23+00:00",
            "title": "A Fast Deep Learning Model for Textual Relevance in Biomedical Information Retrieval",
            "authors": [
                "Sunil Mohan",
                "Nicolas Fiorini",
                "Sun Kim",
                "Zhiyong Lu"
            ],
            "abstract": "Publications in the life sciences are characterized by a large technical\nvocabulary, with many lexical and semantic variations for expressing the same\nconcept. Towards addressing the problem of relevance in biomedical literature\nsearch, we introduce a deep learning model for the relevance of a document's\ntext to a keyword style query. Limited by a relatively small amount of training\ndata, the model uses pre-trained word embeddings. With these, the model first\ncomputes a variable-length Delta matrix between the query and document,\nrepresenting a difference between the two texts, which is then passed through a\ndeep convolution stage followed by a deep feed-forward network to compute a\nrelevance score. This results in a fast model suitable for use in an online\nsearch engine. The model is robust and outperforms comparable state-of-the-art\ndeep learning approaches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.00189v1",
            "date": "2018-03-01 03:15:10+00:00",
            "title": "Growing Story Forest Online from Massive Breaking News",
            "authors": [
                "Bang Liu",
                "Di Niu",
                "Kunfeng Lai",
                "Linglong Kong",
                "Yu Xu"
            ],
            "abstract": "We describe our experience of implementing a news content organization system\nat Tencent that discovers events from vast streams of breaking news and evolves\nnews story structures in an online fashion. Our real-world system has distinct\nrequirements in contrast to previous studies on topic detection and tracking\n(TDT) and event timeline or graph generation, in that we 1) need to accurately\nand quickly extract distinguishable events from massive streams of long text\ndocuments that cover diverse topics and contain highly redundant information,\nand 2) must develop the structures of event stories in an online manner,\nwithout repeatedly restructuring previously formed stories, in order to\nguarantee a consistent user viewing experience. In solving these challenges, we\npropose Story Forest, a set of online schemes that automatically clusters\nstreaming documents into events, while connecting related events in growing\ntrees to tell evolving stories. We conducted extensive evaluation based on 60\nGB of real-world Chinese news data, although our ideas are not\nlanguage-dependent and can easily be extended to other languages, through\ndetailed pilot user experience studies. The results demonstrate the superior\ncapability of Story Forest to accurately identify events and organize news text\ninto a logical structure that is appealing to human readers, compared to\nmultiple existing algorithm frameworks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.00202v1",
            "date": "2018-03-01 04:04:35+00:00",
            "title": "Collaborative Metric Learning Recommendation System: Application to Theatrical Movie Releases",
            "authors": [
                "Miguel Campo",
                "JJ Espinoza",
                "Julie Rieger",
                "Abhinav Taliyan"
            ],
            "abstract": "Product recommendation systems are important for major movie studios during\nthe movie greenlight process and as part of machine learning personalization\npipelines. Collaborative Filtering (CF) models have proved to be effective at\npowering recommender systems for online streaming services with explicit\ncustomer feedback data. CF models do not perform well in scenarios in which\nfeedback data is not available, in cold start situations like new product\nlaunches, and situations with markedly different customer tiers (e.g., high\nfrequency customers vs. casual customers). Generative natural language models\nthat create useful theme-based representations of an underlying corpus of\ndocuments can be used to represent new product descriptions, like new movie\nplots. When combined with CF, they have shown to increase the performance in\ncold start situations. Outside of those cases though in which explicit customer\nfeedback is available, recommender engines must rely on binary purchase data,\nwhich materially degrades performance. Fortunately, purchase data can be\ncombined with product descriptions to generate meaningful representations of\nproducts and customer trajectories in a convenient product space in which\nproximity represents similarity. Learning to measure the distance between\npoints in this space can be accomplished with a deep neural network that trains\non customer histories and on dense vectorizations of product descriptions. We\ndeveloped a system based on Collaborative (Deep) Metric Learning (CML) to\npredict the purchase probabilities of new theatrical releases. We trained and\nevaluated the model using a large dataset of customer histories, and tested the\nmodel for a set of movies that were released outside of the training window.\nInitial experiments show gains relative to models that do not train on\ncollaborative preferences.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.00729v1",
            "date": "2018-03-02 06:18:40+00:00",
            "title": "Representing Verbs as Argument Concepts",
            "authors": [
                "Yu Gong",
                "Kaiqi Zhao",
                "Kenny Q. Zhu"
            ],
            "abstract": "Verbs play an important role in the understanding of natural language text.\nThis paper studies the problem of abstracting the subject and object arguments\nof a verb into a set of noun concepts, known as the \"argument concepts\". This\nset of concepts, whose size is parameterized, represents the fine-grained\nsemantics of a verb. For example, the object of \"enjoy\" can be abstracted into\ntime, hobby and event, etc. We present a novel framework to automatically infer\nhuman readable and machine computable action concepts with high accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00832v1",
            "date": "2018-03-02 12:59:50+00:00",
            "title": "Towards a Question Answering System over the Semantic Web",
            "authors": [
                "Dennis Diefenbach",
                "Andreas Both",
                "Kamal Singh",
                "Pierre Maret"
            ],
            "abstract": "Thanks to the development of the Semantic Web, a lot of new structured data\nhas become available on the Web in the form of knowledge bases (KBs). Making\nthis valuable data accessible and usable for end-users is one of the main goals\nof Question Answering (QA) over KBs. Most current QA systems query one KB, in\none language (namely English). The existing approaches are not designed to be\neasily adaptable to new KBs and languages. We first introduce a new approach\nfor translating natural language questions to SPARQL queries. It is able to\nquery several KBs simultaneously, in different languages, and can easily be\nported to other KBs and languages. In our evaluation, the impact of our\napproach is proven using 5 different well-known and large KBs: Wikidata,\nDBpedia, MusicBrainz, DBLP and Freebase as well as 5 different languages namely\nEnglish, German, French, Italian and Spanish. Second, we show how we integrated\nour approach, to make it easily accessible by the research community and by\nend-users. To summarize, we provided a conceptional solution for multilingual,\nKB-agnostic Question Answering over the Semantic Web. The provided first\napproximation validates this concept.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.01465v3",
            "date": "2018-03-05 02:44:42+00:00",
            "title": "Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation",
            "authors": [
                "Shuming Ma",
                "Xu Sun",
                "Wei Li",
                "Sujian Li",
                "Wenjie Li",
                "Xuancheng Ren"
            ],
            "abstract": "Most recent approaches use the sequence-to-sequence model for paraphrase\ngeneration. The existing sequence-to-sequence model tends to memorize the words\nand the patterns in the training dataset instead of learning the meaning of the\nwords. Therefore, the generated sentences are often grammatically correct but\nsemantically improper. In this work, we introduce a novel model based on the\nencoder-decoder framework, called Word Embedding Attention Network (WEAN). Our\nproposed model generates the words by querying distributed word representations\n(i.e. neural word embeddings), hoping to capturing the meaning of the according\nwords. Following previous work, we evaluate our model on two\nparaphrase-oriented tasks, namely text simplification and short text\nabstractive summarization. Experimental results show that our model outperforms\nthe sequence-to-sequence baseline by the BLEU score of 6.3 and 5.5 on two\nEnglish text simplification datasets, and the ROUGE-2 F1 score of 5.7 on a\nChinese summarization dataset. Moreover, our model achieves state-of-the-art\nperformances on these three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01580v1",
            "date": "2018-03-05 10:09:32+00:00",
            "title": "Calculated attributes of synonym sets",
            "authors": [
                "Andrew Krizhanovsky",
                "Alexander Kirillov"
            ],
            "abstract": "The goal of formalization, proposed in this paper, is to bring together, as\nnear as possible, the theoretic linguistic problem of synonym conception and\nthe computer linguistic methods based generally on empirical intuitive\nunjustified factors. Using the word vector representation we have proposed the\ngeometric approach to mathematical modeling of synonym set (synset). The word\nembedding is based on the neural networks (Skip-gram, CBOW), developed and\nrealized as word2vec program by T. Mikolov. The standard cosine similarity is\nused as the distance between word-vectors. Several geometric characteristics of\nthe synset words are introduced: the interior of synset, the synset word rank\nand centrality. These notions are intended to select the most significant\nsynset words, i.e. the words which senses are the nearest to the sense of a\nsynset. Some experiments with proposed notions, based on RusVectores resources,\nare represented. A brief description of this work can be viewed in slides\nhttps://goo.gl/K82Fei",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.01934v1",
            "date": "2018-03-05 21:29:50+00:00",
            "title": "The morphospace of language networks",
            "authors": [
                "Lu\u00eds F Seoane",
                "Ricard Sol\u00e9"
            ],
            "abstract": "Language can be described as a network of interacting objects with different\nqualitative properties and complexity. These networks include semantic,\nsyntactic, or phonological levels and have been found to provide a new picture\nof language complexity and its evolution. A general approach considers language\nfrom an information theory perspective that incorporates a speaker, a hearer,\nand a noisy channel. The later is often encoded in a matrix connecting the\nsignals used for communication with meanings to be found in the real world.\nMost studies of language evolution deal in a way or another with such\ntheoretical contraption and explore the outcome of diverse forms of selection\non the communication matrix that somewhat optimizes communication. This\nframework naturally introduces networks mediating the communicating agents, but\nno systematic analysis of the underlying landscape of possible language graphs\nhas been developed. Here we present a detailed analysis of network properties\non a generic model of a communication code, which reveals a rather complex and\nheterogeneous morphospace of language networks. Additionally, we use curated\ndata of English words to locate and evaluate real languages within this\nlanguage morphospace. Our findings indicate a surprisingly simple structure in\nhuman language unless particles are introduced in the vocabulary, with the\nability of naming any other concept. These results refine and for the first\ntime complement with empirical data a lasting theoretical tradition around the\nframework of \\emph{least effort language}.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1803.02205v1",
            "date": "2018-03-06 14:21:15+00:00",
            "title": "Code Review Comments: Language Matters",
            "authors": [
                "Vasiliki Efstathiou",
                "Diomidis Spinellis"
            ],
            "abstract": "Recent research provides evidence that effective communication in\ncollaborative software development has significant impact on the software\ndevelopment lifecycle. Although related qualitative and quantitative studies\npoint out textual characteristics of well-formed messages, the underlying\nsemantics of the intertwined linguistic structures still remain largely\nmisinterpreted or ignored. Especially, regarding quality of code reviews the\nimportance of thorough feedback, and explicit rationale is often mentioned but\nrarely linked with related linguistic features. As a first step towards\naddressing this shortcoming, we propose grounding these studies on theories of\nlinguistics. We particularly focus on linguistic structures of coherent speech\nand explain how they can be exploited in practice. We reflect on related\napproaches and examine through a preliminary study on four open source\nprojects, possible links between existing findings and the directions we\nsuggest for detecting textual features of useful code reviews.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1803.02324v2",
            "date": "2018-03-06 18:23:08+00:00",
            "title": "Annotation Artifacts in Natural Language Inference Data",
            "authors": [
                "Suchin Gururangan",
                "Swabha Swayamdipta",
                "Omer Levy",
                "Roy Schwartz",
                "Samuel R. Bowman",
                "Noah A. Smith"
            ],
            "abstract": "Large-scale datasets for natural language inference are created by presenting\ncrowd workers with a sentence (premise), and asking them to generate three new\nsentences (hypotheses) that it entails, contradicts, or is logically neutral\nwith respect to. We show that, in a significant portion of such data, this\nprotocol leaves clues that make it possible to identify the label by looking\nonly at the hypothesis, without observing the premise. Specifically, we show\nthat a simple text categorization model can correctly classify the hypothesis\nalone in about 67% of SNLI (Bowman et. al, 2015) and 53% of MultiNLI (Williams\net. al, 2017). Our analysis reveals that specific linguistic phenomena such as\nnegation and vagueness are highly correlated with certain inference classes.\nOur findings suggest that the success of natural language inference models to\ndate has been overestimated, and that the task remains a hard open problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02400v4",
            "date": "2018-03-02 05:26:28+00:00",
            "title": "Natural Language to Structured Query Generation via Meta-Learning",
            "authors": [
                "Po-Sen Huang",
                "Chenglong Wang",
                "Rishabh Singh",
                "Wen-tau Yih",
                "Xiaodong He"
            ],
            "abstract": "In conventional supervised training, a model is trained to fit all the\ntraining examples. However, having a monolithic model may not always be the\nbest strategy, as examples could vary widely. In this work, we explore a\ndifferent learning protocol that treats each example as a unique pseudo-task,\nby reducing the original learning problem to a few-shot meta-learning scenario\nwith the help of a domain-dependent relevance function. When evaluated on the\nWikiSQL dataset, our approach leads to faster convergence and achieves\n1.1%-5.4% absolute accuracy gains over the non-meta-learning counterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02632v2",
            "date": "2018-03-07 13:13:16+00:00",
            "title": "Extracting Action Sequences from Texts Based on Deep Reinforcement Learning",
            "authors": [
                "Wenfeng Feng",
                "Hankz Hankui Zhuo",
                "Subbarao Kambhampati"
            ],
            "abstract": "Extracting action sequences from natural language texts is challenging, as it\nrequires commonsense inferences based on world knowledge. Although there has\nbeen work on extracting action scripts, instructions, navigation actions, etc.,\nthey require that either the set of candidate actions be provided in advance,\nor that action descriptions are restricted to a specific form, e.g.,\ndescription templates. In this paper, we aim to extract action sequences from\ntexts in free natural language, i.e., without any restricted templates,\nprovided the candidate set of actions is unknown. We propose to extract action\nsequences from texts based on the deep reinforcement learning framework.\nSpecifically, we view \"selecting\" or \"eliminating\" words from texts as\n\"actions\", and the texts associated with actions as \"states\". We then build\nQ-networks to learn the policy of extracting actions and extract plans from the\nlabeled texts. We demonstrate the effectiveness of our approach on several\ndatasets with comparison to state-of-the-art approaches, including online\nexperiments interacting with humans.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.02710v1",
            "date": "2018-03-07 15:18:03+00:00",
            "title": "Generating Contradictory, Neutral, and Entailing Sentences",
            "authors": [
                "Yikang Shen",
                "Shawn Tan",
                "Chin-Wei Huang",
                "Aaron Courville"
            ],
            "abstract": "Learning distributed sentence representations remains an interesting problem\nin the field of Natural Language Processing (NLP). We want to learn a model\nthat approximates the conditional latent space over the representations of a\nlogical antecedent of the given statement. In our paper, we propose an approach\nto generating sentences, conditioned on an input sentence and a logical\ninference label. We do this by modeling the different possibilities for the\noutput sentence as a distribution over the latent representation, which we\ntrain using an adversarial objective. We evaluate the model using two\nstate-of-the-art models for the Recognizing Textual Entailment (RTE) task, and\nmeasure the BLEU scores against the actual sentences as a probe for the\ndiversity of sentences produced by our model. The experiment results show that,\ngiven our framework, we have clear ways to improve the quality and diversity of\ngenerated sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02728v1",
            "date": "2018-03-07 15:51:11+00:00",
            "title": "Towards the Creation of a Large Corpus of Synthetically-Identified Clinical Notes",
            "authors": [
                "Willie Boag",
                "Tristan Naumann",
                "Peter Szolovits"
            ],
            "abstract": "Clinical notes often describe the most important aspects of a patient's\nphysiology and are therefore critical to medical research. However, these notes\nare typically inaccessible to researchers without prior removal of sensitive\nprotected health information (PHI), a natural language processing (NLP) task\nreferred to as deidentification. Tools to automatically de-identify clinical\nnotes are needed but are difficult to create without access to those very same\nnotes containing PHI. This work presents a first step toward creating a large\nsynthetically-identified corpus of clinical notes and corresponding PHI\nannotations in order to facilitate the development de-identification tools.\nFurther, one such tool is evaluated against this corpus in order to understand\nthe advantages and shortcomings of this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03664v1",
            "date": "2018-03-07 07:55:11+00:00",
            "title": "Automating Reading Comprehension by Generating Question and Answer Pairs",
            "authors": [
                "Vishwajeet Kumar",
                "Kireeti Boorla",
                "Yogesh Meena",
                "Ganesh Ramakrishnan",
                "Yuan-Fang Li"
            ],
            "abstract": "Neural network-based methods represent the state-of-the-art in question\ngeneration from text. Existing work focuses on generating only questions from\ntext without concerning itself with answer generation. Moreover, our analysis\nshows that handling rare words and generating the most appropriate question\ngiven a candidate answer are still challenges facing existing approaches. We\npresent a novel two-stage process to generate question-answer pairs from the\ntext. For the first stage, we present alternatives for encoding the span of the\npivotal answer in the sentence using Pointer Networks. In our second stage, we\nemploy sequence to sequence models for question generation, enhanced with rich\nlinguistic features. Finally, global attention and answer encoding are used for\ngenerating the question most relevant to the answer. We motivate and\nlinguistically analyze the role of each component in our framework and consider\ncompositions of these. This analysis is supported by extensive experimental\nevaluations. Using standard evaluation metrics as well as human evaluations,\nour experimental results validate the significant improvement in the quality of\nquestions generated by our framework over the state-of-the-art. The technique\npresented here represents another step towards more automated reading\ncomprehension assessment. We also present a live system \\footnote{Demo of the\nsystem is available at\n\\url{https://www.cse.iitb.ac.in/~vishwajeet/autoqg.html}.} to demonstrate the\neffectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03665v1",
            "date": "2018-03-02 14:47:24+00:00",
            "title": "Syntax-Aware Language Modeling with Recurrent Neural Networks",
            "authors": [
                "Duncan Blythe",
                "Alan Akbik",
                "Roland Vollgraf"
            ],
            "abstract": "Neural language models (LMs) are typically trained using only lexical\nfeatures, such as surface forms of words. In this paper, we argue this deprives\nthe LM of crucial syntactic signals that can be detected at high confidence\nusing existing parsers. We present a simple but highly effective approach for\ntraining neural LMs using both lexical and syntactic information, and a novel\napproach for applying such LMs to unparsed text using sequential Monte Carlo\nsampling. In experiments on a range of corpora and corpus sizes, we show our\napproach consistently outperforms standard lexical LMs in character-level\nlanguage modeling; on the other hand, for word-level models the models are on a\npar with standard language models. These results indicate potential for\nexpanding LMs beyond lexical surface features to higher-level NLP features for\ncharacter-level models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04349v1",
            "date": "2018-03-05 17:07:44+00:00",
            "title": "Linking ImageNet WordNet Synsets with Wikidata",
            "authors": [
                "Finn \u00c5rup Nielsen"
            ],
            "abstract": "The linkage of ImageNet WordNet synsets to Wikidata items will leverage deep\nlearning algorithm with access to a rich multilingual knowledge graph. Here I\nwill describe our on-going efforts in linking the two resources and issues\nfaced in matching the Wikidata and WordNet knowledge graphs. I show an example\non how the linkage can be used in a deep learning setting with real-time image\nclassification and labeling in a non-English language and discuss what\nopportunities lies ahead.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04488v3",
            "date": "2018-03-12 19:46:10+00:00",
            "title": "Concept2vec: Metrics for Evaluating Quality of Embeddings for Ontological Concepts",
            "authors": [
                "Faisal Alshargi",
                "Saeedeh Shekarpour",
                "Tommaso Soru",
                "Amit Sheth"
            ],
            "abstract": "Although there is an emerging trend towards generating embeddings for\nprimarily unstructured data and, recently, for structured data, no systematic\nsuite for measuring the quality of embeddings has been proposed yet. This\ndeficiency is further sensed with respect to embeddings generated for\nstructured data because there are no concrete evaluation metrics measuring the\nquality of the encoded structure as well as semantic patterns in the embedding\nspace. In this paper, we introduce a framework containing three distinct tasks\nconcerned with the individual aspects of ontological concepts: (i) the\ncategorization aspect, (ii) the hierarchical aspect, and (iii) the relational\naspect. Then, in the scope of each task, a number of intrinsic metrics are\nproposed for evaluating the quality of the embeddings. Furthermore, w.r.t. this\nframework, multiple experimental studies were run to compare the quality of the\navailable embedding models. Employing this framework in future research can\nreduce misjudgment and provide greater insight about quality comparisons of\nembeddings for ontological concepts. We positioned our sampled data and code at\nhttps://github.com/alshargi/Concept2vec under GNU General Public License v3.0.",
            "journal ref": "CEUR-WS 2350 (2019) 26",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04579v1",
            "date": "2018-03-12 23:59:37+00:00",
            "title": "It was the training data pruning too!",
            "authors": [
                "Pramod Kaushik Mudrakarta",
                "Ankur Taly",
                "Mukund Sundararajan",
                "Kedar Dhamdhere"
            ],
            "abstract": "We study the current best model (KDG) for question answering on tabular data\nevaluated over the WikiTableQuestions dataset. Previous ablation studies\nperformed against this model attributed the model's performance to certain\naspects of its architecture. In this paper, we find that the model's\nperformance also crucially depends on a certain pruning of the data used to\ntrain the model. Disabling the pruning step drops the accuracy of the model\nfrom 43.3% to 36.3%. The large impact on the performance of the KDG model\nsuggests that the pruning may be a useful pre-processing step in training other\nsemantic parsers as well.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.05030v1",
            "date": "2018-03-04 11:08:16+00:00",
            "title": "Deep-FSMN for Large Vocabulary Continuous Speech Recognition",
            "authors": [
                "Shiliang Zhang",
                "Ming Lei",
                "Zhijie Yan",
                "Lirong Dai"
            ],
            "abstract": "In this paper, we present an improved feedforward sequential memory networks\n(FSMN) architecture, namely Deep-FSMN (DFSMN), by introducing skip connections\nbetween memory blocks in adjacent layers. These skip connections enable the\ninformation flow across different layers and thus alleviate the gradient\nvanishing problem when building very deep structure. As a result, DFSMN\nsignificantly benefits from these skip connections and deep structure. We have\ncompared the performance of DFSMN to BLSTM both with and without lower frame\nrate (LFR) on several large speech recognition tasks, including English and\nMandarin. Experimental results shown that DFSMN can consistently outperform\nBLSTM with dramatic gain, especially trained with LFR using CD-Phone as\nmodeling units. In the 2000 hours Fisher (FSH) task, the proposed DFSMN can\nachieve a word error rate of 9.4% by purely using the cross-entropy criterion\nand decoding with a 3-gram language model, which achieves a 1.5% absolute\nimprovement compared to the BLSTM. In a 20000 hours Mandarin recognition task,\nthe LFR trained DFSMN can achieve more than 20% relative improvement compared\nto the LFR trained BLSTM. Moreover, we can easily design the lookahead filter\norder of the memory blocks in DFSMN to control the latency for real-time\napplications.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1803.05667v1",
            "date": "2018-03-15 10:04:27+00:00",
            "title": "A Study of Recent Contributions on Information Extraction",
            "authors": [
                "Parisa Naderi Golshan",
                "HosseinAli Rahmani Dashti",
                "Shahrzad Azizi",
                "Leila Safari"
            ],
            "abstract": "This paper reports on modern approaches in Information Extraction (IE) and\nits two main sub-tasks of Named Entity Recognition (NER) and Relation\nExtraction (RE). Basic concepts and the most recent approaches in this area are\nreviewed, which mainly include Machine Learning (ML) based approaches and the\nmore recent trend to Deep Learning (DL) based methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.06064v2",
            "date": "2018-03-16 03:07:06+00:00",
            "title": "A Meaning-based Statistical English Math Word Problem Solver",
            "authors": [
                "Chao-Chun Liang",
                "Yu-Shiang Wong",
                "Yi-Chung Lin",
                "Keh-Yih Su"
            ],
            "abstract": "We introduce MeSys, a meaning-based approach, for solving English math word\nproblems (MWPs) via understanding and reasoning in this paper. It first\nanalyzes the text, transforms both body and question parts into their\ncorresponding logic forms, and then performs inference on them. The associated\ncontext of each quantity is represented with proposed role-tags (e.g., nsubj,\nverb, etc.), which provides the flexibility for annotating an extracted math\nquantity with its associated context information (i.e., the physical meaning of\nthis quantity). Statistical models are proposed to select the operator and\noperands. A noisy dataset is designed to assess if a solver solves MWPs mainly\nvia understanding or mechanical pattern matching. Experimental results show\nthat our approach outperforms existing systems on both benchmark datasets and\nthe noisy dataset, which demonstrates that the proposed approach understands\nthe meaning of each quantity in the text more.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.06252v2",
            "date": "2018-03-16 14:47:58+00:00",
            "title": "Joint Recognition of Handwritten Text and Named Entities with a Neural End-to-end Model",
            "authors": [
                "Manuel Carbonell",
                "Mauricio Villegas",
                "Alicia Forn\u00e9s",
                "Josep Llad\u00f3s"
            ],
            "abstract": "When extracting information from handwritten documents, text transcription\nand named entity recognition are usually faced as separate subsequent tasks.\nThis has the disadvantage that errors in the first module affect heavily the\nperformance of the second module. In this work we propose to do both tasks\njointly, using a single neural network with a common architecture used for\nplain text recognition. Experimentally, the work has been tested on a\ncollection of historical marriage records. Results of experiments are presented\nto show the effect on the performance for different configurations: different\nways of encoding the information, doing or not transfer learning and processing\nat text line or multi-line region level. The results are comparable to state of\nthe art reported in the ICDAR 2017 Information Extraction competition, even\nthough the proposed technique does not use any dictionaries, language modeling\nor post processing.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.06500v2",
            "date": "2018-03-17 13:20:37+00:00",
            "title": "Argumentation theory for mathematical argument",
            "authors": [
                "Joseph Corneli",
                "Ursula Martin",
                "Dave Murray-Rust",
                "Gabriela Rino Nesin",
                "Alison Pease"
            ],
            "abstract": "To adequately model mathematical arguments the analyst must be able to\nrepresent the mathematical objects under discussion and the relationships\nbetween them, as well as inferences drawn about these objects and relationships\nas the discourse unfolds. We introduce a framework with these properties, which\nhas been used to analyse mathematical dialogues and expository texts. The\nframework can recover salient elements of discourse at, and within, the\nsentence level, as well as the way mathematical content connects to form larger\nargumentative structures. We show how the framework might be used to support\ncomputational reasoning, and argue that it provides a more natural way to\nexamine the process of proving theorems than do Lamport's structured proofs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06581v3",
            "date": "2018-03-17 22:08:10+00:00",
            "title": "Variational Knowledge Graph Reasoning",
            "authors": [
                "Wenhu Chen",
                "Wenhan Xiong",
                "Xifeng Yan",
                "William Wang"
            ],
            "abstract": "Inferring missing links in knowledge graphs (KG) has attracted a lot of\nattention from the research community. In this paper, we tackle a practical\nquery answering task involving predicting the relation of a given entity pair.\nWe frame this prediction problem as an inference problem in a probabilistic\ngraphical model and aim at resolving it from a variational inference\nperspective. In order to model the relation between the query entity pair, we\nassume that there exists an underlying latent variable (paths connecting two\nnodes) in the KG, which carries the equivalent semantics of their relations.\nHowever, due to the intractability of connections in large KGs, we propose to\nuse variation inference to maximize the evidence lower bound. More\nspecifically, our framework (\\textsc{Diva}) is composed of three modules, i.e.\na posterior approximator, a prior (path finder), and a likelihood (path\nreasoner). By using variational inference, we are able to incorporate them\nclosely into a unified architecture and jointly optimize them to perform KG\nreasoning. With active interactions among these sub-modules, \\textsc{Diva} is\nbetter at handling noise and coping with more complex reasoning scenarios. In\norder to evaluate our method, we conduct the experiment of the link prediction\ntask on multiple datasets and achieve state-of-the-art performances on both\ndatasets.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.07139v2",
            "date": "2018-03-19 19:48:48+00:00",
            "title": "English-Catalan Neural Machine Translation in the Biomedical Domain through the cascade approach",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Noe Casas",
                "Maite Melero"
            ],
            "abstract": "This paper describes the methodology followed to build a neural machine\ntranslation system in the biomedical domain for the English-Catalan language\npair. This task can be considered a low-resourced task from the point of view\nof the domain and the language pair. To face this task, this paper reports\nexperiments on a cascade pivot strategy through Spanish for the neural machine\ntranslation using the English-Spanish SCIELO and Spanish-Catalan El Peri\\'odico\ndatabase. To test the final performance of the system, we have created a new\ntest data set for English-Catalan in the biomedical domain which is freely\navailable on request.",
            "journal ref": "Proceedings of workshop \"MultilingualBIO: Multilingual Biomedical\n  Text Processing\" of the 11th Edition of the Language Resources and Evaluation\n  Conference, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07828v2",
            "date": "2018-03-21 10:06:28+00:00",
            "title": "Expeditious Generation of Knowledge Graph Embeddings",
            "authors": [
                "Tommaso Soru",
                "Stefano Ruberto",
                "Diego Moussallem",
                "Andr\u00e9 Valdestilhas",
                "Alexander Bigerl",
                "Edgard Marx",
                "Diego Esteves"
            ],
            "abstract": "Knowledge Graph Embedding methods aim at representing entities and relations\nin a knowledge base as points or vectors in a continuous vector space. Several\napproaches using embeddings have shown promising results on tasks such as link\nprediction, entity recommendation, question answering, and triplet\nclassification. However, only a few methods can compute low-dimensional\nembeddings of very large knowledge bases without needing state-of-the-art\ncomputational resources. In this paper, we propose KG2Vec, a simple and fast\napproach to Knowledge Graph Embedding based on the skip-gram model. Instead of\nusing a predefined scoring function, we learn it relying on Long Short-Term\nMemories. We show that our embeddings achieve results comparable with the most\nscalable approaches on knowledge graph completion as well as on a new metric.\nYet, KG2Vec can embed large graphs in lesser time by processing more than 250\nmillion triples in less than 7 hours on common hardware.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08035v2",
            "date": "2018-03-21 17:52:42+00:00",
            "title": "Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs",
            "authors": [
                "Xiaolong Wang",
                "Yufei Ye",
                "Abhinav Gupta"
            ],
            "abstract": "We consider the problem of zero-shot recognition: learning a visual\nclassifier for a category with zero training examples, just using the word\nembedding of the category and its relationship to other categories, which\nvisual data are provided. The key to dealing with the unfamiliar or novel\ncategory is to transfer knowledge obtained from familiar classes to describe\nthe unfamiliar class. In this paper, we build upon the recently introduced\nGraph Convolutional Network (GCN) and propose an approach that uses both\nsemantic embeddings and the categorical relationships to predict the\nclassifiers. Given a learned knowledge graph (KG), our approach takes as input\nsemantic embeddings for each node (representing visual category). After a\nseries of graph convolutions, we predict the visual classifier for each\ncategory. During training, the visual classifiers for a few categories are\ngiven to learn the GCN parameters. At test time, these filters are used to\npredict the visual classifiers of unseen categories. We show that our approach\nis robust to noise in the KG. More importantly, our approach provides\nsignificant improvement in performance compared to the current state-of-the-art\nresults (from 2 ~ 3% on some metrics to whopping 20% on a few).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.08419v2",
            "date": "2018-03-22 15:53:59+00:00",
            "title": "The Rapidly Changing Landscape of Conversational Agents",
            "authors": [
                "Vinayak Mathur",
                "Arpit Singh"
            ],
            "abstract": "Conversational agents have become ubiquitous, ranging from goal-oriented\nsystems for helping with reservations to chit-chat models found in modern\nvirtual assistants. In this survey paper, we explore this fascinating field. We\nlook at some of the pioneering work that defined the field and gradually move\nto the current state-of-the-art models. We look at statistical, neural,\ngenerative adversarial network based and reinforcement learning based\napproaches and how they evolved. Along the way we discuss various challenges\nthat the field faces, lack of context in utterances, not having a good\nquantitative metric to compare models, lack of trust in agents because they do\nnot have a consistent persona etc. We structure this paper in a way that\nanswers these pertinent questions and discusses competing approaches to solve\nthem.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.08476v1",
            "date": "2018-03-22 17:22:42+00:00",
            "title": "Word sense induction using word embeddings and community detection in complex networks",
            "authors": [
                "Edilson A. Corr\u00eaa Jr.",
                "Diego R. Amancio"
            ],
            "abstract": "Word Sense Induction (WSI) is the ability to automatically induce word senses\nfrom corpora. The WSI task was first proposed to overcome the limitations of\nmanually annotated corpus that are required in word sense disambiguation\nsystems. Even though several works have been proposed to induce word senses,\nexisting systems are still very limited in the sense that they make use of\nstructured, domain-specific knowledge sources. In this paper, we devise a\nmethod that leverages recent findings in word embeddings research to generate\ncontext embeddings, which are embeddings containing information about the\nsemantical context of a word. In order to induce senses, we modeled the set of\nambiguous words as a complex network. In the generated network, two instances\n(nodes) are connected if the respective context embeddings are similar. Upon\nusing well-established community detection methods to cluster the obtained\ncontext embeddings, we found that the proposed method yields excellent\nperformance for the WSI task. Our method outperformed competing algorithms and\nbaselines, in a completely unsupervised manner and without the need of any\nadditional structured knowledge source.",
            "journal ref": "Physica A, v. 523, p. 180-190, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08721v2",
            "date": "2018-03-23 10:35:42+00:00",
            "title": "Unsupervised Keyphrase Extraction with Multipartite Graphs",
            "authors": [
                "Florian Boudin"
            ],
            "abstract": "We propose an unsupervised keyphrase extraction model that encodes topical\ninformation within a multipartite graph structure. Our model represents\nkeyphrase candidates and topics in a single graph and exploits their mutually\nreinforcing relationship to improve candidate ranking. We further introduce a\nnovel mechanism to incorporate keyphrase selection preferences into the model.\nExperiments conducted on three widely used datasets show significant\nimprovements over state-of-the-art graph-based models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.08863v2",
            "date": "2018-03-23 16:18:27+00:00",
            "title": "Multilingual bottleneck features for subword modeling in zero-resource languages",
            "authors": [
                "Enno Hermann",
                "Sharon Goldwater"
            ],
            "abstract": "How can we effectively develop speech technology for languages where no\ntranscribed data is available? Many existing approaches use no annotated\nresources at all, yet it makes sense to leverage information from large\nannotated corpora in other languages, for example in the form of multilingual\nbottleneck features (BNFs) obtained from a supervised speech recognition\nsystem. In this work, we evaluate the benefits of BNFs for subword modeling\n(feature extraction) in six unseen languages on a word discrimination task.\nFirst we establish a strong unsupervised baseline by combining two existing\nmethods: vocal tract length normalisation (VTLN) and the correspondence\nautoencoder (cAE). We then show that BNFs trained on a single language already\nbeat this baseline; including up to 10 languages results in additional\nimprovements which cannot be matched by just adding more data from a single\nlanguage. Finally, we show that the cAE can improve further on the BNFs if\nhigh-quality same-word pairs are available.",
            "journal ref": "Proc. Interspeech 2018, 2668-2672",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08896v1",
            "date": "2018-03-23 17:17:16+00:00",
            "title": "Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering",
            "authors": [
                "Somak Aditya",
                "Yezhou Yang",
                "Chitta Baral"
            ],
            "abstract": "Many vision and language tasks require commonsense reasoning beyond\ndata-driven image and natural language processing. Here we adopt Visual\nQuestion Answering (VQA) as an example task, where a system is expected to\nanswer a question in natural language about an image. Current state-of-the-art\nsystems attempted to solve the task using deep neural architectures and\nachieved promising performance. However, the resulting systems are generally\nopaque and they struggle in understanding questions for which extra knowledge\nis required. In this paper, we present an explicit reasoning layer on top of a\nset of penultimate neural network based systems. The reasoning layer enables\nreasoning and answering questions where additional knowledge is required, and\nat the same time provides an interpretable interface to the end users.\nSpecifically, the reasoning layer adopts a Probabilistic Soft Logic (PSL) based\nengine to reason over a basket of inputs: visual relations, the semantic parse\nof the question, and background ontological knowledge from word2vec and\nConceptNet. Experimental analysis of the answers and the key evidential\npredicates generated on the VQA dataset validate our approach.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.08983v1",
            "date": "2018-03-23 21:20:00+00:00",
            "title": "Automated Evaluation of Out-of-Context Errors",
            "authors": [
                "Patrick Huber",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "We present a new approach to evaluate computational models for the task of\ntext understanding by the means of out-of-context error detection. Through the\nnovel design of our automated modification process, existing large-scale data\nsources can be adopted for a vast number of text understanding tasks. The data\nis thereby altered on a semantic level, allowing models to be tested against a\nchallenging set of modified text passages that require to comprise a broader\nnarrative discourse. Our newly introduced task targets actual real-world\nproblems of transcription and translation systems by inserting authentic\nout-of-context errors. The automated modification process is applied to the\n2016 TEDTalk corpus. Entirely automating the process allows the adoption of\ncomplete datasets at low cost, facilitating supervised learning procedures and\ndeeper networks to be trained and tested. To evaluate the quality of the\nmodification algorithm a language model and a supervised binary classification\nmodel are trained and tested on the altered dataset. A human baseline\nevaluation is examined to compare the results with human performance. The\noutcome of the evaluation task indicates the difficulty to detect semantic\nerrors for machine-learning algorithms and humans, showing that the errors\ncannot be identified when limited to a single sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09000v1",
            "date": "2018-03-23 22:30:58+00:00",
            "title": "WikiRank: Improving Keyphrase Extraction Based on Background Knowledge",
            "authors": [
                "Yang Yu",
                "Vincent Ng"
            ],
            "abstract": "Keyphrase is an efficient representation of the main idea of documents. While\nbackground knowledge can provide valuable information about documents, they are\nrarely incorporated in keyphrase extraction methods. In this paper, we propose\nWikiRank, an unsupervised method for keyphrase extraction based on the\nbackground knowledge from Wikipedia. Firstly, we construct a semantic graph for\nthe document. Then we transform the keyphrase extraction problem into an\noptimization problem on the graph. Finally, we get the optimal keyphrase set to\nbe the output. Our method obtains improvements over other state-of-art models\nby more than 2% in F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09189v1",
            "date": "2018-03-25 01:53:29+00:00",
            "title": "Scene Graph Parsing as Dependency Parsing",
            "authors": [
                "Yu-Siang Wang",
                "Chenxi Liu",
                "Xiaohui Zeng",
                "Alan Yuille"
            ],
            "abstract": "In this paper, we study the problem of parsing structured knowledge graphs\nfrom textual descriptions. In particular, we consider the scene graph\nrepresentation that considers objects together with their attributes and\nrelations: this representation has been proved useful across a variety of\nvision and language applications. We begin by introducing an alternative but\nequivalent edge-centric view of scene graphs that connect to dependency parses.\nTogether with a careful redesign of label and action space, we combine the\ntwo-stage pipeline used in prior work (generic dependency parsing followed by\nsimple post-processing) into one, enabling end-to-end training. The scene\ngraphs generated by our learned neural dependency parser achieve an F-score\nsimilarity of 49.67% to ground truth graphs on our evaluation set, surpassing\nbest previous approaches by 5%. We further demonstrate the effectiveness of our\nlearned parser on image retrieval applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09401v1",
            "date": "2018-03-26 03:45:56+00:00",
            "title": "HomeGuard: A Smart System to Deal with the Emergency Response of Domestic Violence Victims",
            "authors": [
                "Anik Islam",
                "Arifa Akter",
                "Bayzid Ashik Hossain"
            ],
            "abstract": "Domestic violence is a silent crisis in the developing and underdeveloped\ncountries, though developed countries also remain drowned in the curse of it.\nIn developed countries, victims can easily report and ask help on the contrary\nin developing and underdeveloped countries victims hardly report the crimes and\nwhen it's noticed by the authority it's become too late to save or support the\nvictim. If this kind of problems can be identified at the very beginning of the\nevent and proper actions can be taken, it'll not only help the victim but also\nreduce the domestic violence crimes. This paper proposed a smart system which\ncan extract victim's situation and provide help according to it. Among of the\ndeveloping and underdeveloped countries Bangladesh has been chosen though the\nrate of reporting of domestic violence is low, the extreme report collected by\nauthorities is too high. Case studies collected by different NGO's relating to\ndomestic violence have been studied and applied to extract possible condition\nfor the victims.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.09745v2",
            "date": "2018-03-26 18:00:00+00:00",
            "title": "English verb regularization in books and tweets",
            "authors": [
                "Tyler J. Gray",
                "Andrew J. Reagan",
                "Peter Sheridan Dodds",
                "Christopher M. Danforth"
            ],
            "abstract": "The English language has evolved dramatically throughout its lifespan, to the\nextent that a modern speaker of Old English would be incomprehensible without\ntranslation. One concrete indicator of this process is the movement from\nirregular to regular (-ed) forms for the past tense of verbs. In this study we\nquantify the extent of verb regularization using two vastly disparate datasets:\n(1) Six years of published books scanned by Google (2003--2008), and (2) A\ndecade of social media messages posted to Twitter (2008--2017). We find that\nthe extent of verb regularization is greater on Twitter, taken as a whole, than\nin English Fiction books. Regularization is also greater for tweets geotagged\nin the United States relative to American English books, but the opposite is\ntrue for tweets geotagged in the United Kingdom relative to British English\nbooks. We also find interesting regional variations in regularization across\ncounties in the United States. However, once differences in population are\naccounted for, we do not identify strong correlations with socio-demographic\nvariables such as education or income.",
            "journal ref": "PLOS ONE 13(12): e0209651, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09840v2",
            "date": "2018-03-26 20:56:30+00:00",
            "title": "Empirical Analysis of Foundational Distinctions in Linked Open Data",
            "authors": [
                "Luigi Asprino",
                "Valerio Basile",
                "Paolo Ciancarini",
                "Valentina Presutti"
            ],
            "abstract": "The Web and its Semantic extension (i.e. Linked Open Data) contain open\nglobal-scale knowledge and make it available to potentially intelligent\nmachines that want to benefit from it. Nevertheless, most of Linked Open Data\nlack ontological distinctions and have sparse axiomatisation. For example,\ndistinctions such as whether an entity is inherently a class or an individual,\nor whether it is a physical object or not, are hardly expressed in the data,\nalthough they have been largely studied and formalised by foundational\nontologies (e.g. DOLCE, SUMO). These distinctions belong to common sense too,\nwhich is relevant for many artificial intelligence tasks such as natural\nlanguage understanding, scene recognition, and the like. There is a gap between\nfoundational ontologies, that often formalise or are inspired by pre-existing\nphilosophical theories and are developed with a top-down approach, and Linked\nOpen Data that mostly derive from existing databases or crowd-based effort\n(e.g. DBpedia, Wikidata). We investigate whether machines can learn\nfoundational distinctions over Linked Open Data entities, and if they match\ncommon sense. We want to answer questions such as \"does the DBpedia entity for\ndog refer to a class or to an instance?\". We report on a set of experiments\nbased on machine learning and crowdsourcing that show promising results.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.09845v1",
            "date": "2018-03-27 01:59:56+00:00",
            "title": "Neural Baby Talk",
            "authors": [
                "Jiasen Lu",
                "Jianwei Yang",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "We introduce a novel framework for image captioning that can produce natural\nlanguage explicitly grounded in entities that object detectors find in the\nimage. Our approach reconciles classical slot filling approaches (that are\ngenerally better grounded in images) with modern neural captioning approaches\n(that are generally more natural sounding and accurate). Our approach first\ngenerates a sentence `template' with slot locations explicitly tied to specific\nimage regions. These slots are then filled in by visual concepts identified in\nthe regions by object detectors. The entire architecture (sentence template\ngeneration and slot filling with object detectors) is end-to-end\ndifferentiable. We verify the effectiveness of our proposed model on different\nimage captioning tasks. On standard image captioning and novel object\ncaptioning, our model reaches state-of-the-art on both COCO and Flickr30k\ndatasets. We also demonstrate that our model has unique advantages when the\ntrain and test distributions of scene compositions -- and hence language priors\nof associated captions -- are different. Code has been made available at:\nhttps://github.com/jiasenlu/NeuralBabyTalk",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.11070v2",
            "date": "2018-03-28 02:47:51+00:00",
            "title": "Actor-Critic based Training Framework for Abstractive Summarization",
            "authors": [
                "Piji Li",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "We present a training framework for neural abstractive summarization based on\nactor-critic approaches from reinforcement learning. In the traditional neural\nnetwork based methods, the objective is only to maximize the likelihood of the\npredicted summaries, no other assessment constraints are considered, which may\ngenerate low-quality summaries or even incorrect sentences. To alleviate this\nproblem, we employ an actor-critic framework to enhance the training procedure.\nFor the actor, we employ the typical attention based sequence-to-sequence\n(seq2seq) framework as the policy network for summary generation. For the\ncritic, we combine the maximum likelihood estimator with a well designed global\nsummary quality estimator which is a neural network based binary classifier\naiming to make the generated summaries indistinguishable from the human-written\nones. Policy gradient method is used to conduct the parameter learning. An\nalternating training strategy is proposed to conduct the joint training of the\nactor and critic models. Extensive experiments on some benchmark datasets in\ndifferent languages show that our framework achieves improvements over the\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.11186v1",
            "date": "2018-03-29 17:58:43+00:00",
            "title": "Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering",
            "authors": [
                "Unnat Jain",
                "Svetlana Lazebnik",
                "Alexander Schwing"
            ],
            "abstract": "Human conversation is a complex mechanism with subtle nuances. It is hence an\nambitious goal to develop artificial intelligence agents that can participate\nfluently in a conversation. While we are still far from achieving this goal,\nrecent progress in visual question answering, image captioning, and visual\nquestion generation shows that dialog systems may be realizable in the not too\ndistant future. To this end, a novel dataset was introduced recently and\nencouraging results were demonstrated, particularly for question answering. In\nthis paper, we demonstrate a simple symmetric discriminative baseline, that can\nbe applied to both predicting an answer as well as predicting a question. We\nshow that this method performs on par with the state of the art, even memory\nnet based methods. In addition, for the first time on the visual dialog\ndataset, we assess the performance of a system asking questions, and\ndemonstrate how visual dialog can be generated from discriminative question\ngeneration and question answering.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.11284v1",
            "date": "2018-03-29 23:21:11+00:00",
            "title": "Deep Recurrent Neural Networks for Product Attribute Extraction in eCommerce",
            "authors": [
                "Bodhisattwa Prasad Majumder",
                "Aditya Subramanian",
                "Abhinandan Krishnan",
                "Shreyansh Gandhi",
                "Ajinkya More"
            ],
            "abstract": "Extracting accurate attribute qualities from product titles is a vital\ncomponent in delivering eCommerce customers with a rewarding online shopping\nexperience via an enriched faceted search. We demonstrate the potential of Deep\nRecurrent Networks in this domain, primarily models such as Bidirectional LSTMs\nand Bidirectional LSTM-CRF with or without an attention mechanism. These have\nimproved overall F1 scores, as compared to the previous benchmarks (More et\nal.) by at least 0.0391, showcasing an overall precision of 97.94%, recall of\n94.12% and the F1 score of 0.9599. This has made us achieve a significant\ncoverage of important facets or attributes of products which not only shows the\nefficacy of deep recurrent models over previous machine learning benchmarks but\nalso greatly enhances the overall customer experience while shopping online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00188v3",
            "date": "2018-05-01 05:05:05+00:00",
            "title": "Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems",
            "authors": [
                "Liu Yang",
                "Minghui Qiu",
                "Chen Qu",
                "Jiafeng Guo",
                "Yongfeng Zhang",
                "W. Bruce Croft",
                "Jun Huang",
                "Haiqing Chen"
            ],
            "abstract": "Intelligent personal assistant systems with either text-based or voice-based\nconversational interfaces are becoming increasingly popular around the world.\nRetrieval-based conversation models have the advantages of returning fluent and\ninformative responses. Most existing studies in this area are on open domain\n\"chit-chat\" conversations or task / transaction oriented conversations. More\nresearch is needed for information-seeking conversations. There is also a lack\nof modeling external knowledge beyond the dialog utterances among current\nconversational models. In this paper, we propose a learning framework on the\ntop of deep neural matching networks that leverages external knowledge for\nresponse ranking in information-seeking conversation systems. We incorporate\nexternal knowledge into deep neural models with pseudo-relevance feedback and\nQA correspondence knowledge distillation. Extensive experiments with three\ninformation-seeking conversation data sets including both open benchmarks and\ncommercial data show that, our methods outperform various baseline methods\nincluding several deep text matching models and the state-of-the-art method on\nresponse selection in multi-turn conversations. We also perform analysis over\ndifferent response types, model variations and ranking examples. Our models and\nresearch findings provide new insights on how to utilize external knowledge\nwith deep neural models for response selection and have implications for the\ndesign of the next generation of information-seeking conversation systems.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.00195v1",
            "date": "2018-05-01 05:52:12+00:00",
            "title": "An Annotated Corpus for Machine Reading of Instructions in Wet Lab Protocols",
            "authors": [
                "Chaitanya Kulkarni",
                "Wei Xu",
                "Alan Ritter",
                "Raghu Machiraju"
            ],
            "abstract": "We describe an effort to annotate a corpus of natural language instructions\nconsisting of 622 wet lab protocols to facilitate automatic or semi-automatic\nconversion of protocols into a machine-readable format and benefit biological\nresearch. Experimental results demonstrate the utility of our corpus for\ndeveloping machine learning approaches to shallow semantic parsing of\ninstructional texts. We make our annotated Wet Lab Protocol Corpus available to\nthe research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00250v2",
            "date": "2018-05-01 09:21:34+00:00",
            "title": "Adaptive Scaling for Sparse Detection in Information Extraction",
            "authors": [
                "Hongyu Lin",
                "Yaojie Lu",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "This paper focuses on detection tasks in information extraction, where\npositive instances are sparsely distributed and models are usually evaluated\nusing F-measure on positive classes. These characteristics often result in\ndeficient performance of neural network based detection models. In this paper,\nwe propose adaptive scaling, an algorithm which can handle the positive\nsparsity problem and directly optimize over F-measure via dynamic\ncost-sensitive learning. To this end, we borrow the idea of marginal utility\nfrom economics and propose a theoretical framework for instance importance\nmeasuring without introducing any additional hyper-parameters. Experiments show\nthat our algorithm leads to a more effective and stable training of neural\nnetwork based detection models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00352v1",
            "date": "2018-05-01 14:07:44+00:00",
            "title": "Word2Vec and Doc2Vec in Unsupervised Sentiment Analysis of Clinical Discharge Summaries",
            "authors": [
                "Qufei Chen",
                "Marina Sokolova"
            ],
            "abstract": "In this study, we explored application of Word2Vec and Doc2Vec for sentiment\nanalysis of clinical discharge summaries. We applied unsupervised learning\nsince the data sets did not have sentiment annotations. Note that unsupervised\nlearning is a more realistic scenario than supervised learning which requires\nan access to a training set of sentiment-annotated data. We aim to detect if\nthere exists any underlying bias towards or against a certain disease. We used\nSentiWordNet to establish a gold sentiment standard for the data sets and\nevaluate performance of Word2Vec and Doc2Vec methods. We have shown that the\nWord2vec and Doc2Vec methods complement each other results in sentiment\nanalysis of the data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00444v1",
            "date": "2018-04-20 12:25:59+00:00",
            "title": "Skin Tone Emoji and Sentiment on Twitter",
            "authors": [
                "Steven Coats"
            ],
            "abstract": "In 2015, the Unicode Consortium introduced five skin tone emoji that can be\nused in combination with emoji representing human figures and body parts. In\nthis study, use of the skin tone emoji is analyzed geographically in a large\nsample of data from Twitter. It can be shown that values for the skin tone\nemoji by country correspond approximately to the skin tone of the resident\npopulations, and that a negative correlation exists between tweet sentiment and\ndarker skin tone at the global level. In an era of large-scale migrations and\ncontinued sensitivity to questions of skin color and race, understanding how\nnew language elements such as skin tone emoji are used can help frame our\nunderstanding of how people represent themselves and others in terms of a\nsalient personal appearance attribute.",
            "journal ref": "Proc. 3rd DHN Conference, 122-138.\n  http://ceur-ws.org/Vol-2084/paper10.pdf",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1805.00462v1",
            "date": "2018-04-26 07:14:59+00:00",
            "title": "Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game",
            "authors": [
                "Haichao Zhang",
                "Haonan Yu",
                "Wei Xu"
            ],
            "abstract": "Building intelligent agents that can communicate with and learn from humans\nin natural language is of great value. Supervised language learning is limited\nby the ability of capturing mainly the statistics of training data, and is\nhardly adaptive to new scenarios or flexible for acquiring new knowledge\nwithout inefficient retraining or catastrophic forgetting. We highlight the\nperspective that conversational interaction serves as a natural interface both\nfor language learning and for novel knowledge acquisition and propose a joint\nimitation and reinforcement approach for grounded language learning through an\ninteractive conversational game. The agent trained with this approach is able\nto actively acquire information by asking questions about novel objects and use\nthe just-learned knowledge in subsequent conversations in a one-shot fashion.\nResults compared with other methods verified the effectiveness of the proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00676v1",
            "date": "2018-05-02 08:47:38+00:00",
            "title": "Text to Image Synthesis Using Generative Adversarial Networks",
            "authors": [
                "Cristian Bodnar"
            ],
            "abstract": "Generating images from natural language is one of the primary applications of\nrecent conditional generative models. Besides testing our ability to model\nconditional, highly dimensional distributions, text to image synthesis has many\nexciting and practical applications such as photo editing or computer-aided\ncontent creation. Recent progress has been made using Generative Adversarial\nNetworks (GANs). This material starts with a gentle introduction to these\ntopics and discusses the existent state of the art models. Moreover, I propose\nWasserstein GAN-CLS, a new model for conditional image generation based on the\nWasserstein distance which offers guarantees of stability. Then, I show how the\nnovel loss function of Wasserstein GAN-CLS can be used in a Conditional\nProgressive Growing GAN. In combination with the proposed loss, the model\nboosts by 7.07% the best Inception Score (on the Caltech birds dataset) of the\nmodels which use only the sentence-level visual semantics. The only model which\nperforms better than the Conditional Wasserstein Progressive Growing GAN is the\nrecently proposed AttnGAN which uses word-level visual semantics as well.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.00741v1",
            "date": "2018-05-02 11:33:45+00:00",
            "title": "KNPTC: Knowledge and Neural Machine Translation Powered Chinese Pinyin Typo Correction",
            "authors": [
                "Hengyi Cai",
                "Xingguang Ji",
                "Yonghao Song",
                "Yan Jin",
                "Yang Zhang",
                "Mairgup Mansur",
                "Xiaofang Zhao"
            ],
            "abstract": "Chinese pinyin input methods are very important for Chinese language\nprocessing. Actually, users may make typos inevitably when they input pinyin.\nMoreover, pinyin typo correction has become an increasingly important task with\nthe popularity of smartphones and the mobile Internet. How to exploit the\nknowledge of users typing behaviors and support the typo correction for acronym\npinyin remains a challenging problem. To tackle these challenges, we propose\nKNPTC, a novel approach based on neural machine translation (NMT). In contrast\nto previous work, KNPTC is able to integrate explicit knowledge into NMT for\npinyin typo correction, and is able to learn to correct a variety of typos\nwithout the guidance of manually selected constraints or languagespecific\nfeatures. In this approach, we first obtain the transition probabilities\nbetween adjacent letters based on large-scale real-life datasets. Then, we\nconstruct the \"ground-truth\" alignments of training sentence pairs by utilizing\nthese probabilities. Furthermore, these alignments are integrated into NMT to\ncapture sensible pinyin typo correction patterns. KNPTC is applied to correct\ntypos in real-life datasets, which achieves 32.77% increment on average in\naccuracy rate of typo correction compared against the state-of-the-art system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00912v4",
            "date": "2018-05-02 17:16:48+00:00",
            "title": "Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together",
            "authors": [
                "Tao Shen",
                "Tianyi Zhou",
                "Guodong Long",
                "Jing Jiang",
                "Chengqi Zhang"
            ],
            "abstract": "Neural networks equipped with self-attention have parallelizable computation,\nlight-weight structure, and the ability to capture both long-range and local\ndependencies. Further, their expressive power and performance can be boosted by\nusing a vector to measure pairwise dependency, but this requires to expand the\nalignment matrix to a tensor, which results in memory and computation\nbottlenecks. In this paper, we propose a novel attention mechanism called\n\"Multi-mask Tensorized Self-Attention\" (MTSA), which is as fast and as\nmemory-efficient as a CNN, but significantly outperforms previous\nCNN-/RNN-/attention-based models. MTSA 1) captures both pairwise (token2token)\nand global (source2token) dependencies by a novel compatibility function\ncomposed of dot-product and additive attentions, 2) uses a tensor to represent\nthe feature-wise alignment scores for better expressive power but only requires\nparallelizable matrix multiplications, and 3) combines multi-head with\nmulti-dimensional attentions, and applies a distinct positional mask to each\nhead (subspace), so the memory and computation can be distributed to multiple\nheads, each with sequential information encoded independently. The experiments\nshow that a CNN/RNN-free model based on MTSA achieves state-of-the-art or\ncompetitive performance on nine NLP benchmarks with compelling memory- and\ntime-efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01083v1",
            "date": "2018-05-03 01:57:31+00:00",
            "title": "Scalable Semantic Querying of Text",
            "authors": [
                "Xiaolan Wang",
                "Aaron Feng",
                "Behzad Golshan",
                "Alon Halevy",
                "George Mihaila",
                "Hidekazu Oiwa",
                "Wang-Chiew Tan"
            ],
            "abstract": "We present the KOKO system that takes declarative information extraction to a\nnew level by incorporating advances in natural language processing techniques\nin its extraction language. KOKO is novel in that its extraction language\nsimultaneously supports conditions on the surface of the text and on the\nstructure of the dependency parse tree of sentences, thereby allowing for more\nrefined extractions. KOKO also supports conditions that are forgiving to\nlinguistic variation of expressing concepts and allows to aggregate evidence\nfrom the entire document in order to filter extractions.\n  To scale up, KOKO exploits a multi-indexing scheme and heuristics for\nefficient extractions. We extensively evaluate KOKO over publicly available\ntext corpora. We show that KOKO indices take up the smallest amount of space,\nare notably faster and more effective than a number of prior indexing schemes.\nFinally, we demonstrate KOKO's scale up on a corpus of 5 million Wikipedia\narticles.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1805.01087v1",
            "date": "2018-05-03 02:23:28+00:00",
            "title": "Stack-Pointer Networks for Dependency Parsing",
            "authors": [
                "Xuezhe Ma",
                "Zecong Hu",
                "Jingzhou Liu",
                "Nanyun Peng",
                "Graham Neubig",
                "Eduard Hovy"
            ],
            "abstract": "We introduce a novel architecture for dependency parsing: \\emph{stack-pointer\nnetworks} (\\textbf{\\textsc{StackPtr}}). Combining pointer\nnetworks~\\citep{vinyals2015pointer} with an internal stack, the proposed model\nfirst reads and encodes the whole sentence, then builds the dependency tree\ntop-down (from root-to-leaf) in a depth-first fashion. The stack tracks the\nstatus of the depth-first search and the pointer networks select one child for\nthe word at the top of the stack at each step. The \\textsc{StackPtr} parser\nbenefits from the information of the whole sentence and all previously derived\nsubtree structures, and removes the left-to-right restriction in classical\ntransition-based parsers. Yet, the number of steps for building any (including\nnon-projective) parse tree is linear in the length of the sentence just as\nother transition-based parsers, yielding an efficient decoding algorithm with\n$O(n^2)$ time complexity. We evaluate our model on 29 treebanks spanning 20\nlanguages and different dependency annotation schemas, and achieve\nstate-of-the-art performance on 21 of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01089v2",
            "date": "2018-05-03 02:30:07+00:00",
            "title": "A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification",
            "authors": [
                "Shuming Ma",
                "Xu Sun",
                "Junyang Lin",
                "Xuancheng Ren"
            ],
            "abstract": "Text summarization and sentiment classification both aim to capture the main\nideas of the text but at different levels. Text summarization is to describe\nthe text within a few sentences, while sentiment classification can be regarded\nas a special type of summarization which \"summarizes\" the text into a even more\nabstract fashion, i.e., a sentiment class. Based on this idea, we propose a\nhierarchical end-to-end model for joint learning of text summarization and\nsentiment classification, where the sentiment classification label is treated\nas the further \"summarization\" of the text summarization output. Hence, the\nsentiment classification layer is put upon the text summarization layer, and a\nhierarchical structure is derived. Experimental results on Amazon online\nreviews datasets show that our model achieves better performance than the\nstrong baseline systems on both abstractive summarization and sentiment\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01334v1",
            "date": "2018-05-03 14:46:12+00:00",
            "title": "Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling",
            "authors": [
                "Chenyan Xiong",
                "Zhengzhong Liu",
                "Jamie Callan",
                "Tie-Yan Liu"
            ],
            "abstract": "This paper presents a Kernel Entity Salience Model (KESM) that improves text\nunderstanding and retrieval by better estimating entity salience (importance)\nin documents. KESM represents entities by knowledge enriched distributed\nrepresentations, models the interactions between entities and words by kernels,\nand combines the kernel scores to estimate entity salience. The whole model is\nlearned end-to-end using entity salience labels. The salience model also\nimproves ad hoc search accuracy, providing effective ranking features by\nmodeling the salience of query entities in candidate documents. Our experiments\non two entity salience corpora and two TREC ad hoc search datasets demonstrate\nthe effectiveness of KESM over frequency-based and feature-based methods. We\nalso provide examples showing how KESM conveys its text understanding ability\nlearned from entity salience to search.",
            "journal ref": "In proceedings of SIGIR 2018",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.01460v1",
            "date": "2018-05-02 23:07:31+00:00",
            "title": "Robustness of sentence length measures in written texts",
            "authors": [
                "Denner S. Vieira",
                "Sergio Picoli",
                "Renio S. Mendes"
            ],
            "abstract": "Hidden structural patterns in written texts have been subject of considerable\nresearch in the last decades. In particular, mapping a text into a time series\nof sentence lengths is a natural way to investigate text structure. Typically,\nsentence length has been quantified by using measures based on the number of\nwords and the number of characters, but other variations are possible. To\nquantify the robustness of different sentence length measures, we analyzed a\ndatabase containing about five hundred books in English. For each book, we\nextracted six distinct measures of sentence length, including number of words\nand number of characters (taking into account lemmatization and stop words\nremoval). We compared these six measures for each book by using i) Pearson's\ncoefficient to investigate linear correlations; ii) Kolmogorov--Smirnov test to\ncompare distributions; and iii) detrended fluctuation analysis (DFA) to\nquantify auto-correlations. We have found that all six measures exhibit very\nsimilar behavior, suggesting that sentence length is a robust measure related\nto text structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01553v3",
            "date": "2018-05-03 21:50:34+00:00",
            "title": "A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation",
            "authors": [
                "Tsz Kin Lam",
                "Julia Kreutzer",
                "Stefan Riezler"
            ],
            "abstract": "We present an approach to interactive-predictive neural machine translation\nthat attempts to reduce human effort from three directions: Firstly, instead of\nrequiring humans to select, correct, or delete segments, we employ the idea of\nlearning from human reinforcements in form of judgments on the quality of\npartial translations. Secondly, human effort is further reduced by using the\nentropy of word predictions as uncertainty criterion to trigger feedback\nrequests. Lastly, online updates of the model parameters after every\ninteraction allow the model to adapt quickly. We show in simulation experiments\nthat reward signals on partial translations significantly improve character\nF-score and BLEU compared to feedback on full translations only, while human\neffort can be reduced to an average number of $5$ feedback requests for every\ninput.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01589v1",
            "date": "2018-05-04 02:26:21+00:00",
            "title": "When Politicians Talk About Politics: Identifying Political Tweets of Brazilian Congressmen",
            "authors": [
                "Lucas S. Oliveira",
                "Pedro O. S. Vaz de Melo",
                "Marcelo S. Amaral",
                "Jos\u00e9 Ant\u00f4nio. G. Pinho"
            ],
            "abstract": "Since June 2013, when Brazil faced the largest and most significant mass\nprotests in a generation, a political crisis is in course. In midst of this\ncrisis, Brazilian politicians use social media to communicate with the\nelectorate in order to retain or to grow their political capital. The problem\nis that many controversial topics are in course and deputies may prefer to\navoid such themes in their messages. To characterize this behavior, we propose\na method to accurately identify political and non-political tweets\nindependently of the deputy who posted it and of the time it was posted.\nMoreover, we collected tweets of all congressmen who were active on Twitter and\nworked in the Brazilian parliament from October 2013 to October 2017. To\nevaluate our method, we used word clouds and a topic model to identify the main\npolitical and non-political latent topics in parliamentarian tweets. Both\nresults indicate that our proposal is able to accurately distinguish political\nfrom non-political tweets. Moreover, our analyses revealed a striking fact:\nmore than half of the messages posted by Brazilian deputies are non-political.",
            "journal ref": "2018, THE 12TH INTERNATIONAL AAAI CONFERENCE ON WEB AND SOCIAL\n  MEDIA (ICWSM-18)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1805.02400v4",
            "date": "2018-05-07 08:37:04+00:00",
            "title": "Stay On-Topic: Generating Context-specific Fake Restaurant Reviews",
            "authors": [
                "Mika Juuti",
                "Bo Sun",
                "Tatsuya Mori",
                "N. Asokan"
            ],
            "abstract": "Automatically generated fake restaurant reviews are a threat to online review\nsystems. Recent research has shown that users have difficulties in detecting\nmachine-generated fake reviews hiding among real restaurant reviews. The method\nused in this work (char-LSTM ) has one drawback: it has difficulties staying in\ncontext, i.e. when it generates a review for specific target entity, the\nresulting review may contain phrases that are unrelated to the target, thus\nincreasing its detectability. In this work, we present and evaluate a more\nsophisticated technique based on neural machine translation (NMT) with which we\ncan generate reviews that stay on-topic. We test multiple variants of our\ntechnique using native English speakers on Amazon Mechanical Turk. We\ndemonstrate that reviews generated by the best variant have almost optimal\nundetectability (class-averaged F-score 47%). We conduct a user study with\nskeptical users and show that our method evades detection more frequently\ncompared to the state-of-the-art (average evasion 3.2/4 vs 1.5/4) with\nstatistical significance, at level {\\alpha} = 1% (Section 4.3). We develop very\neffective detection tools and reach average F-score of 97% in classifying\nthese. Although fake reviews are very effective in fooling people, effective\nautomatic detection is still feasible.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1805.02408v2",
            "date": "2018-05-07 09:03:14+00:00",
            "title": "Improving Knowledge Graph Embedding Using Simple Constraints",
            "authors": [
                "Boyang Ding",
                "Quan Wang",
                "Bin Wang",
                "Li Guo"
            ],
            "abstract": "Embedding knowledge graphs (KGs) into continuous vector spaces is a focus of\ncurrent research. Early works performed this task via simple models developed\nover KG triples. Recent attempts focused on either designing more complicated\ntriple scoring models, or incorporating extra information beyond triples. This\npaper, by contrast, investigates the potential of using very simple constraints\nto improve KG embedding. We examine non-negativity constraints on entity\nrepresentations and approximate entailment constraints on relation\nrepresentations. The former help to learn compact and interpretable\nrepresentations for entities. The latter further encode regularities of logical\nentailment between relations into their distributed representations. These\nconstraints impose prior beliefs upon the structure of the embedding space,\nwithout negative impacts on efficiency or scalability. Evaluation on WordNet,\nFreebase, and DBpedia shows that our approach is simple yet surprisingly\neffective, significantly and consistently outperforming competitive baselines.\nThe constraints imposed indeed improve model interpretability, leading to a\nsubstantially increased structuring of the embedding space. Code and data are\navailable at https://github.com/iieir-km/ComplEx-NNE_AER.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.03308v1",
            "date": "2018-05-08 22:22:26+00:00",
            "title": "Investor Reaction to Financial Disclosures Across Topics: An Application of Latent Dirichlet Allocation",
            "authors": [
                "Stefan Feuerriegel",
                "Nicolas Pr\u00f6llochs"
            ],
            "abstract": "This paper provides a holistic study of how stock prices vary in their\nresponse to financial disclosures across different topics. Thereby, we\nspecifically shed light into the extensive amount of filings for which no a\npriori categorization of their content exists. For this purpose, we utilize an\napproach from data mining - namely, latent Dirichlet allocation - as a means of\ntopic modeling. This technique facilitates our task of automatically\ncategorizing, ex ante, the content of more than 70,000 regulatory 8-K filings\nfrom U.S. companies. We then evaluate the subsequent stock market reaction. Our\nempirical evidence suggests a considerable discrepancy among various types of\nnews stories in terms of their relevance and impact on financial markets. For\ninstance, we find a statistically significant abnormal return in response to\nearnings results and credit rating, but also for disclosures regarding business\nstrategy, the health sector, as well as mergers and acquisitions. Our results\nyield findings that benefit managers, investors and policy-makers by indicating\nhow regulatory filings should be structured and the topics most likely to\nprecede changes in stock valuations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03793v1",
            "date": "2018-05-10 02:42:03+00:00",
            "title": "hyperdoc2vec: Distributed Representations of Hypertext Documents",
            "authors": [
                "Jialong Han",
                "Yan Song",
                "Wayne Xin Zhao",
                "Shuming Shi",
                "Haisong Zhang"
            ],
            "abstract": "Hypertext documents, such as web pages and academic papers, are of great\nimportance in delivering information in our daily life. Although being\neffective on plain documents, conventional text embedding methods suffer from\ninformation loss if directly adapted to hyper-documents. In this paper, we\npropose a general embedding approach for hyper-documents, namely, hyperdoc2vec,\nalong with four criteria characterizing necessary information that\nhyper-document embedding models should preserve. Systematic comparisons are\nconducted between hyperdoc2vec and several competitors on two tasks, i.e.,\npaper classification and citation recommendation, in the academic paper domain.\nAnalyses and experiments both validate the superiority of hyperdoc2vec to other\nmodels w.r.t. the four criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03801v1",
            "date": "2018-05-10 03:39:32+00:00",
            "title": "Learning Domain-Sensitive and Sentiment-Aware Word Embeddings",
            "authors": [
                "Bei Shi",
                "Zihao Fu",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "Word embeddings have been widely used in sentiment classification because of\ntheir efficacy for semantic representations of words. Given reviews from\ndifferent domains, some existing methods for word embeddings exploit sentiment\ninformation, but they cannot produce domain-sensitive embeddings. On the other\nhand, some other existing methods can generate domain-sensitive word\nembeddings, but they cannot distinguish words with similar contexts but\nopposite sentiment polarity. We propose a new method for learning\ndomain-sensitive and sentiment-aware embeddings that simultaneously capture the\ninformation of sentiment semantics and domain sensitivity of individual words.\nOur method can automatically determine and produce domain-common embeddings and\ndomain-specific embeddings. The differentiation of domain-common and\ndomain-specific words enables the advantage of data augmentation of common\nsemantics from multiple domains and capture the varied semantics of specific\nwords from different domains at the same time. Experimental results show that\nour model provides an effective way to learn domain-sensitive and\nsentiment-aware word embeddings which benefit sentiment classification at both\nsentence level and lexicon term level.",
            "journal ref": "ACL2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03830v1",
            "date": "2018-05-10 05:47:46+00:00",
            "title": "Towards Inference-Oriented Reading Comprehension: ParallelQA",
            "authors": [
                "Soumya Wadhwa",
                "Varsha Embar",
                "Matthias Grabmair",
                "Eric Nyberg"
            ],
            "abstract": "In this paper, we investigate the tendency of end-to-end neural Machine\nReading Comprehension (MRC) models to match shallow patterns rather than\nperform inference-oriented reasoning on RC benchmarks. We aim to test the\nability of these systems to answer questions which focus on referential\ninference. We propose ParallelQA, a strategy to formulate such questions using\nparallel passages. We also demonstrate that existing neural models fail to\ngeneralize well to this setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03832v2",
            "date": "2018-05-10 05:54:32+00:00",
            "title": "A comparable study of modeling units for end-to-end Mandarin speech recognition",
            "authors": [
                "Wei Zou",
                "Dongwei Jiang",
                "Shuaijiang Zhao",
                "Xiangang Li"
            ],
            "abstract": "End-To-End speech recognition have become increasingly popular in mandarin\nspeech recognition and achieved delightful performance.\n  Mandarin is a tonal language which is different from English and requires\nspecial treatment for the acoustic modeling units. There have been several\ndifferent kinds of modeling units for mandarin such as phoneme, syllable and\nChinese character.\n  In this work, we explore two major end-to-end models: connectionist temporal\nclassification (CTC) model and attention based encoder-decoder model for\nmandarin speech recognition. We compare the performance of three different\nscaled modeling units: context dependent phoneme(CDP), syllable with tone and\nChinese character.\n  We find that all types of modeling units can achieve approximate character\nerror rate (CER) in CTC model and the performance of Chinese character\nattention model is better than syllable attention model. Furthermore, we find\nthat Chinese character is a reasonable unit for mandarin speech recognition. On\nDidiCallcenter task, Chinese character attention model achieves a CER of 5.68%\nand CTC model gets a CER of 7.29%, on the other DidiReading task, CER are 4.89%\nand 5.79%, respectively. Moreover, attention model achieves a better\nperformance than CTC model on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04032v3",
            "date": "2018-05-10 15:56:48+00:00",
            "title": "From Word to Sense Embeddings: A Survey on Vector Representations of Meaning",
            "authors": [
                "Jose Camacho-Collados",
                "Mohammad Taher Pilehvar"
            ],
            "abstract": "Over the past years, distributed semantic representations have proved to be\neffective and flexible keepers of prior knowledge to be integrated into\ndownstream applications. This survey focuses on the representation of meaning.\nWe start from the theoretical background behind word vector space models and\nhighlight one of their major limitations: the meaning conflation deficiency,\nwhich arises from representing a word with all its possible meanings as a\nsingle vector. Then, we explain how this deficiency can be addressed through a\ntransition from the word level to the more fine-grained level of word senses\n(in its broader acceptation) as a method for modelling unambiguous lexical\nmeaning. We present a comprehensive overview of the wide range of techniques in\nthe two main branches of sense representation, i.e., unsupervised and\nknowledge-based. Finally, this survey covers the main evaluation procedures and\napplications for this type of representation, and provides an analysis of four\nof its important aspects: interpretability, sense granularity, adaptability to\ndifferent domains and compositionality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04174v1",
            "date": "2018-05-10 20:42:52+00:00",
            "title": "Joint Embedding of Words and Labels for Text Classification",
            "authors": [
                "Guoyin Wang",
                "Chunyuan Li",
                "Wenlin Wang",
                "Yizhe Zhang",
                "Dinghan Shen",
                "Xinyuan Zhang",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "Word embeddings are effective intermediate representations for capturing\nsemantic regularities between words, when learning the representations of text\nsequences. We propose to view text classification as a label-word joint\nembedding problem: each label is embedded in the same space with the word\nvectors. We introduce an attention framework that measures the compatibility of\nembeddings between text sequences and labels. The attention is learned on a\ntraining set of labeled samples to ensure that, given a text sequence, the\nrelevant words are weighted higher than the irrelevant ones. Our method\nmaintains the interpretability of word embeddings, and enjoys a built-in\nability to leverage alternative sources of information, in addition to input\ntext sequences. Extensive results on the several large text datasets show that\nthe proposed framework outperforms the state-of-the-art methods by a large\nmargin, in terms of both accuracy and speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04212v1",
            "date": "2018-05-11 00:43:59+00:00",
            "title": "Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness",
            "authors": [
                "Vicente Ivan Sanchez Carmona",
                "Jeff Mitchell",
                "Sebastian Riedel"
            ],
            "abstract": "Natural Language Inference is a challenging task that has received\nsubstantial attention, and state-of-the-art models now achieve impressive test\nset performance in the form of accuracy scores. Here, we go beyond this single\nevaluation metric to examine robustness to semantically-valid alterations to\nthe input data. We identify three factors - insensitivity, polarity and unseen\npairs - and compare their impact on three SNLI models under a variety of\nconditions. Our results demonstrate a number of strengths and weaknesses in the\nmodels' ability to generalise to new in-domain instances. In particular, while\nstrong performance is possible on unseen hypernyms, unseen antonyms are more\nchallenging for all the models. More generally, the models suffer from an\ninsensitivity to certain small but semantically significant alterations, and\nare also often influenced by simple statistical correlations between words and\ntraining labels. Overall, we show that evaluations of NLI models can benefit\nfrom studying the influence of factors intrinsic to the models or found in the\ndataset used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04264v2",
            "date": "2018-05-11 07:51:28+00:00",
            "title": "State Gradients for RNN Memory Analysis",
            "authors": [
                "Lyan Verwimp",
                "Hugo Van hamme",
                "Vincent Renkens",
                "Patrick Wambacq"
            ],
            "abstract": "We present a framework for analyzing what the state in RNNs remembers from\nits input embeddings. Our approach is inspired by backpropagation, in the sense\nthat we compute the gradients of the states with respect to the input\nembeddings. The gradient matrix is decomposed with Singular Value Decomposition\nto analyze which directions in the embedding space are best transferred to the\nhidden state space, characterized by the largest singular values. We apply our\napproach to LSTM language models and investigate to what extent and for how\nlong certain classes of words are remembered on average for a certain corpus.\nAdditionally, the extent to which a specific property or relationship is\nremembered by the RNN can be tracked by comparing a vector characterizing that\nproperty with the direction(s) in embedding space that are best preserved in\nhidden state space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04402v2",
            "date": "2018-05-11 13:50:26+00:00",
            "title": "Decision problems for Clark-congruential languages",
            "authors": [
                "Makoto Kanazawa",
                "Tobias Kapp\u00e9"
            ],
            "abstract": "A common question when studying a class of context-free grammars is whether\nequivalence is decidable within this class. We answer this question positively\nfor the class of Clark-congruential grammars, which are of interest to\ngrammatical inference. We also consider the problem of checking whether a given\nCFG is Clark-congruential, and show that it is decidable given that the CFG is\na DCFG.",
            "journal ref": "PMLR 93:3-16, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04437v1",
            "date": "2018-05-11 15:01:00+00:00",
            "title": "Cross-lingual Document Retrieval using Regularized Wasserstein Distance",
            "authors": [
                "Georgios Balikas",
                "Charlotte Laclau",
                "Ievgen Redko",
                "Massih-Reza Amini"
            ],
            "abstract": "Many information retrieval algorithms rely on the notion of a good distance\nthat allows to efficiently compare objects of different nature. Recently, a new\npromising metric called Word Mover's Distance was proposed to measure the\ndivergence between text passages. In this paper, we demonstrate that this\nmetric can be extended to incorporate term-weighting schemes and provide more\naccurate and computationally efficient matching between documents using\nentropic regularization. We evaluate the benefits of both extensions in the\ntask of cross-lingual document retrieval (CLDR). Our experimental results on\neight CLDR problems suggest that the proposed methods achieve remarkable\nimprovements in terms of Mean Reciprocal Rank compared to several baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04688v1",
            "date": "2018-05-12 09:27:53+00:00",
            "title": "Gaussian Mixture Latent Vector Grammars",
            "authors": [
                "Yanpeng Zhao",
                "Liwen Zhang",
                "Kewei Tu"
            ],
            "abstract": "We introduce Latent Vector Grammars (LVeGs), a new framework that extends\nlatent variable grammars such that each nonterminal symbol is associated with a\ncontinuous vector space representing the set of (infinitely many) subtypes of\nthe nonterminal. We show that previous models such as latent variable grammars\nand compositional vector grammars can be interpreted as special cases of LVeGs.\nWe then present Gaussian Mixture LVeGs (GM-LVeGs), a new special case of LVeGs\nthat uses Gaussian mixtures to formulate the weights of production rules over\nsubtypes of nonterminals. A major advantage of using Gaussian mixtures is that\nthe partition function and the expectations of subtype rules can be computed\nusing an extension of the inside-outside algorithm, which enables efficient\ninference and learning. We apply GM-LVeGs to part-of-speech tagging and\nconstituency parsing and show that GM-LVeGs can achieve competitive accuracies.\nOur code is available at https://github.com/zhaoyanpeng/lveg.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04803v1",
            "date": "2018-05-13 01:07:32+00:00",
            "title": "Zero-Shot Dialog Generation with Cross-Domain Latent Actions",
            "authors": [
                "Tiancheng Zhao",
                "Maxine Eskenazi"
            ],
            "abstract": "This paper introduces zero-shot dialog generation (ZSDG), as a step towards\nneural dialog systems that can instantly generalize to new situations with\nminimal data. ZSDG enables an end-to-end generative dialog system to generalize\nto a new domain for which only a domain description is provided and no training\ndialogs are available. Then a novel learning framework, Action Matching, is\nproposed. This algorithm can learn a cross-domain embedding space that models\nthe semantics of dialog responses which, in turn, lets a neural dialog\ngeneration model generalize to new domains. We evaluate our methods on a new\nsynthetic dialog dataset, and an existing human-human dialog dataset. Results\nshow that our method has superior performance in learning dialog models that\nrapidly adapt their behavior to new domains and suggests promising future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04813v2",
            "date": "2018-05-13 03:35:09+00:00",
            "title": "Triangular Architecture for Rare Language Translation",
            "authors": [
                "Shuo Ren",
                "Wenhu Chen",
                "Shujie Liu",
                "Mu Li",
                "Ming Zhou",
                "Shuai Ma"
            ],
            "abstract": "Neural Machine Translation (NMT) performs poor on the low-resource language\npair $(X,Z)$, especially when $Z$ is a rare language. By introducing another\nrich language $Y$, we propose a novel triangular training architecture (TA-NMT)\nto leverage bilingual data $(Y,Z)$ (may be small) and $(X,Y)$ (can be rich) to\nimprove the translation performance of low-resource pairs. In this triangular\narchitecture, $Z$ is taken as the intermediate latent variable, and translation\nmodels of $Z$ are jointly optimized with a unified bidirectional EM algorithm\nunder the goal of maximizing the translation likelihood of $(X,Y)$. Empirical\nresults demonstrate that our method significantly improves the translation\nquality of rare languages on MultiUN and IWSLT2012 datasets, and achieves even\nbetter performance combining back-translation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05062v1",
            "date": "2018-05-14 08:37:50+00:00",
            "title": "Token-level and sequence-level loss smoothing for RNN language models",
            "authors": [
                "Maha Elbayad",
                "Laurent Besacier",
                "Jakob Verbeek"
            ],
            "abstract": "Despite the effectiveness of recurrent neural network language models, their\nmaximum likelihood estimation suffers from two limitations. It treats all\nsentences that do not match the ground truth as equally poor, ignoring the\nstructure of the output space. Second, it suffers from \"exposure bias\": during\ntraining tokens are predicted given ground-truth sequences, while at test time\nprediction is conditioned on generated output sequences. To overcome these\nlimitations we build upon the recent reward augmented maximum likelihood\napproach \\ie sequence-level smoothing that encourages the model to predict\nsentences close to the ground truth according to a given performance metric. We\nextend this approach to token-level loss smoothing, and propose improvements to\nthe sequence-level smoothing approach. Our experiments on two different tasks,\nimage captioning and machine translation, show that token-level and\nsequence-level loss smoothing are complementary, and significantly improve\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05081v2",
            "date": "2018-05-14 09:23:26+00:00",
            "title": "Constructing Narrative Event Evolutionary Graph for Script Event Prediction",
            "authors": [
                "Zhongyang Li",
                "Xiao Ding",
                "Ting Liu"
            ],
            "abstract": "Script event prediction requires a model to predict the subsequent event\ngiven an existing event context. Previous models based on event pairs or event\nchains cannot make full use of dense event connections, which may limit their\ncapability of event prediction. To remedy this, we propose constructing an\nevent graph to better utilize the event network information for script event\nprediction. In particular, we first extract narrative event chains from large\nquantities of news corpus, and then construct a narrative event evolutionary\ngraph (NEEG) based on the extracted chains. NEEG can be seen as a knowledge\nbase that describes event evolutionary principles and patterns. To solve the\ninference problem on NEEG, we present a scaled graph neural network (SGNN) to\nmodel event interactions and learn better event representations. Instead of\ncomputing the representations on the whole graph, SGNN processes only the\nconcerned nodes each time, which makes our model feasible to large-scale\ngraphs. By comparing the similarity between input context event representations\nand candidate event representations, we can choose the most reasonable\nsubsequent event. Experimental results on widely used New York Times corpus\ndemonstrate that our model significantly outperforms state-of-the-art baseline\nmethods, by using standard multiple choice narrative cloze evaluation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.05377v1",
            "date": "2018-05-14 18:50:11+00:00",
            "title": "Large-Scale QA-SRL Parsing",
            "authors": [
                "Nicholas FitzGerald",
                "Julian Michael",
                "Luheng He",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present a new large-scale corpus of Question-Answer driven Semantic Role\nLabeling (QA-SRL) annotations, and the first high-quality QA-SRL parser. Our\ncorpus, QA-SRL Bank 2.0, consists of over 250,000 question-answer pairs for\nover 64,000 sentences across 3 domains and was gathered with a new\ncrowd-sourcing scheme that we show has high precision and good recall at modest\ncost. We also present neural models for two QA-SRL subtasks: detecting argument\nspans for a predicate and generating questions to label the semantic\nrelationship. The best models achieve question accuracy of 82.6% and span-level\naccuracy of 77.6% (under human evaluation) on the full pipelined QA-SRL\nprediction task. They can also, as we show, be used to gather additional\nannotations at low cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05388v1",
            "date": "2018-05-14 19:11:33+00:00",
            "title": "A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors",
            "authors": [
                "Mikhail Khodak",
                "Nikunj Saunshi",
                "Yingyu Liang",
                "Tengyu Ma",
                "Brandon Stewart",
                "Sanjeev Arora"
            ],
            "abstract": "Motivations like domain adaptation, transfer learning, and feature learning\nhave fueled interest in inducing embeddings for rare or unseen words, n-grams,\nsynsets, and other textual features. This paper introduces a la carte\nembedding, a simple and general alternative to the usual word2vec-based\napproaches for building such representations that is based upon recent\ntheoretical results for GloVe-like embeddings. Our method relies mainly on a\nlinear transformation that is efficiently learnable using pretrained word\nvectors and linear regression. This transform is applicable on the fly in the\nfuture when a new text feature or rare word is encountered, even if only a\nsingle usage example is available. We introduce a new dataset showing how the a\nla carte method requires fewer examples of words in context to learn\nhigh-quality embeddings and we obtain state-of-the-art results on a nonce task\nand some unsupervised document classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05470v1",
            "date": "2018-05-09 12:36:04+00:00",
            "title": "Adaptive User-Oriented Direct Load-Control of Residential Flexible Devices",
            "authors": [
                "Davide Frazzetto",
                "Bijay Neupane",
                "Torben Bach Pedersen",
                "Thomas Dyhre Nielsen"
            ],
            "abstract": "Demand Response (DR) schemes are effective tools to maintain a dynamic\nbalance in energy markets with higher integration of fluctuating renewable\nenergy sources. DR schemes can be used to harness residential devices'\nflexibility and to utilize it to achieve social and financial objectives.\nHowever, existing DR schemes suffer from low user participation as they fail at\ntaking into account the users' requirements. First, DR schemes are highly\ndemanding for the users, as users need to provide direct information, e.g. via\nsurveys, on their energy consumption preferences. Second, the user utility\nmodels based on these surveys are hard-coded and do not adapt over time. Third,\nthe existing scheduling techniques require the users to input their energy\nrequirements on a daily basis. As an alternative, this paper proposes a DR\nscheme for user-oriented direct load-control of residential appliances\noperations. Instead of relying on user surveys to evaluate the user utility, we\npropose an online data-driven approach for estimating user utility functions,\npurely based on available load consumption data, that adaptively models the\nusers' preference over time. Our scheme is based on a day-ahead scheduling\ntechnique that transparently prescribes the users with optimal device operation\nschedules that take into account both financial benefits and user-perceived\nquality of service. To model day-ahead user energy demand and flexibility, we\npropose a probabilistic approach for generating flexibility models under\nuncertainty. Results on both real-world and simulated datasets show that our DR\nscheme can provide significant financial benefits while preserving the\nuser-perceived quality of service.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1805.05492v1",
            "date": "2018-05-14 23:10:28+00:00",
            "title": "Did the Model Understand the Question?",
            "authors": [
                "Pramod Kaushik Mudrakarta",
                "Ankur Taly",
                "Mukund Sundararajan",
                "Kedar Dhamdhere"
            ],
            "abstract": "We analyze state-of-the-art deep learning models for three tasks: question\nanswering on (1) images, (2) tables, and (3) passages of text. Using the notion\nof \\emph{attribution} (word importance), we find that these deep networks often\nignore important question terms. Leveraging such behavior, we perturb questions\nto craft a variety of adversarial examples. Our strongest attacks drop the\naccuracy of a visual question answering model from $61.1\\%$ to $19\\%$, and that\nof a tabular question answering model from $33.5\\%$ to $3.3\\%$. Additionally,\nwe show how attributions can strengthen attacks proposed by Jia and Liang\n(2017) on paragraph comprehension models. Our results demonstrate that\nattributions can augment standard measures of accuracy and empower\ninvestigation of model performance. When a model is accurate but for the wrong\nreasons, attributions can surface erroneous logic in the model that indicates\ninadequacies in the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06064v1",
            "date": "2018-05-15 23:13:23+00:00",
            "title": "Paper Abstract Writing through Editing Mechanism",
            "authors": [
                "Qingyun Wang",
                "Zhihao Zhou",
                "Lifu Huang",
                "Spencer Whitehead",
                "Boliang Zhang",
                "Heng Ji",
                "Kevin Knight"
            ],
            "abstract": "We present a paper abstract writing system based on an attentive neural\nsequence-to-sequence model that can take a title as input and automatically\ngenerate an abstract. We design a novel Writing-editing Network that can attend\nto both the title and the previously generated abstract drafts and then\niteratively revise and polish the abstract. With two series of Turing tests,\nwhere the human judges are asked to distinguish the system-generated abstracts\nfrom human-written ones, our system passes Turing tests by junior domain\nexperts at a rate up to 30% and by non-expert at a rate up to 80%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06201v1",
            "date": "2018-05-16 09:10:21+00:00",
            "title": "Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations",
            "authors": [
                "Sosuke Kobayashi"
            ],
            "abstract": "We propose a novel data augmentation for labeled sentences called contextual\naugmentation. We assume an invariance that sentences are natural even if the\nwords in the sentences are replaced with other words with paradigmatic\nrelations. We stochastically replace words with other words that are predicted\nby a bi-directional language model at the word positions. Words predicted\naccording to a context are numerous but appropriate for the augmentation of the\noriginal words. Furthermore, we retrofit a language model with a\nlabel-conditional architecture, which allows the model to augment sentences\nwithout breaking the label-compatibility. Through the experiments for six\nvarious different text classification tasks, we demonstrate that the proposed\nmethod improves classifiers based on the convolutional or recurrent neural\nnetworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06383v1",
            "date": "2018-05-16 15:56:17+00:00",
            "title": "Composing Finite State Transducers on GPUs",
            "authors": [
                "Arturo Argueta",
                "David Chiang"
            ],
            "abstract": "Weighted finite-state transducers (FSTs) are frequently used in language\nprocessing to handle tasks such as part-of-speech tagging and speech\nrecognition. There has been previous work using multiple CPU cores to\naccelerate finite state algorithms, but limited attention has been given to\nparallel graphics processing unit (GPU) implementations. In this paper, we\nintroduce the first (to our knowledge) GPU implementation of the FST\ncomposition operation, and we also discuss the optimizations used to achieve\nthe best performance on this architecture. We show that our approach obtains\nspeedups of up to 6x over our serial implementation and 4.5x over OpenFST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06503v1",
            "date": "2018-05-12 03:08:58+00:00",
            "title": "Weight Initialization in Neural Language Models",
            "authors": [
                "Ameet Deshpande",
                "Vedant Somani"
            ],
            "abstract": "Semantic Similarity is an important application which finds its use in many\ndownstream NLP applications. Though the task is mathematically defined,\nsemantic similarity's essence is to capture the notions of similarity\nimpregnated in humans. Machines use some heuristics to calculate the similarity\nbetween words, but these are typically corpus dependent or are useful for\nspecific domains. The difference between Semantic Similarity and Semantic\nRelatedness motivates the development of new algorithms. For a human, the word\ncar and road are probably as related as car and bus. But this may not be the\ncase for computational methods. Ontological methods are good at encoding\nSemantic Similarity and Vector Space models are better at encoding Semantic\nRelatedness. There is a dearth of methods which leverage ontologies to create\nbetter vector representations. The aim of this proposal is to explore in the\ndirection of a hybrid method which combines statistical/vector space methods\nlike Word2Vec and Ontological methods like WordNet to leverage the advantages\nprovided by both.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06504v1",
            "date": "2018-05-12 15:24:32+00:00",
            "title": "Analogical Reasoning on Chinese Morphological and Semantic Relations",
            "authors": [
                "Shen Li",
                "Zhe Zhao",
                "Renfen Hu",
                "Wensi Li",
                "Tao Liu",
                "Xiaoyong Du"
            ],
            "abstract": "Analogical reasoning is effective in capturing linguistic regularities. This\npaper proposes an analogical reasoning task on Chinese. After delving into\nChinese lexical knowledge, we sketch 68 implicit morphological relations and 28\nexplicit semantic relations. A big and balanced dataset CA8 is then built for\nthis task, including 17813 questions. Furthermore, we systematically explore\nthe influences of vector representations, context features, and corpora on\nanalogical reasoning. With the experiments, CA8 is proved to be a reliable\nbenchmark for evaluating Chinese word embeddings.",
            "journal ref": "Proceedings of the 56th Annual Meeting of the Association for\n  Computational Linguistics (Volume 2: Short Papers), 138--143, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06510v1",
            "date": "2018-05-04 04:46:07+00:00",
            "title": "Facebook Reaction-Based Emotion Classifier as Cue for Sarcasm Detection",
            "authors": [
                "Po Chen Kuo",
                "Fernando H. Calderon Alvarado",
                "Yi-Shin Chen"
            ],
            "abstract": "Online social media users react to content in them based on context. Emotions\nor mood play a significant part of these reactions, which has filled these\nplatforms with opinionated content. Different approaches and applications to\nmake better use of this data are continuously being developed. However, due to\nthe nature of the data, the variety of platforms, and dynamic online user\nbehavior, there are still many issues to be dealt with. It remains a challenge\nto properly obtain a reliable emotional status from a user prior to posting a\ncomment. This work introduces a methodology that explores semi-supervised\nmultilingual emotion detection based on the overlap of Facebook reactions and\ntextual data. With the resulting emotion detection system we evaluate the\npossibility of using emotions and user behavior features for the task of\nsarcasm detection. More than 1 million English and Chinese comments from over\n62,000 public Facebook pages posts have been collected and processed, conducted\nexperiments show acceptable performance metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06511v1",
            "date": "2018-05-09 05:38:14+00:00",
            "title": "Improving End-of-turn Detection in Spoken Dialogues by Detecting Speaker Intentions as a Secondary Task",
            "authors": [
                "Zakaria Aldeneh",
                "Dimitrios Dimitriadis",
                "Emily Mower Provost"
            ],
            "abstract": "This work focuses on the use of acoustic cues for modeling turn-taking in\ndyadic spoken dialogues. Previous work has shown that speaker intentions (e.g.,\nasking a question, uttering a backchannel, etc.) can influence turn-taking\nbehavior and are good predictors of turn-transitions in spoken dialogues.\nHowever, speaker intentions are not readily available for use by automated\nsystems at run-time; making it difficult to use this information to anticipate\na turn-transition. To this end, we propose a multi-task neural approach for\npredicting turn- transitions and speaker intentions simultaneously. Our results\nshow that adding the auxiliary task of speaker intention prediction improves\nthe performance of turn-transition prediction in spoken dialogues, without\nrelying on additional input features during run-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06593v2",
            "date": "2018-05-17 03:39:23+00:00",
            "title": "Cross-Target Stance Classification with Self-Attention Networks",
            "authors": [
                "Chang Xu",
                "Cecile Paris",
                "Surya Nepal",
                "Ross Sparks"
            ],
            "abstract": "In stance classification, the target on which the stance is made defines the\nboundary of the task, and a classifier is usually trained for prediction on the\nsame target. In this work, we explore the potential for generalizing\nclassifiers between different targets, and propose a neural model that can\napply what has been learned from a source target to a destination target. We\nshow that our model can find useful information shared between relevant targets\nwhich improves generalization in certain scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06816v1",
            "date": "2018-05-17 15:18:20+00:00",
            "title": "Annotating Electronic Medical Records for Question Answering",
            "authors": [
                "Preethi Raghavan",
                "Siddharth Patwardhan",
                "Jennifer J. Liang",
                "Murthy V. Devarakonda"
            ],
            "abstract": "Our research is in the relatively unexplored area of question answering\ntechnologies for patient-specific questions over their electronic health\nrecords. A large dataset of human expert curated question and answer pairs is\nan important pre-requisite for developing, training and evaluating any question\nanswering system that is powered by machine learning. In this paper, we\ndescribe a process for creating such a dataset of questions and answers. Our\nmethodology is replicable, can be conducted by medical students as annotators,\nand results in high inter-annotator agreement (0.71 Cohen's kappa). Over the\ncourse of 11 months, 11 medical students followed our annotation methodology,\nresulting in a question answering dataset of 5696 questions over 71 patient\nrecords, of which 1747 questions have corresponding answers generated by the\nmedical students.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07274v1",
            "date": "2018-05-17 10:43:04+00:00",
            "title": "Language Expansion In Text-Based Games",
            "authors": [
                "Ghulam Ahmed Ansari",
                "Sagar J P",
                "Sarath Chandar",
                "Balaraman Ravindran"
            ],
            "abstract": "Text-based games are suitable test-beds for designing agents that can learn\nby interaction with the environment in the form of natural language text. Very\nrecently, deep reinforcement learning based agents have been successfully\napplied for playing text-based games. In this paper, we explore the possibility\nof designing a single agent to play several text-based games and of expanding\nthe agent's vocabulary using the vocabulary of agents trained for multiple\ngames. To this extent, we explore the application of recently proposed policy\ndistillation method for video games to the text-based game setting. We also use\ntext-based games as a test-bed to analyze and hence understand policy\ndistillation approach in detail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07398v1",
            "date": "2018-05-18 18:58:38+00:00",
            "title": "Robust Handling of Polysemy via Sparse Representations",
            "authors": [
                "Abhijit Mahabal",
                "Dan Roth",
                "Sid Mittal"
            ],
            "abstract": "Words are polysemous and multi-faceted, with many shades of meanings. We\nsuggest that sparse distributed representations are more suitable than other,\ncommonly used, (dense) representations to express these multiple facets, and\npresent Category Builder, a working system that, as we show, makes use of\nsparse representations to support multi-faceted lexical representations. We\nargue that the set expansion task is well suited to study these meaning\ndistinctions since a word may belong to multiple sets with a different reason\nfor membership in each. We therefore exhibit the performance of Category\nBuilder on this task, while showing that our representation captures at the\nsame time analogy problems such as \"the Ganga of Egypt\" or \"the Voldemort of\nTolkien\". Category Builder is shown to be a more expressive lexical\nrepresentation and to outperform dense representations such as Word2Vec in some\nanalogy classes despite being shown only two of the three input terms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07513v1",
            "date": "2018-05-19 04:45:04+00:00",
            "title": "Diverse Few-Shot Text Classification with Multiple Metrics",
            "authors": [
                "Mo Yu",
                "Xiaoxiao Guo",
                "Jinfeng Yi",
                "Shiyu Chang",
                "Saloni Potdar",
                "Yu Cheng",
                "Gerald Tesauro",
                "Haoyu Wang",
                "Bowen Zhou"
            ],
            "abstract": "We study few-shot learning in natural language domains. Compared to many\nexisting works that apply either metric-based or optimization-based\nmeta-learning to image domain with low inter-task variance, we consider a more\nrealistic setting, where tasks are diverse. However, it imposes tremendous\ndifficulties to existing state-of-the-art metric-based algorithms since a\nsingle metric is insufficient to capture complex task variations in natural\nlanguage domain. To alleviate the problem, we propose an adaptive metric\nlearning approach that automatically determines the best weighted combination\nfrom a set of metrics obtained from meta-training tasks for a newly seen\nfew-shot task. Extensive quantitative evaluations on real-world sentiment\nanalysis and dialog intent classification datasets demonstrate that the\nproposed method performs favorably against state-of-the-art few shot learning\nalgorithms in terms of predictive accuracy. We make our code and data available\nfor further study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07685v1",
            "date": "2018-05-20 00:57:43+00:00",
            "title": "Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer",
            "authors": [
                "Cicero Nogueira dos Santos",
                "Igor Melnyk",
                "Inkit Padhi"
            ],
            "abstract": "We introduce a new approach to tackle the problem of offensive language in\nonline social media. Our approach uses unsupervised text style transfer to\ntranslate offensive sentences into non-offensive ones. We propose a new method\nfor training encoder-decoders using non-parallel data that combines a\ncollaborative classifier, attention and the cycle consistency loss.\nExperimental results on data from Twitter and Reddit show that our method\noutperforms a state-of-the-art text style transfer system in two out of three\nquantitative metrics and produces reliable non-offensive transferred sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07719v1",
            "date": "2018-05-20 07:19:10+00:00",
            "title": "Balancing Shared Autonomy with Human-Robot Communication",
            "authors": [
                "Rosario Scalise",
                "Yonatan Bisk",
                "Maxwell Forbes",
                "Daqing Yi",
                "Yejin Choi",
                "Siddhartha Srinivasa"
            ],
            "abstract": "Robotic agents that share autonomy with a human should leverage human domain\nknowledge and account for their preferences when completing a task. This extra\nknowledge can dramatically improve plan efficiency and user-satisfaction, but\nthese gains are lost if communicating with a robot is taxing and unnatural. In\nthis paper, we show how viewing humanrobot language through the lens of shared\nautonomy explains the efficiency versus cognitive load trade-offs humans make\nwhen deciding how cooperative and explicit to make their instructions.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1805.07851v1",
            "date": "2018-05-21 00:27:05+00:00",
            "title": "A Text Analysis of Federal Reserve meeting minutes",
            "authors": [
                "Harish Gandhi Ramachandran",
                "Dan DeRose Jr"
            ],
            "abstract": "Recent developments in monetary policy by the Federal Reserve has created a\nneed for an objective method of communication analysis.Using methods developed\nfor text analysis, we present a novel technique of analysis which creates a\nsemantic space defined by various policymakers public comments and places the\ncommittee consensus in the appropriate location. Its then possible to determine\nwhich member of the committee is most closely aligned with the committee\nconsensus over time and create a foundation for further actionable research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.07966v1",
            "date": "2018-05-21 10:10:16+00:00",
            "title": "Aff2Vec: Affect--Enriched Distributional Word Representations",
            "authors": [
                "Sopan Khosla",
                "Niyati Chhaya",
                "Kushal Chawla"
            ],
            "abstract": "Human communication includes information, opinions, and reactions. Reactions\nare often captured by the affective-messages in written as well as verbal\ncommunications. While there has been work in affect modeling and to some extent\naffective content generation, the area of affective word distributions in not\nwell studied. Synsets and lexica capture semantic relationships across words.\nThese models however lack in encoding affective or emotional word\ninterpretations. Our proposed model, Aff2Vec provides a method for enriched\nword embeddings that are representative of affective interpretations of words.\nAff2Vec outperforms the state--of--the--art in intrinsic word-similarity tasks.\nFurther, the use of Aff2Vec representations outperforms baseline embeddings in\ndownstream natural language understanding tasks including sentiment analysis,\npersonality detection, and frustration prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08159v2",
            "date": "2018-05-21 16:25:15+00:00",
            "title": "Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search",
            "authors": [
                "Jinfeng Rao",
                "Wei Yang",
                "Yuhao Zhang",
                "Ferhan Ture",
                "Jimmy Lin"
            ],
            "abstract": "Despite substantial interest in applications of neural networks to\ninformation retrieval, neural ranking models have only been applied to standard\nad hoc retrieval tasks over web pages and newswire documents. This paper\nproposes MP-HCNN (Multi-Perspective Hierarchical Convolutional Neural Network)\na novel neural ranking model specifically designed for ranking short social\nmedia posts. We identify document length, informal language, and heterogeneous\nrelevance signals as features that distinguish documents in our domain, and\npresent a model specifically designed with these characteristics in mind. Our\nmodel uses hierarchical convolutional layers to learn latent semantic\nsoft-match relevance signals at the character, word, and phrase levels. A\npooling-based similarity measurement layer integrates evidence from multiple\ntypes of matches between the query, the social media post, as well as URLs\ncontained in the post. Extensive experiments using Twitter data from the TREC\nMicroblog Tracks 2011--2014 show that our model significantly outperforms prior\nfeature-based as well and existing neural ranking models. To our best\nknowledge, this paper presents the first substantial work tackling search over\nsocial media posts using neural ranking models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.08174v1",
            "date": "2018-05-21 16:50:55+00:00",
            "title": "Reproducibility Report for \"Learning To Count Objects In Natural Images For Visual Question Answering\"",
            "authors": [
                "Shagun Sodhani",
                "Vardaan Pahuja"
            ],
            "abstract": "This is the reproducibility report for the paper \"Learning To Count Objects\nIn Natural Images For Visual QuestionAnswering\"",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.08182v1",
            "date": "2018-05-21 17:03:13+00:00",
            "title": "Party Matters: Enhancing Legislative Embeddings with Author Attributes for Vote Prediction",
            "authors": [
                "Anastassia Kornilova",
                "Daniel Argyle",
                "Vlad Eidelman"
            ],
            "abstract": "Predicting how Congressional legislators will vote is important for\nunderstanding their past and future behavior. However, previous work on\nroll-call prediction has been limited to single session settings, thus did not\nconsider generalization across sessions. In this paper, we show that metadata\nis crucial for modeling voting outcomes in new contexts, as changes between\nsessions lead to changes in the underlying data generation process. We show how\naugmenting bill text with the sponsors' ideologies in a neural network model\ncan achieve an average of a 4% boost in accuracy over the previous\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08389v1",
            "date": "2018-05-22 04:41:37+00:00",
            "title": "Joint Image Captioning and Question Answering",
            "authors": [
                "Jialin Wu",
                "Zeyuan Hu",
                "Raymond J. Mooney"
            ],
            "abstract": "Answering visual questions need acquire daily common knowledge and model the\nsemantic connection among different parts in images, which is too difficult for\nVQA systems to learn from images with the only supervision from answers.\nMeanwhile, image captioning systems with beam search strategy tend to generate\nsimilar captions and fail to diversely describe images. To address the\naforementioned issues, we present a system to have these two tasks compensate\nwith each other, which is capable of jointly producing image captions and\nanswering visual questions. In particular, we utilize question and image\nfeatures to generate question-related captions and use the generated captions\nas additional features to provide new knowledge to the VQA system. For image\ncaptioning, our system attains more informative results in term of the relative\nimprovements on VQA tasks as well as competitive results using automated\nmetrics. Applying our system to the VQA tasks, our results on VQA v2 dataset\nachieve 65.8% using generated captions and 69.1% using annotated captions in\nvalidation set and 68.4% in the test-standard set. Further, an ensemble of 10\nmodels results in 69.7% in the test-standard split.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08455v1",
            "date": "2018-05-22 08:34:10+00:00",
            "title": "Context-Aware Sequence-to-Sequence Models for Conversational Systems",
            "authors": [
                "Silje Christensen",
                "Simen Johnsrud",
                "Massimiliano Ruocco",
                "Heri Ramampiaro"
            ],
            "abstract": "This work proposes a novel approach based on sequence-to-sequence (seq2seq)\nmodels for context-aware conversational systems. Exist- ing seq2seq models have\nbeen shown to be good for generating natural responses in a data-driven\nconversational system. However, they still lack mechanisms to incorporate\nprevious conversation turns. We investigate RNN-based methods that efficiently\nintegrate previous turns as a context for generating responses. Overall, our\nexperimental results based on human judgment demonstrate the feasibility and\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08661v2",
            "date": "2018-05-22 15:26:15+00:00",
            "title": "COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval",
            "authors": [
                "Xirong Li",
                "Chaoxi Xu",
                "Xiaoxu Wang",
                "Weiyu Lan",
                "Zhengxiong Jia",
                "Gang Yang",
                "Jieping Xu"
            ],
            "abstract": "This paper contributes to cross-lingual image annotation and retrieval in\nterms of data and baseline methods. We propose COCO-CN, a novel dataset\nenriching MS-COCO with manually written Chinese sentences and tags. For more\neffective annotation acquisition, we develop a recommendation-assisted\ncollective annotation system, automatically providing an annotator with several\ntags and sentences deemed to be relevant with respect to the pictorial content.\nHaving 20,342 images annotated with 27,218 Chinese sentences and 70,993 tags,\nCOCO-CN is currently the largest Chinese-English dataset that provides a\nunified and challenging platform for cross-lingual image tagging, captioning\nand retrieval. We develop conceptually simple yet effective methods per task\nfor learning from cross-lingual resources. Extensive experiments on the three\ntasks justify the viability of the proposed dataset and methods. Data and code\nare publicly available at https://github.com/li-xirong/coco-cn",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08707v1",
            "date": "2018-05-18 19:33:04+00:00",
            "title": "A syllogistic system for propositions with intermediate quantifiers",
            "authors": [
                "Pasquale Iero",
                "Allan Third",
                "Paul Piwek"
            ],
            "abstract": "This paper describes a formalism that subsumes Peterson's intermediate\nquantifier syllogistic system, and extends the ideas by van Eijck on\nAristotle's logic. Syllogisms are expressed in a concise form making use of and\nextending the Monotonicity Calculus. Contradictory and contrary relationships\nare added so that deduction can derive propositions expressing a form of\nnegation.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1805.08949v1",
            "date": "2018-05-23 03:39:04+00:00",
            "title": "Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow",
            "authors": [
                "Pengcheng Yin",
                "Bowen Deng",
                "Edgar Chen",
                "Bogdan Vasilescu",
                "Graham Neubig"
            ],
            "abstract": "For tasks like code synthesis from natural language, code retrieval, and code\nsummarization, data-driven models have shown great promise. However, creating\nthese models require parallel data between natural language (NL) and code with\nfine-grained alignments. Stack Overflow (SO) is a promising source to create\nsuch a data set: the questions are diverse and most of them have corresponding\nanswers with high-quality code snippets. However, existing heuristic methods\n(e.g., pairing the title of a post with the code in the accepted answer) are\nlimited both in their coverage and the correctness of the NL-code pairs\nobtained. In this paper, we propose a novel method to mine high-quality aligned\ndata from SO using two sets of features: hand-crafted features considering the\nstructure of the extracted snippets, and correspondence features obtained by\ntraining a probabilistic model to capture the correlation between NL and code\nusing neural networks. These features are fed into a classifier that determines\nthe quality of mined NL-code pairs. Experiments using Python and Java as test\nbeds show that the proposed method greatly expands coverage and accuracy over\nexisting mining methods, even when using only a small number of labeled\nexamples. Further, we find that reasonable results are achieved even when\ntraining the classifier on one language and testing on another, showing promise\nfor scaling NL-code mining to a wide variety of programming languages beyond\nthose for which we are able to annotate data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09145v1",
            "date": "2018-05-23 13:34:51+00:00",
            "title": "RDF2Vec-based Classification of Ontology Alignment Changes",
            "authors": [
                "Matthias Jurisch",
                "Bodo Igler"
            ],
            "abstract": "When ontologies cover overlapping topics, the overlap can be represented\nusing ontology alignments. These alignments need to be continuously adapted to\nchanging ontologies. Especially for large ontologies this is a costly task\noften consisting of manual work. Finding changes that do not lead to an\nadaption of the alignment can potentially make this process significantly\neasier. This work presents an approach to finding these changes based on RDF\nembeddings and common classification techniques. To examine the feasibility of\nthis approach, an evaluation on a real-world dataset is presented. In this\nevaluation, the best classifiers reached a precision of 0.8.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09559v2",
            "date": "2018-05-24 09:04:44+00:00",
            "title": "WSD algorithm based on a new method of vector-word contexts proximity calculation via epsilon-filtration",
            "authors": [
                "Alexander Kirillov",
                "Natalia Krizhanovsky",
                "Andrew Krizhanovsky"
            ],
            "abstract": "The problem of word sense disambiguation (WSD) is considered in the article.\nGiven a set of synonyms (synsets) and sentences with these synonyms. It is\nnecessary to select the meaning of the word in the sentence automatically. 1285\nsentences were tagged by experts, namely, one of the dictionary meanings was\nselected by experts for target words. To solve the WSD-problem, an algorithm\nbased on a new method of vector-word contexts proximity calculation is\nproposed. In order to achieve higher accuracy, a preliminary epsilon-filtering\nof words is performed, both in the sentence and in the set of synonyms. An\nextensive program of experiments was carried out. Four algorithms are\nimplemented, including a new algorithm. Experiments have shown that in a number\nof cases the new algorithm shows better results. The developed software and the\ntagged corpus have an open license and are available online. Wiktionary and\nWikisource are used. A brief description of this work can be viewed in slides\n(https://goo.gl/9ak6Gt). Video lecture in Russian on this research is available\nonline (https://youtu.be/-DLmRkepf58).",
            "journal ref": "Transactions of Karelian Research Centre RAS. No. 7. 2018. P.\n  149-163",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.09644v1",
            "date": "2018-05-16 21:06:59+00:00",
            "title": "DINFRA: A One Stop Shop for Computing Multilingual Semantic Relatedness",
            "authors": [
                "Siamak Barzegar",
                "Juliano Efson Sales",
                "Andre Freitas",
                "Siegfried Handschuh",
                "Brian Davis"
            ],
            "abstract": "This demonstration presents an infrastructure for computing multilingual\nsemantic relatedness and correlation for twelve natural languages by using\nthree distributional semantic models (DSMs). Our demonsrator - DInfra\n(Distributional Infrastructure) provides researchers and developers with a\nhighly useful platform for processing large-scale corpora and conducting\nexperiments with distributional semantics. We integrate several multilingual\nDSMs in our webservice so the end user can obtain a result without worrying\nabout the complexities involved in building DSMs. Our webservice allows the\nusers to have easy access to a wide range of comparisons of DSMs with different\nparameters. In addition, users can configure and access DSM parameters using an\neasy to use API.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.09655v3",
            "date": "2018-05-19 19:23:38+00:00",
            "title": "Global-Locally Self-Attentive Dialogue State Tracker",
            "authors": [
                "Victor Zhong",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Dialogue state tracking, which estimates user goals and requests given the\ndialogue context, is an essential part of task-oriented dialogue systems. In\nthis paper, we propose the Global-Locally Self-Attentive Dialogue State Tracker\n(GLAD), which learns representations of the user utterance and previous system\nactions with global-local modules. Our model uses global modules to share\nparameters between estimators for different types (called slots) of dialogue\nstates, and uses local modules to learn slot-specific features. We show that\nthis significantly improves tracking of rare states and achieves\nstate-of-the-art performance on the WoZ and DSTC2 state tracking tasks. GLAD\nobtains 88.1% joint goal accuracy and 97.1% request accuracy on WoZ,\noutperforming prior work by 3.7% and 5.5%. On DSTC2, our model obtains 74.5%\njoint goal accuracy and 97.5% request accuracy, outperforming prior work by\n1.1% and 1.0%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09822v1",
            "date": "2018-05-24 13:09:59+00:00",
            "title": "Filtering and Mining Parallel Data in a Joint Multilingual Space",
            "authors": [
                "Holger Schwenk"
            ],
            "abstract": "We learn a joint multilingual sentence embedding and use the distance between\nsentences in different languages to filter noisy parallel data and to mine for\nparallel data in large news collections. We are able to improve a competitive\nbaseline on the WMT'14 English to German task by 0.3 BLEU by filtering out 25%\nof the training data. The same approach is used to mine additional bitexts for\nthe WMT'14 system and to obtain competitive results on the BUCC shared task to\nidentify parallel sentences in comparable corpora. The approach is generic, it\ncan be applied to many language pairs and it is independent of the architecture\nof the machine translation system.",
            "journal ref": "ACL, July 2018, Melbourne",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09959v2",
            "date": "2018-05-25 03:13:49+00:00",
            "title": "A Sentiment Analysis of Breast Cancer Treatment Experiences and Healthcare Perceptions Across Twitter",
            "authors": [
                "Eric M. Clark",
                "Ted James",
                "Chris A. Jones",
                "Amulya Alapati",
                "Promise Ukandu",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Background: Social media has the capacity to afford the healthcare industry\nwith valuable feedback from patients who reveal and express their medical\ndecision-making process, as well as self-reported quality of life indicators\nboth during and post treatment. In prior work, [Crannell et. al.], we have\nstudied an active cancer patient population on Twitter and compiled a set of\ntweets describing their experience with this disease. We refer to these online\npublic testimonies as \"Invisible Patient Reported Outcomes\" (iPROs), because\nthey carry relevant indicators, yet are difficult to capture by conventional\nmeans of self-report. Methods: Our present study aims to identify tweets\nrelated to the patient experience as an additional informative tool for\nmonitoring public health. Using Twitter's public streaming API, we compiled\nover 5.3 million \"breast cancer\" related tweets spanning September 2016 until\nmid December 2017. We combined supervised machine learning methods with natural\nlanguage processing to sift tweets relevant to breast cancer patient\nexperiences. We analyzed a sample of 845 breast cancer patient and survivor\naccounts, responsible for over 48,000 posts. We investigated tweet content with\na hedonometric sentiment analysis to quantitatively extract emotionally charged\ntopics. Results: We found that positive experiences were shared regarding\npatient treatment, raising support, and spreading awareness. Further\ndiscussions related to healthcare were prevalent and largely negative focusing\non fear of political legislation that could result in loss of coverage.\nConclusions: Social media can provide a positive outlet for patients to discuss\ntheir needs and concerns regarding their healthcare coverage and treatment\nneeds. Capturing iPROs from online communication can help inform healthcare\nprofessionals and lead to more connected and personalized treatment regimens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10190v3",
            "date": "2018-05-25 15:04:17+00:00",
            "title": "Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces",
            "authors": [
                "Alice Coucke",
                "Alaa Saade",
                "Adrien Ball",
                "Th\u00e9odore Bluche",
                "Alexandre Caulier",
                "David Leroy",
                "Cl\u00e9ment Doumouro",
                "Thibault Gisselbrecht",
                "Francesco Caltagirone",
                "Thibaut Lavril",
                "Ma\u00ebl Primet",
                "Joseph Dureau"
            ],
            "abstract": "This paper presents the machine learning architecture of the Snips Voice\nPlatform, a software solution to perform Spoken Language Understanding on\nmicroprocessors typical of IoT devices. The embedded inference is fast and\naccurate while enforcing privacy by design, as no personal user data is ever\ncollected. Focusing on Automatic Speech Recognition and Natural Language\nUnderstanding, we detail our approach to training high-performance Machine\nLearning models that are small enough to run in real-time on small devices.\nAdditionally, we describe a data generation procedure that provides sufficient,\nhigh-quality training data without compromising user privacy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10338v1",
            "date": "2018-05-25 19:27:43+00:00",
            "title": "Zero-Shot Dual Machine Translation",
            "authors": [
                "Lierni Sestorain",
                "Massimiliano Ciaramita",
                "Christian Buck",
                "Thomas Hofmann"
            ],
            "abstract": "Neural Machine Translation (NMT) systems rely on large amounts of parallel\ndata. This is a major challenge for low-resource languages. Building on recent\nwork on unsupervised and semi-supervised methods, we present an approach that\ncombines zero-shot and dual learning. The latter relies on reinforcement\nlearning, to exploit the duality of the machine translation task, and requires\nonly monolingual data for the target language pair. Experiments show that a\nzero-shot dual system, trained on English-French and English-Spanish,\noutperforms by large margins a standard NMT system in zero-shot translation\nperformance on Spanish-French (both directions). The zero-shot dual method\napproaches the performance, within 2.2 BLEU points, of a comparable supervised\nsetting. Our method can obtain improvements also on the setting where a small\namount of parallel data for the zero-shot language pair is available. Adding\nRussian, to extend our experiments to jointly modeling 6 zero-shot translation\ndirections, all directions improve between 4 and 15 BLEU points, again,\nreaching performance near that of the supervised setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10528v2",
            "date": "2018-05-26 19:26:35+00:00",
            "title": "Dependent Gated Reading for Cloze-Style Question Answering",
            "authors": [
                "Reza Ghaeini",
                "Xiaoli Z. Fern",
                "Hamed Shahbazi",
                "Prasad Tadepalli"
            ],
            "abstract": "We present a novel deep learning architecture to address the cloze-style\nquestion answering task. Existing approaches employ reading mechanisms that do\nnot fully exploit the interdependency between the document and the query. In\nthis paper, we propose a novel \\emph{dependent gated reading} bidirectional GRU\nnetwork (DGR) to efficiently model the relationship between the document and\nthe query during encoding and decision making. Our evaluation shows that DGR\nobtains highly competitive performance on well-known machine comprehension\nbenchmarks such as the Children's Book Test (CBT-NE and CBT-CN) and Who DiD\nWhat (WDW, Strict and Relaxed). Finally, we extensively analyze and validate\nour model by ablation and attention studies.",
            "journal ref": "COLING 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10627v3",
            "date": "2018-05-27 13:51:48+00:00",
            "title": "Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning",
            "authors": [
                "Julia Kreutzer",
                "Joshua Uyheng",
                "Stefan Riezler"
            ],
            "abstract": "We present a study on reinforcement learning (RL) from human bandit feedback\nfor sequence-to-sequence learning, exemplified by the task of bandit neural\nmachine translation (NMT). We investigate the reliability of human bandit\nfeedback, and analyze the influence of reliability on the learnability of a\nreward estimator, and the effect of the quality of reward estimates on the\noverall RL task. Our analysis of cardinal (5-point ratings) and ordinal\n(pairwise preferences) feedback shows that their intra- and inter-annotator\n$\\alpha$-agreement is comparable. Best reliability is obtained for standardized\ncardinal feedback, and cardinal feedback is also easiest to learn and\ngeneralize from. Finally, improvements of over 1 BLEU can be obtained by\nintegrating a regression-based reward estimator trained on cardinal feedback\nfor 800 translations into RL for NMT. This shows that RL is possible even from\nsmall amounts of fairly reliable human feedback, pointing to a great potential\nfor applications at larger scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10856v2",
            "date": "2018-05-28 10:36:41+00:00",
            "title": "r-Instance Learning for Missing People Tweets Identification",
            "authors": [
                "Yang Yang",
                "Haoyan Liu",
                "Xia Hu",
                "Jiawei Zhang",
                "Xiaoming Zhang",
                "Zhoujun Li",
                "Philip S. Yu"
            ],
            "abstract": "The number of missing people (i.e., people who get lost) greatly increases in\nrecent years. It is a serious worldwide problem, and finding the missing people\nconsumes a large amount of social resources. In tracking and finding these\nmissing people, timely data gathering and analysis actually play an important\nrole. With the development of social media, information about missing people\ncan get propagated through the web very quickly, which provides a promising way\nto solve the problem. The information in online social media is usually of\nheterogeneous categories, involving both complex social interactions and\ntextual data of diverse structures. Effective fusion of these different types\nof information for addressing the missing people identification problem can be\na great challenge. Motivated by the multi-instance learning problem and\nexisting social science theory of \"homophily\", in this paper, we propose a\nnovel r-instance (RI) learning model.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1805.10956v1",
            "date": "2018-05-28 14:51:27+00:00",
            "title": "Temporal Event Knowledge Acquisition via Identifying Narratives",
            "authors": [
                "Wenlin Yao",
                "Ruihong Huang"
            ],
            "abstract": "Inspired by the double temporality characteristic of narrative texts, we\npropose a novel approach for acquiring rich temporal \"before/after\" event\nknowledge across sentences in narrative stories. The double temporality states\nthat a narrative story often describes a sequence of events following the\nchronological order and therefore, the temporal order of events matches with\ntheir textual order. We explored narratology principles and built a weakly\nsupervised approach that identifies 287k narrative paragraphs from three large\ntext corpora. We then extracted rich temporal event knowledge from these\nnarrative paragraphs. Such event knowledge is shown useful to improve temporal\nrelation classification and outperform several recent neural network models on\nthe narrative cloze task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10973v3",
            "date": "2018-05-28 15:30:21+00:00",
            "title": "GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation",
            "authors": [
                "Taehyeong Kim",
                "Min-Oh Heo",
                "Seonil Son",
                "Kyoung-Wha Park",
                "Byoung-Tak Zhang"
            ],
            "abstract": "The task of multi-image cued story generation, such as visual storytelling\ndataset (VIST) challenge, is to compose multiple coherent sentences from a\ngiven sequence of images. The main difficulty is how to generate image-specific\nsentences within the context of overall images. Here we propose a deep learning\nnetwork model, GLAC Net, that generates visual stories by combining\nglobal-local (glocal) attention and context cascading mechanisms. The model\nincorporates two levels of attention, i.e., overall encoding level and image\nfeature level, to construct image-dependent sentences. While standard attention\nconfiguration needs a large number of parameters, the GLAC Net implements them\nin a very simple way via hard connections from the outputs of encoders or image\nfeatures onto the sentence generators. The coherency of the generated story is\nfurther improved by conveying (cascading) the information of the previous\nsentence to the next sentence serially. We evaluate the performance of the GLAC\nNet on the visual storytelling dataset (VIST) and achieve very competitive\nresults compared to the state-of-the-art techniques. Our code and pre-trained\nmodels are available here.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11025v1",
            "date": "2018-05-25 13:43:56+00:00",
            "title": "Think Visually: Question Answering through Virtual Imagery",
            "authors": [
                "Ankit Goyal",
                "Jian Wang",
                "Jia Deng"
            ],
            "abstract": "In this paper, we study the problem of geometric reasoning in the context of\nquestion-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new\ndeep network architecture designed for answering questions that admit latent\nvisual representations. DSMN learns to generate and reason over such\nrepresentations. Further, we propose two synthetic benchmarks, FloorPlanQA and\nShapeIntersection, to evaluate the geometric reasoning capability of QA\nsystems. Experimental results validate the effectiveness of our proposed DSMN\nfor visual thinking tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11166v1",
            "date": "2018-05-28 20:31:18+00:00",
            "title": "A visual approach for age and gender identification on Twitter",
            "authors": [
                "Miguel A. Alvarez-Carmona",
                "Luis Pellegrin",
                "Manuel Montes-y-G\u00f3mez",
                "Fernando S\u00e1nchez-Vega",
                "Hugo Jair Escalante",
                "A. Pastor L\u00f3pez-Monroy",
                "Luis Villase\u00f1or-Pineda",
                "Esa\u00fa Villatoro-Tello"
            ],
            "abstract": "The goal of Author Profiling (AP) is to identify demographic aspects (e.g.,\nage, gender) from a given set of authors by analyzing their written texts.\nRecently, the AP task has gained interest in many problems related to computer\nforensics, psychology, marketing, but specially in those related with social\nmedia exploitation. As known, social media data is shared through a wide range\nof modalities (e.g., text, images and audio), representing valuable information\nto be exploited for extracting valuable insights from users. Nevertheless, most\nof the current work in AP using social media data has been devoted to analyze\ntextual information only, and there are very few works that have started\nexploring the gender identification using visual information. Contrastingly,\nthis paper focuses in exploiting the visual modality to perform both age and\ngender identification in social media, specifically in Twitter. Our goal is to\nevaluate the pertinence of using visual information in solving the AP task.\nAccordingly, we have extended the Twitter corpus from PAN 2014, incorporating\nposted images from all the users, making a distinction between tweeted and\nretweeted images. Performed experiments provide interesting evidence on the\nusefulness of visual information in comparison with traditional textual\nrepresentations for the AP task.",
            "journal ref": "Miguel A. Alvarez-Carmona, Luis Pellegrin et al. A visual approach\n  for age and gender identification on Twitter. Journal of Intelligent and\n  Fuzzy Systems, vol. 34, no. 5, pp. 3133-3145, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11234v1",
            "date": "2018-05-29 03:39:35+00:00",
            "title": "Table-to-Text: Describing Table Region with Natural Language",
            "authors": [
                "Junwei Bao",
                "Duyu Tang",
                "Nan Duan",
                "Zhao Yan",
                "Yuanhua Lv",
                "Ming Zhou",
                "Tiejun Zhao"
            ],
            "abstract": "In this paper, we present a generative model to generate a natural language\nsentence describing a table region, e.g., a row. The model maps a row from a\ntable to a continuous vector and then generates a natural language sentence by\nleveraging the semantics of a table. To deal with rare words appearing in a\ntable, we develop a flexible copying mechanism that selectively replicates\ncontents from the table in the output sequence. Extensive experiments\ndemonstrate the accuracy of the model and the power of the copying mechanism.\nOn two synthetic datasets, WIKIBIO and SIMPLEQUESTIONS, our model improves the\ncurrent state-of-the-art BLEU-4 score from 34.70 to 40.26 and from 33.32 to\n39.12, respectively. Furthermore, we introduce an open-domain dataset\nWIKITABLETEXT including 13,318 explanatory sentences for 4,962 tables. Our\nmodel achieves a BLEU-4 score of 38.23, which outperforms template based and\nlanguage model based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11404v1",
            "date": "2018-05-11 10:24:11+00:00",
            "title": "iLCM - A Virtual Research Infrastructure for Large-Scale Qualitative Data",
            "authors": [
                "Andreas Niekler",
                "Arnim Bleier",
                "Christian Kahmann",
                "Lisa Posch",
                "Gregor Wiedemann",
                "Kenan Erdogan",
                "Gerhard Heyer",
                "Markus Strohmaier"
            ],
            "abstract": "The iLCM project pursues the development of an integrated research\nenvironment for the analysis of structured and unstructured data in a \"Software\nas a Service\" architecture (SaaS). The research environment addresses\nrequirements for the quantitative evaluation of large amounts of qualitative\ndata with text mining methods as well as requirements for the reproducibility\nof data-driven research designs in the social sciences. For this, the iLCM\nresearch environment comprises two central components. First, the Leipzig\nCorpus Miner (LCM), a decentralized SaaS application for the analysis of large\namounts of news texts developed in a previous Digital Humanities project.\nSecond, the text mining tools implemented in the LCM are extended by an \"Open\nResearch Computing\" (ORC) environment for executable script documents,\nso-called \"notebooks\". This novel integration allows to combine generic,\nhigh-performance methods to process large amounts of unstructured text data and\nwith individual program scripts to address specific research requirements in\ncomputational social science and digital humanities.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.11546v2",
            "date": "2018-05-29 15:53:30+00:00",
            "title": "Like a Baby: Visually Situated Neural Language Acquisition",
            "authors": [
                "Alexander G. Ororbia",
                "Ankur Mali",
                "Matthew A. Kelly",
                "David Reitter"
            ],
            "abstract": "We examine the benefits of visual context in training neural language models\nto perform next-word prediction. A multi-modal neural architecture is\nintroduced that outperform its equivalent trained on language alone with a 2\\%\ndecrease in perplexity, even when no visual context is available at test.\nFine-tuning the embeddings of a pre-trained state-of-the-art bidirectional\nlanguage model (BERT) in the language modeling framework yields a 3.5\\%\nimprovement. The advantage for training with visual context when testing\nwithout is robust across different languages (English, German and Spanish) and\ndifferent models (GRU, LSTM, $\\Delta$-RNN, as well as those that use BERT\nembeddings). Thus, language models perform better when they learn like a baby,\ni.e, in a multi-modal environment. This finding is compatible with the theory\nof situated cognition: language is inseparable from its physical context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11651v2",
            "date": "2018-05-26 06:46:55+00:00",
            "title": "Splitting source code identifiers using Bidirectional LSTM Recurrent Neural Network",
            "authors": [
                "Vadim Markovtsev",
                "Waren Long",
                "Egor Bulychev",
                "Romain Keramitas",
                "Konstantin Slavnov",
                "Gabor Markowski"
            ],
            "abstract": "Programmers make rich use of natural language in the source code they write\nthrough identifiers and comments. Source code identifiers are selected from a\npool of tokens which are strongly related to the meaning, naming conventions,\nand context. These tokens are often combined to produce more precise and\nobvious designations. Such multi-part identifiers count for 97% of all naming\ntokens in the Public Git Archive - the largest dataset of Git repositories to\ndate. We introduce a bidirectional LSTM recurrent neural network to detect\nsubtokens in source code identifiers. We trained that network on 41.7 million\ndistinct splittable identifiers collected from 182,014 open source projects in\nPublic Git Archive, and show that it outperforms several other machine learning\nmodels. The proposed network can be used to improve the upstream models which\nare based on source code identifiers, as well as improving developer experience\nallowing writing code without switching the keyboard case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11850v1",
            "date": "2018-05-30 08:20:55+00:00",
            "title": "Neural Joking Machine : Humorous image captioning",
            "authors": [
                "Kota Yoshida",
                "Munetaka Minoguchi",
                "Kenichiro Wani",
                "Akio Nakamura",
                "Hirokatsu Kataoka"
            ],
            "abstract": "What is an effective expression that draws laughter from human beings? In the\npresent paper, in order to consider this question from an academic standpoint,\nwe generate an image caption that draws a \"laugh\" by a computer. A system that\noutputs funny captions based on the image caption proposed in the computer\nvision field is constructed. Moreover, we also propose the Funny Score, which\nflexibly gives weights according to an evaluation database. The Funny Score\nmore effectively brings out \"laughter\" to optimize a model. In addition, we\nbuild a self-collected BoketeDB, which contains a theme (image) and funny\ncaption (text) posted on \"Bokete\", which is an image Ogiri website. In an\nexperiment, we use BoketeDB to verify the effectiveness of the proposed method\nby comparing the results obtained using the proposed method and those obtained\nusing MS COCO Pre-trained CNN+LSTM, which is the baseline and idiot created by\nhumans. We refer to the proposed method, which uses the BoketeDB pre-trained\nmodel, as the Neural Joking Machine (NJM).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.11867v1",
            "date": "2018-05-30 08:59:44+00:00",
            "title": "Using Inter-Sentence Diverse Beam Search to Reduce Redundancy in Visual Storytelling",
            "authors": [
                "Chao-Chun Hsu",
                "Szu-Min Chen",
                "Ming-Hsun Hsieh",
                "Lun-Wei Ku"
            ],
            "abstract": "Visual storytelling includes two important parts: coherence between the story\nand images as well as the story structure. For image to text neural network\nmodels, similar images in the sequence would provide close information for\nstory generator to obtain almost identical sentence. However, repeatedly\nnarrating same objects or events will undermine a good story structure. In this\npaper, we proposed an inter-sentence diverse beam search to generate a more\nexpressive story. Comparing to some recent models of visual storytelling task,\nwhich generate story without considering the generated sentence of the previous\npicture, our proposed method can avoid generating identical sentence even given\na sequence of similar pictures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12115v1",
            "date": "2018-05-30 17:56:32+00:00",
            "title": "Amnestic Forgery: an Ontology of Conceptual Metaphors",
            "authors": [
                "Aldo Gangemi",
                "Mehwish Alam",
                "Valentina Presutti"
            ],
            "abstract": "This paper presents Amnestic Forgery, an ontology for metaphor semantics,\nbased on MetaNet, which is inspired by the theory of Conceptual Metaphor.\nAmnestic Forgery reuses and extends the Framester schema, as an ideal ontology\ndesign framework to deal with both semiotic and referential aspects of frames,\nroles, mappings, and eventually blending. The description of the resource is\nsupplied by a discussion of its applications, with examples taken from metaphor\ngeneration, and the referential problems of metaphoric mappings. Both schema\nand data are available from the Framester SPARQL endpoint.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12216v1",
            "date": "2018-05-30 20:28:36+00:00",
            "title": "A Web-scale system for scientific knowledge exploration",
            "authors": [
                "Zhihong Shen",
                "Hao Ma",
                "Kuansan Wang"
            ],
            "abstract": "To enable efficient exploration of Web-scale scientific knowledge, it is\nnecessary to organize scientific publications into a hierarchical concept\nstructure. In this work, we present a large-scale system to (1) identify\nhundreds of thousands of scientific concepts, (2) tag these identified concepts\nto hundreds of millions of scientific publications by leveraging both text and\ngraph structure, and (3) build a six-level concept hierarchy with a\nsubsumption-based model. The system builds the most comprehensive cross-domain\nscientific concept ontology published to date, with more than 200 thousand\nconcepts and over one million relationships.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00354v1",
            "date": "2018-06-01 14:02:39+00:00",
            "title": "Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers",
            "authors": [
                "Sandro Pezzelle",
                "Shane Steinert-Threlkeld",
                "Raffaela Bernardi",
                "Jakub Szymanik"
            ],
            "abstract": "We study the role of linguistic context in predicting quantifiers (`few',\n`all'). We collect crowdsourced data from human participants and test various\nmodels in a local (single-sentence) and a global context (multi-sentence)\ncondition. Models significantly out-perform humans in the former setting and\nare only slightly better in the latter. While human performance improves with\nmore linguistic context (especially on proportional quantifiers), model\nperformance suffers. Models are very effective in exploiting lexical and\nmorpho-syntactic patterns; humans are better at genuinely understanding the\nmeaning of the (global) context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00525v1",
            "date": "2018-06-01 19:51:58+00:00",
            "title": "Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7",
            "authors": [
                "Huda Alamri",
                "Vincent Cartillier",
                "Raphael Gontijo Lopes",
                "Abhishek Das",
                "Jue Wang",
                "Irfan Essa",
                "Dhruv Batra",
                "Devi Parikh",
                "Anoop Cherian",
                "Tim K. Marks",
                "Chiori Hori"
            ],
            "abstract": "Scene-aware dialog systems will be able to have conversations with users\nabout the objects and events around them. Progress on such systems can be made\nby integrating state-of-the-art technologies from multiple research areas\nincluding end-to-end dialog systems visual dialog, and video description. We\nintroduce the Audio Visual Scene Aware Dialog (AVSD) challenge and dataset. In\nthis challenge, which is one track of the 7th Dialog System Technology\nChallenges (DSTC7) workshop1, the task is to build a system that generates\nresponses in a dialog about an input video",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00696v3",
            "date": "2018-06-02 20:00:44+00:00",
            "title": "NLP-assisted software testing: A systematic mapping of the literature",
            "authors": [
                "Vahid Garousi",
                "Sara Bauer",
                "Michael Felderer"
            ],
            "abstract": "Context: To reduce manual effort of extracting test cases from\nnatural-language requirements, many approaches based on Natural Language\nProcessing (NLP) have been proposed in the literature. Given the large amount\nof approaches in this area, and since many practitioners are eager to utilize\nsuch techniques, it is important to synthesize and provide an overview of the\nstate-of-the-art in this area. Objective: Our objective is to summarize the\nstate-of-the-art in NLP-assisted software testing which could benefit\npractitioners to potentially utilize those NLP-based techniques. Moreover, this\ncan benefit researchers in providing an overview of the research landscape.\nMethod: To address the above need, we conducted a survey in the form of a\nsystematic literature mapping (classification). After compiling an initial pool\nof 95 papers, we conducted a systematic voting, and our final pool included 67\ntechnical papers. Results: This review paper provides an overview of the\ncontribution types presented in the papers, types of NLP approaches used to\nassist software testing, types of required input requirements, and a review of\ntool support in this area. Some key results we have detected are: (1) only four\nof the 38 tools (11%) presented in the papers are available for download; (2) a\nlarger ratio of the papers (30 of 67) provided a shallow exposure to the NLP\naspects (almost no details). Conclusion: This paper would benefit both\npractitioners and researchers by serving as an \"index\" to the body of knowledge\nin this area. The results could help practitioners utilizing the existing\nNLP-based techniques; this in turn reduces the cost of test-case design and\ndecreases the amount of human resources spent on test activities. After sharing\nthis review with some of our industrial collaborators, initial insights show\nthat this review can indeed be useful and beneficial to practitioners.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1806.00749v1",
            "date": "2018-06-03 08:09:58+00:00",
            "title": "TI-CNN: Convolutional Neural Networks for Fake News Detection",
            "authors": [
                "Yang Yang",
                "Lei Zheng",
                "Jiawei Zhang",
                "Qingcai Cui",
                "Zhoujun Li",
                "Philip S. Yu"
            ],
            "abstract": "With the development of social networks, fake news for various commercial and\npolitical purposes has been appearing in large numbers and gotten widespread in\nthe online world. With deceptive words, people can get infected by the fake\nnews very easily and will share them without any fact-checking. For instance,\nduring the 2016 US president election, various kinds of fake news about the\ncandidates widely spread through both official news media and the online social\nnetworks. These fake news is usually released to either smear the opponents or\nsupport the candidate on their side. The erroneous information in the fake news\nis usually written to motivate the voters' irrational emotion and enthusiasm.\nSuch kinds of fake news sometimes can bring about devastating effects, and an\nimportant goal in improving the credibility of online social networks is to\nidentify the fake news timely. In this paper, we propose to study the fake news\ndetection problem. Automatic fake news identification is extremely hard, since\npure model based fact-checking for news is still an open problem, and few\nexisting models can be applied to solve the problem. With a thorough\ninvestigation of a fake news data, lots of useful explicit features are\nidentified from both the text words and images used in the fake news. Besides\nthe explicit features, there also exist some hidden patterns in the words and\nimages used in fake news, which can be captured with a set of latent features\nextracted via the multiple convolutional layers in our model. A model named as\nTI-CNN (Text and Image information based Convolutinal Neural Network) is\nproposed in this paper. By projecting the explicit and latent features into a\nunified feature space, TI-CNN is trained with both the text and image\ninformation simultaneously. Extensive experiments carried on the real-world\nfake news datasets have demonstrate the effectiveness of TI-CNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00754v1",
            "date": "2018-06-03 08:52:23+00:00",
            "title": "Psychological State in Text: A Limitation of Sentiment Analysis",
            "authors": [
                "Hwiyeol Jo",
                "Jeong Ryu"
            ],
            "abstract": "Starting with the idea that sentiment analysis models should be able to\npredict not only positive or negative but also other psychological states of a\nperson, we implement a sentiment analysis model to investigate the relationship\nbetween the model and emotional state. We first examine psychological\nmeasurements of 64 participants and ask them to write a book report about a\nstory. After that, we train our sentiment analysis model using crawled movie\nreview data. We finally evaluate participants' writings, using the pretrained\nmodel as a concept of transfer learning. The result shows that sentiment\nanalysis model performs good at predicting a score, but the score does not have\nany correlation with human's self-checked sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00793v2",
            "date": "2018-06-03 13:22:10+00:00",
            "title": "Transfer Topic Labeling with Domain-Specific Knowledge Base: An Analysis of UK House of Commons Speeches 1935-2014",
            "authors": [
                "Alexander Herzog",
                "Peter John",
                "Slava Jankin Mikhaylov"
            ],
            "abstract": "Topic models are widely used in natural language processing, allowing\nresearchers to estimate the underlying themes in a collection of documents.\nMost topic models use unsupervised methods and hence require the additional\nstep of attaching meaningful labels to estimated topics. This process of manual\nlabeling is not scalable and suffers from human bias. We present a\nsemi-automatic transfer topic labeling method that seeks to remedy these\nproblems. Domain-specific codebooks form the knowledge-base for automated topic\nlabeling. We demonstrate our approach with a dynamic topic model analysis of\nthe complete corpus of UK House of Commons speeches 1935-2014, using the coding\ninstructions of the Comparative Agendas Project to label topics. We show that\nour method works well for a majority of the topics we estimate; but we also\nfind that institution-specific topics, in particular on subnational governance,\nrequire manual input. We validate our results using human expert coding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00807v5",
            "date": "2018-06-03 15:00:05+00:00",
            "title": "Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator",
            "authors": [
                "Badri N. Patro",
                "Vinod K. Kurmi",
                "Sandeep Kumar",
                "Vinay P. Namboodiri"
            ],
            "abstract": "In this paper, we propose a method for obtaining sentence-level embeddings.\nWhile the problem of securing word-level embeddings is very well studied, we\npropose a novel method for obtaining sentence-level embeddings. This is\nobtained by a simple method in the context of solving the paraphrase generation\ntask. If we use a sequential encoder-decoder model for generating paraphrase,\nwe would like the generated paraphrase to be semantically close to the original\nsentence. One way to ensure this is by adding constraints for true paraphrase\nembeddings to be close and unrelated paraphrase candidate sentence embeddings\nto be far. This is ensured by using a sequential pair-wise discriminator that\nshares weights with the encoder that is trained with a suitable loss function.\nOur loss function penalizes paraphrase sentence embedding distances from being\ntoo large. This loss is used in combination with a sequential encoder-decoder\nnetwork. We also validated our method by evaluating the obtained embeddings for\na sentiment analysis task. The proposed method results in semantic embeddings\nand outperforms the state-of-the-art on the paraphrase generation and sentiment\nanalysis task on standard datasets. These results are also shown to be\nstatistically significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01330v4",
            "date": "2018-06-04 19:03:52+00:00",
            "title": "Closed Form Word Embedding Alignment",
            "authors": [
                "Sunipa Dev",
                "Safia Hassan",
                "Jeff M. Phillips"
            ],
            "abstract": "We develop a family of techniques to align word embeddings which are derived\nfrom different source datasets or created using different mechanisms (e.g.,\nGloVe or word2vec). Our methods are simple and have a closed form to optimally\nrotate, translate, and scale to minimize root mean squared errors or maximize\nthe average cosine similarity between two embeddings of the same vocabulary\ninto the same dimensional space. Our methods extend approaches known as\nAbsolute Orientation, which are popular for aligning objects in\nthree-dimensions, and generalize an approach by Smith etal (ICLR 2017). We\nprove new results for optimal scaling and for maximizing cosine similarity.\nThen we demonstrate how to evaluate the similarity of embeddings from different\nsources or mechanisms, and that certain properties like synonyms and analogies\nare preserved across the embeddings and can be enhanced by simply aligning and\naveraging ensembles of embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01742v2",
            "date": "2018-06-05 15:18:47+00:00",
            "title": "Adapting Neural Text Classification for Improved Software Categorization",
            "authors": [
                "Alexander LeClair",
                "Zachary Eberhart",
                "Collin McMillan"
            ],
            "abstract": "Software Categorization is the task of organizing software into groups that\nbroadly describe the behavior of the software, such as \"editors\" or \"science.\"\nCategorization plays an important role in several maintenance tasks, such as\nrepository navigation and feature elicitation. Current approaches attempt to\ncast the problem as text classification, to make use of the rich body of\nliterature from the NLP domain. However, as we will show in this paper, text\nclassification algorithms are generally not applicable off-the-shelf to source\ncode; we found that they work well when high-level project descriptions are\navailable, but suffer very large performance penalties when classifying source\ncode and comments only. We propose a set of adaptations to a state-of-the-art\nneural classification algorithm and perform two evaluations: one with reference\ndata from Debian end-user programs, and one with a set of C/C++ libraries that\nwe hired professional programmers to annotate. We show that our proposed\napproach achieves performance exceeding that of previous software\nclassification techniques as well as a state-of-the-art neural text\nclassification technique.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1806.02724v2",
            "date": "2018-06-07 15:15:35+00:00",
            "title": "Speaker-Follower Models for Vision-and-Language Navigation",
            "authors": [
                "Daniel Fried",
                "Ronghang Hu",
                "Volkan Cirik",
                "Anna Rohrbach",
                "Jacob Andreas",
                "Louis-Philippe Morency",
                "Taylor Berg-Kirkpatrick",
                "Kate Saenko",
                "Dan Klein",
                "Trevor Darrell"
            ],
            "abstract": "Navigation guided by natural language instructions presents a challenging\nreasoning problem for instruction followers. Natural language instructions\ntypically identify only a few high-level decisions and landmarks rather than\ncomplete low-level motor behaviors; much of the missing information must be\ninferred based on perceptual context. In machine learning settings, this is\ndoubly challenging: it is difficult to collect enough annotated data to enable\nlearning of this reasoning process from scratch, and also difficult to\nimplement the reasoning process using generic sequence models. Here we describe\nan approach to vision-and-language navigation that addresses both these issues\nwith an embedded speaker model. We use this speaker model to (1) synthesize new\ninstructions for data augmentation and to (2) implement pragmatic reasoning,\nwhich evaluates how well candidate action sequences explain an instruction.\nBoth steps are supported by a panoramic action space that reflects the\ngranularity of human-generated instructions. Experiments show that all three\ncomponents of this approach---speaker-driven data augmentation, pragmatic\nreasoning and panoramic action space---dramatically improve the performance of\na baseline instruction follower, more than doubling the success rate over the\nbest existing approach on a standard benchmark.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1806.02814v1",
            "date": "2018-06-07 17:49:54+00:00",
            "title": "Embedding Transfer for Low-Resource Medical Named Entity Recognition: A Case Study on Patient Mobility",
            "authors": [
                "Denis Newman-Griffis",
                "Ayah Zirikly"
            ],
            "abstract": "Functioning is gaining recognition as an important indicator of global\nhealth, but remains under-studied in medical natural language processing\nresearch. We present the first analysis of automatically extracting\ndescriptions of patient mobility, using a recently-developed dataset of free\ntext electronic health records. We frame the task as a named entity recognition\n(NER) problem, and investigate the applicability of NER techniques to mobility\nextraction. As text corpora focused on patient functioning are scarce, we\nexplore domain adaptation of word embeddings for use in a recurrent neural\nnetwork NER system. We find that embeddings trained on a small in-domain corpus\nperform nearly as well as those learned from large out-of-domain corpora, and\nthat domain adaptation techniques yield additional improvements in both\nprecision and recall. Our analysis identifies several significant challenges in\nextracting descriptions of patient mobility, including the length and\ncomplexity of annotated entities and high linguistic variability in mobility\ndescriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02873v1",
            "date": "2018-06-06 07:45:06+00:00",
            "title": "Medical Concept Embedding with Time-Aware Attention",
            "authors": [
                "Xiangrui Cai",
                "Jinyang Gao",
                "Kee Yuan Ngiam",
                "Beng Chin Ooi",
                "Ying Zhang",
                "Xiaojie Yuan"
            ],
            "abstract": "Embeddings of medical concepts such as medication, procedure and diagnosis\ncodes in Electronic Medical Records (EMRs) are central to healthcare analytics.\nPrevious work on medical concept embedding takes medical concepts and EMRs as\nwords and documents respectively. Nevertheless, such models miss out the\ntemporal nature of EMR data. On the one hand, two consecutive medical concepts\ndo not indicate they are temporally close, but the correlations between them\ncan be revealed by the time gap. On the other hand, the temporal scopes of\nmedical concepts often vary greatly (e.g., \\textit{common cold} and\n\\textit{diabetes}). In this paper, we propose to incorporate the temporal\ninformation to embed medical codes. Based on the Continuous Bag-of-Words model,\nwe employ the attention mechanism to learn a \"soft\" time-aware context window\nfor each medical concept. Experiments on public and proprietary datasets\nthrough clustering and nearest neighbour search tasks demonstrate the\neffectiveness of our model, showing that it outperforms five state-of-the-art\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02908v2",
            "date": "2018-06-07 21:29:39+00:00",
            "title": "Is preprocessing of text really worth your time for online comment classification?",
            "authors": [
                "Fahim Mohammad"
            ],
            "abstract": "A large proportion of online comments present on public domains are\nconstructive, however a significant proportion are toxic in nature. The\ncomments contain lot of typos which increases the number of features manifold,\nmaking the ML model difficult to train. Considering the fact that the data\nscientists spend approximately 80% of their time in collecting, cleaning and\norganizing their data [1], we explored how much effort should we invest in the\npreprocessing (transformation) of raw comments before feeding it to the\nstate-of-the-art classification models. With the help of four models on Jigsaw\ntoxic comment classification data, we demonstrated that the training of model\nwithout any transformation produce relatively decent model. Applying even basic\ntransformations, in some cases, lead to worse performance and should be applied\nwith caution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02960v1",
            "date": "2018-06-08 03:49:34+00:00",
            "title": "Representation Learning of Entities and Documents from Knowledge Base Descriptions",
            "authors": [
                "Ikuya Yamada",
                "Hiroyuki Shindo",
                "Yoshiyasu Takefuji"
            ],
            "abstract": "In this paper, we describe TextEnt, a neural network model that learns\ndistributed representations of entities and documents directly from a knowledge\nbase (KB). Given a document in a KB consisting of words and entity annotations,\nwe train our model to predict the entity that the document describes and map\nthe document and its target entity close to each other in a continuous vector\nspace. Our model is trained using a large number of documents extracted from\nWikipedia. The performance of the proposed model is evaluated using two tasks,\nnamely fine-grained entity typing and multiclass text classification. The\nresults demonstrate that our model achieves state-of-the-art performance on\nboth tasks. The code and the trained representations are made available online\nfor further academic research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03357v1",
            "date": "2018-06-08 21:21:19+00:00",
            "title": "Measuring Conversational Productivity in Child Forensic Interviews",
            "authors": [
                "Victor Ardulov",
                "Manoj Kumar",
                "Shanna Williams",
                "Thomas Lyon",
                "Shrikanth Narayanan"
            ],
            "abstract": "Child Forensic Interviewing (FI) presents a challenge for effective\ninformation retrieval and decision making. The high stakes associated with the\nprocess demand that expert legal interviewers are able to effectively establish\na channel of communication and elicit substantive knowledge from the\nchild-client while minimizing potential for experiencing trauma. As a first\nstep toward computationally modeling and producing quality spoken interviewing\nstrategies and a generalized understanding of interview dynamics, we propose a\nnovel methodology to computationally model effectiveness criteria, by applying\nsummarization and topic modeling techniques to objectively measure and rank the\nresponsiveness and conversational productivity of a child during FI. We score\ninformation retrieval by constructing an agenda to represent general topics of\ninterest and measuring alignment with a given response and leveraging lexical\nentrainment for responsiveness. For comparison, we present our methods along\nwith traditional metrics of evaluation and discuss the use of prior information\nfor generating situational awareness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03529v2",
            "date": "2018-06-09 18:55:00+00:00",
            "title": "Learning to Search in Long Documents Using Document Structure",
            "authors": [
                "Mor Geva",
                "Jonathan Berant"
            ],
            "abstract": "Reading comprehension models are based on recurrent neural networks that\nsequentially process the document tokens. As interest turns to answering more\ncomplex questions over longer documents, sequential reading of large portions\nof text becomes a substantial bottleneck. Inspired by how humans use document\nstructure, we propose a novel framework for reading comprehension. We represent\ndocuments as trees, and model an agent that learns to interleave quick\nnavigation through the document tree with more expensive answer extraction. To\nencourage exploration of the document tree, we propose a new algorithm, based\non Deep Q-Network (DQN), which strategically samples tree nodes at training\ntime. Empirically we find our algorithm improves question answering performance\ncompared to DQN and a strong information-retrieval (IR) baseline, and that\nensembling our model with the IR baseline results in further gains in\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03561v1",
            "date": "2018-06-10 00:33:47+00:00",
            "title": "What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?",
            "authors": [
                "Peter Clark"
            ],
            "abstract": "This document gives a knowledge-oriented analysis of about 20 interesting\nRecognizing Textual Entailment (RTE) examples, drawn from the 2005 RTE5\ncompetition test set. The analysis ignores shallow statistical matching\ntechniques between T and H, and rather asks: What would it take to reasonably\ninfer that T implies H? What world knowledge would be needed for this task?\nAlthough such knowledge-intensive techniques have not had much success in RTE\nevaluations, ultimately an intelligent system should be expected to know and\ndeploy this kind of world knowledge required to perform this kind of reasoning.\n  The selected examples are typically ones which our RTE system (called BLUE)\ngot wrong and ones which require world knowledge to answer. In particular, the\nanalysis covers cases where there was near-perfect lexical overlap between T\nand H, yet the entailment was NO, i.e., examples that most likely all current\nRTE systems will have got wrong. A nice example is #341 (page 26), that\nrequires inferring from \"a river floods\" that \"a river overflows its banks\".\nSeems it should be easy, right? Enjoy!",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03648v1",
            "date": "2018-06-10 12:34:00+00:00",
            "title": "Neural Disease Named Entity Extraction with Character-based BiLSTM+CRF in Japanese Medical Text",
            "authors": [
                "Ken Yano"
            ],
            "abstract": "We propose an 'end-to-end' character-based recurrent neural network that\nextracts disease named entities from a Japanese medical text and simultaneously\njudges its modality as either positive or negative; i.e., the mentioned disease\nor symptom is affirmed or negated. The motivation to adopt neural networks is\nto learn effective lexical and structural representation features for Entity\nRecognition and also for Positive/Negative classification from an annotated\ncorpora without explicitly providing any rule-based or manual feature sets. We\nconfirmed the superiority of our method over previous char-based CRF or SVM\nmethods in the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03847v1",
            "date": "2018-06-11 07:52:28+00:00",
            "title": "A Multimodal Classifier Generative Adversarial Network for Carry and Place Tasks from Ambiguous Language Instructions",
            "authors": [
                "Aly Magassouba",
                "Komei Sugiura",
                "Hisashi Kawai"
            ],
            "abstract": "This paper focuses on a multimodal language understanding method for\ncarry-and-place tasks with domestic service robots. We address the case of\nambiguous instructions, that is, when the target area is not specified. For\ninstance \"put away the milk and cereal\" is a natural instruction where there is\nambiguity regarding the target area, considering environments in daily life.\nConventionally, this instruction can be disambiguated from a dialogue system,\nbut at the cost of time and cumbersome interaction. Instead, we propose a\nmultimodal approach, in which the instructions are disambiguated using the\nrobot's state and environment context. We develop the Multi-Modal Classifier\nGenerative Adversarial Network (MMC-GAN) to predict the likelihood of different\ntarget areas considering the robot's physical limitation and the target\nclutter. Our approach, MMC-GAN, significantly improves accuracy compared with\nbaseline methods that use instructions only or simple deep neural networks.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1806.03957v4",
            "date": "2018-06-11 13:25:23+00:00",
            "title": "Prosody Modifications for Question-Answering in Voice-Only Settings",
            "authors": [
                "Aleksandr Chuklin",
                "Aliaksei Severyn",
                "Johanne Trippas",
                "Enrique Alfonseca",
                "Hanna Silen",
                "Damiano Spina"
            ],
            "abstract": "Many popular form factors of digital assistants---such as Amazon Echo, Apple\nHomepod, or Google Home---enable the user to hold a conversation with these\nsystems based only on the speech modality. The lack of a screen presents unique\nchallenges. To satisfy the information need of a user, the presentation of the\nanswer needs to be optimized for such voice-only interactions. In this paper,\nwe propose a task of evaluating the usefulness of audio transformations (i.e.,\nprosodic modifications) for voice-only question answering. We introduce a\ncrowdsourcing setup where we evaluate the quality of our proposed modifications\nalong multiple dimensions corresponding to the informativeness, naturalness,\nand ability of the user to identify key parts of the answer. We offer a set of\nprosodic modifications that highlight potentially important parts of the answer\nusing various acoustic cues. Our experiments show that some of these prosodic\nmodifications lead to better comprehension at the expense of only slightly\ndegraded naturalness of the audio.",
            "journal ref": "Lecture Notes in Computer Science, vol 11696 CLEF 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04189v1",
            "date": "2018-06-11 18:57:49+00:00",
            "title": "Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models",
            "authors": [
                "Minjia Zhang",
                "Xiaodong Liu",
                "Wenhan Wang",
                "Jianfeng Gao",
                "Yuxiong He"
            ],
            "abstract": "Neural language models (NLMs) have recently gained a renewed interest by\nachieving state-of-the-art performance across many natural language processing\n(NLP) tasks. However, NLMs are very computationally demanding largely due to\nthe computational cost of the softmax layer over a large vocabulary. We observe\nthat, in decoding of many NLP tasks, only the probabilities of the top-K\nhypotheses need to be calculated preciously and K is often much smaller than\nthe vocabulary size. This paper proposes a novel softmax layer approximation\nalgorithm, called Fast Graph Decoder (FGD), which quickly identifies, for a\ngiven context, a set of K words that are most likely to occur according to a\nNLM. We demonstrate that FGD reduces the decoding time by an order of magnitude\nwhile attaining close to the full softmax baseline accuracy on neural machine\ntranslation and language modeling tasks. We also prove the theoretical\nguarantee on the softmax approximation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04313v1",
            "date": "2018-06-12 03:25:15+00:00",
            "title": "Embedding Text in Hyperbolic Spaces",
            "authors": [
                "Bhuwan Dhingra",
                "Christopher J. Shallue",
                "Mohammad Norouzi",
                "Andrew M. Dai",
                "George E. Dahl"
            ],
            "abstract": "Natural language text exhibits hierarchical structure in a variety of\nrespects. Ideally, we could incorporate our prior knowledge of this\nhierarchical structure into unsupervised learning algorithms that work on text\ndata. Recent work by Nickel & Kiela (2017) proposed using hyperbolic instead of\nEuclidean embedding spaces to represent hierarchical data and demonstrated\nencouraging results when embedding graphs. In this work, we extend their method\nwith a re-parameterization technique that allows us to learn hyperbolic\nembeddings of arbitrarily parameterized objects. We apply this framework to\nlearn word and sentence embeddings in hyperbolic space in an unsupervised\nmanner from text corpora. The resulting embeddings seem to encode certain\nintuitive notions of hierarchy, such as word-context frequency and phrase\nconstituency. However, the implicit continuous hierarchy in the learned\nhyperbolic space makes interrogating the model's learned hierarchies more\ndifficult than for models that learn explicit edges between items. The learned\nhyperbolic embeddings show improvements over Euclidean embeddings in some --\nbut not all -- downstream tasks, suggesting that hierarchical organization is\nmore useful for some tasks than others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04387v2",
            "date": "2018-06-12 08:22:21+00:00",
            "title": "Knowledge Amalgam: Generating Jokes and Quotes Together",
            "authors": [
                "Bhargav Chippada",
                "Shubajit Saha"
            ],
            "abstract": "Generating humor and quotes are very challenging problems in the field of\ncomputational linguistics and are often tackled separately. In this paper, we\npresent a controlled Long Short-Term Memory (LSTM) architecture which is\ntrained with categorical data like jokes and quotes together by passing\ncategory as an input along with the sequence of words. The idea is that a\nsingle neural net will learn the structure of both jokes and quotes to generate\nthem on demand according to input category. Importantly, we believe the neural\nnet has more knowledge as it's trained on different datasets and hence will\nenable it to generate more creative jokes or quotes from the mixture of\ninformation. May the network generate a funny inspirational joke!",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04510v1",
            "date": "2018-06-08 03:29:30+00:00",
            "title": "Dank Learning: Generating Memes Using Deep Neural Networks",
            "authors": [
                "Abel L Peirson V",
                "E Meltem Tolunay"
            ],
            "abstract": "We introduce a novel meme generation system, which given any image can\nproduce a humorous and relevant caption. Furthermore, the system can be\nconditioned on not only an image but also a user-defined label relating to the\nmeme template, giving a handle to the user on meme content. The system uses a\npretrained Inception-v3 network to return an image embedding which is passed to\nan attention-based deep-layer LSTM model producing the caption - inspired by\nthe widely recognised Show and Tell Model. We implement a modified beam search\nto encourage diversity in the captions. We evaluate the quality of our model\nusing perplexity and human assessment on both the quality of memes generated\nand whether they can be differentiated from real ones. Our model produces\noriginal memes that cannot on the whole be differentiated from real ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04511v1",
            "date": "2018-06-08 14:01:31+00:00",
            "title": "Multilingual Sentiment Analysis: An RNN-Based Framework for Limited Data",
            "authors": [
                "Ethem F. Can",
                "Aysu Ezen-Can",
                "Fazli Can"
            ],
            "abstract": "Sentiment analysis is a widely studied NLP task where the goal is to\ndetermine opinions, emotions, and evaluations of users towards a product, an\nentity or a service that they are reviewing. One of the biggest challenges for\nsentiment analysis is that it is highly language dependent. Word embeddings,\nsentiment lexicons, and even annotated data are language specific. Further,\noptimizing models for each language is very time consuming and labor intensive\nespecially for recurrent neural network models. From a resource perspective, it\nis very challenging to collect data for different languages.\n  In this paper, we look for an answer to the following research question: can\na sentiment analysis model trained on a language be reused for sentiment\nanalysis in other languages, Russian, Spanish, Turkish, and Dutch, where the\ndata is more limited? Our goal is to build a single model in the language with\nthe largest dataset available for the task, and reuse it for languages that\nhave limited resources. For this purpose, we train a sentiment analysis model\nusing recurrent neural networks with reviews in English. We then translate\nreviews in other languages and reuse this model to evaluate the sentiments.\nExperimental results show that our robust approach of single model trained on\nEnglish reviews statistically significantly outperforms the baselines in\nseveral different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04535v1",
            "date": "2018-06-11 09:45:34+00:00",
            "title": "Automatic Target Recovery for Hindi-English Code Mixed Puns",
            "authors": [
                "Srishti Aggarwal",
                "Kritik Mathur",
                "Radhika Mamidi"
            ],
            "abstract": "In order for our computer systems to be more human-like, with a higher\nemotional quotient, they need to be able to process and understand intrinsic\nhuman language phenomena like humour. In this paper, we consider a subtype of\nhumour - puns, which are a common type of wordplay-based jokes. In particular,\nwe consider code-mixed puns which have become increasingly mainstream on social\nmedia, in informal conversations and advertisements and aim to build a system\nwhich can automatically identify the pun location and recover the target of\nsuch puns. We first study and classify code-mixed puns into two categories\nnamely intra-sentential and intra-word, and then propose a four-step algorithm\nto recover the pun targets for puns belonging to the intra-sentential category.\nOur algorithm uses language models, and phonetic similarity-based features to\nget the desired results. We test our approach on a small set of code-mixed\npunning advertisements, and observe that our system is successfully able to\nrecover the targets for 67% of the puns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04616v1",
            "date": "2018-06-12 15:49:14+00:00",
            "title": "Deep Learning to Detect Redundant Method Comments",
            "authors": [
                "Annie Louis",
                "Santanu Kumar Dash",
                "Earl T. Barr",
                "Charles Sutton"
            ],
            "abstract": "Comments in software are critical for maintenance and reuse. But apart from\nprescriptive advice, there is little practical support or quantitative\nunderstanding of what makes a comment useful. In this paper, we introduce the\ntask of identifying comments which are uninformative about the code they are\nmeant to document. To address this problem, we introduce the notion of comment\nentailment from code, high entailment indicating that a comment's natural\nlanguage semantics can be inferred directly from the code. Although not all\nentailed comments are low quality, comments that are too easily inferred, for\nexample, comments that restate the code, are widely discouraged by authorities\non software style. Based on this, we develop a tool called CRAIC which scores\nmethod-level comments for redundancy. Highly redundant comments can then be\nexpanded or alternately removed by the developer. CRAIC uses deep language\nmodels to exploit large software corpora without requiring expensive manual\nannotations of entailment. We show that CRAIC can perform the comment\nentailment task with good agreement with human judgements. Our findings also\nhave implications for documentation tools. For example, we find that common\ntags in Javadoc are at least two times more predictable from code than\nnon-Javadoc sentences, suggesting that Javadoc tags are less informative than\nmore free-form comments",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1806.04860v1",
            "date": "2018-06-13 06:37:42+00:00",
            "title": "Learning Visual Knowledge Memory Networks for Visual Question Answering",
            "authors": [
                "Zhou Su",
                "Chen Zhu",
                "Yinpeng Dong",
                "Dongqi Cai",
                "Yurong Chen",
                "Jianguo Li"
            ],
            "abstract": "Visual question answering (VQA) requires joint comprehension of images and\nnatural language questions, where many questions can't be directly or clearly\nanswered from visual content but require reasoning from structured human\nknowledge with confirmation from visual content. This paper proposes visual\nknowledge memory network (VKMN) to address this issue, which seamlessly\nincorporates structured human knowledge and deep visual features into memory\nnetworks in an end-to-end learning framework. Comparing to existing methods for\nleveraging external knowledge for supporting VQA, this paper stresses more on\ntwo missing mechanisms. First is the mechanism for integrating visual contents\nwith knowledge facts. VKMN handles this issue by embedding knowledge triples\n(subject, relation, target) and deep visual features jointly into the visual\nknowledge features. Second is the mechanism for handling multiple knowledge\nfacts expanding from question and answer pairs. VKMN stores joint embedding\nusing key-value pair structure in the memory networks so that it is easy to\nhandle multiple facts. Experiments show that the proposed method achieves\npromising results on both VQA v1.0 and v2.0 benchmarks, while outperforms\nstate-of-the-art methods on the knowledge-reasoning related questions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1806.04973v1",
            "date": "2018-06-13 12:16:37+00:00",
            "title": "OpenEDGAR: Open Source Software for SEC EDGAR Analysis",
            "authors": [
                "Michael J Bommarito II",
                "Daniel Martin Katz",
                "Eric M Detterman"
            ],
            "abstract": "OpenEDGAR is an open source Python framework designed to rapidly construct\nresearch databases based on the Electronic Data Gathering, Analysis, and\nRetrieval (EDGAR) system operated by the US Securities and Exchange Commission\n(SEC). OpenEDGAR is built on the Django application framework, supports\ndistributed compute across one or more servers, and includes functionality to\n(i) retrieve and parse index and filing data from EDGAR, (ii) build tables for\nkey metadata like form type and filer, (iii) retrieve, parse, and update CIK to\nticker and industry mappings, (iv) extract content and metadata from filing\ndocuments, and (v) search filing document contents. OpenEDGAR is designed for\nuse in both academic research and industrial applications, and is distributed\nunder MIT License at https://github.com/LexPredict/openedgar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05030v1",
            "date": "2018-06-13 13:37:34+00:00",
            "title": "Visually grounded cross-lingual keyword spotting in speech",
            "authors": [
                "Herman Kamper",
                "Michael Roth"
            ],
            "abstract": "Recent work considered how images paired with speech can be used as\nsupervision for building speech systems when transcriptions are not available.\nWe ask whether visual grounding can be used for cross-lingual keyword spotting:\ngiven a text keyword in one language, the task is to retrieve spoken utterances\ncontaining that keyword in another language. This could enable searching\nthrough speech in a low-resource language using text queries in a high-resource\nlanguage. As a proof-of-concept, we use English speech with German queries: we\nuse a German visual tagger to add keyword labels to each training image, and\nthen train a neural network to map English speech to German keywords. Without\nseeing parallel speech-transcriptions or translations, the model achieves a\nprecision at ten of 58%. We show that most erroneous retrievals contain\nequivalent or semantically relevant keywords; excluding these would improve\nP@10 to 91%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05130v3",
            "date": "2018-06-13 16:26:28+00:00",
            "title": "Detecting Speech Act Types in Developer Question/Answer Conversations During Bug Repair",
            "authors": [
                "Andrew Wood",
                "Paige Rodeghero",
                "Ameer Armaly",
                "Collin McMillan"
            ],
            "abstract": "This paper targets the problem of speech act detection in conversations about\nbug repair. We conduct a \"Wizard of Oz\" experiment with 30 professional\nprogrammers, in which the programmers fix bugs for two hours, and use a\nsimulated virtual assistant for help. Then, we use an open coding manual\nannotation procedure to identify the speech act types in the conversations.\nFinally, we train and evaluate a supervised learning algorithm to automatically\ndetect the speech act types in the conversations. In 30 two-hour conversations,\nwe made 2459 annotations and uncovered 26 speech act types. Our automated\ndetection achieved 69% precision and 50% recall. The key application of this\nwork is to advance the state of the art for virtual assistants in software\nengineering. Virtual assistant technology is growing rapidly, though\napplications in software engineering are behind those in other areas, largely\ndue to a lack of relevant data and experiments. This paper targets this problem\nin the area of developer Q/A conversations about bug repair.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1806.05480v1",
            "date": "2018-06-14 11:38:24+00:00",
            "title": "Automatic Language Identification for Romance Languages using Stop Words and Diacritics",
            "authors": [
                "Ciprian-Octavian Truic\u0103",
                "Julien Velcin",
                "Alexandru Boicea"
            ],
            "abstract": "Automatic language identification is a natural language processing problem\nthat tries to determine the natural language of a given content. In this paper\nwe present a statistical method for automatic language identification of\nwritten text using dictionaries containing stop words and diacritics. We\npropose different approaches that combine the two dictionaries to accurately\ndetermine the language of textual corpora. This method was chosen because stop\nwords and diacritics are very specific to a language, although some languages\nhave some similar words and special characters they are not all common. The\nlanguages taken into account were romance languages because they are very\nsimilar and usually it is hard to distinguish between them from a computational\npoint of view. We have tested our method using a Twitter corpus and a news\narticle corpus. Both corpora consists of UTF-8 encoded text, so the diacritics\ncould be taken into account, in the case that the text has no diacritics only\nthe stop words are used to determine the language of the text. The experimental\nresults show that the proposed method has an accuracy of over 90% for small\ntexts and over 99.8% for",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05484v2",
            "date": "2018-06-14 11:47:14+00:00",
            "title": "Nearly Zero-Shot Learning for Semantic Decoding in Spoken Dialogue Systems",
            "authors": [
                "Lina M. Rojas-Barahona",
                "Stefan Ultes",
                "Pawel Budzianowski",
                "I\u00f1igo Casanueva",
                "Milica Gasic",
                "Bo-Hsiang Tseng",
                "Steve Young"
            ],
            "abstract": "This paper presents two ways of dealing with scarce data in semantic decoding\nusing N-Best speech recognition hypotheses. First, we learn features by using a\ndeep learning architecture in which the weights for the unknown and known\ncategories are jointly optimised. Second, an unsupervised method is used for\nfurther tuning the weights. Sharing weights injects prior knowledge to unknown\ncategories. The unsupervised tuning (i.e. the risk minimisation) improves the\nF-Measure when recognising nearly zero-shot data on the DSTC3 corpus. This\nunsupervised method can be applied subject to two assumptions: the rank of the\nclass marginal is assumed to be known and the class-conditional scores of the\nclassifier are assumed to follow a Gaussian distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05521v1",
            "date": "2018-06-14 13:11:36+00:00",
            "title": "SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment",
            "authors": [
                "Jisun An",
                "Haewoon Kwak",
                "Yong-Yeol Ahn"
            ],
            "abstract": "Because word semantics can substantially change across communities and\ncontexts, capturing domain-specific word semantics is an important challenge.\nHere, we propose SEMAXIS, a simple yet powerful framework to characterize word\nsemantics using many semantic axes in word- vector spaces beyond sentiment. We\ndemonstrate that SEMAXIS can capture nuanced semantic representations in\nmultiple online communities. We also show that, when the sentiment axis is\nexamined, SEMAXIS outperforms the state-of-the-art approaches in building\ndomain-specific sentiment lexicons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05645v1",
            "date": "2018-06-14 16:56:44+00:00",
            "title": "Grounded Textual Entailment",
            "authors": [
                "Hoa Trong Vu",
                "Claudio Greco",
                "Aliia Erofeeva",
                "Somayeh Jafaritazehjan",
                "Guido Linders",
                "Marc Tanti",
                "Alberto Testoni",
                "Raffaella Bernardi",
                "Albert Gatt"
            ],
            "abstract": "Capturing semantic relations between sentences, such as entailment, is a\nlong-standing challenge for computational semantics. Logic-based models analyse\nentailment in terms of possible worlds (interpretations, or situations) where a\npremise P entails a hypothesis H iff in all worlds where P is true, H is also\ntrue. Statistical models view this relationship probabilistically, addressing\nit in terms of whether a human would likely infer H from P. In this paper, we\nwish to bridge these two perspectives, by arguing for a visually-grounded\nversion of the Textual Entailment task. Specifically, we ask whether models can\nperform better if, in addition to P and H, there is also an image\n(corresponding to the relevant \"world\" or \"situation\"). We use a multimodal\nversion of the SNLI dataset (Bowman et al., 2015) and we compare \"blind\" and\nvisually-augmented models of textual entailment. We show that visual\ninformation is beneficial, but we also conduct an in-depth error analysis that\nreveals that current multimodal models are not performing \"grounding\" in an\noptimal fashion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05838v1",
            "date": "2018-06-15 07:38:01+00:00",
            "title": "The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities",
            "authors": [
                "Marco Del Tredici",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "We investigate the birth and diffusion of lexical innovations in a large\ndataset of online social communities. We build on sociolinguistic theories and\nfocus on the relation between the spread of a novel term and the social role of\nthe individuals who use it, uncovering characteristics of innovators and\nadopters. Finally, we perform a prediction task that allows us to anticipate\nwhether an innovation will successfully spread within a community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06228v1",
            "date": "2018-06-16 12:05:24+00:00",
            "title": "Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling",
            "authors": [
                "N. Majumder",
                "D. Hazarika",
                "A. Gelbukh",
                "E. Cambria",
                "S. Poria"
            ],
            "abstract": "Multimodal sentiment analysis is a very actively growing field of research. A\npromising area of opportunity in this field is to improve the multimodal fusion\nmechanism. We present a novel feature fusion strategy that proceeds in a\nhierarchical fashion, first fusing the modalities two in two and only then\nfusing all three modalities. On multimodal sentiment analysis of individual\nutterances, our strategy outperforms conventional concatenation of features by\n1%, which amounts to 5% reduction in error rate. On utterance-level multimodal\nsentiment analysis of multi-utterance video clips, for which current\nstate-of-the-art techniques incorporate contextual information from other\nutterances of the same clip, our hierarchical fusion gives up to 2.4% (almost\n10% error rate reduction) over currently used concatenation. The implementation\nof our method is publicly available in the form of open-source code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06371v2",
            "date": "2018-06-17 12:13:37+00:00",
            "title": "Multimodal Grounding for Language Processing",
            "authors": [
                "Lisa Beinborn",
                "Teresa Botschen",
                "Iryna Gurevych"
            ],
            "abstract": "This survey discusses how recent developments in multimodal processing\nfacilitate conceptual grounding of language. We categorize the information flow\nin multimodal processing with respect to cognitive models of human information\nprocessing and analyze different methods for combining multimodal\nrepresentations. Based on this methodological inventory, we discuss the benefit\nof multimodal grounding for a variety of language processing tasks and the\nchallenges that arise. We particularly focus on multimodal grounding of verbs\nwhich play a crucial role for the compositional power of language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06407v1",
            "date": "2018-06-17 16:25:57+00:00",
            "title": "An Improved Text Sentiment Classification Model Using TF-IDF and Next Word Negation",
            "authors": [
                "Bijoyan Das",
                "Sarit Chakraborty"
            ],
            "abstract": "With the rapid growth of Text sentiment analysis, the demand for automatic\nclassification of electronic documents has increased by leaps and bound. The\nparadigm of text classification or text mining has been the subject of many\nresearch works in recent time. In this paper we propose a technique for text\nsentiment classification using term frequency- inverse document frequency\n(TF-IDF) along with Next Word Negation (NWN). We have also compared the\nperformances of binary bag of words model, TF-IDF model and TF-IDF with next\nword negation (TF-IDF-NWN) model for text classification. Our proposed model is\nthen applied on three different text mining algorithms and we found the Linear\nSupport vector machine (LSVM) is the most appropriate to work with our proposed\nmodel. The achieved results show significant increase in accuracy compared to\nearlier methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06411v1",
            "date": "2018-06-17 16:38:48+00:00",
            "title": "Measuring Semantic Coherence of a Conversation",
            "authors": [
                "Svitlana Vakulenko",
                "Maarten de Rijke",
                "Michael Cochez",
                "Vadim Savenkov",
                "Axel Polleres"
            ],
            "abstract": "Conversational systems have become increasingly popular as a way for humans\nto interact with computers. To be able to provide intelligent responses,\nconversational systems must correctly model the structure and semantics of a\nconversation. We introduce the task of measuring semantic (in)coherence in a\nconversation with respect to background knowledge, which relies on the\nidentification of semantic relations between concepts introduced during a\nconversation. We propose and evaluate graph-based and machine learning-based\napproaches for measuring semantic coherence using knowledge graphs, their\nvector space embeddings and word embedding models, as sources of background\nknowledge. We demonstrate how these approaches are able to uncover different\ncoherence patterns in conversations on the Ubuntu Dialogue Corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06478v1",
            "date": "2018-06-18 02:06:46+00:00",
            "title": "Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment",
            "authors": [
                "Muhao Chen",
                "Yingtao Tian",
                "Kai-Wei Chang",
                "Steven Skiena",
                "Carlo Zaniolo"
            ],
            "abstract": "Multilingual knowledge graph (KG) embeddings provide latent semantic\nrepresentations of entities and structured knowledge with cross-lingual\ninferences, which benefit various knowledge-driven cross-lingual NLP tasks.\nHowever, precisely learning such cross-lingual inferences is usually hindered\nby the low coverage of entity alignment in many KGs. Since many multilingual\nKGs also provide literal descriptions of entities, in this paper, we introduce\nan embedding-based approach which leverages a weakly aligned multilingual KG\nfor semi-supervised cross-lingual learning using entity descriptions. Our\napproach performs co-training of two embedding models, i.e. a multilingual KG\nembedding model and a multilingual literal description embedding model. The\nmodels are trained on a large Wikipedia-based trilingual dataset where most\nentity alignment is unknown to training. Experimental results show that the\nperformance of the proposed approach on the entity alignment task improves at\neach iteration of co-training, and eventually reaches a stage at which it\nsignificantly surpasses previous approaches. We also show that our approach has\npromising abilities for zero-shot entity alignment, and cross-lingual KG\ncompletion.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.06734v1",
            "date": "2018-06-18 14:35:14+00:00",
            "title": "Unsupervised Word Segmentation from Speech with Attention",
            "authors": [
                "Pierre Godard",
                "Marcely Zanon-Boito",
                "Lucas Ondel",
                "Alexandre Berard",
                "Fran\u00e7ois Yvon",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "We present a first attempt to perform attentional word segmentation directly\nfrom the speech signal, with the final goal to automatically identify lexical\nunits in a low-resource, unwritten language (UL). Our methodology assumes a\npairing between recordings in the UL with translations in a well-resourced\nlanguage. It uses Acoustic Unit Discovery (AUD) to convert speech into a\nsequence of pseudo-phones that is segmented using neural soft-alignments\nproduced by a neural machine translation model. Evaluation uses an actual Bantu\nUL, Mboshi; comparisons to monolingual and bilingual baselines illustrate the\npotential of attentional word segmentation for language documentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06972v1",
            "date": "2018-06-18 22:29:51+00:00",
            "title": "Comparative Analysis of Neural QA models on SQuAD",
            "authors": [
                "Soumya Wadhwa",
                "Khyathi Raghavi Chandu",
                "Eric Nyberg"
            ],
            "abstract": "The task of Question Answering has gained prominence in the past few decades\nfor testing the ability of machines to understand natural language. Large\ndatasets for Machine Reading have led to the development of neural models that\ncater to deeper language understanding compared to information retrieval tasks.\nDifferent components in these neural architectures are intended to tackle\ndifferent challenges. As a first step towards achieving generalization across\nmultiple domains, we attempt to understand and compare the peculiarities of\nexisting end-to-end neural models on the Stanford Question Answering Dataset\n(SQuAD) by performing quantitative as well as qualitative analysis of the\nresults attained by each of them. We observed that prediction errors reflect\ncertain model-specific biases, which we further discuss in this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07000v4",
            "date": "2018-06-19 01:28:58+00:00",
            "title": "A Syntactically Constrained Bidirectional-Asynchronous Approach for Emotional Conversation Generation",
            "authors": [
                "Jingyuan Li",
                "Xiao Sun"
            ],
            "abstract": "Traditional neural language models tend to generate generic replies with poor\nlogic and no emotion. In this paper, a syntactically constrained\nbidirectional-asynchronous approach for emotional conversation generation\n(E-SCBA) is proposed to address this issue. In our model, pre-generated emotion\nkeywords and topic keywords are asynchronously introduced into the process of\ndecoding. It is much different from most existing methods which generate\nreplies from the first word to the last. Through experiments, the results\nindicate that our approach not only improves the diversity of replies, but\ngains a boost on both logic and emotion compared with baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07139v1",
            "date": "2018-06-19 10:12:25+00:00",
            "title": "Using J-K fold Cross Validation to Reduce Variance When Tuning NLP Models",
            "authors": [
                "Henry B. Moss",
                "David S. Leslie",
                "Paul Rayson"
            ],
            "abstract": "K-fold cross validation (CV) is a popular method for estimating the true\nperformance of machine learning models, allowing model selection and parameter\ntuning. However, the very process of CV requires random partitioning of the\ndata and so our performance estimates are in fact stochastic, with variability\nthat can be substantial for natural language processing tasks. We demonstrate\nthat these unstable estimates cannot be relied upon for effective parameter\ntuning. The resulting tuned parameters are highly sensitive to how our data is\npartitioned, meaning that we often select sub-optimal parameter choices and\nhave serious reproducibility issues.\n  Instead, we propose to use the less variable J-K-fold CV, in which J\nindependent K-fold cross validations are used to assess performance. Our main\ncontributions are extending J-K-fold CV from performance estimation to\nparameter tuning and investigating how to choose J and K. We argue that\nvariability is more important than bias for effective tuning and so advocate\nlower choices of K than are typically seen in the NLP literature, instead use\nthe saved computation to increase J. To demonstrate the generality of our\nrecommendations we investigate a wide range of case-studies: sentiment\nclassification (both general and target-specific), part-of-speech tagging and\ndocument classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07221v1",
            "date": "2018-06-19 13:40:27+00:00",
            "title": "Self-adaptive Privacy Concern Detection for User-generated Content",
            "authors": [
                "Xuan-Son Vu",
                "Lili Jiang"
            ],
            "abstract": "To protect user privacy in data analysis, a state-of-the-art strategy is\ndifferential privacy in which scientific noise is injected into the real\nanalysis output. The noise masks individual's sensitive information contained\nin the dataset. However, determining the amount of noise is a key challenge,\nsince too much noise will destroy data utility while too little noise will\nincrease privacy risk. Though previous research works have designed some\nmechanisms to protect data privacy in different scenarios, most of the existing\nstudies assume uniform privacy concerns for all individuals. Consequently,\nputting an equal amount of noise to all individuals leads to insufficient\nprivacy protection for some users, while over-protecting others. To address\nthis issue, we propose a self-adaptive approach for privacy concern detection\nbased on user personality. Our experimental studies demonstrate the\neffectiveness to address a suitable personalized privacy protection for\ncold-start users (i.e., without their privacy-concern information in training\ndata).",
            "journal ref": "Proceedings of the 19th International Conference on Computational\n  Linguistics and Intelligent Text Processing, 2018",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1806.07346v1",
            "date": "2018-06-15 20:11:22+00:00",
            "title": "A Scalable Machine Learning Approach for Inferring Probabilistic US-LI-RADS Categorization",
            "authors": [
                "Imon Banerjee",
                "Hailey H. Choi",
                "Terry Desser",
                "Daniel L. Rubin"
            ],
            "abstract": "We propose a scalable computerized approach for large-scale inference of\nLiver Imaging Reporting and Data System (LI-RADS) final assessment categories\nin narrative ultrasound (US) reports. Although our model was trained on reports\ncreated using a LI-RADS template, it was also able to infer LI-RADS scoring for\nunstructured reports that were created before the LI-RADS guidelines were\nestablished. No human-labelled data was required in any step of this study; for\ntraining, LI-RADS scores were automatically extracted from those reports that\ncontained structured LI-RADS scores, and it translated the derived knowledge to\nreasoning on unstructured radiology reports. By providing automated LI-RADS\ncategorization, our approach may enable standardizing screening recommendations\nand treatment planning of patients at risk for hepatocellular carcinoma, and it\nmay facilitate AI-based healthcare research with US images by offering large\nscale text mining and data gathering opportunities from standard hospital\nclinical data repositories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07495v1",
            "date": "2018-06-19 23:05:18+00:00",
            "title": "Joint Neural Entity Disambiguation with Output Space Search",
            "authors": [
                "Hamed Shahbazi",
                "Xiaoli Z. Fern",
                "Reza Ghaeini",
                "Chao Ma",
                "Rasha Obeidat",
                "Prasad Tadepalli"
            ],
            "abstract": "In this paper, we present a novel model for entity disambiguation that\ncombines both local contextual information and global evidences through Limited\nDiscrepancy Search (LDS). Given an input document, we start from a complete\nsolution constructed by a local model and conduct a search in the space of\npossible corrections to improve the local solution from a global view point.\nOur search utilizes a heuristic function to focus more on the least confident\nlocal decisions and a pruning function to score the global solutions based on\ntheir local fitness and the global coherences among the predicted entities.\nExperimental results on CoNLL 2003 and TAC 2010 benchmarks verify the\neffectiveness of our model.",
            "journal ref": "Proceedings of COLING 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07832v1",
            "date": "2018-06-20 16:29:01+00:00",
            "title": "StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing",
            "authors": [
                "Pengcheng Yin",
                "Chunting Zhou",
                "Junxian He",
                "Graham Neubig"
            ],
            "abstract": "Semantic parsing is the task of transducing natural language (NL) utterances\ninto formal meaning representations (MRs), commonly represented as tree\nstructures. Annotating NL utterances with their corresponding MRs is expensive\nand time-consuming, and thus the limited availability of labeled data often\nbecomes the bottleneck of data-driven, supervised models. We introduce\nStructVAE, a variational auto-encoding model for semisupervised semantic\nparsing, which learns both from limited amounts of parallel data, and\nreadily-available unlabeled NL utterances. StructVAE models latent MRs not\nobserved in the unlabeled data as tree-structured latent variables. Experiments\non semantic parsing on the ATIS domain and Python code generation show that\nwith extra unlabeled data, StructVAE outperforms strong supervised models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07974v1",
            "date": "2018-06-19 12:43:41+00:00",
            "title": "A Survey of Recent DNN Architectures on the TIMIT Phone Recognition Task",
            "authors": [
                "Josef Michalek",
                "Jan Vanek"
            ],
            "abstract": "In this survey paper, we have evaluated several recent deep neural network\n(DNN) architectures on a TIMIT phone recognition task. We chose the TIMIT\ncorpus due to its popularity and broad availability in the community. It also\nsimulates a low-resource scenario that is helpful in minor languages. Also, we\nprefer the phone recognition task because it is much more sensitive to an\nacoustic model quality than a large vocabulary continuous speech recognition\n(LVCSR) task. In recent years, many DNN published papers reported results on\nTIMIT. However, the reported phone error rates (PERs) were often much higher\nthan a PER of a simple feed-forward (FF) DNN. That was the main motivation of\nthis paper: To provide a baseline DNNs with open-source scripts to easily\nreplicate the baseline results for future papers with lowest possible PERs.\nAccording to our knowledge, the best-achieved PER of this survey is better than\nthe best-published PER to date.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08467v1",
            "date": "2018-06-22 01:58:44+00:00",
            "title": "Paragraph-based complex networks: application to document classification and authenticity verification",
            "authors": [
                "Henrique F. de Arruda",
                "Vanessa Q. Marinho",
                "Luciano da F. Costa",
                "Diego R. Amancio"
            ],
            "abstract": "With the increasing number of texts made available on the Internet, many\napplications have relied on text mining tools to tackle a diversity of\nproblems. A relevant model to represent texts is the so-called word adjacency\n(co-occurrence) representation, which is known to capture mainly syntactical\nfeatures of texts.In this study, we introduce a novel network representation\nthat considers the semantic similarity between paragraphs. Two main properties\nof paragraph networks are considered: (i) their ability to incorporate\ncharacteristics that can discriminate real from artificial, shuffled\nmanuscripts and (ii) their ability to capture syntactical and semantic textual\nfeatures. Our results revealed that real texts are organized into communities,\nwhich turned out to be an important feature for discriminating them from\nartificial texts. Interestingly, we have also found that, differently from\ntraditional co-occurrence networks, the adopted representation is able to\ncapture semantic features. Additionally, the proposed framework was employed to\nanalyze the Voynich manuscript, which was found to be compatible with texts\nwritten in natural languages. Taken together, our findings suggest that the\nproposed methodology can be combined with traditional network models to improve\ntext classification tasks.",
            "journal ref": "Information Processing & Management 56 (3) 479-494, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09030v1",
            "date": "2018-06-23 20:08:56+00:00",
            "title": "On Adversarial Examples for Character-Level Neural Machine Translation",
            "authors": [
                "Javid Ebrahimi",
                "Daniel Lowd",
                "Dejing Dou"
            ],
            "abstract": "Evaluating on adversarial examples has become a standard procedure to measure\nrobustness of deep learning models. Due to the difficulty of creating white-box\nadversarial examples for discrete text input, most analyses of the robustness\nof NLP models have been done through black-box adversarial examples. We\ninvestigate adversarial examples for character-level neural machine translation\n(NMT), and contrast black-box adversaries with a novel white-box adversary,\nwhich employs differentiable string-edit operations to rank adversarial\nchanges. We propose two novel types of attacks which aim to remove or change a\nword in a translation, rather than simply break the NMT. We demonstrate that\nwhite-box adversarial examples are significantly stronger than their black-box\ncounterparts in different attack scenarios, which show more serious\nvulnerabilities than previously known. In addition, after performing\nadversarial training, which takes only 3 times longer than regular training, we\ncan improve the model's robustness significantly.",
            "journal ref": "COLING 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09279v1",
            "date": "2018-06-25 04:17:44+00:00",
            "title": "Framework for Opinion Mining Approach to Augment Education System Performance",
            "authors": [
                "Amritpal Kaur",
                "Harkiran Kaur"
            ],
            "abstract": "The extensive expansion growth of social networking sites allows the people\nto share their views and experiences freely with their peers on internet. Due\nto this, huge amount of data is generated on everyday basis which can be used\nfor the opinion mining to extract the views of people in a particular field.\nOpinion mining finds its applications in many areas such as Tourism, Politics,\neducation and entertainment, etc. It has not been extensively implemented in\narea of education system. This paper discusses the malpractices in the present\nexamination system. In the present scenario, Opinion mining is vastly used for\ndecision making. The authors of this paper have designed a framework by\napplying Na\\\"ive Bayes approach to the education dataset. The various phases of\nNa\\\"ive Bayes approach include three steps: conversion of data into frequency\ntable, making classes of dataset and apply the Na\\\"ive Bayes algorithm equation\nto calculate the probabilities of classes. Finally the highest probability\nclass is the outcome of this prediction. These predictions are used to make\nimprovements in the education system and help to provide better education.",
            "journal ref": "http://ijitce.co.uk/vol8n6.aspx June 2018 Issue Vol.8 No.6",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1806.09835v1",
            "date": "2018-06-26 08:08:30+00:00",
            "title": "Graph-to-Sequence Learning using Gated Graph Neural Networks",
            "authors": [
                "Daniel Beck",
                "Gholamreza Haffari",
                "Trevor Cohn"
            ],
            "abstract": "Many NLP applications can be framed as a graph-to-sequence learning problem.\nPrevious work proposing neural architectures on this setting obtained promising\nresults compared to grammar-based approaches but still rely on linearisation\nheuristics and/or standard recurrent networks to achieve the best performance.\nIn this work, we propose a new model that encodes the full structural\ninformation contained in the graph. Our architecture couples the recently\nproposed Gated Graph Neural Networks with an input transformation that allows\nnodes and edges to have their own hidden representations, while tackling the\nparameter explosion problem present in previous work. Experimental results show\nthat our model outperforms strong baselines in generation from AMR graphs and\nsyntax-based neural machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10306v1",
            "date": "2018-06-27 05:50:05+00:00",
            "title": "Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR",
            "authors": [
                "Yerbolat Khassanov",
                "Eng Siong Chng"
            ],
            "abstract": "In automatic speech recognition (ASR) systems, recurrent neural network\nlanguage models (RNNLM) are used to rescore a word lattice or N-best hypotheses\nlist. Due to the expensive training, the RNNLM's vocabulary set accommodates\nonly small shortlist of most frequent words. This leads to suboptimal\nperformance if an input speech contains many out-of-shortlist (OOS) words. An\neffective solution is to increase the shortlist size and retrain the entire\nnetwork which is highly inefficient. Therefore, we propose an efficient method\nto expand the shortlist set of a pretrained RNNLM without incurring expensive\nretraining and using additional training data. Our method exploits the\nstructure of RNNLM which can be decoupled into three parts: input projection\nlayer, middle layers, and output projection layer. Specifically, our method\nexpands the word embedding matrices in projection layers and keeps the middle\nlayers unchanged. In this approach, the functionality of the pretrained RNNLM\nwill be correctly maintained as long as OOS words are properly modeled in two\nembedding spaces. We propose to model the OOS words by borrowing linguistic\nknowledge from appropriate in-shortlist words. Additionally, we propose to\ngenerate the list of OOS words to expand vocabulary in unsupervised manner by\nautomatically extracting them from ASR output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.10348v1",
            "date": "2018-06-27 08:58:57+00:00",
            "title": "Learning Visually-Grounded Semantics from Contrastive Adversarial Samples",
            "authors": [
                "Haoyue Shi",
                "Jiayuan Mao",
                "Tete Xiao",
                "Yuning Jiang",
                "Jian Sun"
            ],
            "abstract": "We study the problem of grounding distributional representations of texts on\nthe visual domain, namely visual-semantic embeddings (VSE for short). Begin\nwith an insightful adversarial attack on VSE embeddings, we show the limitation\nof current frameworks and image-text datasets (e.g., MS-COCO) both\nquantitatively and qualitatively. The large gap between the number of possible\nconstitutions of real-world semantics and the size of parallel data, to a large\nextent, restricts the model to establish the link between textual semantics and\nvisual concepts. We alleviate this problem by augmenting the MS-COCO image\ncaptioning datasets with textual contrastive adversarial samples. These samples\nare synthesized using linguistic rules and the WordNet knowledge base. The\nconstruction procedure is both syntax- and semantics-aware. The samples enforce\nthe model to ground learned embeddings to concrete concepts within the image.\nThis simple but powerful technique brings a noticeable improvement over the\nbaselines on a diverse set of downstream tasks, in addition to defending\nknown-type adversarial attacks. We release the codes at\nhttps://github.com/ExplorerFreda/VSE-C.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11316v1",
            "date": "2018-06-29 09:36:46+00:00",
            "title": "Fake News Identification on Twitter with Hybrid CNN and RNN Models",
            "authors": [
                "Oluwaseun Ajao",
                "Deepayan Bhowmik",
                "Shahrzad Zargari"
            ],
            "abstract": "The problem associated with the propagation of fake news continues to grow at\nan alarming scale. This trend has generated much interest from politics to\nacademia and industry alike. We propose a framework that detects and classifies\nfake news messages from Twitter posts using hybrid of convolutional neural\nnetworks and long-short term recurrent neural network models. The proposed work\nusing this deep learning approach achieves 82% accuracy. Our approach\nintuitively identifies relevant features associated with fake news stories\nwithout previous knowledge of the domain.",
            "journal ref": "Oluwaseun Ajao, Deepayan Bhowmik, and Shahrzad Zargari. 2018. Fake\n  News Identification on Twitter with Hybrid CNN and RNN Models. In Proceedings\n  of the International Conference on Social Media & Society, Copenhagen,\n  Denmark (SMSociety)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1806.11525v2",
            "date": "2018-06-29 16:40:29+00:00",
            "title": "Counting to Explore and Generalize in Text-based Games",
            "authors": [
                "Xingdi Yuan",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Alessandro Sordoni",
                "Romain Laroche",
                "Remi Tachet des Combes",
                "Matthew Hausknecht",
                "Adam Trischler"
            ],
            "abstract": "We propose a recurrent RL agent with an episodic exploration mechanism that\nhelps discovering good policies in text-based game environments. We show\npromising results on a set of generated text-based games of varying difficulty\nwhere the goal is to collect a coin located at the end of a chain of rooms. In\ncontrast to previous text-based RL approaches, we observe that our agent learns\npolicies that generalize to unseen games of greater difficulty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00303v1",
            "date": "2018-07-01 09:42:10+00:00",
            "title": "Modeling, comprehending and summarizing textual content by graphs",
            "authors": [
                "Vinicius Woloszyn",
                "Guilherme Medeiros Machado",
                "Leandro Krug Wives",
                "Jos\u00e9 Palazzo Moreira de Oliveira"
            ],
            "abstract": "Automatic Text Summarization strategies have been successfully employed to\ndigest text collections and extract its essential content. Usually, summaries\nare generated using textual corpora that belongs to the same domain area where\nthe summary will be used. Nonetheless, there are special cases where it is not\nfound enough textual sources, and one possible alternative is to generate a\nsummary from a different domain. One manner to summarize texts consists of\nusing a graph model. This model allows giving more importance to words\ncorresponding to the main concepts from the target domain found in the\nsummarized text. This gives the reader an overview of the main text concepts as\nwell as their relationships. However, this kind of summarization presents a\nsignificant number of repeated terms when compared to human-generated\nsummaries. In this paper, we present an approach to produce graph-model\nextractive summaries of texts, meeting the target domain exigences and treating\nthe terms repetition problem. To evaluate the proposition, we performed a\nseries of experiments showing that the proposed approach statistically improves\nthe performance of a model based on Graph Centrality, achieving better\ncoverage, accuracy, and recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00735v1",
            "date": "2018-06-27 11:44:35+00:00",
            "title": "hep-th",
            "authors": [
                "Yang-Hui He",
                "Vishnu Jejjala",
                "Brent D. Nelson"
            ],
            "abstract": "We apply techniques in natural language processing, computational\nlinguistics, and machine-learning to investigate papers in hep-th and four\nrelated sections of the arXiv: hep-ph, hep-lat, gr-qc, and math-ph. All of the\ntitles of papers in each of these sections, from the inception of the arXiv\nuntil the end of 2017, are extracted and treated as a corpus which we use to\ntrain the neural network Word2Vec. A comparative study of common n-grams,\nlinear syntactical identities, word cloud and word similarities is carried out.\nWe find notable scientific and sociological differences between the fields. In\nconjunction with support vector machines, we also show that the syntactic\nstructure of the titles in different sub-fields of high energy and mathematical\nphysics are sufficiently different that a neural network can perform a binary\nclassification of formal versus phenomenological sections with 87.1% accuracy,\nand can perform a finer five-fold classification across all sections with 65.1%\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00930v4",
            "date": "2018-07-02 23:54:48+00:00",
            "title": "Neural Random Projections for Language Modelling",
            "authors": [
                "Davide Nunes",
                "Luis Antunes"
            ],
            "abstract": "Neural network-based language models deal with data sparsity problems by\nmapping the large discrete space of words into a smaller continuous space of\nreal-valued vectors. By learning distributed vector representations for words,\neach training sample informs the neural network model about a combinatorial\nnumber of other patterns. In this paper, we exploit the sparsity in natural\nlanguage even further by encoding each unique input word using a fixed sparse\nrandom representation. These sparse codes are then projected onto a smaller\nembedding space which allows for the encoding of word occurrences from a\npossibly unknown vocabulary, along with the creation of more compact language\nmodels using a reduced number of parameters. We investigate the properties of\nour encoding mechanism empirically, by evaluating its performance on the widely\nused Penn Treebank corpus. We show that guaranteeing approximately equidistant\n(nearly orthogonal) vector representations for unique discrete inputs is enough\nto provide the neural network model with enough information to learn --and make\nuse-- of distributed representations for these inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00993v1",
            "date": "2018-07-03 06:36:48+00:00",
            "title": "Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation",
            "authors": [
                "Bin Wang",
                "Zhijian Ou"
            ],
            "abstract": "A new whole-sentence language model - neural trans-dimensional random field\nlanguage model (neural TRF LM), where sentences are modeled as a collection of\nrandom fields, and the potential function is defined by a neural network, has\nbeen introduced and successfully trained by noise-contrastive estimation (NCE).\nIn this paper, we extend NCE and propose dynamic noise-contrastive estimation\n(DNCE) to solve the two problems observed in NCE training. First, a dynamic\nnoise distribution is introduced and trained simultaneously to converge to the\ndata distribution. This helps to significantly cut down the noise sample number\nused in NCE and reduce the training cost. Second, DNCE discriminates between\nsentences generated from the noise distribution and sentences generated from\nthe interpolation of the data distribution and the noise distribution. This\nalleviates the overfitting problem caused by the sparseness of the training\nset. With DNCE, we can successfully and efficiently train neural TRF LMs on\nlarge corpus (about 0.8 billion words) with large vocabulary (about 568 K\nwords). Neural TRF LMs perform as good as LSTM LMs with less parameters and\nbeing 5x~114x faster in rescoring sentences. Interpolating neural TRF LMs with\nLSTM LMs and n-gram LMs can further reduce the error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01122v1",
            "date": "2018-07-03 12:38:11+00:00",
            "title": "Getting the subtext without the text: Scalable multimodal sentiment classification from visual and acoustic modalities",
            "authors": [
                "Nathaniel Blanchard",
                "Daniel Moreira",
                "Aparna Bharati",
                "Walter J. Scheirer"
            ],
            "abstract": "In the last decade, video blogs (vlogs) have become an extremely popular\nmethod through which people express sentiment. The ubiquitousness of these\nvideos has increased the importance of multimodal fusion models, which\nincorporate video and audio features with traditional text features for\nautomatic sentiment detection. Multimodal fusion offers a unique opportunity to\nbuild models that learn from the full depth of expression available to human\nviewers. In the detection of sentiment in these videos, acoustic and video\nfeatures provide clarity to otherwise ambiguous transcripts. In this paper, we\npresent a multimodal fusion model that exclusively uses high-level video and\naudio features to analyze spoken sentences for sentiment. We discard\ntraditional transcription features in order to minimize human intervention and\nto maximize the deployability of our model on at-scale real-world data. We\nselect high-level features for our model that have been successful in nonaffect\ndomains in order to test their generalizability in the sentiment detection\ndomain. We train and test our model on the newly released CMU Multimodal\nOpinion Sentiment and Emotion Intensity (CMUMOSEI) dataset, obtaining an F1\nscore of 0.8049 on the validation set and an F1 score of 0.6325 on the held-out\nchallenge test set.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.01270v5",
            "date": "2018-07-03 16:37:05+00:00",
            "title": "Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study",
            "authors": [
                "Tao Ge",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Neural sequence-to-sequence (seq2seq) approaches have proven to be successful\nin grammatical error correction (GEC). Based on the seq2seq framework, we\npropose a novel fluency boost learning and inference mechanism. Fluency\nboosting learning generates diverse error-corrected sentence pairs during\ntraining, enabling the error correction model to learn how to improve a\nsentence's fluency from more instances, while fluency boosting inference allows\nthe model to correct a sentence incrementally with multiple inference steps.\nCombining fluency boost learning and inference with convolutional seq2seq\nmodels, our approach achieves the state-of-the-art performance: 75.72 (F_{0.5})\non CoNLL-2014 10 annotation dataset and 62.42 (GLEU) on JFLEG test set\nrespectively, becoming the first GEC system that reaches human-level\nperformance (72.58 for CoNLL and 62.37 for JFLEG) on both of the benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01395v1",
            "date": "2018-07-03 23:20:49+00:00",
            "title": "Patient representation learning and interpretable evaluation using clinical notes",
            "authors": [
                "Madhumita Sushil",
                "Simon \u0160uster",
                "Kim Luyckx",
                "Walter Daelemans"
            ],
            "abstract": "We have three contributions in this work: 1. We explore the utility of a\nstacked denoising autoencoder and a paragraph vector model to learn\ntask-independent dense patient representations directly from clinical notes. To\nanalyze if these representations are transferable across tasks, we evaluate\nthem in multiple supervised setups to predict patient mortality, primary\ndiagnostic and procedural category, and gender. We compare their performance\nwith sparse representations obtained from a bag-of-words model. We observe that\nthe learned generalized representations significantly outperform the sparse\nrepresentations when we have few positive instances to learn from, and there is\nan absence of strong lexical features. 2. We compare the model performance of\nthe feature set constructed from a bag of words to that obtained from medical\nconcepts. In the latter case, concepts represent problems, treatments, and\ntests. We find that concept identification does not improve the classification\nperformance. 3. We propose novel techniques to facilitate model\ninterpretability. To understand and interpret the representations, we explore\nthe best encoded features within the patient representations obtained from the\nautoencoder model. Further, we calculate feature sensitivity across two\nnetworks to identify the most significant input features for different\nclassification tasks when we use these pretrained representations as the\nsupervised input. We successfully extract the most influential features for the\npipeline using this technique.",
            "journal ref": "Journal of Biomedical Informatics Volume 84C (2018) pp. 103-113",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01554v1",
            "date": "2018-07-04 13:07:53+00:00",
            "title": "Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding",
            "authors": [
                "Yutai Hou",
                "Yijia Liu",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "In this paper, we study the problem of data augmentation for language\nunderstanding in task-oriented dialogue system. In contrast to previous work\nwhich augments an utterance without considering its relation with other\nutterances, we propose a sequence-to-sequence generation based data\naugmentation framework that leverages one utterance's same semantic\nalternatives in the training data. A novel diversity rank is incorporated into\nthe utterance representation to make the model produce diverse utterances and\nthese diversely augmented utterances help to improve the language understanding\nmodule. Experimental results on the Airline Travel Information System dataset\nand a newly created semantic frame annotation on Stanford Multi-turn,\nMultidomain Dialogue Dataset show that our framework achieves significant\nimprovements of 6.38 and 10.04 F-scores respectively when only a training set\nof hundreds utterances is represented. Case studies also confirm that our\nmethod generates diverse utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01763v3",
            "date": "2018-07-04 20:13:31+00:00",
            "title": "Seq2RDF: An end-to-end application for deriving Triples from Natural Language Text",
            "authors": [
                "Yue Liu",
                "Tongtao Zhang",
                "Zhicheng Liang",
                "Heng Ji",
                "Deborah L. McGuinness"
            ],
            "abstract": "We present an end-to-end approach that takes unstructured textual input and\ngenerates structured output compliant with a given vocabulary. Inspired by\nrecent successes in neural machine translation, we treat the triples within a\ngiven knowledge graph as an independent graph language and propose an\nencoder-decoder framework with an attention mechanism that leverages knowledge\ngraph embeddings. Our model learns the mapping from natural language text to\ntriple representation in the form of subject-predicate-object using the\nselected knowledge graph vocabulary. Experiments on three different data sets\nshow that we achieve competitive F1-Measures over the baselines using our\nsimple yet effective approach. A demo video is included.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01836v1",
            "date": "2018-07-05 03:33:53+00:00",
            "title": "Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering",
            "authors": [
                "Vikas Yadav",
                "Rebecca Sharp",
                "Mihai Surdeanu"
            ],
            "abstract": "While increasingly complex approaches to question answering (QA) have been\nproposed, the true gain of these systems, particularly with respect to their\nexpensive training requirements, can be inflated when they are not compared to\nadequate baselines. Here we propose an unsupervised, simple, and fast alignment\nand information retrieval baseline that incorporates two novel contributions: a\n\\textit{one-to-many alignment} between query and document terms and\n\\textit{negative alignment} as a proxy for discriminative information. Our\napproach not only outperforms all conventional baselines as well as many\nsupervised recurrent neural networks, but also approaches the state of the art\nfor supervised systems on three QA datasets. With only three hyperparameters,\nwe achieve 47\\% P@1 on an 8th grade Science QA dataset, 32.9\\% P@1 on a Yahoo!\nanswers QA dataset and 64\\% MAP on WikiQA. We also achieve 26.56\\% and 58.36\\%\non ARC challenge and easy dataset respectively. In addition to including the\nadditional ARC results in this version of the paper, for the ARC easy set only\nwe also experimented with one additional parameter -- number of justifications\nretrieved.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.02162v1",
            "date": "2018-07-05 19:37:29+00:00",
            "title": "Feature Assisted bi-directional LSTM Model for Protein-Protein Interaction Identification from Biomedical Texts",
            "authors": [
                "Shweta Yadav",
                "Ankit Kumar",
                "Asif Ekbal",
                "Sriparna Saha",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Knowledge about protein-protein interactions is essential in understanding\nthe biological processes such as metabolic pathways, DNA replication, and\ntranscription etc. However, a majority of the existing Protein-Protein\nInteraction (PPI) systems are dependent primarily on the scientific literature,\nwhich is yet not accessible as a structured database. Thus, efficient\ninformation extraction systems are required for identifying PPI information\nfrom the large collection of biomedical texts. Most of the existing systems\nmodel the PPI extraction task as a classification problem and are tailored to\nthe handcrafted feature set including domain dependent features. In this paper,\nwe present a novel method based on deep bidirectional long short-term memory\n(B-LSTM) technique that exploits word sequences and dependency path related\ninformation to identify PPI information from text. This model leverages joint\nmodeling of proteins and relations in a single unified framework, which we name\nas Shortest Dependency Path B-LSTM (sdpLSTM) model. We perform experiments on\ntwo popular benchmark PPI datasets, namely AiMed & BioInfer. The evaluation\nshows the F1-score values of 86.45% and 77.35% on AiMed and BioInfer,\nrespectively. Comparisons with the existing systems show that our proposed\napproach attains state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.02221v2",
            "date": "2018-07-06 01:59:42+00:00",
            "title": "The Data Science of Hollywood: Using Emotional Arcs of Movies to Drive Business Model Innovation in Entertainment Industries",
            "authors": [
                "Marco Del Vecchio",
                "Alexander Kharlamov",
                "Glenn Parry",
                "Ganna Pogrebna"
            ],
            "abstract": "Much of business literature addresses the issues of consumer-centric design:\nhow can businesses design customized services and products which accurately\nreflect consumer preferences? This paper uses data science natural language\nprocessing methodology to explore whether and to what extent emotions shape\nconsumer preferences for media and entertainment content. Using a unique\nfiltered dataset of 6,174 movie scripts, we generate a mapping of screen\ncontent to capture the emotional trajectory of each motion picture. We then\ncombine the obtained mappings into clusters which represent groupings of\nconsumer emotional journeys. These clusters are used to predict overall success\nparameters of the movies including box office revenues, viewer satisfaction\nlevels (captured by IMDb ratings), awards, as well as the number of viewers'\nand critics' reviews. We find that like books all movie stories are dominated\nby 6 basic shapes. The highest box offices are associated with the Man in a\nHole shape which is characterized by an emotional fall followed by an emotional\nrise. This shape results in financially successful movies irrespective of genre\nand production budget. Yet, Man in a Hole succeeds not because it produces most\n\"liked\" movies but because it generates most \"talked about\" movies.\nInterestingly, a carefully chosen combination of production budget and genre\nmay produce a financially successful movie with any emotional shape.\nImplications of this analysis for generating on-demand content and for driving\nbusiness model innovation in entertainment industries are discussed.",
            "journal ref": "Journal of the Operational Research Society, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02383v1",
            "date": "2018-07-06 12:44:31+00:00",
            "title": "Natural Language Processing for Information Extraction",
            "authors": [
                "Sonit Singh"
            ],
            "abstract": "With rise of digital age, there is an explosion of information in the form of\nnews, articles, social media, and so on. Much of this data lies in unstructured\nform and manually managing and effectively making use of it is tedious, boring\nand labor intensive. This explosion of information and need for more\nsophisticated and efficient information handling tools gives rise to\nInformation Extraction(IE) and Information Retrieval(IR) technology.\nInformation Extraction systems takes natural language text as input and\nproduces structured information specified by certain criteria, that is relevant\nto a particular application. Various sub-tasks of IE such as Named Entity\nRecognition, Coreference Resolution, Named Entity Linking, Relation Extraction,\nKnowledge Base reasoning forms the building blocks of various high end Natural\nLanguage Processing (NLP) tasks such as Machine Translation, Question-Answering\nSystem, Natural Language Understanding, Text Summarization and Digital\nAssistants like Siri, Cortana and Google Now. This paper introduces Information\nExtraction technology, its various sub-tasks, highlights state-of-the-art\nresearch in various IE subtasks, current challenges and future research\ndirections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02658v1",
            "date": "2018-07-07 12:44:32+00:00",
            "title": "Robust and Scalable Differentiable Neural Computer for Question Answering",
            "authors": [
                "J\u00f6rg Franke",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "Deep learning models are often not easily adaptable to new tasks and require\ntask-specific adjustments. The differentiable neural computer (DNC), a\nmemory-augmented neural network, is designed as a general problem solver which\ncan be used in a wide range of tasks. But in reality, it is hard to apply this\nmodel to new tasks. We analyze the DNC and identify possible improvements\nwithin the application of question answering. This motivates a more robust and\nscalable DNC (rsDNC). The objective precondition is to keep the general\ncharacter of this model intact while making its application more reliable and\nspeeding up its required training time. The rsDNC is distinguished by a more\nrobust training, a slim memory unit and a bidirectional architecture. We not\nonly achieve new state-of-the-art performance on the bAbI task, but also\nminimize the performance variance between different initializations.\nFurthermore, we demonstrate the simplified applicability of the rsDNC to new\ntasks with passable results on the CNN RC task without adaptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03012v1",
            "date": "2018-07-09 09:43:23+00:00",
            "title": "Constructing a Word Similarity Graph from Vector based Word Representation for Named Entity Recognition",
            "authors": [
                "Miguel Feria",
                "Juan Paolo Balbin",
                "Francis Michael Bautista"
            ],
            "abstract": "In this paper, we discuss a method for identifying a seed word that would\nbest represent a class of named entities in a graphical representation of words\nand their similarities. Word networks, or word graphs, are representations of\nvectorized text where nodes are the words encountered in a corpus, and the\nweighted edges incident on the nodes represent how similar the words are to\neach other. We intend to build a bilingual word graph and identify seed words\nthrough community analysis that would be best used to segment a graph according\nto its named entities, therefore providing an unsupervised way of tagging named\nentities for a bilingual language base.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03053v1",
            "date": "2018-07-09 11:34:21+00:00",
            "title": "A deep learning approach for understanding natural language commands for mobile service robots",
            "authors": [
                "Pedro Henrique Martins",
                "Lu\u00eds Cust\u00f3dio",
                "Rodrigo Ventura"
            ],
            "abstract": "Using natural language to give instructions to robots is challenging, since\nnatural language understanding is still largely an open problem. In this paper\nwe address this problem by restricting our attention to commands modeled as one\naction, plus arguments (also known as slots). For action detection (also called\nintent detection) and slot filling various architectures of Recurrent Neural\nNetworks and Long Short Term Memory (LSTM) networks were evaluated, having\nLSTMs achieved a superior accuracy. As the action requested may not fall within\nthe robots capabilities, a Support Vector Machine(SVM) is used to determine\nwhether it is or not. For the input of the neural networks, several word\nembedding algorithms were compared. Finally, to implement the system in a\nrobot, a ROS package is created using a SMACH state machine. The proposed\nsystem is then evaluated both using well-known datasets and benchmarks in the\ncontext of domestic service robots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03399v1",
            "date": "2018-07-09 21:40:53+00:00",
            "title": "Jointly Embedding Entities and Text with Distant Supervision",
            "authors": [
                "Denis Newman-Griffis",
                "Albert M. Lai",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Learning representations for knowledge base entities and concepts is becoming\nincreasingly important for NLP applications. However, recent entity embedding\nmethods have relied on structured resources that are expensive to create for\nnew domains and corpora. We present a distantly-supervised method for jointly\nlearning embeddings of entities and text from an unnanotated corpus, using only\na list of mappings between entities and surface forms. We learn embeddings from\nopen-domain and biomedical corpora, and compare against prior methods that rely\non human-annotated text or large knowledge graph structure. Our embeddings\ncapture entity similarity and relatedness better than prior work, both in\nexisting biomedical datasets and a new Wikipedia-based dataset that we release\nto the community. Results on analogy completion and entity sense disambiguation\nindicate that entities and words capture complementary information that can be\neffectively combined for downstream use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03586v5",
            "date": "2018-07-10 12:10:16+00:00",
            "title": "Difficulty Controllable Generation of Reading Comprehension Questions",
            "authors": [
                "Yifan Gao",
                "Lidong Bing",
                "Wang Chen",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "We investigate the difficulty levels of questions in reading comprehension\ndatasets such as SQuAD, and propose a new question generation setting, named\nDifficulty-controllable Question Generation (DQG). Taking as input a sentence\nin the reading comprehension paragraph and some of its text fragments (i.e.,\nanswers) that we want to ask questions about, a DQG method needs to generate\nquestions each of which has a given text fragment as its answer, and meanwhile\nthe generation is under the control of specified difficulty labels---the output\nquestions should satisfy the specified difficulty as much as possible. To solve\nthis task, we propose an end-to-end framework to generate questions of\ndesignated difficulty levels by exploring a few important intuitions. For\nevaluation, we prepared the first dataset of reading comprehension questions\nwith difficulty labels. The results show that the question generated by our\nframework not only have better quality under the metrics like BLEU, but also\ncomply with the specified difficulty labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03658v1",
            "date": "2018-07-08 08:49:34+00:00",
            "title": "Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction",
            "authors": [
                "Xiangxi Shi",
                "Jianfei Cai",
                "Jiuxiang Gu",
                "Shafiq Joty"
            ],
            "abstract": "The explosion of video data on the internet requires effective and efficient\ntechnology to generate captions automatically for people who are not able to\nwatch the videos. Despite the great progress of video captioning research,\nparticularly on video feature encoding, the language decoder is still largely\nbased on the prevailing RNN decoder such as LSTM, which tends to prefer the\nfrequent word that aligns with the video. In this paper, we propose a\nboundary-aware hierarchical language decoder for video captioning, which\nconsists of a high-level GRU based language decoder, working as a global\n(caption-level) language model, and a low-level GRU based language decoder,\nworking as a local (phrase-level) language model. Most importantly, we\nintroduce a binary gate into the low-level GRU language decoder to detect the\nlanguage boundaries. Together with other advanced components including joint\nvideo prediction, shared soft attention, and boundary-aware video encoding, our\nintegrated video captioning framework can discover hierarchical language\ninformation and distinguish the subject and the object in a sentence, which are\nusually confusing during the language generation. Extensive experiments on two\nwidely-used video captioning datasets, MSR-Video-to-Text (MSR-VTT)\n\\cite{xu2016msr} and YouTube-to-Text (MSVD) \\cite{chen2011collecting} show that\nour method is highly competitive, compared with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.04441v2",
            "date": "2018-07-12 06:47:15+00:00",
            "title": "Tracking the Evolution of Words with Time-reflective Text Representations",
            "authors": [
                "Roberto Camacho Barranco",
                "Raimundo F. Dos Santos",
                "M. Shahriar Hossain"
            ],
            "abstract": "More than 80% of today's data is unstructured in nature, and these\nunstructured datasets evolve over time. A large part of these datasets are text\ndocuments generated by media outlets, scholarly articles in digital libraries,\nfindings from scientific and professional communities, and social media. Vector\nspace models were developed to analyze text data using data mining and machine\nlearning algorithms. While ample vector space models exist for text data, the\nevolutionary aspect of ever-changing text corpora is still missing in\nvector-based representations. The advent of word embeddings has enabled us to\ncreate a contextual vector space, but the embeddings fail to consider the\ntemporal aspects of the feature space successfully. This paper presents an\napproach to include temporal aspects in feature spaces. The inclusion of the\ntime aspect in the feature space provides vectors for every natural language\nelement, such as words or entities, at every timestamp. Such temporal word\nvectors allow us to track how the meaning of a word changes over time, by\nstudying the changes in its neighborhood. Moreover, a time-reflective text\nrepresentation will pave the way to a new set of text analytic abilities\ninvolving time series for text collections. In this paper, we present a\ntime-reflective vector space model for temporal text data that is able to\ncapture short and long-term changes in the meaning of words. We compare our\napproach with the limited literature on dynamic embeddings. We present\nqualitative and quantitative evaluations using the tracking of semantic\nevolution as the target application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05151v1",
            "date": "2018-07-13 15:51:31+00:00",
            "title": "New/s/leak 2.0 - Multilingual Information Extraction and Visualization for Investigative Journalism",
            "authors": [
                "Gregor Wiedemann",
                "Seid Muhie Yimam",
                "Chris Biemann"
            ],
            "abstract": "Investigative journalism in recent years is confronted with two major\nchallenges: 1) vast amounts of unstructured data originating from large text\ncollections such as leaks or answers to Freedom of Information requests, and 2)\nmulti-lingual data due to intensified global cooperation and communication in\npolitics, business and civil society. Faced with these challenges, journalists\nare increasingly cooperating in international networks. To support such\ncollaborations, we present the new version of new/s/leak 2.0, our open-source\nsoftware for content-based searching of leaks. It includes three novel main\nfeatures: 1) automatic language detection and language-dependent information\nextraction for 40 languages, 2) entity and keyword visualization for efficient\nexploration, and 3) decentral deployment for analysis of confidential data from\nvarious formats. We illustrate the new analysis capabilities with an exemplary\ncase study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05206v2",
            "date": "2018-07-13 17:44:20+00:00",
            "title": "Image Classification for Arabic: Assessing the Accuracy of Direct English to Arabic Translations",
            "authors": [
                "Abdulkareem Alsudais"
            ],
            "abstract": "Image classification is an ongoing research challenge. Most of the available\nresearch focuses on image classification for the English language, however\nthere is very little research on image classification for the Arabic language.\nExpanding image classification to Arabic has several applications. The present\nstudy investigated a method for generating Arabic labels for images of objects.\nThe method used in this study involved a direct English to Arabic translation\nof the labels that are currently available on ImageNet, a database commonly\nused in image classification research. The purpose of this study was to test\nthe accuracy of this method. In this study, 2,887 labeled images were randomly\nselected from ImageNet. All of the labels were translated from English to\nArabic using Google Translate. The accuracy of the translations was evaluated.\nResults indicated that that 65.6% of the Arabic labels were accurate. This\nstudy makes three important contributions to the image classification\nliterature: (1) it determined the baseline level of accuracy for algorithms\nthat provide Arabic labels for images, (2) it provided 1,895 images that are\ntagged with accurate Arabic labels, and (3) provided the accuracy of\ntranslations of image labels from English to Arabic.",
            "journal ref": "IEEE Access 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.05324v1",
            "date": "2018-07-14 03:24:31+00:00",
            "title": "Generating Synthetic Data for Neural Keyword-to-Question Models",
            "authors": [
                "Heng Ding",
                "Krisztian Balog"
            ],
            "abstract": "Search typically relies on keyword queries, but these are often semantically\nambiguous. We propose to overcome this by offering users natural language\nquestions, based on their keyword queries, to disambiguate their intent. This\nkeyword-to-question task may be addressed using neural machine translation\ntechniques. Neural translation models, however, require massive amounts of\ntraining data (keyword-question pairs), which is unavailable for this task. The\nmain idea of this paper is to generate large amounts of synthetic training data\nfrom a small seed set of hand-labeled keyword-question pairs. Since natural\nlanguage questions are available in large quantities, we develop models to\nautomatically generate the corresponding keyword queries. Further, we introduce\nvarious filtering mechanisms to ensure that synthetic training data is of high\nquality. We demonstrate the feasibility of our approach using both automatic\nand manual evaluation. This is an extended version of the article published\nwith the same title in the Proceedings of ICTIR'18.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.05574v1",
            "date": "2018-07-15 16:49:06+00:00",
            "title": "WordNet-Based Information Retrieval Using Common Hypernyms and Combined Features",
            "authors": [
                "Vuong M. Ngo",
                "Tru H. Cao",
                "Tuan M. V. Le"
            ],
            "abstract": "Text search based on lexical matching of keywords is not satisfactory due to\npolysemous and synonymous words. Semantic search that exploits word meanings,\nin general, improves search performance. In this paper, we survey WordNet-based\ninformation retrieval systems, which employ a word sense disambiguation method\nto process queries and documents. The problem is that in many cases a word has\nmore than one possible direct sense, and picking only one of them may give a\nwrong sense for the word. Moreover, the previous systems use only word forms to\nrepresent word senses and their hypernyms. We propose a novel approach that\nuses the most specific common hypernym of the remaining undisambiguated\nmulti-senses of a word, as well as combined WordNet features to represent word\nmeanings. Experiments on a benchmark dataset show that, in terms of the MAP\nmeasure, our search engine is 17.7% better than the lexical search, and at\nleast 9.4% better than all surveyed search systems using WordNet.\n  Keywords Ontology, word sense disambiguation, semantic annotation, semantic\nsearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05642v1",
            "date": "2018-07-16 00:50:00+00:00",
            "title": "LATE Ain'T Earley: A Faster Parallel Earley Parser",
            "authors": [
                "Peter Ahrens",
                "John Feser",
                "Robin Hui"
            ],
            "abstract": "We present the LATE algorithm, an asynchronous variant of the Earley\nalgorithm for parsing context-free grammars. The Earley algorithm is naturally\ntask-based, but is difficult to parallelize because of dependencies between the\ntasks. We present the LATE algorithm, which uses additional data structures to\nmaintain information about the state of the parse so that work items may be\nprocessed in any order. This property allows the LATE algorithm to be sped up\nusing task parallelism. We show that the LATE algorithm can achieve a 120x\nspeedup over the Earley algorithm on a natural language task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06107v1",
            "date": "2018-07-16 21:01:52+00:00",
            "title": "Don't get Lost in Negation: An Effective Negation Handled Dialogue Acts Prediction Algorithm for Twitter Customer Service Conversations",
            "authors": [
                "Mansurul Bhuiyan",
                "Amita Misra",
                "Saurabh Tripathy",
                "Jalal Mahmud",
                "Rama Akkiraju"
            ],
            "abstract": "In the last several years, Twitter is being adopted by the companies as an\nalternative platform to interact with the customers to address their concerns.\nWith the abundance of such unconventional conversation resources, push for\ndeveloping effective virtual agents is more than ever. To address this\nchallenge, a better understanding of such customer service conversations is\nrequired. Lately, there have been several works proposing a novel taxonomy for\nfine-grained dialogue acts as well as develop algorithms for automatic\ndetection of these acts. The outcomes of these works are providing stepping\nstones for the ultimate goal of building efficient and effective virtual\nagents. But none of these works consider handling the notion of negation into\nthe proposed algorithms. In this work, we developed an SVM-based dialogue acts\nprediction algorithm for Twitter customer service conversations where negation\nhandling is an integral part of the end-to-end solution. For negation handling,\nwe propose several efficient heuristics as well as adopt recent state-of- art\nthird party machine learning based solutions. Empirically we show model's\nperformance gain while handling negation compared to when we don't. Our\nexperiments show that for the informal text such as tweets, the heuristic-based\napproach is more effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06270v1",
            "date": "2018-07-17 08:13:19+00:00",
            "title": "Bench-Marking Information Extraction in Semi-Structured Historical Handwritten Records",
            "authors": [
                "Animesh Prasad",
                "Herv\u00e9 D\u00e9jean",
                "Jean-Luc Meunier",
                "Max Weidemann",
                "Johannes Michael",
                "Gundram Leifert"
            ],
            "abstract": "In this report, we present our findings from benchmarking experiments for\ninformation extraction on historical handwritten marriage records Esposalles\nfrom IEHHR - ICDAR 2017 robust reading competition. The information extraction\nis modeled as semantic labeling of the sequence across 2 set of labels. This\ncan be achieved by sequentially or jointly applying handwritten text\nrecognition (HTR) and named entity recognition (NER). We deploy a pipeline\napproach where first we use state-of-the-art HTR and use its output as input\nfor NER. We show that given low resource setup and simple structure of the\nrecords, high performance of HTR ensures overall high performance. We explore\nthe various configurations of conditional random fields and neural networks to\nbenchmark NER on given certain noisy input. The best model on 10-fold\ncross-validation as well as blind test data uses n-gram features with\nbidirectional long short-term memory.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.06638v1",
            "date": "2018-07-17 19:40:28+00:00",
            "title": "Developing a Portable Natural Language Processing Based Phenotyping System",
            "authors": [
                "Himanshu Sharma",
                "Chengsheng Mao",
                "Yizhen Zhang",
                "Haleh Vatani",
                "Liang Yao",
                "Yizhen Zhong",
                "Luke Rasmussen",
                "Guoqian Jiang",
                "Jyotishman Pathak",
                "Yuan Luo"
            ],
            "abstract": "This paper presents a portable phenotyping system that is capable of\nintegrating both rule-based and statistical machine learning based approaches.\nOur system utilizes UMLS to extract clinically relevant features from the\nunstructured text and then facilitates portability across different\ninstitutions and data systems by incorporating OHDSI's OMOP Common Data Model\n(CDM) to standardize necessary data elements. Our system can also store the key\ncomponents of rule-based systems (e.g., regular expression matches) in the\nformat of OMOP CDM, thus enabling the reuse, adaptation and extension of many\nexisting rule-based clinical NLP systems. We experimented with our system on\nthe corpus from i2b2's Obesity Challenge as a pilot study. Our system\nfacilitates portable phenotyping of obesity and its 15 comorbidities based on\nthe unstructured patient discharge summaries, while achieving a performance\nthat often ranked among the top 10 of the challenge participants. This\nstandardization enables a consistent application of numerous rule-based and\nmachine learning based classification techniques downstream.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06926v1",
            "date": "2018-07-18 13:41:57+00:00",
            "title": "Fake news as we feel it: perception and conceptualization of the term \"fake news\" in the media",
            "authors": [
                "Evandro Cunha",
                "Gabriel Magno",
                "Josemar Caetano",
                "Douglas Teixeira",
                "Virgilio Almeida"
            ],
            "abstract": "In this article, we quantitatively analyze how the term \"fake news\" is being\nshaped in news media in recent years. We study the perception and the\nconceptualization of this term in the traditional media using eight years of\ndata collected from news outlets based in 20 countries. Our results not only\ncorroborate previous indications of a high increase in the usage of the\nexpression \"fake news\", but also show contextual changes around this expression\nafter the United States presidential election of 2016. Among other results, we\nfound changes in the related vocabulary, in the mentioned entities, in the\nsurrounding topics and in the contextual polarity around the term \"fake news\",\nsuggesting that this expression underwent a change in perception and\nconceptualization after 2016. These outcomes expand the understandings on the\nusage of the term \"fake news\", helping to comprehend and more accurately\ncharacterize this relevant social phenomenon linked to misinformation and\nmanipulation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07186v1",
            "date": "2018-07-18 23:38:08+00:00",
            "title": "Evaluating Word Embeddings in Multi-label Classification Using Fine-grained Name Typing",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Katharina Kann",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Embedding models typically associate each word with a single real-valued\nvector, representing its different properties. Evaluation methods, therefore,\nneed to analyze the accuracy and completeness of these properties in\nembeddings. This requires fine-grained analysis of embedding subspaces.\nMulti-label classification is an appropriate way to do so. We propose a new\nevaluation method for word embeddings based on multi-label classification given\na word embedding. The task we use is fine-grained name typing: given a large\ncorpus, find all types that a name can refer to based on the name embedding.\nGiven the scale of entities in knowledge bases, we can build datasets for this\ntask that are complementary to the current embedding evaluation datasets in:\nthey are very large, contain fine-grained classes, and allow the direct\nevaluation of embeddings without confounding factors like sentence context",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07351v1",
            "date": "2018-07-19 11:44:10+00:00",
            "title": "Can We Assess Mental Health through Social Media and Smart Devices? Addressing Bias in Methodology and Evaluation",
            "authors": [
                "Adam Tsakalidis",
                "Maria Liakata",
                "Theo Damoulas",
                "Alexandra I. Cristea"
            ],
            "abstract": "Predicting mental health from smartphone and social media data on a\nlongitudinal basis has recently attracted great interest, with very promising\nresults being reported across many studies. Such approaches have the potential\nto revolutionise mental health assessment, if their development and evaluation\nfollows a real world deployment setting. In this work we take a closer look at\nstate-of-the-art approaches, using different mental health datasets and\nindicators, different feature sources and multiple simulations, in order to\nassess their ability to generalise. We demonstrate that under a pragmatic\nevaluation framework, none of the approaches deliver or even approach the\nreported performances. In fact, we show that current state-of-the-art\napproaches can barely outperform the most na\\\"ive baselines in the real-world\nsetting, posing serious questions not only about their deployment ability, but\nalso about the contribution of the derived features for the mental health\nassessment task and how to make better use of such data in the future.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1807.07517v1",
            "date": "2018-07-19 16:14:08+00:00",
            "title": "Using Deep Neural Networks to Translate Multi-lingual Threat Intelligence",
            "authors": [
                "Priyanka Ranade",
                "Sudip Mittal",
                "Anupam Joshi",
                "Karuna Joshi"
            ],
            "abstract": "The multilingual nature of the Internet increases complications in the\ncybersecurity community's ongoing efforts to strategically mine threat\nintelligence from OSINT data on the web. OSINT sources such as social media,\nblogs, and dark web vulnerability markets exist in diverse languages and hinder\nsecurity analysts, who are unable to draw conclusions from intelligence in\nlanguages they don't understand. Although third party translation engines are\ngrowing stronger, they are unsuited for private security environments. First,\nsensitive intelligence is not a permitted input to third party engines due to\nprivacy and confidentiality policies. In addition, third party engines produce\ngeneralized translations that tend to lack exclusive cybersecurity terminology.\nIn this paper, we address these issues and describe our system that enables\nthreat intelligence understanding across unfamiliar languages. We create a\nneural network based system that takes in cybersecurity data in a different\nlanguage and outputs the respective English translation. The English\ntranslation can then be understood by an analyst, and can also serve as input\nto an AI based cyber-defense system that can take mitigative action. As a proof\nof concept, we have created a pipeline which takes Russian threats and\ngenerates its corresponding English, RDF, and vectorized representations. Our\nnetwork optimizes translations on specifically, cybersecurity data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07828v2",
            "date": "2018-07-20 13:18:15+00:00",
            "title": "Towards Functorial Language-Games",
            "authors": [
                "Jules Hedges",
                "Martha Lewis"
            ],
            "abstract": "In categorical compositional semantics of natural language one studies\nfunctors from a category of grammatical derivations (such as a Lambek pregroup)\nto a semantic category (such as real vector spaces). We compositionally build\ngame-theoretic semantics of sentences by taking the semantic category to be the\ncategory whose morphisms are open games. This requires some modifications to\nthe grammar category to compensate for the failure of open games to form a\ncompact closed category. We illustrate the theory using simple examples of\nWittgenstein's language-games.",
            "journal ref": "EPTCS 283, 2018, pp. 89-102",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1807.07961v2",
            "date": "2018-07-20 04:09:08+00:00",
            "title": "Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM",
            "authors": [
                "Yuxiao Chen",
                "Jianbo Yuan",
                "Quanzeng You",
                "Jiebo Luo"
            ],
            "abstract": "Sentiment analysis on large-scale social media data is important to bridge\nthe gaps between social media contents and real world activities including\npolitical election prediction, individual and public emotional status\nmonitoring and analysis, and so on. Although textual sentiment analysis has\nbeen well studied based on platforms such as Twitter and Instagram, analysis of\nthe role of extensive emoji uses in sentiment analysis remains light. In this\npaper, we propose a novel scheme for Twitter sentiment analysis with extra\nattention on emojis. We first learn bi-sense emoji embeddings under positive\nand negative sentimental tweets individually, and then train a sentiment\nclassifier by attending on these bi-sense emoji embeddings with an\nattention-based long short-term memory network (LSTM). Our experiments show\nthat the bi-sense embedding is effective for extracting sentiment-aware\nembeddings of emojis and outperforms the state-of-the-art models. We also\nvisualize the attentions to show that the bi-sense emoji embedding provides\nbetter guidance on the attention mechanism to obtain a more robust\nunderstanding of the semantics and sentiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07964v1",
            "date": "2018-07-20 07:35:43+00:00",
            "title": "Question-Aware Sentence Gating Networks for Question and Answering",
            "authors": [
                "Minjeong Kim",
                "David Keetae Park",
                "Hyungjong Noh",
                "Yeonsoo Lee",
                "Jaegul Choo"
            ],
            "abstract": "Machine comprehension question answering, which finds an answer to the\nquestion given a passage, involves high-level reasoning processes of\nunderstanding and tracking the relevant contents across various semantic units\nsuch as words, phrases, and sentences in a document. This paper proposes the\nnovel question-aware sentence gating networks that directly incorporate the\nsentence-level information into word-level encoding processes. To this end, our\nmodel first learns question-aware sentence representations and then dynamically\ncombines them with word-level representations, resulting in semantically\nmeaningful word representations for QA tasks. Experimental results demonstrate\nthat our approach consistently improves the accuracy over existing baseline\napproaches on various QA datasets and bears the wide applicability to other\nneural network-based QA models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08074v1",
            "date": "2018-07-21 03:12:36+00:00",
            "title": "ScoutBot: A Dialogue System for Collaborative Navigation",
            "authors": [
                "Stephanie M. Lukin",
                "Felix Gervits",
                "Cory J. Hayes",
                "Anton Leuski",
                "Pooja Moolchandani",
                "John G. Rogers III",
                "Carlos Sanchez Amaro",
                "Matthew Marge",
                "Clare R. Voss",
                "David Traum"
            ],
            "abstract": "ScoutBot is a dialogue interface to physical and simulated robots that\nsupports collaborative exploration of environments. The demonstration will\nallow users to issue unconstrained spoken language commands to ScoutBot.\nScoutBot will prompt for clarification if the user's instruction needs\nadditional input. It is trained on human-robot dialogue collected from\nWizard-of-Oz experiments, where robot responses were initiated by a human\nwizard in previous interactions. The demonstration will show a simulated ground\nrobot (Clearpath Jackal) in a simulated environment supported by ROS (Robot\nOperating System).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08204v1",
            "date": "2018-07-21 20:48:53+00:00",
            "title": "Towards Neural Theorem Proving at Scale",
            "authors": [
                "Pasquale Minervini",
                "Matko Bosnjak",
                "Tim Rockt\u00e4schel",
                "Sebastian Riedel"
            ],
            "abstract": "Neural models combining representation learning and reasoning in an\nend-to-end trainable manner are receiving increasing interest. However, their\nuse is severely limited by their computational complexity, which renders them\nunusable on real world datasets. We focus on the Neural Theorem Prover (NTP)\nmodel proposed by Rockt{\\\"{a}}schel and Riedel (2017), a continuous relaxation\nof the Prolog backward chaining algorithm where unification between terms is\nreplaced by the similarity between their embedding representations. For\nanswering a given query, this model needs to consider all possible proof paths,\nand then aggregate results - this quickly becomes infeasible even for small\nKnowledge Bases (KBs). We observe that we can accurately approximate the\ninference process in this model by considering only proof paths associated with\nthe highest proof scores. This enables inference and learning on previously\nimpracticable KBs.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1807.08435v1",
            "date": "2018-07-23 06:01:44+00:00",
            "title": "Question Relevance in Visual Question Answering",
            "authors": [
                "Prakruthi Prabhakar",
                "Nitish Kulkarni",
                "Linghao Zhang"
            ],
            "abstract": "Free-form and open-ended Visual Question Answering systems solve the problem\nof providing an accurate natural language answer to a question pertaining to an\nimage. Current VQA systems do not evaluate if the posed question is relevant to\nthe input image and hence provide nonsensical answers when posed with\nirrelevant questions to an image. In this paper, we solve the problem of\nidentifying the relevance of the posed question to an image. We address the\nproblem as two sub-problems. We first identify if the question is visual or\nnot. If the question is visual, we then determine if it's relevant to the image\nor not. For the second problem, we generate a large dataset from existing\nvisual question answering datasets in order to enable the training of complex\narchitectures and model the relevance of a visual question to an image. We also\ncompare the results of our Long Short-Term Memory Recurrent Neural Network\nbased models to Logistic Regression, XGBoost and multi-layer perceptron based\napproaches to the problem.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.08666v1",
            "date": "2018-07-23 15:14:32+00:00",
            "title": "ASR-free CNN-DTW keyword spotting using multilingual bottleneck features for almost zero-resource languages",
            "authors": [
                "Raghav Menon",
                "Herman Kamper",
                "Emre Yilmaz",
                "John Quinn",
                "Thomas Niesler"
            ],
            "abstract": "We consider multilingual bottleneck features (BNFs) for nearly zero-resource\nkeyword spotting. This forms part of a United Nations effort using keyword\nspotting to support humanitarian relief programmes in parts of Africa where\nlanguages are severely under-resourced. We use 1920 isolated keywords (40\ntypes, 34 minutes) as exemplars for dynamic time warping (DTW) template\nmatching, which is performed on a much larger body of untranscribed speech.\nThese DTW costs are used as targets for a convolutional neural network (CNN)\nkeyword spotter, giving a much faster system than direct DTW. Here we consider\nhow available data from well-resourced languages can improve this CNN-DTW\napproach. We show that multilingual BNFs trained on ten languages improve the\narea under the ROC curve of a CNN-DTW system by 10.9% absolute relative to the\nMFCC baseline. By combining low-resource DTW-based supervision with information\nfrom well-resourced languages, CNN-DTW is a competitive option for low-resource\nkeyword spotting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08669v1",
            "date": "2018-07-23 15:17:04+00:00",
            "title": "Automatic Speech Recognition for Humanitarian Applications in Somali",
            "authors": [
                "Raghav Menon",
                "Astik Biswas",
                "Armin Saeb",
                "John Quinn",
                "Thomas Niesler"
            ],
            "abstract": "We present our first efforts in building an automatic speech recognition\nsystem for Somali, an under-resourced language, using 1.57 hrs of annotated\nspeech for acoustic model training. The system is part of an ongoing effort by\nthe United Nations (UN) to implement keyword spotting systems supporting\nhumanitarian relief programmes in parts of Africa where languages are severely\nunder-resourced. We evaluate several types of acoustic model, including recent\nneural architectures. Language model data augmentation using a combination of\nrecurrent neural networks (RNN) and long short-term memory neural networks\n(LSTMs) as well as the perturbation of acoustic data are also considered. We\nfind that both types of data augmentation are beneficial to performance, with\nour best system using a combination of convolutional neural networks (CNNs),\ntime-delay neural networks (TDNNs) and bi-directional long short term memory\n(BLSTMs) to achieve a word error rate of 53.75%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09434v1",
            "date": "2018-07-25 04:34:17+00:00",
            "title": "Distinctive-attribute Extraction for Image Captioning",
            "authors": [
                "Boeun Kim",
                "Young Han Lee",
                "Hyedong Jung",
                "Choongsang Cho"
            ],
            "abstract": "Image captioning, an open research issue, has been evolved with the progress\nof deep neural networks. Convolutional neural networks (CNNs) and recurrent\nneural networks (RNNs) are employed to compute image features and generate\nnatural language descriptions in the research. In previous works, a caption\ninvolving semantic description can be generated by applying additional\ninformation into the RNNs. In this approach, we propose a distinctive-attribute\nextraction (DaE) which explicitly encourages significant meanings to generate\nan accurate caption describing the overall meaning of the image with their\nunique situation. Specifically, the captions of training images are analyzed by\nterm frequency-inverse document frequency (TF-IDF), and the analyzed semantic\ninformation is trained to extract distinctive-attributes for inferring\ncaptions. The proposed scheme is evaluated on a challenge data, and it improves\nan objective performance while describing images in more detail.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1807.09875v2",
            "date": "2018-07-25 21:42:55+00:00",
            "title": "Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder",
            "authors": [
                "Caio Corro",
                "Ivan Titov"
            ],
            "abstract": "Human annotation for syntactic parsing is expensive, and large resources are\navailable only for a fraction of languages. A question we ask is whether one\ncan leverage abundant unlabeled texts to improve syntactic parsers, beyond just\nusing the texts to obtain more generalisable lexical features (i.e. beyond word\nembeddings). To this end, we propose a novel latent-variable generative model\nfor semi-supervised syntactic dependency parsing. As exact inference is\nintractable, we introduce a differentiable relaxation to obtain approximate\nsamples and compute gradients with respect to the parser parameters. Our method\n(Differentiable Perturb-and-Parse) relies on differentiable dynamic programming\nover stochastically perturbed edge scores. We demonstrate effectiveness of our\napproach with experiments on English, French and Swedish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10104v1",
            "date": "2018-07-26 13:11:51+00:00",
            "title": "Term Set Expansion based on Multi-Context Term Embeddings: an End-to-end Workflow",
            "authors": [
                "Jonathan Mamou",
                "Oren Pereg",
                "Moshe Wasserblat",
                "Ido Dagan",
                "Yoav Goldberg",
                "Alon Eirew",
                "Yael Green",
                "Shira Guskin",
                "Peter Izsak",
                "Daniel Korat"
            ],
            "abstract": "We present SetExpander, a corpus-based system for expanding a seed set of\nterms into a more complete set of terms that belong to the same semantic class.\nSetExpander implements an iterative end-to end workflow for term set expansion.\nIt enables users to easily select a seed set of terms, expand it, view the\nexpanded set, validate it, re-expand the validated set and store it, thus\nsimplifying the extraction of domain-specific fine-grained semantic classes.\nSetExpander has been used for solving real-life use cases including integration\nin an automated recruitment system and an issues and defects resolution system.\nA video demo of SetExpander is available at\nhttps://drive.google.com/open?id=1e545bB87Autsch36DjnJHmq3HWfSd1Rv (some images\nwere blurred for privacy reasons).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1807.10615v1",
            "date": "2018-07-25 08:36:02+00:00",
            "title": "Judging a Book by its Description : Analyzing Gender Stereotypes in the Man Bookers Prize Winning Fiction",
            "authors": [
                "Nishtha Madaan",
                "Sameep Mehta",
                "Shravika Mittal",
                "Ashima Suvarna"
            ],
            "abstract": "The presence of gender stereotypes in many aspects of society is a well-known\nphenomenon. In this paper, we focus on studying and quantifying such\nstereotypes and bias in the Man Bookers Prize winning fiction. We consider 275\nbooks shortlisted for Man Bookers Prize between 1969 and 2017. The gender bias\nis analyzed by semantic modeling of book descriptions on Goodreads. This\nreveals the pervasiveness of gender bias and stereotype in the books on\ndifferent features like occupation, introductions and actions associated to the\ncharacters in the book.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10805v1",
            "date": "2018-07-27 19:07:33+00:00",
            "title": "Improving Neural Sequence Labelling using Additional Linguistic Information",
            "authors": [
                "Mahtab Ahmed",
                "Muhammad Rifayat Samee",
                "Robert E. Mercer"
            ],
            "abstract": "Sequence labelling is the task of assigning categorical labels to a data\nsequence. In Natural Language Processing, sequence labelling can be applied to\nvarious fundamental problems, such as Part of Speech (POS) tagging, Named\nEntity Recognition (NER), and Chunking. In this study, we propose a method to\nadd various linguistic features to the neural sequence framework to improve\nsequence labelling. Besides word level knowledge, sense embeddings are added to\nprovide semantic information. Additionally, selective readings of character\nembeddings are added to capture contextual as well as morphological features\nfor each word in a sentence. Compared to previous methods, these added\nlinguistic features allow us to design a more concise model and perform more\nefficient training. Our proposed architecture achieves state of the art results\non the benchmark datasets of POS, NER, and chunking. Moreover, the convergence\nrate of our model is significantly better than the previous state of the art\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10965v2",
            "date": "2018-07-28 18:26:28+00:00",
            "title": "Ontology-Grounded Topic Modeling for Climate Science Research",
            "authors": [
                "Jennifer Sleeman",
                "Tim Finin",
                "Milton Halem"
            ],
            "abstract": "In scientific disciplines where research findings have a strong impact on\nsociety, reducing the amount of time it takes to understand, synthesize and\nexploit the research is invaluable. Topic modeling is an effective technique\nfor summarizing a collection of documents to find the main themes among them\nand to classify other documents that have a similar mixture of co-occurring\nwords. We show how grounding a topic model with an ontology, extracted from a\nglossary of important domain phrases, improves the topics generated and makes\nthem easier to understand. We apply and evaluate this method to the climate\nscience domain. The result improves the topics generated and supports faster\nresearch understanding, discovery of social networks among researchers, and\nautomatic ontology generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11227v4",
            "date": "2018-07-30 08:28:44+00:00",
            "title": "YouTube AV 50K: An Annotated Corpus for Comments in Autonomous Vehicles",
            "authors": [
                "Tao Li",
                "Lei Lin",
                "Minsoo Choi",
                "Kaiming Fu",
                "Siyuan Gong",
                "Jian Wang"
            ],
            "abstract": "With one billion monthly viewers, and millions of users discussing and\nsharing opinions, comments below YouTube videos are rich sources of data for\nopinion mining and sentiment analysis. We introduce the YouTube AV 50K dataset,\na freely-available collections of more than 50,000 YouTube comments and\nmetadata below autonomous vehicle (AV)-related videos. We describe its creation\nprocess, its content and data format, and discuss its possible usages.\nEspecially, we do a case study of the first self-driving car fatality to\nevaluate the dataset, and show how we can use this dataset to better understand\npublic attitudes toward self-driving cars and public reactions to the accident.\nFuture developments of the dataset are also discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11567v1",
            "date": "2018-07-27 15:31:15+00:00",
            "title": "Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems",
            "authors": [
                "Seonghan Ryu",
                "Seokhwan Kim",
                "Junhwi Choi",
                "Hwanjo Yu",
                "Gary Geunbae Lee"
            ],
            "abstract": "To ensure satisfactory user experience, dialog systems must be able to\ndetermine whether an input sentence is in-domain (ID) or out-of-domain (OOD).\nWe assume that only ID sentences are available as training data because\ncollecting enough OOD sentences in an unbiased way is a laborious and\ntime-consuming job. This paper proposes a novel neural sentence embedding\nmethod that represents sentences in a low-dimensional continuous vector space\nthat emphasizes aspects that distinguish ID cases from OOD cases. We first used\na large set of unlabeled text to pre-train word representations that are used\nto initialize neural sentence embedding. Then we used domain-category analysis\nas an auxiliary task to train neural sentence embedding for OOD sentence\ndetection. After the sentence representations were learned, we used them to\ntrain an autoencoder aimed at OOD sentence detection. We evaluated our method\nby experimentally comparing it to the state-of-the-art methods in an\neight-domain dialog system; our proposed method achieved the highest accuracy\nin all tests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11761v1",
            "date": "2018-07-31 11:18:18+00:00",
            "title": "A First Experiment on Including Text Literals in KGloVe",
            "authors": [
                "Michael Cochez",
                "Martina Garofalo",
                "J\u00e9r\u00f4me Len\u00dfen",
                "Maria Angela Pellegrino"
            ],
            "abstract": "Graph embedding models produce embedding vectors for entities and relations\nin Knowledge Graphs, often without taking literal properties into account. We\nshow an initial idea based on the combination of global graph structure with\nadditional information provided by textual information in properties. Our\ninitial experiment shows that this approach might be useful, but does not\nclearly outperform earlier approaches when evaluated on machine learning tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.00088v1",
            "date": "2018-08-31 23:53:18+00:00",
            "title": "Hierarchical CVAE for Fine-Grained Hate Speech Classification",
            "authors": [
                "Jing Qian",
                "Mai ElSherief",
                "Elizabeth Belding",
                "William Yang Wang"
            ],
            "abstract": "Existing work on automated hate speech detection typically focuses on binary\nclassification or on differentiating among a small set of categories. In this\npaper, we propose a novel method on a fine-grained hate speech classification\ntask, which focuses on differentiating among 40 hate groups of 13 different\nhate group categories. We first explore the Conditional Variational Autoencoder\n(CVAE) as a discriminative model and then extend it to a hierarchical\narchitecture to utilize the additional hate category information for more\naccurate prediction. Experimentally, we show that incorporating the hate\ncategory information for training can significantly improve the classification\nperformance and our proposed model outperforms commonly-used discriminative\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00252v2",
            "date": "2018-09-01 21:12:09+00:00",
            "title": "Parameter Sharing Methods for Multilingual Self-Attentional Translation Models",
            "authors": [
                "Devendra Singh Sachan",
                "Graham Neubig"
            ],
            "abstract": "In multilingual neural machine translation, it has been shown that sharing a\nsingle translation model between multiple languages can achieve competitive\nperformance, sometimes even leading to performance gains over bilingually\ntrained models. However, these improvements are not uniform; often multilingual\nparameter sharing results in a decrease in accuracy due to translation models\nnot being able to accommodate different languages in their limited parameter\nspace. In this work, we examine parameter sharing techniques that strike a\nhappy medium between full sharing and individual training, specifically\nfocusing on the self-attentional Transformer model. We find that the full\nparameter sharing approach leads to increases in BLEU scores mainly when the\ntarget languages are from a similar language family. However, even in the case\nwhere target languages are from different families where full parameter sharing\nleads to a noticeable drop in BLEU scores, our proposed methods for partial\nsharing of parameters can lead to substantial improvements in translation\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00329v1",
            "date": "2018-09-02 12:01:27+00:00",
            "title": "Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet",
            "authors": [
                "Yafang Huang",
                "Hai Zhao"
            ],
            "abstract": "Chinese pinyin input method engine (IME) converts pinyin into character so\nthat Chinese characters can be conveniently inputted into computer through\ncommon keyboard. IMEs work relying on its core component, pinyin-to-character\nconversion (P2C). Usually Chinese IMEs simply predict a list of character\nsequences for user choice only according to user pinyin input at each turn.\nHowever, Chinese inputting is a multi-turn online procedure, which can be\nsupposed to be exploited for further user experience promoting. This paper thus\nfor the first time introduces a sequence-to-sequence model with gated-attention\nmechanism for the core task in IMEs. The proposed neural P2C model is learned\nby encoding previous input utterance as extra context to enable our IME capable\nof predicting character sequence with incomplete pinyin input. Our model is\nevaluated in different benchmark datasets showing great user experience\nimprovement compared to traditional models, which demonstrates the first\nengineering practice of building Chinese aided IME.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00339v1",
            "date": "2018-09-02 14:03:30+00:00",
            "title": "Chittron: An Automatic Bangla Image Captioning System",
            "authors": [
                "Motiur Rahman",
                "Nabeel Mohammed",
                "Nafees Mansoor",
                "Sifat Momen"
            ],
            "abstract": "Automatic image caption generation aims to produce an accurate description of\nan image in natural language automatically. However, Bangla, the fifth most\nwidely spoken language in the world, is lagging considerably in the research\nand development of such domain. Besides, while there are many established data\nsets to related to image annotation in English, no such resource exists for\nBangla yet. Hence, this paper outlines the development of \"Chittron\", an\nautomatic image captioning system in Bangla. Moreover, to address the data set\navailability issue, a collection of 16,000 Bangladeshi contextual images has\nbeen accumulated and manually annotated in Bangla. This data set is then used\nto train a model which integrates a pre-trained VGG16 image embedding model\nwith stacked LSTM layers. The model is trained to predict the caption when the\ninput is an image, one word at a time. The results show that the model has\nsuccessfully been able to learn a working language model and to generate\ncaptions of images quite accurately in many cases. The results are evaluated\nmainly qualitatively. However, BLEU scores are also reported. It is expected\nthat a better result can be obtained with a bigger and more varied data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00385v1",
            "date": "2018-09-02 20:22:48+00:00",
            "title": "Zero-shot User Intent Detection via Capsule Neural Networks",
            "authors": [
                "Congying Xia",
                "Chenwei Zhang",
                "Xiaohui Yan",
                "Yi Chang",
                "Philip S. Yu"
            ],
            "abstract": "User intent detection plays a critical role in question-answering and dialog\nsystems. Most previous works treat intent detection as a classification problem\nwhere utterances are labeled with predefined intents. However, it is\nlabor-intensive and time-consuming to label users' utterances as intents are\ndiversely expressed and novel intents will continually be involved. Instead, we\nstudy the zero-shot intent detection problem, which aims to detect emerging\nuser intents where no labeled utterances are currently available. We propose\ntwo capsule-based architectures: INTENT-CAPSNET that extracts semantic features\nfrom utterances and aggregates them to discriminate existing intents, and\nINTENTCAPSNET-ZSL which gives INTENTCAPSNET the zero-shot learning ability to\ndiscriminate emerging intents via knowledge transfer from existing intents.\nExperiments on two real-world datasets show that our model not only can better\ndiscriminate diversely expressed existing intents, but is also able to\ndiscriminate emerging intents when no labeled utterances are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00540v1",
            "date": "2018-09-03 10:40:26+00:00",
            "title": "Multilingual Clustering of Streaming News",
            "authors": [
                "Sebasti\u00e3o Miranda",
                "Art\u016brs Znoti\u0146\u0161",
                "Shay B. Cohen",
                "Guntis Barzdins"
            ],
            "abstract": "Clustering news across languages enables efficient media monitoring by\naggregating articles from multilingual sources into coherent stories. Doing so\nin an online setting allows scalable processing of massive news streams. To\nthis end, we describe a novel method for clustering an incoming stream of\nmultilingual documents into monolingual and crosslingual story clusters. Unlike\ntypical clustering approaches that consider a small and known number of labels,\nwe tackle the problem of discovering an ever growing number of cluster labels\nin an online fashion, using real news datasets in multiple languages. Our\nmethod is simple to implement, computationally efficient and produces\nstate-of-the-art results on datasets in German, English and Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00549v2",
            "date": "2018-09-03 11:05:00+00:00",
            "title": "Emergence of Communication in an Interactive World with Consistent Speakers",
            "authors": [
                "Ben Bogin",
                "Mor Geva",
                "Jonathan Berant"
            ],
            "abstract": "Training agents to communicate with one another given task-based supervision\nonly has attracted considerable attention recently, due to the growing interest\nin developing models for human-agent interaction. Prior work on the topic\nfocused on simple environments, where training using policy gradient was\nfeasible despite the non-stationarity of the agents during training. In this\npaper, we present a more challenging environment for testing the emergence of\ncommunication from raw pixels, where training using policy gradient fails. We\npropose a new model and training algorithm, that utilizes the structure of a\nlearned representation space to produce more consistent speakers at the initial\nphases of training, which stabilizes learning. We empirically show that our\nalgorithm substantially improves performance compared to policy gradient. We\nalso propose a new alignment-based metric for measuring context-independence in\nemerged communication and find our method increases context-independence\ncompared to policy gradient and other competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00647v1",
            "date": "2018-09-03 16:35:07+00:00",
            "title": "Automatic Event Salience Identification",
            "authors": [
                "Zhengzhong Liu",
                "Chenyan Xiong",
                "Teruko Mitamura",
                "Eduard Hovy"
            ],
            "abstract": "Identifying the salience (i.e. importance) of discourse units is an important\ntask in language understanding. While events play important roles in text\ndocuments, little research exists on analyzing their saliency status. This\npaper empirically studies the Event Salience task and proposes two salience\ndetection models based on content similarities and discourse relations. The\nfirst is a feature based salience model that incorporates similarities among\ndiscourse units. The second is a neural model that captures more complex\nrelations between discourse units. Tested on our new large-scale event salience\ncorpus, both methods significantly outperform the strong frequency baseline,\nwhile our neural model further improves the feature based one by a large\nmargin. Our analyses demonstrate that our neural model captures interesting\nconnections between salience and discourse unit relations (e.g., scripts and\nframe structures).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00782v1",
            "date": "2018-09-04 03:15:56+00:00",
            "title": "Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text",
            "authors": [
                "Haitian Sun",
                "Bhuwan Dhingra",
                "Manzil Zaheer",
                "Kathryn Mazaitis",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "Open Domain Question Answering (QA) is evolving from complex pipelined\nsystems to end-to-end deep neural networks. Specialized neural models have been\ndeveloped for extracting answers from either text alone or Knowledge Bases\n(KBs) alone. In this paper we look at a more practical setting, namely QA over\nthe combination of a KB and entity-linked text, which is appropriate when an\nincomplete KB is available with a large text corpus. Building on recent\nadvances in graph representation learning we propose a novel model, GRAFT-Net,\nfor extracting answers from a question-specific subgraph containing text and KB\nentities and relations. We construct a suite of benchmark tasks for this\nproblem, varying the difficulty of questions, the amount of training data, and\nKB completeness. We show that GRAFT-Net is competitive with the\nstate-of-the-art when tested using either KBs or text alone, and vastly\noutperforms existing methods in the combined setting. Source code is available\nat https://github.com/OceanskySun/GraftNet .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00800v1",
            "date": "2018-09-04 05:33:00+00:00",
            "title": "Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions",
            "authors": [
                "Sho Yokoi",
                "Sosuke Kobayashi",
                "Kenji Fukumizu",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "In this paper, we propose a new kernel-based co-occurrence measure that can\nbe applied to sparse linguistic expressions (e.g., sentences) with a very short\nlearning time, as an alternative to pointwise mutual information (PMI). As well\nas deriving PMI from mutual information, we derive this new measure from the\nHilbert--Schmidt independence criterion (HSIC); thus, we call the new measure\nthe pointwise HSIC (PHSIC). PHSIC can be interpreted as a smoothed variant of\nPMI that allows various similarity metrics (e.g., sentence embeddings) to be\nplugged in as kernels. Moreover, PHSIC can be estimated by simple and fast\n(linear in the size of the data) matrix calculations regardless of whether we\nuse linear or nonlinear kernels. Empirically, in a dialogue response selection\ntask, PHSIC is learned thousands of times faster than an RNN-based PMI while\noutperforming PMI in accuracy. In addition, we also demonstrate that PHSIC is\nbeneficial as a criterion of a data selection task for machine translation\nowing to its ability to give high (low) scores to a consistent (inconsistent)\npair with other pairs.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00812v1",
            "date": "2018-09-04 07:04:55+00:00",
            "title": "RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes",
            "authors": [
                "Semih Yagcioglu",
                "Aykut Erdem",
                "Erkut Erdem",
                "Nazli Ikizler-Cinbis"
            ],
            "abstract": "Understanding and reasoning about cooking recipes is a fruitful research\ndirection towards enabling machines to interpret procedural text. In this work,\nwe introduce RecipeQA, a dataset for multimodal comprehension of cooking\nrecipes. It comprises of approximately 20K instructional recipes with multiple\nmodalities such as titles, descriptions and aligned set of images. With over\n36K automatically generated question-answer pairs, we design a set of\ncomprehension and reasoning tasks that require joint understanding of images\nand text, capturing the temporal flow of events and making sense of procedural\nknowledge. Our preliminary results indicate that RecipeQA will serve as a\nchallenging test bed and an ideal benchmark for evaluating machine\ncomprehension systems. The data and leaderboard are available at\nhttp://hucvl.github.io/recipeqa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00918v2",
            "date": "2018-09-04 12:32:54+00:00",
            "title": "Segmentation-free Compositional $n$-gram Embedding",
            "authors": [
                "Geewook Kim",
                "Kazuki Fukui",
                "Hidetoshi Shimodaira"
            ],
            "abstract": "We propose a new type of representation learning method that models words,\nphrases and sentences seamlessly. Our method does not depend on word\nsegmentation and any human-annotated resources (e.g., word dictionaries), yet\nit is very effective for noisy corpora written in unsegmented languages such as\nChinese and Japanese. The main idea of our method is to ignore word boundaries\ncompletely (i.e., segmentation-free), and construct representations for all\ncharacter $n$-grams in a raw corpus with embeddings of compositional\nsub-$n$-grams. Although the idea is simple, our experiments on various\nbenchmarks and real-world datasets show the efficacy of our proposal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01074v1",
            "date": "2018-09-04 16:28:36+00:00",
            "title": "A Novel Neural Sequence Model with Multiple Attentions for Word Sense Disambiguation",
            "authors": [
                "Mahtab Ahmed",
                "Muhammad Rifayat Samee",
                "Robert E. Mercer"
            ],
            "abstract": "Word sense disambiguation (WSD) is a well researched problem in computational\nlinguistics. Different research works have approached this problem in different\nways. Some state of the art results that have been achieved for this problem\nare by supervised models in terms of accuracy, but they often fall behind\nflexible knowledge-based solutions which use engineered features as well as\nhuman annotators to disambiguate every target word. This work focuses on\nbridging this gap using neural sequence models incorporating the well-known\nattention mechanism. The main gist of our work is to combine multiple\nattentions on different linguistic features through weights and to provide a\nunified framework for doing this. This weighted attention allows the model to\neasily disambiguate the sense of an ambiguous word by attending over a suitable\nportion of a sentence. Our extensive experiments show that multiple attention\nenables a more versatile encoder-decoder model leading to state of the art\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01110v3",
            "date": "2018-09-04 17:31:13+00:00",
            "title": "Text2Scene: Generating Compositional Scenes from Textual Descriptions",
            "authors": [
                "Fuwen Tan",
                "Song Feng",
                "Vicente Ordonez"
            ],
            "abstract": "In this paper, we propose Text2Scene, a model that generates various forms of\ncompositional scene representations from natural language descriptions. Unlike\nrecent works, our method does NOT use Generative Adversarial Networks (GANs).\nText2Scene instead learns to sequentially generate objects and their attributes\n(location, size, appearance, etc) at every time step by attending to different\nparts of the input text and the current status of the generated scene. We show\nthat under minor modifications, the proposed framework can handle the\ngeneration of different forms of scene representations, including cartoon-like\nscenes, object layouts corresponding to real images, and synthetic images. Our\nmethod is not only competitive when compared with state-of-the-art GAN-based\nmethods using automatic metrics and superior based on human judgments but also\nhas the advantage of producing interpretable results.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.01337v1",
            "date": "2018-09-05 05:58:47+00:00",
            "title": "Localizing Moments in Video with Temporal Language",
            "authors": [
                "Lisa Anne Hendricks",
                "Oliver Wang",
                "Eli Shechtman",
                "Josef Sivic",
                "Trevor Darrell",
                "Bryan Russell"
            ],
            "abstract": "Localizing moments in a longer video via natural language queries is a new,\nchallenging task at the intersection of language and video understanding.\nThough moment localization with natural language is similar to other language\nand vision tasks like natural language object retrieval in images, moment\nlocalization offers an interesting opportunity to model temporal dependencies\nand reasoning in text. We propose a new model that explicitly reasons about\ndifferent temporal segments in a video, and shows that temporal context is\nimportant for localizing phrases which include temporal language. To benchmark\nwhether our model, and other recent video localization models, can effectively\nreason about temporal language, we collect the novel TEMPOral reasoning in\nvideo and language (TEMPO) dataset. Our dataset consists of two parts: a\ndataset with real videos and template sentences (TEMPO - Template Language)\nwhich allows for controlled studies on temporal language, and a human language\ndataset which consists of temporal sentences annotated by humans (TEMPO - Human\nLanguage).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.01452v1",
            "date": "2018-09-05 12:04:35+00:00",
            "title": "Sentylic at IEST 2018: Gated Recurrent Neural Network and Capsule Network Based Approach for Implicit Emotion Detection",
            "authors": [
                "Prabod Rathnayaka",
                "Supun Abeysinghe",
                "Chamod Samarajeewa",
                "Isura Manchanayake",
                "Malaka Walpola"
            ],
            "abstract": "In this paper, we present the system we have used for the Implicit WASSA 2018\nImplicit Emotion Shared Task. The task is to predict the emotion of a tweet of\nwhich the explicit mentions of emotion terms have been removed. The idea is to\ncome up with a model which has the ability to implicitly identify the emotion\nexpressed given the context words. We have used a Gated Recurrent Neural\nNetwork (GRU) and a Capsule Network based model for the task. Pre-trained word\nembeddings have been utilized to incorporate contextual knowledge about words\ninto the model. GRU layer learns latent representations using the input word\nembeddings. Subsequent Capsule Network layer learns high-level features from\nthat hidden representation. The proposed model managed to achieve a macro-F1\nscore of 0.692.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01497v1",
            "date": "2018-08-30 00:57:09+00:00",
            "title": "Chinese Discourse Segmentation Using Bilingual Discourse Commonality",
            "authors": [
                "Jingfeng Yang",
                "Sujian Li"
            ],
            "abstract": "Discourse segmentation aims to segment Elementary Discourse Units (EDUs) and\nis a fundamental task in discourse analysis. For Chinese, previous researches\nidentify EDUs just through discriminating the functions of punctuations. In\nthis paper, we argue that Chinese EDUs may not end at the punctuation positions\nand should follow the definition of EDU in RST-DT. With this definition, we\nconduct Chinese discourse segmentation with the help of English labeled\ndata.Using discourse commonality between English and Chinese, we design an\nadversarial neural network framework to extract common language-independent\nfeatures and language-specific features which are useful for discourse\nsegmentation, when there is no or only a small scale of Chinese labeled data\navailable. Experiments on discourse segmentation demonstrate that our models\ncan leverage common features from bilingual data, and learn efficient\nChinese-specific features from a small amount of Chinese labeled data,\noutperforming the baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01610v1",
            "date": "2018-09-05 16:30:08+00:00",
            "title": "Bimodal network architectures for automatic generation of image annotation from text",
            "authors": [
                "Mehdi Moradi",
                "Ali Madani",
                "Yaniv Gur",
                "Yufan Guo",
                "Tanveer Syeda-Mahmood"
            ],
            "abstract": "Medical image analysis practitioners have embraced big data methodologies.\nThis has created a need for large annotated datasets. The source of big data is\ntypically large image collections and clinical reports recorded for these\nimages. In many cases, however, building algorithms aimed at segmentation and\ndetection of disease requires a training dataset with markings of the areas of\ninterest on the image that match with the described anomalies. This process of\nannotation is expensive and needs the involvement of clinicians. In this work\nwe propose two separate deep neural network architectures for automatic marking\nof a region of interest (ROI) on the image best representing a finding\nlocation, given a textual report or a set of keywords. One architecture\nconsists of LSTM and CNN components and is trained end to end with images,\nmatching text, and markings of ROIs for those images. The output layer\nestimates the coordinates of the vertices of a polygonal region. The second\narchitecture uses a network pre-trained on a large dataset of the same image\ntypes for learning feature representations of the findings of interest. We show\nthat for a variety of findings from chest X-ray images, both proposed\narchitectures learn to estimate the ROI, as validated by clinical annotations.\nThere is a clear advantage obtained from the architecture with pre-trained\nimaging network. The centroids of the ROIs marked by this network were on\naverage at a distance equivalent to 5.1% of the image width from the centroids\nof the ground truth ROIs.",
            "journal ref": "Lecture Notes in Computer Science (LNCS 11070), Proceedings of\n  Medical Image Computing & Computer Assisted Intervention (MICCAI 2018)",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.01682v2",
            "date": "2018-09-05 18:18:34+00:00",
            "title": "Deep Relevance Ranking Using Enhanced Document-Query Interactions",
            "authors": [
                "Ryan McDonald",
                "Georgios-Ioannis Brokos",
                "Ion Androutsopoulos"
            ],
            "abstract": "We explore several new models for document relevance ranking, building upon\nthe Deep Relevance Matching Model (DRMM) of Guo et al. (2016). Unlike DRMM,\nwhich uses context-insensitive encodings of terms and query-document term\ninteractions, we inject rich context-sensitive encodings throughout our models,\ninspired by PACRR's (Hui et al., 2017) convolutional n-gram matching features,\nbut extended in several ways including multiple views of query and document\ninputs. We test our models on datasets from the BIOASQ question answering\nchallenge (Tsatsaronis et al., 2015) and TREC ROBUST 2004 (Voorhees, 2005),\nshowing they outperform BM25-based baselines, DRMM, and PACRR.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01797v2",
            "date": "2018-09-06 02:56:58+00:00",
            "title": "Describing a Knowledge Base",
            "authors": [
                "Qingyun Wang",
                "Xiaoman Pan",
                "Lifu Huang",
                "Boliang Zhang",
                "Zhiying Jiang",
                "Heng Ji",
                "Kevin Knight"
            ],
            "abstract": "We aim to automatically generate natural language descriptions about an input\nstructured knowledge base (KB). We build our generation framework based on a\npointer network which can copy facts from the input KB, and add two attention\nmechanisms: (i) slot-aware attention to capture the association between a slot\ntype and its corresponding slot value; and (ii) a new \\emph{table position\nself-attention} to capture the inter-dependencies among related slots. For\nevaluation, besides standard metrics including BLEU, METEOR, and ROUGE, we\npropose a KB reconstruction based metric by extracting a KB from the generation\noutput and comparing it with the input KB. We also create a new data set which\nincludes 106,216 pairs of structured KBs and their corresponding natural\nlanguage descriptions for two distinct entity types. Experiments show that our\napproach significantly outperforms state-of-the-art methods. The reconstructed\nKB achieves 68.8% - 72.6% F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01941v1",
            "date": "2018-09-06 12:24:04+00:00",
            "title": "Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots",
            "authors": [
                "Shaojie Jiang",
                "Maarten de Rijke"
            ],
            "abstract": "Diversity is a long-studied topic in information retrieval that usually\nrefers to the requirement that retrieved results should be non-repetitive and\ncover different aspects. In a conversational setting, an additional dimension\nof diversity matters: an engaging response generation system should be able to\noutput responses that are diverse and interesting. Sequence-to-sequence\n(Seq2Seq) models have been shown to be very effective for response generation.\nHowever, dialogue responses generated by Seq2Seq models tend to have low\ndiversity. In this paper, we review known sources and existing approaches to\nthis low-diversity problem. We also identify a source of low diversity that has\nbeen little studied so far, namely model over-confidence. We sketch several\ndirections for tackling model over-confidence and, hence, the low-diversity\nproblem, including confidence penalties and label smoothing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01962v1",
            "date": "2018-09-06 13:12:27+00:00",
            "title": "Code-switched Language Models Using Dual RNNs and Same-Source Pretraining",
            "authors": [
                "Saurabh Garg",
                "Tanmay Parekh",
                "Preethi Jyothi"
            ],
            "abstract": "This work focuses on building language models (LMs) for code-switched text.\nWe propose two techniques that significantly improve these LMs: 1) A novel\nrecurrent neural network unit with dual components that focus on each language\nin the code-switched text separately 2) Pretraining the LM using synthetic text\nfrom a generative model estimated using the training data. We demonstrate the\neffectiveness of our proposed techniques by reporting perplexities on a\nMandarin-English task and derive significant reductions in perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02040v1",
            "date": "2018-09-06 15:18:14+00:00",
            "title": "Exploring Graph-structured Passage Representation for Multi-hop Reading Comprehension with Graph Neural Networks",
            "authors": [
                "Linfeng Song",
                "Zhiguo Wang",
                "Mo Yu",
                "Yue Zhang",
                "Radu Florian",
                "Daniel Gildea"
            ],
            "abstract": "Multi-hop reading comprehension focuses on one type of factoid question,\nwhere a system needs to properly integrate multiple pieces of evidence to\ncorrectly answer a question. Previous work approximates global evidence with\nlocal coreference information, encoding coreference chains with DAG-styled GRU\nlayers within a gated-attention reader. However, coreference is limited in\nproviding information for rich inference. We introduce a new method for better\nconnecting global evidence, which forms more complex graphs compared to DAGs.\nTo perform evidence integration on our graphs, we investigate two recent graph\nneural networks, namely graph convolutional network (GCN) and graph recurrent\nnetwork (GRN). Experiments on two standard datasets show that richer global\ninformation leads to better answers. Our method performs better than all\npublished results on these datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02079v1",
            "date": "2018-09-06 16:27:32+00:00",
            "title": "Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models",
            "authors": [
                "Tong Niu",
                "Mohit Bansal"
            ],
            "abstract": "We present two categories of model-agnostic adversarial strategies that\nreveal the weaknesses of several generative, task-oriented dialogue models:\nShould-Not-Change strategies that evaluate over-sensitivity to small and\nsemantics-preserving edits, as well as Should-Change strategies that test if a\nmodel is over-stable against subtle yet semantics-changing modifications. We\nnext perform adversarial training with each strategy, employing a max-margin\napproach for negative generative examples. This not only makes the target\ndialogue model more robust to the adversarial inputs, but also helps it perform\nsignificantly better on the original inputs. Moreover, training on all\nstrategies combined achieves further improvements, achieving a new\nstate-of-the-art performance on the original task (also verified via human\nevaluation). In addition to adversarial training, we also address the\nrobustness task at the model-level, by feeding it subword units as both inputs\nand outputs, and show that the resulting model is equally competitive, requires\nonly 1/4 of the original vocabulary size, and is robust to one of the\nadversarial strategies (to which the original model is vulnerable) even without\nadversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02156v2",
            "date": "2018-09-06 18:25:18+00:00",
            "title": "Object Hallucination in Image Captioning",
            "authors": [
                "Anna Rohrbach",
                "Lisa Anne Hendricks",
                "Kaylee Burns",
                "Trevor Darrell",
                "Kate Saenko"
            ],
            "abstract": "Despite continuously improving performance, contemporary image captioning\nmodels are prone to \"hallucinating\" objects that are not actually in a scene.\nOne problem is that standard metrics only measure similarity to ground truth\ncaptions and may not fully capture image relevance. In this work, we propose a\nnew image relevance metric to evaluate current models with veridical visual\nlabels and assess their rate of object hallucination. We analyze how captioning\nmodel architectures and learning objectives contribute to object hallucination,\nexplore when hallucination is likely due to image misclassification or language\npriors, and assess how well current sentence metrics capture object\nhallucination. We investigate these questions on the standard image captioning\nbenchmark, MSCOCO, using a diverse set of models. Our analysis yields several\ninteresting findings, including that models which score best on standard\nsentence metrics do not always have lower hallucination and that models which\nhallucinate more tend to make errors driven by language priors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02208v4",
            "date": "2018-09-06 20:39:23+00:00",
            "title": "Assessing Gender Bias in Machine Translation -- A Case Study with Google Translate",
            "authors": [
                "Marcelo O. R. Prates",
                "Pedro H. C. Avelar",
                "Luis Lamb"
            ],
            "abstract": "Recently there has been a growing concern about machine bias, where trained\nstatistical models grow to reflect controversial societal asymmetries, such as\ngender or racial bias. A significant number of AI tools have recently been\nsuggested to be harmfully biased towards some minority, with reports of racist\ncriminal behavior predictors, Iphone X failing to differentiate between two\nAsian people and Google photos' mistakenly classifying black people as\ngorillas. Although a systematic study of such biases can be difficult, we\nbelieve that automated translation tools can be exploited through gender\nneutral languages to yield a window into the phenomenon of gender bias in AI.\n  In this paper, we start with a comprehensive list of job positions from the\nU.S. Bureau of Labor Statistics (BLS) and used it to build sentences in\nconstructions like \"He/She is an Engineer\" in 12 different gender neutral\nlanguages such as Hungarian, Chinese, Yoruba, and several others. We translate\nthese sentences into English using the Google Translate API, and collect\nstatistics about the frequency of female, male and gender-neutral pronouns in\nthe translated output. We show that GT exhibits a strong tendency towards male\ndefaults, in particular for fields linked to unbalanced gender distribution\nsuch as STEM jobs. We ran these statistics against BLS' data for the frequency\nof female participation in each job position, showing that GT fails to\nreproduce a real-world distribution of female workers. We provide experimental\nevidence that even if one does not expect in principle a 50:50 pronominal\ngender distribution, GT yields male defaults much more frequently than what\nwould be expected from demographic data alone.\n  We are hopeful that this work will ignite a debate about the need to augment\ncurrent statistical translation tools with debiasing techniques which can\nalready be found in the scientific literature.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1809.02494v1",
            "date": "2018-09-07 14:20:32+00:00",
            "title": "Meteorologists and Students: A resource for language grounding of geographical descriptors",
            "authors": [
                "Alejandro Ramos-Soto",
                "Ehud Reiter",
                "Kees van Deemter",
                "Jose M. Alonso",
                "Albert Gatt"
            ],
            "abstract": "We present a data resource which can be useful for research purposes on\nlanguage grounding tasks in the context of geographical referring expression\ngeneration. The resource is composed of two data sets that encompass 25\ndifferent geographical descriptors and a set of associated graphical\nrepresentations, drawn as polygons on a map by two groups of human subjects:\nteenage students and expert meteorologists.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02592v1",
            "date": "2018-09-07 17:34:34+00:00",
            "title": "Logographic Subword Model for Neural Machine Translation",
            "authors": [
                "Yihao Fang",
                "Rong Zheng",
                "Xiaodan Zhu"
            ],
            "abstract": "A novel logographic subword model is proposed to reinterpret logograms as\nabstract subwords for neural machine translation. Our approach drastically\nreduces the size of an artificial neural network, while maintaining comparable\nBLEU scores as those attained with the baseline RNN and CNN seq2seq models. The\nsmaller model size also leads to shorter training and inference time.\nExperiments demonstrate that in the tasks of English-Chinese/Chinese-English\ntranslation, the reduction of those aspects can be from $11\\%$ to as high as\n$77\\%$. Compared to previous subword models, abstract subwords can be applied\nto various logographic languages. Considering most of the logographic languages\nare ancient and very low resource languages, these advantages are very\ndesirable for archaeological computational linguistic applications such as a\nresource-limited offline hand-held Demotic-English translator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02649v1",
            "date": "2018-09-07 19:38:51+00:00",
            "title": "A Transfer-Learnable Natural Language Interface for Databases",
            "authors": [
                "Wenlu Wang",
                "Yingtao Tian",
                "Hongyu Xiong",
                "Haixun Wang",
                "Wei-Shinn Ku"
            ],
            "abstract": "Relational database management systems (RDBMSs) are powerful because they are\nable to optimize and answer queries against any relational database. A natural\nlanguage interface (NLI) for a database, on the other hand, is tailored to\nsupport that specific database. In this work, we introduce a general purpose\ntransfer-learnable NLI with the goal of learning one model that can be used as\nNLI for any relational database. We adopt the data management principle of\nseparating data and its schema, but with the additional support for the\nidiosyncrasy and complexity of natural languages. Specifically, we introduce an\nautomatic annotation mechanism that separates the schema and the data, where\nthe schema also covers knowledge about natural language. Furthermore, we\npropose a customized sequence model that translates annotated natural language\nqueries to SQL statements. We show in experiments that our approach outperforms\nprevious NLI methods on the WikiSQL dataset and the model we learned can be\napplied to another benchmark dataset OVERNIGHT without retraining.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.02687v1",
            "date": "2018-09-07 21:43:30+00:00",
            "title": "Coherence-Aware Neural Topic Modeling",
            "authors": [
                "Ran Ding",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "Topic models are evaluated based on their ability to describe documents well\n(i.e. low perplexity) and to produce topics that carry coherent semantic\nmeaning. In topic modeling so far, perplexity is a direct optimization target.\nHowever, topic coherence, owing to its challenging computation, is not\noptimized for and is only evaluated after training. In this work, under a\nneural variational inference framework, we propose methods to incorporate a\ntopic coherence objective into the training process. We demonstrate that such a\ncoherence-aware topic model exhibits a similar level of perplexity as baseline\nmodels but achieves substantially higher topic coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02719v2",
            "date": "2018-09-07 23:59:22+00:00",
            "title": "What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks",
            "authors": [
                "Haohan Wang",
                "Da Sun",
                "Eric P. Xing"
            ],
            "abstract": "Nature language inference (NLI) task is a predictive task of determining the\ninference relationship of a pair of natural language sentences. With the\nincreasing popularity of NLI, many state-of-the-art predictive models have been\nproposed with impressive performances. However, several works have noticed the\nstatistical irregularities in the collected NLI data set that may result in an\nover-estimated performance of these models and proposed remedies. In this\npaper, we further investigate the statistical irregularities, what we refer as\nconfounding factors, of the NLI data sets. With the belief that some NLI labels\nshould preserve under swapping operations, we propose a simple yet effective\nway (swapping the two text fragments) of evaluating the NLI predictive models\nthat naturally mitigate the observed problems. Further, we continue to train\nthe predictive models with our swapping manner and propose to use the deviation\nof the model's evaluation performances under different percentages of training\ntext fragments to be swapped to describe the robustness of a predictive model.\nOur evaluation metrics leads to some interesting understandings of recent\npublished NLI methods. Finally, we also apply the swapping operation on NLI\nmodels to see the effectiveness of this straightforward method in mitigating\nthe confounding factor problems in training generic sentence embeddings for\nother NLP transfer tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02735v1",
            "date": "2018-09-08 01:49:03+00:00",
            "title": "Operations Guided Neural Networks for High Fidelity Data-To-Text Generation",
            "authors": [
                "Feng Nie",
                "Jinpeng Wang",
                "Jin-Ge Yao",
                "Rong Pan",
                "Chin-Yew Lin"
            ],
            "abstract": "Recent neural models for data-to-text generation are mostly based on\ndata-driven end-to-end training over encoder-decoder networks. Even though the\ngenerated texts are mostly fluent and informative, they often generate\ndescriptions that are not consistent with the input structured data. This is a\ncritical issue especially in domains that require inference or calculations\nover raw data. In this paper, we attempt to improve the fidelity of neural\ndata-to-text generation by utilizing pre-executed symbolic operations. We\npropose a framework called Operation-guided Attention-based\nsequence-to-sequence network (OpAtt), with a specifically designed gating\nmechanism as well as a quantization module for operation results to utilize\ninformation from pre-executed operations. Experiments on two sports datasets\nshow our proposed method clearly improves the fidelity of the generated texts\nto the input structured data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02790v4",
            "date": "2018-09-08 11:54:09+00:00",
            "title": "The Lower The Simpler: Simplifying Hierarchical Recurrent Models",
            "authors": [
                "Chao Wang",
                "Hui Jiang"
            ],
            "abstract": "To improve the training efficiency of hierarchical recurrent models without\ncompromising their performance, we propose a strategy named as `the lower the\nsimpler', which is to simplify the baseline models by making the lower layers\nsimpler than the upper layers. We carry out this strategy to simplify two\ntypical hierarchical recurrent models, namely Hierarchical Recurrent\nEncoder-Decoder (HRED) and R-NET, whose basic building block is GRU.\nSpecifically, we propose Scalar Gated Unit (SGU), which is a simplified variant\nof GRU, and use it to replace the GRUs at the middle layers of HRED and R-NET.\nBesides, we also use Fixed-size Ordinally-Forgetting Encoding (FOFE), which is\nan efficient encoding method without any trainable parameter, to replace the\nGRUs at the bottom layers of HRED and R-NET. The experimental results show that\nthe simplified HRED and the simplified R-NET contain significantly less\ntrainable parameters, consume significantly less training time, and achieve\nslightly better performance than their baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02805v2",
            "date": "2018-09-08 14:14:03+00:00",
            "title": "Faithful Multimodal Explanation for Visual Question Answering",
            "authors": [
                "Jialin Wu",
                "Raymond J. Mooney"
            ],
            "abstract": "AI systems' ability to explain their reasoning is critical to their utility\nand trustworthiness. Deep neural networks have enabled significant progress on\nmany challenging problems such as visual question answering (VQA). However,\nmost of them are opaque black boxes with limited explanatory capability. This\npaper presents a novel approach to developing a high-performing VQA system that\ncan elucidate its answers with integrated textual and visual explanations that\nfaithfully reflect important aspects of its underlying reasoning while\ncapturing the style of comprehensible human explanations. Extensive\nexperimental evaluation demonstrates the advantages of this approach compared\nto competing methods with both automatic evaluation metrics and human\nevaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02823v1",
            "date": "2018-09-08 15:29:47+00:00",
            "title": "Extracting and Analyzing Semantic Relatedness between Cities Using News Articles",
            "authors": [
                "Yingjie Hu",
                "Xinyue Ye",
                "Shih-Lung Shaw"
            ],
            "abstract": "News articles capture a variety of topics about our society. They reflect not\nonly the socioeconomic activities that happened in our physical world, but also\nsome of the cultures, human interests, and public concerns that exist only in\nthe perceptions of people. Cities are frequently mentioned in news articles,\nand two or more cities may co-occur in the same article. Such co-occurrence\noften suggests certain relatedness between the mentioned cities, and the\nrelatedness may be under different topics depending on the contents of the news\narticles. We consider the relatedness under different topics as semantic\nrelatedness. By reading news articles, one can grasp the general semantic\nrelatedness between cities, yet, given hundreds of thousands of news articles,\nit is very difficult, if not impossible, for anyone to manually read them. This\npaper proposes a computational framework which can \"read\" a large number of\nnews articles and extract the semantic relatedness between cities. This\nframework is based on a natural language processing model and employs a machine\nlearning process to identify the main topics of news articles. We describe the\noverall structure of this framework and its individual modules, and then apply\nit to an experimental dataset with more than 500,000 news articles covering the\ntop 100 U.S. cities spanning a 10-year period. We perform exploratory\nvisualization of the extracted semantic relatedness under different topics and\nover multiple years. We also analyze the impact of geographic distance on\nsemantic relatedness and find varied distance decay effects. The proposed\nframework can be used to support large-scale content analysis in city network\nresearch.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1809.02992v1",
            "date": "2018-09-09 15:45:25+00:00",
            "title": "Speeding Up Neural Machine Translation Decoding by Cube Pruning",
            "authors": [
                "Wen Zhang",
                "Liang Huang",
                "Yang Feng",
                "Lei Shen",
                "Qun Liu"
            ],
            "abstract": "Although neural machine translation has achieved promising results, it\nsuffers from slow translation speed. The direct consequence is that a trade-off\nhas to be made between translation quality and speed, thus its performance can\nnot come into full play. We apply cube pruning, a popular technique to speed up\ndynamic programming, into neural machine translation to speed up the\ntranslation. To construct the equivalence class, similar target hidden states\nare combined, leading to less RNN expansion operations on the target side and\nless \\$\\mathrm{softmax}\\$ operations over the large target vocabulary. The\nexperiments show that, at the same or even better translation quality, our\nmethod can translate faster compared with naive beam search by \\$3.3\\times\\$ on\nGPUs and \\$3.5\\times\\$ on CPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03202v1",
            "date": "2018-09-10 09:17:04+00:00",
            "title": "Learning Sequence Encoders for Temporal Knowledge Graph Completion",
            "authors": [
                "Alberto Garc\u00eda-Dur\u00e1n",
                "Sebastijan Duman\u010di\u0107",
                "Mathias Niepert"
            ],
            "abstract": "Research on link prediction in knowledge graphs has mainly focused on static\nmulti-relational data. In this work we consider temporal knowledge graphs where\nrelations between entities may only hold for a time interval or a specific\npoint in time. In line with previous work on static knowledge graphs, we\npropose to address this problem by learning latent entity and relation type\nrepresentations. To incorporate temporal information, we utilize recurrent\nneural networks to learn time-aware representations of relation types which can\nbe used in conjunction with existing latent factorization methods. The proposed\napproach is shown to be robust to common challenges in real-world KGs: the\nsparsity and heterogeneity of temporal expressions. Experiments show the\nbenefits of our approach on four temporal KGs. The data sets are available\nunder a permissive BSD-3 license 1.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.03408v2",
            "date": "2018-09-10 15:46:58+00:00",
            "title": "Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat",
            "authors": [
                "Ravi Shekhar",
                "Aashish Venkatesh",
                "Tim Baumg\u00e4rtner",
                "Elia Bruni",
                "Barbara Plank",
                "Raffaella Bernardi",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "We propose a grounded dialogue state encoder which addresses a foundational\nissue on how to integrate visual grounding with dialogue system components. As\na test-bed, we focus on the GuessWhat?! game, a two-player game where the goal\nis to identify an object in a complex visual scene by asking a sequence of\nyes/no questions. Our visually-grounded encoder leverages synergies between\nguessing and asking questions, as it is trained jointly using multi-task\nlearning. We further enrich our model via a cooperative learning regime. We\nshow that the introduction of both the joint architecture and cooperative\nlearning lead to accuracy improvements over the baseline system. We compare our\napproach to an alternative system which extends the baseline with reinforcement\nlearning. Our in-depth analysis shows that the linguistic skills of the two\nmodels differ dramatically, despite approaching comparable performance levels.\nThis points at the importance of analyzing the linguistic output of competing\nsystems beyond numeric comparison solely based on task success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03449v3",
            "date": "2018-09-10 16:42:22+00:00",
            "title": "Explicit Utilization of General Knowledge in Machine Reading Comprehension",
            "authors": [
                "Chao Wang",
                "Hui Jiang"
            ],
            "abstract": "To bridge the gap between Machine Reading Comprehension (MRC) models and\nhuman beings, which is mainly reflected in the hunger for data and the\nrobustness to noise, in this paper, we explore how to integrate the neural\nnetworks of MRC models with the general knowledge of human beings. On the one\nhand, we propose a data enrichment method, which uses WordNet to extract\ninter-word semantic connections as general knowledge from each given\npassage-question pair. On the other hand, we propose an end-to-end MRC model\nnamed as Knowledge Aided Reader (KAR), which explicitly uses the above\nextracted general knowledge to assist its attention mechanisms. Based on the\ndata enrichment method, KAR is comparable in performance with the\nstate-of-the-art MRC models, and significantly more robust to noise than them.\nWhen only a subset (20%-80%) of the training examples are available, KAR\noutperforms the state-of-the-art MRC models by a large margin, and is still\nreasonably robust to noise.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.03695v1",
            "date": "2018-09-11 06:40:36+00:00",
            "title": "Evaluating Multimodal Representations on Sentence Similarity: vSTS, Visual Semantic Textual Similarity Dataset",
            "authors": [
                "Oier Lopez de Lacalle",
                "Aitor Soroa",
                "Eneko Agirre"
            ],
            "abstract": "In this paper we introduce vSTS, a new dataset for measuring textual\nsimilarity of sentences using multimodal information. The dataset is comprised\nby images along with its respectively textual captions. We describe the dataset\nboth quantitatively and qualitatively, and claim that it is a valid gold\nstandard for measuring automatic multimodal textual similarity systems. We also\ndescribe the initial experiments combining the multimodal information.",
            "journal ref": "ICCV17: second workshop on Closing the Loop Between Vision and\n  Language. Venice, Italy. 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04259v1",
            "date": "2018-09-12 05:15:32+00:00",
            "title": "Generalizing Word Embeddings using Bag of Subwords",
            "authors": [
                "Jinman Zhao",
                "Sidharth Mudgal",
                "Yingyu Liang"
            ],
            "abstract": "We approach the problem of generalizing pre-trained word embeddings beyond\nfixed-size vocabularies without using additional contextual information. We\npropose a subword-level word vector generation model that views words as bags\nof character $n$-grams. The model is simple, fast to train and provides good\nvectors for rare or unseen words. Experiments show that our model achieves\nstate-of-the-art performances in English word similarity task and in joint\nprediction of part-of-speech tag and morphosyntactic attributes in 23\nlanguages, suggesting our model's ability in capturing the relationship between\nwords' textual representations and their embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04283v4",
            "date": "2018-09-12 07:31:06+00:00",
            "title": "Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks",
            "authors": [
                "Shikhar Vashishth",
                "Manik Bhandari",
                "Prateek Yadav",
                "Piyush Rai",
                "Chiranjib Bhattacharyya",
                "Partha Talukdar"
            ],
            "abstract": "Word embeddings have been widely adopted across several NLP applications.\nMost existing word embedding methods utilize sequential context of a word to\nlearn its embedding. While there have been some attempts at utilizing syntactic\ncontext of a word, such methods result in an explosion of the vocabulary size.\nIn this paper, we overcome this problem by proposing SynGCN, a flexible Graph\nConvolution based method for learning word embeddings. SynGCN utilizes the\ndependency context of a word without increasing the vocabulary size. Word\nembeddings learned by SynGCN outperform existing methods on various intrinsic\nand extrinsic tasks and provide an advantage when used with ELMo. We also\npropose SemGCN, an effective framework for incorporating diverse semantic\nknowledge for further enhancing learned word representations. We make the\nsource code of both models available to encourage reproducible research.",
            "journal ref": "57th Annual Meeting of the Association for Computational\n  Linguistics (ACL 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04306v1",
            "date": "2018-09-12 08:31:20+00:00",
            "title": "Chinese Poetry Generation with a Working Memory Model",
            "authors": [
                "Xiaoyuan Yi",
                "Maosong Sun",
                "Ruoyu Li",
                "Zonghan Yang"
            ],
            "abstract": "As an exquisite and concise literary form, poetry is a gem of human culture.\nAutomatic poetry generation is an essential step towards computer creativity.\nIn recent years, several neural models have been designed for this task.\nHowever, among lines of a whole poem, the coherence in meaning and topics still\nremains a big challenge. In this paper, inspired by the theoretical concept in\ncognitive psychology, we propose a novel Working Memory model for poetry\ngeneration. Different from previous methods, our model explicitly maintains\ntopics and informative limited history in a neural memory. During the\ngeneration process, our model reads the most relevant parts from memory slots\nto generate the current line. After each line is generated, it writes the most\nsalient parts of the previous line into memory slots. By dynamic manipulation\nof the memory, our model keeps a coherent information flow and learns to\nexpress each topic flexibly and naturally. We experiment on three different\ngenres of Chinese poetry: quatrain, iambic and chinoiserie lyric. Both\nautomatic and human evaluation results show that our model outperforms current\nstate-of-the-art methods.",
            "journal ref": "In Proceedings of the Twenty-Seventh International Joint\n  Conference on Artificial Intelligence, pages 4553-4559, Stockholm, Sweden,\n  2018",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.04313v1",
            "date": "2018-09-12 08:50:30+00:00",
            "title": "Chinese Poetry Generation with a Salient-Clue Mechanism",
            "authors": [
                "Xiaoyuan Yi",
                "Ruoyu Li",
                "Maosong Sun"
            ],
            "abstract": "As a precious part of the human cultural heritage, Chinese poetry has\ninfluenced people for generations. Automatic poetry composition is a challenge\nfor AI. In recent years, significant progress has been made in this area\nbenefiting from the development of neural networks. However, the coherence in\nmeaning, theme or even artistic conception for a generated poem as a whole\nstill remains a big problem. In this paper, we propose a novel Salient-Clue\nmechanism for Chinese poetry generation. Different from previous work which\ntried to exploit all the context information, our model selects the most\nsalient characters automatically from each so-far generated line to gradually\nform a salient clue, which is utilized to guide successive poem generation\nprocess so as to eliminate interruptions and improve coherence. Besides, our\nmodel can be flexibly extended to control the generated poem in different\naspects, for example, poetry style, which further enhances the coherence.\nExperimental results show that our model is very effective, outperforming three\nstrong baselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.04318v1",
            "date": "2018-09-12 09:03:20+00:00",
            "title": "Neural Melody Composition from Lyrics",
            "authors": [
                "Hangbo Bao",
                "Shaohan Huang",
                "Furu Wei",
                "Lei Cui",
                "Yu Wu",
                "Chuanqi Tan",
                "Songhao Piao",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we study a novel task that learns to compose music from\nnatural language. Given the lyrics as input, we propose a melody composition\nmodel that generates lyrics-conditional melody as well as the exact alignment\nbetween the generated melody and the given lyrics simultaneously. More\nspecifically, we develop the melody composition model based on the\nsequence-to-sequence framework. It consists of two neural encoders to encode\nthe current lyrics and the context melody respectively, and a hierarchical\ndecoder to jointly produce musical notes and the corresponding alignment.\nExperimental results on lyrics-melody pairs of 18,451 pop songs demonstrate the\neffectiveness of our proposed methods. In addition, we apply a singing voice\nsynthesizer software to synthesize the \"singing\" of the lyrics and melodies for\nhuman evaluation. Results indicate that our generated melodies are more\nmelodious and tuneful compared with the baseline method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04556v6",
            "date": "2018-09-10 17:25:46+00:00",
            "title": "Unsupervised Controllable Text Formalization",
            "authors": [
                "Parag Jain",
                "Abhijit Mishra",
                "Amar Prakash Azad",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "We propose a novel framework for controllable natural language\ntransformation. Realizing that the requirement of parallel corpus is\npractically unsustainable for controllable generation tasks, an unsupervised\ntraining scheme is introduced. The crux of the framework is a deep neural\nencoder-decoder that is reinforced with text-transformation knowledge through\nauxiliary modules (called scorers). The scorers, based on off-the-shelf\nlanguage processing tools, decide the learning scheme of the encoder-decoder\nbased on its actions. We apply this framework for the text-transformation task\nof formalizing an input text by improving its readability grade; the degree of\nrequired formalization can be controlled by the user at run-time. Experiments\non public datasets demonstrate the efficacy of our model towards: (a)\ntransforming a given text to a more formal style, and (b) introducing\nappropriate amount of formalness in the output text pertaining to the input\ncontrol. Our code and datasets are released for academic use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04585v1",
            "date": "2018-09-12 17:50:07+00:00",
            "title": "Closed-Book Training to Improve Summarization Encoder Memory",
            "authors": [
                "Yichen Jiang",
                "Mohit Bansal"
            ],
            "abstract": "A good neural sequence-to-sequence summarization model should have a strong\nencoder that can distill and memorize the important information from long input\ntexts so that the decoder can generate salient summaries based on the encoder's\nmemory. In this paper, we aim to improve the memorization capabilities of the\nencoder of a pointer-generator model by adding an additional 'closed-book'\ndecoder without attention and pointer mechanisms. Such a decoder forces the\nencoder to be more selective in the information encoded in its memory state\nbecause the decoder can't rely on the extra information provided by the\nattention and possibly copy modules, and hence improves the entire model. On\nthe CNN/Daily Mail dataset, our 2-decoder model outperforms the baseline\nsignificantly in terms of ROUGE and METEOR metrics, for both cross-entropy and\nreinforced setups (and on human evaluation). Moreover, our model also achieves\nhigher scores in a test-only DUC-2002 generalizability setup. We further\npresent a memory ability test, two saliency metrics, as well as several\nsanity-check ablations (based on fixed-encoder, gradient-flow cut, and model\ncapacity) to prove that the encoder of our 2-decoder model does in fact learn\nstronger memory representations than the baseline encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04686v1",
            "date": "2018-09-12 21:34:03+00:00",
            "title": "Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation",
            "authors": [
                "Akiko Eriguchi",
                "Melvin Johnson",
                "Orhan Firat",
                "Hideto Kazawa",
                "Wolfgang Macherey"
            ],
            "abstract": "Transferring representations from large supervised tasks to downstream tasks\nhas shown promising results in AI fields such as Computer Vision and Natural\nLanguage Processing (NLP). In parallel, the recent progress in Machine\nTranslation (MT) has enabled one to train multilingual Neural MT (NMT) systems\nthat can translate between multiple languages and are also capable of\nperforming zero-shot translation. However, little attention has been paid to\nleveraging representations learned by a multilingual NMT system to enable\nzero-shot multilinguality in other NLP tasks. In this paper, we demonstrate a\nsimple framework, a multilingual Encoder-Classifier, for cross-lingual transfer\nlearning by reusing the encoder from a multilingual NMT system and stitching it\nwith a task-specific classifier component. Our proposed model achieves\nsignificant improvements in the English setup on three benchmark tasks - Amazon\nReviews, SST and SNLI. Further, our system can perform classification in a new\nlanguage for which no classification data was seen during training, showing\nthat zero-shot classification is possible and remarkably competitive. In order\nto understand the underlying factors contributing to this finding, we conducted\na series of analyses on the effect of the shared vocabulary, the training data\ntype for NMT, classifier complexity, encoder representation power, and model\ngeneralization on zero-shot performance. Our results provide strong evidence\nthat the representations learned from multilingual NMT systems are widely\napplicable across languages and tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04708v2",
            "date": "2018-09-12 23:23:46+00:00",
            "title": "Semantically Enhanced Models for Commonsense Knowledge Acquisition",
            "authors": [
                "Ikhlas Alhussien",
                "Erik Cambria",
                "Zhang NengSheng"
            ],
            "abstract": "Commonsense knowledge is paramount to enable intelligent systems. Typically,\nit is characterized as being implicit and ambiguous, hindering thereby the\nautomation of its acquisition. To address these challenges, this paper presents\nsemantically enhanced models to enable reasoning through resolving part of\ncommonsense ambiguity. The proposed models enhance in a knowledge graph\nembedding (KGE) framework for knowledge base completion. Experimental results\nshow the effectiveness of the new semantic models in commonsense reasoning.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.04739v2",
            "date": "2018-09-13 02:00:23+00:00",
            "title": "SafeCity: Understanding Diverse Forms of Sexual Harassment Personal Stories",
            "authors": [
                "Sweta Karlekar",
                "Mohit Bansal"
            ],
            "abstract": "With the recent rise of #MeToo, an increasing number of personal stories\nabout sexual harassment and sexual abuse have been shared online. In order to\npush forward the fight against such harassment and abuse, we present the task\nof automatically categorizing and analyzing various forms of sexual harassment,\nbased on stories shared on the online forum SafeCity. For the labels of\ngroping, ogling, and commenting, our single-label CNN-RNN model achieves an\naccuracy of 86.5%, and our multi-label model achieves a Hamming score of 82.5%.\nFurthermore, we present analysis using LIME, first-derivative saliency\nheatmaps, activation clustering, and embedding visualization to interpret\nneural model predictions and demonstrate how this extracts features that can\nhelp automatically fill out incident reports, identify unsafe areas, avoid\nunsafe practices, and 'pin the creeps'.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05157v2",
            "date": "2018-09-13 20:01:20+00:00",
            "title": "On the Strength of Character Language Models for Multilingual Named Entity Recognition",
            "authors": [
                "Xiaodong Yu",
                "Stephen Mayhew",
                "Mark Sammons",
                "Dan Roth"
            ],
            "abstract": "Character-level patterns have been widely used as features in English Named\nEntity Recognition (NER) systems. However, to date there has been no direct\ninvestigation of the inherent differences between name and non-name tokens in\ntext, nor whether this property holds across multiple languages. This paper\nanalyzes the capabilities of corpus-agnostic Character-level Language Models\n(CLMs) in the binary task of distinguishing name tokens from non-name tokens.\nWe demonstrate that CLMs provide a simple and powerful model for capturing\nthese differences, identifying named entity tokens in a diverse set of\nlanguages at close to the performance of full NER systems. Moreover, by adding\nvery simple CLM-based features we can significantly improve the performance of\nan off-the-shelf NER system for multiple languages.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05255v2",
            "date": "2018-09-14 05:00:40+00:00",
            "title": "SQL-to-Text Generation with Graph-to-Sequence Model",
            "authors": [
                "Kun Xu",
                "Lingfei Wu",
                "Zhiguo Wang",
                "Yansong Feng",
                "Vadim Sheinin"
            ],
            "abstract": "Previous work approaches the SQL-to-text generation task using vanilla\nSeq2Seq models, which may not fully capture the inherent graph-structured\ninformation in SQL query. In this paper, we first introduce a strategy to\nrepresent the SQL query as a directed graph and then employ a graph-to-sequence\nmodel to encode the global structure information into node embeddings. This\nmodel can effectively learn the correlation between the SQL query pattern and\nits interpretation. Experimental results on the WikiSQL dataset and\nStackoverflow dataset show that our model significantly outperforms the Seq2Seq\nand Tree2Seq baselines, achieving the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05524v1",
            "date": "2018-09-14 17:53:53+00:00",
            "title": "Extending Neural Generative Conversational Model using External Knowledge Sources",
            "authors": [
                "Prasanna Parthasarathi",
                "Joelle Pineau"
            ],
            "abstract": "The use of connectionist approaches in conversational agents has been\nprogressing rapidly due to the availability of large corpora. However current\ngenerative dialogue models often lack coherence and are content poor. This work\nproposes an architecture to incorporate unstructured knowledge sources to\nenhance the next utterance prediction in chit-chat type of generative dialogue\nmodels. We focus on Sequence-to-Sequence (Seq2Seq) conversational agents\ntrained with the Reddit News dataset, and consider incorporating external\nknowledge from Wikipedia summaries as well as from the NELL knowledge base. Our\nexperiments show faster training time and improved perplexity when leveraging\nexternal knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05636v1",
            "date": "2018-09-15 02:54:34+00:00",
            "title": "Geo-Text Data and Data-Driven Geospatial Semantics",
            "authors": [
                "Yingjie Hu"
            ],
            "abstract": "Many datasets nowadays contain links between geographic locations and natural\nlanguage texts. These links can be geotags, such as geotagged tweets or\ngeotagged Wikipedia pages, in which location coordinates are explicitly\nattached to texts. These links can also be place mentions, such as those in\nnews articles, travel blogs, or historical archives, in which texts are\nimplicitly connected to the mentioned places. This kind of data is referred to\nas geo-text data. The availability of large amounts of geo-text data brings\nboth challenges and opportunities. On the one hand, it is challenging to\nautomatically process this kind of data due to the unstructured texts and the\ncomplex spatial footprints of some places. On the other hand, geo-text data\noffers unique research opportunities through the rich information contained in\ntexts and the special links between texts and geography. As a result, geo-text\ndata facilitates various studies especially those in data-driven geospatial\nsemantics. This paper discusses geo-text data and related concepts. With a\nfocus on data-driven research, this paper systematically reviews a large number\nof studies that have discovered multiple types of knowledge from geo-text data.\nBased on the literature review, a generalized workflow is extracted and key\nchallenges for future work are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05679v3",
            "date": "2018-09-15 09:13:12+00:00",
            "title": "Graph Convolutional Networks for Text Classification",
            "authors": [
                "Liang Yao",
                "Chengsheng Mao",
                "Yuan Luo"
            ],
            "abstract": "Text classification is an important and classical problem in natural language\nprocessing. There have been a number of studies that applied convolutional\nneural networks (convolution on regular grid, e.g., sequence) to\nclassification. However, only a limited number of studies have explored the\nmore flexible graph convolutional neural networks (convolution on non-grid,\ne.g., arbitrary graph) for the task. In this work, we propose to use graph\nconvolutional networks for text classification. We build a single text graph\nfor a corpus based on word co-occurrence and document word relations, then\nlearn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text\nGCN is initialized with one-hot representation for word and document, it then\njointly learns the embeddings for both words and documents, as supervised by\nthe known class labels for documents. Our experimental results on multiple\nbenchmark datasets demonstrate that a vanilla Text GCN without any external\nword embeddings or knowledge outperforms state-of-the-art methods for text\nclassification. On the other hand, Text GCN also learns predictive word and\ndocument embeddings. In addition, experimental results show that the\nimprovement of Text GCN over state-of-the-art comparison methods become more\nprominent as we lower the percentage of training data, suggesting the\nrobustness of Text GCN to less training data in text classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05726v2",
            "date": "2018-09-15 14:49:23+00:00",
            "title": "Answering Science Exam Questions Using Query Rewriting with Background Knowledge",
            "authors": [
                "Ryan Musa",
                "Xiaoyan Wang",
                "Achille Fokoue",
                "Nicholas Mattei",
                "Maria Chang",
                "Pavan Kapanipathi",
                "Bassem Makni",
                "Kartik Talamadupula",
                "Michael Witbrock"
            ],
            "abstract": "Open-domain question answering (QA) is an important problem in AI and NLP\nthat is emerging as a bellwether for progress on the generalizability of AI\nmethods and techniques. Much of the progress in open-domain QA systems has been\nrealized through advances in information retrieval methods and corpus\nconstruction. In this paper, we focus on the recently introduced ARC Challenge\ndataset, which contains 2,590 multiple choice questions authored for\ngrade-school science exams. These questions are selected to be the most\nchallenging for current QA systems, and current state of the art performance is\nonly slightly better than random chance. We present a system that rewrites a\ngiven question into queries that are used to retrieve supporting text from a\nlarge corpus of science-related text. Our rewriter is able to incorporate\nbackground knowledge from ConceptNet and -- in tandem with a generic textual\nentailment system trained on SciTail that identifies support in the retrieved\nresults -- outperforms several strong baselines on the end-to-end QA task\ndespite only being trained to identify essential terms in the original source\nquestion. We use a generalizable decision methodology over the retrieved\nevidence and answer candidates to select the best answer. By combining query\nrewriting, background knowledge, and textual entailment our system is able to\noutperform several strong baselines on the ARC dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.05807v1",
            "date": "2018-09-16 03:56:21+00:00",
            "title": "Dual Memory Network Model for Biased Product Review Classification",
            "authors": [
                "Yunfei Long",
                "Mingyu Ma",
                "Qin Lu",
                "Rong Xiang",
                "Chu-Ren Huang"
            ],
            "abstract": "In sentiment analysis (SA) of product reviews, both user and product\ninformation are proven to be useful. Current tasks handle user profile and\nproduct information in a unified model which may not be able to learn salient\nfeatures of users and products effectively. In this work, we propose a dual\nuser and product memory network (DUPMN) model to learn user profiles and\nproduct reviews using separate memory networks. Then, the two representations\nare used jointly for sentiment prediction. The use of separate models aims to\ncapture user profiles and product information more effectively. Compared to\nstate-of-the-art unified prediction models, the evaluations on three benchmark\ndatasets, IMDB, Yelp13, and Yelp14, show that our dual learning model gives\nperformance gain of 0.6%, 1.2%, and 0.9%, respectively. The improvements are\nalso deemed very significant measured by p-values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05886v2",
            "date": "2018-09-16 14:36:54+00:00",
            "title": "Meta-Embedding as Auxiliary Task Regularization",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "Word embeddings have been shown to benefit from ensambling several word\nembedding sources, often carried out using straightforward mathematical\noperations over the set of word vectors. More recently, self-supervised\nlearning has been used to find a lower-dimensional representation, similar in\nsize to the individual word embeddings within the ensemble. However, these\nmethods do not use the available manually labeled datasets that are often used\nsolely for the purpose of evaluation. We propose to reconstruct an ensemble of\nword embeddings as an auxiliary task that regularises a main task while both\ntasks share the learned meta-embedding layer. We carry out intrinsic evaluation\n(6 word similarity datasets and 3 analogy datasets) and extrinsic evaluation (4\ndownstream tasks). For intrinsic task evaluation, supervision comes from\nvarious labeled word similarity datasets. Our experimental results show that\nthe performance is improved for all word similarity datasets when compared to\nself-supervised learning methods with a mean increase of $11.33$ in Spearman\ncorrelation. Specifically, the proposed method shows the best performance in 4\nout of 6 of word similarity datasets when using a cosine reconstruction loss\nand Brier's word similarity loss. Moreover, improvements are also made when\nperforming word meta-embedding reconstruction in sequence tagging and sentence\nmeta-embedding for sentence classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05972v5",
            "date": "2018-09-16 22:45:51+00:00",
            "title": "Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization",
            "authors": [
                "Yizhe Zhang",
                "Michel Galley",
                "Jianfeng Gao",
                "Zhe Gan",
                "Xiujun Li",
                "Chris Brockett",
                "Bill Dolan"
            ],
            "abstract": "Responses generated by neural conversational models tend to lack\ninformativeness and diversity. We present Adversarial Information Maximization\n(AIM), an adversarial learning strategy that addresses these two related but\ndistinct problems. To foster response diversity, we leverage adversarial\ntraining that allows distributional matching of synthetic and real responses.\nTo improve informativeness, our framework explicitly optimizes a variational\nlower bound on pairwise mutual information between query and response.\nEmpirical results from automatic and human evaluations demonstrate that our\nmethods significantly boost informativeness and diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06309v3",
            "date": "2018-09-17 16:24:00+00:00",
            "title": "Commonsense for Generative Multi-Hop Question Answering Tasks",
            "authors": [
                "Lisa Bauer",
                "Yicheng Wang",
                "Mohit Bansal"
            ],
            "abstract": "Reading comprehension QA tasks have seen a recent surge in popularity, yet\nmost works have focused on fact-finding extractive QA. We instead focus on a\nmore challenging multi-hop generative task (NarrativeQA), which requires the\nmodel to reason, gather, and synthesize disjoint pieces of information within\nthe context to generate an answer. This type of multi-step reasoning also often\nrequires understanding implicit relations, which humans resolve via external,\nbackground commonsense knowledge. We first present a strong generative baseline\nthat uses a multi-attention mechanism to perform multiple hops of reasoning and\na pointer-generator decoder to synthesize the answer. This model performs\nsubstantially better than previous generative models, and is competitive with\ncurrent state-of-the-art span prediction models. We next introduce a novel\nsystem for selecting grounded multi-hop relational commonsense information from\nConceptNet via a pointwise mutual information and term-frequency based scoring\nfunction. Finally, we effectively use this extracted commonsense information to\nfill in gaps of reasoning between context hops, using a selectively-gated\nattention mechanism. This boosts the model's performance significantly (also\nverified via human evaluation), establishing a new state-of-the-art for the\ntask. We also show promising initial results of the generalizability of our\nbackground knowledge enhancements by demonstrating some improvement on\nQAngaroo-WikiHop, another multi-hop reasoning dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06366v1",
            "date": "2018-09-15 22:11:03+00:00",
            "title": "AUEB at BioASQ 6: Document and Snippet Retrieval",
            "authors": [
                "Georgios-Ioannis Brokos",
                "Polyvios Liosis",
                "Ryan McDonald",
                "Dimitris Pappas",
                "Ion Androutsopoulos"
            ],
            "abstract": "We present AUEB's submissions to the BioASQ 6 document and snippet retrieval\ntasks (parts of Task 6b, Phase A). Our models use novel extensions to deep\nlearning architectures that operate solely over the text of the query and\ncandidate document/snippets. Our systems scored at the top or near the top for\nall batches of the challenge, highlighting the effectiveness of deep learning\nfor these tasks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.06416v1",
            "date": "2018-09-17 19:51:18+00:00",
            "title": "DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning",
            "authors": [
                "Kashyap Popat",
                "Subhabrata Mukherjee",
                "Andrew Yates",
                "Gerhard Weikum"
            ],
            "abstract": "Misinformation such as fake news is one of the big challenges of our society.\nResearch on automated fact-checking has proposed methods based on supervised\nlearning, but these approaches do not consider external evidence apart from\nlabeled training instances. Recent approaches counter this deficit by\nconsidering external sources related to a claim. However, these methods require\nsubstantial feature modeling and rich lexicons. This paper overcomes these\nlimitations of prior work with an end-to-end model for evidence-aware\ncredibility assessment of arbitrary textual claims, without any human\nintervention. It presents a neural network model that judiciously aggregates\nsignals from external evidence articles, the language of these articles and the\ntrustworthiness of their sources. It also derives informative features for\ngenerating user-comprehensible explanations that makes the neural network\npredictions transparent to the end-user. Experiments with four datasets and\nablation studies show the strength of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06491v1",
            "date": "2018-09-18 00:38:30+00:00",
            "title": "Triad-based Neural Network for Coreference Resolution",
            "authors": [
                "Yuanliang Meng",
                "Anna Rumshisky"
            ],
            "abstract": "We propose a triad-based neural network system that generates affinity scores\nbetween entity mentions for coreference resolution. The system simultaneously\naccepts three mentions as input, taking mutual dependency and logical\nconstraints of all three mentions into account, and thus makes more accurate\npredictions than the traditional pairwise approach. Depending on system\nchoices, the affinity scores can be further used in clustering or mention\nranking. Our experiments show that a standard hierarchical clustering using the\nscores produces state-of-art results with gold mentions on the English portion\nof CoNLL 2012 Shared Task. The model does not rely on many handcrafted features\nand is easy to train and use. The triads can also be easily extended to polyads\nof higher orders. To our knowledge, this is the first neural network system to\nmodel mutual dependency of more than two members at mention level.",
            "journal ref": "Proceedings of 27th International Conference on Computational\n  Linguistics (2018) 35-43",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.06537v1",
            "date": "2018-09-18 05:26:40+00:00",
            "title": "Automatic Judgment Prediction via Legal Reading Comprehension",
            "authors": [
                "Shangbang Long",
                "Cunchao Tu",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Automatic judgment prediction aims to predict the judicial results based on\ncase materials. It has been studied for several decades mainly by lawyers and\njudges, considered as a novel and prospective application of artificial\nintelligence techniques in the legal field. Most existing methods follow the\ntext classification framework, which fails to model the complex interactions\namong complementary case materials. To address this issue, we formalize the\ntask as Legal Reading Comprehension according to the legal scenario. Following\nthe working protocol of human judges, LRC predicts the final judgment results\nbased on three types of information, including fact description, plaintiffs'\npleas, and law articles. Moreover, we propose a novel LRC model, AutoJudge,\nwhich captures the complex semantic interactions among facts, pleas, and laws.\nIn experiments, we construct a real-world civil case dataset for LRC.\nExperimental results on this dataset demonstrate that our model achieves\nsignificant improvement over state-of-the-art models. We will publish all\nsource codes and datasets of this work on \\urlgithub.com for further research.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.06559v1",
            "date": "2018-09-18 07:08:59+00:00",
            "title": "User Information Augmented Semantic Frame Parsing using Coarse-to-Fine Neural Networks",
            "authors": [
                "Yilin Shen",
                "Xiangyu Zeng",
                "Yu Wang",
                "Hongxia Jin"
            ],
            "abstract": "Semantic frame parsing is a crucial component in spoken language\nunderstanding (SLU) to build spoken dialog systems. It has two main tasks:\nintent detection and slot filling. Although state-of-the-art approaches showed\ngood results, they require large annotated training data and long training\ntime. In this paper, we aim to alleviate these drawbacks for semantic frame\nparsing by utilizing the ubiquitous user information. We design a novel\ncoarse-to-fine deep neural network model to incorporate prior knowledge of user\ninformation intermediately to better and quickly train a semantic frame parser.\nDue to the lack of benchmark dataset with real user information, we synthesize\nthe simplest type of user information (location and time) on ATIS benchmark\ndata. The results show that our approach leverages such simple user information\nto outperform state-of-the-art approaches by 0.25% for intent detection and\n0.31% for slot filling using standard training data. When using smaller\ntraining data, the performance improvement on intent detection and slot filling\nreaches up to 1.35% and 1.20% respectively. We also show that our approach can\nachieve similar performance as state-of-the-art approaches by using less than\n80% annotated training data. Moreover, the training time to achieve the similar\nperformance is also reduced by over 60%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06639v1",
            "date": "2018-09-18 10:55:03+00:00",
            "title": "Lung Cancer Concept Annotation from Spanish Clinical Narratives",
            "authors": [
                "Marjan Najafabadipour",
                "Juan Manuel Tu\u00f1as",
                "Alejandro Rodr\u00edguez-Gonz\u00e1lez",
                "Ernestina Menasalvas"
            ],
            "abstract": "Recent rapid increase in the generation of clinical data and rapid\ndevelopment of computational science make us able to extract new insights from\nmassive datasets in healthcare industry. Oncological clinical notes are\ncreating rich databases for documenting patients history and they potentially\ncontain lots of patterns that could help in better management of the disease.\nHowever, these patterns are locked within free text (unstructured) portions of\nclinical documents and consequence in limiting health professionals to extract\nuseful information from them and to finally perform Query and Answering (QA)\nprocess in an accurate way. The Information Extraction (IE) process requires\nNatural Language Processing (NLP) techniques to assign semantics to these\npatterns. Therefore, in this paper, we analyze the design of annotators for\nspecific lung cancer concepts that can be integrated over Apache Unstructured\nInformation Management Architecture (UIMA) framework. In addition, we explain\nthe details of generation and storage of annotation outcomes.",
            "journal ref": "Data Integration in the Life Sciences (DILS 2018)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.06641v2",
            "date": "2018-09-18 11:09:49+00:00",
            "title": "Talking to myself: self-dialogues as data for conversational agents",
            "authors": [
                "Joachim Fainberg",
                "Ben Krause",
                "Mihai Dobre",
                "Marco Damonte",
                "Emmanuel Kahembwe",
                "Daniel Duma",
                "Bonnie Webber",
                "Federico Fancellu"
            ],
            "abstract": "Conversational agents are gaining popularity with the increasing ubiquity of\nsmart devices. However, training agents in a data driven manner is challenging\ndue to a lack of suitable corpora. This paper presents a novel method for\ngathering topical, unstructured conversational data in an efficient way:\nself-dialogues through crowd-sourcing. Alongside this paper, we include a\ncorpus of 3.6 million words across 23 topics. We argue the utility of the\ncorpus by comparing self-dialogues with standard two-party conversations as\nwell as data from other corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06951v1",
            "date": "2018-09-18 22:03:48+00:00",
            "title": "Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's Neutrality Norm",
            "authors": [
                "Umashanthi Pavalanathan",
                "Xiaochuang Han",
                "Jacob Eisenstein"
            ],
            "abstract": "Wikipedia has a strong norm of writing in a 'neutral point of view' (NPOV).\nArticles that violate this norm are tagged, and editors are encouraged to make\ncorrections. But the impact of this tagging system has not been quantitatively\nmeasured. Does NPOV tagging help articles to converge to the desired style? Do\nNPOV corrections encourage editors to adopt this style? We study these\nquestions using a corpus of NPOV-tagged articles and a set of lexicons\nassociated with biased language. An interrupted time series analysis shows that\nafter an article is tagged for NPOV, there is a significant decrease in biased\nlanguage in the article, as measured by several lexicons. However, for\nindividual editors, NPOV corrections and talk page discussions yield no\nsignificant change in the usage of words in most of these lexicons, including\nWikipedia's own list of 'words to watch.' This suggests that NPOV tagging and\ndiscussion does improve content, but has less success enculturating editors to\nthe site's linguistic norms.",
            "journal ref": "Umashanthi Pavalanathan, Xiaochuang Han, and Jacob Eisenstein.\n  2018. Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's\n  Neutrality Norm. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 137\n  (November 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06963v3",
            "date": "2018-09-18 23:44:03+00:00",
            "title": "Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension",
            "authors": [
                "Yichong Xu",
                "Xiaodong Liu",
                "Yelong Shen",
                "Jingjing Liu",
                "Jianfeng Gao"
            ],
            "abstract": "We propose a multi-task learning framework to learn a joint Machine Reading\nComprehension (MRC) model that can be applied to a wide range of MRC tasks in\ndifferent domains. Inspired by recent ideas of data selection in machine\ntranslation, we develop a novel sample re-weighting scheme to assign\nsample-specific weights to the loss. Empirical study shows that our approach\ncan be applied to many existing MRC models. Combined with contextual\nrepresentations from pre-trained language models (such as ELMo), we achieve new\nstate-of-the-art results on a set of MRC benchmark datasets. We release our\ncode at https://github.com/xycforgithub/MultiTask-MRC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07607v2",
            "date": "2018-09-20 13:20:58+00:00",
            "title": "Syntactico-Semantic Reasoning using PCFG, MEBN & PP Attachment Ambiguity",
            "authors": [
                "Shrinivasan R Patnaik Patnaikuni",
                "Dr. Sachin R Gengaje"
            ],
            "abstract": "Probabilistic context free grammars (PCFG) have been the core of the\nprobabilistic reasoning based parsers for several years especially in the\ncontext of the NLP. Multi entity bayesian networks (MEBN) a First Order Logic\nprobabilistic reasoning methodology is widely adopted and used method for\nuncertainty reasoning. Further upper ontology like Probabilistic Ontology Web\nLanguage (PR-OWL) built using MEBN takes care of probabilistic ontologies which\nmodel and capture the uncertainties inherent in the domain's semantic\ninformation. The paper attempts to establish a link between probabilistic\nreasoning in PCFG and MEBN by proposing a formal description of PCFG driven by\nMEBN leading to usage of PR-OWL modeled ontologies in PCFG parsers.\nFurthermore, the paper outlines an approach to resolve prepositional phrase\n(PP) attachment ambiguity using the proposed mapping between PCFG and MEBN.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.07950v2",
            "date": "2018-09-21 05:48:54+00:00",
            "title": "CollaboNet: collaboration of deep neural networks for biomedical named entity recognition",
            "authors": [
                "Wonjin Yoon",
                "Chan Ho So",
                "Jinhyuk Lee",
                "Jaewoo Kang"
            ],
            "abstract": "Background: Finding biomedical named entities is one of the most essential\ntasks in biomedical text mining. Recently, deep learning-based approaches have\nbeen applied to biomedical named entity recognition (BioNER) and showed\npromising results. However, as deep learning approaches need an abundant amount\nof training data, a lack of data can hinder performance. BioNER datasets are\nscarce resources and each dataset covers only a small subset of entity types.\nFurthermore, many bio entities are polysemous, which is one of the major\nobstacles in named entity recognition. Results: To address the lack of data and\nthe entity type misclassification problem, we propose CollaboNet which utilizes\na combination of multiple NER models. In CollaboNet, models trained on a\ndifferent dataset are connected to each other so that a target model obtains\ninformation from other collaborator models to reduce false positives. Every\nmodel is an expert on their target entity type and takes turns serving as a\ntarget and a collaborator model during training time. The experimental results\nshow that CollaboNet can be used to greatly reduce the number of false\npositives and misclassified entities including polysemous words. CollaboNet\nachieved state-of-the-art performance in terms of precision, recall and F1\nscore. Conclusions: We demonstrated the benefits of combining multiple models\nfor BioNER. Our model has successfully reduced the number of misclassified\nentities and improved the performance by leveraging multiple datasets annotated\nfor different entity types. Given the state-of-the-art performance of our\nmodel, we believe that CollaboNet can improve the accuracy of downstream\nbiomedical text mining applications such as bio-entity relation extraction.",
            "journal ref": "BMC Bioinformatics 2019, 20(Suppl 10):249",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08390v1",
            "date": "2018-09-22 06:35:07+00:00",
            "title": "Constructing Financial Sentimental Factors in Chinese Market Using Natural Language Processing",
            "authors": [
                "Junfeng Jiang",
                "Jiahao Li"
            ],
            "abstract": "In this paper, we design an integrated algorithm to evaluate the sentiment of\nChinese market. Firstly, with the help of the web browser automation, we crawl\na lot of news and comments from several influential financial websites\nautomatically. Secondly, we use techniques of Natural Language Processing(NLP)\nunder Chinese context, including tokenization, Word2vec word embedding and\nsemantic database WordNet, to compute Senti-scores of these news and comments,\nand then construct the sentimental factor. Here, we build a finance-specific\nsentimental lexicon so that the sentimental factor can reflect the sentiment of\nfinancial market but not the general sentiments as happiness, sadness, etc.\nThirdly, we also implement an adjustment of the standard sentimental factor.\nOur experimental performance shows that there is a significant correlation\nbetween our standard sentimental factor and the Chinese market, and the\nadjusted factor is even more informative, having a stronger correlation with\nthe Chinese market. Therefore, our sentimental factors can be important\nreferences when making investment decisions. Especially during the Chinese\nmarket crash in 2015, the Pearson correlation coefficient of adjusted\nsentimental factor with SSE is 0.5844, which suggests that our model can\nprovide a solid guidance, especially in the special period when the market is\ninfluenced greatly by public sentiment.",
            "journal ref": "None",
            "category": "q-fin.CP"
        },
        {
            "url": "http://arxiv.org/abs/1809.08697v1",
            "date": "2018-09-23 23:45:54+00:00",
            "title": "Textually Enriched Neural Module Networks for Visual Question Answering",
            "authors": [
                "Khyathi Raghavi Chandu",
                "Mary Arpita Pyreddy",
                "Matthieu Felix",
                "Narendra Nath Joshi"
            ],
            "abstract": "Problems at the intersection of language and vision, like visual question\nanswering, have recently been gaining a lot of attention in the field of\nmulti-modal machine learning as computer vision research moves beyond\ntraditional recognition tasks. There has been recent success in visual question\nanswering using deep neural network models which use the linguistic structure\nof the questions to dynamically instantiate network layouts. In the process of\nconverting the question to a network layout, the question is simplified, which\nresults in loss of information in the model. In this paper, we enrich the image\ninformation with textual data using image captions and external knowledge bases\nto generate more coherent answers. We achieve 57.1% overall accuracy on the\ntest-dev open-ended questions from the visual question answering (VQA 1.0) real\nimage dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08711v1",
            "date": "2018-09-24 01:06:46+00:00",
            "title": "Recognizing Film Entities in Podcasts",
            "authors": [
                "Ahmet Salih Gundogdu",
                "Arjun Sanghvi",
                "Keith Harrigian"
            ],
            "abstract": "In this paper, we propose a Named Entity Recognition (NER) system to identify\nfilm titles in podcast audio. Taking inspiration from NER systems for noisy\ntext in social media, we implement a two-stage approach that is robust to\ncomputer transcription errors and does not require significant computational\nexpense to accommodate new film titles/releases. Evaluating on a diverse set of\npodcasts, we demonstrate more than a 20% increase in F1 score across three\nbaseline approaches when combining fuzzy-matching with a linear model aware of\nfilm-specific metadata.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.08761v1",
            "date": "2018-09-24 05:00:05+00:00",
            "title": "Speaker Naming in Movies",
            "authors": [
                "Mahmoud Azab",
                "Mingzhe Wang",
                "Max Smith",
                "Noriyuki Kojima",
                "Jia Deng",
                "Rada Mihalcea"
            ],
            "abstract": "We propose a new model for speaker naming in movies that leverages visual,\ntextual, and acoustic modalities in an unified optimization framework. To\nevaluate the performance of our model, we introduce a new dataset consisting of\nsix episodes of the Big Bang Theory TV show and eighteen full movies covering\ndifferent genres. Our experiments show that our multimodal model significantly\noutperforms several competitive baselines on the average weighted F-score\nmetric. To demonstrate the effectiveness of our framework, we design an\nend-to-end memory network model that leverages our speaker naming model and\nachieves state-of-the-art results on the subtitles task of the MovieQA 2017\nChallenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08826v1",
            "date": "2018-09-24 10:07:27+00:00",
            "title": "Information-Weighted Neural Cache Language Models for ASR",
            "authors": [
                "Lyan Verwimp",
                "Joris Pelemans",
                "Hugo Van hamme",
                "Patrick Wambacq"
            ],
            "abstract": "Neural cache language models (LMs) extend the idea of regular cache language\nmodels by making the cache probability dependent on the similarity between the\ncurrent context and the context of the words in the cache. We make an extensive\ncomparison of 'regular' cache models with neural cache models, both in terms of\nperplexity and WER after rescoring first-pass ASR results. Furthermore, we\npropose two extensions to this neural cache model that make use of the content\nvalue/information weight of the word: firstly, combining the cache probability\nand LM probability with an information-weighted interpolation and secondly,\nselectively adding only content words to the cache. We obtain a 29.9%/32.1%\n(validation/test set) relative improvement in perplexity with respect to a\nbaseline LSTM LM on the WikiText-2 dataset, outperforming previous work on\nneural cache LMs. Additionally, we observe significant WER reductions with\nrespect to the baseline model on the WSJ ASR task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08887v5",
            "date": "2018-09-24 13:03:13+00:00",
            "title": "Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task",
            "authors": [
                "Tao Yu",
                "Rui Zhang",
                "Kai Yang",
                "Michihiro Yasunaga",
                "Dongxu Wang",
                "Zifan Li",
                "James Ma",
                "Irene Li",
                "Qingning Yao",
                "Shanelle Roman",
                "Zilin Zhang",
                "Dragomir Radev"
            ],
            "abstract": "We present Spider, a large-scale, complex and cross-domain semantic parsing\nand text-to-SQL dataset annotated by 11 college students. It consists of 10,181\nquestions and 5,693 unique complex SQL queries on 200 databases with multiple\ntables, covering 138 different domains. We define a new complex and\ncross-domain semantic parsing and text-to-SQL task where different complex SQL\nqueries and databases appear in train and test sets. In this way, the task\nrequires the model to generalize well to both new SQL queries and new database\nschemas. Spider is distinct from most of the previous semantic parsing tasks\nbecause they all use a single database and the exact same programs in the train\nset and the test set. We experiment with various state-of-the-art models and\nthe best model achieves only 12.4% exact matching accuracy on a database split\nsetting. This shows that Spider presents a strong challenge for future\nresearch. Our dataset and task are publicly available at\nhttps://yale-lily.github.io/spider",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08935v1",
            "date": "2018-09-21 12:04:44+00:00",
            "title": "Lexical Bias In Essay Level Prediction",
            "authors": [
                "Georgios Balikas"
            ],
            "abstract": "Automatically predicting the level of non-native English speakers given their\nwritten essays is an interesting machine learning problem. In this work I\npresent the system \"balikasg\" that achieved the state-of-the-art performance in\nthe CAp 2018 data science challenge among 14 systems. I detail the feature\nextraction, feature engineering and model selection steps and I evaluate how\nthese decisions impact the system's performance. The paper concludes with\nremarks for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08962v2",
            "date": "2018-09-24 14:19:59+00:00",
            "title": "WiRe57 : A Fine-Grained Benchmark for Open Information Extraction",
            "authors": [
                "William L\u00e9chelle",
                "Fabrizio Gotti",
                "Philippe Langlais"
            ],
            "abstract": "We build a reference for the task of Open Information Extraction, on five\ndocuments. We tentatively resolve a number of issues that arise, including\ninference and granularity. We seek to better pinpoint the requirements for the\ntask. We produce our annotation guidelines specifying what is correct to\nextract and what is not. In turn, we use this reference to score existing Open\nIE systems. We address the non-trivial problem of evaluating the extractions\nproduced by systems against the reference tuples, and share our evaluation\nscript. Among seven compared extractors, we find the MinIE system to perform\nbest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09408v2",
            "date": "2018-09-25 10:56:20+00:00",
            "title": "Chinese User Service Intention Classification Based on Hybrid Neural Network",
            "authors": [
                "Shengbin Jia",
                "Yang Xiang"
            ],
            "abstract": "In order to satisfy the consumers' increasing personalized service demand,\nthe Intelligent service has arisen. User service intention recognition is an\nimportant challenge for intelligent service system to provide precise service.\nIt is difficult for the intelligent system to understand the semantics of user\ndemand which leads to poor recognition effect, because of the noise in user\nrequirement descriptions. Therefore, a hybrid neural network classification\nmodel based on BiLSTM and CNN is proposed to recognize users service\nintentions. The model can fuse the temporal semantics and spatial semantics of\nthe user descriptions. The experimental results show that our model achieves a\nbetter effect compared with other models, reaching 0.94 on the F1 score.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.09548v1",
            "date": "2018-09-25 15:29:11+00:00",
            "title": "Coordinating and Integrating Faceted Classification with Rich Semantic Modeling",
            "authors": [
                "Robert B. Allen",
                "Jaihyun Park"
            ],
            "abstract": "Faceted classifications define dimensions for the types of entities included.\nIn effect, the facets provide an \"ontological commitment\". We compare a faceted\nthesaurus, the Art and Architecture Thesaurus (AAT), with ontologies derived\nfrom the Basic Formal Ontology (BFO2), which is an upper (or formal) ontology\nwidely used to describe entities in biomedicine. We consider how the AAT and\nBFO2-based ontologies could be coordinated and integrated into a Human Activity\nand Infrastructure Foundry (HAIF). To extend the AAT to enable this\ncoordination and integration, we describe how a wider range of relationships\namong its terms could be introduced. Using these extensions, we explore richer\nmodeling of topics from AAT that deal with Technology. Finally, we consider how\nontology-based frames and semantic role frames can be integrated to make rich\nsemantic statements about changes in the world.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10044v2",
            "date": "2018-09-26 15:01:28+00:00",
            "title": "No One is Perfect: Analysing the Performance of Question Answering Components over the DBpedia Knowledge Graph",
            "authors": [
                "Kuldeep Singh",
                "Ioanna Lytra",
                "Arun Sethupat Radhakrishna",
                "Saeedeh Shekarpour",
                "Maria-Esther Vidal",
                "Jens Lehmann"
            ],
            "abstract": "Question answering (QA) over knowledge graphs has gained significant momentum\nover the past five years due to the increasing availability of large knowledge\ngraphs and the rising importance of question answering for user interaction.\nDBpedia has been the most prominently used knowledge graph in this setting and\nmost approaches currently use a pipeline of processing steps connecting a\nsequence of components. In this article, we analyse and micro evaluate the\nbehaviour of 29 available QA components for DBpedia knowledge graph that were\nreleased by the research community since 2010. As a result, we provide a\nperspective on collective failure cases, suggest characteristics of QA\ncomponents that prevent them from performing better and provide future\nchallenges and research directions for the field.",
            "journal ref": "Journal of Web Semantics (JWS 2020)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.10236v1",
            "date": "2018-09-19 21:58:14+00:00",
            "title": "Learning to Interpret Satellite Images Using Wikipedia",
            "authors": [
                "Evan Sheehan",
                "Burak Uzkent",
                "Chenlin Meng",
                "Zhongyi Tang",
                "Marshall Burke",
                "David Lobell",
                "Stefano Ermon"
            ],
            "abstract": "Despite recent progress in computer vision, fine-grained interpretation of\nsatellite images remains challenging because of a lack of labeled training\ndata. To overcome this limitation, we propose using Wikipedia as a previously\nuntapped source of rich, georeferenced textual information with global\ncoverage. We construct a novel large-scale, multi-modal dataset by pairing\ngeo-referenced Wikipedia articles with satellite imagery of their corresponding\nlocations. To prove the efficacy of this dataset, we focus on the African\ncontinent and train a deep network to classify images based on labels extracted\nfrom articles. We then fine-tune the model on a human annotated dataset and\ndemonstrate that this weak form of supervision can drastically reduce the\nquantity of human annotated labels and time required for downstream tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.10267v1",
            "date": "2018-09-26 23:38:19+00:00",
            "title": "Semantic Sentence Embeddings for Paraphrasing and Text Summarization",
            "authors": [
                "Chi Zhang",
                "Shagan Sah",
                "Thang Nguyen",
                "Dheeraj Peri",
                "Alexander Loui",
                "Carl Salvaggio",
                "Raymond Ptucha"
            ],
            "abstract": "This paper introduces a sentence to vector encoding framework suitable for\nadvanced natural language processing. Our latent representation is shown to\nencode sentences with common semantic information with similar vector\nrepresentations. The vector representation is extracted from an encoder-decoder\nmodel which is trained on sentence paraphrase pairs. We demonstrate the\napplication of the sentence representations for two different tasks -- sentence\nparaphrasing and paragraph summarization, making it attractive for commonly\nused recurrent frameworks that process text. Experimental results help gain\ninsight how vector representations are suitable for advanced language\nembedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10282v1",
            "date": "2018-09-27 00:41:16+00:00",
            "title": "Adaptive Pruning of Neural Language Models for Mobile Devices",
            "authors": [
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "Neural language models (NLMs) exist in an accuracy-efficiency tradeoff space\nwhere better perplexity typically comes at the cost of greater computation\ncomplexity. In a software keyboard application on mobile devices, this\ntranslates into higher power consumption and shorter battery life. This paper\nrepresents the first attempt, to our knowledge, in exploring\naccuracy-efficiency tradeoffs for NLMs. Building on quasi-recurrent neural\nnetworks (QRNNs), we apply pruning techniques to provide a \"knob\" to select\ndifferent operating points. In addition, we propose a simple technique to\nrecover some perplexity using a negligible amount of memory. Our empirical\nevaluations consider both perplexity as well as energy consumption on a\nRaspberry Pi, where we demonstrate which methods provide the best\nperplexity-power consumption operating point. At one operating point, one of\nthe techniques is able to provide energy savings of 40% over the state of the\nart with only a 17% relative increase in perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10617v1",
            "date": "2018-09-27 16:28:18+00:00",
            "title": "Enabling FAIR Research in Earth Science through Research Objects",
            "authors": [
                "Andres Garcia-Silva",
                "Jose Manuel Gomez-Perez",
                "Raul Palma",
                "Marcin Krystek",
                "Simone Mantovani",
                "Federica Foglini",
                "Valentina Grande",
                "Francesco De Leo",
                "Stefano Salvi",
                "Elisa Trasati",
                "Vito Romaniello",
                "Mirko Albani",
                "Cristiano Silvagni",
                "Rosemarie Leone",
                "Fulvio Marelli",
                "Sergio Albani",
                "Michele Lazzarini",
                "Hazel J. Napier",
                "Helen M. Glaves",
                "Timothy Aldridge",
                "Charles Meertens",
                "Fran Boler",
                "Henry W. Loescher",
                "Christine Laney",
                "Melissa A Genazzio",
                "Daniel Crawl",
                "Ilkay Altintas"
            ],
            "abstract": "Data-intensive science communities are progressively adopting FAIR practices\nthat enhance the visibility of scientific breakthroughs and enable reuse. At\nthe core of this movement, research objects contain and describe scientific\ninformation and resources in a way compliant with the FAIR principles and\nsustain the development of key infrastructure and tools. This paper provides an\naccount of the challenges, experiences and solutions involved in the adoption\nof FAIR around research objects over several Earth Science disciplines. During\nthis journey, our work has been comprehensive, with outcomes including: an\nextended research object model adapted to the needs of earth scientists; the\nprovisioning of digital object identifiers (DOI) to enable persistent\nidentification and to give due credit to authors; the generation of\ncontent-based, semantically rich, research object metadata through natural\nlanguage processing, enhancing visibility and reuse through recommendation\nsystems and third-party search engines; and various types of checklists that\nprovide a compact representation of research object quality as a key enabler of\nscientific reuse. All these results have been integrated in ROHub, a platform\nthat provides research object management functionality to a wealth of\napplications and interfaces across different scientific communities. To monitor\nand quantify the community uptake of research objects, we have defined\nindicators and obtained measures via ROHub that are also discussed herein.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.11047v1",
            "date": "2018-09-28 14:11:26+00:00",
            "title": "Cross-situational learning of large lexicons with finite memory",
            "authors": [
                "James Holehouse",
                "Richard A. Blythe"
            ],
            "abstract": "Cross-situational word learning, wherein a learner combines information about\npossible meanings of a word across multiple exposures, has previously been\nshown to be a very powerful strategy to acquire a large lexicon in a short\ntime. However, this success may derive from idealizations that are made when\nmodeling the word-learning process. In particular, an earlier model assumed\nthat a learner could perfectly recall all previous instances of a word's use\nand the inferences that were drawn about its meaning. In this work, we relax\nthis assumption and determine the performance of a model cross-situational\nlearner who forgets word-meaning associations over time. Our main finding is\nthat it is possible for this learner to acquire a human-scale lexicon by\nadulthood with word-exposure and memory-decay rates that are consistent with\nempirical research on childhood word learning, as long as the degree of\nreferential uncertainty is not too high or the learner employs a mutual\nexclusivity constraint. Our findings therefore suggest that successful word\nlearning does not necessarily demand either highly accurate long-term tracking\nof word and meaning statistics or hypothesis-testing strategies.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1809.11155v2",
            "date": "2018-09-28 17:38:36+00:00",
            "title": "SALSA-TEXT : self attentive latent space based adversarial text generation",
            "authors": [
                "Jules Gagnon-Marchand",
                "Hamed Sadeghi",
                "Md. Akmal Haidar",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "Inspired by the success of self attention mechanism and Transformer\narchitecture in sequence transduction and image generation applications, we\npropose novel self attention-based architectures to improve the performance of\nadversarial latent code- based schemes in text generation. Adversarial latent\ncode-based text generation has recently gained a lot of attention due to their\npromising results. In this paper, we take a step to fortify the architectures\nused in these setups, specifically AAE and ARAE. We benchmark two latent\ncode-based methods (AAE and ARAE) designed based on adversarial setups. In our\nexperiments, the Google sentence compression dataset is utilized to compare our\nmethod with these methods using various objective and subjective measures. The\nexperiments demonstrate the proposed (self) attention-based models outperform\nthe state-of-the-art in adversarial code-based text generation.",
            "journal ref": "Canadian AI 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00324v3",
            "date": "2018-09-30 06:25:31+00:00",
            "title": "On the Winograd Schema: Situating Language Understanding in the Data-Information-Knowledge Continuum",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "The Winograd Schema (WS) challenge, proposed as an al-ternative to the Turing\nTest, has become the new standard for evaluating progress in natural language\nunderstanding (NLU). In this paper we will not however be concerned with how\nthis challenge might be addressed. Instead, our aim here is threefold: (i) we\nwill first formally 'situate' the WS challenge in the\ndata-information-knowledge continuum, suggesting where in that continuum a good\nWS resides; (ii) we will show that a WS is just special case of a more general\nphenomenon in language understanding, namely the missing text phenomenon\n(henceforth, MTP) - in particular, we will argue that what we usually call\nthinking in the process of language understanding involves discovering a\nsignificant amount of 'missing text' - text that is not explicitly stated, but\nis often implicitly assumed as shared background knowledge; and (iii) we\nconclude by a brief discussion on why MTP is inconsistent with the data-driven\nand machine learning approach to language understanding.",
            "journal ref": "Proceedings of the 32nd International FLAIRS Conference, 2019",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.00347v1",
            "date": "2018-09-30 09:28:57+00:00",
            "title": "Neural Entity Reasoner for Global Consistency in NER",
            "authors": [
                "Xiaoxiao Yin",
                "Daqi Zheng",
                "Zhengdong Lu",
                "Ruifang Liu"
            ],
            "abstract": "We propose Neural Entity Reasoner (NE-Reasoner), a framework to introduce\nglobal consistency of recognized entities into Neural Reasoner over Named\nEntity Recognition (NER) task. Given an input sentence, the NE-Reasoner layer\ncan infer over multiple entities to increase the global consistency of output\nlabels, which then be transfered into entities for the input of next layer.\nNE-Reasoner inherits and develops some features from Neural Reasoner 1) a\nsymbolic memory, allowing it to exchange entities between layers. 2) the\nspecific interaction-pooling mechanism, allowing it to connect each local word\nto multiple global entities, and 3) the deep architecture, allowing it to\nbootstrap the recognized entity set from coarse to fine. Like human beings,\nNE-Reasoner is able to accommodate ambiguous words and Name Entities that\nrarely or never met before. Despite the symbolic information the model\nintroduced, NE-Reasoner can still be trained effectively in an end-to-end\nmanner via parameter sharing strategy. NE-Reasoner can outperform conventional\nNER models in most cases on both English and Chinese NER datasets. For example,\nit achieves state-of-art on CoNLL-2003 English NER dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00494v1",
            "date": "2018-10-01 00:51:25+00:00",
            "title": "Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering",
            "authors": [
                "Jinhyuk Lee",
                "Seongjun Yun",
                "Hyunjae Kim",
                "Miyoung Ko",
                "Jaewoo Kang"
            ],
            "abstract": "Recently, open-domain question answering (QA) has been combined with machine\ncomprehension models to find answers in a large knowledge source. As\nopen-domain QA requires retrieving relevant documents from text corpora to\nanswer questions, its performance largely depends on the performance of\ndocument retrievers. However, since traditional information retrieval systems\nare not effective in obtaining documents with a high probability of containing\nanswers, they lower the performance of QA systems. Simply extracting more\ndocuments increases the number of irrelevant documents, which also degrades the\nperformance of QA systems. In this paper, we introduce Paragraph Ranker which\nranks paragraphs of retrieved documents for a higher answer recall with less\nnoise. We show that ranking paragraphs and aggregating answers using Paragraph\nRanker improves performance of open-domain QA pipeline on the four open-domain\nQA datasets by 7.8% on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00647v2",
            "date": "2018-09-28 16:00:20+00:00",
            "title": "Real Time Monitoring of Social Media and Digital Press",
            "authors": [
                "I\u00f1aki San Vicente",
                "Xabier Saralegi",
                "Rodrigo Agerri"
            ],
            "abstract": "Talaia is a platform for monitoring social media and digital press. A\nconfigurable crawler gathers content with respect to user defined domains or\ntopics. Crawled data is processed by means of the EliXa Sentiment Analysis\nsystem. A Django powered interface provides data visualization for a user-based\nanalysis of the data. This paper presents the architecture of the system and\ndescribes in detail its different components. To prove the validity of the\napproach, two real use cases are accounted for: one in the cultural domain and\none in the political domain. Evaluation for the sentiment analysis task in both\nscenarios is also provided, showing the capacity for domain adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00660v1",
            "date": "2018-09-21 23:47:42+00:00",
            "title": "Attention-based Encoder-Decoder Networks for Spelling and Grammatical Error Correction",
            "authors": [
                "Sina Ahmadi"
            ],
            "abstract": "Automatic spelling and grammatical correction systems are one of the most\nwidely used tools within natural language applications. In this thesis, we\nassume the task of error correction as a type of monolingual machine\ntranslation where the source sentence is potentially erroneous and the target\nsentence should be the corrected form of the input. Our main focus in this\nproject is building neural network models for the task of error correction. In\nparticular, we investigate sequence-to-sequence and attention-based models\nwhich have recently shown a higher performance than the state-of-the-art of\nmany language processing problems. We demonstrate that neural machine\ntranslation models can be successfully applied to the task of error correction.\n  While the experiments of this research are performed on an Arabic corpus, our\nmethods in this thesis can be easily applied to any language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00663v1",
            "date": "2018-09-24 06:09:20+00:00",
            "title": "Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation",
            "authors": [
                "Xiaoxue Zang",
                "Ashwini Pokle",
                "Marynel V\u00e1zquez",
                "Kevin Chen",
                "Juan Carlos Niebles",
                "Alvaro Soto",
                "Silvio Savarese"
            ],
            "abstract": "We propose an end-to-end deep learning model for translating free-form\nnatural language instructions to a high-level plan for behavioral robot\nnavigation. We use attention models to connect information from both the user\ninstructions and a topological representation of the environment. We evaluate\nour model's performance on a new dataset containing 10,050 pairs of navigation\ninstructions. Our model significantly outperforms baseline approaches.\nFurthermore, our results suggest that it is possible to leverage the\nenvironment map as a relevant knowledge base to facilitate the translation of\nfree-form navigational instruction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00671v2",
            "date": "2018-09-27 10:09:44+00:00",
            "title": "NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation",
            "authors": [
                "Hui Su",
                "Xiaoyu Shen",
                "Wenjie Li",
                "Dietrich Klakow"
            ],
            "abstract": "Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in\nbuilding end-to-end trainable dialogue systems. Though highly efficient in\nlearning the backbone of human-computer communications, they suffer from the\nproblem of strongly favoring short generic responses. In this paper, we argue\nthat a good response should smoothly connect both the preceding dialogue\nhistory and the following conversations. We strengthen this connection through\nmutual information maximization. To sidestep the non-differentiability of\ndiscrete natural language tokens, we introduce an auxiliary continuous code\nspace and map such code space to a learnable prior distribution for generation\npurpose. Experiments on two dialogue datasets validate the effectiveness of our\nmodel, where the generated responses are closely related to the dialogue\ncontext and lead to more interactive conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00924v1",
            "date": "2018-10-01 19:15:57+00:00",
            "title": "Joint On-line Learning of a Zero-shot Spoken Semantic Parser and a Reinforcement Learning Dialogue Manager",
            "authors": [
                "Matthieu Riou",
                "Bassam Jabaian",
                "St\u00e9phane Huet",
                "Fabrice Lef\u00e8vre"
            ],
            "abstract": "Despite many recent advances for the design of dialogue systems, a true\nbottleneck remains the acquisition of data required to train its components.\nUnlike many other language processing applications, dialogue systems require\ninteractions with users, therefore it is complex to develop them with\npre-recorded data. Building on previous works, on-line learning is pursued here\nas a most convenient way to address the issue. Data collection, annotation and\nuse in learning algorithms are performed in a single process. The main\ndifficulties are then: to bootstrap an initial basic system, and to control the\nlevel of additional cost on the user side. Considering that well-performing\nsolutions can be used directly off the shelf for speech recognition and\nsynthesis, the study is focused on learning the spoken language understanding\nand dialogue management modules only. Several variants of joint learning are\ninvestigated and tested with user trials to confirm that the overall on-line\nlearning can be obtained after only a few hundred training dialogues and can\noverstep an expert-based system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00956v1",
            "date": "2018-10-01 20:08:40+00:00",
            "title": "Challenges of Using Text Classifiers for Causal Inference",
            "authors": [
                "Zach Wood-Doughty",
                "Ilya Shpitser",
                "Mark Dredze"
            ],
            "abstract": "Causal understanding is essential for many kinds of decision-making, but\ncausal inference from observational data has typically only been applied to\nstructured, low-dimensional datasets. While text classifiers produce\nlow-dimensional outputs, their use in causal inference has not previously been\nstudied. To facilitate causal analyses based on language data, we consider the\nrole that text classifiers can play in causal inference through established\nmodeling mechanisms from the causality literature on missing data and\nmeasurement error. We demonstrate how to conduct causal analyses using text\nclassifiers on simulated and Yelp data, and discuss the opportunities and\nchallenges of future work that uses text data in causal inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00968v1",
            "date": "2018-10-01 20:40:59+00:00",
            "title": "Utilizing a Transparency-driven Environment toward Trusted Automatic Genre Classification: A Case Study in Journalism History",
            "authors": [
                "Aysenur Bilgin",
                "Laura Hollink",
                "Jacco van Ossenbruggen",
                "Erik Tjong Kim Sang",
                "Kim Smeenk",
                "Frank Harbers",
                "Marcel Broersma"
            ],
            "abstract": "With the growing abundance of unlabeled data in real-world tasks, researchers\nhave to rely on the predictions given by black-boxed computational models.\nHowever, it is an often neglected fact that these models may be scoring high on\naccuracy for the wrong reasons. In this paper, we present a practical impact\nanalysis of enabling model transparency by various presentation forms. For this\npurpose, we developed an environment that empowers non-computer scientists to\nbecome practicing data scientists in their own research field. We demonstrate\nthe gradually increasing understanding of journalism historians through a\nreal-world use case study on automatic genre classification of newspaper\narticles. This study is a first step towards trusted usage of machine learning\npipelines in a responsible way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01114v1",
            "date": "2018-10-02 08:32:08+00:00",
            "title": "Who is Addressed in this Comment? Automatically Classifying Meta-Comments in News Comments",
            "authors": [
                "Marlo H\u00e4ring",
                "Wiebke Loosen",
                "Walid Maalej"
            ],
            "abstract": "User comments have become an essential part of online journalism. However,\nnewsrooms are often overwhelmed by the vast number of diverse comments, for\nwhich a manual analysis is barely feasible. Identifying meta-comments that\naddress or mention newsrooms, individual journalists, or moderators and that\nmay call for reactions is particularly critical. In this paper, we present an\nautomated approach to identify and classify meta-comments. We compare comment\nclassification based on manually extracted features with an end-to-end learning\napproach. We develop, optimize, and evaluate multiple classifiers on a comment\ndataset of the large German online newsroom SPIEGEL Online and the 'One Million\nPosts' corpus of DER STANDARD, an Austrian newspaper. Both optimized\nclassification approaches achieved encouraging $F_{0.5}$ values between 76% and\n91%. We report on the most significant classification features with the results\nof a qualitative analysis and discuss how our work contributes to making\nparticipation in online journalism more constructive.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1810.01480v3",
            "date": "2018-10-02 19:52:38+00:00",
            "title": "Learning to Segment Inputs for NMT Favors Character-Level Processing",
            "authors": [
                "Julia Kreutzer",
                "Artem Sokolov"
            ],
            "abstract": "Most modern neural machine translation (NMT) systems rely on presegmented\ninputs. Segmentation granularity importantly determines the input and output\nsequence lengths, hence the modeling depth, and source and target vocabularies,\nwhich in turn determine model size, computational costs of softmax\nnormalization, and handling of out-of-vocabulary words. However, the current\npractice is to use static, heuristic-based segmentations that are fixed before\nNMT training. This begs the question whether the chosen segmentation is optimal\nfor the translation task. To overcome suboptimal segmentation choices, we\npresent an algorithm for dynamic segmentation based on the Adaptative\nComputation Time algorithm (Graves 2016), that is trainable end-to-end and\ndriven by the NMT objective. In an evaluation on four translation tasks we\nfound that, given the freedom to navigate between different segmentation\nlevels, the model prefers to operate on (almost) character level, providing\nsupport for purely character-level NMT models from a novel angle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02720v1",
            "date": "2018-10-05 14:35:01+00:00",
            "title": "TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation",
            "authors": [
                "Pengcheng Yin",
                "Graham Neubig"
            ],
            "abstract": "We present TRANX, a transition-based neural semantic parser that maps natural\nlanguage (NL) utterances into formal meaning representations (MRs). TRANX uses\na transition system based on the abstract syntax description language for the\ntarget MR, which gives it two major advantages: (1) it is highly accurate,\nusing information from the syntax of the target MR to constrain the output\nspace and model the information flow, and (2) it is highly generalizable, and\ncan easily be applied to new types of MR by just writing a new abstract syntax\ndescription corresponding to the allowable structures in the MR. Experiments on\nfour different semantic parsing and code generation tasks show that our system\nis generalizable, extensible, and effective, registering strong results\ncompared to existing neural semantic parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02980v2",
            "date": "2018-10-06 11:09:54+00:00",
            "title": "Personality facets recognition from text",
            "authors": [
                "Wesley Ramos dos Santos",
                "Ivandre Paraboni"
            ],
            "abstract": "Fundamental Big Five personality traits (e.g., Extraversion) and their facets\n(e.g., Activity) are known to correlate with a broad range of linguistic\nfeatures and, accordingly, the recognition of personality traits from text is a\nwell-known Natural Language Processing task. Labelling text data with facets\ninformation, however, may require the use of lengthy personality inventories,\nand perhaps for that reason existing computational models of this kind are\nusually limited to the recognition of the fundamental traits. Based on these\nobservations, this paper investigates the issue of personality facets\nrecognition from text labelled only with information available from a shorter\npersonality inventory. In doing so, we provide a low-cost model for the\nrecognition of certain personality facets, and present reference results for\nfurther studies in this field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03002v1",
            "date": "2018-10-06 13:25:54+00:00",
            "title": "When logic lays down the law",
            "authors": [
                "Bj\u00f8rn Jespersen",
                "Ana de Almeida Borges",
                "Jorge del Castillo Tierz",
                "Juan Jos\u00e9 Conejero Rodr\u00edguez",
                "Eric Sancho Adamson",
                "Aleix Sol\u00e9 S\u00e1nchez",
                "Nika Pona",
                "Joost J. Joosten"
            ],
            "abstract": "We analyse so-called computable laws, i.e., laws that can be enforced by\nautomatic procedures. These laws should be logically perfect and unambiguous,\nbut sometimes they are not. We use a regulation on road transport to illustrate\nthis issue, and show what some fragments of this regulation would look like if\nrewritten in the image of logic. We further propose desiderata to be fulfilled\nby computable laws, and provide a critical platform from which to assess\nexisting laws and a guideline for composing future ones.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.03067v1",
            "date": "2018-10-07 01:46:27+00:00",
            "title": "Geocoding Without Geotags: A Text-based Approach for reddit",
            "authors": [
                "Keith Harrigian"
            ],
            "abstract": "In this paper, we introduce the first geolocation inference approach for\nreddit, a social media platform where user pseudonymity has thus far made\nsupervised demographic inference difficult to implement and validate. In\nparticular, we design a text-based heuristic schema to generate ground truth\nlocation labels for reddit users in the absence of explicitly geotagged data.\nAfter evaluating the accuracy of our labeling procedure, we train and test\nseveral geolocation inference models across our reddit data set and three\nbenchmark Twitter geolocation data sets. Ultimately, we show that geolocation\nmodels trained and applied on the same domain substantially outperform models\nattempting to transfer training data across domains, even more so on reddit\nwhere platform-specific interest-group metadata can be used to improve\ninferences.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.03167v1",
            "date": "2018-10-07 15:55:58+00:00",
            "title": "Unsupervised Neural Word Segmentation for Chinese via Segmental Language Modeling",
            "authors": [
                "Zhiqing Sun",
                "Zhi-Hong Deng"
            ],
            "abstract": "Previous traditional approaches to unsupervised Chinese word segmentation\n(CWS) can be roughly classified into discriminative and generative models. The\nformer uses the carefully designed goodness measures for candidate\nsegmentation, while the latter focuses on finding the optimal segmentation of\nthe highest generative probability. However, while there exists a trivial way\nto extend the discriminative models into neural version by using neural\nlanguage models, those of generative ones are non-trivial. In this paper, we\npropose the segmental language models (SLMs) for CWS. Our approach explicitly\nfocuses on the segmental nature of Chinese, as well as preserves several\nproperties of language models. In SLMs, a context encoder encodes the previous\ncontext and a segment decoder generates each segment incrementally. As far as\nwe know, we are the first to propose a neural model for unsupervised CWS and\nachieve competitive performance to the state-of-the-art statistical models on\nfour different datasets from SIGHAN 2005 bakeoff.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03450v2",
            "date": "2018-10-03 12:50:56+00:00",
            "title": "Active Learning for New Domains in Natural Language Understanding",
            "authors": [
                "Stanislav Peshterliev",
                "John Kearney",
                "Abhyuday Jagannatha",
                "Imre Kiss",
                "Spyros Matsoukas"
            ],
            "abstract": "We explore active learning (AL) for improving the accuracy of new domains in\na natural language understanding (NLU) system. We propose an algorithm called\nMajority-CRF that uses an ensemble of classification models to guide the\nselection of relevant utterances, as well as a sequence labeling model to help\nprioritize informative examples. Experiments with three domains show that\nMajority-CRF achieves 6.6%-9% relative error rate reduction compared to random\nsampling with the same annotation budget, and statistically significant\nimprovements compared to other AL approaches. Additionally, case studies with\nhuman-in-the-loop AL on six new domains show 4.6%-9% improvement on an existing\nNLU system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03479v1",
            "date": "2018-10-05 14:49:42+00:00",
            "title": "Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding",
            "authors": [
                "Xu Han",
                "Hongsu Wang",
                "Sanqian Zhang",
                "Qunchao Fu",
                "Jun S. Liu"
            ],
            "abstract": "In this paper, we develop a low than character feature embedding called\nradical embedding, and apply it on LSTM model for sentence segmentation of pre\nmodern Chinese texts. The datasets includes over 150 classical Chinese books\nfrom 3 different dynasties and contains different literary styles. LSTM CRF\nmodel is a state of art method for the sequence labeling problem. Our new model\nadds a component of radical embedding, which leads to improved performances.\nExperimental results based on the aforementioned Chinese books demonstrates a\nbetter accuracy than earlier methods on sentence segmentation, especial in Tang\nEpitaph texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03552v3",
            "date": "2018-10-08 16:11:01+00:00",
            "title": "Multi-Source Cross-Lingual Model Transfer: Learning What to Share",
            "authors": [
                "Xilun Chen",
                "Ahmed Hassan Awadallah",
                "Hany Hassan",
                "Wei Wang",
                "Claire Cardie"
            ],
            "abstract": "Modern NLP applications have enjoyed a great boost utilizing neural networks\nmodels. Such deep neural models, however, are not applicable to most human\nlanguages due to the lack of annotated training data for various NLP tasks.\nCross-lingual transfer learning (CLTL) is a viable method for building NLP\nmodels for a low-resource target language by leveraging labeled data from other\n(source) languages. In this work, we focus on the multilingual transfer setting\nwhere training data in multiple source languages is leveraged to further boost\ntarget language performance.\n  Unlike most existing methods that rely only on language-invariant features\nfor CLTL, our approach coherently utilizes both language-invariant and\nlanguage-specific features at instance level. Our model leverages adversarial\nnetworks to learn language-invariant features, and mixture-of-experts models to\ndynamically exploit the similarity between the target language and each\nindividual source language. This enables our model to learn effectively what to\nshare between various languages in the multilingual setup. Moreover, when\ncoupled with unsupervised multilingual embeddings, our model can operate in a\nzero-resource setting where neither target language training data nor\ncross-lingual resources are available. Our model achieves significant\nperformance gains over prior art, as shown in an extensive set of experiments\nover multiple text classification and sequence tagging tasks including a\nlarge-scale industry dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03660v1",
            "date": "2018-10-08 19:05:23+00:00",
            "title": "DepecheMood++: a Bilingual Emotion Lexicon Built Through Simple Yet Powerful Techniques",
            "authors": [
                "Oscar Araque",
                "Lorenzo Gatti",
                "Jacopo Staiano",
                "Marco Guerini"
            ],
            "abstract": "Several lexica for sentiment analysis have been developed and made available\nin the NLP community. While most of these come with word polarity annotations\n(e.g. positive/negative), attempts at building lexica for finer-grained emotion\nanalysis (e.g. happiness, sadness) have recently attracted significant\nattention. Such lexica are often exploited as a building block in the process\nof developing learning models for which emotion recognition is needed, and/or\nused as baselines to which compare the performance of the models. In this work,\nwe contribute two new resources to the community: a) an extension of an\nexisting and widely used emotion lexicon for English; and b) a novel version of\nthe lexicon targeting Italian. Furthermore, we show how simple techniques can\nbe used, both in supervised and unsupervised experimental settings, to boost\nperformances on datasets and tasks of varying degree of domain-specificity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04000v1",
            "date": "2018-10-09 14:02:56+00:00",
            "title": "The combination of context information to enhance simple question answering",
            "authors": [
                "Zhaohui Chao",
                "Lin Li"
            ],
            "abstract": "With the rapid development of knowledge base,question answering based on\nknowledge base has been a hot research issue. In this paper, we focus on\nanswering singlerelation factoid questions based on knowledge base. We build a\nquestion answering system and study the effect of context information on fact\nselection, such as entity's notable type,outdegree. Experimental results show\nthat context information can improve the result of simple question answering.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.04528v1",
            "date": "2018-10-10 13:42:16+00:00",
            "title": "Is there Gender bias and stereotype in Portuguese Word Embeddings?",
            "authors": [
                "Brenda Salenave Santana",
                "Vinicius Woloszyn",
                "Leandro Krug Wives"
            ],
            "abstract": "In this work, we propose an analysis of the presence of gender bias\nassociated with professions in Portuguese word embeddings. The objective of\nthis work is to study gender implications related to stereotyped professions\nfor women and men in the context of the Portuguese language.",
            "journal ref": "The 13th edition of the International Conference on the\n  Computational Processing of Portuguese (PROPOR 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04700v1",
            "date": "2018-10-10 18:36:04+00:00",
            "title": "End-to-End Content and Plan Selection for Data-to-Text Generation",
            "authors": [
                "Sebastian Gehrmann",
                "Falcon Z. Dai",
                "Henry Elder",
                "Alexander M. Rush"
            ],
            "abstract": "Learning to generate fluent natural language from structured data with neural\nnetworks has become an common approach for NLG. This problem can be challenging\nwhen the form of the structured data varies between examples. This paper\npresents a survey of several extensions to sequence-to-sequence models to\naccount for the latent content selection process, particularly variants of copy\nattention and coverage decoding. We further propose a training method based on\ndiverse ensembling to encourage models to learn distinct sentence templates\nduring training. An empirical evaluation of these techniques shows an increase\nin the quality of generated text across five automated metrics, as well as\nhuman evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05237v2",
            "date": "2018-10-11 20:24:13+00:00",
            "title": "SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task",
            "authors": [
                "Tao Yu",
                "Michihiro Yasunaga",
                "Kai Yang",
                "Rui Zhang",
                "Dongxu Wang",
                "Zifan Li",
                "Dragomir Radev"
            ],
            "abstract": "Most existing studies in text-to-SQL tasks do not require generating complex\nSQL queries with multiple clauses or sub-queries, and generalizing to new,\nunseen databases. In this paper we propose SyntaxSQLNet, a syntax tree network\nto address the complex and cross-domain text-to-SQL generation task.\nSyntaxSQLNet employs a SQL specific syntax tree-based decoder with SQL\ngeneration path history and table-aware column attention encoders. We evaluate\nSyntaxSQLNet on the Spider text-to-SQL task, which contains databases with\nmultiple tables and complex SQL queries with multiple SQL clauses and nested\nqueries. We use a database split setting where databases in the test set are\nunseen during training. Experimental results show that SyntaxSQLNet can handle\na significantly greater number of complex SQL examples than prior work,\noutperforming the previous state-of-the-art model by 7.3% in exact matching\naccuracy. We also show that SyntaxSQLNet can further improve the performance by\nan additional 7.5% using a cross-domain augmentation method, resulting in a\n14.8% improvement in total. To our knowledge, we are the first to study this\ncomplex and cross-domain text-to-SQL task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05241v4",
            "date": "2018-10-11 20:40:15+00:00",
            "title": "One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases",
            "authors": [
                "Xingdi Yuan",
                "Tong Wang",
                "Rui Meng",
                "Khushboo Thaker",
                "Peter Brusilovsky",
                "Daqing He",
                "Adam Trischler"
            ],
            "abstract": "Different texts shall by nature correspond to different number of keyphrases.\nThis desideratum is largely missing from existing neural keyphrase generation\nmodels. In this study, we address this problem from both modeling and\nevaluation perspectives.\n  We first propose a recurrent generative model that generates multiple\nkeyphrases as delimiter-separated sequences. Generation diversity is further\nenhanced with two novel techniques by manipulating decoder hidden states. In\ncontrast to previous approaches, our model is capable of generating diverse\nkeyphrases and controlling number of outputs.\n  We further propose two evaluation metrics tailored towards the\nvariable-number generation. We also introduce a new dataset StackEx that\nexpands beyond the only existing genre (i.e., academic writing) in keyphrase\ngeneration tasks. With both previous and new evaluation metrics, our model\noutperforms strong baselines on all datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05474v1",
            "date": "2018-10-12 12:19:56+00:00",
            "title": "Pre-gen metrics: Predicting caption quality metrics without generating captions",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Adrian Muscat"
            ],
            "abstract": "Image caption generation systems are typically evaluated against reference\noutputs. We show that it is possible to predict output quality without\ngenerating the captions, based on the probability assigned by the neural model\nto the reference captions. Such pre-gen metrics are strongly correlated to\nstandard evaluation metrics.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1810.05475v1",
            "date": "2018-10-12 12:23:08+00:00",
            "title": "Quantifying the amount of visual information used by neural caption generators",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Kenneth P. Camilleri"
            ],
            "abstract": "This paper addresses the sensitivity of neural image caption generators to\ntheir visual input. A sensitivity analysis and omission analysis based on image\nfoils is reported, showing that the extent to which image captioning\narchitectures retain and are sensitive to visual information varies depending\non the type of word being generated and the position in the caption as a whole.\nWe motivate this work in the context of broader goals in the field to achieve\nmore explainability in AI.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1810.05983v1",
            "date": "2018-10-14 06:34:18+00:00",
            "title": "Finding Similar Medical Questions from Question Answering Websites",
            "authors": [
                "Yaliang Li",
                "Liuyi Yao",
                "Nan Du",
                "Jing Gao",
                "Qi Li",
                "Chuishi Meng",
                "Chenwei Zhang",
                "Wei Fan"
            ],
            "abstract": "The past few years have witnessed the flourishing of crowdsourced medical\nquestion answering (Q&A) websites. Patients who have medical information\ndemands tend to post questions about their health conditions on these\ncrowdsourced Q&A websites and get answers from other users. However, we observe\nthat a large portion of new medical questions cannot be answered in time or\nreceive only few answers from these websites. On the other hand, we notice that\nsolved questions have great potential to solve this challenge. Motivated by\nthese, we propose an end-to-end system that can automatically find similar\nquestions for unsolved medical questions. By learning the vector presentation\nof unsolved questions and their candidate similar questions, the proposed\nsystem outputs similar questions according to the similarity between vector\nrepresentations. Through the vector representation, the similar questions are\nfound at the question level, and the diversity of medical questions expression\nissue can be addressed. Further, we handle two more important issues, i.e.,\ntraining data generation issue and efficiency issue, associated with the LSTM\ntraining procedure and the retrieval of candidate similar questions. The\neffectiveness of the proposed system is validated on a large-scale real-world\ndataset collected from a crowdsourced maternal-infant Q&A website.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.06033v2",
            "date": "2018-10-14 13:26:34+00:00",
            "title": "Modeling relation paths for knowledge base completion via joint adversarial training",
            "authors": [
                "Chen Li",
                "Xutan Peng",
                "Shanghang Zhang",
                "Hao Peng",
                "Philip S. Yu",
                "Min He",
                "Linfeng Du",
                "Lihong Wang"
            ],
            "abstract": "Knowledge Base Completion (KBC), which aims at determining the missing\nrelations between entity pairs, has received increasing attention in recent\nyears. Most existing KBC methods focus on either embedding the Knowledge Base\n(KB) into a specific semantic space or leveraging the joint probability of\nRandom Walks (RWs) on multi-hop paths. Only a few unified models take both\nsemantic and path-related features into consideration with adequacy. In this\npaper, we propose a novel method to explore the intrinsic relationship between\nthe single relation (i.e. 1-hop path) and multi-hop paths between paired\nentities. We use Hierarchical Attention Networks (HANs) to select important\nrelations in multi-hop paths and encode them into low-dimensional vectors. By\ntreating relations and multi-hop paths as two different input sources, we use a\nfeature extractor, which is shared by two downstream components (i.e. relation\nclassifier and source discriminator), to capture shared/similar information\nbetween them. By joint adversarial training, we encourage our model to extract\nfeatures from the multi-hop paths which are representative for relation\ncompletion. We apply the trained model (except for the source discriminator) to\nseveral large-scale KBs for relation completion. Experimental results show that\nour method outperforms existing path information-based approaches. Since each\nsub-module of our model can be well interpreted, our model can be applied to a\nlarge number of relation learning tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.06638v1",
            "date": "2018-10-12 12:48:34+00:00",
            "title": "U-Net: Machine Reading Comprehension with Unanswerable Questions",
            "authors": [
                "Fu Sun",
                "Linyang Li",
                "Xipeng Qiu",
                "Yang Liu"
            ],
            "abstract": "Machine reading comprehension with unanswerable questions is a new\nchallenging task for natural language processing. A key subtask is to reliably\npredict whether the question is unanswerable. In this paper, we propose a\nunified model, called U-Net, with three important components: answer pointer,\nno-answer pointer, and answer verifier. We introduce a universal node and thus\nprocess the question and its context passage as a single contiguous sequence of\ntokens. The universal node encodes the fused information from both the question\nand passage, and plays an important role to predict whether the question is\nanswerable and also greatly improves the conciseness of the U-Net. Different\nfrom the state-of-art pipeline models, U-Net can be learned in an end-to-end\nfashion. The experimental results on the SQuAD 2.0 dataset show that U-Net can\neffectively predict the unanswerability of questions and achieves an F1 score\nof 71.7 on SQuAD 2.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06644v1",
            "date": "2018-10-09 06:55:04+00:00",
            "title": "An Instance Transfer based Approach Using Enhanced Recurrent Neural Network for Domain Named Entity Recognition",
            "authors": [
                "Lin Li",
                "Yueqing Sun"
            ],
            "abstract": "Recently, neural networks have shown promising results for named entity\nrecognition (NER), which needs a number of labeled data to for model training.\nWhen meeting a new domain (target domain) for NER, there is no or a few labeled\ndata, which makes domain NER much more difficult. As NER has been researched\nfor a long time, some similar domain already has well labelled data (source\ndomain). Therefore, in this paper, we focus on domain NER by studying how to\nutilize the labelled data from such similar source domain for the new target\ndomain. We design a kernel function based instance transfer strategy by getting\nsimilar labelled sentences from a source domain. Moreover, we propose an\nenhanced recurrent neural network (ERNN) by adding an additional layer that\ncombines the source domain labelled data into traditional RNN structure.\nComprehensive experiments are conducted on two datasets. The comparison results\namong HMM, CRF and RNN show that RNN performs bette than others. When there is\nno labelled data in domain target, compared to directly using the source domain\nlabelled data without selecting transferred instances, our enhanced RNN\napproach gets improvement from 0.8052 to 0.9328 in terms of F1 measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06645v1",
            "date": "2018-10-09 07:01:20+00:00",
            "title": "Using Sentiment Representation Learning to Enhance Gender Classification for User Profiling",
            "authors": [
                "Yunpei Zheng",
                "Lin Li",
                "Luo Zhong",
                "Jianwei Zhang",
                "Jinhang Liu"
            ],
            "abstract": "User profiling means exploiting the technology of machine learning to predict\nattributes of users, such as demographic attributes, hobby attributes,\npreference attributes, etc. It's a powerful data support of precision\nmarketing. Existing methods mainly study network behavior, personal\npreferences, post texts to build user profile. Through our data analysis of\nmicro-blog, we find that females show more positive and have richer emotions\nthan males in online social platform. This difference is very conducive to the\ndistinction between genders. Therefore, we argue that sentiment context is\nimportant as well for user profiling.This paper focuses on exploiting microblog\nuser posts to predict one of the demographic labels: gender. We propose a\nSentiment Representation Learning based Multi-Layer Perceptron(SRL-MLP) model\nto classify gender. First we build a sentiment polarity classifier in advance\nby training Long Short-Term Memory(LSTM) model on e-commerce review corpus.\nNext we transfer sentiment representation to a basic MLP network. Last we\nconduct experiments on gender classification by sentiment representation.\nExperimental results show that our approach can improve gender classification\naccuracy by 5.53\\%, from 84.20\\% to 89.73\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06683v3",
            "date": "2018-10-06 20:46:49+00:00",
            "title": "FlowQA: Grasping Flow in History for Conversational Machine Comprehension",
            "authors": [
                "Hsin-Yuan Huang",
                "Eunsol Choi",
                "Wen-tau Yih"
            ],
            "abstract": "Conversational machine comprehension requires the understanding of the\nconversation history, such as previous question/answer pairs, the document\ncontext, and the current question. To enable traditional, single-turn models to\nencode the history comprehensively, we introduce Flow, a mechanism that can\nincorporate intermediate representations generated during the process of\nanswering previous questions, through an alternating parallel processing\nstructure. Compared to approaches that concatenate previous questions/answers\nas input, Flow integrates the latent semantics of the conversation history more\ndeeply. Our model, FlowQA, shows superior performance on two recently proposed\nconversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC). The\neffectiveness of Flow also shows in other tasks. By reducing sequential\ninstruction understanding to conversational machine comprehension, FlowQA\noutperforms the best models on all three domains in SCONE, with +1.8% to +4.4%\nimprovement in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06745v1",
            "date": "2018-10-15 23:06:03+00:00",
            "title": "Can Euroscepticism Contribute to a European Public Sphere? The Europeanization of Media Discourses about Euroscepticism across Six Countries",
            "authors": [
                "Anamaria Dutceac Segesten",
                "Michael Bossetta"
            ],
            "abstract": "This study compares the media discourses about Euroscepticism in 2014 across\nsix countries (United Kingdom, Ireland, France, Spain, Sweden, and Denmark). We\nassess the extent to which the mass media's reporting of Euroscepticism\nindicates the Europeanization of public spheres. Using a mixed-methods approach\ncombining LDA topic modeling and qualitative coding, we find that approximately\n70 per cent of print articles mentioning \"Euroscepticism\" or \"Eurosceptic\" are\nframed in a non-domestic (i.e. European) context. In five of the six cases\nstudied, articles exhibiting a European context are strikingly similar in\ncontent, with the British case as the exception. However, coverage of British\nEuroscepticism drives Europeanization in other Member States. Bivariate\nlogistic regressions further reveal three macro-level structural variables that\nsignificantly correlate with a Europeanized media discourse: newspaper type\n(tabloid or broadsheet), presence of a strong Eurosceptic party, and\nrelationship to the EU budget (net contributor or receiver of EU funds).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06765v1",
            "date": "2018-10-16 00:26:39+00:00",
            "title": "A survey of automatic de-identification of longitudinal clinical narratives",
            "authors": [
                "Vithya Yogarajan",
                "Michael Mayo",
                "Bernhard Pfahringer"
            ],
            "abstract": "Use of medical data, also known as electronic health records, in research\nhelps develop and advance medical science. However, protecting patient\nconfidentiality and identity while using medical data for analysis is crucial.\nMedical data can be in the form of tabular structures (i.e. tables), free-form\nnarratives, and images. This study focuses on medical data in the free form\nlongitudinal text. De-identification of electronic health records provides the\nopportunity to use such data for research without it affecting patient privacy,\nand avoids the need for individual patient consent. In recent years there is\nincreasing interest in developing an accurate, robust and adaptable automatic\nde-identification system for electronic health records. This is mainly due to\nthe dilemma between the availability of an abundance of health data, and the\ninability to use such data in research due to legal and ethical restrictions.\nDe-identification tracks in competitions such as the 2014 i2b2 UTHealth and the\n2016 CEGS N-GRID shared tasks have provided a great platform to advance this\narea. The primary reasons for this include the open source nature of the\ndataset and the fact that raw psychiatric data were used for 2016 competitions.\nThis study focuses on noticeable trend changes in the techniques used in the\ndevelopment of automatic de-identification for longitudinal clinical\nnarratives. More specifically, the shift from using conditional random fields\n(CRF) based systems only or rules (regular expressions, dictionary or\ncombinations) based systems only, to hybrid models (combining CRF and rules),\nand more recently to deep learning based systems. We review the literature and\nresults that arose from the 2014 and the 2016 competitions and discuss the\noutcomes of these systems. We also provide a list of research questions that\nemerged from this survey.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.06818v2",
            "date": "2018-10-16 05:40:03+00:00",
            "title": "Named Entity Analysis and Extraction with Uncommon Words",
            "authors": [
                "Xiaoshi Zhong",
                "Erik Cambria",
                "Jagath C. Rajapakse"
            ],
            "abstract": "Most previous research treats named entity extraction and classification as\nan end-to-end task. We argue that the two sub-tasks should be addressed\nseparately. Entity extraction lies at the level of syntactic analysis while\nentity classification lies at the level of semantic analysis. According to Noam\nChomsky's \"Syntactic Structures,\" pp. 93-94 (Chomsky 1957), syntax is not\nappealed to semantics and semantics does not affect syntax. We analyze two\nbenchmark datasets for the characteristics of named entities, finding that\nuncommon words can distinguish named entities from common text; where uncommon\nwords are the words that hardly appear in common text and they are mainly the\nproper nouns. Experiments validate that lexical and syntactic features achieve\nstate-of-the-art performance on entity extraction and that semantic features do\nnot further improve the extraction performance, in both of our model and the\nstate-of-the-art baselines. With Chomsky's view, we also explain the failure of\njoint syntactic and semantic parsings in other works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07030v1",
            "date": "2018-10-15 08:30:27+00:00",
            "title": "The Trajectory of Voice Onset Time with Vocal Aging",
            "authors": [
                "Xuanda Chen",
                "Ziyu Xiong",
                "Jian Hu"
            ],
            "abstract": "Vocal aging, a universal process of human aging, can largely affect one's\nlanguage use, possibly including some subtle acoustic features of one's\nutterances like Voice Onset Time. To figure out the time effects, Queen\nElizabeth's Christmas speeches are documented and analyzed in the long-term\ntrend. We build statistical models of time dependence in Voice Onset Time,\ncontrolling a wide range of other fixed factors, to present annual variations\nand the simulated trajectory. It is revealed that the variation range of Voice\nOnset Time has been narrowing over fifty years with a slight reduction in the\nmean value, which, possibly, is an effect of diminishing exertion, resulting\nfrom subdued muscle contraction, transcending other non-linguistic factors in\nforming Voice Onset Time patterns over a long time.",
            "journal ref": "Interspeech 2018",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1810.07781v2",
            "date": "2018-10-13 10:28:45+00:00",
            "title": "Responsible team players wanted: an analysis of soft skill requirements in job advertisements",
            "authors": [
                "Federica Calanca",
                "Luiza Sayfullina",
                "Lara Minkus",
                "Claudia Wagner",
                "Eric Malmi"
            ],
            "abstract": "During the past decades the importance of soft skills for labour market\noutcomes has grown substantially. This carries implications for labour market\ninequality, since previous research shows that soft skills are not valued\nequally across race and gender. This work explores the role of soft skills in\njob advertisements by drawing on methods from computational science as well as\non theoretical and empirical insights from economics, sociology and psychology.\nWe present a semi-automatic approach based on crowdsourcing and text mining for\nextracting a list of soft skills. We find that soft skills are a crucial\ncomponent of job ads, especially of low-paid jobs and jobs in female-dominated\nprofessions. Our work shows that soft skills can serve as partial predictors of\nthe gender composition in job categories and that not all soft skills receive\nequal wage returns at the labour market. Especially \"female\" skills are\nfrequently associated with wage penalties. Our results expand the growing\nliterature on the association of soft skills on wage inequality and highlight\ntheir importance for occupational gender segregation at labour markets.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1810.07931v6",
            "date": "2018-10-18 07:43:12+00:00",
            "title": "Unsupervised Neural Text Simplification",
            "authors": [
                "Sai Surya",
                "Abhijit Mishra",
                "Anirban Laha",
                "Parag Jain",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "The paper presents a first attempt towards unsupervised neural text\nsimplification that relies only on unlabeled text corpora. The core framework\nis composed of a shared encoder and a pair of attentional-decoders and gains\nknowledge of simplification through discrimination based-losses and denoising.\nThe framework is trained using unlabeled text collected from en-Wikipedia dump.\nOur analysis (both quantitative and qualitative involving human evaluators) on\na public test data shows that the proposed model can perform\ntext-simplification at both lexical and syntactic levels, competitive to\nexisting supervised methods. Addition of a few labelled pairs also improves the\nperformance further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08076v1",
            "date": "2018-10-18 14:29:50+00:00",
            "title": "Discourse Embellishment Using a Deep Encoder-Decoder Network",
            "authors": [
                "Leonid Berov",
                "Kai Standvoss"
            ],
            "abstract": "We suggest a new NLG task in the context of the discourse generation pipeline\nof computational storytelling systems. This task, textual embellishment, is\ndefined by taking a text as input and generating a semantically equivalent\noutput with increased lexical and syntactic complexity. Ideally, this would\nallow the authors of computational storytellers to implement just lightweight\nNLG systems and use a domain-independent embellishment module to translate its\noutput into more literary text. We present promising first results on this task\nusing LSTM Encoder-Decoder networks trained on the WikiLarge dataset.\nFurthermore, we introduce \"Compiled Computer Tales\", a corpus of\ncomputationally generated stories, that can be used to test the capabilities of\nembellishment algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08272v4",
            "date": "2018-10-18 20:48:08+00:00",
            "title": "BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning",
            "authors": [
                "Maxime Chevalier-Boisvert",
                "Dzmitry Bahdanau",
                "Salem Lahlou",
                "Lucas Willems",
                "Chitwan Saharia",
                "Thien Huu Nguyen",
                "Yoshua Bengio"
            ],
            "abstract": "Allowing humans to interactively train artificial agents to understand\nlanguage instructions is desirable for both practical and scientific reasons,\nbut given the poor data efficiency of the current learning methods, this goal\nmay require substantial research efforts. Here, we introduce the BabyAI\nresearch platform to support investigations towards including humans in the\nloop for grounded language learning. The BabyAI platform comprises an\nextensible suite of 19 levels of increasing difficulty. The levels gradually\nlead the agent towards acquiring a combinatorially rich synthetic language\nwhich is a proper subset of English. The platform also provides a heuristic\nexpert agent for the purpose of simulating a human teacher. We report baseline\nresults and estimate the amount of human involvement that would be required to\ntrain a neural network-based agent on some of the BabyAI levels. We put forward\nstrong evidence that current deep learning methods are not yet sufficiently\nsample efficient when it comes to learning a language with compositional\nproperties.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.08577v1",
            "date": "2018-10-19 16:41:27+00:00",
            "title": "Conceptual Organization is Revealed by Consumer Activity Patterns",
            "authors": [
                "Adam N. Hornsby",
                "Thomas Evans",
                "Peter Riefer",
                "Rosie Prior",
                "Bradley C. Love"
            ],
            "abstract": "Meaning may arise from an element's role or interactions within a larger\nsystem. For example, hitting nails is more central to people's concept of a\nhammer than its particular material composition or other intrinsic features.\nLikewise, the importance of a web page may result from its links with other\npages rather than solely from its content. One example of meaning arising from\nextrinsic relationships are approaches that extract the meaning of word\nconcepts from co-occurrence patterns in large, text corpora. The success of\nthese methods suggest that human activity patterns may reveal conceptual\norganization. However, texts do not directly reflect human activity, but\ninstead serve a communicative function and are usually highly curated or edited\nto suit an audience. Here, we apply methods devised for text to a data source\nthat directly reflects thousands of individuals' activity patterns, namely\nsupermarket purchases. Using product co-occurrence data from nearly 1.3m\nshopping baskets, we trained a topic model to learn 25 high-level concepts (or\n\"topics\"). These topics were found to be comprehensible and coherent by both\nretail experts and consumers. Topics ranged from specific (e.g., ingredients\nfor a stir-fry) to general (e.g., cooking from scratch). Topics tended to be\ngoal-directed and situational, consistent with the notion that human conceptual\nknowledge is tailored to support action. Individual differences in the topics\nsampled predicted basic demographic characteristics. These results suggest that\nhuman activity patterns reveal conceptual organization and may give rise to it.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.08606v1",
            "date": "2018-10-22 17:48:21+00:00",
            "title": "An Exploration of Dropout with RNNs for Natural Language Inference",
            "authors": [
                "Amit Gajbhiye",
                "Sardar Jaf",
                "Noura Al Moubayed",
                "A. Stephen McGough",
                "Steven Bradley"
            ],
            "abstract": "Dropout is a crucial regularization technique for the Recurrent Neural\nNetwork (RNN) models of Natural Language Inference (NLI). However, dropout has\nnot been evaluated for the effectiveness at different layers and dropout rates\nin NLI models. In this paper, we propose a novel RNN model for NLI and\nempirically evaluate the effect of applying dropout at different layers in the\nmodel. We also investigate the impact of varying dropout rates at these layers.\nOur empirical evaluation on a large (Stanford Natural Language Inference\n(SNLI)) and a small (SciTail) dataset suggest that dropout at each feed-forward\nconnection severely affects the model accuracy at increasing dropout rates. We\nalso show that regularizing the embedding layer is efficient for SNLI whereas\nregularizing the recurrent layer improves the accuracy for SciTail. Our model\nachieved an accuracy 86.14% on the SNLI dataset and 77.05% on SciTail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08677v1",
            "date": "2018-10-19 20:22:53+00:00",
            "title": "A neural network to classify metaphorical violence on cable news",
            "authors": [
                "Matthew A. Turner"
            ],
            "abstract": "I present here an experimental system for identifying and annotating metaphor\nin corpora. It is designed to plug in to Metacorps, an experimental web app for\nannotating metaphor. As Metacorps users annotate metaphors, the system will use\nuser annotations as training data. When the system is confident, it will\nsuggest an identification and an annotation. Once approved by the user, this\nbecomes more training data. This naturally allows for transfer learning, where\nthe system can, with some known degree of reliability, classify one class of\nmetaphor after only being trained on another class of metaphor. For example, in\nour metaphorical violence project, metaphors may be classified by the network\nthey were observed on, the grammatical subject or object of the violence\nmetaphor, or the violent word used (hit, attack, beat, etc.).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08680v1",
            "date": "2018-10-19 20:32:36+00:00",
            "title": "Lightweight Convolutional Approaches to Reading Comprehension on SQuAD",
            "authors": [
                "Tobin Bell",
                "Benjamin Penchas"
            ],
            "abstract": "Current state-of-the-art reading comprehension models rely heavily on\nrecurrent neural networks. We explored an entirely different approach to\nquestion answering: a convolutional model. By their nature, these convolutional\nmodels are fast to train and capture local dependencies well, though they can\nstruggle with longer-range dependencies and thus require augmentation to\nachieve comparable performance to RNN-based models. We conducted over two dozen\ncontrolled experiments with convolutional models and various\nkernel/attention/regularization schemes to determine the precise performance\ngains of each strategy, while maintaining a focus on speed. We ultimately\nensembled three models: crossconv (0.5398 dev F1), attnconv (0.5665), and\nmaybeconv (0.5285). The ensembled model was able to achieve a 0.6238 F1 score\nusing the official SQuAD evaluation script. Our individual convolutional model\ncrossconv was able to exceed the performance of the RNN-plus-attention baseline\nby 25% while training 6 times faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08782v3",
            "date": "2018-10-20 09:59:31+00:00",
            "title": "Collective Learning From Diverse Datasets for Entity Typing in the Wild",
            "authors": [
                "Abhishek Abhishek",
                "Amar Prakash Azad",
                "Balaji Ganesan",
                "Ashish Anand",
                "Amit Awekar"
            ],
            "abstract": "Entity typing (ET) is the problem of assigning labels to given entity\nmentions in a sentence. Existing works for ET require knowledge about the\ndomain and target label set for a given test instance. ET in the absence of\nsuch knowledge is a novel problem that we address as ET in the wild. We\nhypothesize that the solution to this problem is to build supervised models\nthat generalize better on the ET task as a whole, rather than a specific\ndataset. In this direction, we propose a Collective Learning Framework (CLF),\nwhich enables learning from diverse datasets in a unified way. The CLF first\ncreates a unified hierarchical label set (UHLS) and a label mapping by\naggregating label information from all available datasets. Then it builds a\nsingle neural network classifier using UHLS, label mapping, and a partial loss\nfunction. The single classifier predicts the finest possible label across all\navailable domains even though these labels may not be present in any\ndomain-specific dataset. We also propose a set of evaluation schemes and\nmetrics to evaluate the performance of models in this novel problem. Extensive\nexperimentation on seven diverse real-world datasets demonstrates the efficacy\nof our CLF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09309v1",
            "date": "2018-10-19 12:57:37+00:00",
            "title": "Real-time Neural-based Input Method",
            "authors": [
                "Jiali Yao",
                "Raphael Shu",
                "Xinjian Li",
                "Katsutoshi Ohtsuki",
                "Hideki Nakayama"
            ],
            "abstract": "The input method is an essential service on every mobile and desktop devices\nthat provides text suggestions. It converts sequential keyboard inputs to the\ncharacters in its target language, which is indispensable for Japanese and\nChinese users. Due to critical resource constraints and limited network\nbandwidth of the target devices, applying neural models to input method is not\nwell explored. In this work, we apply a LSTM-based language model to input\nmethod and evaluate its performance for both prediction and conversion tasks\nwith Japanese BCCWJ corpus. We articulate the bottleneck to be the slow softmax\ncomputation during conversion. To solve the issue, we propose incremental\nsoftmax approximation approach, which computes softmax with a selected subset\nvocabulary and fix the stale probabilities when the vocabulary is updated in\nfuture steps. We refer to this method as incremental selective softmax. The\nresults show a two order speedup for the softmax computation when converting\nJapanese input sequences with a large vocabulary, reaching real-time speed on\ncommodity CPU. We also exploit the model compressing potential to achieve a 92%\nmodel size reduction without losing accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09431v1",
            "date": "2018-10-22 17:56:08+00:00",
            "title": "Proactive Security: Embedded AI Solution for Violent and Abusive Speech Recognition",
            "authors": [
                "Christopher Dane Shulby",
                "Leonardo Pombal",
                "Vitor Jord\u00e3o",
                "Guilherme Ziolle",
                "Bruno Martho",
                "Ant\u00f4nio Postal",
                "Thiago Prochnow"
            ],
            "abstract": "Violence is an epidemic in Brazil and a problem on the rise world-wide.\nMobile devices provide communication technologies which can be used to monitor\nand alert about violent situations. However, current solutions, like panic\nbuttons or safe words, might increase the loss of life in violent situations.\nWe propose an embedded artificial intelligence solution, using natural language\nand speech processing technology, to silently alert someone who can help in\nthis situation. The corpus used contains 400 positive phrases and 800 negative\nphrases, totaling 1,200 sentences which are classified using two well-known\nextraction methods for natural language processing tasks: bag-of-words and word\nembeddings and classified with a support vector machine. We describe the\nproof-of-concept product in development with promising results, indicating a\npath towards a commercial product. More importantly we show that model\nimprovements via word embeddings and data augmentation techniques provide an\nintrinsically robust model. The final embedded solution also has a small\nfootprint of less than 10 MB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09536v6",
            "date": "2018-10-22 20:37:46+00:00",
            "title": "Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks",
            "authors": [
                "Yikang Shen",
                "Shawn Tan",
                "Alessandro Sordoni",
                "Aaron Courville"
            ],
            "abstract": "Natural language is hierarchically structured: smaller units (e.g., phrases)\nare nested within larger units (e.g., clauses). When a larger constituent ends,\nall of the smaller constituents that are nested within it must also be closed.\nWhile the standard LSTM architecture allows different neurons to track\ninformation at different time scales, it does not have an explicit bias towards\nmodeling a hierarchy of constituents. This paper proposes to add such an\ninductive bias by ordering the neurons; a vector of master input and forget\ngates ensures that when a given neuron is updated, all the neurons that follow\nit in the ordering are also updated. Our novel recurrent architecture, ordered\nneurons LSTM (ON-LSTM), achieves good performance on four different tasks:\nlanguage modeling, unsupervised parsing, targeted syntactic evaluation, and\nlogical inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09580v1",
            "date": "2018-10-22 22:10:46+00:00",
            "title": "A Fully Attention-Based Information Retriever",
            "authors": [
                "Alvaro Henrique Chaim Correia",
                "Jorge Luiz Moreira Silva",
                "Thiago de Castro Martins",
                "Fabio Gagliardi Cozman"
            ],
            "abstract": "Recurrent neural networks are now the state-of-the-art in natural language\nprocessing because they can build rich contextual representations and process\ntexts of arbitrary length. However, recent developments on attention mechanisms\nhave equipped feedforward networks with similar capabilities, hence enabling\nfaster computations due to the increase in the number of operations that can be\nparallelized. We explore this new type of architecture in the domain of\nquestion-answering and propose a novel approach that we call Fully Attention\nBased Information Retriever (FABIR). We show that FABIR achieves competitive\nresults in the Stanford Question Answering Dataset (SQuAD) while having fewer\nparameters and being faster at both learning and inference than rival methods.",
            "journal ref": "A. H. C. Correia, J. L. M. Silva, T. d. C. Martins and F. G.\n  Cozman, \"A Fully Attention-Based Information Retriever,\" 2018 International\n  Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 2018,\n  pp. 2799-2806",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09988v1",
            "date": "2018-10-23 17:42:17+00:00",
            "title": "Meta-Learning Multi-task Communication",
            "authors": [
                "Pengfei Liu",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we describe a general framework: Parameters Read-Write\nNetworks (PRaWNs) to systematically analyze current neural models for\nmulti-task learning, in which we find that existing models expect to\ndisentangle features into different spaces while features learned in practice\nare still entangled in shared space, leaving potential hazards for other\ntraining or unseen tasks.\n  We propose to alleviate this problem by incorporating an inductive bias into\nthe process of multi-task learning, that each task can keep informed of not\nonly the knowledge stored in other tasks but the way how other tasks maintain\ntheir knowledge.\n  In practice, we achieve above inductive bias by allowing different tasks to\ncommunicate by passing both hidden variables and gradients explicitly.\n  Experimentally, we evaluate proposed methods on three groups of tasks and two\ntypes of settings (\\textsc{in-task} and \\textsc{out-of-task}). Quantitative and\nqualitative results show their effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10136v1",
            "date": "2018-10-24 00:24:03+00:00",
            "title": "Local Homology of Word Embeddings",
            "authors": [
                "Tadas Tem\u010dinas"
            ],
            "abstract": "Topological data analysis (TDA) has been widely used to make progress on a\nnumber of problems. However, it seems that TDA application in natural language\nprocessing (NLP) is at its infancy. In this paper we try to bridge the gap by\narguing why TDA tools are a natural choice when it comes to analysing word\nembedding data. We describe a parallelisable unsupervised learning algorithm\nbased on local homology of datapoints and show some experimental results on\nword embedding data. We see that local homology of datapoints in word embedding\ndata contains some information that can potentially be used to solve the word\nsense disambiguation problem.",
            "journal ref": "None",
            "category": "math.AT"
        },
        {
            "url": "http://arxiv.org/abs/1810.10165v2",
            "date": "2018-10-24 03:22:08+00:00",
            "title": "Resolving Referring Expressions in Images With Labeled Elements",
            "authors": [
                "Nevan Wichers",
                "Dilek Hakkani-Tur",
                "Jindong Chen"
            ],
            "abstract": "Images may have elements containing text and a bounding box associated with\nthem, for example, text identified via optical character recognition on a\ncomputer screen image, or a natural image with labeled objects. We present an\nend-to-end trainable architecture to incorporate the information from these\nelements and the image to segment/identify the part of the image a natural\nlanguage expression is referring to. We calculate an embedding for each element\nand then project it onto the corresponding location (i.e., the associated\nbounding box) of the image feature map. We show that this architecture gives an\nimprovement in resolving referring expressions, over only using the image, and\nother methods that incorporate the element information. We demonstrate\nexperimental results on the referring expression datasets based on COCO, and on\na webpage image referring expression dataset that we developed.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1810.10181v1",
            "date": "2018-10-24 04:08:22+00:00",
            "title": "Exploiting Deep Representations for Neural Machine Translation",
            "authors": [
                "Zi-Yi Dou",
                "Zhaopeng Tu",
                "Xing Wang",
                "Shuming Shi",
                "Tong Zhang"
            ],
            "abstract": "Advanced neural machine translation (NMT) models generally implement encoder\nand decoder as multiple layers, which allows systems to model complex functions\nand capture complicated linguistic structures. However, only the top layers of\nencoder and decoder are leveraged in the subsequent process, which misses the\nopportunity to exploit the useful information embedded in other layers. In this\nwork, we propose to simultaneously expose all of these signals with layer\naggregation and multi-layer attention mechanisms. In addition, we introduce an\nauxiliary regularization term to encourage different layers to capture diverse\ninformation. Experimental results on widely-used WMT14 English-German and WMT17\nChinese-English translation data demonstrate the effectiveness and universality\nof the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10182v1",
            "date": "2018-10-24 04:08:25+00:00",
            "title": "Modeling Localness for Self-Attention Networks",
            "authors": [
                "Baosong Yang",
                "Zhaopeng Tu",
                "Derek F. Wong",
                "Fandong Meng",
                "Lidia S. Chao",
                "Tong Zhang"
            ],
            "abstract": "Self-attention networks have proven to be of profound value for its strength\nof capturing global dependencies. In this work, we propose to model localness\nfor self-attention networks, which enhances the ability of capturing useful\nlocal context. We cast localness modeling as a learnable Gaussian bias, which\nindicates the central and scope of the local region to be paid more attention.\nThe bias is then incorporated into the original attention distribution to form\na revised distribution. To maintain the strength of capturing long distance\ndependencies and enhance the ability of capturing short-range dependencies, we\nonly apply localness modeling to lower layers of self-attention networks.\nQuantitative and qualitative analyses on Chinese-English and English-German\ntranslation tasks demonstrate the effectiveness and universality of the\nproposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10183v1",
            "date": "2018-10-24 04:08:27+00:00",
            "title": "Multi-Head Attention with Disagreement Regularization",
            "authors": [
                "Jian Li",
                "Zhaopeng Tu",
                "Baosong Yang",
                "Michael R. Lyu",
                "Tong Zhang"
            ],
            "abstract": "Multi-head attention is appealing for the ability to jointly attend to\ninformation from different representation subspaces at different positions. In\nthis work, we introduce a disagreement regularization to explicitly encourage\nthe diversity among multiple attention heads. Specifically, we propose three\ntypes of disagreement regularization, which respectively encourage the\nsubspace, the attended positions, and the output representation associated with\neach attention head to be different from other heads. Experimental results on\nwidely-used WMT14 English-German and WMT17 Chinese-English translation tasks\ndemonstrate the effectiveness and universality of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10297v2",
            "date": "2018-10-24 11:20:02+00:00",
            "title": "A Proof-Theoretic Approach to Scope Ambiguity in Compositional Vector Space Models",
            "authors": [
                "Gijs Jasper Wijnholds"
            ],
            "abstract": "We investigate the extent to which compositional vector space models can be\nused to account for scope ambiguity in quantified sentences (of the form \"Every\nman loves some woman\"). Such sentences containing two quantifiers introduce two\nreadings, a direct scope reading and an inverse scope reading. This ambiguity\nhas been treated in a vector space model using bialgebras by (Hedges and\nSadrzadeh, 2016) and (Sadrzadeh, 2016), though without an explanation of the\nmechanism by which the ambiguity arises. We combine a polarised focussed\nsequent calculus for the non-associative Lambek calculus NL, as described in\n(Moortgat and Moot, 2011), with the vector based approach to quantifier scope\nambiguity. In particular, we establish a procedure for obtaining a vector space\nmodel for quantifier scope ambiguity in a derivational way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10499v1",
            "date": "2018-10-24 17:08:36+00:00",
            "title": "Multi-Multi-View Learning: Multilingual and Multi-Representation Entity Typing",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Knowledge bases (KBs) are paramount in NLP. We employ multiview learning for\nincreasing accuracy and coverage of entity type information in KBs. We rely on\ntwo metaviews: language and representation. For language, we consider\nhigh-resource and low-resource languages from Wikipedia. For representation, we\nconsider representations based on the context distribution of the entity (i.e.,\non its embedding), on the entity's name (i.e., on its surface form) and on its\ndescription in Wikipedia. The two metaviews language and representation can be\nfreely combined: each pair of language and representation (e.g., German\nembedding, English description, Spanish name) is a distinct view. Our\nexperiments on entity typing with fine-grained classes demonstrate the\neffectiveness of multiview learning. We release MVET, a large multiview - and,\nin particular, multilingual - entity typing dataset we created. Mono- and\nmultilingual fine-grained entity typing systems can be evaluated on this\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10942v2",
            "date": "2018-10-25 15:57:45+00:00",
            "title": "Understanding the Role of Two-Sided Argumentation in Online Consumer Reviews: A Language-Based Perspective",
            "authors": [
                "Bernhard Lutz",
                "Nicolas Pr\u00f6llochs",
                "Dirk Neumann"
            ],
            "abstract": "This paper examines the effect of two-sided argumentation on the perceived\nhelpfulness of online consumer reviews. In contrast to previous works, our\nanalysis thereby sheds light on the reception of reviews from a language-based\nperspective. For this purpose, we propose an intriguing text analysis approach\nbased on distributed text representations and multi-instance learning to\noperationalize the two-sidedness of argumentation in review texts. A subsequent\nempirical analysis using a large corpus of Amazon reviews suggests that\ntwo-sided argumentation in reviews significantly increases their helpfulness.\nWe find this effect to be stronger for positive reviews than for negative\nreviews, whereas a higher degree of emotional language weakens the effect. Our\nfindings have immediate implications for retailer platforms, which can utilize\nour results to optimize their customer feedback system and to present more\nuseful product reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11190v1",
            "date": "2018-10-26 05:04:07+00:00",
            "title": "Magnitude: A Fast, Efficient Universal Vector Embedding Utility Package",
            "authors": [
                "Ajay Patel",
                "Alexander Sands",
                "Chris Callison-Burch",
                "Marianna Apidianaki"
            ],
            "abstract": "Vector space embedding models like word2vec, GloVe, fastText, and ELMo are\nextremely popular representations in natural language processing (NLP)\napplications. We present Magnitude, a fast, lightweight tool for utilizing and\nprocessing embeddings. Magnitude is an open source Python package with a\ncompact vector storage file format that allows for efficient manipulation of\nhuge numbers of embeddings. Magnitude performs common operations up to 60 to\n6,000 times faster than Gensim. Magnitude introduces several novel features for\nimproved robustness like out-of-vocabulary lookups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11193v1",
            "date": "2018-10-26 05:44:01+00:00",
            "title": "Integrating Transformer and Paraphrase Rules for Sentence Simplification",
            "authors": [
                "Sanqiang Zhao",
                "Rui Meng",
                "Daqing He",
                "Saptono Andi",
                "Parmanto Bambang"
            ],
            "abstract": "Sentence simplification aims to reduce the complexity of a sentence while\nretaining its original meaning. Current models for sentence simplification\nadopted ideas from ma- chine translation studies and implicitly learned\nsimplification mapping rules from normal- simple sentence pairs. In this paper,\nwe explore a novel model based on a multi-layer and multi-head attention\narchitecture and we pro- pose two innovative approaches to integrate the Simple\nPPDB (A Paraphrase Database for Simplification), an external paraphrase\nknowledge base for simplification that covers a wide range of real-world\nsimplification rules. The experiments show that the integration provides two\nmajor benefits: (1) the integrated model outperforms multiple state- of-the-art\nbaseline models for sentence simplification in the literature (2) through\nanalysis of the rule utilization, the model seeks to select more accurate\nsimplification rules. The code and models used in the paper are available at\nhttps://github.com/ Sanqiang/text_simplification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11438v3",
            "date": "2018-10-26 17:43:44+00:00",
            "title": "American Sign Language fingerspelling recognition in the wild",
            "authors": [
                "Bowen Shi",
                "Aurora Martinez Del Rio",
                "Jonathan Keane",
                "Jonathan Michaux",
                "Diane Brentari",
                "Greg Shakhnarovich",
                "Karen Livescu"
            ],
            "abstract": "We address the problem of American Sign Language fingerspelling recognition\nin the wild, using videos collected from websites. We introduce the largest\ndata set available so far for the problem of fingerspelling recognition, and\nthe first using naturally occurring video data. Using this data set, we present\nthe first attempt to recognize fingerspelling sequences in this challenging\nsetting. Unlike prior work, our video data is extremely challenging due to low\nframe rates and visual variability. To tackle the visual challenges, we train a\nspecial-purpose signing hand detector using a small subset of our data. Given\nthe hand detector output, a sequence model decodes the hypothesized\nfingerspelled letter sequence. For the sequence model, we explore\nattention-based recurrent encoder-decoders and CTC-based approaches. As the\nfirst attempt at fingerspelling recognition in the wild, this work is intended\nto serve as a baseline for future work on sign language recognition in\nrealistic conditions. We find that, as expected, letter error rates are much\nhigher than in previous work on more controlled data, and we analyze the\nsources of error and effects of model variants.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1810.11612v3",
            "date": "2018-10-27 07:27:18+00:00",
            "title": "Handling Imbalanced Dataset in Multi-label Text Categorization using Bagging and Adaptive Boosting",
            "authors": [
                "Genta Indra Winata",
                "Masayu Leylia Khodra"
            ],
            "abstract": "Imbalanced dataset is occurred due to uneven distribution of data available\nin the real world such as disposition of complaints on government offices in\nBandung. Consequently, multi-label text categorization algorithms may not\nproduce the best performance because classifiers tend to be weighed down by the\nmajority of the data and ignore the minority. In this paper, Bagging and\nAdaptive Boosting algorithms are employed to handle the issue and improve the\nperformance of text categorization. The result is evaluated with four\nevaluation metrics such as hamming loss, subset accuracy, example-based\naccuracy and micro-averaged f-measure. Bagging ML-LP with SMO weak classifier\nis the best performer in terms of subset accuracy and example-based accuracy.\nBagging ML-BR with SMO weak classifier has the best micro-averaged f-measure\namong all. In other hand, AdaBoost MH with J48 weak classifier has the lowest\nhamming loss value. Thus, both algorithms have high potential in boosting the\nperformance of text categorization, but only for certain weak classifiers.\nHowever, bagging has more potential than adaptive boosting in increasing the\naccuracy of minority labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11878v2",
            "date": "2018-10-28 20:40:16+00:00",
            "title": "Unsupervised Evaluation Metrics and Learning Criteria for Non-Parallel Textual Transfer",
            "authors": [
                "Richard Yuanzhe Pang",
                "Kevin Gimpel"
            ],
            "abstract": "We consider the problem of automatically generating textual paraphrases with\nmodified attributes or properties, focusing on the setting without parallel\ndata (Hu et al., 2017; Shen et al., 2017). This setting poses challenges for\nevaluation. We show that the metric of post-transfer classification accuracy is\ninsufficient on its own, and propose additional metrics based on semantic\npreservation and fluency as well as a way to combine them into a single overall\nscore. We contribute new loss functions and training strategies to address the\ndifferent metrics. Semantic preservation is addressed by adding a cyclic\nconsistency loss and a loss based on paraphrase pairs, while fluency is\nimproved by integrating losses based on style-specific language models. We\nexperiment with a Yelp sentiment dataset and a new literature dataset that we\npropose, using multiple models that extend prior work (Shen et al., 2017). We\ndemonstrate that our metrics correlate well with human judgments, at both the\nsentence-level and system-level. Automatic and manual evaluation also show\nlarge improvements over the baseline method of Shen et al. (2017). We hope that\nour proposed metrics can speed up system development for new textual transfer\ntasks while also encouraging the community to address our three complementary\naspects of transfer quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11954v1",
            "date": "2018-10-20 16:58:54+00:00",
            "title": "A Knowledge-Grounded Multimodal Search-Based Conversational Agent",
            "authors": [
                "Shubham Agarwal",
                "Ondrej Dusek",
                "Ioannis Konstas",
                "Verena Rieser"
            ],
            "abstract": "Multimodal search-based dialogue is a challenging new task: It extends\nvisually grounded question answering systems into multi-turn conversations with\naccess to an external database. We address this new challenge by learning a\nneural response generation system from the recently released Multimodal\nDialogue (MMD) dataset (Saha et al., 2017). We introduce a knowledge-grounded\nmultimodal conversational model where an encoded knowledge base (KB)\nrepresentation is appended to the decoder input. Our model substantially\noutperforms strong baselines in terms of text-based similarity measures (over 9\nBLEU points, 3 of which are solely due to the use of additional information\nfrom the KB.",
            "journal ref": "Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International\n  Workshop on Search-Oriented Conversational AI, pages 59-66, Brussels,\n  Belgium, October 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12266v2",
            "date": "2018-10-29 17:24:57+00:00",
            "title": "The problem with probabilistic DAG automata for semantic graphs",
            "authors": [
                "Ieva Vasiljeva",
                "Sorcha Gilroy",
                "Adam Lopez"
            ],
            "abstract": "Semantic representations in the form of directed acyclic graphs (DAGs) have\nbeen introduced in recent years, and to model them, we need probabilistic\nmodels of DAGs. One model that has attracted some attention is the DAG\nautomaton, but it has not been studied as a probabilistic model. We show that\nsome DAG automata cannot be made into useful probabilistic models by the nearly\nuniversal strategy of assigning weights to transitions. The problem affects\nsingle-rooted, multi-rooted, and unbounded-degree variants of DAG automata, and\nappears to be pervasive. It does not affect planar variants, but these are\nproblematic for other reasons.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12387v1",
            "date": "2018-10-29 20:13:05+00:00",
            "title": "Language Modeling with Sparse Product of Sememe Experts",
            "authors": [
                "Yihong Gu",
                "Jun Yan",
                "Hao Zhu",
                "Zhiyuan Liu",
                "Ruobing Xie",
                "Maosong Sun",
                "Fen Lin",
                "Leyu Lin"
            ],
            "abstract": "Most language modeling methods rely on large-scale data to statistically\nlearn the sequential patterns of words. In this paper, we argue that words are\natomic language units but not necessarily atomic semantic units. Inspired by\nHowNet, we use sememes, the minimum semantic units in human languages, to\nrepresent the implicit semantics behind words for language modeling, named\nSememe-Driven Language Model (SDLM). More specifically, to predict the next\nword, SDLM first estimates the sememe distribution gave textual context.\nAfterward, it regards each sememe as a distinct semantic expert, and these\nexperts jointly identify the most probable senses and the corresponding word.\nIn this way, SDLM enables language models to work beyond word-level\nmanipulation to fine-grained sememe-level semantics and offers us more powerful\ntools to fine-tune language models and improve the interpretability as well as\nthe robustness of language models. Experiments on language modeling and the\ndownstream application of headline gener- ation demonstrate the significant\neffect of SDLM. Source code and data used in the experiments can be accessed at\nhttps:// github.com/thunlp/SDLM-pytorch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12427v1",
            "date": "2018-10-29 21:58:13+00:00",
            "title": "Parallel Attention Mechanisms in Neural Machine Translation",
            "authors": [
                "Julian Richard Medina",
                "Jugal Kalita"
            ],
            "abstract": "Recent papers in neural machine translation have proposed the strict use of\nattention mechanisms over previous standards such as recurrent and\nconvolutional neural networks (RNNs and CNNs). We propose that by running\ntraditionally stacked encoding branches from encoder-decoder attention- focused\narchitectures in parallel, that even more sequential operations can be removed\nfrom the model and thereby decrease training time. In particular, we modify the\nrecently published attention-based architecture called Transformer by Google,\nby replacing sequential attention modules with parallel ones, reducing the\namount of training time and substantially improving BLEU scores at the same\ntime. Experiments over the English to German and English to French translation\ntasks show that our model establishes a new state of the art.",
            "journal ref": "17th IEEE International Conference on Machine Learning and\n  Applications 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13091v2",
            "date": "2018-10-31 03:44:21+00:00",
            "title": "Towards End-to-End Code-Switching Speech Recognition",
            "authors": [
                "Ne Luo",
                "Dongwei Jiang",
                "Shuaijiang Zhao",
                "Caixia Gong",
                "Wei Zou",
                "Xiangang Li"
            ],
            "abstract": "Code-switching speech recognition has attracted an increasing interest\nrecently, but the need for expert linguistic knowledge has always been a big\nissue. End-to-end automatic speech recognition (ASR) simplifies the building of\nASR systems considerably by predicting graphemes or characters directly from\nacoustic input. In the mean time, the need of expert linguistic knowledge is\nalso eliminated, which makes it an attractive choice for code-switching ASR.\nThis paper presents a hybrid CTC-Attention based end-to-end Mandarin-English\ncode-switching (CS) speech recognition system and studies the effect of hybrid\nCTC-Attention based models, different modeling units, the inclusion of language\nidentification and different decoding strategies on the task of code-switching\nASR. On the SEAME corpus, our system achieves a mixed error rate (MER) of\n34.24%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00225v3",
            "date": "2018-11-01 04:51:20+00:00",
            "title": "Understanding Learning Dynamics Of Language Models with SVCCA",
            "authors": [
                "Naomi Saphra",
                "Adam Lopez"
            ],
            "abstract": "Research has shown that neural models implicitly encode linguistic features,\nbut there has been no research showing \\emph{how} these encodings arise as the\nmodels are trained. We present the first study on the learning dynamics of\nneural language models, using a simple and flexible analysis method called\nSingular Vector Canonical Correlation Analysis (SVCCA), which enables us to\ncompare learned representations across time and across models, without the need\nto evaluate directly on annotated data. We probe the evolution of syntactic,\nsemantic, and topic representations and find that part-of-speech is learned\nearlier than topic; that recurrent layers become more similar to those of a\ntagger during training; and embedding layers less similar. Our results and\nmethods could inform better learning algorithms for NLP models, possibly to\nincorporate linguistic information more effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00228v3",
            "date": "2018-11-01 05:03:26+00:00",
            "title": "A sequential guiding network with attention for image captioning",
            "authors": [
                "Daouda Sow",
                "Zengchang Qin",
                "Mouhamed Niasse",
                "Tao Wan"
            ],
            "abstract": "The recent advances of deep learning in both computer vision (CV) and natural\nlanguage processing (NLP) provide us a new way of understanding semantics, by\nwhich we can deal with more challenging tasks such as automatic description\ngeneration from natural images. In this challenge, the encoder-decoder\nframework has achieved promising performance when a convolutional neural\nnetwork (CNN) is used as image encoder and a recurrent neural network (RNN) as\ndecoder. In this paper, we introduce a sequential guiding network that guides\nthe decoder during word generation. The new model is an extension of the\nencoder-decoder framework with attention that has an additional guiding long\nshort-term memory (LSTM) and can be trained in an end-to-end manner by using\nimage/descriptions pairs. We validate our approach by conducting extensive\nexperiments on a benchmark dataset, i.e., MS COCO Captions. The proposed model\nachieves significant improvement comparing to the other state-of-the-art deep\nlearning models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.00491v3",
            "date": "2018-11-01 16:47:44+00:00",
            "title": "A Corpus for Reasoning About Natural Language Grounded in Photographs",
            "authors": [
                "Alane Suhr",
                "Stephanie Zhou",
                "Ally Zhang",
                "Iris Zhang",
                "Huajun Bai",
                "Yoav Artzi"
            ],
            "abstract": "We introduce a new dataset for joint reasoning about natural language and\nimages, with a focus on semantic diversity, compositionality, and visual\nreasoning challenges. The data contains 107,292 examples of English sentences\npaired with web photographs. The task is to determine whether a natural\nlanguage caption is true about a pair of photographs. We crowdsource the data\nusing sets of visually rich images and a compare-and-contrast task to elicit\nlinguistically diverse language. Qualitative analysis shows the data requires\ncompositional joint reasoning, including about quantities, comparisons, and\nrelations. Evaluation using state-of-the-art visual reasoning methods shows the\ndata presents a strong challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00552v2",
            "date": "2018-11-01 18:00:00+00:00",
            "title": "Multiple-Attribute Text Style Transfer",
            "authors": [
                "Sandeep Subramanian",
                "Guillaume Lample",
                "Eric Michael Smith",
                "Ludovic Denoyer",
                "Marc'Aurelio Ranzato",
                "Y-Lan Boureau"
            ],
            "abstract": "The dominant approach to unsupervised \"style transfer\" in text is based on\nthe idea of learning a latent representation, which is independent of the\nattributes specifying its \"style\". In this paper, we show that this condition\nis not necessary and is not always met in practice, even with domain\nadversarial training that explicitly aims at learning such disentangled\nrepresentations. We thus propose a new model that controls several factors of\nvariation in textual data where this condition on disentanglement is replaced\nwith a simpler mechanism based on back-translation. Our method allows control\nover multiple attributes, like gender, sentiment, product type, etc., and a\nmore fine-grained control on the trade-off between content preservation and\nchange of style with a pooling operator in the latent space. Our experiments\ndemonstrate that the fully entangled model produces better generations, even\nwhen tested on new and more challenging benchmarks comprising reviews with\nmultiple sentences and multiple attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00570v3",
            "date": "2018-11-01 18:11:01+00:00",
            "title": "On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing",
            "authors": [
                "Wasi Uddin Ahmad",
                "Zhisong Zhang",
                "Xuezhe Ma",
                "Eduard Hovy",
                "Kai-Wei Chang",
                "Nanyun Peng"
            ],
            "abstract": "Different languages might have different word orders. In this paper, we\ninvestigate cross-lingual transfer and posit that an order-agnostic model will\nperform better when transferring to distant foreign languages. To test our\nhypothesis, we train dependency parsers on an English corpus and evaluate their\ntransfer performance on 30 other languages. Specifically, we compare encoders\nand decoders based on Recurrent Neural Networks (RNNs) and modified\nself-attentive architectures. The former relies on sequential information while\nthe latter is more flexible at modeling word order. Rigorous experiments and\ndetailed analysis shows that RNN-based architectures transfer well to languages\nthat are close to English, while self-attentive models have better overall\ncross-lingual transferability and perform especially well on distant languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00606v2",
            "date": "2018-11-01 19:39:14+00:00",
            "title": "DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval",
            "authors": [
                "Zhiwen Tang",
                "Grace Hui Yang"
            ],
            "abstract": "Most neural Information Retrieval (Neu-IR) models derive query-to-document\nranking scores based on term-level matching. Inspired by TileBars, a classical\nterm distribution visualization method, in this paper, we propose a novel\nNeu-IR model that handles query-to-document matching at the subtopic and higher\nlevels. Our system first splits the documents into topical segments,\n\"visualizes\" the matchings between the query and the segments, and then feeds\nan interaction matrix into a Neu-IR model, DeepTileBars, to obtain the final\nranking scores. DeepTileBars models the relevance signals occurring at\ndifferent granularities in a document's topic hierarchy. It better captures the\ndiscourse structure of a document and thus the matching patterns. Although its\ndesign and implementation are light-weight, DeepTileBars outperforms other\nstate-of-the-art Neu-IR models on benchmark datasets including the Text\nREtrieval Conference (TREC) 2010-2012 Web Tracks and LETOR 4.0.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00614v1",
            "date": "2018-11-01 19:59:06+00:00",
            "title": "Exploring Semantic Incrementality with Dynamic Syntax and Vector Space Semantics",
            "authors": [
                "Mehrnoosh Sadrzadeh",
                "Matthew Purver",
                "Julian Hough",
                "Ruth Kempson"
            ],
            "abstract": "One of the fundamental requirements for models of semantic processing in\ndialogue is incrementality: a model must reflect how people interpret and\ngenerate language at least on a word-by-word basis, and handle phenomena such\nas fragments, incomplete and jointly-produced utterances. We show that the\nincremental word-by-word parsing process of Dynamic Syntax (DS) can be assigned\na compositional distributional semantics, with the composition operator of DS\ncorresponding to the general operation of tensor contraction from multilinear\nalgebra. We provide abstract semantic decorations for the nodes of DS trees, in\nterms of vectors, tensors, and sums thereof; using the latter to model the\nunderspecified elements crucial to assigning partial representations during\nincremental processing. As a working example, we give an instantiation of this\ntheory using plausibility tensors of compositional distributional semantics,\nand show how our framework can incrementally assign a semantic plausibility\nmeasure as it parses phrases and sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00671v2",
            "date": "2018-11-01 23:10:03+00:00",
            "title": "Dialogue Natural Language Inference",
            "authors": [
                "Sean Welleck",
                "Jason Weston",
                "Arthur Szlam",
                "Kyunghyun Cho"
            ],
            "abstract": "Consistency is a long standing issue faced by dialogue models. In this paper,\nwe frame the consistency of dialogue agents as natural language inference (NLI)\nand create a new natural language inference dataset called Dialogue NLI. We\npropose a method which demonstrates that a model trained on Dialogue NLI can be\nused to improve the consistency of a dialogue model, and evaluate the method\nwith human evaluation and with automatic metrics on a suite of evaluation sets\ndesigned to measure a dialogue model's consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00681v2",
            "date": "2018-11-01 23:50:43+00:00",
            "title": "On the Generation of Medical Question-Answer Pairs",
            "authors": [
                "Sheng Shen",
                "Yaliang Li",
                "Nan Du",
                "Xian Wu",
                "Yusheng Xie",
                "Shen Ge",
                "Tao Yang",
                "Kai Wang",
                "Xingzheng Liang",
                "Wei Fan"
            ],
            "abstract": "Question answering (QA) has achieved promising progress recently. However,\nanswering a question in real-world scenarios like the medical domain is still\nchallenging, due to the requirement of external knowledge and the insufficient\nquantity of high-quality training data. In the light of these challenges, we\nstudy the task of generating medical QA pairs in this paper. With the insight\nthat each medical question can be considered as a sample from the latent\ndistribution of questions given answers, we propose an automated medical QA\npair generation framework, consisting of an unsupervised key phrase detector\nthat explores unstructured material for validity, and a generator that involves\na multi-pass decoder to integrate structural knowledge for diversity. A series\nof experiments have been conducted on a real-world dataset collected from the\nNational Medical Licensing Examination of China. Both automatic evaluation and\nhuman annotation demonstrate the effectiveness of the proposed method. Further\ninvestigation shows that, by incorporating the generated QA pairs for training,\nsignificant improvement in terms of accuracy can be achieved for the\nexamination QA system.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.00692v1",
            "date": "2018-11-02 01:02:49+00:00",
            "title": "Zero-Shot Transfer VQA Dataset",
            "authors": [
                "Yuanpeng Li",
                "Yi Yang",
                "Jianyu Wang",
                "Wei Xu"
            ],
            "abstract": "Acquiring a large vocabulary is an important aspect of human intelligence.\nOnecommon approach for human to populating vocabulary is to learn words\nduringreading or listening, and then use them in writing or speaking. This\nability totransfer from input to output is natural for human, but it is\ndifficult for machines.Human spontaneously performs this knowledge transfer in\ncomplicated multimodaltasks, such as Visual Question Answering (VQA). In order\nto approach human-levelArtificial Intelligence, we hope to equip machines with\nsuch ability. Therefore, toaccelerate this research, we propose a newzero-shot\ntransfer VQA(ZST-VQA)dataset by reorganizing the existing VQA v1.0 dataset in\nthe way that duringtraining, some words appear only in one module (i.e.\nquestions) but not in theother (i.e. answers). In this setting, an intelligent\nmodel should understand andlearn the concepts from one module (i.e. questions),\nand at test time, transfer themto the other (i.e. predict the concepts as\nanswers). We conduct evaluation on thisnew dataset using three existing\nstate-of-the-art VQA neural models. Experimentalresults show a significant drop\nin performance on this dataset, indicating existingmethods do not address the\nzero-shot transfer problem. Besides, our analysis findsthat this may be caused\nby the implicit bias learned during training.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.00739v1",
            "date": "2018-11-02 05:05:26+00:00",
            "title": "An Empirical Exploration of Curriculum Learning for Neural Machine Translation",
            "authors": [
                "Xuan Zhang",
                "Gaurav Kumar",
                "Huda Khayrallah",
                "Kenton Murray",
                "Jeremy Gwinnup",
                "Marianna J Martindale",
                "Paul McNamee",
                "Kevin Duh",
                "Marine Carpuat"
            ],
            "abstract": "Machine translation systems based on deep neural networks are expensive to\ntrain. Curriculum learning aims to address this issue by choosing the order in\nwhich samples are presented during training to help train better models faster.\nWe adopt a probabilistic view of curriculum learning, which lets us flexibly\nevaluate the impact of curricula design, and perform an extensive exploration\non a German-English translation task. Results show that it is possible to\nimprove convergence time at no loss in translation quality. However, results\nare highly sensitive to the choice of sample difficulty criteria, curriculum\nschedule and other hyperparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00770v2",
            "date": "2018-11-02 08:10:21+00:00",
            "title": "A Survey on Natural Language Processing for Fake News Detection",
            "authors": [
                "Ray Oshikawa",
                "Jing Qian",
                "William Yang Wang"
            ],
            "abstract": "Fake news detection is a critical yet challenging problem in Natural Language\nProcessing (NLP). The rapid rise of social networking platforms has not only\nyielded a vast increase in information accessibility but has also accelerated\nthe spread of fake news. Thus, the effect of fake news has been growing,\nsometimes extending to the offline world and threatening public safety. Given\nthe massive amount of Web content, automatic fake news detection is a practical\nNLP problem useful to all online content providers, in order to reduce the\nhuman time and effort to detect and prevent the spread of fake news. In this\npaper, we describe the challenges involved in fake news detection and also\ndescribe related tasks. We systematically review and compare the task\nformulations, datasets and NLP solutions that have been developed for this\ntask, and also discuss the potentials and limitations of them. Based on our\ninsights, we outline promising research directions, including more\nfine-grained, detailed, fair, and practical detection models. We also highlight\nthe difference between fake news detection and other related tasks, and the\nimportance of NLP solutions for fake news detection.",
            "journal ref": "Proceedings of the 12th Language Resources and Evaluation\n  Conference (LREC 2020) pp. 6086-6093",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00780v3",
            "date": "2018-11-02 08:50:11+00:00",
            "title": "A Bayesian Approach for Sequence Tagging with Crowds",
            "authors": [
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "Current methods for sequence tagging, a core task in NLP, are data hungry,\nwhich motivates the use of crowdsourcing as a cheap way to obtain labelled\ndata. However, annotators are often unreliable and current aggregation methods\ncannot capture common types of span annotation errors. To address this, we\npropose a Bayesian method for aggregating sequence tags that reduces errors by\nmodelling sequential dependencies between the annotations as well as the\nground-truth labels. By taking a Bayesian approach, we account for uncertainty\nin the model due to both annotator errors and the lack of data for modelling\nannotators who complete few tasks. We evaluate our model on crowdsourced data\nfor named entity recognition, information extraction and argument mining,\nshowing that our sequential model outperforms the previous state of the art. We\nalso find that our approach can reduce crowdsourcing costs through more\neffective active learning, as it better captures uncertainty in the sequence\nlabels when there are few annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00845v1",
            "date": "2018-11-02 13:22:19+00:00",
            "title": "Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction",
            "authors": [
                "Angrosh Mandya",
                "Danushka Bollegala",
                "Frans Coenen",
                "Katie Atkinson"
            ],
            "abstract": "We propose in this paper a combined model of Long Short Term Memory and\nConvolutional Neural Networks (LSTM-CNN) that exploits word embeddings and\npositional embeddings for cross-sentence n-ary relation extraction. The\nproposed model brings together the properties of both LSTMs and CNNs, to\nsimultaneously exploit long-range sequential information and capture most\ninformative features, essential for cross-sentence n-ary relation extraction.\nThe LSTM-CNN model is evaluated on standard dataset on cross-sentence n-ary\nrelation extraction, where it significantly outperforms baselines such as CNNs,\nLSTMs and also a combined CNN-LSTM model. The paper also shows that the\nLSTM-CNN model outperforms the current state-of-the-art methods on\ncross-sentence n-ary relation extraction.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00854v1",
            "date": "2018-11-01 17:48:18+00:00",
            "title": "Improving Information Retrieval Results for Persian Documents using FarsNet",
            "authors": [
                "Adel Rahimi",
                "Mohammad Bahrani"
            ],
            "abstract": "In this paper, we propose a new method for query expansion, which uses\nFarsNet (Persian WordNet) to find similar tokens related to the query and\nexpand the semantic meaning of the query. For this purpose, we use synonymy\nrelations in FarsNet and extract the related synonyms to query words. This\nalgorithm is used to enhance information retrieval systems and improve search\nresults. The overall evaluation of this system in comparison to the baseline\nmethod (without using query expansion) shows an improvement of about 9 percent\nin Mean Average Precision (MAP).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00907v3",
            "date": "2018-11-02 14:54:50+00:00",
            "title": "Importance of Search and Evaluation Strategies in Neural Dialogue Modeling",
            "authors": [
                "Ilia Kulikov",
                "Alexander H. Miller",
                "Kyunghyun Cho",
                "Jason Weston"
            ],
            "abstract": "We investigate the impact of search strategies in neural dialogue modeling.\nWe first compare two standard search algorithms, greedy and beam search, as\nwell as our newly proposed iterative beam search which produces a more diverse\nset of candidate responses. We evaluate these strategies in realistic full\nconversations with humans and propose a model-based Bayesian calibration to\naddress annotator bias. These conversations are analyzed using two automatic\nmetrics: log-probabilities assigned by the model and utterance diversity. Our\nexperiments reveal that better search algorithms lead to higher rated\nconversations. However, finding the optimal selection mechanism to choose from\na more diverse set of candidates is still an open question.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01012v1",
            "date": "2018-11-02 17:56:23+00:00",
            "title": "Unsupervised Learning of Interpretable Dialog Models",
            "authors": [
                "Dhiraj Madan",
                "Dinesh Raghu",
                "Gaurav Pandey",
                "Sachindra Joshi"
            ],
            "abstract": "Recently several deep learning based models have been proposed for end-to-end\nlearning of dialogs. While these models can be trained from data without the\nneed for any additional annotations, it is hard to interpret them. On the other\nhand, there exist traditional state based dialog systems, where the states of\nthe dialog are discrete and hence easy to interpret. However these states need\nto be handcrafted and annotated in the data. To achieve the best of both\nworlds, we propose Latent State Tracking Network (LSTN) using which we learn an\ninterpretable model in unsupervised manner. The model defines a discrete latent\nvariable at each turn of the conversation which can take a finite set of\nvalues. Since these discrete variables are not present in the training data, we\nuse EM algorithm to train our model in unsupervised manner. In the experiments,\nwe show that LSTN can help achieve interpretability in dialog models without\nmuch decrease in performance compared to end-to-end approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.01124v3",
            "date": "2018-11-02 23:30:54+00:00",
            "title": "Unsupervised Hyperalignment for Multilingual Word Embeddings",
            "authors": [
                "Jean Alaux",
                "Edouard Grave",
                "Marco Cuturi",
                "Armand Joulin"
            ],
            "abstract": "We consider the problem of aligning continuous word representations, learned\nin multiple languages, to a common space. It was recently shown that, in the\ncase of two languages, it is possible to learn such a mapping without\nsupervision. This paper extends this line of work to the problem of aligning\nmultiple languages to a common space. A solution is to independently map all\nlanguages to a pivot language. Unfortunately, this degrades the quality of\nindirect word translation. We thus propose a novel formulation that ensures\ncomposable mappings, leading to better alignments. We evaluate our method by\njointly aligning word vectors in eleven languages, showing consistent\nimprovement with indirect mappings while maintaining competitive performance on\ndirect word translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01127v2",
            "date": "2018-11-02 23:48:46+00:00",
            "title": "Exploiting Explicit Paths for Multi-hop Reading Comprehension",
            "authors": [
                "Souvik Kundu",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Peter Clark"
            ],
            "abstract": "We propose a novel, path-based reasoning approach for the multi-hop reading\ncomprehension task where a system needs to combine facts from multiple passages\nto answer a question. Although inspired by multi-hop reasoning over knowledge\ngraphs, our proposed approach operates directly over unstructured text. It\ngenerates potential paths through passages and scores them without any direct\npath supervision. The proposed model, named PathNet, attempts to extract\nimplicit relations from text through entity pair representations, and compose\nthem to encode each path. To capture additional context, PathNet also composes\nthe passage representations along each path to compute a passage-based\nrepresentation. Unlike previous approaches, our model is then able to explain\nits reasoning via these explicit paths through the passages. We show that our\napproach outperforms prior models on the multi-hop Wikihop dataset, and also\ncan be generalized to apply to the OpenBookQA dataset, matching\nstate-of-the-art performance.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01183v1",
            "date": "2018-11-03 09:29:36+00:00",
            "title": "Unsupervised Identification of Study Descriptors in Toxicology Research: An Experimental Study",
            "authors": [
                "Drahomira Herrmannova",
                "Steven R. Young",
                "Robert M. Patton",
                "Christopher G. Stahl",
                "Nicole C. Kleinstreuer",
                "Mary S. Wolfe"
            ],
            "abstract": "Identifying and extracting data elements such as study descriptors in\npublication full texts is a critical yet manual and labor-intensive step\nrequired in a number of tasks. In this paper we address the question of\nidentifying data elements in an unsupervised manner. Specifically, provided a\nset of criteria describing specific study parameters, such as species, route of\nadministration, and dosing regimen, we develop an unsupervised approach to\nidentify text segments (sentences) relevant to the criteria. A binary\nclassifier trained to identify publications that met the criteria performs\nbetter when trained on the candidate sentences than when trained on sentences\nrandomly picked from the text, supporting the intuition that our method is able\nto accurately identify study descriptors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01294v1",
            "date": "2018-11-03 22:47:59+00:00",
            "title": "Learning Contextual Hierarchical Structure of Medical Concepts with Poincair\u00e9 Embeddings to Clarify Phenotypes",
            "authors": [
                "Brett K. Beaulieu-Jones",
                "Isaac S. Kohane",
                "Andrew L. Beam"
            ],
            "abstract": "Biomedical association studies are increasingly done using clinical concepts,\nand in particular diagnostic codes from clinical data repositories as\nphenotypes. Clinical concepts can be represented in a meaningful, vector space\nusing word embedding models. These embeddings allow for comparison between\nclinical concepts or for straightforward input to machine learning models.\nUsing traditional approaches, good representations require high dimensionality,\nmaking downstream tasks such as visualization more difficult. We applied\nPoincar\\'e embeddings in a 2-dimensional hyperbolic space to a large-scale\nadministrative claims database and show performance comparable to\n100-dimensional embeddings in a euclidean space. We then examine disease\nrelationships under different disease contexts to better understand potential\nphenotypes.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1811.01304v2",
            "date": "2018-11-04 00:26:00+00:00",
            "title": "ColNet: Embedding the Semantics of Web Tables for Column Type Prediction",
            "authors": [
                "Jiaoyan Chen",
                "Ernesto Jimenez-Ruiz",
                "Ian Horrocks",
                "Charles Sutton"
            ],
            "abstract": "Automatically annotating column types with knowledge base (KB) concepts is a\ncritical task to gain a basic understanding of web tables. Current methods rely\non either table metadata like column name or entity correspondences of cells in\nthe KB, and may fail to deal with growing web tables with incomplete meta\ninformation. In this paper we propose a neural network based column type\nannotation framework named ColNet which is able to integrate KB reasoning and\nlookup with machine learning and can automatically train Convolutional Neural\nNetworks for prediction. The prediction model not only considers the contextual\nsemantics within a cell using word representation, but also embeds the\nsemantics of a column by learning locality features from multiple cells. The\nmethod is evaluated with DBPedia and two different web table datasets, T2Dv2\nfrom the general Web and Limaye from Wikipedia pages, and achieves higher\nperformance than the state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01399v2",
            "date": "2018-11-04 16:39:28+00:00",
            "title": "Logic Attention Based Neighborhood Aggregation for Inductive Knowledge Graph Embedding",
            "authors": [
                "Peifeng Wang",
                "Jialong Han",
                "Chenliang Li",
                "Rong Pan"
            ],
            "abstract": "Knowledge graph embedding aims at modeling entities and relations with\nlow-dimensional vectors. Most previous methods require that all entities should\nbe seen during training, which is unpractical for real-world knowledge graphs\nwith new entities emerging on a daily basis. Recent efforts on this issue\nsuggest training a neighborhood aggregator in conjunction with the conventional\nentity and relation embeddings, which may help embed new entities inductively\nvia their existing neighbors. However, their neighborhood aggregators neglect\nthe unordered and unequal natures of an entity's neighbors. To this end, we\nsummarize the desired properties that may lead to effective neighborhood\naggregators. We also introduce a novel aggregator, namely, Logic Attention\nNetwork (LAN), which addresses the properties by aggregating neighbors with\nboth rules- and network-based attention weights. By comparing with conventional\naggregators on two knowledge graph completion tasks, we experimentally validate\nLAN's superiority in terms of the desired properties.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.01409v1",
            "date": "2018-11-04 17:57:07+00:00",
            "title": "Semantic Role Labeling for Knowledge Graph Extraction from Text",
            "authors": [
                "Mehwish Alam",
                "Aldo Gangemi",
                "Valentina Presutti",
                "Diego Reforgiato Recupero"
            ],
            "abstract": "This paper introduces TakeFive, a new semantic role labeling method that\ntransforms a text into a frame-oriented knowledge graph. It performs dependency\nparsing, identifies the words that evoke lexical frames, locates the roles and\nfillers for each frame, runs coercion techniques, and formalises the results as\na knowledge graph. This formal representation complies with the frame semantics\nused in Framester, a factual-linguistic linked data resource. The obtained\nprecision, recall and F1 values indicate that TakeFive is competitive with\nother existing methods such as SEMAFOR, Pikes, PathLSTM and FRED. We finally\ndiscuss how to combine TakeFive and FRED, obtaining higher values of precision,\nrecall and F1.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.01468v1",
            "date": "2018-11-05 00:54:03+00:00",
            "title": "Medical code prediction with multi-view convolution and description-regularized label-dependent attention",
            "authors": [
                "Najmeh Sadoughi",
                "Greg P. Finley",
                "James Fone",
                "Vignesh Murali",
                "Maxim Korenevski",
                "Slava Baryshnikov",
                "Nico Axtmann",
                "Mark Miller",
                "David Suendermann-Oeft"
            ],
            "abstract": "A ubiquitous task in processing electronic medical data is the assignment of\nstandardized codes representing diagnoses and/or procedures to free-text\ndocuments such as medical reports. This is a difficult natural language\nprocessing task that requires parsing long, heterogeneous documents and\nselecting a set of appropriate codes from tens of thousands of\npossibilities---many of which have very few positive training samples. We\npresent a deep learning system that advances the state of the art for the\nMIMIC-III dataset, achieving a new best micro F1-measure of 55.85\\%,\nsignificantly outperforming the previous best result (Mullenbach et al. 2018).\nWe achieve this through a number of enhancements, including two major novel\ncontributions: multi-view convolutional channels, which effectively learn to\nadjust kernel sizes throughout the input; and attention regularization,\nmediated by natural-language code descriptions, which helps overcome sparsity\nfor thousands of uncommon codes. These and other modifications are selected to\naddress difficulties inherent to both automated coding specifically and deep\nlearning generally. Finally, we investigate our accuracy results in detail to\nindividually measure the impact of these contributions and point the way\ntowards future algorithmic improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01727v3",
            "date": "2018-11-01 16:30:33+00:00",
            "title": "AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification",
            "authors": [
                "Ronghui You",
                "Zihan Zhang",
                "Ziye Wang",
                "Suyang Dai",
                "Hiroshi Mamitsuka",
                "Shanfeng Zhu"
            ],
            "abstract": "Extreme multi-label text classification (XMTC) is an important problem in the\nera of big data, for tagging a given text with the most relevant multiple\nlabels from an extremely large-scale label set. XMTC can be found in many\napplications, such as item categorization, web page tagging, and news\nannotation. Traditionally most methods used bag-of-words (BOW) as inputs,\nignoring word context as well as deep semantic information. Recent attempts to\novercome the problems of BOW by deep learning still suffer from 1) failing to\ncapture the important subtext for each label and 2) lack of scalability against\nthe huge number of labels. We propose a new label tree-based deep learning\nmodel for XMTC, called AttentionXML, with two unique features: 1) a multi-label\nattention mechanism with raw text as input, which allows to capture the most\nrelevant part of text to each label; and 2) a shallow and wide probabilistic\nlabel tree (PLT), which allows to handle millions of labels, especially for\n\"tail labels\". We empirically compared the performance of AttentionXML with\nthose of eight state-of-the-art methods over six benchmark datasets, including\nAmazon-3M with around 3 million labels. AttentionXML outperformed all competing\nmethods under all experimental settings. Experimental results also show that\nAttentionXML achieved the best performance against tail labels among label\ntree-based methods. The code and datasets are available at\nhttp://github.com/yourh/AttentionXML .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01734v1",
            "date": "2018-11-02 08:05:36+00:00",
            "title": "Transductive Learning with String Kernels for Cross-Domain Text Classification",
            "authors": [
                "Radu Tudor Ionescu",
                "Andrei M. Butnaru"
            ],
            "abstract": "For many text classification tasks, there is a major problem posed by the\nlack of labeled data in a target domain. Although classifiers for a target\ndomain can be trained on labeled text data from a related source domain, the\naccuracy of such classifiers is usually lower in the cross-domain setting.\nRecently, string kernels have obtained state-of-the-art results in various text\nclassification tasks such as native language identification or automatic essay\nscoring. Moreover, classifiers based on string kernels have been found to be\nrobust to the distribution gap between different domains. In this paper, we\nformally describe an algorithm composed of two simple yet effective\ntransductive learning approaches to further improve the results of string\nkernels in cross-domain settings. By adapting string kernels to the test set\nwithout using the ground-truth test labels, we report significantly better\naccuracy rates in cross-domain English polarity classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01786v1",
            "date": "2018-11-05 15:21:40+00:00",
            "title": "A human-editable Sign Language representation for software editing---and a writing system?",
            "authors": [
                "Michael Filhol"
            ],
            "abstract": "To equip SL with software properly, we need an input system to represent and\nmanipulate signed contents in the same way that every day software allows to\nprocess written text. Refuting the claim that video is good enough a medium to\nserve the purpose, we propose to build a representation that is: editable,\nqueryable, synthesisable and user-friendly---we define those terms upfront. The\nissue being functionally and conceptually linked to that of writing, we study\nexisting writing systems, namely those in use for vocal languages, those\ndesigned and proposed for SLs, and more spontaneous ways in which SL users put\ntheir language in writing. Observing each paradigm in turn, we move on to\npropose a new approach to satisfy our goals of integration in software. We\nfinally open the prospect of our proposition being used outside of this\nrestricted scope, as a writing system in itself, and compare its properties to\nthe other writing systems presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01938v1",
            "date": "2018-11-05 10:46:45+00:00",
            "title": "A personal model of trumpery: Deception detection in a real-world high-stakes setting",
            "authors": [
                "Sophie van der Zee",
                "Ronald Poppe",
                "Alice Havrileck",
                "Aurelien Baillon"
            ],
            "abstract": "Language use reveals information about who we are and how we feel1-3. One of\nthe pioneers in text analysis, Walter Weintraub, manually counted which types\nof words people used in medical interviews and showed that the frequency of\nfirst-person singular pronouns (i.e., I, me, my) was a reliable indicator of\ndepression, with depressed people using I more often than people who are not\ndepressed4. Several studies have demonstrated that language use also differs\nbetween truthful and deceptive statements5-7, but not all differences are\nconsistent across people and contexts, making prediction difficult8. Here we\nshow how well linguistic deception detection performs at the individual level\nby developing a model tailored to a single individual: the current US\npresident. Using tweets fact-checked by an independent third party (Washington\nPost), we found substantial linguistic differences between factually correct\nand incorrect tweets and developed a quantitative model based on these\ndifferences. Next, we predicted whether out-of-sample tweets were either\nfactually correct or incorrect and achieved a 73% overall accuracy. Our results\ndemonstrate the power of linguistic analysis in real-world deception research\nwhen applied at the individual level and provide evidence that factually\nincorrect tweets are not random mistakes of the sender.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02166v2",
            "date": "2018-11-06 05:08:59+00:00",
            "title": "DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction",
            "authors": [
                "Shun Zheng",
                "Xu Han",
                "Yankai Lin",
                "Peilin Yu",
                "Lu Chen",
                "Ling Huang",
                "Zhiyuan Liu",
                "Wei Xu"
            ],
            "abstract": "Pattern-based labeling methods have achieved promising results in alleviating\nthe inevitable labeling noises of distantly supervised neural relation\nextraction. However, these methods require significant expert labor to write\nrelation-specific patterns, which makes them too sophisticated to generalize\nquickly.To ease the labor-intensive workload of pattern writing and enable the\nquick generalization to new relation types, we propose a neural pattern\ndiagnosis framework, DIAG-NRE, that can automatically summarize and refine\nhigh-quality relational patterns from noise data with human experts in the\nloop. To demonstrate the effectiveness of DIAG-NRE, we apply it to two\nreal-world datasets and present both significant and interpretable improvements\nover state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02318v1",
            "date": "2018-11-06 12:28:58+00:00",
            "title": "Recurrent Skipping Networks for Entity Alignment",
            "authors": [
                "Lingbing Guo",
                "Zequn Sun",
                "Ermei Cao",
                "Wei Hu"
            ],
            "abstract": "We consider the problem of learning knowledge graph (KG) embeddings for\nentity alignment (EA). Current methods use the embedding models mainly focusing\non triple-level learning, which lacks the ability of capturing long-term\ndependencies existing in KGs. Consequently, the embedding-based EA methods\nheavily rely on the amount of prior (known) alignment, due to the identity\ninformation in the prior alignment cannot be efficiently propagated from one KG\nto another. In this paper, we propose RSN4EA (recurrent skipping networks for\nEA), which leverages biased random walk sampling for generating long paths\nacross KGs and models the paths with a novel recurrent skipping network (RSN).\nRSN integrates the conventional recurrent neural network (RNN) with residual\nlearning and can largely improve the convergence speed and performance with\nonly a few more parameters. We evaluated RSN4EA on a series of datasets\nconstructed from real-world KGs. Our experimental results showed that it\noutperformed a number of state-of-the-art embedding-based EA methods and also\nachieved comparable performance for KG completion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02320v1",
            "date": "2018-11-06 12:32:27+00:00",
            "title": "Hierarchical Neural Network Architecture In Keyword Spotting",
            "authors": [
                "Yixiao Qu",
                "Sihao Xue",
                "Zhenyi Ying",
                "Hang Zhou",
                "Jue Sun"
            ],
            "abstract": "Keyword Spotting (KWS) provides the start signal of ASR problem, and thus it\nis essential to ensure a high recall rate. However, its real-time property\nrequires low computation complexity. This contradiction inspires people to find\na suitable model which is small enough to perform well in multi environments.\nTo deal with this contradiction, we implement the Hierarchical Neural\nNetwork(HNN), which is proved to be effective in many speech recognition\nproblems. HNN outperforms traditional DNN and CNN even though its model size\nand computation complexity are slightly less. Also, its simple topology\nstructure makes easy to deploy on any device.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02456v1",
            "date": "2018-11-06 16:08:31+00:00",
            "title": "Semantic Term \"Blurring\" and Stochastic \"Barcoding\" for Improved Unsupervised Text Classification",
            "authors": [
                "Robert Frank Martorano III"
            ],
            "abstract": "The abundance of text data being produced in the modern age makes it\nincreasingly important to intuitively group, categorize, or classify text data\nby theme for efficient retrieval and search. Yet, the high dimensionality and\nimprecision of text data, or more generally language as a whole, prove to be\nchallenging when attempting to perform unsupervised document clustering. In\nthis thesis, we present two novel methods for improving unsupervised document\nclustering/classification by theme. The first is to improve document\nrepresentations. We look to exploit \"term neighborhoods\" and \"blur\" semantic\nweight across neighboring terms. These neighborhoods are located in the\nsemantic space afforded by \"word embeddings.\" The second method is for cluster\nrevision, based on what we deem as \"stochastic barcoding\", or \"S- Barcode\"\npatterns. Text data is inherently high dimensional, yet clustering typically\ntakes place in a low dimensional representation space. Our method utilizes\nlower dimension clustering results as initial cluster configurations, and\niteratively revises the configuration in the high dimensional space. We show\nwith experimental results how both of the two methods improve the quality of\ndocument clustering. While this thesis elaborates on the two new conceptual\ncontributions, a joint thesis by David Yan details the feature transformation\nand software architecture we developed for unsupervised document\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02507v2",
            "date": "2018-11-05 05:07:37+00:00",
            "title": "Superregular grammars do not provide additional explanatory power but allow for a compact analysis of animal song",
            "authors": [
                "Takashi Morita",
                "Hiroki Koda"
            ],
            "abstract": "A pervasive belief with regard to the differences between human language and\nanimal vocal sequences (song) is that they belong to different classes of\ncomputational complexity, with animal song belonging to regular languages,\nwhereas human language is superregular. This argument, however, lacks empirical\nevidence since superregular analyses of animal song are understudied. The goal\nof this paper is to perform a superregular analysis of animal song, using data\nfrom gibbons as a case study, and demonstrate that a superregular analysis can\nbe effectively used with non-human data. A key finding is that a superregular\nanalysis does not increase explanatory power but rather provides for compact\nanalysis: Fewer grammatical rules are necessary once superregularity is\nallowed. This pattern is analogous to a previous computational analysis of\nhuman language, and accordingly, the null hypothesis, that human language and\nanimal song are governed by the same type of grammatical systems, cannot be\nrejected.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1811.02528v2",
            "date": "2018-11-06 17:53:54+00:00",
            "title": "Discriminative training of RNNLMs with the average word error criterion",
            "authors": [
                "R\u00e9mi Francis",
                "Tom Ash",
                "Will Williams"
            ],
            "abstract": "In automatic speech recognition (ASR), recurrent neural language models\n(RNNLM) are typically used to refine hypotheses in the form of lattices or\nn-best lists, which are generated by a beam search decoder with a weaker\nlanguage model. The RNNLMs are usually trained generatively using the\nperplexity (PPL) criterion on large corpora of grammatically correct text.\nHowever, the hypotheses are noisy, and the RNNLM doesn't always make the\nchoices that minimise the metric we optimise for, the word error rate (WER). To\naddress this mismatch we propose to use a task specific loss to train an RNNLM\nto discriminate between multiple hypotheses within lattice rescoring scenario.\nBy fine-tuning the RNNLM on lattices with the average edit distance loss, we\nshow that we obtain a 1.9% relative improvement in word error rate over a\npurely generatively trained model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02549v6",
            "date": "2018-11-06 18:44:11+00:00",
            "title": "Language GANs Falling Short",
            "authors": [
                "Massimo Caccia",
                "Lucas Caccia",
                "William Fedus",
                "Hugo Larochelle",
                "Joelle Pineau",
                "Laurent Charlin"
            ],
            "abstract": "Generating high-quality text with sufficient diversity is essential for a\nwide range of Natural Language Generation (NLG) tasks. Maximum-Likelihood (MLE)\nmodels trained with teacher forcing have consistently been reported as weak\nbaselines, where poor performance is attributed to exposure bias (Bengio et\nal., 2015; Ranzato et al., 2015); at inference time, the model is fed its own\nprediction instead of a ground-truth token, which can lead to accumulating\nerrors and poor samples. This line of reasoning has led to an outbreak of\nadversarial based approaches for NLG, on the account that GANs do not suffer\nfrom exposure bias. In this work, we make several surprising observations which\ncontradict common beliefs. First, we revisit the canonical evaluation framework\nfor NLG, and point out fundamental flaws with quality-only evaluation: we show\nthat one can outperform such metrics using a simple, well-known temperature\nparameter to artificially reduce the entropy of the model's conditional\ndistributions. Second, we leverage the control over the quality / diversity\ntrade-off given by this parameter to evaluate models over the whole\nquality-diversity spectrum and find MLE models constantly outperform the\nproposed GAN variants over the whole quality-diversity space. Our results have\nseveral implications: 1) The impact of exposure bias on sample quality is less\nsevere than previously thought, 2) temperature tuning provides a better quality\n/ diversity trade-off than adversarial training while being easier to train,\neasier to cross-validate, and less computationally expensive. Code to reproduce\nthe experiments is available at github.com/pclucas14/GansFallingShort",
            "journal ref": "ICLR 2020 - Proceedings of the Seventh International Conference on\n  Learning Representation",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02750v1",
            "date": "2018-11-07 04:11:15+00:00",
            "title": "The relationship between linguistic expression and symptoms of depression, anxiety, and suicidal thoughts: A longitudinal study of blog content",
            "authors": [
                "B. ODea",
                "T. W. Boonstra",
                "M. E. Larsen",
                "T. Nguyen",
                "S. Venkatesh",
                "H. Christensen"
            ],
            "abstract": "Due to its popularity and availability, social media data may present a new\nway to identify individuals who are experiencing mental illness. By analysing\nblog content, this study aimed to investigate the associations between\nlinguistic features and symptoms of depression, generalised anxiety, and\nsuicidal ideation. This study utilised a longitudinal study design. Individuals\nwho blogged were invited to participate in a study in which they completed\nfortnightly mental health questionnaires including the PHQ9 and GAD7 for a\nperiod of 36 weeks. Linguistic features were extracted from blog data using the\nLIWC tool. Bivariate and multivariate analyses were performed to investigate\nthe correlations between the linguistic features and mental health scores\nbetween subjects. We then used the multivariate regression model to predict\nlongitudinal changes in mood within subjects. A total of 153 participants\nconsented to taking part, with 38 participants completing the required number\nof questionnaires and blog posts during the study period. Between-subject\nanalysis revealed that several linguistic features, including tentativeness and\nnon-fluencies, were significantly associated with depression and anxiety\nsymptoms, but not suicidal thoughts. Within-subject analysis showed no robust\ncorrelations between linguistic features and changes in mental health score.\nThis study provides further support for the relationship between linguistic\nfeatures within social media data and symptoms of depression and anxiety. The\nlack of robust within-subject correlations indicate that the relationship\nobserved at the group level may not generalise to individual changes over time.",
            "journal ref": "PLoS ONE 16(5): e0251787, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02959v2",
            "date": "2018-11-07 16:17:48+00:00",
            "title": "Compositional Language Understanding with Text-based Relational Reasoning",
            "authors": [
                "Koustuv Sinha",
                "Shagun Sodhani",
                "William L. Hamilton",
                "Joelle Pineau"
            ],
            "abstract": "Neural networks for natural language reasoning have largely focused on\nextractive, fact-based question-answering (QA) and common-sense inference.\nHowever, it is also crucial to understand the extent to which neural networks\ncan perform relational reasoning and combinatorial generalization from natural\nlanguage---abilities that are often obscured by annotation artifacts and the\ndominance of language modeling in standard QA benchmarks. In this work, we\npresent a novel benchmark dataset for language understanding that isolates\nperformance on relational reasoning. We also present a neural message-passing\nbaseline and show that this model, which incorporates a relational inductive\nbias, is superior at combinatorial generalization compared to a traditional\nrecurrent neural network approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03169v2",
            "date": "2018-11-07 22:14:32+00:00",
            "title": "Attention Fusion Networks: Combining Behavior and E-mail Content to Improve Customer Support",
            "authors": [
                "Stephane Fotso",
                "Philip Spanoudes",
                "Benjamin C. Ponedel",
                "Brian Reynoso",
                "Janet Ko"
            ],
            "abstract": "Customer support is a central objective at Square as it helps us build and\nmaintain great relationships with our sellers. In order to provide the best\nexperience, we strive to deliver the most accurate and quasi-instantaneous\nresponses to questions regarding our products.\n  In this work, we introduce the Attention Fusion Network model which combines\nsignals extracted from seller interactions on the Square product ecosystem,\nalong with submitted email questions, to predict the most relevant solution to\na seller's inquiry. We show that the innovative combination of two very\ndifferent data sources that are rarely used together, using state-of-the-art\ndeep learning systems outperforms, candidate models that are trained only on a\nsingle source.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03273v1",
            "date": "2018-11-08 05:10:34+00:00",
            "title": "Information Flow in Pregroup Models of Natural Language",
            "authors": [
                "Peter M. Hines"
            ],
            "abstract": "This paper is about pregroup models of natural languages, and how they relate\nto the explicitly categorical use of pregroups in Compositional Distributional\nSemantics and Natural Language Processing. These categorical interpretations\nmake certain assumptions about the nature of natural languages that, when\nstated formally, may be seen to impose strong restrictions on pregroup grammars\nfor natural languages.\n  We formalize this as a hypothesis about the form that pregroup models of\nnatural languages must take, and demonstrate by an artificial language example\nthat these restrictions are not imposed by the pregroup axioms themselves. We\ncompare and contrast the artificial language examples with natural languages\n(using Welsh, a language where the 'noun' type cannot be taken as primitive, as\nan illustrative example).\n  The hypothesis is simply that there must exist a causal connection, or\ninformation flow, between the words of a sentence in a language whose purpose\nis to communicate information. This is not necessarily the case with formal\nlanguages that are simply generated by a series of 'meaning-free' rules. This\nimposes restrictions on the types of pregroup grammars that we expect to find\nin natural languages; we formalize this in algebraic, categorical, and\ngraphical terms.\n  We take some preliminary steps in providing conditions that ensure pregroup\nmodels satisfy these conjectured properties, and discuss the more general forms\nthis hypothesis may take.",
            "journal ref": "EPTCS 283, 2018, pp. 13-27",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03275v1",
            "date": "2018-11-08 05:11:37+00:00",
            "title": "Quantum Semantic Correlations in Hate and Non-Hate Speeches",
            "authors": [
                "Francesco Galofaro",
                "Zeno Toffano",
                "Bich-Li\u00ean Doan"
            ],
            "abstract": "This paper aims to apply the notions of quantum geometry and correlation to\nthe typification of semantic relations between couples of keywords in different\ndocuments. In particular we analysed texts classified as hate / non hate\nspeeches, containing the keywords \"women\", \"white\", and \"black\". The paper\ncompares this approach to cosine similarity, a classical methodology, to cast\nlight on the notion of \"similar meaning\".",
            "journal ref": "EPTCS 283, 2018, pp. 62-74",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.03291v1",
            "date": "2018-11-08 06:51:46+00:00",
            "title": "Doc2Im: document to image conversion through self-attentive embedding",
            "authors": [
                "Mithun Das Gupta"
            ],
            "abstract": "Text classification is a fundamental task in NLP applications. Latest\nresearch in this field has largely been divided into two major sub-fields.\nLearning representations is one sub-field and learning deeper models, both\nsequential and convolutional, which again connects back to the representation\nis the other side. We posit the idea that the stronger the representation is,\nthe simpler classifier models are needed to achieve higher performance. In this\npaper we propose a completely novel direction to text classification research,\nwherein we convert text to a representation very similar to images, such that\nany deep network able to handle images is equally able to handle text. We take\na deeper look at the representation of documents as an image and subsequently\nutilize very simple convolution based models taken as is from computer vision\ndomain. This image can be cropped, re-scaled, re-sampled and augmented just\nlike any other image to work with most of the state-of-the-art large\nconvolution based models which have been designed to handle large image\ndatasets. We show impressive results with some of the latest benchmarks in the\nrelated fields. We perform transfer learning experiments, both from text to\ntext domain and also from image to text domain. We believe this is a paradigm\nshift from the way document understanding and text classification has been\ntraditionally done, and will drive numerous novel research ideas in the\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03325v3",
            "date": "2018-11-08 09:16:19+00:00",
            "title": "Discovering Power Laws in Entity Length",
            "authors": [
                "Xiaoshi Zhong",
                "Erik Cambria",
                "Jagath C. Rajapakse"
            ],
            "abstract": "This paper presents a discovery that the length of the entities in various\ndatasets follows a family of scale-free power law distributions. The concept of\nentity here broadly includes the named entity, entity mention, time expression,\naspect term, and domain-specific entity that are well investigated in natural\nlanguage processing and related areas. The entity length denotes the number of\nwords in an entity. The power law distributions in entity length possess the\nscale-free property and have well-defined means and finite variances. We\nexplain the phenomenon of power laws in entity length by the principle of least\neffort in communication and the preferential mechanism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03514v1",
            "date": "2018-11-08 16:01:35+00:00",
            "title": "Deep Neural Networks for Query Expansion using Word Embeddings",
            "authors": [
                "Ayyoob Imani",
                "Amir Vakili",
                "Ali Montazer",
                "Azadeh Shakery"
            ],
            "abstract": "Query expansion is a method for alleviating the vocabulary mismatch problem\npresent in information retrieval tasks. Previous works have shown that terms\nselected for query expansion by traditional methods such as pseudo-relevance\nfeedback are not always helpful to the retrieval process. In this paper, we\nshow that this is also true for more recently proposed embedding-based query\nexpansion methods. We then introduce an artificial neural network classifier to\npredict the usefulness of query expansion terms. This classifier uses term word\nembeddings as inputs. We perform experiments on four TREC newswire and web\ncollections show that using terms selected by the classifier for expansion\nsignificantly improves retrieval performance when compared to competitive\nbaselines. The results are also shown to be more robust than the baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.03866v1",
            "date": "2018-11-09 11:44:05+00:00",
            "title": "Learning Semantic Representations for Novel Words: Leveraging Both Form and Context",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word embeddings are a key component of high-performing natural language\nprocessing (NLP) systems, but it remains a challenge to learn good\nrepresentations for novel words on the fly, i.e., for words that did not occur\nin the training data. The general problem setting is that word embeddings are\ninduced on an unlabeled training corpus and then a model is trained that embeds\nnovel words into this induced embedding space. Currently, two approaches for\nlearning embeddings of novel words exist: (i) learning an embedding from the\nnovel word's surface-form (e.g., subword n-grams) and (ii) learning an\nembedding from the context in which it occurs. In this paper, we propose an\narchitecture that leverages both sources of information - surface-form and\ncontext - and show that it results in large increases in embedding quality. Our\narchitecture obtains state-of-the-art results on the Definitional Nonce and\nContextual Rare Words datasets. As input, we only require an embedding set and\nan unlabeled corpus for training our architecture to produce embeddings\nappropriate for the induced embedding space. Thus, our model can easily be\nintegrated into any existing NLP system and enhance its capability to handle\nnovel words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03925v1",
            "date": "2018-11-09 14:33:29+00:00",
            "title": "A Hierarchical Framework for Relation Extraction with Reinforcement Learning",
            "authors": [
                "Ryuichi Takanobu",
                "Tianyang Zhang",
                "Jiexi Liu",
                "Minlie Huang"
            ],
            "abstract": "Most existing methods determine relation types only after all the entities\nhave been recognized, thus the interaction between relation types and entity\nmentions is not fully modeled. This paper presents a novel paradigm to deal\nwith relation extraction by regarding the related entities as the arguments of\na relation. We apply a hierarchical reinforcement learning (HRL) framework in\nthis paradigm to enhance the interaction between entity mentions and relation\ntypes. The whole extraction process is decomposed into a hierarchy of two-level\nRL policies for relation detection and entity extraction respectively, so that\nit is more feasible and natural to deal with overlapping relations. Our model\nwas evaluated on public datasets collected via distant supervision, and results\nshow that it gains better performance than existing methods and is more\npowerful for extracting overlapping relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04201v1",
            "date": "2018-11-10 06:05:53+00:00",
            "title": "Adversarially-Trained Normalized Noisy-Feature Auto-Encoder for Text Generation",
            "authors": [
                "Xiang Zhang",
                "Yann LeCun"
            ],
            "abstract": "This article proposes Adversarially-Trained Normalized Noisy-Feature\nAuto-Encoder (ATNNFAE) for byte-level text generation. An ATNNFAE consists of\nan auto-encoder where the internal code is normalized on the unit sphere and\ncorrupted by additive noise. Simultaneously, a replica of the decoder (sharing\nthe same parameters as the AE decoder) is used as the generator and fed with\nrandom latent vectors. An adversarial discriminator is trained to distinguish\ntraining samples reconstructed from the AE from samples produced through the\nrandom-input generator, making the entire generator-discriminator path\ndifferentiable for discrete data like text. The combined effect of noise\ninjection in the code and shared weights between the decoder and the generator\ncan prevent the mode collapsing phenomenon commonly observed in GANs. Since\nperplexity cannot be applied to non-sequential text generation, we propose a\nnew evaluation method using the total variance distance between frequencies of\nhash-coded byte-level n-grams (NGTVD). NGTVD is a single benchmark that can\ncharacterize both the quality and the diversity of the generated texts.\nExperiments are offered in 6 large-scale datasets in Arabic, Chinese and\nEnglish, with comparisons against n-gram baselines and recurrent neural\nnetworks (RNNs). Ablation study on both the noise level and the discriminator\nis performed. We find that RNNs have trouble competing with the n-gram\nbaselines, and the ATNNFAE results are generally competitive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04352v4",
            "date": "2018-11-11 05:07:25+00:00",
            "title": "Open Vocabulary Learning for Neural Chinese Pinyin IME",
            "authors": [
                "Zhuosheng Zhang",
                "Yafang Huang",
                "Hai Zhao"
            ],
            "abstract": "Pinyin-to-character (P2C) conversion is the core component of pinyin-based\nChinese input method engine (IME). However, the conversion is seriously\ncompromised by the ambiguities of Chinese characters corresponding to pinyin as\nwell as the predefined fixed vocabularies. To alleviate such inconveniences, we\npropose a neural P2C conversion model augmented by an online updated vocabulary\nwith a sampling mechanism to support open vocabulary learning during IME\nworking. Our experiments show that the proposed method outperforms commercial\nIMEs and state-of-the-art traditional models on standard corpus and true\ninputting history dataset in terms of multiple metrics and thus the online\nupdated vocabulary indeed helps our IME effectively follows user inputting\nbehavior.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.04441v2",
            "date": "2018-11-11 18:07:44+00:00",
            "title": "End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion",
            "authors": [
                "Chao Shang",
                "Yun Tang",
                "Jing Huang",
                "Jinbo Bi",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Knowledge graph embedding has been an active research topic for knowledge\nbase completion, with progressive improvement from the initial TransE, TransH,\nDistMult et al to the current state-of-the-art ConvE. ConvE uses 2D convolution\nover embeddings and multiple layers of nonlinear features to model knowledge\ngraphs. The model can be efficiently trained and scalable to large knowledge\ngraphs. However, there is no structure enforcement in the embedding space of\nConvE. The recent graph convolutional network (GCN) provides another way of\nlearning graph node embedding by successfully utilizing graph connectivity\nstructure. In this work, we propose a novel end-to-end Structure-Aware\nConvolutional Network (SACN) that takes the benefit of GCN and ConvE together.\nSACN consists of an encoder of a weighted graph convolutional network (WGCN),\nand a decoder of a convolutional network called Conv-TransE. WGCN utilizes\nknowledge graph node structure, node attributes and edge relation types. It has\nlearnable weights that adapt the amount of information from neighbors used in\nlocal aggregation, leading to more accurate embeddings of graph nodes. Node\nattributes in the graph are represented as additional nodes in the WGCN. The\ndecoder Conv-TransE enables the state-of-the-art ConvE to be translational\nbetween entities and relations while keeps the same link prediction performance\nas ConvE. We demonstrate the effectiveness of the proposed SACN on standard\nFB15k-237 and WN18RR datasets, and it gives about 10% relative improvement over\nthe state-of-the-art ConvE in terms of HITS@1, HITS@3 and HITS@10.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.04588v1",
            "date": "2018-11-12 07:09:36+00:00",
            "title": "Differentiating Concepts and Instances for Knowledge Graph Embedding",
            "authors": [
                "Xin Lv",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu"
            ],
            "abstract": "Concepts, which represent a group of different instances sharing common\nproperties, are essential information in knowledge representation. Most\nconventional knowledge embedding methods encode both entities (concepts and\ninstances) and relations as vectors in a low dimensional semantic space\nequally, ignoring the difference between concepts and instances. In this paper,\nwe propose a novel knowledge graph embedding model named TransC by\ndifferentiating concepts and instances. Specifically, TransC encodes each\nconcept in knowledge graph as a sphere and each instance as a vector in the\nsame semantic space. We use the relative positions to model the relations\nbetween concepts and instances (i.e., instanceOf), and the relations between\nconcepts and sub-concepts (i.e., subClassOf). We evaluate our model on both\nlink prediction and triple classification tasks on the dataset based on YAGO.\nExperimental results show that TransC outperforms state-of-the-art methods, and\ncaptures the semantic transitivity for instanceOf and subClassOf relation. Our\ncodes and datasets can be obtained from https:// github.com/davidlvxin/TransC.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing. 2018: 1971-1979",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.04655v2",
            "date": "2018-11-12 10:55:11+00:00",
            "title": "Not Just Depressed: Bipolar Disorder Prediction on Reddit",
            "authors": [
                "Ivan Sekuli\u0107",
                "Matej Gjurkovi\u0107",
                "Jan \u0160najder"
            ],
            "abstract": "Bipolar disorder, an illness characterized by manic and depressive episodes,\naffects more than 60 million people worldwide. We present a preliminary study\non bipolar disorder prediction from user-generated text on Reddit, which relies\non users' self-reported labels. Our benchmark classifiers for bipolar disorder\nprediction outperform the baselines and reach accuracy and F1-scores of above\n86%. Feature analysis shows interesting differences in language use between\nusers with bipolar disorders and the control group, including differences in\nthe use of emotion-expressive words.",
            "journal ref": "WASSA@EMNLP 2018: 72-78",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04791v2",
            "date": "2018-11-09 11:04:40+00:00",
            "title": "Multilingual and Unsupervised Subword Modeling for Zero-Resource Languages",
            "authors": [
                "Enno Hermann",
                "Herman Kamper",
                "Sharon Goldwater"
            ],
            "abstract": "Subword modeling for zero-resource languages aims to learn low-level\nrepresentations of speech audio without using transcriptions or other resources\nfrom the target language (such as text corpora or pronunciation dictionaries).\nA good representation should capture phonetic content and abstract away from\nother types of variability, such as speaker differences and channel noise.\nPrevious work in this area has primarily focused unsupervised learning from\ntarget language data only, and has been evaluated only intrinsically. Here we\ndirectly compare multiple methods, including some that use only target language\nspeech data and some that use transcribed speech from other (non-target)\nlanguages, and we evaluate using two intrinsic measures as well as on a\ndownstream unsupervised word segmentation and clustering task. We find that\ncombining two existing target-language-only methods yields better features than\neither method alone. Nevertheless, even better results are obtained by\nextracting target language bottleneck features using a model trained on other\nlanguages. Cross-lingual training using just one other language is enough to\nprovide this benefit, but multilingual training helps even more. In addition to\nthese results, which hold across both intrinsic measures and the extrinsic\ntask, we discuss the qualitative differences between the different types of\nlearned features.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.05067v2",
            "date": "2018-11-13 02:04:42+00:00",
            "title": "Shall I Compare Thee to a Machine-Written Sonnet? An Approach to Algorithmic Sonnet Generation",
            "authors": [
                "John Benhardt",
                "Peter Hase",
                "Liuyi Zhu",
                "Cynthia Rudin"
            ],
            "abstract": "We provide an approach for generating beautiful poetry. Our sonnet-generation\nalgorithm includes several novel elements that improve over the state of the\nart, leading to metrical, rhyming poetry with many human-like qualities. These\nnovel elements include in-line punctuation, part of speech restrictions, and\nmore appropriate training corpora. Our work is the winner of the 2018 PoetiX\nLiterary Turing Test Award for computer-generated poetry.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.05106v1",
            "date": "2018-11-13 04:57:48+00:00",
            "title": "Interpreting Models by Allowing to Ask",
            "authors": [
                "Sungmin Kang",
                "David Keetae Park",
                "Jaehyuk Chang",
                "Jaegul Choo"
            ],
            "abstract": "Questions convey information about the questioner, namely what one does not\nknow. In this paper, we propose a novel approach to allow a learning agent to\nask what it considers as tricky to predict, in the course of producing a final\noutput. By analyzing when and what it asks, we can make our model more\ntransparent and interpretable. We first develop this idea to propose a general\nframework of deep neural networks that can ask questions, which we call asking\nnetworks. A specific architecture and training process for an asking network is\nproposed for the task of colorization, which is an exemplar one-to-many task\nand thus a task where asking questions is helpful in performing the task\naccurately. Our results show that the model learns to generate meaningful\nquestions, asks difficult questions first, and utilizes the provided hint more\nefficiently than baseline models. We conclude that the proposed asking\nframework makes the learning agent reveal its weaknesses, which poses a\npromising new direction in developing interpretable and interactive models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.05121v1",
            "date": "2018-11-13 06:22:02+00:00",
            "title": "Modeling Local Dependence in Natural Language with Multi-channel Recurrent Neural Networks",
            "authors": [
                "Chang Xu",
                "Weiran Huang",
                "Hongwei Wang",
                "Gang Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) have been widely used in processing natural\nlanguage tasks and achieve huge success. Traditional RNNs usually treat each\ntoken in a sentence uniformly and equally. However, this may miss the rich\nsemantic structure information of a sentence, which is useful for understanding\nnatural languages. Since semantic structures such as word dependence patterns\nare not parameterized, it is a challenge to capture and leverage structure\ninformation. In this paper, we propose an improved variant of RNN,\nMulti-Channel RNN (MC-RNN), to dynamically capture and leverage local semantic\nstructure information. Concretely, MC-RNN contains multiple channels, each of\nwhich represents a local dependence pattern at a time. An attention mechanism\nis introduced to combine these patterns at each step, according to the semantic\ninformation. Then we parameterize structure information by adaptively selecting\nthe most appropriate connection structures among channels. In this way, diverse\nlocal structures and dependence patterns in sentences can be well captured by\nMC-RNN. To verify the effectiveness of MC-RNN, we conduct extensive experiments\non typical natural language processing tasks, including neural machine\ntranslation, abstractive summarization, and language modeling. Experimental\nresults on these tasks all show significant improvements of MC-RNN over current\ntop systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05242v1",
            "date": "2018-11-13 12:10:29+00:00",
            "title": "A Multi-layer LSTM-based Approach for Robot Command Interaction Modeling",
            "authors": [
                "Martino Mensio",
                "Emanuele Bastianelli",
                "Ilaria Tiddi",
                "Giuseppe Rizzo"
            ],
            "abstract": "As the first robotic platforms slowly approach our everyday life, we can\nimagine a near future where service robots will be easily accessible by\nnon-expert users through vocal interfaces. The capability of managing natural\nlanguage would indeed speed up the process of integrating such platform in the\nordinary life. Semantic parsing is a fundamental task of the Natural Language\nUnderstanding process, as it allows extracting the meaning of a user utterance\nto be used by a machine. In this paper, we present a preliminary study to\nsemantically parse user vocal commands for a House Service robot, using a\nmulti-layer Long-Short Term Memory neural network with attention mechanism. The\nsystem is trained on the Human Robot Interaction Corpus, and it is\npreliminarily compared with previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05303v1",
            "date": "2018-11-13 14:06:58+00:00",
            "title": "Translating Natural Language to SQL using Pointer-Generator Networks and How Decoding Order Matters",
            "authors": [
                "Denis Lukovnikov",
                "Nilesh Chakraborty",
                "Jens Lehmann",
                "Asja Fischer"
            ],
            "abstract": "Translating natural language to SQL queries for table-based question\nanswering is a challenging problem and has received significant attention from\nthe research community. In this work, we extend a pointer-generator and\ninvestigate the order-matters problem in semantic parsing for SQL. Even though\nour model is a straightforward extension of a general-purpose\npointer-generator, it outperforms early works for WikiSQL and remains\ncompetitive to concurrently introduced, more complex models. Moreover, we\nprovide a deeper investigation of the potential order-matters problem that\ncould arise due to having multiple correct decoding paths, and investigate the\nuse of REINFORCE as well as a dynamic oracle in this context.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.05402v1",
            "date": "2018-11-13 16:55:52+00:00",
            "title": "Embedding Electronic Health Records for Clinical Information Retrieval",
            "authors": [
                "Xing Wei",
                "Carsten Eickhoff"
            ],
            "abstract": "Neural network representation learning frameworks have recently shown to be\nhighly effective at a wide range of tasks ranging from radiography\ninterpretation via data-driven diagnostics to clinical decision support. This\noften superior performance comes at the price of dramatically increased\ntraining data requirements that cannot be satisfied in every given institution\nor scenario. As a means of countering such data sparsity effects, distant\nsupervision alleviates the need for scarce in-domain data by relying on a\nrelated, resource-rich, task for training.\n  This study presents an end-to-end neural clinical decision support system\nthat recommends relevant literature for individual patients (few available\nresources) via distant supervision on the well-known MIMIC-III collection\n(abundant resource). Our experiments show significant improvements in retrieval\neffectiveness over traditional statistical as well as purely locally supervised\nretrieval models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.05549v1",
            "date": "2018-11-13 22:24:29+00:00",
            "title": "An Analysis of the Semantic Annotation Task on the Linked Data Cloud",
            "authors": [
                "Gagnon Michel",
                "Zouaq Amal",
                "Aranha Francisco",
                "Ensan Faezeh",
                "Jean-Louis Ludovic"
            ],
            "abstract": "Semantic annotation, the process of identifying key-phrases in texts and\nlinking them to concepts in a knowledge base, is an important basis for\nsemantic information retrieval and the Semantic Web uptake. Despite the\nemergence of semantic annotation systems, very few comparative studies have\nbeen published on their performance. In this paper, we provide an evaluation of\nthe performance of existing systems over three tasks: full semantic annotation,\nnamed entity recognition, and keyword detection. More specifically, the\nspotting capability (recognition of relevant surface forms in text) is\nevaluated for all three tasks, whereas the disambiguation (correctly\nassociating an entity from Wikipedia or DBpedia to the spotted surface forms)\nis evaluated only for the first two tasks. Our evaluation is twofold: First, we\ncompute standard precision and recall on the output of semantic annotators on\ndiverse datasets, each best suited for one of the identified tasks. Second, we\nbuild a statistical model using logistic regression to identify significant\nperformance differences. Our results show that systems that provide full\nannotation perform better than named entities annotators and keyword\nextractors, for all three tasks. However, there is still much room for\nimprovement for the identification of the most relevant entities described in a\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05563v1",
            "date": "2018-11-13 23:11:26+00:00",
            "title": "Text Assisted Insight Ranking Using Context-Aware Memory Network",
            "authors": [
                "Qi Zeng",
                "Liangchen Luo",
                "Wenhao Huang",
                "Yang Tang"
            ],
            "abstract": "Extracting valuable facts or informative summaries from multi-dimensional\ntables, i.e. insight mining, is an important task in data analysis and business\nintelligence. However, ranking the importance of insights remains a challenging\nand unexplored task. The main challenge is that explicitly scoring an insight\nor giving it a rank requires a thorough understanding of the tables and costs a\nlot of manual efforts, which leads to the lack of available training data for\nthe insight ranking problem. In this paper, we propose an insight ranking model\nthat consists of two parts: A neural ranking model explores the data\ncharacteristics, such as the header semantics and the data statistical\nfeatures, and a memory network model introduces table structure and context\ninformation into the ranking process. We also build a dataset with text\nassistance. Experimental results show that our approach largely improves the\nranking precision as reported in multi evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05616v1",
            "date": "2018-11-14 03:02:12+00:00",
            "title": "Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector",
            "authors": [
                "Shanchan Wu",
                "Kai Fan",
                "Qiong Zhang"
            ],
            "abstract": "Distant supervised relation extraction has been successfully applied to large\ncorpus with thousands of relations. However, the inevitable wrong labeling\nproblem by distant supervision will hurt the performance of relation\nextraction. In this paper, we propose a method with neural noise converter to\nalleviate the impact of noisy data, and a conditional optimal selector to make\nproper prediction. Our noise converter learns the structured transition matrix\non logit level and captures the property of distant supervised relation\nextraction dataset. The conditional optimal selector on the other hand helps to\nmake proper prediction decision of an entity pair even if the group of\nsentences is overwhelmed by no-relation sentences. We conduct experiments on a\nwidely used dataset and the results show significant improvement over\ncompetitive baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05757v1",
            "date": "2018-10-16 21:56:25+00:00",
            "title": "Automatic event detection in microblogs using incremental machine learning",
            "authors": [
                "Tharindu Rukshan Bandaragoda",
                "Daswin De Silva",
                "Damminda Alahakoon"
            ],
            "abstract": "The global popularity of microblogs has led to an increasing accumulation of\nlarge volumes of text data on microblogging platforms such as Twitter. These\ncorpora are untapped resources to understand social expressions on diverse\nsubjects. Microblog analysis aims to unlock the value of such expressions by\ndiscovering insights and events of significance hidden among swathes of text.\nBesides velocity; diversity of content, brevity, absence of structure and\ntime-sensitivity are key challenges in microblog analysis. In this paper, we\npropose an unsupervised incremental machine learning and event detection\ntechnique to address these challenges. The proposed technique separates a\nmicroblog discussion into topics to address the key problem of diversity. It\nmaintains a record of the evolution of each topic over time. Brevity,\ntime-sensitivity and unstructured nature are addressed by these individual\ntopic pathways which contribute to generate a temporal, topic-driven structure\nof a microblog discussion. The proposed event detection method continuously\nmonitors these topic pathways using multiple domain-independent event\nindicators for events of significance. The autonomous nature of topic\nseparation, topic pathway generation, new topic identification and event\ndetection, appropriates the proposed technique for extensive applications in\nmicroblog analysis. We demonstrate these capabilities on tweets containing\n#microsoft and tweets containing #obama.",
            "journal ref": "Journal of the Association for Information Science and Technology,\n  2017, 68(10), 2394-2411",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.05770v1",
            "date": "2018-11-08 05:11:59+00:00",
            "title": "Internal Wiring of Cartesian Verbs and Prepositions",
            "authors": [
                "Bob Coecke",
                "Martha Lewis",
                "Dan Marsden"
            ],
            "abstract": "Categorical compositional distributional semantics (CCDS) allows one to\ncompute the meaning of phrases and sentences from the meaning of their\nconstituent words. A type-structure carried over from the traditional\ncategorial model of grammar a la Lambek becomes a 'wire-structure' that\nmediates the interaction of word meanings. However, CCDS has a much richer\nlogical structure than plain categorical semantics in that certain words can\nalso be given an 'internal wiring' that either provides their entire meaning or\nreduces the size their meaning space. Previous examples of internal wiring\ninclude relative pronouns and intersective adjectives. Here we establish the\nsame for a large class of well-behaved transitive verbs to which we refer as\nCartesian verbs, and reduce the meaning space from a ternary tensor to a unary\none. Some experimental evidence is also provided.",
            "journal ref": "EPTCS 283, 2018, pp. 75-88",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05825v1",
            "date": "2018-10-09 11:28:19+00:00",
            "title": "Fake Comment Detection Based on Sentiment Analysis",
            "authors": [
                "Su Chang",
                "Xu Zhenzhong",
                "Gao Xuan"
            ],
            "abstract": "With the development of the E-commerce and reviews website, the comment\ninformation is influencing people's life. More and more users share their\nconsumption experience and evaluate the quality of commodity by comment. When\npeople make a decision, they will refer these comments. The dependency of the\ncomments make the fake comment appear. The fake comment is that for profit and\nother bad motivation, business fabricate untrue consumption experience and they\npreach or slander some products. The fake comment is easy to mislead users'\nopinion and decision. The accuracy of humans identifying fake comment is low.\nIt's meaningful to detect fake comment using natural language processing\ntechnology for people getting true comment information. This paper uses the\nsentimental analysis to detect fake comment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05826v1",
            "date": "2018-11-04 22:56:50+00:00",
            "title": "Char2char Generation with Reranking for the E2E NLG Challenge",
            "authors": [
                "Shubham Agarwal",
                "Marc Dymetman",
                "Eric Gaussier"
            ],
            "abstract": "This paper describes our submission to the E2E NLG Challenge. Recently,\nneural seq2seq approaches have become mainstream in NLG, often resorting to\npre- (respectively post-) processing delexicalization (relexicalization) steps\nat the word-level to handle rare words. By contrast, we train a simple\ncharacter level seq2seq model, which requires no pre/post-processing\n(delexicalization, tokenization or even lowercasing), with surprisingly good\nresults. For further improvement, we explore two re-ranking approaches for\nscoring candidates. We also introduce a synthetic dataset creation procedure,\nwhich opens up a new way of creating artificial datasets for Natural Language\nGeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06179v1",
            "date": "2018-11-15 04:58:21+00:00",
            "title": "Implementing a Portable Clinical NLP System with a Common Data Model - a Lisp Perspective",
            "authors": [
                "Yuan Luo",
                "Peter Szolovits"
            ],
            "abstract": "This paper presents a Lisp architecture for a portable NLP system, termed\nLAPNLP, for processing clinical notes. LAPNLP integrates multiple standard,\ncustomized and in-house developed NLP tools. Our system facilitates portability\nacross different institutions and data systems by incorporating an enriched\nCommon Data Model (CDM) to standardize necessary data elements. It utilizes\nUMLS to perform domain adaptation when integrating generic domain NLP tools. It\nalso features stand-off annotations that are specified by positional reference\nto the original document. We built an interval tree based search engine to\nefficiently query and retrieve the stand-off annotations by specifying\npositional requirements. We also developed a utility to convert an inline\nannotation format to stand-off annotations to enable the reuse of clinical text\ndatasets with inline annotations. We experimented with our system on several\nNLP facilitated tasks including computational phenotyping for lymphoma patients\nand semantic relation extraction for clinical notes. These experiments\nshowcased the broader applicability and utility of LAPNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06183v1",
            "date": "2018-11-15 05:12:33+00:00",
            "title": "Characterizing Design Patterns of EHR-Driven Phenotype Extraction Algorithms",
            "authors": [
                "Yizhen Zhong",
                "Luke Rasmussen",
                "Yu Deng",
                "Jennifer Pacheco",
                "Maureen Smith",
                "Justin Starren",
                "Wei-Qi Wei",
                "Peter Speltz",
                "Joshua Denny",
                "Nephi Walton",
                "George Hripcsak",
                "Christopher G Chute",
                "Yuan Luo"
            ],
            "abstract": "The automatic development of phenotype algorithms from Electronic Health\nRecord data with machine learning (ML) techniques is of great interest given\nthe current practice is very time-consuming and resource intensive. The\nextraction of design patterns from phenotype algorithms is essential to\nunderstand their rationale and standard, with great potential to automate the\ndevelopment process. In this pilot study, we perform network visualization on\nthe design patterns and their associations with phenotypes and sites. We\nclassify design patterns using the fragments from previously annotated\nphenotype algorithms as the ground truth. The classification performance is\nused as a proxy for coherence at the attribution level. The bag-of-words\nrepresentation with knowledge-based features generated a good performance in\nthe classification task (0.79 macro-f1 scores). Good classification accuracy\nwith simple features demonstrated the attribution coherence and the feasibility\nof automatic identification of design patterns. Our results point to both the\nfeasibility and challenges of automatic identification of phenotyping design\npatterns, which would power the automatic development of phenotype algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06203v1",
            "date": "2018-11-15 06:40:39+00:00",
            "title": "Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference",
            "authors": [
                "Masashi Yoshikawa",
                "Koji Mineshima",
                "Hiroshi Noji",
                "Daisuke Bekki"
            ],
            "abstract": "In logic-based approaches to reasoning tasks such as Recognizing Textual\nEntailment (RTE), it is important for a system to have a large amount of\nknowledge data. However, there is a tradeoff between adding more knowledge data\nfor improved RTE performance and maintaining an efficient RTE system, as such a\nbig database is problematic in terms of the memory usage and computational\ncomplexity. In this work, we show the processing time of a state-of-the-art\nlogic-based RTE system can be significantly reduced by replacing its\nsearch-based axiom injection (abduction) mechanism by that based on Knowledge\nBase Completion (KBC). We integrate this mechanism in a Coq plugin that\nprovides a proof automation tactic for natural language inference.\nAdditionally, we show empirically that adding new knowledge data contributes to\nbetter RTE performance while not harming the processing speed in this\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06315v2",
            "date": "2018-11-15 12:31:12+00:00",
            "title": "Effect of data reduction on sequence-to-sequence neural TTS",
            "authors": [
                "Javier Latorre",
                "Jakub Lachowicz",
                "Jaime Lorenzo-Trueba",
                "Thomas Merritt",
                "Thomas Drugman",
                "Srikanth Ronanki",
                "Klimkov Viacheslav"
            ],
            "abstract": "Recent speech synthesis systems based on sampling from autoregressive neural\nnetworks models can generate speech almost undistinguishable from human\nrecordings. However, these models require large amounts of data. This paper\nshows that the lack of data from one speaker can be compensated with data from\nother speakers. The naturalness of Tacotron2-like models trained on a blend of\n5k utterances from 7 speakers is better than that of speaker dependent models\ntrained on 15k utterances, but in terms of stability multi-speaker models are\nalways more stable. We also demonstrate that models mixing only 1250 utterances\nfrom a target speaker with 5k utterances from another 6 speakers can produce\nsignificantly better quality than state-of-the-art DNN-guided unit selection\nsystems trained on more than 10 times the data from the target speaker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06567v1",
            "date": "2018-11-15 19:29:26+00:00",
            "title": "Automatic Text Document Summarization using Semantic-based Analysis",
            "authors": [
                "Chandra Shekhar Yadav"
            ],
            "abstract": "Since the advent of the web, the amount of data on wen has been increased\nseveral million folds. In recent years web data generated is more than data\nstored for years. One important data format is text. To answer user queries\nover the internet, and to overcome the problem of information overload one\npossible solution is text document summarization. This not only reduces query\naccess time, but also optimize the document results according to specific users\nrequirements. Summarization of text document can be categorized as abstractive\nand extractive. Most of the work has been done in the direction of Extractive\nsummarization. Extractive summarized result is a subset of original documents\nwith the objective of more content coverage and lea redundancy. Our work is\nbased on Extractive approaches. In the first approach, we are using some\nstatistical features and semantic-based features. To include sentiment as a\nfeature is an idea cached from a view that emotion plays an important role. It\neffectively conveys a message. So, it may play a vital role in text document\nsummarization.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.06596v1",
            "date": "2018-11-15 21:29:26+00:00",
            "title": "On Generality and Knowledge Transferability in Cross-Domain Duplicate Question Detection for Heterogeneous Community Question Answering",
            "authors": [
                "Mohomed Shazan Mohomed Jabbar",
                "Luke Kumar",
                "Hamman Samuel",
                "Mi-Young Kim",
                "Sankalp Prabhakar",
                "Randy Goebel",
                "Osmar Za\u00efane"
            ],
            "abstract": "Duplicate question detection is an ongoing challenge in community question\nanswering because semantically equivalent questions can have significantly\ndifferent words and structures. In addition, the identification of duplicate\nquestions can reduce the resources required for retrieval, when the same\nquestions are not repeated. This study compares the performance of deep neural\nnetworks and gradient tree boosting, and explores the possibility of domain\nadaptation with transfer learning to improve the under-performing target\ndomains for the text-pair duplicates classification task, using three\nheterogeneous datasets: general-purpose Quora, technical Ask Ubuntu, and\nacademic English Stack Exchange. Ultimately, our study exposes the alternative\nhypothesis that the meaning of a \"duplicate\" is not inherently general-purpose,\nbut rather is dependent on the domain of learning, hence reducing the chance of\ntransfer learning through adapting to the domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06630v1",
            "date": "2018-11-15 23:47:39+00:00",
            "title": "Nudging Neural Conversational Model with Domain Knowledge",
            "authors": [
                "Sungjin Lee"
            ],
            "abstract": "Neural conversation models are attractive because one can train a model\ndirectly on dialog examples with minimal labeling. With a small amount of data,\nhowever, they often fail to generalize over test data since they tend to\ncapture spurious features instead of semantically meaningful domain knowledge.\nTo address this issue, we propose a novel approach that allows any human\nteachers to transfer their domain knowledge to the conversation model in the\nform of natural language rules. We tested our method with three different\ndialog datasets. The improved performance across all domains demonstrates the\nefficacy of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07033v1",
            "date": "2018-11-16 21:12:24+00:00",
            "title": "Analyzing Compositionality-Sensitivity of NLI Models",
            "authors": [
                "Yixin Nie",
                "Yicheng Wang",
                "Mohit Bansal"
            ],
            "abstract": "Success in natural language inference (NLI) should require a model to\nunderstand both lexical and compositional semantics. However, through\nadversarial evaluation, we find that several state-of-the-art models with\ndiverse architectures are over-relying on the former and fail to use the\nlatter. Further, this compositionality unawareness is not reflected via\nstandard evaluation on current datasets. We show that removing RNNs in existing\nmodels or shuffling input words during training does not induce large\nperformance loss despite the explicit removal of compositional information.\nTherefore, we propose a compositionality-sensitivity testing setup that\nanalyzes models on natural examples from existing datasets that cannot be\nsolved via lexical features alone (i.e., on which a bag-of-words model gives a\nhigh probability to one wrong label), hence revealing the models' actual\ncompositionality awareness. We show that this setup not only highlights the\nlimited compositional ability of current NLI models, but also differentiates\nmodel performance based on design, e.g., separating shallow bag-of-words models\nfrom deeper, linguistically-grounded tree-based models. Our evaluation setup is\nan important analysis tool: complementing currently existing adversarial and\nlinguistically driven diagnostic evaluations, and exposing opportunities for\nfuture work on evaluating models' compositional understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07039v1",
            "date": "2018-11-16 21:37:59+00:00",
            "title": "Combining Fact Extraction and Verification with Neural Semantic Matching Networks",
            "authors": [
                "Yixin Nie",
                "Haonan Chen",
                "Mohit Bansal"
            ],
            "abstract": "The increasing concern with misinformation has stimulated research efforts on\nautomatic fact checking. The recently-released FEVER dataset introduced a\nbenchmark fact-verification task in which a system is asked to verify a claim\nusing evidential sentences from Wikipedia documents. In this paper, we present\na connected system consisting of three homogeneous neural semantic matching\nmodels that conduct document retrieval, sentence selection, and claim\nverification jointly for fact extraction and verification. For evidence\nretrieval (document retrieval and sentence selection), unlike traditional\nvector space IR models in which queries and sources are matched in some\npre-designed term vector space, we develop neural models to perform deep\nsemantic matching from raw textual input, assuming no intermediate term\nrepresentation and no access to structured external knowledge bases. We also\nshow that Pageview frequency can also help improve the performance of evidence\nretrieval results, that later can be matched by using our neural semantic\nmatching network. For claim verification, unlike previous approaches that\nsimply feed upstream retrieved evidence and the claim to a natural language\ninference (NLI) model, we further enhance the NLI model by providing it with\ninternal semantic relatedness scores (hence integrating it with the evidence\nretrieval modules) and ontological WordNet features. Experiments on the FEVER\ndataset indicate that (1) our neural semantic matching method outperforms\npopular TF-IDF and encoder models, by significant margins on all evidence\nretrieval metrics, (2) the additional relatedness score and WordNet features\nimprove the NLI model via better semantic awareness, and (3) by formalizing all\nthree subtasks as a similar semantic matching problem and improving on all\nthree stages, the complete model is able to achieve the state-of-the-art\nresults on the FEVER test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07223v1",
            "date": "2018-11-17 21:26:20+00:00",
            "title": "Deep Dive into Anonymity: A Large Scale Analysis of Quora Questions",
            "authors": [
                "Binny Mathew",
                "Ritam Dutt",
                "Suman Kalyan Maity",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "Anonymity forms an integral and important part of our digital life. It\nenables us to express our true selves without the fear of judgment. In this\npaper, we investigate the different aspects of anonymity in the social Q&A site\nQuora. The choice of Quora is motivated by the fact that this is one of the\nrare social Q&A sites that allow users to explicitly post anonymous questions\nand such activity in this forum has become normative rather than a taboo.\nThrough an analysis of 5.1 million questions, we observe that at a global scale\nalmost no difference manifests between the linguistic structure of the\nanonymous and the non-anonymous questions. We find that topical mixing at the\nglobal scale to be the primary reason for the absence. However, the differences\nstart to feature once we \"deep dive\" and (topically) cluster the questions and\ncompare the clusters that have high volumes of anonymous questions with those\nthat have low volumes of anonymous questions. In particular, we observe that\nthe choice to post the question as anonymous is dependent on the user's\nperception of anonymity and they often choose to speak about depression,\nanxiety, social ties and personal issues under the guise of anonymity. We\nfurther perform personality trait analysis and observe that the anonymous group\nof users has positive correlation with extraversion, agreeableness, and\nnegative correlation with openness. Subsequently, to gain further insights, we\nbuild an anonymity grid to identify the differences in the perception on\nanonymity of the user posting the question and the community of users answering\nit. We also look into the first response time of the questions and observe that\nit is lowest for topics which talk about personal and sensitive issues, which\nhints toward a higher degree of community support and user engagement.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1811.07236v1",
            "date": "2018-11-17 22:34:20+00:00",
            "title": "Robust cross-domain disfluency detection with pattern match networks",
            "authors": [
                "Vicky Zayats",
                "Mari Ostendorf"
            ],
            "abstract": "In this paper we introduce a novel pattern match neural network architecture\nthat uses neighbor similarity scores as features, eliminating the need for\nfeature engineering in a disfluency detection task. We evaluate the approach in\ndisfluency detection for four different speech genres, showing that the\napproach is as effective as hand-engineered pattern match features when used on\nin-domain data and achieves superior performance in cross-domain scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07430v2",
            "date": "2018-11-19 00:18:48+00:00",
            "title": "Understanding and Measuring Psychological Stress using Social Media",
            "authors": [
                "Sharath Chandra Guntuku",
                "Anneke Buffone",
                "Kokil Jaidka",
                "Johannes Eichstaedt",
                "Lyle Ungar"
            ],
            "abstract": "A body of literature has demonstrated that users' mental health conditions,\nsuch as depression and anxiety, can be predicted from their social media\nlanguage. There is still a gap in the scientific understanding of how\npsychological stress is expressed on social media. Stress is one of the primary\nunderlying causes and correlates of chronic physical illnesses and mental\nhealth conditions. In this paper, we explore the language of psychological\nstress with a dataset of 601 social media users, who answered the Perceived\nStress Scale questionnaire and also consented to share their Facebook and\nTwitter data. Firstly, we find that stressed users post about exhaustion,\nlosing control, increased self-focus and physical pain as compared to posts\nabout breakfast, family-time, and travel by users who are not stressed.\nSecondly, we find that Facebook language is more predictive of stress than\nTwitter language. Thirdly, we demonstrate how the language based models thus\ndeveloped can be adapted and be scaled to measure county-level trends. Since\ncounty-level language is easily available on Twitter using the Streaming API,\nwe explore multiple domain adaptation algorithms to adapt user-level Facebook\nmodels to Twitter language. We find that domain-adapted and scaled social\nmedia-based measurements of stress outperform sociodemographic variables (age,\ngender, race, education, and income), against ground-truth survey-based stress\nmeasurements, both at the user- and the county-level in the U.S. Twitter\nlanguage that scores higher in stress is also predictive of poorer health, less\naccess to facilities and lower socioeconomic status in counties. We conclude\nwith a discussion of the implications of using social media as a new tool for\nmonitoring stress levels of both individuals and counties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07655v1",
            "date": "2018-11-19 12:51:07+00:00",
            "title": "An Influence-based Clustering Model on Twitter",
            "authors": [
                "Abbas Ehsanfar",
                "Mo Mansouri"
            ],
            "abstract": "This paper introduces a temporal framework for detecting and clustering\nemergent and viral topics on social networks. Endogenous and exogenous\ninfluence on developing viral content is explored using a clustering method\nbased on the a user's behavior on social network and a dataset from Twitter\nAPI. Results are discussed by introducing metrics such as popularity,\nburstiness, and relevance score. The results show clear distinction in\ncharacteristics of developed content by the two classes of users.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1811.08129v1",
            "date": "2018-11-20 08:59:53+00:00",
            "title": "Alignment Analysis of Sequential Segmentation of Lexicons to Improve Automatic Cognate Detection",
            "authors": [
                "Pranav A"
            ],
            "abstract": "Ranking functions in information retrieval are often used in search engines\nto recommend the relevant answers to the query. This paper makes use of this\nnotion of information retrieval and applies onto the problem domain of cognate\ndetection. The main contributions of this paper are: (1) positional\nsegmentation, which incorporates the sequential notion; (2) graphical error\nmodelling, which deduces the transformations. The current research work focuses\non classification problem; which is distinguishing whether a pair of words are\ncognates. This paper focuses on a harder problem, whether we could predict a\npossible cognate from the given input. Our study shows that when language\nmodelling smoothing methods are applied as the retrieval functions and used in\nconjunction with positional segmentation and error modelling gives better\nresults than competing baselines, in both classification and prediction of\ncognates.\n  Source code is at: https://github.com/pranav-ust/cognates",
            "journal ref": "Proceedings of ACL 2018, Student Research Workshop. 2018",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.08541v1",
            "date": "2018-11-21 01:48:22+00:00",
            "title": "Neural Machine Translation with Adequacy-Oriented Learning",
            "authors": [
                "Xiang Kong",
                "Zhaopeng Tu",
                "Shuming Shi",
                "Eduard Hovy",
                "Tong Zhang"
            ],
            "abstract": "Although Neural Machine Translation (NMT) models have advanced\nstate-of-the-art performance in machine translation, they face problems like\nthe inadequate translation. We attribute this to that the standard Maximum\nLikelihood Estimation (MLE) cannot judge the real translation quality due to\nits several limitations. In this work, we propose an adequacy-oriented learning\nmechanism for NMT by casting translation as a stochastic policy in\nReinforcement Learning (RL), where the reward is estimated by explicitly\nmeasuring translation adequacy. Benefiting from the sequence-level training of\nRL strategy and a more accurate reward designed specifically for translation,\nour model outperforms multiple strong baselines, including (1) standard and\ncoverage-augmented attention models with MLE-based training, and (2) advanced\nreinforcement and adversarial training strategies with rewards based on both\nword-level BLEU and character-level chrF3. Quantitative and qualitative\nanalyses on different language pairs and NMT architectures demonstrate the\neffectiveness and universality of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08615v1",
            "date": "2018-11-21 07:24:31+00:00",
            "title": "Unsupervised Multimodal Representation Learning across Medical Images and Reports",
            "authors": [
                "Tzu-Ming Harry Hsu",
                "Wei-Hung Weng",
                "Willie Boag",
                "Matthew McDermott",
                "Peter Szolovits"
            ],
            "abstract": "Joint embeddings between medical imaging modalities and associated radiology\nreports have the potential to offer significant benefits to the clinical\ncommunity, ranging from cross-domain retrieval to conditional generation of\nreports to the broader goals of multimodal representation learning. In this\nwork, we establish baseline joint embedding results measured via both local and\nglobal retrieval methods on the soon to be released MIMIC-CXR dataset\nconsisting of both chest X-ray images and the associated radiology reports. We\nexamine both supervised and unsupervised methods on this task and show that for\ndocument retrieval tasks with the learned representations, only a limited\namount of supervision is needed to yield results comparable to those of\nfully-supervised methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.09362v2",
            "date": "2018-11-23 05:13:38+00:00",
            "title": "Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors",
            "authors": [
                "Yansen Wang",
                "Ying Shen",
                "Zhun Liu",
                "Paul Pu Liang",
                "Amir Zadeh",
                "Louis-Philippe Morency"
            ],
            "abstract": "Humans convey their intentions through the usage of both verbal and nonverbal\nbehaviors during face-to-face communication. Speaker intentions often vary\ndynamically depending on different nonverbal contexts, such as vocal patterns\nand facial expressions. As a result, when modeling human language, it is\nessential to not only consider the literal meaning of the words but also the\nnonverbal contexts in which these words appear. To better model human language,\nwe first model expressive nonverbal representations by analyzing the\nfine-grained visual and acoustic patterns that occur during word segments. In\naddition, we seek to capture the dynamic nature of nonverbal intents by\nshifting word representations based on the accompanying nonverbal behaviors. To\nthis end, we propose the Recurrent Attended Variation Embedding Network (RAVEN)\nthat models the fine-grained structure of nonverbal subword sequences and\ndynamically shifts word representations based on nonverbal cues. Our proposed\nmodel achieves competitive performance on two publicly available datasets for\nmultimodal sentiment analysis and emotion recognition. We also visualize the\nshifted word representations in different nonverbal contexts and summarize\ncommon patterns regarding multimodal variations of word representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09368v1",
            "date": "2018-11-23 06:28:41+00:00",
            "title": "Fine Grained Classification of Personal Data Entities",
            "authors": [
                "Riddhiman Dasgupta",
                "Balaji Ganesan",
                "Aswin Kannan",
                "Berthold Reinwald",
                "Arun Kumar"
            ],
            "abstract": "Entity Type Classification can be defined as the task of assigning category\nlabels to entity mentions in documents. While neural networks have recently\nimproved the classification of general entity mentions, pattern matching and\nother systems continue to be used for classifying personal data entities (e.g.\nclassifying an organization as a media company or a government institution for\nGDPR, and HIPAA compliance). We propose a neural model to expand the class of\npersonal data entities that can be classified at a fine grained level, using\nthe output of existing pattern matching systems as additional contextual\nfeatures. We introduce new resources, a personal data entities hierarchy with\n134 types, and two datasets from the Wikipedia pages of elected representatives\nand Enron emails. We hope these resource will aid research in the area of\npersonal data discovery, and to that effect, we provide baseline results on\nthese datasets, and compare our method with state of the art models on\nOntoNotes dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09386v1",
            "date": "2018-11-23 08:30:19+00:00",
            "title": "Explicit Interaction Model towards Text Classification",
            "authors": [
                "Cunxiao Du",
                "Zhaozheng Chin",
                "Fuli Feng",
                "Lei Zhu",
                "Tian Gan",
                "Liqiang Nie"
            ],
            "abstract": "Text classification is one of the fundamental tasks in natural language\nprocessing. Recently, deep neural networks have achieved promising performance\nin the text classification task compared to shallow models. Despite of the\nsignificance of deep models, they ignore the fine-grained (matching signals\nbetween words and classes) classification clues since their classifications\nmainly rely on the text-level representations. To address this problem, we\nintroduce the interaction mechanism to incorporate word-level matching signals\ninto the text classification task. In particular, we design a novel framework,\nEXplicit interAction Model (dubbed as EXAM), equipped with the interaction\nmechanism. We justified the proposed approach on several benchmark datasets\nincluding both multi-label and multi-class text classification tasks. Extensive\nexperimental results demonstrate the superiority of the proposed method. As a\nbyproduct, we have released the codes and parameter settings to facilitate\nother researches.",
            "journal ref": "AAAI 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09755v1",
            "date": "2018-11-24 03:47:04+00:00",
            "title": "Estimation of Inter-Sentiment Correlations Employing Deep Neural Network Models",
            "authors": [
                "Xinzhi Wang",
                "Shengcheng Yuan",
                "Hui Zhang",
                "Yi Liu"
            ],
            "abstract": "This paper focuses on sentiment mining and sentiment correlation analysis of\nweb events. Although neural network models have contributed a lot to mining\ntext information, little attention is paid to analysis of the inter-sentiment\ncorrelations. This paper fills the gap between sentiment calculation and\ninter-sentiment correlations. In this paper, the social emotion is divided into\nsix categories: love, joy, anger, sadness, fear, and surprise. Two deep neural\nnetwork models are presented for sentiment calculation. Three datasets - the\ntitles, the bodies, the comments of news articles - are collected, covering\nboth objective and subjective texts in varying lengths (long and short). From\neach dataset, three kinds of features are extracted: explicit expression,\nimplicit expression, and alphabet characters. The performance of the two models\nare analyzed, with respect to each of the three kinds of the features. There is\ncontroversial phenomenon on the interpretation of anger (fn) and love (gd). In\nsubjective text, other emotions are easily to be considered as anger. By\ncontrast, in objective news bodies and titles, it is easy to regard text as\ncaused love (gd). It means, journalist may want to arouse emotion love by\nwriting news, but cause anger after the news is published. This result reflects\nthe sentiment complexity and unpredictability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10169v1",
            "date": "2018-11-26 04:00:03+00:00",
            "title": "Improving Gated Recurrent Unit Based Acoustic Modeling with Batch Normalization and Enlarged Context",
            "authors": [
                "Jie Li",
                "Yahui Shan",
                "Xiaorui Wang",
                "Yan Li"
            ],
            "abstract": "The use of future contextual information is typically shown to be helpful for\nacoustic modeling. Recently, we proposed a RNN model called minimal gated\nrecurrent unit with input projection (mGRUIP), in which a context module namely\ntemporal convolution, is specifically designed to model the future context.\nThis model, mGRUIP with context module (mGRUIP-Ctx), has been shown to be able\nof utilizing the future context effectively, meanwhile with quite low model\nlatency and computation cost. In this paper, we continue to improve mGRUIP-Ctx\nwith two revisions: applying BN methods and enlarging model context.\nExperimental results on two Mandarin ASR tasks (8400 hours and 60K hours) show\nthat, the revised mGRUIP-Ctx outperform LSTM with a large margin (11% to 38%).\nIt even performs slightly better than a superior BLSTM on the 8400h task, with\n33M less parameters and just 290ms model latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10238v2",
            "date": "2018-11-26 09:12:12+00:00",
            "title": "Learning Latent Beliefs and Performing Epistemic Reasoning for Efficient and Meaningful Dialog Management",
            "authors": [
                "Aishwarya Chhabra",
                "Pratik Saini",
                "Amit Sangroya",
                "C. Anantaram"
            ],
            "abstract": "Many dialogue management frameworks allow the system designer to directly\ndefine belief rules to implement an efficient dialog policy. Because these\nrules are directly defined, the components are said to be hand-crafted. As\ndialogues become more complex, the number of states, transitions, and policy\ndecisions becomes very large. To facilitate the dialog policy design process,\nwe propose an approach to automatically learn belief rules using a supervised\nmachine learning approach. We validate our ideas in Student-Advisor\nconversation domain, where we extract latent beliefs like student is curious,\nconfused and neutral, etc. Further, we also perform epistemic reasoning that\nhelps to tailor the dialog according to student's emotional state and hence\nimprove the overall effectiveness of the dialog system. Our latent belief\nidentification approach shows an accuracy of 87% and this results in efficient\nand meaningful dialog management.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.10403v2",
            "date": "2018-11-22 13:19:07+00:00",
            "title": "Running on Fumes--Preventing Out-of-Gas Vulnerabilities in Ethereum Smart Contracts using Static Resource Analysis",
            "authors": [
                "Elvira Albert",
                "Pablo Gordillo",
                "Albert Rubio",
                "Ilya Sergey"
            ],
            "abstract": "Gas is a measurement unit of the computational effort that it will take to\nexecute every single operation that takes part in the Ethereum blockchain\nplatform. Each instruction executed by the Ethereum Virtual Machine (EVM) has\nan associated gas consumption specified by Ethereum. If a transaction exceeds\nthe amount of gas allotted by the user (known as gas limit), an out-of-gas\nexception is raised. There is a wide family of contract vulnerabilities due to\nout-of-gas behaviours. We report on the design and implementation of GASTAP, a\nGas-Aware Smart contracT Analysis Platform, which takes as input a smart\ncontract (either in EVM, disassembled EVM, or in Solidity source code) and\nautomatically infers sound gas upper bounds for all its public functions. Our\nbounds ensure that if the gas limit paid by the user is higher than our\ninferred gas bounds, the contract is free of out-of-gas vulnerabilities.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10550v1",
            "date": "2018-11-26 17:53:17+00:00",
            "title": "Challenges in the Automatic Analysis of Students' Diagnostic Reasoning",
            "authors": [
                "Claudia Schulz",
                "Christian M. Meyer",
                "Michael Sailer",
                "Jan Kiesewetter",
                "Elisabeth Bauer",
                "Frank Fischer",
                "Martin R. Fischer",
                "Iryna Gurevych"
            ],
            "abstract": "Diagnostic reasoning is a key component of many professions. To improve\nstudents' diagnostic reasoning skills, educational psychologists analyse and\ngive feedback on epistemic activities used by these students while diagnosing,\nin particular, hypothesis generation, evidence generation, evidence evaluation,\nand drawing conclusions. However, this manual analysis is highly\ntime-consuming. We aim to enable the large-scale adoption of diagnostic\nreasoning analysis and feedback by automating the epistemic activity\nidentification. We create the first corpus for this task, comprising diagnostic\nreasoning self-explanations of students from two domains annotated with\nepistemic activities. Based on insights from the corpus creation and the task's\ncharacteristics, we discuss three challenges for the automatic identification\nof epistemic activities using AI methods: the correct identification of\nepistemic activity spans, the reliable distinction of similar epistemic\nactivities, and the detection of overlapping epistemic activities. We propose a\nseparate performance metric for each challenge and thus provide an evaluation\nframework for future research. Indeed, our evaluation of various\nstate-of-the-art recurrent neural network architectures reveals that current\ntechniques fail to address some of these challenges.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.10652v3",
            "date": "2018-11-26 19:23:33+00:00",
            "title": "Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions",
            "authors": [
                "Marcella Cornia",
                "Lorenzo Baraldi",
                "Rita Cucchiara"
            ],
            "abstract": "Current captioning approaches can describe images using black-box\narchitectures whose behavior is hardly controllable and explainable from the\nexterior. As an image can be described in infinite ways depending on the goal\nand the context at hand, a higher degree of controllability is needed to apply\ncaptioning algorithms in complex scenarios. In this paper, we introduce a novel\nframework for image captioning which can generate diverse descriptions by\nallowing both grounding and controllability. Given a control signal in the form\nof a sequence or set of image regions, we generate the corresponding caption\nthrough a recurrent architecture which predicts textual chunks explicitly\ngrounded on regions, following the constraints of the given control.\nExperiments are conducted on Flickr30k Entities and on COCO Entities, an\nextended version of COCO in which we add grounding annotations collected in a\nsemi-automatic manner. Results demonstrate that our method achieves state of\nthe art performances on controllable image captioning, in terms of caption\nquality and diversity. Code and annotations are publicly available at:\nhttps://github.com/aimagelab/show-control-and-tell.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.10667v2",
            "date": "2018-11-26 19:57:14+00:00",
            "title": "Embedding Uncertain Knowledge Graphs",
            "authors": [
                "Xuelu Chen",
                "Muhao Chen",
                "Weijia Shi",
                "Yizhou Sun",
                "Carlo Zaniolo"
            ],
            "abstract": "Embedding models for deterministic Knowledge Graphs (KG) have been\nextensively studied, with the purpose of capturing latent semantic relations\nbetween entities and incorporating the structured knowledge into machine\nlearning. However, there are many KGs that model uncertain knowledge, which\ntypically model the inherent uncertainty of relations facts with a confidence\nscore, and embedding such uncertain knowledge represents an unresolved\nchallenge. The capturing of uncertain knowledge will benefit many\nknowledge-driven applications such as question answering and semantic search by\nproviding more natural characterization of the knowledge. In this paper, we\npropose a novel uncertain KG embedding model UKGE, which aims to preserve both\nstructural and uncertainty information of relation facts in the embedding\nspace. Unlike previous models that characterize relation facts with binary\nclassification techniques, UKGE learns embeddings according to the confidence\nscores of uncertain relation facts. To further enhance the precision of UKGE,\nwe also introduce probabilistic soft logic to infer confidence scores for\nunseen relation facts during training. We propose and evaluate two variants of\nUKGE based on different learning objectives. Experiments are conducted on three\nreal-world uncertain KGs via three tasks, i.e. confidence prediction, relation\nfact ranking, and relation fact classification. UKGE shows effectiveness in\ncapturing uncertain knowledge by achieving promising results on these tasks,\nand consistently outperforms baselines on these tasks.",
            "journal ref": "Proceedings of the AAAI Conference on Artificial Intelligence.\n  Vol. 33. 2019",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.10686v1",
            "date": "2018-11-26 20:56:38+00:00",
            "title": "Beyond \"How may I help you?\": Assisting Customer Service Agents with Proactive Responses",
            "authors": [
                "Mengting Wan",
                "Xin Chen"
            ],
            "abstract": "We study the problem of providing recommended responses to customer service\nagents in live-chat dialogue systems. Smart-reply systems have been widely\napplied in real-world applications (e.g. Gmail, LinkedIn Messaging), where most\nof them can successfully recommend reactive responses. However, we observe a\nmajor limitation of current methods is that they generally have difficulties in\nsuggesting proactive investigation act (e.g. \"Do you perhaps have another\naccount with us?\") due to the lack of long-term context information, which\nindeed act as critical steps for customer service agents to collect information\nand resolve customers' issues. Thus in this work, we propose an end-to-end\nmethod with special focus on suggesting proactive investigative questions to\ncustomer agents in Airbnb's customer service live-chat system. Effectiveness of\nour proposed method can be validated through qualitative and quantitative\nresults.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.10830v2",
            "date": "2018-11-27 06:22:26+00:00",
            "title": "From Recognition to Cognition: Visual Commonsense Reasoning",
            "authors": [
                "Rowan Zellers",
                "Yonatan Bisk",
                "Ali Farhadi",
                "Yejin Choi"
            ],
            "abstract": "Visual understanding goes well beyond object recognition. With one glance at\nan image, we can effortlessly imagine the world beyond the pixels: for\ninstance, we can infer people's actions, goals, and mental states. While this\ntask is easy for humans, it is tremendously difficult for today's vision\nsystems, requiring higher-order cognition and commonsense reasoning about the\nworld. We formalize this task as Visual Commonsense Reasoning. Given a\nchallenging question about an image, a machine must answer correctly and then\nprovide a rationale justifying its answer.\n  Next, we introduce a new dataset, VCR, consisting of 290k multiple choice QA\nproblems derived from 110k movie scenes. The key recipe for generating\nnon-trivial and high-quality problems at scale is Adversarial Matching, a new\napproach to transform rich annotations into multiple choice questions with\nminimal bias. Experimental results show that while humans find VCR easy (over\n90% accuracy), state-of-the-art vision models struggle (~45%).\n  To move towards cognition-level understanding, we present a new reasoning\nengine, Recognition to Cognition Networks (R2C), that models the necessary\nlayered inferences for grounding, contextualization, and reasoning. R2C helps\nnarrow the gap between humans and machines (~65%); still, the challenge is far\nfrom solved, and we provide analysis that suggests avenues for future work.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.10986v5",
            "date": "2018-11-24 07:03:53+00:00",
            "title": "HCqa: Hybrid and Complex Question Answering on Textual Corpus and Knowledge Graph",
            "authors": [
                "Somayeh Asadifar",
                "Mohsen Kahani",
                "Saeedeh Shekarpour"
            ],
            "abstract": "Question Answering (QA) systems provide easy access to the vast amount of\nknowledge without having to know the underlying complex structure of the\nknowledge. The research community has provided ad hoc solutions to the key QA\ntasks, including named entity recognition and disambiguation, relation\nextraction and query building. Furthermore, some have integrated and composed\nthese components to implement many tasks automatically and efficiently.\nHowever, in general, the existing solutions are limited to simple and short\nquestions and still do not address complex questions composed of several\nsub-questions. Exploiting the answer to complex questions is further challenged\nif it requires integrating knowledge from unstructured data sources, i.e.,\ntextual corpus, as well as structured data sources, i.e., knowledge graphs. In\nthis paper, an approach (HCqa) is introduced for dealing with complex questions\nrequiring federating knowledge from a hybrid of heterogeneous data sources\n(structured and unstructured). We contribute in developing (i) a decomposition\nmechanism which extracts sub-questions from potentially long and complex input\nquestions, (ii) a novel comprehensive schema, first of its kind, for extracting\nand annotating relations, and (iii) an approach for executing and aggregating\nthe answers of sub-questions. The evaluation of HCqa showed a superior accuracy\nin the fundamental tasks, such as relation extraction, as well as the\nfederation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11136v1",
            "date": "2018-11-27 17:54:12+00:00",
            "title": "SOC: hunting the underground inside story of the ethereum Social-network Opinion and Comment",
            "authors": [
                "TonTon Hsien-De Huang",
                "Po-Wei Hong",
                "Ying-Tse Lee",
                "Yi-Lun Wang",
                "Chi-Leong Lok",
                "Hung-Yu Kao"
            ],
            "abstract": "The cryptocurrency is attracting more and more attention because of the\nblockchain technology. Ethereum is gaining a significant popularity in\nblockchain community, mainly due to the fact that it is designed in a way that\nenables developers to write smart contracts and decentralized applications\n(Dapps). There are many kinds of cryptocurrency information on the social\nnetwork. The risks and fraud problems behind it have pushed many countries\nincluding the United States, South Korea, and China to make warnings and set up\ncorresponding regulations. However, the security of Ethereum smart contracts\nhas not gained much attention. Through the Deep Learning approach, we propose a\nmethod of sentiment analysis for Ethereum's community comments. In this\nresearch, we first collected the users' cryptocurrency comments from the social\nnetwork and then fed to our LSTM + CNN model for training. Then we made\nprediction through sentiment analysis. With our research result, we have\ndemonstrated that both the precision and the recall of sentiment analysis can\nachieve 0.80+. More importantly, we deploy our sentiment analysis1 on\nRatingToken and Coin Master (mobile application of Cheetah Mobile Blockchain\nSecurity Center23). We can effectively provide detail information to resolve\nthe risks of being fake and fraud problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11365v2",
            "date": "2018-11-28 02:48:25+00:00",
            "title": "Unsupervised Multi-modal Neural Machine Translation",
            "authors": [
                "Yuanhang Su",
                "Kai Fan",
                "Nguyen Bach",
                "C. -C. Jay Kuo",
                "Fei Huang"
            ],
            "abstract": "Unsupervised neural machine translation (UNMT) has recently achieved\nremarkable results with only large monolingual corpora in each language.\nHowever, the uncertainty of associating target with source sentences makes UNMT\ntheoretically an ill-posed problem. This work investigates the possibility of\nutilizing images for disambiguation to improve the performance of UNMT. Our\nassumption is intuitively based on the invariant property of image, i.e., the\ndescription of the same visual content by different languages should be\napproximately similar. We propose an unsupervised multi-modal machine\ntranslation (UMNMT) framework based on the language translation cycle\nconsistency loss conditional on the image, targeting to learn the bidirectional\nmulti-modal translation simultaneously. Through an alternate training between\nmulti-modal and uni-modal, our inference model can translate with or without\nthe image. On the widely used Multi30K dataset, the experimental results of our\napproach are significantly better than those of the text-only UNMT on the 2016\ntest dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.11523v2",
            "date": "2018-11-28 12:42:57+00:00",
            "title": "Sequence Learning with RNNs for Medical Concept Normalization in User-Generated Texts",
            "authors": [
                "Elena Tutubalina",
                "Zulfat Miftahutdinov",
                "Sergey Nikolenko",
                "Valentin Malykh"
            ],
            "abstract": "In this work, we consider the medical concept normalization problem, i.e.,\nthe problem of mapping a disease mention in free-form text to a concept in a\ncontrolled vocabulary, usually to the standard thesaurus in the Unified Medical\nLanguage System (UMLS). This task is challenging since medical terminology is\nvery different when coming from health care professionals or from the general\npublic in the form of social media texts. We approach it as a sequence learning\nproblem, with recurrent neural networks trained to obtain semantic\nrepresentations of one- and multi-word expressions. We develop end-to-end\nneural architectures tailored specifically to medical concept normalization,\nincluding bidirectional LSTM and GRU with an attention mechanism and additional\nsemantic similarity features based on UMLS. Our evaluation over a standard\nbenchmark shows that our model improves over a state of the art baseline for\nclassification based on CNNs.",
            "journal ref": "Journal of Biomedical Informatics. - 2018. - Vol.84, Is.. -\n  P.93-102",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11707v1",
            "date": "2018-11-28 17:51:39+00:00",
            "title": "Few-Shot Generalization Across Dialogue Tasks",
            "authors": [
                "Vladimir Vlasov",
                "Akela Drissner-Schmid",
                "Alan Nichol"
            ],
            "abstract": "Machine-learning based dialogue managers are able to learn complex behaviors\nin order to complete a task, but it is not straightforward to extend their\ncapabilities to new domains. We investigate different policies' ability to\nhandle uncooperative user behavior, and how well expertise in completing one\ntask (such as restaurant reservations) can be reapplied when learning a new one\n(e.g. booking a hotel). We introduce the Recurrent Embedding Dialogue Policy\n(REDP), which embeds system actions and dialogue states in the same vector\nspace. REDP contains a memory component and attention mechanism based on a\nmodified Neural Turing Machine, and significantly outperforms a baseline LSTM\nclassifier on this task. We also show that both our architecture and baseline\nsolve the bAbI dialogue task, achieving 100% test accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11945v1",
            "date": "2018-11-29 03:40:55+00:00",
            "title": "HYPE: A High Performing NLP System for Automatically Detecting Hypoglycemia Events from Electronic Health Record Notes",
            "authors": [
                "Yonghao Jin",
                "Fei Li",
                "Hong Yu"
            ],
            "abstract": "Hypoglycemia is common and potentially dangerous among those treated for\ndiabetes. Electronic health records (EHRs) are important resources for\nhypoglycemia surveillance. In this study, we report the development and\nevaluation of deep learning-based natural language processing systems to\nautomatically detect hypoglycemia events from the EHR narratives. Experts in\nPublic Health annotated 500 EHR notes from patients with diabetes. We used this\nannotated dataset to train and evaluate HYPE, supervised NLP systems for\nhypoglycemia detection. In our experiment, the convolutional neural network\nmodel yielded promising performance $Precision=0.96 \\pm 0.03, Recall=0.86 \\pm\n0.03, F1=0.91 \\pm 0.03$ in a 10-fold cross-validation setting. Despite the\nannotated data is highly imbalanced, our CNN-based HYPE system still achieved a\nhigh performance for hypoglycemia detection. HYPE could be used for EHR-based\nhypoglycemia surveillance and to facilitate clinicians for timely treatment of\nhigh-risk patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12889v3",
            "date": "2018-11-30 17:01:28+00:00",
            "title": "Systematic Generalization: What Is Required and Can It Be Learned?",
            "authors": [
                "Dzmitry Bahdanau",
                "Shikhar Murty",
                "Michael Noukhovitch",
                "Thien Huu Nguyen",
                "Harm de Vries",
                "Aaron Courville"
            ],
            "abstract": "Numerous models for grounded language understanding have been recently\nproposed, including (i) generic models that can be easily adapted to any given\ntask and (ii) intuitively appealing modular models that require background\nknowledge to be instantiated. We compare both types of models in how much they\nlend themselves to a particular form of systematic generalization. Using a\nsynthetic VQA test, we evaluate which models are capable of reasoning about all\npossible object pairs after training on only a small subset of them. Our\nfindings show that the generalization of modular models is much more systematic\nand that it is highly sensitive to the module layout, i.e. to how exactly the\nmodules are connected. We furthermore investigate if modular models that\ngeneralize well could be made more end-to-end by learning their layout and\nparametrization. We find that end-to-end methods from prior work often learn\ninappropriate layouts or parametrizations that do not facilitate systematic\ngeneralization. Our results suggest that, in addition to modularity, systematic\ngeneralization in language understanding may require explicit regularizers or\npriors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12891v1",
            "date": "2018-11-30 17:01:48+00:00",
            "title": "Flexible and Scalable State Tracking Framework for Goal-Oriented Dialogue Systems",
            "authors": [
                "Rahul Goel",
                "Shachi Paul",
                "Tagyoung Chung",
                "Jeremie Lecomte",
                "Arindam Mandal",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Goal-oriented dialogue systems typically rely on components specifically\ndeveloped for a single task or domain. This limits such systems in two\ndifferent ways: If there is an update in the task domain, the dialogue system\nusually needs to be updated or completely re-trained. It is also harder to\nextend such dialogue systems to different and multiple domains. The dialogue\nstate tracker in conventional dialogue systems is one such component - it is\nusually designed to fit a well-defined application domain. For example, it is\ncommon for a state variable to be a categorical distribution over a\nmanually-predefined set of entities (Henderson et al., 2013), resulting in an\ninflexible and hard-to-extend dialogue system. In this paper, we propose a new\napproach for dialogue state tracking that can generalize well over multiple\ndomains without incorporating any domain-specific knowledge. Under this\nframework, discrete dialogue state variables are learned independently and the\ninformation of a predefined set of possible values for dialogue state variables\nis not required. Furthermore, it enables adding arbitrary dialogue context as\nfeatures and allows for multiple values to be associated with a single state\nvariable. These characteristics make it much easier to expand the dialogue\nstate space. We evaluate our framework using the widely used dialogue state\ntracking challenge data set (DSTC2) and show that our framework yields\ncompetitive results with other state-of-the-art results despite incorporating\nlittle domain knowledge. We also show that this framework can benefit from\nwidely available external resources such as pre-trained word embeddings.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.00176v1",
            "date": "2018-12-01 08:13:48+00:00",
            "title": "A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues",
            "authors": [
                "Zhouxing Shi",
                "Minlie Huang"
            ],
            "abstract": "Discourse structures are beneficial for various NLP tasks such as dialogue\nunderstanding, question answering, sentiment analysis, and so on. This paper\npresents a deep sequential model for parsing discourse dependency structures of\nmulti-party dialogues. The proposed model aims to construct a discourse\ndependency tree by predicting dependency relations and constructing the\ndiscourse structure jointly and alternately. It makes a sequential scan of the\nElementary Discourse Units (EDUs) in a dialogue. For each EDU, the model\ndecides to which previous EDU the current one should link and what the\ncorresponding relation type is. The predicted link and relation type are then\nused to build the discourse structure incrementally with a structured encoder.\nDuring link prediction and relation classification, the model utilizes not only\nlocal information that represents the concerned EDUs, but also global\ninformation that encodes the EDU sequence and the discourse structure that is\nalready built at the current step. Experiments show that the proposed model\noutperforms all the state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00235v3",
            "date": "2018-12-01 18:12:35+00:00",
            "title": "Learning to Caption Images through a Lifetime by Asking Questions",
            "authors": [
                "Kevin Shen",
                "Amlan Kar",
                "Sanja Fidler"
            ],
            "abstract": "In order to bring artificial agents into our lives, we will need to go beyond\nsupervised learning on closed datasets to having the ability to continuously\nexpand knowledge. Inspired by a student learning in a classroom, we present an\nagent that can continuously learn by posing natural language questions to\nhumans. Our agent is composed of three interacting modules, one that performs\ncaptioning, another that generates questions and a decision maker that learns\nwhen to ask questions by implicitly reasoning about the uncertainty of the\nagent and expertise of the teacher. As compared to current active learning\nmethods which query images for full captions, our agent is able to ask pointed\nquestions to improve the generated captions. The agent trains on the improved\ncaptions, expanding its knowledge. We show that our approach achieves better\nperformance using less human supervision than the baselines on the challenging\nMSCOCO dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.00350v1",
            "date": "2018-12-02 08:03:12+00:00",
            "title": "A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents",
            "authors": [
                "Heriberto Cuay\u00e1huitl",
                "Seonghan Ryu",
                "Donghyeon Lee",
                "Jihie Kim"
            ],
            "abstract": "The amount of dialogue history to include in a conversational agent is often\nunderestimated and/or set in an empirical and thus possibly naive way. This\nsuggests that principled investigations into optimal context windows are\nurgently needed given that the amount of dialogue history and corresponding\nrepresentations can play an important role in the overall performance of a\nconversational system. This paper studies the amount of history required by\nconversational agents for reliably predicting dialogue rewards. The task of\ndialogue reward prediction is chosen for investigating the effects of varying\namounts of dialogue history and their impact on system performance.\nExperimental results using a dataset of 18K human-human dialogues report that\nlengthy dialogue histories of at least 10 sentences are preferred (25 sentences\nbeing the best in our experiments) over short ones, and that lengthy histories\nare useful for training dialogue reward predictors with strong positive\ncorrelations between target dialogue rewards and predicted ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00382v1",
            "date": "2018-12-02 12:41:03+00:00",
            "title": "Improved and Robust Controversy Detection in General Web Pages Using Semantic Approaches under Large Scale Conditions",
            "authors": [
                "Jasper Linmans",
                "Bob van de Velde",
                "Evangelos Kanoulas"
            ],
            "abstract": "Detecting controversy in general web pages is a daunting task, but\nincreasingly essential to efficiently moderate discussions and effectively\nfilter problematic content. Unfortunately, controversies occur across many\ntopics and domains, with great changes over time. This paper investigates\nneural classifiers as a more robust methodology for controversy detection in\ngeneral web pages. Current models have often cast controversy detection on\ngeneral web pages as Wikipedia linking, or exact lexical matching tasks. The\ndiverse and changing nature of controversies suggest that semantic approaches\nare better able to detect controversy. We train neural networks that can\ncapture semantic information from texts using weak signal data. By leveraging\nthe semantic properties of word embeddings we robustly improve on existing\ncontroversy detection methods. To evaluate model stability over time and to\nunseen topics, we asses model performance under varying training conditions to\ntest cross-temporal, cross-topic, cross-domain performance and annotator\ncongruence. In doing so, we demonstrate that weak-signal based neural\napproaches are closer to human estimates of controversy and are more robust to\nthe inherent variability of controversies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.00436v1",
            "date": "2018-12-02 17:57:04+00:00",
            "title": "Learning Representations of Social Media Users",
            "authors": [
                "Adrian Benton"
            ],
            "abstract": "User representations are routinely used in recommendation systems by platform\ndevelopers, targeted advertisements by marketers, and by public policy\nresearchers to gauge public opinion across demographic groups. Computer\nscientists consider the problem of inferring user representations more\nabstractly; how does one extract a stable user representation - effective for\nmany downstream tasks - from a medium as noisy and complicated as social media?\n  The quality of a user representation is ultimately task-dependent (e.g. does\nit improve classifier performance, make more accurate recommendations in a\nrecommendation system) but there are proxies that are less sensitive to the\nspecific task. Is the representation predictive of latent properties such as a\nperson's demographic features, socioeconomic class, or mental health state? Is\nit predictive of the user's future behavior?\n  In this thesis, we begin by showing how user representations can be learned\nfrom multiple types of user behavior on social media. We apply several\nextensions of generalized canonical correlation analysis to learn these\nrepresentations and evaluate them at three tasks: predicting future hashtag\nmentions, friending behavior, and demographic features. We then show how user\nfeatures can be employed as distant supervision to improve topic model fit.\nFinally, we show how user features can be integrated into and improve existing\nclassifiers in the multitask learning framework. We treat user representations\n- ground truth gender and mental health features - as auxiliary tasks to\nimprove mental health state prediction. We also use distributed user\nrepresentations learned in the first chapter to improve tweet-level stance\nclassifiers, showing that distant user information can inform classification\ntasks at the granularity of a single message.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.00677v2",
            "date": "2018-12-03 11:34:26+00:00",
            "title": "Clinical Document Classification Using Labeled and Unlabeled Data Across Hospitals",
            "authors": [
                "Hamed Hassanzadeh",
                "Mahnoosh Kholghi",
                "Anthony Nguyen",
                "Kevin Chu"
            ],
            "abstract": "Reviewing radiology reports in emergency departments is an essential but\nlaborious task. Timely follow-up of patients with abnormal cases in their\nradiology reports may dramatically affect the patient's outcome, especially if\nthey have been discharged with a different initial diagnosis. Machine learning\napproaches have been devised to expedite the process and detect the cases that\ndemand instant follow up. However, these approaches require a large amount of\nlabeled data to train reliable predictive models. Preparing such a large\ndataset, which needs to be manually annotated by health professionals, is\ncostly and time-consuming. This paper investigates a semi-supervised learning\nframework for radiology report classification across three hospitals. The main\ngoal is to leverage clinical unlabeled data in order to augment the learning\nprocess where limited labeled data is available. To further improve the\nclassification performance, we also integrate a transfer learning technique\ninto the semi-supervised learning pipeline . Our experimental findings show\nthat (1) convolutional neural networks (CNNs), while being independent of any\nproblem-specific feature engineering, achieve significantly higher\neffectiveness compared to conventional supervised learning approaches, (2)\nleveraging unlabeled data in training a CNN-based classifier reduces the\ndependency on labeled data by more than 50% to reach the same performance of a\nfully supervised CNN, and (3) transferring the knowledge gained from available\nlabeled data in an external source hospital significantly improves the\nperformance of a semi-supervised CNN model over their fully supervised\ncounterparts in a target hospital.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01250v1",
            "date": "2018-12-04 07:05:47+00:00",
            "title": "Quantification and Analysis of Scientific Language Variation Across Research Fields",
            "authors": [
                "Pei Zhou",
                "Muhao Chen",
                "Kai-Wei Chang",
                "Carlo Zaniolo"
            ],
            "abstract": "Quantifying differences in terminologies from various academic domains has\nbeen a longstanding problem yet to be solved. We propose a computational\napproach for analyzing linguistic variation among scientific research fields by\ncapturing the semantic change of terms based on a neural language model. The\nmodel is trained on a large collection of literature in five computer science\nresearch fields, for which we obtain field-specific vector representations for\nkey terms, and global vector representations for other words. Several\nquantitative approaches are introduced to identify the terms whose semantics\nhave drastically changed, or remain unchanged across different research fields.\nWe also propose a metric to quantify the overall linguistic variation of\nresearch fields. After quantitative evaluation on human annotated data and\nqualitative comparison with other methods, we show that our model can improve\ncross-disciplinary data collaboration by identifying terms that potentially\ninduce confusion during interdisciplinary studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01260v1",
            "date": "2018-12-04 07:48:00+00:00",
            "title": "Tartan: A retrieval-based socialbot powered by a dynamic finite-state machine architecture",
            "authors": [
                "George Larionov",
                "Zachary Kaden",
                "Hima Varsha Dureddy",
                "Gabriel Bayomi T. Kalejaiye",
                "Mihir Kale",
                "Srividya Pranavi Potharaju",
                "Ankit Parag Shah",
                "Alexander I Rudnicky"
            ],
            "abstract": "This paper describes the Tartan conversational agent built for the 2018 Alexa\nPrize Competition. Tartan is a non-goal-oriented socialbot focused around\nproviding users with an engaging and fluent casual conversation. Tartan's key\nfeatures include an emphasis on structured conversation based on flexible\nfinite-state models and an approach focused on understanding and using\nconversational acts. To provide engaging conversations, Tartan blends\nscript-like yet dynamic responses with data-based generative and retrieval\nmodels. Unique to Tartan is that our dialog manager is modeled as a dynamic\nFinite State Machine. To our knowledge, no other conversational agent\nimplementation has followed this specific structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01431v1",
            "date": "2018-11-30 19:12:04+00:00",
            "title": "Modeling natural language emergence with integral transform theory and reinforcement learning",
            "authors": [
                "Bohdan Khomtchouk",
                "Shyam Sudhakaran"
            ],
            "abstract": "Zipf's law predicts a power-law relationship between word rank and frequency\nin language communication systems and has been widely reported in a variety of\nnatural language processing applications. However, the emergence of natural\nlanguage is often modeled as a function of bias between speaker and listener\ninterests, which lacks a direct way of relating information-theoretic bias to\nZipfian rank. A function of bias also serves as an unintuitive interpretation\nof the communicative effort exchanged between a speaker and a listener. We\ncounter these shortcomings by proposing a novel integral transform and kernel\nfor mapping communicative bias functions to corresponding word frequency-rank\nrepresentations at any arbitrary phase transition point, resulting in a direct\nway to link communicative effort (modeled by speaker/listener bias) to specific\nvocabulary used (represented by word rank). We demonstrate the practical\nutility of our integral transform by showing how a change from bias to rank\nresults in greater accuracy and performance at an image classification task for\nassigning word labels to images randomly subsampled from CIFAR10. We model this\ntask as a reinforcement learning game between a speaker and listener and\ncompare the relative impact of bias and Zipfian word rank on communicative\nperformance (and accuracy) between the two agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01828v1",
            "date": "2018-12-05 06:31:48+00:00",
            "title": "Graph based Question Answering System",
            "authors": [
                "Piyush Mital",
                "Saurabh Agarwal",
                "Bhargavi Neti",
                "Yashodhara Haribhakta",
                "Vibhavari Kamble",
                "Krishnanjan Bhattacharjee",
                "Debashri Das",
                "Swati Mehta",
                "Ajai Kumar"
            ],
            "abstract": "In today's digital age in the dawning era of big data analytics it is not the\ninformation but the linking of information through entities and actions which\ndefines the discourse. Any textual data either available on the Internet off\noff-line (like newspaper data, Wikipedia dump, etc) is basically connect\ninformation which cannot be treated isolated for its wholesome semantics. There\nis a need for an automated retrieval process with proper information extraction\nto structure the data for relevant and fast text analytics. The first big\nchallenge is the conversion of unstructured textual data to structured data.\nUnlike other databases, graph databases handle relationships and connections\nelegantly. Our project aims at developing a graph-based information extraction\nand retrieval system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01879v1",
            "date": "2018-12-05 09:45:52+00:00",
            "title": "An enhanced computational feature selection method for medical synonym identification via bilingualism and multi-corpus training",
            "authors": [
                "K. Lei",
                "S. Si",
                "D. Wen",
                "Y. Shen"
            ],
            "abstract": "Medical synonym identification has been an important part of medical natural\nlanguage processing (NLP). However, in the field of Chinese medical synonym\nidentification, there are problems like low precision and low recall rate. To\nsolve the problem, in this paper, we propose a method for identifying Chinese\nmedical synonyms. We first selected 13 features including Chinese and English\nfeatures. Then we studied the synonym identification results of each feature\nalone and different combinations of the features. Through the comparison among\nidentification results, we present an optimal combination of features for\nChinese medical synonym identification. Experiments show that our selected\nfeatures have achieved 97.37% precision rate, 96.00% recall rate and 97.33% F1\nscore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01884v1",
            "date": "2018-12-05 09:58:54+00:00",
            "title": "MedSim: A Novel Semantic Similarity Measure in Bio-medical Knowledge Graphs",
            "authors": [
                "Kai Lei",
                "Kaiqi Yuan",
                "Qiang Zhang",
                "Ying Shen"
            ],
            "abstract": "We present MedSim, a novel semantic SIMilarity method based on public\nwell-established bio-MEDical knowledge graphs (KGs) and large-scale corpus, to\nstudy the therapeutic substitution of antibiotics. Besides hierarchy and corpus\nof KGs, MedSim further interprets medicine characteristics by constructing\nmulti-dimensional medicine-specific feature vectors. Dataset of 528 antibiotic\npairs scored by doctors is applied for evaluation and MedSim has produced\nstatistically significant improvement over other semantic similarity methods.\nFurthermore, some promising applications of MedSim in drug substitution and\ndrug abuse prevention are presented in case study.",
            "journal ref": "International Conference on Knowledge Science, Engineering and\n  Management KSEM 2018: Knowledge Science, Engineering and Management pp\n  479-490",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01887v1",
            "date": "2018-12-05 10:08:29+00:00",
            "title": "Approach for Semi-automatic Construction of Anti-infective Drug Ontology Based on Entity Linking",
            "authors": [
                "Ying Shen",
                "Yang Deng",
                "Kaiqi Yuan",
                "Li Liu",
                "Yong Liu"
            ],
            "abstract": "Ontology can be used for the interpretation of natural language. To construct\nan anti-infective drug ontology, one needs to design and deploy a\nmethodological step to carry out the entity discovery and linking. Medical\nsynonym resources have been an important part of medical natural language\nprocessing (NLP). However, there are problems such as low precision and low\nrecall rate. In this study, an NLP approach is adopted to generate candidate\nentities. Open ontology is analyzed to extract semantic relations. Six-word\nvector features and word-level features are selected to perform the entity\nlinking. The extraction results of synonyms with a single feature and different\ncombinations of features are studied. Experiments show that our selected\nfeatures have achieved a precision rate of 86.77%, a recall rate of 89.03% and\nan F1 score of 87.89%. This paper finally presents the structure of the\nproposed ontology and its relevant statistical data.",
            "journal ref": "International Conference on Smart Computing and Communication\n  SmartCom 2017: Smart Computing and Communication pp 268-277",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01889v1",
            "date": "2018-12-05 10:10:56+00:00",
            "title": "A Knowledge Graph Based Solution for Entity Discovery and Linking in Open-Domain Questions",
            "authors": [
                "Kai Lei",
                "Bing Zhang",
                "Yong Liu",
                "Yang Deng",
                "Dongyu Zhang",
                "Ying Shen"
            ],
            "abstract": "Named entity discovery and linking is the fundamental and core component of\nquestion answering. In Question Entity Discovery and Linking (QEDL) problem,\ntraditional methods are challenged because multiple entities in one short\nquestion are difficult to be discovered entirely and the incomplete information\nin short text makes entity linking hard to implement. To overcome these\ndifficulties, we proposed a knowledge graph based solution for QEDL and\ndeveloped a system consists of Question Entity Discovery (QED) module and\nEntity Linking (EL) module. The method of QED module is a tradeoff and ensemble\nof two methods. One is the method based on knowledge graph retrieval, which\ncould extract more entities in questions and guarantee the recall rate, the\nother is the method based on Conditional Random Field (CRF), which improves the\nprecision rate. The EL module is treated as a ranking problem and Learning to\nRank (LTR) method with features such as semantic similarity, text similarity\nand entity popularity is utilized to extract and make full use of the\ninformation in short texts. On the official dataset of a shared QEDL evaluation\ntask, our approach could obtain 64.44% F1 score of QED and 64.86% accuracy of\nEL, which ranks the 2nd place and indicates its practical use for QEDL problem.",
            "journal ref": "International Conference on Smart Computing and Communication\n  SmartCom 2017: Smart Computing and Communication pp 181-190",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02142v1",
            "date": "2018-12-05 18:00:36+00:00",
            "title": "End-to-end contextual speech recognition using class language models and a token passing decoder",
            "authors": [
                "Zhehuai Chen",
                "Mahaveer Jain",
                "Yongqiang Wang",
                "Michael L. Seltzer",
                "Christian Fuegen"
            ],
            "abstract": "End-to-end modeling (E2E) of automatic speech recognition (ASR) blends all\nthe components of a traditional speech recognition system into a unified model.\nAlthough it simplifies training and decoding pipelines, the unified model is\nhard to adapt when mismatch exists between training and test data. In this\nwork, we focus on contextual speech recognition, which is particularly\nchallenging for E2E models because it introduces significant mismatch between\ntraining and test data. To improve the performance in the presence of complex\ncontextual information, we propose to use class-based language models(CLM) that\ncan populate the classes with contextdependent information in real-time. To\nenable this approach to scale to a large number of class members and minimize\nsearch errors, we propose a token passing decoder with efficient token\nrecombination for E2E systems for the first time. We evaluate the proposed\nsystem on general and contextual ASR, and achieve relative 62% Word Error\nRate(WER) reduction for contextual ASR without hurting performance for general\nASR. We show that the proposed method performs well without modification of the\ndecoding hyper-parameters across tasks, making it a general solution for E2E\nASR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02305v1",
            "date": "2018-12-06 06:56:16+00:00",
            "title": "Pathology Extraction from Chest X-Ray Radiology Reports: A Performance Study",
            "authors": [
                "Tahsin Mostafiz",
                "Khalid Ashraf"
            ],
            "abstract": "Extraction of relevant pathological terms from radiology reports is important\nfor correct image label generation and disease population studies. In this\nletter, we compare the performance of some known application program interface\n(APIs) for the task of thoracic abnormality extraction from radiology reports.\nWe explored several medical domain specific annotation tools like Medical Text\nIndexer(MTI) with Non-MEDLINE and Mesh On Demand(MOD) options and generic\nNatural Language Understanding (NLU) API provided by the IBM cloud. Our results\nshow that although MTI and MOD are intended for extracting medical terms, their\nperformance is worst compared to generic extraction API like IBM NLU. Finally,\nwe trained a DNN-based Named Entity Recognition (NER) model to extract the key\nconcept words from radiology reports. Our model outperforms the medical\nspecific and generic API performance by a large margin. Our results demonstrate\nthe inadequacy of generic APIs for pathology extraction task and establish the\nimportance of domain specific model training for improved results. We hope that\nthese results motivate the research community to release larger de-identified\nradiology reports corpus for building high accuracy machine learning models for\nthe important task of pathology extraction.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.02536v2",
            "date": "2018-12-06 14:11:25+00:00",
            "title": "Evaluating Architectural Choices for Deep Learning Approaches for Question Answering over Knowledge Bases",
            "authors": [
                "Sherzod Hakimov",
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "The task of answering natural language questions over knowledge bases has\nreceived wide attention in recent years. Various deep learning architectures\nhave been proposed for this task. However, architectural design choices are\ntypically not systematically compared nor evaluated under the same conditions.\nIn this paper, we contribute to a better understanding of the impact of\narchitectural design choices by evaluating four different architectures under\nthe same conditions. We address the task of answering simple questions,\nconsisting in predicting the subject and predicate of a triple given a\nquestion. In order to provide a fair comparison of different architectures, we\nevaluate them under the same strategy for inferring the subject, and compare\ndifferent architectures for inferring the predicate. The architecture for\ninferring the subject is based on a standard LSTM model trained to recognize\nthe span of the subject in the question and on a linking component that links\nthe subject span to an entity in the knowledge base. The architectures for\npredicate inference are based on i) a standard softmax classifier ranging over\nall predicates as output, iii) a model that predicts a low-dimensional encoding\nof the property given entity representation and question, iii) a model that\nlearns to score a pair of subject and predicate given the question as well as\niv) a model based on the well-known FastText model. The comparison of\narchitectures shows that FastText provides better results than other\narchitectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02627v1",
            "date": "2018-12-06 16:01:13+00:00",
            "title": "Relevant Word Order Vectorization for Improved Natural Language Processing in Electronic Healthcare Records",
            "authors": [
                "Jeffrey Thompson",
                "Jinxiang Hu",
                "Dinesh Pal Mudaranthakam",
                "David Streeter",
                "Lisa Neums",
                "Michele Park",
                "Devin C. Koestler",
                "Byron Gajewski",
                "Matthew S. Mayo"
            ],
            "abstract": "Objective: Electronic health records (EHR) represent a rich resource for\nconducting observational studies, supporting clinical trials, and more.\nHowever, much of the relevant information is stored in an unstructured format\nthat makes it difficult to use. Natural language processing approaches that\nattempt to automatically classify the data depend on vectorization algorithms\nthat impose structure on the text, but these algorithms were not designed for\nthe unique characteristics of EHR. Here, we propose a new algorithm for\nstructuring so-called free-text that may help researchers make better use of\nEHR. We call this method Relevant Word Order Vectorization (RWOV).\n  Materials and Methods: As a proof-of-concept, we attempted to classify the\nhormone receptor status of breast cancer patients treated at the University of\nKansas Medical Center during a recent year, from the unstructured text of\npathology reports. Our approach attempts to account for the semi-structured way\nthat healthcare providers often enter information. We compared this approach to\nthe ngrams and word2vec methods.\n  Results: Our approach resulted in the most consistently high accuracy, as\nmeasured by F1 score and area under the receiver operating characteristic curve\n(AUC).\n  Discussion: Our results suggest that methods of structuring free text that\ntake into account its context may show better performance, and that our\napproach is promising.\n  Conclusion: By using a method that accounts for the fact that healthcare\nproviders tend to use certain key words repetitively and that the order of\nthese key words is important, we showed improved performance over methods that\ndo not.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02655v1",
            "date": "2018-12-06 16:45:00+00:00",
            "title": "Feature Analysis for Assessing the Quality of Wikipedia Articles through Supervised Classification",
            "authors": [
                "Elias Bassani",
                "Marco Viviani"
            ],
            "abstract": "Nowadays, thanks to Web 2.0 technologies, people have the possibility to\ngenerate and spread contents on different social media in a very easy way. In\nthis context, the evaluation of the quality of the information that is\navailable online is becoming more and more a crucial issue. In fact, a constant\nflow of contents is generated every day by often unknown sources, which are not\ncertified by traditional authoritative entities. This requires the development\nof appropriate methodologies that can evaluate in a systematic way these\ncontents, based on `objective' aspects connected with them. This would help\nindividuals, who nowadays tend to increasingly form their opinions based on\nwhat they read online and on social media, to come into contact with\ninformation that is actually useful and verified. Wikipedia is nowadays one of\nthe biggest online resources on which users rely as a source of information.\nThe amount of collaboratively generated content that is sent to the online\nencyclopedia every day can let to the possible creation of low-quality articles\n(and, consequently, misinformation) if not properly monitored and revised. For\nthis reason, in this paper, the problem of automatically assessing the quality\nof Wikipedia articles is considered. In particular, the focus is on the\nanalysis of hand-crafted features that can be employed by supervised machine\nlearning techniques to perform the classification of Wikipedia articles on\nqualitative bases. With respect to prior literature, a wider set of\ncharacteristics connected to Wikipedia articles are taken into account and\nillustrated in detail. Evaluations are performed by considering a labeled\ndataset provided in a prior work, and different supervised machine learning\nalgorithms, which produced encouraging results with respect to the considered\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02793v1",
            "date": "2018-12-06 20:35:14+00:00",
            "title": "Generation of Synthetic Electronic Medical Record Text",
            "authors": [
                "Jiaqi Guan",
                "Runzhe Li",
                "Sheng Yu",
                "Xuegong Zhang"
            ],
            "abstract": "Machine learning (ML) and Natural Language Processing (NLP) have achieved\nremarkable success in many fields and have brought new opportunities and high\nexpectation in the analyses of medical data. The most common type of medical\ndata is the massive free-text electronic medical records (EMR). It is widely\nregarded that mining such massive data can bring up important information for\nimproving medical practices as well as for possible new discoveries on complex\ndiseases. However, the free EMR texts are lacking consistent standards, rich of\nprivate information, and limited in availability. Also, as they are accumulated\nfrom everyday practices, it is often hard to have a balanced number of samples\nfor the types of diseases under study. These problems hinder the development of\nML and NLP methods for EMR data analysis. To tackle these problems, we\ndeveloped a model to generate synthetic text of EMRs called Medical Text\nGenerative Adversarial Network or mtGAN. It is based on the GAN framework and\nis trained by the REINFORCE algorithm. It takes disease features as inputs and\ngenerates synthetic texts as EMRs for the corresponding diseases. We evaluate\nthe model from micro-level, macro-level and application-level on a Chinese EMR\ntext dataset. The results show that the method has a good capacity to fit real\ndata and can generate realistic and diverse EMR samples. This provides a novel\nway to avoid potential leakage of patient privacy while still supply sufficient\nwell-controlled cohort data for developing downstream ML and NLP methods. It\ncan also be used as a data augmentation method to assist studies based on real\nEMR data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02825v5",
            "date": "2018-12-05 03:05:08+00:00",
            "title": "Attending to Mathematical Language with Transformers",
            "authors": [
                "Artit Wangperawong"
            ],
            "abstract": "Mathematical expressions were generated, evaluated and used to train neural\nnetwork models based on the transformer architecture. The expressions and their\ntargets were analyzed as a character-level sequence transduction task in which\nthe encoder and decoder are built on attention mechanisms. Three models were\ntrained to understand and evaluate symbolic variables and expressions in\nmathematics: (1) the self-attentive and feed-forward transformer without\nrecurrence or convolution, (2) the universal transformer with recurrence, and\n(3) the adaptive universal transformer with recurrence and adaptive computation\ntime. The models respectively achieved test accuracies as high as 76.1%, 78.8%\nand 84.9% in evaluating the expressions to match the target values. For the\ncases inferred incorrectly, the results differed from the targets by only one\nor two characters. The models notably learned to add, subtract and multiply\nboth positive and negative decimal numbers of variable digits assigned to\nsymbolic variables.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03781v1",
            "date": "2018-12-10 13:36:09+00:00",
            "title": "Inflo: News Categorization and Keyphrase Extraction for Implementation in an Aggregation System",
            "authors": [
                "Pranav A",
                "Nick Sukiennik",
                "Pan Hui"
            ],
            "abstract": "The work herein describes a system for automatic news category and keyphrase\nlabeling, presented in the context of our motivation to improve the speed at\nwhich a user can find relevant and interesting content within an aggregation\nplatform. A set of 12 discrete categories were applied to over 500,000 news\narticles for training a neural network, to be used to facilitate the more\nin-depth task of extracting the most significant keyphrases. The latter was\ndone using three methods: statistical, graphical and numerical, using the\npre-identified category label to improve relevance of extracted phrases. The\nresults are presented in a demo in which the articles are pre-populated via\nNews API, and upon being selected, the category and keyphrase labels will be\ncomputed via the methods explained herein.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.04081v3",
            "date": "2018-12-10 20:52:41+00:00",
            "title": "Chat-crowd: A Dialog-based Platform for Visual Layout Composition",
            "authors": [
                "Paola Cascante-Bonilla",
                "Xuwang Yin",
                "Vicente Ordonez",
                "Song Feng"
            ],
            "abstract": "In this paper we introduce Chat-crowd, an interactive environment for visual\nlayout composition via conversational interactions. Chat-crowd supports\nmultiple agents with two conversational roles: agents who play the role of a\ndesigner are in charge of placing objects in an editable canvas according to\ninstructions or commands issued by agents with a director role. The system can\nbe integrated with crowdsourcing platforms for both synchronous and\nasynchronous data collection and is equipped with comprehensive quality\ncontrols on the performance of both types of agents. We expect that this system\nwill be useful to build multimodal goal-oriented dialog tasks that require\nspatial and geometric reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04199v1",
            "date": "2018-12-11 03:06:02+00:00",
            "title": "Predicting the Effects of News Sentiments on the Stock Market",
            "authors": [
                "Dev Shah",
                "Haruna Isah",
                "Farhana Zulkernine"
            ],
            "abstract": "Stock market forecasting is very important in the planning of business\nactivities. Stock price prediction has attracted many researchers in multiple\ndisciplines including computer science, statistics, economics, finance, and\noperations research. Recent studies have shown that the vast amount of online\ninformation in the public domain such as Wikipedia usage pattern, news stories\nfrom the mainstream media, and social media discussions can have an observable\neffect on investors opinions towards financial markets. The reliability of the\ncomputational models on stock market prediction is important as it is very\nsensitive to the economy and can directly lead to financial loss. In this\npaper, we retrieved, extracted, and analyzed the effects of news sentiments on\nthe stock market. Our main contributions include the development of a sentiment\nanalysis dictionary for the financial sector, the development of a\ndictionary-based sentiment analysis model, and the evaluation of the model for\ngauging the effects of news sentiments on stocks for the pharmaceutical market.\nUsing only news sentiments, we achieved a directional accuracy of 70.59% in\npredicting the trends in short-term stock price movement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04377v2",
            "date": "2018-12-11 13:09:13+00:00",
            "title": "Deep Reader: Information extraction from Document images via relation extraction and Natural Language",
            "authors": [
                "Vishwanath D",
                "Rohit Rahul",
                "Gunjan Sehgal",
                "Swati",
                "Arindam Chowdhury",
                "Monika Sharma",
                "Lovekesh Vig",
                "Gautam Shroff",
                "Ashwin Srinivasan"
            ],
            "abstract": "Recent advancements in the area of Computer Vision with state-of-art Neural\nNetworks has given a boost to Optical Character Recognition (OCR) accuracies.\nHowever, extracting characters/text alone is often insufficient for relevant\ninformation extraction as documents also have a visual structure that is not\ncaptured by OCR. Extracting information from tables, charts, footnotes, boxes,\nheadings and retrieving the corresponding structured representation for the\ndocument remains a challenge and finds application in a large number of\nreal-world use cases. In this paper, we propose a novel enterprise based\nend-to-end framework called DeepReader which facilitates information extraction\nfrom document images via identification of visual entities and populating a\nmeta relational model across different entities in the document image. The\nmodel schema allows for an easy to understand abstraction of the entities\ndetected by the deep vision models and the relationships between them.\nDeepReader has a suite of state-of-the-art vision algorithms which are applied\nto recognize handwritten and printed text, eliminate noisy effects, identify\nthe type of documents and detect visual entities like tables, lines and boxes.\nDeep Reader maps the extracted entities into a rich relational schema so as to\ncapture all the relevant relationships between entities (words, textboxes,\nlines etc) detected in the document. Relevant information and fields can then\nbe extracted from the document by writing SQL queries on top of the\nrelationship tables. A natural language based interface is added on top of the\nrelationship schema so that a non-technical user, specifying the queries in\nnatural language, can fetch the information with minimal effort. In this paper,\nwe also demonstrate many different capabilities of Deep Reader and report\nresults on a real-world use case.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.04718v1",
            "date": "2018-12-05 03:59:41+00:00",
            "title": "Text Data Augmentation Made Simple By Leveraging NLP Cloud APIs",
            "authors": [
                "Claude Coulombe"
            ],
            "abstract": "In practice, it is common to find oneself with far too little text data to\ntrain a deep neural network. This \"Big Data Wall\" represents a challenge for\nminority language communities on the Internet, organizations, laboratories and\ncompanies that compete the GAFAM (Google, Amazon, Facebook, Apple, Microsoft).\nWhile most of the research effort in text data augmentation aims on the\nlong-term goal of finding end-to-end learning solutions, which is equivalent to\n\"using neural networks to feed neural networks\", this engineering work focuses\non the use of practical, robust, scalable and easy-to-implement data\naugmentation pre-processing techniques similar to those that are successful in\ncomputer vision. Several text augmentation techniques have been experimented.\nSome existing ones have been tested for comparison purposes such as noise\ninjection or the use of regular expressions. Others are modified or improved\ntechniques like lexical replacement. Finally more innovative ones, such as the\ngeneration of paraphrases using back-translation or by the transformation of\nsyntactic trees, are based on robust, scalable, and easy-to-use NLP Cloud APIs.\nAll the text augmentation techniques studied, with an amplification factor of\nonly 5, increased the accuracy of the results in a range of 4.3% to 21.6%, with\nsignificant statistical fluctuations, on a standardized task of text polarity\nprediction. Some standard deep neural network architectures were tested: the\nmultilayer perceptron (MLP), the long short-term memory recurrent network\n(LSTM) and the bidirectional LSTM (biLSTM). Classical XGBoost algorithm has\nbeen tested with up to 2.5% improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04840v3",
            "date": "2018-12-12 08:06:30+00:00",
            "title": "Towards Understanding Language through Perception in Situated Human-Robot Interaction: From Word Grounding to Grammar Induction",
            "authors": [
                "Amir Aly",
                "Tadahiro Taniguchi"
            ],
            "abstract": "Robots are widely collaborating with human users in diferent tasks that\nrequire high-level cognitive functions to make them able to discover the\nsurrounding environment. A difcult challenge that we briefy highlight in this\nshort paper is inferring the latent grammatical structure of language, which\nincludes grounding parts of speech (e.g., verbs, nouns, adjectives, and\nprepositions) through visual perception, and induction of Combinatory\nCategorial Grammar (CCG) for phrases. This paves the way towards grounding\nphrases so as to make a robot able to understand human instructions\nappropriately during interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05001v1",
            "date": "2018-12-12 16:11:31+00:00",
            "title": "Temporal Analysis of Entity Relatedness and its Evolution using Wikipedia and DBpedia",
            "authors": [
                "Narumol Prangnawarat",
                "John P. McCrae",
                "Conor Hayes"
            ],
            "abstract": "Many researchers have made use of the Wikipedia network for relatedness and\nsimilarity tasks. However, most approaches use only the most recent information\nand not historical changes in the network. We provide an analysis of entity\nrelatedness using temporal graph-based approaches over different versions of\nthe Wikipedia article link network and DBpedia, which is an open-source\nknowledge base extracted from Wikipedia. We consider creating the Wikipedia\narticle link network as both a union and intersection of edges over multiple\ntime points and present a novel variation of the Jaccard index to weight edges\nbased on their transience. We evaluate our results against the KORE dataset,\nwhich was created in 2010, and show that using the 2010 Wikipedia article link\nnetwork produces the strongest result, suggesting that semantic similarity is\ntime sensitive. We then show that integrating multiple time frames in our\nmethods can give a better overall similarity demonstrating that temporal\nevolution can have an important effect on entity relatedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05035v2",
            "date": "2018-12-12 17:12:58+00:00",
            "title": "Structured Neural Topic Models for Reviews",
            "authors": [
                "Babak Esmaeili",
                "Hongyi Huang",
                "Byron C. Wallace",
                "Jan-Willem van de Meent"
            ],
            "abstract": "We present Variational Aspect-based Latent Topic Allocation (VALTA), a family\nof autoencoding topic models that learn aspect-based representations of\nreviews. VALTA defines a user-item encoder that maps bag-of-words vectors for\ncombined reviews associated with each paired user and item onto structured\nembeddings, which in turn define per-aspect topic weights. We model individual\nreviews in a structured manner by inferring an aspect assignment for each\nsentence in a given review, where the per-aspect topic weights obtained by the\nuser-item encoder serve to define a mixture over topics, conditioned on the\naspect. The result is an autoencoding neural topic model for reviews, which can\nbe trained in a fully unsupervised manner to learn topics that are structured\ninto aspects. Experimental evaluation on large number of datasets demonstrates\nthat aspects are interpretable, yield higher coherence scores than\nnon-structured autoencoding topic model variants, and can be utilized to\nperform aspect-based comparison and genre discovery.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05199v2",
            "date": "2018-12-12 23:49:58+00:00",
            "title": "Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task",
            "authors": [
                "Liang Qiu",
                "Yuanyi Ding",
                "Lei He"
            ],
            "abstract": "In recent years, Recurrent Neural Networks (RNNs) based models have been\napplied to the Slot Filling problem of Spoken Language Understanding and\nachieved the state-of-the-art performances. In this paper, we investigate the\neffect of incorporating pre-trained language models into RNN based Slot Filling\nmodels. Our evaluation on the Airline Travel Information System (ATIS) data\ncorpus shows that we can significantly reduce the size of labeled training data\nand achieve the same level of Slot Filling performance by incorporating extra\nword embedding and language model embedding layers pre-trained on unlabeled\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05366v1",
            "date": "2018-12-13 11:23:18+00:00",
            "title": "Dynamic Feature Generation Network for Answer Selection",
            "authors": [
                "Longxuan Ma",
                "Pengfei Wang",
                "Lei Zhang"
            ],
            "abstract": "Extracting appropriate features to represent a corpus is an important task\nfor textual mining. Previous attention based work usually enhance feature at\nthe lexical level, which lacks the exploration of feature augmentation at the\nsentence level. In this paper, we exploit a Dynamic Feature Generation Network\n(DFGN) to solve this problem. Specifically, DFGN generates features based on a\nvariety of attention mechanisms and attaches features to sentence\nrepresentation. Then a thresholder is designed to filter the mined features\nautomatically. DFGN extracts the most significant characteristics from datasets\nto keep its practicability and robustness. Experimental results on multiple\nwell-known answer selection datasets show that our proposed approach\nsignificantly outperforms state-of-the-art baselines. We give a detailed\nanalysis of the experiments to illustrate why DFGN provides excellent retrieval\nand interpretative ability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05407v1",
            "date": "2018-12-13 13:13:23+00:00",
            "title": "Abstractive Text Summarization by Incorporating Reader Comments",
            "authors": [
                "Shen Gao",
                "Xiuying Chen",
                "Piji Li",
                "Zhaochun Ren",
                "Lidong Bing",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "In neural abstractive summarization field, conventional sequence-to-sequence\nbased models often suffer from summarizing the wrong aspect of the document\nwith respect to the main aspect. To tackle this problem, we propose the task of\nreader-aware abstractive summary generation, which utilizes the reader comments\nto help the model produce better summary about the main aspect. Unlike\ntraditional abstractive summarization task, reader-aware summarization\nconfronts two main challenges: (1) Comments are informal and noisy; (2) jointly\nmodeling the news document and the reader comments is challenging. To tackle\nthe above challenges, we design an adversarial learning model named\nreader-aware summary generator (RASG), which consists of four components: (1) a\nsequence-to-sequence based summary generator; (2) a reader attention module\ncapturing the reader focused aspects; (3) a supervisor modeling the semantic\ngap between the generated summary and reader focused aspects; (4) a goal\ntracker producing the goal for each generation step. The supervisor and the\ngoal tacker are used to guide the training of our framework in an adversarial\nmanner. Extensive experiments are conducted on our large-scale real-world text\nsummarization dataset, and the results show that RASG achieves the\nstate-of-the-art performance in terms of both automatic metrics and human\nevaluations. The experimental results also demonstrate the effectiveness of\neach module in our framework. We release our large-scale dataset for further\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05634v2",
            "date": "2018-12-13 19:07:17+00:00",
            "title": "Adversarial Inference for Multi-Sentence Video Description",
            "authors": [
                "Jae Sung Park",
                "Marcus Rohrbach",
                "Trevor Darrell",
                "Anna Rohrbach"
            ],
            "abstract": "While significant progress has been made in the image captioning task, video\ndescription is still in its infancy due to the complex nature of video data.\nGenerating multi-sentence descriptions for long videos is even more\nchallenging. Among the main issues are the fluency and coherence of the\ngenerated descriptions, and their relevance to the video. Recently,\nreinforcement and adversarial learning based methods have been explored to\nimprove the image captioning models; however, both types of methods suffer from\na number of issues, e.g. poor readability and high redundancy for RL and\nstability issues for GANs. In this work, we instead propose to apply\nadversarial techniques during inference, designing a discriminator which\nencourages better multi-sentence video description. In addition, we find that a\nmulti-discriminator \"hybrid\" design, where each discriminator targets one\naspect of a description, leads to the best results. Specifically, we decouple\nthe discriminator to evaluate on three criteria: 1) visual relevance to the\nvideo, 2) language diversity and fluency, and 3) coherence across sentences.\nOur approach results in more accurate, diverse, and coherent multi-sentence\nvideo descriptions, as shown by automatic as well as human evaluation on the\npopular ActivityNet Captions dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.05718v1",
            "date": "2018-12-13 22:50:04+00:00",
            "title": "Multilayer Network Model of Movie Script",
            "authors": [
                "Youssef Mourchid",
                "Benjamin Renoust",
                "Hocine Cherifi",
                "Mohammed El Hassouni"
            ],
            "abstract": "Network models have been increasingly used in the past years to support\nsummarization and analysis of narratives, such as famous TV series, books and\nnews. Inspired by social network analysis, most of these models focus on the\ncharacters at play. The network model well captures all characters\ninteractions, giving a broad picture of the narration's content. A few works\nwent beyond by introducing additional semantic elements, always captured in a\nsingle layer network. In contrast, we introduce in this work a multilayer\nnetwork model to capture more elements of the narration of a movie from its\nscript: people, locations, and other semantic elements. This model enables new\nmeasures and insights on movies. We demonstrate this model on two very popular\nmovies.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1812.05774v1",
            "date": "2018-12-14 04:12:02+00:00",
            "title": "Don't Classify, Translate: Multi-Level E-Commerce Product Categorization Via Machine Translation",
            "authors": [
                "Maggie Yundi Li",
                "Stanley Kok",
                "Liling Tan"
            ],
            "abstract": "E-commerce platforms categorize their products into a multi-level taxonomy\ntree with thousands of leaf categories. Conventional methods for product\ncategorization are typically based on machine learning classification\nalgorithms. These algorithms take product information as input (e.g., titles\nand descriptions) to classify a product into a leaf category. In this paper, we\npropose a new paradigm based on machine translation. In our approach, we\ntranslate a product's natural language description into a sequence of tokens\nrepresenting a root-to-leaf path in a product taxonomy. In our experiments on\ntwo large real-world datasets, we show that our approach achieves better\npredictive accuracy than a state-of-the-art classification system for product\ncategorization. In addition, we demonstrate that our machine translation models\ncan propose meaningful new paths between previously unconnected nodes in a\ntaxonomy tree, thereby transforming the taxonomy into a directed acyclic graph\n(DAG). We discuss how the resultant taxonomy DAG promotes user-friendly\nnavigation, and how it is more adaptable to new products.",
            "journal ref": "Workshop on Information Technologies and Systems 2018 (WITS2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06081v1",
            "date": "2018-12-14 18:59:41+00:00",
            "title": "A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization",
            "authors": [
                "Sendong Zhao",
                "Ting Liu",
                "Sicheng Zhao",
                "Fei Wang"
            ],
            "abstract": "State-of-the-art studies have demonstrated the superiority of joint modelling\nover pipeline implementation for medical named entity recognition and\nnormalization due to the mutual benefits between the two processes. To exploit\nthese benefits in a more sophisticated way, we propose a novel deep neural\nmulti-task learning framework with explicit feedback strategies to jointly\nmodel recognition and normalization. On one hand, our method benefits from the\ngeneral representations of both tasks provided by multi-task learning. On the\nother hand, our method successfully converts hierarchical tasks into a parallel\nmulti-task setting while maintaining the mutual supports between tasks. Both of\nthese aspects improve the model performance. Experimental results demonstrate\nthat our method performs significantly better than state-of-the-art approaches\non two publicly available medical literature datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06176v1",
            "date": "2018-12-14 21:32:40+00:00",
            "title": "Bootstrapping Conversational Agents With Weak Supervision",
            "authors": [
                "Neil Mallinar",
                "Abhishek Shah",
                "Rajendra Ugrani",
                "Ayush Gupta",
                "Manikandan Gurusankar",
                "Tin Kam Ho",
                "Q. Vera Liao",
                "Yunfeng Zhang",
                "Rachel K. E. Bellamy",
                "Robert Yates",
                "Chris Desmarais",
                "Blake McGregor"
            ],
            "abstract": "Many conversational agents in the market today follow a standard bot\ndevelopment framework which requires training intent classifiers to recognize\nuser input. The need to create a proper set of training examples is often the\nbottleneck in the development process. In many occasions agent developers have\naccess to historical chat logs that can provide a good quantity as well as\ncoverage of training examples. However, the cost of labeling them with tens to\nhundreds of intents often prohibits taking full advantage of these chat logs.\nIn this paper, we present a framework called \\textit{search, label, and\npropagate} (SLP) for bootstrapping intents from existing chat logs using weak\nsupervision. The framework reduces hours to days of labeling effort down to\nminutes of work by using a search engine to find examples, then relies on a\ndata programming approach to automatically expand the labels. We report on a\nuser study that shows positive user feedback for this new approach to build\nconversational agents, and demonstrates the effectiveness of using data\nprogramming for auto-labeling. While the system is developed for training\nconversational agents, the framework has broader application in significantly\nreducing labeling effort for training text classifiers.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.06280v4",
            "date": "2018-12-15 12:51:39+00:00",
            "title": "Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia",
            "authors": [
                "Ikuya Yamada",
                "Akari Asai",
                "Jin Sakuma",
                "Hiroyuki Shindo",
                "Hideaki Takeda",
                "Yoshiyasu Takefuji",
                "Yuji Matsumoto"
            ],
            "abstract": "The embeddings of entities in a large knowledge base (e.g., Wikipedia) are\nhighly beneficial for solving various natural language tasks that involve real\nworld knowledge. In this paper, we present Wikipedia2Vec, a Python-based\nopen-source tool for learning the embeddings of words and entities from\nWikipedia. The proposed tool enables users to learn the embeddings efficiently\nby issuing a single command with a Wikipedia dump file as an argument. We also\nintroduce a web-based demonstration of our tool that allows users to visualize\nand explore the learned embeddings. In our experiments, our tool achieved a\nstate-of-the-art result on the KORE entity relatedness dataset, and competitive\nresults on various standard benchmark datasets. Furthermore, our tool has been\nused as a key component in various recent studies. We publicize the source\ncode, demonstration, and the pretrained embeddings for 12 languages at\nhttps://wikipedia2vec.github.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06624v1",
            "date": "2018-12-17 06:24:03+00:00",
            "title": "Feature Fusion Effects of Tensor Product Representation on (De)Compositional Network for Caption Generation for Images",
            "authors": [
                "Chiranjib Sur"
            ],
            "abstract": "Progress in image captioning is gradually getting complex as researchers try\nto generalized the model and define the representation between visual features\nand natural language processing. This work tried to define such kind of\nrelationship in the form of representation called Tensor Product Representation\n(TPR) which generalized the scheme of language modeling and structuring the\nlinguistic attributes (related to grammar and parts of speech of language)\nwhich will provide a much better structure and grammatically correct sentence.\nTPR enables better and unique representation and structuring of the feature\nspace and will enable better sentence composition from these representations. A\nlarge part of the different ways of defining and improving these TPR are\ndiscussed and their performance with respect to the traditional procedures and\nfeature representations are evaluated for image captioning application. The new\nmodels achieved considerable improvement than the corresponding previous\narchitectures.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.06700v1",
            "date": "2018-12-17 11:20:15+00:00",
            "title": "Hateminers : Detecting Hate speech against Women",
            "authors": [
                "Punyajoy Saha",
                "Binny Mathew",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "With the online proliferation of hate speech, there is an urgent need for\nsystems that can detect such harmful content. In this paper, We present the\nmachine learning models developed for the Automatic Misogyny Identification\n(AMI) shared task at EVALITA 2018. We generate three types of features:\nSentence Embeddings, TF-IDF Vectors, and BOW Vectors to represent each tweet.\nThese features are then concatenated and fed into the machine learning models.\nOur model came First for the English Subtask A and Fifth for the English\nSubtask B. We release our winning model for public use and it's available at\nhttps://github.com/punyajoy/Hateminers-EVALITA.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1812.07023v1",
            "date": "2018-12-17 19:41:37+00:00",
            "title": "From FiLM to Video: Multi-turn Question Answering with Multi-modal Context",
            "authors": [
                "Dat Tien Nguyen",
                "Shikhar Sharma",
                "Hannes Schulz",
                "Layla El Asri"
            ],
            "abstract": "Understanding audio-visual content and the ability to have an informative\nconversation about it have both been challenging areas for intelligent systems.\nThe Audio Visual Scene-aware Dialog (AVSD) challenge, organized as a track of\nthe Dialog System Technology Challenge 7 (DSTC7), proposes a combined task,\nwhere a system has to answer questions pertaining to a video given a dialogue\nwith previous question-answer pairs and the video itself. We propose for this\ntask a hierarchical encoder-decoder model which computes a multi-modal\nembedding of the dialogue context. It first embeds the dialogue history using\ntwo LSTMs. We extract video and audio frames at regular intervals and compute\nsemantic features using pre-trained I3D and VGGish models, respectively. Before\nsummarizing both modalities into fixed-length vectors using LSTMs, we use FiLM\nblocks to condition them on the embeddings of the current question, which\nallows us to reduce the dimensionality considerably. Finally, we use an LSTM\ndecoder that we train with scheduled sampling and evaluate using beam search.\nCompared to the modality-fusing baseline model released by the AVSD challenge\norganizers, our model achieves a relative improvements of more than 16%,\nscoring 0.36 BLEU-4 and more than 33%, scoring 0.997 CIDEr.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07108v2",
            "date": "2018-12-17 23:51:58+00:00",
            "title": "Learning Private Neural Language Modeling with Attentive Aggregation",
            "authors": [
                "Shaoxiong Ji",
                "Shirui Pan",
                "Guodong Long",
                "Xue Li",
                "Jing Jiang",
                "Zi Huang"
            ],
            "abstract": "Mobile keyboard suggestion is typically regarded as a word-level language\nmodeling problem. Centralized machine learning technique requires massive user\ndata collected to train on, which may impose privacy concerns for sensitive\npersonal typing data of users. Federated learning (FL) provides a promising\napproach to learning private language modeling for intelligent personalized\nkeyboard suggestion by training models in distributed clients rather than\ntraining in a central server. To obtain a global model for prediction, existing\nFL algorithms simply average the client models and ignore the importance of\neach client during model aggregation. Furthermore, there is no optimization for\nlearning a well-generalized global model on the central server. To solve these\nproblems, we propose a novel model aggregation with the attention mechanism\nconsidering the contribution of clients models to the global model, together\nwith an optimization technique during server aggregation. Our proposed\nattentive aggregation method minimizes the weighted distance between the server\nmodel and client models through iterative parameters updating while attends the\ndistance between the server model and client models. Through experiments on two\npopular language modeling datasets and a social media dataset, our proposed\nmethod outperforms its counterparts in terms of perplexity and communication\ncost in most settings of comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.07200v1",
            "date": "2018-12-18 07:11:19+00:00",
            "title": "D{\u00e9}tection de locuteurs dans les s{\u00e9}ries TV",
            "authors": [
                "Xavier Bost",
                "Georges Linares"
            ],
            "abstract": "Speaker diarization of audio streams turns out to be particularly challenging\nwhen applied to fictional films, where many characters talk in various acoustic\nconditions (background music, sound effects, variations in intonation...).\nDespite this acoustic variability, such movies exhibit specific visual\npatterns, particularly within dialogue scenes. In this paper, we introduce a\ntwo-step method to achieve speaker diarization in TV series: speaker\ndiarization is first performed locally within scenes visually identified as\ndialogues; then, the hypothesized local speakers are compared to each other\nduring a second clustering process in order to detect recurring speakers: this\nsecond stage of clustering is subject to the constraint that the different\nspeakers involved in the same dialogue have to be assigned to different\nclusters. The performances of our approach are compared to those obtained by\nstandard speaker diarization tools applied to the same data.",
            "journal ref": "Coria 2015, Mar 2015, Paris, France",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1812.07205v2",
            "date": "2018-12-18 07:21:36+00:00",
            "title": "Audiovisual speaker diarization of TV series",
            "authors": [
                "Xavier Bost",
                "Georges Linar\u00e8s",
                "Serigne Gueye"
            ],
            "abstract": "Speaker diarization may be difficult to achieve when applied to narrative\nfilms, where speakers usually talk in adverse acoustic conditions: background\nmusic, sound effects, wide variations in intonation may hide the inter-speaker\nvariability and make audio-based speaker diarization approaches error prone. On\nthe other hand, such fictional movies exhibit strong regularities at the image\nlevel, particularly within dialogue scenes. In this paper, we propose to\nperform speaker diarization within dialogue scenes of TV series by combining\nthe audio and video modalities: speaker diarization is first performed by using\neach modality, the two resulting partitions of the instance set are then\noptimally matched, before the remaining instances, corresponding to cases of\ndisagreement between both modalities, are finally processed. The results\nobtained by applying such a multi-modal approach to fictional films turn out to\noutperform those obtained by relying on a single modality.",
            "journal ref": "2015 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Apr 2015, Brisbane, Australia. IEEE, pp.4799-4803, 2015",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1812.07209v2",
            "date": "2018-12-18 07:29:27+00:00",
            "title": "Constrained speaker diarization of TV series based on visual patterns",
            "authors": [
                "Xavier Bost",
                "Georges Linares"
            ],
            "abstract": "Speaker diarization, usually denoted as the ''who spoke when'' task, turns\nout to be particularly challenging when applied to fictional films, where many\ncharacters talk in various acoustic conditions (background music, sound\neffects...). Despite this acoustic variability , such movies exhibit specific\nvisual patterns in the dialogue scenes. In this paper, we introduce a two-step\nmethod to achieve speaker diarization in TV series: a speaker diarization is\nfirst performed locally in the scenes detected as dialogues; then, the\nhypothesized local speakers are merged in a second agglomerative clustering\nprocess, with the constraint that speakers locally hypothesized to be distinct\nmust not be assigned to the same cluster. The performances of our approach are\ncompared to those obtained by standard speaker diarization tools applied to the\nsame data.",
            "journal ref": "2014 IEEE Spoken Language Technology Workshop (SLT), Dec 2014,\n  South Lake Tahoe, United States. IEEE, pp.390-395, 2014",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1812.07248v3",
            "date": "2018-12-18 09:05:53+00:00",
            "title": "Attend, Copy, Parse -- End-to-end information extraction from documents",
            "authors": [
                "Rasmus Berg Palm",
                "Florian Laws",
                "Ole Winther"
            ],
            "abstract": "Document information extraction tasks performed by humans create data\nconsisting of a PDF or document image input, and extracted string outputs. This\nend-to-end data is naturally consumed and produced when performing the task\nbecause it is valuable in and of itself. It is naturally available, at no\nadditional cost. Unfortunately, state-of-the-art word classification methods\nfor information extraction cannot use this data, instead requiring word-level\nlabels which are expensive to create and consequently not available for many\nreal life tasks. In this paper we propose the Attend, Copy, Parse architecture,\na deep neural network model that can be trained directly on end-to-end data,\nbypassing the need for word-level labels. We evaluate the proposed architecture\non a large diverse set of invoices, and outperform a state-of-the-art\nproduction system based on word classification. We believe our proposed\narchitecture can be used on many real life information extraction tasks where\nword classification cannot be used due to a lack of the required word-level\nlabels.",
            "journal ref": "ICDAR 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08039v1",
            "date": "2018-12-19 15:54:41+00:00",
            "title": "Semantic Frame Parsing for Information Extraction : the CALOR corpus",
            "authors": [
                "Gabriel Marzinotto",
                "Jeremy Auguste",
                "Frederic Bechet",
                "G\u00e9raldine Damnati",
                "Alexis Nasr"
            ],
            "abstract": "This paper presents a publicly available corpus of French encyclopedic\nhistory texts annotated according to the Berkeley FrameNet formalism. The main\ndifference in our approach compared to previous works on semantic parsing with\nFrameNet is that we are not interested here in full text parsing but rather on\npartial parsing. The goal is to select from the FrameNet resources the minimal\nset of frames that are going to be useful for the applicative framework\ntargeted, in our case Information Extraction from encyclopedic documents. Such\nan approach leverages the manual annotation of larger corpora than those\nobtained through full text parsing and therefore opens the door to alternative\nmethods for Frame parsing than those used so far on the FrameNet 1.5 benchmark\ncorpus. The approaches compared in this study rely on an integrated sequence\nlabeling model which jointly optimizes frame identification and semantic role\nsegmentation and identification. The models compared are CRFs and multitasks\nbi-LSTMs.",
            "journal ref": "LREC2018, May 2018, Miyazaki, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08304v1",
            "date": "2018-12-20 01:07:04+00:00",
            "title": "Recommendation System based on Semantic Scholar Mining and Topic modeling: A behavioral analysis of researchers from six conferences",
            "authors": [
                "Hamed Jelodar",
                "Yongli Wang",
                "Mahdi Rabbani",
                "Ru-xin Zhao",
                "Seyedvalyallah Ayobi",
                "Peng Hu",
                "Isma Masood"
            ],
            "abstract": "Recommendation systems have an important place to help online users in the\ninternet society. Recommendation Systems in computer science are of very\npractical use these days in various aspects of the Internet portals, such as\nsocial networks, and library websites. There are several approaches to\nimplement recommendation systems, Latent Dirichlet Allocation (LDA) is one the\npopular techniques in Topic Modeling. Recently, researchers have proposed many\napproaches based on Recommendation Systems and LDA. According to importance of\nthe subject, in this paper we discover the trends of the topics and find\nrelationship between LDA topics and Scholar-Context-documents. In fact, We\napply probabilistic topic modeling based on Gibbs sampling algorithms for a\nsemantic mining from six conference publications in computer science from DBLP\ndataset. According to our experimental results, our semantic framework can be\neffective to help organizations to better organize these conferences and cover\nfuture research topics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.08769v4",
            "date": "2018-12-20 18:53:05+00:00",
            "title": "What are the biases in my word embedding?",
            "authors": [
                "Nathaniel Swinger",
                "Maria De-Arteaga",
                "Neil Thomas Heffernan IV",
                "Mark DM Leiserson",
                "Adam Tauman Kalai"
            ],
            "abstract": "This paper presents an algorithm for enumerating biases in word embeddings.\nThe algorithm exposes a large number of offensive associations related to\nsensitive features such as race and gender on publicly available embeddings,\nincluding a supposedly \"debiased\" embedding. These biases are concerning in\nlight of the widespread use of word embeddings. The associations are identified\nby geometric patterns in word embeddings that run parallel between people's\nnames and common lower-case tokens. The algorithm is highly unsupervised: it\ndoes not even require the sensitive features to be pre-specified. This is\ndesirable because: (a) many forms of discrimination--such as racial\ndiscrimination--are linked to social constructs that may vary depending on the\ncontext, rather than to categories with fixed definitions; and (b) it makes it\neasier to identify biases against intersectional groups, which depend on\ncombinations of sensitive features. The inputs to our algorithm are a list of\ntarget tokens, e.g. names, and a word embedding. It outputs a number of Word\nEmbedding Association Tests (WEATs) that capture various biases present in the\ndata. We illustrate the utility of our approach on publicly available word\nembeddings and lists of names, and evaluate its output using crowdsourcing. We\nalso show how removing names may not remove potential proxy bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08879v1",
            "date": "2018-12-20 22:53:33+00:00",
            "title": "Variational Cross-domain Natural Language Generation for Spoken Dialogue Systems",
            "authors": [
                "Bo-Hsiang Tseng",
                "Florian Kreyssig",
                "Pawel Budzianowski",
                "Inigo Casanueva",
                "Yen-Chen Wu",
                "Stefan Ultes",
                "Milica Gasic"
            ],
            "abstract": "Cross-domain natural language generation (NLG) is still a difficult task\nwithin spoken dialogue modelling. Given a semantic representation provided by\nthe dialogue manager, the language generator should generate sentences that\nconvey desired information. Traditional template-based generators can produce\nsentences with all necessary information, but these sentences are not\nsufficiently diverse. With RNN-based models, the diversity of the generated\nsentences can be high, however, in the process some information is lost. In\nthis work, we improve an RNN-based generator by considering latent information\nat the sentence level during generation using the conditional variational\nautoencoder architecture. We demonstrate that our model outperforms the\noriginal RNN-based generator, while yielding highly diverse sentences. In\naddition, our model performs better when the training data is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09193v1",
            "date": "2018-12-21 15:30:17+00:00",
            "title": "Sources of Complexity in Semantic Frame Parsing for Information Extraction",
            "authors": [
                "Gabriel Marzinotto",
                "Fr\u00e9d\u00e9ric B\u00e9chet",
                "G\u00e9raldine Damnati",
                "Alexis Nasr"
            ],
            "abstract": "This paper describes a Semantic Frame parsing System based on sequence\nlabeling methods, precisely BiLSTM models with highway connections, for\nperforming information extraction on a corpus of French encyclopedic history\ntexts annotated according to the Berkeley FrameNet formalism. The approach\nproposed in this study relies on an integrated sequence labeling model which\njointly optimizes frame identification and semantic role segmentation and\nidentification. The purpose of this study is to analyze the task complexity, to\nhighlight the factors that make Semantic Frame parsing a difficult task and to\nprovide detailed evaluations of the performance on different types of frames\nand sentences.",
            "journal ref": "International FrameNet Workshop 2018, May 2018, Miyazaki, Japan",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09336v1",
            "date": "2018-12-21 19:02:52+00:00",
            "title": "An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition",
            "authors": [
                "Devesh Walawalkar",
                "Yihui He",
                "Rohit Pillai"
            ],
            "abstract": "In this project, we worked on speech recognition, specifically predicting\nindividual words based on both the video frames and audio. Empowered by\nconvolutional neural networks, the recent speech recognition and lip reading\nmodels are comparable to human level performance. We re-implemented and made\nderivations of the state-of-the-art model. Then, we conducted rich experiments\nincluding the effectiveness of attention mechanism, more accurate residual\nnetwork as the backbone with pre-trained weights and the sensitivity of our\nmodel with respect to audio input with/without noise.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.09617v4",
            "date": "2018-12-22 22:53:19+00:00",
            "title": "Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification",
            "authors": [
                "Mozhi Zhang",
                "Yoshinari Fujinuma",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Text classification must sometimes be applied in a low-resource language with\nno labeled training data. However, training data may be available in a related\nlanguage. We investigate whether character-level knowledge transfer from a\nrelated language helps text classification. We present a cross-lingual document\nclassification framework (CACO) that exploits cross-lingual subword similarity\nby jointly training a character-based embedder and a word-based classifier. The\nembedder derives vector representations for input words from their written\nforms, and the classifier makes predictions based on the word vectors. We use a\njoint character representation for both the source language and the target\nlanguage, which allows the embedder to generalize knowledge about source\nlanguage words to target language words with similar forms. We propose a\nmulti-task objective that can further improve the model if additional\ncross-lingual or monolingual resources are available. Experiments confirm that\ncharacter-level knowledge transfer is more data-efficient than word-level\ntransfer between related languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09653v1",
            "date": "2018-12-23 03:47:29+00:00",
            "title": "Supervised Sentiment Classification with CNNs for Diverse SE Datasets",
            "authors": [
                "Achyudh Ram",
                "Meiyappan Nagappan"
            ],
            "abstract": "Sentiment analysis, a popular technique for opinion mining, has been used by\nthe software engineering research community for tasks such as assessing app\nreviews, developer emotions in issue trackers and developer opinions on APIs.\nPast research indicates that state-of-the-art sentiment analysis techniques\nhave poor performance on SE data. This is because sentiment analysis tools are\noften designed to work on non-technical documents such as movie reviews. In\nthis study, we attempt to solve the issues with existing sentiment analysis\ntechniques for SE texts by proposing a hierarchical model based on\nconvolutional neural networks (CNN) and long short-term memory (LSTM) trained\non top of pre-trained word vectors. We assessed our model's performance and\nreliability by comparing it with a number of frequently used sentiment analysis\ntools on five gold standard datasets. Our results show that our model pushes\nthe state of the art further on all datasets in terms of accuracy. We also show\nthat it is possible to get better accuracy after labelling a small sample of\nthe dataset and re-training our model rather than using an unsupervised\nclassifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09664v1",
            "date": "2018-12-23 06:28:30+00:00",
            "title": "Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input",
            "authors": [
                "Junliang Guo",
                "Xu Tan",
                "Di He",
                "Tao Qin",
                "Linli Xu",
                "Tie-Yan Liu"
            ],
            "abstract": "Non-autoregressive translation (NAT) models, which remove the dependence on\nprevious target tokens from the inputs of the decoder, achieve significantly\ninference speedup but at the cost of inferior accuracy compared to\nautoregressive translation (AT) models. Previous work shows that the quality of\nthe inputs of the decoder is important and largely impacts the model accuracy.\nIn this paper, we propose two methods to enhance the decoder inputs so as to\nimprove NAT models. The first one directly leverages a phrase table generated\nby conventional SMT approaches to translate source tokens to target tokens,\nwhich are then fed into the decoder as inputs. The second one transforms\nsource-side word embeddings to target-side word embeddings through\nsentence-level alignment and word-level adversary learning, and then feeds the\ntransformed word embeddings into the decoder as inputs. Experimental results\nshow our method largely outperforms the NAT baseline~\\citep{gu2017non} by\n$5.11$ BLEU scores on WMT14 English-German task and $4.72$ BLEU scores on WMT16\nEnglish-Romanian task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09836v2",
            "date": "2018-12-24 05:29:19+00:00",
            "title": "Moment Matching Training for Neural Machine Translation: A Preliminary Study",
            "authors": [
                "Cong Duy Vu Hoang",
                "Ioan Calapodescu",
                "Marc Dymetman"
            ],
            "abstract": "In previous works, neural sequence models have been shown to improve\nsignificantly if external prior knowledge can be provided, for instance by\nallowing the model to access the embeddings of explicit features during both\ntraining and inference. In this work, we propose a different point of view on\nhow to incorporate prior knowledge in a principled way, using a moment matching\nframework. In this approach, the standard local cross-entropy training of the\nsequential model is combined with a moment matching training mode that\nencourages the equality of the expectations of certain predefined features\nbetween the model distribution and the empirical distribution. In particular,\nwe show how to derive unbiased estimates of some stochastic gradients that are\ncentral to the training, and compare our framework with a formally related one:\npolicy gradient training in reinforcement learning, pointing out some important\ndifferences in terms of the kinds of prior assumptions in both approaches. Our\ninitial results are promising, showing the effectiveness of our proposed\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10230v1",
            "date": "2018-12-26 05:17:03+00:00",
            "title": "Learning to Refine Source Representations for Neural Machine Translation",
            "authors": [
                "Xinwei Geng",
                "Longyue Wang",
                "Xing Wang",
                "Bing Qin",
                "Ting Liu",
                "Zhaopeng Tu"
            ],
            "abstract": "Neural machine translation (NMT) models generally adopt an encoder-decoder\narchitecture for modeling the entire translation process. The encoder\nsummarizes the representation of input sentence from scratch, which is\npotentially a problem if the sentence is ambiguous. When translating a text,\nhumans often create an initial understanding of the source sentence and then\nincrementally refine it along the translation on the target side. Starting from\nthis intuition, we propose a novel encoder-refiner-decoder framework, which\ndynamically refines the source representations based on the generated\ntarget-side information at each decoding step. Since the refining operations\nare time-consuming, we propose a strategy, leveraging the power of\nreinforcement learning models, to decide when to refine at specific decoding\nsteps. Experimental results on both Chinese-English and English-German\ntranslation tasks show that the proposed approach significantly and\nconsistently improves translation performance over the standard encoder-decoder\nframework. Furthermore, when refining strategy is applied, results still show\nreasonable improvement over the baseline without much decrease in decoding\nspeed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10233v3",
            "date": "2018-12-26 05:43:23+00:00",
            "title": "An Investigation of Few-Shot Learning in Spoken Term Classification",
            "authors": [
                "Yangbin Chen",
                "Tom Ko",
                "Lifeng Shang",
                "Xiao Chen",
                "Xin Jiang",
                "Qing Li"
            ],
            "abstract": "In this paper, we investigate the feasibility of applying few-shot learning\nalgorithms to a speech task. We formulate a user-defined scenario of spoken\nterm classification as a few-shot learning problem. In most few-shot learning\nstudies, it is assumed that all the N classes are new in a N-way problem. We\nsuggest that this assumption can be relaxed and define a N+M-way problem where\nN and M are the number of new classes and fixed classes respectively. We\npropose a modification to the Model-Agnostic Meta-Learning (MAML) algorithm to\nsolve the problem. Experiments on the Google Speech Commands dataset show that\nour approach outperforms the conventional supervised learning approach and the\noriginal MAML.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10315v1",
            "date": "2018-12-26 13:50:50+00:00",
            "title": "DBpedia NIF: Open, Large-Scale and Multilingual Knowledge Extraction Corpus",
            "authors": [
                "Milan Dojchinovski",
                "Julio Hernandez",
                "Markus Ackermann",
                "Amit Kirschenbaum",
                "Sebastian Hellmann"
            ],
            "abstract": "In the past decade, the DBpedia community has put significant amount of\neffort on developing technical infrastructure and methods for efficient\nextraction of structured information from Wikipedia. These efforts have been\nprimarily focused on harvesting, refinement and publishing semi-structured\ninformation found in Wikipedia articles, such as information from infoboxes,\ncategorization information, images, wikilinks and citations. Nevertheless,\nstill vast amount of valuable information is contained in the unstructured\nWikipedia article texts. In this paper, we present DBpedia NIF - a large-scale\nand multilingual knowledge extraction corpus. The aim of the dataset is\ntwo-fold: to dramatically broaden and deepen the amount of structured\ninformation in DBpedia, and to provide large-scale and multilingual language\nresource for development of various NLP and IR task. The dataset provides the\ncontent of all articles for 128 Wikipedia languages. We describe the dataset\ncreation process and the NLP Interchange Format (NIF) used to model the\ncontent, links and the structure the information of the Wikipedia articles. The\ndataset has been further enriched with about 25% more links and selected\npartitions published as Linked Data. Finally, we describe the maintenance and\nsustainability plans, and selected use cases of the dataset from the TextExt\nknowledge extraction challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10408v1",
            "date": "2018-12-11 09:48:17+00:00",
            "title": "Hyperbolic Deep Learning for Chinese Natural Language Understanding",
            "authors": [
                "Marko Valentin Micic",
                "Hugo Chu"
            ],
            "abstract": "Recently hyperbolic geometry has proven to be effective in building\nembeddings that encode hierarchical and entailment information. This makes it\nparticularly suited to modelling the complex asymmetrical relationships between\nChinese characters and words. In this paper we first train a large scale\nhyperboloid skip-gram model on a Chinese corpus, then apply the character\nembeddings to a downstream hyperbolic Transformer model derived from the\nprinciples of gyrovector space for Poincare disk model. In our experiments the\ncharacter-based Transformer outperformed its word-based Euclidean equivalent.\nTo the best of our knowledge, this is the first time in Chinese NLP that a\ncharacter-based model outperformed its word-based counterpart, allowing the\ncircumvention of the challenging and domain-dependent task of Chinese Word\nSegmentation (CWS).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10411v2",
            "date": "2018-12-15 01:04:18+00:00",
            "title": "Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languages",
            "authors": [
                "Siddique Latif",
                "Adnan Qayyum",
                "Muhammad Usman",
                "Junaid Qadir"
            ],
            "abstract": "Cross-lingual speech emotion recognition is an important task for practical\napplications. The performance of automatic speech emotion recognition systems\ndegrades in cross-corpus scenarios, particularly in scenarios involving\nmultiple languages or a previously unseen language such as Urdu for which\nlimited or no data is available. In this study, we investigate the problem of\ncross-lingual emotion recognition for Urdu language and contribute URDU---the\nfirst ever spontaneous Urdu-language speech emotion database. Evaluations are\nperformed using three different Western languages against Urdu and experimental\nresults on different possible scenarios suggest various interesting aspects for\ndesigning more adaptive emotion recognition system for such limited languages.\nIn results, selecting training instances of multiple languages can deliver\ncomparable results to baseline and augmentation a fraction of testing language\ndata while training can help to boost accuracy for speech emotion recognition.\nURDU data is publicly available for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10604v1",
            "date": "2018-12-27 03:03:51+00:00",
            "title": "Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction",
            "authors": [
                "Yujin Yuan",
                "Liyuan Liu",
                "Siliang Tang",
                "Zhongfei Zhang",
                "Yueting Zhuang",
                "Shiliang Pu",
                "Fei Wu",
                "Xiang Ren"
            ],
            "abstract": "Distant supervision leverages knowledge bases to automatically label\ninstances, thus allowing us to train relation extractor without human\nannotations. However, the generated training data typically contain massive\nnoise, and may result in poor performances with the vanilla supervised\nlearning. In this paper, we propose to conduct multi-instance learning with a\nnovel Cross-relation Cross-bag Selective Attention (C$^2$SA), which leads to\nnoise-robust training for distant supervised relation extractor. Specifically,\nwe employ the sentence-level selective attention to reduce the effect of noisy\nor mismatched sentences, while the correlation among relations were captured to\nimprove the quality of attention weights. Moreover, instead of treating all\nentity-pairs equally, we try to pay more attention to entity-pairs with a\nhigher quality. Similarly, we adopt the selective attention mechanism to\nachieve this goal. Experiments with two types of relation extractor demonstrate\nthe superiority of the proposed approach over the state-of-the-art, while\nfurther ablation studies verify our intuitions and demonstrate the\neffectiveness of our proposed two techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10628v2",
            "date": "2018-12-27 05:14:49+00:00",
            "title": "Intent Detection and Slots Prompt in a Closed-Domain Chatbot",
            "authors": [
                "Amber Nigam",
                "Prashik Sahare",
                "Kushagra Pandya"
            ],
            "abstract": "In this paper, we introduce a methodology for predicting intent and slots of\na query for a chatbot that answers career-related queries. We take a\nmulti-staged approach where both the processes (intent-classification and\nslot-tagging) inform each other's decision-making in different stages. The\nmodel breaks down the problem into stages, solving one problem at a time and\npassing on relevant results of the current stage to the next, thereby reducing\nsearch space for subsequent stages, and eventually making classification and\ntagging more viable after each stage. We also observe that relaxing rules for a\nfuzzy entity-matching in slot-tagging after each stage (by maintaining a\nseparate Named Entity Tagger per stage) helps us improve performance, although\nat a slight cost of false-positives. Our model has achieved state-of-the-art\nperformance with F1-score of 77.63% for intent-classification and 82.24% for\nslot-tagging on our dataset that we would publicly release along with the\npaper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10720v1",
            "date": "2018-12-27 13:36:41+00:00",
            "title": "QRFA: A Data-Driven Model of Information-Seeking Dialogues",
            "authors": [
                "Svitlana Vakulenko",
                "Kate Revoredo",
                "Claudio Di Ciccio",
                "Maarten de Rijke"
            ],
            "abstract": "Understanding the structure of interaction processes helps us to improve\ninformation-seeking dialogue systems. Analyzing an interaction process boils\ndown to discovering patterns in sequences of alternating utterances exchanged\nbetween a user and an agent. Process mining techniques have been successfully\napplied to analyze structured event logs, discovering the underlying process\nmodels or evaluating whether the observed behavior is in conformance with the\nknown process. In this paper, we apply process mining techniques to discover\npatterns in conversational transcripts and extract a new model of\ninformation-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our\nresults are grounded in an empirical evaluation across multiple conversational\ndatasets from different domains, which was never attempted before. We show that\nthe QRFA model better reflects conversation flows observed in real\ninformation-seeking conversations than models proposed previously. Moreover,\nQRFA allows us to identify malfunctioning in dialogue system transcripts as\ndeviations from the expected conversation flow described by the model via\nconformance analysis.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.10757v1",
            "date": "2018-12-27 16:17:30+00:00",
            "title": "Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize",
            "authors": [
                "Chandra Khatri",
                "Behnam Hedayatnia",
                "Anu Venkatesh",
                "Jeff Nunn",
                "Yi Pan",
                "Qing Liu",
                "Han Song",
                "Anna Gottardi",
                "Sanjeev Kwatra",
                "Sanju Pancholi",
                "Ming Cheng",
                "Qinglang Chen",
                "Lauren Stubel",
                "Karthik Gopalakrishnan",
                "Kate Bland",
                "Raefer Gabriel",
                "Arindam Mandal",
                "Dilek Hakkani-Tur",
                "Gene Hwang",
                "Nate Michel",
                "Eric King",
                "Rohit Prasad"
            ],
            "abstract": "Building open domain conversational systems that allow users to have engaging\nconversations on topics of their choice is a challenging task. Alexa Prize was\nlaunched in 2016 to tackle the problem of achieving natural, sustained,\ncoherent and engaging open-domain dialogs. In the second iteration of the\ncompetition in 2018, university teams advanced the state of the art by using\ncontext in dialog models, leveraging knowledge graphs for language\nunderstanding, handling complex utterances, building statistical and\nhierarchical dialog managers, and leveraging model-driven signals from user\nresponses. The 2018 competition also included the provision of a suite of tools\nand models to the competitors including the CoBot (conversational bot) toolkit,\ntopic and dialog act detection models, conversation evaluators, and a sensitive\ncontent detection model so that the competing teams could focus on building\nknowledge-rich, coherent and engaging multi-turn dialog systems. This paper\noutlines the advances developed by the university teams as well as the Alexa\nPrize team to achieve the common goal of advancing the science of\nConversational AI. We address several key open-ended problems such as\nconversational speech recognition, open domain natural language understanding,\ncommonsense reasoning, statistical dialog management, and dialog evaluation.\nThese collaborative efforts have driven improved experiences by Alexa users to\nan average rating of 3.61, the median duration of 2 mins 18 seconds, and\naverage turns to 14.6, increases of 14%, 92%, 54% respectively since the launch\nof the 2018 competition. For conversational speech recognition, we have\nimproved our relative Word Error Rate by 55% and our relative Entity Error Rate\nby 34% since the launch of the Alexa Prize. Socialbots improved in quality\nsignificantly more rapidly in 2018, in part due to the release of the CoBot\ntoolkit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10847v1",
            "date": "2018-12-27 23:42:06+00:00",
            "title": "The Clickbait Challenge 2017: Towards a Regression Model for Clickbait Strength",
            "authors": [
                "Martin Potthast",
                "Tim Gollub",
                "Matthias Hagen",
                "Benno Stein"
            ],
            "abstract": "Clickbait has grown to become a nuisance to social media users and social\nmedia operators alike. Malicious content publishers misuse social media to\nmanipulate as many users as possible to visit their websites using clickbait\nmessages. Machine learning technology may help to handle this problem, giving\nrise to automatic clickbait detection. To accelerate progress in this\ndirection, we organized the Clickbait Challenge 2017, a shared task inviting\nthe submission of clickbait detectors for a comparative evaluation. A total of\n13 detectors have been submitted, achieving significant improvements over the\nprevious state of the art in terms of detection performance. Also, many of the\nsubmitted approaches have been published open source, rendering them\nreproducible, and a good starting point for newcomers. While the 2017 challenge\nhas passed, we maintain the evaluation system and answer to new registrations\nin support of the ongoing research on better clickbait detectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10991v1",
            "date": "2018-12-28 13:49:49+00:00",
            "title": "The role of grammar in transition-probabilities of subsequent words in English text",
            "authors": [
                "Rudolf Hanel",
                "Stefan Thurner"
            ],
            "abstract": "Sentence formation is a highly structured, history-dependent, and\nsample-space reducing (SSR) process. While the first word in a sentence can be\nchosen from the entire vocabulary, typically, the freedom of choosing\nsubsequent words gets more and more constrained by grammar and context, as the\nsentence progresses. This sample-space reducing property offers a natural\nexplanation of Zipf's law in word frequencies, however, it fails to capture the\nstructure of the word-to-word transition probability matrices of English text.\nHere we adopt the view that grammatical constraints (such as\nsubject--predicate--object) locally re-order the word order in sentences that\nare sampled with a SSR word generation process. We demonstrate that\nsuperimposing grammatical structure -- as a local word re-ordering\n(permutation) process -- on a sample-space reducing process is sufficient to\nexplain both, word frequencies and word-to-word transition probabilities. We\ncompare the quality of the grammatically ordered SSR model in reproducing\nseveral test statistics of real texts with other text generation models, such\nas the Bernoulli model, the Simon model, and the Monkey typewriting model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11275v1",
            "date": "2018-12-29 03:32:09+00:00",
            "title": "End-to-end neural relation extraction using deep biaffine attention",
            "authors": [
                "Dat Quoc Nguyen",
                "Karin Verspoor"
            ],
            "abstract": "We propose a neural network model for joint extraction of named entities and\nrelations between them, without any hand-crafted features. The key contribution\nof our model is to extend a BiLSTM-CRF-based entity recognition model with a\ndeep biaffine attention layer to model second-order interactions between latent\nfeatures for relation classification, specifically attending to the role of an\nentity in a directional relationship. On the benchmark \"relation and entity\nrecognition\" dataset CoNLL04, experimental results show that our model\noutperforms previous models, producing new state-of-the-art performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11321v1",
            "date": "2018-12-29 09:34:23+00:00",
            "title": "Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction",
            "authors": [
                "Ningyu Zhang",
                "Shumin Deng",
                "Zhanlin Sun",
                "Xi Chen",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "A capsule is a group of neurons, whose activity vector represents the\ninstantiation parameters of a specific type of entity. In this paper, we\nexplore the capsule networks used for relation extraction in a multi-instance\nmulti-label learning framework and propose a novel neural approach based on\ncapsule networks with attention mechanisms. We evaluate our method with\ndifferent benchmarks, and it is demonstrated that our method improves the\nprecision of the predicted relations. Particularly, we show that capsule\nnetworks improve multiple entity pairs relation extraction.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.11709v1",
            "date": "2018-12-31 07:09:59+00:00",
            "title": "Cross-language Citation Recommendation via Hierarchical Representation Learning on Heterogeneous Graph",
            "authors": [
                "Zhuoren Jiang",
                "Yue Yin",
                "Liangcai Gao",
                "Yao Lu",
                "Xiaozhong Liu"
            ],
            "abstract": "While the volume of scholarly publications has increased at a frenetic pace,\naccessing and consuming the useful candidate papers, in very large digital\nlibraries, is becoming an essential and challenging task for scholars.\nUnfortunately, because of language barrier, some scientists (especially the\njunior ones or graduate students who do not master other languages) cannot\nefficiently locate the publications hosted in a foreign language repository. In\nthis study, we propose a novel solution, cross-language citation recommendation\nvia Hierarchical Representation Learning on Heterogeneous Graph (HRLHG), to\naddress this new problem. HRLHG can learn a representation function by mapping\nthe publications, from multilingual repositories, to a low-dimensional joint\nembedding space from various kinds of vertexes and relations on a heterogeneous\ngraph. By leveraging both global (task specific) plus local (task independent)\ninformation as well as a novel supervised hierarchical random walk algorithm,\nthe proposed method can optimize the publication representations by maximizing\nthe likelihood of locating the important cross-language neighborhoods on the\ngraph. Experiment results show that the proposed method can not only outperform\nstate-of-the-art baseline models, but also improve the interpretability of the\nrepresentation model for cross-language citation recommendation task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.00056v2",
            "date": "2018-12-31 22:05:05+00:00",
            "title": "Entity Synonym Discovery via Multipiece Bilateral Context Matching",
            "authors": [
                "Chenwei Zhang",
                "Yaliang Li",
                "Nan Du",
                "Wei Fan",
                "Philip S. Yu"
            ],
            "abstract": "Being able to automatically discover synonymous entities in an open-world\nsetting benefits various tasks such as entity disambiguation or knowledge graph\ncanonicalization. Existing works either only utilize entity features, or rely\non structured annotations from a single piece of context where the entity is\nmentioned. To leverage diverse contexts where entities are mentioned, in this\npaper, we generalize the distributional hypothesis to a multi-context setting\nand propose a synonym discovery framework that detects entity synonyms from\nfree-text corpora with considerations on effectiveness and robustness. As one\nof the key components in synonym discovery, we introduce a neural network model\nSYNONYMNET to determine whether or not two given entities are synonym with each\nother. Instead of using entities features, SYNONYMNET makes use of multiple\npieces of contexts in which the entity is mentioned, and compares the\ncontext-level similarity via a bilateral matching schema. Experimental results\ndemonstrate that the proposed model is able to detect synonym sets that are not\nobserved during training on both generic and domain-specific datasets:\nWiki+Freebase, PubMed+UMLS, and MedBook+MKG, with up to 4.16% improvement in\nterms of Area Under the Curve and 3.19% in terms of Mean Average Precision\ncompared to the best baseline method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00388v2",
            "date": "2018-01-01 03:43:30+00:00",
            "title": "Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases",
            "authors": [
                "Walid Shalaby",
                "Wlodek Zadrozny",
                "Hongxia Jin"
            ],
            "abstract": "Text representations using neural word embeddings have proven effective in\nmany NLP applications. Recent researches adapt the traditional word embedding\nmodels to learn vectors of multiword expressions (concepts/entities). However,\nthese methods are limited to textual knowledge bases (e.g., Wikipedia). In this\npaper, we propose a novel and simple technique for integrating the knowledge\nabout concepts from two large scale knowledge bases of different structure\n(Wikipedia and Probase) in order to learn concept representations. We adapt the\nefficient skip-gram model to seamlessly learn from the knowledge in Wikipedia\ntext and Probase concept graph. We evaluate our concept embedding models on two\ntasks: (1) analogical reasoning, where we achieve a state-of-the-art\nperformance of 91% on semantic analogies, (2) concept categorization, where we\nachieve a state-of-the-art performance on two benchmark datasets achieving\ncategorization accuracy of 100% on one and 98% on the other. Additionally, we\npresent a case study to evaluate our model on unsupervised argument type\nidentification for neural semantic parsing. We demonstrate the competitive\naccuracy of our unsupervised method and its ability to better generalize to out\nof vocabulary entity mentions compared to the tedious and error prone methods\nwhich depend on gazetteers and regular expressions.",
            "journal ref": "Inf Retrieval J (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.00632v2",
            "date": "2018-01-02 12:50:12+00:00",
            "title": "Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes",
            "authors": [
                "Cedric De Boom",
                "Thomas Demeester",
                "Bart Dhoedt"
            ],
            "abstract": "Recurrent neural networks are nowadays successfully used in an abundance of\napplications, going from text, speech and image processing to recommender\nsystems. Backpropagation through time is the algorithm that is commonly used to\ntrain these networks on specific tasks. Many deep learning frameworks have\ntheir own implementation of training and sampling procedures for recurrent\nneural networks, while there are in fact multiple other possibilities to choose\nfrom and other parameters to tune. In existing literature this is very often\noverlooked or ignored. In this paper we therefore give an overview of possible\ntraining and sampling schemes for character-level recurrent neural networks to\nsolve the task of predicting the next token in a given sequence. We test these\ndifferent schemes on a variety of datasets, neural network architectures and\nparameter settings, and formulate a number of take-home recommendations. The\nchoice of training and sampling scheme turns out to be subject to a number of\ntrade-offs, such as training stability, sampling time, model performance and\nimplementation effort, but is largely independent of the data. Perhaps the most\nsurprising result is that transferring hidden states for correctly initializing\nthe model on subsequences often leads to unstable training behavior depending\non the dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.00841v1",
            "date": "2018-01-02 21:29:41+00:00",
            "title": "Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer",
            "authors": [
                "Kanishka Rao",
                "Ha\u015fim Sak",
                "Rohit Prabhavalkar"
            ],
            "abstract": "We investigate training end-to-end speech recognition models with the\nrecurrent neural network transducer (RNN-T): a streaming, all-neural,\nsequence-to-sequence architecture which jointly learns acoustic and language\nmodel components from transcribed acoustic data. We explore various model\narchitectures and demonstrate how the model can be improved further if\nadditional text or pronunciation data are available. The model consists of an\n`encoder', which is initialized from a connectionist temporal\nclassification-based (CTC) acoustic model, and a `decoder' which is partially\ninitialized from a recurrent neural network language model trained on text data\nalone. The entire neural network is trained with the RNN-T loss and directly\noutputs the recognized transcript as a sequence of graphemes, thus performing\nend-to-end speech recognition. We find that performance can be improved further\nthrough the use of sub-word units (`wordpieces') which capture longer context\nand significantly reduce substitution errors. The best RNN-T system, a\ntwelve-layer LSTM encoder with a two-layer LSTM decoder trained with 30,000\nwordpieces as output targets achieves a word error rate of 8.5\\% on\nvoice-search and 5.2\\% on voice-dictation tasks and is comparable to a\nstate-of-the-art baseline at 8.3\\% on voice-search and 5.4\\% voice-dictation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.01102v2",
            "date": "2018-01-03 18:23:41+00:00",
            "title": "Social Media Analysis based on Semanticity of Streaming and Batch Data",
            "authors": [
                "Barathi Ganesh HB"
            ],
            "abstract": "Languages shared by people differ in different regions based on their\naccents, pronunciation and word usages. In this era sharing of language takes\nplace mainly through social media and blogs. Every second swing of such a micro\nposts exist which induces the need of processing those micro posts, in-order to\nextract knowledge out of it. Knowledge extraction differs with respect to the\napplication in which the research on cognitive science fed the necessities for\nthe same. This work further moves forward such a research by extracting\nsemantic information of streaming and batch data in applications like Named\nEntity Recognition and Author Profiling. In the case of Named Entity\nRecognition context of a single micro post has been utilized and context that\nlies in the pool of micro posts were utilized to identify the sociolect aspects\nof the author of those micro posts. In this work Conditional Random Field has\nbeen utilized to do the entity recognition and a novel approach has been\nproposed to find the sociolect aspects of the author (Gender, Age group).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02243v1",
            "date": "2018-01-07 20:26:00+00:00",
            "title": "Trading the Twitter Sentiment with Reinforcement Learning",
            "authors": [
                "Catherine Xiao",
                "Wanfeng Chen"
            ],
            "abstract": "This paper is to explore the possibility to use alternative data and\nartificial intelligence techniques to trade stocks. The efficacy of the daily\nTwitter sentiment on predicting the stock return is examined using machine\nlearning methods. Reinforcement learning(Q-learning) is applied to generate the\noptimal trading policy based on the sentiment signal. The predicting power of\nthe sentiment signal is more significant if the stock price is driven by the\nexpectation of the company growth and when the company has a major event that\ndraws the public attention. The optimal trading strategy based on reinforcement\nlearning outperforms the trading strategy based on the machine learning\nprediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1801.02668v2",
            "date": "2018-01-08 20:07:35+00:00",
            "title": "Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time",
            "authors": [
                "Ting-Hao 'Kenneth' Huang",
                "Joseph Chee Chang",
                "Jeffrey P. Bigham"
            ],
            "abstract": "Crowd-powered conversational assistants have been shown to be more robust\nthan automated systems, but do so at the cost of higher response latency and\nmonetary costs. A promising direction is to combine the two approaches for high\nquality, low latency, and low cost solutions. In this paper, we introduce\nEvorus, a crowd-powered conversational assistant built to automate itself over\ntime by (i) allowing new chatbots to be easily integrated to automate more\nscenarios, (ii) reusing prior crowd answers, and (iii) learning to\nautomatically approve response candidates. Our 5-month-long deployment with 80\nparticipants and 281 conversations shows that Evorus can automate itself\nwithout compromising conversation quality. Crowd-AI architectures have long\nbeen proposed as a way to reduce cost and latency for crowd-powered systems;\nEvorus demonstrates how automation can be introduced successfully in a deployed\nsystem. Its architecture allows future researchers to make further innovation\non the underlying automated components in the context of a deployed open domain\ndialog system.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1801.02808v1",
            "date": "2018-01-09 06:11:50+00:00",
            "title": "Lifelong Learning for Sentiment Classification",
            "authors": [
                "Zhiyuan Chen",
                "Nianzu Ma",
                "Bing Liu"
            ],
            "abstract": "This paper proposes a novel lifelong learning (LL) approach to sentiment\nclassification. LL mimics the human continuous learning process, i.e.,\nretaining the knowledge learned from past tasks and use it to help future\nlearning. In this paper, we first discuss LL in general and then LL for\nsentiment classification in particular. The proposed LL approach adopts a\nBayesian optimization framework based on stochastic gradient descent. Our\nexperimental results show that the proposed method outperforms baseline methods\nsignificantly, which demonstrates that lifelong learning is a promising\nresearch direction.",
            "journal ref": "ACL 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.02832v1",
            "date": "2018-01-09 08:23:46+00:00",
            "title": "Biomedical Question Answering via Weighted Neural Network Passage Retrieval",
            "authors": [
                "Ferenc Galk\u00f3",
                "Carsten Eickhoff"
            ],
            "abstract": "The amount of publicly available biomedical literature has been growing\nrapidly in recent years, yet question answering systems still struggle to\nexploit the full potential of this source of data. In a preliminary processing\nstep, many question answering systems rely on retrieval models for identifying\nrelevant documents and passages. This paper proposes a weighted cosine distance\nretrieval scheme based on neural network word embeddings. Our experiments are\nbased on publicly available data and tasks from the BioASQ biomedical question\nanswering challenge and demonstrate significant performance gains over a wide\nrange of state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.03032v1",
            "date": "2018-01-09 17:00:24+00:00",
            "title": "Topical Stance Detection for Twitter: A Two-Phase LSTM Model Using Attention",
            "authors": [
                "Kuntal Dey",
                "Ritvik Shrivastava",
                "Saroj Kaushik"
            ],
            "abstract": "The topical stance detection problem addresses detecting the stance of the\ntext content with respect to a given topic: whether the sentiment of the given\ntext content is in FAVOR of (positive), is AGAINST (negative), or is NONE\n(neutral) towards the given topic. Using the concept of attention, we develop a\ntwo-phase solution. In the first phase, we classify subjectivity - whether a\ngiven tweet is neutral or subjective with respect to the given topic. In the\nsecond phase, we classify sentiment of the subjective tweets (ignoring the\nneutral tweets) - whether a given subjective tweet has a FAVOR or AGAINST\nstance towards the topic. We propose a Long Short-Term memory (LSTM) based deep\nneural network for each phase, and embed attention at each of the phases. On\nthe SemEval 2016 stance detection Twitter task dataset, we obtain a best-case\nmacro F-score of 68.84% and a best-case accuracy of 60.2%, outperforming the\nexisting deep learning based solutions. Our framework, T-PAN, is the first in\nthe topical stance detection literature, that uses deep learning within a\ntwo-phase architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03911v2",
            "date": "2018-01-11 18:24:02+00:00",
            "title": "Stochastic Learning of Nonstationary Kernels for Natural Language Modeling",
            "authors": [
                "Sahil Garg",
                "Greg Ver Steeg",
                "Aram Galstyan"
            ],
            "abstract": "Natural language processing often involves computations with semantic or\nsyntactic graphs to facilitate sophisticated reasoning based on structural\nrelationships. While convolution kernels provide a powerful tool for comparing\ngraph structure based on node (word) level relationships, they are difficult to\ncustomize and can be computationally expensive. We propose a generalization of\nconvolution kernels, with a nonstationary model, for better expressibility of\nnatural languages in supervised settings. For a scalable learning of the\nparameters introduced with our model, we propose a novel algorithm that\nleverages stochastic sampling on k-nearest neighbor graphs, along with\napproximations based on locality-sensitive hashing. We demonstrate the\nadvantages of our approach on a challenging real-world (structured inference)\nproblem of automatically extracting biological models from the text of\nscientific papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04354v5",
            "date": "2018-01-13 00:42:30+00:00",
            "title": "Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers",
            "authors": [
                "Ji Gao",
                "Jack Lanchantin",
                "Mary Lou Soffa",
                "Yanjun Qi"
            ],
            "abstract": "Although various techniques have been proposed to generate adversarial\nsamples for white-box attacks on text, little attention has been paid to\nblack-box attacks, which are more realistic scenarios. In this paper, we\npresent a novel algorithm, DeepWordBug, to effectively generate small text\nperturbations in a black-box setting that forces a deep-learning classifier to\nmisclassify a text input. We employ novel scoring strategies to identify the\ncritical tokens that, if modified, cause the classifier to make an incorrect\nprediction. Simple character-level transformations are applied to the\nhighest-ranked tokens in order to minimize the edit distance of the\nperturbation, yet change the original classification. We evaluated DeepWordBug\non eight real-world text datasets, including text classification, sentiment\nanalysis, and spam detection. We compare the result of DeepWordBug with two\nbaselines: Random (Black-box) and Gradient (White-box). Our experimental\nresults indicate that DeepWordBug reduces the prediction accuracy of current\nstate-of-the-art deep-learning models, including a decrease of 68\\% on average\nfor a Word-LSTM model and 48\\% on average for a Char-CNN model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04433v1",
            "date": "2018-01-13 12:58:43+00:00",
            "title": "Detecting Offensive Language in Tweets Using Deep Learning",
            "authors": [
                "Georgios K. Pitsilis",
                "Heri Ramampiaro",
                "Helge Langseth"
            ],
            "abstract": "This paper addresses the important problem of discerning hateful content in\nsocial media. We propose a detection scheme that is an ensemble of Recurrent\nNeural Network (RNN) classifiers, and it incorporates various features\nassociated with user-related information, such as the users' tendency towards\nracism or sexism. These data are fed as input to the above classifiers along\nwith the word frequency vectors derived from the textual content. Our approach\nhas been evaluated on a publicly available corpus of 16k tweets, and the\nresults demonstrate its effectiveness in comparison to existing state of the\nart solutions. More specifically, our scheme can successfully distinguish\nracism and sexism messages from normal text, and achieve higher classification\nquality than current state-of-the-art algorithms.",
            "journal ref": "Applied Intelligence, 48(12), 4730-4742 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04554v1",
            "date": "2018-01-14 13:28:19+00:00",
            "title": "DCDistance: A Supervised Text Document Feature extraction based on class labels",
            "authors": [
                "Charles Henrique Porto Ferreira",
                "Debora Maria Rossi de Medeiros",
                "Fabricio Olivetti de Fran\u00e7a"
            ],
            "abstract": "Text Mining is a field that aims at extracting information from textual data.\nOne of the challenges of such field of study comes from the pre-processing\nstage in which a vector (and structured) representation should be extracted\nfrom unstructured data. The common extraction creates large and sparse vectors\nrepresenting the importance of each term to a document. As such, this usually\nleads to the curse-of-dimensionality that plagues most machine learning\nalgorithms. To cope with this issue, in this paper we propose a new supervised\nfeature extraction and reduction algorithm, named DCDistance, that creates\nfeatures based on the distance between a document to a representative of each\nclass label. As such, the proposed technique can reduce the features set in\nmore than 99% of the original set. Additionally, this algorithm was also\ncapable of improving the classification accuracy over a set of benchmark\ndatasets when compared to traditional and state-of-the-art features selection\nalgorithms.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.04813v1",
            "date": "2018-01-15 14:11:57+00:00",
            "title": "Predicting Movie Genres Based on Plot Summaries",
            "authors": [
                "Quan Hoang"
            ],
            "abstract": "This project explores several Machine Learning methods to predict movie\ngenres based on plot summaries. Naive Bayes, Word2Vec+XGBoost and Recurrent\nNeural Networks are used for text classification, while K-binary\ntransformation, rank method and probabilistic classification with learned\nprobability threshold are employed for the multi-label problem involved in the\ngenre tagging task.Experiments with more than 250,000 movies show that\nemploying the Gated Recurrent Units (GRU) neural networks for the probabilistic\nclassification with learned probability threshold approach achieves the best\nresult on the test set. The model attains a Jaccard Index of 50.0%, a F-score\nof 0.56, and a hit rate of 80.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.04958v1",
            "date": "2018-01-15 19:23:21+00:00",
            "title": "Topic Modeling on Health Journals with Regularized Variational Inference",
            "authors": [
                "Robert Giaquinto",
                "Arindam Banerjee"
            ],
            "abstract": "Topic modeling enables exploration and compact representation of a corpus.\nThe CaringBridge (CB) dataset is a massive collection of journals written by\npatients and caregivers during a health crisis. Topic modeling on the CB\ndataset, however, is challenging due to the asynchronous nature of multiple\nauthors writing about their health journeys. To overcome this challenge we\nintroduce the Dynamic Author-Persona topic model (DAP), a probabilistic\ngraphical model designed for temporal corpora with multiple authors. The\nnovelty of the DAP model lies in its representation of authors by a persona ---\nwhere personas capture the propensity to write about certain topics over time.\nFurther, we present a regularized variational inference algorithm, which we use\nto encourage the DAP model's personas to be distinct. Our results show\nsignificant improvements over competing topic models --- particularly after\nregularization, and highlight the DAP model's unique ability to capture common\njourneys shared by different authors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05096v1",
            "date": "2018-01-16 02:05:50+00:00",
            "title": "Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification",
            "authors": [
                "Komei Sugiura",
                "Hisashi Kawai"
            ],
            "abstract": "The target task of this study is grounded language understanding for domestic\nservice robots (DSRs). In particular, we focus on instruction understanding for\nshort sentences where verbs are missing. This task is of critical importance to\nbuild communicative DSRs because manipulation is essential for DSRs. Existing\ninstruction understanding methods usually estimate missing information only\nfrom non-grounded knowledge; therefore, whether the predicted action is\nphysically executable or not was unclear.\n  In this paper, we present a grounded instruction understanding method to\nestimate appropriate objects given an instruction and situation. We extend the\nGenerative Adversarial Nets (GAN) and build a GAN-based classifier using latent\nrepresentations. To quantitatively evaluate the proposed method, we have\ndeveloped a data set based on the standard data set used for Visual QA.\nExperimental results have shown that the proposed method gives the better\nresult than baseline methods.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1801.05164v3",
            "date": "2018-01-16 09:09:58+00:00",
            "title": "Embedding a $\u03b8$-invariant code into a complete one",
            "authors": [
                "Jean N\u00e9raud",
                "Carla Selmi"
            ],
            "abstract": "Let A be a finite or countable alphabet and let $\\theta$ be a literal\n(anti-)automorphism onto A * (by definition, such a correspondence is\ndeterminated by a permutation of the alphabet). This paper deals with sets\nwhich are invariant under $\\theta$ ($\\theta$-invariant for short) that is,\nlanguages L such that $\\theta$ (L) is a subset of L.We establish an extension\nof the famous defect theorem. With regards to the so-called notion of\ncompleteness, we provide a series of examples of finite complete\n$\\theta$-invariant codes. Moreover, we establish a formula which allows to\nembed any non-complete $\\theta$-invariant code into a complete one. As a\nconsequence, in the family of the so-called thin $\\theta$--invariant codes,\nmaximality and completeness are two equivalent notions.",
            "journal ref": "Theoretical Computer Science, Elsevier",
            "category": "cs.DM"
        },
        {
            "url": "http://arxiv.org/abs/1801.05453v2",
            "date": "2018-01-16 19:21:48+00:00",
            "title": "Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs",
            "authors": [
                "W. James Murdoch",
                "Peter J. Liu",
                "Bin Yu"
            ],
            "abstract": "The driving force behind the recent success of LSTMs has been their ability\nto learn complex and non-linear relationships. Consequently, our inability to\ndescribe these relationships has led to LSTMs being characterized as black\nboxes. To this end, we introduce contextual decomposition (CD), an\ninterpretation algorithm for analysing individual predictions made by standard\nLSTMs, without any changes to the underlying model. By decomposing the output\nof a LSTM, CD captures the contributions of combinations of words or variables\nto the final prediction of an LSTM. On the task of sentiment analysis with the\nYelp and SST data sets, we show that CD is able to reliably identify words and\nphrases of contrasting sentiment, and how they are combined to yield the LSTM's\nfinal prediction. Using the phrase-level labels in SST, we also demonstrate\nthat CD is able to successfully extract positive and negative negations from an\nLSTM, something which has not previously been done.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05617v1",
            "date": "2018-01-17 10:38:20+00:00",
            "title": "Automatic Detection of Cyberbullying in Social Media Text",
            "authors": [
                "Cynthia Van Hee",
                "Gilles Jacobs",
                "Chris Emmery",
                "Bart Desmet",
                "Els Lefever",
                "Ben Verhoeven",
                "Guy De Pauw",
                "Walter Daelemans",
                "V\u00e9ronique Hoste"
            ],
            "abstract": "While social media offer great communication opportunities, they also\nincrease the vulnerability of young people to threatening situations online.\nRecent studies report that cyberbullying constitutes a growing problem among\nyoungsters. Successful prevention depends on the adequate detection of\npotentially harmful messages and the information overload on the Web requires\nintelligent systems to identify potential risks automatically. The focus of\nthis paper is on automatic cyberbullying detection in social media text by\nmodelling posts written by bullies, victims, and bystanders of online bullying.\nWe describe the collection and fine-grained annotation of a training corpus for\nEnglish and Dutch and perform a series of binary classification experiments to\ndetermine the feasibility of automatic cyberbullying detection. We make use of\nlinear support vector machines exploiting a rich feature set and investigate\nwhich information sources contribute the most for this particular task.\nExperiments on a holdout test set reveal promising results for the detection of\ncyberbullying-related posts. After optimisation of the hyperparameters, the\nclassifier yields an F1-score of 64% and 61% for English and Dutch\nrespectively, and considerably outperforms baseline systems based on keywords\nand word unigrams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06024v1",
            "date": "2018-01-18 14:10:37+00:00",
            "title": "Natural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations",
            "authors": [
                "Gino Brunner",
                "Yuyi Wang",
                "Roger Wattenhofer",
                "Michael Weigelt"
            ],
            "abstract": "We train multi-task autoencoders on linguistic tasks and analyze the learned\nhidden sentence representations. The representations change significantly when\ntranslation and part-of-speech decoders are added. The more decoders a model\nemploys, the better it clusters sentences according to their syntactic\nsimilarity, as the representation space becomes less entangled. We explore the\nstructure of the representation space by interpolating between sentences, which\nyields interesting pseudo-English sentences, many of which have recognizable\nsyntactic structure. Lastly, we point out an interesting property of our\nmodels: The difference-vector between two sentences can be added to change a\nthird sentence with similar features in a meaningful way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06146v5",
            "date": "2018-01-18 17:54:52+00:00",
            "title": "Universal Language Model Fine-tuning for Text Classification",
            "authors": [
                "Jeremy Howard",
                "Sebastian Ruder"
            ],
            "abstract": "Inductive transfer learning has greatly impacted computer vision, but\nexisting approaches in NLP still require task-specific modifications and\ntraining from scratch. We propose Universal Language Model Fine-tuning\n(ULMFiT), an effective transfer learning method that can be applied to any task\nin NLP, and introduce techniques that are key for fine-tuning a language model.\nOur method significantly outperforms the state-of-the-art on six text\nclassification tasks, reducing the error by 18-24% on the majority of datasets.\nFurthermore, with only 100 labeled examples, it matches the performance of\ntraining from scratch on 100x more data. We open-source our pretrained models\nand code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06176v3",
            "date": "2018-01-18 18:57:33+00:00",
            "title": "Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning",
            "authors": [
                "Baolin Peng",
                "Xiujun Li",
                "Jianfeng Gao",
                "Jingjing Liu",
                "Kam-Fai Wong",
                "Shang-Yu Su"
            ],
            "abstract": "Training a task-completion dialogue agent via reinforcement learning (RL) is\ncostly because it requires many interactions with real users. One common\nalternative is to use a user simulator. However, a user simulator usually lacks\nthe language complexity of human interlocutors and the biases in its design may\ntend to degrade the agent. To address these issues, we present Deep Dyna-Q,\nwhich to our knowledge is the first deep RL framework that integrates planning\nfor task-completion dialogue policy learning. We incorporate into the dialogue\nagent a model of the environment, referred to as the world model, to mimic real\nuser response and generate simulated experience. During dialogue policy\nlearning, the world model is constantly updated with real user experience to\napproach real user behavior, and in turn, the dialogue agent is optimized using\nboth real experience and simulated experience. The effectiveness of our\napproach is demonstrated on a movie-ticket booking task in both simulated and\nhuman-in-the-loop settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06287v1",
            "date": "2018-01-19 04:02:04+00:00",
            "title": "What Does a TextCNN Learn?",
            "authors": [
                "Linyuan Gong",
                "Ruyi Ji"
            ],
            "abstract": "TextCNN, the convolutional neural network for text, is a useful deep learning\nalgorithm for sentence classification tasks such as sentiment analysis and\nquestion classification. However, neural networks have long been known as black\nboxes because interpreting them is a challenging task. Researchers have\ndeveloped several tools to understand a CNN for image classification by deep\nvisualization, but research about deep TextCNNs is still insufficient. In this\npaper, we are trying to understand what a TextCNN learns on two classical NLP\ndatasets. Our work focuses on functions of different convolutional kernels and\ncorrelations between convolutional kernels.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1801.06294v5",
            "date": "2018-01-19 05:04:21+00:00",
            "title": "Multi-Task Pharmacovigilance Mining from Social Media Posts",
            "authors": [
                "Shaika Chowdhury",
                "Chenwei Zhang",
                "Philip S. Yu"
            ],
            "abstract": "Social media has grown to be a crucial information source for\npharmacovigilance studies where an increasing number of people post adverse\nreactions to medical drugs that are previously unreported. Aiming to\neffectively monitor various aspects of Adverse Drug Reactions (ADRs) from\ndiversely expressed social medical posts, we propose a multi-task neural\nnetwork framework that learns several tasks associated with ADR monitoring with\ndifferent levels of supervisions collectively. Besides being able to correctly\nclassify ADR posts and accurately extract ADR mentions from online posts, the\nproposed framework is also able to further understand reasons for which the\ndrug is being taken, known as 'indication', from the given social media post. A\ncoverage-based attention mechanism is adopted in our framework to help the\nmodel properly identify 'phrasal' ADRs and Indications that are attentive to\nmultiple words in a post. Our framework is applicable in situations where\nlimited parallel data for different pharmacovigilance tasks are available.We\nevaluate the proposed framework on real-world Twitter datasets, where the\nproposed model outperforms the state-of-the-art alternatives of each individual\ntask consistently.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.06482v1",
            "date": "2018-01-19 16:27:36+00:00",
            "title": "Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms",
            "authors": [
                "Sweta Agrawal",
                "Amit Awekar"
            ],
            "abstract": "Harassment by cyberbullies is a significant phenomenon on the social media.\nExisting works for cyberbullying detection have at least one of the following\nthree bottlenecks. First, they target only one particular social media platform\n(SMP). Second, they address just one topic of cyberbullying. Third, they rely\non carefully handcrafted features of the data. We show that deep learning based\nmodels can overcome all three bottlenecks. Knowledge learned by these models on\none dataset can be transferred to other datasets. We performed extensive\nexperiments using three real-world datasets: Formspring (12k posts), Twitter\n(16k posts), and Wikipedia(100k posts). Our experiments provide several useful\ninsights about cyberbullying detection. To the best of our knowledge, this is\nthe first work that systematically analyzes cyberbullying detection on various\ntopics across multiple SMPs using deep learning based models and transfer\nlearning.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.07175v2",
            "date": "2018-01-22 16:19:52+00:00",
            "title": "Adversarial Texts with Gradient Methods",
            "authors": [
                "Zhitao Gong",
                "Wenlu Wang",
                "Bo Li",
                "Dawn Song",
                "Wei-Shinn Ku"
            ],
            "abstract": "Adversarial samples for images have been extensively studied in the\nliterature. Among many of the attacking methods, gradient-based methods are\nboth effective and easy to compute. In this work, we propose a framework to\nadapt the gradient attacking methods on images to text domain. The main\ndifficulties for generating adversarial texts with gradient methods are i) the\ninput space is discrete, which makes it difficult to accumulate small noise\ndirectly in the inputs, and ii) the measurement of the quality of the\nadversarial texts is difficult. We tackle the first problem by searching for\nadversarials in the embedding space and then reconstruct the adversarial texts\nvia nearest neighbor search. For the latter problem, we employ the Word Mover's\nDistance (WMD) to quantify the quality of adversarial texts. Through extensive\nexperiments on three datasets, IMDB movie reviews, Reuters-2 and Reuters-5\nnewswires, we show that our framework can leverage gradient attacking methods\nto generate very high-quality adversarial texts that are only a few words\ndifferent from the original texts. There are many cases where we can change one\nword to alter the label of the whole piece of text. We successfully incorporate\nFGM and DeepFool into our framework. In addition, we empirically show that WMD\nis closely related to the quality of adversarial texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07861v1",
            "date": "2018-01-24 05:11:57+00:00",
            "title": "Improving Review Representations with User Attention and Product Attention for Sentiment Classification",
            "authors": [
                "Zhen Wu",
                "Xin-Yu Dai",
                "Cunyan Yin",
                "Shujian Huang",
                "Jiajun Chen"
            ],
            "abstract": "Neural network methods have achieved great success in reviews sentiment\nclassification. Recently, some works achieved improvement by incorporating user\nand product information to generate a review representation. However, in\nreviews, we observe that some words or sentences show strong user's preference,\nand some others tend to indicate product's characteristic. The two kinds of\ninformation play different roles in determining the sentiment label of a\nreview. Therefore, it is not reasonable to encode user and product information\ntogether into one representation. In this paper, we propose a novel framework\nto encode user and product information. Firstly, we apply two individual\nhierarchical neural networks to generate two representations, with user\nattention or with product attention. Then, we design a combined strategy to\nmake full use of the two representations for training and final prediction. The\nexperimental results show that our model obviously outperforms other\nstate-of-the-art methods on IMDB and Yelp datasets. Through the visualization\nof attention over words related to user or product, we validate our observation\nmentioned above.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07875v2",
            "date": "2018-01-24 06:38:06+00:00",
            "title": "Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection",
            "authors": [
                "Michael Bloodgood"
            ],
            "abstract": "This paper investigates and evaluates support vector machine active learning\nalgorithms for use with imbalanced datasets, which commonly arise in many\napplications such as information extraction applications. Algorithms based on\nclosest-to-hyperplane selection and query-by-committee selection are combined\nwith methods for addressing imbalance such as positive amplification based on\nprevalence statistics from initial random samples. Three algorithms (ClosestPA,\nQBagPA, and QBoostPA) are presented and carefully evaluated on datasets for\ntext classification and relation extraction. The ClosestPA algorithm is shown\nto consistently outperform the other two in a variety of ways and insights are\nprovided as to why this is the case.",
            "journal ref": "In Proceedings of the 2018 IEEE 12th International Conference on\n  Semantic Computing (ICSC), pages 148-155, Laguna Hills, CA, USA, January\n  2018. IEEE",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.07883v2",
            "date": "2018-01-24 07:32:29+00:00",
            "title": "Deep Learning for Sentiment Analysis : A Survey",
            "authors": [
                "Lei Zhang",
                "Shuai Wang",
                "Bing Liu"
            ],
            "abstract": "Deep learning has emerged as a powerful machine learning technique that\nlearns multiple layers of representations or features of the data and produces\nstate-of-the-art prediction results. Along with the success of deep learning in\nmany other application domains, deep learning is also popularly used in\nsentiment analysis in recent years. This paper first gives an overview of deep\nlearning and then provides a comprehensive survey of its current applications\nin sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.07887v2",
            "date": "2018-01-24 07:47:05+00:00",
            "title": "Impact of Batch Size on Stopping Active Learning for Text Classification",
            "authors": [
                "Garrett Beatty",
                "Ethan Kochis",
                "Michael Bloodgood"
            ],
            "abstract": "When using active learning, smaller batch sizes are typically more efficient\nfrom a learning efficiency perspective. However, in practice due to speed and\nhuman annotator considerations, the use of larger batch sizes is necessary.\nWhile past work has shown that larger batch sizes decrease learning efficiency\nfrom a learning curve perspective, it remains an open question how batch size\nimpacts methods for stopping active learning. We find that large batch sizes\ndegrade the performance of a leading stopping method over and above the\ndegradation that results from reduced learning efficiency. We analyze this\ndegradation and find that it can be mitigated by changing the window size\nparameter of how many past iterations of learning are taken into account when\nmaking the stopping decision. We find that when using larger batch sizes,\nstopping methods are more effective when smaller window sizes are used.",
            "journal ref": "In Proceedings of the 2018 IEEE 12th International Conference on\n  Semantic Computing (ICSC), pages 306-307, Laguna Hills, CA, USA, January\n  2018. IEEE",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.07948v5",
            "date": "2018-01-24 12:11:09+00:00",
            "title": "Empirical observations of ultraslow diffusion driven by the fractional dynamics in languages: Dynamical statistical properties of word counts of already popular words",
            "authors": [
                "Hayafumi Watanabe"
            ],
            "abstract": "Ultraslow diffusion (i.e. logarithmic diffusion) has been extensively studied\ntheoretically, but has hardly been observed empirically. In this paper,\nfirstly, we find the ultraslow-like diffusion of the time-series of word counts\nof already popular words by analysing three different nationwide language\ndatabases: (i) newspaper articles (Japanese), (ii) blog articles (Japanese),\nand (iii) page views of Wikipedia (English, French, Chinese, and Japanese).\nSecondly, we use theoretical analysis to show that this diffusion is basically\nexplained by the random walk model with the power-law forgetting with the\nexponent $\\beta \\approx 0.5$, which is related to the fractional Langevin\nequation. The exponent $\\beta$ characterises the speed of forgetting and $\\beta\n\\approx 0.5$ corresponds to (i) the border (or thresholds) between the\nstationary and the nonstationary and (ii) the right-in-the-middle dynamics\nbetween the IID noise for $\\beta=1$ and the normal random walk for $\\beta=0$.\nThirdly, the generative model of the time-series of word counts of already\npopular words, which is a kind of Poisson process with the Poisson parameter\nsampled by the above-mentioned random walk model, can almost reproduce not only\nthe empirical mean-squared displacement but also the power spectrum density and\nthe probability density function.",
            "journal ref": "Phys. Rev. E 98, 012308 (2018)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1801.08186v3",
            "date": "2018-01-24 20:54:26+00:00",
            "title": "MAttNet: Modular Attention Network for Referring Expression Comprehension",
            "authors": [
                "Licheng Yu",
                "Zhe Lin",
                "Xiaohui Shen",
                "Jimei Yang",
                "Xin Lu",
                "Mohit Bansal",
                "Tamara L. Berg"
            ],
            "abstract": "In this paper, we address referring expression comprehension: localizing an\nimage region described by a natural language expression. While most recent work\ntreats expressions as a single unit, we propose to decompose them into three\nmodular components related to subject appearance, location, and relationship to\nother objects. This allows us to flexibly adapt to expressions containing\ndifferent types of information in an end-to-end framework. In our model, which\nwe call the Modular Attention Network (MAttNet), two types of attention are\nutilized: language-based attention that learns the module weights as well as\nthe word/phrase attention that each module should focus on; and visual\nattention that allows the subject and relationship modules to focus on relevant\nimage components. Module weights combine scores from all three modules\ndynamically to output an overall score. Experiments show that MAttNet\noutperforms previous state-of-art methods by a large margin on both\nbounding-box-level and pixel-level comprehension tasks. Demo and code are\nprovided.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1801.09251v2",
            "date": "2018-01-28 17:14:13+00:00",
            "title": "Multi-Pointer Co-Attention Networks for Recommendation",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Many recent state-of-the-art recommender systems such as D-ATT, TransNet and\nDeepCoNN exploit reviews for representation learning. This paper proposes a new\nneural architecture for recommendation with reviews. Our model operates on a\nmulti-hierarchical paradigm and is based on the intuition that not all reviews\nare created equal, i.e., only a select few are important. The importance,\nhowever, should be dynamically inferred depending on the current target. To\nthis end, we propose a review-by-review pointer-based learning scheme that\nextracts important reviews, subsequently matching them in a word-by-word\nfashion. This enables not only the most informative reviews to be utilized for\nprediction but also a deeper word-level interaction. Our pointer-based method\noperates with a novel gumbel-softmax based pointer mechanism that enables the\nincorporation of discrete vectors within differentiable neural architectures.\nOur pointer mechanism is co-attentive in nature, learning pointers which are\nco-dependent on user-item relationships. Finally, we propose a multi-pointer\nlearning scheme that learns to combine multiple views of interactions between\nuser and item. Overall, we demonstrate the effectiveness of our proposed model\nvia extensive experiments on \\textbf{24} benchmark datasets from Amazon and\nYelp. Empirical results show that our approach significantly outperforms\nexisting state-of-the-art, with up to 19% and 71% relative improvement when\ncompared to TransNet and DeepCoNN respectively. We study the behavior of our\nmulti-pointer learning mechanism, shedding light on evidence aggregation\npatterns in review-based recommender systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.09637v1",
            "date": "2018-01-29 17:24:27+00:00",
            "title": "Geospatial distributions reflect rates of evolution of features of language",
            "authors": [
                "Henri Kauhanen",
                "Deepthi Gopal",
                "Tobias Galla",
                "Ricardo Berm\u00fadez-Otero"
            ],
            "abstract": "Different structural features of human language change at different rates and\nthus exhibit different temporal stabilities. Existing methods of linguistic\nstability estimation depend upon the prior genealogical classification of the\nworld's languages into language families; these methods result in unreliable\nstability estimates for features which are sensitive to horizontal transfer\nbetween families and whenever data are aggregated from families of divergent\ntime depths. To overcome these problems, we describe a method of stability\nestimation without family classifications, based on mathematical modelling and\nthe analysis of contemporary geospatial distributions of linguistic features.\nRegressing the estimates produced by our model against those of a genealogical\nmethod, we report broad agreement but also important differences. In\nparticular, we show that our approach is not liable to some of the false\npositives and false negatives incurred by the genealogical method. Our results\nsuggest that the historical evolution of a linguistic feature leaves a\nfootprint in its global geospatial distribution, and that rates of evolution\ncan be recovered from these distributions by treating language dynamics as a\nspatially extended stochastic process.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1801.09788v1",
            "date": "2018-01-29 22:43:32+00:00",
            "title": "Evaluating approaches for supervised semantic labeling",
            "authors": [
                "Natalia Ruemmele",
                "Yuriy Tyshetskiy",
                "Alex Collins"
            ],
            "abstract": "Relational data sources are still one of the most popular ways to store\nenterprise or Web data, however, the issue with relational schema is the lack\nof a well-defined semantic description. A common ontology provides a way to\nrepresent the meaning of a relational schema and can facilitate the integration\nof heterogeneous data sources within a domain. Semantic labeling is achieved by\nmapping attributes from the data sources to the classes and properties in the\nontology. We formulate this problem as a multi-class classification problem\nwhere previously labeled data sources are used to learn rules for labeling new\ndata sources. The majority of existing approaches for semantic labeling have\nfocused on data integration challenges such as naming conflicts and semantic\nheterogeneity. In addition, machine learning approaches typically have issues\naround class imbalance, lack of labeled instances and relative importance of\nattributes. To address these issues, we develop a new machine learning model\nwith engineered features as well as two deep learning models which do not\nrequire extensive feature engineering. We evaluate our new approaches with the\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1801.09851v4",
            "date": "2018-01-30 04:44:14+00:00",
            "title": "Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning",
            "authors": [
                "Xuan Wang",
                "Yu Zhang",
                "Xiang Ren",
                "Yuhao Zhang",
                "Marinka Zitnik",
                "Jingbo Shang",
                "Curtis Langlotz",
                "Jiawei Han"
            ],
            "abstract": "Motivation: State-of-the-art biomedical named entity recognition (BioNER)\nsystems often require handcrafted features specific to each entity type, such\nas genes, chemicals and diseases. Although recent studies explored using neural\nnetwork models for BioNER to free experts from manual feature engineering, the\nperformance remains limited by the available training data for each entity\ntype. Results: We propose a multi-task learning framework for BioNER to\ncollectively use the training data of different types of entities and improve\nthe performance on each of them. In experiments on 15 benchmark BioNER\ndatasets, our multi-task model achieves substantially better performance\ncompared with state-of-the-art BioNER systems and baseline neural sequence\nlabeling models. Further analysis shows that the large performance gains come\nfrom sharing character- and word-level information among relevant biomedical\nentities across differently labeled corpora.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1801.09975v2",
            "date": "2018-01-30 13:18:51+00:00",
            "title": "Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media",
            "authors": [
                "Semiha Makinist",
                "Ibrahim Riza Hallac",
                "Betul Ay Karakus",
                "Galip Aydin"
            ],
            "abstract": "A public dataset, with a variety of properties suitable for sentiment\nanalysis [1], event prediction, trend detection and other text mining\napplications, is needed in order to be able to successfully perform analysis\nstudies. The vast majority of data on social media is text-based and it is not\npossible to directly apply machine learning processes into these raw data,\nsince several different processes are required to prepare the data before the\nimplementation of the algorithms. For example, different misspellings of same\nword enlarge the word vector space unnecessarily, thereby it leads to reduce\nthe success of the algorithm and increase the computational power requirement.\nThis paper presents an improved Turkish dataset with an effective spelling\ncorrection algorithm based on Hadoop [2]. The collected data is recorded on the\nHadoop Distributed File System and the text based data is processed by\nMapReduce programming model. This method is suitable for the storage and\nprocessing of large sized text based social media data. In this study, movie\nreviews have been automatically recorded with Apache ManifoldCF (MCF) [3] and\ndata clusters have been created. Various methods compared such as Levenshtein\nand Fuzzy String Matching have been proposed to create a public dataset from\ncollected data. Experimental results show that the proposed algorithm, which\ncan be used as an open source dataset in sentiment analysis studies, have been\nperformed successfully to the detection and correction of spelling errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10063v1",
            "date": "2018-01-28 09:46:22+00:00",
            "title": "Characterisation of (Sub)sequential Rational Functions over a General Class Monoids",
            "authors": [
                "Stefan Gerdjikov"
            ],
            "abstract": "In this technical report we describe a general class of monoids for which\n(sub)sequential rational can be characterised in terms of a congruence relation\nin the flavour of Myhill-Nerode relation. The class of monoids that we consider\ncan be described in terms of natural algebraic axioms, contains the free\nmonoids, groups, the tropical monoid, and is closed under Cartesian.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1801.10253v2",
            "date": "2018-01-30 23:19:49+00:00",
            "title": "The New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval",
            "authors": [
                "Spencer Cappallo",
                "Stacey Svetlichnaya",
                "Pierre Garrigues",
                "Thomas Mensink",
                "Cees G. M. Snoek"
            ],
            "abstract": "Over the past decade, emoji have emerged as a new and widespread form of\ndigital communication, spanning diverse social networks and spoken languages.\nWe propose to treat these ideograms as a new modality in their own right,\ndistinct in their semantic structure from both the text in which they are often\nembedded as well as the images which they resemble. As a new modality, emoji\npresent rich novel possibilities for representation and interaction. In this\npaper, we explore the challenges that arise naturally from considering the\nemoji modality through the lens of multimedia research. Specifically, the ways\nin which emoji can be related to other common modalities such as text and\nimages. To do so, we first present a large scale dataset of real-world emoji\nusage collected from Twitter. This dataset contains examples of both text-emoji\nand image-emoji relationships. We present baseline results on the challenge of\npredicting emoji from both text and images, using state-of-the-art neural\nnetworks. Further, we offer a first consideration into the problem of how to\naccount for new, unseen emoji - a relevant issue as the emoji vocabulary\ncontinues to expand on a yearly basis. Finally, we present results for\nmultimedia retrieval using emoji as queries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00047v2",
            "date": "2018-03-30 20:17:31+00:00",
            "title": "Conditional End-to-End Audio Transforms",
            "authors": [
                "Albert Haque",
                "Michelle Guo",
                "Prateek Verma"
            ],
            "abstract": "We present an end-to-end method for transforming audio from one style to\nanother. For the case of speech, by conditioning on speaker identities, we can\ntrain a single model to transform words spoken by multiple people into multiple\ntarget voices. For the case of music, we can specify musical instruments and\nachieve the same result. Architecturally, our method is a fully-differentiable\nsequence-to-sequence model based on convolutional and hierarchical recurrent\nneural networks. It is designed to capture long-term acoustic dependencies,\nrequires minimal post-processing, and produces realistic audio transforms.\nAblation studies confirm that our model can separate speaker and instrument\nproperties from acoustic content at different receptive fields. Empirically,\nour method achieves competitive performance on community-standard datasets.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1804.00306v2",
            "date": "2018-04-01 15:41:01+00:00",
            "title": "Revisiting Skip-Gram Negative Sampling Model with Rectification",
            "authors": [
                "Cun Mu",
                "Guang Yang",
                "Zheng Yan"
            ],
            "abstract": "We revisit skip-gram negative sampling (SGNS), one of the most popular\nneural-network based approaches to learning distributed word representation. We\nfirst point out the ambiguity issue undermining the SGNS model, in the sense\nthat the word vectors can be entirely distorted without changing the objective\nvalue. To resolve the issue, we investigate the intrinsic structures in\nsolution that a good word embedding model should deliver. Motivated by this, we\nrectify the SGNS model with quadratic regularization, and show that this simple\nmodification suffices to structure the solution in the desired manner. A\ntheoretical justification is presented, which provides novel insights into\nquadratic regularization . Preliminary experiments are also conducted on\nGoogle's analytical reasoning task to support the modified SGNS model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00401v1",
            "date": "2018-04-02 05:36:38+00:00",
            "title": "An End-to-end Neural Natural Language Interface for Databases",
            "authors": [
                "Prasetya Utama",
                "Nathaniel Weir",
                "Fuat Basik",
                "Carsten Binnig",
                "Ugur Cetintemel",
                "Benjamin H\u00e4ttasch",
                "Amir Ilkhechi",
                "Shekar Ramaswamy",
                "Arif Usta"
            ],
            "abstract": "The ability to extract insights from new data sets is critical for decision\nmaking. Visual interactive tools play an important role in data exploration\nsince they provide non-technical users with an effective way to visually\ncompose queries and comprehend the results. Natural language has recently\ngained traction as an alternative query interface to databases with the\npotential to enable non-expert users to formulate complex questions and\ninformation needs efficiently and effectively. However, understanding natural\nlanguage questions and translating them accurately to SQL is a challenging\ntask, and thus Natural Language Interfaces for Databases (NLIDBs) have not yet\nmade their way into practical tools and commercial products.\n  In this paper, we present DBPal, a novel data exploration tool with a natural\nlanguage interface. DBPal leverages recent advances in deep models to make\nquery understanding more robust in the following ways: First, DBPal uses a deep\nmodel to translate natural language statements to SQL, making the translation\nprocess more robust to paraphrasing and other linguistic variations. Second, to\nsupport the users in phrasing questions without knowing the database schema and\nthe query features, DBPal provides a learned auto-completion model that\nsuggests partial query extensions to users during query formulation and thus\nhelps to write complex queries.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1804.00425v1",
            "date": "2018-04-02 07:58:23+00:00",
            "title": "High-quality nonparallel voice conversion based on cycle-consistent adversarial network",
            "authors": [
                "Fuming Fang",
                "Junichi Yamagishi",
                "Isao Echizen",
                "Jaime Lorenzo-Trueba"
            ],
            "abstract": "Although voice conversion (VC) algorithms have achieved remarkable success\nalong with the development of machine learning, superior performance is still\ndifficult to achieve when using nonparallel data. In this paper, we propose\nusing a cycle-consistent adversarial network (CycleGAN) for nonparallel\ndata-based VC training. A CycleGAN is a generative adversarial network (GAN)\noriginally developed for unpaired image-to-image translation. A subjective\nevaluation of inter-gender conversion demonstrated that the proposed method\nsignificantly outperformed a method based on the Merlin open source neural\nnetwork speech synthesis system (a parallel VC system adapted for our setup)\nand a GAN-based parallel VC system. This is the first research to show that the\nperformance of a nonparallel VC method can exceed that of state-of-the-art\nparallel VC methods.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.00644v2",
            "date": "2018-04-02 17:45:57+00:00",
            "title": "Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yifan Gong",
                "Biing-Hwang",
                "Juang"
            ],
            "abstract": "The teacher-student (T/S) learning has been shown effective in unsupervised\ndomain adaptation [1]. It is a form of transfer learning, not in terms of the\ntransfer of recognition decisions, but the knowledge of posteriori\nprobabilities in the source domain as evaluated by the teacher model. It learns\nto handle the speaker and environment variability inherent in and restricted to\nthe speech signal in the target domain without proactively addressing the\nrobustness to other likely conditions. Performance degradation may thus ensue.\nIn this work, we advance T/S learning by proposing adversarial T/S learning to\nexplicitly achieve condition-robust unsupervised domain adaptation. In this\nmethod, a student acoustic model and a condition classifier are jointly\noptimized to minimize the Kullback-Leibler divergence between the output\ndistributions of the teacher and student models, and simultaneously, to\nmin-maximize the condition classification loss. A condition-invariant deep\nfeature is learned in the adapted student model through this procedure. We\nfurther propose multi-factorial adversarial T/S learning which suppresses\ncondition variabilities caused by multiple factors simultaneously. Evaluated\nwith the noisy CHiME-3 test set, the proposed methods achieve relative word\nerror rate improvements of 44.60% and 5.38%, respectively, over a clean source\nmodel and a strong T/S learning baseline model.",
            "journal ref": "2018 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Calgary, Canada",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.00732v3",
            "date": "2018-04-02 21:09:30+00:00",
            "title": "Speaker-Invariant Training via Adversarial Learning",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Zhuo Chen",
                "Yong Zhao",
                "Vadim Mazalov",
                "Yifan Gong",
                "Biing-Hwang",
                "Juang"
            ],
            "abstract": "We propose a novel adversarial multi-task learning scheme, aiming at actively\ncurtailing the inter-talker feature variability while maximizing its senone\ndiscriminability so as to enhance the performance of a deep neural network\n(DNN) based ASR system. We call the scheme speaker-invariant training (SIT). In\nSIT, a DNN acoustic model and a speaker classifier network are jointly\noptimized to minimize the senone (tied triphone state) classification loss, and\nsimultaneously mini-maximize the speaker classification loss. A\nspeaker-invariant and senone-discriminative deep feature is learned through\nthis adversarial multi-task learning. With SIT, a canonical DNN acoustic model\nwith significantly reduced variance in its output probabilities is learned with\nno explicit speaker-independent (SI) transformations or speaker-specific\nrepresentations used in training or testing. Evaluated on the CHiME-3 dataset,\nthe SIT achieves 4.99% relative word error rate (WER) improvement over the\nconventional SI acoustic model. With additional unsupervised speaker\nadaptation, the speaker-adapted (SA) SIT model achieves 4.86% relative WER gain\nover the SA SI acoustic model.",
            "journal ref": "2018 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Calgary, Canada",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.00823v4",
            "date": "2018-04-03 04:47:22+00:00",
            "title": "Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks",
            "authors": [
                "Kun Xu",
                "Lingfei Wu",
                "Zhiguo Wang",
                "Yansong Feng",
                "Michael Witbrock",
                "Vadim Sheinin"
            ],
            "abstract": "The celebrated Sequence to Sequence learning (Seq2Seq) technique and its\nnumerous variants achieve excellent performance on many tasks. However, many\nmachine learning tasks have inputs naturally represented as graphs; existing\nSeq2Seq models face a significant challenge in achieving accurate conversion\nfrom graph form to the appropriate sequence. To address this challenge, we\nintroduce a novel general end-to-end graph-to-sequence neural encoder-decoder\nmodel that maps an input graph to a sequence of vectors and uses an\nattention-based LSTM method to decode the target sequence from these vectors.\nOur method first generates the node and graph embeddings using an improved\ngraph-based neural network with a novel aggregation strategy to incorporate\nedge direction information in the node embeddings. We further introduce an\nattention mechanism that aligns node embeddings and the decoding sequence to\nbetter cope with large graphs. Experimental results on bAbI, Shortest Path, and\nNatural Language Generation tasks demonstrate that our model achieves\nstate-of-the-art performance and significantly outperforms existing graph\nneural networks, Seq2Seq, and Tree2Seq models; using the proposed\nbi-directional node embedding aggregation strategy, the model can converge\nrapidly to the optimal performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.00831v2",
            "date": "2018-04-03 05:31:35+00:00",
            "title": "AttnConvnet at SemEval-2018 Task 1: Attention-based Convolutional Neural Networks for Multi-label Emotion Classification",
            "authors": [
                "Yanghoon Kim",
                "Hwanhee Lee",
                "Kyomin Jung"
            ],
            "abstract": "In this paper, we propose an attention-based classifier that predicts\nmultiple emotions of a given sentence. Our model imitates human's two-step\nprocedure of sentence understanding and it can effectively represent and\nclassify sentences. With emoji-to-meaning preprocessing and extra lexicon\nutilization, we further improve the model performance. We train and evaluate\nour model with data provided by SemEval-2018 task 1-5, each sentence of which\nhas several labels among 11 given sentiments. Our model achieves 5-th/1-th rank\nin English/Spanish respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00920v1",
            "date": "2018-04-03 11:43:36+00:00",
            "title": "Speech waveform synthesis from MFCC sequences with generative adversarial networks",
            "authors": [
                "Lauri Juvela",
                "Bajibabu Bollepalli",
                "Xin Wang",
                "Hirokazu Kameoka",
                "Manu Airaksinen",
                "Junichi Yamagishi",
                "Paavo Alku"
            ],
            "abstract": "This paper proposes a method for generating speech from filterbank mel\nfrequency cepstral coefficients (MFCC), which are widely used in speech\napplications, such as ASR, but are generally considered unusable for speech\nsynthesis. First, we predict fundamental frequency and voicing information from\nMFCCs with an autoregressive recurrent neural net. Second, the spectral\nenvelope information contained in MFCCs is converted to all-pole filters, and a\npitch-synchronous excitation model matched to these filters is trained.\nFinally, we introduce a generative adversarial network -based noise model to\nadd a realistic high-frequency stochastic component to the modeled excitation\nsignal. The results show that high quality speech reconstruction can be\nobtained, given only MFCC information at test time.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.00982v1",
            "date": "2018-04-03 14:17:09+00:00",
            "title": "360\u00b0 Stance Detection",
            "authors": [
                "Sebastian Ruder",
                "John Glover",
                "Afshin Mehrabani",
                "Parsa Ghaffari"
            ],
            "abstract": "The proliferation of fake news and filter bubbles makes it increasingly\ndifficult to form an unbiased, balanced opinion towards a topic. To ameliorate\nthis, we propose 360{\\deg} Stance Detection, a tool that aggregates news with\nmultiple perspectives on a topic. It presents them on a spectrum ranging from\nsupport to opposition, enabling the user to base their opinion on multiple\npieces of diverse evidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.00987v2",
            "date": "2018-03-31 13:01:29+00:00",
            "title": "A Language for Function Signature Representations",
            "authors": [
                "Kyle Richardson"
            ],
            "abstract": "Recent work by (Richardson and Kuhn, 2017a,b; Richardson et al., 2018) looks\nat semantic parser induction and question answering in the domain of source\ncode libraries and APIs. In this brief note, we formalize the representations\nbeing learned in these studies and introduce a simple domain specific language\nand a systematic translation from this language to first-order logic. By\nrecasting the target representations in terms of classical logic, we aim to\nbroaden the applicability of existing code datasets for investigating more\ncomplex natural language understanding and reasoning problems in the software\ndomain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01189v2",
            "date": "2018-04-03 23:10:36+00:00",
            "title": "Real-Time Prediction of the Duration of Distribution System Outages",
            "authors": [
                "Aaron Jaech",
                "Baosen Zhang",
                "Mari Ostendorf",
                "Daniel S. Kirschen"
            ],
            "abstract": "This paper addresses the problem of predicting duration of unplanned power\noutages, using historical outage records to train a series of neural network\npredictors. The initial duration prediction is made based on environmental\nfactors, and it is updated based on incoming field reports using natural\nlanguage processing to automatically analyze the text. Experiments using 15\nyears of outage records show good initial results and improved performance\nleveraging text. Case studies show that the language processing identifies\nphrases that point to outage causes and repair steps.",
            "journal ref": "None",
            "category": "cs.SY"
        },
        {
            "url": "http://arxiv.org/abs/1804.01452v1",
            "date": "2018-04-04 15:03:08+00:00",
            "title": "Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input",
            "authors": [
                "David Harwath",
                "Adri\u00e0 Recasens",
                "D\u00eddac Sur\u00eds",
                "Galen Chuang",
                "Antonio Torralba",
                "James Glass"
            ],
            "abstract": "In this paper, we explore neural network models that learn to associate\nsegments of spoken audio captions with the semantically relevant portions of\nnatural images that they refer to. We demonstrate that these audio-visual\nassociative localizations emerge from network-internal representations learned\nas a by-product of training to perform an image-audio retrieval task. Our\nmodels operate directly on the image pixels and speech waveform, and do not\nrely on any conventional supervision in the form of labels, segmentations, or\nalignments between the modalities during training. We perform analysis using\nthe Places 205 and ADE20k datasets demonstrating that our models implicitly\nlearn semantically-coupled object and word detectors.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.01486v3",
            "date": "2018-04-04 16:02:54+00:00",
            "title": "Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data",
            "authors": [
                "Andrew L. Beam",
                "Benjamin Kompa",
                "Allen Schmaltz",
                "Inbar Fried",
                "Griffin Weber",
                "Nathan P. Palmer",
                "Xu Shi",
                "Tianxi Cai",
                "Isaac S. Kohane"
            ],
            "abstract": "Word embeddings are a popular approach to unsupervised learning of word\nrelationships that are widely used in natural language processing. In this\narticle, we present a new set of embeddings for medical concepts learned using\nan extremely large collection of multimodal medical data. Leaning on recent\ntheoretical insights, we demonstrate how an insurance claims database of 60\nmillion members, a collection of 20 million clinical notes, and 1.7 million\nfull text biomedical journal articles can be combined to embed concepts into a\ncommon space, resulting in the largest ever set of embeddings for 108,477\nmedical concepts. To evaluate our approach, we present a new benchmark\nmethodology based on statistical power specifically designed to test embeddings\nof medical concepts. Our approach, called cui2vec, attains state-of-the-art\nperformance relative to previous methods in most instances. Finally, we provide\na downloadable set of pre-trained embeddings for other researchers to use, as\nwell as an online tool for interactive exploration of the cui2vec embeddings",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01720v2",
            "date": "2018-04-05 08:13:37+00:00",
            "title": "Finding beans in burgers: Deep semantic-visual embedding with localization",
            "authors": [
                "Martin Engilberge",
                "Louis Chevallier",
                "Patrick P\u00e9rez",
                "Matthieu Cord"
            ],
            "abstract": "Several works have proposed to learn a two-path neural network that maps\nimages and texts, respectively, to a same shared Euclidean space where geometry\ncaptures useful semantic relationships. Such a multi-modal embedding can be\ntrained and used for various tasks, notably image captioning. In the present\nwork, we introduce a new architecture of this type, with a visual path that\nleverages recent space-aware pooling mechanisms. Combined with a textual path\nwhich is jointly trained from scratch, our semantic-visual embedding offers a\nversatile model. Once trained under the supervision of captioned images, it\nyields new state-of-the-art performance on cross-modal retrieval. It also\nallows the localization of new concepts from the embedding space into any input\nimage, delivering state-of-the-art result on the visual grounding of phrases.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.01963v1",
            "date": "2018-04-05 17:21:48+00:00",
            "title": "Automated Classification of Text Sentiment",
            "authors": [
                "Emmanuel Dufourq",
                "Bruce A. Bassett"
            ],
            "abstract": "The ability to identify sentiment in text, referred to as sentiment analysis,\nis one which is natural to adult humans. This task is, however, not one which a\ncomputer can perform by default. Identifying sentiments in an automated,\nalgorithmic manner will be a useful capability for business and research in\ntheir search to understand what consumers think about their products or\nservices and to understand human sociology. Here we propose two new Genetic\nAlgorithms (GAs) for the task of automated text sentiment analysis. The GAs\nlearn whether words occurring in a text corpus are either sentiment or\namplifier words, and their corresponding magnitude. Sentiment words, such as\n'horrible', add linearly to the final sentiment. Amplifier words in contrast,\nwhich are typically adjectives/adverbs like 'very', multiply the sentiment of\nthe following word. This increases, decreases or negates the sentiment of the\nfollowing word. The sentiment of the full text is then the sum of these terms.\nThis approach grows both a sentiment and amplifier dictionary which can be\nreused for other purposes and fed into other machine learning algorithms. We\nreport the results of multiple experiments conducted on large Amazon data sets.\nThe results reveal that our proposed approach was able to outperform several\npublic and/or commercial sentiment analysis algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02135v3",
            "date": "2018-04-06 05:27:14+00:00",
            "title": "Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder",
            "authors": [
                "Kei Akuzawa",
                "Yusuke Iwasawa",
                "Yutaka Matsuo"
            ],
            "abstract": "Recent advances in neural autoregressive models have improve the performance\nof speech synthesis (SS). However, as they lack the ability to model global\ncharacteristics of speech (such as speaker individualities or speaking styles),\nparticularly when these characteristics have not been labeled, making neural\nautoregressive SS systems more expressive is still an open issue. In this\npaper, we propose to combine VoiceLoop, an autoregressive SS model, with\nVariational Autoencoder (VAE). This approach, unlike traditional autoregressive\nSS systems, uses VAE to model the global characteristics explicitly, enabling\nthe expressiveness of the synthesized speech to be controlled in an\nunsupervised manner. Experiments using the VCTK and Blizzard2012 datasets show\nthe VAE helps VoiceLoop to generate higher quality speech and to control the\nexpressions in its synthesized speech by incorporating global characteristics\ninto the speech generating process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02204v1",
            "date": "2018-04-06 11:05:53+00:00",
            "title": "Sequence Training of DNN Acoustic Models With Natural Gradient",
            "authors": [
                "Adnan Haider",
                "Philip C. Woodland"
            ],
            "abstract": "Deep Neural Network (DNN) acoustic models often use discriminative sequence\ntraining that optimises an objective function that better approximates the word\nerror rate (WER) than frame-based training. Sequence training is normally\nimplemented using Stochastic Gradient Descent (SGD) or Hessian Free (HF)\ntraining. This paper proposes an alternative batch style optimisation framework\nthat employs a Natural Gradient (NG) approach to traverse through the parameter\nspace. By correcting the gradient according to the local curvature of the\nKL-divergence, the NG optimisation process converges more quickly than HF.\nFurthermore, the proposed NG approach can be applied to any sequence\ndiscriminative training criterion. The efficacy of the NG method is shown using\nexperiments on a Multi-Genre Broadcast (MGB) transcription task that\ndemonstrates both the computational efficiency and the accuracy of the\nresulting DNN models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02233v2",
            "date": "2018-04-06 12:36:28+00:00",
            "title": "Forex trading and Twitter: Spam, bots, and reputation manipulation",
            "authors": [
                "Igor Mozeti\u010d",
                "Peter Gabrov\u0161ek",
                "Petra Kralj Novak"
            ],
            "abstract": "Currency trading (Forex) is the largest world market in terms of volume. We\nanalyze trading and tweeting about the EUR-USD currency pair over a period of\nthree years. First, a large number of tweets were manually labeled, and a\nTwitter stance classification model is constructed. The model then classifies\nall the tweets by the trading stance signal: buy, hold, or sell (EUR vs. USD).\nThe Twitter stance is compared to the actual currency rates by applying the\nevent study methodology, well-known in financial economics. It turns out that\nthere are large differences in Twitter stance distribution and potential\ntrading returns between the four groups of Twitter users: trading robots,\nspammers, trading companies, and individual traders. Additionally, we observe\nattempts of reputation manipulation by post festum removal of tweets with poor\npredictions, and deleting/reposting of identical tweets to increase the\nvisibility without tainting one's Twitter timeline.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.02341v1",
            "date": "2018-04-06 16:12:51+00:00",
            "title": "Compositional Obverter Communication Learning From Raw Visual Input",
            "authors": [
                "Edward Choi",
                "Angeliki Lazaridou",
                "Nando de Freitas"
            ],
            "abstract": "One of the distinguishing aspects of human language is its compositionality,\nwhich allows us to describe complex environments with limited vocabulary.\nPreviously, it has been shown that neural network agents can learn to\ncommunicate in a highly structured, possibly compositional language based on\ndisentangled input (e.g. hand- engineered features). Humans, however, do not\nlearn to communicate based on well-summarized features. In this work, we train\nneural agents to simultaneously develop visual perception from raw image\npixels, and learn to communicate with a sequence of discrete symbols. The\nagents play an image description game where the image contains factors such as\ncolors and shapes. We train the agents using the obverter technique where an\nagent introspects to generate messages that maximize its own understanding.\nThrough qualitative analysis, visualization and a zero-shot test, we show that\nthe agents can develop, out of raw image pixels, a language with compositional\nproperties, given a proper pressure from the environment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.02525v1",
            "date": "2018-04-07 07:50:50+00:00",
            "title": "Quootstrap: Scalable Unsupervised Extraction of Quotation-Speaker Pairs from Large News Corpora via Bootstrapping",
            "authors": [
                "Dario Pavllo",
                "Tiziano Piccardi",
                "Robert West"
            ],
            "abstract": "We propose Quootstrap, a method for extracting quotations, as well as the\nnames of the speakers who uttered them, from large news corpora. Whereas prior\nwork has addressed this problem primarily with supervised machine learning, our\napproach follows a fully unsupervised bootstrapping paradigm. It leverages the\nredundancy present in large news corpora, more precisely, the fact that the\nsame quotation often appears across multiple news articles in slightly\ndifferent contexts. Starting from a few seed patterns, such as [\"Q\", said S.],\nour method extracts a set of quotation-speaker pairs (Q, S), which are in turn\nused for discovering new patterns expressing the same quotations; the process\nis then repeated with the larger pattern set. Our algorithm is highly scalable,\nwhich we demonstrate by running it on the large ICWSM 2011 Spinn3r corpus.\nValidating our results against a crowdsourced ground truth, we obtain 90%\nprecision at 40% recall using a single seed pattern, with significantly higher\nrecall values for more frequently reported (and thus likely more interesting)\nquotations. Finally, we showcase the usefulness of our algorithm's output for\ncomputational social science by analyzing the sentiment expressed in our\nextracted quotations.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.02549v1",
            "date": "2018-04-07 12:16:50+00:00",
            "title": "A comparison of recent waveform generation and acoustic modeling methods for neural-network-based speech synthesis",
            "authors": [
                "Xin Wang",
                "Jaime Lorenzo-Trueba",
                "Shinji Takaki",
                "Lauri Juvela",
                "Junichi Yamagishi"
            ],
            "abstract": "Recent advances in speech synthesis suggest that limitations such as the\nlossy nature of the amplitude spectrum with minimum phase approximation and the\nover-smoothing effect in acoustic modeling can be overcome by using advanced\nmachine learning approaches. In this paper, we build a framework in which we\ncan fairly compare new vocoding and acoustic modeling techniques with\nconventional approaches by means of a large scale crowdsourced evaluation.\nResults on acoustic models showed that generative adversarial networks and an\nautoregressive (AR) model performed better than a normal recurrent network and\nthe AR model performed best. Evaluation on vocoders by using the same AR\nacoustic model demonstrated that a Wavenet vocoder outperformed classical\nsource-filter-based vocoders. Particularly, generated speech waveforms from the\ncombination of AR acoustic model and Wavenet vocoder achieved a similar score\nof speech quality to vocoded speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.02617v1",
            "date": "2018-04-08 03:18:13+00:00",
            "title": "Language Modeling with Generative Adversarial Networks",
            "authors": [
                "Mehrad Moradshahi",
                "Utkarsh Contractor"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have been promising in the field of\nimage generation, however, they have been hard to train for language\ngeneration. GANs were originally designed to output differentiable values, so\ndiscrete language generation is challenging for them which causes high levels\nof instability in training GANs. Consequently, past work has resorted to\npre-training with maximum-likelihood or training GANs without pre-training with\na WGAN objective with a gradient penalty. In this study, we present a\ncomparison of those approaches. Furthermore, we present the results of some\nexperiments that indicate better training and convergence of Wasserstein GANs\n(WGANs) when a weaker regularization term is enforcing the Lipschitz\nconstraint.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1804.02657v1",
            "date": "2018-04-08 09:18:56+00:00",
            "title": "Emotion Orientated Recommendation System for Hiroshima Tourist by Fuzzy Petri Net",
            "authors": [
                "Takumi Ichimura",
                "Issei Tachibana"
            ],
            "abstract": "We developed an Android Smartophone application software for tourist\ninformation system. Especially, the agent system recommends the sightseeing\nspot and local hospitality corresponding to the current feelings. The system\nsuch as concierge can estimate user's emotion and mood by Emotion Generating\nCalculations and Mental State Transition Network. In this paper, the system\ndecides the next candidates for spots and foods by the reasoning of fuzzy Petri\nNet in order to make more smooth communication between human and smartphone.\nThe system was developed for Hiroshima Tourist Information and described some\nhospitality about the concierge system.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1804.02812v2",
            "date": "2018-04-09 04:31:43+00:00",
            "title": "Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations",
            "authors": [
                "Ju-chieh Chou",
                "Cheng-chieh Yeh",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Recently, cycle-consistent adversarial network (Cycle-GAN) has been\nsuccessfully applied to voice conversion to a different speaker without\nparallel data, although in those approaches an individual model is needed for\neach target speaker. In this paper, we propose an adversarial learning\nframework for voice conversion, with which a single model can be trained to\nconvert the voice to many different speakers, all without parallel data, by\nseparating the speaker characteristics from the linguistic content in speech\nsignals. An autoencoder is first trained to extract speaker-independent latent\nrepresentations and speaker embedding separately using another auxiliary\nspeaker classifier to regularize the latent representation. The decoder then\ntakes the speaker-independent latent representation and the target speaker\nembedding as the input to generate the voice of the target speaker with the\nlinguistic content of the source utterance. The quality of decoder output is\nfurther improved by patching with the residual signal produced by another pair\nof generator and discriminator. A target speaker set size of 20 was tested in\nthe preliminary experiments, and very good voice quality was obtained.\nConventional voice conversion metrics are reported. We also show that the\nspeaker information has been properly reduced from the latent representations.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.03052v1",
            "date": "2018-04-09 15:15:37+00:00",
            "title": "Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech",
            "authors": [
                "David Harwath",
                "Galen Chuang",
                "James Glass"
            ],
            "abstract": "In this paper, we explore the learning of neural network embeddings for\nnatural images and speech waveforms describing the content of those images.\nThese embeddings are learned directly from the waveforms without the use of\nlinguistic transcriptions or conventional speech recognition technology. While\nprior work has investigated this setting in the monolingual case using English\nspeech data, this work represents the first effort to apply these techniques to\nlanguages beyond English. Using spoken captions collected in English and Hindi,\nwe show that the same model architecture can be successfully applied to both\nlanguages. Further, we demonstrate that training a multilingual model\nsimultaneously on both languages offers improved performance over the\nmonolingual models. Finally, we show that these models are capable of\nperforming semantic cross-lingual speech-to-speech retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03240v1",
            "date": "2018-03-28 16:06:29+00:00",
            "title": "Deep Attention Model for Triage of Emergency Department Patients",
            "authors": [
                "Djordje Gligorijevic",
                "Jelena Stojanovic",
                "Wayne Satz",
                "Ivan Stojkovic",
                "Kathrin Schreyer",
                "Daniel Del Portal",
                "Zoran Obradovic"
            ],
            "abstract": "Optimization of patient throughput and wait time in emergency departments\n(ED) is an important task for hospital systems. For that reason, Emergency\nSeverity Index (ESI) system for patient triage was introduced to help guide\nmanual estimation of acuity levels, which is used by nurses to rank the\npatients and organize hospital resources. However, despite improvements that it\nbrought to managing medical resources, such triage system greatly depends on\nnurse's subjective judgment and is thus prone to human errors. Here, we propose\na novel deep model based on the word attention mechanism designed for\npredicting a number of resources an ED patient would need. Our approach\nincorporates routinely available continuous and nominal (structured) data with\nmedical text (unstructured) data, including patient's chief complaint, past\nmedical history, medication list, and nurse assessment collected for 338,500 ED\nvisits over three years in a large urban hospital. Using both structured and\nunstructured data, the proposed approach achieves the AUC of $\\sim 88\\%$ for\nthe task of identifying resource intensive patients (binary classification),\nand the accuracy of $\\sim 44\\%$ for predicting exact category of number of\nresources (multi-class classification task), giving an estimated lift over\nnurses' performance by 16\\% in accuracy. Furthermore, the attention mechanism\nof the proposed model provides interpretability by assigning attention scores\nfor nurses' notes which is crucial for decision making and implementation of\nsuch approaches in the real systems working on human health.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1804.03317v3",
            "date": "2018-04-10 02:39:41+00:00",
            "title": "Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN",
            "authors": [
                "Yingqi Qu",
                "Jie Liu",
                "Liangyi Kang",
                "Qinfeng Shi",
                "Dan Ye"
            ],
            "abstract": "With the rapid growth of knowledge bases (KBs), question answering over\nknowledge base, a.k.a. KBQA has drawn huge attention in recent years. Most of\nthe existing KBQA methods follow so called encoder-compare framework. They map\nthe question and the KB facts to a common embedding space, in which the\nsimilarity between the question vector and the fact vectors can be conveniently\ncomputed. This, however, inevitably loses original words interaction\ninformation. To preserve more original information, we propose an attentive\nrecurrent neural network with similarity matrix based convolutional neural\nnetwork (AR-SMCNN) model, which is able to capture comprehensive hierarchical\ninformation utilizing the advantages of both RNN and CNN. We use RNN to capture\nsemantic-level correlation by its sequential modeling nature, and use an\nattention mechanism to keep track of the entities and relations simultaneously.\nMeanwhile, we use a similarity matrix based CNN with two-directions pooling to\nextract literal-level words interaction matching utilizing CNNs strength of\nmodeling spatial correlation among data. Moreover, we have developed a new\nheuristic extension method for entity detection, which significantly decreases\nthe effect of noise. Our method has outperformed the state-of-the-arts on\nSimpleQuestion benchmark in both accuracy and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03396v2",
            "date": "2018-04-10 08:31:03+00:00",
            "title": "QA4IE: A Question Answering based Framework for Information Extraction",
            "authors": [
                "Lin Qiu",
                "Hao Zhou",
                "Yanru Qu",
                "Weinan Zhang",
                "Suoheng Li",
                "Shu Rong",
                "Dongyu Ru",
                "Lihua Qian",
                "Kewei Tu",
                "Yong Yu"
            ],
            "abstract": "Information Extraction (IE) refers to automatically extracting structured\nrelation tuples from unstructured texts. Common IE solutions, including\nRelation Extraction (RE) and open IE systems, can hardly handle cross-sentence\ntuples, and are severely restricted by limited relation types as well as\ninformal relation specifications (e.g., free-text based relation tuples). In\norder to overcome these weaknesses, we propose a novel IE framework named\nQA4IE, which leverages the flexible question answering (QA) approaches to\nproduce high quality relation triples across sentences. Based on the framework,\nwe develop a large IE benchmark with high quality human evaluation. This\nbenchmark contains 293K documents, 2M golden relation triples, and 636 relation\ntypes. We compare our system with some IE baselines on our benchmark and the\nresults show that our system achieves great improvements.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.03424v2",
            "date": "2018-04-10 10:00:36+00:00",
            "title": "A Hierarchical Latent Structure for Variational Conversation Modeling",
            "authors": [
                "Yookoon Park",
                "Jaemin Cho",
                "Gunhee Kim"
            ],
            "abstract": "Variational autoencoders (VAE) combined with hierarchical RNNs have emerged\nas a powerful framework for conversation modeling. However, they suffer from\nthe notorious degeneration problem, where the decoders learn to ignore latent\nvariables and reduce to vanilla RNNs. We empirically show that this degeneracy\noccurs mostly due to two reasons. First, the expressive power of hierarchical\nRNN decoders is often high enough to model the data using only its decoding\ndistributions without relying on the latent variables. Second, the conditional\nVAE structure whose generation process is conditioned on a context, makes the\nrange of training targets very sparse; that is, the RNN decoders can easily\noverfit to the training data ignoring the latent variables. To solve the\ndegeneration problem, we propose a novel model named Variational Hierarchical\nConversation RNNs (VHCR), involving two key ideas of (1) using a hierarchical\nstructure of latent variables, and (2) exploiting an utterance drop\nregularization. With evaluations on two datasets of Cornell Movie Dialog and\nUbuntu Dialog Corpus, we show that our VHCR successfully utilizes latent\nvariables and outperforms state-of-the-art models for conversation generation.\nMoreover, it can perform several new utterance control tasks, thanks to its\nhierarchical latent structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03540v2",
            "date": "2018-04-10 13:54:05+00:00",
            "title": "Mining Social Media for Newsgathering: A Review",
            "authors": [
                "Arkaitz Zubiaga"
            ],
            "abstract": "Social media is becoming an increasingly important data source for learning\nabout breaking news and for following the latest developments of ongoing news.\nThis is in part possible thanks to the existence of mobile devices, which\nallows anyone with access to the Internet to post updates from anywhere,\nleading in turn to a growing presence of citizen journalism. Consequently,\nsocial media has become a go-to resource for journalists during the process of\nnewsgathering. Use of social media for newsgathering is however challenging,\nand suitable tools are needed in order to facilitate access to useful\ninformation for reporting. In this paper, we provide an overview of research in\ndata mining and natural language processing for mining social media for\nnewsgathering. We discuss five different areas that researchers have worked on\nto mitigate the challenges inherent to social media newsgathering: news\ndiscovery, curation of news, validation and verification of content,\nnewsgathering dashboards, and other tasks. We outline the progress made so far\nin the field, summarise the current challenges as well as discuss future\ndirections in the use of computational journalism to assist with social media\nnewsgathering. This review is relevant to computer scientists researching news\nin social media as well as for interdisciplinary researchers interested in the\nintersection of computer science and journalism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.03608v1",
            "date": "2018-04-10 15:59:45+00:00",
            "title": "Imagine This! Scripts to Compositions to Videos",
            "authors": [
                "Tanmay Gupta",
                "Dustin Schwenk",
                "Ali Farhadi",
                "Derek Hoiem",
                "Aniruddha Kembhavi"
            ],
            "abstract": "Imagining a scene described in natural language with realistic layout and\nappearance of entities is the ultimate test of spatial, visual, and semantic\nworld knowledge. Towards this goal, we present the Composition, Retrieval, and\nFusion Network (CRAFT), a model capable of learning this knowledge from\nvideo-caption data and applying it while generating videos from novel captions.\nCRAFT explicitly predicts a temporal-layout of mentioned entities (characters\nand objects), retrieves spatio-temporal entity segments from a video database\nand fuses them to generate scene videos. Our contributions include sequential\ntraining of components of CRAFT while jointly modeling layout and appearances,\nand losses that encourage learning compositional representations for retrieval.\nWe evaluate CRAFT on semantic fidelity to caption, composition consistency, and\nvisual quality. CRAFT outperforms direct pixel generation approaches and\ngeneralizes well to unseen captions and to unseen video databases with no text\nannotations. We demonstrate CRAFT on FLINTSTONES, a new richly annotated\nvideo-caption dataset with over 25000 videos. For a glimpse of videos generated\nby CRAFT, see https://youtu.be/688Vv86n0z8.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1804.03782v3",
            "date": "2018-04-11 02:10:55+00:00",
            "title": "CoT: Cooperative Training for Generative Modeling of Discrete Data",
            "authors": [
                "Sidi Lu",
                "Lantao Yu",
                "Siyuan Feng",
                "Yaoming Zhu",
                "Weinan Zhang",
                "Yong Yu"
            ],
            "abstract": "In this paper, we study the generative models of sequential discrete data. To\ntackle the exposure bias problem inherent in maximum likelihood estimation\n(MLE), generative adversarial networks (GANs) are introduced to penalize the\nunrealistic generated samples. To exploit the supervision signal from the\ndiscriminator, most previous models leverage REINFORCE to address the\nnon-differentiable problem of sequential discrete data. However, because of the\nunstable property of the training signal during the dynamic process of\nadversarial training, the effectiveness of REINFORCE, in this case, is hardly\nguaranteed. To deal with such a problem, we propose a novel approach called\nCooperative Training (CoT) to improve the training of sequence generative\nmodels. CoT transforms the min-max game of GANs into a joint maximization\nframework and manages to explicitly estimate and optimize Jensen-Shannon\ndivergence. Moreover, CoT works without the necessity of pre-training via MLE,\nwhich is crucial to the success of previous methods. In the experiments,\ncompared to existing state-of-the-art methods, CoT shows superior or at least\ncompetitive performance on sample quality, diversity, as well as training\nstability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1804.03980v1",
            "date": "2018-04-11 13:48:08+00:00",
            "title": "Emergent Communication through Negotiation",
            "authors": [
                "Kris Cao",
                "Angeliki Lazaridou",
                "Marc Lanctot",
                "Joel Z Leibo",
                "Karl Tuyls",
                "Stephen Clark"
            ],
            "abstract": "Multi-agent reinforcement learning offers a way to study how communication\ncould emerge in communities of agents needing to solve specific problems. In\nthis paper, we study the emergence of communication in the negotiation\nenvironment, a semi-cooperative model of agent interaction. We introduce two\ncommunication protocols -- one grounded in the semantics of the game, and one\nwhich is \\textit{a priori} ungrounded and is a form of cheap talk. We show that\nself-interested agents can use the pre-grounded communication channel to\nnegotiate fairly, but are unable to effectively use the ungrounded channel.\nHowever, prosocial agents do learn to use cheap talk to find an optimal\nnegotiating strategy, suggesting that cooperation is necessary for language to\nemerge. We also study communication behaviour in a setting where one agent\ninteracts with agents in a community with different levels of prosociality and\nshow how agent identifiability can aid negotiation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.03984v1",
            "date": "2018-04-11 13:51:19+00:00",
            "title": "Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input",
            "authors": [
                "Angeliki Lazaridou",
                "Karl Moritz Hermann",
                "Karl Tuyls",
                "Stephen Clark"
            ],
            "abstract": "The ability of algorithms to evolve or learn (compositional) communication\nprotocols has traditionally been studied in the language evolution literature\nthrough the use of emergent communication tasks. Here we scale up this research\nby using contemporary deep learning methods and by training\nreinforcement-learning neural network agents on referential communication\ngames. We extend previous work, in which agents were trained in symbolic\nenvironments, by developing agents which are able to learn from raw pixel data,\na more challenging and realistic input representation. We find that the degree\nof structure found in the input data affects the nature of the emerged\nprotocols, and thereby corroborate the hypothesis that structured compositional\nlanguage is most likely to emerge when agents perceive the world as being\nstructured.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.04053v1",
            "date": "2018-04-03 09:21:11+00:00",
            "title": "EmoRL: Continuous Acoustic Emotion Classification using Deep Reinforcement Learning",
            "authors": [
                "Egor Lakomkin",
                "Mohammad Ali Zamani",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Acoustically expressed emotions can make communication with a robot more\nefficient. Detecting emotions like anger could provide a clue for the robot\nindicating unsafe/undesired situations. Recently, several deep neural\nnetwork-based models have been proposed which establish new state-of-the-art\nresults in affective state evaluation. These models typically start processing\nat the end of each utterance, which not only requires a mechanism to detect the\nend of an utterance but also makes it difficult to use them in a real-time\ncommunication scenario, e.g. human-robot interaction. We propose the EmoRL\nmodel that triggers an emotion classification as soon as it gains enough\nconfidence while listening to a person speaking. As a result, we minimize the\nneed for segmenting the audio signal for classification and achieve lower\nlatency as the audio signal is processed incrementally. The method is\ncompetitive with the accuracy of a strong baseline model, while allowing much\nearlier prediction.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1804.04058v1",
            "date": "2018-04-05 23:31:44+00:00",
            "title": "Analyzing Self-Driving Cars on Twitter",
            "authors": [
                "Rizwan Sadiq",
                "Mohsin Khan"
            ],
            "abstract": "This paper studies users' perception regarding a controversial product,\nnamely self-driving (autonomous) cars. To find people's opinion regarding this\nnew technology, we used an annotated Twitter dataset, and extracted the topics\nin positive and negative tweets using an unsupervised, probabilistic model\nknown as topic modeling. We later used the topics, as well as linguist and\nTwitter specific features to classify the sentiment of the tweets. Regarding\nthe opinions, the result of our analysis shows that people are optimistic and\nexcited about the future technology, but at the same time they find it\ndangerous and not reliable. For the classification task, we found Twitter\nspecific features, such as hashtags as well as linguistic features such as\nemphatic words among top attributes in classifying the sentiment of the tweets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1804.04095v1",
            "date": "2018-04-11 17:00:27+00:00",
            "title": "Predicting Twitter User Socioeconomic Attributes with Network and Language Information",
            "authors": [
                "Nikolaos Aletras",
                "Benjamin Paul Chamberlain"
            ],
            "abstract": "Inferring socioeconomic attributes of social media users such as occupation\nand income is an important problem in computational social science. Automated\ninference of such characteristics has applications in personalised recommender\nsystems, targeted computational advertising and online political campaigning.\nWhile previous work has shown that language features can reliably predict\nsocioeconomic attributes on Twitter, employing information coming from users'\nsocial networks has not yet been explored for such complex user\ncharacteristics. In this paper, we describe a method for predicting the\noccupational class and the income of Twitter users given information extracted\nfrom their extended networks by learning a low-dimensional vector\nrepresentation of users, i.e. graph embeddings. We use this representation to\ntrain predictive models for occupational class and income. Results on two\npublicly available datasets show that our method consistently outperforms the\nstate-of-the-art methods in both tasks. We also obtain further significant\nimprovements when we combine graph embeddings with textual features,\ndemonstrating that social network and language information are complementary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.04164v2",
            "date": "2018-04-06 17:44:23+00:00",
            "title": "Understanding Actors and Evaluating Personae with Gaussian Embeddings",
            "authors": [
                "Hannah Kim",
                "Denys Katerenchuk",
                "Daniel Billet",
                "Jun Huan",
                "Haesun Park",
                "Boyang Li"
            ],
            "abstract": "Understanding narrative content has become an increasingly popular topic.\nNonetheless, research on identifying common types of narrative characters, or\npersonae, is impeded by the lack of automatic and broad-coverage evaluation\nmethods. We argue that computationally modeling actors provides benefits,\nincluding novel evaluation mechanisms for personae. Specifically, we propose\ntwo actor-modeling tasks, cast prediction and versatility ranking, which can\ncapture complementary aspects of the relation between actors and the characters\nthey portray. For an actor model, we present a technique for embedding actors,\nmovies, character roles, genres, and descriptive keywords as Gaussian\ndistributions and translation vectors, where the Gaussian variance corresponds\nto actors' versatility. Empirical results indicate that (1) the technique\nconsiderably outperforms TransE (Bordes et al. 2013) and ablation baselines and\n(2) automatically identified persona topics (Bamman, O'Connor, and Smith 2013)\nyield statistically significant improvements in both tasks, whereas simplistic\npersona descriptors including age and gender perform inconsistently, validating\nprior research.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1804.04191v1",
            "date": "2018-04-11 19:59:44+00:00",
            "title": "Automatically Infer Human Traits and Behavior from Social Media Data",
            "authors": [
                "Shimei Pan",
                "Tao Ding"
            ],
            "abstract": "Given the complexity of human minds and their behavioral flexibility, it\nrequires sophisticated data analysis to sift through a large amount of human\nbehavioral evidence to model human minds and to predict human behavior. People\ncurrently spend a significant amount of time on social media such as Twitter\nand Facebook. Thus many aspects of their lives and behaviors have been\ndigitally captured and continuously archived on these platforms. This makes\nsocial media a great source of large, rich and diverse human behavioral\nevidence. In this paper, we survey the recent work on applying machine learning\nto infer human traits and behavior from social media data. We will also point\nout several future research directions.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.04205v1",
            "date": "2018-04-11 20:23:23+00:00",
            "title": "Learning Topics using Semantic Locality",
            "authors": [
                "Ziyi Zhao",
                "Krittaphat Pugdeethosapol",
                "Sheng Lin",
                "Zhe Li",
                "Caiwen Ding",
                "Yanzhi Wang",
                "Qinru Qiu"
            ],
            "abstract": "The topic modeling discovers the latent topic probability of the given text\ndocuments. To generate the more meaningful topic that better represents the\ngiven document, we proposed a new feature extraction technique which can be\nused in the data preprocessing stage. The method consists of three steps.\nFirst, it generates the word/word-pair from every single document. Second, it\napplies a two-way TF-IDF algorithm to word/word-pair for semantic filtering.\nThird, it uses the K-means algorithm to merge the word pairs that have the\nsimilar semantic meaning.\n  Experiments are carried out on the Open Movie Database (OMDb), Reuters\nDataset and 20NewsGroup Dataset. The mean Average Precision score is used as\nthe evaluation metric. Comparing our results with other state-of-the-art topic\nmodels, such as Latent Dirichlet allocation and traditional Restricted\nBoltzmann Machines. Our proposed data preprocessing can improve the generated\ntopic accuracy by up to 12.99\\%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1804.04212v3",
            "date": "2018-04-11 20:37:35+00:00",
            "title": "Word2Vec applied to Recommendation: Hyperparameters Matter",
            "authors": [
                "Hugo Caselles-Dupr\u00e9",
                "Florian Lesaint",
                "Jimena Royo-Letelier"
            ],
            "abstract": "Skip-gram with negative sampling, a popular variant of Word2vec originally\ndesigned and tuned to create word embeddings for Natural Language Processing,\nhas been used to create item embeddings with successful applications in\nrecommendation. While these fields do not share the same type of data, neither\nevaluate on the same tasks, recommendation applications tend to use the same\nalready tuned hyperparameters values, even if optimal hyperparameters values\nare often known to be data and task dependent. We thus investigate the marginal\nimportance of each hyperparameter in a recommendation setting through large\nhyperparameter grid searches on various datasets. Results reveal that\noptimizing neglected hyperparameters, namely negative sampling distribution,\nnumber of epochs, subsampling parameter and window-size, significantly improves\nperformance on a recommendation task, and can increase it by an order of\nmagnitude. Importantly, we find that optimal hyperparameters configurations for\nNatural Language Processing tasks and Recommendation tasks are noticeably\ndifferent.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.04262v1",
            "date": "2018-04-12 00:14:10+00:00",
            "title": "The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods",
            "authors": [
                "Jaime Lorenzo-Trueba",
                "Junichi Yamagishi",
                "Tomoki Toda",
                "Daisuke Saito",
                "Fernando Villavicencio",
                "Tomi Kinnunen",
                "Zhenhua Ling"
            ],
            "abstract": "We present the Voice Conversion Challenge 2018, designed as a follow up to\nthe 2016 edition with the aim of providing a common framework for evaluating\nand comparing different state-of-the-art voice conversion (VC) systems. The\nobjective of the challenge was to perform speaker conversion (i.e. transform\nthe vocal identity) of a source speaker to a target speaker while maintaining\nlinguistic information. As an update to the previous challenge, we considered\nboth parallel and non-parallel data to form the Hub and Spoke tasks,\nrespectively. A total of 23 teams from around the world submitted their\nsystems, 11 of them additionally participated in the optional Spoke task. A\nlarge-scale crowdsourced perceptual evaluation was then carried out to rate the\nsubmitted converted speech in terms of naturalness and similarity to the target\nspeaker identity. In this paper, we present a brief summary of the\nstate-of-the-art techniques for VC, followed by a detailed explanation of the\nchallenge tasks and the results that were obtained.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.05260v1",
            "date": "2018-04-14 18:24:06+00:00",
            "title": "ClassiNet -- Predicting Missing Features for Short-Text Classification",
            "authors": [
                "Danushka Bollegala",
                "Vincent Atanasov",
                "Takanori Maehara",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "The fundamental problem in short-text classification is \\emph{feature\nsparseness} -- the lack of feature overlap between a trained model and a test\ninstance to be classified. We propose \\emph{ClassiNet} -- a network of\nclassifiers trained for predicting missing features in a given instance, to\novercome the feature sparseness problem. Using a set of unlabeled training\ninstances, we first learn binary classifiers as feature predictors for\npredicting whether a particular feature occurs in a given instance. Next, each\nfeature predictor is represented as a vertex $v_i$ in the ClassiNet where a\none-to-one correspondence exists between feature predictors and vertices. The\nweight of the directed edge $e_{ij}$ connecting a vertex $v_i$ to a vertex\n$v_j$ represents the conditional probability that given $v_i$ exists in an\ninstance, $v_j$ also exists in the same instance. We show that ClassiNets\ngeneralize word co-occurrence graphs by considering implicit co-occurrences\nbetween features. We extract numerous features from the trained ClassiNet to\novercome feature sparseness. In particular, for a given instance $\\vec{x}$, we\nfind similar features from ClassiNet that did not appear in $\\vec{x}$, and\nappend those features in the representation of $\\vec{x}$. Moreover, we propose\na method based on graph propagation to find features that are indirectly\nrelated to a given short-text. We evaluate ClassiNets on several benchmark\ndatasets for short-text classification. Our experimental results show that by\nusing ClassiNet, we can statistically significantly improve the accuracy in\nshort-text classification tasks, without having to use any external resources\nsuch as thesauri for finding related features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.05306v1",
            "date": "2018-04-15 05:50:27+00:00",
            "title": "Transcribing Lyrics From Commercial Song Audio: The First Step Towards Singing Content Processing",
            "authors": [
                "Che-Ping Tsai",
                "Yi-Lin Tuan",
                "Lin-shan Lee"
            ],
            "abstract": "Spoken content processing (such as retrieval and browsing) is maturing, but\nthe singing content is still almost completely left out. Songs are human voice\ncarrying plenty of semantic information just as speech, and may be considered\nas a special type of speech with highly flexible prosody. The various problems\nin song audio, for example the significantly changing phone duration over\nhighly flexible pitch contours, make the recognition of lyrics from song audio\nmuch more difficult. This paper reports an initial attempt towards this goal.\nWe collected music-removed version of English songs directly from commercial\nsinging content. The best results were obtained by TDNN-LSTM with data\naugmentation with 3-fold speed perturbation plus some special approaches. The\nWER achieved (73.90%) was significantly lower than the baseline (96.21%), but\nstill relatively high.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1804.05448v1",
            "date": "2018-04-15 23:04:57+00:00",
            "title": "Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning",
            "authors": [
                "Xin Wang",
                "Yuan-Fang Wang",
                "William Yang Wang"
            ],
            "abstract": "A major challenge for video captioning is to combine audio and visual cues.\nExisting multi-modal fusion methods have shown encouraging results in video\nunderstanding. However, the temporal structures of multiple modalities at\ndifferent granularities are rarely explored, and how to selectively fuse the\nmulti-modal representations at different levels of details remains uncharted.\nIn this paper, we propose a novel hierarchically aligned cross-modal attention\n(HACA) framework to learn and selectively fuse both global and local temporal\ndynamics of different modalities. Furthermore, for the first time, we validate\nthe superior performance of the deep audio features on the video captioning\ntask. Finally, our HACA model significantly outperforms the previous best\nsystems and achieves new state-of-the-art results on the widely used MSR-VTT\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06201v2",
            "date": "2018-04-17 12:32:23+00:00",
            "title": "LCMR: Local and Centralized Memories for Collaborative Filtering with Unstructured Text",
            "authors": [
                "Guangneng Hu",
                "Yu Zhang",
                "Qiang Yang"
            ],
            "abstract": "Collaborative filtering (CF) is the key technique for recommender systems.\nPure CF approaches exploit the user-item interaction data (e.g., clicks, likes,\nand views) only and suffer from the sparsity issue. Items are usually\nassociated with content information such as unstructured text (e.g., abstracts\nof articles and reviews of products). CF can be extended to leverage text. In\nthis paper, we develop a unified neural framework to exploit interaction data\nand content information seamlessly. The proposed framework, called LCMR, is\nbased on memory networks and consists of local and centralized memories for\nexploiting content information and interaction data, respectively. By modeling\ncontent information as local memories, LCMR attentively learns what to exploit\nwith the guidance of user-item interaction. On real-world datasets, LCMR shows\nbetter performance by comparing with various baselines in terms of the hit\nratio and NDCG metrics. We further conduct analyses to understand how local and\ncentralized memories work for the proposed framework.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.06451v2",
            "date": "2018-04-17 19:39:26+00:00",
            "title": "Multi-Reward Reinforced Summarization with Saliency and Entailment",
            "authors": [
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Abstractive text summarization is the task of compressing and rewriting a\nlong document into a short summary while maintaining saliency, directed logical\nentailment, and non-redundancy. In this work, we address these three important\naspects of a good summary via a reinforcement learning approach with two novel\nreward functions: ROUGESal and Entail, on top of a coverage-based baseline. The\nROUGESal reward modifies the ROUGE metric by up-weighting the salient\nphrases/words detected via a keyphrase classifier. The Entail reward gives high\n(length-normalized) scores to logically-entailed summaries using an entailment\nclassifier. Further, we show superior performance improvement when these\nrewards are combined with traditional metric (ROUGE) based rewards, via our\nnovel and effective multi-reward approach of optimizing multiple rewards\nsimultaneously in alternate mini-batches. Our method achieves the new\nstate-of-the-art results (including human evaluation) on the CNN/Daily Mail\ndataset as well as strong improvements in a test-only transfer setup on\nDUC-2002.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06775v2",
            "date": "2018-04-18 15:02:10+00:00",
            "title": "Unspeech: Unsupervised Speech Context Embeddings",
            "authors": [
                "Benjamin Milde",
                "Chris Biemann"
            ],
            "abstract": "We introduce \"Unspeech\" embeddings, which are based on unsupervised learning\nof context feature representations for spoken language. The embeddings were\ntrained on up to 9500 hours of crawled English speech data without\ntranscriptions or speaker information, by using a straightforward learning\nobjective based on context and non-context discrimination with negative\nsampling. We use a Siamese convolutional neural network architecture to train\nUnspeech embeddings and evaluate them on speaker comparison, utterance\nclustering and as a context feature in TDNN-HMM acoustic models trained on\nTED-LIUM, comparing it to i-vector baselines. Particularly decoding\nout-of-domain speech data from the recently released Common Voice corpus shows\nconsistent WER reductions. We release our source code and pre-trained Unspeech\nmodels under a permissive open source license.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1804.06786v2",
            "date": "2018-04-18 15:23:04+00:00",
            "title": "Quantifying the visual concreteness of words and topics in multimodal datasets",
            "authors": [
                "Jack Hessel",
                "David Mimno",
                "Lillian Lee"
            ],
            "abstract": "Multimodal machine learning algorithms aim to learn visual-textual\ncorrespondences. Previous work suggests that concepts with concrete visual\nmanifestations may be easier to learn than concepts with abstract ones. We give\nan algorithm for automatically computing the visual concreteness of words and\ntopics within multimodal datasets. We apply the approach in four settings,\nranging from image captions to images/text scraped from historical books. In\naddition to enabling explorations of concepts in multimodal datasets, our\nconcreteness scores predict the capacity of machine learning algorithms to\nlearn textual/visual relationships. We find that 1) concrete concepts are\nindeed easier to learn; 2) the large number of algorithms we consider have\nsimilar failure cases; 3) the precise positive relationship between\nconcreteness and performance varies between datasets. We conclude with\nrecommendations for using concreteness scores to facilitate future multimodal\nresearch.",
            "journal ref": "2018 North American Chapter of the Association for Computational\n  Linguistics: Human Language Technologies (NAACL HLT)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.06870v2",
            "date": "2018-04-18 18:39:17+00:00",
            "title": "Object Ordering with Bidirectional Matchings for Visual Reasoning",
            "authors": [
                "Hao Tan",
                "Mohit Bansal"
            ],
            "abstract": "Visual reasoning with compositional natural language instructions, e.g.,\nbased on the newly-released Cornell Natural Language Visual Reasoning (NLVR)\ndataset, is a challenging task, where the model needs to have the ability to\ncreate an accurate mapping between the diverse phrases and the several objects\nplaced in complex arrangements in the image. Further, this mapping needs to be\nprocessed to answer the question in the statement given the ordering and\nrelationship of the objects across three similar images. In this paper, we\npropose a novel end-to-end neural model for the NLVR task, where we first use\njoint bidirectional attention to build a two-way conditioning between the\nvisual information and the language phrases. Next, we use an RL-based pointer\nnetwork to sort and process the varying number of unordered objects (so as to\nmatch the order of the statement phrases) in each of the three images and then\npool over the three decisions. Our model achieves strong improvements (of 4-6%\nabsolute) over the state-of-the-art on both the structured representation and\nraw image versions of the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07247v4",
            "date": "2018-04-19 16:06:29+00:00",
            "title": "Semantic Text Analysis for Detection of Compromised Accounts on Social Networks",
            "authors": [
                "Dominic Seyler",
                "Lunan Li",
                "ChengXiang Zhai"
            ],
            "abstract": "Compromised accounts on social networks are regular user accounts that have\nbeen taken over by an entity with malicious intent. Since the adversary\nexploits the already established trust of a compromised account, it is crucial\nto detect these accounts to limit the damage they can cause. We propose a novel\ngeneral framework for semantic analysis of text messages coming out from an\naccount to detect compromised accounts. Our framework is built on the\nobservation that normal users will use language that is measurably different\nfrom the language that an adversary would use when the account is compromised.\nWe propose to use the difference of language models of users and adversaries to\ndefine novel interpretable semantic features for measuring semantic incoherence\nin a message stream. We study the effectiveness of the proposed semantic\nfeatures using a Twitter data set. Evaluation results show that the proposed\nframework is effective for discovering compromised accounts on social networks\nand a KL-divergence-based language model feature works best.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.07789v1",
            "date": "2018-04-20 18:30:18+00:00",
            "title": "Generating Descriptions from Structured Data Using a Bifocal Attention Mechanism and Gated Orthogonalization",
            "authors": [
                "Preksha Nema",
                "Shreyas Shetty",
                "Parag Jain",
                "Anirban Laha",
                "Karthik Sankaranarayanan",
                "Mitesh M. Khapra"
            ],
            "abstract": "In this work, we focus on the task of generating natural language\ndescriptions from a structured table of facts containing fields (such as\nnationality, occupation, etc) and values (such as Indian, actor, director,\netc). One simple choice is to treat the table as a sequence of fields and\nvalues and then use a standard seq2seq model for this task. However, such a\nmodel is too generic and does not exploit task-specific characteristics. For\nexample, while generating descriptions from a table, a human would attend to\ninformation at two levels: (i) the fields (macro level) and (ii) the values\nwithin the field (micro level). Further, a human would continue attending to a\nfield for a few timesteps till all the information from that field has been\nrendered and then never return back to this field (because there is nothing\nleft to say about it). To capture this behavior we use (i) a fused bifocal\nattention mechanism which exploits and combines this micro and macro level\ninformation and (ii) a gated orthogonalization mechanism which tries to ensure\nthat a field is remembered for a few time steps and then forgotten. We\nexperiment with a recently released dataset which contains fact tables about\npeople and their corresponding one line biographical descriptions in English.\nIn addition, we also introduce two similar datasets for French and German. Our\nexperiments show that the proposed model gives 21% relative improvement over a\nrecently proposed state of the art method and 10% relative improvement over\nbasic seq2seq models. The code and the datasets developed as a part of this\nwork are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07855v3",
            "date": "2018-04-20 23:06:44+00:00",
            "title": "Subgoal Discovery for Hierarchical Dialogue Policy Learning",
            "authors": [
                "Da Tang",
                "Xiujun Li",
                "Jianfeng Gao",
                "Chong Wang",
                "Lihong Li",
                "Tony Jebara"
            ],
            "abstract": "Developing agents to engage in complex goal-oriented dialogues is challenging\npartly because the main learning signals are very sparse in long conversations.\nIn this paper, we propose a divide-and-conquer approach that discovers and\nexploits the hidden structure of the task to enable efficient policy learning.\nFirst, given successful example dialogues, we propose the Subgoal Discovery\nNetwork (SDN) to divide a complex goal-oriented task into a set of simpler\nsubgoals in an unsupervised fashion. We then use these subgoals to learn a\nmulti-level policy by hierarchical reinforcement learning. We demonstrate our\nmethod by building a dialogue agent for the composite task of travel planning.\nExperiments with simulated and real users show that our approach performs\ncompetitively against a state-of-the-art method that requires human-defined\nsubgoals. Moreover, we show that the learned subgoals are often human\ncomprehensible.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.07944v1",
            "date": "2018-04-21 11:12:25+00:00",
            "title": "Variational Inference In Pachinko Allocation Machines",
            "authors": [
                "Akash Srivastava",
                "Charles Sutton"
            ],
            "abstract": "The Pachinko Allocation Machine (PAM) is a deep topic model that allows\nrepresenting rich correlation structures among topics by a directed acyclic\ngraph over topics. Because of the flexibility of the model, however,\napproximate inference is very difficult. Perhaps for this reason, only a small\nnumber of potential PAM architectures have been explored in the literature. In\nthis paper we present an efficient and flexible amortized variational inference\nmethod for PAM, using a deep inference network to parameterize the approximate\nposterior distribution in a manner similar to the variational autoencoder. Our\ninference method produces more coherent topics than state-of-art inference\nmethods for PAM while being an order of magnitude faster, which allows\nexploration of a wider range of PAM architectures than have previously been\nstudied.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08234v1",
            "date": "2018-04-23 03:07:45+00:00",
            "title": "PeRView: A Framework for Personalized Review Selection Using Micro-Reviews",
            "authors": [
                "Muhmmad Al-Khiza'ay",
                "Noora Alallaq",
                "Qusay Alanoz",
                "Adil Al-Azzawi",
                "N. Maheswari"
            ],
            "abstract": "In the contemporary era, social media has its influence on people in making\ndecisions. The proliferation of online reviews with diversified and verbose\ncontent often causes problems inaccurate decision making. Since online reviews\nhave an impact on people of all walks of life while taking decisions, choosing\nappropriate reviews based on the podsolization consisting is very important\nsince it relies on using such micro-reviews consistency to evaluate the review\nset section. Micro-reviews are very concise and directly talk about product or\nservice instead of having unnecessary verbose content. Thus, micro-reviews can\nhelp in choosing reviews based on their personalized consistency that is\nrelated to directly or indirectly to the main profile of the reviews.\nPersonalized reviews selection that is highly relevant with high personalized\ncoverage in terms of matching with micro-reviews is the main problem that is\nconsidered in this paper. Furthermore, personalization with user preferences\nwhile making review selection is also considered based on the personalized\nusers' profile. Towards this end, we proposed a framework known as PeRView for\npersonalized review selection using micro-reviews based on the proposed\nevaluation metric approach which considering two main factors (personalized\nmatching score and subset size). Personalized Review Selection Algorithm (PRSA)\nis proposed which makes use of multiple similarity measures merged to have\nhighly efficient personalized reviews matching function for selection. The\nexperimental results based on using reviews dataset which is collected from\nYELP.COM while micro-reviews dataset is obtained from Foursqure.COM. show that\nthe personalized reviews selection is a very empirical case of study.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1804.08261v1",
            "date": "2018-04-23 06:52:13+00:00",
            "title": "Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network",
            "authors": [
                "Zhongliang Yang",
                "Yongfeng Huang",
                "Yiran Jiang",
                "Yuxi Sun",
                "Yu-Jin Zhan",
                "Pengcheng Luo"
            ],
            "abstract": "Automatically extracting useful information from electronic medical records\nalong with conducting disease diagnoses is a promising task for both clinical\ndecision support(CDS) and neural language processing(NLP). Most of the existing\nsystems are based on artificially constructed knowledge bases, and then\nauxiliary diagnosis is done by rule matching. In this study, we present a\nclinical intelligent decision approach based on Convolutional Neural\nNetworks(CNN), which can automatically extract high-level semantic information\nof electronic medical records and then perform automatic diagnosis without\nartificial construction of rules or knowledge bases. We use collected 18,590\ncopies of the real-world clinical electronic medical records to train and test\nthe proposed model. Experimental results show that the proposed model can\nachieve 98.67\\% accuracy and 96.02\\% recall, which strongly supports that using\nconvolutional neural network to automatically learn high-level semantic\nfeatures of electronic medical records and then conduct assist diagnosis is\nfeasible and effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08420v2",
            "date": "2018-04-18 21:33:00+00:00",
            "title": "Exploiting Partially Annotated Data for Temporal Relation Extraction",
            "authors": [
                "Qiang Ning",
                "Zhongzhi Yu",
                "Chuchu Fan",
                "Dan Roth"
            ],
            "abstract": "Annotating temporal relations (TempRel) between events described in natural\nlanguage is known to be labor intensive, partly because the total number of\nTempRels is quadratic in the number of events. As a result, only a small number\nof documents are typically annotated, limiting the coverage of various\nlexical/semantic phenomena. In order to improve existing approaches, one\npossibility is to make use of the readily available, partially annotated data\n(P as in partial) that cover more documents. However, missing annotations in P\nare known to hurt, rather than help, existing systems. This work is a case\nstudy in exploring various usages of P for TempRel extraction. Results show\nthat despite missing annotations, P is still a useful supervision signal for\nthis task within a constrained bootstrapping learning framework. The system\ndescribed in this system is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08438v2",
            "date": "2018-04-23 13:54:47+00:00",
            "title": "A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment",
            "authors": [
                "Tomi Kinnunen",
                "Jaime Lorenzo-Trueba",
                "Junichi Yamagishi",
                "Tomoki Toda",
                "Daisuke Saito",
                "Fernando Villavicencio",
                "Zhenhua Ling"
            ],
            "abstract": "Voice conversion (VC) aims at conversion of speaker characteristic without\naltering content. Due to training data limitations and modeling imperfections,\nit is difficult to achieve believable speaker mimicry without introducing\nprocessing artifacts; performance assessment of VC, therefore, usually involves\nboth speaker similarity and quality evaluation by a human panel. As a\ntime-consuming, expensive, and non-reproducible process, it hinders rapid\nprototyping of new VC technology. We address artifact assessment using an\nalternative, objective approach leveraging from prior work on spoofing\ncountermeasures (CMs) for automatic speaker verification. Therein, CMs are used\nfor rejecting `fake' inputs such as replayed, synthetic or converted speech but\ntheir potential for automatic speech artifact assessment remains unknown. This\nstudy serves to fill that gap. As a supplement to subjective results for the\n2018 Voice Conversion Challenge (VCC'18) data, we configure a standard\nconstant-Q cepstral coefficient CM to quantify the extent of processing\nartifacts. Equal error rate (EER) of the CM, a confusability index of VC\nsamples with real human speech, serves as our artifact measure. Two clusters of\nVCC'18 entries are identified: low-quality ones with detectable artifacts (low\nEERs), and higher quality ones with less artifacts. None of the VCC'18 systems,\nhowever, is perfect: all EERs are < 30 % (the `ideal' value would be 50 %). Our\npreliminary findings suggest potential of CMs outside of their original\napplication, as a supplemental optimization and benchmarking tool to enhance VC\ntechnology.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.08454v2",
            "date": "2018-04-23 14:11:17+00:00",
            "title": "Attention Based Natural Language Grounding by Navigating Virtual Environment",
            "authors": [
                "Akilesh B",
                "Abhishek Sinha",
                "Mausoom Sarkar",
                "Balaji Krishnamurthy"
            ],
            "abstract": "In this work, we focus on the problem of grounding language by training an\nagent to follow a set of natural language instructions and navigate to a target\nobject in an environment. The agent receives visual information through raw\npixels and a natural language instruction telling what task needs to be\nachieved and is trained in an end-to-end way. We develop an attention mechanism\nfor multi-modal fusion of visual and textual modalities that allows the agent\nto learn to complete the task and achieve language grounding. Our experimental\nresults show that our attention mechanism outperforms the existing multi-modal\nfusion mechanisms proposed for both 2D and 3D environments in order to solve\nthe above-mentioned task in terms of both speed and success rate. We show that\nthe learnt textual representations are semantically meaningful as they follow\nvector arithmetic in the embedding space. The effectiveness of our attention\napproach over the contemporary fusion mechanisms is also highlighted from the\ntextual embeddings learnt by the different approaches. We also show that our\nmodel generalizes effectively to unseen scenarios and exhibit zero-shot\ngeneralization capabilities both in 2D and 3D environments. The code for our 2D\nenvironment as well as the models that we developed for both 2D and 3D are\navailable at https://github.com/rl-lang-grounding/rl-lang-ground.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.08559v1",
            "date": "2018-04-23 16:52:49+00:00",
            "title": "False Information on Web and Social Media: A Survey",
            "authors": [
                "Srijan Kumar",
                "Neil Shah"
            ],
            "abstract": "False information can be created and spread easily through the web and social\nmedia platforms, resulting in widespread real-world impact. Characterizing how\nfalse information proliferates on social platforms and why it succeeds in\ndeceiving readers are critical to develop efficient detection algorithms and\ntools for early detection. A recent surge of research in this area has aimed to\naddress the key issues using methods based on feature engineering, graph\nmining, and information modeling. Majority of the research has primarily\nfocused on two broad categories of false information: opinion-based (e.g., fake\nreviews), and fact-based (e.g., false news and hoaxes). Therefore, in this\nwork, we present a comprehensive survey spanning diverse aspects of false\ninformation, namely (i) the actors involved in spreading false information,\n(ii) rationale behind successfully deceiving readers, (iii) quantifying the\nimpact of false information, (iv) measuring its characteristics across\ndifferent dimensions, and finally, (iv) algorithms developed to detect false\ninformation. In doing so, we create a unified framework to describe these\nrecent methods and highlight a number of important directions for future\nresearch.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.08782v1",
            "date": "2018-04-23 23:45:30+00:00",
            "title": "Towards an Unsupervised Entrainment Distance in Conversational Speech using Deep Neural Networks",
            "authors": [
                "Md Nasir",
                "Brian Baucom",
                "Shrikanth Narayanan",
                "Panayiotis Georgiou"
            ],
            "abstract": "Entrainment is a known adaptation mechanism that causes interaction\nparticipants to adapt or synchronize their acoustic characteristics.\nUnderstanding how interlocutors tend to adapt to each other's speaking style\nthrough entrainment involves measuring a range of acoustic features and\ncomparing those via multiple signal comparison methods. In this work, we\npresent a turn-level distance measure obtained in an unsupervised manner using\na Deep Neural Network (DNN) model, which we call Neural Entrainment Distance\n(NED). This metric establishes a framework that learns an embedding from the\npopulation-wide entrainment in an unlabeled training corpus. We use the\nframework for a set of acoustic features and validate the measure\nexperimentally by showing its efficacy in distinguishing real conversations\nfrom fake ones created by randomly shuffling speaker turns. Moreover, we show\nreal world evidence of the validity of the proposed measure. We find that high\nvalue of NED is associated with high ratings of emotional bond in suicide\nassessment interviews, which is consistent with prior studies.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.09028v1",
            "date": "2018-04-24 13:40:09+00:00",
            "title": "Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications",
            "authors": [
                "Guy Hadash",
                "Einat Kermany",
                "Boaz Carmeli",
                "Ofer Lavi",
                "George Kour",
                "Alon Jacovi"
            ],
            "abstract": "Existing applications include a huge amount of knowledge that is out of reach\nfor deep neural networks. This paper presents a novel approach for integrating\ncalls to existing applications into deep learning architectures. Using this\napproach, we estimate each application's functionality with an estimator, which\nis implemented as a deep neural network (DNN). The estimator is then embedded\ninto a base network that we direct into complying with the application's\ninterface during an end-to-end optimization process. At inference time, we\nreplace each estimator with its existing application counterpart and let the\nbase network solve the task by interacting with the existing application. Using\nthis 'Estimate and Replace' method, we were able to train a DNN end-to-end with\nless data and outperformed a matching DNN that did not interact with the\nexternal application.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1804.09148v1",
            "date": "2018-04-24 17:18:01+00:00",
            "title": "Automated Detection of Adverse Drug Reactions in the Biomedical Literature Using Convolutional Neural Networks and Biomedical Word Embeddings",
            "authors": [
                "Diego Saldana Miranda"
            ],
            "abstract": "Monitoring the biomedical literature for cases of Adverse Drug Reactions\n(ADRs) is a critically important and time consuming task in pharmacovigilance.\nThe development of computer assisted approaches to aid this process in\ndifferent forms has been the subject of many recent works. One particular area\nthat has shown promise is the use of Deep Neural Networks, in particular,\nConvolutional Neural Networks (CNNs), for the detection of ADR relevant\nsentences. Using token-level convolutions and general purpose word embeddings,\nthis architecture has shown good performance relative to more traditional\nmodels as well as Long Short Term Memory (LSTM) models. In this work, we\nevaluate and compare two different CNN architectures using the ADE corpus. In\naddition, we show that by de-duplicating the ADR relevant sentences, we can\ngreatly reduce overoptimism in the classification results. Finally, we evaluate\nthe use of word embeddings specifically developed for biomedical text and show\nthat they lead to a better performance in this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09160v2",
            "date": "2018-04-24 17:41:24+00:00",
            "title": "No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling",
            "authors": [
                "Xin Wang",
                "Wenhu Chen",
                "Yuan-Fang Wang",
                "William Yang Wang"
            ],
            "abstract": "Though impressive results have been achieved in visual captioning, the task\nof generating abstract stories from photo streams is still a little-tapped\nproblem. Different from captions, stories have more expressive language styles\nand contain many imaginary concepts that do not appear in the images. Thus it\nposes challenges to behavioral cloning algorithms. Furthermore, due to the\nlimitations of automatic metrics on evaluating story quality, reinforcement\nlearning methods with hand-crafted rewards also face difficulties in gaining an\noverall performance boost. Therefore, we propose an Adversarial REward Learning\n(AREL) framework to learn an implicit reward function from human\ndemonstrations, and then optimize policy search with the learned reward\nfunction. Though automatic eval- uation indicates slight performance boost over\nstate-of-the-art (SOTA) methods in cloning expert behaviors, human evaluation\nshows that our approach achieves significant improvement in generating more\nhuman-like stories than SOTA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09298v2",
            "date": "2018-04-25 00:24:39+00:00",
            "title": "Recent Progresses in Deep Learning based Acoustic Models (Updated)",
            "authors": [
                "Dong Yu",
                "Jinyu Li"
            ],
            "abstract": "In this paper, we summarize recent progresses made in deep learning based\nacoustic models and the motivation and insights behind the surveyed techniques.\nWe first discuss acoustic models that can effectively exploit variable-length\ncontextual information, such as recurrent neural networks (RNNs), convolutional\nneural networks (CNNs), and their various combination with other models. We\nthen describe acoustic models that are optimized end-to-end with emphasis on\nfeature representations learned jointly with rest of the system, the\nconnectionist temporal classification (CTC) criterion, and the attention-based\nsequence-to-sequence model. We further illustrate robustness issues in speech\nrecognition systems, and discuss acoustic model adaptation, speech enhancement\nand separation, and robust training strategies. We also cover modeling\ntechniques that lead to more efficient decoding and discuss possible future\ndirections in acoustic model research.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.09299v2",
            "date": "2018-04-25 00:32:45+00:00",
            "title": "Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models",
            "authors": [
                "Hendrik Strobelt",
                "Sebastian Gehrmann",
                "Michael Behrisch",
                "Adam Perer",
                "Hanspeter Pfister",
                "Alexander M. Rush"
            ],
            "abstract": "Neural Sequence-to-Sequence models have proven to be accurate and robust for\nmany sequence prediction tasks, and have become the standard approach for\nautomatic translation of text. The models work in a five stage blackbox process\nthat involves encoding a source sequence to a vector space and then decoding\nout to a new target sequence. This process is now standard, but like many deep\nlearning methods remains quite difficult to understand or debug. In this work,\nwe present a visual analysis tool that allows interaction with a trained\nsequence-to-sequence model through each stage of the translation process. The\naim is to identify which patterns have been learned and to detect model errors.\nWe demonstrate the utility of our tool through several real-world large-scale\nsequence-to-sequence use cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09530v1",
            "date": "2018-04-25 13:06:29+00:00",
            "title": "Strong Baselines for Neural Semi-supervised Learning under Domain Shift",
            "authors": [
                "Sebastian Ruder",
                "Barbara Plank"
            ],
            "abstract": "Novel neural models have been proposed in recent years for learning under\ndomain shift. Most models, however, only evaluate on a single task, on\nproprietary datasets, or compare to weak baselines, which makes comparison of\nmodels difficult. In this paper, we re-evaluate classic general-purpose\nbootstrapping approaches in the context of neural networks under domain shifts\nvs. recent neural approaches and propose a novel multi-task tri-training method\nthat reduces the time and space complexity of classic tri-training. Extensive\nexperiments on two benchmarks are negative: while our novel method establishes\na new state-of-the-art for sentiment analysis, it does not fare consistently\nthe best. More importantly, we arrive at the somewhat surprising conclusion\nthat classic tri-training, with some additions, outperforms the state of the\nart. We conclude that classic approaches constitute an important and strong\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09541v1",
            "date": "2018-04-23 11:33:43+00:00",
            "title": "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension",
            "authors": [
                "Adams Wei Yu",
                "David Dohan",
                "Minh-Thang Luong",
                "Rui Zhao",
                "Kai Chen",
                "Mohammad Norouzi",
                "Quoc V. Le"
            ],
            "abstract": "Current end-to-end machine reading and question answering (Q\\&A) models are\nprimarily based on recurrent neural networks (RNNs) with attention. Despite\ntheir success, these models are often slow for both training and inference due\nto the sequential nature of RNNs. We propose a new Q\\&A architecture called\nQANet, which does not require recurrent networks: Its encoder consists\nexclusively of convolution and self-attention, where convolution models local\ninteractions and self-attention models global interactions. On the SQuAD\ndataset, our model is 3x to 13x faster in training and 4x to 9x faster in\ninference, while achieving equivalent accuracy to recurrent models. The\nspeed-up gain allows us to train the model with much more data. We hence\ncombine our model with data generated by backtranslation from a neural machine\ntranslation model. On the SQuAD dataset, our single model, trained with\naugmented data, achieves 84.6 F1 score on the test set, which is significantly\nbetter than the best published F1 score of 81.8.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.09713v1",
            "date": "2018-04-25 22:54:06+00:00",
            "title": "End-to-End Multimodal Speech Recognition",
            "authors": [
                "Shruti Palaskar",
                "Ramon Sanabria",
                "Florian Metze"
            ],
            "abstract": "Transcription or sub-titling of open-domain videos is still a challenging\ndomain for Automatic Speech Recognition (ASR) due to the data's challenging\nacoustics, variable signal processing and the essentially unrestricted domain\nof the data. In previous work, we have shown that the visual channel --\nspecifically object and scene features -- can help to adapt the acoustic model\n(AM) and language model (LM) of a recognizer, and we are now expanding this\nwork to end-to-end approaches. In the case of a Connectionist Temporal\nClassification (CTC)-based approach, we retain the separation of AM and LM,\nwhile for a sequence-to-sequence (S2S) approach, both information sources are\nadapted together, in a single model. This paper also analyzes the behavior of\nCTC and S2S models on noisy video data (How-To corpus), and compares it to\nresults on the clean Wall Street Journal (WSJ) corpus, providing insight into\nthe robustness of both approaches.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.09843v1",
            "date": "2018-04-26 00:43:49+00:00",
            "title": "Hierarchical Density Order Embeddings",
            "authors": [
                "Ben Athiwaratkun",
                "Andrew Gordon Wilson"
            ],
            "abstract": "By representing words with probability densities rather than point vectors,\nprobabilistic word embeddings can capture rich and interpretable semantic\ninformation and uncertainty. The uncertainty information can be particularly\nmeaningful in capturing entailment relationships -- whereby general words such\nas \"entity\" correspond to broad distributions that encompass more specific\nwords such as \"animal\" or \"instrument\". We introduce density order embeddings,\nwhich learn hierarchical representations through encapsulation of probability\ndensities. In particular, we propose simple yet effective loss functions and\ndistance metrics, as well as graph-based schemes to select negative samples to\nbetter learn hierarchical density representations. Our approach provides\nstate-of-the-art performance on the WordNet hypernym relationship prediction\ntask and the challenging HyperLex lexical entailment dataset -- while retaining\na rich and interpretable density representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10080v1",
            "date": "2018-04-26 14:22:01+00:00",
            "title": "On deep speaker embeddings for text-independent speaker recognition",
            "authors": [
                "Sergey Novoselov",
                "Andrey Shulipa",
                "Ivan Kremnev",
                "Alexandr Kozlov",
                "Vadim Shchemelinin"
            ],
            "abstract": "We investigate deep neural network performance in the textindependent speaker\nrecognition task. We demonstrate that using angular softmax activation at the\nlast classification layer of a classification neural network instead of a\nsimple softmax activation allows to train a more generalized discriminative\nspeaker embedding extractor. Cosine similarity is an effective metric for\nspeaker verification in this embedding space. We also address the problem of\nchoosing an architecture for the extractor. We found that deep networks with\nresidual frame level connections outperform wide but relatively shallow\narchitectures. This paper also proposes several improvements for previous\nDNN-based extractor systems to increase the speaker recognition accuracy. We\nshow that the discriminatively trained similarity metric learning approach\noutperforms the standard LDA-PLDA method as an embedding backend. The results\nobtained on Speakers in the Wild and NIST SRE 2016 evaluation sets demonstrate\nrobustness of the proposed systems when dealing with close to real-life\nconditions.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1804.10202v1",
            "date": "2018-04-26 08:11:16+00:00",
            "title": "Sounding Board: A User-Centric and Content-Driven Social Chatbot",
            "authors": [
                "Hao Fang",
                "Hao Cheng",
                "Maarten Sap",
                "Elizabeth Clark",
                "Ari Holtzman",
                "Yejin Choi",
                "Noah A. Smith",
                "Mari Ostendorf"
            ],
            "abstract": "We present Sounding Board, a social chatbot that won the 2017 Amazon Alexa\nPrize. The system architecture consists of several components including spoken\nlanguage processing, dialogue management, language generation, and content\nmanagement, with emphasis on user-centric and content-driven design. We also\nshare insights gained from large-scale online logs based on 160,000\nconversations with real-world users.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1804.10752v2",
            "date": "2018-04-28 06:54:11+00:00",
            "title": "Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese",
            "authors": [
                "Shiyu Zhou",
                "Linhao Dong",
                "Shuang Xu",
                "Bo Xu"
            ],
            "abstract": "Sequence-to-sequence attention-based models have recently shown very\npromising results on automatic speech recognition (ASR) tasks, which integrate\nan acoustic, pronunciation and language model into a single neural network. In\nthese models, the Transformer, a new sequence-to-sequence attention-based model\nrelying entirely on self-attention without using RNNs or convolutions, achieves\na new single-model state-of-the-art BLEU on neural machine translation (NMT)\ntasks. Since the outstanding performance of the Transformer, we extend it to\nspeech and concentrate on it as the basic architecture of sequence-to-sequence\nattention-based model on Mandarin Chinese ASR tasks. Furthermore, we\ninvestigate a comparison between syllable based model and context-independent\nphoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese.\nAdditionally, a greedy cascading decoder with the Transformer is proposed for\nmapping CI-phoneme sequences and syllable sequences into word sequences.\nExperiments on HKUST datasets demonstrate that syllable based model with the\nTransformer performs better than CI-phoneme based counterpart, and achieves a\ncharacter error rate (CER) of \\emph{$28.77\\%$}, which is competitive to the\nstate-of-the-art CER of $28.0\\%$ by the joint CTC-attention based\nencoder-decoder network.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.10922v1",
            "date": "2018-04-29 12:49:29+00:00",
            "title": "OPA2Vec: combining formal and informal content of biomedical ontologies to improve similarity-based prediction",
            "authors": [
                "Fatima Zohra Smaili",
                "Xin Gao",
                "Robert Hoehndorf"
            ],
            "abstract": "Motivation: Ontologies are widely used in biology for data annotation,\nintegration, and analysis. In addition to formally structured axioms,\nontologies contain meta-data in the form of annotation axioms which provide\nvaluable pieces of information that characterize ontology classes. Annotations\ncommonly used in ontologies include class labels, descriptions, or synonyms.\nDespite being a rich source of semantic information, the ontology meta-data are\ngenerally unexploited by ontology-based analysis methods such as semantic\nsimilarity measures. Results: We propose a novel method, OPA2Vec, to generate\nvector representations of biological entities in ontologies by combining formal\nontology axioms and annotation axioms from the ontology meta-data. We apply a\nWord2Vec model that has been pre-trained on PubMed abstracts to produce feature\nvectors from our collected data. We validate our method in two different ways:\nfirst, we use the obtained vector representations of proteins as a similarity\nmeasure to predict protein-protein interaction (PPI) on two different datasets.\nSecond, we evaluate our method on predicting gene-disease associations based on\nphenotype similarity by generating vector representations of genes and diseases\nusing a phenotype ontology, and applying the obtained vectors to predict\ngene-disease associations. These two experiments are just an illustration of\nthe possible applications of our method. OPA2Vec can be used to produce vector\nrepresentations of any biomedical entity given any type of biomedical ontology.\nAvailability: https://github.com/bio-ontology-research-group/opa2vec Contact:\nrobert.hoehndorf@kaust.edu.sa and xin.gao@kaust.edu.sa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10974v1",
            "date": "2018-04-29 18:27:43+00:00",
            "title": "From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction",
            "authors": [
                "Zihang Dai",
                "Qizhe Xie",
                "Eduard Hovy"
            ],
            "abstract": "In this work, we study the credit assignment problem in reward augmented\nmaximum likelihood (RAML) learning, and establish a theoretical equivalence\nbetween the token-level counterpart of RAML and the entropy regularized\nreinforcement learning. Inspired by the connection, we propose two sequence\nprediction algorithms, one extending RAML with fine-grained credit assignment\nand the other improving Actor-Critic with a systematic entropy regularization.\nOn two benchmark datasets, we show the proposed algorithms outperform RAML and\nActor-Critic respectively, providing new alternatives to sequence prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11046v4",
            "date": "2018-04-30 04:41:33+00:00",
            "title": "Automatic Documentation of ICD Codes with Far-Field Speech Recognition",
            "authors": [
                "Albert Haque",
                "Corinna Fukushima"
            ],
            "abstract": "Documentation errors increase healthcare costs and cause unnecessary patient\ndeaths. As the standard language for diagnoses and billing, ICD codes serve as\nthe foundation for medical documentation worldwide. Despite the prevalence of\nelectronic medical records, hospitals still witness high levels of ICD\nmiscoding. In this paper, we propose to automatically document ICD codes with\nfar-field speech recognition. Far-field speech occurs when the microphone is\nlocated several meters from the source, as is common with smart homes and\nsecurity systems. Our method combines acoustic signal processing with recurrent\nneural networks to recognize and document ICD codes in real time. To evaluate\nour model, we collected a far-field speech dataset of ICD-10 codes and found\nour model to achieve 87% accuracy with a BLEU score of 85%. By sampling from an\nunsupervised medical language model, our method is able to outperform existing\nmethods. Overall, this work shows the potential of automatic speech recognition\nto provide efficient, accurate, and cost-effective healthcare documentation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1804.11067v1",
            "date": "2018-04-30 07:55:55+00:00",
            "title": "Staircase Network: structural language identification via hierarchical attentive units",
            "authors": [
                "Trung Ngo Trong",
                "Ville Hautam\u00e4ki",
                "Kristiina Jokinen"
            ],
            "abstract": "Language recognition system is typically trained directly to optimize\nclassification error on the target language labels, without using the external,\nor meta-information in the estimation of the model parameters. However labels\nare not independent of each other, there is a dependency enforced by, for\nexample, the language family, which affects negatively on classification. The\nother external information sources (e.g. audio encoding, telephony or video\nspeech) can also decrease classification accuracy. In this paper, we attempt to\nsolve these issues by constructing a deep hierarchical neural network, where\ndifferent levels of meta-information are encapsulated by attentive prediction\nunits and also embedded into the training progress. The proposed method learns\nauxiliary tasks to obtain robust internal representation and to construct a\nvariant of attentive units within the hierarchical model. The final result is\nthe structural prediction of the target language and a closely related language\nfamily. The algorithm reflects a \"staircase\" way of learning in both its\narchitecture and training, advancing from the fundamental audio encoding to the\nlanguage family level and finally to the target language level. This process\nnot only improves generalization but also tackles the issues of imbalanced\nclass priors and channel variability in the deep neural network model. Our\nexperimental findings show that the proposed architecture outperforms the\nstate-of-the-art i-vector approaches on both small and big language corpora by\na significant margin.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1804.11146v1",
            "date": "2018-04-30 12:14:32+00:00",
            "title": "Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings",
            "authors": [
                "Micael Carvalho",
                "R\u00e9mi Cad\u00e8ne",
                "David Picard",
                "Laure Soulier",
                "Nicolas Thome",
                "Matthieu Cord"
            ],
            "abstract": "Designing powerful tools that support cooking activities has rapidly gained\npopularity due to the massive amounts of available data, as well as recent\nadvances in machine learning that are capable of analyzing them. In this paper,\nwe propose a cross-modal retrieval model aligning visual and textual data (like\npictures of dishes and their recipes) in a shared representation space. We\ndescribe an effective learning scheme, capable of tackling large-scale\nproblems, and validate it on the Recipe1M dataset containing nearly 1 million\npicture-recipe pairs. We show the effectiveness of our approach regarding\nprevious state-of-the-art models and present qualitative results over\ncomputational cooking use cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.11243v1",
            "date": "2018-04-02 19:57:29+00:00",
            "title": "An Event Detection Approach Based On Twitter Hashtags",
            "authors": [
                "Shih-Feng Yang",
                "Julia Taylor Rayz"
            ],
            "abstract": "Twitter is one of the most popular microblogging services in the world. The\ngreat amount of information within Twitter makes it an important information\nchannel for people to learn and share news. Twitter hashtag is an popular\nfeature that can be viewed as human-labeled information which people use to\nidentify the topic of a tweet. Many researchers have proposed event-detection\napproaches that can monitor Twitter data and determine whether special events,\nsuch as accidents, extreme weather, earthquakes, or crimes take place. Although\nmany approaches use hashtags as one of their features, few of them explicitly\nfocus on the effectiveness of using hashtags on event detection. In this study,\nwe proposed an event detection approach that utilizes hashtags in tweets. We\nadopted the feature extraction used in STREAMCUBE and applied a clustering\nK-means approach to it. The experiments demonstrated that the K-means approach\nperformed better than STREAMCUBE in the clustering results. A discussion on\noptimal K values for the K-means approach is also provided.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1804.11258v3",
            "date": "2018-04-30 15:04:21+00:00",
            "title": "Toward Diverse Text Generation with Inverse Reinforcement Learning",
            "authors": [
                "Zhan Shi",
                "Xinchi Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Text generation is a crucial task in NLP. Recently, several adversarial\ngenerative models have been proposed to improve the exposure bias problem in\ntext generation. Though these models gain great success, they still suffer from\nthe problems of reward sparsity and mode collapse. In order to address these\ntwo problems, in this paper, we employ inverse reinforcement learning (IRL) for\ntext generation. Specifically, the IRL framework learns a reward function on\ntraining data, and then an optimal policy to maximum the expected total reward.\nSimilar to the adversarial models, the reward and policy function in IRL are\noptimized alternately. Our method has two advantages: (1) the reward function\ncan produce more dense reward signals. (2) the generation policy, trained by\n\"entropy regularized\" policy gradient, encourages to generate more diversified\ntexts. Experiment results demonstrate that our proposed method can generate\nhigher quality texts than the previous methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00265v1",
            "date": "2018-08-01 11:06:08+00:00",
            "title": "Interpretable Visual Question Answering by Visual Grounding from Attention Supervision Mining",
            "authors": [
                "Yundong Zhang",
                "Juan Carlos Niebles",
                "Alvaro Soto"
            ],
            "abstract": "A key aspect of VQA models that are interpretable is their ability to ground\ntheir answers to relevant regions in the image. Current approaches with this\ncapability rely on supervised learning and human annotated groundings to train\nattention mechanisms inside the VQA architecture. Unfortunately, obtaining\nhuman annotations specific for visual grounding is difficult and expensive. In\nthis work, we demonstrate that we can effectively train a VQA architecture with\ngrounding supervision that can be automatically obtained from available region\ndescriptions and object annotations. We also show that our model trained with\nthis mined supervision generates visual groundings that achieve a higher\ncorrelation with respect to manually-annotated groundings, meanwhile achieving\nstate-of-the-art VQA accuracy.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1808.00423v1",
            "date": "2018-08-01 17:04:48+00:00",
            "title": "Seq2Seq and Multi-Task Learning for joint intent and content extraction for domain specific interpreters",
            "authors": [
                "Marc Velay",
                "Fabrice Daniel"
            ],
            "abstract": "This study evaluates the performances of an LSTM network for detecting and\nextracting the intent and content of com- mands for a financial chatbot. It\npresents two techniques, sequence to sequence learning and Multi-Task Learning,\nwhich might improve on the previous task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.00563v1",
            "date": "2018-08-01 21:00:50+00:00",
            "title": "Data Augmentation for Robust Keyword Spotting under Playback Interference",
            "authors": [
                "Anirudh Raju",
                "Sankaran Panchapagesan",
                "Xing Liu",
                "Arindam Mandal",
                "Nikko Strom"
            ],
            "abstract": "Accurate on-device keyword spotting (KWS) with low false accept and false\nreject rate is crucial to customer experience for far-field voice control of\nconversational agents. It is particularly challenging to maintain low false\nreject rate in real world conditions where there is (a) ambient noise from\nexternal sources such as TV, household appliances, or other speech that is not\ndirected at the device (b) imperfect cancellation of the audio playback from\nthe device, resulting in residual echo, after being processed by the Acoustic\nEcho Cancellation (AEC) system. In this paper, we propose a data augmentation\nstrategy to improve keyword spotting performance under these challenging\nconditions. The training set audio is artificially corrupted by mixing in music\nand TV/movie audio, at different signal to interference ratios. Our results\nshow that we get around 30-45% relative reduction in false reject rates, at a\nrange of false alarm rates, under audio playback from such devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.00665v1",
            "date": "2018-08-02 04:25:30+00:00",
            "title": "Investigating accuracy of pitch-accent annotations in neural network-based speech synthesis and denoising effects",
            "authors": [
                "Hieu-Thi Luong",
                "Xin Wang",
                "Junichi Yamagishi",
                "Nobuyuki Nishizawa"
            ],
            "abstract": "We investigated the impact of noisy linguistic features on the performance of\na Japanese speech synthesis system based on neural network that uses WaveNet\nvocoder. We compared an ideal system that uses manually corrected linguistic\nfeatures including phoneme and prosodic information in training and test sets\nagainst a few other systems that use corrupted linguistic features. Both\nsubjective and objective results demonstrate that corrupted linguistic\nfeatures, especially those in the test set, affected the ideal system's\nperformance significantly in a statistical sense due to a mismatched condition\nbetween the training and test sets. Interestingly, while an utterance-level\nTuring test showed that listeners had a difficult time differentiating\nsynthetic speech from natural speech, it further indicated that adding noise to\nthe linguistic features in the training set can partially reduce the effect of\nthe mismatch, regularize the model, and help the system perform better when\nlinguistic features of the test set are noisy.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1808.01175v1",
            "date": "2018-08-03 12:57:15+00:00",
            "title": "Content-driven, unsupervised clustering of news articles through multiscale graph partitioning",
            "authors": [
                "M. Tarik Altuncu",
                "Sophia N. Yaliraki",
                "Mauricio Barahona"
            ],
            "abstract": "The explosion in the amount of news and journalistic content being generated\nacross the globe, coupled with extended and instantaneous access to information\nthrough online media, makes it difficult and time-consuming to monitor news\ndevelopments and opinion formation in real time. There is an increasing need\nfor tools that can pre-process, analyse and classify raw text to extract\ninterpretable content; specifically, identifying topics and content-driven\ngroupings of articles. We present here such a methodology that brings together\npowerful vector embeddings from Natural Language Processing with tools from\nGraph Theory that exploit diffusive dynamics on graphs to reveal natural\npartitions across scales. Our framework uses a recent deep neural network text\nanalysis methodology (Doc2vec) to represent text in vector form and then\napplies a multi-scale community detection method (Markov Stability) to\npartition a similarity graph of document vectors. The method allows us to\nobtain clusters of documents with similar content, at different levels of\nresolution, in an unsupervised manner. We showcase our approach with the\nanalysis of a corpus of 9,000 news articles published by Vox Media over one\nyear. Our results show consistent groupings of documents according to content\nwithout a priori assumptions about the number or type of clusters to be found.\nThe multilevel clustering reveals a quasi-hierarchy of topics and subtopics\nwith increased intelligibility and improved topic coherence as compared to\nexternal taxonomy services and standard topic detection methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01371v2",
            "date": "2018-08-03 21:44:29+00:00",
            "title": "Large Scale Language Modeling: Converging on 40GB of Text in Four Hours",
            "authors": [
                "Raul Puri",
                "Robert Kirby",
                "Nikolai Yakovenko",
                "Bryan Catanzaro"
            ],
            "abstract": "Recent work has shown how to train Convolutional Neural Networks (CNNs)\nrapidly on large image datasets, then transfer the knowledge gained from these\nmodels to a variety of tasks. Following [Radford 2017], in this work, we\ndemonstrate similar scalability and transfer for Recurrent Neural Networks\n(RNNs) for Natural Language tasks. By utilizing mixed precision arithmetic and\na 32k batch size distributed across 128 NVIDIA Tesla V100 GPUs, we are able to\ntrain a character-level 4096-dimension multiplicative LSTM (mLSTM) for\nunsupervised text reconstruction over 3 epochs of the 40 GB Amazon Reviews\ndataset in four hours. This runtime compares favorably with previous work\ntaking one month to train the same size and configuration for one epoch over\nthe same dataset. Converging large batch RNN models can be challenging. Recent\nwork has suggested scaling the learning rate as a function of batch size, but\nwe find that simply scaling the learning rate as a function of batch size leads\neither to significantly worse convergence or immediate divergence for this\nproblem. We provide a learning rate schedule that allows our model to converge\nwith a 32k batch size. Since our model converges over the Amazon Reviews\ndataset in hours, and our compute requirement of 128 Tesla V100 GPUs, while\nsubstantial, is commercially available, this work opens up large scale\nunsupervised NLP training to most commercial applications and deep learning\nresearchers. A model can be trained over most public or private text datasets\novernight.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.01535v1",
            "date": "2018-08-04 21:10:03+00:00",
            "title": "Triplet Network with Attention for Speaker Diarization",
            "authors": [
                "Huan Song",
                "Megan Willi",
                "Jayaraman J. Thiagarajan",
                "Visar Berisha",
                "Andreas Spanias"
            ],
            "abstract": "In automatic speech processing systems, speaker diarization is a crucial\nfront-end component to separate segments from different speakers. Inspired by\nthe recent success of deep neural networks (DNNs) in semantic inferencing,\ntriplet loss-based architectures have been successfully used for this problem.\nHowever, existing work utilizes conventional i-vectors as the input\nrepresentation and builds simple fully connected networks for metric learning,\nthus not fully leveraging the modeling power of DNN architectures. This paper\ninvestigates the importance of learning effective representations from the\nsequences directly in metric learning pipelines for speaker diarization. More\nspecifically, we propose to employ attention models to learn embeddings and the\nmetric jointly in an end-to-end fashion. Experiments are conducted on the\nCALLHOME conversational speech corpus. The diarization results demonstrate\nthat, besides providing a unified model, the proposed approach achieves\nimproved performance when compared against existing approaches.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1808.01591v1",
            "date": "2018-08-05 09:50:47+00:00",
            "title": "LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation",
            "authors": [
                "Pankaj Gupta",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Recurrent neural networks (RNNs) are temporal networks and cumulative in\nnature that have shown promising results in various natural language processing\ntasks. Despite their success, it still remains a challenge to understand their\nhidden behavior. In this work, we analyze and interpret the cumulative nature\nof RNN via a proposed technique named as Layer-wIse-Semantic-Accumulation\n(LISA) for explaining decisions and detecting the most likely (i.e., saliency)\npatterns that the network relies on while decision making. We demonstrate (1)\nLISA: \"How an RNN accumulates or builds semantics during its sequential\nprocessing for a given text example and expected response\" (2) Example2pattern:\n\"How the saliency patterns look like for each category in the data according to\nthe network in decision making\". We analyse the sensitiveness of RNNs about\ndifferent inputs to check the increase or decrease in prediction scores and\nfurther extract the saliency patterns learned by the network. We employ two\nrelation classification datasets: SemEval 10 Task 8 and TAC KBP Slot Filling to\nexplain RNN predictions via the LISA and example2pattern.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01843v1",
            "date": "2018-08-06 12:24:19+00:00",
            "title": "An Efficient Approach to Learning Chinese Judgment Document Similarity Based on Knowledge Summarization",
            "authors": [
                "Yinglong Ma",
                "Peng Zhang",
                "Jiangang Ma"
            ],
            "abstract": "A previous similar case in common law systems can be used as a reference with\nrespect to the current case such that identical situations can be treated\nsimilarly in every case. However, current approaches for judgment document\nsimilarity computation failed to capture the core semantics of judgment\ndocuments and therefore suffer from lower accuracy and higher computation\ncomplexity. In this paper, a knowledge block summarization based machine\nlearning approach is proposed to compute the semantic similarity of Chinese\njudgment documents. By utilizing domain ontologies for judgment documents, the\ncore semantics of Chinese judgment documents is summarized based on knowledge\nblocks. Then the WMD algorithm is used to calculate the similarity between\nknowledge blocks. At last, the related experiments were made to illustrate that\nour approach is very effective and efficient in achieving higher accuracy and\nfaster computation speed in comparison with the traditional approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.01916v1",
            "date": "2018-08-06 14:00:40+00:00",
            "title": "Residual Memory Networks: Feed-forward approach to learn long temporal dependencies",
            "authors": [
                "Murali Karthick Baskar",
                "Martin Karafiat",
                "Lukas Burget",
                "Karel Vesely",
                "Frantisek Grezl",
                "Jan Honza Cernocky"
            ],
            "abstract": "Training deep recurrent neural network (RNN) architectures is complicated due\nto the increased network complexity. This disrupts the learning of higher order\nabstracts using deep RNN. In case of feed-forward networks training deep\nstructures is simple and faster while learning long-term temporal information\nis not possible. In this paper we propose a residual memory neural network\n(RMN) architecture to model short-time dependencies using deep feed-forward\nlayers having residual and time delayed connections. The residual connection\npaves way to construct deeper networks by enabling unhindered flow of gradients\nand the time delay units capture temporal information with shared weights. The\nnumber of layers in RMN signifies both the hierarchical processing depth and\ntemporal depth. The computational complexity in training RMN is significantly\nless when compared to deep recurrent networks. RMN is further extended as\nbi-directional RMN (BRMN) to capture both past and future information.\nExperimental analysis is done on AMI corpus to substantiate the capability of\nRMN in learning long-term information and hierarchical information. Recognition\nperformance of RMN trained with 300 hours of Switchboard corpus is compared\nwith various state-of-the-art LVCSR systems. The results indicate that RMN and\nBRMN gains 6 % and 3.8 % relative improvement over LSTM and BLSTM networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.01935v1",
            "date": "2018-08-06 14:40:31+00:00",
            "title": "Audio Tagging With Connectionist Temporal Classification Model Using Sequential Labelled Data",
            "authors": [
                "Yuanbo Hou",
                "Qiuqiang Kong",
                "Shengchen Li"
            ],
            "abstract": "Audio tagging aims to predict one or several labels in an audio clip. Many\nprevious works use weakly labelled data (WLD) for audio tagging, where only\npresence or absence of sound events is known, but the order of sound events is\nunknown. To use the order information of sound events, we propose sequential\nlabelled data (SLD), where both the presence or absence and the order\ninformation of sound events are known. To utilize SLD in audio tagging, we\npropose a Convolutional Recurrent Neural Network followed by a Connectionist\nTemporal Classification (CRNN-CTC) objective function to map from an audio clip\nspectrogram to SLD. Experiments show that CRNN-CTC obtains an Area Under Curve\n(AUC) score of 0.986 in audio tagging, outperforming the baseline CRNN of 0.908\nand 0.815 with Max Pooling and Average Pooling, respectively. In addition, we\nshow CRNN-CTC has the ability to predict the order of sound events in an audio\nclip.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1808.02113v1",
            "date": "2018-08-06 21:05:55+00:00",
            "title": "Paying Attention to Attention: Highlighting Influential Samples in Sequential Analysis",
            "authors": [
                "Cynthia Freeman",
                "Jonathan Merriman",
                "Abhinav Aggarwal",
                "Ian Beaver",
                "Abdullah Mueen"
            ],
            "abstract": "In (Yang et al. 2016), a hierarchical attention network (HAN) is created for\ndocument classification. The attention layer can be used to visualize text\ninfluential in classifying the document, thereby explaining the model's\nprediction. We successfully applied HAN to a sequential analysis task in the\nform of real-time monitoring of turn taking in conversations. However, we\ndiscovered instances where the attention weights were uniform at the stopping\npoint (indicating all turns were equivalently influential to the classifier),\npreventing meaningful visualization for real-time human review or classifier\nimprovement. We observed that attention weights for turns fluctuated as the\nconversations progressed, indicating turns had varying influence based on\nconversation state. Leveraging this observation, we develop a method to create\nmore informative real-time visuals (as confirmed by human reviewers) in cases\nof uniform attention weights using the changes in turn importance as a\nconversation progresses over time.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.02632v1",
            "date": "2018-08-08 05:39:00+00:00",
            "title": "Question-Guided Hybrid Convolution for Visual Question Answering",
            "authors": [
                "Peng Gao",
                "Pan Lu",
                "Hongsheng Li",
                "Shuang Li",
                "Yikang Li",
                "Steven Hoi",
                "Xiaogang Wang"
            ],
            "abstract": "In this paper, we propose a novel Question-Guided Hybrid Convolution (QGHC)\nnetwork for Visual Question Answering (VQA). Most state-of-the-art VQA methods\nfuse the high-level textual and visual features from the neural network and\nabandon the visual spatial information when learning multi-modal features.To\naddress these problems, question-guided kernels generated from the input\nquestion are designed to convolute with visual features for capturing the\ntextual and visual relationship in the early stage. The question-guided\nconvolution can tightly couple the textual and visual information but also\nintroduce more parameters when learning kernels. We apply the group\nconvolution, which consists of question-independent kernels and\nquestion-dependent kernels, to reduce the parameter size and alleviate\nover-fitting. The hybrid convolution can generate discriminative multi-modal\nfeatures with fewer parameters. The proposed approach is also complementary to\nexisting bilinear pooling fusion and attention based VQA methods. By\nintegrating with them, our method could further boost the performance.\nExtensive experiments on public VQA datasets validate the effectiveness of\nQGHC.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1808.02939v1",
            "date": "2018-08-08 20:59:26+00:00",
            "title": "Towards Learning Fine-Grained Disentangled Representations from Speech",
            "authors": [
                "Yuan Gong",
                "Christian Poellabauer"
            ],
            "abstract": "Learning disentangled representations of high-dimensional data is currently\nan active research area. However, compared to the field of computer vision,\nless work has been done for speech processing. In this paper, we provide a\nreview of two representative efforts on this topic and propose the novel\nconcept of fine-grained disentangled speech representation learning.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1808.03227v1",
            "date": "2018-07-27 19:08:12+00:00",
            "title": "Identifying Protein-Protein Interaction using Tree LSTM and Structured Attention",
            "authors": [
                "Mahtab Ahmed",
                "Jumayel Islam",
                "Muhammad Rifayat Samee",
                "Robert E. Mercer"
            ],
            "abstract": "Identifying interactions between proteins is important to understand\nunderlying biological processes. Extracting a protein-protein interaction (PPI)\nfrom the raw text is often very difficult. Previous supervised learning methods\nhave used handcrafted features on human-annotated data sets. In this paper, we\npropose a novel tree recurrent neural network with structured attention\narchitecture for doing PPI. Our architecture achieves state of the art results\n(precision, recall, and F1-score) on the AIMed and BioInfer benchmark data\nsets. Moreover, our models achieve a significant improvement over previous best\nmodels without any explicit feature extraction. Our experimental results show\nthat traditional recurrent networks have inferior performance compared to tree\nrecurrent networks for the supervised PPI problem.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1808.03370v1",
            "date": "2018-08-09 23:09:16+00:00",
            "title": "Fast Flexible Function Dispatch in Julia",
            "authors": [
                "Jeff Bezanson",
                "Jake Bolewski",
                "Jiahao Chen"
            ],
            "abstract": "Technical computing is a challenging application area for programming\nlanguages to address. This is evinced by the unusually large number of\nspecialized languages in the area (e.g. MATLAB, R), and the complexity of\ncommon software stacks, often involving multiple languages and custom code\ngenerators. We believe this is ultimately due to key characteristics of the\ndomain: highly complex operators, a need for extensive code specialization for\nperformance, and a desire for permissive high-level programming styles allowing\nproductive experimentation. The Julia language attempts to provide a more\neffective structure for this kind of programming by allowing programmers to\nexpress complex polymorphic behaviors using dynamic multiple dispatch over\nparametric types. The forms of extension and reuse permitted by this paradigm\nhave proven valuable for technical computing. We report on how this approach\nhas allowed domain experts to express useful abstractions while simultaneously\nproviding a natural path to better performance for high-level technical code.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03703v2",
            "date": "2018-08-10 20:46:32+00:00",
            "title": "LemmaTag: Jointly Tagging and Lemmatizing for Morphologically-Rich Languages with BRNNs",
            "authors": [
                "Daniel Kondratyuk",
                "Tom\u00e1\u0161 Gaven\u010diak",
                "Milan Straka",
                "Jan Haji\u010d"
            ],
            "abstract": "We present LemmaTag, a featureless neural network architecture that jointly\ngenerates part-of-speech tags and lemmas for sentences by using bidirectional\nRNNs with character-level and word-level embeddings. We demonstrate that both\ntasks benefit from sharing the encoding part of the network, predicting tag\nsubcategories, and using the tagger output as an input to the lemmatizer. We\nevaluate our model across several languages with complex morphology, which\nsurpasses state-of-the-art accuracy in both part-of-speech tagging and\nlemmatization in Czech, German, and Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03726v3",
            "date": "2018-08-10 23:21:07+00:00",
            "title": "Learning to Represent Bilingual Dictionaries",
            "authors": [
                "Muhao Chen",
                "Yingtao Tian",
                "Haochen Chen",
                "Kai-Wei Chang",
                "Steven Skiena",
                "Carlo Zaniolo"
            ],
            "abstract": "Bilingual word embeddings have been widely used to capture the similarity of\nlexical semantics in different human languages. However, many applications,\nsuch as cross-lingual semantic search and question answering, can be largely\nbenefited from the cross-lingual correspondence between sentences and lexicons.\nTo bridge this gap, we propose a neural embedding model that leverages\nbilingual dictionaries. The proposed model is trained to map the literal word\ndefinitions to the cross-lingual target words, for which we explore with\ndifferent sentence encoding techniques. To enhance the learning process on\nlimited resources, our model adopts several critical learning strategies,\nincluding multi-task learning on different bridges of languages, and joint\nlearning of the dictionary model with a bilingual word embedding model.\nExperimental evaluation focuses on two applications. The results of the\ncross-lingual reverse dictionary retrieval task show our model's promising\nability of comprehending bilingual concepts based on descriptions, and\nhighlight the effectiveness of proposed learning strategies in improving\nperformance. Meanwhile, our model effectively addresses the bilingual\nparaphrase identification problem and significantly outperforms previous\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03733v2",
            "date": "2018-08-11 01:14:50+00:00",
            "title": "Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering",
            "authors": [
                "Di Jiang",
                "Yuanfeng Song",
                "Rongzhong Lian",
                "Siqi Bao",
                "Jinhua Peng",
                "Huang He",
                "Hua Wu"
            ],
            "abstract": "In the last decade, a variety of topic models have been proposed for text\nengineering. However, except Probabilistic Latent Semantic Analysis (PLSA) and\nLatent Dirichlet Allocation (LDA), most of existing topic models are seldom\napplied or considered in industrial scenarios. This phenomenon is caused by the\nfact that there are very few convenient tools to support these topic models so\nfar. Intimidated by the demanding expertise and labor of designing and\nimplementing parameter inference algorithms, software engineers are prone to\nsimply resort to PLSA/LDA, without considering whether it is proper for their\nproblem at hand or not. In this paper, we propose a configurable topic modeling\nframework named Familia, in order to bridge the huge gap between academic\nresearch fruits and current industrial practice. Familia supports an important\nline of topic models that are widely applicable in text engineering scenarios.\nIn order to relieve burdens of software engineers without knowledge of Bayesian\nnetworks, Familia is able to conduct automatic parameter inference for a\nvariety of topic models. Simply through changing the data organization of\nFamilia, software engineers are able to easily explore a broad spectrum of\nexisting topic models or even design their own topic models, and find the one\nthat best suits the problem at hand. With its superior extendability, Familia\nhas a novel sampling mechanism that strikes balance between effectiveness and\nefficiency of parameter inference. Furthermore, Familia is essentially a big\ntopic modeling framework that supports parallel parameter inference and\ndistributed parameter storage. The utilities and necessity of Familia are\ndemonstrated in real-life industrial applications. Familia would significantly\nenlarge software engineers' arsenal of topic models and pave the way for\nutilizing highly customized topic models in real-life problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03793v1",
            "date": "2018-08-11 12:16:09+00:00",
            "title": "Document Informed Neural Autoregressive Topic Models",
            "authors": [
                "Pankaj Gupta",
                "Florian Buettner",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Context information around words helps in determining their actual meaning,\nfor example \"networks\" used in contexts of artificial neural networks or\nbiological neuron networks. Generative topic models infer topic-word\ndistributions, taking no or only little context into account. Here, we extend a\nneural autoregressive topic model to exploit the full context information\naround words in a document in a language modeling fashion. This results in an\nimproved performance in terms of generalization, interpretability and\napplicability. We apply our modeling approach to seven data sets from various\ndomains and demonstrate that our approach consistently outperforms\nstateof-the-art generative topic models. With the learned representations, we\nshow on an average a gain of 9.6% (0.57 Vs 0.52) in precision at retrieval\nfraction 0.02 and 7.2% (0.582 Vs 0.543) in F1 for text categorization.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.03835v1",
            "date": "2018-08-11 16:47:58+00:00",
            "title": "jLDADMM: A Java package for the LDA and DMM topic models",
            "authors": [
                "Dat Quoc Nguyen"
            ],
            "abstract": "In this technical report, we present jLDADMM---an easy-to-use Java toolkit\nfor conventional topic models. jLDADMM is released to provide alternatives for\ntopic modeling on normal or short texts. It provides implementations of the\nLatent Dirichlet Allocation topic model and the one-topic-per-document\nDirichlet Multinomial Mixture model (i.e. mixture of unigrams), using collapsed\nGibbs sampling. In addition, jLDADMM supplies a document clustering evaluation\nto compare topic models. jLDADMM is open-source and available to download at:\nhttps://github.com/datquocnguyen/jLDADMM",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.03894v1",
            "date": "2018-08-12 05:42:26+00:00",
            "title": "Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference",
            "authors": [
                "Reza Ghaeini",
                "Xiaoli Z. Fern",
                "Prasad Tadepalli"
            ],
            "abstract": "Deep learning models have achieved remarkable success in natural language\ninference (NLI) tasks. While these models are widely explored, they are hard to\ninterpret and it is often unclear how and why they actually work. In this\npaper, we take a step toward explaining such deep learning based models through\na case study on a popular neural model for NLI. In particular, we propose to\ninterpret the intermediate layers of NLI models by visualizing the saliency of\nattention and LSTM gating signals. We present several examples for which our\nmethods are able to reveal interesting insights and identify the critical\ninformation contributing to the model decisions.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03986v2",
            "date": "2018-08-12 18:56:56+00:00",
            "title": "Multimodal Differential Network for Visual Question Generation",
            "authors": [
                "Badri N. Patro",
                "Sandeep Kumar",
                "Vinod K. Kurmi",
                "Vinay P. Namboodiri"
            ],
            "abstract": "Generating natural questions from an image is a semantic task that requires\nusing visual and language modality to learn multimodal representations. Images\ncan have multiple visual and language contexts that are relevant for generating\nquestions namely places, captions, and tags. In this paper, we propose the use\nof exemplars for obtaining the relevant context. We obtain this by using a\nMultimodal Differential Network to produce natural and engaging questions. The\ngenerated questions show a remarkable similarity to the natural questions as\nvalidated by a human study. Further, we observe that the proposed approach\nsubstantially improves over state-of-the-art benchmarks on the quantitative\nmetrics (BLEU, METEOR, ROUGE, and CIDEr).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04108v1",
            "date": "2018-08-13 08:46:42+00:00",
            "title": "Towards Audio to Scene Image Synthesis using Generative Adversarial Network",
            "authors": [
                "Chia-Hung Wan",
                "Shun-Po Chuang",
                "Hung-Yi Lee"
            ],
            "abstract": "Humans can imagine a scene from a sound. We want machines to do so by using\nconditional generative adversarial networks (GANs). By applying the techniques\nincluding spectral norm, projection discriminator and auxiliary classifier,\ncompared with naive conditional GAN, the model can generate images with better\nquality in terms of both subjective and objective evaluations. Almost\nthree-fourth of people agree that our model have the ability to generate images\nrelated to sounds. By inputting different volumes of the same sound, our model\noutput different scales of changes based on the volumes, showing that our model\ntruly knows the relationship between sounds and images to some extent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04216v1",
            "date": "2018-08-10 10:09:54+00:00",
            "title": "Effective Unsupervised Author Disambiguation with Relative Frequencies",
            "authors": [
                "Tobias Backes"
            ],
            "abstract": "This work addresses the problem of author name homonymy in the Web of\nScience. Aiming for an efficient, simple and straightforward solution, we\nintroduce a novel probabilistic similarity measure for author name\ndisambiguation based on feature overlap. Using the researcher-ID available for\na subset of the Web of Science, we evaluate the application of this measure in\nthe context of agglomeratively clustering author mentions. We focus on a\nconcise evaluation that shows clearly for which problem setups and at which\ntime during the clustering process our approach works best. In contrast to most\nother works in this field, we are sceptical towards the performance of author\nname disambiguation methods in general and compare our approach to the trivial\nsingle-cluster baseline. Our results are presented separately for each correct\nclustering size as we can explain that, when treating all cases together, the\ntrivial baseline and more sophisticated approaches are hardly distinguishable\nin terms of evaluation results. Our model shows state-of-the-art performance\nfor all correct clustering sizes without any discriminative training and with\ntuning only one convergence parameter.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.04334v1",
            "date": "2018-08-13 17:20:20+00:00",
            "title": "Angular-Based Word Meta-Embedding Learning",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "Ensembling word embeddings to improve distributed word representations has\nshown good success for natural language processing tasks in recent years. These\napproaches either carry out straightforward mathematical operations over a set\nof vectors or use unsupervised learning to find a lower-dimensional\nrepresentation. This work compares meta-embeddings trained for different\nlosses, namely loss functions that account for angular distance between the\nreconstructed embedding and the target and those that account normalized\ndistances based on the vector length. We argue that meta-embeddings are better\nto treat the ensemble set equally in unsupervised learning as the respective\nquality of each embedding is unknown for upstream tasks prior to\nmeta-embedding. We show that normalization methods that account for this such\nas cosine and KL-divergence objectives outperform meta-embedding trained on\nstandard $\\ell_1$ and $\\ell_2$ loss on \\textit{defacto} word similarity and\nrelatedness datasets and find it outperforms existing meta-learning strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04359v2",
            "date": "2018-08-10 22:09:43+00:00",
            "title": "Community Regularization of Visually-Grounded Dialog",
            "authors": [
                "Akshat Agarwal",
                "Swaminathan Gurumurthy",
                "Vasu Sharma",
                "Mike Lewis",
                "Katia Sycara"
            ],
            "abstract": "The task of conducting visually grounded dialog involves learning\ngoal-oriented cooperative dialog between autonomous agents who exchange\ninformation about a scene through several rounds of questions and answers in\nnatural language. We posit that requiring artificial agents to adhere to the\nrules of human language, while also requiring them to maximize information\nexchange through dialog is an ill-posed problem. We observe that humans do not\nstray from a common language because they are social creatures who live in\ncommunities, and have to communicate with many people everyday, so it is far\neasier to stick to a common language even at the cost of some efficiency loss.\nUsing this as inspiration, we propose and evaluate a multi-agent\ncommunity-based dialog framework where each agent interacts with, and learns\nfrom, multiple agents, and show that this community-enforced regularization\nresults in more relevant and coherent dialog (as judged by human evaluators)\nwithout sacrificing task performance (as judged by quantitative metrics).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1808.04444v2",
            "date": "2018-08-09 18:44:38+00:00",
            "title": "Character-Level Language Modeling with Deeper Self-Attention",
            "authors": [
                "Rami Al-Rfou",
                "Dokook Choe",
                "Noah Constant",
                "Mandy Guo",
                "Llion Jones"
            ],
            "abstract": "LSTMs and other RNN variants have shown strong performance on character-level\nlanguage modeling. These models are typically trained using truncated\nbackpropagation through time, and it is common to assume that their success\nstems from their ability to remember long-term contexts. In this paper, we show\nthat a deep (64-layer) transformer model with fixed context outperforms RNN\nvariants by a large margin, achieving state of the art on two popular\nbenchmarks: 1.13 bits per character on text8 and 1.06 on enwik8. To get good\nresults at this depth, we show that it is important to add auxiliary losses,\nboth at intermediate network layers and intermediate sequence positions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04446v2",
            "date": "2018-08-03 14:32:02+00:00",
            "title": "Visual Reasoning with Multi-hop Feature Modulation",
            "authors": [
                "Florian Strub",
                "Mathieu Seurin",
                "Ethan Perez",
                "Harm de Vries",
                "J\u00e9r\u00e9mie Mary",
                "Philippe Preux",
                "Aaron Courville",
                "Olivier Pietquin"
            ],
            "abstract": "Recent breakthroughs in computer vision and natural language processing have\nspurred interest in challenging multi-modal tasks such as visual\nquestion-answering and visual dialogue. For such tasks, one successful approach\nis to condition image-based convolutional network computation on language via\nFeature-wise Linear Modulation (FiLM) layers, i.e., per-channel scaling and\nshifting. We propose to generate the parameters of FiLM layers going up the\nhierarchy of a convolutional network in a multi-hop fashion rather than all at\nonce, as in prior work. By alternating between attending to the language input\nand generating FiLM layer parameters, this approach is better able to scale to\nsettings with longer input sequences such as dialogue. We demonstrate that\nmulti-hop FiLM generation achieves state-of-the-art for the short input\nsequence task ReferIt --- on-par with single-hop FiLM generation --- while also\nsignificantly outperforming prior state-of-the-art and single-hop FiLM\ngeneration on the GuessWhat?! visual dialogue task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1808.04538v1",
            "date": "2018-08-14 05:45:25+00:00",
            "title": "Text-to-Image-to-Text Translation using Cycle Consistent Adversarial Networks",
            "authors": [
                "Satya Krishna Gorti",
                "Jeremy Ma"
            ],
            "abstract": "Text-to-Image translation has been an active area of research in the recent\npast. The ability for a network to learn the meaning of a sentence and generate\nan accurate image that depicts the sentence shows ability of the model to think\nmore like humans. Popular methods on text to image translation make use of\nGenerative Adversarial Networks (GANs) to generate high quality images based on\ntext input, but the generated images don't always reflect the meaning of the\nsentence given to the model as input. We address this issue by using a\ncaptioning network to caption on generated images and exploit the distance\nbetween ground truth captions and generated captions to improve the network\nfurther. We show extensive comparisons between our method and existing methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.04706v2",
            "date": "2018-08-08 22:26:08+00:00",
            "title": "Neural Machine Translation Inspired Binary Code Similarity Comparison beyond Function Pairs",
            "authors": [
                "Fei Zuo",
                "Xiaopeng Li",
                "Patrick Young",
                "Lannan Luo",
                "Qiang Zeng",
                "Zhexin Zhang"
            ],
            "abstract": "Binary code analysis allows analyzing binary code without having access to\nthe corresponding source code. A binary, after disassembly, is expressed in an\nassembly language. This inspires us to approach binary analysis by leveraging\nideas and techniques from Natural Language Processing (NLP), a rich area\nfocused on processing text of various natural languages. We notice that binary\ncode analysis and NLP share a lot of analogical topics, such as semantics\nextraction, summarization, and classification. This work utilizes these ideas\nto address two important code similarity comparison problems. (I) Given a pair\nof basic blocks for different instruction set architectures (ISAs), determining\nwhether their semantics is similar or not; and (II) given a piece of code of\ninterest, determining if it is contained in another piece of assembly code for\na different ISA. The solutions to these two problems have many applications,\nsuch as cross-architecture vulnerability discovery and code plagiarism\ndetection. We implement a prototype system INNEREYE and perform a comprehensive\nevaluation. A comparison between our approach and existing approaches to\nProblem I shows that our system outperforms them in terms of accuracy,\nefficiency and scalability. And the case studies utilizing the system\ndemonstrate that our solution to Problem II is effective. Moreover, this\nresearch showcases how to apply ideas and techniques from NLP to large-scale\nbinary code analysis.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1808.04926v2",
            "date": "2018-08-14 23:59:26+00:00",
            "title": "How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks",
            "authors": [
                "Divyansh Kaushik",
                "Zachary C. Lipton"
            ],
            "abstract": "Many recent papers address reading comprehension, where examples consist of\n(question, passage, answer) tuples. Presumably, a model must combine\ninformation from both questions and passages to predict corresponding answers.\nHowever, despite intense interest in the topic, with hundreds of published\npapers vying for leaderboard dominance, basic questions about the difficulty of\nmany popular benchmarks remain unanswered. In this paper, we establish sensible\nbaselines for the bAbI, SQuAD, CBT, CNN, and Who-did-What datasets, finding\nthat question- and passage-only models often perform surprisingly well. On $14$\nout of $20$ bAbI tasks, passage-only models achieve greater than $50\\%$\naccuracy, sometimes matching the full model. Interestingly, while CBT provides\n$20$-sentence stories only the last is needed for comparably accurate\nprediction. By comparison, SQuAD and CNN appear better-constructed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.04928v1",
            "date": "2018-08-15 00:10:55+00:00",
            "title": "Deep EHR: Chronic Disease Prediction Using Medical Notes",
            "authors": [
                "Jingshu Liu",
                "Zachariah Zhang",
                "Narges Razavian"
            ],
            "abstract": "Early detection of preventable diseases is important for better disease\nmanagement, improved inter-ventions, and more efficient health-care resource\nallocation. Various machine learning approacheshave been developed to utilize\ninformation in Electronic Health Record (EHR) for this task. Majorityof\nprevious attempts, however, focus on structured fields and lose the vast amount\nof information inthe unstructured notes. In this work we propose a general\nmulti-task framework for disease onsetprediction that combines both free-text\nmedical notes and structured information. We compareperformance of different\ndeep learning architectures including CNN, LSTM and hierarchical models.In\ncontrast to traditional text-based prediction models, our approach does not\nrequire disease specificfeature engineering, and can handle negations and\nnumerical values that exist in the text. Ourresults on a cohort of about 1\nmillion patients show that models using text outperform modelsusing just\nstructured data, and that models capable of using numerical values and\nnegations in thetext, in addition to the raw text, further improve performance.\nAdditionally, we compare differentvisualization methods for medical\nprofessionals to interpret model predictions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.05535v1",
            "date": "2018-08-16 15:19:34+00:00",
            "title": "Combining time-series and textual data for taxi demand prediction in event areas: a deep learning approach",
            "authors": [
                "Filipe Rodrigues",
                "Ioulia Markou",
                "Francisco Pereira"
            ],
            "abstract": "Accurate time-series forecasting is vital for numerous areas of application\nsuch as transportation, energy, finance, economics, etc. However, while modern\ntechniques are able to explore large sets of temporal data to build forecasting\nmodels, they typically neglect valuable information that is often available\nunder the form of unstructured text. Although this data is in a radically\ndifferent format, it often contains contextual explanations for many of the\npatterns that are observed in the temporal data. In this paper, we propose two\ndeep learning architectures that leverage word embeddings, convolutional layers\nand attention mechanisms for combining text information with time-series data.\nWe apply these approaches for the problem of taxi demand forecasting in event\nareas. Using publicly available taxi data from New York, we empirically show\nthat by fusing these two complementary cross-modal sources of information, the\nproposed models are able to significantly reduce the error in the forecasts.",
            "journal ref": "Rodrigues, F., Markou, I., Pereira, F. Combining time-series and\n  textual data for taxi demand prediction in event areas: a deep learning\n  approach. In Information Fusion, Elsevier, 2018",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1808.05697v3",
            "date": "2018-08-16 22:46:40+00:00",
            "title": "Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study",
            "authors": [
                "Aditya Siddhant",
                "Zachary C. Lipton"
            ],
            "abstract": "Several recent papers investigate Active Learning (AL) for mitigating the\ndata dependence of deep learning for natural language processing. However, the\napplicability of AL to real-world problems remains an open question. While in\nsupervised learning, practitioners can try many different methods, evaluating\neach against a validation set before selecting a model, AL affords no such\nluxury. Over the course of one AL run, an agent annotates its dataset\nexhausting its labeling budget. Thus, given a new task, an active learner has\nno opportunity to compare models and acquisition functions. This paper provides\na large scale empirical study of deep active learning, addressing multiple\ntasks and, for each, multiple datasets, multiple models, and a full suite of\nacquisition functions. We find that across all settings, Bayesian active\nlearning by disagreement, using uncertainty estimates provided either by\nDropout or Bayes-by Backprop significantly improves over i.i.d. baselines and\nusually outperforms classic uncertainty sampling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05857v1",
            "date": "2018-07-21 00:19:13+00:00",
            "title": "ELICA: An Automated Tool for Dynamic Extraction of Requirements Relevant Information",
            "authors": [
                "Zahra Shakeri Hossein Abad",
                "Vincenzo Gervasi",
                "Didar Zowghi",
                "Ken Barker"
            ],
            "abstract": "Requirements elicitation requires extensive knowledge and deep understanding\nof the problem domain where the final system will be situated. However, in many\nsoftware development projects, analysts are required to elicit the requirements\nfrom an unfamiliar domain, which often causes communication barriers between\nanalysts and stakeholders. In this paper, we propose a requirements ELICitation\nAid tool (ELICA) to help analysts better understand the target application\ndomain by dynamic extraction and labeling of requirements-relevant knowledge.\nTo extract the relevant terms, we leverage the flexibility and power of\nWeighted Finite State Transducers (WFSTs) in dynamic modeling of natural\nlanguage processing tasks. In addition to the information conveyed through\ntext, ELICA captures and processes non-linguistic information about the\nintention of speakers such as their confidence level, analytical tone, and\nemotions. The extracted information is made available to the analysts as a set\nof labeled snippets with highlighted relevant terms which can also be exported\nas an artifact of the Requirements Engineering (RE) process. The application\nand usefulness of ELICA are demonstrated through a case study. This study shows\nhow pre-existing relevant information about the application domain and the\ninformation captured during an elicitation meeting, such as the conversation\nand stakeholders' intentions, can be captured and used to support analysts\nachieving their tasks.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1808.05906v1",
            "date": "2018-08-16 09:02:37+00:00",
            "title": "Story Disambiguation: Tracking Evolving News Stories across News and Social Streams",
            "authors": [
                "Bichen Shi",
                "Thanh-Binh Le",
                "Neil Hurley",
                "Georgiana Ifrim"
            ],
            "abstract": "Following a particular news story online is an important but difficult task,\nas the relevant information is often scattered across different domains/sources\n(e.g., news articles, blogs, comments, tweets), presented in various formats\nand language styles, and may overlap with thousands of other stories. In this\nwork we join the areas of topic tracking and entity disambiguation, and propose\na framework named Story Disambiguation - a cross-domain story tracking approach\nthat builds on real-time entity disambiguation and a learning-to-rank framework\nto represent and update the rich semantic structure of news stories. Given a\ntarget news story, specified by a seed set of documents, the goal is to\neffectively select new story-relevant documents from an incoming document\nstream. We represent stories as entity graphs and we model the story tracking\nproblem as a learning-to-rank task. This enables us to track content with high\naccuracy, from multiple domains, in real-time. We study a range of text, entity\nand graph based features to understand which type of features are most\neffective for representing stories. We further propose new semi-supervised\nlearning techniques to automatically update the story representation over time.\nOur empirical study shows that we outperform the accuracy of state-of-the-art\nmethods for tracking mixed-domain document streams, while requiring fewer\nlabeled data to seed the tracked stories. This is particularly the case for\nlocal news stories that are easily over shadowed by other trending stories, and\nfor complex news stories with ambiguous content in noisy stream environments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.05927v1",
            "date": "2018-08-17 16:47:52+00:00",
            "title": "Characterizing the public perception of WhatsApp through the lens of media",
            "authors": [
                "Josemar Alves Caetano",
                "Gabriel Magno",
                "Evandro Cunha",
                "Wagner Meira Jr.",
                "Humberto T. Marques-Neto",
                "Virgilio Almeida"
            ],
            "abstract": "WhatsApp is, as of 2018, a significant component of the global information\nand communication infrastructure, especially in developing countries. However,\nprobably due to its strong end-to-end encryption, WhatsApp became an attractive\nplace for the dissemination of misinformation, extremism and other forms of\nundesirable behavior. In this paper, we investigate the public perception of\nWhatsApp through the lens of media. We analyze two large datasets of news and\nshow the kind of content that is being associated with WhatsApp in different\nregions of the world and over time. Our analyses include the examination of\nnamed entities, general vocabulary, and topics addressed in news articles that\nmention WhatsApp, as well as the polarity of these texts. Among other results,\nwe demonstrate that the vocabulary and topics around the term \"whatsapp\" in the\nmedia have been changing over the years and in 2018 concentrate on matters\nrelated to misinformation, politics and criminal scams. More generally, our\nfindings are useful to understand the impact that tools like WhatsApp play in\nthe contemporary society and how they are seen by the communities themselves.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1808.06021v2",
            "date": "2018-08-17 23:50:01+00:00",
            "title": "What do the US West Coast Public Libraries Post on Twitter?",
            "authors": [
                "Amir Karami",
                "Matthew Collins"
            ],
            "abstract": "Twitter has provided a great opportunity for public libraries to disseminate\ninformation for a variety of purposes. Twitter data have been applied in\ndifferent domains such as health, politics, and history. There are thousands of\npublic libraries in the US, but no study has yet investigated the content of\ntheir social media posts like tweets to find their interests. Moreover,\ntraditional content analysis of Twitter content is not an efficient task for\nexploring thousands of tweets. Therefore, there is a need for automatic methods\nto overcome the limitations of manual methods. This paper proposes a\ncomputational approach to collecting and analyzing using Twitter Application\nProgramming Interfaces (API) and investigates more than 138,000 tweets from 48\nUS west coast libraries using topic modeling. We found 20 topics and assigned\nthem to five categories including public relations, book, event, training, and\nsocial good. Our results show that the US west coast libraries are more\ninterested in using Twitter for public relations and book-related events. This\nresearch has both practical and theoretical applications for libraries as well\nas other organizations to explore social media actives of their customer and\nthemselves.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1808.06022v2",
            "date": "2018-08-18 00:00:19+00:00",
            "title": "Characterizing Transgender Health Issues in Twitter",
            "authors": [
                "Amir Karami",
                "Frank Webb",
                "Vanessa L. Kitzie"
            ],
            "abstract": "Although there are millions of transgender people in the world, a lack of\ninformation exists about their health issues. This issue has consequences for\nthe medical field, which only has a nascent understanding of how to identify\nand meet this population's health-related needs. Social media sites like\nTwitter provide new opportunities for transgender people to overcome these\nbarriers by sharing their personal health experiences. Our research employs a\ncomputational framework to collect tweets from self-identified transgender\nusers, detect those that are health-related, and identify their information\nneeds. This framework is significant because it provides a macro-scale\nperspective on an issue that lacks investigation at national or demographic\nlevels. Our findings identified 54 distinct health-related topics that we\ngrouped into 7 broader categories. Further, we found both linguistic and\ntopical differences in the health-related information shared by transgender men\n(TM) as com-pared to transgender women (TW). These findings can help inform\nmedical and policy-based strategies for health interventions within transgender\ncommunities. Also, our proposed approach can inform the development of\ncomputational strategies to identify the health-related information needs of\nother marginalized populations.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1808.06170v1",
            "date": "2018-08-19 06:21:58+00:00",
            "title": "Linked Recurrent Neural Networks",
            "authors": [
                "Zhiwei Wang",
                "Yao Ma",
                "Dawei Yin",
                "Jiliang Tang"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) have been proven to be effective in modeling\nsequential data and they have been applied to boost a variety of tasks such as\ndocument classification, speech recognition and machine translation. Most of\nexisting RNN models have been designed for sequences assumed to be identically\nand independently distributed (i.i.d). However, in many real-world\napplications, sequences are naturally linked. For example, web documents are\nconnected by hyperlinks; and genes interact with each other. On the one hand,\nlinked sequences are inherently not i.i.d., which poses tremendous challenges\nto existing RNN models. On the other hand, linked sequences offer link\ninformation in addition to the sequential information, which enables\nunprecedented opportunities to build advanced RNN models. In this paper, we\nstudy the problem of RNN for linked sequences. In particular, we introduce a\nprincipled approach to capture link information and propose a linked Recurrent\nNeural Network (LinkedRNN), which models sequential and link information\ncoherently. We conduct experiments on real-world datasets from multiple domains\nand the experimental results validate the effectiveness of the proposed\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06288v1",
            "date": "2018-08-20 02:36:19+00:00",
            "title": "Multimodal speech synthesis architecture for unsupervised speaker adaptation",
            "authors": [
                "Hieu-Thi Luong",
                "Junichi Yamagishi"
            ],
            "abstract": "This paper proposes a new architecture for speaker adaptation of\nmulti-speaker neural-network speech synthesis systems, in which an unseen\nspeaker's voice can be built using a relatively small amount of speech data\nwithout transcriptions. This is sometimes called \"unsupervised speaker\nadaptation\". More specifically, we concatenate the layers to the audio inputs\nwhen performing unsupervised speaker adaptation while we concatenate them to\nthe text inputs when synthesizing speech from text. Two new training schemes\nfor the new architecture are also proposed in this paper. These training\nschemes are not limited to speech synthesis, other applications are suggested.\nExperimental results show that the proposed model not only enables adaptation\nto unseen speakers using untranscribed speech but it also improves the\nperformance of multi-speaker modeling and speaker adaptation using transcribed\naudio files.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1808.06640v2",
            "date": "2018-08-20 18:20:01+00:00",
            "title": "Adversarial Removal of Demographic Attributes from Text Data",
            "authors": [
                "Yanai Elazar",
                "Yoav Goldberg"
            ],
            "abstract": "Recent advances in Representation Learning and Adversarial Training seem to\nsucceed in removing unwanted features from the learned representation. We show\nthat demographic information of authors is encoded in -- and can be recovered\nfrom -- the intermediate representations learned by text-based neural\nclassifiers. The implication is that decisions of classifiers trained on\ntextual data are not agnostic to -- and likely condition on -- demographic\nattributes. When attempting to remove such demographic information using\nadversarial training, we find that while the adversarial component achieves\nchance-level development-set accuracy during training, a post-hoc classifier,\ntrained on the encoded sentences from the first part, still manages to reach\nsubstantially higher classification accuracies on the same data. This behavior\nis consistent across several tasks, demographic properties and datasets. We\nexplore several techniques to improve the effectiveness of the adversarial\ncomponent. Our main conclusion is a cautionary one: do not rely on the\nadversarial training to achieve invariant representation to sensitive features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.06885v3",
            "date": "2018-08-21 13:07:53+00:00",
            "title": "Multi-Source Pointer Network for Product Title Summarization",
            "authors": [
                "Fei Sun",
                "Peng Jiang",
                "Hanxiao Sun",
                "Changhua Pei",
                "Wenwu Ou",
                "Xiaobo Wang"
            ],
            "abstract": "In this paper, we study the product title summarization problem in E-commerce\napplications for display on mobile devices. Comparing with conventional\nsentence summarization, product title summarization has some extra and\nessential constraints. For example, factual errors or loss of the key\ninformation are intolerable for E-commerce applications. Therefore, we abstract\ntwo more constraints for product title summarization: (i) do not introduce\nirrelevant information; (ii) retain the key information (e.g., brand name and\ncommodity name). To address these issues, we propose a novel multi-source\npointer network by adding a new knowledge encoder for pointer network. The\nfirst constraint is handled by pointer mechanism. For the second constraint, we\nrestore the key information by copying words from the knowledge encoder with\nthe help of the soft gating mechanism. For evaluation, we build a large\ncollection of real-world product titles along with human-written short titles.\nExperimental results demonstrate that our model significantly outperforms the\nother baselines. Finally, online deployment of our proposed model has yielded a\nsignificant business impact, as measured by the click-through rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07036v3",
            "date": "2018-08-21 17:46:12+00:00",
            "title": "QuAC : Question Answering in Context",
            "authors": [
                "Eunsol Choi",
                "He He",
                "Mohit Iyyer",
                "Mark Yatskar",
                "Wen-tau Yih",
                "Yejin Choi",
                "Percy Liang",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present QuAC, a dataset for Question Answering in Context that contains\n14K information-seeking QA dialogs (100K questions in total). The dialogs\ninvolve two crowd workers: (1) a student who poses a sequence of freeform\nquestions to learn as much as possible about a hidden Wikipedia text, and (2) a\nteacher who answers the questions by providing short excerpts from the text.\nQuAC introduces challenges not found in existing machine comprehension\ndatasets: its questions are often more open-ended, unanswerable, or only\nmeaningful within the dialog context, as we show in a detailed qualitative\nevaluation. We also report results for a number of reference models, including\na recently state-of-the-art reading comprehension architecture extended to\nmodel dialog context. Our best model underperforms humans by 20 F1, suggesting\nthat there is significant room for future work on this data. Dataset, baseline,\nand leaderboard available at http://quac.ai.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07042v2",
            "date": "2018-08-21 17:52:02+00:00",
            "title": "CoQA: A Conversational Question Answering Challenge",
            "authors": [
                "Siva Reddy",
                "Danqi Chen",
                "Christopher D. Manning"
            ],
            "abstract": "Humans gather information by engaging in conversations involving a series of\ninterconnected questions and answers. For machines to assist in information\ngathering, it is therefore essential to enable them to answer conversational\nquestions. We introduce CoQA, a novel dataset for building Conversational\nQuestion Answering systems. Our dataset contains 127k questions with answers,\nobtained from 8k conversations about text passages from seven diverse domains.\nThe questions are conversational, and the answers are free-form text with their\ncorresponding evidence highlighted in the passage. We analyze CoQA in depth and\nshow that conversational questions have challenging phenomena not present in\nexisting reading comprehension datasets, e.g., coreference and pragmatic\nreasoning. We evaluate strong conversational and reading comprehension models\non CoQA. The best system obtains an F1 score of 65.4%, which is 23.4 points\nbehind human performance (88.8%), indicating there is ample room for\nimprovement. We launch CoQA as a challenge to the community at\nhttp://stanfordnlp.github.io/coqa/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07187v2",
            "date": "2018-08-22 02:18:40+00:00",
            "title": "Neural Latent Extractive Document Summarization",
            "authors": [
                "Xingxing Zhang",
                "Mirella Lapata",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Extractive summarization models require sentence-level labels, which are\nusually created heuristically (e.g., with rule-based methods) given that most\nsummarization datasets only have document-summary pairs. Since these labels\nmight be suboptimal, we propose a latent variable extractive model where\nsentences are viewed as latent variables and sentences with activated variables\nare used to infer gold summaries. During training the loss comes\n\\emph{directly} from gold summaries. Experiments on the CNN/Dailymail dataset\nshow that our model improves over a strong extractive baseline trained on\nheuristically approximated labels and also performs competitively to several\nrecent models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07228v1",
            "date": "2018-08-22 05:57:13+00:00",
            "title": "Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents",
            "authors": [
                "Ganbin Zhou",
                "Rongyu Cao",
                "Xiang Ao",
                "Ping Luo",
                "Fen Lin",
                "Leyu Lin",
                "Qing He"
            ],
            "abstract": "In this study, we focus on extracting knowledgeable snippets and annotating\nknowledgeable documents from Web corpus, consisting of the documents from\nsocial media and We-media. Informally, knowledgeable snippets refer to the text\ndescribing concepts, properties of entities, or relations among entities, while\nknowledgeable documents are the ones with enough knowledgeable snippets. These\nknowledgeable snippets and documents could be helpful in multiple applications,\nsuch as knowledge base construction and knowledge-oriented service. Previous\nstudies extracted the knowledgeable snippets using the pattern-based method.\nHere, we propose the semantic-based method for this task. Specifically, a CNN\nbased model is developed to extract knowledgeable snippets and annotate\nknowledgeable documents simultaneously. Additionally, a \"low-level sharing,\nhigh-level splitting\" structure of CNN is designed to handle the documents from\ndifferent content domains. Compared with building multiple domain-specific\nCNNs, this joint model not only critically saves the training time, but also\nimproves the prediction accuracy visibly. The superiority of the proposed\nmethod is demonstrated in a real dataset from Wechat public platform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07383v1",
            "date": "2018-08-22 14:30:03+00:00",
            "title": "Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding",
            "authors": [
                "Deunsol Yoon",
                "Dongbok Lee",
                "SangKeun Lee"
            ],
            "abstract": "In this paper, we propose Dynamic Self-Attention (DSA), a new self-attention\nmechanism for sentence embedding. We design DSA by modifying dynamic routing in\ncapsule network (Sabouretal.,2017) for natural language processing. DSA attends\nto informative words with a dynamic weight vector. We achieve new\nstate-of-the-art results among sentence encoding methods in Stanford Natural\nLanguage Inference (SNLI) dataset with the least number of parameters, while\nshowing comparative results in Stanford Sentiment Treebank (SST) dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.07561v2",
            "date": "2018-08-22 20:53:37+00:00",
            "title": "Training Deeper Neural Machine Translation Models with Transparent Attention",
            "authors": [
                "Ankur Bapna",
                "Mia Xu Chen",
                "Orhan Firat",
                "Yuan Cao",
                "Yonghui Wu"
            ],
            "abstract": "While current state-of-the-art NMT models, such as RNN seq2seq and\nTransformers, possess a large number of parameters, they are still shallow in\ncomparison to convolutional models used for both text and vision applications.\nIn this work we attempt to train significantly (2-3x) deeper Transformer and\nBi-RNN encoders for machine translation. We propose a simple modification to\nthe attention mechanism that eases the optimization of deeper models, and\nresults in consistent gains of 0.7-1.1 BLEU on the benchmark WMT'14\nEnglish-German and WMT'15 Czech-English tasks for both architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07645v3",
            "date": "2018-08-23 06:34:32+00:00",
            "title": "Playing 20 Question Game with Policy-Based Reinforcement Learning",
            "authors": [
                "Huang Hu",
                "Xianchao Wu",
                "Bingfeng Luo",
                "Chongyang Tao",
                "Can Xu",
                "Wei Wu",
                "Zhan Chen"
            ],
            "abstract": "The 20 Questions (Q20) game is a well known game which encourages deductive\nreasoning and creativity. In the game, the answerer first thinks of an object\nsuch as a famous person or a kind of animal. Then the questioner tries to guess\nthe object by asking 20 questions. In a Q20 game system, the user is considered\nas the answerer while the system itself acts as the questioner which requires a\ngood strategy of question selection to figure out the correct object and win\nthe game. However, the optimal policy of question selection is hard to be\nderived due to the complexity and volatility of the game environment. In this\npaper, we propose a novel policy-based Reinforcement Learning (RL) method,\nwhich enables the questioner agent to learn the optimal policy of question\nselection through continuous interactions with users. To facilitate training,\nwe also propose to use a reward network to estimate the more informative\nreward. Compared to previous methods, our RL method is robust to noisy answers\nand does not rely on the Knowledge Base of objects. Experimental results show\nthat our RL method clearly outperforms an entropy-based engineering system and\nhas competitive performance in a noisy-free simulation environment.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1808.07724v1",
            "date": "2018-08-23 12:47:01+00:00",
            "title": "Mapping Text to Knowledge Graph Entities using Multi-Sense LSTMs",
            "authors": [
                "Dimitri Kartsaklis",
                "Mohammad Taher Pilehvar",
                "Nigel Collier"
            ],
            "abstract": "This paper addresses the problem of mapping natural language text to\nknowledge base entities. The mapping process is approached as a composition of\na phrase or a sentence into a point in a multi-dimensional entity space\nobtained from a knowledge graph. The compositional model is an LSTM equipped\nwith a dynamic disambiguation mechanism on the input word embeddings (a\nMulti-Sense LSTM), addressing polysemy issues. Further, the knowledge base\nspace is prepared by collecting random walks from a graph enhanced with textual\nfeatures, which act as a set of semantic bridges between text and knowledge\nbase entities. The ideas of this work are demonstrated on large-scale\ntext-to-entity mapping and entity classification tasks, with state of the art\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.07910v1",
            "date": "2018-08-23 19:17:24+00:00",
            "title": "The Importance of Generation Order in Language Modeling",
            "authors": [
                "Nicolas Ford",
                "Daniel Duckworth",
                "Mohammad Norouzi",
                "George E. Dahl"
            ],
            "abstract": "Neural language models are a critical component of state-of-the-art systems\nfor machine translation, summarization, audio transcription, and other tasks.\nThese language models are almost universally autoregressive in nature,\ngenerating sentences one token at a time from left to right. This paper studies\nthe influence of token generation order on model quality via a novel two-pass\nlanguage model that produces partially-filled sentence \"templates\" and then\nfills in missing tokens. We compare various strategies for structuring these\ntwo passes and observe a surprisingly large variation in model quality. We find\nthe most effective strategy generates function words in the first pass followed\nby content words in the second. We believe these experimental results justify a\nmore extensive investigation of generation order for neural language models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.07982v1",
            "date": "2018-08-24 02:14:43+00:00",
            "title": "Proximal Policy Optimization and its Dynamic Version for Sequence Generation",
            "authors": [
                "Yi-Lin Tuan",
                "Jinzhi Zhang",
                "Yujia Li",
                "Hung-yi Lee"
            ],
            "abstract": "In sequence generation task, many works use policy gradient for model\noptimization to tackle the intractable backpropagation issue when maximizing\nthe non-differentiable evaluation metrics or fooling the discriminator in\nadversarial learning. In this paper, we replace policy gradient with proximal\npolicy optimization (PPO), which is a proved more efficient reinforcement\nlearning algorithm, and propose a dynamic approach for PPO (PPO-dynamic). We\ndemonstrate the efficacy of PPO and PPO-dynamic on conditional sequence\ngeneration tasks including synthetic experiment and chit-chat chatbot. The\nresults show that PPO and PPO-dynamic can beat policy gradient by stability and\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08149v3",
            "date": "2018-08-24 14:17:01+00:00",
            "title": "From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information",
            "authors": [
                "Hengru Xu",
                "Shen Li",
                "Renfen Hu",
                "Si Li",
                "Sheng Gao"
            ],
            "abstract": "Dropout is used to avoid overfitting by randomly dropping units from the\nneural networks during training. Inspired by dropout, this paper presents\nGI-Dropout, a novel dropout method integrating with global information to\nimprove neural networks for text classification. Unlike the traditional dropout\nmethod in which the units are dropped randomly according to the same\nprobability, we aim to use explicit instructions based on global information of\nthe dataset to guide the training process. With GI-Dropout, the model is\nsupposed to pay more attention to inapparent features or patterns. Experiments\ndemonstrate the effectiveness of the dropout with global information on seven\ntext classification tasks, including sentiment analysis and topic\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08270v5",
            "date": "2018-08-24 19:22:12+00:00",
            "title": "Robust Text Classifier on Test-Time Budgets",
            "authors": [
                "Md Rizwan Parvez",
                "Tolga Bolukbasi",
                "Kai-Wei Chang",
                "Venkatesh Saligrama"
            ],
            "abstract": "We propose a generic and interpretable learning framework for building robust\ntext classification model that achieves accuracy comparable to full models\nunder test-time budget constraints. Our approach learns a selector to identify\nwords that are relevant to the prediction tasks and passes them to the\nclassifier for processing. The selector is trained jointly with the classifier\nand directly learns to incorporate with the classifier. We further propose a\ndata aggregation scheme to improve the robustness of the classifier. Our\nlearning framework is general and can be incorporated with any type of text\nclassification model. On real-world data, we show that the proposed approach\nimproves the performance of a given classifier and speeds up the model with a\nmere loss in accuracy performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.08316v3",
            "date": "2018-08-24 21:29:53+00:00",
            "title": "A Trio Neural Model for Dynamic Entity Relatedness Ranking",
            "authors": [
                "Tu Ngoc Nguyen",
                "Tuan Tran",
                "Wolfgang Nejdl"
            ],
            "abstract": "Measuring entity relatedness is a fundamental task for many natural language\nprocessing and information retrieval applications. Prior work often studies\nentity relatedness in static settings and an unsupervised manner. However,\nentities in real-world are often involved in many different relationships,\nconsequently entity-relations are very dynamic over time. In this work, we\npropose a neural networkbased approach for dynamic entity relatedness,\nleveraging the collective attention as supervision. Our model is capable of\nlearning rich and different entity representations in a joint framework.\nThrough extensive experiments on large-scale datasets, we demonstrate that our\nmethod achieves better results than competitive baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.08493v1",
            "date": "2018-08-26 01:17:50+00:00",
            "title": "Contextual Parameter Generation for Universal Neural Machine Translation",
            "authors": [
                "Emmanouil Antonios Platanios",
                "Mrinmaya Sachan",
                "Graham Neubig",
                "Tom Mitchell"
            ],
            "abstract": "We propose a simple modification to existing neural machine translation (NMT)\nmodels that enables using a single universal model to translate between\nmultiple languages while allowing for language specific parameterization, and\nthat can also be used for domain adaptation. Our approach requires no changes\nto the model architecture of a standard NMT system, but instead introduces a\nnew component, the contextual parameter generator (CPG), that generates the\nparameters of the system (e.g., weights in a neural network). This parameter\ngenerator accepts source and target language embeddings as input, and generates\nthe parameters for the encoder and the decoder, respectively. The rest of the\nmodel remains unchanged and is shared across all languages. We show how this\nsimple modification enables the system to use monolingual data for training and\nalso perform zero-shot translation. We further show it is able to surpass\nstate-of-the-art performance for both the IWSLT-15 and IWSLT-17 datasets and\nthat the learned language embeddings are able to uncover interesting\nrelationships between languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08538v1",
            "date": "2018-08-26 11:58:04+00:00",
            "title": "Nowcasting the Stance of Social Media Users in a Sudden Vote: The Case of the Greek Referendum",
            "authors": [
                "Adam Tsakalidis",
                "Nikolaos Aletras",
                "Alexandra I. Cristea",
                "Maria Liakata"
            ],
            "abstract": "Modelling user voting intention in social media is an important research\narea, with applications in analysing electorate behaviour, online political\ncampaigning and advertising. Previous approaches mainly focus on predicting\nnational general elections, which are regularly scheduled and where data of\npast results and opinion polls are available. However, there is no evidence of\nhow such models would perform during a sudden vote under time-constrained\ncircumstances. That poses a more challenging task compared to traditional\nelections, due to its spontaneous nature. In this paper, we focus on the 2015\nGreek bailout referendum, aiming to nowcast on a daily basis the voting\nintention of 2,197 Twitter users. We propose a semi-supervised multiple\nconvolution kernel learning approach, leveraging temporally sensitive text and\nnetwork information. Our evaluation under a real-time simulation framework\ndemonstrates the effectiveness and robustness of our approach against\ncompetitive baselines, achieving a significant 20% increase in F-score compared\nto solely text-based models.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1808.08609v1",
            "date": "2018-08-26 18:36:20+00:00",
            "title": "Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge",
            "authors": [
                "Pasquale Minervini",
                "Sebastian Riedel"
            ],
            "abstract": "Adversarial examples are inputs to machine learning models designed to cause\nthe model to make a mistake. They are useful for understanding the shortcomings\nof machine learning models, interpreting their results, and for regularisation.\nIn NLP, however, most example generation strategies produce input text by using\nknown, pre-specified semantic transformations, requiring significant manual\neffort and in-depth understanding of the problem and domain. In this paper, we\ninvestigate the problem of automatically generating adversarial examples that\nviolate a set of given First-Order Logic constraints in Natural Language\nInference (NLI). We reduce the problem of identifying such adversarial examples\nto a combinatorial optimisation problem, by maximising a quantity measuring the\ndegree of violation of such constraints and by using a language model for\ngenerating linguistically-plausible examples. Furthermore, we propose a method\nfor adversarially regularising neural NLI models for incorporating background\nknowledge. Our results show that, while the proposed method does not always\nimprove results on the SNLI and MultiNLI datasets, it significantly and\nconsistently increases the predictive accuracy on adversarially-crafted\ndatasets -- up to a 79.6% relative improvement -- while drastically reducing\nthe number of background knowledge violations. Furthermore, we show that\nadversarial examples transfer among model architectures, and that the proposed\nadversarial training procedure improves the robustness of NLI models to\nadversarial examples.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.08703v3",
            "date": "2018-08-27 06:51:07+00:00",
            "title": "Generating Text through Adversarial Training using Skip-Thought Vectors",
            "authors": [
                "Afroz Ahamad"
            ],
            "abstract": "GANs have been shown to perform exceedingly well on tasks pertaining to image\ngeneration and style transfer. In the field of language modelling, word\nembeddings such as GLoVe and word2vec are state-of-the-art methods for applying\nneural network models on textual data. Attempts have been made to utilize GANs\nwith word embeddings for text generation. This study presents an approach to\ntext generation using Skip-Thought sentence embeddings with GANs based on\ngradient penalty functions and f-measures. The proposed architecture aims to\nreproduce writing style in the generated text by modelling the way of\nexpression at a sentence level across all the works of an author. Extensive\nexperiments were run in different embedding settings on a variety of tasks\nincluding conditional text generation and language generation. The model\noutperforms baseline text generation networks across several automated\nevaluation metrics like BLEU-n, METEOR and ROUGE. Further, wide applicability\nand effectiveness in real life tasks are demonstrated through human judgement\nscores.",
            "journal ref": "\"Proceedings of the 2019 Conference of the North {A}merican\n  Chapter of the Association for Computational Linguistics: Student Research\n  Workshop, Jun 2019, Pages 53-60\"",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08720v1",
            "date": "2018-08-27 07:55:41+00:00",
            "title": "Predefined Sparseness in Recurrent Sequence Models",
            "authors": [
                "Thomas Demeester",
                "Johannes Deleu",
                "Fr\u00e9deric Godin",
                "Chris Develder"
            ],
            "abstract": "Inducing sparseness while training neural networks has been shown to yield\nmodels with a lower memory footprint but similar effectiveness to dense models.\nHowever, sparseness is typically induced starting from a dense model, and thus\nthis advantage does not hold during training. We propose techniques to enforce\nsparseness upfront in recurrent sequence models for NLP applications, to also\nbenefit training. First, in language modeling, we show how to increase hidden\nstate sizes in recurrent layers without increasing the number of parameters,\nleading to more expressive models. Second, for sequence labeling, we show that\nword embeddings with predefined sparseness lead to similar performance as dense\nembeddings, at a fraction of the number of trainable parameters.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.08773v3",
            "date": "2018-08-27 10:37:16+00:00",
            "title": "Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach",
            "authors": [
                "Pratik Jawanpuria",
                "Arjun Balgovind",
                "Anoop Kunchukuttan",
                "Bamdev Mishra"
            ],
            "abstract": "We propose a novel geometric approach for learning bilingual mappings given\nmonolingual embeddings and a bilingual dictionary. Our approach decouples\nlearning the transformation from the source language to the target language\ninto (a) learning rotations for language-specific embeddings to align them to a\ncommon space, and (b) learning a similarity metric in the common space to model\nsimilarities between the embeddings. We model the bilingual mapping problem as\nan optimization problem on smooth Riemannian manifolds. We show that our\napproach outperforms previous approaches on the bilingual lexicon induction and\ncross-lingual word similarity tasks. We also generalize our framework to\nrepresent multiple languages in a common latent space. In particular, the\nlatent space representations for several languages are learned jointly, given\nbilingual dictionaries for multiple language pairs. We illustrate the\neffectiveness of joint learning for multiple languages in zero-shot word\ntranslation setting. Our implementation is available at\nhttps://github.com/anoopkunchukuttan/geomm .",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.08836v1",
            "date": "2018-08-27 13:19:49+00:00",
            "title": "A strong baseline for question relevancy ranking",
            "authors": [
                "Ana V. Gonz\u00e1lez-Gardu\u00f1o",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "The best systems at the SemEval-16 and SemEval-17 community question\nanswering shared tasks -- a task that amounts to question relevancy ranking --\ninvolve complex pipelines and manual feature engineering. Despite this, many of\nthese still fail at beating the IR baseline, i.e., the rankings provided by\nGoogle's search engine. We present a strong baseline for question relevancy\nranking by training a simple multi-task feed forward network on a bag of 14\ndistance measures for the input question pair. This baseline model, which is\nfast to train and uses only language-independent features, outperforms the best\nshared task systems on the task of retrieving relevant previously asked\nquestions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.08858v1",
            "date": "2018-08-27 14:17:08+00:00",
            "title": "Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised",
            "authors": [
                "Stefanos Angelidis",
                "Mirella Lapata"
            ],
            "abstract": "We present a neural framework for opinion summarization from online product\nreviews which is knowledge-lean and only requires light supervision (e.g., in\nthe form of product domain labels and user-provided ratings). Our method\ncombines two weakly supervised components to identify salient opinions and form\nextractive summaries from multiple reviews: an aspect extractor trained under a\nmulti-task objective, and a sentiment predictor based on multiple instance\nlearning. We introduce an opinion summarization dataset that includes a\ntraining set of product reviews from six diverse domains and human-annotated\ndevelopment and test sets with gold standard aspect annotations, salience\nlabels, and opinion summaries. Automatic evaluation shows significant\nimprovements over baselines, and a large-scale study indicates that our opinion\nsummaries are preferred by human judges according to multiple criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09334v2",
            "date": "2018-08-28 14:47:33+00:00",
            "title": "A Discriminative Latent-Variable Model for Bilingual Lexicon Induction",
            "authors": [
                "Sebastian Ruder",
                "Ryan Cotterell",
                "Yova Kementchedjhieva",
                "Anders S\u00f8gaard"
            ],
            "abstract": "We introduce a novel discriminative latent-variable model for the task of\nbilingual lexicon induction. Our model combines the bipartite matching\ndictionary prior of Haghighi et al. (2008) with a state-of-the-art\nembedding-based approach. To train the model, we derive an efficient Viterbi EM\nalgorithm. We provide empirical improvements on six language pairs under two\nmetrics and show that the prior theoretically and empirically helps to mitigate\nthe hubness problem. We also demonstrate how previous work may be viewed as a\nsimilarly fashioned latent-variable model, albeit with a different prior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09442v2",
            "date": "2018-08-28 17:59:08+00:00",
            "title": "Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning",
            "authors": [
                "Shang-Yu Su",
                "Xiujun Li",
                "Jianfeng Gao",
                "Jingjing Liu",
                "Yun-Nung Chen"
            ],
            "abstract": "This paper presents a Discriminative Deep Dyna-Q (D3Q) approach to improving\nthe effectiveness and robustness of Deep Dyna-Q (DDQ), a recently proposed\nframework that extends the Dyna-Q algorithm to integrate planning for\ntask-completion dialogue policy learning. To obviate DDQ's high dependency on\nthe quality of simulated experiences, we incorporate an RNN-based discriminator\nin D3Q to differentiate simulated experience from real user experience in order\nto control the quality of training data. Experiments show that D3Q\nsignificantly outperforms DDQ by controlling the quality of simulated\nexperience used for planning. The effectiveness and robustness of D3Q is\nfurther demonstrated in a domain extension setting, where the agent's\ncapability of adapting to a changing environment is tested.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09551v1",
            "date": "2018-08-28 21:44:26+00:00",
            "title": "Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules?",
            "authors": [
                "Fr\u00e9deric Godin",
                "Kris Demuynck",
                "Joni Dambre",
                "Wesley De Neve",
                "Thomas Demeester"
            ],
            "abstract": "Character-level features are currently used in different neural network-based\nnatural language processing algorithms. However, little is known about the\ncharacter-level patterns those models learn. Moreover, models are often\ncompared only quantitatively while a qualitative analysis is missing. In this\npaper, we investigate which character-level patterns neural networks learn and\nif those patterns coincide with manually-defined word segmentations and\nannotations. To that end, we extend the contextual decomposition technique\n(Murdoch et al. 2018) to convolutional neural networks which allows us to\ncompare convolutional neural networks and bidirectional long short-term memory\nnetworks. We evaluate and compare these models for the task of morphological\ntagging on three morphologically different languages and show that these models\nimplicitly discover understandable linguistic rules. Our implementation can be\nfound at https://github.com/FredericGodin/ContextualDecomposition-NLP .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09586v1",
            "date": "2018-08-29 00:03:04+00:00",
            "title": "Complexity and mission computability of adaptive computing systems",
            "authors": [
                "Venkat R. Dasari",
                "Mee Seong Im",
                "Billy Geerhart"
            ],
            "abstract": "There is a subset of computational problems that are computable in polynomial\ntime for which an existing algorithm may not complete due to a lack of high\nperformance technology on a mission field. We define a subclass of\ndeterministic polynomial time complexity class called mission class, as many\npolynomial problems are not computable in mission time. By focusing on such\nsubclass of languages in the context for successful military applications, we\nalso discuss their computational and communicational constraints. We\ninvestigate feasible (non)linear models that will minimize energy and maximize\nmemory, efficiency, and computational power, and also provide an approximate\nsolution obtained within a pre-determined length of computation time using\nlimited resources so that an optimal solution to a language could be\ndetermined.",
            "journal ref": "None",
            "category": "math.OC"
        },
        {
            "url": "http://arxiv.org/abs/1808.09633v1",
            "date": "2018-08-29 04:23:02+00:00",
            "title": "Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment",
            "authors": [
                "Dinghan Shen",
                "Xinyuan Zhang",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "Network embeddings, which learn low-dimensional representations for each\nvertex in a large-scale network, have received considerable attention in recent\nyears. For a wide range of applications, vertices in a network are typically\naccompanied by rich textual information such as user profiles, paper abstracts,\netc. We propose to incorporate semantic features into network embeddings by\nmatching important words between text sequences for all pairs of vertices. We\nintroduce a word-by-word alignment framework that measures the compatibility of\nembeddings between word pairs, and then adaptively accumulates these alignment\nfeatures with a simple yet effective aggregation function. In experiments, we\nevaluate the proposed framework on three real-world benchmarks for downstream\ntasks, including link prediction and multi-label vertex classification. Results\ndemonstrate that our model outperforms state-of-the-art network embedding\nmethods by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09634v1",
            "date": "2018-08-29 04:32:42+00:00",
            "title": "Voice Conversion Based on Cross-Domain Features Using Variational Auto Encoders",
            "authors": [
                "Wen-Chin Huang",
                "Hsin-Te Hwang",
                "Yu-Huai Peng",
                "Yu Tsao",
                "Hsin-Min Wang"
            ],
            "abstract": "An effective approach to non-parallel voice conversion (VC) is to utilize\ndeep neural networks (DNNs), specifically variational auto encoders (VAEs), to\nmodel the latent structure of speech in an unsupervised manner. A previous\nstudy has confirmed the ef- fectiveness of VAE using the STRAIGHT spectra for\nVC. How- ever, VAE using other types of spectral features such as mel- cepstral\ncoefficients (MCCs), which are related to human per- ception and have been\nwidely used in VC, have not been prop- erly investigated. Instead of using one\nspecific type of spectral feature, it is expected that VAE may benefit from\nusing multi- ple types of spectral features simultaneously, thereby improving\nthe capability of VAE for VC. To this end, we propose a novel VAE framework\n(called cross-domain VAE, CDVAE) for VC. Specifically, the proposed framework\nutilizes both STRAIGHT spectra and MCCs by explicitly regularizing multiple\nobjectives in order to constrain the behavior of the learned encoder and de-\ncoder. Experimental results demonstrate that the proposed CD- VAE framework\noutperforms the conventional VAE framework in terms of subjective tests.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1808.09648v2",
            "date": "2018-08-29 05:53:17+00:00",
            "title": "Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms",
            "authors": [
                "Avikalp Srivastava",
                "Hsin Wen Liu",
                "Sumio Fujita"
            ],
            "abstract": "Question categorization and expert retrieval methods have been crucial for\ninformation organization and accessibility in community question & answering\n(CQA) platforms. Research in this area, however, has dealt with only the text\nmodality. With the increasing multimodal nature of web content, we focus on\nextending these methods for CQA questions accompanied by images. Specifically,\nwe leverage the success of representation learning for text and images in the\nvisual question answering (VQA) domain, and adapt the underlying concept and\narchitecture for automated category classification and expert retrieval on\nimage-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of\nYahoo! Answers.\n  To the best of our knowledge, this is the first work to tackle the\nmultimodality challenge in CQA, and to adapt VQA models for tasks on a more\necologically valid source of visual questions. Our analysis of the differences\nbetween visual QA and community QA data drives our proposal of novel\naugmentations of an attention method tailored for CQA, and use of auxiliary\ntasks for learning better grounding features. Our final model markedly\noutperforms the text-only and VQA model baselines for both tasks of\nclassification and expert retrieval on real-world multimodal CQA data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09658v1",
            "date": "2018-08-29 06:49:49+00:00",
            "title": "APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning",
            "authors": [
                "Yang Gao",
                "Christian M. Meyer",
                "Iryna Gurevych"
            ],
            "abstract": "We propose a method to perform automatic document summarisation without using\nreference summaries. Instead, our method interactively learns from users'\npreferences. The merit of preference-based interactive summarisation is that\npreferences are easier for users to provide than reference summaries. Existing\npreference-based interactive learning methods suffer from high sample\ncomplexity, i.e. they need to interact with the oracle for many rounds in order\nto converge. In this work, we propose a new objective function, which enables\nus to leverage active learning, preference learning and reinforcement learning\ntechniques in order to reduce the sample complexity. Both simulation and\nreal-user experiments suggest that our method significantly advances the state\nof the art. Our source code is freely available at\nhttps://github.com/UKPLab/emnlp2018-april.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09663v6",
            "date": "2018-08-29 07:18:29+00:00",
            "title": "Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations",
            "authors": [
                "Sidak Pal Singh",
                "Andreas Hug",
                "Aymeric Dieuleveut",
                "Martin Jaggi"
            ],
            "abstract": "We present a framework for building unsupervised representations of entities\nand their compositions, where each entity is viewed as a probability\ndistribution rather than a vector embedding. In particular, this distribution\nis supported over the contexts which co-occur with the entity and are embedded\nin a suitable low-dimensional space. This enables us to consider representation\nlearning from the perspective of Optimal Transport and take advantage of its\ntools such as Wasserstein distance and barycenters. We elaborate how the method\ncan be applied for obtaining unsupervised representations of text and\nillustrate the performance (quantitatively as well as qualitatively) on tasks\nsuch as measuring sentence similarity, word entailment and similarity, where we\nempirically observe significant gains (e.g., 4.1% relative improvement over\nSent2vec, GenSen).\n  The key benefits of the proposed approach include: (a) capturing uncertainty\nand polysemy via modeling the entities as distributions, (b) utilizing the\nunderlying geometry of the particular task (with the ground cost), (c)\nsimultaneously providing interpretability with the notion of optimal transport\nbetween contexts and (d) easy applicability on top of existing point embedding\nmethods. The code, as well as prebuilt histograms, are available under\nhttps://github.com/context-mover/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09744v1",
            "date": "2018-08-29 12:02:11+00:00",
            "title": "Rule induction for global explanation of trained models",
            "authors": [
                "Madhumita Sushil",
                "Simon \u0160uster",
                "Walter Daelemans"
            ],
            "abstract": "Understanding the behavior of a trained network and finding explanations for\nits outputs is important for improving the network's performance and\ngeneralization ability, and for ensuring trust in automated systems. Several\napproaches have previously been proposed to identify and visualize the most\nimportant features by analyzing a trained network. However, the relations\nbetween different features and classes are lost in most cases. We propose a\ntechnique to induce sets of if-then-else rules that capture these relations to\nglobally explain the predictions of a network. We first calculate the\nimportance of the features in the trained network. We then weigh the original\ninputs with these feature importance scores, simplify the transformed input\nspace, and finally fit a rule induction model to explain the model predictions.\nWe find that the output rule-sets can explain the predictions of a neural\nnetwork trained for 4-class text classification from the 20 newsgroups dataset\nto a macro-averaged F-score of 0.80. We make the code available at\nhttps://github.com/clips/interpret_with_rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09889v1",
            "date": "2018-08-27 16:12:36+00:00",
            "title": "Zero-shot Transfer Learning for Semantic Parsing",
            "authors": [
                "Javid Dadashkarimi",
                "Alexander Fabbri",
                "Sekhar Tatikonda",
                "Dragomir R. Radev"
            ],
            "abstract": "While neural networks have shown impressive performance on large datasets,\napplying these models to tasks where little data is available remains a\nchallenging problem.\n  In this paper we propose to use feature transfer in a zero-shot experimental\nsetting on the task of semantic parsing.\n  We first introduce a new method for learning the shared space between\nmultiple domains based on the prediction of the domain label for each example.\n  Our experiments support the superiority of this method in a zero-shot\nexperimental setting in terms of accuracy metrics compared to state-of-the-art\ntechniques.\n  In the second part of this paper we study the impact of individual domains\nand examples on semantic parsing performance.\n  We use influence functions to this aim and investigate the sensitivity of\ndomain-label classification loss on each example.\n  Our findings reveal that cross-domain adversarial attacks identify useful\nexamples for training even from the domains the least similar to the target\ndomain. Augmenting our training data with these influential examples further\nboosts our accuracy at both the token and the sequence level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09890v1",
            "date": "2018-08-27 21:37:42+00:00",
            "title": "An Adaptive Conversational Bot Framework",
            "authors": [
                "Isak Czeresnia Etinger"
            ],
            "abstract": "How can we enable users to heavily specify criteria for database queries in a\nuser-friendly way? This paper describes a general framework of a conversational\nbot that extracts meaningful information from user's sentences, that asks\nsubsequent questions to complete missing information, and that adjusts its\nquestions and information-extraction parameters for later conversations\ndepending on users' behavior. Additionally, we provide a comparison of existing\ntools and give novel techniques to implement such framework. Finally, we\nexemplify the framework with a bot to query movies in a database, whose code is\navailable for Microsoft employees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.09942v1",
            "date": "2018-08-29 17:43:11+00:00",
            "title": "Neural Compositional Denotational Semantics for Question Answering",
            "authors": [
                "Nitish Gupta",
                "Mike Lewis"
            ],
            "abstract": "Answering compositional questions requiring multi-step reasoning is\nchallenging. We introduce an end-to-end differentiable model for interpreting\nquestions about a knowledge graph (KG), which is inspired by formal approaches\nto semantics. Each span of text is represented by a denotation in a KG and a\nvector that captures ungrounded aspects of meaning. Learned composition modules\nrecursively combine constituent spans, culminating in a grounding for the\ncomplete sentence which answers the question. For example, to interpret \"not\ngreen\", the model represents \"green\" as a set of KG entities and \"not\" as a\ntrainable ungrounded vector---and then uses this vector to parameterize a\ncomposition function that performs a complement operation. For each sentence,\nwe build a parse chart subsuming all possible parses, allowing the model to\njointly learn both the composition operators and output structure by gradient\ndescent from end-task supervision. The model learns a variety of challenging\nsemantic operators, such as quantifiers, disjunctions and composed relations,\nand infers latent syntactic structure. It also generalizes well to longer\nquestions than seen in its training data, in contrast to RNN, its tree-based\nvariants, and semantic parsing baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10009v1",
            "date": "2018-08-29 18:40:26+00:00",
            "title": "Learning a Policy for Opportunistic Active Learning",
            "authors": [
                "Aishwarya Padmakumar",
                "Peter Stone",
                "Raymond J. Mooney"
            ],
            "abstract": "Active learning identifies data points to label that are expected to be the\nmost useful in improving a supervised model. Opportunistic active learning\nincorporates active learning into interactive tasks that constrain possible\nqueries during interactions. Prior work has shown that opportunistic active\nlearning can be used to improve grounding of natural language descriptions in\nan interactive object retrieval task. In this work, we use reinforcement\nlearning for such an object retrieval task, to learn a policy that effectively\ntrades off task completion with model improvement that would benefit future\ntasks.",
            "journal ref": "EMNLP 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10088v2",
            "date": "2018-08-30 02:33:02+00:00",
            "title": "End-to-end Speech Recognition with Adaptive Computation Steps",
            "authors": [
                "Mohan Li",
                "Min Liu",
                "Masanori Hattori"
            ],
            "abstract": "In this paper, we present Adaptive Computation Steps (ACS) algo-rithm, which\nenables end-to-end speech recognition models to dy-namically decide how many\nframes should be processed to predict a linguistic output. The model that\napplies ACS algorithm follows the encoder-decoder framework, while unlike the\nattention-based mod-els, it produces alignments independently at the encoder\nside using the correlation between adjacent frames. Thus, predictions can be\nmade as soon as sufficient acoustic information is received, which makes the\nmodel applicable in online cases. Besides, a small change is made to the\ndecoding stage of the encoder-decoder framework, which allows the prediction to\nexploit bidirectional contexts. We verify the ACS algorithm on a Mandarin\nspeech corpus AIShell-1, and it achieves a 31.2% CER in the online occasion,\ncompared to the 32.4% CER of the attention-based model. To fully demonstrate\nthe advantage of ACS algorithm, offline experiments are conducted, in which our\nACS model achieves an 18.7% CER, outperforming the attention-based counterpart\nwith the CER of 22.0%.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1808.10128v1",
            "date": "2018-08-30 05:51:30+00:00",
            "title": "Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis",
            "authors": [
                "Yu-An Chung",
                "Yuxuan Wang",
                "Wei-Ning Hsu",
                "Yu Zhang",
                "RJ Skerry-Ryan"
            ],
            "abstract": "Although end-to-end text-to-speech (TTS) models such as Tacotron have shown\nexcellent results, they typically require a sizable set of high-quality <text,\naudio> pairs for training, which are expensive to collect. In this paper, we\npropose a semi-supervised training framework to improve the data efficiency of\nTacotron. The idea is to allow Tacotron to utilize textual and acoustic\nknowledge contained in large, publicly-available text and speech corpora.\nImportantly, these external data are unpaired and potentially noisy.\nSpecifically, first we embed each word in the input text into word vectors and\ncondition the Tacotron encoder on them. We then use an unpaired speech corpus\nto pre-train the Tacotron decoder in the acoustic domain. Finally, we fine-tune\nthe model using available paired data. We demonstrate that the proposed\nframework enables Tacotron to generate intelligible speech using less than half\nan hour of paired training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.10556v1",
            "date": "2018-08-31 00:15:53+00:00",
            "title": "Speaker Fluency Level Classification Using Machine Learning Techniques",
            "authors": [
                "Alan Preciado-Grijalva",
                "Ramon F. Brena"
            ],
            "abstract": "Level assessment for foreign language students is necessary for putting them\nin the right level group, furthermore, interviewing students is a very\ntime-consuming task, so we propose to automate the evaluation of speaker\nfluency level by implementing machine learning techniques. This work presents\nan audio processing system capable of classifying the level of fluency of\nnon-native English speakers using five different machine learning models. As a\nfirst step, we have built our own dataset, which consists of labeled audio\nconversations in English between people ranging in different fluency\ndomains/classes (low, intermediate, high). We segment the audio conversations\ninto 5s non-overlapped audio clips to perform feature extraction on them. We\nstart by extracting Mel cepstral coefficients from the audios, selecting 20\ncoefficients is an appropriate quantity for our data. We thereafter extracted\nzero-crossing rate, root mean square energy and spectral flux features, proving\nthat this improves model performance. Out of a total of 1424 audio segments,\nwith 70% training data and 30% test data, one of our trained models (support\nvector machine) achieved a classification accuracy of 94.39%, whereas the other\nfour models passed an 89% classification accuracy threshold.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1808.10568v2",
            "date": "2018-08-31 01:55:09+00:00",
            "title": "Multi-Hop Knowledge Graph Reasoning with Reward Shaping",
            "authors": [
                "Xi Victoria Lin",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Multi-hop reasoning is an effective approach for query answering (QA) over\nincomplete knowledge graphs (KGs). The problem can be formulated in a\nreinforcement learning (RL) setup, where a policy-based agent sequentially\nextends its inference path until it reaches a target. However, in an incomplete\nKG environment, the agent receives low-quality rewards corrupted by false\nnegatives in the training data, which harms generalization at test time.\nFurthermore, since no golden action sequence is used for training, the agent\ncan be misled by spurious search trajectories that incidentally lead to the\ncorrect answer. We propose two modeling advances to address both issues: (1) we\nreduce the impact of false negative supervision by adopting a pretrained\none-hop embedding model to estimate the reward of unobserved facts; (2) we\ncounter the sensitivity to spurious paths of on-policy RL by forcing the agent\nto explore a diverse set of paths using randomly generated edge masks. Our\napproach significantly improves over existing path-based KGQA models on several\nbenchmark datasets and is comparable or better than embedding-based models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1808.10792v2",
            "date": "2018-08-31 14:55:52+00:00",
            "title": "Bottom-Up Abstractive Summarization",
            "authors": [
                "Sebastian Gehrmann",
                "Yuntian Deng",
                "Alexander M. Rush"
            ],
            "abstract": "Neural network-based methods for abstractive summarization produce outputs\nthat are more fluent than other techniques, but which can be poor at content\nselection. This work proposes a simple technique for addressing this issue: use\na data-efficient content selector to over-determine phrases in a source\ndocument that should be part of the summary. We use this selector as a\nbottom-up attention step to constrain the model to likely phrases. We show that\nthis approach improves the ability to compress text, while still generating\nfluent summaries. This two-step process is both simpler and higher performing\nthan other end-to-end content selection models, leading to significant\nimprovements on ROUGE for both the CNN-DM and NYT corpus. Furthermore, the\ncontent selector can be trained with as little as 1,000 sentences, making it\neasy to transfer a trained summarizer to a new domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00254v1",
            "date": "2018-02-01 12:00:45+00:00",
            "title": "Phonetic and Graphemic Systems for Multi-Genre Broadcast Transcription",
            "authors": [
                "Yu Wang",
                "Xie Chen",
                "Mark Gales",
                "Anton Ragni",
                "Jeremy Wong"
            ],
            "abstract": "State-of-the-art English automatic speech recognition systems typically use\nphonetic rather than graphemic lexicons. Graphemic systems are known to perform\nless well for English as the mapping from the written form to the spoken form\nis complicated. However, in recent years the representational power of\ndeep-learning based acoustic models has improved, raising interest in graphemic\nacoustic models for English, due to the simplicity of generating the lexicon.\nIn this paper, phonetic and graphemic models are compared for an English\nMulti-Genre Broadcast transcription task. A range of acoustic models based on\nlattice-free MMI training are constructed using phonetic and graphemic\nlexicons. For this task, it is found that having a long-span temporal history\nreduces the difference in performance between the two forms of models. In\naddition, system combination is examined, using parameter smoothing and\nhypothesis combination. As the combination approaches become more complicated\nthe difference between the phonetic and graphemic systems further decreases.\nFinally, for all configurations examined the combination of phonetic and\ngraphemic systems yields consistent gains.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1802.00382v1",
            "date": "2018-02-01 16:46:00+00:00",
            "title": "Classifying medical notes into standard disease codes using Machine Learning",
            "authors": [
                "Amitabha Karmakar"
            ],
            "abstract": "We investigate the automatic classification of patient discharge notes into\nstandard disease labels. We find that Convolutional Neural Networks with\nAttention outperform previous algorithms used in this task, and suggest further\nareas for improvement.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.00396v2",
            "date": "2018-02-01 17:08:48+00:00",
            "title": "Disunited Nations? A Multiplex Network Approach to Detecting Preference Affinity Blocs using Texts and Votes",
            "authors": [
                "Caleb Pomeroy",
                "Niheer Dasandi",
                "Slava J. Mikhaylov"
            ],
            "abstract": "This paper contributes to an emerging literature that models votes and text\nin tandem to better understand polarization of expressed preferences. It\nintroduces a new approach to estimate preference polarization in\nmultidimensional settings, such as international relations, based on\ndevelopments in the natural language processing and network science literatures\n-- namely word embeddings, which retain valuable syntactical qualities of human\nlanguage, and community detection in multilayer networks, which locates densely\nconnected actors across multiple, complex networks. We find that the employment\nof these tools in tandem helps to better estimate states' foreign policy\npreferences expressed in UN votes and speeches beyond that permitted by votes\nalone. The utility of these located affinity blocs is demonstrated through an\napplication to conflict onset in International Relations, though these tools\nwill be of interest to all scholars faced with the measurement of preferences\nand polarization in multidimensional settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.00923v1",
            "date": "2018-02-03 06:29:17+00:00",
            "title": "Multi-attention Recurrent Network for Human Communication Comprehension",
            "authors": [
                "Amir Zadeh",
                "Paul Pu Liang",
                "Soujanya Poria",
                "Prateek Vij",
                "Erik Cambria",
                "Louis-Philippe Morency"
            ],
            "abstract": "Human face-to-face communication is a complex multimodal signal. We use words\n(language modality), gestures (vision modality) and changes in tone (acoustic\nmodality) to convey our intentions. Humans easily process and understand\nface-to-face communication, however, comprehending this form of communication\nremains a significant challenge for Artificial Intelligence (AI). AI must\nunderstand each modality and the interactions between them that shape human\ncommunication. In this paper, we present a novel neural architecture for\nunderstanding human communication called the Multi-attention Recurrent Network\n(MARN). The main strength of our model comes from discovering interactions\nbetween modalities through time using a neural component called the\nMulti-attention Block (MAB) and storing them in the hybrid memory of a\nrecurrent component called the Long-short Term Hybrid Memory (LSTHM). We\nperform extensive comparisons on six publicly available datasets for multimodal\nsentiment analysis, speaker trait recognition and emotion recognition. MARN\nshows state-of-the-art performance on all the datasets.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.00924v1",
            "date": "2018-02-03 06:30:09+00:00",
            "title": "Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning",
            "authors": [
                "Minghai Chen",
                "Sen Wang",
                "Paul Pu Liang",
                "Tadas Baltru\u0161aitis",
                "Amir Zadeh",
                "Louis-Philippe Morency"
            ],
            "abstract": "With the increasing popularity of video sharing websites such as YouTube and\nFacebook, multimodal sentiment analysis has received increasing attention from\nthe scientific community. Contrary to previous works in multimodal sentiment\nanalysis which focus on holistic information in speech segments such as bag of\nwords representations and average facial expression intensity, we develop a\nnovel deep architecture for multimodal sentiment analysis that performs\nmodality fusion at the word level. In this paper, we propose the Gated\nMultimodal Embedding LSTM with Temporal Attention (GME-LSTM(A)) model that is\ncomposed of 2 modules. The Gated Multimodal Embedding alleviates the\ndifficulties of fusion when there are noisy modalities. The LSTM with Temporal\nAttention performs word level fusion at a finer fusion resolution between input\nmodalities and attends to the most important time steps. As a result, the\nGME-LSTM(A) is able to better model the multimodal structure of speech through\ntime and perform better sentiment comprehension. We demonstrate the\neffectiveness of this approach on the publicly-available Multimodal Corpus of\nSentiment Intensity and Subjectivity Analysis (CMU-MOSI) dataset by achieving\nstate-of-the-art sentiment classification and regression results. Qualitative\nanalysis on our model emphasizes the importance of the Temporal Attention Layer\nin sentiment prediction because the additional acoustic and visual modalities\nare noisy. We also demonstrate the effectiveness of the Gated Multimodal\nEmbedding in selectively filtering these noisy modalities out. Our results and\nanalysis open new areas in the study of sentiment analysis in human\ncommunication and provide new models for multimodal fusion.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.01405v1",
            "date": "2018-01-31 22:18:56+00:00",
            "title": "Comparing approaches for mitigating intergroup variability in personality recognition",
            "authors": [
                "Guozhen An",
                "Rivka Levitan"
            ],
            "abstract": "Personality have been found to predict many life outcomes, and there have\nbeen huge interests on automatic personality recognition from a speaker's\nutterance. Previously, we achieved accuracies between 37%-44% for three-way\nclassification of high, medium or low for each of the Big Five personality\ntraits (Openness to Experience, Conscientiousness, Extraversion, Agreeableness,\nNeuroticism). We show here that we can improve performance on this task by\naccounting for heterogeneity of gender and L1 in our data, which has English\nspeech from female and male native speakers of Chinese and Standard American\nEnglish (SAE). We experiment with personalizing models by L1 and gender and\nnormalizing features by speaker, L1 group, and/or gender.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1802.01433v4",
            "date": "2018-01-31 01:35:46+00:00",
            "title": "Interactive Grounded Language Acquisition and Generalization in a 2D World",
            "authors": [
                "Haonan Yu",
                "Haichao Zhang",
                "Wei Xu"
            ],
            "abstract": "We build a virtual agent for learning language in a 2D maze-like world. The\nagent sees images of the surrounding environment, listens to a virtual teacher,\nand takes actions to receive rewards. It interactively learns the teacher's\nlanguage from scratch based on two language use cases: sentence-directed\nnavigation and question answering. It learns simultaneously the visual\nrepresentations of the world, the language, and the action control. By\ndisentangling language grounding from other computational routines and sharing\na concept detection function between language grounding and prediction, the\nagent reliably interpolates and extrapolates to interpret sentences that\ncontain new word combinations or new words missing from training sentences. The\nnew words are transferred from the answers of language prediction. Such a\nlanguage ability is trained and evaluated on a population of over 1.6 million\ndistinct sentences consisting of 119 object words, 8 color words, 9\nspatial-relation words, and 50 grammatical words. The proposed model\nsignificantly outperforms five comparison methods for interpreting zero-shot\nsentences. In addition, we demonstrate human-interpretable intermediate outputs\nof the model in the appendix.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01812v1",
            "date": "2018-02-06 06:18:56+00:00",
            "title": "Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation",
            "authors": [
                "Junyang Lin",
                "Shuming Ma",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Attention-based sequence-to-sequence model has proved successful in Neural\nMachine Translation (NMT). However, the attention without consideration of\ndecoding history, which includes the past information in the decoder and the\nattention mechanism, often causes much repetition. To address this problem, we\npropose the decoding-history-based Adaptive Control of Attention (ACA) for the\nNMT model. ACA learns to control the attention by keeping track of the decoding\nhistory and the current information with a memory vector, so that the model can\ntake the translated contents and the current information into consideration.\nExperiments on Chinese-English translation and the English-Vietnamese\ntranslation have demonstrated that our model significantly outperforms the\nstrong baselines. The analysis shows that our model is capable of generating\ntranslation with less repetition and higher accuracy. The code will be\navailable at https://github.com/lancopku",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.01886v1",
            "date": "2018-02-06 11:30:32+00:00",
            "title": "Texygen: A Benchmarking Platform for Text Generation Models",
            "authors": [
                "Yaoming Zhu",
                "Sidi Lu",
                "Lei Zheng",
                "Jiaxian Guo",
                "Weinan Zhang",
                "Jun Wang",
                "Yong Yu"
            ],
            "abstract": "We introduce Texygen, a benchmarking platform to support research on\nopen-domain text generation models. Texygen has not only implemented a majority\nof text generation models, but also covered a set of metrics that evaluate the\ndiversity, the quality and the consistency of the generated texts. The Texygen\nplatform could help standardize the research on text generation and facilitate\nthe sharing of fine-tuned open-source implementations among researchers for\ntheir work. As a consequence, this would help in improving the reproductivity\nand reliability of future research work in text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02032v1",
            "date": "2018-02-06 16:19:05+00:00",
            "title": "Improving Variational Encoder-Decoders in Dialogue Generation",
            "authors": [
                "Xiaoyu Shen",
                "Hui Su",
                "Shuzi Niu",
                "Vera Demberg"
            ],
            "abstract": "Variational encoder-decoders (VEDs) have shown promising results in dialogue\ngeneration. However, the latent variable distributions are usually approximated\nby a much simpler model than the powerful RNN structure used for encoding and\ndecoding, yielding the KL-vanishing problem and inconsistent training\nobjective. In this paper, we separate the training step into two phases: The\nfirst phase learns to autoencode discrete texts into continuous embeddings,\nfrom which the second phase learns to generalize latent representations by\nreconstructing the encoded embedding. In this case, latent variables are\nsampled by transforming Gaussian noise through multi-layer perceptrons and are\ntrained with a separate VED model, which has the potential of realizing a much\nmore flexible distribution. We compare our model with current popular models\nand the experiment demonstrates substantial improvement in both metric-based\nand human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02163v1",
            "date": "2018-02-06 19:00:12+00:00",
            "title": "How to Make Causal Inferences Using Texts",
            "authors": [
                "Naoki Egami",
                "Christian J. Fong",
                "Justin Grimmer",
                "Margaret E. Roberts",
                "Brandon M. Stewart"
            ],
            "abstract": "New text as data techniques offer a great promise: the ability to inductively\ndiscover measures that are useful for testing social science theories of\ninterest from large collections of text. We introduce a conceptual framework\nfor making causal inferences with discovered measures as a treatment or\noutcome. Our framework enables researchers to discover high-dimensional textual\ninterventions and estimate the ways that observed treatments affect text-based\noutcomes. We argue that nearly all text-based causal inferences depend upon a\nlatent representation of the text and we provide a framework to learn the\nlatent representation. But estimating this latent representation, we show,\ncreates new risks: we may introduce an identification problem or overfit. To\naddress these risks we describe a split-sample framework and apply it to\nestimate causal effects from an experiment on immigration attitudes and a study\non bureaucratic response. Our work provides a rigorous foundation for\ntext-based causal inferences.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1802.02550v7",
            "date": "2018-02-07 18:06:42+00:00",
            "title": "Semi-Amortized Variational Autoencoders",
            "authors": [
                "Yoon Kim",
                "Sam Wiseman",
                "Andrew C. Miller",
                "David Sontag",
                "Alexander M. Rush"
            ],
            "abstract": "Amortized variational inference (AVI) replaces instance-specific local\ninference with a global inference network. While AVI has enabled efficient\ntraining of deep generative models such as variational autoencoders (VAE),\nrecent empirical work suggests that inference networks can produce suboptimal\nvariational parameters. We propose a hybrid approach, to use AVI to initialize\nthe variational parameters and run stochastic variational inference (SVI) to\nrefine them. Crucially, the local SVI procedure is itself differentiable, so\nthe inference network and generative model can be trained end-to-end with\ngradient-based optimization. This semi-amortized approach enables the use of\nrich generative models without experiencing the posterior-collapse phenomenon\ncommon in training VAEs for problems like text generation. Experiments show\nthis approach outperforms strong autoregressive and variational baselines on\nstandard text and image datasets.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1802.02561v2",
            "date": "2018-02-07 18:36:38+00:00",
            "title": "Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning",
            "authors": [
                "Hamza Harkous",
                "Kassem Fawaz",
                "R\u00e9mi Lebret",
                "Florian Schaub",
                "Kang G. Shin",
                "Karl Aberer"
            ],
            "abstract": "Privacy policies are the primary channel through which companies inform users\nabout their data collection and sharing practices. These policies are often\nlong and difficult to comprehend. Short notices based on information extracted\nfrom privacy policies have been shown to be useful but face a significant\nscalability hurdle, given the number of policies and their evolution over time.\nCompanies, users, researchers, and regulators still lack usable and scalable\ntools to cope with the breadth and depth of privacy policies. To address these\nhurdles, we propose an automated framework for privacy policy analysis\n(Polisis). It enables scalable, dynamic, and multi-dimensional queries on\nnatural language privacy policies. At the core of Polisis is a privacy-centric\nlanguage model, built with 130K privacy policies, and a novel hierarchy of\nneural-network classifiers that accounts for both high-level aspects and\nfine-grained details of privacy practices. We demonstrate Polisis' modularity\nand utility with two applications supporting structured and free-form querying.\nThe structured querying application is the automated assignment of privacy\nicons from privacy policies. With Polisis, we can achieve an accuracy of 88.4%\non this task. The second application, PriBot, is the first freeform\nquestion-answering system for privacy policies. We show that PriBot can produce\na correct answer among its top-3 results for 82% of the test questions. Using\nan MTurk user study with 700 participants, we show that at least one of\nPriBot's top-3 answers is relevant to users for 89% of the test questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02607v2",
            "date": "2018-02-07 19:30:17+00:00",
            "title": "Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context Modeling",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Haoqi Li",
                "Kevin Knight",
                "Panayiotis Georgiou"
            ],
            "abstract": "Automatic speech recognition (ASR) systems often make unrecoverable errors\ndue to subsystem pruning (acoustic, language and pronunciation models); for\nexample pruning words due to acoustics using short-term context, prior to\nrescoring with long-term context based on linguistics. In this work we model\nASR as a phrase-based noisy transformation channel and propose an error\ncorrection system that can learn from the aggregate errors of all the\nindependent modules constituting the ASR and attempt to invert those. The\nproposed system can exploit long-term context using a neural network language\nmodel and can better choose between existing ASR output possibilities as well\nas re-introduce previously pruned or unseen (out-of-vocabulary) phrases. It\nprovides corrections under poorly performing ASR conditions without degrading\nany accurate transcriptions; such corrections are greater on top of\nout-of-domain and mismatched data ASR. Our system consistently provides\nimprovements over the baseline ASR, even when baseline is further optimized\nthrough recurrent neural network language model rescoring. This demonstrates\nthat any ASR improvements can be exploited independently and that our proposed\nsystem can potentially still provide benefits on highly optimized ASR. Finally,\nwe present an extensive analysis of the type of errors corrected by our system.",
            "journal ref": "APSIPA Transactions on Signal and Information Processing 8.\n  Cambridge University Press: e8, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02656v1",
            "date": "2018-02-07 22:05:18+00:00",
            "title": "Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition",
            "authors": [
                "Xuesong Yang",
                "Kartik Audhkhasi",
                "Andrew Rosenberg",
                "Samuel Thomas",
                "Bhuvana Ramabhadran",
                "Mark Hasegawa-Johnson"
            ],
            "abstract": "The performance of automatic speech recognition systems degrades with\nincreasing mismatch between the training and testing scenarios. Differences in\nspeaker accents are a significant source of such mismatch. The traditional\napproach to deal with multiple accents involves pooling data from several\naccents during training and building a single model in multi-task fashion,\nwhere tasks correspond to individual accents. In this paper, we explore an\nalternate model where we jointly learn an accent classifier and a multi-task\nacoustic model. Experiments on the American English Wall Street Journal and\nBritish English Cambridge corpora demonstrate that our joint model outperforms\nthe strong multi-task acoustic model baseline. We obtain a 5.94% relative\nimprovement in word error rate on British English, and 9.47% relative\nimprovement on American English. This illustrates that jointly modeling with\naccent information improves acoustic model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02745v2",
            "date": "2018-02-08 08:25:51+00:00",
            "title": "Learning Inductive Biases with Simple Neural Networks",
            "authors": [
                "Reuben Feinman",
                "Brenden M. Lake"
            ],
            "abstract": "People use rich prior knowledge about the world in order to efficiently learn\nnew concepts. These priors - also known as \"inductive biases\" - pertain to the\nspace of internal models considered by a learner, and they help the learner\nmake inferences that go beyond the observed data. A recent study found that\ndeep neural networks optimized for object recognition develop the shape bias\n(Ritter et al., 2017), an inductive bias possessed by children that plays an\nimportant role in early word learning. However, these networks use\nunrealistically large quantities of training data, and the conditions required\nfor these biases to develop are not well understood. Moreover, it is unclear\nhow the learning dynamics of these networks relate to developmental processes\nin childhood. We investigate the development and influence of the shape bias in\nneural networks using controlled datasets of abstract patterns and synthetic\nimages, allowing us to systematically vary the quantity and form of the\nexperience provided to the learning algorithms. We find that simple neural\nnetworks develop a shape bias after seeing as few as 3 examples of 4 object\ncategories. The development of these biases predicts the onset of vocabulary\nacceleration in our networks, consistent with the developmental process in\nchildren.",
            "journal ref": "Feinman, R. and Lake, B.M. (2018). Learning inductive biases with\n  simple neural networks. In Proceedings of the 40th Annual Meeting of the\n  Cognitive Science Society",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.02892v1",
            "date": "2018-02-06 20:30:59+00:00",
            "title": "Efficient Large-Scale Multi-Modal Classification",
            "authors": [
                "D. Kiela",
                "E. Grave",
                "A. Joulin",
                "T. Mikolov"
            ],
            "abstract": "While the incipient internet was largely text-based, the modern digital world\nis becoming increasingly multi-modal. Here, we examine multi-modal\nclassification where one modality is discrete, e.g. text, and the other is\ncontinuous, e.g. visual representations transferred from a convolutional neural\nnetwork. In particular, we focus on scenarios where we have to be able to\nclassify large quantities of data quickly. We investigate various methods for\nperforming multi-modal fusion and analyze their trade-offs in terms of\nclassification accuracy and computational efficiency. Our findings indicate\nthat the inclusion of continuous information improves performance over\ntext-only on a range of multi-modal classification tasks, even with simple\nfusion methods. In addition, we experiment with discretizing the continuous\nfeatures in order to speed up and simplify the fusion process even further. Our\nresults show that fusion with discretized features outperforms text-only\nclassification, at a fraction of the computational cost of full multi-modal\nfusion, with the additional benefit of improved interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03052v1",
            "date": "2018-02-08 21:26:03+00:00",
            "title": "WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-Hop Inference",
            "authors": [
                "Peter A. Jansen",
                "Elizabeth Wainwright",
                "Steven Marmorstein",
                "Clayton T. Morrison"
            ],
            "abstract": "Developing methods of automated inference that are able to provide users with\ncompelling human-readable justifications for why the answer to a question is\ncorrect is critical for domains such as science and medicine, where user trust\nand detecting costly errors are limiting factors to adoption. One of the\ncentral barriers to training question answering models on explainable inference\ntasks is the lack of gold explanations to serve as training data. In this paper\nwe present a corpus of explanations for standardized science exams, a recent\nchallenge task for question answering. We manually construct a corpus of\ndetailed explanations for nearly all publicly available standardized elementary\nscience question (approximately 1,680 3rd through 5th grade questions) and\nrepresent these as \"explanation graphs\" -- sets of lexically overlapping\nsentences that describe how to arrive at the correct answer to a question\nthrough a combination of domain and world knowledge. We also provide an\nexplanation-centered tablestore, a collection of semi-structured tables that\ncontain the knowledge to construct these elementary science explanations.\nTogether, these two knowledge resources map out a substantial portion of the\nknowledge required for answering and explaining elementary science exams, and\nprovide both structured and free-text training data for the explainable\ninference task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03753v1",
            "date": "2018-02-11 15:37:37+00:00",
            "title": "Sample Efficient Deep Reinforcement Learning for Dialogue Systems with Large Action Spaces",
            "authors": [
                "Gell\u00e9rt Weisz",
                "Pawe\u0142 Budzianowski",
                "Pei-Hao Su",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "In spoken dialogue systems, we aim to deploy artificial intelligence to build\nautomated dialogue agents that can converse with humans. A part of this effort\nis the policy optimisation task, which attempts to find a policy describing how\nto respond to humans, in the form of a function taking the current state of the\ndialogue and returning the response of the system. In this paper, we\ninvestigate deep reinforcement learning approaches to solve this problem.\nParticular attention is given to actor-critic methods, off-policy reinforcement\nlearning with experience replay, and various methods aimed at reducing the bias\nand variance of estimators. When combined, these methods result in the\npreviously proposed ACER algorithm that gave competitive results in gaming\nenvironments. These environments however are fully observable and have a\nrelatively small action set so in this paper we examine the application of ACER\nto dialogue policy optimisation. We show that this method beats the current\nstate-of-the-art in deep learning approaches for spoken dialogue systems. This\nnot only leads to a more sample efficient algorithm that can train faster, but\nalso allows us to apply the algorithm in more difficult environments than\nbefore. We thus experiment with learning in a very large action space, which\nhas two orders of magnitude more actions than previously considered. We find\nthat ACER trains significantly faster than the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.03881v3",
            "date": "2018-02-12 04:08:06+00:00",
            "title": "Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog",
            "authors": [
                "Sang-Woo Lee",
                "Yu-Jung Heo",
                "Byoung-Tak Zhang"
            ],
            "abstract": "Goal-oriented dialog has been given attention due to its numerous\napplications in artificial intelligence. Goal-oriented dialogue tasks occur\nwhen a questioner asks an action-oriented question and an answerer responds\nwith the intent of letting the questioner know a correct action to take. To ask\nthe adequate question, deep learning and reinforcement learning have been\nrecently applied. However, these approaches struggle to find a competent\nrecurrent neural questioner, owing to the complexity of learning a series of\nsentences. Motivated by theory of mind, we propose \"Answerer in Questioner's\nMind\" (AQM), a novel information theoretic algorithm for goal-oriented dialog.\nWith AQM, a questioner asks and infers based on an approximated probabilistic\nmodel of the answerer. The questioner figures out the answerer's intention via\nselecting a plausible question by explicitly calculating the information gain\nof the candidate intentions and possible answers to each question. We test our\nframework on two goal-oriented visual dialog tasks: \"MNIST Counting Dialog\" and\n\"GuessWhat?!\". In our experiments, AQM outperforms comparative algorithms by a\nlarge margin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1802.04223v2",
            "date": "2018-02-12 18:07:34+00:00",
            "title": "SparseMAP: Differentiable Sparse Structured Inference",
            "authors": [
                "Vlad Niculae",
                "Andr\u00e9 F. T. Martins",
                "Mathieu Blondel",
                "Claire Cardie"
            ],
            "abstract": "Structured prediction requires searching over a combinatorial number of\nstructures. To tackle it, we introduce SparseMAP: a new method for sparse\nstructured inference, and its natural loss function. SparseMAP automatically\nselects only a few global structures: it is situated between MAP inference,\nwhich picks a single structure, and marginal inference, which assigns\nprobability mass to all structures, including implausible ones. Importantly,\nSparseMAP can be computed using only calls to a MAP oracle, making it\napplicable to problems with intractable marginal inference, e.g., linear\nassignment. Sparsity makes gradient backpropagation efficient regardless of the\nstructure, enabling us to augment deep neural networks with generic and sparse\nstructured hidden layers. Experiments in dependency parsing and natural\nlanguage inference reveal competitive accuracy, improved interpretability, and\nthe ability to capture natural language ambiguities, which is attractive for\npipeline systems.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1802.04335v1",
            "date": "2018-02-12 20:05:26+00:00",
            "title": "Neural Program Search: Solving Programming Tasks from Description and Examples",
            "authors": [
                "Illia Polosukhin",
                "Alexander Skidanov"
            ],
            "abstract": "We present a Neural Program Search, an algorithm to generate programs from\nnatural language description and a small number of input/output examples. The\nalgorithm combines methods from Deep Learning and Program Synthesis fields by\ndesigning rich domain-specific language (DSL) and defining efficient search\nalgorithm guided by a Seq2Tree model on it. To evaluate the quality of the\napproach we also present a semi-synthetic dataset of descriptions with test\nexamples and corresponding programs. We show that our algorithm significantly\noutperforms a sequence-to-sequence model with attention baseline.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.04394v5",
            "date": "2018-02-12 23:27:23+00:00",
            "title": "M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search",
            "authors": [
                "Yelong Shen",
                "Jianshu Chen",
                "Po-Sen Huang",
                "Yuqing Guo",
                "Jianfeng Gao"
            ],
            "abstract": "Learning to walk over a graph towards a target node for a given query and a\nsource node is an important problem in applications such as knowledge base\ncompletion (KBC). It can be formulated as a reinforcement learning (RL) problem\nwith a known state transition model. To overcome the challenge of sparse\nrewards, we develop a graph-walking agent called M-Walk, which consists of a\ndeep recurrent neural network (RNN) and Monte Carlo Tree Search (MCTS). The RNN\nencodes the state (i.e., history of the walked path) and maps it separately to\na policy and Q-values. In order to effectively train the agent from sparse\nrewards, we combine MCTS with the neural policy to generate trajectories\nyielding more positive rewards. From these trajectories, the network is\nimproved in an off-policy manner using Q-learning, which modifies the RNN\npolicy via parameter sharing. Our proposed RL algorithm repeatedly applies this\npolicy-improvement step to learn the model. At test time, MCTS is combined with\nthe neural policy to predict the target node. Experimental results on several\ngraph-walking benchmarks show that M-Walk is able to learn better policies than\nother RL-based methods, which are mainly based on policy gradients. M-Walk also\noutperforms traditional KBC baselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.04675v1",
            "date": "2018-02-13 15:13:28+00:00",
            "title": "Attention based Sentence Extraction from Scientific Articles using Pseudo-Labeled data",
            "authors": [
                "Parth Mehta",
                "Gaurav Arora",
                "Prasenjit Majumder"
            ],
            "abstract": "In this work, we present a weakly supervised sentence extraction technique\nfor identifying important sentences in scientific papers that are worthy of\ninclusion in the abstract. We propose a new attention based deep learning\narchitecture that jointly learns to identify important content, as well as the\ncue phrases that are indicative of summary worthy sentences. We propose a new\ncontext embedding technique for determining the focus of a given paper using\ntopic models and use it jointly with an LSTM based sequence encoder to learn\nattention weights across the sentence words. We use a collection of articles\npublicly available through ACL anthology for our experiments. Our system\nachieves a performance that is better, in terms of several ROUGE metrics, as\ncompared to several state of art extractive techniques. It also generates more\ncoherent summaries and preserves the overall structure of the document.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.05300v4",
            "date": "2018-02-14 19:48:50+00:00",
            "title": "Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise",
            "authors": [
                "Dan Hendrycks",
                "Mantas Mazeika",
                "Duncan Wilson",
                "Kevin Gimpel"
            ],
            "abstract": "The growing importance of massive datasets used for deep learning makes\nrobustness to label noise a critical property for classifiers to have. Sources\nof label noise include automatic labeling, non-expert labeling, and label\ncorruption by data poisoning adversaries. Numerous previous works assume that\nno source of labels can be trusted. We relax this assumption and assume that a\nsmall subset of the training data is trusted. This enables substantial label\ncorruption robustness performance gains. In addition, particularly severe label\nnoise can be combated by using a set of trusted data with clean labels. We\nutilize trusted data by proposing a loss correction technique that utilizes\ntrusted examples in a data-efficient manner to mitigate the effects of label\nnoise on deep neural network classifiers. Across vision and natural language\nprocessing tasks, we experiment with various label noises at several strengths,\nand show that our method significantly outperforms existing methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.05415v2",
            "date": "2018-02-15 06:17:51+00:00",
            "title": "Teaching Machines to Code: Neural Markup Generation with Visual Attention",
            "authors": [
                "Sumeet S. Singh"
            ],
            "abstract": "We present a neural transducer model with visual attention that learns to\ngenerate LaTeX markup of a real-world math formula given its image. Applying\nsequence modeling and transduction techniques that have been very successful\nacross modalities such as natural language, image, handwriting, speech and\naudio; we construct an image-to-markup model that learns to produce\nsyntactically and semantically correct LaTeX markup code over 150 words long\nand achieves a BLEU score of 89%; improving upon the previous state-of-art for\nthe Im2Latex problem. We also demonstrate with heat-map visualization how\nattention helps in interpreting the model and can pinpoint (detect and\nlocalize) symbols on the image accurately despite having been trained without\nany bounding box data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.05630v2",
            "date": "2018-02-15 16:05:02+00:00",
            "title": "CNN+LSTM Architecture for Speech Emotion Recognition with Data Augmentation",
            "authors": [
                "Caroline Etienne",
                "Guillaume Fidanza",
                "Andrei Petrovskii",
                "Laurence Devillers",
                "Benoit Schmauch"
            ],
            "abstract": "In this work we design a neural network for recognizing emotions in speech,\nusing the IEMOCAP dataset. Following the latest advances in audio analysis, we\nuse an architecture involving both convolutional layers, for extracting\nhigh-level features from raw spectrograms, and recurrent ones for aggregating\nlong-term dependencies. We examine the techniques of data augmentation with\nvocal track length perturbation, layer-wise optimizer adjustment, batch\nnormalization of recurrent layers and obtain highly competitive results of\n64.5% for weighted accuracy and 61.7% for unweighted accuracy on four emotions.",
            "journal ref": "Workshop on Speech, Music and Mind 2018",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1802.05694v1",
            "date": "2018-02-15 18:22:58+00:00",
            "title": "Multinomial Adversarial Networks for Multi-Domain Text Classification",
            "authors": [
                "Xilun Chen",
                "Claire Cardie"
            ],
            "abstract": "Many text classification tasks are known to be highly domain-dependent.\nUnfortunately, the availability of training data can vary drastically across\ndomains. Worse still, for some domains there may not be any annotated data at\nall. In this work, we propose a multinomial adversarial network (MAN) to tackle\nthe text classification problem in this real-world multidomain setting (MDTC).\nWe provide theoretical justifications for the MAN framework, proving that\ndifferent instances of MANs are essentially minimizers of various f-divergence\nmetrics (Ali and Silvey, 1966) among multiple probability distributions. MANs\nare thus a theoretically sound generalization of traditional adversarial\nnetworks that discriminate over two distributions. More specifically, for the\nMDTC task, MAN learns features that are invariant across multiple domains by\nresorting to its ability to reduce the divergence among the feature\ndistributions of each domain. We present experimental results showing that MANs\nsignificantly outperform the prior art on the MDTC task. We also show that MANs\nachieve state-of-the-art performance for domains with no labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05695v2",
            "date": "2018-02-15 18:25:32+00:00",
            "title": "Explainable Prediction of Medical Codes from Clinical Text",
            "authors": [
                "James Mullenbach",
                "Sarah Wiegreffe",
                "Jon Duke",
                "Jimeng Sun",
                "Jacob Eisenstein"
            ],
            "abstract": "Clinical notes are text documents that are created by clinicians for each\npatient encounter. They are typically accompanied by medical codes, which\ndescribe the diagnosis and treatment. Annotating these codes is labor intensive\nand error prone; furthermore, the connection between the codes and the text is\nnot annotated, obscuring the reasons and details behind specific diagnoses and\ntreatments. We present an attentional convolutional network that predicts\nmedical codes from clinical text. Our method aggregates information across the\ndocument using a convolutional neural network, and uses an attention mechanism\nto select the most relevant segments for each of the thousands of possible\ncodes. The method is accurate, achieving precision@8 of 0.71 and a Micro-F1 of\n0.54, which are both better than the prior state of the art. Furthermore,\nthrough an interpretability evaluation by a physician, we show that the\nattention mechanism identifies meaningful explanations for each code assignment",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.05853v1",
            "date": "2018-02-16 07:45:53+00:00",
            "title": "Articulatory information and Multiview Features for Large Vocabulary Continuous Speech Recognition",
            "authors": [
                "Vikramjit Mitra",
                "Wen Wang",
                "Chris Bartels",
                "Horacio Franco",
                "Dimitra Vergyri"
            ],
            "abstract": "This paper explores the use of multi-view features and their discriminative\ntransforms in a convolutional deep neural network (CNN) architecture for a\ncontinuous large vocabulary speech recognition task. Mel-filterbank energies\nand perceptually motivated forced damped oscillator coefficient (DOC) features\nare used after feature-space maximum-likelihood linear regression (fMLLR)\ntransforms, which are combined and fed as a multi-view feature to a single CNN\nacoustic model. Use of multi-view feature representation demonstrated\nsignificant reduction in word error rates (WERs) compared to the use of\nindividual features by themselves. In addition, when articulatory information\nwas used as an additional input to a fused deep neural network (DNN) and CNN\nacoustic model, it was found to demonstrate further reduction in WER for the\nSwitchboard subset and the CallHome subset (containing partly non-native\naccented speech) of the NIST 2000 conversational telephone speech test set,\nreducing the error rate by 12% relative to the baseline in both cases. This\nwork shows that multi-view features in association with articulatory\ninformation can improve speech recognition robustness to spontaneous and\nnon-native speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06003v1",
            "date": "2018-02-13 11:33:27+00:00",
            "title": "Structured-based Curriculum Learning for End-to-end English-Japanese Speech Translation",
            "authors": [
                "Takatomo Kano",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Sequence-to-sequence attentional-based neural network architectures have been\nshown to provide a powerful model for machine translation and speech\nrecognition. Recently, several works have attempted to extend the models for\nend-to-end speech translation task. However, the usefulness of these models\nwere only investigated on language pairs with similar syntax and word order\n(e.g., English-French or English-Spanish). In this work, we focus on end-to-end\nspeech translation tasks on syntactically distant language pairs (e.g.,\nEnglish-Japanese) that require distant word reordering.\n  To guide the encoder-decoder attentional model to learn this difficult\nproblem, we propose a structured-based curriculum learning strategy.\n  Unlike conventional curriculum learning that gradually emphasizes difficult\ndata examples, we formalize learning strategies from easier network structures\nto more difficult network structures. Here, we start the training with\nend-to-end encoder-decoder for speech recognition or text-based machine\ntranslation task then gradually move to end-to-end speech translation task. The\nexperiment results show that the proposed approach could provide significant\nimprovements in comparison with the one without curriculum learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06006v3",
            "date": "2018-02-14 18:24:41+00:00",
            "title": "Neural Voice Cloning with a Few Samples",
            "authors": [
                "Sercan O. Arik",
                "Jitong Chen",
                "Kainan Peng",
                "Wei Ping",
                "Yanqi Zhou"
            ],
            "abstract": "Voice cloning is a highly desired feature for personalized speech interfaces.\nNeural network based speech synthesis has been shown to generate high quality\nspeech for a large number of speakers. In this paper, we introduce a neural\nvoice cloning system that takes a few audio samples as input. We study two\napproaches: speaker adaptation and speaker encoding. Speaker adaptation is\nbased on fine-tuning a multi-speaker generative model with a few cloning\nsamples. Speaker encoding is based on training a separate model to directly\ninfer a new speaker embedding from cloning audios and to be used with a\nmulti-speaker generative model. In terms of naturalness of the speech and its\nsimilarity to original speaker, both approaches can achieve good performance,\neven with very few cloning audios. While speaker adaptation can achieve better\nnaturalness and similarity, the cloning time or required memory for the speaker\nencoding approach is significantly less, making it favorable for low-resource\ndeployment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06007v1",
            "date": "2018-02-14 19:44:24+00:00",
            "title": "Authorship Attribution Using the Chaos Game Representation",
            "authors": [
                "Daniel Lichtblau",
                "Catalin Stoean"
            ],
            "abstract": "The Chaos Game Representation, a method for creating images from nucleotide\nsequences, is modified to make images from chunks of text documents. Machine\nlearning methods are then applied to train classifiers based on authorship.\nExperiments are conducted on several benchmark data sets in English, including\nthe widely used Federalist Papers, and one in Portuguese. Validation results\nfor the trained classifiers are competitive with the best methods in prior\nliterature. The methodology is also successfully applied for text\ncategorization with encouraging results. One classifier method is moreover seen\nto hold promise for the task of digital fingerprinting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06024v2",
            "date": "2018-02-16 16:50:27+00:00",
            "title": "Towards a Continuous Knowledge Learning Engine for Chatbots",
            "authors": [
                "Sahisnu Mazumder",
                "Nianzu Ma",
                "Bing Liu"
            ],
            "abstract": "Although chatbots have been very popular in recent years, they still have\nsome serious weaknesses which limit the scope of their applications. One major\nweakness is that they cannot learn new knowledge during the conversation\nprocess, i.e., their knowledge is fixed beforehand and cannot be expanded or\nupdated during conversation. In this paper, we propose to build a general\nknowledge learning engine for chatbots to enable them to continuously and\ninteractively learn new knowledge during conversations. As time goes by, they\nbecome more and more knowledgeable and better and better at learning and\nconversation. We model the task as an open-world knowledge base completion\nproblem and propose a novel technique called lifelong interactive learning and\ninference (LiLi) to solve it. LiLi works by imitating how humans acquire\nknowledge and perform inference during an interactive conversation. Our\nexperimental results show LiLi is highly promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06209v1",
            "date": "2018-02-17 08:39:47+00:00",
            "title": "Sentiment Analysis on Speaker Specific Speech Data",
            "authors": [
                "Maghilnan S",
                "Rajesh Kumar M"
            ],
            "abstract": "Sentiment analysis has evolved over past few decades, most of the work in it\nrevolved around textual sentiment analysis with text mining techniques. But\naudio sentiment analysis is still in a nascent stage in the research community.\nIn this proposed research, we perform sentiment analysis on speaker\ndiscriminated speech transcripts to detect the emotions of the individual\nspeakers involved in the conversation. We analyzed different techniques to\nperform speaker discrimination and sentiment analysis to find efficient\nalgorithms to perform this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06428v1",
            "date": "2018-02-18 19:27:24+00:00",
            "title": "Improving Mild Cognitive Impairment Prediction via Reinforcement Learning and Dialogue Simulation",
            "authors": [
                "Fengyi Tang",
                "Kaixiang Lin",
                "Ikechukwu Uchendu",
                "Hiroko H. Dodge",
                "Jiayu Zhou"
            ],
            "abstract": "Mild cognitive impairment (MCI) is a prodromal phase in the progression from\nnormal aging to dementia, especially Alzheimers disease. Even though there is\nmild cognitive decline in MCI patients, they have normal overall cognition and\nthus is challenging to distinguish from normal aging. Using transcribed data\nobtained from recorded conversational interactions between participants and\ntrained interviewers, and applying supervised learning models to these data, a\nrecent clinical trial has shown a promising result in differentiating MCI from\nnormal aging. However, the substantial amount of interactions with medical\nstaff can still incur significant medical care expenses in practice. In this\npaper, we propose a novel reinforcement learning (RL) framework to train an\nefficient dialogue agent on existing transcripts from clinical trials.\nSpecifically, the agent is trained to sketch disease-specific lexical\nprobability distribution, and thus to converse in a way that maximizes the\ndiagnosis accuracy and minimizes the number of conversation turns. We evaluate\nthe performance of the proposed reinforcement learning framework on the MCI\ndiagnosis from a real clinical trial. The results show that while using only a\nfew turns of conversation, our framework can significantly outperform\nstate-of-the-art supervised learning approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.06757v1",
            "date": "2018-02-06 11:58:58+00:00",
            "title": "Deep Inference of Personality Traits by Integrating Image and Word Use in Social Networks",
            "authors": [
                "Guillem Cucurull",
                "Pau Rodr\u00edguez",
                "V. Oguz Yazici",
                "Josep M. Gonfaus",
                "F. Xavier Roca",
                "Jordi Gonz\u00e0lez"
            ],
            "abstract": "Social media, as a major platform for communication and information exchange,\nis a rich repository of the opinions and sentiments of 2.3 billion users about\na vast spectrum of topics. To sense the whys of certain social user's demands\nand cultural-driven interests, however, the knowledge embedded in the 1.8\nbillion pictures which are uploaded daily in public profiles has just started\nto be exploited since this process has been typically been text-based.\nFollowing this trend on visual-based social analysis, we present a novel\nmethodology based on Deep Learning to build a combined image-and-text based\npersonality trait model, trained with images posted together with words found\nhighly correlated to specific personality traits. So the key contribution here\nis to explore whether OCEAN personality trait modeling can be addressed based\non images, here called \\emph{Mind{P}ics}, appearing with certain tags with\npsychological insights. We found that there is a correlation between those\nposted images and their accompanying texts, which can be successfully modeled\nusing deep neural networks for personality estimation. The experimental results\nare consistent with previous cyber-psychology results based on texts or images.\nIn addition, classification results on some traits show that some patterns\nemerge in the set of images corresponding to a specific text, in essence to\nthose representing an abstract concept. These results open new avenues of\nresearch for further refining the proposed personality model under the\nsupervision of psychology experts.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1802.06861v1",
            "date": "2018-02-16 07:42:35+00:00",
            "title": "Interpreting DNN output layer activations: A strategy to cope with unseen data in speech recognition",
            "authors": [
                "Vikramjit Mitra",
                "Horacio Franco"
            ],
            "abstract": "Unseen data can degrade performance of deep neural net acoustic models. To\ncope with unseen data, adaptation techniques are deployed. For unlabeled unseen\ndata, one must generate some hypothesis given an existing model, which is used\nas the label for model adaptation. However, assessing the goodness of the\nhypothesis can be difficult, and an erroneous hypothesis can lead to poorly\ntrained models. In such cases, a strategy to select data having reliable\nhypothesis can ensure better model adaptation. This work proposes a\ndata-selection strategy for DNN model adaptation, where DNN output layer\nactivations are used to ascertain the goodness of a generated hypothesis. In a\nDNN acoustic model, the output layer activations are used to generate target\nclass probabilities. Under unseen data conditions, the difference between the\nmost probable target and the next most probable target is decreased compared to\nthe same for seen data, indicating that the model may be uncertain while\ngenerating its hypothesis. This work proposes a strategy to assess a model's\nperformance by analyzing the output layer activations by using a distance\nmeasure between the most likely target and the next most likely target, which\nis used for data selection for performing unsupervised adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06894v2",
            "date": "2018-02-19 22:33:56+00:00",
            "title": "Learning Hidden Markov Models from Pairwise Co-occurrences with Application to Topic Modeling",
            "authors": [
                "Kejun Huang",
                "Xiao Fu",
                "Nicholas D. Sidiropoulos"
            ],
            "abstract": "We present a new algorithm for identifying the transition and emission\nprobabilities of a hidden Markov model (HMM) from the emitted data.\nExpectation-maximization becomes computationally prohibitive for long\nobservation records, which are often required for identification. The new\nalgorithm is particularly suitable for cases where the available sample size is\nlarge enough to accurately estimate second-order output probabilities, but not\nhigher-order ones. We show that if one is only able to obtain a reliable\nestimate of the pairwise co-occurrence probabilities of the emissions, it is\nstill possible to uniquely identify the HMM if the emission probability is\n\\emph{sufficiently scattered}. We apply our method to hidden topic Markov\nmodeling, and demonstrate that we can learn topics with higher quality if\ndocuments are modeled as observations of HMMs sharing the same emission (topic)\nprobability, compared to the simple but widely used bag-of-words model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06901v3",
            "date": "2018-02-19 22:57:54+00:00",
            "title": "Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement",
            "authors": [
                "Jason Lee",
                "Elman Mansimov",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose a conditional non-autoregressive neural sequence model based on\niterative refinement. The proposed model is designed based on the principles of\nlatent variable models and denoising autoencoders, and is generally applicable\nto any sequence generation task. We extensively evaluate the proposed model on\nmachine translation (En-De and En-Ro) and image caption generation, and observe\nthat it significantly speeds up decoding while maintaining the generation\nquality comparable to the autoregressive counterpart.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.06941v1",
            "date": "2018-02-20 02:51:34+00:00",
            "title": "Distilling Knowledge Using Parallel Data for Far-field Speech Recognition",
            "authors": [
                "Jiangyan Yi",
                "Jianhua Tao",
                "Zhengqi Wen",
                "Bin Liu"
            ],
            "abstract": "In order to improve the performance for far-field speech recognition, this\npaper proposes to distill knowledge from the close-talking model to the\nfar-field model using parallel data. The close-talking model is called the\nteacher model. The far-field model is called the student model. The student\nmodel is trained to imitate the output distributions of the teacher model. This\nconstraint can be realized by minimizing the Kullback-Leibler (KL) divergence\nbetween the output distribution of the student model and the teacher model.\nExperimental results on AMI corpus show that the best student model achieves up\nto 4.7% absolute word error rate (WER) reduction when compared with the\nconventionally-trained baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07089v2",
            "date": "2018-02-20 12:42:07+00:00",
            "title": "Attentive Tensor Product Learning",
            "authors": [
                "Qiuyuan Huang",
                "Li Deng",
                "Dapeng Wu",
                "Chang Liu",
                "Xiaodong He"
            ],
            "abstract": "This paper proposes a new architecture - Attentive Tensor Product Learning\n(ATPL) - to represent grammatical structures in deep learning models. ATPL is a\nnew architecture to bridge this gap by exploiting Tensor Product\nRepresentations (TPR), a structured neural-symbolic model developed in\ncognitive science, aiming to integrate deep learning with explicit language\nstructures and rules. The key ideas of ATPL are: 1) unsupervised learning of\nrole-unbinding vectors of words via TPR-based deep neural network; 2) employing\nattention modules to compute TPR; and 3) integration of TPR with typical deep\nlearning architectures including Long Short-Term Memory (LSTM) and Feedforward\nNeural Network (FFNN). The novelty of our approach lies in its ability to\nextract the grammatical structure of a sentence by using role-unbinding\nvectors, which are obtained in an unsupervised manner. This ATPL approach is\napplied to 1) image captioning, 2) part of speech (POS) tagging, and 3)\nconstituency parsing of a sentence. Experimental results demonstrate the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07420v2",
            "date": "2018-02-21 04:09:26+00:00",
            "title": "Sequence-based Multi-lingual Low Resource Speech Recognition",
            "authors": [
                "Siddharth Dalmia",
                "Ramon Sanabria",
                "Florian Metze",
                "Alan W. Black"
            ],
            "abstract": "Techniques for multi-lingual and cross-lingual speech recognition can help in\nlow resource scenarios, to bootstrap systems and enable analysis of new\nlanguages and domains. End-to-end approaches, in particular sequence-based\ntechniques, are attractive because of their simplicity and elegance. While it\nis possible to integrate traditional multi-lingual bottleneck feature\nextractors as front-ends, we show that end-to-end multi-lingual training of\nsequence models is effective on context independent models trained using\nConnectionist Temporal Classification (CTC) loss. We show that our model\nimproves performance on Babel languages by over 6% absolute in terms of\nword/phoneme error rate when compared to mono-lingual systems built in the same\nsetting for these languages. We also show that the trained model can be adapted\ncross-lingually to an unseen language using just 25% of the target data. We\nshow that training on multiple languages is important for very low resource\ncross-lingual target scenarios, but not for multi-lingual testing scenarios.\nHere, it appears beneficial to include large well prepared datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.07860v2",
            "date": "2018-02-22 00:37:49+00:00",
            "title": "Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics",
            "authors": [
                "Arindam Jati",
                "Panayiotis Georgiou"
            ],
            "abstract": "Learning speaker-specific features is vital in many applications like speaker\nrecognition, diarization and speech recognition. This paper provides a novel\napproach, we term Neural Predictive Coding (NPC), to learn speaker-specific\ncharacteristics in a completely unsupervised manner from large amounts of\nunlabeled training data that even contain many non-speech events and\nmulti-speaker audio streams. The NPC framework exploits the proposed short-term\nactive-speaker stationarity hypothesis which assumes two temporally-close short\nspeech segments belong to the same speaker, and thus a common representation\nthat can encode the commonalities of both the segments, should capture the\nvocal characteristics of that speaker. We train a convolutional deep siamese\nnetwork to produce \"speaker embeddings\" by learning to separate `same' vs\n`different' speaker pairs which are generated from an unlabeled data of audio\nstreams. Two sets of experiments are done in different scenarios to evaluate\nthe strength of NPC embeddings and compare with state-of-the-art in-domain\nsupervised methods. First, two speaker identification experiments with\ndifferent context lengths are performed in a scenario with comparatively\nlimited within-speaker channel variability. NPC embeddings are found to perform\nthe best at short duration experiment, and they provide complementary\ninformation to i-vectors for full utterance experiments. Second, a large scale\nspeaker verification task having a wide range of within-speaker channel\nvariability is adopted as an upper-bound experiment where comparisons are drawn\nwith in-domain supervised methods.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  vol. 27, no. 10, pp. 1577-1589, Oct. 2019",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1802.07997v1",
            "date": "2018-02-22 11:55:28+00:00",
            "title": "Generating High-Quality Query Suggestion Candidates for Task-Based Search",
            "authors": [
                "Heng Ding",
                "Shuo Zhang",
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "We address the task of generating query suggestions for task-based search.\nThe current state of the art relies heavily on suggestions provided by a major\nsearch engine. In this paper, we solve the task without reliance on search\nengines. Specifically, we focus on the first step of a two-stage pipeline\napproach, which is dedicated to the generation of query suggestion candidates.\nWe present three methods for generating candidate suggestions and apply them on\nmultiple information sources. Using a purpose-built test collection, we find\nthat these methods are able to generate high-quality suggestion candidates.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.08010v1",
            "date": "2018-02-22 12:30:13+00:00",
            "title": "Towards an Understanding of Entity-Oriented Search Intents",
            "authors": [
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "Entity-oriented search deals with a wide variety of information needs, from\ndisplaying direct answers to interacting with services. In this work, we aim to\nunderstand what are prominent entity-oriented search intents and how they can\nbe fulfilled. We develop a scheme of entity intent categories, and use them to\nannotate a sample of queries. Specifically, we annotate unique query refiners\non the level of entity types. We observe that, on average, over half of those\nrefiners seek to interact with a service, while over a quarter of the refiners\nsearch for information that may be looked up in a knowledge base.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1802.08129v1",
            "date": "2018-02-15 19:12:03+00:00",
            "title": "Multimodal Explanations: Justifying Decisions and Pointing to the Evidence",
            "authors": [
                "Dong Huk Park",
                "Lisa Anne Hendricks",
                "Zeynep Akata",
                "Anna Rohrbach",
                "Bernt Schiele",
                "Trevor Darrell",
                "Marcus Rohrbach"
            ],
            "abstract": "Deep models that are both effective and explainable are desirable in many\nsettings; prior explainable models have been unimodal, offering either\nimage-based visualization of attention weights or text-based generation of\npost-hoc justifications. We propose a multimodal approach to explanation, and\nargue that the two modalities provide complementary explanatory strengths. We\ncollect two new datasets to define and evaluate this task, and propose a novel\nmodel which can provide joint textual rationale generation and attention\nvisualization. Our datasets define visual and textual justifications of a\nclassification decision for activity recognition tasks (ACT-X) and for visual\nquestion answering tasks (VQA-X). We quantitatively show that training with the\ntextual explanations not only yields better textual justification models, but\nalso better localizes the evidence that supports the decision. We also\nqualitatively show cases where visual explanation is more insightful than\ntextual explanation, and vice versa, supporting our thesis that multimodal\nexplanation models offer significant benefits over unimodal approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.08218v4",
            "date": "2018-02-22 18:16:53+00:00",
            "title": "VizWiz Grand Challenge: Answering Visual Questions from Blind People",
            "authors": [
                "Danna Gurari",
                "Qing Li",
                "Abigale J. Stangl",
                "Anhong Guo",
                "Chi Lin",
                "Kristen Grauman",
                "Jiebo Luo",
                "Jeffrey P. Bigham"
            ],
            "abstract": "The study of algorithms to automatically answer visual questions currently is\nmotivated by visual question answering (VQA) datasets constructed in artificial\nVQA settings. We propose VizWiz, the first goal-oriented VQA dataset arising\nfrom a natural VQA setting. VizWiz consists of over 31,000 visual questions\noriginating from blind people who each took a picture using a mobile phone and\nrecorded a spoken question about it, together with 10 crowdsourced answers per\nvisual question. VizWiz differs from the many existing VQA datasets because (1)\nimages are captured by blind photographers and so are often poor quality, (2)\nquestions are spoken and so are more conversational, and (3) often visual\nquestions cannot be answered. Evaluation of modern algorithms for answering\nvisual questions and deciding if a visual question is answerable reveals that\nVizWiz is a challenging dataset. We introduce this dataset to encourage a\nlarger community to develop more generalized algorithms that can assist blind\npeople.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1802.08301v1",
            "date": "2018-02-22 21:13:47+00:00",
            "title": "Content-Based Citation Recommendation",
            "authors": [
                "Chandra Bhagavatula",
                "Sergey Feldman",
                "Russell Power",
                "Waleed Ammar"
            ],
            "abstract": "We present a content-based method for recommending citations in an academic\npaper draft. We embed a given query document into a vector space, then use its\nnearest neighbors as candidates, and rerank the candidates using a\ndiscriminative model trained to distinguish between observed and unobserved\ncitations. Unlike previous work, our method does not require metadata such as\nauthor names which can be missing, e.g., during the peer review process.\nWithout using metadata, our method outperforms the best reported results on\nPubMed and DBLP datasets with relative improvements of over 18% in F1@20 and\nover 22% in MRR. We show empirically that, although adding metadata improves\nthe performance on standard metrics, it favors self-citations which are less\nuseful in a citation recommendation setup. We release an online portal\n(http://labs.semanticscholar.org/citeomatic/) for citation recommendation based\non our method, and a new dataset OpenCorpus of 7 million research articles to\nfacilitate future research on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08314v1",
            "date": "2018-02-22 22:01:05+00:00",
            "title": "High Order Recurrent Neural Networks for Acoustic Modelling",
            "authors": [
                "Chao Zhang",
                "Philip Woodland"
            ],
            "abstract": "Vanishing long-term gradients are a major issue in training standard\nrecurrent neural networks (RNNs), which can be alleviated by long short-term\nmemory (LSTM) models with memory cells. However, the extra parameters\nassociated with the memory cells mean an LSTM layer has four times as many\nparameters as an RNN with the same hidden vector size. This paper addresses the\nvanishing gradient problem using a high order RNN (HORNN) which has additional\nconnections from multiple previous time steps. Speech recognition experiments\nusing British English multi-genre broadcast (MGB3) data showed that the\nproposed HORNN architectures for rectified linear unit and sigmoid activation\nfunctions reduced word error rates (WER) by 4.2% and 6.3% over the\ncorresponding RNNs, and gave similar WERs to a (projected) LSTM while using\nonly 20%--50% of the recurrent layer parameters and computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08375v2",
            "date": "2018-02-23 03:44:15+00:00",
            "title": "Reusing Weights in Subword-aware Neural Language Models",
            "authors": [
                "Zhenisbek Assylbekov",
                "Rustem Takhanov"
            ],
            "abstract": "We propose several ways of reusing subword embeddings and other weights in\nsubword-aware neural language models. The proposed techniques do not benefit a\ncompetitive character-aware model, but some of them improve the performance of\nsyllable- and morpheme-aware models while showing significant reductions in\nmodel sizes. We discover a simple hands-on principle: in a multi-layer input\nembedding model, layers should be tied consecutively bottom-up if reused at\noutput. Our best morpheme-aware model with properly reused weights beats the\ncompetitive word-level model by a large margin across multiple languages and\nhas 20%-87% fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08614v1",
            "date": "2018-02-23 15:56:07+00:00",
            "title": "Visualizing the Flow of Discourse with a Concept Ontology",
            "authors": [
                "Baoxu Shi",
                "Tim Weninger"
            ],
            "abstract": "Understanding and visualizing human discourse has long being a challenging\ntask. Although recent work on argument mining have shown success in classifying\nthe role of various sentences, the task of recognizing concepts and\nunderstanding the ways in which they are discussed remains challenging. Given\nan email thread or a transcript of a group discussion, our task is to extract\nthe relevant concepts and understand how they are referenced and re-referenced\nthroughout the discussion. In the present work, we present a preliminary\napproach for extracting and visualizing group discourse by adapting Wikipedia's\ncategory hierarchy to be an external concept ontology. From a user study, we\nfound that our method achieved better results than 4 strong alternative\napproaches, and we illustrate our visualization method based on the extracted\ndiscourse flows.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.08690v1",
            "date": "2018-02-23 19:00:01+00:00",
            "title": "\"You are no Jack Kennedy\": On Media Selection of Highlights from Presidential Debates",
            "authors": [
                "Chenhao Tan",
                "Hao Peng",
                "Noah A. Smith"
            ],
            "abstract": "Political speeches and debates play an important role in shaping the images\nof politicians, and the public often relies on media outlets to select bits of\npolitical communication from a large pool of utterances. It is an important\nresearch question to understand what factors impact this selection process.\n  To quantitatively explore the selection process, we build a three- decade\ndataset of presidential debate transcripts and post-debate coverage. We first\nexamine the effect of wording and propose a binary classification framework\nthat controls for both the speaker and the debate situation. We find that\ncrowdworkers can only achieve an accuracy of 60% in this task, indicating that\nmedia choices are not entirely obvious. Our classifiers outperform crowdworkers\non average, mainly in primary debates. We also compare important factors from\ncrowdworkers' free-form explanations with those from data-driven methods and\nfind interesting differences. Few crowdworkers mentioned that \"context\nmatters\", whereas our data show that well-quoted sentences are more distinct\nfrom the previous utterance by the same speaker than less-quoted sentences.\nFinally, we examine the aggregate effect of media preferences towards different\nwordings to understand the extent of fragmentation among media outlets. By\nanalyzing a bipartite graph built from quoting behavior in our data, we observe\na decreasing trend in bipartisan coverage.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1802.09059v1",
            "date": "2018-02-25 18:51:53+00:00",
            "title": "One Single Deep Bidirectional LSTM Network for Word Sense Disambiguation of Text Data",
            "authors": [
                "Ahmad Pesaranghader",
                "Ali Pesaranghader",
                "Stan Matwin",
                "Marina Sokolova"
            ],
            "abstract": "Due to recent technical and scientific advances, we have a wealth of\ninformation hidden in unstructured text data such as offline/online narratives,\nresearch articles, and clinical reports. To mine these data properly,\nattributable to their innate ambiguity, a Word Sense Disambiguation (WSD)\nalgorithm can avoid numbers of difficulties in Natural Language Processing\n(NLP) pipeline. However, considering a large number of ambiguous words in one\nlanguage or technical domain, we may encounter limiting constraints for proper\ndeployment of existing WSD models. This paper attempts to address the problem\nof one-classifier-per-one-word WSD algorithms by proposing a single\nBidirectional Long Short-Term Memory (BLSTM) network which by considering\nsenses and context sequences works on all ambiguous words collectively.\nEvaluated on SensEval-3 benchmark, we show the result of our model is\ncomparable with top-performing WSD algorithms. We also discuss how applying\nadditional modifications alleviates the model fault and the need for more\ntraining data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.09777v1",
            "date": "2018-02-27 08:55:05+00:00",
            "title": "Gaussian meta-embeddings for efficient scoring of a heavy-tailed PLDA model",
            "authors": [
                "Niko Brummer",
                "Anna Silnova",
                "Lukas Burget",
                "Themos Stafylakis"
            ],
            "abstract": "Embeddings in machine learning are low-dimensional representations of complex\ninput patterns, with the property that simple geometric operations like\nEuclidean distances and dot products can be used for classification and\ncomparison tasks. The proposed meta-embeddings are special embeddings that live\nin more general inner product spaces. They are designed to propagate\nuncertainty to the final output in speaker recognition and similar\napplications. The familiar Gaussian PLDA model (GPLDA) can be re-formulated as\nan extractor for Gaussian meta-embeddings (GMEs), such that likelihood ratio\nscores are given by Hilbert space inner products between Gaussian likelihood\nfunctions. GMEs extracted by the GPLDA model have fixed precisions and do not\npropagate uncertainty. We show that a generalization to heavy-tailed PLDA gives\nGMEs with variable precisions, which do propagate uncertainty. Experiments on\nNIST SRE 2010 and 2016 show that the proposed method applied to i-vectors\nwithout length normalization is up to 20% more accurate than GPLDA applied to\nlength-normalized ivectors.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1802.09913v2",
            "date": "2018-02-27 14:38:43+00:00",
            "title": "Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces",
            "authors": [
                "Isabelle Augenstein",
                "Sebastian Ruder",
                "Anders S\u00f8gaard"
            ],
            "abstract": "We combine multi-task learning and semi-supervised learning by inducing a\njoint embedding space between disparate label spaces and learning transfer\nfunctions between label embeddings, enabling us to jointly leverage unlabelled\ndata and auxiliary, annotated datasets. We evaluate our approach on a variety\nof sequence classification tasks with disparate label spaces. We outperform\nstrong single and multi-task baselines and achieve a new state-of-the-art for\ntopic-based sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.09914v1",
            "date": "2018-02-23 16:50:16+00:00",
            "title": "High-Dimensional Vector Semantics",
            "authors": [
                "M. Andrecut"
            ],
            "abstract": "In this paper we explore the \"vector semantics\" problem from the perspective\nof \"almost orthogonal\" property of high-dimensional random vectors. We show\nthat this intriguing property can be used to \"memorize\" random vectors by\nsimply adding them, and we provide an efficient probabilistic solution to the\nset membership problem. Also, we discuss several applications to word context\nvector embeddings, document sentences similarity, and spam filtering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00057v2",
            "date": "2018-02-19 06:51:01+00:00",
            "title": "A Neural Multi-sequence Alignment TeCHnique (NeuMATCH)",
            "authors": [
                "Pelin Dogan",
                "Boyang Li",
                "Leonid Sigal",
                "Markus Gross"
            ],
            "abstract": "The alignment of heterogeneous sequential data (video to text) is an\nimportant and challenging problem. Standard techniques for this task, including\nDynamic Time Warping (DTW) and Conditional Random Fields (CRFs), suffer from\ninherent drawbacks. Mainly, the Markov assumption implies that, given the\nimmediate past, future alignment decisions are independent of further history.\nThe separation between similarity computation and alignment decision also\nprevents end-to-end training. In this paper, we propose an end-to-end neural\narchitecture where alignment actions are implemented as moving data between\nstacks of Long Short-term Memory (LSTM) blocks. This flexible architecture\nsupports a large variety of alignment tasks, including one-to-one, one-to-many,\nskipping unmatched elements, and (with extensions) non-monotonic alignment.\nExtensive experiments on semi-synthetic and real datasets show that our\nalgorithm outperforms state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.00344v1",
            "date": "2018-03-01 12:38:13+00:00",
            "title": "A Deep Learning Approach for Multimodal Deception Detection",
            "authors": [
                "Gangeshwar Krishnamurthy",
                "Navonil Majumder",
                "Soujanya Poria",
                "Erik Cambria"
            ],
            "abstract": "Automatic deception detection is an important task that has gained momentum\nin computational linguistics due to its potential applications. In this paper,\nwe propose a simple yet tough to beat multi-modal neural model for deception\ndetection. By combining features from different modalities such as video,\naudio, and text along with Micro-Expression features, we show that detecting\ndeception in real life videos can be more accurate. Experimental results on a\ndataset of real-life deception videos show that our model outperforms existing\ntechniques for deception detection with an accuracy of 96.14% and ROC-AUC of\n0.9799.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00721v1",
            "date": "2018-03-02 05:07:33+00:00",
            "title": "Age Group Classification with Speech and Metadata Multimodality Fusion",
            "authors": [
                "Denys Katerenchuk"
            ],
            "abstract": "Children comprise a significant proportion of TV viewers and it is worthwhile\nto customize the experience for them. However, identifying who is a child in\nthe audience can be a challenging task. Identifying gender and age from audio\ncommands is a well-studied problem but is still very challenging to get good\naccuracy when the utterances are typically only a couple of seconds long. We\npresent initial studies of a novel method which combines utterances with user\nmetadata. In particular, we develop an ensemble of different machine learning\ntechniques on different subsets of data to improve child detection. Our initial\nresults show a 9.2\\% absolute improvement over the baseline, leading to a\nstate-of-the-art performance.",
            "journal ref": "Proceedings of the 15th Conference of the European Chapter of the\n  Association for Computational Linguistics: Volume 2, Short Papers, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.00860v1",
            "date": "2018-03-02 14:21:16+00:00",
            "title": "Can we steal your vocal identity from the Internet?: Initial investigation of cloning Obama's voice using GAN, WaveNet and low-quality found data",
            "authors": [
                "Jaime Lorenzo-Trueba",
                "Fuming Fang",
                "Xin Wang",
                "Isao Echizen",
                "Junichi Yamagishi",
                "Tomi Kinnunen"
            ],
            "abstract": "Thanks to the growing availability of spoofing databases and rapid advances\nin using them, systems for detecting voice spoofing attacks are becoming more\nand more capable, and error rates close to zero are being reached for the\nASVspoof2015 database. However, speech synthesis and voice conversion paradigms\nthat are not considered in the ASVspoof2015 database are appearing. Such\nexamples include direct waveform modelling and generative adversarial networks.\nWe also need to investigate the feasibility of training spoofing systems using\nonly low-quality found data. For that purpose, we developed a generative\nadversarial network-based speech enhancement system that improves the quality\nof speech data found in publicly available sources. Using the enhanced data, we\ntrained state-of-the-art text-to-speech and voice conversion models and\nevaluated them in terms of perceptual speech quality and speaker similarity.\nThe results show that the enhancement models significantly improved the SNR of\nlow-quality degraded data found in publicly available sources and that they\nsignificantly improved the perceptual cleanliness of the source speech without\nsignificantly degrading the naturalness of the voice. However, the results also\nshow limitations when generating speech with the low-quality found data.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1803.00886v1",
            "date": "2018-02-27 12:45:16+00:00",
            "title": "Deep factorization for speech signal",
            "authors": [
                "Lantian Li",
                "Dong Wang",
                "Yixiang Chen",
                "Ying Shi",
                "Zhiyuan Tang",
                "Thomas Fang Zheng"
            ],
            "abstract": "Various informative factors mixed in speech signals, leading to great\ndifficulty when decoding any of the factors. An intuitive idea is to factorize\neach speech frame into individual informative factors, though it turns out to\nbe highly difficult. Recently, we found that speaker traits, which were assumed\nto be long-term distributional properties, are actually short-time patterns,\nand can be learned by a carefully designed deep neural network (DNN). This\ndiscovery motivated a cascade deep factorization (CDF) framework that will be\npresented in this paper. The proposed framework infers speech factors in a\nsequential way, where factors previously inferred are used as conditional\nvariables when inferring other factors. We will show that this approach can\neffectively factorize speech signals, and using these factors, the original\nspeech spectrum can be recovered with a high accuracy. This factorization and\nreconstruction approach provides potential values for many speech processing\ntasks, e.g., speaker recognition and emotion recognition, as will be\ndemonstrated in the paper.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1803.01271v2",
            "date": "2018-03-04 00:20:29+00:00",
            "title": "An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling",
            "authors": [
                "Shaojie Bai",
                "J. Zico Kolter",
                "Vladlen Koltun"
            ],
            "abstract": "For most deep learning practitioners, sequence modeling is synonymous with\nrecurrent networks. Yet recent results indicate that convolutional\narchitectures can outperform recurrent networks on tasks such as audio\nsynthesis and machine translation. Given a new sequence modeling task or\ndataset, which architecture should one use? We conduct a systematic evaluation\nof generic convolutional and recurrent architectures for sequence modeling. The\nmodels are evaluated across a broad range of standard tasks that are commonly\nused to benchmark recurrent networks. Our results indicate that a simple\nconvolutional architecture outperforms canonical recurrent networks such as\nLSTMs across a diverse range of tasks and datasets, while demonstrating longer\neffective memory. We conclude that the common association between sequence\nmodeling and recurrent networks should be reconsidered, and convolutional\nnetworks should be regarded as a natural starting point for sequence modeling\ntasks. To assist related work, we have made code available at\nhttp://github.com/locuslab/TCN .",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.01686v5",
            "date": "2018-02-27 02:47:13+00:00",
            "title": "On Extended Long Short-term Memory and Dependent Bidirectional Recurrent Neural Network",
            "authors": [
                "Yuanhang Su",
                "C. -C. Jay Kuo"
            ],
            "abstract": "In this work, we first analyze the memory behavior in three recurrent neural\nnetworks (RNN) cells; namely, the simple RNN (SRN), the long short-term memory\n(LSTM) and the gated recurrent unit (GRU), where the memory is defined as a\nfunction that maps previous elements in a sequence to the current output. Our\nstudy shows that all three of them suffer rapid memory decay. Then, to\nalleviate this effect, we introduce trainable scaling factors that act like an\nattention mechanism to adjust memory decay adaptively. The new design is called\nthe extended LSTM (ELSTM). Finally, to design a system that is robust to\nprevious erroneous predictions, we propose a dependent bidirectional recurrent\nneural network (DBRNN). Extensive experiments are conducted on different\nlanguage tasks to demonstrate the superiority of the proposed ELSTM and DBRNN\nsolutions. The ELTSM has achieved up to 30% increase in the labeled attachment\nscore (LAS) as compared to LSTM and GRU in the dependency parsing (DP) task.\nOur models also outperform other state-of-the-art models such as bi-attention\nand convolutional sequence to sequence (convseq2seq) by close to 10% in the\nLAS. The code is released as an open source\n(https://github.com/yuanhangsu/ELSTM-DBRNN)",
            "journal ref": "Neurocomputing 356 (2019): 151-161",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.01937v1",
            "date": "2018-03-05 21:35:04+00:00",
            "title": "ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks",
            "authors": [
                "Kavita Ganesan"
            ],
            "abstract": "Evaluation of summarization tasks is extremely crucial to determining the\nquality of machine generated summaries. Over the last decade, ROUGE has become\nthe standard automatic evaluation measure for evaluating summarization tasks.\nWhile ROUGE has been shown to be effective in capturing n-gram overlap between\nsystem and human composed summaries, there are several limitations with the\nexisting ROUGE measures in terms of capturing synonymous concepts and coverage\nof topics. Thus, often times ROUGE scores do not reflect the true quality of\nsummaries and prevents multi-faceted evaluation of summaries (i.e. by topics,\nby overall content coverage and etc). In this paper, we introduce ROUGE 2.0,\nwhich has several updated measures of ROUGE: ROUGE-N+Synonyms, ROUGE-Topic,\nROUGE-Topic+Synonyms, ROUGE-TopicUniq and ROUGE-TopicUniq+Synonyms; all of\nwhich are improvements over the core ROUGE measures.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.02088v1",
            "date": "2018-03-06 10:13:29+00:00",
            "title": "Explain Yourself: A Natural Language Interface for Scrutable Autonomous Robots",
            "authors": [
                "Francisco J. Chiyah Garcia",
                "David A. Robb",
                "Xingkun Liu",
                "Atanas Laskov",
                "Pedro Patron",
                "Helen Hastie"
            ],
            "abstract": "Autonomous systems in remote locations have a high degree of autonomy and\nthere is a need to explain what they are doing and why in order to increase\ntransparency and maintain trust. Here, we describe a natural language chat\ninterface that enables vehicle behaviour to be queried by the user. We obtain\nan interpretable model of autonomy through having an expert 'speak out-loud'\nand provide explanations during a mission. This approach is agnostic to the\ntype of autonomy model and as expert and operator are from the same user-group,\nwe predict that these explanations will align well with the operator's mental\nmodel, increase transparency and assist with operator training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02238v2",
            "date": "2018-03-06 15:07:40+00:00",
            "title": "Precise but Natural Specification for Robot Tasks",
            "authors": [
                "Ivan Gavran",
                "Brendon Boldt",
                "Eva Darulova",
                "Rupak Majumdar"
            ],
            "abstract": "We present Flipper, a natural language interface for describing high-level\ntask specifications for robots that are compiled into robot actions. Flipper\nstarts with a formal core language for task planning that allows expressing\nrich temporal specifications and uses a semantic parser to provide a natural\nlanguage interface. Flipper provides immediate visual feedback by executing an\nautomatically constructed plan of the task in a graphical user interface. This\nallows the user to resolve potentially ambiguous interpretations. Flipper\nextends itself via naturalization: its users can add definitions for\nutterances, from which Flipper induces new rules and adds them to the core\nlanguage, gradually growing a more and more natural task specification\nlanguage. Flipper improves the naturalization by generalizing the definition\nprovided by users. Unlike other task-specification systems, Flipper enables\nnatural language interactions while maintaining the expressive power and formal\nprecision of a programming language. We show through an initial user study that\nnatural language interactions and generalization can considerably ease the\ndescription of tasks. Moreover, over time, users employ more and more concepts\noutside of the initial core language. Such extensions are available to the\nFlipper community, and users can use concepts that others have defined.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1803.02839v1",
            "date": "2018-03-07 19:06:08+00:00",
            "title": "The emergent algebraic structure of RNNs and embeddings in NLP",
            "authors": [
                "Sean A. Cantrell"
            ],
            "abstract": "We examine the algebraic and geometric properties of a uni-directional GRU\nand word embeddings trained end-to-end on a text classification task. A\nhyperparameter search over word embedding dimension, GRU hidden dimension, and\na linear combination of the GRU outputs is performed. We conclude that words\nnaturally embed themselves in a Lie group and that RNNs form a nonlinear\nrepresentation of the group. Appealing to these results, we propose a novel\nclass of recurrent-like neural networks and a word embedding scheme.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.02893v1",
            "date": "2018-03-07 22:02:10+00:00",
            "title": "An efficient framework for learning sentence representations",
            "authors": [
                "Lajanugen Logeswaran",
                "Honglak Lee"
            ],
            "abstract": "In this work we propose a simple and efficient framework for learning\nsentence representations from unlabelled data. Drawing inspiration from the\ndistributional hypothesis and recent work on learning sentence representations,\nwe reformulate the problem of predicting the context in which a sentence\nappears as a classification problem. Given a sentence and its context, a\nclassifier distinguishes context sentences from other contrastive sentences\nbased on their vector representations. This allows us to efficiently learn\ndifferent types of encoding functions, and we show that the model learns\nhigh-quality sentence representations. We demonstrate that our sentence\nrepresentations outperform state-of-the-art unsupervised and supervised\nrepresentation learning methods on several downstream NLP tasks that involve\nunderstanding sentence semantics while achieving an order of magnitude speedup\nin training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03018v1",
            "date": "2018-03-08 09:43:04+00:00",
            "title": "Cross-domain Recommendation via Deep Domain Adaptation",
            "authors": [
                "Heishiro Kanagawa",
                "Hayato Kobayashi",
                "Nobuyuki Shimizu",
                "Yukihiro Tagami",
                "Taiji Suzuki"
            ],
            "abstract": "The behavior of users in certain services could be a clue that can be used to\ninfer their preferences and may be used to make recommendations for other\nservices they have never used. However, the cross-domain relationships between\nitems and user consumption patterns are not simple, especially when there are\nfew or no common users and items across domains. To address this problem, we\npropose a content-based cross-domain recommendation method for cold-start users\nthat does not require user- and item- overlap. We formulate recommendation as\nextreme multi-class classification where labels (items) corresponding to the\nusers are predicted. With this formulation, the problem is reduced to a domain\nadaptation setting, in which a classifier trained in the source domain is\nadapted to the target domain. For this, we construct a neural network that\ncombines an architecture for domain adaptation, Domain Separation Network, with\na denoising autoencoder for item representation. We assess the performance of\nour approach in experiments on a pair of data sets collected from movie and\nnews services of Yahoo! JAPAN and show that our approach outperforms several\nbaseline methods including a cross-domain collaborative filtering method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.03232v1",
            "date": "2018-03-08 18:05:18+00:00",
            "title": "Feudal Reinforcement Learning for Dialogue Management in Large Domains",
            "authors": [
                "I\u00f1igo Casanueva",
                "Pawe\u0142 Budzianowski",
                "Pei-Hao Su",
                "Stefan Ultes",
                "Lina Rojas-Barahona",
                "Bo-Hsiang Tseng",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Reinforcement learning (RL) is a promising approach to solve dialogue policy\noptimisation. Traditional RL algorithms, however, fail to scale to large\ndomains due to the curse of dimensionality. We propose a novel Dialogue\nManagement architecture, based on Feudal RL, which decomposes the decision into\ntwo steps; a first step where a master policy selects a subset of primitive\nactions, and a second step where a primitive action is chosen from the selected\nsubset. The structural information included in the domain ontology is used to\nabstract the dialogue state space, taking the decisions at each step using\ndifferent parts of the abstracted state. This, combined with an information\nsharing mechanism between slots, increases the scalability to large domains. We\nshow that an implementation of this approach, based on Deep-Q Networks,\nsignificantly outperforms previous state of the art in several dialogue domains\nand environments, without the need of any additional reward signal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03370v1",
            "date": "2018-03-09 03:28:36+00:00",
            "title": "Expert Finding in Heterogeneous Bibliographic Networks with Locally-trained Embeddings",
            "authors": [
                "Huan Gui",
                "Qi Zhu",
                "Liyuan Liu",
                "Aston Zhang",
                "Jiawei Han"
            ],
            "abstract": "Expert finding is an important task in both industry and academia. It is\nchallenging to rank candidates with appropriate expertise for various queries.\nIn addition, different types of objects interact with one another, which\nnaturally forms heterogeneous information networks. We study the task of expert\nfinding in heterogeneous bibliographical networks based on two aspects: textual\ncontent analysis and authority ranking. Regarding the textual content analysis,\nwe propose a new method for query expansion via locally-trained embedding\nlearning with concept hierarchy as guidance, which is particularly tailored for\nspecific queries with narrow semantic meanings. Compared with global embedding\nlearning, locally-trained embedding learning projects the terms into a latent\nsemantic space constrained on relevant topics, therefore it preserves more\nprecise and subtle information for specific queries. Considering the candidate\nranking, the heterogeneous information network structure, while being largely\nignored in the previous studies of expert finding, provides additional\ninformation. Specifically, different types of interactions among objects play\ndifferent roles. We propose a ranking algorithm to estimate the authority of\nobjects in the network, treating each strongly-typed edge type individually. To\ndemonstrate the effectiveness of the proposed framework, we apply the proposed\nmethod to a large-scale bibliographical dataset with over two million entries\nand one million researcher candidates. The experiment results show that the\nproposed framework outperforms existing methods for both general and specific\nqueries.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.03376v1",
            "date": "2018-03-09 03:50:24+00:00",
            "title": "Learning Approximate Inference Networks for Structured Prediction",
            "authors": [
                "Lifu Tu",
                "Kevin Gimpel"
            ],
            "abstract": "Structured prediction energy networks (SPENs; Belanger & McCallum 2016) use\nneural network architectures to define energy functions that can capture\narbitrary dependencies among parts of structured outputs. Prior work used\ngradient descent for inference, relaxing the structured output to a set of\ncontinuous variables and then optimizing the energy with respect to them. We\nreplace this use of gradient descent with a neural network trained to\napproximate structured argmax inference. This \"inference network\" outputs\ncontinuous values that we treat as the output structure. We develop\nlarge-margin training criteria for joint training of the structured energy\nfunction and inference network. On multi-label classification we report\nspeed-ups of 10-60x compared to (Belanger et al, 2017) while also improving\naccuracy. For sequence labeling with simple structured energies, our approach\nperforms comparably to exact inference while being much faster at test time. We\nthen demonstrate improved accuracy by augmenting the energy with a \"label\nlanguage model\" that scores entire output label sequences, showing it can\nimprove handling of long-distance dependencies in part-of-speech tagging.\nFinally, we show how inference networks can replace dynamic programming for\ntest-time inference in conditional random fields, suggestive for their general\nuse for fast inference in structured settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03481v3",
            "date": "2018-03-09 12:06:04+00:00",
            "title": "Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping",
            "authors": [
                "Akira Taniguchi",
                "Yoshinobu Hagiwara",
                "Tadahiro Taniguchi",
                "Tetsunari Inamura"
            ],
            "abstract": "We propose a novel online learning algorithm, called SpCoSLAM 2.0, for\nspatial concepts and lexical acquisition with high accuracy and scalability.\nPreviously, we proposed SpCoSLAM as an online learning algorithm based on\nunsupervised Bayesian probabilistic model that integrates multimodal place\ncategorization, lexical acquisition, and SLAM. However, our original algorithm\nhad limited estimation accuracy owing to the influence of the early stages of\nlearning, and increased computational complexity with added training data.\nTherefore, we introduce techniques such as fixed-lag rejuvenation to reduce the\ncalculation time while maintaining an accuracy higher than that of the original\nalgorithm. The results show that, in terms of estimation accuracy, the proposed\nalgorithm exceeds the original algorithm and is comparable to batch learning.\nIn addition, the calculation time of the proposed algorithm does not depend on\nthe amount of training data and becomes constant for each step of the scalable\nalgorithm. Our approach will contribute to the realization of long-term spatial\nlanguage interactions between humans and robots.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1803.03667v1",
            "date": "2018-03-06 12:24:42+00:00",
            "title": "Co-occurrence of the Benford-like and Zipf Laws Arising from the Texts Representing Human and Artificial Languages",
            "authors": [
                "Evgeny Shulzinger",
                "Irina Legchenkova",
                "Edward Bormashenko"
            ],
            "abstract": "We demonstrate that large texts, representing human (English, Russian,\nUkrainian) and artificial (C++, Java) languages, display quantitative patterns\ncharacterized by the Benford-like and Zipf laws. The frequency of a word\nfollowing the Zipf law is inversely proportional to its rank, whereas the total\nnumbers of a certain word appearing in the text generate the uneven\nBenford-like distribution of leading numbers. Excluding the most popular words\nessentially improves the correlation of actual textual data with the Zipfian\ndistribution, whereas the Benford distribution of leading numbers (arising from\nthe overall amount of a certain word) is insensitive to the same elimination\nprocedure. The calculated values of the moduli of slopes of double\nlogarithmical plots for artificial languages (C++, Java) are markedly larger\nthan those for human ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.03697v1",
            "date": "2018-03-09 21:26:13+00:00",
            "title": "Community Interaction and Conflict on the Web",
            "authors": [
                "Srijan Kumar",
                "William L. Hamilton",
                "Jure Leskovec",
                "Dan Jurafsky"
            ],
            "abstract": "Users organize themselves into communities on web platforms. These\ncommunities can interact with one another, often leading to conflicts and toxic\ninteractions. However, little is known about the mechanisms of interactions\nbetween communities and how they impact users.\n  Here we study intercommunity interactions across 36,000 communities on\nReddit, examining cases where users of one community are mobilized by negative\nsentiment to comment in another community. We show that such conflicts tend to\nbe initiated by a handful of communities---less than 1% of communities start\n74% of conflicts. While conflicts tend to be initiated by highly active\ncommunity members, they are carried out by significantly less active members.\nWe find that conflicts are marked by formation of echo chambers, where users\nprimarily talk to other users from their own community. In the long-term,\nconflicts have adverse effects and reduce the overall activity of users in the\ntargeted communities.\n  Our analysis of user interactions also suggests strategies for mitigating the\nnegative impact of conflicts---such as increasing direct engagement between\nattackers and defenders. Further, we accurately predict whether a conflict will\noccur by creating a novel LSTM model that combines graph embeddings, user,\ncommunity, and text features. This model can be used toreate early-warning\nsystems for community moderators to prevent conflicts. Altogether, this work\npresents a data-driven view of community interactions and conflict, and paves\nthe way towards healthier online communities.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1803.03827v2",
            "date": "2018-03-10 15:52:08+00:00",
            "title": "Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions",
            "authors": [
                "Albert Gatt",
                "Marc Tanti",
                "Adrian Muscat",
                "Patrizia Paggio",
                "Reuben A. Farrugia",
                "Claudia Borg",
                "Kenneth P. Camilleri",
                "Mike Rosner",
                "Lonneke van der Plas"
            ],
            "abstract": "The past few years have witnessed renewed interest in NLP tasks at the\ninterface between vision and language. One intensively-studied problem is that\nof automatically generating text from images. In this paper, we extend this\nproblem to the more specific domain of face description. Unlike scene\ndescriptions, face descriptions are more fine-grained and rely on attributes\nextracted from the image, rather than objects and relations. Given that no data\nexists for this task, we present an ongoing crowdsourcing study to collect a\ncorpus of descriptions of face images taken `in the wild'. To gain a better\nunderstanding of the variation we find in face description and the possible\nissues that this may raise, we also conducted an annotation study on a subset\nof the corpus. Primarily, we found descriptions to refer to a mixture of\nattributes, not only physical, but also emotional and inferential, which is\nbound to create further challenges for current image-to-text methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04596v1",
            "date": "2018-03-13 02:09:06+00:00",
            "title": "Automatic Detection of Online Jihadist Hate Speech",
            "authors": [
                "Tom De Smedt",
                "Guy De Pauw",
                "Pieter Van Ostaeyen"
            ],
            "abstract": "We have developed a system that automatically detects online jihadist hate\nspeech with over 80% accuracy, by using techniques from Natural Language\nProcessing and Machine Learning. The system is trained on a corpus of 45,000\nsubversive Twitter messages collected from October 2014 to December 2016. We\npresent a qualitative and quantitative analysis of the jihadist rhetoric in the\ncorpus, examine the network of Twitter users, outline the technical procedure\nused to train the system, and discuss examples of use.",
            "journal ref": "CLiPS Technical Report Series 7 (2018) 1-31",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.04715v1",
            "date": "2018-03-13 10:30:55+00:00",
            "title": "Hierarchical Learning of Cross-Language Mappings through Distributed Vector Representations for Code",
            "authors": [
                "Nghi D. Q. Bui",
                "Lingxiao Jiang"
            ],
            "abstract": "Translating a program written in one programming language to another can be\nuseful for software development tasks that need functionality implementations\nin different languages. Although past studies have considered this problem,\nthey may be either specific to the language grammars, or specific to certain\nkinds of code elements (e.g., tokens, phrases, API uses). This paper proposes a\nnew approach to automatically learn cross-language representations for various\nkinds of structural code elements that may be used for program translation. Our\nkey idea is two folded: First, we normalize and enrich code token streams with\nadditional structural and semantic information, and train cross-language vector\nrepresentations for the tokens (a.k.a. shared embeddings based on word2vec, a\nneural-network-based technique for producing word embeddings; Second,\nhierarchically from bottom up, we construct shared embeddings for code elements\nof higher levels of granularity (e.g., expressions, statements, methods) from\nthe embeddings for their constituents, and then build mappings among code\nelements across languages based on similarities among embeddings.\n  Our preliminary evaluations on about 40,000 Java and C# source files from 9\nsoftware projects show that our approach can automatically learn shared\nembeddings for various code elements in different languages and identify their\ncross-language mappings with reasonable Mean Average Precision scores. When\ncompared with an existing tool for mapping library API methods, our approach\nidentifies many more mappings accurately. The mapping results and code can be\naccessed at\nhttps://github.com/bdqnghi/hierarchical-programming-language-mapping. We\nbelieve that our idea for learning cross-language vector representations with\ncode structural information can be a useful step towards automated program\ntranslation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.04884v1",
            "date": "2018-03-13 15:35:42+00:00",
            "title": "IDEL: In-Database Entity Linking with Neural Embeddings",
            "authors": [
                "Torsten Kilias",
                "Alexander L\u00f6ser",
                "Felix A. Gers",
                "Richard Koopmanschap",
                "Ying Zhang",
                "Martin Kersten"
            ],
            "abstract": "We present a novel architecture, In-Database Entity Linking (IDEL), in which\nwe integrate the analytics-optimized RDBMS MonetDB with neural text mining\nabilities. Our system design abstracts core tasks of most neural entity linking\nsystems for MonetDB. To the best of our knowledge, this is the first defacto\nimplemented system integrating entity-linking in a database. We leverage the\nability of MonetDB to support in-database-analytics with user defined functions\n(UDFs) implemented in Python. These functions call machine learning libraries\nfor neural text mining, such as TensorFlow. The system achieves zero cost for\ndata shipping and transformation by utilizing MonetDB's ability to embed Python\nprocesses in the database kernel and exchange data in NumPy arrays. IDEL\nrepresents text and relational data in a joint vector space with neural\nembeddings and can compensate errors with ambiguous entity representations. For\ndetecting matching entities, we propose a novel similarity function based on\njoint neural embeddings which are learned via minimizing pairwise contrastive\nranking loss. This function utilizes a high dimensional index structures for\nfast retrieval of matching entities. Our first implementation and experiments\nusing the WebNLG corpus show the effectiveness and the potentials of IDEL.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1803.05058v1",
            "date": "2018-03-13 21:40:59+00:00",
            "title": "Investigating the Effect of Music and Lyrics on Spoken-Word Recognition",
            "authors": [
                "Odette Scharenborg",
                "Martha Larson"
            ],
            "abstract": "Background music in social interaction settings can hinder conversation. Yet,\nlittle is known of how specific properties of music impact speech processing.\nThis paper addresses this knowledge gap by investigating 1) whether the masking\neffect of background music with lyrics is larger than that of music without\nlyrics, and 2) whether the masking effect is larger for more complex music. To\nanswer these questions, a word identification experiment was run in which Dutch\nparticipants listened to Dutch CVC words embedded in stretches of background\nmusic in two conditions, with and without lyrics, and at three SNRs. Three\nsongs were used of different genres and complexities. Music stretches with and\nwithout lyrics were sampled from the same song in order to control for factors\nbeyond the presence of lyrics. The results showed a clear negative impact of\nthe presence of lyrics in background music on spoken-word recognition. This\nimpact is independent of complexity. The results suggest that social spaces\n(e.g., restaurants, caf\\'es and bars) should make careful choices of music to\npromote conversation, and open a path for future work.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1803.05160v1",
            "date": "2018-03-14 08:16:48+00:00",
            "title": "How to evaluate sentiment classifiers for Twitter time-ordered data?",
            "authors": [
                "Igor Mozeti\u010d",
                "Luis Torgo",
                "Vitor Cerqueira",
                "Jasmina Smailovi\u0107"
            ],
            "abstract": "Social media are becoming an increasingly important source of information\nabout the public mood regarding issues such as elections, Brexit, stock market,\netc. In this paper we focus on sentiment classification of Twitter data.\nConstruction of sentiment classifiers is a standard text mining task, but here\nwe address the question of how to properly evaluate them as there is no settled\nway to do so. Sentiment classes are ordered and unbalanced, and Twitter\nproduces a stream of time-ordered data. The problem we address concerns the\nprocedures used to obtain reliable estimates of performance measures, and\nwhether the temporal ordering of the training and test data matters. We\ncollected a large set of 1.5 million tweets in 13 European languages. We\ncreated 138 sentiment models and out-of-sample datasets, which are used as a\ngold standard for evaluations. The corresponding 138 in-sample datasets are\nused to empirically compare six different estimation procedures: three variants\nof cross-validation, and three variants of sequential validation (where test\nset always follows the training set). We find no significant difference between\nthe best cross-validation and sequential validation. However, we observe that\nall cross-validation variants tend to overestimate the performance, while the\nsequential methods tend to underestimate it. Standard cross-validation with\nrandom selection of examples is significantly worse than the blocked\ncross-validation, and should not be used to evaluate classifiers in\ntime-ordered data scenarios.",
            "journal ref": "PLoS ONE 13(3): e0194317, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05457v1",
            "date": "2018-03-14 18:04:21+00:00",
            "title": "Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge",
            "authors": [
                "Peter Clark",
                "Isaac Cowhey",
                "Oren Etzioni",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Carissa Schoenick",
                "Oyvind Tafjord"
            ],
            "abstract": "We present a new question set, text corpus, and baselines assembled to\nencourage AI research in advanced question answering. Together, these\nconstitute the AI2 Reasoning Challenge (ARC), which requires far more powerful\nknowledge and reasoning than previous challenges such as SQuAD or SNLI. The ARC\nquestion set is partitioned into a Challenge Set and an Easy Set, where the\nChallenge Set contains only questions answered incorrectly by both a\nretrieval-based algorithm and a word co-occurence algorithm. The dataset\ncontains only natural, grade-school science questions (authored for human\ntests), and is the largest public-domain set of this kind (7,787 questions). We\ntest several baselines on the Challenge Set, including leading neural models\nfrom the SQuAD and SNLI tasks, and find that none are able to significantly\noutperform a random baseline, reflecting the difficult nature of this task. We\nare also releasing the ARC Corpus, a corpus of 14M science sentences relevant\nto the task, and implementations of the three neural baseline models tested.\nCan your model perform better? We pose ARC as a challenge to the community.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1803.06390v1",
            "date": "2018-03-16 20:19:56+00:00",
            "title": "Corpus Statistics in Text Classification of Online Data",
            "authors": [
                "Marina Sokolova",
                "Victoria Bobicev"
            ],
            "abstract": "Transformation of Machine Learning (ML) from a boutique science to a\ngenerally accepted technology has increased importance of reproduction and\ntransportability of ML studies. In the current work, we investigate how corpus\ncharacteristics of textual data sets correspond to text classification results.\nWe work with two data sets gathered from sub-forums of an online health-related\nforum. Our empirical results are obtained for a multi-class sentiment analysis\napplication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.06643v1",
            "date": "2018-03-18 11:28:12+00:00",
            "title": "The Web as a Knowledge-base for Answering Complex Questions",
            "authors": [
                "Alon Talmor",
                "Jonathan Berant"
            ],
            "abstract": "Answering complex questions is a time-consuming activity for humans that\nrequires reasoning and integration of information. Recent work on reading\ncomprehension made headway in answering simple questions, but tackling complex\nquestions is still an ongoing research challenge. Conversely, semantic parsers\nhave been successful at handling compositionality, but only when the\ninformation resides in a target knowledge-base. In this paper, we present a\nnovel framework for answering broad and complex questions, assuming answering\nsimple questions is possible using a search engine and a reading comprehension\nmodel. We propose to decompose complex questions into a sequence of simple\nquestions, and compute the final answer from the sequence of answers. To\nillustrate the viability of our approach, we create a new dataset of complex\nquestions, ComplexWebQuestions, and present a model that decomposes questions\nand interacts with the web to compute an answer. We empirically demonstrate\nthat question decomposition improves performance from 20.8 precision@1 to 27.5\nprecision@1 on this new dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07133v1",
            "date": "2018-03-15 07:54:30+00:00",
            "title": "Neural Text Generation: Past, Present and Beyond",
            "authors": [
                "Sidi Lu",
                "Yaoming Zhu",
                "Weinan Zhang",
                "Jun Wang",
                "Yong Yu"
            ],
            "abstract": "This paper presents a systematic survey on recent development of neural text\ngeneration models. Specifically, we start from recurrent neural network\nlanguage models with the traditional maximum likelihood estimation training\nscheme and point out its shortcoming for text generation. We thus introduce the\nrecently proposed methods for text generation based on reinforcement learning,\nre-parametrization tricks and generative adversarial nets (GAN) techniques. We\ncompare different properties of these models and the corresponding techniques\nto handle their common problems such as gradient vanishing and generation\ndiversity. Finally, we conduct a benchmarking experiment with different types\nof neural text generation models on two well-known datasets and discuss the\nempirical results along with the aforementioned model properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07416v1",
            "date": "2018-03-16 18:49:22+00:00",
            "title": "Tensor2Tensor for Neural Machine Translation",
            "authors": [
                "Ashish Vaswani",
                "Samy Bengio",
                "Eugene Brevdo",
                "Francois Chollet",
                "Aidan N. Gomez",
                "Stephan Gouws",
                "Llion Jones",
                "\u0141ukasz Kaiser",
                "Nal Kalchbrenner",
                "Niki Parmar",
                "Ryan Sepassi",
                "Noam Shazeer",
                "Jakob Uszkoreit"
            ],
            "abstract": "Tensor2Tensor is a library for deep learning models that is well-suited for\nneural machine translation and includes the reference implementation of the\nstate-of-the-art Transformer model.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.07427v2",
            "date": "2018-03-19 02:23:30+00:00",
            "title": "Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselines",
            "authors": [
                "Soujanya Poria",
                "Navonil Majumder",
                "Devamanyu Hazarika",
                "Erik Cambria",
                "Alexander Gelbukh",
                "Amir Hussain"
            ],
            "abstract": "We compile baselines, along with dataset split, for multimodal sentiment\nanalysis. In this paper, we explore three different deep-learning based\narchitectures for multimodal sentiment classification, each improving upon the\nprevious. Further, we evaluate these architectures with multiple datasets with\nfixed train/test partition. We also discuss some major issues, frequently\nignored in multimodal sentiment analysis research, e.g., role of\nspeaker-exclusive models, importance of different modalities, and\ngeneralizability. This framework illustrates the different facets of analysis\nto be considered while performing multimodal sentiment analysis and, hence,\nserves as a new benchmark for future research in this emerging field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07724v1",
            "date": "2018-03-21 03:05:58+00:00",
            "title": "Attention on Attention: Architectures for Visual Question Answering (VQA)",
            "authors": [
                "Jasdeep Singh",
                "Vincent Ying",
                "Alex Nutkiewicz"
            ],
            "abstract": "Visual Question Answering (VQA) is an increasingly popular topic in deep\nlearning research, requiring coordination of natural language processing and\ncomputer vision modules into a single architecture. We build upon the model\nwhich placed first in the VQA Challenge by developing thirteen new attention\nmechanisms and introducing a simplified classifier. We performed 300 GPU hours\nof extensive hyperparameter and architecture searches and were able to achieve\nan evaluation score of 64.78%, outperforming the existing state-of-the-art\nsingle model's validation score of 63.15%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.07729v2",
            "date": "2018-03-21 03:21:38+00:00",
            "title": "Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation",
            "authors": [
                "Xin Wang",
                "Wenhan Xiong",
                "Hongmin Wang",
                "William Yang Wang"
            ],
            "abstract": "Existing research studies on vision and language grounding for robot\nnavigation focus on improving model-free deep reinforcement learning (DRL)\nmodels in synthetic environments. However, model-free DRL models do not\nconsider the dynamics in the real-world environments, and they often fail to\ngeneralize to new scenes. In this paper, we take a radical approach to bridge\nthe gap between synthetic studies and real-world practices---We propose a\nnovel, planned-ahead hybrid reinforcement learning model that combines\nmodel-free and model-based reinforcement learning to solve a real-world\nvision-language navigation task. Our look-ahead module tightly integrates a\nlook-ahead policy model with an environment model that predicts the next state\nand the reward. Experimental results suggest that our proposed method\nsignificantly outperforms the baselines and achieves the best on the real-world\nRoom-to-Room dataset. Moreover, our scalable method is more generalizable when\ntransferring to unseen environments.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1803.07738v1",
            "date": "2018-03-21 03:52:26+00:00",
            "title": "Speech Emotion Recognition Considering Local Dynamic Features",
            "authors": [
                "Haotian Guan",
                "Zhilei Liu",
                "Longbiao Wang",
                "Jianwu Dang",
                "Ruiguo Yu"
            ],
            "abstract": "Recently, increasing attention has been directed to the study of the speech\nemotion recognition, in which global acoustic features of an utterance are\nmostly used to eliminate the content differences. However, the expression of\nspeech emotion is a dynamic process, which is reflected through dynamic\ndurations, energies, and some other prosodic information when one speaks. In\nthis paper, a novel local dynamic pitch probability distribution feature, which\nis obtained by drawing the histogram, is proposed to improve the accuracy of\nspeech emotion recognition. Compared with most of the previous works using\nglobal features, the proposed method takes advantage of the local dynamic\ninformation conveyed by the emotional speech. Several experiments on Berlin\nDatabase of Emotional Speech are conducted to verify the effectiveness of the\nproposed method. The experimental results demonstrate that the local dynamic\ninformation obtained with the proposed method is more effective for speech\nemotion recognition than the traditional global features.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1803.08240v1",
            "date": "2018-03-22 06:25:47+00:00",
            "title": "An Analysis of Neural Language Modeling at Multiple Scales",
            "authors": [
                "Stephen Merity",
                "Nitish Shirish Keskar",
                "Richard Socher"
            ],
            "abstract": "Many of the leading approaches in language modeling introduce novel, complex\nand specialized architectures. We take existing state-of-the-art word level\nlanguage models based on LSTMs and QRNNs and extend them to both larger\nvocabularies as well as character-level granularity. When properly tuned, LSTMs\nand QRNNs achieve state-of-the-art results on character-level (Penn Treebank,\nenwik8) and word-level (WikiText-103) datasets, respectively. Results are\nobtained in only 12 hours (WikiText-103) to 2 days (enwik8) using a single\nmodern GPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08312v3",
            "date": "2018-03-22 11:38:53+00:00",
            "title": "Learning Eligibility in Cancer Clinical Trials using Deep Neural Networks",
            "authors": [
                "Aurelia Bustos",
                "Antonio Pertusa"
            ],
            "abstract": "Interventional cancer clinical trials are generally too restrictive, and some\npatients are often excluded on the basis of comorbidity, past or concomitant\ntreatments, or the fact that they are over a certain age. The efficacy and\nsafety of new treatments for patients with these characteristics are,\ntherefore, not defined. In this work, we built a model to automatically predict\nwhether short clinical statements were considered inclusion or exclusion\ncriteria. We used protocols from cancer clinical trials that were available in\npublic registries from the last 18 years to train word-embeddings, and we\nconstructed a~dataset of 6M short free-texts labeled as eligible or not\neligible. A text classifier was trained using deep neural networks, with\npre-trained word-embeddings as inputs, to predict whether or not short\nfree-text statements describing clinical information were considered eligible.\nWe additionally analyzed the semantic reasoning of the word-embedding\nrepresentations obtained and were able to identify equivalent treatments for a\ntype of tumor analogous with the drugs used to treat other tumors. We show that\nrepresentation learning using {deep} neural networks can be successfully\nleveraged to extract the medical knowledge from clinical trial protocols for\npotentially assisting practitioners when prescribing treatments.",
            "journal ref": "Applied Sciences, 8(7), 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08790v1",
            "date": "2018-03-22 15:30:03+00:00",
            "title": "Sentiment Analysis of Comments on Rohingya Movement with Support Vector Machine",
            "authors": [
                "Hemayet Ahmed Chowdhury",
                "Tanvir Alam Nibir",
                "Md. Saiful Islam"
            ],
            "abstract": "The Rohingya Movement and Crisis caused a huge uproar in the political and\neconomic state of Bangladesh. Refugee movement is a recurring event and a large\namount of data in the form of opinions remains on social media such as\nFacebook, with very little analysis done on them.To analyse the comments based\non all Rohingya related posts, we had to create and modify a classifier based\non the Support Vector Machine algorithm. The code is implemented in python and\nuses scikit-learn library. A dataset on Rohingya analysis is not currently\navailable so we had to use our own data set of 2500 positive and 2500 negative\ncomments. We specifically used a support vector machine with linear kernel. A\nprevious experiment was performed by us on the same dataset using the naive\nbayes algorithm, but that did not yield impressive results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.08793v1",
            "date": "2018-03-21 16:14:22+00:00",
            "title": "Exploring the Naturalness of Buggy Code with Recurrent Neural Networks",
            "authors": [
                "Jack Lanchantin",
                "Ji Gao"
            ],
            "abstract": "Statistical language models are powerful tools which have been used for many\ntasks within natural language processing. Recently, they have been used for\nother sequential data such as source code.(Ray et al., 2015) showed that it is\npossible train an n-gram source code language mode, and use it to predict buggy\nlines in code by determining \"unnatural\" lines via entropy with respect to the\nlanguage model. In this work, we propose using a more advanced language\nmodeling technique, Long Short-term Memory recurrent neural networks, to model\nsource code and classify buggy lines based on entropy. We show that our method\nslightly outperforms an n-gram model in the buggy line classification task\nusing AUC.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1803.08869v2",
            "date": "2018-03-23 16:30:06+00:00",
            "title": "On the difficulty of a distributional semantics of spoken language",
            "authors": [
                "Grzegorz Chrupa\u0142a",
                "Lieke Gelderloos",
                "\u00c1kos K\u00e1d\u00e1r",
                "Afra Alishahi"
            ],
            "abstract": "In the domain of unsupervised learning most work on speech has focused on\ndiscovering low-level constructs such as phoneme inventories or word-like\nunits. In contrast, for written language, where there is a large body of work\non unsupervised induction of semantic representations of words, whole sentences\nand longer texts. In this study we examine the challenges of adapting these\napproaches from written to spoken language. We conjecture that unsupervised\nlearning of the semantics of spoken language becomes feasible if we abstract\nfrom the surface variability. We simulate this setting with a dataset of\nutterances spoken by a realistic but uniform synthetic voice. We evaluate two\nsimple unsupervised models which, to varying degrees of success, learn semantic\nrepresentations of speech fragments. Finally we present inconclusive results on\nhuman speech, and discuss the challenges inherent in learning distributional\nsemantic representations on unrestricted natural spoken language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.08966v1",
            "date": "2018-03-23 20:14:51+00:00",
            "title": "Counterexamples for Robotic Planning Explained in Structured Language",
            "authors": [
                "Lu Feng",
                "Mahsa Ghasemi",
                "Kai-Wei Chang",
                "Ufuk Topcu"
            ],
            "abstract": "Automated techniques such as model checking have been used to verify models\nof robotic mission plans based on Markov decision processes (MDPs) and generate\ncounterexamples that may help diagnose requirement violations. However, such\nartifacts may be too complex for humans to understand, because existing\nrepresentations of counterexamples typically include a large number of paths or\na complex automaton. To help improve the interpretability of counterexamples,\nwe define a notion of explainable counterexample, which includes a set of\nstructured natural language sentences to describe the robotic behavior that\nlead to a requirement violation in an MDP model of robotic mission plan. We\npropose an approach based on mixed-integer linear programming for generating\nexplainable counterexamples that are minimal, sound and complete. We\ndemonstrate the usefulness of the proposed approach via a case study of\nwarehouse robots planning.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1803.09017v1",
            "date": "2018-03-23 23:56:49+00:00",
            "title": "Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis",
            "authors": [
                "Yuxuan Wang",
                "Daisy Stanton",
                "Yu Zhang",
                "RJ Skerry-Ryan",
                "Eric Battenberg",
                "Joel Shor",
                "Ying Xiao",
                "Fei Ren",
                "Ye Jia",
                "Rif A. Saurous"
            ],
            "abstract": "In this work, we propose \"global style tokens\" (GSTs), a bank of embeddings\nthat are jointly trained within Tacotron, a state-of-the-art end-to-end speech\nsynthesis system. The embeddings are trained with no explicit labels, yet learn\nto model a large range of acoustic expressiveness. GSTs lead to a rich set of\nsignificant results. The soft interpretable \"labels\" they generate can be used\nto control synthesis in novel ways, such as varying speed and speaking style -\nindependently of the text content. They can also be used for style transfer,\nreplicating the speaking style of a single audio clip across an entire\nlong-form text corpus. When trained on noisy, unlabeled found data, GSTs learn\nto factorize noise and speaker identity, providing a path towards highly\nscalable but robust speech synthesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09047v1",
            "date": "2018-03-24 02:52:58+00:00",
            "title": "Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron",
            "authors": [
                "RJ Skerry-Ryan",
                "Eric Battenberg",
                "Ying Xiao",
                "Yuxuan Wang",
                "Daisy Stanton",
                "Joel Shor",
                "Ron J. Weiss",
                "Rob Clark",
                "Rif A. Saurous"
            ],
            "abstract": "We present an extension to the Tacotron speech synthesis architecture that\nlearns a latent embedding space of prosody, derived from a reference acoustic\nrepresentation containing the desired prosody. We show that conditioning\nTacotron on this learned embedding space results in synthesized audio that\nmatches the prosody of the reference signal with fine time detail even when the\nreference and synthesis speakers are different. Additionally, we show that a\nreference prosody embedding can be used to synthesize text that is different\nfrom that of the reference utterance. We define several quantitative and\nsubjective metrics for evaluating prosody transfer, and report results with\naccompanying audio samples from single-speaker and 44-speaker Tacotron models\non a prosody transfer task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09074v1",
            "date": "2018-03-24 08:10:04+00:00",
            "title": "Multi-range Reasoning for Machine Comprehension",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "We propose MRU (Multi-Range Reasoning Units), a new fast compositional\nencoder for machine comprehension (MC). Our proposed MRU encoders are\ncharacterized by multi-ranged gating, executing a series of parameterized\ncontract-and-expand layers for learning gating vectors that benefit from long\nand short-term dependencies. The aims of our approach are as follows: (1)\nlearning representations that are concurrently aware of long and short-term\ncontext, (2) modeling relationships between intra-document blocks and (3) fast\nand efficient sequence encoding. We show that our proposed encoder demonstrates\npromising results both as a standalone encoder and as well as a complementary\nbuilding block. We conduct extensive experiments on three challenging MC\ndatasets, namely RACE, SearchQA and NarrativeQA, achieving highly competitive\nperformance on all. On the RACE benchmark, our model outperforms DFN (Dynamic\nFusion Networks) by 1.5%-6% without using any recurrent or convolution layers.\nSimilarly, we achieve competitive performance relative to AMANDA on the\nSearchQA benchmark and BiDAF on the NarrativeQA benchmark without using any\nLSTM/GRU layers. Finally, incorporating MRU encoders with standard BiLSTM\narchitectures further improves performance, achieving state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.09123v1",
            "date": "2018-03-24 15:04:17+00:00",
            "title": "Equation Embeddings",
            "authors": [
                "Kriste Krstovski",
                "David M. Blei"
            ],
            "abstract": "We present an unsupervised approach for discovering semantic representations\nof mathematical equations. Equations are challenging to analyze because each is\nunique, or nearly unique. Our method, which we call equation embeddings, finds\ngood representations of equations by using the representations of their\nsurrounding words. We used equation embeddings to analyze four collections of\nscientific articles from the arXiv, covering four computer science domains\n(NLP, IR, AI, and ML) and $\\sim$98.5k equations. Quantitatively, we found that\nequation embeddings provide better models when compared to existing word\nembedding approaches. Qualitatively, we found that equation embeddings provide\ncoherent semantic representations of equations and can capture semantic\nsimilarity to other equations and to words.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1803.09551v1",
            "date": "2018-03-26 12:46:13+00:00",
            "title": "Collaborative Filtering with Topic and Social Latent Factors Incorporating Implicit Feedback",
            "authors": [
                "Guang-Neng Hu",
                "Xin-Yu Dai",
                "Feng-Yu Qiu",
                "Rui Xia",
                "Tao Li",
                "Shu-Jian Huang",
                "Jia-Jun Chen"
            ],
            "abstract": "Recommender systems (RSs) provide an effective way of alleviating the\ninformation overload problem by selecting personalized items for different\nusers. Latent factors based collaborative filtering (CF) has become the popular\napproaches for RSs due to its accuracy and scalability. Recently, online social\nnetworks and user-generated content provide diverse sources for recommendation\nbeyond ratings. Although {\\em social matrix factorization} (Social MF) and {\\em\ntopic matrix factorization} (Topic MF) successfully exploit social relations\nand item reviews, respectively, both of them ignore some useful information. In\nthis paper, we investigate the effective data fusion by combining the\naforementioned approaches. First, we propose a novel model {\\em \\mbox{MR3}} to\njointly model three sources of information (i.e., ratings, item reviews, and\nsocial relations) effectively for rating prediction by aligning the latent\nfactors and hidden topics. Second, we incorporate the implicit feedback from\nratings into the proposed model to enhance its capability and to demonstrate\nits flexibility. We achieve more accurate rating prediction on real-life\ndatasets over various state-of-the-art methods. Furthermore, we measure the\ncontribution from each of the three data sources and the impact of implicit\nfeedback from ratings, followed by the sensitivity analysis of hyperparameters.\nEmpirical studies demonstrate the effectiveness and efficacy of our proposed\nmodel and its extension.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1803.09578v1",
            "date": "2018-03-26 13:35:14+00:00",
            "title": "Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "Developing state-of-the-art approaches for specific tasks is a major driving\nforce in our research community. Depending on the prestige of the task,\npublishing it can come along with a lot of visibility. The question arises how\nreliable are our evaluation methodologies to compare approaches?\n  One common methodology to identify the state-of-the-art is to partition data\ninto a train, a development and a test set. Researchers can train and tune\ntheir approach on some part of the dataset and then select the model that\nworked best on the development set for a final evaluation on unseen test data.\nTest scores from different approaches are compared, and performance differences\nare tested for statistical significance.\n  In this publication, we show that there is a high risk that a statistical\nsignificance in this type of evaluation is not due to a superior learning\napproach. Instead, there is a high risk that the difference is due to chance.\nFor example for the CoNLL 2003 NER dataset we observed in up to 26% of the\ncases type I errors (false positives) with a threshold of p < 0.05, i.e.,\nfalsely concluding a statistically significant difference between two identical\napproaches.\n  We prove that this evaluation setup is unsuitable to compare learning\napproaches. We formalize alternative evaluation setups based on score\ndistributions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1803.09816v1",
            "date": "2018-03-26 19:56:21+00:00",
            "title": "Spectral feature mapping with mimic loss for robust speech recognition",
            "authors": [
                "Deblin Bagchi",
                "Peter Plantinga",
                "Adam Stiff",
                "Eric Fosler-Lussier"
            ],
            "abstract": "For the task of speech enhancement, local learning objectives are agnostic to\nphonetic structures helpful for speech recognition. We propose to add a global\ncriterion to ensure de-noised speech is useful for downstream tasks like ASR.\nWe first train a spectral classifier on clean speech to predict senone labels.\nThen, the spectral classifier is joined with our speech enhancer as a noisy\nspeech recognizer. This model is taught to imitate the output of the spectral\nclassifier alone on clean speech. This \\textit{mimic loss} is combined with the\ntraditional local criterion to train the speech enhancer to produce de-noised\nspeech. Feeding the de-noised speech to an off-the-shelf Kaldi training recipe\nfor the CHiME-2 corpus shows significant improvements in WER.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1803.10132v3",
            "date": "2018-03-27 15:23:12+00:00",
            "title": "Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition",
            "authors": [
                "Ke Wang",
                "Junbo Zhang",
                "Sining Sun",
                "Yujun Wang",
                "Fei Xiang",
                "Lei Xie"
            ],
            "abstract": "We investigate the use of generative adversarial networks (GANs) in speech\ndereverberation for robust speech recognition. GANs have been recently studied\nfor speech enhancement to remove additive noises, but there still lacks of a\nwork to examine their ability in speech dereverberation and the advantages of\nusing GANs have not been fully established. In this paper, we provide deep\ninvestigations in the use of GAN-based dereverberation front-end in ASR. First,\nwe study the effectiveness of different dereverberation networks (the generator\nin GAN) and find that LSTM leads a significant improvement as compared with\nfeed-forward DNN and CNN in our dataset. Second, further adding residual\nconnections in the deep LSTMs can boost the performance as well. Finally, we\nfind that, for the success of GAN, it is important to update the generator and\nthe discriminator using the same mini-batch data during training. Moreover,\nusing reverberant spectrogram as a condition to discriminator, as suggested in\nprevious studies, may degrade the performance. In summary, our GAN-based\ndereverberation front-end achieves 14%-19% relative CER reduction as compared\nto the baseline DNN dereverberation network when tested on a strong\nmulti-condition training acoustic model.",
            "journal ref": "Proceedings of Interspeech, 2018, pp. 1581-1585",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1803.10146v3",
            "date": "2018-03-27 15:39:46+00:00",
            "title": "Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model",
            "authors": [
                "Ke Wang",
                "Junbo Zhang",
                "Yujun Wang",
                "Lei Xie"
            ],
            "abstract": "Speaker adaptation aims to estimate a speaker specific acoustic model from a\nspeaker independent one to minimize the mismatch between the training and\ntesting conditions arisen from speaker variabilities. A variety of neural\nnetwork adaptation methods have been proposed since deep learning models have\nbecome the main stream. But there still lacks an experimental comparison\nbetween different methods, especially when DNN-based acoustic models have been\nadvanced greatly. In this paper, we aim to close this gap by providing an\nempirical evaluation of three typical speaker adaptation methods: LIN, LHUC and\nKLD. Adaptation experiments, with different size of adaptation data, are\nconducted on a strong TDNN-LSTM acoustic model. More challengingly, here, the\nsource and target we are concerned with are standard Mandarin speaker model and\naccented Mandarin speaker model. We compare the performances of different\nmethods and their combinations. Speaker adaptation performance is also examined\nby speaker's accent degree.",
            "journal ref": "Proceedings of Interspeech, 2018, pp. 2429-2433",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1803.10299v3",
            "date": "2018-03-27 20:12:39+00:00",
            "title": "Multi-Modal Data Augmentation for End-to-End ASR",
            "authors": [
                "Adithya Renduchintala",
                "Shuoyang Ding",
                "Matthew Wiesner",
                "Shinji Watanabe"
            ],
            "abstract": "We present a new end-to-end architecture for automatic speech recognition\n(ASR) that can be trained using \\emph{symbolic} input in addition to the\ntraditional acoustic input. This architecture utilizes two separate encoders:\none for acoustic input and another for symbolic input, both sharing the\nattention and decoder parameters. We call this architecture a multi-modal data\naugmentation network (MMDA), as it can support multi-modal (acoustic and\nsymbolic) input and enables seamless mixing of large text datasets with\nsignificantly smaller transcribed speech corpora during training. We study\ndifferent ways of transforming large text corpora into a symbolic form suitable\nfor training our MMDA network. Our best MMDA setup obtains small improvements\non character error rate (CER), and as much as 7-10\\% relative word error rate\n(WER) improvement over a baseline both with and without an external language\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10525v1",
            "date": "2018-03-28 11:06:15+00:00",
            "title": "Machine Speech Chain with One-shot Speaker Adaptation",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "In previous work, we developed a closed-loop speech chain model based on deep\nlearning, in which the architecture enabled the automatic speech recognition\n(ASR) and text-to-speech synthesis (TTS) components to mutually improve their\nperformance. This was accomplished by the two parts teaching each other using\nboth labeled and unlabeled data. This approach could significantly improve\nmodel performance within a single-speaker speech dataset, but only a slight\nincrease could be gained in multi-speaker tasks. Furthermore, the model is\nstill unable to handle unseen speakers. In this paper, we present a new speech\nchain mechanism by integrating a speaker recognition model inside the loop. We\nalso propose extending the capability of TTS to handle unseen speakers by\nimplementing one-shot speaker adaptation. This enables TTS to mimic voice\ncharacteristics from one speaker to another with only a one-shot speaker\nsample, even from a text without any speaker information. In the speech chain\nloop mechanism, ASR also benefits from the ability to further learn an\narbitrary speaker's characteristics from the generated speech waveform,\nresulting in a significant improvement in the recognition rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.10916v1",
            "date": "2018-03-29 03:32:59+00:00",
            "title": "Attention-based End-to-End Models for Small-Footprint Keyword Spotting",
            "authors": [
                "Changhao Shan",
                "Junbo Zhang",
                "Yujun Wang",
                "Lei Xie"
            ],
            "abstract": "In this paper, we propose an attention-based end-to-end neural approach for\nsmall-footprint keyword spotting (KWS), which aims to simplify the pipelines of\nbuilding a production-quality KWS system. Our model consists of an encoder and\nan attention mechanism. The encoder transforms the input signal into a high\nlevel representation using RNNs. Then the attention mechanism weights the\nencoder features and generates a fixed-length vector. Finally, by linear\ntransformation and softmax function, the vector becomes a score used for\nkeyword detection. We also evaluate the performance of different encoder\narchitectures, including LSTM, GRU and CRNN. Experiments on real-world wake-up\ndata show that our approach outperforms the recent Deep KWS approach by a large\nmargin and the best performance is achieved by CRNN. To be more specific, with\n~84K parameters, our attention-based model achieves 1.02% false rejection rate\n(FRR) at 1.0 false alarm (FA) per hour.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1805.00063v3",
            "date": "2018-04-30 19:10:43+00:00",
            "title": "Adversarial Semantic Alignment for Improved Image Captions",
            "authors": [
                "Pierre L. Dognin",
                "Igor Melnyk",
                "Youssef Mroueh",
                "Jarret Ross",
                "Tom Sercu"
            ],
            "abstract": "In this paper we study image captioning as a conditional GAN training,\nproposing both a context-aware LSTM captioner and co-attentive discriminator,\nwhich enforces semantic alignment between images and captions. We empirically\nfocus on the viability of two training methods: Self-critical Sequence Training\n(SCST) and Gumbel Straight-Through (ST) and demonstrate that SCST shows more\nstable gradient behavior and improved results over Gumbel ST, even without\naccessing discriminator gradients directly. We also address the problem of\nautomatic evaluation for captioning models and introduce a new semantic score,\nand show its correlation to human judgement. As an evaluation paradigm, we\nargue that an important criterion for a captioner is the ability to generalize\nto compositions of objects that do not usually co-occur together. To this end,\nwe introduce a small captioned Out of Context (OOC) test set. The OOC set,\ncombined with our semantic score, are the proposed new diagnosis tools for the\ncaptioning community. When evaluated on OOC and MS-COCO benchmarks, we show\nthat SCST-based training has a strong performance in both semantic score and\nhuman evaluation, promising to be a valuable new approach for efficient\ndiscrete GAN training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.00150v1",
            "date": "2018-05-01 02:14:00+00:00",
            "title": "Memory-augmented Dialogue Management for Task-oriented Dialogue Systems",
            "authors": [
                "Zheng Zhang",
                "Minlie Huang",
                "Zhongzhou Zhao",
                "Feng Ji",
                "Haiqing Chen",
                "Xiaoyan Zhu"
            ],
            "abstract": "Dialogue management (DM) decides the next action of a dialogue system\naccording to the current dialogue state, and thus plays a central role in\ntask-oriented dialogue systems. Since dialogue management requires to have\naccess to not only local utterances, but also the global semantics of the\nentire dialogue session, modeling the long-range history information is a\ncritical issue. To this end, we propose a novel Memory-Augmented Dialogue\nmanagement model (MAD) which employs a memory controller and two additional\nmemory structures, i.e., a slot-value memory and an external memory. The\nslot-value memory tracks the dialogue state by memorizing and updating the\nvalues of semantic slots (for instance, cuisine, price, and location), and the\nexternal memory augments the representation of hidden states of traditional\nrecurrent neural networks through storing more context information. To update\nthe dialogue state efficiently, we also propose slot-level attention on user\nutterances to extract specific semantic information for each slot. Experiments\nshow that our model can obtain state-of-the-art performance and outperforms\nexisting baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00314v1",
            "date": "2018-04-23 14:51:46+00:00",
            "title": "Object Counts! Bringing Explicit Detections Back into Image Captioning",
            "authors": [
                "Josiah Wang",
                "Pranava Madhyastha",
                "Lucia Specia"
            ],
            "abstract": "The use of explicit object detectors as an intermediate step to image\ncaptioning - which used to constitute an essential stage in early work - is\noften bypassed in the currently dominant end-to-end approaches, where the\nlanguage model is conditioned directly on a mid-level image embedding. We argue\nthat explicit detections provide rich semantic information, and can thus be\nused as an interpretable representation to better understand why end-to-end\nimage captioning systems work well. We provide an in-depth analysis of\nend-to-end image captioning by exploring a variety of cues that can be derived\nfrom such object detections. Our study reveals that end-to-end image captioning\nsystems rely on matching image representations to generate captions, and that\nencoding the frequency, size and position of objects are complementary and all\nplay a role in forming a good image representation. It also reveals that\ndifferent object categories contribute in different ways towards image\ncaptioning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.00460v1",
            "date": "2018-04-27 11:27:45+00:00",
            "title": "Customized Image Narrative Generation via Interactive Visual Question Generation and Answering",
            "authors": [
                "Andrew Shin",
                "Yoshitaka Ushiku",
                "Tatsuya Harada"
            ],
            "abstract": "Image description task has been invariably examined in a static manner with\nqualitative presumptions held to be universally applicable, regardless of the\nscope or target of the description. In practice, however, different viewers may\npay attention to different aspects of the image, and yield different\ndescriptions or interpretations under various contexts. Such diversity in\nperspectives is difficult to derive with conventional image description\ntechniques. In this paper, we propose a customized image narrative generation\ntask, in which the users are interactively engaged in the generation process by\nproviding answers to the questions. We further attempt to learn the user's\ninterest via repeating such interactive stages, and to automatically reflect\nthe interest in descriptions for new images. Experimental results demonstrate\nthat our model can generate a variety of descriptions from single image that\ncover a wider range of topics than conventional models, while being\ncustomizable to the target user of interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00579v1",
            "date": "2018-05-02 00:06:53+00:00",
            "title": "Convolutional-Recurrent Neural Networks for Speech Enhancement",
            "authors": [
                "Han Zhao",
                "Shuayb Zarar",
                "Ivan Tashev",
                "Chin-Hui Lee"
            ],
            "abstract": "We propose an end-to-end model based on convolutional and recurrent neural\nnetworks for speech enhancement. Our model is purely data-driven and does not\nmake any assumptions about the type or the stationarity of the noise. In\ncontrast to existing methods that use multilayer perceptrons (MLPs), we employ\nboth convolutional and recurrent neural network architectures. Thus, our\napproach allows us to exploit local structures in both the frequency and\ntemporal domains. By incorporating prior knowledge of speech signals into the\ndesign of model structures, we build a model that is more data-efficient and\nachieves better generalization on both seen and unseen noise. Based on\nexperiments with synthetic data, we demonstrate that our model outperforms\nexisting methods, improving PESQ by up to 0.6 on seen noise and 0.64 on unseen\nnoise.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1805.00604v3",
            "date": "2018-05-02 02:30:20+00:00",
            "title": "Text-Independent Speaker Verification Using Long Short-Term Memory Networks",
            "authors": [
                "Aryan Mobiny",
                "Mohammad Najarian"
            ],
            "abstract": "In this paper, an architecture based on Long Short-Term Memory Networks has\nbeen proposed for the text-independent scenario which is aimed to capture the\ntemporal speaker-related information by operating over traditional speech\nfeatures. For speaker verification, at first, a background model must be\ncreated for speaker representation. Then, in enrollment stage, the speaker\nmodels will be created based on the enrollment utterances. For this work, the\nmodel will be trained in an end-to-end fashion to combine the first two stages.\nThe main goal of end-to-end training is the model being optimized to be\nconsistent with the speaker verification protocol. The end- to-end training\njointly learns the background and speaker models by creating the representation\nspace. The LSTM architecture is trained to create a discrimination space for\nvalidating the match and non-match pairs for speaker verification. The proposed\narchitecture demonstrate its superiority in the text-independent compared to\nother traditional methods.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.00625v2",
            "date": "2018-05-02 05:05:32+00:00",
            "title": "Multimodal Utterance-level Affect Analysis using Visual, Audio and Text Features",
            "authors": [
                "Didan Deng",
                "Yuqian Zhou",
                "Jimin Pi",
                "Bertram E. Shi"
            ],
            "abstract": "The integration of information across multiple modalities and across time is\na promising way to enhance the emotion recognition performance of affective\nsystems. Much previous work has focused on instantaneous emotion recognition.\nThe 2018 One-Minute Gradual-Emotion Recognition (OMG-Emotion) challenge, which\nwas held in conjunction with the IEEE World Congress on Computational\nIntelligence, encouraged participants to address long-term emotion recognition\nby integrating cues from multiple modalities, including facial expression,\naudio and language. Intuitively, a multi-modal inference network should be able\nto leverage information from each modality and their correlations to improve\nrecognition over that achievable by a single modality network. We describe here\na multi-modal neural architecture that integrates visual information over time\nusing an LSTM, and combines it with utterance level audio and text cues to\nrecognize human sentiment from multimodal clips. Our model outperforms the\nunimodal baseline, achieving the concordance correlation coefficients (CCC) of\n0.400 on the arousal task, and 0.353 on the valence task.",
            "journal ref": "None",
            "category": "eess.IV"
        },
        {
            "url": "http://arxiv.org/abs/1805.00900v1",
            "date": "2018-05-02 16:34:01+00:00",
            "title": "Images & Recipes: Retrieval in the cooking context",
            "authors": [
                "Micael Carvalho",
                "R\u00e9mi Cad\u00e8ne",
                "David Picard",
                "Laure Soulier",
                "Matthieu Cord"
            ],
            "abstract": "Recent advances in the machine learning community allowed different use cases\nto emerge, as its association to domains like cooking which created the\ncomputational cuisine. In this paper, we tackle the picture-recipe alignment\nproblem, having as target application the large-scale retrieval task (finding a\nrecipe given a picture, and vice versa). Our approach is validated on the\nRecipe1M dataset, composed of one million image-recipe pairs and additional\nclass information, for which we achieve state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.01060v1",
            "date": "2018-05-03 00:10:10+00:00",
            "title": "Multimodal Emotion Recognition for One-Minute-Gradual Emotion Challenge",
            "authors": [
                "Ziqi Zheng",
                "Chenjie Cao",
                "Xingwei Chen",
                "Guoqiang Xu"
            ],
            "abstract": "The continuous dimensional emotion modelled by arousal and valence can depict\ncomplex changes of emotions. In this paper, we present our works on arousal and\nvalence predictions for One-Minute-Gradual (OMG) Emotion Challenge. Multimodal\nrepresentations are first extracted from videos using a variety of acoustic,\nvideo and textual models and support vector machine (SVM) is then used for\nfusion of multimodal signals to make final predictions. Our solution achieves\nConcordant Correlation Coefficient (CCC) scores of 0.397 and 0.520 on arousal\nand valence respectively for the validation dataset, which outperforms the\nbaseline systems with the best CCC scores of 0.15 and 0.23 on arousal and\nvalence by a large margin.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.01216v3",
            "date": "2018-05-03 10:52:26+00:00",
            "title": "Disentangling Language and Knowledge in Task-Oriented Dialogs",
            "authors": [
                "Dinesh Raghu",
                "Nikhil Gupta",
                "Mausam"
            ],
            "abstract": "The Knowledge Base (KB) used for real-world applications, such as booking a\nmovie or restaurant reservation, keeps changing over time. End-to-end neural\nnetworks trained for these task-oriented dialogs are expected to be immune to\nany changes in the KB. However, existing approaches breakdown when asked to\nhandle such changes. We propose an encoder-decoder architecture (BoSsNet) with\na novel Bag-of-Sequences (BoSs) memory, which facilitates the disentangled\nlearning of the response's language model and its knowledge incorporation.\nConsequently, the KB can be modified with new knowledge without a drop in\ninterpretability. We find that BoSsNet outperforms state-of-the-art models,\nwith considerable improvements (> 10\\%) on bAbI OOV test sets and other\nhuman-human datasets. We also systematically modify existing datasets to\nmeasure disentanglement and show BoSsNet to be robust to KB modifications.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.01252v2",
            "date": "2018-05-03 12:24:39+00:00",
            "title": "Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback",
            "authors": [
                "Carolin Lawrence",
                "Stefan Riezler"
            ],
            "abstract": "Counterfactual learning from human bandit feedback describes a scenario where\nuser feedback on the quality of outputs of a historic system is logged and used\nto improve a target system. We show how to apply this learning framework to\nneural semantic parsing. From a machine learning perspective, the key challenge\nlies in a proper reweighting of the estimator so as to avoid known degeneracies\nin counterfactual learning, while still being applicable to stochastic gradient\noptimization. To conduct experiments with human users, we devise an easy-to-use\ninterface to collect human feedback on semantic parses. Our work is the first\nto show that semantic parsers can be improved significantly by counterfactual\nlearning from logged human feedback data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01369v1",
            "date": "2018-05-03 15:21:44+00:00",
            "title": "Framewise approach in multimodal emotion recognition in OMG challenge",
            "authors": [
                "Grigoriy Sterling",
                "Andrey Belyaev",
                "Maxim Ryabov"
            ],
            "abstract": "In this report we described our approach achieves $53\\%$ of unweighted\naccuracy over $7$ emotions and $0.05$ and $0.09$ mean squared errors for\narousal and valence in OMG emotion recognition challenge. Our results were\nobtained with ensemble of single modality models trained on voice and face data\nfrom video separately. We consider each stream as a sequence of frames. Next we\nestimated features from frames and handle it with recurrent neural network. As\naudio frame we mean short $0.4$ second spectrogram interval. For features\nestimation for face pictures we used own ResNet neural network pretrained on\nAffectNet database. Each short spectrogram was considered as a picture and\nprocessed by convolutional network too. As a base audio model we used ResNet\npretrained in speaker recognition task. Predictions from both modalities were\nfused on decision level and improve single-channel approaches by a few percent",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.01416v1",
            "date": "2018-05-03 16:42:20+00:00",
            "title": "Dimensional emotion recognition using visual and textual cues",
            "authors": [
                "Pedro M. Ferreira",
                "Diogo Pernes",
                "Kelwin Fernandes",
                "Ana Rebelo",
                "Jaime S. Cardoso"
            ],
            "abstract": "This paper addresses the problem of automatic emotion recognition in the\nscope of the One-Minute Gradual-Emotional Behavior challenge (OMG-Emotion\nchallenge). The underlying objective of the challenge is the automatic\nestimation of emotion expressions in the two-dimensional emotion representation\nspace (i.e., arousal and valence). The adopted methodology is a weighted\nensemble of several models from both video and text modalities. For video-based\nrecognition, two different types of visual cues (i.e., face and facial\nlandmarks) were considered to feed a multi-input deep neural network. Regarding\nthe text modality, a sequential model based on a simple recurrent architecture\nwas implemented. In addition, we also introduce a model based on high-level\nfeatures in order to embed domain knowledge in the learning process.\nExperimental results on the OMG-Emotion validation set demonstrate the\neffectiveness of the implemented ensemble model as it clearly outperforms the\ncurrent baseline methods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.02214v1",
            "date": "2018-05-06 13:53:50+00:00",
            "title": "Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens",
            "authors": [
                "Marek Rei",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Can attention- or gradient-based visualization techniques be used to infer\ntoken-level labels for binary sequence tagging problems, using networks trained\nonly on sentence-level labels? We construct a neural network architecture based\non soft attention, train it as a binary sentence classifier and evaluate\nagainst token-level annotation on four different datasets. Inferring token\nlabels from a network provides a method for quantitatively evaluating what the\nmodel is learning, along with generating useful feedback in assistance systems.\nOur results indicate that attention-based methods are able to predict\ntoken-level labels more accurately, compared to gradient-based methods,\nsometimes even rivaling the supervised oracle network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02474v1",
            "date": "2018-05-07 12:36:54+00:00",
            "title": "Sentence-State LSTM for Text Representation",
            "authors": [
                "Yue Zhang",
                "Qi Liu",
                "Linfeng Song"
            ],
            "abstract": "Bi-directional LSTMs are a powerful tool for text representation. On the\nother hand, they have been shown to suffer various limitations due to their\nsequential nature. We investigate an alternative LSTM structure for encoding\ntext, which consists of a parallel state for each word. Recurrent steps are\nused to perform local and global information exchange between words\nsimultaneously, rather than incremental reading of a sequence of words. Results\non various classification and sequence labelling benchmarks show that the\nproposed model has strong representation power, giving highly competitive\nperformances compared to stacked BiLSTM models with similar parameter numbers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02856v1",
            "date": "2018-05-08 06:46:03+00:00",
            "title": "Reasoning with Sarcasm by Reading In-between",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui",
                "Jian Su"
            ],
            "abstract": "Sarcasm is a sophisticated speech act which commonly manifests on social\ncommunities such as Twitter and Reddit. The prevalence of sarcasm on the social\nweb is highly disruptive to opinion mining systems due to not only its tendency\nof polarity flipping but also usage of figurative language. Sarcasm commonly\nmanifests with a contrastive theme either between positive-negative sentiments\nor between literal-figurative scenarios. In this paper, we revisit the notion\nof modeling contrast in order to reason with sarcasm. More specifically, we\npropose an attention-based neural model that looks in-between instead of\nacross, enabling it to explicitly model contrast and incongruity. We conduct\nextensive experiments on six benchmark datasets from Twitter, Reddit and the\nInternet Argument Corpus. Our proposed model not only achieves state-of-the-art\nperformance on all datasets but also enjoys improved interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02867v2",
            "date": "2018-05-08 07:34:17+00:00",
            "title": "Online normalizer calculation for softmax",
            "authors": [
                "Maxim Milakov",
                "Natalia Gimelshein"
            ],
            "abstract": "The Softmax function is ubiquitous in machine learning, multiple previous\nworks suggested faster alternatives for it. In this paper we propose a way to\ncompute classical Softmax with fewer memory accesses and hypothesize that this\nreduction in memory accesses should improve Softmax performance on actual\nhardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to\n1.3x and Softmax+TopK combined and fused by up to 5x.",
            "journal ref": "None",
            "category": "cs.PF"
        },
        {
            "url": "http://arxiv.org/abs/1805.02917v1",
            "date": "2018-05-08 09:27:46+00:00",
            "title": "Interpretable Adversarial Perturbation in Input Embedding Space for Text",
            "authors": [
                "Motoki Sato",
                "Jun Suzuki",
                "Hiroyuki Shindo",
                "Yuji Matsumoto"
            ],
            "abstract": "Following great success in the image processing field, the idea of\nadversarial training has been applied to tasks in the natural language\nprocessing (NLP) field. One promising approach directly applies adversarial\ntraining developed in the image processing field to the input word embedding\nspace instead of the discrete input space of texts. However, this approach\nabandons such interpretability as generating adversarial texts to significantly\nimprove the performance of NLP tasks. This paper restores interpretability to\nsuch methods by restricting the directions of perturbations toward the existing\nwords in the input embedding space. As a result, we can straightforwardly\nreconstruct each input with perturbations to an actual text by considering the\nperturbations to be the replacement of words in the sentence while maintaining\nor even improving the task performance.",
            "journal ref": "IJCAI-ECAI-2018",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.02958v1",
            "date": "2018-05-08 11:54:06+00:00",
            "title": "A Regression Model of Recurrent Deep Neural Networks for Noise Robust Estimation of the Fundamental Frequency Contour of Speech",
            "authors": [
                "Akihiro Kato",
                "Tomi Kinnunen"
            ],
            "abstract": "The fundamental frequency (F0) contour of speech is a key aspect to represent\nspeech prosody that finds use in speech and spoken language analysis such as\nvoice conversion and speech synthesis as well as speaker and language\nidentification. This work proposes new methods to estimate the F0 contour of\nspeech using deep neural networks (DNNs) and recurrent neural networks (RNNs).\nThey are trained using supervised learning with the ground truth of F0\ncontours. The latest prior research addresses this problem first as a\nframe-by-frame-classification problem followed by sequence tracking using deep\nneural network hidden Markov model (DNN-HMM) hybrid architecture. This study,\nhowever, tackles the problem as a regression problem instead, in order to\nobtain F0 contours with higher frequency resolution from clean and noisy\nspeech. Experiments using PTDB-TUG corpus contaminated with additive noise\n(NOISEX-92) show the proposed method improves gross pitch error (GPE) by more\nthan 25 % at signal-to-noise ratios (SNRs) between -10 dB and +10 dB as\ncompared with one of the most noise-robust F0 trackers, PEFAC. Furthermore, the\nperformance on fine pitch error (FPE) is improved by approximately 20 % against\na state-of-the-art DNN-HMM-based approach.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.03162v1",
            "date": "2018-05-08 16:56:15+00:00",
            "title": "Polite Dialogue Generation Without Parallel Data",
            "authors": [
                "Tong Niu",
                "Mohit Bansal"
            ],
            "abstract": "Stylistic dialogue response generation, with valuable applications in\npersonality-based conversational agents, is a challenging task because the\nresponse needs to be fluent, contextually-relevant, as well as\nparalinguistically accurate. Moreover, parallel datasets for\nregular-to-stylistic pairs are usually unavailable. We present three\nweakly-supervised models that can generate diverse polite (or rude) dialogue\nresponses without parallel data. Our late fusion model (Fusion) merges the\ndecoder of an encoder-attention-decoder dialogue model with a language model\ntrained on stand-alone polite utterances. Our label-fine-tuning (LFT) model\nprepends to each source sequence a politeness-score scaled label (predicted by\nour state-of-the-art politeness classifier) during training, and at test time\nis able to generate polite, neutral, and rude responses by simply scaling the\nlabel embedding by the corresponding score. Our reinforcement learning model\n(Polite-RL) encourages politeness generation by assigning rewards proportional\nto the politeness classifier score of the sampled response. We also present two\nretrieval-based polite dialogue model baselines. Human evaluation validates\nthat while the Fusion and the retrieval-based models achieve politeness with\npoorer context-relevance, the LFT and Polite-RL models can produce\nsignificantly more polite responses without sacrificing dialogue quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03294v1",
            "date": "2018-05-08 21:27:04+00:00",
            "title": "Improved training of end-to-end attention models for speech recognition",
            "authors": [
                "Albert Zeyer",
                "Kazuki Irie",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "Sequence-to-sequence attention-based models on subword units allow simple\nopen-vocabulary end-to-end speech recognition. In this work, we show that such\nmodels can achieve competitive results on the Switchboard 300h and LibriSpeech\n1000h tasks. In particular, we report the state-of-the-art word error rates\n(WER) of 3.54% on the dev-clean and 3.82% on the test-clean evaluation subsets\nof LibriSpeech. We introduce a new pretraining scheme by starting with a high\ntime reduction factor and lowering it during training, which is crucial both\nfor convergence and final performance. In some experiments, we also use an\nauxiliary CTC loss function to help the convergence. In addition, we train long\nshort-term memory (LSTM) language models on subword units. By shallow fusion,\nwe report up to 27% relative improvements in WER over the attention baseline\nwithout a language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03322v1",
            "date": "2018-05-08 23:59:04+00:00",
            "title": "Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Panayiotis Georgiou"
            ],
            "abstract": "Children speech recognition is challenging mainly due to the inherent high\nvariability in children's physical and articulatory characteristics and\nexpressions. This variability manifests in both acoustic constructs and\nlinguistic usage due to the rapidly changing developmental stage in children's\nlife. Part of the challenge is due to the lack of large amounts of available\nchildren speech data for efficient modeling. This work attempts to address the\nkey challenges using transfer learning from adult's models to children's models\nin a Deep Neural Network (DNN) framework for children's Automatic Speech\nRecognition (ASR) task evaluating on multiple children's speech corpora with a\nlarge vocabulary. The paper presents a systematic and an extensive analysis of\nthe proposed transfer learning technique considering the key factors affecting\nchildren's speech recognition from prior literature. Evaluations are presented\non (i) comparisons of earlier GMM-HMM and the newer DNN Models, (ii)\neffectiveness of standard adaptation techniques versus transfer learning, (iii)\nvarious adaptation configurations in tackling the variabilities present in\nchildren speech, in terms of (a) acoustic spectral variability, and (b)\npronunciation variability and linguistic constraints. Our Analysis spans over\n(i) number of DNN model parameters (for adaptation), (ii) amount of adaptation\ndata, (iii) ages of children, (iv) age dependent-independent adaptation.\nFinally, we provide Recommendations on (i) the favorable strategies over\nvarious aforementioned - analyzed parameters, and (ii) potential future\nresearch directions and relevant challenges/problems persisting in DNN based\nASR for children's speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.03379v1",
            "date": "2018-05-09 05:44:19+00:00",
            "title": "Opinion Fraud Detection via Neural Autoencoder Decision Forest",
            "authors": [
                "Manqing Dong",
                "Lina Yao",
                "Xianzhi Wang",
                "Boualem Benatallah",
                "Chaoran Huang",
                "Xiaodong Ning"
            ],
            "abstract": "Online reviews play an important role in influencing buyers' daily purchase\ndecisions. However, fake and meaningless reviews, which cannot reflect users'\ngenuine purchase experience and opinions, widely exist on the Web and pose\ngreat challenges for users to make right choices. Therefore,it is desirable to\nbuild a fair model that evaluates the quality of products by distinguishing\nspamming reviews. We present an end-to-end trainable unified model to leverage\nthe appealing properties from Autoencoder and random forest. A stochastic\ndecision tree model is implemented to guide the global parameter learning\nprocess. Extensive experiments were conducted on a large Amazon review dataset.\nThe proposed model consistently outperforms a series of compared methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03435v1",
            "date": "2018-05-09 09:41:51+00:00",
            "title": "Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks",
            "authors": [
                "Vitalii Zhelezniak",
                "Dan Busbridge",
                "April Shen",
                "Samuel L. Smith",
                "Nils Y. Hammerla"
            ],
            "abstract": "Experimental evidence indicates that simple models outperform complex deep\nnetworks on many unsupervised similarity tasks. We provide a simple yet\nrigorous explanation for this behaviour by introducing the concept of an\noptimal representation space, in which semantically close symbols are mapped to\nrepresentations that are close under a similarity measure induced by the\nmodel's objective function. In addition, we present a straightforward procedure\nthat, without any retraining or architectural modifications, allows deep\nrecurrent models to perform equally well (and sometimes better) when compared\nto shallow models. To validate our analysis, we conduct a set of consistent\nempirical evaluations and introduce several new sentence embedding models in\nthe process. Even though this work is presented within the context of natural\nlanguage processing, the insights are readily applicable to other domains that\nrely on distributed representations for transfer tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.03616v3",
            "date": "2018-05-09 16:56:41+00:00",
            "title": "A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization",
            "authors": [
                "Li Wang",
                "Junlin Yao",
                "Yunzhe Tao",
                "Li Zhong",
                "Wei Liu",
                "Qiang Du"
            ],
            "abstract": "In this paper, we propose a deep learning approach to tackle the automatic\nsummarization tasks by incorporating topic information into the convolutional\nsequence-to-sequence (ConvS2S) model and using self-critical sequence training\n(SCST) for optimization. Through jointly attending to topics and word-level\nalignment, our approach can improve coherence, diversity, and informativeness\nof generated summaries via a biased probability generation mechanism. On the\nother hand, reinforcement training, like SCST, directly optimizes the proposed\nmodel with respect to the non-differentiable metric ROUGE, which also avoids\nthe exposure bias during inference. We carry out the experimental evaluation\nwith state-of-the-art methods over the Gigaword, DUC-2004, and LCSTS datasets.\nThe empirical results demonstrate the superiority of our proposed method in the\nabstractive summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03620v1",
            "date": "2018-05-09 17:08:03+00:00",
            "title": "On the Limitations of Unsupervised Bilingual Dictionary Induction",
            "authors": [
                "Anders S\u00f8gaard",
                "Sebastian Ruder",
                "Ivan Vuli\u0107"
            ],
            "abstract": "Unsupervised machine translation---i.e., not assuming any cross-lingual\nsupervision signal, whether a dictionary, translations, or comparable\ncorpora---seems impossible, but nevertheless, Lample et al. (2018) recently\nproposed a fully unsupervised machine translation (MT) model. The model relies\nheavily on an adversarial, unsupervised alignment of word embedding spaces for\nbilingual dictionary induction (Conneau et al., 2018), which we examine here.\nOur results identify the limitations of current unsupervised MT: unsupervised\nbilingual dictionary induction performs much worse on morphologically rich\nlanguages that are not dependent marking, when monolingual corpora from\ndifferent domains or different embedding algorithms are used. We show that a\nsimple trick, exploiting a weak supervision signal from identical words,\nenables more robust induction, and establish a near-perfect correlation between\nunsupervised bilingual dictionary induction performance and a previously\nunexplored graph similarity metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03642v3",
            "date": "2018-05-09 04:06:30+00:00",
            "title": "Adversarial Contrastive Estimation",
            "authors": [
                "Avishek Joey Bose",
                "Huan Ling",
                "Yanshuai Cao"
            ],
            "abstract": "Learning by contrasting positive and negative samples is a general strategy\nadopted by many methods. Noise contrastive estimation (NCE) for word embeddings\nand translating embeddings for knowledge graphs are examples in NLP employing\nthis approach. In this work, we view contrastive learning as an abstraction of\nall such methods and augment the negative sampler into a mixture distribution\ncontaining an adversarially learned sampler. The resulting adaptive sampler\nfinds harder negative examples, which forces the main model to learn a better\nrepresentation of the data. We evaluate our proposal on learning word\nembeddings, order embeddings and knowledge graph embeddings and observe both\nfaster convergence and improved results on multiple metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03687v2",
            "date": "2018-05-08 11:58:20+00:00",
            "title": "Statistical Analysis on E-Commerce Reviews, with Sentiment Classification using Bidirectional Recurrent Neural Network (RNN)",
            "authors": [
                "Abien Fred Agarap"
            ],
            "abstract": "Understanding customer sentiments is of paramount importance in marketing\nstrategies today. Not only will it give companies an insight as to how\ncustomers perceive their products and/or services, but it will also give them\nan idea on how to improve their offers. This paper attempts to understand the\ncorrelation of different variables in customer reviews on a women clothing\ne-commerce, and to classify each review whether it recommends the reviewed\nproduct or not and whether it consists of positive, negative, or neutral\nsentiment. To achieve these goals, we employed univariate and multivariate\nanalyses on dataset features except for review titles and review texts, and we\nimplemented a bidirectional recurrent neural network (RNN) with long-short term\nmemory unit (LSTM) for recommendation and sentiment classification. Results\nhave shown that a recommendation is a strong indicator of a positive sentiment\nscore, and vice-versa. On the other hand, ratings in product reviews are fuzzy\nindicators of sentiment scores. We also found out that the bidirectional LSTM\nwas able to reach an F1-score of 0.88 for recommendation classification, and\n0.93 for sentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03716v1",
            "date": "2018-05-09 20:05:58+00:00",
            "title": "Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum",
            "authors": [
                "Omer Levy",
                "Kenton Lee",
                "Nicholas FitzGerald",
                "Luke Zettlemoyer"
            ],
            "abstract": "LSTMs were introduced to combat vanishing gradients in simple RNNs by\naugmenting them with gated additive recurrent connections. We present an\nalternative view to explain the success of LSTMs: the gates themselves are\nversatile recurrent models that provide more representational power than\npreviously appreciated. We do this by decoupling the LSTM's gates from the\nembedded simple RNN, producing a new class of RNNs where the recurrence\ncomputes an element-wise weighted sum of context-independent functions of the\ninput. Ablations on a range of problems demonstrate that the gating mechanism\nalone performs as well as an LSTM in most settings, strongly suggesting that\nthe gates are doing much more in practice than just alleviating vanishing\ngradients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.03989v2",
            "date": "2018-05-10 14:11:51+00:00",
            "title": "Global Encoding for Abstractive Summarization",
            "authors": [
                "Junyang Lin",
                "Xu Sun",
                "Shuming Ma",
                "Qi Su"
            ],
            "abstract": "In neural abstractive summarization, the conventional sequence-to-sequence\n(seq2seq) model often suffers from repetition and semantic irrelevance. To\ntackle the problem, we propose a global encoding framework, which controls the\ninformation flow from the encoder to the decoder based on the global\ninformation of the source context. It consists of a convolutional gated unit to\nperform global encoding to improve the representations of the source-side\ninformation. Evaluations on the LCSTS and the English Gigaword both demonstrate\nthat our model outperforms the baseline models, and the analysis shows that our\nmodel is capable of reducing repetition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04247v2",
            "date": "2018-05-11 06:13:56+00:00",
            "title": "Reciprocal Attention Fusion for Visual Question Answering",
            "authors": [
                "Moshiur R Farazi",
                "Salman H Khan"
            ],
            "abstract": "Existing attention mechanisms either attend to local image grid or object\nlevel features for Visual Question Answering (VQA). Motivated by the\nobservation that questions can relate to both object instances and their parts,\nwe propose a novel attention mechanism that jointly considers reciprocal\nrelationships between the two levels of visual details. The bottom-up attention\nthus generated is further coalesced with the top-down information to only focus\non the scene elements that are most relevant to a given question. Our design\nhierarchically fuses multi-modal information i.e., language, object- and\ngird-level features, through an efficient tensor decomposition scheme. The\nproposed model improves the state-of-the-art single model performances from\n67.9% to 68.2% on VQAv1 and from 65.7% to 67.4% on VQAv2, demonstrating a\nsignificant boost.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.04609v3",
            "date": "2018-05-11 22:40:59+00:00",
            "title": "Textual Membership Queries",
            "authors": [
                "Jonathan Zarecki",
                "Shaul Markovitch"
            ],
            "abstract": "Human labeling of data can be very time-consuming and expensive, yet, in many\ncases it is critical for the success of the learning process. In order to\nminimize human labeling efforts, we propose a novel active learning solution\nthat does not rely on existing sources of unlabeled data. It uses a small\namount of labeled data as the core set for the synthesis of useful membership\nqueries (MQs) - unlabeled instances generated by an algorithm for human\nlabeling. Our solution uses modification operators, functions that modify\ninstances to some extent. We apply the operators on a small set of instances\n(core set), creating a set of new membership queries. Using this framework, we\nlook at the instance space as a search space and apply search algorithms in\norder to generate new examples highly relevant to the learner. We implement\nthis framework in the textual domain and test it on several text classification\ntasks and show improved classifier performance as more MQs are labeled and\nincorporated into the training set. To the best of our knowledge, this is the\nfirst work on membership queries in the textual domain.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.04661v1",
            "date": "2018-05-12 06:00:47+00:00",
            "title": "Examining a hate speech corpus for hate speech detection and popularity prediction",
            "authors": [
                "Filip Klubi\u010dka",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "As research on hate speech becomes more and more relevant every day, most of\nit is still focused on hate speech detection. By attempting to replicate a hate\nspeech detection experiment performed on an existing Twitter corpus annotated\nfor hate speech, we highlight some issues that arise from doing research in the\nfield of hate speech, which is essentially still in its infancy. We take a\ncritical look at the training corpus in order to understand its biases, while\nalso using it to venture beyond hate speech detection and investigate whether\nit can be used to shed light on other facets of research, such as popularity of\nhate tweets.",
            "journal ref": "In Proceedings of 4REAL Workshop 9-16 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04680v1",
            "date": "2018-05-12 07:52:59+00:00",
            "title": "AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples",
            "authors": [
                "Dongyeop Kang",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Eduard Hovy"
            ],
            "abstract": "We consider the problem of learning textual entailment models with limited\nsupervision (5K-10K training examples), and present two complementary\napproaches for it. First, we propose knowledge-guided adversarial example\ngenerators for incorporating large lexical resources in entailment models via\nonly a handful of rule templates. Second, to make the entailment model - a\ndiscriminator - more robust, we propose the first GAN-style approach for\ntraining it using a natural language example generator that iteratively adjusts\nbased on the discriminator's performance. We demonstrate effectiveness using\ntwo entailment datasets, where the proposed methods increase accuracy by 4.7%\non SciTail and by 2.8% on a 1% training sub-sample of SNLI. Notably, even a\nsingle hand-written rule, negate, improves the accuracy on the negation\nexamples in SNLI by 6.1%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.04908v1",
            "date": "2018-05-13 16:28:32+00:00",
            "title": "On the Practical Computational Power of Finite Precision RNNs for Language Recognition",
            "authors": [
                "Gail Weiss",
                "Yoav Goldberg",
                "Eran Yahav"
            ],
            "abstract": "While Recurrent Neural Networks (RNNs) are famously known to be Turing\ncomplete, this relies on infinite precision in the states and unbounded\ncomputation time. We consider the case of RNNs with finite precision whose\ncomputation time is linear in the input length. Under these limitations, we\nshow that different RNN variants have different computational power. In\nparticular, we show that the LSTM and the Elman-RNN with ReLU activation are\nstrictly stronger than the RNN with a squashing activation and the GRU. This is\nachieved because LSTMs and ReLU-RNNs can easily implement counting behavior. We\nshow empirically that the LSTM does indeed learn to effectively use the\ncounting mechanism.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.05225v2",
            "date": "2018-05-14 15:23:40+00:00",
            "title": "RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition",
            "authors": [
                "Albert Zeyer",
                "Tamer Alkhouli",
                "Hermann Ney"
            ],
            "abstract": "We compare the fast training and decoding speed of RETURNN of attention\nmodels for translation, due to fast CUDA LSTM kernels, and a fast pure\nTensorFlow beam search decoder. We show that a layer-wise pretraining scheme\nfor recurrent attention models gives over 1% BLEU improvement absolute and it\nallows to train deeper recurrent encoder networks. Promising preliminary\nresults on max. expected BLEU training are presented. We are able to train\nstate-of-the-art models for translation and end-to-end models for speech\nrecognition and show results on WMT 2017 and Switchboard. The flexibility of\nRETURNN allows a fast research feedback loop to experiment with alternative\narchitectures, and its generality allows to use it on a wide range of\napplications.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1805.05361v1",
            "date": "2018-05-14 18:04:28+00:00",
            "title": "NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing",
            "authors": [
                "Dinghan Shen",
                "Qinliang Su",
                "Paidamoyo Chapfuwa",
                "Wenlin Wang",
                "Guoyin Wang",
                "Lawrence Carin",
                "Ricardo Henao"
            ],
            "abstract": "Semantic hashing has become a powerful paradigm for fast similarity search in\nmany information retrieval systems. While fairly successful, previous\ntechniques generally require two-stage training, and the binary constraints are\nhandled ad-hoc. In this paper, we present an end-to-end Neural Architecture for\nSemantic Hashing (NASH), where the binary hashing codes are treated as\nBernoulli latent variables. A neural variational inference framework is\nproposed for training, where gradients are directly back-propagated through the\ndiscrete latent variable to optimize the hash function. We also draw\nconnections between proposed method and rate-distortion theory, which provides\na theoretical foundation for the effectiveness of the proposed framework.\nExperimental results on three public datasets demonstrate that our method\nsignificantly outperforms several state-of-the-art models on both unsupervised\nand supervised scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05491v1",
            "date": "2018-05-14 22:59:56+00:00",
            "title": "Crowdbreaks: Tracking Health Trends using Public Social Media Data and Crowdsourcing",
            "authors": [
                "Martin Mueller",
                "Marcel Salath\u00e9"
            ],
            "abstract": "In the past decade, tracking health trends using social media data has shown\ngreat promise, due to a powerful combination of massive adoption of social\nmedia around the world, and increasingly potent hardware and software that\nenables us to work with these new big data streams. At the same time, many\nchallenging problems have been identified. First, there is often a mismatch\nbetween how rapidly online data can change, and how rapidly algorithms are\nupdated, which means that there is limited reusability for algorithms trained\non past data as their performance decreases over time. Second, much of the work\nis focusing on specific issues during a specific past period in time, even\nthough public health institutions would need flexible tools to assess multiple\nevolving situations in real time. Third, most tools providing such capabilities\nare proprietary systems with little algorithmic or data transparency, and thus\nlittle buy-in from the global public health and research community. Here, we\nintroduce Crowdbreaks, an open platform which allows tracking of health trends\nby making use of continuous crowdsourced labelling of public social media\ncontent. The system is built in a way which automatizes the typical workflow\nfrom data collection, filtering, labelling and training of machine learning\nclassifiers and therefore can greatly accelerate the research process in the\npublic health domain. This work introduces the technical aspects of the\nplatform and explores its future use cases.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1805.05574v1",
            "date": "2018-05-15 05:46:23+00:00",
            "title": "Improved ASR for Under-Resourced Languages Through Multi-Task Learning with Acoustic Landmarks",
            "authors": [
                "Di He",
                "Boon Pang Lim",
                "Xuesong Yang",
                "Mark Hasegawa-Johnson",
                "Deming Chen"
            ],
            "abstract": "Furui first demonstrated that the identity of both consonant and vowel can be\nperceived from the C-V transition; later, Stevens proposed that acoustic\nlandmarks are the primary cues for speech perception, and that steady-state\nregions are secondary or supplemental. Acoustic landmarks are perceptually\nsalient, even in a language one doesn't speak, and it has been demonstrated\nthat non-speakers of the language can identify features such as the primary\narticulator of the landmark. These factors suggest a strategy for developing\nlanguage-independent automatic speech recognition: landmarks can potentially be\nlearned once from a suitably labeled corpus and rapidly applied to many other\nlanguages. This paper proposes enhancing the cross-lingual portability of a\nneural network by using landmarks as the secondary task in multi-task learning\n(MTL). The network is trained in a well-resourced source language with both\nphone and landmark labels (English), then adapted to an under-resourced target\nlanguage with only word labels (Iban). Landmark-tasked MTL reduces\nsource-language phone error rate by 2.9% relative, and reduces target-language\nword error rate by 1.9%-5.9% depending on the amount of target-language\ntraining data. These results suggest that landmark-tasked MTL causes the DNN to\nlearn hidden-node features that are useful for cross-lingual adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.05622v3",
            "date": "2018-05-15 08:15:40+00:00",
            "title": "Stories for Images-in-Sequence by using Visual and Narrative Components",
            "authors": [
                "Marko Smilevski",
                "Ilija Lalkovski",
                "Gjorgji Madjarov"
            ],
            "abstract": "Recent research in AI is focusing towards generating narrative stories about\nvisual scenes. It has the potential to achieve more human-like understanding\nthan just basic description generation of images- in-sequence. In this work, we\npropose a solution for generating stories for images-in-sequence that is based\non the Sequence to Sequence model. As a novelty, our encoder model is composed\nof two separate encoders, one that models the behaviour of the image sequence\nand other that models the sentence-story generated for the previous image in\nthe sequence of images. By using the image sequence encoder we capture the\ntemporal dependencies between the image sequence and the sentence-story and by\nusing the previous sentence-story encoder we achieve a better story flow. Our\nsolution generates long human-like stories that not only describe the visual\ncontext of the image sequence but also contains narrative and evaluative\nlanguage. The obtained results were confirmed by manual human evaluation.",
            "journal ref": "ICT Innovations 2018. Engineering and Life Sciences. ICT 2018.\n  Communications in Computer and Information Science, vol 940. Springer, Cham\n  (2018) pp. 148-159",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.05631v2",
            "date": "2018-05-15 08:23:56+00:00",
            "title": "Complexity Reduction in the Negotiation of New Lexical Conventions",
            "authors": [
                "William Schueller",
                "Vittorio Loreto",
                "Pierre-Yves Oudeyer"
            ],
            "abstract": "In the process of collectively inventing new words for new concepts in a\npopulation, conflicts can quickly become numerous, in the form of synonymy and\nhomonymy. Remembering all of them could cost too much memory, and remembering\ntoo few may slow down the overall process. Is there an efficient behavior that\ncould help balance the two? The Naming Game is a multi-agent computational\nmodel for the emergence of language, focusing on the negotiation of new lexical\nconventions, where a common lexicon self-organizes but going through a phase of\nhigh complexity. Previous work has been done on the control of complexity\ngrowth in this particular model, by allowing agents to actively choose what\nthey talk about. However, those strategies were relying on ad hoc heuristics\nhighly dependent on fine-tuning of parameters. We define here a new principled\nmeasure and a new strategy, based on the beliefs of each agent on the global\nstate of the population. The measure does not rely on heavy computation, and is\ncognitively plausible. The new strategy yields an efficient control of\ncomplexity growth, along with a faster agreement process. Also, we show that\nshort-term memory is enough to build relevant beliefs about the global lexicon.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/1805.05826v1",
            "date": "2018-05-15 14:45:33+00:00",
            "title": "A Purely End-to-end System for Multi-speaker Speech Recognition",
            "authors": [
                "Hiroshi Seki",
                "Takaaki Hori",
                "Shinji Watanabe",
                "Jonathan Le Roux",
                "John R. Hershey"
            ],
            "abstract": "Recently, there has been growing interest in multi-speaker speech\nrecognition, where the utterances of multiple speakers are recognized from\ntheir mixture. Promising techniques have been proposed for this task, but\nearlier works have required additional training data such as isolated source\nsignals or senone alignments for effective learning. In this paper, we propose\na new sequence-to-sequence framework to directly decode multiple label\nsequences from a single speech sequence by unifying source separation and\nspeech recognition functions in an end-to-end manner. We further propose a new\nobjective function to improve the contrast between the hidden vectors to avoid\ngenerating similar hypotheses. Experimental results show that the model is\ndirectly able to learn a mapping from a speech mixture to multiple label\nsequences, achieving 83.1 % relative improvement compared to a model trained\nwithout the proposed objective. Interestingly, the results are comparable to\nthose produced by previous end-to-end works featuring explicit separation and\nrecognition modules.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1805.06061v1",
            "date": "2018-05-15 23:03:01+00:00",
            "title": "SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines",
            "authors": [
                "Roy Schwartz",
                "Sam Thomson",
                "Noah A. Smith"
            ],
            "abstract": "Recurrent and convolutional neural networks comprise two distinct families of\nmodels that have proven to be useful for encoding natural language utterances.\nIn this paper we present SoPa, a new model that aims to bridge these two\napproaches. SoPa combines neural representation learning with weighted\nfinite-state automata (WFSAs) to learn a soft version of traditional surface\npatterns. We show that SoPa is an extension of a one-layer CNN, and that such\nCNNs are equivalent to a restricted version of SoPa, and accordingly, to a\nrestricted form of WFSA. Empirically, on three text classification tasks, SoPa\nis comparable or better than both a BiLSTM (RNN) baseline and a CNN baseline,\nand is particularly useful in small data settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06150v1",
            "date": "2018-05-16 06:29:18+00:00",
            "title": "FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learning",
            "authors": [
                "Pararth Shah",
                "Marek Fiser",
                "Aleksandra Faust",
                "J. Chase Kew",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Understanding and following directions provided by humans can enable robots\nto navigate effectively in unknown situations. We present FollowNet, an\nend-to-end differentiable neural architecture for learning multi-modal\nnavigation policies. FollowNet maps natural language instructions as well as\nvisual and depth inputs to locomotion primitives. FollowNet processes\ninstructions using an attention mechanism conditioned on its visual and depth\ninput to focus on the relevant parts of the command while performing the\nnavigation task. Deep reinforcement learning (RL) a sparse reward learns\nsimultaneously the state representation, the attention function, and control\npolicies. We evaluate our agent on a dataset of complex natural language\ndirections that guide the agent through a rich and realistic dataset of\nsimulated homes. We show that the FollowNet agent learns to execute previously\nunseen instructions described with a similar vocabulary, and successfully\nnavigates along paths not encountered during training. The agent shows 30%\nimprovement over a baseline model without the attention mechanism, with 52%\nsuccess rate at novel instructions.",
            "journal ref": "Third Workshop in Machine Learning in the Planning and Control of\n  Robot Motion at ICRA, 2018",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1805.06239v2",
            "date": "2018-05-16 10:43:47+00:00",
            "title": "A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese",
            "authors": [
                "Shiyu Zhou",
                "Linhao Dong",
                "Shuang Xu",
                "Bo Xu"
            ],
            "abstract": "The choice of modeling units is critical to automatic speech recognition\n(ASR) tasks. Conventional ASR systems typically choose context-dependent states\n(CD-states) or context-dependent phonemes (CD-phonemes) as their modeling\nunits. However, it has been challenged by sequence-to-sequence attention-based\nmodels, which integrate an acoustic, pronunciation and language model into a\nsingle neural network. On English ASR tasks, previous attempts have already\nshown that the modeling unit of graphemes can outperform that of phonemes by\nsequence-to-sequence attention-based model.\n  In this paper, we are concerned with modeling units on Mandarin Chinese ASR\ntasks using sequence-to-sequence attention-based models with the Transformer.\nFive modeling units are explored including context-independent phonemes\n(CI-phonemes), syllables, words, sub-words and characters. Experiments on HKUST\ndatasets demonstrate that the lexicon free modeling units can outperform\nlexicon related modeling units in terms of character error rate (CER). Among\nfive modeling units, character based model performs best and establishes a new\nstate-of-the-art CER of $26.64\\%$ on HKUST datasets without a hand-designed\nlexicon and an extra language model integration, which corresponds to a $4.8\\%$\nrelative improvement over the existing best CER of $28.0\\%$ by the joint\nCTC-attention based encoder-decoder network.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.06242v2",
            "date": "2018-05-16 10:51:56+00:00",
            "title": "Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks",
            "authors": [
                "Chandrakant Bothe",
                "Sven Magg",
                "Cornelius Weber",
                "Stefan Wermter"
            ],
            "abstract": "Recent approaches for dialogue act recognition have shown that context from\npreceding utterances is important to classify the subsequent one. It was shown\nthat the performance improves rapidly when the context is taken into account.\nWe propose an utterance-level attention-based bidirectional recurrent neural\nnetwork (Utt-Att-BiRNN) model to analyze the importance of preceding utterances\nto classify the current one. In our setup, the BiRNN is given the input set of\ncurrent and preceding utterances. Our model outperforms previous models that\nuse only preceding utterances as context on the used corpus. Another\ncontribution of the article is to discover the amount of information in each\nutterance to classify the subsequent one and to show that context-based\nlearning not only improves the performance but also achieves higher confidence\nin the classification. We use character- and word-level features to represent\nthe utterances. The results are presented for character and word feature\nrepresentations and as an ensemble model of both representations. We found that\nwhen classifying short utterances, the closest preceding utterances contributes\nto a higher degree.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06280v1",
            "date": "2018-05-16 12:58:18+00:00",
            "title": "A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks",
            "authors": [
                "Chandrakant Bothe",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Dialogue act recognition is an important part of natural language\nunderstanding. We investigate the way dialogue act corpora are annotated and\nthe learning approaches used so far. We find that the dialogue act is\ncontext-sensitive within the conversation for most of the classes.\nNevertheless, previous models of dialogue act classification work on the\nutterance-level and only very few consider context. We propose a novel\ncontext-based learning method to classify dialogue acts using a character-level\nlanguage model utterance representation, and we notice significant improvement.\nWe evaluate this method on the Switchboard Dialogue Act corpus, and our results\nshow that the consideration of the preceding utterances as a context of the\ncurrent utterance improves dialogue act detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06289v1",
            "date": "2018-05-02 10:38:57+00:00",
            "title": "Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets",
            "authors": [
                "Firoj Alam",
                "Shafiq Joty",
                "Muhammad Imran"
            ],
            "abstract": "During time-critical situations such as natural disasters, rapid\nclassification of data posted on social networks by affected people is useful\nfor humanitarian organizations to gain situational awareness and to plan\nresponse efforts. However, the scarcity of labeled data in the early hours of a\ncrisis hinders machine learning tasks thus delays crisis response. In this\nwork, we propose to use an inductive semi-supervised technique to utilize\nunlabeled data, which is often abundant at the onset of a crisis event, along\nwith fewer labeled data. Specif- ically, we adopt a graph-based deep learning\nframework to learn an inductive semi-supervised model. We use two real-world\ncrisis datasets from Twitter to evaluate the proposed approach. Our results\nshow significant improvements using unlabeled data as compared to only using\nlabeled data.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1805.06297v2",
            "date": "2018-05-16 13:23:48+00:00",
            "title": "A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre"
            ],
            "abstract": "Recent work has managed to learn cross-lingual word embeddings without\nparallel data by mapping monolingual embeddings to a shared space through\nadversarial training. However, their evaluation has focused on favorable\nconditions, using comparable corpora or closely-related languages, and we show\nthat they often fail in more realistic scenarios. This work proposes an\nalternative approach based on a fully unsupervised initialization that\nexplicitly exploits the structural similarity of the embeddings, and a robust\nself-learning algorithm that iteratively improves this solution. Our method\nsucceeds in all tested scenarios and obtains the best published results in\nstandard datasets, even surpassing previous supervised systems. Our\nimplementation is released as an open source project at\nhttps://github.com/artetxem/vecmap",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06502v2",
            "date": "2018-05-10 09:11:43+00:00",
            "title": "First Experiments with Neural Translation of Informal to Formal Mathematics",
            "authors": [
                "Qingxiang Wang",
                "Cezary Kaliszyk",
                "Josef Urban"
            ],
            "abstract": "We report on our experiments to train deep neural networks that automatically\ntranslate informalized LaTeX-written Mizar texts into the formal Mizar\nlanguage. To the best of our knowledge, this is the first time when neural\nnetworks have been adopted in the formalization of mathematics. Using Luong et\nal.'s neural machine translation model (NMT), we tested our aligned\ninformal-formal corpora against various hyperparameters and evaluated their\nresults. Our experiments show that our best performing model configurations are\nable to generate correct Mizar statements on 65.73\\% of the inference data,\nwith the union of all models covering 79.17\\%. These results indicate that\nformalization through artificial neural network is a promising approach for\nautomated formalization of mathematics. We present several case studies to\nillustrate our results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06524v1",
            "date": "2018-05-10 06:11:27+00:00",
            "title": "Hybrid Adaptive Fuzzy Extreme Learning Machine for text classification",
            "authors": [
                "Ming Li",
                "Peilun Xiao",
                "Ju Zhang"
            ],
            "abstract": "In traditional ELM and its improved versions suffer from the problems of\noutliers or noises due to overfitting and imbalance due to distribution. We\npropose a novel hybrid adaptive fuzzy ELM(HA-FELM), which introduces a fuzzy\nmembership function to the traditional ELM method to deal with the above\nproblems. We define the fuzzy membership function not only basing on the\ndistance between each sample and the center of the class but also the density\namong samples which based on the quantum harmonic oscillator model. The\nproposed fuzzy membership function overcomes the shortcoming of the traditional\nfuzzy membership function and could make itself adjusted according to the\nspecific distribution of different samples adaptively. Experiments show the\nproposed HA-FELM can produce better performance than SVM, ELM, and RELM in text\nclassification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.06525v1",
            "date": "2018-05-10 06:10:46+00:00",
            "title": "Text classification based on ensemble extreme learning machine",
            "authors": [
                "Ming Li",
                "Peilun Xiao",
                "Ju Zhang"
            ],
            "abstract": "In this paper, we propose a novel approach based on cost-sensitive ensemble\nweighted extreme learning machine; we call this approach AE1-WELM. We apply\nthis approach to text classification. AE1-WELM is an algorithm including\nbalanced and imbalanced multiclassification for text classification. Weighted\nELM assigning the different weights to the different samples improves the\nclassification accuracy to a certain extent, but weighted ELM considers the\ndifferences between samples in the different categories only and ignores the\ndifferences between samples within the same categories. We measure the\nimportance of the documents by the sample information entropy, and generate\ncost-sensitive matrix and factor based on the document importance, then embed\nthe cost-sensitive weighted ELM into the AdaBoost.M1 framework seamlessly.\nVector space model(VSM) text representation produces the high dimensions and\nsparse features which increase the burden of ELM. To overcome this problem, we\ndevelop a text classification framework combining the word vector and AE1-WELM.\nThe experimental results show that our method provides an accurate, reliable\nand effective solution for text classification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1805.06549v1",
            "date": "2018-05-16 22:50:17+00:00",
            "title": "Defoiling Foiled Image Captions",
            "authors": [
                "Pranava Madhyastha",
                "Josiah Wang",
                "Lucia Specia"
            ],
            "abstract": "We address the task of detecting foiled image captions, i.e. identifying\nwhether a caption contains a word that has been deliberately replaced by a\nsemantically similar word, thus rendering it inaccurate with respect to the\nimage being described. Solving this problem should in principle require a\nfine-grained understanding of images to detect linguistically valid\nperturbations in captions. In such contexts, encoding sufficiently descriptive\nimage information becomes a key challenge. In this paper, we demonstrate that\nit is possible to solve this task using simple, interpretable yet powerful\nrepresentations based on explicit object information. Our models achieve\nstate-of-the-art performance on a standard dataset, with scores exceeding those\nachieved by humans on the task. We also measure the upper-bound performance of\nour models using gold standard annotations. Our analysis reveals that the\nsimpler model performs well even without image information, suggesting that the\ndataset contains strong linguistic bias.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.06606v2",
            "date": "2018-05-17 05:51:00+00:00",
            "title": "Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data",
            "authors": [
                "Chan Woo Lee",
                "Kyu Ye Song",
                "Jihoon Jeong",
                "Woo Yong Choi"
            ],
            "abstract": "Emotion recognition has become a popular topic of interest, especially in the\nfield of human computer interaction. Previous works involve unimodal analysis\nof emotion, while recent efforts focus on multi-modal emotion recognition from\nvision and speech. In this paper, we propose a new method of learning about the\nhidden representations between just speech and text data using convolutional\nattention networks. Compared to the shallow model which employs simple\nconcatenation of feature vectors, the proposed attention model performs much\nbetter in classifying emotion from speech and text data contained in the\nCMU-MOSEI dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06960v2",
            "date": "2018-05-17 20:32:08+00:00",
            "title": "Ask No More: Deciding when to guess in referential visual dialogue",
            "authors": [
                "Ravi Shekhar",
                "Tim Baumgartner",
                "Aashish Venkatesh",
                "Elia Bruni",
                "Raffaella Bernardi",
                "Raquel Fernandez"
            ],
            "abstract": "Our goal is to explore how the abilities brought in by a dialogue manager can\nbe included in end-to-end visually grounded conversational agents. We make\ninitial steps towards this general goal by augmenting a task-oriented visual\ndialogue model with a decision-making component that decides whether to ask a\nfollow-up question to identify a target referent in an image, or to stop the\nconversation to make a guess. Our analyses show that adding a decision making\ncomponent produces dialogues that are less repetitive and that include fewer\nunnecessary questions, thus potentially leading to more efficient and less\nunnatural interactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06966v1",
            "date": "2018-05-17 21:00:03+00:00",
            "title": "Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems",
            "authors": [
                "Florian Kreyssig",
                "Inigo Casanueva",
                "Pawel Budzianowski",
                "Milica Gasic"
            ],
            "abstract": "User Simulators are one of the major tools that enable offline training of\ntask-oriented dialogue systems. For this task the Agenda-Based User Simulator\n(ABUS) is often used. The ABUS is based on hand-crafted rules and its output is\nin semantic form. Issues arise from both properties such as limited diversity\nand the inability to interface a text-level belief tracker. This paper\nintroduces the Neural User Simulator (NUS) whose behaviour is learned from a\ncorpus and which generates natural language, hence needing a less labelled\ndataset than simulators generating a semantic output. In comparison to much of\nthe past work on this topic, which evaluates user simulators on corpus-based\nmetrics, we use the NUS to train the policy of a reinforcement learning based\nSpoken Dialogue System. The NUS is compared to the ABUS by evaluating the\npolicies that were trained using the simulators. Cross-model evaluation is\nperformed i.e. training on one simulator and testing on the other. Furthermore,\nthe trained policies are tested on real users. In both evaluation tasks the NUS\noutperformed the ABUS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07024v1",
            "date": "2018-05-18 02:33:53+00:00",
            "title": "Gated Recurrent Unit Based Acoustic Modeling with Future Context",
            "authors": [
                "Jie Li",
                "Xiaorui Wang",
                "Yuanyuan Zhao",
                "Yan Li"
            ],
            "abstract": "The use of future contextual information is typically shown to be helpful for\nacoustic modeling. However, for the recurrent neural network (RNN), it's not so\neasy to model the future temporal context effectively, meanwhile keep lower\nmodel latency. In this paper, we attempt to design a RNN acoustic model that\nbeing capable of utilizing the future context effectively and directly, with\nthe model latency and computation cost as low as possible. The proposed model\nis based on the minimal gated recurrent unit (mGRU) with an input projection\nlayer inserted in it. Two context modules, temporal encoding and temporal\nconvolution, are specifically designed for this architecture to model the\nfuture context. Experimental results on the Switchboard task and an internal\nMandarin ASR task show that, the proposed model performs much better than long\nshort-term memory (LSTM) and mGRU models, whereas enables online decoding with\na maximum latency of 170 ms. This model even outperforms a very strong\nbaseline, TDNN-LSTM, with smaller model latency and almost half less\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07340v2",
            "date": "2018-05-18 17:46:25+00:00",
            "title": "Improved Sentence Modeling using Suffix Bidirectional LSTM",
            "authors": [
                "Siddhartha Brahma"
            ],
            "abstract": "Recurrent neural networks have become ubiquitous in computing representations\nof sequential data, especially textual data in natural language processing. In\nparticular, Bidirectional LSTMs are at the heart of several neural models\nachieving state-of-the-art performance in a wide variety of tasks in NLP.\nHowever, BiLSTMs are known to suffer from sequential bias - the contextual\nrepresentation of a token is heavily influenced by tokens close to it in a\nsentence. We propose a general and effective improvement to the BiLSTM model\nwhich encodes each suffix and prefix of a sequence of tokens in both forward\nand reverse directions. We call our model Suffix Bidirectional LSTM or\nSuBiLSTM. This introduces an alternate bias that favors long range\ndependencies. We apply SuBiLSTMs to several tasks that require sentence\nmodeling. We demonstrate that using SuBiLSTM instead of a BiLSTM in existing\nmodels leads to improvements in performance in learning general sentence\nrepresentations, text classification, textual entailment and paraphrase\ndetection. Using SuBiLSTM we achieve new state-of-the-art results for\nfine-grained sentiment classification and question classification.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.07443v1",
            "date": "2018-05-18 21:04:08+00:00",
            "title": "Multi-view Sentence Representation Learning",
            "authors": [
                "Shuai Tang",
                "Virginia R. de Sa"
            ],
            "abstract": "Multi-view learning can provide self-supervision when different views are\navailable of the same data. The distributional hypothesis provides another form\nof useful self-supervision from adjacent sentences which are plentiful in large\nunlabelled corpora. Motivated by the asymmetry in the two hemispheres of the\nhuman brain as well as the observation that different learning architectures\ntend to emphasise different aspects of sentence meaning, we create a unified\nmulti-view sentence representation learning framework, in which, one view\nencodes the input sentence with a Recurrent Neural Network (RNN), and the other\nview encodes it with a simple linear model, and the training objective is to\nmaximise the agreement specified by the adjacent context information between\ntwo views. We show that, after training, the vectors produced from our\nmulti-view training provide improved representations over the single-view\ntraining, and the combination of different views gives further representational\nimprovement and demonstrates solid transferability on standard downstream\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07467v2",
            "date": "2018-05-18 22:59:18+00:00",
            "title": "Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces",
            "authors": [
                "Yu-An Chung",
                "Wei-Hung Weng",
                "Schrasing Tong",
                "James Glass"
            ],
            "abstract": "Recent research has shown that word embedding spaces learned from text\ncorpora of different languages can be aligned without any parallel data\nsupervision. Inspired by the success in unsupervised cross-lingual word\nembeddings, in this paper we target learning a cross-modal alignment between\nthe embedding spaces of speech and text learned from corpora of their\nrespective modalities in an unsupervised fashion. The proposed framework learns\nthe individual speech and text embedding spaces, and attempts to align the two\nspaces via adversarial training, followed by a refinement procedure. We show\nhow our framework could be used to perform spoken word classification and\ntranslation, and the results on these two tasks demonstrate that the\nperformance of our unsupervised alignment approach is comparable to its\nsupervised counterpart. Our framework is especially useful for developing\nautomatic speech recognition (ASR) and speech-to-text translation systems for\nlow- or zero-resource languages, which have little parallel audio-text data for\ntraining modern supervised ASR and speech-to-text translation models, but\naccount for the majority of the languages spoken across the world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07475v3",
            "date": "2018-05-18 23:31:03+00:00",
            "title": "Learning to Repair Software Vulnerabilities with Generative Adversarial Networks",
            "authors": [
                "Jacob Harer",
                "Onur Ozdemir",
                "Tomo Lazovich",
                "Christopher P. Reale",
                "Rebecca L. Russell",
                "Louis Y. Kim",
                "Peter Chin"
            ],
            "abstract": "Motivated by the problem of automated repair of software vulnerabilities, we\npropose an adversarial learning approach that maps from one discrete source\ndomain to another target domain without requiring paired labeled examples or\nsource and target domains to be bijections. We demonstrate that the proposed\nadversarial learning approach is an effective technique for repairing software\nvulnerabilities, performing close to seq2seq approaches that require labeled\npairs. The proposed Generative Adversarial Network approach is\napplication-agnostic in that it can be applied to other problems similar to\ncode repair, such as grammar correction or sentiment translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.07616v2",
            "date": "2018-05-19 15:51:43+00:00",
            "title": "Do Neural Network Cross-Modal Mappings Really Bridge Modalities?",
            "authors": [
                "Guillem Collell",
                "Marie-Francine Moens"
            ],
            "abstract": "Feed-forward networks are widely used in cross-modal applications to bridge\nmodalities by mapping distributed vectors of one modality to the other, or to a\nshared space. The predicted vectors are then used to perform e.g., retrieval or\nlabeling. Thus, the success of the whole system relies on the ability of the\nmapping to make the neighborhood structure (i.e., the pairwise similarities) of\nthe predicted vectors akin to that of the target vectors. However, whether this\nis achieved has not been investigated yet. Here, we propose a new similarity\nmeasure and two ad hoc experiments to shed light on this issue. In three\ncross-modal benchmarks we learn a large number of language-to-vision and\nvision-to-language neural network mappings (up to five layers) using a rich\ndiversity of image and text features and loss functions. Our results reveal\nthat, surprisingly, the neighborhood structure of the predicted vectors\nconsistently resembles more that of the input vectors than that of the target\nvectors. In a second experiment, we further show that untrained nets do not\nsignificantly disrupt the neighborhood (i.e., semantic) structure of the input\nvectors.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1805.07932v2",
            "date": "2018-05-21 07:58:31+00:00",
            "title": "Bilinear Attention Networks",
            "authors": [
                "Jin-Hwa Kim",
                "Jaehyun Jun",
                "Byoung-Tak Zhang"
            ],
            "abstract": "Attention networks in multimodal learning provide an efficient way to utilize\ngiven visual information selectively. However, the computational cost to learn\nattention distributions for every pair of multimodal input channels is\nprohibitively expensive. To solve this problem, co-attention builds two\nseparate attention distributions for each modality neglecting the interaction\nbetween multimodal inputs. In this paper, we propose bilinear attention\nnetworks (BAN) that find bilinear attention distributions to utilize given\nvision-language information seamlessly. BAN considers bilinear interactions\namong two groups of input channels, while low-rank bilinear pooling extracts\nthe joint representations for each pair of channels. Furthermore, we propose a\nvariant of multimodal residual networks to exploit eight-attention maps of the\nBAN efficiently. We quantitatively and qualitatively evaluate our model on\nvisual question answering (VQA 2.0) and Flickr30k Entities datasets, showing\nthat BAN significantly outperforms previous methods and achieves new\nstate-of-the-arts on both datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.08154v1",
            "date": "2018-05-21 16:18:41+00:00",
            "title": "Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers",
            "authors": [
                "Georgios P. Spithourakis",
                "Sebastian Riedel"
            ],
            "abstract": "Numeracy is the ability to understand and work with numbers. It is a\nnecessary skill for composing and understanding documents in clinical,\nscientific, and other technical domains. In this paper, we explore different\nstrategies for modelling numerals with language models, such as memorisation\nand digit-by-digit composition, and propose a novel neural architecture that\nuses a continuous probability density function to model numerals from an open\nvocabulary. Our evaluation on clinical and scientific datasets shows that using\nhierarchical models to distinguish numerals from words improves a perplexity\nmetric on the subset of numerals by 2 and 4 orders of magnitude, respectively,\nover non-hierarchical models. A combination of strategies can further improve\nperplexity. Our continuous probability density function model reduces mean\nabsolute percentage errors by 18% and 54% in comparison to the second best\nstrategy for each dataset, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.08329v2",
            "date": "2018-05-22 00:16:39+00:00",
            "title": "Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents",
            "authors": [
                "Haonan Yu",
                "Xiaochen Lian",
                "Haichao Zhang",
                "Wei Xu"
            ],
            "abstract": "Recently there has been a rising interest in training agents, embodied in\nvirtual environments, to perform language-directed tasks by deep reinforcement\nlearning. In this paper, we propose a simple but effective neural language\ngrounding module for embodied agents that can be trained end to end from\nscratch taking raw pixels, unstructured linguistic commands, and sparse rewards\nas the inputs. We model the language grounding process as a language-guided\ntransformation of visual features, where latent sentence embeddings are used as\nthe transformation matrices. In several language-directed navigation tasks that\nfeature challenging partial observability and require simple reasoning, our\nmodule significantly outperforms the state of the art. We also release\nXWorld3D, an easy-to-customize 3D environment that can potentially be modified\nto evaluate a variety of embodied agents.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.09197v3",
            "date": "2018-05-23 14:38:39+00:00",
            "title": "ASR-based Features for Emotion Recognition: A Transfer Learning Approach",
            "authors": [
                "No\u00e9 Tits",
                "Kevin El Haddad",
                "Thierry Dutoit"
            ],
            "abstract": "During the last decade, the applications of signal processing have\ndrastically improved with deep learning. However areas of affecting computing\nsuch as emotional speech synthesis or emotion recognition from spoken language\nremains challenging. In this paper, we investigate the use of a neural\nAutomatic Speech Recognition (ASR) as a feature extractor for emotion\nrecognition. We show that these features outperform the eGeMAPS feature set to\npredict the valence and arousal emotional dimensions, which means that the\naudio-to-text mapping learning by the ASR system contain information related to\nthe emotional dimensions in spontaneous speech. We also examine the\nrelationship between first layers (closer to speech) and last layers (closer to\ntext) of the ASR and valence/arousal.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.09208v2",
            "date": "2018-05-23 14:55:39+00:00",
            "title": "Pushing the bounds of dropout",
            "authors": [
                "G\u00e1bor Melis",
                "Charles Blundell",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Karl Moritz Hermann",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "We show that dropout training is best understood as performing MAP estimation\nconcurrently for a family of conditional models whose objectives are themselves\nlower bounded by the original dropout objective. This discovery allows us to\npick any model from this family after training, which leads to a substantial\nimprovement on regularisation-heavy language modelling. The family includes\nmodels that compute a power mean over the sampled dropout masks, and their less\nstochastic subvariants with tighter and higher lower bounds than the fully\nstochastic dropout objective. We argue that since the deterministic\nsubvariant's bound is equal to its objective, and the highest amongst these\nmodels, the predominant view of it as a good approximation to MC averaging is\nmisleading. Rather, deterministic dropout is the best available approximation\nto the true objective.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1805.09355v1",
            "date": "2018-05-23 18:03:40+00:00",
            "title": "Scoring Lexical Entailment with a Supervised Directional Similarity Network",
            "authors": [
                "Marek Rei",
                "Daniela Gerz",
                "Ivan Vuli\u0107"
            ],
            "abstract": "We present the Supervised Directional Similarity Network (SDSN), a novel\nneural architecture for learning task-specific transformation functions on top\nof general-purpose word embeddings. Relying on only a limited amount of\nsupervision from task-specific scores on a subset of the vocabulary, our\narchitecture is able to generalise and transform a general-purpose\ndistributional vector space to model the relation of lexical entailment.\nExperiments show excellent performance on scoring graded lexical entailment,\nraising the state-of-the-art on the HyperLex dataset by approximately 25%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09657v4",
            "date": "2018-05-20 10:33:00+00:00",
            "title": "Learning compositionally through attentive guidance",
            "authors": [
                "Dieuwke Hupkes",
                "Anand Singh",
                "Kris Korrel",
                "German Kruszewski",
                "Elia Bruni"
            ],
            "abstract": "While neural network models have been successfully applied to domains that\nrequire substantial generalisation skills, recent studies have implied that\nthey struggle when solving the task they are trained on requires inferring its\nunderlying compositional structure. In this paper, we introduce Attentive\nGuidance, a mechanism to direct a sequence to sequence model equipped with\nattention to find more compositional solutions. We test it on two tasks,\ndevised precisely to assess the compositional capabilities of neural models,\nand we show that vanilla sequence to sequence models with attention overfit the\ntraining distribution, while the guided versions come up with compositional\nsolutions that fit the training and testing distributions almost equally well.\nMoreover, the learned solutions generalise even in cases where the training and\ntesting distributions strongly diverge. In this way, we demonstrate that\nsequence to sequence models are capable of finding compositional solutions\nwithout requiring extra components. These results helps to disentangle the\ncauses for the lack of systematic compositionality in neural networks, which\ncan in turn fuel future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09772v2",
            "date": "2018-04-27 15:33:50+00:00",
            "title": "Auto-Detection of Safety Issues in Baby Products",
            "authors": [
                "Graham Bleaney",
                "Matthew Kuzyk",
                "Julian Man",
                "Hossein Mayanloo",
                "H. R. Tizhoosh"
            ],
            "abstract": "Every year, thousands of people receive consumer product related injuries.\nResearch indicates that online customer reviews can be processed to\nautonomously identify product safety issues. Early identification of safety\nissues can lead to earlier recalls, and thus fewer injuries and deaths. A\ndataset of product reviews from Amazon.com was compiled, along with\n\\emph{SaferProducts.gov} complaints and recall descriptions from the Consumer\nProduct Safety Commission (CPSC) and European Commission Rapid Alert system. A\nsystem was built to clean the collected text and to extract relevant features.\nDimensionality reduction was performed by computing feature relevance through a\nRandom Forest and discarding features with low information gain. Various\nclassifiers were analyzed, including Logistic Regression, SVMs,\nNa{\\\"i}ve-Bayes, Random Forests, and an Ensemble classifier. Experimentation\nwith various features and classifier combinations resulted in a logistic\nregression model with 66\\% precision in the top 50 reviews surfaced. This\nclassifier outperforms all benchmarks set by related literature and consumer\nproduct safety professionals.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.09780v1",
            "date": "2018-05-24 16:58:24+00:00",
            "title": "Mining Procedures from Technical Support Documents",
            "authors": [
                "Abhirut Gupta",
                "Abhay Khosla",
                "Gautam Singh",
                "Gargi Dasgupta"
            ],
            "abstract": "Guided troubleshooting is an inherent task in the domain of technical support\nservices. When a customer experiences an issue with the functioning of a\ntechnical service or a product, an expert user helps guide the customer through\na set of steps comprising a troubleshooting procedure. The objective is to\nidentify the source of the problem through a set of diagnostic steps and\nobservations, and arrive at a resolution. Procedures containing these set of\ndiagnostic steps and observations in response to different problems are common\nartifacts in the body of technical support documentation. The ability to use\nmachine learning and linguistics to understand and leverage these procedures\nfor applications like intelligent chatbots or robotic process automation, is\ncrucial. Existing research on question answering or intelligent chatbots does\nnot look within procedures or deep-understand them. In this paper, we outline a\nsystem for mining procedures from technical support documents. We create models\nfor solving important subproblems like extraction of procedures, identifying\ndecision points within procedures, identifying blocks of instructions\ncorresponding to these decision points and mapping instructions within a\ndecision block. We also release a dataset containing our manual annotations on\npublicly available support documents, to promote further research on the\nproblem.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1805.09843v1",
            "date": "2018-05-24 18:27:21+00:00",
            "title": "Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms",
            "authors": [
                "Dinghan Shen",
                "Guoyin Wang",
                "Wenlin Wang",
                "Martin Renqiang Min",
                "Qinliang Su",
                "Yizhe Zhang",
                "Chunyuan Li",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "Many deep learning architectures have been proposed to model the\ncompositionality in text sequences, requiring a substantial number of\nparameters and expensive computations. However, there has not been a rigorous\nevaluation regarding the added value of sophisticated compositional functions.\nIn this paper, we conduct a point-by-point comparative study between Simple\nWord-Embedding-based Models (SWEMs), consisting of parameter-free pooling\noperations, relative to word-embedding-based RNN/CNN models. Surprisingly,\nSWEMs exhibit comparable or even superior performance in the majority of cases\nconsidered. Based upon this understanding, we propose two additional pooling\nstrategies over learned word embeddings: (i) a max-pooling operation for\nimproved interpretability; and (ii) a hierarchical pooling operation, which\npreserves spatial (n-gram) information within text sequences. We present\nexperiments on 17 datasets encompassing three tasks: (i) (long) document\nclassification; (ii) text sequence matching; and (iii) short text tasks,\nincluding classification and tagging. The source code and datasets can be\nobtained from https:// github.com/dinghanshen/SWEM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09906v2",
            "date": "2018-05-24 21:24:14+00:00",
            "title": "Diffusion Maps for Textual Network Embedding",
            "authors": [
                "Xinyuan Zhang",
                "Yitong Li",
                "Dinghan Shen",
                "Lawrence Carin"
            ],
            "abstract": "Textual network embedding leverages rich text information associated with the\nnetwork to learn low-dimensional vectorial representations of vertices. Rather\nthan using typical natural language processing (NLP) approaches, recent\nresearch exploits the relationship of texts on the same edge to graphically\nembed text. However, these models neglect to measure the complete level of\nconnectivity between any two texts in the graph. We present diffusion maps for\ntextual network embedding (DMTE), integrating global structural information of\nthe graph to capture the semantic relatedness between texts, with a\ndiffusion-convolution operation applied on the text inputs. In addition, a new\nobjective function is designed to efficiently preserve the high-order proximity\nusing the graph diffusion. Experimental results show that the proposed approach\noutperforms state-of-the-art methods on the vertex-classification and\nlink-prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10547v1",
            "date": "2018-05-26 22:02:05+00:00",
            "title": "Using Syntax to Ground Referring Expressions in Natural Images",
            "authors": [
                "Volkan Cirik",
                "Taylor Berg-Kirkpatrick",
                "Louis-Philippe Morency"
            ],
            "abstract": "We introduce GroundNet, a neural network for referring expression recognition\n-- the task of localizing (or grounding) in an image the object referred to by\na natural language expression. Our approach to this task is the first to rely\non a syntactic analysis of the input referring expression in order to inform\nthe structure of the computation graph. Given a parse tree for an input\nexpression, we explicitly map the syntactic constituents and relationships\npresent in the tree to a composed graph of neural modules that defines our\narchitecture for performing localization. This syntax-based approach aids\nlocalization of \\textit{both} the target object and auxiliary supporting\nobjects mentioned in the expression. As a result, GroundNet is more\ninterpretable than previous methods: we can (1) determine which phrase of the\nreferring expression points to which object in the image and (2) track how the\nlocalization of the target object is determined by the network. We study this\nproperty empirically by introducing a new set of annotations on the GoogleRef\ndataset to evaluate localization of supporting objects. Our experiments show\nthat GroundNet achieves state-of-the-art accuracy in identifying supporting\nobjects, while maintaining comparable performance in the localization of target\nobjects.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.10796v1",
            "date": "2018-05-28 07:40:33+00:00",
            "title": "Convolutional neural network compression for natural language processing",
            "authors": [
                "Krzysztof Wr\u00f3bel",
                "Marcin Pietro\u0144",
                "Maciej Wielgosz",
                "Micha\u0142 Karwatowski",
                "Kazimierz Wiatr"
            ],
            "abstract": "Convolutional neural networks are modern models that are very efficient in\nmany classification tasks. They were originally created for image processing\npurposes. Then some trials were performed to use them in different domains like\nnatural language processing. The artificial intelligence systems (like humanoid\nrobots) are very often based on embedded systems with constraints on memory,\npower consumption etc. Therefore convolutional neural network because of its\nmemory capacity should be reduced to be mapped to given hardware. In this\npaper, results are presented of compressing the efficient convolutional neural\nnetworks for sentiment analysis. The main steps are quantization and pruning\nprocesses. The method responsible for mapping compressed network to FPGA and\nresults of this implementation are presented. The described simulations showed\nthat 5-bit width is enough to have no drop in accuracy from floating point\nversion of the network. Additionally, significant memory footprint reduction\nwas achieved (from 85% up to 93%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.10799v1",
            "date": "2018-05-28 07:52:42+00:00",
            "title": "Interactive Text2Pickup Network for Natural Language based Human-Robot Collaboration",
            "authors": [
                "Hyemin Ahn",
                "Sungjoon Choi",
                "Nuri Kim",
                "Geonho Cha",
                "Songhwai Oh"
            ],
            "abstract": "In this paper, we propose the Interactive Text2Pickup (IT2P) network for\nhuman-robot collaboration which enables an effective interaction with a human\nuser despite the ambiguity in user's commands. We focus on the task where a\nrobot is expected to pick up an object instructed by a human, and to interact\nwith the human when the given instruction is vague. The proposed network\nunderstands the command from the human user and estimates the position of the\ndesired object first. To handle the inherent ambiguity in human language\ncommands, a suitable question which can resolve the ambiguity is generated. The\nuser's answer to the question is combined with the initial command and given\nback to the network, resulting in more accurate estimation. The experiment\nresults show that given unambiguous commands, the proposed method can estimate\nthe position of the requested object with an accuracy of 98.49% based on our\ntest dataset. Given ambiguous language commands, we show that the accuracy of\nthe pick up task increases by 1.94 times after incorporating the information\nobtained from the interaction.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1805.10844v1",
            "date": "2018-05-28 09:49:56+00:00",
            "title": "A Stochastic Decoder for Neural Machine Translation",
            "authors": [
                "Philip Schulz",
                "Wilker Aziz",
                "Trevor Cohn"
            ],
            "abstract": "The process of translation is ambiguous, in that there are typically many\nvalid trans- lations for a given sentence. This gives rise to significant\nvariation in parallel cor- pora, however, most current models of machine\ntranslation do not account for this variation, instead treating the prob- lem\nas a deterministic process. To this end, we present a deep generative model of\nmachine translation which incorporates a chain of latent variables, in order to\nac- count for local lexical and syntactic varia- tion in parallel corpora. We\nprovide an in- depth analysis of the pitfalls encountered in variational\ninference for training deep generative models. Experiments on sev- eral\ndifferent language pairs demonstrate that the model consistently improves over\nstrong baselines.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1805.11004v1",
            "date": "2018-05-28 16:05:39+00:00",
            "title": "Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation",
            "authors": [
                "Han Guo",
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "An accurate abstractive summary of a document should contain all its salient\ninformation and should be logically entailed by the input document. We improve\nthese important aspects of abstractive summarization via multi-task learning\nwith the auxiliary tasks of question generation and entailment generation,\nwhere the former teaches the summarization model how to look for salient\nquestioning-worthy details, and the latter teaches the model how to rewrite a\nsummary which is a directed-logical subset of the input document. We also\npropose novel multi-task architectures with high-level (semantic)\nlayer-specific sharing across multiple encoder and decoder layers of the three\ntasks, as well as soft-sharing mechanisms (and show performance ablations and\nanalysis examples of each contribution). Overall, we achieve statistically\nsignificant improvements over the state-of-the-art on both the CNN/DailyMail\nand Gigaword datasets, as well as on the DUC-2002 transfer setup. We also\npresent several quantitative and qualitative analysis studies of our model's\nlearned saliency and entailment skills.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11080v1",
            "date": "2018-05-28 17:49:10+00:00",
            "title": "Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting",
            "authors": [
                "Yen-Chun Chen",
                "Mohit Bansal"
            ],
            "abstract": "Inspired by how humans summarize long documents, we propose an accurate and\nfast summarization model that first selects salient sentences and then rewrites\nthem abstractively (i.e., compresses and paraphrases) to generate a concise\noverall summary. We use a novel sentence-level policy gradient method to bridge\nthe non-differentiable computation between these two neural networks in a\nhierarchical way, while maintaining language fluency. Empirically, we achieve\nthe new state-of-the-art on all metrics (including human evaluation) on the\nCNN/Daily Mail dataset, as well as significantly higher abstractiveness scores.\nMoreover, by first operating at the sentence-level and then the word-level, we\nenable parallel decoding of our neural generative model that results in\nsubstantially faster (10-20x) inference speed as well as 4x faster training\nconvergence than previous long-paragraph encoder-decoder models. We also\ndemonstrate the generalization of our model on the test-only DUC-2002 dataset,\nwhere we achieve higher scores than a state-of-the-art model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11154v2",
            "date": "2018-05-25 13:34:52+00:00",
            "title": "Refining Source Representations with Relation Networks for Neural Machine Translation",
            "authors": [
                "Wen Zhang",
                "Jiawei Hu",
                "Yang Feng",
                "Qun Liu"
            ],
            "abstract": "Although neural machine translation with the encoder-decoder framework has\nachieved great success recently, it still suffers drawbacks of forgetting\ndistant information, which is an inherent disadvantage of recurrent neural\nnetwork structure, and disregarding relationship between source words during\nencoding step. Whereas in practice, the former information and relationship are\noften useful in current step. We target on solving these problems and thus\nintroduce relation networks to learn better representations of the source. The\nrelation networks are able to facilitate memorization capability of recurrent\nneural network via associating source words with each other, this would also\nhelp retain their relationships. Then the source representations and all the\nrelations are fed into the attention component together while decoding, with\nthe main encoder-decoder framework unchanged. Experiments on several datasets\nshow that our method can improve the translation performance significantly over\nthe conventional encoder-decoder model and even outperform the approach\ninvolving supervised syntactic knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11222v1",
            "date": "2018-05-29 02:35:15+00:00",
            "title": "Unsupervised Alignment of Embeddings with Wasserstein Procrustes",
            "authors": [
                "Edouard Grave",
                "Armand Joulin",
                "Quentin Berthet"
            ],
            "abstract": "We consider the task of aligning two sets of points in high dimension, which\nhas many applications in natural language processing and computer vision. As an\nexample, it was recently shown that it is possible to infer a bilingual\nlexicon, without supervised data, by aligning word embeddings trained on\nmonolingual data. These recent advances are based on adversarial training to\nlearn the mapping between the two embeddings. In this paper, we propose to use\nan alternative formulation, based on the joint estimation of an orthogonal\nmatrix and a permutation matrix. While this problem is not convex, we propose\nto initialize our optimization algorithm by using a convex relaxation,\ntraditionally considered for the graph isomorphism problem. We propose a\nstochastic algorithm to minimize our cost function on large scale problems.\nFinally, we evaluate our method on the problem of unsupervised word\ntranslation, by aligning word embeddings trained on monolingual data. On this\ntask, our method obtains state of the art results, while requiring less\ncomputational resources than competing approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1805.11350v1",
            "date": "2018-05-29 10:41:08+00:00",
            "title": "Fully Statistical Neural Belief Tracking",
            "authors": [
                "Nikola Mrk\u0161i\u0107",
                "Ivan Vuli\u0107"
            ],
            "abstract": "This paper proposes an improvement to the existing data-driven Neural Belief\nTracking (NBT) framework for Dialogue State Tracking (DST). The existing NBT\nmodel uses a hand-crafted belief state update mechanism which involves an\nexpensive manual retuning step whenever the model is deployed to a new dialogue\ndomain. We show that this update mechanism can be learned jointly with the\nsemantic decoding and context modelling parts of the NBT model, eliminating the\nlast rule-based module from this DST framework. We propose two different\nstatistical update mechanisms and show that dialogue dynamics can be modelled\nwith a very small number of additional model parameters. In our DST evaluation\nover three languages, we show that this model achieves competitive performance\nand provides a robust framework for building resource-light DST models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11535v1",
            "date": "2018-05-29 15:14:41+00:00",
            "title": "CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation",
            "authors": [
                "Yi Tay",
                "Anh Tuan Luu",
                "Siu Cheung Hui"
            ],
            "abstract": "Dating and romantic relationships not only play a huge role in our personal\nlives but also collectively influence and shape society. Today, many romantic\npartnerships originate from the Internet, signifying the importance of\ntechnology and the web in modern dating. In this paper, we present a text-based\ncomputational approach for estimating the relationship compatibility of two\nusers on social media. Unlike many previous works that propose reciprocal\nrecommender systems for online dating websites, we devise a distant supervision\nheuristic to obtain real world couples from social platforms such as Twitter.\nOur approach, the CoupleNet is an end-to-end deep learning based estimator that\nanalyzes the social profiles of two users and subsequently performs a\nsimilarity match between the users. Intuitively, our approach performs both\nuser profiling and match-making within a unified end-to-end framework.\nCoupleNet utilizes hierarchical recurrent neural models for learning\nrepresentations of user profiles and subsequently coupled attention mechanisms\nto fuse information aggregated from two users. To the best of our knowledge,\nour approach is the first data-driven deep learning approach for our novel\nrelationship recommendation problem. We benchmark our CoupleNet against several\nmachine learning and deep learning baselines. Experimental results show that\nour approach outperforms all approaches significantly in terms of precision.\nQualitative analysis shows that our model is capable of also producing\nexplainable results to users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.11818v1",
            "date": "2018-05-30 06:03:21+00:00",
            "title": "Visual Referring Expression Recognition: What Do Systems Actually Learn?",
            "authors": [
                "Volkan Cirik",
                "Louis-Philippe Morency",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We present an empirical analysis of the state-of-the-art systems for\nreferring expression recognition -- the task of identifying the object in an\nimage referred to by a natural language expression -- with the goal of gaining\ninsight into how these systems reason about language and vision. Surprisingly,\nwe find strong evidence that even sophisticated and linguistically-motivated\nmodels for this task may ignore the linguistic structure, instead relying on\nshallow correlations introduced by unintended biases in the data selection and\nannotation process. For example, we show that a system trained and tested on\nthe input image $\\textit{without the input referring expression}$ can achieve a\nprecision of 71.2% in top-2 predictions. Furthermore, a system that predicts\nonly the object category given the input can achieve a precision of 84.2% in\ntop-2 predictions. These surprisingly positive results for what should be\ndeficient prediction scenarios suggest that careful analysis of what our models\nare learning -- and further, how our data is constructed -- is critical as we\nseek to make substantive progress on grounded language tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12164v3",
            "date": "2018-05-30 18:19:38+00:00",
            "title": "What the Vec? Towards Probabilistically Grounded Embeddings",
            "authors": [
                "Carl Allen",
                "Ivana Bala\u017eevi\u0107",
                "Timothy Hospedales"
            ],
            "abstract": "Word2Vec (W2V) and GloVe are popular, fast and efficient word embedding\nalgorithms. Their embeddings are widely used and perform well on a variety of\nnatural language processing tasks. Moreover, W2V has recently been adopted in\nthe field of graph embedding, where it underpins several leading algorithms.\nHowever, despite their ubiquity and relatively simple model architecture, a\ntheoretical understanding of what the embedding parameters of W2V and GloVe\nlearn and why that is useful in downstream tasks has been lacking. We show that\ndifferent interactions between PMI vectors reflect semantic word relationships,\nsuch as similarity and paraphrasing, that are encoded in low dimensional word\nembeddings under a suitable projection, theoretically explaining why embeddings\nof W2V and GloVe work. As a consequence, we also reveal an interesting\nmathematical interconnection between the considered semantic relationships\nthemselves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12352v2",
            "date": "2018-05-31 07:25:04+00:00",
            "title": "DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder",
            "authors": [
                "Xiaodong Gu",
                "Kyunghyun Cho",
                "Jung-Woo Ha",
                "Sunghun Kim"
            ],
            "abstract": "Variational autoencoders~(VAEs) have shown a promise in data-driven\nconversation modeling. However, most VAE conversation models match the\napproximate posterior distribution over the latent variables to a simple prior\nsuch as standard normal distribution, thereby restricting the generated\nresponses to a relatively simple (e.g., unimodal) scope. In this paper, we\npropose DialogWAE, a conditional Wasserstein autoencoder~(WAE) specially\ndesigned for dialogue modeling. Unlike VAEs that impose a simple distribution\nover the latent variables, DialogWAE models the distribution of data by\ntraining a GAN within the latent variable space. Specifically, our model\nsamples from the prior and posterior distributions over the latent variables by\ntransforming context-dependent random noise using neural networks and minimizes\nthe Wasserstein distance between the two distributions. We further develop a\nGaussian mixture prior network to enrich the latent space. Experiments on two\npopular datasets show that DialogWAE outperforms the state-of-the-art\napproaches in generating more coherent, informative and diverse responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12393v1",
            "date": "2018-05-31 09:39:14+00:00",
            "title": "KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings",
            "authors": [
                "Yuyu Zhang",
                "Hanjun Dai",
                "Kamil Toraman",
                "Le Song"
            ],
            "abstract": "The AI2 Reasoning Challenge (ARC), a new benchmark dataset for question\nanswering (QA) has been recently released. ARC only contains natural science\nquestions authored for human exams, which are hard to answer and require\nadvanced logic reasoning. On the ARC Challenge Set, existing state-of-the-art\nQA systems fail to significantly outperform random baseline, reflecting the\ndifficult nature of this task. In this paper, we propose a novel framework for\nanswering science exam questions, which mimics human solving process in an\nopen-book exam. To address the reasoning challenge, we construct contextual\nknowledge graphs respectively for the question itself and supporting sentences.\nOur model learns to reason with neural embeddings of both knowledge graphs.\nExperiments on the ARC Challenge Set show that our model outperforms the\nprevious state-of-the-art QA systems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.00258v1",
            "date": "2018-06-01 09:54:32+00:00",
            "title": "A Survey of Domain Adaptation for Neural Machine Translation",
            "authors": [
                "Chenhui Chu",
                "Rui Wang"
            ],
            "abstract": "Neural machine translation (NMT) is a deep learning based approach for\nmachine translation, which yields the state-of-the-art translation performance\nin scenarios where large-scale parallel corpora are available. Although the\nhigh-quality and domain-specific translation is crucial in the real world,\ndomain-specific corpora are usually scarce or nonexistent, and thus vanilla NMT\nperforms poorly in such scenarios. Domain adaptation that leverages both\nout-of-domain parallel corpora as well as monolingual corpora for in-domain\ntranslation, is very important for domain-specific translation. In this paper,\nwe give a comprehensive survey of the state-of-the-art domain adaptation\ntechniques for NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00358v2",
            "date": "2018-06-01 14:06:45+00:00",
            "title": "A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset",
            "authors": [
                "Michael Boratko",
                "Harshit Padigela",
                "Divyendra Mikkilineni",
                "Pritish Yuvraj",
                "Rajarshi Das",
                "Andrew McCallum",
                "Maria Chang",
                "Achille Fokoue-Nkoutche",
                "Pavan Kapanipathi",
                "Nicholas Mattei",
                "Ryan Musa",
                "Kartik Talamadupula",
                "Michael Witbrock"
            ],
            "abstract": "The recent work of Clark et al. introduces the AI2 Reasoning Challenge (ARC)\nand the associated ARC dataset that partitions open domain, complex science\nquestions into an Easy Set and a Challenge Set. That paper includes an analysis\nof 100 questions with respect to the types of knowledge and reasoning required\nto answer them; however, it does not include clear definitions of these types,\nnor does it offer information about the quality of the labels. We propose a\ncomprehensive set of definitions of knowledge and reasoning types necessary for\nanswering the questions in the ARC dataset. Using ten annotators and a\nsophisticated annotation interface, we analyze the distribution of labels\nacross the Challenge Set and statistics related to them. Additionally, we\ndemonstrate that although naive information retrieval methods return sentences\nthat are irrelevant to answering the query, sufficient supporting text is often\npresent in the (ARC) corpus. Evaluating with human-selected relevant sentences\nimproves the performance of a neural machine comprehension model by 42 points.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.00512v1",
            "date": "2018-06-01 19:08:30+00:00",
            "title": "Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training",
            "authors": [
                "Maohua Zhu",
                "Jason Clemons",
                "Jeff Pool",
                "Minsoo Rhu",
                "Stephen W. Keckler",
                "Yuan Xie"
            ],
            "abstract": "Exploiting sparsity enables hardware systems to run neural networks faster\nand more energy-efficiently. However, most prior sparsity-centric optimization\ntechniques only accelerate the forward pass of neural networks and usually\nrequire an even longer training process with iterative pruning and retraining.\nWe observe that artificially inducing sparsity in the gradients of the gates in\nan LSTM cell has little impact on the training quality. Further, we can enforce\nstructured sparsity in the gate gradients to make the LSTM backward pass up to\n45% faster than the state-of-the-art dense approach and 168% faster than the\nstate-of-the-art sparsifying method on modern GPUs. Though the structured\nsparsifying method can impact the accuracy of a model, this performance gap can\nbe eliminated by mixing our sparse training method and the standard dense\ntraining method. Experimental results show that the mixed method can achieve\ncomparable results in a shorter time span than using purely dense training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.00588v1",
            "date": "2018-06-02 06:18:15+00:00",
            "title": "Fast Locality Sensitive Hashing for Beam Search on GPU",
            "authors": [
                "Xing Shi",
                "Shizhen Xu",
                "Kevin Knight"
            ],
            "abstract": "We present a GPU-based Locality Sensitive Hashing (LSH) algorithm to speed up\nbeam search for sequence models. We utilize the winner-take-all (WTA) hash,\nwhich is based on relative ranking order of hidden dimensions and thus\nresilient to perturbations in numerical values. Our algorithm is designed by\nfully considering the underling architecture of CUDA-enabled GPUs\n(Algorithm/Architecture Co-design): 1) A parallel Cuckoo hash table is applied\nfor LSH code lookup (guaranteed O(1) lookup time); 2) Candidate lists are\nshared across beams to maximize the parallelism; 3) Top frequent words are\nmerged into candidate lists to improve performance. Experiments on 4\nlarge-scale neural machine translation models demonstrate that our algorithm\ncan achieve up to 4x speedup on softmax module, and 2x overall speedup without\nhurting BLEU on GPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00615v2",
            "date": "2018-06-02 09:58:50+00:00",
            "title": "Multiplex Communities and the Emergence of International Conflict",
            "authors": [
                "Caleb Pomeroy",
                "Niheer Dasandi",
                "Slava Jankin Mikhaylov"
            ],
            "abstract": "Advances in community detection reveal new insights into multiplex and\nmultilayer networks. Less work, however, investigates the relationship between\nthese communities and outcomes in social systems. We leverage these advances to\nshed light on the relationship between the cooperative mesostructure of the\ninternational system and the onset of interstate conflict. We detect\ncommunities based upon weaker signals of affinity expressed in United Nations\nvotes and speeches, as well as stronger signals observed across multiple layers\nof bilateral cooperation. Communities of diplomatic affinity display an\nexpected negative relationship with conflict onset. Ties in communities based\nupon observed cooperation, however, display no effect under a standard model\nspecification and a positive relationship with conflict under an alternative\nspecification. These results align with some extant hypotheses but also point\nto a paucity in our understanding of the relationship between community\nstructure and behavioral outcomes in networks.",
            "journal ref": "PLoS ONE 14(10): e0223040 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00628v1",
            "date": "2018-06-02 12:59:18+00:00",
            "title": "A Novel Framework for Recurrent Neural Networks with Enhancing Information Processing and Transmission between Units",
            "authors": [
                "Xi Chen",
                "Zhihong Deng",
                "Gehui Shen",
                "Ting Huang"
            ],
            "abstract": "This paper proposes a novel framework for recurrent neural networks (RNNs)\ninspired by the human memory models in the field of cognitive neuroscience to\nenhance information processing and transmission between adjacent RNNs' units.\nThe proposed framework for RNNs consists of three stages that is working\nmemory, forget, and long-term store. The first stage includes taking input data\ninto sensory memory and transferring it to working memory for preliminary\ntreatment. And the second stage mainly focuses on proactively forgetting the\nsecondary information rather than the primary in the working memory. And\nfinally, we get the long-term store normally using some kind of RNN's unit. Our\nframework, which is generalized and simple, is evaluated on 6 datasets which\nfall into 3 different tasks, corresponding to text classification, image\nclassification and language modelling. Experiments reveal that our framework\ncan obviously improve the performance of traditional recurrent neural networks.\nAnd exploratory task shows the ability of our framework of correctly forgetting\nthe secondary information.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1806.00738v1",
            "date": "2018-06-03 05:09:54+00:00",
            "title": "Contextualize, Show and Tell: A Neural Visual Storyteller",
            "authors": [
                "Diana Gonzalez-Rico",
                "Gibran Fuentes-Pineda"
            ],
            "abstract": "We present a neural model for generating short stories from image sequences,\nwhich extends the image description model by Vinyals et al. (Vinyals et al.,\n2015). This extension relies on an encoder LSTM to compute a context vector of\neach story from the image sequence. This context vector is used as the first\nstate of multiple independent decoder LSTMs, each of which generates the\nportion of the story corresponding to each image in the sequence by taking the\nimage embedding as the first input. Our model showed competitive results with\nthe METEOR metric and human ratings in the internal track of the Visual\nStorytelling Challenge 2018.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.00778v1",
            "date": "2018-06-03 12:22:28+00:00",
            "title": "Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Attention is typically used to select informative sub-phrases that are used\nfor prediction. This paper investigates the novel use of attention as a form of\nfeature augmentation, i.e, casted attention. We propose Multi-Cast Attention\nNetworks (MCAN), a new attention mechanism and general model architecture for a\npotpourri of ranking tasks in the conversational modeling and question\nanswering domains. Our approach performs a series of soft attention operations,\neach time casting a scalar feature upon the inner word embeddings. The key idea\nis to provide a real-valued hint (feature) to a subsequent encoder layer and is\ntargeted at improving the representation learning process. There are several\nadvantages to this design, e.g., it allows an arbitrary number of attention\nmechanisms to be casted, allowing for multiple attention types (e.g.,\nco-attention, intra-attention) and attention variants (e.g., alignment-pooling,\nmax-pooling, mean-pooling) to be executed simultaneously. This not only\neliminates the costly need to tune the nature of the co-attention layer, but\nalso provides greater extents of explainability to practitioners. Via extensive\nexperiments on four well-known benchmark datasets, we show that MCAN achieves\nstate-of-the-art performance. On the Ubuntu Dialogue Corpus, MCAN outperforms\nexisting state-of-the-art models by $9\\%$. MCAN also achieves the best\nperforming score to date on the well-studied TrecQA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01264v2",
            "date": "2018-06-01 19:41:07+00:00",
            "title": "OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]",
            "authors": [
                "Guineng Zheng",
                "Subhabrata Mukherjee",
                "Xin Luna Dong",
                "Feifei Li"
            ],
            "abstract": "Extraction of missing attribute values is to find values describing an\nattribute of interest from a free text input. Most past related work on\nextraction of missing attribute values work with a closed world assumption with\nthe possible set of values known beforehand, or use dictionaries of values and\nhand-crafted features. How can we discover new attribute values that we have\nnever seen before? Can we do this with limited human annotation or supervision?\nWe study this problem in the context of product catalogs that often have\nmissing values for many attributes of interest.\n  In this work, we leverage product profile information such as titles and\ndescriptions to discover missing values of product attributes. We develop a\nnovel deep tagging model OpenTag for this extraction problem with the following\ncontributions: (1) we formalize the problem as a sequence tagging task, and\npropose a joint model exploiting recurrent neural networks (specifically,\nbidirectional LSTM) to capture context and semantics, and Conditional Random\nFields (CRF) to enforce tagging consistency, (2) we develop a novel attention\nmechanism to provide interpretable explanation for our model's decisions, (3)\nwe propose a novel sampling strategy exploring active learning to reduce the\nburden of human annotation. OpenTag does not use any dictionary or hand-crafted\nfeatures as in prior works. Extensive experiments in real-life datasets in\ndifferent domains show that OpenTag with our active learning strategy discovers\nnew attribute values from as few as 150 annotated samples (reduction in 3.3x\namount of annotation effort) with a high F-score of 83%, outperforming\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01351v2",
            "date": "2018-06-01 06:47:28+00:00",
            "title": "Document Chunking and Learning Objective Generation for Instruction Design",
            "authors": [
                "Khoi-Nguyen Tran",
                "Jey Han Lau",
                "Danish Contractor",
                "Utkarsh Gupta",
                "Bikram Sengupta",
                "Christopher J. Butler",
                "Mukesh Mohania"
            ],
            "abstract": "Instructional Systems Design is the practice of creating of instructional\nexperiences that make the acquisition of knowledge and skill more efficient,\neffective, and appealing. Specifically in designing courses, an hour of\ntraining material can require between 30 to 500 hours of effort in sourcing and\norganizing reference data for use in just the preparation of course material.\nIn this paper, we present the first system of its kind that helps reduce the\neffort associated with sourcing reference material and course creation. We\npresent algorithms for document chunking and automatic generation of learning\nobjectives from content, creating descriptive content metadata to improve\ncontent-discoverability. Unlike existing methods, the learning objectives\ngenerated by our system incorporate pedagogically motivated Bloom's verbs. We\ndemonstrate the usefulness of our methods using real world data from the\nbanking industry and through a live deployment at a large pharmaceutical\ncompany.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01353v1",
            "date": "2018-06-01 12:01:48+00:00",
            "title": "Natural Language Generation for Electronic Health Records",
            "authors": [
                "Scott Lee"
            ],
            "abstract": "A variety of methods existing for generating synthetic electronic health\nrecords (EHRs), but they are not capable of generating unstructured text, like\nemergency department (ED) chief complaints, history of present illness or\nprogress notes. Here, we use the encoder-decoder model, a deep learning\nalgorithm that features in many contemporary machine translation systems, to\ngenerate synthetic chief complaints from discrete variables in EHRs, like age\ngroup, gender, and discharge diagnosis. After being trained end-to-end on\nauthentic records, the model can generate realistic chief complaint text that\npreserves much of the epidemiological information in the original data. As a\nside effect of the model's optimization goal, these synthetic chief complaints\nare also free of relatively uncommon abbreviation and misspellings, and they\ninclude none of the personally-identifiable information (PII) that was in the\ntraining data, suggesting it may be used to support the de-identification of\ntext in EHRs. When combined with algorithms like generative adversarial\nnetworks (GANs), our model could be used to generate fully-synthetic EHRs,\nfacilitating data sharing between healthcare providers and researchers and\nimproving our ability to develop machine learning methods tailored to the\ninformation in healthcare data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.01526v1",
            "date": "2018-06-05 07:36:36+00:00",
            "title": "Leolani: a reference machine with a theory of mind for social communication",
            "authors": [
                "Piek Vossen",
                "Selene Baez",
                "Lenka Baj\u010deti\u0107",
                "Bram Kraaijeveld"
            ],
            "abstract": "Our state of mind is based on experiences and what other people tell us. This\nmay result in conflicting information, uncertainty, and alternative facts. We\npresent a robot that models relativity of knowledge and perception within\nsocial interaction following principles of the theory of mind. We utilized\nvision and speech capabilities on a Pepper robot to build an interaction model\nthat stores the interpretations of perceptions and conversations in combination\nwith provenance on its sources. The robot learns directly from what people tell\nit, possibly in relation to its perception. We demonstrate how the robot's\ncommunication is driven by hunger to acquire more knowledge from and on people\nand objects, to resolve uncertainties and conflicts, and to share awareness of\nthe per- ceived environment. Likewise, the robot can make reference to the\nworld and its knowledge about the world and the encounters with people that\nyielded this knowledge.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.02782v2",
            "date": "2018-06-07 16:53:12+00:00",
            "title": "Training Augmentation with Adversarial Examples for Robust Speech Recognition",
            "authors": [
                "Sining Sun",
                "Ching-Feng Yeh",
                "Mari Ostendorf",
                "Mei-Yuh Hwang",
                "Lei Xie"
            ],
            "abstract": "This paper explores the use of adversarial examples in training speech\nrecognition systems to increase robustness of deep neural network acoustic\nmodels. During training, the fast gradient sign method is used to generate\nadversarial examples augmenting the original training data. Different from\nconventional data augmentation based on data transformations, the examples are\ndynamically generated based on current acoustic model parameters. We assess the\nimpact of adversarial data augmentation in experiments on the Aurora-4 and\nCHiME-4 single-channel tasks, showing improved robustness against noise and\nchannel variation. Further improvement is obtained when combining adversarial\nexamples with teacher/student training, leading to a 23% relative word error\nrate reduction on Aurora-4.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02847v2",
            "date": "2018-06-07 18:13:08+00:00",
            "title": "A Simple Method for Commonsense Reasoning",
            "authors": [
                "Trieu H. Trinh",
                "Quoc V. Le"
            ],
            "abstract": "Commonsense reasoning is a long-standing challenge for deep learning. For\nexample, it is difficult to use neural networks to tackle the Winograd Schema\ndataset (Levesque et al., 2011). In this paper, we present a simple method for\ncommonsense reasoning with neural networks, using unsupervised learning. Key to\nour method is the use of language models, trained on a massive amount of\nunlabled data, to score multiple choice questions posed by commonsense\nreasoning tests. On both Pronoun Disambiguation and Winograd Schema challenges,\nour models outperform previous state-of-the-art methods by a large margin,\nwithout using expensive annotated knowledge bases or hand-engineered features.\nWe train an array of large RNN language models that operate at word or\ncharacter level on LM-1-Billion, CommonCrawl, SQuAD, Gutenberg Books, and a\ncustomized corpus for this task and show that diversity of training data plays\nan important role in test performance. Further analysis also shows that our\nsystem successfully discovers important features of the context that decide the\ncorrect answer, indicating a good grasp of commonsense knowledge.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.02863v1",
            "date": "2018-06-07 18:59:26+00:00",
            "title": "Semi-supervised and Transfer learning approaches for low resource sentiment classification",
            "authors": [
                "Rahul Gupta",
                "Saurabh Sahu",
                "Carol Espy-Wilson",
                "Shrikanth Narayanan"
            ],
            "abstract": "Sentiment classification involves quantifying the affective reaction of a\nhuman to a document, media item or an event. Although researchers have\ninvestigated several methods to reliably infer sentiment from lexical, speech\nand body language cues, training a model with a small set of labeled datasets\nis still a challenge. For instance, in expanding sentiment analysis to new\nlanguages and cultures, it may not always be possible to obtain comprehensive\nlabeled datasets. In this paper, we investigate the application of\nsemi-supervised and transfer learning methods to improve performances on low\nresource sentiment classification tasks. We experiment with extracting dense\nfeature representations, pre-training and manifold regularization in enhancing\nthe performance of sentiment classification systems. Our goal is a coherent\nimplementation of these methods and we evaluate the gains achieved by these\nmethods in matched setting involving training and testing on a single corpus\nsetting as well as two cross corpora settings. In both the cases, our\nexperiments demonstrate that the proposed methods can significantly enhance the\nmodel performance against a purely supervised approach, particularly in cases\ninvolving a handful of training data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1806.02901v1",
            "date": "2018-06-07 20:57:22+00:00",
            "title": "Probabilistic FastText for Multi-Sense Word Embeddings",
            "authors": [
                "Ben Athiwaratkun",
                "Andrew Gordon Wilson",
                "Anima Anandkumar"
            ],
            "abstract": "We introduce Probabilistic FastText, a new model for word embeddings that can\ncapture multiple word senses, sub-word structure, and uncertainty information.\nIn particular, we represent each word with a Gaussian mixture density, where\nthe mean of a mixture component is given by the sum of n-grams. This\nrepresentation allows the model to share statistical strength across sub-word\nstructures (e.g. Latin roots), producing accurate representations of rare,\nmisspelt, or even unseen words. Moreover, each component of the mixture can\ncapture a different word sense. Probabilistic FastText outperforms both\nFastText, which has no probabilistic model, and dictionary-level probabilistic\nembeddings, which do not incorporate subword structures, on several\nword-similarity benchmarks, including English RareWord and foreign language\ndatasets. We also achieve state-of-art performance on benchmarks that measure\nability to discern different meanings. Thus, the proposed model is the first to\nachieve multi-sense representations while having enriched semantics on rare\nwords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.02934v1",
            "date": "2018-06-08 01:18:10+00:00",
            "title": "Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations",
            "authors": [
                "Ashwin Kalyan",
                "Stefan Lee",
                "Anitha Kannan",
                "Dhruv Batra"
            ],
            "abstract": "Many structured prediction problems (particularly in vision and language\ndomains) are ambiguous, with multiple outputs being correct for an input - e.g.\nthere are many ways of describing an image, multiple ways of translating a\nsentence; however, exhaustively annotating the applicability of all possible\noutputs is intractable due to exponentially large output spaces (e.g. all\nEnglish sentences). In practice, these problems are cast as multi-class\nprediction, with the likelihood of only a sparse set of annotations being\nmaximized - unfortunately penalizing for placing beliefs on plausible but\nunannotated outputs. We make and test the following hypothesis - for a given\ninput, the annotations of its neighbors may serve as an additional supervisory\nsignal. Specifically, we propose an objective that transfers supervision from\nneighboring examples. We first study the properties of our developed method in\na controlled toy setup before reporting results on multi-label classification\nand two image-grounded sequence modeling tasks - captioning and question\ngeneration. We evaluate using standard task-specific metrics and measures of\noutput diversity, finding consistent improvements over standard maximum\nlikelihood training and other baselines.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1806.02988v1",
            "date": "2018-06-08 06:57:16+00:00",
            "title": "Towards Binary-Valued Gates for Robust LSTM Training",
            "authors": [
                "Zhuohan Li",
                "Di He",
                "Fei Tian",
                "Wei Chen",
                "Tao Qin",
                "Liwei Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "Long Short-Term Memory (LSTM) is one of the most widely used recurrent\nstructures in sequence modeling. It aims to use gates to control information\nflow (e.g., whether to skip some information or not) in the recurrent\ncomputations, although its practical implementation based on soft gates only\npartially achieves this goal. In this paper, we propose a new way for LSTM\ntraining, which pushes the output values of the gates towards 0 or 1. By doing\nso, we can better control the information flow: the gates are mostly open or\nclosed, instead of in a middle state, which makes the results more\ninterpretable. Empirical studies show that (1) Although it seems that we\nrestrict the model capacity, there is no performance drop: we achieve better or\ncomparable performances due to its better generalization ability; (2) The\noutputs of gates are not sensitive to their inputs: we can easily compress the\nLSTM unit in multiple ways, e.g., low-rank approximation and low-precision\napproximation. The compressed models are even better than the baseline models\nwithout compression.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.03125v1",
            "date": "2018-06-08 12:55:37+00:00",
            "title": "Text Classification based on Word Subspace with Term-Frequency",
            "authors": [
                "Erica K. Shimomoto",
                "Lincon S. Souza",
                "Bernardo B. Gatto",
                "Kazuhiro Fukui"
            ],
            "abstract": "Text classification has become indispensable due to the rapid increase of\ntext in digital form. Over the past three decades, efforts have been made to\napproach this task using various learning algorithms and statistical models\nbased on bag-of-words (BOW) features. Despite its simple implementation, BOW\nfeatures lack semantic meaning representation. To solve this problem, neural\nnetworks started to be employed to learn word vectors, such as the word2vec.\nWord2vec embeds word semantic structure into vectors, where the angle between\nvectors indicates the meaningful similarity between words. To measure the\nsimilarity between texts, we propose the novel concept of word subspace, which\ncan represent the intrinsic variability of features in a set of word vectors.\nThrough this concept, it is possible to model text from word vectors while\nholding semantic information. To incorporate the word frequency directly in the\nsubspace model, we further extend the word subspace to the term-frequency (TF)\nweighted word subspace. Based on these new concepts, text classification can be\nperformed under the mutual subspace method (MSM) framework. The validity of our\nmodeling is shown through experiments on the Reuters text database, comparing\nthe results to various state-of-art algorithms.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1806.03688v1",
            "date": "2018-06-10 16:55:40+00:00",
            "title": "LexNLP: Natural language processing and information extraction for legal and regulatory texts",
            "authors": [
                "Michael J Bommarito II",
                "Daniel Martin Katz",
                "Eric M Detterman"
            ],
            "abstract": "LexNLP is an open source Python package focused on natural language\nprocessing and machine learning for legal and regulatory text. The package\nincludes functionality to (i) segment documents, (ii) identify key text such as\ntitles and section headings, (iii) extract over eighteen types of structured\ninformation like distances and dates, (iv) extract named entities such as\ncompanies and geopolitical entities, (v) transform text into features for model\ntraining, and (vi) build unsupervised and supervised models such as word\nembedding or tagging models. LexNLP includes pre-trained models based on\nthousands of unit tests drawn from real documents available from the SEC EDGAR\ndatabase as well as various judicial and regulatory proceedings. LexNLP is\ndesigned for use in both academic research and industrial applications, and is\ndistributed at https://github.com/LexPredict/lexpredict-lexnlp.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03692v1",
            "date": "2018-06-10 17:05:31+00:00",
            "title": "Deconvolution-Based Global Decoding for Neural Machine Translation",
            "authors": [
                "Junyang Lin",
                "Xu Sun",
                "Xuancheng Ren",
                "Shuming Ma",
                "Jinsong Su",
                "Qi Su"
            ],
            "abstract": "A great proportion of sequence-to-sequence (Seq2Seq) models for Neural\nMachine Translation (NMT) adopt Recurrent Neural Network (RNN) to generate\ntranslation word by word following a sequential order. As the studies of\nlinguistics have proved that language is not linear word sequence but sequence\nof complex structure, translation at each step should be conditioned on the\nwhole target-side context. To tackle the problem, we propose a new NMT model\nthat decodes the sequence with the guidance of its structural prediction of the\ncontext of the target sequence. Our model generates translation based on the\nstructural prediction of the target-side context so that the translation can be\nfreed from the bind of sequential order. Experimental results demonstrate that\nour model is more competitive compared with the state-of-the-art methods, and\nthe analysis reflects that our model is also robust to translating sentences of\ndifferent lengths and it also reduces repetition with the instruction from the\ntarget-side context for decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.03831v1",
            "date": "2018-06-11 06:58:19+00:00",
            "title": "Interactive Visual Grounding of Referring Expressions for Human-Robot Interaction",
            "authors": [
                "Mohit Shridhar",
                "David Hsu"
            ],
            "abstract": "This paper presents INGRESS, a robot system that follows human natural\nlanguage instructions to pick and place everyday objects. The core issue here\nis the grounding of referring expressions: infer objects and their\nrelationships from input images and language expressions. INGRESS allows for\nunconstrained object categories and unconstrained language expressions.\nFurther, it asks questions to disambiguate referring expressions interactively.\nTo achieve these, we take the approach of grounding by generation and propose a\ntwo-stage neural network model for grounding. The first stage uses a neural\nnetwork to generate visual descriptions of objects, compares them with the\ninput language expression, and identifies a set of candidate objects. The\nsecond stage uses another neural network to examine all pairwise relations\nbetween the candidates and infers the most likely referred object. The same\nneural networks are used for both grounding and question generation for\ndisambiguation. Experiments show that INGRESS outperformed a state-of-the-art\nmethod on the RefCOCO dataset and in robot experiments with humans.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1806.04168v1",
            "date": "2018-06-11 18:18:00+00:00",
            "title": "Straight to the Tree: Constituency Parsing with Neural Syntactic Distance",
            "authors": [
                "Yikang Shen",
                "Zhouhan Lin",
                "Athul Paul Jacob",
                "Alessandro Sordoni",
                "Aaron Courville",
                "Yoshua Bengio"
            ],
            "abstract": "In this work, we propose a novel constituency parsing scheme. The model\npredicts a vector of real-valued scalars, named syntactic distances, for each\nsplit position in the input sentence. The syntactic distances specify the order\nin which the split points will be selected, recursively partitioning the input,\nin a top-down fashion. Compared to traditional shift-reduce parsing schemes,\nour approach is free from the potential problem of compounding errors, while\nbeing faster and easier to parallelize. Our model achieves competitive\nperformance amongst single model, discriminative parsers in the PTB dataset and\noutperforms previous models in the CTB dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04458v3",
            "date": "2018-06-12 12:16:54+00:00",
            "title": "Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Prediction",
            "authors": [
                "Artem Sokolov",
                "Julian Hitschler",
                "Mayumi Ohta",
                "Stefan Riezler"
            ],
            "abstract": "Stochastic zeroth-order (SZO), or gradient-free, optimization allows to\noptimize arbitrary functions by relying only on function evaluations under\nparameter perturbations, however, the iteration complexity of SZO methods\nsuffers a factor proportional to the dimensionality of the perturbed function.\nWe show that in scenarios with natural sparsity patterns as in structured\nprediction applications, this factor can be reduced to the expected number of\nactive features over input-output pairs. We give a general proof that applies\nsparse SZO optimization to Lipschitz-continuous, nonconvex, stochastic\nobjectives, and present an experimental evaluation on linear bandit structured\nprediction tasks with sparse word-based feature representations that confirm\nour theoretical results.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1806.04550v2",
            "date": "2018-06-12 14:19:48+00:00",
            "title": "Deep State Space Models for Unconditional Word Generation",
            "authors": [
                "Florian Schmidt",
                "Thomas Hofmann"
            ],
            "abstract": "Autoregressive feedback is considered a necessity for successful\nunconditional text generation using stochastic sequence models. However, such\nfeedback is known to introduce systematic biases into the training process and\nit obscures a principle of generation: committing to global information and\nforgetting local nuances. We show that a non-autoregressive deep state space\nmodel with a clear separation of global and local uncertainty can be built from\nonly two ingredients: An independent noise source and a deterministic\ntransition function. Recent advances on flow-based variational inference can be\nused to train an evidence lower-bound without resorting to annealing, auxiliary\nlosses or similar measures. The result is a highly interpretable generative\nmodel on par with comparable auto-regressive models on the task of word\ngeneration.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.04558v4",
            "date": "2018-06-12 14:29:22+00:00",
            "title": "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis",
            "authors": [
                "Ye Jia",
                "Yu Zhang",
                "Ron J. Weiss",
                "Quan Wang",
                "Jonathan Shen",
                "Fei Ren",
                "Zhifeng Chen",
                "Patrick Nguyen",
                "Ruoming Pang",
                "Ignacio Lopez Moreno",
                "Yonghui Wu"
            ],
            "abstract": "We describe a neural network-based system for text-to-speech (TTS) synthesis\nthat is able to generate speech audio in the voice of many different speakers,\nincluding those unseen during training. Our system consists of three\nindependently trained components: (1) a speaker encoder network, trained on a\nspeaker verification task using an independent dataset of noisy speech from\nthousands of speakers without transcripts, to generate a fixed-dimensional\nembedding vector from seconds of reference speech from a target speaker; (2) a\nsequence-to-sequence synthesis network based on Tacotron 2, which generates a\nmel spectrogram from text, conditioned on the speaker embedding; (3) an\nauto-regressive WaveNet-based vocoder that converts the mel spectrogram into a\nsequence of time domain waveform samples. We demonstrate that the proposed\nmodel is able to transfer the knowledge of speaker variability learned by the\ndiscriminatively-trained speaker encoder to the new task, and is able to\nsynthesize natural speech from speakers that were not seen during training. We\nquantify the importance of training the speaker encoder on a large and diverse\nspeaker set in order to obtain the best generalization performance. Finally, we\nshow that randomly sampled speaker embeddings can be used to synthesize speech\nin the voice of novel speakers dissimilar from those used in training,\nindicating that the model has learned a high quality speaker representation.",
            "journal ref": "Advances in Neural Information Processing Systems 31 (2018),\n  4485-4495",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04841v1",
            "date": "2018-06-13 04:07:22+00:00",
            "title": "A Study of Enhancement, Augmentation, and Autoencoder Methods for Domain Adaptation in Distant Speech Recognition",
            "authors": [
                "Hao Tang",
                "Wei-Ning Hsu",
                "Francois Grondin",
                "James Glass"
            ],
            "abstract": "Speech recognizers trained on close-talking speech do not generalize to\ndistant speech and the word error rate degradation can be as large as 40%\nabsolute. Most studies focus on tackling distant speech recognition as a\nseparate problem, leaving little effort to adapting close-talking speech\nrecognizers to distant speech. In this work, we review several approaches from\na domain adaptation perspective. These approaches, including speech\nenhancement, multi-condition training, data augmentation, and autoencoders, all\ninvolve a transformation of the data between domains. We conduct experiments on\nthe AMI data set, where these approaches can be realized under the same\ncontrolled setting. These approaches lead to different amounts of improvement\nunder their respective assumptions. The purpose of this paper is to quantify\nand characterize the performance gap between the two domains, setting up the\nbasis for studying adaptation of speech recognizers from close-talking speech\nto distant speech. Our results also have implications for improving distant\nspeech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05059v2",
            "date": "2018-06-12 05:13:04+00:00",
            "title": "Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages",
            "authors": [
                "Shiyu Zhou",
                "Shuang Xu",
                "Bo Xu"
            ],
            "abstract": "Sequence-to-sequence attention-based models integrate an acoustic,\npronunciation and language model into a single neural network, which make them\nvery suitable for multilingual automatic speech recognition (ASR). In this\npaper, we are concerned with multilingual speech recognition on low-resource\nlanguages by a single Transformer, one of sequence-to-sequence attention-based\nmodels. Sub-words are employed as the multilingual modeling unit without using\nany pronunciation lexicon. First, we show that a single multilingual ASR\nTransformer performs well on low-resource languages despite of some language\nconfusion. We then look at incorporating language information into the model by\ninserting the language symbol at the beginning or at the end of the original\nsub-words sequence under the condition of language information being known\nduring training. Experiments on CALLHOME datasets demonstrate that the\nmultilingual ASR Transformer with the language symbol at the end performs\nbetter and can obtain relatively 10.5\\% average word error rate (WER) reduction\ncompared to SHL-MLSTM with residual learning. We go on to show that, assuming\nthe language information being known during training and testing, about\nrelatively 12.4\\% average WER reduction can be observed compared to SHL-MLSTM\nwith residual learning through giving the language symbol as the sentence start\ntoken.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1806.05138v1",
            "date": "2018-06-13 16:35:32+00:00",
            "title": "Generative Neural Machine Translation",
            "authors": [
                "Harshil Shah",
                "David Barber"
            ],
            "abstract": "We introduce Generative Neural Machine Translation (GNMT), a latent variable\narchitecture which is designed to model the semantics of the source and target\nsentences. We modify an encoder-decoder translation model by adding a latent\nvariable as a language agnostic representation which is encouraged to learn the\nmeaning of the sentence. GNMT achieves competitive BLEU scores on pure\ntranslation tasks, and is superior when there are missing words in the source\nsentence. We augment the model to facilitate multilingual translation and\nsemi-supervised learning without adding parameters. This framework\nsignificantly reduces overfitting when there is limited paired data available,\nand is effective for translating between pairs of languages not seen during\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05177v1",
            "date": "2018-06-13 10:59:33+00:00",
            "title": "fMRI Semantic Category Decoding using Linguistic Encoding of Word Embeddings",
            "authors": [
                "Subba Reddy Oota",
                "Naresh Manwani",
                "Bapi Raju S"
            ],
            "abstract": "The dispute of how the human brain represents conceptual knowledge has been\nargued in many scientific fields. Brain imaging studies have shown that the\nspatial patterns of neural activation in the brain are correlated with thinking\nabout different semantic categories of words (for example, tools, animals, and\nbuildings) or when viewing the related pictures. In this paper, we present a\ncomputational model that learns to predict the neural activation captured in\nfunctional magnetic resonance imaging (fMRI) data of test words. Unlike the\nmodels with hand-crafted features that have been used in the literature, in\nthis paper we propose a novel approach wherein decoding models are built with\nfeatures extracted from popular linguistic encodings of Word2Vec, GloVe,\nMeta-Embeddings in conjunction with the empirical fMRI data associated with\nviewing several dozen concrete nouns. We compared these models with several\nother models that use word features extracted from FastText, Randomly-generated\nfeatures, Mitchell's 25 features [1]. The experimental results show that the\npredicted fMRI images using Meta-Embeddings meet the state-of-the-art\nperformance. Although models with features from GloVe and Word2Vec predict fMRI\nimages similar to the state-of-the-art model, model with features from\nMeta-Embeddings predicts significantly better. The proposed scheme that uses\npopular linguistic encoding offers a simple and easy approach for semantic\ndecoding from fMRI experiments.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1806.05178v1",
            "date": "2018-06-13 12:57:19+00:00",
            "title": "Generating Sentences Using a Dynamic Canvas",
            "authors": [
                "Harshil Shah",
                "Bowen Zheng",
                "David Barber"
            ],
            "abstract": "We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word\nlevel generative model for natural language. It uses a recurrent neural network\nwith a dynamic attention and canvas memory mechanism to iteratively construct\nsentences. By viewing the state of the memory at intermediate stages and where\nthe model is placing its attention, we gain insight into how it constructs\nsentences. We demonstrate that AUTR learns a meaningful latent representation\nfor each sentence, and achieves competitive log-likelihood lower bounds whilst\nbeing computationally efficient. It is effective at generating and\nreconstructing sentences, as well as imputing missing words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05180v1",
            "date": "2018-06-13 15:38:09+00:00",
            "title": "A Retrospective Analysis of the Fake News Challenge Stance Detection Task",
            "authors": [
                "Andreas Hanselowski",
                "Avinesh PVS",
                "Benjamin Schiller",
                "Felix Caspelherr",
                "Debanjan Chaudhuri",
                "Christian M. Meyer",
                "Iryna Gurevych"
            ],
            "abstract": "The 2017 Fake News Challenge Stage 1 (FNC-1) shared task addressed a stance\nclassification task as a crucial first step towards detecting fake news. To\ndate, there is no in-depth analysis paper to critically discuss FNC-1's\nexperimental setup, reproduce the results, and draw conclusions for\nnext-generation stance classification methods. In this paper, we provide such\nan in-depth analysis for the three top-performing systems. We first find that\nFNC-1's proposed evaluation metric favors the majority class, which can be\neasily classified, and thus overestimates the true discriminative power of the\nmethods. Therefore, we propose a new F1-based metric yielding a changed system\nranking. Next, we compare the features and architectures used, which leads to a\nnovel feature-rich stacked LSTM model that performs on par with the best\nsystems, but is superior in predicting minority classes. To understand the\nmethods' ability to generalize, we derive a new dataset and perform both\nin-domain and cross-domain experiments. Our qualitative and quantitative study\nhelps interpreting the original FNC-1 scores and understand which features help\nimproving performance and why. Our new dataset and all source code used during\nthe reproduction study are publicly available for future research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1806.05559v2",
            "date": "2018-06-13 13:57:13+00:00",
            "title": "Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation",
            "authors": [
                "Francis Gr\u00e9goire",
                "Philippe Langlais"
            ],
            "abstract": "Parallel sentence extraction is a task addressing the data sparsity problem\nfound in multilingual natural language processing applications. We propose a\nbidirectional recurrent neural network based approach to extract parallel\nsentences from collections of multilingual texts. Our experiments with noisy\nparallel corpora show that we can achieve promising results against a\ncompetitive baseline by removing the need of specific feature engineering or\nadditional external resources. To justify the utility of our approach, we\nextract sentence pairs from Wikipedia articles to train machine translation\nsystems and show significant improvements in translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05662v3",
            "date": "2018-06-14 17:41:19+00:00",
            "title": "GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations",
            "authors": [
                "Zhilin Yang",
                "Jake Zhao",
                "Bhuwan Dhingra",
                "Kaiming He",
                "William W. Cohen",
                "Ruslan Salakhutdinov",
                "Yann LeCun"
            ],
            "abstract": "Modern deep transfer learning approaches have mainly focused on learning\ngeneric feature vectors from one task that are transferable to other tasks,\nsuch as word embeddings in language and pretrained convolutional features in\nvision. However, these approaches usually transfer unary features and largely\nignore more structured graphical representations. This work explores the\npossibility of learning generic latent relational graphs that capture\ndependencies between pairs of data units (e.g., words or pixels) from\nlarge-scale unlabeled data and transferring the graphs to downstream tasks. Our\nproposed transfer learning framework improves performance on various tasks\nincluding question answering, natural language inference, sentiment analysis,\nand image classification. We also show that the learned graphs are generic\nenough to be transferred to different embeddings on which the graphs have not\nbeen trained (including GloVe embeddings, ELMo embeddings, and task-specific\nRNN hidden unit), or embedding-free units such as image pixels.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.05740v2",
            "date": "2018-06-14 20:48:41+00:00",
            "title": "Using Search Queries to Understand Health Information Needs in Africa",
            "authors": [
                "Rediet Abebe",
                "Shawndra Hill",
                "Jennifer Wortman Vaughan",
                "Peter M. Small",
                "H. Andrew Schwartz"
            ],
            "abstract": "The lack of comprehensive, high-quality health data in developing nations\ncreates a roadblock for combating the impacts of disease. One key challenge is\nunderstanding the health information needs of people in these nations. Without\nunderstanding people's everyday needs, concerns, and misconceptions, health\norganizations and policymakers lack the ability to effectively target education\nand programming efforts. In this paper, we propose a bottom-up approach that\nuses search data from individuals to uncover and gain insight into health\ninformation needs in Africa. We analyze Bing searches related to HIV/AIDS,\nmalaria, and tuberculosis from all 54 African nations. For each disease, we\nautomatically derive a set of common search themes or topics, revealing a\nwide-spread interest in various types of information, including disease\nsymptoms, drugs, concerns about breastfeeding, as well as stigma, beliefs in\nnatural cures, and other topics that may be hard to uncover through traditional\nsurveys. We expose the different patterns that emerge in health information\nneeds by demographic groups (age and sex) and country. We also uncover\ndiscrepancies in the quality of content returned by search engines to users by\ntopic. Combined, our results suggest that search data can help illuminate\nhealth information needs in Africa and inform discussions on health policy and\ntargeted education efforts both on- and offline.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1806.06176v3",
            "date": "2018-06-16 03:48:50+00:00",
            "title": "Learning Factorized Multimodal Representations",
            "authors": [
                "Yao-Hung Hubert Tsai",
                "Paul Pu Liang",
                "Amir Zadeh",
                "Louis-Philippe Morency",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Learning multimodal representations is a fundamentally complex research\nproblem due to the presence of multiple heterogeneous sources of information.\nAlthough the presence of multiple modalities provides additional valuable\ninformation, there are two key challenges to address when learning from\nmultimodal data: 1) models must learn the complex intra-modal and cross-modal\ninteractions for prediction and 2) models must be robust to unexpected missing\nor noisy modalities during testing. In this paper, we propose to optimize for a\njoint generative-discriminative objective across multimodal data and labels. We\nintroduce a model that factorizes representations into two sets of independent\nfactors: multimodal discriminative and modality-specific generative factors.\nMultimodal discriminative factors are shared across all modalities and contain\njoint multimodal features required for discriminative tasks such as sentiment\nprediction. Modality-specific generative factors are unique for each modality\nand contain the information required for generating data. Experimental results\nshow that our model is able to learn meaningful multimodal representations that\nachieve state-of-the-art or competitive performance on six multimodal datasets.\nOur model demonstrates flexible generative capabilities by conditioning on\nindependent factors and can reconstruct missing modalities without\nsignificantly impacting performance. Lastly, we interpret our factorized\nrepresentations to understand the interactions that influence multimodal\nlearning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.06208v2",
            "date": "2018-06-16 08:31:06+00:00",
            "title": "Offline Extraction of Indic Regional Language from Natural Scene Image using Text Segmentation and Deep Convolutional Sequence",
            "authors": [
                "Sauradip Nag",
                "Pallab Kumar Ganguly",
                "Sumit Roy",
                "Sourab Jha",
                "Krishna Bose",
                "Abhishek Jha",
                "Kousik Dasgupta"
            ],
            "abstract": "Regional language extraction from a natural scene image is always a\nchallenging proposition due to its dependence on the text information extracted\nfrom Image. Text Extraction on the other hand varies on different lighting\ncondition, arbitrary orientation, inadequate text information, heavy background\ninfluence over text and change of text appearance. This paper presents a novel\nunified method for tackling the above challenges. The proposed work uses an\nimage correction and segmentation technique on the existing Text Detection\nPipeline an Efficient and Accurate Scene Text Detector (EAST). EAST uses\nstandard PVAnet architecture to select features and non maximal suppression to\ndetect text from image. Text recognition is done using combined architecture of\nMaxOut convolution neural network (CNN) and Bidirectional long short term\nmemory (LSTM) network. After recognizing text using the Deep Learning based\napproach, the native Languages are translated to English and tokenized using\nstandard Text Tokenizers. The tokens that very likely represent a location is\nused to find the Global Positioning System (GPS) coordinates of the location\nand subsequently the regional languages spoken in that location is extracted.\nThe proposed method is tested on a self generated dataset collected from\nGovernment of India dataset and experimented on Standard Dataset to evaluate\nthe performance of the proposed technique. Comparative study with a few\nstate-of-the-art methods on text detection, recognition and extraction of\nregional language from images shows that the proposed method outperforms the\nexisting methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1806.06301v1",
            "date": "2018-06-16 21:46:59+00:00",
            "title": "Biased Embeddings from Wild Data: Measuring, Understanding and Removing",
            "authors": [
                "Adam Sutton",
                "Thomas Lansdall-Welfare",
                "Nello Cristianini"
            ],
            "abstract": "Many modern Artificial Intelligence (AI) systems make use of data embeddings,\nparticularly in the domain of Natural Language Processing (NLP). These\nembeddings are learnt from data that has been gathered \"from the wild\" and have\nbeen found to contain unwanted biases. In this paper we make three\ncontributions towards measuring, understanding and removing this problem. We\npresent a rigorous way to measure some of these biases, based on the use of\nword lists created for social psychology applications; we observe how gender\nbias in occupations reflects actual gender bias in the same occupations in the\nreal world; and finally we demonstrate how a simple projection can\nsignificantly reduce the effects of embedding bias. All this is part of an\nongoing effort to understand how trust can be built into AI systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06342v2",
            "date": "2018-06-17 06:57:30+00:00",
            "title": "Extending Recurrent Neural Aligner for Streaming End-to-End Speech Recognition in Mandarin",
            "authors": [
                "Linhao Dong",
                "Shiyu Zhou",
                "Wei Chen",
                "Bo Xu"
            ],
            "abstract": "End-to-end models have been showing superiority in Automatic Speech\nRecognition (ASR). At the same time, the capacity of streaming recognition has\nbecome a growing requirement for end-to-end models. Following these trends, an\nencoder-decoder recurrent neural network called Recurrent Neural Aligner (RNA)\nhas been freshly proposed and shown its competitiveness on two English ASR\ntasks. However, it is not clear if RNA can be further improved and applied to\nother spoken language. In this work, we explore the applicability of RNA in\nMandarin Chinese and present four effective extensions: In the encoder, we\nredesign the temporal down-sampling and introduce a powerful convolutional\nstructure. In the decoder, we utilize a regularizer to smooth the output\ndistribution and conduct joint training with a language model. On two Mandarin\nChinese conversational telephone speech recognition (MTS) datasets, our\nExtended-RNA obtains promising performance. Particularly, it achieves 27.7%\ncharacter error rate (CER), which is superior to current state-of-the-art\nresult on the popular HKUST task.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1806.06349v1",
            "date": "2018-06-17 08:44:55+00:00",
            "title": "Incorporating Chinese Characters of Words for Lexical Sememe Prediction",
            "authors": [
                "Huiming Jin",
                "Hao Zhu",
                "Zhiyuan Liu",
                "Ruobing Xie",
                "Maosong Sun",
                "Fen Lin",
                "Leyu Lin"
            ],
            "abstract": "Sememes are minimum semantic units of concepts in human languages, such that\neach word sense is composed of one or multiple sememes. Words are usually\nmanually annotated with their sememes by linguists, and form linguistic\ncommon-sense knowledge bases widely used in various NLP tasks. Recently, the\nlexical sememe prediction task has been introduced. It consists of\nautomatically recommending sememes for words, which is expected to improve\nannotation efficiency and consistency. However, existing methods of lexical\nsememe prediction typically rely on the external context of words to represent\nthe meaning, which usually fails to deal with low-frequency and\nout-of-vocabulary words. To address this issue for Chinese, we propose a novel\nframework to take advantage of both internal character information and external\ncontext information of words. We experiment on HowNet, a Chinese sememe\nknowledge base, and demonstrate that our framework outperforms state-of-the-art\nbaselines by a large margin, and maintains a robust performance even for\nlow-frequency words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06513v1",
            "date": "2018-06-18 06:49:00+00:00",
            "title": "Semi-tied Units for Efficient Gating in LSTM and Highway Networks",
            "authors": [
                "Chao Zhang",
                "Philip Woodland"
            ],
            "abstract": "Gating is a key technique used for integrating information from multiple\nsources by long short-term memory (LSTM) models and has recently also been\napplied to other models such as the highway network. Although gating is\npowerful, it is rather expensive in terms of both computation and storage as\neach gating unit uses a separate full weight matrix. This issue can be severe\nsince several gates can be used together in e.g. an LSTM cell. This paper\nproposes a semi-tied unit (STU) approach to solve this efficiency issue, which\nuses one shared weight matrix to replace those in all the units in the same\nlayer. The approach is termed \"semi-tied\" since extra parameters are used to\nseparately scale each of the shared output values. These extra scaling factors\nare associated with the network activation functions and result in the use of\nparameterised sigmoid, hyperbolic tangent, and rectified linear unit functions.\nSpeech recognition experiments using British English multi-genre broadcast data\nshowed that using STUs can reduce the calculation and storage cost by a factor\nof three for highway networks and four for LSTMs, while giving similar word\nerror rates to the original models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06583v1",
            "date": "2018-06-18 10:22:18+00:00",
            "title": "Nonparametric Topic Modeling with Neural Inference",
            "authors": [
                "Xuefei Ning",
                "Yin Zheng",
                "Zhuxi Jiang",
                "Yu Wang",
                "Huazhong Yang",
                "Junzhou Huang"
            ],
            "abstract": "This work focuses on combining nonparametric topic models with Auto-Encoding\nVariational Bayes (AEVB). Specifically, we first propose iTM-VAE, where the\ntopics are treated as trainable parameters and the document-specific topic\nproportions are obtained by a stick-breaking construction. The inference of\niTM-VAE is modeled by neural networks such that it can be computed in a simple\nfeed-forward manner. We also describe how to introduce a hyper-prior into\niTM-VAE so as to model the uncertainty of the prior parameter. Actually, the\nhyper-prior technique is quite general and we show that it can be applied to\nother AEVB based models to alleviate the {\\it collapse-to-prior} problem\nelegantly. Moreover, we also propose HiTM-VAE, where the document-specific\ntopic distributions are generated in a hierarchical manner. HiTM-VAE is even\nmore flexible and can generate topic distributions with better variability.\nExperimental results on 20News and Reuters RCV1-V2 datasets show that the\nproposed models outperform the state-of-the-art baselines significantly. The\nadvantages of the hyper-prior technique and the hierarchical model construction\nare also confirmed by experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06950v1",
            "date": "2018-06-18 23:08:15+00:00",
            "title": "GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking",
            "authors": [
                "Patrick H. Chen",
                "Si Si",
                "Yang Li",
                "Ciprian Chelba",
                "Cho-jui Hsieh"
            ],
            "abstract": "Model compression is essential for serving large deep neural nets on devices\nwith limited resources or applications that require real-time responses. As a\ncase study, a state-of-the-art neural language model usually consists of one or\nmore recurrent layers sandwiched between an embedding layer used for\nrepresenting input tokens and a softmax layer for generating output tokens. For\nproblems with a very large vocabulary size, the embedding and the softmax\nmatrices can account for more than half of the model size. For instance, the\nbigLSTM model achieves state-of- the-art performance on the One-Billion-Word\n(OBW) dataset with around 800k vocabulary, and its word embedding and softmax\nmatrices use more than 6GBytes space, and are responsible for over 90% of the\nmodel parameters. In this paper, we propose GroupReduce, a novel compression\nmethod for neural language models, based on vocabulary-partition (block) based\nlow-rank matrix approximation and the inherent frequency distribution of tokens\n(the power-law distribution of words). The experimental results show our method\ncan significantly outperform traditional compression methods such as low-rank\napproximation and pruning. On the OBW dataset, our method achieved 6.6 times\ncompression rate for the embedding and softmax matrices, and when combined with\nquantization, our method can achieve 26 times compression rate, which\ntranslates to a factor of 12.8 times compression for the entire model with very\nlittle degradation in perplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.06998v1",
            "date": "2018-06-19 01:24:32+00:00",
            "title": "Private Text Classification",
            "authors": [
                "Leif W. Hanlen",
                "Richard Nock",
                "Hanna Suominen",
                "Neil Bacon"
            ],
            "abstract": "Confidential text corpora exist in many forms, but do not allow arbitrary\nsharing. We explore how to use such private corpora using privacy preserving\ntext analytics. We construct typical text processing applications using\nappropriate privacy preservation techniques (including homomorphic encryption,\nRademacher operators and secure computation). We set out the preliminary\nmaterials from Rademacher operators for binary classifiers, and then construct\nbasic text processing approaches to match those binary classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07072v1",
            "date": "2018-06-19 07:14:54+00:00",
            "title": "A New COLD Feature based Handwriting Analysis for Ethnicity/Nationality Identification",
            "authors": [
                "Sauradip Nag",
                "Palaiahnakote Shivakumara",
                "Wu Yirui",
                "Umapada Pal",
                "Tong Lu"
            ],
            "abstract": "Identifying crime for forensic investigating teams when crimes involve people\nof different nationals is challenging. This paper proposes a new method for\nethnicity (nationality) identification based on Cloud of Line Distribution\n(COLD) features of handwriting components. The proposed method, at first,\nexplores tangent angle for the contour pixels in each row and the mean of\nintensity values of each row in an image for segmenting text lines. For\nsegmented text lines, we use tangent angle and direction of base lines to\nremove rule lines in the image. We use polygonal approximation for finding\ndominant points for contours of edge components. Then the proposed method\nconnects the nearest dominant points of every dominant point, which results in\nline segments of dominant point pairs. For each line segment, the proposed\nmethod estimates angle and length, which gives a point in polar domain. For all\nthe line segments, the proposed method generates dense points in polar domain,\nwhich results in COLD distribution. As character component shapes change,\naccording to nationals, the shape of the distribution changes. This observation\nis extracted based on distance from pixels of distribution to Principal Axis of\nthe distribution. Then the features are subjected to an SVM classifier for\nidentifying nationals. Experiments are conducted on a complex dataset, which\nshow the proposed method is effective and outperforms the existing method",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1806.07098v2",
            "date": "2018-06-19 08:32:49+00:00",
            "title": "End-to-End Speech Recognition From the Raw Waveform",
            "authors": [
                "Neil Zeghidour",
                "Nicolas Usunier",
                "Gabriel Synnaeve",
                "Ronan Collobert",
                "Emmanuel Dupoux"
            ],
            "abstract": "State-of-the-art speech recognition systems rely on fixed, hand-crafted\nfeatures such as mel-filterbanks to preprocess the waveform before the training\npipeline. In this paper, we study end-to-end systems trained directly from the\nraw waveform, building on two alternatives for trainable replacements of\nmel-filterbanks that use a convolutional architecture. The first one is\ninspired by gammatone filterbanks (Hoshen et al., 2015; Sainath et al, 2015),\nand the second one by the scattering transform (Zeghidour et al., 2017). We\npropose two modifications to these architectures and systematically compare\nthem to mel-filterbanks, on the Wall Street Journal dataset. The first\nmodification is the addition of an instance normalization layer, which greatly\nimproves on the gammatone-based trainable filterbanks and speeds up the\ntraining of the scattering-based filterbanks. The second one relates to the\nlow-pass filter used in these approaches. These modifications consistently\nimprove performances for both approaches, and remove the need for a careful\ninitialization in scattering-based trainable filterbanks. In particular, we\nshow a consistent improvement in word error rate of the trainable filterbanks\nrelatively to comparable mel-filterbanks. It is the first time end-to-end\nmodels trained from the raw signal significantly outperform mel-filterbanks on\na large vocabulary task under clean recording conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07304v1",
            "date": "2018-06-19 15:21:37+00:00",
            "title": "Dynamic Multi-Level Multi-Task Learning for Sentence Simplification",
            "authors": [
                "Han Guo",
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Sentence simplification aims to improve readability and understandability,\nbased on several operations such as splitting, deletion, and paraphrasing.\nHowever, a valid simplified sentence should also be logically entailed by its\ninput sentence. In this work, we first present a strong pointer-copy mechanism\nbased sequence-to-sequence sentence simplification model, and then improve its\nentailment and paraphrasing capabilities via multi-task learning with related\nauxiliary tasks of entailment and paraphrase generation. Moreover, we propose a\nnovel 'multi-level' layered soft sharing approach where each auxiliary task\nshares different (higher versus lower) level layers of the sentence\nsimplification model, depending on the task's semantic versus lexico-syntactic\nnature. We also introduce a novel multi-armed bandit based training approach\nthat dynamically learns how to effectively switch across tasks during\nmulti-task learning. Experiments on multiple popular datasets demonstrate that\nour model outperforms competitive simplification systems in SARI and FKGL\nautomatic metrics, and human evaluation. Further, we present several ablation\nanalyses on alternative layer sharing methods, soft versus hard sharing,\ndynamic multi-armed bandit sampling approaches, and our model's learned\nentailment and paraphrasing skills.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07407v1",
            "date": "2018-06-19 18:03:33+00:00",
            "title": "Speaker Adapted Beamforming for Multi-Channel Automatic Speech Recognition",
            "authors": [
                "Tobias Menne",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "This paper presents, in the context of multi-channel ASR, a method to adapt a\nmask based, statistically optimal beamforming approach to a speaker of\ninterest. The beamforming vector of the statistically optimal beamformer is\ncomputed by utilizing speech and noise masks, which are estimated by a neural\nnetwork. The proposed adaptation approach is based on the integration of the\nbeamformer, which includes the mask estimation network, and the acoustic model\nof the ASR system. This allows for the propagation of the training error, from\nthe acoustic modeling cost function, all the way through the beamforming\noperation and through the mask estimation network. By using the results of a\nfirst pass recognition and by keeping all other parameters fixed, the mask\nestimation network can therefore be fine tuned by retraining. Utterances of a\nspeaker of interest can thus be used in a two pass approach, to optimize the\nbeamforming for the speech characteristics of that specific speaker. It is\nshown that this approach improves the ASR performance of a state-of-the-art\nmulti-channel ASR system on the CHiME-4 data. Furthermore the effect of the\nadaptation on the estimated speech masks is discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07573v1",
            "date": "2018-06-20 06:54:17+00:00",
            "title": "Extracting News Events from Microblogs",
            "authors": [
                "\u00d8ystein Repp",
                "Heri Ramampiaro"
            ],
            "abstract": "Twitter stream has become a large source of information for many people, but\nthe magnitude of tweets and the noisy nature of its content have made\nharvesting the knowledge from Twitter a challenging task for researchers for a\nlong time. Aiming at overcoming some of the main challenges of extracting the\nhidden information from tweet streams, this work proposes a new approach for\nreal-time detection of news events from the Twitter stream. We divide our\napproach into three steps. The first step is to use a neural network or deep\nlearning to detect news-relevant tweets from the stream. The second step is to\napply a novel streaming data clustering algorithm to the detected news tweets\nto form news events. The third and final step is to rank the detected events\nbased on the size of the event clusters and growth speed of the tweet\nfrequencies. We evaluate the proposed system on a large, publicly available\ncorpus of annotated news events from Twitter. As part of the evaluation, we\ncompare our approach with a related state-of-the-art solution. Overall, our\nexperiments and user-based evaluation show that our approach on detecting\ncurrent (real) news events delivers a state-of-the-art performance.",
            "journal ref": "In Journal of Statistics and Management Systems, 21(4), pp.\n  695-723. Taylor & Francis",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07713v1",
            "date": "2018-06-20 13:21:53+00:00",
            "title": "Using Neural Network for Identifying Clickbaits in Online News Media",
            "authors": [
                "Amin Omidvar",
                "Hui Jiang",
                "Aijun An"
            ],
            "abstract": "Online news media sometimes use misleading headlines to lure users to open\nthe news article. These catchy headlines that attract users but disappointed\nthem at the end, are called Clickbaits. Because of the importance of automatic\nclickbait detection in online medias, lots of machine learning methods were\nproposed and employed to find the clickbait headlines. In this research, a\nmodel using deep learning methods is proposed to find the clickbaits in\nClickbait Challenge 2017's dataset. The proposed model gained the first rank in\nthe Clickbait Challenge 2017 in terms of Mean Squared Error. Also, data\nanalytics and visualization techniques are employed to explore and discover the\nprovided dataset to get more insight from the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.07722v1",
            "date": "2018-06-15 09:29:05+00:00",
            "title": "Stylized innovation: interrogating incrementally available randomised dictionaries",
            "authors": [
                "Paul Kinsler"
            ],
            "abstract": "Inspired by recent work of Fink, Reeves, Palma and Farr (2017) on innovation\nin language, gastronomy, and technology, I study how new symbol discovery\nmanifests itself in terms of additional \"word\" vocabulary being available from\ndictionaries generated from a finite number of symbols. Several distinct\ndictionary generation models are investigated using numerical simulation, with\nemphasis on the scaling of knowledge as dictionary generators and parameters\nare varied, and the role of which order the symbols are discovered in.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.07978v1",
            "date": "2018-06-20 20:37:28+00:00",
            "title": "The Corpus Replication Task",
            "authors": [
                "Tobias Eichinger"
            ],
            "abstract": "In the field of Natural Language Processing (NLP), we revisit the well-known\nword embedding algorithm word2vec. Word embeddings identify words by vectors\nsuch that the words' distributional similarity is captured. Unexpectedly,\nbesides semantic similarity even relational similarity has been shown to be\ncaptured in word embeddings generated by word2vec, whence two questions arise.\nFirstly, which kind of relations are representable in continuous space and\nsecondly, how are relations built. In order to tackle these questions we\npropose a bottom-up point of view. We call generating input text for which\nword2vec outputs target relations solving the Corpus Replication Task. Deeming\ngeneralizations of this approach to any set of relations possible, we expect\nsolving of the Corpus Replication Task to provide partial answers to the\nquestions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.08202v1",
            "date": "2018-06-21 12:35:23+00:00",
            "title": "Metadata Enrichment of Multi-Disciplinary Digital Library: A Semantic-based Approach",
            "authors": [
                "Hussein T. Al-Natsheh",
                "Lucie Martinet",
                "Fabrice Muhlenbach",
                "Fabien Rico",
                "Djamel A. Zighed"
            ],
            "abstract": "In the scientific digital libraries, some papers from different research\ncommunities can be described by community-dependent keywords even if they share\na semantically similar topic. Articles that are not tagged with enough keyword\nvariations are poorly indexed in any information retrieval system which limits\npotentially fruitful exchanges between scientific disciplines. In this paper,\nwe introduce a novel experimentally designed pipeline for multi-label\nsemantic-based tagging developed for open-access metadata digital libraries.\nThe approach starts by learning from a standard scientific categorization and a\nsample of topic tagged articles to find semantically relevant articles and\nenrich its metadata accordingly. Our proposed pipeline aims to enable\nresearchers reaching articles from various disciplines that tend to use\ndifferent terminologies. It allows retrieving semantically relevant articles\ngiven a limited known variation of search terms. In addition to achieving an\naccuracy that is higher than an expanded query based method using a topic\nsynonym set extracted from a semantic network, our experiments also show a\nhigher computational scalability versus other comparable techniques. We created\na new benchmark extracted from the open-access metadata of a scientific digital\nlibrary and published it along with the experiment code to allow further\nresearch in the topic.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08409v2",
            "date": "2018-06-21 19:43:13+00:00",
            "title": "End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features",
            "authors": [
                "Chiori Hori",
                "Huda Alamri",
                "Jue Wang",
                "Gordon Wichern",
                "Takaaki Hori",
                "Anoop Cherian",
                "Tim K. Marks",
                "Vincent Cartillier",
                "Raphael Gontijo Lopes",
                "Abhishek Das",
                "Irfan Essa",
                "Dhruv Batra",
                "Devi Parikh"
            ],
            "abstract": "Dialog systems need to understand dynamic visual scenes in order to have\nconversations with users about the objects and events around them. Scene-aware\ndialog systems for real-world applications could be developed by integrating\nstate-of-the-art technologies from multiple research areas, including:\nend-to-end dialog technologies, which generate system responses using models\ntrained from dialog data; visual question answering (VQA) technologies, which\nanswer questions about images using learned image features; and video\ndescription technologies, in which descriptions/captions are generated from\nvideos using multimodal information. We introduce a new dataset of dialogs\nabout videos of human behaviors. Each dialog is a typed conversation that\nconsists of a sequence of 10 question-and-answer(QA) pairs between two Amazon\nMechanical Turk (AMT) workers. In total, we collected dialogs on roughly 9,000\nvideos. Using this new dataset for Audio Visual Scene-aware dialog (AVSD), we\ntrained an end-to-end conversation model that generates responses in a dialog\nabout a video. Our experiments demonstrate that using multimodal features that\nwere developed for multimodal attention-based video description enhances the\nquality of generated dialog about dynamic scenes (videos). Our dataset, model\ncode and pretrained models will be publicly available for a new Video\nScene-Aware Dialog challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08462v2",
            "date": "2018-06-22 01:11:40+00:00",
            "title": "Stochastic Wasserstein Autoencoder for Probabilistic Sentence Generation",
            "authors": [
                "Hareesh Bahuleyan",
                "Lili Mou",
                "Hao Zhou",
                "Olga Vechtomova"
            ],
            "abstract": "The variational autoencoder (VAE) imposes a probabilistic distribution\n(typically Gaussian) on the latent space and penalizes the Kullback--Leibler\n(KL) divergence between the posterior and prior. In NLP, VAEs are extremely\ndifficult to train due to the problem of KL collapsing to zero. One has to\nimplement various heuristics such as KL weight annealing and word dropout in a\ncarefully engineered manner to successfully train a VAE for text. In this\npaper, we propose to use the Wasserstein autoencoder (WAE) for probabilistic\nsentence generation, where the encoder could be either stochastic or\ndeterministic. We show theoretically and empirically that, in the original WAE,\nthe stochastically encoded Gaussian distribution tends to become a Dirac-delta\nfunction, and we propose a variant of WAE that encourages the stochasticity of\nthe encoder. Experimental results show that the latent space learned by WAE\nexhibits properties of continuity and smoothness as in VAEs, while\nsimultaneously achieving much higher BLEU scores for sentence reconstruction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08621v1",
            "date": "2018-06-22 12:15:35+00:00",
            "title": "Weakly Supervised Training of Speaker Identification Models",
            "authors": [
                "Martin Karu",
                "Tanel Alum\u00e4e"
            ],
            "abstract": "We propose an approach for training speaker identification models in a weakly\nsupervised manner. We concentrate on the setting where the training data\nconsists of a set of audio recordings and the speaker annotation is provided\nonly at the recording level. The method uses speaker diarization to find unique\nspeakers in each recording, and i-vectors to project the speech of each speaker\nto a fixed-dimensional vector. A neural network is then trained to map\ni-vectors to speakers, using a special objective function that allows to\noptimize the model using recording-level speaker labels. We report experiments\non two different real-world datasets. On the VoxCeleb dataset, the method\nprovides 94.6% accuracy on a closed set speaker identification task, surpassing\nthe baseline performance by a large margin. On an Estonian broadcast news\ndataset, the method provides 66% time-weighted speaker identification recall at\n93% precision.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1806.08727v1",
            "date": "2018-06-20 00:30:29+00:00",
            "title": "Jack the Reader - A Machine Reading Framework",
            "authors": [
                "Dirk Weissenborn",
                "Pasquale Minervini",
                "Tim Dettmers",
                "Isabelle Augenstein",
                "Johannes Welbl",
                "Tim Rockt\u00e4schel",
                "Matko Bo\u0161njak",
                "Jeff Mitchell",
                "Thomas Demeester",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "Many Machine Reading and Natural Language Understanding tasks require reading\nsupporting text in order to answer questions. For example, in Question\nAnswering, the supporting text can be newswire or Wikipedia articles; in\nNatural Language Inference, premises can be seen as the supporting text and\nhypotheses as questions. Providing a set of useful primitives operating in a\nsingle framework of related tasks would allow for expressive modelling, and\neasier model comparison and replication. To that end, we present Jack the\nReader (Jack), a framework for Machine Reading that allows for quick model\nprototyping by component reuse, evaluation of new models on existing datasets\nas well as integrating new datasets and applying them on a growing set of\nimplemented baseline models. Jack is currently supporting (but not limited to)\nthree tasks: Question Answering, Natural Language Inference, and Link\nPrediction. It is developed with the aim of increasing research efficiency and\ncode reuse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08730v1",
            "date": "2018-06-20 16:39:26+00:00",
            "title": "The Natural Language Decathlon: Multitask Learning as Question Answering",
            "authors": [
                "Bryan McCann",
                "Nitish Shirish Keskar",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Deep learning has improved performance on many natural language processing\n(NLP) tasks individually. However, general NLP models cannot emerge within a\nparadigm that focuses on the particularities of a single metric, dataset, and\ntask. We introduce the Natural Language Decathlon (decaNLP), a challenge that\nspans ten tasks: question answering, machine translation, summarization,\nnatural language inference, sentiment analysis, semantic role labeling,\nzero-shot relation extraction, goal-oriented dialogue, semantic parsing, and\ncommonsense pronoun resolution. We cast all tasks as question answering over a\ncontext. Furthermore, we present a new Multitask Question Answering Network\n(MQAN) jointly learns all tasks in decaNLP without any task-specific modules or\nparameters in the multitask setting. MQAN shows improvements in transfer\nlearning for machine translation and named entity recognition, domain\nadaptation for sentiment analysis and natural language inference, and zero-shot\ncapabilities for text classification. We demonstrate that the MQAN's\nmulti-pointer-generator decoder is key to this success and performance further\nimproves with an anti-curriculum training strategy. Though designed for\ndecaNLP, MQAN also achieves state of the art results on the WikiSQL semantic\nparsing task in the single-task setting. We also release code for procuring and\nprocessing data, training and evaluating models, and reproducing all\nexperiments for decaNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08748v2",
            "date": "2018-06-22 16:19:46+00:00",
            "title": "Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory",
            "authors": [
                "Heeyoul Choi"
            ],
            "abstract": "Recurrent neural networks (RNNs) have been drawing much attention with great\nsuccess in many applications like speech recognition and neural machine\ntranslation. Long short-term memory (LSTM) is one of the most popular RNN units\nin deep learning applications. LSTM transforms the input and the previous\nhidden states to the next states with the affine transformation, multiplication\noperations and a nonlinear activation function, which makes a good data\nrepresentation for a given task. The affine transformation includes rotation\nand reflection, which change the semantic or syntactic information of\ndimensions in the hidden states. However, considering that a model interprets\nthe output sequence of LSTM over the whole input sequence, the dimensions of\nthe states need to keep the same type of semantic or syntactic information\nregardless of the location in the sequence. In this paper, we propose a simple\nvariant of the LSTM unit, persistent recurrent unit (PRU), where each dimension\nof hidden states keeps persistent information across time, so that the space\nkeeps the same meaning over the whole sequence. In addition, to improve the\nnonlinear transformation power, we add a feedforward layer in the PRU\nstructure. In the experiment, we evaluate our proposed methods with three\ndifferent tasks, and the results confirm that our methods have better\nperformance than the conventional LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.08760v3",
            "date": "2018-06-22 16:39:37+00:00",
            "title": "Combination of Domain Knowledge and Deep Learning for Sentiment Analysis",
            "authors": [
                "Khuong Vo",
                "Dang Pham",
                "Mao Nguyen",
                "Trung Mai",
                "Tho Quan"
            ],
            "abstract": "The emerging technique of deep learning has been widely applied in many\ndifferent areas. However, when adopted in a certain specific domain, this\ntechnique should be combined with domain knowledge to improve efficiency and\naccuracy. In particular, when analyzing the applications of deep learning in\nsentiment analysis, we found that the current approaches are suffering from the\nfollowing drawbacks: (i) the existing works have not paid much attention to the\nimportance of different types of sentiment terms, which is an important concept\nin this area; and (ii) the loss function currently employed does not well\nreflect the degree of error of sentiment misclassification. To overcome such\nproblem, we propose to combine domain knowledge with deep learning. Our\nproposal includes using sentiment scores, learnt by quadratic programming, to\naugment training data; and introducing the penalty matrix for enhancing the\nloss function of cross entropy. When experimented, we achieved a significant\nimprovement in classification results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09010v1",
            "date": "2018-06-23 17:42:32+00:00",
            "title": "Evaluating Gammatone Frequency Cepstral Coefficients with Neural Networks for Emotion Recognition from Speech",
            "authors": [
                "Gabrielle K. Liu"
            ],
            "abstract": "Current approaches to speech emotion recognition focus on speech features\nthat can capture the emotional content of a speech signal. Mel Frequency\nCepstral Coefficients (MFCCs) are one of the most commonly used representations\nfor audio speech recognition and classification. This paper proposes Gammatone\nFrequency Cepstral Coefficients (GFCCs) as a potentially better representation\nof speech signals for emotion recognition. The effectiveness of MFCC and GFCC\nrepresentations are compared and evaluated over emotion and intensity\nclassification tasks with fully connected and recurrent neural network\narchitectures. The results provide evidence that GFCCs outperform MFCCs in\nspeech emotion recognition.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1806.09029v1",
            "date": "2018-06-23 20:02:55+00:00",
            "title": "Improving Text-to-SQL Evaluation Methodology",
            "authors": [
                "Catherine Finegan-Dollak",
                "Jonathan K. Kummerfeld",
                "Li Zhang",
                "Karthik Ramanathan",
                "Sesh Sadasivam",
                "Rui Zhang",
                "Dragomir Radev"
            ],
            "abstract": "To be informative, an evaluation must measure how well systems generalize to\nrealistic unseen data. We identify limitations of and propose improvements to\ncurrent evaluations of text-to-SQL systems. First, we compare human-generated\nand automatically generated questions, characterizing properties of queries\nnecessary for real-world applications. To facilitate evaluation on multiple\ndatasets, we release standardized and improved versions of seven existing\ndatasets and one new text-to-SQL dataset. Second, we show that the current\ndivision of data into training and test sets measures robustness to variations\nin the way questions are asked, but only partially tests how well systems\ngeneralize to new queries; therefore, we propose a complementary dataset split\nfor evaluation of future work. Finally, we demonstrate how the common practice\nof anonymizing variables during evaluation removes an important challenge of\nthe task. Our observations highlight key difficulties, and our methodology\nenables effective measurement of future development.",
            "journal ref": "ACL (2018) 351-360",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09055v2",
            "date": "2018-06-24 00:06:13+00:00",
            "title": "DARTS: Differentiable Architecture Search",
            "authors": [
                "Hanxiao Liu",
                "Karen Simonyan",
                "Yiming Yang"
            ],
            "abstract": "This paper addresses the scalability challenge of architecture search by\nformulating the task in a differentiable manner. Unlike conventional approaches\nof applying evolution or reinforcement learning over a discrete and\nnon-differentiable search space, our method is based on the continuous\nrelaxation of the architecture representation, allowing efficient search of the\narchitecture using gradient descent. Extensive experiments on CIFAR-10,\nImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in\ndiscovering high-performance convolutional architectures for image\nclassification and recurrent architectures for language modeling, while being\norders of magnitude faster than state-of-the-art non-differentiable techniques.\nOur implementation has been made publicly available to facilitate further\nresearch on efficient architecture search algorithms.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.09202v1",
            "date": "2018-06-24 20:02:25+00:00",
            "title": "Balanced News Using Constrained Bandit-based Personalization",
            "authors": [
                "Sayash Kapoor",
                "Vijay Keswani",
                "Nisheeth K. Vishnoi",
                "L. Elisa Celis"
            ],
            "abstract": "We present a prototype for a news search engine that presents balanced\nviewpoints across liberal and conservative articles with the goal of\nde-polarizing content and allowing users to escape their filter bubble. The\nbalancing is done according to flexible user-defined constraints, and leverages\nrecent advances in constrained bandit optimization. We showcase our balanced\nnews feed by displaying it side-by-side with the news feed produced by a\ntraditional (polarized) feed.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1806.09325v1",
            "date": "2018-06-25 08:35:56+00:00",
            "title": "Single-channel Speech Dereverberation via Generative Adversarial Training",
            "authors": [
                "Chenxing Li",
                "Tieqiang Wang",
                "Shuang Xu",
                "Bo Xu"
            ],
            "abstract": "In this paper, we propose a single-channel speech dereverberation system\n(DeReGAT) based on convolutional, bidirectional long short-term memory and deep\nfeed-forward neural network (CBLDNN) with generative adversarial training\n(GAT). In order to obtain better speech quality instead of only minimizing a\nmean square error (MSE), GAT is employed to make the dereverberated speech\nindistinguishable form the clean samples. Besides, our system can deal with\nwide range reverberation and be well adapted to variant environments. The\nexperimental results show that the proposed model outperforms weighted\nprediction error (WPE) and deep neural network-based systems. In addition,\nDeReGAT is extended to an online speech dereverberation scenario, which reports\ncomparable performance with the offline case.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1806.09511v1",
            "date": "2018-06-25 14:57:52+00:00",
            "title": "A Hierarchical Deep Learning Natural Language Parser for Fashion",
            "authors": [
                "Jos\u00e9 Marcelino",
                "Jo\u00e3o Faria",
                "Lu\u00eds Ba\u00eda",
                "Ricardo Gamelas Sousa"
            ],
            "abstract": "This work presents a hierarchical deep learning natural language parser for\nfashion. Our proposal intends not only to recognize fashion-domain entities but\nalso to expose syntactic and morphologic insights. We leverage the usage of an\narchitecture of specialist models, each one for a different task (from parsing\nto entity recognition). Such architecture renders a hierarchical model able to\ncapture the nuances of the fashion language. The natural language parser is\nable to deal with textual ambiguities which are left unresolved by our\ncurrently existing solution. Our empirical results establish a robust baseline,\nwhich justifies the use of hierarchical architectures of deep learning models\nwhile opening new research avenues to explore.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1806.09514v1",
            "date": "2018-06-25 15:01:54+00:00",
            "title": "The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems",
            "authors": [
                "Adaeze Adigwe",
                "No\u00e9 Tits",
                "Kevin El Haddad",
                "Sarah Ostadabbas",
                "Thierry Dutoit"
            ],
            "abstract": "In this paper, we present a database of emotional speech intended to be\nopen-sourced and used for synthesis and generation purpose. It contains data\nfor male and female actors in English and a male actor in French. The database\ncovers 5 emotion classes so it could be suitable to build synthesis and voice\ntransformation systems with the potential to control the emotional dimension in\na continuous way. We show the data's efficiency by building a simple MLP system\nconverting neutral to angry speech style and evaluate it via a CMOS perception\ntest. Even though the system is a very simple one, the test show the efficiency\nof the data which is promising for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09533v1",
            "date": "2018-06-22 15:37:35+00:00",
            "title": "Using NLP on news headlines to predict index trends",
            "authors": [
                "Marc Velay",
                "Fabrice Daniel"
            ],
            "abstract": "This paper attempts to provide a state of the art in trend prediction using\nnews headlines. We present the research done on predicting DJIA trends using\nNatural Language Processing. We will explain the different algorithms we have\nused as well as the various embedding techniques attempted. We rely on\nstatistical and deep learning models in order to extract information from the\ncorpuses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09542v1",
            "date": "2018-06-25 15:54:45+00:00",
            "title": "Mapping Unparalleled Clinical Professional and Consumer Languages with Embedding Alignment",
            "authors": [
                "Wei-Hung Weng",
                "Peter Szolovits"
            ],
            "abstract": "Mapping and translating professional but arcane clinical jargons to consumer\nlanguage is essential to improve the patient-clinician communication.\nResearchers have used the existing biomedical ontologies and consumer health\nvocabulary dictionary to translate between the languages. However, such\napproaches are limited by expert efforts to manually build the dictionary,\nwhich is hard to be generalized and scalable. In this work, we utilized the\nembeddings alignment method for the word mapping between unparalleled clinical\nprofessional and consumer language embeddings. To map semantically similar\nwords in two different word embeddings, we first independently trained word\nembeddings on both the corpus with abundant clinical professional terms and the\nother with mainly healthcare consumer terms. Then, we aligned the embeddings by\nthe Procrustes algorithm. We also investigated the approach with the\nadversarial training with refinement. We evaluated the quality of the alignment\nthrough the similar words retrieval both by computing the model precision and\nas well as judging qualitatively by human. We show that the Procrustes\nalgorithm can be performant for the professional consumer language embeddings\nalignment, whereas adversarial training with refinement may find some relations\nbetween two languages.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.09736v1",
            "date": "2018-06-26 00:12:14+00:00",
            "title": "Computational Analysis of Insurance Complaints: GEICO Case Study",
            "authors": [
                "Amir Karami",
                "Noelle M. Pendergraft"
            ],
            "abstract": "The online environment has provided a great opportunity for insurance\npolicyholders to share their complaints with respect to different services.\nThese complaints can reveal valuable information for insurance companies who\nseek to improve their services; however, analyzing a huge number of online\ncomplaints is a complicated task for human and must involve computational\nmethods to create an efficient process. This research proposes a computational\napproach to characterize the major topics of a large number of online\ncomplaints. Our approach is based on using the topic modeling approach to\ndisclose the latent semantic of complaints. The proposed approach deployed on\nthousands of GEICO negative reviews. Analyzing 1,371 GEICO complaints indicates\nthat there are 30 major complains in four categories: (1) customer service, (2)\ninsurance coverage, paperwork, policy, and reports, (3) legal issues, and (4)\ncosts, estimates, and payments. This research approach can be used in other\napplications to explore a large number of reviews.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1806.09764v2",
            "date": "2018-06-26 02:31:35+00:00",
            "title": "Deep Generative Models with Learnable Knowledge Constraints",
            "authors": [
                "Zhiting Hu",
                "Zichao Yang",
                "Ruslan Salakhutdinov",
                "Xiaodan Liang",
                "Lianhui Qin",
                "Haoye Dong",
                "Eric Xing"
            ],
            "abstract": "The broad set of deep generative models (DGMs) has achieved remarkable\nadvances. However, it is often difficult to incorporate rich structured domain\nknowledge with the end-to-end DGMs. Posterior regularization (PR) offers a\nprincipled framework to impose structured constraints on probabilistic models,\nbut has limited applicability to the diverse DGMs that can lack a Bayesian\nformulation or even explicit density evaluation. PR also requires constraints\nto be fully specified a priori, which is impractical or suboptimal for complex\nknowledge with learnable uncertain parts. In this paper, we establish\nmathematical correspondence between PR and reinforcement learning (RL), and,\nbased on the connection, expand PR to learn constraints as the extrinsic reward\nin RL. The resulting algorithm is model-agnostic to apply to any DGMs, and is\nflexible to adapt arbitrary constraints with the model jointly. Experiments on\nhuman image generation and templated sentence generation show models with\nlearned knowledge constraints by our algorithm greatly improve over base\ngenerative models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.09827v1",
            "date": "2018-06-26 07:50:37+00:00",
            "title": "Unveiling the semantic structure of text documents using paragraph-aware Topic Models",
            "authors": [
                "Sim\u00f3n Roca-Sotelo",
                "Jer\u00f3nimo Arenas-Garc\u00eda"
            ],
            "abstract": "Classic Topic Models are built under the Bag Of Words assumption, in which\nword position is ignored for simplicity. Besides, symmetric priors are\ntypically used in most applications. In order to easily learn topics with\ndifferent properties among the same corpus, we propose a new line of work in\nwhich the paragraph structure is exploited. Our proposal is based on the\nfollowing assumption: in many text document corpora there are formal\nconstraints shared across all the collection, e.g. sections. When this\nassumption is satisfied, some paragraphs may be related to general concepts\nshared by all documents in the corpus, while others would contain the genuine\ndescription of documents. Assuming each paragraph can be semantically more\ngeneral, specific, or hybrid, we look for ways to measure this, transferring\nthis distinction to topics and being able to learn what we call specific and\ngeneral topics. Experiments show that this is a proper methodology to highlight\ncertain paragraphs in structured documents at the same time we learn\ninteresting and more diverse topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.09932v1",
            "date": "2018-06-26 12:05:33+00:00",
            "title": "Text-Independent Speaker Verification Based on Deep Neural Networks and Segmental Dynamic Time Warping",
            "authors": [
                "Mohamed Adel",
                "Mohamed Afify",
                "Akram Gaballah"
            ],
            "abstract": "In this paper we present a new method for text-independent speaker\nverification that combines segmental dynamic time warping (SDTW) and the\nd-vector approach. The d-vectors, generated from a feed forward deep neural\nnetwork trained to distinguish between speakers, are used as features to\nperform alignment and hence calculate the overall distance between the\nenrolment and test utterances.We present results on the NIST 2008 data set for\nspeaker verification where the proposed method outperforms the conventional\ni-vector baseline with PLDA scores and outperforms d-vector approach with local\ndistances based on cosine and PLDA scores. Also score combination with the\ni-vector/PLDA baseline leads to significant gains over both methods.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1806.10478v2",
            "date": "2018-06-27 13:40:49+00:00",
            "title": "Neural Machine Translation for Query Construction and Composition",
            "authors": [
                "Tommaso Soru",
                "Edgard Marx",
                "Andr\u00e9 Valdestilhas",
                "Diego Esteves",
                "Diego Moussallem",
                "Gustavo Publio"
            ],
            "abstract": "Research on question answering with knowledge base has recently seen an\nincreasing use of deep architectures. In this extended abstract, we study the\napplication of the neural machine translation paradigm for question parsing. We\nemploy a sequence-to-sequence model to learn graph patterns in the SPARQL graph\nquery language and their compositions. Instead of inducing the programs through\nquestion-answer pairs, we expect a semi-supervised approach, where alignments\nbetween questions and queries are built through templates. We argue that the\ncoverage of language utterances can be expanded using late notable works in\nnatural language generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11420v1",
            "date": "2018-06-29 14:02:04+00:00",
            "title": "Discourse-Wizard: Discovering Deep Discourse Structure in your Conversation with RNNs",
            "authors": [
                "Chandrakant Bothe",
                "Sven Magg",
                "Cornelius Weber",
                "Stefan Wermter"
            ],
            "abstract": "Spoken language understanding is one of the key factors in a dialogue system,\nand a context in a conversation plays an important role to understand the\ncurrent utterance. In this work, we demonstrate the importance of context\nwithin the dialogue for neural network models through an online web interface\nlive demo. We developed two different neural models: a model that does not use\ncontext and a context-based model. The no-context model classifies dialogue\nacts at an utterance-level whereas the context-based model takes some preceding\nutterances into account. We make these trained neural models available as a\nlive demo called Discourse-Wizard using a modular server architecture. The live\ndemo provides an easy to use interface for conversational analysis and for\ndiscovering deep discourse structures in a conversation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.11532v2",
            "date": "2018-06-29 16:56:07+00:00",
            "title": "TextWorld: A Learning Environment for Text-based Games",
            "authors": [
                "Marc-Alexandre C\u00f4t\u00e9",
                "\u00c1kos K\u00e1d\u00e1r",
                "Xingdi Yuan",
                "Ben Kybartas",
                "Tavian Barnes",
                "Emery Fine",
                "James Moore",
                "Ruo Yu Tao",
                "Matthew Hausknecht",
                "Layla El Asri",
                "Mahmoud Adada",
                "Wendy Tay",
                "Adam Trischler"
            ],
            "abstract": "We introduce TextWorld, a sandbox learning environment for the training and\nevaluation of RL agents on text-based games. TextWorld is a Python library that\nhandles interactive play-through of text games, as well as backend functions\nlike state tracking and reward assignment. It comes with a curated list of\ngames whose features and challenges we have analyzed. More significantly, it\nenables users to handcraft or automatically generate new games. Its generative\nmechanisms give precise control over the difficulty, scope, and language of\nconstructed games, and can be used to relax challenges inherent to commercial\ntext games like partial observability and sparse rewards. By generating sets of\nvaried but similar games, TextWorld can also be used to study generalization\nand transfer learning. We cast text-based games in the Reinforcement Learning\nformalism, use our framework to develop a set of benchmark games, and evaluate\nseveral baseline agents on this set and the curated list.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.00099v2",
            "date": "2018-06-30 00:57:15+00:00",
            "title": "Generating Titles for Web Tables",
            "authors": [
                "Braden Hancock",
                "Hongrae Lee",
                "Cong Yu"
            ],
            "abstract": "Descriptive titles provide crucial context for interpreting tables that are\nextracted from web pages and are a key component of table-based web\napplications. Prior approaches have attempted to produce titles by selecting\nexisting text snippets associated with the table. These approaches, however,\nare limited by their dependence on suitable titles existing a priori. In our\nuser study, we observe that the relevant information for the title tends to be\nscattered across the page, and often--more than 80% of the time--does not\nappear verbatim anywhere in the page. We propose instead the application of a\nsequence-to-sequence neural network model as a more generalizable means of\ngenerating high-quality titles. This is accomplished by extracting many text\nsnippets that have potentially relevant information to the table, encoding them\ninto an input sequence, and using both copy and generation mechanisms in the\ndecoder to balance relevance and readability of the generated title. We\nvalidate this approach with human evaluation on sample web tables and report\nthat while sequence models with only a copy mechanism or only a generation\nmechanism are easily outperformed by simple selection-based baselines, the\nmodel with both capabilities outperforms them all, approaching the quality of\ncrowdsourced titles while training on fewer than ten thousand examples. To the\nbest of our knowledge, the proposed technique is the first to consider text\ngeneration methods for table titles and establishes a new state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00122v1",
            "date": "2018-06-30 04:07:00+00:00",
            "title": "A Constrained Coupled Matrix-Tensor Factorization for Learning Time-evolving and Emerging Topics",
            "authors": [
                "Sanaz Bahargam",
                "Evangelos E. Papalexakis"
            ],
            "abstract": "Topic discovery has witnessed a significant growth as a field of data mining\nat large. In particular, time-evolving topic discovery, where the evolution of\na topic is taken into account has been instrumental in understanding the\nhistorical context of an emerging topic in a dynamic corpus. Traditionally,\ntime-evolving topic discovery has focused on this notion of time. However,\nespecially in settings where content is contributed by a community or a crowd,\nan orthogonal notion of time is the one that pertains to the level of expertise\nof the content creator: the more experienced the creator, the more advanced the\ntopic. In this paper, we propose a novel time-evolving topic discovery method\nwhich, in addition to the extracted topics, is able to identify the evolution\nof that topic over time, as well as the level of difficulty of that topic, as\nit is inferred by the level of expertise of its main contributors. Our method\nis based on a novel formulation of Constrained Coupled Matrix-Tensor\nFactorization, which adopts constraints well-motivated for, and, as we\ndemonstrate, are essential for high-quality topic discovery. We qualitatively\nevaluate our approach using real data from the Physics and also Programming\nStack Exchange forum, and we were able to identify topics of varying levels of\ndifficulty which can be linked to external events, such as the announcement of\ngravitational waves by the LIGO lab in Physics forum. We provide a quantitative\nevaluation of our method by conducting a user study where experts were asked to\njudge the coherence and quality of the extracted topics. Finally, our proposed\nmethod has implications for automatic curriculum design using the extracted\ntopics, where the notion of the level of difficulty is necessary for the proper\nmodeling of prerequisites and advanced concepts.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.00181v1",
            "date": "2018-06-30 14:06:54+00:00",
            "title": "The Historical Significance of Textual Distances",
            "authors": [
                "Ted Underwood"
            ],
            "abstract": "Measuring similarity is a basic task in information retrieval, and now often\na building-block for more complex arguments about cultural change. But do\nmeasures of textual similarity and distance really correspond to evidence about\ncultural proximity and differentiation? To explore that question empirically,\nthis paper compares textual and social measures of the similarities between\ngenres of English-language fiction. Existing measures of textual similarity\n(cosine similarity on tf-idf vectors or topic vectors) are also compared to new\nstrategies that use supervised learning to anchor textual measurement in a\nsocial context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00560v3",
            "date": "2018-07-02 09:34:34+00:00",
            "title": "Weight-importance sparse training in keyword spotting",
            "authors": [
                "Sihao Xue",
                "Zhenyi Ying",
                "Fan Mo",
                "Min Wang",
                "Jue Sun"
            ],
            "abstract": "Large size models are implemented in recently ASR system to deal with complex\nspeech recognition problems. The num- ber of parameters in these models makes\nthem hard to deploy, especially on some resource-short devices such as car\ntablet. Besides this, at most of time, ASR system is used to deal with\nreal-time problem such as keyword spotting (KWS). It is contradictory to the\nfact that large model requires long com- putation time. To deal with this\nproblem, we apply some sparse algo- rithms to reduces number of parameters in\nsome widely used models, Deep Neural Network (DNN) KWS, which requires real\nshort computation time. We can prune more than 90 % even 95% of parameters in\nthe model with tiny effect decline. And the sparse model performs better than\nbaseline models which has same order number of parameters. Besides this, sparse\nalgorithm can lead us to find rational model size au- tomatically for certain\nproblem without concerning choosing an original model size.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.00745v2",
            "date": "2018-07-02 15:35:02+00:00",
            "title": "Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data",
            "authors": [
                "Michael A. Hedderich",
                "Dietrich Klakow"
            ],
            "abstract": "Manually labeled corpora are expensive to create and often not available for\nlow-resource languages or domains. Automatic labeling approaches are an\nalternative way to obtain labeled data in a quicker and cheaper way. However,\nthese labels often contain more errors which can deteriorate a classifier's\nperformance when trained on this data. We propose a noise layer that is added\nto a neural network architecture. This allows modeling the noise and train on a\ncombination of clean and noisy data. We show that in a low-resource NER task we\ncan improve performance by up to 35% by using additional, noisy data and\nhandling the noise.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.00752v1",
            "date": "2018-07-02 15:42:00+00:00",
            "title": "Waveform to Single Sinusoid Regression to Estimate the F0 Contour from Noisy Speech Using Recurrent Deep Neural Networks",
            "authors": [
                "Akihiro Kato",
                "Tomi Kinnunen"
            ],
            "abstract": "The fundamental frequency (F0) represents pitch in speech that determines\nprosodic characteristics of speech and is needed in various tasks for speech\nanalysis and synthesis. Despite decades of research on this topic, F0\nestimation at low signal-to-noise ratios (SNRs) in unexpected noise conditions\nremains difficult. This work proposes a new approach to noise robust F0\nestimation using a recurrent neural network (RNN) trained in a supervised\nmanner. Recent studies employ deep neural networks (DNNs) for F0 tracking as a\nframe-by-frame classification task into quantised frequency states but we\npropose waveform-to-sinusoid regression instead to achieve both noise\nrobustness and accurate estimation with increased frequency resolution.\n  Experimental results with PTDB-TUG corpus contaminated by additive noise\n(NOISEX-92) demonstrate that the proposed method improves gross pitch error\n(GPE) rate and fine pitch error (FPE) by more than 35 % at SNRs between -10 dB\nand +10 dB compared with well-known noise robust F0 tracker, PEFAC.\nFurthermore, the proposed method also outperforms state-of-the-art DNN-based\napproaches by more than 15 % in terms of both FPE and GPE rate over the\npreceding SNR range.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.00818v1",
            "date": "2018-07-02 13:04:52+00:00",
            "title": "Improving part-of-speech tagging via multi-task learning and character-level word representations",
            "authors": [
                "Daniil Anastasyev",
                "Ilya Gusev",
                "Eugene Indenbom"
            ],
            "abstract": "In this paper, we explore the ways to improve POS-tagging using various types\nof auxiliary losses and different word representations. As a baseline, we\nutilized a BiLSTM tagger, which is able to achieve state-of-the-art results on\nthe sequence labelling tasks. We developed a new method for character-level\nword representation using feedforward neural network. Such representation gave\nus better results in terms of speed and performance of the model. We also\napplied a novel technique of pretraining such word representations with\nexisting word vectors. Finally, we designed a new variant of auxiliary loss for\nsequence labelling tasks: an additional prediction of the neighbour labels.\nSuch loss forces a model to learn the dependencies in-side a sequence of labels\nand accelerates the process of training. We test these methods on English and\nRussian languages.",
            "journal ref": "Computational Linguistics and Intellectual Technologies, Papers\n  from the Annual International Conference \"Dialogue\" (2018) Issue 17, 14-27",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.00868v1",
            "date": "2018-07-02 19:47:22+00:00",
            "title": "Exploring End-to-End Techniques for Low-Resource Speech Recognition",
            "authors": [
                "Vladimir Bataev",
                "Maxim Korenevsky",
                "Ivan Medennikov",
                "Alexander Zatvornitskiy"
            ],
            "abstract": "In this work we present simple grapheme-based system for low-resource speech\nrecognition using Babel data for Turkish spontaneous speech (80 hours). We have\ninvestigated different neural network architectures performance, including\nfully-convolutional, recurrent and ResNet with GRU. Different features and\nnormalization techniques are compared as well. We also proposed CTC-loss\nmodification using segmentation during training, which leads to improvement\nwhile decoding with small beam size. Our best model achieved word error rate of\n45.8%, which is the best reported result for end-to-end systems using in-domain\ndata for this task, according to our knowledge.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1807.01337v1",
            "date": "2018-07-03 18:25:44+00:00",
            "title": "COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks",
            "authors": [
                "Piero Molino",
                "Huaixiu Zheng",
                "Yi-Chia Wang"
            ],
            "abstract": "For a company looking to provide delightful user experiences, it is of\nparamount importance to take care of any customer issues. This paper proposes\nCOTA, a system to improve speed and reliability of customer support for end\nusers through automated ticket classification and answers selection for support\nrepresentatives. Two machine learning and natural language processing\ntechniques are demonstrated: one relying on feature engineering (COTA v1) and\nthe other exploiting raw signals through deep learning architectures (COTA v2).\nCOTA v1 employs a new approach that converts the multi-classification task into\na ranking problem, demonstrating significantly better performance in the case\nof thousands of classes. For COTA v2, we propose an Encoder-Combiner-Decoder, a\nnovel deep learning architecture that allows for heterogeneous input and output\nfeature types and injection of prior knowledge through network architecture\nchoices. This paper compares these models and their variants on the task of\nticket classification and answer selection, showing model COTA v2 outperforms\nCOTA v1, and analyzes their inner workings and shortcomings. Finally, an A/B\ntest is conducted in a production setting validating the real-world impact of\nCOTA in reducing issue resolution time by 10 percent without reducing customer\nsatisfaction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.01670v2",
            "date": "2018-07-04 16:38:49+00:00",
            "title": "Encoding Spatial Relations from Natural Language",
            "authors": [
                "Tiago Ramalho",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Frederic Besse",
                "S. M. Ali Eslami",
                "G\u00e1bor Melis",
                "Fabio Viola",
                "Phil Blunsom",
                "Karl Moritz Hermann"
            ],
            "abstract": "Natural language processing has made significant inroads into learning the\nsemantics of words through distributional approaches, however representations\nlearnt via these methods fail to capture certain kinds of information implicit\nin the real world. In particular, spatial relations are encoded in a way that\nis inconsistent with human spatial reasoning and lacking invariance to\nviewpoint changes. We present a system capable of capturing the semantics of\nspatial relations such as behind, left of, etc from natural language. Our key\ncontributions are a novel multi-modal objective based on generating images of\nscenes from their textual descriptions, and a new dataset on which to train it.\nWe demonstrate that internal representations are robust to meaning preserving\ntransformations of descriptions (paraphrase invariance), while viewpoint\ninvariance is an emergent property of the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.01956v1",
            "date": "2018-07-05 12:15:34+00:00",
            "title": "Neural Language Codes for Multilingual Acoustic Models",
            "authors": [
                "Markus M\u00fcller",
                "Sebastian St\u00fcker",
                "Alex Waibel"
            ],
            "abstract": "Multilingual Speech Recognition is one of the most costly AI problems,\nbecause each language (7,000+) and even different accents require their own\nacoustic models to obtain best recognition performance. Even though they all\nuse the same phoneme symbols, each language and accent imposes its own coloring\nor \"twang\". Many adaptive approaches have been proposed, but they require\nfurther training, additional data and generally are inferior to monolingually\ntrained models. In this paper, we propose a different approach that uses a\nlarge multilingual model that is \\emph{modulated} by the codes generated by an\nancillary network that learns to code useful differences between the \"twangs\"\nor human language.\n  We use Meta-Pi networks to have one network (the language code net) gate the\nactivity of neurons in another (the acoustic model nets). Our results show that\nduring recognition multilingual Meta-Pi networks quickly adapt to the proper\nlanguage coloring without retraining or new data, and perform better than\nmonolingually trained networks. The model was evaluated by training acoustic\nmodeling nets and modulating language code nets jointly and optimize them for\nbest recognition performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02314v1",
            "date": "2018-07-06 08:49:56+00:00",
            "title": "JUMPER: Learning When to Make Classification Decisions in Reading",
            "authors": [
                "Xianggen Liu",
                "Lili Mou",
                "Haotian Cui",
                "Zhengdong Lu",
                "Sen Song"
            ],
            "abstract": "In early years, text classification is typically accomplished by\nfeature-based machine learning models; recently, deep neural networks, as a\npowerful learning machine, make it possible to work with raw input as the text\nstands. However, exiting end-to-end neural networks lack explicit\ninterpretation of the prediction. In this paper, we propose a novel framework,\nJUMPER, inspired by the cognitive process of text reading, that models text\nclassification as a sequential decision process. Basically, JUMPER is a neural\nsystem that scans a piece of text sequentially and makes classification\ndecisions at the time it wishes. Both the classification result and when to\nmake the classification are part of the decision process, which is controlled\nby a policy network and trained with reinforcement learning. Experimental\nresults show that a properly trained JUMPER has the following properties: (1)\nIt can make decisions whenever the evidence is enough, therefore reducing total\ntext reading by 30-40% and often finding the key rationale of prediction. (2)\nIt achieves classification accuracy better than or comparable to\nstate-of-the-art models in several benchmark and industrial datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.02322v5",
            "date": "2018-07-06 09:15:05+00:00",
            "title": "Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing",
            "authors": [
                "Chen Liang",
                "Mohammad Norouzi",
                "Jonathan Berant",
                "Quoc Le",
                "Ni Lao"
            ],
            "abstract": "We present Memory Augmented Policy Optimization (MAPO), a simple and novel\nway to leverage a memory buffer of promising trajectories to reduce the\nvariance of policy gradient estimate. MAPO is applicable to deterministic\nenvironments with discrete actions, such as structured prediction and\ncombinatorial optimization tasks. We express the expected return objective as a\nweighted sum of two terms: an expectation over the high-reward trajectories\ninside the memory buffer, and a separate expectation over trajectories outside\nthe buffer. To make an efficient algorithm of MAPO, we propose: (1) memory\nweight clipping to accelerate and stabilize training; (2) systematic\nexploration to discover high-reward trajectories; (3) distributed sampling from\ninside and outside of the memory buffer to scale up training. MAPO improves the\nsample efficiency and robustness of policy gradient, especially on tasks with\nsparse rewards. We evaluate MAPO on weakly supervised program synthesis from\nnatural language (semantic parsing). On the WikiTableQuestions benchmark, we\nimprove the state-of-the-art by 2.6%, achieving an accuracy of 46.3%. On the\nWikiSQL benchmark, MAPO achieves an accuracy of 74.9% with only weak\nsupervision, outperforming several strong baselines with full supervision. Our\nsource code is available at\nhttps://github.com/crazydonkey200/neural-symbolic-machines",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.02340v2",
            "date": "2018-07-06 10:17:44+00:00",
            "title": "Testing Untestable Neural Machine Translation: An Industrial Case",
            "authors": [
                "Wujie Zheng",
                "Wenyu Wang",
                "Dian Liu",
                "Changrong Zhang",
                "Qinsong Zeng",
                "Yuetang Deng",
                "Wei Yang",
                "Pinjia He",
                "Tao Xie"
            ],
            "abstract": "Neural Machine Translation (NMT) has been widely adopted recently due to its\nadvantages compared with the traditional Statistical Machine Translation (SMT).\nHowever, an NMT system still often produces translation failures due to the\ncomplexity of natural language and sophistication in designing neural networks.\nWhile in-house black-box system testing based on reference translations (i.e.,\nexamples of valid translations) has been a common practice for NMT quality\nassurance, an increasingly critical industrial practice, named in-vivo testing,\nexposes unseen types or instances of translation failures when real users are\nusing a deployed industrial NMT system. To fill the gap of lacking test oracle\nfor in-vivo testing of an NMT system, in this paper, we propose a new approach\nfor automatically identifying translation failures, without requiring reference\ntranslations for a translation task; our approach can directly serve as a test\noracle for in-vivo testing. Our approach focuses on properties of natural\nlanguage translation that can be checked systematically and uses information\nfrom both the test inputs (i.e., the texts to be translated) and the test\noutputs (i.e., the translations under inspection) of the NMT system. Our\nevaluation conducted on real-world datasets shows that our approach can\neffectively detect targeted property violations as translation failures. Our\nexperiences on deploying our approach in both production and development\nenvironments of WeChat (a messenger app with over one billion monthly active\nusers) demonstrate high effectiveness of our approach along with high industry\nimpact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.02391v1",
            "date": "2018-07-05 03:34:22+00:00",
            "title": "Extracting Actionable Knowledge from Domestic Violence Discourses on Social Media",
            "authors": [
                "Sudha Subramani",
                "Manjula O'Connor"
            ],
            "abstract": "Domestic Violence (DV) is considered as big social issue and there exists a\nstrong relationship between DV and health impacts of the public. Existing\nresearch studies have focused on social media to track and analyse real world\nevents like emerging trends, natural disasters, user sentiment analysis,\npolitical opinions, and health care. However there is less attention given on\nsocial welfare issues like DV and its impact on public health. Recently, the\nvictims of DV turned to social media platforms to express their feelings in the\nform of posts and seek the social and emotional support, for sympathetic\nencouragement, to show compassion and empathy among public. But, it is\ndifficult to mine the actionable knowledge from large conversational datasets\nfrom social media due to the characteristics of high dimensions, short, noisy,\nhuge volume, high velocity, and so on. Hence, this paper will propose a novel\nframework to model and discover the various themes related to DV from the\npublic domain. The proposed framework would possibly provide unprecedentedly\nvaluable information to the public health researchers, national family health\norganizations, government and public with data enrichment and consolidation to\nimprove the social welfare of the community. Thus provides actionable knowledge\nby monitoring and analysing continuous and rich user generated content.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.02471v1",
            "date": "2018-07-05 07:17:21+00:00",
            "title": "A Review of Different Word Embeddings for Sentiment Classification using Deep Learning",
            "authors": [
                "Debadri Dutta"
            ],
            "abstract": "The web is loaded with textual content, and Natural Language Processing is a\nstandout amongst the most vital fields in Machine Learning. But when data is\nhuge simple Machine Learning algorithms are not able to handle it and that is\nwhen Deep Learning comes into play which based on Neural Networks. However\nsince neural networks cannot process raw text, we have to change over them\nthrough some diverse strategies of word embedding. This paper demonstrates\nthose distinctive word embedding strategies implemented on an Amazon Review\nDataset, which has two sentiments to be classified: Happy and Unhappy based on\nnumerous customer reviews. Moreover we demonstrate the distinction in accuracy\nwith a discourse about which word embedding to apply when.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.02854v1",
            "date": "2018-07-08 17:33:43+00:00",
            "title": "Replicated Siamese LSTM in Ticketing System for Similarity Learning and Retrieval in Asymmetric Texts",
            "authors": [
                "Pankaj Gupta",
                "Bernt Andrassy",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "The goal of our industrial ticketing system is to retrieve a relevant\nsolution for an input query, by matching with historical tickets stored in\nknowledge base. A query is comprised of subject and description, while a\nhistorical ticket consists of subject, description and solution. To retrieve a\nrelevant solution, we use textual similarity paradigm to learn similarity in\nthe query and historical tickets. The task is challenging due to significant\nterm mismatch in the query and ticket pairs of asymmetric lengths, where\nsubject is a short text but description and solution are multi-sentence texts.\nWe present a novel Replicated Siamese LSTM model to learn similarity in\nasymmetric text pairs, that gives 22% and 7% gain (Accuracy@10) for retrieval\ntask, respectively over unsupervised and supervised baselines. We also show\nthat the topic and distributed semantic features for short and long texts\nimproved both similarity learning and retrieval.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.03052v1",
            "date": "2018-07-09 11:34:13+00:00",
            "title": "Position-aware Self-attention with Relative Positional Encodings for Slot Filling",
            "authors": [
                "Ivan Bilan",
                "Benjamin Roth"
            ],
            "abstract": "This paper describes how to apply self-attention with relative positional\nencodings to the task of relation extraction. We propose to use the\nself-attention encoder layer together with an additional position-aware\nattention layer that takes into account positions of the query and the object\nin the sentence. The self-attention encoder also uses a custom implementation\nof relative positional encodings which allow each word in the sentence to take\ninto account its left and right context. The evaluation of the model is done on\nthe TACRED dataset. The proposed model relies only on attention (no recurrent\nor convolutional layers are used), while improving performance w.r.t. the\nprevious state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03367v3",
            "date": "2018-07-09 20:05:24+00:00",
            "title": "Talk the Walk: Navigating New York City through Grounded Dialogue",
            "authors": [
                "Harm de Vries",
                "Kurt Shuster",
                "Dhruv Batra",
                "Devi Parikh",
                "Jason Weston",
                "Douwe Kiela"
            ],
            "abstract": "We introduce \"Talk The Walk\", the first large-scale dialogue dataset grounded\nin action and perception. The task involves two agents (a \"guide\" and a\n\"tourist\") that communicate via natural language in order to achieve a common\ngoal: having the tourist navigate to a given target location. The task and\ndataset, which are described in detail, are challenging and their full solution\nis an open problem that we pose to the community. We (i) focus on the task of\ntourist localization and develop the novel Masked Attention for Spatial\nConvolutions (MASC) mechanism that allows for grounding tourist utterances into\nthe guide's map, (ii) show it yields significant improvements for both emergent\nand natural language communication, and (iii) using this method, we establish\nnon-trivial baselines on the full task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1807.03396v3",
            "date": "2018-07-09 21:31:49+00:00",
            "title": "On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition",
            "authors": [
                "Hao Tang",
                "James Glass"
            ],
            "abstract": "Recurrent neural networks have been the dominant models for many speech and\nlanguage processing tasks. However, we understand little about the behavior and\nthe class of functions recurrent networks can realize. Moreover, the heuristics\nused during training complicate the analyses. In this paper, we study recurrent\nnetworks' ability to learn long-term dependency in the context of speech\nrecognition. We consider two decoding approaches, online and batch decoding,\nand show the classes of functions to which the decoding approaches correspond.\nWe then draw a connection between batch decoding and a popular training\napproach for recurrent networks, truncated backpropagation through time.\nChanging the decoding approach restricts the amount of past history recurrent\nnetworks can use for prediction, allowing us to analyze their ability to\nremember. Empirically, we utilize long-term dependency in subphonetic states,\nphonemes, and words, and show how the design decisions, such as the decoding\napproach, lookahead, context frames, and consecutive prediction, characterize\nthe behavior of recurrent networks. Finally, we draw a connection between\nMarkov processes and vanishing gradients. These results have implications for\nstudying the long-term dependency in speech data and how these properties are\nlearned by recurrent networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03756v2",
            "date": "2018-07-10 16:59:12+00:00",
            "title": "Latent Alignment and Variational Attention",
            "authors": [
                "Yuntian Deng",
                "Yoon Kim",
                "Justin Chiu",
                "Demi Guo",
                "Alexander M. Rush"
            ],
            "abstract": "Neural attention has become central to many state-of-the-art models in\nnatural language processing and related domains. Attention networks are an\neasy-to-train and effective method for softly simulating alignment; however,\nthe approach does not marginalize over latent alignments in a probabilistic\nsense. This property makes it difficult to compare attention to other alignment\napproaches, to compose it with probabilistic models, and to perform posterior\ninference conditioned on observed data. A related latent approach, hard\nattention, fixes these issues, but is generally harder to train and less\naccurate. This work considers variational attention networks, alternatives to\nsoft and hard attention for learning latent variable alignment models, with\ntighter approximation bounds based on amortized variational inference. We\nfurther propose methods for reducing the variance of gradients to make these\napproaches computationally feasible. Experiments show that for machine\ntranslation and visual question answering, inefficient exact latent variable\nmodels outperform standard neural attention, but these gains go away when using\nhard attention based training. On the other hand, variational attention retains\nmost of the performance gain but with training speed comparable to neural\nattention.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1807.03819v3",
            "date": "2018-07-10 18:39:15+00:00",
            "title": "Universal Transformers",
            "authors": [
                "Mostafa Dehghani",
                "Stephan Gouws",
                "Oriol Vinyals",
                "Jakob Uszkoreit",
                "\u0141ukasz Kaiser"
            ],
            "abstract": "Recurrent neural networks (RNNs) sequentially process data by updating their\nstate with each new data point, and have long been the de facto choice for\nsequence modeling tasks. However, their inherently sequential computation makes\nthem slow to train. Feed-forward and convolutional architectures have recently\nbeen shown to achieve superior results on some sequence modeling tasks such as\nmachine translation, with the added advantage that they concurrently process\nall inputs in the sequence, leading to easy parallelization and faster training\ntimes. Despite these successes, however, popular feed-forward sequence models\nlike the Transformer fail to generalize in many simple tasks that recurrent\nmodels handle with ease, e.g. copying strings or even simple logical inference\nwhen the string or formula lengths exceed those observed at training time. We\npropose the Universal Transformer (UT), a parallel-in-time self-attentive\nrecurrent sequence model which can be cast as a generalization of the\nTransformer model and which addresses these issues. UTs combine the\nparallelizability and global receptive field of feed-forward sequence models\nlike the Transformer with the recurrent inductive bias of RNNs. We also add a\ndynamic per-position halting mechanism and find that it improves accuracy on\nseveral tasks. In contrast to the standard Transformer, under certain\nassumptions, UTs can be shown to be Turing-complete. Our experiments show that\nUTs outperform standard Transformers on a wide range of algorithmic and\nlanguage understanding tasks, including the challenging LAMBADA language\nmodeling task where UTs achieve a new state of the art, and machine translation\nwhere UTs achieve a 0.9 BLEU improvement over Transformers on the WMT14 En-De\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03915v2",
            "date": "2018-07-11 01:13:13+00:00",
            "title": "Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis",
            "authors": [
                "Hai Pham",
                "Thomas Manzini",
                "Paul Pu Liang",
                "Barnabas Poczos"
            ],
            "abstract": "Multimodal machine learning is a core research area spanning the language,\nvisual and acoustic modalities. The central challenge in multimodal learning\ninvolves learning representations that can process and relate information from\nmultiple modalities. In this paper, we propose two methods for unsupervised\nlearning of joint multimodal representations using sequence to sequence\n(Seq2Seq) methods: a \\textit{Seq2Seq Modality Translation Model} and a\n\\textit{Hierarchical Seq2Seq Modality Translation Model}. We also explore\nmultiple different variations on the multimodal inputs and outputs of these\nseq2seq models. Our experiments on multimodal sentiment analysis using the\nCMU-MOSI dataset indicate that our methods learn informative multimodal\nrepresentations that outperform the baselines and achieve improved performance\non multimodal sentiment analysis, specifically in the Bimodal case where our\nmodel is able to improve F1 Score by twelve points. We also discuss future\ndirections for multimodal Seq2Seq methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04687v1",
            "date": "2018-07-12 15:53:29+00:00",
            "title": "Making Efficient Use of a Domain Expert's Time in Relation Extraction",
            "authors": [
                "Linara Adilova",
                "Sven Giesselbach",
                "Stefan R\u00fcping"
            ],
            "abstract": "Scarcity of labeled data is one of the most frequent problems faced in\nmachine learning. This is particularly true in relation extraction in text\nmining, where large corpora of texts exists in many application domains, while\nlabeling of text data requires an expert to invest much time to read the\ndocuments. Overall, state-of-the art models, like the convolutional neural\nnetwork used in this paper, achieve great results when trained on large enough\namounts of labeled data. However, from a practical point of view the question\narises whether this is the most efficient approach when one takes the manual\neffort of the expert into account. In this paper, we report on an alternative\napproach where we first construct a relation extraction model using distant\nsupervision, and only later make use of a domain expert to refine the results.\nDistant supervision provides a mean of labeling data given known relations in a\nknowledge base, but it suffers from noisy labeling. We introduce an active\nlearning based extension, that allows our neural network to incorporate expert\nfeedback and report on first results on a complex data set.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.04715v2",
            "date": "2018-07-12 16:43:32+00:00",
            "title": "Orthogonal Matching Pursuit for Text Classification",
            "authors": [
                "Konstantinos Skianis",
                "Nikolaos Tziortziotis",
                "Michalis Vazirgiannis"
            ],
            "abstract": "In text classification, the problem of overfitting arises due to the high\ndimensionality, making regularization essential. Although classic regularizers\nprovide sparsity, they fail to return highly accurate models. On the contrary,\nstate-of-the-art group-lasso regularizers provide better results at the expense\nof low sparsity. In this paper, we apply a greedy variable selection algorithm,\ncalled Orthogonal Matching Pursuit, for the text classification task. We also\nextend standard group OMP by introducing overlapping Group OMP to handle\noverlapping groups of features. Empirical analysis verifies that both OMP and\noverlapping GOMP constitute powerful regularizers, able to produce effective\nand very sparse models. Code and data are available online:\nhttps://github.com/y3nk0/OMP-for-Text-Classification .",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.04863v2",
            "date": "2018-07-12 23:37:27+00:00",
            "title": "Avoiding Latent Variable Collapse With Generative Skip Models",
            "authors": [
                "Adji B. Dieng",
                "Yoon Kim",
                "Alexander M. Rush",
                "David M. Blei"
            ],
            "abstract": "Variational autoencoders learn distributions of high-dimensional data. They\nmodel data with a deep latent-variable model and then fit the model by\nmaximizing a lower bound of the log marginal likelihood. VAEs can capture\ncomplex distributions, but they can also suffer from an issue known as \"latent\nvariable collapse,\" especially if the likelihood model is powerful.\nSpecifically, the lower bound involves an approximate posterior of the latent\nvariables; this posterior \"collapses\" when it is set equal to the prior, i.e.,\nwhen the approximate posterior is independent of the data. While VAEs learn\ngood generative models, latent variable collapse prevents them from learning\nuseful representations. In this paper, we propose a simple new way to avoid\nlatent variable collapse by including skip connections in our generative model;\nthese connections enforce strong links between the latent variables and the\nlikelihood function. We study generative skip models both theoretically and\nempirically. Theoretically, we prove that skip models increase the mutual\ninformation between the observations and the inferred latent variables.\nEmpirically, we study images (MNIST and Omniglot) and text (Yahoo). Compared to\nexisting VAE architectures, we show that generative skip models maintain\nsimilar predictive performance but lead to less collapse and provide more\nmeaningful representations of the data.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1807.04905v1",
            "date": "2018-07-13 04:19:03+00:00",
            "title": "Ultra-Fine Entity Typing",
            "authors": [
                "Eunsol Choi",
                "Omer Levy",
                "Yejin Choi",
                "Luke Zettlemoyer"
            ],
            "abstract": "We introduce a new entity typing task: given a sentence with an entity\nmention, the goal is to predict a set of free-form phrases (e.g. skyscraper,\nsongwriter, or criminal) that describe appropriate types for the target entity.\nThis formulation allows us to use a new type of distant supervision at large\nscale: head words, which indicate the type of the noun phrases they appear in.\nWe show that these ultra-fine types can be crowd-sourced, and introduce new\nevaluation sets that are much more diverse and fine-grained than existing\nbenchmarks. We present a model that can predict open types, and is trained\nusing a multitask objective that pools our new head-word supervision with prior\nsupervision from entity linking. Experimental results demonstrate that our\nmodel is effective in predicting entity types at varying granularity; it\nachieves state of the art performance on an existing fine-grained entity typing\nbenchmark, and sets baselines for our newly-introduced datasets. Our data and\nmodel can be downloaded from: http://nlp.cs.washington.edu/entity_type",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04978v2",
            "date": "2018-07-13 09:06:07+00:00",
            "title": "Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units",
            "authors": [
                "Zhangyu Xiao",
                "Zhijian Ou",
                "Wei Chu",
                "Hui Lin"
            ],
            "abstract": "In this paper, we present an end-to-end automatic speech recognition system,\nwhich successfully employs subword units in a hybrid CTC-Attention based\nsystem. The subword units are obtained by the byte-pair encoding (BPE)\ncompression algorithm. Compared to using words as modeling units, using\ncharacters or subword units does not suffer from the out-of-vocabulary (OOV)\nproblem. Furthermore, using subword units further offers a capability in\nmodeling longer context than using characters. We evaluate different systems\nover the LibriSpeech 1000h dataset. The subword-based hybrid CTC-Attention\nsystem obtains 6.8% word error rate (WER) on the test_clean subset without any\ndictionary or external language model. This represents a significant\nimprovement (a 12.8% WER relative reduction) over the character-based hybrid\nCTC-Attention system.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.05154v1",
            "date": "2018-07-13 15:57:39+00:00",
            "title": "Deep Enhanced Representation for Implicit Discourse Relation Recognition",
            "authors": [
                "Hongxiao Bai",
                "Hai Zhao"
            ],
            "abstract": "Implicit discourse relation recognition is a challenging task as the relation\nprediction without explicit connectives in discourse parsing needs\nunderstanding of text spans and cannot be easily derived from surface features\nfrom the input sentence pairs. Thus, properly representing the text is very\ncrucial to this task. In this paper, we propose a model augmented with\ndifferent grained text representations, including character, subword, word,\nsentence, and sentence pair levels. The proposed deeper model is evaluated on\nthe benchmark treebank and achieves state-of-the-art accuracy with greater than\n48% in 11-way and $F_1$ score greater than 50% in 4-way classifications for the\nfirst time according to our best knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05849v1",
            "date": "2018-07-11 04:51:41+00:00",
            "title": "Neural Chinese Word Segmentation with Dictionary Knowledge",
            "authors": [
                "Junxin Liu",
                "Fangzhao Wu",
                "Chuhan Wu",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "Chinese word segmentation (CWS) is an important task for Chinese NLP.\nRecently, many neural network based methods have been proposed for CWS.\nHowever, these methods require a large number of labeled sentences for model\ntraining, and usually cannot utilize the useful information in Chinese\ndictionary. In this paper, we propose two methods to exploit the dictionary\ninformation for CWS. The first one is based on pseudo labeled data generation,\nand the second one is based on multi-task learning. The experimental results on\ntwo benchmark datasets validate that our approach can effectively improve the\nperformance of Chinese word segmentation, especially when training data is\ninsufficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.05855v1",
            "date": "2018-07-11 05:34:09+00:00",
            "title": "A Fast-Converged Acoustic Modeling for Korean Speech Recognition: A Preliminary Study on Time Delay Neural Network",
            "authors": [
                "Hosung Park",
                "Donghyun Lee",
                "Minkyu Lim",
                "Yoseb Kang",
                "Juneseok Oh",
                "Ji-Hwan Kim"
            ],
            "abstract": "In this paper, a time delay neural network (TDNN) based acoustic model is\nproposed to implement a fast-converged acoustic modeling for Korean speech\nrecognition. The TDNN has an advantage in fast-convergence where the amount of\ntraining data is limited, due to subsampling which excludes duplicated weights.\nThe TDNN showed an absolute improvement of 2.12% in terms of character error\nrate compared to feed forward neural network (FFNN) based modelling for Korean\nspeech corpora. The proposed model converged 1.67 times faster than a\nFFNN-based model did.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06414v1",
            "date": "2018-07-16 12:29:23+00:00",
            "title": "Combining a Context Aware Neural Network with a Denoising Autoencoder for Measuring String Similarities",
            "authors": [
                "Mehdi Ben Lazreg",
                "Morten Goodwin"
            ],
            "abstract": "Measuring similarities between strings is central for many established and\nfast growing research areas including information retrieval, biology, and\nnatural language processing. The traditional approach for string similarity\nmeasurements is to define a metric over a word space that quantifies and sums\nup the differences between characters in two strings. The state-of-the-art in\nthe area has, surprisingly, not evolved much during the last few decades. The\nmajority of the metrics are based on a simple comparison between character and\ncharacter distributions without consideration for the context of the words.\nThis paper proposes a string metric that encompasses similarities between\nstrings based on (1) the character similarities between the words including.\nNon-Standard and standard spellings of the same words, and (2) the context of\nthe words. Our proposal is a neural network composed of a denoising autoencoder\nand what we call a context encoder specifically designed to find similarities\nbetween the words based on their context. The experimental results show that\nthe resulting metrics succeeds in 85.4\\% of the cases in finding the correct\nversion of a non-standard spelling among the closest words, compared to 63.2\\%\nwith the established Normalised-Levenshtein distance. Besides, we show that\nwords used in similar context are with our approach calculated to be similar\nthan words with different contexts, which is a desirable property missing in\nestablished string metrics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.06441v1",
            "date": "2018-07-12 09:40:21+00:00",
            "title": "A Comparison of Adaptation Techniques and Recurrent Neural Network Architectures",
            "authors": [
                "Jan Vanek",
                "Josef Michalek",
                "Jan Zelinka",
                "Josef Psutka"
            ],
            "abstract": "Recently, recurrent neural networks have become state-of-the-art in acoustic\nmodeling for automatic speech recognition. The long short-term memory (LSTM)\nunits are the most popular ones. However, alternative units like gated\nrecurrent unit (GRU) and its modifications outperformed LSTM in some\npublications. In this paper, we compared five neural network (NN) architectures\nwith various adaptation and feature normalization techniques. We have evaluated\nfeature-space maximum likelihood linear regression, five variants of i-vector\nadaptation and two variants of cepstral mean normalization. The most adaptation\nand normalization techniques were developed for feed-forward NNs and, according\nto results in this paper, not all of them worked also with RNNs. For\nexperiments, we have chosen a well known and available TIMIT phone recognition\ntask. The phone recognition is much more sensitive to the quality of AM than\nlarge vocabulary task with a complex language model. Also, we published the\nopen-source scripts to easily replicate the results and to help continue the\ndevelopment.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.06610v1",
            "date": "2018-07-17 18:15:14+00:00",
            "title": "Learning Noise-Invariant Representations for Robust Speech Recognition",
            "authors": [
                "Davis Liang",
                "Zhiheng Huang",
                "Zachary C. Lipton"
            ],
            "abstract": "Despite rapid advances in speech recognition, current models remain brittle\nto superficial perturbations to their inputs. Small amounts of noise can\ndestroy the performance of an otherwise state-of-the-art model. To harden\nmodels against background noise, practitioners often perform data augmentation,\nadding artificially-noised examples to the training set, carrying over the\noriginal label. In this paper, we hypothesize that a clean example and its\nsuperficially perturbed counterparts shouldn't merely map to the same class ---\nthey should map to the same representation. We propose\ninvariant-representation-learning (IRL): At each training iteration, for each\ntraining example,we sample a noisy counterpart. We then apply a penalty term to\ncoerce matched representations at each layer (above some chosen layer). Our key\nresults, demonstrated on the Librispeech dataset are the following: (i) IRL\nsignificantly reduces character error rates (CER) on both 'clean' (3.3% vs\n6.5%) and 'other' (11.0% vs 18.1%) test sets; (ii) on several out-of-domain\nnoise settings (different from those seen during training), IRL's benefits are\neven more pronounced. Careful ablations confirm that our results are not simply\ndue to shrinking activations at the chosen layers.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.06736v1",
            "date": "2018-07-18 01:59:26+00:00",
            "title": "Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis",
            "authors": [
                "Jing-Xuan Zhang",
                "Zhen-Hua Ling",
                "Li-Rong Dai"
            ],
            "abstract": "This paper proposes a forward attention method for the sequenceto- sequence\nacoustic modeling of speech synthesis. This method is motivated by the nature\nof the monotonic alignment from phone sequences to acoustic sequences. Only the\nalignment paths that satisfy the monotonic condition are taken into\nconsideration at each decoder timestep. The modified attention probabilities at\neach timestep are computed recursively using a forward algorithm. A transition\nagent for forward attention is further proposed, which helps the attention\nmechanism to make decisions whether to move forward or stay at each decoder\ntimestep. Experimental results show that the proposed forward attention method\nachieves faster convergence speed and higher stability than the baseline\nattention method. Besides, the method of forward attention with transition\nagent can also help improve the naturalness of synthetic speech and control the\nspeed of synthetic speech effectively.",
            "journal ref": "IEEE International Conference on Acoustics, Speech and Signal\n  Processing (2018) 4789-4793",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.06978v1",
            "date": "2018-07-18 14:42:35+00:00",
            "title": "Improving Explainable Recommendations with Synthetic Reviews",
            "authors": [
                "Sixun Ouyang",
                "Aonghus Lawlor",
                "Felipe Costa",
                "Peter Dolog"
            ],
            "abstract": "An important task for a recommender system to provide interpretable\nexplanations for the user. This is important for the credibility of the system.\nCurrent interpretable recommender systems tend to focus on certain features\nknown to be important to the user and offer their explanations in a structured\nform. It is well known that user generated reviews and feedback from reviewers\nhave strong leverage over the users' decisions. On the other hand, recent text\ngeneration works have been shown to generate text of similar quality to human\nwritten text, and we aim to show that generated text can be successfully used\nto explain recommendations.\n  In this paper, we propose a framework consisting of popular review-oriented\ngeneration models aiming to create personalised explanations for\nrecommendations. The interpretations are generated at both character and word\nlevels. We build a dataset containing reviewers' feedback from the Amazon books\nreview dataset. Our cross-domain experiments are designed to bridge from\nnatural language processing to the recommender system domain. Besides language\nmodel evaluation methods, we employ DeepCoNN, a novel review-oriented\nrecommender system using a deep neural network, to evaluate the recommendation\nperformance of generated reviews by root mean square error (RMSE). We\ndemonstrate that the synthetic personalised reviews have better recommendation\nperformance than human written reviews. To our knowledge, this presents the\nfirst machine-generated natural language explanations for rating prediction.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.07147v3",
            "date": "2018-07-17 15:13:20+00:00",
            "title": "Guess who? Multilingual approach for the automated generation of author-stylized poetry",
            "authors": [
                "Alexey Tikhonov",
                "Ivan P. Yamshchikov"
            ],
            "abstract": "This paper addresses the problem of stylized text generation in a\nmultilingual setup. A version of a language model based on a long short-term\nmemory (LSTM) artificial neural network with extended phonetic and semantic\nembeddings is used for stylized poetry generation. The quality of the resulting\npoems generated by the network is estimated through bilingual evaluation\nunderstudy (BLEU), a survey and a new cross-entropy based metric that is\nsuggested for the problems of such type. The experiments show that the proposed\nmodel consistently outperforms random sample and vanilla-LSTM baselines, humans\nalso tend to associate machine generated texts with the target author.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07149v1",
            "date": "2018-07-17 03:52:26+00:00",
            "title": "A Hand-Held Multimedia Translation and Interpretation System with Application to Diet Management",
            "authors": [
                "Albert Parra",
                "Andrew W. Haddad",
                "Mireille Boutin",
                "Edward J. Delp"
            ],
            "abstract": "We propose a network independent, hand-held system to translate and\ndisambiguate foreign restaurant menu items in real-time. The system is based on\nthe use of a portable multimedia device, such as a smartphones or a PDA. An\naccurate and fast translation is obtained using a Machine Translation engine\nand a context-specific corpora to which we apply two pre-processing steps,\ncalled translation standardization and $n$-gram consolidation. The phrase-table\ngenerated is orders of magnitude lighter than the ones commonly used in market\napplications, thus making translations computationally less expensive, and\ndecreasing the battery usage. Translation ambiguities are mitigated using\nmultimedia information including images of dishes and ingredients, along with\ningredient lists. We implemented a prototype of our system on an iPod Touch\nSecond Generation for English speakers traveling in Spain. Our tests indicate\nthat our translation method yields higher accuracy than translation engines\nsuch as Google Translate, and does so almost instantaneously. The memory\nrequirements of the application, including the database of images, are also\nwell within the limits of the device. By combining it with a database of\nnutritional information, our proposed system can be used to help individuals\nwho follow a medical diet maintain this diet while traveling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07187v1",
            "date": "2018-07-18 23:45:33+00:00",
            "title": "Efficient Training on Very Large Corpora via Gramian Estimation",
            "authors": [
                "Walid Krichene",
                "Nicolas Mayoraz",
                "Steffen Rendle",
                "Li Zhang",
                "Xinyang Yi",
                "Lichan Hong",
                "Ed Chi",
                "John Anderson"
            ],
            "abstract": "We study the problem of learning similarity functions over very large corpora\nusing neural network embedding models. These models are typically trained using\nSGD with sampling of random observed and unobserved pairs, with a number of\nsamples that grows quadratically with the corpus size, making it expensive to\nscale to very large corpora. We propose new efficient methods to train these\nmodels without having to sample unobserved pairs. Inspired by matrix\nfactorization, our approach relies on adding a global quadratic penalty to all\npairs of examples and expressing this term as the matrix-inner-product of two\ngeneralized Gramians. We show that the gradient of this term can be efficiently\ncomputed by maintaining estimates of the Gramians, and develop variance\nreduction schemes to improve the quality of the estimates. We conduct\nlarge-scale experiments that show a significant improvement in training time\nand generalization quality compared to traditional sampling methods.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1807.07255v2",
            "date": "2018-07-19 06:41:05+00:00",
            "title": "Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts",
            "authors": [
                "Can Xu",
                "Wei Wu",
                "Yu Wu"
            ],
            "abstract": "We study open domain dialogue generation with dialogue acts designed to\nexplain how people engage in social chat. To imitate human behavior, we propose\nmanaging the flow of human-machine interactions with the dialogue acts as\npolicies. The policies and response generation are jointly learned from\nhuman-human conversations, and the former is further optimized with a\nreinforcement learning approach. With the dialogue acts, we achieve significant\nimprovement over state-of-the-art methods on response quality for given\ncontexts and dialogue length in both machine-machine simulation and\nhuman-machine conversation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07545v1",
            "date": "2018-07-19 17:23:13+00:00",
            "title": "Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks",
            "authors": [
                "Jo\u00e3o Loula",
                "Marco Baroni",
                "Brenden M. Lake"
            ],
            "abstract": "Systematic compositionality is the ability to recombine meaningful units with\nregular and predictable outcomes, and it's seen as key to humans' capacity for\ngeneralization in language. Recent work has studied systematic compositionality\nin modern seq2seq models using generalization to novel navigation instructions\nin a grounded environment as a probing tool, requiring models to quickly\nbootstrap the meaning of new words. We extend this framework here to settings\nwhere the model needs only to recombine well-trained functional words (such as\n\"around\" and \"right\") in novel contexts. Our findings confirm and strengthen\nthe earlier ones: seq2seq models can be impressively good at generalizing to\nnovel combinations of previously-seen input, but only when they receive\nextensive training on the specific pattern to be generalized (e.g.,\ngeneralizing from many examples of \"X around right\" to \"jump around right\"),\nwhile failing when generalization requires novel application of compositional\nrules (e.g., inferring the meaning of \"around right\" from those of \"right\" and\n\"around\").",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07741v1",
            "date": "2018-07-20 08:40:10+00:00",
            "title": "Learning Representations for Soft Skill Matching",
            "authors": [
                "Luiza Sayfullina",
                "Eric Malmi",
                "Juho Kannala"
            ],
            "abstract": "Employers actively look for talents having not only specific hard skills but\nalso various soft skills. To analyze the soft skill demands on the job market,\nit is important to be able to detect soft skill phrases from job advertisements\nautomatically. However, a naive matching of soft skill phrases can lead to\nfalse positive matches when a soft skill phrase, such as friendly, is used to\ndescribe a company, a team, or another entity, rather than a desired candidate.\n  In this paper, we propose a phrase-matching-based approach which\ndifferentiates between soft skill phrases referring to a candidate vs.\nsomething else. The disambiguation is formulated as a binary text\nclassification problem where the prediction is made for the potential soft\nskill based on the context where it occurs. To inform the model about the soft\nskill for which the prediction is made, we develop several approaches,\nincluding soft skill masking and soft skill tagging.\n  We compare several neural network based approaches, including CNN, LSTM and\nHierarchical Attention Model. The proposed tagging-based input representation\nusing LSTM achieved the highest recall of 83.92% on the job dataset when fixing\na precision to 95%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07752v1",
            "date": "2018-07-20 09:19:08+00:00",
            "title": "Twitter Sentiment Analysis System",
            "authors": [
                "Shaunak Joshi",
                "Deepali Deshpande"
            ],
            "abstract": "Social media is increasingly used by humans to express their feelings and\nopinions in the form of short text messages. Detecting sentiments in the text\nhas a wide range of applications including identifying anxiety or depression of\nindividuals and measuring well-being or mood of a community. Sentiments can be\nexpressed in many ways that can be seen such as facial expression and gestures,\nspeech and by written text. Sentiment Analysis in text documents is essentially\na content-based classification problem involving concepts from the domains of\nNatural Language Processing as well as Machine Learning. In this paper,\nsentiment recognition based on textual data and the techniques used in\nsentiment analysis are discussed.",
            "journal ref": "International Journal of Computer Applications (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07779v1",
            "date": "2018-07-20 10:37:31+00:00",
            "title": "A Generalized Vector Space Model for Ontology-Based Information Retrieval",
            "authors": [
                "Vuong M. Ngo",
                "Tru H. Cao"
            ],
            "abstract": "Named entities (NE) are objects that are referred to by names such as people,\norganizations and locations. Named entities and keywords are important to the\nmeaning of a document. We propose a generalized vector space model that\ncombines named entities and keywords. In the model, we take into account\ndifferent ontological features of named entities, namely, aliases, classes and\nidentifiers. Moreover, we use entity classes to represent the latent\ninformation of interrogative words in Wh-queries, which are ignored in\ntraditional keyword-based searching. We have implemented and tested the\nproposed model on a TREC dataset, as presented and discussed in the paper.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.07965v2",
            "date": "2018-07-20 09:55:09+00:00",
            "title": "An Efficient End-to-End Neural Model for Handwritten Text Recognition",
            "authors": [
                "Arindam Chowdhury",
                "Lovekesh Vig"
            ],
            "abstract": "Offline handwritten text recognition from images is an important problem for\nenterprises attempting to digitize large volumes of handmarked scanned\ndocuments/reports. Deep recurrent models such as Multi-dimensional LSTMs have\nbeen shown to yield superior performance over traditional Hidden Markov Model\nbased approaches that suffer from the Markov assumption and therefore lack the\nrepresentational power of RNNs. In this paper we introduce a novel approach\nthat combines a deep convolutional network with a recurrent Encoder-Decoder\nnetwork to map an image to a sequence of characters corresponding to the text\npresent in the image. The entire model is trained end-to-end using Focal Loss,\nan improvement over the standard Cross-Entropy loss that addresses the class\nimbalance problem, inherent to text recognition. To enhance the decoding\ncapacity of the model, Beam Search algorithm is employed which searches for the\nbest sequence out of a set of hypotheses based on a joint distribution of\nindividual characters. Our model takes as input a downsampled version of the\noriginal image thereby making it both computationally and memory efficient. The\nexperimental results were benchmarked against two publicly available datasets,\nIAM and RIMES. We surpass the state-of-the-art word level accuracy on the\nevaluation set of both datasets by 3.5% & 1.1%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.07982v2",
            "date": "2018-07-20 18:09:44+00:00",
            "title": "Visitors to urban greenspace have higher sentiment and lower negativity on Twitter",
            "authors": [
                "Aaron J. Schwartz",
                "Peter Sheridan Dodds",
                "Jarlath P. M. O'Neil-Dunne",
                "Christopher M. Danforth",
                "Taylor H. Ricketts"
            ],
            "abstract": "With more people living in cities, we are witnessing a decline in exposure to\nnature. A growing body of research has demonstrated an association between\nnature contact and improved mood. Here, we used Twitter and the Hedonometer, a\nworld analysis tool, to investigate how sentiment, or the estimated happiness\nof the words people write, varied before, during, and after visits to San\nFrancisco's urban park system. We found that sentiment was substantially higher\nduring park visits and remained elevated for several hours following the visit.\nLeveraging differences in vegetative cover across park types, we explored how\ndifferent types of outdoor public spaces may contribute to subjective\nwell-being. Tweets during visits to Regional Parks, which are greener and have\ngreater vegetative cover, exhibited larger increases in sentiment than tweets\nduring visits to Civic Plazas and Squares. Finally, we analyzed word\nfrequencies to explore several mechanisms theorized to link nature exposure\nwith mental and cognitive benefits. Negation words such as 'no', 'not', and\n'don't' decreased in frequency during visits to urban parks. These results can\nbe used by urban planners and public health officials to better target nature\ncontact recommendations for growing urban populations.",
            "journal ref": "People Nat. 2019; 00: 1- 10",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1807.08076v1",
            "date": "2018-07-21 03:21:56+00:00",
            "title": "Consequences and Factors of Stylistic Differences in Human-Robot Dialogue",
            "authors": [
                "Stephanie M. Lukin",
                "Kimberly A. Pollard",
                "Claire Bonial",
                "Matthew Marge",
                "Cassidy Henry",
                "Ron Arstein",
                "David Traum",
                "Clare R. Voss"
            ],
            "abstract": "This paper identifies stylistic differences in instruction-giving observed in\na corpus of human-robot dialogue. Differences in verbosity and structure (i.e.,\nsingle-intent vs. multi-intent instructions) arose naturally without\nrestrictions or prior guidance on how users should speak with the robot.\nDifferent styles were found to produce different rates of miscommunication, and\ncorrelations were found between style differences and individual user\nvariation, trust, and interaction experience with the robot. Understanding\npotential consequences and factors that influence style can inform design of\ndialogue systems that are robust to natural variation from human users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08089v4",
            "date": "2018-07-21 06:07:46+00:00",
            "title": "Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval",
            "authors": [
                "Yi-Chen Chen",
                "Sung-Feng Huang",
                "Chia-Hao Shen",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Word embedding or Word2Vec has been successful in offering semantics for text\nwords learned from the context of words. Audio Word2Vec was shown to offer\nphonetic structures for spoken words (signal segments for words) learned from\nsignals within spoken words. This paper proposes a two-stage framework to\nperform phonetic-and-semantic embedding on spoken words considering the context\nof the spoken words. Stage 1 performs phonetic embedding with speaker\ncharacteristics disentangled. Stage 2 then performs semantic embedding in\naddition. We further propose to evaluate the phonetic-and-semantic nature of\nthe audio embeddings obtained in Stage 2 by parallelizing with text embeddings.\nIn general, phonetic structure and semantics inevitably disturb each other. For\nexample the words \"brother\" and \"sister\" are close in semantics but very\ndifferent in phonetic structure, while the words \"brother\" and \"bother\" are in\nthe other way around. But phonetic-and-semantic embedding is attractive, as\nshown in the initial experiments on spoken document retrieval. Not only spoken\ndocuments including the spoken query can be retrieved based on the phonetic\nstructures, but spoken documents semantically related to the query but not\nincluding the query can also be retrieved based on the semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08280v1",
            "date": "2018-07-22 13:10:30+00:00",
            "title": "Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "A sequence-to-sequence model is a neural network module for mapping two\nsequences of different lengths. The sequence-to-sequence model has three core\nmodules: encoder, decoder, and attention. Attention is the bridge that connects\nthe encoder and decoder modules and improves model performance in many tasks.\nIn this paper, we propose two ideas to improve sequence-to-sequence model\nperformance by enhancing the attention module. First, we maintain the history\nof the location and the expected context from several previous time-steps.\nSecond, we apply multiscale convolution from several previous attention vectors\nto the current decoder state. We utilized our proposed framework for\nsequence-to-sequence speech recognition and text-to-speech systems. The results\nreveal that our proposed extension could improve performance significantly\ncompared to a standard attention baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08447v1",
            "date": "2018-07-23 06:47:57+00:00",
            "title": "LinkNBed: Multi-Graph Representation Learning with Entity Linkage",
            "authors": [
                "Rakshit Trivedi",
                "Bunyamin Sisman",
                "Jun Ma",
                "Christos Faloutsos",
                "Hongyuan Zha",
                "Xin Luna Dong"
            ],
            "abstract": "Knowledge graphs have emerged as an important model for studying complex\nmulti-relational data. This has given rise to the construction of numerous\nlarge scale but incomplete knowledge graphs encoding information extracted from\nvarious resources. An effective and scalable approach to jointly learn over\nmultiple graphs and eventually construct a unified graph is a crucial next step\nfor the success of knowledge-based inference for many downstream applications.\nTo this end, we propose LinkNBed, a deep relational learning framework that\nlearns entity and relationship representations across multiple graphs. We\nidentify entity linkage across graphs as a vital component to achieve our goal.\nWe design a novel objective that leverage entity linkage and build an efficient\nmulti-task training procedure. Experiments on link prediction and entity\nlinkage demonstrate substantial improvements over the state-of-the-art\nrelational learning approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.08465v1",
            "date": "2018-07-23 07:52:52+00:00",
            "title": "Multimodal Social Media Analysis for Gang Violence Prevention",
            "authors": [
                "Philipp Blandfort",
                "Desmond Patton",
                "William R. Frey",
                "Svebor Karaman",
                "Surabhi Bhargava",
                "Fei-Tzin Lee",
                "Siddharth Varia",
                "Chris Kedzie",
                "Michael B. Gaskell",
                "Rossano Schifanella",
                "Kathleen McKeown",
                "Shih-Fu Chang"
            ],
            "abstract": "Gang violence is a severe issue in major cities across the U.S. and recent\nstudies [Patton et al. 2017] have found evidence of social media communications\nthat can be linked to such violence in communities with high rates of exposure\nto gang activity. In this paper we partnered computer scientists with social\nwork researchers, who have domain expertise in gang violence, to analyze how\npublic tweets with images posted by youth who mention gang associations on\nTwitter can be leveraged to automatically detect psychosocial factors and\nconditions that could potentially assist social workers and violence outreach\nworkers in prevention and early intervention programs. To this end, we\ndeveloped a rigorous methodology for collecting and annotating tweets. We\ngathered 1,851 tweets and accompanying annotations related to visual concepts\nand the psychosocial codes: aggression, loss, and substance use. These codes\nare relevant to social work interventions, as they represent possible pathways\nto violence on social media. We compare various methods for classifying tweets\ninto these three classes, using only the text of the tweet, only the image of\nthe tweet, or both modalities as input to the classifier. In particular, we\nanalyze the usefulness of mid-level visual concepts and the role of different\nmodalities for this tweet classification task. Our experiments show that\nindividually, text information dominates classification performance of the loss\nclass, while image information dominates the aggression and substance use\nclasses. Our multimodal approach provides a very promising improvement (18%\nrelative in mean average precision) over the best single modality approach.\nFinally, we also illustrate the complexity of understanding social media data\nand elaborate on open challenges.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.08484v2",
            "date": "2018-07-23 08:57:44+00:00",
            "title": "AceKG: A Large-scale Knowledge Graph for Academic Data Mining",
            "authors": [
                "Ruijie Wang",
                "Yuchen Yan",
                "Jialu Wang",
                "Yuting Jia",
                "Ye Zhang",
                "Weinan Zhang",
                "Xinbing Wang"
            ],
            "abstract": "Most existing knowledge graphs (KGs) in academic domains suffer from problems\nof insufficient multi-relational information, name ambiguity and improper data\nformat for large-scale machine processing. In this paper, we present AceKG, a\nnew large-scale KG in academic domain. AceKG not only provides clean academic\ninformation, but also offers a large-scale benchmark dataset for researchers to\nconduct challenging data mining projects including link prediction, community\ndetection and scholar classification. Specifically, AceKG describes 3.13\nbillion triples of academic facts based on a consistent ontology, including\nnecessary properties of papers, authors, fields of study, venues and\ninstitutes, as well as the relations among them. To enrich the proposed\nknowledge graph, we also perform entity alignment with existing databases and\nrule-based inference. Based on AceKG, we conduct experiments of three typical\nacademic data mining tasks and evaluate several state-of- the-art knowledge\nembedding and network representation learning approaches on the benchmark\ndatasets built from AceKG. Finally, we discuss several promising research\ndirections that benefit from AceKG.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.09561v1",
            "date": "2018-07-25 12:56:25+00:00",
            "title": "Enhancing keyword correlation for event detection in social networks using SVD and k-means: Twitter case study",
            "authors": [
                "Ahmad Hany Hossny",
                "Terry Moschou",
                "Grant Osborne",
                "Lewis Mitchell",
                "Nick Lothian"
            ],
            "abstract": "Extracting textual features from tweets is a challenging process due to the\nnoisy nature of the content and the weak signal of most of the words used. In\nthis paper, we propose using singular value decomposition (SVD) with clustering\nto enhance the signals of the textual features in the tweets to improve the\ncorrelation with events. The proposed technique applies SVD to the time series\nvector for each feature to factorize the matrix of feature/day counts, in order\nto ensure the independence of the feature vectors. Afterwards, the k-means\nclustering is applied to build a look-up table that maps members of each\ncluster to the cluster-centroid. The lookup table is used to map each feature\nin the original data to the centroid of its cluster, then we calculate the sum\nof the term frequency vectors of all features in each cluster to the\nterm-frequency-vector of the cluster centroid. To test the technique we\ncalculated the correlations of the cluster centroids with the golden standard\nrecord (GSR) vector before and after summing the vectors of the cluster members\nto the centroid-vector. The proposed method is applied to multiple correlation\ntechniques including the Pearson, Spearman, distance correlation and Kendal\nTao. The experiments have also considered the different word forms and lengths\nof the features including keywords, n-grams, skip-grams and bags-of-words. The\ncorrelation results are enhanced significantly as the highest correlation\nscores have increased from 0.3 to 0.6, and the average correlation scores have\nincreased from 0.3 to 0.4.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1807.09597v2",
            "date": "2018-07-23 06:29:43+00:00",
            "title": "Acoustic-to-Word Recognition with Sequence-to-Sequence Models",
            "authors": [
                "Shruti Palaskar",
                "Florian Metze"
            ],
            "abstract": "Acoustic-to-Word recognition provides a straightforward solution to\nend-to-end speech recognition without needing external decoding, language model\nre-scoring or lexicon. While character-based models offer a natural solution to\nthe out-of-vocabulary problem, word models can be simpler to decode and may\nalso be able to directly recognize semantically meaningful units. We present\neffective methods to train Sequence-to-Sequence models for direct word-level\nrecognition (and character-level recognition) and show an absolute improvement\nof 4.4-5.0\\% in Word Error Rate on the Switchboard corpus compared to prior\nwork. In addition to these promising results, word-based models are more\ninterpretable than character models, which have to be composed into words using\na separate decoding step. We analyze the encoder hidden states and the\nattention behavior, and show that location-aware attention naturally represents\nwords as a single speech-word-vector, despite spanning multiple frames in the\ninput. We finally show that the Acoustic-to-Word model also learns to segment\nspeech into words with a mean standard deviation of 3 frames as compared with\nhuman annotated forced-alignments for the Switchboard corpus.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.09623v1",
            "date": "2018-07-25 14:15:40+00:00",
            "title": "Repartitioning of the ComplexWebQuestions Dataset",
            "authors": [
                "Alon Talmor",
                "Jonathan Berant"
            ],
            "abstract": "Recently, Talmor and Berant (2018) introduced ComplexWebQuestions - a dataset\nfocused on answering complex questions by decomposing them into a sequence of\nsimpler questions and extracting the answer from retrieved web snippets. In\ntheir work the authors used a pre-trained reading comprehension (RC) model\n(Salant and Berant, 2018) to extract the answer from the web snippets. In this\nshort note we show that training a RC model directly on the training data of\nComplexWebQuestions reveals a leakage from the training set to the test set\nthat allows to obtain unreasonably high performance. As a solution, we\nconstruct a new partitioning of ComplexWebQuestions that does not suffer from\nthis leakage and publicly release it. We also perform an empirical evaluation\non these two datasets and show that training a RC model on the training data\nsubstantially improves state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09639v1",
            "date": "2018-07-25 14:43:46+00:00",
            "title": "Finding Better Subword Segmentation for Neural Machine Translation",
            "authors": [
                "Yingting Wu",
                "Hai Zhao"
            ],
            "abstract": "For different language pairs, word-level neural machine translation (NMT)\nmodels with a fixed-size vocabulary suffer from the same problem of\nrepresenting out-of-vocabulary (OOV) words. The common practice usually\nreplaces all these rare or unknown words with a <UNK> token, which limits the\ntranslation performance to some extent. Most of recent work handled such a\nproblem by splitting words into characters or other specially extracted subword\nunits to enable open-vocabulary translation. Byte pair encoding (BPE) is one of\nthe successful attempts that has been shown extremely competitive by providing\neffective subword segmentation for NMT systems. In this paper, we extend the\nBPE style segmentation to a general unsupervised framework with three\nstatistical measures: frequency (FRQ), accessor variety (AV) and description\nlength gain (DLG). We test our approach on two translation tasks: German to\nEnglish and Chinese to English. The experimental results show that AV and DLG\nenhanced systems outperform the FRQ baseline in the frequency weighted schemes\nat different significant levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.09842v1",
            "date": "2018-07-24 04:14:51+00:00",
            "title": "Understanding and representing the semantics of large structured documents",
            "authors": [
                "Muhammad Mahbubur Rahman",
                "Tim Finin"
            ],
            "abstract": "Understanding large, structured documents like scholarly articles, requests\nfor proposals or business reports is a complex and difficult task. It involves\ndiscovering a document's overall purpose and subject(s), understanding the\nfunction and meaning of its sections and subsections, and extracting low level\nentities and facts about them. In this research, we present a deep learning\nbased document ontology to capture the general purpose semantic structure and\ndomain specific semantic concepts from a large number of academic articles and\nbusiness documents. The ontology is able to describe different functional parts\nof a document, which can be used to enhance semantic indexing for a better\nunderstanding by human beings and machines. We evaluate our models through\nextensive experiments on datasets of scholarly articles from arXiv and Request\nfor Proposal documents.",
            "journal ref": "Semantic Deep Learning at ISWC 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10675v1",
            "date": "2018-07-26 17:05:20+00:00",
            "title": "Resource-Size matters: Improving Neural Named Entity Recognition with Optimized Large Corpora",
            "authors": [
                "Sajawel Ahmed",
                "Alexander Mehler"
            ],
            "abstract": "This study improves the performance of neural named entity recognition by a\nmargin of up to 11% in F-score on the example of a low-resource language like\nGerman, thereby outperforming existing baselines and establishing a new\nstate-of-the-art on each single open-source dataset. Rather than designing\ndeeper and wider hybrid neural architectures, we gather all available resources\nand perform a detailed optimization and grammar-dependent morphological\nprocessing consisting of lemmatization and part-of-speech tagging prior to\nexposing the raw data to any training process. We test our approach in a\nthreefold monolingual experimental setup of a) single, b) joint, and c)\noptimized training and shed light on the dependency of downstream-tasks on the\nsize of corpora used to compute word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.10857v2",
            "date": "2018-07-27 23:33:33+00:00",
            "title": "A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition",
            "authors": [
                "Shubham Toshniwal",
                "Anjuli Kannan",
                "Chung-Cheng Chiu",
                "Yonghui Wu",
                "Tara N Sainath",
                "Karen Livescu"
            ],
            "abstract": "Attention-based recurrent neural encoder-decoder models present an elegant\nsolution to the automatic speech recognition problem. This approach folds the\nacoustic model, pronunciation model, and language model into a single network\nand requires only a parallel corpus of speech and text for training. However,\nunlike in conventional approaches that combine separate acoustic and language\nmodels, it is not clear how to use additional (unpaired) text. While there has\nbeen previous work on methods addressing this problem, a thorough comparison\namong methods is still lacking. In this paper, we compare a suite of past\nmethods and some of our own proposed methods for using unpaired text data to\nimprove encoder-decoder models. For evaluation, we use the medium-sized\nSwitchboard data set and the large-scale Google voice search and dictation data\nsets. Our results confirm the benefits of using unpaired text across a range of\nmethods and data sets. Surprisingly, for first-pass decoding, the rather simple\napproach of shallow fusion performs best across data sets. However, for Google\ndata sets we find that cold fusion has a lower oracle error rate and\noutperforms other approaches after second-pass rescoring on the Google voice\nsearch data set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.10984v2",
            "date": "2018-07-28 23:54:59+00:00",
            "title": "Domain Robust Feature Extraction for Rapid Low Resource ASR Development",
            "authors": [
                "Siddharth Dalmia",
                "Xinjian Li",
                "Florian Metze",
                "Alan W. Black"
            ],
            "abstract": "Developing a practical speech recognizer for a low resource language is\nchallenging, not only because of the (potentially unknown) properties of the\nlanguage, but also because test data may not be from the same domain as the\navailable training data. In this paper, we focus on the latter challenge, i.e.\ndomain mismatch, for systems trained using a sequence-based criterion. We\ndemonstrate the effectiveness of using a pre-trained English recognizer, which\nis robust to such mismatched conditions, as a domain normalizing feature\nextractor on a low resource language. In our example, we use Turkish\nConversational Speech and Broadcast News data. This enables rapid development\nof speech recognizers for new languages which can easily adapt to any domain.\nTesting in various cross-domain scenarios, we achieve relative improvements of\naround 25% in phoneme error rate, with improvements being around 50% for some\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11024v1",
            "date": "2018-07-29 09:05:21+00:00",
            "title": "Opinion Spam Recognition Method for Online Reviews using Ontological Features",
            "authors": [
                "L. H. Nguyen",
                "N. T. H. Pham",
                "V. M. Ngo"
            ],
            "abstract": "Nowadays, there are a lot of people using social media opinions to make their\ndecision on buying products or services. Opinion spam detection is a hard\nproblem because fake reviews can be made by organizations as well as\nindividuals for different purposes. They write fake reviews to mislead readers\nor automated detection system by promoting or demoting target products to\npromote them or to damage their reputations. In this paper, we pro-pose a new\napproach using knowledge-based Ontology to detect opinion spam with high\naccuracy (higher than 75%). Keywords: Opinion spam, Fake review, E-commercial,\nOntology.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.11125v2",
            "date": "2018-07-29 23:51:08+00:00",
            "title": "Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems",
            "authors": [
                "Xiujun Li",
                "Yu Wang",
                "Siqi Sun",
                "Sarah Panda",
                "Jingjing Liu",
                "Jianfeng Gao"
            ],
            "abstract": "This proposal introduces a Dialogue Challenge for building end-to-end\ntask-completion dialogue systems, with the goal of encouraging the dialogue\nresearch community to collaborate and benchmark on standard datasets and\nunified experimental environment. In this special session, we will release\nhuman-annotated conversational data in three domains (movie-ticket booking,\nrestaurant reservation, and taxi booking), as well as an experiment platform\nwith built-in simulators in each domain, for training and evaluation purposes.\nThe final submitted systems will be evaluated both in simulated setting and by\nhuman judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11284v1",
            "date": "2018-07-30 11:00:59+00:00",
            "title": "Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition",
            "authors": [
                "Pavel Denisov",
                "Ngoc Thang Vu",
                "Marc Ferras Font"
            ],
            "abstract": "In this paper, we investigate the use of adversarial learning for\nunsupervised adaptation to unseen recording conditions, more specifically,\nsingle microphone far-field speech. We adapt neural networks based acoustic\nmodels trained with close-talk clean speech to the new recording conditions\nusing untranscribed adaptation data. Our experimental results on Italian\nSPEECON data set show that our proposed method achieves 19.8% relative word\nerror rate (WER) reduction compared to the unadapted models. Furthermore, this\nadaptation method is beneficial even when performed on data from another\nlanguage (i.e. French) giving 12.6% relative WER reduction.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.11582v2",
            "date": "2018-07-27 11:10:03+00:00",
            "title": "A Hierarchical Approach to Neural Context-Aware Modeling",
            "authors": [
                "Patrick Huber",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "We present a new recurrent neural network topology to enhance\nstate-of-the-art machine learning systems by incorporating a broader context.\nOur approach overcomes recent limitations with extended narratives through a\nmulti-layered computational approach to generate an abstract context\nrepresentation. Therefore, the developed system captures the narrative on\nword-level, sentence-level, and context-level. Through the hierarchical set-up,\nour proposed model summarizes the most salient information on each level and\ncreates an abstract representation of the extended context. We subsequently use\nthis representation to enhance neural language processing systems on the task\nof semantic error detection. To show the potential of the newly introduced\ntopology, we compare the approach against a context-agnostic set-up including a\nstandard neural language model and a supervised binary classification network.\nThe performance measures on the error detection task show the advantage of the\nhierarchical context-aware topologies, improving the baseline by 12.75%\nrelative for unsupervised models and 20.37% relative for supervised models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11679v1",
            "date": "2018-07-31 06:38:54+00:00",
            "title": "Wasserstein GAN and Waveform Loss-based Acoustic Model Training for Multi-speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder",
            "authors": [
                "Yi Zhao",
                "Shinji Takaki",
                "Hieu-Thi Luong",
                "Junichi Yamagishi",
                "Daisuke Saito",
                "Nobuaki Minematsu"
            ],
            "abstract": "Recent neural networks such as WaveNet and sampleRNN that learn directly from\nspeech waveform samples have achieved very high-quality synthetic speech in\nterms of both naturalness and speaker similarity even in multi-speaker\ntext-to-speech synthesis systems. Such neural networks are being used as an\nalternative to vocoders and hence they are often called neural vocoders. The\nneural vocoder uses acoustic features as local condition parameters, and these\nparameters need to be accurately predicted by another acoustic model. However,\nit is not yet clear how to train this acoustic model, which is problematic\nbecause the final quality of synthetic speech is significantly affected by the\nperformance of the acoustic model. Significant degradation happens, especially\nwhen predicted acoustic features have mismatched characteristics compared to\nnatural ones. In order to reduce the mismatched characteristics between natural\nand generated acoustic features, we propose frameworks that incorporate either\na conditional generative adversarial network (GAN) or its variant, Wasserstein\nGAN with gradient penalty (WGAN-GP), into multi-speaker speech synthesis that\nuses the WaveNet vocoder. We also extend the GAN frameworks and use the\ndiscretized mixture logistic loss of a well-trained WaveNet in addition to mean\nsquared error and adversarial losses as parts of objective functions.\nExperimental results show that acoustic models trained using the WGAN-GP\nframework using back-propagated discretized-mixture-of-logistics (DML) loss\nachieves the highest subjective evaluation scores in terms of both quality and\nspeaker similarity.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1807.11689v2",
            "date": "2018-07-31 07:19:36+00:00",
            "title": "Neural Article Pair Modeling for Wikipedia Sub-article Matching",
            "authors": [
                "Muhao Chen",
                "Changping Meng",
                "Gang Huang",
                "Carlo Zaniolo"
            ],
            "abstract": "Nowadays, editors tend to separate different subtopics of a long Wiki-pedia\narticle into multiple sub-articles. This separation seeks to improve human\nreadability. However, it also has a deleterious effect on many Wikipedia-based\ntasks that rely on the article-as-concept assumption, which requires each\nentity (or concept) to be described solely by one article. This underlying\nassumption significantly simplifies knowledge representation and extraction,\nand it is vital to many existing technologies such as automated knowledge base\nconstruction, cross-lingual knowledge alignment, semantic search and data\nlineage of Wikipedia entities. In this paper we provide an approach to match\nthe scattered sub-articles back to their corresponding main-articles, with the\nintent of facilitating automated Wikipedia curation and processing. The\nproposed model adopts a hierarchical learning structure that combines multiple\nvariants of neural document pair encoders with a comprehensive set of explicit\nfeatures. A large crowdsourced dataset is created to support the evaluation and\nfeature extraction for the task. Based on the large dataset, the proposed model\nachieves promising results of cross-validation and significantly outperforms\nprevious approaches. Large-scale serving on the entire English Wikipedia also\nproves the practicability and scalability of the proposed model by effectively\nextracting a vast collection of newly paired main and sub-articles.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1807.11838v1",
            "date": "2018-07-31 14:31:17+00:00",
            "title": "Extensible Grounding of Speech for Robot Instruction",
            "authors": [
                "Jonathan Connell"
            ],
            "abstract": "Spoken language is a convenient interface for commanding a mobile robot. Yet\nfor this to work a number of base terms must be grounded in perceptual and\nmotor skills. We detail the language processing used on our robot ELI and\nexplain how this grounding is performed, how it interacts with user gestures,\nand how it handles phenomena such as anaphora. More importantly, however, there\nare certain concepts which the robot cannot be preprogrammed with, such as the\nnames of various objects in a household or the nature of specific tasks it may\nbe requested to perform. In these cases it is vital that there exist a method\nfor extending the grounding, essentially \"learning by being told\". We describe\nhow this was successfully implemented for learning new nouns and verbs in a\ntabletop setting. Creating this language learning kernel may be the last\nexplicit programming the robot ever needs - the core mechanism could eventually\nbe used for imparting a vast amount of knowledge, much as a child learns from\nits parents and teachers.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1809.00068v1",
            "date": "2018-08-31 22:01:45+00:00",
            "title": "Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection",
            "authors": [
                "Wei Wang",
                "Taro Watanabe",
                "Macduff Hughes",
                "Tetsuji Nakagawa",
                "Ciprian Chelba"
            ],
            "abstract": "Measuring domain relevance of data and identifying or selecting well-fit\ndomain data for machine translation (MT) is a well-studied topic, but denoising\nis not yet. Denoising is concerned with a different type of data quality and\ntries to reduce the negative impact of data noise on MT training, in\nparticular, neural MT (NMT) training. This paper generalizes methods for\nmeasuring and selecting data for domain MT and applies them to denoising NMT\ntraining. The proposed approach uses trusted data and a denoising curriculum\nrealized by online data selection. Intrinsic and extrinsic evaluations of the\napproach show its significant effectiveness for NMT to train on data with\nsevere noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00204v1",
            "date": "2018-09-01 15:11:12+00:00",
            "title": "Improving Visual Relationship Detection using Semantic Modeling of Scene Descriptions",
            "authors": [
                "Stephan Baier",
                "Yunpu Ma",
                "Volker Tresp"
            ],
            "abstract": "Structured scene descriptions of images are useful for the automatic\nprocessing and querying of large image databases. We show how the combination\nof a semantic and a visual statistical model can improve on the task of mapping\nimages to their associated scene description. In this paper we consider scene\ndescriptions which are represented as a set of triples (subject, predicate,\nobject), where each triple consists of a pair of visual objects, which appear\nin the image, and the relationship between them (e.g. man-riding-elephant,\nman-wearing-hat). We combine a standard visual model for object detection,\nbased on convolutional neural networks, with a latent variable model for link\nprediction. We apply multiple state-of-the-art link prediction methods and\ncompare their capability for visual relationship detection. One of the main\nadvantages of link prediction methods is that they can also generalize to\ntriples, which have never been observed in the training data. Our experimental\nresults on the recently published Stanford Visual Relationship dataset, a\nchallenging real world dataset, show that the integration of a semantic model\nusing link prediction methods can significantly improve the results for visual\nrelationship detection. Our combined approach achieves superior performance\ncompared to the state-of-the-art method from the Stanford computer vision\ngroup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00224v1",
            "date": "2018-09-01 17:21:01+00:00",
            "title": "Finding the Answers with Definition Models",
            "authors": [
                "Jack Parry"
            ],
            "abstract": "Inspired by a previous attempt to answer crossword questions using neural\nnetworks (Hill, Cho, Korhonen, & Bengio, 2015), this dissertation implements\nextensions to improve the performance of this existing definition model on the\ntask of answering crossword questions. A discussion and evaluation of the\noriginal implementation finds that there are some ways in which the recurrent\nneural model could be extended. Insights from related fields neural language\nmodeling and neural machine translation provide the justification and means\nrequired for these extensions. Two extensions are applied to the LSTM encoder,\nfirst taking the average of LSTM states across the sequence and secondly using\na bidirectional LSTM, both implementations serve to improve model performance\non a definitions and crossword test set. In order to improve performance on\ncrossword questions, the training data is increased to include crossword\nquestions and answers, and this serves to improve results on definitions as\nwell as crossword questions. The final experiments are conducted using sub-word\nunit segmentation, first on the source side and then later preliminary\nexperimentation is conducted to facilitate character-level output. Initially,\nan exact reproduction of the baseline results proves unsuccessful. Despite\nthis, the extensions improve performance, allowing the definition model to\nsurpass the performance of the recurrent neural network variants of the\nprevious work (Hill, et al., 2015).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.00345v1",
            "date": "2018-09-02 14:29:05+00:00",
            "title": "IntentsKB: A Knowledge Base of Entity-Oriented Search Intents",
            "authors": [
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "We address the problem of constructing a knowledge base of entity-oriented\nsearch intents. Search intents are defined on the level of entity types, each\ncomprising of a high-level intent category (property, website, service, or\nother), along with a cluster of query terms used to express that intent. These\nmachine-readable statements can be leveraged in various applications, e.g., for\ngenerating entity cards or query recommendations. By structuring\nservice-oriented search intents, we take one step towards making entities\nactionable. The main contribution of this paper is a pipeline of components we\ndevelop to construct a knowledge base of entity intents. We evaluate\nperformance both component-wise and end-to-end, and demonstrate that our\napproach is able to generate high-quality data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.00410v1",
            "date": "2018-09-02 23:49:31+00:00",
            "title": "Modeling Topical Coherence in Discourse without Supervision",
            "authors": [
                "Disha Shrivastava",
                "Abhijit Mishra",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "Coherence of text is an important attribute to be measured for both manually\nand automatically generated discourse; but well-defined quantitative metrics\nfor it are still elusive. In this paper, we present a metric for scoring\ntopical coherence of an input paragraph on a real-valued scale by analyzing its\nunderlying topical structure. We first extract all possible topics that the\nsentences of a paragraph of text are related to. Coherence of this text is then\nmeasured by computing: (a) the degree of uncertainty of the topics with respect\nto the paragraph, and (b) the relatedness between these topics. All components\nof our modular framework rely only on unlabeled data and WordNet, thus making\nit completely unsupervised, which is an important feature for general-purpose\nusage of any metric. Experiments are conducted on two datasets - a publicly\navailable dataset for essay grading (representing human discourse), and a\nsynthetic dataset constructed by mixing content from multiple paragraphs\ncovering diverse topics. Our evaluation shows that the measured coherence\nscores are positively correlated with the ground truth for both the datasets.\nFurther validation to our coherence scores is provided by conducting human\nevaluation on the synthetic data, showing a significant agreement of 79.3%",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00414v1",
            "date": "2018-09-03 00:04:49+00:00",
            "title": "Hypernyms Through Intra-Article Organization in Wikipedia",
            "authors": [
                "Disha Shrivastava",
                "Sreyash Kenkre",
                "Santosh Penubothula"
            ],
            "abstract": "We introduce a new measure for unsupervised hypernym detection and\ndirectionality. The motivation is to keep the measure computationally light and\nportatable across languages. We show that the relative physical location of\nwords in explanatory articles captures the directionality property. Further,\nthe phrases in section titles of articles about the word, capture the semantic\nsimilarity needed for hypernym detection task. We experimentally show that the\ncombination of features coming from these two simple measures suffices to\nproduce results comparable with the best unsupervised measures in terms of the\naverage precision.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.00494v1",
            "date": "2018-09-03 08:37:33+00:00",
            "title": "Belittling the Source: Trustworthiness Indicators to Obfuscate Fake News on the Web",
            "authors": [
                "Diego Esteves",
                "Aniketh Janardhan Reddy",
                "Piyush Chawla",
                "Jens Lehmann"
            ],
            "abstract": "With the growth of the internet, the number of fake-news online has been\nproliferating every year. The consequences of such phenomena are manifold,\nranging from lousy decision-making process to bullying and violence episodes.\nTherefore, fact-checking algorithms became a valuable asset. To this aim, an\nimportant step to detect fake-news is to have access to a credibility score for\na given information source. However, most of the widely used Web indicators\nhave either been shut-down to the public (e.g., Google PageRank) or are not\nfree for use (Alexa Rank). Further existing databases are short-manually\ncurated lists of online sources, which do not scale. Finally, most of the\nresearch on the topic is theoretical-based or explore confidential data in a\nrestricted simulation environment. In this paper we explore current research,\nhighlight the challenges and propose solutions to tackle the problem of\nclassifying websites into a credibility scale. The proposed model automatically\nextracts source reputation cues and computes a credibility factor, providing\nvaluable insights which can help in belittling dubious and confirming trustful\nunknown websites. Experimental results outperform state of the art in the\n2-classes and 5-classes setting.",
            "journal ref": "EMNLP 2018: Conference on Empirical Methods in Natural Language\n  Processing (The First Workshop on Fact Extraction and Verification)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.00509v1",
            "date": "2018-09-03 09:07:17+00:00",
            "title": "DeFactoNLP: Fact Verification using Entity Recognition, TFIDF Vector Comparison and Decomposable Attention",
            "authors": [
                "Aniketh Janardhan Reddy",
                "Gil Rocha",
                "Diego Esteves"
            ],
            "abstract": "In this paper, we describe DeFactoNLP, the system we designed for the FEVER\n2018 Shared Task. The aim of this task was to conceive a system that can not\nonly automatically assess the veracity of a claim but also retrieve evidence\nsupporting this assessment from Wikipedia. In our approach, the Wikipedia\ndocuments whose Term Frequency-Inverse Document Frequency (TFIDF) vectors are\nmost similar to the vector of the claim and those documents whose names are\nsimilar to those of the named entities (NEs) mentioned in the claim are\nidentified as the documents which might contain evidence. The sentences in\nthese documents are then supplied to a textual entailment recognition module.\nThis module calculates the probability of each sentence supporting the claim,\ncontradicting the claim or not providing any relevant information to assess the\nveracity of the claim. Various features computed using these probabilities are\nfinally used by a Random Forest classifier to determine the overall\ntruthfulness of the claim. The sentences which support this classification are\nreturned as evidence. Our approach achieved a 0.4277 evidence F1-score, a\n0.5136 label accuracy and a 0.3833 FEVER score.",
            "journal ref": "EMNLP 2018: Conference on Empirical Methods in Natural Language\n  Processing (The First Workshop on Fact Extraction and Verification)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.00653v1",
            "date": "2018-09-03 16:52:19+00:00",
            "title": "Towards Dynamic Computation Graphs via Sparse Latent Structure",
            "authors": [
                "Vlad Niculae",
                "Andr\u00e9 F. T. Martins",
                "Claire Cardie"
            ],
            "abstract": "Deep NLP models benefit from underlying structures in the data---e.g., parse\ntrees---typically extracted using off-the-shelf parsers. Recent attempts to\njointly learn the latent structure encounter a tradeoff: either make\nfactorization assumptions that limit expressiveness, or sacrifice end-to-end\ndifferentiability. Using the recently proposed SparseMAP inference, which\nretrieves a sparse distribution over latent structures, we propose a novel\napproach for end-to-end learning of latent structure predictors jointly with a\ndownstream predictor. To the best of our knowledge, our method is the first to\nenable unrestricted dynamic computation graph construction from the global\nlatent structure, while maintaining differentiability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00656v1",
            "date": "2018-09-03 16:57:54+00:00",
            "title": "Exploring the Landscape of Relational Syllogistic Logics",
            "authors": [
                "Alex Kruckman",
                "Lawrence S. Moss"
            ],
            "abstract": "This paper explores relational syllogistic logics, a family of logical\nsystems related to reasoning about relations in extensions of the classical\nsyllogistic. These are all decidable logical systems. We prove completeness\ntheorems and complexity results for a natural subfamily of relational\nsyllogistic logics, parametrized by constructors for terms and for sentences.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1809.00741v1",
            "date": "2018-09-03 23:13:00+00:00",
            "title": "\"Read My Lips\": Using Automatic Text Analysis to Classify Politicians by Party and Ideology",
            "authors": [
                "Eitan Sapiro-Gheiler"
            ],
            "abstract": "The increasing digitization of political speech has opened the door to\nstudying a new dimension of political behavior using text analysis. This work\ninvestigates the value of word-level statistical data from the US Congressional\nRecord--which contains the full text of all speeches made in the US\nCongress--for studying the ideological positions and behavior of senators.\nApplying machine learning techniques, we use this data to automatically\nclassify senators according to party, obtaining accuracy in the 70-95% range\ndepending on the specific method used. We also show that using text to predict\nDW-NOMINATE scores, a common proxy for ideology, does not improve upon these\nalready-successful results. This classification deteriorates when applied to\ntext from sessions of Congress that are four or more years removed from the\ntraining set, pointing to a need on the part of voters to dynamically update\nthe heuristics they use to evaluate party based on political speech. Text-based\npredictions are less accurate than those based on voting behavior, supporting\nthe theory that roll-call votes represent greater commitment on the part of\npoliticians and are thus a more accurate reflection of their ideological\npreferences. However, the overall success of the machine learning approaches\nstudied here demonstrates that political speeches are highly predictive of\npartisan affiliation. In addition to these findings, this work also introduces\nthe computational tools and methods relevant to the use of political speech\ndata.",
            "journal ref": "None",
            "category": "econ.GN"
        },
        {
            "url": "http://arxiv.org/abs/1809.00794v2",
            "date": "2018-09-04 04:40:34+00:00",
            "title": "Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation",
            "authors": [
                "Zhiting Hu",
                "Haoran Shi",
                "Bowen Tan",
                "Wentao Wang",
                "Zichao Yang",
                "Tiancheng Zhao",
                "Junxian He",
                "Lianhui Qin",
                "Di Wang",
                "Xuezhe Ma",
                "Zhengzhong Liu",
                "Xiaodan Liang",
                "Wangrong Zhu",
                "Devendra Singh Sachan",
                "Eric P. Xing"
            ],
            "abstract": "We introduce Texar, an open-source toolkit aiming to support the broad set of\ntext generation tasks that transform any inputs into natural language, such as\nmachine translation, summarization, dialog, content manipulation, and so forth.\nWith the design goals of modularity, versatility, and extensibility in mind,\nTexar extracts common patterns underlying the diverse tasks and methodologies,\ncreates a library of highly reusable modules, and allows arbitrary model\narchitectures and algorithmic paradigms. In Texar, model architecture,\ninference, and learning processes are properly decomposed. Modules at a high\nconcept level can be freely assembled and plugged in/swapped out. The toolkit\nalso supports a rich set of large-scale pretrained models. Texar is thus\nparticularly suitable for researchers and practitioners to do fast prototyping\nand experimentation. The versatile toolkit also fosters technique sharing\nacross different text generation tasks. Texar supports both TensorFlow and\nPyTorch, and is released under Apache License 2.0 at https://www.texar.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.00832v1",
            "date": "2018-09-04 08:31:21+00:00",
            "title": "Improving the Expressiveness of Deep Learning Frameworks with Recursion",
            "authors": [
                "Eunji Jeong",
                "Joo Seong Jeong",
                "Soojeong Kim",
                "Gyeong-In Yu",
                "Byung-Gon Chun"
            ],
            "abstract": "Recursive neural networks have widely been used by researchers to handle\napplications with recursively or hierarchically structured data. However,\nembedded control flow deep learning frameworks such as TensorFlow, Theano,\nCaffe2, and MXNet fail to efficiently represent and execute such neural\nnetworks, due to lack of support for recursion. In this paper, we add recursion\nto the programming model of existing frameworks by complementing their design\nwith recursive execution of dataflow graphs as well as additional APIs for\nrecursive definitions. Unlike iterative implementations, which can only\nunderstand the topological index of each node in recursive data structures, our\nrecursive implementation is able to exploit the recursive relationships between\nnodes for efficient execution based on parallel computation. We present an\nimplementation on TensorFlow and evaluation results with various recursive\nneural network models, showing that our recursive implementation not only\nconveys the recursive nature of recursive neural networks better than other\nimplementations, but also uses given resources more effectively to reduce\ntraining and inference time.",
            "journal ref": "EuroSys 2018: Thirteenth EuroSys Conference, April 23-26, 2018,\n  Porto, Portugal",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.00836v1",
            "date": "2018-09-04 08:41:53+00:00",
            "title": "A Recurrent Neural Network for Sentiment Quantification",
            "authors": [
                "Andrea Esuli",
                "Alejandro Moreo Fern\u00e1ndez",
                "Fabrizio Sebastiani"
            ],
            "abstract": "Quantification is a supervised learning task that consists in predicting,\ngiven a set of classes C and a set D of unlabelled items, the prevalence (or\nrelative frequency) p(c|D) of each class c in C. Quantification can in\nprinciple be solved by classifying all the unlabelled items and counting how\nmany of them have been attributed to each class. However, this \"classify and\ncount\" approach has been shown to yield suboptimal quantification accuracy;\nthis has established quantification as a task of its own, and given rise to a\nnumber of methods specifically devised for it. We propose a recurrent neural\nnetwork architecture for quantification (that we call QuaNet) that observes the\nclassification predictions to learn higher-order \"quantification embeddings\",\nwhich are then refined by incorporating quantification predictions of simple\nclassify-and-count-like methods. We test {QuaNet on sentiment quantification on\ntext, showing that it substantially outperforms several state-of-the-art\nbaselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.00934v1",
            "date": "2018-08-31 14:45:33+00:00",
            "title": "A Deep Neural Network Sentence Level Classification Method with Context Information",
            "authors": [
                "Xingyi Song",
                "Johann Petrak",
                "Angus Roberts"
            ],
            "abstract": "In the sentence classification task, context formed from sentences adjacent\nto the sentence being classified can provide important information for\nclassification. This context is, however, often ignored. Where methods do make\nuse of context, only small amounts are considered, making it difficult to\nscale. We present a new method for sentence classification, Context-LSTM-CNN,\nthat makes use of potentially large contexts. The method also utilizes\nlong-range dependencies within the sentence being classified, using an LSTM,\nand short-span features, using a stacked CNN. Our experiments demonstrate that\nthis approach consistently improves over previous methods on two different\ndatasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01201v2",
            "date": "2018-09-04 19:06:18+00:00",
            "title": "Random Language Model",
            "authors": [
                "E. DeGiuli"
            ],
            "abstract": "Many complex generative systems use languages to create structured objects.\nWe consider a model of random languages, defined by weighted context-free\ngrammars. As the distribution of grammar weights broadens, a transition is\nfound from a random phase, in which sentences are indistinguishable from noise,\nto an organized phase in which nontrivial information is carried. This marks\nthe emergence of deep structure in the language, and can be understood by a\ncompetition between energy and entropy.",
            "journal ref": "Phys. Rev. Lett. 122, 128301 (2019)",
            "category": "cond-mat.dis-nn"
        },
        {
            "url": "http://arxiv.org/abs/1809.01229v1",
            "date": "2018-09-04 20:09:01+00:00",
            "title": "t-Exponential Memory Networks for Question-Answering Machines",
            "authors": [
                "Kyriakos Tolias",
                "Sotirios Chatzis"
            ],
            "abstract": "Recent advances in deep learning have brought to the fore models that can\nmake multiple computational steps in the service of completing a task; these\nare capable of describ- ing long-term dependencies in sequential data. Novel\nrecurrent attention models over possibly large external memory modules\nconstitute the core mechanisms that enable these capabilities. Our work\naddresses learning subtler and more complex underlying temporal dynamics in\nlanguage modeling tasks that deal with sparse sequential data. To this end, we\nimprove upon these recent advances, by adopting concepts from the field of\nBayesian statistics, namely variational inference. Our proposed approach\nconsists in treating the network parameters as latent variables with a prior\ndistribution imposed over them. Our statistical assumptions go beyond the\nstandard practice of postulating Gaussian priors. Indeed, to allow for handling\noutliers, which are prevalent in long observed sequences of multivariate data,\nmultivariate t-exponential distributions are imposed. On this basis, we proceed\nto infer corresponding posteriors; these can be used for inference and\nprediction at test time, in a way that accounts for the uncertainty in the\navailable sparse training data. Specifically, to allow for our approach to best\nexploit the merits of the t-exponential family, our method considers a new\nt-divergence measure, which generalizes the concept of the Kullback-Leibler\ndivergence. We perform an extensive experimental evaluation of our approach,\nusing challenging language modeling benchmarks, and illustrate its superiority\nover existing state-of-the-art techniques.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.01272v1",
            "date": "2018-09-04 23:22:28+00:00",
            "title": "Unsupervised Statistical Machine Translation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre"
            ],
            "abstract": "While modern machine translation has relied on large parallel corpora, a\nrecent line of work has managed to train Neural Machine Translation (NMT)\nsystems from monolingual corpora only (Artetxe et al., 2018c; Lample et al.,\n2018). Despite the potential of this approach for low-resource settings,\nexisting systems are far behind their supervised counterparts, limiting their\npractical interest. In this paper, we propose an alternative approach based on\nphrase-based Statistical Machine Translation (SMT) that significantly closes\nthe gap with supervised systems. Our method profits from the modular\narchitecture of SMT: we first induce a phrase table from monolingual corpora\nthrough cross-lingual embedding mappings, combine it with an n-gram language\nmodel, and fine-tune hyperparameters through an unsupervised MERT variant. In\naddition, iterative backtranslation improves results further, yielding, for\ninstance, 14.08 and 26.22 BLEU points in WMT 2014 English-German and\nEnglish-French, respectively, an improvement of more than 7-10 BLEU points over\nprevious unsupervised systems, and closing the gap with supervised SMT (Moses\ntrained on Europarl) down to 2-5 BLEU points. Our implementation is available\nat https://github.com/artetxem/monoses",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01316v3",
            "date": "2018-09-05 04:15:13+00:00",
            "title": "Learning User Preferences and Understanding Calendar Contexts for Event Scheduling",
            "authors": [
                "Donghyeon Kim",
                "Jinhyuk Lee",
                "Donghee Choi",
                "Jaehoon Choi",
                "Jaewoo Kang"
            ],
            "abstract": "With online calendar services gaining popularity worldwide, calendar data has\nbecome one of the richest context sources for understanding human behavior.\nHowever, event scheduling is still time-consuming even with the development of\nonline calendars. Although machine learning based event scheduling models have\nautomated scheduling processes to some extent, they often fail to understand\nsubtle user preferences and complex calendar contexts with event titles written\nin natural language. In this paper, we propose Neural Event Scheduling\nAssistant (NESA) which learns user preferences and understands calendar\ncontexts, directly from raw online calendars for fully automated and highly\neffective event scheduling. We leverage over 593K calendar events for NESA to\nlearn scheduling personal events, and we further utilize NESA for\nmulti-attendee event scheduling. NESA successfully incorporates deep neural\nnetworks such as Bidirectional Long Short-Term Memory, Convolutional Neural\nNetwork, and Highway Network for learning the preferences of each user and\nunderstanding calendar context based on natural languages. The experimental\nresults show that NESA significantly outperforms previous baseline models in\nterms of various evaluation metrics on both personal and multi-attendee event\nscheduling tasks. Our qualitative analysis demonstrates the effectiveness of\neach layer in NESA and learned user preferences.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.01341v2",
            "date": "2018-09-05 06:07:31+00:00",
            "title": "Embedding Multimodal Relational Data for Knowledge Base Completion",
            "authors": [
                "Pouya Pezeshkpour",
                "Liyan Chen",
                "Sameer Singh"
            ],
            "abstract": "Representing entities and relations in an embedding space is a well-studied\napproach for machine learning on relational data. Existing approaches, however,\nprimarily focus on simple link structure between a finite set of entities,\nignoring the variety of data types that are often used in knowledge bases, such\nas text, images, and numerical values. In this paper, we propose multimodal\nknowledge base embeddings (MKBE) that use different neural encoders for this\nvariety of observed data, and combine them with existing relational models to\nlearn embeddings of the entities and multimodal data. Further, using these\nlearned embedings and different neural decoders, we introduce a novel\nmultimodal imputation model to generate missing multimodal values, like text\nand images, from information in the knowledge base. We enrich existing\nrelational datasets to create two novel benchmarks that contain additional\ninformation such as textual descriptions and images of the original entities.\nWe demonstrate that our models utilize this additional information effectively\nto provide more accurate link prediction, achieving state-of-the-art results\nwith a considerable gap of 5-7% over existing methods. Further, we evaluate the\nquality of our generated multimodal values via a user study. We have release\nthe datasets and the open-source implementation of our models at\nhttps://github.com/pouyapez/mkbe",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.01477v1",
            "date": "2018-08-31 19:31:05+00:00",
            "title": "A Supervised Learning Approach For Heading Detection",
            "authors": [
                "Sahib Singh Budhiraja",
                "Vijay Mago"
            ],
            "abstract": "As the Portable Document Format (PDF) file format increases in popularity,\nresearch in analysing its structure for text extraction and analysis is\nnecessary. Detecting headings can be a crucial component of classifying and\nextracting meaningful data. This research involves training a supervised\nlearning model to detect headings with features carefully selected through\nrecursive feature elimination. The best performing classifier had an accuracy\nof 96.95%, sensitivity of 0.986 and a specificity of 0.953. This research into\nheading detection contributes to the field of PDF based text extraction and can\nbe applied to the automation of large scale PDF text analysis in a variety of\nprofessional and policy based contexts.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01478v2",
            "date": "2018-09-02 02:56:25+00:00",
            "title": "Weakly-Supervised Neural Text Classification",
            "authors": [
                "Yu Meng",
                "Jiaming Shen",
                "Chao Zhang",
                "Jiawei Han"
            ],
            "abstract": "Deep neural networks are gaining increasing popularity for the classic text\nclassification task, due to their strong expressive power and less requirement\nfor feature engineering. Despite such attractiveness, neural text\nclassification models suffer from the lack of training data in many real-world\napplications. Although many semi-supervised and weakly-supervised text\nclassification models exist, they cannot be easily applied to deep neural\nmodels and meanwhile support limited supervision types. In this paper, we\npropose a weakly-supervised method that addresses the lack of training data in\nneural text classification. Our method consists of two modules: (1) a\npseudo-document generator that leverages seed information to generate\npseudo-labeled documents for model pre-training, and (2) a self-training module\nthat bootstraps on real unlabeled data for model refinement. Our method has the\nflexibility to handle different types of weak supervision and can be easily\nintegrated into existing deep neural models for text classification. We have\nperformed extensive experiments on three real-world datasets from different\ndomains. The results demonstrate that our proposed method achieves inspiring\nperformance without requiring excessive training data and outperforms baseline\nmethods significantly.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01479v5",
            "date": "2018-09-03 14:06:11+00:00",
            "title": "UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification",
            "authors": [
                "Andreas Hanselowski",
                "Hao Zhang",
                "Zile Li",
                "Daniil Sorokin",
                "Benjamin Schiller",
                "Claudia Schulz",
                "Iryna Gurevych"
            ],
            "abstract": "The Fact Extraction and VERification (FEVER) shared task was launched to\nsupport the development of systems able to verify claims by extracting\nsupporting or refuting facts from raw text. The shared task organizers provide\na large-scale dataset for the consecutive steps involved in claim verification,\nin particular, document retrieval, fact extraction, and claim classification.\nIn this paper, we present our claim verification pipeline approach, which,\naccording to the preliminary results, scored third in the shared task, out of\n23 competing systems. For the document retrieval, we implemented a new entity\nlinking approach. In order to be able to rank candidate facts and classify a\nclaim on the basis of several selected facts, we introduce two extensions to\nthe Enhanced LSTM (ESIM).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01494v1",
            "date": "2018-08-28 19:44:51+00:00",
            "title": "Interpretation of Natural Language Rules in Conversational Machine Reading",
            "authors": [
                "Marzieh Saeidi",
                "Max Bartolo",
                "Patrick Lewis",
                "Sameer Singh",
                "Tim Rockt\u00e4schel",
                "Mike Sheldon",
                "Guillaume Bouchard",
                "Sebastian Riedel"
            ],
            "abstract": "Most work in machine reading focuses on question answering problems where the\nanswer is directly expressed in the text to read. However, many real-world\nquestion answering problems require the reading of text not because it contains\nthe literal answer, but because it contains a recipe to derive an answer\ntogether with the reader's background knowledge. One example is the task of\ninterpreting regulations to answer \"Can I...?\" or \"Do I have to...?\" questions\nsuch as \"I am working in Canada. Do I have to carry on paying UK National\nInsurance?\" after reading a UK government website about this topic. This task\nrequires both the interpretation of rules and the application of background\nknowledge. It is further complicated due to the fact that, in practice, most\nquestions are underspecified, and a human assistant will regularly have to ask\nclarification questions such as \"How long have you been working abroad?\" when\nthe answer cannot be directly derived from the question and text. In this\npaper, we formalise this task and develop a crowd-sourcing strategy to collect\n32k task instances based on real-world rules and crowd-generated questions and\nscenarios. We analyse the challenges of this task and assess its difficulty by\nevaluating the performance of rule-based and machine-learning baselines. We\nobserve promising results when no background knowledge is necessary, and\nsubstantial room for improvement whenever background knowledge is needed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01495v1",
            "date": "2018-08-29 15:11:49+00:00",
            "title": "A Reinforcement Learning-driven Translation Model for Search-Oriented Conversational Systems",
            "authors": [
                "Wafa Aissa",
                "Laure Soulier",
                "Ludovic Denoyer"
            ],
            "abstract": "Search-oriented conversational systems rely on information needs expressed in\nnatural language (NL). We focus here on the understanding of NL expressions for\nbuilding keyword-based queries. We propose a reinforcement-learning-driven\ntranslation model framework able to 1) learn the translation from NL\nexpressions to queries in a supervised way, and, 2) to overcome the lack of\nlarge-scale dataset by framing the translation model as a word selection\napproach and injecting relevance feedback in the learning process. Experiments\nare carried out on two TREC datasets and outline the effectiveness of our\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01496v1",
            "date": "2018-08-29 21:11:09+00:00",
            "title": "Learning Gender-Neutral Word Embeddings",
            "authors": [
                "Jieyu Zhao",
                "Yichao Zhou",
                "Zeyu Li",
                "Wei Wang",
                "Kai-Wei Chang"
            ],
            "abstract": "Word embedding models have become a fundamental component in a wide range of\nNatural Language Processing (NLP) applications. However, embeddings trained on\nhuman-generated corpora have been demonstrated to inherit strong gender\nstereotypes that reflect social constructs. To address this concern, in this\npaper, we propose a novel training procedure for learning gender-neutral word\nembeddings. Our approach aims to preserve gender information in certain\ndimensions of word vectors while compelling other dimensions to be free of\ngender influence. Based on the proposed method, we generate a Gender-Neutral\nvariant of GloVe (GN-GloVe). Quantitative and qualitative experiments\ndemonstrate that GN-GloVe successfully isolates gender information without\nsacrificing the functionality of the embedding model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01498v2",
            "date": "2018-08-30 13:54:45+00:00",
            "title": "Skip-gram word embeddings in hyperbolic space",
            "authors": [
                "Matthias Leimeister",
                "Benjamin J. Wilson"
            ],
            "abstract": "Recent work has demonstrated that embeddings of tree-like graphs in\nhyperbolic space surpass their Euclidean counterparts in performance by a large\nmargin. Inspired by these results and scale-free structure in the word\nco-occurrence graph, we present an algorithm for learning word embeddings in\nhyperbolic space from free text. An objective function based on the hyperbolic\ndistance is derived and included in the skip-gram negative-sampling\narchitecture of word2vec. The hyperbolic word embeddings are then evaluated on\nword similarity and analogy benchmarks. The results demonstrate the potential\nof hyperbolic word embeddings, particularly in low dimensions, though without\nclear superiority over their Euclidean counterparts. We further discuss\nsubtleties in the formulation of the analogy task in curved spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01499v2",
            "date": "2018-09-01 00:15:30+00:00",
            "title": "Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts",
            "authors": [
                "Samuel Carton",
                "Qiaozhu Mei",
                "Paul Resnick"
            ],
            "abstract": "We introduce an adversarial method for producing high-recall explanations of\nneural text classifier decisions. Building on an existing architecture for\nextractive explanations via hard attention, we add an adversarial layer which\nscans the residual of the attention for remaining predictive signal. Motivated\nby the important domain of detecting personal attacks in social media comments,\nwe additionally demonstrate the importance of manually setting a semantically\nappropriate `default' behavior for the model by explicitly manipulating its\nbias term. We develop a validation set of human-annotated personal attacks to\nevaluate the impact of these changes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01534v1",
            "date": "2018-09-05 16:44:04+00:00",
            "title": "Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models",
            "authors": [
                "Daniel Watson",
                "Nasser Zalmout",
                "Nizar Habash"
            ],
            "abstract": "Text normalization is an important enabling technology for several NLP tasks.\nRecently, neural-network-based approaches have outperformed well-established\nmodels in this task. However, in languages other than English, there has been\nlittle exploration in this direction. Both the scarcity of annotated data and\nthe complexity of the language increase the difficulty of the problem. To\naddress these challenges, we use a sequence-to-sequence model with\ncharacter-based attention, which in addition to its self-learned character\nembeddings, uses word embeddings pre-trained with an approach that also models\nsubword information. This provides the neural model with access to more\nlinguistic information especially suitable for text normalization, without\nlarge parallel corpora. We show that providing the model with word-level\nfeatures bridges the gap for the neural network approach to achieve a\nstate-of-the-art F1 score on a standard Arabic language correction shared task\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01696v2",
            "date": "2018-09-05 19:14:11+00:00",
            "title": "TVQA: Localized, Compositional Video Question Answering",
            "authors": [
                "Jie Lei",
                "Licheng Yu",
                "Mohit Bansal",
                "Tamara L. Berg"
            ],
            "abstract": "Recent years have witnessed an increasing interest in image-based\nquestion-answering (QA) tasks. However, due to data limitations, there has been\nmuch less work on video-based QA. In this paper, we present TVQA, a large-scale\nvideo QA dataset based on 6 popular TV shows. TVQA consists of 152,545 QA pairs\nfrom 21,793 clips, spanning over 460 hours of video. Questions are designed to\nbe compositional in nature, requiring systems to jointly localize relevant\nmoments within a clip, comprehend subtitle-based dialogue, and recognize\nrelevant visual concepts. We provide analyses of this new dataset as well as\nseveral baselines and a multi-stream end-to-end trainable neural network\nframework for the TVQA task. The dataset is publicly available at\nhttp://tvqa.cs.unc.edu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01771v1",
            "date": "2018-09-06 00:31:51+00:00",
            "title": "An Analysis of Hierarchical Text Classification Using Word Embeddings",
            "authors": [
                "Roger A. Stein",
                "Patricia A. Jaques",
                "Joao F. Valiati"
            ],
            "abstract": "Efficient distributed numerical word representation models (word embeddings)\ncombined with modern machine learning algorithms have recently yielded\nconsiderable improvement on automatic document classification tasks. However,\nthe effectiveness of such techniques has not been assessed for the hierarchical\ntext classification (HTC) yet. This study investigates the application of those\nmodels and algorithms on this specific problem by means of experimentation and\nanalysis. We trained classification models with prominent machine learning\nalgorithm implementations---fastText, XGBoost, SVM, and Keras' CNN---and\nnoticeable word embeddings generation methods---GloVe, word2vec, and\nfastText---with publicly available data and evaluated them with measures\nspecifically appropriate for the hierarchical context. FastText achieved an\n${}_{LCA}F_1$ of 0.893 on a single-labeled version of the RCV1 dataset. An\nanalysis indicates that using word embeddings and its flavors is a very\npromising approach for HTC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01812v1",
            "date": "2018-09-06 04:11:46+00:00",
            "title": "Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency",
            "authors": [
                "Zhuang Ma",
                "Michael Collins"
            ],
            "abstract": "Noise Contrastive Estimation (NCE) is a powerful parameter estimation method\nfor log-linear models, which avoids calculation of the partition function or\nits derivatives at each training step, a computationally demanding step in many\ncases. It is closely related to negative sampling methods, now widely used in\nNLP. This paper considers NCE-based estimation of conditional models.\nConditional models are frequently encountered in practice; however there has\nnot been a rigorous theoretical analysis of NCE in this setting, and we will\nargue there are subtle but important questions when generalizing NCE to the\nconditional case. In particular, we analyze two variants of NCE for conditional\nmodels: one based on a classification objective, the other based on a ranking\nobjective. We show that the ranking-based variant of NCE gives consistent\nparameter estimates under weaker assumptions than the classification-based\nmethod; we analyze the statistical efficiency of the ranking-based and\nclassification-based variants of NCE; finally we describe experiments on\nsynthetic data and language modeling showing the effectiveness and trade-offs\nof both methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.01816v1",
            "date": "2018-09-06 04:36:22+00:00",
            "title": "Visual Coreference Resolution in Visual Dialog using Neural Module Networks",
            "authors": [
                "Satwik Kottur",
                "Jos\u00e9 M. F. Moura",
                "Devi Parikh",
                "Dhruv Batra",
                "Marcus Rohrbach"
            ],
            "abstract": "Visual dialog entails answering a series of questions grounded in an image,\nusing dialog history as context. In addition to the challenges found in visual\nquestion answering (VQA), which can be seen as one-round dialog, visual dialog\nencompasses several more. We focus on one such problem called visual\ncoreference resolution that involves determining which words, typically noun\nphrases and pronouns, co-refer to the same entity/object instance in an image.\nThis is crucial, especially for pronouns (e.g., `it'), as the dialog agent must\nfirst link it to a previous coreference (e.g., `boat'), and only then can rely\non the visual grounding of the coreference `boat' to reason about the pronoun\n`it'. Prior work (in visual dialog) models visual coreference resolution either\n(a) implicitly via a memory network over history, or (b) at a coarse level for\nthe entire question; and not explicitly at a phrase level of granularity. In\nthis work, we propose a neural module network architecture for visual dialog by\nintroducing two novel modules - Refer and Exclude - that perform explicit,\ngrounded, coreference resolution at a finer word level. We demonstrate the\neffectiveness of our model on MNIST Dialog, a visually simple yet\ncoreference-wise complex dataset, by achieving near perfect accuracy, and on\nVisDial, a large and challenging visual dialog dataset on real images, where\nour model outperforms other approaches, and is more interpretable, grounded,\nand consistent qualitatively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.01943v1",
            "date": "2018-09-06 12:26:24+00:00",
            "title": "Cascaded Mutual Modulation for Visual Reasoning",
            "authors": [
                "Yiqun Yao",
                "Jiaming Xu",
                "Feng Wang",
                "Bo Xu"
            ],
            "abstract": "Visual reasoning is a special visual question answering problem that is\nmulti-step and compositional by nature, and also requires intensive text-vision\ninteractions. We propose CMM: Cascaded Mutual Modulation as a novel end-to-end\nvisual reasoning model. CMM includes a multi-step comprehension process for\nboth question and image. In each step, we use a Feature-wise Linear Modulation\n(FiLM) technique to enable textual/visual pipeline to mutually control each\nother. Experiments show that CMM significantly outperforms most related models,\nand reach state-of-the-arts on two visual reasoning benchmarks: CLEVR and NLVR,\ncollected from both synthetic and natural languages. Ablation studies confirm\nthat both our multistep framework and our visual-guided language modulation are\ncritical to the task. Our code is available at\nhttps://github.com/FlamingHorizon/CMM-VR.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.01997v2",
            "date": "2018-09-06 13:57:03+00:00",
            "title": "Dual Ask-Answer Network for Machine Reading Comprehension",
            "authors": [
                "Han Xiao",
                "Feng Wang",
                "Jianfeng Yan",
                "Jingyao Zheng"
            ],
            "abstract": "There are three modalities in the reading comprehension setting: question,\nanswer and context. The task of question answering or question generation aims\nto infer an answer or a question when given the counterpart based on context.\nWe present a novel two-way neural sequence transduction model that connects\nthree modalities, allowing it to learn two tasks simultaneously and mutually\nbenefit one another. During training, the model receives\nquestion-context-answer triplets as input and captures the cross-modal\ninteraction via a hierarchical attention process. Unlike previous joint\nlearning paradigms that leverage the duality of question generation and\nquestion answering at data level, we solve such dual tasks at the architecture\nlevel by mirroring the network structure and partially sharing components at\ndifferent layers. This enables the knowledge to be transferred from one task to\nanother, helping the model to find a general representation for each modality.\nThe evaluation on four public datasets shows that our dual-learning model\noutperforms the mono-learning counterpart as well as the state-of-the-art joint\nmodels on both question answering and question generation tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.02094v1",
            "date": "2018-09-06 17:08:21+00:00",
            "title": "Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "I\u00f1igo Lopez-Gazpio",
                "Eneko Agirre"
            ],
            "abstract": "Following the recent success of word embeddings, it has been argued that\nthere is no such thing as an ideal representation for words, as different\nmodels tend to capture divergent and often mutually incompatible aspects like\nsemantics/syntax and similarity/relatedness. In this paper, we show that each\nembedding model captures more information than directly apparent. A linear\ntransformation that adjusts the similarity order of the model without any\nexternal resource can tailor it to achieve better results in those aspects,\nproviding a new perspective on how embeddings encode divergent linguistic\ninformation. In addition, we explore the relation between intrinsic and\nextrinsic evaluation, as the effect of our transformations in downstream tasks\nis higher for unsupervised systems than for supervised ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02251v2",
            "date": "2018-09-06 23:42:21+00:00",
            "title": "Adversarial Feature-Mapping for Speech Enhancement",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yifan Gong",
                "Biing-Hwang",
                "Juang"
            ],
            "abstract": "Feature-mapping with deep neural networks is commonly used for single-channel\nspeech enhancement, in which a feature-mapping network directly transforms the\nnoisy features to the corresponding enhanced ones and is trained to minimize\nthe mean square errors between the enhanced and clean features. In this paper,\nwe propose an adversarial feature-mapping (AFM) method for speech enhancement\nwhich advances the feature-mapping approach with adversarial learning. An\nadditional discriminator network is introduced to distinguish the enhanced\nfeatures from the real clean ones. The two networks are jointly optimized to\nminimize the feature-mapping loss and simultaneously mini-maximize the\ndiscrimination loss. The distribution of the enhanced features is further\npushed towards that of the clean features through this adversarial multi-task\ntraining. To achieve better performance on ASR task, senone-aware (SA) AFM is\nfurther proposed in which an acoustic model network is jointly trained with the\nfeature-mapping and discriminator networks to optimize the senone\nclassification loss in addition to the AFM losses. Evaluated on the CHiME-3\ndataset, the proposed AFM achieves 16.95% and 5.27% relative word error rate\n(WER) improvements over the real noisy data and the feature-mapping baseline\nrespectively and the SA-AFM achieves 9.85% relative WER improvement over the\nmulti-conditional acoustic model.",
            "journal ref": "Interspeech 2018",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1809.02253v2",
            "date": "2018-09-06 23:55:49+00:00",
            "title": "Cycle-Consistent Speech Enhancement",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yifan Gong",
                "Biing-Hwang",
                "Juang"
            ],
            "abstract": "Feature mapping using deep neural networks is an effective approach for\nsingle-channel speech enhancement. Noisy features are transformed to the\nenhanced ones through a mapping network and the mean square errors between the\nenhanced and clean features are minimized. In this paper, we propose a\ncycle-consistent speech enhancement (CSE) in which an additional inverse\nmapping network is introduced to reconstruct the noisy features from the\nenhanced ones. A cycle-consistent constraint is enforced to minimize the\nreconstruction loss. Similarly, a backward cycle of mappings is performed in\nthe opposite direction with the same networks and losses. With\ncycle-consistency, the speech structure is well preserved in the enhanced\nfeatures while noise is effectively reduced such that the feature-mapping\nnetwork generalizes better to unseen data. In cases where only unparalleled\nnoisy and clean data is available for training, two discriminator networks are\nused to distinguish the enhanced and noised features from the clean and noisy\nones. The discrimination losses are jointly optimized with reconstruction\nlosses through adversarial multi-task learning. Evaluated on the CHiME-3\ndataset, the proposed CSE achieves 19.60% and 6.69% relative word error rate\nimprovements respectively when using or without using parallel clean and noisy\nspeech data.",
            "journal ref": "Interspeech 2018",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1809.02306v1",
            "date": "2018-09-07 04:17:40+00:00",
            "title": "Unsupervised Cross-lingual Word Embedding by Multilingual Neural Language Models",
            "authors": [
                "Takashi Wada",
                "Tomoharu Iwata"
            ],
            "abstract": "We propose an unsupervised method to obtain cross-lingual embeddings without\nany parallel data or pre-trained word embeddings. The proposed model, which we\ncall multilingual neural language models, takes sentences of multiple languages\nas an input. The proposed model contains bidirectional LSTMs that perform as\nforward and backward language models, and these networks are shared among all\nthe languages. The other parameters, i.e. word embeddings and linear\ntransformation between hidden states and outputs, are specific to each\nlanguage. The shared LSTMs can capture the common sentence structure among all\nlanguages. Accordingly, word embeddings of each language are mapped into a\ncommon latent space, making it possible to measure the similarity of words\nacross multiple languages. We evaluate the quality of the cross-lingual word\nembeddings on a word alignment task. Our experiments demonstrate that our model\ncan obtain cross-lingual embeddings of much higher quality than existing\nunsupervised models when only a small amount of monolingual data (i.e. 50k\nsentences) are available, or the domains of monolingual data are different\nacross languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02343v1",
            "date": "2018-09-07 08:18:01+00:00",
            "title": "Exploiting local and global performance of candidate systems for aggregation of summarization techniques",
            "authors": [
                "Parth Mehta",
                "Prasenjit Majumder"
            ],
            "abstract": "With an ever growing number of extractive summarization techniques being\nproposed, there is less clarity then ever about how good each system is\ncompared to the rest. Several studies highlight the variance in performance of\nthese systems with change in datasets or even across documents within the same\ncorpus. An effective way to counter this variance and to make the systems more\nrobust could be to use inputs from multiple systems when generating a summary.\nIn the present work, we define a novel way of creating such ensemble by\nexploiting similarity between the content of candidate summaries to estimate\ntheir reliability. We define GlobalRank which captures the performance of a\ncandidate system on an overall corpus and LocalRank which estimates its\nperformance on a given document cluster. We then use these two scores to assign\na weight to each individual systems, which is then used to generate the new\naggregate ranking. Experiments on DUC2003 and DUC 2004 datasets show a\nsignificant improvement in terms of ROUGE score, over existing sate-of-art\ntechniques.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.02393v2",
            "date": "2018-09-07 10:35:42+00:00",
            "title": "Improving Neural Question Generation using Answer Separation",
            "authors": [
                "Yanghoon Kim",
                "Hwanhee Lee",
                "Joongbo Shin",
                "Kyomin Jung"
            ],
            "abstract": "Neural question generation (NQG) is the task of generating a question from a\ngiven passage with deep neural networks. Previous NQG models suffer from a\nproblem that a significant proportion of the generated questions include words\nin the question target, resulting in the generation of unintended questions. In\nthis paper, we propose answer-separated seq2seq, which better utilizes the\ninformation from both the passage and the target answer. By replacing the\ntarget answer in the original passage with a special token, our model learns to\nidentify which interrogative word should be used. We also propose a new module\ntermed keyword-net, which helps the model better capture the key information in\nthe target answer and generate an appropriate question. Experimental results\ndemonstrate that our answer separation method significantly reduces the number\nof improper questions which include answers. Consequently, our model\nsignificantly outperforms previous state-of-the-art NQG models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02637v2",
            "date": "2018-09-07 18:47:20+00:00",
            "title": "Neural Generation of Diverse Questions using Answer Focus, Contextual and Linguistic Features",
            "authors": [
                "Vrindavan Harrison",
                "Marilyn Walker"
            ],
            "abstract": "Question Generation is the task of automatically creating questions from\ntextual input. In this work we present a new Attentional Encoder--Decoder\nRecurrent Neural Network model for automatic question generation. Our model\nincorporates linguistic features and an additional sentence embedding to\ncapture meaning at both sentence and word levels. The linguistic features are\ndesigned to capture information related to named entity recognition, word case,\nand entity coreference resolution. In addition our model uses a copying\nmechanism and a special answer signal that enables generation of numerous\ndiverse questions on a given sentence. Our model achieves state of the art\nresults of 19.98 Bleu_4 on a benchmark Question Generation dataset,\noutperforming all previously published results by a significant margin. A human\nevaluation also shows that these added features improve the quality of the\ngenerated questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.02731v3",
            "date": "2018-09-08 01:20:45+00:00",
            "title": "Exploiting Invertible Decoders for Unsupervised Sentence Representation Learning",
            "authors": [
                "Shuai Tang",
                "Virginia R. de Sa"
            ],
            "abstract": "The encoder-decoder models for unsupervised sentence representation learning\ntend to discard the decoder after being trained on a large unlabelled corpus,\nsince only the encoder is needed to map the input sentence into a vector\nrepresentation. However, parameters learnt in the decoder also contain useful\ninformation about language. In order to utilise the decoder after learning, we\npresent two types of decoding functions whose inverse can be easily derived\nwithout expensive inverse calculation. Therefore, the inverse of the decoding\nfunction serves as another encoder that produces sentence representations. We\nshow that, with careful design of the decoding functions, the model learns good\nsentence representations, and the ensemble of the representations produced from\nthe encoder and the inverse of the decoder demonstrate even better\ngeneralisation ability and solid transferability.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1809.02836v1",
            "date": "2018-09-08 17:04:53+00:00",
            "title": "Context-Free Transductions with Neural Stacks",
            "authors": [
                "Yiding Hao",
                "William Merrill",
                "Dana Angluin",
                "Robert Frank",
                "Noah Amsel",
                "Andrew Benz",
                "Simon Mendelsohn"
            ],
            "abstract": "This paper analyzes the behavior of stack-augmented recurrent neural network\n(RNN) models. Due to the architectural similarity between stack RNNs and\npushdown transducers, we train stack RNN models on a number of tasks, including\nstring reversal, context-free language modelling, and cumulative XOR\nevaluation. Examining the behavior of our networks, we show that\nstack-augmented RNNs can discover intuitive stack-based strategies for solving\nour tasks. However, stack RNNs are more difficult to train than classical\narchitectures such as LSTMs. Rather than employ stack-based strategies, more\ncomplex networks often find approximate solutions by using the stack as\nunstructured memory.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1809.03044v1",
            "date": "2018-09-09 21:08:57+00:00",
            "title": "How clever is the FiLM model, and how clever can it be?",
            "authors": [
                "Alexander Kuhnle",
                "Huiyuan Xie",
                "Ann Copestake"
            ],
            "abstract": "The FiLM model achieves close-to-perfect performance on the diagnostic CLEVR\ndataset and is distinguished from other such models by having a comparatively\nsimple and easily transferable architecture. In this paper, we investigate in\nmore detail the ability of FiLM to learn various linguistic constructions. Our\nmain results show that (a) FiLM is not able to learn relational statements\nstraight away except for very simple instances, (b) training on a broader set\nof instances as well as pretraining on simpler instance types can help\nalleviate these learning difficulties, (c) mixing is less robust than\npretraining and very sensitive to the compositional structure of the dataset.\nOverall, our results suggest that the approach of big all-encompassing datasets\nand the paradigm of \"the effectiveness of data\" may have fundamental\nlimitations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03051v1",
            "date": "2018-09-09 22:33:20+00:00",
            "title": "Attentional Multi-Reading Sarcasm Detection",
            "authors": [
                "Reza Ghaeini",
                "Xiaoli Z. Fern",
                "Prasad Tadepalli"
            ],
            "abstract": "Recognizing sarcasm often requires a deep understanding of multiple sources\nof information, including the utterance, the conversational context, and real\nworld facts. Most of the current sarcasm detection systems consider only the\nutterance in isolation. There are some limited attempts toward taking into\naccount the conversational context. In this paper, we propose an interpretable\nend-to-end model that combines information from both the utterance and the\nconversational context to detect sarcasm, and demonstrate its effectiveness\nthrough empirical evaluations. We also study the behavior of the proposed model\nto provide explanations for the model's decisions. Importantly, our model is\ncapable of determining the impact of utterance and conversational context on\nthe model's decisions. Finally, we provide an ablation study to illustrate the\nimpact of different components of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03194v3",
            "date": "2018-09-10 09:10:25+00:00",
            "title": "Improving Response Selection in Multi-Turn Dialogue Systems by Incorporating Domain Knowledge",
            "authors": [
                "Debanjan Chaudhuri",
                "Agustinus Kristiadi",
                "Jens Lehmann",
                "Asja Fischer"
            ],
            "abstract": "Building systems that can communicate with humans is a core problem in\nArtificial Intelligence. This work proposes a novel neural network architecture\nfor response selection in an end-to-end multi-turn conversational dialogue\nsetting. The architecture applies context level attention and incorporates\nadditional external knowledge provided by descriptions of domain-specific\nwords. It uses a bi-directional Gated Recurrent Unit (GRU) for encoding context\nand responses and learns to attend over the context words given the latent\nresponse representation and vice versa.In addition, it incorporates external\ndomain specific information using another GRU for encoding the domain keyword\ndescriptions. This allows better representation of domain-specific keywords in\nresponses and hence improves the overall performance. Experimental results show\nthat our model outperforms all other state-of-the-art methods for response\nselection in multi-turn conversations.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.03416v2",
            "date": "2018-09-10 15:55:15+00:00",
            "title": "Identifying Relationships Among Sentences in Court Case Transcripts Using Discourse Relations",
            "authors": [
                "Gathika Ratnayaka",
                "Thejan Rupasinghe",
                "Nisansa de Silva",
                "Menuka Warushavithana",
                "Viraj Gamage",
                "Amal Shehan Perera"
            ],
            "abstract": "Case Law has a significant impact on the proceedings of legal cases.\nTherefore, the information that can be obtained from previous court cases is\nvaluable to lawyers and other legal officials when performing their duties.\nThis paper describes a methodology of applying discourse relations between\nsentences when processing text documents related to the legal domain. In this\nstudy, we developed a mechanism to classify the relationships that can be\nobserved among sentences in transcripts of United States court cases. First, we\ndefined relationship types that can be observed between sentences in court case\ntranscripts. Then we classified pairs of sentences according to the\nrelationship type by combining a machine learning model and a rule-based\napproach. The results obtained through our system were evaluated using human\njudges. To the best of our knowledge, this is the first study where discourse\nrelationships between sentences have been used to determine relationships among\nsentences in legal court case transcripts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.03707v2",
            "date": "2018-09-11 07:22:28+00:00",
            "title": "Answering Visual What-If Questions: From Actions to Predicted Scene Descriptions",
            "authors": [
                "M. Wagner",
                "H. Basevi",
                "R. Shetty",
                "W. Li",
                "M. Malinowski",
                "M. Fritz",
                "A. Leonardis"
            ],
            "abstract": "In-depth scene descriptions and question answering tasks have greatly\nincreased the scope of today's definition of scene understanding. While such\ntasks are in principle open ended, current formulations primarily focus on\ndescribing only the current state of the scenes under consideration. In\ncontrast, in this paper, we focus on the future states of the scenes which are\nalso conditioned on actions. We posit this as a question answering task, where\nan answer has to be given about a future scene state, given observations of the\ncurrent scene, and a question that includes a hypothetical action. Our solution\nis a hybrid model which integrates a physics engine into a question answering\narchitecture in order to anticipate future scene states resulting from\nobject-object interactions caused by an action. We demonstrate first results on\nthis challenging new problem and compare to baselines, where we outperform\nfully data-driven end-to-end learning approaches.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.04019v1",
            "date": "2018-09-11 16:43:52+00:00",
            "title": "Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data",
            "authors": [
                "Emilia Apostolova",
                "R. Andrew Kreek"
            ],
            "abstract": "Industry datasets used for text classification are rarely created for that\npurpose. In most cases, the data and target predictions are a by-product of\naccumulated historical data, typically fraught with noise, present in both the\ntext-based document, as well as in the targeted labels. In this work, we\naddress the question of how well performance metrics computed on noisy,\nhistorical data reflect the performance on the intended future machine learning\nmodel input. The results demonstrate the utility of dirty training datasets\nused to build prediction models for cleaner (and different) prediction inputs.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.04206v3",
            "date": "2018-09-12 00:40:40+00:00",
            "title": "Temporal Pattern Attention for Multivariate Time Series Forecasting",
            "authors": [
                "Shun-Yao Shih",
                "Fan-Keng Sun",
                "Hung-yi Lee"
            ],
            "abstract": "Forecasting multivariate time series data, such as prediction of electricity\nconsumption, solar power production, and polyphonic piano pieces, has numerous\nvaluable applications. However, complex and non-linear interdependencies\nbetween time steps and series complicate the task. To obtain accurate\nprediction, it is crucial to model long-term dependency in time series data,\nwhich can be achieved to some good extent by recurrent neural network (RNN)\nwith attention mechanism. Typical attention mechanism reviews the information\nat each previous time step and selects the relevant information to help\ngenerate the outputs, but it fails to capture the temporal patterns across\nmultiple time steps. In this paper, we propose to use a set of filters to\nextract time-invariant temporal patterns, which is similar to transforming time\nseries data into its \"frequency domain\". Then we proposed a novel attention\nmechanism to select relevant time series, and use its \"frequency domain\"\ninformation for forecasting. We applied the proposed model on several\nreal-world tasks and achieved state-of-the-art performance in all of them with\nonly one exception.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.04344v1",
            "date": "2018-09-12 10:11:39+00:00",
            "title": "The Wisdom of MaSSeS: Majority, Subjectivity, and Semantic Similarity in the Evaluation of VQA",
            "authors": [
                "Shailza Jolly",
                "Sandro Pezzelle",
                "Tassilo Klein",
                "Andreas Dengel",
                "Moin Nabi"
            ],
            "abstract": "We introduce MASSES, a simple evaluation metric for the task of Visual\nQuestion Answering (VQA). In its standard form, the VQA task is operationalized\nas follows: Given an image and an open-ended question in natural language,\nsystems are required to provide a suitable answer. Currently, model performance\nis evaluated by means of a somehow simplistic metric: If the predicted answer\nis chosen by at least 3 human annotators out of 10, then it is 100% correct.\nThough intuitively valuable, this metric has some important limitations. First,\nit ignores whether the predicted answer is the one selected by the Majority\n(MA) of annotators. Second, it does not account for the quantitative\nSubjectivity (S) of the answers in the sample (and dataset). Third, information\nabout the Semantic Similarity (SES) of the responses is completely neglected.\nBased on such limitations, we propose a multi-component metric that accounts\nfor all these issues. We show that our metric is effective in providing a more\nfine-grained evaluation both on the quantitative and qualitative level.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.04458v1",
            "date": "2018-09-12 13:57:06+00:00",
            "title": "Unsupervised Representation Learning of Speech for Dialect Identification",
            "authors": [
                "Suwon Shon",
                "Wei-Ning Hsu",
                "James Glass"
            ],
            "abstract": "In this paper, we explore the use of a factorized hierarchical variational\nautoencoder (FHVAE) model to learn an unsupervised latent representation for\ndialect identification (DID). An FHVAE can learn a latent space that separates\nthe more static attributes within an utterance from the more dynamic attributes\nby encoding them into two different sets of latent variables. Useful factors\nfor dialect identification, such as phonetic or linguistic content, are encoded\nby a segmental latent variable, while irrelevant factors that are relatively\nconstant within a sequence, such as a channel or a speaker information, are\nencoded by a sequential latent variable. The disentanglement property makes the\nsegmental latent variable less susceptible to channel and speaker variation,\nand thus reduces degradation from channel domain mismatch. We demonstrate that\non fully-supervised DID tasks, an end-to-end model trained on the features\nextracted from the FHVAE model achieves the best performance, compared to the\nsame model trained on conventional acoustic features and an i-vector based\nsystem. Moreover, we also show that the proposed approach can leverage a large\namount of unlabeled data for FHVAE training to learn domain-invariant features\nfor DID, and significantly improve the performance in a low-resource condition,\nwhere the labels for the in-domain data are not available.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1809.04482v1",
            "date": "2018-09-11 07:14:30+00:00",
            "title": "The Visual QA Devil in the Details: The Impact of Early Fusion and Batch Norm on CLEVR",
            "authors": [
                "Mateusz Malinowski",
                "Carl Doersch"
            ],
            "abstract": "Visual QA is a pivotal challenge for higher-level reasoning, requiring\nunderstanding language, vision, and relationships between many objects in a\nscene. Although datasets like CLEVR are designed to be unsolvable without such\ncomplex relational reasoning, some surprisingly simple feed-forward, \"holistic\"\nmodels have recently shown strong performance on this dataset. These models\nlack any kind of explicit iterative, symbolic reasoning procedure, which are\nhypothesized to be necessary for counting objects, narrowing down the set of\nrelevant objects based on several attributes, etc. The reason for this strong\nperformance is poorly understood. Hence, our work analyzes such models, and\nfinds that minor architectural elements are crucial to performance. In\nparticular, we find that \\textit{early fusion} of language and vision provides\nlarge performance improvements. This contrasts with the late fusion approaches\npopular at the dawn of Visual QA. We propose a simple module we call Multimodal\nCore, which we hypothesize performs the fundamental operations for multimodal\ntasks. We believe that understanding why these elements are so important to\ncomplex question answering will aid the design of better-performing algorithms\nfor Visual QA while minimizing hand-engineering effort.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.04553v1",
            "date": "2018-09-12 16:44:46+00:00",
            "title": "End-to-end Audiovisual Speech Activity Detection with Bimodal Recurrent Neural Models",
            "authors": [
                "Fei Tao",
                "Carlos Busso"
            ],
            "abstract": "Speech activity detection (SAD) plays an important role in current speech\nprocessing systems, including automatic speech recognition (ASR). SAD is\nparticularly difficult in environments with acoustic noise. A practical\nsolution is to incorporate visual information, increasing the robustness of the\nSAD approach. An audiovisual system has the advantage of being robust to\ndifferent speech modes (e.g., whisper speech) or background noise. Recent\nadvances in audiovisual speech processing using deep learning have opened\nopportunities to capture in a principled way the temporal relationships between\nacoustic and visual features. This study explores this idea proposing a\n\\emph{bimodal recurrent neural network} (BRNN) framework for SAD. The approach\nmodels the temporal dynamic of the sequential audiovisual data, improving the\naccuracy and robustness of the proposed SAD system. Instead of estimating\nhand-crafted features, the study investigates an end-to-end training approach,\nwhere acoustic and visual features are directly learned from the raw data\nduring training. The experimental evaluation considers a large audiovisual\ncorpus with over 60.8 hours of recordings, collected from 105 speakers. The\nresults demonstrate that the proposed framework leads to absolute improvements\nup to 1.2% under practical scenarios over a VAD baseline using only audio\nimplemented with deep neural network (DNN). The proposed approach achieves\n92.7% F1-score when it is evaluated using the sensors from a portable tablet\nunder noisy acoustic environment, which is only 1.0% lower than the performance\nobtained under ideal conditions (e.g., clean speech obtained with a high\ndefinition camera and a close-talking microphone).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04560v2",
            "date": "2018-09-12 16:53:13+00:00",
            "title": "Game-Based Video-Context Dialogue",
            "authors": [
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Current dialogue systems focus more on textual and speech context knowledge\nand are usually based on two speakers. Some recent work has investigated static\nimage-based dialogue. However, several real-world human interactions also\ninvolve dynamic visual context (similar to videos) as well as dialogue\nexchanges among multiple speakers. To move closer towards such multimodal\nconversational skills and visually-situated applications, we introduce a new\nvideo-context, many-speaker dialogue dataset based on live-broadcast soccer\ngame videos and chats from Twitch.tv. This challenging testbed allows us to\ndevelop visually-grounded dialogue models that should generate relevant\ntemporal and spatial event language from the live video, while also being\nrelevant to the chat history. For strong baselines, we also present several\ndiscriminative and generative models, e.g., based on tridirectional attention\nflow (TriDAF). We evaluate these models via retrieval ranking-recall, automatic\nphrase-matching metrics, as well as human evaluation studies. We also present\ndataset analyses, model ablations, and visualizations to understand the\ncontribution of different modalities and model components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.04705v1",
            "date": "2018-09-12 23:10:23+00:00",
            "title": "Distilled Wasserstein Learning for Word Embedding and Topic Modeling",
            "authors": [
                "Hongteng Xu",
                "Wenlin Wang",
                "Wei Liu",
                "Lawrence Carin"
            ],
            "abstract": "We propose a novel Wasserstein method with a distillation mechanism, yielding\njoint learning of word embeddings and topics. The proposed method is based on\nthe fact that the Euclidean distance between word embeddings may be employed as\nthe underlying distance in the Wasserstein topic model. The word distributions\nof topics, their optimal transports to the word distributions of documents, and\nthe embeddings of words are learned in a unified framework. When learning the\ntopic model, we leverage a distilled underlying distance matrix to update the\ntopic distributions and smoothly calculate the corresponding optimal\ntransports. Such a strategy provides the updating of word embeddings with\nrobust guidance, improving the algorithmic convergence. As an application, we\nfocus on patient admission records, in which the proposed method embeds the\ncodes of diseases and procedures and learns the topics of admissions, obtaining\nsuperior performance on clinically-meaningful disease network construction,\nmortality prediction as a function of admission codes, and procedure\nrecommendation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.05053v1",
            "date": "2018-09-13 16:39:53+00:00",
            "title": "XNLI: Evaluating Cross-lingual Sentence Representations",
            "authors": [
                "Alexis Conneau",
                "Guillaume Lample",
                "Ruty Rinott",
                "Adina Williams",
                "Samuel R. Bowman",
                "Holger Schwenk",
                "Veselin Stoyanov"
            ],
            "abstract": "State-of-the-art natural language processing systems rely on supervision in\nthe form of annotated data to learn competent models. These models are\ngenerally trained on data in a single language (usually English), and cannot be\ndirectly used beyond that language. Since collecting data in every language is\nnot realistic, there has been a growing interest in cross-lingual language\nunderstanding (XLU) and low-resource cross-language transfer. In this work, we\nconstruct an evaluation set for XLU by extending the development and test sets\nof the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15\nlanguages, including low-resource languages such as Swahili and Urdu. We hope\nthat our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence\nunderstanding by providing an informative standard evaluation task. In\naddition, we provide several baselines for multilingual sentence understanding,\nincluding two based on machine translation systems, and two that use parallel\ndata to train aligned multilingual bag-of-words and LSTM encoders. We find that\nXNLI represents a practical and challenging evaluation suite, and that directly\ntranslating the test data yields the best performance among available\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05054v2",
            "date": "2018-09-13 16:42:21+00:00",
            "title": "IncSQL: Training Incremental Text-to-SQL Parsers with Non-Deterministic Oracles",
            "authors": [
                "Tianze Shi",
                "Kedar Tatwawadi",
                "Kaushik Chakrabarti",
                "Yi Mao",
                "Oleksandr Polozov",
                "Weizhu Chen"
            ],
            "abstract": "We present a sequence-to-action parsing approach for the natural language to\nSQL task that incrementally fills the slots of a SQL query with feasible\nactions from a pre-defined inventory. To account for the fact that typically\nthere are multiple correct SQL queries with the same or very similar semantics,\nwe draw inspiration from syntactic parsing techniques and propose to train our\nsequence-to-action models with non-deterministic oracles. We evaluate our\nmodels on the WikiSQL dataset and achieve an execution accuracy of 83.7% on the\ntest set, a 2.1% absolute improvement over the models trained with traditional\nstatic oracles assuming a single correct target SQL query. When further\ncombined with the execution-guided decoding strategy, our model sets a new\nstate-of-the-art performance at an execution accuracy of 87.1%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05275v1",
            "date": "2018-09-14 06:51:01+00:00",
            "title": "Learning to Fingerprint the Latent Structure in Question Articulation",
            "authors": [
                "Kumar Mrityunjay",
                "Guntur Ravindra"
            ],
            "abstract": "Abstract Machine understanding of questions is tightly related to recognition\nof articulation in the context of the computational capabilities of an\nunderlying processing algorithm. In this paper a mathematical model to capture\nand distinguish the latent structure in the articulation of questions is\npresented. We propose an objective-driven approach to represent this latent\nstructure and show that such an approach is beneficial when examples of\ncomplementary objectives are not available. We show that the latent structure\ncan be represented as a system that maximizes a cost function related to the\nunderlying objective. Further, we show that the optimization formulation can be\napproximated to building a memory of patterns represented as a trained neural\nauto-encoder. Experimental evaluation using many clusters of questions, each\nrelated to an objective, shows 80% recognition accuracy and negligible false\npositive across these clusters of questions. We then extend the same memory to\na related task where the goal is to iteratively refine a dataset of questions\nbased on the latent articulation. We also demonstrate a refinement scheme\ncalled K-fingerprints, that achieves nearly 100% recognition with negligible\nfalse positive across the different clusters of questions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.05724v2",
            "date": "2018-09-15 14:37:46+00:00",
            "title": "Improving Natural Language Inference Using External Knowledge in the Science Questions Domain",
            "authors": [
                "Xiaoyan Wang",
                "Pavan Kapanipathi",
                "Ryan Musa",
                "Mo Yu",
                "Kartik Talamadupula",
                "Ibrahim Abdelaziz",
                "Maria Chang",
                "Achille Fokoue",
                "Bassem Makni",
                "Nicholas Mattei",
                "Michael Witbrock"
            ],
            "abstract": "Natural Language Inference (NLI) is fundamental to many Natural Language\nProcessing (NLP) applications including semantic search and question answering.\nThe NLI problem has gained significant attention thanks to the release of large\nscale, challenging datasets. Present approaches to the problem largely focus on\nlearning-based methods that use only textual information in order to classify\nwhether a given premise entails, contradicts, or is neutral with respect to a\ngiven hypothesis. Surprisingly, the use of methods based on structured\nknowledge -- a central topic in artificial intelligence -- has not received\nmuch attention vis-a-vis the NLI problem. While there are many open knowledge\nbases that contain various types of reasoning information, their use for NLI\nhas not been well explored. To address this, we present a combination of\ntechniques that harness knowledge graphs to improve performance on the NLI\nproblem in the science questions domain. We present the results of applying our\ntechniques on text, graph, and text-to-graph based models, and discuss\nimplications for the use of external knowledge in solving the NLI problem. Our\nmodel achieves the new state-of-the-art performance on the NLI problem over the\nSciTail science questions dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1809.05814v1",
            "date": "2018-09-16 04:21:38+00:00",
            "title": "Development of deep learning algorithms to categorize free-text notes pertaining to diabetes: convolution neural networks achieve higher accuracy than support vector machines",
            "authors": [
                "Boyi Yang",
                "Adam Wright"
            ],
            "abstract": "Health professionals can use natural language processing (NLP) technologies\nwhen reviewing electronic health records (EHR). Machine learning free-text\nclassifiers can help them identify problems and make critical decisions. We aim\nto develop deep learning neural network algorithms that identify EHR progress\nnotes pertaining to diabetes and validate the algorithms at two institutions.\nThe data used are 2,000 EHR progress notes retrieved from patients with\ndiabetes and all notes were annotated manually as diabetic or non-diabetic.\nSeveral deep learning classifiers were developed, and their performances were\nevaluated with the area under the ROC curve (AUC). The convolutional neural\nnetwork (CNN) model with a separable convolution layer accurately identified\ndiabetes-related notes in the Brigham and Womens Hospital testing set with the\nhighest AUC of 0.975. Deep learning classifiers can be used to identify EHR\nprogress notes pertaining to diabetes. In particular, the CNN-based classifier\ncan achieve a higher AUC than an SVM-based classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.05916v1",
            "date": "2018-09-16 17:17:56+00:00",
            "title": "Curriculum-Based Neighborhood Sampling For Sequence Prediction",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "The task of multi-step ahead prediction in language models is challenging\nconsidering the discrepancy between training and testing. At test time, a\nlanguage model is required to make predictions given past predictions as input,\ninstead of the past targets that are provided during training. This difference,\nknown as exposure bias, can lead to the compounding of errors along a generated\nsequence at test time.\n  In order to improve generalization in neural language models and address\ncompounding errors, we propose a curriculum learning based method that\ngradually changes an initially deterministic teacher policy to a gradually more\nstochastic policy, which we refer to as \\textit{Nearest-Neighbor Replacement\nSampling}. A chosen input at a given timestep is replaced with a sampled\nnearest neighbor of the past target with a truncated probability proportional\nto the cosine similarity between the original word and its top $k$ most similar\nwords. This allows the teacher to explore alternatives when the teacher\nprovides a sub-optimal policy or when the initial policy is difficult for the\nlearner to model. The proposed strategy is straightforward, online and requires\nlittle additional memory requirements. We report our main findings on two\nlanguage modelling benchmarks and find that the proposed approach performs\nparticularly well when used in conjunction with scheduled sampling, that too\nattempts to mitigate compounding errors in language models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.06004v2",
            "date": "2018-09-17 03:08:58+00:00",
            "title": "Open-world Learning and Application to Product Classification",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "P. Yu"
            ],
            "abstract": "Classic supervised learning makes the closed-world assumption, meaning that\nclasses seen in testing must have been seen in training. However, in the\ndynamic world, new or unseen class examples may appear constantly. A model\nworking in such an environment must be able to reject unseen classes (not seen\nor used in training). If enough data is collected for the unseen classes, the\nsystem should incrementally learn to accept/classify them. This learning\nparadigm is called open-world learning (OWL). Existing OWL methods all need\nsome form of re-training to accept or include the new classes in the overall\nmodel. In this paper, we propose a meta-learning approach to the problem. Its\nkey novelty is that it only needs to train a meta-classifier, which can then\ncontinually accept new classes when they have enough labeled data for the\nmeta-classifier to use, and also detect/reject future unseen classes. No\nre-training of the meta-classifier or a new overall classifier covering all old\nand new classes is needed. In testing, the method only uses the examples of the\nseen classes (including the newly added classes) on-the-fly for classification\nand rejection. Experimental results demonstrate the effectiveness of the new\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06213v1",
            "date": "2018-09-11 02:13:45+00:00",
            "title": "Context-Dependent Diffusion Network for Visual Relationship Detection",
            "authors": [
                "Zhen Cui",
                "Chunyan Xu",
                "Wenming Zheng",
                "Jian Yang"
            ],
            "abstract": "Visual relationship detection can bridge the gap between computer vision and\nnatural language for scene understanding of images. Different from pure object\nrecognition tasks, the relation triplets of subject-predicate-object lie on an\nextreme diversity space, such as \\textit{person-behind-person} and\n\\textit{car-behind-building}, while suffering from the problem of combinatorial\nexplosion. In this paper, we propose a context-dependent diffusion network\n(CDDN) framework to deal with visual relationship detection. To capture the\ninteractions of different object instances, two types of graphs, word semantic\ngraph and visual scene graph, are constructed to encode global context\ninterdependency. The semantic graph is built through language priors to model\nsemantic correlations across objects, whilst the visual scene graph defines the\nconnections of scene objects so as to utilize the surrounding scene\ninformation. For the graph-structured data, we design a diffusion network to\nadaptively aggregate information from contexts, which can effectively learn\nlatent representations of visual relationships and well cater to visual\nrelationship detection in view of its isomorphic invariance to graphs.\nExperiments on two widely-used datasets demonstrate that our proposed method is\nmore effective and achieves the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.06214v1",
            "date": "2018-09-11 11:07:26+00:00",
            "title": "Unsupervised Stylish Image Description Generation via Domain Layer Norm",
            "authors": [
                "Cheng Kuan Chen",
                "Zhu Feng Pan",
                "Min Sun",
                "Ming-Yu Liu"
            ],
            "abstract": "Most of the existing works on image description focus on generating\nexpressive descriptions. The only few works that are dedicated to generating\nstylish (e.g., romantic, lyric, etc.) descriptions suffer from limited style\nvariation and content digression. To address these limitations, we propose a\ncontrollable stylish image description generation model. It can learn to\ngenerate stylish image descriptions that are more related to image content and\ncan be trained with the arbitrary monolingual corpus without collecting new\npaired image and stylish descriptions. Moreover, it enables users to generate\nvarious stylish descriptions by plugging in style-specific parameters to\ninclude new styles into the existing model. We achieve this capability via a\nnovel layer normalization layer design, which we will refer to as the Domain\nLayer Norm (DLN). Extensive experimental validation and user study on various\nstylish image description generation tasks are conducted to show the\ncompetitive advantages of the proposed model.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.06709v2",
            "date": "2018-09-15 12:48:16+00:00",
            "title": "Document Informed Neural Autoregressive Topic Models with Distributional Prior",
            "authors": [
                "Pankaj Gupta",
                "Yatin Chaudhary",
                "Florian Buettner",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We address two challenges in topic models: (1) Context information around\nwords helps in determining their actual meaning, e.g., \"networks\" used in the\ncontexts \"artificial neural networks\" vs. \"biological neuron networks\".\nGenerative topic models infer topic-word distributions, taking no or only\nlittle context into account. Here, we extend a neural autoregressive topic\nmodel to exploit the full context information around words in a document in a\nlanguage modeling fashion. The proposed model is named as iDocNADE. (2) Due to\nthe small number of word occurrences (i.e., lack of context) in short text and\ndata sparsity in a corpus of few documents, the application of topic models is\nchallenging on such texts. Therefore, we propose a simple and efficient way of\nincorporating external knowledge into neural autoregressive topic models: we\nuse embeddings as a distributional prior. The proposed variants are named as\nDocNADEe and iDocNADEe.\n  We present novel neural autoregressive topic model variants that consistently\noutperform state-of-the-art generative topic models in terms of generalization,\ninterpretability (topic coherence) and applicability (retrieval and\nclassification) over 7 long-text and 8 short-text datasets from diverse\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.06858v2",
            "date": "2018-09-18 13:31:22+00:00",
            "title": "FRAGE: Frequency-Agnostic Word Representation",
            "authors": [
                "Chengyue Gong",
                "Di He",
                "Xu Tan",
                "Tao Qin",
                "Liwei Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "Continuous word representation (aka word embedding) is a basic building block\nin many neural network-based models used in natural language processing tasks.\nAlthough it is widely accepted that words with similar semantics should be\nclose to each other in the embedding space, we find that word embeddings\nlearned in several tasks are biased towards word frequency: the embeddings of\nhigh-frequency and low-frequency words lie in different subregions of the\nembedding space, and the embedding of a rare word and a popular word can be far\nfrom each other even if they are semantically similar. This makes learned word\nembeddings ineffective, especially for rare words, and consequently limits the\nperformance of these neural network models. In this paper, we develop a neat,\nsimple yet effective way to learn \\emph{FRequency-AGnostic word Embedding}\n(FRAGE) using adversarial training. We conducted comprehensive studies on ten\ndatasets across four natural language processing tasks, including word\nsimilarity, language modeling, machine translation and text classification.\nResults show that with FRAGE, we achieve higher performance than the baselines\nin all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07257v1",
            "date": "2018-09-19 15:50:18+00:00",
            "title": "MTLE: A Multitask Learning Encoder of Visual Feature Representations for Video and Movie Description",
            "authors": [
                "Oliver Nina",
                "Washington Garcia",
                "Scott Clouse",
                "Alper Yilmaz"
            ],
            "abstract": "Learning visual feature representations for video analysis is a daunting task\nthat requires a large amount of training samples and a proper generalization\nframework. Many of the current state of the art methods for video captioning\nand movie description rely on simple encoding mechanisms through recurrent\nneural networks to encode temporal visual information extracted from video\ndata. In this paper, we introduce a novel multitask encoder-decoder framework\nfor automatic semantic description and captioning of video sequences. In\ncontrast to current approaches, our method relies on distinct decoders that\ntrain a visual encoder in a multitask fashion. Our system does not depend\nsolely on multiple labels and allows for a lack of training data working even\nwith datasets where only one single annotation is viable per video. Our method\nshows improved performance over current state of the art methods in several\nmetrics on multi-caption and single-caption datasets. To the best of our\nknowledge, our method is the first method to use a multitask approach for\nencoding video features. Our method demonstrates its robustness on the Large\nScale Movie Description Challenge (LSMDC) 2017 where our method won the movie\ndescription task and its results were ranked among other competitors as the\nmost helpful for the visually impaired.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.07269v2",
            "date": "2018-09-19 16:08:50+00:00",
            "title": "Towards Dialogue-based Navigation with Multivariate Adaptation driven by Intention and Politeness for Social Robots",
            "authors": [
                "Chandrakant Bothe",
                "Fernando Garcia",
                "Arturo Cruz Maya",
                "Amit Kumar Pandey",
                "Stefan Wermter"
            ],
            "abstract": "Service robots need to show appropriate social behaviour in order to be\ndeployed in social environments such as healthcare, education, retail, etc.\nSome of the main capabilities that robots should have are navigation and\nconversational skills. If the person is impatient, the person might want a\nrobot to navigate faster and vice versa. Linguistic features that indicate\npoliteness can provide social cues about a person's patient and impatient\nbehaviour. The novelty presented in this paper is to dynamically incorporate\npoliteness in robotic dialogue systems for navigation. Understanding the\npoliteness in users' speech can be used to modulate the robot behaviour and\nresponses. Therefore, we developed a dialogue system to navigate in an indoor\nenvironment, which produces different robot behaviours and responses based on\nusers' intention and degree of politeness. We deploy and test our system with\nthe Pepper robot that adapts to the changes in user's politeness.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1809.07282v3",
            "date": "2018-09-19 16:21:12+00:00",
            "title": "Modeling Online Discourse with Coupled Distributed Topics",
            "authors": [
                "Nikita Srivatsan",
                "Zachary Wojtowicz",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "In this paper, we propose a deep, globally normalized topic model that\nincorporates structural relationships connecting documents in socially\ngenerated corpora, such as online forums. Our model (1) captures discursive\ninteractions along observed reply links in addition to traditional topic\ninformation, and (2) incorporates latent distributed representations arranged\nin a deep architecture, which enables a GPU-based mean-field inference\nprocedure that scales efficiently to large data. We apply our model to a new\nsocial media dataset consisting of 13M comments mined from the popular internet\nforum Reddit, a domain that poses significant challenges to models that do not\naccount for relationships connecting user comments. We evaluate against\nexisting methods across multiple metrics including perplexity and metadata\nprediction, and qualitatively analyze the learned interaction patterns.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.07853v3",
            "date": "2018-09-20 20:57:43+00:00",
            "title": "On Folding and Twisting (and whatknot): towards a characterization of workspaces in syntax",
            "authors": [
                "Diego Gabriel Krivochen"
            ],
            "abstract": "Syntactic theory has traditionally adopted a constructivist approach, in\nwhich a set of atomic elements are manipulated by combinatory operations to\nyield derived, complex elements. Syntactic structure is thus seen as the result\nor discrete recursive combinatorics over lexical items which get assembled into\nphrases, which are themselves combined to form sentences. This view is common\nto European and American structuralism (e.g., Benveniste, 1971; Hockett, 1958)\nand different incarnations of generative grammar, transformational and\nnon-transformational (Chomsky, 1956, 1995; and Kaplan & Bresnan, 1982; Gazdar,\n1982). Since at least Uriagereka (2002), there has been some attention paid to\nthe fact that syntactic operations must apply somewhere, particularly when\ncopying and movement operations are considered. Contemporary syntactic theory\nhas thus somewhat acknowledged the importance of formalizing aspects of the\nspaces in which elements are manipulated, but it is still a vastly\nunderexplored area. In this paper we explore the consequences of\nconceptualizing syntax as a set of topological operations applying over spaces\nrather than over discrete elements. We argue that there are empirical\nadvantages in such a view for the treatment of long-distance dependencies and\ncross-derivational dependencies: constraints on possible configurations emerge\nfrom the dynamics of the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07999v1",
            "date": "2018-09-21 09:19:12+00:00",
            "title": "Multimodal Dual Attention Memory for Video Story Question Answering",
            "authors": [
                "Kyung-Min Kim",
                "Seong-Ho Choi",
                "Jin-Hwa Kim",
                "Byoung-Tak Zhang"
            ],
            "abstract": "We propose a video story question-answering (QA) architecture, Multimodal\nDual Attention Memory (MDAM). The key idea is to use a dual attention mechanism\nwith late fusion. MDAM uses self-attention to learn the latent concepts in\nscene frames and captions. Given a question, MDAM uses the second attention\nover these latent concepts. Multimodal fusion is performed after the dual\nattention processes (late fusion). Using this processing pipeline, MDAM learns\nto infer a high-level vision-language joint representation from an abstraction\nof the full video content. We evaluate MDAM on PororoQA and MovieQA datasets\nwhich have large-scale QA annotations on cartoon videos and movies,\nrespectively. For both datasets, MDAM achieves new state-of-the-art results\nwith significant margins compared to the runner-up models. We confirm the best\nperformance of the dual attention mechanism combined with late fusion by\nablation studies. We also perform qualitative analysis by visualizing the\ninference mechanisms of MDAM.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1809.08291v1",
            "date": "2018-09-21 20:01:30+00:00",
            "title": "Opacity, Obscurity, and the Geometry of Question-Asking",
            "authors": [
                "Christina Boyce-Jacino",
                "Simon DeDeo"
            ],
            "abstract": "Asking questions is a pervasive human activity, but little is understood\nabout what makes them difficult to answer. An analysis of a pair of large\ndatabases, of New York Times crosswords and questions from the quiz-show\nJeopardy, establishes two orthogonal dimensions of question difficulty:\nobscurity (the rarity of the answer) and opacity (the indirectness of question\ncues, operationalized with word2vec). The importance of opacity, and the role\nof synergistic information in resolving it, suggests that accounts of\ndifficulty in terms of prior expectations captures only a part of the\nquestion-asking process. A further regression analysis shows the presence of\nadditional dimensions to question-asking: question complexity, the answer's\nlocal network density, cue intersection, and the presence of signal words. Our\nwork shows how question-askers can help their interlocutors by using contextual\ncues, or, conversely, how a particular kind of unfamiliarity with the domain in\nquestion can make it harder for individuals to learn from others. Taken\ntogether, these results suggest how Bayesian models of question difficulty can\nbe supplemented by process models and accounts of the heuristics individuals\nuse to navigate conceptual spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08396v3",
            "date": "2018-09-22 07:08:20+00:00",
            "title": "The Privacy Policy Landscape After the GDPR",
            "authors": [
                "Thomas Linden",
                "Rishabh Khandelwal",
                "Hamza Harkous",
                "Kassem Fawaz"
            ],
            "abstract": "The EU General Data Protection Regulation (GDPR) is one of the most demanding\nand comprehensive privacy regulations of all time. A year after it went into\neffect, we study its impact on the landscape of privacy policies online. We\nconduct the first longitudinal, in-depth, and at-scale assessment of privacy\npolicies before and after the GDPR. We gauge the complete consumption cycle of\nthese policies, from the first user impressions until the compliance\nassessment. We create a diverse corpus of two sets of 6,278 unique\nEnglish-language privacy policies from inside and outside the EU, covering\ntheir pre-GDPR and the post-GDPR versions. The results of our tests and\nanalyses suggest that the GDPR has been a catalyst for a major overhaul of the\nprivacy policies inside and outside the EU. This overhaul of the policies,\nmanifesting in extensive textual changes, especially for the EU-based websites,\ncomes at mixed benefits to the users. While the privacy policies have become\nconsiderably longer, our user study with 470 participants on Amazon MTurk\nindicates a significant improvement in the visual representation of privacy\npolicies from the users' perspective for the EU websites. We further develop a\nnew workflow for the automated assessment of requirements in privacy policies.\nUsing this workflow, we show that privacy policies cover more data practices\nand are more consistent with seven compliance requirements post the GDPR. We\nalso assess how transparent the organizations are with their privacy practices\nby performing specificity analysis. In this analysis, we find evidence for\npositive changes triggered by the GDPR, with the specificity level improving on\naverage. Still, we find the landscape of privacy policies to be in a\ntransitional phase; many policies still do not meet several key GDPR\nrequirements or their improved coverage comes with reduced specificity.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1809.08399v1",
            "date": "2018-09-22 07:37:39+00:00",
            "title": "Relating Zipf's law to textual information",
            "authors": [
                "Weibing Deng",
                "Armen E. Allahverdyan"
            ],
            "abstract": "Zipf's law is the main regularity of quantitative linguistics. Despite of\nmany works devoted to foundations of this law, it is still unclear whether it\nis only a statistical regularity, or it has deeper relations with\ninformation-carrying structures of the text. This question relates to that of\ndistinguishing a meaningful text (written in an unknown system) from a\nmeaningless set of symbols that mimics statistical features of a text. Here we\ncontribute to resolving these questions by comparing features of the first half\nof a text (from the beginning to the middle) to its second half. This\ncomparison can uncover hidden effects, because the halves have the same values\nof many parameters (style, genre, author's vocabulary {\\it etc}). In all\nstudied texts we saw that for the first half Zipf's law applies from smaller\nranks than in the second half, i.e. the law applies better to the first half.\nAlso, words that hold Zipf's law in the first half are distributed more\nhomogeneously over the text. These features do allow to distinguish a\nmeaningful text from a random sequence of words. Our findings correlate with a\nnumber of textual characteristics that hold in most cases we studied: the first\nhalf is lexically richer, has longer and less repetitive words, more and\nshorter sentences, more punctuation signs and more paragraphs. These\ndifferences between the halves indicate on a higher hierarchic level of text\norganization that so far went unnoticed in text linguistics. They relate the\nvalidity of Zipf's law to textual information. A complete description of this\neffect requires new models, though one existing model can account for some of\nits aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08510v1",
            "date": "2018-09-23 01:55:46+00:00",
            "title": "Towards Language Agnostic Universal Representations",
            "authors": [
                "Armen Aghajanyan",
                "Xia Song",
                "Saurabh Tiwary"
            ],
            "abstract": "When a bilingual student learns to solve word problems in math, we expect the\nstudent to be able to solve these problem in both languages the student is\nfluent in,even if the math lessons were only taught in one language. However,\ncurrent representations in machine learning are language dependent. In this\nwork, we present a method to decouple the language from the problem by learning\nlanguage agnostic representations and therefore allowing training a model in\none language and applying to a different one in a zero shot fashion. We learn\nthese representations by taking inspiration from linguistics and formalizing\nUniversal Grammar as an optimization process (Chomsky, 2014; Montague, 1970).\nWe demonstrate the capabilities of these representations by showing that the\nmodels trained on a single language using language agnostic representations\nachieve very similar accuracies in other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08718v1",
            "date": "2018-09-24 01:46:05+00:00",
            "title": "Central Bank Communication and the Yield Curve: A Semi-Automatic Approach using Non-Negative Matrix Factorization",
            "authors": [
                "Ancil Crayton"
            ],
            "abstract": "Communication is now a standard tool in the central bank's monetary policy\ntoolkit. Theoretically, communication provides the central bank an opportunity\nto guide public expectations, and it has been shown empirically that central\nbank communication can lead to financial market fluctuations. However, there\nhas been little research into which dimensions or topics of information are\nmost important in causing these fluctuations. We develop a semi-automatic\nmethodology that summarizes the FOMC statements into its main themes,\nautomatically selects the best model based on coherency, and assesses whether\nthere is a significant impact of these themes on the shape of the U.S Treasury\nyield curve using topic modeling methods from the machine learning literature.\nOur findings suggest that the FOMC statements can be decomposed into three\ntopics: (i) information related to the economic conditions and the mandates,\n(ii) information related to monetary policy tools and intermediate targets, and\n(iii) information related to financial markets and the financial crisis. We\nfind that statements are most influential during the financial crisis and the\neffects are mostly present in the curvature of the yield curve through\ninformation related to the financial theme.",
            "journal ref": "None",
            "category": "econ.GN"
        },
        {
            "url": "http://arxiv.org/abs/1809.08738v3",
            "date": "2018-09-24 03:23:07+00:00",
            "title": "Scalable inference of topic evolution via models for latent geometric structures",
            "authors": [
                "Mikhail Yurochkin",
                "Zhiwei Fan",
                "Aritra Guha",
                "Paraschos Koutris",
                "XuanLong Nguyen"
            ],
            "abstract": "We develop new models and algorithms for learning the temporal dynamics of\nthe topic polytopes and related geometric objects that arise in topic model\nbased inference. Our model is nonparametric Bayesian and the corresponding\ninference algorithm is able to discover new topics as the time progresses. By\nexploiting the connection between the modeling of topic polytope evolution,\nBeta-Bernoulli process and the Hungarian matching algorithm, our method is\nshown to be several orders of magnitude faster than existing topic modeling\napproaches, as demonstrated by experiments working with several million\ndocuments in under two dozens of minutes.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1809.08799v1",
            "date": "2018-09-24 08:37:02+00:00",
            "title": "Chargrid: Towards Understanding 2D Documents",
            "authors": [
                "Anoop Raveendra Katti",
                "Christian Reisswig",
                "Cordula Guder",
                "Sebastian Brarda",
                "Steffen Bickel",
                "Johannes H\u00f6hne",
                "Jean Baptiste Faddoul"
            ],
            "abstract": "We introduce a novel type of text representation that preserves the 2D layout\nof a document. This is achieved by encoding each document page as a\ntwo-dimensional grid of characters. Based on this representation, we present a\ngeneric document understanding pipeline for structured documents. This pipeline\nmakes use of a fully convolutional encoder-decoder network that predicts a\nsegmentation mask and bounding boxes. We demonstrate its capabilities on an\ninformation extraction task from invoices and show that it significantly\noutperforms approaches based on sequential text or document images.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08899v1",
            "date": "2018-09-20 14:29:22+00:00",
            "title": "Neural network approach to classifying alarming student responses to online assessment",
            "authors": [
                "Christopher M. Ormerod",
                "Amy E. Harris"
            ],
            "abstract": "Automated scoring engines are increasingly being used to score the free-form\ntext responses that students give to questions. Such engines are not designed\nto appropriately deal with responses that a human reader would find alarming\nsuch as those that indicate an intention to self-harm or harm others, responses\nthat allude to drug abuse or sexual abuse or any response that would elicit\nconcern for the student writing the response. Our neural network models have\nbeen designed to help identify these anomalous responses from a large\ncollection of typical responses that students give. The responses identified by\nthe neural network can be assessed for urgency, severity, and validity more\nquickly by a team of reviewers than otherwise possible. Given the anomalous\nnature of these types of responses, our goal is to maximize the chance of\nflagging these responses for review given the constraint that only a fixed\npercentage of responses can viably be assessed by a team of reviewers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.08909v2",
            "date": "2018-09-18 19:34:54+00:00",
            "title": "Language Identification with Deep Bottleneck Features",
            "authors": [
                "Zhanyu Ma",
                "Hong Yu"
            ],
            "abstract": "In this paper we proposed an end-to-end short utterances speech language\nidentification(SLD) approach based on a Long Short Term Memory (LSTM) neural\nnetwork which is special suitable for SLD application in intelligent vehicles.\nFeatures used for LSTM learning are generated by a transfer learning method.\nBottle-neck features of a deep neural network (DNN) which are trained for\nmandarin acoustic-phonetic classification are used for LSTM training. In order\nto improve the SLD accuracy of short utterances a phase vocoder based\ntime-scale modification(TSM) method is used to reduce and increase speech rated\nof the test utterance. By splicing the normal, speech rate reduced and\nincreased utterances, we can extend length of test utterances so as to improved\nimproved the performance of the SLD system. The experimental results on\nAP17-OLR database shows that the proposed methods can improve the performance\nof SLD, especially on short utterance with 1s and 3s durations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.08927v1",
            "date": "2018-09-21 08:27:01+00:00",
            "title": "Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives",
            "authors": [
                "Jing Han",
                "Zixing Zhang",
                "Nicholas Cummins",
                "Bj\u00f6rn Schuller"
            ],
            "abstract": "Over the past few years, adversarial training has become an extremely active\nresearch topic and has been successfully applied to various Artificial\nIntelligence (AI) domains. As a potentially crucial technique for the\ndevelopment of the next generation of emotional AI systems, we herein provide a\ncomprehensive overview of the application of adversarial training to affective\ncomputing and sentiment analysis. Various representative adversarial training\nalgorithms are explained and discussed accordingly, aimed at tackling diverse\nchallenges associated with emotional AI systems. Further, we highlight a range\nof potential future research directions. We expect that this overview will help\nfacilitate the development of adversarial training for affective computing and\nsentiment analysis in both the academic and industrial communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.09190v1",
            "date": "2018-09-24 19:46:24+00:00",
            "title": "From Audio to Semantics: Approaches to end-to-end spoken language understanding",
            "authors": [
                "Parisa Haghani",
                "Arun Narayanan",
                "Michiel Bacchiani",
                "Galen Chuang",
                "Neeraj Gaur",
                "Pedro Moreno",
                "Rohit Prabhavalkar",
                "Zhongdi Qu",
                "Austin Waters"
            ],
            "abstract": "Conventional spoken language understanding systems consist of two main\ncomponents: an automatic speech recognition module that converts audio to a\ntranscript, and a natural language understanding module that transforms the\nresulting text (or top N hypotheses) into a set of domains, intents, and\narguments. These modules are typically optimized independently. In this paper,\nwe formulate audio to semantic understanding as a sequence-to-sequence problem\n[1]. We propose and compare various encoder-decoder based approaches that\noptimize both modules jointly, in an end-to-end manner. Evaluations on a\nreal-world task show that 1) having an intermediate text representation is\ncrucial for the quality of the predicted semantics, especially the intent\narguments and 2) jointly optimizing the full system improves overall accuracy\nof prediction. Compared to independently trained models, our best jointly\ntrained model achieves similar domain and intent prediction F1 scores, but\nimproves argument word error rate by 18% relative.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1809.10274v1",
            "date": "2018-09-27 00:11:25+00:00",
            "title": "Semantically Invariant Text-to-Image Generation",
            "authors": [
                "Shagan Sah",
                "Dheeraj Peri",
                "Ameya Shringi",
                "Chi Zhang",
                "Miguel Dominguez",
                "Andreas Savakis",
                "Ray Ptucha"
            ],
            "abstract": "Image captioning has demonstrated models that are capable of generating\nplausible text given input images or videos. Further, recent work in image\ngeneration has shown significant improvements in image quality when text is\nused as a prior. Our work ties these concepts together by creating an\narchitecture that can enable bidirectional generation of images and text. We\ncall this network Multi-Modal Vector Representation (MMVR). Along with MMVR, we\npropose two improvements to the text conditioned image generation. Firstly, a\nn-gram metric based cost function is introduced that generalizes the caption\nwith respect to the image. Secondly, multiple semantically similar sentences\nare shown to help in generating better images. Qualitative and quantitative\nevaluations demonstrate that MMVR improves upon existing text conditioned image\ngeneration results by over 20%, while integrating visual and text modalities.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.10522v1",
            "date": "2018-09-27 14:01:19+00:00",
            "title": "Consistency and Variation in Kernel Neural Ranking Model",
            "authors": [
                "Mary Arpita Pyreddy",
                "Varshini Ramaseshan",
                "Narendra Nath Joshi",
                "Zhuyun Dai",
                "Chenyan Xiong",
                "Jamie Callan",
                "Zhiyuan Liu"
            ],
            "abstract": "This paper studies the consistency of the kernel-based neural ranking model\nK-NRM, a recent state-of-the-art neural IR model, which is important for\nreproducible research and deployment in the industry. We find that K-NRM has\nlow variance on relevance-based metrics across experimental trials. In spite of\nthis low variance in overall performance, different trials produce different\ndocument rankings for individual queries. The main source of variance in our\nexperiments was found to be different latent matching patterns captured by\nK-NRM. In the IR-customized word embeddings learned by K-NRM, the\nquery-document word pairs follow two different matching patterns that are\nequally effective, but align word pairs differently in the embedding space. The\ndifferent latent matching patterns enable a simple yet effective approach to\nconstruct ensemble rankers, which improve K-NRM's effectiveness and\ngeneralization abilities.",
            "journal ref": "Mary Arpita Pyreddy et al. 2018. Consistency and Variation in\n  Kernel Neural Ranking Model. In The 41st International ACM SIGIR Conference\n  on Research & Development in Information Retrieval (SIGIR '18). ACM, New\n  York, NY, USA, 961-964",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1809.10542v1",
            "date": "2018-09-27 14:36:00+00:00",
            "title": "Towards a classification of Lindenmayer systems",
            "authors": [
                "Diego Krivochen",
                "Douglas Saddy"
            ],
            "abstract": "In this paper we will attempt to classify Lindenmayer systems based on\nproperties of sets of rules and the kind of strings those rules generate. This\nclassification will be referred to as a parametrization of the L-space: the\nL-space is the phase space in which all possible L-developments are\nrepresented. This space is infinite, because there is no halting algorithm for\nL-grammars; but it is also subjected to hard conditions, because there are\ngrammars and developments which are not possible states of an L-system: a very\nwell-known example is the space of normal grammars. Just as the space of normal\ngrammars is parametrized into Regular, Context-Free, Context-Sensitive, and\nUnrestricted (with proper containment relations holding among them; see\nChomsky, 1959: Theorem 1), we contend here that the L-space is a very rich\nlandscape of grammars which cluster into kinds that are not mutually\ntranslatable.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10804v1",
            "date": "2018-09-28 00:14:10+00:00",
            "title": "Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks",
            "authors": [
                "Ivan Girardi",
                "Pengfei Ji",
                "An-phi Nguyen",
                "Nora Hollenstein",
                "Adam Ivankay",
                "Lorenz Kuhn",
                "Chiara Marchiori",
                "Ce Zhang"
            ],
            "abstract": "We present an operational component of a real-world patient triage system.\nGiven a specific patient presentation, the system is able to assess the level\nof medical urgency and issue the most appropriate recommendation in terms of\nbest point of care and time to treat. We use an attention-based convolutional\nneural network architecture trained on 600,000 doctor notes in German. We\ncompare two approaches, one that uses the full text of the medical notes and\none that uses only a selected list of medical entities extracted from the text.\nThese approaches achieve 79% and 66% precision, respectively, but on a\nconfidence threshold of 0.6, precision increases to 85% and 75%, respectively.\nIn addition, a method to detect warning symptoms is implemented to render the\nclassification task transparent from a medical perspective. The method is based\non the learning of attention scores and a method of automatic validation using\nthe same data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.10835v1",
            "date": "2018-09-28 03:06:31+00:00",
            "title": "Embedded-State Latent Conditional Random Fields for Sequence Labeling",
            "authors": [
                "Dung Thai",
                "Sree Harsha Ramesh",
                "Shikhar Murty",
                "Luke Vilnis",
                "Andrew McCallum"
            ],
            "abstract": "Complex textual information extraction tasks are often posed as sequence\nlabeling or \\emph{shallow parsing}, where fields are extracted using local\nlabels made consistent through probabilistic inference in a graphical model\nwith constrained transitions. Recently, it has become common to locally\nparametrize these models using rich features extracted by recurrent neural\nnetworks (such as LSTM), while enforcing consistent outputs through a simple\nlinear-chain model, representing Markovian dependencies between successive\nlabels. However, the simple graphical model structure belies the often complex\nnon-local constraints between output labels. For example, many fields, such as\na first name, can only occur a fixed number of times, or in the presence of\nother fields. While RNNs have provided increasingly powerful context-aware\nlocal features for sequence tagging, they have yet to be integrated with a\nglobal graphical model of similar expressivity in the output distribution. Our\nmodel goes beyond the linear chain CRF to incorporate multiple hidden states\nper output label, but parametrizes their transitions parsimoniously with\nlow-rank log-potential scoring matrices, effectively learning an embedding\nspace for hidden states. This augmented latent space of inference variables\ncomplements the rich feature representation of the RNN, and allows exact global\ninference obeying complex, learned non-local output constraints. We experiment\nwith several datasets and show that the model outperforms baseline CRF+RNN\nmodels when global output constraints are necessary at inference-time, and\nexplore the interpretable latent structure.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.11068v1",
            "date": "2018-09-28 14:49:27+00:00",
            "title": "Spoken Pass-Phrase Verification in the i-vector Space",
            "authors": [
                "Hossein Zeinali",
                "Lukas Burget",
                "Hossein Sameti",
                "Jan Cernocky"
            ],
            "abstract": "The task of spoken pass-phrase verification is to decide whether a test\nutterance contains the same phrase as given enrollment utterances. Beside other\napplications, pass-phrase verification can complement an independent speaker\nverification subsystem in text-dependent speaker verification. It can also be\nused for liveness detection by verifying that the user is able to correctly\nrespond to a randomly prompted phrase. In this paper, we build on our previous\nwork on i-vector based text-dependent speaker verification, where we have shown\nthat i-vectors extracted using phrase specific Hidden Markov Models (HMMs) or\nusing Deep Neural Network (DNN) based bottle-neck (BN) features help to reject\nutterances with wrong pass-phrases. We apply the same i-vector extraction\ntechniques to the stand-alone task of speaker-independent spoken pass-phrase\nclassification and verification. The experiments on RSR2015 and RedDots\ndatabases show that very simple scoring techniques (e.g. cosine distance\nscoring) applied to such i-vectors can provide results superior to those\npreviously published on the same data.",
            "journal ref": "Proc. Odyssey 2018 The Speaker and Language Recognition Workshop",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1810.00428v1",
            "date": "2018-09-30 17:31:52+00:00",
            "title": "Efficient Sequence Labeling with Actor-Critic Training",
            "authors": [
                "Saeed Najafi",
                "Colin Cherry",
                "Grzegorz Kondrak"
            ],
            "abstract": "Neural approaches to sequence labeling often use a Conditional Random Field\n(CRF) to model their output dependencies, while Recurrent Neural Networks (RNN)\nare used for the same purpose in other tasks. We set out to establish RNNs as\nan attractive alternative to CRFs for sequence labeling. To do so, we address\none of the RNN's most prominent shortcomings, the fact that it is not exposed\nto its own errors with the maximum-likelihood training. We frame the prediction\nof the output sequence as a sequential decision-making process, where we train\nthe network with an adjusted actor-critic algorithm (AC-RNN). We\ncomprehensively compare this strategy with maximum-likelihood training for both\nRNNs and CRFs on three structured-output tasks. The proposed AC-RNN efficiently\nmatches the performance of the CRF on NER and CCG tagging, and outperforms it\non Machine Transliteration. We also show that our training strategy is\nsignificantly better than other techniques for addressing RNN's exposure bias,\nsuch as Scheduled Sampling, and Self-Critical policy training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.00521v1",
            "date": "2018-10-01 03:58:00+00:00",
            "title": "A Simple Machine Learning Method for Commonsense Reasoning? A Short Commentary on Trinh & Le (2018)",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "This is a short Commentary on Trinh & Le (2018) (\"A Simple Method for\nCommonsense Reasoning\") that outlines three serious flaws in the cited paper\nand discusses why data-driven approaches cannot be considered as serious models\nfor the commonsense reasoning needed in natural language understanding in\ngeneral, and in reference resolution, in particular.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.00664v1",
            "date": "2018-09-24 10:54:52+00:00",
            "title": "Text Similarity in Vector Space Models: A Comparative Study",
            "authors": [
                "Omid Shahmirzadi",
                "Adam Lugowski",
                "Kenneth Younge"
            ],
            "abstract": "Automatic measurement of semantic text similarity is an important task in\nnatural language processing. In this paper, we evaluate the performance of\ndifferent vector space models to perform this task. We address the real-world\nproblem of modeling patent-to-patent similarity and compare TFIDF (and related\nextensions), topic models (e.g., latent semantic indexing), and neural models\n(e.g., paragraph vectors). Contrary to expectations, the added computational\ncost of text embedding methods is justified only when: 1) the target text is\ncondensed; and 2) the similarity comparison is trivial. Otherwise, TFIDF\nperforms surprisingly well in other cases: in particular for longer and more\ntechnical texts or for making finer-grained distinctions between nearest\nneighbors. Unexpectedly, extensions to the TFIDF method, such as adding noun\nphrases or calculating term weights incrementally, were not helpful in our\ncontext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00668v1",
            "date": "2018-09-26 14:25:40+00:00",
            "title": "Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection",
            "authors": [
                "Sudhanshu Kasewa",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "Grammatical error correction, like other machine learning tasks, greatly\nbenefits from large quantities of high quality training data, which is\ntypically expensive to produce. While writing a program to automatically\ngenerate realistic grammatical errors would be difficult, one could learn the\ndistribution of naturallyoccurring errors and attempt to introduce them into\nother datasets. Initial work on inducing errors in this way using statistical\nmachine translation has shown promise; we investigate cheaply constructing\nsynthetic samples, given a small corpus of human-annotated data, using an\noff-the-rack attentive sequence-to-sequence model and a straight-forward\npost-processing procedure. Our approach yields error-filled artificial data\nthat helps a vanilla bi-directional LSTM to outperform the previous state of\nthe art at grammatical error detection, and a previously introduced model to\ngain further improvements of over 5% $F_{0.5}$ score. When attempting to\ndetermine if a given sentence is synthetic, a human annotator at best achieves\n39.39 $F_1$ score, indicating that our model generates mostly human-like\ninstances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.00679v1",
            "date": "2018-09-28 00:51:24+00:00",
            "title": "Direct optimization of F-measure for retrieval-based personal question answering",
            "authors": [
                "Rasool Fakoor",
                "Amanjit Kainth",
                "Siamak Shakeri",
                "Christopher Winestock",
                "Abdel-rahman Mohamed",
                "Ruhi Sarikaya"
            ],
            "abstract": "Recent advances in spoken language technologies and the introduction of many\ncustomer facing products, have given rise to a wide customer reliance on smart\npersonal assistants for many of their daily tasks. In this paper, we present a\nsystem to reduce users' cognitive load by extending personal assistants with\nlong-term personal memory where users can store and retrieve by voice,\narbitrary pieces of information. The problem is framed as a neural retrieval\nbased question answering system where answers are selected from previously\nstored user memories. We propose to directly optimize the end-to-end retrieval\nperformance, measured by the F1-score, using reinforcement learning, leading to\nbetter performance on our experimental test set(s).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.00782v1",
            "date": "2018-10-01 16:03:49+00:00",
            "title": "The Profiling Machine: Active Generalization over Knowledge",
            "authors": [
                "Filip Ilievski",
                "Eduard Hovy",
                "Qizhe Xie",
                "Piek Vossen"
            ],
            "abstract": "The human mind is a powerful multifunctional knowledge storage and management\nsystem that performs generalization, type inference, anomaly detection,\nstereotyping, and other tasks. A dynamic KR system that appropriately profiles\nover sparse inputs to provide complete expectations for unknown facets can help\nwith all these tasks. In this paper, we introduce the task of profiling,\ninspired by theories and findings in social psychology about the potential of\nprofiles for reasoning and information processing. We describe two generic\nstate-of-the-art neural architectures that can be easily instantiated as\nprofiling machines to generate expectations and applied to any kind of\nknowledge to fill gaps. We evaluate these methods against Wikidata and crowd\nexpectations, and compare the results to gain insight in the nature of\nknowledge captured by various profiling methods. We make all code and data\navailable to facilitate future research.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.01064v4",
            "date": "2018-10-02 04:51:33+00:00",
            "title": "Improving Sentence Representations with Consensus Maximisation",
            "authors": [
                "Shuai Tang",
                "Virginia R. de Sa"
            ],
            "abstract": "Consensus maximisation learning can provide self-supervision when different\nviews are available of the same data. The distributional hypothesis provides\nanother form of useful self-supervision from adjacent sentences which are\nplentiful in large unlabelled corpora. Motivated by the observation that\ndifferent learning architectures tend to emphasise different aspects of\nsentence meaning, we present a new self-supervised learning framework for\nlearning sentence representations which minimises the disagreement between two\nviews of the same sentence where one view encodes the sentence with a recurrent\nneural network (RNN), and the other view encodes the same sentence with a\nsimple linear model. After learning, the individual views (networks) result in\nhigher quality sentence representations than their single-view learnt\ncounterparts (learnt using only the distributional hypothesis) as judged by\nperformance on standard downstream tasks. An ensemble of both views provides\neven better generalisation on both supervised and unsupervised downstream\ntasks. Also, importantly the ensemble of views trained with consensus\nmaximisation between the two different architectures performs better on\ndownstream tasks than an analogous ensemble made from the single-view trained\ncounterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01165v2",
            "date": "2018-10-02 10:35:13+00:00",
            "title": "Semi-supervised Text Regression with Conditional Generative Adversarial Networks",
            "authors": [
                "Tao Li",
                "Xudong Liu",
                "Shihan Su"
            ],
            "abstract": "Enormous online textual information provides intriguing opportunities for\nunderstandings of social and economic semantics. In this paper, we propose a\nnovel text regression model based on a conditional generative adversarial\nnetwork (GAN), with an attempt to associate textual data and social outcomes in\na semi-supervised manner. Besides promising potential of predicting\ncapabilities, our superiorities are twofold: (i) the model works with\nunbalanced datasets of limited labelled data, which align with real-world\nscenarios; and (ii) predictions are obtained by an end-to-end framework,\nwithout explicitly selecting high-level representations. Finally we point out\nrelated datasets for experiments and future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.01371v3",
            "date": "2018-10-02 17:01:28+00:00",
            "title": "Efficient Dialog Policy Learning via Positive Memory Retention",
            "authors": [
                "Rui Zhao",
                "Volker Tresp"
            ],
            "abstract": "This paper is concerned with the training of recurrent neural networks as\ngoal-oriented dialog agents using reinforcement learning. Training such agents\nwith policy gradients typically requires a large amount of samples. However,\nthe collection of the required data in form of conversations between chat-bots\nand human agents is time-consuming and expensive. To mitigate this problem, we\ndescribe an efficient policy gradient method using positive memory retention,\nwhich significantly increases the sample-efficiency. We show that our method is\n10 times more sample-efficient than policy gradients in extensive experiments\non a new synthetic number guessing game. Moreover, in a real-word visual object\ndiscovery game, the proposed method is twice as sample-efficient as policy\ngradients and shows state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.01398v2",
            "date": "2018-10-02 17:44:44+00:00",
            "title": "Optimal Completion Distillation for Sequence Learning",
            "authors": [
                "Sara Sabour",
                "William Chan",
                "Mohammad Norouzi"
            ],
            "abstract": "We present Optimal Completion Distillation (OCD), a training procedure for\noptimizing sequence to sequence models based on edit distance. OCD is\nefficient, has no hyper-parameters of its own, and does not require pretraining\nor joint optimization with conditional log-likelihood. Given a partial sequence\ngenerated by the model, we first identify the set of optimal suffixes that\nminimize the total edit distance, using an efficient dynamic programming\nalgorithm. Then, for each position of the generated sequence, we use a target\ndistribution that puts equal probability on the first token of all the optimal\nsuffixes. OCD achieves the state-of-the-art performance on end-to-end speech\nrecognition, on both Wall Street Journal and Librispeech datasets, achieving\n$9.3\\%$ WER and $4.5\\%$ WER respectively.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.01869v1",
            "date": "2018-10-03 13:22:44+00:00",
            "title": "Machine Learning Suites for Online Toxicity Detection",
            "authors": [
                "David Noever"
            ],
            "abstract": "To identify and classify toxic online commentary, the modern tools of data\nscience transform raw text into key features from which either thresholding or\nlearning algorithms can make predictions for monitoring offensive\nconversations. We systematically evaluate 62 classifiers representing 19 major\nalgorithmic families against features extracted from the Jigsaw dataset of\nWikipedia comments. We compare the classifiers based on statistically\nsignificant differences in accuracy and relative execution time. Among these\nclassifiers for identifying toxic comments, tree-based algorithms provide the\nmost transparently explainable rules and rank-order the predictive contribution\nof each feature. Among 28 features of syntax, sentiment, emotion and outlier\nword dictionaries, a simple bad word list proves most predictive of offensive\ncommentary.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.02047v6",
            "date": "2018-10-04 04:02:11+00:00",
            "title": "Classical linear logic, cobordisms and categorical semantics of categorial grammars",
            "authors": [
                "Sergey Slavnov"
            ],
            "abstract": "We propose a categorial grammar based on classical multiplicative linear\nlogic.\n  This can be seen as an extension of abstract categorial grammars (ACG) and is\nat least as expressive. However, constituents of {\\it linear logic grammars\n(LLG)} are not abstract ${\\lambda}$-terms, but simply tuples of words with\nlabeled endpoints, we call them {\\it multiwords}. At least, this gives a\nconcrete and intuitive representation of ACG.\n  A key observation is that the class of multiwords has a fundamental algebraic\nstructure. Namely, multiwords can be organized in a category, very similar to\nthe category of topological cobordisms. This category is symmetric monoidal\nclosed and compact closed and thus is a model of linear $\\lambda$-calculus and\nclassical linear logic. We think that this category is interesting on its own\nright. In particular, it might provide categorical representation for other\nformalisms.\n  On the other hand, many models of language semantics are based on commutative\nlogic or, more generally, on symmetric monoidal closed categories. But the\ncategory of {\\it word cobordisms} is a category of language elements, which is\nitself symmetric monoidal closed and independent of any grammar. Thus, it might\nprove useful in understanding language semantics as well.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1810.02229v1",
            "date": "2018-10-04 14:09:20+00:00",
            "title": "Italian Event Detection Goes Deep Learning",
            "authors": [
                "Tommaso Caselli"
            ],
            "abstract": "This paper reports on a set of experiments with different word embeddings to\ninitialize a state-of-the-art Bi-LSTM-CRF network for event detection and\nclassification in Italian, following the EVENTI evaluation exercise. The net-\nwork obtains a new state-of-the-art result by improving the F1 score for\ndetection of 1.3 points, and of 6.5 points for classification, by using a\nsingle step approach. The results also provide further evidence that embeddings\nhave a major impact on the performance of such architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.02338v2",
            "date": "2018-10-04 17:38:50+00:00",
            "title": "Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding",
            "authors": [
                "Kexin Yi",
                "Jiajun Wu",
                "Chuang Gan",
                "Antonio Torralba",
                "Pushmeet Kohli",
                "Joshua B. Tenenbaum"
            ],
            "abstract": "We marry two powerful ideas: deep representation learning for visual\nrecognition and language understanding, and symbolic program execution for\nreasoning. Our neural-symbolic visual question answering (NS-VQA) system first\nrecovers a structural scene representation from the image and a program trace\nfrom the question. It then executes the program on the scene representation to\nobtain an answer. Incorporating symbolic structure as prior knowledge offers\nthree unique advantages. First, executing programs on a symbolic space is more\nrobust to long program traces; our model can solve complex reasoning tasks\nbetter, achieving an accuracy of 99.8% on the CLEVR dataset. Second, the model\nis more data- and memory-efficient: it performs well after learning on a small\nnumber of training data; it can also encode an image into a compact\nrepresentation, requiring less storage than existing methods for offline\nquestion answering. Third, symbolic program execution offers full transparency\nto the reasoning process; we are thus able to interpret and diagnose each\nexecution step.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.02358v2",
            "date": "2018-10-03 19:48:38+00:00",
            "title": "Transfer Learning via Unsupervised Task Discovery for Visual Question Answering",
            "authors": [
                "Hyeonwoo Noh",
                "Taehoon Kim",
                "Jonghwan Mun",
                "Bohyung Han"
            ],
            "abstract": "We study how to leverage off-the-shelf visual and linguistic data to cope\nwith out-of-vocabulary answers in visual question answering task. Existing\nlarge-scale visual datasets with annotations such as image class labels,\nbounding boxes and region descriptions are good sources for learning rich and\ndiverse visual concepts. However, it is not straightforward how the visual\nconcepts can be captured and transferred to visual question answering models\ndue to missing link between question dependent answering models and visual data\nwithout question. We tackle this problem in two steps: 1) learning a task\nconditional visual classifier, which is capable of solving diverse\nquestion-specific visual recognition tasks, based on unsupervised task\ndiscovery and 2) transferring the task conditional visual classifier to visual\nquestion answering models. Specifically, we employ linguistic knowledge sources\nsuch as structured lexical database (e.g. WordNet) and visual descriptions for\nunsupervised task discovery, and transfer a learned task conditional visual\nclassifier as an answering unit in a visual question answering model. We\nempirically show that the proposed algorithm generalizes to out-of-vocabulary\nanswers successfully using the knowledge transferred from the visual dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.02802v1",
            "date": "2018-10-05 17:37:37+00:00",
            "title": "POIReviewQA: A Semantically Enriched POI Retrieval and Question Answering Dataset",
            "authors": [
                "Gengchen Mai",
                "Krzysztof Janowicz",
                "Cheng He",
                "Sumang Liu",
                "Ni Lao"
            ],
            "abstract": "Many services that perform information retrieval for Points of Interest (POI)\nutilize a Lucene-based setup with spatial filtering. While this type of system\nis easy to implement it does not make use of semantics but relies on direct\nword matches between a query and reviews leading to a loss in both precision\nand recall. To study the challenging task of semantically enriching POIs from\nunstructured data in order to support open-domain search and question answering\n(QA), we introduce a new dataset POIReviewQA. It consists of 20k questions\n(e.g.\"is this restaurant dog friendly?\") for 1022 Yelp business types. For each\nquestion we sampled 10 reviews, and annotated each sentence in the reviews\nwhether it answers the question and what the corresponding answer is. To test a\nsystem's ability to understand the text we adopt an information retrieval\nevaluation by ranking all the review sentences for a question based on the\nlikelihood that they answer this question. We build a Lucene-based baseline\nmodel, which achieves 77.0% AUC and 48.8% MAP. A sentence embedding-based model\nachieves 79.2% AUC and 41.8% MAP, indicating that the dataset presents a\nchallenging problem for future research by the GIR community. The result\ntechnology can help exploit the thematic content of web documents and social\nmedia for characterisation of locations.",
            "journal ref": "12th Workshop on Geographic Information Retrieval (GIR 2018)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.02938v1",
            "date": "2018-10-06 05:25:24+00:00",
            "title": "Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Learning a matching function between two text sequences is a long standing\nproblem in NLP research. This task enables many potential applications such as\nquestion answering and paraphrase identification. This paper proposes Co-Stack\nResidual Affinity Networks (CSRAN), a new and universal neural architecture for\nthis problem. CSRAN is a deep architecture, involving stacked (multi-layered)\nrecurrent encoders. Stacked/Deep architectures are traditionally difficult to\ntrain, due to the inherent weaknesses such as difficulty with feature\npropagation and vanishing gradients. CSRAN incorporates two novel components to\ntake advantage of the stacked architecture. Firstly, it introduces a new\nbidirectional alignment mechanism that learns affinity weights by fusing\nsequence pairs across stacked hierarchies. Secondly, it leverages a multi-level\nattention refinement component between stacked recurrent layers. The key\nintuition is that, by leveraging information across all network hierarchies, we\ncan not only improve gradient flow but also improve overall performance. We\nconduct extensive experiments on six well-studied text sequence matching\ndatasets, achieving state-of-the-art performance on all.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03031v1",
            "date": "2018-10-06 17:42:19+00:00",
            "title": "Text-based Sentiment Analysis and Music Emotion Recognition",
            "authors": [
                "Erion \u00c7ano"
            ],
            "abstract": "Sentiment polarity of tweets, blog posts or product reviews has become highly\nattractive and is utilized in recommender systems, market predictions, business\nintelligence and more. Deep learning techniques are becoming top performers on\nanalyzing such texts. There are however several problems that need to be solved\nfor efficient use of deep neural networks on text mining and text polarity\nanalysis. First, deep neural networks need to be fed with data sets that are\nbig in size as well as properly labeled. Second, there are various\nuncertainties regarding the use of word embedding vectors: should they be\ngenerated from the same data set that is used to train the model or it is\nbetter to source them from big and popular collections? Third, to simplify\nmodel creation it is convenient to have generic neural network architectures\nthat are effective and can adapt to various texts, encapsulating much of design\ncomplexity. This thesis addresses the above problems to provide methodological\nand practical insights for utilizing neural networks on sentiment analysis of\ntexts and achieving state of the art results. Regarding the first problem, the\neffectiveness of various crowdsourcing alternatives is explored and two\nmedium-sized and emotion-labeled song data sets are created utilizing social\ntags. To address the second problem, a series of experiments with large text\ncollections of various contents and domains were conducted, trying word\nembeddings of various parameters. Regarding the third problem, a series of\nexperiments involving convolution and max-pooling neural layers were conducted.\nCombining convolutions of words, bigrams, and trigrams with regional\nmax-pooling layers in a couple of stacks produced the best results. The derived\narchitecture achieves competitive performance on sentiment polarity analysis of\nmovie, business and product reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03430v1",
            "date": "2018-10-08 13:25:05+00:00",
            "title": "Cross Script Hindi English NER Corpus from Wikipedia",
            "authors": [
                "Mohd Zeeshan Ansari",
                "Tanvir Ahmad",
                "Md Arshad Ali"
            ],
            "abstract": "The text generated on social media platforms is essentially a mixed lingual\ntext. The mixing of language in any form produces considerable amount of\ndifficulty in language processing systems. Moreover, the advancements in\nlanguage processing research depends upon the availability of standard corpora.\nThe development of mixed lingual Indian Named Entity Recognition (NER) systems\nare facing obstacles due to unavailability of the standard evaluation corpora.\nSuch corpora may be of mixed lingual nature in which text is written using\nmultiple languages predominantly using a single script only. The motivation of\nour work is to emphasize the automatic generation such kind of corpora in order\nto encourage mixed lingual Indian NER. The paper presents the preparation of a\nCross Script Hindi-English Corpora from Wikipedia category pages. The corpora\nis successfully annotated using standard CoNLL-2003 categories of PER, LOC,\nORG, and MISC. Its evaluation is carried out on a variety of machine learning\nalgorithms and favorable results are achieved.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.03445v1",
            "date": "2018-10-04 14:25:36+00:00",
            "title": "Building a language evolution tree based on word vector combination model",
            "authors": [
                "Zhu Gao",
                "Yanhui Jiang",
                "Junhui Gao"
            ],
            "abstract": "In this paper, we try to explore the evolution of language through case\ncalculations. First, we chose the novels of eleven British writers from 1400 to\n2005 and found the corresponding works; Then, we use the natural language\nprocessing tool to construct the corresponding eleven corpora, and calculate\nthe respective word vectors of 100 high-frequency words in eleven corpora;\nNext, for each corpus, we concatenate the 100 word vectors from beginning to\nend into one; Finally, we use the similarity comparison and hierarchical\nclustering method to generate the relationship tree between the combined eleven\nword vectors. This tree represents the relationship between eleven corpora. We\nfound that in the tree generated by clustering, the distance between the corpus\nand the year corresponding to the corpus are basically the same. This means\nthat we have discovered a specific language evolution tree. To verify the\nstability and versatility of this method, we add three other themes: Dickens's\neight works, the 19th century poets' works, and art criticism of recent 60\nyears. For these four themes, we tested different parameters such as the time\nspan of the corpus, the time interval between the corpora, the dimension of the\nword vector, and the number of high-frequency public words. The results show\nthat this is fairly stable and versatile.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03449v1",
            "date": "2018-10-03 08:43:11+00:00",
            "title": "Exploiting Contextual Information via Dynamic Memory Network for Event Detection",
            "authors": [
                "Shaobo Liu",
                "Rui Cheng",
                "Xiaoming Yu",
                "Xueqi Cheng"
            ],
            "abstract": "The task of event detection involves identifying and categorizing event\ntriggers. Contextual information has been shown effective on the task. However,\nexisting methods which utilize contextual information only process the context\nonce. We argue that the context can be better exploited by processing the\ncontext multiple times, allowing the model to perform complex reasoning and to\ngenerate better context representation, thus improving the overall performance.\nMeanwhile, dynamic memory network (DMN) has demonstrated promising capability\nin capturing contextual information and has been applied successfully to\nvarious tasks. In light of the multi-hop mechanism of the DMN to model the\ncontext, we propose the trigger detection dynamic memory network (TD-DMN) to\ntackle the event detection problem. We performed a five-fold cross-validation\non the ACE-2005 dataset and experimental results show that the multi-hop\nmechanism does improve the performance and the proposed model achieves best\n$F_1$ score compared to the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03459v1",
            "date": "2018-10-04 08:53:42+00:00",
            "title": "Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling",
            "authors": [
                "Jaejin Cho",
                "Murali Karthick Baskar",
                "Ruizhi Li",
                "Matthew Wiesner",
                "Sri Harish Mallidi",
                "Nelson Yalta",
                "Martin Karafiat",
                "Shinji Watanabe",
                "Takaaki Hori"
            ],
            "abstract": "Sequence-to-sequence (seq2seq) approach for low-resource ASR is a relatively\nnew direction in speech research. The approach benefits by performing model\ntraining without using lexicon and alignments. However, this poses a new\nproblem of requiring more data compared to conventional DNN-HMM systems. In\nthis work, we attempt to use data from 10 BABEL languages to build a\nmulti-lingual seq2seq model as a prior model, and then port them towards 4\nother BABEL languages using transfer learning approach. We also explore\ndifferent architectures for improving the prior multilingual seq2seq model. The\npaper also discusses the effect of integrating a recurrent neural network\nlanguage model (RNNLM) with a seq2seq model during decoding. Experimental\nresults show that the transfer learning approach from the multilingual model\nshows substantial gains over monolingual models across all 4 BABEL languages.\nIncorporating an RNNLM also brings significant improvements in terms of %WER,\nand achieves recognition performance comparable to the models trained with\ntwice more training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03480v1",
            "date": "2018-10-05 16:15:14+00:00",
            "title": "Text Classification of the Precursory Accelerating Seismicity Corpus: Inference on some Theoretical Trends in Earthquake Predictability Research from 1988 to 2018",
            "authors": [
                "Arnaud Mignan"
            ],
            "abstract": "Text analytics based on supervised machine learning classifiers has shown\ngreat promise in a multitude of domains, but has yet to be applied to\nSeismology. We test various standard models (Naive Bayes, k-Nearest Neighbors,\nSupport Vector Machines, and Random Forests) on a seismological corpus of 100\narticles related to the topic of precursory accelerating seismicity, spanning\nfrom 1988 to 2010. This corpus was labelled in Mignan (2011) with the precursor\nwhether explained by critical processes (i.e., cascade triggering) or by other\nprocesses (such as signature of main fault loading). We investigate rather the\nclassification process can be automatized to help analyze larger corpora in\norder to better understand trends in earthquake predictability research. We\nfind that the Naive Bayes model performs best, in agreement with the machine\nlearning literature for the case of small datasets, with cross-validation\naccuracies of 86% for binary classification. For a refined multiclass\nclassification ('non-critical process' < 'agnostic' < 'critical process\nassumed' < 'critical process demonstrated'), we obtain up to 78% accuracy.\nPrediction on a dozen of articles published since 2011 shows however a weak\ngeneralization with a F1-score of 60%, only slightly better than a random\nclassifier, which can be explained by a change of authorship and use of\ndifferent terminologies. Yet, the model shows F1-scores greater than 80% for\nthe two multiclass extremes ('non-critical process' versus 'critical process\ndemonstrated') while it falls to random classifier results (around 25%) for\npapers labelled 'agnostic' or 'critical process assumed'. Those results are\nencouraging in view of the small size of the corpus and of the high degree of\nabstraction of the labelling. Domain knowledge engineering remains essential\nbut can be made transparent by an investigation of Naive Bayes keyword\nposterior probabilities.",
            "journal ref": "Journal of Seismology, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.03655v1",
            "date": "2018-10-08 18:50:54+00:00",
            "title": "Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks",
            "authors": [
                "Takuya Yoshioka",
                "Hakan Erdogan",
                "Zhuo Chen",
                "Xiong Xiao",
                "Fil Alleva"
            ],
            "abstract": "The goal of this work is to develop a meeting transcription system that can\nrecognize speech even when utterances of different speakers are overlapped.\nWhile speech overlaps have been regarded as a major obstacle in accurately\ntranscribing meetings, a traditional beamformer with a single output has been\nexclusively used because previously proposed speech separation techniques have\ncritical constraints for application to real meetings. This paper proposes a\nnew signal processing module, called an unmixing transducer, and describes its\nimplementation using a windowed BLSTM. The unmixing transducer has a fixed\nnumber, say J, of output channels, where J may be different from the number of\nmeeting attendees, and transforms an input multi-channel acoustic signal into J\ntime-synchronous audio streams. Each utterance in the meeting is separated and\nemitted from one of the output channels. Then, each output signal can be simply\nfed to a speech recognition back-end for segmentation and transcription. Our\nmeeting transcription system using the unmixing transducer outperforms a system\nbased on a state-of-the-art neural mask-based beamformer by 10.8%. Significant\nimprovements are observed in overlapped segments. To the best of our knowledge,\nthis is the first report that applies overlapped speech recognition to\nunconstrained real meeting audio.",
            "journal ref": "Proc. Interspeech 2018, 3038-3042",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.03947v4",
            "date": "2018-10-09 13:04:25+00:00",
            "title": "textTOvec: Deep Contextualized Neural Autoregressive Topic Models of Language with Distributed Compositional Prior",
            "authors": [
                "Pankaj Gupta",
                "Yatin Chaudhary",
                "Florian Buettner",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We address two challenges of probabilistic topic modelling in order to better\nestimate the probability of a word in a given context, i.e., P(word|context):\n(1) No Language Structure in Context: Probabilistic topic models ignore word\norder by summarizing a given context as a \"bag-of-word\" and consequently the\nsemantics of words in the context is lost. The LSTM-LM learns a vector-space\nrepresentation of each word by accounting for word order in local collocation\npatterns and models complex characteristics of language (e.g., syntax and\nsemantics), while the TM simultaneously learns a latent representation from the\nentire document and discovers the underlying thematic structure. We unite two\ncomplementary paradigms of learning the meaning of word occurrences by\ncombining a TM (e.g., DocNADE) and a LM in a unified probabilistic framework,\nnamed as ctx-DocNADE. (2) Limited Context and/or Smaller training corpus of\ndocuments: In settings with a small number of word occurrences (i.e., lack of\ncontext) in short text or data sparsity in a corpus of few documents, the\napplication of TMs is challenging. We address this challenge by incorporating\nexternal knowledge into neural autoregressive topic models via a language\nmodelling approach: we use word embeddings as input of a LSTM-LM with the aim\nto improve the word-topic mapping on a smaller and/or short-text corpus. The\nproposed DocNADE extension is named as ctx-DocNADEe.\n  We present novel neural autoregressive topic model variants coupled with\nneural LMs and embeddings priors that consistently outperform state-of-the-art\ngenerative TMs in terms of generalization (perplexity), interpretability (topic\ncoherence) and applicability (retrieval and classification) over 6 long-text\nand 8 short-text datasets from diverse domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.04755v1",
            "date": "2018-10-10 21:42:29+00:00",
            "title": "Leveraging Textual Specifications for Grammar-based Fuzzing of Network Protocols",
            "authors": [
                "Samuel Jero",
                "Maria Leonor Pacheco",
                "Dan Goldwasser",
                "Cristina Nita-Rotaru"
            ],
            "abstract": "Grammar-based fuzzing is a technique used to find software vulnerabilities by\ninjecting well-formed inputs generated following rules that encode application\nsemantics. Most grammar-based fuzzers for network protocols rely on human\nexperts to manually specify these rules. In this work we study automated\nlearning of protocol rules from textual specifications (i.e. RFCs). We evaluate\nthe automatically extracted protocol rules by applying them to a\nstate-of-the-art fuzzer for transport protocols and show that it leads to a\nsmaller number of test cases while finding the same attacks as the system that\nuses manually specified rules.",
            "journal ref": "The Thirty-First AAAI Conference on Innovative Applications of\n  Artificial Intelligence, IAAI 2019",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1810.05102v2",
            "date": "2018-10-11 16:07:20+00:00",
            "title": "Neural Relation Extraction Within and Across Sentence Boundaries",
            "authors": [
                "Pankaj Gupta",
                "Subburam Rajaram",
                "Hinrich Sch\u00fctze",
                "Bernt Andrassy",
                "Thomas Runkler"
            ],
            "abstract": "Past work in relation extraction mostly focuses on binary relation between\nentity pairs within single sentence. Recently, the NLP community has gained\ninterest in relation extraction in entity pairs spanning multiple sentences. In\nthis paper, we propose a novel architecture for this task: inter-sentential\ndependency-based neural networks (iDepNN). iDepNN models the shortest and\naugmented dependency paths via recurrent and recursive neural networks to\nextract relationships within (intra-) and across (inter-) sentence boundaries.\nCompared to SVM and neural network baselines, iDepNN is more robust to false\npositives in relationships spanning sentences.\n  We evaluate our models on four datasets from newswire (MUC6) and medical\n(BioNLP shared task) domains that achieve state-of-the-art performance and show\na better balance in precision and recall for inter-sentential relationships. We\nperform better than 11 teams participating in the BioNLP shared task 2016 and\nachieve a gain of 5.2% (0.587 vs 0.558) in F1 over the winning team. We also\nrelease the crosssentence annotations for MUC6.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.05436v1",
            "date": "2018-10-12 10:02:23+00:00",
            "title": "HiTR: Hierarchical Topic Model Re-estimation for Measuring Topical Diversity of Documents",
            "authors": [
                "Hosein Azarbonyad",
                "Mostafa Dehghani",
                "Tom Kenter",
                "Maarten Marx",
                "Jaap Kamps",
                "Maarten de Rijke"
            ],
            "abstract": "A high degree of topical diversity is often considered to be an important\ncharacteristic of interesting text documents. A recent proposal for measuring\ntopical diversity identifies three distributions for assessing the diversity of\ndocuments: distributions of words within documents, words within topics, and\ntopics within documents. Topic models play a central role in this approach and,\nhence, their quality is crucial to the efficacy of measuring topical diversity.\nThe quality of topic models is affected by two causes: generality and impurity\nof topics. General topics only include common information of a background\ncorpus and are assigned to most of the documents. Impure topics contain words\nthat are not related to the topic. Impurity lowers the interpretability of\ntopic models. Impure topics are likely to get assigned to documents\nerroneously. We propose a hierarchical re-estimation process aimed at removing\ngenerality and impurity. Our approach has three re-estimation components: (1)\ndocument re-estimation, which removes general words from the documents; (2)\ntopic re-estimation, which re-estimates the distribution over words of each\ntopic; and (3) topic assignment re-estimation, which re-estimates for each\ndocument its distributions over topics. For measuring topical diversity of text\ndocuments, our HiTR approach improves over the state-of-the-art measured on\nPubMed dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06306v1",
            "date": "2018-10-15 12:34:05+00:00",
            "title": "Improving Topic Models with Latent Feature Word Representations",
            "authors": [
                "Dat Quoc Nguyen",
                "Richard Billingsley",
                "Lan Du",
                "Mark Johnson"
            ],
            "abstract": "Probabilistic topic models are widely used to discover latent topics in\ndocument collections, while latent feature vector representations of words have\nbeen used to obtain high performance in many NLP tasks. In this paper, we\nextend two different Dirichlet multinomial topic models by incorporating latent\nfeature vector representations of words trained on very large corpora to\nimprove the word-topic mapping learnt on a smaller corpus. Experimental results\nshow that by using information from the external corpora, our new models\nproduce significant improvements on topic coherence, document clustering and\ndocument classification tasks, especially on datasets with few or short\ndocuments.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  vol. 3, pp. 299-313, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06526v2",
            "date": "2018-10-15 17:19:18+00:00",
            "title": "Structured Content Preservation for Unsupervised Text Style Transfer",
            "authors": [
                "Youzhi Tian",
                "Zhiting Hu",
                "Zhou Yu"
            ],
            "abstract": "Text style transfer aims to modify the style of a sentence while keeping its\ncontent unchanged. Recent style transfer systems often fail to faithfully\npreserve the content after changing the style. This paper proposes a structured\ncontent preserving model that leverages linguistic information in the\nstructured fine-grained supervisions to better preserve the style-independent\ncontent during style transfer. In particular, we achieve the goal by devising\nrich model objectives based on both the sentence's lexical information and a\nlanguage model that conditions on content. The resulting model therefore is\nencouraged to retain the semantic meaning of the target sentences. We perform\nextensive experiments that compare our model to other existing approaches in\nthe tasks of sentiment and political slant transfer. Our model achieves\nsignificant improvement in terms of both content preservation and style\ntransfer in automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06635v1",
            "date": "2018-10-02 07:23:42+00:00",
            "title": "Semi-supervised and Active-learning Scenarios: Efficient Acoustic Model Refinement for a Low Resource Indian Language",
            "authors": [
                "Maharajan Chellapriyadharshini",
                "Anoop Toffy",
                "Srinivasa Raghavan K. M.",
                "V Ramasubramanian"
            ],
            "abstract": "We address the problem of efficient acoustic-model refinement (continuous\nretraining) using semi-supervised and active learning for a low resource Indian\nlanguage, wherein the low resource constraints are having i) a small labeled\ncorpus from which to train a baseline `seed' acoustic model and ii) a large\ntraining corpus without orthographic labeling or from which to perform a data\nselection for manual labeling at low costs. The proposed semi-supervised\nlearning decodes the unlabeled large training corpus using the seed model and\nthrough various protocols, selects the decoded utterances with high reliability\nusing confidence levels (that correlate to the WER of the decoded utterances)\nand iterative bootstrapping. The proposed active learning protocol uses\nconfidence level based metric to select the decoded utterances from the large\nunlabeled corpus for further labeling. The semi-supervised learning protocols\ncan offer a WER reduction, from a poorly trained seed model, by as much as 50%\nof the best WER-reduction realizable from the seed model's WER, if the large\ncorpus were labeled and used for acoustic-model training. The active learning\nprotocols allow that only 60% of the entire training corpus be manually\nlabeled, to reach the same performance as the entire data.",
            "journal ref": "Proc. Interspeech 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06640v2",
            "date": "2018-10-11 22:50:38+00:00",
            "title": "Adversarial Text Generation Without Reinforcement Learning",
            "authors": [
                "David Donahue",
                "Anna Rumshisky"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have experienced a recent surge in\npopularity, performing competitively in a variety of tasks, especially in\ncomputer vision. However, GAN training has shown limited success in natural\nlanguage processing. This is largely because sequences of text are discrete,\nand thus gradients cannot propagate from the discriminator to the generator.\nRecent solutions use reinforcement learning to propagate approximate gradients\nto the generator, but this is inefficient to train. We propose to utilize an\nautoencoder to learn a low-dimensional representation of sentences. A GAN is\nthen trained to generate its own vectors in this space, which decode to\nrealistic utterances. We report both random and interpolated samples from the\ngenerator. Visualization of sentence vectors indicate our model correctly\nlearns the latent space of the autoencoder. Both human ratings and BLEU scores\nshow that our model generates realistic text against competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.06665v1",
            "date": "2018-10-15 20:22:44+00:00",
            "title": "Stop Illegal Comments: A Multi-Task Deep Learning Approach",
            "authors": [
                "Ahmed Elnaggar",
                "Bernhard Waltl",
                "Ingo Glaser",
                "J\u00f6rg Landthaler",
                "Elena Scepankova",
                "Florian Matthes"
            ],
            "abstract": "Deep learning methods are often difficult to apply in the legal domain due to\nthe large amount of labeled data required by deep learning methods. A recent\nnew trend in the deep learning community is the application of multi-task\nmodels that enable single deep neural networks to perform more than one task at\nthe same time, for example classification and translation tasks. These powerful\nnovel models are capable of transferring knowledge among different tasks or\ntraining sets and therefore could open up the legal domain for many deep\nlearning applications. In this paper, we investigate the transfer learning\ncapabilities of such a multi-task model on a classification task on the\npublicly available Kaggle toxic comment dataset for classifying illegal\ncomments and we can report promising results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.06667v2",
            "date": "2018-10-15 20:26:44+00:00",
            "title": "Deep Transfer Reinforcement Learning for Text Summarization",
            "authors": [
                "Yaser Keneshloo",
                "Naren Ramakrishnan",
                "Chandan K. Reddy"
            ],
            "abstract": "Deep neural networks are data hungry models and thus face difficulties when\nattempting to train on small text datasets. Transfer learning is a potential\nsolution but their effectiveness in the text domain is not as explored as in\nareas such as image analysis. In this paper, we study the problem of transfer\nlearning for text summarization and discuss why existing state-of-the-art\nmodels fail to generalize well on other (unseen) datasets. We propose a\nreinforcement learning framework based on a self-critic policy gradient\napproach which achieves good generalization and state-of-the-art results on a\nvariety of datasets. Through an extensive set of experiments, we also show the\nability of our proposed framework to fine-tune the text summarization model\nusing only a few training samples. To the best of our knowledge, this is the\nfirst work that studies transfer learning in text summarization and provides a\ngeneric solution that works well on unseen data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.06673v1",
            "date": "2018-10-15 20:38:00+00:00",
            "title": "Named-Entity Linking Using Deep Learning For Legal Documents: A Transfer Learning Approach",
            "authors": [
                "Ahmed Elnaggar",
                "Robin Otto",
                "Florian Matthes"
            ],
            "abstract": "In the legal domain it is important to differentiate between words in\ngeneral, and afterwards to link the occurrences of the same entities. The topic\nto solve these challenges is called Named-Entity Linking (NEL). Current\nsupervised neural networks designed for NEL use publicly available datasets for\ntraining and testing. However, this paper focuses especially on the aspect of\napplying transfer learning approach using networks trained for NEL to legal\ndocuments. Experiments show consistent improvement in the legal datasets that\nwere created from the European Union law in the scope of this research. Using\ntransfer learning approach, we reached F1-score of 98.90\\% and 98.01\\% on the\nlegal small and large test dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.06682v2",
            "date": "2018-10-15 20:50:05+00:00",
            "title": "Trellis Networks for Sequence Modeling",
            "authors": [
                "Shaojie Bai",
                "J. Zico Kolter",
                "Vladlen Koltun"
            ],
            "abstract": "We present trellis networks, a new architecture for sequence modeling. On the\none hand, a trellis network is a temporal convolutional network with special\nstructure, characterized by weight tying across depth and direct injection of\nthe input into deep layers. On the other hand, we show that truncated recurrent\nnetworks are equivalent to trellis networks with special sparsity structure in\ntheir weight matrices. Thus trellis networks with general weight matrices\ngeneralize truncated recurrent networks. We leverage these connections to\ndesign high-performing trellis networks that absorb structural and algorithmic\nelements from both recurrent and convolutional models. Experiments demonstrate\nthat trellis networks outperform the current state of the art methods on a\nvariety of challenging benchmarks, including word-level language modeling and\ncharacter-level language modeling tasks, and stress tests designed to evaluate\nlong-term memory retention. The code is available at\nhttps://github.com/locuslab/trellisnet .",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.06695v1",
            "date": "2018-10-10 09:57:32+00:00",
            "title": "Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms",
            "authors": [
                "Giancarlo D. Salton",
                "Robert J. Ross",
                "John D. Kelleher"
            ],
            "abstract": "Idioms pose problems to almost all Machine Translation systems. This type of\nlanguage is very frequent in day-to-day language use and cannot be simply\nignored. The recent interest in memory augmented models in the field of\nLanguage Modelling has aided the systems to achieve good results by bridging\nlong-distance dependencies. In this paper we explore the use of such techniques\ninto a Neural Machine Translation system to help in translation of idiomatic\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07217v2",
            "date": "2018-10-16 18:20:02+00:00",
            "title": "Hierarchical Generative Modeling for Controllable Speech Synthesis",
            "authors": [
                "Wei-Ning Hsu",
                "Yu Zhang",
                "Ron J. Weiss",
                "Heiga Zen",
                "Yonghui Wu",
                "Yuxuan Wang",
                "Yuan Cao",
                "Ye Jia",
                "Zhifeng Chen",
                "Jonathan Shen",
                "Patrick Nguyen",
                "Ruoming Pang"
            ],
            "abstract": "This paper proposes a neural sequence-to-sequence text-to-speech (TTS) model\nwhich can control latent attributes in the generated speech that are rarely\nannotated in the training data, such as speaking style, accent, background\nnoise, and recording conditions. The model is formulated as a conditional\ngenerative model based on the variational autoencoder (VAE) framework, with two\nlevels of hierarchical latent variables. The first level is a categorical\nvariable, which represents attribute groups (e.g. clean/noisy) and provides\ninterpretability. The second level, conditioned on the first, is a multivariate\nGaussian variable, which characterizes specific attribute configurations (e.g.\nnoise level, speaking rate) and enables disentangled fine-grained control over\nthese attributes. This amounts to using a Gaussian mixture model (GMM) for the\nlatent distribution. Extensive evaluation demonstrates its ability to control\nthe aforementioned attributes. In particular, we train a high-quality\ncontrollable TTS model on real found data, which is capable of inferring\nspeaker and style attributes from a noisy utterance and use it to synthesize\nclean speech with controllable speaking style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07320v1",
            "date": "2018-10-16 23:57:37+00:00",
            "title": "Exploring Sentence Vector Spaces through Automatic Summarization",
            "authors": [
                "Adly Templeton",
                "Jugal Kalita"
            ],
            "abstract": "Given vector representations for individual words, it is necessary to compute\nvector representations of sentences for many applications in a compositional\nmanner, often using artificial neural networks.\n  Relatively little work has explored the internal structure and properties of\nsuch sentence vectors. In this paper, we explore the properties of sentence\nvectors in the context of automatic summarization. In particular, we show that\ncosine similarity between sentence vectors and document vectors is strongly\ncorrelated with sentence importance and that vector semantics can identify and\ncorrect gaps between the sentences chosen so far and the document. In addition,\nwe identify specific dimensions which are linked to effective summaries. To our\nknowledge, this is the first time specific dimensions of sentence embeddings\nhave been connected to sentence properties. We also compare the features of\ndifferent methods of sentence embeddings. Many of these insights have\napplications in uses of sentence embeddings far beyond summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07513v1",
            "date": "2018-10-16 08:54:50+00:00",
            "title": "Multi-Task Deep Learning for Legal Document Translation, Summarization and Multi-Label Classification",
            "authors": [
                "Ahmed Elnaggar",
                "Christoph Gebendorfer",
                "Ingo Glaser",
                "Florian Matthes"
            ],
            "abstract": "The digitalization of the legal domain has been ongoing for a couple of\nyears. In that process, the application of different machine learning (ML)\ntechniques is crucial. Tasks such as the classification of legal documents or\ncontract clauses as well as the translation of those are highly relevant. On\nthe other side, digitized documents are barely accessible in this field,\nparticularly in Germany. Today, deep learning (DL) is one of the hot topics\nwith many publications and various applications. Sometimes it provides results\noutperforming the human level. Hence this technique may be feasible for the\nlegal domain as well. However, DL requires thousands of samples to provide\ndecent results. A potential solution to this problem is multi-task DL to enable\ntransfer learning. This approach may be able to overcome the data scarcity\nproblem in the legal domain, specifically for the German language. We applied\nthe state of the art multi-task model on three tasks: translation,\nsummarization, and multi-label classification. The experiments were conducted\non legal document corpora utilizing several task combinations as well as\nvarious model parameters. The goal was to find the optimal configuration for\nthe tasks at hand within the legal domain. The multi-task DL approach\noutperformed the state of the art results in all three tasks. This opens a new\ndirection to integrate DL technology more efficiently in the legal domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07767v1",
            "date": "2018-09-20 07:14:10+00:00",
            "title": "Architecture of Text Mining Application in Analyzing Public Sentiments of West Java Governor Election using Naive Bayes Classification",
            "authors": [
                "Suryanto Nugroho",
                "Prihandoko"
            ],
            "abstract": "The selection of West Java governor is one event that seizes the attention of\nthe public is no exception to social media users. Public opinion on a\nprospective regional leader can help predict electability and tendency of\nvoters. Data that can be used by the opinion mining process can be obtained\nfrom Twitter. Because the data is very varied form and very unstructured, it\nmust be managed and uninformed using data pre-processing techniques into\nsemi-structured data. This semi-structured information is followed by a\nclassification stage to categorize the opinion into negative or positive\nopinions. The research methodology uses a literature study where the research\nwill examine previous research on a similar topic. The purpose of this study is\nto find the right architecture to develop it into the application of twitter\nopinion mining to know public sentiments toward the election of the governor of\nwest java. The result of this research is that Twitter opinion mining is part\nof text mining where opinions in Twitter if they want to be classified, must go\nthrough the preprocessing text stage first. The preprocessing step required\nfrom twitter data is cleansing, case folding, POS Tagging and stemming. The\nresulting text mining architecture is an architecture that can be used for text\nmining research with different topics.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1810.08732v1",
            "date": "2018-10-20 02:00:35+00:00",
            "title": "Named Entity Recognition on Twitter for Turkish using Semi-supervised Learning with Word Embeddings",
            "authors": [
                "Eda Okur",
                "Hakan Demir",
                "Arzucan \u00d6zg\u00fcr"
            ],
            "abstract": "Recently, due to the increasing popularity of social media, the necessity for\nextracting information from informal text types, such as microblog texts, has\ngained significant attention. In this study, we focused on the Named Entity\nRecognition (NER) problem on informal text types for Turkish. We utilized a\nsemi-supervised learning approach based on neural networks. We applied a fast\nunsupervised method for learning continuous representations of words in vector\nspace. We made use of these obtained word embeddings, together with language\nindependent features that are engineered to work better on informal text types,\nfor generating a Turkish NER system on microblog texts. We evaluated our\nTurkish NER system on Twitter messages and achieved better F-score performances\nthan the published results of previously proposed NER systems on Turkish\ntweets. Since we did not employ any language dependent features, we believe\nthat our method can be easily adapted to microblog texts in other\nmorphologically rich languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.08802v1",
            "date": "2018-10-20 13:27:04+00:00",
            "title": "Hierarchical Text Generation using an Outline",
            "authors": [
                "Mehdi Drissi",
                "Olivia Watkins",
                "Jugal Kalita"
            ],
            "abstract": "Many challenges in natural language processing require generating text,\nincluding language translation, dialogue generation, and speech recognition.\nFor all of these problems, text generation becomes more difficult as the text\nbecomes longer. Current language models often struggle to keep track of\ncoherence for long pieces of text. Here, we attempt to have the model construct\nand use an outline of the text it generates to keep it focused. We find that\nthe usage of an outline improves perplexity. We do not find that using the\noutline improves human evaluation over a simpler baseline, revealing a\ndiscrepancy in perplexity and human perception. Similarly, hierarchical\ngeneration is not found to improve human evaluation scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09302v6",
            "date": "2018-10-22 14:10:01+00:00",
            "title": "BioSentVec: creating sentence embeddings for biomedical texts",
            "authors": [
                "Qingyu Chen",
                "Yifan Peng",
                "Zhiyong Lu"
            ],
            "abstract": "Sentence embeddings have become an essential part of today's natural language\nprocessing (NLP) systems, especially together advanced deep learning methods.\nAlthough pre-trained sentence encoders are available in the general domain,\nnone exists for biomedical texts to date. In this work, we introduce\nBioSentVec: the first open set of sentence embeddings trained with over 30\nmillion documents from both scholarly articles in PubMed and clinical notes in\nthe MIMIC-III Clinical Database. We evaluate BioSentVec embeddings in two\nsentence pair similarity tasks in different text genres. Our benchmarking\nresults demonstrate that the BioSentVec embeddings can better capture sentence\nsemantics compared to the other competitive alternatives and achieve\nstate-of-the-art performance in both tasks. We expect BioSentVec to facilitate\nthe research and development in biomedical text mining and to complement the\nexisting resources in biomedical word embeddings. BioSentVec is publicly\navailable at https://github.com/ncbi-nlp/BioSentVec",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09305v1",
            "date": "2018-10-18 05:29:41+00:00",
            "title": "WikiHow: A Large Scale Text Summarization Dataset",
            "authors": [
                "Mahnaz Koupaee",
                "William Yang Wang"
            ],
            "abstract": "Sequence-to-sequence models have recently gained the state of the art\nperformance in summarization. However, not too many large-scale high-quality\ndatasets are available and almost all the available ones are mainly news\narticles with specific writing style. Moreover, abstractive human-style systems\ninvolving description of the content at a deeper level require data with higher\nlevels of abstraction. In this paper, we present WikiHow, a dataset of more\nthan 230,000 article and summary pairs extracted and constructed from an online\nknowledge base written by different human authors. The articles span a wide\nrange of topics and therefore represent high diversity styles. We evaluate the\nperformance of the existing methods on WikiHow to present its challenges and\nset some baselines to further improve it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09311v1",
            "date": "2018-10-19 07:27:24+00:00",
            "title": "Revisiting Distributional Correspondence Indexing: A Python Reimplementation and New Experiments",
            "authors": [
                "Alejandro Moreo",
                "Andrea Esuli",
                "Fabrizio Sebastiani"
            ],
            "abstract": "This paper introduces PyDCI, a new implementation of Distributional\nCorrespondence Indexing (DCI) written in Python. DCI is a transfer learning\nmethod for cross-domain and cross-lingual text classification for which we had\nprovided an implementation (here called JaDCI) built on top of JaTeCS, a Java\nframework for text classification. PyDCI is a stand-alone version of DCI that\nexploits scikit-learn and the SciPy stack. We here report on new experiments\nthat we have carried out in order to test PyDCI, and in which we use as\nbaselines new high-performing methods that have appeared after DCI was\noriginally proposed. These experiments show that, thanks to a few subtle ways\nin which we have improved DCI, PyDCI outperforms both JaDCI and the\nabove-mentioned high-performing methods, and delivers the best known results on\nthe two popular benchmarks on which we had tested DCI, i.e.,\nMultiDomainSentiment (a.k.a. MDS -- for cross-domain adaptation) and\nWebis-CLS-10 (for cross-lingual adaptation). PyDCI, together with the code\nallowing to replicate our experiments, is available at\nhttps://github.com/AlexMoreo/pydci .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09312v1",
            "date": "2018-10-18 05:18:04+00:00",
            "title": "Analyzing and Interpreting Convolutional Neural Networks in NLP",
            "authors": [
                "Mahnaz Koupaee",
                "William Yang Wang"
            ],
            "abstract": "Convolutional neural networks have been successfully applied to various NLP\ntasks. However, it is not obvious whether they model different linguistic\npatterns such as negation, intensification, and clause compositionality to help\nthe decision-making process. In this paper, we apply visualization techniques\nto observe how the model can capture different linguistic features and how\nthese features can affect the performance of the model. Later on, we try to\nidentify the model errors and their sources. We believe that interpreting CNNs\nis the first step to understand the underlying semantic features which can\nraise awareness to further improve the performance and explainability of CNN\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09593v1",
            "date": "2018-10-22 23:19:43+00:00",
            "title": "MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare",
            "authors": [
                "Edward Choi",
                "Cao Xiao",
                "Walter F. Stewart",
                "Jimeng Sun"
            ],
            "abstract": "Deep learning models exhibit state-of-the-art performance for many predictive\nhealthcare tasks using electronic health records (EHR) data, but these models\ntypically require training data volume that exceeds the capacity of most\nhealthcare systems. External resources such as medical ontologies are used to\nbridge the data volume constraint, but this approach is often not directly\napplicable or useful because of inconsistencies with terminology. To solve the\ndata insufficiency challenge, we leverage the inherent multilevel structure of\nEHR data and, in particular, the encoded relationships among medical codes. We\npropose Multilevel Medical Embedding (MiME) which learns the multilevel\nembedding of EHR data while jointly performing auxiliary prediction tasks that\nrely on this inherent EHR structure without the need for external labels. We\nconducted two prediction tasks, heart failure prediction and sequential disease\nprediction, where MiME outperformed baseline methods in diverse evaluation\nsettings. In particular, MiME consistently outperformed all baselines when\npredicting heart failure on datasets of different volumes, especially\ndemonstrating the greatest performance improvement (15% relative gain in PR-AUC\nover the best baseline) on the smallest dataset, demonstrating its ability to\neffectively model the multilevel structure of EHR data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.09597v1",
            "date": "2018-10-22 23:37:31+00:00",
            "title": "Biomedical Document Clustering and Visualization based on the Concepts of Diseases",
            "authors": [
                "Setu Shah",
                "Xiao Luo"
            ],
            "abstract": "Document clustering is a text mining technique used to provide better\ndocument search and browsing in digital libraries or online corpora. A lot of\nresearch has been done on biomedical document clustering that is based on using\nexisting ontology. But, associations and co-occurrences of the medical concepts\nare not well represented by using ontology. In this research, a vector\nrepresentation of concepts of diseases and similarity measurement between\nconcepts are proposed. They identify the closest concepts of diseases in the\ncontext of a corpus. Each document is represented by using the vector space\nmodel. A weight scheme is proposed to consider both local content and\nassociations between concepts. A Self-Organizing Map is used as document\nclustering algorithm. The vector projection and visualization features of SOM\nenable visualization and analysis of the clusters distributions and\nrelationships on the two dimensional space. The experimental results show that\nthe proposed document clustering framework generates meaningful clusters and\nfacilitate visualization of the clusters based on the concepts of diseases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.09630v1",
            "date": "2018-10-23 02:16:12+00:00",
            "title": "A Neural Compositional Paradigm for Image Captioning",
            "authors": [
                "Bo Dai",
                "Sanja Fidler",
                "Dahua Lin"
            ],
            "abstract": "Mainstream captioning models often follow a sequential structure to generate\ncaptions, leading to issues such as introduction of irrelevant semantics, lack\nof diversity in the generated captions, and inadequate generalization\nperformance. In this paper, we present an alternative paradigm for image\ncaptioning, which factorizes the captioning procedure into two stages: (1)\nextracting an explicit semantic representation from the given image; and (2)\nconstructing the caption based on a recursive compositional procedure in a\nbottom-up manner. Compared to conventional ones, our paradigm better preserves\nthe semantic content through an explicit factorization of semantics and syntax.\nBy using the compositional generation procedure, caption construction follows a\nrecursive structure, which naturally fits the properties of human language.\nMoreover, the proposed compositional procedure requires less data to train,\ngeneralizes better, and yields more diverse captions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1810.09807v1",
            "date": "2018-10-23 12:09:37+00:00",
            "title": "PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution",
            "authors": [
                "Hong Chen",
                "Zhenhua Fan",
                "Hao Lu",
                "Alan L. Yuille",
                "Shu Rong"
            ],
            "abstract": "We introduce PreCo, a large-scale English dataset for coreference resolution.\nThe dataset is designed to embody the core challenges in coreference, such as\nentity representation, by alleviating the challenge of low overlap between\ntraining and test sets and enabling separated analysis of mention detection and\nmention clustering. To strengthen the training-test overlap, we collect a large\ncorpus of about 38K documents and 12.4M words which are mostly from the\nvocabulary of English-speaking preschoolers. Experiments show that with higher\ntraining-test overlap, error analysis on PreCo is more efficient than the one\non OntoNotes, a popular existing dataset. Furthermore, we annotate singleton\nmentions making it possible for the first time to quantify the influence that a\nmention detector makes on coreference resolution performance. The dataset is\nfreely available at https://preschool-lab.github.io/PreCo/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10126v7",
            "date": "2018-10-23 23:14:27+00:00",
            "title": "Area Attention",
            "authors": [
                "Yang Li",
                "Lukasz Kaiser",
                "Samy Bengio",
                "Si Si"
            ],
            "abstract": "Existing attention mechanisms are trained to attend to individual items in a\ncollection (the memory) with a predefined, fixed granularity, e.g., a word\ntoken or an image grid. We propose area attention: a way to attend to areas in\nthe memory, where each area contains a group of items that are structurally\nadjacent, e.g., spatially for a 2D memory such as images, or temporally for a\n1D memory such as natural language sentences. Importantly, the shape and the\nsize of an area are dynamically determined via learning, which enables a model\nto attend to information with varying granularity. Area attention can easily\nwork with existing model architectures such as multi-head attention for\nsimultaneously attending to multiple areas in the memory. We evaluate area\nattention on two tasks: neural machine translation (both character and\ntoken-level) and image captioning, and improve upon strong (state-of-the-art)\nbaselines in all the cases. These improvements are obtainable with a basic form\nof area attention that is parameter free.",
            "journal ref": "ICML 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.10147v2",
            "date": "2018-10-24 01:18:08+00:00",
            "title": "FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation",
            "authors": [
                "Xu Han",
                "Hao Zhu",
                "Pengfei Yu",
                "Ziyun Wang",
                "Yuan Yao",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "We present a Few-Shot Relation Classification Dataset (FewRel), consisting of\n70, 000 sentences on 100 relations derived from Wikipedia and annotated by\ncrowdworkers. The relation of each sentence is first recognized by distant\nsupervision methods, and then filtered by crowdworkers. We adapt the most\nrecent state-of-the-art few-shot learning methods for relation classification\nand conduct a thorough evaluation of these methods. Empirical results show that\neven the most competitive few-shot learning models struggle on this task,\nespecially as compared with humans. We also show that a range of different\nreasoning skills are needed to solve our task. These results indicate that\nfew-shot relation classification remains an open problem and still requires\nfurther research. Our detailed analysis points multiple directions for future\nresearch. All details and resources about the dataset and baselines are\nreleased on http://zhuhao.me/fewrel.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.10222v1",
            "date": "2018-10-24 07:34:45+00:00",
            "title": "Universal Language Model Fine-Tuning with Subword Tokenization for Polish",
            "authors": [
                "Piotr Czapla",
                "Jeremy Howard",
                "Marcin Kardas"
            ],
            "abstract": "Universal Language Model for Fine-tuning [arXiv:1801.06146] (ULMFiT) is one\nof the first NLP methods for efficient inductive transfer learning.\nUnsupervised pretraining results in improvements on many NLP tasks for English.\nIn this paper, we describe a new method that uses subword tokenization to adapt\nULMFiT to languages with high inflection. Our approach results in a new\nstate-of-the-art for the Polish language, taking first place in Task 3 of\nPolEval'18. After further training, our final model outperformed the second\nbest model by 35%. We have open-sourced our pretrained models and code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10419v1",
            "date": "2018-10-24 14:30:39+00:00",
            "title": "Effective extractive summarization using frequency-filtered entity relationship graphs",
            "authors": [
                "Archit Sakhadeo",
                "Nisheeth Srivastava"
            ],
            "abstract": "Word frequency-based methods for extractive summarization are easy to\nimplement and yield reasonable results across languages. However, they have\nsignificant limitations - they ignore the role of context, they offer uneven\ncoverage of topics in a document, and sometimes are disjointed and hard to\nread. We use a simple premise from linguistic typology - that English sentences\nare complete descriptors of potential interactions between entities, usually in\nthe order subject-verb-object - to address a subset of these difficulties. We\nhave developed a hybrid model of extractive summarization that combines\nword-frequency based keyword identification with information from automatically\ngenerated entity relationship graphs to select sentences for summaries.\nComparative evaluation with word-frequency and topic word-based methods shows\nthat the proposed method is competitive by conventional ROUGE standards, and\nyields moderately more informative summaries on average, as assessed by a large\npanel (N=94) of human raters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10665v2",
            "date": "2018-10-25 00:46:16+00:00",
            "title": "Engaging Image Captioning Via Personality",
            "authors": [
                "Kurt Shuster",
                "Samuel Humeau",
                "Hexiang Hu",
                "Antoine Bordes",
                "Jason Weston"
            ],
            "abstract": "Standard image captioning tasks such as COCO and Flickr30k are factual,\nneutral in tone and (to a human) state the obvious (e.g., \"a man playing a\nguitar\"). While such tasks are useful to verify that a machine understands the\ncontent of an image, they are not engaging to humans as captions. With this in\nmind we define a new task, Personality-Captions, where the goal is to be as\nengaging to humans as possible by incorporating controllable style and\npersonality traits. We collect and release a large dataset of 201,858 of such\ncaptions conditioned over 215 possible traits. We build models that combine\nexisting work from (i) sentence representations (Mazare et al., 2018) with\nTransformers trained on 1.7 billion dialogue examples; and (ii) image\nrepresentations (Mahajan et al., 2018) with ResNets trained on 3.5 billion\nsocial media images. We obtain state-of-the-art performance on Flickr30k and\nCOCO, and strong performance on our new task. Finally, online evaluations\nvalidate that our task and models are engaging to humans, with our best model\nclose to human performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1810.10802v1",
            "date": "2018-10-25 09:24:13+00:00",
            "title": "Tackling Sequence to Sequence Mapping Problems with Neural Networks",
            "authors": [
                "Lei Yu"
            ],
            "abstract": "In Natural Language Processing (NLP), it is important to detect the\nrelationship between two sequences or to generate a sequence of tokens given\nanother observed sequence. We call the type of problems on modelling sequence\npairs as sequence to sequence (seq2seq) mapping problems. A lot of research has\nbeen devoted to finding ways of tackling these problems, with traditional\napproaches relying on a combination of hand-crafted features, alignment models,\nsegmentation heuristics, and external linguistic resources. Although great\nprogress has been made, these traditional approaches suffer from various\ndrawbacks, such as complicated pipeline, laborious feature engineering, and the\ndifficulty for domain adaptation. Recently, neural networks emerged as a\npromising solution to many problems in NLP, speech recognition, and computer\nvision. Neural models are powerful because they can be trained end to end,\ngeneralise well to unseen examples, and the same framework can be easily\nadapted to a new domain.\n  The aim of this thesis is to advance the state-of-the-art in seq2seq mapping\nproblems with neural networks. We explore solutions from three major aspects:\ninvestigating neural models for representing sequences, modelling interactions\nbetween sequences, and using unpaired data to boost the performance of neural\nmodels. For each aspect, we propose novel models and evaluate their efficacy on\nvarious tasks of seq2seq mapping.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.10927v2",
            "date": "2018-10-25 15:27:23+00:00",
            "title": "Bayesian Compression for Natural Language Processing",
            "authors": [
                "Nadezhda Chirkova",
                "Ekaterina Lobacheva",
                "Dmitry Vetrov"
            ],
            "abstract": "In natural language processing, a lot of the tasks are successfully solved\nwith recurrent neural networks, but such models have a huge number of\nparameters. The majority of these parameters are often concentrated in the\nembedding layer, which size grows proportionally to the vocabulary length. We\npropose a Bayesian sparsification technique for RNNs which allows compressing\nthe RNN dozens or hundreds of times without time-consuming hyperparameters\ntuning. We also generalize the model for vocabulary sparsification to filter\nout unnecessary words and compress the RNN even further. We show that the\nchoice of the kept words is interpretable. Code is available on github:\nhttps://github.com/tipt0p/SparseBayesianRNN",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11181v2",
            "date": "2018-10-26 03:58:26+00:00",
            "title": "Neural Modular Control for Embodied Question Answering",
            "authors": [
                "Abhishek Das",
                "Georgia Gkioxari",
                "Stefan Lee",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We present a modular approach for learning policies for navigation over long\nplanning horizons from language input. Our hierarchical policy operates at\nmultiple timescales, where the higher-level master policy proposes subgoals to\nbe executed by specialized sub-policies. Our choice of subgoals is\ncompositional and semantic, i.e. they can be sequentially combined in arbitrary\norderings, and assume human-interpretable descriptions (e.g. 'exit room', 'find\nkitchen', 'find refrigerator', etc.).\n  We use imitation learning to warm-start policies at each level of the\nhierarchy, dramatically increasing sample efficiency, followed by reinforcement\nlearning. Independent reinforcement learning at each level of hierarchy enables\nsub-policies to adapt to consequences of their actions and recover from errors.\nSubsequent joint hierarchical training enables the master policy to adapt to\nthe sub-policies.\n  On the challenging EQA (Das et al., 2018) benchmark in House3D (Wu et al.,\n2018), requiring navigating diverse realistic indoor environments, our approach\noutperforms prior work by a significant margin, both in terms of navigation and\nquestion answering.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.11367v1",
            "date": "2018-10-22 20:05:42+00:00",
            "title": "LAMVI-2: A Visual Tool for Comparing and Tuning Word Embedding Models",
            "authors": [
                "Xin Rong",
                "Joshua Luckson",
                "Eytan Adar"
            ],
            "abstract": "Tuning machine learning models, particularly deep learning architectures, is\na complex process. Automated hyperparameter tuning algorithms often depend on\nspecific optimization metrics. However, in many situations, a developer trades\none metric against another: accuracy versus overfitting, precision versus\nrecall, smaller models and accuracy, etc. With deep learning, not only are the\nmodel's representations opaque, the model's behavior when parameters \"knobs\"\nare changed may also be unpredictable. Thus, picking the \"best\" model often\nrequires time-consuming model comparison. In this work, we introduce LAMVI-2, a\nvisual analytics system to support a developer in comparing hyperparameter\nsettings and outcomes. By focusing on word-embedding models (\"deep learning for\ntext\") we integrate views to compare both high-level statistics as well as\ninternal model behaviors (e.g., comparing word 'distances'). We demonstrate how\ndevelopers can work with LAMVI-2 to more quickly and accurately narrow down an\nappropriate and effective application-specific model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11414v1",
            "date": "2018-10-24 17:44:57+00:00",
            "title": "A Text Classification Application: Poet Detection from Poetry",
            "authors": [
                "Durmus Ozkan Sahin",
                "Oguz Emre Kural",
                "Erdal Kilic",
                "Armagan Karabina"
            ],
            "abstract": "With the widespread use of the internet, the size of the text data increases\nday by day. Poems can be given as an example of the growing text. In this\nstudy, we aim to classify poetry according to poet. Firstly, data set\nconsisting of three different poetry of poets written in English have been\nconstructed. Then, text categorization techniques are implemented on it.\nChi-Square technique are used for feature selection. In addition, five\ndifferent classification algorithms are tried. These algorithms are Sequential\nminimal optimization, Naive Bayes, C4.5 decision tree, Random Forest and\nk-nearest neighbors. Although each classifier showed very different results,\nover the 70% classification success rate was taken by sequential minimal\noptimization technique.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.11497v1",
            "date": "2018-10-26 18:44:52+00:00",
            "title": "Parsing Coordination for Spoken Language Understanding",
            "authors": [
                "Sanchit Agarwal",
                "Rahul Goel",
                "Tagyoung Chung",
                "Abhishek Sethi",
                "Arindam Mandal",
                "Spyros Matsoukas"
            ],
            "abstract": "Typical spoken language understanding systems provide narrow semantic parses\nusing a domain-specific ontology. The parses contain intents and slots that are\ndirectly consumed by downstream domain applications. In this work we discuss\nexpanding such systems to handle compound entities and intents by introducing a\ndomain-agnostic shallow parser that handles linguistic coordination. We show\nthat our model for parsing coordination learns domain-independent and\nslot-independent features and is able to segment conjunct boundaries of many\ndifferent phrasal categories. We also show that using adversarial training can\nbe effective for improving generalization across different slot types for\ncoordination parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11804v1",
            "date": "2018-10-28 12:17:27+00:00",
            "title": "Robots Learning to Say `No': Prohibition and Rejective Mechanisms in Acquisition of Linguistic Negation",
            "authors": [
                "Frank F\u00f6rster",
                "Joe Saunders",
                "Hagen Lehmann",
                "Chrystopher L. Nehaniv"
            ],
            "abstract": "`No' belongs to the first ten words used by children and embodies the first\nactive form of linguistic negation. Despite its early occurrence the details of\nits acquisition process remain largely unknown. The circumstance that `no'\ncannot be construed as a label for perceptible objects or events puts it\noutside of the scope of most modern accounts of language acquisition. Moreover,\nmost symbol grounding architectures will struggle to ground the word due to its\nnon-referential character. In an experimental study involving the child-like\nhumanoid robot iCub that was designed to illuminate the acquisition process of\nnegation words, the robot is deployed in several rounds of speech-wise\nunconstrained interaction with na\\\"ive participants acting as its language\nteachers. The results corroborate the hypothesis that affect or volition plays\na pivotal role in the socially distributed acquisition process. Negation words\nare prosodically salient within prohibitive utterances and negative intent\ninterpretations such that they can be easily isolated from the teacher's speech\nsignal. These words subsequently may be grounded in negative affective states.\nHowever, observations of the nature of prohibitive acts and the temporal\nrelationships between its linguistic and extra-linguistic components raise\nserious questions over the suitability of Hebbian-type algorithms for language\ngrounding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.11906v1",
            "date": "2018-10-28 23:40:54+00:00",
            "title": "Semi-Supervised Translation with MMD Networks",
            "authors": [
                "Mark Hamilton"
            ],
            "abstract": "This work aims to improve semi-supervised learning in a neural network\narchitecture by introducing a hybrid supervised and unsupervised cost function.\nThe unsupervised component is trained using a differentiable estimator of the\nMaximum Mean Discrepancy (MMD) distance between the network output and the\ntarget dataset. We introduce the notion of an $n$-channel network and several\nmethods to improve performance of these nets based on supervised\npre-initialization, and multi-scale kernels. This work investigates the\neffectiveness of these methods on language translation where very few quality\ntranslations are known \\textit{a priori}. We also present a thorough\ninvestigation of the hyper-parameter space of this method on both synthetic\ndata.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.11945v2",
            "date": "2018-10-29 04:05:35+00:00",
            "title": "STFT spectral loss for training a neural speech waveform model",
            "authors": [
                "Shinji Takaki",
                "Toru Nakashika",
                "Xin Wang",
                "Junichi Yamagishi"
            ],
            "abstract": "This paper proposes a new loss using short-time Fourier transform (STFT)\nspectra for the aim of training a high-performance neural speech waveform model\nthat predicts raw continuous speech waveform samples directly. Not only\namplitude spectra but also phase spectra obtained from generated speech\nwaveforms are used to calculate the proposed loss. We also mathematically show\nthat training of the waveform model on the basis of the proposed loss can be\ninterpreted as maximum likelihood training that assumes the amplitude and phase\nspectra of generated speech waveforms following Gaussian and von Mises\ndistributions, respectively. Furthermore, this paper presents a simple network\narchitecture as the speech waveform model, which is composed of uni-directional\nlong short-term memories (LSTMs) and an auto-regressive structure. Experimental\nresults showed that the proposed neural model synthesized high-quality speech\nwaveforms.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.11960v2",
            "date": "2018-10-29 05:25:21+00:00",
            "title": "Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language",
            "authors": [
                "Yusuke Yasuda",
                "Xin Wang",
                "Shinji Takaki",
                "Junichi Yamagishi"
            ],
            "abstract": "End-to-end speech synthesis is a promising approach that directly converts\nraw text to speech. Although it was shown that Tacotron2 outperforms classical\npipeline systems with regards to naturalness in English, its applicability to\nother languages is still unknown. Japanese could be one of the most difficult\nlanguages for which to achieve end-to-end speech synthesis, largely due to its\ncharacter diversity and pitch accents. Therefore, state-of-the-art systems are\nstill based on a traditional pipeline framework that requires a separate text\nanalyzer and duration model. Towards end-to-end Japanese speech synthesis, we\nextend Tacotron to systems with self-attention to capture long-term\ndependencies related to pitch accents and compare their audio quality with\nclassical pipeline systems under various conditions to show their pros and\ncons. In a large-scale listening test, we investigated the impacts of the\npresence of accentual-type labels, the use of force or predicted alignments,\nand acoustic features used as local condition parameters of the Wavenet\nvocoder. Our results reveal that although the proposed systems still do not\nmatch the quality of a top-line pipeline system for Japanese, we show important\nstepping stones towards end-to-end Japanese speech synthesis.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.11975v2",
            "date": "2018-10-29 06:46:37+00:00",
            "title": "On Controllable Sparse Alternatives to Softmax",
            "authors": [
                "Anirban Laha",
                "Saneem A. Chemmengath",
                "Priyanka Agrawal",
                "Mitesh M. Khapra",
                "Karthik Sankaranarayanan",
                "Harish G. Ramaswamy"
            ],
            "abstract": "Converting an n-dimensional vector to a probability distribution over n\nobjects is a commonly used component in many machine learning tasks like\nmulticlass classification, multilabel classification, attention mechanisms etc.\nFor this, several probability mapping functions have been proposed and employed\nin literature such as softmax, sum-normalization, spherical softmax, and\nsparsemax, but there is very little understanding in terms how they relate with\neach other. Further, none of the above formulations offer an explicit control\nover the degree of sparsity. To address this, we develop a unified framework\nthat encompasses all these formulations as special cases. This framework\nensures simple closed-form solutions and existence of sub-gradients suitable\nfor learning via backpropagation. Within this framework, we propose two novel\nsparse formulations, sparsegen-lin and sparsehourglass, that seek to provide a\ncontrol over the degree of desired sparsity. We further develop novel convex\nloss functions that help induce the behavior of aforementioned formulations in\nthe multilabel classification setting, showing improved performance. We also\ndemonstrate empirically that the proposed formulations, when used to compute\nattention weights, achieve better or comparable performance on standard seq2seq\ntasks like neural machine translation and abstractive summarization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.12001v2",
            "date": "2018-10-29 08:52:31+00:00",
            "title": "Cascaded CNN-resBiLSTM-CTC: An End-to-End Acoustic Model For Speech Recognition",
            "authors": [
                "Xinpei Zhou",
                "Jiwei Li",
                "Xi Zhou"
            ],
            "abstract": "Automatic speech recognition (ASR) tasks are resolved by end-to-end deep\nlearning models, which benefits us by less preparation of raw data, and easier\ntransformation between languages. We propose a novel end-to-end deep learning\nmodel architecture namely cascaded CNN-resBiLSTM-CTC. In the proposed model, we\nadd residual blocks in BiLSTM layers to extract sophisticated phoneme and\nsemantic information together, and apply cascaded structure to pay more\nattention mining information of hard negative samples. By applying both simple\nFast Fourier Transform (FFT) technique and n-gram language model (LM) rescoring\nmethod, we manage to achieve word error rate (WER) of 3.41% on LibriSpeech test\nclean corpora. Furthermore, we propose a new batch-varied method to speed up\nthe training process in length-varied tasks, which result in 25% less training\ntime.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.12051v1",
            "date": "2018-10-29 10:53:31+00:00",
            "title": "Speaking style adaptation in Text-To-Speech synthesis using Sequence-to-sequence models with attention",
            "authors": [
                "Bajibabu Bollepalli",
                "Lauri Juvela",
                "Paavo Alku"
            ],
            "abstract": "Currently, there are increasing interests in text-to-speech (TTS) synthesis\nto use sequence-to-sequence models with attention. These models are end-to-end\nmeaning that they learn both co-articulation and duration properties directly\nfrom text and speech. Since these models are entirely data-driven, they need\nlarge amounts of data to generate synthetic speech with good quality. However,\nin challenging speaking styles, such as Lombard speech, it is difficult to\nrecord sufficiently large speech corpora. Therefore, in this study we propose a\ntransfer learning method to adapt a sequence-to-sequence based TTS system of\nnormal speaking style to Lombard style. Moreover, we experiment with a WaveNet\nvocoder in synthesis of Lombard speech. We conducted subjective evaluations to\nassess the performance of the adapted TTS systems. The subjective evaluation\nresults indicated that an adaptation system with the WaveNet vocoder clearly\noutperformed the conventional deep neural network based TTS system in synthesis\nof Lombard speech.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1810.12085v1",
            "date": "2018-10-26 16:36:27+00:00",
            "title": "Extractive Summarization of EHR Discharge Notes",
            "authors": [
                "Emily Alsentzer",
                "Anne Kim"
            ],
            "abstract": "Patient summarization is essential for clinicians to provide coordinated care\nand practice effective communication. Automated summarization has the potential\nto save time, standardize notes, aid clinical decision making, and reduce\nmedical errors. Here we provide an upper bound on extractive summarization of\ndischarge notes and develop an LSTM model to sequentially label topics of\nhistory of present illness notes. We achieve an F1 score of 0.876, which\nindicates that this model can be employed to create a dataset for evaluation of\nextractive summarization methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.12118v1",
            "date": "2018-10-26 12:34:21+00:00",
            "title": "Finding Answers from the Word of God: Domain Adaptation for Neural Networks in Biblical Question Answering",
            "authors": [
                "Helen Jiahe Zhao",
                "Jiamou Liu"
            ],
            "abstract": "Question answering (QA) has significantly benefitted from deep learning\ntechniques in recent years. However, domain-specific QA remains a challenge due\nto the significant amount of data required to train a neural network. This\npaper studies the answer sentence selection task in the Bible domain and answer\nquestions by selecting relevant verses from the Bible. For this purpose, we\ncreate a new dataset BibleQA based on bible trivia questions and propose three\nneural network models for our task. We pre-train our models on a large-scale QA\ndataset, SQuAD, and investigate the effect of transferring weights on model\naccuracy. Furthermore, we also measure the model accuracies with different\nanswer context lengths and different Bible translations. We affirm that\ntransfer learning has a noticeable improvement in the model accuracy. We\nachieve relatively good results with shorter context lengths, whereas longer\ncontext lengths decreased model accuracy. We also find that using a more modern\nBible translation in the dataset has a positive effect on the task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.12366v1",
            "date": "2018-10-29 19:14:26+00:00",
            "title": "Do Explanations make VQA Models more Predictable to a Human?",
            "authors": [
                "Arjun Chandrasekaran",
                "Viraj Prabhu",
                "Deshraj Yadav",
                "Prithvijit Chattopadhyay",
                "Devi Parikh"
            ],
            "abstract": "A rich line of research attempts to make deep neural networks more\ntransparent by generating human-interpretable 'explanations' of their decision\nprocess, especially for interactive tasks like Visual Question Answering (VQA).\nIn this work, we analyze if existing explanations indeed make a VQA model --\nits responses as well as failures -- more predictable to a human. Surprisingly,\nwe find that they do not. On the other hand, we find that human-in-the-loop\napproaches that treat the model as a black-box do.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1810.12406v1",
            "date": "2018-10-29 20:59:56+00:00",
            "title": "Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks",
            "authors": [
                "Patrick H. Chen",
                "Si Si",
                "Sanjiv Kumar",
                "Yang Li",
                "Cho-Jui Hsieh"
            ],
            "abstract": "Neural language models have been widely used in various NLP tasks, including\nmachine translation, next word prediction and conversational agents. However,\nit is challenging to deploy these models on mobile devices due to their slow\nprediction speed, where the bottleneck is to compute top candidates in the\nsoftmax layer. In this paper, we introduce a novel softmax layer approximation\nalgorithm by exploiting the clustering structure of context vectors. Our\nalgorithm uses a light-weight screening model to predict a much smaller set of\ncandidate words based on the given context, and then conducts an exact softmax\nonly within that subset. Training such a procedure end-to-end is challenging as\ntraditional clustering methods are discrete and non-differentiable, and thus\nunable to be used with back-propagation in the training process. Using the\nGumbel softmax, we are able to train the screening model end-to-end on the\ntraining set to exploit data distribution. The algorithm achieves an order of\nmagnitude faster inference than the original softmax layer for predicting\ntop-$k$ words in various tasks such as beam search in machine translation or\nnext words prediction. For example, for machine translation task on German to\nEnglish dataset with around 25K vocabulary, we can achieve 20.4 times speed up\nwith 98.9\\% precision@1 and 99.3\\% precision@5 with the original softmax layer\nprediction, while state-of-the-art ~\\citep{MSRprediction} only achieves 6.7x\nspeedup with 98.7\\% precision@1 and 98.1\\% precision@5 for the same task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.12566v1",
            "date": "2018-10-30 08:11:45+00:00",
            "title": "Almost-unsupervised Speech Recognition with Close-to-zero Resource Based on Phonetic Structures Learned from Very Small Unpaired Speech and Text Data",
            "authors": [
                "Yi-Chen Chen",
                "Chia-Hao Shen",
                "Sung-Feng Huang",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Producing a large amount of annotated speech data for training ASR systems\nremains difficult for more than 95% of languages all over the world which are\nlow-resourced. However, we note human babies start to learn the language by the\nsounds of a small number of exemplar words without hearing a large amount of\ndata. We initiate some preliminary work in this direction in this paper. Audio\nWord2Vec is used to obtain embeddings of spoken words which carry phonetic\ninformation extracted from the signals. An autoencoder is used to generate\nembeddings of text words based on the articulatory features for the phoneme\nsequences. Both sets of embeddings for spoken and text words describe similar\nphonetic structures among words in their respective latent spaces. A mapping\nrelation from the audio embeddings to text embeddings actually gives the\nword-level ASR. This can be learned by aligning a small number of spoken words\nand the corresponding text words in the embedding spaces. In the initial\nexperiments only 200 annotated spoken words and one hour of speech data without\nannotation gave a word accuracy of 27.5%, which is low but a good starting\npoint.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12698v1",
            "date": "2018-10-30 12:23:35+00:00",
            "title": "Compositional Attention Networks for Interpretability in Natural Language Question Answering",
            "authors": [
                "Muru Selvakumar",
                "Suriyadeepan Ramamoorthy",
                "Vaidheeswaran Archana",
                "Malaikannan Sankarasubbu"
            ],
            "abstract": "MAC Net is a compositional attention network designed for Visual Question\nAnswering. We propose a modified MAC net architecture for Natural Language\nQuestion Answering. Question Answering typically requires Language\nUnderstanding and multi-step Reasoning. MAC net's unique architecture - the\nseparation between memory and control, facilitates data-driven iterative\nreasoning. This makes it an ideal candidate for solving tasks that involve\nlogical reasoning. Our experiments with 20 bAbI tasks demonstrate the value of\nMAC net as a data-efficient and interpretable architecture for Natural Language\nQuestion Answering. The transparent nature of MAC net provides a highly\ngranular view of the reasoning steps taken by the network in answering a query.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.12735v2",
            "date": "2018-10-30 13:49:37+00:00",
            "title": "Spoken Language Understanding on the Edge",
            "authors": [
                "Alaa Saade",
                "Alice Coucke",
                "Alexandre Caulier",
                "Joseph Dureau",
                "Adrien Ball",
                "Th\u00e9odore Bluche",
                "David Leroy",
                "Cl\u00e9ment Doumouro",
                "Thibault Gisselbrecht",
                "Francesco Caltagirone",
                "Thibaut Lavril",
                "Ma\u00ebl Primet"
            ],
            "abstract": "We consider the problem of performing Spoken Language Understanding (SLU) on\nsmall devices typical of IoT applications. Our contributions are twofold.\nFirst, we outline the design of an embedded, private-by-design SLU system and\nshow that it has performance on par with cloud-based commercial solutions.\nSecond, we release the datasets used in our experiments in the interest of\nreproducibility and in the hope that they can prove useful to the SLU\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12752v2",
            "date": "2018-10-30 14:08:30+00:00",
            "title": "Long Short-Term Attention",
            "authors": [
                "Guoqiang Zhong",
                "Xin Lin",
                "Kang Chen",
                "Qingyang Li",
                "Kaizhu Huang"
            ],
            "abstract": "Attention is an important cognition process of humans, which helps humans\nconcentrate on critical information during their perception and learning.\nHowever, although many machine learning models can remember information of\ndata, they have no the attention mechanism. For example, the long short-term\nmemory (LSTM) network is able to remember sequential information, but it cannot\npay special attention to part of the sequences. In this paper, we present a\nnovel model called long short-term attention (LSTA), which seamlessly\nintegrates the attention mechanism into the inner cell of LSTM. More than\nprocessing long short term dependencies, LSTA can focus on important\ninformation of the sequences with the attention mechanism. Extensive\nexperiments demonstrate that LSTA outperforms LSTM and related models on the\nsequence learning tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.12754v1",
            "date": "2018-10-30 14:09:19+00:00",
            "title": "Recurrent Attention Unit",
            "authors": [
                "Guoqiang Zhong",
                "Guohua Yue",
                "Xiao Ling"
            ],
            "abstract": "Recurrent Neural Network (RNN) has been successfully applied in many sequence\nlearning problems. Such as handwriting recognition, image description, natural\nlanguage processing and video motion analysis. After years of development,\nresearchers have improved the internal structure of the RNN and introduced many\nvariants. Among others, Gated Recurrent Unit (GRU) is one of the most widely\nused RNN model. However, GRU lacks the capability of adaptively paying\nattention to certain regions or locations, so that it may cause information\nredundancy or loss during leaning. In this paper, we propose a RNN model,\ncalled Recurrent Attention Unit (RAU), which seamlessly integrates the\nattention mechanism into the interior of GRU by adding an attention gate. The\nattention gate can enhance GRU's ability to remember long-term memory and help\nmemory cells quickly discard unimportant content. RAU is capable of extracting\ninformation from the sequential data by adaptively selecting a sequence of\nregions or locations and pay more attention to the selected regions during\nlearning. Extensive experiments on image classification, sentiment\nclassification and language modeling show that RAU consistently outperforms GRU\nand other baseline methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1810.12780v4",
            "date": "2018-10-30 14:44:24+00:00",
            "title": "Advancing PICO Element Detection in Biomedical Text via Deep Neural Networks",
            "authors": [
                "Di Jin",
                "Peter Szolovits"
            ],
            "abstract": "In evidence-based medicine (EBM), defining a clinical question in terms of\nthe specific patient problem aids the physicians to efficiently identify\nappropriate resources and search for the best available evidence for medical\ntreatment. In order to formulate a well-defined, focused clinical question, a\nframework called PICO is widely used, which identifies the sentences in a given\nmedical text that belong to the four components typically reported in clinical\ntrials: Participants/Problem (P), Intervention (I), Comparison (C) and Outcome\n(O). In this work, we propose a novel deep learning model for recognizing PICO\nelements in biomedical abstracts. Based on the previous state-of-the-art\nbidirectional long-short term memory (biLSTM) plus conditional random field\n(CRF) architecture, we add another layer of biLSTM upon the sentence\nrepresentation vectors so that the contextual information from surrounding\nsentences can be gathered to help infer the interpretation of the current one.\nIn addition, we propose two methods to further generalize and improve the\nmodel: adversarial training and unsupervised pre-training over large corpora.\nWe tested our proposed approach over two benchmark datasets. One is the\nPubMed-PICO dataset, where our best results outperform the previous best by\n5.5%, 7.9%, and 5.8% for P, I, and O elements in terms of F1 score,\nrespectively. And for the other dataset named NICTA-PIBOSO, the improvements\nfor P/I/O elements are 2.4%, 13.6%, and 1.0% in F1 score, respectively.\nOverall, our proposed deep learning model can obtain unprecedented PICO element\ndetection accuracy while avoiding the need for any manual feature selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12859v1",
            "date": "2018-10-30 16:58:36+00:00",
            "title": "JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis",
            "authors": [
                "Jaejun Lee",
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "Used for simple commands recognition on devices from smart routers to mobile\nphones, keyword spotting systems are everywhere. Ubiquitous as well are web\napplications, which have grown in popularity and complexity over the last\ndecade with significant improvements in usability under cross-platform\nconditions. However, despite their obvious advantage in natural language\ninteraction, voice-enabled web applications are still far and few between. In\nthis work, we attempt to bridge this gap by bringing keyword spotting\ncapabilities directly into the browser. To our knowledge, we are the first to\ndemonstrate a fully-functional implementation of convolutional neural networks\nin pure JavaScript that runs in any standards-compliant browser. We also apply\nnetwork slimming, a model compression technique, to explore the\naccuracy-efficiency tradeoffs, reporting latency measurements on a range of\ndevices and software. Overall, our robust, cross-device implementation for\nkeyword spotting realizes a new paradigm for serving neural network\napplications, and one of our slim models reduces latency by 66% with a minimal\ndecrease in accuracy of 4% from 94% to 90%.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1810.12897v1",
            "date": "2018-10-30 17:49:46+00:00",
            "title": "Topic-Specific Sentiment Analysis Can Help Identify Political Ideology",
            "authors": [
                "Sumit Bhatia",
                "Deepak P"
            ],
            "abstract": "Ideological leanings of an individual can often be gauged by the sentiment\none expresses about different issues. We propose a simple framework that\nrepresents a political ideology as a distribution of sentiment polarities\ntowards a set of topics. This representation can then be used to detect\nideological leanings of documents (speeches, news articles, etc.) based on the\nsentiments expressed towards different topics. Experiments performed using a\nwidely used dataset show the promise of our proposed approach that achieves\ncomparable performance to other methods despite being much simpler and more\ninterpretable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.12936v1",
            "date": "2018-10-30 18:03:12+00:00",
            "title": "NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval",
            "authors": [
                "Canjia Li",
                "Yingfei Sun",
                "Ben He",
                "Le Wang",
                "Kai Hui",
                "Andrew Yates",
                "Le Sun",
                "Jungang Xu"
            ],
            "abstract": "Pseudo-relevance feedback (PRF) is commonly used to boost the performance of\ntraditional information retrieval (IR) models by using top-ranked documents to\nidentify and weight new query terms, thereby reducing the effect of\nquery-document vocabulary mismatches. While neural retrieval models have\nrecently demonstrated strong results for ad-hoc retrieval, combining them with\nPRF is not straightforward due to incompatibilities between existing PRF\napproaches and neural architectures. To bridge this gap, we propose an\nend-to-end neural PRF framework that can be used with existing neural IR models\nby embedding different neural models as building blocks. Extensive experiments\non two standard test collections confirm the effectiveness of the proposed NPRF\nframework in improving the performance of two state-of-the-art neural IR\nmodels.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.13024v2",
            "date": "2018-10-30 22:39:02+00:00",
            "title": "Bi-Directional Lattice Recurrent Neural Networks for Confidence Estimation",
            "authors": [
                "Qiujia Li",
                "Preben Ness",
                "Anton Ragni",
                "Mark Gales"
            ],
            "abstract": "The standard approach to mitigate errors made by an automatic speech\nrecognition system is to use confidence scores associated with each predicted\nword. In the simplest case, these scores are word posterior probabilities\nwhilst more complex schemes utilise bi-directional recurrent neural network\n(BiRNN) models. A number of upstream and downstream applications, however, rely\non confidence scores assigned not only to 1-best hypotheses but to all words\nfound in confusion networks or lattices. These include but are not limited to\nspeaker adaptation, semi-supervised training and information retrieval.\nAlthough word posteriors could be used in those applications as confidence\nscores, they are known to have reliability issues. To make improved confidence\nscores more generally available, this paper shows how BiRNNs can be extended\nfrom 1-best sequences to confusion network and lattice structures. Experiments\nare conducted using one of the Cambridge University submissions to the IARPA\nOpenKWS 2016 competition. The results show that confusion network and\nlattice-based BiRNNs can provide a significant improvement in confidence\nestimation.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.13025v1",
            "date": "2018-10-30 22:39:54+00:00",
            "title": "Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks",
            "authors": [
                "Anton Ragni",
                "Qiujia Li",
                "Mark Gales",
                "Yu Wang"
            ],
            "abstract": "The standard approach to assess reliability of automatic speech\ntranscriptions is through the use of confidence scores. If accurate, these\nscores provide a flexible mechanism to flag transcription errors for upstream\nand downstream applications. One challenging type of errors that recognisers\nmake are deletions. These errors are not accounted for by the standard\nconfidence estimation schemes and are hard to rectify in the upstream and\ndownstream processing. High deletion rates are prominent in limited resource\nand highly mismatched training/testing conditions studied under IARPA Babel and\nMaterial programs. This paper looks at the use of bidirectional recurrent\nneural networks to yield confidence estimates in predicted as well as deleted\nwords. Several simple schemes are examined for combination. To assess\nusefulness of this approach, the combined confidence score is examined for\nuntranscribed data selection that favours transcriptions with lower deletion\nerrors. Experiments are conducted using IARPA Babel/Material program languages.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.13048v1",
            "date": "2018-10-31 00:23:16+00:00",
            "title": "Attentive Filtering Networks for Audio Replay Attack Detection",
            "authors": [
                "Cheng-I Lai",
                "Alberto Abad",
                "Korin Richmond",
                "Junichi Yamagishi",
                "Najim Dehak",
                "Simon King"
            ],
            "abstract": "An attacker may use a variety of techniques to fool an automatic speaker\nverification system into accepting them as a genuine user. Anti-spoofing\nmethods meanwhile aim to make the system robust against such attacks. The\nASVspoof 2017 Challenge focused specifically on replay attacks, with the\nintention of measuring the limits of replay attack detection as well as\ndeveloping countermeasures against them. In this work, we propose our replay\nattacks detection system - Attentive Filtering Network, which is composed of an\nattention-based filtering mechanism that enhances feature representations in\nboth the frequency and time domains, and a ResNet-based classifier. We show\nthat the network enables us to visualize the automatically acquired feature\nrepresentations that are helpful for spoofing detection. Attentive Filtering\nNetwork attains an evaluation EER of 8.99$\\%$ on the ASVspoof 2017 Version 2.0\ndataset. With system fusion, our best system further obtains a 30$\\%$ relative\nimprovement over the ASVspoof 2017 enhanced baseline system.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.13088v2",
            "date": "2018-10-31 03:10:37+00:00",
            "title": "Attention-based sequence-to-sequence model for speech recognition: development of state-of-the-art system on LibriSpeech and its application to non-native English",
            "authors": [
                "Yan Yin",
                "Ramon Prieto",
                "Bin Wang",
                "Jianwei Zhou",
                "Yiwei Gu",
                "Yang Liu",
                "Hui Lin"
            ],
            "abstract": "Recent research has shown that attention-based sequence-to-sequence models\nsuch as Listen, Attend, and Spell (LAS) yield comparable results to\nstate-of-the-art ASR systems on various tasks. In this paper, we describe the\ndevelopment of such a system and demonstrate its performance on two tasks:\nfirst we achieve a new state-of-the-art word error rate of 3.43% on the test\nclean subset of LibriSpeech English data; second on non-native English speech,\nincluding both read speech and spontaneous speech, we obtain very competitive\nresults compared to a conventional system built with the most updated Kaldi\nrecipe.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13107v1",
            "date": "2018-10-31 05:05:37+00:00",
            "title": "End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "The speech chain mechanism integrates automatic speech recognition (ASR) and\ntext-to-speech synthesis (TTS) modules into a single cycle during training. In\nour previous work, we applied a speech chain mechanism as a semi-supervised\nlearning. It provides the ability for ASR and TTS to assist each other when\nthey receive unpaired data and let them infer the missing pair and optimize the\nmodel with reconstruction loss. If we only have speech without transcription,\nASR generates the most likely transcription from the speech data, and then TTS\nuses the generated transcription to reconstruct the original speech features.\nHowever, in previous papers, we just limited our back-propagation to the\nclosest module, which is the TTS part. One reason is that back-propagating the\nerror through the ASR is challenging due to the output of the ASR are discrete\ntokens, creating non-differentiability between the TTS and ASR. In this paper,\nwe address this problem and describe how to thoroughly train a speech chain\nend-to-end for reconstruction loss using a straight-through estimator (ST).\nExperimental results revealed that, with sampling from ST-Gumbel-Softmax, we\nwere able to update ASR parameters and improve the ASR performances by 11\\%\nrelative CER reduction compared to the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.13407v2",
            "date": "2018-10-31 17:07:14+00:00",
            "title": "On The Inductive Bias of Words in Acoustics-to-Word Models",
            "authors": [
                "Hao Tang",
                "James Glass"
            ],
            "abstract": "Acoustics-to-word models are end-to-end speech recognizers that use words as\ntargets without relying on pronunciation dictionaries or graphemes. These\nmodels are notoriously difficult to train due to the lack of linguistic\nknowledge. It is also unclear how the amount of training data impacts the\noptimization and generalization of such models. In this work, we study the\noptimization and generalization of acoustics-to-word models under different\namounts of training data. In addition, we study three types of inductive bias,\nleveraging a pronunciation dictionary, word boundary annotations, and\nconstraints on word durations. We find that constraining word durations leads\nto the most improvement. Finally, we analyze the word embedding space learned\nby the model, and find that the space has a structure dominated by the\npronunciation of words. This suggests that the contexts of words, instead of\ntheir phonetic structure, should be the future focus of inductive bias in\nacoustics-to-word models.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.00062v1",
            "date": "2018-10-31 18:54:27+00:00",
            "title": "An Interdisciplinary Comparison of Sequence Modeling Methods for Next-Element Prediction",
            "authors": [
                "Niek Tax",
                "Irene Teinemaa",
                "Sebastiaan J. van Zelst"
            ],
            "abstract": "Data of sequential nature arise in many application domains in forms of, e.g.\ntextual data, DNA sequences, and software execution traces. Different research\ndisciplines have developed methods to learn sequence models from such datasets:\n(i) in the machine learning field methods such as (hidden) Markov models and\nrecurrent neural networks have been developed and successfully applied to a\nwide-range of tasks, (ii) in process mining process discovery techniques aim to\ngenerate human-interpretable descriptive models, and (iii) in the grammar\ninference field the focus is on finding descriptive models in the form of\nformal grammars. Despite their different focuses, these fields share a common\ngoal - learning a model that accurately describes the behavior in the\nunderlying data. Those sequence models are generative, i.e, they can predict\nwhat elements are likely to occur after a given unfinished sequence. So far,\nthese fields have developed mainly in isolation from each other and no\ncomparison exists. This paper presents an interdisciplinary experimental\nevaluation that compares sequence modeling techniques on the task of\nnext-element prediction on four real-life sequence datasets. The results\nindicate that machine learning techniques that generally have no aim at\ninterpretability in terms of accuracy outperform techniques from the process\nmining and grammar inference fields that aim to yield interpretable models.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1811.00135v1",
            "date": "2018-10-31 22:04:22+00:00",
            "title": "Dirichlet Variational Autoencoder for Text Modeling",
            "authors": [
                "Yijun Xiao",
                "Tiancheng Zhao",
                "William Yang Wang"
            ],
            "abstract": "We introduce an improved variational autoencoder (VAE) for text modeling with\ntopic information explicitly modeled as a Dirichlet latent variable. By\nproviding the proposed model topic awareness, it is more superior at\nreconstructing input texts. Furthermore, due to the inherent interactions\nbetween the newly introduced Dirichlet variable and the conventional\nmultivariate Gaussian variable, the model is less prone to KL divergence\nvanishing. We derive the variational lower bound for the new model and conduct\nexperiments on four different data sets. The results show that the proposed\nmodel is superior at text reconstruction across the latent space and\nclassifications on learned representations have higher test accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00147v1",
            "date": "2018-10-31 22:59:57+00:00",
            "title": "DOLORES: Deep Contextualized Knowledge Graph Embeddings",
            "authors": [
                "Haoyu Wang",
                "Vivek Kulkarni",
                "William Yang Wang"
            ],
            "abstract": "We introduce a new method DOLORES for learning knowledge graph embeddings\nthat effectively captures contextual cues and dependencies among entities and\nrelations. First, we note that short paths on knowledge graphs comprising of\nchains of entities and relations can encode valuable information regarding\ntheir contextual usage. We operationalize this notion by representing knowledge\ngraphs not as a collection of triples but as a collection of entity-relation\nchains, and learn embeddings for entities and relations using deep neural\nmodels that capture such contextual usage. In particular, our model is based on\nBi-Directional LSTMs and learn deep representations of entities and relations\nfrom constructed entity-relation chains. We show that these representations can\nvery easily be incorporated into existing models to significantly advance the\nstate of the art on several knowledge graph prediction tasks like link\nprediction, triple classification, and missing relation type prediction (in\nsome cases by at least 9.5%).",
            "journal ref": "Automated Knowledge Base Construction (AKBC), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00196v2",
            "date": "2018-11-01 02:45:57+00:00",
            "title": "Towards Explainable NLP: A Generative Explanation Framework for Text Classification",
            "authors": [
                "Hui Liu",
                "Qingyu Yin",
                "William Yang Wang"
            ],
            "abstract": "Building explainable systems is a critical problem in the field of Natural\nLanguage Processing (NLP), since most machine learning models provide no\nexplanations for the predictions. Existing approaches for explainable machine\nlearning systems tend to focus on interpreting the outputs or the connections\nbetween inputs and outputs. However, the fine-grained information is often\nignored, and the systems do not explicitly generate the human-readable\nexplanations. To better alleviate this problem, we propose a novel generative\nexplanation framework that learns to make classification decisions and generate\nfine-grained explanations at the same time. More specifically, we introduce the\nexplainable factor and the minimum risk training approach that learn to\ngenerate more reasonable explanations. We construct two new datasets that\ncontain summaries, rating scores, and fine-grained reasons. We conduct\nexperiments on both datasets, comparing with several strong neural network\nbaseline systems. Experimental results show that our method surpasses all\nbaselines on both datasets, and is able to generate concise explanations at the\nsame time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00198v1",
            "date": "2018-11-01 03:04:09+00:00",
            "title": "MOHONE: Modeling Higher Order Network Effects in KnowledgeGraphs via Network Infused Embeddings",
            "authors": [
                "Hao Yu",
                "Vivek Kulkarni",
                "William Wang"
            ],
            "abstract": "Many knowledge graph embedding methods operate on triples and are therefore\nimplicitly limited by a very local view of the entire knowledge graph. We\npresent a new framework MOHONE to effectively model higher order network\neffects in knowledge-graphs, thus enabling one to capture varying degrees of\nnetwork connectivity (from the local to the global). Our framework is generic,\nexplicitly models the network scale, and captures two different aspects of\nsimilarity in networks: (a) shared local neighborhood and (b) structural\nrole-based similarity. First, we introduce methods that learn network\nrepresentations of entities in the knowledge graph capturing these varied\naspects of similarity. We then propose a fast, efficient method to incorporate\nthe information captured by these network representations into existing\nknowledge graph embeddings. We show that our method consistently and\nsignificantly improves the performance on link prediction of several different\nknowledge-graph embedding methods including TRANSE, TRANSD, DISTMULT, and\nCOMPLEX(by at least 4 points or 17% in some cases).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00239v2",
            "date": "2018-11-01 05:22:01+00:00",
            "title": "Progressive Memory Banks for Incremental Domain Adaptation",
            "authors": [
                "Nabiha Asghar",
                "Lili Mou",
                "Kira A. Selby",
                "Kevin D. Pantasdo",
                "Pascal Poupart",
                "Xin Jiang"
            ],
            "abstract": "This paper addresses the problem of incremental domain adaptation (IDA) in\nnatural language processing (NLP). We assume each domain comes one after\nanother, and that we could only access data in the current domain. The goal of\nIDA is to build a unified model performing well on all the domains that we have\nencountered. We adopt the recurrent neural network (RNN) widely used in NLP,\nbut augment it with a directly parameterized memory bank, which is retrieved by\nan attention mechanism at each step of RNN transition. The memory bank provides\na natural way of IDA: when adapting our model to a new domain, we progressively\nadd new slots to the memory bank, which increases the number of parameters, and\nthus the model capacity. We learn the new memory slots and fine-tune existing\nparameters by back-propagation. Experimental results show that our approach\nachieves significantly better performance than fine-tuning alone. Compared with\nexpanding hidden states, our approach is more robust for old domains, shown by\nboth empirical and theoretical results. Our model also outperforms previous\nwork of IDA including elastic weight consolidation and progressive neural\nnetworks in the experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00403v2",
            "date": "2018-11-01 14:17:01+00:00",
            "title": "Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models",
            "authors": [
                "Herman Kamper"
            ],
            "abstract": "We investigate unsupervised models that can map a variable-duration speech\nsegment to a fixed-dimensional representation. In settings where unlabelled\nspeech is the only available resource, such acoustic word embeddings can form\nthe basis for \"zero-resource\" speech search, discovery and indexing systems.\nMost existing unsupervised embedding methods still use some supervision, such\nas word or phoneme boundaries. Here we propose the encoder-decoder\ncorrespondence autoencoder (EncDec-CAE), which, instead of true word segments,\nuses automatically discovered segments: an unsupervised term discovery system\nfinds pairs of words of the same unknown type, and the EncDec-CAE is trained to\nreconstruct one word given the other as input. We compare it to a standard\nencoder-decoder autoencoder (AE), a variational AE with a prior over its latent\nembedding, and downsampling. EncDec-CAE outperforms its closest competitor by\n24% relative in average precision on two languages in a word discrimination\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00513v2",
            "date": "2018-11-01 17:32:44+00:00",
            "title": "Auditing Data Provenance in Text-Generation Models",
            "authors": [
                "Congzheng Song",
                "Vitaly Shmatikov"
            ],
            "abstract": "To help enforce data-protection regulations such as GDPR and detect\nunauthorized uses of personal data, we develop a new \\emph{model auditing}\ntechnique that helps users check if their data was used to train a machine\nlearning model. We focus on auditing deep-learning models that generate\nnatural-language text, including word prediction and dialog generation. These\nmodels are at the core of popular online services and are often trained on\npersonal data such as users' messages, searches, chats, and comments.\n  We design and evaluate a black-box auditing method that can detect, with very\nfew queries to a model, if a particular user's texts were used to train it\n(among thousands of other users). We empirically show that our method can\nsuccessfully audit well-generalized models that are not overfitted to the\ntraining data. We also analyze how text-generation models memorize word\nsequences and explain why this memorization makes them amenable to auditing.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00641v1",
            "date": "2018-11-01 21:38:18+00:00",
            "title": "Online Embedding Compression for Text Classification using Low Rank Matrix Factorization",
            "authors": [
                "Anish Acharya",
                "Rahul Goel",
                "Angeliki Metallinou",
                "Inderjit Dhillon"
            ],
            "abstract": "Deep learning models have become state of the art for natural language\nprocessing (NLP) tasks, however deploying these models in production system\nposes significant memory constraints. Existing compression methods are either\nlossy or introduce significant latency. We propose a compression method that\nleverages low rank matrix factorization during training,to compress the word\nembedding layer which represents the size bottleneck for most NLP models. Our\nmodels are trained, compressed and then further re-trained on the downstream\ntask to recover accuracy while maintaining the reduced size. Empirically, we\nshow that the proposed method can achieve 90% compression with minimal impact\nin accuracy for sentence classification tasks, and outperforms alternative\nmethods like fixed-point quantization or offline word embedding compression. We\nalso analyze the inference time and storage space for our method through FLOP\ncalculations, showing that we can compress DNN models by a configurable ratio\nand regain accuracy loss without introducing additional latency compared to\nfixed point quantization. Finally, we introduce a novel learning rate schedule,\nthe Cyclically Annealed Learning Rate (CALR), which we empirically demonstrate\nto outperform other popular adaptive learning rate algorithms on a sentence\nclassification benchmark.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.00659v1",
            "date": "2018-11-01 22:24:25+00:00",
            "title": "Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications",
            "authors": [
                "Deren Lei",
                "Zichen Sun",
                "Yijun Xiao",
                "William Yang Wang"
            ],
            "abstract": "Deep neural networks with remarkably strong generalization performances are\nusually over-parameterized. Despite explicit regularization strategies are used\nfor practitioners to avoid over-fitting, the impacts are often small. Some\ntheoretical studies have analyzed the implicit regularization effect of\nstochastic gradient descent (SGD) on simple machine learning models with\ncertain assumptions. However, how it behaves practically in state-of-the-art\nmodels and real-world datasets is still unknown. To bridge this gap, we study\nthe role of SGD implicit regularization in deep learning systems. We show pure\nSGD tends to converge to minimas that have better generalization performances\nin multiple natural language processing (NLP) tasks. This phenomenon coexists\nwith dropout, an explicit regularizer. In addition, neural network's finite\nlearning capability does not impact the intrinsic nature of SGD's implicit\nregularization effect. Specifically, under limited training samples or with\ncertain corrupted labels, the implicit regularization effect remains strong. We\nfurther analyze the stability by varying the weight initialization range. We\ncorroborate these experimental findings with a decision boundary visualization\nusing a 3-layer neural network for interpretation. Altogether, our work enables\na deepened understanding on how implicit regularization affects the deep\nlearning model and sheds light on the future study of the over-parameterized\nmodel's generalization ability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00707v1",
            "date": "2018-11-02 02:15:46+00:00",
            "title": "Training Neural Speech Recognition Systems with Synthetic Speech Augmentation",
            "authors": [
                "Jason Li",
                "Ravi Gadde",
                "Boris Ginsburg",
                "Vitaly Lavrukhin"
            ],
            "abstract": "Building an accurate automatic speech recognition (ASR) system requires a\nlarge dataset that contains many hours of labeled speech samples produced by a\ndiverse set of speakers. The lack of such open free datasets is one of the main\nissues preventing advancements in ASR research. To address this problem, we\npropose to augment a natural speech dataset with synthetic speech. We train\nvery large end-to-end neural speech recognition models using the LibriSpeech\ndataset augmented with synthetic speech. These new models achieve state of the\nart Word Error Rate (WER) for character-level based models without an external\nlanguage model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00717v1",
            "date": "2018-11-02 02:54:39+00:00",
            "title": "Dirichlet belief networks for topic structure learning",
            "authors": [
                "He Zhao",
                "Lan Du",
                "Wray Buntine",
                "Mingyuan Zhou"
            ],
            "abstract": "Recently, considerable research effort has been devoted to developing deep\narchitectures for topic models to learn topic structures. Although several deep\nmodels have been proposed to learn better topic proportions of documents, how\nto leverage the benefits of deep structures for learning word distributions of\ntopics has not yet been rigorously studied. Here we propose a new multi-layer\ngenerative process on word distributions of topics, where each layer consists\nof a set of topics and each topic is drawn from a mixture of the topics of the\nlayer above. As the topics in all layers can be directly interpreted by words,\nthe proposed model is able to discover interpretable topic hierarchies. As a\nself-contained module, our model can be flexibly adapted to different kinds of\ntopic models to improve their modelling accuracy and interpretability.\nExtensive experiments on text corpora demonstrate the advantages of the\nproposed model.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00869v1",
            "date": "2018-11-02 14:08:53+00:00",
            "title": "Comparison of Classification Algorithms Used Medical Documents Categorization",
            "authors": [
                "Durmus Ozkan Sahin",
                "Erdal Kilic"
            ],
            "abstract": "Volume of text based documents have been increasing day by day. Medical\ndocuments are located within this growing text documents. In this study, the\ntechniques used for text classification applied on medical documents and\nevaluated classification performance. Used data sets are multi class and multi\nlabelled. Chi Square (CHI) technique was used for feature selection also SMO,\nNB, C4.5, RF and KNN algorithms was used for classification. The aim of this\nstudy, success of various classifiers is evaluated on multi class and multi\nlabel data sets consisting of medical documents. The first 400 features, while\nthe most successful in the KNN classifier, feature number 400 and after the SMO\nhas become the most successful classifier.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.00937v2",
            "date": "2018-11-02 15:34:29+00:00",
            "title": "CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge",
            "authors": [
                "Alon Talmor",
                "Jonathan Herzig",
                "Nicholas Lourie",
                "Jonathan Berant"
            ],
            "abstract": "When answering a question, people often draw upon their rich world knowledge\nin addition to the particular context. Recent work has focused primarily on\nanswering questions given some relevant document or context, and required very\nlittle general background. To investigate question answering with prior\nknowledge, we present CommonsenseQA: a challenging new dataset for commonsense\nquestion answering. To capture common sense beyond associations, we extract\nfrom ConceptNet (Speer et al., 2017) multiple target concepts that have the\nsame semantic relation to a single source concept. Crowd-workers are asked to\nauthor multiple-choice questions that mention the source concept and\ndiscriminate in turn between each of the target concepts. This encourages\nworkers to create questions with complex semantics that often require prior\nknowledge. We create 12,247 questions through this procedure and demonstrate\nthe difficulty of our task with a large number of strong baselines. Our best\nbaseline is based on BERT-large (Devlin et al., 2018) and obtains 56% accuracy,\nwell below human performance, which is 89%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.00998v1",
            "date": "2018-11-02 17:31:53+00:00",
            "title": "Analysing Dropout and Compounding Errors in Neural Language Models",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "This paper carries out an empirical analysis of various dropout techniques\nfor language modelling, such as Bernoulli dropout, Gaussian dropout, Curriculum\nDropout, Variational Dropout and Concrete Dropout. Moreover, we propose an\nextension of variational dropout to concrete dropout and curriculum dropout\nwith varying schedules. We find these extensions to perform well when compared\nto standard dropout approaches, particularly variational curriculum dropout\nwith a linear schedule. Largest performance increases are made when applying\ndropout on the decoder layer. Lastly, we analyze where most of the errors occur\nat test time as a post-analysis step to determine if the well-known problem of\ncompounding errors is apparent and to what end do the proposed methods mitigate\nthis issue for each dataset. We report results on a 2-hidden layer LSTM, GRU\nand Highway network with embedding dropout, dropout on the gated hidden layers\nand the output projection layer for each model. We report our results on\nPenn-TreeBank and WikiText-2 word-level language modelling datasets, where the\nformer reduces the long-tail distribution through preprocessing and one which\npreserves rare words in the training and test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01001v1",
            "date": "2018-11-02 17:37:39+00:00",
            "title": "On Evaluating the Generalization of LSTM Models in Formal Languages",
            "authors": [
                "Mirac Suzgun",
                "Yonatan Belinkov",
                "Stuart M. Shieber"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) are theoretically Turing-complete and\nestablished themselves as a dominant model for language processing. Yet, there\nstill remains an uncertainty regarding their language learning capabilities. In\nthis paper, we empirically evaluate the inductive learning capabilities of Long\nShort-Term Memory networks, a popular extension of simple RNNs, to learn simple\nformal languages, in particular $a^nb^n$, $a^nb^nc^n$, and $a^nb^nc^nd^n$. We\ninvestigate the influence of various aspects of learning, such as training data\nregimes and model capacity, on the generalization to unobserved samples. We\nfind striking differences in model performances under different training\nsettings and highlight the need for careful analysis and assessment when making\nclaims about the learning capabilities of neural network models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01135v1",
            "date": "2018-11-03 00:29:41+00:00",
            "title": "Content preserving text generation with attribute controls",
            "authors": [
                "Lajanugen Logeswaran",
                "Honglak Lee",
                "Samy Bengio"
            ],
            "abstract": "In this work, we address the problem of modifying textual attributes of\nsentences. Given an input sentence and a set of attribute labels, we attempt to\ngenerate sentences that are compatible with the conditioning information. To\nensure that the model generates content compatible sentences, we introduce a\nreconstruction loss which interpolates between auto-encoding and\nback-translation loss components. We propose an adversarial loss to enforce\ngenerated samples to be attribute compatible and realistic. Through\nquantitative, qualitative and human evaluations we demonstrate that our model\nis capable of generating fluent sentences that better reflect the conditioning\ninformation compared to prior methods. We further demonstrate that the model is\ncapable of simultaneously controlling multiple attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01136v2",
            "date": "2018-11-03 00:34:05+00:00",
            "title": "Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings",
            "authors": [
                "Mikel Artetxe",
                "Holger Schwenk"
            ],
            "abstract": "Machine translation is highly sensitive to the size and quality of the\ntraining data, which has led to an increasing interest in collecting and\nfiltering large parallel corpora. In this paper, we propose a new method for\nthis task based on multilingual sentence embeddings. In contrast to previous\napproaches, which rely on nearest neighbor retrieval with a hard threshold over\ncosine similarity, our proposed method accounts for the scale inconsistencies\nof this measure, considering the margin between a given sentence pair and its\nclosest candidates instead. Our experiments show large improvements over\nexisting methods. We outperform the best published results on the BUCC mining\ntask and the UN reconstruction task by more than 10 F1 and 30 precision points,\nrespectively. Filtering the English-German ParaCrawl corpus with our approach,\nwe obtain 31.2 BLEU points on newstest2014, an improvement of more than one\npoint over the best official filtered version.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01237v1",
            "date": "2018-11-03 15:50:27+00:00",
            "title": "Relation Mention Extraction from Noisy Data with Hierarchical Reinforcement Learning",
            "authors": [
                "Jun Feng",
                "Minlie Huang",
                "Yijie Zhang",
                "Yang Yang",
                "Xiaoyan Zhu"
            ],
            "abstract": "In this paper we address a task of relation mention extraction from noisy\ndata: extracting representative phrases for a particular relation from noisy\nsentences that are collected via distant supervision. Despite its significance\nand value in many downstream applications, this task is less studied on noisy\ndata. The major challenges exists in 1) the lack of annotation on mention\nphrases, and more severely, 2) handling noisy sentences which do not express a\nrelation at all. To address the two challenges, we formulate the task as a\nsemi-Markov decision process and propose a novel hierarchical reinforcement\nlearning model. Our model consists of a top-level sentence selector to remove\nnoisy sentences, a low-level mention extractor to extract relation mentions,\nand a reward estimator to provide signals to guide data denoising and mention\nextraction without explicit annotations. Experimental results show that our\nmodel is effective to extract relation mentions from noisy data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01299v1",
            "date": "2018-11-03 23:43:38+00:00",
            "title": "SimplerVoice: A Key Message & Visual Description Generator System for Illiteracy",
            "authors": [
                "Minh N. B. Nguyen",
                "Samuel Thomas",
                "Anne E. Gattiker",
                "Sujatha Kashyap",
                "Kush R. Varshney"
            ],
            "abstract": "We introduce SimplerVoice: a key message and visual description generator\nsystem to help low-literate adults navigate the information-dense world with\nconfidence, on their own. SimplerVoice can automatically generate sensible\nsentences describing an unknown object, extract semantic meanings of the object\nusage in the form of a query string, then, represent the string as multiple\ntypes of visual guidance (pictures, pictographs, etc.). We demonstrate\nSimplerVoice system in a case study of generating grocery products' manuals\nthrough a mobile application. To evaluate, we conducted a user study on\nSimplerVoice's generated description in comparison to the information\ninterpreted by users from other methods: the original product package and\nsearch engines' top result, in which SimplerVoice achieved the highest\nperformance score: 4.82 on 5-point mean opinion score scale. Our result shows\nthat SimplerVoice is able to provide low-literate end-users with simple yet\ninformative components to help them understand how to use the grocery products,\nand that the system may potentially provide benefits in other real-world use\ncases",
            "journal ref": "Data For Good Exchange 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01302v1",
            "date": "2018-11-04 00:02:42+00:00",
            "title": "Adversarial Gain",
            "authors": [
                "Peter Henderson",
                "Koustuv Sinha",
                "Rosemary Nan Ke",
                "Joelle Pineau"
            ],
            "abstract": "Adversarial examples can be defined as inputs to a model which induce a\nmistake - where the model output is different than that of an oracle, perhaps\nin surprising or malicious ways. Original models of adversarial attacks are\nprimarily studied in the context of classification and computer vision tasks.\nWhile several attacks have been proposed in natural language processing (NLP)\nsettings, they often vary in defining the parameters of an attack and what a\nsuccessful attack would look like. The goal of this work is to propose a\nunifying model of adversarial examples suitable for NLP tasks in both\ngenerative and classification settings. We define the notion of adversarial\ngain: based in control theory, it is a measure of the change in the output of a\nsystem relative to the perturbation of the input (caused by the so-called\nadversary) presented to the learner. This definition, as we show, can be used\nunder different feature spaces and distance conditions to determine attack or\ndefense effectiveness across different intuitive manifolds. This notion of\nadversarial gain not only provides a useful way for evaluating adversaries and\ndefenses, but can act as a building block for future work in robustness under\nadversaries due to its rooted nature in stability and manifold theory.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.01307v1",
            "date": "2018-11-04 01:23:47+00:00",
            "title": "Towards Unsupervised Speech-to-Text Translation",
            "authors": [
                "Yu-An Chung",
                "Wei-Hung Weng",
                "Schrasing Tong",
                "James Glass"
            ],
            "abstract": "We present a framework for building speech-to-text translation (ST) systems\nusing only monolingual speech and text corpora, in other words, speech\nutterances from a source language and independent text from a target language.\nAs opposed to traditional cascaded systems and end-to-end architectures, our\nsystem does not require any labeled data (i.e., transcribed source audio or\nparallel source and target text corpora) during training, making it especially\napplicable to language pairs with very few or even zero bilingual resources.\nThe framework initializes the ST system with a cross-modal bilingual dictionary\ninferred from the monolingual corpora, that maps every source speech segment\ncorresponding to a spoken word to its target text translation. For unseen\nsource speech utterances, the system first performs word-by-word translation on\neach speech segment in the utterance. The translation is improved by leveraging\na language model and a sequence denoising autoencoder to provide prior\nknowledge about the target language. Experimental results show that our\nunsupervised system achieves comparable BLEU scores to supervised end-to-end\nmodels despite the lack of supervision. We also provide an ablation analysis to\nexamine the utility of each component in our system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01382v1",
            "date": "2018-11-04 14:45:50+00:00",
            "title": "Neural CRF transducers for sequence labeling",
            "authors": [
                "Kai Hu",
                "Zhijian Ou",
                "Min Hu",
                "Junlan Feng"
            ],
            "abstract": "Conditional random fields (CRFs) have been shown to be one of the most\nsuccessful approaches to sequence labeling. Various linear-chain neural CRFs\n(NCRFs) are developed to implement the non-linear node potentials in CRFs, but\nstill keeping the linear-chain hidden structure. In this paper, we propose NCRF\ntransducers, which consists of two RNNs, one extracting features from\nobservations and the other capturing (theoretically infinite) long-range\ndependencies between labels. Different sequence labeling methods are evaluated\nover POS tagging, chunking and NER (English, Dutch). Experiment results show\nthat NCRF transducers achieve consistent improvements over linear-chain NCRFs\nand RNN transducers across all the four tasks, and can improve state-of-the-art\nresults.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.01690v2",
            "date": "2018-11-02 06:32:58+00:00",
            "title": "Cycle-consistency training for end-to-end speech recognition",
            "authors": [
                "Takaaki Hori",
                "Ramon Astudillo",
                "Tomoki Hayashi",
                "Yu Zhang",
                "Shinji Watanabe",
                "Jonathan Le Roux"
            ],
            "abstract": "This paper presents a method to train end-to-end automatic speech recognition\n(ASR) models using unpaired data. Although the end-to-end approach can\neliminate the need for expert knowledge such as pronunciation dictionaries to\nbuild ASR systems, it still requires a large amount of paired data, i.e.,\nspeech utterances and their transcriptions. Cycle-consistency losses have been\nrecently proposed as a way to mitigate the problem of limited paired data.\nThese approaches compose a reverse operation with a given transformation, e.g.,\ntext-to-speech (TTS) with ASR, to build a loss that only requires unsupervised\ndata, speech in this example. Applying cycle consistency to ASR models is not\ntrivial since fundamental information, such as speaker traits, are lost in the\nintermediate text bottleneck. To solve this problem, this work presents a loss\nthat is based on the speech encoder state sequence instead of the raw speech\nsignal. This is achieved by training a Text-To-Encoder model and defining a\nloss based on the encoder reconstruction error. Experimental results on the\nLibriSpeech corpus show that the proposed cycle-consistency training reduced\nthe word error rate by 14.7% from an initial model trained with 100-hour paired\ndata, using an additional 360 hours of audio data without transcriptions. We\nalso investigate the use of text-only data mainly for language modeling to\nfurther improve the performance in the unpaired data training scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01710v1",
            "date": "2018-10-31 01:31:10+00:00",
            "title": "Weakly Supervised Grammatical Error Correction using Iterative Decoding",
            "authors": [
                "Jared Lichtarge",
                "Christopher Alberti",
                "Shankar Kumar",
                "Noam Shazeer",
                "Niki Parmar"
            ],
            "abstract": "We describe an approach to Grammatical Error Correction (GEC) that is\neffective at making use of models trained on large amounts of weakly supervised\nbitext. We train the Transformer sequence-to-sequence model on 4B tokens of\nWikipedia revisions and employ an iterative decoding strategy that is tailored\nto the loosely-supervised nature of the Wikipedia training corpus. Finetuning\non the Lang-8 corpus and ensembling yields an F0.5 of 58.3 on the CoNLL'14\nbenchmark and a GLEU of 62.4 on JFLEG. The combination of weakly supervised\ntraining and iterative decoding obtains an F0.5 of 48.2 on CoNLL'14 even\nwithout using any labeled GEC data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01713v1",
            "date": "2018-10-30 19:43:17+00:00",
            "title": "Word Mover's Embedding: From Word2Vec to Document Embedding",
            "authors": [
                "Lingfei Wu",
                "Ian E. H. Yen",
                "Kun Xu",
                "Fangli Xu",
                "Avinash Balakrishnan",
                "Pin-Yu Chen",
                "Pradeep Ravikumar",
                "Michael J. Witbrock"
            ],
            "abstract": "While the celebrated Word2Vec technique yields semantically rich\nrepresentations for individual words, there has been relatively less success in\nextending to generate unsupervised sentences or documents embeddings. Recent\nwork has demonstrated that a distance measure between documents called\n\\emph{Word Mover's Distance} (WMD) that aligns semantically similar words,\nyields unprecedented KNN classification accuracy. However, WMD is expensive to\ncompute, and it is hard to extend its use beyond a KNN classifier. In this\npaper, we propose the \\emph{Word Mover's Embedding } (WME), a novel approach to\nbuilding an unsupervised document (sentence) embedding from pre-trained word\nembeddings. In our experiments on 9 benchmark text classification datasets and\n22 textual similarity tasks, the proposed technique consistently matches or\noutperforms state-of-the-art techniques, with significantly higher accuracy on\nproblems of short length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01747v3",
            "date": "2018-11-02 16:41:26+00:00",
            "title": "The Knowref Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution",
            "authors": [
                "Ali Emami",
                "Paul Trichelair",
                "Adam Trischler",
                "Kaheer Suleman",
                "Hannes Schulz",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "We introduce a new benchmark for coreference resolution and NLI, Knowref,\nthat targets common-sense understanding and world knowledge. Previous\ncoreference resolution tasks can largely be solved by exploiting the number and\ngender of the antecedents, or have been handcrafted and do not reflect the\ndiversity of naturally occurring text. We present a corpus of over 8,000\nannotated text passages with ambiguous pronominal anaphora. These instances are\nboth challenging and realistic. We show that various coreference systems,\nwhether rule-based, feature-rich, or neural, perform significantly worse on the\ntask than humans, who display high inter-annotator agreement. To explain this\nperformance gap, we show empirically that state-of-the art models often fail to\ncapture context, instead relying on the gender or number of candidate\nantecedents to make a decision. We then use problem-specific insights to\npropose a data-augmentation trick called antecedent switching to alleviate this\ntendency in models. Finally, we show that antecedent switching yields promising\nresults on other tasks as well: we use it to achieve state-of-the-art results\non the GAP coreference task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01778v2",
            "date": "2018-11-05 15:11:24+00:00",
            "title": "How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG",
            "authors": [
                "Paul Trichelair",
                "Ali Emami",
                "Adam Trischler",
                "Kaheer Suleman",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Recent studies have significantly improved the state-of-the-art on\ncommon-sense reasoning (CSR) benchmarks like the Winograd Schema Challenge\n(WSC) and SWAG. The question we ask in this paper is whether improved\nperformance on these benchmarks represents genuine progress towards\ncommon-sense-enabled systems. We make case studies of both benchmarks and\ndesign protocols that clarify and qualify the results of previous work by\nanalyzing threats to the validity of previous experimental designs. Our\nprotocols account for several properties prevalent in common-sense benchmarks\nincluding size limitations, structural regularities, and variable instance\ndifficulty.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.01824v4",
            "date": "2018-11-05 16:12:04+00:00",
            "title": "Structured Neural Summarization",
            "authors": [
                "Patrick Fernandes",
                "Miltiadis Allamanis",
                "Marc Brockschmidt"
            ],
            "abstract": "Summarization of long sequences into a concise statement is a core problem in\nnatural language processing, requiring non-trivial understanding of the input.\nBased on the promising results of graph neural networks on highly structured\ndata, we develop a framework to extend existing sequence encoders with a graph\ncomponent that can reason about long-distance relationships in weakly\nstructured data such as text. In an extensive evaluation, we show that the\nresulting hybrid sequence-graph models outperform both pure sequence models as\nwell as pure graph models on a range of summarization tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.01833v1",
            "date": "2018-10-29 00:28:26+00:00",
            "title": "Accelerating System Log Processing by Semi-supervised Learning: A Technical Report",
            "authors": [
                "Guofu Li",
                "Pengjia Zhu",
                "Zhiyi Chen"
            ],
            "abstract": "There is an increasing need for more automated system-log analysis tools for\nlarge scale online system in a timely manner. However, conventional way to\nmonitor and classify the log output based on keyword list does not scale well\nfor complex system in which codes contributed by a large group of developers,\nwith diverse ways of encoding the error messages, often with misleading pre-set\nlabels. In this paper, we propose that the design of a large scale online log\nanalysis should follow the \"Least Prior Knowledge Principle\", in which\nunsupervised or semi-supervised solution with the minimal prior knowledge of\nthe log should be encoded directly. Thereby, we report our experience in\ndesigning a two-stage machine learning based method, in which the system logs\nare regarded as the output of a quasi-natural language, pre-filtered by a\nperplexity score threshold, and then undergo a fine-grained classification\nprocedure. Tests on empirical data show that our method has obvious advantage\nregarding to the processing speed and classification accuracy.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1811.01910v2",
            "date": "2018-11-05 18:39:54+00:00",
            "title": "Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks",
            "authors": [
                "Edward Collins",
                "Nikolai Rozanov",
                "Bingbing Zhang"
            ],
            "abstract": "Classification tasks are usually analysed and improved through new model\narchitectures or hyperparameter optimisation but the underlying properties of\ndatasets are discovered on an ad-hoc basis as errors occur. However,\nunderstanding the properties of the data is crucial in perfecting models. In\nthis paper we analyse exactly which characteristics of a dataset best determine\nhow difficult that dataset is for the task of text classification. We then\npropose an intuitive measure of difficulty for text classification datasets\nwhich is simple and fast to calculate. We show that this measure generalises to\nunseen data by comparing it to state-of-the-art datasets and results. This\nmeasure can be used to analyse the precise source of errors in a dataset and\nallows fast estimation of how difficult a dataset is to learn. We searched for\nthis measure by training 12 classical and neural network based models on 78\nreal-world datasets, then use a genetic algorithm to discover the best measure\nof difficulty. Our difficulty-calculating code ( https://github.com/Wluper/edm\n) and datasets ( http://data.wluper.com ) are publicly available.",
            "journal ref": "ACL, CoNLL(K18-1037), 22, 380--391, (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.01931v1",
            "date": "2018-11-03 21:27:56+00:00",
            "title": "DAPPER: Scaling Dynamic Author Persona Topic Model to Billion Word Corpora",
            "authors": [
                "Robert Giaquinto",
                "Arindam Banerjee"
            ],
            "abstract": "Extracting common narratives from multi-author dynamic text corpora requires\ncomplex models, such as the Dynamic Author Persona (DAP) topic model. However,\nsuch models are complex and can struggle to scale to large corpora, often\nbecause of challenging non-conjugate terms. To overcome such challenges, in\nthis paper we adapt new ideas in approximate inference to the DAP model,\nresulting in the DAP Performed Exceedingly Rapidly (DAPPER) topic model.\nSpecifically, we develop Conjugate-Computation Variational Inference (CVI)\nbased variational Expectation-Maximization (EM) for learning the model,\nyielding fast, closed form updates for each document, replacing iterative\noptimization in earlier work. Our results show significant improvements in\nmodel fit and training time without needing to compromise the model's temporal\nstructure or the application of Regularized Variation Inference (RVI). We\ndemonstrate the scalability and effectiveness of the DAPPER model by extracting\nhealth journeys from the CaringBridge corpus --- a collection of 9 million\njournals written by 200,000 authors during health crises.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1811.02050v2",
            "date": "2018-11-05 21:57:09+00:00",
            "title": "Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation",
            "authors": [
                "Ye Jia",
                "Melvin Johnson",
                "Wolfgang Macherey",
                "Ron J. Weiss",
                "Yuan Cao",
                "Chung-Cheng Chiu",
                "Naveen Ari",
                "Stella Laurenzo",
                "Yonghui Wu"
            ],
            "abstract": "End-to-end Speech Translation (ST) models have many potential advantages when\ncompared to the cascade of Automatic Speech Recognition (ASR) and text Machine\nTranslation (MT) models, including lowered inference latency and the avoidance\nof error compounding. However, the quality of end-to-end ST is often limited by\na paucity of training data, since it is difficult to collect large parallel\ncorpora of speech and translated transcript pairs. Previous studies have\nproposed the use of pre-trained components and multi-task learning in order to\nbenefit from weakly supervised training data, such as speech-to-transcript or\ntext-to-foreign-text pairs. In this paper, we demonstrate that using\npre-trained MT or text-to-speech (TTS) synthesis models to convert weakly\nsupervised data into speech-to-translation pairs for ST training can be more\neffective than multi-task learning. Furthermore, we demonstrate that a high\nquality end-to-end ST model can be trained using only weakly supervised\ndatasets, and that synthetic data sourced from unlabeled monolingual text or\nspeech can be used to improve performance. Finally, we discuss methods for\navoiding overfitting to synthetic speech with a quantitative ablation study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02062v1",
            "date": "2018-11-05 22:21:51+00:00",
            "title": "End-to-End Monaural Multi-speaker ASR System without Pretraining",
            "authors": [
                "Xuankai Chang",
                "Yanmin Qian",
                "Kai Yu",
                "Shinji Watanabe"
            ],
            "abstract": "Recently, end-to-end models have become a popular approach as an alternative\nto traditional hybrid models in automatic speech recognition (ASR). The\nmulti-speaker speech separation and recognition task is a central task in\ncocktail party problem. In this paper, we present a state-of-the-art monaural\nmulti-speaker end-to-end automatic speech recognition model. In contrast to\nprevious studies on the monaural multi-speaker speech recognition, this\nend-to-end framework is trained to recognize multiple label sequences\ncompletely from scratch. The system only requires the speech mixture and\ncorresponding label sequences, without needing any indeterminate supervisions\nobtained from non-mixture speech or corresponding labels/alignments. Moreover,\nwe exploited using the individual attention module for each separated speaker\nand the scheduled sampling to further improve the performance. Finally, we\nevaluate the proposed model on the 2-speaker mixed speech generated from the\nWSJ corpus and the wsj0-2mix dataset, which is a speech separation and\nrecognition benchmark. The experiments demonstrate that the proposed methods\ncan improve the performance of the end-to-end model in separating the\noverlapping speech and recognizing the separated streams. From the results, the\nproposed model leads to ~10.0% relative performance gains in terms of CER and\nWER respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02066v1",
            "date": "2018-11-05 22:31:00+00:00",
            "title": "How to Improve Your Speaker Embeddings Extractor in Generic Toolkits",
            "authors": [
                "Hossein Zeinali",
                "Lukas Burget",
                "Johan Rohdin",
                "Themos Stafylakis",
                "Jan Cernocky"
            ],
            "abstract": "Recently, speaker embeddings extracted with deep neural networks became the\nstate-of-the-art method for speaker verification. In this paper we aim to\nfacilitate its implementation on a more generic toolkit than Kaldi, which we\nanticipate to enable further improvements on the method. We examine several\ntricks in training, such as the effects of normalizing input features and\npooled statistics, different methods for preventing overfitting as well as\nalternative non-linearities that can be used instead of Rectifier Linear Units.\nIn addition, we investigate the difference in performance between TDNN and CNN,\nand between two types of attention mechanism. Experimental results on Speaker\nin the Wild, SRE 2016 and SRE 2018 datasets demonstrate the effectiveness of\nthe proposed implementation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.02122v2",
            "date": "2018-11-06 01:54:22+00:00",
            "title": "Robust and fine-grained prosody control of end-to-end speech synthesis",
            "authors": [
                "Younggun Lee",
                "Taesu Kim"
            ],
            "abstract": "We propose prosody embeddings for emotional and expressive speech synthesis\nnetworks. The proposed methods introduce temporal structures in the embedding\nnetworks, thus enabling fine-grained control of the speaking style of the\nsynthesized speech. The temporal structures can be designed either on the\nspeech side or the text side, leading to different control resolutions in time.\nThe prosody embedding networks are plugged into end-to-end speech synthesis\nnetworks and trained without any other supervision except for the target speech\nfor synthesizing. It is demonstrated that the prosody embedding networks\nlearned to extract prosodic features. By adjusting the learned prosody\nfeatures, we could change the pitch and amplitude of the synthesized speech\nboth at the frame level and the phoneme level. We also introduce the temporal\nnormalization of prosody embeddings, which shows better robustness against\nspeaker perturbations during prosody transfer tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02172v1",
            "date": "2018-11-06 05:47:52+00:00",
            "title": "Neural Phrase-to-Phrase Machine Translation",
            "authors": [
                "Jiangtao Feng",
                "Lingpeng Kong",
                "Po-Sen Huang",
                "Chong Wang",
                "Da Huang",
                "Jiayuan Mao",
                "Kan Qiao",
                "Dengyong Zhou"
            ],
            "abstract": "In this paper, we propose Neural Phrase-to-Phrase Machine Translation\n(NP$^2$MT). Our model uses a phrase attention mechanism to discover relevant\ninput (source) segments that are used by a decoder to generate output (target)\nphrases. We also design an efficient dynamic programming algorithm to decode\nsegments that allows the model to be trained faster than the existing neural\nphrase-based machine translation method by Huang et al. (2018). Furthermore,\nour method can naturally integrate with external phrase dictionaries during\ndecoding. Empirical experiments show that our method achieves comparable\nperformance with the state-of-the art methods on benchmark datasets. However,\nwhen the training and testing data are from different distributions or domains,\nour method performs better.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02182v1",
            "date": "2018-11-06 06:23:57+00:00",
            "title": "Unpaired Speech Enhancement by Acoustic and Adversarial Supervision for Speech Recognition",
            "authors": [
                "Geonmin Kim",
                "Hwaran Lee",
                "Bo-Kyeong Kim",
                "Sang-Hoon Oh",
                "Soo-Young Lee"
            ],
            "abstract": "Many speech enhancement methods try to learn the relationship between noisy\nand clean speech, obtained using an acoustic room simulator. We point out\nseveral limitations of enhancement methods relying on clean speech targets; the\ngoal of this work is proposing an alternative learning algorithm, called\nacoustic and adversarial supervision (AAS). AAS makes the enhanced output both\nmaximizing the likelihood of transcription on the pre-trained acoustic model\nand having general characteristics of clean speech, which improve\ngeneralization on unseen noisy speeches. We employ the connectionist temporal\nclassification and the unpaired conditional boundary equilibrium generative\nadversarial network as the loss function of AAS. AAS is tested on two datasets\nincluding additive noise without and with reverberation, Librispeech + DEMAND\nand CHiME-4. By visualizing the enhanced speech with different loss\ncombinations, we demonstrate the role of each supervision. AAS achieves a lower\nword error rate than other state-of-the-art methods using the clean speech\ntarget in both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02480v3",
            "date": "2018-11-06 16:35:01+00:00",
            "title": "Face Landmark-based Speaker-Independent Audio-Visual Speech Enhancement in Multi-Talker Environments",
            "authors": [
                "Giovanni Morrone",
                "Luca Pasa",
                "Vadim Tikhanoff",
                "Sonia Bergamaschi",
                "Luciano Fadiga",
                "Leonardo Badino"
            ],
            "abstract": "In this paper, we address the problem of enhancing the speech of a speaker of\ninterest in a cocktail party scenario when visual information of the speaker of\ninterest is available. Contrary to most previous studies, we do not learn\nvisual features on the typically small audio-visual datasets, but use an\nalready available face landmark detector (trained on a separate image dataset).\nThe landmarks are used by LSTM-based models to generate time-frequency masks\nwhich are applied to the acoustic mixed-speech spectrogram. Results show that:\n(i) landmark motion features are very effective features for this task, (ii)\nsimilarly to previous work, reconstruction of the target speaker's spectrogram\nmediated by masking is significantly more accurate than direct spectrogram\nreconstruction, and (iii) the best masks depend on both motion landmark\nfeatures and the input mixed-speech spectrogram. To the best of our knowledge,\nour proposed models are the first models trained and evaluated on the limited\nsize GRID and TCD-TIMIT datasets, that achieve speaker-independent speech\nenhancement in a multi-talker setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02701v1",
            "date": "2018-11-06 23:25:45+00:00",
            "title": "Proceedings of the 2018 Workshop on Compositional Approaches in Physics, NLP, and Social Sciences",
            "authors": [
                "Martha Lewis",
                "Bob Coecke",
                "Jules Hedges",
                "Dimitri Kartsaklis",
                "Dan Marsden"
            ],
            "abstract": "The ability to compose parts to form a more complex whole, and to analyze a\nwhole as a combination of elements, is desirable across disciplines. This\nworkshop bring together researchers applying compositional approaches to\nphysics, NLP, cognitive science, and game theory. Within NLP, a long-standing\naim is to represent how words can combine to form phrases and sentences. Within\nthe framework of distributional semantics, words are represented as vectors in\nvector spaces. The categorical model of Coecke et al. [2010], inspired by\nquantum protocols, has provided a convincing account of compositionality in\nvector space models of NLP. There is furthermore a history of vector space\nmodels in cognitive science. Theories of categorization such as those developed\nby Nosofsky [1986] and Smith et al. [1988] utilise notions of distance between\nfeature vectors. More recently G\\\"ardenfors [2004, 2014] has developed a model\nof concepts in which conceptual spaces provide geometric structures, and\ninformation is represented by points, vectors and regions in vector spaces. The\nsame compositional approach has been applied to this formalism, giving\nconceptual spaces theory a richer model of compositionality than previously\n[Bolt et al., 2018]. Compositional approaches have also been applied in the\nstudy of strategic games and Nash equilibria. In contrast to classical game\ntheory, where games are studied monolithically as one global object,\ncompositional game theory works bottom-up by building large and complex games\nfrom smaller components. Such an approach is inherently difficult since the\ninteraction between games has to be considered. Research into categorical\ncompositional methods for this field have recently begun [Ghani et al., 2018].\nMoreover, the interaction between the three disciplines of cognitive science,\nlinguistics and game theory is a fertile ground for research. Game theory in\ncognitive science is a well-established area [Camerer, 2011]. Similarly game\ntheoretic approaches have been applied in linguistics [J\\\"ager, 2008]. Lastly,\nthe study of linguistics and cognitive science is intimately intertwined\n[Smolensky and Legendre, 2006, Jackendoff, 2007]. Physics supplies\ncompositional approaches via vector spaces and categorical quantum theory,\nallowing the interplay between the three disciplines to be examined.",
            "journal ref": "EPTCS 283, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02735v3",
            "date": "2018-11-07 02:38:13+00:00",
            "title": "CNN-based MultiChannel End-to-End Speech Recognition for everyday home environments",
            "authors": [
                "Nelson Yalta",
                "Shinji Watanabe",
                "Takaaki Hori",
                "Kazuhiro Nakadai",
                "Tetsuya Ogata"
            ],
            "abstract": "Casual conversations involving multiple speakers and noises from surrounding\ndevices are common in everyday environments, which degrades the performances of\nautomatic speech recognition systems. These challenging characteristics of\nenvironments are the target of the CHiME-5 challenge. By employing a\nconvolutional neural network (CNN)-based multichannel end-to-end speech\nrecognition system, this study attempts to overcome the presents difficulties\nin everyday environments. The system comprises of an attention-based\nencoder-decoder neural network that directly generates a text as an output from\na sound input. The multichannel CNN encoder, which uses residual connections\nand batch renormalization, is trained with augmented data, including white\nnoise injection. The experimental results show that the word error rate is\nreduced by 8.5% and 0.6% absolute from a single channel end-to-end and the best\nbaseline (LF-MMI TDNN) on the CHiME-5 corpus, respectively.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.02765v2",
            "date": "2018-11-07 05:33:07+00:00",
            "title": "Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning",
            "authors": [
                "Xin Wang",
                "Jiawei Wu",
                "Da Zhang",
                "Yu Su",
                "William Yang Wang"
            ],
            "abstract": "Although promising results have been achieved in video captioning, existing\nmodels are limited to the fixed inventory of activities in the training corpus,\nand do not generalize to open vocabulary scenarios. Here we introduce a novel\ntask, zero-shot video captioning, that aims at describing out-of-domain videos\nof unseen activities. Videos of different activities usually require different\ncaptioning strategies in many aspects, i.e. word selection, semantic\nconstruction, and style expression etc, which poses a great challenge to depict\nnovel activities without paired training data. But meanwhile, similar\nactivities share some of those aspects in common. Therefore, We propose a\nprincipled Topic-Aware Mixture of Experts (TAMoE) model for zero-shot video\ncaptioning, which learns to compose different experts based on different topic\nembeddings, implicitly transferring the knowledge learned from seen activities\nto unseen ones. Besides, we leverage external topic-related text corpus to\nconstruct the topic embedding for each activity, which embodies the most\nrelevant semantic vectors within the topic. Empirical results not only validate\nthe effectiveness of our method in utilizing semantic knowledge for video\ncaptioning, but also show its strong generalization ability when describing\nnovel activities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.02770v1",
            "date": "2018-11-07 05:53:21+00:00",
            "title": "Promising Accurate Prefix Boosting for sequence-to-sequence ASR",
            "authors": [
                "Murali Karthick Baskar",
                "Luk\u00e1\u0161 Burget",
                "Shinji Watanabe",
                "Martin Karafi\u00e1t",
                "Takaaki Hori",
                "Jan Honza \u010cernock\u00fd"
            ],
            "abstract": "In this paper, we present promising accurate prefix boosting (PAPB), a\ndiscriminative training technique for attention based sequence-to-sequence\n(seq2seq) ASR. PAPB is devised to unify the training and testing scheme in an\neffective manner. The training procedure involves maximizing the score of each\npartial correct sequence obtained during beam search compared to other\nhypotheses. The training objective also includes minimization of token\n(character) error rate. PAPB shows its efficacy by achieving 10.8\\% and 3.8\\%\nWER with and without RNNLM respectively on Wall Street Journal dataset.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03115v1",
            "date": "2018-11-07 19:09:40+00:00",
            "title": "Blockwise Parallel Decoding for Deep Autoregressive Models",
            "authors": [
                "Mitchell Stern",
                "Noam Shazeer",
                "Jakob Uszkoreit"
            ],
            "abstract": "Deep autoregressive sequence-to-sequence models have demonstrated impressive\nperformance across a wide variety of tasks in recent years. While common\narchitecture classes such as recurrent, convolutional, and self-attention\nnetworks make different trade-offs between the amount of computation needed per\nlayer and the length of the critical path at training time, generation still\nremains an inherently sequential process. To overcome this limitation, we\npropose a novel blockwise parallel decoding scheme in which we make predictions\nfor multiple time steps in parallel then back off to the longest prefix\nvalidated by a scoring model. This allows for substantial theoretical\nimprovements in generation speed when applied to architectures that can process\noutput sequences in parallel. We verify our approach empirically through a\nseries of experiments using state-of-the-art self-attention models for machine\ntranslation and image super-resolution, achieving iteration reductions of up to\n2x over a baseline greedy decoder with no loss in quality, or up to 7x in\nexchange for a slight decrease in performance. In terms of wall-clock time, our\nfastest models exhibit real-time speedups of up to 4x over standard greedy\ndecoding.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.03255v1",
            "date": "2018-11-08 03:54:55+00:00",
            "title": "Phonetic-attention scoring for deep speaker features in speaker verification",
            "authors": [
                "Lantian Li",
                "Zhiyuan Tang",
                "Ying Shi",
                "Dong Wang"
            ],
            "abstract": "Recent studies have shown that frame-level deep speaker features can be\nderived from a deep neural network with the training target set to discriminate\nspeakers by a short speech segment. By pooling the frame-level features,\nutterance-level representations, called d-vectors, can be derived and used in\nthe automatic speaker verification (ASV) task. This simple average pooling,\nhowever, is inherently sensitive to the phonetic content of the utterance. An\ninteresting idea borrowed from machine translation is the attention-based\nmechanism, where the contribution of an input word to the translation at a\nparticular time is weighted by an attention score. This score reflects the\nrelevance of the input word and the present translation. We can use the same\nidea to align utterances with different phonetic contents. This paper proposes\na phonetic-attention scoring approach for d-vector systems. By this approach,\nan attention score is computed for each frame pair. This score reflects the\nsimilarity of the two frames in phonetic content, and is used to weigh the\ncontribution of this frame pair in the utterance-based scoring. This new\nscoring approach emphasizes the frame pairs with similar phonetic contents,\nwhich essentially provides a soft alignment for utterances with any phonetic\ncontents. Experimental results show that compared with the naive average\npooling, this phonetic-attention scoring approach can deliver consistent\nperformance improvement in ASV tasks of both text-dependent and\ntext-independent.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03258v2",
            "date": "2018-11-08 04:03:09+00:00",
            "title": "Gaussian-Constrained training for speaker verification",
            "authors": [
                "Lantian Li",
                "Zhiyuan Tang",
                "Ying Shi",
                "Dong Wang"
            ],
            "abstract": "Neural models, in particular the d-vector and x-vector architectures, have\nproduced state-of-the-art performance on many speaker verification tasks.\nHowever, two potential problems of these neural models deserve more\ninvestigation. Firstly, both models suffer from `information leak', which means\nthat some parameters participating in model training will be discarded during\ninference, i.e, the layers that are used as the classifier. Secondly, these\nmodels do not regulate the distribution of the derived speaker vectors. This\n`unconstrained distribution' may degrade the performance of the subsequent\nscoring component, e.g., PLDA. This paper proposes a Gaussian-constrained\ntraining approach that (1) discards the parametric classifier, and (2) enforces\nthe distribution of the derived speaker vectors to be Gaussian. Our experiments\non the VoxCeleb and SITW databases demonstrated that this new training approach\nproduced more representative and regular speaker embeddings, leading to\nconsistent performance improvement.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03276v1",
            "date": "2018-11-08 05:12:46+00:00",
            "title": "Classical Copying versus Quantum Entanglement in Natural Language: The Case of VP-ellipsis",
            "authors": [
                "Gijs Wijnholds",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "This paper compares classical copying and quantum entanglement in natural\nlanguage by considering the case of verb phrase (VP) ellipsis. VP ellipsis is a\nnon-linear linguistic phenomenon that requires the reuse of resources, making\nit the ideal test case for a comparative study of different copying behaviours\nin compositional models of natural language. Following the line of research in\ncompositional distributional semantics set out by (Coecke et al., 2010) we\ndevelop an extension of the Lambek calculus which admits a controlled form of\ncontraction to deal with the copying of linguistic resources. We then develop\ntwo different compositional models of distributional meaning for this calculus.\nIn the first model, we follow the categorical approach of (Coecke et al., 2013)\nin which a functorial passage sends the proofs of the grammar to linear maps on\nvector spaces and we use Frobenius algebras to allow for copying. In the second\ncase, we follow the more traditional approach that one finds in categorial\ngrammars, whereby an intermediate step interprets proofs as non-linear lambda\nterms, using multiple variable occurrences that model classical copying. As a\ncase study, we apply the models to derive different readings of ambiguous\nelliptical phrases and compare the analyses that each model provides.",
            "journal ref": "EPTCS 283, 2018, pp. 103-119",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.03277v1",
            "date": "2018-11-08 05:14:19+00:00",
            "title": "Towards Compositional Distributional Discourse Analysis",
            "authors": [
                "Bob Coecke",
                "Giovanni de Felice",
                "Dan Marsden",
                "Alexis Toumi"
            ],
            "abstract": "Categorical compositional distributional semantics provide a method to derive\nthe meaning of a sentence from the meaning of its individual words: the\ngrammatical reduction of a sentence automatically induces a linear map for\ncomposing the word vectors obtained from distributional semantics. In this\npaper, we extend this passage from word-to-sentence to sentence-to-discourse\ncomposition. To achieve this we introduce a notion of basic anaphoric\ndiscourses as a mid-level representation between natural language discourse\nformalised in terms of basic discourse representation structures (DRS); and\nknowledge base queries over the Semantic Web as described by basic graph\npatterns in the Resource Description Framework (RDF). This provides a\nhigh-level specification for compositional algorithms for question answering\nand anaphora resolution, and allows us to give a picture of natural language\nunderstanding as a process involving both statistical and logical resources.",
            "journal ref": "EPTCS 283, 2018, pp. 1-12",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.03423v1",
            "date": "2018-10-31 15:41:22+00:00",
            "title": "dAIrector: Automatic Story Beat Generation through Knowledge Synthesis",
            "authors": [
                "Markus Eger",
                "Kory W. Mathewson"
            ],
            "abstract": "dAIrector is an automated director which collaborates with humans\nstorytellers for live improvisational performances and writing assistance.\ndAIrector can be used to create short narrative arcs through contextual plot\ngeneration. In this work, we present the system architecture, a quantitative\nevaluation of design choices, and a case-study usage of the system which\nprovides qualitative feedback from a professional improvisational performer. We\npresent relevant metrics for the understudied domain of human-machine creative\ngeneration, specifically long-form narrative creation. We include, alongside\npublication, open-source code so that others may test, evaluate, and run the\ndAIrector.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1811.03451v1",
            "date": "2018-11-07 09:59:24+00:00",
            "title": "Analysis of Multilingual Sequence-to-Sequence speech recognition systems",
            "authors": [
                "Martin Karafi\u00e1t",
                "Murali Karthick Baskar",
                "Shinji Watanabe",
                "Takaaki Hori",
                "Matthew Wiesner",
                "Jan \"Honza'' \u010cernock\u00fd"
            ],
            "abstract": "This paper investigates the applications of various multilingual approaches\ndeveloped in conventional hidden Markov model (HMM) systems to\nsequence-to-sequence (seq2seq) automatic speech recognition (ASR). On a set\ncomposed of Babel data, we first show the effectiveness of multi-lingual\ntraining with stacked bottle-neck (SBN) features. Then we explore various\narchitectures and training strategies of multi-lingual seq2seq models based on\nCTC-attention networks including combinations of output layer, CTC and/or\nattention component re-training. We also investigate the effectiveness of\nlanguage-transfer learning in a very low resource scenario when the target\nlanguage is not included in the original multi-lingual training data.\nInterestingly, we found multilingual features superior to multilingual models,\nand this finding suggests that we can efficiently combine the benefits of the\nHMM system with the seq2seq system through these multilingual feature\ntechniques.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03790v1",
            "date": "2018-11-09 06:15:08+00:00",
            "title": "Can We Use Speaker Recognition Technology to Attack Itself? Enhancing Mimicry Attacks Using Automatic Target Speaker Selection",
            "authors": [
                "Tomi Kinnunen",
                "Rosa Gonz\u00e1lez Hautam\u00e4ki",
                "Ville Vestman",
                "Md Sahidullah"
            ],
            "abstract": "We consider technology-assisted mimicry attacks in the context of automatic\nspeaker verification (ASV). We use ASV itself to select targeted speakers to be\nattacked by human-based mimicry. We recorded 6 naive mimics for whom we select\ntarget celebrities from VoxCeleb1 and VoxCeleb2 corpora (7,365 potential\ntargets) using an i-vector system. The attacker attempts to mimic the selected\ntarget, with the utterances subjected to ASV tests using an independently\ndeveloped x-vector system. Our main finding is negative: even if some of the\nattacker scores against the target speakers were slightly increased, our mimics\ndid not succeed in spoofing the x-vector system. Interestingly, however, the\nrelative ordering of the selected targets (closest, furthest, median) are\nconsistent between the systems, which suggests some level of transferability\nbetween the systems.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03875v2",
            "date": "2018-11-09 12:14:20+00:00",
            "title": "Multimodal One-Shot Learning of Speech and Images",
            "authors": [
                "Ryan Eloff",
                "Herman A. Engelbrecht",
                "Herman Kamper"
            ],
            "abstract": "Imagine a robot is shown new concepts visually together with spoken tags,\ne.g. \"milk\", \"eggs\", \"butter\". After seeing one paired audio-visual example per\nclass, it is shown a new set of unseen instances of these objects, and asked to\npick the \"milk\". Without receiving any hard labels, could it learn to match the\nnew continuous speech input to the correct visual instance? Although unimodal\none-shot learning has been studied, where one labelled example in a single\nmodality is given per class, this example motivates multimodal one-shot\nlearning. Our main contribution is to formally define this task, and to propose\nseveral baseline and advanced models. We use a dataset of paired spoken and\nvisual digits to specifically investigate recent advances in Siamese\nconvolutional neural networks. Our best Siamese model achieves twice the\naccuracy of a nearest neighbour model using pixel-distance over images and\ndynamic time warping over speech in 11-way cross-modal matching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04210v2",
            "date": "2018-11-10 07:54:13+00:00",
            "title": "Densely Connected Attention Propagation for Reading Comprehension",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui",
                "Jian Su"
            ],
            "abstract": "We propose DecaProp (Densely Connected Attention Propagation), a new densely\nconnected neural architecture for reading comprehension (RC). There are two\ndistinct characteristics of our model. Firstly, our model densely connects all\npairwise layers of the network, modeling relationships between passage and\nquery across all hierarchical levels. Secondly, the dense connectors in our\nnetwork are learned via attention instead of standard residual skip-connectors.\nTo this end, we propose novel Bidirectional Attention Connectors (BAC) for\nefficiently forging connections throughout the network. We conduct extensive\nexperiments on four challenging RC benchmarks. Our proposed approach achieves\nstate-of-the-art results on all four, outperforming existing baselines by up to\n$2.6\\%-14.2\\%$ in absolute F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04319v3",
            "date": "2018-11-10 21:45:07+00:00",
            "title": "Playing by the Book: An Interactive Game Approach for Action Graph Extraction from Text",
            "authors": [
                "Ronen Tamari",
                "Hiroyuki Shindo",
                "Dafna Shahaf",
                "Yuji Matsumoto"
            ],
            "abstract": "Understanding procedural text requires tracking entities, actions and effects\nas the narrative unfolds. We focus on the challenging real-world problem of\naction-graph extraction from material science papers, where language is highly\nspecialized and data annotation is expensive and scarce. We propose a novel\napproach, Text2Quest, where procedural text is interpreted as instructions for\nan interactive game. A learning agent completes the game by executing the\nprocedure correctly in a text-based simulated lab environment. The framework\ncan complement existing approaches and enables richer forms of learning\ncompared to static texts. We discuss potential limitations and advantages of\nthe approach, and release a prototype proof-of-concept, hoping to encourage\nresearch in this direction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.04369v1",
            "date": "2018-11-11 08:21:55+00:00",
            "title": "User Modeling for Task Oriented Dialogues",
            "authors": [
                "Izzeddin Gur",
                "Dilek Hakkani-Tur",
                "Gokhan Tur",
                "Pararth Shah"
            ],
            "abstract": "We introduce end-to-end neural network based models for simulating users of\ntask-oriented dialogue systems. User simulation in dialogue systems is crucial\nfrom two different perspectives: (i) automatic evaluation of different dialogue\nmodels, and (ii) training task-oriented dialogue systems. We design a\nhierarchical sequence-to-sequence model that first encodes the initial user\ngoal and system turns into fixed length representations using Recurrent Neural\nNetworks (RNN). It then encodes the dialogue history using another RNN layer.\nAt each turn, user responses are decoded from the hidden representations of the\ndialogue level RNN. This hierarchical user simulator (HUS) approach allows the\nmodel to capture undiscovered parts of the user goal without the need of an\nexplicit dialogue state tracking. We further develop several variants by\nutilizing a latent variable model to inject random variations into user\nresponses to promote diversity in simulated user responses and a novel goal\nregularization mechanism to penalize divergence of user responses from the\ninitial user goal. We evaluate the proposed models on movie ticket booking\ndomain by systematically interacting each user simulator with various dialogue\nsystem policies trained with different objectives and users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04454v1",
            "date": "2018-11-11 19:02:50+00:00",
            "title": "ReDecode Framework for Iterative Improvement in Paraphrase Generation",
            "authors": [
                "Milan Aggarwal",
                "Nupur Kumari",
                "Ayush Bansal",
                "Balaji Krishnamurthy"
            ],
            "abstract": "Generating paraphrases, that is, different variations of a sentence conveying\nthe same meaning, is an important yet challenging task in NLP. Automatically\ngenerating paraphrases has its utility in many NLP tasks like question\nanswering, information retrieval, conversational systems to name a few. In this\npaper, we introduce iterative refinement of generated paraphrases within VAE\nbased generation framework. Current sequence generation models lack the\ncapability to (1) make improvements once the sentence is generated; (2) rectify\nerrors made while decoding. We propose a technique to iteratively refine the\noutput using multiple decoders, each one attending on the output sentence\ngenerated by the previous decoder. We improve current state of the art results\nsignificantly - with over 9% and 28% absolute increase in METEOR scores on\nQuora question pairs and MSCOCO datasets respectively. We also show\nqualitatively through examples that our re-decoding approach generates better\nparaphrases compared to a single decoder by rectifying errors and making\nimprovements in paraphrase structure, inducing variations and introducing new\nbut semantically coherent information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04568v1",
            "date": "2018-11-12 06:02:19+00:00",
            "title": "Vectorization of hypotheses and speech for faster beam search in encoder decoder-based speech recognition",
            "authors": [
                "Hiroshi Seki",
                "Takaaki Hori",
                "Shinji Watanabe"
            ],
            "abstract": "Attention-based encoder decoder network uses a left-to-right beam search\nalgorithm in the inference step. The current beam search expands hypotheses and\ntraverses the expanded hypotheses at the next time step. This traversal is\nimplemented using a for-loop program in general, and it leads to speed down of\nthe recognition process. In this paper, we propose a parallelism technique for\nbeam search, which accelerates the search process by vectorizing multiple\nhypotheses to eliminate the for-loop program. We also propose a technique to\nbatch multiple speech utterances for off-line recognition use, which reduces\nthe for-loop program with regard to the traverse of multiple utterances. This\nextension is not trivial during beam search unlike during training due to\nseveral pruning and thresholding techniques for efficient decoding. In\naddition, our method can combine scores of external modules, RNNLM and CTC, in\na batch as shallow fusion. We achieved 3.7 x speedup compared with the original\nbeam search algorithm by vectoring hypotheses, and achieved 10.5 x speedup by\nfurther changing processing unit to GPU.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.04860v1",
            "date": "2018-11-12 17:06:53+00:00",
            "title": "Bio-YODIE: A Named Entity Linking System for Biomedical Text",
            "authors": [
                "Genevieve Gorrell",
                "Xingyi Song",
                "Angus Roberts"
            ],
            "abstract": "Ever-expanding volumes of biomedical text require automated semantic\nannotation techniques to curate and put to best use. An established field of\nresearch seeks to link mentions in text to knowledge bases such as those\nincluded in the UMLS (Unified Medical Language System), in order to enable a\nmore sophisticated understanding. This work has yielded good results for tasks\nsuch as curating literature, but increasingly, annotation systems are more\nbroadly applied. Medical vocabularies are expanding in size, and with them the\nextent of term ambiguity. Document collections are increasing in size and\ncomplexity, creating a greater need for speed and robustness. Furthermore, as\nthe technologies are turned to new tasks, requirements change; for example\ngreater coverage of expressions may be required in order to annotate patient\nrecords, and greater accuracy may be needed for applications that affect\npatients. This places new demands on the approaches currently in use. In this\nwork, we present a new system, Bio-YODIE, and compare it to two other popular\nsystems in order to give guidance about suitable approaches in different\nscenarios and how systems might be designed to accommodate future needs.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.04890v4",
            "date": "2018-11-12 18:13:40+00:00",
            "title": "When do Words Matter? Understanding the Impact of Lexical Choice on Audience Perception using Individual Treatment Effect Estimation",
            "authors": [
                "Zhao Wang",
                "Aron Culotta"
            ],
            "abstract": "Studies across many disciplines have shown that lexical choice can affect\naudience perception. For example, how users describe themselves in a social\nmedia profile can affect their perceived socio-economic status. However, we\nlack general methods for estimating the causal effect of lexical choice on the\nperception of a specific sentence. While randomized controlled trials may\nprovide good estimates, they do not scale to the potentially millions of\ncomparisons necessary to consider all lexical choices. Instead, in this paper,\nwe first offer two classes of methods to estimate the effect on perception of\nchanging one word to another in a given sentence. The first class of algorithms\nbuilds upon quasi-experimental designs to estimate individual treatment effects\nfrom observational data. The second class treats treatment effect estimation as\na classification problem. We conduct experiments with three data sources (Yelp,\nTwitter, and Airbnb), finding that the algorithmic estimates align well with\nthose produced by randomized-control trials. Additionally, we find that it is\npossible to transfer treatment effect classifiers across domains and still\nmaintain high accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.04903v2",
            "date": "2018-11-12 18:42:33+00:00",
            "title": "Stream attention-based multi-array end-to-end speech recognition",
            "authors": [
                "Xiaofei Wang",
                "Ruizhi Li",
                "Sri Harish Mallid",
                "Takaaki Hori",
                "Shinji Watanabe",
                "Hynek Hermansky"
            ],
            "abstract": "Automatic Speech Recognition (ASR) using multiple microphone arrays has\nachieved great success in the far-field robustness. Taking advantage of all the\ninformation that each array shares and contributes is crucial in this task.\nMotivated by the advances of joint Connectionist Temporal Classification\n(CTC)/attention mechanism in the End-to-End (E2E) ASR, a stream attention-based\nmulti-array framework is proposed in this work. Microphone arrays, acting as\ninformation streams, are activated by separate encoders and decoded under the\ninstruction of both CTC and attention networks. In terms of attention, a\nhierarchical structure is adopted. On top of the regular attention networks,\nstream attention is introduced to steer the decoder toward the most informative\nencoders. Experiments have been conducted on AMI and DIRHA multi-array corpora\nusing the encoder-decoder architecture. Compared with the best single-array\nresults, the proposed framework has achieved relative Word Error Rates (WERs)\nreduction of 3.7% and 9.7% in the two datasets, respectively, which is better\nthan conventional strategies as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.04983v1",
            "date": "2018-11-12 20:02:00+00:00",
            "title": "Unseen Word Representation by Aligning Heterogeneous Lexical Semantic Spaces",
            "authors": [
                "Victor Prokhorov",
                "Mohammad Taher Pilehvar",
                "Dimitri Kartsaklis",
                "Pietro Lio",
                "Nigel Collier"
            ],
            "abstract": "Word embedding techniques heavily rely on the abundance of training data for\nindividual words. Given the Zipfian distribution of words in natural language\ntexts, a large number of words do not usually appear frequently or at all in\nthe training data. In this paper we put forward a technique that exploits the\nknowledge encoded in lexical resources, such as WordNet, to induce embeddings\nfor unseen words. Our approach adapts graph embedding and cross-lingual vector\nspace transformation techniques in order to merge lexical knowledge encoded in\nontologies with that derived from corpus statistics. We show that the approach\ncan provide consistent performance improvements across multiple evaluation\nbenchmarks: in-vitro, on multiple rare word similarity datasets, and in-vivo,\nin two downstream text classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05013v1",
            "date": "2018-11-12 21:45:41+00:00",
            "title": "Blindfold Baselines for Embodied QA",
            "authors": [
                "Ankesh Anand",
                "Eugene Belilovsky",
                "Kyle Kastner",
                "Hugo Larochelle",
                "Aaron Courville"
            ],
            "abstract": "We explore blindfold (question-only) baselines for Embodied Question\nAnswering. The EmbodiedQA task requires an agent to answer a question by\nintelligently navigating in a simulated environment, gathering necessary visual\ninformation only through first-person vision before finally answering.\nConsequently, a blindfold baseline which ignores the environment and visual\ninformation is a degenerate solution, yet we show through our experiments on\nthe EQAv1 dataset that a simple question-only baseline achieves\nstate-of-the-art results on the EmbodiedQA task in all cases except when the\nagent is spawned extremely close to the object.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.05097v2",
            "date": "2018-11-13 04:37:11+00:00",
            "title": "Exploring RNN-Transducer for Chinese Speech Recognition",
            "authors": [
                "Senmao Wang",
                "Pan Zhou",
                "Wei Chen",
                "Jia Jia",
                "Lei Xie"
            ],
            "abstract": "End-to-end approaches have drawn much attention recently for significantly\nsimplifying the construction of an automatic speech recognition (ASR) system.\nRNN transducer (RNN-T) is one of the popular end-to-end methods. Previous\nstudies have shown that RNN-T is difficult to train and a very complex training\nprocess is needed for a reasonable performance. In this paper, we explore RNN-T\nfor a Chinese large vocabulary continuous speech recognition (LVCSR) task and\naim to simplify the training process while maintaining performance. First, a\nnew strategy of learning rate decay is proposed to accelerate the model\nconvergence. Second, we find that adding convolutional layers at the beginning\nof the network and using ordered data can discard the pre-training process of\nthe encoder without loss of performance. Besides, we design experiments to find\na balance among the usage of GPU memory, training circle and model performance.\nFinally, we achieve 16.9% character error rate (CER) on our test set which is\n2% absolute improvement from a strong BLSTM CE system with language model\ntrained on the same text corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05247v2",
            "date": "2018-11-13 12:23:37+00:00",
            "title": "An Online Attention-based Model for Speech Recognition",
            "authors": [
                "Ruchao Fan",
                "Pan Zhou",
                "Wei Chen",
                "Jia Jia",
                "Gang Liu"
            ],
            "abstract": "Attention-based end-to-end models such as Listen, Attend and Spell (LAS),\nsimplify the whole pipeline of traditional automatic speech recognition (ASR)\nsystems and become popular in the field of speech recognition. In previous\nwork, researchers have shown that such architectures can acquire comparable\nresults to state-of-the-art ASR systems, especially when using a bidirectional\nencoder and global soft attention (GSA) mechanism. However, bidirectional\nencoder and GSA are two obstacles for real-time speech recognition. In this\nwork, we aim to stream LAS baseline by removing the above two obstacles. On the\nencoder side, we use a latency-controlled (LC) bidirectional structure to\nreduce the delay of forward computation. Meanwhile, an adaptive monotonic\nchunk-wise attention (AMoChA) mechanism is proposed to replace GSA for the\ncalculation of attention weight distribution. Furthermore, we propose two\nmethods to alleviate the huge performance degradation when combining LC and\nAMoChA. Finally, we successfully acquire an online LAS model, LC-AMoChA, which\nhas only 3.5% relative performance reduction to LAS baseline on our internal\nMandarin corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05250v2",
            "date": "2018-11-13 12:28:03+00:00",
            "title": "Modality Attention for End-to-End Audio-visual Speech Recognition",
            "authors": [
                "Pan Zhou",
                "Wenwen Yang",
                "Wei Chen",
                "Yanfeng Wang",
                "Jia Jia"
            ],
            "abstract": "Audio-visual speech recognition (AVSR) system is thought to be one of the\nmost promising solutions for robust speech recognition, especially in noisy\nenvironment. In this paper, we propose a novel multimodal attention based\nmethod for audio-visual speech recognition which could automatically learn the\nfused representation from both modalities based on their importance. Our method\nis realized using state-of-the-art sequence-to-sequence (Seq2seq)\narchitectures. Experimental results show that relative improvements from 2% up\nto 36% over the auditory modality alone are obtained depending on the different\nsignal-to-noise-ratio (SNR). Compared to the traditional feature concatenation\nmethods, our proposed approach can achieve better recognition performance under\nboth clean and noisy conditions. We believe modality attention based end-to-end\nmethod can be easily generalized to other multimodal tasks with correlated\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05270v1",
            "date": "2018-11-13 13:09:58+00:00",
            "title": "Predicting Distresses using Deep Learning of Text Segments in Annual Reports",
            "authors": [
                "Rastin Matin",
                "Casper Hansen",
                "Christian Hansen",
                "Pia M\u00f8lgaard"
            ],
            "abstract": "Corporate distress models typically only employ the numerical financial\nvariables in the firms' annual reports. We develop a model that employs the\nunstructured textual data in the reports as well, namely the auditors' reports\nand managements' statements. Our model consists of a convolutional recurrent\nneural network which, when concatenated with the numerical financial variables,\nlearns a descriptive representation of the text that is suited for corporate\ndistress prediction. We find that the unstructured data provides a\nstatistically significant enhancement of the distress prediction performance,\nin particular for large firms where accurate predictions are of the utmost\nimportance. Furthermore, we find that auditors' reports are more informative\nthan managements' statements and that a joint model including both managements'\nstatements and auditors' reports displays no enhancement relative to a model\nincluding only auditors' reports. Our model demonstrates a direct improvement\nover existing state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05370v1",
            "date": "2018-11-13 15:44:31+00:00",
            "title": "Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents",
            "authors": [
                "Aditya Siddhant",
                "Anuj Goyal",
                "Angeliki Metallinou"
            ],
            "abstract": "User interaction with voice-powered agents generates large amounts of\nunlabeled utterances. In this paper, we explore techniques to efficiently\ntransfer the knowledge from these unlabeled utterances to improve model\nperformance on Spoken Language Understanding (SLU) tasks. We use Embeddings\nfrom Language Model (ELMo) to take advantage of unlabeled data by learning\ncontextualized word representations. Additionally, we propose ELMo-Light\n(ELMoL), a faster and simpler unsupervised pre-training method for SLU. Our\nfindings suggest unsupervised pre-training on a large corpora of unlabeled\nutterances leads to significantly better SLU performance compared to training\nfrom scratch and it can even outperform conventional supervised transfer.\nAdditionally, we show that the gains from unsupervised transfer techniques can\nbe further improved by supervised transfer. The improvements are more\npronounced in low resource settings and when using only 1000 labeled in-domain\nsamples, our techniques match the performance of training from scratch on\n10-15x more labeled in-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05467v1",
            "date": "2018-11-13 06:49:08+00:00",
            "title": "Towards Neural Machine Translation for African Languages",
            "authors": [
                "Jade Z. Abbott",
                "Laura Martinus"
            ],
            "abstract": "Given that South African education is in crisis, strategies for improvement\nand sustainability of high-quality, up-to-date education must be explored. In\nthe migration of education online, inclusion of machine translation for\nlow-resourced local languages becomes necessary. This paper aims to spur the\nuse of current neural machine translation (NMT) techniques for low-resourced\nlocal languages. The paper demonstrates state-of-the-art performance on\nEnglish-to-Setswana translation using the Autshumato dataset. The use of the\nTransformer architecture beat previous techniques by 5.33 BLEU points. This\ndemonstrates the promise of using current NMT techniques for African languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05468v1",
            "date": "2018-11-13 13:12:02+00:00",
            "title": "Few-shot Learning for Named Entity Recognition in Medical Text",
            "authors": [
                "Maximilian Hofer",
                "Andrey Kormilitzin",
                "Paul Goldberg",
                "Alejo Nevado-Holgado"
            ],
            "abstract": "Deep neural network models have recently achieved state-of-the-art\nperformance gains in a variety of natural language processing (NLP) tasks\n(Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the\navailability of large amounts of annotated examples, without which\nstate-of-the-art performance is rarely achievable. This is especially\ninconvenient for the many NLP fields where annotated examples are scarce, such\nas medical text. To improve NLP models in this situation, we evaluate five\nimprovements on named entity recognition (NER) tasks when only ten annotated\nexamples are available: (1) layer-wise initialization with pre-trained weights,\n(2) hyperparameter tuning, (3) combining pre-training data, (4) custom word\nembeddings, and (5) optimizing out-of-vocabulary (OOV) words. Experimental\nresults show that the F1 score of 69.3% achievable by state-of-the-art models\ncan be improved to 78.87%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05475v2",
            "date": "2018-11-13 17:31:49+00:00",
            "title": "ML-Net: multi-label classification of biomedical texts with deep neural networks",
            "authors": [
                "Jingcheng Du",
                "Qingyu Chen",
                "Yifan Peng",
                "Yang Xiang",
                "Cui Tao",
                "Zhiyong Lu"
            ],
            "abstract": "In multi-label text classification, each textual document can be assigned\nwith one or more labels. Due to this nature, the multi-label text\nclassification task is often considered to be more challenging compared to the\nbinary or multi-class text classification problems. As an important task with\nbroad applications in biomedicine such as assigning diagnosis codes, a number\nof different computational methods (e.g. training and combining binary\nclassifiers for each label) have been proposed in recent years. However, many\nsuffered from modest accuracy and efficiency, with only limited success in\npractical use. We propose ML-Net, a novel deep learning framework, for\nmulti-label classification of biomedical texts. As an end-to-end system, ML-Net\ncombines a label prediction network with an automated label count prediction\nmechanism to output an optimal set of labels by leveraging both predicted\nconfidence score of each label and the contextual information in the target\ndocument. We evaluate ML-Net on three independent, publicly-available corpora\nin two kinds of text genres: biomedical literature and clinical notes. For\nevaluation, example-based measures such as precision, recall and f-measure are\nused. ML-Net is compared with several competitive machine learning baseline\nmodels. Our benchmarking results show that ML-Net compares favorably to the\nstate-of-the-art methods in multi-label classification of biomedical texts.\nML-NET is also shown to be robust when evaluated on different text genres in\nbiomedicine. Unlike traditional machine learning methods, ML-Net does not\nrequire human efforts in feature engineering and is highly efficient and\nscalable approach to tasks with a large set of labels (no need to build\nindividual classifiers for each separate label). Finally, ML-NET is able to\ndynamically estimate the label count based on the document context in a more\nsystematic and accurate manner.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.05542v2",
            "date": "2018-11-14 02:02:18+00:00",
            "title": "Extractive Summary as Discrete Latent Variables",
            "authors": [
                "Aran Komatsuzaki"
            ],
            "abstract": "In this paper, we compare various methods to compress a text using a neural\nmodel. We find that extracting tokens as latent variables significantly\noutperforms the state-of-the-art discrete latent variable models such as\nVQ-VAE. Furthermore, we compare various extractive compression schemes. There\nare two best-performing methods that perform equally. One method is to simply\nchoose the tokens with the highest tf-idf scores. Another is to train a\nbidirectional language model similar to ELMo and choose the tokens with the\nhighest loss. If we consider any subsequence of a text to be a text in a\nbroader sense, we conclude that language is a strong compression code of\nitself. Our finding justifies the high quality of generation achieved with\nhierarchical method, as their latent variables are nothing but natural language\nsummary. We also conclude that there is a hierarchy in language such that an\nentire text can be predicted much more easily based on a sequence of a small\nnumber of keywords, which can be easily found by classical methods as tf-idf.\nWe speculate that this extraction process may be useful for unsupervised\nhierarchical text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05544v1",
            "date": "2018-11-12 16:19:22+00:00",
            "title": "An Introductory Survey on Attention Mechanisms in NLP Problems",
            "authors": [
                "Dichao Hu"
            ],
            "abstract": "First derived from human intuition, later adapted to machine translation for\nautomatic token alignment, attention mechanism, a simple method that can be\nused for encoding sequence data based on the importance score each element is\nassigned, has been widely applied to and attained significant improvement in\nvarious tasks in natural language processing, including sentiment\nclassification, text summarization, question answering, dependency parsing,\netc. In this paper, we survey through recent works and conduct an introductory\nsummary of the attention mechanism in different NLP problems, aiming to provide\nour readers with basic knowledge on this widely used method, discuss its\ndifferent variants for different tasks, explore its association with other\ntechniques in machine learning, and examine methods for evaluating its\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05768v1",
            "date": "2018-11-06 16:12:00+00:00",
            "title": "Parser Extraction of Triples in Unstructured Text",
            "authors": [
                "Shaun D'Souza"
            ],
            "abstract": "The web contains vast repositories of unstructured text. We investigate the\nopportunity for building a knowledge graph from these text sources. We generate\na set of triples which can be used in knowledge gathering and integration. We\ndefine the architecture of a language compiler for processing\nsubject-predicate-object triples using the OpenNLP parser. We implement a\ndepth-first search traversal on the POS tagged syntactic tree appending\npredicate and object information. A parser enables higher precision and higher\nrecall extractions of syntactic relationships across conjunction boundaries. We\nare able to extract 2-2.5 times the correct extractions of ReVerb. The\nextractions are used in a variety of semantic web applications and question\nanswering. We verify extraction of 50,000 triples on the ClueWeb dataset.",
            "journal ref": "IAES International Journal of Artificial Intelligence (IJ-AI),\n  5(4):143-148, 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05949v1",
            "date": "2018-11-14 18:32:18+00:00",
            "title": "Jointly Learning to Label Sentences and Tokens",
            "authors": [
                "Marek Rei",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Learning to construct text representations in end-to-end systems can be\ndifficult, as natural languages are highly compositional and task-specific\nannotated datasets are often limited in size. Methods for directly supervising\nlanguage composition can allow us to guide the models based on existing\nknowledge, regularizing them towards more robust and interpretable\nrepresentations. In this paper, we investigate how objectives at different\ngranularities can be used to learn better language representations and we\npropose an architecture for jointly learning to label sentences and tokens. The\npredictions at each level are combined together using an attention mechanism,\nwith token-level labels also acting as explicit supervision for composing\nsentence-level representations. Our experiments show that by learning to\nperform these tasks jointly on multiple levels, the model achieves substantial\nimprovements for both sentence classification and sequence labeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06096v1",
            "date": "2018-11-14 22:19:03+00:00",
            "title": "Automatic Grammar Augmentation for Robust Voice Command Recognition",
            "authors": [
                "Yang Yang",
                "Anusha Lalitha",
                "Jinwon Lee",
                "Chris Lott"
            ],
            "abstract": "This paper proposes a novel pipeline for automatic grammar augmentation that\nprovides a significant improvement in the voice command recognition accuracy\nfor systems with small footprint acoustic model (AM). The improvement is\nachieved by augmenting the user-defined voice command set, also called grammar\nset, with alternate grammar expressions. For a given grammar set, a set of\npotential grammar expressions (candidate set) for augmentation is constructed\nfrom an AM-specific statistical pronunciation dictionary that captures the\nconsistent patterns and errors in the decoding of AM induced by variations in\npronunciation, pitch, tempo, accent, ambiguous spellings, and noise conditions.\nUsing this candidate set, greedy optimization based and cross-entropy-method\n(CEM) based algorithms are considered to search for an augmented grammar set\nwith improved recognition accuracy utilizing a command-specific dataset. Our\nexperiments show that the proposed pipeline along with algorithms considered in\nthis paper significantly reduce the mis-detection and mis-classification rate\nwithout increasing the false-alarm rate. Experiments also demonstrate the\nconsistent superior performance of CEM method over greedy-based algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06156v1",
            "date": "2018-11-15 03:38:20+00:00",
            "title": "Exploiting Sentence Embedding for Medical Question Answering",
            "authors": [
                "Yu Hao",
                "Xien Liu",
                "Ji Wu",
                "Ping Lv"
            ],
            "abstract": "Despite the great success of word embedding, sentence embedding remains a\nnot-well-solved problem. In this paper, we present a supervised learning\nframework to exploit sentence embedding for the medical question answering\ntask. The learning framework consists of two main parts: 1) a sentence\nembedding producing module, and 2) a scoring module. The former is developed\nwith contextual self-attention and multi-scale techniques to encode a sentence\ninto an embedding tensor. This module is shortly called Contextual\nself-Attention Multi-scale Sentence Embedding (CAMSE). The latter employs two\nscoring strategies: Semantic Matching Scoring (SMS) and Semantic Association\nScoring (SAS). SMS measures similarity while SAS captures association between\nsentence pairs: a medical question concatenated with a candidate choice, and a\npiece of corresponding supportive evidence. The proposed framework is examined\nby two Medical Question Answering(MedicalQA) datasets which are collected from\nreal-world applications: medical exam and clinical diagnosis based on\nelectronic medical records (EMR). The comparison results show that our proposed\nframework achieved significant improvements compared to competitive baseline\napproaches. Additionally, a series of controlled experiments are also conducted\nto illustrate that the multi-scale strategy and the contextual self-attention\nlayer play important roles for producing effective sentence embedding, and the\ntwo kinds of scoring strategies are highly complementary to each other for\nquestion answering problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.06439v2",
            "date": "2018-11-15 15:50:29+00:00",
            "title": "HCU400: An Annotated Dataset for Exploring Aural Phenomenology Through Causal Uncertainty",
            "authors": [
                "Ishwarya Ananthabhotla",
                "David B. Ramsay",
                "Joseph A. Paradiso"
            ],
            "abstract": "The way we perceive a sound depends on many aspects-- its ecological\nfrequency, acoustic features, typicality, and most notably, its identified\nsource. In this paper, we present the HCU400: a dataset of 402 sounds ranging\nfrom easily identifiable everyday sounds to intentionally obscured artificial\nones. It aims to lower the barrier for the study of aural phenomenology as the\nlargest available audio dataset to include an analysis of causal attribution.\nEach sample has been annotated with crowd-sourced descriptions, as well as\nfamiliarity, imageability, arousal, and valence ratings. We extend existing\ncalculations of causal uncertainty, automating and generalizing them with word\nembeddings. Upon analysis we find that individuals will provide less polarized\nemotion ratings as a sound's source becomes increasingly ambiguous; individual\nratings of familiarity and imageability, on the other hand, diverge as\nuncertainty increases despite a clear negative trend on average.",
            "journal ref": "IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP). IEEE, 2019",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.06477v1",
            "date": "2018-11-15 17:09:53+00:00",
            "title": "Multi-cell LSTM Based Neural Language Model",
            "authors": [
                "Thomas Cherian",
                "Akshay Badola",
                "Vineet Padmanabhan"
            ],
            "abstract": "Language models, being at the heart of many NLP problems, are always of great\ninterest to researchers. Neural language models come with the advantage of\ndistributed representations and long range contexts. With its particular\ndynamics that allow the cycling of information within the network, `Recurrent\nneural network' (RNN) becomes an ideal paradigm for neural language modeling.\nLong Short-Term Memory (LSTM) architecture solves the inadequacies of the\nstandard RNN in modeling long-range contexts. In spite of a plethora of RNN\nvariants, possibility to add multiple memory cells in LSTM nodes was seldom\nexplored. Here we propose a multi-cell node architecture for LSTMs and study\nits applicability for neural language modeling. The proposed multi-cell LSTM\nlanguage models outperform the state-of-the-art results on well-known Penn\nTreebank (PTB) setup.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1811.06943v1",
            "date": "2018-11-16 17:52:25+00:00",
            "title": "Automatic Paper Summary Generation from Visual and Textual Information",
            "authors": [
                "Shintaro Yamamoto",
                "Yoshihiro Fukuhara",
                "Ryota Suzuki",
                "Shigeo Morishima",
                "Hirokatsu Kataoka"
            ],
            "abstract": "Due to the recent boom in artificial intelligence (AI) research, including\ncomputer vision (CV), it has become impossible for researchers in these fields\nto keep up with the exponentially increasing number of manuscripts. In response\nto this situation, this paper proposes the paper summary generation (PSG) task\nusing a simple but effective method to automatically generate an academic paper\nsummary from raw PDF data. We realized PSG by combination of vision-based\nsupervised components detector and language-based unsupervised important\nsentence extractor, which is applicable for a trained format of manuscripts. We\nshow the quantitative evaluation of ability of simple vision-based components\nextraction, and the qualitative evaluation that our system can extract both\nvisual item and sentence that are helpful for understanding. After processing\nvia our PSG, the 979 manuscripts accepted by the Conference on Computer Vision\nand Pattern Recognition (CVPR) 2018 are available. It is believed that the\nproposed method will provide a better way for researchers to stay caught with\nimportant academic papers.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.07021v2",
            "date": "2018-11-16 20:25:23+00:00",
            "title": "Investigating the Effects of Word Substitution Errors on Sentence Embeddings",
            "authors": [
                "Rohit Voleti",
                "Julie M. Liss",
                "Visar Berisha"
            ],
            "abstract": "A key initial step in several natural language processing (NLP) tasks\ninvolves embedding phrases of text to vectors of real numbers that preserve\nsemantic meaning. To that end, several methods have been recently proposed with\nimpressive results on semantic similarity tasks. However, all of these\napproaches assume that perfect transcripts are available when generating the\nembeddings. While this is a reasonable assumption for analysis of written text,\nit is limiting for analysis of transcribed text. In this paper we investigate\nthe effects of word substitution errors, such as those coming from automatic\nspeech recognition errors (ASR), on several state-of-the-art sentence embedding\nmethods. To do this, we propose a new simulator that allows the experimenter to\ninduce ASR-plausible word substitution errors in a corpus at a desired word\nerror rate. We use this simulator to evaluate the robustness of several\nsentence embedding methods. Our results show that pre-trained neural sentence\nencoders are both robust to ASR errors and perform well on textual similarity\ntasks after errors are introduced. Meanwhile, unweighted averages of word\nvectors perform well with perfect transcriptions, but their performance\ndegrades rapidly on textual similarity tasks for text with word substitution\nerrors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07078v1",
            "date": "2018-11-17 02:29:18+00:00",
            "title": "An Affect-Rich Neural Conversational Model with Biased Attention and Weighted Cross-Entropy Loss",
            "authors": [
                "Peixiang Zhong",
                "Di Wang",
                "Chunyan Miao"
            ],
            "abstract": "Affect conveys important implicit information in human communication. Having\nthe capability to correctly express affect during human-machine conversations\nis one of the major milestones in artificial intelligence. In recent years,\nextensive research on open-domain neural conversational models has been\nconducted. However, embedding affect into such models is still under explored.\nIn this paper, we propose an end-to-end affect-rich open-domain neural\nconversational model that produces responses not only appropriate in syntax and\nsemantics, but also with rich affect. Our model extends the Seq2Seq model and\nadopts VAD (Valence, Arousal and Dominance) affective notations to embed each\nword with affects. In addition, our model considers the effect of negators and\nintensifiers via a novel affective attention mechanism, which biases attention\ntowards affect-rich words in input sentences. Lastly, we train our model with\nan affect-incorporated objective function to encourage the generation of\naffect-rich words in the output responses. Evaluations based on both perplexity\nand human evaluations show that our model outperforms the state-of-the-art\nbaseline model of comparable size in producing natural and affect-rich\nresponses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07208v1",
            "date": "2018-11-17 18:38:09+00:00",
            "title": "Emergence of linguistic conventions in multi-agent reinforcement learning",
            "authors": [
                "Dorota Lipowska",
                "Adam Lipowski"
            ],
            "abstract": "Recently, emergence of signaling conventions, among which language is a prime\nexample, draws a considerable interdisciplinary interest ranging from game\ntheory, to robotics to evolutionary linguistics. Such a wide spectrum of\nresearch is based on much different assumptions and methodologies, but\ncomplexity of the problem precludes formulation of a unifying and commonly\naccepted explanation. We examine formation of signaling conventions in a\nframework of a multi-agent reinforcement learning model. When the network of\ninteractions between agents is a complete graph or a sufficiently dense random\ngraph, a global consensus is typically reached with the emerging language being\na nearly unique object-word mapping or containing some synonyms and homonyms.\nOn finite-dimensional lattices, the model gets trapped in disordered\nconfigurations with a local consensus only. Such a trapping can be avoided by\nintroducing a population renewal, which in the presence of superlinear\nreinforcement restores an ordinary surface-tension driven coarsening and\nconsiderably enhances formation of efficient signaling.",
            "journal ref": "PLoS ONE 13(11): e0208095 (2018)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1811.07234v1",
            "date": "2018-11-17 22:21:12+00:00",
            "title": "Improving Automatic Source Code Summarization via Deep Reinforcement Learning",
            "authors": [
                "Yao Wan",
                "Zhou Zhao",
                "Min Yang",
                "Guandong Xu",
                "Haochao Ying",
                "Jian Wu",
                "Philip S. Yu"
            ],
            "abstract": "Code summarization provides a high level natural language description of the\nfunction performed by code, as it can benefit the software maintenance, code\ncategorization and retrieval. To the best of our knowledge, most\nstate-of-the-art approaches follow an encoder-decoder framework which encodes\nthe code into a hidden space and then decode it into natural language space,\nsuffering from two major drawbacks: a) Their encoders only consider the\nsequential content of code, ignoring the tree structure which is also critical\nfor the task of code summarization, b) Their decoders are typically trained to\npredict the next word by maximizing the likelihood of next ground-truth word\nwith previous ground-truth word given. However, it is expected to generate the\nentire sequence from scratch at test time. This discrepancy can cause an\n\\textit{exposure bias} issue, making the learnt decoder suboptimal. In this\npaper, we incorporate an abstract syntax tree structure as well as sequential\ncontent of code snippets into a deep reinforcement learning framework (i.e.,\nactor-critic network). The actor network provides the confidence of predicting\nthe next word according to current state. On the other hand, the critic network\nevaluates the reward value of all possible extensions of the current state and\ncan provide global guidance for explorations. We employ an advantage reward\ncomposed of BLEU metric to train both networks. Comprehensive experiments on a\nreal-world dataset show the effectiveness of our proposed model when compared\nwith some state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1811.07253v1",
            "date": "2018-11-18 01:36:05+00:00",
            "title": "Quantifying Uncertainties in Natural Language Processing Tasks",
            "authors": [
                "Yijun Xiao",
                "William Yang Wang"
            ],
            "abstract": "Reliable uncertainty quantification is a first step towards building\nexplainable, transparent, and accountable artificial intelligent systems.\nRecent progress in Bayesian deep learning has made such quantification\nrealizable. In this paper, we propose novel methods to study the benefits of\ncharacterizing model and data uncertainties for natural language processing\n(NLP) tasks. With empirical experiments on sentiment analysis, named entity\nrecognition, and language modeling using convolutional and recurrent neural\nnetwork models, we show that explicitly modeling uncertainties is not only\nnecessary to measure output confidence levels, but also useful at enhancing\nmodel performances in various NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07435v1",
            "date": "2018-11-19 00:32:50+00:00",
            "title": "Limitations of Source-Filter Coupling In Phonation",
            "authors": [
                "Debasish Ray Mohapatra",
                "Sidney Fels"
            ],
            "abstract": "The coupling of vocal fold (source) and vocal tract (filter) is one of the\nmost critical factors in source-filter articulation theory. The traditional\nlinear source-filter theory has been challenged by current research which\nclearly shows the impact of acoustic loading on the dynamic behavior of the\nvocal fold vibration as well as the variations in the glottal flow pulses\nshape. This paper outlines the underlying mechanism of source-filter\ninteractions; demonstrates the design and working principles of coupling for\nthe various existing vocal cord and vocal tract biomechanical models. For our\nstudy, we have considered self-oscillating lumped-element models of the\nacoustic source and computational models of the vocal tract as articulators. To\nunderstand the limitations of source-filter interactions which are associated\nwith each of those models, we compare them concerning their mechanical design,\nacoustic and physiological characteristics and aerodynamic simulation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.07453v2",
            "date": "2018-11-19 01:57:05+00:00",
            "title": "The PyTorch-Kaldi Speech Recognition Toolkit",
            "authors": [
                "Mirco Ravanelli",
                "Titouan Parcollet",
                "Yoshua Bengio"
            ],
            "abstract": "The availability of open-source software is playing a remarkable role in the\npopularization of speech recognition and deep learning. Kaldi, for instance, is\nnowadays an established framework used to develop state-of-the-art speech\nrecognizers. PyTorch is used to build neural networks with the Python language\nand has recently spawn tremendous interest within the machine learning\ncommunity thanks to its simplicity and flexibility.\n  The PyTorch-Kaldi project aims to bridge the gap between these popular\ntoolkits, trying to inherit the efficiency of Kaldi and the flexibility of\nPyTorch. PyTorch-Kaldi is not only a simple interface between these software,\nbut it embeds several useful features for developing modern speech recognizers.\nFor instance, the code is specifically designed to naturally plug-in\nuser-defined acoustic models. As an alternative, users can exploit several\npre-implemented neural networks that can be customized using intuitive\nconfiguration files. PyTorch-Kaldi supports multiple feature and label streams\nas well as combinations of neural networks, enabling the use of complex neural\narchitectures. The toolkit is publicly-released along with a rich documentation\nand is designed to properly work locally or on HPC clusters.\n  Experiments, that are conducted on several datasets and tasks, show that\nPyTorch-Kaldi can effectively be used to develop modern state-of-the-art speech\nrecognizers.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.07485v1",
            "date": "2018-11-19 03:51:19+00:00",
            "title": "Visual-Texual Emotion Analysis with Deep Coupled Video and Danmu Neural Networks",
            "authors": [
                "Chenchen Li",
                "Jialin Wang",
                "Hongwei Wang",
                "Miao Zhao",
                "Wenjie Li",
                "Xiaotie Deng"
            ],
            "abstract": "User emotion analysis toward videos is to automatically recognize the general\nemotional status of viewers from the multimedia content embedded in the online\nvideo stream. Existing works fall in two categories: 1) visual-based methods,\nwhich focus on visual content and extract a specific set of features of videos.\nHowever, it is generally hard to learn a mapping function from low-level video\npixels to high-level emotion space due to great intra-class variance. 2)\ntextual-based methods, which focus on the investigation of user-generated\ncomments associated with videos. The learned word representations by\ntraditional linguistic approaches typically lack emotion information and the\nglobal comments usually reflect viewers' high-level understandings rather than\ninstantaneous emotions. To address these limitations, in this paper, we propose\nto jointly utilize video content and user-generated texts simultaneously for\nemotion analysis. In particular, we introduce exploiting a new type of\nuser-generated texts, i.e., \"danmu\", which are real-time comments floating on\nthe video and contain rich information to convey viewers' emotional opinions.\nTo enhance the emotion discriminativeness of words in textual feature\nextraction, we propose Emotional Word Embedding (EWE) to learn text\nrepresentations by jointly considering their semantics and emotions.\nAfterwards, we propose a novel visual-textual emotion analysis model with Deep\nCoupled Video and Danmu Neural networks (DCVDN), in which visual and textual\nfeatures are synchronously extracted and fused to form a comprehensive\nrepresentation by deep-canonically-correlated-autoencoder-based multi-view\nlearning. Through extensive experiments on a self-crawled real-world\nvideo-danmu dataset, we prove that DCVDN significantly outperforms the\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.07550v1",
            "date": "2018-11-19 08:23:34+00:00",
            "title": "Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning",
            "authors": [
                "Yuexin Wu",
                "Xiujun Li",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Yiming Yang"
            ],
            "abstract": "Training task-completion dialogue agents with reinforcement learning usually\nrequires a large number of real user experiences. The Dyna-Q algorithm extends\nQ-learning by integrating a world model, and thus can effectively boost\ntraining efficiency using simulated experiences generated by the world model.\nThe effectiveness of Dyna-Q, however, depends on the quality of the world model\n- or implicitly, the pre-specified ratio of real vs. simulated experiences used\nfor Q-learning. To this end, we extend the recently proposed Deep Dyna-Q (DDQ)\nframework by integrating a switcher that automatically determines whether to\nuse a real or simulated experience for Q-learning. Furthermore, we explore the\nuse of active learning for improving sample efficiency, by encouraging the\nworld model to generate simulated experiences in the state-action space where\nthe agent has not (fully) explored. Our results show that by combining switcher\nand active learning, the new framework named as Switch-based Active Deep Dyna-Q\n(Switch-DDQ), leads to significant improvement over DDQ and Q-learning\nbaselines in both simulation and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.07600v2",
            "date": "2018-11-19 10:45:13+00:00",
            "title": "A Trustworthy, Responsible and Interpretable System to Handle Chit Chat in Conversational Bots",
            "authors": [
                "Parag Agrawal",
                "Anshuman Suri",
                "Tulasi Menon"
            ],
            "abstract": "Most often, chat-bots are built to solve the purpose of a search engine or a\nhuman assistant: Their primary goal is to provide information to the user or\nhelp them complete a task. However, these chat-bots are incapable of responding\nto unscripted queries like \"Hi, what's up\", \"What's your favourite food\". Human\nevaluation judgments show that 4 humans come to a consensus on the intent of a\ngiven query which is from chat domain only 77% of the time, thus making it\nevident how non-trivial this task is. In our work, we show why it is difficult\nto break the chitchat space into clearly defined intents. We propose a system\nto handle this task in chat-bots, keeping in mind scalability,\ninterpretability, appropriateness, trustworthiness, relevance and coverage. Our\nwork introduces a pipeline for query understanding in chitchat using\nhierarchical intents as well as a way to use seq-seq auto-generation models in\nprofessional bots. We explore an interpretable model for chat domain detection\nand also show how various components such as adult/offensive classification,\ngrammars/regex patterns, curated personality based responses, generic guided\nevasive responses and response generation models can be combined in a scalable\nway to solve this problem.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.07882v2",
            "date": "2018-11-19 18:58:42+00:00",
            "title": "Guiding Policies with Language via Meta-Learning",
            "authors": [
                "John D. Co-Reyes",
                "Abhishek Gupta",
                "Suvansh Sanjeev",
                "Nick Altieri",
                "Jacob Andreas",
                "John DeNero",
                "Pieter Abbeel",
                "Sergey Levine"
            ],
            "abstract": "Behavioral skills or policies for autonomous agents are conventionally\nlearned from reward functions, via reinforcement learning, or from\ndemonstrations, via imitation learning. However, both modes of task\nspecification have their disadvantages: reward functions require manual\nengineering, while demonstrations require a human expert to be able to actually\nperform the task in order to generate the demonstration. Instruction following\nfrom natural language instructions provides an appealing alternative: in the\nsame way that we can specify goals to other humans simply by speaking or\nwriting, we would like to be able to specify tasks for our machines. However, a\nsingle instruction may be insufficient to fully communicate our intent or, even\nif it is, may be insufficient for an autonomous agent to actually understand\nhow to perform the desired task. In this work, we propose an interactive\nformulation of the task specification problem, where iterative language\ncorrections are provided to an autonomous agent, guiding it in acquiring the\ndesired skill. Our proposed language-guided policy learning algorithm can\nintegrate an instruction and a sequence of corrections to acquire new skills\nvery quickly. In our experiments, we show that this method can enable a policy\nto follow instructions and corrections for simulated navigation and\nmanipulation tasks, substantially outperforming direct, non-interactive\ninstruction following.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.08008v1",
            "date": "2018-11-19 22:23:59+00:00",
            "title": "End-to-End Retrieval in Continuous Space",
            "authors": [
                "Daniel Gillick",
                "Alessandro Presta",
                "Gaurav Singh Tomar"
            ],
            "abstract": "Most text-based information retrieval (IR) systems index objects by words or\nphrases. These discrete systems have been augmented by models that use\nembeddings to measure similarity in continuous space. But continuous-space\nmodels are typically used just to re-rank the top candidates. We consider the\nproblem of end-to-end continuous retrieval, where standard approximate nearest\nneighbor (ANN) search replaces the usual discrete inverted index, and rely\nentirely on distances between learned embeddings. By training simple models\nspecifically for retrieval, with an appropriate model architecture, we improve\non a discrete baseline by 8% and 26% (MAP) on two similar-question retrieval\ntasks. We also discuss the problem of evaluation for retrieval systems, and\nshow how to modify existing pairwise similarity datasets for this purpose.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.08065v2",
            "date": "2018-11-20 04:10:23+00:00",
            "title": "Learning Robust Heterogeneous Signal Features from Parallel Neural Network for Audio Sentiment Analysis",
            "authors": [
                "Feiyang Chen",
                "Ziqian Luo"
            ],
            "abstract": "Audio Sentiment Analysis is a popular research area which extends the\nconventional text-based sentiment analysis to depend on the effectiveness of\nacoustic features extracted from speech. However, current progress on audio\nsentiment analysis mainly focuses on extracting homogeneous acoustic features\nor doesn't fuse heterogeneous features effectively. In this paper, we propose\nan utterance-based deep neural network model, which has a parallel combination\nof Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based\nnetwork, to obtain representative features termed Audio Sentiment Vector (ASV),\nthat can maximally reflect sentiment information in an audio. Specifically, our\nmodel is trained by utterance-level labels and ASV can be extracted and fused\ncreatively from two branches. In the CNN model branch, spectrum graphs produced\nby signals are fed as inputs while in the LSTM model branch, inputs include\nspectral features and cepstrum coefficient extracted from dependent utterances\nin audio. Besides, Bidirectional Long Short-Term Memory (BiLSTM) with attention\nmechanism is used for feature fusion. Extensive experiments have been conducted\nto show our model can recognize audio sentiment precisely and quickly, and\ndemonstrate our ASV is better than traditional acoustic features or vectors\nextracted from other deep learning models. Furthermore, experimental results\nindicate that the proposed model outperforms the state-of-the-art approach by\n9.33\\% on Multimodal Opinion-level Sentiment Intensity dataset (MOSI) dataset.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.08162v1",
            "date": "2018-11-20 10:12:55+00:00",
            "title": "DeepZip: Lossless Data Compression using Recurrent Neural Networks",
            "authors": [
                "Mohit Goyal",
                "Kedar Tatwawadi",
                "Shubham Chandak",
                "Idoia Ochoa"
            ],
            "abstract": "Sequential data is being generated at an unprecedented pace in various forms,\nincluding text and genomic data. This creates the need for efficient\ncompression mechanisms to enable better storage, transmission and processing of\nsuch data. To solve this problem, many of the existing compressors attempt to\nlearn models for the data and perform prediction-based compression. Since\nneural networks are known as universal function approximators with the\ncapability to learn arbitrarily complex mappings, and in practice show\nexcellent performance in prediction tasks, we explore and devise methods to\ncompress sequential data using neural network predictors. We combine recurrent\nneural network predictors with an arithmetic coder and losslessly compress a\nvariety of synthetic, text and genomic datasets. The proposed compressor\noutperforms Gzip on the real datasets and achieves near-optimal compression for\nthe synthetic datasets. The results also help understand why and where neural\nnetworks are good alternatives for traditional finite context models",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08417v1",
            "date": "2018-11-20 18:47:50+00:00",
            "title": "WEST: Word Encoded Sequence Transducers",
            "authors": [
                "Ehsan Variani",
                "Ananda Theertha Suresh",
                "Mitchel Weintraub"
            ],
            "abstract": "Most of the parameters in large vocabulary models are used in embedding layer\nto map categorical features to vectors and in softmax layer for classification\nweights. This is a bottle-neck in memory constraint on-device training\napplications like federated learning and on-device inference applications like\nautomatic speech recognition (ASR). One way of compressing the embedding and\nsoftmax layers is to substitute larger units such as words with smaller\nsub-units such as characters. However, often the sub-unit models perform poorly\ncompared to the larger unit models. We propose WEST, an algorithm for encoding\ncategorical features and output classes with a sequence of random or domain\ndependent sub-units and demonstrate that this transduction can lead to\nsignificant compression without compromising performance. WEST bridges the gap\nbetween larger unit and sub-unit models and can be interpreted as a MaxEnt\nmodel over sub-unit features, which can be of independent interest.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.08465v4",
            "date": "2018-11-20 19:54:41+00:00",
            "title": "Fading of collective attention shapes the evolution of linguistic variants",
            "authors": [
                "Diego E Shalom",
                "Mariano Sigman",
                "Gabriel Mindlin",
                "Marcos A Trevisan"
            ],
            "abstract": "Language change involves the competition between alternative linguistic forms\n(1). The spontaneous evolution of these forms typically results in monotonic\ngrowths or decays (2, 3) like in winner-take-all attractor behaviors. In the\ncase of the Spanish past subjunctive, the spontaneous evolution of its two\ncompeting forms (ended in -ra and -se) was perturbed by the appearance of the\nRoyal Spanish Academy in 1713, which enforced the spelling of both forms as\nperfectly interchangeable variants (4), at a moment in which the -ra form was\ndominant (5). Time series extracted from a massive corpus of books (6) reveal\nthat this regulation in fact produced a transient renewed interest for the old\nform -se which, once faded, left the -ra again as the dominant form up to the\npresent day. We show that time series are successfully explained by a\ntwo-dimensional linear model that integrates an imitative and a novelty\ncomponent. The model reveals that the temporal scale over which collective\nattention fades is in inverse proportion to the verb frequency. The integration\nof the two basic mechanisms of imitation and attention to novelty allows to\nunderstand diverse competing objects, with lifetimes that range from hours for\nmemes and news (7, 8) to decades for verbs, suggesting the existence of a\ngeneral mechanism underlying cultural evolution.",
            "journal ref": "Phys. Rev. E 100, 020102 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.08705v1",
            "date": "2018-11-21 12:14:12+00:00",
            "title": "Inline Detection of Domain Generation Algorithms with Context-Sensitive Word Embeddings",
            "authors": [
                "Joewie J. Koh",
                "Barton Rhodes"
            ],
            "abstract": "Domain generation algorithms (DGAs) are frequently employed by malware to\ngenerate domains used for connecting to command-and-control (C2) servers.\nRecent work in DGA detection leveraged deep learning architectures like\nconvolutional neural networks (CNNs) and character-level long short-term memory\nnetworks (LSTMs) to classify domains. However, these classifiers perform poorly\nwith wordlist-based DGA families, which generate domains by pseudorandomly\nconcatenating dictionary words. We propose a novel approach that combines\ncontext-sensitive word embeddings with a simple fully-connected classifier to\nperform classification of domains based on word-level information. The word\nembeddings were pre-trained on a large unrelated corpus and left frozen during\nthe training on domain data. The resulting small number of trainable parameters\nenabled extremely short training durations, while the transfer of language\nknowledge stored in the representations allowed for high-performing models with\nsmall training datasets. We show that this architecture reliably outperformed\nexisting techniques on wordlist-based DGA families with just 30 DGA training\nexamples and achieved state-of-the-art performance with around 100 DGA training\nexamples, all while requiring an order of magnitude less time to train compared\nto current techniques. Of special note is the technique's performance on the\nmatsnu DGA: the classifier attained a 89.5% detection rate with a 1:1,000 false\npositive rate (FPR) after training on only 30 examples of the DGA domains, and\na 91.2% detection rate with a 1:10,000 FPR after 90 examples. Considering that\nsome of these DGAs have wordlists of several hundred words, our results\ndemonstrate that this technique does not rely on the classifier learning the\nDGA wordlists. Instead, the classifier is able to learn the semantic signatures\nof the wordlist-based DGA families.",
            "journal ref": "Proceedings of the 2018 IEEE International Conference on Big Data,\n  2018, pp. 2966-2971",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1811.08935v1",
            "date": "2018-11-14 08:54:24+00:00",
            "title": "A Study of Language and Classifier-independent Feature Analysis for Vocal Emotion Recognition",
            "authors": [
                "Fatemeh Noroozi",
                "Marina Marjanovic",
                "Angelina Njegus",
                "Sergio Escalera",
                "Gholamreza Anbarjafari"
            ],
            "abstract": "Every speech signal carries implicit information about the emotions, which\ncan be extracted by speech processing methods. In this paper, we propose an\nalgorithm for extracting features that are independent from the spoken language\nand the classification method to have comparatively good recognition\nperformance on different languages independent from the employed classification\nmethods. The proposed algorithm is composed of three stages. In the first\nstage, we propose a feature ranking method analyzing the state-of-the-art voice\nquality features. In the second stage, we propose a method for finding the\nsubset of the common features for each language and classifier. In the third\nstage, we compare our approach with the recognition rate of the\nstate-of-the-art filter methods. We use three databases with different\nlanguages, namely, Polish, Serbian and English. Also three different\nclassifiers, namely, nearest neighbour, support vector machine and gradient\ndescent neural network, are employed. It is shown that our method for selecting\nthe most significant language-independent and method-independent features in\nmany cases outperforms state-of-the-art filter methods.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.09010v1",
            "date": "2018-11-22 03:46:44+00:00",
            "title": "Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective",
            "authors": [
                "Zhong-Qiu Wang",
                "Ke Tan",
                "DeLiang Wang"
            ],
            "abstract": "This study investigates phase reconstruction for deep learning based monaural\ntalker-independent speaker separation in the short-time Fourier transform\n(STFT) domain. The key observation is that, for a mixture of two sources, with\ntheir magnitudes accurately estimated and under a geometric constraint, the\nabsolute phase difference between each source and the mixture can be uniquely\ndetermined; in addition, the source phases at each time-frequency (T-F) unit\ncan be narrowed down to only two candidates. To pick the right candidate, we\npropose three algorithms based on iterative phase reconstruction, group delay\nestimation, and phase-difference sign prediction. State-of-the-art results are\nobtained on the publicly available wsj0-2mix and 3mix corpus.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.09021v1",
            "date": "2018-11-22 04:37:55+00:00",
            "title": "Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes",
            "authors": [
                "Bo Li",
                "Yu Zhang",
                "Tara Sainath",
                "Yonghui Wu",
                "William Chan"
            ],
            "abstract": "We present two end-to-end models: Audio-to-Byte (A2B) and Byte-to-Audio\n(B2A), for multilingual speech recognition and synthesis. Prior work has\npredominantly used characters, sub-words or words as the unit of choice to\nmodel text. These units are difficult to scale to languages with large\nvocabularies, particularly in the case of multilingual processing. In this\nwork, we model text via a sequence of Unicode bytes, specifically, the UTF-8\nvariable length byte sequence for each character. Bytes allow us to avoid large\nsoftmaxes in languages with large vocabularies, and share representations in\nmultilingual models. We show that bytes are superior to grapheme characters\nover a wide variety of languages in monolingual end-to-end speech recognition.\nAdditionally, our multilingual byte model outperform each respective single\nlanguage baseline on average by 4.4% relatively. In Japanese-English\ncode-switching speech, our multilingual byte model outperform our monolingual\nbaseline by 38.6% relatively. Finally, we present an end-to-end multilingual\nspeech synthesis model using byte representations which matches the performance\nof our monolingual baselines.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.09364v4",
            "date": "2018-11-23 05:24:15+00:00",
            "title": "Learning pronunciation from a foreign language in speech synthesis networks",
            "authors": [
                "Younggun Lee",
                "Suwon Shon",
                "Taesu Kim"
            ],
            "abstract": "Although there are more than 6,500 languages in the world, the pronunciations\nof many phonemes sound similar across the languages. When people learn a\nforeign language, their pronunciation often reflects their native language's\ncharacteristics. This motivates us to investigate how the speech synthesis\nnetwork learns the pronunciation from datasets from different languages. In\nthis study, we are interested in analyzing and taking advantage of multilingual\nspeech synthesis network. First, we train the speech synthesis network\nbilingually in English and Korean and analyze how the network learns the\nrelations of phoneme pronunciation between the languages. Our experimental\nresult shows that the learned phoneme embedding vectors are located closer if\ntheir pronunciations are similar across the languages. Consequently, the\ntrained networks can synthesize the English speakers' Korean speech and vice\nversa. Using this result, we propose a training framework to utilize\ninformation from a different language. To be specific, we pre-train a speech\nsynthesis network using datasets from both high-resource language and\nlow-resource language, then we fine-tune the network using the low-resource\nlanguage dataset. Finally, we conducted more simulations on 10 different\nlanguages to show it is generally extendable to other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.09529v1",
            "date": "2018-11-23 16:00:51+00:00",
            "title": "Competency Questions and SPARQL-OWL Queries Dataset and Analysis",
            "authors": [
                "Dawid Wisniewski",
                "Jedrzej Potoniec",
                "Agnieszka Lawrynowicz",
                "C. Maria Keet"
            ],
            "abstract": "Competency Questions (CQs) are natural language questions outlining and\nconstraining the scope of knowledge represented by an ontology. Despite that\nCQs are a part of several ontology engineering methodologies, we have observed\nthat the actual publication of CQs for the available ontologies is very limited\nand even scarcer is the publication of their respective formalisations in terms\nof, e.g., SPARQL queries. This paper aims to contribute to addressing the\nengineering shortcomings of using CQs in ontology development, to facilitate\nwider use of CQs. In order to understand the relation between CQs and the\nqueries over the ontology to test the CQs on an ontology, we gather, analyse,\nand publicly release a set of 234 CQs and their translations to SPARQL-OWL for\nseveral ontologies in different domains developed by different groups. We\nanalysed the CQs in two principal ways. The first stage focused on a linguistic\nanalysis of the natural language text itself, i.e., a lexico-syntactic analysis\nwithout any presuppositions of ontology elements, and a subsequent step of\nsemantic analysis in order to find patterns. This increased diversity of CQ\nsources resulted in a 5-fold increase of hitherto published patterns, to 106\ndistinct CQ patterns, which have a limited subset of few patterns shared across\nthe CQ sets from the different ontologies. Next, we analysed the relation\nbetween the found CQ patterns and the 46 SPARQL-OWL query signatures, which\nrevealed that one CQ pattern may be realised by more than one SPARQL-OWL query\nsignature, and vice versa. We hope that our work will contribute to\nestablishing common practices, templates, automation, and user tools that will\nsupport CQ formulation, formalisation, execution, and general management.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.09607v1",
            "date": "2018-11-21 19:20:43+00:00",
            "title": "Towards Emotion Recognition: A Persistent Entropy Application",
            "authors": [
                "R. Gonzalez-Diaz",
                "E. Paluzo-Hidalgo",
                "J. F. Quesada"
            ],
            "abstract": "Emotion recognition and classification is a very active area of research. In\nthis paper, we present a first approach to emotion classification using\npersistent entropy and support vector machines. A topology-based model is\napplied to obtain a single real number from each raw signal. These data are\nused as input of a support vector machine to classify signals into 8 different\nemotions (calm, happy, sad, angry, fearful, disgust and surprised).",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.09688v1",
            "date": "2018-11-16 01:35:09+00:00",
            "title": "A Voice Controlled E-Commerce Web Application",
            "authors": [
                "Mandeep Singh Kandhari",
                "Farhana Zulkernine",
                "Haruna Isah"
            ],
            "abstract": "Automatic voice-controlled systems have changed the way humans interact with\na computer. Voice or speech recognition systems allow a user to make a\nhands-free request to the computer, which in turn processes the request and\nserves the user with appropriate responses. After years of research and\ndevelopments in machine learning and artificial intelligence, today\nvoice-controlled technologies have become more efficient and are widely applied\nin many domains to enable and improve human-to-human and human-to-computer\ninteractions. The state-of-the-art e-commerce applications with the help of web\ntechnologies offer interactive and user-friendly interfaces. However, there are\nsome instances where people, especially with visual disabilities, are not able\nto fully experience the serviceability of such applications. A voice-controlled\nsystem embedded in a web application can enhance user experience and can\nprovide voice as a means to control the functionality of e-commerce websites.\nIn this paper, we propose a taxonomy of speech recognition systems (SRS) and\npresent a voice-controlled commodity purchase e-commerce application using IBM\nWatson speech-to-text to demonstrate its usability. The prototype can be\nextended to other application scenarios such as government service kiosks and\nenable analytics of the converted text data for scenarios such as medical\ndiagnosis at the clinics.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1811.09725v2",
            "date": "2018-11-23 23:13:09+00:00",
            "title": "Interpretable Convolutional Filters with SincNet",
            "authors": [
                "Mirco Ravanelli",
                "Yoshua Bengio"
            ],
            "abstract": "Deep learning is currently playing a crucial role toward higher levels of\nartificial intelligence. This paradigm allows neural networks to learn complex\nand abstract representations, that are progressively obtained by combining\nsimpler ones. Nevertheless, the internal \"black-box\" representations\nautomatically discovered by current neural architectures often suffer from a\nlack of interpretability, making of primary interest the study of explainable\nmachine learning techniques. This paper summarizes our recent efforts to\ndevelop a more interpretable neural model for directly processing speech from\nthe raw waveform. In particular, we propose SincNet, a novel Convolutional\nNeural Network (CNN) that encourages the first layer to discover more\nmeaningful filters by exploiting parametrized sinc functions. In contrast to\nstandard CNNs, which learn all the elements of each filter, only low and high\ncutoff frequencies of band-pass filters are directly learned from data. This\ninductive bias offers a very compact way to derive a customized filter-bank\nfront-end, that only depends on some parameters with a clear physical meaning.\nOur experiments, conducted on both speaker and speech recognition, show that\nthe proposed architecture converges faster, performs better, and is more\ninterpretable than standard CNNs.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.09740v2",
            "date": "2018-11-24 01:33:39+00:00",
            "title": "Connecting the Dots Between MLE and RL for Sequence Prediction",
            "authors": [
                "Bowen Tan",
                "Zhiting Hu",
                "Zichao Yang",
                "Ruslan Salakhutdinov",
                "Eric Xing"
            ],
            "abstract": "Sequence prediction models can be learned from example sequences with a\nvariety of training algorithms. Maximum likelihood learning is simple and\nefficient, yet can suffer from compounding error at test time. Reinforcement\nlearning such as policy gradient addresses the issue but can have prohibitively\npoor exploration efficiency. A rich set of other algorithms such as RAML, SPG,\nand data noising, have also been developed from different perspectives. This\npaper establishes a formal connection between these algorithms. We present a\ngeneralized entropy regularized policy optimization formulation, and show that\nthe apparently distinct algorithms can all be reformulated as special instances\nof the framework, with the only difference being the configurations of a reward\nfunction and a couple of hyperparameters. The unified interpretation offers a\nsystematic view of the varying properties of exploration and learning\nefficiency. Besides, inspired from the framework, we present a new algorithm\nthat dynamically interpolates among the family of algorithms for scheduled\nsequence model learning. Experiments on machine translation, text\nsummarization, and game imitation learning demonstrate the superiority of the\nproposed algorithm.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.09786v1",
            "date": "2018-11-24 08:15:50+00:00",
            "title": "Recurrently Controlled Recurrent Networks",
            "authors": [
                "Yi Tay",
                "Luu Anh Tuan",
                "Siu Cheung Hui"
            ],
            "abstract": "Recurrent neural networks (RNNs) such as long short-term memory and gated\nrecurrent units are pivotal building blocks across a broad spectrum of sequence\nmodeling problems. This paper proposes a recurrently controlled recurrent\nnetwork (RCRN) for expressive and powerful sequence encoding. More concretely,\nthe key idea behind our approach is to learn the recurrent gating functions\nusing recurrent networks. Our architecture is split into two components - a\ncontroller cell and a listener cell whereby the recurrent controller actively\ninfluences the compositionality of the listener cell. We conduct extensive\nexperiments on a myriad of tasks in the NLP domain such as sentiment analysis\n(SST, IMDb, Amazon reviews, etc.), question classification (TREC), entailment\nclassification (SNLI, SciTail), answer selection (WikiQA, TrecQA) and reading\ncomprehension (NarrativeQA). Across all 26 datasets, our results demonstrate\nthat RCRN not only consistently outperforms BiLSTMs but also stacked BiLSTMs,\nsuggesting that our controller architecture might be a suitable replacement for\nthe widely adopted stacked architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10092v2",
            "date": "2018-11-25 20:49:58+00:00",
            "title": "Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation",
            "authors": [
                "Xin Wang",
                "Qiuyuan Huang",
                "Asli Celikyilmaz",
                "Jianfeng Gao",
                "Dinghan Shen",
                "Yuan-Fang Wang",
                "William Yang Wang",
                "Lei Zhang"
            ],
            "abstract": "Vision-language navigation (VLN) is the task of navigating an embodied agent\nto carry out natural language instructions inside real 3D environments. In this\npaper, we study how to address three critical challenges for this task: the\ncross-modal grounding, the ill-posed feedback, and the generalization problems.\nFirst, we propose a novel Reinforced Cross-Modal Matching (RCM) approach that\nenforces cross-modal grounding both locally and globally via reinforcement\nlearning (RL). Particularly, a matching critic is used to provide an intrinsic\nreward to encourage global matching between instructions and trajectories, and\na reasoning navigator is employed to perform cross-modal grounding in the local\nvisual scene. Evaluation on a VLN benchmark dataset shows that our RCM model\nsignificantly outperforms previous methods by 10% on SPL and achieves the new\nstate-of-the-art performance. To improve the generalizability of the learned\npolicy, we further introduce a Self-Supervised Imitation Learning (SIL) method\nto explore unseen environments by imitating its own past, good decisions. We\ndemonstrate that SIL can approximate a better and more efficient policy, which\ntremendously minimizes the success rate performance gap between seen and unseen\nenvironments (from 30.7% to 11.7%).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.10229v1",
            "date": "2018-11-26 08:28:21+00:00",
            "title": "Augmenting Robot Knowledge Consultants with Distributed Short Term Memory",
            "authors": [
                "Tom Williams",
                "Ravenna Thielstrom",
                "Evan Krause",
                "Bradley Oosterveld",
                "Matthias Scheutz"
            ],
            "abstract": "Human-robot communication in situated environments involves a complex\ninterplay between knowledge representations across a wide variety of\nmodalities. Crucially, linguistic information must be associated with\nrepresentations of objects, locations, people, and goals, which may be\nrepresented in very different ways. In previous work, we developed a Consultant\nFramework that facilitates modality-agnostic access to information distributed\nacross a set of heterogeneously represented knowledge sources. In this work, we\ndraw inspiration from cognitive science to augment these distributed knowledge\nsources with Short Term Memory Buffers to create an STM-augmented algorithm for\nreferring expression generation. We then discuss the potential performance\nbenefits of this approach and insights from cognitive science that may inform\nfuture refinements in the design of our approach.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1811.10369v1",
            "date": "2018-11-26 13:56:57+00:00",
            "title": "ParsRec: A Novel Meta-Learning Approach to Recommending Bibliographic Reference Parsers",
            "authors": [
                "Dominika Tkaczyk",
                "Rohit Gupta",
                "Riccardo Cinti",
                "Joeran Beel"
            ],
            "abstract": "Bibliographic reference parsers extract machine-readable metadata such as\nauthor names, title, journal, and year from bibliographic reference strings. To\nextract the metadata, the parsers apply heuristics or machine learning.\nHowever, no reference parser, and no algorithm, consistently gives the best\nresults in every scenario. For instance, one tool may be best in extracting\ntitles in ACM citation style, but only third best when APA is used. Another\ntool may be best in extracting English author names, while another one is best\nfor noisy data (i.e. inconsistent citation styles). In this paper, which is an\nextended version of our recent RecSys poster, we address the problem of\nreference parsing from a recommender-systems and meta-learning perspective. We\npropose ParsRec, a meta-learning based recommender-system that recommends the\npotentially most effective parser for a given reference string. ParsRec\nrecommends one out of 10 open-source parsers: Anystyle-Parser, Biblio, CERMINE,\nCitation, Citation-Parser, GROBID, ParsCit, PDFSSA4MET, Reference Tagger, and\nScience Parse. We evaluate ParsRec on 105k references from chemistry. We\npropose two approaches to meta-learning recommendations. The first approach\nlearns the best parser for an entire reference string. The second approach\nlearns the best parser for each metadata type in a reference string. The second\napproach achieved a 2.6% increase in F1 (0.909 vs. 0.886) over the best single\nparser (GROBID), reducing the false positive rate by 20.2% (0.075 vs. 0.094),\nand the false negative rate by 18.9% (0.107 vs. 0.132).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.10422v1",
            "date": "2018-11-22 12:55:40+00:00",
            "title": "Creating a contemporary corpus of similes in Serbian by using natural language processing",
            "authors": [
                "Nikola Milosevic",
                "Goran Nenadic"
            ],
            "abstract": "Simile is a figure of speech that compares two things through the use of\nconnection words, but where comparison is not intended to be taken literally.\nThey are often used in everyday communication, but they are also a part of\nlinguistic cultural heritage. In this paper we present a methodology for\nsemi-automated collection of similes from the World Wide Web using text mining\nand machine learning techniques. We expanded an existing corpus by collecting\n442 similes from the internet and adding them to the existing corpus collected\nby Vuk Stefanovic Karadzic that contained 333 similes. We, also, introduce\ncrowdsourcing to the collection of figures of speech, which helped us to build\ncorpus containing 787 unique similes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10475v1",
            "date": "2018-11-26 16:07:36+00:00",
            "title": "Sentence Encoding with Tree-constrained Relation Networks",
            "authors": [
                "Lei Yu",
                "Cyprien de Masson d'Autume",
                "Chris Dyer",
                "Phil Blunsom",
                "Lingpeng Kong",
                "Wang Ling"
            ],
            "abstract": "The meaning of a sentence is a function of the relations that hold between\nits words. We instantiate this relational view of semantics in a series of\nneural models based on variants of relation networks (RNs) which represent a\nset of objects (for us, words forming a sentence) in terms of representations\nof pairs of objects. We propose two extensions to the basic RN model for\nnatural language. First, building on the intuition that not all word pairs are\nequally informative about the meaning of a sentence, we use constraints based\non both supervised and unsupervised dependency syntax to control which\nrelations influence the representation. Second, since higher-order relations\nare poorly captured by a sum of pairwise relations, we use a recurrent\nextension of RNs to propagate information so as to form representations of\nhigher order relations. Experiments on sentence classification, sentence pair\nclassification, and machine translation reveal that, while basic RNs are only\nmodestly effective for sentence representation, recurrent RNs with latent\nsyntax are a reliably powerful representational device.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10990v1",
            "date": "2018-11-15 22:59:25+00:00",
            "title": "Generating Responses Expressing Emotion in an Open-domain Dialogue System",
            "authors": [
                "Chenyang Huang",
                "Osmar R. Za\u00efane"
            ],
            "abstract": "Neural network-based Open-ended conversational agents automatically generate\nresponses based on predictive models learned from a large number of pairs of\nutterances. The generated responses are typically acceptable as a sentence but\nare often dull, generic, and certainly devoid of any emotion. In this paper, we\npresent neural models that learn to express a given emotion in the generated\nresponse. We propose four models and evaluate them against 3 baselines. An\nencoder-decoder framework-based model with multiple attention layers provides\nthe best overall performance in terms of expressing the required emotion. While\nit does not outperform other models on all emotions, it presents promising\nresults in most cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10999v1",
            "date": "2018-11-16 07:09:30+00:00",
            "title": "Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification",
            "authors": [
                "Zheng Li",
                "Ying Wei",
                "Yu Zhang",
                "Xiang Zhang",
                "Xin Li",
                "Qiang Yang"
            ],
            "abstract": "Aspect-level sentiment classification (ASC) aims at identifying sentiment\npolarities towards aspects in a sentence, where the aspect can behave as a\ngeneral Aspect Category (AC) or a specific Aspect Term (AT). However, due to\nthe especially expensive and labor-intensive labeling, existing public corpora\nin AT-level are all relatively small. Meanwhile, most of the previous methods\nrely on complicated structures with given scarce data, which largely limits the\nefficacy of the neural models. In this paper, we exploit a new direction named\ncoarse-to-fine task transfer, which aims to leverage knowledge learned from a\nrich-resource source domain of the coarse-grained AC task, which is more easily\naccessible, to improve the learning in a low-resource target domain of the\nfine-grained AT task. To resolve both the aspect granularity inconsistency and\nfeature mismatch between domains, we propose a Multi-Granularity Alignment\nNetwork (MGAN). In MGAN, a novel Coarse2Fine attention guided by an auxiliary\ntask can help the AC task modeling at the same fine-grained level with the AT\ntask. To alleviate the feature false alignment, a contrastive feature alignment\nmethod is adopted to align aspect-specific feature representations\nsemantically. In addition, a large-scale multi-domain dataset for the AC task\nis provided. Empirically, extensive experiments demonstrate the effectiveness\nof the MGAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11001v2",
            "date": "2018-11-17 20:12:40+00:00",
            "title": "Unsupervised Post-processing of Word Vectors via Conceptor Negation",
            "authors": [
                "Tianlin Liu",
                "Lyle Ungar",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Word vectors are at the core of many natural language processing tasks.\nRecently, there has been interest in post-processing word vectors to enrich\ntheir semantic information. In this paper, we introduce a novel word vector\npost-processing technique based on matrix conceptors (Jaeger2014), a family of\nregularized identity maps. More concretely, we propose to use conceptors to\nsuppress those latent features of word vectors having high variances. The\nproposed method is purely unsupervised: it does not rely on any corpus or\nexternal linguistic database. We evaluate the post-processed word vectors on a\nbattery of intrinsic lexical evaluation tasks, showing that the proposed method\nconsistently outperforms existing state-of-the-art alternatives. We also show\nthat post-processed word vectors can be used for the downstream natural\nlanguage processing task of dialogue state tracking, yielding improved results\nin different dialogue domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11002v1",
            "date": "2018-11-17 20:20:20+00:00",
            "title": "Correcting the Common Discourse Bias in Linear Representation of Sentences using Conceptors",
            "authors": [
                "Tianlin Liu",
                "Jo\u00e3o Sedoc",
                "Lyle Ungar"
            ],
            "abstract": "Distributed representations of words, better known as word embeddings, have\nbecome important building blocks for natural language processing tasks.\nNumerous studies are devoted to transferring the success of unsupervised word\nembeddings to sentence embeddings. In this paper, we introduce a simple\nrepresentation of sentences in which a sentence embedding is represented as a\nweighted average of word vectors followed by a soft projection. We demonstrate\nthe effectiveness of this proposed method on the clinical semantic textual\nsimilarity task of the BioCreative/OHNLP Challenge 2018.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11005v2",
            "date": "2018-11-23 13:04:12+00:00",
            "title": "Application of Clinical Concept Embeddings for Heart Failure Prediction in UK EHR data",
            "authors": [
                "Spiros Denaxas",
                "Pontus Stenetorp",
                "Sebastian Riedel",
                "Maria Pikoula",
                "Richard Dobson",
                "Harry Hemingway"
            ],
            "abstract": "Electronic health records (EHR) are increasingly being used for constructing\ndisease risk prediction models. Feature engineering in EHR data however is\nchallenging due to their highly dimensional and heterogeneous nature.\nLow-dimensional representations of EHR data can potentially mitigate these\nchallenges. In this paper, we use global vectors (GloVe) to learn word\nembeddings for diagnoses and procedures recorded using 13 million ontology\nterms across 2.7 million hospitalisations in national UK EHR. We demonstrate\nthe utility of these embeddings by evaluating their performance in identifying\npatients which are at higher risk of being hospitalised for congestive heart\nfailure. Our findings indicate that embeddings can enable the creation of\nrobust EHR-derived disease risk prediction models and address some the\nlimitations associated with manual clinical feature engineering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11008v1",
            "date": "2018-11-25 01:36:12+00:00",
            "title": "Sentiment Analysis of Financial News Articles using Performance Indicators",
            "authors": [
                "Srikumar Krishnamoorthy"
            ],
            "abstract": "Mining financial text documents and understanding the sentiments of\nindividual investors, institutions and markets is an important and challenging\nproblem in the literature. Current approaches to mine sentiments from financial\ntexts largely rely on domain specific dictionaries. However, dictionary based\nmethods often fail to accurately predict the polarity of financial texts. This\npaper aims to improve the state-of-the-art and introduces a novel sentiment\nanalysis approach that employs the concept of financial and non-financial\nperformance indicators. It presents an association rule mining based\nhierarchical sentiment classifier model to predict the polarity of financial\ntexts as positive, neutral or negative. The performance of the proposed model\nis evaluated on a benchmark financial dataset. The model is also compared\nagainst other state-of-the-art dictionary and machine learning based approaches\nand the results are found to be quite promising. The novel use of performance\nindicators for financial sentiment analysis offers interesting and useful\ninsights.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.11017v1",
            "date": "2018-11-24 17:50:41+00:00",
            "title": "Latent Dirichlet Allocation with Residual Convolutional Neural Network Applied in Evaluating Credibility of Chinese Listed Companies",
            "authors": [
                "Mohan Zhang",
                "Zhichao Luo",
                "Hai Lu"
            ],
            "abstract": "This project demonstrated a methodology to estimating cooperate credibility\nwith a Natural Language Processing approach. As cooperate transparency impacts\nboth the credibility and possible future earnings of the firm, it is an\nimportant factor to be considered by banks and investors on risk assessments of\nlisted firms. This approach of estimating cooperate credibility can bypass\nhuman bias and inconsistency in the risk assessment, the use of large\nquantitative data and neural network models provides more accurate estimation\nin a more efficient manner compare to manual assessment. At the beginning, the\nmodel will employs Latent Dirichlet Allocation and THU Open Chinese Lexicon\nfrom Tsinghua University to classify topics in articles which are potentially\nrelated to corporate credibility. Then with the keywords related to each\ntopics, we trained a residual convolutional neural network with data labeled\naccording to surveys of fund manager and accountant's opinion on corporate\ncredibility. After the training, we run the model with preprocessed news\nreports regarding to all of the 3065 listed companies, the model is supposed to\ngive back companies ranking based on the level of their transparency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11041v1",
            "date": "2018-11-08 05:11:18+00:00",
            "title": "Translating and Evolving: Towards a Model of Language Change in DisCoCat",
            "authors": [
                "Tai-Danae Bradley",
                "Martha Lewis",
                "Jade Master",
                "Brad Theilman"
            ],
            "abstract": "The categorical compositional distributional (DisCoCat) model of meaning\ndeveloped by Coecke et al. (2010) has been successful in modeling various\naspects of meaning. However, it fails to model the fact that language can\nchange. We give an approach to DisCoCat that allows us to represent language\nmodels and translations between them, enabling us to describe translations from\none language to another, or changes within the same language. We unify the\nproduct space representation given in (Coecke et al., 2010) and the functorial\ndescription in (Kartsaklis et al., 2013), in a way that allows us to view a\nlanguage as a catalogue of meanings. We formalize the notion of a lexicon in\nDisCoCat, and define a dictionary of meanings between two lexicons. All this is\ndone within the framework of monoidal categories. We give examples of how to\napply our methods, and give a concrete suggestion for compositional translation\nin corpora.",
            "journal ref": "EPTCS 283, 2018, pp. 50-61",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.11078v2",
            "date": "2018-11-27 16:26:17+00:00",
            "title": "Refined WaveNet Vocoder for Variational Autoencoder Based Voice Conversion",
            "authors": [
                "Wen-Chin Huang",
                "Yi-Chiao Wu",
                "Hsin-Te Hwang",
                "Patrick Lumban Tobing",
                "Tomoki Hayashi",
                "Kazuhiro Kobayashi",
                "Tomoki Toda",
                "Yu Tsao",
                "Hsin-Min Wang"
            ],
            "abstract": "This paper presents a refinement framework of WaveNet vocoders for\nvariational autoencoder (VAE) based voice conversion (VC), which reduces the\nquality distortion caused by the mismatch between the training data and testing\ndata. Conventional WaveNet vocoders are trained with natural acoustic features\nbut conditioned on the converted features in the conversion stage for VC, and\nsuch a mismatch often causes significant quality and similarity degradation. In\nthis work, we take advantage of the particular structure of VAEs to refine\nWaveNet vocoders with the self-reconstructed features generated by VAE, which\nare of similar characteristics with the converted features while having the\nsame temporal structure with the target natural features. We analyze these\nfeatures and show that the self-reconstructed features are similar to the\nconverted features. Objective and subjective experimental results demonstrate\nthe effectiveness of our proposed framework.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.11456v2",
            "date": "2018-11-28 09:20:13+00:00",
            "title": "GIRNet: Interleaved Multi-Task Recurrent State Sequence Models",
            "authors": [
                "Divam Gupta",
                "Tanmoy Chakraborty",
                "Soumen Chakrabarti"
            ],
            "abstract": "In several natural language tasks, labeled sequences are available in\nseparate domains (say, languages), but the goal is to label sequences with\nmixed domain (such as code-switched text). Or, we may have available models for\nlabeling whole passages (say, with sentiments), which we would like to exploit\ntoward better position-specific label inference (say, target-dependent\nsentiment annotation). A key characteristic shared across such tasks is that\ndifferent positions in a primary instance can benefit from different `experts'\ntrained from auxiliary data, but labeled primary instances are scarce, and\nlabeling the best expert for each position entails unacceptable cognitive\nburden. We propose GITNet, a unified position-sensitive multi-task recurrent\nneural network (RNN) architecture for such applications. Auxiliary and primary\ntasks need not share training instances. Auxiliary RNNs are trained over\nauxiliary instances. A primary instance is also submitted to each auxiliary\nRNN, but their state sequences are gated and merged into a novel composite\nstate sequence tailored to the primary inference task. Our approach is in sharp\ncontrast to recent multi-task networks like the cross-stitch and sluice\nnetwork, which do not control state transfer at such fine granularity. We\ndemonstrate the superiority of GIRNet using three applications: sentiment\nclassification of code-switched passages, part-of-speech tagging of\ncode-switched text, and target position-sensitive annotation of sentiment in\nmonolingual passages. In all cases, we establish new state-of-the-art\nperformance beyond recent competitive baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.11683v2",
            "date": "2018-11-28 17:05:27+00:00",
            "title": "Multi-level Multimodal Common Semantic Space for Image-Phrase Grounding",
            "authors": [
                "Hassan Akbari",
                "Svebor Karaman",
                "Surabhi Bhargava",
                "Brian Chen",
                "Carl Vondrick",
                "Shih-Fu Chang"
            ],
            "abstract": "We address the problem of phrase grounding by lear ing a multi-level common\nsemantic space shared by the textual and visual modalities. We exploit multiple\nlevels of feature maps of a Deep Convolutional Neural Network, as well as\ncontextualized word and sentence embeddings extracted from a character-based\nlanguage model. Following dedicated non-linear mappings for visual features at\neach level, word, and sentence embeddings, we obtain multiple instantiations of\nour common semantic space in which comparisons between any target text and the\nvisual content is performed with cosine similarity. We guide the model by a\nmulti-level multimodal attention mechanism which outputs attended visual\nfeatures at each level. The best level is chosen to be compared with text\ncontent for maximizing the pertinence scores of image-sentence pairs of the\nground truth. Experiments conducted on three publicly available datasets show\nsignificant performance gains (20%-60% relative) over the state-of-the-art in\nphrase localization and set a new performance record on those datasets. We\nprovide a detailed ablation study to show the contribution of each element of\nour approach and release our code on GitHub.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.12157v1",
            "date": "2018-09-26 13:43:01+00:00",
            "title": "Universal Network Representation for Heterogeneous Information Networks",
            "authors": [
                "Ruiqi Hu",
                "Celina Ping Yu",
                "Sai-Fu Fung",
                "Shirui Pan",
                "Haishuai Wang",
                "Guodong Long"
            ],
            "abstract": "Network representation aims to represent the nodes in a network as continuous\nand compact vectors, and has attracted much attention in recent years due to\nits ability to capture complex structure relationships inside networks.\nHowever, existing network representation methods are commonly designed for\nhomogeneous information networks where all the nodes (entities) of a network\nare of the same type, e.g., papers in a citation network. In this paper, we\npropose a universal network representation approach (UNRA), that represents\ndifferent types of nodes in heterogeneous information networks in a continuous\nand common vector space. The UNRA is built on our latest mutually updated\nneural language module, which simultaneously captures inter-relationship among\nhomogeneous nodes and node-content correlation. Relationships between different\ntypes of nodes are also assembled and learned in a unified framework.\nExperiments validate that the UNRA achieves outstanding performance, compared\nto six other state-of-the-art algorithms, in node representation, node\nclassification, and network visualization. In node classification, the UNRA\nachieves a 3\\% to 132\\% performance improvement in terms of accuracy.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1811.12239v1",
            "date": "2018-11-29 15:20:30+00:00",
            "title": "Counterfactual Learning from Human Proofreading Feedback for Semantic Parsing",
            "authors": [
                "Carolin Lawrence",
                "Stefan Riezler"
            ],
            "abstract": "In semantic parsing for question-answering, it is often too expensive to\ncollect gold parses or even gold answers as supervision signals. We propose to\nconvert model outputs into a set of human-understandable statements which allow\nnon-expert users to act as proofreaders, providing error markings as learning\nsignals to the parser. Because model outputs were suggested by a historic\nsystem, we operate in a counterfactual, or off-policy, learning setup. We\nintroduce new estimators which can effectively leverage the given feedback and\nwhich avoid known degeneracies in counterfactual learning, while still being\napplicable to stochastic gradient optimization for neural semantic parsing.\nFurthermore, we discuss how our feedback collection method can be seamlessly\nintegrated into deployed virtual personal assistants that embed a semantic\nparser. Our work is the first to show that semantic parsers can be improved\nsignificantly by counterfactual learning from logged human feedback data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12276v2",
            "date": "2018-11-29 16:10:41+00:00",
            "title": "Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning",
            "authors": [
                "Mengqi Jin",
                "Mohammad Taha Bahadori",
                "Aaron Colak",
                "Parminder Bhatia",
                "Busra Celikkaya",
                "Ram Bhakta",
                "Selvan Senthivel",
                "Mohammed Khalilia",
                "Daniel Navarro",
                "Borui Zhang",
                "Tiberiu Doman",
                "Arun Ravi",
                "Matthieu Liger",
                "Taha Kass-hout"
            ],
            "abstract": "Clinical text provides essential information to estimate the acuity of a\npatient during hospital stays in addition to structured clinical data. In this\nstudy, we explore how clinical text can complement a clinical predictive\nlearning task. We leverage an internal medical natural language processing\nservice to perform named entity extraction and negation detection on clinical\nnotes and compose selected entities into a new text corpus to train document\nrepresentations. We then propose a multimodal neural network to jointly train\ntime series signals and unstructured clinical text representations to predict\nthe in-hospital mortality risk for ICU patients. Our model outperforms the\nbenchmark by 2% AUC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12354v7",
            "date": "2018-11-29 18:06:22+00:00",
            "title": "Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments",
            "authors": [
                "Howard Chen",
                "Alane Suhr",
                "Dipendra Misra",
                "Noah Snavely",
                "Yoav Artzi"
            ],
            "abstract": "We study the problem of jointly reasoning about language and vision through a\nnavigation and spatial reasoning task. We introduce the Touchdown task and\ndataset, where an agent must first follow navigation instructions in a\nreal-life visual urban environment, and then identify a location described in\nnatural language to find a hidden object at the goal position. The data\ncontains 9,326 examples of English instructions and spatial descriptions paired\nwith demonstrations. Empirical analysis shows the data presents an open\nchallenge to existing methods, and qualitative linguistic analysis shows that\nthe data displays richer use of spatial reasoning compared to related\nresources.",
            "journal ref": "Published in CVPR 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1811.12640v2",
            "date": "2018-11-30 06:55:20+00:00",
            "title": "Inferring Concept Prerequisite Relations from Online Educational Resources",
            "authors": [
                "Sudeshna Roy",
                "Meghana Madhyastha",
                "Sheril Lawrence",
                "Vaibhav Rajan"
            ],
            "abstract": "The Internet has rich and rapidly increasing sources of high quality\neducational content. Inferring prerequisite relations between educational\nconcepts is required for modern large-scale online educational technology\napplications such as personalized recommendations and automatic curriculum\ncreation. We present PREREQ, a new supervised learning method for inferring\nconcept prerequisite relations. PREREQ is designed using latent representations\nof concepts obtained from the Pairwise Latent Dirichlet Allocation model, and a\nneural network based on the Siamese network architecture. PREREQ can learn\nunknown concept prerequisites from course prerequisites and labeled concept\nprerequisite data. It outperforms state-of-the-art approaches on benchmark\ndatasets and can effectively learn from very less training data. PREREQ can\nalso use unlabeled video playlists, a steadily growing source of training data,\nto learn concept prerequisites, thus obviating the need for manual annotation\nof course prerequisites.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12739v2",
            "date": "2018-11-30 11:38:54+00:00",
            "title": "Neural separation of observed and unobserved distributions",
            "authors": [
                "Tavi Halperin",
                "Ariel Ephrat",
                "Yedid Hoshen"
            ],
            "abstract": "Separating mixed distributions is a long standing challenge for machine\nlearning and signal processing. Most current methods either rely on making\nstrong assumptions on the source distributions or rely on having training\nsamples of each source in the mixture. In this work, we introduce a new\nmethod---Neural Egg Separation---to tackle the scenario of extracting a signal\nfrom an unobserved distribution additively mixed with a signal from an observed\ndistribution. Our method iteratively learns to separate the known distribution\nfrom progressively finer estimates of the unknown distribution. In some\nsettings, Neural Egg Separation is initialization sensitive, we therefore\nintroduce Latent Mixture Masking which ensures a good initialization. Extensive\nexperiments on audio and image separation tasks show that our method\noutperforms current methods that use the same level of supervision, and often\nachieves similar performance to full supervision.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.00049v1",
            "date": "2018-11-28 23:34:06+00:00",
            "title": "The Indus Script and Economics. A Role for Indus Seals and Tablets in Rationing and Administration of Labor",
            "authors": [
                "Rajesh P. N. Rao"
            ],
            "abstract": "The Indus script remains one of the last major undeciphered scripts of the\nancient world. We focus here on Indus inscriptions on a group of miniature\ntablets discovered by Meadow and Kenoyer in Harappa in 1997. By drawing\nparallels with proto-Elamite and proto-Cuneiform inscriptions, we explore how\nthese miniature tablets may have been used to record rations allocated to\nporters or laborers. We then show that similar inscriptions are found on stamp\nseals, leading to the potentially provocative conclusion that rather than\nsimply indicating ownership of property, Indus seals may have been used for\ngenerating tokens, tablets and sealings for repetitive economic transactions\nsuch as rations and exchange of canonical amounts of goods, grains, animals,\nand labor in a barter-based economy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00315v2",
            "date": "2018-12-02 03:27:15+00:00",
            "title": "A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities",
            "authors": [
                "Xinyi Zhou",
                "Reza Zafarani"
            ],
            "abstract": "The explosive growth in fake news and its erosion to democracy, justice, and\npublic trust has increased the demand for fake news detection and intervention.\nThis survey reviews and evaluates methods that can detect fake news from four\nperspectives: (1) the false knowledge it carries, (2) its writing style, (3)\nits propagation patterns, and (4) the credibility of its source. The survey\nalso highlights some potential research tasks based on the review. In\nparticular, we identify and detail related fundamental theories across various\ndisciplines to encourage interdisciplinary research on fake news. We hope this\nsurvey can facilitate collaborative efforts among experts in computer and\ninformation sciences, social sciences, political science, and journalism to\nresearch fake news, where such efforts can lead to fake news detection that is\nnot only efficient but more importantly, explainable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.00381v2",
            "date": "2018-12-02 12:38:58+00:00",
            "title": "Towards Automatic Discovery of Cybercrime Supply Chains",
            "authors": [
                "Rasika Bhalerao",
                "Maxwell Aliapoulios",
                "Ilia Shumailov",
                "Sadia Afroz",
                "Damon McCoy"
            ],
            "abstract": "Cybercrime forums enable modern criminal entrepreneurs to collaborate with\nother criminals into increasingly efficient and sophisticated criminal\nendeavors. Understanding the connections between different products and\nservices can often illuminate effective interventions. However, generating this\nunderstanding of supply chains currently requires time-consuming manual effort.\n  In this paper, we propose a language-agnostic method to automatically extract\nsupply chains from cybercrime forum posts and replies. Our supply chain\ndetection algorithm can identify 36% and 58% relevant chains within major\nEnglish and Russian forums, respectively, showing improvements over the\nbaselines of 13% and 36%, respectively. Our analysis of the automatically\ngenerated supply chains demonstrates underlying connections between products\nand services within these forums. For example, the extracted supply chain\nilluminated the connection between hack-for-hire services and the selling of\nrare and valuable `OG' accounts, which has only recently been reported. The\nunderstanding of connections between products and services exposes potentially\neffective intervention points.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1812.00855v1",
            "date": "2018-12-03 16:00:48+00:00",
            "title": "Towards Solving Text-based Games by Producing Adaptive Action Spaces",
            "authors": [
                "Ruo Yu Tao",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Xingdi Yuan",
                "Layla El Asri"
            ],
            "abstract": "To solve a text-based game, an agent needs to formulate valid text commands\nfor a given context and find the ones that lead to success. Recent attempts at\nsolving text-based games with deep reinforcement learning have focused on the\nlatter, i.e., learning to act optimally when valid actions are known in\nadvance. In this work, we propose to tackle the first task and train a model\nthat generates the set of all valid commands for a given context. We try three\ngenerative models on a dataset generated with Textworld. The best model can\ngenerate valid commands which were unseen at training and achieve high $F_1$\nscore on the test set.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.00898v1",
            "date": "2018-12-03 16:51:35+00:00",
            "title": "Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning",
            "authors": [
                "Aishwarya Agrawal",
                "Mateusz Malinowski",
                "Felix Hill",
                "Ali Eslami",
                "Oriol Vinyals",
                "Tejas Kulkarni"
            ],
            "abstract": "Advances in Deep Reinforcement Learning have led to agents that perform well\nacross a variety of sensory-motor domains. In this work, we study the setting\nin which an agent must learn to generate programs for diverse scenes\nconditioned on a given symbolic instruction. Final goals are specified to our\nagent via images of the scenes. A symbolic instruction consistent with the goal\nimages is used as the conditioning input for our policies. Since a single\ninstruction corresponds to a diverse set of different but still consistent\nend-goal images, the agent needs to learn to generate a distribution over\nprograms given an instruction. We demonstrate that with simple changes to the\nreinforced adversarial learning objective, we can learn instruction conditioned\npolicies to achieve the corresponding diverse set of goals. Most importantly,\nour agent's stochastic policy is shown to more accurately capture the diversity\nin the goal distribution than a fixed pixel-based reward function baseline. We\ndemonstrate the efficacy of our approach on two domains: (1) drawing MNIST\ndigits with a paint software conditioned on instructions and (2) constructing\nscenes in a 3D editor that satisfies a certain instruction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.01245v2",
            "date": "2018-12-04 06:42:49+00:00",
            "title": "Transferable Natural Language Interface to Structured Queries aided by Adversarial Generation",
            "authors": [
                "Hongyu Xiong",
                "Ruixiao Sun"
            ],
            "abstract": "A natural language interface (NLI) to structured query is intriguing due to\nits wide industrial applications and high economical values. In this work, we\ntackle the problem of domain adaptation for NLI with limited data on target\ndomain. Two important approaches are considered: (a) effective\ngeneral-knowledge-learning on source domain semantic parsing, and (b) data\naugmentation on target domain. We present a Structured Query Inference Network\n(SQIN) to enhance learning for domain adaptation, by separating schema\ninformation from NL and decoding SQL in a more structural-aware manner; we also\npropose a GAN-based augmentation technique (AugmentGAN) to mitigate the issue\nof lacking target domain data. We report solid results on GeoQuery, Overnight,\nand WikiSQL to demonstrate state-of-the-art performances for both in-domain and\ndomain-transfer tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01525v1",
            "date": "2018-12-04 16:55:25+00:00",
            "title": "A Face-to-Face Neural Conversation Model",
            "authors": [
                "Hang Chu",
                "Daiqing Li",
                "Sanja Fidler"
            ],
            "abstract": "Neural networks have recently become good at engaging in dialog. However,\ncurrent approaches are based solely on verbal text, lacking the richness of a\nreal face-to-face conversation. We propose a neural conversation model that\naims to read and generate facial gestures alongside with text. This allows our\nmodel to adapt its response based on the \"mood\" of the conversation. In\nparticular, we introduce an RNN encoder-decoder that exploits the movement of\nfacial muscles, as well as the verbal conversation. The decoder consists of two\nlayers, where the lower layer aims at generating the verbal response and coarse\nfacial expressions, while the second layer fills in the subtle gestures, making\nthe generated output more smooth and natural. We train our neural network by\nhaving it \"watch\" 250 movies. We showcase our joint face-text model in\ngenerating more natural conversations through automatic metrics and a human\nstudy. We demonstrate an example application with a face-to-face chatting\navatar.",
            "journal ref": "CVPR (2018) 7113-7121",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.01628v2",
            "date": "2018-12-04 19:06:00+00:00",
            "title": "Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Mark O. Riedl"
            ],
            "abstract": "Text-based adventure games provide a platform on which to explore\nreinforcement learning in the context of a combinatorial action space, such as\nnatural language. We present a deep reinforcement learning architecture that\nrepresents the game state as a knowledge graph which is learned during\nexploration. This graph is used to prune the action space, enabling more\nefficient exploration. The question of which action to take can be reduced to a\nquestion-answering task, a form of transfer learning that pre-trains certain\nparts of our architecture. In experiments using the TextWorld framework, we\nshow that our proposed technique can learn a control policy faster than\nbaseline alternatives. We have also open-sourced our code at\nhttps://github.com/rajammanabrolu/KG-DQN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.01969v2",
            "date": "2018-12-05 13:00:04+00:00",
            "title": "Summarizing Videos with Attention",
            "authors": [
                "Jiri Fajtl",
                "Hajar Sadeghi Sokeh",
                "Vasileios Argyriou",
                "Dorothy Monekosso",
                "Paolo Remagnino"
            ],
            "abstract": "In this work we propose a novel method for supervised, keyshots based video\nsummarization by applying a conceptually simple and computationally efficient\nsoft, self-attention mechanism. Current state of the art methods leverage\nbi-directional recurrent networks such as BiLSTM combined with attention. These\nnetworks are complex to implement and computationally demanding compared to\nfully connected networks. To that end we propose a simple, self-attention based\nnetwork for video summarization which performs the entire sequence to sequence\ntransformation in a single feed forward pass and single backward pass during\ntraining. Our method sets a new state of the art results on two benchmarks\nTvSum and SumMe, commonly used in this domain.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.02303v4",
            "date": "2018-12-05 04:06:27+00:00",
            "title": "Neural Abstractive Text Summarization with Sequence-to-Sequence Models",
            "authors": [
                "Tian Shi",
                "Yaser Keneshloo",
                "Naren Ramakrishnan",
                "Chandan K. Reddy"
            ],
            "abstract": "In the past few years, neural abstractive text summarization with\nsequence-to-sequence (seq2seq) models have gained a lot of popularity. Many\ninteresting techniques have been proposed to improve seq2seq models, making\nthem capable of handling different challenges, such as saliency, fluency and\nhuman readability, and generate high-quality summaries. Generally speaking,\nmost of these techniques differ in one of these three categories: network\nstructure, parameter inference, and decoding/generation. There are also other\nconcerns, such as efficiency and parallelism for training a model. In this\npaper, we provide a comprehensive literature survey on different seq2seq models\nfor abstractive text summarization from the viewpoint of network structures,\ntraining strategies, and summary generation algorithms. Several models were\nfirst proposed for language modeling and generation tasks, such as machine\ntranslation, and later applied to abstractive text summarization. Hence, we\nalso provide a brief review of these models. As part of this survey, we also\ndevelop an open source library, namely, Neural Abstractive Text Summarizer\n(NATS) toolkit, for the abstractive text summarization. An extensive set of\nexperiments have been conducted on the widely used CNN/Daily Mail dataset to\nexamine the effectiveness of several different neural network components.\nFinally, we benchmark two models implemented in NATS on the two recently\nreleased datasets, namely, Newsroom and Bytecup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02307v4",
            "date": "2018-11-29 23:33:59+00:00",
            "title": "EvoMSA: A Multilingual Evolutionary Approach for Sentiment Analysis",
            "authors": [
                "Mario Graff",
                "Sabino Miranda-Jim\u00e9nez",
                "Eric S. Tellez",
                "Daniela Moctezuma"
            ],
            "abstract": "Sentiment analysis (SA) is a task related to understanding people's feelings\nin written text; the starting point would be to identify the polarity level\n(positive, neutral or negative) of a given text, moving on to identify emotions\nor whether a text is humorous or not. This task has been the subject of several\nresearch competitions in a number of languages, e.g., English, Spanish, and\nArabic, among others. In this contribution, we propose an SA system, namely\nEvoMSA, that unifies our participating systems in various SA competitions,\nmaking it domain independent and multilingual by processing text using only\nlanguage-independent techniques. EvoMSA is a classifier, based on Genetic\nProgramming, that works by combining the output of different text classifiers\nand text models to produce the final prediction. We analyze EvoMSA on different\nSA competitions to provide a global overview of its performance, and as the\nresults show, EvoMSA is competitive obtaining top rankings in several SA\ncompetitions. Furthermore, we performed an analysis of EvoMSA's components to\nmeasure their contribution to the performance; the idea is to facilitate a\npractitioner or newcomer to implement a competitive SA classifier. Finally, it\nis worth to mention that EvoMSA is available as open-source software.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02308v1",
            "date": "2018-11-28 10:37:24+00:00",
            "title": "On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition",
            "authors": [
                "Jan Kremer",
                "Lasse Borgholt",
                "Lars Maal\u00f8e"
            ],
            "abstract": "End-to-end automatic speech recognition (ASR) commonly transcribes audio\nsignals into sequences of characters while its performance is evaluated by\nmeasuring the word-error rate (WER). This suggests that predicting sequences of\nwords directly may be helpful instead. However, training with word-level\nsupervision can be more difficult due to the sparsity of examples per label\nclass. In this paper we analyze an end-to-end ASR model that combines a\nword-and-character representation in a multi-task learning (MTL) framework. We\nshow that it improves on the WER and study how the word-level model can benefit\nfrom character-level supervision by analyzing the learned inductive preference\nbias of each model component empirically. We find that by adding\ncharacter-level supervision, the MTL model interpolates between recognizing\nmore frequent words (preferred by the word-level model) and shorter words\n(preferred by the character-level model).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.02455v1",
            "date": "2018-12-06 10:57:29+00:00",
            "title": "The USTC-NEL Speech Translation system at IWSLT 2018",
            "authors": [
                "Dan Liu",
                "Junhua Liu",
                "Wu Guo",
                "Shifu Xiong",
                "Zhiqiang Ma",
                "Rui Song",
                "Chongliang Wu",
                "Quan Liu"
            ],
            "abstract": "This paper describes the USTC-NEL system to the speech translation task of\nthe IWSLT Evaluation 2018. The system is a conventional pipeline system which\ncontains 3 modules: speech recognition, post-processing and machine\ntranslation. We train a group of hybrid-HMM models for our speech recognition,\nand for machine translation we train transformer based neural machine\ntranslation models with speech recognition output style text as input.\nExperiments conducted on the IWSLT 2018 task indicate that, compared to\nbaseline system from KIT, our system achieved 14.9 BLEU improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03258v1",
            "date": "2018-12-08 03:17:29+00:00",
            "title": "Political Popularity Analysis in Social Media",
            "authors": [
                "Amir Karami",
                "Aida Elkouri"
            ],
            "abstract": "Popularity is a critical success factor for a politician and her/his party to\nwin in elections and implement their plans. Finding the reasons behind the\npopularity can provide a stable political movement. This research attempts to\nmeasure popularity in Twitter using a mixed method. In recent years, Twitter\ndata has provided an excellent opportunity for exploring public opinions by\nanalyzing a large number of tweets. This study has collected and examined 4.5\nmillion tweets related to a US politician, Senator Bernie Sanders. This study\ninvestigated eight economic reasons behind the senator's popularity in Twitter.\nThis research has benefits for politicians, informatics experts, and\npolicymakers to explore public opinion. The collected data will also be\navailable for further investigation.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1812.03260v1",
            "date": "2018-12-08 03:21:26+00:00",
            "title": "An Exploratory Study of (#)Exercise in the Twittersphere",
            "authors": [
                "George Shaw",
                "Amir Karami"
            ],
            "abstract": "Social media analytics allows us to extract, analyze, and establish semantic\nfrom user-generated contents in social media platforms. This study utilized a\nmixed method including a three-step process of data collection, topic modeling,\nand data annotation for recognizing exercise related patterns. Based on the\nfindings, 86% of the detected topics were identified as meaningful topics after\nconducting the data annotation process. The most discussed exercise-related\ntopics were physical activity (18.7%), lifestyle behaviors (6.6%), and dieting\n(4%). The results from our experiment indicate that the exploratory data\nanalysis is a practical approach to summarizing the various characteristics of\ntext data for different health and medical applications.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1812.03509v1",
            "date": "2018-12-09 16:05:43+00:00",
            "title": "Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning",
            "authors": [
                "Ziming Li",
                "Julia Kiseleva",
                "Maarten de Rijke"
            ],
            "abstract": "The performance of adversarial dialogue generation models relies on the\nquality of the reward signal produced by the discriminator. The reward signal\nfrom a poor discriminator can be very sparse and unstable, which may lead the\ngenerator to fall into a local optimum or to produce nonsense replies. To\nalleviate the first problem, we first extend a recently proposed adversarial\ndialogue generation method to an adversarial imitation learning solution. Then,\nin the framework of adversarial inverse reinforcement learning, we propose a\nnew reward model for dialogue generation that can provide a more accurate and\nprecise reward signal for generator training. We evaluate the performance of\nthe resulting model with automatic metrics and human evaluations in two\nannotation settings. Our experimental results demonstrate that our model can\ngenerate more high-quality responses and achieve higher overall performance\nthan the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03519v1",
            "date": "2018-12-09 16:44:56+00:00",
            "title": "Deep-Net: Deep Neural Network for Cyber Security Use Cases",
            "authors": [
                "Vinayakumar R",
                "Barathi Ganesh HB",
                "Prabaharan Poornachandran",
                "Anand Kumar M",
                "Soman KP"
            ],
            "abstract": "Deep neural networks (DNNs) have witnessed as a powerful approach in this\nyear by solving long-standing Artificial intelligence (AI) supervised and\nunsupervised tasks exists in natural language processing, speech processing,\ncomputer vision and others. In this paper, we attempt to apply DNNs on three\ndifferent cyber security use cases: Android malware classification, incident\ndetection and fraud detection. The data set of each use case contains real\nknown benign and malicious activities samples. The efficient network\narchitecture for DNN is chosen by conducting various trails of experiments for\nnetwork parameters and network structures. The experiments of such chosen\nefficient configurations of DNNs are run up to 1000 epochs with learning rate\nset in the range [0.01-0.5]. Experiments of DNN performed well in comparison to\nthe classical machine learning algorithms in all cases of experiments of cyber\nsecurity use cases. This is due to the fact that DNNs implicitly extract and\nbuild better features, identifies the characteristics of the data that lead to\nbetter accuracy. The best accuracy obtained by DNN and XGBoost on Android\nmalware classification 0.940 and 0.741, incident detection 1.00 and 0.997 fraud\ndetection 0.972 and 0.916 respectively.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.03919v2",
            "date": "2018-12-10 16:57:21+00:00",
            "title": "Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settings",
            "authors": [
                "Matthew Wiesner",
                "Adithya Renduchintala",
                "Shinji Watanabe",
                "Chunxi Liu",
                "Najim Dehak",
                "Sanjeev Khudanpur"
            ],
            "abstract": "We explore training attention-based encoder-decoder ASR in low-resource\nsettings. These models perform poorly when trained on small amounts of\ntranscribed speech, in part because they depend on having sufficient\ntarget-side text to train the attention and decoder networks. In this paper we\naddress this shortcoming by pretraining our network parameters using only\ntext-based data and transcribed speech from other languages. We analyze the\nrelative contributions of both sources of data. Across 3 test languages, our\ntext-based approach resulted in a 20% average relative improvement over a\ntext-based augmentation technique without pretraining. Using transcribed speech\nfrom nearby languages gives a further 20-30% relative reduction in character\nerror rate.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.04013v1",
            "date": "2018-12-10 19:00:09+00:00",
            "title": "L\u00e9vy Flights of the Collective Imagination",
            "authors": [
                "William H. W. Thompson",
                "Zachary Wojtowicz",
                "Simon DeDeo"
            ],
            "abstract": "We present a structured random-walk model that captures key aspects of how\npeople communicate in groups. Our model takes the form of a correlated L\\'{e}vy\nflight that quantifies the balance between focused discussion of an idea and\nlong-distance leaps in semantic space. We apply our model to three cases of\nincreasing structural complexity: philosophical texts by Aristotle, Hume, and\nKant; four days of parliamentary debate during the French Revolution; and\nbranching comment trees on the discussion website Reddit. In the philosophical\nand parliamentary cases, the model parameters that describe this balance\nconverge under coarse-graining to limit regions that demonstrate the emergence\nof large-scale structure, a result which is robust to translation between\nlanguages. Meanwhile, we find that the political forum we consider on Reddit\nexhibits a debate-like pattern, while communities dedicated to the discussion\nof science and news show much less temporal order, and may make use of the\nemergent, tree-like topology of comment replies to structure their epistemic\nexplorations. Our model allows us to quantify the ways in which social\ntechnologies such as parliamentary procedures and online commenting systems\nshape the joint exploration of ideas.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1812.04224v1",
            "date": "2018-12-11 05:25:38+00:00",
            "title": "On the Dimensionality of Word Embedding",
            "authors": [
                "Zi Yin",
                "Yuanyuan Shen"
            ],
            "abstract": "In this paper, we provide a theoretical understanding of word embedding and\nits dimensionality. Motivated by the unitary-invariance of word embedding, we\npropose the Pairwise Inner Product (PIP) loss, a novel metric on the\ndissimilarity between word embeddings. Using techniques from matrix\nperturbation theory, we reveal a fundamental bias-variance trade-off in\ndimensionality selection for word embeddings. This bias-variance trade-off\nsheds light on many empirical observations which were previously unexplained,\nfor example the existence of an optimal dimensionality. Moreover, new insights\nand discoveries, like when and how word embeddings are robust to over-fitting,\nare revealed. By optimizing over the bias-variance trade-off of the PIP loss,\nwe can explicitly answer the open question of dimensionality selection for word\nembedding.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.04238v1",
            "date": "2018-12-11 07:04:44+00:00",
            "title": "Machine Translation : From Statistical to modern Deep-learning practices",
            "authors": [
                "Siddhant Srivastava",
                "Anupam Shukla",
                "Ritu Tiwari"
            ],
            "abstract": "Machine translation (MT) is an area of study in Natural Language processing\nwhich deals with the automatic translation of human language, from one language\nto another by the computer. Having a rich research history spanning nearly\nthree decades, Machine translation is one of the most sought after area of\nresearch in the linguistics and computational community. In this paper, we\ninvestigate the models based on deep learning that have achieved substantial\nprogress in recent years and becoming the prominent method in MT. We shall\ndiscuss the two main deep-learning based Machine Translation methods, one at\ncomponent or domain level which leverages deep learning models to enhance the\nefficacy of Statistical Machine Translation (SMT) and end-to-end deep learning\nmodels in MT which uses neural networks to find correspondence between the\nsource and target languages using the encoder-decoder architecture. We conclude\nthis paper by providing a time line of the major research problems solved by\nthe researchers and also provide a comprehensive overview of present areas of\nresearch in Neural Machine Translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04342v2",
            "date": "2018-12-11 12:00:06+00:00",
            "title": "Learning latent representations for style control and transfer in end-to-end speech synthesis",
            "authors": [
                "Ya-Jie Zhang",
                "Shifeng Pan",
                "Lei He",
                "Zhen-Hua Ling"
            ],
            "abstract": "In this paper, we introduce the Variational Autoencoder (VAE) to an\nend-to-end speech synthesis model, to learn the latent representation of\nspeaking styles in an unsupervised manner. The style representation learned\nthrough VAE shows good properties such as disentangling, scaling, and\ncombination, which makes it easy for style control. Style transfer can be\nachieved in this framework by first inferring style representation through the\nrecognition network of VAE, then feeding it into TTS network to guide the style\nin synthesizing speech. To avoid Kullback-Leibler (KL) divergence collapse in\ntraining, several techniques are adopted. Finally, the proposed model shows\ngood performance of style control and outperforms Global Style Token (GST)\nmodel in ABX preference tests on style transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04606v3",
            "date": "2018-12-11 18:49:50+00:00",
            "title": "Deep Anomaly Detection with Outlier Exposure",
            "authors": [
                "Dan Hendrycks",
                "Mantas Mazeika",
                "Thomas Dietterich"
            ],
            "abstract": "It is important to detect anomalous inputs when deploying machine learning\nsystems. The use of larger and more complex inputs in deep learning magnifies\nthe difficulty of distinguishing between anomalous and in-distribution\nexamples. At the same time, diverse image and text data are available in\nenormous quantities. We propose leveraging these data to improve deep anomaly\ndetection by training anomaly detectors against an auxiliary dataset of\noutliers, an approach we call Outlier Exposure (OE). This enables anomaly\ndetectors to generalize and detect unseen anomalies. In extensive experiments\non natural language processing and small- and large-scale vision tasks, we find\nthat Outlier Exposure significantly improves detection performance. We also\nobserve that cutting-edge generative models trained on CIFAR-10 may assign\nhigher likelihoods to SVHN images than to CIFAR-10 images; we use OE to\nmitigate this issue. We also analyze the flexibility and robustness of Outlier\nExposure, and identify characteristics of the auxiliary dataset that improve\nperformance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.04616v3",
            "date": "2018-12-10 20:00:36+00:00",
            "title": "Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs",
            "authors": [
                "Sachin Kumar",
                "Yulia Tsvetkov"
            ],
            "abstract": "The Softmax function is used in the final layer of nearly all existing\nsequence-to-sequence models for language generation. However, it is usually the\nslowest layer to compute which limits the vocabulary size to a subset of most\nfrequent types; and it has a large memory footprint. We propose a general\ntechnique for replacing the softmax layer with a continuous embedding layer.\nOur primary innovations are a novel probabilistic loss, and a training and\ninference procedure in which we generate a probability distribution over\npre-trained word embeddings, instead of a multinomial distribution over the\nvocabulary obtained via softmax. We evaluate this new class of\nsequence-to-sequence models with continuous outputs on the task of neural\nmachine translation. We show that our models obtain upto 2.5x speed-up in\ntraining time while performing on par with the state-of-the-art models in terms\nof translation quality. These models are capable of handling very large\nvocabularies without compromising on translation quality. They also produce\nmore meaningful errors than in the softmax-based models, as these errors\ntypically lie in a subspace of the vector space of the reference translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.04722v1",
            "date": "2018-12-06 03:16:06+00:00",
            "title": "Context is Key: New Approaches to Neural Coherence Modeling",
            "authors": [
                "David McClure",
                "Shayne O'Brien",
                "Deb Roy"
            ],
            "abstract": "We formulate coherence modeling as a regression task and propose two novel\nmethods to combine techniques from our setup with pairwise approaches. The\nfirst of our methods is a model that we call \"first-next,\" which operates\nsimilarly to selection sorting but conditions decision-making on information\nabout already-sorted sentences. The second consists of a technique for adding\ncontext to regression-based models by concatenating sentence-level\nrepresentations with an encoding of its corresponding out-of-order paragraph.\nThis latter model achieves Kendall-tau distance and positional accuracy scores\nthat match or exceed the current state-of-the-art on these metrics. Our results\nsuggest that many of the gains that come from more complex, machine-translation\ninspired approaches can be achieved with simpler, more efficient models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.05083v2",
            "date": "2018-12-12 18:44:23+00:00",
            "title": "Adversarial Learning of Semantic Relevance in Text to Image Synthesis",
            "authors": [
                "Miriam Cha",
                "Youngjune L. Gwon",
                "H. T. Kung"
            ],
            "abstract": "We describe a new approach that improves the training of generative\nadversarial nets (GANs) for synthesizing diverse images from a text input. Our\napproach is based on the conditional version of GANs and expands on previous\nwork leveraging an auxiliary task in the discriminator. Our generated images\nare not limited to certain classes and do not suffer from mode collapse while\nsemantically matching the text input. A key to our training methods is how to\nform positive and negative training examples with respect to the class label of\na given image. Instead of selecting random training examples, we perform\nnegative sampling based on the semantic distance from a positive example in the\nclass. We evaluate our approach using the Oxford-102 flower dataset, adopting\nthe inception score and multi-scale structural similarity index (MS-SSIM)\nmetrics to assess discriminability and diversity of the generated images. The\nempirical results indicate greater diversity in the generated images,\nespecially when we gradually select more negative training examples closer to a\npositive example in the semantic space.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.05253v4",
            "date": "2018-12-13 03:41:58+00:00",
            "title": "Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice",
            "authors": [
                "Yan Deng",
                "Lei He",
                "Frank Soong"
            ],
            "abstract": "Neural TTS has shown it can generate high quality synthesized speech. In this\npaper, we investigate the multi-speaker latent space to improve neural TTS for\nadapting the system to new speakers with only several minutes of speech or\nenhancing a premium voice by utilizing the data from other speakers for richer\ncontextual coverage and better generalization. A multi-speaker neural TTS model\nis built with the embedded speaker information in both spectral and speaker\nlatent space. The experimental results show that, with less than 5 minutes of\ntraining data from a new speaker, the new model can achieve an MOS score of\n4.16 in naturalness and 4.64 in speaker similarity close to human recordings\n(4.74). For a well-trained premium voice, we can achieve an MOS score of 4.5\nfor out-of-domain texts, which is comparable to an MOS of 4.58 for professional\nrecordings, and significantly outperforms single speaker result of 4.28.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.05271v1",
            "date": "2018-12-13 05:32:43+00:00",
            "title": "TextBugger: Generating Adversarial Text Against Real-world Applications",
            "authors": [
                "Jinfeng Li",
                "Shouling Ji",
                "Tianyu Du",
                "Bo Li",
                "Ting Wang"
            ],
            "abstract": "Deep Learning-based Text Understanding (DLTU) is the backbone technique\nbehind various applications, including question answering, machine translation,\nand text classification. Despite its tremendous popularity, the security\nvulnerabilities of DLTU are still largely unknown, which is highly concerning\ngiven its increasing use in security-sensitive applications such as sentiment\nanalysis and toxic content detection. In this paper, we show that DLTU is\ninherently vulnerable to adversarial text attacks, in which maliciously crafted\ntexts trigger target DLTU systems and services to misbehave. Specifically, we\npresent TextBugger, a general attack framework for generating adversarial\ntexts. In contrast to prior works, TextBugger differs in significant ways: (i)\neffective -- it outperforms state-of-the-art attacks in terms of attack success\nrate; (ii) evasive -- it preserves the utility of benign text, with 94.9\\% of\nthe adversarial text correctly recognized by human readers; and (iii) efficient\n-- it generates adversarial text with computational complexity sub-linear to\nthe text length. We empirically evaluate TextBugger on a set of real-world DLTU\nsystems and services used for sentiment analysis and toxic content detection,\ndemonstrating its effectiveness, evasiveness, and efficiency. For instance,\nTextBugger achieves 100\\% success rate on the IMDB dataset based on Amazon AWS\nComprehend within 4.61 seconds and preserves 97\\% semantic similarity. We\nfurther discuss possible defense mechanisms to mitigate such attack and the\nadversary's potential countermeasures, which leads to promising directions for\nfurther research.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1812.05288v4",
            "date": "2018-12-13 07:02:54+00:00",
            "title": "Dynamic Transfer Learning for Named Entity Recognition",
            "authors": [
                "Parminder Bhatia",
                "Kristjan Arumae",
                "Busra Celikkaya"
            ],
            "abstract": "State-of-the-art named entity recognition (NER) systems have been improving\ncontinuously using neural architectures over the past several years. However,\nmany tasks including NER require large sets of annotated data to achieve such\nperformance. In particular, we focus on NER from clinical notes, which is one\nof the most fundamental and critical problems for medical text analysis. Our\nwork centers on effectively adapting these neural architectures towards\nlow-resource settings using parameter transfer methods. We complement a\nstandard hierarchical NER model with a general transfer learning framework\nconsisting of parameter sharing between the source and target tasks, and\nshowcase scores significantly above the baseline architecture. These sharing\nschemes require an exponential search over tied parameter sets to generate an\noptimal configuration. To mitigate the problem of exhaustively searching for\nmodel optimization, we propose the Dynamic Transfer Networks (DTN), a gated\narchitecture which learns the appropriate parameter sharing scheme between\nsource and target datasets. DTN achieves the improvements of the optimized\ntransfer learning framework with just a single training setting, effectively\nremoving the need for exponential search.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.05692v1",
            "date": "2018-12-12 14:32:16+00:00",
            "title": "Bayesian Sparsification of Gated Recurrent Neural Networks",
            "authors": [
                "Ekaterina Lobacheva",
                "Nadezhda Chirkova",
                "Dmitry Vetrov"
            ],
            "abstract": "Bayesian methods have been successfully applied to sparsify weights of neural\nnetworks and to remove structure units from the networks, e. g. neurons. We\napply and further develop this approach for gated recurrent architectures.\nSpecifically, in addition to sparsification of individual weights and neurons,\nwe propose to sparsify preactivations of gates and information flow in LSTM. It\nmakes some gates and information flow components constant, speeds up forward\npass and improves compression. Moreover, the resulting structure of gate\nsparsity is interpretable and depends on the task. Code is available on github:\nhttps://github.com/tipt0p/SparseBayesianRNN",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.05920v2",
            "date": "2018-12-13 16:01:11+00:00",
            "title": "Speech and Speaker Recognition from Raw Waveform with SincNet",
            "authors": [
                "Mirco Ravanelli",
                "Yoshua Bengio"
            ],
            "abstract": "Deep neural networks can learn complex and abstract representations, that are\nprogressively obtained by combining simpler ones. A recent trend in speech and\nspeaker recognition consists in discovering these representations starting from\nraw audio samples directly. Differently from standard hand-crafted features\nsuch as MFCCs or FBANK, the raw waveform can potentially help neural networks\ndiscover better and more customized representations. The high-dimensional raw\ninputs, however, can make training significantly more challenging. This paper\nsummarizes our recent efforts to develop a neural architecture that efficiently\nprocesses speech from audio waveforms. In particular, we propose SincNet, a\nnovel Convolutional Neural Network (CNN) that encourages the first layer to\ndiscover meaningful filters by exploiting parametrized sinc functions. In\ncontrast to standard CNNs, which learn all the elements of each filter, only\nlow and high cutoff frequencies of band-pass filters are directly learned from\ndata. This inductive bias offers a very compact way to derive a customized\nfront-end, that only depends on some parameters with a clear physical meaning.\nOur experiments, conducted on both speaker and speech recognition, show that\nthe proposed architecture converges faster, performs better, and is more\ncomputationally efficient than standard CNNs.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.06038v1",
            "date": "2018-12-14 17:28:20+00:00",
            "title": "Inferring the size of the causal universe: features and fusion of causal attribution networks",
            "authors": [
                "Daniel Berenberg",
                "James P. Bagrow"
            ],
            "abstract": "Cause-and-effect reasoning, the attribution of effects to causes, is one of\nthe most powerful and unique skills humans possess. Multiple surveys are\nmapping out causal attributions as networks, but it is unclear how well these\nefforts can be combined. Further, the total size of the collective causal\nattribution network held by humans is currently unknown, making it challenging\nto assess the progress of these surveys. Here we study three causal attribution\nnetworks to determine how well they can be combined into a single network.\nCombining these networks requires dealing with ambiguous nodes, as nodes\nrepresent written descriptions of causes and effects and different descriptions\nmay exist for the same concept. We introduce NetFUSES, a method for combining\nnetworks with ambiguous nodes. Crucially, treating the different causal\nattributions networks as independent samples allows us to use their overlap to\nestimate the total size of the collective causal attribution network. We find\nthat existing surveys capture 5.77% $\\pm$ 0.781% of the $\\approx$293 000 causes\nand effects estimated to exist, and 0.198% $\\pm$ 0.174% of the $\\approx$10 200\n000 attributed cause-effect relationships.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1812.06083v1",
            "date": "2018-12-13 22:23:51+00:00",
            "title": "Coupled Representation Learning for Domains, Intents and Slots in Spoken Language Understanding",
            "authors": [
                "JIhwan Lee",
                "Dongchan Kim",
                "Ruhi Sarikaya",
                "Young-Bum Kim"
            ],
            "abstract": "Representation learning is an essential problem in a wide range of\napplications and it is important for performing downstream tasks successfully.\nIn this paper, we propose a new model that learns coupled representations of\ndomains, intents, and slots by taking advantage of their hierarchical\ndependency in a Spoken Language Understanding system. Our proposed model learns\nthe vector representation of intents based on the slots tied to these intents\nby aggregating the representations of the slots. Similarly, the vector\nrepresentation of a domain is learned by aggregating the representations of the\nintents tied to a specific domain. To the best of our knowledge, it is the\nfirst approach to jointly learning the representations of domains, intents, and\nslots using their hierarchical relationships. The experimental results\ndemonstrate the effectiveness of the representations learned by our model, as\nevidenced by improved performance on the contextual cross-domain reranking\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06158v1",
            "date": "2018-12-14 20:39:47+00:00",
            "title": "Few-shot classification in Named Entity Recognition Task",
            "authors": [
                "Alexander Fritzler",
                "Varvara Logacheva",
                "Maksim Kretov"
            ],
            "abstract": "For many natural language processing (NLP) tasks the amount of annotated data\nis limited. This urges a need to apply semi-supervised learning techniques,\nsuch as transfer learning or meta-learning. In this work we tackle Named Entity\nRecognition (NER) task using Prototypical Network - a metric learning\ntechnique. It learns intermediate representations of words which cluster well\ninto named entity classes. This property of the model allows classifying words\nwith extremely limited number of training examples, and can potentially be used\nas a zero-shot learning method. By coupling this technique with transfer\nlearning we achieve well-performing classifiers trained on only 20 instances of\na target class.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06199v1",
            "date": "2018-12-14 23:03:41+00:00",
            "title": "Inter-sentence Relation Extraction for Associating Biological Context with Events in Biomedical Texts",
            "authors": [
                "Enrique Noriega-Atala",
                "Paul D. Hein",
                "Shraddha S. Thumsi",
                "Zechy Wong",
                "Xia Wang",
                "Clayton T. Morrison"
            ],
            "abstract": "We present an analysis of the problem of identifying biological context and\nassociating it with biochemical events in biomedical texts. This constitutes a\nnon-trivial, inter-sentential relation extraction task. We focus on biological\ncontext as descriptions of the species, tissue type and cell type that are\nassociated with biochemical events. We describe the properties of an annotated\ncorpus of context-event relations and present and evaluate several classifiers\nfor context-event association trained on syntactic, distance and frequency\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06401v1",
            "date": "2018-12-16 06:12:45+00:00",
            "title": "What's to know? Uncertainty as a Guide to Asking Goal-oriented Questions",
            "authors": [
                "Ehsan Abbasnejad",
                "Qi Wu",
                "Javen Shi",
                "Anton van den Hengel"
            ],
            "abstract": "One of the core challenges in Visual Dialogue problems is asking the question\nthat will provide the most useful information towards achieving the required\nobjective. Encouraging an agent to ask the right questions is difficult because\nwe don't know a-priori what information the agent will need to achieve its\ntask, and we don't have an explicit model of what it knows already. We propose\na solution to this problem based on a Bayesian model of the uncertainty in the\nimplicit model maintained by the visual dialogue agent, and in the function\nused to select an appropriate output. By selecting the question that minimises\nthe predicted regret with respect to this implicit model the agent actively\nreduces ambiguity. The Bayesian model of uncertainty also enables a principled\nmethod for identifying when enough information has been acquired, and an action\nshould be selected. We evaluate our approach on two goal-oriented dialogue\ndatasets, one for visual-based collaboration task and the other for a\nnegotiation-based task. Our uncertainty-aware information-seeking model\noutperforms its counterparts in these two challenging problems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.06410v2",
            "date": "2018-12-16 07:32:02+00:00",
            "title": "NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding",
            "authors": [
                "Yongqi Zhang",
                "Quanming Yao",
                "Yingxia Shao",
                "Lei Chen"
            ],
            "abstract": "Knowledge Graph (KG) embedding is a fundamental problem in data mining\nresearch with many real-world applications. It aims to encode the entities and\nrelations in the graph into low dimensional vector space, which can be used for\nsubsequent algorithms. Negative sampling, which samples negative triplets from\nnon-observed ones in the training data, is an important step in KG embedding.\nRecently, generative adversarial network (GAN), has been introduced in negative\nsampling. By sampling negative triplets with large scores, these methods avoid\nthe problem of vanishing gradient and thus obtain better performance. However,\nusing GAN makes the original model more complex and hard to train, where\nreinforcement learning must be used. In this paper, motivated by the\nobservation that negative triplets with large scores are important but rare, we\npropose to directly keep track of them with the cache. However, how to sample\nfrom and update the cache are two important questions. We carefully design the\nsolutions, which are not only efficient but also achieve a good balance between\nexploration and exploitation. In this way, our method acts as a \"distilled\"\nversion of previous GA-based methods, which does not waste training time on\nadditional parameters to fit the full distribution of negative triplets. The\nextensive experiments show that our method can gain significant improvement in\nvarious KG embedding models, and outperform the state-of-the-art negative\nsampling methods based on GAN.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.06417v3",
            "date": "2018-12-16 08:18:37+00:00",
            "title": "Visual Dialogue without Vision or Dialogue",
            "authors": [
                "Daniela Massiceti",
                "Puneet K. Dokania",
                "N. Siddharth",
                "Philip H. S. Torr"
            ],
            "abstract": "We characterise some of the quirks and shortcomings in the exploration of\nVisual Dialogue - a sequential question-answering task where the questions and\ncorresponding answers are related through given visual stimuli. To do so, we\ndevelop an embarrassingly simple method based on Canonical Correlation Analysis\n(CCA) that, on the standard dataset, achieves near state-of-the-art performance\non mean rank (MR). In direct contrast to current complex and over-parametrised\narchitectures that are both compute and time intensive, our method ignores the\nvisual stimuli, ignores the sequencing of dialogue, does not need gradients,\nuses off-the-shelf feature extractors, has at least an order of magnitude fewer\nparameters, and learns in practically no time. We argue that these results are\nindicative of issues in current approaches to Visual Dialogue and conduct\nanalyses to highlight implicit dataset biases and effects of over-constrained\nevaluation metrics. Our code is publicly available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.06705v1",
            "date": "2018-12-17 11:26:42+00:00",
            "title": "Conditional BERT Contextual Augmentation",
            "authors": [
                "Xing Wu",
                "Shangwen Lv",
                "Liangjun Zang",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "We propose a novel data augmentation method for labeled sentences called\nconditional BERT contextual augmentation. Data augmentation methods are often\napplied to prevent overfitting and improve generalization of deep neural\nnetwork models. Recently proposed contextual augmentation augments labeled\nsentences by randomly replacing words with more varied substitutions predicted\nby language model. BERT demonstrates that a deep bidirectional language model\nis more powerful than either an unidirectional language model or the shallow\nconcatenation of a forward and backward model. We retrofit BERT to conditional\nBERT by introducing a new conditional masked language model\\footnote{The term\n\"conditional masked language model\" appeared once in original BERT paper, which\nindicates context-conditional, is equivalent to term \"masked language model\".\nIn our paper, \"conditional masked language model\" indicates we apply extra\nlabel-conditional constraint to the \"masked language model\".} task. The well\ntrained conditional BERT can be applied to enhance contextual augmentation.\nExperiments on six various different text classification tasks show that our\nmethod can be easily applied to both convolutional or recurrent neural networks\nclassifier to obtain obvious improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06722v1",
            "date": "2018-12-17 11:59:53+00:00",
            "title": "TechKG: A Large-Scale Chinese Technology-Oriented Knowledge Graph",
            "authors": [
                "Feiliang Ren",
                "Yining Hou",
                "Yan Li",
                "Linfeng Pan",
                "Yi Zhang",
                "Xiaobo Liang",
                "Yongkang Liu",
                "Yu Guo",
                "Rongsheng Zhao",
                "Ruicheng Ming",
                "Huiming Wu"
            ],
            "abstract": "Knowledge graph is a kind of valuable knowledge base which would benefit lots\nof AI-related applications. Up to now, lots of large-scale knowledge graphs\nhave been built. However, most of them are non-Chinese and designed for general\npurpose. In this work, we introduce TechKG, a large scale Chinese knowledge\ngraph that is technology-oriented. It is built automatically from massive\ntechnical papers that are published in Chinese academic journals of different\nresearch domains. Some carefully designed heuristic rules are used to extract\nhigh quality entities and relations. Totally, it comprises of over 260 million\ntriplets that are built upon more than 52 million entities which come from 38\nresearch domains. Our preliminary ex-periments indicate that TechKG has high\nadaptability and can be used as a dataset for many diverse AI-related\napplications. We released TechKG at: http://www.techkg.cn.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.06834v3",
            "date": "2018-12-17 15:26:29+00:00",
            "title": "A Tutorial on Deep Latent Variable Models of Natural Language",
            "authors": [
                "Yoon Kim",
                "Sam Wiseman",
                "Alexander M. Rush"
            ],
            "abstract": "There has been much recent, exciting work on combining the complementary\nstrengths of latent variable models and deep learning. Latent variable modeling\nmakes it easy to explicitly specify model constraints through conditional\nindependence properties, while deep learning makes it possible to parameterize\nthese conditional likelihoods with powerful function approximators. While these\n\"deep latent variable\" models provide a rich, flexible framework for modeling\nmany real-world phenomena, difficulties exist: deep parameterizations of\nconditional likelihoods usually make posterior inference intractable, and\nlatent variable objectives often complicate backpropagation by introducing\npoints of non-differentiability. This tutorial explores these issues in depth\nthrough the lens of variational inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.06953v1",
            "date": "2018-12-17 18:50:36+00:00",
            "title": "Persian Vowel recognition with MFCC and ANN on PCVC speech dataset",
            "authors": [
                "Saber Malekzadeh",
                "Mohammad Hossein Gholizadeh",
                "Seyed Naser Razavi"
            ],
            "abstract": "In this paper a new method for recognition of consonant-vowel phonemes\ncombination on a new Persian speech dataset titled as PCVC (Persian\nConsonant-Vowel Combination) is proposed which is used to recognize Persian\nphonemes. In PCVC dataset, there are 20 sets of audio samples from 10 speakers\nwhich are combinations of 23 consonant and 6 vowel phonemes of Persian\nlanguage. In each sample, there is a combination of one vowel and one\nconsonant. First, the consonant phoneme is pronounced and just after it, the\nvowel phoneme is pronounced. Each sound sample is a frame of 2 seconds of\naudio. In every 2 seconds, there is an average of 0.5 second speech and the\nrest is silence. In this paper, the proposed method is the implementations of\nthe MFCC (Mel Frequency Cepstrum Coefficients) on every partitioned sound\nsample. Then, every train sample of MFCC vector is given to a multilayer\nperceptron feed-forward ANN (Artificial Neural Network) for training process.\nAt the end, the test samples are examined on ANN model for phoneme recognition.\nAfter training and testing process, the results are presented in recognition of\nvowels. Then, the average percent of recognition for vowel phonemes are\ncomputed.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1812.06974v1",
            "date": "2018-12-17 16:15:13+00:00",
            "title": "Analogy Search Engine: Finding Analogies in Cross-Domain Research Papers",
            "authors": [
                "Jieli Zhou",
                "Yuntao Zhou",
                "Yi Xu"
            ],
            "abstract": "In recent years, with the rapid proliferation of research publications in the\nfield of Artificial Intelligence, it is becoming increasingly difficult for\nresearchers to effectively keep up with all the latest research in one's own\ndomains. However, history has shown that scientific breakthroughs often come\nfrom collaborations of researchers from different domains. Traditional search\nalgorithms like Lexical search, which look for literal matches or synonyms and\nvariants of the query words, are not effective for discovering cross-domain\nresearch papers and meeting the needs of researchers in this age of information\noverflow. In this paper, we developed and tested an innovative semantic search\nengine, Analogy Search Engine (ASE), for 2000 AI research paper abstracts\nacross domains like Language Technologies, Robotics, Machine Learning,\nComputational Biology, Human Computer Interactions, etc. ASE combines recent\ntheories and methods from Computational Analogy and Natural Language Processing\nto go beyond keyword-based lexical search and discover the deeper analogical\nrelationships among research paper abstracts. We experimentally show that ASE\nis capable of finding more interesting and useful research papers than baseline\nelasticsearch. Furthermore, we believe that the methods used in ASE go beyond\nacademic paper and will benefit many other document search tasks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.07104v1",
            "date": "2018-12-11 13:10:14+00:00",
            "title": "Reading Industrial Inspection Sheets by Inferring Visual Relations",
            "authors": [
                "Rohit Rahul",
                "Arindam Chowdhury",
                "Animesh",
                "Samarth Mittal",
                "Lovekesh Vig"
            ],
            "abstract": "The traditional mode of recording faults in heavy factory equipment has been\nvia hand marked inspection sheets, wherein a machine engineer manually marks\nthe faulty machine regions on a paper outline of the machine. Over the years,\nmillions of such inspection sheets have been recorded and the data within these\nsheets has remained inaccessible. However, with industries going digital and\nwaking up to the potential value of fault data for machine health monitoring,\nthere is an increased impetus towards digitization of these hand marked\ninspection records. To target this digitization, we propose a novel visual\npipeline combining state of the art deep learning models, with domain knowledge\nand low level vision techniques, followed by inference of visual relationships.\nOur framework is robust to the presence of both static and non-static\nbackground in the document, variability in the machine template diagrams,\nunstructured shape of graphical objects to be identified and variability in the\nstrokes of handwritten text. The proposed pipeline incorporates a capsule and\nspatial transformer network based classifier for accurate text reading, and a\ncustomized CTPN network for text detection in addition to hybrid techniques for\narrow detection and dialogue cloud removal. We have tested our approach on a\nreal world dataset of 50 inspection sheets for large containers and boilers.\nThe results are visually appealing and the pipeline achieved an accuracy of\n87.1% for text detection and 94.6% for text reading.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.07617v2",
            "date": "2018-12-18 19:34:32+00:00",
            "title": "Towards Deep Conversational Recommendations",
            "authors": [
                "Raymond Li",
                "Samira Kahou",
                "Hannes Schulz",
                "Vincent Michalski",
                "Laurent Charlin",
                "Chris Pal"
            ],
            "abstract": "There has been growing interest in using neural networks and deep learning\ntechniques to create dialogue systems. Conversational recommendation is an\ninteresting setting for the scientific exploration of dialogue with natural\nlanguage as the associated discourse involves goal-driven dialogue that often\ntransforms naturally into more free-form chat. This paper provides two\ncontributions. First, until now there has been no publicly available\nlarge-scale dataset consisting of real-world dialogues centered around\nrecommendations. To address this issue and to facilitate our exploration here,\nwe have collected ReDial, a dataset consisting of over 10,000 conversations\ncentered around the theme of providing movie recommendations. We make this data\navailable to the community for further research. Second, we use this dataset to\nexplore multiple facets of conversational recommendations. In particular we\nexplore new neural architectures, mechanisms, and methods suitable for\ncomposing conversational recommendation systems. Our dataset allows us to\nsystematically probe model sub-components addressing different parts of the\noverall problem domain ranging from: sentiment analysis and cold-start\nrecommendation generation to detailed aspects of how natural language is used\nin this setting in the real world. We combine such sub-components into a\nfull-blown dialogue system and examine its behavior.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.08046v1",
            "date": "2018-12-19 16:02:08+00:00",
            "title": "Cyberbullying Detection in Social Networks Using Deep Learning Based Models; A Reproducibility Study",
            "authors": [
                "Maral Dadvar",
                "Kai Eckert"
            ],
            "abstract": "Cyberbullying is a disturbing online misbehaviour with troubling\nconsequences. It appears in different forms, and in most of the social\nnetworks, it is in textual format. Automatic detection of such incidents\nrequires intelligent systems. Most of the existing studies have approached this\nproblem with conventional machine learning models and the majority of the\ndeveloped models in these studies are adaptable to a single social network at a\ntime. In recent studies, deep learning based models have found their way in the\ndetection of cyberbullying incidents, claiming that they can overcome the\nlimitations of the conventional models, and improve the detection performance.\nIn this paper, we investigate the findings of a recent literature in this\nregard. We successfully reproduced the findings of this literature and\nvalidated their findings using the same datasets, namely Wikipedia, Twitter,\nand Formspring, used by the authors. Then we expanded our work by applying the\ndeveloped methods on a new YouTube dataset (~54k posts by ~4k users) and\ninvestigated the performance of the models in new social media platforms. We\nalso transferred and evaluated the performance of the models trained on one\nplatform to another platform. Our findings show that the deep learning based\nmodels outperform the machine learning models previously applied to the same\nYouTube dataset. We believe that the deep learning based models can also\nbenefit from integrating other sources of information and looking into the\nimpact of profile information of the users in social networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08092v1",
            "date": "2018-12-19 17:10:14+00:00",
            "title": "A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics",
            "authors": [
                "Martin Gerlach",
                "Francesc Font-Clos"
            ],
            "abstract": "The use of Project Gutenberg (PG) as a text corpus has been extremely popular\nin statistical analysis of language for more than 25 years. However, in\ncontrast to other major linguistic datasets of similar importance, no\nconsensual full version of PG exists to date. In fact, most PG studies so far\neither consider only a small number of manually selected books, leading to\npotential biased subsets, or employ vastly different pre-processing strategies\n(often specified in insufficient details), raising concerns regarding the\nreproducibility of published results. In order to address these shortcomings,\nhere we present the Standardized Project Gutenberg Corpus (SPGC), an open\nscience approach to a curated version of the complete PG data containing more\nthan 50,000 books and more than $3 \\times 10^9$ word-tokens. Using different\nsources of annotated metadata, we not only provide a broad characterization of\nthe content of PG, but also show different examples highlighting the potential\nof SPGC for investigating language variability across time, subjects, and\nauthors. We publish our methodology in detail, the code to download and process\nthe data, as well as the obtained corpus itself on 3 different levels of\ngranularity (raw text, timeseries of word tokens, and counts of words). In this\nway, we provide a reproducible, pre-processed, full-size version of Project\nGutenberg as a new scientific resource for corpus linguistics, natural language\nprocessing, and information retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08126v1",
            "date": "2018-12-19 18:10:18+00:00",
            "title": "Generating Diverse and Meaningful Captions",
            "authors": [
                "Annika Lindh",
                "Robert J. Ross",
                "Abhijit Mahalunkar",
                "Giancarlo Salton",
                "John D. Kelleher"
            ],
            "abstract": "Image Captioning is a task that requires models to acquire a multi-modal\nunderstanding of the world and to express this understanding in natural\nlanguage text. While the state-of-the-art for this task has rapidly improved in\nterms of n-gram metrics, these models tend to output the same generic captions\nfor similar images. In this work, we address this limitation and train a model\nthat generates more diverse and specific captions through an unsupervised\ntraining approach that incorporates a learning signal from an Image Retrieval\nmodel. We summarize previous results and improve the state-of-the-art on\ncaption diversity and novelty. We make our source code publicly available\nonline.",
            "journal ref": "Artificial Neural Networks and Machine Learning - ICANN 2018 (pp.\n  176-187). Springer International Publishing",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.08318v1",
            "date": "2018-12-20 02:25:45+00:00",
            "title": "Generating lyrics with variational autoencoder and multi-modal artist embeddings",
            "authors": [
                "Olga Vechtomova",
                "Hareesh Bahuleyan",
                "Amirpasha Ghabussi",
                "Vineet John"
            ],
            "abstract": "We present a system for generating song lyrics lines conditioned on the style\nof a specified artist. The system uses a variational autoencoder with artist\nembeddings. We propose the pre-training of artist embeddings with the\nrepresentations learned by a CNN classifier, which is trained to predict\nartists based on MEL spectrograms of their song clips. This work is the first\nstep towards combining audio and text modalities of songs for generating lyrics\nconditioned on the artist's style. Our preliminary results suggest that there\nis a benefit in initializing artists' embeddings with the representations\nlearned by a spectrogram classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08425v1",
            "date": "2018-12-20 08:56:52+00:00",
            "title": "A Survey of Hierarchy Identification in Social Networks",
            "authors": [
                "Denys Katerenchuk"
            ],
            "abstract": "Humans are social by nature. Throughout history, people have formed\ncommunities and built relationships. Most relationships with coworkers,\nfriends, and family are developed during face-to-face interactions. These\nrelationships are established through explicit means of communications such as\nwords and implicit such as intonation, body language, etc. By analyzing human\ninteractions we can derive information about the relationships and influence\namong conversation participants. However, with the development of the Internet,\npeople started to communicate through text in online social networks.\nInterestingly, they brought their communicational habits to the Internet. Many\nsocial network users form relationships with each other and establish\ncommunities with leaders and followers. Recognizing these hierarchical\nrelationships is an important task because it will help to understand social\nnetworks and predict future trends, improve recommendations, better target\nadvertisement, and improve national security by identifying leaders of\nanonymous terror groups. In this work, I provide an overview of current\nresearch in this area and present the state-of-the-art approaches to deal with\nthe problem of identifying hierarchical relationships in social networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08658v3",
            "date": "2018-12-20 16:04:05+00:00",
            "title": "nocaps: novel object captioning at scale",
            "authors": [
                "Harsh Agrawal",
                "Karan Desai",
                "Yufei Wang",
                "Xinlei Chen",
                "Rishabh Jain",
                "Mark Johnson",
                "Dhruv Batra",
                "Devi Parikh",
                "Stefan Lee",
                "Peter Anderson"
            ],
            "abstract": "Image captioning models have achieved impressive results on datasets\ncontaining limited visual concepts and large amounts of paired image-caption\ntraining data. However, if these models are to ever function in the wild, a\nmuch larger variety of visual concepts must be learned, ideally from less\nsupervision. To encourage the development of image captioning models that can\nlearn visual concepts from alternative data sources, such as object detection\ndatasets, we present the first large-scale benchmark for this task. Dubbed\n'nocaps', for novel object captioning at scale, our benchmark consists of\n166,100 human-generated captions describing 15,100 images from the OpenImages\nvalidation and test sets. The associated training data consists of COCO\nimage-caption pairs, plus OpenImages image-level labels and object bounding\nboxes. Since OpenImages contains many more classes than COCO, nearly 400 object\nclasses seen in test images have no or very few associated training captions\n(hence, nocaps). We extend existing novel object captioning models to establish\nstrong baselines for this benchmark and provide analysis to guide future work\non this task.",
            "journal ref": "IEEE International Conference on Computer Vision (ICCV) 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1812.08947v1",
            "date": "2018-12-21 05:02:02+00:00",
            "title": "Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach",
            "authors": [
                "Chuan Qin",
                "Hengshu Zhu",
                "Tong Xu",
                "Chen Zhu",
                "Liang Jiang",
                "Enhong Chen",
                "Hui Xiong"
            ],
            "abstract": "The wide spread use of online recruitment services has led to information\nexplosion in the job market. As a result, the recruiters have to seek the\nintelligent ways for Person Job Fit, which is the bridge for adapting the right\njob seekers to the right positions. Existing studies on Person Job Fit have a\nfocus on measuring the matching degree between the talent qualification and the\njob requirements mainly based on the manual inspection of human resource\nexperts despite of the subjective, incomplete, and inefficient nature of the\nhuman judgement. To this end, in this paper, we propose a novel end to end\nAbility aware Person Job Fit Neural Network model, which has a goal of reducing\nthe dependence on manual labour and can provide better interpretation about the\nfitting results. The key idea is to exploit the rich information available at\nabundant historical job application data. Specifically, we propose a word level\nsemantic representation for both job requirements and job seekers' experiences\nbased on Recurrent Neural Network. Along this line, four hierarchical ability\naware attention strategies are designed to measure the different importance of\njob requirements for semantic representation, as well as measuring the\ndifferent contribution of each job experience to a specific ability\nrequirement. Finally, extensive experiments on a large scale real world data\nset clearly validate the effectiveness and interpretability of the APJFNN\nframework compared with several baselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.08951v2",
            "date": "2018-12-21 05:13:03+00:00",
            "title": "Analysis Methods in Neural Language Processing: A Survey",
            "authors": [
                "Yonatan Belinkov",
                "James Glass"
            ],
            "abstract": "The field of natural language processing has seen impressive progress in\nrecent years, with neural network models replacing many of the traditional\nsystems. A plethora of new models have been proposed, many of which are thought\nto be opaque compared to their feature-rich counterparts. This has led\nresearchers to analyze, interpret, and evaluate neural networks in novel and\nmore fine-grained ways. In this survey paper, we review analysis methods in\nneural language processing, categorize them according to prominent research\ntrends, highlight existing limitations, and point to potential directions for\nfuture work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.08989v2",
            "date": "2018-12-21 08:01:31+00:00",
            "title": "The Design and Implementation of XiaoIce, an Empathetic Social Chatbot",
            "authors": [
                "Li Zhou",
                "Jianfeng Gao",
                "Di Li",
                "Heung-Yeung Shum"
            ],
            "abstract": "This paper describes the development of Microsoft XiaoIce, the most popular\nsocial chatbot in the world. XiaoIce is uniquely designed as an AI companion\nwith an emotional connection to satisfy the human need for communication,\naffection, and social belonging. We take into account both intelligent quotient\n(IQ) and emotional quotient (EQ) in system design, cast human-machine social\nchat as decision-making over Markov Decision Processes (MDPs), and optimize\nXiaoIce for long-term user engagement, measured in expected Conversation-turns\nPer Session (CPS). We detail the system architecture and key components\nincluding dialogue manager, core chat, skills, and an empathetic computing\nmodule. We show how XiaoIce dynamically recognizes human feelings and states,\nunderstands user intent, and responds to user needs throughout long\nconversations. Since her launch in 2014, XiaoIce has communicated with over 660\nmillion active users and succeeded in establishing long-term relationships with\nmany of them. Analysis of large scale online logs shows that XiaoIce has\nachieved an average CPS of 23, which is significantly higher than that of other\nchatbots and even human conversations.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1812.09195v1",
            "date": "2018-12-21 15:32:59+00:00",
            "title": "Learning to Navigate the Web",
            "authors": [
                "Izzeddin Gur",
                "Ulrich Rueckert",
                "Aleksandra Faust",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Learning in environments with large state and action spaces, and sparse\nrewards, can hinder a Reinforcement Learning (RL) agent's learning through\ntrial-and-error. For instance, following natural language instructions on the\nWeb (such as booking a flight ticket) leads to RL settings where input\nvocabulary and number of actionable elements on a page can grow very large.\nEven though recent approaches improve the success rate on relatively simple\nenvironments with the help of human demonstrations to guide the exploration,\nthey still fail in environments where the set of possible instructions can\nreach millions. We approach the aforementioned problems from a different\nperspective and propose guided RL approaches that can generate unbounded amount\nof experience for an agent to learn from. Instead of learning from a\ncomplicated instruction with a large vocabulary, we decompose it into multiple\nsub-instructions and schedule a curriculum in which an agent is tasked with a\ngradually increasing subset of these relatively easier sub-instructions. In\naddition, when the expert demonstrations are not available, we propose a novel\nmeta-learning framework that generates new instruction following tasks and\ntrains the agent more effectively. We train DQN, deep reinforcement learning\nagent, with Q-value function approximated with a novel QWeb neural network\narchitecture on these smaller, synthetic instructions. We evaluate the ability\nof our agent to generalize to new instructions on World of Bits benchmark, on\nforms with up to 100 elements, supporting 14 million possible instructions. The\nQWeb agent outperforms the baseline without using any human demonstration\nachieving 100% success rate on several difficult environments.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.09244v3",
            "date": "2018-12-21 16:37:29+00:00",
            "title": "Symbolic inductive bias for visually grounded learning of spoken language",
            "authors": [
                "Grzegorz Chrupa\u0142a"
            ],
            "abstract": "A widespread approach to processing spoken language is to first automatically\ntranscribe it into text. An alternative is to use an end-to-end approach:\nrecent works have proposed to learn semantic embeddings of spoken language from\nimages with spoken captions, without an intermediate transcription step. We\npropose to use multitask learning to exploit existing transcribed speech within\nthe end-to-end setting. We describe a three-task architecture which combines\nthe objectives of matching spoken captions with corresponding images, speech\nwith text, and text with images. We show that the addition of the speech/text\ntask leads to substantial performance improvements on image retrieval when\ncompared to training the speech/image task in isolation. We conjecture that\nthis is due to a strong inductive bias transcribed speech provides to the\nmodel, and offer supporting evidence for this.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09650v1",
            "date": "2018-12-23 03:35:56+00:00",
            "title": "Improving Context-Aware Semantic Relationships in Sparse Mobile Datasets",
            "authors": [
                "Peter Hansel",
                "Nik Marda",
                "William Yin"
            ],
            "abstract": "Traditional semantic similarity models often fail to encapsulate the external\ncontext in which texts are situated. However, textual datasets generated on\nmobile platforms can help us build a truer representation of semantic\nsimilarity by introducing multimodal data. This is especially important in\nsparse datasets, making solely text-driven interpretation of context more\ndifficult. In this paper, we develop new algorithms for building external\nfeatures into sentence embeddings and semantic similarity scores. Then, we test\nthem on embedding spaces on data from Twitter, using each tweet's time and\ngeolocation to better understand its context. Ultimately, we show that applying\nPCA with eight components to the embedding space and appending multimodal\nfeatures yields the best outcomes. This yields a considerable improvement over\npure text-based approaches for discovering similar tweets. Our results suggest\nthat our new algorithm can help improve semantic understanding in various\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.09652v1",
            "date": "2018-12-23 03:44:03+00:00",
            "title": "A Cross-Architecture Instruction Embedding Model for Natural Language Processing-Inspired Binary Code Analysis",
            "authors": [
                "Kimberly Redmond",
                "Lannan Luo",
                "Qiang Zeng"
            ],
            "abstract": "Given a closed-source program, such as most of proprietary software and\nviruses, binary code analysis is indispensable for many tasks, such as code\nplagiarism detection and malware analysis. Today, source code is very often\ncompiled for various architectures, making cross-architecture binary code\nanalysis increasingly important. A binary, after being disassembled, is\nexpressed in an assembly languages. Thus, recent work starts exploring Natural\nLanguage Processing (NLP) inspired binary code analysis. In NLP, words are\nusually represented in high-dimensional vectors (i.e., embeddings) to\nfacilitate further processing, which is one of the most common and critical\nsteps in many NLP tasks. We regard instructions as words in NLP-inspired binary\ncode analysis, and aim to represent instructions as embeddings as well.\n  To facilitate cross-architecture binary code analysis, our goal is that\nsimilar instructions, regardless of their architectures, have embeddings close\nto each other. To this end, we propose a joint learning approach to generating\ninstruction embeddings that capture not only the semantics of instructions\nwithin an architecture, but also their semantic relationships across\narchitectures. To the best of our knowledge, this is the first work on building\ncross-architecture instruction embedding model. As a showcase, we apply the\nmodel to resolving one of the most fundamental problems for binary code\nsimilarity comparison---semantics-based basic block comparison, and the\nsolution outperforms the code statistics based approach. It demonstrates that\nit is promising to apply the model to other cross-architecture binary code\nanalysis tasks.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1812.09718v2",
            "date": "2018-12-23 14:35:58+00:00",
            "title": "Optimizing Answer Set Computation via Heuristic-Based Decomposition",
            "authors": [
                "Francesco Calimeri",
                "Simona Perri",
                "Jessica Zangari"
            ],
            "abstract": "Answer Set Programming (ASP) is a purely declarative formalism developed in\nthe field of logic programming and nonmonotonic reasoning: computational\nproblems are encoded by logic programs whose answer sets, corresponding to\nsolutions, are computed by an ASP system. Different, semantically equivalent,\nprograms can be defined for the same problem; however, performance of systems\nevaluating them might significantly vary. We propose an approach for\nautomatically transforming an input logic program into an equivalent one that\ncan be evaluated more efficiently. One can make use of existing\ntree-decomposition techniques for rewriting selected rules into a set of\nmultiple ones; the idea is to guide and adaptively apply them on the basis of\nproper new heuristics, to obtain a smart rewriting algorithm to be integrated\ninto an ASP system. The method is rather general: it can be adapted to any\nsystem and implement different preference policies. Furthermore, we define a\nset of new heuristics tailored at optimizing grounding, one of the main phases\nof the ASP computation; we use them in order to implement the approach into the\nASP system DLV, in particular into its grounding subsystem I-DLV, and carry out\nan extensive experimental activity for assessing the impact of the proposal.\nUnder consideration in Theory and Practice of Logic Programming (TPLP).",
            "journal ref": "Theory and Practice of Logic Programming 19 (2019) 603-628",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.09798v1",
            "date": "2018-12-23 23:57:07+00:00",
            "title": "Pansori: ASR Corpus Generation from Open Online Video Contents",
            "authors": [
                "Yoona Choi",
                "Bowon Lee"
            ],
            "abstract": "This paper introduces Pansori, a program used to create ASR (automatic speech\nrecognition) corpora from online video contents. It utilizes a cloud-based\nspeech API to easily create a corpus in different languages. Using this\nprogram, we semi-automatically generated the Pansori-TEDxKR dataset from Korean\nTED conference talks with community-transcribed subtitles. It is the first\nhigh-quality corpus for the Korean language freely available for independent\nresearch. Pansori is released as an open-source software and the generated\ncorpus is released under a permissive public license for community use and\nparticipation.",
            "journal ref": "Proceedings of IEEE Seoul Section Student Paper Contest 2018,\n  Hongik University, pp. 117--121, Nov 2018",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.10119v1",
            "date": "2018-12-25 15:24:04+00:00",
            "title": "Sequence to Sequence Learning for Query Expansion",
            "authors": [
                "Salah Zaiem",
                "Fatiha Sadat"
            ],
            "abstract": "Using sequence to sequence algorithms for query expansion has not been\nexplored yet in Information Retrieval literature nor in Question-Answering's.\nWe tried to fill this gap in the literature with a custom Query Expansion\nengine trained and tested on open datasets. Starting from open datasets, we\nbuilt a Query Expansion training set using sentence-embeddings-based Keyword\nExtraction. We therefore assessed the ability of the Sequence to Sequence\nneural networks to capture expanding relations in the words embeddings' space.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.10234v1",
            "date": "2018-12-26 05:54:34+00:00",
            "title": "A New Concept of Deep Reinforcement Learning based Augmented General Sequence Tagging System",
            "authors": [
                "Yu Wang",
                "Abhishek Patel",
                "Hongxia Jin"
            ],
            "abstract": "In this paper, a new deep reinforcement learning based augmented general\nsequence tagging system is proposed. The new system contains two parts: a deep\nneural network (DNN) based sequence tagging model and a deep reinforcement\nlearning (DRL) based augmented tagger. The augmented tagger helps improve\nsystem performance by modeling the data with minority tags. The new system is\nevaluated on SLU and NLU sequence tagging tasks using ATIS and CoNLL-2003\nbenchmark datasets, to demonstrate the new system's outstanding performance on\ngeneral tagging tasks. Evaluated by F1 scores, it shows that the new system\noutperforms the current state-of-the-art model on ATIS dataset by 1.9% and that\non CoNLL-2003 dataset by 1.4%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.10235v1",
            "date": "2018-12-26 05:55:42+00:00",
            "title": "A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling",
            "authors": [
                "Yu Wang",
                "Yilin Shen",
                "Hongxia Jin"
            ],
            "abstract": "Intent detection and slot filling are two main tasks for building a spoken\nlanguage understanding(SLU) system. Multiple deep learning based models have\ndemonstrated good results on these tasks . The most effective algorithms are\nbased on the structures of sequence to sequence models (or \"encoder-decoder\"\nmodels), and generate the intents and semantic tags either using separate\nmodels or a joint model. Most of the previous studies, however, either treat\nthe intent detection and slot filling as two separate parallel tasks, or use a\nsequence to sequence model to generate both semantic tags and intent. Most of\nthese approaches use one (joint) NN based model (including encoder-decoder\nstructure) to model two tasks, hence may not fully take advantage of the\ncross-impact between them. In this paper, new Bi-model based RNN semantic frame\nparsing network structures are designed to perform the intent detection and\nslot filling tasks jointly, by considering their cross-impact to each other\nusing two correlated bidirectional LSTMs (BLSTM). Our Bi-model structure with a\ndecoder achieves state-of-the-art result on the benchmark ATIS data, with about\n0.5$\\%$ intent accuracy improvement and 0.9 $\\%$ slot filling improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10382v1",
            "date": "2018-12-12 02:38:56+00:00",
            "title": "The Global Anchor Method for Quantifying Linguistic Shifts and Domain Adaptation",
            "authors": [
                "Zi Yin",
                "Vin Sachidananda",
                "Balaji Prabhakar"
            ],
            "abstract": "Language is dynamic, constantly evolving and adapting with respect to time,\ndomain or topic. The adaptability of language is an active research area, where\nresearchers discover social, cultural and domain-specific changes in language\nusing distributional tools such as word embeddings. In this paper, we introduce\nthe global anchor method for detecting corpus-level language shifts. We show\nboth theoretically and empirically that the global anchor method is equivalent\nto the alignment method, a widely-used method for comparing word embeddings, in\nterms of detecting corpus-level language shifts. Despite their equivalence in\nterms of detection abilities, we demonstrate that the global anchor method is\nsuperior in terms of applicability as it can compare embeddings of different\ndimensionalities. Furthermore, the global anchor method has implementation and\nparallelization advantages. We show that the global anchor method reveals fine\nstructures in the evolution of language and domain adaptation. When combined\nwith the graph Laplacian technique, the global anchor method recovers the\nevolution trajectory and domain clustering of disparate text corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10387v1",
            "date": "2018-12-13 12:48:40+00:00",
            "title": "Same but Different: Distant Supervision for Predicting and Understanding Entity Linking Difficulty",
            "authors": [
                "Renato Stoffalette Jo\u00e3o",
                "Pavlos Fafalios",
                "Stefan Dietze"
            ],
            "abstract": "Entity Linking (EL) is the task of automatically identifying entity mentions\nin a piece of text and resolving them to a corresponding entity in a reference\nknowledge base like Wikipedia. There is a large number of EL tools available\nfor different types of documents and domains, yet EL remains a challenging task\nwhere the lack of precision on particularly ambiguous mentions often spoils the\nusefulness of automated disambiguation results in real applications. A priori\napproximations of the difficulty to link a particular entity mention can\nfacilitate flagging of critical cases as part of semi-automated EL systems,\nwhile detecting latent factors that affect the EL performance, like\ncorpus-specific features, can provide insights on how to improve a system based\non the special characteristics of the underlying corpus. In this paper, we\nfirst introduce a consensus-based method to generate difficulty labels for\nentity mentions on arbitrary corpora. The difficulty labels are then exploited\nas training data for a supervised classification task able to predict the EL\ndifficulty of entity mentions using a variety of features. Experiments over a\ncorpus of news articles show that EL difficulty can be estimated with high\naccuracy, revealing also latent features that affect EL performance. Finally,\nevaluation results demonstrate the effectiveness of the proposed method to\ninform semi-automated EL pipelines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10400v1",
            "date": "2018-12-12 10:34:21+00:00",
            "title": "Detecting weak and strong Islamophobic hate speech on social media",
            "authors": [
                "Bertie Vidgen",
                "Taha Yasseri"
            ],
            "abstract": "Islamophobic hate speech on social media inflicts considerable harm on both\ntargeted individuals and wider society, and also risks reputational damage for\nthe host platforms. Accordingly, there is a pressing need for robust tools to\ndetect and classify Islamophobic hate speech at scale. Previous research has\nlargely approached the detection of Islamophobic hate speech on social media as\na binary task. However, the varied nature of Islamophobia means that this is\noften inappropriate for both theoretically-informed social science and\neffectively monitoring social media. Drawing on in-depth conceptual work we\nbuild a multi-class classifier which distinguishes between non-Islamophobic,\nweak Islamophobic and strong Islamophobic content. Accuracy is 77.6% and\nbalanced accuracy is 83%. We apply the classifier to a dataset of 109,488\ntweets produced by far right Twitter accounts during 2017. Whilst most tweets\nare not Islamophobic, weak Islamophobia is considerably more prevalent (36,963\ntweets) than strong (14,895 tweets). Our main input feature is a gloVe word\nembeddings model trained on a newly collected corpus of 140 million tweets. It\noutperforms a generic word embeddings model by 5.9 percentage points,\ndemonstrating the importan4ce of context. Unexpectedly, we also find that a\none-against-one multi class SVM outperforms a deep learning algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10401v1",
            "date": "2018-12-12 15:38:46+00:00",
            "title": "Word Embedding based on Low-Rank Doubly Stochastic Matrix Decomposition",
            "authors": [
                "Denis Sedov",
                "Zhirong Yang"
            ],
            "abstract": "Word embedding, which encodes words into vectors, is an important starting\npoint in natural language processing and commonly used in many text-based\nmachine learning tasks. However, in most current word embedding approaches, the\nsimilarity in embedding space is not optimized in the learning. In this paper\nwe propose a novel neighbor embedding method which directly learns an embedding\nsimplex where the similarities between the mapped words are optimal in terms of\nminimal discrepancy to the input neighborhoods. Our method is built upon\ntwo-step random walks between words via topics and thus able to better reveal\nthe topics among the words. Experiment results indicate that our method,\ncompared with another existing word embedding approach, is more favorable for\nvarious queries.",
            "journal ref": "Cheng, L., Leung, A., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol.\n  11303, pp. 90-100. Springer, Cham (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10424v4",
            "date": "2018-12-13 21:00:05+00:00",
            "title": "Measuring Societal Biases from Text Corpora with Smoothed First-Order Co-occurrence",
            "authors": [
                "Navid Rekabsaz",
                "Robert West",
                "James Henderson",
                "Allan Hanbury"
            ],
            "abstract": "Text corpora are widely used resources for measuring societal biases and\nstereotypes. The common approach to measuring such biases using a corpus is by\ncalculating the similarities between the embedding vector of a word (like\nnurse) and the vectors of the representative words of the concepts of interest\n(such as genders). In this study, we show that, depending on what one aims to\nquantify as bias, this commonly-used approach can introduce non-relevant\nconcepts into bias measurement. We propose an alternative approach to bias\nmeasurement utilizing the smoothed first-order co-occurrence relations between\nthe word and the representative concept words, which we derive by\nreconstructing the co-occurrence estimates inherent in word embedding models.\nWe compare these approaches by conducting several experiments on the scenario\nof measuring gender bias of occupational words, according to an English\nWikipedia corpus. Our experiments show higher correlations of the measured\ngender bias with the actual gender bias statistics of the U.S. job market - on\ntwo collections and with a variety of word embedding models - using the\nfirst-order approach in comparison with the vector similarity-based approaches.\nThe first-order approach also suggests a more severe bias towards female in a\nfew specific occupations than the other approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10464v2",
            "date": "2018-12-26 18:58:39+00:00",
            "title": "Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond",
            "authors": [
                "Mikel Artetxe",
                "Holger Schwenk"
            ],
            "abstract": "We introduce an architecture to learn joint multilingual sentence\nrepresentations for 93 languages, belonging to more than 30 different families\nand written in 28 different scripts. Our system uses a single BiLSTM encoder\nwith a shared BPE vocabulary for all languages, which is coupled with an\nauxiliary decoder and trained on publicly available parallel corpora. This\nenables us to learn a classifier on top of the resulting embeddings using\nEnglish annotated data only, and transfer it to any of the 93 languages without\nany modification. Our experiments in cross-lingual natural language inference\n(XNLI dataset), cross-lingual document classification (MLDoc dataset) and\nparallel corpus mining (BUCC dataset) show the effectiveness of our approach.\nWe also introduce a new test set of aligned sentences in 112 languages, and\nshow that our sentence embeddings obtain strong results in multilingual\nsimilarity search even for low-resource languages. Our implementation, the\npre-trained encoder and the multilingual test set are available at\nhttps://github.com/facebookresearch/LASER",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.10549v1",
            "date": "2018-12-18 14:17:56+00:00",
            "title": "Automatic Summarization of Natural Language",
            "authors": [
                "Marc Everett Johnson"
            ],
            "abstract": "Automatic summarization of natural language is a current topic in computer\nscience research and industry, studied for decades because of its usefulness\nacross multiple domains. For example, summarization is necessary to create\nreviews such as this one. Research and applications have achieved some success\nin extractive summarization (where key sentences are curated), however,\nabstractive summarization (synthesis and re-stating) is a hard problem and\ngenerally unsolved in computer science. This literature review contrasts\nhistorical progress up through current state of the art, comparing dimensions\nsuch as: extractive vs. abstractive, supervised vs. unsupervised, NLP (Natural\nLanguage Processing) vs Knowledge-based, deep learning vs algorithms,\nstructured vs. unstructured sources, and measurement metrics such as Rouge and\nBLEU. Multiple dimensions are contrasted since current research uses\ncombinations of approaches as seen in the review matrix. Throughout this\nsummary, synthesis and critique is provided. This review concludes with\ninsights for improved abstractive summarization measurement, with surprising\nimplications for detecting understanding and comprehension in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11158v1",
            "date": "2018-12-28 18:44:49+00:00",
            "title": "MEETING BOT: Reinforcement Learning for Dialogue Based Meeting Scheduling",
            "authors": [
                "Vishwanath D",
                "Lovekesh Vig",
                "Gautam Shroff",
                "Puneet Agarwal"
            ],
            "abstract": "In this paper we present Meeting Bot, a reinforcement learning based\nconversational system that interacts with multiple users to schedule meetings.\nThe system is able to interpret user utterences and map them to preferred time\nslots, which are then fed to a reinforcement learning (RL) system with the goal\nof converging on an agreeable time slot. The RL system is able to adapt to user\npreferences and environmental changes in meeting arrival rate while still\nscheduling effectively. Learning is performed via policy gradient with\nexploration, by utilizing an MLP as an approximator of the policy function.\nResults demonstrate that the system outperforms standard scheduling algorithms\nin terms of overall scheduling efficiency. Additionally, the system is able to\nadapt its strategy to situations when users consistently reject or accept\nmeetings in certain slots (such as Friday afternoon versus Thursday morning),\nor when the meeting is called by members who are at a more senior designation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1812.11270v1",
            "date": "2018-12-29 03:04:26+00:00",
            "title": "Weakly-Supervised Hierarchical Text Classification",
            "authors": [
                "Yu Meng",
                "Jiaming Shen",
                "Chao Zhang",
                "Jiawei Han"
            ],
            "abstract": "Hierarchical text classification, which aims to classify text documents into\na given hierarchy, is an important task in many real-world applications.\nRecently, deep neural models are gaining increasing popularity for text\nclassification due to their expressive power and minimum requirement for\nfeature engineering. However, applying deep neural networks for hierarchical\ntext classification remains challenging, because they heavily rely on a large\namount of training data and meanwhile cannot easily determine appropriate\nlevels of documents in the hierarchical setting. In this paper, we propose a\nweakly-supervised neural method for hierarchical text classification. Our\nmethod does not require a large amount of training data but requires only\neasy-to-provide weak supervision signals such as a few class-related documents\nor keywords. Our method effectively leverages such weak supervision signals to\ngenerate pseudo documents for model pre-training, and then performs\nself-training on real unlabeled data to iteratively refine the model. During\nthe training process, our model features a hierarchical neural structure, which\nmimics the given hierarchy and is capable of determining the proper levels for\ndocuments with a blocking mechanism. Experiments on three datasets from\ndifferent domains demonstrate the efficacy of our method compared with a\ncomprehensive set of baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.11561v1",
            "date": "2018-12-30 15:39:57+00:00",
            "title": "Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching",
            "authors": [
                "Chen Qu",
                "Feng Ji",
                "Minghui Qiu",
                "Liu Yang",
                "Zhiyu Min",
                "Haiqing Chen",
                "Jun Huang",
                "W. Bruce Croft"
            ],
            "abstract": "Deep text matching approaches have been widely studied for many applications\nincluding question answering and information retrieval systems. To deal with a\ndomain that has insufficient labeled data, these approaches can be used in a\nTransfer Learning (TL) setting to leverage labeled data from a resource-rich\nsource domain. To achieve better performance, source domain data selection is\nessential in this process to prevent the \"negative transfer\" problem. However,\nthe emerging deep transfer models do not fit well with most existing data\nselection methods, because the data selection policy and the transfer learning\nmodel are not jointly trained, leading to sub-optimal training efficiency.\n  In this paper, we propose a novel reinforced data selector to select\nhigh-quality source domain data to help the TL model. Specifically, the data\nselector \"acts\" on the source domain data to find a subset for optimization of\nthe TL model, and the performance of the TL model can provide \"rewards\" in turn\nto update the selector. We build the reinforced data selector based on the\nactor-critic framework and integrate it to a DNN based transfer learning model,\nresulting in a Reinforced Transfer Learning (RTL) method. We perform a thorough\nexperimental evaluation on two major tasks for text matching, namely,\nparaphrase identification and natural language inference. Experimental results\nshow the proposed RTL can significantly improve the performance of the TL\nmodel. We further investigate different settings of states, rewards, and policy\noptimization methods to examine the robustness of our method. Last, we conduct\na case study on the selected data and find our method is able to select source\ndomain data whose Wasserstein distance is close to the target domain data. This\nis reasonable and intuitive as such source domain data can provide more\ntransferability power to the model.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1812.11737v2",
            "date": "2018-12-31 09:41:04+00:00",
            "title": "The meaning of \"most\" for visual question answering models",
            "authors": [
                "Alexander Kuhnle",
                "Ann Copestake"
            ],
            "abstract": "The correct interpretation of quantifier statements in the context of a\nvisual scene requires non-trivial inference mechanisms. For the example of\n\"most\", we discuss two strategies which rely on fundamentally different\ncognitive concepts. Our aim is to identify what strategy deep learning models\nfor visual question answering learn when trained on such questions. To this\nend, we carefully design data to replicate experiments from psycholinguistics\nwhere the same question was investigated for humans. Focusing on the FiLM\nvisual question answering model, our experiments indicate that a form of\napproximate number system emerges whose performance declines with more\ndifficult scenes as predicted by Weber's law. Moreover, we identify confounding\nfactors, like spatial arrangement of the scene, which impede the effectiveness\nof this system.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1901.00399v1",
            "date": "2018-12-31 16:04:16+00:00",
            "title": "Unary and Binary Classification Approaches and their Implications for Authorship Verification",
            "authors": [
                "Oren Halvani",
                "Christian Winter",
                "Lukas Graner"
            ],
            "abstract": "Retrieving indexed documents, not by their topical content but their writing\nstyle opens the door for a number of applications in information retrieval\n(IR). One application is to retrieve textual content of a certain author X,\nwhere the queried IR system is provided beforehand with a set of reference\ntexts of X. Authorship verification (AV), which is a research subject in the\nfield of digital text forensics, is suitable for this purpose. The task of AV\nis to determine if two documents (i.e. an indexed and a reference document)\nhave been written by the same author X. Even though AV represents a unary\nclassification problem, a number of existing approaches consider it as a binary\nclassification task. However, the underlying classification model of an AV\nmethod has a number of serious implications regarding its prerequisites,\nevaluability, and applicability. In our comprehensive literature review, we\nobserved several misunderstandings regarding the differentiation of unary and\nbinary AV approaches that require consideration. The objective of this paper\nis, therefore, to clarify these by proposing clear criteria and new properties\nthat aim to improve the characterization of existing and future AV approaches.\nGiven both, we investigate the applicability of eleven existing unary and\nbinary AV methods as well as four generic unary classification algorithms on\ntwo self-compiled corpora. Furthermore, we highlight an important issue\nconcerning the evaluation of AV methods based on fixed decision criterions,\nwhich has not been paid attention in previous AV studies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.00400v1",
            "date": "2018-12-31 16:30:21+00:00",
            "title": "Sentence-Level Sentiment Analysis of Financial News Using Distributed Text Representations and Multi-Instance Learning",
            "authors": [
                "Bernhard Lutz",
                "Nicolas Pr\u00f6llochs",
                "Dirk Neumann"
            ],
            "abstract": "Researchers and financial professionals require robust computerized tools\nthat allow users to rapidly operationalize and assess the semantic textual\ncontent in financial news. However, existing methods commonly work at the\ndocument-level while deeper insights into the actual structure and the\nsentiment of individual sentences remain blurred. As a result, investors are\nrequired to apply the utmost attention and detailed, domain-specific knowledge\nin order to assess the information on a fine-grained basis. To facilitate this\nmanual process, this paper proposes the use of distributed text representations\nand multi-instance learning to transfer information from the document-level to\nthe sentence-level. Compared to alternative approaches, this method features\nsuperior predictive performance while preserving context and interpretability.\nOur analysis of a manually-labeled dataset yields a predictive accuracy of up\nto 69.90%, exceeding the performance of alternative approaches by at least 3.80\npercentage points. Accordingly, this study not only benefits investors with\nregard to their financial decision-making, but also helps companies to\ncommunicate their messages as intended.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.00439v1",
            "date": "2018-12-25 00:31:22+00:00",
            "title": "Deep Representation Learning for Clustering of Health Tweets",
            "authors": [
                "Oguzhan Gencoglu"
            ],
            "abstract": "Twitter has been a prominent social media platform for mining\npopulation-level health data and accurate clustering of health-related tweets\ninto topics is important for extracting relevant health insights. In this work,\nwe propose deep convolutional autoencoders for learning compact representations\nof health-related tweets, further to be employed in clustering. We compare our\nmethod to several conventional tweet representation methods including\nbag-of-words, term frequency-inverse document frequency, Latent Dirichlet\nAllocation and Non-negative Matrix Factorization with 3 different clustering\nalgorithms. Our results show that the clustering performance using proposed\nrepresentation learning scheme significantly outperforms that of conventional\nmethods for all experiments of different number of clusters. In addition, we\npropose a constraint on the learned representations during the neural network\ntraining in order to further enhance the clustering performance. All in all,\nthis study introduces utilization of deep neural network-based architectures,\ni.e., deep convolutional autoencoders, for learning informative representations\nof health-related tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00519v2",
            "date": "2018-12-31 18:48:20+00:00",
            "title": "Pull out all the stops: Textual analysis via punctuation sequences",
            "authors": [
                "Alexandra N. M. Darmon",
                "Marya Bazzi",
                "Sam D. Howison",
                "Mason A. Porter"
            ],
            "abstract": "Whether enjoying the lucid prose of a favorite author or slogging through\nsome other writer's cumbersome, heavy-set prattle (full of parentheses, em\ndashes, compound adjectives, and Oxford commas), readers will notice stylistic\nsignatures not only in word choice and grammar, but also in punctuation itself.\nIndeed, visual sequences of punctuation from different authors produce\nmarvelously different (and visually striking) sequences. Punctuation is a\nlargely overlooked stylistic feature in \"stylometry\", the quantitative analysis\nof written text. In this paper, we examine punctuation sequences in a corpus of\nliterary documents and ask the following questions: Are the properties of such\nsequences a distinctive feature of different authors? Is it possible to\ndistinguish literary genres based on their punctuation sequences? Do the\npunctuation styles of authors evolve over time? Are we on to something\ninteresting in trying to do stylometry without words, or are we full of sound\nand fury (signifying nothing)?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03604v1",
            "date": "2018-01-11 01:23:50+00:00",
            "title": "Conversational AI: The Science Behind the Alexa Prize",
            "authors": [
                "Ashwin Ram",
                "Rohit Prasad",
                "Chandra Khatri",
                "Anu Venkatesh",
                "Raefer Gabriel",
                "Qing Liu",
                "Jeff Nunn",
                "Behnam Hedayatnia",
                "Ming Cheng",
                "Ashish Nagar",
                "Eric King",
                "Kate Bland",
                "Amanda Wartick",
                "Yi Pan",
                "Han Song",
                "Sk Jayadevan",
                "Gene Hwang",
                "Art Pettigrue"
            ],
            "abstract": "Conversational agents are exploding in popularity. However, much work remains\nin the area of social conversation as well as free-form conversation over a\nbroad range of domains and topics. To advance the state of the art in\nconversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar\nuniversity competition where sixteen selected university teams were challenged\nto build conversational agents, known as socialbots, to converse coherently and\nengagingly with humans on popular topics such as Sports, Politics,\nEntertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers\nthe academic community a unique opportunity to perform research with a live\nsystem used by millions of users. The competition provided university teams\nwith real user conversational data at scale, along with the user-provided\nratings and feedback augmented with annotations by the Alexa team. This enabled\nteams to effectively iterate and make improvements throughout the competition\nwhile being evaluated in real-time through live user interactions. To build\ntheir socialbots, university teams combined state-of-the-art techniques with\nnovel strategies in the areas of Natural Language Understanding, Context\nModeling, Dialog Management, Response Generation, and Knowledge Acquisition. To\nsupport the efforts of participating teams, the Alexa Prize team made\nsignificant scientific and engineering investments to build and improve\nConversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice\nUser Experience, and tools for traffic management and scalability. This paper\noutlines the advances created by the university teams as well as the Alexa\nPrize team to achieve the common goal of solving the problem of Conversational\nAI.",
            "journal ref": "Alexa.Prize.Proceedings\n  https://developer.amazon.com/alexaprize/proceedings accessed (2018)-01-01",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1801.03622v1",
            "date": "2018-01-11 03:20:02+00:00",
            "title": "Topic-based Evaluation for Conversational Bots",
            "authors": [
                "Fenfei Guo",
                "Angeliki Metallinou",
                "Chandra Khatri",
                "Anirudh Raju",
                "Anu Venkatesh",
                "Ashwin Ram"
            ],
            "abstract": "Dialog evaluation is a challenging problem, especially for non task-oriented\ndialogs where conversational success is not well-defined. We propose to\nevaluate dialog quality using topic-based metrics that describe the ability of\na conversational bot to sustain coherent and engaging conversations on a topic,\nand the diversity of topics that a bot can handle. To detect conversation\ntopics per utterance, we adopt Deep Average Networks (DAN) and train a topic\nclassifier on a variety of question and query data categorized into multiple\ntopics. We propose a novel extension to DAN by adding a topic-word attention\ntable that allows the system to jointly capture topic keywords in an utterance\nand perform topic classification. We compare our proposed topic based metrics\nwith the ratings provided by users and show that our metrics both correlate\nwith and complement human judgment. Our analysis is performed on tens of\nthousands of real human-bot dialogs from the Alexa Prize competition and\nhighlights user expectations for conversational bots.",
            "journal ref": "Nips.Workshop.ConversationalAI 2017-12-08",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.03625v2",
            "date": "2018-01-11 03:30:00+00:00",
            "title": "On Evaluating and Comparing Open Domain Dialog Systems",
            "authors": [
                "Anu Venkatesh",
                "Chandra Khatri",
                "Ashwin Ram",
                "Fenfei Guo",
                "Raefer Gabriel",
                "Ashish Nagar",
                "Rohit Prasad",
                "Ming Cheng",
                "Behnam Hedayatnia",
                "Angeliki Metallinou",
                "Rahul Goel",
                "Shaohua Yang",
                "Anirudh Raju"
            ],
            "abstract": "Conversational agents are exploding in popularity. However, much work remains\nin the area of non goal-oriented conversations, despite significant growth in\nresearch interest over recent years. To advance the state of the art in\nconversational AI, Amazon launched the Alexa Prize, a 2.5-million dollar\nuniversity competition where sixteen selected university teams built\nconversational agents to deliver the best social conversational experience.\nAlexa Prize provided the academic community with the unique opportunity to\nperform research with a live system used by millions of users. The subjectivity\nassociated with evaluating conversations is key element underlying the\nchallenge of building non-goal oriented dialogue systems. In this paper, we\npropose a comprehensive evaluation strategy with multiple metrics designed to\nreduce subjectivity by selecting metrics which correlate well with human\njudgement. The proposed metrics provide granular analysis of the conversational\nagents, which is not captured in human ratings. We show that these metrics can\nbe used as a reasonable proxy for human judgment. We provide a mechanism to\nunify the metrics for selecting the top performing agents, which has also been\napplied throughout the Alexa Prize competition. To our knowledge, to date it is\nthe largest setting for evaluating agents with millions of conversations and\nhundreds of thousands of ratings from users. We believe that this work is a\nstep towards an automatic evaluation process for conversational AIs.",
            "journal ref": "NIPS.Workshop.ConversationalAI 2017-12-08\n  http://alborz-geramifard.com/workshops/nips17-Conversational-AI/Main.html\n  accessed 2018-01-01",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.05088v1",
            "date": "2018-01-16 01:26:33+00:00",
            "title": "Real-time Road Traffic Information Detection Through Social Media",
            "authors": [
                "Chandra Khatri"
            ],
            "abstract": "In current study, a mechanism to extract traffic related information such as\ncongestion and incidents from textual data from the internet is proposed. The\ncurrent source of data is Twitter. As the data being considered is extremely\nlarge in size automated models are developed to stream, download, and mine the\ndata in real-time. Furthermore, if any tweet has traffic related information\nthen the models should be able to infer and extract this data.\n  Currently, the data is collected only for United States and a total of\n120,000 geo-tagged traffic related tweets are extracted, while six million\ngeo-tagged non-traffic related tweets are retrieved and classification models\nare trained. Furthermore, this data is used for various kinds of spatial and\ntemporal analysis. A mechanism to calculate level of traffic congestion,\nsafety, and traffic perception for cities in U.S. is proposed. Traffic\ncongestion and safety rankings for the various urban areas are obtained and\nthen they are statistically validated with existing widely adopted rankings.\nTraffic perception depicts the attitude and perception of people towards the\ntraffic.\n  It is also seen that traffic related data when visualized spatially and\ntemporally provides the same pattern as the actual traffic flows for various\nurban areas. When visualized at the city level, it is clearly visible that the\nflow of tweets is similar to flow of vehicles and that the traffic related\ntweets are representative of traffic within the cities. With all the findings\nin current study, it is shown that significant amount of traffic related\ninformation can be extracted from Twitter and other sources on internet.\nFurthermore, Twitter and these data sources are freely available and are not\nbound by spatial and temporal limitations. That is, wherever there is a user\nthere is a potential for data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1801.06700v1",
            "date": "2018-01-20 17:22:06+00:00",
            "title": "A Deep Reinforcement Learning Chatbot (Short Version)",
            "authors": [
                "Iulian V. Serban",
                "Chinnadhurai Sankar",
                "Mathieu Germain",
                "Saizheng Zhang",
                "Zhouhan Lin",
                "Sandeep Subramanian",
                "Taesup Kim",
                "Michael Pieper",
                "Sarath Chandar",
                "Nan Rosemary Ke",
                "Sai Rajeswar",
                "Alexandre de Brebisson",
                "Jose M. R. Sotelo",
                "Dendi Suhubdy",
                "Vincent Michalski",
                "Alexandre Nguyen",
                "Joelle Pineau",
                "Yoshua Bengio"
            ],
            "abstract": "We present MILABOT: a deep reinforcement learning chatbot developed by the\nMontreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize\ncompetition. MILABOT is capable of conversing with humans on popular small talk\ntopics through both speech and text. The system consists of an ensemble of\nnatural language generation and retrieval models, including neural network and\ntemplate-based models. By applying reinforcement learning to crowdsourced data\nand real-world user interactions, the system has been trained to select an\nappropriate response from the models in its ensemble. The system has been\nevaluated through A/B testing with real-world users, where it performed\nsignificantly better than other systems. The results highlight the potential of\ncoupling ensemble systems with deep reinforcement learning as a fruitful path\nfor developing real-world, open-domain conversational agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.01155v1",
            "date": "2018-04-03 20:18:11+00:00",
            "title": "Socioeconomic Dependencies of Linguistic Patterns in Twitter: A Multivariate Analysis",
            "authors": [
                "Jacob Levy Abitbol",
                "M\u00e1rton Karsai",
                "Jean-Philippe Magu\u00e9",
                "Jean-Pierre Chevrot",
                "Eric Fleury"
            ],
            "abstract": "Our usage of language is not solely reliant on cognition but is arguably\ndetermined by myriad external factors leading to a global variability of\nlinguistic patterns. This issue, which lies at the core of sociolinguistics and\nis backed by many small-scale studies on face-to-face communication, is\naddressed here by constructing a dataset combining the largest French Twitter\ncorpus to date with detailed socioeconomic maps obtained from national census\nin France. We show how key linguistic variables measured in individual Twitter\nstreams depend on factors like socioeconomic status, location, time, and the\nsocial network of individuals. We found that (i) people of higher socioeconomic\nstatus, active to a greater degree during the daytime, use a more standard\nlanguage; (ii) the southern part of the country is more prone to use more\nstandard language than the northern one, while locally the used variety or\ndialect is determined by the spatial distribution of socioeconomic status; and\n(iii) individuals connected in the social network are closer linguistically\nthan disconnected ones, even after the effects of status homophily have been\nremoved. Our results inform sociolinguistic theory and may inspire novel\nlearning methods for the inference of socioeconomic status of people from the\nway they tweet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.02173v1",
            "date": "2018-04-06 09:03:29+00:00",
            "title": "On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks",
            "authors": [
                "Egor Lakomkin",
                "Mohammad Ali Zamani",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Speech emotion recognition (SER) is an important aspect of effective\nhuman-robot collaboration and received a lot of attention from the research\ncommunity. For example, many neural network-based architectures were proposed\nrecently and pushed the performance to a new level. However, the applicability\nof such neural SER models trained only on in-domain data to noisy conditions is\ncurrently under-researched. In this work, we evaluate the robustness of\nstate-of-the-art neural acoustic emotion recognition models in human-robot\ninteraction scenarios. We hypothesize that a robot's ego noise, room\nconditions, and various acoustic events that can occur in a home environment\ncan significantly affect the performance of a model. We conduct several\nexperiments on the iCub robot platform and propose several novel ways to reduce\nthe gap between the model's performance during training and testing in\nreal-world conditions. Furthermore, we observe large improvements in the model\nperformance on the robot and demonstrate the necessity of introducing several\ndata augmentation techniques like overlaying background noise and loudness\nvariations to improve the robustness of the neural approaches.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1804.03201v2",
            "date": "2018-04-09 19:44:29+00:00",
            "title": "Scalable Factorized Hierarchical Variational Autoencoder Training",
            "authors": [
                "Wei-Ning Hsu",
                "James Glass"
            ],
            "abstract": "Deep generative models have achieved great success in unsupervised learning\nwith the ability to capture complex nonlinear relationships between latent\ngenerating factors and observations. Among them, a factorized hierarchical\nvariational autoencoder (FHVAE) is a variational inference-based model that\nformulates a hierarchical generative process for sequential data. Specifically,\nan FHVAE model can learn disentangled and interpretable representations, which\nhave been proven useful for numerous speech applications, such as speaker\nverification, robust speech recognition, and voice conversion. However, as we\nwill elaborate in this paper, the training algorithm proposed in the original\npaper is not scalable to datasets of thousands of hours, which makes this model\nless applicable on a larger scale. After identifying limitations in terms of\nruntime, memory, and hyperparameter optimization, we propose a hierarchical\nsampling training algorithm to address all three issues. Our proposed method is\nevaluated comprehensively on a wide variety of datasets, ranging from 3 to\n1,000 hours and involving different types of generating factors, such as\nrecording conditions and noise types. In addition, we also present a new\nvisualization method for qualitatively evaluating the performance with respect\nto the interpretability and disentanglement. Models trained with our proposed\nalgorithm demonstrate the desired characteristics on all the datasets.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1804.05374v2",
            "date": "2018-04-15 15:52:16+00:00",
            "title": "Twin Regularization for online speech recognition",
            "authors": [
                "Mirco Ravanelli",
                "Dmitriy Serdyuk",
                "Yoshua Bengio"
            ],
            "abstract": "Online speech recognition is crucial for developing natural human-machine\ninterfaces. This modality, however, is significantly more challenging than\noff-line ASR, since real-time/low-latency constraints inevitably hinder the use\nof future information, that is known to be very helpful to perform robust\npredictions. A popular solution to mitigate this issue consists of feeding\nneural acoustic models with context windows that gather some future frames.\nThis introduces a latency which depends on the number of employed look-ahead\nfeatures. This paper explores a different approach, based on estimating the\nfuture rather than waiting for it. Our technique encourages the hidden\nrepresentations of a unidirectional recurrent network to embed some useful\ninformation about the future. Inspired by a recently proposed technique called\nTwin Networks, we add a regularization term that forces forward hidden states\nto be as close as possible to cotemporal backward ones, computed by a \"twin\"\nneural network running backwards in time. The experiments, conducted on a\nnumber of datasets, recurrent architectures, input features, and acoustic\nconditions, have shown the effectiveness of this approach. One important\nadvantage is that our method does not introduce any additional computation at\ntest time if compared to standard unidirectional recurrent networks.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1804.09558v2",
            "date": "2018-04-24 15:34:33+00:00",
            "title": "A Visual Distance for WordNet",
            "authors": [
                "Raquel P\u00e9rez-Arnal",
                "Armand Vilalta",
                "Dario Garcia-Gasulla",
                "Ulises Cort\u00e9s",
                "Eduard Ayguad\u00e9",
                "Jesus Labarta"
            ],
            "abstract": "Measuring the distance between concepts is an important field of study of\nNatural Language Processing, as it can be used to improve tasks related to the\ninterpretation of those same concepts. WordNet, which includes a wide variety\nof concepts associated with words (i.e., synsets), is often used as a source\nfor computing those distances. In this paper, we explore a distance for WordNet\nsynsets based on visual features, instead of lexical ones. For this purpose, we\nextract the graphic features generated within a deep convolutional neural\nnetworks trained with ImageNet and use those features to generate a\nrepresentative of each synset. Based on those representatives, we define a\ndistance measure of synsets, which complements the traditional lexical\ndistances. Finally, we propose some experiments to evaluate its performance and\ncompare it with the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1804.10204v1",
            "date": "2018-04-26 13:14:22+00:00",
            "title": "End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction",
            "authors": [
                "Zhong-Qiu Wang",
                "Jonathan Le Roux",
                "DeLiang Wang",
                "John R. Hershey"
            ],
            "abstract": "This paper proposes an end-to-end approach for single-channel\nspeaker-independent multi-speaker speech separation, where time-frequency (T-F)\nmasking, the short-time Fourier transform (STFT), and its inverse are\nrepresented as layers within a deep network. Previous approaches, rather than\ncomputing a loss on the reconstructed signal, used a surrogate loss based on\nthe target STFT magnitudes. This ignores reconstruction error introduced by\nphase inconsistency. In our approach, the loss function is directly defined on\nthe reconstructed signals, which are optimized for best separation. In\naddition, we train through unfolded iterations of a phase reconstruction\nalgorithm, represented as a series of STFT and inverse STFT layers. While mask\nvalues are typically limited to lie between zero and one for approaches using\nthe mixture phase for reconstruction, this limitation is less relevant if the\nestimated magnitudes are to be used together with phase reconstruction. We thus\npropose several novel activation functions for the output layer of the T-F\nmasking, to allow mask values beyond one. On the publicly-available wsj0-2mix\ndataset, our approach achieves state-of-the-art 12.6 dB scale-invariant\nsignal-to-distortion ratio (SI-SDR) and 13.1 dB SDR, revealing new\npossibilities for deep learning based phase reconstruction and representing a\nfundamental progress towards solving the notoriously-hard cocktail party\nproblem.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1808.00300v1",
            "date": "2018-08-01 12:39:43+00:00",
            "title": "Learning Visual Question Answering by Bootstrapping Hard Attention",
            "authors": [
                "Mateusz Malinowski",
                "Carl Doersch",
                "Adam Santoro",
                "Peter Battaglia"
            ],
            "abstract": "Attention mechanisms in biological perception are thought to select subsets\nof perceptual information for more sophisticated processing which would be\nprohibitive to perform on all sensory inputs. In computer vision, however,\nthere has been relatively little exploration of hard attention, where some\ninformation is selectively ignored, in spite of the success of soft attention,\nwhere information is re-weighted and aggregated, but never filtered out. Here,\nwe introduce a new approach for hard attention and find it achieves very\ncompetitive performance on a recently-released visual question answering\ndatasets, equalling and in some cases surpassing similar soft attention\narchitectures while entirely ignoring some features. Even though the hard\nattention mechanism is thought to be non-differentiable, we found that the\nfeature magnitudes correlate with semantic relevance, and provide a useful\nsignal for our mechanism's attentional selection criterion. Because hard\nattention selects important features of the input information, it can also be\nmore efficient than analogous soft attention mechanisms. This is especially\nimportant for recent approaches that use non-local pairwise operations, whereby\ncomputational and memory costs are quadratic in the size of the set of\nfeatures.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1808.00525v2",
            "date": "2018-07-30 14:29:27+00:00",
            "title": "The impact of imbalanced training data on machine learning for author name disambiguation",
            "authors": [
                "Jinseok Kim",
                "Jenna Kim"
            ],
            "abstract": "In supervised machine learning for author name disambiguation, negative\ntraining data are often dominantly larger than positive training data. This\npaper examines how the ratios of negative to positive training data can affect\nthe performance of machine learning algorithms to disambiguate author names in\nbibliographic records. On multiple labeled datasets, three classifiers -\nLogistic Regression, Na\\\"ive Bayes, and Random Forest - are trained through\nrepresentative features such as coauthor names, and title words extracted from\nthe same training data but with various positive-negative training data ratios.\nResults show that increasing negative training data can improve disambiguation\nperformance but with a few percent of performance gains and sometimes degrade\nit. Logistic Regression and Na\\\"ive Bayes learn optimal disambiguation models\neven with a base ratio (1:1) of positive and negative training data. Also, the\nperformance improvement by Random Forest tends to quickly saturate roughly\nafter 1:10 ~ 1:15. These findings imply that contrary to the common practice\nusing all training data, name disambiguation algorithms can be trained using\npart of negative training data without degrading much disambiguation\nperformance while increasing computational efficiency. This study calls for\nmore attention from author name disambiguation scholars to methods for machine\nlearning from imbalanced data.",
            "journal ref": "Kim, J. & Kim, J. (2018). The impact of imbalanced training data\n  on machine learning for author name disambiguation. Scientometrics",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1808.01410v1",
            "date": "2018-08-04 02:21:07+00:00",
            "title": "Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis",
            "authors": [
                "Daisy Stanton",
                "Yuxuan Wang",
                "RJ Skerry-Ryan"
            ],
            "abstract": "Global Style Tokens (GSTs) are a recently-proposed method to learn latent\ndisentangled representations of high-dimensional data. GSTs can be used within\nTacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to\nuncover expressive factors of variation in speaking style. In this work, we\nintroduce the Text-Predicted Global Style Token (TP-GST) architecture, which\ntreats GST combination weights or style embeddings as \"virtual\" speaking style\nlabels within Tacotron. TP-GST learns to predict stylistic renderings from text\nalone, requiring neither explicit labels during training nor auxiliary inputs\nfor inference. We show that, when trained on a dataset of expressive speech,\nour system generates audio with more pitch and energy variation than two\nstate-of-the-art baseline models. We further demonstrate that TP-GSTs can\nsynthesize speech with background noise removed, and corroborate these analyses\nwith positive results on human-rated listener preference audiobook tasks.\nFinally, we demonstrate that multi-speaker TP-GST models successfully factorize\nspeaker identity and speaking style. We provide a website with audio samples\nfor each of our findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.02022v1",
            "date": "2018-08-06 03:04:35+00:00",
            "title": "Principles for Developing a Knowledge Graph of Interlinked Events from News Headlines on Twitter",
            "authors": [
                "Saeedeh Shekarpour",
                "Ankita Saxena",
                "Krishnaprasad Thirunarayan",
                "Valerie L. Shalin",
                "Amit Sheth"
            ],
            "abstract": "The ever-growing datasets published on Linked Open Data mainly contain\nencyclopedic information. However, there is a lack of quality structured and\nsemantically annotated datasets extracted from unstructured real-time sources.\nIn this paper, we present principles for developing a knowledge graph of\ninterlinked events using the case study of news headlines published on Twitter\nwhich is a real-time and eventful source of fresh information. We represent the\nessential pipeline containing the required tasks ranging from choosing\nbackground data model, event annotation (i.e., event recognition and\nclassification), entity annotation and eventually interlinking events. The\nstate-of-the-art is limited to domain-specific scenarios for recognizing and\nclassifying events, whereas this paper plays the role of a domain-agnostic\nroad-map for developing a knowledge graph of interlinked events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1808.03920v1",
            "date": "2018-08-12 10:04:45+00:00",
            "title": "Multimodal Language Analysis with Recurrent Multistage Fusion",
            "authors": [
                "Paul Pu Liang",
                "Ziyin Liu",
                "Amir Zadeh",
                "Louis-Philippe Morency"
            ],
            "abstract": "Computational modeling of human multimodal language is an emerging research\narea in natural language processing spanning the language, visual and acoustic\nmodalities. Comprehending multimodal language requires modeling not only the\ninteractions within each modality (intra-modal interactions) but more\nimportantly the interactions between modalities (cross-modal interactions). In\nthis paper, we propose the Recurrent Multistage Fusion Network (RMFN) which\ndecomposes the fusion problem into multiple stages, each of them focused on a\nsubset of multimodal signals for specialized, effective fusion. Cross-modal\ninteractions are modeled using this multistage fusion approach which builds\nupon intermediate representations of previous stages. Temporal and intra-modal\ninteractions are modeled by integrating our proposed fusion approach with a\nsystem of recurrent neural networks. The RMFN displays state-of-the-art\nperformance in modeling human multimodal language across three public datasets\nrelating to multimodal sentiment analysis, emotion recognition, and speaker\ntraits recognition. We provide visualizations to show that each stage of fusion\nfocuses on a different subset of multimodal signals, learning increasingly\ndiscriminative multimodal representations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1808.05902v1",
            "date": "2018-08-17 15:32:24+00:00",
            "title": "Learning Supervised Topic Models for Classification and Regression from Crowds",
            "authors": [
                "Filipe Rodrigues",
                "Mariana Louren\u00e7o",
                "Bernardete Ribeiro",
                "Francisco Pereira"
            ],
            "abstract": "The growing need to analyze large collections of documents has led to great\ndevelopments in topic modeling. Since documents are frequently associated with\nother related variables, such as labels or ratings, much interest has been\nplaced on supervised topic models. However, the nature of most annotation\ntasks, prone to ambiguity and noise, often with high volumes of documents, deem\nlearning under a single-annotator assumption unrealistic or unpractical for\nmost real-world applications. In this article, we propose two supervised topic\nmodels, one for classification and another for regression problems, which\naccount for the heterogeneity and biases among different annotators that are\nencountered in practice when learning from crowds. We develop an efficient\nstochastic variational inference algorithm that is able to scale to very large\ndatasets, and we empirically demonstrate the advantages of the proposed model\nover state-of-the-art approaches.",
            "journal ref": "Rodrigues, F., Lourenco, M., Ribeiro, B. and Pereira, F.C., 2017.\n  Learning supervised topic models for classification and regression from\n  crowds. IEEE transactions on pattern analysis and machine intelligence,\n  39(12), pp.2409-2422",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1808.09037v1",
            "date": "2018-08-27 21:28:46+00:00",
            "title": "Volatility in the Issue Attention Economy",
            "authors": [
                "Chico Q. Camargo",
                "Scott A. Hale",
                "Peter John",
                "Helen Z. Margetts"
            ],
            "abstract": "Recent election surprises and regime changes have left the impression that\npolitics has become more fast-moving and unstable. While modern politics does\nseem more volatile, there is little systematic evidence to support this claim.\nThis paper seeks to address this gap in knowledge by reporting data over the\nlast seventy years using public opinion polls and traditional media data from\nthe UK and Germany. These countries are good cases to study because both have\nexperienced considerable changes in electoral behaviour and have new political\nparties during the time period studied. We measure volatility in public opinion\nand in media coverage using approaches from information theory, tracking the\nchange in word-use patterns across over 700,000 articles. Our preliminary\nanalysis suggests an increase in the number of opinion issues over time and a\ngrowth in lack of predictability of the media series from the 1970s.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1802.00209v3",
            "date": "2018-02-01 09:35:33+00:00",
            "title": "Dual Recurrent Attention Units for Visual Question Answering",
            "authors": [
                "Ahmed Osman",
                "Wojciech Samek"
            ],
            "abstract": "Visual Question Answering (VQA) requires AI models to comprehend data in two\ndomains, vision and text. Current state-of-the-art models use learned attention\nmechanisms to extract relevant information from the input domains to answer a\ncertain question. Thus, robust attention mechanisms are essential for powerful\nVQA models. In this paper, we propose a recurrent attention mechanism and show\nits benefits compared to the traditional convolutional approach. We perform two\nablation studies to evaluate recurrent attention. First, we introduce a\nbaseline VQA model with visual attention and test the performance difference\nbetween convolutional and recurrent attention on the VQA 2.0 dataset. Secondly,\nwe design an architecture for VQA which utilizes dual (textual and visual)\nRecurrent Attention Units (RAUs). Using this model, we show the effect of all\npossible combinations of recurrent and convolutional dual attention. Our single\nmodel outperforms the first place winner on the VQA 2016 challenge and to the\nbest of our knowledge, it is the second best performing single model on the VQA\n1.0 dataset. Furthermore, our model noticeably improves upon the winner of the\nVQA 2017 challenge. Moreover, we experiment replacing attention mechanisms in\nstate-of-the-art models with our RAUs and show increased performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1802.01786v1",
            "date": "2018-02-06 03:55:37+00:00",
            "title": "Mining Public Opinion about Economic Issues: Twitter and the U.S. Presidential Election",
            "authors": [
                "Amir Karami",
                "London S. Bennett",
                "Xiaoyun He"
            ],
            "abstract": "Opinion polls have been the bridge between public opinion and politicians in\nelections. However, developing surveys to disclose people's feedback with\nrespect to economic issues is limited, expensive, and time-consuming. In recent\nyears, social media such as Twitter has enabled people to share their opinions\nregarding elections. Social media has provided a platform for collecting a\nlarge amount of social media data. This paper proposes a computational public\nopinion mining approach to explore the discussion of economic issues in social\nmedia during an election. Current related studies use text mining methods\nindependently for election analysis and election prediction; this research\ncombines two text mining methods: sentiment analysis and topic modeling. The\nproposed approach has effectively been deployed on millions of tweets to\nanalyze economic concerns of people during the 2012 US presidential election.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1802.03268v2",
            "date": "2018-02-09 14:14:37+00:00",
            "title": "Efficient Neural Architecture Search via Parameter Sharing",
            "authors": [
                "Hieu Pham",
                "Melody Y. Guan",
                "Barret Zoph",
                "Quoc V. Le",
                "Jeff Dean"
            ],
            "abstract": "We propose Efficient Neural Architecture Search (ENAS), a fast and\ninexpensive approach for automatic model design. In ENAS, a controller learns\nto discover neural network architectures by searching for an optimal subgraph\nwithin a large computational graph. The controller is trained with policy\ngradient to select a subgraph that maximizes the expected reward on the\nvalidation set. Meanwhile the model corresponding to the selected subgraph is\ntrained to minimize a canonical cross entropy loss. Thanks to parameter sharing\nbetween child models, ENAS is fast: it delivers strong empirical performances\nusing much fewer GPU-hours than all existing automatic model design approaches,\nand notably, 1000x less expensive than standard Neural Architecture Search. On\nthe Penn Treebank dataset, ENAS discovers a novel architecture that achieves a\ntest perplexity of 55.8, establishing a new state-of-the-art among all methods\nwithout post-training processing. On the CIFAR-10 dataset, ENAS designs novel\narchitectures that achieve a test error of 2.89%, which is on par with NASNet\n(Zoph et al., 2018), whose test error is 2.65%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1802.05383v1",
            "date": "2018-02-15 02:00:54+00:00",
            "title": "Deep Learning Based Speech Beamforming",
            "authors": [
                "Kaizhi Qian",
                "Yang Zhang",
                "Shiyu Chang",
                "Xuesong Yang",
                "Dinei Florencio",
                "Mark Hasegawa-Johnson"
            ],
            "abstract": "Multi-channel speech enhancement with ad-hoc sensors has been a challenging\ntask. Speech model guided beamforming algorithms are able to recover natural\nsounding speech, but the speech models tend to be oversimplified or the\ninference would otherwise be too complicated. On the other hand, deep learning\nbased enhancement approaches are able to learn complicated speech distributions\nand perform efficient inference, but they are unable to deal with variable\nnumber of input channels. Also, deep learning approaches introduce a lot of\nerrors, particularly in the presence of unseen noise types and settings. We\nhave therefore proposed an enhancement framework called DEEPBEAM, which\ncombines the two complementary classes of algorithms. DEEPBEAM introduces a\nbeamforming filter to produce natural sounding speech, but the filter\ncoefficients are determined with the help of a monaural speech enhancement\nneural network. Experiments on synthetic and real-world data show that DEEPBEAM\nis able to produce clean, dry and natural sounding speech, and is robust\nagainst unseen noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.06412v2",
            "date": "2018-02-18 17:54:19+00:00",
            "title": "Improved TDNNs using Deep Kernels and Frequency Dependent Grid-RNNs",
            "authors": [
                "Florian Kreyssig",
                "Chao Zhang",
                "Philip Woodland"
            ],
            "abstract": "Time delay neural networks (TDNNs) are an effective acoustic model for large\nvocabulary speech recognition. The strength of the model can be attributed to\nits ability to effectively model long temporal contexts. However, current TDNN\nmodels are relatively shallow, which limits the modelling capability. This\npaper proposes a method of increasing the network depth by deepening the kernel\nused in the TDNN temporal convolutions. The best performing kernel consists of\nthree fully connected layers with a residual (ResNet) connection from the\noutput of the first to the output of the third. The addition of\nspectro-temporal processing as the input to the TDNN in the form of a\nconvolutional neural network (CNN) and a newly designed Grid-RNN was\ninvestigated. The Grid-RNN strongly outperforms a CNN if different sets of\nparameters for different frequency bands are used and can be further enhanced\nby using a bi-directional Grid-RNN. Experiments using the multi-genre broadcast\n(MGB3) English data (275h) show that deep kernel TDNNs reduces the word error\nrate (WER) by 6% relative and when combined with the frequency dependent\nGrid-RNN gives a relative WER reduction of 9%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1802.10411v1",
            "date": "2018-02-28 13:53:34+00:00",
            "title": "Distance entropy cartography characterises centrality in complex networks",
            "authors": [
                "Massimo Stella",
                "Manlio De Domenico"
            ],
            "abstract": "We introduce distance entropy as a measure of homogeneity in the distribution\nof path lengths between a given node and its neighbours in a complex network.\nDistance entropy defines a new centrality measure whose properties are\ninvestigated for a variety of synthetic network models. By coupling distance\nentropy information with closeness centrality, we introduce a network\ncartography which allows one to reduce the degeneracy of ranking based on\ncloseness alone. We apply this methodology to the empirical multiplex lexical\nnetwork encoding the linguistic relationships known to English speaking\ntoddlers. We show that the distance entropy cartography better predicts how\nchildren learn words compared to closeness centrality. Our results highlight\nthe importance of distance entropy for gaining insights from distance patterns\nin complex networks.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1803.02551v1",
            "date": "2018-03-07 07:30:36+00:00",
            "title": "Extracting Domain Invariant Features by Unsupervised Learning for Robust Automatic Speech Recognition",
            "authors": [
                "Wei-Ning Hsu",
                "James Glass"
            ],
            "abstract": "The performance of automatic speech recognition (ASR) systems can be\nsignificantly compromised by previously unseen conditions, which is typically\ndue to a mismatch between training and testing distributions. In this paper, we\naddress robustness by studying domain invariant features, such that domain\ninformation becomes transparent to ASR systems, resolving the mismatch problem.\nSpecifically, we investigate a recent model, called the Factorized Hierarchical\nVariational Autoencoder (FHVAE). FHVAEs learn to factorize sequence-level and\nsegment-level attributes into different latent variables without supervision.\nWe argue that the set of latent variables that contain segment-level\ninformation is our desired domain invariant feature for ASR. Experiments are\nconducted on Aurora-4 and CHiME-4, which demonstrate 41% and 27% absolute word\nerror rate reductions respectively on mismatched domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1803.05307v1",
            "date": "2018-03-13 10:59:24+00:00",
            "title": "Deep CNN based feature extractor for text-prompted speaker recognition",
            "authors": [
                "Sergey Novoselov",
                "Oleg Kudashev",
                "Vadim Schemelinin",
                "Ivan Kremnev",
                "Galina Lavrentyeva"
            ],
            "abstract": "Deep learning is still not a very common tool in speaker verification field.\nWe study deep convolutional neural network performance in the text-prompted\nspeaker verification task. The prompted passphrase is segmented into word\nstates - i.e. digits -to test each digit utterance separately. We train a\nsingle high-level feature extractor for all states and use cosine similarity\nmetric for scoring. The key feature of our network is the Max-Feature-Map\nactivation function, which acts as an embedded feature selector. By using\nmultitask learning scheme to train the high-level feature extractor we were\nable to surpass the classic baseline systems in terms of quality and achieved\nimpressive results for such a novice approach, getting 2.85% EER on the RSR2015\nevaluation set. Fusion of the proposed and the baseline systems improves this\nresult.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1803.07679v1",
            "date": "2018-03-20 22:25:29+00:00",
            "title": "Product Characterisation towards Personalisation: Learning Attributes from Unstructured Data to Recommend Fashion Products",
            "authors": [
                "\u00c2ngelo Cardoso",
                "Fabio Daolio",
                "Sa\u00fal Vargas"
            ],
            "abstract": "In this paper, we describe a solution to tackle a common set of challenges in\ne-commerce, which arise from the fact that new products are continually being\nadded to the catalogue. The challenges involve properly personalising the\ncustomer experience, forecasting demand and planning the product range. We\nargue that the foundational piece to solve all of these problems is having\nconsistent and detailed information about each product, information that is\nrarely available or consistent given the multitude of suppliers and types of\nproducts. We describe in detail the architecture and methodology implemented at\nASOS, one of the world's largest fashion e-commerce retailers, to tackle this\nproblem. We then show how this quantitative understanding of the products can\nbe leveraged to improve recommendations in a hybrid recommender system\napproach.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1803.08471v3",
            "date": "2018-03-22 17:14:29+00:00",
            "title": "Locally Private Bayesian Inference for Count Models",
            "authors": [
                "Aaron Schein",
                "Zhiwei Steven Wu",
                "Alexandra Schofield",
                "Mingyuan Zhou",
                "Hanna Wallach"
            ],
            "abstract": "We present a general method for privacy-preserving Bayesian inference in\nPoisson factorization, a broad class of models that includes some of the most\nwidely used models in the social sciences. Our method satisfies limited\nprecision local privacy, a generalization of local differential privacy, which\nwe introduce to formulate privacy guarantees appropriate for sparse count data.\nWe develop an MCMC algorithm that approximates the locally private posterior\nover model parameters given data that has been locally privatized by the\ngeometric mechanism (Ghosh et al., 2012). Our solution is based on two\ninsights: 1) a novel reinterpretation of the geometric mechanism in terms of\nthe Skellam distribution (Skellam, 1946) and 2) a general theorem that relates\nthe Skellam to the Bessel distribution (Yuan & Kalbfleisch, 2000). We\ndemonstrate our method in two case studies on real-world email data in which we\nshow that our method consistently outperforms the commonly-used naive approach,\nobtaining higher quality topics in text and more accurate link prediction in\nnetworks. On some tasks, our privacy-preserving method even outperforms\nnon-private inference which conditions on the true data.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1803.10384v1",
            "date": "2018-03-28 02:12:48+00:00",
            "title": "Topic Modeling Based Multi-modal Depression Detection",
            "authors": [
                "Yuan Gong",
                "Christian Poellabauer"
            ],
            "abstract": "Major depressive disorder is a common mental disorder that affects almost 7%\nof the adult U.S. population. The 2017 Audio/Visual Emotion Challenge (AVEC)\nasks participants to build a model to predict depression levels based on the\naudio, video, and text of an interview ranging between 7-33 minutes. Since\naveraging features over the entire interview will lose most temporal\ninformation, how to discover, capture, and preserve useful temporal details for\nsuch a long interview are significant challenges. Therefore, we propose a novel\ntopic modeling based approach to perform context-aware analysis of the\nrecording. Our experiments show that the proposed approach outperforms\ncontext-unaware methods and the challenge baselines for all metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.00254v1",
            "date": "2018-05-01 09:39:19+00:00",
            "title": "Joint Bootstrapping Machines for High Confidence Relation Extraction",
            "authors": [
                "Pankaj Gupta",
                "Benjamin Roth",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Semi-supervised bootstrapping techniques for relationship extraction from\ntext iteratively expand a set of initial seed instances. Due to the lack of\nlabeled data, a key challenge in bootstrapping is semantic drift: if a false\npositive instance is added during an iteration, then all following iterations\nare contaminated. We introduce BREX, a new bootstrapping method that protects\nagainst such contamination by highly effective confidence assessment. This is\nachieved by using entity and template seeds jointly (as opposed to just one as\nin previous work), by expanding entities and templates in parallel and in a\nmutually constraining fashion in each iteration and by introducing\nhigherquality similarity measures for templates. Experimental results show that\nBREX achieves an F1 that is 0.13 (0.87 vs. 0.74) better than the state of the\nart for four relationships.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.01156v1",
            "date": "2018-05-03 08:12:39+00:00",
            "title": "Supervector Compression Strategies to Speed up I-Vector System Development",
            "authors": [
                "Ville Vestman",
                "Tomi Kinnunen"
            ],
            "abstract": "The front-end factor analysis (FEFA), an extension of principal component\nanalysis (PPCA) tailored to be used with Gaussian mixture models (GMMs), is\ncurrently the prevalent approach to extract compact utterance-level features\n(i-vectors) for automatic speaker verification (ASV) systems. Little research\nhas been conducted comparing FEFA to the conventional PPCA applied to maximum a\nposteriori (MAP) adapted GMM supervectors. We study several alternative\nmethods, including PPCA, factor analysis (FA), and two supervised approaches,\nsupervised PPCA (SPPCA) and the recently proposed probabilistic partial least\nsquares (PPLS), to compress MAP-adapted GMM supervectors. The resulting\ni-vectors are used in ASV tasks with a probabilistic linear discriminant\nanalysis (PLDA) back-end. We experiment on two different datasets, on the\ntelephone condition of NIST SRE 2010 and on the recent VoxCeleb corpus\ncollected from YouTube videos containing celebrity interviews recorded in\nvarious acoustical and technical conditions. The results suggest that, in terms\nof ASV accuracy, the supervector compression approaches are on a par with FEFA.\nThe supervised approaches did not result in improved performance. In comparison\nto FEFA, we obtained more than hundred-fold (100x) speedups in the total\nvariability model (TVM) training using the PPCA and FA supervector compression\napproaches.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1805.02356v1",
            "date": "2018-05-07 06:12:32+00:00",
            "title": "Multimodal Machine Translation with Reinforcement Learning",
            "authors": [
                "Xin Qian",
                "Ziyi Zhong",
                "Jieli Zhou"
            ],
            "abstract": "Multimodal machine translation is one of the applications that integrates\ncomputer vision and language processing. It is a unique task given that in the\nfield of machine translation, many state-of-the-arts algorithms still only\nemploy textual information. In this work, we explore the effectiveness of\nreinforcement learning in multimodal machine translation. We present a novel\nalgorithm based on the Advantage Actor-Critic (A2C) algorithm that specifically\ncater to the multimodal machine translation task of the EMNLP 2018 Third\nConference on Machine Translation (WMT18). We experiment our proposed algorithm\non the Multi30K multilingual English-German image description dataset and the\nFlickr30K image entity dataset. Our model takes two channels of inputs, image\nand text, uses translation evaluation metrics as training rewards, and achieves\nbetter results than supervised learning MLE baseline models. Furthermore, we\ndiscuss the prospects and limitations of using reinforcement learning for\nmachine translation. Our experiment results suggest a promising reinforcement\nlearning solution to the general task of multimodal sequence to sequence\nlearning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.02924v1",
            "date": "2018-05-08 09:51:34+00:00",
            "title": "Comparing phonemes and visemes with DNN-based lipreading",
            "authors": [
                "Kwanchiva Thangthai",
                "Helen L Bear",
                "Richard Harvey"
            ],
            "abstract": "There is debate if phoneme or viseme units are the most effective for a\nlipreading system. Some studies use phoneme units even though phonemes describe\nunique short sounds; other studies tried to improve lipreading accuracy by\nfocusing on visemes with varying results. We compare the performance of a\nlipreading system by modeling visual speech using either 13 viseme or 38\nphoneme units. We report the accuracy of our system at both word and unit\nlevels. The evaluation task is large vocabulary continuous speech using the\nTCD-TIMIT corpus. We complete our visual speech modeling via hybrid DNN-HMMs\nand our visual speech decoder is a Weighted Finite-State Transducer (WFST). We\nuse DCT and Eigenlips as a representation of mouth ROI image. The phoneme\nlipreading system word accuracy outperforms the viseme based system word\naccuracy. However, the phoneme system achieved lower accuracy at the unit level\nwhich shows the importance of the dictionary for decoding classification\noutputs into words.",
            "journal ref": "BMVC Lipreading Workshop 2017",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.05345v1",
            "date": "2018-05-14 18:00:03+00:00",
            "title": "Conversations Gone Awry: Detecting Early Signs of Conversational Failure",
            "authors": [
                "Justine Zhang",
                "Jonathan P. Chang",
                "Cristian Danescu-Niculescu-Mizil",
                "Lucas Dixon",
                "Yiqing Hua",
                "Nithum Thain",
                "Dario Taraborelli"
            ],
            "abstract": "One of the main challenges online social systems face is the prevalence of\nantisocial behavior, such as harassment and personal attacks. In this work, we\nintroduce the task of predicting from the very start of a conversation whether\nit will get out of hand. As opposed to detecting undesirable behavior after the\nfact, this task aims to enable early, actionable prediction at a time when the\nconversation might still be salvaged.\n  To this end, we develop a framework for capturing pragmatic devices---such as\npoliteness strategies and rhetorical prompts---used to start a conversation,\nand analyze their relation to its future trajectory. Applying this framework in\na controlled setting, we demonstrate the feasibility of detecting early warning\nsigns of antisocial behavior in online discussions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.06879v1",
            "date": "2018-05-17 17:40:12+00:00",
            "title": "Neural language representations predict outcomes of scientific research",
            "authors": [
                "James P. Bagrow",
                "Daniel Berenberg",
                "Joshua Bongard"
            ],
            "abstract": "Many research fields codify their findings in standard formats, often by\nreporting correlations between quantities of interest. But the space of all\ntestable correlates is far larger than scientific resources can currently\naddress, so the ability to accurately predict correlations would be useful to\nplan research and allocate resources. Using a dataset of approximately 170,000\ncorrelational findings extracted from leading social science journals, we show\nthat a trained neural network can accurately predict the reported correlations\nusing only the text descriptions of the correlates. Accurate predictive models\nsuch as these can guide scientists towards promising untested correlates,\nbetter quantify the information gained from new findings, and has implications\nfor moving artificial intelligence systems from predicting structures to\npredicting relationships in the real world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09687v1",
            "date": "2018-05-15 07:05:52+00:00",
            "title": "Corpus Conversion Service: A machine learning platform to ingest documents at scale [Poster abstract]",
            "authors": [
                "Peter W J Staar",
                "Michele Dolfi",
                "Christoph Auer",
                "Costas Bekas"
            ],
            "abstract": "Over the past few decades, the amount of scientific articles and technical\nliterature has increased exponentially in size. Consequently, there is a great\nneed for systems that can ingest these documents at scale and make their\ncontent discoverable. Unfortunately, both the format of these documents (e.g.\nthe PDF format or bitmap images) as well as the presentation of the data (e.g.\ncomplex tables) make the extraction of qualitative and quantitive data\nextremely challenging. We present a platform to ingest documents at scale which\nis powered by Machine Learning techniques and allows the user to train custom\nmodels on document collections. We show precision/recall results greater than\n97% with regard to conversion to structured formats, as well as scaling\nevidence for each of the microservices constituting the platform.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1805.09701v2",
            "date": "2018-05-24 14:43:30+00:00",
            "title": "R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering",
            "authors": [
                "Pan Lu",
                "Lei Ji",
                "Wei Zhang",
                "Nan Duan",
                "Ming Zhou",
                "Jianyong Wang"
            ],
            "abstract": "Recently, Visual Question Answering (VQA) has emerged as one of the most\nsignificant tasks in multimodal learning as it requires understanding both\nvisual and textual modalities. Existing methods mainly rely on extracting image\nand question features to learn their joint feature embedding via multimodal\nfusion or attention mechanism. Some recent studies utilize external\nVQA-independent models to detect candidate entities or attributes in images,\nwhich serve as semantic knowledge complementary to the VQA task. However, these\ncandidate entities or attributes might be unrelated to the VQA task and have\nlimited semantic capacities. To better utilize semantic knowledge in images, we\npropose a novel framework to learn visual relation facts for VQA. Specifically,\nwe build up a Relation-VQA (R-VQA) dataset based on the Visual Genome dataset\nvia a semantic similarity module, in which each data consists of an image, a\ncorresponding question, a correct answer and a supporting relation fact. A\nwell-defined relation detector is then adopted to predict visual\nquestion-related relation facts. We further propose a multi-step attention\nmodel composed of visual attention and semantic attention sequentially to\nextract related visual knowledge and semantic knowledge. We conduct\ncomprehensive experiments on the two benchmark datasets, demonstrating that our\nmodel achieves state-of-the-art performance and verifying the benefit of\nconsidering visual relation facts.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1805.10364v1",
            "date": "2018-05-25 21:06:56+00:00",
            "title": "Detecting Deceptive Reviews using Generative Adversarial Networks",
            "authors": [
                "Hojjat Aghakhani",
                "Aravind Machiry",
                "Shirin Nilizadeh",
                "Christopher Kruegel",
                "Giovanni Vigna"
            ],
            "abstract": "In the past few years, consumer review sites have become the main target of\ndeceptive opinion spam, where fictitious opinions or reviews are deliberately\nwritten to sound authentic. Most of the existing work to detect the deceptive\nreviews focus on building supervised classifiers based on syntactic and lexical\npatterns of an opinion. With the successful use of Neural Networks on various\nclassification applications, in this paper, we propose FakeGAN a system that\nfor the first time augments and adopts Generative Adversarial Networks (GANs)\nfor a text classification task, in particular, detecting deceptive reviews.\nUnlike standard GAN models which have a single Generator and Discriminator\nmodel, FakeGAN uses two discriminator models and one generative model. The\ngenerator is modeled as a stochastic policy agent in reinforcement learning\n(RL), and the discriminators use Monte Carlo search algorithm to estimate and\npass the intermediate action-value as the RL reward to the generator. Providing\nthe generator model with two discriminator models avoids the mod collapse issue\nby learning from both distributions of truthful and deceptive reviews. Indeed,\nour experiments show that using two discriminators provides FakeGAN high\nstability, which is a known issue for GAN architectures. While FakeGAN is built\nupon a semi-supervised classifier, known for less accuracy, our evaluation\nresults on a dataset of TripAdvisor hotel reviews show the same performance in\nterms of accuracy as of the state-of-the-art approaches that apply supervised\nmachine learning. These results indicate that GANs can be effective for text\nclassification tasks. Specifically, FakeGAN is effective at detecting deceptive\nreviews.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1805.11264v1",
            "date": "2018-05-29 06:45:02+00:00",
            "title": "Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data",
            "authors": [
                "Wei-Ning Hsu",
                "James Glass"
            ],
            "abstract": "Multimodal sensory data resembles the form of information perceived by humans\nfor learning, and are easy to obtain in large quantities. Compared to unimodal\ndata, synchronization of concepts between modalities in such data provides\nsupervision for disentangling the underlying explanatory factors of each\nmodality. Previous work leveraging multimodal data has mainly focused on\nretaining only the modality-invariant factors while discarding the rest. In\nthis paper, we present a partitioned variational autoencoder (PVAE) and several\ntraining objectives to learn disentangled representations, which encode not\nonly the shared factors, but also modality-dependent ones, into separate latent\nvariables. Specifically, PVAE integrates a variational inference framework and\na multimodal generative model that partitions the explanatory factors and\nconditions only on the relevant subset of them for generation. We evaluate our\nmodel on two parallel speech/image datasets, and demonstrate its ability to\nlearn disentangled representations by qualitatively exploring within-modality\nand cross-modality conditional generation with semantics and styles specified\nby examples. For quantitative analysis, we evaluate the classification accuracy\nof automatically discovered semantic units. Our PVAE can achieve over 99%\naccuracy on both modalities.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1805.11752v5",
            "date": "2018-05-30 00:05:53+00:00",
            "title": "Multi-turn Dialogue Response Generation in an Adversarial Learning Framework",
            "authors": [
                "Oluwatobi Olabiyi",
                "Alan Salimov",
                "Anish Khazane",
                "Erik T. Mueller"
            ],
            "abstract": "We propose an adversarial learning approach for generating multi-turn\ndialogue responses. Our proposed framework, hredGAN, is based on conditional\ngenerative adversarial networks (GANs). The GAN's generator is a modified\nhierarchical recurrent encoder-decoder network (HRED) and the discriminator is\na word-level bidirectional RNN that shares context and word embeddings with the\ngenerator. During inference, noise samples conditioned on the dialogue history\nare used to perturb the generator's latent space to generate several possible\nresponses. The final response is the one ranked best by the discriminator. The\nhredGAN shows improved performance over existing methods: (1) it generalizes\nbetter than networks trained using only the log-likelihood criterion, and (2)\nit generates longer, more informative and more diverse responses with high\nutterance and topic relevance even with limited training data. This improvement\nis demonstrated on the Movie triples and Ubuntu dialogue datasets using both\nautomatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1805.12316v1",
            "date": "2018-05-31 04:40:32+00:00",
            "title": "Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data",
            "authors": [
                "Puyudi Yang",
                "Jianbo Chen",
                "Cho-Jui Hsieh",
                "Jane-Ling Wang",
                "Michael I. Jordan"
            ],
            "abstract": "We present a probabilistic framework for studying adversarial attacks on\ndiscrete data. Based on this framework, we derive a perturbation-based method,\nGreedy Attack, and a scalable learning-based method, Gumbel Attack, that\nillustrate various tradeoffs in the design of attacks. We demonstrate the\neffectiveness of these methods using both quantitative metrics and human\nevaluation on various state-of-the-art models for text classification,\nincluding a word-based CNN, a character-based CNN and an LSTM. As as example of\nour results, we show that the accuracy of character-based convolutional\nnetworks drops to the level of random selection by modifying only five\ncharacters through Greedy Attack.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1806.00047v1",
            "date": "2018-05-31 18:42:26+00:00",
            "title": "Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning",
            "authors": [
                "Valts Blukis",
                "Nataly Brukhim",
                "Andrew Bennett",
                "Ross A. Knepper",
                "Yoav Artzi"
            ],
            "abstract": "We introduce a method for following high-level navigation instructions by\nmapping directly from images, instructions and pose estimates to continuous\nlow-level velocity commands for real-time control. The Grounded Semantic\nMapping Network (GSMN) is a fully-differentiable neural network architecture\nthat builds an explicit semantic map in the world reference frame by\nincorporating a pinhole camera projection model within the network. The\ninformation stored in the map is learned from experience, while the\nlocal-to-world transformation is computed explicitly. We train the model using\nDAggerFM, a modified variant of DAgger that trades tabular convergence\nguarantees for improved training speed and memory use. We test GSMN in virtual\nenvironments on a realistic quadcopter simulator and show that incorporating an\nexplicit mapping and grounding modules allows GSMN to outperform strong neural\nbaselines and almost reach an expert policy performance. Finally, we analyze\nthe learned map representations and show that using an explicit map leads to an\ninterpretable instruction-following model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1806.03497v1",
            "date": "2018-06-09 16:07:02+00:00",
            "title": "Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction",
            "authors": [
                "Siyuan Qi",
                "Baoxiong Jia",
                "Song-Chun Zhu"
            ],
            "abstract": "Future predictions on sequence data (e.g., videos or audios) require the\nalgorithms to capture non-Markovian and compositional properties of high-level\nsemantics. Context-free grammars are natural choices to capture such\nproperties, but traditional grammar parsers (e.g., Earley parser) only take\nsymbolic sentences as inputs. In this paper, we generalize the Earley parser to\nparse sequence data which is neither segmented nor labeled. This generalized\nEarley parser integrates a grammar parser with a classifier to find the optimal\nsegmentation and labels, and makes top-down future predictions. Experiments\nshow that our method significantly outperforms other approaches for future\nhuman activity prediction.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1806.04284v1",
            "date": "2018-06-12 00:58:59+00:00",
            "title": "iParaphrasing: Extracting Visually Grounded Paraphrases via an Image",
            "authors": [
                "Chenhui Chu",
                "Mayu Otani",
                "Yuta Nakashima"
            ],
            "abstract": "A paraphrase is a restatement of the meaning of a text in other words.\nParaphrases have been studied to enhance the performance of many natural\nlanguage processing tasks. In this paper, we propose a novel task iParaphrasing\nto extract visually grounded paraphrases (VGPs), which are different phrasal\nexpressions describing the same visual concept in an image. These extracted\nVGPs have the potential to improve language and image multimodal tasks such as\nvisual question answering and image captioning. How to model the similarity\nbetween VGPs is the key of iParaphrasing. We apply various existing methods as\nwell as propose a novel neural network-based method with image attention, and\nreport the results of the first attempt toward iParaphrasing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.04872v1",
            "date": "2018-06-13 07:14:15+00:00",
            "title": "Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition",
            "authors": [
                "Wei-Ning Hsu",
                "Hao Tang",
                "James Glass"
            ],
            "abstract": "The current trend in automatic speech recognition is to leverage large\namounts of labeled data to train supervised neural network models.\nUnfortunately, obtaining data for a wide range of domains to train robust\nmodels can be costly. However, it is relatively inexpensive to collect large\namounts of unlabeled data from domains that we want the models to generalize\nto. In this paper, we propose a novel unsupervised adaptation method that\nlearns to synthesize labeled data for the target domain from unlabeled\nin-domain data and labeled out-of-domain data. We first learn without\nsupervision an interpretable latent representation of speech that encodes\nlinguistic and nuisance factors (e.g., speaker and channel) using different\nlatent variables. To transform a labeled out-of-domain utterance without\naltering its transcript, we transform the latent nuisance variables while\nmaintaining the linguistic variables. To demonstrate our approach, we focus on\na channel mismatch setting, where the domain of interest is distant\nconversational speech, and labels are only available for close-talking speech.\nOur proposed method is evaluated on the AMI dataset, outperforming all\nbaselines and bridging the gap between unadapted and in-domain models by over\n77% without using any parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1806.05337v2",
            "date": "2018-06-14 02:41:03+00:00",
            "title": "Hierarchical interpretations for neural network predictions",
            "authors": [
                "Chandan Singh",
                "W. James Murdoch",
                "Bin Yu"
            ],
            "abstract": "Deep neural networks (DNNs) have achieved impressive predictive performance\ndue to their ability to learn complex, non-linear relationships between\nvariables. However, the inability to effectively visualize these relationships\nhas led to DNNs being characterized as black boxes and consequently limited\ntheir applications. To ameliorate this problem, we introduce the use of\nhierarchical interpretations to explain DNN predictions through our proposed\nmethod, agglomerative contextual decomposition (ACD). Given a prediction from a\ntrained DNN, ACD produces a hierarchical clustering of the input features,\nalong with the contribution of each cluster to the final prediction. This\nhierarchy is optimized to identify clusters of features that the DNN learned\nare predictive. Using examples from Stanford Sentiment Treebank and ImageNet,\nwe show that ACD is effective at diagnosing incorrect predictions and\nidentifying dataset bias. Through human experiments, we demonstrate that ACD\nenables users both to identify the more accurate of two DNNs and to better\ntrust a DNN's outputs. We also find that ACD's hierarchy is largely robust to\nadversarial perturbations, implying that it captures fundamental aspects of the\ninput and ignores spurious noise.",
            "journal ref": "ICLR 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.02599v1",
            "date": "2018-07-07 01:14:10+00:00",
            "title": "From Text to Topics in Healthcare Records: An Unsupervised Graph Partitioning Methodology",
            "authors": [
                "M. Tarik Altuncu",
                "Erik Mayer",
                "Sophia N. Yaliraki",
                "Mauricio Barahona"
            ],
            "abstract": "Electronic Healthcare Records contain large volumes of unstructured data,\nincluding extensive free text. Yet this source of detailed information often\nremains under-used because of a lack of methodologies to extract interpretable\ncontent in a timely manner. Here we apply network-theoretical tools to analyse\nfree text in Hospital Patient Incident reports from the National Health\nService, to find clusters of documents with similar content in an unsupervised\nmanner at different levels of resolution. We combine deep neural network\nparagraph vector text-embedding with multiscale Markov Stability community\ndetection applied to a sparsified similarity graph of document vectors, and\nshowcase the approach on incident reports from Imperial College Healthcare NHS\nTrust, London. The multiscale community structure reveals different levels of\nmeaning in the topics of the dataset, as shown by descriptive terms extracted\nfrom the clusters of records. We also compare a posteriori against hand-coded\ncategories assigned by healthcare personnel, and show that our approach\noutperforms LDA-based models. Our content clusters exhibit good correspondence\nwith two levels of hand-coded categories, yet they also provide further medical\ndetail in certain areas and reveal complementary descriptors of incidents\nbeyond the external classification taxonomy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.03100v3",
            "date": "2018-07-09 13:20:28+00:00",
            "title": "Robust Text-to-SQL Generation with Execution-Guided Decoding",
            "authors": [
                "Chenglong Wang",
                "Kedar Tatwawadi",
                "Marc Brockschmidt",
                "Po-Sen Huang",
                "Yi Mao",
                "Oleksandr Polozov",
                "Rishabh Singh"
            ],
            "abstract": "We consider the problem of neural semantic parsing, which translates natural\nlanguage questions into executable SQL queries. We introduce a new mechanism,\nexecution guidance, to leverage the semantics of SQL. It detects and excludes\nfaulty programs during the decoding procedure by conditioning on the execution\nof partially generated program. The mechanism can be used with any\nautoregressive generative model, which we demonstrate on four state-of-the-art\nrecurrent or template-based semantic parsing models. We demonstrate that\nexecution guidance universally improves model performance on various\ntext-to-SQL datasets with different scales and query complexity: WikiSQL, ATIS,\nand GeoQuery. As a result, we achieve new state-of-the-art execution accuracy\nof 83.8% on WikiSQL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.04723v1",
            "date": "2018-07-12 16:59:28+00:00",
            "title": "The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach",
            "authors": [
                "Iulian Vlad Serban",
                "Chinnadhurai Sankar",
                "Michael Pieper",
                "Joelle Pineau",
                "Yoshua Bengio"
            ],
            "abstract": "Deep reinforcement learning has recently shown many impressive successes.\nHowever, one major obstacle towards applying such methods to real-world\nproblems is their lack of data-efficiency. To this end, we propose the\nBottleneck Simulator: a model-based reinforcement learning method which\ncombines a learned, factorized transition model of the environment with rollout\nsimulations to learn an effective policy from few examples. The learned\ntransition model employs an abstract, discrete (bottleneck) state, which\nincreases sample efficiency by reducing the number of model parameters and by\nexploiting structural properties of the environment. We provide a mathematical\nanalysis of the Bottleneck Simulator in terms of fixed points of the learned\npolicy, which reveals how performance is affected by four distinct sources of\nerror: an error related to the abstract space structure, an error related to\nthe transition model estimation variance, an error related to the transition\nmodel estimation bias, and an error related to the transition model class bias.\nFinally, we evaluate the Bottleneck Simulator on two natural language\nprocessing tasks: a text adventure game and a real-world, complex dialogue\nresponse selection task. On both tasks, the Bottleneck Simulator yields\nexcellent performance beating competing approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.06538v6",
            "date": "2018-07-17 16:34:47+00:00",
            "title": "Cavity Filling: Pseudo-Feature Generation for Multi-Class Imbalanced Data Problems in Deep Learning",
            "authors": [
                "Tomohiko Konno",
                "Michiaki Iwazume"
            ],
            "abstract": "Herein, we generate pseudo-features based on the multivariate probability\ndistributions obtained from the feature maps in layers of trained deep neural\nnetworks. Further, we augment the minor-class data based on these generated\npseudo-features to overcome the imbalanced data problems. The proposed method,\ni.e., cavity filling, improves the deep learning capabilities in several\nproblems because all the real-world data are observed to be imbalanced.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.07281v3",
            "date": "2018-07-19 08:15:41+00:00",
            "title": "ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech",
            "authors": [
                "Wei Ping",
                "Kainan Peng",
                "Jitong Chen"
            ],
            "abstract": "In this work, we propose a new solution for parallel wave generation by\nWaveNet. In contrast to parallel WaveNet (van den Oord et al., 2018), we\ndistill a Gaussian inverse autoregressive flow from the autoregressive WaveNet\nby minimizing a regularized KL divergence between their highly-peaked output\ndistributions. Our method computes the KL divergence in closed-form, which\nsimplifies the training algorithm and provides very efficient distillation. In\naddition, we introduce the first text-to-wave neural architecture for speech\nsynthesis, which is fully convolutional and enables fast end-to-end training\nfrom scratch. It significantly outperforms the previous pipeline that connects\na text-to-spectrogram model to a separately trained WaveNet (Ping et al.,\n2018). We also successfully distill a parallel waveform synthesizer conditioned\non the hidden representation in this end-to-end model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.08133v1",
            "date": "2018-07-21 11:55:17+00:00",
            "title": "What is not where: the challenge of integrating spatial representations into deep learning architectures",
            "authors": [
                "John D. Kelleher",
                "Simon Dobnik"
            ],
            "abstract": "This paper examines to what degree current deep learning architectures for\nimage caption generation capture spatial language. On the basis of the\nevaluation of examples of generated captions from the literature we argue that\nsystems capture what objects are in the image data but not where these objects\nare located: the captions generated by these systems are the output of a\nlanguage model conditioned on the output of an object detector that cannot\ncapture fine-grained location information. Although language models provide\nuseful knowledge for image captions, we argue that deep learning image\ncaptioning architectures should also model geometric relations between objects.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1807.09844v2",
            "date": "2018-07-21 11:37:15+00:00",
            "title": "Modular Mechanistic Networks: On Bridging Mechanistic and Phenomenological Models with Deep Neural Networks in Natural Language Processing",
            "authors": [
                "Simon Dobnik",
                "John D. Kelleher"
            ],
            "abstract": "Natural language processing (NLP) can be done using either top-down (theory\ndriven) and bottom-up (data driven) approaches, which we call mechanistic and\nphenomenological respectively. The approaches are frequently considered to\nstand in opposition to each other. Examining some recent approaches in deep\nlearning we argue that deep neural networks incorporate both perspectives and,\nfurthermore, that leveraging this aspect of deep learning may help in solving\ncomplex problems within language technology, such as modelling language and\nperception in the domain of spatial cognition.",
            "journal ref": "CLASP Papers in Computational Linguistics Vol. 1: Proceedings of\n  the Conference on Logic and Machine Learning in Natural Language (LaML 2017).\n  ISSN: 2002-9764. URI: http://hdl.handle.net/2077/54911",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1807.11089v1",
            "date": "2018-07-29 17:36:08+00:00",
            "title": "Towards Automatic Speech Identification from Vocal Tract Shape Dynamics in Real-time MRI",
            "authors": [
                "Pramit Saha",
                "Praneeth Srungarapu",
                "Sidney Fels"
            ],
            "abstract": "Vocal tract configurations play a vital role in generating distinguishable\nspeech sounds, by modulating the airflow and creating different resonant\ncavities in speech production. They contain abundant information that can be\nutilized to better understand the underlying speech production mechanism. As a\nstep towards automatic mapping of vocal tract shape geometry to acoustics, this\npaper employs effective video action recognition techniques, like Long-term\nRecurrent Convolutional Networks (LRCN) models, to identify different\nvowel-consonant-vowel (VCV) sequences from dynamic shaping of the vocal tract.\nSuch a model typically combines a CNN based deep hierarchical visual feature\nextractor with Recurrent Networks, that ideally makes the network\nspatio-temporally deep enough to learn the sequential dynamics of a short video\nclip for video classification tasks. We use a database consisting of 2D\nreal-time MRI of vocal tract shaping during VCV utterances by 17 speakers. The\ncomparative performances of this class of algorithms under various parameter\nsettings and for various classification tasks are discussed. Interestingly, the\nresults show a marked difference in the model performance in the context of\nspeech classification with respect to generic sequence or video classification\ntasks.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1809.04214v1",
            "date": "2018-09-12 01:32:11+00:00",
            "title": "Automatic, Personalized, and Flexible Playlist Generation using Reinforcement Learning",
            "authors": [
                "Shun-Yao Shih",
                "Heng-Yu Chi"
            ],
            "abstract": "Songs can be well arranged by professional music curators to form a riveting\nplaylist that creates engaging listening experiences. However, it is\ntime-consuming for curators to timely rearrange these playlists for fitting\ntrends in future. By exploiting the techniques of deep learning and\nreinforcement learning, in this paper, we consider music playlist generation as\na language modeling problem and solve it by the proposed attention language\nmodel with policy gradient. We develop a systematic and interactive approach so\nthat the resulting playlists can be tuned flexibly according to user\npreferences. Considering a playlist as a sequence of words, we first train our\nattention RNN language model on baseline recommended playlists. By optimizing\nsuitable imposed reward functions, the model is thus refined for corresponding\npreferences. The experimental results demonstrate that our approach not only\ngenerates coherent playlists automatically but is also able to flexibly\nrecommend personalized playlists for diversity, novelty and freshness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1809.07559v2",
            "date": "2018-09-20 10:32:22+00:00",
            "title": "Permutation Invariant Gaussian Matrix Models",
            "authors": [
                "Sanjaye Ramgoolam"
            ],
            "abstract": "Permutation invariant Gaussian matrix models were recently developed for\napplications in computational linguistics. A 5-parameter family of models was\nsolved. In this paper, we use a representation theoretic approach to solve the\ngeneral 13-parameter Gaussian model, which can be viewed as a zero-dimensional\nquantum field theory. We express the two linear and eleven quadratic terms in\nthe action in terms of representation theoretic parameters. These parameters\nare coefficients of simple quadratic expressions in terms of appropriate linear\ncombinations of the matrix variables transforming in specific irreducible\nrepresentations of the symmetric group $S_D$ where $D$ is the size of the\nmatrices. They allow the identification of constraints which ensure a\nconvergent Gaussian measure and well-defined expectation values for polynomial\nfunctions of the random matrix at all orders. A graph-theoretic interpretation\nis known to allow the enumeration of permutation invariants of matrices at\nlinear, quadratic and higher orders. We express the expectation values of all\nthe quadratic graph-basis invariants and a selection of cubic and quartic\ninvariants in terms of the representation theoretic parameters of the model.",
            "journal ref": "None",
            "category": "hep-th"
        },
        {
            "url": "http://arxiv.org/abs/1810.01395v2",
            "date": "2018-10-02 17:36:23+00:00",
            "title": "Phasebook and Friends: Leveraging Discrete Representations for Source Separation",
            "authors": [
                "Jonathan Le Roux",
                "Gordon Wichern",
                "Shinji Watanabe",
                "Andy Sarroff",
                "John R. Hershey"
            ],
            "abstract": "Deep learning based speech enhancement and source separation systems have\nrecently reached unprecedented levels of quality, to the point that performance\nis reaching a new ceiling. Most systems rely on estimating the magnitude of a\ntarget source by estimating a real-valued mask to be applied to a\ntime-frequency representation of the mixture signal. A limiting factor in such\napproaches is a lack of phase estimation: the phase of the mixture is most\noften used when reconstructing the estimated time-domain signal. Here, we\npropose \"magbook\", \"phasebook\", and \"combook\", three new types of layers based\non discrete representations that can be used to estimate complex time-frequency\nmasks. Magbook layers extend classical sigmoidal units and a recently\nintroduced convex softmax activation for mask-based magnitude estimation.\nPhasebook layers use a similar structure to give an estimate of the phase mask\nwithout suffering from phase wrapping issues. Combook layers are an alternative\nto the magbook-phasebook combination that directly estimate complex masks. We\npresent various training and inference schemes involving these representations,\nand explain in particular how to include them in an end-to-end learning\nframework. We also present an oracle study to assess upper bounds on\nperformance for various types of masks using discrete phase representations. We\nevaluate the proposed methods on the wsj0-2mix dataset, a well-studied corpus\nfor single-channel speaker-independent speaker separation, matching the\nperformance of state-of-the-art mask-based approaches without requiring\nadditional phase reconstruction steps.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1810.05512v4",
            "date": "2018-10-09 09:41:15+00:00",
            "title": "Federated Learning for Keyword Spotting",
            "authors": [
                "David Leroy",
                "Alice Coucke",
                "Thibaut Lavril",
                "Thibault Gisselbrecht",
                "Joseph Dureau"
            ],
            "abstract": "We propose a practical approach based on federated learning to solve\nout-of-domain issues with continuously running embedded speech-based models\nsuch as wake word detectors. We conduct an extensive empirical study of the\nfederated averaging algorithm for the \"Hey Snips\" wake word based on a\ncrowdsourced dataset that mimics a federation of wake word users. We\nempirically demonstrate that using an adaptive averaging strategy inspired from\nAdam in place of standard weighted model averaging highly reduces the number of\ncommunication rounds required to reach our target performance. The associated\nupstream communication costs per user are estimated at 8 MB, which is a\nreasonable in the context of smart home voice assistants. Additionally, the\ndataset used for these experiments is being open sourced with the aim of\nfostering further transparent research in the application of federated learning\nto speech data.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.07382v3",
            "date": "2018-10-17 04:30:02+00:00",
            "title": "Analysis of Railway Accidents' Narratives Using Deep Learning",
            "authors": [
                "Mojtaba Heidarysafa",
                "Kamran Kowsari",
                "Laura E. Barnes",
                "Donald E. Brown"
            ],
            "abstract": "Automatic understanding of domain specific texts in order to extract useful\nrelationships for later use is a non-trivial task. One such relationship would\nbe between railroad accidents' causes and their correspondent descriptions in\nreports. From 2001 to 2016 rail accidents in the U.S. cost more than $4.6B.\nRailroads involved in accidents are required to submit an accident report to\nthe Federal Railroad Administration (FRA). These reports contain a variety of\nfixed field entries including primary cause of the accidents (a coded variable\nwith 389 values) as well as a narrative field which is a short text description\nof the accident. Although these narratives provide more information than a\nfixed field entry, the terminologies used in these reports are not easy to\nunderstand by a non-expert reader. Therefore, providing an assisting method to\nfill in the primary cause from such domain specific texts(narratives) would\nhelp to label the accidents with more accuracy. Another important question for\ntransportation safety is whether the reported accident cause is consistent with\nnarrative description. To address these questions, we applied deep learning\nmethods together with powerful word embeddings such as Word2Vec and GloVe to\nclassify accident cause values for the primary cause field using the text in\nthe narratives. The results show that such approaches can both accurately\nclassify accident causes based on report narratives and find important\ninconsistencies in accident reporting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1810.07652v1",
            "date": "2018-10-16 09:54:37+00:00",
            "title": "Fine-tuning on Clean Data for End-to-End Speech Translation: FBK @ IWSLT 2018",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Roberto Dess\u00ec",
                "Roldano Cattoni",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "This paper describes FBK's submission to the end-to-end English-German speech\ntranslation task at IWSLT 2018. Our system relies on a state-of-the-art model\nbased on LSTMs and CNNs, where the CNNs are used to reduce the temporal\ndimension of the audio input, which is in general much higher than machine\ntranslation input. Our model was trained only on the audio-to-text parallel\ndata released for the task, and fine-tuned on cleaned subsets of the original\ntraining corpus. The addition of weight normalization and label smoothing\nimproved the baseline system by 1.0 BLEU point on our validation set. The final\nsubmission also featured checkpoint averaging within a training run and\nensemble decoding of models trained during multiple runs. On test data, our\nbest single model obtained a BLEU score of 9.7, while the ensemble obtained a\nBLEU score of 10.24.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1810.12091v1",
            "date": "2018-10-12 12:22:34+00:00",
            "title": "Embedding Geographic Locations for Modelling the Natural Environment using Flickr Tags and Structured Data",
            "authors": [
                "Shelan S. Jeawak",
                "Christopher B. Jones",
                "Steven Schockaert"
            ],
            "abstract": "Meta-data from photo-sharing websites such as Flickr can be used to obtain\nrich bag-of-words descriptions of geographic locations, which have proven\nvaluable, among others, for modelling and predicting ecological features. One\nimportant insight from previous work is that the descriptions obtained from\nFlickr tend to be complementary to the structured information that is available\nfrom traditional scientific resources. To better integrate these two diverse\nsources of information, in this paper we consider a method for learning vector\nspace embeddings of geographic locations. We show experimentally that this\nmethod improves on existing approaches, especially in cases where structured\ninformation is available.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1810.12730v2",
            "date": "2018-10-29 15:20:32+00:00",
            "title": "Audiovisual speaker conversion: jointly and simultaneously transforming facial expression and acoustic characteristics",
            "authors": [
                "Fuming Fang",
                "Xin Wang",
                "Junichi Yamagishi",
                "Isao Echizen"
            ],
            "abstract": "An audiovisual speaker conversion method is presented for simultaneously\ntransforming the facial expressions and voice of a source speaker into those of\na target speaker. Transforming the facial and acoustic features together makes\nit possible for the converted voice and facial expressions to be highly\ncorrelated and for the generated target speaker to appear and sound natural. It\nuses three neural networks: a conversion network that fuses and transforms the\nfacial and acoustic features, a waveform generation network that produces the\nwaveform from both the converted facial and acoustic features, and an image\nreconstruction network that outputs an RGB facial image also based on both the\nconverted features. The results of experiments using an emotional audiovisual\ndatabase showed that the proposed method achieved significantly higher\nnaturalness compared with one that separately transformed acoustic and facial\nfeatures.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.02736v3",
            "date": "2018-11-07 02:38:49+00:00",
            "title": "Learning acoustic word embeddings with phonetically associated triplet network",
            "authors": [
                "Hyungjun Lim",
                "Younggwan Kim",
                "Youngmoon Jung",
                "Myunghun Jung",
                "Hoirin Kim"
            ],
            "abstract": "Previous researches on acoustic word embeddings used in query-by-example\nspoken term detection have shown remarkable performance improvements when using\na triplet network. However, the triplet network is trained using only a limited\ninformation about acoustic similarity between words. In this paper, we propose\na novel architecture, phonetically associated triplet network (PATN), which\naims at increasing discriminative power of acoustic word embeddings by\nutilizing phonetic information as well as word identity. The proposed model is\nlearned to minimize a combined loss function that was made by introducing a\ncross entropy loss to the lower layer of LSTM-based triplet network. We\nobserved that the proposed method performs significantly better than the\nbaseline triplet network on a word discrimination task with the WSJ dataset\nresulting in over 20% relative improvement in recall rate at 1.0 false alarm\nper hour. Finally, we examined the generalization ability by conducting the\nout-of-domain test on the RM dataset.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.03700v2",
            "date": "2018-11-08 22:37:55+00:00",
            "title": "A Comparison of Lattice-free Discriminative Training Criteria for Purely Sequence-Trained Neural Network Acoustic Models",
            "authors": [
                "Chao Weng",
                "Dong Yu"
            ],
            "abstract": "In this work, three lattice-free (LF) discriminative training criteria for\npurely sequence-trained neural network acoustic models are compared on LVCSR\ntasks, namely maximum mutual information (MMI), boosted maximum mutual\ninformation (bMMI) and state-level minimum Bayes risk (sMBR). We demonstrate\nthat, analogous to LF-MMI, a neural network acoustic model can also be trained\nfrom scratch using LF-bMMI or LF-sMBR criteria respectively without the need of\ncross-entropy pre-training. Furthermore, experimental results on\nSwitchboard-300hrs and Switchboard+Fisher-2100hrs datasets show that models\ntrained with LF-bMMI consistently outperform those trained with plain LF-MMI\nand achieve a relative word error rate (WER) reduction of 5% over competitive\ntemporal convolution projected LSTM (TDNN-LSTMP) LF-MMI baselines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.04179v2",
            "date": "2018-11-10 02:57:38+00:00",
            "title": "Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction",
            "authors": [
                "Valts Blukis",
                "Dipendra Misra",
                "Ross A. Knepper",
                "Yoav Artzi"
            ],
            "abstract": "We propose an approach for mapping natural language instructions and raw\nobservations to continuous control of a quadcopter drone. Our model predicts\ninterpretable position-visitation distributions indicating where the agent\nshould go during execution and where it should stop, and uses the predicted\ndistributions to select the actions to execute. This two-step model\ndecomposition allows for simple and efficient training using a combination of\nsupervised learning and imitation learning. We evaluate our approach with a\nrealistic drone simulator, and demonstrate absolute task-completion accuracy\nimprovements of 16.85% over two state-of-the-art instruction-following methods.",
            "journal ref": "In Conference on Robot Learning (pp. 505-518) (2018)",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1811.05536v1",
            "date": "2018-11-13 21:45:20+00:00",
            "title": "Staging Human-computer Dialogs: An Application of the Futamura Projections",
            "authors": [
                "Brandon M. Williams",
                "Saverio Perugini"
            ],
            "abstract": "We demonstrate an application of the Futamura Projections to human-computer\ninteraction, and particularly to staging human-computer dialogs. Specifically,\nby providing staging analogs to the classical Futamura Projections, we\ndemonstrate that the Futamura Projections can be applied to the staging of\nhuman-computer dialogs in addition to the execution of programs.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05540v1",
            "date": "2018-11-09 17:12:47+00:00",
            "title": "Native Language Identification using i-vector",
            "authors": [
                "Ahmed Nazim Uddin",
                "Md Ashequr Rahman",
                "Md. Rafidul Islam",
                "Mohammad Ariful Haque"
            ],
            "abstract": "The task of determining a speaker's native language based only on his\nspeeches in a second language is known as Native Language Identification or\nNLI. Due to its increasing applications in various domains of speech signal\nprocessing, this has emerged as an important research area in recent times. In\nthis paper we have proposed an i-vector based approach to develop an automatic\nNLI system using MFCC and GFCC features. For evaluation of our approach, we\nhave tested our framework on the 2016 ComParE Native language sub-challenge\ndataset which has English language speakers from 11 different native language\nbackgrounds. Our proposed method outperforms the baseline system with an\nimprovement in accuracy by 21.95% for the MFCC feature based i-vector framework\nand 22.81% for the GFCC feature based i-vector framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05711v1",
            "date": "2018-11-14 10:08:19+00:00",
            "title": "From Free Text to Clusters of Content in Health Records: An Unsupervised Graph Partitioning Approach",
            "authors": [
                "M. Tarik Altuncu",
                "Erik Mayer",
                "Sophia N. Yaliraki",
                "Mauricio Barahona"
            ],
            "abstract": "Electronic Healthcare records contain large volumes of unstructured data in\ndifferent forms. Free text constitutes a large portion of such data, yet this\nsource of richly detailed information often remains under-used in practice\nbecause of a lack of suitable methodologies to extract interpretable content in\na timely manner. Here we apply network-theoretical tools to the analysis of\nfree text in Hospital Patient Incident reports in the English National Health\nService, to find clusters of reports in an unsupervised manner and at different\nlevels of resolution based directly on the free text descriptions contained\nwithin them. To do so, we combine recently developed deep neural network\ntext-embedding methodologies based on paragraph vectors with multi-scale Markov\nStability community detection applied to a similarity graph of documents\nobtained from sparsified text vector similarities. We showcase the approach\nwith the analysis of incident reports submitted in Imperial College Healthcare\nNHS Trust, London. The multiscale community structure reveals levels of meaning\nwith different resolution in the topics of the dataset, as shown by relevant\ndescriptive terms extracted from the groups of records, as well as by comparing\na posteriori against hand-coded categories assigned by healthcare personnel.\nOur content communities exhibit good correspondence with well-defined\nhand-coded categories, yet our results also provide further medical detail in\ncertain areas as well as revealing complementary descriptors of incidents\nbeyond the external classification. We also discuss how the method can be used\nto monitor reports over time and across different healthcare providers, and to\ndetect emerging trends that fall outside of pre-existing categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.05760v1",
            "date": "2018-10-10 20:51:03+00:00",
            "title": "A Multimodal Approach towards Emotion Recognition of Music using Audio and Lyrical Content",
            "authors": [
                "Aniruddha Bhattacharya",
                "K. V. Kadambari"
            ],
            "abstract": "We propose MoodNet - A Deep Convolutional Neural Network based architecture\nto effectively predict the emotion associated with a piece of music given its\naudio and lyrical content.We evaluate different architectures consisting of\nvarying number of two-dimensional convolutional and subsampling layers,followed\nby dense layers.We use Mel-Spectrograms to represent the audio content and word\nembeddings-specifically 100 dimensional word vectors, to represent the textual\ncontent represented by the lyrics.We feed input data from both modalities to\nour MoodNet architecture.The output from both the modalities are then fused as\na fully connected layer and softmax classfier is used to predict the category\nof emotion.Using F1-score as our metric,our results show excellent performance\nof MoodNet over the two datasets we experimented on-The MIREX Multimodal\ndataset and the Million Song Dataset.Our experiments reflect the hypothesis\nthat more complex models perform better with more training data.We also observe\nthat lyrics outperform audio as a better expressed modality and conclude that\ncombining and using features from multiple modalities for prediction tasks\nresult in superior performance in comparison to using a single modality as\ninput.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1811.07240v2",
            "date": "2018-11-17 22:45:15+00:00",
            "title": "Representation Mixing for TTS Synthesis",
            "authors": [
                "Kyle Kastner",
                "Jo\u00e3o Felipe Santos",
                "Yoshua Bengio",
                "Aaron Courville"
            ],
            "abstract": "Recent character and phoneme-based parametric TTS systems using deep learning\nhave shown strong performance in natural speech generation. However, the choice\nbetween character or phoneme input can create serious limitations for practical\ndeployment, as direct control of pronunciation is crucial in certain cases. We\ndemonstrate a simple method for combining multiple types of linguistic\ninformation in a single encoder, named representation mixing, enabling flexible\nchoice between character, phoneme, or mixed representations during inference.\nExperiments and user studies on a public audiobook corpus show the efficacy of\nour approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.07514v2",
            "date": "2018-11-19 06:04:13+00:00",
            "title": "NSEEN: Neural Semantic Embedding for Entity Normalization",
            "authors": [
                "Shobeir Fakhraei",
                "Joel Mathew",
                "Jose Luis Ambite"
            ],
            "abstract": "Much of human knowledge is encoded in text, available in scientific\npublications, books, and the web. Given the rapid growth of these resources, we\nneed automated methods to extract such knowledge into machine-processable\nstructures, such as knowledge graphs. An important task in this process is\nentity normalization, which consists of mapping noisy entity mentions in text\nto canonical entities in well-known reference sets. However, entity\nnormalization is a challenging problem; there often are many textual forms for\na canonical entity that may not be captured in the reference set, and entities\nmentioned in text may include many syntactic variations, or errors. The problem\nis particularly acute in scientific domains, such as biology. To address this\nproblem, we have developed a general, scalable solution based on a deep Siamese\nneural network model to embed the semantic information about the entities, as\nwell as their syntactic variations. We use these embeddings for fast mapping of\nnew entities to large reference sets, and empirically show the effectiveness of\nour framework in challenging bio-entity normalization datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1811.07684v2",
            "date": "2018-11-19 13:51:10+00:00",
            "title": "Efficient keyword spotting using dilated convolutions and gating",
            "authors": [
                "Alice Coucke",
                "Mohammed Chlieh",
                "Thibault Gisselbrecht",
                "David Leroy",
                "Mathieu Poumeyrol",
                "Thibaut Lavril"
            ],
            "abstract": "We explore the application of end-to-end stateless temporal modeling to\nsmall-footprint keyword spotting as opposed to recurrent networks that model\nlong-term temporal dependencies using internal states. We propose a model\ninspired by the recent success of dilated convolutions in sequence modeling\napplications, allowing to train deeper architectures in resource-constrained\nconfigurations. Gated activations and residual connections are also added,\nfollowing a similar configuration to WaveNet. In addition, we apply a custom\ntarget labeling that back-propagates loss from specific frames of interest,\ntherefore yielding higher accuracy and only requiring to detect the end of the\nkeyword. Our experimental results show that our model outperforms a max-pooling\nloss trained recurrent neural network using LSTM cells, with a significant\ndecrease in false rejection rate. The underlying dataset - \"Hey Snips\"\nutterances recorded by over 2.2K different speakers - has been made publicly\navailable to establish an open reference for wake-word detection.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1811.07901v4",
            "date": "2018-11-19 19:00:01+00:00",
            "title": "On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection",
            "authors": [
                "Vivian Lai",
                "Chenhao Tan"
            ],
            "abstract": "Humans are the final decision makers in critical tasks that involve ethical\nand legal concerns, ranging from recidivism prediction, to medical diagnosis,\nto fighting against fake news. Although machine learning models can sometimes\nachieve impressive performance in these tasks, these tasks are not amenable to\nfull automation. To realize the potential of machine learning for improving\nhuman decisions, it is important to understand how assistance from machine\nlearning models affects human performance and human agency.\n  In this paper, we use deception detection as a testbed and investigate how we\ncan harness explanations and predictions of machine learning models to improve\nhuman performance while retaining human agency. We propose a spectrum between\nfull human agency and full automation, and develop varying levels of machine\nassistance along the spectrum that gradually increase the influence of machine\npredictions. We find that without showing predicted labels, explanations alone\nslightly improve human performance in the end task. In comparison, human\nperformance is greatly improved by showing predicted labels (>20% relative\nimprovement) and can be further improved by explicitly suggesting strong\nmachine performance. Interestingly, when predicted labels are shown,\nexplanations of machine predictions induce a similar level of accuracy as an\nexplicit statement of strong machine performance. Our results demonstrate a\ntradeoff between human performance and human agency and show that explanations\nof machine predictions can moderate this tradeoff.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1811.08853v1",
            "date": "2018-11-21 17:59:56+00:00",
            "title": "Resource Mention Extraction for MOOC Discussion Forums",
            "authors": [
                "Ya-Hui An",
                "Liangming Pan",
                "Min-Yen Kan",
                "Qiang Dong",
                "Yan Fu"
            ],
            "abstract": "In discussions hosted on discussion forums for MOOCs, references to online\nlearning resources are often of central importance. They contextualize the\ndiscussion, anchoring the discussion participants' presentation of the issues\nand their understanding. However they are usually mentioned in free text,\nwithout appropriate hyperlinking to their associated resource. Automated\nlearning resource mention hyperlinking and categorization will facilitate\ndiscussion and searching within MOOC forums, and also benefit the\ncontextualization of such resources across disparate views. We propose the\nnovel problem of learning resource mention identification in MOOC forums. As\nthis is a novel task with no publicly available data, we first contribute a\nlarge-scale labeled dataset, dubbed the Forum Resource Mention (FoRM) dataset,\nto facilitate our current research and future research on this task. We then\nformulate this task as a sequence tagging problem and investigate solution\narchitectures to address the problem. Importantly, we identify two major\nchallenges that hinder the application of sequence tagging models to the task:\n(1) the diversity of resource mention expression, and (2) long-range contextual\ndependencies. We address these challenges by incorporating character-level and\nthread context information into a LSTM-CRF model. First, we incorporate a\ncharacter encoder to address the out-of-vocabulary problem caused by the\ndiversity of mention expressions. Second, to address the context dependency\nchallenge, we encode thread contexts using an RNN-based context encoder, and\napply the attention mechanism to selectively leverage useful context\ninformation during sequence tagging. Experiments on FoRM show that the proposed\nmethod improves the baseline deep sequence tagging models notably,\nsignificantly bettering performance on instances that exemplify the two\nchallenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.10561v1",
            "date": "2018-11-26 18:06:36+00:00",
            "title": "CLEAR: A Dataset for Compositional Language and Elementary Acoustic Reasoning",
            "authors": [
                "Jerome Abdelnour",
                "Giampiero Salvi",
                "Jean Rouat"
            ],
            "abstract": "We introduce the task of acoustic question answering (AQA) in the area of\nacoustic reasoning. In this task an agent learns to answer questions on the\nbasis of acoustic context. In order to promote research in this area, we\npropose a data generation paradigm adapted from CLEVR (Johnson et al. 2017). We\ngenerate acoustic scenes by leveraging a bank elementary sounds. We also\nprovide a number of functional programs that can be used to compose questions\nand answers that exploit the relationships between the attributes of the\nelementary sounds in each scene. We provide AQA datasets of various sizes as\nwell as the data generation code. As a preliminary experiment to validate our\ndata, we report the accuracy of current state of the art visual question\nanswering models when they are applied to the AQA task without modifications.\nAlthough there is a plethora of question answering tasks based on text, image\nor video data, to our knowledge, we are the first to propose answering\nquestions directly on audio streams. We hope this contribution will facilitate\nthe development of research in the area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1811.12181v1",
            "date": "2018-11-26 21:09:20+00:00",
            "title": "What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning",
            "authors": [
                "Irene Li",
                "Alexander R. Fabbri",
                "Robert R. Tung",
                "Dragomir R. Radev"
            ],
            "abstract": "Recent years have witnessed the rising popularity of Natural Language\nProcessing (NLP) and related fields such as Artificial Intelligence (AI) and\nMachine Learning (ML). Many online courses and resources are available even for\nthose without a strong background in the field. Often the student is curious\nabout a specific topic but does not quite know where to begin studying. To\nanswer the question of \"what should one learn first,\" we apply an\nembedding-based method to learn prerequisite relations for course concepts in\nthe domain of NLP. We introduce LectureBank, a dataset containing 1,352 English\nlecture files collected from university courses which are each classified\naccording to an existing taxonomy as well as 208 manually-labeled prerequisite\nrelation topics, which is publicly available. The dataset will be useful for\neducational purposes such as lecture preparation and organization as well as\napplications such as reading list generation. Additionally, we experiment with\nneural graph-based networks and non-neural classifiers to learn these\nprerequisite relations from our dataset.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1811.12254v1",
            "date": "2018-11-29 15:37:45+00:00",
            "title": "The Effect of Heterogeneous Data for Alzheimer's Disease Detection from Speech",
            "authors": [
                "Aparna Balagopalan",
                "Jekaterina Novikova",
                "Frank Rudzicz",
                "Marzyeh Ghassemi"
            ],
            "abstract": "Speech datasets for identifying Alzheimer's disease (AD) are generally\nrestricted to participants performing a single task, e.g. describing an image\nshown to them. As a result, models trained on linguistic features derived from\nsuch datasets may not be generalizable across tasks. Building on prior work\ndemonstrating that same-task data of healthy participants helps improve AD\ndetection on a single-task dataset of pathological speech, we augment an\nAD-specific dataset consisting of subjects describing a picture with multi-task\nhealthy data. We demonstrate that normative data from multiple speech-based\ntasks helps improve AD detection by up to 9%. Visualization of decision\nboundaries reveals that models trained on a combination of structured picture\ndescriptions and unstructured conversational speech have the least out-of-task\nerror and show the most potential to generalize to multiple tasks. We analyze\nthe impact of age of the added samples and if they affect fairness in\nclassification. We also provide explanations for a possible inductive bias\neffect across tasks using model-agnostic feature anchors. This work highlights\nthe need for heterogeneous datasets for encoding changes in multiple facets of\ncognition and for developing a task-independent AD detection model.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.00271v2",
            "date": "2018-12-01 21:48:28+00:00",
            "title": "Learning Speaker Representations with Mutual Information",
            "authors": [
                "Mirco Ravanelli",
                "Yoshua Bengio"
            ],
            "abstract": "Learning good representations is of crucial importance in deep learning.\nMutual Information (MI) or similar measures of statistical dependence are\npromising tools for learning these representations in an unsupervised way. Even\nthough the mutual information between two random variables is hard to measure\ndirectly in high dimensional spaces, some recent studies have shown that an\nimplicit optimization of MI can be achieved with an encoder-discriminator\narchitecture similar to that of Generative Adversarial Networks (GANs). In this\nwork, we learn representations that capture speaker identities by maximizing\nthe mutual information between the encoded representations of chunks of speech\nrandomly sampled from the same sentence. The proposed encoder relies on the\nSincNet architecture and transforms raw speech waveform into a compact feature\nvector. The discriminator is fed by either positive samples (of the joint\ndistribution of encoded chunks) or negative samples (from the product of the\nmarginals) and is trained to separate them. We report experiments showing that\nthis approach effectively learns useful speaker representations, leading to\npromising results on speaker identification and verification tasks. Our\nexperiments consider both unsupervised and semi-supervised settings and compare\nthe performance achieved with different objective functions.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.01995v4",
            "date": "2018-12-03 05:30:34+00:00",
            "title": "Deep Learning Model for Finding New Superconductors",
            "authors": [
                "Tomohiko Konno",
                "Hodaka Kurokawa",
                "Fuyuki Nabeshima",
                "Yuki Sakishita",
                "Ryo Ogawa",
                "Iwao Hosako",
                "Atsutaka Maeda"
            ],
            "abstract": "Exploration of new superconductors still relies on the experience and\nintuition of experts and is largely a process of experimental trial and error.\nIn one study, only 3% of the candidate materials showed superconductivity.\nHere, we report the first deep learning model for finding new superconductors.\nWe introduced the method named \"reading periodic table\" which represented the\nperiodic table in a way that allows deep learning to learn to read the periodic\ntable and to learn the law of elements for the purpose of discovering novel\nsuperconductors that are outside the training data. It is recognized that it is\ndifficult for deep learning to predict something outside the training data.\nAlthough we used only the chemical composition of materials as information, we\nobtained an $R^{2}$ value of 0.92 for predicting $T_\\text{c}$ for materials in\na database of superconductors. We also introduced the method named \"garbage-in\"\nto create synthetic data of non-superconductors that do not exist.\nNon-superconductors are not reported, but the data must be required for deep\nlearning to distinguish between superconductors and non-superconductors. We\nobtained three remarkable results. The deep learning can predict\nsuperconductivity for a material with a precision of 62%, which shows the\nusefulness of the model; it found the recently discovered superconductor CaBi2\nand another one Hf0.5Nb0.2V2Zr0.3, neither of which is in the superconductor\ndatabase; and it found Fe-based high-temperature superconductors (discovered in\n2008) from the training data before 2008. These results open the way for the\ndiscovery of new high-temperature superconductor families. The candidate\nmaterials list, data, and method are openly available from the link\nhttps://github.com/tomo835g/Deep-Learning-to-find-Superconductors.",
            "journal ref": "Phys. Rev. B 103, 014509 (2021)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.02309v1",
            "date": "2018-11-28 07:48:27+00:00",
            "title": "The MeSH-gram Neural Network Model: Extending Word Embedding Vectors with MeSH Concepts for UMLS Semantic Similarity and Relatedness in the Biomedical Domain",
            "authors": [
                "Sa\u00efd Abdedda\u00efm",
                "Sylvestre Vimard",
                "Lina Fatima Soualmia"
            ],
            "abstract": "Eliciting semantic similarity between concepts in the biomedical domain\nremains a challenging task. Recent approaches founded on embedding vectors have\ngained in popularity as they risen to efficiently capture semantic\nrelationships The underlying idea is that two words that have close meaning\ngather similar contexts. In this study, we propose a new neural network model\nnamed MeSH-gram which relies on a straighforward approach that extends the\nskip-gram neural network model by considering MeSH (Medical Subject Headings)\ndescriptors instead words. Trained on publicly available corpus PubMed MEDLINE,\nMeSH-gram is evaluated on reference standards manually annotated for semantic\nsimilarity. MeSH-gram is first compared to skip-gram with vectors of size 300\nand at several windows contexts. A deeper comparison is performed with tewenty\nexisting models. All the obtained results of Spearman's rank correlations\nbetween human scores and computed similarities show that MeSH-gram outperforms\nthe skip-gram model, and is comparable to the best methods but that need more\ncomputation and external resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1812.03483v3",
            "date": "2018-12-09 13:18:02+00:00",
            "title": "To Reverse the Gradient or Not: An Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition",
            "authors": [
                "Yossi Adi",
                "Neil Zeghidour",
                "Ronan Collobert",
                "Nicolas Usunier",
                "Vitaliy Liptchinsky",
                "Gabriel Synnaeve"
            ],
            "abstract": "Transcribed datasets typically contain speaker identity for each instance in\nthe data. We investigate two ways to incorporate this information during\ntraining: Multi-Task Learning and Adversarial Learning. In multi-task learning,\nthe goal is speaker prediction; we expect a performance improvement with this\njoint training if the two tasks of speech recognition and speaker recognition\nshare a common set of underlying features. In contrast, adversarial learning is\na means to learn representations invariant to the speaker. We then expect\nbetter performance if this learnt invariance helps generalizing to new\nspeakers. While the two approaches seem natural in the context of speech\nrecognition, they are incompatible because they correspond to opposite\ngradients back-propagated to the model. In order to better understand the\neffect of these approaches in terms of error rates, we compare both strategies\nin controlled settings. Moreover, we explore the use of additional\nuntranscribed data in a semi-supervised, adversarial learning manner to improve\nerror rates. Our results show that deep models trained on big datasets already\ndevelop invariant representations to speakers without any auxiliary loss. When\nconsidering adversarial learning and multi-task learning, the impact on the\nacoustic model seems minor. However, models trained in a semi-supervised manner\ncan improve error-rates.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.04155v4",
            "date": "2018-12-10 23:48:25+00:00",
            "title": "Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention",
            "authors": [
                "Khanh Nguyen",
                "Debadeepta Dey",
                "Chris Brockett",
                "Bill Dolan"
            ],
            "abstract": "We present Vision-based Navigation with Language-based Assistance (VNLA), a\ngrounded vision-language task where an agent with visual perception is guided\nvia language to find objects in photorealistic indoor environments. The task\nemulates a real-world scenario in that (a) the requester may not know how to\nnavigate to the target objects and thus makes requests by only specifying\nhigh-level end-goals, and (b) the agent is capable of sensing when it is lost\nand querying an advisor, who is more qualified at the task, to obtain language\nsubgoals to make progress. To model language-based assistance, we develop a\ngeneral framework termed Imitation Learning with Indirect Intervention (I3L),\nand propose a solution that is effective on the VNLA task. Empirical results\nshow that this approach significantly improves the success rate of the learning\nagent over other baselines in both seen and unseen environments. Our code and\ndata are publicly available at https://github.com/debadeepta/vnla .",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.05710v5",
            "date": "2018-12-12 05:17:23+00:00",
            "title": "FPETS : Fully Parallel End-to-End Text-to-Speech System",
            "authors": [
                "Dabiao Ma",
                "Zhiba Su",
                "Wenxuan Wang",
                "Yuhao Lu"
            ],
            "abstract": "End-to-end Text-to-speech (TTS) system can greatly improve the quality of\nsynthesised speech. But it usually suffers form high time latency due to its\nauto-regressive structure. And the synthesised speech may also suffer from some\nerror modes, e.g. repeated words, mispronunciations, and skipped words. In this\npaper, we propose a novel non-autoregressive, fully parallel end-to-end TTS\nsystem (FPETS). It utilizes a new alignment model and the recently proposed\nU-shape convolutional structure, UFANS. Different from RNN, UFANS can capture\nlong term information in a fully parallel manner. Trainable position encoding\nand two-step training strategy are used for learning better alignments.\nExperimental results show FPETS utilizes the power of parallel computation and\nreaches a significant speed up of inference compared with state-of-the-art\nend-to-end TTS systems. More specifically, FPETS is 600X faster than Tacotron2,\n50X faster than DCTTS and 10X faster than Deep Voice3. And FPETS can generates\naudios with equal or better quality and fewer errors comparing with other\nsystem. As far as we know, FPETS is the first end-to-end TTS system which is\nfully parallel.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.07805v1",
            "date": "2018-12-19 08:33:31+00:00",
            "title": "Unifying Topic, Sentiment & Preference in an HDP-Based Rating Regression Model for Online Reviews",
            "authors": [
                "Zheng Chen",
                "Yong Zhang",
                "Yue Shang",
                "Xiaohua Hu"
            ],
            "abstract": "This paper proposes a new HDP based online review rating regression model\nnamed Topic-Sentiment-Preference Regression Analysis (TSPRA). TSPRA combines\ntopics (i.e. product aspects), word sentiment and user preference as regression\nfactors, and is able to perform topic clustering, review rating prediction,\nsentiment analysis and what we invent as \"critical aspect\" analysis altogether\nin one framework. TSPRA extends sentiment approaches by integrating the key\nconcept \"user preference\" in collaborative filtering (CF) models into\nconsideration, while it is distinct from current CF models by decoupling \"user\npreference\" and \"sentiment\" as independent factors. Our experiments conducted\non 22 Amazon datasets show overwhelming better performance in rating\npredication against a state-of-art model FLAME (2015) in terms of error,\nPearson's Correlation and number of inverted pairs. For sentiment analysis, we\ncompare the derived word sentiments against a public sentiment resource\nSenticNet3 and our sentiment estimations clearly make more sense in the context\nof online reviews. Last, as a result of the de-correlation of \"user preference\"\nfrom \"sentiment\", TSPRA is able to evaluate a new concept \"critical aspects\",\ndefined as the product aspects seriously concerned by users but negatively\ncommented in reviews. Improvement to such \"critical aspects\" could be most\neffective to enhance user experience.",
            "journal ref": "Asian Conference on Machine Learning. 2016",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.07809v2",
            "date": "2018-12-19 08:38:21+00:00",
            "title": "Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities",
            "authors": [
                "Hai Pham",
                "Paul Pu Liang",
                "Thomas Manzini",
                "Louis-Philippe Morency",
                "Barnabas Poczos"
            ],
            "abstract": "Multimodal sentiment analysis is a core research area that studies speaker\nsentiment expressed from the language, visual, and acoustic modalities. The\ncentral challenge in multimodal learning involves inferring joint\nrepresentations that can process and relate information from these modalities.\nHowever, existing work learns joint representations by requiring all modalities\nas input and as a result, the learned representations may be sensitive to noisy\nor missing modalities at test time. With the recent success of sequence to\nsequence (Seq2Seq) models in machine translation, there is an opportunity to\nexplore new ways of learning joint representations that may not require all\ninput modalities at test time. In this paper, we propose a method to learn\nrobust joint representations by translating between modalities. Our method is\nbased on the key insight that translation from a source to a target modality\nprovides a method of learning joint representations using only the source\nmodality as input. We augment modality translations with a cycle consistency\nloss to ensure that our joint representations retain maximal information from\nall modalities. Once our translation model is trained with paired multimodal\ndata, we only need data from the source modality at test time for final\nsentiment prediction. This ensures that our model remains robust from\nperturbations or missing information in the other modalities. We train our\nmodel with a coupled translation-prediction objective and it achieves new\nstate-of-the-art results on multimodal sentiment analysis datasets: CMU-MOSI,\nICT-MMMO, and YouTube. Additional experiments show that our model learns\nincreasingly discriminative joint representations with more input modalities\nwhile maintaining robustness to missing or perturbed modalities.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1812.09323v1",
            "date": "2018-12-23 01:58:39+00:00",
            "title": "Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching",
            "authors": [
                "Chih-Kuan Yeh",
                "Jianshu Chen",
                "Chengzhu Yu",
                "Dong Yu"
            ],
            "abstract": "We consider the problem of training speech recognition systems without using\nany labeled data, under the assumption that the learner can only access to the\ninput utterances and a phoneme language model estimated from a non-overlapping\ncorpus. We propose a fully unsupervised learning algorithm that alternates\nbetween solving two sub-problems: (i) learn a phoneme classifier for a given\nset of phoneme segmentation boundaries, and (ii) refining the phoneme\nboundaries based on a given classifier. To solve the first sub-problem, we\nintroduce a novel unsupervised cost function named Segmental Empirical Output\nDistribution Matching, which generalizes the work in (Liu et al., 2017) to\nsegmental structures. For the second sub-problem, we develop an approximate MAP\napproach to refining the boundaries obtained from Wang et al. (2017).\nExperimental results on TIMIT dataset demonstrate the success of this fully\nunsupervised phoneme recognition system, which achieves a phone error rate\n(PER) of 41.6%. Although it is still far away from the state-of-the-art\nsupervised systems, we show that with oracle boundaries and matching language\nmodel, the PER could be improved to 32.5%.This performance approaches the\nsupervised system of the same model architecture, demonstrating the great\npotential of the proposed method.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1812.10061v1",
            "date": "2018-12-25 08:02:01+00:00",
            "title": "Noise Flooding for Detecting Audio Adversarial Examples Against Automatic Speech Recognition",
            "authors": [
                "Krishan Rajaratnam",
                "Jugal Kalita"
            ],
            "abstract": "Neural models enjoy widespread use across a variety of tasks and have grown\nto become crucial components of many industrial systems. Despite their\neffectiveness and extensive popularity, they are not without their exploitable\nflaws. Initially applied to computer vision systems, the generation of\nadversarial examples is a process in which seemingly imperceptible\nperturbations are made to an image, with the purpose of inducing a deep\nlearning based classifier to misclassify the image. Due to recent trends in\nspeech processing, this has become a noticeable issue in speech recognition\nmodels. In late 2017, an attack was shown to be quite effective against the\nSpeech Commands classification model. Limited-vocabulary speech classifiers,\nsuch as the Speech Commands model, are used quite frequently in a variety of\napplications, particularly in managing automated attendants in telephony\ncontexts. As such, adversarial examples produced by this attack could have\nreal-world consequences. While previous work in defending against these\nadversarial examples has investigated using audio preprocessing to reduce or\ndistort adversarial noise, this work explores the idea of flooding particular\nfrequency bands of an audio signal with random noise in order to detect\nadversarial examples. This technique of flooding, which does not require\nretraining or modifying the model, is inspired by work done in computer vision\nand builds on the idea that speech classifiers are relatively robust to natural\nnoise. A combined defense incorporating 5 different frequency bands for\nflooding the signal with noise outperformed other existing defenses in the\naudio space, detecting adversarial examples with 91.8% precision and 93.5%\nrecall.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1812.10479v1",
            "date": "2018-12-25 14:35:08+00:00",
            "title": "Multimodal deep learning for short-term stock volatility prediction",
            "authors": [
                "Marcelo Sardelich",
                "Suresh Manandhar"
            ],
            "abstract": "Stock market volatility forecasting is a task relevant to assessing market\nrisk. We investigate the interaction between news and prices for the\none-day-ahead volatility prediction using state-of-the-art deep learning\napproaches. The proposed models are trained either end-to-end or using sentence\nencoders transfered from other tasks. We evaluate a broad range of stock market\nsectors, namely Consumer Staples, Energy, Utilities, Heathcare, and Financials.\nOur experimental results show that adding news improves the volatility\nforecasting as compared to the mainstream models that rely only on price data.\nIn particular, our model outperforms the widely-recognized GARCH(1,1) model for\nall sectors in terms of coefficient of determination $R^2$, $MSE$ and $MAE$,\nachieving the best performance when training from both news and price data.",
            "journal ref": "None",
            "category": "q-fin.ST"
        },
        {
            "url": "http://arxiv.org/abs/1804.10188v7",
            "date": "2018-04-26 17:39:28+00:00",
            "title": "Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach",
            "authors": [
                "Sahil Garg",
                "Irina Rish",
                "Guillermo Cecchi",
                "Palash Goyal",
                "Sarik Ghazarian",
                "Shuyang Gao",
                "Greg Ver Steeg",
                "Aram Galstyan"
            ],
            "abstract": "We propose a novel dialogue modeling framework, the first-ever nonparametric\nkernel functions based approach for dialogue modeling, which learns kernelized\nhashcodes as compressed text representations; unlike traditional deep learning\nmodels, it handles well relatively small datasets, while also scaling to large\nones. We also derive a novel lower bound on mutual information, used as a\nmodel-selection criterion favoring representations with better alignment\nbetween the utterances of participants in a collaborative dialogue setting, as\nwell as higher predictability of the generated responses. As demonstrated on\nthree real-life datasets, including prominently psychotherapy sessions, the\nproposed approach significantly outperforms several state-of-art neural network\nbased dialogue systems, both in terms of computational efficiency, reducing\ntraining time from days or weeks to hours, and the response quality, achieving\nan order of magnitude improvement over competitors in frequency of being chosen\nas the best model by human evaluators.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1809.04397v1",
            "date": "2018-09-11 05:12:15+00:00",
            "title": "Isolated and Ensemble Audio Preprocessing Methods for Detecting Adversarial Examples against Automatic Speech Recognition",
            "authors": [
                "Krishan Rajaratnam",
                "Kunal Shah",
                "Jugal Kalita"
            ],
            "abstract": "An adversarial attack is an exploitative process in which minute alterations\nare made to natural inputs, causing the inputs to be misclassified by neural\nmodels. In the field of speech recognition, this has become an issue of\nincreasing significance. Although adversarial attacks were originally\nintroduced in computer vision, they have since infiltrated the realm of speech\nrecognition. In 2017, a genetic attack was shown to be quite potent against the\nSpeech Commands Model. Limited-vocabulary speech classifiers, such as the\nSpeech Commands Model, are used in a variety of applications, particularly in\ntelephony; as such, adversarial examples produced by this attack pose as a\nmajor security threat. This paper explores various methods of detecting these\nadversarial examples with combinations of audio preprocessing. One particular\ncombined defense incorporating compressions, speech coding, filtering, and\naudio panning was shown to be quite effective against the attack on the Speech\nCommands Model, detecting audio adversarial examples with 93.5% precision and\n91.2% recall.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1811.10996v1",
            "date": "2018-11-14 15:46:57+00:00",
            "title": "CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling",
            "authors": [
                "Ning Miao",
                "Hao Zhou",
                "Lili Mou",
                "Rui Yan",
                "Lei Li"
            ],
            "abstract": "In real-world applications of natural language generation, there are often\nconstraints on the target sentences in addition to fluency and naturalness\nrequirements. Existing language generation techniques are usually based on\nrecurrent neural networks (RNNs). However, it is non-trivial to impose\nconstraints on RNNs while maintaining generation quality, since RNNs generate\nsentences sequentially (or with beam search) from the first word to the last.\nIn this paper, we propose CGMH, a novel approach using Metropolis-Hastings\nsampling for constrained sentence generation. CGMH allows complicated\nconstraints such as the occurrence of multiple keywords in the target\nsentences, which cannot be handled in traditional RNN-based approaches.\nMoreover, CGMH works in the inference stage, and does not require parallel\ncorpora for training. We evaluate our method on a variety of tasks, including\nkeywords-to-sentence generation, unsupervised sentence paraphrasing, and\nunsupervised sentence error correction. CGMH achieves high performance compared\nwith previous supervised methods for sentence generation. Our code is released\nat https://github.com/NingMiao/CGMH",
            "journal ref": "None",
            "category": "cs.CL"
        }
    ],
    "2019": [
        {
            "url": "http://arxiv.org/abs/1902.00164v1",
            "date": "2019-02-01 03:43:51+00:00",
            "title": "DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension",
            "authors": [
                "Kai Sun",
                "Dian Yu",
                "Jianshu Chen",
                "Dong Yu",
                "Yejin Choi",
                "Claire Cardie"
            ],
            "abstract": "We present DREAM, the first dialogue-based multiple-choice reading\ncomprehension dataset. Collected from English-as-a-foreign-language\nexaminations designed by human experts to evaluate the comprehension level of\nChinese learners of English, our dataset contains 10,197 multiple-choice\nquestions for 6,444 dialogues. In contrast to existing reading comprehension\ndatasets, DREAM is the first to focus on in-depth multi-turn multi-party\ndialogue understanding. DREAM is likely to present significant challenges for\nexisting reading comprehension systems: 84% of answers are non-extractive, 85%\nof questions require reasoning beyond a single sentence, and 34% of questions\nalso involve commonsense knowledge.\n  We apply several popular neural reading comprehension models that primarily\nexploit surface information within the text and find them to, at best, just\nbarely outperform a rule-based approach. We next investigate the effects of\nincorporating dialogue structure and different kinds of general world knowledge\ninto both rule-based and (neural and non-neural) machine learning-based reading\ncomprehension models. Experimental results on the DREAM dataset show the\neffectiveness of dialogue structure and general world knowledge. DREAM will be\navailable at https://dataset.org/dream/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00184v1",
            "date": "2019-02-01 05:02:59+00:00",
            "title": "A Simple Regularization-based Algorithm for Learning Cross-Domain Word Embeddings",
            "authors": [
                "Wei Yang",
                "Wei Lu",
                "Vincent W. Zheng"
            ],
            "abstract": "Learning word embeddings has received a significant amount of attention\nrecently. Often, word embeddings are learned in an unsupervised manner from a\nlarge collection of text. The genre of the text typically plays an important\nrole in the effectiveness of the resulting embeddings. How to effectively train\nword embedding models using data from different domains remains a problem that\nis underexplored. In this paper, we present a simple yet effective method for\nlearning word embeddings based on text from different domains. We demonstrate\nthe effectiveness of our approach through extensive experiments on various\ndown-stream NLP tasks.",
            "journal ref": "D17-1312, 2017, 2898-2904",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00193v4",
            "date": "2019-02-01 05:49:45+00:00",
            "title": "Massively Multilingual Transfer for NER",
            "authors": [
                "Afshin Rahimi",
                "Yuan Li",
                "Trevor Cohn"
            ],
            "abstract": "In cross-lingual transfer, NLP models over one or more source languages are\napplied to a low-resource target language. While most prior work has used a\nsingle source model or a few carefully selected models, here we consider a\n`massive' setting with many such models. This setting raises the problem of\npoor transfer, particularly from distant languages. We propose two techniques\nfor modulating the transfer, suitable for zero-shot or few-shot learning,\nrespectively. Evaluating on named entity recognition, we show that our\ntechniques are much more effective than strong baselines, including standard\nensembling, and our unsupervised method rivals oracle selection of the single\nbest individual model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00330v1",
            "date": "2019-02-01 13:58:37+00:00",
            "title": "Joint Entity Linking with Deep Reinforcement Learning",
            "authors": [
                "Zheng Fang",
                "Yanan Cao",
                "Dongjie Zhang",
                "Qian Li",
                "Zhenyu Zhang",
                "Yanbing Liu"
            ],
            "abstract": "Entity linking is the task of aligning mentions to corresponding entities in\na given knowledge base. Previous studies have highlighted the necessity for\nentity linking systems to capture the global coherence. However, there are two\ncommon weaknesses in previous global models. First, most of them calculate the\npairwise scores between all candidate entities and select the most relevant\ngroup of entities as the final result. In this process, the consistency among\nwrong entities as well as that among right ones are involved, which may\nintroduce noise data and increase the model complexity. Second, the cues of\npreviously disambiguated entities, which could contribute to the disambiguation\nof the subsequent mentions, are usually ignored by previous models. To address\nthese problems, we convert the global linking into a sequence decision problem\nand propose a reinforcement learning model which makes decisions from a global\nperspective. Our model makes full use of the previous referred entities and\nexplores the long-term influence of current selection on subsequent decisions.\nWe conduct experiments on different types of datasets, the results show that\nour model outperforms state-of-the-art systems and has better generalization\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00438v3",
            "date": "2019-02-01 16:23:17+00:00",
            "title": "tax2vec: Constructing Interpretable Features from Taxonomies for Short Text Classification",
            "authors": [
                "Bla\u017e \u0160krlj",
                "Matej Martinc",
                "Jan Kralj",
                "Nada Lavra\u010d",
                "Senja Pollak"
            ],
            "abstract": "The use of background knowledge is largely unexploited in text classification\ntasks. This paper explores word taxonomies as means for constructing new\nsemantic features, which may improve the performance and robustness of the\nlearned classifiers. We propose tax2vec, a parallel algorithm for constructing\ntaxonomy-based features, and demonstrate its use on six short text\nclassification problems: prediction of gender, personality type, age, news\ntopics, drug side effects and drug effectiveness. The constructed semantic\nfeatures, in combination with fast linear classifiers, tested against strong\nbaselines such as hierarchical attention neural networks, achieves comparable\nclassification results on short text documents. The algorithm's performance is\nalso tested in a few-shot learning setting, indicating that the inclusion of\nsemantic features can improve the performance in data-scarce situations. The\ntax2vec capability to extract corpus-specific semantic keywords is also\ndemonstrated. Finally, we investigate the semantic space of potential features,\nwhere we observe a similarity with the well known Zipf's law.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00489v1",
            "date": "2019-02-01 18:22:33+00:00",
            "title": "Human acceptability judgements for extractive sentence compression",
            "authors": [
                "Abram Handler",
                "Brian Dillon",
                "Brendan O'Connor"
            ],
            "abstract": "Recent approaches to English-language sentence compression rely on parallel\ncorpora consisting of sentence-compression pairs. However, a sentence may be\nshortened in many different ways, which each might be suited to the needs of a\nparticular application. Therefore, in this work, we collect and model\ncrowdsourced judgements of the acceptability of many possible sentence\nshortenings. We then show how a model of such judgements can be used to support\na flexible approach to the compression task. We release our model and dataset\nfor future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00496v1",
            "date": "2019-02-01 18:36:09+00:00",
            "title": "Examining the Presence of Gender Bias in Customer Reviews Using Word Embedding",
            "authors": [
                "A. Mishra",
                "H. Mishra",
                "S. Rathee"
            ],
            "abstract": "Humans have entered the age of algorithms. Each minute, algorithms shape\ncountless preferences from suggesting a product to a potential life partner. In\nthe marketplace algorithms are trained to learn consumer preferences from\ncustomer reviews because user-generated reviews are considered the voice of\ncustomers and a valuable source of information to firms. Insights mined from\nreviews play an indispensable role in several business activities ranging from\nproduct recommendation, targeted advertising, promotions, segmentation etc. In\nthis research, we question whether reviews might hold stereotypic gender bias\nthat algorithms learn and propagate Utilizing data from millions of\nobservations and a word embedding approach, GloVe, we show that algorithms\ndesigned to learn from human language output also learn gender bias. We also\nexamine why such biases occur: whether the bias is caused because of a negative\nbias against females or a positive bias for males. We examine the impact of\ngender bias in reviews on choice and conclude with policy implications for\nfemale consumers, especially when they are unaware of the bias, and the ethical\nimplications for firms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00508v1",
            "date": "2019-02-01 18:59:27+00:00",
            "title": "How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions",
            "authors": [
                "Goran Glavas",
                "Robert Litschko",
                "Sebastian Ruder",
                "Ivan Vulic"
            ],
            "abstract": "Cross-lingual word embeddings (CLEs) enable multilingual modeling of meaning\nand facilitate cross-lingual transfer of NLP models. Despite their ubiquitous\nusage in downstream tasks, recent increasingly popular projection-based CLE\nmodels are almost exclusively evaluated on a single task only: bilingual\nlexicon induction (BLI). Even BLI evaluations vary greatly, hindering our\nability to correctly interpret performance and properties of different CLE\nmodels. In this work, we make the first step towards a comprehensive evaluation\nof cross-lingual word embeddings. We thoroughly evaluate both supervised and\nunsupervised CLE models on a large number of language pairs in the BLI task and\nthree downstream tasks, providing new insights concerning the ability of\ncutting-edge CLE models to support cross-lingual NLP. We empirically\ndemonstrate that the performance of CLE models largely depends on the task at\nhand and that optimizing CLE models for BLI can result in deteriorated\ndownstream performance. We indicate the most robust supervised and unsupervised\nCLE models and emphasize the need to reassess existing baselines, which still\ndisplay competitive performance across the board. We hope that our work will\ncatalyze further work on CLE evaluation and model analysis.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00551v1",
            "date": "2019-01-08 06:44:40+00:00",
            "title": "Deconstructing Word Embeddings",
            "authors": [
                "Koushik Varma Kalidindi"
            ],
            "abstract": "A review of Word Embedding Models through a deconstructive approach reveals\ntheir several shortcomings and inconsistencies. These include instability of\nthe vector representations, a distorted analogical reasoning, geometric\nincompatibility with linguistic features, and the inconsistencies in the corpus\ndata. A new theoretical embedding model, Derridian Embedding, is proposed in\nthis paper. Contemporary embedding models are evaluated qualitatively in terms\nof how adequate they are in relation to the capabilities of a Derridian\nEmbedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00555v1",
            "date": "2019-01-31 16:12:14+00:00",
            "title": "Riconoscimento ortografico per apostrofo ed espressioni polirematiche",
            "authors": [
                "Massimiliano Polito"
            ],
            "abstract": "The work presents two algorithms of manipulation and comparison between\nstrings whose purpose is the orthographic recognition of the apostrophe and of\nthe compound expressions. The theory supporting general reasoning refers to the\nbasic concept of EditDistance, the improvements that ensure the achievement of\nthe objective are achieved with the aid of tools borrowed from the use of\ntechniques for processing large amounts of data on distributed platforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00595v3",
            "date": "2019-02-02 00:32:11+00:00",
            "title": "Character-based Surprisal as a Model of Reading Difficulty in the Presence of Error",
            "authors": [
                "Michael Hahn",
                "Frank Keller",
                "Yonatan Bisk",
                "Yonatan Belinkov"
            ],
            "abstract": "Intuitively, human readers cope easily with errors in text; typos,\nmisspelling, word substitutions, etc. do not unduly disrupt natural reading.\nPrevious work indicates that letter transpositions result in increased reading\ntimes, but it is unclear if this effect generalizes to more natural errors. In\nthis paper, we report an eye-tracking study that compares two error types\n(letter transpositions and naturally occurring misspelling) and two error rates\n(10% or 50% of all words contain errors). We find that human readers show\nunimpaired comprehension in spite of these errors, but error words cause more\nreading difficulty than correct words. Also, transpositions are more difficult\nthan misspellings, and a high error rate increases difficulty for all words,\nincluding correct ones. We then present a computational model that uses\ncharacter-based (rather than traditional word-based) surprisal to account for\nthese results. The model explains that transpositions are harder than\nmisspellings because they contain unexpected letter combinations. It also\nexplains the error rate effect: upcoming words are more difficultto predict\nwhen the context is degraded, leading to increased surprisal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00679v1",
            "date": "2019-02-02 09:30:26+00:00",
            "title": "Natural Language Processing, Sentiment Analysis and Clinical Analytics",
            "authors": [
                "Adil Rajput"
            ],
            "abstract": "Recent advances in Big Data has prompted health care practitioners to utilize\nthe data available on social media to discern sentiment and emotions\nexpression. Health Informatics and Clinical Analytics depend heavily on\ninformation gathered from diverse sources. Traditionally, a healthcare\npractitioner will ask a patient to fill out a questionnaire that will form the\nbasis of diagnosing the medical condition. However, medical practitioners have\naccess to many sources of data including the patients writings on various\nmedia. Natural Language Processing (NLP) allows researchers to gather such data\nand analyze it to glean the underlying meaning of such writings. The field of\nsentiment analysis (applied to many other domains) depend heavily on techniques\nutilized by NLP. This work will look into various prevalent theories underlying\nthe NLP field and how they can be leveraged to gather users sentiments on\nsocial media. Such sentiments can be culled over a period of time thus\nminimizing the errors introduced by data input and other stressors.\nFurthermore, we look at some applications of sentiment analysis and application\nof NLP to mental health. The reader will also learn about the NLTK toolkit that\nimplements various NLP theories and how they can make the data scavenging\nprocess a lot easier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00702v1",
            "date": "2019-02-02 11:59:28+00:00",
            "title": "Making a Case for Social Media Corpus for Detecting Depression",
            "authors": [
                "Adil Rajput",
                "Samara Ahmed"
            ],
            "abstract": "The social media platform provides an opportunity to gain valuable insights\ninto user behaviour. Users mimic their internal feelings and emotions in a\ndisinhibited fashion using natural language. Techniques in Natural Language\nProcessing have helped researchers decipher standard documents and cull\ntogether inferences from massive amount of data. A representative corpus is a\nprerequisite for NLP and one of the challenges we face today is the\nnon-standard and noisy language that exists on the internet. Our work focuses\non building a corpus from social media that is focused on detecting mental\nillness. We use depression as a case study and demonstrate the effectiveness of\nusing such a corpus for helping practitioners detect such cases. Our results\nshow a high correlation between our Social Media Corpus and the standard corpus\nfor depression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00756v1",
            "date": "2019-02-02 17:34:19+00:00",
            "title": "Graph Neural Networks with Generated Parameters for Relation Extraction",
            "authors": [
                "Hao Zhu",
                "Yankai Lin",
                "Zhiyuan Liu",
                "Jie Fu",
                "Tat-seng Chua",
                "Maosong Sun"
            ],
            "abstract": "Recently, progress has been made towards improving relational reasoning in\nmachine learning field. Among existing models, graph neural networks (GNNs) is\none of the most effective approaches for multi-hop relational reasoning. In\nfact, multi-hop relational reasoning is indispensable in many natural language\nprocessing tasks such as relation extraction. In this paper, we propose to\ngenerate the parameters of graph neural networks (GP-GNNs) according to natural\nlanguage sentences, which enables GNNs to process relational reasoning on\nunstructured text inputs. We verify GP-GNNs in relation extraction from text.\nExperimental results on a human-annotated dataset and two distantly supervised\ndatasets show that our model achieves significant improvements compared to\nbaselines. We also perform a qualitative analysis to demonstrate that our model\ncould discover more accurate relations by multi-hop relational reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00821v2",
            "date": "2019-02-03 00:32:46+00:00",
            "title": "Review Conversational Reading Comprehension",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Inspired by conversational reading comprehension (CRC), this paper studies a\nnovel task of leveraging reviews as a source to build an agent that can answer\nmulti-turn questions from potential consumers of online businesses. We first\nbuild a review CRC dataset and then propose a novel task-aware pre-tuning step\nrunning between language model (e.g., BERT) pre-training and domain-specific\nfine-tuning. The proposed pre-tuning requires no data annotation, but can\ngreatly enhance the performance on our end task. Experimental results show that\nthe proposed approach is highly effective and has competitive performance as\nthe supervised approach. The dataset is available at\n\\url{https://github.com/howardhsu/RCRC}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00863v2",
            "date": "2019-02-03 08:19:42+00:00",
            "title": "Neural Extractive Text Summarization with Syntactic Compression",
            "authors": [
                "Jiacheng Xu",
                "Greg Durrett"
            ],
            "abstract": "Recent neural network approaches to summarization are largely either\nselection-based extraction or generation-based abstraction. In this work, we\npresent a neural model for single-document summarization based on joint\nextraction and syntactic compression. Our model chooses sentences from the\ndocument, identifies possible compressions based on constituency parses, and\nscores those compressions with a neural model to produce the final summary. For\nlearning, we construct oracle extractive-compressive summaries, then learn both\nof our components jointly with this supervision. Experimental results on the\nCNN/Daily Mail and New York Times datasets show that our model achieves strong\nperformance (comparable to state-of-the-art systems) as evaluated by ROUGE.\nMoreover, our approach outperforms an off-the-shelf compression module, and\nhuman and manual evaluation shows that our model's output generally remains\ngrammatical.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00913v1",
            "date": "2019-02-03 16:03:29+00:00",
            "title": "Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings",
            "authors": [
                "Matt Le",
                "Stephen Roller",
                "Laetitia Papaxanthos",
                "Douwe Kiela",
                "Maximilian Nickel"
            ],
            "abstract": "We consider the task of inferring is-a relationships from large text corpora.\nFor this purpose, we propose a new method combining hyperbolic embeddings and\nHearst patterns. This approach allows us to set appropriate constraints for\ninferring concept hierarchies from distributional contexts while also being\nable to predict missing is-a relationships and to correct wrong extractions.\nMoreover -- and in contrast with other methods -- the hierarchical nature of\nhyperbolic space allows us to learn highly efficient representations and to\nimprove the taxonomic consistency of the inferred hierarchies. Experimentally,\nwe show that our approach achieves state-of-the-art performance on several\ncommonly-used benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00972v2",
            "date": "2019-02-03 21:38:29+00:00",
            "title": "Universal Lemmatizer: A Sequence to Sequence Model for Lemmatizing Universal Dependencies Treebanks",
            "authors": [
                "Jenna Kanerva",
                "Filip Ginter",
                "Tapio Salakoski"
            ],
            "abstract": "In this paper we present a novel lemmatization method based on a\nsequence-to-sequence neural network architecture and morphosyntactic context\nrepresentation. In the proposed method, our context-sensitive lemmatizer\ngenerates the lemma one character at a time based on the surface form\ncharacters and its morphosyntactic features obtained from a morphological\ntagger. We argue that a sliding window context representation suffers from\nsparseness, while in majority of cases the morphosyntactic features of a word\nbring enough information to resolve lemma ambiguities while keeping the context\nrepresentation dense and more practical for machine learning systems.\nAdditionally, we study two different data augmentation methods utilizing\nautoencoder training and morphological transducers especially beneficial for\nlow resource languages. We evaluate our lemmatizer on 52 different languages\nand 76 different treebanks, showing that our system outperforms all latest\nbaseline systems. Compared to the best overall baseline, UDPipe Future, our\nsystem outperforms it on 62 out of 76 treebanks reducing errors on average by\n19% relative. The lemmatizer together with all trained models is made available\nas a part of the Turku-neural-parsing-pipeline under the Apache 2.0 license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00993v3",
            "date": "2019-02-03 23:44:10+00:00",
            "title": "Improving Question Answering with External Knowledge",
            "authors": [
                "Xiaoman Pan",
                "Kai Sun",
                "Dian Yu",
                "Jianshu Chen",
                "Heng Ji",
                "Claire Cardie",
                "Dong Yu"
            ],
            "abstract": "We focus on multiple-choice question answering (QA) tasks in subject areas\nsuch as science, where we require both broad background knowledge and the facts\nfrom the given subject-area reference corpus. In this work, we explore simple\nyet effective methods for exploiting two sources of external knowledge for\nsubject-area QA. The first enriches the original subject-area reference corpus\nwith relevant text snippets extracted from an open-domain resource (i.e.,\nWikipedia) that cover potentially ambiguous concepts in the question and answer\noptions. As in other QA research, the second method simply increases the amount\nof training data by appending additional in-domain subject-area instances.\n  Experiments on three challenging multiple-choice science QA tasks (i.e.,\nARC-Easy, ARC-Challenge, and OpenBookQA) demonstrate the effectiveness of our\nmethods: in comparison to the previous state-of-the-art, we obtain absolute\ngains in accuracy of up to 8.1%, 13.0%, and 12.8%, respectively. While we\nobserve consistent gains when we introduce knowledge from Wikipedia, we find\nthat employing additional QA training instances is not uniformly helpful:\nperformance degrades when the added instances exhibit a higher level of\ndifficulty than the original training data. As one of the first studies on\nexploiting unstructured external knowledge for subject-area QA, we hope our\nmethods, observations, and discussion of the exposed limitations may shed light\non further developments in the area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01007v4",
            "date": "2019-02-04 01:54:19+00:00",
            "title": "Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference",
            "authors": [
                "R. Thomas McCoy",
                "Ellie Pavlick",
                "Tal Linzen"
            ],
            "abstract": "A machine learning system can score well on a given test set by relying on\nheuristics that are effective for frequent example types but break down in more\nchallenging cases. We study this issue within natural language inference (NLI),\nthe task of determining whether one sentence entails another. We hypothesize\nthat statistical NLI models may adopt three fallible syntactic heuristics: the\nlexical overlap heuristic, the subsequence heuristic, and the constituent\nheuristic. To determine whether models have adopted these heuristics, we\nintroduce a controlled evaluation set called HANS (Heuristic Analysis for NLI\nSystems), which contains many examples where the heuristics fail. We find that\nmodels trained on MNLI, including BERT, a state-of-the-art model, perform very\npoorly on HANS, suggesting that they have indeed adopted these heuristics. We\nconclude that there is substantial room for improvement in NLI systems, and\nthat the HANS dataset can motivate and measure progress in this area",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01069v2",
            "date": "2019-02-04 07:55:47+00:00",
            "title": "A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization",
            "authors": [
                "Wonseok Hwang",
                "Jinyeong Yim",
                "Seunghyun Park",
                "Minjoon Seo"
            ],
            "abstract": "We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to\nachieve human performance in WikiSQL dataset. We revisit and discuss diverse\npopular methods in NL2SQL literature, take a full advantage of BERT {Devlin et\nal., 2018) through an effective table contextualization method, and coherently\ncombine them, outperforming the previous state of the art by 8.2% and 2.5% in\nlogical form and execution accuracy, respectively. We particularly note that\nBERT with a seq2seq decoder leads to a poor performance in the task, indicating\nthe importance of a careful design when using such large pretrained models. We\nalso provide a comprehensive analysis on the dataset and our model, which can\nbe helpful for designing future NL2SQL datsets and models. We especially show\nthat our model's performance is near the upper bound in WikiSQL, where we\nobserve that a large portion of the evaluation errors are due to wrong\nannotations, and our model is already exceeding human performance by 1.3% in\nexecution accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01109v2",
            "date": "2019-02-04 10:23:39+00:00",
            "title": "Strategies for Structuring Story Generation",
            "authors": [
                "Angela Fan",
                "Mike Lewis",
                "Yann Dauphin"
            ],
            "abstract": "Writers generally rely on plans or sketches to write long stories, but most\ncurrent language models generate word by word from left to right. We explore\ncoarse-to-fine models for creating narrative texts of several hundred words,\nand introduce new models which decompose stories by abstracting over actions\nand entities. The model first generates the predicate-argument structure of the\ntext, where different mentions of the same entity are marked with placeholder\ntokens. It then generates a surface realization of the predicate-argument\nstructure, and finally replaces the entity placeholders with context-sensitive\nnames and references. Human judges prefer the stories from our models to a wide\nrange of previous approaches to hierarchical text generation. Extensive\nanalysis shows that our methods can help improve the diversity and coherence of\nevents and entities in generated stories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01349v2",
            "date": "2019-02-04 18:11:36+00:00",
            "title": "An Argument-Marker Model for Syntax-Agnostic Proto-Role Labeling",
            "authors": [
                "Juri Opitz",
                "Anette Frank"
            ],
            "abstract": "Semantic proto-role labeling (SPRL) is an alternative to semantic role\nlabeling (SRL) that moves beyond a categorical definition of roles, following\nDowty's feature-based view of proto-roles. This theory determines agenthood vs.\npatienthood based on a participant's instantiation of more or less typical\nagent vs. patient properties, such as, for example, volition in an event. To\nperform SPRL, we develop an ensemble of hierarchical models with self-attention\nand concurrently learned predicate-argument-markers. Our method is competitive\nwith the state-of-the art, overall outperforming previous work in two\nformulations of the task (multi-label and multi-variate Likert scale\nprediction). In contrast to previous work, our results do not depend on gold\nargument heads derived from supplementary gold tree banks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01382v3",
            "date": "2019-02-04 18:48:45+00:00",
            "title": "The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English",
            "authors": [
                "Francisco Guzm\u00e1n",
                "Peng-Jen Chen",
                "Myle Ott",
                "Juan Pino",
                "Guillaume Lample",
                "Philipp Koehn",
                "Vishrav Chaudhary",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "For machine translation, a vast majority of language pairs in the world are\nconsidered low-resource because they have little parallel data available.\nBesides the technical challenges of learning with limited supervision, it is\ndifficult to evaluate methods trained on low-resource language pairs because of\nthe lack of freely and publicly available benchmarks. In this work, we\nintroduce the FLoRes evaluation datasets for Nepali-English and\nSinhala-English, based on sentences translated from Wikipedia. Compared to\nEnglish, these are languages with very different morphology and syntax, for\nwhich little out-of-domain parallel data is available and for which relatively\nlarge amounts of monolingual data are freely available. We describe our process\nto collect and cross-check the quality of translations, and we report baseline\nperformance using several learning settings: fully supervised, weakly\nsupervised, semi-supervised, and fully unsupervised. Our experiments\ndemonstrate that current state-of-the-art methods perform rather poorly on this\nbenchmark, posing a challenge to the research community working on low-resource\nMT. Data and code to reproduce our experiments are available at\nhttps://github.com/facebookresearch/flores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01390v2",
            "date": "2019-02-04 18:58:18+00:00",
            "title": "Fine-Grained Temporal Relation Extraction",
            "authors": [
                "Siddharth Vashishtha",
                "Benjamin Van Durme",
                "Aaron Steven White"
            ],
            "abstract": "We present a novel semantic framework for modeling temporal relations and\nevent durations that maps pairs of events to real-valued scales. We use this\nframework to construct the largest temporal relations dataset to date, covering\nthe entirety of the Universal Dependencies English Web Treebank. We use this\ndataset to train models for jointly predicting fine-grained temporal relations\nand event durations. We report strong results on our data and show the efficacy\nof a transfer-learning approach for predicting categorical relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01615v2",
            "date": "2019-02-05 09:50:47+00:00",
            "title": "Restructuring Conversations using Discourse Relations for Zero-shot Abstractive Dialogue Summarization",
            "authors": [
                "Prakhar Ganesh",
                "Saket Dingliwal"
            ],
            "abstract": "Dialogue summarization is a challenging problem due to the informal and\nunstructured nature of conversational data. Recent advances in abstractive\nsummarization have been focused on data-hungry neural models and adapting these\nmodels to a new domain requires the availability of domain-specific manually\nannotated corpus created by linguistic experts. We propose a zero-shot\nabstractive dialogue summarization method that uses discourse relations to\nprovide structure to conversations, and then uses an out-of-the-box document\nsummarization model to create final summaries. Experiments on the AMI and ICSI\nmeeting corpus, with document summarization models like PGN and BART, shows\nthat our method improves the ROGUE score by up to 3 points, and even performs\ncompetitively against other state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01955v2",
            "date": "2019-02-05 22:16:15+00:00",
            "title": "On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition",
            "authors": [
                "Kazuki Irie",
                "Rohit Prabhavalkar",
                "Anjuli Kannan",
                "Antoine Bruguier",
                "David Rybach",
                "Patrick Nguyen"
            ],
            "abstract": "In conventional speech recognition, phoneme-based models outperform\ngrapheme-based models for non-phonetic languages such as English. The\nperformance gap between the two typically reduces as the amount of training\ndata is increased. In this work, we examine the impact of the choice of\nmodeling unit for attention-based encoder-decoder models. We conduct\nexperiments on the LibriSpeech 100hr, 460hr, and 960hr tasks, using various\ntarget units (phoneme, grapheme, and word-piece); across all tasks, we find\nthat grapheme or word-piece models consistently outperform phoneme-based\nmodels, even though they are evaluated without a lexicon or an external\nlanguage model. We also investigate model complementarity: we find that we can\nimprove WERs by up to 9% relative by rescoring N-best lists generated from a\nstrong word-piece based baseline with either the phoneme or the grapheme model.\nRescoring an N-best list generated by the phonemic system, however, provides\nlimited improvements. Further analysis shows that the word-piece-based models\nproduce more diverse N-best hypotheses, and thus lower oracle WERs, than\nphonemic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02078v3",
            "date": "2019-02-06 09:21:55+00:00",
            "title": "Word Embeddings for Entity-annotated Texts",
            "authors": [
                "Satya Almasian",
                "Andreas Spitz",
                "Michael Gertz"
            ],
            "abstract": "Learned vector representations of words are useful tools for many information\nretrieval and natural language processing tasks due to their ability to capture\nlexical semantics. However, while many such tasks involve or even rely on named\nentities as central components, popular word embedding models have so far\nfailed to include entities as first-class citizens. While it seems intuitive\nthat annotating named entities in the training corpus should result in more\nintelligent word features for downstream tasks, performance issues arise when\npopular embedding approaches are naively applied to entity annotated corpora.\nNot only are the resulting entity embeddings less useful than expected, but one\nalso finds that the performance of the non-entity word embeddings degrades in\ncomparison to those trained on the raw, unannotated corpus. In this paper, we\ninvestigate approaches to jointly train word and entity embeddings on a large\ncorpus with automatically annotated and linked entities. We discuss two\ndistinct approaches to the generation of such embeddings, namely the training\nof state-of-the-art embeddings on raw-text and annotated versions of the\ncorpus, as well as node embeddings of a co-occurrence graph representation of\nthe annotated corpus. We compare the performance of annotated embeddings and\nclassical word embeddings on a variety of word similarity, analogy, and\nclustering evaluation tasks, and investigate their performance in\nentity-specific tasks. Our findings show that it takes more than training\npopular word embedding models on an annotated corpus to create entity\nembeddings with acceptable performance on common test cases. Based on these\nresults, we discuss how and when node embeddings of the co-occurrence graph\nrepresentation of the text can restore the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02160v2",
            "date": "2019-01-24 17:10:02+00:00",
            "title": "Squared English Word: A Method of Generating Glyph to Use Super Characters for Sentiment Analysis",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Catherine Chi",
                "Wenhan Zhang",
                "Michael Lin"
            ],
            "abstract": "The Super Characters method addresses sentiment analysis problems by first\nconverting the input text into images and then applying 2D-CNN models to\nclassify the sentiment. It achieves state of the art performance on many\nbenchmark datasets. However, it is not as straightforward to apply in Latin\nlanguages as in Asian languages. Because the 2D-CNN model is designed to\nrecognize two-dimensional images, it is better if the inputs are in the form of\nglyphs. In this paper, we propose SEW (Squared English Word) method generating\na squared glyph for each English word by drawing Super Characters images of\neach English word at the alphabet level, combining the squared glyph together\ninto a whole Super Characters image at the sentence level, and then applying\nthe CNN model to classify the sentiment within the sentence. We applied the SEW\nmethod to Wikipedia dataset and obtained a 2.1% accuracy gain compared to the\noriginal Super Characters method. For multi-modal data with both structured\ntabular data and unstructured natural language text, the modified SEW method\nintegrates the data into a single image and classifies sentiment with one\nunified CNN model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02161v1",
            "date": "2019-01-21 07:02:52+00:00",
            "title": "AD3: Attentive Deep Document Dater",
            "authors": [
                "Swayambhu Nath Ray",
                "Shib Sankar Dasgupta",
                "Partha Talukdar"
            ],
            "abstract": "Knowledge of the creation date of documents facilitates several tasks such as\nsummarization, event extraction, temporally focused information extraction etc.\nUnfortunately, for most of the documents on the Web, the time-stamp metadata is\neither missing or can't be trusted. Thus, predicting creation time from\ndocument content itself is an important task. In this paper, we propose\nAttentive Deep Document Dater (AD3), an attention-based neural document dating\nsystem which utilizes both context and temporal information in documents in a\nflexible and principled manner. We perform extensive experimentation on\nmultiple real-world datasets to demonstrate the effectiveness of AD3 over\nneural and non-neural baselines.",
            "journal ref": "DBLP:conf/emnlp/RayDT18 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02176v1",
            "date": "2019-01-26 10:26:55+00:00",
            "title": "A Linear-complexity Multi-biometric Forensic Document Analysis System, by Fusing the Stylome and Signature Modalities",
            "authors": [
                "Sayyed-Ali Hossayni",
                "Yousef Alizadeh-Q",
                "Vahid Tavana",
                "Seyed M. Hosseini Nejad",
                "Mohammad-R Akbarzadeh-T",
                "Esteve Del Acebo",
                "Josep Lluis De la Rosa i Esteva",
                "Enrico Grosso",
                "Massimo Tistarelli",
                "Przemyslaw Kudlacik"
            ],
            "abstract": "Forensic Document Analysis (FDA) addresses the problem of finding the\nauthorship of a given document. Identification of the document writer via a\nnumber of its modalities (e.g. handwriting, signature, linguistic writing style\n(i.e. stylome), etc.) has been studied in the FDA state-of-the-art. But, no\nresearch is conducted on the fusion of stylome and signature modalities. In\nthis paper, we propose such a bimodal FDA system (which has vast applications\nin judicial, police-related, and historical documents analysis) with a focus on\ntime-complexity. The proposed bimodal system can be trained and tested with\nlinear time complexity. For this purpose, we first revisit Multinomial Na\\\"ive\nBayes (MNB), as the best state-of-the-art linear-complexity authorship\nattribution system and, then, prove its superior accuracy to the well-known\nlinear-complexity classifiers in the state-of-the-art. Then, we propose a fuzzy\nversion of MNB for being fused with a state-of-the-art well-known\nlinear-complexity fuzzy signature recognition system. For the evaluation\npurposes, we construct a chimeric dataset, composed of signatures and textual\ncontents of different letters. Despite its linear-complexity, the proposed\nmulti-biometric system is proven to meaningfully improve its state-of-the-art\nunimodal counterparts, regarding the accuracy, F-Score, Detection Error\nTrade-off (DET), Cumulative Match Characteristics (CMC), and Match Score\nHistograms (MSH) evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02179v1",
            "date": "2019-01-25 01:47:13+00:00",
            "title": "Assessing Partisan Traits of News Text Attributions",
            "authors": [
                "Logan Martel",
                "Edward Newell",
                "Drew Margolin",
                "Derek Ruths"
            ],
            "abstract": "On the topic of journalistic integrity, the current state of accurate,\nimpartial news reporting has garnered much debate in context to the 2016 US\nPresidential Election. In pursuit of computational evaluation of news text, the\nstatements (attributions) ascribed by media outlets to sources provide a common\ncategory of evidence on which to operate. In this paper, we develop an approach\nto compare partisan traits of news text attributions and apply it to\ncharacterize differences in statements ascribed to candidate, Hilary Clinton,\nand incumbent President, Donald Trump. In doing so, we present a model trained\non over 600 in-house annotated attributions to identify each candidate with\naccuracy > 88%. Finally, we discuss insights from its performance for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02326v1",
            "date": "2019-02-06 18:42:18+00:00",
            "title": "Extending a model for ontology-based Arabic-English machine translation",
            "authors": [
                "Neama Abdulaziz Dahan",
                "Fadl Mutaher Ba-Alwi"
            ],
            "abstract": "The acceleration in telecommunication needs leads to many groups of research,\nespecially in communication facilitating and Machine Translation fields. While\npeople contact with others having different languages and cultures, they need\nto have instant translations. However, the available instant translators are\nstill providing somewhat bad Arabic-English Translations, for instance when\ntranslating books or articles, the meaning is not totally accurate. Therefore,\nusing the semantic web techniques to deal with the homographs and homonyms\nsemantically, the aim of this research is to extend a model for the\nontology-based Arabic-English Machine Translation, named NAN, which simulate\nthe human way in translation. The experimental results show that NAN\ntranslation is approximately more similar to the Human Translation than the\nother instant translators. The resulted translation will help getting the\ntranslated texts in the target language somewhat correctly and semantically\nmore similar to human translations for the Non-Arabic Natives and the\nNon-English natives.",
            "journal ref": "International Journal of Artificial Intelligence and Applications\n  (IJAIA), Vol.10, No.1, January 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02704v4",
            "date": "2019-02-07 16:01:43+00:00",
            "title": "Understanding Chat Messages for Sticker Recommendation in Messaging Apps",
            "authors": [
                "Abhishek Laddha",
                "Mohamed Hanoosh",
                "Debdoot Mukherjee",
                "Parth Patwa",
                "Ankur Narang"
            ],
            "abstract": "Stickers are popularly used in messaging apps such as Hike to visually\nexpress a nuanced range of thoughts and utterances to convey exaggerated\nemotions. However, discovering the right sticker from a large and ever\nexpanding pool of stickers while chatting can be cumbersome. In this paper, we\ndescribe a system for recommending stickers in real time as the user is typing\nbased on the context of the conversation. We decompose the sticker\nrecommendation (SR) problem into two steps. First, we predict the message that\nthe user is likely to send in the chat. Second, we substitute the predicted\nmessage with an appropriate sticker. Majority of Hike's messages are in the\nform of text which is transliterated from users' native language to the Roman\nscript. This leads to numerous orthographic variations of the same message and\nmakes accurate message prediction challenging. To address this issue, we learn\ndense representations of chat messages employing character level convolution\nnetwork in an unsupervised manner. We use them to cluster the messages that\nhave the same meaning. In the subsequent steps, we predict the message cluster\ninstead of the message. Our approach does not depend on human labelled data\n(except for validation), leading to fully automatic updation and tuning\npipeline for the underlying models. We also propose a novel hybrid message\nprediction model, which can run with low latency on low-end phones that have\nsevere computational limitations. Our described system has been deployed for\nmore than $6$ months and is being used by millions of users along with hundreds\nof thousands of expressive stickers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02709v1",
            "date": "2019-02-07 16:12:41+00:00",
            "title": "Aspect Specific Opinion Expression Extraction using Attention based LSTM-CRF Network",
            "authors": [
                "Abhishek Laddha",
                "Arjun Mukherjee"
            ],
            "abstract": "Opinion phrase extraction is one of the key tasks in fine-grained sentiment\nanalysis. While opinion expressions could be generic subjective expressions,\naspect specific opinion expressions contain both the aspect as well as the\nopinion expression within the original sentence context. In this work, we\nformulate the task as an instance of token-level sequence labeling. When\nmultiple aspects are present in a sentence, detection of opinion phrase\nboundary becomes difficult and label of each word depend not only upon the\nsurrounding words but also with the concerned aspect. We propose a neural\nnetwork architecture with bidirectional LSTM (Bi-LSTM) and a novel attention\nmechanism. Bi-LSTM layer learns the various sequential pattern among the words\nwithout requiring any hand-crafted features. The attention mechanism captures\nthe importance of context words on a particular aspect opinion expression when\nmultiple aspects are present in a sentence via location and content based\nmemory. A Conditional Random Field (CRF) model is incorporated in the final\nlayer to explicitly model the dependencies among the output labels.\nExperimental results on Hotel dataset from Tripadvisor.com showed that our\napproach outperformed several state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03052v1",
            "date": "2019-02-08 12:27:26+00:00",
            "title": "Models of Visually Grounded Speech Signal Pay Attention To Nouns: a Bilingual Experiment on English and Japanese",
            "authors": [
                "William N. Havard",
                "Jean-Pierre Chevrot",
                "Laurent Besacier"
            ],
            "abstract": "We investigate the behaviour of attention in neural models of visually\ngrounded speech trained on two languages: English and Japanese. Experimental\nresults show that attention focuses on nouns and this behaviour holds true for\ntwo very typologically different languages. We also draw parallels between\nartificial neural attention and human attention and show that neural attention\nfocuses on word endings as it has been theorised for human attention. Finally,\nwe investigate how two visually grounded monolingual models can be used to\nperform cross-lingual speech-to-speech retrieval. For both languages, the\nenriched bilingual (speech-image) corpora with part-of-speech tags and forced\nalignments are distributed to the community for reproducible research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03499v1",
            "date": "2019-02-09 21:47:05+00:00",
            "title": "Multilingual Neural Machine Translation With Soft Decoupled Encoding",
            "authors": [
                "Xinyi Wang",
                "Hieu Pham",
                "Philip Arthur",
                "Graham Neubig"
            ],
            "abstract": "Multilingual training of neural machine translation (NMT) systems has led to\nimpressive accuracy improvements on low-resource languages. However, there are\nstill significant challenges in efficiently learning word representations in\nthe face of paucity of data. In this paper, we propose Soft Decoupled Encoding\n(SDE), a multilingual lexicon encoding framework specifically designed to share\nlexical-level information intelligently without requiring heuristic\npreprocessing such as pre-segmenting the data. SDE represents a word by its\nspelling through a character encoding, and its semantic meaning through a\nlatent embedding space shared by all languages. Experiments on a standard\ndataset of four low-resource languages show consistent improvements over strong\nmultilingual NMT baselines, with gains of up to 2 BLEU on one of the tested\nlanguages, achieving the new state-of-the-art on all four language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03658v1",
            "date": "2019-02-10 19:40:07+00:00",
            "title": "Word embeddings for idiolect identification",
            "authors": [
                "Konstantinos Perifanos",
                "Eirini Florou",
                "Dionysis Goutsos"
            ],
            "abstract": "The term idiolect refers to the unique and distinctive use of language of an\nindividual and it is the theoretical foundation of Authorship Attribution. In\nthis paper we are focusing on learning distributed representations (embeddings)\nof social media users that reflect their writing style. These representations\ncan be considered as stylistic fingerprints of the authors. We are exploring\nthe performance of the two main flavours of distributed representations, namely\nembeddings produced by Neural Probabilistic Language models (such as word2vec)\nand matrix factorization (such as GloVe).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03659v1",
            "date": "2019-02-10 19:46:24+00:00",
            "title": "Neural embeddings for metaphor detection in a corpus of Greek texts",
            "authors": [
                "Eirini Florou",
                "Konstantinos Perifanos",
                "Dionysis Goutsos"
            ],
            "abstract": "One of the major challenges that NLP faces is metaphor detection, especially\nby automatic means, a task that becomes even more difficult for languages\nlacking in linguistic resources and tools. Our purpose is the automatic\ndifferentiation between literal and metaphorical meaning in authentic\nnon-annotated phrases from the Corpus of Greek Texts by means of computational\nmethods of machine learning. For this purpose the theoretical background of\ndistributional semantics is discussed and employed. Distributional Semantics\nTheory develops concepts and methods for the quantification and classification\nof semantic similarities displayed by linguistic elements in large amounts of\nlinguistic data according to their distributional properties. In accordance\nwith this model, the approach followed in the thesis takes into account the\nlinguistic context for the computation of the distributional representation of\nphrases in geometrical space, as well as for their comparison with the\ndistributional representations of other phrases, whose function in speech is\nalready \"known\" with the objective to reach conclusions about their literal or\nmetaphorical function in the specific linguistic context. This procedure aims\nat dealing with the lack of linguistic resources for the Greek language, as the\nalmost impossible up to now semantic comparison between \"phrases\", takes the\nform of an arithmetical comparison of their distributional representations in\ngeometrical space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.04260v8",
            "date": "2019-02-12 07:07:16+00:00",
            "title": "Table2answer: Read the database and answer without SQL",
            "authors": [
                "Tong Guo",
                "Huilin Gao"
            ],
            "abstract": "Semantic parsing is the task of mapping natural language to logic form. In\nquestion answering, semantic parsing can be used to map the question to logic\nform and execute the logic form to get the answer. One key problem for semantic\nparsing is the hard label work. We study this problem in another way: we do not\nuse the logic form any more. Instead we only use the schema and answer info. We\nthink that the logic form step can be injected into the deep model. The reason\nwhy we think removing the logic form step is possible is that human can do the\ntask without explicit logic form. We use BERT-based model and do the experiment\nin the WikiSQL dataset, which is a large natural language to SQL dataset. Our\nexperimental evaluations that show that our model can achieves the baseline\nresults in WikiSQL dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.04574v2",
            "date": "2019-02-12 15:49:40+00:00",
            "title": "Machine Reading Comprehension for Answer Re-Ranking in Customer Support Chatbots",
            "authors": [
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Recent advances in deep neural networks, language modeling and language\ngeneration have introduced new ideas to the field of conversational agents. As\na result, deep neural models such as sequence-to-sequence, Memory Networks, and\nthe Transformer have become key ingredients of state-of-the-art dialog systems.\nWhile those models are able to generate meaningful responses even in unseen\nsituation, they need a lot of training data to build a reliable model. Thus,\nmost real-world systems stuck to traditional approaches based on information\nretrieval and even hand-crafted rules, due to their robustness and\neffectiveness, especially for narrow-focused conversations. Here, we present a\nmethod that adapts a deep neural architecture from the domain of machine\nreading comprehension to re-rank the suggested answers from different models\nusing the question as context. We train our model using negative sampling based\non question-answer pairs from the Twitter Customer Support Dataset.The\nexperimental results show that our re-ranking framework can improve the\nperformance in terms of word overlap and semantics both for individual models\nas well as for model combinations.",
            "journal ref": "Information 2019, 10, 82",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.04793v1",
            "date": "2019-02-13 09:00:16+00:00",
            "title": "SECTOR: A Neural Model for Coherent Topic Segmentation and Classification",
            "authors": [
                "Sebastian Arnold",
                "Rudolf Schneider",
                "Philippe Cudr\u00e9-Mauroux",
                "Felix A. Gers",
                "Alexander L\u00f6ser"
            ],
            "abstract": "When searching for information, a human reader first glances over a document,\nspots relevant sections and then focuses on a few sentences for resolving her\nintention. However, the high variance of document structure complicates to\nidentify the salient topic of a given section at a glance. To tackle this\nchallenge, we present SECTOR, a model to support machine reading systems by\nsegmenting documents into coherent sections and assigning topic labels to each\nsection. Our deep neural network architecture learns a latent topic embedding\nover the course of a document. This can be leveraged to classify local topics\nfrom plain text and segment a document at topic shifts. In addition, we\ncontribute WikiSection, a publicly available dataset with 242k labeled sections\nin English and German from two distinct domains: diseases and cities. From our\nextensive evaluation of 20 architectures, we report a highest score of 71.6% F1\nfor the segmentation and classification of 30 topics from the English city\ndomain, scored by our SECTOR LSTM model with bloom filter embeddings and\nbidirectional segmentation. This is a significant improvement of 29.5 points F1\ncompared to state-of-the-art CNN classifiers with baseline segmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.04911v2",
            "date": "2019-02-13 14:13:21+00:00",
            "title": "Learning to Select Knowledge for Response Generation in Dialog Systems",
            "authors": [
                "Rongzhong Lian",
                "Min Xie",
                "Fan Wang",
                "Jinhua Peng",
                "Hua Wu"
            ],
            "abstract": "End-to-end neural models for intelligent dialogue systems suffer from the\nproblem of generating uninformative responses. Various methods were proposed to\ngenerate more informative responses by leveraging external knowledge. However,\nfew previous work has focused on selecting appropriate knowledge in the\nlearning process. The inappropriate selection of knowledge could prohibit the\nmodel from learning to make full use of the knowledge. Motivated by this, we\npropose an end-to-end neural model which employs a novel knowledge selection\nmechanism where both prior and posterior distributions over knowledge are used\nto facilitate knowledge selection. Specifically, a posterior distribution over\nknowledge is inferred from both utterances and responses, and it ensures the\nappropriate selection of knowledge during the training process. Meanwhile, a\nprior distribution, which is inferred from utterances only, is used to\napproximate the posterior distribution so that appropriate knowledge can be\nselected even without responses during the inference process. Compared with the\nprevious work, our model can better incorporate appropriate knowledge in\nresponse generation. Experiments on both automatic and human evaluation verify\nthe superiority of our model over previous baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05085v1",
            "date": "2019-02-13 19:01:51+00:00",
            "title": "Leveraging Newswire Treebanks for Parsing Conversational Data with Argument Scrambling",
            "authors": [
                "Riyaz Ahmad Bhat",
                "Irshad Ahmad Bhat",
                "Dipti Misra Sharma"
            ],
            "abstract": "We investigate the problem of parsing conversational data of\nmorphologically-rich languages such as Hindi where argument scrambling occurs\nfrequently. We evaluate a state-of-the-art non-linear transition-based parsing\nsystem on a new dataset containing 506 dependency trees for sentences from\nBollywood (Hindi) movie scripts and Twitter posts of Hindi monolingual\nspeakers. We show that a dependency parser trained on a newswire treebank is\nstrongly biased towards the canonical structures and degrades when applied to\nconversational data. Inspired by Transformational Generative Grammar, we\nmitigate the sampling bias by generating all theoretically possible alternative\nword orders of a clause from the existing (kernel) structures in the treebank.\nTraining our parser on canonical and transformed structures improves\nperformance on conversational data by around 9% LAS over the baseline newswire\nparser.",
            "journal ref": "Proceedings of the 15th International Conference on Parsing\n  Technologies, pages 61-66, Pisa, Italy; September 20-22, 2017. Association\n  for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05196v1",
            "date": "2019-02-14 03:07:53+00:00",
            "title": "Categorical Metadata Representation for Customized Text Classification",
            "authors": [
                "Jihyeok Kim",
                "Reinald Kim Amplayo",
                "Kyungjae Lee",
                "Sua Sung",
                "Minji Seo",
                "Seung-won Hwang"
            ],
            "abstract": "The performance of text classification has improved tremendously using\nintelligently engineered neural-based models, especially those injecting\ncategorical metadata as additional information, e.g., using user/product\ninformation for sentiment classification. These information have been used to\nmodify parts of the model (e.g., word embeddings, attention mechanisms) such\nthat results can be customized according to the metadata. We observe that\ncurrent representation methods for categorical metadata, which are devised for\nhuman consumption, are not as effective as claimed in popular classification\nmethods, outperformed even by simple concatenation of categorical features in\nthe final layer of the sentence encoder. We conjecture that categorical\nfeatures are harder to represent for machine use, as available context only\nindirectly describes the category, and even such context is often scarce (for\ntail category). To this end, we propose to use basis vectors to effectively\nincorporate categorical metadata on various parts of a neural-based model. This\nadditionally decreases the number of parameters dramatically, especially when\nthe number of categorical features is large. Extensive experiments on various\ndatasets with different properties are performed and show that through our\nmethod, we can represent categorical metadata more effectively to customize\nparts of the model, including unexplored ones, and increase the performance of\nthe model greatly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05309v1",
            "date": "2019-02-14 11:40:58+00:00",
            "title": "Transfer Learning for Sequence Labeling Using Source Model and Target Data",
            "authors": [
                "Lingzhen Chen",
                "Alessandro Moschitti"
            ],
            "abstract": "In this paper, we propose an approach for transferring the knowledge of a\nneural model for sequence labeling, learned from the source domain, to a new\nmodel trained on a target domain, where new label categories appear. Our\ntransfer learning (TL) techniques enable to adapt the source model using the\ntarget data and new categories, without accessing to the source data. Our\nsolution consists in adding new neurons in the output layer of the target model\nand transferring parameters from the source model, which are then fine-tuned\nwith the target data. Additionally, we propose a neural adapter to learn the\ndifference between the source and the target label distribution, which provides\nadditional important information to the target model. Our experiments on Named\nEntity Recognition show that (i) the learned knowledge in the source model can\nbe effectively transferred when the target data contains new categories and\n(ii) our neural adapter further improves such transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05766v1",
            "date": "2019-02-15 11:03:52+00:00",
            "title": "Context-Aware Self-Attention Networks",
            "authors": [
                "Baosong Yang",
                "Jian Li",
                "Derek Wong",
                "Lidia S. Chao",
                "Xing Wang",
                "Zhaopeng Tu"
            ],
            "abstract": "Self-attention model have shown its flexibility in parallel computation and\nthe effectiveness on modeling both long- and short-term dependencies. However,\nit calculates the dependencies between representations without considering the\ncontextual information, which have proven useful for modeling dependencies\namong neural representations in various natural language tasks. In this work,\nwe focus on improving self-attention networks through capturing the richness of\ncontext. To maintain the simplicity and flexibility of the self-attention\nnetworks, we propose to contextualize the transformations of the query and key\nlayers, which are used to calculates the relevance between elements.\nSpecifically, we leverage the internal representations that embed both global\nand deep contexts, thus avoid relying on external resources. Experimental\nresults on WMT14 English-German and WMT17 Chinese-English translation tasks\ndemonstrate the effectiveness and universality of the proposed methods.\nFurthermore, we conducted extensive analyses to quantity how the context\nvectors participate in the self-attention model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06006v3",
            "date": "2019-02-15 23:28:36+00:00",
            "title": "Contextual Word Representations: A Contextual Introduction",
            "authors": [
                "Noah A. Smith"
            ],
            "abstract": "This introduction aims to tell the story of how we put words into computers.\nIt is part of the story of the field of natural language processing (NLP), a\nbranch of artificial intelligence. It targets a wide audience with a basic\nunderstanding of computer programming, but avoids a detailed mathematical\ntreatment, and it does not present any algorithms. It also does not focus on\nany particular application of NLP such as translation, question answering, or\ninformation extraction. The ideas presented here were developed by many\nresearchers over many decades, so the citations are not exhaustive but rather\ndirect the reader to a handful of papers that are, in the author's view,\nseminal. After reading this document, you should have a general understanding\nof word vectors (also known as word embeddings): why they exist, what problems\nthey solve, where they come from, how they have changed over time, and what\nsome of the open questions about them are. Readers already familiar with word\nvectors are advised to skip to Section 5 for the discussion of the most recent\nadvance, contextual word vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06022v1",
            "date": "2019-02-16 01:28:12+00:00",
            "title": "A Fully Differentiable Beam Search Decoder",
            "authors": [
                "Ronan Collobert",
                "Awni Hannun",
                "Gabriel Synnaeve"
            ],
            "abstract": "We introduce a new beam search decoder that is fully differentiable, making\nit possible to optimize at training time through the inference procedure. Our\ndecoder allows us to combine models which operate at different granularities\n(e.g. acoustic and language models). It can be used when target sequences are\nnot aligned to input sequences by considering all possible alignments between\nthe two. We demonstrate our approach scales by applying it to speech\nrecognition, jointly training acoustic and word-level language models. The\nsystem is end-to-end, with gradients flowing through the whole architecture\nfrom the word-level transcriptions. Recent research efforts have shown that\ndeep neural networks with attention-based mechanisms are powerful enough to\nsuccessfully train an acoustic model from the final transcription, while\nimplicitly learning a language model. Instead, we show that it is possible to\ndiscriminatively train an acoustic model jointly with an explicit and possibly\npre-trained language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06092v1",
            "date": "2019-02-16 11:27:21+00:00",
            "title": "Exploring Language Similarities with Dimensionality Reduction Technique",
            "authors": [
                "Sangarshanan Veeraraghavan"
            ],
            "abstract": "In recent years several novel models were developed to process natural\nlanguage, development of accurate language translation systems have helped us\novercome geographical barriers and communicate ideas effectively. These models\nare developed mostly for a few languages that are widely used while other\nlanguages are ignored. Most of the languages that are spoken share lexical,\nsyntactic and sematic similarity with several other languages and knowing this\ncan help us leverage the existing model to build more specific and accurate\nmodels that can be used for other languages, so here I have explored the idea\nof representing several known popular languages in a lower dimension such that\ntheir similarities can be visualized using simple 2 dimensional plots. This can\neven help us understand newly discovered languages that may not share its\nvocabulary with any of the existing languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06242v1",
            "date": "2019-02-17 10:51:15+00:00",
            "title": "A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine",
            "authors": [
                "Omar Al-Harbi"
            ],
            "abstract": "Unlike other languages, the Arabic language has a morphological complexity\nwhich makes the Arabic sentiment analysis is a challenging task. Moreover, the\npresence of the dialects in the Arabic texts have made the sentiment analysis\ntask is more challenging, due to the absence of specific rules that govern the\nwriting or speaking system. Generally, one of the problems of sentiment\nanalysis is the high dimensionality of the feature vector. To resolve this\nproblem, many feature selection methods have been proposed. In contrast to the\ndialectal Arabic language, these selection methods have been investigated\nwidely for the English language. This work investigated the effect of feature\nselection methods and their combinations on dialectal Arabic sentiment\nclassification. The feature selection methods are Information Gain (IG),\nCorrelation, Support Vector Machine (SVM), Gini Index (GI), and Chi-Square. A\nnumber of experiments were carried out on dialectical Jordanian reviews with\nusing an SVM classifier. Furthermore, the effect of different term weighting\nschemes, stemmers, stop words removal, and feature models on the performance\nwere investigated. The experimental results showed that the best performance of\nthe SVM classifier was obtained after the SVM and correlation feature selection\nmethods had been combined with the uni-gram model.",
            "journal ref": "IJCSNS International Journal of Computer Science and Network\n  Security, VOL.19 No.1, January 2019, 167-176",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06635v1",
            "date": "2019-02-18 16:26:01+00:00",
            "title": "Investigating the Effect of Segmentation Methods on Neural Model based Sentiment Analysis on Informal Short Texts in Turkish",
            "authors": [
                "Fatih Kurt",
                "Dilek Kisa",
                "Pinar Karagoz"
            ],
            "abstract": "This work investigates segmentation approaches for sentiment analysis on\ninformal short texts in Turkish. The two building blocks of the proposed work\nare segmentation and deep neural network model. Segmentation focuses on\npreprocessing of text with different methods. These methods are grouped in\nfour: morphological, sub-word, tokenization, and hybrid approaches. We analyzed\nseveral variants for each of these four methods. The second stage focuses on\nevaluation of the neural model for sentiment analysis. The performance of each\nsegmentation method is evaluated under Convolutional Neural Network (CNN) and\nRecurrent Neural Network (RNN) model proposed in the literature for sentiment\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06734v1",
            "date": "2019-02-14 20:00:30+00:00",
            "title": "Author Profiling for Hate Speech Detection",
            "authors": [
                "Pushkar Mishra",
                "Marco Del Tredici",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "The rapid growth of social media in recent years has fed into some highly\nundesirable phenomena such as proliferation of abusive and offensive language\non the Internet. Previous research suggests that such hateful content tends to\ncome from users who share a set of common stereotypes and form communities\naround them. The current state-of-the-art approaches to hate speech detection\nare oblivious to user and community information and rely entirely on textual\n(i.e., lexical and semantic) cues. In this paper, we propose a novel approach\nto this problem that incorporates community-based profiling features of Twitter\nusers. Experimenting with a dataset of 16k tweets, we show that our methods\nsignificantly outperform the current state of the art in hate speech detection.\nFurther, we conduct a qualitative analysis of model characteristics. We release\nour code, pre-trained models and all the resources used in the public domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07023v2",
            "date": "2019-02-19 12:34:40+00:00",
            "title": "A Walk-based Model on Entity Graphs for Relation Extraction",
            "authors": [
                "Fenia Christopoulou",
                "Makoto Miwa",
                "Sophia Ananiadou"
            ],
            "abstract": "We present a novel graph-based neural network model for relation extraction.\nOur model treats multiple pairs in a sentence simultaneously and considers\ninteractions among them. All the entities in a sentence are placed as nodes in\na fully-connected graph structure. The edges are represented with\nposition-aware contexts around the entity pairs. In order to consider different\nrelation paths between two entities, we construct up to l-length walks between\neach pair. The resulting walks are merged and iteratively used to update the\nedge representations into longer walks representations. We show that the model\nachieves performance comparable to the state-of-the-art systems on the ACE 2005\ndataset without using any external tools.",
            "journal ref": "Proceedings of the 56th Annual Meeting of the Association for\n  Computational Linguistics (Volume 2: Short Papers), 2018, pages 81-88",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07110v1",
            "date": "2019-02-19 16:00:38+00:00",
            "title": "A novel repetition normalized adversarial reward for headline generation",
            "authors": [
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "While reinforcement learning can effectively improve language generation\nmodels, it often suffers from generating incoherent and repetitive phrases\n\\cite{paulus2017deep}. In this paper, we propose a novel repetition normalized\nadversarial reward to mitigate these problems. Our repetition penalized reward\ncan greatly reduce the repetition rate and adversarial training mitigates\ngenerating incoherent phrases. Our model significantly outperforms the baseline\nmodel on ROUGE-1\\,(+3.24), ROUGE-L\\,(+2.25), and a decreased repetition-rate\n(-4.98\\%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07250v1",
            "date": "2019-02-10 13:28:25+00:00",
            "title": "Neural Machine Translation for Cebuano to Tagalog with Subword Unit Translation",
            "authors": [
                "Kristine Mae M. Adlaon",
                "Nelson Marcos"
            ],
            "abstract": "The Philippines is an archipelago composed of 7, 641 different islands with\nmore than 150 different languages. This linguistic differences and diversity,\nthough may be seen as a beautiful feature, have contributed to the difficulty\nin the promotion of educational and cultural development of different domains\nin the country. An effective machine translation system solely dedicated to\ncater Philippine languages will surely help bridge this gap. In this research\nwork, a never before applied approach for language translation to a Philippine\nlanguage was used for a Cebuano to Tagalog translator. A Recurrent Neural\nNetwork was used to implement the translator using OpenNMT sequence modeling\ntool in TensorFlow. The performance of the translation was evaluated using the\nBLEU Score metric. For the Cebuano to Tagalog translation, BLEU produced a\nscore of 20.01. A subword unit translation for verbs and copyable approach was\nperformed where commonly seen mistranslated words from the source to the target\nwere corrected. The BLEU score increased to 22.87. Though slightly higher, this\nscore still indicates that the translation is somehow understandable but is not\nyet considered as a good translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07613v1",
            "date": "2019-02-20 16:00:12+00:00",
            "title": "Phoneme Level Language Models for Sequence Based Low Resource ASR",
            "authors": [
                "Siddharth Dalmia",
                "Xinjian Li",
                "Alan W Black",
                "Florian Metze"
            ],
            "abstract": "Building multilingual and crosslingual models help bring different languages\ntogether in a language universal space. It allows models to share parameters\nand transfer knowledge across languages, enabling faster and better adaptation\nto a new language. These approaches are particularly useful for low resource\nlanguages. In this paper, we propose a phoneme-level language model that can be\nused multilingually and for crosslingual adaptation to a target language. We\nshow that our model performs almost as well as the monolingual models by using\nsix times fewer parameters, and is capable of better adaptation to languages\nnot seen during training in a low resource scenario. We show that these\nphoneme-level language models can be used to decode sequence based\nConnectionist Temporal Classification (CTC) acoustic model outputs to obtain\ncomparable word error rates with Weighted Finite State Transducer (WFST) based\ndecoding in Babel languages. We also show that these phoneme-level language\nmodels outperform WFST decoding in various low-resource conditions like\nadapting to a new language and domain mismatch between training and testing\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07669v3",
            "date": "2019-02-20 17:28:51+00:00",
            "title": "ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing",
            "authors": [
                "Mark Neumann",
                "Daniel King",
                "Iz Beltagy",
                "Waleed Ammar"
            ],
            "abstract": "Despite recent advances in natural language processing, many statistical\nmodels for processing text perform extremely poorly under domain shift.\nProcessing biomedical and clinical text is a critically important application\narea of natural language processing, for which there are few robust, practical,\npublicly available models. This paper describes scispaCy, a new tool for\npractical biomedical/scientific text processing, which heavily leverages the\nspaCy library. We detail the performance of two packages of models released in\nscispaCy and demonstrate their robustness on several tasks and datasets. Models\nand code are available at https://allenai.github.io/scispacy/",
            "journal ref": "Proceedings of the 18th BioNLP Workshop and Shared Task (2019)\n  319-327",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07938v1",
            "date": "2019-02-21 09:53:04+00:00",
            "title": "Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts",
            "authors": [
                "Rezka Leonandya",
                "Fariz Ikhwantri"
            ],
            "abstract": "Named entity recognition (NER) is an important task in NLP, which is all the\nmore challenging in conversational domain with their noisy facets. Moreover,\nconversational texts are often available in limited amount, making supervised\ntasks infeasible. To learn from small data, strong inductive biases are\nrequired. Previous work relied on hand-crafted features to encode these biases\nuntil transfer learning emerges. Here, we explore a transfer learning method,\nnamely language model pretraining, on NER task in Indonesian conversational\ntexts. We utilize large unlabeled data (generic domain) to be transferred to\nconversational texts, enabling supervised training on limited in-domain data.\nWe report two transfer learning variants, namely supervised model fine-tuning\nand unsupervised pretrained LM fine-tuning. Our experiments show that both\nvariants outperform baseline neural models when trained on small data (100\nsentences), yielding an absolute improvement of 32 points of test F1 score.\nFurthermore, we find that the pretrained LM encodes part-of-speech information\nwhich is a strong predictor for NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08050v1",
            "date": "2019-02-21 13:50:56+00:00",
            "title": "Deep Short Text Classification with Knowledge Powered Attention",
            "authors": [
                "Jindong Chen",
                "Yizhou Hu",
                "Jingping Liu",
                "Yanghua Xiao",
                "Haiyun Jiang"
            ],
            "abstract": "Short text classification is one of important tasks in Natural Language\nProcessing (NLP). Unlike paragraphs or documents, short texts are more\nambiguous since they have not enough contextual information, which poses a\ngreat challenge for classification. In this paper, we retrieve knowledge from\nexternal knowledge source to enhance the semantic representation of short\ntexts. We take conceptual information as a kind of knowledge and incorporate it\ninto deep neural networks. For the purpose of measuring the importance of\nknowledge, we introduce attention mechanisms and propose deep Short Text\nClassification with Knowledge powered Attention (STCKA). We utilize Concept\ntowards Short Text (C- ST) attention and Concept towards Concept Set (C-CS)\nattention to acquire the weight of concepts from two aspects. And we classify a\nshort text with the help of conceptual information. Unlike traditional\napproaches, our model acts like a human being who has intrinsic ability to make\ndecisions based on observation (i.e., training data for machines) and pays more\nattention to important knowledge. We also conduct extensive experiments on four\npublic datasets for different tasks. The experimental results and case studies\nshow that our model outperforms the state-of-the-art methods, justifying the\neffectiveness of knowledge powered attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08061v1",
            "date": "2019-02-21 14:19:00+00:00",
            "title": "Development of a classifiers/quantifiers dictionary towards French-Japanese MT",
            "authors": [
                "Mutsuko Tomokiyo",
                "Mathieu Mangeot",
                "Christian Boitet"
            ],
            "abstract": "Although classifiers/quantifiers (CQs) expressions appear frequently in\neveryday communications or written documents, they are described neither in\nclassical bilingual paper dictionaries , nor in machine-readable dictionaries.\nThe paper describes a CQs dictionary, edited from the corpus we have annotated,\nand its usage in the framework of French-Japanese machine translation (MT). CQs\ntreatment in MT often causes problems of lexical ambiguity, polylexical phrase\nrecognition difficulties in analysis and doubtful output in\ntransfer-generation, in particular for distant languages pairs like French and\nJapanese. Our basic treatment of CQs is to annotate the corpus by UNL-UWs\n(Universal Networking Language-Universal words) 1 , and then to produce a\nbilingual or multilingual dictionary of CQs, based on synonymy through identity\nof UWs.",
            "journal ref": "MT Summit 2017, Sep 2017, Nagoya, Japan",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08342v1",
            "date": "2019-02-22 02:31:41+00:00",
            "title": "Aspect-Sentiment Embeddings for Company Profiling and Employee Opinion Mining",
            "authors": [
                "Rajiv Bajpai",
                "Devamanyu Hazarika",
                "Kunal Singh",
                "Sruthi Gorantla",
                "Erik Cambria",
                "Roger Zimmerman"
            ],
            "abstract": "With the multitude of companies and organizations abound today, ranking them\nand choosing one out of the many is a difficult and cumbersome task. Although\nthere are many available metrics that rank companies, there is an inherent need\nfor a generalized metric that takes into account the different aspects that\nconstitute employee opinions of the companies. In this work, we aim to overcome\nthe aforementioned problem by generating aspect-sentiment based embedding for\nthe companies by looking into reliable employee reviews of them. We created a\ncomprehensive dataset of company reviews from the famous website Glassdoor.com\nand employed a novel ensemble approach to perform aspect-level sentiment\nanalysis. Although a relevant amount of work has been done on reviews centered\non subjects like movies, music, etc., this work is the first of its kind. We\nalso provide several insights from the collated embeddings, thus helping users\ngain a better understanding of their options as well as select companies using\ncustomized preferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08564v2",
            "date": "2019-02-22 17:11:03+00:00",
            "title": "Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax",
            "authors": [
                "Yinfei Yang",
                "Gustavo Hernandez Abrego",
                "Steve Yuan",
                "Mandy Guo",
                "Qinlan Shen",
                "Daniel Cer",
                "Yun-hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "In this paper, we present an approach to learn multilingual sentence\nembeddings using a bi-directional dual-encoder with additive margin softmax.\nThe embeddings are able to achieve state-of-the-art results on the United\nNations (UN) parallel corpus retrieval task. In all the languages tested, the\nsystem achieves P@1 of 86% or higher. We use pairs retrieved by our approach to\ntrain NMT models that achieve similar performance to models trained on gold\npairs. We explore simple document-level embeddings constructed by averaging our\nsentence embeddings. On the UN document-level retrieval task, document\nembeddings achieve around 97% on P@1 for all experimented language pairs.\nLastly, we evaluate the proposed model on the BUCC mining task. The learned\nembeddings with raw cosine similarity scores achieve competitive results\ncompared to current state-of-the-art models, and with a second-stage scorer we\nachieve a new state-of-the-art level on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08654v2",
            "date": "2019-02-22 19:59:47+00:00",
            "title": "What makes a good conversation? How controllable attributes affect human judgments",
            "authors": [
                "Abigail See",
                "Stephen Roller",
                "Douwe Kiela",
                "Jason Weston"
            ],
            "abstract": "A good conversation requires balance -- between simplicity and detail;\nstaying on topic and changing it; asking questions and answering them. Although\ndialogue agents are commonly evaluated via human judgments of overall quality,\nthe relationship between quality and these individual factors is less\nwell-studied. In this work, we examine two controllable neural text generation\nmethods, conditional training and weighted decoding, in order to control four\nimportant attributes for chitchat dialogue: repetition, specificity,\nresponse-relatedness and question-asking. We conduct a large-scale human\nevaluation to measure the effect of these control parameters on multi-turn\ninteractive conversations on the PersonaChat task. We provide a detailed\nanalysis of their relationship to high-level aspects of conversation, and show\nthat by controlling combinations of these variables our models obtain clear\nimprovements in human quality judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08691v4",
            "date": "2019-02-22 22:24:37+00:00",
            "title": "Enhancing Clinical Concept Extraction with Contextual Embeddings",
            "authors": [
                "Yuqi Si",
                "Jingqi Wang",
                "Hua Xu",
                "Kirk Roberts"
            ],
            "abstract": "Neural network-based representations (\"embeddings\") have dramatically\nadvanced natural language processing (NLP) tasks, including clinical NLP tasks\nsuch as concept extraction. Recently, however, more advanced embedding methods\nand representations (e.g., ELMo, BERT) have further pushed the state-of-the-art\nin NLP, yet there are no common best practices for how to integrate these\nrepresentations into clinical tasks. The purpose of this study, then, is to\nexplore the space of possible options in utilizing these new models for\nclinical concept extraction, including comparing these to traditional word\nembedding methods (word2vec, GloVe, fastText). Both off-the-shelf open-domain\nembeddings and pre-trained clinical embeddings from MIMIC-III are evaluated. We\nexplore a battery of embedding methods consisting of traditional word\nembeddings and contextual embeddings, and compare these on four concept\nextraction corpora: i2b2 2010, i2b2 2012, SemEval 2014, and SemEval 2015. We\nalso analyze the impact of the pre-training time of a large language model like\nELMo or BERT on the extraction performance. Last, we present an intuitive way\nto understand the semantic information encoded by contextual embeddings.\nContextual embeddings pre-trained on a large clinical corpus achieves new\nstate-of-the-art performances across all concept extraction tasks. The\nbest-performing model outperforms all state-of-the-art methods with respective\nF1-measures of 90.25, 93.18 (partial), 80.74, and 81.65. We demonstrate the\npotential of contextual embeddings through the state-of-the-art performance\nthese methods achieve on clinical concept extraction. Additionally, we\ndemonstrate contextual embeddings encode valuable semantic information not\naccounted for in traditional word representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08795v2",
            "date": "2019-02-23 14:25:51+00:00",
            "title": "VCWE: Visual Character-Enhanced Word Embeddings",
            "authors": [
                "Chi Sun",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Chinese is a logographic writing system, and the shape of Chinese characters\ncontain rich syntactic and semantic information. In this paper, we propose a\nmodel to learn Chinese word embeddings via three-level composition: (1) a\nconvolutional neural network to extract the intra-character compositionality\nfrom the visual shape of a character; (2) a recurrent neural network with\nself-attention to compose character representation into word embeddings; (3)\nthe Skip-Gram framework to capture non-compositionality directly from the\ncontextual information. Evaluations demonstrate the superior performance of our\nmodel on four tasks: word similarity, sentiment analysis, named entity\nrecognition and part-of-speech tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08816v1",
            "date": "2019-02-23 17:04:54+00:00",
            "title": "Augmenting Neural Machine Translation with Knowledge Graphs",
            "authors": [
                "Diego Moussallem",
                "Mihael Ar\u010dan",
                "Axel-Cyrille Ngonga Ngomo",
                "Paul Buitelaar"
            ],
            "abstract": "While neural networks have been used extensively to make substantial progress\nin the machine translation task, they are known for being heavily dependent on\nthe availability of large amounts of training data. Recent efforts have tried\nto alleviate the data sparsity problem by augmenting the training data using\ndifferent strategies, such as back-translation. Along with the data scarcity,\nthe out-of-vocabulary words, mostly entities and terminological expressions,\npose a difficult challenge to Neural Machine Translation systems. In this\npaper, we hypothesize that knowledge graphs enhance the semantic feature\nextraction of neural models, thus optimizing the translation of entities and\nterminological expressions in texts and consequently leading to a better\ntranslation quality. We hence investigate two different strategies for\nincorporating knowledge graphs into neural models without modifying the neural\nnetwork architectures. We also examine the effectiveness of our augmentation\nmethod to recurrent and non-recurrent (self-attentional) neural architectures.\nOur knowledge graph augmented neural translation model, dubbed KG-NMT, achieves\nsignificant and consistent improvements of +3 BLEU, METEOR and chrF3 on average\non the newstest datasets between 2014 and 2018 for WMT English-German\ntranslation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08830v1",
            "date": "2019-02-23 19:21:08+00:00",
            "title": "Categorization in the Wild: Generalizing Cognitive Models to Naturalistic Data across Languages",
            "authors": [
                "Lea Frermann",
                "Mirella Lapata"
            ],
            "abstract": "Categories such as animal or furniture are acquired at an early age and play\nan important role in processing, organizing, and communicating world knowledge.\nCategories exist across cultures: they allow to efficiently represent the\ncomplexity of the world, and members of a community strongly agree on their\nnature, revealing a shared mental representation. Models of category learning\nand representation, however, are typically tested on data from small-scale\nexperiments involving small sets of concepts with artificially restricted\nfeatures; and experiments predominantly involve participants of selected\ncultural and socio-economical groups (very often involving western native\nspeakers of English such as U.S. college students) . This work investigates\nwhether models of categorization generalize (a) to rich and noisy data\napproximating the environment humans live in; and (b) across languages and\ncultures. We present a Bayesian cognitive model designed to jointly learn\ncategories and their structured representation from natural language text which\nallows us to (a) evaluate performance on a large scale, and (b) apply our model\nto a diverse set of languages. We show that meaningful categories comprising\nhundreds of concepts and richly structured featural representations emerge\nacross languages. Our work illustrates the potential of recent advances in\ncomputational modeling and large scale naturalistic datasets for cognitive\nscience research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08832v1",
            "date": "2019-02-23 19:21:24+00:00",
            "title": "Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses",
            "authors": [
                "Ananya B. Sai",
                "Mithun Das Gupta",
                "Mitesh M. Khapra",
                "Mukundhan Srinivasan"
            ],
            "abstract": "Automatically evaluating the quality of dialogue responses for unstructured\ndomains is a challenging problem. ADEM(Lowe et al. 2017) formulated the\nautomatic evaluation of dialogue systems as a learning problem and showed that\nsuch a model was able to predict responses which correlate significantly with\nhuman judgements, both at utterance and system level. Their system was shown to\nhave beaten word-overlap metrics such as BLEU with large margins. We start with\nthe question of whether an adversary can game the ADEM model. We design a\nbattery of targeted attacks at the neural network based ADEM evaluation system\nand show that automatic evaluation of dialogue systems still has a long way to\ngo. ADEM can get confused with a variation as simple as reversing the word\norder in the text! We report experiments on several such adversarial scenarios\nthat draw out counterintuitive scores on the dialogue responses. We take a\nsystematic look at the scoring function proposed by ADEM and connect it to\nlinear system theory to predict the shortcomings evident in the system. We also\ndevise an attack that can fool such a system to rate a response generation\nsystem as favorable. Finally, we allude to future research directions of using\nthe adversarial attacks to design a truly automated dialogue evaluation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08850v3",
            "date": "2019-02-23 21:35:54+00:00",
            "title": "Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation",
            "authors": [
                "Radu Tudor Ionescu",
                "Andrei M. Butnaru"
            ],
            "abstract": "In this paper, we propose a novel representation for text documents based on\naggregating word embedding vectors into document embeddings. Our approach is\ninspired by the Vector of Locally-Aggregated Descriptors used for image\nrepresentation, and it works as follows. First, the word embeddings gathered\nfrom a collection of documents are clustered by k-means in order to learn a\ncodebook of semnatically-related word embeddings. Each word embedding is then\nassociated to its nearest cluster centroid (codeword). The Vector of\nLocally-Aggregated Word Embeddings (VLAWE) representation of a document is then\ncomputed by accumulating the differences between each codeword vector and each\nword vector (from the document) associated to the respective codeword. We plug\nthe VLAWE representation, which is learned in an unsupervised manner, into a\nclassifier and show that it is useful for a diverse set of text classification\ntasks. We compare our approach with a broad range of recent state-of-the-art\nmethods, demonstrating the effectiveness of our approach. Furthermore, we\nobtain a considerable improvement on the Movie Review data set, reporting an\naccuracy of 93.3%, which represents an absolute gain of 10% over the\nstate-of-the-art approach. Our code is available at\nhttps://github.com/raduionescu/vlawe-boswe/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08852v2",
            "date": "2019-02-23 21:53:48+00:00",
            "title": "Evidence Sentence Extraction for Machine Reading Comprehension",
            "authors": [
                "Hai Wang",
                "Dian Yu",
                "Kai Sun",
                "Jianshu Chen",
                "Dong Yu",
                "David McAllester",
                "Dan Roth"
            ],
            "abstract": "Remarkable success has been achieved in the last few years on some limited\nmachine reading comprehension (MRC) tasks. However, it is still difficult to\ninterpret the predictions of existing MRC models. In this paper, we focus on\nextracting evidence sentences that can explain or support the answers of\nmultiple-choice MRC tasks, where the majority of answer options cannot be\ndirectly extracted from reference documents.\n  Due to the lack of ground truth evidence sentence labels in most cases, we\napply distant supervision to generate imperfect labels and then use them to\ntrain an evidence sentence extractor. To denoise the noisy labels, we apply a\nrecently proposed deep probabilistic logic learning framework to incorporate\nboth sentence-level and cross-sentence linguistic indicators for indirect\nsupervision. We feed the extracted evidence sentences into existing MRC models\nand evaluate the end-to-end performance on three challenging multiple-choice\nMRC datasets: MultiRC, RACE, and DREAM, achieving comparable or better\nperformance than the same models that take as input the full reference\ndocument. To the best of our knowledge, this is the first work extracting\nevidence sentences for multiple-choice MRC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08856v1",
            "date": "2019-02-23 22:17:08+00:00",
            "title": "ABI Neural Ensemble Model for Gender Prediction Adapt Bar-Ilan Submission for the CLIN29 Shared Task on Gender Prediction",
            "authors": [
                "Eva Vanmassenhove",
                "Amit Moryossef",
                "Alberto Poncelas",
                "Andy Way",
                "Dimitar Shterionov"
            ],
            "abstract": "We present our system for the CLIN29 shared task on cross-genre gender\ndetection for Dutch. We experimented with a multitude of neural models (CNN,\nRNN, LSTM, etc.), more \"traditional\" models (SVM, RF, LogReg, etc.), different\nfeature sets as well as data pre-processing. The final results suggested that\nusing tokenized, non-lowercased data works best for most of the neural models,\nwhile a combination of word clusters, character trigrams and word lists showed\nto be most beneficial for the majority of the more \"traditional\" (that is,\nnon-neural) models, beating features used in previous tasks such as n-grams,\ncharacter n-grams, part-of-speech tags and combinations thereof. In\ncontradiction with the results described in previous comparable shared tasks,\nour neural models performed better than our best traditional approaches with\nour best feature set-up. Our final model consisted of a weighted ensemble model\ncombining the top 25 models. Our final model won both the in-domain gender\nprediction task and the cross-genre challenge, achieving an average accuracy of\n64.93% on the in-domain gender prediction task, and 56.26% on cross-genre\ngender prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08899v1",
            "date": "2019-02-24 06:40:47+00:00",
            "title": "The ARIEL-CMU Systems for LoReHLT18",
            "authors": [
                "Aditi Chaudhary",
                "Siddharth Dalmia",
                "Junjie Hu",
                "Xinjian Li",
                "Austin Matthews",
                "Aldrian Obaja Muis",
                "Naoki Otani",
                "Shruti Rijhwani",
                "Zaid Sheikh",
                "Nidhi Vyas",
                "Xinyi Wang",
                "Jiateng Xie",
                "Ruochen Xu",
                "Chunting Zhou",
                "Peter J. Jansen",
                "Yiming Yang",
                "Lori Levin",
                "Florian Metze",
                "Teruko Mitamura",
                "David R. Mortensen",
                "Graham Neubig",
                "Eduard Hovy",
                "Alan W Black",
                "Jaime Carbonell",
                "Graham V. Horwood",
                "Shabnam Tafreshi",
                "Mona Diab",
                "Efsun S. Kayi",
                "Noura Farra",
                "Kathleen McKeown"
            ],
            "abstract": "This paper describes the ARIEL-CMU submissions to the Low Resource Human\nLanguage Technologies (LoReHLT) 2018 evaluations for the tasks Machine\nTranslation (MT), Entity Discovery and Linking (EDL), and detection of\nSituation Frames in Text and Speech (SF Text and Speech).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08906v2",
            "date": "2019-02-24 07:44:00+00:00",
            "title": "On the Use of Emojis to Train Emotion Classifiers",
            "authors": [
                "Wegdan Hussien",
                "Mahmoud Al-Ayyoub",
                "Yahya Tashtoush",
                "Mohammed Al-Kabi"
            ],
            "abstract": "Nowadays, the automatic detection of emotions is employed by many\napplications in different fields like security informatics, e-learning, humor\ndetection, targeted advertising, etc. Many of these applications focus on\nsocial media and treat this problem as a classification problem, which requires\npreparing training data. The typical method for annotating the training data by\nhuman experts is considered time consuming, labor intensive and sometimes prone\nto error. Moreover, such an approach is not easily extensible to new\ndomains/languages since such extensions require annotating new training data.\nIn this study, we propose a distant supervised learning approach where the\ntraining sentences are automatically annotated based on the emojis they have.\nSuch training data would be very cheap to produce compared with the manually\ncreated training data, thus, much larger training data can be easily obtained.\nOn the other hand, this training data would naturally have lower quality as it\nmay contain some errors in the annotation. Nonetheless, we experimentally show\nthat training classifiers on cheap, large and possibly erroneous data annotated\nusing this approach leads to more accurate results compared with training the\nsame classifiers on the more expensive, much smaller and error-free manually\nannotated training data. Our experiments are conducted on an in-house dataset\nof emotional Arabic tweets and the classifiers we consider are: Support Vector\nMachine (SVM), Multinomial Naive Bayes (MNB) and Random Forest (RF). In\naddition to experimenting with single classifiers, we also consider using an\nensemble of classifiers. The results show that using an automatically annotated\ntraining data (that is only one order of magnitude larger than the manually\nannotated one) gives better results in almost all settings considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08912v1",
            "date": "2019-02-24 09:59:33+00:00",
            "title": "Unlexicalized Transition-based Discontinuous Constituency Parsing",
            "authors": [
                "Maximin Coavoux",
                "Beno\u00eet Crabb\u00e9",
                "Shay B. Cohen"
            ],
            "abstract": "Lexicalized parsing models are based on the assumptions that (i) constituents\nare organized around a lexical head (ii) bilexical statistics are crucial to\nsolve ambiguities. In this paper, we introduce an unlexicalized\ntransition-based parser for discontinuous constituency structures, based on a\nstructure-label transition system and a bi-LSTM scoring system. We compare it\nto lexicalized parsing models in order to address the question of\nlexicalization in the context of discontinuous constituency parsing. Our\nexperiments show that unlexicalized models systematically achieve higher\nresults than lexicalized models, and provide additional empirical evidence that\nlexicalization is not necessary to achieve strong parsing results. Our best\nunlexicalized model sets a new state of the art on English and German\ndiscontinuous constituency treebanks. We further provide a per-phenomenon\nanalysis of its errors on discontinuous constituents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08939v2",
            "date": "2019-02-24 13:18:27+00:00",
            "title": "Text Analysis in Adversarial Settings: Does Deception Leave a Stylistic Trace?",
            "authors": [
                "Tommi Gr\u00f6ndahl",
                "N. Asokan"
            ],
            "abstract": "Textual deception constitutes a major problem for online security. Many\nstudies have argued that deceptiveness leaves traces in writing style, which\ncould be detected using text classification techniques. By conducting an\nextensive literature review of existing empirical work, we demonstrate that\nwhile certain linguistic features have been indicative of deception in certain\ncorpora, they fail to generalize across divergent semantic domains. We suggest\nthat deceptiveness as such leaves no content-invariant stylistic trace, and\ntextual similarity measures provide superior means of classifying texts as\npotentially deceptive. Additionally, we discuss forms of deception beyond\nsemantic content, focusing on hiding author identity by writing style\nobfuscation. Surveying the literature on both author identification and\nobfuscation techniques, we conclude that current style transformation methods\nfail to achieve reliable obfuscation while simultaneously ensuring semantic\nfaithfulness to the original text. We propose that future work in style\ntransformation should pay particular attention to disallowing semantically\ndrastic changes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09087v1",
            "date": "2019-02-25 04:46:52+00:00",
            "title": "Lattice CNNs for Matching Based Chinese Question Answering",
            "authors": [
                "Yuxuan Lai",
                "Yansong Feng",
                "Xiaohan Yu",
                "Zheng Wang",
                "Kun Xu",
                "Dongyan Zhao"
            ],
            "abstract": "Short text matching often faces the challenges that there are great word\nmismatch and expression diversity between the two texts, which would be further\naggravated in languages like Chinese where there is no natural space to segment\nwords explicitly. In this paper, we propose a novel lattice based CNN model\n(LCNs) to utilize multi-granularity information inherent in the word lattice\nwhile maintaining strong ability to deal with the introduced noisy information\nfor matching based question answering in Chinese. We conduct extensive\nexperiments on both document based question answering and knowledge based\nquestion answering tasks, and experimental results show that the LCNs models\ncan significantly outperform the state-of-the-art matching models and strong\nbaselines by taking advantages of better ability to distill rich but\ndiscriminative information from the word lattice input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09113v2",
            "date": "2019-02-25 07:07:38+00:00",
            "title": "Star-Transformer",
            "authors": [
                "Qipeng Guo",
                "Xipeng Qiu",
                "Pengfei Liu",
                "Yunfan Shao",
                "Xiangyang Xue",
                "Zheng Zhang"
            ],
            "abstract": "Although Transformer has achieved great successes on many NLP tasks, its\nheavy structure with fully-connected attention connections leads to\ndependencies on large training data. In this paper, we present\nStar-Transformer, a lightweight alternative by careful sparsification. To\nreduce model complexity, we replace the fully-connected structure with a\nstar-shaped topology, in which every two non-adjacent nodes are connected\nthrough a shared relay node. Thus, complexity is reduced from quadratic to\nlinear, while preserving capacity to capture both local composition and\nlong-range dependency. The experiments on four tasks (22 datasets) show that\nStar-Transformer achieved significant improvements against the standard\nTransformer for the modestly sized datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09183v1",
            "date": "2019-02-25 10:31:57+00:00",
            "title": "Joint Multi-Domain Learning for Automatic Short Answer Grading",
            "authors": [
                "Swarnadeep Saha",
                "Tejas I. Dhamecha",
                "Smit Marvaniya",
                "Peter Foltz",
                "Renuka Sindhgatta",
                "Bikram Sengupta"
            ],
            "abstract": "One of the fundamental challenges towards building any intelligent tutoring\nsystem is its ability to automatically grade short student answers. A typical\nautomatic short answer grading system (ASAG) grades student answers across\nmultiple domains (or subjects). Grading student answers requires building a\nsupervised machine learning model that evaluates the similarity of the student\nanswer with the reference answer(s). We observe that unlike typical textual\nsimilarity or entailment tasks, the notion of similarity is not universal here.\nOn one hand, para-phrasal constructs of the language can indicate similarity\nindependent of the domain. On the other hand, two words, or phrases, that are\nnot strict synonyms of each other, might mean the same in certain domains.\nBuilding on this observation, we propose JMD-ASAG, the first joint multidomain\ndeep learning architecture for automatic short answer grading that performs\ndomain adaptation by learning generic and domain-specific aspects from the\nlimited domain-wise training data. JMD-ASAG not only learns the domain-specific\ncharacteristics but also overcomes the dependence on a large corpus by learning\nthe generic characteristics from the task-specific data itself. On a\nlarge-scale industry dataset and a benchmarking dataset, we show that our model\nperforms significantly better than existing techniques which either learn\ndomain-specific models or adapt a generic similarity scoring model from a large\ncorpus. Further, on the benchmarking dataset, we report state-of-the-art\nresults against all existing non-neural and neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09271v1",
            "date": "2019-02-25 14:09:03+00:00",
            "title": "Relation Extraction using Explicit Context Conditioning",
            "authors": [
                "Gaurav Singh",
                "Parminder Bhatia"
            ],
            "abstract": "Relation Extraction (RE) aims to label relations between groups of marked\nentities in raw text. Most current RE models learn context-aware\nrepresentations of the target entities that are then used to establish relation\nbetween them. This works well for intra-sentence RE and we call them\nfirst-order relations. However, this methodology can sometimes fail to capture\ncomplex and long dependencies. To address this, we hypothesize that at times\ntwo target entities can be explicitly connected via a context token. We refer\nto such indirect relations as second-order relations and describe an efficient\nimplementation for computing them. These second-order relation scores are then\ncombined with first-order relation scores. Our empirical results show that the\nproposed method leads to state-of-the-art performance over two biomedical\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09314v2",
            "date": "2019-02-25 14:51:46+00:00",
            "title": "Attentional Encoder Network for Targeted Sentiment Classification",
            "authors": [
                "Youwei Song",
                "Jiahai Wang",
                "Tao Jiang",
                "Zhiyue Liu",
                "Yanghui Rao"
            ],
            "abstract": "Targeted sentiment classification aims at determining the sentimental\ntendency towards specific targets. Most of the previous approaches model\ncontext and target words with RNN and attention. However, RNNs are difficult to\nparallelize and truncated backpropagation through time brings difficulty in\nremembering long-term patterns. To address this issue, this paper proposes an\nAttentional Encoder Network (AEN) which eschews recurrence and employs\nattention based encoders for the modeling between context and target. We raise\nthe label unreliability issue and introduce label smoothing regularization. We\nalso apply pre-trained BERT to this task and obtain new state-of-the-art\nresults. Experiments and analysis demonstrate the effectiveness and lightweight\nof our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09381v4",
            "date": "2019-02-25 15:49:10+00:00",
            "title": "EAT: a simple and versatile semantic representation format for multi-purpose NLP",
            "authors": [
                "Tommi Gr\u00f6ndahl"
            ],
            "abstract": "Semantic representations are central in many NLP tasks that require\nhuman-interpretable data. The conjunctivist framework - primarily developed by\nPietroski (2005, 2018) - obtains expressive representations with only a few\nbasic semantic types and relations systematically linked to syntactic\npositions. While representational simplicity is crucial for computational\napplications, such findings have not yet had major influence on NLP. We present\nthe first generic semantic representation format for NLP directly based on\nthese insights. We name the format EAT due to its basis in the Event-, Agent-,\nand Theme arguments in Neo-Davidsonian logical forms. It builds on the idea\nthat similar tripartite argument relations are ubiquitous across categories,\nand can be constructed from grammatical structure without additional lexical\ninformation. We present a detailed exposition of EAT and how it relates to\nother prevalent formats used in prior work, such as Abstract Meaning\nRepresentation (AMR) and Minimal Recursion Semantics (MRS). EAT stands out in\ntwo respects: simplicity and versatility. Uniquely, EAT discards semantic\nmetapredicates, and instead represents semantic roles entirely via positional\nencoding. This is made possible by limiting the number of roles to only three;\na major decrease from the many dozens recognized in e.g. AMR and MRS. EAT's\nsimplicity makes it exceptionally versatile in application. First, we show that\ndrastically reducing semantic roles based on EAT benefits text generation from\nMRS in the test settings of Hajdik et al. (2019). Second, we implement the\nderivation of EAT from a syntactic parse, and apply this for parallel corpus\ngeneration between grammatical classes. Third, we train an encoder-decoder LSTM\nnetwork to map EAT to English. Finally, we use both the encoder-decoder network\nand a rule-based alternative to conduct grammatical transformation from\nEAT-input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09514v4",
            "date": "2019-02-25 18:54:10+00:00",
            "title": "Lost in Machine Translation: A Method to Reduce Meaning Loss",
            "authors": [
                "Reuben Cohn-Gordon",
                "Noah Goodman"
            ],
            "abstract": "A desideratum of high-quality translation systems is that they preserve\nmeaning, in the sense that two sentences with different meanings should not\ntranslate to one and the same sentence in another language. However,\nstate-of-the-art systems often fail in this regard, particularly in cases where\nthe source and target languages partition the \"meaning space\" in different\nways. For instance, \"I cut my finger.\" and \"I cut my finger off.\" describe\ndifferent states of the world but are translated to French (by both Fairseq and\nGoogle Translate) as \"Je me suis coupe le doigt.\", which is ambiguous as to\nwhether the finger is detached. More generally, translation systems are\ntypically many-to-one (non-injective) functions from source to target language,\nwhich in many cases results in important distinctions in meaning being lost in\ntranslation. Building on Bayesian models of informative utterance production,\nwe present a method to define a less ambiguous translation system in terms of\nan underlying pre-trained neural sequence-to-sequence model. This method\nincreases injectivity, resulting in greater preservation of meaning as measured\nby improvement in cycle-consistency, without impeding translation quality\n(measured by BLEU score).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09666v2",
            "date": "2019-02-25 23:54:40+00:00",
            "title": "Predicting the Type and Target of Offensive Posts in Social Media",
            "authors": [
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Preslav Nakov",
                "Sara Rosenthal",
                "Noura Farra",
                "Ritesh Kumar"
            ],
            "abstract": "As offensive content has become pervasive in social media, there has been\nmuch research in identifying potentially offensive messages. However, previous\nwork on this topic did not consider the problem as a whole, but rather focused\non detecting very specific types of offensive content, e.g., hate speech,\ncyberbulling, or cyber-aggression. In contrast, here we target several\ndifferent kinds of offensive content. In particular, we model the task\nhierarchically, identifying the type and the target of offensive messages in\nsocial media. For this purpose, we complied the Offensive Language\nIdentification Dataset (OLID), a new dataset with tweets annotated for\noffensive content using a fine-grained three-layer annotation scheme, which we\nmake publicly available. We discuss the main similarities and differences\nbetween OLID and pre-existing datasets for hate speech identification,\naggression detection, and similar tasks. We further experiment with and we\ncompare the performance of different machine learning models on OLID.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09674v1",
            "date": "2019-02-26 00:45:56+00:00",
            "title": "Developing and Using Special-Purpose Lexicons for Cohort Selection from Clinical Notes",
            "authors": [
                "Samarth Rawal",
                "Ashok Prakash",
                "Soumya Adhya",
                "Sidharth Kulkarni",
                "Saadat Anwar",
                "Chitta Baral",
                "Murthy Devarakonda"
            ],
            "abstract": "Background and Significance: Selecting cohorts for a clinical trial typically\nrequires costly and time-consuming manual chart reviews resulting in poor\nparticipation. To help automate the process, National NLP Clinical Challenges\n(N2C2) conducted a shared challenge by defining 13 criteria for clinical trial\ncohort selection and by providing training and test datasets. This research was\nmotivated by the N2C2 challenge.\n  Methods: We broke down the task into 13 independent subtasks corresponding to\neach criterion and implemented subtasks using rules or a supervised machine\nlearning model. Each task critically depended on knowledge resources in the\nform of task-specific lexicons, for which we developed a novel model-driven\napproach. The approach allowed us to first expand the lexicon from a seed set\nand then remove noise from the list, thus improving the accuracy.\n  Results: Our system achieved an overall F measure of 0.9003 at the challenge,\nand was statistically tied for the first place out of 45 participants. The\nmodel-driven lexicon development and further debugging the rules/code on the\ntraining set improved overall F measure to 0.9140, overtaking the best\nnumerical result at the challenge.\n  Discussion: Cohort selection, like phenotype extraction and classification,\nis amenable to rule-based or simple machine learning methods, however, the\nlexicons involved, such as medication names or medical terms referring to a\nmedical problem, critically determine the overall accuracy. Automated lexicon\ndevelopment has the potential for scalability and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09697v2",
            "date": "2019-02-26 01:50:09+00:00",
            "title": "Polyglot Contextual Representations Improve Crosslingual Transfer",
            "authors": [
                "Phoebe Mulcaire",
                "Jungo Kasai",
                "Noah A. Smith"
            ],
            "abstract": "We introduce Rosita, a method to produce multilingual contextual word\nrepresentations by training a single language model on text from multiple\nlanguages. Our method combines the advantages of contextual word\nrepresentations with those of multilingual representation learning. We produce\nlanguage models from dissimilar language pairs (English/Arabic and\nEnglish/Chinese) and use them in dependency parsing, semantic role labeling,\nand named entity recognition, with comparisons to monolingual and\nnon-contextual variants. Our results provide further evidence for the benefits\nof polyglot learning, in which representations are shared across multiple\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09781v1",
            "date": "2019-02-26 07:57:22+00:00",
            "title": "Recursive Subtree Composition in LSTM-Based Dependency Parsing",
            "authors": [
                "Miryam de Lhoneux",
                "Miguel Ballesteros",
                "Joakim Nivre"
            ],
            "abstract": "The need for tree structure modelling on top of sequence modelling is an open\nissue in neural dependency parsing. We investigate the impact of adding a tree\nlayer on top of a sequential model by recursively composing subtree\nrepresentations (composition) in a transition-based parser that uses features\nextracted by a BiLSTM. Composition seems superfluous with such a model,\nsuggesting that BiLSTMs capture information about subtrees. We perform model\nablations to tease out the conditions under which composition helps. When\nablating the backward LSTM, performance drops and composition does not recover\nmuch of the gap. When ablating the forward LSTM, performance drops less\ndramatically and composition recovers a substantial part of the gap, indicating\nthat a forward LSTM and composition capture similar information. We take the\nbackward LSTM to be related to lookahead features and the forward LSTM to the\nrich history-based features both crucial for transition-based parsers. To\ncapture history-based information, composition is better than a forward LSTM on\nits own, but it is even better to have a forward LSTM as part of a BiLSTM. We\ncorrelate results with language properties, showing that the improved lookahead\nof a backward LSTM is especially important for head-final languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09802v1",
            "date": "2019-02-26 08:46:15+00:00",
            "title": "Semantic Hilbert Space for Text Representation Learning",
            "authors": [
                "Benyou Wang",
                "Qiuchi Li",
                "Massimo Melucci",
                "Dawei Song"
            ],
            "abstract": "Capturing the meaning of sentences has long been a challenging task. Current\nmodels tend to apply linear combinations of word features to conduct semantic\ncomposition for bigger-granularity units e.g. phrases, sentences, and\ndocuments. However, the semantic linearity does not always hold in human\nlanguage. For instance, the meaning of the phrase `ivory tower' can not be\ndeduced by linearly combining the meanings of `ivory' and `tower'. To address\nthis issue, we propose a new framework that models different levels of semantic\nunits (e.g. sememe, word, sentence, and semantic abstraction) on a single\n\\textit{Semantic Hilbert Space}, which naturally admits a non-linear semantic\ncomposition by means of a complex-valued vector word representation. An\nend-to-end neural network~\\footnote{https://github.com/wabyking/qnn} is\nproposed to implement the framework in the text classification task, and\nevaluation results on six benchmarking text classification datasets demonstrate\nthe effectiveness, robustness and self-explanation power of the proposed model.\nFurthermore, intuitive case studies are conducted to help end users to\nunderstand how the framework works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09875v1",
            "date": "2019-02-26 11:55:35+00:00",
            "title": "Improving a tf-idf weighted document vector embedding",
            "authors": [
                "Craig W. Schmidt"
            ],
            "abstract": "We examine a number of methods to compute a dense vector embedding for a\ndocument in a corpus, given a set of word vectors such as those from word2vec\nor GloVe. We describe two methods that can improve upon a simple weighted sum,\nthat are optimal in the sense that they maximizes a particular weighted cosine\nsimilarity measure.\n  We consider several weighting functions, including inverse document frequency\n(idf), smooth inverse frequency (SIF), and the sub-sampling function used in\nword2vec. We find that idf works best for our applications. We also use common\ncomponent removal proposed by Arora et al. as a post-process and find it is\nhelpful in most cases.\n  We compare these embeddings variations to the doc2vec embedding on a new\nevaluation task using TripAdvisor reviews, and also on the CQADupStack\nbenchmark from the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10068v2",
            "date": "2019-02-26 17:29:43+00:00",
            "title": "Entity Recognition at First Sight: Improving NER with Eye Movement Information",
            "authors": [
                "Nora Hollenstein",
                "Ce Zhang"
            ],
            "abstract": "Previous research shows that eye-tracking data contains information about the\nlexical and syntactic properties of text, which can be used to improve natural\nlanguage processing models. In this work, we leverage eye movement features\nfrom three corpora with recorded gaze information to augment a state-of-the-art\nneural model for named entity recognition (NER) with gaze embeddings. These\ncorpora were manually annotated with named entity labels. Moreover, we show how\ngaze features, generalized on word type level, eliminate the need for recorded\neye-tracking data at test time. The gaze-augmented models for NER using\ntoken-level and type-level features outperform the baselines. We present the\nbenefits of eye-tracking features by evaluating the NER models on both\nindividual datasets as well as in cross-domain settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10118v1",
            "date": "2019-02-26 18:53:22+00:00",
            "title": "Multi-Task Learning with Contextualized Word Representations for Extented Named Entity Recognition",
            "authors": [
                "Thai-Hoang Pham",
                "Khai Mai",
                "Nguyen Minh Trung",
                "Nguyen Tuan Duc",
                "Danushka Bolegala",
                "Ryohei Sasano",
                "Satoshi Sekine"
            ],
            "abstract": "Fine-Grained Named Entity Recognition (FG-NER) is critical for many NLP\napplications. While classical named entity recognition (NER) has attracted a\nsubstantial amount of research, FG-NER is still an open research domain. The\ncurrent state-of-the-art (SOTA) model for FG-NER relies heavily on manual\nefforts for building a dictionary and designing hand-crafted features. The\nend-to-end framework which achieved the SOTA result for NER did not get the\ncompetitive result compared to SOTA model for FG-NER. In this paper, we\ninvestigate how effective multi-task learning approaches are in an end-to-end\nframework for FG-NER in different aspects. Our experiments show that using\nmulti-task learning approaches with contextualized word representation can help\nan end-to-end neural network model achieve SOTA results without using any\nadditional manual effort for creating data and designing features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10193v3",
            "date": "2019-02-26 20:11:27+00:00",
            "title": "On the Idiosyncrasies of the Mandarin Chinese Classifier System",
            "authors": [
                "Shijia Liu",
                "Hongyuan Mei",
                "Adina Williams",
                "Ryan Cotterell"
            ],
            "abstract": "While idiosyncrasies of the Chinese classifier system have been a richly\nstudied topic among linguists (Adams and Conklin, 1973; Erbaugh, 1986; Lakoff,\n1986), not much work has been done to quantify them with statistical methods.\nIn this paper, we introduce an information-theoretic approach to measuring\nidiosyncrasy; we examine how much the uncertainty in Mandarin Chinese\nclassifiers can be reduced by knowing semantic information about the nouns that\nthe classifiers modify. Using the empirical distribution of classifiers from\nthe parsed Chinese Gigaword corpus (Graff et al., 2005), we compute the mutual\ninformation (in bits) between the distribution over classifiers and\ndistributions over other linguistic quantities. We investigate whether semantic\nclasses of nouns and adjectives differ in how much they reduce uncertainty in\nclassifier choice, and find that it is not fully idiosyncratic; while there are\nno obvious trends for the majority of semantic classes, shape nouns reduce\nuncertainty in classifier choice the most.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10236v2",
            "date": "2019-02-26 21:33:48+00:00",
            "title": "Learning When Not to Answer: A Ternary Reward Structure for Reinforcement Learning based Question Answering",
            "authors": [
                "Fr\u00e9deric Godin",
                "Anjishnu Kumar",
                "Arpit Mittal"
            ],
            "abstract": "In this paper, we investigate the challenges of using reinforcement learning\nagents for question-answering over knowledge graphs for real-world\napplications. We examine the performance metrics used by state-of-the-art\nsystems and determine that they are inadequate for such settings. More\nspecifically, they do not evaluate the systems correctly for situations when\nthere is no answer available and thus agents optimized for these metrics are\npoor at modeling confidence. We introduce a simple new performance metric for\nevaluating question-answering agents that is more representative of practical\nusage conditions, and optimize for this metric by extending the binary reward\nstructure used in prior work to a ternary reward structure which also rewards\nan agent for not answering a question rather than giving an incorrect answer.\nWe show that this can drastically improve the precision of answered questions\nwhile only not answering a limited number of previously correctly answered\nquestions. Employing a supervised learning strategy using depth-first-search\npaths to bootstrap the reinforcement learning algorithm further improves\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10296v2",
            "date": "2019-02-27 01:43:48+00:00",
            "title": "A Framework for Decoding Event-Related Potentials from Text",
            "authors": [
                "Shaorong Yan",
                "Aaron Steven White"
            ],
            "abstract": "We propose a novel framework for modeling event-related potentials (ERPs)\ncollected during reading that couples pre-trained convolutional decoders with a\nlanguage model. Using this framework, we compare the abilities of a variety of\nexisting and novel sentence processing models to reconstruct ERPs. We find that\nmodern contextual word embeddings underperform surprisal-based models but that,\ncombined, the two outperform either on its own.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10326v1",
            "date": "2019-02-27 04:28:33+00:00",
            "title": "CN-Probase: A Data-driven Approach for Large-scale Chinese Taxonomy Construction",
            "authors": [
                "Jindong Chen",
                "Ao Wang",
                "Jiangjie Chen",
                "Yanghua Xiao",
                "Zhendong Chu",
                "Jingping Liu",
                "Jiaqing Liang",
                "Wei Wang"
            ],
            "abstract": "Taxonomies play an important role in machine intelligence. However, most\nwell-known taxonomies are in English, and non-English taxonomies, especially\nChinese ones, are still very rare. In this paper, we focus on automatic Chinese\ntaxonomy construction and propose an effective generation and verification\nframework to build a large-scale and high-quality Chinese taxonomy. In the\ngeneration module, we extract isA relations from multiple sources of Chinese\nencyclopedia, which ensures the coverage. To further improve the precision of\ntaxonomy, we apply three heuristic approaches in verification module. As a\nresult, we construct the largest Chinese taxonomy with high precision about 95%\ncalled CN-Probase. Our taxonomy has been deployed on Aliyun, with over 82\nmillion API calls in six months.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10374v1",
            "date": "2019-02-27 07:54:54+00:00",
            "title": "Domain-Constrained Advertising Keyword Generation",
            "authors": [
                "Hao Zhou",
                "Minlie Huang",
                "Yishun Mao",
                "Changlei Zhu",
                "Peng Shu",
                "Xiaoyan Zhu"
            ],
            "abstract": "Advertising (ad for short) keyword suggestion is important for sponsored\nsearch to improve online advertising and increase search revenue. There are two\ncommon challenges in this task. First, the keyword bidding problem: hot ad\nkeywords are very expensive for most of the advertisers because more\nadvertisers are bidding on more popular keywords, while unpopular keywords are\ndifficult to discover. As a result, most ads have few chances to be presented\nto the users. Second, the inefficient ad impression issue: a large proportion\nof search queries, which are unpopular yet relevant to many ad keywords, have\nno ads presented on their search result pages. Existing retrieval-based or\nmatching-based methods either deteriorate the bidding competition or are unable\nto suggest novel keywords to cover more queries, which leads to inefficient ad\nimpressions. To address the above issues, this work investigates to use\ngenerative neural networks for keyword generation in sponsored search. Given a\npurchased keyword (a word sequence) as input, our model can generate a set of\nkeywords that are not only relevant to the input but also satisfy the domain\nconstraint which enforces that the domain category of a generated keyword is as\nexpected. Furthermore, a reinforcement learning algorithm is proposed to\nadaptively utilize domain-specific information in keyword generation. Offline\nevaluation shows that the proposed model can generate keywords that are\ndiverse, novel, relevant to the source keyword, and accordant with the domain\nconstraint. Online evaluation shows that generative models can improve coverage\n(COV), click-through rate (CTR), and revenue per mille (RPM) substantially in\nsponsored search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10418v1",
            "date": "2019-02-27 09:56:19+00:00",
            "title": "Learning to Generate Questions by Learning What not to Generate",
            "authors": [
                "Bang Liu",
                "Mingjun Zhao",
                "Di Niu",
                "Kunfeng Lai",
                "Yancheng He",
                "Haojie Wei",
                "Yu Xu"
            ],
            "abstract": "Automatic question generation is an important technique that can improve the\ntraining of question answering, help chatbots to start or continue a\nconversation with humans, and provide assessment materials for educational\npurposes. Existing neural question generation models are not sufficient mainly\ndue to their inability to properly model the process of how each word in the\nquestion is selected, i.e., whether repeating the given passage or being\ngenerated from a vocabulary. In this paper, we propose our Clue Guided Copy\nNetwork for Question Generation (CGC-QG), which is a sequence-to-sequence\ngenerative model with copying mechanism, yet employing a variety of novel\ncomponents and techniques to boost the performance of question generation. In\nCGC-QG, we design a multi-task labeling strategy to identify whether a question\nword should be copied from the input passage or be generated instead, guiding\nthe model to learn the accurate boundaries between copying and generation.\nFurthermore, our input passage encoder takes as input, among a diverse range of\nother features, the prediction made by a clue word predictor, which helps\nidentify whether each word in the input passage is a potential clue to be\ncopied into the target question. The clue word predictor is designed based on a\nnovel application of Graph Convolutional Networks onto a syntactic dependency\ntree representation of each passage, thus being able to predict clue words only\nbased on their context in the passage and their relative positions to the\nanswer in the tree. We jointly train the clue prediction as well as question\ngeneration with multi-task learning and a number of practical strategies to\nreduce the complexity. Extensive evaluations show that our model significantly\nimproves the performance of question generation and out-performs all previous\nstate-of-the-art neural question generation models by a substantial margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10461v3",
            "date": "2019-02-27 11:14:16+00:00",
            "title": "Multilingual Neural Machine Translation with Knowledge Distillation",
            "authors": [
                "Xu Tan",
                "Yi Ren",
                "Di He",
                "Tao Qin",
                "Zhou Zhao",
                "Tie-Yan Liu"
            ],
            "abstract": "Multilingual machine translation, which translates multiple languages with a\nsingle model, has attracted much attention due to its efficiency of offline\ntraining and online serving. However, traditional multilingual translation\nusually yields inferior accuracy compared with the counterpart using individual\nmodels for each language pair, due to language diversity and model capacity\nlimitations. In this paper, we propose a distillation-based approach to boost\nthe accuracy of multilingual machine translation. Specifically, individual\nmodels are first trained and regarded as teachers, and then the multilingual\nmodel is trained to fit the training data and match the outputs of individual\nmodels simultaneously through knowledge distillation. Experiments on IWSLT, WMT\nand Ted talk translation datasets demonstrate the effectiveness of our method.\nParticularly, we show that one model is enough to handle multiple languages (up\nto 44 languages in our experiment), with comparable or even better accuracy\nthan individual models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10482v2",
            "date": "2019-02-27 12:16:55+00:00",
            "title": "Induction Networks for Few-Shot Text Classification",
            "authors": [
                "Ruiying Geng",
                "Binhua Li",
                "Yongbin Li",
                "Xiaodan Zhu",
                "Ping Jian",
                "Jian Sun"
            ],
            "abstract": "Text classification tends to struggle when data is deficient or when it needs\nto adapt to unseen classes. In such challenging scenarios, recent studies have\nused meta-learning to simulate the few-shot task, in which new queries are\ncompared to a small support set at the sample-wise level. However, this\nsample-wise comparison may be severely disturbed by the various expressions in\nthe same class. Therefore, we should be able to learn a general representation\nof each class in the support set and then compare it to new queries. In this\npaper, we propose a novel Induction Network to learn such a generalized\nclass-wise representation, by innovatively leveraging the dynamic routing\nalgorithm in meta-learning. In this way, we find the model is able to induce\nand generalize better. We evaluate the proposed model on a well-studied\nsentiment classification dataset (English) and a real-world dialogue intent\nclassification dataset (Chinese). Experiment results show that on both\ndatasets, the proposed model significantly outperforms the existing\nstate-of-the-art approaches, proving the effectiveness of class-wise\ngeneralization in few-shot text classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10526v3",
            "date": "2019-02-27 13:48:59+00:00",
            "title": "DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion",
            "authors": [
                "Mor Geva",
                "Eric Malmi",
                "Idan Szpektor",
                "Jonathan Berant"
            ],
            "abstract": "Sentence fusion is the task of joining several independent sentences into a\nsingle coherent text. Current datasets for sentence fusion are small and\ninsufficient for training modern neural models. In this paper, we propose a\nmethod for automatically-generating fusion examples from raw text and present\nDiscoFuse, a large scale dataset for discourse-based sentence fusion. We author\na set of rules for identifying a diverse set of discourse phenomena in raw\ntext, and decomposing the text into two independent sentences. We apply our\napproach on two document collections: Wikipedia and Sports articles, yielding\n60 million fusion examples annotated with discourse information required to\nreconstruct the fused text. We develop a sequence-to-sequence model on\nDiscoFuse and thoroughly analyze its strengths and weaknesses with respect to\nthe various discourse phenomena, using both automatic as well as human\nevaluation. Finally, we conduct transfer learning experiments with WebSplit, a\nrecent dataset for text simplification. We show that pretraining on DiscoFuse\nsubstantially improves performance on WebSplit when viewed as a sentence fusion\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10580v1",
            "date": "2019-02-27 15:19:36+00:00",
            "title": "Multiresolution Graph Attention Networks for Relevance Matching",
            "authors": [
                "Ting Zhang",
                "Bang Liu",
                "Di Niu",
                "Kunfeng Lai",
                "Yu Xu"
            ],
            "abstract": "A large number of deep learning models have been proposed for the text\nmatching problem, which is at the core of various typical natural language\nprocessing (NLP) tasks. However, existing deep models are mainly designed for\nthe semantic matching between a pair of short texts, such as paraphrase\nidentification and question answering, and do not perform well on the task of\nrelevance matching between short-long text pairs. This is partially due to the\nfact that the essential characteristics of short-long text matching have not\nbeen well considered in these deep models. More specifically, these methods\nfail to handle extreme length discrepancy between text pieces and neither can\nthey fully characterize the underlying structural information in long text\ndocuments. In this paper, we are especially interested in relevance matching\nbetween a piece of short text and a long document, which is critical to\nproblems like query-document matching in information retrieval and web\nsearching. To extract the structural information of documents, an undirected\ngraph is constructed, with each vertex representing a keyword and the weight of\nan edge indicating the degree of interaction between keywords. Based on the\nkeyword graph, we further propose a Multiresolution Graph Attention Network to\nlearn multi-layered representations of vertices through a Graph Convolutional\nNetwork (GCN), and then match the short text snippet with the graphical\nrepresentation of the document with the attention mechanisms applied over each\nlayer of the GCN. Experimental results on two datasets demonstrate that our\ngraph approach outperforms other state-of-the-art deep matching models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10584v1",
            "date": "2019-02-27 15:26:45+00:00",
            "title": "When a Tweet is Actually Sexist. A more Comprehensive Classification of Different Online Harassment Categories and The Challenges in NLP",
            "authors": [
                "Sima Sharifirad",
                "Stan Matwin"
            ],
            "abstract": "Sexism is very common in social media and makes the boundaries of freedom\ntighter for feminist and female users. There is still no comprehensive\nclassification of sexism attracting natural language processing techniques.\nCategorizing sexism in social media in the categories of hostile or benevolent\nsexism are so general that simply ignores the other types of sexism happening\nin these media. This paper proposes a more comprehensive and in-depth\ncategories of online harassment in social media e.g. twitter into the following\ncategories, \"Indirect harassment\", \"Information threat\", \"sexual harassment\",\n\"Physical harassment\" and \"Not sexist\" and address the challenge of labeling\nthem along with presenting the classification result of the categories. It is\npreliminary work applying machine learning to learn the concept of sexism and\ndistinguishes itself by looking at more precise categories of sexism in social\nmedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10618v2",
            "date": "2019-02-27 16:16:37+00:00",
            "title": "Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition",
            "authors": [
                "Vered Shwartz",
                "Ido Dagan"
            ],
            "abstract": "Building meaningful phrase representations is challenging because phrase\nmeanings are not simply the sum of their constituent meanings. Lexical\ncomposition can shift the meanings of the constituent words and introduce\nimplicit information. We tested a broad range of textual representations for\ntheir capacity to address these issues. We found that as expected,\ncontextualized word representations perform better than static word embeddings,\nmore so on detecting meaning shift than in recovering implicit information, in\nwhich their performance is still far from that of humans. Our evaluation suite,\nincluding 5 tasks related to lexical composition effects, can serve future\nresearch aiming to improve such representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10623v2",
            "date": "2019-02-27 16:33:32+00:00",
            "title": "Zoho at SemEval-2019 Task 9: Semi-supervised Domain Adaptation using Tri-training for Suggestion Mining",
            "authors": [
                "Sai Prasanna",
                "Sri Ananda Seelan"
            ],
            "abstract": "This paper describes our submission for the SemEval-2019 Suggestion Mining\ntask. A simple Convolutional Neural Network (CNN) classifier with contextual\nword representations from a pre-trained language model was used for sentence\nclassification. The model is trained using tri-training, a semi-supervised\nbootstrapping mechanism for labelling unseen data. Tri-training proved to be an\neffective technique to accommodate domain shift for cross-domain suggestion\nmining (Subtask B) where there is no hand labelled training data. For in-domain\nevaluation (Subtask A), we use the same technique to augment the training set.\nOur system ranks thirteenth in Subtask A with an $F_1$-score of 68.07 and third\nin Subtask B with an $F_1$-score of 81.94.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10649v1",
            "date": "2019-02-27 17:32:15+00:00",
            "title": "F10-SGD: Fast Training of Elastic-net Linear Models for Text Classification and Named-entity Recognition",
            "authors": [
                "Stanislav Peshterliev",
                "Alexander Hsieh",
                "Imre Kiss"
            ],
            "abstract": "Voice-assistants text classification and named-entity recognition (NER)\nmodels are trained on millions of example utterances. Because of the large\ndatasets, long training time is one of the bottlenecks for releasing improved\nmodels. In this work, we develop F10-SGD, a fast optimizer for text\nclassification and NER elastic-net linear models. On internal datasets, F10-SGD\nprovides 4x reduction in training time compared to the OWL-QN optimizer without\nloss of accuracy or increase in model size. Furthermore, we incorporate biased\nsampling that prioritizes harder examples towards the end of the training. As a\nresult, in addition to faster training, we were able to obtain statistically\nsignificant accuracy improvements for NER.\n  On public datasets, F10-SGD obtains 22% faster training time compared to\nFastText for text classification. And, 4x reduction in training time compared\nto CRFSuite OWL-QN for NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10909v1",
            "date": "2019-02-28 05:54:16+00:00",
            "title": "BERT for Joint Intent Classification and Slot Filling",
            "authors": [
                "Qian Chen",
                "Zhu Zhuo",
                "Wen Wang"
            ],
            "abstract": "Intent classification and slot filling are two essential tasks for natural\nlanguage understanding. They often suffer from small-scale human-labeled\ntraining data, resulting in poor generalization capability, especially for rare\nwords. Recently a new language representation model, BERT (Bidirectional\nEncoder Representations from Transformers), facilitates pre-training deep\nbidirectional representations on large-scale unlabeled corpora, and has created\nstate-of-the-art models for a wide variety of natural language processing tasks\nafter simple fine-tuning. However, there has not been much effort on exploring\nBERT for natural language understanding. In this work, we propose a joint\nintent classification and slot filling model based on BERT. Experimental\nresults demonstrate that our proposed model achieves significant improvement on\nintent classification accuracy, slot filling F1, and sentence-level semantic\nframe accuracy on several public benchmark datasets, compared to the\nattention-based recurrent neural network models and slot-gated models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10985v3",
            "date": "2019-02-28 10:06:20+00:00",
            "title": "Better, Faster, Stronger Sequence Tagging Constituent Parsers",
            "authors": [
                "David Vilares",
                "Mostafa Abdou",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Sequence tagging models for constituent parsing are faster, but less accurate\nthan other types of parsers. In this work, we address the following weaknesses\nof such constituent parsers: (a) high error rates around closing brackets of\nlong constituents, (b) large label sets, leading to sparsity, and (c) error\npropagation arising from greedy decoding. To effectively close brackets, we\ntrain a model that learns to switch between tagging schemes. To reduce\nsparsity, we decompose the label set and use multi-task learning to jointly\nlearn to predict sublabels. Finally, we mitigate issues from greedy decoding\nthrough auxiliary losses and sentence-level fine-tuning with policy gradient.\nCombining these techniques, we clearly surpass the performance of sequence\ntagging constituent parsers on the English and Chinese Penn Treebanks, and\nreduce their parsing time even further. On the SPMRL datasets, we observe even\ngreater improvements across the board, including a new state of the art on\nBasque, Hebrew, Polish and Swedish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11049v2",
            "date": "2019-02-28 12:33:17+00:00",
            "title": "Evaluating Rewards for Question Generation Models",
            "authors": [
                "Tom Hosking",
                "Sebastian Riedel"
            ],
            "abstract": "Recent approaches to question generation have used modifications to a Seq2Seq\narchitecture inspired by advances in machine translation. Models are trained\nusing teacher forcing to optimise only the one-step-ahead prediction. However,\nat test time, the model is asked to generate a whole sequence, causing errors\nto propagate through the generation process (exposure bias). A number of\nauthors have proposed countering this bias by optimising for a reward that is\nless tightly coupled to the training data, using reinforcement learning. We\noptimise directly for quality metrics, including a novel approach using a\ndiscriminator learned directly from the training data. We confirm that policy\ngradient methods can be used to decouple training from the ground truth,\nleading to increases in the metrics used as rewards. We perform a human\nevaluation, and show that although these metrics have previously been assumed\nto be good proxies for question quality, they are poorly aligned with human\njudgement and the model simply learns to exploit the weaknesses of the reward\nsource.",
            "journal ref": "Proceedings of the 2019 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, Volume 1 (Long and Short Papers):2278-2283",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11060v1",
            "date": "2019-02-28 12:55:31+00:00",
            "title": "Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions",
            "authors": [
                "Daniel Ortega",
                "Chia-Yu Li",
                "Gisela Vallejo",
                "Pavel Denisov",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper presents our latest investigations on dialog act (DA)\nclassification on automatically generated transcriptions. We propose a novel\napproach that combines convolutional neural networks (CNNs) and conditional\nrandom fields (CRFs) for context modeling in DA classification. We explore the\nimpact of transcriptions generated from different automatic speech recognition\nsystems such as hybrid TDNN/HMM and End-to-End systems on the final\nperformance. Experimental results on two benchmark datasets (MRDA and SwDA)\nshow that the combination CNN and CRF improves consistently the accuracy.\nFurthermore, they show that although the word error rates are comparable,\nEnd-to-End ASR system seems to be more suitable for DA classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11145v2",
            "date": "2019-02-28 15:20:41+00:00",
            "title": "Adversarial Training for Satire Detection: Controlling for Confounding Variables",
            "authors": [
                "Robert McHardy",
                "Heike Adel",
                "Roman Klinger"
            ],
            "abstract": "The automatic detection of satire vs. regular news is relevant for downstream\napplications (for instance, knowledge base population) and to improve the\nunderstanding of linguistic characteristics of satire. Recent approaches build\nupon corpora which have been labeled automatically based on article sources. We\nhypothesize that this encourages the models to learn characteristics for\ndifferent publication sources (e.g., \"The Onion\" vs. \"The Guardian\") rather\nthan characteristics of satire, leading to poor generalization performance to\nunseen publication sources. We therefore propose a novel model for satire\ndetection with an adversarial component to control for the confounding variable\nof publication source. On a large novel data set collected from German news\n(which we make available to the research community), we observe comparable\nsatire classification performance and, as desired, a considerable drop in\npublication classification performance with adversarial training. Our analysis\nshows that the adversarial component is crucial for the model to learn to pay\nattention to linguistic properties of satire.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11291v2",
            "date": "2019-02-28 18:49:10+00:00",
            "title": "FastFusionNet: New State-of-the-Art for DAWNBench SQuAD",
            "authors": [
                "Felix Wu",
                "Boyi Li",
                "Lequn Wang",
                "Ni Lao",
                "John Blitzer",
                "Kilian Q. Weinberger"
            ],
            "abstract": "In this technical report, we introduce FastFusionNet, an efficient variant of\nFusionNet [12]. FusionNet is a high performing reading comprehension\narchitecture, which was designed primarily for maximum retrieval accuracy with\nless regard towards computational requirements. For FastFusionNets we remove\nthe expensive CoVe layers [21] and substitute the BiLSTMs with far more\nefficient SRU layers [19]. The resulting architecture obtains state-of-the-art\nresults on DAWNBench [5] while achieving the lowest training and inference time\non SQuAD [25] to-date. The code is available at\nhttps://github.com/felixgwu/FastFusionNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00297v1",
            "date": "2019-01-02 08:47:38+00:00",
            "title": "A Deep Learning Approach for Similar Languages, Varieties and Dialects",
            "authors": [
                "Vidya Prasad K",
                "Akarsh S",
                "Vinayakumar R",
                "Soman KP"
            ],
            "abstract": "Deep learning mechanisms are prevailing approaches in recent days for the\nvarious tasks in natural language processing, speech recognition, image\nprocessing and many others. To leverage this we use deep learning based\nmechanism specifically Bidirectional- Long Short-Term Memory (B-LSTM) for the\ntask of dialectic identification in Arabic and German broadcast speech and Long\nShort-Term Memory (LSTM) for discriminating between similar Languages. Two\nunique B-LSTM models are created using the Large-vocabulary Continuous Speech\nRecognition (LVCSR) based lexical features and a fixed length of 400 per\nutterance bottleneck features generated by i-vector framework. These models\nwere evaluated on the VarDial 2017 datasets for the tasks Arabic, German\ndialect identification with dialects of Egyptian, Gulf, Levantine, North\nAfrican, and MSA for Arabic and Basel, Bern, Lucerne, and Zurich for German.\nAlso for the task of Discriminating between Similar Languages like Bosnian,\nCroatian and Serbian. The B-LSTM model showed accuracy of 0.246 on lexical\nfeatures and accuracy of 0.577 bottleneck features of i-Vector framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01183v2",
            "date": "2019-01-04 15:52:49+00:00",
            "title": "Aspect Category Detection via Topic-Attention Network",
            "authors": [
                "Sajad Movahedi",
                "Erfan Ghadery",
                "Heshaam Faili",
                "Azadeh Shakery"
            ],
            "abstract": "The e-commerce has started a new trend in natural language processing through\nsentiment analysis of user-generated reviews. Different consumers have\ndifferent concerns about various aspects of a specific product or service.\nAspect category detection, as a subtask of aspect-based sentiment analysis,\ntackles the problem of categorizing a given review sentence into a set of\npre-defined aspect categories. In recent years, deep learning approaches have\nbrought revolutionary advances in multiple branches of natural language\nprocessing including sentiment analysis. In this paper, we propose a deep\nneural network method based on attention mechanism to identify different aspect\ncategories of a given review sentence. Our model utilizes several attentions\nwith different topic contexts, enabling it to attend to different parts of a\nreview sentence based on different topics. Experimental results on two datasets\nin the restaurant domain released by SemEval workshop demonstrates that our\napproach outperforms existing methods on both datasets. Visualization of the\ntopic attention weights shows the effectiveness of our model in identifying\nwords related to different topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01239v1",
            "date": "2019-01-04 18:38:11+00:00",
            "title": "Speaker Adaptation for End-to-End CTC Models",
            "authors": [
                "Ke Li",
                "Jinyu Li",
                "Yong Zhao",
                "Kshitiz Kumar",
                "Yifan Gong"
            ],
            "abstract": "We propose two approaches for speaker adaptation in end-to-end (E2E)\nautomatic speech recognition systems. One is Kullback-Leibler divergence (KLD)\nregularization and the other is multi-task learning (MTL). Both approaches aim\nto address the data sparsity especially output target sparsity issue of speaker\nadaptation in E2E systems. The KLD regularization adapts a model by forcing the\noutput distribution from the adapted model to be close to the unadapted one.\nThe MTL utilizes a jointly trained auxiliary task to improve the performance of\nthe main task. We investigated our approaches on E2E connectionist temporal\nclassification (CTC) models with three different types of output units.\nExperiments on the Microsoft short message dictation task demonstrated that MTL\noutperforms KLD regularization. In particular, the MTL adaptation obtained\n8.8\\% and 4.0\\% relative word error rate reductions (WERRs) for supervised and\nunsupervised adaptations for the word CTC model, and 9.6% and 3.8% relative\nWERRs for the mix-unit CTC model, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01466v1",
            "date": "2019-01-05 20:27:19+00:00",
            "title": "Addressing Objects and Their Relations: The Conversational Entity Dialogue Model",
            "authors": [
                "Stefan Ultes",
                "Pawe\u0142\\ Budzianowski",
                "I\u00f1igo Casanueva",
                "Lina Rojas-Barahona",
                "Bo-Hsiang Tseng",
                "Yen-Chen Wu",
                "Steve Young",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Statistical spoken dialogue systems usually rely on a single- or multi-domain\ndialogue model that is restricted in its capabilities of modelling complex\ndialogue structures, e.g., relations. In this work, we propose a novel dialogue\nmodel that is centred around entities and is able to model relations as well as\nmultiple entities of the same type. We demonstrate in a prototype\nimplementation benefits of relation modelling on the dialogue level and show\nthat a trained policy using these relations outperforms the multi-domain\nbaseline. Furthermore, we show that by modelling the relations on the dialogue\nlevel, the system is capable of processing relations present in the user input\nand even learns to address them in the system response.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01574v1",
            "date": "2019-01-06 17:20:24+00:00",
            "title": "A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing",
            "authors": [
                "Yunsu Kim",
                "Andreas Guta",
                "Joern Wuebker",
                "Hermann Ney"
            ],
            "abstract": "This work systematically analyzes the smoothing effect of vocabulary\nreduction for phrase translation models. We extensively compare various\nword-level vocabularies to show that the performance of smoothing is not\nsignificantly affected by the choice of vocabulary. This result provides\nempirical evidence that the standard phrase translation model is extremely\nsparse. Our experiments also reveal that vocabulary reduction is more effective\nfor smoothing large-scale phrase tables.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01577v1",
            "date": "2019-01-06 17:29:24+00:00",
            "title": "Unsupervised Training for Large Vocabulary Translation Using Sparse Lexicon and Word Classes",
            "authors": [
                "Yunsu Kim",
                "Julian Schamper",
                "Hermann Ney"
            ],
            "abstract": "We address for the first time unsupervised training for a translation task\nwith hundreds of thousands of vocabulary words. We scale up the\nexpectation-maximization (EM) algorithm to learn a large translation table\nwithout any parallel text or seed lexicon. First, we solve the memory\nbottleneck and enforce the sparsity with a simple thresholding scheme for the\nlexicon. Second, we initialize the lexicon training with word classes, which\nefficiently boosts the performance. Our methods produced promising results on\ntwo large-scale unsupervised translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01642v1",
            "date": "2019-01-07 01:57:21+00:00",
            "title": "Text Mining Customer Reviews For Aspect-based Restaurant Rating",
            "authors": [
                "Jovelyn C. Cuizon",
                "Jesserine Lopez",
                "Danica Rose Jones"
            ],
            "abstract": "This study applies text mining to analyze customer reviews and automatically\nassign a collective restaurant star rating based on five predetermined aspects:\nambiance, cost, food, hygiene, and service. The application provides a web and\nmobile crowd sourcing platform where users share dining experiences and get\ninsights about the strengths and weaknesses of a restaurant through user\ncontributed feedback. Text reviews are tokenized into sentences. Noun-adjective\npairs are extracted from each sentence using Stanford Core NLP library and are\nassociated to aspects based on the bag of associated words fed into the system.\nThe sentiment weight of the adjectives is determined through AFINN library. An\noverall restaurant star rating is computed based on the individual aspect\nrating. Further, a word cloud is generated to provide visual display of the\nmost frequently occurring terms in the reviews. The more feedbacks are added\nthe more reflective the sentiment score to the restaurants' performance.",
            "journal ref": "International Journal of Computer Science & Information Technology\n  (IJCSIT) Vol 10, No 6, December 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01824v2",
            "date": "2019-01-07 14:17:29+00:00",
            "title": "Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots",
            "authors": [
                "Jia-Chen Gu",
                "Zhen-Hua Ling",
                "Quan Liu"
            ],
            "abstract": "In this paper, we propose an interactive matching network (IMN) for the\nmulti-turn response selection task. First, IMN constructs word representations\nfrom three aspects to address the challenge of out-of-vocabulary (OOV) words.\nSecond, an attentive hierarchical recurrent encoder (AHRE), which is capable of\nencoding sentences hierarchically and generating more descriptive\nrepresentations by aggregating with an attention mechanism, is designed.\nFinally, the bidirectional interactions between whole multi-turn contexts and\nresponse candidates are calculated to derive the matching information between\nthem. Experiments on four public datasets show that IMN outperforms the\nbaseline models on all metrics, achieving a new state-of-the-art performance\nand demonstrating compatibility across domains for multi-turn response\nselection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01911v1",
            "date": "2019-01-07 16:38:16+00:00",
            "title": "Stance Classification for Rumour Analysis in Twitter: Exploiting Affective Information and Conversation Structure",
            "authors": [
                "Endang Wahyu Pamungkas",
                "Valerio Basile",
                "Viviana Patti"
            ],
            "abstract": "Analysing how people react to rumours associated with news in social media is\nan important task to prevent the spreading of misinformation, which is nowadays\nwidely recognized as a dangerous tendency. In social media conversations, users\nshow different stances and attitudes towards rumourous stories. Some users take\na definite stance, supporting or denying the rumour at issue, while others just\ncomment it, or ask for additional evidence related to the veracity of the\nrumour. On this line, a new shared task has been proposed at SemEval-2017 (Task\n8, SubTask A), which is focused on rumour stance classification in English\ntweets. The goal is predicting user stance towards emerging rumours in Twitter,\nin terms of supporting, denying, querying, or commenting the original rumour,\nlooking at the conversation threads originated by the rumour. This paper\ndescribes a new approach to this task, where the use of conversation-based and\naffective-based features, covering different facets of affect, has been\nexplored. Our classification model outperforms the best-performing systems for\nstance classification at SemEval-2017 Task 8, showing the effectiveness of the\nfeature set proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02222v1",
            "date": "2019-01-08 09:48:41+00:00",
            "title": "Multi-turn Inference Matching Network for Natural Language Inference",
            "authors": [
                "Chunhua Liu",
                "Shan Jiang",
                "Hainan Yu",
                "Dong Yu"
            ],
            "abstract": "Natural Language Inference (NLI) is a fundamental and challenging task in\nNatural Language Processing (NLP). Most existing methods only apply one-pass\ninference process on a mixed matching feature, which is a concatenation of\ndifferent matching features between a premise and a hypothesis. In this paper,\nwe propose a new model called Multi-turn Inference Matching Network (MIMN) to\nperform multi-turn inference on different matching features. In each turn, the\nmodel focuses on one particular matching feature instead of the mixed matching\nfeature. To enhance the interaction between different matching features, a\nmemory component is employed to store the history inference information. The\ninference of each turn is performed on the current matching feature and the\nmemory. We conduct experiments on three different NLI datasets. The\nexperimental results show that our model outperforms or achieves the\nstate-of-the-art performance on all the three datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02252v1",
            "date": "2019-01-08 11:06:42+00:00",
            "title": "DEMN: Distilled-Exposition Enhanced Matching Network for Story Comprehension",
            "authors": [
                "Chunhua Liu",
                "Haiou Zhang",
                "Shan Jiang",
                "Dong Yu"
            ],
            "abstract": "This paper proposes a Distilled-Exposition Enhanced Matching Network (DEMN)\nfor story-cloze test, which is still a challenging task in story comprehension.\nWe divide a complete story into three narrative segments: an\n\\textit{exposition}, a \\textit{climax}, and an \\textit{ending}. The model\nconsists of three modules: input module, matching module, and distillation\nmodule. The input module provides semantic representations for the three\nsegments and then feeds them into the other two modules. The matching module\ncollects interaction features between the ending and the climax. The\ndistillation module distills the crucial semantic information in the exposition\nand infuses it into the matching module in two different ways. We evaluate our\nsingle and ensemble model on ROCStories Corpus \\cite{Mostafazadeh2016ACA},\nachieving an accuracy of 80.1\\% and 81.2\\% on the test set respectively. The\nexperimental results demonstrate that our DEMN model achieves a\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02257v1",
            "date": "2019-01-08 11:15:07+00:00",
            "title": "Multi-Perspective Fusion Network for Commonsense Reading Comprehension",
            "authors": [
                "Chunhua Liu",
                "Yan Zhao",
                "Qingyi Si",
                "Haiou Zhang",
                "Bohan Li",
                "Dong Yu"
            ],
            "abstract": "Commonsense Reading Comprehension (CRC) is a significantly challenging task,\naiming at choosing the right answer for the question referring to a narrative\npassage, which may require commonsense knowledge inference. Most of the\nexisting approaches only fuse the interaction information of choice, passage,\nand question in a simple combination manner from a \\emph{union} perspective,\nwhich lacks the comparison information on a deeper level. Instead, we propose a\nMulti-Perspective Fusion Network (MPFN), extending the single fusion method\nwith multiple perspectives by introducing the \\emph{difference} and\n\\emph{similarity} fusion\\deleted{along with the \\emph{union}}. More\ncomprehensive and accurate information can be captured through the three types\nof fusion. We design several groups of experiments on MCScript dataset\n\\cite{Ostermann:LREC18:MCScript} to evaluate the effectiveness of the three\ntypes of fusion respectively. From the experimental results, we can conclude\nthat the difference fusion is comparable with union fusion, and the similarity\nfusion needs to be activated by the union fusion. The experimental result also\nshows that our MPFN model achieves the state-of-the-art with an accuracy of\n83.52\\% on the official test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02262v2",
            "date": "2019-01-08 11:27:58+00:00",
            "title": "Multi-style Generative Reading Comprehension",
            "authors": [
                "Kyosuke Nishida",
                "Itsumi Saito",
                "Kosuke Nishida",
                "Kazutoshi Shinoda",
                "Atsushi Otsuka",
                "Hisako Asano",
                "Junji Tomita"
            ],
            "abstract": "This study tackles generative reading comprehension (RC), which consists of\nanswering questions based on textual evidence and natural language generation\n(NLG). We propose a multi-style abstractive summarization model for question\nanswering, called Masque. The proposed model has two key characteristics.\nFirst, unlike most studies on RC that have focused on extracting an answer span\nfrom the provided passages, our model instead focuses on generating a summary\nfrom the question and multiple passages. This serves to cover various answer\nstyles required for real-world applications. Second, whereas previous studies\nbuilt a specific model for each answer style because of the difficulty of\nacquiring one general model, our approach learns multi-style answers within a\nmodel to improve the NLG capability for all styles involved. This also enables\nour model to give an answer in the target style. Experiments show that our\nmodel achieves state-of-the-art performance on the Q&A task and the Q&A + NLG\ntask of MS MARCO 2.1 and the summary task of NarrativeQA. We observe that the\ntransfer of the style-independent NLG capability to the target style is the key\nto its success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02490v1",
            "date": "2019-01-08 19:59:33+00:00",
            "title": "Choosing the Right Word: Using Bidirectional LSTM Tagger for Writing Support Systems",
            "authors": [
                "Victor Makarenkov",
                "Lior Rokach",
                "Bracha Shapira"
            ],
            "abstract": "Scientific writing is difficult. It is even harder for those for whom English\nis a second language (ESL learners). Scholars around the world spend a\nsignificant amount of time and resources proofreading their work before\nsubmitting it for review or publication.\n  In this paper we present a novel machine learning based application for\nproper word choice task. Proper word choice is a generalization the lexical\nsubstitution (LS) and grammatical error correction (GEC) tasks. We demonstrate\nand evaluate the usefulness of applying bidirectional Long Short Term Memory\n(LSTM) tagger, for this task. While state-of-the-art grammatical error\ncorrection uses error-specific classifiers and machine translation methods, we\ndemonstrate an unsupervised method that is based solely on a high quality text\ncorpus and does not require manually annotated data. We use a bidirectional\nRecurrent Neural Network (RNN) with LSTM for learning the proper word choice\nbased on a word's sentential context. We demonstrate and evaluate our\napplication on both a domain-specific (scientific), writing task and a\ngeneral-purpose writing task. We show that our domain-specific and\ngeneral-purpose models outperform state-of-the-art general context learning. As\nan additional contribution of this research, we also share our code,\npre-trained models, and a new ESL learner test set with the research community.",
            "journal ref": "Elsevier Engineering Applications of Artificial Intelligence,\n  Volume 84, September 2019, Pages 1-10",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02534v1",
            "date": "2019-01-08 21:57:30+00:00",
            "title": "Team Papelo: Transformer Networks at FEVER",
            "authors": [
                "Christopher Malon"
            ],
            "abstract": "We develop a system for the FEVER fact extraction and verification challenge\nthat uses a high precision entailment classifier based on transformer networks\npretrained with language modeling, to classify a broad set of potential\nevidence. The precision of the entailment classifier allows us to enhance\nrecall by considering every statement from several articles to decide upon each\nclaim. We include not only the articles best matching the claim text by TFIDF\nscore, but read additional articles whose titles match named entities and\ncapitalized expressions occurring in the claim text. The entailment module\nevaluates potential evidence one statement at a time, together with the title\nof the page the evidence came from (providing a hint about possible pronoun\nantecedents). In preliminary evaluation, the system achieves .5736 FEVER score,\n.6108 label accuracy, and .6485 evidence F1 on the FEVER shared task test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02609v3",
            "date": "2019-01-09 05:45:41+00:00",
            "title": "Sequential Attention-based Network for Noetic End-to-End Response Selection",
            "authors": [
                "Qian Chen",
                "Wen Wang"
            ],
            "abstract": "The noetic end-to-end response selection challenge as one track in Dialog\nSystem Technology Challenges 7 (DSTC7) aims to push the state of the art of\nutterance classification for real world goal-oriented dialog systems, for which\nparticipants need to select the correct next utterances from a set of\ncandidates for the multi-turn context. This paper describes our systems that\nare ranked the top on both datasets under this challenge, one focused and small\n(Advising) and the other more diverse and large (Ubuntu). Previous\nstate-of-the-art models use hierarchy-based (utterance-level and token-level)\nneural networks to explicitly model the interactions among different turns'\nutterances for context modeling. In this paper, we investigate a sequential\nmatching model based only on chain sequence for multi-turn response selection.\nOur results demonstrate that the potentials of sequential matching approaches\nhave not yet been fully exploited in the past for multi-turn response\nselection. In addition to ranking the top in the challenge, the proposed model\noutperforms all previous models, including state-of-the-art hierarchy-based\nmodels, and achieves new state-of-the-art performances on two large-scale\npublic multi-turn response selection benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02646v1",
            "date": "2019-01-09 09:19:28+00:00",
            "title": "What do Language Representations Really Represent?",
            "authors": [
                "Johannes Bjerva",
                "Robert \u00d6stling",
                "Maria Han Veiga",
                "J\u00f6rg Tiedemann",
                "Isabelle Augenstein"
            ],
            "abstract": "A neural language model trained on a text corpus can be used to induce\ndistributed representations of words, such that similar words end up with\nsimilar representations. If the corpus is multilingual, the same model can be\nused to learn distributed representations of languages, such that similar\nlanguages end up with similar representations. We show that this holds even\nwhen the multilingual corpus has been translated into English, by picking up\nthe faint signal left by the source languages. However, just like it is a\nthorny problem to separate semantic from syntactic similarity in word\nrepresentations, it is not obvious what type of similarity is captured by\nlanguage representations. We investigate correlations and causal relationships\nbetween language representations learned from translations on one hand, and\ngenetic, geographical, and several levels of structural similarity between\nlanguages on the other. Of these, structural similarity is found to correlate\nmost strongly with language representation similarity, while genetic\nrelationships---a convenient benchmark used for evaluation in previous\nwork---appears to be a confounding factor. Apart from implications about\ntranslation effects, we see this more generally as a case where NLP and\nlinguistic typology can interact and benefit one another.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02998v1",
            "date": "2019-01-10 02:23:18+00:00",
            "title": "Sentence Rewriting for Semantic Parsing",
            "authors": [
                "Bo Chen",
                "Le Sun",
                "Xianpei Han",
                "Bo An"
            ],
            "abstract": "A major challenge of semantic parsing is the vocabulary mismatch problem\nbetween natural language and target ontology. In this paper, we propose a\nsentence rewriting based semantic parsing method, which can effectively resolve\nthe mismatch problem by rewriting a sentence into a new form which has the same\nstructure with its target logical form. Specifically, we propose two\nsentence-rewriting methods for two common types of mismatch: a dictionary-based\nmethod for 1-N mismatch and a template-based method for N-1 mismatch. We\nevaluate our entence rewriting based semantic parser on the benchmark semantic\nparsing dataset -- WEBQUESTIONS. Experimental results show that our system\noutperforms the base system with a 3.4% gain in F1, and generates logical forms\nmore accurately and parses sentences more robustly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03116v2",
            "date": "2019-01-10 12:06:31+00:00",
            "title": "Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques",
            "authors": [
                "Joel Escud\u00e9 Font",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "Neural machine translation has significantly pushed forward the quality of\nthe field. However, there are remaining big issues with the output translations\nand one of them is fairness. Neural models are trained on large text corpora\nwhich contain biases and stereotypes. As a consequence, models inherit these\nsocial biases. Recent methods have shown results in reducing gender bias in\nother natural language processing tools such as word embeddings. We take\nadvantage of the fact that word embeddings are used in neural machine\ntranslation to propose a method to equalize gender biases in neural machine\ntranslation using these representations. Specifically, we propose, experiment\nand analyze the integration of two debiasing techniques over GloVe embeddings\nin the Transformer translation architecture. We evaluate our proposed system on\nthe WMT English-Spanish benchmark task, showing gains up to one BLEU point. As\nfor the gender bias evaluation, we generate a test set of occupations and we\nshow that our proposed system learns to equalize existing biases from the\nbaseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03438v4",
            "date": "2019-01-11 00:25:10+00:00",
            "title": "Linguistic Analysis of Pretrained Sentence Encoders with Acceptability Judgments",
            "authors": [
                "Alex Warstadt",
                "Samuel R. Bowman"
            ],
            "abstract": "Recent work on evaluating grammatical knowledge in pretrained sentence\nencoders gives a fine-grained view of a small number of phenomena. We introduce\na new analysis dataset that also has broad coverage of linguistic phenomena. We\nannotate the development set of the Corpus of Linguistic Acceptability (CoLA;\nWarstadt et al., 2018) for the presence of 13 classes of syntactic phenomena\nincluding various forms of argument alternations, movement, and modification.\nWe use this analysis set to investigate the grammatical knowledge of three\npretrained encoders: BERT (Devlin et al., 2018), GPT (Radford et al., 2018),\nand the BiLSTM baseline from Warstadt et al. We find that these models have a\nstrong command of complex or non-canonical argument structures like\nditransitives (Sue gave Dan a book) and passives (The book was read). Sentences\nwith long distance dependencies like questions (What do you think I ate?)\nchallenge all models, but for these, BERT and GPT have a distinct advantage\nover the baseline. We conclude that recent sentence encoders, despite showing\nnear-human performance on acceptability classification overall, still fail to\nmake fine-grained grammaticality distinctions for many complex syntactic\nstructures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03459v1",
            "date": "2019-01-11 02:28:38+00:00",
            "title": "From Plots to Endings: A Reinforced Pointer Generator for Story Ending Generation",
            "authors": [
                "Yan Zhao",
                "Lu Liu",
                "Chunhua Liu",
                "Ruoyao Yang",
                "Dong Yu"
            ],
            "abstract": "We introduce a new task named Story Ending Generation (SEG), whic-h aims at\ngenerating a coherent story ending from a sequence of story plot. Wepropose a\nframework consisting of a Generator and a Reward Manager for thistask. The\nGenerator follows the pointer-generator network with coverage mech-anism to\ndeal with out-of-vocabulary (OOV) and repetitive words. Moreover, amixed loss\nmethod is introduced to enable the Generator to produce story endingsof high\nsemantic relevance with story plots. In the Reward Manager, the rewardis\ncomputed to fine-tune the Generator with policy-gradient reinforcement\nlearn-ing (PGRL). We conduct experiments on the recently-introduced\nROCStoriesCorpus. We evaluate our model in both automatic evaluation and human\nevalua-tion. Experimental results show that our model exceeds the\nsequence-to-sequencebaseline model by 15.75% and 13.57% in terms of CIDEr and\nconsistency scorerespectively.",
            "journal ref": "Natural Language Processing and Chinese Computing. NLPCC 2018.\n  Lecture Notes in Computer Science, vol 11108. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03461v1",
            "date": "2019-01-11 02:43:12+00:00",
            "title": "Dialog System Technology Challenge 7",
            "authors": [
                "Koichiro Yoshino",
                "Chiori Hori",
                "Julien Perez",
                "Luis Fernando D'Haro",
                "Lazaros Polymenakos",
                "Chulaka Gunasekara",
                "Walter S. Lasecki",
                "Jonathan K. Kummerfeld",
                "Michel Galley",
                "Chris Brockett",
                "Jianfeng Gao",
                "Bill Dolan",
                "Xiang Gao",
                "Huda Alamari",
                "Tim K. Marks",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "This paper introduces the Seventh Dialog System Technology Challenges (DSTC),\nwhich use shared datasets to explore the problem of building dialog systems.\nRecently, end-to-end dialog modeling approaches have been applied to various\ndialog tasks. The seventh DSTC (DSTC7) focuses on developing technologies\nrelated to end-to-end dialog systems for (1) sentence selection, (2) sentence\ngeneration and (3) audio visual scene aware dialog. This paper summarizes the\noverall setup and results of DSTC7, including detailed descriptions of the\ndifferent tracks and provided datasets. We also describe overall trends in the\nsubmitted systems and the key results. Each track introduced new datasets and\nparticipants achieved impressive results using state-of-the-art end-to-end\ntechnologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03601v1",
            "date": "2019-01-11 14:51:02+00:00",
            "title": "Advanced Rich Transcription System for Estonian Speech",
            "authors": [
                "Tanel Alum\u00e4e",
                "Ottokar Tilk",
                "Asadullah"
            ],
            "abstract": "This paper describes the current TT\\\"U speech transcription system for\nEstonian speech. The system is designed to handle semi-spontaneous speech, such\nas broadcast conversations, lecture recordings and interviews recorded in\ndiverse acoustic conditions. The system is based on the Kaldi toolkit.\nMulti-condition training using background noise profiles extracted\nautomatically from untranscribed data is used to improve the robustness of the\nsystem. Out-of-vocabulary words are recovered using a phoneme n-gram based\ndecoding subgraph and a FST-based phoneme-to-grapheme model. The system\nachieves a word error rate of 8.1% on a test set of broadcast conversations.\nThe system also performs punctuation recovery and speaker identification.\nSpeaker identification models are trained using a recently proposed weakly\nsupervised training method.",
            "journal ref": "Series: Frontiers in Artificial Intelligence and Applications;\n  Ebook Volume 307: Human Language Technologies -- The Baltic Perspective, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03644v1",
            "date": "2019-01-11 16:42:43+00:00",
            "title": "ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation",
            "authors": [
                "J. Edward Hu",
                "Rachel Rudinger",
                "Matt Post",
                "Benjamin Van Durme"
            ],
            "abstract": "We present ParaBank, a large-scale English paraphrase dataset that surpasses\nprior work in both quantity and quality. Following the approach of ParaNMT, we\ntrain a Czech-English neural machine translation (NMT) system to generate novel\nparaphrases of English reference sentences. By adding lexical constraints to\nthe NMT decoding procedure, however, we are able to produce multiple\nhigh-quality sentential paraphrases per source sentence, yielding an English\nparaphrase resource with more than 4 billion generated tokens and exhibiting\ngreater lexical diversity. Using human judgments, we also demonstrate that\nParaBank's paraphrases improve over ParaNMT on both semantic similarity and\nfluency. Finally, we use ParaBank to train a monolingual NMT model with the\nsame support for lexically-constrained decoding for sentence rewriting tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03735v2",
            "date": "2019-01-11 20:27:25+00:00",
            "title": "EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference",
            "authors": [
                "Abhilasha Ravichander",
                "Aakanksha Naik",
                "Carolyn Rose",
                "Eduard Hovy"
            ],
            "abstract": "Quantitative reasoning is a higher-order reasoning skill that any intelligent\nnatural language understanding system can reasonably be expected to handle. We\npresent EQUATE (Evaluating Quantitative Understanding Aptitude in Textual\nEntailment), a new framework for quantitative reasoning in textual entailment.\nWe benchmark the performance of 9 published NLI models on EQUATE, and find that\non average, state-of-the-art methods do not achieve an absolute improvement\nover a majority-class baseline, suggesting that they do not implicitly learn to\nreason with quantities. We establish a new baseline Q-REAS that manipulates\nquantities symbolically. In comparison to the best performing NLI model, it\nachieves success on numerical reasoning tests (+24.2%), but has limited verbal\nreasoning capabilities (-8.1%). We hope our evaluation framework will support\nthe development of models of quantitative reasoning in language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03788v1",
            "date": "2019-01-12 02:50:40+00:00",
            "title": "Semi-interactive Attention Network for Answer Understanding in Reverse-QA",
            "authors": [
                "Qing Yin",
                "Guan Luo",
                "Xiaodong Zhu",
                "Qinghua Hu",
                "Ou Wu"
            ],
            "abstract": "Question answering (QA) is an important natural language processing (NLP)\ntask and has received much attention in academic research and industry\ncommunities. Existing QA studies assume that questions are raised by humans and\nanswers are generated by machines. Nevertheless, in many real applications,\nmachines are also required to determine human needs or perceive human states.\nIn such scenarios, machines may proactively raise questions and humans supply\nanswers. Subsequently, machines should attempt to understand the true meaning\nof these answers. This new QA approach is called reverse-QA (rQA) throughout\nthis paper. In this work, the human answer understanding problem is\ninvestigated and solved by classifying the answers into predefined answer-label\ncategories (e.g., True, False, Uncertain). To explore the relationships between\nquestions and answers, we use the interactive attention network (IAN) model and\npropose an improved structure called semi-interactive attention network\n(Semi-IAN). Two Chinese data sets for rQA are compiled. We evaluate several\nconventional text classification models for comparison, and experimental\nresults indicate the promising performance of our proposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03859v1",
            "date": "2019-01-12 13:00:30+00:00",
            "title": "What comes next? Extractive summarization by next-sentence prediction",
            "authors": [
                "Jingyun Liu",
                "Jackie C. K. Cheung",
                "Annie Louis"
            ],
            "abstract": "Existing approaches to automatic summarization assume that a length limit for\nthe summary is given, and view content selection as an optimization problem to\nmaximize informativeness and minimize redundancy within this budget. This\nframework ignores the fact that human-written summaries have rich internal\nstructure which can be exploited to train a summarization system. We present\nNEXTSUM, a novel approach to summarization based on a model that predicts the\nnext sentence to include in the summary using not only the source article, but\nalso the summary produced so far. We show that such a model successfully\ncaptures summary-specific discourse moves, and leads to better content\nselection performance, in addition to automatically predicting how long the\ntarget summary should be. We perform experiments on the New York Times\nAnnotated Corpus of summaries, where NEXTSUM outperforms lead and content-model\nsummarization baselines by significant margins. We also show that the lengths\nof summaries produced by our system correlates with the lengths of the\nhuman-written gold standards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03866v1",
            "date": "2019-01-12 14:29:35+00:00",
            "title": "HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering",
            "authors": [
                "Liang Pang",
                "Yanyan Lan",
                "Jiafeng Guo",
                "Jun Xu",
                "Lixin Su",
                "Xueqi Cheng"
            ],
            "abstract": "This paper is concerned with open-domain question answering (i.e., OpenQA).\nRecently, some works have viewed this problem as a reading comprehension (RC)\ntask, and directly applied successful RC models to it. However, the\nperformances of such models are not so good as that in the RC task. In our\nopinion, the perspective of RC ignores three characteristics in OpenQA task: 1)\nmany paragraphs without the answer span are included in the data collection; 2)\nmultiple answer spans may exist within one given paragraph; 3) the end position\nof an answer span is dependent with the start position. In this paper, we first\npropose a new probabilistic formulation of OpenQA, based on a three-level\nhierarchical structure, i.e.,~the question level, the paragraph level and the\nanswer span level. Then a Hierarchical Answer Spans Model (HAS-QA) is designed\nto capture each probability. HAS-QA has the ability to tackle the above three\nproblems, and experiments on public OpenQA datasets show that it significantly\noutperforms traditional RC baselines and recent OpenQA baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04112v1",
            "date": "2019-01-14 03:34:27+00:00",
            "title": "Unsupervised Neural Machine Translation with SMT as Posterior Regularization",
            "authors": [
                "Shuo Ren",
                "Zhirui Zhang",
                "Shujie Liu",
                "Ming Zhou",
                "Shuai Ma"
            ],
            "abstract": "Without real bilingual corpus available, unsupervised Neural Machine\nTranslation (NMT) typically requires pseudo parallel data generated with the\nback-translation method for the model training. However, due to weak\nsupervision, the pseudo data inevitably contain noises and errors that will be\naccumulated and reinforced in the subsequent training process, leading to bad\ntranslation performance. To address this issue, we introduce phrase based\nStatistic Machine Translation (SMT) models which are robust to noisy data, as\nposterior regularizations to guide the training of unsupervised NMT models in\nthe iterative back-translation process. Our method starts from SMT models built\nwith pre-trained language models and word-level translation tables inferred\nfrom cross-lingual embeddings. Then SMT and NMT models are optimized jointly\nand boost each other incrementally in a unified EM framework. In this way, (1)\nthe negative effect caused by errors in the iterative back-translation process\ncan be alleviated timely by SMT filtering noises from its phrase tables;\nmeanwhile, (2) NMT can compensate for the deficiency of fluency inherent in\nSMT. Experiments conducted on en-fr and en-de translation tasks show that our\nmethod outperforms the strong baseline and achieves new state-of-the-art\nunsupervised machine translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04379v2",
            "date": "2019-01-14 16:23:35+00:00",
            "title": "Towards Using Context-Dependent Symbols in CTC Without State-Tying Decision Trees",
            "authors": [
                "Jan Chorowski",
                "Adrian Lancucki",
                "Bartosz Kostka",
                "Michal Zapotoczny"
            ],
            "abstract": "Deep neural acoustic models benefit from context-dependent (CD) modeling of\noutput symbols. We consider direct training of CTC networks with CD outputs,\nand identify two issues. The first one is frame-level normalization of\nprobabilities in CTC, which induces strong language modeling behavior that\nleads to overfitting and interference with external language models. The second\none is poor generalization in the presence of numerous lexical units like\ntriphones or tri-chars. We mitigate the former with utterance-level\nnormalization of probabilities. The latter typically requires reducing the CD\nsymbol inventory with state-tying decision trees, which have to be transferred\nfrom classical GMM-HMM systems. We replace the trees with a CD symbol embedding\nnetwork, which saves parameters and ensures generalization to unseen and\nundersampled CD symbols. The embedding network is trained together with the\nrest of the acoustic model and removes one of the last cases in which neural\nsystems have to be bootstrapped from GMM-HMM ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04587v2",
            "date": "2019-01-14 22:19:35+00:00",
            "title": "Human few-shot learning of compositional instructions",
            "authors": [
                "Brenden M. Lake",
                "Tal Linzen",
                "Marco Baroni"
            ],
            "abstract": "People learn in fast and flexible ways that have not been emulated by\nmachines. Once a person learns a new verb \"dax,\" he or she can effortlessly\nunderstand how to \"dax twice,\" \"walk and dax,\" or \"dax vigorously.\" There have\nbeen striking recent improvements in machine learning for natural language\nprocessing, yet the best algorithms require vast amounts of experience and\nstruggle to generalize new concepts in compositional ways. To better understand\nthese distinctively human abilities, we study the compositional skills of\npeople through language-like instruction learning tasks. Our results show that\npeople can learn and use novel functional concepts from very few examples\n(few-shot learning), successfully applying familiar functions to novel inputs.\nPeople can also compose concepts in complex ways that go beyond the provided\ndemonstrations. Two additional experiments examined the assumptions and\ninductive biases that people make when solving these tasks, revealing three\nbiases: mutual exclusivity, one-to-one mappings, and iconic concatenation. We\ndiscuss the implications for cognitive modeling and the potential for building\nmachines with more human-like language learning capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04787v2",
            "date": "2019-01-15 12:16:13+00:00",
            "title": "A Tweet Dataset Annotated for Named Entity Recognition and Stance Detection",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck",
                "Fazli Can"
            ],
            "abstract": "Annotated datasets in different domains are critical for many supervised\nlearning-based solutions to related problems and for the evaluation of the\nproposed solutions. Topics in natural language processing (NLP) similarly\nrequire annotated datasets to be used for such purposes. In this paper, we\ntarget at two NLP problems, named entity recognition and stance detection, and\npresent the details of a tweet dataset in Turkish annotated for named entity\nand stance information. Within the course of the current study, both the named\nentity and stance annotations of the included tweets are made publicly\navailable, although previously the dataset has been publicly shared with stance\nannotations only. We believe that this dataset will be useful for uncovering\nthe possible relationships between named entity recognition and stance\ndetection in tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05041v1",
            "date": "2019-01-15 20:50:28+00:00",
            "title": "Answering Comparative Questions: Better than Ten-Blue-Links?",
            "authors": [
                "Matthias Schildw\u00e4chter",
                "Alexander Bondarenko",
                "Julian Zenker",
                "Matthias Hagen",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We present CAM (comparative argumentative machine), a novel open-domain IR\nsystem to argumentatively compare objects with respect to information extracted\nfrom the Common Crawl. In a user study, the participants obtained 15% more\naccurate answers using CAM compared to a \"traditional\" keyword-based search and\nwere 20% faster in finding the answer to comparative questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05066v1",
            "date": "2019-01-15 21:56:32+00:00",
            "title": "Investigating Antigram Behaviour using Distributional Semantics",
            "authors": [
                "Saptarshi Sengupta"
            ],
            "abstract": "Language is an extremely interesting subject to study, each day presenting\nnew challenges and new topics for research. Words in particular have several\nunique characteristics which when explored, prove to be astonishing. Anagrams\nand Antigrams are such words possessing these amazing properties. The presented\nwork is an exploration into generating anagrams from a given word and\ndetermining whether there exists antigram relationships between the pairs of\ngenerated anagrams in light of the Word2Vec distributional semantic similarity\nmodel. The experiments conducted, showed promising results for detecting\nantigrams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05180v1",
            "date": "2019-01-16 08:47:47+00:00",
            "title": "Formal models of Structure Building in Music, Language and Animal Songs",
            "authors": [
                "Willem Zuidema",
                "Dieuwke Hupkes",
                "Geraint Wiggins",
                "Constance Scharff",
                "Martin Rohrmeier"
            ],
            "abstract": "Human language, music and a variety of animal vocalisations constitute ways\nof sonic communication that exhibit remarkable structural complexity. While the\ncomplexities of language and possible parallels in animal communication have\nbeen discussed intensively, reflections on the complexity of music and animal\nsong, and their comparisons are underrepresented. In some ways, music and\nanimal songs are more comparable to each other than to language, as\npropositional semantics cannot be used as as indicator of communicative success\nor well-formedness, and notions of grammaticality are less easily defined. This\nreview brings together accounts of the principles of structure building in\nlanguage, music and animal song, relating them to the corresponding models in\nformal language theory, with a special focus on evaluating the benefits of\nusing the Chomsky hierarchy (CH). We further discuss common misunderstandings\nand shortcomings concerning the CH, as well as extensions or augmentations of\nit that address some of these issues, and suggest ways to move beyond.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05219v1",
            "date": "2019-01-16 10:40:18+00:00",
            "title": "Sentence transition matrix: An efficient approach that preserves sentence semantics",
            "authors": [
                "Myeongjun Jang",
                "Pilsung Kang"
            ],
            "abstract": "Sentence embedding is a significant research topic in the field of natural\nlanguage processing (NLP). Generating sentence embedding vectors reflecting the\nintrinsic meaning of a sentence is a key factor to achieve an enhanced\nperformance in various NLP tasks such as sentence classification and document\nsummarization. Therefore, various sentence embedding models based on supervised\nand unsupervised learning have been proposed after the advent of researches\nregarding the distributed representation of words. They were evaluated through\nsemantic textual similarity (STS) tasks, which measure the degree of semantic\npreservation of a sentence and neural network-based supervised embedding models\ngenerally yielded state-of-the-art performance. However, these models have a\nlimitation in that they have multiple parameters to update, thereby requiring a\ntremendous amount of labeled training data. In this study, we propose an\nefficient approach that learns a transition matrix that refines a sentence\nembedding vector to reflect the latent semantic meaning of a sentence. The\nproposed method has two practical advantages; (1) it can be applied to any\nsentence embedding method, and (2) it can achieve robust performance in STS\ntasks irrespective of the number of training examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05280v1",
            "date": "2019-01-16 13:48:53+00:00",
            "title": "Dependency or Span, End-to-End Uniform Semantic Role Labeling",
            "authors": [
                "Zuchao Li",
                "Shexia He",
                "Hai Zhao",
                "Yiqing Zhang",
                "Zhuosheng Zhang",
                "Xi Zhou",
                "Xiang Zhou"
            ],
            "abstract": "Semantic role labeling (SRL) aims to discover the predicateargument structure\nof a sentence. End-to-end SRL without syntactic input has received great\nattention. However, most of them focus on either span-based or dependency-based\nsemantic representation form and only show specific model optimization\nrespectively. Meanwhile, handling these two SRL tasks uniformly was less\nsuccessful. This paper presents an end-to-end model for both dependency and\nspan SRL with a unified argument representation to deal with two different\ntypes of argument annotations in a uniform fashion. Furthermore, we jointly\npredict all predicates and arguments, especially including long-term ignored\npredicate identification subtask. Our single model achieves new\nstate-of-the-art results on both span (CoNLL 2005, 2012) and dependency (CoNLL\n2008, 2009) SRL benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05287v1",
            "date": "2019-01-16 14:01:15+00:00",
            "title": "Assessing BERT's Syntactic Abilities",
            "authors": [
                "Yoav Goldberg"
            ],
            "abstract": "I assess the extent to which the recently introduced BERT model captures\nEnglish syntactic phenomena, using (1) naturally-occurring subject-verb\nagreement stimuli; (2) \"coloreless green ideas\" subject-verb agreement stimuli,\nin which content words in natural sentences are randomly replaced with words\nsharing the same part-of-speech and inflection; and (3) manually crafted\nstimuli for subject-verb agreement and reflexive anaphora phenomena. The BERT\nmodel performs remarkably well on all cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05816v1",
            "date": "2019-01-17 14:42:38+00:00",
            "title": "Robust Chinese Word Segmentation with Contextualized Word Representations",
            "authors": [
                "Yung-Sung Chuang"
            ],
            "abstract": "In recent years, after the neural-network-based method was proposed, the\naccuracy of the Chinese word segmentation task has made great progress.\nHowever, when dealing with out-of-vocabulary words, there is still a large\nerror rate. We used a simple bidirectional LSTM architecture and a large-scale\npretrained language model to generate high-quality contextualize character\nrepresentations, which successfully reduced the weakness of the ambiguous\nmeanings of each Chinese character that widely appears in Chinese characters,\nand hence effectively reduced OOV error rate. State-of-the-art performance is\nachieved on many datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06079v1",
            "date": "2019-01-18 04:16:56+00:00",
            "title": "Chinese Word Segmentation: Another Decade Review (2007-2017)",
            "authors": [
                "Hai Zhao",
                "Deng Cai",
                "Changning Huang",
                "Chunyu Kit"
            ],
            "abstract": "This paper reviews the development of Chinese word segmentation (CWS) in the\nmost recent decade, 2007-2017. Special attention was paid to the deep learning\ntechnologies that has already permeated into most areas of natural language\nprocessing (NLP). The basic view we have arrived at is that compared to\ntraditional supervised learning methods, neural network based methods have not\nshown any superior performance. The most critical challenge still lies on\nbalancing of recognition of in-vocabulary (IV) and out-of-vocabulary (OOV)\nwords. However, as neural models have potentials to capture the essential\nlinguistic structure of natural language, we are optimistic about significant\nprogresses may arrive in the near future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06103v1",
            "date": "2019-01-18 06:48:53+00:00",
            "title": "Exploring Semi-supervised Variational Autoencoders for Biomedical Relation Extraction",
            "authors": [
                "Yijia Zhang",
                "Zhiyong Lu"
            ],
            "abstract": "The biomedical literature provides a rich source of knowledge such as\nprotein-protein interactions (PPIs), drug-drug interactions (DDIs) and\nchemical-protein interactions (CPIs). Biomedical relation extraction aims to\nautomatically extract biomedical relations from biomedical text for various\nbiomedical research. State-of-the-art methods for biomedical relation\nextraction are primarily based on supervised machine learning and therefore\ndepend on (sufficient) labeled data. However, creating large sets of training\ndata is prohibitively expensive and labor-intensive, especially so in\nbiomedicine as domain knowledge is required. In contrast, there is a large\namount of unlabeled biomedical text available in PubMed. Hence, computational\nmethods capable of employing unlabeled data to reduce the burden of manual\nannotation are of particular interest in biomedical relation extraction. We\npresent a novel semi-supervised approach based on variational autoencoder (VAE)\nfor biomedical relation extraction. Our model consists of the following three\nparts, a classifier, an encoder and a decoder. The classifier is implemented\nusing multi-layer convolutional neural networks (CNNs), and the encoder and\ndecoder are implemented using both bidirectional long short-term memory\nnetworks (Bi-LSTMs) and CNNs, respectively. The semi-supervised mechanism\nallows our model to learn features from both the labeled and unlabeled data. We\nevaluate our method on multiple public PPI, DDI and CPI corpora. Experimental\nresults show that our method effectively exploits the unlabeled data to improve\nthe performance and reduce the dependence on labeled data. To our best\nknowledge, this is the first semi-supervised VAE-based method for (biomedical)\nrelation extraction. Our results suggest that exploiting such unlabeled data\ncan be greatly beneficial to improved performance in various biomedical\nrelation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06283v1",
            "date": "2019-01-18 14:52:34+00:00",
            "title": "Improving Sequence-to-Sequence Learning via Optimal Transport",
            "authors": [
                "Liqun Chen",
                "Yizhe Zhang",
                "Ruiyi Zhang",
                "Chenyang Tao",
                "Zhe Gan",
                "Haichao Zhang",
                "Bai Li",
                "Dinghan Shen",
                "Changyou Chen",
                "Lawrence Carin"
            ],
            "abstract": "Sequence-to-sequence models are commonly trained via maximum likelihood\nestimation (MLE). However, standard MLE training considers a word-level\nobjective, predicting the next word given the previous ground-truth partial\nsentence. This procedure focuses on modeling local syntactic patterns, and may\nfail to capture long-range semantic structure. We present a novel solution to\nalleviate these issues. Our approach imposes global sequence-level guidance via\nnew supervision based on optimal transport, enabling the overall\ncharacterization and preservation of semantic features. We further show that\nthis method can be understood as a Wasserstein gradient flow trying to match\nour model to the ground truth sequence distribution. Extensive experiments are\nconducted to validate the utility of the proposed approach, showing consistent\nimprovements over a wide variety of NLP tasks, including machine translation,\nabstractive text summarization, and image captioning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06436v2",
            "date": "2019-01-18 22:43:17+00:00",
            "title": "Modeling Latent Sentence Structure in Neural Machine Translation",
            "authors": [
                "Jasmijn Bastings",
                "Wilker Aziz",
                "Ivan Titov",
                "Khalil Sima'an"
            ],
            "abstract": "Recently it was shown that linguistic structure predicted by a supervised\nparser can be beneficial for neural machine translation (NMT). In this work we\ninvestigate a more challenging setup: we incorporate sentence structure as a\nlatent variable in a standard NMT encoder-decoder and induce it in such a way\nas to benefit the translation task. We consider German-English and\nJapanese-English translation benchmarks and observe that when using RNN\nencoders the model makes no or very limited use of the structure induction\napparatus. In contrast, CNN and word-embedding-based encoders rely on latent\ngraphs and force them to encode useful, potentially long-distance,\ndependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06543v2",
            "date": "2019-01-19 15:40:08+00:00",
            "title": "MOROCO: The Moldavian and Romanian Dialectal Corpus",
            "authors": [
                "Andrei M. Butnaru",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this work, we introduce the MOldavian and ROmanian Dialectal COrpus\n(MOROCO), which is freely available for download at\nhttps://github.com/butnaruandrei/MOROCO. The corpus contains 33564 samples of\ntext (with over 10 million tokens) collected from the news domain. The samples\nbelong to one of the following six topics: culture, finance, politics, science,\nsports and tech. The data set is divided into 21719 samples for training, 5921\nsamples for validation and another 5924 samples for testing. For each sample,\nwe provide corresponding dialectal and category labels. This allows us to\nperform empirical studies on several classification tasks such as (i) binary\ndiscrimination of Moldavian versus Romanian text samples, (ii) intra-dialect\nmulti-class categorization by topic and (iii) cross-dialect multi-class\ncategorization by topic. We perform experiments using a shallow approach based\non string kernels, as well as a novel deep approach based on character-level\nconvolutional neural networks containing Squeeze-and-Excitation blocks. We also\npresent and analyze the most discriminative features of our best performing\nmodel, before and after named entity removal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06796v3",
            "date": "2019-01-21 05:55:42+00:00",
            "title": "Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey",
            "authors": [
                "Wei Emma Zhang",
                "Quan Z. Sheng",
                "Ahoud Alhazmi",
                "Chenliang Li"
            ],
            "abstract": "With the development of high computational devices, deep neural networks\n(DNNs), in recent years, have gained significant popularity in many Artificial\nIntelligence (AI) applications. However, previous efforts have shown that DNNs\nwere vulnerable to strategically modified samples, named adversarial examples.\nThese samples are generated with some imperceptible perturbations but can fool\nthe DNNs to give false predictions. Inspired by the popularity of generating\nadversarial examples for image DNNs, research efforts on attacking DNNs for\ntextual applications emerges in recent years. However, existing perturbation\nmethods for images cannotbe directly applied to texts as text data is discrete.\nIn this article, we review research works that address this difference and\ngeneratetextual adversarial examples on DNNs. We collect, select, summarize,\ndiscuss and analyze these works in a comprehensive way andcover all the related\ninformation to make the article self-contained. Finally, drawing on the\nreviewed literature, we provide further discussions and suggestions on this\ntopic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07003v1",
            "date": "2019-01-21 17:24:54+00:00",
            "title": "Chemical Names Standardization using Neural Sequence to Sequence Model",
            "authors": [
                "Junlang Zhan",
                "Hai Zhao"
            ],
            "abstract": "Chemical information extraction is to convert chemical knowledge in text into\ntrue chemical database, which is a text processing task heavily relying on\nchemical compound name identification and standardization. Once a systematic\nname for a chemical compound is given, it will naturally and much simply\nconvert the name into the eventually required molecular formula. However, for\nmany chemical substances, they have been shown in many other names besides\ntheir systematic names which poses a great challenge for this task. In this\npaper, we propose a framework to do the auto standardization from the\nnon-systematic names to the corresponding systematic names by using the\nspelling error correction, byte pair encoding tokenization and neural sequence\nto sequence model. Our framework is trained end to end and is fully\ndata-driven. Our standardization accuracy on the test dataset achieves 54.04%\nwhich has a great improvement compared to previous state-of-the-art result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07129v1",
            "date": "2019-01-22 00:29:27+00:00",
            "title": "An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation",
            "authors": [
                "Xiang Kong",
                "Bohan Li",
                "Graham Neubig",
                "Eduard Hovy",
                "Yiming Yang"
            ],
            "abstract": "In this work, we propose a method for neural dialogue response generation\nthat allows not only generating semantically reasonable responses according to\nthe dialogue history, but also explicitly controlling the sentiment of the\nresponse via sentiment labels. Our proposed model is based on the paradigm of\nconditional adversarial learning; the training of a sentiment-controlled\ndialogue generator is assisted by an adversarial discriminator which assesses\nthe fluency and feasibility of the response generating from the dialogue\nhistory and a given sentiment label. Because of the flexibility of our\nframework, the generator could be a standard sequence-to-sequence (SEQ2SEQ)\nmodel or a more complicated one such as a conditional variational\nautoencoder-based SEQ2SEQ model. Experimental results using automatic and human\nevaluation both demonstrate that our proposed framework is able to generate\nboth semantically reasonable and sentiment-controlled dialogue responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07291v1",
            "date": "2019-01-22 13:22:34+00:00",
            "title": "Cross-lingual Language Model Pretraining",
            "authors": [
                "Guillaume Lample",
                "Alexis Conneau"
            ],
            "abstract": "Recent studies have demonstrated the efficiency of generative pretraining for\nEnglish natural language understanding. In this work, we extend this approach\nto multiple languages and show the effectiveness of cross-lingual pretraining.\nWe propose two methods to learn cross-lingual language models (XLMs): one\nunsupervised that only relies on monolingual data, and one supervised that\nleverages parallel data with a new cross-lingual language model objective. We\nobtain state-of-the-art results on cross-lingual classification, unsupervised\nand supervised machine translation. On XNLI, our approach pushes the state of\nthe art by an absolute gain of 4.9% accuracy. On unsupervised machine\ntranslation, we obtain 34.3 BLEU on WMT'16 German-English, improving the\nprevious state of the art by more than 9 BLEU. On supervised machine\ntranslation, we obtain a new state of the art of 38.5 BLEU on WMT'16\nRomanian-English, outperforming the previous best approach by more than 4 BLEU.\nOur code and pretrained models will be made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07426v1",
            "date": "2019-01-22 15:42:51+00:00",
            "title": "Deep learning and sub-word-unit approach in written art generation",
            "authors": [
                "Krzysztof Wo\u0142k",
                "Emilia Zawadzka-Gosk",
                "Wojciech Czarnowski"
            ],
            "abstract": "Automatic poetry generation is novel and interesting application of natural\nlanguage processing research. It became more popular during the last few years\ndue to the rapid development of technology and neural computing power. This\nline of research can be applied to the study of linguistics and literature, for\nsocial science experiments, or simply for entertainment. The most effective\nknown method of artificial poem generation uses recurrent neural networks\n(RNN). We also used RNNs to generate poems in the style of Adam Mickiewicz. Our\nnetwork was trained on the Sir Thaddeus poem. For data pre-processing, we used\na specialized stemming tool, which is one of the major innovations and\ncontributions of this work. Our experiment was conducted on the source text,\ndivided into sub-word units (at a level of resolution close to syllables). This\napproach is novel and is not often employed in the published literature. The\nsubwords units seem to be a natural choice for analysis of the Polish language,\nas the language is morphologically rich due to cases, gender forms and a large\nvocabulary. Moreover, Sir Thaddeus contains rhymes, so the analysis of\nsyllables can be meaningful. We verified our model with different settings for\nthe temperature parameter, which controls the randomness of the generated text.\nWe also compared our results with similar models trained on the same text but\ndivided into characters (which is the most common approach alongside the use of\nfull word units). The differences were tremendous. Our solution generated much\nbetter poems that were able to follow the metre and vocabulary of the source\ndata text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07475v1",
            "date": "2019-01-22 17:17:02+00:00",
            "title": "Debugging Frame Semantic Role Labeling",
            "authors": [
                "Alexandre Kabbach"
            ],
            "abstract": "We propose a quantitative and qualitative analysis of the performances of\nstatistical models for frame semantic structure extraction. We report on a\nreplication study on FrameNet 1.7 data and show that preprocessing toolkits\nplay a major role in argument identification performances, observing gains\nsimilar in their order of magnitude to those reported by recent models for\nframe semantic parsing. We report on the robustness of a recent statistical\nclassifier for frame semantic parsing to lexical configurations of\npredicate-argument structures, relying on an artificially augmented dataset\ngenerated using a rule-based algorithm combining valence pattern matching and\nlexical substitution. We prove that syntactic pre-processing plays a major role\nin the performances of statistical classifiers to argument identification, and\ndiscuss the core reasons of syntactic mismatch between dependency parsers\noutput and FrameNet syntactic formalism. Finally, we suggest new leads for\nimproving statistical models for frame semantic parsing, including joint\nsyntax-semantic parsing relying on FrameNet syntactic formalism, latent classes\ninference via split-and-merge algorithms and neural network architectures\nrelying on rich input representations of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07651v3",
            "date": "2019-01-22 23:55:49+00:00",
            "title": "Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings",
            "authors": [
                "Hwiyeol Jo",
                "Ceyda Cinarel"
            ],
            "abstract": "We propose a novel and simple method for semi-supervised text classification.\nThe method stems from the hypothesis that a classifier with pretrained word\nembeddings always outperforms the same classifier with randomly initialized\nword embeddings, as empirically observed in NLP tasks. Our method first builds\ntwo sets of classifiers as a form of model ensemble, and then initializes their\nword embeddings differently: one using random, the other using pretrained word\nembeddings. We focus on different predictions between the two classifiers on\nunlabeled data while following the self-training framework. We also use\nearly-stopping in meta-epoch to improve the performance of our method. Our\nmethod, Delta-training, outperforms the self-training and the co-training\nframework in 4 different text classification datasets, showing robustness\nagainst error accumulation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07656v1",
            "date": "2019-01-23 00:25:48+00:00",
            "title": "Attenuating Bias in Word Vectors",
            "authors": [
                "Sunipa Dev",
                "Jeff Phillips"
            ],
            "abstract": "Word vector representations are well developed tools for various NLP and\nMachine Learning tasks and are known to retain significant semantic and\nsyntactic structure of languages. But they are prone to carrying and amplifying\nbias which can perpetrate discrimination in various applications. In this work,\nwe explore new simple ways to detect the most stereotypically gendered words in\nan embedding and remove the bias from them. We verify how names are masked\ncarriers of gender bias and then use that as a tool to attenuate bias in\nembeddings. Further, we extend this property of names to show how names can be\nused to detect other types of bias in the embeddings such as bias based on\nrace, ethnicity, and age.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07688v1",
            "date": "2019-01-23 02:11:11+00:00",
            "title": "Context-Sensitive Malicious Spelling Error Correction",
            "authors": [
                "Hongyu Gong",
                "Yuchen Li",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "Misspelled words of the malicious kind work by changing specific keywords and\nare intended to thwart existing automated applications for cyber-environment\ncontrol such as harassing content detection on the Internet and email spam\ndetection. In this paper, we focus on malicious spelling correction, which\nrequires an approach that relies on the context and the surface forms of\ntargeted keywords. In the context of two applications--profanity detection and\nemail spam detection--we show that malicious misspellings seriously degrade\ntheir performance. We then propose a context-sensitive approach for malicious\nspelling correction using word embeddings and demonstrate its superior\nperformance compared to state-of-the-art spell checkers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07744v2",
            "date": "2019-01-23 06:37:12+00:00",
            "title": "Automated Essay Scoring based on Two-Stage Learning",
            "authors": [
                "Jiawei Liu",
                "Yang Xu",
                "Yaguang Zhu"
            ],
            "abstract": "Current state-of-art feature-engineered and end-to-end Automated Essay Score\n(AES) methods are proven to be unable to detect adversarial samples, e.g. the\nessays composed of permuted sentences and the prompt-irrelevant essays.\nFocusing on the problem, we develop a Two-Stage Learning Framework (TSLF) which\nintegrates the advantages of both feature-engineered and end-to-end AES models.\nIn experiments, we compare TSLF against a number of strong baselines, and the\nresults demonstrate the effectiveness and robustness of our models. TSLF\nsurpasses all the baselines on five-eighths of prompts and achieves new\nstate-of-the-art average performance when without negative samples. After\nadding some adversarial essays to the original datasets, TSLF outperforms the\nfeature-engineered and end-to-end baselines to a great extent, and shows great\nrobustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07867v1",
            "date": "2019-01-23 13:23:10+00:00",
            "title": "Context based Analysis of Lexical Semantics for Hindi Language",
            "authors": [
                "Mohd Zeeshan Ansari",
                "Lubna Khan"
            ],
            "abstract": "A word having multiple senses in a text introduces the lexical semantic task\nto find out which particular sense is appropriate for the given context. One\nsuch task is Word sense disambiguation which refers to the identification of\nthe most appropriate meaning of the polysemous word in a given context using\ncomputational algorithms. The language processing research in Hindi, the\nofficial language of India, and other Indian languages is restricted by\nunavailability of the standard corpus. For Hindi word sense disambiguation\nalso, the large corpus is not available. In this work, we prepared the text\ncontaining new senses of certain words leading to the enrichment of the\nsense-tagged Hindi corpus of sixty polysemous words. Furthermore, we analyzed\ntwo novel lexical associations for Hindi word sense disambiguation based on the\ncontextual features of the polysemous word. The evaluation of these methods is\ncarried out over learning algorithms and favorable results are achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07931v3",
            "date": "2019-01-23 14:54:53+00:00",
            "title": "Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Jekaterina Novikova",
                "Verena Rieser"
            ],
            "abstract": "This paper provides a comprehensive analysis of the first shared task on\nEnd-to-End Natural Language Generation (NLG) and identifies avenues for future\nresearch based on the results. This shared task aimed to assess whether recent\nend-to-end NLG systems can generate more complex output by learning from\ndatasets containing higher lexical richness, syntactic complexity and diverse\ndiscourse phenomena. Introducing novel automatic and human metrics, we compare\n62 systems submitted by 17 institutions, covering a wide range of approaches,\nincluding machine learning architectures -- with the majority implementing\nsequence-to-sequence models (seq2seq) -- as well as systems based on\ngrammatical rules and templates. Seq2seq-based systems have demonstrated a\ngreat potential for NLG in the challenge. We find that seq2seq systems\ngenerally score high in terms of word-overlap metrics and human evaluations of\nnaturalness -- with the winning SLUG system (Juraska et al., 2018) being\nseq2seq-based. However, vanilla seq2seq models often fail to correctly express\na given meaning representation if they lack a strong semantic control mechanism\napplied during decoding. Moreover, seq2seq models can be outperformed by\nhand-engineered systems in terms of overall quality, as well as complexity,\nlength and diversity of outputs. This research has influenced, inspired and\nmotivated a number of recent studies outwith the original competition, which we\nalso summarise as part of this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08014v2",
            "date": "2019-01-23 17:18:50+00:00",
            "title": "Sentiment and Sarcasm Classification with Multitask Learning",
            "authors": [
                "Navonil Majumder",
                "Soujanya Poria",
                "Haiyun Peng",
                "Niyati Chhaya",
                "Erik Cambria",
                "Alexander Gelbukh"
            ],
            "abstract": "Sentiment classification and sarcasm detection are both important natural\nlanguage processing (NLP) tasks. Sentiment is always coupled with sarcasm where\nintensive emotion is expressed. Nevertheless, most literature considers them as\ntwo separate tasks. We argue that knowledge in sarcasm detection can also be\nbeneficial to sentiment classification and vice versa. We show that these two\ntasks are correlated, and present a multi-task learning-based framework using a\ndeep neural network that models this correlation to improve the performance of\nboth tasks in a multi-task learning setting. Our method outperforms the state\nof the art by 3-4% in the benchmark dataset.",
            "journal ref": "IEEE Intelligent Systems 34(3) (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08149v2",
            "date": "2019-01-23 22:08:01+00:00",
            "title": "TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents",
            "authors": [
                "Thomas Wolf",
                "Victor Sanh",
                "Julien Chaumond",
                "Clement Delangue"
            ],
            "abstract": "We introduce a new approach to generative data-driven dialogue systems (e.g.\nchatbots) called TransferTransfo which is a combination of a Transfer learning\nbased training scheme and a high-capacity Transformer model. Fine-tuning is\nperformed by using a multi-task objective which combines several unsupervised\nprediction tasks. The resulting fine-tuned model shows strong improvements over\nthe current state-of-the-art end-to-end conversational models like memory\naugmented seq2seq and information-retrieval models. On the privately held\nPERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this\napproach obtains a new state-of-the-art, with respective perplexity, Hits@1 and\nF1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute\nimprovement) and 19.5 (20 % absolute improvement).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08163v1",
            "date": "2019-01-23 23:19:45+00:00",
            "title": "Semantic Relation Classification via Bidirectional LSTM Networks with Entity-aware Attention using Latent Entity Typing",
            "authors": [
                "Joohong Lee",
                "Sangwoo Seo",
                "Yong Suk Choi"
            ],
            "abstract": "Classifying semantic relations between entity pairs in sentences is an\nimportant task in Natural Language Processing (NLP). Most previous models for\nrelation classification rely on the high-level lexical and syntactic features\nobtained by NLP tools such as WordNet, dependency parser, part-of-speech (POS)\ntagger, and named entity recognizers (NER). In addition, state-of-the-art\nneural models based on attention mechanisms do not fully utilize information of\nentity that may be the most crucial features for relation classification. To\naddress these issues, we propose a novel end-to-end recurrent neural model\nwhich incorporates an entity-aware attention mechanism with a latent entity\ntyping (LET) method. Our model not only utilizes entities and their latent\ntypes as features effectively but also is more interpretable by visualizing\nattention mechanisms applied to our model and results of LET. Experimental\nresults on the SemEval-2010 Task 8, one of the most popular relation\nclassification task, demonstrate that our model outperforms existing\nstate-of-the-art models without any high-level features.",
            "journal ref": "Symmetry 2019, 11 (6), 785",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08319v1",
            "date": "2019-01-24 10:02:45+00:00",
            "title": "A review of sentiment computation methods with R packages",
            "authors": [
                "Maurizio Naldi"
            ],
            "abstract": "Four packages in R are analyzed to carry out sentiment analysis. All packages\nallow to define custom dictionaries. Just one - Sentiment R - properly accounts\nfor the presence of negators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08625v1",
            "date": "2019-01-24 19:49:43+00:00",
            "title": "Automatic Parallel Corpus Creation for Hindi-English News Translation Task",
            "authors": [
                "Aditya Kumar Pathak",
                "Priyankit Acharya",
                "Dilpreet Kaur",
                "Rakesh Chandra Balabantaray"
            ],
            "abstract": "The parallel corpus for multilingual NLP tasks, deep learning applications\nlike Statistical Machine Translation Systems is very important. The parallel\ncorpus of Hindi-English language pair available for news translation task till\ndate is of very limited size as per the requirement of the systems are\nconcerned. In this work we have developed an automatic parallel corpus\ngeneration system prototype, which creates Hindi-English parallel corpus for\nnews translation task. Further to verify the quality of generated parallel\ncorpus we have experimented by taking various performance metrics and the\nresults are quite interesting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08634v3",
            "date": "2019-01-24 20:22:14+00:00",
            "title": "A BERT Baseline for the Natural Questions",
            "authors": [
                "Chris Alberti",
                "Kenton Lee",
                "Michael Collins"
            ],
            "abstract": "This technical note describes a new baseline for the Natural Questions. Our\nmodel is based on BERT and reduces the gap between the model F1 scores reported\nin the original dataset paper and the human upper bound by 30% and 50% relative\nfor the long and short answer tasks respectively. This baseline has been\nsubmitted to the official NQ leaderboard at\nai.google.com/research/NaturalQuestions. Code, preprocessed data and pretrained\nmodel are available at\nhttps://github.com/google-research/language/tree/master/language/question_answering/bert_joint.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08746v4",
            "date": "2019-01-25 05:57:24+00:00",
            "title": "BioBERT: a pre-trained biomedical language representation model for biomedical text mining",
            "authors": [
                "Jinhyuk Lee",
                "Wonjin Yoon",
                "Sungdong Kim",
                "Donghyeon Kim",
                "Sunkyu Kim",
                "Chan Ho So",
                "Jaewoo Kang"
            ],
            "abstract": "Biomedical text mining is becoming increasingly important as the number of\nbiomedical documents rapidly grows. With the progress in natural language\nprocessing (NLP), extracting valuable information from biomedical literature\nhas gained popularity among researchers, and deep learning has boosted the\ndevelopment of effective biomedical text mining models. However, directly\napplying the advancements in NLP to biomedical text mining often yields\nunsatisfactory results due to a word distribution shift from general domain\ncorpora to biomedical corpora. In this article, we investigate how the recently\nintroduced pre-trained language model BERT can be adapted for biomedical\ncorpora. We introduce BioBERT (Bidirectional Encoder Representations from\nTransformers for Biomedical Text Mining), which is a domain-specific language\nrepresentation model pre-trained on large-scale biomedical corpora. With almost\nthe same architecture across tasks, BioBERT largely outperforms BERT and\nprevious state-of-the-art models in a variety of biomedical text mining tasks\nwhen pre-trained on biomedical corpora. While BERT obtains performance\ncomparable to that of previous state-of-the-art models, BioBERT significantly\noutperforms them on the following three representative biomedical text mining\ntasks: biomedical named entity recognition (0.62% F1 score improvement),\nbiomedical relation extraction (2.80% F1 score improvement) and biomedical\nquestion answering (12.24% MRR improvement). Our analysis results show that\npre-training BERT on biomedical corpora helps it to understand complex\nbiomedical texts. We make the pre-trained weights of BioBERT freely available\nat https://github.com/naver/biobert-pretrained, and the source code for\nfine-tuning BioBERT available at https://github.com/dmis-lab/biobert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09115v1",
            "date": "2019-01-25 23:25:18+00:00",
            "title": "Context in Neural Machine Translation: A Review of Models and Evaluations",
            "authors": [
                "Andrei Popescu-Belis"
            ],
            "abstract": "This review paper discusses how context has been used in neural machine\ntranslation (NMT) in the past two years (2017-2018). Starting with a brief\nretrospect on the rapid evolution of NMT models, the paper then reviews studies\nthat evaluate NMT output from various perspectives, with emphasis on those\nanalyzing limitations of the translation of contextual phenomena. In a\nsubsequent version, the paper will then present the main methods that were\nproposed to leverage context for improving translation quality, and\ndistinguishes methods that aim to improve the translation of specific phenomena\nfrom those that consider a wider unstructured context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09128v1",
            "date": "2019-01-26 00:35:35+00:00",
            "title": "Language Model Pre-training for Hierarchical Document Representations",
            "authors": [
                "Ming-Wei Chang",
                "Kristina Toutanova",
                "Kenton Lee",
                "Jacob Devlin"
            ],
            "abstract": "Hierarchical neural architectures are often used to capture long-distance\ndependencies and have been applied to many document-level tasks such as\nsummarization, document segmentation, and sentiment analysis. However,\neffective usage of such a large context can be difficult to learn, especially\nin the case where there is limited labeled data available. Building on the\nrecent success of language model pretraining methods for learning flat\nrepresentations of text, we propose algorithms for pre-training hierarchical\ndocument representations from unlabeled data. Unlike prior work, which has\nfocused on pre-training contextual token representations or context-independent\n{sentence/paragraph} representations, our hierarchical document representations\ninclude fixed-length sentence/paragraph representations which integrate\ncontextual information from the entire documents. Experiments on document\nsegmentation, document-level question answering, and extractive document\nsummarization demonstrate the effectiveness of the proposed pre-training\nalgorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09219v2",
            "date": "2019-01-26 14:18:57+00:00",
            "title": "Implicit Dimension Identification in User-Generated Text with LSTM Networks",
            "authors": [
                "Victor Makarenkov",
                "Ido Guy",
                "Niva Hazon",
                "Tamar Meisels",
                "Bracha Shapira",
                "Lior Rokach"
            ],
            "abstract": "In the process of online storytelling, individual users create and consume\nhighly diverse content that contains a great deal of implicit beliefs and not\nplainly expressed narrative. It is hard to manually detect these implicit\nbeliefs, intentions and moral foundations of the writers. We study and\ninvestigate two different tasks, each of which reflect the difficulty of\ndetecting an implicit user's knowledge, intent or belief that may be based on\nwriter's moral foundation: 1) political perspective detection in news articles\n2) identification of informational vs. conversational questions in community\nquestion answering (CQA) archives and. In both tasks we first describe new\ninteresting annotated datasets and make the datasets publicly available.\nSecond, we compare various classification algorithms, and show the differences\nin their performance on both tasks. Third, in political perspective detection\ntask we utilize a narrative representation language of local press to identify\nperspective differences between presumably neutral American and British press.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09296v1",
            "date": "2019-01-27 00:46:27+00:00",
            "title": "Variational Smoothing in Recurrent Neural Network Language Models",
            "authors": [
                "Lingpeng Kong",
                "Gabor Melis",
                "Wang Ling",
                "Lei Yu",
                "Dani Yogatama"
            ],
            "abstract": "We present a new theoretical perspective of data noising in recurrent neural\nnetwork language models (Xie et al., 2017). We show that each variant of data\nnoising is an instance of Bayesian recurrent neural networks with a particular\nvariational distribution (i.e., a mixture of Gaussians whose weights depend on\nstatistics derived from the corpus such as the unigram distribution). We use\nthis insight to propose a more principled method to apply at prediction time\nand propose natural extensions to data noising under the variational framework.\nIn particular, we propose variational smoothing with tied input and output\nembedding matrices and an element-wise variational smoothing method. We\nempirically verify our analysis on two benchmark language modeling datasets and\ndemonstrate performance improvements over existing data noising methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09381v2",
            "date": "2019-01-27 14:15:37+00:00",
            "title": "Dual Co-Matching Network for Multi-choice Reading Comprehension",
            "authors": [
                "Shuailiang Zhang",
                "Hai Zhao",
                "Yuwei Wu",
                "Zhuosheng Zhang",
                "Xi Zhou",
                "Xiang Zhou"
            ],
            "abstract": "Multi-choice reading comprehension is a challenging task that requires\ncomplex reasoning procedure. Given passage and question, a correct answer need\nto be selected from a set of candidate answers. In this paper, we propose\n\\textbf{D}ual \\textbf{C}o-\\textbf{M}atching \\textbf{N}etwork (\\textbf{DCMN})\nwhich model the relationship among passage, question and answer\nbidirectionally. Different from existing approaches which only calculate\nquestion-aware or option-aware passage representation, we calculate\npassage-aware question representation and passage-aware answer representation\nat the same time. To demonstrate the effectiveness of our model, we evaluate\nour model on a large-scale multiple choice machine reading comprehension\ndataset (i.e. RACE). Experimental result show that our proposed model achieves\nnew state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09444v2",
            "date": "2019-01-27 21:40:10+00:00",
            "title": "Promoting Diversity for End-to-End Conversation Response Generation",
            "authors": [
                "Yu-Ping Ruan",
                "Zhen-Hua Ling",
                "Quan Liu",
                "Jia-Chen Gu",
                "Xiaodan Zhu"
            ],
            "abstract": "We present our work on Track 2 in the Dialog System Technology Challenges 7\n(DSTC7). The DSTC7-Track 2 aims to evaluate the response generation of fully\ndata-driven conversation models in knowledge-grounded settings, which provides\nthe contextual-relevant factual texts. The Sequenceto-Sequence models have been\nwidely used for end-to-end generative conversation modelling and achieved\nimpressive results. However, they tend to output dull and repeated responses in\nprevious studies. Our work aims to promote the diversity for end-to-end\nconversation response generation, which follows a two-stage pipeline: 1)\nGenerate multiple responses. At this stage, two different models are proposed,\ni.e., a variational generative (VariGen) model and a retrieval based\n(Retrieval) model. 2) Rank and return the most related response by training a\ntopic coherence discrimination (TCD) model for the ranking process. According\nto the official evaluation results, our proposed Retrieval and VariGen systems\nranked first and second respectively on objective diversity metrics, i.e.,\nEntropy, among all participant systems. And the VariGen system ranked second on\nNIST and METEOR metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09672v2",
            "date": "2019-01-28 14:22:15+00:00",
            "title": "Personalized Dialogue Generation with Diversified Traits",
            "authors": [
                "Yinhe Zheng",
                "Guanyi Chen",
                "Minlie Huang",
                "Song Liu",
                "Xuan Zhu"
            ],
            "abstract": "Endowing a dialogue system with particular personality traits is essential to\ndeliver more human-like conversations. However, due to the challenge of\nembodying personality via language expression and the lack of large-scale\npersona-labeled dialogue data, this research problem is still far from\nwell-studied. In this paper, we investigate the problem of incorporating\nexplicit personality traits in dialogue generation to deliver personalized\ndialogues.\n  To this end, firstly, we construct PersonalDialog, a large-scale multi-turn\ndialogue dataset containing various traits from a large number of speakers. The\ndataset consists of 20.83M sessions and 56.25M utterances from 8.47M speakers.\nEach utterance is associated with a speaker who is marked with traits like Age,\nGender, Location, Interest Tags, etc. Several anonymization schemes are\ndesigned to protect the privacy of each speaker. This large-scale dataset will\nfacilitate not only the study of personalized dialogue generation, but also\nother researches on sociolinguistics or social science.\n  Secondly, to study how personality traits can be captured and addressed in\ndialogue generation, we propose persona-aware dialogue generation models within\nthe sequence to sequence learning framework. Explicit personality traits\n(structured by key-value pairs) are embedded using a trait fusion module.\nDuring the decoding process, two techniques, namely persona-aware attention and\npersona-aware bias, are devised to capture and address trait-related\ninformation. Experiments demonstrate that our model is able to address proper\ntraits in different contexts. Case studies also show interesting results for\nthis challenging research problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09755v1",
            "date": "2019-01-28 15:55:22+00:00",
            "title": "Language Independent Sequence Labelling for Opinion Target Extraction",
            "authors": [
                "Rodrigo Agerri",
                "German Rigau"
            ],
            "abstract": "In this research note we present a language independent system to model\nOpinion Target Extraction (OTE) as a sequence labelling task. The system\nconsists of a combination of clustering features implemented on top of a simple\nset of shallow local features. Experiments on the well known Aspect Based\nSentiment Analysis (ABSA) benchmarks show that our approach is very competitive\nacross languages, obtaining best results for six languages in seven different\ndatasets. Furthermore, the results provide further insights into the behaviour\nof clustering features for sequence labelling tasks. The system and models\ngenerated in this work are available for public use and to facilitate\nreproducibility of results.",
            "journal ref": "Artificial Intelligence (2018), 268: 65-85",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09785v2",
            "date": "2019-01-28 16:33:25+00:00",
            "title": "Evaluating Word Embedding Models: Methods and Experimental Results",
            "authors": [
                "Bin Wang",
                "Angela Wang",
                "Fenxiao Chen",
                "Yuncheng Wang",
                "C. -C. Jay Kuo"
            ],
            "abstract": "Extensive evaluation on a large number of word embedding models for language\nprocessing applications is conducted in this work. First, we introduce popular\nword embedding models and discuss desired properties of word models and\nevaluation methods (or evaluators). Then, we categorize evaluators into\nintrinsic and extrinsic two types. Intrinsic evaluators test the quality of a\nrepresentation independent of specific natural language processing tasks while\nextrinsic evaluators use word embeddings as input features to a downstream task\nand measure changes in performance metrics specific to that task. We report\nexperimental results of intrinsic and extrinsic evaluators on six word\nembedding models. It is shown that different evaluators focus on different\naspects of word models, and some are more correlated with natural language\nprocessing tasks. Finally, we adopt correlation analysis to study performance\nconsistency of extrinsic and intrinsic evalutors.",
            "journal ref": "APSIPA Transactions on Signal and Information Processing 8 (2019)\n  e19",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09957v1",
            "date": "2019-01-28 19:35:17+00:00",
            "title": "OpenHowNet: An Open Sememe-based Lexical Knowledge Base",
            "authors": [
                "Fanchao Qi",
                "Chenghao Yang",
                "Zhiyuan Liu",
                "Qiang Dong",
                "Maosong Sun",
                "Zhendong Dong"
            ],
            "abstract": "In this paper, we present an open sememe-based lexical knowledge base\nOpenHowNet. Based on well-known HowNet, OpenHowNet comprises three components:\ncore data which is composed of more than 100 thousand senses annotated with\nsememes, OpenHowNet Web which gives a brief introduction to OpenHowNet as well\nas provides online exhibition of OpenHowNet information, and OpenHowNet API\nwhich includes several useful APIs such as accessing OpenHowNet core data and\ndrawing sememe tree structures of senses. In the main text, we first give some\nbackgrounds including definition of sememe and details of HowNet. And then we\nintroduce some previous HowNet and sememe-based research works. Last but not\nleast, we detail the constituents of OpenHowNet and their basic features and\nfunctionalities. Additionally, we briefly make a summary and list some future\nworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10188v1",
            "date": "2019-01-29 09:24:15+00:00",
            "title": "An Arabic Dependency Treebank in the Travel Domain",
            "authors": [
                "Dima Taji",
                "Jamila El Gizuli",
                "Nizar Habash"
            ],
            "abstract": "In this paper we present a dependency treebank of travel domain sentences in\nModern Standard Arabic. The text comes from a translation of the English\nequivalent sentences in the Basic Traveling Expressions Corpus. The treebank\ndependency representation is in the style of the Columbia Arabic Treebank. The\npaper motivates the effort and discusses the construction process and\nguidelines. We also present parsing results and discuss the effect of domain\nand genre difference on parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10196v1",
            "date": "2019-01-29 10:00:54+00:00",
            "title": "Divide and Generate: Neural Generation of Complex Sentences",
            "authors": [
                "Tomoya Ogata",
                "Mamoru Komachi",
                "Tomoya Takatani"
            ],
            "abstract": "We propose a task to generate a complex sentence from a simple sentence in\norder to amplify various kinds of responses in the database. We first divide a\ncomplex sentence into a main clause and a subordinate clause to learn a\ngenerator model of modifiers, and then use the model to generate a modifier\nclause to create a complex sentence from a simple sentence. We present an\nautomatic evaluation metric to estimate the quality of the models and show that\na pipeline model outperforms an end-to-end model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10430v2",
            "date": "2019-01-29 18:01:35+00:00",
            "title": "Pay Less Attention with Lightweight and Dynamic Convolutions",
            "authors": [
                "Felix Wu",
                "Angela Fan",
                "Alexei Baevski",
                "Yann N. Dauphin",
                "Michael Auli"
            ],
            "abstract": "Self-attention is a useful mechanism to build generative models for language\nand images. It determines the importance of context elements by comparing each\nelement to the current time step. In this paper, we show that a very\nlightweight convolution can perform competitively to the best reported\nself-attention results. Next, we introduce dynamic convolutions which are\nsimpler and more efficient than self-attention. We predict separate convolution\nkernels based solely on the current time-step in order to determine the\nimportance of context elements. The number of operations required by this\napproach scales linearly in the input length, whereas self-attention is\nquadratic. Experiments on large-scale machine translation, language modeling\nand abstractive summarization show that dynamic convolutions improve over\nstrong self-attention models. On the WMT'14 English-German test set dynamic\nconvolutions achieve a new state of the art of 29.7 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10444v1",
            "date": "2019-01-29 18:44:01+00:00",
            "title": "No Training Required: Exploring Random Encoders for Sentence Classification",
            "authors": [
                "John Wieting",
                "Douwe Kiela"
            ],
            "abstract": "We explore various methods for computing sentence representations from\npre-trained word embeddings without any training, i.e., using nothing but\nrandom parameterizations. Our aim is to put sentence embeddings on more solid\nfooting by 1) looking at how much modern sentence embeddings gain over random\nmethods---as it turns out, surprisingly little; and by 2) providing the field\nwith more appropriate baselines going forward---which are, as it turns out,\nquite strong. We also make important observations about proper experimental\nprotocol for sentence classification evaluation, together with recommendations\nfor future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10457v1",
            "date": "2019-01-29 18:58:29+00:00",
            "title": "Universal Dependency Parsing from Scratch",
            "authors": [
                "Peng Qi",
                "Timothy Dozat",
                "Yuhao Zhang",
                "Christopher D. Manning"
            ],
            "abstract": "This paper describes Stanford's system at the CoNLL 2018 UD Shared Task. We\nintroduce a complete neural pipeline system that takes raw text as input, and\nperforms all tasks required by the shared task, ranging from tokenization and\nsentence segmentation, to POS tagging and dependency parsing. Our single system\nsubmission achieved very competitive performance on big treebanks. Moreover,\nafter fixing an unfortunate bug, our corrected system would have placed the\n2nd, 1st, and 3rd on the official evaluation metrics LAS,MLAS, and BLEX, and\nwould have outperformed all submission systems on low-resource treebank\ncategories on all metrics by a large margin. We further show the effectiveness\nof different model components through extensive ablation studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10623v2",
            "date": "2019-01-30 00:34:05+00:00",
            "title": "End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis",
            "authors": [
                "Lin Xu",
                "Qixian Zhou",
                "Ke Gong",
                "Xiaodan Liang",
                "Jianheng Tang",
                "Liang Lin"
            ],
            "abstract": "Beyond current conversational chatbots or task-oriented dialogue systems that\nhave attracted increasing attention, we move forward to develop a dialogue\nsystem for automatic medical diagnosis that converses with patients to collect\nadditional symptoms beyond their self-reports and automatically makes a\ndiagnosis. Besides the challenges for conversational dialogue systems (e.g.\ntopic transition coherency and question understanding), automatic medical\ndiagnosis further poses more critical requirements for the dialogue rationality\nin the context of medical knowledge and symptom-disease relations. Existing\ndialogue systems (Madotto, Wu, and Fung 2018; Wei et al. 2018; Li et al. 2017)\nmostly rely on data-driven learning and cannot be able to encode extra expert\nknowledge graph. In this work, we propose an End-to-End Knowledge-routed\nRelational Dialogue System (KR-DS) that seamlessly incorporates rich medical\nknowledge graph into the topic transition in dialogue management, and makes it\ncooperative with natural language understanding and natural language\ngeneration. A novel Knowledge-routed Deep Q-network (KR-DQN) is introduced to\nmanage topic transitions, which integrates a relational refinement branch for\nencoding relations among different symptoms and symptom-disease pairs, and a\nknowledge-routed graph branch for topic decision-making. Extensive experiments\non a public medical dialogue dataset show our KR-DS significantly beats\nstate-of-the-art methods (by more than 8% in diagnosis accuracy). We further\nshow the superiority of our KR-DS on a newly collected medical dialogue system\ndataset, which is more challenging retaining original self-reports and\nconversational data between patients and doctors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10680v1",
            "date": "2019-01-30 05:49:17+00:00",
            "title": "Effective weakly supervised semantic frame induction using expression sharing in hierarchical hidden Markov models",
            "authors": [
                "Janneke van de Loo",
                "Jort F. Gemmeke",
                "Guy De Pauw",
                "Bart Ons",
                "Walter Daelemans",
                "Hugo Van hamme"
            ],
            "abstract": "We present a framework for the induction of semantic frames from utterances\nin the context of an adaptive command-and-control interface. The system is\ntrained on an individual user's utterances and the corresponding semantic\nframes representing controls. During training, no prior information on the\nalignment between utterance segments and frame slots and values is available.\nIn addition, semantic frames in the training data can contain information that\nis not expressed in the utterances. To tackle this weakly supervised\nclassification task, we propose a framework based on Hidden Markov Models\n(HMMs). Structural modifications, resulting in a hierarchical HMM, and an\nextension called expression sharing are introduced to minimize the amount of\ntraining time and effort required for the user.\n  The dataset used for the present study is PATCOR, which contains commands\nuttered in the context of a vocally guided card game, Patience. Experiments\nwere carried out on orthographic and phonetic transcriptions of commands,\nsegmented on different levels of n-gram granularity. The experimental results\nshow positive effects of all the studied system extensions, with some effect\ndifferences between the different input representations. Moreover, evaluation\nexperiments on held-out data with the optimal system configuration show that\nthe extended system is able to achieve high accuracies with relatively small\namounts of training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10746v1",
            "date": "2019-01-30 10:21:04+00:00",
            "title": "Reference-less Quality Estimation of Text Simplification Systems",
            "authors": [
                "Louis Martin",
                "Samuel Humeau",
                "Pierre-Emmanuel Mazar\u00e9",
                "Antoine Bordes",
                "\u00c9ric Villemonte de La Clergerie",
                "Beno\u00eet Sagot"
            ],
            "abstract": "The evaluation of text simplification (TS) systems remains an open challenge.\nAs the task has common points with machine translation (MT), TS is often\nevaluated using MT metrics such as BLEU. However, such metrics require high\nquality reference data, which is rarely available for TS. TS has the advantage\nover MT of being a monolingual task, which allows for direct comparisons to be\nmade between the simplified text and its original version. In this paper, we\ncompare multiple approaches to reference-less quality estimation of\nsentence-level text simplification systems, based on the dataset used for the\nQATS 2016 shared task. We distinguish three different dimensions:\ngram-maticality, meaning preservation and simplicity. We show that n-gram-based\nMT metrics such as BLEU and METEOR correlate the most with human judgment of\ngrammaticality and meaning preservation, whereas simplicity is best evaluated\nby basic length-based metrics.",
            "journal ref": "1st Workshop on Automatic Text Adaptation (ATA), Nov 2018,\n  Tilburg, Netherlands. https://www.ida.liu.se/~evere22/ATA-18/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10879v6",
            "date": "2019-01-30 15:04:16+00:00",
            "title": "Span Model for Open Information Extraction on Accurate Corpus",
            "authors": [
                "Junlang Zhan",
                "Hai Zhao"
            ],
            "abstract": "Open information extraction (Open IE) is a challenging task especially due to\nits brittle data basis. Most of Open IE systems have to be trained on\nautomatically built corpus and evaluated on inaccurate test set. In this work,\nwe first alleviate this difficulty from both sides of training and test sets.\nFor the former, we propose an improved model design to more sufficiently\nexploit training dataset. For the latter, we present our accurately\nre-annotated benchmark test set (Re-OIE6) according to a series of linguistic\nobservation and analysis. Then, we introduce a span model instead of previous\nadopted sequence labeling formulization for n-ary Open IE. Our newly introduced\nmodel achieves new state-of-the-art performance on both benchmark evaluation\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11196v2",
            "date": "2019-01-31 03:20:52+00:00",
            "title": "EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks",
            "authors": [
                "Jason Wei",
                "Kai Zou"
            ],
            "abstract": "We present EDA: easy data augmentation techniques for boosting performance on\ntext classification tasks. EDA consists of four simple but powerful operations:\nsynonym replacement, random insertion, random swap, and random deletion. On\nfive text classification tasks, we show that EDA improves performance for both\nconvolutional and recurrent neural networks. EDA demonstrates particularly\nstrong results for smaller datasets; on average, across five datasets, training\nwith EDA while using only 50% of the available training set achieved the same\naccuracy as normal training with all available data. We also performed\nextensive ablation studies and suggest parameters for practical use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11359v1",
            "date": "2019-01-31 14:05:02+00:00",
            "title": "Adding Interpretable Attention to Neural Translation Models Improves Word Alignment",
            "authors": [
                "Thomas Zenkel",
                "Joern Wuebker",
                "John DeNero"
            ],
            "abstract": "Multi-layer models with multiple attention heads per layer provide superior\ntranslation quality compared to simpler and shallower models, but determining\nwhat source context is most relevant to each target word is more challenging as\na result. Therefore, deriving high-accuracy word alignments from the\nactivations of a state-of-the-art neural machine translation model is an open\nchallenge. We propose a simple model extension to the Transformer architecture\nthat makes use of its hidden representations and is restricted to attend solely\non encoder information to predict the next word. It can be trained on bilingual\ndata without word-alignment information. We further introduce a novel alignment\ninference procedure which applies stochastic gradient descent to directly\noptimize the attention activations towards a given target word. The resulting\nalignments dramatically outperform the naive approach to interpreting\nTransformer attention activations, and are comparable to Giza++ on two publicly\navailable data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11429v2",
            "date": "2019-01-31 15:30:37+00:00",
            "title": "Decomposing Generalization: Models of Generic, Habitual, and Episodic Statements",
            "authors": [
                "Venkata Subrahmanyan Govindarajan",
                "Benjamin Van Durme",
                "Aaron Steven White"
            ],
            "abstract": "We present a novel semantic framework for modeling linguistic expressions of\ngeneralization---generic, habitual, and episodic statements---as combinations\nof simple, real-valued referential properties of predicates and their\narguments. We use this framework to construct a dataset covering the entirety\nof the Universal Dependencies English Web Treebank. We use this dataset to\nprobe the efficacy of type-level and token-level information---including\nhand-engineered features and static (GloVe) and contextual (ELMo) word\nembeddings---for predicting expressions of generalization. Data and code are\navailable at decomp.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11504v2",
            "date": "2019-01-31 18:07:25+00:00",
            "title": "Multi-Task Deep Neural Networks for Natural Language Understanding",
            "authors": [
                "Xiaodong Liu",
                "Pengcheng He",
                "Weizhu Chen",
                "Jianfeng Gao"
            ],
            "abstract": "In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for\nlearning representations across multiple natural language understanding (NLU)\ntasks. MT-DNN not only leverages large amounts of cross-task data, but also\nbenefits from a regularization effect that leads to more general\nrepresentations in order to adapt to new tasks and domains. MT-DNN extends the\nmodel proposed in Liu et al. (2015) by incorporating a pre-trained\nbidirectional transformer language model, known as BERT (Devlin et al., 2018).\nMT-DNN obtains new state-of-the-art results on ten NLU tasks, including SNLI,\nSciTail, and eight out of nine GLUE tasks, pushing the GLUE benchmark to 82.7%\n(2.2% absolute improvement). We also demonstrate using the SNLI and SciTail\ndatasets that the representations learned by MT-DNN allow domain adaptation\nwith substantially fewer in-domain labels than the pre-trained BERT\nrepresentations. The code and pre-trained models are publicly available at\nhttps://github.com/namisan/mt-dnn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00041v1",
            "date": "2019-02-28 19:35:09+00:00",
            "title": "Reinforcement Learning based Curriculum Optimization for Neural Machine Translation",
            "authors": [
                "Gaurav Kumar",
                "George Foster",
                "Colin Cherry",
                "Maxim Krikun"
            ],
            "abstract": "We consider the problem of making efficient use of heterogeneous training\ndata in neural machine translation (NMT). Specifically, given a training\ndataset with a sentence-level feature such as noise, we seek an optimal\ncurriculum, or order for presenting examples to the system during training. Our\ncurriculum framework allows examples to appear an arbitrary number of times,\nand thus generalizes data weighting, filtering, and fine-tuning schemes. Rather\nthan relying on prior knowledge to design a curriculum, we use reinforcement\nlearning to learn one automatically, jointly with the NMT system, in the course\nof a single training run. We show that this approach can beat uniform and\nfiltering baselines on Paracrawl and WMT English-to-French datasets by up to\n+3.4 BLEU, and match the performance of a hand-designed, state-of-the-art\ncurriculum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00089v3",
            "date": "2019-02-28 22:26:12+00:00",
            "title": "Massively Multilingual Neural Machine Translation",
            "authors": [
                "Roee Aharoni",
                "Melvin Johnson",
                "Orhan Firat"
            ],
            "abstract": "Multilingual neural machine translation (NMT) enables training a single model\nthat supports translation from multiple source languages into multiple target\nlanguages. In this paper, we push the limits of multilingual NMT in terms of\nnumber of languages being used. We perform extensive experiments in training\nmassively multilingual NMT models, translating up to 102 languages to and from\nEnglish within a single model. We explore different setups for training such\nmodels and analyze the trade-offs between translation quality and various\nmodeling decisions. We report results on the publicly available TED talks\nmultilingual corpus where we show that massively multilingual many-to-many\nmodels are effective in low resource settings, outperforming the previous\nstate-of-the-art while supporting up to 59 languages. Our experiments on a\nlarge-scale dataset with 102 languages to and from English and up to one\nmillion examples per direction also show promising results, surpassing strong\nbilingual baselines and encouraging future work on massively multilingual NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00138v3",
            "date": "2019-03-01 03:08:03+00:00",
            "title": "Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data",
            "authors": [
                "Wei Zhao",
                "Liang Wang",
                "Kewei Shen",
                "Ruoyu Jia",
                "Jingming Liu"
            ],
            "abstract": "Neural machine translation systems have become state-of-the-art approaches\nfor Grammatical Error Correction (GEC) task. In this paper, we propose a\ncopy-augmented architecture for the GEC task by copying the unchanged words\nfrom the source sentence to the target sentence. Since the GEC suffers from not\nhaving enough labeled training data to achieve high accuracy. We pre-train the\ncopy-augmented architecture with a denoising auto-encoder using the unlabeled\nOne Billion Benchmark and make comparisons between the fully pre-trained model\nand a partially pre-trained model. It is the first time copying words from the\nsource context and fully pre-training a sequence to sequence model are\nexperimented on the GEC task. Moreover, We add token-level and sentence-level\nmulti-task learning for the GEC task. The evaluation results on the CoNLL-2014\ntest set show that our approach outperforms all recently published\nstate-of-the-art results by a large margin. The code and pre-trained models are\nreleased at https://github.com/zhawe01/fairseq-gec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00149v1",
            "date": "2019-03-01 03:57:15+00:00",
            "title": "Chinese-Japanese Unsupervised Neural Machine Translation Using Sub-character Level Information",
            "authors": [
                "Longtu Zhang",
                "Mamoru Komachi"
            ],
            "abstract": "Unsupervised neural machine translation (UNMT) requires only monolingual data\nof similar language pairs during training and can produce bi-directional\ntranslation models with relatively good performance on alphabetic languages\n(Lample et al., 2018). However, no research has been done to logographic\nlanguage pairs. This study focuses on Chinese-Japanese UNMT trained by data\ncontaining sub-character (ideograph or stroke) level information which is\ndecomposed from character level data. BLEU scores of both character and\nsub-character level systems were compared against each other and the results\nshowed that despite the effectiveness of UNMT on character level data,\nsub-character level data could further enhance the performance, in which the\nstroke level system outperformed the ideograph level system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00161v2",
            "date": "2019-03-01 05:32:01+00:00",
            "title": "DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs",
            "authors": [
                "Dheeru Dua",
                "Yizhong Wang",
                "Pradeep Dasigi",
                "Gabriel Stanovsky",
                "Sameer Singh",
                "Matt Gardner"
            ],
            "abstract": "Reading comprehension has recently seen rapid progress, with systems matching\nhumans on the most popular datasets for the task. However, a large body of work\nhas highlighted the brittleness of these systems, showing that there is much\nwork left to be done. We introduce a new English reading comprehension\nbenchmark, DROP, which requires Discrete Reasoning Over the content of\nParagraphs. In this crowdsourced, adversarially-created, 96k-question\nbenchmark, a system must resolve references in a question, perhaps to multiple\ninput positions, and perform discrete operations over them (such as addition,\ncounting, or sorting). These operations require a much more comprehensive\nunderstanding of the content of paragraphs than what was necessary for prior\ndatasets. We apply state-of-the-art methods from both the reading comprehension\nand semantic parsing literature on this dataset and show that the best systems\nonly achieve 32.7% F1 on our generalized accuracy metric, while expert human\nperformance is 96.0%. We additionally present a new model that combines reading\ncomprehension methods with simple numerical reasoning to achieve 47.0% F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00172v2",
            "date": "2019-03-01 06:26:50+00:00",
            "title": "Open Information Extraction from Question-Answer Pairs",
            "authors": [
                "Nikita Bhutani",
                "Yoshihiko Suhara",
                "Wang-Chiew Tan",
                "Alon Halevy",
                "H. V. Jagadish"
            ],
            "abstract": "Open Information Extraction (OpenIE) extracts meaningful structured tuples\nfrom free-form text. Most previous work on OpenIE considers extracting data\nfrom one sentence at a time. We describe NeurON, a system for extracting tuples\nfrom question-answer pairs. Since real questions and answers often contain\nprecisely the information that users care about, such information is\nparticularly desirable to extend a knowledge base with.\n  NeurON addresses several challenges. First, an answer text is often hard to\nunderstand without knowing the question, and second, relevant information can\nspan multiple sentences. To address these, NeurON formulates extraction as a\nmulti-source sequence-to-sequence learning task, wherein it combines\ndistributed representations of a question and an answer to generate knowledge\nfacts. We describe experiments on two real-world datasets that demonstrate that\nNeurON can find a significant number of new and interesting facts to extend a\nknowledge base compared to state-of-the-art OpenIE methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00665v2",
            "date": "2019-03-02 09:42:54+00:00",
            "title": "Towards NLP with Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks for Offensive Language Identification in Social Media",
            "authors": [
                "Andrei-Bogdan Puiu",
                "Andrei-Octavian Brabete"
            ],
            "abstract": "This short paper presents the design decisions taken and challenges\nencountered in completing SemEval Task 6, which poses the problem of\nidentifying and categorizing offensive language in tweets. Our proposed\nsolutions explore Deep Learning techniques, Linear Support Vector\nclassification and Random Forests to identify offensive tweets, to classify\noffenses as targeted or untargeted and eventually to identify the target\nsubject type.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00933v2",
            "date": "2019-03-03 16:07:10+00:00",
            "title": "Detecting dementia in Mandarin Chinese using transfer learning from a parallel corpus",
            "authors": [
                "Bai Li",
                "Yi-Te Hsu",
                "Frank Rudzicz"
            ],
            "abstract": "Machine learning has shown promise for automatic detection of Alzheimer's\ndisease (AD) through speech; however, efforts are hampered by a scarcity of\ndata, especially in languages other than English. We propose a method to learn\na correspondence between independently engineered lexicosyntactic features in\ntwo languages, using a large parallel corpus of out-of-domain movie dialogue\ndata. We apply it to dementia detection in Mandarin Chinese, and demonstrate\nthat our method outperforms both unilingual and machine translation-based\nbaselines. This appears to be the first study that transfers feature domains in\ndetecting cognitive decline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00943v2",
            "date": "2019-03-03 17:08:00+00:00",
            "title": "Structural Supervision Improves Learning of Non-Local Grammatical Dependencies",
            "authors": [
                "Ethan Wilcox",
                "Peng Qian",
                "Richard Futrell",
                "Miguel Ballesteros",
                "Roger Levy"
            ],
            "abstract": "State-of-the-art LSTM language models trained on large corpora learn\nsequential contingencies in impressive detail and have been shown to acquire a\nnumber of non-local grammatical dependencies with some success. Here we\ninvestigate whether supervision with hierarchical structure enhances learning\nof a range of grammatical dependencies, a question that has previously been\naddressed only for subject-verb agreement. Using controlled experimental\nmethods from psycholinguistics, we compare the performance of word-based LSTM\nmodels versus two models that represent hierarchical structure and deploy it in\nleft-to-right processing: Recurrent Neural Network Grammars (RNNGs) (Dyer et\nal., 2016) and a incrementalized version of the Parsing-as-Language-Modeling\nconfiguration from Chariak et al., (2016). Models are tested on a diverse range\nof configurations for two classes of non-local grammatical dependencies in\nEnglish---Negative Polarity licensing and Filler--Gap Dependencies. Using the\nsame training data across models, we find that structurally-supervised models\noutperform the LSTM, with the RNNG demonstrating best results on both types of\ngrammatical dependencies and even learning many of the Island Constraints on\nthe filler--gap dependency. Structural supervision thus provides data\nefficiency advantages over purely string-based training of neural language\nmodels in acquiring human-like generalizations about non-local grammatical\ndependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01039v4",
            "date": "2019-03-04 01:37:52+00:00",
            "title": "SECNLP: A Survey of Embeddings in Clinical Natural Language Processing",
            "authors": [
                "Kalyan KS",
                "S Sangeetha"
            ],
            "abstract": "Traditional representations like Bag of words are high dimensional, sparse\nand ignore the order as well as syntactic and semantic information. Distributed\nvector representations or embeddings map variable length text to dense fixed\nlength vectors as well as capture the prior knowledge which can transferred to\ndownstream tasks. Even though embedding has become de facto standard for\nrepresentations in deep learning based NLP tasks in both general and clinical\ndomains, there is no survey paper which presents a detailed review of\nembeddings in Clinical Natural Language Processing. In this survey paper, we\ndiscuss various medical corpora and their characteristics, medical codes and\npresent a brief overview as well as comparison of popular embeddings models. We\nclassify clinical embeddings into nine types and discuss each embedding type in\ndetail. We discuss various evaluation methods followed by possible solutions to\nvarious challenges in clinical embeddings. Finally, we conclude with some of\nthe future directions which will advance the research in clinical embeddings.",
            "journal ref": "Journal of Biomedical Informatics, Volume 101, January 2020,\n  103323",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01080v2",
            "date": "2019-03-04 05:38:29+00:00",
            "title": "From Knowledge Map to Mind Map: Artificial Imagination",
            "authors": [
                "Ruixue Liu",
                "Baoyang Chen",
                "Xiaoyu Guo",
                "Yan Dai",
                "Meng Chen",
                "Zhijie Qiu",
                "Xiaodong He"
            ],
            "abstract": "Imagination is one of the most important factors which makes an artistic\npainting unique and impressive. With the rapid development of Artificial\nIntelligence, more and more researchers try to create painting with AI\ntechnology automatically. However, lacking of imagination is still a main\nproblem for AI painting. In this paper, we propose a novel approach to inject\nrich imagination into a special painting art Mind Map creation. We firstly\nconsider lexical and phonological similarities of seed word, then learn and\ninherit original painting style of the author, and finally apply Dadaism and\nimpossibility of improvisation principles into painting process. We also design\nseveral metrics for imagination evaluation. Experimental results show that our\nproposed method can increase imagination of painting and also improve its\noverall quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01284v1",
            "date": "2019-03-04 14:46:20+00:00",
            "title": "Relation Extraction Datasets in the Digital Humanities Domain and their Evaluation with Word Embeddings",
            "authors": [
                "Gerhard Wohlgenannt",
                "Ekaterina Chernyak",
                "Dmitry Ilvovsky",
                "Ariadna Barinova",
                "Dmitry Mouromtsev"
            ],
            "abstract": "In this research, we manually create high-quality datasets in the digital\nhumanities domain for the evaluation of language models, specifically word\nembedding models. The first step comprises the creation of unigram and n-gram\ndatasets for two fantasy novel book series for two task types each, analogy and\ndoesn't-match. This is followed by the training of models on the two book\nseries with various popular word embedding model types such as word2vec, GloVe,\nfastText, or LexVec. Finally, we evaluate the suitability of word embedding\nmodels for such specific relation extraction tasks in a situation of comparably\nsmall corpus sizes. In the evaluations, we also investigate and analyze\nparticular aspects such as the impact of corpus term frequencies and task\ndifficulty on accuracy. The datasets, and the underlying system and word\nembedding models are available on github and can be easily extended with new\ndatasets and tasks, be used to reproduce the presented results, or be\ntransferred to other domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01411v1",
            "date": "2019-03-04 18:10:52+00:00",
            "title": "Polylingual Wordnet",
            "authors": [
                "Mihael Arcan",
                "John McCrae",
                "Paul Buitelaar"
            ],
            "abstract": "Princeton WordNet is one of the most important resources for natural language\nprocessing, but is only available for English. While it has been translated\nusing the expand approach to many other languages, this is an expensive manual\nprocess. Therefore it would be beneficial to have a high-quality automatic\ntranslation approach that would support NLP techniques, which rely on WordNet\nin new languages. The translation of wordnets is fundamentally complex because\nof the need to translate all senses of a word including low frequency senses,\nwhich is very challenging for current machine translation approaches. For this\nreason we leverage existing translations of WordNet in other languages to\nidentify contextual information for wordnet senses from a large set of generic\nparallel corpora. We evaluate our approach using 10 translated wordnets for\nEuropean languages. Our experiment shows a significant improvement over\ntranslation without any contextual information. Furthermore, we evaluate how\nthe choice of pivot languages affects performance of multilingual word sense\ndisambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01698v3",
            "date": "2019-03-05 06:56:12+00:00",
            "title": "Improving Cross-Domain Chinese Word Segmentation with Word Embeddings",
            "authors": [
                "Yuxiao Ye",
                "Yue Zhang",
                "Weikang Li",
                "Likun Qiu",
                "Jian Sun"
            ],
            "abstract": "Cross-domain Chinese Word Segmentation (CWS) remains a challenge despite\nrecent progress in neural-based CWS. The limited amount of annotated data in\nthe target domain has been the key obstacle to a satisfactory performance. In\nthis paper, we propose a semi-supervised word-based approach to improving\ncross-domain CWS given a baseline segmenter. Particularly, our model only\ndeploys word embeddings trained on raw text in the target domain, discarding\ncomplex hand-crafted features and domain-specific dictionaries. Innovative\nsubsampling and negative sampling methods are proposed to derive word\nembeddings optimized for CWS. We conduct experiments on five datasets in\nspecial domains, covering domains in novels, medicine, and patent. Results show\nthat our model can obviously improve cross-domain CWS, especially in the\nsegmentation of domain-specific noun entities. The word F-measure increases by\nover 3.0% on four datasets, outperforming state-of-the-art semi-supervised and\nunsupervised cross-domain CWS approaches with a large margin. We make our code\nand data available on Github.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01891v3",
            "date": "2019-03-05 15:17:18+00:00",
            "title": "Language and Dialect Identification of Cuneiform Texts",
            "authors": [
                "Tommi Jauhiainen",
                "Heidi Jauhiainen",
                "Tero Alstola",
                "Krister Lind\u00e9n"
            ],
            "abstract": "This article introduces a corpus of cuneiform texts from which the dataset\nfor the use of the Cuneiform Language Identification (CLI) 2019 shared task was\nderived as well as some preliminary language identification experiments\nconducted using that corpus. We also describe the CLI dataset and how it was\nderived from the corpus. In addition, we provide some baseline language\nidentification results using the CLI dataset. To the best of our knowledge, the\nexperiments detailed here are the first time automatic language identification\nmethods have been used on cuneiform data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02188v3",
            "date": "2019-03-06 06:09:02+00:00",
            "title": "Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases",
            "authors": [
                "Yu Chen",
                "Lingfei Wu",
                "Mohammed J. Zaki"
            ],
            "abstract": "When answering natural language questions over knowledge bases (KBs),\ndifferent question components and KB aspects play different roles. However,\nmost existing embedding-based methods for knowledge base question answering\n(KBQA) ignore the subtle inter-relationships between the question and the KB\n(e.g., entity types, relation paths and context). In this work, we propose to\ndirectly model the two-way flow of interactions between the questions and the\nKB via a novel Bidirectional Attentive Memory Network, called BAMnet. Requiring\nno external resources and only very few hand-crafted features, on the\nWebQuestions benchmark, our method significantly outperforms existing\ninformation-retrieval based methods, and remains competitive with\n(hand-crafted) semantic parsing based methods. Also, since we use attention\nmechanisms, our method offers better interpretability compared to other\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02230v3",
            "date": "2019-03-06 08:08:01+00:00",
            "title": "Dixit: Interactive Visual Storytelling via Term Manipulation",
            "authors": [
                "Chao-Chun Hsu",
                "Yu-Hua Chen",
                "Zi-Yuan Chen",
                "Hsin-Yu Lin",
                "Ting-Hao 'Kenneth' Huang",
                "Lun-Wei Ku"
            ],
            "abstract": "In this paper, we introduce Dixit, an interactive visual storytelling system\nthat the user interacts with iteratively to compose a short story for a photo\nsequence. The user initiates the process by uploading a sequence of photos.\nDixit first extracts text terms from each photo which describe the objects\n(e.g., boy, bike) or actions (e.g., sleep) in the photo, and then allows the\nuser to add new terms or remove existing terms. Dixit then generates a short\nstory based on these terms. Behind the scenes, Dixit uses an LSTM-based model\ntrained on image caption data and FrameNet to distill terms from each image and\nutilizes a transformer decoder to compose a context-coherent story. Users\nchange images or terms iteratively with Dixit to create the most ideal story.\nDixit also allows users to manually edit and rate stories. The proposed\nprocedure opens up possibilities for interpretable and controllable visual\nstorytelling, allowing users to understand the story formation rationale and to\nintervene in the generation process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02419v1",
            "date": "2019-03-06 14:38:28+00:00",
            "title": "KBQA: Learning Question Answering over QA Corpora and Knowledge Bases",
            "authors": [
                "Wanyun Cui",
                "Yanghua Xiao",
                "Haixun Wang",
                "Yangqiu Song",
                "Seung-won Hwang",
                "Wei Wang"
            ],
            "abstract": "Question answering (QA) has become a popular way for humans to access\nbillion-scale knowledge bases. Unlike web search, QA over a knowledge base\ngives out accurate and concise results, provided that natural language\nquestions can be understood and mapped precisely to structured queries over the\nknowledge base. The challenge, however, is that a human can ask one question in\nmany different ways. Previous approaches have natural limits due to their\nrepresentations: rule based approaches only understand a small set of \"canned\"\nquestions, while keyword based or synonym based approaches cannot fully\nunderstand the questions. In this paper, we design a new kind of question\nrepresentation: templates, over a billion scale knowledge base and a million\nscale QA corpora. For example, for questions about a city's population, we\nlearn templates such as What's the population of $city?, How many people are\nthere in $city?. We learned 27 million templates for 2782 intents. Based on\nthese templates, our QA system KBQA effectively supports binary factoid\nquestions, as well as complex questions which are composed of a series of\nbinary factoid questions. Furthermore, we expand predicates in RDF knowledge\nbase, which boosts the coverage of knowledge base by 57 times. Our QA system\nbeats all other state-of-art works on both effectiveness and efficiency over\nQALD benchmarks.",
            "journal ref": "Proceedings of the VLDB Endowment, Volume 10 Issue 5, January 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02588v3",
            "date": "2019-03-06 19:22:24+00:00",
            "title": "Sentence Embedding Alignment for Lifelong Relation Extraction",
            "authors": [
                "Hong Wang",
                "Wenhan Xiong",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Shiyu Chang",
                "William Yang Wang"
            ],
            "abstract": "Conventional approaches to relation extraction usually require a fixed set of\npre-defined relations. Such requirement is hard to meet in many real\napplications, especially when new data and relations are emerging incessantly\nand it is computationally expensive to store all data and re-train the whole\nmodel every time new data and relations come in. We formulate such a\nchallenging problem as lifelong relation extraction and investigate\nmemory-efficient incremental learning methods without catastrophically\nforgetting knowledge learned from previous tasks. We first investigate a\nmodified version of the stochastic gradient methods with a replay memory, which\nsurprisingly outperforms recent state-of-the-art lifelong learning methods. We\nfurther propose to improve this approach to alleviate the forgetting problem by\nanchoring the sentence embedding space. Specifically, we utilize an explicit\nalignment model to mitigate the sentence embedding distortion of the learned\nmodel when training on new data and new relations. Experiment results on\nmultiple benchmarks show that our proposed method significantly outperforms the\nstate-of-the-art lifelong learning approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02591v1",
            "date": "2019-03-06 19:42:19+00:00",
            "title": "Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing",
            "authors": [
                "Wenhan Xiong",
                "Jiawei Wu",
                "Deren Lei",
                "Mo Yu",
                "Shiyu Chang",
                "Xiaoxiao Guo",
                "William Yang Wang"
            ],
            "abstract": "Existing entity typing systems usually exploit the type hierarchy provided by\nknowledge base (KB) schema to model label correlations and thus improve the\noverall performance. Such techniques, however, are not directly applicable to\nmore open and practical scenarios where the type set is not restricted by KB\nschema and includes a vast number of free-form types. To model the underly-ing\nlabel correlations without access to manually annotated label structures, we\nintroduce a novel label-relational inductive bias, represented by a graph\npropagation layer that effectively encodes both global label co-occurrence\nstatistics and word-level similarities.On a large dataset with over 10,000\nfree-form types, the graph-enhanced model equipped with an attention-based\nmatching module is able to achieve a much higher recall score while maintaining\na high-level precision. Specifically, it achieves a 15.3% relative F1\nimprovement and also less inconsistency in the outputs. We further show that a\nsimple modification of our proposed graph layer can also improve the\nperformance on a conventional and widely-tested dataset that only includes\nKB-schema types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02652v1",
            "date": "2019-03-06 23:14:49+00:00",
            "title": "Multi-Instance Learning for End-to-End Knowledge Base Question Answering",
            "authors": [
                "Mengxi Wei",
                "Yifan He",
                "Qiong Zhang",
                "Luo Si"
            ],
            "abstract": "End-to-end training has been a popular approach for knowledge base question\nanswering (KBQA). However, real world applications often contain answers of\nvaried quality for users' questions. It is not appropriate to treat all\navailable answers of a user question equally.\n  This paper proposes a novel approach based on multiple instance learning to\naddress the problem of noisy answers by exploring consensus among answers to\nthe same question in training end-to-end KBQA models. In particular, the QA\npairs are organized into bags with dynamic instance selection and different\noptions of instance weighting. Curriculum learning is utilized to select\ninstance bags during training. On the public CQA dataset, the new method\nsignificantly improves both entity accuracy and the Rouge-L score over a\nstate-of-the-art end-to-end KBQA baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02671v1",
            "date": "2019-03-07 00:54:02+00:00",
            "title": "Creation and Evaluation of Datasets for Distributional Semantics Tasks in the Digital Humanities Domain",
            "authors": [
                "Gerhard Wohlgenannt",
                "Ariadna Barinova",
                "Dmitry Ilvovsky",
                "Ekaterina Chernyak"
            ],
            "abstract": "Word embeddings are already well studied in the general domain, usually\ntrained on large text corpora, and have been evaluated for example on word\nsimilarity and analogy tasks, but also as an input to downstream NLP processes.\nIn contrast, in this work we explore the suitability of word embedding\ntechnologies in the specialized digital humanities domain. After training\nembedding models of various types on two popular fantasy novel book series, we\nevaluate their performance on two task types: term analogies, and word\nintrusion. To this end, we manually construct test datasets with domain\nexperts. Among the contributions are the evaluation of various word embedding\ntechniques on the different task types, with the findings that even embeddings\ntrained on small corpora perform well for example on the word intrusion task.\nFurthermore, we provide extensive and high-quality datasets in digital\nhumanities for further investigation, as well as the implementation to easily\nreproduce or extend the experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02784v1",
            "date": "2019-03-07 09:22:35+00:00",
            "title": "Arabic natural language processing: An overview",
            "authors": [
                "Imane Guellil",
                "Houda Sa\u00e2dane",
                "Faical Azouaou",
                "Billel Gueni",
                "Damien Nouvel"
            ],
            "abstract": "Arabic is recognised as the 4th most used language of the Internet. Arabic\nhas three main varieties: (1) classical Arabic (CA), (2) Modern Standard Arabic\n(MSA), (3) Arabic Dialect (AD). MSA and AD could be written either in Arabic or\nin Roman script (Arabizi), which corresponds to Arabic written with Latin\nletters, numerals and punctuation. Due to the complexity of this language and\nthe number of corresponding challenges for NLP, many surveys have been\nconducted, in order to synthesise the work done on Arabic. However these\nsurveys principally focus on two varieties of Arabic (MSA and AD, written in\nArabic letters only), they are slightly old (no such survey since 2015) and\ntherefore do not cover recent resources and tools. To bridge the gap, we\npropose a survey focusing on 90 recent research papers (74% of which were\npublished after 2015). Our study presents and classifies the work done on the\nthree varieties of Arabic, by concentrating on both Arabic and Arabizi, and\nassociates each work to its publicly available resources whenever available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02852v1",
            "date": "2019-03-07 11:38:36+00:00",
            "title": "Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models",
            "authors": [
                "Thomas Drugman",
                "Janne Pylkkonen",
                "Reinhard Kneser"
            ],
            "abstract": "The goal of this paper is to simulate the benefits of jointly applying active\nlearning (AL) and semi-supervised training (SST) in a new speech recognition\napplication. Our data selection approach relies on confidence filtering, and\nits impact on both the acoustic and language models (AM and LM) is studied.\nWhile AL is known to be beneficial to AM training, we show that it also carries\nout substantial improvements to the LM when combined with SST. Sophisticated\nconfidence models, on the other hand, did not prove to yield any data selection\ngain. Our results indicate that, while SST is crucial at the beginning of the\nlabeling process, its gains degrade rapidly as AL is set in place. The final\nsimulation reports that AL allows a transcription cost reduction of about 70%\nover random selection. Alternatively, for a fixed transcription budget, the\nproposed approach improves the word error rate by about 12.5% relative.",
            "journal ref": "Interspeech, pp. 2318-2322, 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02930v1",
            "date": "2019-03-07 14:20:01+00:00",
            "title": "Neural Language Modeling with Visual Features",
            "authors": [
                "Antonios Anastasopoulos",
                "Shankar Kumar",
                "Hank Liao"
            ],
            "abstract": "Multimodal language models attempt to incorporate non-linguistic features for\nthe language modeling task. In this work, we extend a standard recurrent neural\nnetwork (RNN) language model with features derived from videos. We train our\nmodels on data that is two orders-of-magnitude bigger than datasets used in\nprior work. We perform a thorough exploration of model architectures for\ncombining visual and text features. Our experiments on two corpora (YouCookII\nand 20bn-something-something-v2) show that the best performing architecture\nconsists of middle fusion of visual and text features, yielding over 25%\nrelative improvement in perplexity. We report analysis that provides insights\ninto why our multimodal language model improves upon a standard RNN language\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02953v3",
            "date": "2019-03-06 16:55:58+00:00",
            "title": "SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA",
            "authors": [
                "Daniel Hershcovich",
                "Zohar Aizenbud",
                "Leshem Choshen",
                "Elior Sulem",
                "Ari Rappoport",
                "Omri Abend"
            ],
            "abstract": "We present the SemEval 2019 shared task on UCCA parsing in English, German\nand French, and discuss the participating systems and results. UCCA is a\ncross-linguistically applicable framework for semantic representation, which\nbuilds on extensive typological work and supports rapid annotation. UCCA poses\na challenge for existing parsing techniques, as it exhibits reentrancy\n(resulting in DAG structures), discontinuous structures and non-terminal nodes\ncorresponding to complex semantic units. The shared task has yielded\nimprovements over the state-of-the-art baseline in all languages and settings.\nFull results can be found in the task's website\n\\url{https://competitions.codalab.org/competitions/19160}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03033v1",
            "date": "2019-03-07 16:43:50+00:00",
            "title": "Option Comparison Network for Multiple-choice Reading Comprehension",
            "authors": [
                "Qiu Ran",
                "Peng Li",
                "Weiwei Hu",
                "Jie Zhou"
            ],
            "abstract": "Multiple-choice reading comprehension (MCRC) is the task of selecting the\ncorrect answer from multiple options given a question and an article. Existing\nMCRC models typically either read each option independently or compute a\nfixed-length representation for each option before comparing them. However,\nhumans typically compare the options at multiple-granularity level before\nreading the article in detail to make reasoning more efficient. Mimicking\nhumans, we propose an option comparison network (OCN) for MCRC which compares\noptions at word-level to better identify their correlations to help reasoning.\nSpecially, each option is encoded into a vector sequence using a skimmer to\nretain fine-grained information as much as possible. An attention mechanism is\nleveraged to compare these sequences vector-by-vector to identify more subtle\ncorrelations between options, which is potentially valuable for reasoning.\nExperimental results on the human English exam MCRC dataset RACE show that our\nmodel outperforms existing methods significantly. Moreover, it is also the\nfirst model that surpasses Amazon Mechanical Turker performance on the whole\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03243v2",
            "date": "2019-03-08 01:46:37+00:00",
            "title": "Context-Aware Cross-Lingual Mapping",
            "authors": [
                "Hanan Aldarmaki",
                "Mona Diab"
            ],
            "abstract": "Cross-lingual word vectors are typically obtained by fitting an orthogonal\nmatrix that maps the entries of a bilingual dictionary from a source to a\ntarget vector space. Word vectors, however, are most commonly used for sentence\nor document-level representations that are calculated as the weighted average\nof word embeddings. In this paper, we propose an alternative to word-level\nmapping that better reflects sentence-level cross-lingual similarity. We\nincorporate context in the transformation matrix by directly mapping the\naveraged embeddings of aligned sentences in a parallel corpus. We also\nimplement cross-lingual mapping of deep contextualized word embeddings using\nparallel sentences with word alignments. In our experiments, both approaches\nresulted in cross-lingual sentence embeddings that outperformed\ncontext-independent word mapping in sentence translation retrieval.\nFurthermore, the sentence-level transformation could be used for word-level\nmapping without loss in word translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03260v1",
            "date": "2019-03-08 03:02:28+00:00",
            "title": "Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State",
            "authors": [
                "Richard Futrell",
                "Ethan Wilcox",
                "Takashi Morita",
                "Peng Qian",
                "Miguel Ballesteros",
                "Roger Levy"
            ],
            "abstract": "We deploy the methods of controlled psycholinguistic experimentation to shed\nlight on the extent to which the behavior of neural network language models\nreflects incremental representations of syntactic state. To do so, we examine\nmodel behavior on artificial sentences containing a variety of syntactically\ncomplex structures. We test four models: two publicly available LSTM sequence\nmodels of English (Jozefowicz et al., 2016; Gulordava et al., 2018) trained on\nlarge datasets; an RNNG (Dyer et al., 2016) trained on a small, parsed dataset;\nand an LSTM trained on the same small corpus as the RNNG. We find evidence that\nthe LSTMs trained on large datasets represent syntactic state over large spans\nof text in a way that is comparable to the RNNG, while the LSTM trained on the\nsmall dataset does not or does so only weakly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03467v1",
            "date": "2019-03-08 14:33:34+00:00",
            "title": "Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection",
            "authors": [
                "Amit Moryossef",
                "Roee Aharoni",
                "Yoav Goldberg"
            ],
            "abstract": "When translating from a language that does not morphologically mark\ninformation such as gender and number into a language that does, translation\nsystems must \"guess\" this missing information, often leading to incorrect\ntranslations in the given context. We propose a black-box approach for\ninjecting the missing information to a pre-trained neural machine translation\nsystem, allowing to control the morphological variations in the generated\ntranslations without changing the underlying model or training data. We\nevaluate our method on an English to Hebrew translation task, and show that it\nis effective in injecting the gender and number information and that supplying\nthe correct information improves the translation accuracy in up to 2.3 BLEU on\na female-speaker test set for a state-of-the-art online black-box system.\nFinally, we perform a fine-grained syntactic analysis of the generated\ntranslations that shows the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03530v2",
            "date": "2019-03-08 16:20:42+00:00",
            "title": "Fast Prototyping a Dialogue Comprehension System for Nurse-Patient Conversations on Symptom Monitoring",
            "authors": [
                "Zhengyuan Liu",
                "Hazel Lim",
                "Nur Farah Ain Binte Suhaimi",
                "Shao Chuen Tong",
                "Sharon Ong",
                "Angela Ng",
                "Sheldon Lee",
                "Michael R. Macdonald",
                "Savitha Ramasamy",
                "Pavitra Krishnaswamy",
                "Wai Leng Chow",
                "Nancy F. Chen"
            ],
            "abstract": "Data for human-human spoken dialogues for research and development are\ncurrently very limited in quantity, variety, and sources; such data are even\nscarcer in healthcare. In this work, we investigate fast prototyping of a\ndialogue comprehension system by leveraging on minimal nurse-to-patient\nconversations. We propose a framework inspired by nurse-initiated clinical\nsymptom monitoring conversations to construct a simulated human-human dialogue\ndataset, embodying linguistic characteristics of spoken interactions like\nthinking aloud, self-contradiction, and topic drift. We then adopt an\nestablished bidirectional attention pointer network on this simulated dataset,\nachieving more than 80% F1 score on a held-out test set from real-world\nnurse-to-patient conversations. The ability to automatically comprehend\nconversations in the healthcare domain by exploiting only limited data has\nimplications for improving clinical workflows through red flag symptom\ndetection and triaging capabilities. We demonstrate the feasibility for\nefficient and effective extraction, retrieval and comprehension of symptom\nchecking information discussed in multi-turn human-human spoken conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03862v2",
            "date": "2019-03-09 19:56:47+00:00",
            "title": "Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them",
            "authors": [
                "Hila Gonen",
                "Yoav Goldberg"
            ],
            "abstract": "Word embeddings are widely used in NLP for a vast range of tasks. It was\nshown that word embeddings derived from text corpora reflect gender biases in\nsociety. This phenomenon is pervasive and consistent across different word\nembedding models, causing serious concern. Several recent works tackle this\nproblem, and propose methods for significantly reducing this gender bias in\nword embeddings, demonstrating convincing results. However, we argue that this\nremoval is superficial. While the bias is indeed substantially reduced\naccording to the provided bias definition, the actual effect is mostly hiding\nthe bias, not removing it. The gender bias information is still reflected in\nthe distances between \"gender-neutralized\" words in the debiased embeddings,\nand can be recovered from them. We present a series of experiments to support\nthis claim, for two debiasing methods. We conclude that existing bias removal\ntechniques are insufficient, and should not be trusted for providing\ngender-neutral modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04153v2",
            "date": "2019-03-11 07:46:03+00:00",
            "title": "HLT@SUDA at SemEval 2019 Task 1: UCCA Graph Parsing as Constituent Tree Parsing",
            "authors": [
                "Wei Jiang",
                "Zhenghua Li",
                "Yu Zhang",
                "Min Zhang"
            ],
            "abstract": "This paper describes a simple UCCA semantic graph parsing approach. The key\nidea is to convert a UCCA semantic graph into a constituent tree, in which\nextra labels are deliberately designed to mark remote edges and discontinuous\nnodes for future recovery. In this way, we can make use of existing syntactic\nparsing techniques. Based on the data statistics, we recover discontinuous\nnodes directly according to the output labels of the constituent parser and use\na biaffine classification model to recover the more complex remote edges. The\nclassification model and the constituent parser are simultaneously trained\nunder the multi-task learning framework. We use the multilingual BERT as extra\nfeatures in the open tracks. Our system ranks the first place in the six\nEnglish/German closed/open tracks among seven participating systems. For the\nseventh cross-lingual track, where there is little training data for French, we\npropose a language embedding approach to utilize English and German training\ndata, and our result ranks the second place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04167v2",
            "date": "2019-03-11 08:20:13+00:00",
            "title": "Partially Shuffling the Training Data to Improve Language Models",
            "authors": [
                "Ofir Press"
            ],
            "abstract": "Although SGD requires shuffling the training data between epochs, currently\nnone of the word-level language modeling systems do this. Naively shuffling all\nsentences in the training data would not permit the model to learn\ninter-sentence dependencies. Here we present a method that partially shuffles\nthe training data between epochs. This method makes each batch random, while\nkeeping most sentence ordering intact. It achieves new state of the art results\non word-level language modeling on both the Penn Treebank and WikiText-2\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04190v2",
            "date": "2019-03-11 09:48:39+00:00",
            "title": "Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning",
            "authors": [
                "Weipeng Huang",
                "Xingyi Cheng",
                "Kunlong Chen",
                "Taifeng Wang",
                "Wei Chu"
            ],
            "abstract": "The ambiguous annotation criteria lead to divergence of Chinese Word\nSegmentation (CWS) datasets in various granularities. Multi-criteria Chinese\nword segmentation aims to capture various annotation criteria among datasets\nand leverage their common underlying knowledge. In this paper, we propose a\ndomain adaptive segmenter to exploit diverse criteria of various datasets. Our\nmodel is based on Bidirectional Encoder Representations from Transformers\n(BERT), which is responsible for introducing open-domain knowledge. Private and\nshared projection layers are proposed to capture domain-specific knowledge and\ncommon knowledge, respectively. We also optimize computational efficiency via\ndistillation, quantization, and compiler optimization. Experiments show that\nour segmenter outperforms the previous state of the art (SOTA) models on 10 CWS\ndatasets with superior efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04433v2",
            "date": "2019-03-11 16:48:25+00:00",
            "title": "ETNLP: a visual-aided systematic approach to select pre-trained embeddings for a downstream task",
            "authors": [
                "Xuan-Son Vu",
                "Thanh Vu",
                "Son N. Tran",
                "Lili Jiang"
            ],
            "abstract": "Given many recent advanced embedding models, selecting pre-trained word\nembedding (a.k.a., word representation) models best fit for a specific\ndownstream task is non-trivial. In this paper, we propose a systematic\napproach, called ETNLP, for extracting, evaluating, and visualizing multiple\nsets of pre-trained word embeddings to determine which embeddings should be\nused in a downstream task. For extraction, we provide a method to extract\nsubsets of the embeddings to be used in the downstream task. For evaluation, we\nanalyse the quality of pre-trained embeddings using an input word analogy list.\nFinally, we visualize the word representations in the embedding space to\nexplore the embedded words interactively.\n  We demonstrate the effectiveness of the proposed approach on our pre-trained\nword embedding models in Vietnamese to select which models are suitable for a\nnamed entity recognition (NER) task. Specifically, we create a large Vietnamese\nword analogy list to evaluate and select the pre-trained embedding models for\nthe task. We then utilize the selected embeddings for the NER task and achieve\nthe new state-of-the-art results on the task benchmark dataset. We also apply\nthe approach to another downstream task of privacy-guaranteed embedding\nselection, and show that it helps users quickly select the most suitable\nembeddings. In addition, we create an open-source system using the proposed\nsystematic approach to facilitate similar studies on other NLP tasks. The\nsource code and data are available at https://github.com/vietnlp/etnlp.",
            "journal ref": "Proceedings of the International Conference Recent Advances in\n  Natural Language Processing (RANLP), 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04484v1",
            "date": "2019-03-11 18:11:23+00:00",
            "title": "The Truth and Nothing but the Truth: Multimodal Analysis for Deception Detection",
            "authors": [
                "Mimansa Jaiswal",
                "Sairam Tabibu",
                "Rajiv Bajpai"
            ],
            "abstract": "We propose a data-driven method for automatic deception detection in\nreal-life trial data using visual and verbal cues. Using OpenFace with facial\naction unit recognition, we analyze the movement of facial features of the\nwitness when posed with questions and the acoustic patterns using OpenSmile. We\nthen perform a lexical analysis on the spoken words, emphasizing the use of\npauses and utterance breaks, feeding that to a Support Vector Machine to test\ndeceit or truth prediction. We then try out a method to incorporate\nutterance-based fusion of visual and lexical analysis, using string based\nmatching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04521v3",
            "date": "2019-03-11 18:12:28+00:00",
            "title": "Practical Semantic Parsing for Spoken Language Understanding",
            "authors": [
                "Marco Damonte",
                "Rahul Goel",
                "Tagyoung Chung"
            ],
            "abstract": "Executable semantic parsing is the task of converting natural language\nutterances into logical forms that can be directly used as queries to get a\nresponse. We build a transfer learning framework for executable semantic\nparsing. We show that the framework is effective for Question Answering (Q&A)\nas well as for Spoken Language Understanding (SLU). We further investigate the\ncase where a parser on a new domain can be learned by exploiting data on other\ndomains, either via multi-task learning between the target domain and an\nauxiliary domain or via pre-training on the auxiliary domain and fine-tuning on\nthe target domain. With either flavor of transfer learning, we are able to\nimprove performance on most domains; we experiment with public data sets such\nas Overnight and NLmaps as well as with commercial SLU data. The experiments\ncarried out on data sets that are different in nature show how executable\nsemantic parsing can unify different areas of NLP such as Q&A and SLU.",
            "journal ref": "Proceedings of NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04715v1",
            "date": "2019-03-12 03:55:13+00:00",
            "title": "Context-Aware Learning for Neural Machine Translation",
            "authors": [
                "S\u00e9bastien Jean",
                "Kyunghyun Cho"
            ],
            "abstract": "Interest in larger-context neural machine translation, including\ndocument-level and multi-modal translation, has been growing. Multiple works\nhave proposed new network architectures or evaluation schemes, but potentially\nhelpful context is still sometimes ignored by larger-context translation\nmodels. In this paper, we propose a novel learning algorithm that explicitly\nencourages a neural translation model to take into account additional context\nusing a multilevel pair-wise ranking loss. We evaluate the proposed learning\nalgorithm with a transformer-based larger-context translation system on\ndocument-level translation. By comparing performance using actual and random\ncontexts, we show that a model trained with the proposed algorithm is more\nsensitive to the additional context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04739v1",
            "date": "2019-03-12 05:52:41+00:00",
            "title": "Syllable-based Neural Named Entity Recognition for Myanmar Language",
            "authors": [
                "Hsu Myat Mo",
                "Khin Mar Soe"
            ],
            "abstract": "Named Entity Recognition (NER) for Myanmar Language is essential to Myanmar\nnatural language processing research work. In this work, NER for Myanmar\nlanguage is treated as a sequence tagging problem and the effectiveness of deep\nneural networks on NER for Myanmar language has been investigated. Experiments\nare performed by applying deep neural network architectures on syllable level\nMyanmar contexts. Very first manually annotated NER corpus for Myanmar language\nis also constructed and proposed. In developing our in-house NER corpus,\nsentences from online news website and also sentences supported from\nALT-Parallel-Corpus are also used. This ALT corpus is one part of the Asian\nLanguage Treebank (ALT) project under ASEAN IVO. This paper contributes the\nfirst evaluation of neural network models on NER task for Myanmar language. The\nexperimental results show that those neural sequence models can produce\npromising results compared to the baseline CRF model. Among those neural\narchitectures, bidirectional LSTM network added CRF layer above gives the\nhighest F-score value. This work also aims to discover the effectiveness of\nneural network approaches to Myanmar textual processing as well as to promote\nfurther researches on this understudied language.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol.8,\n  No.1, February 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05041v1",
            "date": "2019-03-12 16:42:39+00:00",
            "title": "Character Eyes: Seeing Language through Character-Level Taggers",
            "authors": [
                "Yuval Pinter",
                "Marc Marone",
                "Jacob Eisenstein"
            ],
            "abstract": "Character-level models have been used extensively in recent years in NLP\ntasks as both supplements and replacements for closed-vocabulary token-level\nword representations. In one popular architecture, character-level LSTMs are\nused to feed token representations into a sequence tagger predicting\ntoken-level annotations such as part-of-speech (POS) tags. In this work, we\nexamine the behavior of POS taggers across languages from the perspective of\nindividual hidden units within the character LSTM. We aggregate the behavior of\nthese units into language-level metrics which quantify the challenges that\ntaggers face on languages with different morphological properties, and identify\nlinks between synthesis and affixation preference and emergent behavior of the\nhidden tagger layer. In a comparative experiment, we show how modifying the\nbalance between forward and backward hidden units affects model arrangement and\nperformance in these types of languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05164v3",
            "date": "2019-03-12 19:26:39+00:00",
            "title": "Scaling Multi-Domain Dialogue State Tracking via Query Reformulation",
            "authors": [
                "Pushpendre Rastogi",
                "Arpit Gupta",
                "Tongfei Chen",
                "Lambert Mathias"
            ],
            "abstract": "We present a novel approach to dialogue state tracking and referring\nexpression resolution tasks. Successful contextual understanding of multi-turn\nspoken dialogues requires resolving referring expressions across turns and\ntracking the entities relevant to the conversation across turns. Tracking\nconversational state is particularly challenging in a multi-domain scenario\nwhen there exist multiple spoken language understanding (SLU) sub-systems, and\neach SLU sub-system operates on its domain-specific meaning representation.\nWhile previous approaches have addressed the disparate schema issue by learning\ncandidate transformations of the meaning representation, in this paper, we\ninstead model the reference resolution as a dialogue context-aware user query\nreformulation task -- the dialog state is serialized to a sequence of natural\nlanguage tokens representing the conversation. We develop our model for query\nreformulation using a pointer-generator network and a novel multi-task learning\nsetup. In our experiments, we show a significant improvement in absolute F1 on\nan internal as well as a, soon to be released, public benchmark respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05225v1",
            "date": "2019-03-12 21:24:25+00:00",
            "title": "Bootstrapping Method for Developing Part-of-Speech Tagged Corpus in Low Resource Languages Tagset - A Focus on an African Igbo",
            "authors": [
                "Onyenwe Ikechukwu E",
                "Onyedinma Ebele G",
                "Aniegwu Godwin E",
                "Ezeani Ignatius M"
            ],
            "abstract": "Most languages, especially in Africa, have fewer or no established\npart-of-speech (POS) tagged corpus. However, POS tagged corpus is essential for\nnatural language processing (NLP) to support advanced researches such as\nmachine translation, speech recognition, etc. Even in cases where there is no\nPOS tagged corpus, there are some languages for which parallel texts are\navailable online. The task of POS tagging a new language corpus with a new\ntagset usually face a bootstrapping problem at the initial stages of the\nannotation process. The unavailability of automatic taggers to help the human\nannotator makes the annotation process to appear infeasible to quickly produce\nadequate amounts of POS tagged corpus for advanced NLP research and training\nthe taggers. In this paper, we demonstrate the efficacy of a POS annotation\nmethod that employed the services of two automatic approaches to assist POS\ntagged corpus creation for a novel language in NLP. The two approaches are\ncross-lingual and monolingual POS tags projection. We used cross-lingual to\nautomatically create an initial 'errorful' tagged corpus for a target language\nvia word-alignment. The resources for creating this are derived from a source\nlanguage rich in NLP resources. A monolingual method is applied to clean the\ninduce noise via an alignment process and to transform the source language tags\nto the target language tags. We used English and Igbo as our case study. This\nis possible because there are parallel texts that exist between English and\nIgbo, and the source language English has available NLP resources. The results\nof the experiment show a steady improvement in accuracy and rate of tags\ntransformation with score ranges of 6.13% to 83.79% and 8.67% to 98.37%\nrespectively. The rate of tags transformation evaluates the rate at which\nsource language tags are translated to target language tags.",
            "journal ref": "International Journal on Natural Language Computing (IJNLC) Vol\n  8(1) (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05260v2",
            "date": "2019-03-12 23:35:32+00:00",
            "title": "Syntax-aware Neural Semantic Role Labeling with Supertags",
            "authors": [
                "Jungo Kasai",
                "Dan Friedman",
                "Robert Frank",
                "Dragomir Radev",
                "Owen Rambow"
            ],
            "abstract": "We introduce a new syntax-aware model for dependency-based semantic role\nlabeling that outperforms syntax-agnostic models for English and Spanish. We\nuse a BiLSTM to tag the text with supertags extracted from dependency parses,\nand we feed these supertags, along with words and parts of speech, into a deep\nhighway BiLSTM for semantic role labeling. Our model combines the strengths of\nearlier models that performed SRL on the basis of a full dependency parse with\nmore recent models that use no syntactic information at all. Our local and\nnon-ensemble model achieves state-of-the-art performance on the CoNLL 09\nEnglish and Spanish datasets. SRL models benefit from syntactic information,\nand we show that supertagging is a simple, powerful, and robust way to\nincorporate syntax into a neural SRL system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05261v1",
            "date": "2019-03-12 23:40:27+00:00",
            "title": "End-To-End Speech Recognition Using A High Rank LSTM-CTC Based Model",
            "authors": [
                "Yangyang Shi",
                "Mei-Yuh Hwang",
                "Xin Lei"
            ],
            "abstract": "Long Short Term Memory Connectionist Temporal Classification (LSTM-CTC) based\nend-to-end models are widely used in speech recognition due to its simplicity\nin training and efficiency in decoding. In conventional LSTM-CTC based models,\na bottleneck projection matrix maps the hidden feature vectors obtained from\nLSTM to softmax output layer. In this paper, we propose to use a high rank\nprojection layer to replace the projection matrix. The output from the high\nrank projection layer is a weighted combination of vectors that are projected\nfrom the hidden feature vectors via different projection matrices and\nnon-linear activation function. The high rank projection layer is able to\nimprove the expressiveness of LSTM-CTC models. The experimental results show\nthat on Wall Street Journal (WSJ) corpus and LibriSpeech data set, the proposed\nmethod achieves 4%-6% relative word error rate (WER) reduction over the\nbaseline CTC system. They outperform other published CTC based end-to-end (E2E)\nmodels under the condition that no external data or data augmentation is\napplied. Code has been made available at https://github.com/mobvoi/lstm_ctc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05396v1",
            "date": "2019-03-13 10:23:38+00:00",
            "title": "Sub-event detection from Twitter streams as a sequence labeling problem",
            "authors": [
                "Giannis Bekoulis",
                "Johannes Deleu",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "This paper introduces improved methods for sub-event detection in social\nmedia streams, by applying neural sequence models not only on the level of\nindividual posts, but also directly on the stream level. Current approaches to\nidentify sub-events within a given event, such as a goal during a soccer match,\nessentially do not exploit the sequential nature of social media streams. We\naddress this shortcoming by framing the sub-event detection problem in social\nmedia streams as a sequence labeling task and adopt a neural sequence\narchitecture that explicitly accounts for the chronological order of posts.\nSpecifically, we (i) establish a neural baseline that outperforms a graph-based\nstate-of-the-art method for binary sub-event detection (2.7% micro-F1\nimprovement), as well as (ii) demonstrate superiority of a recurrent neural\nnetwork model on the posts sequence level for labeled sub-events (2.4%\nbin-level F1 improvement over non-sequential models).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05498v1",
            "date": "2019-03-13 14:03:50+00:00",
            "title": "Overview of the Ugglan Entity Discovery and Linking System",
            "authors": [
                "Marcus Klang",
                "Firas Dib",
                "Pierre Nugues"
            ],
            "abstract": "Ugglan is a system designed to discover named entities and link them to\nunique identifiers in a knowledge base. It is based on a combination of a name\nand nominal dictionary derived from Wikipedia and Wikidata, a named entity\nrecognition module (NER) using fixed ordinally-forgetting encoding (FOFE)\ntrained on the TAC EDL data from 2014-2016, a candidate generation module from\nthe Wikipedia link graph across multiple editions, a PageRank link and\ncooccurrence graph disambiguator, and finally a reranker trained on the TAC EDL\n2015-2016 data.",
            "journal ref": "Proceedings of the Tenth Text Analysis Conference (TAC 2017)\n  November 13-14, 2017, National Institute of Standards and Technology,\n  Gaithersburg, Maryland, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05683v1",
            "date": "2019-03-13 19:01:00+00:00",
            "title": "Low-Resource Syntactic Transfer with Unsupervised Source Reordering",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Michael Collins"
            ],
            "abstract": "We describe a cross-lingual transfer method for dependency parsing that takes\ninto account the problem of word order differences between source and target\nlanguages. Our model only relies on the Bible, a considerably smaller parallel\ndata than the commonly used parallel data in transfer methods. We use the\nconcatenation of projected trees from the Bible corpus, and the gold-standard\ntreebanks in multiple source languages along with cross-lingual word\nrepresentations. We demonstrate that reordering the source treebanks before\ntraining on them for a target language improves the accuracy of languages\noutside the European language family. Our experiments on 68 treebanks (38\nlanguages) in the Universal Dependencies corpus achieve a high accuracy for all\nlanguages. Among them, our experiments on 16 treebanks of 12 non-European\nlanguages achieve an average UAS absolute improvement of 3.3% over a\nstate-of-the-art method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05759v3",
            "date": "2019-03-13 23:45:31+00:00",
            "title": "Consistent Dialogue Generation with Self-supervised Feature Learning",
            "authors": [
                "Yizhe Zhang",
                "Xiang Gao",
                "Sungjin Lee",
                "Chris Brockett",
                "Michel Galley",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "Generating responses that are consistent with the dialogue context is one of\nthe central challenges in building engaging conversational agents. In this\npaper, we propose a neural conversation model that generates consistent\nresponses by maintaining certain features related to topics and personas\nthroughout the conversation. Unlike past work that requires external\nsupervision such as user identities, which are often unavailable or classified\nas sensitive information, our approach trains topic and persona feature\nextractors in a self-supervised way by utilizing the natural structure of\ndialogue data. Moreover, we adopt a binary feature representation and introduce\na feature disentangling loss which, paired with controllable response\ngeneration techniques, allows us to promote or demote certain learned topics\nand personas features. The evaluation result demonstrates the model's\ncapability of capturing meaningful topics and personas features, and the\nincorporation of the learned features brings significant improvement in terms\nof the quality of generated responses on two datasets, even comparing with\nmodel which explicit persona information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05929v3",
            "date": "2019-03-14 11:59:01+00:00",
            "title": "Absit invidia verbo: Comparing Deep Learning methods for offensive language",
            "authors": [
                "Silvia Sapora",
                "Bogdan Lazarescu",
                "Christo Lolov"
            ],
            "abstract": "This document describes our approach to building an Offensive Language\nClassifier. More specifically, the OffensEval 2019 competition required us to\nbuild three classifiers with slightly different goals:\n  - Offensive language identification: would classify a tweet as offensive or\nnot.\n  - Automatic categorization of offense types: would recognize if the target of\nthe offense was an individual or not.\n  - Offense target identification: would identify the target of the offense\nbetween an individual, group or other.\n  In this report, we will discuss the different architectures, algorithms and\npre-processing strategies we tried, together with a detailed description of the\ndesigns of our final classifiers and the reasons we choose them over others.\n  We evaluated our classifiers on the official test set provided for the\nOffenseEval 2019 competition, obtaining a macro-averaged F1-score of 0.7189 for\nTask A, 0.6708 on Task B and 0.5442 on Task C.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06353v1",
            "date": "2019-03-15 04:08:49+00:00",
            "title": "Formality Style Transfer with Hybrid Textual Annotations",
            "authors": [
                "Ruochen Xu",
                "Tao Ge",
                "Furu Wei"
            ],
            "abstract": "Formality style transformation is the task of modifying the formality of a\ngiven sentence without changing its content. Its challenge is the lack of\nlarge-scale sentence-aligned parallel data. In this paper, we propose an\nomnivorous model that takes parallel data and formality-classified data jointly\nto alleviate the data sparsity issue. We empirically demonstrate the\neffectiveness of our approach by achieving the state-of-art performance on a\nrecently proposed benchmark dataset of formality transfer. Furthermore, our\nmodel can be readily adapted to other unsupervised text style transfer tasks\nlike unsupervised sentiment transfer and achieve competitive results on three\nwidely recognized benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06400v2",
            "date": "2019-03-15 08:06:03+00:00",
            "title": "Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages",
            "authors": [
                "Shauli Ravfogel",
                "Yoav Goldberg",
                "Tal Linzen"
            ],
            "abstract": "How do typological properties such as word order and morphological case\nmarking affect the ability of neural sequence models to acquire the syntax of a\nlanguage? Cross-linguistic comparisons of RNNs' syntactic performance (e.g., on\nsubject-verb agreement prediction) are complicated by the fact that any two\nlanguages differ in multiple typological properties, as well as by differences\nin training corpus. We propose a paradigm that addresses these issues: we\ncreate synthetic versions of English, which differ from English in one or more\ntypological parameters, and generate corpora for those languages based on a\nparsed English corpus. We report a series of experiments in which RNNs were\ntrained to predict agreement features for verbs in each of those synthetic\nlanguages. Among other findings, (1) performance was higher in\nsubject-verb-object order (as in English) than in subject-object-verb order (as\nin Japanese), suggesting that RNNs have a recency bias; (2) predicting\nagreement with both subject and object (polypersonal agreement) improves over\npredicting each separately, suggesting that underlying syntactic knowledge\ntransfers across the two tasks; and (3) overt morphological case makes\nagreement prediction significantly easier, regardless of word order.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06409v1",
            "date": "2019-03-15 08:48:20+00:00",
            "title": "Automatic assessment of spoken language proficiency of non-native children",
            "authors": [
                "Roberto Gretter",
                "Katharina Allgaier",
                "Svetlana Tchistiakova",
                "Daniele Falavigna"
            ],
            "abstract": "This paper describes technology developed to automatically grade Italian\nstudents (ages 9-16) on their English and German spoken language proficiency.\nThe students' spoken answers are first transcribed by an automatic speech\nrecognition (ASR) system and then scored using a feedforward neural network\n(NN) that processes features extracted from the automatic transcriptions.\nIn-domain acoustic models, employing deep neural networks (DNNs), are derived\nby adapting the parameters of an original out of domain DNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06494v5",
            "date": "2019-03-15 12:44:36+00:00",
            "title": "Content Differences in Syntactic and Semantic Representations",
            "authors": [
                "Daniel Hershcovich",
                "Omri Abend",
                "Ari Rappoport"
            ],
            "abstract": "Syntactic analysis plays an important role in semantic parsing, but the\nnature of this role remains a topic of ongoing debate. The debate has been\nconstrained by the scarcity of empirical comparative studies between syntactic\nand semantic schemes, which hinders the development of parsing methods informed\nby the details of target schemes and constructions. We target this gap, and\ntake Universal Dependencies (UD) and UCCA as a test case. After abstracting\naway from differences of convention or formalism, we find that most content\ndivergences can be ascribed to: (1) UCCA's distinction between a Scene and a\nnon-Scene; (2) UCCA's distinction between primary relations, secondary ones and\nparticipants; (3) different treatment of multi-word expressions, and (4)\ndifferent treatment of inter-clause linkage. We further discuss the long tail\nof cases where the two schemes take markedly different approaches. Finally, we\nshow that the proposed comparison methodology can be used for fine-grained\nevaluation of UCCA parsing, highlighting both challenges and potential sources\nfor improvement. The substantial differences between the schemes suggest that\nsemantic parsers are likely to benefit downstream text understanding\napplications beyond their syntactic counterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06620v2",
            "date": "2019-03-15 16:04:11+00:00",
            "title": "On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models",
            "authors": [
                "Paul Michel",
                "Xian Li",
                "Graham Neubig",
                "Juan Miguel Pino"
            ],
            "abstract": "Adversarial examples --- perturbations to the input of a model that elicit\nlarge changes in the output --- have been shown to be an effective way of\nassessing the robustness of sequence-to-sequence (seq2seq) models. However,\nthese perturbations only indicate weaknesses in the model if they do not change\nthe input so significantly that it legitimately results in changes in the\nexpected output. This fact has largely been ignored in the evaluations of the\ngrowing body of related literature. Using the example of untargeted attacks on\nmachine translation (MT), we propose a new evaluation framework for adversarial\nattacks on seq2seq models that takes the semantic equivalence of the pre- and\npost-perturbation input into account. Using this framework, we demonstrate that\nexisting methods may not preserve meaning in general, breaking the\naforementioned assumption that source side perturbations should not result in\nchanges in the expected output. We further use this framework to demonstrate\nthat adding additional constraints on attacks allows for adversarial\nperturbations that are more meaning-preserving, but nonetheless largely change\nthe output sequence. Finally, we show that performing untargeted adversarial\ntraining with meaning-preserving attacks is beneficial to the model in terms of\nadversarial robustness, without hurting test performance. A toolkit\nimplementing our evaluation framework is released at\nhttps://github.com/pmichel31415/teapot-nlp.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06901v1",
            "date": "2019-03-16 09:46:29+00:00",
            "title": "Emotion Action Detection and Emotion Inference: the Task and Dataset",
            "authors": [
                "Pengyuan Liu",
                "Chengyu Du",
                "Shuofeng Zhao",
                "Chenghao Zhu"
            ],
            "abstract": "Many Natural Language Processing works on emotion analysis only focus on\nsimple emotion classification without exploring the potentials of putting\nemotion into \"event context\", and ignore the analysis of emotion-related\nevents. One main reason is the lack of this kind of corpus. Here we present\nCause-Emotion-Action Corpus, which manually annotates not only emotion, but\nalso cause events and action events. We propose two new tasks based on the\ndata-set: emotion causality and emotion inference. The first task is to extract\na triple (cause, emotion, action). The second task is to infer the probable\nemotion. We are currently releasing the data-set with 10,603 samples and 15,892\nevents, basic statistic analysis and baseline on both emotion causality and\nemotion inference tasks. Baseline performance demonstrates that there is much\nroom for both tasks to be improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06939v1",
            "date": "2019-03-16 14:59:13+00:00",
            "title": "Improving Lemmatization of Non-Standard Languages with Joint Learning",
            "authors": [
                "Enrique Manjavacas",
                "\u00c1kos K\u00e1d\u00e1r",
                "Mike Kestemont"
            ],
            "abstract": "Lemmatization of standard languages is concerned with (i) abstracting over\nmorphological differences and (ii) resolving token-lemma ambiguities of\ninflected words in order to map them to a dictionary headword. In the present\npaper we aim to improve lemmatization performance on a set of non-standard\nhistorical languages in which the difficulty is increased by an additional\naspect (iii): spelling variation due to lacking orthographic standards. We\napproach lemmatization as a string-transduction task with an encoder-decoder\narchitecture which we enrich with sentence context information using a\nhierarchical sentence encoder. We show significant improvements over the\nstate-of-the-art when training the sentence encoder jointly for lemmatization\nand language modeling. Crucially, our architecture does not require POS or\nmorphological annotations, which are not always available for historical\ncorpora. Additionally, we also test the proposed model on a set of\ntypologically diverse standard languages showing results on par or better than\na model without enhanced sentence representations and previous state-of-the-art\nsystems. Finally, to encourage future work on processing of non-standard\nvarieties, we release the dataset of non-standard languages underlying the\npresent study, based on openly accessible sources.",
            "journal ref": "NAACL-HLT 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06963v1",
            "date": "2019-03-16 17:31:29+00:00",
            "title": "Imbalanced multi-label classification using multi-task learning with extractive summarization",
            "authors": [
                "John Brandt"
            ],
            "abstract": "Extractive summarization and imbalanced multi-label classification often\nrequire vast amounts of training data to avoid overfitting. In situations where\ntraining data is expensive to generate, leveraging information between tasks is\nan attractive approach to increasing the amount of available information. This\npaper employs multi-task training of an extractive summarizer and an RNN-based\nclassifier to improve summarization and classification accuracy by 50% and 75%,\nrespectively, relative to RNN baselines. We hypothesize that concatenating\nsentence encodings based on document and class context increases\ngeneralizability for highly variable corpuses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07037v2",
            "date": "2019-03-17 07:07:44+00:00",
            "title": "Audio De-identification: A New Entity Recognition Task",
            "authors": [
                "Ido Cohn",
                "Itay Laish",
                "Genady Beryozkin",
                "Gang Li",
                "Izhak Shafran",
                "Idan Szpektor",
                "Tzvika Hartman",
                "Avinatan Hassidim",
                "Yossi Matias"
            ],
            "abstract": "Named Entity Recognition (NER) has been mostly studied in the context of\nwritten text. Specifically, NER is an important step in de-identification\n(de-ID) of medical records, many of which are recorded conversations between a\npatient and a doctor. In such recordings, audio spans with personal information\nshould be redacted, similar to the redaction of sensitive character spans in\nde-ID for written text. The application of NER in the context of audio\nde-identification has yet to be fully investigated. To this end, we define the\ntask of audio de-ID, in which audio spans with entity mentions should be\ndetected. We then present our pipeline for this task, which involves Automatic\nSpeech Recognition (ASR), NER on the transcript text, and text-to-audio\nalignment. Finally, we introduce a novel metric for audio de-ID and a new\nevaluation benchmark consisting of a large labeled segment of the Switchboard\nand Fisher audio datasets and detail our pipeline's results on it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07113v2",
            "date": "2019-03-17 15:46:05+00:00",
            "title": "Question Answering via Web Extracted Tables and Pipelined Models",
            "authors": [
                "Bhavya Karki",
                "Fan Hu",
                "Nithin Haridas",
                "Suhail Barot",
                "Zihua Liu",
                "Lucile Callebert",
                "Matthias Grabmair",
                "Anthony Tomasic"
            ],
            "abstract": "In this paper, we describe a dataset and baseline result for a question\nanswering that utilizes web tables. It contains commonly asked questions on the\nweb and their corresponding answers found in tables on websites. Our dataset is\nnovel in that every question is paired with a table of a different signature.\nIn particular, the dataset contains two classes of tables: entity-instance\ntables and the key-value tables. Each QA instance comprises a table of either\nkind, a natural language question, and a corresponding structured SQL query. We\nbuild our model by dividing question answering into several tasks, including\ntable retrieval and question element classification, and conduct experiments to\nmeasure the performance of each task. We extract various features specific to\neach task and compose a full pipeline which constructs the SQL query from its\nparts. Our work provides qualitative results and error analysis for each task,\nand identifies in detail the reasoning required to generate SQL expressions\nfrom natural language questions. This analysis of reasoning informs future\nmodels based on neural machine learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07137v1",
            "date": "2019-03-17 17:42:29+00:00",
            "title": "Topic-Guided Variational Autoencoders for Text Generation",
            "authors": [
                "Wenlin Wang",
                "Zhe Gan",
                "Hongteng Xu",
                "Ruiyi Zhang",
                "Guoyin Wang",
                "Dinghan Shen",
                "Changyou Chen",
                "Lawrence Carin"
            ],
            "abstract": "We propose a topic-guided variational autoencoder (TGVAE) model for text\ngeneration. Distinct from existing variational autoencoder (VAE) based\napproaches, which assume a simple Gaussian prior for the latent code, our model\nspecifies the prior as a Gaussian mixture model (GMM) parametrized by a neural\ntopic module. Each mixture component corresponds to a latent topic, which\nprovides guidance to generate sentences under the topic. The neural topic\nmodule and the VAE-based neural sequence module in our model are learned\njointly. In particular, a sequence of invertible Householder transformations is\napplied to endow the approximate posterior of the latent code with high\nflexibility during model inference. Experimental results show that our TGVAE\noutperforms alternative approaches on both unconditional and conditional text\ngeneration, which can generate semantically-meaningful sentences with various\ntopics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07161v1",
            "date": "2019-03-17 20:26:56+00:00",
            "title": "Technical notes: Syntax-aware Representation Learning With Pointer Networks",
            "authors": [
                "Matteo Grella"
            ],
            "abstract": "This is a work-in-progress report, which aims to share preliminary results of\na novel sequence-to-sequence schema for dependency parsing that relies on a\ncombination of a BiLSTM and two Pointer Networks (Vinyals et al., 2015), in\nwhich the final softmax function has been replaced with the logistic\nregression. The two pointer networks co-operate to develop a latent syntactic\nknowledge, by learning the lexical properties of \"selection\" and the lexical\nproperties of \"selectability\", respectively. At the moment and without\nfine-tuning, the parser implementation gets a UAS of 93.14% on the English\nPenn-treebank (Marcus et al., 1993) annotated with Stanford Dependencies: 2-3%\nunder the SOTA but yet attractive as a baseline of the approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07319v1",
            "date": "2019-03-18 09:21:30+00:00",
            "title": "What You Say and How You Say it: Joint Modeling of Topics and Discourse in Microblog Conversations",
            "authors": [
                "Jichuan Zeng",
                "Jing Li",
                "Yulan He",
                "Cuiyun Gao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "This paper presents an unsupervised framework for jointly modeling topic\ncontent and discourse behavior in microblog conversations. Concretely, we\npropose a neural model to discover word clusters indicating what a conversation\nconcerns (i.e., topics) and those reflecting how participants voice their\nopinions (i.e., discourse). Extensive experiments show that our model can yield\nboth coherent topics and meaningful discourse behavior. Further study shows\nthat our topic and discourse representations can benefit the classification of\nmicroblog messages, especially when they are jointly trained with the\nclassifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07397v1",
            "date": "2019-03-11 11:02:24+00:00",
            "title": "Un duel probabiliste pour d\u00e9partager deux pr\u00e9sidents (LIA @ DEFT'2005)",
            "authors": [
                "Marc El-B\u00e8ze",
                "Juan-Manuel Torres-Moreno",
                "Fr\u00e9d\u00e9ric B\u00e9chet"
            ],
            "abstract": "We present a set of probabilistic models applied to binary classification as\ndefined in the DEFT'05 challenge. The challenge consisted a mixture of two\ndifferents problems in Natural Language Processing : identification of author\n(a sequence of Fran\\c{c}ois Mitterrand's sentences might have been inserted\ninto a speech of Jacques Chirac) and thematic break detection (the subjects\naddressed by the two authors are supposed to be different). Markov chains,\nBayes models and an adaptative process have been used to identify the paternity\nof these sequences. A probabilistic model of the internal coherence of speeches\nwhich has been employed to identify thematic breaks. Adding this model has\nshown to improve the quality results. A comparison with different approaches\ndemostrates the superiority of a strategy that combines learning, coherence and\nadaptation. Applied to the DEFT'05 data test the results in terms of precision\n(0.890), recall (0.955) and Fscore (0.925) measure are very promising.",
            "journal ref": "RNTI (E10)776:1889-1918, 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07402v2",
            "date": "2019-03-18 12:54:22+00:00",
            "title": "Neutron: An Implementation of the Transformer Translation Model and its Variants",
            "authors": [
                "Hongfei Xu",
                "Qiuhui Liu"
            ],
            "abstract": "The Transformer translation model is easier to parallelize and provides\nbetter performance compared to recurrent seq2seq models, which makes it popular\namong industry and research community. We implement the Neutron in this work,\nincluding the Transformer model and its several variants from most recent\nresearches. It is highly optimized, easy to modify and provides comparable\nperformance with interesting features while keeping readability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07416v1",
            "date": "2019-03-14 02:56:03+00:00",
            "title": "Complexity-entropy analysis at different levels of organization in written language",
            "authors": [
                "E. Estevez-Rams",
                "A. Mesa Rodriguez",
                "D. Estevez-Moya"
            ],
            "abstract": "Written language is complex. A written text can be considered an attempt to\nconvey a meaningful message which ends up being constrained by language rules,\ncontext dependence and highly redundant in its use of resources. Despite all\nthese constraints, unpredictability is an essential element of natural\nlanguage. Here we present the use of entropic measures to assert the balance\nbetween predictability and surprise in written text. In short, it is possible\nto measure innovation and context preservation in a document. It is shown that\nthis can also be done at the different levels of organization of a text. The\ntype of analysis presented is reasonably general, and can also be used to\nanalyze the same balance in other complex messages such as DNA, where a\nhierarchy of organizational levels are known to exist.",
            "journal ref": "PLOS ONE, 2019, https://doi.org/10.1371/journal.pone.0214863",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07435v2",
            "date": "2019-03-18 13:38:54+00:00",
            "title": "The emergence of number and syntax units in LSTM language models",
            "authors": [
                "Yair Lakretz",
                "German Kruszewski",
                "Theo Desbordes",
                "Dieuwke Hupkes",
                "Stanislas Dehaene",
                "Marco Baroni"
            ],
            "abstract": "Recent work has shown that LSTMs trained on a generic language modeling\nobjective capture syntax-sensitive generalizations such as long-distance number\nagreement. We have however no mechanistic understanding of how they accomplish\nthis remarkable feat. Some have conjectured it depends on heuristics that do\nnot truly take hierarchical structure into account. We present here a detailed\nstudy of the inner mechanics of number tracking in LSTMs at the single neuron\nlevel. We discover that long-distance number information is largely managed by\ntwo `number units'. Importantly, the behaviour of these units is partially\ncontrolled by other units independently shown to track syntactic structure. We\nconclude that LSTMs are, to some extent, implementing genuinely syntactic\nprocessing mechanisms, paving the way to a more general understanding of\ngrammatical encoding in LSTMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07785v1",
            "date": "2019-03-19 01:19:06+00:00",
            "title": "Cloze-driven Pretraining of Self-attention Networks",
            "authors": [
                "Alexei Baevski",
                "Sergey Edunov",
                "Yinhan Liu",
                "Luke Zettlemoyer",
                "Michael Auli"
            ],
            "abstract": "We present a new approach for pretraining a bi-directional transformer model\nthat provides significant performance gains across a variety of language\nunderstanding problems. Our model solves a cloze-style word reconstruction\ntask, where each word is ablated and must be predicted given the rest of the\ntext. Experiments demonstrate large performance gains on GLUE and new state of\nthe art results on NER as well as constituency parsing benchmarks, consistent\nwith the concurrently introduced BERT model. We also present a detailed\nanalysis of a number of factors that contribute to effective pretraining,\nincluding data domain and size, model capacity, and variations on the cloze\nobjective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07879v2",
            "date": "2019-03-19 08:37:04+00:00",
            "title": "Hybrid Approaches for our Participation to the n2c2 Challenge on Cohort Selection for Clinical Trials",
            "authors": [
                "Xavier Tannier",
                "Nicolas Paris",
                "Hugo Cisneros",
                "Christel Daniel",
                "Matthieu Doutreligne",
                "Catherine Duclos",
                "Nicolas Griffon",
                "Claire Hassen-Khodja",
                "Ivan Lerner",
                "Adrien Parrot",
                "\u00c9ric Sadou",
                "Cyrina Saussol",
                "Pascal Vaillant"
            ],
            "abstract": "Objective: Natural language processing can help minimize human intervention\nin identifying patients meeting eligibility criteria for clinical trials, but\nthere is still a long way to go to obtain a general and systematic approach\nthat is useful for researchers. We describe two methods taking a step in this\ndirection and present their results obtained during the n2c2 challenge on\ncohort selection for clinical trials. Materials and Methods: The first method\nis a weakly supervised method using an unlabeled corpus (MIMIC) to build a\nsilver standard, by producing semi-automatically a small and very precise set\nof rules to detect some samples of positive and negative patients. This silver\nstandard is then used to train a traditional supervised model. The second\nmethod is a terminology-based approach where a medical expert selects the\nappropriate concepts, and a procedure is defined to search the terms and check\nthe structural or temporal constraints. Results: On the n2c2 dataset containing\nannotated data about 13 selection criteria on 288 patients, we obtained an\noverall F1-measure of 0.8969, which is the third best result out of 45\nparticipant teams, with no statistically significant difference with the\nbest-ranked team. Discussion: Both approaches obtained very encouraging results\nand apply to different types of criteria. The weakly supervised method requires\nexplicit descriptions of positive and negative examples in some reports. The\nterminology-based method is very efficient when medical concepts carry most of\nthe relevant information. Conclusion: It is unlikely that much more annotated\ndata will be soon available for the task of identifying a wide range of patient\nphenotypes. One must focus on weakly or non-supervised learning methods using\nboth structured and unstructured data and relying on a comprehensive\nrepresentation of the patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07917v1",
            "date": "2019-03-19 10:18:57+00:00",
            "title": "CVIT-MT Systems for WAT-2018",
            "authors": [
                "Jerin Philip",
                "Vinay P. Namboodiri",
                "C. V. Jawahar"
            ],
            "abstract": "This document describes the machine translation system used in the\nsubmissions of IIIT-Hyderabad CVIT-MT for the WAT-2018 English-Hindi\ntranslation task. Performance is evaluated on the associated corpus provided by\nthe organizers. We experimented with convolutional sequence to sequence\narchitectures. We also train with additional data obtained through\nbacktranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07926v2",
            "date": "2019-03-19 10:40:23+00:00",
            "title": "compare-mt: A Tool for Holistic Comparison of Language Generation Systems",
            "authors": [
                "Graham Neubig",
                "Zi-Yi Dou",
                "Junjie Hu",
                "Paul Michel",
                "Danish Pruthi",
                "Xinyi Wang",
                "John Wieting"
            ],
            "abstract": "In this paper, we describe compare-mt, a tool for holistic analysis and\ncomparison of the results of systems for language generation tasks such as\nmachine translation. The main goal of the tool is to give the user a high-level\nand coherent view of the salient differences between systems that can then be\nused to guide further analysis or system improvement. It implements a number of\ntools to do so, such as analysis of accuracy of generation of particular types\nof words, bucketed histograms of sentence accuracies or counts based on salient\ncharacteristics, and extraction of characteristic $n$-grams for each system. It\nalso has a number of advanced features such as use of linguistic labels, source\nside data, or comparison of log likelihoods for probabilistic models, and also\naims to be easily extensible by users to new types of analysis. The code is\navailable at https://github.com/neulab/compare-mt",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08097v2",
            "date": "2019-03-19 16:35:29+00:00",
            "title": "Natural Language Generation at Scale: A Case Study for Open Domain Question Answering",
            "authors": [
                "Alessandra Cervone",
                "Chandra Khatri",
                "Rahul Goel",
                "Behnam Hedayatnia",
                "Anu Venkatesh",
                "Dilek Hakkani-Tur",
                "Raefer Gabriel"
            ],
            "abstract": "Current approaches to Natural Language Generation (NLG) for dialog mainly\nfocus on domain-specific, task-oriented applications (e.g. restaurant booking)\nusing limited ontologies (up to 20 slot types), usually without considering the\nprevious conversation context. Furthermore, these approaches require large\namounts of data for each domain, and do not benefit from examples that may be\navailable for other domains. This work explores the feasibility of applying\nstatistical NLG to scenarios requiring larger ontologies, such as multi-domain\ndialog applications or open-domain question answering (QA) based on knowledge\ngraphs. We model NLG through an Encoder-Decoder framework using a large dataset\nof interactions between real-world users and a conversational agent for\nopen-domain QA. First, we investigate the impact of increasing the number of\nslot types on the generation quality and experiment with different partitions\nof the QA data with progressively larger ontologies (up to 369 slot types).\nSecond, we perform multi-task learning experiments between open-domain QA and\ntask-oriented dialog, and benchmark our model on a popular NLG dataset.\nMoreover, we experiment with using the conversational context as an additional\ninput to improve response generation quality. Our experiments show the\nfeasibility of learning statistical NLG models for open-domain QA with larger\nontologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08237v3",
            "date": "2019-03-19 19:49:12+00:00",
            "title": "When redundancy is useful: A Bayesian approach to 'overinformative' referring expressions",
            "authors": [
                "Judith Degen",
                "Robert D. Hawkins",
                "Caroline Graf",
                "Elisa Kreiss",
                "Noah D. Goodman"
            ],
            "abstract": "Referring is one of the most basic and prevalent uses of language. How do\nspeakers choose from the wealth of referring expressions at their disposal?\nRational theories of language use have come under attack for decades for not\nbeing able to account for the seemingly irrational overinformativeness\nubiquitous in referring expressions. Here we present a novel production model\nof referring expressions within the Rational Speech Act framework that treats\nspeakers as agents that rationally trade off cost and informativeness of\nutterances. Crucially, we relax the assumption that informativeness is computed\nwith respect to a deterministic Boolean semantics, in favor of a\nnon-deterministic continuous semantics. This innovation allows us to capture a\nlarge number of seemingly disparate phenomena within one unified framework: the\nbasic asymmetry in speakers' propensity to overmodify with color rather than\nsize; the increase in overmodification in complex scenes; the increase in\novermodification with atypical features; and the increase in specificity in\nnominal reference as a function of typicality. These findings cast a new light\non the production of referring expressions: rather than being wastefully\noverinformative, reference is usefully redundant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08445v1",
            "date": "2019-03-20 11:19:58+00:00",
            "title": "Left-to-Right Dependency Parsing with Pointer Networks",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We propose a novel transition-based algorithm that straightforwardly parses\nsentences from left to right by building $n$ attachments, with $n$ being the\nlength of the input sentence. Similarly to the recent stack-pointer parser by\nMa et al. (2018), we use the pointer network framework that, given a word, can\ndirectly point to a position from the sentence. However, our left-to-right\napproach is simpler than the original top-down stack-pointer parser (not\nrequiring a stack) and reduces transition sequence length in half, from 2$n$-1\nactions to $n$. This results in a quadratic non-projective parser that runs\ntwice as fast as the original while achieving the best accuracy to date on the\nEnglish PTB dataset (96.04% UAS, 94.43% LAS) among fully-supervised\nsingle-model dependency parsers, and improves over the former top-down\ntransition system in the majority of languages tested.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08450v3",
            "date": "2019-03-20 11:28:06+00:00",
            "title": "Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding",
            "authors": [
                "Jonggu Kim",
                "Jong-Hyeok Lee"
            ],
            "abstract": "To capture salient contextual information for spoken language understanding\n(SLU) of a dialogue, we propose time-aware models that automatically learn the\nlatent time-decay function of the history without a manual time-decay function.\nWe also propose a method to identify and label the current speaker to improve\nthe SLU accuracy. In experiments on the benchmark dataset used in Dialog State\nTracking Challenge 4, the proposed models achieved significantly higher F1\nscores than the state-of-the-art contextual models. Finally, we analyze the\neffectiveness of the introduced models in detail. The analysis demonstrates\nthat the proposed methods were effective to improve SLU accuracy individually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08678v2",
            "date": "2019-03-20 18:11:59+00:00",
            "title": "Probing the Need for Visual Context in Multimodal Machine Translation",
            "authors": [
                "Ozan Caglayan",
                "Pranava Madhyastha",
                "Lucia Specia",
                "Lo\u00efc Barrault"
            ],
            "abstract": "Current work on multimodal machine translation (MMT) has suggested that the\nvisual modality is either unnecessary or only marginally beneficial. We posit\nthat this is a consequence of the very simple, short and repetitive sentences\nused in the only available dataset for the task (Multi30K), rendering the\nsource text sufficient as context. In the general case, however, we believe\nthat it is possible to combine visual and textual information in order to\nground translations. In this paper we probe the contribution of the visual\nmodality to state-of-the-art MMT models by conducting a systematic analysis\nwhere we partially deprive the models from source-side textual context. Our\nresults show that under limited textual context, models are capable of\nleveraging the visual input to generate better translations. This contradicts\nthe current belief that MMT models disregard the visual modality because of\neither the quality of the image features or the way they are integrated into\nthe model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08788v2",
            "date": "2019-03-21 01:01:22+00:00",
            "title": "Selective Attention for Context-aware Neural Machine Translation",
            "authors": [
                "Sameen Maruf",
                "Andr\u00e9 F. T. Martins",
                "Gholamreza Haffari"
            ],
            "abstract": "Despite the progress made in sentence-level NMT, current systems still fall\nshort at achieving fluent, good quality translation for a full document. Recent\nworks in context-aware NMT consider only a few previous sentences as context\nand may not scale to entire documents. To this end, we propose a novel and\nscalable top-down approach to hierarchical attention for context-aware NMT\nwhich uses sparse attention to selectively focus on relevant sentences in the\ndocument context and then attends to key words in those sentences. We also\npropose single-level attention approaches based on sentence or word-level\ninformation in the context. The document-level context representation, produced\nfrom these attention modules, is integrated into the encoder or decoder of the\nTransformer model depending on whether we use monolingual or bilingual context.\nOur experiments and evaluation on English-German datasets in different document\nMT settings show that our selective attention approach not only significantly\noutperforms context-agnostic baselines but also surpasses context-aware\nbaselines in most cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08808v1",
            "date": "2019-03-19 17:31:44+00:00",
            "title": "Bidirectional Recurrent Models for Offensive Tweet Classification",
            "authors": [
                "Aleix Cambray",
                "Norbert Podsadowski"
            ],
            "abstract": "In this paper we propose four deep recurrent architectures to tackle the task\nof offensive tweet detection as well as further classification into targeting\nand subject of said targeting. Our architectures are based on LSTMs and GRUs,\nwe present a simple bidirectional LSTM as a baseline system and then further\nincrease the complexity of the models by adding convolutional layers and\nimplementing a split-process-merge architecture with LSTM and GRU as\nprocessors. Multiple pre-processing techniques were also investigated. The\nvalidation F1-score results from each model are presented for the three\nsubtasks as well as the final F1-score performance on the private competition\ntest set. It was found that model complexity did not necessarily yield better\nresults. Our best-performing model was also the simplest, a bidirectional LSTM;\nclosely followed by a two-branch bidirectional LSTM and GRU architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08855v5",
            "date": "2019-03-21 07:19:45+00:00",
            "title": "Linguistic Knowledge and Transferability of Contextual Representations",
            "authors": [
                "Nelson F. Liu",
                "Matt Gardner",
                "Yonatan Belinkov",
                "Matthew E. Peters",
                "Noah A. Smith"
            ],
            "abstract": "Contextual word representations derived from large-scale neural language\nmodels are successful across a diverse set of NLP tasks, suggesting that they\nencode useful and transferable features of language. To shed light on the\nlinguistic knowledge they capture, we study the representations produced by\nseveral recent pretrained contextualizers (variants of ELMo, the OpenAI\ntransformer language model, and BERT) with a suite of seventeen diverse probing\ntasks. We find that linear models trained on top of frozen contextual\nrepresentations are competitive with state-of-the-art task-specific models in\nmany cases, but fail on tasks requiring fine-grained linguistic knowledge\n(e.g., conjunct identification). To investigate the transferability of\ncontextual word representations, we quantify differences in the transferability\nof individual layers within contextualizers, especially between recurrent\nneural networks (RNNs) and transformers. For instance, higher layers of RNNs\nare more task-specific, while transformer layers do not exhibit the same\nmonotonic trend. In addition, to better understand what makes contextual word\nrepresentations transferable, we compare language model pretraining with eleven\nsupervised pretraining tasks. For any given task, pretraining on a closely\nrelated task yields better performance than language model pretraining (which\nis better on average) when the pretraining dataset is fixed. However, language\nmodel pretraining on more data gives the best results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08905v1",
            "date": "2019-03-21 10:09:43+00:00",
            "title": "RAP-Net: Recurrent Attention Pooling Networks for Dialogue Response Selection",
            "authors": [
                "Chao-Wei Huang",
                "Ting-Rui Chiang",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "The response selection has been an emerging research topic due to the growing\ninterest in dialogue modeling, where the goal of the task is to select an\nappropriate response for continuing dialogues. To further push the end-to-end\ndialogue model toward real-world scenarios, the seventh Dialog System\nTechnology Challenge (DSTC7) proposed a challenging track based on real chatlog\ndatasets. The competition focuses on dialogue modeling with several advanced\ncharacteristics: (1) natural language diversity, (2) capability of precisely\nselecting a proper response from a large set of candidates or the scenario\nwithout any correct answer, and (3) knowledge grounding. This paper introduces\nrecurrent attention pooling networks (RAP-Net), a novel framework for response\nselection, which can well estimate the relevance between the dialogue contexts\nand the candidates. The proposed RAP-Net is shown to be effective and can be\ngeneralized across different datasets and settings in the DSTC7 experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08953v1",
            "date": "2019-03-21 12:39:02+00:00",
            "title": "Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer",
            "authors": [
                "Ting-Rui Chiang",
                "Chao-Wei Huang",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "With the increasing research interest in dialogue response generation, there\nis an emerging branch formulating this task as selecting next sentences, where\ngiven the partial dialogue contexts, the goal is to determine the most probable\nnext sentence. Following the recent success of the Transformer model, this\npaper proposes (1) a new variant of attention mechanism based on multi-head\nattention, called highway attention, and (2) a recurrent model based on\ntransformer and the proposed highway attention, so-called Highway Recurrent\nTransformer. Experiments on the response selection task in the seventh Dialog\nSystem Technology Challenge (DSTC7) show the capability of the proposed model\nof modeling both utterance-level and dialogue-level information; the\neffectiveness of each module is further analyzed as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08983v3",
            "date": "2019-03-19 20:22:02+00:00",
            "title": "SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)",
            "authors": [
                "Marcos Zampieri",
                "Shervin Malmasi",
                "Preslav Nakov",
                "Sara Rosenthal",
                "Noura Farra",
                "Ritesh Kumar"
            ],
            "abstract": "We present the results and the main findings of SemEval-2019 Task 6 on\nIdentifying and Categorizing Offensive Language in Social Media (OffensEval).\nThe task was based on a new dataset, the Offensive Language Identification\nDataset (OLID), which contains over 14,000 English tweets. It featured three\nsub-tasks. In sub-task A, the goal was to discriminate between offensive and\nnon-offensive posts. In sub-task B, the focus was on the type of offensive\ncontent in the post. Finally, in sub-task C, systems had to detect the target\nof the offensive posts. OffensEval attracted a large number of participants and\nit was one of the most popular tasks in SemEval-2019. In total, about 800 teams\nsigned up to participate in the task, and 115 of them submitted results, which\nwe present and analyze in this report.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09424v1",
            "date": "2019-03-22 09:54:36+00:00",
            "title": "An end-to-end Neural Network Framework for Text Clustering",
            "authors": [
                "Jie Zhou",
                "Xingyi Cheng",
                "Jinchao Zhang"
            ],
            "abstract": "The unsupervised text clustering is one of the major tasks in natural\nlanguage processing (NLP) and remains a difficult and complex problem.\nConventional \\mbox{methods} generally treat this task using separated steps,\nincluding text representation learning and clustering the representations. As\nan improvement, neural methods have also been introduced for continuous\nrepresentation learning to address the sparsity problem. However, the\nmulti-step process still deviates from the unified optimization target.\nEspecially the second step of cluster is generally performed with conventional\nmethods such as k-Means. We propose a pure neural framework for text clustering\nin an end-to-end manner. It jointly learns the text representation and the\nclustering model. Our model works well when the context can be obtained, which\nis nearly always the case in the field of NLP. We have our method\n\\mbox{evaluated} on two widely used benchmarks: IMDB movie reviews for\nsentiment classification and $20$-Newsgroup for topic categorization. Despite\nits simplicity, experiments show the model outperforms previous clustering\nmethods by a large margin. Furthermore, the model is also verified on English\nwiki dataset as a large corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09442v2",
            "date": "2019-03-22 11:01:16+00:00",
            "title": "LINSPECTOR: Multilingual Probing Tasks for Word Representations",
            "authors": [
                "G\u00f6zde G\u00fcl \u015eahin",
                "Clara Vania",
                "Ilia Kuznetsov",
                "Iryna Gurevych"
            ],
            "abstract": "Despite an ever growing number of word representation models introduced for a\nlarge number of languages, there is a lack of a standardized technique to\nprovide insights into what is captured by these models. Such insights would\nhelp the community to get an estimate of the downstream task performance, as\nwell as to design more informed neural architectures, while avoiding extensive\nexperimentation which requires substantial computational resources not all\nresearchers have access to. A recent development in NLP is to use simple\nclassification tasks, also called probing tasks, that test for a single\nlinguistic feature such as part-of-speech. Existing studies mostly focus on\nexploring the linguistic information encoded by the continuous representations\nof English text. However, from a typological perspective the morphologically\npoor English is rather an outlier: the information encoded by the word order\nand function words in English is often stored on a morphological level in other\nlanguages. To address this, we introduce 15 type-level probing tasks such as\ncase marking, possession, word length, morphological tag count and pseudoword\nidentification for 24 languages. We present a reusable methodology for creation\nand evaluation of such tests in a multilingual setting. We then present\nexperiments on several diverse multilingual word embedding models, in which we\nrelate the probing task performance for a diverse set of languages to a range\nof five classic NLP tasks: POS-tagging, dependency parsing, semantic role\nlabeling, named entity recognition and natural language inference. We find that\na number of probing tests have significantly high positive correlation to the\ndownstream tasks, especially for morphologically rich languages. We show that\nour tests can be used to explore word embeddings or black-box neural models for\nlinguistic cues in a multilingual setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09460v1",
            "date": "2019-03-22 11:55:21+00:00",
            "title": "Data Augmentation via Dependency Tree Morphing for Low-Resource Languages",
            "authors": [
                "G\u00f6zde G\u00fcl \u015eahin",
                "Mark Steedman"
            ],
            "abstract": "Neural NLP systems achieve high scores in the presence of sizable training\ndataset. Lack of such datasets leads to poor system performances in the case\nlow-resource languages. We present two simple text augmentation techniques\nusing dependency trees, inspired from image processing. We crop sentences by\nremoving dependency links, and we rotate sentences by moving the tree fragments\naround the root. We apply these techniques to augment the training sets of\nlow-resource languages in Universal Dependencies project. We implement a\ncharacter-level sequence tagging model and evaluate the augmented datasets on\npart-of-speech tagging task. We show that crop and rotate provides improvements\nover the models trained with non-augmented data for majority of the languages,\nespecially for languages with rich case marking systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09588v1",
            "date": "2019-03-22 16:29:18+00:00",
            "title": "Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence",
            "authors": [
                "Chi Sun",
                "Luyao Huang",
                "Xipeng Qiu"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA), which aims to identify fine-grained\nopinion polarity towards a specific aspect, is a challenging subtask of\nsentiment analysis (SA). In this paper, we construct an auxiliary sentence from\nthe aspect and convert ABSA to a sentence-pair classification task, such as\nquestion answering (QA) and natural language inference (NLI). We fine-tune the\npre-trained model from BERT and achieve new state-of-the-art results on\nSentiHood and SemEval-2014 Task 4 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09722v2",
            "date": "2019-03-22 22:14:51+00:00",
            "title": "Pre-trained Language Model Representations for Language Generation",
            "authors": [
                "Sergey Edunov",
                "Alexei Baevski",
                "Michael Auli"
            ],
            "abstract": "Pre-trained language model representations have been successful in a wide\nrange of language understanding tasks. In this paper, we examine different\nstrategies to integrate pre-trained representations into sequence to sequence\nmodels and apply it to neural machine translation and abstractive\nsummarization. We find that pre-trained representations are most effective when\nadded to the encoder network which slows inference by only 14%. Our experiments\nin machine translation show gains of up to 5.3 BLEU in a simulated\nresource-poor setup. While returns diminish with more labeled data, we still\nobserve improvements when millions of sentence-pairs are available. Finally, on\nabstractive summarization we achieve a new state of the art on the full text\nversion of CNN/DailyMail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09813v1",
            "date": "2019-03-23 12:19:11+00:00",
            "title": "Knowledge-Grounded Response Generation with Deep Attentional Latent-Variable Model",
            "authors": [
                "Hao-Tong Ye",
                "Kai-Ling Lo",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "End-to-end dialogue generation has achieved promising results without using\nhandcrafted features and attributes specific for each task and corpus. However,\none of the fatal drawbacks in such approaches is that they are unable to\ngenerate informative utterances, so it limits their usage from some real-world\nconversational applications. This paper attempts at generating diverse and\ninformative responses with a variational generation model, which contains a\njoint attention mechanism conditioning on the information from both dialogue\ncontexts and extra knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09878v2",
            "date": "2019-03-23 20:25:40+00:00",
            "title": "Expanding the Text Classification Toolbox with Cross-Lingual Embeddings",
            "authors": [
                "Meryem M'hamdi",
                "Robert West",
                "Andreea Hossmann",
                "Michael Baeriswyl",
                "Claudiu Musat"
            ],
            "abstract": "Most work in text classification and Natural Language Processing (NLP)\nfocuses on English or a handful of other languages that have text corpora of\nhundreds of millions of words. This is creating a new version of the digital\ndivide: the artificial intelligence (AI) divide. Transfer-based approaches,\nsuch as Cross-Lingual Text Classification (CLTC) - the task of categorizing\ntexts written in different languages into a common taxonomy, are a promising\nsolution to the emerging AI divide. Recent work on CLTC has focused on\ndemonstrating the benefits of using bilingual word embeddings as features,\nrelegating the CLTC problem to a mere benchmark based on a simple averaged\nperceptron.\n  In this paper, we explore more extensively and systematically two flavors of\nthe CLTC problem: news topic classification and textual churn intent detection\n(TCID) in social media. In particular, we test the hypothesis that embeddings\nwith context are more effective, by multi-tasking the learning of multilingual\nword embeddings and text classification; we explore neural architectures for\nCLTC; and we move from bi- to multi-lingual word embeddings. For all\narchitectures, types of word embeddings and datasets, we notice a consistent\ngain trend in favor of multilingual joint training, especially for\nlow-resourced languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10104v1",
            "date": "2019-03-25 02:26:54+00:00",
            "title": "Argument Mining for Understanding Peer Reviews",
            "authors": [
                "Xinyu Hua",
                "Mitko Nikolov",
                "Nikhil Badugu",
                "Lu Wang"
            ],
            "abstract": "Peer-review plays a critical role in the scientific writing and publication\necosystem. To assess the efficiency and efficacy of the reviewing process, one\nessential element is to understand and evaluate the reviews themselves. In this\nwork, we study the content and structure of peer reviews under the argument\nmining framework, through automatically detecting (1) argumentative\npropositions put forward by reviewers, and (2) their types (e.g., evaluating\nthe work or making suggestions for improvement). We first collect 14.2K reviews\nfrom major machine learning and natural language processing venues. 400 reviews\nare annotated with 10,386 propositions and corresponding types of Evaluation,\nRequest, Fact, Reference, or Quote. We then train state-of-the-art proposition\nsegmentation and classification models on the data to evaluate their utilities\nand identify new challenges for this new domain, motivating future directions\nfor argument mining. Further experiments show that proposition usage varies\nacross venues in amount, type, and topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10126v3",
            "date": "2019-03-25 04:09:59+00:00",
            "title": "Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction",
            "authors": [
                "Peng Xu",
                "Denilson Barbosa"
            ],
            "abstract": "Knowledge Bases (KBs) require constant up-dating to reflect changes to the\nworld they represent. For general purpose KBs, this is often done through\nRelation Extraction (RE), the task of predicting KB relations expressed in text\nmentioning entities known to the KB. One way to improve RE is to use KB\nEmbeddings (KBE) for link prediction. However, despite clear connections\nbetween RE and KBE, little has been done toward properly unifying these models\nsystematically. We help close the gap with a framework that unifies the\nlearning of RE and KBE models leading to significant improvements over the\nstate-of-the-art in RE. The code is available at\nhttps://github.com/billy-inn/HRERE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10238v1",
            "date": "2019-03-25 11:00:20+00:00",
            "title": "Aligning Vector-spaces with Noisy Supervised Lexicons",
            "authors": [
                "Noa Yehezkel Lubin",
                "Jacob Goldberger",
                "Yoav Goldberg"
            ],
            "abstract": "The problem of learning to translate between two vector spaces given a set of\naligned points arises in several application areas of NLP. Current solutions\nassume that the lexicon which defines the alignment pairs is noise-free. We\nconsider the case where the set of aligned points is allowed to contain an\namount of noise, in the form of incorrect lexicon pairs and show that this\narises in practice by analyzing the edited dictionaries after the cleaning\nprocess. We demonstrate that such noise substantially degrades the accuracy of\nthe learned translation when using current methods. We propose a model that\naccounts for noisy pairs. This is achieved by introducing a generative model\nwith a compatible iterative EM algorithm. The algorithm jointly learns the\nnoise level in the lexicon, finds the set of noisy pairs, and learns the\nmapping between the spaces. We demonstrate the effectiveness of our proposed\nalgorithm on two alignment problems: bilingual word embedding translation, and\nmapping between diachronic embedding spaces for recovering the semantic shifts\nof words across time periods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10318v2",
            "date": "2019-03-25 13:42:45+00:00",
            "title": "Fine-tune BERT for Extractive Summarization",
            "authors": [
                "Yang Liu"
            ],
            "abstract": "BERT, a pre-trained Transformer model, has achieved ground-breaking\nperformance on multiple NLP tasks. In this paper, we describe BERTSUM, a simple\nvariant of BERT, for extractive summarization. Our system is the state of the\nart on the CNN/Dailymail dataset, outperforming the previous best-performed\nsystem by 1.65 on ROUGE-L. The codes to reproduce our results are available at\nhttps://github.com/nlpyang/BertSum",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10548v1",
            "date": "2019-03-25 18:45:29+00:00",
            "title": "Recognizing Arrow Of Time In The Short Stories",
            "authors": [
                "Fahimeh Hosseini",
                "Hosein Fooladi",
                "Mohammad Reza Samsami"
            ],
            "abstract": "Recognizing arrow of time in short stories is a challenging task. i.e., given\nonly two paragraphs, determining which comes first and which comes next is a\ndifficult task even for humans. In this paper, we have collected and curated a\nnovel dataset for tackling this challenging task. We have shown that a\npre-trained BERT architecture achieves reasonable accuracy on the task, and\noutperforms RNN-based architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10625v2",
            "date": "2019-03-25 23:05:11+00:00",
            "title": "Neural Grammatical Error Correction with Finite State Transducers",
            "authors": [
                "Felix Stahlberg",
                "Christopher Bryant",
                "Bill Byrne"
            ],
            "abstract": "Grammatical error correction (GEC) is one of the areas in natural language\nprocessing in which purely neural models have not yet superseded more\ntraditional symbolic models. Hybrid systems combining phrase-based statistical\nmachine translation (SMT) and neural sequence models are currently among the\nmost effective approaches to GEC. However, both SMT and neural\nsequence-to-sequence models require large amounts of annotated data. Language\nmodel based GEC (LM-GEC) is a promising alternative which does not rely on\nannotated training data. We show how to improve LM-GEC by applying modelling\ntechniques based on finite state transducers. We report further gains by\nrescoring with neural language models. We show that our methods developed for\nLM-GEC can also be used with SMT systems if annotated training data is\navailable. Our best system outperforms the best published result on the\nCoNLL-2014 test set, and achieves far better relative improvements over the SMT\nbaselines than previous hybrid systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10635v1",
            "date": "2019-03-26 00:01:47+00:00",
            "title": "Federated Learning Of Out-Of-Vocabulary Words",
            "authors": [
                "Mingqing Chen",
                "Rajiv Mathews",
                "Tom Ouyang",
                "Fran\u00e7oise Beaufays"
            ],
            "abstract": "We demonstrate that a character-level recurrent neural network is able to\nlearn out-of-vocabulary (OOV) words under federated learning settings, for the\npurpose of expanding the vocabulary of a virtual keyboard for smartphones\nwithout exporting sensitive text to servers. High-frequency words can be\nsampled from the trained generative model by drawing from the joint posterior\ndirectly. We study the feasibility of the approach in two settings: (1) using\nsimulated federated learning on a publicly available non-IID per-user dataset\nfrom a popular social networking website, (2) using federated learning on data\nhosted on user mobile devices. The model achieves good recall and precision\ncompared to ground-truth OOV words in setting (1). With (2) we demonstrate the\npracticality of this approach by showing that we can learn meaningful OOV words\nwith good character-level prediction accuracy and cross entropy loss.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10671v2",
            "date": "2019-03-26 04:33:23+00:00",
            "title": "Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus",
            "authors": [
                "Hongyu Gong",
                "Suma Bhat",
                "Lingfei Wu",
                "Jinjun Xiong",
                "Wen-mei Hwu"
            ],
            "abstract": "Text style transfer rephrases a text from a source style (e.g., informal) to\na target style (e.g., formal) while keeping its original meaning. Despite the\nsuccess existing works have achieved using a parallel corpus for the two\nstyles, transferring text style has proven significantly more challenging when\nthere is no parallel training corpus. In this paper, we address this challenge\nby using a reinforcement-learning-based generator-evaluator architecture. Our\ngenerator employs an attention-based encoder-decoder to transfer a sentence\nfrom the source style to the target style. Our evaluator is an adversarially\ntrained style discriminator with semantic and syntactic constraints that score\nthe generated sentence for style, meaning preservation, and fluency.\nExperimental results on two different style transfer tasks (sentiment transfer\nand formality transfer) show that our model outperforms state-of-the-art\napproaches. Furthermore, we perform a manual evaluation that demonstrates the\neffectiveness of the proposed method using subjective metrics of generated text\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10675v1",
            "date": "2019-03-26 04:42:17+00:00",
            "title": "Document Similarity for Texts of Varying Lengths via Hidden Topics",
            "authors": [
                "Hongyu Gong",
                "Tarek Sakakini",
                "Suma Bhat",
                "Jinjun Xiong"
            ],
            "abstract": "Measuring similarity between texts is an important task for several\napplications. Available approaches to measure document similarity are\ninadequate for document pairs that have non-comparable lengths, such as a long\ndocument and its summary. This is because of the lexical, contextual and the\nabstraction gaps between a long document of rich details and its concise\nsummary of abstract information. In this paper, we present a document matching\napproach to bridge this gap, by comparing the texts in a common space of hidden\ntopics. We evaluate the matching algorithm on two matching tasks and find that\nit consistently and widely outperforms strong baselines. We also highlight the\nbenefits of incorporating domain knowledge to text matching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10676v3",
            "date": "2019-03-26 05:11:46+00:00",
            "title": "SciBERT: A Pretrained Language Model for Scientific Text",
            "authors": [
                "Iz Beltagy",
                "Kyle Lo",
                "Arman Cohan"
            ],
            "abstract": "Obtaining large-scale annotated data for NLP tasks in the scientific domain\nis challenging and expensive. We release SciBERT, a pretrained language model\nbased on BERT (Devlin et al., 2018) to address the lack of high-quality,\nlarge-scale labeled scientific data. SciBERT leverages unsupervised pretraining\non a large multi-domain corpus of scientific publications to improve\nperformance on downstream scientific NLP tasks. We evaluate on a suite of tasks\nincluding sequence tagging, sentence classification and dependency parsing,\nwith datasets from a variety of scientific domains. We demonstrate\nstatistically significant improvements over BERT and achieve new\nstate-of-the-art results on several of these tasks. The code and pretrained\nmodels are available at https://github.com/allenai/scibert/.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10728v2",
            "date": "2019-03-26 08:26:58+00:00",
            "title": "A Silver Standard Corpus of Human Phenotype-Gene Relations",
            "authors": [
                "Diana Sousa",
                "Andre Lamurias",
                "Francisco M. Couto"
            ],
            "abstract": "Human phenotype-gene relations are fundamental to fully understand the origin\nof some phenotypic abnormalities and their associated diseases. Biomedical\nliterature is the most comprehensive source of these relations, however, we\nneed Relation Extraction tools to automatically recognize them. Most of these\ntools require an annotated corpus and to the best of our knowledge, there is no\ncorpus available annotated with human phenotype-gene relations. This paper\npresents the Phenotype-Gene Relations (PGR) corpus, a silver standard corpus of\nhuman phenotype and gene annotations and their relations. The corpus consists\nof 1712 abstracts, 5676 human phenotype annotations, 13835 gene annotations,\nand 4283 relations. We generated this corpus using Named-Entity Recognition\ntools, whose results were partially evaluated by eight curators, obtaining a\nprecision of 87.01%. By using the corpus we were able to obtain promising\nresults with two state-of-the-art deep learning tools, namely 78.05% of\nprecision. The PGR corpus was made publicly available to the research\ncommunity.",
            "journal ref": "Proceedings of the 2019 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, Volume 1 (Long and Short Papers). 2019. pp. 1487-1492",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10915v1",
            "date": "2019-03-26 14:19:19+00:00",
            "title": "Language Model Adaptation for Language and Dialect Identification of Text",
            "authors": [
                "Tommi Jauhiainen",
                "Krister Lind\u00e9n",
                "Heidi Jauhiainen"
            ],
            "abstract": "This article describes an unsupervised language model adaptation approach\nthat can be used to enhance the performance of language identification methods.\nThe approach is applied to a current version of the HeLI language\nidentification method, which is now called HeLI 2.0. We describe the HeLI 2.0\nmethod in detail. The resulting system is evaluated using the datasets from the\nGerman dialect identification and Indo-Aryan language identification shared\ntasks of the VarDial workshops 2017 and 2018. The new approach with language\nidentification provides considerably higher F1-scores than the previous HeLI\nmethod or the other systems which participated in the shared tasks. The results\nindicate that unsupervised language model adaptation should be considered as an\noption in all language identification tasks, especially in those where\nencountering out-of-domain data is likely.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10921v2",
            "date": "2019-03-26 14:32:55+00:00",
            "title": "A New Approach for Semi-automatic Building and Extending a Multilingual Terminology Thesaurus",
            "authors": [
                "Adam Rambousek",
                "Ales Horak",
                "Vit Suchomel",
                "Vit Baisa"
            ],
            "abstract": "This paper describes a new system for semi-automatically building, extending\nand managing a terminological thesaurus---a multilingual terminology dictionary\nenriched with relationships between the terms themselves to form a thesaurus.\nThe system allows to radically enhance the workflow of current terminology\nexpert groups, where most of the editing decisions still come from\nintrospection. The presented system supplements the lexicographic process with\nnatural language processing techniques, which are seamlessly integrated to the\nthesaurus editing environment. The system's methodology and the resulting\nthesaurus are closely connected to new domain corpora in the six languages\ninvolved. They are used for term usage examples as well as for the automatic\nextraction of new candidate terms. The terminological thesaurus is now\naccessible via a web-based application, which a) presents rich detailed\ninformation on each term, b) visualizes term relations, and c) displays\nreal-life usage examples of the term in the domain-related documents and in the\ncontext-based similar terms. Furthermore, the specialized corpora are used to\ndetect candidate translations of terms from the central language (Czech) to the\nother languages (English, French, German, Russian and Slovak) as well as to\ndetect broader Czech terms, which help to place new terms in the actual\nthesaurus hierarchy. This project has been realized as a terminological\nthesaurus of land surveying, but the presented tools and methodology are\nreusable for other terminology domains.",
            "journal ref": "International Journal on Artifical Intelligence Tools, World\n  Scientific Publishing, 2019, vol. 28, No 2",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10950v3",
            "date": "2019-03-26 15:14:31+00:00",
            "title": "A Probabilistic Generative Model of Linguistic Typology",
            "authors": [
                "Johannes Bjerva",
                "Yova Kementchedjhieva",
                "Ryan Cotterell",
                "Isabelle Augenstein"
            ],
            "abstract": "In the principles-and-parameters framework, the structural features of\nlanguages depend on parameters that may be toggled on or off, with a single\nparameter often dictating the status of multiple features. The implied\ncovariance between features inspires our probabilisation of this line of\nlinguistic inquiry---we develop a generative model of language based on\nexponential-family matrix factorisation. By modelling all languages and\nfeatures within the same architecture, we show how structural similarities\nbetween languages can be exploited to predict typological features with\nnear-perfect accuracy, outperforming several baselines on the task of\npredicting held-out features. Furthermore, we show that language embeddings\npre-trained on monolingual text allow for generalisation to unobserved\nlanguages. This finding has clear practical and also theoretical implications:\nthe results confirm what linguists have hypothesised, i.e.~that there are\nsignificant correlations between typological features and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11222v2",
            "date": "2019-03-27 01:57:18+00:00",
            "title": "ner and pos when nothing is capitalized",
            "authors": [
                "Stephen Mayhew",
                "Tatiana Tsygankova",
                "Dan Roth"
            ],
            "abstract": "For those languages which use it, capitalization is an important signal for\nthe fundamental NLP tasks of Named Entity Recognition (NER) and Part of Speech\n(POS) tagging. In fact, it is such a strong signal that model performance on\nthese tasks drops sharply in common lowercased scenarios, such as noisy web\ntext or machine translation outputs. In this work, we perform a systematic\nanalysis of solutions to this problem, modifying only the casing of the train\nor test data using lowercasing and truecasing methods. While prior work and\nfirst impressions might suggest training a caseless model, or using a truecaser\nat test time, we show that the most effective strategy is a concatenation of\ncased and lowercased training data, producing a single model with high\nperformance on both cased and uncased text. As shown in our experiments, this\nresult holds across tasks and input representations. Finally, we show that our\nproposed solution gives an 8% F1 improvement in mention detection on noisy\nout-of-domain Twitter data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11269v3",
            "date": "2019-03-27 07:15:21+00:00",
            "title": "CSS10: A Collection of Single Speaker Speech Datasets for 10 Languages",
            "authors": [
                "Kyubyong Park",
                "Thomas Mulc"
            ],
            "abstract": "We describe our development of CSS10, a collection of single speaker speech\ndatasets for ten languages. It is composed of short audio clips from LibriVox\naudiobooks and their aligned texts. To validate its quality we train two neural\ntext-to-speech models on each dataset. Subsequently, we conduct Mean Opinion\nScore tests on the synthesized speech samples. We make our datasets,\npre-trained models, and test resources publicly available. We hope they will be\nused for future speech tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11283v2",
            "date": "2019-03-27 08:16:10+00:00",
            "title": "Grammatical Error Correction and Style Transfer via Zero-shot Monolingual Translation",
            "authors": [
                "Elizaveta Korotkova",
                "Agnes Luhtaru",
                "Maksym Del",
                "Krista Liin",
                "Daiga Deksne",
                "Mark Fishel"
            ],
            "abstract": "Both grammatical error correction and text style transfer can be viewed as\nmonolingual sequence-to-sequence transformation tasks, but the scarcity of\ndirectly annotated data for either task makes them unfeasible for most\nlanguages. We present an approach that does both tasks within the same trained\nmodel, and only uses regular language parallel data, without requiring\nerror-corrected or style-adapted texts. We apply our model to three languages\nand present a thorough evaluation on both tasks, showing that the model is\nreliable for a number of error types and style transfer aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11340v2",
            "date": "2019-03-27 10:50:23+00:00",
            "title": "Multilevel Text Normalization with Sequence-to-Sequence Networks and Multisource Learning",
            "authors": [
                "Tatyana Ruzsics",
                "Tanja Samard\u017ei\u0107"
            ],
            "abstract": "We define multilevel text normalization as sequence-to-sequence processing\nthat transforms naturally noisy text into a sequence of normalized units of\nmeaning (morphemes) in three steps: 1) writing normalization, 2) lemmatization,\n3) canonical segmentation. These steps are traditionally considered separate\nNLP tasks, with diverse solutions, evaluation schemes and data sources. We\nexploit the fact that all these tasks involve sub-word sequence-to-sequence\ntransformation to propose a systematic solution for all of them using neural\nencoder-decoder technology. The specific challenge that we tackle in this paper\nis integrating the traditional know-how on separate tasks into the neural\nsequence-to-sequence framework to improve the state of the art. We address this\nchallenge by enriching the general framework with mechanisms that allow\nprocessing the information on multiple levels of text organization (characters,\nmorphemes, words, sentences) in combination with structural information\n(multilevel language model, part-of-speech) and heterogeneous sources (text,\ndictionaries). We show that our solution consistently improves on the current\nmethods in all three steps. In addition, we analyze the performance of our\nsystem to show the specific contribution of the integrating components to the\noverall improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11393v1",
            "date": "2019-03-27 12:56:37+00:00",
            "title": "Learning semantic sentence representations from visually grounded language without lexical knowledge",
            "authors": [
                "Danny Merkx",
                "Stefan Frank"
            ],
            "abstract": "Current approaches to learning semantic representations of sentences often\nuse prior word-level knowledge. The current study aims to leverage visual\ninformation in order to capture sentence level semantics without the need for\nword embeddings. We use a multimodal sentence encoder trained on a corpus of\nimages with matching text captions to produce visually grounded sentence\nembeddings. Deep Neural Networks are trained to map the two modalities to a\ncommon embedding space such that for an image the corresponding caption can be\nretrieved and vice versa. We show that our model achieves results comparable to\nthe current state-of-the-art on two popular image-caption retrieval benchmark\ndata sets: MSCOCO and Flickr8k. We evaluate the semantic content of the\nresulting sentence embeddings using the data from the Semantic Textual\nSimilarity benchmark task and show that the multimodal embeddings correlate\nwell with human semantic similarity judgements. The system achieves\nstate-of-the-art results on several of these benchmarks, which shows that a\nsystem trained solely on multimodal data, without assuming any word\nrepresentations, is able to capture sentence level semantics. Importantly, this\nresult shows that we do not need prior knowledge of lexical level semantics in\norder to model sentence level semantics. These findings demonstrate the\nimportance of visual information in semantics.",
            "journal ref": "Natural Language Engineering, Volume 25 - Issue 4 - July 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11410v2",
            "date": "2019-03-27 13:21:51+00:00",
            "title": "Structural Neural Encoders for AMR-to-text Generation",
            "authors": [
                "Marco Damonte",
                "Shay B. Cohen"
            ],
            "abstract": "AMR-to-text generation is a problem recently introduced to the NLP community,\nin which the goal is to generate sentences from Abstract Meaning Representation\n(AMR) graphs. Sequence-to-sequence models can be used to this end by converting\nthe AMR graphs to strings. Approaching the problem while working directly with\ngraphs requires the use of graph-to-sequence models that encode the AMR graph\ninto a vector representation. Such encoding has been shown to be beneficial in\nthe past, and unlike sequential encoding, it allows us to explicitly capture\nreentrant structures in the AMR graphs. We investigate the extent to which\nreentrancies (nodes with multiple parents) have an impact on AMR-to-text\ngeneration by comparing graph encoders to tree encoders, where reentrancies are\nnot preserved. We show that improvements in the treatment of reentrancies and\nlong-range dependencies contribute to higher overall scores for graph encoders.\nOur best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of\nthe art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of\nthe art by 1.24 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11437v1",
            "date": "2019-03-27 14:11:18+00:00",
            "title": "Using Monolingual Data in Neural Machine Translation: a Systematic Study",
            "authors": [
                "Franck Burlot",
                "Fran\u00e7ois Yvon"
            ],
            "abstract": "Neural Machine Translation (MT) has radically changed the way systems are\ndeveloped. A major difference with the previous generation (Phrase-Based MT) is\nthe way monolingual target data, which often abounds, is used in these two\nparadigms. While Phrase-Based MT can seamlessly integrate very large language\nmodels trained on billions of sentences, the best option for Neural MT\ndevelopers seems to be the generation of artificial parallel data through\n\\textsl{back-translation} - a technique that fails to fully take advantage of\nexisting datasets. In this paper, we conduct a systematic study of\nback-translation, comparing alternative uses of monolingual data, as well as\nmultiple data generation procedures. Our findings confirm that back-translation\nis very effective and give new explanations as to why this is the case. We also\nintroduce new data simulation techniques that are almost as effective, yet much\ncheaper to implement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11770v2",
            "date": "2019-03-28 03:09:11+00:00",
            "title": "An Improved Approach for Semantic Graph Composition with CCG",
            "authors": [
                "Austin Blodgett",
                "Nathan Schneider"
            ],
            "abstract": "This paper builds on previous work using Combinatory Categorial Grammar (CCG)\nto derive a transparent syntax-semantics interface for Abstract Meaning\nRepresentation (AMR) parsing. We define new semantics for the CCG combinators\nthat is better suited to deriving AMR graphs. In particular, we define\nrelation-wise alternatives for the application and composition combinators:\nthese require that the two constituents being combined overlap in one AMR\nrelation. We also provide a new semantics for type raising, which is necessary\nfor certain constructions. Using these mechanisms, we suggest an analysis of\neventive nouns, which present a challenge for deriving AMR graphs. Our\ntheoretical analysis will facilitate future work on robust and transparent AMR\nparsing using CCG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11771v4",
            "date": "2019-03-28 03:09:33+00:00",
            "title": "A Large-Scale Multi-Length Headline Corpus for Analyzing Length-Constrained Headline Generation Model Evaluation",
            "authors": [
                "Yuta Hitomi",
                "Yuya Taguchi",
                "Hideaki Tamori",
                "Ko Kikuta",
                "Jiro Nishitoba",
                "Naoaki Okazaki",
                "Kentaro Inui",
                "Manabu Okumura"
            ],
            "abstract": "Browsing news articles on multiple devices is now possible. The lengths of\nnews article headlines have precise upper bounds, dictated by the size of the\ndisplay of the relevant device or interface. Therefore, controlling the length\nof headlines is essential when applying the task of headline generation to news\nproduction. However, because there is no corpus of headlines of multiple\nlengths for a given article, previous research on controlling output length in\nheadline generation has not discussed whether the system outputs could be\nadequately evaluated without multiple references of different lengths. In this\npaper, we introduce two corpora, which are Japanese News Corpus (JNC) and\nJApanese MUlti-Length Headline Corpus (JAMUL), to confirm the validity of\nprevious evaluation settings. The JNC provides common supervision data for\nheadline generation. The JAMUL is a large-scale evaluation dataset for\nheadlines of three different lengths composed by professional editors. We\nreport new findings on these corpora; for example, although the longest length\nreference summary can appropriately evaluate the existing methods controlling\noutput length, this evaluation setting has several problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11783v3",
            "date": "2019-03-28 04:39:34+00:00",
            "title": "A dataset for resolving referring expressions in spoken dialogue via contextual query rewrites (CQR)",
            "authors": [
                "Michael Regan",
                "Pushpendre Rastogi",
                "Arpit Gupta",
                "Lambert Mathias"
            ],
            "abstract": "We present Contextual Query Rewrite (CQR) a dataset for multi-domain\ntask-oriented spoken dialogue systems that is an extension of the Stanford\ndialog corpus (Eric et al., 2017a). While previous approaches have addressed\nthe issue of diverse schemas by learning candidate transformations (Naik et\nal., 2018), we instead model the reference resolution task as a user query\nreformulation task, where the dialog state is serialized into a natural\nlanguage query that can be executed by the downstream spoken language\nunderstanding system. In this paper, we describe our methodology for creating\nthe query reformulation extension to the dialog corpus, and present an initial\nset of experiments to establish a baseline for the CQR task. We have released\nthe corpus to the public [1] to support further research in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11848v2",
            "date": "2019-03-28 09:27:23+00:00",
            "title": "Sogou Machine Reading Comprehension Toolkit",
            "authors": [
                "Jindou Wu",
                "Yunlun Yang",
                "Chao Deng",
                "Hongyi Tang",
                "Bingning Wang",
                "Haoze Sun",
                "Ting Yao",
                "Qi Zhang"
            ],
            "abstract": "Machine reading comprehension have been intensively studied in recent years,\nand neural network-based models have shown dominant performances. In this\npaper, we present a Sogou Machine Reading Comprehension (SMRC) toolkit that can\nbe used to provide the fast and efficient development of modern machine\ncomprehension models, including both published models and original prototypes.\nTo achieve this goal, the toolkit provides dataset readers, a flexible\npreprocessing pipeline, necessary neural network components, and built-in\nmodels, which make the whole process of data preparation, model construction,\nand training easier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11850v1",
            "date": "2019-03-28 09:30:16+00:00",
            "title": "Mining Discourse Markers for Unsupervised Sentence Representation Learning",
            "authors": [
                "Damien Sileo",
                "Tim Van-De-Cruys",
                "Camille Pradel",
                "Philippe Muller"
            ],
            "abstract": "Current state of the art systems in NLP heavily rely on manually annotated\ndatasets, which are expensive to construct. Very little work adequately\nexploits unannotated data -- such as discourse markers between sentences --\nmainly because of data sparseness and ineffective extraction methods. In the\npresent work, we propose a method to automatically discover sentence pairs with\nrelevant discourse markers, and apply it to massive amounts of data. Our\nresulting dataset contains 174 discourse markers with at least 10k examples\neach, even for rare markers such as coincidentally or amazingly We use the\nresulting data as supervision for learning transferable sentence embeddings. In\naddition, we show that even though sentence representation learning through\nprediction of discourse markers yields state of the art results across\ndifferent transfer tasks, it is not clear that our models made use of the\nsemantic relation between sentences, thus leaving room for further\nimprovements. Our datasets are publicly available\n(https://github.com/synapse-developpement/Discovery)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12157v1",
            "date": "2019-03-28 17:43:09+00:00",
            "title": "Resilient Combination of Complementary CNN and RNN Features for Text Classification through Attention and Ensembling",
            "authors": [
                "Athanasios Giannakopoulos",
                "Maxime Coriou",
                "Andreea Hossmann",
                "Michael Baeriswyl",
                "Claudiu Musat"
            ],
            "abstract": "State-of-the-art methods for text classification include several distinct\nsteps of pre-processing, feature extraction and post-processing. In this work,\nwe focus on end-to-end neural architectures and show that the best performance\nin text classification is obtained by combining information from different\nneural modules. Concretely, we combine convolution, recurrent and attention\nmodules with ensemble methods and show that they are complementary. We\nintroduce ECGA, an end-to-end go-to architecture for novel text classification\ntasks. We prove that it is efficient and robust, as it attains or surpasses the\nstate-of-the-art on varied datasets, including both low and high data regimes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12238v2",
            "date": "2019-03-28 19:43:57+00:00",
            "title": "Modeling Acoustic-Prosodic Cues for Word Importance Prediction in Spoken Dialogues",
            "authors": [
                "Sushant Kafle",
                "Cecilia O. Alm",
                "Matt Huenerfauth"
            ],
            "abstract": "Prosodic cues in conversational speech aid listeners in discerning a message.\nWe investigate whether acoustic cues in spoken dialogue can be used to identify\nthe importance of individual words to the meaning of a conversation turn.\nIndividuals who are Deaf and Hard of Hearing often rely on real-time captions\nin live meetings. Word error rate, a traditional metric for evaluating\nautomatic speech recognition, fails to capture that some words are more\nimportant for a system to transcribe correctly than others. We present and\nevaluate neural architectures that use acoustic features for 3-class word\nimportance prediction. Our model performs competitively against\nstate-of-the-art text-based word-importance prediction models, and it\ndemonstrates particular benefits when operating on imperfect ASR output.",
            "journal ref": "Proceedings of the Eighth Workshop on Speech and Language\n  Processing for Assistive Technologies. 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12271v1",
            "date": "2019-03-28 21:12:59+00:00",
            "title": "In Search of Meaning: Lessons, Resources and Next Steps for Computational Analysis of Financial Discourse",
            "authors": [
                "Mahmoud El-Haj",
                "Paul Rayson",
                "Martin Walker",
                "Steven Young",
                "Vasiliki Simaki"
            ],
            "abstract": "We critically assess mainstream accounting and finance research applying\nmethods from computational linguistics (CL) to study financial discourse. We\nalso review common themes and innovations in the literature and assess the\nincremental contributions of work applying CL methods over manual content\nanalysis. Key conclusions emerging from our analysis are: (a) accounting and\nfinance research is behind the curve in terms of CL methods generally and word\nsense disambiguation in particular; (b) implementation issues mean the proposed\nbenefits of CL are often less pronounced than proponents suggest; (c)\nstructural issues limit practical relevance; and (d) CL methods and high\nquality manual analysis represent complementary approaches to analyzing\nfinancial discourse. We describe four CL tools that have yet to gain traction\nin mainstream AF research but which we believe offer promising ways to enhance\nthe study of meaning in financial discourse. The four tools are named entity\nrecognition (NER), summarization, semantics and corpus linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12306v1",
            "date": "2019-03-29 00:44:14+00:00",
            "title": "Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognition",
            "authors": [
                "Shane Settle",
                "Kartik Audhkhasi",
                "Karen Livescu",
                "Michael Picheny"
            ],
            "abstract": "Direct acoustics-to-word (A2W) systems for end-to-end automatic speech\nrecognition are simpler to train, and more efficient to decode with, than\nsub-word systems. However, A2W systems can have difficulties at training time\nwhen data is limited, and at decoding time when recognizing words outside the\ntraining vocabulary. To address these shortcomings, we investigate the use of\nrecently proposed acoustic and acoustically grounded word embedding techniques\nin A2W systems. The idea is based on treating the final pre-softmax weight\nmatrix of an AWE recognizer as a matrix of word embedding vectors, and using an\nexternally trained set of word embeddings to improve the quality of this\nmatrix. In particular we introduce two ideas: (1) Enforcing similarity at\ntraining time between the external embeddings and the recognizer weights, and\n(2) using the word embeddings at test time for predicting out-of-vocabulary\nwords. Our word embedding model is acoustically grounded, that is it is learned\njointly with acoustic embeddings so as to encode the words' acoustic-phonetic\ncontent; and it is parametric, so that it can embed any arbitrary (potentially\nout-of-vocabulary) sequence of characters. We find that both techniques improve\nthe performance of an A2W recognizer on conversational telephone speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12431v2",
            "date": "2019-03-29 10:15:45+00:00",
            "title": "Train One Get One Free: Partially Supervised Neural Network for Bug Report Duplicate Detection and Clustering",
            "authors": [
                "Lahari Poddar",
                "Leonardo Neves",
                "William Brendel",
                "Luis Marujo",
                "Sergey Tulyakov",
                "Pradeep Karuturi"
            ],
            "abstract": "Tracking user reported bugs requires considerable engineering effort in going\nthrough many repetitive reports and assigning them to the correct teams. This\npaper proposes a neural architecture that can jointly (1) detect if two bug\nreports are duplicates, and (2) aggregate them into latent topics. Leveraging\nthe assumption that learning the topic of a bug is a sub-task for detecting\nduplicates, we design a loss function that can jointly perform both tasks but\nneeds supervision for only duplicate classification, achieving topic clustering\nin an unsupervised fashion. We use a two-step attention module that uses\nself-attention for topic clustering and conditional attention for duplicate\ndetection. We study the characteristics of two types of real world datasets\nthat have been marked for duplicate bugs by engineers and by non-technical\nannotators. The results demonstrate that our model not only can outperform\nstate-of-the-art methods for duplicate classification on both cases, but can\nalso learn meaningful latent clusters without additional supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12452v1",
            "date": "2019-03-29 11:24:40+00:00",
            "title": "A framework for fake review detection in online consumer electronics retailers",
            "authors": [
                "Rodrigo Barbado",
                "Oscar Araque",
                "Carlos A. Iglesias"
            ],
            "abstract": "The impact of online reviews on businesses has grown significantly during\nlast years, being crucial to determine business success in a wide array of\nsectors, ranging from restaurants, hotels to e-commerce. Unfortunately, some\nusers use unethical means to improve their online reputation by writing fake\nreviews of their businesses or competitors. Previous research has addressed\nfake review detection in a number of domains, such as product or business\nreviews in restaurants and hotels. However, in spite of its economical\ninterest, the domain of consumer electronics businesses has not yet been\nthoroughly studied. This article proposes a feature framework for detecting\nfake reviews that has been evaluated in the consumer electronics domain. The\ncontributions are fourfold: (i) Construction of a dataset for classifying fake\nreviews in the consumer electronics domain in four different cities based on\nscraping techniques; (ii) definition of a feature framework for fake review\ndetection; (iii) development of a fake review classification method based on\nthe proposed framework and (iv) evaluation and analysis of the results for each\nof the cities under study. We have reached an 82% F-Score on the classification\ntask and the Ada Boost classifier has been proven to be the best one by\nstatistical means according to the Friedman test.",
            "journal ref": "Information Processing & Management, ISSN 0306-4573, Volume 56,\n  Issue 4, July 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12453v2",
            "date": "2019-03-29 11:26:14+00:00",
            "title": "Frowning Frodo, Wincing Leia, and a Seriously Great Friendship: Learning to Classify Emotional Relationships of Fictional Characters",
            "authors": [
                "Evgeny Kim",
                "Roman Klinger"
            ],
            "abstract": "The development of a fictional plot is centered around characters who closely\ninteract with each other forming dynamic social networks. In literature\nanalysis, such networks have mostly been analyzed without particular relation\ntypes or focusing on roles which the characters take with respect to each\nother. We argue that an important aspect for the analysis of stories and their\ndevelopment is the emotion between characters. In this paper, we combine these\naspects into a unified framework to classify emotional relationships of\nfictional characters. We formalize it as a new task and describe the annotation\nof a corpus, based on fan-fiction short stories. The extraction pipeline which\nwe propose consists of character identification (which we treat as given by an\noracle here) and the relation classification. For the latter, we provide\nresults using several approaches previously proposed for relation\nidentification with neural methods. The best result of 0.45 F1 is achieved with\na GRU with character position indicators on the task of predicting undirected\nemotion relations in the associated social network graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12457v3",
            "date": "2019-03-29 11:57:24+00:00",
            "title": "Towards Knowledge-Based Personalized Product Description Generation in E-commerce",
            "authors": [
                "Qibin Chen",
                "Junyang Lin",
                "Yichang Zhang",
                "Hongxia Yang",
                "Jingren Zhou",
                "Jie Tang"
            ],
            "abstract": "Quality product descriptions are critical for providing competitive customer\nexperience in an e-commerce platform. An accurate and attractive description\nnot only helps customers make an informed decision but also improves the\nlikelihood of purchase. However, crafting a successful product description is\ntedious and highly time-consuming. Due to its importance, automating the\nproduct description generation has attracted considerable interests from both\nresearch and industrial communities. Existing methods mainly use templates or\nstatistical methods, and their performance could be rather limited. In this\npaper, we explore a new way to generate the personalized product description by\ncombining the power of neural networks and knowledge base. Specifically, we\npropose a KnOwledge Based pErsonalized (or KOBE) product description generation\nmodel in the context of e-commerce. In KOBE, we extend the encoder-decoder\nframework, the Transformer, to a sequence modeling formulation using\nself-attention. In order to make the description both informative and\npersonalized, KOBE considers a variety of important factors during text\ngeneration, including product aspects, user categories, and knowledge base,\netc. Experiments on real-world datasets demonstrate that the proposed method\nout-performs the baseline on various metrics. KOBE can achieve an improvement\nof 9.7% over state-of-the-arts in terms of BLEU. We also present several case\nstudies as the anecdotal evidence to further prove the effectiveness of the\nproposed approach. The framework has been deployed in Taobao, the largest\nonline e-commerce platform in China.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12626v1",
            "date": "2019-03-29 17:22:40+00:00",
            "title": "Integrating Semantic Knowledge to Tackle Zero-shot Text Classification",
            "authors": [
                "Jingqing Zhang",
                "Piyawat Lertvittayakumjorn",
                "Yike Guo"
            ],
            "abstract": "Insufficient or even unavailable training data of emerging classes is a big\nchallenge of many classification tasks, including text classification.\nRecognising text documents of classes that have never been seen in the learning\nstage, so-called zero-shot text classification, is therefore difficult and only\nlimited previous works tackled this problem. In this paper, we propose a\ntwo-phase framework together with data augmentation and feature augmentation to\nsolve this problem. Four kinds of semantic knowledge (word embeddings, class\ndescriptions, class hierarchy, and a general knowledge graph) are incorporated\ninto the proposed framework to deal with instances of unseen classes\neffectively. Experimental results show that each and the combination of the two\nphases achieve the best overall accuracy compared with baselines and recent\napproaches in classifying real-world texts under the zero-shot scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00118v5",
            "date": "2019-03-29 23:28:01+00:00",
            "title": "Structured Minimally Supervised Learning for Neural Relation Extraction",
            "authors": [
                "Fan Bai",
                "Alan Ritter"
            ],
            "abstract": "We present an approach to minimally supervised relation extraction that\ncombines the benefits of learned representations and structured learning, and\naccurately predicts sentence-level relation mentions given only\nproposition-level supervision from a KB. By explicitly reasoning about missing\ndata during learning, our approach enables large-scale training of 1D\nconvolutional neural networks while mitigating the issue of label noise\ninherent in distant supervision. Our approach achieves state-of-the-art results\non minimally supervised sentential relation extraction, outperforming a number\nof baselines, including a competitive approach that uses the attention layer of\na purely neural model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00157v3",
            "date": "2019-03-30 06:48:32+00:00",
            "title": "Linguistic generalization and compositionality in modern artificial neural networks",
            "authors": [
                "Marco Baroni"
            ],
            "abstract": "In the last decade, deep artificial neural networks have achieved astounding\nperformance in many natural language processing tasks. Given the high\nproductivity of language, these models must possess effective generalization\nabilities. It is widely assumed that humans handle linguistic productivity by\nmeans of algebraic compositional rules: Are deep networks similarly\ncompositional? After reviewing the main innovations characterizing current deep\nlanguage processing networks, I discuss a set of studies suggesting that deep\nnetworks are capable of subtle grammar-dependent generalizations, but also that\nthey do not rely on systematic compositional rules. I argue that the intriguing\nbehaviour of these devices (still awaiting a full understanding) should be of\ninterest to linguists and cognitive scientists, as it offers a new perspective\non possible computational strategies to deal with linguistic productivity\nbeyond rule-based compositionality, and it might lead to new insights into the\nless systematic generalization patterns that also appear in natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00313v1",
            "date": "2019-03-31 00:48:42+00:00",
            "title": "Modeling Drug-Disease Relations with Linguistic and Knowledge Graph Constraints",
            "authors": [
                "Bruno Godefroy",
                "Christopher Potts"
            ],
            "abstract": "FDA drug labels are rich sources of information about drugs and drug-disease\nrelations, but their complexity makes them challenging texts to analyze in\nisolation. To overcome this, we situate these labels in two health knowledge\ngraphs: one built from precise structured information about drugs and diseases,\nand another built entirely from a database of clinical narrative texts using\nsimple heuristic methods. We show that Probabilistic Soft Logic models defined\nover these graphs are superior to text-only and relation-only variants, and\nthat the clinical narratives graph delivers exceptional results with little\nmanual effort. Finally, we release a new dataset of drug labels with\nannotations for five distinct drug-disease relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00365v1",
            "date": "2019-03-31 09:23:17+00:00",
            "title": "SART - Similarity, Analogies, and Relatedness for Tatar Language: New Benchmark Datasets for Word Embeddings Evaluation",
            "authors": [
                "Albina Khusainova",
                "Adil Khan",
                "Ad\u00edn Ram\u00edrez Rivera"
            ],
            "abstract": "There is a huge imbalance between languages currently spoken and\ncorresponding resources to study them. Most of the attention naturally goes to\nthe \"big\" languages: those which have the largest presence in terms of media\nand number of speakers. Other less represented languages sometimes do not even\nhave a good quality corpus to study them. In this paper, we tackle this\nimbalance by presenting a new set of evaluation resources for Tatar, a language\nof the Turkic language family which is mainly spoken in Tatarstan Republic,\nRussia.\n  We present three datasets: Similarity and Relatedness datasets that consist\nof human scored word pairs and can be used to evaluate semantic models; and\nAnalogies dataset that comprises analogy questions and allows to explore\nsemantic, syntactic, and morphological aspects of language modeling. All three\ndatasets build upon existing datasets for the English language and follow the\nsame structure. However, they are not mere translations. They take into account\nspecifics of the Tatar language and expand beyond the original datasets. We\nevaluate state-of-the-art word embedding models for two languages using our\nproposed datasets for Tatar and the original datasets for English and report\nour findings on performance comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00585v2",
            "date": "2019-04-01 06:45:45+00:00",
            "title": "Using Similarity Measures to Select Pretraining Data for NER",
            "authors": [
                "Xiang Dai",
                "Sarvnaz Karimi",
                "Ben Hachey",
                "Cecile Paris"
            ],
            "abstract": "Word vectors and Language Models (LMs) pretrained on a large amount of\nunlabelled data can dramatically improve various Natural Language Processing\n(NLP) tasks. However, the measure and impact of similarity between pretraining\ndata and target task data are left to intuition. We propose three\ncost-effective measures to quantify different aspects of similarity between\nsource pretraining and target task data. We demonstrate that these measures are\ngood predictors of the usefulness of pretrained models for Named Entity\nRecognition (NER) over 30 data pairs. Results also suggest that pretrained LMs\nare more effective and more predictable than pretrained word vectors, but\npretrained word vectors are better when pretraining data is dissimilar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00615v1",
            "date": "2019-04-01 07:49:19+00:00",
            "title": "Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle",
            "authors": [
                "Maximin Coavoux",
                "Shay B. Cohen"
            ],
            "abstract": "We introduce a novel transition system for discontinuous constituency\nparsing. Instead of storing subtrees in a stack --i.e. a data structure with\nlinear-time sequential access-- the proposed system uses a set of parsing\nitems, with constant-time random access. This change makes it possible to\nconstruct any discontinuous constituency tree in exactly $4n - 2$ transitions\nfor a sentence of length $n$. At each parsing step, the parser considers every\nitem in the set to be combined with a focus item and to construct a new\nconstituent in a bottom-up fashion. The parsing strategy is based on the\nassumption that most syntactic structures can be parsed incrementally and that\nthe set --the memory of the parser-- remains reasonably small on average.\nMoreover, we introduce a provably correct dynamic oracle for the new transition\nsystem, and present the first experiments in discontinuous constituency parsing\nusing a dynamic oracle. Our parser obtains state-of-the-art results on three\nEnglish and German discontinuous treebanks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00639v1",
            "date": "2019-04-01 08:47:40+00:00",
            "title": "Multimodal Machine Translation with Embedding Prediction",
            "authors": [
                "Tosho Hirasawa",
                "Hayahide Yamagishi",
                "Yukio Matsumura",
                "Mamoru Komachi"
            ],
            "abstract": "Multimodal machine translation is an attractive application of neural machine\ntranslation (NMT). It helps computers to deeply understand visual objects and\ntheir relations with natural languages. However, multimodal NMT systems suffer\nfrom a shortage of available training data, resulting in poor performance for\ntranslating rare words. In NMT, pretrained word embeddings have been shown to\nimprove NMT of low-resource domains, and a search-based approach is proposed to\naddress the rare word problem. In this study, we effectively combine these two\napproaches in the context of multimodal NMT and explore how we can take full\nadvantage of pretrained word embeddings to better translate rare words. We\nreport overall performance improvements of 1.24 METEOR and 2.49 BLEU and\nachieve an improvement of 7.67 F-score for rare word translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00648v1",
            "date": "2019-04-01 09:10:14+00:00",
            "title": "Recognizing Musical Entities in User-generated Content",
            "authors": [
                "Lorenzo Porcaro",
                "Horacio Saggion"
            ],
            "abstract": "Recognizing Musical Entities is important for Music Information Retrieval\n(MIR) since it can improve the performance of several tasks such as music\nrecommendation, genre classification or artist similarity. However, most entity\nrecognition systems in the music domain have concentrated on formal texts (e.g.\nartists' biographies, encyclopedic articles, etc.), ignoring rich and noisy\nuser-generated content. In this work, we present a novel method to recognize\nmusical entities in Twitter content generated by users following a classical\nmusic radio channel. Our approach takes advantage of both formal radio schedule\nand users' tweets to improve entity recognition. We instantiate several machine\nlearning algorithms to perform entity recognition combining task-specific and\ncorpus-based features. We also show how to improve recognition results by\njointly considering formal and user-generated content",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00669v2",
            "date": "2019-04-01 09:49:16+00:00",
            "title": "Syntactic Interchangeability in Word Embedding Models",
            "authors": [
                "Daniel Hershcovich",
                "Assaf Toledo",
                "Alon Halfon",
                "Noam Slonim"
            ],
            "abstract": "Nearest neighbors in word embedding models are commonly observed to be\nsemantically similar, but the relations between them can vary greatly. We\ninvestigate the extent to which word embedding models preserve syntactic\ninterchangeability, as reflected by distances between word vectors, and the\neffect of hyper-parameters---context window size in particular. We use part of\nspeech (POS) as a proxy for syntactic interchangeability, as generally\nspeaking, words with the same POS are syntactically valid in the same contexts.\nWe also investigate the relationship between interchangeability and similarity\nas judged by commonly-used word similarity benchmarks, and correlate the result\nwith the performance of word embedding models on these benchmarks. Our results\nwill inform future research and applications in the selection of word embedding\nmodel, suggesting a principle for an appropriate selection of the context\nwindow size parameter depending on the use-case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00676v1",
            "date": "2019-04-01 09:57:54+00:00",
            "title": "Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs",
            "authors": [
                "Debjit Paul",
                "Anette Frank"
            ],
            "abstract": "To make machines better understand sentiments, research needs to move from\npolarity identification to understanding the reasons that underlie the\nexpression of sentiment. Categorizing the goals or needs of humans is one way\nto explain the expression of sentiment in text. Humans are good at\nunderstanding situations described in natural language and can easily connect\nthem to the character's psychological needs using commonsense knowledge. We\npresent a novel method to extract, rank, filter and select multi-hop relation\npaths from a commonsense knowledge resource to interpret the expression of\nsentiment in terms of their underlying human needs. We efficiently integrate\nthe acquired knowledge paths in a neural model that interfaces context\nrepresentations with knowledge using a gated attention mechanism. We assess the\nmodel's performance on a recently published dataset for categorizing human\nneeds. Selectively integrating knowledge paths boosts performance and\nestablishes a new state-of-the-art. Our model offers interpretability through\nthe learned attention map over commonsense knowledge paths. Human evaluation\nhighlights the relevance of the encoded knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00688v1",
            "date": "2019-04-01 10:22:42+00:00",
            "title": "Automatic text summarization: What has been done and what has to be done",
            "authors": [
                "Abdelkrime Aries",
                "Djamel eddine Zegour",
                "Walid Khaled Hidouci"
            ],
            "abstract": "Summaries are important when it comes to process huge amounts of information.\nTheir most important benefit is saving time, which we do not have much\nnowadays. Therefore, a summary must be short, representative and readable.\nGenerating summaries automatically can be beneficial for humans, since it can\nsave time and help selecting relevant documents. Automatic summarization and,\nin particular, Automatic text summarization (ATS) is not a new research field;\nIt was known since the 50s. Since then, researchers have been active to find\nthe perfect summarization method. In this article, we will discuss different\nworks in automatic summarization, especially the recent ones. We will present\nsome problems and limits which prevent works to move forward. Most of these\nchallenges are much more related to the nature of processed languages. These\nchallenges are interesting for academics and developers, as a path to follow in\nthis field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00796v1",
            "date": "2019-03-29 05:03:15+00:00",
            "title": "Making Neural Machine Reading Comprehension Faster",
            "authors": [
                "Debajyoti Chatterjee"
            ],
            "abstract": "This study aims at solving the Machine Reading Comprehension problem where\nquestions have to be answered given a context passage. The challenge is to\ndevelop a computationally faster model which will have improved inference time.\nState of the art in many natural language understanding tasks, BERT model, has\nbeen used and knowledge distillation method has been applied to train two\nsmaller models. The developed models are compared with other models which have\nbeen developed with the same intention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00929v2",
            "date": "2019-04-01 15:59:18+00:00",
            "title": "Unsupervised Abbreviation Disambiguation Contextual disambiguation using word embeddings",
            "authors": [
                "Manuel Ciosici",
                "Tobias Sommer",
                "Ira Assent"
            ],
            "abstract": "Abbreviations often have several distinct meanings, often making their use in\ntext ambiguous. Expanding them to their intended meaning in context is\nimportant for Machine Reading tasks such as document search, recommendation and\nquestion answering. Existing approaches mostly rely on manually labeled\nexamples of abbreviations and their correct long-forms. Such data sets are\ncostly to create and result in trained models with limited applicability and\nflexibility. Importantly, most current methods must be subjected to a full\nempirical evaluation in order to understand their limitations, which is\ncumbersome in practice.\n  In this paper, we present an entirely unsupervised abbreviation\ndisambiguation method (called UAD) that picks up abbreviation definitions from\nunstructured text. Creating distinct tokens per meaning, we learn context\nrepresentations as word vectors. We demonstrate how to further boost\nabbreviation disambiguation performance by obtaining better context\nrepresentations using additional unstructured text. Our method is the first\nabbreviation disambiguation approach with a transparent model that allows\nperformance analysis without requiring full-scale evaluation, making it highly\nrelevant for real-world deployments.\n  In our thorough empirical evaluation, UAD achieves high performance on large\nreal-world data sets from different domains and outperforms both baseline and\nstate-of-the-art methods. UAD scales well and supports thousands of\nabbreviations with multiple different meanings within a single model.\n  In order to spur more research into abbreviation disambiguation, we publish a\nnew data set, that we also use in our experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00930v1",
            "date": "2019-04-01 16:03:59+00:00",
            "title": "Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation",
            "authors": [
                "Nikolai Vogler",
                "Craig Stewart",
                "Graham Neubig"
            ],
            "abstract": "Simultaneous interpretation, the translation of speech from one language to\nanother in real-time, is an inherently difficult and strenuous task. One of the\ngreatest challenges faced by interpreters is the accurate translation of\ndifficult terminology like proper names, numbers, or other entities.\nIntelligent computer-assisted interpreting (CAI) tools that could analyze the\nspoken word and detect terms likely to be untranslated by an interpreter could\nreduce translation error and improve interpreter performance. In this paper, we\npropose a task of predicting which terminology simultaneous interpreters will\nleave untranslated, and examine methods that perform this task using supervised\nsequence taggers. We describe a number of task-specific features explicitly\ndesigned to indicate when an interpreter may struggle with translating a word.\nExperimental results on a newly-annotated version of the NAIST Simultaneous\nTranslation Corpus (Shimizu et al., 2014) indicate the promise of our proposed\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01032v3",
            "date": "2019-04-01 18:01:08+00:00",
            "title": "Learning to Stop in Structured Prediction for Neural Machine Translation",
            "authors": [
                "Mingbo Ma",
                "Renjie Zheng",
                "Liang Huang"
            ],
            "abstract": "Beam search optimization resolves many issues in neural machine translation.\nHowever, this method lacks principled stopping criteria and does not learn how\nto stop during training, and the model naturally prefers the longer hypotheses\nduring the testing time in practice since they use the raw score instead of the\nprobability-based score. We propose a novel ranking method which enables an\noptimal beam search stopping criteria. We further introduce a structured\nprediction loss function which penalizes suboptimal finished candidates\nproduced by beam search during training. Experiments of neural machine\ntranslation on both synthetic data and real languages (German-to-English and\nChinese-to-English) demonstrate our proposed methods lead to better length and\nBLEU score.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01038v1",
            "date": "2019-04-01 18:05:02+00:00",
            "title": "fairseq: A Fast, Extensible Toolkit for Sequence Modeling",
            "authors": [
                "Myle Ott",
                "Sergey Edunov",
                "Alexei Baevski",
                "Angela Fan",
                "Sam Gross",
                "Nathan Ng",
                "David Grangier",
                "Michael Auli"
            ],
            "abstract": "fairseq is an open-source sequence modeling toolkit that allows researchers\nand developers to train custom models for translation, summarization, language\nmodeling, and other text generation tasks. The toolkit is based on PyTorch and\nsupports distributed training across multiple GPUs and machines. We also\nsupport fast mixed-precision training and inference on modern GPUs. A demo\nvideo can be found at https://www.youtube.com/watch?v=OtgDdWtHvto",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01130v1",
            "date": "2019-04-01 22:21:14+00:00",
            "title": "PAWS: Paraphrase Adversaries from Word Scrambling",
            "authors": [
                "Yuan Zhang",
                "Jason Baldridge",
                "Luheng He"
            ],
            "abstract": "Existing paraphrase identification datasets lack sentence pairs that have\nhigh lexical overlap without being paraphrases. Models trained on such data\nfail to distinguish pairs like flights from New York to Florida and flights\nfrom Florida to New York. This paper introduces PAWS (Paraphrase Adversaries\nfrom Word Scrambling), a new dataset with 108,463 well-formed paraphrase and\nnon-paraphrase pairs with high lexical overlap. Challenging pairs are generated\nby controlled word swapping and back translation, followed by fluency and\nparaphrase judgments by human raters. State-of-the-art models trained on\nexisting datasets have dismal performance on PAWS (<40% accuracy); however,\nincluding PAWS training data for these models improves their accuracy to 85%\nwhile maintaining performance on existing tasks. In contrast, models that do\nnot capture non-local contextual information fail even with PAWS training\nexamples. As such, PAWS provides an effective instrument for driving further\nprogress on models that better exploit structure, context, and pairwise\ncomparisons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01172v3",
            "date": "2019-04-02 02:09:01+00:00",
            "title": "Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches",
            "authors": [
                "Shane Storks",
                "Qiaozi Gao",
                "Joyce Y. Chai"
            ],
            "abstract": "In the NLP community, recent years have seen a surge of research activities\nthat address machines' ability to perform deep language understanding which\ngoes beyond what is explicitly stated in text, rather relying on reasoning and\nknowledge of the world. Many benchmark tasks and datasets have been created to\nsupport the development and evaluation of such natural language inference\nability. As these benchmarks become instrumental and a driving force for the\nNLP research community, this paper aims to provide an overview of recent\nbenchmarks, relevant knowledge resources, and state-of-the-art learning and\ninference approaches in order to support a better understanding of this growing\nfield.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01173v1",
            "date": "2019-04-02 02:09:05+00:00",
            "title": "A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations",
            "authors": [
                "Mingda Chen",
                "Qingming Tang",
                "Sam Wiseman",
                "Kevin Gimpel"
            ],
            "abstract": "We propose a generative model for a sentence that uses two latent variables,\nwith one intended to represent the syntax of the sentence and the other to\nrepresent its semantics. We show we can achieve better disentanglement between\nsemantic and syntactic representations by training with multiple losses,\nincluding losses that exploit aligned paraphrastic sentences and word-order\ninformation. We also investigate the effect of moving from bag-of-words to\nrecurrent neural network modules. We evaluate our models as well as several\npopular pretrained embeddings on standard semantic similarity tasks and novel\nsyntactic similarity tasks. Empirically, we find that the model with the best\nperforming syntactic and semantic representations also gives rise to the most\ndisentangled representations.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01246v1",
            "date": "2019-04-02 07:08:25+00:00",
            "title": "UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answering",
            "authors": [
                "Zi-Yuan Chen",
                "Chih-Hung Chang",
                "Yi-Pei Chen",
                "Jijnasa Nayak",
                "Lun-Wei Ku"
            ],
            "abstract": "In relation extraction for knowledge-based question answering, searching from\none entity to another entity via a single relation is called \"one hop\". In\nrelated work, an exhaustive search from all one-hop relations, two-hop\nrelations, and so on to the max-hop relations in the knowledge graph is\nnecessary but expensive. Therefore, the number of hops is generally restricted\nto two or three. In this paper, we propose UHop, an unrestricted-hop framework\nwhich relaxes this restriction by use of a transition-based search framework to\nreplace the relation-chain-based search one. We conduct experiments on\nconventional 1- and 2-hop questions as well as lengthy questions, including\ndatasets such as WebQSP, PathQuestion, and Grid World. Results show that the\nproposed framework enables the ability to halt, works well with\nstate-of-the-art models, achieves competitive performance without exhaustive\nsearches, and opens the performance gap for long relation paths.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01297v1",
            "date": "2019-04-02 08:57:55+00:00",
            "title": "Temporal and Aspectual Entailment",
            "authors": [
                "Thomas Kober",
                "Sander Bijl de Vroe",
                "Mark Steedman"
            ],
            "abstract": "Inferences regarding \"Jane's arrival in London\" from predications such as\n\"Jane is going to London\" or \"Jane has gone to London\" depend on tense and\naspect of the predications. Tense determines the temporal location of the\npredication in the past, present or future of the time of utterance. The\naspectual auxiliaries on the other hand specify the internal constituency of\nthe event, i.e. whether the event of \"going to London\" is completed and whether\nits consequences hold at that time or not. While tense and aspect are among the\nmost important factors for determining natural language inference, there has\nbeen very little work to show whether modern NLP models capture these semantic\nconcepts. In this paper we propose a novel entailment dataset and analyse the\nability of a range of recently proposed NLP models to perform inference on\ntemporal predications. We show that the models encode a substantial amount of\nmorphosyntactic information relating to tense and aspect, but fail to model\ninferences that require reasoning with these semantic properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01301v2",
            "date": "2019-04-02 09:04:57+00:00",
            "title": "Pragmatically Informative Text Generation",
            "authors": [
                "Sheng Shen",
                "Daniel Fried",
                "Jacob Andreas",
                "Dan Klein"
            ],
            "abstract": "We improve the informativeness of models for conditional text generation\nusing techniques from computational pragmatics. These techniques formulate\nlanguage production as a game between speakers and listeners, in which a\nspeaker should generate output text that a listener can use to correctly\nidentify the original input that the text describes. While such approaches are\nwidely used in cognitive science and grounded language learning, they have\nreceived less attention for more standard language generation tasks. We\nconsider two pragmatic modeling methods for text generation: one where\npragmatics is imposed by information preservation, and another where pragmatics\nis imposed by explicit modeling of distractors. We find that these methods\nimprove the performance of strong existing systems for abstractive\nsummarization and generation from structured meaning representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01464v3",
            "date": "2019-04-02 14:38:37+00:00",
            "title": "Training Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text",
            "authors": [
                "Toms Bergmanis",
                "Sharon Goldwater"
            ],
            "abstract": "Lemmatization aims to reduce the sparse data problem by relating the\ninflected forms of a word to its dictionary form. Using context can help, both\nfor unseen and ambiguous words. Yet most context-sensitive approaches require\nfull lemma-annotated sentences for training, which may be scarce or unavailable\nin low-resource languages. In addition (as shown here), in a low-resource\nsetting, a lemmatizer can learn more from $n$ labeled examples of distinct\nwords (types) than from $n$ (contiguous) labeled tokens, since the latter\ncontain far fewer distinct types. To combine the efficiency of type-based\nlearning with the benefits of context, we propose a way to train a\ncontext-sensitive lemmatizer with little or no labeled corpus data, using\ninflection tables from the UniMorph project and raw text examples from\nWikipedia that provide sentence contexts for the unambiguous UniMorph examples.\nDespite these being unambiguous examples, the model successfully generalizes\nfrom them, leading to improved results (both overall, and especially on unseen\nwords) in comparison to a baseline that does not use context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01587v1",
            "date": "2019-04-02 18:00:02+00:00",
            "title": "Asking the Right Question: Inferring Advice-Seeking Intentions from Personal Narratives",
            "authors": [
                "Liye Fu",
                "Jonathan P. Chang",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "People often share personal narratives in order to seek advice from others.\nTo properly infer the narrator's intention, one needs to apply a certain degree\nof common sense and social intuition. To test the capabilities of NLP systems\nto recover such intuition, we introduce the new task of inferring what is the\nadvice-seeking goal behind a personal narrative. We formulate this as a cloze\ntest, where the goal is to identify which of two advice-seeking questions was\nremoved from a given narrative.\n  The main challenge in constructing this task is finding pairs of semantically\nplausible advice-seeking questions for given narratives. To address this\nchallenge, we devise a method that exploits commonalities in experiences people\nshare online to automatically extract pairs of questions that are appropriate\ncandidates for the cloze task. This results in a dataset of over 20,000\npersonal narratives, each matched with a pair of related advice-seeking\nquestions: one actually intended by the narrator, and the other one not. The\ndataset covers a very broad array of human experiences, from dating, to career\noptions, to stolen iPads. We use human annotation to determine the degree to\nwhich the task relies on common sense and social intuition in addition to a\nsemantic understanding of the narrative. By introducing several baselines for\nthis new task we demonstrate its feasibility and identify avenues for better\nmodeling the intention of the narrator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01596v2",
            "date": "2019-04-02 18:00:09+00:00",
            "title": "Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings",
            "authors": [
                "Dorottya Demszky",
                "Nikhil Garg",
                "Rob Voigt",
                "James Zou",
                "Matthew Gentzkow",
                "Jesse Shapiro",
                "Dan Jurafsky"
            ],
            "abstract": "We provide an NLP framework to uncover four linguistic dimensions of\npolitical polarization in social media: topic choice, framing, affect and\nillocutionary force. We quantify these aspects with existing lexical methods,\nand propose clustering of tweet embeddings as a means to identify salient\ntopics for analysis across events; human evaluations show that our approach\ngenerates more cohesive topics than traditional LDA-based models. We apply our\nmethods to study 4.4M tweets on 21 mass shootings. We provide evidence that the\ndiscussion of these events is highly polarized politically and that this\npolarization is primarily driven by partisan differences in framing rather than\ntopic choice. We identify framing devices, such as grounding and the\ncontrasting use of the terms \"terrorist\" and \"crazy\", that contribute to\npolarization. Results pertaining to topic choice, affect and illocutionary\nforce suggest that Republicans focus more on the shooter and event-specific\nfacts (news) while Democrats focus more on the victims and call for policy\nchanges. Our work contributes to a deeper understanding of the way group\ndivisions manifest in language and to computational methods for studying them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01606v2",
            "date": "2019-04-02 18:17:49+00:00",
            "title": "Inferring Which Medical Treatments Work from Reports of Clinical Trials",
            "authors": [
                "Eric Lehman",
                "Jay DeYoung",
                "Regina Barzilay",
                "Byron C. Wallace"
            ],
            "abstract": "How do we know if a particular medical treatment actually works? Ideally one\nwould consult all available evidence from relevant clinical trials.\nUnfortunately, such results are primarily disseminated in natural language\nscientific articles, imposing substantial burden on those trying to make sense\nof them. In this paper, we present a new task and corpus for making this\nunstructured evidence actionable. The task entails inferring reported findings\nfrom a full-text article describing a randomized controlled trial (RCT) with\nrespect to a given intervention, comparator, and outcome of interest, e.g.,\ninferring if an article provides evidence supporting the use of aspirin to\nreduce risk of stroke, as compared to placebo.\n  We present a new corpus for this task comprising 10,000+ prompts coupled with\nfull-text articles describing RCTs. Results using a suite of models --- ranging\nfrom heuristic (rule-based) approaches to attentive neural architectures ---\ndemonstrate the difficulty of the task, which we believe largely owes to the\nlengthy, technical input texts. To facilitate further work on this important,\nchallenging problem we make the corpus, documentation, a website and\nleaderboard, and code for baselines and evaluation available at\nhttp://evidence-inference.ebm-nlp.com/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01608v2",
            "date": "2019-04-02 18:22:09+00:00",
            "title": "Structural Scaffolds for Citation Intent Classification in Scientific Publications",
            "authors": [
                "Arman Cohan",
                "Waleed Ammar",
                "Madeleine van Zuylen",
                "Field Cady"
            ],
            "abstract": "Identifying the intent of a citation in scientific papers (e.g., background\ninformation, use of methods, comparing results) is critical for machine reading\nof individual publications and automated analysis of the scientific literature.\nWe propose structural scaffolds, a multitask model to incorporate structural\ninformation of scientific papers into citations for effective classification of\ncitation intents. Our model achieves a new state-of-the-art on an existing ACL\nanthology dataset (ACL-ARC) with a 13.3% absolute increase in F1 score, without\nrelying on external linguistic resources or hand-engineered features as done in\nexisting methods. In addition, we introduce a new dataset of citation intents\n(SciCite) which is more than five times larger and covers multiple scientific\ndomains compared with existing datasets. Our code and data are available at:\nhttps://github.com/allenai/scicite.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01617v2",
            "date": "2019-04-02 18:44:04+00:00",
            "title": "Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Learning high-quality embeddings for rare words is a hard problem because of\nsparse context information. Mimicking (Pinter et al., 2017) has been proposed\nas a solution: given embeddings learned by a standard algorithm, a model is\nfirst trained to reproduce embeddings of frequent words from their surface form\nand then used to compute embeddings for rare words. In this paper, we introduce\nattentive mimicking: the mimicking model is given access not only to a word's\nsurface form, but also to all available contexts and learns to attend to the\nmost informative and reliable contexts for computing an embedding. In an\nevaluation on four tasks, we show that attentive mimicking outperforms previous\nwork for both rare and medium-frequency words. Thus, compared to previous work,\nattentive mimicking improves embeddings for a much larger part of the\nvocabulary, including the medium-frequency range.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01684v3",
            "date": "2019-04-02 21:59:35+00:00",
            "title": "Impact of ASR on Alzheimer's Disease Detection: All Errors are Equal, but Deletions are More Equal than Others",
            "authors": [
                "Aparna Balagopalan",
                "Ksenia Shkaruta",
                "Jekaterina Novikova"
            ],
            "abstract": "Automatic Speech Recognition (ASR) is a critical component of any\nfully-automated speech-based dementia detection model. However, despite years\nof speech recognition research, little is known about the impact of ASR\naccuracy on dementia detection. In this paper, we experiment with controlled\namounts of artificially generated ASR errors and investigate their influence on\ndementia detection. We find that deletion errors affect detection performance\nthe most, due to their impact on the features of syntactic complexity and\ndiscourse representation in speech. We show the trend to be generalisable\nacross two different datasets for cognitive impairment detection. As a\nconclusion, we propose optimising the ASR to reflect a higher penalty for\ndeletion errors in order to improve dementia detection performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01735v1",
            "date": "2019-04-03 01:29:48+00:00",
            "title": "Multi-Modal Generative Adversarial Network for Short Product Title Generation in Mobile E-Commerce",
            "authors": [
                "Jian-Guo Zhang",
                "Pengcheng Zou",
                "Zhao Li",
                "Yao Wan",
                "Xiuming Pan",
                "Yu Gong",
                "Philip S. Yu"
            ],
            "abstract": "Nowadays, more and more customers browse and purchase products in favor of\nusing mobile E-Commerce Apps such as Taobao and Amazon. Since merchants are\nusually inclined to describe redundant and over-informative product titles to\nattract attentions from customers, it is important to concisely display short\nproduct titles on limited screen of mobile phones. To address this discrepancy,\nprevious studies mainly consider textual information of long product titles and\nlacks of human-like view during training and evaluation process. In this paper,\nwe propose a Multi-Modal Generative Adversarial Network (MM-GAN) for short\nproduct title generation in E-Commerce, which innovatively incorporates image\ninformation and attribute tags from product, as well as textual information\nfrom original long titles. MM-GAN poses short title generation as a\nreinforcement learning process, where the generated titles are evaluated by the\ndiscriminator in a human-like view. Extensive experiments on a large-scale\nE-Commerce dataset demonstrate that our algorithm outperforms other\nstate-of-the-art methods. Moreover, we deploy our model into a real-world\nonline E-Commerce environment and effectively boost the performance of click\nthrough rate and click conversion rate by 1.66% and 1.87%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01783v1",
            "date": "2019-04-03 05:58:07+00:00",
            "title": "Multi-task Learning for Chinese Word Usage Errors Detection",
            "authors": [
                "Jinbin Zhang",
                "Heng Wang"
            ],
            "abstract": "Chinese word usage errors often occur in non-native Chinese learners'\nwriting. It is very helpful for non-native Chinese learners to detect them\nautomatically when learning writing. In this paper, we propose a novel\napproach, which takes advantages of different auxiliary tasks, such as\nPOS-tagging prediction and word log frequency prediction, to help the task of\nChinese word usage error detection. With the help of these auxiliary tasks, we\nachieve the state-of-the-art results on the performances on the HSK corpus\ndata, without any other extra data.",
            "journal ref": "Proceedings of the 3rd IEEE International Conference on\n  Computational Intelligence and Applications (ICCIA 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01825v1",
            "date": "2019-04-03 08:11:57+00:00",
            "title": "Cross-lingual transfer learning for spoken language understanding",
            "authors": [
                "Quynh Ngoc Thi Do",
                "Judith Gaspers"
            ],
            "abstract": "Typically, spoken language understanding (SLU) models are trained on\nannotated data which are costly to gather. Aiming to reduce data needs for\nbootstrapping a SLU system for a new language, we present a simple but\neffective weight transfer approach using data from another language. The\napproach is evaluated with our promising multi-task SLU framework developed\ntowards different languages. We evaluate our approach on the ATIS and a\nreal-world SLU dataset, showing that i) our monolingual models outperform the\nstate-of-the-art, ii) we can reduce data amounts needed for bootstrapping a SLU\nsystem for a new language greatly, and iii) while multitask training improves\nover separate training, different weight transfer settings may work best for\ndifferent SLU modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01938v1",
            "date": "2019-04-03 11:57:25+00:00",
            "title": "Unsupervised Deep Structured Semantic Models for Commonsense Reasoning",
            "authors": [
                "Shuohang Wang",
                "Sheng Zhang",
                "Yelong Shen",
                "Xiaodong Liu",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Jing Jiang"
            ],
            "abstract": "Commonsense reasoning is fundamental to natural language understanding. While\ntraditional methods rely heavily on human-crafted features and knowledge bases,\nwe explore learning commonsense knowledge from a large amount of raw text via\nunsupervised learning. We propose two neural network models based on the Deep\nStructured Semantic Models (DSSM) framework to tackle two classic commonsense\nreasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation\n(PDP). Evaluation shows that the proposed models effectively capture contextual\ninformation in the sentence and co-reference information between pronouns and\nnouns, and achieve significant improvement over previous state-of-the-art\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02036v1",
            "date": "2019-04-03 14:46:38+00:00",
            "title": "A Large-Scale Comparison of Historical Text Normalization Systems",
            "authors": [
                "Marcel Bollmann"
            ],
            "abstract": "There is no consensus on the state-of-the-art approach to historical text\nnormalization. Many techniques have been proposed, including rule-based\nmethods, distance metrics, character-based statistical machine translation, and\nneural encoder--decoder models, but studies have used different datasets,\ndifferent evaluation methods, and have come to different conclusions. This\npaper presents the largest study of historical text normalization done so far.\nWe critically survey the existing literature and report experiments on eight\nlanguages, comparing systems spanning all categories of proposed normalization\ntechniques, analysing the effect of training data quantity, and using different\nevaluation methods. The datasets and scripts are made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02141v3",
            "date": "2019-04-03 17:56:38+00:00",
            "title": "CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition",
            "authors": [
                "Yuying Zhu",
                "Guoxin Wang",
                "B\u00f6rje F. Karlsson"
            ],
            "abstract": "Named entity recognition (NER) in Chinese is essential but difficult because\nof the lack of natural delimiters. Therefore, Chinese Word Segmentation (CWS)\nis usually considered as the first step for Chinese NER. However, models based\non word-level embeddings and lexicon features often suffer from segmentation\nerrors and out-of-vocabulary (OOV) words. In this paper, we investigate a\nConvolutional Attention Network called CAN for Chinese NER, which consists of a\ncharacter-based convolutional neural network (CNN) with local-attention layer\nand a gated recurrent unit (GRU) with global self-attention layer to capture\nthe information from adjacent characters and sentence contexts. Also, compared\nto other models, not depending on any external resources like lexicons and\nemploying small size of char embeddings make our model more practical.\nExtensive experimental results show that our approach outperforms\nstate-of-the-art methods without word embedding and external lexicon resources\non different domain datasets including Weibo, MSRA and Chinese Resume NER\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02142v2",
            "date": "2019-04-03 17:56:48+00:00",
            "title": "Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders",
            "authors": [
                "Andrew Drozdov",
                "Pat Verga",
                "Mohit Yadav",
                "Mohit Iyyer",
                "Andrew McCallum"
            ],
            "abstract": "We introduce deep inside-outside recursive autoencoders (DIORA), a\nfully-unsupervised method for discovering syntax that simultaneously learns\nrepresentations for constituents within the induced tree. Our approach predicts\neach word in an input sentence conditioned on the rest of the sentence and uses\ninside-outside dynamic programming to consider all possible binary trees over\nthe sentence. At test time the CKY algorithm extracts the highest scoring\nparse. DIORA achieves a new state-of-the-art F1 in unsupervised binary\nconstituency parsing (unlabeled) in two benchmark datasets, WSJ and MultiNLI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02181v1",
            "date": "2019-04-03 18:05:02+00:00",
            "title": "Probing Biomedical Embeddings from Language Models",
            "authors": [
                "Qiao Jin",
                "Bhuwan Dhingra",
                "William W. Cohen",
                "Xinghua Lu"
            ],
            "abstract": "Contextualized word embeddings derived from pre-trained language models (LMs)\nshow significant improvements on downstream NLP tasks. Pre-training on\ndomain-specific corpora, such as biomedical articles, further improves their\nperformance. In this paper, we conduct probing experiments to determine what\nadditional information is carried intrinsically by the in-domain trained\ncontextualized embeddings. For this we use the pre-trained LMs as fixed feature\nextractors and restrict the downstream task models to not have additional\nsequence modeling layers. We compare BERT, ELMo, BioBERT and BioELMo, a\nbiomedical version of ELMo trained on 10M PubMed abstracts. Surprisingly, while\nfine-tuned BioBERT is better than BioELMo in biomedical NER and NLI tasks, as a\nfixed feature extractor BioELMo outperforms BioBERT in our probing tasks. We\nuse visualization and nearest neighbor analysis to show that better encoding of\nentity-type and relational information leads to this superiority.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02228v2",
            "date": "2019-04-03 20:12:10+00:00",
            "title": "The Effect of Downstream Classification Tasks for Evaluating Sentence Embeddings",
            "authors": [
                "Peter Potash"
            ],
            "abstract": "One popular method for quantitatively evaluating the utility of sentence\nembeddings involves using them in downstream language processing tasks that\nrequire sentence representations as input. One simple such task is\nclassification, where the sentence representations are used to train and test\nmodels on several classification datasets. We argue that by evaluating sentence\nrepresentations in such a manner, the goal of the representations becomes\nlearning a low-dimensional factorization of a sentence-task label matrix. We\nshow how characteristics of this matrix can affect the ability for a\nlow-dimensional factorization to perform as sentence representations in a suite\nof classification tasks. Primarily, sentences that have more labels across all\npossible classification tasks have a higher reconstruction loss, however the\ngeneral nature of this effect is ultimately dependent on the overall\ndistribution of labels across all possible sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02232v2",
            "date": "2019-04-03 20:29:10+00:00",
            "title": "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Question-answering plays an important role in e-commerce as it allows\npotential customers to actively seek crucial information about products or\nservices to help their purchase decision making. Inspired by the recent success\nof machine reading comprehension (MRC) on formal documents, this paper explores\nthe potential of turning customer reviews into a large source of knowledge that\ncan be exploited to answer user questions.~We call this problem Review Reading\nComprehension (RRC). To the best of our knowledge, no existing work has been\ndone on RRC. In this work, we first build an RRC dataset called ReviewRC based\non a popular benchmark for aspect-based sentiment analysis. Since ReviewRC has\nlimited training examples for RRC (and also for aspect-based sentiment\nanalysis), we then explore a novel post-training approach on the popular\nlanguage model BERT to enhance the performance of fine-tuning of BERT for RRC.\nTo show the generality of the approach, the proposed post-training is also\napplied to some other review-based tasks such as aspect extraction and aspect\nsentiment classification in aspect-based sentiment analysis. Experimental\nresults demonstrate that the proposed post-training is highly effective. The\ndatasets and code are available at https://www.cs.uic.edu/~hxu/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02244v1",
            "date": "2019-04-03 21:29:14+00:00",
            "title": "Multi-task Learning for Japanese Predicate Argument Structure Analysis",
            "authors": [
                "Hikaru Omori",
                "Mamoru Komachi"
            ],
            "abstract": "An event-noun is a noun that has an argument structure similar to a\npredicate. Recent works, including those considered state-of-the-art, ignore\nevent-nouns or build a single model for solving both Japanese predicate\nargument structure analysis (PASA) and event-noun argument structure analysis\n(ENASA). However, because there are interactions between predicates and\nevent-nouns, it is not sufficient to target only predicates. To address this\nproblem, we present a multi-task learning method for PASA and ENASA. Our\nmulti-task models improved the performance of both tasks compared to a\nsingle-task model by sharing knowledge from each task. Moreover, in PASA, our\nmodels achieved state-of-the-art results in overall F1 scores on the NAIST Text\nCorpus. In addition, this is the first work to employ neural networks in ENASA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02246v2",
            "date": "2019-04-03 21:38:25+00:00",
            "title": "Learning Outside the Box: Discourse-level Features Improve Metaphor Identification",
            "authors": [
                "Jesse Mu",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "Most current approaches to metaphor identification use restricted linguistic\ncontexts, e.g. by considering only a verb's arguments or the sentence\ncontaining a phrase. Inspired by pragmatic accounts of metaphor, we argue that\nbroader discourse features are crucial for better metaphor identification. We\ntrain simple gradient boosting classifiers on representations of an utterance\nand its surrounding discourse learned with a variety of document embedding\nmethods, obtaining near state-of-the-art results on the 2018 VU Amsterdam\nmetaphor identification task without the complex metaphor-specific features or\ndeep neural architectures employed by other systems. A qualitative analysis\nfurther confirms the need for broader context in metaphor processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02281v1",
            "date": "2019-04-04 00:30:20+00:00",
            "title": "Answer-based Adversarial Training for Generating Clarification Questions",
            "authors": [
                "Sudha Rao",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We present an approach for generating clarification questions with the goal\nof eliciting new information that would make the given textual context more\ncomplete. We propose that modeling hypothetical answers (to clarification\nquestions) as latent variables can guide our approach into generating more\nuseful clarification questions. We develop a Generative Adversarial Network\n(GAN) where the generator is a sequence-to-sequence model and the discriminator\nis a utility function that models the value of updating the context with the\nanswer to the clarification question. We evaluate on two datasets, using both\nautomatic metrics and human judgments of usefulness, specificity and relevance,\nshowing that our approach outperforms both a retrieval-based model and\nablations that exclude the utility model and the adversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02295v1",
            "date": "2019-04-04 01:18:56+00:00",
            "title": "Evaluating Style Transfer for Text",
            "authors": [
                "Remi Mir",
                "Bjarke Felbo",
                "Nick Obradovich",
                "Iyad Rahwan"
            ],
            "abstract": "Research in the area of style transfer for text is currently bottlenecked by\na lack of standard evaluation practices. This paper aims to alleviate this\nissue by experimentally identifying best practices with a Yelp sentiment\ndataset. We specify three aspects of interest (style transfer intensity,\ncontent preservation, and naturalness) and show how to obtain more reliable\nmeasures of them from human evaluation than in previous work. We propose a set\nof metrics for automated evaluation and demonstrate that they are more strongly\ncorrelated and in agreement with human judgment: direction-corrected Earth\nMover's Distance, Word Mover's Distance on style-masked texts, and adversarial\nclassification for the respective aspects. We also show that the three examined\nmodels exhibit tradeoffs between aspects of interest, demonstrating the\nimportance of evaluating style transfer models at specific points of their\ntradeoff plots. We release software with our evaluation metrics to facilitate\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02306v4",
            "date": "2019-04-04 02:03:19+00:00",
            "title": "A Simple Joint Model for Improved Contextual Neural Lemmatization",
            "authors": [
                "Chaitanya Malaviya",
                "Shijie Wu",
                "Ryan Cotterell"
            ],
            "abstract": "English verbs have multiple forms. For instance, talk may also appear as\ntalks, talked or talking, depending on the context. The NLP task of\nlemmatization seeks to map these diverse forms back to a canonical one, known\nas the lemma. We present a simple joint neural model for lemmatization and\nmorphological tagging that achieves state-of-the-art results on 20 languages\nfrom the Universal Dependencies corpora. Our paper describes the model in\naddition to training and decoding procedures. Error analysis indicates that\njoint morphological tagging and lemmatization is especially helpful in\nlow-resource lemmatization and languages that display a larger degree of\nmorphological complexity. Code and pre-trained models are available at\nhttps://sigmorphon.github.io/sharedtasks/2019/task2/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02321v1",
            "date": "2019-04-04 02:57:59+00:00",
            "title": "Guiding Extractive Summarization with Question-Answering Rewards",
            "authors": [
                "Kristjan Arumae",
                "Fei Liu"
            ],
            "abstract": "Highlighting while reading is a natural behavior for people to track salient\ncontent of a document. It would be desirable to teach an extractive summarizer\nto do the same. However, a major obstacle to the development of a supervised\nsummarizer is the lack of ground-truth. Manual annotation of extraction units\nis cost-prohibitive, whereas acquiring labels by automatically aligning human\nabstracts and source documents can yield inferior results. In this paper we\ndescribe a novel framework to guide a supervised, extractive summarization\nsystem with question-answering rewards. We argue that quality summaries should\nserve as a document surrogate to answer important questions, and such\nquestion-answer pairs can be conveniently obtained from human abstracts. The\nsystem learns to promote summaries that are informative, fluent, and perform\ncompetitively on question-answering. Our results compare favorably with those\nreported by strong summarization baselines as evaluated by automatic metrics\nand human assessors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02331v1",
            "date": "2019-04-04 03:22:40+00:00",
            "title": "Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation",
            "authors": [
                "Jiawei Wu",
                "Xin Wang",
                "William Yang Wang"
            ],
            "abstract": "The overreliance on large parallel corpora significantly limits the\napplicability of machine translation systems to the majority of language pairs.\nBack-translation has been dominantly used in previous approaches for\nunsupervised neural machine translation, where pseudo sentence pairs are\ngenerated to train the models with a reconstruction loss. However, the pseudo\nsentences are usually of low quality as translation errors accumulate during\ntraining. To avoid this fundamental issue, we propose an alternative but more\neffective approach, extract-edit, to extract and then edit real sentences from\nthe target monolingual corpora. Furthermore, we introduce a comparative\ntranslation loss to evaluate the translated target sentences and thus train the\nunsupervised translation systems. Experiments show that the proposed approach\nconsistently outperforms the previous state-of-the-art unsupervised machine\ntranslation systems across two benchmarks (English-French and English-German)\nand two low-resource language pairs (English-Romanian and English-Russian) by\nmore than 2 (up to 3.63) BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02342v2",
            "date": "2019-04-04 04:33:15+00:00",
            "title": "Text Generation from Knowledge Graphs with Graph Transformers",
            "authors": [
                "Rik Koncel-Kedziorski",
                "Dhanush Bekal",
                "Yi Luan",
                "Mirella Lapata",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Generating texts which express complex ideas spanning multiple sentences\nrequires a structured representation of their content (document plan), but\nthese representations are prohibitively expensive to manually produce. In this\nwork, we address the problem of generating coherent multi-sentence texts from\nthe output of an information extraction system, and in particular a knowledge\ngraph. Graphical knowledge representations are ubiquitous in computing, but\npose a significant challenge for text generation techniques due to their\nnon-hierarchical nature, collapsing of long-distance dependencies, and\nstructural variety. We introduce a novel graph transforming encoder which can\nleverage the relational structure of such knowledge graphs without imposing\nlinearization or hierarchical constraints. Incorporated into an encoder-decoder\nsetup, we provide an end-to-end trainable system for graph-to-text generation\nthat we apply to the domain of scientific text. Automatic and human evaluations\nshow that our technique produces more informative texts which exhibit better\ndocument structure than competitive encoder-decoder methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02347v3",
            "date": "2019-04-04 05:02:38+00:00",
            "title": "Document-Level $N$-ary Relation Extraction with Multiscale Representation Learning",
            "authors": [
                "Robin Jia",
                "Cliff Wong",
                "Hoifung Poon"
            ],
            "abstract": "Most information extraction methods focus on binary relations expressed\nwithin single sentences. In high-value domains, however, $n$-ary relations are\nof great demand (e.g., drug-gene-mutation interactions in precision oncology).\nSuch relations often involve entity mentions that are far apart in the\ndocument, yet existing work on cross-sentence relation extraction is generally\nconfined to small text spans (e.g., three consecutive sentences), which\nseverely limits recall. In this paper, we propose a novel multiscale neural\narchitecture for document-level $n$-ary relation extraction. Our system\ncombines representations learned over various text spans throughout the\ndocument and across the subrelation hierarchy. Widening the system's purview to\nthe entire document maximizes potential recall. Moreover, by integrating weak\nsignals across the document, multiscale modeling increases precision, even in\nthe presence of noisy labels from distant supervision. Experiments on\nbiomedical machine reading show that our approach substantially outperforms\nprevious $n$-ary relation extraction methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02357v3",
            "date": "2019-04-04 05:42:53+00:00",
            "title": "Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation",
            "authors": [
                "Seraphina Goldfarb-Tarrant",
                "Haining Feng",
                "Nanyun Peng"
            ],
            "abstract": "Story composition is a challenging problem for machines and even for humans.\nWe present a neural narrative generation system that interacts with humans to\ngenerate stories. Our system has different levels of human interaction, which\nenables us to understand at what stage of story-writing human collaboration is\nmost productive, both to improving story quality and human engagement in the\nwriting process. We compare different varieties of interaction in\nstory-writing, story-planning, and diversity controls under time constraints,\nand show that increased types of human collaboration at both planning and\nwriting stages results in a 10-50% improvement in story quality as compared to\nless interactive baselines. We also show an accompanying increase in user\nengagement and satisfaction with stories as compared to our own less\ninteractive systems and to previous turn-taking approaches to interaction.\nFinally, we find that humans tasked with collaboratively improving a particular\ncharacteristic of a story are in fact able to do so, which has implications for\nfuture uses of human-in-the-loop systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02373v1",
            "date": "2019-04-04 06:37:19+00:00",
            "title": "Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesis",
            "authors": [
                "Yanyao Bian",
                "Changbin Chen",
                "Yongguo Kang",
                "Zhenglin Pan"
            ],
            "abstract": "Speech style control and transfer techniques aim to enrich the diversity and\nexpressiveness of synthesized speech. Existing approaches model all speech\nstyles into one representation, lacking the ability to control a specific\nspeech feature independently. To address this issue, we introduce a novel\nmulti-reference structure to Tacotron and propose intercross training approach,\nwhich together ensure that each sub-encoder of the multi-reference encoder\nindependently disentangles and controls a specific style. Experimental results\nshow that our model is able to control and transfer desired speech styles\nindividually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02461v1",
            "date": "2019-04-04 10:30:52+00:00",
            "title": "ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems",
            "authors": [
                "Inigo Jauregi Unanue",
                "Ehsan Zare Borzeshi",
                "Nazanin Esmaili",
                "Massimo Piccardi"
            ],
            "abstract": "Regularization of neural machine translation is still a significant problem,\nespecially in low-resource settings. To mollify this problem, we propose\nregressing word embeddings (ReWE) as a new regularization technique in a system\nthat is jointly trained to predict the next word in the translation\n(categorical value) and its word embedding (continuous value). Such a joint\ntraining allows the proposed system to learn the distributional properties\nrepresented by the word embeddings, empirically improving the generalization to\nunseen sentences. Experiments over three translation datasets have showed a\nconsistent improvement over a strong baseline, ranging between 0.91 and 2.54\nBLEU points, and also a marked improvement over a state-of-the-art system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02464v1",
            "date": "2019-04-04 10:38:33+00:00",
            "title": "Composition of Sentence Embeddings:Lessons from Statistical Relational Learning",
            "authors": [
                "Damien Sileo",
                "Tim Van-De-Cruys",
                "Camille Pradel",
                "Philippe Muller"
            ],
            "abstract": "Various NLP problems -- such as the prediction of sentence similarity,\nentailment, and discourse relations -- are all instances of the same general\ntask: the modeling of semantic relations between a pair of textual elements. A\npopular model for such problems is to embed sentences into fixed size vectors,\nand use composition functions (e.g. concatenation or sum) of those vectors as\nfeatures for the prediction. At the same time, composition of embeddings has\nbeen a main focus within the field of Statistical Relational Learning (SRL)\nwhose goal is to predict relations between entities (typically from knowledge\nbase triples). In this article, we show that previous work on relation\nprediction between texts implicitly uses compositions from baseline SRL models.\nWe show that such compositions are not expressive enough for several tasks\n(e.g. natural language inference). We build on recent SRL models to address\ntextual relational problems, showing that they are more expressive, and can\nalleviate issues from simpler compositions. The resulting models significantly\nimprove the state of the art in both transferable sentence representation\nlearning and relation prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02594v2",
            "date": "2019-04-04 15:02:20+00:00",
            "title": "Dialogue Act Classification with Context-Aware Self-Attention",
            "authors": [
                "Vipul Raheja",
                "Joel Tetreault"
            ],
            "abstract": "Recent work in Dialogue Act classification has treated the task as a sequence\nlabeling problem using hierarchical deep neural networks. We build on this\nprior work by leveraging the effectiveness of a context-aware self-attention\nmechanism coupled with a hierarchical recurrent neural network. We conduct\nextensive evaluations on standard Dialogue Act classification datasets and show\nsignificant improvement over state-of-the-art results on the Switchboard\nDialogue Act (SwDA) Corpus. We also investigate the impact of different\nutterance-level representation learning methods and show that our method is\neffective at capturing utterance-level semantic text representations while\nmaintaining high accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02619v1",
            "date": "2019-04-04 15:44:39+00:00",
            "title": "Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions",
            "authors": [
                "Awni Hannun",
                "Ann Lee",
                "Qiantong Xu",
                "Ronan Collobert"
            ],
            "abstract": "We propose a fully convolutional sequence-to-sequence encoder architecture\nwith a simple and efficient decoder. Our model improves WER on LibriSpeech\nwhile being an order of magnitude more efficient than a strong RNN baseline.\nKey to our approach is a time-depth separable convolution block which\ndramatically reduces the number of parameters in the model while keeping the\nreceptive field large. We also give a stable and efficient beam search\ninference procedure which allows us to effectively integrate a language model.\nCoupled with a convolutional language model, our time-depth separable\nconvolution architecture improves by more than 22% relative WER over the best\npreviously reported sequence-to-sequence results on the noisy LibriSpeech test\nset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02651v1",
            "date": "2019-04-04 16:44:44+00:00",
            "title": "ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions",
            "authors": [
                "Soham Parikh",
                "Ananya B. Sai",
                "Preksha Nema",
                "Mitesh M. Khapra"
            ],
            "abstract": "The task of Reading Comprehension with Multiple Choice Questions, requires a\nhuman (or machine) to read a given passage, question pair and select one of the\nn given options. The current state of the art model for this task first\ncomputes a question-aware representation for the passage and then selects the\noption which has the maximum similarity with this representation. However, when\nhumans perform this task they do not just focus on option selection but use a\ncombination of elimination and selection. Specifically, a human would first try\nto eliminate the most irrelevant option and then read the passage again in the\nlight of this new information (and perhaps ignore portions corresponding to the\neliminated option). This process could be repeated multiple times till the\nreader is finally ready to select the correct option. We propose ElimiNet, a\nneural network-based model which tries to mimic this process. Specifically, it\nhas gates which decide whether an option can be eliminated given the passage,\nquestion pair and if so it tries to make the passage representation orthogonal\nto this eliminated option (akin to ignoring portions of the passage\ncorresponding to the eliminated option). The model makes multiple rounds of\npartial elimination to refine the passage representation and finally uses a\nselection module to pick the best option. We evaluate our model on the recently\nreleased large scale RACE dataset and show that it outperforms the current\nstate of the art model on 7 out of the $13$ question types in this dataset.\nFurther, we show that taking an ensemble of our elimination-selection based\nmethod with a selection based method gives us an improvement of 3.1% over the\nbest-reported performance on this dataset.",
            "journal ref": "Proceedings of the Twenty-Seventh International Joint Conference\n  on Artificial Intelligence (2018) Main track. Pages 4272-4278",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02660v1",
            "date": "2019-04-04 16:56:01+00:00",
            "title": "Recommendations for Datasets for Source Code Summarization",
            "authors": [
                "Alexander LeClair",
                "Collin McMillan"
            ],
            "abstract": "Source Code Summarization is the task of writing short, natural language\ndescriptions of source code. The main use for these descriptions is in software\ndocumentation e.g. the one-sentence Java method descriptions in JavaDocs. Code\nsummarization is rapidly becoming a popular research problem, but progress is\nrestrained due to a lack of suitable datasets. In addition, a lack of community\nstandards for creating datasets leads to confusing and unreproducible research\nresults -- we observe swings in performance of more than 33% due only to\nchanges in dataset design. In this paper, we make recommendations for these\nstandards from experimental results. We release a dataset based on prior work\nof over 2.1m pairs of Java methods and one sentence method descriptions from\nover 28k Java projects. We describe the dataset and point out key differences\nfrom natural language data, to guide and support future researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02665v1",
            "date": "2019-04-04 17:00:48+00:00",
            "title": "Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples",
            "authors": [
                "Soham Parikh",
                "Ananya B. Sai",
                "Preksha Nema",
                "Mitesh M. Khapra"
            ],
            "abstract": "When humans learn to perform a difficult task (say, reading comprehension\n(RC) over longer passages), it is typically the case that their performance\nimproves significantly on an easier version of this task (say, RC over shorter\npassages). Ideally, we would want an intelligent agent to also exhibit such a\nbehavior. However, on experimenting with state of the art RC models using the\nstandard RACE dataset, we observe that this is not true. Specifically, we see\ncounter-intuitive results wherein even when we show frustratingly easy examples\nto the model at test time, there is hardly any improvement in its performance.\nWe refer to this as non-adversarial evaluation as opposed to adversarial\nevaluation. Such non-adversarial examples allow us to assess the utility of\nspecialized neural components. For example, we show that even for easy examples\nwhere the answer is clearly embedded in the passage, the neural components\ndesigned for paying attention to relevant portions of the passage fail to serve\ntheir intended purpose. We believe that the non-adversarial dataset created as\na part of this work would complement the research on adversarial evaluation and\ngive a more realistic assessment of the ability of RC models. All the datasets\nand codes developed as a part of this work will be made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02668v4",
            "date": "2019-04-04 17:04:30+00:00",
            "title": "Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets",
            "authors": [
                "Nelson F. Liu",
                "Roy Schwartz",
                "Noah A. Smith"
            ],
            "abstract": "Several datasets have recently been constructed to expose brittleness in\nmodels trained on existing benchmarks. While model performance on these\nchallenge datasets is significantly lower compared to the original benchmark,\nit is unclear what particular weaknesses they reveal. For example, a challenge\ndataset may be difficult because it targets phenomena that current models\ncannot capture, or because it simply exploits blind spots in a model's specific\ntraining set. We introduce inoculation by fine-tuning, a new analysis method\nfor studying challenge datasets by exposing models (the metaphorical patient)\nto a small amount of data from the challenge dataset (a metaphorical pathogen)\nand assessing how well they can adapt. We apply our method to analyze the NLI\n\"stress tests\" (Naik et al., 2018) and the Adversarial SQuAD dataset (Jia and\nLiang, 2017). We show that after slight exposure, some of these datasets are no\nlonger challenging, while others remain difficult. Our results indicate that\nfailures on challenge datasets may lead to very different conclusions about\nmodels, training datasets, and the challenge datasets themselves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02671v1",
            "date": "2019-04-04 17:12:25+00:00",
            "title": "Studying Cultural Differences in Emoji Usage across the East and the West",
            "authors": [
                "Sharath Chandra Guntuku",
                "Mingyang Li",
                "Louis Tay",
                "Lyle H. Ungar"
            ],
            "abstract": "Global acceptance of Emojis suggests a cross-cultural, normative use of\nEmojis. Meanwhile, nuances in Emoji use across cultures may also exist due to\nlinguistic differences in expressing emotions and diversity in conceptualizing\ntopics. Indeed, literature in cross-cultural psychology has found both\nnormative and culture-specific ways in which emotions are expressed. In this\npaper, using social media, we compare the Emoji usage based on frequency,\ncontext, and topic associations across countries in the East (China and Japan)\nand the West (United States, United Kingdom, and Canada). Across the East and\nthe West, our study examines a) similarities and differences on the usage of\ndifferent categories of Emojis such as People, Food \\& Drink, Travel \\& Places\netc., b) potential mapping of Emoji use differences with previously identified\ncultural differences in users' expression about diverse concepts such as death,\nmoney emotions and family, and c) relative correspondence of validated\npsycho-linguistic categories with Ekman's emotions. The analysis of Emoji use\nin the East and the West reveals recognizable normative and culture specific\npatterns. This research reveals the ways in which Emojis can be used for\ncross-cultural communication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02682v1",
            "date": "2019-04-04 17:38:16+00:00",
            "title": "Advancing NLP with Cognitive Language Processing Signals",
            "authors": [
                "Nora Hollenstein",
                "Maria Barrett",
                "Marius Troendle",
                "Francesco Bigiolli",
                "Nicolas Langer",
                "Ce Zhang"
            ],
            "abstract": "When we read, our brain processes language and generates cognitive processing\ndata such as gaze patterns and brain activity. These signals can be recorded\nwhile reading. Cognitive language processing data such as eye-tracking features\nhave shown improvements on single NLP tasks. We analyze whether using such\nhuman features can show consistent improvement across tasks and data sources.\nWe present an extensive investigation of the benefits and limitations of using\ncognitive processing data for NLP. Specifically, we use gaze and EEG features\nto augment models of named entity recognition, relation classification, and\nsentiment analysis. These methods significantly outperform the baselines and\nshow the potential and current limitations of employing human language\nprocessing data for NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02755v1",
            "date": "2019-04-04 19:17:04+00:00",
            "title": "ExCL: Extractive Clip Localization Using Natural Language Descriptions",
            "authors": [
                "Soham Ghosh",
                "Anuva Agarwal",
                "Zarana Parekh",
                "Alexander Hauptmann"
            ],
            "abstract": "The task of retrieving clips within videos based on a given natural language\nquery requires cross-modal reasoning over multiple frames. Prior approaches\nsuch as sliding window classifiers are inefficient, while text-clip similarity\ndriven ranking-based approaches such as segment proposal networks are far more\ncomplicated. In order to select the most relevant video clip corresponding to\nthe given text description, we propose a novel extractive approach that\npredicts the start and end frames by leveraging cross-modal interactions\nbetween the text and video - this removes the need to retrieve and re-rank\nmultiple proposal segments. Using recurrent networks we encode the two\nmodalities into a joint representation which is then used in different variants\nof start-end frame predictor networks. Through extensive experimentation and\nablative analysis, we demonstrate that our simple and elegant approach\nsignificantly outperforms state of the art on two datasets and has comparable\nperformance on a third.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02767v1",
            "date": "2019-04-04 19:47:17+00:00",
            "title": "Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification",
            "authors": [
                "Reno Kriz",
                "Jo\u00e3o Sedoc",
                "Marianna Apidianaki",
                "Carolina Zheng",
                "Gaurav Kumar",
                "Eleni Miltsakaki",
                "Chris Callison-Burch"
            ],
            "abstract": "Sentence simplification is the task of rewriting texts so they are easier to\nunderstand. Recent research has applied sequence-to-sequence (Seq2Seq) models\nto this task, focusing largely on training-time improvements via reinforcement\nlearning and memory augmentation. One of the main problems with applying\ngeneric Seq2Seq models for simplification is that these models tend to copy\ndirectly from the original sentence, resulting in outputs that are relatively\nlong and complex. We aim to alleviate this issue through the use of two main\ntechniques. First, we incorporate content word complexities, as predicted with\na leveled word complexity model, into our loss function during training.\nSecond, we generate a large set of diverse candidate simplifications at test\ntime, and rerank these to promote fluency, adequacy, and simplicity. Here, we\nmeasure simplicity through a novel sentence complexity model. These extensions\nallow our models to perform competitively with state-of-the-art systems while\ngenerating simpler sentences. We report standard automatic and human evaluation\nmetrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02800v1",
            "date": "2019-04-04 21:53:41+00:00",
            "title": "Improving Dialogue State Tracking by Discerning the Relevant Context",
            "authors": [
                "Sanuj Sharma",
                "Prafulla Kumar Choubey",
                "Ruihong Huang"
            ],
            "abstract": "A typical conversation comprises of multiple turns between participants where\nthey go back-and-forth between different topics. At each user turn, dialogue\nstate tracking (DST) aims to estimate user's goal by processing the current\nutterance. However, in many turns, users implicitly refer to the previous goal,\nnecessitating the use of relevant dialogue history. Nonetheless, distinguishing\nrelevant history is challenging and a popular method of using dialogue recency\nfor that is inefficient. We, therefore, propose a novel framework for DST that\nidentifies relevant historical context by referring to the past utterances\nwhere a particular slot-value changes and uses that together with weighted\nsystem utterance to identify the relevant context. Specifically, we use the\ncurrent user utterance and the most recent system utterance to determine the\nrelevance of a system utterance. Empirical analyses show that our method\nimproves joint goal accuracy by 2.75% and 2.36% on WoZ 2.0 and MultiWoZ 2.0\nrestaurant domain datasets respectively over the previous state-of-the-art GLAD\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02927v1",
            "date": "2019-04-05 08:14:08+00:00",
            "title": "Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models --- Is Single-Corpus Evaluation Enough?",
            "authors": [
                "Masato Mita",
                "Tomoya Mizumoto",
                "Masahiro Kaneko",
                "Ryo Nagata",
                "Kentaro Inui"
            ],
            "abstract": "This study explores the necessity of performing cross-corpora evaluation for\ngrammatical error correction (GEC) models. GEC models have been previously\nevaluated based on a single commonly applied corpus: the CoNLL-2014 benchmark.\nHowever, the evaluation remains incomplete because the task difficulty varies\ndepending on the test corpus and conditions such as the proficiency levels of\nthe writers and essay topics. To overcome this limitation, we evaluate the\nperformance of several GEC models, including NMT-based (LSTM, CNN, and\ntransformer) and an SMT-based model, against various learner corpora\n(CoNLL-2013, CoNLL-2014, FCE, JFLEG, ICNALE, and KJ). Evaluation results reveal\nthat the models' rankings considerably vary depending on the corpus, indicating\nthat single-corpus evaluation is insufficient for GEC models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02954v1",
            "date": "2019-04-05 09:24:36+00:00",
            "title": "Alternative Weighting Schemes for ELMo Embeddings",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "ELMo embeddings (Peters et. al, 2018) had a huge impact on the NLP community\nand may recent publications use these embeddings to boost the performance for\ndownstream NLP tasks. However, integration of ELMo embeddings in existent NLP\narchitectures is not straightforward. In contrast to traditional word\nembeddings, like GloVe or word2vec embeddings, the bi-directional language\nmodel of ELMo produces three 1024 dimensional vectors per token in a sentence.\nPeters et al. proposed to learn a task-specific weighting of these three\nvectors for downstream tasks. However, this proposed weighting scheme is not\nfeasible for certain tasks, and, as we will show, it does not necessarily yield\noptimal performance. We evaluate different methods that combine the three\nvectors from the language model in order to achieve the best possible\nperformance in downstream NLP tasks. We notice that the third layer of the\npublished language model often decreases the performance. By learning a\nweighted average of only the first two layers, we are able to improve the\nperformance for many datasets. Due to the reduced complexity of the language\nmodel, we have a training speed-up of 19-44% for the downstream task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02981v1",
            "date": "2019-04-05 10:19:56+00:00",
            "title": "NL-FIIT at SemEval-2019 Task 9: Neural Model Ensemble for Suggestion Mining",
            "authors": [
                "Samuel Pecar",
                "Marian Simko",
                "Maria Bielikova"
            ],
            "abstract": "In this paper, we present neural model architecture submitted to the\nSemEval-2019 Task 9 competition: \"Suggestion Mining from Online Reviews and\nForums\". We participated in both subtasks for domain specific and also\ncross-domain suggestion mining. We proposed a recurrent neural network\narchitecture that employs Bi-LSTM layers and also self-attention mechanism. Our\narchitecture tries to encode words via word representations using ELMo and\nensembles multiple models to achieve better results. We performed experiments\nwith different setups of our proposed model involving weighting of prediction\nclasses for loss function. Our best model achieved in official test evaluation\nscore of 0.6816 for subtask A and 0.6850 for subtask B. In official results, we\nachieved 12th and 10th place in subtasks A and B, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02996v1",
            "date": "2019-04-05 11:29:57+00:00",
            "title": "Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models",
            "authors": [
                "Victor Prokhorov",
                "Mohammad Taher Pilehvar",
                "Nigel Collier"
            ],
            "abstract": "We present a novel method for mapping unrestricted text to knowledge graph\nentities by framing the task as a sequence-to-sequence problem. Specifically,\ngiven the encoded state of an input text, our decoder directly predicts paths\nin the knowledge graph, starting from the root and ending at the target node\nfollowing hypernym-hyponym relationships. In this way, and in contrast to other\ntext-to-entity mapping systems, our model outputs hierarchically structured\npredictions that are fully interpretable in the context of the underlying\nontology, in an end-to-end manner. We present a proof-of-concept experiment\nwith encouraging results, comparable to those of state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03035v1",
            "date": "2019-04-05 12:40:28+00:00",
            "title": "Identifying and Reducing Gender Bias in Word-Level Language Models",
            "authors": [
                "Shikha Bordia",
                "Samuel R. Bowman"
            ],
            "abstract": "Many text corpora exhibit socially problematic biases, which can be\npropagated or amplified in the models trained on such data. For example, doctor\ncooccurs more frequently with male pronouns than female pronouns. In this study\nwe (i) propose a metric to measure gender bias; (ii) measure bias in a text\ncorpus and the text generated from a recurrent neural network language model\ntrained on the text corpus; (iii) propose a regularization loss term for the\nlanguage model that minimizes the projection of encoder-trained embeddings onto\nan embedding subspace that encodes gender; (iv) finally, evaluate efficacy of\nour proposed method on reducing gender bias. We find this regularization method\nto be effective in reducing gender bias up to an optimal weight assigned to the\nloss term, beyond which the model becomes unstable as the perplexity increases.\nWe replicate this study on three training corpora---Penn Treebank, WikiText-2,\nand CNN/Daily Mail---resulting in similar conclusions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03111v2",
            "date": "2019-04-05 15:10:09+00:00",
            "title": "PoMo: Generating Entity-Specific Post-Modifiers in Context",
            "authors": [
                "Jun Seok Kang",
                "Robert L. Logan IV",
                "Zewei Chu",
                "Yang Chen",
                "Dheeru Dua",
                "Kevin Gimpel",
                "Sameer Singh",
                "Niranjan Balasubramanian"
            ],
            "abstract": "We introduce entity post-modifier generation as an instance of a\ncollaborative writing task. Given a sentence about a target entity, the task is\nto automatically generate a post-modifier phrase that provides contextually\nrelevant information about the entity. For example, for the sentence, \"Barack\nObama, _______, supported the #MeToo movement.\", the phrase \"a father of two\ngirls\" is a contextually relevant post-modifier. To this end, we build PoMo, a\npost-modifier dataset created automatically from news articles reflecting a\njournalistic need for incorporating entity information that is relevant to a\nparticular news event. PoMo consists of more than 231K sentences with\npost-modifiers and associated facts extracted from Wikidata for around 57K\nunique entities. We use crowdsourcing to show that modeling contextual\nrelevance is necessary for accurate post-modifier generation. We adapt a number\nof existing generation approaches as baselines for this dataset. Our results\nshow there is large room for improvement in terms of both identifying relevant\nfacts to include (knowing which claims are relevant gives a >20% improvement in\nBLEU score), and generating appropriate post-modifier text for the context\n(providing relevant claims is not sufficient for accurate generation). We\nconduct an error analysis that suggests promising directions for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03122v1",
            "date": "2019-04-05 15:31:28+00:00",
            "title": "Outlier Detection for Improved Data Quality and Diversity in Dialog Systems",
            "authors": [
                "Stefan Larson",
                "Anish Mahendran",
                "Andrew Lee",
                "Jonathan K. Kummerfeld",
                "Parker Hill",
                "Michael A. Laurenzano",
                "Johann Hauswald",
                "Lingjia Tang",
                "Jason Mars"
            ],
            "abstract": "In a corpus of data, outliers are either errors: mistakes in the data that\nare counterproductive, or are unique: informative samples that improve model\nrobustness. Identifying outliers can lead to better datasets by (1) removing\nnoise in datasets and (2) guiding collection of additional data to fill gaps.\nHowever, the problem of detecting both outlier types has received relatively\nlittle attention in NLP, particularly for dialog systems. We introduce a simple\nand effective technique for detecting both erroneous and unique samples in a\ncorpus of short texts using neural sentence embeddings combined with\ndistance-based outlier detection. We also present a novel data collection\npipeline built atop our detection technique to automatically and iteratively\nmine unique data samples while discarding erroneous samples. Experiments show\nthat our outlier detection technique is effective at finding errors while our\ndata collection pipeline yields highly diverse corpora that in turn produce\nmore robust intent classification and slot-filling models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03164v1",
            "date": "2019-04-05 17:13:01+00:00",
            "title": "Exploring Fine-Tuned Embeddings that Model Intensifiers for Emotion Analysis",
            "authors": [
                "Laura Bostan",
                "Roman Klinger"
            ],
            "abstract": "Adjective phrases like \"a little bit surprised\", \"completely shocked\", or\n\"not stunned at all\" are not handled properly by currently published\nstate-of-the-art emotion classification and intensity prediction systems which\nuse pre-dominantly non-contextualized word embeddings as input. Based on this\nfinding, we analyze differences between embeddings used by these systems in\nregard to their capability of handling such cases. Furthermore, we argue that\nintensifiers in context of emotion words need special treatment, as is\nestablished for sentiment polarity classification, but not for more\nfine-grained emotion prediction. To resolve this issue, we analyze different\naspects of a post-processing pipeline which enriches the word representations\nof such phrases. This includes expansion of semantic spaces at the phrase level\nand sub-word level followed by retrofitting to emotion lexica. We evaluate the\nimpact of these steps with A La Carte and Bag-of-Substrings extensions based on\npretrained GloVe, Word2vec, and fastText embeddings against a crowd-sourced\ncorpus of intensity annotations for tweets containing our focus phrases. We\nshow that the fastText-based models do not gain from handling these specific\nphrases under inspection. For Word2vec embeddings, we show that our\npost-processing pipeline improves the results by up to 8% on a novel dataset\ndensely populated with intensifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03225v1",
            "date": "2019-04-05 18:33:22+00:00",
            "title": "Distinguishing Clinical Sentiment: The Importance of Domain Adaptation in Psychiatric Patient Health Records",
            "authors": [
                "Eben Holderness",
                "Philip Cawkwell",
                "Kirsten Bolton",
                "James Pustejovsky",
                "Mei-Hua Hall"
            ],
            "abstract": "Recently natural language processing (NLP) tools have been developed to\nidentify and extract salient risk indicators in electronic health records\n(EHRs). Sentiment analysis, although widely used in non-medical areas for\nimproving decision making, has been studied minimally in the clinical setting.\nIn this study, we undertook, to our knowledge, the first domain adaptation of\nsentiment analysis to psychiatric EHRs by defining psychiatric clinical\nsentiment, performing an annotation project, and evaluating multiple\nsentence-level sentiment machine learning (ML) models. Results indicate that\noff-the-shelf sentiment analysis tools fail in identifying clinically positive\nor negative polarity, and that the definition of clinical sentiment that we\nprovide is learnable with relatively small amounts of training data. This\nproject is an initial step towards further refining sentiment analysis methods\nfor clinical use. Our long-term objective is to incorporate the results of this\nproject as part of a machine learning model that predicts inpatient readmission\nrisk. We hope that this work will initiate a discussion concerning domain\nadaptation of sentiment analysis to the clinical setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03256v1",
            "date": "2019-04-05 20:04:04+00:00",
            "title": "Cross-Lingual Transfer of Semantic Roles: From Raw Text to Semantic Roles",
            "authors": [
                "Maryam Aminian",
                "Mohammad Sadegh Rasooli",
                "Mona Diab"
            ],
            "abstract": "We describe a transfer method based on annotation projection to develop a\ndependency-based semantic role labeling system for languages for which no\nsupervised linguistic information other than parallel data is available. Unlike\nprevious work that presumes the availability of supervised features such as\nlemmas, part-of-speech tags, and dependency parse trees, we only make use of\nword and character features. Our deep model considers using character-based\nrepresentations as well as unsupervised stem embeddings to alleviate the need\nfor supervised features. Our experiments outperform a state-of-the-art method\nthat uses supervised lexico-syntactic features on 6 out of 7 languages in the\nUniversal Proposition Bank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03262v1",
            "date": "2019-04-05 20:18:51+00:00",
            "title": "Extracting Factual Min/Max Age Information from Clinical Trial Studies",
            "authors": [
                "Yufang Hou",
                "Debasis Ganguly",
                "Lea A. Deleris",
                "Francesca Bonin"
            ],
            "abstract": "Population age information is an essential characteristic of clinical trials.\nIn this paper, we focus on extracting minimum and maximum (min/max) age values\nfor the study samples from clinical research articles. Specifically, we\ninvestigate the use of a neural network model for question answering to address\nthis information extraction task. The min/max age QA model is trained on the\nmassive structured clinical study records from ClinicalTrials.gov. For each\narticle, based on multiple min and max age values extracted from the QA model,\nwe predict both actual min/max age values for the study samples and filter out\nnon-factual age expressions. Our system improves the results over (i) a passage\nretrieval based IE system and (ii) a CRF-based system by a large margin when\nevaluated on an annotated dataset consisting of 50 research papers on smoking\ncessation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03279v2",
            "date": "2019-04-05 21:02:12+00:00",
            "title": "Generate, Filter, and Rank: Grammaticality Classification for Production-Ready NLG Systems",
            "authors": [
                "Ashwini Challa",
                "Kartikeya Upasani",
                "Anusha Balakrishnan",
                "Rajen Subba"
            ],
            "abstract": "Neural approaches to Natural Language Generation (NLG) have been promising\nfor goal-oriented dialogue. One of the challenges of productionizing these\napproaches, however, is the ability to control response quality, and ensure\nthat generated responses are acceptable. We propose the use of a generate,\nfilter, and rank framework, in which candidate responses are first filtered to\neliminate unacceptable responses, and then ranked to select the best response.\nWhile acceptability includes grammatical correctness and semantic correctness,\nwe focus only on grammaticality classification in this paper, and show that\nexisting datasets for grammatical error correction don't correctly capture the\ndistribution of errors that data-driven generators are likely to make. We\nrelease a grammatical classification and semantic correctness classification\ndataset for the weather domain that consists of responses generated by 3\ndata-driven NLG systems. We then explore two supervised learning approaches\n(CNNs and GBDTs) for classifying grammaticality. Our experiments show that\ngrammaticality classification is very sensitive to the distribution of errors\nin the data, and that these distributions vary significantly with both the\nsource of the response as well as the domain. We show that it's possible to\nachieve high precision with reasonable recall on our dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03296v1",
            "date": "2019-04-05 21:52:18+00:00",
            "title": "A General Framework for Information Extraction using Dynamic Span Graphs",
            "authors": [
                "Yi Luan",
                "Dave Wadden",
                "Luheng He",
                "Amy Shah",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce a general framework for several information extraction tasks\nthat share span representations using dynamically constructed span graphs. The\ngraphs are constructed by selecting the most confident entity spans and linking\nthese nodes with confidence-weighted relation types and coreferences. The\ndynamic span graph allows coreference and relation type confidences to\npropagate through the graph to iteratively refine the span representations.\nThis is unlike previous multi-task frameworks for information extraction in\nwhich the only interaction between tasks is in the shared first-layer LSTM. Our\nframework significantly outperforms the state-of-the-art on multiple\ninformation extraction tasks across multiple datasets reflecting different\ndomains. We further observe that the span enumeration approach is good at\ndetecting nested span entities, with significant F1 score improvement on the\nACE dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03300v2",
            "date": "2019-04-05 21:58:27+00:00",
            "title": "A Multi-task Learning Approach for Named Entity Recognition using Local Detection",
            "authors": [
                "Nargiza Nosirova",
                "Mingbin Xu",
                "Hui Jiang"
            ],
            "abstract": "Named entity recognition (NER) systems that perform well require task-related\nand manually annotated datasets. However, they are expensive to develop, and\nare thus limited in size. As there already exists a large number of NER\ndatasets that share a certain degree of relationship but differ in content, it\nis important to explore the question of whether such datasets can be combined\nas a simple method for improving NER performance. To investigate this, we\ndeveloped a novel locally detecting multitask model using FFNNs. The model\nrelies on encoding variable-length sequences of words into theoretically\nlossless and unique fixed-size representations. We applied this method to\nseveral well-known NER tasks and compared the results of our model to baseline\nmodels as well as other published results. As a result, we observed competitive\nperformance in nearly all of the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03305v2",
            "date": "2019-04-05 22:10:15+00:00",
            "title": "Effective Context and Fragment Feature Usage for Named Entity Recognition",
            "authors": [
                "Nargiza Nosirova",
                "Mingbin Xu",
                "Hui Jiang"
            ],
            "abstract": "In this paper, we explore a new approach to named entity recognition (NER)\nwith the goal of learning from context and fragment features more effectively,\ncontributing to the improvement of overall recognition performance. We use the\nrecent fixed-size ordinally forgetting encoding (FOFE) method to fully encode\neach sentence fragment and its left-right contexts into a fixed-size\nrepresentation. Next, we organize the context and fragment features into\ngroups, and feed each feature group to dedicated fully-connected layers.\nFinally, we merge each group's final dedicated layers and add a shared layer\nleading to a single output. The outcome of our experiments show that, given\nonly tokenized text and trained word embeddings, our system outperforms our\nbaseline models, and is competitive to the state-of-the-arts of various\nwell-known NER tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03310v1",
            "date": "2019-04-05 22:36:12+00:00",
            "title": "Gender Bias in Contextualized Word Embeddings",
            "authors": [
                "Jieyu Zhao",
                "Tianlu Wang",
                "Mark Yatskar",
                "Ryan Cotterell",
                "Vicente Ordonez",
                "Kai-Wei Chang"
            ],
            "abstract": "In this paper, we quantify, analyze and mitigate gender bias exhibited in\nELMo's contextualized word vectors. First, we conduct several intrinsic\nanalyses and find that (1) training data for ELMo contains significantly more\nmale than female entities, (2) the trained ELMo embeddings systematically\nencode gender information and (3) ELMo unequally encodes gender information\nabout male and female entities. Then, we show that a state-of-the-art\ncoreference system that depends on ELMo inherits its bias and demonstrates\nsignificant bias on the WinoBias probing corpus. Finally, we explore two\nmethods to mitigate such gender bias and show that the bias demonstrated on\nWinoBias can be eliminated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03323v3",
            "date": "2019-04-06 00:34:39+00:00",
            "title": "Publicly Available Clinical BERT Embeddings",
            "authors": [
                "Emily Alsentzer",
                "John R. Murphy",
                "Willie Boag",
                "Wei-Hung Weng",
                "Di Jin",
                "Tristan Naumann",
                "Matthew B. A. McDermott"
            ],
            "abstract": "Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT\n(Devlin et al., 2018) have dramatically improved performance for many natural\nlanguage processing (NLP) tasks in recent months. However, these models have\nbeen minimally explored on specialty corpora, such as clinical text; moreover,\nin the clinical domain, no publicly-available pre-trained BERT models yet\nexist. In this work, we address this need by exploring and releasing BERT\nmodels for clinical text: one for generic clinical text and another for\ndischarge summaries specifically. We demonstrate that using a domain-specific\nmodel yields performance improvements on three common clinical NLP tasks as\ncompared to nonspecific embeddings. These domain-specific models are not as\nperformant on two clinical de-identification tasks, and argue that this is a\nnatural consequence of the differences between de-identified source text and\nsynthetically non de-identified task text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03339v1",
            "date": "2019-04-06 02:24:30+00:00",
            "title": "ThisIsCompetition at SemEval-2019 Task 9: BERT is unstable for out-of-domain samples",
            "authors": [
                "Cheoneum Park",
                "Juae Kim",
                "Hyeon-gu Lee",
                "Reinald Kim Amplayo",
                "Harksoo Kim",
                "Jungyun Seo",
                "Changki Lee"
            ],
            "abstract": "This paper describes our system, Joint Encoders for Stable Suggestion\nInference (JESSI), for the SemEval 2019 Task 9: Suggestion Mining from Online\nReviews and Forums. JESSI is a combination of two sentence encoders: (a) one\nusing multiple pre-trained word embeddings learned from log-bilinear regression\n(GloVe) and translation (CoVe) models, and (b) one on top of word encodings\nfrom a pre-trained deep bidirectional transformer (BERT). We include a domain\nadversarial training module when training for out-of-domain samples. Our\nexperiments show that while BERT performs exceptionally well for in-domain\nsamples, several runs of the model show that it is unstable for out-of-domain\nsamples. The problem is mitigated tremendously by (1) combining BERT with a\nnon-BERT encoder, and (2) using an RNN-based classifier on top of BERT. Our\nfinal models obtained second place with 77.78\\% F-Score on Subtask A (i.e.\nin-domain) and achieved an F-Score of 79.59\\% on Subtask B (i.e.\nout-of-domain), even without using any additional external data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03366v1",
            "date": "2019-04-06 05:42:19+00:00",
            "title": "The Steep Road to Happily Ever After: An Analysis of Current Visual Storytelling Models",
            "authors": [
                "Yatri Modi",
                "Natalie Parde"
            ],
            "abstract": "Visual storytelling is an intriguing and complex task that only recently\nentered the research arena. In this work, we survey relevant work to date, and\nconduct a thorough error analysis of three very recent approaches to visual\nstorytelling. We categorize and provide examples of common types of errors, and\nidentify key shortcomings in current work. Finally, we make recommendations for\naddressing these limitations in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03409v1",
            "date": "2019-04-06 10:12:27+00:00",
            "title": "Parallelizable Stack Long Short-Term Memory",
            "authors": [
                "Shuoyang Ding",
                "Philipp Koehn"
            ],
            "abstract": "Stack Long Short-Term Memory (StackLSTM) is useful for various applications\nsuch as parsing and string-to-tree neural machine translation, but it is also\nknown to be notoriously difficult to parallelize for GPU training due to the\nfact that the computations are dependent on discrete operations. In this paper,\nwe tackle this problem by utilizing state access patterns of StackLSTM to\nhomogenize computations with regard to different discrete operations. Our\nparsing experiments show that the method scales up almost linearly with\nincreasing batch size, and our parallelized PyTorch implementation trains\nsignificantly faster compared to the Dynet C++ implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03417v1",
            "date": "2019-04-06 10:55:11+00:00",
            "title": "Speeding Up Natural Language Parsing by Reusing Partial Results",
            "authors": [
                "Michalina Strzyz",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "This paper proposes a novel technique that applies case-based reasoning in\norder to generate templates for reusable parse tree fragments, based on PoS\ntags of bigrams and trigrams that demonstrate low variability in their\nsyntactic analyses from prior data. The aim of this approach is to improve the\nspeed of dependency parsers by avoiding redundant calculations. This can be\nresolved by applying the predefined templates that capture results of previous\nsyntactic analyses and directly assigning the stored structure to a new n-gram\nthat matches one of the templates, instead of parsing a similar text fragment\nagain. The study shows that using a heuristic approach to select and reuse the\npartial results increases parsing speed by reducing the input length to be\nprocessed by a parser. The increase in parsing speed comes at some expense of\naccuracy. Experiments on English show promising results: the input dimension\ncan be reduced by more than 20% at the cost of less than 3 points of Unlabeled\nAttachment Score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03450v1",
            "date": "2019-04-06 14:02:13+00:00",
            "title": "UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs",
            "authors": [
                "Jian Zhu",
                "Zuoyu Tian",
                "Sandra K\u00fcbler"
            ],
            "abstract": "This paper describes the UM-IU@LING's system for the SemEval 2019 Task 6:\nOffensEval. We take a mixed approach to identify and categorize hate speech in\nsocial media. In subtask A, we fine-tuned a BERT based classifier to detect\nabusive content in tweets, achieving a macro F1 score of 0.8136 on the test\ndata, thus reaching the 3rd rank out of 103 submissions. In subtasks B and C,\nwe used a linear SVM with selected character n-gram features. For subtask C,\nour system could identify the target of abuse with a macro F1 score of 0.5243,\nranking it 27th out of 65 submissions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03454v1",
            "date": "2019-04-06 14:18:25+00:00",
            "title": "An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction",
            "authors": [
                "Wang Chen",
                "Hou Pong Chan",
                "Piji Li",
                "Lidong Bing",
                "Irwin King"
            ],
            "abstract": "In this paper, we present a novel integrated approach for keyphrase\ngeneration (KG). Unlike previous works which are purely extractive or\ngenerative, we first propose a new multi-task learning framework that jointly\nlearns an extractive model and a generative model. Besides extracting\nkeyphrases, the output of the extractive model is also employed to rectify the\ncopy probability distribution of the generative model, such that the generative\nmodel can better identify important contents from the given document. Moreover,\nwe retrieve similar documents with the given document from training data and\nuse their associated keyphrases as external knowledge for the generative model\nto produce more accurate keyphrases. For further exploiting the power of\nextraction and retrieval, we propose a neural-based merging module to combine\nand re-rank the predicted keyphrases from the enhanced generative model, the\nextractive model, and the retrieved keyphrases. Experiments on the five KG\nbenchmarks demonstrate that our integrated approach outperforms the\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03518v1",
            "date": "2019-04-06 19:56:14+00:00",
            "title": "Tracking Discrete and Continuous Entity State for Process Understanding",
            "authors": [
                "Aditya Gupta",
                "Greg Durrett"
            ],
            "abstract": "Procedural text, which describes entities and their interactions as they\nundergo some process, depicts entities in a uniquely nuanced way. First, each\nentity may have some observable discrete attributes, such as its state or\nlocation; modeling these involves imposing global structure and enforcing\nconsistency. Second, an entity may have properties which are not made explicit\nbut can be effectively induced and tracked by neural networks. In this paper,\nwe propose a structured neural architecture that reflects this dual nature of\nentity evolution. The model tracks each entity recurrently, updating its hidden\ncontinuous representation at each step to contain relevant state information.\nThe global discrete state structure is explicitly modeled with a neural CRF\nover the changing hidden representation of the entity. This CRF can explicitly\ncapture constraints on entity states over time, enforcing that, for example, an\nentity cannot move to a location after it is destroyed. We evaluate the\nperformance of our proposed model on QA tasks over process paragraphs in the\nProPara dataset and find that our model achieves state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03651v2",
            "date": "2019-04-07 13:47:28+00:00",
            "title": "SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression",
            "authors": [
                "Christos Baziotis",
                "Ion Androutsopoulos",
                "Ioannis Konstas",
                "Alexandros Potamianos"
            ],
            "abstract": "Neural sequence-to-sequence models are currently the dominant approach in\nseveral natural language processing tasks, but require large parallel corpora.\nWe present a sequence-to-sequence-to-sequence autoencoder (SEQ^3), consisting\nof two chained encoder-decoder pairs, with words used as a sequence of discrete\nlatent variables. We apply the proposed model to unsupervised abstractive\nsentence compression, where the first and last sequences are the input and\nreconstructed sentences, respectively, while the middle sequence is the\ncompressed sentence. Constraining the length of the latent word sequences\nforces the model to distill important information from the input. A pretrained\nlanguage model, acting as a prior over the latent sequences, encourages the\ncompressed sentences to be human-readable. Continuous relaxations enable us to\nsample from categorical distributions, allowing gradient-based optimization,\nunlike alternatives that rely on reinforcement learning. The proposed model\ndoes not require parallel text-summary pairs, achieving promising results in\nunsupervised sentence compression on benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03799v2",
            "date": "2019-04-08 01:50:45+00:00",
            "title": "Enriching Rare Word Representations in Neural Language Models by Embedding Matrix Augmentation",
            "authors": [
                "Yerbolat Khassanov",
                "Zhiping Zeng",
                "Van Tung Pham",
                "Haihua Xu",
                "Eng Siong Chng"
            ],
            "abstract": "The neural language models (NLM) achieve strong generalization capability by\nlearning the dense representation of words and using them to estimate\nprobability distribution function. However, learning the representation of rare\nwords is a challenging problem causing the NLM to produce unreliable\nprobability estimates. To address this problem, we propose a method to enrich\nrepresentations of rare words in pre-trained NLM and consequently improve its\nprobability estimation performance. The proposed method augments the word\nembedding matrices of pre-trained NLM while keeping other parameters unchanged.\nSpecifically, our method updates the embedding vectors of rare words using\nembedding vectors of other semantically and syntactically similar words. To\nevaluate the proposed method, we enrich the rare street names in the\npre-trained NLM and use it to rescore 100-best hypotheses output from the\nSingapore English speech recognition system. The enriched NLM reduces the word\nerror rate by 6% relative and improves the recognition accuracy of the rare\nwords by 16% absolute as compared to the baseline NLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03802v2",
            "date": "2019-04-08 02:16:44+00:00",
            "title": "Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data",
            "authors": [
                "Yerbolat Khassanov",
                "Haihua Xu",
                "Van Tung Pham",
                "Zhiping Zeng",
                "Eng Siong Chng",
                "Chongjia Ni",
                "Bin Ma"
            ],
            "abstract": "The lack of code-switch training data is one of the major concerns in the\ndevelopment of end-to-end code-switching automatic speech recognition (ASR)\nmodels. In this work, we propose a method to train an improved end-to-end\ncode-switching ASR using only monolingual data. Our method encourages the\ndistributions of output token embeddings of monolingual languages to be\nsimilar, and hence, promotes the ASR model to easily code-switch between\nlanguages. Specifically, we propose to use Jensen-Shannon divergence and cosine\ndistance based constraints. The former will enforce output embeddings of\nmonolingual languages to possess similar distributions, while the later simply\nbrings the centroids of two distributions to be close to each other.\nExperimental results demonstrate high effectiveness of the proposed method,\nyielding up to 4.5% absolute mixed error rate improvement on Mandarin-English\ncode-switching ASR task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03879v2",
            "date": "2019-04-08 08:00:25+00:00",
            "title": "Improving Domain Adaptation Translation with Domain Invariant and Specific Information",
            "authors": [
                "Shuhao Gu",
                "Yang Feng",
                "Qun Liu"
            ],
            "abstract": "In domain adaptation for neural machine translation, translation performance\ncan benefit from separating features into domain-specific features and common\nfeatures. In this paper, we propose a method to explicitly model the two kinds\nof information in the encoder-decoder framework so as to exploit out-of-domain\ndata in in-domain training. In our method, we maintain a private encoder and a\nprivate decoder for each domain which are used to model domain-specific\ninformation. In the meantime, we introduce a common encoder and a common\ndecoder shared by all the domains which can only have domain-independent\ninformation flow through. Besides, we add a discriminator to the shared encoder\nand employ adversarial training for the whole model to reinforce the\nperformance of information separation and machine translation simultaneously.\nExperiment results show that our method can outperform competitive baselines\ngreatly on multiple data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03922v1",
            "date": "2019-04-08 10:02:05+00:00",
            "title": "Crosslingual Document Embedding as Reduced-Rank Ridge Regression",
            "authors": [
                "Martin Josifoski",
                "Ivan S. Paskov",
                "Hristo S. Paskov",
                "Martin Jaggi",
                "Robert West"
            ],
            "abstract": "There has recently been much interest in extending vector-based word\nrepresentations to multiple languages, such that words can be compared across\nlanguages. In this paper, we shift the focus from words to documents and\nintroduce a method for embedding documents written in any language into a\nsingle, language-independent vector space. For training, our approach leverages\na multilingual corpus where the same concept is covered in multiple languages\n(but not necessarily via exact translations), such as Wikipedia. Our method,\nCr5 (Crosslingual reduced-rank ridge regression), starts by training a\nridge-regression-based classifier that uses language-specific bag-of-word\nfeatures in order to predict the concept that a given document is about. We\nshow that, when constraining the learned weight matrix to be of low rank, it\ncan be factored to obtain the desired mappings from language-specific\nbags-of-words to language-independent embeddings. As opposed to most prior\nmethods, which use pretrained monolingual word vectors, postprocess them to\nmake them crosslingual, and finally average word vectors to obtain document\nvectors, Cr5 is trained end-to-end and is thus natively crosslingual as well as\ndocument-level. Moreover, since our algorithm uses the singular value\ndecomposition as its core operation, it is highly scalable. Experiments show\nthat our method achieves state-of-the-art performance on a crosslingual\ndocument retrieval task. Finally, although not trained for embedding sentences\nand words, it also achieves competitive performance on crosslingual sentence\nand word retrieval tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04032v3",
            "date": "2019-04-08 12:56:34+00:00",
            "title": "Evaluation of Greek Word Embeddings",
            "authors": [
                "Stamatis Outsios",
                "Christos Karatsalos",
                "Konstantinos Skianis",
                "Michalis Vazirgiannis"
            ],
            "abstract": "Since word embeddings have been the most popular input for many NLP tasks,\nevaluating their quality is of critical importance. Most research efforts are\nfocusing on English word embeddings. This paper addresses the problem of\nconstructing and evaluating such models for the Greek language. We created a\nnew word analogy corpus considering the original English Word2vec word analogy\ncorpus and some specific linguistic aspects of the Greek language as well.\nMoreover, we created a Greek version of WordSim353 corpora for a basic\nevaluation of word similarities. We tested seven word vector models and our\nevaluation showed that we are able to create meaningful representations. Last,\nwe discovered that the morphological complexity of the Greek language and\npolysemy can influence the quality of the resulting word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04073v1",
            "date": "2019-04-05 03:49:22+00:00",
            "title": "Abusive Language Detection with Graph Convolutional Networks",
            "authors": [
                "Pushkar Mishra",
                "Marco Del Tredici",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "Abuse on the Internet represents a significant societal problem of our time.\nPrevious research on automated abusive language detection in Twitter has shown\nthat community-based profiling of users is a promising technique for this task.\nHowever, existing approaches only capture shallow properties of online\ncommunities by modeling follower-following relationships. In contrast, working\nwith graph convolutional networks (GCNs), we present the first approach that\ncaptures not only the structure of online communities but also the linguistic\nbehavior of the users within them. We show that such a heterogeneous\ngraph-structured modeling of communities significantly advances the current\nstate of the art in abusive language detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04163v1",
            "date": "2019-04-08 16:16:01+00:00",
            "title": "Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization",
            "authors": [
                "Yangyang Shi",
                "Mei-Yuh Hwang",
                "Xin Lei",
                "Haoyu Sheng"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) have dominated language modeling because of\ntheir superior performance over traditional N-gram based models. In many\napplications, a large Recurrent Neural Network language model (RNNLM) or an\nensemble of several RNNLMs is used. These models have large memory footprints\nand require heavy computation. In this paper, we examine the effect of applying\nknowledge distillation in reducing the model size for RNNLMs. In addition, we\npropose a trust regularization method to improve the knowledge distillation\ntraining for RNNLMs. Using knowledge distillation with trust regularization, we\nreduce the parameter size to a third of that of the previously published best\nmodel while maintaining the state-of-the-art perplexity result on Penn Treebank\ndata. In a speech recognition N-bestrescoring task, we reduce the RNNLM model\nsize to 18.5% of the baseline system, with no degradation in word error\nrate(WER) performance on Wall Street Journal data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04307v1",
            "date": "2019-04-08 19:18:09+00:00",
            "title": "Word Similarity Datasets for Thai: Construction and Evaluation",
            "authors": [
                "Ponrudee Netisopakul",
                "Gerhard Wohlgenannt",
                "Aleksei Pulich"
            ],
            "abstract": "Distributional semantics in the form of word embeddings are an essential\ningredient to many modern natural language processing systems. The\nquantification of semantic similarity between words can be used to evaluate the\nability of a system to perform semantic interpretation. To this end, a number\nof word similarity datasets have been created for the English language over the\nlast decades. For Thai language few such resources are available. In this work,\nwe create three Thai word similarity datasets by translating and re-rating the\npopular WordSim-353, SimLex-999 and SemEval-2017-Task-2 datasets. The three\ndatasets contain 1852 word pairs in total and have different characteristics in\nterms of difficulty, domain coverage, and notion of similarity (relatedness\nvs.~similarity). These features help to gain a broader picture of the\nproperties of an evaluated word embedding model. We include baseline\nevaluations with existing Thai embedding models, and identify the high ratio of\nout-of-vocabulary words as one of the biggest challenges. All datasets,\nevaluation results, and a tool for easy evaluation of new Thai embedding models\nare available to the NLP community online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04398v1",
            "date": "2019-04-08 23:43:16+00:00",
            "title": "Disfluencies and Human Speech Transcription Errors",
            "authors": [
                "Vicky Zayats",
                "Trang Tran",
                "Richard Wright",
                "Courtney Mansfield",
                "Mari Ostendorf"
            ],
            "abstract": "This paper explores contexts associated with errors in transcrip-tion of\nspontaneous speech, shedding light on human perceptionof disfluencies and other\nconversational speech phenomena. Anew version of the Switchboard corpus is\nprovided with disfluency annotations for careful speech transcripts, together\nwith results showing the impact of transcription errors on evaluation of\nautomatic disfluency detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04428v2",
            "date": "2019-04-09 02:34:30+00:00",
            "title": "Text Generation with Exemplar-based Adaptive Decoding",
            "authors": [
                "Hao Peng",
                "Ankur P. Parikh",
                "Manaal Faruqui",
                "Bhuwan Dhingra",
                "Dipanjan Das"
            ],
            "abstract": "We propose a novel conditioned text generation model. It draws inspiration\nfrom traditional template-based text generation techniques, where the source\nprovides the content (i.e., what to say), and the template influences how to\nsay it. Building on the successful encoder-decoder paradigm, it first encodes\nthe content representation from the given input text; to produce the output, it\nretrieves exemplar text from the training data as \"soft templates,\" which are\nthen used to construct an exemplar-specific decoder. We evaluate the proposed\nmodel on abstractive text summarization and data-to-text generation. Empirical\nresults show that this model achieves strong performance and outperforms\ncomparable baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04446v1",
            "date": "2019-04-09 03:25:53+00:00",
            "title": "HiGRU: Hierarchical Gated Recurrent Units for Utterance-level Emotion Recognition",
            "authors": [
                "Wenxiang Jiao",
                "Haiqin Yang",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "In this paper, we address three challenges in utterance-level emotion\nrecognition in dialogue systems: (1) the same word can deliver different\nemotions in different contexts; (2) some emotions are rarely seen in general\ndialogues; (3) long-range contextual information is hard to be effectively\ncaptured. We therefore propose a hierarchical Gated Recurrent Unit (HiGRU)\nframework with a lower-level GRU to model the word-level inputs and an\nupper-level GRU to capture the contexts of utterance-level embeddings.\nMoreover, we promote the framework to two variants, HiGRU with individual\nfeatures fusion (HiGRU-f) and HiGRU with self-attention and features fusion\n(HiGRU-sf), so that the word/utterance-level individual inputs and the\nlong-range contextual information can be sufficiently utilized. Experiments on\nthree dialogue emotion datasets, IEMOCAP, Friends, and EmotionPush demonstrate\nthat our proposed HiGRU models attain at least 8.7%, 7.5%, 6.0% improvement\nover the state-of-the-art methods on each dataset, respectively. Particularly,\nby utilizing only the textual feature in IEMOCAP, our HiGRU models gain at\nleast 3.8% improvement over the state-of-the-art conversational memory network\n(CMN) with the trimodal features of text, video, and audio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04461v2",
            "date": "2019-04-09 04:27:31+00:00",
            "title": "Mixing syntagmatic and paradigmatic information for concept detection",
            "authors": [
                "Louis Chartrand",
                "Mohamed Bouguessa"
            ],
            "abstract": "In the last decades, philosophers have begun using empirical data for\nconceptual analysis, but corpus-based conceptual analysis has so far failed to\ndevelop, in part because of the absence of reliable methods to automatically\ndetect concepts in textual data. Previous attempts have shown that topic models\ncan constitute efficient concept detection heuristics, but while they leverage\nthe syntagmatic relations in a corpus, they fail to exploit paradigmatic\nrelations, and thus probably fail to model concepts accurately. In this\narticle, we show that using a topic model that models concepts on a space of\nword embeddings (Hu and Tsujii, 2016) can lead to significant increases in\nconcept detection performance, as well as enable the target concept to be\nexpressed in more flexible ways using word vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04479v4",
            "date": "2019-04-09 06:06:54+00:00",
            "title": "Who Needs Words? Lexicon-Free Speech Recognition",
            "authors": [
                "Tatiana Likhomanenko",
                "Gabriel Synnaeve",
                "Ronan Collobert"
            ],
            "abstract": "Lexicon-free speech recognition naturally deals with the problem of\nout-of-vocabulary (OOV) words. In this paper, we show that character-based\nlanguage models (LM) can perform as well as word-based LMs for speech\nrecognition, in word error rates (WER), even without restricting the decoding\nto a lexicon. We study character-based LMs and show that convolutional LMs can\neffectively leverage large (character) contexts, which is key for good speech\nrecognition performance downstream. We specifically show that the lexicon-free\ndecoding performance (WER) on utterances with OOV words using character-based\nLMs is better than lexicon-based decoding, both with character or word-based\nLMs.",
            "journal ref": "Proc. Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04498v1",
            "date": "2019-04-09 07:26:25+00:00",
            "title": "A Hierarchical Decoding Model For Spoken Language Understanding From Unaligned Data",
            "authors": [
                "Zijian Zhao",
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "Spoken language understanding (SLU) systems can be trained on two types of\nlabelled data: aligned or unaligned. Unaligned data do not require word by word\nannotation and is easier to be obtained. In the paper, we focus on spoken\nlanguage understanding from unaligned data whose annotation is a set of\nact-slot-value triples. Previous works usually focus on improve slot-value pair\nprediction and estimate dialogue act types separately, which ignores the\nhierarchical structure of the act-slot-value triples. Here, we propose a novel\nhierarchical decoding model which dynamically parses act, slot and value in a\nstructured way and employs pointer network to handle out-of-vocabulary (OOV)\nvalues. Experiments on DSTC2 dataset, a benchmark unaligned dataset, show that\nthe proposed model not only outperforms previous state-of-the-art model, but\nalso can be generalized effectively and efficiently to unseen act-slot type\npairs and OOV values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04790v2",
            "date": "2019-04-09 17:10:15+00:00",
            "title": "APE at Scale and its Implications on MT Evaluation Biases",
            "authors": [
                "Markus Freitag",
                "Isaac Caswell",
                "Scott Roy"
            ],
            "abstract": "In this work, we train an Automatic Post-Editing (APE) model and use it to\nreveal biases in standard Machine Translation (MT) evaluation procedures. The\ngoal of our APE model is to correct typical errors introduced by the\ntranslation process, and convert the \"translationese\" output into natural text.\nOur APE model is trained entirely on monolingual data that has been round-trip\ntranslated through English, to mimic errors that are similar to the ones\nintroduced by NMT. We apply our model to the output of existing NMT systems,\nand demonstrate that, while the human-judged quality improves in all cases,\nBLEU scores drop with forward-translated test sets. We verify these results for\nthe WMT18 English to German, WMT15 English to French, and WMT16 English to\nRomanian tasks. Furthermore, we selectively apply our APE model on the output\nof the top submissions of the most recent WMT evaluation campaigns. We see\nquality improvements on all tasks of up to 2.5 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04792v2",
            "date": "2019-04-09 17:13:36+00:00",
            "title": "Quizbowl: The Case for Incremental Question Answering",
            "authors": [
                "Pedro Rodriguez",
                "Shi Feng",
                "Mohit Iyyer",
                "He He",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Scholastic trivia competitions test knowledge and intelligence through\nmastery of question answering. Modern question answering benchmarks are one\nvariant of the Turing test. Specifically, answering a set of questions as well\nas a human is a minimum bar towards demonstrating human-like intelligence. This\npaper makes the case that the format of one competition -- where participants\ncan answer in the middle of hearing a question (incremental) -- better\ndifferentiates the skill between (human or machine) players. Additionally,\nmerging a sequential decision-making sub-task with question answering (QA)\nprovides a good setting for research in model calibration and opponent\nmodeling. Thus, embedded in this task are three machine learning challenges:\n(1) factoid QA over thousands of Wikipedia-like answers, (2) calibration of the\nQA model's confidence scores, and (3) sequential decision-making that\nincorporates knowledge of the QA model, its calibration, and what the opponent\nmay do. We make two contributions: (1) collecting and curating a large factoid\nQA dataset and an accompanying gameplay dataset, and (2) developing a model\nthat addresses these three machine learning challenges. In addition to offline\nevaluation, we pitted our model against some of the most accomplished trivia\nplayers in the world in a series of exhibition matches spanning several years.\nThroughout this paper, we show that collaborations with the vibrant trivia\ncommunity have contributed to the quality of our dataset, spawned new research\ndirections, and doubled as an exciting way to engage the public with research\nin machine learning and natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04896v1",
            "date": "2019-04-09 20:35:25+00:00",
            "title": "Performance Monitoring for End-to-End Speech Recognition",
            "authors": [
                "Ruizhi Li",
                "Gregory Sell",
                "Hynek Hermansky"
            ],
            "abstract": "Measuring performance of an automatic speech recognition (ASR) system without\nground-truth could be beneficial in many scenarios, especially with data from\nunseen domains, where performance can be highly inconsistent. In conventional\nASR systems, several performance monitoring (PM) techniques have been\nwell-developed to monitor performance by looking at tri-phone posteriors or\npre-softmax activations from neural network acoustic modeling. However,\nstrategies for monitoring more recently developed end-to-end ASR systems have\nnot yet been explored, and so that is the focus of this paper. We adapt\nprevious PM measures (Entropy, M-measure and Auto-encoder) and apply our\nproposed RNN predictor in the end-to-end setting. These measures utilize the\ndecoder output layer and attention probability vectors, and their predictive\npower is measured with simple linear models. Our findings suggest that\ndecoder-level features are more feasible and informative than attention-level\nprobabilities for PM measures, and that M-measure on the decoder posteriors\nachieves the best overall predictive performance with an average prediction\nerror 8.8%. Entropy measures and RNN-based prediction also show competitive\npredictability, especially for unseen conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04900v1",
            "date": "2019-04-09 20:39:07+00:00",
            "title": "Data Selection with Cluster-Based Language Difference Models and Cynical Selection",
            "authors": [
                "Luc\u00eda Santamar\u00eda",
                "Amittai Axelrod"
            ],
            "abstract": "We present and apply two methods for addressing the problem of selecting\nrelevant training data out of a general pool for use in tasks such as machine\ntranslation. Building on existing work on class-based language difference\nmodels, we first introduce a cluster-based method that uses Brown clusters to\ncondense the vocabulary of the corpora. Secondly, we implement the cynical data\nselection method, which incrementally constructs a training corpus to\nefficiently model the task corpus. Both the cluster-based and the cynical data\nselection approaches are used for the first time within a machine translation\nsystem, and we perform a head-to-head comparison. Our intrinsic evaluations\nshow that both new methods outperform the standard Moore-Lewis approach\n(cross-entropy difference), in terms of better perplexity and OOV rates on\nin-domain data. The cynical approach converges much quicker, covering nearly\nall of the in-domain vocabulary with 84% less data than the other methods.\nFurthermore, the new approaches can be used to select machine translation\ntraining data for training better systems. Our results confirm that class-based\nselection using Brown clusters is a viable alternative to POS-based class-based\nmethods, and removes the reliance on a part-of-speech tagger. Additionally, we\nare able to validate the recently proposed cynical data selection method,\nshowing that its performance in SMT models surpasses that of traditional\ncross-entropy difference methods and more closely matches the sentence length\nof the task corpus.",
            "journal ref": "Proceedings of the International Workshop on Spoken Language\n  Translation (IWSLT) 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05055v1",
            "date": "2019-04-10 08:24:06+00:00",
            "title": "A Variational Approach to Weakly Supervised Document-Level Multi-Aspect Sentiment Classification",
            "authors": [
                "Ziqian Zeng",
                "Wenxuan Zhou",
                "Xin Liu",
                "Yangqiu Song"
            ],
            "abstract": "In this paper, we propose a variational approach to weakly supervised\ndocument-level multi-aspect sentiment classification. Instead of using\nuser-generated ratings or annotations provided by domain experts, we use\ntarget-opinion word pairs as \"supervision.\" These word pairs can be extracted\nby using dependency parsers and simple rules. Our objective is to predict an\nopinion word given a target word while our ultimate goal is to learn a\nsentiment polarity classifier to predict the sentiment polarity of each aspect\ngiven a document. By introducing a latent variable, i.e., the sentiment\npolarity, to the objective function, we can inject the sentiment polarity\nclassifier to the objective via the variational lower bound. We can learn a\nsentiment polarity classifier by optimizing the lower bound. We show that our\nmethod can outperform weakly supervised baselines on TripAdvisor and\nBeerAdvocate datasets and can be comparable to the state-of-the-art supervised\nmethod with hundreds of labels per aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05152v1",
            "date": "2019-04-10 12:56:50+00:00",
            "title": "NLPR@SRPOL at SemEval-2019 Task 6 and Task 5: Linguistically enhanced deep learning offensive sentence classifier",
            "authors": [
                "Alessandro Seganti",
                "Helena Sobol",
                "Iryna Orlova",
                "Hannam Kim",
                "Jakub Staniszewski",
                "Tymoteusz Krumholc",
                "Krystian Koziel"
            ],
            "abstract": "The paper presents a system developed for the SemEval-2019 competition Task 5\nhat-Eval Basile et al. (2019) (team name: LU Team) and Task 6 OffensEval\nZampieri et al. (2019b) (team name: NLPR@SRPOL), where we achieved 2nd position\nin Subtask C. The system combines in an ensemble several models (LSTM,\nTransformer, OpenAI's GPT, Random forest, SVM) with various embeddings (custom,\nELMo, fastText, Universal Encoder) together with additional linguistic features\n(number of blacklisted words, special characters, etc.). The system works with\na multi-tier blacklist and a large corpus of crawled data, annotated for\ngeneral offensiveness. In the paper we do an extensive analysis of our results\nand show how the combination of features and embedding affect the performance\nof the models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05255v1",
            "date": "2019-04-10 15:52:13+00:00",
            "title": "Simple BERT Models for Relation Extraction and Semantic Role Labeling",
            "authors": [
                "Peng Shi",
                "Jimmy Lin"
            ],
            "abstract": "We present simple BERT-based models for relation extraction and semantic role\nlabeling. In recent years, state-of-the-art performance has been achieved using\nneural models by incorporating lexical and syntactic features such as\npart-of-speech tags and dependency trees. In this paper, extensive experiments\non datasets for these two tasks show that without using any external features,\na simple BERT-based model can achieve state-of-the-art performance. To our\nknowledge, we are the first to successfully apply BERT in this manner. Our\nmodels provide strong baselines for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05298v1",
            "date": "2019-04-10 16:59:29+00:00",
            "title": "CNM: An Interpretable Complex-valued Network for Matching",
            "authors": [
                "Qiuchi Li",
                "Benyou Wang",
                "Massimo Melucci"
            ],
            "abstract": "This paper seeks to model human language by the mathematical framework of\nquantum physics. With the well-designed mathematical formulations in quantum\nphysics, this framework unifies different linguistic units in a single\ncomplex-valued vector space, e.g. words as particles in quantum states and\nsentences as mixed systems. A complex-valued network is built to implement this\nframework for semantic matching. With well-constrained complex-valued\ncomponents, the network admits interpretations to explicit physical meanings.\nThe proposed complex-valued network for matching (CNM) achieves comparable\nperformances to strong CNN and RNN baselines on two benchmarking question\nanswering (QA) datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05527v1",
            "date": "2019-04-11 04:38:35+00:00",
            "title": "Modeling Global Syntactic Variation in English Using Dialect Classification",
            "authors": [
                "Jonathan Dunn"
            ],
            "abstract": "This paper evaluates global-scale dialect identification for 14 national\nvarieties of English as a means for studying syntactic variation. The paper\nmakes three main contributions: (i) introducing data-driven language mapping as\na method for selecting the inventory of national varieties to include in the\ntask; (ii) producing a large and dynamic set of syntactic features using\ngrammar induction rather than focusing on a few hand-selected features such as\nfunction words; and (iii) comparing models across both web corpora and social\nmedia corpora in order to measure the robustness of syntactic variation across\nregisters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05529v1",
            "date": "2019-04-11 04:41:46+00:00",
            "title": "Frequency vs. Association for Constraint Selection in Usage-Based Construction Grammar",
            "authors": [
                "Jonathan Dunn"
            ],
            "abstract": "A usage-based Construction Grammar (CxG) posits that slot-constraints\ngeneralize from common exemplar constructions. But what is the best model of\nconstraint generalization? This paper evaluates competing frequency-based and\nassociation-based models across eight languages using a metric derived from the\nMinimum Description Length paradigm. The experiments show that\nassociation-based models produce better generalizations across all languages by\na significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05542v1",
            "date": "2019-04-11 05:41:27+00:00",
            "title": "Scalable Cross-Lingual Transfer of Neural Sentence Embeddings",
            "authors": [
                "Hanan Aldarmaki",
                "Mona Diab"
            ],
            "abstract": "We develop and investigate several cross-lingual alignment approaches for\nneural sentence embedding models, such as the supervised inference classifier,\nInferSent, and sequential encoder-decoder models. We evaluate three alignment\nframeworks applied to these models: joint modeling, representation transfer\nlearning, and sentence mapping, using parallel text to guide the alignment. Our\nresults support representation transfer as a scalable approach for modular\ncross-lingual alignment of neural sentence embeddings, where we observe better\nperformance compared to joint models in intrinsic and extrinsic evaluations,\nparticularly with smaller sets of parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05557v1",
            "date": "2019-04-11 07:08:09+00:00",
            "title": "Searching News Articles Using an Event Knowledge Graph Leveraged by Wikidata",
            "authors": [
                "Charlotte Rudnik",
                "Thibault Ehrhart",
                "Olivier Ferret",
                "Denis Teyssou",
                "Rapha\u00ebl Troncy",
                "Xavier Tannier"
            ],
            "abstract": "News agencies produce thousands of multimedia stories describing events\nhappening in the world that are either scheduled such as sports competitions,\npolitical summits and elections, or breaking events such as military conflicts,\nterrorist attacks, natural disasters, etc. When writing up those stories,\njournalists refer to contextual background and to compare with past similar\nevents. However, searching for precise facts described in stories is hard. In\nthis paper, we propose a general method that leverages the Wikidata knowledge\nbase to produce semantic annotations of news articles. Next, we describe a\nsemantic search engine that supports both keyword based search in news articles\nand structured data search providing filters for properties belonging to\nspecific event schemas that are automatically inferred.",
            "journal ref": "WikiWorkshop at The Web Conference 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05569v1",
            "date": "2019-04-11 07:57:51+00:00",
            "title": "A high quality and phonetic balanced speech corpus for Vietnamese",
            "authors": [
                "Pham Ngoc Phuong",
                "Quoc Truong Do",
                "Luong Chi Mai"
            ],
            "abstract": "This paper presents a high quality Vietnamese speech corpus that can be used\nfor analyzing Vietnamese speech characteristic as well as building speech\nsynthesis models. The corpus consists of 5400 clean-speech utterances spoken by\n12 speakers including 6 males and 6 females. The corpus is designed with\nphonetic balanced in mind so that it can be used for speech synthesis,\nespecially, speech adaptation approaches. Specifically, all speakers utter a\ncommon dataset contains 250 phonetic balanced sentences. To increase the\nvariety of speech context, each speaker also utters another 200 non-shared,\nphonetic-balanced sentences. The speakers are selected to cover a wide range of\nage and come from different regions of the North of Vietnam. The audios are\nrecorded in a soundproof studio room, they are sampling at 48 kHz, 16 bits PCM,\nmono channel.",
            "journal ref": "Oriental COCOSDA 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05588v1",
            "date": "2019-04-11 09:06:24+00:00",
            "title": "Modeling the Complexity and Descriptive Adequacy of Construction Grammars",
            "authors": [
                "Jonathan Dunn"
            ],
            "abstract": "This paper uses the Minimum Description Length paradigm to model the\ncomplexity of CxGs (operationalized as the encoding size of a grammar)\nalongside their descriptive adequacy (operationalized as the encoding size of a\ncorpus given a grammar). These two quantities are combined to measure the\nquality of potential CxGs against unannotated corpora, supporting\ndiscovery-device CxGs for English, Spanish, French, German, and Italian. The\nresults show (i) that these grammars provide significant generalizations as\nmeasured using compression and (ii) that more complex CxGs with access to\nmultiple levels of representation provide greater generalizations than\nsingle-representation CxGs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05862v4",
            "date": "2019-04-11 17:29:30+00:00",
            "title": "wav2vec: Unsupervised Pre-training for Speech Recognition",
            "authors": [
                "Steffen Schneider",
                "Alexei Baevski",
                "Ronan Collobert",
                "Michael Auli"
            ],
            "abstract": "We explore unsupervised pre-training for speech recognition by learning\nrepresentations of raw audio. wav2vec is trained on large amounts of unlabeled\naudio data and the resulting representations are then used to improve acoustic\nmodel training. We pre-train a simple multi-layer convolutional neural network\noptimized via a noise contrastive binary classification task. Our experiments\non WSJ reduce WER of a strong character-based log-mel filterbank baseline by up\nto 36% when only a few hours of transcribed data is available. Our approach\nachieves 2.43% WER on the nov92 test set. This outperforms Deep Speech 2, the\nbest reported character-based system in the literature while using two orders\nof magnitude less labeled training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05929v1",
            "date": "2019-04-11 19:04:57+00:00",
            "title": "Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation",
            "authors": [
                "Ori Shapira",
                "David Gabay",
                "Yang Gao",
                "Hadar Ronen",
                "Ramakanth Pasunuru",
                "Mohit Bansal",
                "Yael Amsterdamer",
                "Ido Dagan"
            ],
            "abstract": "Conducting a manual evaluation is considered an essential part of summary\nevaluation methodology. Traditionally, the Pyramid protocol, which exhaustively\ncompares system summaries to references, has been perceived as very reliable,\nproviding objective scores. Yet, due to the high cost of the Pyramid method and\nthe required expertise, researchers resorted to cheaper and less thorough\nmanual evaluation methods, such as Responsiveness and pairwise comparison,\nattainable via crowdsourcing. We revisit the Pyramid approach, proposing a\nlightweight sampling-based version that is crowdsourcable. We analyze the\nperformance of our method in comparison to original expert-based Pyramid\nevaluations, showing higher correlation relative to the common Responsiveness\nmethod. We release our crowdsourced Summary-Content-Units, along with all\ncrowdsourcing scripts, for future evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05953v1",
            "date": "2019-04-11 20:50:18+00:00",
            "title": "Strong Baselines for Complex Word Identification across Multiple Languages",
            "authors": [
                "Pierre Finnimore",
                "Elisabeth Fritzsch",
                "Daniel King",
                "Alison Sneyd",
                "Aneeq Ur Rehman",
                "Fernando Alva-Manchego",
                "Andreas Vlachos"
            ],
            "abstract": "Complex Word Identification (CWI) is the task of identifying which words or\nphrases in a sentence are difficult to understand by a target audience. The\nlatest CWI Shared Task released data for two settings: monolingual (i.e. train\nand test in the same language) and cross-lingual (i.e. test in a language not\nseen during training). The best monolingual models relied on language-dependent\nfeatures, which do not generalise in the cross-lingual setting, while the best\ncross-lingual model used neural networks with multi-task learning. In this\npaper, we present monolingual and cross-lingual CWI models that perform as well\nas (or better than) most models submitted to the latest CWI Shared Task. We\nshow that carefully selected features and simple learning models can achieve\nstate-of-the-art performance, and result in strong baselines for future\ndevelopment in this area. Finally, we discuss how inconsistencies in the\nannotation of the data can explain some of the results obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06002v1",
            "date": "2019-04-12 01:54:01+00:00",
            "title": "Modeling Interpersonal Linguistic Coordination in Conversations using Word Mover's Distance",
            "authors": [
                "Md Nasir",
                "Sandeep Nallan Chakravarthula",
                "Brian Baucom",
                "David C. Atkins",
                "Panayiotis Georgiou",
                "Shrikanth Narayanan"
            ],
            "abstract": "Linguistic coordination is a well-established phenomenon in spoken\nconversations and often associated with positive social behaviors and outcomes.\nWhile there have been many attempts to measure lexical coordination or\nentrainment in literature, only a few have explored coordination in syntactic\nor semantic space. In this work, we attempt to combine these different aspects\nof coordination into a single measure by leveraging distances in a neural word\nrepresentation space. In particular, we adopt the recently proposed Word\nMover's Distance with word2vec embeddings and extend it to measure the\ndissimilarity in language used in multiple consecutive speaker turns. To\nvalidate our approach, we apply this measure for two case studies in the\nclinical psychology domain. We find that our proposed measure is correlated\nwith the therapist's empathy towards their patient in Motivational Interviewing\nand with affective behaviors in Couples Therapy. In both case studies, our\nproposed metric exhibits higher correlation than previously proposed measures.\nWhen applied to the couples with relationship improvement, we also notice a\nsignificant decrease in the proposed measure over the course of therapy,\nindicating higher linguistic coordination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06101v1",
            "date": "2019-04-12 08:48:03+00:00",
            "title": "A Crowdsourced Frame Disambiguation Corpus with Ambiguity",
            "authors": [
                "Anca Dumitrache",
                "Lora Aroyo",
                "Chris Welty"
            ],
            "abstract": "We present a resource for the task of FrameNet semantic frame disambiguation\nof over 5,000 word-sentence pairs from the Wikipedia corpus. The annotations\nwere collected using a novel crowdsourcing approach with multiple workers per\nsentence to capture inter-annotator disagreement. In contrast to the typical\napproach of attributing the best single frame to each word, we provide a list\nof frames with disagreement-based scores that express the confidence with which\neach frame applies to the word. This is based on the idea that inter-annotator\ndisagreement is at least partly caused by ambiguity that is inherent to the\ntext and frames. We have found many examples where the semantics of individual\nframes overlap sufficiently to make them acceptable alternatives for\ninterpreting a sentence. We have argued that ignoring this ambiguity creates an\noverly arbitrary target for training and evaluating natural language processing\nsystems - if humans cannot agree, why would we expect the correct answer from a\nmachine to be any different? To process this data we also utilized an expanded\nlemma-set provided by the Framester system, which merges FN with WordNet to\nenhance coverage. Our dataset includes annotations of 1,000 sentence-word pairs\nwhose lemmas are not part of FN. Finally we present metrics for evaluating\nframe disambiguation systems that account for ambiguity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06217v2",
            "date": "2019-04-12 13:05:06+00:00",
            "title": "Political Text Scaling Meets Computational Semantics",
            "authors": [
                "Federico Nanni",
                "Goran Glavas",
                "Simone Paolo Ponzetto",
                "Heiner Stuckenschmidt"
            ],
            "abstract": "During the last fifteen years, text scaling approaches have become a central\nelement for the text-as-data community. However, they are based on the\nassumption that latent positions can be captured just by modeling\nword-frequency information from the different documents under study. We\nchallenge this by presenting a new semantically aware unsupervised scaling\nalgorithm, SemScale, which relies upon distributional representations of the\ndocuments under study. We conduct an extensive quantitative analysis over a\ncollection of speeches from the European Parliament in five different languages\nand from two different legislations, in order to understand whether a) an\napproach that is aware of semantics would better capture known underlying\npolitical dimensions compared to a frequency-based scaling method, b) such\npositioning correlates in particular with a specific subset of linguistic\ntraits, compared to the use of the entire text, and c) these findings hold\nacross different languages. To support further research on this new branch of\ntext scaling approaches, we release the employed dataset and evaluation\nsetting, an easy-to-use online demo, and a Python implementation of SemScale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06234v1",
            "date": "2019-04-12 13:52:29+00:00",
            "title": "IIT (BHU) Varanasi at MSR-SRST 2018: A Language Model Based Approach for Natural Language Generation",
            "authors": [
                "Shreyansh Singh",
                "Avi Chawla",
                "Ayush Sharma",
                "Anil Kumar Singh"
            ],
            "abstract": "This paper describes our submission system for the Shallow Track of Surface\nRealization Shared Task 2018 (SRST'18). The task was to convert genuine UD\nstructures, from which word order information had been removed and the tokens\nhad been lemmatized, into their correct sentential form. We divide the problem\nstatement into two parts, word reinflection and correct word order prediction.\nFor the first sub-problem, we use a Long Short Term Memory based\nEncoder-Decoder approach. For the second sub-problem, we present a Language\nModel (LM) based approach. We apply two different sub-approaches in the LM\nBased approach and the combined result of these two approaches is considered as\nthe final output of the system.",
            "journal ref": "Proceedings of the 1st Workshop on Multilingual Surface\n  Realisation (MSR), 56th Annual Meeting of the Association for Computational\n  Linguistics (ACL), July 2018, Melbourne, Australia",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06286v2",
            "date": "2019-04-12 15:46:31+00:00",
            "title": "CITE: A Corpus of Image-Text Discourse Relations",
            "authors": [
                "Malihe Alikhani",
                "Sreyasi Nag Chowdhury",
                "Gerard de Melo",
                "Matthew Stone"
            ],
            "abstract": "This paper presents a novel crowd-sourced resource for multimodal discourse:\nour resource characterizes inferences in image-text contexts in the domain of\ncooking recipes in the form of coherence relations. Like previous corpora\nannotating discourse structure between text arguments, such as the Penn\nDiscourse Treebank, our new corpus aids in establishing a better understanding\nof natural communication and common-sense reasoning, while our findings have\nimplications for a wide range of applications, such as understanding and\ngeneration of multimodal documents.",
            "journal ref": "2019 Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06470v1",
            "date": "2019-04-13 02:48:49+00:00",
            "title": "Legal Area Classification: A Comparative Study of Text Classifiers on Singapore Supreme Court Judgments",
            "authors": [
                "Jerrold Soh Tsin Howe",
                "Lim How Khang",
                "Ian Ernst Chai"
            ],
            "abstract": "This paper conducts a comparative study on the performance of various machine\nlearning (``ML'') approaches for classifying judgments into legal areas. Using\na novel dataset of 6,227 Singapore Supreme Court judgments, we investigate how\nstate-of-the-art NLP methods compare against traditional statistical models\nwhen applied to a legal corpus that comprised few but lengthy documents. All\napproaches tested, including topic model, word embedding, and language\nmodel-based classifiers, performed well with as little as a few hundred\njudgments. However, more work needs to be done to optimize state-of-the-art\nmethods for the legal domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06472v2",
            "date": "2019-04-13 02:59:48+00:00",
            "title": "A Repository of Conversational Datasets",
            "authors": [
                "Matthew Henderson",
                "Pawe\u0142 Budzianowski",
                "I\u00f1igo Casanueva",
                "Sam Coope",
                "Daniela Gerz",
                "Girish Kumar",
                "Nikola Mrk\u0161i\u0107",
                "Georgios Spithourakis",
                "Pei-Hao Su",
                "Ivan Vuli\u0107",
                "Tsung-Hsien Wen"
            ],
            "abstract": "Progress in Machine Learning is often driven by the availability of large\ndatasets, and consistent evaluation metrics for comparing modeling approaches.\nTo this end, we present a repository of conversational datasets consisting of\nhundreds of millions of examples, and a standardised evaluation procedure for\nconversational response selection models using '1-of-100 accuracy'. The\nrepository contains scripts that allow researchers to reproduce the standard\ndatasets, or to adapt the pre-processing and data filtering steps to their\nneeds. We introduce and evaluate several competitive baselines for\nconversational response selection, whose implementations are shared in the\nrepository, as well as a neural encoder model that is trained on the entire\ntraining set.",
            "journal ref": "Proceedings of the Workshop on NLP for Conversational AI (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06682v1",
            "date": "2019-04-14 11:52:40+00:00",
            "title": "From News to Medical: Cross-domain Discourse Segmentation",
            "authors": [
                "Elisa Ferracane",
                "Titan Page",
                "Junyi Jessy Li",
                "Katrin Erk"
            ],
            "abstract": "The first step in discourse analysis involves dividing a text into segments.\nWe annotate the first high-quality small-scale medical corpus in English with\ndiscourse segments and analyze how well news-trained segmenters perform on this\ndomain. While we expectedly find a drop in performance, the nature of the\nsegmentation errors suggests some problems can be addressed earlier in the\npipeline, while others would require expanding the corpus to a trainable size\nto learn the nuances of the medical domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06779v1",
            "date": "2019-04-14 23:20:34+00:00",
            "title": "No Adjective Ordering Mystery, and No Raven Paradox, Just an Ontological Mishap",
            "authors": [
                "Walid S. Saba"
            ],
            "abstract": "In the concluding remarks of Ontological Promiscuity Hobbs (1985) made what\nwe believe to be a very insightful observation: given that semantics is an\nattempt at specifying the relation between language and the world, if \"one can\nassume a theory of the world that is isomorphic to the way we talk about it ...\nthen semantics becomes nearly trivial\". But how exactly can we rectify our\nlogical formalisms so that semantics, an endeavor that has occupied the most\npenetrating minds for over two centuries, can become (nearly) trivial, and what\nexactly does it mean to assume a theory of the world in our semantics? In this\npaper we hope to provide answers for both questions. First, we believe that a\ncommonsense theory of the world can (and should) be embedded in our semantic\nformalisms resulting in a logical semantics grounded in commonsense\nmetaphysics. Moreover, we believe the first step to accomplishing this vision\nis rectifying what we think was a crucial oversight in logical semantics,\nnamely the failure to distinguish between two fundamentally different types of\nconcepts: (i) ontological concepts, that correspond to what Cocchiarella (2001)\ncalls first-intension concepts and are types in a strongly-typed ontology; and\n(ii) logical concepts (or second intension concepts), that are predicates\ncorresponding to properties of (and relations between) objects of various\nontological types1. In such a framework, which we will refer to henceforth by\nontologik, it will be shown how type unification and other type operations can\nbe used to account for the `missing text phenomenon' (MTP) (see Saba, 2019a)\nthat is at the heart of most challenges in the semantics of natural language,\nby uncovering the significant amount of missing text that is never explicitly\nstated in everyday discourse, but is often implicitly assumed as shared\nbackground knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06828v1",
            "date": "2019-04-15 03:40:16+00:00",
            "title": "Pun Generation with Surprise",
            "authors": [
                "He He",
                "Nanyun Peng",
                "Percy Liang"
            ],
            "abstract": "We tackle the problem of generating a pun sentence given a pair of homophones\n(e.g., \"died\" and \"dyed\"). Supervised text generation is inappropriate due to\nthe lack of a large corpus of puns, and even if such a corpus existed, mimicry\nis at odds with generating novel content. In this paper, we propose an\nunsupervised approach to pun generation using a corpus of unhumorous text and\nwhat we call the local-global surprisal principle: we posit that in a pun\nsentence, there is a strong association between the pun word (e.g., \"dyed\") and\nthe distant context, as well as a strong association between the alternative\nword (e.g., \"died\") and the immediate context. This contrast creates surprise\nand thus humor. We instantiate this principle for pun generation in two ways:\n(i) as a measure based on the ratio of probabilities under a language model,\nand (ii) a retrieve-and-edit approach based on words suggested by a skip-gram\nmodel. Human evaluation shows that our retrieve-and-edit approach generates\npuns successfully 31% of the time, tripling the success rate of a neural\ngeneration baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07142v1",
            "date": "2019-04-15 15:51:15+00:00",
            "title": "Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation",
            "authors": [
                "Sebastian Gehrmann",
                "Steven Layne",
                "Franck Dernoncourt"
            ],
            "abstract": "Titles of short sections within long documents support readers by guiding\ntheir focus towards relevant passages and by providing anchor-points that help\nto understand the progression of the document. The positive effects of section\ntitles are even more pronounced when measured on readers with less developed\nreading abilities, for example in communities with limited labeled text\nresources.\n  We, therefore, aim to develop techniques to generate section titles in\nlow-resource environments. In particular, we present an extractive pipeline for\nsection title generation by first selecting the most salient sentence and then\napplying deletion-based compression. Our compression approach is based on a\nSemi-Markov Conditional Random Field that leverages unsupervised\nword-representations such as ELMo or BERT, eliminating the need for a complex\nencoder-decoder architecture. The results show that this approach leads to\ncompetitive performance with sequence-to-sequence models with high resources,\nwhile strongly outperforming it with low resources. In a human-subject study\nacross subjects with varying reading abilities, we find that our section titles\nimprove the speed of completing comprehension tasks while retaining similar\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07209v1",
            "date": "2019-04-15 17:33:38+00:00",
            "title": "Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation",
            "authors": [
                "Matthias Sperber",
                "Graham Neubig",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "Speech translation has traditionally been approached through cascaded models\nconsisting of a speech recognizer trained on a corpus of transcribed speech,\nand a machine translation system trained on parallel texts. Several recent\nworks have shown the feasibility of collapsing the cascade into a single,\ndirect model that can be trained in an end-to-end fashion on a corpus of\ntranslated speech. However, experiments are inconclusive on whether the cascade\nor the direct model is stronger, and have only been conducted under the\nunrealistic assumption that both are trained on equal amounts of data, ignoring\nother available speech recognition and machine translation corpora.\n  In this paper, we demonstrate that direct speech translation models require\nmore data to perform well than cascaded models, and while they allow including\nauxiliary data through multi-task training, they are poor at exploiting such\ndata, putting them at a severe disadvantage. As a remedy, we propose the use of\nend-to-end trainable models with two attention mechanisms, the first\nestablishing source speech to source text alignments, the second modeling\nsource to target text alignment. We show that such models naturally decompose\ninto multi-task-trainable recognition and translation tasks and propose an\nattention-passing technique that alleviates error propagation issues in a\nprevious formulation of a model with two attention stages. Our proposed model\noutperforms all examined baselines and is able to exploit auxiliary training\ndata much more effectively than direct attentional models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07334v1",
            "date": "2019-04-15 21:36:21+00:00",
            "title": "Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection",
            "authors": [
                "Masahiro Kaneko",
                "Mamoru Komachi"
            ],
            "abstract": "It is known that a deep neural network model pre-trained with large-scale\ndata greatly improves the accuracy of various tasks, especially when there are\nresource constraints. However, the information needed to solve a given task can\nvary, and simply using the output of the final layer is not necessarily\nsufficient. Moreover, to our knowledge, exploiting large language\nrepresentation models to detect grammatical errors has not yet been studied. In\nthis work, we investigate the effect of utilizing information not only from the\nfinal layer but also from intermediate layers of a pre-trained language\nrepresentation model to detect grammatical errors. We propose a multi-head\nmulti-layer attention model that determines the appropriate layers in\nBidirectional Encoder Representation from Transformers (BERT). The proposed\nmethod achieved the best scores on three datasets for grammatical error\ndetection tasks, outperforming the current state-of-the-art method by 6.0\npoints on FCE, 8.2 points on CoNLL14, and 12.2 points on JFLEG in terms of\nF_0.5. We also demonstrate that by using multi-head multi-layer attention, our\nmodel can exploit a broader range of information for each token in a sentence\nthan a model that uses only the final layer's information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07391v1",
            "date": "2019-04-16 01:30:00+00:00",
            "title": "Be Concise and Precise: Synthesizing Open-Domain Entity Descriptions from Facts",
            "authors": [
                "Rajarshi Bhowmik",
                "Gerard de Melo"
            ],
            "abstract": "Despite being vast repositories of factual information, cross-domain\nknowledge graphs, such as Wikidata and the Google Knowledge Graph, only\nsparsely provide short synoptic descriptions for entities. Such descriptions\nthat briefly identify the most discernible features of an entity provide\nreaders with a near-instantaneous understanding of what kind of entity they are\nbeing presented. They can also aid in tasks such as named entity\ndisambiguation, ontological type determination, and answering entity queries.\nGiven the rapidly increasing numbers of entities in knowledge graphs, a fully\nautomated synthesis of succinct textual descriptions from underlying factual\ninformation is essential. To this end, we propose a novel fact-to-sequence\nencoder-decoder model with a suitable copy mechanism to generate concise and\nprecise textual descriptions of entities. In an in-depth evaluation, we\ndemonstrate that our method significantly outperforms state-of-the-art\nalternatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07418v1",
            "date": "2019-04-16 02:48:11+00:00",
            "title": "Positional Encoding to Control Output Sequence Length",
            "authors": [
                "Sho Takase",
                "Naoaki Okazaki"
            ],
            "abstract": "Neural encoder-decoder models have been successful in natural language\ngeneration tasks. However, real applications of abstractive summarization must\nconsider additional constraint that a generated summary should not exceed a\ndesired length. In this paper, we propose a simple but effective extension of a\nsinusoidal positional encoding (Vaswani et al., 2017) to enable neural\nencoder-decoder model to preserves the length constraint. Unlike in previous\nstudies where that learn embeddings representing each length, the proposed\nmethod can generate a text of any length even if the target length is not\npresent in training data. The experimental results show that the proposed\nmethod can not only control the generation length but also improve the ROUGE\nscores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07733v1",
            "date": "2019-04-16 14:39:21+00:00",
            "title": "Subjective Assessment of Text Complexity: A Dataset for German Language",
            "authors": [
                "Babak Naderi",
                "Salar Mohtaj",
                "Kaspar Ensikat",
                "Sebastian M\u00f6ller"
            ],
            "abstract": "This paper presents TextComplexityDE, a dataset consisting of 1000 sentences\nin German language taken from 23 Wikipedia articles in 3 different\narticle-genres to be used for developing text-complexity predictor models and\nautomatic text simplification in German language. The dataset includes\nsubjective assessment of different text-complexity aspects provided by German\nlearners in level A and B. In addition, it contains manual simplification of\n250 of those sentences provided by native speakers and subjective assessment of\nthe simplified sentences by participants from the target group. The subjective\nratings were collected using both laboratory studies and crowdsourcing\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07839v1",
            "date": "2019-04-16 17:41:49+00:00",
            "title": "UTFPR at SemEval-2019 Task 5: Hate Speech Identification with Recurrent Neural Networks",
            "authors": [
                "Gustavo Henrique Paetzold",
                "Shervin Malmasi",
                "Marcos Zampieri"
            ],
            "abstract": "In this paper we revisit the problem of automatically identifying hate speech\nin posts from social media. We approach the task using a system based on\nminimalistic compositional Recurrent Neural Networks (RNN). We tested our\napproach on the SemEval-2019 Task 5: Multilingual Detection of Hate Speech\nAgainst Immigrants and Women in Twitter (HatEval) shared task dataset. The\ndataset made available by the HatEval organizers contained English and Spanish\nposts retrieved from Twitter annotated with respect to the presence of hateful\ncontent and its target. In this paper we present the results obtained by our\nsystem in comparison to the other entries in the shared task. Our system\nachieved competitive performance ranking 7th in sub-task A out of 62 systems in\nthe English track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07904v1",
            "date": "2019-04-16 18:13:39+00:00",
            "title": "Mitigating the Impact of Speech Recognition Errors on Spoken Question Answering by Adversarial Domain Adaptation",
            "authors": [
                "Chia-Hsuan Lee",
                "Yun-Nung Chen",
                "Hung-Yi Lee"
            ],
            "abstract": "Spoken question answering (SQA) is challenging due to complex reasoning on\ntop of the spoken documents. The recent studies have also shown the\ncatastrophic impact of automatic speech recognition (ASR) errors on SQA.\nTherefore, this work proposes to mitigate the ASR errors by aligning the\nmismatch between ASR hypotheses and their corresponding reference\ntranscriptions. An adversarial model is applied to this domain adaptation task,\nwhich forces the model to learn domain-invariant features the QA model can\neffectively utilize in order to improve the SQA results. The experiments\nsuccessfully demonstrate the effectiveness of our proposed model, and the\nresults are better than the previous best model by 2% EM score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07953v1",
            "date": "2019-04-16 20:09:41+00:00",
            "title": "Semantic Characteristics of Schizophrenic Speech",
            "authors": [
                "Kfir Bar",
                "Vered Zilberstein",
                "Ido Ziv",
                "Heli Baram",
                "Nachum Dershowitz",
                "Samuel Itzikowitz",
                "Eiran Vadim Harel"
            ],
            "abstract": "Natural language processing tools are used to automatically detect\ndisturbances in transcribed speech of schizophrenia inpatients who speak\nHebrew. We measure topic mutation over time and show that controls maintain\nmore cohesive speech than inpatients. We also examine differences in how\ninpatients and controls use adjectives and adverbs to describe content words\nand show that the ones used by controls are more common than the those of\ninpatients. We provide experimental results and show their potential for\nautomatically detecting schizophrenia in patients by means only of their speech\npatterns.",
            "journal ref": "CLPsych at NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07994v2",
            "date": "2019-04-16 21:50:32+00:00",
            "title": "A Systematic Study of Leveraging Subword Information for Learning Word Representations",
            "authors": [
                "Yi Zhu",
                "Ivan Vuli\u0107",
                "Anna Korhonen"
            ],
            "abstract": "The use of subword-level information (e.g., characters, character n-grams,\nmorphemes) has become ubiquitous in modern word representation learning. Its\nimportance is attested especially for morphologically rich languages which\ngenerate a large number of rare words. Despite a steadily increasing interest\nin such subword-informed word representations, their systematic comparative\nanalysis across typologically diverse languages and different tasks is still\nmissing. In this work, we deliver such a study focusing on the variation of two\ncrucial components required for subword-level integration into word\nrepresentation models: 1) segmentation of words into subword units, and 2)\nsubword composition functions to obtain final word representations. We propose\na general framework for learning subword-informed word representations that\nallows for easy experimentation with different segmentation and composition\ncomponents, also including more advanced techniques based on position\nembeddings and self-attention. Using the unified framework, we run experiments\nover a large number of subword-informed word representation configurations (60\nin total) on 3 tasks (general and rare word similarity, dependency parsing,\nfine-grained entity typing) for 5 languages representing 3 language types. Our\nmain results clearly indicate that there is no \"one-sizefits-all\"\nconfiguration, as performance is both language- and task-dependent. We also\nshow that configurations based on unsupervised segmentation (e.g., BPE,\nMorfessor) are sometimes comparable to or even outperform the ones based on\nsupervised word segmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08075v1",
            "date": "2019-04-17 04:00:52+00:00",
            "title": "End-to-End Speech Translation with Knowledge Distillation",
            "authors": [
                "Yuchen Liu",
                "Hao Xiong",
                "Zhongjun He",
                "Jiajun Zhang",
                "Hua Wu",
                "Haifeng Wang",
                "Chengqing Zong"
            ],
            "abstract": "End-to-end speech translation (ST), which directly translates from source\nlanguage speech into target language text, has attracted intensive attentions\nin recent years. Compared to conventional pipeline systems, end-to-end ST\nmodels have advantages of lower latency, smaller model size and less error\npropagation. However, the combination of speech recognition and text\ntranslation in one model is more difficult than each of these two tasks. In\nthis paper, we propose a knowledge distillation approach to improve ST model by\ntransferring the knowledge from text translation model. Specifically, we first\ntrain a text translation model, regarded as a teacher model, and then ST model\nis trained to learn output probabilities from teacher model through knowledge\ndistillation. Experiments on English- French Augmented LibriSpeech and\nEnglish-Chinese TED corpus show that end-to-end ST is possible to implement on\nboth similar and dissimilar language pairs. In addition, with the instruction\nof teacher model, end-to-end ST model can gain significant improvements by over\n3.5 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08292v1",
            "date": "2019-04-17 14:34:12+00:00",
            "title": "Amobee at SemEval-2019 Tasks 5 and 6: Multiple Choice CNN Over Contextual Embedding",
            "authors": [
                "Alon Rozental",
                "Dadi Biton"
            ],
            "abstract": "This article describes Amobee's participation in \"HatEval: Multilingual\ndetection of hate speech against immigrants and women in Twitter\" (task 5) and\n\"OffensEval: Identifying and Categorizing Offensive Language in Social Media\"\n(task 6). The goal of task 5 was to detect hate speech targeted to women and\nimmigrants. The goal of task 6 was to identify and categorized offensive\nlanguage in social media, and identify offense target. We present a novel type\nof convolutional neural network called \"Multiple Choice CNN\" (MC-CNN) that we\nused over our newly developed contextual embedding, Rozental et al. (2019). For\nboth tasks we used this architecture and achieved 4th place out of 69\nparticipants with an F1 score of 0.53 in task 5, in task 6 achieved 2nd place\n(out of 75) in Sub-task B - automatic categorization of offense types (our\nmodel reached places 18/2/7 out of 103/75/65 for sub-tasks A, B and C\nrespectively in task 6).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08301v1",
            "date": "2019-04-17 14:59:45+00:00",
            "title": "Automatic Accuracy Prediction for AMR Parsing",
            "authors": [
                "Juri Opitz",
                "Anette Frank"
            ],
            "abstract": "Abstract Meaning Representation (AMR) represents sentences as directed,\nacyclic and rooted graphs, aiming at capturing their meaning in a machine\nreadable format. AMR parsing converts natural language sentences into such\ngraphs. However, evaluating a parser on new data by means of comparison to\nmanually created AMR graphs is very costly. Also, we would like to be able to\ndetect parses of questionable quality, or preferring results of alternative\nsystems by selecting the ones for which we can assess good quality. We propose\nAMR accuracy prediction as the task of predicting several metrics of\ncorrectness for an automatically generated AMR parse - in absence of the\ncorresponding gold parse. We develop a neural end-to-end multi-output\nregression model and perform three case studies: firstly, we evaluate the\nmodel's capacity of predicting AMR parse accuracies and test whether it can\nreliably assign high scores to gold parses. Secondly, we perform parse\nselection based on predicted parse accuracies of candidate parses from\nalternative systems, with the aim of improving overall results. Finally, we\npredict system ranks for submissions from two AMR shared tasks on the basis of\ntheir predicted parse accuracy averages. All experiments are carried out across\ntwo different domains and show that our method is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08398v3",
            "date": "2019-04-17 17:55:18+00:00",
            "title": "DocBERT: BERT for Document Classification",
            "authors": [
                "Ashutosh Adhikari",
                "Achyudh Ram",
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "We present, to our knowledge, the first application of BERT to document\nclassification. A few characteristics of the task might lead one to think that\nBERT is not the most appropriate model: syntactic structures matter less for\ncontent categories, documents can often be longer than typical BERT input, and\ndocuments often have multiple labels. Nevertheless, we show that a\nstraightforward classification model using BERT is able to achieve the state of\nthe art across four popular datasets. To address the computational expense\nassociated with BERT inference, we distill knowledge from BERT-large to small\nbidirectional LSTMs, reaching BERT-base parity on multiple datasets using 30x\nfewer parameters. The primary contribution of our paper is improved baselines\nthat can provide the foundation for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08455v3",
            "date": "2019-04-17 19:03:07+00:00",
            "title": "Headline Generation: Learning from Decomposable Document Titles",
            "authors": [
                "Oleg Vasilyev",
                "Tom Grek",
                "John Bohannon"
            ],
            "abstract": "We propose a novel method for generating titles for unstructured text\ndocuments. We reframe the problem as a sequential question-answering task. A\ndeep neural network is trained on document-title pairs with decomposable\ntitles, meaning that the vocabulary of the title is a subset of the vocabulary\nof the document. To train the model we use a corpus of millions of publicly\navailable document-title pairs: news articles and headlines. We present the\nresults of a randomized double-blind trial in which subjects were unaware of\nwhich titles were human or machine-generated. When trained on approximately 1.5\nmillion news articles, the model generates headlines that humans judge to be as\ngood or better than the original human-written headlines in the majority of\ncases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08533v2",
            "date": "2019-04-17 23:19:25+00:00",
            "title": "One Homonym per Translation",
            "authors": [
                "Bradley Hauer",
                "Grzegorz Kondrak"
            ],
            "abstract": "The study of homonymy is vital to resolving fundamental problems in lexical\nsemantics. In this paper, we propose four hypotheses that characterize the\nunique behavior of homonyms in the context of translations, discourses,\ncollocations, and sense clusters. We present a new annotated homonym resource\nthat allows us to test our hypotheses on existing WSD resources. The results of\nthe experiments provide strong empirical evidence for the hypotheses. This\nstudy represents a step towards a computational method for distinguishing\nbetween homonymy and polysemy, and constructing a definitive inventory of\ncoarse-grained senses.",
            "journal ref": "Proceedings of the AAAI Conference on Artificial Intelligence.\n  Vol. 34. 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08535v4",
            "date": "2019-04-17 23:30:17+00:00",
            "title": "Neural Constituency Parsing of Speech Transcripts",
            "authors": [
                "Paria Jamshid Lou",
                "Yufei Wang",
                "Mark Johnson"
            ],
            "abstract": "This paper studies the performance of a neural self-attentive parser on\ntranscribed speech. Speech presents parsing challenges that do not appear in\nwritten text, such as the lack of punctuation and the presence of speech\ndisfluencies (including filled pauses, repetitions, corrections, etc.).\nDisfluencies are especially problematic for conventional syntactic parsers,\nwhich typically fail to find any EDITED disfluency nodes at all. This motivated\nthe development of special disfluency detection systems, and special mechanisms\nadded to parsers specifically to handle disfluencies. However, we show here\nthat neural parsers can find EDITED disfluency nodes, and the best neural\nparsers find them with an accuracy surpassing that of specialized disfluency\ndetection systems, thus making these specialized mechanisms unnecessary. This\npaper also investigates a modified loss function that puts more weight on\nEDITED nodes. It also describes tree-transformations that simplify the\ndisfluency detection task by providing alternative encodings of disfluencies\nand syntactic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08654v2",
            "date": "2019-04-18 09:47:06+00:00",
            "title": "Analytical Methods for Interpretable Ultradense Word Embeddings",
            "authors": [
                "Philipp Dufter",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word embeddings are useful for a wide variety of tasks, but they lack\ninterpretability. By rotating word spaces, interpretable dimensions can be\nidentified while preserving the information contained in the embeddings without\nany loss. In this work, we investigate three methods for making word spaces\ninterpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and\nDensRay, a new method we propose. In contrast to Densifier, DensRay can be\ncomputed in closed form, is hyperparameter-free and thus more robust than\nDensifier. We evaluate the three methods on lexicon induction and set-based\nword analogy. In addition we provide qualitative insights as to how\ninterpretable word spaces can be used for removing gender bias from embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09020v1",
            "date": "2019-04-18 21:33:15+00:00",
            "title": "Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands",
            "authors": [
                "Giovanni Campagna",
                "Silei Xu",
                "Mehrad Moradshahi",
                "Richard Socher",
                "Monica S. Lam"
            ],
            "abstract": "To understand diverse natural language commands, virtual assistants today are\ntrained with numerous labor-intensive, manually annotated sentences. This paper\npresents a methodology and the Genie toolkit that can handle new compound\ncommands with significantly less manual effort. We advocate formalizing the\ncapability of virtual assistants with a Virtual Assistant Programming Language\n(VAPL) and using a neural semantic parser to translate natural language into\nVAPL code. Genie needs only a small realistic set of input sentences for\nvalidating the neural model. Developers write templates to synthesize data;\nGenie uses crowdsourced paraphrases and data augmentation, along with the\nsynthesized data, to train a semantic parser. We also propose design principles\nthat make VAPL languages amenable to natural language translation. We apply\nthese principles to revise ThingTalk, the language used by the Almond virtual\nassistant. We use Genie to build the first semantic parser that can support\ncompound virtual assistants commands with unquoted free-form parameters. Genie\nachieves a 62% accuracy on realistic user inputs. We demonstrate Genie's\ngenerality by showing a 19% and 31% improvement over the previous state of the\nart on a music skill, aggregate functions, and access control.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09051v2",
            "date": "2019-04-19 02:19:43+00:00",
            "title": "Query-focused Sentence Compression in Linear Time",
            "authors": [
                "Abram Handler",
                "Brendan O'Connor"
            ],
            "abstract": "Search applications often display shortened sentences which must contain\ncertain query terms and must fit within the space constraints of a user\ninterface. This work introduces a new transition-based sentence compression\ntechnique developed for such settings. Our query-focused method constructs\nlength and lexically constrained compressions in linear time, by growing a\nsubgraph in the dependency parse of a sentence. This theoretically efficient\napproach achieves an 11X empirical speedup over baseline ILP methods, while\nbetter reconstructing gold constrained shortenings. Such speedups help\nquery-focused applications, because users are measurably hindered by interface\nlags. Additionally, our technique does not require an ILP solver or a GPU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09072v1",
            "date": "2019-04-19 04:26:36+00:00",
            "title": "Identifying Offensive Posts and Targeted Offense from Twitter",
            "authors": [
                "Haimin Zhang",
                "Debanjan Mahata",
                "Simra Shahid",
                "Laiba Mehnaz",
                "Sarthak Anand",
                "Yaman Singla",
                "Rajiv Ratn Shah",
                "Karan Uppal"
            ],
            "abstract": "In this paper we present our approach and the system description for Sub-task\nA and Sub Task B of SemEval 2019 Task 6: Identifying and Categorizing Offensive\nLanguage in Social Media. Sub-task A involves identifying if a given tweet is\noffensive or not, and Sub Task B involves detecting if an offensive tweet is\ntargeted towards someone (group or an individual). Our models for Sub-task A is\nbased on an ensemble of Convolutional Neural Network, Bidirectional LSTM with\nattention, and Bidirectional LSTM + Bidirectional GRU, whereas for Sub-task B,\nwe rely on a set of heuristics derived from the training data and manual\nobservation. We provide detailed analysis of the results obtained using the\ntrained models. Our team ranked 5th out of 103 participants in Sub-task A,\nachieving a macro F1 score of 0.807, and ranked 8th out of 75 participants in\nSub Task B achieving a macro F1 of 0.695.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09076v1",
            "date": "2019-04-19 04:38:32+00:00",
            "title": "Suggestion Mining from Online Reviews using ULMFiT",
            "authors": [
                "Sarthak Anand",
                "Debanjan Mahata",
                "Kartik Aggarwal",
                "Laiba Mehnaz",
                "Simra Shahid",
                "Haimin Zhang",
                "Yaman Kumar",
                "Rajiv Ratn Shah",
                "Karan Uppal"
            ],
            "abstract": "In this paper we present our approach and the system description for Sub Task\nA of SemEval 2019 Task 9: Suggestion Mining from Online Reviews and Forums.\nGiven a sentence, the task asks to predict whether the sentence consists of a\nsuggestion or not. Our model is based on Universal Language Model Fine-tuning\nfor Text Classification. We apply various pre-processing techniques before\ntraining the language and the classification model. We further provide detailed\nanalysis of the results obtained using the trained model. Our team ranked 10th\nout of 34 participants, achieving an F1 score of 0.7011. We publicly share our\nimplementation at https://github.com/isarth/SemEval9_MIDAS",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09077v2",
            "date": "2019-04-19 04:45:44+00:00",
            "title": "Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT",
            "authors": [
                "Shijie Wu",
                "Mark Dredze"
            ],
            "abstract": "Pretrained contextual representation models (Peters et al., 2018; Devlin et\nal., 2018) have pushed forward the state-of-the-art on many NLP tasks. A new\nrelease of BERT (Devlin, 2018) includes a model simultaneously pretrained on\n104 languages with impressive performance for zero-shot cross-lingual transfer\non a natural language inference task. This paper explores the broader\ncross-lingual potential of mBERT (multilingual) as a zero shot language\ntransfer model on 5 NLP tasks covering a total of 39 languages from various\nlanguage families: NLI, document classification, NER, POS tagging, and\ndependency parsing. We compare mBERT with the best-published methods for\nzero-shot cross-lingual transfer and find mBERT competitive on each task.\nAdditionally, we investigate the most effective strategy for utilizing mBERT in\nthis manner, determine to what extent mBERT generalizes away from language\nspecific features, and measure factors that influence cross-lingual transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09086v2",
            "date": "2019-04-19 05:56:45+00:00",
            "title": "Learning Programmatic Idioms for Scalable Semantic Parsing",
            "authors": [
                "Srinivasan Iyer",
                "Alvin Cheung",
                "Luke Zettlemoyer"
            ],
            "abstract": "Programmers typically organize executable source code using high-level coding\npatterns or idiomatic structures such as nested loops, exception handlers and\nrecursive blocks, rather than as individual code tokens. In contrast, state of\nthe art (SOTA) semantic parsers still map natural language instructions to\nsource code by building the code syntax tree one node at a time. In this paper,\nwe introduce an iterative method to extract code idioms from large source code\ncorpora by repeatedly collapsing most-frequent depth-2 subtrees of their syntax\ntrees, and train semantic parsers to apply these idioms during decoding.\nApplying idiom-based decoding on a recent context-dependent semantic parsing\ntask improves the SOTA by 2.2\\% BLEU score while reducing training time by more\nthan 50\\%. This improved speed enables us to scale up the model by training on\nan extended training set that is 5$\\times$ larger, to further move up the SOTA\nby an additional 2.3\\% BLEU and 0.9\\% exact match. Finally, idioms also\nsignificantly improve accuracy of semantic parsing to SQL on the ATIS-SQL\ndataset, when training data is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09107v4",
            "date": "2019-04-19 07:53:06+00:00",
            "title": "Code-Switching for Enhancing NMT with Pre-Specified Translation",
            "authors": [
                "Kai Song",
                "Yue Zhang",
                "Heng Yu",
                "Weihua Luo",
                "Kun Wang",
                "Min Zhang"
            ],
            "abstract": "Leveraging user-provided translation to constrain NMT has practical\nsignificance. Existing methods can be classified into two main categories,\nnamely the use of placeholder tags for lexicon words and the use of hard\nconstraints during decoding. Both methods can hurt translation fidelity for\nvarious reasons. We investigate a data augmentation method, making\ncode-switched training data by replacing source phrases with their target\ntranslations. Our method does not change the MNT model or decoding algorithm,\nallowing the model to learn lexicon translations by copying source-side target\nwords. Extensive experiments show that our method achieves consistent\nimprovements over existing approaches, improving translation of constrained\nwords without hurting unconstrained words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09108v1",
            "date": "2019-04-19 07:59:25+00:00",
            "title": "Recognizing the vocabulary of Brazilian popular newspapers with a free-access computational dictionary",
            "authors": [
                "Maria Jos\u00e9 Finatto",
                "Oto Vale",
                "Eric Laporte"
            ],
            "abstract": "We report an experiment to check the identification of a set of words in\npopular written Portuguese with two versions of a computational dictionary of\nBrazilian Portuguese, DELAF PB 2004 and DELAF PB 2015. This dictionary is\nfreely available for use in linguistic analyses of Brazilian Portuguese and\nother researches, which justifies critical study. The vocabulary comes from the\nPorPopular corpus, made of popular newspapers Di{\\'a}rio Ga{\\'u}cho (DG) and\nMassa! (MA). From DG, we retained a set of texts with 984.465 words (tokens),\npublished in 2008, with the spelling used before the Portuguese Language\nOrthographic Agreement adopted in 2009. From MA, we examined papers of 2012,\n2014 e 2015, with 215.776 words (tokens), all with the new spelling. The\nchecking involved: a) generating lists of words (types) occurring in DG and MA;\nb) comparing them with the entry lists of both versions of DELAF PB; c)\nassessing the coverage of this vocabulary; d) proposing ways of incorporating\nthe items not covered. The results of the work show that an average of 19% of\nthe types in DG were not found in DELAF PB 2004 or 2015. In MA, this average is\n13%. Switching versions of the dictionary affected slightly the performance in\nrecognizing the words.",
            "journal ref": "Alfa Revista de Linguistica, 2019, 63 (1), pp.67-85",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09223v1",
            "date": "2019-04-19 15:10:56+00:00",
            "title": "ERNIE: Enhanced Representation through Knowledge Integration",
            "authors": [
                "Yu Sun",
                "Shuohuan Wang",
                "Yukun Li",
                "Shikun Feng",
                "Xuyi Chen",
                "Han Zhang",
                "Xin Tian",
                "Danxiang Zhu",
                "Hao Tian",
                "Hua Wu"
            ],
            "abstract": "We present a novel language representation model enhanced by knowledge called\nERNIE (Enhanced Representation through kNowledge IntEgration). Inspired by the\nmasking strategy of BERT, ERNIE is designed to learn language representation\nenhanced by knowledge masking strategies, which includes entity-level masking\nand phrase-level masking. Entity-level strategy masks entities which are\nusually composed of multiple words.Phrase-level strategy masks the whole phrase\nwhich is composed of several words standing together as a conceptual\nunit.Experimental results show that ERNIE outperforms other baseline methods,\nachieving new state-of-the-art results on five Chinese natural language\nprocessing tasks including natural language inference, semantic similarity,\nnamed entity recognition, sentiment analysis and question answering. We also\ndemonstrate that ERNIE has more powerful knowledge inference capacity on a\ncloze test.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09286v2",
            "date": "2019-04-19 17:58:29+00:00",
            "title": "Unifying Question Answering, Text Classification, and Regression via Span Extraction",
            "authors": [
                "Nitish Shirish Keskar",
                "Bryan McCann",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Even as pre-trained language encoders such as BERT are shared across many\ntasks, the output layers of question answering, text classification, and\nregression models are significantly different. Span decoders are frequently\nused for question answering, fixed-class, classification layers for text\nclassification, and similarity-scoring layers for regression tasks, We show\nthat this distinction is not necessary and that all three can be unified as\nspan extraction. A unified, span-extraction approach leads to superior or\ncomparable performance in supplementary supervised pre-trained, low-data, and\nmulti-task learning experiments on several question answering, text\nclassification, and regression benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09407v1",
            "date": "2019-04-20 06:21:52+00:00",
            "title": "Self-imitating Feedback Generation Using GAN for Computer-Assisted Pronunciation Training",
            "authors": [
                "Seung Hee Yang",
                "Minhwa Chung"
            ],
            "abstract": "Self-imitating feedback is an effective and learner-friendly method for\nnon-native learners in Computer-Assisted Pronunciation Training. Acoustic\ncharacteristics in native utterances are extracted and transplanted onto\nlearner's own speech input, and given back to the learner as a corrective\nfeedback. Previous works focused on speech conversion using prosodic\ntransplantation techniques based on PSOLA algorithm. Motivated by the visual\ndifferences found in spectrograms of native and non-native speeches, we\ninvestigated applying GAN to generate self-imitating feedback by utilizing\ngenerator's ability through adversarial training. Because this mapping is\nhighly under-constrained, we also adopt cycle consistency loss to encourage the\noutput to preserve the global structure, which is shared by native and\nnon-native utterances. Trained on 97,200 spectrogram images of short utterances\nproduced by native and non-native speakers of Korean, the generator is able to\nsuccessfully transform the non-native spectrogram input to a spectrogram with\nproperties of self-imitating feedback. Furthermore, the transformed spectrogram\nshows segmental corrections that cannot be obtained by prosodic\ntransplantation. Perceptual test comparing the self-imitating and correcting\nabilities of our method with the baseline PSOLA method shows that the\ngenerative approach with cycle consistency loss is promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09442v1",
            "date": "2019-04-20 12:56:39+00:00",
            "title": "Personalized sentence generation using generative adversarial networks with author-specific word usage",
            "authors": [
                "Chenhan Yuan",
                "Yi-Chin Huang"
            ],
            "abstract": "The author-specific word usage is a vital feature to let readers perceive the\nwriting style of the author. In this work, a personalized sentence generation\nmethod based on generative adversarial networks (GANs) is proposed to cope with\nthis issue. The frequently used function word and content word are incorporated\nnot only as the input features but also as the sentence structure constraint\nfor the GAN training. For the sentence generation with the related topics\ndecided by the user, the Named Entity Recognition (NER) information of the\ninput words is also used in the network training. We compared the proposed\nmethod with the GAN-based sentence generation methods, and the experimental\nresults showed that the generated sentences using our method are more similar\nto the original sentences of the same author based on the objective evaluation\nsuch as BLEU and SimHash score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09446v1",
            "date": "2019-04-20 13:19:24+00:00",
            "title": "Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings",
            "authors": [
                "Haozhou Wang",
                "James Henderson",
                "Paola Merlo"
            ],
            "abstract": "Distributed representations of words which map each word to a continuous\nvector have proven useful in capturing important linguistic information not\nonly in a single language but also across different languages. Current\nunsupervised adversarial approaches show that it is possible to build a mapping\nmatrix that align two sets of monolingual word embeddings together without high\nquality parallel data such as a dictionary or a sentence-aligned corpus.\nHowever, without post refinement, the performance of these methods' preliminary\nmapping is not good, leading to poor performance for typologically distant\nlanguages.\n  In this paper, we propose a weakly-supervised adversarial training method to\novercome this limitation, based on the intuition that mapping across languages\nis better done at the concept level than at the word level. We propose a\nconcept-based adversarial training method which for most languages improves the\nperformance of previous unsupervised adversarial methods, especially for\ntypologically distant language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09482v1",
            "date": "2019-04-20 19:11:00+00:00",
            "title": "Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding",
            "authors": [
                "Xiaodong Liu",
                "Pengcheng He",
                "Weizhu Chen",
                "Jianfeng Gao"
            ],
            "abstract": "This paper explores the use of knowledge distillation to improve a Multi-Task\nDeep Neural Network (MT-DNN) (Liu et al., 2019) for learning text\nrepresentations across multiple natural language understanding tasks. Although\nensemble learning can improve model performance, serving an ensemble of large\nDNNs such as MT-DNN can be prohibitively expensive. Here we apply the knowledge\ndistillation method (Hinton et al., 2015) in the multi-task learning setting.\nFor each task, we train an ensemble of different MT-DNNs (teacher) that\noutperforms any single model, and then train a single MT-DNN (student) via\nmulti-task learning to \\emph{distill} knowledge from these ensemble teachers.\nWe show that the distilled MT-DNN significantly outperforms the original MT-DNN\non 7 out of 9 GLUE tasks, pushing the GLUE benchmark (single model) to 83.7\\%\n(1.5\\% absolute improvement\\footnote{ Based on the GLUE leaderboard at\nhttps://gluebenchmark.com/leaderboard as of April 1, 2019.}). The code and\npre-trained models will be made publicly available at\nhttps://github.com/namisan/mt-dnn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09521v3",
            "date": "2019-04-21 00:42:22+00:00",
            "title": "Few-Shot NLG with Pre-Trained Language Model",
            "authors": [
                "Zhiyu Chen",
                "Harini Eavani",
                "Wenhu Chen",
                "Yinyin Liu",
                "William Yang Wang"
            ],
            "abstract": "Neural-based end-to-end approaches to natural language generation (NLG) from\nstructured data or knowledge are data-hungry, making their adoption for\nreal-world applications difficult with limited data. In this work, we propose\nthe new task of \\textit{few-shot natural language generation}. Motivated by how\nhumans tend to summarize tabular data, we propose a simple yet effective\napproach and show that it not only demonstrates strong performance but also\nprovides good generalization across domains. The design of the model\narchitecture is based on two aspects: content selection from input data and\nlanguage modeling to compose coherent sentences, which can be acquired from\nprior knowledge. With just 200 training examples, across multiple domains, we\nshow that our approach achieves very reasonable performances and outperforms\nthe strongest baseline by an average of over 8.0 BLEU points improvement. Our\ncode and data can be found at \\url{https://github.com/czyssrs/Few-Shot-NLG}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09535v3",
            "date": "2019-04-21 03:11:24+00:00",
            "title": "NeuronBlocks: Building Your NLP DNN Models Like Playing Lego",
            "authors": [
                "Ming Gong",
                "Linjun Shou",
                "Wutao Lin",
                "Zhijie Sang",
                "Quanjia Yan",
                "Ze Yang",
                "Feixiang Cheng",
                "Daxin Jiang"
            ],
            "abstract": "Deep Neural Networks (DNN) have been widely employed in industry to address\nvarious Natural Language Processing (NLP) tasks. However, many engineers find\nit a big overhead when they have to choose from multiple frameworks, compare\ndifferent types of models, and understand various optimization mechanisms. An\nNLP toolkit for DNN models with both generality and flexibility can greatly\nimprove the productivity of engineers by saving their learning cost and guiding\nthem to find optimal solutions to their tasks. In this paper, we introduce\nNeuronBlocks\\footnote{Code: \\url{https://github.com/Microsoft/NeuronBlocks}}\n\\footnote{Demo: \\url{https://youtu.be/x6cOpVSZcdo}}, a toolkit encapsulating a\nsuite of neural network modules as building blocks to construct various DNN\nmodels with complex architecture. This toolkit empowers engineers to build,\ntrain, and test various NLP models through simple configuration of JSON files.\nThe experiments on several NLP datasets such as GLUE, WikiQA and CoNLL-2003\ndemonstrate the effectiveness of NeuronBlocks.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09540v1",
            "date": "2019-04-21 04:13:55+00:00",
            "title": "Fact Discovery from Knowledge Base via Facet Decomposition",
            "authors": [
                "Zihao Fu",
                "Yankai Lin",
                "Zhiyuan Liu",
                "Wai Lam"
            ],
            "abstract": "During the past few decades, knowledge bases (KBs) have experienced rapid\ngrowth. Nevertheless, most KBs still suffer from serious incompletion.\nResearchers proposed many tasks such as knowledge base completion and relation\nprediction to help build the representation of KBs. However, there are some\nissues unsettled towards enriching the KBs. Knowledge base completion and\nrelation prediction assume that we know two elements of the fact triples and we\nare going to predict the missing one. This assumption is too restricted in\npractice and prevents it from discovering new facts directly. To address this\nissue, we propose a new task, namely, fact discovery from knowledge base. This\ntask only requires that we know the head entity and the goal is to discover\nfacts associated with the head entity. To tackle this new problem, we propose a\nnovel framework that decomposes the discovery problem into several facet\ndiscovery components. We also propose a novel auto-encoder based facet\ncomponent to estimate some facets of the fact. Besides, we propose a feedback\nlearning component to share the information between each facet. We evaluate our\nframework using a benchmark dataset and the experimental results show that our\nframework achieves promising results. We also conduct extensive analysis of our\nframework in discovering different kinds of facts. The source code of this\npaper can be obtained from https://github.com/thunlp/FFD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09545v4",
            "date": "2019-04-21 05:54:30+00:00",
            "title": "Good-Enough Compositional Data Augmentation",
            "authors": [
                "Jacob Andreas"
            ],
            "abstract": "We propose a simple data augmentation protocol aimed at providing a\ncompositional inductive bias in conditional and unconditional sequence models.\nUnder this protocol, synthetic training examples are constructed by taking real\ntraining examples and replacing (possibly discontinuous) fragments with other\nfragments that appear in at least one similar environment. The protocol is\nmodel-agnostic and useful for a variety of tasks. Applied to neural\nsequence-to-sequence models, it reduces error rate by as much as 87% on\ndiagnostic tasks from the SCAN dataset and 16% on a semantic parsing task.\nApplied to n-gram language models, it reduces perplexity by roughly 1% on small\ncorpora in several languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09552v2",
            "date": "2019-04-21 07:07:48+00:00",
            "title": "Understanding the Stability of Medical Concept Embeddings",
            "authors": [
                "Grace E. Lee",
                "Aixin Sun"
            ],
            "abstract": "Frequency is one of the major factors for training quality word embeddings.\nSeveral work has recently discussed the stability of word embeddings in general\ndomain and suggested factors influencing the stability. In this work, we\nconduct a detailed analysis on the stability of concept embeddings in medical\ndomain, particularly the relation with concept frequency. The analysis reveals\nthe surprising high stability of low-frequency concepts: low-frequency (<100)\nconcepts have the same high stability as high-frequency (>1000) concepts. To\ndevelop a deeper understanding of this finding, we propose a new factor, the\nnoisiness of context words, which influences the stability of medical concept\nembeddings, regardless of frequency. We evaluate the proposed factor by showing\nthe linear correlation with the stability of medical concept embeddings. The\ncorrelations are clear and consistent with various groups of medical concepts.\nBased on the linear relations, we make suggestions on ways to adjust the\nnoisiness of context words for the improvement of stability. Finally, we\ndemonstrate that the proposed factor extends to the word embedding stability in\ngeneral domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09636v1",
            "date": "2019-04-21 17:46:24+00:00",
            "title": "Model Compression with Multi-Task Knowledge Distillation for Web-scale Question Answering System",
            "authors": [
                "Ze Yang",
                "Linjun Shou",
                "Ming Gong",
                "Wutao Lin",
                "Daxin Jiang"
            ],
            "abstract": "Deep pre-training and fine-tuning models (like BERT, OpenAI GPT) have\ndemonstrated excellent results in question answering areas. However, due to the\nsheer amount of model parameters, the inference speed of these models is very\nslow. How to apply these complex models to real business scenarios becomes a\nchallenging but practical problem. Previous works often leverage model\ncompression approaches to resolve this problem. However, these methods usually\ninduce information loss during the model compression procedure, leading to\nincomparable results between compressed model and the original model. To tackle\nthis challenge, we propose a Multi-task Knowledge Distillation Model (MKDM for\nshort) for web-scale Question Answering system, by distilling knowledge from\nmultiple teacher models to a light-weight student model. In this way, more\ngeneralized knowledge can be transferred. The experiment results show that our\nmethod can significantly outperform the baseline methods and even achieve\ncomparable results with the original teacher models, along with significant\nspeedup of model inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09646v2",
            "date": "2019-04-21 19:07:07+00:00",
            "title": "Dynamic Past and Future for Neural Machine Translation",
            "authors": [
                "Zaixiang Zheng",
                "Shujian Huang",
                "Zhaopeng Tu",
                "Xin-Yu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Previous studies have shown that neural machine translation (NMT) models can\nbenefit from explicitly modeling translated (Past) and untranslated (Future) to\ngroups of translated and untranslated contents through parts-to-wholes\nassignment. The assignment is learned through a novel variant of\nrouting-by-agreement mechanism (Sabour et al., 2017), namely {\\em Guided\nDynamic Routing}, where the translating status at each decoding step {\\em\nguides} the routing process to assign each source word to its associated group\n(i.e., translated or untranslated content) represented by a capsule, enabling\ntranslation to be made from holistic context. Experiments show that our\napproach achieves substantial improvements over both RNMT and Transformer by\nproducing more adequate translations. Extensive analysis demonstrates that our\nmethod is highly interpretable, which is able to recognize the translated and\nuntranslated contents as expected.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09675v3",
            "date": "2019-04-21 23:08:53+00:00",
            "title": "BERTScore: Evaluating Text Generation with BERT",
            "authors": [
                "Tianyi Zhang",
                "Varsha Kishore",
                "Felix Wu",
                "Kilian Q. Weinberger",
                "Yoav Artzi"
            ],
            "abstract": "We propose BERTScore, an automatic evaluation metric for text generation.\nAnalogously to common metrics, BERTScore computes a similarity score for each\ntoken in the candidate sentence with each token in the reference sentence.\nHowever, instead of exact matches, we compute token similarity using contextual\nembeddings. We evaluate using the outputs of 363 machine translation and image\ncaptioning systems. BERTScore correlates better with human judgments and\nprovides stronger model selection performance than existing metrics. Finally,\nwe use an adversarial paraphrase detection task to show that BERTScore is more\nrobust to challenging examples when compared to existing metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09678v2",
            "date": "2019-04-21 23:37:30+00:00",
            "title": "UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages",
            "authors": [
                "Ehsaneddin Asgari",
                "Fabienne Braune",
                "Benjamin Roth",
                "Christoph Ringlstetter",
                "Mohammad R. K. Mofrad"
            ],
            "abstract": "In this paper, we introduce UniSent universal sentiment lexica for $1000+$\nlanguages. Sentiment lexica are vital for sentiment analysis in absence of\ndocument-level annotations, a very common scenario for low-resource languages.\nTo the best of our knowledge, UniSent is the largest sentiment resource to date\nin terms of the number of covered languages, including many low resource ones.\nIn this work, we use a massively parallel Bible corpus to project sentiment\ninformation from English to other languages for sentiment analysis on Twitter\ndata. We introduce a method called DomDrift to mitigate the huge domain\nmismatch between Bible and Twitter by a confidence weighting scheme that uses\ndomain-specific embeddings to compare the nearest neighbors for a candidate\nsentiment word in the source (Bible) and target (Twitter) domain. We evaluate\nthe quality of UniSent in a subset of languages for which manually created\nground truth was available, Macedonian, Czech, German, Spanish, and French. We\nshow that the quality of UniSent is comparable to manually created sentiment\nresources when it is used as the sentiment seed for the task of word sentiment\nprediction on top of embedding representations. In addition, we show that\nemoticon sentiments could be reliably predicted in the Twitter domain using\nonly UniSent and monolingual embeddings in German, Spanish, French, and\nItalian. With the publication of this paper, we release the UniSent sentiment\nlexica.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09679v3",
            "date": "2019-04-21 23:49:02+00:00",
            "title": "Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension",
            "authors": [
                "Kai Sun",
                "Dian Yu",
                "Dong Yu",
                "Claire Cardie"
            ],
            "abstract": "Machine reading comprehension tasks require a machine reader to answer\nquestions relevant to the given document. In this paper, we present the first\nfree-form multiple-Choice Chinese machine reading Comprehension dataset (C^3),\ncontaining 13,369 documents (dialogues or more formally written mixed-genre\ntexts) and their associated 19,577 multiple-choice free-form questions\ncollected from Chinese-as-a-second-language examinations.\n  We present a comprehensive analysis of the prior knowledge (i.e., linguistic,\ndomain-specific, and general world knowledge) needed for these real-world\nproblems. We implement rule-based and popular neural methods and find that\nthere is still a significant performance gap between the best performing model\n(68.5%) and human readers (96.0%), especially on problems that require prior\nknowledge. We further study the effects of distractor plausibility and data\naugmentation based on translated relevant datasets for English on model\nperformance. We expect C^3 to present great challenges to existing systems as\nanswering 86.8% of questions requires both knowledge within and beyond the\naccompanying document, and we hope that C^3 can serve as a platform to study\nhow to leverage various kinds of prior knowledge to better understand a given\nwritten or orally oriented text. C^3 is available at https://dataset.org/c3/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09688v4",
            "date": "2019-04-22 00:55:37+00:00",
            "title": "Fine-Grained Argument Unit Recognition and Classification",
            "authors": [
                "Dietrich Trautmann",
                "Johannes Daxenberger",
                "Christian Stab",
                "Hinrich Sch\u00fctze",
                "Iryna Gurevych"
            ],
            "abstract": "Prior work has commonly defined argument retrieval from heterogeneous\ndocument collections as a sentence-level classification task. Consequently,\nargument retrieval suffers both from low recall and from sentence segmentation\nerrors making it difficult for humans and machines to consume the arguments. In\nthis work, we argue that the task should be performed on a more fine-grained\nlevel of sequence labeling. For this, we define the task as Argument Unit\nRecognition and Classification (AURC). We present a dataset of arguments from\nheterogeneous sources annotated as spans of tokens within a sentence, as well\nas with a corresponding stance. We show that and how such difficult argument\nannotations can be effectively collected through crowdsourcing with high\ninterannotator agreement. The new benchmark, AURC-8, contains up to 15% more\narguments per topic as compared to annotations on the sentence level. We\nidentify a number of methods targeted at AURC sequence labeling, achieving\nclose to human performance on known domains. Further analysis also reveals\nthat, contrary to previous approaches, our methods are more robust against\nsentence segmentation errors. We publicly release our code and the AURC-8\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09720v1",
            "date": "2019-04-22 05:06:48+00:00",
            "title": "Understanding Roles and Entities: Datasets and Models for Natural Language Inference",
            "authors": [
                "Arindam Mitra",
                "Ishan Shrivastava",
                "Chitta Baral"
            ],
            "abstract": "We present two new datasets and a novel attention mechanism for Natural\nLanguage Inference (NLI). Existing neural NLI models, even though when trained\non existing large datasets, do not capture the notion of entity and role well\nand often end up making mistakes such as \"Peter signed a deal\" can be inferred\nfrom \"John signed a deal\". The two datasets have been developed to mitigate\nsuch issues and make the systems better at understanding the notion of\n\"entities\" and \"roles\". After training the existing architectures on the new\ndataset we observe that the existing architectures does not perform well on one\nof the new benchmark. We then propose a modification to the \"word-to-word\"\nattention function which has been uniformly reused across several popular NLI\narchitectures. The resulting architectures perform as well as their unmodified\ncounterparts on the existing benchmarks and perform significantly well on the\nnew benchmark for \"roles\" and \"entities\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09728v3",
            "date": "2019-04-22 05:36:37+00:00",
            "title": "SocialIQA: Commonsense Reasoning about Social Interactions",
            "authors": [
                "Maarten Sap",
                "Hannah Rashkin",
                "Derek Chen",
                "Ronan LeBras",
                "Yejin Choi"
            ],
            "abstract": "We introduce Social IQa, the first largescale benchmark for commonsense\nreasoning about social situations. Social IQa contains 38,000 multiple choice\nquestions for probing emotional and social intelligence in a variety of\neveryday situations (e.g., Q: \"Jordan wanted to tell Tracy a secret, so Jordan\nleaned towards Tracy. Why did Jordan do this?\" A: \"Make sure no one else could\nhear\"). Through crowdsourcing, we collect commonsense questions along with\ncorrect and incorrect answers about social interactions, using a new framework\nthat mitigates stylistic artifacts in incorrect answers by asking workers to\nprovide the right answer to a different but related question. Empirical results\nshow that our benchmark is challenging for existing question-answering models\nbased on pretrained language models, compared to human performance (>20% gap).\nNotably, we further establish Social IQa as a resource for transfer learning of\ncommonsense knowledge, achieving state-of-the-art performance on multiple\ncommonsense reasoning tasks (Winograd Schemas, COPA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09745v2",
            "date": "2019-04-22 06:57:43+00:00",
            "title": "Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference",
            "authors": [
                "Nikita Kitaev",
                "Dan Klein"
            ],
            "abstract": "We present a constituency parsing algorithm that, like a supertagger, works\nby assigning labels to each word in a sentence. In order to maximally leverage\ncurrent neural architectures, the model scores each word's tags in parallel,\nwith minimal task-specific structure. After scoring, a left-to-right\nreconciliation phase extracts a tree in (empirically) linear time. Our parser\nachieves 95.4 F1 on the WSJ test set while also achieving substantial speedups\ncompared to current state-of-the-art parsers with comparable accuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09751v2",
            "date": "2019-04-22 07:17:18+00:00",
            "title": "The Curious Case of Neural Text Degeneration",
            "authors": [
                "Ari Holtzman",
                "Jan Buys",
                "Li Du",
                "Maxwell Forbes",
                "Yejin Choi"
            ],
            "abstract": "Despite considerable advancements with deep neural language models, the\nenigma of neural text degeneration persists when these models are tested as\ntext generators. The counter-intuitive empirical observation is that even\nthough the use of likelihood as training objective leads to high quality models\nfor a broad range of language understanding tasks, using likelihood as a\ndecoding objective leads to text that is bland and strangely repetitive.\n  In this paper, we reveal surprising distributional differences between human\ntext and machine text. In addition, we find that decoding strategies alone can\ndramatically effect the quality of machine text, even when generated from\nexactly the same neural language model. Our findings motivate Nucleus Sampling,\na simple but effective method to draw the best out of neural generation. By\nsampling text from the dynamic nucleus of the probability distribution, which\nallows for diversity while effectively truncating the less reliable tail of the\ndistribution, the resulting text better demonstrates the quality of human text,\nyielding enhanced diversity without sacrificing fluency and coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09824v1",
            "date": "2019-04-22 12:35:38+00:00",
            "title": "Judging Chemical Reaction Practicality From Positive Sample only Learning",
            "authors": [
                "Shu Jiang",
                "Zhuosheng Zhang",
                "Hai Zhao",
                "Jiangtong Li",
                "Yang Yang",
                "Bao-Liang Lu",
                "Ning Xia"
            ],
            "abstract": "Chemical reaction practicality is the core task among all symbol intelligence\nbased chemical information processing, for example, it provides indispensable\nclue for further automatic synthesis route inference. Considering that chemical\nreactions have been represented in a language form, we propose a new solution\nto generally judge the practicality of organic reaction without considering\ncomplex quantum physical modeling or chemistry knowledge. While tackling the\npracticality judgment as a machine learning task from positive and negative\n(chemical reaction) samples, all existing studies have to carefully handle the\nserious insufficiency issue on the negative samples. We propose an\nauto-construction method to well solve the extensively existed long-term\ndifficulty. Experimental results show our model can effectively predict the\npracticality of chemical reactions, which achieves a high accuracy of 99.76\\%\non real large-scale chemical lab reaction practicality judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10162v1",
            "date": "2019-04-23 05:58:54+00:00",
            "title": "Multi-Task Learning for Argumentation Mining",
            "authors": [
                "Tobias Kahse"
            ],
            "abstract": "Multi-task learning has recently become a very active field in deep learning\nresearch. In contrast to learning a single task in isolation, multiple tasks\nare learned at the same time, thereby utilizing the training signal of related\ntasks to improve the performance on the respective machine learning tasks.\nRelated work shows various successes in different domains when applying this\nparadigm and this thesis extends the existing empirical results by evaluating\nmulti-task learning in four different scenarios: argumentation mining,\nepistemic segmentation, argumentation component segmentation, and\ngrapheme-to-phoneme conversion. We show that multi-task learning can, indeed,\nimprove the performance compared to single-task learning in all these\nscenarios, but may also hurt the performance. Therefore, we investigate the\nreasons for successful and less successful applications of this paradigm and\nfind that dataset properties such as entropy or the size of the label inventory\nare good indicators for a potential multi-task learning success and that\nmulti-task learning is particularly useful if the task at hand suffers from\ndata sparsity, i.e. a lack of training data. Moreover, multi-task learning is\nparticularly effective for long input sequences in our experiments. We have\nobserved this trend in all evaluated scenarios. Finally, we develop a highly\nconfigurable and extensible sequence tagging framework which supports\nmulti-task learning to conduct our empirical experiments and to aid future\nresearch regarding the multi-task learning paradigm and natural language\nprocessing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10195v1",
            "date": "2019-04-23 08:28:11+00:00",
            "title": "Empirical Evaluation of Leveraging Named Entities for Arabic Sentiment Analysis",
            "authors": [
                "Hala Mulki",
                "Hatem Haddad",
                "Mourad Gridach",
                "Ismail Babaoglu"
            ],
            "abstract": "Social media reflects the public attitudes towards specific events. Events\nare often related to persons, locations or organizations, the so-called Named\nEntities. This can define Named Entities as sentiment-bearing components. In\nthis paper, we dive beyond Named Entities recognition to the exploitation of\nsentiment-annotated Named Entities in Arabic sentiment analysis. Therefore, we\ndevelop an algorithm to detect the sentiment of Named Entities based on the\nmajority of attitudes towards them. This enabled tagging Named Entities with\nproper tags and, thus, including them in a sentiment analysis framework of two\nmodels: supervised and lexicon-based. Both models were applied on datasets of\nmulti-dialectal content. The results revealed that Named Entities have no\nconsiderable impact on the supervised model, while employing them in the\nlexicon-based model improved the classification performance and outperformed\nmost of the baseline systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10419v2",
            "date": "2019-04-23 16:55:04+00:00",
            "title": "GumDrop at the DISRPT2019 Shared Task: A Model Stacking Approach to Discourse Unit Segmentation and Connective Detection",
            "authors": [
                "Yue Yu",
                "Yilun Zhu",
                "Yang Liu",
                "Yan Liu",
                "Siyao Peng",
                "Mackenzie Gong",
                "Amir Zeldes"
            ],
            "abstract": "In this paper we present GumDrop, Georgetown University's entry at the DISRPT\n2019 Shared Task on automatic discourse unit segmentation and connective\ndetection. Our approach relies on model stacking, creating a heterogeneous\nensemble of classifiers, which feed into a metalearner for each final task. The\nsystem encompasses three trainable component stacks: one for sentence\nsplitting, one for discourse unit segmentation and one for connective\ndetection. The flexibility of each ensemble allows the system to generalize\nwell to datasets of different sizes and with varying levels of homogeneity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10622v2",
            "date": "2019-04-24 03:09:47+00:00",
            "title": "Objective Assessment of Social Skills Using Automated Language Analysis for Identification of Schizophrenia and Bipolar Disorder",
            "authors": [
                "Rohit Voleti",
                "Stephanie Woolridge",
                "Julie M. Liss",
                "Melissa Milanovic",
                "Christopher R. Bowie",
                "Visar Berisha"
            ],
            "abstract": "Several studies have shown that speech and language features, automatically\nextracted from clinical interviews or spontaneous discourse, have diagnostic\nvalue for mental disorders such as schizophrenia and bipolar disorder. They\ntypically make use of a large feature set to train a classifier for\ndistinguishing between two groups of interest, i.e. a clinical and control\ngroup. However, a purely data-driven approach runs the risk of overfitting to a\nparticular data set, especially when sample sizes are limited. Here, we first\ndown-select the set of language features to a small subset that is related to a\nwell-validated test of functional ability, the Social Skills Performance\nAssessment (SSPA). This helps establish the concurrent validity of the selected\nfeatures. We use only these features to train a simple classifier to\ndistinguish between groups of interest. Linear regression reveals that a subset\nof language features can effectively model the SSPA, with a correlation\ncoefficient of 0.75. Furthermore, the same feature set can be used to build a\nstrong binary classifier to distinguish between healthy controls and a clinical\ngroup (AUC = 0.96) and also between patients within the clinical group with\nschizophrenia and bipolar I disorder (AUC = 0.83).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10635v1",
            "date": "2019-04-24 04:16:44+00:00",
            "title": "Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings",
            "authors": [
                "Sarik Ghazarian",
                "Johnny Tian-Zheng Wei",
                "Aram Galstyan",
                "Nanyun Peng"
            ],
            "abstract": "Despite advances in open-domain dialogue systems, automatic evaluation of\nsuch systems is still a challenging problem. Traditional reference-based\nmetrics such as BLEU are ineffective because there could be many valid\nresponses for a given context that share no common words with reference\nresponses. A recent work proposed Referenced metric and Unreferenced metric\nBlended Evaluation Routine (RUBER) to combine a learning-based metric, which\npredicts relatedness between a generated response and a given query, with\nreference-based metric; it showed high correlation with human judgments. In\nthis paper, we explore using contextualized word embeddings to compute more\naccurate relatedness scores, thus better evaluation metrics. Experiments show\nthat our evaluation metrics outperform RUBER, which is trained on static\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10637v1",
            "date": "2019-04-24 04:31:46+00:00",
            "title": "Who Blames Whom in a Crisis? Detecting Blame Ties from News Articles Using Neural Networks",
            "authors": [
                "Shuailong Liang",
                "Olivia Nicol",
                "Yue Zhang"
            ],
            "abstract": "Blame games tend to follow major disruptions, be they financial crises,\nnatural disasters or terrorist attacks. To study how the blame game evolves and\nshapes the dominant crisis narratives is of great significance, as sense-making\nprocesses can affect regulatory outcomes, social hierarchies, and cultural\nnorms. However, it takes tremendous time and efforts for social scientists to\nmanually examine each relevant news article and extract the blame ties (A\nblames B). In this study, we define a new task, Blame Tie Extraction, and\nconstruct a new dataset related to the United States financial crisis\n(2007-2010) from The New York Times, The Wall Street Journal and USA Today. We\nbuild a Bi-directional Long Short-Term Memory (BiLSTM) network for contexts\nwhere the entities appear in and it learns to automatically extract such blame\nties at the document level. Leveraging the large unsupervised model such as\nGloVe and ELMo, our best model achieves an F1 score of 70% on the test set for\nblame tie extraction, making it a useful tool for social scientists to extract\nblame ties more efficiently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10641v1",
            "date": "2019-04-24 05:03:28+00:00",
            "title": "Detecting Machine-Translated Paragraphs by Matching Similar Words",
            "authors": [
                "Hoang-Quoc Nguyen-Son",
                "Tran Phuong Thao",
                "Seira Hidano",
                "Shinsaku Kiyomoto"
            ],
            "abstract": "Machine-translated text plays an important role in modern life by smoothing\ncommunication from various communities using different languages. However,\nunnatural translation may lead to misunderstanding, a detector is thus needed\nto avoid the unfortunate mistakes. While a previous method measured the\nnaturalness of continuous words using a N-gram language model, another method\nmatched noncontinuous words across sentences but this method ignores such words\nin an individual sentence. We have developed a method matching similar words\nthroughout the paragraph and estimating the paragraph-level coherence, that can\nidentify machine-translated text. Experiment evaluates on 2000 English\nhuman-generated and 2000 English machine-translated paragraphs from German\nshowing that the coherence-based method achieves high performance (accuracy =\n87.0%; equal error rate = 13.0%). It is efficiently better than previous\nmethods (best accuracy = 72.4%; equal error rate = 29.7%). Similar experiments\non Dutch and Japanese obtain 89.2% and 97.9% accuracy, respectively. The\nresults demonstrate the persistence of the proposed method in various languages\nwith different resource levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10887v1",
            "date": "2019-04-24 15:54:46+00:00",
            "title": "Listening between the Lines: Learning Personal Attributes from Conversations",
            "authors": [
                "Anna Tigunova",
                "Andrew Yates",
                "Paramita Mirza",
                "Gerhard Weikum"
            ],
            "abstract": "Open-domain dialogue agents must be able to converse about many topics while\nincorporating knowledge about the user into the conversation. In this work we\naddress the acquisition of such knowledge, for personalization in downstream\nWeb applications, by extracting personal attributes from conversations. This\nproblem is more challenging than the established task of information extraction\nfrom scientific publications or Wikipedia articles, because dialogues often\ngive merely implicit cues about the speaker. We propose methods for inferring\npersonal attributes, such as profession, age or family status, from\nconversations using deep learning. Specifically, we propose several Hidden\nAttribute Models, which are neural networks leveraging attention mechanisms and\nembeddings. Our methods are trained on a per-predicate basis to output rankings\nof object values for a given subject-predicate combination (e.g., ranking the\ndoctor and nurse professions high when speakers talk about patients, emergency\nrooms, etc). Experiments with various conversational texts including Reddit\ndiscussions, movie scripts and a collection of crowdsourced personal dialogues\ndemonstrate the viability of our methods and their superior performance\ncompared to state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10997v1",
            "date": "2019-04-24 18:23:05+00:00",
            "title": "Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors",
            "authors": [
                "Nicholas Ruiz",
                "Mattia Antonino Di Gangi",
                "Nicola Bertoldi",
                "Marcello Federico"
            ],
            "abstract": "Machine translation systems are conventionally trained on textual resources\nthat do not model phenomena that occur in spoken language. While the evaluation\nof neural machine translation systems on textual inputs is actively researched\nin the literature , little has been discovered about the complexities of\ntranslating spoken language data with neural models. We introduce and motivate\ninteresting problems one faces when considering the translation of automatic\nspeech recognition (ASR) outputs on neural machine translation (NMT) systems.\nWe test the robustness of sentence encoding approaches for NMT encoder-decoder\nmodeling, focusing on word-based over byte-pair encoding. We compare the\ntranslation of utterances containing ASR errors in state-of-the-art NMT\nencoder-decoder systems against a strong phrase-based machine translation\nbaseline in order to better understand which phenomena present in ASR outputs\nare better represented under the NMT framework than approaches that represent\ntranslation as a linear model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11018v2",
            "date": "2019-04-24 19:00:13+00:00",
            "title": "Toponym Identification in Epidemiology Articles - A Deep Learning Approach",
            "authors": [
                "MohammadReza Davari",
                "Leila Kosseim",
                "Tien D. Bui"
            ],
            "abstract": "When analyzing the spread of viruses, epidemiologists often need to identify\nthe location of infected hosts. This information can be found in public\ndatabases, such as GenBank, however, information provided in these databases\nare usually limited to the country or state level. More fine-grained\nlocalization information requires phylogeographers to manually read relevant\nscientific articles. In this work we propose an approach to automate the\nprocess of place name identification from medical (epidemiology) articles. The\nfocus of this paper is to propose a deep learning based model for toponym\ndetection and experiment with the use of external linguistic features and\ndomain specific information. The model was evaluated using a collection of 105\nepidemiology articles from PubMed Central provided by the recent SemEval task\n12. Our best detection model achieves an F1 score of $80.13\\%$, a significant\nimprovement compared to the state of the art of $69.84\\%$. These results\nunderline the importance of domain specific embedding as well as specific\nlinguistic features in toponym detection in medical journals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11024v1",
            "date": "2019-04-24 19:09:35+00:00",
            "title": "Phonetically-Oriented Word Error Alignment for Speech Recognition Error Analysis in Speech Translation",
            "authors": [
                "Nicholas Ruiz",
                "Marcello Federico"
            ],
            "abstract": "We propose a variation to the commonly used Word Error Rate (WER) metric for\nspeech recognition evaluation which incorporates the alignment of phonemes, in\nthe absence of time boundary information. After computing the Levenshtein\nalignment on words in the reference and hypothesis transcripts, spans of\nadjacent errors are converted into phonemes with word and syllable boundaries\nand a phonetic Levenshtein alignment is performed. The aligned phonemes are\nrecombined into aligned words that adjust the word alignment labels in each\nerror region. We demonstrate that our Phonetically-Oriented Word Error Rate\n(POWER) yields similar scores to WER with the added advantages of better word\nalignments and the ability to capture one-to-many word alignments corresponding\nto homophonic errors in speech recognition hypotheses. These improved\nalignments allow us to better trace the impact of Levenshtein error types on\ndownstream tasks such as speech translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11473v2",
            "date": "2019-04-25 17:37:28+00:00",
            "title": "Terminologies augmented recurrent neural network model for clinical named entity recognition",
            "authors": [
                "Ivan Lerner",
                "Nicolas Paris",
                "Xavier Tannier"
            ],
            "abstract": "We aimed to enhance the performance of a supervised model for clinical\nnamed-entity recognition (NER) using medical terminologies. In order to\nevaluate our system in French, we built a corpus for 5 types of clinical\nentities. We used a terminology-based system as baseline, built upon UMLS and\nSNOMED. Then, we evaluated a biGRU-CRF, and an hybrid system using the\nprediction of the terminology-based system as feature for the biGRU-CRF. In\nEnglish, we evaluated the NER systems on the i2b2-2009 Medication Challenge for\nDrug name recognition, which contained 8,573 entities for 268 documents. In\nFrench, we built APcNER, a corpus of 147 documents annotated for 5 entities\n(drug name, sign or symptom, disease or disorder, diagnostic procedure or lab\ntest and therapeutic procedure). We evaluated each NER systems using exact and\npartial match definition of F-measure for NER. The APcNER contains 4,837\nentities which took 28 hours to annotate, the inter-annotator agreement was\nacceptable for Drug name in exact match (85%) and acceptable for other entity\ntypes in non-exact match (>70%). For drug name recognition on both i2b2-2009\nand APcNER, the biGRU-CRF performed better that the terminology-based system,\nwith an exact-match F-measure of 91.1% versus 73% and 81.9% versus 75%\nrespectively. Moreover, the hybrid system outperformed the biGRU-CRF, with an\nexact-match F-measure of 92.2% versus 91.1% (i2b2-2009) and 88.4% versus 81.9%\n(APcNER). On APcNER corpus, the micro-average F-measure of the hybrid system on\nthe 5 entities was 69.5% in exact match, and 84.1% in non-exact match. APcNER\nis a French corpus for clinical-NER of five type of entities which covers a\nlarge variety of document types. Extending supervised model with terminology\nallowed for an easy performance gain, especially in low regimes of entities,\nand established near state of the art results on the i2b2-2009 corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11544v2",
            "date": "2019-04-25 19:15:16+00:00",
            "title": "Probing What Different NLP Tasks Teach Machines about Function Word Comprehension",
            "authors": [
                "Najoung Kim",
                "Roma Patel",
                "Adam Poliak",
                "Alex Wang",
                "Patrick Xia",
                "R. Thomas McCoy",
                "Ian Tenney",
                "Alexis Ross",
                "Tal Linzen",
                "Benjamin Van Durme",
                "Samuel R. Bowman",
                "Ellie Pavlick"
            ],
            "abstract": "We introduce a set of nine challenge tasks that test for the understanding of\nfunction words. These tasks are created by structurally mutating sentences from\nexisting datasets to target the comprehension of specific types of function\nwords (e.g., prepositions, wh-words). Using these probing tasks, we explore the\neffects of various pretraining objectives for sentence encoders (e.g., language\nmodeling, CCG supertagging and natural language inference (NLI)) on the learned\nrepresentations. Our results show that pretraining on language modeling\nperforms the best on average across our probing tasks, supporting its\nwidespread use for pretraining state-of-the-art NLP models, and CCG\nsupertagging and NLI pretraining perform comparably. Overall, no pretraining\nobjective dominates across the board, and our function word probing tasks\nhighlight several intuitive differences between pretraining objectives, e.g.,\nthat NLI helps the comprehension of negation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11564v1",
            "date": "2019-04-25 20:01:08+00:00",
            "title": "Neural Text Generation from Rich Semantic Representations",
            "authors": [
                "Valerie Hajdik",
                "Jan Buys",
                "Michael W. Goodman",
                "Emily M. Bender"
            ],
            "abstract": "We propose neural models to generate high-quality text from structured\nrepresentations based on Minimal Recursion Semantics (MRS). MRS is a rich\nsemantic representation that encodes more precise semantic detail than other\nrepresentations such as Abstract Meaning Representation (AMR). We show that a\nsequence-to-sequence model that maps a linearization of Dependency MRS, a\ngraph-based representation of MRS, to English text can achieve a BLEU score of\n66.11 when trained on gold data. The performance can be improved further using\na high-precision, broad coverage grammar-based parser to generate a large\nsilver training corpus, achieving a final BLEU score of 77.17 on the full test\nset, and 83.37 on the subset of test data most closely matching the silver data\ndomain. Our results suggest that MRS-based representations are a good choice\nfor applications that need both structured semantics and the ability to produce\nnatural language text as output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11660v2",
            "date": "2019-04-26 03:00:19+00:00",
            "title": "Transformers with convolutional context for ASR",
            "authors": [
                "Abdelrahman Mohamed",
                "Dmytro Okhonko",
                "Luke Zettlemoyer"
            ],
            "abstract": "The recent success of transformer networks for neural machine translation and\nother NLP tasks has led to a surge in research work trying to apply it for\nspeech recognition. Recent efforts studied key research questions around ways\nof combining positional embedding with speech features, and stability of\noptimization for large scale learning of transformer networks. In this paper,\nwe propose replacing the sinusoidal positional embedding for transformers with\nconvolutionally learned input representations. These contextual representations\nprovide subsequent transformer blocks with relative positional information\nneeded for discovering long-range relationships between local concepts. The\nproposed system has favorable optimization characteristics where our reported\nresults are produced with fixed learning rate of 1.0 and no warmup steps. The\nproposed model achieves a competitive 4.7% and 12.9% WER on the Librispeech\n``test clean'' and ``test other'' subsets when no extra LM text is provided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11783v2",
            "date": "2019-04-26 11:56:35+00:00",
            "title": "Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors",
            "authors": [
                "Anne Lauscher",
                "Goran Glava\u0161"
            ],
            "abstract": "Word embeddings have recently been shown to reflect many of the pronounced\nsocietal biases (e.g., gender bias or racial bias). Existing studies are,\nhowever, limited in scope and do not investigate the consistency of biases\nacross relevant dimensions like embedding models, types of texts, and different\nlanguages. In this work, we present a systematic study of biases encoded in\ndistributional word vector spaces: we analyze how consistent the bias effects\nare across languages, corpora, and embedding models. Furthermore, we analyze\nthe cross-lingual biases encoded in bilingual embedding spaces, indicative of\nthe effects of bias transfer encompassed in cross-lingual transfer of NLP\nmodels. Our study yields some unexpected findings, e.g., that biases can be\nemphasized or downplayed by different embedding models or that user-generated\ncontent may be less biased than encyclopedic text. We hope our work catalyzes\nbias research in NLP and informs the development of bias reduction techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11942v1",
            "date": "2019-04-26 17:21:59+00:00",
            "title": "Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding",
            "authors": [
                "Rujun Han",
                "Mengyue Liang",
                "Bashar Alhafni",
                "Nanyun Peng"
            ],
            "abstract": "Learning causal and temporal relationships between events is an important\nstep towards deeper story and commonsense understanding. Though there are\nabundant datasets annotated with event relations for story comprehension, many\nhave no empirical results associated with them. In this work, we establish\nstrong baselines for event temporal relation extraction on two under-explored\nstory narrative datasets: Richer Event Description (RED) and Causal and\nTemporal Relation Scheme (CaTeRS). To the best of our knowledge, these are the\nfirst results reported on these two datasets. We demonstrate that neural\nnetwork-based models can outperform some strong traditional linguistic\nfeature-based models. We also conduct comparative studies to show the\ncontribution of adopting contextualized word embeddings (BERT) for event\ntemporal relation extraction from stories. Detailed analyses are offered to\nbetter understand the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12087v1",
            "date": "2019-04-27 01:51:55+00:00",
            "title": "Experiments in Cuneiform Language Identification",
            "authors": [
                "Gustavo Henrique Paetzold",
                "Marcos Zampieri"
            ],
            "abstract": "This paper presents methods to discriminate between languages and dialects\nwritten in Cuneiform script, one of the first writing systems in the world. We\nreport the results obtained by the PZ team in the Cuneiform Language\nIdentification (CLI) shared task organized within the scope of the VarDial\nEvaluation Campaign 2019. The task included two languages, Sumerian and\nAkkadian. The latter is divided into six dialects: Old Babylonian, Middle\nBabylonian peripheral, Standard Babylonian, Neo Babylonian, Late Babylonian,\nand Neo Assyrian. We approach the task using a meta-classifier trained on\nvarious SVM models and we show the effectiveness of the system for this task.\nOur submission achieved 0.738 F1 score in discriminating between the seven\nlanguages and dialects and it was ranked fourth in the competition among eight\nteams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12104v1",
            "date": "2019-04-27 04:24:07+00:00",
            "title": "Several Experiments on Investigating Pretraining and Knowledge-Enhanced Models for Natural Language Inference",
            "authors": [
                "Tianda Li",
                "Xiaodan Zhu",
                "Quan Liu",
                "Qian Chen",
                "Zhigang Chen",
                "Si Wei"
            ],
            "abstract": "Natural language inference (NLI) is among the most challenging tasks in\nnatural language understanding. Recent work on unsupervised pretraining that\nleverages unsupervised signals such as language-model and sentence prediction\nobjectives has shown to be very effective on a wide range of NLP problems. It\nwould still be desirable to further understand how it helps NLI; e.g., if it\nlearns artifacts in data annotation or instead learn true inference knowledge.\nIn addition, external knowledge that does not exist in the limited amount of\nNLI training data may be added to NLI models in two typical ways, e.g., from\nhuman-created resources or an unsupervised pretraining paradigm. We runs\nseveral experiments here to investigate whether they help NLI in the same way,\nand if not,how?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12166v1",
            "date": "2019-04-27 15:15:14+00:00",
            "title": "HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning",
            "authors": [
                "Hitomi Yanaka",
                "Koji Mineshima",
                "Daisuke Bekki",
                "Kentaro Inui",
                "Satoshi Sekine",
                "Lasha Abzianidze",
                "Johan Bos"
            ],
            "abstract": "Large crowdsourced datasets are widely used for training and evaluating\nneural models on natural language inference (NLI). Despite these efforts,\nneural models have a hard time capturing logical inferences, including those\nlicensed by phrase replacements, so-called monotonicity reasoning. Since no\nlarge dataset has been developed for monotonicity reasoning, it is still\nunclear whether the main obstacle is the size of datasets or the model\narchitectures themselves. To investigate this issue, we introduce a new\ndataset, called HELP, for handling entailments with lexical and logical\nphenomena. We add it to training data for the state-of-the-art neural models\nand evaluate them on test sets for monotonicity phenomena. The results showed\nthat our data augmentation improved the overall accuracy. We also find that the\nimprovement is better on monotonicity inferences with lexical replacements than\non downward inferences with disjunction and modification. This suggests that\nsome types of inferences can be improved by our data augmentation while others\nare immune to it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12213v1",
            "date": "2019-04-27 21:14:21+00:00",
            "title": "Towards Recognizing Phrase Translation Processes: Experiments on English-French",
            "authors": [
                "Yuming Zhai",
                "Pooyan Safari",
                "Gabriel Illouz",
                "Alexandre Allauzen",
                "Anne Vilnat"
            ],
            "abstract": "When translating phrases (words or group of words), human translators,\nconsciously or not, resort to different translation processes apart from the\nliteral translation, such as Idiom Equivalence, Generalization,\nParticularization, Semantic Modulation, etc. Translators and linguists (such as\nVinay and Darbelnet, Newmark, etc.) have proposed several typologies to\ncharacterize the different translation processes. However, to the best of our\nknowledge, there has not been effort to automatically classify these\nfine-grained translation processes. Recently, an English-French parallel corpus\nof TED Talks has been manually annotated with translation process categories,\nalong with established annotation guidelines. Based on these annotated\nexamples, we propose an automatic classification of translation processes at\nsubsentential level. Experimental results show that we can distinguish\nnon-literal translation from literal translation with an accuracy of 87.09%,\nand 55.20% for classifying among five non-literal translation processes. This\nwork demonstrates that it is possible to automatically classify translation\nprocesses. Even with a small amount of annotated examples, our experiments show\nthe directions that we can follow in future work. One of our long term\nobjectives is leveraging this automatic classification to better control\nparaphrase extraction from bilingual parallel corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12324v1",
            "date": "2019-04-28 13:57:54+00:00",
            "title": "OPIEC: An Open Information Extraction Corpus",
            "authors": [
                "Kiril Gashteovski",
                "Sebastian Wanner",
                "Sven Hertling",
                "Samuel Broscheit",
                "Rainer Gemulla"
            ],
            "abstract": "Open information extraction (OIE) systems extract relations and their\narguments from natural language text in an unsupervised manner. The resulting\nextractions are a valuable resource for downstream tasks such as knowledge base\nconstruction, open question answering, or event schema induction. In this\npaper, we release, describe, and analyze an OIE corpus called OPIEC, which was\nextracted from the text of English Wikipedia. OPIEC complements the available\nOIE resources: It is the largest OIE corpus publicly available to date (over\n340M triples) and contains valuable metadata such as provenance information,\nconfidence scores, linguistic annotations, and semantic annotations including\nspatial and temporal information. We analyze the OPIEC corpus by comparing its\ncontent with knowledge bases such as DBpedia or YAGO, which are also based on\nWikipedia. We found that most of the facts between entities present in OPIEC\ncannot be found in DBpedia and/or YAGO, that OIE facts often differ in the\nlevel of specificity compared to knowledge base facts, and that OIE open\nrelations are generally highly polysemous. We believe that the OPIEC corpus is\na valuable resource for future research on automated knowledge base\nconstruction.",
            "journal ref": "In Proceedings of the Conference of Automatic Knowledge Base\n  Construction (AKBC) 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12550v1",
            "date": "2019-04-29 10:21:58+00:00",
            "title": "Semantic Matching of Documents from Heterogeneous Collections: A Simple and Transparent Method for Practical Applications",
            "authors": [
                "Mark-Christoph M\u00fcller"
            ],
            "abstract": "We present a very simple, unsupervised method for the pairwise matching of\ndocuments from heterogeneous collections. We demonstrate our method with the\nConcept-Project matching task, which is a binary classification task involving\npairs of documents from heterogeneous collections. Although our method only\nemploys standard resources without any domain- or task-specific modifications,\nit clearly outperforms the more complex system of the original authors. In\naddition, our method is transparent, because it provides explicit information\nabout how a similarity score was computed, and efficient, because it is based\non the aggregation of (pre-computable) word-level similarities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.13018v1",
            "date": "2019-04-30 02:18:23+00:00",
            "title": "A self-attention based deep learning method for lesion attribute detection from CT reports",
            "authors": [
                "Yifan Peng",
                "Ke Yan",
                "Veit Sandfort",
                "Ronald M. Summers",
                "Zhiyong Lu"
            ],
            "abstract": "In radiology, radiologists not only detect lesions from the medical image,\nbut also describe them with various attributes such as their type, location,\nsize, shape, and intensity. While these lesion attributes are rich and useful\nin many downstream clinical applications, how to extract them from the\nradiology reports is less studied. This paper outlines a novel deep learning\nmethod to automatically extract attributes of lesions of interest from the\nclinical text. Different from classical CNN models, we integrated the\nmulti-head self-attention mechanism to handle the long-distance information in\nthe sentence, and to jointly correlate different portions of sentence\nrepresentation subspaces in parallel. Evaluation on an in-house corpus\ndemonstrates that our method can achieve high performance with 0.848 in\nprecision, 0.788 in recall, and 0.815 in F-score. The new method and\nconstructed corpus will enable us to build automatic systems with a\nhigher-level understanding of the radiological world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.13178v1",
            "date": "2019-04-30 11:51:52+00:00",
            "title": "Fine-grained Entity Recognition with Reduced False Negatives and Large Type Coverage",
            "authors": [
                "Abhishek Abhishek",
                "Sanya Bathla Taneja",
                "Garima Malik",
                "Ashish Anand",
                "Amit Awekar"
            ],
            "abstract": "Fine-grained Entity Recognition (FgER) is the task of detecting and\nclassifying entity mentions to a large set of types spanning diverse domains\nsuch as biomedical, finance and sports. We observe that when the type set spans\nseveral domains, detection of entity mention becomes a limitation for\nsupervised learning models. The primary reason being lack of dataset where\nentity boundaries are properly annotated while covering a large spectrum of\nentity types. Our work directly addresses this issue. We propose Heuristics\nAllied with Distant Supervision (HAnDS) framework to automatically construct a\nquality dataset suitable for the FgER task. HAnDS framework exploits the high\ninterlink among Wikipedia and Freebase in a pipelined manner, reducing\nannotation errors introduced by naively using distant supervision approach.\nUsing HAnDS framework, we create two datasets, one suitable for building FgER\nsystems recognizing up to 118 entity types based on the FIGER type hierarchy\nand another for up to 1115 entity types based on the TypeNet hierarchy. Our\nextensive empirical experimentation warrants the quality of the generated\ndatasets. Along with this, we also provide a manually annotated dataset for\nbenchmarking FgER systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00195v3",
            "date": "2019-05-01 06:03:56+00:00",
            "title": "Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors",
            "authors": [
                "Trung Trinh",
                "Tho Quan",
                "Trung Mai"
            ],
            "abstract": "Most of the information on the Internet is represented in the form of\nmicrotexts, which are short text snippets such as news headlines or tweets.\nThese sources of information are abundant, and mining these data could uncover\nmeaningful insights. Topic modeling is one of the popular methods to extract\nknowledge from a collection of documents; however, conventional topic models\nsuch as latent Dirichlet allocation (LDA) are unable to perform well on short\ndocuments, mostly due to the scarcity of word co-occurrence statistics embedded\nin the data. The objective of our research is to create a topic model that can\nachieve great performances on microtexts while requiring a small runtime for\nscalability to large datasets. To solve the lack of information of microtexts,\nwe allow our method to take advantage of word embeddings for additional\nknowledge of relationships between words. For speed and scalability, we apply\nautoencoding variational Bayes, an algorithm that can perform efficient\nblack-box inference in probabilistic models. The result of our work is a novel\ntopic model called the nested variational autoencoder, which is a distribution\nthat takes into account word vectors and is parameterized by a neural network\narchitecture. For optimization, the model is trained to approximate the\nposterior distribution of the original LDA model. Experiments show the\nimprovements of our model on microtexts as well as its runtime advantage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00472v1",
            "date": "2019-05-01 19:55:46+00:00",
            "title": "A system for the 2019 Sentiment, Emotion and Cognitive State Task of DARPAs LORELEI project",
            "authors": [
                "Victor R Martinez",
                "Anil Ramakrishna",
                "Ming-Chang Chiu",
                "Karan Singla",
                "Shrikanth Narayanan"
            ],
            "abstract": "During the course of a Humanitarian Assistance-Disaster Relief (HADR) crisis,\nthat can happen anywhere in the world, real-time information is often posted\nonline by the people in need of help which, in turn, can be used by different\nstakeholders involved with management of the crisis. Automated processing of\nsuch posts can considerably improve the effectiveness of such efforts; for\nexample, understanding the aggregated emotion from affected populations in\nspecific areas may help inform decision-makers on how to best allocate\nresources for an effective disaster response. However, these efforts may be\nseverely limited by the availability of resources for the local language. The\nongoing DARPA project Low Resource Languages for Emergent Incidents (LORELEI)\naims to further language processing technologies for low resource languages in\nthe context of such a humanitarian crisis. In this work, we describe our\nsubmission for the 2019 Sentiment, Emotion and Cognitive state (SEC) pilot task\nof the LORELEI project. We describe a collection of sentiment analysis systems\nincluded in our submission along with the features extracted. Our fielded\nsystems obtained the best results in both English and Spanish language\nevaluations of the SEC pilot task.",
            "journal ref": "2019 8th International Conference on Affective Computing and\n  Intelligent Interaction (ACII)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00724v2",
            "date": "2019-05-02 13:24:55+00:00",
            "title": "KnowBias: A Novel AI Method to Detect Polarity in Online Content",
            "authors": [
                "Aditya Saligrama"
            ],
            "abstract": "We propose a novel training and inference method for detecting political bias\nin long text content such as newspaper opinion articles. Obtaining long text\ndata and annotations at sufficient scale for training is difficult, but it is\nrelatively easy to extract political polarity from tweets through their\nauthorship; as such, we train on tweets and perform inference on articles.\nUniversal sentence encoders and other existing methods that aim to address this\ndomain-adaptation scenario deliver inaccurate and inconsistent predictions on\narticles, which we show is due to a difference in opinion concentration between\ntweets and articles. We propose a two-step classification scheme that utilizes\na neutral detector trained on tweets to remove neutral sentences from articles\nin order to align opinion concentration and therefore improve accuracy on that\ndomain.\n  We evaluate our two-step approach using a variety of test suites, including a\nset of tweets and long-form articles where annotations were crowd-sourced to\ndecrease label noise, measuring accuracy and Spearman-rho rank correlation. In\npractice, KnowBias achieves a high accuracy of 86 (rho = 0.65) on these tweets\nand 75 (rho = 0.69) on long-form articles. While we validate our method on\npolitical bias, our scheme is general and can be readily applied to other\nsettings, where there exist such domain mismatches between source and target\ndomains. Our implementation is available for public use at https://knowbias.ml.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00982v1",
            "date": "2019-05-02 22:01:57+00:00",
            "title": "Context awareness and embedding for biomedical event extraction",
            "authors": [
                "Shankai Yan",
                "Ka-Chun Wong"
            ],
            "abstract": "Motivation: Biomedical event detection is fundamental for information\nextraction in molecular biology and biomedical research. The detected events\nform the central basis for comprehensive biomedical knowledge fusion,\nfacilitating the digestion of massive information influx from literature.\nLimited by the feature context, the existing event detection models are mostly\napplicable for a single task. A general and scalable computational model is\ndesiderated for biomedical knowledge management. Results: We consider and\npropose a bottom-up detection framework to identify the events from recognized\narguments. To capture the relations between the arguments, we trained a\nbi-directional Long Short-Term Memory (LSTM) network to model their context\nembedding. Leveraging the compositional attributes, we further derived the\ncandidate samples for training event classifiers. We built our models on the\ndatasets from BioNLP Shared Task for evaluations. Our method achieved the\naverage F-scores of 0.81 and 0.92 on BioNLPST-BGI and BioNLPST-BB datasets\nrespectively. Comparing with 7 state-of-the-art methods, our method nearly\ndoubled the existing F-score performance (0.92 vs 0.56) on the BioNLPST-BB\ndataset. Case studies were conducted to reveal the underlying reasons.\nAvailability: https://github.com/cskyan/evntextrc",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01338v1",
            "date": "2019-05-03 18:38:39+00:00",
            "title": "Effectiveness of Self Normalizing Neural Networks for Text Classification",
            "authors": [
                "Avinash Madasu",
                "Vijjini Anvesh Rao"
            ],
            "abstract": "Self Normalizing Neural Networks(SNN) proposed on Feed Forward Neural\nNetworks(FNN) outperform regular FNN architectures in various machine learning\ntasks. Particularly in the domain of Computer Vision, the activation function\nScaled Exponential Linear Units (SELU) proposed for SNNs, perform better than\nother non linear activations such as ReLU. The goal of SNN is to produce a\nnormalized output for a normalized input. Established neural network\narchitectures like feed forward networks and Convolutional Neural Networks(CNN)\nlack the intrinsic nature of normalizing outputs. Hence, requiring additional\nlayers such as Batch Normalization. Despite the success of SNNs, their\ncharacteristic features on other network architectures like CNN haven't been\nexplored, especially in the domain of Natural Language Processing. In this\npaper we aim to show the effectiveness of proposed, Self Normalizing\nConvolutional Neural Networks(SCNN) on text classification. We analyze their\nperformance with the standard CNN architecture used on several text\nclassification datasets. Our experiments demonstrate that SCNN achieves\ncomparable results to standard CNN model with significantly fewer parameters.\nFurthermore it also outperforms CNN with equal number of parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01420v1",
            "date": "2019-05-04 03:22:55+00:00",
            "title": "Contextualization of Morphological Inflection",
            "authors": [
                "Ekaterina Vylomova",
                "Ryan Cotterell",
                "Timothy Baldwin",
                "Trevor Cohn",
                "Jason Eisner"
            ],
            "abstract": "Critical to natural language generation is the production of correctly\ninflected text. In this paper, we isolate the task of predicting a fully\ninflected sentence from its partially lemmatized version. Unlike traditional\nmorphological inflection or surface realization, our task input does not\nprovide ``gold'' tags that specify what morphological features to realize on\neach lemmatized word; rather, such features must be inferred from sentential\ncontext. We develop a neural hybrid graphical model that explicitly\nreconstructs morphological features before predicting the inflected forms, and\ncompare this to a system that directly predicts the inflected forms without\nrelying on any morphological annotation. We experiment on several typologically\ndiverse languages from the Universal Dependencies treebanks, showing the\nutility of incorporating linguistically-motivated latent variables into NLP\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01715v1",
            "date": "2019-05-05 16:53:03+00:00",
            "title": "A Parallel Corpus of Theses and Dissertations Abstracts",
            "authors": [
                "Felipe Soares",
                "Gabrielli Harumi Yamashita",
                "Michel Jose Anzanello"
            ],
            "abstract": "In Brazil, the governmental body responsible for overseeing and coordinating\npost-graduate programs, CAPES, keeps records of all theses and dissertations\npresented in the country. Information regarding such documents can be accessed\nonline in the Theses and Dissertations Catalog (TDC), which contains abstracts\nin Portuguese and English, and additional metadata. Thus, this database can be\na potential source of parallel corpora for the Portuguese and English\nlanguages. In this article, we present the development of a parallel corpus\nfrom TDC, which is made available by CAPES under the open data initiative.\nApproximately 240,000 documents were collected and aligned using the Hunalign\ntool. We demonstrate the capability of our developed corpus by training\nStatistical Machine Translation (SMT) and Neural Machine Translation (NMT)\nmodels for both language directions, followed by a comparison with Google\nTranslate (GT). Both translation models presented better BLEU scores than GT,\nwith NMT system being the most accurate one. Sentence alignment was also\nmanually evaluated, presenting an average of 82.30% correctly aligned\nsentences. Our parallel corpus is freely available in TMX format, with\ncomplementary information regarding document metadata",
            "journal ref": "Computational Processing of the Portuguese Language 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01739v1",
            "date": "2019-05-05 19:57:32+00:00",
            "title": "HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings",
            "authors": [
                "Saba Anwar",
                "Dmitry Ustalov",
                "Nikolay Arefyev",
                "Simone Paolo Ponzetto",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We present our system for semantic frame induction that showed the best\nperformance in Subtask B.1 and finished as the runner-up in Subtask A of the\nSemEval 2019 Task 2 on unsupervised semantic frame induction (QasemiZadeh et\nal., 2019). Our approach separates this task into two independent steps: verb\nclustering using word and their context embeddings and role labeling by\ncombining these embeddings with syntactical features. A simple combination of\nthese steps shows very competitive results and can be extended to process other\ndatasets and languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01799v3",
            "date": "2019-05-06 02:39:31+00:00",
            "title": "RSL19BD at DBDC4: Ensemble of Decision Tree-based and LSTM-based Models",
            "authors": [
                "Chih-Hao Wang",
                "Sosuke Kato",
                "Tetsuya Sakai"
            ],
            "abstract": "RSL19BD (Waseda University Sakai Laboratory) participated in the Fourth\nDialogue Breakdown Detection Challenge (DBDC4) and submitted five runs to both\nEnglish and Japanese subtasks. In these runs, we utilise the Decision\nTree-based model and the Long Short-Term Memory-based (LSTM-based) model\nfollowing the approaches of RSL17BD and KTH in the Third Dialogue Breakdown\nDetection Challenge (DBDC3) respectively. The Decision Tree-based model follows\nthe approach of RSL17BD but utilises RandomForestRegressor instead of\nExtraTreesRegressor. In addition, instead of predicting the mean and the\nvariance of the probability distribution of the three breakdown labels, it\npredicts the probability of each label directly. The LSTM-based model follows\nthe approach of KTH with some changes in the architecture and utilises\nConvolutional Neural Network (CNN) to perform text feature extraction. In\naddition, instead of targeting the single breakdown label and minimising the\ncategorical cross entropy loss, it targets the probability distribution of the\nthree breakdown labels and minimises the mean squared error. Run 1 utilises a\nDecision Tree-based model; Run 2 utilises an LSTM-based model; Run 3 performs\nan ensemble of 5 LSTM-based models; Run 4 performs an ensemble of Run 1 and Run\n2; Run 5 performs an ensemble of Run 1 and Run 3. Run 5 statistically\nsignificantly outperformed all other runs in terms of MSE (NB, PB, B) for the\nEnglish data and all other runs except Run 4 in terms of MSE (NB, PB, B) for\nthe Japanese data (alpha level = 0.05).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01852v1",
            "date": "2019-05-06 07:33:00+00:00",
            "title": "A Large Parallel Corpus of Full-Text Scientific Articles",
            "authors": [
                "Felipe Soares",
                "Viviane Pereira Moreira",
                "Karin Becker"
            ],
            "abstract": "The Scielo database is an important source of scientific information in Latin\nAmerica, containing articles from several research domains. A striking\ncharacteristic of Scielo is that many of its full-text contents are presented\nin more than one language, thus being a potential source of parallel corpora.\nIn this article, we present the development of a parallel corpus from Scielo in\nthree languages: English, Portuguese, and Spanish. Sentences were automatically\naligned using the Hunalign algorithm for all language pairs, and for a subset\nof trilingual articles also. We demonstrate the capabilities of our corpus by\ntraining a Statistical Machine Translation system (Moses) for each language\npair, which outperformed related works on scientific articles. Sentence\nalignment was also manually evaluated, presenting an average of 98.8% correctly\naligned sentences across all languages. Our parallel corpus is freely available\nin the TMX format, with complementary information regarding article metadata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01855v1",
            "date": "2019-05-06 07:36:59+00:00",
            "title": "UFRGS Participation on the WMT Biomedical Translation Shared Task",
            "authors": [
                "Felipe Soares",
                "Karin Becker"
            ],
            "abstract": "This paper describes the machine translation systems developed by the\nUniversidade Federal do Rio Grande do Sul (UFRGS) team for the biomedical\ntranslation shared task. Our systems are based on statistical machine\ntranslation and neural machine translation, using the Moses and OpenNMT\ntoolkits, respectively. We participated in four translation directions for the\nEnglish/Spanish and English/Portuguese language pairs. To create our training\ndata, we concatenated several parallel corpora, both from in-domain and\nout-of-domain sources, as well as terminological resources from UMLS. Our\nsystems achieved the best BLEU scores according to the official shared task\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01896v4",
            "date": "2019-05-06 09:32:28+00:00",
            "title": "Distributional Semantics and Linguistic Theory",
            "authors": [
                "Gemma Boleda"
            ],
            "abstract": "Distributional semantics provides multi-dimensional, graded, empirically\ninduced word representations that successfully capture many aspects of meaning\nin natural languages, as shown in a large body of work in computational\nlinguistics; yet, its impact in theoretical linguistics has so far been\nlimited. This review provides a critical discussion of the literature on\ndistributional semantics, with an emphasis on methods and results that are of\nrelevance for theoretical linguistics, in three areas: semantic change,\npolysemy and composition, and the grammar-semantics interface (specifically,\nthe interface of semantics with syntax and with derivational morphology). The\nreview aims at fostering greater cross-fertilization of theoretical and\ncomputational approaches to language, as a means to advance our collective\nknowledge of how it works.",
            "journal ref": "Annu. Rev. Linguist. 6:213-34 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01961v1",
            "date": "2019-04-02 02:01:28+00:00",
            "title": "Evaluating the Portability of an NLP System for Processing Echocardiograms: A Retrospective, Multi-site Observational Study",
            "authors": [
                "Prakash Adekkanattu",
                "Guoqian Jiang",
                "Yuan Luo",
                "Paul R. Kingsbury",
                "Zhenxing Xu",
                "Luke V. Rasmussen",
                "Jennifer A. Pacheco",
                "Richard C. Kiefer",
                "Daniel J. Stone",
                "Pascal S. Brandt",
                "Liang Yao",
                "Yizhen Zhong",
                "Yu Deng",
                "Fei Wang",
                "Jessica S. Ancker",
                "Thomas R. Campion",
                "Jyotishman Pathak"
            ],
            "abstract": "While natural language processing (NLP) of unstructured clinical narratives\nholds the potential for patient care and clinical research, portability of NLP\napproaches across multiple sites remains a major challenge. This study\ninvestigated the portability of an NLP system developed initially at the\nDepartment of Veterans Affairs (VA) to extract 27 key cardiac concepts from\nfree-text or semi-structured echocardiograms from three academic medical\ncenters: Weill Cornell Medicine, Mayo Clinic and Northwestern Medicine. While\nthe NLP system showed high precision and recall measurements for four target\nconcepts (aortic valve regurgitation, left atrium size at end systole, mitral\nvalve regurgitation, tricuspid valve regurgitation) across all sites, we found\nmoderate or poor results for the remaining concepts and the NLP system\nperformance varied between individual sites.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01967v1",
            "date": "2019-04-24 03:23:38+00:00",
            "title": "PhonSenticNet: A Cognitive Approach to Microtext Normalization for Concept-Level Sentiment Analysis",
            "authors": [
                "Ranjan Satapathy",
                "Aalind Singh",
                "Erik Cambria"
            ],
            "abstract": "With the current upsurge in the usage of social media platforms, the trend of\nusing short text (microtext) in place of standard words has seen a significant\nrise. The usage of microtext poses a considerable performance issue in\nconcept-level sentiment analysis, since models are trained on standard words.\nThis paper discusses the impact of coupling sub-symbolic (phonetics) with\nsymbolic (machine learning) Artificial Intelligence to transform the\nout-of-vocabulary concepts into their standard in-vocabulary form. The phonetic\ndistance is calculated using the Sorensen similarity algorithm. The\nphonetically similar invocabulary concepts thus obtained are then used to\ncompute the correct polarity value, which was previously being miscalculated\nbecause of the presence of microtext. Our proposed framework increases the\naccuracy of polarity detection by 6% as compared to the earlier model. This\nalso validates the fact that microtext normalization is a necessary\npre-requisite for the sentiment analysis task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01974v1",
            "date": "2019-04-19 08:21:01+00:00",
            "title": "A Novel Task-Oriented Text Corpus in Silent Speech Recognition and its Natural Language Generation Construction Method",
            "authors": [
                "Dong Cao",
                "Dongdong Zhang",
                "HaiBo Chen"
            ],
            "abstract": "Millions of people with severe speech disorders around the world may regain\ntheir communication capabilities through techniques of silent speech\nrecognition (SSR). Using electroencephalography (EEG) as a biomarker for speech\ndecoding has been popular for SSR. However, the lack of SSR text corpus has\nimpeded the development of this technique. Here, we construct a novel\ntask-oriented text corpus, which is utilized in the field of SSR. In the\nprocess of construction, we propose a task-oriented hybrid construction method\nbased on natural language generation algorithm. The algorithm focuses on the\nstrategy of data-to-text generation, and has two advantages including\nlinguistic quality and high diversity. These two advantages use template-based\nmethod and deep neural networks respectively. In an SSR experiment with the\ngenerated text corpus, analysis results show that the performance of our hybrid\nconstruction method outperforms the pure method such as template-based natural\nlanguage generation or neural natural language generation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01975v1",
            "date": "2019-04-18 22:17:21+00:00",
            "title": "Point-less: More Abstractive Summarization with Pointer-Generator Networks",
            "authors": [
                "Freek Boutkan",
                "Jorn Ranzijn",
                "David Rau",
                "Eelco van der Wel"
            ],
            "abstract": "The Pointer-Generator architecture has shown to be a big improvement for\nabstractive summarization seq2seq models. However, the summaries produced by\nthis model are largely extractive as over 30% of the generated sentences are\ncopied from the source text. This work proposes a multihead attention\nmechanism, pointer dropout, and two new loss functions to promote more\nabstractive summaries while maintaining similar ROUGE scores. Both the\nmultihead attention and dropout do not improve N-gram novelty, however, the\ndropout acts as a regularizer which improves the ROUGE score. The new loss\nfunction achieves significantly higher novel N-grams and sentences, at the cost\nof a slightly lower ROUGE score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01976v1",
            "date": "2019-04-23 15:15:12+00:00",
            "title": "TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks",
            "authors": [
                "Md. Akmal Haidar",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "Text generation is of particular interest in many NLP applications such as\nmachine translation, language modeling, and text summarization. Generative\nadversarial networks (GANs) achieved a remarkable success in high quality image\ngeneration in computer vision,and recently, GANs have gained lots of interest\nfrom the NLP community as well. However, achieving similar success in NLP would\nbe more challenging due to the discrete nature of text. In this work, we\nintroduce a method using knowledge distillation to effectively exploit GAN\nsetup for text generation. We demonstrate how autoencoders (AEs) can be used\nfor providing a continuous representation of sentences, which is a smooth\nrepresentation that assign non-zero probabilities to more than one word. We\ndistill this representation to train the generator to synthesize similar smooth\nrepresentations. We perform a number of experiments to validate our idea using\ndifferent datasets and show that our proposed approach yields better\nperformance in terms of the BLEU score and Jensen-Shannon distance (JSD)\nmeasure compared to traditional GAN-based text generation approaches without\npre-training.",
            "journal ref": "32nd Canadian Conference on Artificial Intelligence 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02239v1",
            "date": "2019-05-06 19:04:57+00:00",
            "title": "English-Bhojpuri SMT System: Insights from the Karaka Model",
            "authors": [
                "Atul Kr. Ojha"
            ],
            "abstract": "This thesis has been divided into six chapters namely: Introduction, Karaka\nModel and it impacts on Dependency Parsing, LT Resources for Bhojpuri,\nEnglish-Bhojpuri SMT System: Experiment, Evaluation of EB-SMT System, and\nConclusion. Chapter one introduces this PhD research by detailing the\nmotivation of the study, the methodology used for the study and the literature\nreview of the existing MT related work in Indian Languages. Chapter two talks\nof the theoretical background of Karaka and Karaka model. Along with this, it\ntalks about previous related work. It also discusses the impacts of the Karaka\nmodel in NLP and dependency parsing. It compares Karaka dependency and\nUniversal Dependency. It also presents a brief idea of the implementation of\nthese models in the SMT system for English-Bhojpuri language pair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02265v3",
            "date": "2019-05-06 21:14:41+00:00",
            "title": "Comprehensible Context-driven Text Game Playing",
            "authors": [
                "Xusen Yin",
                "Jonathan May"
            ],
            "abstract": "In order to train a computer agent to play a text-based computer game, we\nmust represent each hidden state of the game. A Long Short-Term Memory (LSTM)\nmodel running over observed texts is a common choice for state construction.\nHowever, a normal Deep Q-learning Network (DQN) for such an agent requires\nmillions of steps of training or more to converge. As such, an LSTM-based DQN\ncan take tens of days to finish the training process. Though we can use a\nConvolutional Neural Network (CNN) as a text-encoder to construct states much\nfaster than the LSTM, doing so without an understanding of the syntactic\ncontext of the words being analyzed can slow convergence. In this paper, we use\na fast CNN to encode position- and syntax-oriented structures extracted from\nobserved texts as states. We additionally augment the reward signal in a\nuniversal and practical manner. Together, we show that our improvements can not\nonly speed up the process by one order of magnitude but also learn a superior\nagent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02869v1",
            "date": "2019-05-08 02:12:18+00:00",
            "title": "Automatic Inference of Minimalist Grammars using an SMT-Solver",
            "authors": [
                "Sagar Indurkhya"
            ],
            "abstract": "We introduce (1) a novel parser for Minimalist Grammars (MG), encoded as a\nsystem of first-order logic formulae that may be evaluated using an SMT-solver,\nand (2) a novel procedure for inferring Minimalist Grammars using this parser.\nThe input to this procedure is a sequence of sentences that have been annotated\nwith syntactic relations such as semantic role labels (connecting arguments to\npredicates) and subject-verb agreement. The output of this procedure is a set\nof minimalist grammars, each of which is able to parse the sentences in the\ninput sequence such that the parse for a sentence has the same syntactic\nrelations as those specified in the annotation for that sentence. We applied\nthis procedure to a set of sentences annotated with syntactic relations and\nevaluated the inferred grammars using cost functions inspired by the Minimum\nDescription Length principle and the Subset principle. Inferred grammars that\nwere optimal with respect to certain combinations of these cost functions were\nfound to align with contemporary theories of syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02878v1",
            "date": "2019-05-08 02:56:43+00:00",
            "title": "Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations",
            "authors": [
                "Meishan Zhang",
                "Zhenghua Li",
                "Guohong Fu",
                "Min Zhang"
            ],
            "abstract": "Syntax has been demonstrated highly effective in neural machine translation\n(NMT). Previous NMT models integrate syntax by representing 1-best tree outputs\nfrom a well-trained parsing system, e.g., the representative Tree-RNN and\nTree-Linearization methods, which may suffer from error propagation. In this\nwork, we propose a novel method to integrate source-side syntax implicitly for\nNMT. The basic idea is to use the intermediate hidden representations of a\nwell-trained end-to-end dependency parser, which are referred to as\nsyntax-aware word representations (SAWRs). Then, we simply concatenate such\nSAWRs with ordinary word embeddings to enhance basic NMT models. The method can\nbe straightforwardly integrated into the widely-used sequence-to-sequence\n(Seq2Seq) NMT models. We start with a representative RNN-based Seq2Seq baseline\nsystem, and test the effectiveness of our proposed method on two benchmark\ndatasets of the Chinese-English and English-Vietnamese translation tasks,\nrespectively. Experimental results show that the proposed approach is able to\nbring significant BLEU score improvements on the two datasets compared with the\nbaseline, 1.74 points for Chinese-English translation and 0.80 point for\nEnglish-Vietnamese translation, respectively. In addition, the approach also\noutperforms the explicit Tree-RNN and Tree-Linearization methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02973v1",
            "date": "2019-05-08 09:24:48+00:00",
            "title": "On the Feasibility of Automated Detection of Allusive Text Reuse",
            "authors": [
                "Enrique Manjavacas",
                "Brian Long",
                "Mike Kestemont"
            ],
            "abstract": "The detection of allusive text reuse is particularly challenging due to the\nsparse evidence on which allusive references rely---commonly based on none or\nvery few shared words. Arguably, lexical semantics can be resorted to since\nuncovering semantic relations between words has the potential to increase the\nsupport underlying the allusion and alleviate the lexical sparsity. A further\nobstacle is the lack of evaluation benchmark corpora, largely due to the highly\ninterpretative character of the annotation process. In the present paper, we\naim to elucidate the feasibility of automated allusion detection. We approach\nthe matter from an Information Retrieval perspective in which referencing texts\nact as queries and referenced texts as relevant documents to be retrieved, and\nestimate the difficulty of benchmark corpus compilation by a novel\ninter-annotator agreement study on query segmentation. Furthermore, we\ninvestigate to what extent the integration of lexical semantic information\nderived from distributional models and ontologies can aid retrieving cases of\nallusive reuse. The results show that (i) despite low agreement scores, using\nmanual queries considerably improves retrieval performance with respect to a\nwindowing approach, and that (ii) retrieval performance can be moderately\nboosted with distributional semantics.",
            "journal ref": "NAACL-HLT (LaTeCH-CLfL Workshop) 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.03197v3",
            "date": "2019-05-08 16:33:51+00:00",
            "title": "Unified Language Model Pre-training for Natural Language Understanding and Generation",
            "authors": [
                "Li Dong",
                "Nan Yang",
                "Wenhui Wang",
                "Furu Wei",
                "Xiaodong Liu",
                "Yu Wang",
                "Jianfeng Gao",
                "Ming Zhou",
                "Hsiao-Wuen Hon"
            ],
            "abstract": "This paper presents a new Unified pre-trained Language Model (UniLM) that can\nbe fine-tuned for both natural language understanding and generation tasks. The\nmodel is pre-trained using three types of language modeling tasks:\nunidirectional, bidirectional, and sequence-to-sequence prediction. The unified\nmodeling is achieved by employing a shared Transformer network and utilizing\nspecific self-attention masks to control what context the prediction conditions\non. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0\nand CoQA question answering tasks. Moreover, UniLM achieves new\nstate-of-the-art results on five natural language generation datasets,\nincluding improving the CNN/DailyMail abstractive summarization ROUGE-L to\n40.51 (2.04 absolute improvement), the Gigaword abstractive summarization\nROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question\nanswering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question\ngeneration BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7\ndocument-grounded dialog response generation NIST-4 to 2.67 (human performance\nis 2.65). The code and pre-trained models are available at\nhttps://github.com/microsoft/unilm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.03423v1",
            "date": "2019-05-09 03:01:54+00:00",
            "title": "Targeted Sentiment Analysis: A Data-Driven Categorization",
            "authors": [
                "Jiaxin Pei",
                "Aixin Sun",
                "Chenliang Li"
            ],
            "abstract": "Targeted sentiment analysis (TSA), also known as aspect based sentiment\nanalysis (ABSA), aims at detecting fine-grained sentiment polarity towards\ntargets in a given opinion document. Due to the lack of labeled datasets and\neffective technology, TSA had been intractable for many years. The newly\nreleased datasets and the rapid development of deep learning technologies are\nkey enablers for the recent significant progress made in this area. However,\nthe TSA tasks have been defined in various ways with different understandings\ntowards basic concepts like `target' and `aspect'. In this paper, we categorize\nthe different tasks and highlight the differences in the available datasets and\ntheir specific tasks. We then further discuss the challenges related to data\ncollection and data annotation which are overlooked in many previous studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.03969v2",
            "date": "2019-05-10 07:04:12+00:00",
            "title": "Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network",
            "authors": [
                "Wenmian Yang",
                "Weijia Jia",
                "XIaojie Zhou",
                "Yutao Luo"
            ],
            "abstract": "The Legal Judgment Prediction (LJP) is to determine judgment results based on\nthe fact descriptions of the cases. LJP usually consists of multiple subtasks,\nsuch as applicable law articles prediction, charges prediction, and the term of\nthe penalty prediction. These multiple subtasks have topological dependencies,\nthe results of which affect and verify each other. However, existing methods\nuse dependencies of results among multiple subtasks inefficiently. Moreover,\nfor cases with similar descriptions but different penalties, current methods\ncannot predict accurately because the word collocation information is ignored.\nIn this paper, we propose a Multi-Perspective Bi-Feedback Network with the Word\nCollocation Attention mechanism based on the topology structure among subtasks.\nSpecifically, we design a multi-perspective forward prediction and backward\nverification framework to utilize result dependencies among multiple subtasks\neffectively. To distinguish cases with similar descriptions but different\npenalties, we integrate word collocations features of fact descriptions into\nthe network via an attention mechanism. The experimental results show our model\nachieves significant improvements over baselines on all prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04051v1",
            "date": "2019-05-10 10:14:41+00:00",
            "title": "Restoring Arabic vowels through omission-tolerant dictionary lookup",
            "authors": [
                "Alexis Amid Neme",
                "S\u00e9bastien Paumier"
            ],
            "abstract": "Vowels in Arabic are optional orthographic symbols written as diacritics\nabove or below letters. In Arabic texts, typically more than 97 percent of\nwritten words do not explicitly show any of the vowels they contain; that is to\nsay, depending on the author, genre and field, less than 3 percent of words\ninclude any explicit vowel. Although numerous studies have been published on\nthe issue of restoring the omitted vowels in speech technologies, little\nattention has been given to this problem in papers dedicated to written Arabic\ntechnologies.f In this research, we present Arabic-Unitex, an Arabic Language\nResource, with emphasis on vowel representation and encoding. Specifically, we\npresent two dozens of rules formalizing a detailed description of vowel\nomission in written text. They are typographical rules integrated into\nlarge-coverage resources for morphological annotation. For restoring vowels,\nour resources are capable of identifying words in which the vowels are not\nshown, as well as words in which the vowels are partially or fully included. By\ntaking into account these rules, our resources are able to compute and restore\nfor each word form a list of compatible fully vowelized candidates through\nomission-tolerant dictionary lookup. Our program performs the analysis of 5000\nwords/second for running text (20 pages/second). Based on these comprehensive\nlinguistic resources, we created a spell checker that detects any\ninvalid/misplaced vowel in a fully or partially vowelized form. Finally, our\nresources provide a lexical coverage of more than 99 percent of the words used\nin popular newspapers, and restore vowels in words (out of context) simply and\nefficiently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04562v1",
            "date": "2019-05-11 17:50:12+00:00",
            "title": "Semantic categories of artifacts and animals reflect efficient coding",
            "authors": [
                "Noga Zaslavsky",
                "Terry Regier",
                "Naftali Tishby",
                "Charles Kemp"
            ],
            "abstract": "It has been argued that semantic categories across languages reflect pressure\nfor efficient communication. Recently, this idea has been cast in terms of a\ngeneral information-theoretic principle of efficiency, the Information\nBottleneck (IB) principle, and it has been shown that this principle accounts\nfor the emergence and evolution of named color categories across languages,\nincluding soft structure and patterns of inconsistent naming. However, it is\nnot yet clear to what extent this account generalizes to semantic domains other\nthan color. Here we show that it generalizes to two qualitatively different\nsemantic domains: names for containers, and for animals. First, we show that\ncontainer naming in Dutch and French is near-optimal in the IB sense, and that\nIB broadly accounts for soft categories and inconsistent naming patterns in\nboth languages. Second, we show that a hierarchy of animal categories derived\nfrom IB captures cross-linguistic tendencies in the growth of animal\ntaxonomies. Taken together, these findings suggest that fundamental\ninformation-theoretic principles of efficient coding may shape semantic\ncategories across languages and across domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04727v1",
            "date": "2019-05-12 14:19:28+00:00",
            "title": "A Comparison of Techniques for Sentiment Classification of Film Reviews",
            "authors": [
                "Milan Gritta"
            ],
            "abstract": "We undertake the task of comparing lexicon-based sentiment classification of\nfilm reviews with machine learning approaches. We look at existing\nmethodologies and attempt to emulate and improve on them using a 'given'\nlexicon and a bag-of-words approach. We also utilise syntactical information\nsuch as part-of-speech and dependency relations. We will show that a simple\nlexicon-based classification achieves good results however machine learning\ntechniques prove to be the superior tool. We also show that more features do\nnot necessarily deliver better performance as well as elaborate on three\nfurther enhancements not tested in this article.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05293v1",
            "date": "2019-05-13 21:36:43+00:00",
            "title": "Towards Content Transfer through Grounded Text Generation",
            "authors": [
                "Shrimai Prabhumoye",
                "Chris Quirk",
                "Michel Galley"
            ],
            "abstract": "Recent work in neural generation has attracted significant interest in\ncontrolling the form of text, such as style, persona, and politeness. However,\nthere has been less work on controlling neural text generation for content.\nThis paper introduces the notion of Content Transfer for long-form text\ngeneration, where the task is to generate a next sentence in a document that\nboth fits its context and is grounded in a content-rich external textual source\nsuch as a news story. Our experiments on Wikipedia data show significant\nimprovements against competitive baselines. As another contribution of this\npaper, we release a benchmark dataset of 640k Wikipedia referenced sentences\npaired with the source articles to encourage exploration of this new task.",
            "journal ref": "Proc. NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05395v3",
            "date": "2019-05-14 05:20:41+00:00",
            "title": "A Brief Survey of Multilingual Neural Machine Translation",
            "authors": [
                "Raj Dabre",
                "Chenhui Chu",
                "Anoop Kunchukuttan"
            ],
            "abstract": "We present a survey on multilingual neural machine translation (MNMT), which\nhas gained a lot of traction in the recent years. MNMT has been useful in\nimproving translation quality as a result of knowledge transfer. MNMT is more\npromising and interesting than its statistical machine translation counterpart\nbecause end-to-end modeling and distributed representations open new avenues.\nMany approaches have been proposed in order to exploit multilingual parallel\ncorpora for improving translation quality. However, the lack of a comprehensive\nsurvey makes it difficult to determine which approaches are promising and hence\ndeserve further exploration. In this paper, we present an in-depth survey of\nexisting literature on MNMT. We categorize various approaches based on the\nresource scenarios as well as underlying modeling principles. We hope this\npaper will serve as a starting point for researchers and engineers interested\nin MNMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05407v1",
            "date": "2019-05-14 06:18:35+00:00",
            "title": "On the number of k-skip-n-grams",
            "authors": [
                "Dmytro Krasnoshtan"
            ],
            "abstract": "The paper proves that the number of k-skip-n-grams for a corpus of size $L$\nis $$\\frac{Ln + n + k' - n^2 - nk'}{n} \\cdot \\binom{n-1+k'}{n-1}$$ where $k' =\n\\min(L - n + 1, k)$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05460v2",
            "date": "2019-05-14 08:55:04+00:00",
            "title": "Cognitive Graph for Multi-Hop Reading Comprehension at Scale",
            "authors": [
                "Ming Ding",
                "Chang Zhou",
                "Qibin Chen",
                "Hongxia Yang",
                "Jie Tang"
            ],
            "abstract": "We propose a new CogQA framework for multi-hop question answering in\nweb-scale documents. Inspired by the dual process theory in cognitive science,\nthe framework gradually builds a \\textit{cognitive graph} in an iterative\nprocess by coordinating an implicit extraction module (System 1) and an\nexplicit reasoning module (System 2). While giving accurate answers, our\nframework further provides explainable reasoning paths. Specifically, our\nimplementation based on BERT and graph neural network efficiently handles\nmillions of documents for multi-hop reasoning questions in the HotpotQA\nfullwiki dataset, achieving a winning joint $F_1$ score of 34.9 on the\nleaderboard, compared to 23.6 of the best competitor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05513v2",
            "date": "2019-05-14 10:51:55+00:00",
            "title": "Deep Residual Output Layers for Neural Language Generation",
            "authors": [
                "Nikolaos Pappas",
                "James Henderson"
            ],
            "abstract": "Many tasks, including language generation, benefit from learning the\nstructure of the output space, particularly when the space of output labels is\nlarge and the data is sparse. State-of-the-art neural language models\nindirectly capture the output space structure in their classifier weights since\nthey lack parameter sharing across output labels. Learning shared output label\nmappings helps, but existing methods have limited expressivity and are prone to\noverfitting. In this paper, we investigate the usefulness of more powerful\nshared mappings for output labels, and propose a deep residual output mapping\nwith dropout between layers to better capture the structure of the output space\nand avoid overfitting. Evaluations on three language generation tasks show that\nour output label mapping can match or improve state-of-the-art recurrent and\nself-attention architectures, and suggest that the classifier does not\nnecessarily need to be high-rank to better model natural language if it is\nbetter at capturing the structure of the output space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05529v4",
            "date": "2019-05-14 11:44:33+00:00",
            "title": "Entity-Relation Extraction as Multi-Turn Question Answering",
            "authors": [
                "Xiaoya Li",
                "Fan Yin",
                "Zijun Sun",
                "Xiayu Li",
                "Arianna Yuan",
                "Duo Chai",
                "Mingxin Zhou",
                "Jiwei Li"
            ],
            "abstract": "In this paper, we propose a new paradigm for the task of entity-relation\nextraction. We cast the task as a multi-turn question answering problem, i.e.,\nthe extraction of entities and relations is transformed to the task of\nidentifying answer spans from the context. This multi-turn QA formalization\ncomes with several key advantages: firstly, the question query encodes\nimportant information for the entity/relation class we want to identify;\nsecondly, QA provides a natural way of jointly modeling entity and relation;\nand thirdly, it allows us to exploit the well developed machine reading\ncomprehension (MRC) models. Experiments on the ACE and the CoNLL04 corpora\ndemonstrate that the proposed paradigm significantly outperforms previous best\nmodels. We are able to obtain the state-of-the-art results on all of the ACE04,\nACE05 and CoNLL04 datasets, increasing the SOTA results on the three datasets\nto 49.4 (+1.0), 60.2 (+0.6) and 68.9 (+2.1), respectively. Additionally, we\nconstruct a newly developed dataset RESUME in Chinese, which requires\nmulti-step reasoning to construct entity dependencies, as opposed to the\nsingle-step dependency extraction in the triplet exaction in previous datasets.\nThe proposed multi-turn QA model also achieves the best performance on the\nRESUME dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05538v1",
            "date": "2019-05-14 12:07:13+00:00",
            "title": "Assessing the Difficulty of Classifying ConceptNet Relations in a Multi-Label Classification Setting",
            "authors": [
                "Maria Becker",
                "Michael Staniek",
                "Vivi Nastase",
                "Anette Frank"
            ],
            "abstract": "Commonsense knowledge relations are crucial for advanced NLU tasks. We\nexamine the learnability of such relations as represented in CONCEPTNET, taking\ninto account their specific properties, which can make relation classification\ndifficult: a given concept pair can be linked by multiple relation types, and\nrelations can have multi-word arguments of diverse semantic types. We explore a\nneural open world multi-label classification approach that focuses on the\nevaluation of classification accuracy for individual relations. Based on an\nin-depth study of the specific properties of the CONCEPTNET resource, we\ninvestigate the impact of different relation representations and model\nvariations. Our analysis reveals that the complexity of argument types and\nrelation ambiguity are the most important challenges to address. We design a\ncustomized evaluation method to address the incompleteness of the resource that\ncan be expanded in future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05543v2",
            "date": "2019-05-14 12:14:27+00:00",
            "title": "The Language of Legal and Illegal Activity on the Darknet",
            "authors": [
                "Leshem Choshen",
                "Dan Eldad",
                "Daniel Hershcovich",
                "Elior Sulem",
                "Omri Abend"
            ],
            "abstract": "The non-indexed parts of the Internet (the Darknet) have become a haven for\nboth legal and illegal anonymous activity. Given the magnitude of these\nnetworks, scalably monitoring their activity necessarily relies on automated\ntools, and notably on NLP tools. However, little is known about what\ncharacteristics texts communicated through the Darknet have, and how well\noff-the-shelf NLP tools do on this domain. This paper tackles this gap and\nperforms an in-depth investigation of the characteristics of legal and illegal\ntext in the Darknet, comparing it to a clear net website with similar content\nas a control condition. Taking drug-related websites as a test case, we find\nthat texts for selling legal and illegal drugs have several linguistic\ncharacteristics that distinguish them from one another, as well as from the\ncontrol condition, among them the distribution of POS tags, and the coverage of\ntheir named entities in Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05550v2",
            "date": "2019-05-14 12:30:53+00:00",
            "title": "A Dynamic Evolutionary Framework for Timeline Generation based on Distributed Representations",
            "authors": [
                "Dongyun Liang",
                "Guohua Wang",
                "Jing Nie"
            ],
            "abstract": "Given the collection of timestamped web documents related to the evolving\ntopic, timeline summarization (TS) highlights its most important events in the\nform of relevant summaries to represent the development of a topic over time.\nMost of the previous work focuses on fully-observable ranking models and\ndepends on hand-designed features or complex mechanisms that may not generalize\nwell. We present a novel dynamic framework for evolutionary timeline generation\nleveraging distributed representations, which dynamically finds the most likely\nsequence of evolutionary summaries in the timeline, called the Viterbi\ntimeline, and reduces the impact of events that irrelevant or repeated to the\ntopic. The assumptions of the coherence and the global view run through our\nmodel. We explore adjacent relevance to constrain timeline coherence and make\nsure the events evolve on the same topic with a global view. Experimental\nresults demonstrate that our framework is feasible to extract summaries for\ntimeline generation, outperforms various competitive baselines, and achieves\nthe state-of-the-art performance as an unsupervised approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05583v3",
            "date": "2019-05-14 13:17:26+00:00",
            "title": "How to Fine-Tune BERT for Text Classification?",
            "authors": [
                "Chi Sun",
                "Xipeng Qiu",
                "Yige Xu",
                "Xuanjing Huang"
            ],
            "abstract": "Language model pre-training has proven to be useful in learning universal\nlanguage representations. As a state-of-the-art language model pre-training\nmodel, BERT (Bidirectional Encoder Representations from Transformers) has\nachieved amazing results in many language understanding tasks. In this paper,\nwe conduct exhaustive experiments to investigate different fine-tuning methods\nof BERT on text classification task and provide a general solution for BERT\nfine-tuning. Finally, the proposed solution obtains new state-of-the-art\nresults on eight widely-studied text classification datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05621v3",
            "date": "2019-05-14 14:03:21+00:00",
            "title": "Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation",
            "authors": [
                "Ning Dai",
                "Jianze Liang",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Disentangling the content and style in the latent space is prevalent in\nunpaired text style transfer. However, two major issues exist in most of the\ncurrent neural models. 1) It is difficult to completely strip the style\ninformation from the semantics for a sentence. 2) The recurrent neural network\n(RNN) based encoder and decoder, mediated by the latent representation, cannot\nwell deal with the issue of the long-term dependency, resulting in poor\npreservation of non-stylistic semantic content. In this paper, we propose the\nStyle Transformer, which makes no assumption about the latent representation of\nsource sentence and equips the power of attention mechanism in Transformer to\nachieve better style transfer and better content preservation.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05677v3",
            "date": "2019-05-14 15:39:58+00:00",
            "title": "Sense Vocabulary Compression through the Semantic Knowledge of WordNet for Neural Word Sense Disambiguation",
            "authors": [
                "Lo\u00efc Vial",
                "Benjamin Lecouteux",
                "Didier Schwab"
            ],
            "abstract": "In this article, we tackle the issue of the limited quantity of manually\nsense annotated corpora for the task of word sense disambiguation, by\nexploiting the semantic relationships between senses such as synonymy,\nhypernymy and hyponymy, in order to compress the sense vocabulary of Princeton\nWordNet, and thus reduce the number of different sense tags that must be\nobserved to disambiguate all words of the lexical database. We propose two\ndifferent methods that greatly reduces the size of neural WSD models, with the\nbenefit of improving their coverage without additional training data, and\nwithout impacting their precision. In addition to our method, we present a WSD\nsystem which relies on pre-trained BERT word vectors in order to achieve\nresults that significantly outperform the state of the art on all WSD\nevaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05698v2",
            "date": "2019-05-07 08:36:27+00:00",
            "title": "SuperChat: Dialogue Generation by Transfer Learning from Vision to Language using Two-dimensional Word Embedding and Pretrained ImageNet CNN Models",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Michael Lin",
                "Charles Young",
                "Jason Dong",
                "Wenhan Zhang",
                "Patrick Dong"
            ],
            "abstract": "The recent work of Super Characters method using two-dimensional word\nembedding achieved state-of-the-art results in text classification tasks,\nshowcasing the promise of this new approach. This paper borrows the idea of\nSuper Characters method and two-dimensional embedding, and proposes a method of\ngenerating conversational response for open domain dialogues. The experimental\nresults on a public dataset shows that the proposed SuperChat method generates\nhigh quality responses. An interactive demo is ready to show at the workshop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05812v1",
            "date": "2019-05-14 19:42:43+00:00",
            "title": "Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis",
            "authors": [
                "Md Shad Akhtar",
                "Dushyant Singh Chauhan",
                "Deepanway Ghosal",
                "Soujanya Poria",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Related tasks often have inter-dependence on each other and perform better\nwhen solved in a joint framework. In this paper, we present a deep multi-task\nlearning framework that jointly performs sentiment and emotion analysis both.\nThe multi-modal inputs (i.e., text, acoustic and visual frames) of a video\nconvey diverse and distinctive information, and usually do not have equal\ncontribution in the decision making. We propose a context-level inter-modal\nattention framework for simultaneously predicting the sentiment and expressed\nemotions of an utterance. We evaluate our proposed approach on CMU-MOSEI\ndataset for multi-modal sentiment and emotion analysis. Evaluation results\nsuggest that multi-task learning framework offers improvement over the\nsingle-task framework. The proposed approach reports new state-of-the-art\nperformance for both sentiment analysis and emotion analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05816v1",
            "date": "2019-05-14 20:03:54+00:00",
            "title": "Curriculum Learning for Domain Adaptation in Neural Machine Translation",
            "authors": [
                "Xuan Zhang",
                "Pamela Shapiro",
                "Gaurav Kumar",
                "Paul McNamee",
                "Marine Carpuat",
                "Kevin Duh"
            ],
            "abstract": "We introduce a curriculum learning approach to adapt generic neural machine\ntranslation models to a specific domain. Samples are grouped by their\nsimilarities to the domain of interest and each group is fed to the training\nalgorithm with a particular schedule. This approach is simple to implement on\ntop of any neural framework or architecture, and consistently outperforms both\nunadapted and adapted baselines in experiments with two distinct domains and\ntwo language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05877v1",
            "date": "2019-05-14 23:11:46+00:00",
            "title": "Extraction and Analysis of Clinically Important Follow-up Recommendations in a Large Radiology Dataset",
            "authors": [
                "Wilson Lau",
                "Thomas H Payne",
                "Ozlem Uzuner",
                "Meliha Yetisgen"
            ],
            "abstract": "Communication of follow-up recommendations when abnormalities are identified\non imaging studies is prone to error. In this paper, we present a natural\nlanguage processing approach based on deep learning to automatically identify\nclinically important recommendations in radiology reports. Our approach first\nidentifies the recommendation sentences and then extracts reason, test, and\ntime frame of the identified recommendations. To train our extraction models,\nwe created a corpus of 567 radiology reports annotated for recommendation\ninformation. Our extraction models achieved 0.92 f-score for recommendation\nsentence, 0.65 f-score for reason, 0.73 f-score for test, and 0.84 f-score for\ntime frame. We applied the extraction models to a set of over 3.3 million\nradiology reports and analyzed the adherence of follow-up recommendations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05950v2",
            "date": "2019-05-15 05:47:23+00:00",
            "title": "BERT Rediscovers the Classical NLP Pipeline",
            "authors": [
                "Ian Tenney",
                "Dipanjan Das",
                "Ellie Pavlick"
            ],
            "abstract": "Pre-trained text encoders have rapidly advanced the state of the art on many\nNLP tasks. We focus on one such model, BERT, and aim to quantify where\nlinguistic information is captured within the network. We find that the model\nrepresents the steps of the traditional NLP pipeline in an interpretable and\nlocalizable way, and that the regions responsible for each step appear in the\nexpected sequence: POS tagging, parsing, NER, semantic roles, then coreference.\nQualitative analysis reveals that the model can and often does adjust this\npipeline dynamically, revising lower-level decisions on the basis of\ndisambiguating information from higher-level representations.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics (2019) 4593-4601",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05979v2",
            "date": "2019-05-15 07:05:46+00:00",
            "title": "When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion",
            "authors": [
                "Elena Voita",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "Though machine translation errors caused by the lack of context beyond one\nsentence have long been acknowledged, the development of context-aware NMT\nsystems is hampered by several problems. Firstly, standard metrics are not\nsensitive to improvements in consistency in document-level translations.\nSecondly, previous work on context-aware NMT assumed that the sentence-aligned\nparallel data consisted of complete documents while in most practical scenarios\nsuch document-level data constitutes only a fraction of the available parallel\ndata. To address the first issue, we perform a human study on an\nEnglish-Russian subtitles dataset and identify deixis, ellipsis and lexical\ncohesion as three main sources of inconsistency. We then create test sets\ntargeting these phenomena. To address the second shortcoming, we consider a\nset-up in which a much larger amount of sentence-level data is available\ncompared to that aligned at the document level. We introduce a model that is\nsuitable for this scenario and demonstrate major gains over a context-agnostic\nbaseline on our new benchmarks without sacrificing performance as measured with\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06196v4",
            "date": "2019-05-15 14:04:47+00:00",
            "title": "Dual Supervised Learning for Natural Language Understanding and Generation",
            "authors": [
                "Shang-Yu Su",
                "Chao-Wei Huang",
                "Yun-Nung Chen"
            ],
            "abstract": "Natural language understanding (NLU) and natural language generation (NLG)\nare both critical research topics in the NLP field. Natural language\nunderstanding is to extract the core semantic meaning from the given\nutterances, while natural language generation is opposite, of which the goal is\nto construct corresponding sentences based on the given semantics. However,\nsuch dual relationship has not been investigated in the literature. This paper\nproposes a new learning framework for language understanding and generation on\ntop of dual supervised learning, providing a way to exploit the duality. The\npreliminary experiments show that the proposed approach boosts the performance\nfor both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06241v2",
            "date": "2019-05-15 15:22:01+00:00",
            "title": "Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing",
            "authors": [
                "Ben Bogin",
                "Matt Gardner",
                "Jonathan Berant"
            ],
            "abstract": "Research on parsing language to SQL has largely ignored the structure of the\ndatabase (DB) schema, either because the DB was very simple, or because it was\nobserved at both training and test time. In Spider, a recently-released\ntext-to-SQL dataset, new and complex DBs are given at test time, and so the\nstructure of the DB schema can inform the predicted SQL query. In this paper,\nwe present an encoder-decoder semantic parser, where the structure of the DB\nschema is encoded with a graph neural network, and this representation is later\nused at both encoding and decoding time. Evaluation shows that encoding the\nschema structure improves our parser accuracy from 33.8% to 39.4%, dramatically\nabove the current state of the art, which is at 19.7%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06290v2",
            "date": "2019-05-15 16:47:11+00:00",
            "title": "A Surprisingly Robust Trick for Winograd Schema Challenge",
            "authors": [
                "Vid Kocijan",
                "Ana-Maria Cretu",
                "Oana-Maria Camburu",
                "Yordan Yordanov",
                "Thomas Lukasiewicz"
            ],
            "abstract": "The Winograd Schema Challenge (WSC) dataset WSC273 and its inference\ncounterpart WNLI are popular benchmarks for natural language understanding and\ncommonsense reasoning. In this paper, we show that the performance of three\nlanguage models on WSC273 strongly improves when fine-tuned on a similar\npronoun disambiguation problem dataset (denoted WSCR). We additionally generate\na large unsupervised WSC-like dataset. By fine-tuning the BERT language model\nboth on the introduced and on the WSCR dataset, we achieve overall accuracies\nof 72.5% and 74.7% on WSC273 and WNLI, improving the previous state-of-the-art\nsolutions by 8.8% and 9.6%, respectively. Furthermore, our fine-tuned models\nare also consistently more robust on the \"complex\" subsets of WSC273,\nintroduced by Trichelair et al. (2018).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06316v1",
            "date": "2019-05-15 17:48:56+00:00",
            "title": "What do you learn from context? Probing for sentence structure in contextualized word representations",
            "authors": [
                "Ian Tenney",
                "Patrick Xia",
                "Berlin Chen",
                "Alex Wang",
                "Adam Poliak",
                "R Thomas McCoy",
                "Najoung Kim",
                "Benjamin Van Durme",
                "Samuel R. Bowman",
                "Dipanjan Das",
                "Ellie Pavlick"
            ],
            "abstract": "Contextualized representation models such as ELMo (Peters et al., 2018a) and\nBERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a\ndiverse array of downstream NLP tasks. Building on recent token-level probing\nwork, we introduce a novel edge probing task design and construct a broad suite\nof sub-sentence tasks derived from the traditional structured NLP pipeline. We\nprobe word-level contextual representations from four recent models and\ninvestigate how they encode sentence structure across a range of syntactic,\nsemantic, local, and long-range phenomena. We find that existing models trained\non language modeling and translation produce strong representations for\nsyntactic phenomena, but only offer comparably small improvements on semantic\ntasks over a non-contextual baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06319v2",
            "date": "2019-05-15 17:51:09+00:00",
            "title": "Exact Hard Monotonic Attention for Character-Level Transduction",
            "authors": [
                "Shijie Wu",
                "Ryan Cotterell"
            ],
            "abstract": "Many common character-level, string-to-string transduction tasks, e.g.\ngraphemeto-phoneme conversion and morphological inflection, consist almost\nexclusively of monotonic transduction. Neural sequence-to-sequence models with\nsoft attention, which are non-monotonic, often outperform popular monotonic\nmodels. In this work, we ask the following question: Is monotonicity really a\nhelpful inductive bias in these tasks? We develop a hard attention\nsequence-to-sequence model that enforces strict monotonicity and learns a\nlatent alignment jointly while learning to transduce. With the help of dynamic\nprogramming, we are able to compute the exact marginalization over all\nmonotonic alignments. Our models achieve state-of-the-art performance on\nmorphological inflection. Furthermore, we find strong performance on two other\ncharacter-level transduction tasks. Code is available at\nhttps://github.com/shijie-wu/neural-transducer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06512v1",
            "date": "2019-05-16 03:38:14+00:00",
            "title": "Incorporating Sememes into Chinese Definition Modeling",
            "authors": [
                "Liner Yang",
                "Cunliang Kong",
                "Yun Chen",
                "Yang Liu",
                "Qinan Fan",
                "Erhong Yang"
            ],
            "abstract": "Chinese definition modeling is a challenging task that generates a dictionary\ndefinition in Chinese for a given Chinese word. To accomplish this task, we\nconstruct the Chinese Definition Modeling Corpus (CDM), which contains triples\nof word, sememes and the corresponding definition. We present two novel models\nto improve Chinese definition modeling: the Adaptive-Attention model (AAM) and\nthe Self- and Adaptive-Attention Model (SAAM). AAM successfully incorporates\nsememes for generating the definition with an adaptive attention mechanism. It\nhas the capability to decide which sememes to focus on and when to pay\nattention to sememes. SAAM further replaces recurrent connections in AAM with\nself-attention and relies entirely on the attention mechanism, reducing the\npath length between word, sememes and definition. Experiments on CDM\ndemonstrate that by incorporating sememes, our best proposed model can\noutperform the state-of-the-art method by +6.0 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06643v1",
            "date": "2019-05-16 10:27:17+00:00",
            "title": "Machine Learning based English Sentiment Analysis",
            "authors": [
                "T. N. T. Tran",
                "L. K. N. Nguyen",
                "V. M. Ngo"
            ],
            "abstract": "Sentiment analysis or opinion mining aims to determine attitudes, judgments\nand opinions of customers for a product or a service. This is a great system to\nhelp manufacturers or servicers know the satisfaction level of customers about\ntheir products or services. From that, they can have appropriate adjustments.\nWe use a popular machine learning method, being Support Vector Machine, combine\nwith the library in Waikato Environment for Knowledge Analysis (WEKA) to build\nJava web program which analyzes the sentiment of English comments belongs one\nin four types of woman products. That are dresses, handbags, shoes and rings.\nWe have developed and test our system with a training set having 300 comments\nand a test set having 400 comments. The experimental results of the system\nabout precision, recall and F measures for positive comments are 89.3%, 95.0%\nand 92,.1%; for negative comments are 97.1%, 78.5% and 86.8%; and for neutral\ncomments are 76.7%, 86.2% and 81.2%.",
            "journal ref": "Journal of Science and Technology, Vietnam Academy of Science and\n  Technology, Vol. 52, No. 4D, pp. 142-155 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06647v1",
            "date": "2019-05-16 10:31:48+00:00",
            "title": "Using Entity Relations for Opinion Mining of Vietnamese Comments",
            "authors": [
                "P. T. Nguyen",
                "L. T. Le",
                "V. M. Ngo",
                "P. M. Nguyen"
            ],
            "abstract": "In this paper, we propose several novel techniques to extract and mining\nopinions of Vietnamese reviews of customers about a number of products traded\non e-commerce in Vietnam. The assessment is based on the emotional level of\ncustomers on a specific product such as mobile and laptop. We exploit the\nfeatures of the products because they are much interested by customers and have\nmany products in the Vietnam e-commerce market. Thence, it can be known the\nfavorites and dislikes of customers about exploited products.",
            "journal ref": "Journal of Science and Technology, Vietnam Academy of Science and\n  Technology, Vol. 52, No. 4D, pp. 120-132 (2014)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06649v1",
            "date": "2019-05-16 10:43:58+00:00",
            "title": "What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue",
            "authors": [
                "Laura Aina",
                "Carina Silberer",
                "Matthijs Westera",
                "Ionut-Teodor Sorodoc",
                "Gemma Boleda"
            ],
            "abstract": "Humans use language to refer to entities in the external world. Motivated by\nthis, in recent years several models that incorporate a bias towards learning\nentity representations have been proposed. Such entity-centric models have\nshown empirical success, but we still know little about why. In this paper we\nanalyze the behavior of two recently proposed entity-centric models in a\nreferential task, Entity Linking in Multi-party Dialogue (SemEval 2018 Task 4).\nWe show that these models outperform the state of the art on this task, and\nthat they do better on lower frequency entities than a counterpart model that\nis not entity-centric, with the same model size. We argue that making models\nentity-centric naturally fosters good architectural decisions. However, we also\nshow that these models do not really build entity representations and that they\nmake poor use of linguistic context. These negative results underscore the need\nfor model analysis, to test whether the motivations for particular\narchitectures are borne out in how models behave when deployed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06831v2",
            "date": "2019-05-15 13:52:53+00:00",
            "title": "Towards Interlingua Neural Machine Translation",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Common intermediate language representation in neural machine translation can\nbe used to extend bilingual to multilingual systems by incremental training. In\nthis paper, we propose a new architecture based on introducing an interlingual\nloss as an additional training objective. By adding and forcing this\ninterlingual loss, we are able to train multiple encoders and decoders for each\nlanguage, sharing a common intermediate representation. Translation results on\nthe low-resourced tasks (Turkish-English and Kazakh-English tasks, from the\npopular Workshop on Machine Translation benchmark) show the following BLEU\nimprovements up to 2.8. However, results on a larger dataset (Russian-English\nand Kazakh-English, from the same baselines) show BLEU loses if the same\namount. While our system is only providing improvements for the low-resourced\ntasks in terms of translation quality, our system is capable of quickly\ndeploying new language pairs without retraining the rest of the system, which\nmay be a game-changer in some situations (i.e. in a disaster crisis where\ninternational help is required towards a small region or to develop some\ntranslation system for a client). Precisely, what is most relevant from our\narchitecture is that it is capable of: (1) reducing the number of production\nsystems, with respect to the number of languages, from quadratic to linear (2)\nincrementally adding a new language in the system without retraining languages\npreviously there and (3) allowing for translations from the new language to all\nthe others present in the system",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06837v2",
            "date": "2019-05-16 15:27:19+00:00",
            "title": "Tracing cultural diachronic semantic shifts in Russian using word embeddings: test sets and baselines",
            "authors": [
                "Vadim Fomin",
                "Daria Bakshandaeva",
                "Julia Rodina",
                "Andrey Kutuzov"
            ],
            "abstract": "The paper introduces manually annotated test sets for the task of tracing\ndiachronic (temporal) semantic shifts in Russian. The two test sets are\ncomplementary in that the first one covers comparatively strong semantic\nchanges occurring to nouns and adjectives from pre-Soviet to Soviet times,\nwhile the second one covers comparatively subtle socially and culturally\ndetermined shifts occurring in years from 2000 to 2014. Additionally, the\nsecond test set offers more granular classification of shifts degree, but is\nlimited to only adjectives.\n  The introduction of the test sets allowed us to evaluate several\nwell-established algorithms of semantic shifts detection (posing this as a\nclassification problem), most of which have never been tested on Russian\nmaterial. All of these algorithms use distributional word embedding models\ntrained on the corresponding in-domain corpora. The resulting scores provide\nsolid comparison baselines for future studies tackling similar tasks. We\npublish the datasets, code and the trained models in order to facilitate\nfurther research in automatically detecting temporal semantic shifts for\nRussian words, with time periods of different granularities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06906v1",
            "date": "2019-05-16 16:57:35+00:00",
            "title": "Gated Convolutional Neural Networks for Domain Adaptation",
            "authors": [
                "Avinash Madasu",
                "Vijjini Anvesh Rao"
            ],
            "abstract": "Domain Adaptation explores the idea of how to maximize performance on a\ntarget domain, distinct from source domain, upon which the classifier was\ntrained. This idea has been explored for the task of sentiment analysis\nextensively. The training of reviews pertaining to one domain and evaluation on\nanother domain is widely studied for modeling a domain independent algorithm.\nThis further helps in understanding correlation between domains. In this paper,\nwe show that Gated Convolutional Neural Networks (GCN) perform effectively at\nlearning sentiment analysis in a manner where domain dependant knowledge is\nfiltered out using its gates. We perform our experiments on multiple gate\narchitectures: Gated Tanh ReLU Unit (GTRU), Gated Tanh Unit (GTU) and Gated\nLinear Unit (GLU). Extensive experimentation on two standard datasets relevant\nto the task, reveal that training with Gated Convolutional Neural Networks give\nsignificantly better performance on target domains than regular convolution and\nrecurrent based architectures. While complex architectures like attention,\nfilter domain specific knowledge as well, their complexity order is remarkably\nhigh as compared to gated architectures. GCNs rely on convolution hence gaining\nan upper hand through parallelization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06933v3",
            "date": "2019-05-16 17:51:44+00:00",
            "title": "Dynamically Fused Graph Network for Multi-hop Reasoning",
            "authors": [
                "Yunxuan Xiao",
                "Yanru Qu",
                "Lin Qiu",
                "Hao Zhou",
                "Lei Li",
                "Weinan Zhang",
                "Yong Yu"
            ],
            "abstract": "Text-based question answering (TBQA) has been studied extensively in recent\nyears. Most existing approaches focus on finding the answer to a question\nwithin a single paragraph. However, many difficult questions require multiple\nsupporting evidence from scattered text among two or more documents. In this\npaper, we propose Dynamically Fused Graph Network(DFGN), a novel method to\nanswer those questions requiring multiple scattered evidence and reasoning over\nthem. Inspired by human's step-by-step reasoning behavior, DFGN includes a\ndynamic fusion layer that starts from the entities mentioned in the given\nquery, explores along the entity graph dynamically built from the text, and\ngradually finds relevant supporting entities from the given documents. We\nevaluate DFGN on HotpotQA, a public TBQA dataset requiring multi-hop reasoning.\nDFGN achieves competitive results on the public board. Furthermore, our\nanalysis shows DFGN produces interpretable reasoning chains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07000v1",
            "date": "2019-05-16 19:13:44+00:00",
            "title": "IMHO Fine-Tuning Improves Claim Detection",
            "authors": [
                "Tuhin Chakrabarty",
                "Christopher Hidey",
                "Kathleen McKeown"
            ],
            "abstract": "Claims are the central component of an argument. Detecting claims across\ndifferent domains or data sets can often be challenging due to their varying\nconceptualization. We propose to alleviate this problem by fine tuning a\nlanguage model using a Reddit corpus of 5.5 million opinionated claims. These\nclaims are self-labeled by their authors using the internet acronyms IMO/IMHO\n(in my (humble) opinion). Empirical results show that using this approach\nimproves the state of art performance across four benchmark argumentation data\nsets by an average of 4 absolute F1 points in claim detection. As these data\nsets include diverse domains such as social media and student essays this\nimprovement demonstrates the robustness of fine-tuning on this novel corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07002v2",
            "date": "2019-05-16 19:14:18+00:00",
            "title": "Towards Automatic Generation of Shareable Synthetic Clinical Notes Using Neural Language Models",
            "authors": [
                "Oren Melamud",
                "Chaitanya Shivade"
            ],
            "abstract": "Large-scale clinical data is invaluable to driving many computational\nscientific advances today. However, understandable concerns regarding patient\nprivacy hinder the open dissemination of such data and give rise to suboptimal\nsiloed research. De-identification methods attempt to address these concerns\nbut were shown to be susceptible to adversarial attacks. In this work, we focus\non the vast amounts of unstructured natural language data stored in clinical\nnotes and propose to automatically generate synthetic clinical notes that are\nmore amenable to sharing using generative models trained on real de-identified\nrecords. To evaluate the merit of such notes, we measure both their privacy\npreservation properties as well as utility in training clinical NLP models.\nExperiments using neural language models yield notes whose utility is close to\nthat of the real ones in some clinical NLP tasks, yet leave ample room for\nfuture improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07129v3",
            "date": "2019-05-17 06:24:16+00:00",
            "title": "ERNIE: Enhanced Language Representation with Informative Entities",
            "authors": [
                "Zhengyan Zhang",
                "Xu Han",
                "Zhiyuan Liu",
                "Xin Jiang",
                "Maosong Sun",
                "Qun Liu"
            ],
            "abstract": "Neural language representation models such as BERT pre-trained on large-scale\ncorpora can well capture rich semantic patterns from plain text, and be\nfine-tuned to consistently improve the performance of various NLP tasks.\nHowever, the existing pre-trained language models rarely consider incorporating\nknowledge graphs (KGs), which can provide rich structured knowledge facts for\nbetter language understanding. We argue that informative entities in KGs can\nenhance language representation with external knowledge. In this paper, we\nutilize both large-scale textual corpora and KGs to train an enhanced language\nrepresentation model (ERNIE), which can take full advantage of lexical,\nsyntactic, and knowledge information simultaneously. The experimental results\nhave demonstrated that ERNIE achieves significant improvements on various\nknowledge-driven tasks, and meanwhile is comparable with the state-of-the-art\nmodel BERT on other common NLP tasks. The source code of this paper can be\nobtained from https://github.com/thunlp/ERNIE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07213v1",
            "date": "2019-05-17 11:39:21+00:00",
            "title": "Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language",
            "authors": [
                "Yuri Kuratov",
                "Mikhail Arkhipov"
            ],
            "abstract": "The paper introduces methods of adaptation of multilingual masked language\nmodels for a specific language. Pre-trained bidirectional language models show\nstate-of-the-art performance on a wide range of tasks including reading\ncomprehension, natural language inference, and sentiment analysis. At the\nmoment there are two alternative approaches to train such models: monolingual\nand multilingual. While language specific models show superior performance,\nmultilingual models allow to perform a transfer from one language to another\nand solve tasks for different languages simultaneously. This work shows that\ntransfer learning from a multilingual model to monolingual model results in\nsignificant growth of performance on such tasks as reading comprehension,\nparaphrase detection, and sentiment analysis. Furthermore, multilingual\ninitialization of monolingual model substantially reduces training time.\nPre-trained models for the Russian language are open sourced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07321v1",
            "date": "2019-05-17 15:19:17+00:00",
            "title": "Availability-Based Production Predicts Speakers' Real-time Choices of Mandarin Classifiers",
            "authors": [
                "Meilin Zhan",
                "Roger Levy"
            ],
            "abstract": "Speakers often face choices as to how to structure their intended message\ninto an utterance. Here we investigate the influence of contextual\npredictability on the encoding of linguistic content manifested by speaker\nchoice in a classifier language. In English, a numeral modifies a noun directly\n(e.g., three computers). In classifier languages such as Mandarin Chinese, it\nis obligatory to use a classifier (CL) with the numeral and the noun (e.g.,\nthree CL.machinery computer, three CL.general computer). While different nouns\nare compatible with different specific classifiers, there is a general\nclassifier \"ge\" (CL.general) that can be used with most nouns. When the\nupcoming noun is less predictable, the use of a more specific classifier would\nreduce surprisal at the noun thus potentially facilitate comprehension\n(predicted by Uniform Information Density, Levy & Jaeger, 2007), but the use of\nthat more specific classifier may be dispreferred from a production standpoint\nif accessing the general classifier is always available (predicted by\nAvailability-Based Production; Bock, 1987; Ferreira & Dell, 2000). Here we use\na picture-naming experiment showing that Availability-Based Production predicts\nspeakers' real-time choices of Mandarin classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07374v2",
            "date": "2019-05-17 17:03:11+00:00",
            "title": "Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs",
            "authors": [
                "Ming Tu",
                "Guangtao Wang",
                "Jing Huang",
                "Yun Tang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Multi-hop reading comprehension (RC) across documents poses new challenge\nover single-document RC because it requires reasoning over multiple documents\nto reach the final answer. In this paper, we propose a new model to tackle the\nmulti-hop RC problem. We introduce a heterogeneous graph with different types\nof nodes and edges, which is named as Heterogeneous Document-Entity (HDE)\ngraph. The advantage of HDE graph is that it contains different granularity\nlevels of information including candidates, documents and entities in specific\ndocument contexts. Our proposed model can do reasoning over the HDE graph with\nnodes representation initialized with co-attention and self-attention based\ncontext encoders. We employ Graph Neural Networks (GNN) based message passing\nalgorithms to accumulate evidences on the proposed HDE graph. Evaluated on the\nblind test set of the Qangaroo WikiHop data set, our HDE graph based single\nmodel delivers competitive result, and the ensemble model achieves the\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07458v4",
            "date": "2019-05-17 20:20:22+00:00",
            "title": "Neural Metric Learning for Fast End-to-End Relation Extraction",
            "authors": [
                "Tung Tran",
                "Ramakanth Kavuluru"
            ],
            "abstract": "Relation extraction (RE) is an indispensable information extraction task in\nseveral disciplines. RE models typically assume that named entity recognition\n(NER) is already performed in a previous step by another independent model.\nSeveral recent efforts, under the theme of end-to-end RE, seek to exploit\ninter-task correlations by modeling both NER and RE tasks jointly. Earlier work\nin this area commonly reduces the task to a table-filling problem wherein an\nadditional expensive decoding step involving beam search is applied to obtain\nglobally consistent cell labels. In efforts that do not employ table-filling,\nglobal optimization in the form of CRFs with Viterbi decoding for the NER\ncomponent is still necessary for competitive performance. We introduce a novel\nneural architecture utilizing the table structure, based on repeated\napplications of 2D convolutions for pooling local dependency and metric-based\nfeatures, that improves on the state-of-the-art without the need for global\noptimization. We validate our model on the ADE and CoNLL04 datasets for\nend-to-end RE and demonstrate $\\approx 1\\%$ gain (in F-score) over prior best\nresults with training and testing times that are seven to ten times faster ---\nthe latter highly advantageous for time-sensitive end user applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07464v1",
            "date": "2019-05-17 20:29:40+00:00",
            "title": "A Multi-Task Learning Framework for Extracting Drugs and Their Interactions from Drug Labels",
            "authors": [
                "Tung Tran",
                "Ramakanth Kavuluru",
                "Halil Kilicoglu"
            ],
            "abstract": "Preventable adverse drug reactions as a result of medical errors present a\ngrowing concern in modern medicine. As drug-drug interactions (DDIs) may cause\nadverse reactions, being able to extracting DDIs from drug labels into\nmachine-readable form is an important effort in effectively deploying drug\nsafety information. The DDI track of TAC 2018 introduces two large\nhand-annotated test sets for the task of extracting DDIs from structured\nproduct labels with linkage to standard terminologies. Herein, we describe our\napproach to tackling tasks one and two of the DDI track, which corresponds to\nnamed entity recognition (NER) and sentence-level relation extraction\nrespectively. Namely, our approach resembles a multi-task learning framework\ndesigned to jointly model various sub-tasks including NER and interaction type\nand outcome prediction. On NER, our system ranked second (among eight teams) at\n33.00% and 38.25% F1 on Test Sets 1 and 2 respectively. On relation extraction,\nour system ranked second (among four teams) at 21.59% and 23.55% on Test Sets 1\nand 2 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07584v1",
            "date": "2019-05-18 13:11:31+00:00",
            "title": "Microblog Hashtag Generation via Encoding Conversation Contexts",
            "authors": [
                "Yue Wang",
                "Jing Li",
                "Irwin King",
                "Michael R. Lyu",
                "Shuming Shi"
            ],
            "abstract": "Automatic hashtag annotation plays an important role in content understanding\nfor microblog posts. To date, progress made in this field has been restricted\nto phrase selection from limited candidates, or word-level hashtag discovery\nusing topic models. Different from previous work considering hashtags to be\ninseparable, our work is the first effort to annotate hashtags with a novel\nsequence generation framework via viewing the hashtag as a short sequence of\nwords. Moreover, to address the data sparsity issue in processing short\nmicroblog posts, we propose to jointly model the target posts and the\nconversation contexts initiated by them with bidirectional attention. Extensive\nexperimental results on two large-scale datasets, newly collected from English\nTwitter and Chinese Weibo, show that our model significantly outperforms\nstate-of-the-art models based on classification. Further studies demonstrate\nour ability to effectively generate rare and even unseen hashtags, which is\nhowever not possible for most existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07588v1",
            "date": "2019-05-18 13:35:33+00:00",
            "title": "BERTSel: Answer Selection with Pre-trained Models",
            "authors": [
                "Dongfang Li",
                "Yifei Yu",
                "Qingcai Chen",
                "Xinyu Li"
            ],
            "abstract": "Recently, pre-trained models have been the dominant paradigm in natural\nlanguage processing. They achieved remarkable state-of-the-art performance\nacross a wide range of related tasks, such as textual entailment, natural\nlanguage inference, question answering, etc. BERT, proposed by Devlin et.al.,\nhas achieved a better marked result in GLUE leaderboard with a deep transformer\narchitecture. Despite its soaring popularity, however, BERT has not yet been\napplied to answer selection. This task is different from others with a few\nnuances: first, modeling the relevance and correctness of candidates matters\ncompared to semantic relatedness and syntactic structure; second, the length of\nan answer may be different from other candidates and questions. In this paper.\nwe are the first to explore the performance of fine-tuning BERT for answer\nselection. We achieved STOA results across five popular datasets, demonstrating\nthe success of pre-trained models in this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07595v1",
            "date": "2019-05-18 14:32:53+00:00",
            "title": "Semantic flow in language networks",
            "authors": [
                "Edilson A. Corr\u00eaa Jr.",
                "Vanessa Q. Marinho",
                "Diego R. Amancio"
            ],
            "abstract": "In this study we propose a framework to characterize documents based on their\nsemantic flow. The proposed framework encompasses a network-based model that\nconnected sentences based on their semantic similarity. Semantic fields are\ndetected using standard community detection methods. as the story unfolds,\ntransitions between semantic fields are represent in Markov networks, which in\nturned are characterized via network motifs (subgraphs). Here we show that the\nproposed framework can be used to classify books according to their style and\npublication dates. Remarkably, even without a systematic optimization of\nparameters, philosophy and investigative books were discriminated with an\naccuracy rate of 92.5%. Because this model captures semantic features of texts,\nit could be used as an additional feature in traditional network-based models\nof texts that capture only syntactical/stylistic information, as it is the case\nof word adjacency (co-occurrence) networks.",
            "journal ref": "Physica A, v. 557, p. 124895, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07695v2",
            "date": "2019-05-19 07:12:22+00:00",
            "title": "Structured Summarization of Academic Publications",
            "authors": [
                "Alexios Gidiotis",
                "Grigorios Tsoumakas"
            ],
            "abstract": "We propose SUSIE, a novel summarization method that can work with\nstate-of-the-art summarization models in order to produce structured scientific\nsummaries for academic articles. We also created PMC-SA, a new dataset of\nacademic publications, suitable for the task of structured summarization with\nneural networks. We apply SUSIE combined with three different summarization\nmodels on the new PMC-SA dataset and we show that the proposed method improves\nthe performance of all models by as much as 4 ROUGE points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07719v2",
            "date": "2019-05-19 10:20:09+00:00",
            "title": "Earlier Attention? Aspect-Aware LSTM for Aspect-Based Sentiment Analysis",
            "authors": [
                "Bowen Xing",
                "Lejian Liao",
                "Dandan Song",
                "Jingang Wang",
                "Fuzheng Zhang",
                "Zhongyuan Wang",
                "Heyan Huang"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) aims to predict fine-grained\nsentiments of comments with respect to given aspect terms or categories. In\nprevious ABSA methods, the importance of aspect has been realized and verified.\nMost existing LSTM-based models take aspect into account via the attention\nmechanism, where the attention weights are calculated after the context is\nmodeled in the form of contextual vectors. However, aspect-related information\nmay be already discarded and aspect-irrelevant information may be retained in\nclassic LSTM cells in the context modeling process, which can be improved to\ngenerate more effective context representations. This paper proposes a novel\nvariant of LSTM, termed as aspect-aware LSTM (AA-LSTM), which incorporates\naspect information into LSTM cells in the context modeling stage before the\nattention mechanism. Therefore, our AA-LSTM can dynamically produce\naspect-aware contextual representations. We experiment with several\nrepresentative LSTM-based models by replacing the classic LSTM cells with the\nAA-LSTM cells. Experimental results on SemEval-2014 Datasets demonstrate the\neffectiveness of AA-LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07791v1",
            "date": "2019-05-19 18:28:34+00:00",
            "title": "Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction",
            "authors": [
                "Yinfei Yang",
                "Oshin Agarwal",
                "Chris Tar",
                "Byron C. Wallace",
                "Ani Nenkova"
            ],
            "abstract": "Modern NLP systems require high-quality annotated data. In specialized\ndomains, expert annotations may be prohibitively expensive. An alternative is\nto rely on crowdsourcing to reduce costs at the risk of introducing noise. In\nthis paper we demonstrate that directly modeling instance difficulty can be\nused to improve model performance, and to route instances to appropriate\nannotators. Our difficulty prediction model combines two learned\nrepresentations: a `universal' encoder trained on out-of-domain data, and a\ntask-specific encoder. Experiments on a complex biomedical information\nextraction task using expert and lay annotators show that: (i) simply excluding\nfrom the training data instances predicted to be difficult yields a small boost\nin performance; (ii) using difficulty scores to weight instances during\ntraining provides further, consistent gains; (iii) assigning instances\npredicted to be difficult to domain experts is an effective strategy for task\nrouting. Our experiments confirm the expectation that for specialized tasks\nexpert annotations are higher quality than crowd labels, and hence preferable\nto obtain if practical. Moreover, augmenting small amounts of expert data with\na larger set of lay annotations leads to further improvements in model\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07830v1",
            "date": "2019-05-19 23:57:23+00:00",
            "title": "HellaSwag: Can a Machine Really Finish Your Sentence?",
            "authors": [
                "Rowan Zellers",
                "Ari Holtzman",
                "Yonatan Bisk",
                "Ali Farhadi",
                "Yejin Choi"
            ],
            "abstract": "Recent work by Zellers et al. (2018) introduced a new task of commonsense\nnatural language inference: given an event description such as \"A woman sits at\na piano,\" a machine must select the most likely followup: \"She sets her fingers\non the keys.\" With the introduction of BERT, near human-level performance was\nreached. Does this mean that machines can perform human level commonsense\ninference?\n  In this paper, we show that commonsense inference still proves difficult for\neven state-of-the-art models, by presenting HellaSwag, a new challenge dataset.\nThough its questions are trivial for humans (>95% accuracy), state-of-the-art\nmodels struggle (<48%). We achieve this via Adversarial Filtering (AF), a data\ncollection paradigm wherein a series of discriminators iteratively select an\nadversarial set of machine-generated wrong answers. AF proves to be\nsurprisingly robust. The key insight is to scale up the length and complexity\nof the dataset examples towards a critical 'Goldilocks' zone wherein generated\ntext is ridiculous to humans, yet often misclassified by state-of-the-art\nmodels.\n  Our construction of HellaSwag, and its resulting difficulty, sheds light on\nthe inner workings of deep pretrained models. More broadly, it suggests a new\npath forward for NLP research, in which benchmarks co-evolve with the evolving\nstate-of-the-art in an adversarial way, so as to present ever-harder\nchallenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07856v1",
            "date": "2019-05-20 03:14:11+00:00",
            "title": "Target Based Speech Act Classification in Political Campaign Text",
            "authors": [
                "Shivashankar Subramanian",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "We study pragmatics in political campaign text, through analysis of speech\nacts and the target of each utterance. We propose a new annotation schema\nincorporating domain-specific speech acts, such as commissive-action, and\npresent a novel annotated corpus of media releases and speech transcripts from\nthe 2016 Australian election cycle. We show how speech acts and target\nreferents can be modeled as sequential classification, and evaluate several\ntechniques, exploiting contextualized word representations, semi-supervised\nlearning, task dependencies and speaker meta-data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08160v2",
            "date": "2019-05-20 15:07:36+00:00",
            "title": "Interpretable Neural Predictions with Differentiable Binary Variables",
            "authors": [
                "Jasmijn Bastings",
                "Wilker Aziz",
                "Ivan Titov"
            ],
            "abstract": "The success of neural networks comes hand in hand with a desire for more\ninterpretability. We focus on text classifiers and make them more interpretable\nby having them provide a justification, a rationale, for their predictions. We\napproach this problem by jointly training two neural network models: a latent\nmodel that selects a rationale (i.e. a short and informative part of the input\ntext), and a classifier that learns from the words in the rationale alone.\nPrevious work proposed to assign binary latent masks to input positions and to\npromote short selections via sparsity-inducing penalties such as L0\nregularisation. We propose a latent model that mixes discrete and continuous\nbehaviour allowing at the same time for binary selections and gradient-based\ntraining without REINFORCE. In our formulation, we can tractably compute the\nexpected value of penalties such as L0, which allows us to directly optimise\nthe model towards a pre-specified text selection rate. We show that our\napproach is competitive with previous work on rationale extraction, and explore\nfurther uses in attention mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08181v2",
            "date": "2019-05-20 15:53:09+00:00",
            "title": "A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks",
            "authors": [
                "\u00c1lvaro Peris",
                "Francisco Casacuberta"
            ],
            "abstract": "We present a demonstration of a neural interactive-predictive system for\ntackling multimodal sequence to sequence tasks. The system generates text\npredictions to different sequence to sequence tasks: machine translation, image\nand video captioning. These predictions are revised by a human agent, who\nintroduces corrections in the form of characters. The system reacts to each\ncorrection, providing alternative hypotheses, compelling with the feedback\nprovided by the user. The final objective is to reduce the human effort\nrequired during this correction process.\n  This system is implemented following a client-server architecture. For\naccessing the system, we developed a website, which communicates with the\nneural model, hosted in a local server. From this website, the different tasks\ncan be tackled following the interactive-predictive framework. We open-source\nall the code developed for building this system. The demonstration in hosted in\nhttp://casmacat.prhlt.upv.es/interactive-seq2seq.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08205v2",
            "date": "2019-05-20 16:44:00+00:00",
            "title": "Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation",
            "authors": [
                "Jiaqi Guo",
                "Zecheng Zhan",
                "Yan Gao",
                "Yan Xiao",
                "Jian-Guang Lou",
                "Ting Liu",
                "Dongmei Zhang"
            ],
            "abstract": "We present a neural approach called IRNet for complex and cross-domain\nText-to-SQL. IRNet aims to address two challenges: 1) the mismatch between\nintents expressed in natural language (NL) and the implementation details in\nSQL; 2) the challenge in predicting columns caused by the large number of\nout-of-domain words. Instead of end-to-end synthesizing a SQL query, IRNet\ndecomposes the synthesis process into three phases. In the first phase, IRNet\nperforms a schema linking over a question and a database schema. Then, IRNet\nadopts a grammar-based neural model to synthesize a SemQL query which is an\nintermediate representation that we design to bridge NL and SQL. Finally, IRNet\ndeterministically infers a SQL query from the synthesized SemQL query with\ndomain knowledge. On the challenging Text-to-SQL benchmark Spider, IRNet\nachieves 46.7% accuracy, obtaining 19.5% absolute improvement over previous\nstate-of-the-art approaches. At the time of writing, IRNet achieves the first\nposition on the Spider leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08212v1",
            "date": "2019-05-20 16:54:43+00:00",
            "title": "Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation",
            "authors": [
                "Xinyi Wang",
                "Graham Neubig"
            ],
            "abstract": "To improve low-resource Neural Machine Translation (NMT) with multilingual\ncorpora, training on the most related high-resource language only is often more\neffective than using all data available (Neubig and Hu, 2018). However, it is\npossible that an intelligent data selection strategy can further improve\nlow-resource NMT with data from other auxiliary languages. In this paper, we\nseek to construct a sampling distribution over all multilingual data, so that\nit minimizes the training loss of the low-resource language. Based on this\nformulation, we propose an efficient algorithm, Target Conditioned Sampling\n(TCS), which first samples a target sentence, and then conditionally samples\nits source sentence. Experiments show that TCS brings significant gains of up\nto 2 BLEU on three of four languages we test, with minimal training overhead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08284v1",
            "date": "2019-05-20 18:22:18+00:00",
            "title": "Enriching Pre-trained Language Model with Entity Information for Relation Classification",
            "authors": [
                "Shanchan Wu",
                "Yifan He"
            ],
            "abstract": "Relation classification is an important NLP task to extract relations between\nentities. The state-of-the-art methods for relation classification are\nprimarily based on Convolutional or Recurrent Neural Networks. Recently, the\npre-trained BERT model achieves very successful results in many NLP\nclassification / sequence labeling tasks. Relation classification differs from\nthose tasks in that it relies on information of both the sentence and the two\ntarget entities. In this paper, we propose a model that both leverages the\npre-trained BERT language model and incorporates information from the target\nentities to tackle the relation classification task. We locate the target\nentities and transfer the information through the pre-trained architecture and\nincorporate the corresponding encoding of the two entities. We achieve\nsignificant improvement over the state-of-the-art method on the SemEval-2010\ntask 8 relational dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08377v1",
            "date": "2019-05-20 23:10:42+00:00",
            "title": "Word Usage Similarity Estimation with Sentence Representations and Automatic Substitutes",
            "authors": [
                "Aina Gar\u00ed Soler",
                "Marianna Apidianaki",
                "Alexandre Allauzen"
            ],
            "abstract": "Usage similarity estimation addresses the semantic proximity of word\ninstances in different contexts. We apply contextualized (ELMo and BERT) word\nand sentence embeddings to this task, and propose supervised models that\nleverage these representations for prediction. Our models are further assisted\nby lexical substitute annotations automatically assigned to word instances by\ncontext2vec, a neural model that relies on a bidirectional LSTM. We perform an\nextensive comparison of existing word and sentence representations on benchmark\ndatasets addressing both graded and binary similarity. The best performing\nmodels outperform previous methods in both settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08407v3",
            "date": "2019-05-21 02:13:03+00:00",
            "title": "Generating Logical Forms from Graph Representations of Text and Entities",
            "authors": [
                "Peter Shaw",
                "Philip Massey",
                "Angelica Chen",
                "Francesco Piccinno",
                "Yasemin Altun"
            ],
            "abstract": "Structured information about entities is critical for many semantic parsing\ntasks. We present an approach that uses a Graph Neural Network (GNN)\narchitecture to incorporate information about relevant entities and their\nrelations during parsing. Combined with a decoder copy mechanism, this approach\nprovides a conceptually simple mechanism to generate logical forms with\nentities. We demonstrate that this approach is competitive with the\nstate-of-the-art across several tasks without pre-training, and outperforms\nexisting approaches when combined with BERT pre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08511v2",
            "date": "2019-05-21 09:23:56+00:00",
            "title": "Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction",
            "authors": [
                "Kosuke Nishida",
                "Kyosuke Nishida",
                "Masaaki Nagata",
                "Atsushi Otsuka",
                "Itsumi Saito",
                "Hisako Asano",
                "Junji Tomita"
            ],
            "abstract": "Question answering (QA) using textual sources for purposes such as reading\ncomprehension (RC) has attracted much attention. This study focuses on the task\nof explainable multi-hop QA, which requires the system to return the answer\nwith evidence sentences by reasoning and gathering disjoint pieces of the\nreference texts. It proposes the Query Focused Extractor (QFE) model for\nevidence extraction and uses multi-task learning with the QA model. QFE is\ninspired by extractive summarization models; compared with the existing method,\nwhich extracts each evidence sentence independently, it sequentially extracts\nevidence sentences by using an RNN with an attention mechanism on the question\nsentence. It enables QFE to consider the dependency among the evidence\nsentences and cover important information in the question sentence.\nExperimental results show that QFE with a simple RC baseline model achieves a\nstate-of-the-art evidence extraction score on HotpotQA. Although designed for\nRC, it also achieves a state-of-the-art evidence extraction score on FEVER,\nwhich is a recognizing textual entailment task on a large textual database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08675v1",
            "date": "2019-05-21 14:47:35+00:00",
            "title": "MultiWiki: Interlingual Text Passage Alignment in Wikipedia",
            "authors": [
                "Simon Gottschalk",
                "Elena Demidova"
            ],
            "abstract": "In this article we address the problem of text passage alignment across\ninterlingual article pairs in Wikipedia. We develop methods that enable the\nidentification and interlinking of text passages written in different languages\nand containing overlapping information. Interlingual text passage alignment can\nenable Wikipedia editors and readers to better understand language-specific\ncontext of entities, provide valuable insights in cultural differences and\nbuild a basis for qualitative analysis of the articles. An important challenge\nin this context is the trade-off between the granularity of the extracted text\npassages and the precision of the alignment. Whereas short text passages can\nresult in more precise alignment, longer text passages can facilitate a better\noverview of the differences in an article pair. To better understand these\naspects from the user perspective, we conduct a user study at the example of\nthe German, Russian and the English Wikipedia and collect a user-annotated\nbenchmark. Then we propose MultiWiki -- a method that adopts an integrated\napproach to the text passage alignment using semantic similarity measures and\ngreedy algorithms and achieves precise results with respect to the user-defined\nalignment. MultiWiki demonstration is publicly available and currently supports\nfour language pairs.",
            "journal ref": "ACM Transactions on the Web 11 (1), 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08704v2",
            "date": "2019-05-21 15:41:18+00:00",
            "title": "AMR Parsing as Sequence-to-Graph Transduction",
            "authors": [
                "Sheng Zhang",
                "Xutai Ma",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "We propose an attention-based model that treats AMR parsing as\nsequence-to-graph transduction. Unlike most AMR parsers that rely on\npre-trained aligners, external semantic resources, or data augmentation, our\nproposed parser is aligner-free, and it can be effectively trained with limited\namounts of labeled AMR data. Our experimental results outperform all previously\nreported SMATCH scores, on both AMR 2.0 (76.3% F1 on LDC2017T10) and AMR 1.0\n(70.2% F1 on LDC2014T12).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08780v1",
            "date": "2019-05-21 17:54:09+00:00",
            "title": "A Comparative Analysis of Distributional Term Representations for Author Profiling in Social Media",
            "authors": [
                "Miguel \u00c1. \u00c1lvarez-Carmona",
                "Esa\u00fa Villatoro-Tello",
                "Manuel Montes-y-G\u00f3mez",
                "Luis Villase\u00f1or-Pienda"
            ],
            "abstract": "Author Profiling (AP) aims at predicting specific characteristics from a\ngroup of authors by analyzing their written documents. Many research has been\nfocused on determining suitable features for modeling writing patterns from\nauthors. Reported results indicate that content-based features continue to be\nthe most relevant and discriminant features for solving this task. Thus, in\nthis paper, we present a thorough analysis regarding the appropriateness of\ndifferent distributional term representations (DTR) for the AP task. In this\nregard, we introduce a novel framework for supervised AP using these\nrepresentations and, supported on it. We approach a comparative analysis of\nrepresentations such as DOR, TCOR, SSR, and word2vec in the AP problem. We also\ncompare the performance of the DTRs against classic approaches including\npopular topic-based methods. The obtained results indicate that DTRs are\nsuitable for solving the AP task in social media domains as they achieve\ncompetitive results while providing meaningful interpretability.",
            "journal ref": "Journal of Intelligent & Fuzzy Systems, vol. 36, no. 5, pp.\n  4857-4868, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08794v1",
            "date": "2019-05-21 15:14:34+00:00",
            "title": "EventKG - the Hub of Event Knowledge on the Web - and Biographical Timeline Generation",
            "authors": [
                "Simon Gottschalk",
                "Elena Demidova"
            ],
            "abstract": "One of the key requirements to facilitate the semantic analytics of\ninformation regarding contemporary and historical events on the Web, in the\nnews and in social media is the availability of reference knowledge\nrepositories containing comprehensive representations of events, entities and\ntemporal relations. Existing knowledge graphs, with popular examples including\nDBpedia, YAGO and Wikidata, focus mostly on entity-centric information and are\ninsufficient in terms of their coverage and completeness with respect to events\nand temporal relations. In this article we address this limitation, formalise\nthe concept of a temporal knowledge graph and present its instantiation -\nEventKG. EventKG is a multilingual event-centric temporal knowledge graph that\nincorporates over 690 thousand events and over 2.3 million temporal relations\nobtained from several large-scale knowledge graphs and semi-structured sources\nand makes them available through a canonical RDF representation. Whereas\npopular entities often possess hundreds of relations within a temporal\nknowledge graph such as EventKG, generating a concise overview of the most\nimportant temporal relations for a given entity is a challenging task. In this\narticle we demonstrate an application of EventKG to biographical timeline\ngeneration, where we adopt a distant supervision method to identify relations\nmost relevant for an entity biography. Our evaluation results provide insights\non the characteristics of EventKG and demonstrate the effectiveness of the\nproposed biographical timeline generation method.",
            "journal ref": "Semantic Web Journal (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08836v1",
            "date": "2019-05-21 19:13:16+00:00",
            "title": "Sample Efficient Text Summarization Using a Single Pre-Trained Transformer",
            "authors": [
                "Urvashi Khandelwal",
                "Kevin Clark",
                "Dan Jurafsky",
                "Lukasz Kaiser"
            ],
            "abstract": "Language model (LM) pre-training has resulted in impressive performance and\nsample efficiency on a variety of language understanding tasks. However, it\nremains unclear how to best use pre-trained LMs for generation tasks such as\nabstractive summarization, particularly to enhance sample efficiency. In these\nsequence-to-sequence settings, prior work has experimented with loading\npre-trained weights into the encoder and/or decoder networks, but used\nnon-pre-trained encoder-decoder attention weights. We instead use a pre-trained\ndecoder-only network, where the same Transformer LM both encodes the source and\ngenerates the summary. This ensures that all parameters in the network,\nincluding those governing attention over source states, have been pre-trained\nbefore the fine-tuning step. Experiments on the CNN/Daily Mail dataset show\nthat our pre-trained Transformer LM substantially improves over pre-trained\nTransformer encoder-decoder networks in limited-data settings. For instance, it\nachieves 13.1 ROUGE-2 using only 1% of the training data (~3000 examples),\nwhile pre-trained encoder-decoder models score 2.3 ROUGE-2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08868v3",
            "date": "2019-05-21 20:56:50+00:00",
            "title": "Look Again at the Syntax: Relational Graph Convolutional Network for Gendered Ambiguous Pronoun Resolution",
            "authors": [
                "Yinchuan Xu",
                "Junlin Yang"
            ],
            "abstract": "Gender bias has been found in existing coreference resolvers. In order to\neliminate gender bias, a gender-balanced dataset Gendered Ambiguous Pronouns\n(GAP) has been released and the best baseline model achieves only 66.9% F1.\nBidirectional Encoder Representations from Transformers (BERT) has broken\nseveral NLP task records and can be used on GAP dataset. However, fine-tune\nBERT on a specific task is computationally expensive. In this paper, we propose\nan end-to-end resolver by combining pre-trained BERT with Relational Graph\nConvolutional Network (R-GCN). R-GCN is used for digesting structural syntactic\ninformation and learning better task-specific embeddings. Empirical results\ndemonstrate that, under explicit syntactic supervision and without the need to\nfine tune BERT, R-GCN's embeddings outperform the original BERT embeddings on\nthe coreference task. Our work significantly improves the snippet-context\nbaseline F1 score on GAP dataset from 66.9% to 80.3%. We participated in the\n2019 GAP Coreference Shared Task, and our codes are available online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08945v1",
            "date": "2019-05-22 04:11:13+00:00",
            "title": "Corpus Augmentation by Sentence Segmentation for Low-Resource Neural Machine Translation",
            "authors": [
                "Jinyi Zhang",
                "Tadahiro Matsumoto"
            ],
            "abstract": "Neural Machine Translation (NMT) has been proven to achieve impressive\nresults. The NMT system translation results depend strongly on the size and\nquality of parallel corpora. Nevertheless, for many language pairs, no\nrich-resource parallel corpora exist. As described in this paper, we propose a\ncorpus augmentation method by segmenting long sentences in a corpus using\nback-translation and generating pseudo-parallel sentence pairs. The experiment\nresults of the Japanese-Chinese and Chinese-Japanese translation with\nJapanese-Chinese scientific paper excerpt corpus (ASPEC-JC) show that the\nmethod improves translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08949v3",
            "date": "2019-05-22 04:38:06+00:00",
            "title": "Recent Advances in Neural Question Generation",
            "authors": [
                "Liangming Pan",
                "Wenqiang Lei",
                "Tat-Seng Chua",
                "Min-Yen Kan"
            ],
            "abstract": "Emerging research in Neural Question Generation (NQG) has started to\nintegrate a larger variety of inputs, and generating questions requiring higher\nlevels of cognition. These trends point to NQG as a bellwether for NLP, about\nhow human intelligence embodies the skills of curiosity and integration.\n  We present a comprehensive survey of neural question generation, examining\nthe corpora, methodologies, and evaluation methods. From this, we elaborate on\nwhat we see as emerging on NQG's trend: in terms of the learning paradigms,\ninput modalities, and cognitive levels considered by NQG. We end by pointing\nout the potential directions ahead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09135v2",
            "date": "2019-05-22 13:40:38+00:00",
            "title": "A Joint Named-Entity Recognizer for Heterogeneous Tag-sets Using a Tag Hierarchy",
            "authors": [
                "Genady Beryozkin",
                "Yoel Drori",
                "Oren Gilon",
                "Tzvika Hartman",
                "Idan Szpektor"
            ],
            "abstract": "We study a variant of domain adaptation for named-entity recognition where\nmultiple, heterogeneously tagged training sets are available. Furthermore, the\ntest tag-set is not identical to any individual training tag-set. Yet, the\nrelations between all tags are provided in a tag hierarchy, covering the test\ntags as a combination of training tags. This setting occurs when various\ndatasets are created using different annotation schemes. This is also the case\nof extending a tag-set with a new tag by annotating only the new tag in a new\ndataset. We propose to use the given tag hierarchy to jointly learn a neural\nnetwork that shares its tagging layer among all tag-sets. We compare this model\nto combining independent models and to a model based on the multitasking\napproach. Our experiments show the benefit of the tag-hierarchy model,\nespecially when facing non-trivial consolidation of tag-sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09139v1",
            "date": "2019-05-22 13:45:05+00:00",
            "title": "Sentence Length",
            "authors": [
                "G\u00e1bor Borb\u00e9ly",
                "Andr\u00e1s Kornai"
            ],
            "abstract": "The distribution of sentence length in ordinary language is not well captured\nby the existing models. Here we survey previous models of sentence length and\npresent our random walk model that offers both a better fit with the data and a\nbetter understanding of the distribution. We develop a generalization of KL\ndivergence, discuss measuring the noise inherent in a corpus, and present a\nhyperparameter-free Bayesian model comparison method that has strong conceptual\nties to Minimal Description Length modeling. The models we obtain require only\na few dozen bits, orders of magnitude less than the naive nonparametric MDL\nmodels would.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09418v2",
            "date": "2019-05-23 01:13:24+00:00",
            "title": "Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned",
            "authors": [
                "Elena Voita",
                "David Talbot",
                "Fedor Moiseev",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "Multi-head self-attention is a key component of the Transformer, a\nstate-of-the-art architecture for neural machine translation. In this work we\nevaluate the contribution made by individual attention heads in the encoder to\nthe overall performance of the model and analyze the roles played by them. We\nfind that the most important and confident heads play consistent and often\nlinguistically-interpretable roles. When pruning heads using a method based on\nstochastic gates and a differentiable relaxation of the L0 penalty, we observe\nthat specialized heads are last to be pruned. Our novel pruning method removes\nthe vast majority of heads without seriously affecting performance. For\nexample, on the English-Russian WMT dataset, pruning 38 out of 48 encoder heads\nresults in a drop of only 0.15 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09439v1",
            "date": "2019-05-23 02:52:10+00:00",
            "title": "GWU NLP Lab at SemEval-2019 Task 3: EmoContext: Effective Contextual Information in Models for Emotion Detection in Sentence-level in a Multigenre Corpus",
            "authors": [
                "Shabnam Tafreshi",
                "Mona Diab"
            ],
            "abstract": "In this paper we present an emotion classifier model submitted to the\nSemEval-2019 Task 3: EmoContext. The task objective is to classify emotion\n(i.e. happy, sad, angry) in a 3-turn conversational data set. We formulate the\ntask as a classification problem and introduce a Gated Recurrent Neural Network\n(GRU) model with attention layer, which is bootstrapped with contextual\ninformation and trained with a multigenre corpus. We utilize different word\nembeddings to empirically select the most suited one to represent our features.\nWe train the model with a multigenre emotion corpus to leverage using all\navailable training sets to bootstrap the results. We achieved overall %56.05\nf1-score and placed 144.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09531v2",
            "date": "2019-05-23 08:33:56+00:00",
            "title": "MCScript2.0: A Machine Comprehension Corpus Focused on Script Events and Participants",
            "authors": [
                "Simon Ostermann",
                "Michael Roth",
                "Manfred Pinkal"
            ],
            "abstract": "We introduce MCScript2.0, a machine comprehension corpus for the end-to-end\nevaluation of script knowledge. MCScript2.0 contains approx. 20,000 questions\non approx. 3,500 texts, crowdsourced based on a new collection process that\nresults in challenging questions. Half of the questions cannot be answered from\nthe reading texts, but require the use of commonsense and, in particular,\nscript knowledge. We give a thorough analysis of our corpus and show that while\nthe task is not challenging to humans, existing machine comprehension models\nfail to perform well on the data, even if they make use of a commonsense\nknowledge base. The dataset is available at\nhttp://www.sfb1102.uni-saarland.de/?page_id=2582",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09866v2",
            "date": "2019-05-23 18:43:59+00:00",
            "title": "Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor",
            "authors": [
                "Malvina Nissim",
                "Rik van Noord",
                "Rob van der Goot"
            ],
            "abstract": "Analogies such as \"man is to king as woman is to X\" are often used to\nillustrate the amazing power of word embeddings. Concurrently, they have also\nbeen used to expose how strongly human biases are encoded in vector spaces\nbuilt on natural language, like \"man is to computer programmer as woman is to\nhomemaker\". Recent work has shown that analogies are in fact not such a\ndiagnostic for bias, and other methods have been proven to be more apt to the\ntask. However, beside the intrinsic problems with the analogy task as a bias\ndetection tool, in this paper we show that a series of issues related to how\nanalogies have been implemented and used might have yielded a distorted picture\nof bias in word embeddings. Human biases are present in word embeddings and\nneed to be addressed. Analogies, though, are probably not the right tool to do\nso. Also, the way they have been most often used has exacerbated some possibly\nnon-existing biases and perhaps hid others. Because they are still widely\npopular, and some of them have become classics within and outside the NLP\ncommunity, we deem it important to provide a series of clarifications that\nshould put well-known, and potentially new cases into the right perspective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10024v1",
            "date": "2019-05-24 04:16:46+00:00",
            "title": "An Analysis of Source-Side Grammatical Errors in NMT",
            "authors": [
                "Antonios Anastasopoulos"
            ],
            "abstract": "The quality of Neural Machine Translation (NMT) has been shown to\nsignificantly degrade when confronted with source-side noise. We present the\nfirst large-scale study of state-of-the-art English-to-German NMT on real\ngrammatical noise, by evaluating on several Grammar Correction corpora. We\npresent methods for evaluating NMT robustness without true references, and we\nuse them for extensive analysis of the effects that different grammatical\nerrors have on the NMT output. We also introduce a technique for visualizing\nthe divergence distribution caused by a source-side error, which allows for\nadditional insights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10039v1",
            "date": "2019-05-24 05:29:33+00:00",
            "title": "Outline Generation: Understanding the Inherent Content Structure of Documents",
            "authors": [
                "Ruqing Zhang",
                "Jiafeng Guo",
                "Yixing Fan",
                "Yanyan Lan",
                "Xueqi Cheng"
            ],
            "abstract": "In this paper, we introduce and tackle the Outline Generation (OG) task,\nwhich aims to unveil the inherent content structure of a multi-paragraph\ndocument by identifying its potential sections and generating the corresponding\nsection headings. Without loss of generality, the OG task can be viewed as a\nnovel structured summarization task. To generate a sound outline, an ideal OG\nmodel should be able to capture three levels of coherence, namely the coherence\nbetween context paragraphs, that between a section and its heading, and that\nbetween context headings. The first one is the foundation for section\nidentification, while the latter two are critical for consistent heading\ngeneration. In this work, we formulate the OG task as a hierarchical structured\nprediction problem, i.e., to first predict a sequence of section boundaries and\nthen a sequence of section headings accordingly. We propose a novel\nhierarchical structured neural generation model, named HiStGen, for the task.\nOur model attempts to capture the three-level coherence via the following ways.\nFirst, we introduce a Markov paragraph dependency mechanism between context\nparagraphs for section identification. Second, we employ a section-aware\nattention mechanism to ensure the semantic coherence between a section and its\nheading. Finally, we leverage a Markov heading dependency mechanism and a\nreview mechanism between context headings to improve the consistency and\neliminate duplication between section headings. Besides, we build a novel\nWIKIOG dataset, a public collection which consists of over 1.75 million\ndocument-outline pairs for research on the OG task. Experimental results on our\nbenchmark dataset demonstrate that our model can significantly outperform\nseveral state-of-the-art sequential generation models for the OG task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10044v1",
            "date": "2019-05-24 05:48:49+00:00",
            "title": "BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions",
            "authors": [
                "Christopher Clark",
                "Kenton Lee",
                "Ming-Wei Chang",
                "Tom Kwiatkowski",
                "Michael Collins",
                "Kristina Toutanova"
            ],
            "abstract": "In this paper we study yes/no questions that are naturally occurring ---\nmeaning that they are generated in unprompted and unconstrained settings. We\nbuild a reading comprehension dataset, BoolQ, of such questions, and show that\nthey are unexpectedly challenging. They often query for complex, non-factoid\ninformation, and require difficult entailment-like inference to solve. We also\nexplore the effectiveness of a range of transfer learning baselines. We find\nthat transferring from entailment data is more effective than transferring from\nparaphrase or extractive QA data, and that it, surprisingly, continues to be\nvery beneficial even when starting from massive pre-trained language models\nsuch as BERT. Our best method trains BERT on MultiNLI and then re-trains it on\nour train set. It achieves 80.4% accuracy compared to 90% accuracy of human\nannotators (and 62% majority-baseline), leaving a significant gap for future\nwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10060v1",
            "date": "2019-05-24 07:02:05+00:00",
            "title": "A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer",
            "authors": [
                "Fuli Luo",
                "Peng Li",
                "Jie Zhou",
                "Pengcheng Yang",
                "Baobao Chang",
                "Zhifang Sui",
                "Xu Sun"
            ],
            "abstract": "Unsupervised text style transfer aims to transfer the underlying style of\ntext but keep its main content unchanged without parallel data. Most existing\nmethods typically follow two steps: first separating the content from the\noriginal style, and then fusing the content with the desired style. However,\nthe separation in the first step is challenging because the content and style\ninteract in subtle ways in natural language. Therefore, in this paper, we\npropose a dual reinforcement learning framework to directly transfer the style\nof the text via a one-step mapping model, without any separation of content and\nstyle. Specifically, we consider the learning of the source-to-target and\ntarget-to-source mappings as a dual task, and two rewards are designed based on\nsuch a dual structure to reflect the style accuracy and content preservation,\nrespectively. In this way, the two one-step mapping models can be trained via\nreinforcement learning, without any use of parallel data. Automatic evaluations\nshow that our model outperforms the state-of-the-art systems by a large margin,\nespecially with more than 8 BLEU points improvement averaged on two benchmark\ndatasets. Human evaluations also validate the effectiveness of our model in\nterms of style accuracy, content preservation and fluency. Our code and data,\nincluding outputs of all baselines and our model are available at\nhttps://github.com/luofuli/DualLanST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10238v1",
            "date": "2019-05-24 13:52:14+00:00",
            "title": "Incorporating Context and External Knowledge for Pronoun Coreference Resolution",
            "authors": [
                "Hongming Zhang",
                "Yan Song",
                "Yangqiu Song"
            ],
            "abstract": "Linking pronominal expressions to the correct references requires, in many\ncases, better analysis of the contextual information and external knowledge. In\nthis paper, we propose a two-layer model for pronoun coreference resolution\nthat leverages both context and external knowledge, where a knowledge attention\nmechanism is designed to ensure the model leveraging the appropriate source of\nexternal knowledge based on different context. Experimental results demonstrate\nthe validity and effectiveness of our model, where it outperforms\nstate-of-the-art models by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10247v1",
            "date": "2019-05-24 14:18:18+00:00",
            "title": "Contextual Out-of-Domain Utterance Handling With Counterfeit Data Augmentation",
            "authors": [
                "Sungjin Lee",
                "Igor Shalyminov"
            ],
            "abstract": "Neural dialog models often lack robustness to anomalous user input and\nproduce inappropriate responses which leads to frustrating user experience.\nAlthough there are a set of prior approaches to out-of-domain (OOD) utterance\ndetection, they share a few restrictions: they rely on OOD data or multiple\nsub-domains, and their OOD detection is context-independent which leads to\nsuboptimal performance in a dialog. The goal of this paper is to propose a\nnovel OOD detection method that does not require OOD data by utilizing\ncounterfeit OOD turns in the context of a dialog. For the sake of fostering\nfurther research, we also release new dialog datasets which are 3 publicly\navailable dialog corpora augmented with OOD turns in a controllable way. Our\nmethod outperforms state-of-the-art dialog models equipped with a conventional\nOOD detection mechanism by a large margin in the presence of OOD utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10431v1",
            "date": "2019-05-24 20:06:04+00:00",
            "title": "What Syntactic Structures block Dependencies in RNN Language Models?",
            "authors": [
                "Ethan Wilcox",
                "Roger Levy",
                "Richard Futrell"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) trained on a language modeling task have\nbeen shown to acquire a number of non-local grammatical dependencies with some\nsuccess. Here, we provide new evidence that RNN language models are sensitive\nto hierarchical syntactic structure by investigating the filler--gap dependency\nand constraints on it, known as syntactic islands. Previous work is\ninconclusive about whether RNNs learn to attenuate their expectations for gaps\nin island constructions in particular or in any sufficiently complex syntactic\nenvironment. This paper gives new evidence for the former by providing control\nstudies that have been lacking so far. We demonstrate that two state-of-the-art\nRNN models are are able to maintain the filler--gap dependency through\nunbounded sentential embeddings and are also sensitive to the hierarchical\nrelationship between the filler and the gap. Next, we demonstrate that the\nmodels are able to maintain possessive pronoun gender expectations through\nisland constructions---this control case rules out the possibility that island\nconstructions block all information flow in these networks. We also evaluate\nthree untested islands constraints: coordination islands, left branch islands,\nand sentential subject islands. Models are able to learn left branch islands\nand learn coordination islands gradiently, but fail to learn sentential subject\nislands. Through these controls and new tests, we provide evidence that model\nbehavior is due to finer-grained expectations than gross syntactic complexity,\nbut also that the models are conspicuously un-humanlike in some of their\nperformance characteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10453v2",
            "date": "2019-05-24 21:32:28+00:00",
            "title": "A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translation",
            "authors": [
                "Shuoyang Ding",
                "Adithya Renduchintala",
                "Kevin Duh"
            ],
            "abstract": "Most neural machine translation systems are built upon subword units\nextracted by methods such as Byte-Pair Encoding (BPE) or wordpiece. However,\nthe choice of number of merge operations is generally made by following\nexisting recipes. In this paper, we conduct a systematic exploration on\ndifferent numbers of BPE merge operations to understand how it interacts with\nthe model architecture, the strategy to build vocabularies and the language\npair. Our exploration could provide guidance for selecting proper BPE\nconfigurations in the future. Most prominently: we show that for LSTM-based\narchitectures, it is necessary to experiment with a wide range of different BPE\noperations as there is no typical optimal BPE configuration, whereas for\nTransformer architectures, smaller BPE size tends to be a typically optimal\nchoice. We urge the community to make prudent choices with subword merge\noperations, as our experiments indicate that a sub-optimal BPE configuration\nalone could easily reduce the system performance by 3-4 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10464v3",
            "date": "2019-05-24 22:11:57+00:00",
            "title": "Debiasing Word Embeddings Improves Multimodal Machine Translation",
            "authors": [
                "Tosho Hirasawa",
                "Mamoru Komachi"
            ],
            "abstract": "In recent years, pretrained word embeddings have proved useful for multimodal\nneural machine translation (NMT) models to address the shortage of available\ndatasets. However, the integration of pretrained word embeddings has not yet\nbeen explored extensively. Further, pretrained word embeddings in high\ndimensional spaces have been reported to suffer from the hubness problem.\nAlthough some debiasing techniques have been proposed to address this problem\nfor other natural language processing tasks, they have seldom been studied for\nmultimodal NMT models. In this study, we examine various kinds of word\nembeddings and introduce two debiasing techniques for three multimodal NMT\nmodels and two language pairs -- English-German translation and English-French\ntranslation. With our optimal settings, the overall performance of multimodal\nmodels was improved by up to +1.93 BLEU and +2.02 METEOR for English-German\ntranslation and +1.73 BLEU and +0.95 METEOR for English-French translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10486v1",
            "date": "2019-05-24 23:57:11+00:00",
            "title": "Designing a Symbolic Intermediate Representation for Neural Surface Realization",
            "authors": [
                "Henry Elder",
                "Jennifer Foster",
                "James Barry",
                "Alexander O'Connor"
            ],
            "abstract": "Generated output from neural NLG systems often contain errors such as\nhallucination, repetition or contradiction. This work focuses on designing a\nsymbolic intermediate representation to be used in multi-stage neural\ngeneration with the intention of reducing the frequency of failed outputs. We\nshow that surface realization from this intermediate representation is of high\nquality and when the full system is applied to the E2E dataset it outperforms\nthe winner of the E2E challenge. Furthermore, by breaking out the surface\nrealization step from typically end-to-end neural systems, we also provide a\nframework for non-neural content selection and planning systems to potentially\ntake advantage of semi-supervised pretraining of neural surface realization\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10515v2",
            "date": "2019-05-25 03:57:16+00:00",
            "title": "SuperCaptioning: Image Captioning Using Two-dimensional Word Embedding",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Michael Lin",
                "Charles Young",
                "Patrick Dong",
                "Wenhan Zhang",
                "Jason Dong"
            ],
            "abstract": "Language and vision are processed as two different modal in current work for\nimage captioning. However, recent work on Super Characters method shows the\neffectiveness of two-dimensional word embedding, which converts text\nclassification problem into image classification problem. In this paper, we\npropose the SuperCaptioning method, which borrows the idea of two-dimensional\nword embedding from Super Characters method, and processes the information of\nlanguage and vision together in one single CNN model. The experimental results\non Flickr30k data shows the proposed method gives high quality image captions.\nAn interactive demo is ready to show at the workshop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10523v1",
            "date": "2019-05-25 05:28:03+00:00",
            "title": "Soft Contextual Data Augmentation for Neural Machine Translation",
            "authors": [
                "Jinhua Zhu",
                "Fei Gao",
                "Lijun Wu",
                "Yingce Xia",
                "Tao Qin",
                "Wengang Zhou",
                "Xueqi Cheng",
                "Tie-Yan Liu"
            ],
            "abstract": "While data augmentation is an important trick to boost the accuracy of deep\nlearning methods in computer vision tasks, its study in natural language tasks\nis still very limited. In this paper, we present a novel data augmentation\nmethod for neural machine translation. Different from previous augmentation\nmethods that randomly drop, swap or replace words with other words in a\nsentence, we softly augment a randomly chosen word in a sentence by its\ncontextual mixture of multiple related words. More accurately, we replace the\none-hot representation of a word by a distribution (provided by a language\nmodel) over the vocabulary, i.e., replacing the embedding of this word by a\nweighted combination of multiple semantically similar words. Since the weights\nof those words depend on the contextual information of the word to be replaced,\nthe newly generated sentences capture much richer information than previous\naugmentation methods. Experimental results on both small scale and large scale\nmachine translation datasets demonstrate the superiority of our method over\nstrong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10650v3",
            "date": "2019-05-25 18:27:28+00:00",
            "title": "Are Sixteen Heads Really Better than One?",
            "authors": [
                "Paul Michel",
                "Omer Levy",
                "Graham Neubig"
            ],
            "abstract": "Attention is a powerful and ubiquitous mechanism for allowing neural models\nto focus on particular salient pieces of information by taking their weighted\naverage when making predictions. In particular, multi-headed attention is a\ndriving force behind many recent state-of-the-art NLP models such as\nTransformer-based MT models and BERT. These models apply multiple attention\nmechanisms in parallel, with each attention \"head\" potentially focusing on\ndifferent parts of the input, which makes it possible to express sophisticated\nfunctions beyond the simple weighted average. In this paper we make the\nsurprising observation that even if models have been trained using multiple\nheads, in practice, a large percentage of attention heads can be removed at\ntest time without significantly impacting performance. In fact, some layers can\neven be reduced to a single head. We further examine greedy algorithms for\npruning down models, and the potential speed, memory efficiency, and accuracy\nimprovements obtainable therefrom. Finally, we analyze the results with respect\nto which parts of the model are more reliant on having multiple heads, and\nprovide precursory evidence that training dynamics play a role in the gains\nprovided by multi-head attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10726v2",
            "date": "2019-05-26 04:49:29+00:00",
            "title": "SemBleu: A Robust Metric for AMR Parsing Evaluation",
            "authors": [
                "Linfeng Song",
                "Daniel Gildea"
            ],
            "abstract": "Evaluating AMR parsing accuracy involves comparing pairs of AMR graphs. The\nmajor evaluation metric, SMATCH (Cai and Knight, 2013), searches for one-to-one\nmappings between the nodes of two AMRs with a greedy hill-climbing algorithm,\nwhich leads to search errors. We propose SEMBLEU, a robust metric that extends\nBLEU (Papineni et al., 2002) to AMRs. It does not suffer from search errors and\nconsiders non-local correspondences in addition to local ones. SEMBLEU is fully\ncontent-driven and punishes situations where a system's output does not\npreserve most information from the input. Preliminary experiments on both\nsentence and corpus levels show that SEMBLEU has slightly higher consistency\nwith human judgments than SMATCH. Our code is available at\nhttp://github.com/freesunshine0316/sembleu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10752v1",
            "date": "2019-05-26 07:45:44+00:00",
            "title": "TIGS: An Inference Algorithm for Text Infilling with Gradient Search",
            "authors": [
                "Dayiheng Liu",
                "Jie Fu",
                "Pengfei Liu",
                "Jiancheng Lv"
            ],
            "abstract": "Text infilling is defined as a task for filling in the missing part of a\nsentence or paragraph, which is suitable for many real-world natural language\ngeneration scenarios. However, given a well-trained sequential generative\nmodel, generating missing symbols conditioned on the context is challenging for\nexisting greedy approximate inference algorithms. In this paper, we propose an\niterative inference algorithm based on gradient search, which is the first\ninference algorithm that can be broadly applied to any neural sequence\ngenerative models for text infilling tasks. We compare the proposed method with\nstrong baselines on three text infilling tasks with various mask ratios and\ndifferent mask strategies. The results show that our proposed method is\neffective and efficient for fill-in-the-blank tasks, consistently outperforming\nall baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10810v1",
            "date": "2019-05-26 14:54:52+00:00",
            "title": "Evaluation of basic modules for isolated spelling error correction in Polish texts",
            "authors": [
                "Szymon Rutkowski"
            ],
            "abstract": "Spelling error correction is an important problem in natural language\nprocessing, as a prerequisite for good performance in downstream tasks as well\nas an important feature in user-facing applications. For texts in Polish\nlanguage, there exist works on specific error correction solutions, often\ndeveloped for dealing with specialized corpora, but not evaluations of many\ndifferent approaches on big resources of errors. We begin to address this\nproblem by testing some basic and promising methods on PlEWi, a corpus of\nannotated spelling extracted from Polish Wikipedia. These modules may be\nfurther combined with appropriate solutions for error detection and context\nawareness. Following our results, combining edit distance with cosine distance\nof semantic vectors may be suggested for interpretable systems, while an LSTM,\nparticularly enhanced by ELMo embeddings, seems to offer the best raw\nperformance.",
            "journal ref": "Human Language Technologies as a Challenge for Computer Science\n  and Linguistics. Ed. Zygmunt Vetulani and Patrick Paroubek. Pozna\\'n:\n  Wydawnictwo Nauka i Innowacje, 2019. Pp. 173-176",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10886v1",
            "date": "2019-05-26 21:35:21+00:00",
            "title": "Where's My Head? Definition, Dataset and Models for Numeric Fused-Heads Identification and Resolution",
            "authors": [
                "Yanai Elazar",
                "Yoav Goldberg"
            ],
            "abstract": "We provide the first computational treatment of fused-heads constructions\n(FH), focusing on the numeric fused-heads (NFH). FHs constructions are noun\nphrases (NPs) in which the head noun is missing and is said to be `fused' with\nits dependent modifier. This missing information is implicit and is important\nfor sentence understanding. The missing references are easily filled in by\nhumans but pose a challenge for computational models. We formulate the handling\nof FH as a two stages process: identification of the FH construction and\nresolution of the missing head. We explore the NFH phenomena in large corpora\nof English text and create (1) a dataset and a highly accurate method for NFH\nidentification; (2) a 10k examples (1M tokens) crowd-sourced dataset of NFH\nresolution; and (3) a neural baseline for the NFH resolution task. We release\nour code and dataset, in hope to foster further research into this challenging\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10892v1",
            "date": "2019-05-26 21:50:15+00:00",
            "title": "Extreme Multi-Label Legal Text Classification: A case study in EU Legislation",
            "authors": [
                "Ilias Chalkidis",
                "Manos Fergadiotis",
                "Prodromos Malakasiotis",
                "Nikolaos Aletras",
                "Ion Androutsopoulos"
            ],
            "abstract": "We consider the task of Extreme Multi-Label Text Classification (XMTC) in the\nlegal domain. We release a new dataset of 57k legislative documents from\nEURLEX, the European Union's public document database, annotated with concepts\nfrom EUROVOC, a multidisciplinary thesaurus. The dataset is substantially\nlarger than previous EURLEX datasets and suitable for XMTC, few-shot and\nzero-shot learning. Experimenting with several neural classifiers, we show that\nBIGRUs with self-attention outperform the current multi-label state-of-the-art\nmethods, which employ label-wise attention. Replacing CNNs with BIGRUs in\nlabel-wise attention networks leads to the best overall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11037v1",
            "date": "2019-05-27 08:28:51+00:00",
            "title": "Harry Potter and the Action Prediction Challenge from Natural Language",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We explore the challenge of action prediction from textual descriptions of\nscenes, a testbed to approximate whether text inference can be used to predict\nupcoming actions. As a case of study, we consider the world of the Harry Potter\nfantasy novels and inferring what spell will be cast next given a fragment of a\nstory. Spells act as keywords that abstract actions (e.g. 'Alohomora' to open a\ndoor) and denote a response to the environment. This idea is used to\nautomatically build HPAC, a corpus containing 82,836 samples and 85 actions. We\nthen evaluate different baselines. Among the tested models, an LSTM-based\napproach obtains the best performance for frequent actions and large scene\ndescriptions, but approaches such as logistic regression behave well on\ninfrequent actions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11558v1",
            "date": "2019-05-28 01:15:11+00:00",
            "title": "Leap-LSTM: Enhancing Long Short-Term Memory for Text Categorization",
            "authors": [
                "Ting Huang",
                "Gehui Shen",
                "Zhi-Hong Deng"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) are widely used in the field of natural\nlanguage processing (NLP), ranging from text categorization to question\nanswering and machine translation. However, RNNs generally read the whole text\nfrom beginning to end or vice versa sometimes, which makes it inefficient to\nprocess long texts. When reading a long document for a categorization task,\nsuch as topic categorization, large quantities of words are irrelevant and can\nbe skipped. To this end, we propose Leap-LSTM, an LSTM-enhanced model which\ndynamically leaps between words while reading texts. At each step, we utilize\nseveral feature encoders to extract messages from preceding texts, following\ntexts and the current word, and then determine whether to skip the current\nword. We evaluate Leap-LSTM on several text categorization tasks: sentiment\nanalysis, news categorization, ontology classification and topic\nclassification, with five benchmark data sets. The experimental results show\nthat our model reads faster and predicts better than standard LSTM. Compared to\nprevious models which can also skip words, our model achieves better trade-offs\nbetween performance and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11658v3",
            "date": "2019-05-28 07:46:50+00:00",
            "title": "DSReg: Using Distant Supervision as a Regularizer",
            "authors": [
                "Yuxian Meng",
                "Muyu Li",
                "Xiaoya Li",
                "Wei Wu",
                "Jiwei Li"
            ],
            "abstract": "In this paper, we aim at tackling a general issue in NLP tasks where some of\nthe negative examples are highly similar to the positive examples, i.e.,\nhard-negative examples. We propose the distant supervision as a regularizer\n(DSReg) approach to tackle this issue. The original task is converted to a\nmulti-task learning problem, in which distant supervision is used to retrieve\nhard-negative examples. The obtained hard-negative examples are then used as a\nregularizer. The original target objective of distinguishing positive examples\nfrom negative examples is jointly optimized with the auxiliary task objective\nof distinguishing softened positive (i.e., hard-negative examples plus positive\nexamples) from easy-negative examples. In the neural context, this can be done\nby outputting the same representation from the last neural layer to different\n$softmax$ functions. Using this strategy, we can improve the performance of\nbaseline models in a range of different NLP tasks, including text\nclassification, sequence labeling and reading comprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11684v1",
            "date": "2019-05-28 08:46:56+00:00",
            "title": "On Measuring Gender Bias in Translation of Gender-neutral Pronouns",
            "authors": [
                "Won Ik Cho",
                "Ji Won Kim",
                "Seok Min Kim",
                "Nam Soo Kim"
            ],
            "abstract": "Ethics regarding social bias has recently thrown striking issues in natural\nlanguage processing. Especially for gender-related topics, the need for a\nsystem that reduces the model bias has grown in areas such as image captioning,\ncontent recommendation, and automated employment. However, detection and\nevaluation of gender bias in the machine translation systems are not yet\nthoroughly investigated, for the task being cross-lingual and challenging to\ndefine. In this paper, we propose a scheme for making up a test set that\nevaluates the gender bias in a machine translation system, with Korean, a\nlanguage with gender-neutral pronouns. Three word/phrase sets are primarily\nconstructed, each incorporating positive/negative expressions or occupations;\nall the terms are gender-independent or at least not biased to one side\nseverely. Then, additional sentence lists are constructed concerning formality\nof the pronouns and politeness of the sentences. With the generated sentence\nset of size 4,236 in total, we evaluate gender bias in conventional machine\ntranslation systems utilizing the proposed measure, which is termed here as\ntranslation gender bias index (TGBI). The corpus and the code for evaluation is\navailable on-line.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11806v3",
            "date": "2019-05-28 13:38:29+00:00",
            "title": "An Incremental Turn-Taking Model For Task-Oriented Dialog Systems",
            "authors": [
                "Andrei C. Coman",
                "Koichiro Yoshino",
                "Yukitoshi Murase",
                "Satoshi Nakamura",
                "Giuseppe Riccardi"
            ],
            "abstract": "In a human-machine dialog scenario, deciding the appropriate time for the\nmachine to take the turn is an open research problem. In contrast, humans\nengaged in conversations are able to timely decide when to interrupt the\nspeaker for competitive or non-competitive reasons. In state-of-the-art\nturn-by-turn dialog systems the decision on the next dialog action is taken at\nthe end of the utterance. In this paper, we propose a token-by-token prediction\nof the dialog state from incremental transcriptions of the user utterance. To\nidentify the point of maximal understanding in an ongoing utterance, we a)\nimplement an incremental Dialog State Tracker which is updated on a token basis\n(iDST) b) re-label the Dialog State Tracking Challenge 2 (DSTC2) dataset and c)\nadapt it to the incremental turn-taking experimental scenario. The re-labeling\nconsists of assigning a binary value to each token in the user utterance that\nallows to identify the appropriate point for taking the turn. Finally, we\nimplement an incremental Turn Taking Decider (iTTD) that is trained on these\nnew labels for the turn-taking decision. We show that the proposed model can\nachieve a better performance compared to a deterministic handcrafted\nturn-taking algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11836v1",
            "date": "2019-05-27 08:54:26+00:00",
            "title": "Specific polysemy of the brief sapiential units",
            "authors": [
                "Marie-Christine Bornes-Varol",
                "Marie-Sol Ortola",
                "Gronoff Jean-Daniel"
            ],
            "abstract": "In this paper we explain how we deal with the problems related to the\nconstitution of the Aliento database, the complexity of which has to do with\nthe type of phrases we work with, the differences between languages, the type\nof information we want to see emerge. The correct tagging of the specific\npolysemy of brief sapiential units is an important step in the preparation of\nthe text within the corpus which will be submitted to compute similarities and\nposterity of the units.",
            "journal ref": "Annual Meeting of the German Society for Computer Science -\n  Informatik 2010, Sep 2010, Leipzig, Germany. pp.523 - 529",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11901v1",
            "date": "2019-05-28 15:59:21+00:00",
            "title": "Revisiting Low-Resource Neural Machine Translation: A Case Study",
            "authors": [
                "Rico Sennrich",
                "Biao Zhang"
            ],
            "abstract": "It has been shown that the performance of neural machine translation (NMT)\ndrops starkly in low-resource conditions, underperforming phrase-based\nstatistical machine translation (PBSMT) and requiring large amounts of\nauxiliary data to achieve competitive results. In this paper, we re-assess the\nvalidity of these results, arguing that they are the result of lack of system\nadaptation to low-resource settings. We discuss some pitfalls to be aware of\nwhen training low-resource NMT systems, and recent techniques that have shown\nto be especially helpful in low-resource settings, resulting in a set of best\npractices for low-resource NMT. In our experiments on German--English with\ndifferent amounts of IWSLT14 training data, we show that, without the use of\nany auxiliary monolingual or multilingual data, an optimized NMT system can\noutperform PBSMT with far less data than previously claimed. We also apply\nthese techniques to a low-resource Korean-English dataset, surpassing\npreviously reported results by 4 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11912v2",
            "date": "2019-05-28 16:17:41+00:00",
            "title": "A Cross-Domain Transferable Neural Coherence Model",
            "authors": [
                "Peng Xu",
                "Hamidreza Saghir",
                "Jin Sung Kang",
                "Teng Long",
                "Avishek Joey Bose",
                "Yanshuai Cao",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Coherence is an important aspect of text quality and is crucial for ensuring\nits readability. One important limitation of existing coherence models is that\ntraining on one domain does not easily generalize to unseen categories of text.\nPrevious work advocates for generative models for cross-domain generalization,\nbecause for discriminative models, the space of incoherent sentence orderings\nto discriminate against during training is prohibitively large. In this work,\nwe propose a local discriminative neural model with a much smaller negative\nsampling space that can efficiently learn against incorrect orderings. The\nproposed coherence model is simple in structure, yet it significantly\noutperforms previous state-of-art methods on a standard benchmark dataset on\nthe Wall Street Journal corpus, as well as in multiple new challenging settings\nof transfer to unseen categories of discourse on Wikipedia articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12065v1",
            "date": "2019-05-28 20:11:47+00:00",
            "title": "Automatic Ambiguity Detection",
            "authors": [
                "Richard Sproat",
                "Jan van Santen"
            ],
            "abstract": "Most work on sense disambiguation presumes that one knows beforehand -- e.g.\nfrom a thesaurus -- a set of polysemous terms. But published lists invariably\ngive only partial coverage. For example, the English word tan has several\nobvious senses, but one may overlook the abbreviation for tangent. In this\npaper, we present an algorithm for identifying interesting polysemous terms and\nmeasuring their degree of polysemy, given an unlabeled corpus. The algorithm\ninvolves: (i) collecting all terms within a k-term window of the target term;\n(ii) computing the inter-term distances of the contextual terms, and reducing\nthe multi-dimensional distance space to two dimensions using standard methods;\n(iii) converting the two-dimensional representation into radial coordinates and\nusing isotonic/antitonic regression to compute the degree to which the\ndistribution deviates from a single-peak model. The amount of deviation is the\nproposed polysemy index",
            "journal ref": "International Conference on Spoken Language Processing, 1998",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12069v1",
            "date": "2019-05-28 20:21:19+00:00",
            "title": "SEMA: an Extended Semantic Evaluation Metric for AMR",
            "authors": [
                "Rafael T. Anchieta",
                "Marco A. S. Cabezudo",
                "Thiago A. S. Pardo"
            ],
            "abstract": "Abstract Meaning Representation (AMR) is a recently designed semantic\nrepresentation language intended to capture the meaning of a sentence, which\nmay be represented as a single-rooted directed acyclic graph with labeled nodes\nand edges. The automatic evaluation of this structure plays an important role\nin the development of better systems, as well as for semantic annotation.\nDespite there is one available metric, smatch, it has some drawbacks. For\ninstance, smatch creates a self-relation on the root of the graph, has weights\nfor different error types, and does not take into account the dependence of the\nelements in the AMR structure. With these drawbacks, smatch masks several\nproblems of the AMR parsers and distorts the evaluation of the AMRs. In view of\nthis, in this paper, we introduce an extended metric to evaluate AMR parsers,\nwhich deals with the drawbacks of the smatch metric. Finally, we compare both\nmetrics, using four well-known AMR parsers, and we argue that our metric is\nmore refined, robust, fairer, and faster than smatch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12198v1",
            "date": "2019-05-29 03:32:38+00:00",
            "title": "Ensuring Readability and Data-fidelity using Head-modifier Templates in Deep Type Description Generation",
            "authors": [
                "Jiangjie Chen",
                "Ao Wang",
                "Haiyun Jiang",
                "Suo Feng",
                "Chenguang Li",
                "Yanghua Xiao"
            ],
            "abstract": "A type description is a succinct noun compound which helps human and machines\nto quickly grasp the informative and distinctive information of an entity.\nEntities in most knowledge graphs (KGs) still lack such descriptions, thus\ncalling for automatic methods to supplement such information. However, existing\ngenerative methods either overlook the grammatical structure or make factual\nmistakes in generated texts. To solve these problems, we propose a\nhead-modifier template-based method to ensure the readability and data fidelity\nof generated type descriptions. We also propose a new dataset and two automatic\nmetrics for this task. Experiments show that our method improves substantially\ncompared with baselines and achieves state-of-the-art performance on both\ndatasets.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics. 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12277v1",
            "date": "2019-05-29 08:58:52+00:00",
            "title": "Learning Task-specific Representation for Novel Words in Sequence Labeling",
            "authors": [
                "Minlong Peng",
                "Qi Zhang",
                "Xiaoyu Xing",
                "Tao Gui",
                "Jinlan Fu",
                "Xuanjing Huang"
            ],
            "abstract": "Word representation is a key component in neural-network-based sequence\nlabeling systems. However, representations of unseen or rare words trained on\nthe end task are usually poor for appreciable performance. This is commonly\nreferred to as the out-of-vocabulary (OOV) problem. In this work, we address\nthe OOV problem in sequence labeling using only training data of the task. To\nthis end, we propose a novel method to predict representations for OOV words\nfrom their surface-forms (e.g., character sequence) and contexts. The method is\nspecifically designed to avoid the error propagation problem suffered by\nexisting approaches in the same paradigm. To evaluate its effectiveness, we\nperformed extensive empirical studies on four part-of-speech tagging (POS)\ntasks and four named entity recognition (NER) tasks. Experimental results show\nthat the proposed method can achieve better or competitive performance on the\nOOV problem compared with existing state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12304v3",
            "date": "2019-05-29 10:07:12+00:00",
            "title": "Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning",
            "authors": [
                "Dayiheng Liu",
                "Jie Fu",
                "Yidan Zhang",
                "Chris Pal",
                "Jiancheng Lv"
            ],
            "abstract": "Typical methods for unsupervised text style transfer often rely on two key\ningredients: 1) seeking the explicit disentanglement of the content and the\nattributes, and 2) troublesome adversarial learning. In this paper, we show\nthat neither of these components is indispensable. We propose a new framework\nthat utilizes the gradients to revise the sentence in a continuous space during\ninference to achieve text style transfer. Our method consists of three key\ncomponents: a variational auto-encoder (VAE), some attribute predictors (one\nfor each attribute), and a content predictor. The VAE and the two types of\npredictors enable us to perform gradient-based optimization in the continuous\nspace, which is mapped from sentences in a discrete space, to find the\nrepresentation of a target sentence with the desired attributes and preserved\ncontent. Moreover, the proposed method naturally has the ability to\nsimultaneously manipulate multiple fine-grained attributes, such as sentence\nlength and the presence of specific words, when performing text style transfer\ntasks. Compared with previous adversarial learning based methods, the proposed\nmethod is more interpretable, controllable and easier to train. Extensive\nexperimental studies on three popular text style transfer tasks show that the\nproposed method significantly outperforms five state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12335v2",
            "date": "2019-05-29 11:28:37+00:00",
            "title": "TopExNet: Entity-Centric Network Topic Exploration in News Streams",
            "authors": [
                "Andreas Spitz",
                "Satya Almasian",
                "Michael Gertz"
            ],
            "abstract": "The recent introduction of entity-centric implicit network representations of\nunstructured text offers novel ways for exploring entity relations in document\ncollections and streams efficiently and interactively. Here, we present\nTopExNet as a tool for exploring entity-centric network topics in streams of\nnews articles. The application is available as a web service at\nhttps://topexnet.ifi.uni-heidelberg.de/ .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12598v2",
            "date": "2019-05-29 17:20:11+00:00",
            "title": "Towards better substitution-based word sense induction",
            "authors": [
                "Asaf Amrami",
                "Yoav Goldberg"
            ],
            "abstract": "Word sense induction (WSI) is the task of unsupervised clustering of word\nusages within a sentence to distinguish senses. Recent work obtain strong\nresults by clustering lexical substitutes derived from pre-trained RNN language\nmodels (ELMo). Adapting the method to BERT improves the scores even further. We\nextend the previous method to support a dynamic rather than a fixed number of\nclusters as supported by other prominent methods, and propose a method for\ninterpreting the resulting clusters by associating them with their most\ninformative substitutes. We then perform extensive error analysis revealing the\nremaining sources of errors in the WSI task.\n  Our code is available at https://github.com/asafamr/bertwsi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12676v2",
            "date": "2019-05-29 18:49:03+00:00",
            "title": "The (Non-)Utility of Structural Features in BiLSTM-based Dependency Parsers",
            "authors": [
                "Agnieszka Falenska",
                "Jonas Kuhn"
            ],
            "abstract": "Classical non-neural dependency parsers put considerable effort on the design\nof feature functions. Especially, they benefit from information coming from\nstructural features, such as features drawn from neighboring tokens in the\ndependency tree. In contrast, their BiLSTM-based successors achieve\nstate-of-the-art performance without explicit information about the structural\ncontext. In this paper we aim to answer the question: How much structural\ncontext are the BiLSTM representations able to capture implicitly? We show that\nfeatures drawn from partial subtrees become redundant when the BiLSTMs are\nused. We provide a deep insight into information flow in transition- and\ngraph-based neural architectures to demonstrate where the implicit information\ncomes from when the parsers make their decisions. Finally, with model ablations\nwe demonstrate that the structural context is not only present in the models,\nbut it significantly influences their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12688v2",
            "date": "2019-05-29 19:19:47+00:00",
            "title": "Choosing Transfer Languages for Cross-Lingual Learning",
            "authors": [
                "Yu-Hsiang Lin",
                "Chian-Yu Chen",
                "Jean Lee",
                "Zirui Li",
                "Yuyan Zhang",
                "Mengzhou Xia",
                "Shruti Rijhwani",
                "Junxian He",
                "Zhisong Zhang",
                "Xuezhe Ma",
                "Antonios Anastasopoulos",
                "Patrick Littell",
                "Graham Neubig"
            ],
            "abstract": "Cross-lingual transfer, where a high-resource transfer language is used to\nimprove the accuracy of a low-resource task language, is now an invaluable tool\nfor improving performance of natural language processing (NLP) on low-resource\nlanguages. However, given a particular task language, it is not clear which\nlanguage to transfer from, and the standard strategy is to select languages\nbased on ad hoc criteria, usually the intuition of the experimenter. Since a\nlarge number of features contribute to the success of cross-lingual transfer\n(including phylogenetic similarity, typological properties, lexical overlap, or\nsize of available data), even the most enlightened experimenter rarely\nconsiders all these factors for the particular task at hand. In this paper, we\nconsider this task of automatically selecting optimal transfer languages as a\nranking problem, and build models that consider the aforementioned features to\nperform this prediction. In experiments on representative NLP tasks, we\ndemonstrate that our model predicts good transfer languages much better than ad\nhoc baselines considering single features in isolation, and glean insights on\nwhat features are most informative for each different NLP tasks, which may\ninform future ad hoc selection even without use of our method. Code, data, and\npre-trained models are available at https://github.com/neulab/langrank",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12713v1",
            "date": "2019-05-29 20:40:24+00:00",
            "title": "Geolocating Political Events in Text",
            "authors": [
                "Andrew Halterman"
            ],
            "abstract": "This work introduces a general method for automatically finding the locations\nwhere political events in text occurred. Using a novel set of 8,000 labeled\nsentences, I create a method to link automatically extracted events and\nlocations in text. The model achieves human level performance on the annotation\ntask and outperforms previous event geolocation systems. It can be applied to\nmost event extraction systems across geographic contexts. I formalize the\nevent--location linking task, describe the neural network model, describe the\npotential uses of such a system in political science, and demonstrate a\nworkflow to answer an open question on the role of conventional military\noffensives in causing civilian casualties in the Syrian civil war.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12801v2",
            "date": "2019-05-30 00:43:02+00:00",
            "title": "Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function",
            "authors": [
                "Yusu Qian",
                "Urwa Muaz",
                "Ben Zhang",
                "Jae Won Hyun"
            ],
            "abstract": "Gender bias exists in natural language datasets which neural language models\ntend to learn, resulting in biased text generation. In this research, we\npropose a debiasing approach based on the loss function modification. We\nintroduce a new term to the loss function which attempts to equalize the\nprobabilities of male and female words in the output. Using an array of bias\nevaluation metrics, we provide empirical evidence that our approach\nsuccessfully mitigates gender bias in language models without increasing\nperplexity. In comparison to existing debiasing strategies, data augmentation,\nand word embedding debiasing, our method performs better in several aspects,\nespecially in reducing gender bias in occupation words. Finally, we introduce a\ncombination of data augmentation and our approach, and show that it outperforms\nexisting strategies in all bias evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12848v1",
            "date": "2019-05-30 04:28:19+00:00",
            "title": "A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension",
            "authors": [
                "Yasuhito Ohsugi",
                "Itsumi Saito",
                "Kyosuke Nishida",
                "Hisako Asano",
                "Junji Tomita"
            ],
            "abstract": "Conversational machine comprehension (CMC) requires understanding the context\nof multi-turn dialogue. Using BERT, a pre-training language model, has been\nsuccessful for single-turn machine comprehension, while modeling multiple turns\nof question answering with BERT has not been established because BERT has a\nlimit on the number and the length of input sequences. In this paper, we\npropose a simple but effective method with BERT for CMC. Our method uses BERT\nto encode a paragraph independently conditioned with each question and each\nanswer in a multi-turn context. Then, the method predicts an answer on the\nbasis of the paragraph representations encoded with BERT. The experiments with\nrepresentative CMC datasets, QuAC and CoQA, show that our method outperformed\nrecently published methods (+0.8 F1 on QuAC and +2.1 F1 on CoQA). In addition,\nwe conducted a detailed analysis of the effects of the number and types of\ndialogue history on the accuracy of CMC, and we found that the gold answer\nhistory, which may not be given in an actual conversation, contributed to the\nmodel performance most on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12926v2",
            "date": "2019-05-30 09:32:03+00:00",
            "title": "Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation",
            "authors": [
                "Ke Wang",
                "Hang Hua",
                "Xiaojun Wan"
            ],
            "abstract": "Unsupervised text attribute transfer automatically transforms a text to alter\na specific attribute (e.g. sentiment) without using any parallel data, while\nsimultaneously preserving its attribute-independent content. The dominant\napproaches are trying to model the content-independent attribute separately,\ne.g., learning different attributes' representations or using multiple\nattribute-specific decoders. However, it may lead to inflexibility from the\nperspective of controlling the degree of transfer or transferring over multiple\naspects at the same time. To address the above problems, we propose a more\nflexible unsupervised text attribute transfer framework which replaces the\nprocess of modeling attribute with minimal editing of latent representations\nbased on an attribute classifier. Specifically, we first propose a\nTransformer-based autoencoder to learn an entangled latent representation for a\ndiscrete text, then we transform the attribute transfer task to an optimization\nproblem and propose the Fast-Gradient-Iterative-Modification algorithm to edit\nthe latent representation until conforming to the target attribute. Extensive\nexperimental results demonstrate that our model achieves very competitive\nperformance on three public data sets. Furthermore, we also show that our model\ncan not only control the degree of transfer freely but also allow to transfer\nover multiple aspects at the same time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13068v2",
            "date": "2019-05-30 14:17:39+00:00",
            "title": "Unbabel's Submission to the WMT2019 APE Shared Task: BERT-based Encoder-Decoder for Automatic Post-Editing",
            "authors": [
                "Ant\u00f3nio V. Lopes",
                "M. Amin Farajian",
                "Gon\u00e7alo M. Correia",
                "Jonay Trenous",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "This paper describes Unbabel's submission to the WMT2019 APE Shared Task for\nthe English-German language pair. Following the recent rise of large, powerful,\npre-trained models, we adapt the BERT pretrained model to perform Automatic\nPost-Editing in an encoder-decoder framework. Analogously to dual-encoder\narchitectures we develop a BERT-based encoder-decoder (BED) model in which a\nsingle pretrained BERT encoder receives both the source src and machine\ntranslation tgt strings. Furthermore, we explore a conservativeness factor to\nconstrain the APE system to perform fewer edits. As the official results show,\nwhen trained on a weighted combination of in-domain and artificial training\ndata, our BED system with the conservativeness penalty improves significantly\nthe translations of a strong Neural Machine Translation system by $-0.78$ and\n$+1.23$ in terms of TER and BLEU, respectively. Finally, our submission\nachieves a new state-of-the-art, ex-aequo, in English-German APE of NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13319v1",
            "date": "2019-05-30 21:28:12+00:00",
            "title": "MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms",
            "authors": [
                "Aida Amini",
                "Saadia Gabriel",
                "Peter Lin",
                "Rik Koncel-Kedziorski",
                "Yejin Choi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce a large-scale dataset of math word problems and an interpretable\nneural math problem solver that learns to map problems to operation programs.\nDue to annotation challenges, current datasets in this domain have been either\nrelatively small in scale or did not offer precise operational annotations over\ndiverse problem types. We introduce a new representation language to model\nprecise operation programs corresponding to each math problem that aim to\nimprove both the performance and the interpretability of the learned models.\nUsing this representation language, our new dataset, MathQA, significantly\nenhances the AQuA dataset with fully-specified operational programs. We\nadditionally introduce a neural sequence-to-program model enhanced with\nautomatic problem categorization. Our experiments show improvements over\ncompetitive baselines in our MathQA as well as the AQuA dataset. The results\nare still significantly lower than human performance indicating that the\ndataset poses new challenges for future research. Our dataset is available at:\nhttps://math-qa.github.io/math-QA/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13322v2",
            "date": "2019-05-30 21:37:45+00:00",
            "title": "Assessing The Factual Accuracy of Generated Text",
            "authors": [
                "Ben Goodrich",
                "Vinay Rao",
                "Mohammad Saleh",
                "Peter J Liu"
            ],
            "abstract": "We propose a model-based metric to estimate the factual accuracy of generated\ntext that is complementary to typical scoring schemes like ROUGE\n(Recall-Oriented Understudy for Gisting Evaluation) and BLEU (Bilingual\nEvaluation Understudy). We introduce and release a new large-scale dataset\nbased on Wikipedia and Wikidata to train relation classifiers and end-to-end\nfact extraction models. The end-to-end models are shown to be able to extract\ncomplete sets of facts from datasets with full pages of text. We then analyse\nmultiple models that estimate factual accuracy on a Wikipedia text\nsummarization task, and show their efficacy compared to ROUGE and other\nmodel-free variants by conducting a human evaluation study.",
            "journal ref": "The 25th ACM SIGKDD Conference on Knowledge Discovery and Data\n  Mining (KDD '19), August 4--8, 2019, Anchorage, AK, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13324v1",
            "date": "2019-05-30 21:40:46+00:00",
            "title": "A Lightweight Recurrent Network for Sequence Modeling",
            "authors": [
                "Biao Zhang",
                "Rico Sennrich"
            ],
            "abstract": "Recurrent networks have achieved great success on various sequential tasks\nwith the assistance of complex recurrent units, but suffer from severe\ncomputational inefficiency due to weak parallelization. One direction to\nalleviate this issue is to shift heavy computations outside the recurrence. In\nthis paper, we propose a lightweight recurrent network, or LRN. LRN uses input\nand forget gates to handle long-range dependencies as well as gradient\nvanishing and explosion, with all parameter related calculations factored\noutside the recurrence. The recurrence in LRN only manipulates the weight\nassigned to each token, tightly connecting LRN with self-attention networks. We\napply LRN as a drop-in replacement of existing recurrent units in several\nneural sequential models. Extensive experiments on six NLP tasks show that LRN\nyields the best running efficiency with little or no loss in model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13326v1",
            "date": "2019-05-30 21:43:57+00:00",
            "title": "Grammar-based Neural Text-to-SQL Generation",
            "authors": [
                "Kevin Lin",
                "Ben Bogin",
                "Mark Neumann",
                "Jonathan Berant",
                "Matt Gardner"
            ],
            "abstract": "The sequence-to-sequence paradigm employed by neural text-to-SQL models\ntypically performs token-level decoding and does not consider generating SQL\nhierarchically from a grammar. Grammar-based decoding has shown significant\nimprovements for other semantic parsing tasks, but SQL and other general\nprogramming languages have complexities not present in logical formalisms that\nmake writing hierarchical grammars difficult. We introduce techniques to handle\nthese complexities, showing how to construct a schema-dependent grammar with\nminimal over-generation. We analyze these techniques on ATIS and Spider, two\nchallenging text-to-SQL datasets, demonstrating that they yield 14--18\\%\nrelative reductions in error.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13354v1",
            "date": "2019-05-30 23:41:35+00:00",
            "title": "DiaBLa: A Corpus of Bilingual Spontaneous Written Dialogues for Machine Translation",
            "authors": [
                "Rachel Bawden",
                "Sophie Rosset",
                "Thomas Lavergne",
                "Eric Bilinski"
            ],
            "abstract": "We present a new English-French test set for the evaluation of Machine\nTranslation (MT) for informal, written bilingual dialogue. The test set\ncontains 144 spontaneous dialogues (5,700+ sentences) between native English\nand French speakers, mediated by one of two neural MT systems in a range of\nrole-play settings. The dialogues are accompanied by fine-grained\nsentence-level judgments of MT quality, produced by the dialogue participants\nthemselves, as well as by manually normalised versions and reference\ntranslations produced a posteriori. The motivation for the corpus is two-fold:\nto provide (i) a unique resource for evaluating MT models, and (ii) a corpus\nfor the analysis of MT-mediated communication. We provide a preliminary\nanalysis of the corpus to confirm that the participants' judgments reveal\nperceptible differences in MT quality between the two MT systems used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13359v1",
            "date": "2019-05-31 00:08:27+00:00",
            "title": "Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data",
            "authors": [
                "Fahad AlGhamdi",
                "Mona Diab"
            ],
            "abstract": "Linguistic Code Switching (CS) is a phenomenon that occurs when multilingual\nspeakers alternate between two or more languages/dialects within a single\nconversation. Processing CS data is especially challenging in intra-sentential\ndata given state-of-the-art monolingual NLP technologies since such\ntechnologies are geared toward the processing of one language at a time. In\nthis paper, we address the problem of Part-of-Speech tagging (POS) in the\ncontext of linguistic code switching (CS). We explore leveraging multiple\nneural network architectures to measure the impact of different pre-trained\nembeddings methods on POS tagging CS data. We investigate the landscape in four\nCS language pairs, Spanish-English, Hindi-English, Modern Standard Arabic-\nEgyptian Arabic dialect (MSA-EGY), and Modern Standard Arabic- Levantine Arabic\ndialect (MSA-LEV). Our results show that multilingual embedding (e.g., MSA-EGY\nand MSA-LEV) helps closely related languages (EGY/LEV) but adds noise to the\nlanguages that are distant (SPA/HIN). Finally, we show that our proposed models\noutperform state-of-the-art CS taggers for MSA-EGY language pair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13370v1",
            "date": "2019-05-31 01:15:36+00:00",
            "title": "Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning",
            "authors": [
                "Tahira Naseem",
                "Abhishek Shah",
                "Hui Wan",
                "Radu Florian",
                "Salim Roukos",
                "Miguel Ballesteros"
            ],
            "abstract": "Our work involves enriching the Stack-LSTM transition-based AMR parser\n(Ballesteros and Al-Onaizan, 2017) by augmenting training with Policy Learning\nand rewarding the Smatch score of sampled graphs. In addition, we also combined\nseveral AMR-to-text alignments with an attention mechanism and we supplemented\nthe parser with pre-processed concept identification, named entities and\ncontextualized embeddings. We achieve a highly competitive performance that is\ncomparable to the best published results. We show an in-depth study ablating\neach of the new components of the parser",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13413v1",
            "date": "2019-05-31 05:02:50+00:00",
            "title": "Improving Open Information Extraction via Iterative Rank-Aware Learning",
            "authors": [
                "Zhengbao Jiang",
                "Pengcheng Yin",
                "Graham Neubig"
            ],
            "abstract": "Open information extraction (IE) is the task of extracting open-domain\nassertions from natural language sentences. A key step in open IE is confidence\nmodeling, ranking the extractions based on their estimated quality to adjust\nprecision and recall of extracted assertions. We found that the extraction\nlikelihood, a confidence measure used by current supervised open IE systems, is\nnot well calibrated when comparing the quality of assertions extracted from\ndifferent sentences. We propose an additional binary classification loss to\ncalibrate the likelihood to make it more globally comparable, and an iterative\nlearning process, where extractions generated by the open IE model are\nincrementally included as training samples to help the model learn from trial\nand error. Experiments on OIE2016 demonstrate the effectiveness of our method.\nCode and data are available at https://github.com/jzbjyb/oie_rank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13416v1",
            "date": "2019-05-31 05:06:00+00:00",
            "title": "Fine-Grained Spoiler Detection from Large-Scale Review Corpora",
            "authors": [
                "Mengting Wan",
                "Rishabh Misra",
                "Ndapa Nakashole",
                "Julian McAuley"
            ],
            "abstract": "This paper presents computational approaches for automatically detecting\ncritical plot twists in reviews of media products. First, we created a\nlarge-scale book review dataset that includes fine-grained spoiler annotations\nat the sentence-level, as well as book and (anonymized) user information.\nSecond, we carefully analyzed this dataset, and found that: spoiler language\ntends to be book-specific; spoiler distributions vary greatly across books and\nreview authors; and spoiler sentences tend to jointly appear in the latter part\nof reviews. Third, inspired by these findings, we developed an end-to-end\nneural network architecture to detect spoiler sentences in review corpora.\nQuantitative and qualitative results demonstrate that the proposed method\nsubstantially outperforms existing baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13443v1",
            "date": "2019-05-31 07:03:03+00:00",
            "title": "Symbol Emergence as an Interpersonal Multimodal Categorization",
            "authors": [
                "Yoshinobu Hagiwara",
                "Hiroyoshi Kobayashi",
                "Akira Taniguchi",
                "Tadahiro Taniguchi"
            ],
            "abstract": "This study focuses on category formation for individual agents and the\ndynamics of symbol emergence in a multi-agent system through semiotic\ncommunication. Semiotic communication is defined, in this study, as the\ngeneration and interpretation of signs associated with the categories formed\nthrough the agent's own sensory experience or by exchange of signs with other\nagents. From the viewpoint of language evolution and symbol emergence,\norganization of a symbol system in a multi-agent system is considered as a\nbottom-up and dynamic process, where individual agents share the meaning of\nsigns and categorize sensory experience. A constructive computational model can\nexplain the mutual dependency of the two processes and has mathematical support\nthat guarantees a symbol system's emergence and sharing within the multi-agent\nsystem. In this paper, we describe a new computational model that represents\nsymbol emergence in a two-agent system based on a probabilistic generative\nmodel for multimodal categorization. It models semiotic communication via a\nprobabilistic rejection based on the receiver's own belief. We have found that\nthe dynamics by which cognitively independent agents create a symbol system\nthrough their semiotic communication can be regarded as the inference process\nof a hidden variable in an interpersonal multimodal categorizer, if we define\nthe rejection probability based on the Metropolis-Hastings algorithm. The\nvalidity of the proposed model and algorithm for symbol emergence is also\nverified in an experiment with two agents observing daily objects in the\nreal-world environment. The experimental results demonstrate that our model\nreproduces the phenomena of symbol emergence, which does not require a teacher\nwho would know a pre-existing symbol system. Instead, the multi-agent system\ncan form and use a symbol system without having pre-existing categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13464v3",
            "date": "2019-05-31 08:42:27+00:00",
            "title": "Effective writing style imitation via combinatorial paraphrasing",
            "authors": [
                "Tommi Gr\u00f6ndahl",
                "N. Asokan"
            ],
            "abstract": "Stylometry can be used to profile or deanonymize authors against their will\nbased on writing style. Style transfer provides a defence. Current techniques\ntypically use either encoder-decoder architectures or rule-based algorithms.\nCrucially, style transfer must reliably retain original semantic content to be\nactually deployable. We conduct a multifaceted evaluation of three\nstate-of-the-art encoder-decoder style transfer techniques, and show that all\nfail at semantic retainment. In particular, they do not produce appropriate\nparaphrases, but only retain original content in the trivial case of exactly\nreproducing the text. To mitigate this problem we propose ParChoice: a\ntechnique based on the combinatorial application of multiple paraphrasing\nalgorithms. ParChoice strongly outperforms the encoder-decoder baselines in\nsemantic retainment. Additionally, compared to baselines that achieve\nnon-negligible semantic retainment, ParChoice has superior style transfer\nperformance. We also apply ParChoice to multi-author style imitation (not\nconsidered by prior work), where we achieve up to 75% imitation success among\nfive authors. Furthermore, when compared to two state-of-the-art rule-based\nstyle transfer techniques, ParChoice has markedly better semantic retainment.\nCombining ParChoice with the best performing rule-based baseline (Mutant-X)\nalso reaches the highest style transfer success on the Brennan-Greenstadt and\nExtended-Brennan-Greenstadt corpora, with much less impact on original meaning\nthan when using the rule-based baseline techniques alone. Finally, we highlight\na critical problem that afflicts all current style transfer techniques: the\nadversary can use the same technique for thwarting style transfer via\nadversarial training. We show that adding randomness to style transfer helps to\nmitigate the effectiveness of adversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13656v3",
            "date": "2019-05-31 14:54:46+00:00",
            "title": "Investigating an Effective Character-level Embedding in Korean Sentence Classification",
            "authors": [
                "Won Ik Cho",
                "Seok Min Kim",
                "Nam Soo Kim"
            ],
            "abstract": "Different from the writing systems of many Romance and Germanic languages,\nsome languages or language families show complex conjunct forms in character\ncomposition. For such cases where the conjuncts consist of the components\nrepresenting consonant(s) and vowel, various character encoding schemes can be\nadopted beyond merely making up a one-hot vector. However, there has been\nlittle work done on intra-language comparison regarding performances using each\nrepresentation. In this study, utilizing the Korean language which is\ncharacter-rich and agglutinative, we investigate an encoding scheme that is the\nmost effective among Jamo-level one-hot, character-level one-hot,\ncharacter-level dense, and character-level multi-hot. Classification\nperformance with each scheme is evaluated on two corpora: one on binary\nsentiment analysis of movie reviews, and the other on multi-class\nidentification of intention types. The result displays that the character-level\nfeatures show higher performance in general, although the Jamo-level features\nmay show compatibility with the attention-based models if guaranteed adequate\nparameter set size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13714v1",
            "date": "2019-05-31 16:49:57+00:00",
            "title": "Do Human Rationales Improve Machine Explanations?",
            "authors": [
                "Julia Strout",
                "Ye Zhang",
                "Raymond J. Mooney"
            ],
            "abstract": "Work on \"learning with rationales\" shows that humans providing explanations\nto a machine learning system can improve the system's predictive accuracy.\nHowever, this work has not been connected to work in \"explainable AI\" which\nconcerns machines explaining their reasoning to humans. In this work, we show\nthat learning with rationales can also improve the quality of the machine's\nexplanations as evaluated by human judges. Specifically, we present experiments\nshowing that, for CNN- based text classification, explanations generated using\n\"supervised attention\" are judged superior to explanations generated using\nnormal unsupervised attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00048v1",
            "date": "2019-05-31 19:57:49+00:00",
            "title": "Thinking Slow about Latency Evaluation for Simultaneous Machine Translation",
            "authors": [
                "Colin Cherry",
                "George Foster"
            ],
            "abstract": "Simultaneous machine translation attempts to translate a source sentence\nbefore it is finished being spoken, with applications to translation of spoken\nlanguage for live streaming and conversation. Since simultaneous systems trade\nquality to reduce latency, having an effective and interpretable latency metric\nis crucial. We introduce a variant of the recently proposed Average Lagging\n(AL) metric, which we call Differentiable Average Lagging (DAL). It\ndistinguishes itself by being differentiable and internally consistent to its\nunderlying mathematical model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00072v1",
            "date": "2019-05-31 20:45:15+00:00",
            "title": "Improving the Similarity Measure of Determinantal Point Processes for Extractive Multi-Document Summarization",
            "authors": [
                "Sangwoo Cho",
                "Logan Lebanoff",
                "Hassan Foroosh",
                "Fei Liu"
            ],
            "abstract": "The most important obstacles facing multi-document summarization include\nexcessive redundancy in source descriptions and the looming shortage of\ntraining data. These obstacles prevent encoder-decoder models from being used\ndirectly, but optimization-based methods such as determinantal point processes\n(DPPs) are known to handle them well. In this paper we seek to strengthen a\nDPP-based method for extractive multi-document summarization by presenting a\nnovel similarity measure inspired by capsule networks. The approach measures\nredundancy between a pair of sentences based on surface form and semantic\ninformation. We show that our DPP system with improved similarity measure\nperforms competitively, outperforming strong summarization baselines on\nbenchmark datasets. Our findings are particularly meaningful for summarizing\ndocuments created by multiple authors containing redundant yet lexically\ndiverse expressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00077v1",
            "date": "2019-05-31 20:58:08+00:00",
            "title": "Scoring Sentence Singletons and Pairs for Abstractive Summarization",
            "authors": [
                "Logan Lebanoff",
                "Kaiqiang Song",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Seokhwan Kim",
                "Walter Chang",
                "Fei Liu"
            ],
            "abstract": "When writing a summary, humans tend to choose content from one or two\nsentences and merge them into a single summary sentence. However, the\nmechanisms behind the selection of one or multiple source sentences remain\npoorly understood. Sentence fusion assumes multi-sentence input; yet sentence\nselection methods only work with single sentences and not combinations of them.\nThere is thus a crucial gap between sentence selection and fusion to support\nsummarizing by both compressing single sentences and fusing pairs. This paper\nattempts to bridge the gap by ranking sentence singletons and pairs together in\na unified space. Our proposed framework attempts to model human methodology by\nselecting either a single sentence or a pair of sentences, then compressing or\nfusing the sentence(s) to produce a summary sentence. We conduct extensive\nexperiments on both single- and multi-document summarization datasets and\nreport findings on sentence selection and abstraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00114v1",
            "date": "2019-05-31 22:47:56+00:00",
            "title": "Examining Structure of Word Embeddings with PCA",
            "authors": [
                "Tom\u00e1\u0161 Musil"
            ],
            "abstract": "In this paper we compare structure of Czech word embeddings for English-Czech\nneural machine translation (NMT), word2vec and sentiment analysis. We show that\nalthough it is possible to successfully predict part of speech (POS) tags from\nword embeddings of word2vec and various translation models, not all of the\nembedding spaces show the same structure. The information about POS is present\nin word2vec embeddings, but the high degree of organization by POS in the NMT\ndecoder suggests that this information is more important for machine\ntranslation and therefore the NMT model represents it in more direct way. Our\nmethod is based on correlation of principal component analysis (PCA) dimensions\nwith categorical linguistic data. We also show that further examining\nhistograms of classes along the principal component is important to understand\nthe structure of representation of information in embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00138v1",
            "date": "2019-06-01 03:05:31+00:00",
            "title": "Efficient Adaptation of Pretrained Transformers for Abstractive Summarization",
            "authors": [
                "Andrew Hoang",
                "Antoine Bosselut",
                "Asli Celikyilmaz",
                "Yejin Choi"
            ],
            "abstract": "Large-scale learning of transformer language models has yielded improvements\non a variety of natural language understanding tasks. Whether they can be\neffectively adapted for summarization, however, has been less explored, as the\nlearned representations are less seamlessly integrated into existing neural\ntext production architectures. In this work, we propose two solutions for\nefficiently adapting pretrained transformer language models as text\nsummarizers: source embeddings and domain-adaptive training. We test these\nsolutions on three abstractive summarization datasets, achieving new state of\nthe art performance on two of them. Finally, we show that these improvements\nare achieved by producing more focused summaries with fewer superfluous and\nthat performance improvements are more pronounced on more abstractive datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00238v1",
            "date": "2019-06-01 15:01:16+00:00",
            "title": "Adversarial Generation and Encoding of Nested Texts",
            "authors": [
                "Alon Rozental"
            ],
            "abstract": "In this paper we propose a new language model called AGENT, which stands for\nAdversarial Generation and Encoding of Nested Texts. AGENT is designed for\nencoding, generating and refining documents that consist of a long and coherent\ntext, such as an entire book, provided they are hierarchically annotated\n(nested). i.e. divided into sentences, paragraphs and chapters. The core idea\nof our system is learning vector representations for each level of the text\nhierarchy (sentences, paragraphs, etc...), and train each such representation\nto perform 3 tasks: The task of reconstructing the sequence of vectors from a\nlower level that was used to create the representation, and generalized\nversions of the Masked Language Modeling (MLM) and \"Next Sentence Prediction\"\ntasks from BERT Devlin et al. [2018]. Additionally we present a new adversarial\nmodel for long text generation and suggest a way to improve the coherence of\nthe generated text by traversing its vector representation tree.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00247v2",
            "date": "2019-06-01 15:37:19+00:00",
            "title": "COS960: A Chinese Word Similarity Dataset of 960 Word Pairs",
            "authors": [
                "Junjie Huang",
                "Fanchao Qi",
                "Chenghao Yang",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Word similarity computation is a widely recognized task in the field of\nlexical semantics. Most proposed tasks test on similarity of word pairs of\nsingle morpheme, while few works focus on words of two morphemes or more\nmorphemes. In this work, we propose COS960, a benchmark dataset with 960 pairs\nof Chinese wOrd Similarity, where all the words have two morphemes in three\nPart of Speech (POS) tags with their human annotated similarity rather than\nrelatedness. We give a detailed description of dataset construction and\nannotation process, and test on a range of word embedding models. The dataset\nof this paper can be obtained from https://github.com/thunlp/COS960.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00266v1",
            "date": "2019-06-01 18:35:12+00:00",
            "title": "How to best use Syntax in Semantic Role Labelling",
            "authors": [
                "Yufei Wang",
                "Mark Johnson",
                "Stephen Wan",
                "Yifang Sun",
                "Wei Wang"
            ],
            "abstract": "There are many different ways in which external information might be used in\nan NLP task. This paper investigates how external syntactic information can be\nused most effectively in the Semantic Role Labeling (SRL) task. We evaluate\nthree different ways of encoding syntactic parses and three different ways of\ninjecting them into a state-of-the-art neural ELMo-based SRL sequence labelling\nmodel. We show that using a constituency representation as input features\nimproves performance the most, achieving a new state-of-the-art for\nnon-ensemble SRL models on the in-domain CoNLL'05 and CoNLL'12 benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00274v1",
            "date": "2019-06-01 19:17:03+00:00",
            "title": "\"President Vows to Cut <Taxes> Hair\": Dataset and Analysis of Creative Text Editing for Humorous Headlines",
            "authors": [
                "Nabil Hossain",
                "John Krumm",
                "Michael Gamon"
            ],
            "abstract": "We introduce, release, and analyze a new dataset, called Humicroedit, for\nresearch in computational humor. Our publicly available data consists of\nregular English news headlines paired with versions of the same headlines that\ncontain simple replacement edits designed to make them funny. We carefully\ncurated crowdsourced editors to create funny headlines and judges to score a to\na total of 15,095 edited headlines, with five judges per headline. The simple\nedits, usually just a single word replacement, mean we can apply\nstraightforward analysis techniques to determine what makes our edited\nheadlines humorous. We show how the data support classic theories of humor,\nsuch as incongruity, superiority, and setup/punchline. Finally, we develop\nbaseline classifiers that can predict whether or not an edited headline is\nfunny, which is a first step toward automatically generating humorous headlines\nas an approach to creating topical humor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00295v1",
            "date": "2019-06-01 21:29:20+00:00",
            "title": "Multimodal Transformer for Unaligned Multimodal Language Sequences",
            "authors": [
                "Yao-Hung Hubert Tsai",
                "Shaojie Bai",
                "Paul Pu Liang",
                "J. Zico Kolter",
                "Louis-Philippe Morency",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Human language is often multimodal, which comprehends a mixture of natural\nlanguage, facial gestures, and acoustic behaviors. However, two major\nchallenges in modeling such multimodal human language time-series data exist:\n1) inherent data non-alignment due to variable sampling rates for the sequences\nfrom each modality; and 2) long-range dependencies between elements across\nmodalities. In this paper, we introduce the Multimodal Transformer (MulT) to\ngenerically address the above issues in an end-to-end manner without explicitly\naligning the data. At the heart of our model is the directional pairwise\ncrossmodal attention, which attends to interactions between multimodal\nsequences across distinct time steps and latently adapt streams from one\nmodality to another. Comprehensive experiments on both aligned and non-aligned\nmultimodal time-series show that our model outperforms state-of-the-art methods\nby a large margin. In addition, empirical analysis suggests that correlated\ncrossmodal signals are able to be captured by the proposed crossmodal attention\nmechanism in MulT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00300v3",
            "date": "2019-06-01 22:02:39+00:00",
            "title": "Latent Retrieval for Weakly Supervised Open Domain Question Answering",
            "authors": [
                "Kenton Lee",
                "Ming-Wei Chang",
                "Kristina Toutanova"
            ],
            "abstract": "Recent work on open domain question answering (QA) assumes strong supervision\nof the supporting evidence and/or assumes a blackbox information retrieval (IR)\nsystem to retrieve evidence candidates. We argue that both are suboptimal,\nsince gold evidence is not always available, and QA is fundamentally different\nfrom IR. We show for the first time that it is possible to jointly learn the\nretriever and reader from question-answer string pairs and without any IR\nsystem. In this setting, evidence retrieval from all of Wikipedia is treated as\na latent variable. Since this is impractical to learn from scratch, we\npre-train the retriever with an Inverse Cloze Task. We evaluate on open\nversions of five QA datasets. On datasets where the questioner already knows\nthe answer, a traditional IR system such as BM25 is sufficient. On datasets\nwhere a user is genuinely seeking an answer, we show that learned retrieval is\ncrucial, outperforming BM25 by up to 19 points in exact match.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00347v3",
            "date": "2019-06-02 05:16:06+00:00",
            "title": "Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation",
            "authors": [
                "Ronghang Hu",
                "Daniel Fried",
                "Anna Rohrbach",
                "Dan Klein",
                "Trevor Darrell",
                "Kate Saenko"
            ],
            "abstract": "Vision-and-Language Navigation (VLN) requires grounding instructions, such as\n\"turn right and stop at the door\", to routes in a visual environment. The\nactual grounding can connect language to the environment through multiple\nmodalities, e.g. \"stop at the door\" might ground into visual objects, while\n\"turn right\" might rely only on the geometric structure of a route. We\ninvestigate where the natural language empirically grounds under two recent\nstate-of-the-art VLN models. Surprisingly, we discover that visual features may\nactually hurt these models: models which only use route structure, ablating\nvisual features, outperform their visual counterparts in unseen new\nenvironments on the benchmark Room-to-Room dataset. To better use all the\navailable modalities, we propose to decompose the grounding procedure into a\nset of expert models with access to different modalities (including object\ndetections) and ensemble them at prediction time, improving the performance of\nstate-of-the-art models on the VLN task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00376v1",
            "date": "2019-06-02 09:50:12+00:00",
            "title": "Domain Adaptation of Neural Machine Translation by Lexicon Induction",
            "authors": [
                "Junjie Hu",
                "Mengzhou Xia",
                "Graham Neubig",
                "Jaime Carbonell"
            ],
            "abstract": "It has been previously noted that neural machine translation (NMT) is very\nsensitive to domain shift. In this paper, we argue that this is a dual effect\nof the highly lexicalized nature of NMT, resulting in failure for sentences\nwith large numbers of unknown words, and lack of supervision for\ndomain-specific words. To remedy this problem, we propose an unsupervised\nadaptation method which fine-tunes a pre-trained out-of-domain NMT model using\na pseudo-in-domain corpus. Specifically, we perform lexicon induction to\nextract an in-domain lexicon, and construct a pseudo-parallel in-domain corpus\nby performing word-for-word back-translation of monolingual in-domain target\nsentences. In five domains over twenty pairwise adaptation settings and two\nmodel architectures, our method achieves consistent improvements without using\nany in-domain parallel sentences, improving up to 14 BLEU over unadapted\nmodels, and up to 2 BLEU over strong back-translation baselines.",
            "journal ref": "published at the 57th Annual Meeting of the Association for\n  Computational Linguistics (ACL). July 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00408v1",
            "date": "2019-06-02 14:00:05+00:00",
            "title": "Domain Adaptive Inference for Neural Machine Translation",
            "authors": [
                "Danielle Saunders",
                "Felix Stahlberg",
                "Adria de Gispert",
                "Bill Byrne"
            ],
            "abstract": "We investigate adaptive ensemble weighting for Neural Machine Translation,\naddressing the case of improving performance on a new and potentially unknown\ndomain without sacrificing performance on the original domain. We adapt\nsequentially across two Spanish-English and three English-German tasks,\ncomparing unregularized fine-tuning, L2 and Elastic Weight Consolidation. We\nthen report a novel scheme for adaptive NMT ensemble decoding by extending\nBayesian Interpolation with source information, and show strong improvements\nacross test domains without access to the domain label.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00424v1",
            "date": "2019-06-02 15:27:51+00:00",
            "title": "Plain English Summarization of Contracts",
            "authors": [
                "Laura Manor",
                "Junyi Jessy Li"
            ],
            "abstract": "Unilateral contracts, such as terms of service, play a substantial role in\nmodern digital life. However, few users read these documents before accepting\nthe terms within, as they are too long and the language too complicated. We\npropose the task of summarizing such legal documents in plain English, which\nwould enable users to have a better understanding of the terms they are\naccepting.\n  We propose an initial dataset of legal text snippets paired with summaries\nwritten in plain English. We verify the quality of these summaries manually and\nshow that they involve heavy abstraction, compression, and simplification.\nInitial experiments show that unsupervised extractive summarization methods do\nnot perform well on this task due to the level of abstraction and style\ndifferences. We conclude with a call for resource and technique development for\nsimplification and style transfer for legal language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00500v1",
            "date": "2019-06-02 22:55:14+00:00",
            "title": "A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions",
            "authors": [
                "Sashank Santhanam",
                "Samira Shaikh"
            ],
            "abstract": "One of the hardest problems in the area of Natural Language Processing and\nArtificial Intelligence is automatically generating language that is coherent\nand understandable to humans. Teaching machines how to converse as humans do\nfalls under the broad umbrella of Natural Language Generation. Recent years\nhave seen unprecedented growth in the number of research articles published on\nthis subject in conferences and journals both by academic and industry\nresearchers. There have also been several workshops organized alongside\ntop-tier NLP conferences dedicated specifically to this problem. All this\nactivity makes it hard to clearly define the state of the field and reason\nabout its future directions. In this work, we provide an overview of this\nimportant and thriving area, covering traditional approaches, statistical\napproaches and also approaches that use deep neural networks. We provide a\ncomprehensive review towards building open domain dialogue systems, an\nimportant application of natural language generation. We find that,\npredominantly, the approaches for building dialogue systems use seq2seq or\nlanguage models architecture. Notably, we identify three important areas of\nfurther research towards building more effective dialogue systems: 1)\nincorporating larger context, including conversation context and world\nknowledge; 2) adding personae or personality in the NLG system; and 3)\novercoming dull and generic responses that affect the quality of\nsystem-produced responses. We provide pointers on how to tackle these open\nproblems through the use of cognitive architectures that mimic human language\nunderstanding and generation capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00549v1",
            "date": "2019-06-03 03:45:15+00:00",
            "title": "Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment",
            "authors": [
                "Siqi Bao",
                "Huang He",
                "Fan Wang",
                "Rongzhong Lian",
                "Hua Wu"
            ],
            "abstract": "In this paper, a novel Generation-Evaluation framework is developed for\nmulti-turn conversations with the objective of letting both participants know\nmore about each other. For the sake of rational knowledge utilization and\ncoherent conversation flow, a dialogue strategy which controls knowledge\nselection is instantiated and continuously adapted via reinforcement learning.\nUnder the deployed strategy, knowledge grounded conversations are conducted\nwith two dialogue agents. The generated dialogues are comprehensively evaluated\non aspects like informativeness and coherence, which are aligned with our\nobjective and human instinct. These assessments are integrated as a compound\nreward to guide the evolution of dialogue strategy via policy gradient.\nComprehensive experiments have been carried out on the publicly available\ndataset, demonstrating that the proposed method outperforms the other\nstate-of-the-art approaches significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00550v1",
            "date": "2019-06-03 03:47:37+00:00",
            "title": "Global Textual Relation Embedding for Relational Understanding",
            "authors": [
                "Zhiyu Chen",
                "Hanwen Zha",
                "Honglei Liu",
                "Wenhu Chen",
                "Xifeng Yan",
                "Yu Su"
            ],
            "abstract": "Pre-trained embeddings such as word embeddings and sentence embeddings are\nfundamental tools facilitating a wide range of downstream NLP tasks. In this\nwork, we investigate how to learn a general-purpose embedding of textual\nrelations, defined as the shortest dependency path between entities. Textual\nrelation embedding provides a level of knowledge between word/phrase level and\nsentence level, and we show that it can facilitate downstream tasks requiring\nrelational understanding of the text. To learn such an embedding, we create the\nlargest distant supervision dataset by linking the entire English ClueWeb09\ncorpus to Freebase. We use global co-occurrence statistics between textual and\nknowledge base relations as the supervision signal to train the embedding.\nEvaluation on two relational understanding tasks demonstrates the usefulness of\nthe learned textual relation embedding. The data and code can be found at\nhttps://github.com/czyssrs/GloREPlus",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00556v1",
            "date": "2019-06-03 03:57:11+00:00",
            "title": "Fluent Translations from Disfluent Speech in End-to-End Speech Translation",
            "authors": [
                "Elizabeth Salesky",
                "Matthias Sperber",
                "Alex Waibel"
            ],
            "abstract": "Spoken language translation applications for speech suffer due to\nconversational speech phenomena, particularly the presence of disfluencies.\nWith the rise of end-to-end speech translation models, processing steps such as\ndisfluency removal that were previously an intermediate step between speech\nrecognition and machine translation need to be incorporated into model\narchitectures. We use a sequence-to-sequence model to translate from noisy,\ndisfluent speech to fluent text with disfluencies removed using the recently\ncollected `copy-edited' references for the Fisher Spanish-English dataset. We\nare able to directly generate fluent translations and introduce considerations\nabout how to evaluate success on this task. This work provides a baseline for a\nnew task, the translation of conversational speech with joint removal of\ndisfluencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00565v1",
            "date": "2019-06-03 04:29:22+00:00",
            "title": "Controllable Paraphrase Generation with a Syntactic Exemplar",
            "authors": [
                "Mingda Chen",
                "Qingming Tang",
                "Sam Wiseman",
                "Kevin Gimpel"
            ],
            "abstract": "Prior work on controllable text generation usually assumes that the\ncontrolled attribute can take on one of a small set of values known a priori.\nIn this work, we propose a novel task, where the syntax of a generated sentence\nis controlled rather by a sentential exemplar. To evaluate quantitatively with\nstandard metrics, we create a novel dataset with human annotations. We also\ndevelop a variational model with a neural module specifically designed for\ncapturing syntactic knowledge and several multitask training objectives to\npromote disentangled representation learning. Empirically, the proposed model\nis observed to achieve improvements over baselines and learn to capture\ndesirable characteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00575v3",
            "date": "2019-06-03 05:00:09+00:00",
            "title": "Jointly Learning Semantic Parser and Natural Language Generator via Dual Information Maximization",
            "authors": [
                "Hai Ye",
                "Wenjie Li",
                "Lu Wang"
            ],
            "abstract": "Semantic parsing aims to transform natural language (NL) utterances into\nformal meaning representations (MRs), whereas an NL generator achieves the\nreverse: producing a NL description for some given MRs. Despite this intrinsic\nconnection, the two tasks are often studied separately in prior work. In this\npaper, we model the duality of these two tasks via a joint learning framework,\nand demonstrate its effectiveness of boosting the performance on both tasks.\nConcretely, we propose a novel method of dual information maximization (DIM) to\nregularize the learning process, where DIM empirically maximizes the\nvariational lower bounds of expected joint distributions of NL and MRs. We\nfurther extend DIM to a semi-supervision setup (SemiDIM), which leverages\nunlabeled data of both tasks. Experiments on three datasets of dialogue\nmanagement and code generation (and summarization) show that performance on\nboth semantic parsing and NL generation can be consistently improved by DIM, in\nboth supervised and semi-supervised setups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00584v1",
            "date": "2019-06-03 05:42:33+00:00",
            "title": "A Semi-Supervised Approach for Low-Resourced Text Generation",
            "authors": [
                "Hongyu Zang",
                "Xiaojun Wan"
            ],
            "abstract": "Recently, encoder-decoder neural models have achieved great success on text\ngeneration tasks. However, one problem of this kind of models is that their\nperformances are usually limited by the scale of well-labeled data, which are\nvery expensive to get. The low-resource (of labeled data) problem is quite\ncommon in different task generation tasks, but unlabeled data are usually\nabundant. In this paper, we propose a method to make use of the unlabeled data\nto improve the performance of such models in the low-resourced circumstances.\nWe use denoising auto-encoder (DAE) and language model (LM) based reinforcement\nlearning (RL) to enhance the training of encoder and decoder with unlabeled\ndata. Our method shows adaptability for different text generation tasks, and\nmakes significant improvements over basic text generation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00591v1",
            "date": "2019-06-03 06:21:38+00:00",
            "title": "Evaluating Gender Bias in Machine Translation",
            "authors": [
                "Gabriel Stanovsky",
                "Noah A. Smith",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present the first challenge set and evaluation protocol for the analysis\nof gender bias in machine translation (MT). Our approach uses two recent\ncoreference resolution datasets composed of English sentences which cast\nparticipants into non-stereotypical gender roles (e.g., \"The doctor asked the\nnurse to help her in the operation\"). We devise an automatic gender bias\nevaluation method for eight target languages with grammatical gender, based on\nmorphological analysis (e.g., the use of female inflection for the word\n\"doctor\"). Our analyses show that four popular industrial MT systems and two\nrecent state-of-the-art academic MT models are significantly prone to\ngender-biased translation errors for all tested target languages. Our data and\ncode are made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00663v3",
            "date": "2019-06-03 09:39:33+00:00",
            "title": "Semantically Constrained Multilayer Annotation: The Case of Coreference",
            "authors": [
                "Jakob Prange",
                "Nathan Schneider",
                "Omri Abend"
            ],
            "abstract": "We propose a coreference annotation scheme as a layer on top of the Universal\nConceptual Cognitive Annotation foundational layer, treating units in\npredicate-argument structure as a basis for entity and event mentions. We argue\nthat this allows coreference annotators to sidestep some of the challenges\nfaced in other schemes, which do not enforce consistency with\npredicate-argument structure and vary widely in what kinds of mentions they\nannotate and how. The proposed approach is examined with a pilot annotation\nstudy and compared with annotations from other schemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00790v2",
            "date": "2019-06-03 13:28:33+00:00",
            "title": "Multi-task Pairwise Neural Ranking for Hashtag Segmentation",
            "authors": [
                "Mounica Maddela",
                "Wei Xu",
                "Daniel Preo\u0163iuc-Pietro"
            ],
            "abstract": "Hashtags are often employed on social media and beyond to add metadata to a\ntextual utterance with the goal of increasing discoverability, aiding search,\nor providing additional semantics. However, the semantic content of hashtags is\nnot straightforward to infer as these represent ad-hoc conventions which\nfrequently include multiple words joined together and can include abbreviations\nand unorthodox spellings. We build a dataset of 12,594 hashtags split into\nindividual segments and propose a set of approaches for hashtag segmentation by\nframing it as a pairwise ranking problem between candidate segmentations. Our\nnovel neural approaches demonstrate 24.6% error reduction in hashtag\nsegmentation accuracy compared to the current state-of-the-art method. Finally,\nwe demonstrate that a deeper understanding of hashtag semantics obtained\nthrough segmentation is useful for downstream applications such as sentiment\nanalysis, for which we achieved a 2.6% increase in average recall on the\nSemEval 2017 sentiment analysis dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00839v1",
            "date": "2019-06-03 14:37:57+00:00",
            "title": "Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling",
            "authors": [
                "Sandeep Attree"
            ],
            "abstract": "This paper presents a strong set of results for resolving gendered ambiguous\npronouns on the Gendered Ambiguous Pronouns shared task. The model presented\nhere draws upon the strengths of state-of-the-art language and coreference\nresolution models, and introduces a novel evidence-based deep learning\narchitecture. Injecting evidence from the coreference models compliments the\nbase architecture, and analysis shows that the model is not hindered by their\nweaknesses, specifically gender bias. The modularity and simplicity of the\narchitecture make it very easy to extend for further improvement and applicable\nto other NLP problems. Evaluation on GAP test data results in a\nstate-of-the-art performance at 92.5% F1 (gender bias of 0.97), edging closer\nto the human performance of 96.6%. The end-to-end solution presented here\nplaced 1st in the Kaggle competition, winning by a significant lead. The code\nis available at https://github.com/sattree/gap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01010v1",
            "date": "2019-06-03 18:17:09+00:00",
            "title": "A computational linguistic study of personal recovery in bipolar disorder",
            "authors": [
                "Glorianna Jagfeld"
            ],
            "abstract": "Mental health research can benefit increasingly fruitfully from computational\nlinguistics methods, given the abundant availability of language data in the\ninternet and advances of computational tools. This interdisciplinary project\nwill collect and analyse social media data of individuals diagnosed with\nbipolar disorder with regard to their recovery experiences. Personal recovery -\nliving a satisfying and contributing life along symptoms of severe mental\nhealth issues - so far has only been investigated qualitatively with structured\ninterviews and quantitatively with standardised questionnaires with mainly\nEnglish-speaking participants in Western countries. Complementary to this\nevidence, computational linguistic methods allow us to analyse first-person\naccounts shared online in large quantities, representing unstructured settings\nand a more heterogeneous, multilingual population, to draw a more complete\npicture of the aspects and mechanisms of personal recovery in bipolar disorder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01037v2",
            "date": "2019-06-03 19:30:51+00:00",
            "title": "Better Character Language Modeling Through Morphology",
            "authors": [
                "Terra Blevins",
                "Luke Zettlemoyer"
            ],
            "abstract": "We incorporate morphological supervision into character language models\n(CLMs) via multitasking and show that this addition improves bits-per-character\n(BPC) performance across 24 languages, even when the morphology data and\nlanguage modeling data are disjoint. Analyzing the CLMs shows that inflected\nwords benefit more from explicitly modeling morphology than uninflected words,\nand that morphological supervision improves performance even as the amount of\nlanguage modeling data grows. We then transfer morphological supervision across\nlanguages to improve language modeling performance in the low-resource setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01038v1",
            "date": "2019-06-03 19:33:13+00:00",
            "title": "Transforming Complex Sentences into a Semantic Hierarchy",
            "authors": [
                "Christina Niklaus",
                "Matthias Cetto",
                "Andre Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We present an approach for recursively splitting and rephrasing complex\nEnglish sentences into a novel semantic hierarchy of simplified sentences, with\neach of them presenting a more regular structure that may facilitate a wide\nvariety of artificial intelligence tasks, such as machine translation (MT) or\ninformation extraction (IE). Using a set of hand-crafted transformation rules,\ninput sentences are recursively transformed into a two-layered hierarchical\nrepresentation in the form of core sentences and accompanying contexts that are\nlinked via rhetorical relations. In this way, the semantic relationship of the\ndecomposed constituents is preserved in the output, maintaining its\ninterpretability for downstream applications. Both a thorough manual analysis\nand automatic evaluation across three datasets from two different domains\ndemonstrate that the proposed syntactic simplification approach outperforms the\nstate of the art in structural text simplification. Moreover, an extrinsic\nevaluation shows that when applying our framework as a preprocessing step the\nperformance of state-of-the-art Open IE systems can be improved by up to 346%\nin precision and 52% in recall. To enable reproducible research, all code is\nprovided online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01081v1",
            "date": "2019-06-03 21:13:07+00:00",
            "title": "Handling Divergent Reference Texts when Evaluating Table-to-Text Generation",
            "authors": [
                "Bhuwan Dhingra",
                "Manaal Faruqui",
                "Ankur Parikh",
                "Ming-Wei Chang",
                "Dipanjan Das",
                "William W. Cohen"
            ],
            "abstract": "Automatically constructed datasets for generating text from semi-structured\ndata (tables), such as WikiBio, often contain reference texts that diverge from\nthe information in the corresponding semi-structured data. We show that metrics\nwhich rely solely on the reference texts, such as BLEU and ROUGE, show poor\ncorrelation with human judgments when those references diverge. We propose a\nnew metric, PARENT, which aligns n-grams from the reference and generated texts\nto the semi-structured data before computing their precision and recall.\nThrough a large scale human evaluation study of table-to-text models for\nWikiBio, we show that PARENT correlates with human judgments better than\nexisting text generation metrics. We also adapt and evaluate the information\nextraction based evaluation proposed by Wiseman et al (2017), and show that\nPARENT has comparable correlation to it, while being easier to use. We show\nthat PARENT is also applicable when the reference texts are elicited from\nhumans using the data from the WebNLG challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01135v2",
            "date": "2019-06-04 00:11:52+00:00",
            "title": "Simultaneous Translation with Flexible Policy via Restricted Imitation Learning",
            "authors": [
                "Baigong Zheng",
                "Renjie Zheng",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "Simultaneous translation is widely useful but remains one of the most\ndifficult tasks in NLP. Previous work either uses fixed-latency policies, or\ntrain a complicated two-staged model using reinforcement learning. We propose a\nmuch simpler single model that adds a `delay' token to the target vocabulary,\nand design a restricted dynamic oracle to greatly simplify training.\nExperiments on Chinese<->English simultaneous translation show that our work\nleads to flexible policies that achieve better BLEU scores and lower latencies\ncompared to both fixed and RL-learned policies.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01145v1",
            "date": "2019-06-04 00:51:23+00:00",
            "title": "System Demo for Transfer Learning across Vision and Text using Domain Specific CNN Accelerator for On-Device NLP Applications",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Michael Lin",
                "Wenhan Zhang",
                "Patrick Dong",
                "Charles Young",
                "Jason Dong"
            ],
            "abstract": "Power-efficient CNN Domain Specific Accelerator (CNN-DSA) chips are currently\navailable for wide use in mobile devices. These chips are mainly used in\ncomputer vision applications. However, the recent work of Super Characters\nmethod for text classification and sentiment analysis tasks using\ntwo-dimensional CNN models has also achieved state-of-the-art results through\nthe method of transfer learning from vision to text. In this paper, we\nimplemented the text classification and sentiment analysis applications on\nmobile devices using CNN-DSA chips. Compact network representations using\none-bit and three-bits precision for coefficients and five-bits for activations\nare used in the CNN-DSA chip with power consumption less than 300mW. For edge\ndevices under memory and compute constraints, the network is further compressed\nby approximating the external Fully Connected (FC) layers within the CNN-DSA\nchip. At the workshop, we have two system demonstrations for NLP tasks. The\nfirst demo classifies the input English Wikipedia sentence into one of the 14\nontologies. The second demo classifies the Chinese online-shopping review into\npositive or negative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01149v1",
            "date": "2019-06-04 01:13:20+00:00",
            "title": "Improving Long Distance Slot Carryover in Spoken Dialogue Systems",
            "authors": [
                "Tongfei Chen",
                "Chetan Naik",
                "Hua He",
                "Pushpendre Rastogi",
                "Lambert Mathias"
            ],
            "abstract": "Tracking the state of the conversation is a central component in\ntask-oriented spoken dialogue systems. One such approach for tracking the\ndialogue state is slot carryover, where a model makes a binary decision if a\nslot from the context is relevant to the current turn. Previous work on the\nslot carryover task used models that made independent decisions for each slot.\nA close analysis of the results show that this approach results in poor\nperformance over longer context dialogues. In this paper, we propose to jointly\nmodel the slots. We propose two neural network architectures, one based on\npointer networks that incorporate slot ordering information, and the other\nbased on transformer networks that uses self attention mechanism to model the\nslot interdependencies. Our experiments on an internal dialogue benchmark\ndataset and on the public DSTC2 dataset demonstrate that our proposed models\nare able to resolve longer distance slot references and are able to achieve\ncompetitive performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01154v5",
            "date": "2019-06-04 01:54:42+00:00",
            "title": "Detecting Local Insights from Global Labels: Supervised & Zero-Shot Sequence Labeling via a Convolutional Decomposition",
            "authors": [
                "Allen Schmaltz"
            ],
            "abstract": "Detecting, constraining, manipulating, and reasoning over features in the\nhigh-dimensional space that is language is the primary concern of modern\ncomputational linguistics, and a key focus of AI, more generally. We propose a\ngeneral framework for addressing the first two of these four facets, coupling\nsharp feature detection with a matching method for introspecting inference.\nSpecifically, we present and analyze binary labeling via a convolutional\ndecomposition (BLADE), a sequence labeling approach based on a decomposition of\nthe filter-ngram interactions of a convolutional neural network and a linear\nlayer. BLADE can be viewed as a maxpool attention-style mechanism on the final\nlayer of a network, and it enables flexibility in producing predictions at --\nand defining loss functions for -- varying label granularities, from the\nfully-supervised sequence labeling setting to the challenging zero-shot\nsequence labeling setting, in which we seek token-level predictions but only\nhave access to labels at the document- or sentence-level for training.\nImportantly, BLADE enables a matching method, exemplar auditing, useful for\nanalyzing the model and data, and empirically, as part of an inference-time\ndecision rule. This introspection method provides a means, in some settings, of\nupdating the model (via a database) without explicit re-training, opening the\npossibility for end-users to make local updates, or for annotators to\nprogressively add fine-grained labels (or other meta-data). We assess this\nframework -- and suitability limitations -- on a series of binary\nclassification tasks and at varying label resolutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01181v1",
            "date": "2019-06-04 03:30:22+00:00",
            "title": "Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations",
            "authors": [
                "Jiatao Gu",
                "Yong Wang",
                "Kyunghyun Cho",
                "Victor O. K. Li"
            ],
            "abstract": "Zero-shot translation, translating between language pairs on which a Neural\nMachine Translation (NMT) system has never been trained, is an emergent\nproperty when training the system in multilingual settings. However, naive\ntraining for zero-shot NMT easily fails, and is sensitive to hyper-parameter\nsetting. The performance typically lags far behind the more conventional\npivot-based approach which translates twice using a third language as a pivot.\nIn this work, we address the degeneracy problem due to capturing spurious\ncorrelations by quantitatively analyzing the mutual information between\nlanguage IDs of the source and decoded sentences. Inspired by this analysis, we\npropose to use two simple but effective approaches: (1) decoder pre-training;\n(2) back-translation. These methods show significant improvement (4~22 BLEU\npoints) over the vanilla zero-shot translation on three challenging\nmultilingual datasets, and achieve similar or better results than the\npivot-based approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01183v2",
            "date": "2019-06-04 03:33:51+00:00",
            "title": "Back Attention Knowledge Transfer for Low-resource Named Entity Recognition",
            "authors": [
                "Linghao Sun",
                "Huixiong Yi",
                "Yong Liu",
                "Huanhuan Chen",
                "Chunyan Miao"
            ],
            "abstract": "In recent years, great success has been achieved in the field of natural\nlanguage processing (NLP), thanks in part to the considerable amount of\nannotated resources. For named entity recognition (NER), most languages do not\nhave such an abundance of labeled data as English, so the performances of those\nlanguages are relatively lower. To improve the performance, we propose a\ngeneral approach called Back Attention Network (BAN). BAN uses a translation\nsystem to translate other language sentences into English and then applies a\nnew mechanism named back attention knowledge transfer to obtain task-specific\ninformation from pre-trained high-resource languages NER model. This strategy\ncan transfer high-layer features of well-trained model and enrich the semantic\nrepresentations of the original language. Experiments on three different\nlanguage datasets indicate that the proposed approach outperforms other\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01185v1",
            "date": "2019-06-04 03:43:37+00:00",
            "title": "Joint Effects of Context and User History for Predicting Online Conversation Re-entries",
            "authors": [
                "Xingshan Zeng",
                "Jing Li",
                "Lu Wang",
                "Kam-Fai Wong"
            ],
            "abstract": "As the online world continues its exponential growth, interpersonal\ncommunication has come to play an increasingly central role in opinion\nformation and change. In order to help users better engage with each other\nonline, we study a challenging problem of re-entry prediction foreseeing\nwhether a user will come back to a conversation they once participated in. We\nhypothesize that both the context of the ongoing conversations and the users'\nprevious chatting history will affect their continued interests in future\nengagement. Specifically, we propose a neural framework with three main layers,\neach modeling context, user history, and interactions between them, to explore\nhow the conversation context and user chatting history jointly result in their\nre-entry behavior. We experiment with two large-scale datasets collected from\nTwitter and Reddit. Results show that our proposed framework with bi-attention\nachieves an F1 score of 61.1 on Twitter conversations, outperforming the\nstate-of-the-art methods from previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01213v3",
            "date": "2019-06-04 06:07:56+00:00",
            "title": "Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis",
            "authors": [
                "Jialong Tang",
                "Ziyao Lu",
                "Jinsong Su",
                "Yubin Ge",
                "Linfeng Song",
                "Le Sun",
                "Jiebo Luo"
            ],
            "abstract": "In aspect-level sentiment classification (ASC), it is prevalent to equip\ndominant neural models with attention mechanisms, for the sake of acquiring the\nimportance of each context word on the given aspect. However, such a mechanism\ntends to excessively focus on a few frequent words with sentiment polarities,\nwhile ignoring infrequent ones. In this paper, we propose a progressive\nself-supervised attention learning approach for neural ASC models, which\nautomatically mines useful attention supervision information from a training\ncorpus to refine attention mechanisms. Specifically, we iteratively conduct\nsentiment predictions on all training instances. Particularly, at each\niteration, the context word with the maximum attention weight is extracted as\nthe one with active/misleading influence on the correct/incorrect prediction of\nevery instance, and then the word itself is masked for subsequent iterations.\nFinally, we augment the conventional training objective with a regularization\nterm, which enables ASC models to continue equally focusing on the extracted\nactive context words while decreasing weights of those misleading ones.\nExperimental results on multiple datasets show that our proposed approach\nyields better attention mechanisms, leading to substantial improvements over\nthe two state-of-the-art neural ASC models. Source code and trained models are\navailable at https://github.com/DeepLearnXMU/PSSAttention.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01230v1",
            "date": "2019-06-04 07:02:22+00:00",
            "title": "From Independent Prediction to Re-ordered Prediction: Integrating Relative Position and Global Label Information to Emotion Cause Identification",
            "authors": [
                "Zixiang Ding",
                "Huihui He",
                "Mengran Zhang",
                "Rui Xia"
            ],
            "abstract": "Emotion cause identification aims at identifying the potential causes that\nlead to a certain emotion expression in text. Several techniques including rule\nbased methods and traditional machine learning methods have been proposed to\naddress this problem based on manually designed rules and features. More\nrecently, some deep learning methods have also been applied to this task, with\nthe attempt to automatically capture the causal relationship of emotion and its\ncauses embodied in the text. In this work, we find that in addition to the\ncontent of the text, there are another two kinds of information, namely\nrelative position and global labels, that are also very important for emotion\ncause identification. To integrate such information, we propose a model based\non the neural network architecture to encode the three elements ($i.e.$, text\ncontent, relative position and global label), in an unified and end-to-end\nfashion. We introduce a relative position augmented embedding learning\nalgorithm, and transform the task from an independent prediction problem to a\nreordered prediction problem, where the dynamic global label information is\nincorporated. Experimental results on a benchmark emotion cause dataset show\nthat our model achieves new state-of-the-art performance and performs\nsignificantly better than a number of competitive baselines. Further analysis\nshows the effectiveness of the relative position augmented embedding learning\nalgorithm and the reordered prediction mechanism with dynamic global labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01236v1",
            "date": "2019-06-04 07:10:16+00:00",
            "title": "RTHN: A RNN-Transformer Hierarchical Network for Emotion Cause Extraction",
            "authors": [
                "Rui Xia",
                "Mengran Zhang",
                "Zixiang Ding"
            ],
            "abstract": "The emotion cause extraction (ECE) task aims at discovering the potential\ncauses behind a certain emotion expression in a document. Techniques including\nrule-based methods, traditional machine learning methods and deep neural\nnetworks have been proposed to solve this task. However, most of the previous\nwork considered ECE as a set of independent clause classification problems and\nignored the relations between multiple clauses in a document. In this work, we\npropose a joint emotion cause extraction framework, named RNN-Transformer\nHierarchical Network (RTHN), to encode and classify multiple clauses\nsynchronously. RTHN is composed of a lower word-level encoder based on RNNs to\nencode multiple words in each clause, and an upper clause-level encoder based\non Transformer to learn the correlation between multiple clauses in a document.\nWe furthermore propose ways to encode the relative position and global\npredication information into Transformer that can capture the causality between\nclauses and make RTHN more efficient. We finally achieve the best performance\namong 12 compared systems and improve the F1 score of the state-of-the-art from\n72.69\\% to 76.77\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01239v1",
            "date": "2019-06-04 07:13:49+00:00",
            "title": "Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies",
            "authors": [
                "Shuhei Kurita",
                "Anders S\u00f8gaard"
            ],
            "abstract": "In Semantic Dependency Parsing (SDP), semantic relations form directed\nacyclic graphs, rather than trees. We propose a new iterative predicate\nselection (IPS) algorithm for SDP. Our IPS algorithm combines the graph-based\nand transition-based parsing approaches in order to handle multiple semantic\nhead words. We train the IPS model using a combination of multi-task learning\nand task-specific policy gradient training. Trained this way, IPS achieves a\nnew state of the art on the SemEval 2015 Task 18 datasets. Furthermore, we\nobserve that policy gradient training learns an easy-first strategy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01243v1",
            "date": "2019-06-04 07:27:35+00:00",
            "title": "Learning to Explain: Answering Why-Questions via Rephrasing",
            "authors": [
                "Allen Nie",
                "Erin D. Bennett",
                "Noah D. Goodman"
            ],
            "abstract": "Providing plausible responses to why questions is a challenging but critical\ngoal for language based human-machine interaction. Explanations are challenging\nin that they require many different forms of abstract knowledge and reasoning.\nPrevious work has either relied on human-curated structured knowledge bases or\ndetailed domain representation to generate satisfactory explanations. They are\nalso often limited to ranking pre-existing explanation choices. In our work, we\ncontribute to the under-explored area of generating natural language\nexplanations for general phenomena. We automatically collect large datasets of\nexplanation-phenomenon pairs which allow us to train sequence-to-sequence\nmodels to generate natural language explanations. We compare different training\nstrategies and evaluate their performance using both automatic scores and human\nratings. We demonstrate that our strategy is sufficient to generate highly\nplausible explanations for general open-domain phenomena compared to other\nmodels trained on different datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01265v3",
            "date": "2019-06-04 08:29:00+00:00",
            "title": "ChID: A Large-scale Chinese IDiom Dataset for Cloze Test",
            "authors": [
                "Chujie Zheng",
                "Minlie Huang",
                "Aixin Sun"
            ],
            "abstract": "Cloze-style reading comprehension in Chinese is still limited due to the lack\nof various corpora. In this paper we propose a large-scale Chinese cloze test\ndataset ChID, which studies the comprehension of idiom, a unique language\nphenomenon in Chinese. In this corpus, the idioms in a passage are replaced by\nblank symbols and the correct answer needs to be chosen from well-designed\ncandidate idioms. We carefully study how the design of candidate idioms and the\nrepresentation of idioms affect the performance of state-of-the-art models.\nResults show that the machine accuracy is substantially worse than that of\nhuman, indicating a large space for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01267v1",
            "date": "2019-06-04 08:29:26+00:00",
            "title": "Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts",
            "authors": [
                "Rui Xia",
                "Zixiang Ding"
            ],
            "abstract": "Emotion cause extraction (ECE), the task aimed at extracting the potential\ncauses behind certain emotions in text, has gained much attention in recent\nyears due to its wide applications. However, it suffers from two shortcomings:\n1) the emotion must be annotated before cause extraction in ECE, which greatly\nlimits its applications in real-world scenarios; 2) the way to first annotate\nemotion and then extract the cause ignores the fact that they are mutually\nindicative. In this work, we propose a new task: emotion-cause pair extraction\n(ECPE), which aims to extract the potential pairs of emotions and corresponding\ncauses in a document. We propose a 2-step approach to address this new ECPE\ntask, which first performs individual emotion extraction and cause extraction\nvia multi-task learning, and then conduct emotion-cause pairing and filtering.\nThe experimental results on a benchmark emotion cause corpus prove the\nfeasibility of the ECPE task as well as the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01268v1",
            "date": "2019-06-04 08:29:33+00:00",
            "title": "Exploiting Sentential Context for Neural Machine Translation",
            "authors": [
                "Xing Wang",
                "Zhaopeng Tu",
                "Longyue Wang",
                "Shuming Shi"
            ],
            "abstract": "In this work, we present novel approaches to exploit sentential context for\nneural machine translation (NMT). Specifically, we first show that a shallow\nsentential context extracted from the top encoder layer only, can improve\ntranslation performance via contextualizing the encoding representations of\nindividual words. Next, we introduce a deep sentential context, which\naggregates the sentential context representations from all the internal layers\nof the encoder to form a more comprehensive context representation.\nExperimental results on the WMT14 English-to-German and English-to-French\nbenchmarks show that our model consistently improves performance over the\nstrong TRANSFORMER model (Vaswani et al., 2017), demonstrating the necessity\nand effectiveness of exploiting sentential context for NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01280v1",
            "date": "2019-06-04 08:56:56+00:00",
            "title": "Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection",
            "authors": [
                "Maria Corkery",
                "Yevgen Matusevych",
                "Sharon Goldwater"
            ],
            "abstract": "The cognitive mechanisms needed to account for the English past tense have\nlong been a subject of debate in linguistics and cognitive science. Neural\nnetwork models were proposed early on, but were shown to have clear flaws.\nRecently, however, Kirov and Cotterell (2018) showed that modern\nencoder-decoder (ED) models overcome many of these flaws. They also presented\nevidence that ED models demonstrate humanlike performance in a nonce-word task.\nHere, we look more closely at the behaviour of their model in this task. We\nfind that (1) the model exhibits instability across multiple simulations in\nterms of its correlation with human data, and (2) even when results are\naggregated across simulations (treating each simulation as an individual human\nparticipant), the fit to the human data is not strong---worse than an older\nrule-based model. These findings hold up through several alternative training\nregimes and evaluation measures. Although other neural architectures might do\nbetter, we conclude that there is still insufficient evidence to claim that\nneural nets are a good cognitive model for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01282v1",
            "date": "2019-06-04 08:58:14+00:00",
            "title": "Lattice-Based Transformer Encoder for Neural Machine Translation",
            "authors": [
                "Fengshun Xiao",
                "Jiangtong Li",
                "Hai Zhao",
                "Rui Wang",
                "Kehai Chen"
            ],
            "abstract": "Neural machine translation (NMT) takes deterministic sequences for source\nrepresentations. However, either word-level or subword-level segmentations have\nmultiple choices to split a source sequence with different word segmentors or\ndifferent subword vocabulary sizes. We hypothesize that the diversity in\nsegmentations may affect the NMT performance. To integrate different\nsegmentations with the state-of-the-art NMT model, Transformer, we propose\nlattice-based encoders to explore effective word or subword representation in\nan automatic way during training. We propose two methods: 1) lattice positional\nencoding and 2) lattice-aware self-attention. These two methods can be used\ntogether and show complementary to each other to further improve translation\nperformance. Experiment results show superiorities of lattice-based encoders in\nword-level and subword-level representations over conventional Transformer\nencoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01327v1",
            "date": "2019-06-04 10:34:33+00:00",
            "title": "How Large Are Lions? Inducing Distributions over Quantitative Attributes",
            "authors": [
                "Yanai Elazar",
                "Abhijit Mahabal",
                "Deepak Ramachandran",
                "Tania Bedrax-Weiss",
                "Dan Roth"
            ],
            "abstract": "Most current NLP systems have little knowledge about quantitative attributes\nof objects and events. We propose an unsupervised method for collecting\nquantitative information from large amounts of web data, and use it to create a\nnew, very large resource consisting of distributions over physical quantities\nassociated with objects, adjectives, and verbs which we call Distributions over\nQuantitative (DoQ). This contrasts with recent work in this area which has\nfocused on making only relative comparisons such as \"Is a lion bigger than a\nwolf?\". Our evaluation shows that DoQ compares favorably with state of the art\nresults on existing datasets for relative comparisons of nouns and adjectives,\nand on a new dataset we introduce.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01334v2",
            "date": "2019-06-04 10:51:32+00:00",
            "title": "Curate and Generate: A Corpus and Method for Joint Control of Semantics and Style in Neural NLG",
            "authors": [
                "Shereen Oraby",
                "Vrindavan Harrison",
                "Abteen Ebrahimi",
                "Marilyn Walker"
            ],
            "abstract": "Neural natural language generation (NNLG) from structured meaning\nrepresentations has become increasingly popular in recent years. While we have\nseen progress with generating syntactically correct utterances that preserve\nsemantics, various shortcomings of NNLG systems are clear: new tasks require\nnew training data which is not available or straightforward to acquire, and\nmodel outputs are simple and may be dull and repetitive. This paper addresses\nthese two critical challenges in NNLG by: (1) scalably (and at no cost)\ncreating training datasets of parallel meaning representations and reference\ntexts with rich style markup by using data from freely available and naturally\ndescriptive user reviews, and (2) systematically exploring how the style markup\nenables joint control of semantic and stylistic aspects of neural model output.\nWe present YelpNLG, a corpus of 300,000 rich, parallel meaning representations\nand highly stylistically varied reference texts spanning different restaurant\nattributes, and describe a novel methodology that can be scalably reused to\ngenerate NLG datasets for other domains. The experiments show that the models\ncontrol important aspects, including lexical choice of adjectives, output\nlength, and sentiment, allowing the models to successfully hit multiple style\ntargets without sacrificing semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01343v1",
            "date": "2019-06-04 11:03:49+00:00",
            "title": "A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching",
            "authors": [
                "Jihun Choi",
                "Taeuk Kim",
                "Sang-goo Lee"
            ],
            "abstract": "We present a latent variable model for predicting the relationship between a\npair of text sequences. Unlike previous auto-encoding--based approaches that\nconsider each sequence separately, our proposed framework utilizes both\nsequences within a single model by generating a sequence that has a given\nrelationship with a source sequence. We further extend the cross-sentence\ngenerating framework to facilitate semi-supervised training. We also define\nnovel semantic constraints that lead the decoder network to generate\nsemantically plausible and diverse sequences. We demonstrate the effectiveness\nof the proposed model from quantitative and qualitative experiments, while\nachieving state-of-the-art results on semi-supervised natural language\ninference and paraphrase identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01351v2",
            "date": "2019-06-04 11:25:14+00:00",
            "title": "TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talks",
            "authors": [
                "Guy Lev",
                "Michal Shmueli-Scheuer",
                "Jonathan Herzig",
                "Achiya Jerbi",
                "David Konopnicki"
            ],
            "abstract": "Currently, no large-scale training data is available for the task of\nscientific paper summarization. In this paper, we propose a novel method that\nautomatically generates summaries for scientific papers, by utilizing videos of\ntalks at scientific conferences. We hypothesize that such talks constitute a\ncoherent and concise description of the papers' content, and can form the basis\nfor good summaries. We collected 1716 papers and their corresponding videos,\nand created a dataset of paper summaries. A model trained on this dataset\nachieves similar performance as models trained on a dataset of summaries\ncreated manually. In addition, we validated the quality of our summaries by\nhuman experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01359v1",
            "date": "2019-06-04 11:46:37+00:00",
            "title": "NNE: A Dataset for Nested Named Entity Recognition in English Newswire",
            "authors": [
                "Nicky Ringland",
                "Xiang Dai",
                "Ben Hachey",
                "Sarvnaz Karimi",
                "Cecile Paris",
                "James R. Curran"
            ],
            "abstract": "Named entity recognition (NER) is widely used in natural language processing\napplications and downstream tasks. However, most NER tools target flat\nannotation from popular datasets, eschewing the semantic information available\nin nested entity mentions. We describe NNE---a fine-grained, nested named\nentity dataset over the full Wall Street Journal portion of the Penn Treebank\n(PTB). Our annotation comprises 279,795 mentions of 114 entity types with up to\n6 layers of nesting. We hope the public release of this large dataset for\nEnglish newswire will encourage development of new techniques for nested NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01361v1",
            "date": "2019-06-04 11:47:23+00:00",
            "title": "HighRES: Highlight-based Reference-less Evaluation of Summarization",
            "authors": [
                "Hardy",
                "Shashi Narayan",
                "Andreas Vlachos"
            ],
            "abstract": "There has been substantial progress in summarization research enabled by the\navailability of novel, often large-scale, datasets and recent advances on\nneural network-based approaches. However, manual evaluation of the system\ngenerated summaries is inconsistent due to the difficulty the task poses to\nhuman non-expert readers. To address this issue, we propose a novel approach\nfor manual evaluation, Highlight-based Reference-less Evaluation of\nSummarization (HighRES), in which summaries are assessed by multiple annotators\nagainst the source document via manually highlighted salient content in the\nlatter. Thus summary assessment on the source document by human judges is\nfacilitated, while the highlights can be used for evaluating multiple systems.\nTo validate our approach we employ crowd-workers to augment with highlights a\nrecently proposed dataset and compare two state-of-the-art systems. We\ndemonstrate that HighRES improves inter-annotator agreement in comparison to\nusing the source document directly, while they help emphasize differences among\nsystems that would be ignored under other evaluation approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01373v1",
            "date": "2019-06-04 12:30:02+00:00",
            "title": "Relational Word Embeddings",
            "authors": [
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "While word embeddings have been shown to implicitly encode various forms of\nattributional knowledge, the extent to which they capture relational\ninformation is far more limited. In previous work, this limitation has been\naddressed by incorporating relational knowledge from external knowledge bases\nwhen learning the word embedding. Such strategies may not be optimal, however,\nas they are limited by the coverage of available resources and conflate\nsimilarity with other forms of relatedness. As an alternative, in this paper we\npropose to encode relational knowledge in a separate word embedding, which is\naimed to be complementary to a given standard word embedding. This relational\nword embedding is still learned from co-occurrence statistics, and can thus be\nused even when no external knowledge base is available. Our analysis shows that\nrelational word vectors do indeed capture information that is complementary to\nwhat is encoded in standard word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01378v2",
            "date": "2019-06-04 12:39:10+00:00",
            "title": "Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning",
            "authors": [
                "Minlong Peng",
                "Xiaoyu Xing",
                "Qi Zhang",
                "Jinlan Fu",
                "Xuanjing Huang"
            ],
            "abstract": "In this work, we explore the way to perform named entity recognition (NER)\nusing only unlabeled data and named entity dictionaries. To this end, we\nformulate the task as a positive-unlabeled (PU) learning problem and\naccordingly propose a novel PU learning algorithm to perform the task. We prove\nthat the proposed algorithm can unbiasedly and consistently estimate the task\nloss as if there is fully labeled data. A key feature of the proposed method is\nthat it does not require the dictionaries to label every entity within a\nsentence, and it even does not require the dictionaries to label all of the\nwords constituting an entity. This greatly reduces the requirement on the\nquality of the dictionaries and makes our method generalize well with quite\nsimple dictionaries. Empirical studies on four public NER datasets demonstrate\nthe effectiveness of our proposed method. We have published the source code at\n\\url{https://github.com/v-mipeng/LexiconNER}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01392v1",
            "date": "2019-06-04 13:04:38+00:00",
            "title": "Recognising Agreement and Disagreement between Stances with Reason Comparing Networks",
            "authors": [
                "Chang Xu",
                "Cecile Paris",
                "Surya Nepal",
                "Ross Sparks"
            ],
            "abstract": "We identify agreement and disagreement between utterances that express\nstances towards a topic of discussion. Existing methods focus mainly on\nconversational settings, where dialogic features are used for (dis)agreement\ninference. We extend this scope and seek to detect stance (dis)agreement in a\nbroader setting, where independent stance-bearing utterances, which prevail in\nmany stance corpora and real-world scenarios, are compared. To cope with such\nnon-dialogic utterances, we find that the reasons uttered to back up a specific\nstance can help predict stance (dis)agreements. We propose a reason comparing\nnetwork (RCN) to leverage reason information for stance comparison. Empirical\nresults on a well-known stance corpus show that our method can discover useful\nreason information, enabling it to outperform several baselines in stance\n(dis)agreement detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01472v2",
            "date": "2019-06-04 14:21:22+00:00",
            "title": "Evaluating Discourse in Structured Text Representations",
            "authors": [
                "Elisa Ferracane",
                "Greg Durrett",
                "Junyi Jessy Li",
                "Katrin Erk"
            ],
            "abstract": "Discourse structure is integral to understanding a text and is helpful in\nmany NLP tasks. Learning latent representations of discourse is an attractive\nalternative to acquiring expensive labeled discourse data. Liu and Lapata\n(2018) propose a structured attention mechanism for text classification that\nderives a tree over a text, akin to an RST discourse tree. We examine this\nmodel in detail, and evaluate on additional discourse-relevant tasks and\ndatasets, in order to assess whether the structured attention improves\nperformance on the end task and whether it captures a text's discourse\nstructure. We find the learned latent trees have little to no structure and\ninstead focus on lexical cues; even after obtaining more structured trees with\nproposed model modifications, the trees are still far from capturing discourse\nstructure when compared to discourse dependency trees from an existing\ndiscourse parser. Finally, ablation studies show the structured attention\nprovides little benefit, sometimes even hurting performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01512v1",
            "date": "2019-05-28 23:53:02+00:00",
            "title": "LeafNATS: An Open-Source Toolkit and Live Demo System for Neural Abstractive Text Summarization",
            "authors": [
                "Tian Shi",
                "Ping Wang",
                "Chandan K. Reddy"
            ],
            "abstract": "Neural abstractive text summarization (NATS) has received a lot of attention\nin the past few years from both industry and academia. In this paper, we\nintroduce an open-source toolkit, namely LeafNATS, for training and evaluation\nof different sequence-to-sequence based models for the NATS task, and for\ndeploying the pre-trained models to real-world applications. The toolkit is\nmodularized and extensible in addition to maintaining competitive performance\nin the NATS task. A live news blogging system has also been implemented to\ndemonstrate how these models can aid blog/news editors by providing them\nsuggestions of headlines and summaries of their articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01514v1",
            "date": "2019-05-28 13:11:23+00:00",
            "title": "Adaptive Region Embedding for Text Classification",
            "authors": [
                "Liuyu Xiang",
                "Xiaoming Jin",
                "Lan Yi",
                "Guiguang Ding"
            ],
            "abstract": "Deep learning models such as convolutional neural networks and recurrent\nnetworks are widely applied in text classification. In spite of their great\nsuccess, most deep learning models neglect the importance of modeling context\ninformation, which is crucial to understanding texts. In this work, we propose\nthe Adaptive Region Embedding to learn context representation to improve text\nclassification. Specifically, a metanetwork is learned to generate a context\nmatrix for each region, and each word interacts with its corresponding context\nmatrix to produce the regional representation for further classification.\nCompared to previous models that are designed to capture context information,\nour model contains less parameters and is more flexible. We extensively\nevaluate our method on 8 benchmark datasets for text classification. The\nexperimental results prove that our method achieves state-of-the-art\nperformances and effectively avoids word ambiguity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01543v2",
            "date": "2019-06-04 15:58:54+00:00",
            "title": "Training Neural Response Selection for Task-Oriented Dialogue Systems",
            "authors": [
                "Matthew Henderson",
                "Ivan Vuli\u0107",
                "Daniela Gerz",
                "I\u00f1igo Casanueva",
                "Pawe\u0142 Budzianowski",
                "Sam Coope",
                "Georgios Spithourakis",
                "Tsung-Hsien Wen",
                "Nikola Mrk\u0161i\u0107",
                "Pei-Hao Su"
            ],
            "abstract": "Despite their popularity in the chatbot literature, retrieval-based models\nhave had modest impact on task-oriented dialogue systems, with the main\nobstacle to their application being the low-data regime of most task-oriented\ndialogue tasks. Inspired by the recent success of pretraining in language\nmodelling, we propose an effective method for deploying response selection in\ntask-oriented dialogue. To train response selection models for task-oriented\ndialogue tasks, we propose a novel method which: 1) pretrains the response\nselection model on large general-domain conversational corpora; and then 2)\nfine-tunes the pretrained model for the target dialogue domain, relying only on\nthe small in-domain dataset to capture the nuances of the given dialogue\ndomain. Our evaluation on six diverse application domains, ranging from\ne-commerce to banking, demonstrates the effectiveness of the proposed training\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01569v1",
            "date": "2019-06-04 16:36:53+00:00",
            "title": "Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation",
            "authors": [
                "Benjamin Heinzerling",
                "Michael Strube"
            ],
            "abstract": "Pretrained contextual and non-contextual subword embeddings have become\navailable in over 250 languages, allowing massively multilingual NLP. However,\nwhile there is no dearth of pretrained embeddings, the distinct lack of\nsystematic evaluations makes it difficult for practitioners to choose between\nthem. In this work, we conduct an extensive evaluation comparing non-contextual\nsubword embeddings, namely FastText and BPEmb, and a contextual representation\nmethod, namely BERT, on multilingual named entity recognition and\npart-of-speech tagging. We find that overall, a combination of BERT, BPEmb, and\ncharacter representations works best across languages and tasks. A more\ndetailed analysis reveals different strengths and weaknesses: Multilingual BERT\nperforms well in medium- to high-resource languages, but is outperformed by\nnon-contextual subword embeddings in a low-resource setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01573v1",
            "date": "2019-06-04 16:37:44+00:00",
            "title": "A Study of Feature Extraction techniques for Sentiment Analysis",
            "authors": [
                "Avinash Madasu",
                "Sivasankar E"
            ],
            "abstract": "Sentiment Analysis refers to the study of systematically extracting the\nmeaning of subjective text . When analysing sentiments from the subjective text\nusing Machine Learning techniques,feature extraction becomes a significant\npart. We perform a study on the performance of feature extraction techniques\nTF-IDF(Term Frequency-Inverse Document Frequency) and Doc2vec (Document to\nVector) using Cornell movie review datasets, UCI sentiment labeled datasets,\nstanford movie review datasets,effectively classifying the text into positive\nand negative polarities by using various pre-processing methods like\neliminating StopWords and Tokenization which increases the performance of\nsentiment analysis in terms of accuracy and time taken by the classifier.The\nfeatures obtained after applying feature extraction techniques on the text\nsentences are trained and tested using the classifiers Logistic\nRegression,Support Vector Machines,K-Nearest Neighbours , Decision Tree and\nBernoulli Nave Bayes",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01575v1",
            "date": "2019-06-04 16:41:15+00:00",
            "title": "Pitfalls in the Evaluation of Sentence Embeddings",
            "authors": [
                "Steffen Eger",
                "Andreas R\u00fcckl\u00e9",
                "Iryna Gurevych"
            ],
            "abstract": "Deep learning models continuously break new records across different NLP\ntasks. At the same time, their success exposes weaknesses of model evaluation.\nHere, we compile several key pitfalls of evaluation of sentence embeddings, a\ncurrently very popular NLP paradigm. These pitfalls include the comparison of\nembeddings of different sizes, normalization of embeddings, and the low (and\ndiverging) correlations between transfer and probing tasks. Our motivation is\nto challenge the current evaluation of sentence embeddings and to provide an\neasy-to-access reference for future research. Based on our insights, we also\nrecommend better practices for better future evaluations of sentence\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01617v1",
            "date": "2019-06-04 17:51:03+00:00",
            "title": "Self-Attentional Models for Lattice Inputs",
            "authors": [
                "Matthias Sperber",
                "Graham Neubig",
                "Ngoc-Quan Pham",
                "Alex Waibel"
            ],
            "abstract": "Lattices are an efficient and effective method to encode ambiguity of\nupstream systems in natural language processing tasks, for example to compactly\ncapture multiple speech recognition hypotheses, or to represent multiple\nlinguistic analyses. Previous work has extended recurrent neural networks to\nmodel lattice inputs and achieved improvements in various tasks, but these\nmodels suffer from very slow computation speeds. This paper extends the\nrecently proposed paradigm of self-attention to handle lattice inputs.\nSelf-attention is a sequence modeling technique that relates inputs to one\nanother by computing pairwise similarities and has gained popularity for both\nits strong results and its computational efficiency. To extend such models to\nhandle lattices, we introduce probabilistic reachability masks that incorporate\nlattice structure into the model and support lattice scores if available. We\nalso propose a method for adapting positional embeddings to lattice structures.\nWe apply the proposed model to a speech translation task and find that it\noutperforms all examined baselines while being much faster to compute than\nprevious neural lattice models during both training and inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01659v2",
            "date": "2019-06-04 18:03:13+00:00",
            "title": "Towards Lossless Encoding of Sentences",
            "authors": [
                "Gabriele Prato",
                "Mathieu Duchesneau",
                "Sarath Chandar",
                "Alain Tapp"
            ],
            "abstract": "A lot of work has been done in the field of image compression via machine\nlearning, but not much attention has been given to the compression of natural\nlanguage. Compressing text into lossless representations while making features\neasily retrievable is not a trivial task, yet has huge benefits. Most methods\ndesigned to produce feature rich sentence embeddings focus solely on performing\nwell on downstream tasks and are unable to properly reconstruct the original\nsequence from the learned embedding. In this work, we propose a near lossless\nmethod for encoding long sequences of texts as well as all of their\nsub-sequences into feature rich representations. We test our method on\nsentiment analysis and show good performance across all sub-sentence and\nsentence embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01661v2",
            "date": "2019-06-04 18:04:14+00:00",
            "title": "Detecting Syntactic Change Using a Neural Part-of-Speech Tagger",
            "authors": [
                "William Merrill",
                "Gigi Felice Stark",
                "Robert Frank"
            ],
            "abstract": "We train a diachronic long short-term memory (LSTM) part-of-speech tagger on\na large corpus of American English from the 19th, 20th, and 21st centuries. We\nanalyze the tagger's ability to implicitly learn temporal structure between\nyears, and the extent to which this knowledge can be transferred to date new\nsentences. The learned year embeddings show a strong linear correlation between\ntheir first principal component and time. We show that temporal information\nencoded in the model can be used to predict novel sentences' years of\ncomposition relatively well. Comparisons to a feedforward baseline suggest that\nthe temporal change learned by the LSTM is syntactic rather than purely\nlexical. Thus, our results suggest that our tagger is implicitly learning to\nmodel syntactic change in American English over the course of the 19th, 20th,\nand early 21st centuries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01685v1",
            "date": "2019-06-04 19:05:11+00:00",
            "title": "Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain",
            "authors": [
                "Samuel L\u00e4ubli",
                "Chantal Amrhein",
                "Patrick D\u00fcggelin",
                "Beatriz Gonzalez",
                "Alena Zwahlen",
                "Martin Volk"
            ],
            "abstract": "Neural machine translation (NMT) has set new quality standards in automatic\ntranslation, yet its effect on post-editing productivity is still pending\nthorough investigation. We empirically test how the inclusion of NMT, in\naddition to domain-specific translation memories and termbases, impacts speed\nand quality in professional translation of financial texts. We find that even\nwith language pairs that have received little attention in research settings\nand small amounts of in-domain data for system adaptation, NMT post-editing\nallows for substantial time savings and leads to equal or slightly better\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01688v1",
            "date": "2019-06-04 19:19:46+00:00",
            "title": "Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change",
            "authors": [
                "Haim Dubossarsky",
                "Simon Hengchen",
                "Nina Tahmasebi",
                "Dominik Schlechtweg"
            ],
            "abstract": "State-of-the-art models of lexical semantic change detection suffer from\nnoise stemming from vector space alignment. We have empirically tested the\nTemporal Referencing method for lexical semantic change and show that, by\navoiding alignment, it is less affected by this noise. We show that, trained on\na diachronic corpus, the skip-gram with negative sampling architecture with\ntemporal referencing outperforms alignment models on a synthetic task as well\nas a manual testset. We introduce a principled way to simulate lexical semantic\nchange and systematically control for possible biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01698v1",
            "date": "2019-06-04 19:41:10+00:00",
            "title": "Open Sesame: Getting Inside BERT's Linguistic Knowledge",
            "authors": [
                "Yongjie Lin",
                "Yi Chern Tan",
                "Robert Frank"
            ],
            "abstract": "How and to what extent does BERT encode syntactically-sensitive hierarchical\ninformation or positionally-sensitive linear information? Recent work has shown\nthat contextual representations like BERT perform well on tasks that require\nsensitivity to linguistic structure. We present here two studies which aim to\nprovide a better understanding of the nature of BERT's representations. The\nfirst of these focuses on the identification of structurally-defined elements\nusing diagnostic classifiers, while the second explores BERT's representation\nof subject-verb agreement and anaphor-antecedent dependencies through a\nquantitative assessment of self-attention vectors. In both cases, we find that\nBERT encodes positional information about word tokens well on its lower layers,\nbut switches to a hierarchically-oriented encoding on higher layers. We\nconclude then that BERT's representations do indeed model linguistically\nrelevant aspects of hierarchical structure, though they do not appear to show\nthe sharp sensitivity to hierarchical structure that is found in human\nprocessing of reflexive anaphora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01749v3",
            "date": "2019-06-04 23:00:43+00:00",
            "title": "Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model",
            "authors": [
                "Alexander R. Fabbri",
                "Irene Li",
                "Tianwei She",
                "Suyi Li",
                "Dragomir R. Radev"
            ],
            "abstract": "Automatic generation of summaries from multiple news articles is a valuable\ntool as the number of online publications grows rapidly. Single document\nsummarization (SDS) systems have benefited from advances in neural\nencoder-decoder model thanks to the availability of large datasets. However,\nmulti-document summarization (MDS) of news articles has been limited to\ndatasets of a couple of hundred examples. In this paper, we introduce\nMulti-News, the first large-scale MDS news dataset. Additionally, we propose an\nend-to-end model which incorporates a traditional extractive summarization\nmodel with a standard SDS model and achieves competitive results on MDS\ndatasets. We benchmark several methods on Multi-News and release our data and\ncode in hope that this work will promote advances in summarization in the\nmulti-document setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01753v1",
            "date": "2019-06-04 23:36:50+00:00",
            "title": "Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution",
            "authors": [
                "Shany Barhom",
                "Vered Shwartz",
                "Alon Eirew",
                "Michael Bugert",
                "Nils Reimers",
                "Ido Dagan"
            ],
            "abstract": "Recognizing coreferring events and entities across multiple texts is crucial\nfor many NLP applications. Despite the task's importance, research focus was\ngiven mostly to within-document entity coreference, with rather little\nattention to the other variants. We propose a neural architecture for\ncross-document coreference resolution. Inspired by Lee et al (2012), we jointly\nmodel entity and event coreference. We represent an event (entity) mention\nusing its lexical span, surrounding context, and relation to entity (event)\nmentions via predicate-arguments structures. Our model outperforms the previous\nstate-of-the-art event coreference model on ECB+, while providing the first\nentity coreference results on this corpus. Our analysis confirms that all our\nrepresentation elements, including the mention span itself, its context, and\nthe relation to other mentions contribute to the model's success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01762v1",
            "date": "2019-06-05 00:23:50+00:00",
            "title": "Entity-Centric Contextual Affective Analysis",
            "authors": [
                "Anjalie Field",
                "Yulia Tsvetkov"
            ],
            "abstract": "While contextualized word representations have improved state-of-the-art\nbenchmarks in many NLP tasks, their potential usefulness for social-oriented\ntasks remains largely unexplored. We show how contextualized word embeddings\ncan be used to capture affect dimensions in portrayals of people. We evaluate\nour methodology quantitatively, on held-out affect lexicons, and qualitatively,\nthrough case examples. We find that contextualized word representations do\nencode meaningful affect information, but they are heavily biased towards their\ntraining data, which limits their usefulness to in-domain analyses. We\nultimately use our method to examine differences in portrayals of men and\nwomen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01781v1",
            "date": "2019-06-05 01:56:50+00:00",
            "title": "Generating Multiple Diverse Responses with Multi-Mapping and Posterior Mapping Selection",
            "authors": [
                "Chaotao Chen",
                "Jinhua Peng",
                "Fan Wang",
                "Jun Xu",
                "Hua Wu"
            ],
            "abstract": "In human conversation an input post is open to multiple potential responses,\nwhich is typically regarded as a one-to-many problem. Promising approaches\nmainly incorporate multiple latent mechanisms to build the one-to-many\nrelationship. However, without accurate selection of the latent mechanism\ncorresponding to the target response during training, these methods suffer from\na rough optimization of latent mechanisms. In this paper, we propose a\nmulti-mapping mechanism to better capture the one-to-many relationship, where\nmultiple mapping modules are employed as latent mechanisms to model the\nsemantic mappings from an input post to its diverse responses. For accurate\noptimization of latent mechanisms, a posterior mapping selection module is\ndesigned to select the corresponding mapping module according to the target\nresponse for further optimization. We also introduce an auxiliary matching loss\nto facilitate the optimization of posterior mapping selection. Empirical\nresults demonstrate the superiority of our model in generating multiple diverse\nand informative responses over the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01788v1",
            "date": "2019-06-05 02:24:15+00:00",
            "title": "Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference",
            "authors": [
                "He Bai",
                "Yu Zhou",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Dialogue contexts are proven helpful in the spoken language understanding\n(SLU) system and they are typically encoded with explicit memory\nrepresentations. However, most of the previous models learn the context memory\nwith only one objective to maximizing the SLU performance, leaving the context\nmemory under-exploited. In this paper, we propose a new dialogue logistic\ninference (DLI) task to consolidate the context memory jointly with SLU in the\nmulti-task framework. DLI is defined as sorting a shuffled dialogue session\ninto its original logical order and shares the same memory encoder and\nretrieval mechanism as the SLU model. Our experimental results show that\nvarious popular contextual SLU models can benefit from our approach, and\nimprovements are quite impressive, especially in slot filling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01834v1",
            "date": "2019-06-05 05:40:26+00:00",
            "title": "Automatic Generation of High Quality CCGbanks for Parser Domain Adaptation",
            "authors": [
                "Masashi Yoshikawa",
                "Hiroshi Noji",
                "Koji Mineshima",
                "Daisuke Bekki"
            ],
            "abstract": "We propose a new domain adaptation method for Combinatory Categorial Grammar\n(CCG) parsing, based on the idea of automatic generation of CCG corpora\nexploiting cheaper resources of dependency trees. Our solution is conceptually\nsimple, and not relying on a specific parser architecture, making it applicable\nto the current best-performing parsers. We conduct extensive parsing\nexperiments with detailed discussion; on top of existing benchmark datasets on\n(1) biomedical texts and (2) question sentences, we create experimental\ndatasets of (3) speech conversation and (4) math problems. When applied to the\nproposed method, an off-the-shelf CCG parser shows significant performance\ngains, improving from 90.7% to 96.6% on speech conversation, and from 88.5% to\n96.8% on math problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01926v1",
            "date": "2019-06-05 10:34:56+00:00",
            "title": "A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity",
            "authors": [
                "Yoshinari Fujinuma",
                "Jordan Boyd-Graber",
                "Michael J. Paul"
            ],
            "abstract": "Cross-lingual word embeddings encode the meaning of words from different\nlanguages into a shared low-dimensional space. An important requirement for\nmany downstream tasks is that word similarity should be independent of language\n- i.e., word vectors within one language should not be more similar to each\nother than to words in another language. We measure this characteristic using\nmodularity, a network measurement that measures the strength of clusters in a\ngraph. Modularity has a moderate to strong correlation with three downstream\ntasks, even though modularity is based only on the structure of embeddings and\ndoes not require any external resources. We show through experiments that\nmodularity can serve as an intrinsic validation metric to improve unsupervised\ncross-lingual word embeddings, particularly on distant language pairs in\nlow-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01973v2",
            "date": "2019-06-05 12:28:00+00:00",
            "title": "A Hierarchical Decoder with Three-level Hierarchical Attention to Generate Abstractive Summaries of Interleaved Texts",
            "authors": [
                "Sanjeev Kumar Karn",
                "Francine Chen",
                "Yan-Ying Chen",
                "Ulli Waltinger",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Interleaved texts, where posts belonging to different threads occur in one\nsequence, are a common occurrence, e.g., online chat conversations. To quickly\nobtain an overview of such texts, existing systems first disentangle the posts\nby threads and then extract summaries from those threads. The major issues with\nsuch systems are error propagation and non-fluent summary. To address those, we\npropose an end-to-end trainable hierarchical encoder-decoder system. We also\nintroduce a novel hierarchical attention mechanism which combines three levels\nof information from an interleaved text, i.e, posts, phrases and words, and\nimplicitly disentangles the threads. We evaluated the proposed system on\nmultiple interleaved text datasets, and it out-performs a SOTA two-step system\nby 20-40%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02002v1",
            "date": "2019-06-05 12:54:14+00:00",
            "title": "Every child should have parents: a taxonomy refinement algorithm based on hyperbolic term embeddings",
            "authors": [
                "Rami Aly",
                "Shantanu Acharya",
                "Alexander Ossa",
                "Arne K\u00f6hn",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "We introduce the use of Poincar\\'e embeddings to improve existing\nstate-of-the-art approaches to domain-specific taxonomy induction from text as\na signal for both relocating wrong hyponym terms within a (pre-induced)\ntaxonomy as well as for attaching disconnected terms in a taxonomy. This method\nsubstantially improves previous state-of-the-art results on the SemEval-2016\nTask 13 on taxonomy extraction. We demonstrate the superiority of Poincar\\'e\nembeddings over distributional semantic representations, supporting the\nhypothesis that they can better capture hierarchical lexical-semantic\nrelationships than embeddings in the Euclidean space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02041v2",
            "date": "2019-06-05 14:15:47+00:00",
            "title": "Imitation Learning for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Bingzhen Wei",
                "Mingxuan Wang",
                "Hao Zhou",
                "Junyang Lin",
                "Jun Xie",
                "Xu Sun"
            ],
            "abstract": "Non-autoregressive translation models (NAT) have achieved impressive\ninference speedup. A potential issue of the existing NAT algorithms, however,\nis that the decoding is conducted in parallel, without directly considering\nprevious context. In this paper, we propose an imitation learning framework for\nnon-autoregressive machine translation, which still enjoys the fast translation\nspeed but gives comparable translation performance compared to its\nauto-regressive counterpart. We conduct experiments on the IWSLT16, WMT14 and\nWMT16 datasets. Our proposed model achieves a significant speedup over the\nautoregressive models, while keeping the translation quality comparable to the\nautoregressive models. By sampling sentence length in parallel at inference\ntime, we achieve the performance of 31.85 BLEU on WMT16 Ro$\\rightarrow$En and\n30.68 BLEU on IWSLT16 En$\\rightarrow$De.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02045v2",
            "date": "2019-06-05 14:23:01+00:00",
            "title": "The FRENK Datasets of Socially Unacceptable Discourse in Slovene and English",
            "authors": [
                "Nikola Ljube\u0161i\u0107",
                "Darja Fi\u0161er",
                "Toma\u017e Erjavec"
            ],
            "abstract": "In this paper we present datasets of Facebook comment threads to mainstream\nmedia posts in Slovene and English developed inside the Slovene national\nproject FRENK which cover two topics, migrants and LGBT, and are manually\nannotated for different types of socially unacceptable discourse (SUD). The\nmain advantages of these datasets compared to the existing ones are identical\nsampling procedures, producing comparable data across languages and an\nannotation schema that takes into account six types of SUD and five targets at\nwhich SUD is directed. We describe the sampling and annotation procedures, and\nanalyze the annotation distributions and inter-annotator agreements. We\nconsider this dataset to be an important milestone in understanding and\ncombating SUD for both languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02053v1",
            "date": "2019-06-05 14:36:59+00:00",
            "title": "KAS-term: Extracting Slovene Terms from Doctoral Theses via Supervised Machine Learning",
            "authors": [
                "Nikola Ljube\u0161i\u0107",
                "Darja Fi\u0161er",
                "Toma\u017e Erjavec"
            ],
            "abstract": "This paper presents a dataset and supervised learning experiments for term\nextraction from Slovene academic texts. Term candidates in the dataset were\nextracted via morphosyntactic patterns and annotated for their termness by four\nannotators. Experiments on the dataset show that most co-occurrence statistics,\napplied after morphosyntactic patterns and a frequency threshold, perform close\nto random and that the results can be significantly improved by combining, with\nsupervised machine learning, all the seven statistic measures included in the\ndataset. On multi-word terms the model using all statistics obtains an AUC of\n0.736 while the best single statistic produces only AUC 0.590. Among many\nadditional candidate features, only adding multi-word morphosyntactic pattern\ninformation and length of the single-word term candidates achieves further\nimprovements of the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02059v1",
            "date": "2019-06-05 14:56:06+00:00",
            "title": "Neural Legal Judgment Prediction in English",
            "authors": [
                "Ilias Chalkidis",
                "Ion Androutsopoulos",
                "Nikolaos Aletras"
            ],
            "abstract": "Legal judgment prediction is the task of automatically predicting the outcome\nof a court case, given a text describing the case's facts. Previous work on\nusing neural models for this task has focused on Chinese; only feature-based\nmodels (e.g., using bags of words and topics) have been considered in English.\nWe release a new English legal judgment prediction dataset, containing cases\nfrom the European Court of Human Rights. We evaluate a broad variety of neural\nmodels on the new dataset, establishing strong baselines that surpass previous\nfeature-based models in three tasks: (1) binary violation classification; (2)\nmulti-label classification; (3) case importance prediction. We also explore if\nmodels are biased towards demographic information via data anonymization. As a\nside-product, we propose a hierarchical version of BERT, which bypasses BERT's\nlength limitation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02079v1",
            "date": "2019-06-05 15:41:22+00:00",
            "title": "Learning to Rank for Plausible Plausibility",
            "authors": [
                "Zhongyang Li",
                "Tongfei Chen",
                "Benjamin Van Durme"
            ],
            "abstract": "Researchers illustrate improvements in contextual encoding strategies via\nresultant performance on a battery of shared Natural Language Understanding\n(NLU) tasks. Many of these tasks are of a categorical prediction variety: given\na conditioning context (e.g., an NLI premise), provide a label based on an\nassociated prompt (e.g., an NLI hypothesis). The categorical nature of these\ntasks has led to common use of a cross entropy log-loss objective during\ntraining. We suggest this loss is intuitively wrong when applied to\nplausibility tasks, where the prompt by design is neither categorically\nentailed nor contradictory given the context. Log-loss naturally drives models\nto assign scores near 0.0 or 1.0, in contrast to our proposed use of a\nmargin-based loss. Following a discussion of our intuition, we describe a\nconfirmation study based on an extreme, synthetically curated task derived from\nMultiNLI. We find that a margin-based loss leads to a more plausible model of\nplausibility. Finally, we illustrate improvements on the Choice Of Plausible\nAlternative (COPA) task through this change in loss.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02128v1",
            "date": "2019-05-17 05:13:22+00:00",
            "title": "Recovering Dropped Pronouns in Chinese Conversations via Modeling Their Referents",
            "authors": [
                "Jingxuan Yang",
                "Jianzhuo Tong",
                "Si Li",
                "Sheng Gao",
                "Jun Guo",
                "Nianwen Xue"
            ],
            "abstract": "Pronouns are often dropped in Chinese sentences, and this happens more\nfrequently in conversational genres as their referents can be easily understood\nfrom context. Recovering dropped pronouns is essential to applications such as\nInformation Extraction where the referents of these dropped pronouns need to be\nresolved, or Machine Translation when Chinese is the source language. In this\nwork, we present a novel end-to-end neural network model to recover dropped\npronouns in conversational data. Our model is based on a structured attention\nmechanism that models the referents of dropped pronouns utilizing both\nsentence-level and word-level information. Results on three different\nconversational genres show that our approach achieves a significant improvement\nover the current state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02136v1",
            "date": "2019-05-23 14:48:39+00:00",
            "title": "LMF Reloaded",
            "authors": [
                "Laurent Romary",
                "Mohamed Khemakhem",
                "Fahad Khan",
                "Jack Bowers",
                "Nicoletta Calzolari",
                "Monte George",
                "Mandy Pet",
                "Piotr Ba\u0144ski"
            ],
            "abstract": "Lexical Markup Framework (LMF) or ISO 24613 [1] is a de jure standard that\nprovides a framework for modelling and encoding lexical information in\nretrodigitised print dictionaries and NLP lexical databases. An in-depth review\nis currently underway within the standardisation subcommittee ,\nISO-TC37/SC4/WG4, to find a more modular, flexible and durable follow up to the\noriginal LMF standard published in 2008. In this paper we will present some of\nthe major improvements which have so far been implemented in the new version of\nLMF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02192v1",
            "date": "2019-06-05 14:41:01+00:00",
            "title": "Large-Scale Multi-Label Text Classification on EU Legislation",
            "authors": [
                "Ilias Chalkidis",
                "Manos Fergadiotis",
                "Prodromos Malakasiotis",
                "Ion Androutsopoulos"
            ],
            "abstract": "We consider Large-Scale Multi-Label Text Classification (LMTC) in the legal\ndomain. We release a new dataset of 57k legislative documents from EURLEX,\nannotated with ~4.3k EUROVOC labels, which is suitable for LMTC, few- and\nzero-shot learning. Experimenting with several neural classifiers, we show that\nBIGRUs with label-wise attention perform better than other current state of the\nart methods. Domain-specific WORD2VEC and context-sensitive ELMO embeddings\nfurther improve performance. We also find that considering only particular\nzones of the documents is sufficient. This allows us to bypass BERT's maximum\ntext length limit and fine-tune BERT, obtaining the best results in all but\nzero-shot learning cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02243v1",
            "date": "2019-06-05 18:40:53+00:00",
            "title": "Energy and Policy Considerations for Deep Learning in NLP",
            "authors": [
                "Emma Strubell",
                "Ananya Ganesh",
                "Andrew McCallum"
            ],
            "abstract": "Recent progress in hardware and methodology for training neural networks has\nushered in a new generation of large networks trained on abundant data. These\nmodels have obtained notable gains in accuracy across many NLP tasks. However,\nthese accuracy improvements depend on the availability of exceptionally large\ncomputational resources that necessitate similarly substantial energy\nconsumption. As a result these models are costly to train and develop, both\nfinancially, due to the cost of hardware and electricity or cloud compute time,\nand environmentally, due to the carbon footprint required to fuel modern tensor\nprocessing hardware. In this paper we bring this issue to the attention of NLP\nresearchers by quantifying the approximate financial and environmental costs of\ntraining a variety of recently successful neural network models for NLP. Based\non these findings, we propose actionable recommendations to reduce costs and\nimprove equity in NLP research and practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02276v1",
            "date": "2019-06-05 19:45:21+00:00",
            "title": "An Imitation Learning Approach to Unsupervised Parsing",
            "authors": [
                "Bowen Li",
                "Lili Mou",
                "Frank Keller"
            ],
            "abstract": "Recently, there has been an increasing interest in unsupervised parsers that\noptimize semantically oriented objectives, typically using reinforcement\nlearning. Unfortunately, the learned trees often do not match actual syntax\ntrees well. Shen et al. (2018) propose a structured attention mechanism for\nlanguage modeling (PRPN), which induces better syntactic structures but relies\non ad hoc heuristics. Also, their model lacks interpretability as it is not\ngrounded in parsing actions. In our work, we propose an imitation learning\napproach to unsupervised parsing, where we transfer the syntactic knowledge\ninduced by the PRPN to a Tree-LSTM model with discrete parsing actions. Its\npolicy is then refined by Gumbel-Softmax training towards a semantically\noriented objective. We evaluate our approach on the All Natural Language\nInference dataset and show that it achieves a new state of the art in terms of\nparsing $F$-score, outperforming our base models, including the PRPN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02358v8",
            "date": "2019-06-05 23:36:06+00:00",
            "title": "Survey on Publicly Available Sinhala Natural Language Processing Tools and Research",
            "authors": [
                "Nisansa de Silva"
            ],
            "abstract": "Sinhala is the native language of the Sinhalese people who make up the\nlargest ethnic group of Sri Lanka. The language belongs to the globe-spanning\nlanguage tree, Indo-European. However, due to poverty in both linguistic and\neconomic capital, Sinhala, in the perspective of Natural Language Processing\ntools and research, remains a resource-poor language which has neither the\neconomic drive its cousin English has nor the sheer push of the law of numbers\na language such as Chinese has. A number of research groups from Sri Lanka have\nnoticed this dearth and the resultant dire need for proper tools and research\nfor Sinhala natural language processing. However, due to various reasons, these\nattempts seem to lack coordination and awareness of each other. The objective\nof this paper is to fill that gap of a comprehensive literature survey of the\npublicly available Sinhala natural language tools and research so that the\nresearchers working in this field can better utilize contributions of their\npeers. As such, we shall be uploading this paper to arXiv and perpetually\nupdate it periodically to reflect the advances made in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02361v1",
            "date": "2019-06-06 00:02:37+00:00",
            "title": "Explain Yourself! Leveraging Language Models for Commonsense Reasoning",
            "authors": [
                "Nazneen Fatema Rajani",
                "Bryan McCann",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Deep learning models perform poorly on tasks that require commonsense\nreasoning, which often necessitates some form of world-knowledge or reasoning\nover information not immediately present in the input. We collect human\nexplanations for commonsense reasoning in the form of natural language\nsequences and highlighted annotations in a new dataset called Common Sense\nExplanations (CoS-E). We use CoS-E to train language models to automatically\ngenerate explanations that can be used during training and inference in a novel\nCommonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the\nstate-of-the-art by 10% on the challenging CommonsenseQA task. We further study\ncommonsense reasoning in DNNs using both human and auto-generated explanations\nincluding transfer to out-of-domain tasks. Empirical results indicate that we\ncan effectively leverage language models for commonsense reasoning.",
            "journal ref": "In Proceedings of the Association for Computational Linguistics\n  (ACL), 2019. Florence, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02402v1",
            "date": "2019-06-06 03:53:58+00:00",
            "title": "An Analysis of Emotion Communication Channels in Fan Fiction: Towards Emotional Storytelling",
            "authors": [
                "Evgeny Kim",
                "Roman Klinger"
            ],
            "abstract": "Centrality of emotion for the stories told by humans is underpinned by\nnumerous studies in literature and psychology. The research in automatic\nstorytelling has recently turned towards emotional storytelling, in which\ncharacters' emotions play an important role in the plot development. However,\nthese studies mainly use emotion to generate propositional statements in the\nform \"A feels affection towards B\" or \"A confronts B\". At the same time,\nemotional behavior does not boil down to such propositional descriptions, as\nhumans display complex and highly variable patterns in communicating their\nemotions, both verbally and non-verbally. In this paper, we analyze how\nemotions are expressed non-verbally in a corpus of fan fiction short stories.\nOur analysis shows that stories written by humans convey character emotions\nalong various non-verbal channels. We find that some non-verbal channels, such\nas facial expressions and voice characteristics of the characters, are more\nstrongly associated with joy, while gestures and body postures are more likely\nto occur with trust. Based on our analysis, we argue that automatic\nstorytelling systems should take variability of emotion into account when\ngenerating descriptions of characters' emotions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02430v4",
            "date": "2019-06-06 05:58:42+00:00",
            "title": "Shift-of-Perspective Identification Within Legal Cases",
            "authors": [
                "Gathika Ratnayaka",
                "Thejan Rupasinghe",
                "Nisansa de Silva",
                "Viraj Salaka Gamage",
                "Menuka Warushavithana",
                "Amal Shehan Perera"
            ],
            "abstract": "Arguments, counter-arguments, facts, and evidence obtained via documents\nrelated to previous court cases are of essential need for legal professionals.\nTherefore, the process of automatic information extraction from documents\ncontaining legal opinions related to court cases can be considered to be of\nsignificant importance. This study is focused on the identification of\nsentences in legal opinion texts which convey different perspectives on a\ncertain topic or entity. We combined several approaches based on semantic\nanalysis, open information extraction, and sentiment analysis to achieve our\nobjective. Then, our methodology was evaluated with the help of human judges.\nThe outcomes of the evaluation demonstrate that our system is successful in\ndetecting situations where two sentences deliver different opinions on the same\ntopic or entity. The proposed methodology can be used to facilitate other\ninformation extraction tasks related to the legal domain. One such task is the\nautomated detection of counter arguments for a given argument. Another is the\nidentification of opponent parties in a court case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02437v1",
            "date": "2019-06-06 06:31:21+00:00",
            "title": "GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling",
            "authors": [
                "Yijin Liu",
                "Fandong Meng",
                "Jinchao Zhang",
                "Jinan Xu",
                "Yufeng Chen",
                "Jie Zhou"
            ],
            "abstract": "Current state-of-the-art systems for sequence labeling are typically based on\nthe family of Recurrent Neural Networks (RNNs). However, the shallow\nconnections between consecutive hidden states of RNNs and insufficient modeling\nof global information restrict the potential performance of those models. In\nthis paper, we try to address these issues, and thus propose a Global Context\nenhanced Deep Transition architecture for sequence labeling named GCDT. We\ndeepen the state transition path at each position in a sentence, and further\nassign every token with a global representation learned from the entire\nsentence. Experiments on two standard sequence labeling tasks show that, given\nonly training data and the ubiquitous word embeddings (Glove), our GCDT\nachieves 91.96 F1 on the CoNLL03 NER task and 95.43 F1 on the CoNLL2000\nChunking task, which outperforms the best reported results under the same\nsettings. Furthermore, by leveraging BERT as an additional resource, we\nestablish new state-of-the-art results with 93.47 F1 on NER and 97.30 F1 on\nChunking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02443v1",
            "date": "2019-06-06 07:02:04+00:00",
            "title": "Robust Neural Machine Translation with Doubly Adversarial Inputs",
            "authors": [
                "Yong Cheng",
                "Lu Jiang",
                "Wolfgang Macherey"
            ],
            "abstract": "Neural machine translation (NMT) often suffers from the vulnerability to\nnoisy perturbations in the input. We propose an approach to improving the\nrobustness of NMT models, which consists of two parts: (1) attack the\ntranslation model with adversarial source examples; (2) defend the translation\nmodel with adversarial target inputs to improve its robustness against the\nadversarial source inputs.For the generation of adversarial inputs, we propose\na gradient-based method to craft adversarial examples informed by the\ntranslation loss over the clean inputs.Experimental results on Chinese-English\nand English-German translation tasks demonstrate that our approach achieves\nsignificant improvements ($2.8$ and $1.6$ BLEU points) over Transformer on\nstandard clean benchmarks as well as exhibiting higher robustness on noisy\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02479v2",
            "date": "2019-06-06 08:44:10+00:00",
            "title": "Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Sampling",
            "authors": [
                "Dominik Schlechtweg",
                "Cennet Oguz",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We simulate first- and second-order context overlap and show that Skip-Gram\nwith Negative Sampling is similar to Singular Value Decomposition in capturing\nsecond-order co-occurrence information, while Pointwise Mutual Information is\nagnostic to it. We support the results with an empirical study finding that the\nmodels react differently when provided with additional second-order\ninformation. Our findings reveal a basic property of Skip-Gram with Negative\nSampling and point towards an explanation of its success on a variety of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02505v1",
            "date": "2019-06-06 10:22:24+00:00",
            "title": "Fine-Grained Entity Typing in Hyperbolic Space",
            "authors": [
                "Federico L\u00f3pez",
                "Benjamin Heinzerling",
                "Michael Strube"
            ],
            "abstract": "How can we represent hierarchical information present in large type\ninventories for entity typing? We study the ability of hyperbolic embeddings to\ncapture hierarchical relations between mentions in context and their target\ntypes in a shared vector space. We evaluate on two datasets and investigate two\ndifferent techniques for creating a large hierarchical entity type inventory:\nfrom an expert-generated ontology and by automatically mining type\nco-occurrences. We find that the hyperbolic model yields improvements over its\nEuclidean counterpart in some, but not all cases. Our analysis suggests that\nthe adequacy of this geometry depends on the granularity of the type inventory\nand the way hierarchical relations are inferred.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02510v1",
            "date": "2019-06-06 10:44:41+00:00",
            "title": "Derivational Morphological Relations in Word Embeddings",
            "authors": [
                "Tom\u00e1\u0161 Musil",
                "Jon\u00e1\u0161 Vidra",
                "David Mare\u010dek"
            ],
            "abstract": "Derivation is a type of a word-formation process which creates new words from\nexisting ones by adding, changing or deleting affixes. In this paper, we\nexplore the potential of word embeddings to identify properties of word\nderivations in the morphologically rich Czech language. We extract derivational\nrelations between pairs of words from DeriNet, a Czech lexical network, which\norganizes almost one million Czech lemmata into derivational trees. For each\nsuch pair, we compute the difference of the embeddings of the two words, and\nperform unsupervised clustering of the resulting vectors. Our results show that\nthese clusters largely match manually annotated semantic categories of the\nderivational relations (e.g. the relation 'bake--baker' belongs to category\n'actor', and a correct clustering puts it into the same cluster as\n'govern--governor').",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02525v1",
            "date": "2019-06-06 11:31:24+00:00",
            "title": "Cross-Lingual Training for Automatic Question Generation",
            "authors": [
                "Vishwajeet Kumar",
                "Nitish Joshi",
                "Arijit Mukherjee",
                "Ganesh Ramakrishnan",
                "Preethi Jyothi"
            ],
            "abstract": "Automatic question generation (QG) is a challenging problem in natural\nlanguage understanding. QG systems are typically built assuming access to a\nlarge number of training instances where each instance is a question and its\ncorresponding answer. For a new language, such training instances are hard to\nobtain making the QG problem even more challenging. Using this as our\nmotivation, we study the reuse of an available large QG dataset in a secondary\nlanguage (e.g. English) to learn a QG model for a primary language (e.g. Hindi)\nof interest. For the primary language, we assume access to a large amount of\nmonolingual text but only a small QG dataset. We propose a cross-lingual QG\nmodel which uses the following training regime: (i) Unsupervised pretraining of\nlanguage models in both primary and secondary languages and (ii) joint\nsupervised training for QG in both languages. We demonstrate the efficacy of\nour proposed approach using two different primary languages, Hindi and Chinese.\nWe also create and release a new question answering dataset for Hindi\nconsisting of 6555 sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02563v2",
            "date": "2019-06-06 13:12:35+00:00",
            "title": "Measuring the compositionality of noun-noun compounds over time",
            "authors": [
                "Prajit Dhar",
                "Janis Pagel",
                "Lonneke van der Plas"
            ],
            "abstract": "We present work in progress on the temporal progression of compositionality\nin noun-noun compounds. Previous work has proposed computational methods for\ndetermining the compositionality of compounds. These methods try to\nautomatically determine how transparent the meaning of the compound as a whole\nis with respect to the meaning of its parts. We hypothesize that such a\nproperty might change over time. We use the time-stamped Google Books corpus\nfor our diachronic investigations, and first examine whether the vector-based\nsemantic spaces extracted from this corpus are able to predict compositionality\nratings, despite their inherent limitations. We find that using temporal\ninformation helps predicting the ratings, although correlation with the ratings\nis lower than reported for other corpora. Finally, we show changes in\ncompositionality over time for a selection of compounds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02622v2",
            "date": "2019-06-06 14:53:04+00:00",
            "title": "Generating Question-Answer Hierarchies",
            "authors": [
                "Kalpesh Krishna",
                "Mohit Iyyer"
            ],
            "abstract": "The process of knowledge acquisition can be viewed as a question-answer game\nbetween a student and a teacher in which the student typically starts by asking\nbroad, open-ended questions before drilling down into specifics (Hintikka,\n1981; Hakkarainen and Sintonen, 2002). This pedagogical perspective motivates a\nnew way of representing documents. In this paper, we present SQUASH\n(Specificity-controlled Question-Answer Hierarchies), a novel and challenging\ntext generation task that converts an input document into a hierarchy of\nquestion-answer pairs. Users can click on high-level questions (e.g., \"Why did\nFrodo leave the Fellowship?\") to reveal related but more specific questions\n(e.g., \"Who did Frodo leave with?\"). Using a question taxonomy loosely based on\nLehnert (1978), we classify questions in existing reading comprehension\ndatasets as either \"general\" or \"specific\". We then use these labels as input\nto a pipelined system centered around a conditional neural language model. We\nextensively evaluate the quality of the generated QA hierarchies through\ncrowdsourced experiments and report strong empirical results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02780v1",
            "date": "2019-06-06 19:16:16+00:00",
            "title": "Syntactically Supervised Transformers for Faster Neural Machine Translation",
            "authors": [
                "Nader Akoury",
                "Kalpesh Krishna",
                "Mohit Iyyer"
            ],
            "abstract": "Standard decoders for neural machine translation autoregressively generate a\nsingle target token per time step, which slows inference especially for long\noutputs. While architectural advances such as the Transformer fully parallelize\nthe decoder computations at training time, inference still proceeds\nsequentially. Recent developments in non- and semi- autoregressive decoding\nproduce multiple tokens per time step independently of the others, which\nimproves inference speed but deteriorates translation quality. In this work, we\npropose the syntactically supervised Transformer (SynST), which first\nautoregressively predicts a chunked parse tree before generating all of the\ntarget tokens in one shot conditioned on the predicted parse. A series of\ncontrolled experiments demonstrates that SynST decodes sentences ~ 5x faster\nthan the baseline autoregressive Transformer while achieving higher BLEU scores\nthan most competing methods on En-De and En-Fr datasets.",
            "journal ref": "Association for Computational Linguistics (2019) 1269-1281",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02782v1",
            "date": "2019-06-06 19:25:30+00:00",
            "title": "From Receptive to Productive: Learning to Use Confusing Words through Automatically Selected Example Sentences",
            "authors": [
                "Chieh-Yang Huang",
                "Yi-Ting Huang",
                "Mei-Hua Chen",
                "Lun-Wei Ku"
            ],
            "abstract": "Knowing how to use words appropriately has been a key to improving language\nproficiency. Previous studies typically discuss how students learn receptively\nto select the correct candidate from a set of confusing words in the\nfill-in-the-blank task where specific context is given. In this paper, we go\none step further, assisting students to learn to use confusing words\nappropriately in a productive task: sentence translation. We leverage the\nGiveMeExample system, which suggests example sentences for each confusing word,\nto achieve this goal. In this study, students learn to differentiate the\nconfusing words by reading the example sentences, and then choose the\nappropriate word(s) to complete the sentence translation task. Results show\nstudents made substantial progress in terms of sentence structure. In addition,\nhighly proficient students better managed to learn confusing words. In view of\nthe influence of the first language on learners, we further propose an\neffective approach to improve the quality of the suggested sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02829v1",
            "date": "2019-06-06 21:53:53+00:00",
            "title": "Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications",
            "authors": [
                "Wei Zhao",
                "Haiyun Peng",
                "Steffen Eger",
                "Erik Cambria",
                "Min Yang"
            ],
            "abstract": "Obstacles hindering the development of capsule networks for challenging NLP\napplications include poor scalability to large output spaces and less reliable\nrouting processes. In this paper, we introduce: 1) an agreement score to\nevaluate the performance of routing processes at instance level; 2) an adaptive\noptimizer to enhance the reliability of routing; 3) capsule compression and\npartial routing to improve the scalability of capsule networks. We validate our\napproach on two NLP tasks, namely: multi-label text classification and question\nanswering. Experimental results show that our approach considerably improves\nover strong competitors on both tasks. In addition, we gain the best results in\nlow-resource settings with few training instances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02868v2",
            "date": "2019-06-07 02:30:31+00:00",
            "title": "Modeling financial analysts' decision making via the pragmatics and semantics of earnings calls",
            "authors": [
                "Katherine A. Keith",
                "Amanda Stent"
            ],
            "abstract": "Every fiscal quarter, companies hold earnings calls in which company\nexecutives respond to questions from analysts. After these calls, analysts\noften change their price target recommendations, which are used in equity\nresearch reports to help investors make decisions. In this paper, we examine\nanalysts' decision making behavior as it pertains to the language content of\nearnings calls. We identify a set of 20 pragmatic features of analysts'\nquestions which we correlate with analysts' pre-call investor recommendations.\nWe also analyze the degree to which semantic and pragmatic features from an\nearnings call complement market data in predicting analysts' post-call changes\nin price targets. Our results show that earnings calls are moderately\npredictive of analysts' decisions even though these decisions are influenced by\na number of other factors including private communication with company\nexecutives and market conditions. A breakdown of model errors indicates\ndisparate performance on calls from different market sectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02923v1",
            "date": "2019-06-07 06:42:24+00:00",
            "title": "Preference-based Interactive Multi-Document Summarisation",
            "authors": [
                "Yang Gao",
                "Christian M. Meyer",
                "Iryna Gurevych"
            ],
            "abstract": "Interactive NLP is a promising paradigm to close the gap between automatic\nNLP systems and the human upper bound. Preference-based interactive learning\nhas been successfully applied, but the existing methods require several\nthousand interaction rounds even in simulations with perfect user feedback. In\nthis paper, we study preference-based interactive summarisation. To reduce the\nnumber of interaction rounds, we propose the Active Preference-based\nReInforcement Learning (APRIL) framework. APRIL uses Active Learning to query\nthe user, Preference Learning to learn a summary ranking function from the\npreferences, and neural Reinforcement Learning to efficiently search for the\n(near-)optimal summary. Our results show that users can easily provide reliable\npreferences over summaries and that APRIL outperforms the state-of-the-art\npreference-based interactive method in both simulation and real-user\nexperiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02979v1",
            "date": "2019-06-07 09:19:47+00:00",
            "title": "A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains",
            "authors": [
                "Dominik Schlechtweg",
                "Anna H\u00e4tty",
                "Marco del Tredici",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We perform an interdisciplinary large-scale evaluation for detecting lexical\nsemantic divergences in a diachronic and in a synchronic task: semantic sense\nchanges across time, and semantic sense changes across domains. Our work\naddresses the superficialness and lack of comparison in assessing models of\ndiachronic lexical change, by bringing together and extending benchmark models\non a common state-of-the-art evaluation task. In addition, we demonstrate that\nthe same evaluation task and modelling approaches can successfully be utilised\nfor the synchronic detection of domain-specific sense divergences in the field\nof term extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03007v1",
            "date": "2019-06-07 11:05:30+00:00",
            "title": "On the Compositionality Prediction of Noun Phrases using Poincar\u00e9 Embeddings",
            "authors": [
                "Abhik Jana",
                "Dmitry Puzyrev",
                "Alexander Panchenko",
                "Pawan Goyal",
                "Chris Biemann",
                "Animesh Mukherjee"
            ],
            "abstract": "The compositionality degree of multiword expressions indicates to what extent\nthe meaning of a phrase can be derived from the meaning of its constituents and\ntheir grammatical relations. Prediction of (non)-compositionality is a task\nthat has been frequently addressed with distributional semantic models. We\nintroduce a novel technique to blend hierarchical information with\ndistributional information for predicting compositionality. In particular, we\nuse hypernymy information of the multiword and its constituents encoded in the\nform of the recently introduced Poincar\\'e embeddings in addition to the\ndistributional information to detect compositionality for noun phrases. Using a\nweighted average of the distributional similarity and a Poincar\\'e similarity\nfunction, we obtain consistent and substantial, statistically significant\nimprovement across three gold standard datasets over state-of-the-art models\nbased on distributional information only. Unlike traditional approaches that\nsolely use an unsupervised setting, we have also framed the problem as a\nsupervised task, obtaining comparable improvements. Further, we publicly\nrelease our Poincar\\'e embeddings, which are trained on the output of\nhandcrafted lexical-syntactic patterns on a large corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03008v2",
            "date": "2019-06-07 11:06:11+00:00",
            "title": "RankQA: Neural Question Answering with Answer Re-Ranking",
            "authors": [
                "Bernhard Kratzwald",
                "Anna Eigenmann",
                "Stefan Feuerriegel"
            ],
            "abstract": "The conventional paradigm in neural question answering (QA) for narrative\ncontent is limited to a two-stage process: first, relevant text passages are\nretrieved and, subsequently, a neural network for machine comprehension\nextracts the likeliest answer. However, both stages are largely isolated in the\nstatus quo and, hence, information from the two phases is never properly fused.\nIn contrast, this work proposes RankQA: RankQA extends the conventional\ntwo-stage process in neural QA with a third stage that performs an additional\nanswer re-ranking. The re-ranking leverages different features that are\ndirectly extracted from the QA pipeline, i.e., a combination of retrieval and\ncomprehension features. While our intentionally simple design allows for an\nefficient, data-sparse estimation, it nevertheless outperforms more complex QA\nsystems by a significant margin: in fact, RankQA achieves state-of-the-art\nperformance on 3 out of 4 benchmark datasets. Furthermore, its performance is\nespecially superior in settings where the size of the corpus is dynamic. Here\nthe answer re-ranking provides an effective remedy against the underlying\nnoise-information trade-off due to a variable corpus size. As a consequence,\nRankQA represents a novel, powerful, and thus challenging baseline for future\nresearch in content-based QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03088v1",
            "date": "2019-06-07 13:31:09+00:00",
            "title": "Improving Relation Extraction by Pre-trained Language Representations",
            "authors": [
                "Christoph Alt",
                "Marc H\u00fcbner",
                "Leonhard Hennig"
            ],
            "abstract": "Current state-of-the-art relation extraction methods typically rely on a set\nof lexical, syntactic, and semantic features, explicitly computed in a\npre-processing step. Training feature extraction models requires additional\nannotated language resources, which severely restricts the applicability and\nportability of relation extraction to novel languages. Similarly,\npre-processing introduces an additional source of error. To address these\nlimitations, we introduce TRE, a Transformer for Relation Extraction, extending\nthe OpenAI Generative Pre-trained Transformer [Radford et al., 2018]. Unlike\nprevious relation extraction models, TRE uses pre-trained deep language\nrepresentations instead of explicit linguistic features to inform the relation\nclassification and combines it with the self-attentive Transformer architecture\nto effectively model long-range dependencies between entity mentions. TRE\nallows us to learn implicit linguistic features solely from plain text corpora\nby unsupervised pre-training, before fine-tuning the learned language\nrepresentations on the relation extraction task. TRE obtains a new\nstate-of-the-art result on the TACRED and SemEval 2010 Task 8 datasets,\nachieving a test F1 of 67.4 and 87.1, respectively. Furthermore, we observe a\nsignificant increase in sample efficiency. With only 20% of the training\nexamples, TRE matches the performance of our baselines and our model trained\nfrom scratch on 100% of the TACRED dataset. We open-source our trained models,\nexperiments, and source code.",
            "journal ref": "Proceedings of AKBC 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03134v1",
            "date": "2019-06-07 14:45:49+00:00",
            "title": "Word Embeddings for the Armenian Language: Intrinsic and Extrinsic Evaluation",
            "authors": [
                "Karen Avetisyan",
                "Tsolak Ghukasyan"
            ],
            "abstract": "In this work, we intrinsically and extrinsically evaluate and compare\nexisting word embedding models for the Armenian language. Alongside, new\nembeddings are presented, trained using GloVe, fastText, CBOW, SkipGram\nalgorithms. We adapt and use the word analogy task in intrinsic evaluation of\nembeddings. For extrinsic evaluation, two tasks are employed: morphological\ntagging and text classification. Tagging is performed on a deep neural network,\nusing ArmTDP v2.3 dataset. For text classification, we propose a corpus of news\narticles categorized into 7 classes. The datasets are made public to serve as\nbenchmarks for future models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03209v2",
            "date": "2019-06-07 16:17:31+00:00",
            "title": "Building a Production Model for Retrieval-Based Chatbots",
            "authors": [
                "Kyle Swanson",
                "Lili Yu",
                "Christopher Fox",
                "Jeremy Wohlwend",
                "Tao Lei"
            ],
            "abstract": "Response suggestion is an important task for building human-computer\nconversation systems. Recent approaches to conversation modeling have\nintroduced new model architectures with impressive results, but relatively\nlittle attention has been paid to whether these models would be practical in a\nproduction setting. In this paper, we describe the unique challenges of\nbuilding a production retrieval-based conversation system, which selects\noutputs from a whitelist of candidate responses. To address these challenges,\nwe propose a dual encoder architecture which performs rapid inference and\nscales well with the size of the whitelist. We also introduce and compare two\nmethods for generating whitelists, and we carry out a comprehensive analysis of\nthe model and whitelists. Experimental results on a large, proprietary help\ndesk chat dataset, including both offline metrics and a human evaluation,\nindicate production-quality performance and illustrate key lessons about\nconversation modeling in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03221v1",
            "date": "2019-06-07 16:41:52+00:00",
            "title": "Data-to-text Generation with Entity Modeling",
            "authors": [
                "Ratish Puduppully",
                "Li Dong",
                "Mirella Lapata"
            ],
            "abstract": "Recent approaches to data-to-text generation have shown great promise thanks\nto the use of large-scale datasets and the application of neural network\narchitectures which are trained end-to-end. These models rely on representation\nlearning to select content appropriately, structure it coherently, and\nverbalize it grammatically, treating entities as nothing more than vocabulary\ntokens. In this work we propose an entity-centric neural architecture for\ndata-to-text generation. Our model creates entity-specific representations\nwhich are dynamically updated. Text is generated conditioned on the data input\nand entity memory representations using hierarchical attention at each time\nstep. We present experiments on the RotoWire benchmark and a (five times\nlarger) new dataset on the baseball domain which we create. Our results show\nthat the proposed model outperforms competitive baselines in automatic and\nhuman evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03249v3",
            "date": "2019-06-07 17:29:18+00:00",
            "title": "Learning Word Embeddings with Domain Awareness",
            "authors": [
                "Guoyin Wang",
                "Yan Song",
                "Yue Zhang",
                "Dong Yu"
            ],
            "abstract": "Word embeddings are traditionally trained on a large corpus in an\nunsupervised setting, with no specific design for incorporating domain\nknowledge. This can lead to unsatisfactory performances when training data\noriginate from heterogeneous domains. In this paper, we propose two novel\nmechanisms for domain-aware word embedding training, namely domain indicator\nand domain attention, which integrate domain-specific knowledge into the widely\nused SG and CBOW models, respectively. The two methods are based on a joint\nlearning paradigm and ensure that words in a target domain are intensively\nfocused when trained on a source domain corpus. Qualitative and quantitative\nevaluation confirm the validity and effectiveness of our models. Compared to\nbaseline methods, our method is particularly effective in near-cold-start\nscenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03338v1",
            "date": "2019-06-07 21:36:49+00:00",
            "title": "Dissecting Content and Context in Argumentative Relation Analysis",
            "authors": [
                "Juri Opitz",
                "Anette Frank"
            ],
            "abstract": "When assessing relations between argumentative units (e.g., support or\nattack), computational systems often exploit disclosing indicators or markers\nthat are not part of elementary argumentative units (EAUs) themselves, but are\ngained from their context (position in paragraph, preceding tokens, etc.). We\nshow that this dependency is much stronger than previously assumed. In fact, we\nshow that by completely masking the EAU text spans and only feeding information\nfrom their context, a competitive system may function even better. We argue\nthat an argument analysis system that relies more on discourse context than the\nargument's content is unsafe, since it can easily be tricked. To alleviate this\nissue, we separate argumentative units from their context such that the system\nis forced to model and rely on an EAU's content. We show that the resulting\nclassification system is more robust, and argue that such models are better\nsuited for predicting argumentative relations across documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03380v1",
            "date": "2019-06-08 03:32:00+00:00",
            "title": "Clinical Concept Extraction for Document-Level Coding",
            "authors": [
                "Sarah Wiegreffe",
                "Edward Choi",
                "Sherry Yan",
                "Jimeng Sun",
                "Jacob Eisenstein"
            ],
            "abstract": "The text of clinical notes can be a valuable source of patient information\nand clinical assessments. Historically, the primary approach for exploiting\nclinical notes has been information extraction: linking spans of text to\nconcepts in a detailed domain ontology. However, recent work has demonstrated\nthe potential of supervised machine learning to extract document-level codes\ndirectly from the raw text of clinical notes. We propose to bridge the gap\nbetween the two approaches with two novel syntheses: (1) treating extracted\nconcepts as features, which are used to supplement or replace the text of the\nnote; (2) treating extracted concepts as labels, which are used to learn a\nbetter representation of the text. Unfortunately, the resulting concepts do not\nyield performance gains on the document-level clinical coding task. We explore\npossible explanations and future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03492v1",
            "date": "2019-06-08 17:40:33+00:00",
            "title": "Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations",
            "authors": [
                "Rui Zhang",
                "Caitlin Westerfield",
                "Sungrok Shim",
                "Garrett Bingham",
                "Alexander Fabbri",
                "Neha Verma",
                "William Hu",
                "Dragomir Radev"
            ],
            "abstract": "In this paper, we propose to boost low-resource cross-lingual document\nretrieval performance with deep bilingual query-document representations. We\nmatch queries and documents in both source and target languages with four\ncomponents, each of which is implemented as a term interaction-based deep\nneural network with cross-lingual word embeddings as input. By including query\nlikelihood scores as extra features, our model effectively learns to rerank the\nretrieved documents by using a small number of relevance labels for\nlow-resource language pairs. Due to the shared cross-lingual word embedding\nspace, the model can also be directly applied to another language pair without\nany training label. Experimental results on the MATERIAL dataset show that our\nmodel outperforms the competitive translation-based baselines on\nEnglish-Swahili, English-Tagalog, and English-Somali cross-lingual information\nretrieval tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03497v1",
            "date": "2019-06-08 18:31:33+00:00",
            "title": "This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation",
            "authors": [
                "Rui Zhang",
                "Joel Tetreault"
            ],
            "abstract": "Given the overwhelming number of emails, an effective subject line becomes\nessential to better inform the recipient of the email's content. In this paper,\nwe propose and study the task of email subject line generation: automatically\ngenerating an email subject line from the email body. We create the first\ndataset for this task and find that email subject line generation favor\nextremely abstractive summary which differentiates it from news headline\ngeneration or news single document summarization. We then develop a novel deep\nlearning method and compare it to several baselines as well as recent\nstate-of-the-art text summarization systems. We also investigate the efficacy\nof several automatic metrics based on correlations with human judgments and\npropose a new automatic evaluation metric. Our system outperforms competitive\nbaselines given both automatic and human evaluations. To our knowledge, this is\nthe first work to tackle the problem of effective email subject line\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03508v1",
            "date": "2019-06-08 19:27:31+00:00",
            "title": "Sentence Centrality Revisited for Unsupervised Summarization",
            "authors": [
                "Hao Zheng",
                "Mirella Lapata"
            ],
            "abstract": "Single document summarization has enjoyed renewed interests in recent years\nthanks to the popularity of neural network models and the availability of\nlarge-scale datasets. In this paper we develop an unsupervised approach arguing\nthat it is unrealistic to expect large-scale and high-quality training data to\nbe available or created for different types of summaries, domains, or\nlanguages. We revisit a popular graph-based ranking algorithm and modify how\nnode (aka sentence) centrality is computed in two ways: (a)~we employ BERT, a\nstate-of-the-art neural representation learning model to better capture\nsentential meaning and (b)~we build graphs with directed edges arguing that the\ncontribution of any two nodes to their respective centrality is influenced by\ntheir relative position in a document. Experimental results on three news\nsummarization datasets representative of different languages and writing styles\nshow that our approach outperforms strong baselines by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03538v1",
            "date": "2019-06-08 23:18:07+00:00",
            "title": "Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims",
            "authors": [
                "Sihao Chen",
                "Daniel Khashabi",
                "Wenpeng Yin",
                "Chris Callison-Burch",
                "Dan Roth"
            ],
            "abstract": "One key consequence of the information revolution is a significant increase\nand a contamination of our information supply. The practice of fact checking\nwon't suffice to eliminate the biases in text data we observe, as the degree of\nfactuality alone does not determine whether biases exist in the spectrum of\nopinions visible to us. To better understand controversial issues, one needs to\nview them from a diverse yet comprehensive set of perspectives. For example,\nthere are many ways to respond to a claim such as \"animals should have lawful\nrights\", and these responses form a spectrum of perspectives, each with a\nstance relative to this claim and, ideally, with evidence supporting it.\nInherently, this is a natural language understanding task, and we propose to\naddress it as such. Specifically, we propose the task of substantiated\nperspective discovery where, given a claim, a system is expected to discover a\ndiverse set of well-corroborated perspectives that take a stance with respect\nto the claim. Each perspective should be substantiated by evidence paragraphs\nwhich summarize pertinent results and facts. We construct PERSPECTRUM, a\ndataset of claims, perspectives and evidence, making use of online debate\nwebsites to create the initial data collection, and augmenting it using search\nengines in order to expand and diversify our dataset. We use crowd-sourcing to\nfilter out noise and ensure high-quality data. Our dataset contains 1k claims,\naccompanied with pools of 10k and 8k perspective sentences and evidence\nparagraphs, respectively. We provide a thorough analysis of the dataset to\nhighlight key underlying language understanding challenges, and show that human\nbaselines across multiple subtasks far outperform ma-chine baselines built upon\nstate-of-the-art NLP techniques. This poses a challenge and opportunity for the\nNLP community to address.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03634v2",
            "date": "2019-06-09 13:12:45+00:00",
            "title": "Learning to Predict Novel Noun-Noun Compounds",
            "authors": [
                "Prajit Dhar",
                "Lonneke van der Plas"
            ],
            "abstract": "We introduce temporally and contextually-aware models for the novel task of\npredicting unseen but plausible concepts, as conveyed by noun-noun compounds in\na time-stamped corpus. We train compositional models on observed compounds,\nmore specifically the composed distributed representations of their\nconstituents across a time-stamped corpus, while giving it corrupted instances\n(where head or modifier are replaced by a random constituent) as negative\nevidence. The model captures generalisations over this data and learns what\ncombinations give rise to plausible compounds and which ones do not. After\ntraining, we query the model for the plausibility of automatically generated\nnovel combinations and verify whether the classifications are accurate. For our\nbest model, we find that in around 85% of the cases, the novel compounds\ngenerated are attested in previously unseen data. An additional estimated 5%\nare plausible despite not being attested in the recent corpus, based on\njudgments from independent human raters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03656v1",
            "date": "2019-06-09 15:18:53+00:00",
            "title": "Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification",
            "authors": [
                "Tu Vu",
                "Mohit Iyyer"
            ],
            "abstract": "While paragraph embedding models are remarkably effective for downstream\nclassification tasks, what they learn and encode into a single vector remains\nopaque. In this paper, we investigate a state-of-the-art paragraph embedding\nmethod proposed by Zhang et al. (2017) and discover that it cannot reliably\ntell whether a given sentence occurs in the input paragraph or not. We\nformulate a sentence content task to probe for this basic linguistic property\nand find that even a much simpler bag-of-words method has no trouble solving\nit. This result motivates us to replace the reconstruction-based objective of\nZhang et al. (2017) with our sentence content probe objective in a\nsemi-supervised setting. Despite its simplicity, our objective improves over\nparagraph reconstruction in terms of (1) downstream classification accuracies\non benchmark datasets, (2) faster training, and (3) better generalization\nability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03677v1",
            "date": "2019-06-09 17:28:37+00:00",
            "title": "Happy Together: Learning and Understanding Appraisal From Natural Language",
            "authors": [
                "Arun Rajendran",
                "Chiyu Zhang",
                "Muhammad Abdul-Mageed"
            ],
            "abstract": "In this paper, we explore various approaches for learning two types of\nappraisal components from happy language. We focus on 'agency' of the author\nand the 'sociality' involved in happy moments based on the HappyDB dataset. We\ndevelop models based on deep neural networks for the task, including uni- and\nbi-directional long short-term memory networks, with and without attention. We\nalso experiment with a number of novel embedding methods, such as embedding\nfrom neural machine translation (as in CoVe) and embedding from language models\n(as in ELMo). We compare our results to those acquired by several traditional\nmachine learning methods. Our best models achieve 87.97% accuracy on agency and\n93.13% accuracy on sociality, both of which are significantly higher than our\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03692v1",
            "date": "2019-06-09 19:14:18+00:00",
            "title": "UBC-NLP at SemEval-2019 Task 6:Ensemble Learning of Offensive Content With Enhanced Training Data",
            "authors": [
                "Arun Rajendran",
                "Chiyu Zhang",
                "Muhammad Abdul-Mageed"
            ],
            "abstract": "We examine learning offensive content on Twitter with limited, imbalanced\ndata. For the purpose, we investigate the utility of using various data\nenhancement methods with a host of classical ensemble classifiers. Among the 75\nparticipating teams in SemEval-2019 sub-task B, our system ranks 6th (with\n0.706 macro F1-score). For sub-task C, among the 65 participating teams, our\nsystem ranks 9th (with 0.587 macro F1-score).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03717v1",
            "date": "2019-06-09 21:39:46+00:00",
            "title": "Argument Generation with Retrieval, Planning, and Realization",
            "authors": [
                "Xinyu Hua",
                "Zhe Hu",
                "Lu Wang"
            ],
            "abstract": "Automatic argument generation is an appealing but challenging task. In this\npaper, we study the specific problem of counter-argument generation, and\npresent a novel framework, CANDELA. It consists of a powerful retrieval system\nand a novel two-step generation model, where a text planning decoder first\ndecides on the main talking points and a proper language style for each\nsentence, then a content realization decoder reflects the decisions and\nconstructs an informative paragraph-level argument. Furthermore, our generation\nmodel is empowered by a retrieval system indexed with 12 million articles\ncollected from Wikipedia and popular English news media, which provides access\nto high-quality content with diversity. Automatic evaluation on a large-scale\ndataset collected from Reddit shows that our model yields significantly higher\nBLEU, ROUGE, and METEOR scores than the state-of-the-art and non-trivial\ncomparisons. Human evaluation further indicates that our system arguments are\nmore appropriate for refutation and richer in content.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03731v1",
            "date": "2019-06-09 22:46:12+00:00",
            "title": "Is Attention Interpretable?",
            "authors": [
                "Sofia Serrano",
                "Noah A. Smith"
            ],
            "abstract": "Attention mechanisms have recently boosted performance on a range of NLP\ntasks. Because attention layers explicitly weight input components'\nrepresentations, it is also often assumed that attention can be used to\nidentify information that models found important (e.g., specific contextualized\nword tokens). We test whether that assumption holds by manipulating attention\nweights in already-trained text classification models and analyzing the\nresulting differences in their predictions. While we observe some ways in which\nhigher attention weights correlate with greater impact on model predictions, we\nalso find many ways in which this does not hold, i.e., where gradient-based\nrankings of attention weights better predict their effects than their\nmagnitudes. We conclude that while attention noisily predicts input components'\noverall importance to a model, it is by no means a fail-safe indicator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03753v2",
            "date": "2019-06-10 01:10:34+00:00",
            "title": "Out-of-Vocabulary Embedding Imputation with Grounded Language Information by Graph Convolutional Networks",
            "authors": [
                "Ziyi Yang",
                "Chenguang Zhu",
                "Vin Sachidananda",
                "Eric Darve"
            ],
            "abstract": "Due to the ubiquitous use of embeddings as input representations for a wide\nrange of natural language tasks, imputation of embeddings for rare and unseen\nwords is a critical problem in language processing. Embedding imputation\ninvolves learning representations for rare or unseen words during the training\nof an embedding model, often in a post-hoc manner. In this paper, we propose an\napproach for embedding imputation which uses grounded information in the form\nof a knowledge graph. This is in contrast to existing approaches which\ntypically make use of vector space properties or subword information. We\npropose an online method to construct a graph from grounded information and\ndesign an algorithm to map from the resulting graphical structure to the space\nof the pre-trained embeddings. Finally, we evaluate our approach on a range of\nrare and unseen word tasks across various domains and show that our model can\nlearn better representations. For example, on the Card-660 task our method\nimproves Pearson's and Spearman's correlation coefficients upon the\nstate-of-the-art by 11% and 17.8% respectively using GloVe embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03783v1",
            "date": "2019-06-10 03:15:00+00:00",
            "title": "Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks",
            "authors": [
                "Hongyu Lin",
                "Yaojie Lu",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "Sequential labeling-based NER approaches restrict each word belonging to at\nmost one entity mention, which will face a serious problem when recognizing\nnested entity mentions. In this paper, we propose to resolve this problem by\nmodeling and leveraging the head-driven phrase structures of entity mentions,\ni.e., although a mention can nest other mentions, they will not share the same\nhead word. Specifically, we propose Anchor-Region Networks (ARNs), a\nsequence-to-nuggets architecture for nested mention detection. ARNs first\nidentify anchor words (i.e., possible head words) of all mentions, and then\nrecognize the mention boundaries for each anchor word by exploiting regular\nphrase structures. Furthermore, we also design Bag Loss, an objective function\nwhich can train ARNs in an end-to-end manner without using any anchor word\nannotation. Experiments show that ARNs achieve the state-of-the-art performance\non three standard nested entity mention detection benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03785v1",
            "date": "2019-06-10 03:28:57+00:00",
            "title": "Generalized Data Augmentation for Low-Resource Translation",
            "authors": [
                "Mengzhou Xia",
                "Xiang Kong",
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "Translation to or from low-resource languages LRLs poses challenges for\nmachine translation in terms of both adequacy and fluency. Data augmentation\nutilizing large amounts of monolingual data is regarded as an effective way to\nalleviate these problems. In this paper, we propose a general framework for\ndata augmentation in low-resource machine translation that not only uses\ntarget-side monolingual data, but also pivots through a related high-resource\nlanguage HRL. Specifically, we experiment with a two-step pivoting method to\nconvert high-resource data to the LRL, making use of available resources to\nbetter approximate the true data distribution of the LRL. First, we inject LRL\nwords into HRL sentences through an induced bilingual dictionary. Second, we\nfurther edit these modified sentences using a modified unsupervised machine\ntranslation framework. Extensive experiments on four low-resource datasets show\nthat under extreme low-resource settings, our data augmentation techniques\nimprove translation quality by up to~1.5 to~8 BLEU points compared to\nsupervised back-translation baselines",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03820v1",
            "date": "2019-06-10 07:22:05+00:00",
            "title": "Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification",
            "authors": [
                "Minghao Hu",
                "Yuxing Peng",
                "Zhen Huang",
                "Dongsheng Li",
                "Yiwei Lv"
            ],
            "abstract": "Open-domain targeted sentiment analysis aims to detect opinion targets along\nwith their sentiment polarities from a sentence. Prior work typically\nformulates this task as a sequence tagging problem. However, such formulation\nsuffers from problems such as huge search space and sentiment inconsistency. To\naddress these problems, we propose a span-based extract-then-classify\nframework, where multiple opinion targets are directly extracted from the\nsentence under the supervision of target span boundaries, and corresponding\npolarities are then classified using their span representations. We further\ninvestigate three approaches under this framework, namely the pipeline, joint,\nand collapsed models. Experiments on three benchmark datasets show that our\napproach consistently outperforms the sequence tagging baseline. Moreover, we\nfind that the pipeline model achieves the best performance compared with the\nother two models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03889v1",
            "date": "2019-06-10 10:40:39+00:00",
            "title": "Topic-Aware Neural Keyphrase Generation for Social Media Language",
            "authors": [
                "Yue Wang",
                "Jing Li",
                "Hou Pong Chan",
                "Irwin King",
                "Michael R. Lyu",
                "Shuming Shi"
            ],
            "abstract": "A huge volume of user-generated content is daily produced on social media. To\nfacilitate automatic language understanding, we study keyphrase prediction,\ndistilling salient information from massive posts. While most existing methods\nextract words from source posts to form keyphrases, we propose a\nsequence-to-sequence (seq2seq) based neural keyphrase generation framework,\nenabling absent keyphrases to be created. Moreover, our model, being\ntopic-aware, allows joint modeling of corpus-level latent topic\nrepresentations, which helps alleviate the data sparsity that widely exhibited\nin social media language. Experiments on three datasets collected from English\nand Chinese social media platforms show that our model significantly\noutperforms both extraction and generation models that do not exploit latent\ntopics. Further discussions show that our model learns meaningful topics, which\ninterprets its superiority in social media keyphrase generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03952v1",
            "date": "2019-06-10 12:57:56+00:00",
            "title": "Multimodal Logical Inference System for Visual-Textual Entailment",
            "authors": [
                "Riko Suzuki",
                "Hitomi Yanaka",
                "Masashi Yoshikawa",
                "Koji Mineshima",
                "Daisuke Bekki"
            ],
            "abstract": "A large amount of research about multimodal inference across text and vision\nhas been recently developed to obtain visually grounded word and sentence\nrepresentations. In this paper, we use logic-based representations as unified\nmeaning representations for texts and images and present an unsupervised\nmultimodal logical inference system that can effectively prove entailment\nrelations between them. We show that by combining semantic parsing and theorem\nproving, the system can handle semantically complex sentences for\nvisual-textual inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04040v1",
            "date": "2019-06-10 14:48:10+00:00",
            "title": "The University of Helsinki submissions to the WMT19 news translation task",
            "authors": [
                "Aarne Talman",
                "Umut Sulubacak",
                "Ra\u00fal V\u00e1zquez",
                "Yves Scherrer",
                "Sami Virpioja",
                "Alessandro Raganato",
                "Arvi Hurskainen",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "In this paper, we present the University of Helsinki submissions to the WMT\n2019 shared task on news translation in three language pairs: English-German,\nEnglish-Finnish and Finnish-English. This year, we focused first on cleaning\nand filtering the training data using multiple data-filtering approaches,\nresulting in much smaller and cleaner training sets. For English-German, we\ntrained both sentence-level transformer models and compared different\ndocument-level translation approaches. For Finnish-English and English-Finnish\nwe focused on different segmentation approaches, and we also included a\nrule-based system for English-Finnish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04041v1",
            "date": "2019-06-10 14:48:54+00:00",
            "title": "CAiRE_HKUST at SemEval-2019 Task 3: Hierarchical Attention for Dialogue Emotion Classification",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Jamin Shin",
                "Yan Xu",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "Detecting emotion from dialogue is a challenge that has not yet been\nextensively surveyed. One could consider the emotion of each dialogue turn to\nbe independent, but in this paper, we introduce a hierarchical approach to\nclassify emotion, hypothesizing that the current emotional state depends on\nprevious latent emotions. We benchmark several feature-based classifiers using\npre-trained word and emotion embeddings, state-of-the-art end-to-end neural\nnetwork models, and Gaussian processes for automatic hyper-parameter search. In\nour experiments, hierarchical architectures consistently give significant\nimprovements, and our best model achieves a 76.77% F1-score on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04068v1",
            "date": "2019-06-10 15:20:32+00:00",
            "title": "Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations",
            "authors": [
                "Ethan Wilcox",
                "Roger Levy",
                "Richard Futrell"
            ],
            "abstract": "Deep learning sequence models have led to a marked increase in performance\nfor a range of Natural Language Processing tasks, but it remains an open\nquestion whether they are able to induce proper hierarchical generalizations\nfor representing natural language from linear input alone. Work using\nartificial languages as training input has shown that LSTMs are capable of\ninducing the stack-like data structures required to represent context-free and\ncertain mildly context-sensitive languages---formal language classes which\ncorrespond in theory to the hierarchical structures of natural language. Here\nwe present a suite of experiments probing whether neural language models\ntrained on linguistic data induce these stack-like data structures and deploy\nthem while incrementally predicting words. We study two natural language\nphenomena: center embedding sentences and syntactic island constraints on the\nfiller--gap dependency. In order to properly predict words in these structures,\na model must be able to temporarily suppress certain expectations and then\nrecover those expectations later, essentially pushing and popping these\nexpectations on a stack. Our results provide evidence that models can\nsuccessfully suppress and recover expectations in many cases, but do not fully\nrecover their previous grammatical state.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04082v1",
            "date": "2019-06-10 15:53:20+00:00",
            "title": "Char-RNN for Word Stress Detection in East Slavic Languages",
            "authors": [
                "Ekaterina Chernyak",
                "Maria Ponomareva",
                "Kirill Milintsevich"
            ],
            "abstract": "We explore how well a sequence labeling approach, namely, recurrent neural\nnetwork, is suited for the task of resource-poor and POS tagging free word\nstress detection in the Russian, Ukranian, Belarusian languages. We present new\ndatasets, annotated with the word stress, for the three languages and compare\nseveral RNN models trained on three languages and explore possible applications\nof the transfer learning for the task. We show that it is possible to train a\nmodel in a cross-lingual setting and that using additional languages improves\nthe quality of the results.",
            "journal ref": "2019, In Proceedings of the Sixth Workshop on NLP for Similar\n  Languages, Varieties and Dialects, pages 35-41,TOBEFILLED-Ann Arbor,\n  Michigan, Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04099v1",
            "date": "2019-06-10 16:20:48+00:00",
            "title": "AGRR-2019: A Corpus for Gapping Resolution in Russian",
            "authors": [
                "Maria Ponomareva",
                "Kira Droganova",
                "Ivan Smurov",
                "Tatiana Shavrina"
            ],
            "abstract": "This paper provides a comprehensive overview of the gapping dataset for\nRussian that consists of 7.5k sentences with gapping (as well as 15k relevant\nnegative sentences) and comprises data from various genres: news, fiction,\nsocial media and technical texts. The dataset was prepared for the Automatic\nGapping Resolution Shared Task for Russian (AGRR-2019) - a competition aimed at\nstimulating the development of NLP tools and methods for processing of\nellipsis.\n  In this paper, we pay special attention to the gapping resolution methods\nthat were introduced within the shared task as well as an alternative test set\nthat illustrates that our corpus is a diverse and representative subset of\nRussian language gapping sufficient for effective utilization of machine\nlearning techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04106v1",
            "date": "2019-06-10 16:27:34+00:00",
            "title": "Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards",
            "authors": [
                "Hou Pong Chan",
                "Wang Chen",
                "Lu Wang",
                "Irwin King"
            ],
            "abstract": "Generating keyphrases that summarize the main points of a document is a\nfundamental task in natural language processing. Although existing generative\nmodels are capable of predicting multiple keyphrases for an input document as\nwell as determining the number of keyphrases to generate, they still suffer\nfrom the problem of generating too few keyphrases. To address this problem, we\npropose a reinforcement learning (RL) approach for keyphrase generation, with\nan adaptive reward function that encourages a model to generate both sufficient\nand accurate keyphrases. Furthermore, we introduce a new evaluation method that\nincorporates name variations of the ground-truth keyphrases using the Wikipedia\nknowledge base. Thus, our evaluation method can more robustly evaluate the\nquality of predicted keyphrases. Extensive experiments on five real-world\ndatasets of different scales demonstrate that our RL approach consistently and\nsignificantly improves the performance of the state-of-the-art generative\nmodels with both conventional and new evaluation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04129v1",
            "date": "2019-06-10 17:08:29+00:00",
            "title": "Modeling Noisiness to Recognize Named Entities using Multitask Neural Networks on Social Media",
            "authors": [
                "Gustavo Aguilar",
                "A. Pastor L\u00f3pez-Monroy",
                "Fabio A. Gonz\u00e1lez",
                "Thamar Solorio"
            ],
            "abstract": "Recognizing named entities in a document is a key task in many NLP\napplications. Although current state-of-the-art approaches to this task reach a\nhigh performance on clean text (e.g. newswire genres), those algorithms\ndramatically degrade when they are moved to noisy environments such as social\nmedia domains. We present two systems that address the challenges of processing\nsocial media data using character-level phonetics and phonology, word\nembeddings, and Part-of-Speech tags as features. The first model is a multitask\nend-to-end Bidirectional Long Short-Term Memory (BLSTM)-Conditional Random\nField (CRF) network whose output layer contains two CRF classifiers. The second\nmodel uses a multitask BLSTM network as feature extractor that transfers the\nlearning to a CRF classifier for the final prediction. Our systems outperform\nthe current F1 scores of the state of the art on the Workshop on Noisy\nUser-generated Text 2017 dataset by 2.45% and 3.69%, establishing a more\nsuitable approach for social media environments.",
            "journal ref": "Proceedings of the 2018 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, Volume 1 (Long Papers), 1401-1412",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04135v1",
            "date": "2019-06-10 17:12:30+00:00",
            "title": "A Multi-task Approach for Named Entity Recognition in Social Media Data",
            "authors": [
                "Gustavo Aguilar",
                "Suraj Maharjan",
                "Adrian Pastor L\u00f3pez-Monroy",
                "Thamar Solorio"
            ],
            "abstract": "Named Entity Recognition for social media data is challenging because of its\ninherent noisiness. In addition to improper grammatical structures, it contains\nspelling inconsistencies and numerous informal abbreviations. We propose a\nnovel multi-task approach by employing a more general secondary task of Named\nEntity (NE) segmentation together with the primary task of fine-grained NE\ncategorization. The multi-task neural network architecture learns higher order\nfeature representations from word and character sequences along with basic\nPart-of-Speech tags and gazetteer information. This neural network acts as a\nfeature extractor to feed a Conditional Random Fields classifier. We were able\nto obtain the first position in the 3rd Workshop on Noisy User-generated Text\n(WNUT-2017) with a 41.86% entity F1-score and a 40.24% surface F1-score.",
            "journal ref": "Proceedings of the 3rd Workshop on Noisy User-generated Text,\n  2017, 148-153",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04138v1",
            "date": "2019-06-10 17:17:45+00:00",
            "title": "Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task",
            "authors": [
                "Gustavo Aguilar",
                "Fahad AlGhamdi",
                "Victor Soto",
                "Mona Diab",
                "Julia Hirschberg",
                "Thamar Solorio"
            ],
            "abstract": "In the third shared task of the Computational Approaches to Linguistic\nCode-Switching (CALCS) workshop, we focus on Named Entity Recognition (NER) on\ncode-switched social-media data. We divide the shared task into two\ncompetitions based on the English-Spanish (ENG-SPA) and Modern Standard\nArabic-Egyptian (MSA-EGY) language pairs. We use Twitter data and 9 entity\ntypes to establish a new dataset for code-switched NER benchmarks. In addition\nto the CS phenomenon, the diversity of the entities and the social media\nchallenges make the task considerably hard to process. As a result, the best\nscores of the competitions are 63.76% and 71.61% for ENG-SPA and MSA-EGY,\nrespectively. We present the scores of 9 participants and discuss the most\ncommon challenges among submissions.",
            "journal ref": "Proceedings of the Third Workshop on Computational Approaches to\n  Linguistic Code-Switching, 2018, 138-147",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04225v1",
            "date": "2019-06-10 18:53:18+00:00",
            "title": "Label-Agnostic Sequence Labeling by Copying Nearest Neighbors",
            "authors": [
                "Sam Wiseman",
                "Karl Stratos"
            ],
            "abstract": "Retrieve-and-edit based approaches to structured prediction, where structures\nassociated with retrieved neighbors are edited to form new structures, have\nrecently attracted increased interest. However, much recent work merely\nconditions on retrieved structures (e.g., in a sequence-to-sequence framework),\nrather than explicitly manipulating them. We show we can perform accurate\nsequence labeling by explicitly (and only) copying labels from retrieved\nneighbors. Moreover, because this copying is label-agnostic, we can achieve\nimpressive performance in zero-shot sequence-labeling tasks. We additionally\nconsider a dynamic programming approach to sequence labeling in the presence of\nretrieved neighbors, which allows for controlling the number of distinct\n(copied) segments used to form a prediction, and leads to both more\ninterpretable and accurate predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04229v1",
            "date": "2019-06-10 19:00:59+00:00",
            "title": "Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering",
            "authors": [
                "Claudio Greco",
                "Barbara Plank",
                "Raquel Fern\u00e1ndez",
                "Raffaella Bernardi"
            ],
            "abstract": "We study the issue of catastrophic forgetting in the context of neural\nmultimodal approaches to Visual Question Answering (VQA). Motivated by evidence\nfrom psycholinguistics, we devise a set of linguistically-informed VQA tasks,\nwhich differ by the types of questions involved (Wh-questions and polar\nquestions). We test what impact task difficulty has on continual learning, and\nwhether the order in which a child acquires question types facilitates\ncomputational models. Our results show that dramatic forgetting is at play and\nthat task difficulty and order matter. Two well-known current continual\nlearning methods mitigate the problem only to a limiting degree.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04287v1",
            "date": "2019-06-03 13:47:40+00:00",
            "title": "Chinese Embedding via Stroke and Glyph Information: A Dual-channel View",
            "authors": [
                "Hanqing Tao",
                "Shiwei Tong",
                "Tong Xu",
                "Qi Liu",
                "Enhong Chen"
            ],
            "abstract": "Recent studies have consistently given positive hints that morphology is\nhelpful in enriching word embeddings. In this paper, we argue that Chinese word\nembeddings can be substantially enriched by the morphological information\nhidden in characters which is reflected not only in strokes order sequentially,\nbut also in character glyphs spatially. Then, we propose a novel Dual-channel\nWord Embedding (DWE) model to realize the joint learning of sequential and\nspatial information of characters. Through the evaluation on both word\nsimilarity and word analogy tasks, our model shows its rationality and\nsuperiority in modelling the morphology of Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04291v1",
            "date": "2019-06-10 21:54:33+00:00",
            "title": "Automated Curriculum Learning for Turn-level Spoken Language Understanding with Weak Supervision",
            "authors": [
                "Hao Lang",
                "Wen Wang"
            ],
            "abstract": "We propose a learning approach for turn-level spoken language understanding,\nwhich facilitates a user to speak one or more utterances compositionally in a\nturn for completing a task (e.g., voice ordering). A typical pipelined approach\nfor these understanding tasks requires non-trivial annotation effort for\ndeveloping its multiple components. Also, the pipeline is difficult to port to\na new domain or scale up. To address these problems, we propose an end-to-end\nstatistical model with weak supervision. We employ randomized beam search with\nmemory augmentation (RBSMA) to solve complicated problems for which long\npromising trajectories are usually difficult to explore. Furthermore,\nconsidering the diversity of problem complexity, we explore automated\ncurriculum learning (CL) for weak supervision to accelerate exploration and\nlearning. We evaluate the proposed approach on real-world user logs of a\ncommercial voice ordering system. Results demonstrate that when trained on a\nsmall number of end-to-end annotated sessions collected with low cost, our\nmodel performs comparably to the deployed pipelined system, saving the\ndevelopment labor over an order of magnitude. The RBSMA algorithm improves the\ntest set accuracy by 7.8% relative compared to the standard beam search.\nAutomated CL leads to better generalization and further improves the test set\naccuracy by 5% relative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04341v1",
            "date": "2019-06-11 01:31:41+00:00",
            "title": "What Does BERT Look At? An Analysis of BERT's Attention",
            "authors": [
                "Kevin Clark",
                "Urvashi Khandelwal",
                "Omer Levy",
                "Christopher D. Manning"
            ],
            "abstract": "Large pre-trained neural networks such as BERT have had great recent success\nin NLP, motivating a growing body of research investigating what aspects of\nlanguage they are able to learn from unlabeled data. Most recent analysis has\nfocused on model outputs (e.g., language model surprisal) or internal vector\nrepresentations (e.g., probing classifiers). Complementary to these works, we\npropose methods for analyzing the attention mechanisms of pre-trained models\nand apply them to BERT. BERT's attention heads exhibit patterns such as\nattending to delimiter tokens, specific positional offsets, or broadly\nattending over the whole sentence, with heads in the same layer often\nexhibiting similar behaviors. We further show that certain attention heads\ncorrespond well to linguistic notions of syntax and coreference. For example,\nwe find heads that attend to the direct objects of verbs, determiners of nouns,\nobjects of prepositions, and coreferent mentions with remarkably high accuracy.\nLastly, we propose an attention-based probing classifier and use it to further\ndemonstrate that substantial syntactic information is captured in BERT's\nattention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04362v1",
            "date": "2019-06-11 03:00:34+00:00",
            "title": "A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Xueliang Zhao",
                "Chongyang Tao",
                "Wei Wu",
                "Can Xu",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "We present a document-grounded matching network (DGMN) for response selection\nthat can power a knowledge-aware retrieval-based chatbot system. The challenges\nof building such a model lie in how to ground conversation contexts with\nbackground documents and how to recognize important information in the\ndocuments for matching. To overcome the challenges, DGMN fuses information in a\ndocument and a context into representations of each other, and dynamically\ndetermines if grounding is necessary and importance of different parts of the\ndocument and the context through hierarchical interaction with a response at\nthe matching step. Empirical studies on two public data sets indicate that DGMN\ncan significantly improve upon state-of-the-art methods and at the same time\nenjoys good interpretability.",
            "journal ref": "IJCAI 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04382v1",
            "date": "2019-06-11 04:07:37+00:00",
            "title": "DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain",
            "authors": [
                "Yichong Xu",
                "Xiaodong Liu",
                "Chunyuan Li",
                "Hoifung Poon",
                "Jianfeng Gao"
            ],
            "abstract": "This paper describes our competing system to enter the MEDIQA-2019\ncompetition. We use a multi-source transfer learning approach to transfer the\nknowledge from MT-DNN and SciBERT to natural language understanding tasks in\nthe medical domain. For transfer learning fine-tuning, we use multi-task\nlearning on NLI, RQE and QA tasks on general and medical domains to improve\nperformance. The proposed methods are proved effective for natural language\nunderstanding in the medical domain, and we rank the first place on the QA\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04413v1",
            "date": "2019-06-11 06:55:04+00:00",
            "title": "Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems",
            "authors": [
                "Jiazhan Feng",
                "Chongyang Tao",
                "Wei Wu",
                "Yansong Feng",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "We study learning of a matching model for response selection in\nretrieval-based dialogue systems. The problem is equally important with\ndesigning the architecture of a model, but is less explored in existing\nliterature. To learn a robust matching model from noisy training data, we\npropose a general co-teaching framework with three specific teaching strategies\nthat cover both teaching with loss functions and teaching with data curriculum.\nUnder the framework, we simultaneously learn two matching models with\nindependent training sets. In each iteration, one model transfers the knowledge\nlearned from its training set to the other model, and at the same time receives\nthe guide from the other model on how to overcome noise in training. Through\nbeing both a teacher and a student, the two models learn from each other and\nget improved together. Evaluation results on two public data sets indicate that\nthe proposed learning approach can generally and significantly improve the\nperformance of existing matching models.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04466v1",
            "date": "2019-06-11 09:53:17+00:00",
            "title": "Self-Supervised Learning for Contextualized Extractive Summarization",
            "authors": [
                "Hong Wang",
                "Xin Wang",
                "Wenhan Xiong",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Shiyu Chang",
                "William Yang Wang"
            ],
            "abstract": "Existing models for extractive summarization are usually trained from scratch\nwith a cross-entropy loss, which does not explicitly capture the global context\nat the document level. In this paper, we aim to improve this task by\nintroducing three auxiliary pre-training tasks that learn to capture the\ndocument-level context in a self-supervised fashion. Experiments on the\nwidely-used CNN/DM dataset validate the effectiveness of the proposed auxiliary\ntasks. Furthermore, we show that after pre-training, a clean model with simple\nbuilding blocks is able to outperform previous state-of-the-art that are\ncarefully designed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04571v3",
            "date": "2019-06-11 13:22:24+00:00",
            "title": "Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology",
            "authors": [
                "Ran Zmigrod",
                "Sabrina J. Mielke",
                "Hanna Wallach",
                "Ryan Cotterell"
            ],
            "abstract": "Gender stereotypes are manifest in most of the world's languages and are\nconsequently propagated or amplified by NLP systems. Although research has\nfocused on mitigating gender stereotypes in English, the approaches that are\ncommonly employed produce ungrammatical sentences in morphologically rich\nlanguages. We present a novel approach for converting between\nmasculine-inflected and feminine-inflected sentences in such languages. For\nSpanish and Hebrew, our approach achieves F1 scores of 82% and 73% at the level\nof tags and accuracies of 90% and 87% at the level of forms. By evaluating our\napproach using four different languages, we show that, on average, it reduces\ngender stereotyping by a factor of 2.5 without any sacrifice to grammaticality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04618v1",
            "date": "2019-06-11 14:18:43+00:00",
            "title": "Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension",
            "authors": [
                "Minghao Hu",
                "Yuxing Peng",
                "Zhen Huang",
                "Dongsheng Li"
            ],
            "abstract": "This paper considers the reading comprehension task in which multiple\ndocuments are given as input. Prior work has shown that a pipeline of\nretriever, reader, and reranker can improve the overall performance. However,\nthe pipeline system is inefficient since the input is re-encoded within each\nmodule, and is unable to leverage upstream components to help downstream\ntraining. In this work, we present RE$^3$QA, a unified question answering model\nthat combines context retrieving, reading comprehension, and answer reranking\nto predict the final answer. Unlike previous pipelined approaches, RE$^3$QA\nshares contextualized text representation across different components, and is\ncarefully designed to use high-quality upstream outputs (e.g., retrieved\ncontext or candidate answers) for directly supervising downstream modules\n(e.g., the reader or the reranker). As a result, the whole network can be\ntrained end-to-end to avoid the context inconsistency problem. Experiments show\nthat our model outperforms the pipelined baseline and achieves state-of-the-art\nresults on two versions of TriviaQA and two variants of SQuAD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04655v1",
            "date": "2019-06-11 15:35:43+00:00",
            "title": "Journal Name Extraction from Japanese Scientific News Articles",
            "authors": [
                "Masato Kikuchi",
                "Mitsuo Yoshida",
                "Kyoji Umemura"
            ],
            "abstract": "In Japanese scientific news articles, although the research results are\ndescribed clearly, the article's sources tend to be uncited. This makes it\ndifficult for readers to know the details of the research. In this paper, we\naddress the task of extracting journal names from Japanese scientific news\narticles. We hypothesize that a journal name is likely to occur in a specific\ncontext. To support the hypothesis, we construct a character-based method and\nextract journal names using this method. This method only uses the left and\nright context features of journal names. The results of the journal name\nextractions suggest that the distribution hypothesis plays an important role in\nidentifying the journal names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04687v1",
            "date": "2019-06-11 16:39:11+00:00",
            "title": "Generating Summaries with Topic Templates and Structured Convolutional Decoders",
            "authors": [
                "Laura Perez-Beltrachini",
                "Yang Liu",
                "Mirella Lapata"
            ],
            "abstract": "Existing neural generation approaches create multi-sentence text as a single\nsequence. In this paper we propose a structured convolutional decoder that is\nguided by the content structure of target summaries. We compare our model with\nexisting sequential decoders on three data sets representing different domains.\nAutomatic and human evaluation demonstrate that our summaries have better\ncontent coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04701v1",
            "date": "2019-06-11 17:06:49+00:00",
            "title": "HEAD-QA: A Healthcare Dataset for Complex Reasoning",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present HEAD-QA, a multi-choice question answering testbed to encourage\nresearch on complex reasoning. The questions come from exams to access a\nspecialized position in the Spanish healthcare system, and are challenging even\nfor highly specialized humans. We then consider monolingual (Spanish) and\ncross-lingual (to English) experiments with information retrieval and neural\ntechniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the\nresults lag well behind human performance, demonstrating its usefulness as a\nbenchmark for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04726v2",
            "date": "2019-06-11 17:56:08+00:00",
            "title": "What Kind of Language Is Hard to Language-Model?",
            "authors": [
                "Sabrina J. Mielke",
                "Ryan Cotterell",
                "Kyle Gorman",
                "Brian Roark",
                "Jason Eisner"
            ],
            "abstract": "How language-agnostic are current state-of-the-art NLP tools? Are there some\ntypes of language that are easier to model with current methods? In prior work\n(Cotterell et al., 2018) we attempted to address this question for language\nmodeling, and observed that recurrent neural network language models do not\nperform equally well over all the high-resource European languages found in the\nEuroparl corpus. We speculated that inflectional morphology may be the primary\nculprit for the discrepancy. In this paper, we extend these earlier experiments\nto cover 69 languages from 13 language families using a multilingual Bible\ncorpus. Methodologically, we introduce a new paired-sample multiplicative\nmixed-effects model to obtain language difficulty coefficients from\nat-least-pairwise parallel corpora. In other words, the model is aware of\ninter-sentence variation and can handle missing data. Exploiting this model, we\nshow that \"translationese\" is not any easier to model than natively written\nlanguage in a fair comparison. Trying to answer the question of what features\ndifficult languages have in common, we try and fail to reproduce our earlier\n(Cotterell et al., 2018) observation about morphological complexity and instead\nreveal far simpler statistics of the data that seem to drive complexity in a\nmuch larger sample.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04760v1",
            "date": "2019-06-11 18:18:29+00:00",
            "title": "Unsupervised Discovery of Gendered Language through Latent-Variable Modeling",
            "authors": [
                "Alexander Hoyle",
                "Wolf-Sonkin",
                "Hanna Wallach",
                "Isabelle Augenstein",
                "Ryan Cotterell"
            ],
            "abstract": "Studying the ways in which language is gendered has long been an area of\ninterest in sociolinguistics. Studies have explored, for example, the speech of\nmale and female characters in film and the language used to describe male and\nfemale politicians. In this paper, we aim not to merely study this phenomenon\nqualitatively, but instead to quantify the degree to which the language used to\ndescribe men and women is different and, moreover, different in a positive or\nnegative way. To that end, we introduce a generative latent-variable model that\njointly represents adjective (or verb) choice, with its sentiment, given the\nnatural gender of a head (or dependent) noun. We find that there are\nsignificant differences between descriptions of male and female nouns and that\nthese differences align with common gender stereotypes: Positive adjectives\nused to describe women are more often related to their bodies than adjectives\nused to describe men.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04761v1",
            "date": "2019-06-11 18:18:39+00:00",
            "title": "PerspectroScope: A Window to the World of Diverse Perspectives",
            "authors": [
                "Sihao Chen",
                "Daniel Khashabi",
                "Chris Callison-Burch",
                "Dan Roth"
            ],
            "abstract": "This work presents PerspectroScope, a web-based system which lets users query\na discussion-worthy natural language claim, and extract and visualize various\nperspectives in support or against the claim, along with evidence supporting\neach perspective. The system thus lets users explore various perspectives that\ncould touch upon aspects of the issue at hand.The system is built as a\ncombination of retrieval engines and learned textual-entailment-like\nclassifiers built using a few recent developments in natural language\nunderstanding. To make the system more adaptive, expand its coverage, and\nimprove its decisions over time, our platform employs various mechanisms to get\ncorrections from the users.\n  PerspectroScope is available at github.com/CogComp/perspectroscope.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04772v1",
            "date": "2019-06-11 19:03:09+00:00",
            "title": "A Systematic Comparison of English Noun Compound Representations",
            "authors": [
                "Vered Shwartz"
            ],
            "abstract": "Building meaningful representations of noun compounds is not trivial since\nmany of them scarcely appear in the corpus. To that end, composition functions\napproximate the distributional representation of a noun compound by combining\nits constituent distributional vectors. In the more general case, phrase\nembeddings have been trained by minimizing the distance between the vectors\nrepresenting paraphrases. We compare various types of noun compound\nrepresentations, including distributional, compositional, and paraphrase-based\nrepresentations, through a series of tasks and analyses, and with an extensive\nnumber of underlying word embeddings. We find that indeed, in most cases,\ncomposition functions produce higher quality representations than\ndistributional ones, and they improve with computational power. No single\nfunction performs best in all scenarios, suggesting that a joint training\nobjective may produce improved representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04836v1",
            "date": "2019-06-11 21:37:51+00:00",
            "title": "Unmasking Bias in News",
            "authors": [
                "Javier S\u00e1nchez-Junquera",
                "Paolo Rosso",
                "Manuel Montes-y-G\u00f3mez",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "We present experiments on detecting hyperpartisanship in news using a\n'masking' method that allows us to assess the role of style vs. content for the\ntask at hand. Our results corroborate previous research on this task in that\ntopic related features yield better results than stylistic ones. We\nadditionally show that competitive results can be achieved by simply including\nhigher-length n-grams, which suggests the need to develop more challenging\ndatasets and tasks that address implicit and more subtle forms of bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04960v2",
            "date": "2019-06-12 06:30:01+00:00",
            "title": "Towards Geocoding Spatial Expressions",
            "authors": [
                "Hussein S. Al-Olimat",
                "Valerie L. Shalin",
                "Krishnaprasad Thirunarayan",
                "Joy Prakash Sain"
            ],
            "abstract": "Imprecise composite location references formed using ad hoc spatial\nexpressions in English text makes the geocoding task challenging for both\ninference and evaluation. Typically such spatial expressions fill in\nunestablished areas with new toponyms for finer spatial referents. For example,\nthe spatial extent of the ad hoc spatial expression \"north of\" or \"50 minutes\naway from\" in relation to the toponym \"Dayton, OH\" refers to an ambiguous,\nimprecise area, requiring translation from this qualitative representation to a\nquantitative one with precise semantics using systems such as WGS84. Here we\nhighlight the challenges of geocoding such referents and propose a formal\nrepresentation that employs background knowledge, semantic approximations and\nrules, and fuzzy linguistic variables. We also discuss an appropriate\nevaluation technique for the task that is based on human contextualized and\nsubjective judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04991v1",
            "date": "2019-06-12 08:08:38+00:00",
            "title": "Incremental Learning from Scratch for Task-Oriented Dialogue Systems",
            "authors": [
                "Weikang Wang",
                "Jiajun Zhang",
                "Qian Li",
                "Mei-Yuh Hwang",
                "Chengqing Zong",
                "Zhifei Li"
            ],
            "abstract": "Clarifying user needs is essential for existing task-oriented dialogue\nsystems. However, in real-world applications, developers can never guarantee\nthat all possible user demands are taken into account in the design phase.\nConsequently, existing systems will break down when encountering unconsidered\nuser needs. To address this problem, we propose a novel incremental learning\nframework to design task-oriented dialogue systems, or for short Incremental\nDialogue System (IDS), without pre-defining the exhaustive list of user needs.\nSpecifically, we introduce an uncertainty estimation module to evaluate the\nconfidence of giving correct responses. If there is high confidence, IDS will\nprovide responses to users. Otherwise, humans will be involved in the dialogue\nprocess, and IDS can learn from human intervention through an online learning\nmodule. To evaluate our method, we propose a new dataset which simulates\nunanticipated user needs in the deployment stage. Experiments show that IDS is\nrobust to unconsidered user actions, and can update itself online by smartly\nselecting only the most effective training data, and hence attains better\nperformance with less annotation cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05000v1",
            "date": "2019-06-12 08:29:24+00:00",
            "title": "Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records",
            "authors": [
                "Max Friedrich",
                "Arne K\u00f6hn",
                "Gregor Wiedemann",
                "Chris Biemann"
            ],
            "abstract": "De-identification is the task of detecting protected health information (PHI)\nin medical text. It is a critical step in sanitizing electronic health records\n(EHRs) to be shared for research. Automatic de-identification classifierscan\nsignificantly speed up the sanitization process. However, obtaining a large and\ndiverse dataset to train such a classifier that works wellacross many types of\nmedical text poses a challenge as privacy laws prohibit the sharing of raw\nmedical records. We introduce a method to create privacy-preserving shareable\nrepresentations of medical text (i.e. they contain no PHI) that does not\nrequire expensive manual pseudonymization. These representations can be shared\nbetween organizations to create unified datasets for training de-identification\nmodels. Our representation allows training a simple LSTM-CRF de-identification\nmodel to an F1 score of 97.4%, which is comparable to a strong baseline that\nexposes private information in its representation. A robust, widely available\nde-identification classifier based on our representation could potentially\nenable studies for which de-identification would otherwise be too costly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05039v1",
            "date": "2019-06-12 09:55:20+00:00",
            "title": "Concept Discovery through Information Extraction in Restaurant Domain",
            "authors": [
                "Nadeesha Pathirana",
                "Sandaru Seneviratne",
                "Rangika Samarawickrama",
                "Shane Wolff",
                "Charith Chitraranjan",
                "Uthayasanker Thayasivam",
                "Tharindu Ranasinghe"
            ],
            "abstract": "Concept identification is a crucial step in understanding and building a\nknowledge base for any particular domain. However, it is not a simple task in\nvery large domains such as restaurants and hotel. In this paper, a novel\napproach of identifying a concept hierarchy and classifying unseen words into\nidentified concepts related to restaurant domain is presented. Sorting,\nidentifying, classifying of domain-related words manually is tedious and\ntherefore, the proposed process is automated to a great extent. Word embedding,\nhierarchical clustering, classification algorithms are effectively used to\nobtain concepts related to the restaurant domain. Further, this approach can\nalso be extended to create a semi-automatic ontology on restaurant domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05061v1",
            "date": "2019-06-12 11:21:05+00:00",
            "title": "Probing Multilingual Sentence Representations With X-Probe",
            "authors": [
                "Vinit Ravishankar",
                "Lilja \u00d8vrelid",
                "Erik Velldal"
            ],
            "abstract": "This paper extends the task of probing sentence representations for\nlinguistic insight in a multilingual domain. In doing so, we make two\ncontributions: first, we provide datasets for multilingual probing, derived\nfrom Wikipedia, in five languages, viz. English, French, German, Spanish and\nRussian. Second, we evaluate six sentence encoders for each language, each\ntrained by mapping sentence representations to English sentence\nrepresentations, using sentences in a parallel corpus. We discover that\ncross-lingually mapped representations are often better at retaining certain\nlinguistic information than representations derived from English encoders\ntrained on natural language inference (NLI) as a downstream task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05149v1",
            "date": "2019-06-12 14:07:40+00:00",
            "title": "Putting words in context: LSTM language models and lexical ambiguity",
            "authors": [
                "Laura Aina",
                "Kristina Gulordava",
                "Gemma Boleda"
            ],
            "abstract": "In neural network models of language, words are commonly represented using\ncontext-invariant representations (word embeddings) which are then put in\ncontext in the hidden layers. Since words are often ambiguous, representing the\ncontextually relevant information is not trivial. We investigate how an LSTM\nlanguage model deals with lexical ambiguity in English, designing a method to\nprobe its hidden representations for lexical and contextual information about\nwords. We find that both types of information are represented to a large\nextent, but also that there is room for improvement for contextual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05218v1",
            "date": "2019-06-12 15:49:31+00:00",
            "title": "Monotonic Infinite Lookback Attention for Simultaneous Machine Translation",
            "authors": [
                "Naveen Arivazhagan",
                "Colin Cherry",
                "Wolfgang Macherey",
                "Chung-Cheng Chiu",
                "Semih Yavuz",
                "Ruoming Pang",
                "Wei Li",
                "Colin Raffel"
            ],
            "abstract": "Simultaneous machine translation begins to translate each source sentence\nbefore the source speaker is finished speaking, with applications to live and\nstreaming scenarios. Simultaneous systems must carefully schedule their reading\nof the source sentence to balance quality against latency. We present the first\nsimultaneous translation system to learn an adaptive schedule jointly with a\nneural machine translation (NMT) model that attends over all source tokens read\nthus far. We do so by introducing Monotonic Infinite Lookback (MILk) attention,\nwhich maintains both a hard, monotonic attention head to schedule the reading\nof the source sentence, and a soft attention head that extends from the\nmonotonic head back to the beginning of the source. We show that MILk's\nadaptive schedule allows it to arrive at latency-quality trade-offs that are\nfavorable to those of a recently proposed wait-k strategy for many latency\nvalues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05416v1",
            "date": "2019-06-12 22:50:31+00:00",
            "title": "Synthetic QA Corpora Generation with Roundtrip Consistency",
            "authors": [
                "Chris Alberti",
                "Daniel Andor",
                "Emily Pitler",
                "Jacob Devlin",
                "Michael Collins"
            ],
            "abstract": "We introduce a novel method of generating synthetic question answering\ncorpora by combining models of question generation and answer extraction, and\nby filtering the results to ensure roundtrip consistency. By pretraining on the\nresulting corpora we obtain significant improvements on SQuAD2 and NQ,\nestablishing a new state-of-the-art on the latter. Our synthetic data\ngeneration models, for both question generation and answer extraction, can be\nfully reproduced by finetuning a publicly available BERT model on the\nextractive subsets of SQuAD2 and NQ. We also describe a more powerful variant\nthat does full sequence-to-sequence pretraining for question generation,\nobtaining exact match and F1 at less than 0.1% and 0.4% from human performance\non SQuAD2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05447v1",
            "date": "2019-06-11 17:49:57+00:00",
            "title": "Cued@wmt19:ewc&lms",
            "authors": [
                "Felix Stahlberg",
                "Danielle Saunders",
                "Adria de Gispert",
                "Bill Byrne"
            ],
            "abstract": "Two techniques provide the fabric of the Cambridge University Engineering\nDepartment's (CUED) entry to the WMT19 evaluation campaign: elastic weight\nconsolidation (EWC) and different forms of language modelling (LMs). We report\nsubstantial gains by fine-tuning very strong baselines on former WMT test sets\nusing a combination of checkpoint averaging and EWC. A sentence-level\nTransformer LM and a document-level LM based on a modified Transformer\narchitecture yield further gains. As in previous years, we also extract\n$n$-gram probabilities from SMT lattices which can be seen as a\nsource-conditioned $n$-gram LM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05474v2",
            "date": "2019-06-13 04:07:12+00:00",
            "title": "Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets",
            "authors": [
                "Yifan Peng",
                "Shankai Yan",
                "Zhiyong Lu"
            ],
            "abstract": "Inspired by the success of the General Language Understanding Evaluation\nbenchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE)\nbenchmark to facilitate research in the development of pre-training language\nrepresentations in the biomedicine domain. The benchmark consists of five tasks\nwith ten datasets that cover both biomedical and clinical texts with different\ndataset sizes and difficulties. We also evaluate several baselines based on\nBERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and\nMIMIC-III clinical notes achieves the best results. We make the datasets,\npre-trained models, and codes publicly available at\nhttps://github.com/ncbi-nlp/BLUE_Benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05491v1",
            "date": "2019-06-13 05:44:16+00:00",
            "title": "A Computational Analysis of Natural Languages to Build a Sentence Structure Aware Artificial Neural Network",
            "authors": [
                "Alberto Calderone"
            ],
            "abstract": "Natural languages are complexly structured entities. They exhibit\ncharacterising regularities that can be exploited to link them one another. In\nthis work, I compare two morphological aspects of languages: Written Patterns\nand Sentence Structure. I show how languages spontaneously group by similarity\nin both analyses and derive an average language distance. Finally, exploiting\nSentence Structure I developed an Artificial Neural Network capable of\ndistinguishing languages suggesting that not only word roots but also\ngrammatical sentence structure is a characterising trait which alone suffice to\nidentify them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05506v1",
            "date": "2019-06-13 06:52:45+00:00",
            "title": "Character n-gram Embeddings to Improve RNN Language Models",
            "authors": [
                "Sho Takase",
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "This paper proposes a novel Recurrent Neural Network (RNN) language model\nthat takes advantage of character information. We focus on character n-grams\nbased on research in the field of word embedding construction (Wieting et al.\n2016). Our proposed method constructs word embeddings from character n-gram\nembeddings and combines them with ordinary word embeddings. We demonstrate that\nthe proposed method achieves the best perplexities on the language modeling\ndatasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct\nexperiments on application tasks: machine translation and headline generation.\nThe experimental results indicate that our proposed method also positively\naffects these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05518v1",
            "date": "2019-06-13 07:35:05+00:00",
            "title": "Know What You Don't Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories",
            "authors": [
                "Sina Zarrie\u00df",
                "David Schlangen"
            ],
            "abstract": "Zero-shot learning in Language & Vision is the task of correctly labelling\n(or naming) objects of novel categories. Another strand of work in L&V aims at\npragmatically informative rather than ``correct'' object descriptions, e.g. in\nreference games. We combine these lines of research and model zero-shot\nreference games, where a speaker needs to successfully refer to a novel object\nin an image. Inspired by models of \"rational speech acts\", we extend a neural\ngenerator to become a pragmatic speaker reasoning about uncertain object\ncategories. As a result of this reasoning, the generator produces fewer nouns\nand names of distractor categories as compared to a literal speaker. We show\nthat this conversational strategy for dealing with novel objects often improves\ncommunicative success, in terms of resolution accuracy of an automatic\nlistener.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05551v1",
            "date": "2019-06-13 08:55:06+00:00",
            "title": "Lattice Transformer for Speech Translation",
            "authors": [
                "Pei Zhang",
                "Boxing Chen",
                "Niyu Ge",
                "Kai Fan"
            ],
            "abstract": "Recent advances in sequence modeling have highlighted the strengths of the\ntransformer architecture, especially in achieving state-of-the-art machine\ntranslation results. However, depending on the up-stream systems, e.g., speech\nrecognition, or word segmentation, the input to translation system can vary\ngreatly. The goal of this work is to extend the attention mechanism of the\ntransformer to naturally consume the lattice in addition to the traditional\nsequential input. We first propose a general lattice transformer for speech\ntranslation where the input is the output of the automatic speech recognition\n(ASR) which contains multiple paths and posterior scores. To leverage the extra\ninformation from the lattice structure, we develop a novel controllable lattice\nattention mechanism to obtain latent representations. On the LDC\nSpanish-English speech translation corpus, our experiments show that lattice\ntransformer generalizes significantly better and outperforms both a transformer\nbaseline and a lattice LSTM. Additionally, we validate our approach on the WMT\n2017 Chinese-English translation task with lattice inputs from different BPE\nsegmentations. In this task, we also observe the improvements over strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05572v2",
            "date": "2019-06-13 09:42:51+00:00",
            "title": "Proactive Human-Machine Conversation with Explicit Conversation Goals",
            "authors": [
                "Wenquan Wu",
                "Zhen Guo",
                "Xiangyang Zhou",
                "Hua Wu",
                "Xiyuan Zhang",
                "Rongzhong Lian",
                "Haifeng Wang"
            ],
            "abstract": "Though great progress has been made for human-machine conversation, current\ndialogue system is still in its infancy: it usually converses passively and\nutters words more as a matter of response, rather than on its own initiatives.\nIn this paper, we take a radical step towards building a human-like\nconversational agent: endowing it with the ability of proactively leading the\nconversation (introducing a new topic or maintaining the current topic). To\nfacilitate the development of such conversation systems, we create a new\ndataset named DuConv where one acts as a conversation leader and the other acts\nas the follower. The leader is provided with a knowledge graph and asked to\nsequentially change the discussion topics, following the given conversation\ngoal, and meanwhile keep the dialogue as natural and engaging as possible.\nDuConv enables a very challenging task as the model needs to both understand\ndialogue and plan over the given knowledge graph. We establish baseline results\non this dataset (about 270K utterances and 30k dialogues) using several\nstate-of-the-art models. Experimental results show that dialogue models that\nplan over the knowledge graph can make full use of related knowledge to\ngenerate more diverse multi-turn conversations. The baseline systems along with\nthe dataset are publicly available",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05612v1",
            "date": "2019-06-13 11:46:12+00:00",
            "title": "Antonym-Synonym Classification Based on New Sub-space Embeddings",
            "authors": [
                "Muhammad Asif Ali",
                "Yifang Sun",
                "Xiaoling Zhou",
                "Wei Wang",
                "Xiang Zhao"
            ],
            "abstract": "Distinguishing antonyms from synonyms is a key challenge for many NLP\napplications focused on the lexical-semantic relation extraction. Existing\nsolutions relying on large-scale corpora yield low performance because of huge\ncontextual overlap of antonym and synonym pairs. We propose a novel approach\nentirely based on pre-trained embeddings. We hypothesize that the pre-trained\nembeddings comprehend a blend of lexical-semantic information and we may\ndistill the task-specific information using Distiller, a model proposed in this\npaper. Later, a classifier is trained based on features constructed from the\ndistilled sub-spaces along with some word level features to distinguish\nantonyms from synonyms. Experimental results show that the proposed model\noutperforms existing research on antonym synonym distinction in both speed and\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05667v2",
            "date": "2019-06-11 06:59:56+00:00",
            "title": "Generating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding",
            "authors": [
                "Junyi Li",
                "Wayne Xin Zhao",
                "Ji-Rong Wen",
                "Yang Song"
            ],
            "abstract": "Generating long and informative review text is a challenging natural language\ngeneration task. Previous work focuses on word-level generation, neglecting the\nimportance of topical and syntactic characteristics from natural languages. In\nthis paper, we propose a novel review generation model by characterizing an\nelaborately designed aspect-aware coarse-to-fine generation process. First, we\nmodel the aspect transitions to capture the overall content flow. Then, to\ngenerate a sentence, an aspect-aware sketch will be predicted using an\naspect-aware decoder. Finally, another decoder fills in the semantic slots by\ngenerating corresponding words. Our approach is able to jointly utilize aspect\nsemantics, syntactic sketch, and context information. Extensive experiments\nresults have demonstrated the effectiveness of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05683v1",
            "date": "2019-06-11 17:56:29+00:00",
            "title": "Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation",
            "authors": [
                "Nima Pourdamghani",
                "Nada Aldarrab",
                "Marjan Ghazvininejad",
                "Kevin Knight",
                "Jonathan May"
            ],
            "abstract": "Given a rough, word-by-word gloss of a source language sentence, target\nlanguage natives can uncover the latent, fully-fluent rendering of the\ntranslation. In this work we explore this intuition by breaking translation\ninto a two step process: generating a rough gloss by means of a dictionary and\nthen `translating' the resulting pseudo-translation, or `Translationese' into a\nfully fluent translation. We build our Translationese decoder once from a\nmish-mash of parallel data that has the target language in common and then can\nbuild dictionaries on demand using unsupervised techniques, resulting in\nrapidly generated unsupervised neural MT systems for many source languages. We\napply this process to 14 test languages, obtaining better or comparable\ntranslation results on high-resource languages than previously published\nunsupervised MT studies, and obtaining good quality results for low-resource\nlanguages that have never been used in an unsupervised MT scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05725v1",
            "date": "2019-06-13 14:41:00+00:00",
            "title": "Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text",
            "authors": [
                "Bidisha Samanta",
                "Niloy Ganguly",
                "Soumen Chakrabarti"
            ],
            "abstract": "Multilingual writers and speakers often alternate between two languages in a\nsingle discourse, a practice called \"code-switching\". Existing sentiment\ndetection methods are usually trained on sentiment-labeled monolingual text.\nManually labeled code-switched text, especially involving minority languages,\nis extremely rare. Consequently, the best monolingual methods perform\nrelatively poorly on code-switched text. We present an effective technique for\nsynthesizing labeled code-switched text from labeled monolingual text, which is\nmore readily available. The idea is to replace carefully selected subtrees of\nconstituency parses of sentences in the resource-rich language with suitable\ntoken spans selected from automatic translations to the resource-poor language.\nBy augmenting scarce human-labeled code-switched text with plentiful synthetic\ncode-switched text, we achieve significant improvements in sentiment labeling\naccuracy (1.5%, 5.11%, 7.20%) for three different language pairs\n(English-Hindi, English-Spanish and English-Bengali). We also get significant\ngains for hate speech detection: 4% improvement using only synthetic text and\n6% if augmented with real text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05765v2",
            "date": "2019-06-13 15:49:56+00:00",
            "title": "Anti dependency distance minimization in short sequences. A graph theoretic approach",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Dependency distance minimization (DDm) is a word order principle favouring\nthe placement of syntactically related words close to each other in sentences.\nMassive evidence of the principle has been reported for more than a decade with\nthe help of syntactic dependency treebanks where long sentences abound.\nHowever, it has been predicted theoretically that the principle is more likely\nto be beaten in short sequences by the principle of surprisal minimization\n(predictability maximization). Here we introduce a simple binomial test to\nverify such a hypothesis. In short sentences, we find anti-DDm for some\nlanguages from different families. Our analysis of the syntactic dependency\nstructures suggests that anti-DDm is produced by star trees.",
            "journal ref": "Ferrer-i-Cancho, R. & G\\'omez-Rodr\\'iguez (2021). Anti dependency\n  distance minimization in short sequences. A graph theoretic approach. Journal\n  of Quantitative Linguistics 28 (1), 50-76",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05786v1",
            "date": "2019-06-13 16:22:44+00:00",
            "title": "UCAM Biomedical translation at WMT19: Transfer learning multi-domain ensembles",
            "authors": [
                "Danielle Saunders",
                "Felix Stahlberg",
                "Bill Byrne"
            ],
            "abstract": "The 2019 WMT Biomedical translation task involved translating Medline\nabstracts. We approached this using transfer learning to obtain a series of\nstrong neural models on distinct domains, and combining them into multi-domain\nensembles. We further experiment with an adaptive language-model ensemble\nweighting scheme. Our submission achieved the best submitted results on both\ndirections of English-Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05807v2",
            "date": "2019-06-13 16:49:35+00:00",
            "title": "Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index",
            "authors": [
                "Minjoon Seo",
                "Jinhyuk Lee",
                "Tom Kwiatkowski",
                "Ankur P. Parikh",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Existing open-domain question answering (QA) models are not suitable for\nreal-time usage because they need to process several long documents on-demand\nfor every input query. In this paper, we introduce the query-agnostic indexable\nrepresentation of document phrases that can drastically speed up open-domain QA\nand also allows us to reach long-tail targets. In particular, our dense-sparse\nphrase encoding effectively captures syntactic, semantic, and lexical\ninformation of the phrases and eliminates the pipeline filtering of context\ndocuments. Leveraging optimization strategies, our model can be trained in a\nsingle 4-GPU server and serve entire Wikipedia (up to 60 billion phrases) under\n2TB with CPUs only. Our experiments on SQuAD-Open show that our model is more\naccurate than DrQA (Chen et al., 2017) with 6000x reduced computational cost,\nwhich translates into at least 58x faster end-to-end inference benchmark on\nCPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05887v1",
            "date": "2019-06-13 18:35:00+00:00",
            "title": "Sentiment analysis is not solved! Assessing and probing sentiment classification",
            "authors": [
                "Jeremy Barnes",
                "Lilja \u00d8vrelid",
                "Erik Velldal"
            ],
            "abstract": "Neural methods for SA have led to quantitative improvements over previous\napproaches, but these advances are not always accompanied with a thorough\nanalysis of the qualitative differences. Therefore, it is not clear what\noutstanding conceptual challenges for sentiment analysis remain. In this work,\nwe attempt to discover what challenges still prove a problem for sentiment\nclassifiers for English and to provide a challenging dataset. We collect the\nsubset of sentences that an (oracle) ensemble of state-of-the-art sentiment\nclassifiers misclassify and then annotate them for 18 linguistic and\nparalinguistic phenomena, such as negation, sarcasm, modality, etc. The dataset\nis available at https://github.com/ltgoslo/assessing_and_probing_sentiment.\nFinally, we provide a case study that demonstrates the usefulness of the\ndataset to probe the performance of a given sentiment classifier with respect\nto linguistic phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05889v1",
            "date": "2019-06-13 18:50:17+00:00",
            "title": "On the Effect of Word Order on Cross-lingual Sentiment Analysis",
            "authors": [
                "\u00c0lex R. Atrio",
                "Toni Badia",
                "Jeremy Barnes"
            ],
            "abstract": "Current state-of-the-art models for sentiment analysis make use of word order\neither explicitly by pre-training on a language modeling objective or\nimplicitly by using recurrent neural networks (RNNs) or convolutional networks\n(CNNs). This is a problem for cross-lingual models that use bilingual\nembeddings as features, as the difference in word order between source and\ntarget languages is not resolved. In this work, we explore reordering as a\npre-processing step for sentence-level cross-lingual sentiment classification\nwith two language combinations (English-Spanish, English-Catalan). We find that\nwhile reordering helps both models, CNNS are more sensitive to local\nreorderings, while global reordering benefits RNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05906v2",
            "date": "2019-06-13 19:30:12+00:00",
            "title": "Meaning to Form: Measuring Systematicity as Information",
            "authors": [
                "Tiago Pimentel",
                "Arya D. McCarthy",
                "Dami\u00e1n E. Blasi",
                "Brian Roark",
                "Ryan Cotterell"
            ],
            "abstract": "A longstanding debate in semiotics centers on the relationship between\nlinguistic signs and their corresponding semantics: is there an arbitrary\nrelationship between a word form and its meaning, or does some systematic\nphenomenon pervade? For instance, does the character bigram \\textit{gl} have\nany systematic relationship to the meaning of words like \\textit{glisten},\n\\textit{gleam} and \\textit{glow}? In this work, we offer a holistic\nquantification of the systematicity of the sign using mutual information and\nrecurrent neural networks. We employ these in a data-driven and massively\nmultilingual approach to the question, examining 106 languages. We find a\nstatistically significant reduction in entropy when modeling a word form\nconditioned on its semantic representation. Encouragingly, we also recover\nwell-attested English examples of systematic affixes. We conclude with the\nmeta-point: Our approximate effect size (measured in bits) is quite\nsmall---despite some amount of systematicity between form and meaning, an\narbitrary relationship and its resulting benefits dominate human language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05993v1",
            "date": "2019-06-14 02:53:26+00:00",
            "title": "Conceptor Debiasing of Word Representations Evaluated on WEAT",
            "authors": [
                "Saket Karve",
                "Lyle Ungar",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Bias in word embeddings such as Word2Vec has been widely investigated, and\nmany efforts made to remove such bias. We show how to use conceptors debiasing\nto post-process both traditional and contextualized word embeddings. Our\nconceptor debiasing can simultaneously remove racial and gender biases and,\nunlike standard debiasing methods, can make effect use of heterogeneous lists\nof biased words. We show that conceptor debiasing diminishes racial and gender\nbias of word representations as measured using the Word Embedding Association\nTest (WEAT) of Caliskan et al. (2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06003v1",
            "date": "2019-06-14 03:27:11+00:00",
            "title": "Cost-sensitive Regularization for Label Confusion-aware Event Detection",
            "authors": [
                "Hongyu Lin",
                "Yaojie Lu",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "In supervised event detection, most of the mislabeling occurs between a small\nnumber of confusing type pairs, including trigger-NIL pairs and sibling\nsub-types of the same coarse type. To address this label confusion problem,\nthis paper proposes cost-sensitive regularization, which can force the training\nprocedure to concentrate more on optimizing confusing type pairs. Specifically,\nwe introduce a cost-weighted term into the training loss, which penalizes more\non mislabeling between confusing label pairs. Furthermore, we also propose two\nestimators which can effectively measure such label confusion based on\ninstance-level or population-level statistics. Experiments on TAC-KBP 2017\ndatasets demonstrate that the proposed method can significantly improve the\nperformances of different models in both English and Chinese event detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06045v1",
            "date": "2019-06-14 06:35:10+00:00",
            "title": "Learning to Ask Unanswerable Questions for Machine Reading Comprehension",
            "authors": [
                "Haichao Zhu",
                "Li Dong",
                "Furu Wei",
                "Wenhui Wang",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Machine reading comprehension with unanswerable questions is a challenging\ntask. In this work, we propose a data augmentation technique by automatically\ngenerating relevant unanswerable questions according to an answerable question\npaired with its corresponding paragraph that contains the answer. We introduce\na pair-to-sequence model for unanswerable question generation, which\neffectively captures the interactions between the question and the paragraph.\nWe also present a way to construct training data for our question generation\nmodels by leveraging the existing reading comprehension dataset. Experimental\nresults show that the pair-to-sequence model performs consistently better\ncompared with the sequence-to-sequence baseline. We further use the\nautomatically generated unanswerable questions as a means of data augmentation\non the SQuAD 2.0 dataset, yielding 1.9 absolute F1 improvement with BERT-base\nmodel and 1.7 absolute F1 improvement with BERT-large model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06050v1",
            "date": "2019-06-14 06:53:57+00:00",
            "title": "Neural Response Generation with Meta-Words",
            "authors": [
                "Can Xu",
                "Wei Wu",
                "Chongyang Tao",
                "Huang Hu",
                "Matt Schuerman",
                "Ying Wang"
            ],
            "abstract": "We present open domain response generation with meta-words. A meta-word is a\nstructured record that describes various attributes of a response, and thus\nallows us to explicitly model the one-to-many relationship within open domain\ndialogues and perform response generation in an explainable and controllable\nmanner. To incorporate meta-words into generation, we enhance the\nsequence-to-sequence architecture with a goal tracking memory network that\nformalizes meta-word expression as a goal and manages the generation process to\nachieve the goal with a state memory panel and a state controller. Experimental\nresults on two large-scale datasets indicate that our model can significantly\noutperform several state-of-the-art generation models in terms of response\nrelevance, response diversity, accuracy of one-to-many modeling, accuracy of\nmeta-word expression, and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06056v1",
            "date": "2019-06-14 07:29:29+00:00",
            "title": "Microsoft AI Challenge India 2018: Learning to Rank Passages for Web Question Answering with Deep Attention Networks",
            "authors": [
                "Chaitanya Sai Alaparthi"
            ],
            "abstract": "This paper describes our system for The Microsoft AI Challenge India 2018:\nRanking Passages for Web Question Answering. The system uses the biLSTM network\nwith co-attention mechanism between query and passage representations.\nAdditionally, we use self attention on embeddings to increase the lexical\ncoverage by allowing the system to take union over different embeddings. We\nalso incorporate hand-crafted features to improve the system performance. Our\nsystem achieved a Mean Reciprocal Rank (MRR) of 0.67 on eval-1 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06127v3",
            "date": "2019-06-14 11:12:20+00:00",
            "title": "DocRED: A Large-Scale Document-Level Relation Extraction Dataset",
            "authors": [
                "Yuan Yao",
                "Deming Ye",
                "Peng Li",
                "Xu Han",
                "Yankai Lin",
                "Zhenghao Liu",
                "Zhiyuan Liu",
                "Lixin Huang",
                "Jie Zhou",
                "Maosong Sun"
            ],
            "abstract": "Multiple entities in a document generally exhibit complex inter-sentence\nrelations, and cannot be well handled by existing relation extraction (RE)\nmethods that typically focus on extracting intra-sentence relations for single\nentity pairs. In order to accelerate the research on document-level RE, we\nintroduce DocRED, a new dataset constructed from Wikipedia and Wikidata with\nthree features: (1) DocRED annotates both named entities and relations, and is\nthe largest human-annotated dataset for document-level RE from plain text; (2)\nDocRED requires reading multiple sentences in a document to extract entities\nand infer their relations by synthesizing all information of the document; (3)\nalong with the human-annotated data, we also offer large-scale distantly\nsupervised data, which enables DocRED to be adopted for both supervised and\nweakly supervised scenarios. In order to verify the challenges of\ndocument-level RE, we implement recent state-of-the-art methods for RE and\nconduct a thorough evaluation of these methods on DocRED. Empirical results\nshow that DocRED is challenging for existing RE methods, which indicates that\ndocument-level RE remains an open problem and requires further efforts. Based\non the detailed analysis on the experiments, we discuss multiple promising\ndirections for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06216v1",
            "date": "2019-06-14 14:14:42+00:00",
            "title": "Improving Visual Question Answering by Referring to Generated Paragraph Captions",
            "authors": [
                "Hyounghun Kim",
                "Mohit Bansal"
            ],
            "abstract": "Paragraph-style image captions describe diverse aspects of an image as\nopposed to the more common single-sentence captions that only provide an\nabstract description of the image. These paragraph captions can hence contain\nsubstantial information of the image for tasks such as visual question\nanswering. Moreover, this textual information is complementary with visual\ninformation present in the image because it can discuss both more abstract\nconcepts and more explicit, intermediate symbolic information about objects,\nevents, and scenes that can directly be matched with the textual question and\ncopied into the textual answer (i.e., via easier modality match). Hence, we\npropose a combined Visual and Textual Question Answering (VTQA) model which\ntakes as input a paragraph caption as well as the corresponding image, and\nanswers the given question based on both inputs. In our model, the inputs are\nfused to extract related information by cross-attention (early fusion), then\nfused again in the form of consensus (late fusion), and finally expected\nanswers are given an extra score to enhance the chance of selection (later\nfusion). Empirical results show that paragraph captions, even when\nautomatically generated (via an RL-based encoder-decoder model), help correctly\nanswer more visual questions. Overall, our joint model, when trained on the\nVisual Genome dataset, significantly improves the VQA performance over a strong\nbaseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06362v1",
            "date": "2019-06-14 18:39:34+00:00",
            "title": "Comparison of Diverse Decoding Methods from Conditional Language Models",
            "authors": [
                "Daphne Ippolito",
                "Reno Kriz",
                "Maria Kustikova",
                "Jo\u00e3o Sedoc",
                "Chris Callison-Burch"
            ],
            "abstract": "While conditional language models have greatly improved in their ability to\noutput high-quality natural language, many NLP applications benefit from being\nable to generate a diverse set of candidate sequences. Diverse decoding\nstrategies aim to, within a given-sized candidate list, cover as much of the\nspace of high-quality outputs as possible, leading to improvements for tasks\nthat re-rank and combine candidate outputs. Standard decoding methods, such as\nbeam search, optimize for generating high likelihood sequences rather than\ndiverse ones, though recent work has focused on increasing diversity in these\nmethods. In this work, we perform an extensive survey of decoding-time\nstrategies for generating diverse outputs from conditional language models. We\nalso show how diversity can be improved without sacrificing quality by\nover-sampling additional candidates, then filtering to the desired number.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06401v1",
            "date": "2019-06-14 21:00:20+00:00",
            "title": "\"My Way of Telling a Story\": Persona based Grounded Story Generation",
            "authors": [
                "Shrimai Prabhumoye",
                "Khyathi Raghavi Chandu",
                "Ruslan Salakhutdinov",
                "Alan W Black"
            ],
            "abstract": "Visual storytelling is the task of generating stories based on a sequence of\nimages. Inspired by the recent works in neural generation focusing on\ncontrolling the form of text, this paper explores the idea of generating these\nstories in different personas. However, one of the main challenges of\nperforming this task is the lack of a dataset of visual stories in different\npersonas. Having said that, there are independent datasets for both visual\nstorytelling and annotated sentences for various persona. In this paper we\ndescribe an approach to overcome this by getting labelled persona data from a\ndifferent task and leveraging those annotations to perform persona based story\ngeneration. We inspect various ways of incorporating personality in both the\nencoder and the decoder representations to steer the generation in the target\ndirection. To this end, we propose five models which are incremental extensions\nto the baseline model to perform the task at hand. In our experiments we use\nfive different personas to guide the generation process. We find that the\nmodels based on our hypotheses perform better at capturing words while\ngenerating stories in the target persona.",
            "journal ref": "Storytelling Workshop at ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06448v2",
            "date": "2019-06-15 01:18:41+00:00",
            "title": "Can neural networks understand monotonicity reasoning?",
            "authors": [
                "Hitomi Yanaka",
                "Koji Mineshima",
                "Daisuke Bekki",
                "Kentaro Inui",
                "Satoshi Sekine",
                "Lasha Abzianidze",
                "Johan Bos"
            ],
            "abstract": "Monotonicity reasoning is one of the important reasoning skills for any\nintelligent natural language inference (NLI) model in that it requires the\nability to capture the interaction between lexical and syntactic structures.\nSince no test set has been developed for monotonicity reasoning with wide\ncoverage, it is still unclear whether neural models can perform monotonicity\nreasoning in a proper way. To investigate this issue, we introduce the\nMonotonicity Entailment Dataset (MED). Performance by state-of-the-art NLI\nmodels on the new test set is substantially worse, under 55%, especially on\ndownward reasoning. In addition, analysis using a monotonicity-driven data\naugmentation method showed that these models might be limited in their\ngeneralization ability in upward and downward reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06493v1",
            "date": "2019-06-15 08:13:32+00:00",
            "title": "A weakly supervised sequence tagging and grammar induction approach to semantic frame slot filling",
            "authors": [
                "Janneke van de Loo",
                "Guy De Pauw",
                "Walter Daelemans"
            ],
            "abstract": "This paper describes continuing work on semantic frame slot filling for a\ncommand and control task using a weakly-supervised approach. We investigate the\nadvantages of using retraining techniques that take the output of a\nhierarchical hidden markov model as input to two inductive approaches: (1)\ndiscriminative sequence labelers based on conditional random fields and\nmemory-based learning and (2) probabilistic context-free grammar induction.\nExperimental results show that this setup can significantly improve F-scores\nwithout the need for additional information sources. Furthermore, qualitative\nanalysis shows that the weakly supervised technique is able to automatically\ninduce an easily interpretable and syntactically appropriate grammar for the\ndomain and task at hand.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06550v1",
            "date": "2019-06-15 12:47:18+00:00",
            "title": "Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks",
            "authors": [
                "Ahmad Aghaebrahimian",
                "Mark Cieliebak"
            ],
            "abstract": "We report our ongoing work about a new deep architecture working in tandem\nwith a statistical test procedure for jointly training texts and their label\ndescriptions for multi-label and multi-class classification tasks. A\nstatistical hypothesis testing method is used to extract the most informative\nwords for each given class. These words are used as a class description for\nmore label-aware text classification. Intuition is to help the model to\nconcentrate on more informative words rather than more frequent ones. The model\nleverages the use of label descriptions in addition to the input text to\nenhance text classification performance. Our method is entirely data-driven,\nhas no dependency on other sources of information than the training data, and\nis adaptable to different classification problems by providing appropriate\ntraining data without major hyper-parameter tuning. We trained and tested our\nsystem on several publicly available datasets, where we managed to improve the\nstate-of-the-art on one set with a high margin, and to obtain competitive\nresults on all other ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06678v1",
            "date": "2019-06-16 13:10:33+00:00",
            "title": "Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification",
            "authors": [
                "Zhi-Xiu Ye",
                "Zhen-Hua Ling"
            ],
            "abstract": "This paper presents a multi-level matching and aggregation network (MLMAN)\nfor few-shot relation classification. Previous studies on this topic adopt\nprototypical networks, which calculate the embedding vector of a query instance\nand the prototype vector of each support set independently. In contrast, our\nproposed MLMAN model encodes the query instance and each support set in an\ninteractive way by considering their matching information at both local and\ninstance levels. The final class prototype for each support set is obtained by\nattentive aggregation over the representations of its support instances, where\nthe weights are calculated using the query instance. Experimental results\ndemonstrate the effectiveness of our proposed methods, which achieve a new\nstate-of-the-art performance on the FewRel dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06703v1",
            "date": "2019-06-16 14:33:45+00:00",
            "title": "Using Automatically Extracted Minimum Spans to Disentangle Coreference Evaluation from Boundary Detection",
            "authors": [
                "Nafise Sadat Moosavi",
                "Leo Born",
                "Massimo Poesio",
                "Michael Strube"
            ],
            "abstract": "The common practice in coreference resolution is to identify and evaluate the\nmaximum span of mentions. The use of maximum spans tangles coreference\nevaluation with the challenges of mention boundary detection like prepositional\nphrase attachment. To address this problem, minimum spans are manually\nannotated in smaller corpora. However, this additional annotation is costly and\ntherefore, this solution does not scale to large corpora. In this paper, we\npropose the MINA algorithm for automatically extracting minimum spans to\nbenefit from minimum span evaluation in all corpora. We show that the extracted\nminimum spans by MINA are consistent with those that are manually annotated by\nexperts. Our experiments show that using minimum spans is in particular\nimportant in cross-dataset coreference evaluation, in which detected mention\nboundaries are noisier due to domain shift. We will integrate MINA into\nhttps://github.com/ns-moosavi/coval for reporting standard coreference scores\nbased on both maximum and automatically detected minimum spans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06718v1",
            "date": "2019-06-16 15:37:54+00:00",
            "title": "Neural Decipherment via Minimum-Cost Flow: from Ugaritic to Linear B",
            "authors": [
                "Jiaming Luo",
                "Yuan Cao",
                "Regina Barzilay"
            ],
            "abstract": "In this paper we propose a novel neural approach for automatic decipherment\nof lost languages. To compensate for the lack of strong supervision signal, our\nmodel design is informed by patterns in language change documented in\nhistorical linguistics. The model utilizes an expressive sequence-to-sequence\nmodel to capture character-level correspondences between cognates. To\neffectively train the model in an unsupervised manner, we innovate the training\nprocedure by formalizing it as a minimum-cost flow problem. When applied to the\ndecipherment of Ugaritic, we achieve a 5.5% absolute improvement over\nstate-of-the-art results. We also report the first automatic results in\ndeciphering Linear B, a syllabic language related to ancient Greek, where our\nmodel correctly translates 67.3% of cognates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06870v1",
            "date": "2019-06-17 07:07:01+00:00",
            "title": "Robust Zero-Shot Cross-Domain Slot Filling with Example Values",
            "authors": [
                "Darsh J Shah",
                "Raghav Gupta",
                "Amir A Fayazi",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Task-oriented dialog systems increasingly rely on deep learning-based slot\nfilling models, usually needing extensive labeled training data for target\ndomains. Often, however, little to no target domain training data may be\navailable, or the training and target domain schemas may be misaligned, as is\ncommon for web forms on similar websites. Prior zero-shot slot filling models\nuse slot descriptions to learn concepts, but are not robust to misaligned\nschemas. We propose utilizing both the slot description and a small number of\nexamples of slot values, which may be easily available, to learn semantic\nrepresentations of slots which are transferable across domains and robust to\nmisaligned schemas. Our approach outperforms state-of-the-art models on two\nmulti-domain datasets, especially in the low-data setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06893v1",
            "date": "2019-06-17 08:30:55+00:00",
            "title": "Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling",
            "authors": [
                "Yifan Gao",
                "Piji Li",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "We study the problem of generating interconnected questions in\nquestion-answering style conversations. Compared with previous works which\ngenerate questions based on a single sentence (or paragraph), this setting is\ndifferent in two major aspects: (1) Questions are highly conversational. Almost\nhalf of them refer back to conversation history using coreferences. (2) In a\ncoherent conversation, questions have smooth transitions between turns. We\npropose an end-to-end neural model with coreference alignment and conversation\nflow modeling. The coreference alignment modeling explicitly aligns coreferent\nmentions in conversation history with corresponding pronominal references in\ngenerated questions, which makes generated questions interconnected to\nconversation history. The conversation flow modeling builds a coherent\nconversation by starting questioning on the first few sentences in a text\npassage and smoothly shifting the focus to later parts. Extensive experiments\nshow that our system outperforms several baselines and can generate highly\nconversational questions. The code implementation is released at\nhttps://github.com/Evan-Gao/conversational-QG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06905v2",
            "date": "2019-06-17 08:57:38+00:00",
            "title": "Manipulating the Difficulty of C-Tests",
            "authors": [
                "Ji-Ung Lee",
                "Erik Schwan",
                "Christian M. Meyer"
            ],
            "abstract": "We propose two novel manipulation strategies for increasing and decreasing\nthe difficulty of C-tests automatically. This is a crucial step towards\ngenerating learner-adaptive exercises for self-directed language learning and\npreparing language assessment tests. To reach the desired difficulty level, we\nmanipulate the size and the distribution of gaps based on absolute and relative\ngap difficulty predictions. We evaluate our approach in corpus-based\nexperiments and in a user study with 60 participants. We find that both\nstrategies are able to generate C-tests with the desired difficulty level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06906v1",
            "date": "2019-06-17 09:00:01+00:00",
            "title": "An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis",
            "authors": [
                "Ruidan He",
                "Wee Sun Lee",
                "Hwee Tou Ng",
                "Daniel Dahlmeier"
            ],
            "abstract": "Aspect-based sentiment analysis produces a list of aspect terms and their\ncorresponding sentiments for a natural language sentence. This task is usually\ndone in a pipeline manner, with aspect term extraction performed first,\nfollowed by sentiment predictions toward the extracted aspect terms. While\neasier to develop, such an approach does not fully exploit joint information\nfrom the two subtasks and does not use all available sources of training\ninformation that might be helpful, such as document-level labeled sentiment\ncorpus. In this paper, we propose an interactive multi-task learning network\n(IMN) which is able to jointly learn multiple related tasks simultaneously at\nboth the token level as well as the document level. Unlike conventional\nmulti-task learning methods that rely on learning common features for the\ndifferent tasks, IMN introduces a message passing architecture where\ninformation is iteratively passed to different tasks through a shared set of\nlatent variables. Experimental results demonstrate superior performance of the\nproposed method against multiple baselines on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06916v2",
            "date": "2019-06-17 09:33:27+00:00",
            "title": "BERE: An accurate distantly supervised biomedical entity relation extraction network",
            "authors": [
                "Lixiang Hong",
                "JinJian Lin",
                "Jiang Tao",
                "Jianyang Zeng"
            ],
            "abstract": "Automated entity relation extraction (RE) from literature provides an\nimportant source for constructing biomedical database, which is more efficient\nand extensible than manual curation. However, existing RE models usually ignore\nthe information contained in sentence structures and target entities. In this\npaper, we propose BERE, a deep learning based model which uses Gumbel Tree-GRU\nto learn sentence structures and joint embedding to incorporate entity\ninformation. It also employs word-level attention for improved relation\nextraction and sentence-level attention to suit the distantly supervised\ndataset. Because the existing dataset are relatively small, we further\nconstruct a much larger drug-target interaction extraction (DTIE) dataset by\ndistant supervision. Experiments conducted on both DDIExtraction 2013 task and\nDTIE dataset show our model's effectiveness over state-of-the-art baselines in\nterms of F1 measures and PR curves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06947v1",
            "date": "2019-06-17 11:01:20+00:00",
            "title": "Open Domain Event Extraction Using Neural Latent Variable Models",
            "authors": [
                "Xiao Liu",
                "Heyan Huang",
                "Yue Zhang"
            ],
            "abstract": "We consider open domain event extraction, the task of extracting unconstraint\ntypes of events from news clusters. A novel latent variable neural model is\nconstructed, which is scalable to very large corpus. A dataset is collected and\nmanually annotated, with task-specific evaluation metrics being designed.\nResults show that the proposed unsupervised model gives better performance\ncompared to the state-of-the-art method for event schema induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07020v1",
            "date": "2019-06-14 14:56:55+00:00",
            "title": "Attention-based Modeling for Emotion Detection and Classification in Textual Conversations",
            "authors": [
                "Waleed Ragheb",
                "J\u00e9r\u00f4me Az\u00e9",
                "Sandra Bringay",
                "Maximilien Servajean"
            ],
            "abstract": "This paper addresses the problem of modeling textual conversations and\ndetecting emotions. Our proposed model makes use of 1) deep transfer learning\nrather than the classical shallow methods of word embedding; 2) self-attention\nmechanisms to focus on the most important parts of the texts and 3) turn-based\nconversational modeling for classifying the emotions. The approach does not\nrely on any hand-crafted features or lexicons. Our model was evaluated on the\ndata provided by the SemEval-2019 shared task on contextual emotion detection\nin text. The model shows very competitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07040v1",
            "date": "2019-06-17 14:01:33+00:00",
            "title": "Making Fast Graph-based Algorithms with Graph Metric Embeddings",
            "authors": [
                "Andrey Kutuzov",
                "Mohammad Dorgham",
                "Oleksiy Oliynyk",
                "Chris Biemann",
                "Alexander Panchenko"
            ],
            "abstract": "The computation of distance measures between nodes in graphs is inefficient\nand does not scale to large graphs. We explore dense vector representations as\nan effective way to approximate the same information: we introduce a simple yet\nefficient and effective approach for learning graph embeddings. Instead of\ndirectly operating on the graph structure, our method takes structural measures\nof pairwise node similarities into account and learns dense node\nrepresentations reflecting user-defined graph distance measures, such as\ne.g.the shortest path distance or distance measures that take information\nbeyond the graph structure into account. We demonstrate a speed-up of several\norders of magnitude when predicting word similarity by vector operations on our\nembeddings as opposed to directly computing the respective path-based measures,\nwhile outperforming various other graph embeddings on semantic similarity and\nword sense disambiguation tasks and show evaluations on the WordNet graph and\ntwo knowledge base graphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07108v1",
            "date": "2019-06-17 16:18:28+00:00",
            "title": "Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing",
            "authors": [
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Ming Zhou",
                "Jian Yin"
            ],
            "abstract": "In this paper, we present an approach to incorporate retrieved datapoints as\nsupporting evidence for context-dependent semantic parsing, such as generating\nsource code conditioned on the class environment. Our approach naturally\ncombines a retrieval model and a meta-learner, where the former learns to find\nsimilar datapoints from the training data, and the latter considers retrieved\ndatapoints as a pseudo task for fast adaptation. Specifically, our retriever is\na context-aware encoder-decoder model with a latent variable which takes\ncontext environment into consideration, and our meta-learner learns to utilize\nretrieved datapoints in a model-agnostic meta-learning paradigm for fast\nadaptation. We conduct experiments on CONCODE and CSQA datasets, where the\ncontext refers to class environment in JAVA codes and conversational history,\nrespectively. We use sequence-to-action model as the base semantic parser,\nwhich performs the state-of-the-art accuracy on both datasets. Results show\nthat both the context-aware retriever and the meta-learning strategy improve\naccuracy, and our approach performs better than retrieve-and-edit baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07241v2",
            "date": "2019-06-17 19:48:41+00:00",
            "title": "Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling",
            "authors": [
                "Robert L. Logan IV",
                "Nelson F. Liu",
                "Matthew E. Peters",
                "Matt Gardner",
                "Sameer Singh"
            ],
            "abstract": "Modeling human language requires the ability to not only generate fluent text\nbut also encode factual knowledge. However, traditional language models are\nonly capable of remembering facts seen at training time, and often have\ndifficulty recalling them. To address this, we introduce the knowledge graph\nlanguage model (KGLM), a neural language model with mechanisms for selecting\nand copying facts from a knowledge graph that are relevant to the context.\nThese mechanisms enable the model to render information it has never seen\nbefore, as well as generate out-of-vocabulary tokens. We also introduce the\nLinked WikiText-2 dataset, a corpus of annotated text aligned to the Wikidata\nknowledge graph whose contents (roughly) match the popular WikiText-2\nbenchmark. In experiments, we demonstrate that the KGLM achieves significantly\nbetter performance than a strong baseline language model. We additionally\ncompare different language model's ability to complete sentences requiring\nfactual knowledge, showing that the KGLM outperforms even very large language\nmodels in generating facts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07280v1",
            "date": "2019-06-17 21:31:40+00:00",
            "title": "A Structured Distributional Model of Sentence Meaning and Processing",
            "authors": [
                "Emmanuele Chersoni",
                "Enrico Santus",
                "Ludovica Pannitto",
                "Alessandro Lenci",
                "Philippe Blache",
                "Chu-Ren Huang"
            ],
            "abstract": "Most compositional distributional semantic models represent sentence meaning\nwith a single vector. In this paper, we propose a Structured Distributional\nModel (SDM) that combines word embeddings with formal semantics and is based on\nthe assumption that sentences represent events and situations. The semantic\nrepresentation of a sentence is a formal structure derived from Discourse\nRepresentation Theory and containing distributional vectors. This structure is\ndynamically and incrementally built by integrating knowledge about events and\ntheir typical participants, as they are activated by lexical items. Event\nknowledge is modeled as a graph extracted from parsed corpora and encoding\nroles and relationships between participants that are represented as\ndistributional vectors. SDM is grounded on extensive psycholinguistic research\nshowing that generalized knowledge about events stored in semantic memory plays\na key role in sentence comprehension. We evaluate SDM on two recently\nintroduced compositionality datasets, and our results show that combining a\nsimple compositional model with event knowledge constantly improves\nperformances, even with different types of word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07285v1",
            "date": "2019-06-17 21:53:10+00:00",
            "title": "Tabula nearly rasa: Probing the Linguistic Knowledge of Character-Level Neural Language Models Trained on Unsegmented Text",
            "authors": [
                "Michael Hahn",
                "Marco Baroni"
            ],
            "abstract": "Recurrent neural networks (RNNs) have reached striking performance in many\nnatural language processing tasks. This has renewed interest in whether these\ngeneric sequence processing devices are inducing genuine linguistic knowledge.\nNearly all current analytical studies, however, initialize the RNNs with a\nvocabulary of known words, and feed them tokenized input during training. We\npresent a multi-lingual study of the linguistic knowledge encoded in RNNs\ntrained as character-level language models, on input data with word boundaries\nremoved. These networks face a tougher and more cognitively realistic task,\nhaving to discover any useful linguistic unit from scratch based on input\nstatistics. The results show that our \"near tabula rasa\" RNNs are mostly able\nto solve morphological, syntactic and semantic tasks that intuitively\npresuppose word-level knowledge, and indeed they learned, to some extent, to\ntrack word boundaries. Our study opens the door to speculations about the\nnecessity of an explicit, rigid word lexicon in language learning and usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07337v1",
            "date": "2019-06-18 01:58:56+00:00",
            "title": "Measuring Bias in Contextualized Word Representations",
            "authors": [
                "Keita Kurita",
                "Nidhi Vyas",
                "Ayush Pareek",
                "Alan W Black",
                "Yulia Tsvetkov"
            ],
            "abstract": "Contextual word embeddings such as BERT have achieved state of the art\nperformance in numerous NLP tasks. Since they are optimized to capture the\nstatistical properties of training data, they tend to pick up on and amplify\nsocial stereotypes present in the data as well. In this study, we (1)~propose a\ntemplate-based method to quantify bias in BERT; (2)~show that this method\nobtains more consistent results in capturing social biases than the traditional\ncosine based method; and (3)~conduct a case study, evaluating gender bias in a\ndownstream task of Gender Pronoun Resolution. Although our case study focuses\non gender bias, the proposed technique is generalizable to unveiling other\nbiases, including in multiclass settings, such as racial and religious biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07382v1",
            "date": "2019-06-18 05:14:17+00:00",
            "title": "Curriculum Learning Strategies for Hindi-English Codemixed Sentiment Analysis",
            "authors": [
                "Anirudh Dahiya",
                "Neeraj Battan",
                "Manish Shrivastava",
                "Dipti Mishra Sharma"
            ],
            "abstract": "Sentiment Analysis and other semantic tasks are commonly used for social\nmedia textual analysis to gauge public opinion and make sense from the noise on\nsocial media. The language used on social media not only commonly diverges from\nthe formal language, but is compounded by codemixing between languages,\nespecially in large multilingual societies like India.\n  Traditional methods for learning semantic NLP tasks have long relied on end\nto end task specific training, requiring expensive data creation process, even\nmore so for deep learning methods. This challenge is even more severe for\nresource scarce texts like codemixed language pairs, with lack of well learnt\nrepresentations as model priors, and task specific datasets can be few and\nsmall in quantities to efficiently exploit recent deep learning approaches. To\naddress above challenges, we introduce curriculum learning strategies for\nsemantic tasks in code-mixed Hindi-English (Hi-En) texts, and investigate\nvarious training strategies for enhancing model performance. Our method\noutperforms the state of the art methods for Hi-En codemixed sentiment analysis\nby 3.31% accuracy, and also shows better model robustness in terms of\nconvergence, and variance in test performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07429v1",
            "date": "2019-06-18 07:57:22+00:00",
            "title": "Modeling Semantic Relationship in Multi-turn Conversations with Hierarchical Latent Variables",
            "authors": [
                "Lei Shen",
                "Yang Feng",
                "Haolan Zhan"
            ],
            "abstract": "Multi-turn conversations consist of complex semantic structures, and it is\nstill a challenge to generate coherent and diverse responses given previous\nutterances. It's practical that a conversation takes place under a background,\nmeanwhile, the query and response are usually most related and they are\nconsistent in topic but also different in content. However, little work focuses\non such hierarchical relationship among utterances. To address this problem, we\npropose a Conversational Semantic Relationship RNN (CSRR) model to construct\nthe dependency explicitly. The model contains latent variables in three\nhierarchies. The discourse-level one captures the global background, the\npair-level one stands for the common topic information between query and\nresponse, and the utterance-level ones try to represent differences in content.\nExperimental results show that our model significantly improves the quality of\nresponses in terms of fluency, coherence and diversity compared to baseline\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07525v1",
            "date": "2019-06-18 12:27:25+00:00",
            "title": "Mimicking Human Process: Text Representation via Latent Semantic Clustering for Classification",
            "authors": [
                "Xiaoye Tan",
                "Rui Yan",
                "Chongyang Tao",
                "Mingrui Wu"
            ],
            "abstract": "Considering that words with different characteristic in the text have\ndifferent importance for classification, grouping them together separately can\nstrengthen the semantic expression of each part. Thus we propose a new text\nrepresentation scheme by clustering words according to their latent semantics\nand composing them together to get a set of cluster vectors, which are then\nconcatenated as the final text representation. Evaluation on five\nclassification benchmarks proves the effectiveness of our method. We further\nconduct visualization analysis showing statistical clustering results and\nverifying the validity of our motivation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07544v2",
            "date": "2019-06-18 13:17:13+00:00",
            "title": "Transfer Learning for Causal Sentence Detection",
            "authors": [
                "Manolis Kyriakakis",
                "Ion Androutsopoulos",
                "Joan Gin\u00e9s i Ametll\u00e9",
                "Artur Saudabayev"
            ],
            "abstract": "We consider the task of detecting sentences that express causality, as a step\ntowards mining causal relations from texts. To bypass the scarcity of causal\ninstances in relation extraction datasets, we exploit transfer learning, namely\nELMO and BERT, using a bidirectional GRU with self-attention (BIGRUATT) as a\nbaseline. We experiment with both generic public relation extraction datasets\nand a new biomedical causal sentence detection dataset, a subset of which we\nmake publicly available. We find that transfer learning helps only in very\nsmall datasets. With larger datasets, BIGRUATT reaches a performance plateau,\nthen larger datasets and transfer learning do not help.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07555v1",
            "date": "2019-06-18 13:29:04+00:00",
            "title": "Automatic learner summary assessment for reading comprehension",
            "authors": [
                "Menglin Xia",
                "Ekaterina Kochmar",
                "Ted Briscoe"
            ],
            "abstract": "Automating the assessment of learner summaries provides a useful tool for\nassessing learner reading comprehension. We present a summarization task for\nevaluating non-native reading comprehension and propose three novel approaches\nto automatically assess the learner summaries. We evaluate our models on two\ndatasets we created and show that our models outperform traditional approaches\nthat rely on exact word match on this task. Our best model produces quality\nassessments close to professional examiners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07562v1",
            "date": "2019-06-18 13:32:20+00:00",
            "title": "Hyperintensional Reasoning based on Natural Language Knowledge Base",
            "authors": [
                "Marie Du\u017e\u00ed",
                "Ale\u0161 Hor\u00e1k"
            ],
            "abstract": "The success of automated reasoning techniques over large natural-language\ntexts heavily relies on a fine-grained analysis of natural language\nassumptions. While there is a common agreement that the analysis should be\nhyperintensional, most of the automatic reasoning systems are still based on an\nintensional logic, at the best. In this paper, we introduce the system of\nreasoning based on a fine-grained, hyperintensional analysis. To this end we\napply Tichy's Transparent Intensional Logic (TIL) with its procedural\nsemantics. TIL is a higher-order, hyperintensional logic of partial functions,\nin particular apt for a fine-grained natural-language analysis. Within TIL we\nrecognise three kinds of context, namely extensional, intensional and\nhyperintensional, in which a particular natural-language term, or rather its\nmeaning, can occur. Having defined the three kinds of context and implemented\nan algorithm of context recognition, we are in a position to develop and\nimplement an extensional logic of hyperintensions with the inference machine\nthat should neither over-infer nor under-infer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07580v1",
            "date": "2019-06-18 13:46:21+00:00",
            "title": "Text Readability Assessment for Second Language Learners",
            "authors": [
                "Menglin Xia",
                "Ekaterina Kochmar",
                "Ted Briscoe"
            ],
            "abstract": "This paper addresses the task of readability assessment for the texts aimed\nat second language (L2) learners. One of the major challenges in this task is\nthe lack of significantly sized level-annotated data. For the present work, we\ncollected a dataset of CEFR-graded texts tailored for learners of English as an\nL2 and investigated text readability assessment for both native and L2\nlearners. We applied a generalization method to adapt models trained on larger\nnative corpora to estimate text readability for learners, and explored domain\nadaptation and self-learning techniques to make use of the native data to\nimprove system performance on the limited L2 data. In our experiments, the best\nperforming model for readability on learner texts achieves an accuracy of 0.797\nand PCC of $0.938$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07592v1",
            "date": "2019-06-18 14:06:40+00:00",
            "title": "Towards Robust Named Entity Recognition for Historic German",
            "authors": [
                "Stefan Schweter",
                "Johannes Baiter"
            ],
            "abstract": "Recent advances in language modeling using deep neural networks have shown\nthat these models learn representations, that vary with the network depth from\nmorphology to semantic relationships like co-reference. We apply pre-trained\nlanguage models to low-resource named entity recognition for Historic German.\nWe show on a series of experiments that character-based pre-trained language\nmodels do not run into trouble when faced with low-resource datasets. Our\npre-trained character-based language models improve upon classical CRF-based\nmethods and previous work on Bi-LSTMs by boosting F1 score performance by up to\n6%. Our pre-trained language and NER models are publicly available under\nhttps://github.com/stefan-it/historic-ner .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07599v1",
            "date": "2019-06-18 14:18:31+00:00",
            "title": "LTG-Oslo Hierarchical Multi-task Network: The importance of negation for document-level sentiment in Spanish",
            "authors": [
                "Jeremy Barnes"
            ],
            "abstract": "This paper details LTG-Oslo team's participation in the sentiment track of\nthe NEGES 2019 evaluation campaign. We participated in the task with a\nhierarchical multi-task network, which used shared lower-layers in a deep\nBiLSTM to predict negation, while the higher layers were dedicated to\npredicting document-level sentiment. The multi-task component shows promise as\na way to incorporate information on negation into deep neural sentiment\nclassifiers, despite the fact that the absolute results on the test set were\nrelatively low for a binary classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07610v2",
            "date": "2019-06-18 14:31:58+00:00",
            "title": "Improving Sentiment Analysis with Multi-task Learning of Negation",
            "authors": [
                "Jeremy Barnes",
                "Erik Velldal",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "Sentiment analysis is directly affected by compositional phenomena in\nlanguage that act on the prior polarity of the words and phrases found in the\ntext. Negation is the most prevalent of these phenomena and in order to\ncorrectly predict sentiment, a classifier must be able to identify negation and\ndisentangle the effect that its scope has on the final polarity of a text. This\npaper proposes a multi-task approach to explicitly incorporate information\nabout negation in sentiment analysis, which we show outperforms learning\nnegation implicitly in a data-driven manner. We describe our approach, a\ncascading neural architecture with selective sharing of LSTM layers, and show\nthat explicitly training the model with negation as an auxiliary task helps\nimprove the main task of sentiment analysis. The effect is demonstrated across\nseveral different standard English-language data sets for both tasks and we\nanalyze several aspects of our system related to its performance, varying types\nand amounts of input data and different multi-task setups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07651v1",
            "date": "2019-06-18 15:46:08+00:00",
            "title": "Scheduled Sampling for Transformers",
            "authors": [
                "Tsvetomila Mihaylova",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Scheduled sampling is a technique for avoiding one of the known problems in\nsequence-to-sequence generation: exposure bias. It consists of feeding the\nmodel a mix of the teacher forced embeddings and the model predictions from the\nprevious step in training time. The technique has been used for improving the\nmodel performance with recurrent neural networks (RNN). In the Transformer\nmodel, unlike the RNN, the generation of a new word attends to the full\nsentence generated so far, not only to the last word, and it is not\nstraightforward to apply the scheduled sampling technique. We propose some\nstructural changes to allow scheduled sampling to be applied to Transformer\narchitecture, via a two-pass decoding strategy. Experiments on two language\npairs achieve performance close to a teacher-forcing baseline and show that\nthis technique is promising for further exploration.",
            "journal ref": "ACL 2019 Student Research Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07662v1",
            "date": "2019-06-18 16:00:56+00:00",
            "title": "State-of-the-Art Vietnamese Word Segmentation",
            "authors": [
                "Song Nguyen Duc Cong",
                "Quoc Hung Ngo",
                "Rachsuda Jiamthapthaksin"
            ],
            "abstract": "Word segmentation is the first step of any tasks in Vietnamese language\nprocessing. This paper reviews stateof-the-art approaches and systems for word\nsegmentation in Vietnamese. To have an overview of all stages from building\ncorpora to developing toolkits, we discuss building the corpus stage,\napproaches applied to solve the word segmentation and existing toolkits to\nsegment words in Vietnamese sentences. In addition, this study shows clearly\nthe motivations on building corpus and implementing machine learning techniques\nto improve the accuracy for Vietnamese word segmentation. According to our\nobservation, this study also reports a few of achivements and limitations in\nexisting Vietnamese word segmentation systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07701v1",
            "date": "2019-06-18 17:30:30+00:00",
            "title": "Distilling Translations with Visual Awareness",
            "authors": [
                "Julia Ive",
                "Pranava Madhyastha",
                "Lucia Specia"
            ],
            "abstract": "Previous work on multimodal machine translation has shown that visual\ninformation is only needed in very specific cases, for example in the presence\nof ambiguous words where the textual context is not sufficient. As a\nconsequence, models tend to learn to ignore this information. We propose a\ntranslate-and-refine approach to this problem where images are only used by a\nsecond stage decoder. This approach is trained jointly to generate a good first\ndraft translation and to improve over this draft by (i) making better use of\nthe target language textual context (both left and right-side contexts) and\n(ii) making use of visual context. This approach leads to the state of the art\nresults. Additionally, we show that it has the ability to recover from\nerroneous or missing words in the source language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07808v1",
            "date": "2019-06-18 20:56:11+00:00",
            "title": "Adaptation of Machine Translation Models with Back-translated Data using Transductive Data Selection Methods",
            "authors": [
                "Alberto Poncelas",
                "Gideon Maillette de Buy Wenniger",
                "Andy Way"
            ],
            "abstract": "Data selection has proven its merit for improving Neural Machine Translation\n(NMT), when applied to authentic data. But the benefit of using synthetic data\nin NMT training, produced by the popular back-translation technique, raises the\nquestion if data selection could also be useful for synthetic data?\n  In this work we use Infrequent N-gram Recovery (INR) and Feature Decay\nAlgorithms (FDA), two transductive data selection methods to obtain subsets of\nsentences from synthetic data. These methods ensure that selected sentences\nshare n-grams with the test set so the NMT model can be adapted to translate\nit.\n  Performing data selection on back-translated data creates new challenges as\nthe source-side may contain noise originated by the model used in the\nback-translation. Hence, finding n-grams present in the test set become more\ndifficult. Despite that, in our work we show that adapting a model with a\nselection of synthetic data is an useful approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07854v1",
            "date": "2019-06-19 00:13:04+00:00",
            "title": "Surf at MEDIQA 2019: Improving Performance of Natural Language Inference in the Clinical Domain by Adopting Pre-trained Language Model",
            "authors": [
                "Jiin Nam",
                "Seunghyun Yoon",
                "Kyomin Jung"
            ],
            "abstract": "While deep learning techniques have shown promising results in many natural\nlanguage processing (NLP) tasks, it has not been widely applied to the clinical\ndomain. The lack of large datasets and the pervasive use of domain-specific\nlanguage (i.e. abbreviations and acronyms) in the clinical domain causes slower\nprogress in NLP tasks than that of the general NLP tasks. To fill this gap, we\nemploy word/subword-level based models that adopt large-scale data-driven\nmethods such as pre-trained language models and transfer learning in analyzing\ntext for the clinical domain. Empirical results demonstrate the superiority of\nthe proposed methods by achieving 90.6% accuracy in medical domain natural\nlanguage inference task. Furthermore, we inspect the independent strengths of\nthe proposed approaches in quantitative and qualitative manners. This analysis\nwill help researchers to select necessary components in building models for the\nmedical domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07978v2",
            "date": "2019-06-19 08:56:02+00:00",
            "title": "Multilingual Multi-Domain Adaptation Approaches for Neural Machine Translation",
            "authors": [
                "Chenhui Chu",
                "Raj Dabre"
            ],
            "abstract": "In this paper, we propose two novel methods for domain adaptation for the\nattention-only neural machine translation (NMT) model, i.e., the Transformer.\nOur methods focus on training a single translation model for multiple domains\nby either learning domain specialized hidden state representations or predictor\nbiases for each domain. We combine our methods with a previously proposed\nblack-box method called mixed fine tuning, which is known to be highly\neffective for domain adaptation. In addition, we incorporate multilingualism\ninto the domain adaptation framework. Experiments show that multilingual\nmulti-domain adaptation can significantly improve both resource-poor in-domain\nand resource-rich out-of-domain translations, and the combination of our\nmethods with mixed fine tuning achieves the best performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08069v1",
            "date": "2019-06-19 12:39:09+00:00",
            "title": "The Effect of Translationese in Machine Translation Test Sets",
            "authors": [
                "Mike Zhang",
                "Antonio Toral"
            ],
            "abstract": "The effect of translationese has been studied in the field of machine\ntranslation (MT), mostly with respect to training data. We study in depth the\neffect of translationese on test data, using the test sets from the last three\neditions of WMT's news shared task, containing 17 translation directions. We\nshow evidence that (i) the use of translationese in test sets results in\ninflated human evaluation scores for MT systems; (ii) in some cases system\nrankings do change and (iii) the impact translationese has on a translation\ndirection is inversely correlated to the translation quality attainable by\nstate-of-the-art MT systems for that direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08104v1",
            "date": "2019-06-19 14:00:15+00:00",
            "title": "EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing",
            "authors": [
                "Yue Dong",
                "Zichao Li",
                "Mehdi Rezagholizadeh",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "We present the first sentence simplification model that learns explicit edit\noperations (ADD, DELETE, and KEEP) via a neural programmer-interpreter\napproach. Most current neural sentence simplification systems are variants of\nsequence-to-sequence models adopted from machine translation. These methods\nlearn to simplify sentences as a byproduct of the fact that they are trained on\ncomplex-simple sentence pairs. By contrast, our neural programmer-interpreter\nis directly trained to predict explicit edit operations on targeted parts of\nthe input sentence, resembling the way that humans might perform simplification\nand revision. Our model outperforms previous state-of-the-art neural sentence\nsimplification models (without external knowledge) by large margins on three\nbenchmark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89\nWikiSmall, +1.41 Newsela), and is judged by humans to produce overall better\nand simpler output sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08286v1",
            "date": "2019-06-19 18:08:06+00:00",
            "title": "Incorporating Priors with Feature Attribution on Text Classification",
            "authors": [
                "Frederick Liu",
                "Besim Avci"
            ],
            "abstract": "Feature attribution methods, proposed recently, help users interpret the\npredictions of complex models. Our approach integrates feature attributions\ninto the objective function to allow machine learning practitioners to\nincorporate priors in model building. To demonstrate the effectiveness our\ntechnique, we apply it to two tasks: (1) mitigating unintended bias in text\nclassifiers by neutralizing identity terms; (2) improving classifier\nperformance in a scarce data setting by forcing the model to focus on toxic\nterms. Our approach adds an L2 distance loss between feature attributions and\ntask-specific prior values to the objective. Our experiments show that i) a\nclassifier trained with our technique reduces undesired model biases without a\ntrade off on the original task; ii) incorporating priors helps model\nperformance in scarce data settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08287v1",
            "date": "2019-06-19 18:08:33+00:00",
            "title": "Embedding time expressions for deep temporal ordering models",
            "authors": [
                "Tanya Goyal",
                "Greg Durrett"
            ],
            "abstract": "Data-driven models have demonstrated state-of-the-art performance in\ninferring the temporal ordering of events in text. However, these models often\noverlook explicit temporal signals, such as dates and time windows. Rule-based\nmethods can be used to identify the temporal links between these time\nexpressions (timexes), but they fail to capture timexes' interactions with\nevents and are hard to integrate with the distributed representations of neural\nnet models. In this paper, we introduce a framework to infuse temporal\nawareness into such models by learning a pre-trained model to embed timexes. We\ngenerate synthetic data consisting of pairs of timexes, then train a character\nLSTM to learn embeddings and classify the timexes' temporal relation. We\nevaluate the utility of these embeddings in the context of a strong neural\nmodel for event temporal ordering, and show a small increase in performance on\nthe MATRES dataset and more substantial gains on an automatically collected\ndataset with more frequent event-timex interactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08318v4",
            "date": "2019-06-19 19:21:16+00:00",
            "title": "REflex: Flexible Framework for Relation Extraction in Multiple Domains",
            "authors": [
                "Geeticka Chauhan",
                "Matthew B. A. McDermott",
                "Peter Szolovits"
            ],
            "abstract": "Systematic comparison of methods for relation extraction (RE) is difficult\nbecause many experiments in the field are not described precisely enough to be\ncompletely reproducible and many papers fail to report ablation studies that\nwould highlight the relative contributions of their various combined\ntechniques. In this work, we build a unifying framework for RE, applying this\non three highly used datasets (from the general, biomedical and clinical\ndomains) with the ability to be extendable to new datasets. By performing a\nsystematic exploration of modeling, pre-processing and training methodologies,\nwe find that choices of pre-processing are a large contributor performance and\nthat omission of such information can further hinder fair comparison. Other\ninsights from our exploration allow us to provide recommendations for future\nresearch in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08379v1",
            "date": "2019-06-19 21:56:25+00:00",
            "title": "Considerations for the Interpretation of Bias Measures of Word Embeddings",
            "authors": [
                "Inom Mirzaev",
                "Anthony Schulte",
                "Michael Conover",
                "Sam Shah"
            ],
            "abstract": "Word embedding spaces are powerful tools for capturing latent semantic\nrelationships between terms in corpora, and have become widely popular for\nbuilding state-of-the-art natural language processing algorithms. However,\nstudies have shown that societal biases present in text corpora may be\nincorporated into the word embedding spaces learned from them. Thus, there is\nan ethical concern that human-like biases contained in the corpora and their\nderived embedding spaces might be propagated, or even amplified with the usage\nof the biased embedding spaces in downstream applications. In an attempt to\nquantify these biases so that they may be better understood and studied,\nseveral bias metrics have been proposed. We explore the statistical properties\nof these proposed measures in the context of their cited applications as well\nas their supposed utilities. We find that there are caveats to the simple\ninterpretation of these metrics as proposed. We find that the bias metric\nproposed by Bolukbasi et al. 2016 is highly sensitive to embedding\nhyper-parameter selection, and that in many cases, the variance due to the\nselection of some hyper-parameters is greater than the variance in the metric\ndue to corpus selection, while in fewer cases the bias rankings of corpora vary\nwith hyper-parameter selection. In light of these observations, it may be the\ncase that bias estimates should not be thought to directly measure the\nproperties of the underlying corpus, but rather the properties of the specific\nembedding spaces in question, particularly in the context of hyper-parameter\nselections used to generate them. Hence, bias metrics of spaces generated with\ndiffering hyper-parameters should be compared only with explicit consideration\nof the embedding-learning algorithms particular configurations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08393v2",
            "date": "2019-06-19 23:20:57+00:00",
            "title": "Robust Machine Translation with Domain Sensitive Pseudo-Sources: Baidu-OSU WMT19 MT Robustness Shared Task System Report",
            "authors": [
                "Renjie Zheng",
                "Hairong Liu",
                "Mingbo Ma",
                "Baigong Zheng",
                "Liang Huang"
            ],
            "abstract": "This paper describes the machine translation system developed jointly by\nBaidu Research and Oregon State University for WMT 2019 Machine Translation\nRobustness Shared Task. Translation of social media is a very challenging\nproblem, since its style is very different from normal parallel corpora (e.g.\nNews) and also include various types of noises. To make it worse, the amount of\nsocial media parallel corpora is extremely limited. In this paper, we use a\ndomain sensitive training method which leverages a large amount of parallel\ndata from popular domains together with a little amount of parallel data from\nsocial media. Furthermore, we generate a parallel dataset with pseudo noisy\nsource sentences which are back-translated from monolingual data using a model\ntrained by a similar domain sensitive way. We achieve more than 10 BLEU\nimprovement in both En-Fr and Fr-En translation compared with the baseline\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08401v2",
            "date": "2019-06-20 00:59:22+00:00",
            "title": "Hierarchical Document Encoder for Parallel Corpus Mining",
            "authors": [
                "Mandy Guo",
                "Yinfei Yang",
                "Keith Stevens",
                "Daniel Cer",
                "Heming Ge",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "We explore using multilingual document embeddings for nearest neighbor mining\nof parallel data. Three document-level representations are investigated: (i)\ndocument embeddings generated by simply averaging multilingual sentence\nembeddings; (ii) a neural bag-of-words (BoW) document encoding model; (iii) a\nhierarchical multilingual document encoder (HiDE) that builds on our\nsentence-level model. The results show document embeddings derived from\nsentence-level averaging are surprisingly effective for clean datasets, but\nsuggest models trained hierarchically at the document-level are more effective\non noisy data. Analysis experiments demonstrate our hierarchical models are\nvery robust to variations in the underlying sentence embedding quality. Using\ndocument embeddings trained with HiDE achieves state-of-the-art performance on\nUnited Nations (UN) parallel document mining, 94.9% P@1 for en-fr and 97.3% P@1\nfor en-es.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08449v1",
            "date": "2019-06-20 05:33:30+00:00",
            "title": "Multi-Grained Named Entity Recognition",
            "authors": [
                "Congying Xia",
                "Chenwei Zhang",
                "Tao Yang",
                "Yaliang Li",
                "Nan Du",
                "Xian Wu",
                "Wei Fan",
                "Fenglong Ma",
                "Philip Yu"
            ],
            "abstract": "This paper presents a novel framework, MGNER, for Multi-Grained Named Entity\nRecognition where multiple entities or entity mentions in a sentence could be\nnon-overlapping or totally nested. Different from traditional approaches\nregarding NER as a sequential labeling task and annotate entities\nconsecutively, MGNER detects and recognizes entities on multiple granularities:\nit is able to recognize named entities without explicitly assuming\nnon-overlapping or totally nested structures. MGNER consists of a Detector that\nexamines all possible word segments and a Classifier that categorizes entities.\nIn addition, contextual information and a self-attention mechanism are utilized\nthroughout the framework to improve the NER performance. Experimental results\nshow that MGNER outperforms current state-of-the-art baselines up to 4.4% in\nterms of the F1 score among nested/non-overlapping NER tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08570v1",
            "date": "2019-06-20 12:05:13+00:00",
            "title": "Hindi Question Generation Using Dependency Structures",
            "authors": [
                "Kaveri Anuranjana",
                "Vijjini Anvesh Rao",
                "Radhika Mamidi"
            ],
            "abstract": "Hindi question answering systems suffer from a lack of data. To address the\nsame, this paper presents an approach towards automatic question generation. We\npresent a rule-based system for question generation in Hindi by formalizing\nquestion transformation methods based on karaka-dependency theory. We use a\nHindi dependency parser to mark the karaka roles and use IndoWordNet a Hindi\nontology to detect the semantic category of the karaka role heads to generate\nthe interrogatives. We analyze how one sentence can have multiple generations\nfrom the same karaka role's rule. The generations are manually annotated by\nmultiple annotators on a semantic and syntactic scale for evaluation. Further,\nwe constrain our generation with the help of various semantic and syntactic\nfilters so as to improve the generation quality. Using these methods, we are\nable to generate diverse questions, significantly more than number of sentences\nfed to the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08584v1",
            "date": "2019-06-20 12:49:17+00:00",
            "title": "Improving Zero-shot Translation with Language-Independent Constraints",
            "authors": [
                "Ngoc-Quan Pham",
                "Jan Niehues",
                "Thanh-Le Ha",
                "Alex Waibel"
            ],
            "abstract": "An important concern in training multilingual neural machine translation\n(NMT) is to translate between language pairs unseen during training, i.e\nzero-shot translation. Improving this ability kills two birds with one stone by\nproviding an alternative to pivot translation which also allows us to better\nunderstand how the model captures information between languages.\n  In this work, we carried out an investigation on this capability of the\nmultilingual NMT models. First, we intentionally create an encoder architecture\nwhich is independent with respect to the source language. Such experiments shed\nlight on the ability of NMT encoders to learn multilingual representations, in\ngeneral. Based on such proof of concept, we were able to design regularization\nmethods into the standard Transformer model, so that the whole architecture\nbecomes more robust in zero-shot conditions. We investigated the behaviour of\nsuch models on the standard IWSLT 2017 multilingual dataset. We achieved an\naverage improvement of 2.23 BLEU points across 12 language pairs compared to\nthe zero-shot performance of a state-of-the-art multilingual system.\nAdditionally, we carry out further experiments in which the effect is confirmed\neven for language pairs with multiple intermediate pivots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08593v2",
            "date": "2019-06-20 13:16:37+00:00",
            "title": "Conflict as an Inverse of Attention in Sequence Relationship",
            "authors": [
                "Rajarshee Mitra"
            ],
            "abstract": "Attention is a very efficient way to model the relationship between two\nsequences by comparing how similar two intermediate representations are.\nInitially demonstrated in NMT, it is a standard in all NLU tasks today when\nefficient interaction between sequences is considered. However, we show that\nattention, by virtue of its composition, works best only when it is given that\nthere is a match somewhere between two sequences. It does not very well adapt\nto cases when there is no similarity between two sequences or if the\nrelationship is contrastive. We propose an Conflict model which is very similar\nto how attention works but which emphasizes mostly on how well two sequences\nrepel each other and finally empirically show how this method in conjunction\nwith attention can boost the overall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08646v1",
            "date": "2019-06-19 11:04:51+00:00",
            "title": "Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction",
            "authors": [
                "Christoph Alt",
                "Marc H\u00fcbner",
                "Leonhard Hennig"
            ],
            "abstract": "Distantly supervised relation extraction is widely used to extract relational\nfacts from text, but suffers from noisy labels. Current relation extraction\nmethods try to alleviate the noise by multi-instance learning and by providing\nsupporting linguistic and contextual information to more efficiently guide the\nrelation classification. While achieving state-of-the-art results, we observed\nthese models to be biased towards recognizing a limited set of relations with\nhigh precision, while ignoring those in the long tail. To address this gap, we\nutilize a pre-trained language model, the OpenAI Generative Pre-trained\nTransformer (GPT) [Radford et al., 2018]. The GPT and similar models have been\nshown to capture semantic and syntactic features, and also a notable amount of\n\"common-sense\" knowledge, which we hypothesize are important features for\nrecognizing a more diverse set of relations. By extending the GPT to the\ndistantly supervised setting, and fine-tuning it on the NYT10 dataset, we show\nthat it predicts a larger set of distinct relation types with high confidence.\nManual and automated evaluation of our model shows that it achieves a\nstate-of-the-art AUC score of 0.422 on the NYT10 dataset, and performs\nespecially well at higher recall levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08717v1",
            "date": "2019-06-20 16:04:38+00:00",
            "title": "A New Statistical Approach for Comparing Algorithms for Lexicon Based Sentiment Analysis",
            "authors": [
                "Mateus Machado",
                "Evandro Ruiz",
                "Kuruvilla Joseph Abraham"
            ],
            "abstract": "Lexicon based sentiment analysis usually relies on the identification of\nvarious words to which a numerical value corresponding to sentiment can be\nassigned. In principle, classifiers can be obtained from these algorithms by\ncomparison with human annotation, which is considered the gold standard. In\npractise this is difficult in languages such as Portuguese where there is a\npaucity of human annotated texts. Thus in order to compare algorithms, a next\nbest step is to directly compare different algorithms with each other without\nreferring to human annotation. In this paper we develop methods for a\nstatistical comparison of algorithms which does not rely on human annotation or\non known class labels. We will motivate the use of marginal homogeneity tests,\nas well as log linear models within the framework of maximum likelihood\nestimation We will also show how some uncertainties present in lexicon based\nsentiment analysis may be similar to those which occur in human annotated\ntweets. We will also show how the variability in the output of different\nalgorithms is lexicon dependent, and quantify this variability in the output\nwithin the framework of log linear models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08885v1",
            "date": "2019-06-20 22:39:44+00:00",
            "title": "Low-Resource Corpus Filtering using Multilingual Sentence Embeddings",
            "authors": [
                "Vishrav Chaudhary",
                "Yuqing Tang",
                "Francisco Guzm\u00e1n",
                "Holger Schwenk",
                "Philipp Koehn"
            ],
            "abstract": "In this paper, we describe our submission to the WMT19 low-resource parallel\ncorpus filtering shared task. Our main approach is based on the LASER toolkit\n(Language-Agnostic SEntence Representations), which uses an encoder-decoder\narchitecture trained on a parallel corpus to obtain multilingual sentence\nrepresentations. We then use the representations directly to score and filter\nthe noisy parallel sentences without additionally training a scoring function.\nWe contrast our approach to other promising methods and show that LASER yields\nstrong results. Finally, we produce an ensemble of different scoring methods\nand obtain additional gains. Our submission achieved the best overall\nperformance for both the Nepali-English and Sinhala-English 1M tasks by a\nmargin of 1.3 and 1.4 BLEU respectively, as compared to the second best\nsystems. Moreover, our experiments show that this technique is promising for\nlow and even no-resource scenarios.",
            "journal ref": "Conference on Machine Translation (WMT) 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08931v1",
            "date": "2019-06-21 03:25:30+00:00",
            "title": "Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data",
            "authors": [
                "Wei Ye",
                "Bo Li",
                "Rui Xie",
                "Zhonghao Sheng",
                "Long Chen",
                "Shikun Zhang"
            ],
            "abstract": "In practical scenario, relation extraction needs to first identify entity\npairs that have relation and then assign a correct relation class. However, the\nnumber of non-relation entity pairs in context (negative instances) usually far\nexceeds the others (positive instances), which negatively affects a model's\nperformance. To mitigate this problem, we propose a multi-task architecture\nwhich jointly trains a model to perform relation identification with\ncross-entropy loss and relation classification with ranking loss. Meanwhile, we\nobserve that a sentence may have multiple entities and relation mentions, and\nthe patterns in which the entities appear in a sentence may contain useful\nsemantic information that can be utilized to distinguish between positive and\nnegative instances. Thus we further incorporate the embeddings of\ncharacter-wise/word-wise BIO tag from the named entity recognition task into\ncharacter/word embeddings to enrich the input representation. Experiment\nresults show that our proposed approach can significantly improve the\nperformance of a baseline model with more than 10% absolute increase in\nF1-score, and outperform the state-of-the-art models on ACE 2005 Chinese and\nEnglish corpus. Moreover, BIO tag embeddings are particularly effective and can\nbe used to improve other models as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08972v1",
            "date": "2019-06-21 06:27:17+00:00",
            "title": "A Deep Generative Model for Code-Switched Text",
            "authors": [
                "Bidisha Samanta",
                "Sharmila Reddy",
                "Hussain Jagirdar",
                "Niloy Ganguly",
                "Soumen Chakrabarti"
            ],
            "abstract": "Code-switching, the interleaving of two or more languages within a sentence\nor discourse is pervasive in multilingual societies. Accurate language models\nfor code-switched text are critical for NLP tasks. State-of-the-art\ndata-intensive neural language models are difficult to train well from scarce\nlanguage-labeled code-switched text. A potential solution is to use deep\ngenerative models to synthesize large volumes of realistic code-switched text.\nAlthough generative adversarial networks and variational autoencoders can\nsynthesize plausible monolingual text from continuous latent space, they cannot\nadequately address code-switched text, owing to their informal style and\ncomplex interplay between the constituent languages. We introduce VACS, a novel\nvariational autoencoder architecture specifically tailored to code-switching\nphenomena. VACS encodes to and decodes from a two-level hierarchical\nrepresentation, which models syntactic contextual signals in the lower level,\nand language switching signals in the upper layer. Sampling representations\nfrom the prior and decoding them produced well-formed, diverse code-switched\nsentences. Extensive experiments show that using synthetic code-switched text\nwith natural monolingual data results in significant (33.06%) drop in\nperplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08976v1",
            "date": "2019-06-21 06:39:11+00:00",
            "title": "Mitigating Gender Bias in Natural Language Processing: Literature Review",
            "authors": [
                "Tony Sun",
                "Andrew Gaut",
                "Shirlyn Tang",
                "Yuxin Huang",
                "Mai ElSherief",
                "Jieyu Zhao",
                "Diba Mirza",
                "Elizabeth Belding",
                "Kai-Wei Chang",
                "William Yang Wang"
            ],
            "abstract": "As Natural Language Processing (NLP) and Machine Learning (ML) tools rise in\npopularity, it becomes increasingly vital to recognize the role they play in\nshaping societal biases and stereotypes. Although NLP models have shown success\nin modeling various applications, they propagate and may even amplify gender\nbias found in text corpora. While the study of bias in artificial intelligence\nis not new, methods to mitigate gender bias in NLP are relatively nascent. In\nthis paper, we review contemporary studies on recognizing and mitigating gender\nbias in NLP. We discuss gender bias based on four forms of representation bias\nand analyze methods recognizing gender bias. Furthermore, we discuss the\nadvantages and drawbacks of existing gender debiasing methods. Finally, we\ndiscuss future studies for recognizing and mitigating gender bias in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08996v1",
            "date": "2019-06-21 08:10:05+00:00",
            "title": "Incremental Adaptation of NMT for Professional Post-editors: A User Study",
            "authors": [
                "Miguel Domingo",
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "\u00c1lvaro Peris",
                "Alexandre Helle",
                "Amando Estela",
                "Laurent Bi\u00e9",
                "Francisco Casacuberta",
                "Manuel Herranz"
            ],
            "abstract": "A common use of machine translation in the industry is providing initial\ntranslation hypotheses, which are later supervised and post-edited by a human\nexpert. During this revision process, new bilingual data are continuously\ngenerated. Machine translation systems can benefit from these new data,\nincrementally updating the underlying models under an online learning paradigm.\nWe conducted a user study on this scenario, for a neural machine translation\nsystem. The experimentation was carried out by professional translators, with a\nvast experience in machine translation post-editing. The results showed a\nreduction in the required amount of human effort needed when post-editing the\noutputs of the system, improvements in the translation quality and a positive\nperception of the adaptive system by the users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09000v1",
            "date": "2019-06-21 08:19:49+00:00",
            "title": "Demonstration of a Neural Machine Translation System with Online Learning for Translators",
            "authors": [
                "Miguel Domingo",
                "Mercedes Garc\u00eda-Mart\u00ednez",
                "Amando Estela",
                "Laurent Bi\u00e9",
                "Alexandre Helle",
                "\u00c1lvaro Peris",
                "Francisco Casacuberta",
                "Manuerl Herranz"
            ],
            "abstract": "We introduce a demonstration of our system, which implements online learning\nfor neural machine translation in a production environment. These techniques\nallow the system to continuously learn from the corrections provided by the\ntranslators. We implemented an end-to-end platform integrating our machine\ntranslation servers to one of the most common user interfaces for professional\ntranslators: SDL Trados Studio. Our objective was to save post-editing effort\nas the machine is continuously learning from human choices and adapting the\nmodels to a specific domain or user style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09246v1",
            "date": "2019-06-21 17:11:46+00:00",
            "title": "CUNI System for the WMT19 Robustness Task",
            "authors": [
                "Jind\u0159ich Helcl",
                "Jind\u0159ich Libovick\u00fd",
                "Martin Popel"
            ],
            "abstract": "We present our submission to the WMT19 Robustness Task. Our baseline system\nis the Charles University (CUNI) Transformer system trained for the WMT18\nshared task on News Translation. Quantitative results show that the CUNI\nTransformer system is already far more robust to noisy input than the\nLSTM-based baseline provided by the task organizers. We further improved the\nperformance of our model by fine-tuning on the in-domain noisy data without\ninfluencing the translation quality on the news domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09285v1",
            "date": "2019-06-21 18:23:17+00:00",
            "title": "SurfCon: Synonym Discovery on Privacy-Aware Clinical Data",
            "authors": [
                "Zhen Wang",
                "Xiang Yue",
                "Soheil Moosavinasab",
                "Yungui Huang",
                "Simon Lin",
                "Huan Sun"
            ],
            "abstract": "Unstructured clinical texts contain rich health-related information. To\nbetter utilize the knowledge buried in clinical texts, discovering synonyms for\na medical query term has become an important task. Recent automatic synonym\ndiscovery methods leveraging raw text information have been developed. However,\nto preserve patient privacy and security, it is usually quite difficult to get\naccess to large-scale raw clinical texts. In this paper, we study a new setting\nnamed synonym discovery on privacy-aware clinical data (i.e., medical terms\nextracted from the clinical texts and their aggregated co-occurrence counts,\nwithout raw clinical texts). To solve the problem, we propose a new framework\nSurfCon that leverages two important types of information in the privacy-aware\nclinical data, i.e., the surface form information, and the global context\ninformation for synonym discovery. In particular, the surface form module\nenables us to detect synonyms that look similar while the global context module\nplays a complementary role to discover synonyms that are semantically similar\nbut in different surface forms, and both allow us to deal with the OOV query\nissue (i.e., when the query is not found in the given data). We conduct\nextensive experiments and case studies on publicly available privacy-aware\nclinical data, and show that SurfCon can outperform strong baseline methods by\nlarge margins under various settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09302v1",
            "date": "2019-06-21 19:34:19+00:00",
            "title": "Neural Machine Translating from Natural Language to SPARQL",
            "authors": [
                "Xiaoyu Yin",
                "Dagmar Gromann",
                "Sebastian Rudolph"
            ],
            "abstract": "SPARQL is a highly powerful query language for an ever-growing number of\nLinked Data resources and Knowledge Graphs. Using it requires a certain\nfamiliarity with the entities in the domain to be queried as well as expertise\nin the language's syntax and semantics, none of which average human web users\ncan be assumed to possess. To overcome this limitation, automatically\ntranslating natural language questions to SPARQL queries has been a vibrant\nfield of research. However, to this date, the vast success of deep learning\nmethods has not yet been fully propagated to this research problem. This paper\ncontributes to filling this gap by evaluating the utilization of eight\ndifferent Neural Machine Translation (NMT) models for the task of translating\nfrom natural language to the structured query language SPARQL. While\nhighlighting the importance of high-quantity and high-quality datasets, the\nresults show a dominance of a CNN-based architecture with a BLEU score of up to\n98 and accuracy of up to 94%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09320v1",
            "date": "2019-06-20 11:09:12+00:00",
            "title": "Neural Collective Entity Linking Based on Recurrent Random Walk Network Learning",
            "authors": [
                "Mengge Xue",
                "Weiming Cai",
                "Jinsong Su",
                "Linfeng Song",
                "Yubin Ge",
                "Yubao Liu",
                "Bin Wang"
            ],
            "abstract": "Benefiting from the excellent ability of neural networks on learning semantic\nrepresentations, existing studies for entity linking (EL) have resorted to\nneural networks to exploit both the local mention-to-entity compatibility and\nthe global interdependence between different EL decisions for target entity\ndisambiguation. However, most neural collective EL methods depend entirely upon\nneural networks to automatically model the semantic dependencies between\ndifferent EL decisions, which lack of the guidance from external knowledge. In\nthis paper, we propose a novel end-to-end neural network with recurrent\nrandom-walk layers for collective EL, which introduces external knowledge to\nmodel the semantic interdependence between different EL decisions.\nSpecifically, we first establish a model based on local context features, and\nthen stack random-walk layers to reinforce the evidence for related EL\ndecisions into high-probability decisions, where the semantic interdependence\nbetween candidate entities is mainly induced from an external knowledge base.\nFinally, a semantic regularizer that preserves the collective EL decisions\nconsistency is incorporated into the conventional objective function, so that\nthe external knowledge base can be fully exploited in collective EL decisions.\nExperimental results and in-depth analysis on various datasets show that our\nmodel achieves better performance than other state-of-the-art models. Our code\nand data are released at \\url{https://github.com/DeepLearnXMU/RRWEL}.",
            "journal ref": "IJCAI 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09379v1",
            "date": "2019-06-22 03:24:32+00:00",
            "title": "Evaluating Computational Language Models with Scaling Properties of Natural Language",
            "authors": [
                "Shuntaro Takahashi",
                "Kumiko Tanaka-Ishii"
            ],
            "abstract": "In this article, we evaluate computational models of natural language with\nrespect to the universal statistical behaviors of natural language. Statistical\nmechanical analyses have revealed that natural language text is characterized\nby scaling properties, which quantify the global structure in the vocabulary\npopulation and the long memory of a text. We study whether five scaling\nproperties (given by Zipf's law, Heaps' law, Ebeling's method, Taylor's law,\nand long-range correlation analysis) can serve for evaluation of computational\nmodels. Specifically, we test $n$-gram language models, a probabilistic\ncontext-free grammar (PCFG), language models based on Simon/Pitman-Yor\nprocesses, neural language models, and generative adversarial networks (GANs)\nfor text generation. Our analysis reveals that language models based on\nrecurrent neural networks (RNNs) with a gating mechanism (i.e., long short-term\nmemory, LSTM; a gated recurrent unit, GRU; and quasi-recurrent neural networks,\nQRNNs) are the only computational models that can reproduce the long memory\nbehavior of natural language. Furthermore, through comparison with recently\nproposed model-based evaluation methods, we find that the exponent of Taylor's\nlaw is a good indicator of model quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09532v1",
            "date": "2019-06-23 02:00:40+00:00",
            "title": "Smaller Text Classifiers with Discriminative Cluster Embeddings",
            "authors": [
                "Mingda Chen",
                "Kevin Gimpel"
            ],
            "abstract": "Word embedding parameters often dominate overall model sizes in neural\nmethods for natural language processing. We reduce deployed model sizes of text\nclassifiers by learning a hard word clustering in an end-to-end manner. We use\nthe Gumbel-Softmax distribution to maximize over the latent clustering while\nminimizing the task loss. We propose variations that selectively assign\nadditional parameters to words, which further improves accuracy while still\nremaining parameter-efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09535v1",
            "date": "2019-06-23 02:17:46+00:00",
            "title": "Variational Sequential Labelers for Semi-Supervised Learning",
            "authors": [
                "Mingda Chen",
                "Qingming Tang",
                "Karen Livescu",
                "Kevin Gimpel"
            ],
            "abstract": "We introduce a family of multitask variational methods for semi-supervised\nsequence labeling. Our model family consists of a latent-variable generative\nmodel and a discriminative labeler. The generative models use latent variables\nto define the conditional probability of a word given its context, drawing\ninspiration from word prediction objectives commonly used in learning word\nembeddings. The labeler helps inject discriminative information into the latent\nspace. We explore several latent variable configurations, including ones with\nhierarchical structure, which enables the model to account for both\nlabel-specific and word-specific information. Our models consistently\noutperform standard sequential baselines on 8 sequence labeling datasets, and\nimprove further with unlabeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09556v1",
            "date": "2019-06-23 05:28:03+00:00",
            "title": "DAL: Dual Adversarial Learning for Dialogue Generation",
            "authors": [
                "Shaobo Cui",
                "Rongzhong Lian",
                "Di Jiang",
                "Yuanfeng Song",
                "Siqi Bao",
                "Yong Jiang"
            ],
            "abstract": "In open-domain dialogue systems, generative approaches have attracted much\nattention for response generation. However, existing methods are heavily\nplagued by generating safe responses and unnatural responses. To alleviate\nthese two problems, we propose a novel framework named Dual Adversarial\nLearning (DAL) for high-quality response generation. DAL is the first work to\ninnovatively utilizes the duality between query generation and response\ngeneration to avoid safe responses and increase the diversity of the generated\nresponses. Additionally, DAL uses adversarial learning to mimic human judges\nand guides the system to generate natural responses. Experimental results\ndemonstrate that DAL effectively improves both diversity and overall quality of\nthe generated responses. DAL outperforms the state-of-the-art methods regarding\nautomatic metrics and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09675v1",
            "date": "2019-06-24 00:25:18+00:00",
            "title": "Evaluating the Supervised and Zero-shot Performance of Multi-lingual Translation Models",
            "authors": [
                "Chris Hokamp",
                "John Glover",
                "Demian Gholipour"
            ],
            "abstract": "We study several methods for full or partial sharing of the decoder\nparameters of multilingual NMT models. We evaluate both fully supervised and\nzero-shot translation performance in 110 unique translation directions using\nonly the WMT 2019 shared task parallel datasets for training. We use additional\ntest sets and re-purpose evaluation methods recently used for unsupervised MT\nin order to evaluate zero-shot translation performance for language pairs where\nno gold-standard parallel data is available. To our knowledge, this is the\nlargest evaluation of multi-lingual translation yet conducted in terms of the\ntotal size of the training data we use, and in terms of the diversity of\nzero-shot translation pairs we evaluate. We conduct an in-depth evaluation of\nthe translation performance of different models, highlighting the trade-offs\nbetween methods of sharing decoder parameters. We find that models which have\ntask-specific decoder parameters outperform models where decoder parameters are\nfully shared across all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09719v1",
            "date": "2019-06-24 04:46:10+00:00",
            "title": "On the Definition of Japanese Word",
            "authors": [
                "Yugo Murawaki"
            ],
            "abstract": "The annotation guidelines for Universal Dependencies (UD) stipulate that the\nbasic units of dependency annotation are syntactic words, but it is not clear\nwhat are syntactic words in Japanese. Departing from the long tradition of\nusing phrasal units called bunsetsu for dependency parsing, the current UD\nJapanese treebanks adopt the Short Unit Words. However, we argue that they are\nnot syntactic word as specified by the annotation guidelines. Although we find\nnon-mainstream attempts to linguistically define Japanese words, such\ndefinitions have never been applied to corpus annotation. We discuss the costs\nand benefits of adopting the rather unfamiliar criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09741v1",
            "date": "2019-06-24 06:35:36+00:00",
            "title": "Decomposable Neural Paraphrase Generation",
            "authors": [
                "Zichao Li",
                "Xin Jiang",
                "Lifeng Shang",
                "Qun Liu"
            ],
            "abstract": "Paraphrasing exists at different granularity levels, such as lexical level,\nphrasal level and sentential level. This paper presents Decomposable Neural\nParaphrase Generator (DNPG), a Transformer-based model that can learn and\ngenerate paraphrases of a sentence at different levels of granularity in a\ndisentangled way. Specifically, the model is composed of multiple encoders and\ndecoders with different structures, each of which corresponds to a specific\ngranularity. The empirical study shows that the decomposition mechanism of DNPG\nmakes paraphrase generation more interpretable and controllable. Based on DNPG,\nwe further develop an unsupervised domain adaptation method for paraphrase\ngeneration. Experimental results show that the proposed model achieves\ncompetitive in-domain performance compared to the state-of-the-art neural\nmodels, and significantly better performance when adapting to a new domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09774v1",
            "date": "2019-06-24 08:20:10+00:00",
            "title": "Emotionally-Aware Chatbots: A Survey",
            "authors": [
                "Endang Wahyu Pamungkas"
            ],
            "abstract": "Textual conversational agent or chatbots' development gather tremendous\ntraction from both academia and industries in recent years. Nowadays, chatbots\nare widely used as an agent to communicate with a human in some services such\nas booking assistant, customer service, and also a personal partner. The\nbiggest challenge in building chatbot is to build a humanizing machine to\nimprove user engagement. Some studies show that emotion is an important aspect\nto humanize machine, including chatbot. In this paper, we will provide a\nsystematic review of approaches in building an emotionally-aware chatbot (EAC).\nAs far as our knowledge, there is still no work focusing on this area. We\npropose three research question regarding EAC studies. We start with the\nhistory and evolution of EAC, then several approaches to build EAC by previous\nstudies, and some available resources in building EAC. Based on our\ninvestigation, we found that in the early development, EAC exploits a simple\nrule-based approach while now most of EAC use neural-based approach. We also\nnotice that most of EAC contain emotion classifier in their architecture, which\nutilize several available affective resources. We also predict that the\ndevelopment of EAC will continue to gain more and more attention from scholars,\nnoted by some recent studies propose new datasets for building EAC in various\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09795v1",
            "date": "2019-06-24 09:08:29+00:00",
            "title": "Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding",
            "authors": [
                "Shohei Tanaka",
                "Koichiro Yoshino",
                "Katsuhito Sudoh",
                "Satoshi Nakamura"
            ],
            "abstract": "We propose a novel method for selecting coherent and diverse responses for a\ngiven dialogue context. The proposed method re-ranks response candidates\ngenerated from conversational models by using event causality relations between\nevents in a dialogue history and response candidates (e.g., ``be stressed out''\nprecedes ``relieve stress''). We use distributed event representation based on\nthe Role Factored Tensor Model for a robust matching of event causality\nrelations due to limited event causality knowledge of the system. Experimental\nresults showed that the proposed method improved coherency and dialogue\ncontinuity of system responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09821v1",
            "date": "2019-06-24 09:55:21+00:00",
            "title": "Classification and Clustering of Arguments with Contextualized Word Embeddings",
            "authors": [
                "Nils Reimers",
                "Benjamin Schiller",
                "Tilman Beck",
                "Johannes Daxenberger",
                "Christian Stab",
                "Iryna Gurevych"
            ],
            "abstract": "We experiment with two recent contextualized word embedding methods (ELMo and\nBERT) in the context of open-domain argument search. For the first time, we\nshow how to leverage the power of contextualized word embeddings to classify\nand cluster topic-dependent arguments, achieving impressive results on both\ntasks and across multiple datasets. For argument classification, we improve the\nstate-of-the-art for the UKP Sentential Argument Mining Corpus by 20.8\npercentage points and for the IBM Debater - Evidence Sentences dataset by 7.4\npercentage points. For the understudied task of argument clustering, we propose\na pre-training step which improves by 7.8 percentage points over strong\nbaselines on a novel dataset, and by 12.3 percentage points for the Argument\nFacet Similarity (AFS) Corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09912v1",
            "date": "2019-06-17 06:32:24+00:00",
            "title": "KaWAT: A Word Analogy Task Dataset for Indonesian",
            "authors": [
                "Kemal Kurniawan"
            ],
            "abstract": "We introduced KaWAT (Kata Word Analogy Task), a new word analogy task dataset\nfor Indonesian. We evaluated on it several existing pretrained Indonesian word\nembeddings and embeddings trained on Indonesian online news corpus. We also\ntested them on two downstream tasks and found that pretrained word embeddings\nhelped either by reducing the training epochs or yielding significant\nperformance gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09978v1",
            "date": "2019-06-21 13:03:47+00:00",
            "title": "Multilingual Named Entity Recognition Using Pretrained Embeddings, Attention Mechanism and NCRF",
            "authors": [
                "Anton A. Emelyanov",
                "Ekaterina Artemova"
            ],
            "abstract": "In this paper we tackle multilingual named entity recognition task. We use\nthe BERT Language Model as embeddings with bidirectional recurrent network,\nattention, and NCRF on the top. We apply multilingual BERT only as embedder\nwithout any fine-tuning. We test out model on the dataset of the BSNLP shared\ntask, which consists of texts in Bulgarian, Czech, Polish and Russian\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10007v1",
            "date": "2019-06-24 14:59:12+00:00",
            "title": "Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation",
            "authors": [
                "Daniel Loureiro",
                "Alipio Jorge"
            ],
            "abstract": "Contextual embeddings represent a new generation of semantic representations\nlearned from Neural Language Modelling (NLM) that addresses the issue of\nmeaning conflation hampering traditional word embeddings. In this work, we show\nthat contextual embeddings can be used to achieve unprecedented gains in Word\nSense Disambiguation (WSD) tasks. Our approach focuses on creating sense-level\nembeddings with full-coverage of WordNet, and without recourse to explicit\nknowledge of sense distributions or task-specific modelling. As a result, a\nsimple Nearest Neighbors (k-NN) method using our representations is able to\nconsistently surpass the performance of previous systems using powerful neural\nsequencing models. We also analyse the robustness of our approach when ignoring\npart-of-speech and lemma features, requiring disambiguation against the full\nsense inventory, and revealing shortcomings to be improved. Finally, we explore\napplications of our sense embeddings for concept-level analyses of contextual\nembeddings and their respective NLMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10256v2",
            "date": "2019-06-24 22:31:33+00:00",
            "title": "Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis",
            "authors": [
                "Jayadev Bhaskaran",
                "Isha Bhallamudi"
            ],
            "abstract": "In this work, we investigate the presence of occupational gender stereotypes\nin sentiment analysis models. Such a task has implications for reducing\nimplicit biases in these models, which are being applied to an increasingly\nwide variety of downstream tasks. We release a new gender-balanced dataset of\n800 sentences pertaining to specific professions and propose a methodology for\nusing it as a test bench to evaluate sentiment analysis models. We evaluate the\npresence of occupational gender stereotypes in 3 different models using our\napproach, and explore their relationship with societal perceptions of\noccupations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10282v2",
            "date": "2019-06-25 00:43:32+00:00",
            "title": "Saliency-driven Word Alignment Interpretation for Neural Machine Translation",
            "authors": [
                "Shuoyang Ding",
                "Hainan Xu",
                "Philipp Koehn"
            ],
            "abstract": "Despite their original goal to jointly learn to align and translate, Neural\nMachine Translation (NMT) models, especially Transformer, are often perceived\nas not learning interpretable word alignments. In this paper, we show that NMT\nmodels do learn interpretable word alignments, which could only be revealed\nwith proper interpretation methods. We propose a series of such methods that\nare model-agnostic, are able to be applied either offline or online, and do not\nrequire parameter update or architectural change. We show that under the force\ndecoding setup, the alignments induced by our interpretation method are of\nbetter quality than fast-align for some systems, and when performing free\ndecoding, they agree well with the alignments induced by automatic alignment\ntools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10519v1",
            "date": "2019-06-24 08:18:12+00:00",
            "title": "Embedding Projection for Targeted Cross-Lingual Sentiment: Model Comparisons and a Real-World Study",
            "authors": [
                "Jeremy Barnes",
                "Roman Klinger"
            ],
            "abstract": "Sentiment analysis benefits from large, hand-annotated resources in order to\ntrain and test machine learning models, which are often data hungry. While some\nlanguages, e.g., English, have a vast array of these resources, most\nunder-resourced languages do not, especially for fine-grained sentiment tasks,\nsuch as aspect-level or targeted sentiment analysis. To improve this situation,\nwe propose a cross-lingual approach to sentiment analysis that is applicable to\nunder-resourced languages and takes into account target-level information. This\nmodel incorporates sentiment information into bilingual distributional\nrepresentations, by jointly optimizing them for semantics and sentiment,\nshowing state-of-the-art performance at sentence-level when combined with\nmachine translation. The adaptation to targeted sentiment analysis on multiple\ndomains shows that our model outperforms other projection-based bilingual\nembedding methods on binary targeted sentiment tasks. Our analysis on ten\nlanguages demonstrates that the amount of unlabeled monolingual data has\nsurprisingly little effect on the sentiment results. As expected, the choice of\nannotated source language for projection to a target leads to better results\nfor source-target language pairs which are similar. Therefore, our results\nsuggest that more efforts should be spent on the creation of resources for less\nsimilar languages to those which are resource-rich already. Finally, a domain\nmismatch leads to a decreased performance. This suggests resources in any\nlanguage should ideally cover varieties of domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10724v3",
            "date": "2019-06-25 18:56:36+00:00",
            "title": "Model-based annotation of coreference",
            "authors": [
                "Rahul Aralikatte",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Humans do not make inferences over texts, but over models of what texts are\nabout. When annotators are asked to annotate coreferent spans of text, it is\ntherefore a somewhat unnatural task. This paper presents an alternative in\nwhich we preprocess documents, linking entities to a knowledge base, and turn\nthe coreference annotation task -- in our case limited to pronouns -- into an\nannotation task where annotators are asked to assign pronouns to entities.\nModel-based annotation is shown to lead to faster annotation and higher\ninter-annotator agreement, and we argue that it also opens up for an\nalternative approach to coreference resolution. We present two new coreference\nbenchmark datasets, for English Wikipedia and English teacher-student\ndialogues, and evaluate state-of-the-art coreference resolvers on them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10907v1",
            "date": "2019-06-26 08:28:51+00:00",
            "title": "Leveraging Text Repetitions and Denoising Autoencoders in OCR Post-correction",
            "authors": [
                "Kai Hakala",
                "Aleksi Vesanto",
                "Niko Miekka",
                "Tapio Salakoski",
                "Filip Ginter"
            ],
            "abstract": "A common approach for improving OCR quality is a post-processing step based\non models correcting misdetected characters and tokens. These models are\ntypically trained on aligned pairs of OCR read text and their manually\ncorrected counterparts. In this paper we show that the requirement of manually\ncorrected training data can be alleviated by estimating the OCR errors from\nrepeating text spans found in large OCR read text corpora and generating\nsynthetic training examples following this error distribution. We use the\ngenerated data for training a character-level neural seq2seq model and evaluate\nthe performance of the suggested model on a manually corrected corpus of\nFinnish newspapers mostly from the 19th century. The results show that a clear\nimprovement over the underlying OCR system as well as previously suggested\nmodels utilizing uniformly generated noise can be achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11024v1",
            "date": "2019-06-26 12:27:05+00:00",
            "title": "Sharing Attention Weights for Fast Transformer",
            "authors": [
                "Tong Xiao",
                "Yinqiao Li",
                "Jingbo Zhu",
                "Zhengtao Yu",
                "Tongran Liu"
            ],
            "abstract": "Recently, the Transformer machine translation system has shown strong results\nby stacking attention layers on both the source and target-language sides. But\nthe inference of this model is slow due to the heavy use of dot-product\nattention in auto-regressive decoding. In this paper we speed up Transformer\nvia a fast and lightweight attention model. More specifically, we share\nattention weights in adjacent layers and enable the efficient re-use of hidden\nstates in a vertical manner. Moreover, the sharing policy can be jointly\nlearned with the MT model. We test our approach on ten WMT and NIST OpenMT\ntasks. Experimental results show that it yields an average of 1.3X speed-up\n(with almost no decrease in BLEU) on top of a state-of-the-art implementation\nthat has already adopted a cache for fast inference. Also, our approach obtains\na 1.8X speed-up when it works with the \\textsc{Aan} model. This is even 16\ntimes faster than the baseline with no use of the attention cache.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11085v1",
            "date": "2019-06-26 13:27:39+00:00",
            "title": "Enhancing PIO Element Detection in Medical Text Using Contextualized Embedding",
            "authors": [
                "Hichem Mezaoui",
                "Aleksandr Gontcharov",
                "Isuru Gunasekara"
            ],
            "abstract": "In this paper, we investigate a new approach to Population, Intervention and\nOutcome (PIO) element detection, a common task in Evidence Based Medicine\n(EBM). The purpose of this study is two-fold: to build a training dataset for\nPIO element detection with minimum redundancy and ambiguity and to investigate\npossible options in utilizing state of the art embedding methods for the task\nof PIO element detection. For the former purpose, we build a new and improved\ndataset by investigating the shortcomings of previously released datasets. For\nthe latter purpose, we leverage the state of the art text embedding,\nBidirectional Encoder Representations from Transformers (BERT), and build a\nmulti-label classifier. We show that choosing a domain specific pre-trained\nembedding further optimizes the performance of the classifier. Furthermore, we\nshow that the model could be enhanced by using ensemble methods and boosting\ntechniques provided that features are adequately chosen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11384v1",
            "date": "2019-06-26 23:34:22+00:00",
            "title": "Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis",
            "authors": [
                "Junyi Du",
                "He Jiang",
                "Jiaming Shen",
                "Xiang Ren"
            ],
            "abstract": "Cognitive task analysis (CTA) is a type of analysis in applied psychology\naimed at eliciting and representing the knowledge and thought processes of\ndomain experts. In CTA, often heavy human labor is involved to parse the\ninterview transcript into structured knowledge (e.g., flowchart for different\nactions). To reduce human efforts and scale the process, automated CTA\ntranscript parsing is desirable. However, this task has unique challenges as\n(1) it requires the understanding of long-range context information in\nconversational text; and (2) the amount of labeled data is limited and\nindirect---i.e., context-aware, noisy, and low-resource. In this paper, we\npropose a weakly-supervised information extraction framework for automated CTA\ntranscript parsing. We partition the parsing process into a sequence labeling\ntask and a text span-pair relation extraction task, with distant supervision\nfrom human-curated protocol files. To model long-range context information for\nextracting sentence relations, neighbor sentences are involved as a part of\ninput. Different types of models for capturing context dependency are then\napplied. We manually annotate real-world CTA transcripts to facilitate the\nevaluation of the parsing tasks",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11455v2",
            "date": "2019-06-27 06:40:42+00:00",
            "title": "PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation",
            "authors": [
                "Ruixuan Luo",
                "Jingjing Xu",
                "Yi Zhang",
                "Xuancheng Ren",
                "Xu Sun"
            ],
            "abstract": "Chinese word segmentation (CWS) is a fundamental step of Chinese natural\nlanguage processing. In this paper, we build a new toolkit, named PKUSEG, for\nmulti-domain word segmentation. Unlike existing single-model toolkits, PKUSEG\ntargets at multi-domain word segmentation and provides separate models for\ndifferent domains, such as web, medicine, and tourism. The new toolkit also\nsupports POS tagging and model training to adapt to various application\nscenarios. Experiments show that PKUSEG achieves high performance on multiple\ndomains. The toolkit is now freely and publicly available for the usage of\nresearch and industry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11483v1",
            "date": "2019-06-27 08:03:02+00:00",
            "title": "Morphological Irregularity Correlates with Frequency",
            "authors": [
                "Shijie Wu",
                "Ryan Cotterell",
                "Timothy J. O'Donnell"
            ],
            "abstract": "We present a study of morphological irregularity. Following recent work, we\ndefine an information-theoretic measure of irregularity based on the\npredictability of forms in a language. Using a neural transduction model, we\nestimate this quantity for the forms in 28 languages. We first present several\nvalidatory and exploratory analyses of irregularity. We then show that our\nanalyses provide evidence for a correlation between irregularity and frequency:\nhigher frequency items are more likely to be irregular and irregular items are\nmore likely be highly frequent. To our knowledge, this result is the first of\nits breadth and confirms longstanding proposals from the linguistics\nliterature. The correlation is more robust when aggregated at the level of\nwhole paradigms--providing support for models of linguistic structure in which\ninflected forms are unified by abstract underlying stems or lexemes. Code is\navailable at https://github.com/shijie-wu/neural-transducer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11511v1",
            "date": "2019-06-27 09:05:32+00:00",
            "title": "Inducing Syntactic Trees from BERT Representations",
            "authors": [
                "Rudolf Rosa",
                "David Mare\u010dek"
            ],
            "abstract": "We use the English model of BERT and explore how a deletion of one word in a\nsentence changes representations of other words. Our hypothesis is that\nremoving a reducible word (e.g. an adjective) does not affect the\nrepresentation of other words so much as removing e.g. the main verb, which\nmakes the sentence ungrammatical and of \"high surprise\" for the language model.\nWe estimate reducibilities of individual words and also of longer continuous\nphrases (word n-grams), study their syntax-related properties, and then also\nuse them to induce full dependency trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11565v2",
            "date": "2019-06-27 11:46:48+00:00",
            "title": "EmotionX-KU: BERT-Max based Contextual Emotion Classifier",
            "authors": [
                "Kisu Yang",
                "Dongyub Lee",
                "Taesun Whang",
                "Seolhwa Lee",
                "Heuiseok Lim"
            ],
            "abstract": "We propose a contextual emotion classifier based on a transferable language\nmodel and dynamic max pooling, which predicts the emotion of each utterance in\na dialogue. A representative emotion analysis task, EmotionX, requires to\nconsider contextual information from colloquial dialogues and to deal with a\nclass imbalance problem. To alleviate these problems, our model leverages the\nself-attention based transferable language model and the weighted cross entropy\nloss. Furthermore, we apply post-training and fine-tuning mechanisms to enhance\nthe domain adaptability of our model and utilize several machine learning\ntechniques to improve its performance. We conduct experiments on two\nemotion-labeled datasets named Friends and EmotionPush. As a result, our model\noutperforms the previous state-of-the-art model and also shows competitive\nperformance in the EmotionX 2019 challenge. The code will be available in the\nGithub page.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11608v2",
            "date": "2019-06-27 13:15:12+00:00",
            "title": "Simple Natural Language Processing Tools for Danish",
            "authors": [
                "Leon Derczynski"
            ],
            "abstract": "This technical note describes a set of baseline tools for automatic\nprocessing of Danish text. The tools are machine-learning based, using natural\nlanguage processing models trained over previously annotated documents. They\nare maintained at ITU Copenhagen and will always be freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11746v2",
            "date": "2019-06-27 15:50:59+00:00",
            "title": "Compositional Semantic Parsing Across Graphbanks",
            "authors": [
                "Matthias Lindemann",
                "Jonas Groschwitz",
                "Alexander Koller"
            ],
            "abstract": "Most semantic parsers that map sentences to graph-based meaning\nrepresentations are hand-designed for specific graphbanks. We present a\ncompositional neural semantic parser which achieves, for the first time,\ncompetitive accuracies across a diverse range of graphbanks. Incorporating BERT\nembeddings and multi-task learning improves the accuracy further, setting new\nstates of the art on DM, PAS, PSD, AMR 2015 and EDS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11751v1",
            "date": "2019-06-27 15:53:58+00:00",
            "title": "The Impact of Preprocessing on Arabic-English Statistical and Neural Machine Translation",
            "authors": [
                "Mai Oudah",
                "Amjad Almahairi",
                "Nizar Habash"
            ],
            "abstract": "Neural networks have become the state-of-the-art approach for machine\ntranslation (MT) in many languages. While linguistically-motivated tokenization\ntechniques were shown to have significant effects on the performance of\nstatistical MT, it remains unclear if those techniques are well suited for\nneural MT. In this paper, we systematically compare neural and statistical MT\nmodels for Arabic-English translation on data preprecossed by various prominent\ntokenization schemes. Furthermore, we consider a range of data and vocabulary\nsizes and compare their effect on both approaches. Our empirical results show\nthat the best choice of tokenization scheme is largely based on the type of\nmodel and the size of data. We also show that we can gain significant\nimprovements using a system selection that combines the output from neural and\nstatistical MT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11861v1",
            "date": "2019-06-27 18:23:27+00:00",
            "title": "Relating Simple Sentence Representations in Deep Neural Networks and the Brain",
            "authors": [
                "Sharmistha Jat",
                "Hao Tang",
                "Partha Talukdar",
                "Tom Mitchell"
            ],
            "abstract": "What is the relationship between sentence representations learned by deep\nrecurrent models against those encoded by the brain? Is there any\ncorrespondence between hidden layers of these recurrent models and brain\nregions when processing sentences? Can these deep models be used to synthesize\nbrain data which can then be utilized in other extrinsic tasks? We investigate\nthese questions using sentences with simple syntax and semantics (e.g., The\nbone was eaten by the dog.). We consider multiple neural network architectures,\nincluding recently proposed ELMo and BERT. We use magnetoencephalography (MEG)\nbrain recording data collected from human subjects when they were reading these\nsimple sentences.\n  Overall, we find that BERT's activations correlate the best with MEG brain\ndata. We also find that the deep network representation can be used to generate\nbrain data from new sentences to augment existing brain data.\n  To the best of our knowledge, this is the first work showing that the MEG\nbrain recording when reading a word in a sentence can be used to distinguish\nearlier words in the sentence. Our exploration is also the first to use deep\nneural network representations to generate synthetic brain data and to show\nthat it helps in improving subsequent stimuli decoding task accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11930v1",
            "date": "2019-06-27 19:40:09+00:00",
            "title": "Training Models to Extract Treatment Plans from Clinical Notes Using Contents of Sections with Headings",
            "authors": [
                "Ananya Poddar",
                "Bharath Dandala",
                "Murthy Devarakonda"
            ],
            "abstract": "Objective: Using natural language processing (NLP) to find sentences that\nstate treatment plans in a clinical note, would automate plan extraction and\nwould further enable their use in tools that help providers and care managers.\nHowever, as in the most NLP tasks on clinical text, creating gold standard to\ntrain and test NLP models is tedious and expensive. Fortuitously, sometimes but\nnot always clinical notes contain sections with a heading that identifies the\nsection as a plan. Leveraging contents of such labeled sections as a noisy\ntraining data, we assessed accuracy of NLP models trained with the data.\n  Methods: We used common variations of plan headings and rule-based heuristics\nto find plan sections with headings in clinical notes, and we extracted\nsentences from them and formed a noisy training data of plan sentences. We\ntrained Support Vector Machine (SVM) and Convolutional Neural Network (CNN)\nmodels with the data. We measured accuracy of the trained models on the noisy\ndataset using ten-fold cross validation and separately on a set-aside manually\nannotated dataset.\n  Results: About 13% of 117,730 clinical notes contained treatment plans\nsections with recognizable headings in the 1001 longitudinal patient records\nthat were obtained from Cleveland Clinic under an IRB approval. We were able to\nextract and create a noisy training data of 13,492 plan sentences from the\nclinical notes. CNN achieved best F measures, 0.91 and 0.97 in the\ncross-validation and set-aside evaluation experiments respectively. SVM\nslightly underperformed with F measures of 0.89 and 0.96 in the same\nexperiments.\n  Conclusion: Our study showed that the training supervised learning models\nusing noisy plan sentences was effective in identifying them in all clinical\nnotes. More broadly, sections with informal headings in clinical notes can be a\ngood source for generating effective training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11943v2",
            "date": "2019-06-27 20:24:55+00:00",
            "title": "Findings of the First Shared Task on Machine Translation Robustness",
            "authors": [
                "Xian Li",
                "Paul Michel",
                "Antonios Anastasopoulos",
                "Yonatan Belinkov",
                "Nadir Durrani",
                "Orhan Firat",
                "Philipp Koehn",
                "Graham Neubig",
                "Juan Pino",
                "Hassan Sajjad"
            ],
            "abstract": "We share the findings of the first shared task on improving robustness of\nMachine Translation (MT). The task provides a testbed representing challenges\nfacing MT models deployed in the real world, and facilitates new approaches to\nimprove models; robustness to noisy input and domain mismatch. We focus on two\nlanguage pairs (English-French and English-Japanese), and the submitted systems\nare evaluated on a blind test set consisting of noisy comments on Reddit and\nprofessionally sourced translations. As a new task, we received 23 submissions\nby 11 participating teams from universities, companies, national labs, etc. All\nsubmitted systems achieved large improvements over baselines, with the best\nimprovement having +22.33 BLEU. We evaluated submissions by both human judgment\nand automatic evaluation (BLEU), which shows high correlations (Pearson's r =\n0.94 and 0.95). Furthermore, we conducted a qualitative analysis of the\nsubmitted systems using compare-mt, which revealed their salient differences in\nhandling challenges in this task. Such analysis provides additional insights\nwhen there is occasional disagreement between human judgment and BLEU, e.g.\nsystems better at producing colloquial expressions received higher score from\nhuman judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00168v1",
            "date": "2019-06-29 09:00:45+00:00",
            "title": "The CUED's Grammatical Error Correction Systems for BEA-2019",
            "authors": [
                "Felix Stahlberg",
                "Bill Byrne"
            ],
            "abstract": "We describe two entries from the Cambridge University Engineering Department\nto the BEA 2019 Shared Task on grammatical error correction. Our submission to\nthe low-resource track is based on prior work on using finite state transducers\ntogether with strong neural language models. Our system for the restricted\ntrack is a purely neural system consisting of neural language models and neural\nmachine translation models trained with back-translation and a combination of\ncheckpoint averaging and fine-tuning -- without the help of any additional\ntools like spell checkers. The latter system has been used inside a separate\nsystem combination entry in cooperation with the Cambridge University Computer\nLab.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00184v2",
            "date": "2019-06-29 11:47:22+00:00",
            "title": "Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settings",
            "authors": [
                "Marcely Zanon Boito",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "Since Bahdanau et al. [1] first introduced attention for neural machine\ntranslation, most sequence-to-sequence models made use of attention mechanisms\n[2, 3, 4]. While they produce soft-alignment matrices that could be interpreted\nas alignment between target and source languages, we lack metrics to quantify\ntheir quality, being unclear which approach produces the best alignments. This\npaper presents an empirical evaluation of 3 main sequence-to-sequence models\n(CNN, RNN and Transformer-based) for word discovery from unsegmented phoneme\nsequences. This task consists in aligning word sequences in a source language\nwith phoneme sequences in a target language, inferring from it word\nsegmentation on the target side [5]. Evaluating word segmentation quality can\nbe seen as an extrinsic evaluation of the soft-alignment matrices produced\nduring training. Our experiments in a low-resource scenario on Mboshi and\nEnglish languages (both aligned to French) show that RNNs surprisingly\noutperform CNNs and Transformer for this task. Our results are confirmed by an\nintrinsic evaluation of alignment quality through the use of Average Normalized\nEntropy (ANE). Lastly, we improve our best word discovery model by using an\nalignment entropy confidence measure that accumulates ANE over all the\noccurrences of a given alignment pair in the collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00218v2",
            "date": "2019-06-29 15:15:16+00:00",
            "title": "Latent Variable Sentiment Grammar",
            "authors": [
                "Liwen Zhang",
                "Kewei Tu",
                "Yue Zhang"
            ],
            "abstract": "Neural models have been investigated for sentiment classification over\nconstituent trees. They learn phrase composition automatically by encoding tree\nstructures but do not explicitly model sentiment composition, which requires to\nencode sentiment class labels. To this end, we investigate two formalisms with\ndeep sentiment representations that capture sentiment subtype expressions by\nlatent variables and Gaussian mixture vectors, respectively. Experiments on\nStanford Sentiment Treebank (SST) show the effectiveness of sentiment grammar\nover vanilla neural encoders. Using ELMo embeddings, our method gives the best\nresults on this benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00326v1",
            "date": "2019-06-30 06:03:32+00:00",
            "title": "Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes",
            "authors": [
                "Jie Cao",
                "Michael Tanana",
                "Zac E. Imel",
                "Eric Poitras",
                "David C. Atkins",
                "Vivek Srikumar"
            ],
            "abstract": "Automatically analyzing dialogue can help understand and guide behavior in\ndomains such as counseling, where interactions are largely mediated by\nconversation. In this paper, we study modeling behavioral codes used to asses a\npsychotherapy treatment style called Motivational Interviewing (MI), which is\neffective for addressing substance abuse and related problems. Specifically, we\naddress the problem of providing real-time guidance to therapists with a\ndialogue observer that (1) categorizes therapist and client MI behavioral codes\nand, (2) forecasts codes for upcoming utterances to help guide the conversation\nand potentially alert the therapist. For both tasks, we define neural network\nmodels that build upon recent successes in dialogue modeling. Our experiments\ndemonstrate that our models can outperform several baselines for both tasks. We\nalso report the results of a careful analysis that reveals the impact of the\nvarious network design tradeoffs for modeling therapy dialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00409v1",
            "date": "2019-06-30 16:32:28+00:00",
            "title": "Evaluating Language Model Finetuning Techniques for Low-resource Languages",
            "authors": [
                "Jan Christian Blaise Cruz",
                "Charibeth Cheng"
            ],
            "abstract": "Unlike mainstream languages (such as English and French), low-resource\nlanguages often suffer from a lack of expert-annotated corpora and benchmark\nresources that make it hard to apply state-of-the-art techniques directly. In\nthis paper, we alleviate this scarcity problem for the low-resourced Filipino\nlanguage in two ways. First, we introduce a new benchmark language modeling\ndataset in Filipino which we call WikiText-TL-39. Second, we show that language\nmodel finetuning techniques such as BERT and ULMFiT can be used to consistently\ntrain robust classifiers in low-resource settings, experiencing at most a\n0.0782 increase in validation error when the number of training examples is\ndecreased from 10K to 1K while finetuning using a privately-held sentiment\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00448v1",
            "date": "2019-06-30 20:17:52+00:00",
            "title": "Self-Supervised Dialogue Learning",
            "authors": [
                "Jiawei Wu",
                "Xin Wang",
                "William Yang Wang"
            ],
            "abstract": "The sequential order of utterances is often meaningful in coherent dialogues,\nand the order changes of utterances could lead to low-quality and incoherent\nconversations. We consider the order information as a crucial supervised signal\nfor dialogue learning, which, however, has been neglected by many previous\ndialogue systems. Therefore, in this paper, we introduce a self-supervised\nlearning task, inconsistent order detection, to explicitly capture the flow of\nconversation in dialogues. Given a sampled utterance pair triple, the task is\nto predict whether it is ordered or misordered. Then we propose a\nsampling-based self-supervised network SSN to perform the prediction with\nsampled triple references from previous dialogue history. Furthermore, we\ndesign a joint learning framework where SSN can guide the dialogue systems\ntowards more coherent and relevant dialogue learning through adversarial\ntraining. We demonstrate that the proposed methods can be applied to both\nopen-domain and task-oriented dialogue scenarios, and achieve the new\nstate-of-the-art performance on the OpenSubtitiles and Movie-Ticket Booking\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00505v1",
            "date": "2019-07-01 00:43:45+00:00",
            "title": "Few-Shot Representation Learning for Out-Of-Vocabulary Words",
            "authors": [
                "Ziniu Hu",
                "Ting Chen",
                "Kai-Wei Chang",
                "Yizhou Sun"
            ],
            "abstract": "Existing approaches for learning word embeddings often assume there are\nsufficient occurrences for each word in the corpus, such that the\nrepresentation of words can be accurately estimated from their contexts.\nHowever, in real-world scenarios, out-of-vocabulary (a.k.a. OOV) words that do\nnot appear in training corpus emerge frequently. It is challenging to learn\naccurate representations of these words with only a few observations. In this\npaper, we formulate the learning of OOV embeddings as a few-shot regression\nproblem, and address it by training a representation function to predict the\noracle embedding vector (defined as embedding trained with abundant\nobservations) based on limited observations. Specifically, we propose a novel\nhierarchical attention-based architecture to serve as the neural regression\nfunction, with which the context information of a word is encoded and\naggregated from K observations. Furthermore, our approach can leverage\nModel-Agnostic Meta-Learning (MAML) for adapting the learned model to the new\ncorpus fast and robustly. Experiments show that the proposed approach\nsignificantly outperforms existing methods in constructing accurate embeddings\nfor OOV words, and improves downstream tasks where these embeddings are\nutilized.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00620v6",
            "date": "2019-07-01 09:14:45+00:00",
            "title": "Using Database Rule for Weak Supervised Text-to-SQL Generation",
            "authors": [
                "Tong Guo",
                "Huilin Gao"
            ],
            "abstract": "We present a simple way to do the task of text-to-SQL problem with weak\nsupervision. We call it Rule-SQL. Given the question and the answer from the\ndatabase table without the SQL logic form, Rule-SQL use the rules based on\ntable column names and question string for the SQL exploration first and then\nuse the explored SQL for supervised training. We design several rules for\nreducing the exploration search space. For the deep model, we leverage BERT for\nthe representation layer and separate the model to SELECT, AGG and WHERE parts.\nThe experiment result on WikiSQL outperforms the strong baseline of full\nsupervision and is comparable to the start-of-the-art weak supervised mothods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00659v2",
            "date": "2019-07-01 11:13:19+00:00",
            "title": "Modernizing Historical Documents: a User Study",
            "authors": [
                "Miguel Domingo",
                "Francisco Casacuberta"
            ],
            "abstract": "Accessibility to historical documents is mostly limited to scholars. This is\ndue to the language barrier inherent in human language and the linguistic\nproperties of these documents. Given a historical document, modernization aims\nto generate a new version of it, written in the modern version of the\ndocument's language. Its goal is to tackle the language barrier, decreasing the\ncomprehension difficulty and making historical documents accessible to a\nbroader audience. In this work, we proposed a new neural machine translation\napproach that profits from modern documents to enrich its systems. We tested\nthis approach with both automatic and human evaluation, and conducted a user\nstudy. Results showed that modernization is successfully reaching its goal,\nalthough it still has room for improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00735v2",
            "date": "2019-06-28 09:28:00+00:00",
            "title": "From Bilingual to Multilingual Neural Machine Translation by Incremental Training",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-Juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Multilingual Neural Machine Translation approaches are based on the use of\ntask-specific models and the addition of one more language can only be done by\nretraining the whole system. In this work, we propose a new training schedule\nthat allows the system to scale to more languages without modification of the\nprevious components based on joint training and language-independent\nencoder/decoder modules allowing for zero-shot translation. This work in\nprogress shows close results to the state-of-the-art in the WMT task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00810v1",
            "date": "2019-07-01 14:15:16+00:00",
            "title": "Multilingual, Multi-scale and Multi-layer Visualization of Intermediate Representations",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Elora Lacroux",
                "Pere-Pau V\u00e1zquez"
            ],
            "abstract": "The main alternatives nowadays to deal with sequences are Recurrent Neural\nNetworks (RNN), Convolutional Neural Networks (CNN) architectures and the\nTransformer. In this context, RNN's, CNN's and Transformer have most commonly\nbeen used as an encoder-decoder architecture with multiple layers in each\nmodule. Far beyond this, these architectures are the basis for the contextual\nword embeddings which are revolutionizing most natural language downstream\napplications. However, intermediate layer representations in sequence-based\narchitectures can be difficult to interpret. To make each layer representation\nwithin these architectures more accessible and meaningful, we introduce a\nweb-based tool that visualizes them both at the sentence and token level. We\npresent three use cases. The first analyses gender issues in contextual word\nembeddings. The second and third are showing multilingual intermediate\nrepresentations for sentences and tokens and the evolution of these\nintermediate representations along the multiple layers of the decoder and in\nthe context of multilingual machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00900v2",
            "date": "2019-07-01 16:16:39+00:00",
            "title": "Post-editese: an Exacerbated Translationese",
            "authors": [
                "Antonio Toral"
            ],
            "abstract": "Post-editing (PE) machine translation (MT) is widely used for dissemination\nbecause it leads to higher productivity than human translation from scratch\n(HT). In addition, PE translations are found to be of equal or better quality\nthan HTs. However, most such studies measure quality solely as the number of\nerrors. We conduct a set of computational analyses in which we compare PE\nagainst HT on three different datasets that cover five translation directions\nwith measures that address different translation universals and laws of\ntranslation: simplification, normalisation and interference. We find out that\nPEs are simpler and more normalised and have a higher degree of interference\nfrom the source language than HTs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00962v2",
            "date": "2019-07-01 17:56:41+00:00",
            "title": "Claim Extraction in Biomedical Publications using Deep Discourse Model and Transfer Learning",
            "authors": [
                "Titipat Achakulvisut",
                "Chandra Bhagavatula",
                "Daniel Acuna",
                "Konrad Kording"
            ],
            "abstract": "Claims are a fundamental unit of scientific discourse. The exponential growth\nin the number of scientific publications makes automatic claim extraction an\nimportant problem for researchers who are overwhelmed by this information\noverload. Such an automated claim extraction system is useful for both manual\nand programmatic exploration of scientific knowledge. In this paper, we\nintroduce a new dataset of 1,500 scientific abstracts from the biomedical\ndomain with expert annotations for each sentence indicating whether the\nsentence presents a scientific claim. We introduce a new model for claim\nextraction and compare it to several baseline models including rule-based and\ndeep learning techniques. Moreover, we show that using a transfer learning\napproach with a fine-tuning step allows us to improve performance from a large\ndiscourse-annotated dataset. Our final model increases F1-score by over 14\npercent points compared to a baseline model without transfer learning. We\nrelease a publicly accessible tool for discourse and claims prediction along\nwith an annotation tool. We discuss further applications beyond biomedical\nliterature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01118v5",
            "date": "2019-07-02 01:14:13+00:00",
            "title": "Neural Machine Reading Comprehension: Methods and Trends",
            "authors": [
                "Shanshan Liu",
                "Xin Zhang",
                "Sheng Zhang",
                "Hui Wang",
                "Weiming Zhang"
            ],
            "abstract": "Machine reading comprehension (MRC), which requires a machine to answer\nquestions based on a given context, has attracted increasing attention with the\nincorporation of various deep-learning techniques over the past few years.\nAlthough research on MRC based on deep learning is flourishing, there remains a\nlack of a comprehensive survey summarizing existing approaches and recent\ntrends, which motivated the work presented in this article. Specifically, we\ngive a thorough review of this research field, covering different aspects\nincluding (1) typical MRC tasks: their definitions, differences, and\nrepresentative datasets; (2) the general architecture of neural MRC: the main\nmodules and prevalent approaches to each; and (3) new trends: some emerging\nareas in neural MRC as well as the corresponding challenges. Finally,\nconsidering what has been achieved so far, the survey also envisages what the\nfuture may hold by discussing the open issues left to be addressed.",
            "journal ref": "J. Applied Sciences, 2019, 9(18): 3698",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01166v1",
            "date": "2019-07-02 04:54:17+00:00",
            "title": "Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems",
            "authors": [
                "Hung Le",
                "Doyen Sahoo",
                "Nancy F. Chen",
                "Steven C. H. Hoi"
            ],
            "abstract": "Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is\nconducted based on visual and audio aspects of a given video, is significantly\nmore challenging than traditional image or text-grounded dialogue systems\nbecause (1) feature space of videos span across multiple picture frames, making\nit difficult to obtain semantic information; and (2) a dialogue agent must\nperceive and process information from different modalities (audio, video,\ncaption, etc.) to obtain a comprehensive understanding. Most existing work is\nbased on RNNs and sequence-to-sequence architectures, which are not very\neffective for capturing complex long-term dependencies (like in videos). To\novercome this, we propose Multimodal Transformer Networks (MTN) to encode\nvideos and incorporate information from different modalities. We also propose\nquery-aware attention through an auto-encoder to extract query-aware features\nfrom non-text modalities. We develop a training procedure to simulate\ntoken-level decoding to improve the quality of generated responses during\ninference. We get state of the art performance on Dialogue System Technology\nChallenge 7 (DSTC7). Our model also generalizes to another multimodal\nvisual-grounded dialogue task, and obtains promising performance. We\nimplemented our models using PyTorch and the code is released at\nhttps://github.com/henryhungle/MTN.",
            "journal ref": "Association for Computational Linguistics (2019) 5612-5623",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01272v2",
            "date": "2019-07-02 10:06:33+00:00",
            "title": "Discourse Understanding and Factual Consistency in Abstractive Summarization",
            "authors": [
                "Saadia Gabriel",
                "Antoine Bosselut",
                "Jeff Da",
                "Ari Holtzman",
                "Jan Buys",
                "Kyle Lo",
                "Asli Celikyilmaz",
                "Yejin Choi"
            ],
            "abstract": "We introduce a general framework for abstractive summarization with factual\nconsistency and distinct modeling of the narrative flow in an output summary.\nOur work addresses current limitations of models for abstractive summarization\nthat often hallucinate information or generate summaries with coherence issues.\n  To generate abstractive summaries with factual consistency and narrative\nflow, we propose Cooperative Generator -- Discriminator Networks (Co-opNet), a\nnovel transformer-based framework where a generator works with a discriminator\narchitecture to compose coherent long-form summaries. We explore four different\ndiscriminator objectives which each capture a different aspect of coherence,\nincluding whether salient spans of generated abstracts are hallucinated or\nappear in the input context, and the likelihood of sentence adjacency in\ngenerated abstracts. We measure the ability of Co-opNet to learn these\nobjectives with arXiv scientific papers, using the abstracts as a proxy for\ngold long-form scientific article summaries. Empirical results from automatic\nand human evaluations demonstrate that Co-opNet learns to summarize with\nconsiderably improved global coherence compared to competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01279v1",
            "date": "2019-07-02 10:11:23+00:00",
            "title": "Improving Robustness in Real-World Neural Machine Translation Engines",
            "authors": [
                "Rohit Gupta",
                "Patrik Lambert",
                "Raj Nath Patel",
                "John Tinsley"
            ],
            "abstract": "As a commercial provider of machine translation, we are constantly training\nengines for a variety of uses, languages, and content types. In each case,\nthere can be many variables, such as the amount of training data available, and\nthe quality requirements of the end user. These variables can have an impact on\nthe robustness of Neural MT engines. On the whole, Neural MT cures many ills of\nother MT paradigms, but at the same time, it has introduced a new set of\nchallenges to address. In this paper, we describe some of the specific issues\nwith practical NMT and the approaches we take to improve model robustness in\nreal-world scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01302v1",
            "date": "2019-07-02 11:33:52+00:00",
            "title": "Latent Dirichlet Allocation Based Acoustic Data Selection for Automatic Speech Recognition",
            "authors": [
                "Mortaza",
                "Doulaty",
                "Thomas Hain"
            ],
            "abstract": "Selecting in-domain data from a large pool of diverse and out-of-domain data\nis a non-trivial problem. In most cases simply using all of the available data\nwill lead to sub-optimal and in some cases even worse performance compared to\ncarefully selecting a matching set. This is true even for data-inefficient\nneural models. Acoustic Latent Dirichlet Allocation (aLDA) is shown to be\nuseful in a variety of speech technology related tasks, including domain\nadaptation of acoustic models for automatic speech recognition and entity\nlabeling for information retrieval. In this paper we propose to use aLDA as a\ndata similarity criterion in a data selection framework. Given a large pool of\nout-of-domain and potentially mismatched data, the task is to select the\nbest-matching training data to a set of representative utterances sampled from\na target domain. Our target data consists of around 32 hours of meeting data\n(both far-field and close-talk) and the pool contains 2k hours of meeting,\ntalks, voice search, dictation, command-and-control, audio books, lectures,\ngeneric media and telephony speech data. The proposed technique for training\ndata selection, significantly outperforms random selection, posterior-based\nselection as well as using all of the available data.",
            "journal ref": "Proc. of Interspeech (2019), Graz, Austria",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01417v2",
            "date": "2019-07-02 14:53:30+00:00",
            "title": "Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns",
            "authors": [
                "Julien Fauqueur",
                "Ashok Thillaisundaram",
                "Theodosia Togia"
            ],
            "abstract": "Knowledge base construction is crucial for summarising, understanding and\ninferring relationships between biomedical entities. However, for many\npractical applications such as drug discovery, the scarcity of relevant facts\n(e.g. gene X is therapeutic target for disease Y) severely limits a domain\nexpert's ability to create a usable knowledge base, either directly or by\ntraining a relation extraction model.\n  In this paper, we present a simple and effective method of extracting new\nfacts with a pre-specified binary relationship type from the biomedical\nliterature, without requiring any training data or hand-crafted rules. Our\nsystem discovers, ranks and presents the most salient patterns to domain\nexperts in an interpretable form. By marking patterns as compatible with the\ndesired relationship type, experts indirectly batch-annotate candidate pairs\nwhose relationship is expressed with such patterns in the literature. Even with\na complete absence of seed data, experts are able to discover thousands of\nhigh-quality pairs with the desired relationship within minutes. When a small\nnumber of relevant pairs do exist - even when their relationship is more\ngeneral (e.g. gene X is biologically associated with disease Y) than the\nrelationship of interest - our system leverages them in order to i) learn a\nbetter ranking of the patterns to be annotated or ii) generate weakly labelled\npairs in a fully automated manner.\n  We evaluate our method both intrinsically and via a downstream knowledge base\ncompletion task, and show that it is an effective way of constructing knowledge\nbases when few or no relevant facts are already available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01468v1",
            "date": "2019-07-02 15:55:42+00:00",
            "title": "How we do things with words: Analyzing text as social and cultural data",
            "authors": [
                "Dong Nguyen",
                "Maria Liakata",
                "Simon DeDeo",
                "Jacob Eisenstein",
                "David Mimno",
                "Rebekah Tromble",
                "Jane Winters"
            ],
            "abstract": "In this article we describe our experiences with computational text analysis.\nWe hope to achieve three primary goals. First, we aim to shed light on thorny\nissues not always at the forefront of discussions about computational text\nanalysis methods. Second, we hope to provide a set of best practices for\nworking with thick social and cultural concepts. Our guidance is based on our\nown experiences and is therefore inherently imperfect. Still, given our\ndiversity of disciplinary backgrounds and research practices, we hope to\ncapture a range of ideas and identify commonalities that will resonate for\nmany. And this leads to our final goal: to help promote interdisciplinary\ncollaborations. Interdisciplinary insights and partnerships are essential for\nrealizing the full potential of any computational text analysis that involves\nsocial and cultural concepts, and the more we are able to bridge these divides,\nthe more fruitful we believe our work will be.",
            "journal ref": "Front. Artif. Intell. 3:62 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01668v1",
            "date": "2019-07-02 22:29:53+00:00",
            "title": "Data mining Mandarin tone contour shapes",
            "authors": [
                "Shuo Zhang"
            ],
            "abstract": "In spontaneous speech, Mandarin tones that belong to the same tone category\nmay exhibit many different contour shapes. We explore the use of data mining\nand NLP techniques for understanding the variability of tones in a large corpus\nof Mandarin newscast speech. First, we adapt a graph-based approach to\ncharacterize the clusters (fuzzy types) of tone contour shapes observed in each\ntone n-gram category. Second, we show correlations between these realized\ncontour shape types and a bag of automatically extracted linguistic features.\nWe discuss the implications of the current study within the context of\nphonological and information theory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01686v1",
            "date": "2019-06-30 09:18:31+00:00",
            "title": "Machine Reading Comprehension: a Literature Review",
            "authors": [
                "Xin Zhang",
                "An Yang",
                "Sujian Li",
                "Yizhong Wang"
            ],
            "abstract": "Machine reading comprehension aims to teach machines to understand a text\nlike a human and is a new challenging direction in Artificial Intelligence.\nThis article summarizes recent advances in MRC, mainly focusing on two aspects\n(i.e., corpus and techniques). The specific characteristics of various MRC\ncorpus are listed and compared. The main ideas of some typical MRC techniques\nare also described.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01968v1",
            "date": "2019-07-03 14:31:45+00:00",
            "title": "Depth Growing for Neural Machine Translation",
            "authors": [
                "Lijun Wu",
                "Yiren Wang",
                "Yingce Xia",
                "Fei Tian",
                "Fei Gao",
                "Tao Qin",
                "Jianhuang Lai",
                "Tie-Yan Liu"
            ],
            "abstract": "While very deep neural networks have shown effectiveness for computer vision\nand text classification applications, how to increase the network depth of\nneural machine translation (NMT) models for better translation quality remains\na challenging problem. Directly stacking more blocks to the NMT model results\nin no improvement and even reduces performance. In this work, we propose an\neffective two-stage approach with three specially designed components to\nconstruct deeper NMT models, which result in significant improvements over the\nstrong Transformer baselines on WMT$14$ English$\\to$German and\nEnglish$\\to$French translation tasks\\footnote{Our code is available at\n\\url{https://github.com/apeterswu/Depth_Growing_NMT}}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02030v1",
            "date": "2019-07-03 16:50:50+00:00",
            "title": "Real-time Claim Detection from News Articles and Retrieval of Semantically-Similar Factchecks",
            "authors": [
                "Ben Adler",
                "Giacomo Boscaini-Gilroy"
            ],
            "abstract": "Factchecking has always been a part of the journalistic process. However with\nnewsroom budgets shrinking it is coming under increasing pressure just as the\namount of false information circulating is on the rise. We therefore propose a\nmethod to increase the efficiency of the factchecking process, using the latest\ndevelopments in Natural Language Processing (NLP). This method allows us to\ncompare incoming claims to an existing corpus and return similar, factchecked,\nclaims in a live system-allowing factcheckers to work simultaneously without\nduplicating their work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02090v1",
            "date": "2019-07-03 18:08:16+00:00",
            "title": "Learning Multi-Party Turn-Taking Models from Dialogue Logs",
            "authors": [
                "Maira Gatti de Bayser",
                "Paulo Cavalin",
                "Claudio Pinhanez",
                "Bianca Zadrozny"
            ],
            "abstract": "This paper investigates the application of machine learning (ML) techniques\nto enable intelligent systems to learn multi-party turn-taking models from\ndialogue logs. The specific ML task consists of determining who speaks next,\nafter each utterance of a dialogue, given who has spoken and what was said in\nthe previous utterances. With this goal, this paper presents comparisons of the\naccuracy of different ML techniques such as Maximum Likelihood Estimation\n(MLE), Support Vector Machines (SVM), and Convolutional Neural Networks (CNN)\narchitectures, with and without utterance data. We present three corpora: the\nfirst with dialogues from an American TV situated comedy (chit-chat), the\nsecond with logs from a financial advice multi-bot system and the third with a\ncorpus created from the Multi-Domain Wizard-of-Oz dataset (both are\ntopic-oriented). The results show: (i) the size of the corpus has a very\npositive impact on the accuracy for the content-based deep learning approaches\nand those models perform best in the larger datasets; and (ii) if the dialogue\ndataset is small and topic-oriented (but with few topics), it is sufficient to\nuse an agent-only MLE or SVM models, although slightly higher accuracies can be\nachieved with the use of the content of the utterances with a CNN model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02298v3",
            "date": "2019-07-04 09:40:27+00:00",
            "title": "A Comparative Analysis of Knowledge-Intensive and Data-Intensive Semantic Parsers",
            "authors": [
                "Junjie Cao",
                "Zi Lin",
                "Weiwei Sun",
                "Xiaojun Wan"
            ],
            "abstract": "We present a phenomenon-oriented comparative analysis of the two dominant\napproaches in task-independent semantic parsing: classic, knowledge-intensive\nand neural, data-intensive models. To reflect state-of-the-art neural NLP\ntechnologies, we introduce a new target structure-centric parser that can\nproduce semantic graphs much more accurately than previous data-driven parsers.\nWe then show that, in spite of comparable performance overall, knowledge- and\ndata-intensive models produce different types of errors, in a way that can be\nexplained by their theoretical properties. This analysis leads to new\ndirections for parser development.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02326v2",
            "date": "2019-07-04 10:58:58+00:00",
            "title": "Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation",
            "authors": [
                "Tsz Kin Lam",
                "Shigehiko Schamoni",
                "Stefan Riezler"
            ],
            "abstract": "We propose an interactive-predictive neural machine translation framework for\neasier model personalization using reinforcement and imitation learning. During\nthe interactive translation process, the user is asked for feedback on\nuncertain locations identified by the system. Responses are weak feedback in\nthe form of \"keep\" and \"delete\" edits, and expert demonstrations in the form of\n\"substitute\" edits. Conditioning on the collected feedback, the system creates\nalternative translations via constrained beam search. In simulation experiments\non two language pairs our systems get close to the performance of supervised\ntraining with much less human effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02423v1",
            "date": "2019-07-04 14:32:39+00:00",
            "title": "Morphological Word Embeddings",
            "authors": [
                "Ryan Cotterell",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Linguistic similarity is multi-faceted. For instance, two words may be\nsimilar with respect to semantics, syntax, or morphology inter alia. Continuous\nword-embeddings have been shown to capture most of these shades of similarity\nto some degree. This work considers guiding word-embeddings with\nmorphologically annotated data, a form of semi-supervised learning, encouraging\nthe vectors to encode a word's morphology, i.e., words close in the embedded\nspace share morphological features. We extend the log-bilinear model to this\nend and show that indeed our learned embeddings achieve this, using German as a\ncase study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02581v2",
            "date": "2019-07-04 20:37:44+00:00",
            "title": "Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study",
            "authors": [
                "Derek Howard",
                "Marta Maslej",
                "Justin Lee",
                "Jacob Ritchie",
                "Geoffrey Woollard",
                "Leon French"
            ],
            "abstract": "Mental illness affects a significant portion of the worldwide population.\nOnline mental health forums can provide a supportive environment for those\nafflicted and also generate a large amount of data which can be mined to\npredict mental health states using machine learning methods. We benchmark\nmultiple methods of text feature representation for social media posts and\ncompare their downstream use with automated machine learning (AutoML) tools to\ntriage content for moderator attention. We used 1588 labeled posts from the\nCLPsych 2017 shared task collected from the Reachout.com forum (Milne et al.,\n2019). Posts were represented using lexicon based tools including VADER,\nEmpath, LIWC and also used pre-trained artificial neural network models\nincluding DeepMoji, Universal Sentence Encoder, and GPT-1. We used TPOT and\nauto-sklearn as AutoML tools to generate classifiers to triage the posts. The\ntop-performing system used features derived from the GPT-1 model, which was\nfinetuned on over 150,000 unlabeled posts from Reachout.com. Our top system had\na macro averaged F1 score of 0.572, providing a new state-of-the-art result on\nthe CLPsych 2017 task. This was achieved without additional information from\nmeta-data or preceding posts. Error analyses revealed that this top system\noften misses expressions of hopelessness. We additionally present\nvisualizations that aid understanding of the learned classifiers. We show that\ntransfer learning is an effective strategy for predicting risk with relatively\nlittle labeled data. We note that finetuning of pretrained language models\nprovides further gains when large amounts of unlabeled text is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02679v1",
            "date": "2019-07-05 05:19:46+00:00",
            "title": "Improving Chemical Named Entity Recognition in Patents with Contextualized Word Embeddings",
            "authors": [
                "Zenan Zhai",
                "Dat Quoc Nguyen",
                "Saber A. Akhondi",
                "Camilo Thorne",
                "Christian Druckenbrodt",
                "Trevor Cohn",
                "Michelle Gregory",
                "Karin Verspoor"
            ],
            "abstract": "Chemical patents are an important resource for chemical information. However,\nfew chemical Named Entity Recognition (NER) systems have been evaluated on\npatent documents, due in part to their structural and linguistic complexity. In\nthis paper, we explore the NER performance of a BiLSTM-CRF model utilising\npre-trained word embeddings, character-level word representations and\ncontextualized ELMo word representations for chemical patents. We compare word\nembeddings pre-trained on biomedical and chemical patent corpora. The effect of\ntokenizers optimized for the chemical domain on NER performance in chemical\npatents is also explored. The results on two patent corpora show that\ncontextualized word representations generated from ELMo substantially improve\nchemical NER performance w.r.t. the current state-of-the-art. We also show that\ndomain-specific resources such as word embeddings trained on chemical patents\nand chemical-specific tokenizers have a positive impact on NER performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02684v4",
            "date": "2019-07-05 05:44:21+00:00",
            "title": "Head-Driven Phrase Structure Grammar Parsing on Penn Treebank",
            "authors": [
                "Junru Zhou",
                "Hai Zhao"
            ],
            "abstract": "Head-driven phrase structure grammar (HPSG) enjoys a uniform formalism\nrepresenting rich contextual syntactic and even semantic meanings. This paper\nmakes the first attempt to formulate a simplified HPSG by integrating\nconstituent and dependency formal representations into head-driven phrase\nstructure. Then two parsing algorithms are respectively proposed for two\nconverted tree representations, division span and joint span. As HPSG encodes\nboth constituent and dependency structure information, the proposed HPSG\nparsers may be regarded as a sort of joint decoder for both types of structures\nand thus are evaluated in terms of extracted or converted constituent and\ndependency parsing trees. Our parser achieves new state-of-the-art performance\nfor both parsing tasks on Penn Treebank (PTB) and Chinese Penn Treebank,\nverifying the effectiveness of joint learning constituent and dependency\nstructures. In details, we report 96.33 F1 of constituent parsing and 97.20\\%\nUAS of dependency parsing on PTB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03040v1",
            "date": "2019-07-05 22:41:02+00:00",
            "title": "BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer",
            "authors": [
                "Guan-Lin Chao",
                "Ian Lane"
            ],
            "abstract": "An important yet rarely tackled problem in dialogue state tracking (DST) is\nscalability for dynamic ontology (e.g., movie, restaurant) and unseen slot\nvalues. We focus on a specific condition, where the ontology is unknown to the\nstate tracker, but the target slot value (except for none and dontcare),\npossibly unseen during training, can be found as word segment in the dialogue\ncontext. Prior approaches often rely on candidate generation from n-gram\nenumeration or slot tagger outputs, which can be inefficient or suffer from\nerror propagation. We propose BERT-DST, an end-to-end dialogue state tracker\nwhich directly extracts slot values from the dialogue context. We use BERT as\ndialogue context encoder whose contextualized language representations are\nsuitable for scalable DST to identify slot values from their semantic context.\nFurthermore, we employ encoder parameter sharing across all slots with two\nadvantages: (1) Number of parameters does not grow linearly with the ontology.\n(2) Language representation knowledge can be transferred among slots. Empirical\nevaluation shows BERT-DST with cross-slot parameter sharing outperforms prior\nwork on the benchmark scalable DST datasets Sim-M and Sim-R, and achieves\ncompetitive performance on the standard DSTC2 and WOZ 2.0 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03060v1",
            "date": "2019-07-06 02:14:30+00:00",
            "title": "Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translation",
            "authors": [
                "Aizhan Imankulova",
                "Raj Dabre",
                "Atsushi Fujita",
                "Kenji Imamura"
            ],
            "abstract": "This paper proposes a novel multilingual multistage fine-tuning approach for\nlow-resource neural machine translation (NMT), taking a challenging\nJapanese--Russian pair for benchmarking. Although there are many solutions for\nlow-resource scenarios, such as multilingual NMT and back-translation, we have\nempirically confirmed their limited success when restricted to in-domain data.\nWe therefore propose to exploit out-of-domain data through transfer learning,\nby using it to first train a multilingual NMT model followed by multistage\nfine-tuning on in-domain parallel and back-translated pseudo-parallel data. Our\napproach, which combines domain adaptation, multilingualism, and\nback-translation, helps improve the translation quality by more than 3.7 BLEU\npoints, over a strong baseline, for this extremely low-resource scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03070v1",
            "date": "2019-07-06 03:58:04+00:00",
            "title": "Short Text Conversation Based on Deep Neural Network and Analysis on Evaluation Measures",
            "authors": [
                "Hsiang-En Cherng",
                "Chia-Hui Chang"
            ],
            "abstract": "With the development of Natural Language Processing, Automatic\nquestion-answering system such as Waston, Siri, Alexa, has become one of the\nmost important NLP applications. Nowadays, enterprises try to build automatic\ncustom service chatbots to save human resources and provide a 24-hour customer\nservice. Evaluation of chatbots currently relied greatly on human annotation\nwhich cost a plenty of time. Thus, has initiated a new Short Text Conversation\nsubtask called Dialogue Quality (DQ) and Nugget Detection (ND) which aim to\nautomatically evaluate dialogues generated by chatbots. In this paper, we solve\nthe DQ and ND subtasks by deep neural network. We proposed two models for both\nDQ and ND subtasks which is constructed by hierarchical structure: embedding\nlayer, utterance layer, context layer and memory layer, to hierarchical learn\ndialogue representation from word level, sentence level, context level to long\nrange context level. Furthermore, we apply gating and attention mechanism at\nutterance layer and context layer to improve the performance. We also tried\nBERT to replace embedding layer and utterance layer as sentence representation.\nThe result shows that BERT produced a better utterance representation than\nmulti-stack CNN for both DQ and ND subtasks and outperform other models\nproposed by other researches. The evaluation measures are proposed by , that\nis, NMD, RSNOD for DQ and JSD, RNSS for ND, which is not traditional evaluation\nmeasures such as accuracy, precision, recall and f1-score. Thus, we have done a\nseries of experiments by using traditional evaluation measures and analyze the\nperformance and error.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03112v1",
            "date": "2019-07-06 10:45:45+00:00",
            "title": "Best Practices for Learning Domain-Specific Cross-Lingual Embeddings",
            "authors": [
                "Lena Shakurova",
                "Beata Nyari",
                "Chao Li",
                "Mihai Rotaru"
            ],
            "abstract": "Cross-lingual embeddings aim to represent words in multiple languages in a\nshared vector space by capturing semantic similarities across languages. They\nare a crucial component for scaling tasks to multiple languages by transferring\nknowledge from languages with rich resources to low-resource languages. A\ncommon approach to learning cross-lingual embeddings is to train monolingual\nembeddings separately for each language and learn a linear projection from the\nmonolingual spaces into a shared space, where the mapping relies on a small\nseed dictionary. While there are high-quality generic seed dictionaries and\npre-trained cross-lingual embeddings available for many language pairs, there\nis little research on how they perform on specialised tasks. In this paper, we\ninvestigate the best practices for constructing the seed dictionary for a\nspecific domain. We evaluate the embeddings on the sequence labelling task of\nCurriculum Vitae parsing and show that the size of a bilingual dictionary, the\nfrequency of the dictionary words in the domain corpora and the source of data\n(task-specific vs generic) influence the performance. We also show that the\nless training data is available in the low-resource language, the more the\nconstruction of the bilingual dictionary matters, and demonstrate that some of\nthe choices are crucial in the zero-shot transfer learning case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03187v1",
            "date": "2019-07-06 21:05:54+00:00",
            "title": "Applying a Pre-trained Language Model to Spanish Twitter Humor Prediction",
            "authors": [
                "Bobak Farzin",
                "Piotr Czapla",
                "Jeremy Howard"
            ],
            "abstract": "Our entry into the HAHA 2019 Challenge placed $3^{rd}$ in the classification\ntask and $2^{nd}$ in the regression task. We describe our system and\ninnovations, as well as comparing our results to a Naive Bayes baseline. A\nlarge Twitter based corpus allowed us to train a language model from scratch\nfocused on Spanish and transfer that knowledge to our competition model. To\novercome the inherent errors in some labels we reduce our class confidence with\nlabel smoothing in the loss function. All the code for our project is included\nin a GitHub repository for easy reference and to enable replication by others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03224v1",
            "date": "2019-07-07 05:41:55+00:00",
            "title": "Joint Lifelong Topic Model and Manifold Ranking for Document Summarization",
            "authors": [
                "Jianying Lin",
                "Rui Liu",
                "Quanye Jia"
            ],
            "abstract": "Due to the manifold ranking method has a significant effect on the ranking of\nunknown data based on known data by using a weighted network, many researchers\nuse the manifold ranking method to solve the document summarization task.\nHowever, their models only consider the original features but ignore the\nsemantic features of sentences when they construct the weighted networks for\nthe manifold ranking method. To solve this problem, we proposed two improved\nmodels based on the manifold ranking method. One is combining the topic model\nand manifold ranking method (JTMMR) to solve the document summarization task.\nThis model not only uses the original feature, but also uses the semantic\nfeature to represent the document, which can improve the accuracy of the\nmanifold ranking method. The other one is combining the lifelong topic model\nand manifold ranking method (JLTMMR). On the basis of the JTMMR, this model\nadds the constraint of knowledge to improve the quality of the topic. At the\nsame time, we also add the constraint of the relationship between documents to\ndig out a better document semantic features. The JTMMR model can improve the\neffect of the manifold ranking method by using the better semantic feature.\nExperiments show that our models can achieve a better result than other\nbaseline models for multi-document summarization task. At the same time, our\nmodels also have a good performance on the single document summarization task.\nAfter combining with a few basic surface features, our model significantly\noutperforms some model based on deep learning in recent years. After that, we\nalso do an exploring work for lifelong machine learning by analyzing the effect\nof adding feedback. Experiments show that the effect of adding feedback to our\nmodel is significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03227v1",
            "date": "2019-07-07 06:02:05+00:00",
            "title": "Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Thien Huu Nguyen",
                "Dejing Dou"
            ],
            "abstract": "Event factuality prediction (EFP) is the task of assessing the degree to\nwhich an event mentioned in a sentence has happened. For this task, both\nsyntactic and semantic information are crucial to identify the important\ncontext words. The previous work for EFP has only combined these information in\na simple way that cannot fully exploit their coordination. In this work, we\nintroduce a novel graph-based neural network for EFP that can integrate the\nsemantic and syntactic information more effectively. Our experiments\ndemonstrate the advantage of the proposed model for EFP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03228v1",
            "date": "2019-07-07 06:05:56+00:00",
            "title": "Zero-Shot Open Entity Typing as Type-Compatible Grounding",
            "authors": [
                "Ben Zhou",
                "Daniel Khashabi",
                "Chen-Tse Tsai",
                "Dan Roth"
            ],
            "abstract": "The problem of entity-typing has been studied predominantly in supervised\nlearning fashion, mostly with task-specific annotations (for coarse types) and\nsometimes with distant supervision (for fine types). While such approaches have\nstrong performance within datasets, they often lack the flexibility to transfer\nacross text genres and to generalize to new type taxonomies. In this work we\npropose a zero-shot entity typing approach that requires no annotated data and\ncan flexibly identify newly defined types. Given a type taxonomy defined as\nBoolean functions of FREEBASE \"types\", we ground a given mention to a set of\ntype-compatible Wikipedia entries and then infer the target mention's types\nusing an inference algorithm that makes use of the types of these entries. We\nevaluate our system on a broad range of datasets, including standard\nfine-grained and coarse-grained entity typing datasets, and also a dataset in\nthe biological domain. Our system is shown to be competitive with\nstate-of-the-art supervised NER systems and outperforms them on out-of-domain\ndatasets. We also show that our system significantly outperforms other\nzero-shot fine typing systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03230v1",
            "date": "2019-07-07 06:21:47+00:00",
            "title": "Improving Cross-Domain Performance for Relation Extraction via Dependency Prediction and Information Flow Control",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Thien Huu Nguyen",
                "Dejing Dou"
            ],
            "abstract": "Relation Extraction (RE) is one of the fundamental tasks in Information\nExtraction and Natural Language Processing. Dependency trees have been shown to\nbe a very useful source of information for this task. The current deep learning\nmodels for relation extraction has mainly exploited this dependency information\nby guiding their computation along the structures of the dependency trees. One\npotential problem with this approach is it might prevent the models from\ncapturing important context information beyond syntactic structures and cause\nthe poor cross-domain generalization. This paper introduces a novel method to\nuse dependency trees in RE for deep learning models that jointly predicts\ndependency and semantics relations. We also propose a new mechanism to control\nthe information flow in the model based on the input entity mentions. Our\nextensive experiments on benchmark datasets show that the proposed model\noutperforms the existing methods for RE significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03468v2",
            "date": "2019-07-08 09:09:45+00:00",
            "title": "Correct-and-Memorize: Learning to Translate from Interactive Revisions",
            "authors": [
                "Rongxiang Weng",
                "Hao Zhou",
                "Shujian Huang",
                "Lei Li",
                "Yifan Xia",
                "Jiajun Chen"
            ],
            "abstract": "State-of-the-art machine translation models are still not on par with human\ntranslators. Previous work takes human interactions into the neural machine\ntranslation process to obtain improved results in target languages. However,\nnot all model-translation errors are equal -- some are critical while others\nare minor. In the meanwhile, the same translation mistakes occur repeatedly in\na similar context. To solve both issues, we propose CAMIT, a novel method for\ntranslating in an interactive environment. Our proposed method works with\ncritical revision instructions, therefore allows human to correct arbitrary\nwords in model-translated sentences. In addition, CAMIT learns from and softly\nmemorizes revision actions based on the context, alleviating the issue of\nrepeating mistakes. Experiments in both ideal and real interactive translation\nsettings demonstrate that our proposed \\method enhances machine translation\nresults significantly while requires fewer revision instructions from human\ncompared to previous methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03491v1",
            "date": "2019-07-08 10:17:28+00:00",
            "title": "Searching for Effective Neural Extractive Summarization: What Works and What's Next",
            "authors": [
                "Ming Zhong",
                "Pengfei Liu",
                "Danqing Wang",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "The recent years have seen remarkable success in the use of deep neural\nnetworks on text summarization.\n  However, there is no clear understanding of \\textit{why} they perform so\nwell, or \\textit{how} they might be improved.\n  In this paper, we seek to better understand how neural extractive\nsummarization systems could benefit from different types of model\narchitectures, transferable knowledge and\n  learning schemas. Additionally, we find an effective way to improve current\nframeworks and achieve the state-of-the-art result on CNN/DailyMail by a large\nmargin based on our\n  observations and analyses. Hopefully, our work could provide more clues for\nfuture research on extractive summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03513v1",
            "date": "2019-07-08 11:13:42+00:00",
            "title": "Early Discovery of Emerging Entities in Microblogs",
            "authors": [
                "Satoshi Akasaki",
                "Naoki Yoshinaga",
                "Masashi Toyoda"
            ],
            "abstract": "Keeping up to date on emerging entities that appear every day is\nindispensable for various applications, such as social-trend analysis and\nmarketing research. Previous studies have attempted to detect unseen entities\nthat are not registered in a particular knowledge base as emerging entities and\nconsequently find non-emerging entities since the absence of entities in\nknowledge bases does not guarantee their emergence. We therefore introduce a\nnovel task of discovering truly emerging entities when they have just been\nintroduced to the public through microblogs and propose an effective method\nbased on time-sensitive distant supervision, which exploits distinctive\nearly-stage contexts of emerging entities. Experimental results with a\nlarge-scale Twitter archive show that the proposed method achieves 83.2%\nprecision of the top 500 discovered emerging entities, which outperforms\nbaselines based on unseen entity recognition with burst detection. Besides\nnotable emerging entities, our method can discover massive long-tail and\nhomographic emerging entities. An evaluation of relative recall shows that the\nmethod detects 80.4% emerging entities newly registered in Wikipedia; 92.4% of\nthem are discovered earlier than their registration in Wikipedia, and the\naverage lead-time is more than one year (571 days).",
            "journal ref": "IJCAI2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03663v1",
            "date": "2019-07-08 15:01:17+00:00",
            "title": "Knowledge-aware Pronoun Coreference Resolution",
            "authors": [
                "Hongming Zhang",
                "Yan Song",
                "Yangqiu Song",
                "Dong Yu"
            ],
            "abstract": "Resolving pronoun coreference requires knowledge support, especially for\nparticular domains (e.g., medicine). In this paper, we explore how to leverage\ndifferent types of knowledge to better resolve pronoun coreference with a\nneural model. To ensure the generalization ability of our model, we directly\nincorporate knowledge in the format of triplets, which is the most common\nformat of modern knowledge graphs, instead of encoding it with features or\nrules as that in conventional approaches. Moreover, since not all knowledge is\nhelpful in certain contexts, to selectively use them, we propose a knowledge\nattention module, which learns to select and use informative knowledge based on\ncontexts, to enhance our model. Experimental results on two datasets from\ndifferent domains prove the validity and effectiveness of our model, where it\noutperforms state-of-the-art baselines by a large margin. Moreover, since our\nmodel learns to use external knowledge rather than only fitting the training\ndata, it also demonstrates superior performance to baselines in the\ncross-domain setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03871v1",
            "date": "2019-07-05 12:29:46+00:00",
            "title": "A Study of the Effect of Resolving Negation and Sentiment Analysis in Recognizing Text Entailment for Arabic",
            "authors": [
                "Fatima T. AL-Khawaldeh"
            ],
            "abstract": "Recognizing the entailment relation showed that its influence to extract the\nsemantic inferences in wide-ranging natural language processing domains (text\nsummarization, question answering, etc.) and enhanced the results of their\noutput. For Arabic language, few attempts concerns with Arabic entailment\nproblem. This paper aims to increase the entailment accuracy for Arabic texts\nby resolving negation of the text-hypothesis pair and determining the polarity\nof the text-hypothesis pair whether it is Positive, Negative or Neutral. It is\nnoticed that the absence of negation detection feature gives inaccurate results\nwhen detecting the entailment relation since the negation revers the truth. The\nnegation words are considered stop words and removed from the text-hypothesis\npair which may lead wrong entailment decision. Another case not solved\npreviously, it is impossible that the positive text entails negative text and\nvice versa. In this paper, in order to classify the text-hypothesis pair\npolarity, a sentiment analysis tool is used. We show that analyzing the\npolarity of the text-hypothesis pair increases the entailment accuracy. to\nevaluate our approach we used a dataset for Arabic textual entailment (ArbTEDS)\nconsisted of 618 text-hypothesis pairs and showed that the Arabic entailment\naccuracy is increased by resolving negation for entailment relation and\nanalyzing the polarity of the text-hypothesis pair.",
            "journal ref": "World of Computer Science and Information Technology Journal\n  (WCSIT) ISSN: 2221-0741 Vol. 5, No. 7, 124-128, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03927v1",
            "date": "2019-07-09 01:11:23+00:00",
            "title": "NTT's Machine Translation Systems for WMT19 Robustness Task",
            "authors": [
                "Soichiro Murakami",
                "Makoto Morishita",
                "Tsutomu Hirao",
                "Masaaki Nagata"
            ],
            "abstract": "This paper describes NTT's submission to the WMT19 robustness task. This task\nmainly focuses on translating noisy text (e.g., posts on Twitter), which\npresents different difficulties from typical translation tasks such as news.\nOur submission combined techniques including utilization of a synthetic corpus,\ndomain adaptation, and a placeholder mechanism, which significantly improved\nover the previous baseline. Experimental results revealed the placeholder\nmechanism, which temporarily replaces the non-standard tokens including emojis\nand emoticons with special placeholder tokens during translation, improves\ntranslation accuracy even with noisy texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03975v1",
            "date": "2019-07-09 03:58:46+00:00",
            "title": "Implicit Discourse Relation Identification for Open-domain Dialogues",
            "authors": [
                "Mingyu Derek Ma",
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Wen Cui",
                "Marilyn Walker"
            ],
            "abstract": "Discourse relation identification has been an active area of research for\nmany years, and the challenge of identifying implicit relations remains largely\nan unsolved task, especially in the context of an open-domain dialogue system.\nPrevious work primarily relies on a corpora of formal text which is inherently\nnon-dialogic, i.e., news and journals. This data however is not suitable to\nhandle the nuances of informal dialogue nor is it capable of navigating the\nplethora of valid topics present in open-domain dialogue. In this paper, we\ndesigned a novel discourse relation identification pipeline specifically tuned\nfor open-domain dialogue systems. We firstly propose a method to automatically\nextract the implicit discourse relation argument pairs and labels from a\ndataset of dialogic turns, resulting in a novel corpus of discourse relation\npairs; the first of its kind to attempt to identify the discourse relations\nconnecting the dialogic turns in open-domain discourse. Moreover, we have taken\nthe first steps to leverage the dialogue features unique to our task to further\nimprove the identification of such relations by performing feature ablation and\nincorporating dialogue features to enhance the state-of-the-art model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04126v2",
            "date": "2019-07-09 13:06:31+00:00",
            "title": "Sentiment and position-taking analysis of parliamentary debates: A systematic literature review",
            "authors": [
                "Gavin Abercrombie",
                "Riza Batista-Navarro"
            ],
            "abstract": "Parliamentary and legislative debate transcripts provide access to\ninformation concerning the opinions, positions and policy preferences of\nelected politicians. They attract attention from researchers from a wide\nvariety of backgrounds, from political and social sciences to computer science.\nAs a result, the problem of automatic sentiment and position-taking analysis\nhas been tackled from different perspectives, using varying approaches and\nmethods, and with relatively little collaboration or cross-pollination of\nideas. The existing research is scattered across publications from various\nfields and venues. In this article we present the results of a systematic\nliterature review of 61 studies, all of which address the automatic analysis of\nthe sentiment and opinions expressed and positions taken by speakers in\nparliamentary (and other legislative) debates. In this review, we discuss the\navailable research with regard to the aims and objectives of the researchers\nwho work on these problems, the automatic analysis tasks they undertake, and\nthe approaches and methods they use. We conclude by summarizing their findings,\ndiscussing the challenges of applying computational analysis to parliamentary\ndebates, and suggesting possible avenues for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04307v1",
            "date": "2019-07-09 17:46:17+00:00",
            "title": "Multilingual Universal Sentence Encoder for Semantic Retrieval",
            "authors": [
                "Yinfei Yang",
                "Daniel Cer",
                "Amin Ahmad",
                "Mandy Guo",
                "Jax Law",
                "Noah Constant",
                "Gustavo Hernandez Abrego",
                "Steve Yuan",
                "Chris Tar",
                "Yun-Hsuan Sung",
                "Brian Strope",
                "Ray Kurzweil"
            ],
            "abstract": "We introduce two pre-trained retrieval focused multilingual sentence encoding\nmodels, respectively based on the Transformer and CNN model architectures. The\nmodels embed text from 16 languages into a single semantic space using a\nmulti-task trained dual-encoder that learns tied representations using\ntranslation based bridge tasks (Chidambaram al., 2018). The models provide\nperformance that is competitive with the state-of-the-art on: semantic\nretrieval (SR), translation pair bitext retrieval (BR) and retrieval question\nanswering (ReQA). On English transfer learning tasks, our sentence-level\nembeddings approach, and in some cases exceed, the performance of monolingual,\nEnglish only, sentence embedding models. Our models are made available for\ndownload on TensorFlow Hub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04347v1",
            "date": "2019-07-09 18:05:59+00:00",
            "title": "Cross-Domain Generalization of Neural Constituency Parsers",
            "authors": [
                "Daniel Fried",
                "Nikita Kitaev",
                "Dan Klein"
            ],
            "abstract": "Neural parsers obtain state-of-the-art results on benchmark treebanks for\nconstituency parsing -- but to what degree do they generalize to other domains?\nWe present three results about the generalization of neural parsers in a\nzero-shot setting: training on trees from one corpus and evaluating on\nout-of-domain corpora. First, neural and non-neural parsers generalize\ncomparably to new domains. Second, incorporating pre-trained encoder\nrepresentations into neural parsers substantially improves their performance\nacross all domains, but does not give a larger relative improvement for\nout-of-domain treebanks. Finally, despite the rich input representations they\nlearn, neural parsers still benefit from structured output prediction of output\ntrees, yielding higher exact match accuracy and stronger generalization both to\nlarger text spans and to out-of-domain corpora. We analyze generalization on\nEnglish and Chinese corpora, and in the process obtain state-of-the-art parsing\nresults for the Brown, Genia, and English Web treebanks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04380v1",
            "date": "2019-07-09 19:40:47+00:00",
            "title": "Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference",
            "authors": [
                "Yonatan Belinkov",
                "Adam Poliak",
                "Stuart M. Shieber",
                "Benjamin Van Durme",
                "Alexander M. Rush"
            ],
            "abstract": "Natural Language Inference (NLI) datasets often contain hypothesis-only\nbiases---artifacts that allow models to achieve non-trivial performance without\nlearning whether a premise entails a hypothesis. We propose two probabilistic\nmethods to build models that are more robust to such biases and better transfer\nacross datasets. In contrast to standard approaches to NLI, our methods predict\nthe probability of a premise given a hypothesis and NLI label, discouraging\nmodels from ignoring the premise. We evaluate our methods on synthetic and\nexisting NLI datasets by training on datasets containing biases and testing on\ndatasets containing no (or different) hypothesis-only biases. Our results\nindicate that these methods can make NLI models more robust to dataset-specific\nartifacts, transferring better than a baseline architecture in 9 out of 12 NLI\ndatasets. Additionally, we provide an extensive analysis of the interplay of\nour methods with known biases in NLI datasets, as well as the effects of\nencouraging models to ignore biases and fine-tuning on target datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04389v1",
            "date": "2019-07-09 20:04:24+00:00",
            "title": "On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference",
            "authors": [
                "Yonatan Belinkov",
                "Adam Poliak",
                "Stuart M. Shieber",
                "Benjamin Van Durme",
                "Alexander M. Rush"
            ],
            "abstract": "Popular Natural Language Inference (NLI) datasets have been shown to be\ntainted by hypothesis-only biases. Adversarial learning may help models ignore\nsensitive biases and spurious correlations in data. We evaluate whether\nadversarial learning can be used in NLI to encourage models to learn\nrepresentations free of hypothesis-only biases. Our analyses indicate that the\nrepresentations learned via adversarial learning may be less biased, with only\nsmall drops in NLI accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04492v1",
            "date": "2019-07-10 02:43:37+00:00",
            "title": "Exploiting user-frequency information for mining regionalisms from Social Media texts",
            "authors": [
                "Juan Manuel P\u00e9rez",
                "Dami\u00e1n E. Aleman",
                "Santiago N. Kalinowski",
                "Agust\u00edn Gravano"
            ],
            "abstract": "The task of detecting regionalisms (expressions or words used in certain\nregions) has traditionally relied on the use of questionnaires and surveys, and\nhas also heavily depended on the expertise and intuition of the surveyor. The\nirruption of Social Media and its microblogging services has produced an\nunprecedented wealth of content, mainly informal text generated by users,\nopening new opportunities for linguists to extend their studies of language\nvariation. Previous work on automatic detection of regionalisms depended mostly\non word frequencies. In this work, we present a novel metric based on\nInformation Theory that incorporates user frequency. We tested this metric on a\ncorpus of Argentinian Spanish tweets in two ways: via manual annotation of the\nrelevance of the retrieved terms, and also as a feature selection method for\ngeolocation of users. In either case, our metric outperformed other techniques\nbased solely in word frequency, suggesting that measuring the amount of users\nthat produce a word is informative. This tool has helped lexicographers\ndiscover several unregistered words of Argentinian Spanish, as well as\ndifferent meanings assigned to registered words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04618v1",
            "date": "2019-07-10 11:14:53+00:00",
            "title": "Lingua Custodia at WMT'19: Attempts to Control Terminology",
            "authors": [
                "Franck Burlot"
            ],
            "abstract": "This paper describes Lingua Custodia's submission to the WMT'19 news shared\ntask for German-to-French on the topic of the EU elections. We report\nexperiments on the adaptation of the terminology of a machine translation\nsystem to a specific topic, aimed at providing more accurate translations of\nspecific entities like political parties and person names, given that the\nshared task provided no in-domain training parallel data dealing with the\nrestricted topic. Our primary submission to the shared task uses\nbacktranslation generated with a type of decoding allowing the insertion of\nconstraints in the output in order to guarantee the correct translation of\nspecific terms that are not necessarily observed in the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04744v1",
            "date": "2019-07-10 14:20:43+00:00",
            "title": "Modeling Semantic Compositionality with Sememe Knowledge",
            "authors": [
                "Fanchao Qi",
                "Junjie Huang",
                "Chenghao Yang",
                "Zhiyuan Liu",
                "Xiao Chen",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Semantic compositionality (SC) refers to the phenomenon that the meaning of a\ncomplex linguistic unit can be composed of the meanings of its constituents.\nMost related works focus on using complicated compositionality functions to\nmodel SC while few works consider external knowledge in models. In this paper,\nwe verify the effectiveness of sememes, the minimum semantic units of human\nlanguages, in modeling SC by a confirmatory experiment. Furthermore, we make\nthe first attempt to incorporate sememe knowledge into SC models, and employ\nthe sememeincorporated models in learning representations of multiword\nexpressions, a typical task of SC. In experiments, we implement our models by\nincorporating knowledge from a famous sememe knowledge base HowNet and perform\nboth intrinsic and extrinsic evaluations. Experimental results show that our\nmodels achieve significant performance boost as compared to the baseline\nmethods without considering sememe knowledge. We further conduct quantitative\nanalysis and case studies to demonstrate the effectiveness of applying sememe\nknowledge in modeling SC. All the code and data of this paper can be obtained\non https://github.com/thunlp/Sememe-SC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04780v2",
            "date": "2019-07-10 15:16:36+00:00",
            "title": "ReQA: An Evaluation for End-to-End Answer Retrieval Models",
            "authors": [
                "Amin Ahmad",
                "Noah Constant",
                "Yinfei Yang",
                "Daniel Cer"
            ],
            "abstract": "Popular QA benchmarks like SQuAD have driven progress on the task of\nidentifying answer spans within a specific passage, with models now surpassing\nhuman performance. However, retrieving relevant answers from a huge corpus of\ndocuments is still a challenging problem, and places different requirements on\nthe model architecture. There is growing interest in developing scalable answer\nretrieval models trained end-to-end, bypassing the typical document retrieval\nstep. In this paper, we introduce Retrieval Question-Answering (ReQA), a\nbenchmark for evaluating large-scale sentence-level answer retrieval models. We\nestablish baselines using both neural encoding models as well as classical\ninformation retrieval techniques. We release our evaluation code to encourage\nfurther work on this challenging task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04829v1",
            "date": "2019-07-10 17:14:47+00:00",
            "title": "BAM! Born-Again Multi-Task Networks for Natural Language Understanding",
            "authors": [
                "Kevin Clark",
                "Minh-Thang Luong",
                "Urvashi Khandelwal",
                "Christopher D. Manning",
                "Quoc V. Le"
            ],
            "abstract": "It can be challenging to train multi-task neural networks that outperform or\neven match their single-task counterparts. To help address this, we propose\nusing knowledge distillation where single-task models teach a multi-task model.\nWe enhance this training with teacher annealing, a novel method that gradually\ntransitions the model from distillation to supervised learning, helping the\nmulti-task model surpass its single-task teachers. We evaluate our approach by\nmulti-task fine-tuning BERT on the GLUE benchmark. Our method consistently\nimproves over standard single-task and multi-task training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04954v1",
            "date": "2019-07-10 23:12:17+00:00",
            "title": "Modelling the Socialization of Creative Agents in a Master-Apprentice Setting: The Case of Movie Title Puns",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Khalid Alnajjar"
            ],
            "abstract": "This paper presents work on modelling the social psychological aspect of\nsocialization in the case of a computationally creative master-apprentice\nsystem. In each master-apprentice pair, the master, a genetic algorithm, is\nseen as a parent for its apprentice, which is an NMT based sequence-to-sequence\nmodel. The effect of different parenting styles on the creative output of each\npair is in the focus of this study. This approach brings a novel view point to\ncomputational social creativity, which has mainly focused in the past on\ncomputationally creative agents being on a socially equal level, whereas our\napproach studies the phenomenon in the context of a social hierarchy.",
            "journal ref": "Proceedings of the 10th International Conference on Computational\n  Creativity. Grace, K., Cook, M., Ventura, D. & Maher, M. L. (eds.).\n  Association for Computational Creativity, p. 266-273 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05048v1",
            "date": "2019-07-11 08:41:53+00:00",
            "title": "No Word is an Island -- A Transformation Weighting Model for Semantic Composition",
            "authors": [
                "Corina Dima",
                "Dani\u00ebl de Kok",
                "Neele Witte",
                "Erhard Hinrichs"
            ],
            "abstract": "Composition models of distributional semantics are used to construct phrase\nrepresentations from the representations of their words. Composition models are\ntypically situated on two ends of a spectrum. They either have a small number\nof parameters but compose all phrases in the same way, or they perform\nword-specific compositions at the cost of a far larger number of parameters. In\nthis paper we propose transformation weighting (TransWeight), a composition\nmodel that consistently outperforms existing models on nominal compounds,\nadjective-noun phrases and adverb-adjective phrases in English, German and\nDutch. TransWeight drastically reduces the number of parameters needed compared\nto the best model in the literature by composing similar words in the same way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05340v1",
            "date": "2019-07-09 04:39:24+00:00",
            "title": "Neural or Statistical: An Empirical Study on Language Models for Chinese Input Recommendation on Mobile",
            "authors": [
                "Hainan Zhang",
                "Yanyan Lan",
                "Jiafeng Guo",
                "Jun Xu",
                "Xueqi Cheng"
            ],
            "abstract": "Chinese input recommendation plays an important role in alleviating human\ncost in typing Chinese words, especially in the scenario of mobile\napplications. The fundamental problem is to predict the conditional probability\nof the next word given the sequence of previous words. Therefore, statistical\nlanguage models, i.e.~n-grams based models, have been extensively used on this\ntask in real application. However, the characteristics of extremely different\ntyping behaviors usually lead to serious sparsity problem, even n-gram with\nsmoothing will fail. A reasonable approach to tackle this problem is to use the\nrecently proposed neural models, such as probabilistic neural language model,\nrecurrent neural network and word2vec. They can leverage more semantically\nsimilar words for estimating the probability. However, there is no conclusion\non which approach of the two will work better in real application. In this\npaper, we conduct an extensive empirical study to show the differences between\nstatistical and neural language models. The experimental results show that the\ntwo different approach have individual advantages, and a hybrid approach will\nbring a significant improvement.",
            "journal ref": "LNCS2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05403v1",
            "date": "2019-07-11 17:35:20+00:00",
            "title": "Incrementalizing RASA's Open-Source Natural Language Understanding Pipeline",
            "authors": [
                "Andrew Rafla",
                "Casey Kennington"
            ],
            "abstract": "As spoken dialogue systems and chatbots are gaining more widespread adoption,\ncommercial and open-sourced services for natural language understanding are\nemerging. In this paper, we explain how we altered the open-source RASA natural\nlanguage understanding pipeline to process incrementally (i.e., word-by-word),\nfollowing the incremental unit framework proposed by Schlangen and Skantze. To\ndo so, we altered existing RASA components to process incrementally, and added\nan update-incremental intent recognition model as a component to RASA. Our\nevaluations on the Snips dataset show that our changes allow RASA to function\nas an effective incremental natural language understanding service.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05524v1",
            "date": "2019-07-11 23:40:56+00:00",
            "title": "Solving Hard Coreference Problems",
            "authors": [
                "Haoruo Peng",
                "Daniel Khashabi",
                "Dan Roth"
            ],
            "abstract": "Coreference resolution is a key problem in natural language understanding\nthat still escapes reliable solutions. One fundamental difficulty has been that\nof resolving instances involving pronouns since they often require deep\nlanguage understanding and use of background knowledge. In this paper, we\npropose an algorithmic solution that involves a new representation for the\nknowledge required to address hard coreference problems, along with a\nconstrained optimization framework that uses this knowledge in coreference\ndecision making. Our representation, Predicate Schemas, is instantiated with\nknowledge acquired in an unsupervised way, and is compiled automatically into\nconstraints that impact the coreference decision. We present a general\ncoreference resolution system that significantly improves state-of-the-art\nperformance on hard, Winograd-style, pronoun resolution cases, while still\nperforming at the state-of-the-art level on standard coreference resolution\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05611v2",
            "date": "2019-07-12 08:16:26+00:00",
            "title": "GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition",
            "authors": [
                "Hui Chen",
                "Zijia Lin",
                "Guiguang Ding",
                "Jianguang Lou",
                "Yusen Zhang",
                "Borje Karlsson"
            ],
            "abstract": "The dominant approaches for named entity recognition (NER) mostly adopt\ncomplex recurrent neural networks (RNN), e.g., long-short-term-memory (LSTM).\nHowever, RNNs are limited by their recurrent nature in terms of computational\nefficiency. In contrast, convolutional neural networks (CNN) can fully exploit\nthe GPU parallelism with their feedforward architectures. However, little\nattention has been paid to performing NER with CNNs, mainly owing to their\ndifficulties in capturing the long-term context information in a sequence. In\nthis paper, we propose a simple but effective CNN-based network for NER, i.e.,\ngated relation network (GRN), which is more capable than common CNNs in\ncapturing long-term context. Specifically, in GRN we firstly employ CNNs to\nexplore the local context features of each word. Then we model the relations\nbetween words and use them as gates to fuse local context features into global\nones for predicting labels. Without using recurrent layers that process a\nsentence in a sequential manner, our GRN allows computations to be performed in\nparallel across the entire sentence. Experiments on two benchmark NER datasets\n(i.e., CoNLL2003 and Ontonotes 5.0) show that, our proposed GRN can achieve\nstate-of-the-art performance with or without external knowledge. It also enjoys\nlower time costs to train and test.We have made the code publicly available at\nhttps://github.com/HuiChen24/NER-GRN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05757v1",
            "date": "2019-07-12 14:13:53+00:00",
            "title": "Automated Word Stress Detection in Russian",
            "authors": [
                "Maria Ponomareva",
                "Kirill Milintsevich",
                "Ekaterina Chernyak",
                "Anatoly Starostin"
            ],
            "abstract": "In this study we address the problem of automated word stress detection in\nRussian using character level models and no part-speech-taggers. We use a\nsimple bidirectional RNN with LSTM nodes and achieve the accuracy of 90% or\nhigher. We experiment with two training datasets and show that using the data\nfrom an annotated corpus is much more efficient than using a dictionary, since\nit allows us to take into account word frequencies and the morphological\ncontext of the word.",
            "journal ref": "Published in Proceedings of the First Workshop on Subword and\n  Character Level Models in NLP, pages 31 35, Copenhagen, Denmark, September 7,\n  2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05774v2",
            "date": "2019-07-12 14:55:06+00:00",
            "title": "Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems",
            "authors": [
                "Pawe\u0142 Budzianowski",
                "Ivan Vuli\u0107"
            ],
            "abstract": "Data scarcity is a long-standing and crucial challenge that hinders quick\ndevelopment of task-oriented dialogue systems across multiple domains:\ntask-oriented dialogue models are expected to learn grammar, syntax, dialogue\nreasoning, decision making, and language generation from absurdly small amounts\nof task-specific data. In this paper, we demonstrate that recent progress in\nlanguage modeling pre-training and transfer learning shows promise to overcome\nthis problem. We propose a task-oriented dialogue model that operates solely on\ntext input: it effectively bypasses explicit policy and language generation\nmodules. Building on top of the TransferTransfo framework (Wolf et al., 2019)\nand generative model pre-training (Radford et al., 2019), we validate the\napproach on complex multi-domain task-oriented dialogues from the MultiWOZ\ndataset. Our automatic and human evaluations show that the proposed model is on\npar with a strong task-specific neural baseline. In the long run, our approach\nholds promise to mitigate the data scarcity problem, and to support the\nconstruction of more engaging and more eloquent task-oriented conversational\nagents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05789v1",
            "date": "2019-07-06 04:40:48+00:00",
            "title": "Generating Sentences from Disentangled Syntactic and Semantic Spaces",
            "authors": [
                "Yu Bao",
                "Hao Zhou",
                "Shujian Huang",
                "Lei Li",
                "Lili Mou",
                "Olga Vechtomova",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Variational auto-encoders (VAEs) are widely used in natural language\ngeneration due to the regularization of the latent space. However, generating\nsentences from the continuous latent space does not explicitly model the\nsyntactic information. In this paper, we propose to generate sentences from\ndisentangled syntactic and semantic spaces. Our proposed method explicitly\nmodels syntactic information in the VAE's latent space by using the linearized\ntree sequence, leading to better performance of language generation.\nAdditionally, the advantage of sampling in the disentangled syntactic and\nsemantic latent spaces enables us to perform novel applications, such as the\nunsupervised paraphrase generation and syntax-transfer generation. Experimental\nresults show that our proposed model achieves similar or better performance in\nvarious tasks, compared with state-of-the-art related work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05791v2",
            "date": "2019-07-10 23:57:30+00:00",
            "title": "WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia",
            "authors": [
                "Holger Schwenk",
                "Vishrav Chaudhary",
                "Shuo Sun",
                "Hongyu Gong",
                "Francisco Guzm\u00e1n"
            ],
            "abstract": "We present an approach based on multilingual sentence embeddings to\nautomatically extract parallel sentences from the content of Wikipedia articles\nin 85 languages, including several dialects or low-resource languages. We do\nnot limit the the extraction process to alignments with English, but\nsystematically consider all possible language pairs. In total, we are able to\nextract 135M parallel sentences for 1620 different language pairs, out of which\nonly 34M are aligned with English. This corpus of parallel sentences is freely\navailable at\nhttps://github.com/facebookresearch/LASER/tree/master/tasks/WikiMatrix. To get\nan indication on the quality of the extracted bitexts, we train neural MT\nbaseline systems on the mined data only for 1886 languages pairs, and evaluate\nthem on the TED corpus, achieving strong BLEU scores for many language pairs.\nThe WikiMatrix bitexts seem to be particularly interesting to train MT systems\nbetween distant languages without the need to pivot through English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05854v1",
            "date": "2019-07-12 17:23:30+00:00",
            "title": "The University of Edinburgh's Submissions to the WMT19 News Translation Task",
            "authors": [
                "Rachel Bawden",
                "Nikolay Bogoychev",
                "Ulrich Germann",
                "Roman Grundkiewicz",
                "Faheem Kirefu",
                "Antonio Valerio Miceli Barone",
                "Alexandra Birch"
            ],
            "abstract": "The University of Edinburgh participated in the WMT19 Shared Task on News\nTranslation in six language directions: English-to-Gujarati,\nGujarati-to-English, English-to-Chinese, Chinese-to-English, German-to-English,\nand English-to-Czech. For all translation directions, we created or used\nback-translations of monolingual data in the target language as additional\nsynthetic training data. For English-Gujarati, we also explored semi-supervised\nMT with cross-lingual language model pre-training, and translation pivoting\nthrough Hindi. For translation to and from Chinese, we investigated\ncharacter-based tokenisation vs. sub-word segmentation of Chinese text. For\nGerman-to-English, we studied the impact of vast amounts of back-translated\ntraining data on translation quality, gaining a few additional insights over\nEdunov et al. (2018). For English-to-Czech, we compared different\npre-processing and tokenisation regimes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06042v1",
            "date": "2019-07-13 10:04:37+00:00",
            "title": "Cross-Lingual Transfer Learning for Question Answering",
            "authors": [
                "Chia-Hsuan Lee",
                "Hung-Yi Lee"
            ],
            "abstract": "Deep learning based question answering (QA) on English documents has achieved\nsuccess because there is a large amount of English training examples. However,\nfor most languages, training examples for high-quality QA models are not\navailable. In this paper, we explore the problem of cross-lingual transfer\nlearning for QA, where a source language task with plentiful annotations is\nutilized to improve the performance of a QA model on a target language task\nwith limited available annotations. We examine two different approaches. A\nmachine translation (MT) based approach translates the source language into the\ntarget language, or vice versa. Although the MT-based approach brings\nimprovement, it assumes the availability of a sentence-level translation\nsystem. A GAN-based approach incorporates a language discriminator to learn\nlanguage-universal feature representations, and consequentially transfer\nknowledge from the source language. The GAN-based approach rivals the\nperformance of the MT-based approach with fewer linguistic resources. Applying\nboth approaches simultaneously yield the best results. We use two English\nbenchmark datasets, SQuAD and NewsQA, as source language data, and show\nsignificant improvements over a number of established baselines on a Chinese QA\ntask. We achieve the new state-of-the-art on the Chinese QA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06080v2",
            "date": "2019-07-13 14:01:27+00:00",
            "title": "A Relational Memory-based Embedding Model for Triple Classification and Search Personalization",
            "authors": [
                "Dai Quoc Nguyen",
                "Tu Dinh Nguyen",
                "Dinh Phung"
            ],
            "abstract": "Knowledge graph embedding methods often suffer from a limitation of\nmemorizing valid triples to predict new ones for triple classification and\nsearch personalization problems. To this end, we introduce a novel embedding\nmodel, named R-MeN, that explores a relational memory network to encode\npotential dependencies in relationship triples. R-MeN considers each triple as\na sequence of 3 input vectors that recurrently interact with a memory using a\ntransformer self-attention mechanism. Thus R-MeN encodes new information from\ninteractions between the memory and each input vector to return a corresponding\nvector. Consequently, R-MeN feeds these 3 returned vectors to a convolutional\nneural network-based decoder to produce a scalar score for the triple.\nExperimental results show that our proposed R-MeN obtains state-of-the-art\nresults on SEARCH17 for the search personalization task, and on WN11 and FB13\nfor the triple classification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06170v1",
            "date": "2019-07-14 05:47:53+00:00",
            "title": "Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation",
            "authors": [
                "Marcin Junczys-Dowmunt"
            ],
            "abstract": "This paper describes the Microsoft Translator submissions to the WMT19 news\ntranslation shared task for English-German. Our main focus is document-level\nneural machine translation with deep transformer models. We start with strong\nsentence-level baselines, trained on large-scale data created via\ndata-filtering and noisy back-translation and find that back-translation seems\nto mainly help with translationese input. We explore fine-tuning techniques,\ndeeper models and different ensembling strategies to counter these effects.\nUsing document boundaries present in the authentic and synthetic parallel data,\nwe create sequences of up to 1000 subword segments and train transformer\ntranslation models. We experiment with data augmentation techniques for the\nsmaller authentic data with document-boundaries and for larger authentic data\nwithout boundaries. We further explore multi-task training for the\nincorporation of document-level source language monolingual data via the\nBERT-objective on the encoder and two-pass decoding for combinations of\nsentence-level and document-level systems. Based on preliminary human\nevaluation results, evaluators strongly prefer the document-level systems over\nour comparable sentence-level system. The document-level systems also seem to\nscore higher than the human references in source-based direct assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06210v1",
            "date": "2019-07-14 11:56:20+00:00",
            "title": "Simple Automatic Post-editing for Arabic-Japanese Machine Translation",
            "authors": [
                "Ella Noll",
                "Mai Oudah",
                "Nizar Habash"
            ],
            "abstract": "A common bottleneck for developing machine translation (MT) systems for some\nlanguage pairs is the lack of direct parallel translation data sets, in general\nand in certain domains. Alternative solutions such as zero-shot models or\npivoting techniques are successful in getting a strong baseline, but are often\nbelow the more supported language-pair systems. In this paper, we focus on\nArabic-Japanese machine translation, a less studied language pair; and we work\nwith a unique parallel corpus of Arabic news articles that were manually\ntranslated to Japanese. We use this parallel corpus to adapt a state-of-the-art\ndomain/genre agnostic neural MT system via a simple automatic post-editing\ntechnique. Our results and detailed analysis suggest that this approach is\nquite viable for less supported language pairs in specific domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06292v1",
            "date": "2019-07-14 22:20:59+00:00",
            "title": "TWEETQA: A Social Media Focused Question Answering Dataset",
            "authors": [
                "Wenhan Xiong",
                "Jiawei Wu",
                "Hong Wang",
                "Vivek Kulkarni",
                "Mo Yu",
                "Shiyu Chang",
                "Xiaoxiao Guo",
                "William Yang Wang"
            ],
            "abstract": "With social media becoming increasingly pop-ular on which lots of news and\nreal-time eventsare reported, developing automated questionanswering systems is\ncritical to the effective-ness of many applications that rely on real-time\nknowledge. While previous datasets haveconcentrated on question answering (QA)\nforformal text like news and Wikipedia, wepresent the first large-scale dataset\nfor QA oversocial media data. To ensure that the tweetswe collected are useful,\nwe only gather tweetsused by journalists to write news articles. Wethen ask\nhuman annotators to write questionsand answers upon these tweets. Unlike\notherQA datasets like SQuAD in which the answersare extractive, we allow the\nanswers to be ab-stractive. We show that two recently proposedneural models\nthat perform well on formaltexts are limited in their performance when ap-plied\nto our dataset. In addition, even the fine-tuned BERT model is still lagging\nbehind hu-man performance with a large margin. Our re-sults thus point to the\nneed of improved QAsystems targeting social media text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06488v1",
            "date": "2019-07-15 13:19:44+00:00",
            "title": "Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Task",
            "authors": [
                "Alexandre B\u00e9rard",
                "Ioan Calapodescu",
                "Claude Roux"
            ],
            "abstract": "This paper describes the systems that we submitted to the WMT19 Machine\nTranslation robustness task. This task aims to improve MT's robustness to noise\nfound on social media, like informal language, spelling mistakes and other\northographic variations. The organizers provide parallel data extracted from a\nsocial media website in two language pairs: French-English and Japanese-English\n(in both translation directions). The goal is to obtain the best scores on\nunseen test sets from the same source, according to automatic metrics (BLEU)\nand human evaluation. We proposed one single and one ensemble system for each\ntranslation direction. Our ensemble models ranked first in all language pairs,\naccording to BLEU evaluation. We discuss the pre-processing choices that we\nmade, and present our solutions for robustness to noise and domain adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06616v1",
            "date": "2019-07-15 17:22:54+00:00",
            "title": "Facebook FAIR's WMT19 News Translation Task Submission",
            "authors": [
                "Nathan Ng",
                "Kyra Yee",
                "Alexei Baevski",
                "Myle Ott",
                "Michael Auli",
                "Sergey Edunov"
            ],
            "abstract": "This paper describes Facebook FAIR's submission to the WMT19 shared news\ntranslation task. We participate in two language pairs and four language\ndirections, English <-> German and English <-> Russian. Following our\nsubmission from last year, our baseline systems are large BPE-based transformer\nmodels trained with the Fairseq sequence modeling toolkit which rely on sampled\nback-translations. This year we experiment with different bitext data filtering\nschemes, as well as with adding filtered back-translated data. We also ensemble\nand fine-tune our models on domain-specific data, then decode using noisy\nchannel model reranking. Our submissions are ranked first in all four\ndirections of the human evaluation campaign. On En->De, our system\nsignificantly outperforms other systems as well as human translations. This\nsystem improves upon our WMT'18 submission by 4.5 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06944v2",
            "date": "2019-07-16 11:33:04+00:00",
            "title": "Language comparison via network topology",
            "authors": [
                "Bla\u017e \u0160krlj",
                "Senja Pollak"
            ],
            "abstract": "Modeling relations between languages can offer understanding of language\ncharacteristics and uncover similarities and differences between languages.\nAutomated methods applied to large textual corpora can be seen as opportunities\nfor novel statistical studies of language development over time, as well as for\nimproving cross-lingual natural language processing techniques. In this work,\nwe first propose how to represent textual data as a directed, weighted network\nby the text2net algorithm. We next explore how various fast,\nnetwork-topological metrics, such as network community structure, can be used\nfor cross-lingual comparisons. In our experiments, we employ eight different\nnetwork topology metrics, and empirically showcase on a parallel corpus, how\nthe methods can be used for modeling the relations between nine selected\nlanguages. We demonstrate that the proposed method scales to large corpora\nconsisting of hundreds of thousands of aligned sentences on an of-the-shelf\nlaptop. We observe that on the one hand properties such as communities, capture\nsome of the known differences between the languages, while others can be seen\nas novel opportunities for linguistic studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07073v1",
            "date": "2019-07-16 15:34:58+00:00",
            "title": "RadioTalk: a large-scale corpus of talk radio transcripts",
            "authors": [
                "Doug Beeferman",
                "William Brannon",
                "Deb Roy"
            ],
            "abstract": "We introduce RadioTalk, a corpus of speech recognition transcripts sampled\nfrom talk radio broadcasts in the United States between October of 2018 and\nMarch of 2019. The corpus is intended for use by researchers in the fields of\nnatural language processing, conversational analysis, and the social sciences.\nThe corpus encompasses approximately 2.8 billion words of automatically\ntranscribed speech from 284,000 hours of radio, together with metadata about\nthe speech, such as geographical location, speaker turn boundaries, gender, and\nradio program information. In this paper we summarize why and how we prepared\nthe corpus, give some descriptive statistics on stations, shows and speakers,\nand carry out a few high-level analyses.",
            "journal ref": "Proc. Interspeech 2019, 564-568 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07265v1",
            "date": "2019-07-16 21:20:49+00:00",
            "title": "You Write Like You Eat: Stylistic variation as a predictor of social stratification",
            "authors": [
                "Angelo Basile",
                "Albert Gatt",
                "Malvina Nissim"
            ],
            "abstract": "Inspired by Labov's seminal work on stylistic variation as a function of\nsocial stratification, we develop and compare neural models that predict a\nperson's presumed socio-economic status, obtained through distant\nsupervision,from their writing style on social media. The focus of our work is\non identifying the most important stylistic parameters to predict\nsocio-economic group. In particular, we show the effectiveness of\nmorpho-syntactic features as stylistic predictors of socio-economic group,in\ncontrast to lexical features, which are good predictors of topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07328v1",
            "date": "2019-07-17 04:29:27+00:00",
            "title": "Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering",
            "authors": [
                "Peng Wu",
                "Shujian Huang",
                "Rongxiang Weng",
                "Zaixiang Zheng",
                "Jianbing Zhang",
                "Xiaohui Yan",
                "Jiajun Chen"
            ],
            "abstract": "Relation detection is a core step in many natural language process\napplications including knowledge base question answering. Previous efforts show\nthat single-fact questions could be answered with high accuracy. However, one\ncritical problem is that current approaches only get high accuracy for\nquestions whose relations have been seen in the training data. But for unseen\nrelations, the performance will drop rapidly. The main reason for this problem\nis that the representations for unseen relations are missing. In this paper, we\npropose a simple mapping method, named representation adapter, to learn the\nrepresentation mapping for both seen and unseen relations based on previously\nlearned relation embedding. We employ the adversarial objective and the\nreconstruction objective to improve the mapping performance. We re-organize the\npopular SimpleQuestion dataset to reveal and evaluate the problem of detecting\nunseen relations. Experiments show that our method can greatly improve the\nperformance of unseen relations while the performance for those seen part is\nkept comparable to the state-of-the-art. Our code and data are available at\nhttps://github.com/wudapeng268/KBQA-Adapter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07347v1",
            "date": "2019-07-17 06:03:17+00:00",
            "title": "Fake News Detection as Natural Language Inference",
            "authors": [
                "Kai-Chou Yang",
                "Timothy Niven",
                "Hung-Yu Kao"
            ],
            "abstract": "This report describes the entry by the Intelligent Knowledge Management (IKM)\nLab in the WSDM 2019 Fake News Classification challenge. We treat the task as\nnatural language inference (NLI). We individually train a number of the\nstrongest NLI models as well as BERT. We ensemble these results and retrain\nwith noisy labels in two stages. We analyze transitivity relations in the train\nand test sets and determine a set of test cases that can be reliably classified\non this basis. The remainder of test cases are classified by our ensemble. Our\nentry achieves test set accuracy of 88.063% for 3rd place in the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07355v2",
            "date": "2019-07-17 06:26:20+00:00",
            "title": "Probing Neural Network Comprehension of Natural Language Arguments",
            "authors": [
                "Timothy Niven",
                "Hung-Yu Kao"
            ],
            "abstract": "We are surprised to find that BERT's peak performance of 77% on the Argument\nReasoning Comprehension Task reaches just three points below the average\nuntrained human baseline. However, we show that this result is entirely\naccounted for by exploitation of spurious statistical cues in the dataset. We\nanalyze the nature of these cues and demonstrate that a range of models all\nexploit them. This analysis informs the construction of an adversarial dataset\non which all models achieve random accuracy. Our adversarial dataset provides a\nmore robust assessment of argument comprehension and should be adopted as the\nstandard in future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07507v2",
            "date": "2019-07-17 13:31:31+00:00",
            "title": "Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe",
            "authors": [
                "Guntis Barzdins",
                "Eduards Sidorovics"
            ],
            "abstract": "It has long been speculated that deep neural networks function by discovering\na hierarchical set of domain-specific core concepts or patterns, which are\nfurther combined to recognize even more elaborate concepts for the\nclassification or other machine learning tasks. Meanwhile disentangling the\nactual core concepts engrained in the word embeddings (like word2vec or BERT)\nor deep convolutional image recognition neural networks (like PG-GAN) is\ndifficult and some success there has been achieved only recently. In this paper\nwe propose a novel neural network nonlinearity named Differentiable\nDisentanglement Filter (DDF) which can be transparently inserted into any\nexisting neural network layer to automatically disentangle the core concepts\nused by that layer. The DDF probe is inspired by the obscure properties of the\nhyper-dimensional computing theory. The DDF proof-of-concept implementation is\nshown to disentangle concepts within the neural 3D scene representation - a\ntask vital for visual grounding of natural language narratives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07526v1",
            "date": "2019-07-17 13:52:54+00:00",
            "title": "Almawave-SLU: A new dataset for SLU in Italian",
            "authors": [
                "Valentina Bellomaria",
                "Giuseppe Castellucci",
                "Andrea Favalli",
                "Raniero Romagnoli"
            ],
            "abstract": "The widespread use of conversational and question answering systems made it\nnecessary to improve the performances of speaker intent detection and\nunderstanding of related semantic slots, i.e., Spoken Language Understanding\n(SLU). Often, these tasks are approached with supervised learning methods,\nwhich needs considerable labeled datasets. This paper presents the first\nItalian dataset for SLU. It is derived through a semi-automatic procedure and\nis used as a benchmark of various open source and commercial systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07950v2",
            "date": "2019-07-18 09:37:38+00:00",
            "title": "What Should/Do/Can LSTMs Learn When Parsing Auxiliary Verb Constructions?",
            "authors": [
                "Miryam de Lhoneux",
                "Sara Stymne",
                "Joakim Nivre"
            ],
            "abstract": "There is a growing interest in investigating what neural NLP models learn\nabout language. A prominent open question is the question of whether or not it\nis necessary to model hierarchical structure. We present a linguistic\ninvestigation of a neural parser adding insights to this question. We look at\ntransitivity and agreement information of auxiliary verb constructions (AVCs)\nin comparison to finite main verbs (FMVs). This comparison is motivated by\ntheoretical work in dependency grammar and in particular the work of Tesni\\`ere\n(1959) where AVCs and FMVs are both instances of a nucleus, the basic unit of\nsyntax. An AVC is a dissociated nucleus, it consists of at least two words, and\nan FMV is its non-dissociated counterpart, consisting of exactly one word. We\nsuggest that the representation of AVCs and FMVs should capture similar\ninformation. We use diagnostic classifiers to probe agreement and transitivity\ninformation in vectors learned by a transition-based neural parser in four\ntypologically different languages. We find that the parser learns different\ninformation about AVCs and FMVs if only sequential models (BiLSTMs) are used in\nthe architecture but similar information when a recursive layer is used. We\nfind explanations for why this is the case by looking closely at how\ninformation is learned in the network and looking at what happens with\ndifferent dependency representations of AVCs. We conclude that there may be\nbenefits to using a recursive layer in dependency parsing and that we have not\nyet found the best way to integrate it in our parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08158v1",
            "date": "2019-07-18 16:59:40+00:00",
            "title": "Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models",
            "authors": [
                "Gongbo Tang",
                "Rico Sennrich",
                "Joakim Nivre"
            ],
            "abstract": "In this paper, we try to understand neural machine translation (NMT) via\nsimplifying NMT architectures and training encoder-free NMT models. In an\nencoder-free model, the sums of word embeddings and positional embeddings\nrepresent the source. The decoder is a standard Transformer or recurrent neural\nnetwork that directly attends to embeddings via attention mechanisms.\nExperimental results show (1) that the attention mechanism in encoder-free\nmodels acts as a strong feature extractor, (2) that the word embeddings in\nencoder-free models are competitive to those in conventional models, (3) that\nnon-contextualized source representations lead to a big performance drop, and\n(4) that encoder-free models have different effects on alignment quality for\nGerman-English and Chinese-English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08184v1",
            "date": "2019-07-18 17:47:22+00:00",
            "title": "Evaluating the Utility of Document Embedding Vector Difference for Relation Learning",
            "authors": [
                "Jingyuan Zhang",
                "Timothy Baldwin"
            ],
            "abstract": "Recent work has demonstrated that vector offsets obtained by subtracting\npretrained word embedding vectors can be used to predict lexical relations with\nsurprising accuracy. Inspired by this finding, in this paper, we extend the\nidea to the document level, in generating document-level embeddings,\ncalculating the distance between them, and using a linear classifier to\nclassify the relation between the documents. In the context of duplicate\ndetection and dialogue act tagging tasks, we show that document-level\ndifference vectors have utility in assessing document-level similarity, but\nperform less well in multi-relational classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08236v1",
            "date": "2019-07-18 18:32:16+00:00",
            "title": "SLATE: A Super-Lightweight Annotation Tool for Experts",
            "authors": [
                "Jonathan K. Kummerfeld"
            ],
            "abstract": "Many annotation tools have been developed, covering a wide variety of tasks\nand providing features like user management, pre-processing, and automatic\nlabeling. However, all of these tools use Graphical User Interfaces, and often\nrequire substantial effort to install and configure. This paper presents a new\nannotation tool that is designed to fill the niche of a lightweight interface\nfor users with a terminal-based workflow. Slate supports annotation at\ndifferent scales (spans of characters, tokens, and lines, or a document) and of\ndifferent types (free text, labels, and links), with easily customisable\nkeybindings, and unicode support. In a user study comparing with other tools it\nwas consistently the easiest to install and use. Slate fills a need not met by\nexisting systems, and has already been used to annotate two corpora, one of\nwhich involved over 250 hours of annotation effort.",
            "journal ref": "ACL: Demonstrations (2019) 7-12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08243v1",
            "date": "2019-07-18 18:47:33+00:00",
            "title": "Joint Learning of Named Entity Recognition and Entity Linking",
            "authors": [
                "Pedro Henrique Martins",
                "Zita Marinho",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Named entity recognition (NER) and entity linking (EL) are two fundamentally\nrelated tasks, since in order to perform EL, first the mentions to entities\nhave to be detected. However, most entity linking approaches disregard the\nmention detection part, assuming that the correct mentions have been previously\ndetected. In this paper, we perform joint learning of NER and EL to leverage\ntheir relatedness and obtain a more robust and generalisable system. For that,\nwe introduce a model inspired by the Stack-LSTM approach (Dyer et al., 2015).\nWe observe that, in fact, doing multi-task learning of NER and EL improves the\nperformance in both tasks when comparing with models trained with individual\nobjectives. Furthermore, we achieve results competitive with the\nstate-of-the-art in both NER and EL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08469v2",
            "date": "2019-07-19 11:39:41+00:00",
            "title": "Exploring sentence informativeness",
            "authors": [
                "Syrielle Montariol",
                "Aina Gar\u00ed Soler",
                "Alexandre Allauzen"
            ],
            "abstract": "This study is a preliminary exploration of the concept of informativeness\n-how much information a sentence gives about a word it contains- and its\npotential benefits to building quality word representations from scarce data.\nWe propose several sentence-level classifiers to predict informativeness, and\nwe perform a manual annotation on a set of sentences. We conclude that these\ntwo measures correspond to different notions of informativeness. However, our\nexperiments show that using the classifiers' predictions to train word\nembeddings has an impact on embedding quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08540v1",
            "date": "2019-07-19 15:02:16+00:00",
            "title": "Predicting Human Activities from User-Generated Content",
            "authors": [
                "Steven R. Wilson",
                "Rada Mihalcea"
            ],
            "abstract": "The activities we do are linked to our interests, personality, political\npreferences, and decisions we make about the future. In this paper, we explore\nthe task of predicting human activities from user-generated content. We collect\na dataset containing instances of social media users writing about a range of\neveryday activities. We then use a state-of-the-art sentence embedding\nframework tailored to recognize the semantics of human activities and perform\nan automatic clustering of these activities. We train a neural network model to\nmake predictions about which clusters contain activities that were performed by\na given user based on the text of their previous posts and self-description.\nAdditionally, we explore the degree to which incorporating inferred user traits\ninto our model helps with this prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08672v1",
            "date": "2019-07-19 20:09:03+00:00",
            "title": "Discourse-Based Evaluation of Language Understanding",
            "authors": [
                "Damien Sileo",
                "Tim Van-de-Cruys",
                "Camille Pradel",
                "Philippe Muller"
            ],
            "abstract": "We introduce DiscEval, a compilation of $11$ evaluation datasets with a focus\non discourse, that can be used for evaluation of English Natural Language\nUnderstanding when considering meaning as use. We make the case that evaluation\nwith discourse tasks is overlooked and that Natural Language Inference (NLI)\npretraining may not lead to the learning really universal representations.\nDiscEval can also be used as supplementary training data for multi-task\nlearning-based systems, and is publicly available, alongside the code for\ngathering and preprocessing the datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08722v1",
            "date": "2019-07-19 22:57:21+00:00",
            "title": "What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks",
            "authors": [
                "Shashi Narayan",
                "Shay B. Cohen",
                "Mirella Lapata"
            ],
            "abstract": "We introduce 'extreme summarization', a new single-document summarization\ntask which aims at creating a short, one-sentence news summary answering the\nquestion ``What is the article about?''. We argue that extreme summarization,\nby nature, is not amenable to extractive strategies and requires an abstractive\nmodeling approach. In the hope of driving research on this task further: (a) we\ncollect a real-world, large scale dataset by harvesting online articles from\nthe British Broadcasting Corporation (BBC); and (b) propose a novel abstractive\nmodel which is conditioned on the article's topics and based entirely on\nconvolutional neural networks. We demonstrate experimentally that this\narchitecture captures long-range dependencies in a document and recognizes\npertinent content, outperforming an oracle extractive system and\nstate-of-the-art abstractive approaches when evaluated automatically and by\nhumans on the extreme summarization dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08854v3",
            "date": "2019-07-20 18:49:36+00:00",
            "title": "Incremental Transformer with Deliberation Decoder for Document Grounded Conversations",
            "authors": [
                "Zekang Li",
                "Cheng Niu",
                "Fandong Meng",
                "Yang Feng",
                "Qian Li",
                "Jie Zhou"
            ],
            "abstract": "Document Grounded Conversations is a task to generate dialogue responses when\nchatting about the content of a given document. Obviously, document knowledge\nplays a critical role in Document Grounded Conversations, while existing\ndialogue models do not exploit this kind of knowledge effectively enough. In\nthis paper, we propose a novel Transformer-based architecture for multi-turn\ndocument grounded conversations. In particular, we devise an Incremental\nTransformer to encode multi-turn utterances along with knowledge in related\ndocuments. Motivated by the human cognitive process, we design a two-pass\ndecoder (Deliberation Decoder) to improve context coherence and knowledge\ncorrectness. Our empirical study on a real-world Document Grounded Dataset\nproves that responses generated by our model significantly outperform\ncompetitive baselines on both context coherence and knowledge relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08889v1",
            "date": "2019-07-21 01:20:07+00:00",
            "title": "The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correction",
            "authors": [
                "Phu Mon Htut",
                "Joel Tetreault"
            ],
            "abstract": "In recent years, sequence-to-sequence models have been very effective for\nend-to-end grammatical error correction (GEC). As creating human-annotated\nparallel corpus for GEC is expensive and time-consuming, there has been work on\nartificial corpus generation with the aim of creating sentences that contain\nrealistic grammatical errors from grammatically correct sentences. In this\npaper, we investigate the impact of using recent neural models for generating\nerrors to help neural models to correct errors. We conduct a battery of\nexperiments on the effect of data size, models, and comparison with a\nrule-based approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08948v1",
            "date": "2019-07-21 10:00:28+00:00",
            "title": "Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation",
            "authors": [
                "Shantipriya Parida",
                "Ond\u0159ej Bojar",
                "Satya Ranjan Dash"
            ],
            "abstract": "Visual Genome is a dataset connecting structured image information with\nEnglish language. We present ``Hindi Visual Genome'', a multimodal dataset\nconsisting of text and images suitable for English-Hindi multimodal machine\ntranslation task and multimodal research. We have selected short English\nsegments (captions) from Visual Genome along with associated images and\nautomatically translated them to Hindi with manual post-editing which took the\nassociated images into account. We prepared a set of 31525 segments,\naccompanied by a challenge test set of 1400 segments. This challenge test set\nwas created by searching for (particularly) ambiguous English words based on\nthe embedding similarity and manually selecting those where the image helps to\nresolve the ambiguity.\n  Our dataset is the first for multimodal English-Hindi machine translation,\nfreely available for non-commercial research purposes. Our Hindi version of\nVisual Genome also allows to create Hindi image labelers or other practical\ntools.\n  Hindi Visual Genome also serves in Workshop on Asian Translation (WAT) 2019\nMulti-Modal Translation Task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09190v1",
            "date": "2019-07-22 09:01:35+00:00",
            "title": "ELI5: Long Form Question Answering",
            "authors": [
                "Angela Fan",
                "Yacine Jernite",
                "Ethan Perez",
                "David Grangier",
                "Jason Weston",
                "Michael Auli"
            ],
            "abstract": "We introduce the first large-scale corpus for long-form question answering, a\ntask requiring elaborate and in-depth answers to open-ended questions. The\ndataset comprises 270K threads from the Reddit forum ``Explain Like I'm Five''\n(ELI5) where an online community provides answers to questions which are\ncomprehensible by five year olds. Compared to existing datasets, ELI5 comprises\ndiverse questions requiring multi-sentence answers. We provide a large set of\nweb documents to help answer the question. Automatic and human evaluations show\nthat an abstractive model trained with a multi-task objective outperforms\nconventional Seq2Seq, language modeling, as well as a strong extractive\nbaseline. However, our best model is still far from human performance since\nraters prefer gold responses in over 86% of cases, leaving ample opportunity\nfor future improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09312v1",
            "date": "2019-07-22 13:25:27+00:00",
            "title": "Syntax-aware Neural Semantic Role Labeling",
            "authors": [
                "Qingrong Xia",
                "Zhenghua Li",
                "Min Zhang",
                "Meishan Zhang",
                "Guohong Fu",
                "Rui Wang",
                "Luo Si"
            ],
            "abstract": "Semantic role labeling (SRL), also known as shallow semantic parsing, is an\nimportant yet challenging task in NLP. Motivated by the close correlation\nbetween syntactic and semantic structures, traditional discrete-feature-based\nSRL approaches make heavy use of syntactic features. In contrast,\ndeep-neural-network-based approaches usually encode the input sentence as a\nword sequence without considering the syntactic structures. In this work, we\ninvestigate several previous approaches for encoding syntactic trees, and make\na thorough study on whether extra syntax-aware representations are beneficial\nfor neural SRL models. Experiments on the benchmark CoNLL-2005 dataset show\nthat syntax-aware SRL approaches can effectively improve performance over a\nstrong baseline with external word representations from ELMo. With the extra\nsyntax-aware representations, our approaches achieve new state-of-the-art 85.6\nF1 (single model) and 86.6 F1 (ensemble) on the test data, outperforming the\ncorresponding strong baselines with ELMo by 0.8 and 1.0, respectively. Detailed\nerror analysis are conducted to gain more insights on the investigated\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09669v1",
            "date": "2019-07-23 03:05:39+00:00",
            "title": "EmotionX-HSU: Adopting Pre-trained BERT for Emotion Classification",
            "authors": [
                "Linkai Luo",
                "Yue Wang"
            ],
            "abstract": "This paper describes our approach to the EmotionX-2019, the shared task of\nSocialNLP 2019. To detect emotion for each utterance of two datasets from the\nTV show Friends and Facebook chat log EmotionPush, we propose two-step deep\nlearning based methodology: (i) encode each of the utterance into a sequence of\nvectors that represent its meaning; and (ii) use a simply softmax classifier to\npredict one of the emotions amongst four candidates that an utterance may\ncarry. Notice that the source of labeled utterances is not rich, we utilise a\nwell-trained model, known as BERT, to transfer part of the knowledge learned\nfrom a large amount of corpus to our model. We then focus on fine-tuning our\nmodel until it well fits to the in-domain data. The performance of the proposed\nmodel is evaluated by micro-F1 scores, i.e., 79.1% and 86.2% for the testsets\nof Friends and EmotionPush, respectively. Our model ranks 3rd among 11\nsubmissions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09699v1",
            "date": "2019-07-23 05:31:54+00:00",
            "title": "Learning to Select, Track, and Generate for Data-to-Text",
            "authors": [
                "Hayate Iso",
                "Yui Uehara",
                "Tatsuya Ishigaki",
                "Hiroshi Noji",
                "Eiji Aramaki",
                "Ichiro Kobayashi",
                "Yusuke Miyao",
                "Naoaki Okazaki",
                "Hiroya Takamura"
            ],
            "abstract": "We propose a data-to-text generation model with two modules, one for tracking\nand the other for text generation. Our tracking module selects and keeps track\nof salient information and memorizes which record has been mentioned. Our\ngeneration module generates a summary conditioned on the state of tracking\nmodule. Our model is considered to simulate the human-like writing process that\ngradually selects the information by determining the intermediate variables\nwhile writing the summary. In addition, we also explore the effectiveness of\nthe writer information for generation. Experimental results show that our model\noutperforms existing models in all evaluation metrics even without writer\ninformation. Incorporating writer information further improves the performance,\ncontributing to content planning and surface realization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09724v1",
            "date": "2019-07-23 07:15:23+00:00",
            "title": "Towards Unsupervised Grammatical Error Correction using Statistical Machine Translation with Synthetic Comparable Corpus",
            "authors": [
                "Satoru Katsumata",
                "Mamoru Komachi"
            ],
            "abstract": "We introduce unsupervised techniques based on phrase-based statistical\nmachine translation for grammatical error correction (GEC) trained on a pseudo\nlearner corpus created by Google Translation. We verified our GEC system\nthrough experiments on various GEC dataset, includi ng a low resource track of\nthe shared task at Building Educational Applications 2019 (BEA 2019). As a\nresult, we achieved an F_0.5 score of 28.31 points with the test data of the\nlow resource track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09899v4",
            "date": "2019-07-23 14:20:21+00:00",
            "title": "Modeling question asking using neural program generation",
            "authors": [
                "Ziyun Wang",
                "Brenden M. Lake"
            ],
            "abstract": "People ask questions that are far richer, more informative, and more creative\nthan current AI systems. We propose a neuro-symbolic framework for modeling\nhuman question asking, which represents questions as formal programs and\ngenerates programs with an encoder-decoder based deep neural network. From\nextensive experiments using an information-search game, we show that our method\ncan predict which questions humans are likely to ask in unconstrained settings.\nWe also propose a novel grammar-based question generation framework trained\nwith reinforcement learning, which is able to generate creative questions\nwithout supervised human data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10129v1",
            "date": "2019-07-23 21:05:37+00:00",
            "title": "CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology",
            "authors": [
                "Aditi Chaudhary",
                "Elizabeth Salesky",
                "Gayatri Bhat",
                "David R. Mortensen",
                "Jaime G. Carbonell",
                "Yulia Tsvetkov"
            ],
            "abstract": "This paper presents the submission by the CMU-01 team to the SIGMORPHON 2019\ntask 2 of Morphological Analysis and Lemmatization in Context. This task\nrequires us to produce the lemma and morpho-syntactic description of each token\nin a sequence, for 107 treebanks. We approach this task with a hierarchical\nneural conditional random field (CRF) model which predicts each coarse-grained\nfeature (eg. POS, Case, etc.) independently. However, most treebanks are\nunder-resourced, thus making it challenging to train deep neural models for\nthem. Hence, we propose a multi-lingual transfer training regime where we\ntransfer from multiple related languages that share similar typology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10136v1",
            "date": "2019-07-23 21:18:41+00:00",
            "title": "Dr.Quad at MEDIQA 2019: Towards Textual Inference and Question Entailment using contextualized representations",
            "authors": [
                "Vinayshekhar Bannihatti Kumar",
                "Ashwin Srinivasan",
                "Aditi Chaudhary",
                "James Route",
                "Teruko Mitamura",
                "Eric Nyberg"
            ],
            "abstract": "This paper presents the submissions by Team Dr.Quad to the ACL-BioNLP 2019\nshared task on Textual Inference and Question Entailment in the Medical Domain.\nOur system is based on the prior work Liu et al. (2019) which uses a multi-task\nobjective function for textual entailment. In this work, we explore different\nstrategies for generalizing state-of-the-art language understanding models to\nthe specialized medical domain. Our results on the shared task demonstrate that\nincorporating domain knowledge through data augmentation is a powerful strategy\nfor addressing challenges posed by specialized domains such as medicine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10302v3",
            "date": "2019-07-24 08:49:01+00:00",
            "title": "Fine-Grained Sentence Functions for Short-Text Conversation",
            "authors": [
                "Wei Bi",
                "Jun Gao",
                "Xiaojiang Liu",
                "Shuming Shi"
            ],
            "abstract": "Sentence function is an important linguistic feature referring to a user's\npurpose in uttering a specific sentence. The use of sentence function has shown\npromising results to improve the performance of conversation models. However,\nthere is no large conversation dataset annotated with sentence functions. In\nthis work, we collect a new Short-Text Conversation dataset with manually\nannotated SEntence FUNctions (STC-Sefun). Classification models are trained on\nthis dataset to (i) recognize the sentence function of new data in a large\ncorpus of short-text conversations; (ii) estimate a proper sentence function of\nthe response given a test query. We later train conversation models conditioned\non the sentence functions, including information retrieval-based and neural\ngenerative models. Experimental results demonstrate that the use of sentence\nfunctions can help improve the quality of the returned responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10352v2",
            "date": "2019-07-24 10:30:39+00:00",
            "title": "Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task",
            "authors": [
                "Fabio Kepler",
                "Jonay Tr\u00e9nous",
                "Marcos Treviso",
                "Miguel Vera",
                "Ant\u00f3nio G\u00f3is",
                "M. Amin Farajian",
                "Ant\u00f3nio V. Lopes",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "We present the contribution of the Unbabel team to the WMT 2019 Shared Task\non Quality Estimation. We participated on the word, sentence, and\ndocument-level tracks, encompassing 3 language pairs: English-German,\nEnglish-Russian, and English-French. Our submissions build upon the recent\nOpenKiwi framework: we combine linear, neural, and predictor-estimator systems\nwith new transfer learning approaches using BERT and XLM pre-trained models. We\ncompare systems individually and propose new ensemble techniques for word and\nsentence-level predictions. We also propose a simple technique for converting\nword labels into document-level predictions. Overall, our submitted systems\nachieve the best results on all tracks and language pairs by a considerable\nmargin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10362v1",
            "date": "2019-07-24 11:01:24+00:00",
            "title": "Translator2Vec: Understanding and Representing Human Post-Editors",
            "authors": [
                "Ant\u00f3nio G\u00f3is",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "The combination of machines and humans for translation is effective, with\nmany studies showing productivity gains when humans post-edit\nmachine-translated output instead of translating from scratch. To take full\nadvantage of this combination, we need a fine-grained understanding of how\nhuman translators work, and which post-editing styles are more effective than\nothers. In this paper, we release and analyze a new dataset with document-level\npost-editing action sequences, including edit operations from keystrokes, mouse\nactions, and waiting times. Our dataset comprises 66,268 full document sessions\npost-edited by 332 humans, the largest of the kind released to date. We show\nthat action sequences are informative enough to identify post-editors\naccurately, compared to baselines that only look at the initial and final text.\nWe build on this to learn and visualize continuous representations of\npost-editors, and we show that these representations improve the downstream\ntask of predicting post-editing time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10371v1",
            "date": "2019-07-24 11:37:08+00:00",
            "title": "Automatic Generation of Personalized Comment Based on User Profile",
            "authors": [
                "Wenhuan Zeng",
                "Abulikemu Abuduweili",
                "Lei Li",
                "Pengcheng Yang"
            ],
            "abstract": "Comments on social media are very diverse, in terms of content, style and\nvocabulary, which make generating comments much more challenging than other\nexisting natural language generation~(NLG) tasks. Besides, since different user\nhas different expression habits, it is necessary to take the user's profile\ninto consideration when generating comments. In this paper, we introduce the\ntask of automatic generation of personalized comment~(AGPC) for social media.\nBased on tens of thousands of users' real comments and corresponding user\nprofiles on weibo, we propose Personalized Comment Generation Network~(PCGN)\nfor AGPC. The model utilizes user feature embedding with a gated memory and\nattends to user description to model personality of users. In addition,\nexternal user representation is taken into consideration during the decoding to\nenhance the comments generation. Experimental results show that our model can\ngenerate natural, human-like and personalized comments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10449v2",
            "date": "2019-07-24 13:48:16+00:00",
            "title": "Distributional Analysis of Polysemous Function Words",
            "authors": [
                "Sebastian Pado",
                "Daniel Hole"
            ],
            "abstract": "In this paper, we are concerned with the phenomenon of function word\npolysemy. We adopt the framework of distributional semantics, which\ncharacterizes word meaning by observing occurrence contexts in large corpora\nand which is in principle well situated to model polysemy. Nevertheless,\nfunction words were traditionally considered as impossible to analyze\ndistributionally due to their highly flexible usage patterns. We establish that\ncontextualized word embeddings, the most recent generation of distributional\nmethods, offer hope in this regard. Using the German reflexive pronoun 'sich'\nas an example, we find that contextualized word embeddings capture\ntheoretically motivated word senses for 'sich' to the extent to which these\nsenses are mirrored systematically in linguistic usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10568v2",
            "date": "2019-07-24 17:18:48+00:00",
            "title": "Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References",
            "authors": [
                "Prakhar Gupta",
                "Shikib Mehri",
                "Tiancheng Zhao",
                "Amy Pavel",
                "Maxine Eskenazi",
                "Jeffrey P. Bigham"
            ],
            "abstract": "The aim of this paper is to mitigate the shortcomings of automatic evaluation\nof open-domain dialog systems through multi-reference evaluation. Existing\nmetrics have been shown to correlate poorly with human judgement, particularly\nin open-domain dialog. One alternative is to collect human annotations for\nevaluation, which can be expensive and time consuming. To demonstrate the\neffectiveness of multi-reference evaluation, we augment the test set of\nDailyDialog with multiple references. A series of experiments show that the use\nof multiple references results in improved correlation between several\nautomatic metrics and human judgement for both the quality and the diversity of\nsystem output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10641v2",
            "date": "2019-07-24 18:11:59+00:00",
            "title": "WinoGrande: An Adversarial Winograd Schema Challenge at Scale",
            "authors": [
                "Keisuke Sakaguchi",
                "Ronan Le Bras",
                "Chandra Bhagavatula",
                "Yejin Choi"
            ],
            "abstract": "The Winograd Schema Challenge (WSC) (Levesque, Davis, and Morgenstern 2011),\na benchmark for commonsense reasoning, is a set of 273 expert-crafted pronoun\nresolution problems originally designed to be unsolvable for statistical models\nthat rely on selectional preferences or word associations. However, recent\nadvances in neural language models have already reached around 90% accuracy on\nvariants of WSC. This raises an important question whether these models have\ntruly acquired robust commonsense capabilities or whether they rely on spurious\nbiases in the datasets that lead to an overestimation of the true capabilities\nof machine commonsense. To investigate this question, we introduce WinoGrande,\na large-scale dataset of 44k problems, inspired by the original WSC design, but\nadjusted to improve both the scale and the hardness of the dataset. The key\nsteps of the dataset construction consist of (1) a carefully designed\ncrowdsourcing procedure, followed by (2) systematic bias reduction using a\nnovel AfLite algorithm that generalizes human-detectable word associations to\nmachine-detectable embedding associations. The best state-of-the-art methods on\nWinoGrande achieve 59.4-79.1%, which are 15-35% below human performance of\n94.0%, depending on the amount of the training data allowed. Furthermore, we\nestablish new state-of-the-art results on five related benchmarks - WSC\n(90.1%), DPR (93.1%), COPA (90.6%), KnowRef (85.6%), and Winogender (97.1%).\nThese results have dual implications: on one hand, they demonstrate the\neffectiveness of WinoGrande when used as a resource for transfer learning. On\nthe other hand, they raise a concern that we are likely to be overestimating\nthe true capabilities of machine commonsense across all these benchmarks. We\nemphasize the importance of algorithmic bias reduction in existing and future\nbenchmarks to mitigate such overestimation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10676v1",
            "date": "2019-07-23 15:49:20+00:00",
            "title": "Semantic Web for Machine Translation: Challenges and Directions",
            "authors": [
                "Diego Moussallem",
                "Matthias Wauer",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "A large number of machine translation approaches have recently been developed\nto facilitate the fluid migration of content across languages. However, the\nliterature suggests that many obstacles must still be dealt with to achieve\nbetter automatic translations. One of these obstacles is lexical and syntactic\nambiguity. A promising way of overcoming this problem is using Semantic Web\ntechnologies. This article is an extended abstract of our systematic review on\nmachine translation approaches that rely on Semantic Web technologies for\nimproving the translation of texts. Overall, we present the challenges and\nopportunities in the use of Semantic Web technologies in Machine Translation.\nMoreover, our research suggests that while Semantic Web technologies can\nenhance the quality of machine translation outputs for various problems, the\ncombination of both is still in its infancy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10873v1",
            "date": "2019-07-25 07:37:41+00:00",
            "title": "Summary Refinement through Denoising",
            "authors": [
                "Nikola I. Nikolov",
                "Alessandro Calmanovici",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "We propose a simple method for post-processing the outputs of a text\nsummarization system in order to refine its overall quality. Our approach is to\ntrain text-to-text rewriting models to correct information redundancy errors\nthat may arise during summarization. We train on synthetically generated noisy\nsummaries, testing three different types of noise that introduce out-of-context\ninformation within each summary. When applied on top of extractive and\nabstractive summarization baselines, our summary denoising models yield metric\nimprovements while reducing redundancy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11049v1",
            "date": "2019-07-25 13:45:48+00:00",
            "title": "Grammatical Sequence Prediction for Real-Time Neural Semantic Parsing",
            "authors": [
                "Chunyang Xiao",
                "Christoph Teichmann",
                "Konstantine Arkoudas"
            ],
            "abstract": "While sequence-to-sequence (seq2seq) models achieve state-of-the-art\nperformance in many natural language processing tasks, they can be too slow for\nreal-time applications. One performance bottleneck is predicting the most\nlikely next token over a large vocabulary; methods to circumvent this\nbottleneck are a current research topic. We focus specifically on using seq2seq\nmodels for semantic parsing, where we observe that grammars often exist which\nspecify valid formal representations of utterance semantics. By developing a\ngeneric approach for restricting the predictions of a seq2seq model to\ngrammatically permissible continuations, we arrive at a widely applicable\ntechnique for speeding up semantic parsing. The technique leads to a 74%\nspeed-up on an in-house dataset with a large vocabulary, compared to the same\nneural model without grammatical restrictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11062v1",
            "date": "2019-07-25 14:00:18+00:00",
            "title": "HireNet: a Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews",
            "authors": [
                "L\u00e9o Hemamou",
                "Ghazi Felhi",
                "Vincent Vandenbussche",
                "Jean-Claude Martin",
                "Chlo\u00e9 Clavel"
            ],
            "abstract": "New technologies drastically change recruitment techniques. Some research\nprojects aim at designing interactive systems that help candidates practice job\ninterviews. Other studies aim at the automatic detection of social signals\n(e.g. smile, turn of speech, etc...) in videos of job interviews. These studies\nare limited with respect to the number of interviews they process, but also by\nthe fact that they only analyze simulated job interviews (e.g. students\npretending to apply for a fake position). Asynchronous video interviewing tools\nhave become mature products on the human resources market, and thus, a popular\nstep in the recruitment process. As part of a project to help recruiters, we\ncollected a corpus of more than 7000 candidates having asynchronous video job\ninterviews for real positions and recording videos of themselves answering a\nset of questions. We propose a new hierarchical attention model called HireNet\nthat aims at predicting the hirability of the candidates as evaluated by\nrecruiters. In HireNet, an interview is considered as a sequence of questions\nand answers containing salient socials signals. Two contextual sources of\ninformation are modeled in HireNet: the words contained in the question and in\nthe job position. Our model achieves better F1-scores than previous approaches\nfor each modality (verbal content, audio and video). Results from early and\nlate multimodal fusion suggest that more sophisticated fusion schemes are\nneeded to improve on the monomodal results. Finally, some examples of moments\ncaptured by the attention mechanisms suggest our model could potentially be\nused to help finding key moments in an asynchronous job interview.",
            "journal ref": "Vol 33 (2019): Proceedings of the Thirty-Third AAAI Conference on\n  Artificial Intelligence",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11065v2",
            "date": "2019-07-25 14:03:06+00:00",
            "title": "DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks",
            "authors": [
                "Lin Zehui",
                "Pengfei Liu",
                "Luyao Huang",
                "Junkun Chen",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Variants dropout methods have been designed for the fully-connected layer,\nconvolutional layer and recurrent layer in neural networks, and shown to be\neffective to avoid overfitting. As an appealing alternative to recurrent and\nconvolutional layers, the fully-connected self-attention layer surprisingly\nlacks a specific dropout method. This paper explores the possibility of\nregularizing the attention weights in Transformers to prevent different\ncontextualized feature vectors from co-adaption. Experiments on a wide range of\ntasks show that DropAttention can improve performance and reduce overfitting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11158v1",
            "date": "2019-07-25 16:04:09+00:00",
            "title": "Cross-Lingual Transfer for Distantly Supervised and Low-resources Indonesian NER",
            "authors": [
                "Fariz Ikhwantri"
            ],
            "abstract": "Manually annotated corpora for low-resource languages are usually small in\nquantity (gold), or large but distantly supervised (silver). Inspired by recent\nprogress of injecting pre-trained language model (LM) on many Natural Language\nProcessing (NLP) task, we proposed to fine-tune pre-trained language model from\nhigh-resources languages to low-resources languages to improve the performance\nof both scenarios. Our empirical experiment demonstrates significant\nimprovement when fine-tuning pre-trained language model in cross-lingual\ntransfer scenarios for small gold corpus and competitive results in large\nsilver compare to supervised cross-lingual transfer, which will be useful when\nthere is no parallel annotation in the same task to begin. We compare our\nproposed method of cross-lingual transfer using pre-trained LM to different\nsources of transfer such as mono-lingual LM and Part-of-Speech tagging (POS) in\nthe downstream task of both large silver and small gold NER dataset by\nexploiting character-level input of bi-directional language model task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11315v1",
            "date": "2019-07-25 21:33:29+00:00",
            "title": "Time Masking: Leveraging Temporal Information in Spoken Dialogue Systems",
            "authors": [
                "Rylan Conway",
                "Lambert Mathias"
            ],
            "abstract": "In a spoken dialogue system, dialogue state tracker (DST) components track\nthe state of the conversation by updating a distribution of values associated\nwith each of the slots being tracked for the current user turn, using the\ninteractions until then. Much of the previous work has relied on modeling the\nnatural order of the conversation, using distance based offsets as an\napproximation of time. In this work, we hypothesize that leveraging the\nwall-clock temporal difference between turns is crucial for finer-grained\ncontrol of dialogue scenarios. We develop a novel approach that applies a {\\it\ntime mask}, based on the wall-clock time difference, to the associated slot\nembeddings and empirically demonstrate that our proposed approach outperforms\nexisting approaches that leverage distance offsets, on both an internal\nbenchmark dataset as well as DSTC2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11438v3",
            "date": "2019-07-26 08:54:36+00:00",
            "title": "LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations",
            "authors": [
                "Max Eichler",
                "G\u00f6zde G\u00fcl \u015eahin",
                "Iryna Gurevych"
            ],
            "abstract": "We present LINSPECTOR WEB, an open source multilingual inspector to analyze\nword representations. Our system provides researchers working in low-resource\nsettings with an easily accessible web based probing tool to gain quick\ninsights into their word embeddings especially outside of the English language.\nTo do this we employ 16 simple linguistic probing tasks such as gender, case\nmarking, and tense for a diverse set of 28 languages. We support probing of\nstatic word embeddings along with pretrained AllenNLP models that are commonly\nused for NLP downstream tasks such as named entity recognition, natural\nlanguage inference and dependency parsing. The results are visualized in a\npolar chart and also provided as a table. LINSPECTOR WEB is available as an\noffline tool or at https://linspector.ukp.informatik.tu-darmstadt.de.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11512v1",
            "date": "2019-07-26 12:29:37+00:00",
            "title": "Investigating Self-Attention Network for Chinese Word Segmentation",
            "authors": [
                "Leilei Gan",
                "Yue Zhang"
            ],
            "abstract": "Neural network has become the dominant method for Chinese word segmentation.\nMost existing models cast the task as sequence labeling, using BiLSTM-CRF for\nrepresenting the input and making output predictions. Recently, attention-based\nsequence models have emerged as a highly competitive alternative to LSTMs,\nwhich allow better running speed by parallelization of computation. We\ninvestigate self attention network for Chinese word segmentation, making\ncomparisons between BiLSTM-CRF models. In addition, the influence of\ncontextualized character embeddings is investigated using BERT, and a method is\nproposed for integrating word information into SAN segmentation. Results show\nthat SAN gives highly competitive results compared with BiLSTMs, with BERT and\nword information further improving segmentation for in-domain and cross-domain\nsegmentation. Our final models give the best results for 6 heterogenous domain\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11521v1",
            "date": "2019-07-25 07:41:45+00:00",
            "title": "Deep Ranking Based Cost-sensitive Multi-label Learning for Distant Supervision Relation Extraction",
            "authors": [
                "Hai Ye",
                "Zhunchen Luo"
            ],
            "abstract": "Knowledge base provides a potential way to improve the intelligence of\ninformation retrieval (IR) systems, for that knowledge base has numerous\nrelations between entities which can help the IR systems to conduct inference\nfrom one entity to another entity. Relation extraction is one of the\nfundamental techniques to construct a knowledge base. Distant supervision is a\nsemi-supervised learning method for relation extraction which learns with\nlabeled and unlabeled data. However, this approach suffers the problem of\nrelation overlapping in which one entity tuple may have multiple relation\nfacts. We believe that relation types can have latent connections, which we\ncall class ties, and can be exploited to enhance relation extraction. However,\nthis property between relation classes has not been fully explored before. In\nthis paper, to exploit class ties between relations to improve relation\nextraction, we propose a general ranking based multi-label learning framework\ncombined with convolutional neural networks, in which ranking based loss\nfunctions with regularization technique are introduced to learn the latent\nconnections between relations. Furthermore, to deal with the problem of class\nimbalance in distant supervision relation extraction, we further adopt\ncost-sensitive learning to rescale the costs from the positive and negative\nlabels. Extensive experiments on a widely used dataset show the effectiveness\nof our model to exploit class ties and to relieve class imbalance problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11692v1",
            "date": "2019-07-26 17:48:29+00:00",
            "title": "RoBERTa: A Robustly Optimized BERT Pretraining Approach",
            "authors": [
                "Yinhan Liu",
                "Myle Ott",
                "Naman Goyal",
                "Jingfei Du",
                "Mandar Joshi",
                "Danqi Chen",
                "Omer Levy",
                "Mike Lewis",
                "Luke Zettlemoyer",
                "Veselin Stoyanov"
            ],
            "abstract": "Language model pretraining has led to significant performance gains but\ncareful comparison between different approaches is challenging. Training is\ncomputationally expensive, often done on private datasets of different sizes,\nand, as we will show, hyperparameter choices have significant impact on the\nfinal results. We present a replication study of BERT pretraining (Devlin et\nal., 2019) that carefully measures the impact of many key hyperparameters and\ntraining data size. We find that BERT was significantly undertrained, and can\nmatch or exceed the performance of every model published after it. Our best\nmodel achieves state-of-the-art results on GLUE, RACE and SQuAD. These results\nhighlight the importance of previously overlooked design choices, and raise\nquestions about the source of recently reported improvements. We release our\nmodels and code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11769v3",
            "date": "2019-07-26 19:43:07+00:00",
            "title": "Automatically Learning Construction Injury Precursors from Text",
            "authors": [
                "Henrietta Baker",
                "Matthew R. Hallowell",
                "Antoine J. -P. Tixier"
            ],
            "abstract": "In light of the increasing availability of digitally recorded safety reports\nin the construction industry, it is important to develop methods to exploit\nthese data to improve our understanding of safety incidents and ability to\nlearn from them. In this study, we compare several approaches to automatically\nlearn injury precursors from raw construction accident reports. More precisely,\nwe experiment with two state-of-the-art deep learning architectures for Natural\nLanguage Processing (NLP), Convolutional Neural Networks (CNN) and Hierarchical\nAttention Networks (HAN), and with the established Term Frequency - Inverse\nDocument Frequency representation (TF-IDF) + Support Vector Machine (SVM)\napproach. For each model, we provide a method to identify (after training) the\ntextual patterns that are, on average, the most predictive of each safety\noutcome. We show that among those pieces of text, valid injury precursors can\nbe found. The proposed methods can also be used by the user to visualize and\nunderstand the models' predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11779v3",
            "date": "2019-07-26 20:04:57+00:00",
            "title": "Supervised and Unsupervised Neural Approaches to Text Readability",
            "authors": [
                "Matej Martinc",
                "Senja Pollak",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "We present a set of novel neural supervised and unsupervised approaches for\ndetermining the readability of documents. In the unsupervised setting, we\nleverage neural language models, whereas in the supervised setting, three\ndifferent neural classification architectures are tested. We show that the\nproposed neural unsupervised approach is robust, transferable across languages\nand allows adaptation to a specific readability task and data set. By\nsystematic comparison of several neural architectures on a number of benchmark\nand new labelled readability datasets in two languages, this study also offers\na comprehensive analysis of different neural approaches to readability\nclassification. We expose their strengths and weaknesses, compare their\nperformance to current state-of-the-art classification approaches to\nreadability, which in most cases still rely on extensive feature engineering,\nand propose possibilities for improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11907v1",
            "date": "2019-07-27 13:30:56+00:00",
            "title": "Nefnir: A high accuracy lemmatizer for Icelandic",
            "authors": [
                "Svanhv\u00edt Lilja Ing\u00f3lfsd\u00f3ttir",
                "Hrafn Loftsson",
                "J\u00f3n Fri\u00f0rik Da\u00f0ason",
                "Krist\u00edn Bjarnad\u00f3ttir"
            ],
            "abstract": "Lemmatization, finding the basic morphological form of a word in a corpus, is\nan important step in many natural language processing tasks when working with\nmorphologically rich languages. We describe and evaluate Nefnir, a new open\nsource lemmatizer for Icelandic. Nefnir uses suffix substitution rules, derived\nfrom a large morphological database, to lemmatize tagged text. Evaluation shows\nthat for correctly tagged text, Nefnir obtains an accuracy of 99.55%, and for\ntext tagged with a PoS tagger, the accuracy obtained is 96.88%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11983v1",
            "date": "2019-07-27 21:51:52+00:00",
            "title": "A Hybrid Neural Network Model for Commonsense Reasoning",
            "authors": [
                "Pengcheng He",
                "Xiaodong Liu",
                "Weizhu Chen",
                "Jianfeng Gao"
            ],
            "abstract": "This paper proposes a hybrid neural network (HNN) model for commonsense\nreasoning. An HNN consists of two component models, a masked language model and\na semantic similarity model, which share a BERT-based contextual encoder but\nuse different model-specific input and output layers. HNN obtains new\nstate-of-the-art results on three classic commonsense reasoning tasks, pushing\nthe WNLI benchmark to 89%, the Winograd Schema Challenge (WSC) benchmark to\n75.1%, and the PDP60 benchmark to 90.0%. An ablation study shows that language\nmodels and semantic similarity models are complementary approaches to\ncommonsense reasoning, and HNN effectively combines the strengths of both. The\ncode and pre-trained models will be publicly available at\nhttps://github.com/namisan/mt-dnn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12009v1",
            "date": "2019-07-28 03:57:41+00:00",
            "title": "Representation Degeneration Problem in Training Natural Language Generation Models",
            "authors": [
                "Jun Gao",
                "Di He",
                "Xu Tan",
                "Tao Qin",
                "Liwei Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "We study an interesting problem in training neural network-based models for\nnatural language generation tasks, which we call the \\emph{representation\ndegeneration problem}. We observe that when training a model for natural\nlanguage generation tasks through likelihood maximization with the weight tying\ntrick, especially with big training datasets, most of the learnt word\nembeddings tend to degenerate and be distributed into a narrow cone, which\nlargely limits the representation power of word embeddings. We analyze the\nconditions and causes of this problem and propose a novel regularization method\nto address it. Experiments on language modeling and machine translation show\nthat our method can largely mitigate the representation degeneration problem\nand achieve better performance than baseline algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12108v4",
            "date": "2019-07-28 16:52:09+00:00",
            "title": "CAiRE: An Empathetic Neural Chatbot",
            "authors": [
                "Zhaojiang Lin",
                "Peng Xu",
                "Genta Indra Winata",
                "Farhad Bin Siddique",
                "Zihan Liu",
                "Jamin Shin",
                "Pascale Fung"
            ],
            "abstract": "In this paper, we present an end-to-end empathetic conversation agent CAiRE.\nOur system adapts TransferTransfo (Wolf et al., 2019) learning approach that\nfine-tunes a large-scale pre-trained language model with multi-task objectives:\nresponse language modeling, response prediction and dialogue emotion detection.\nWe evaluate our model on the recently proposed empathetic-dialogues dataset\n(Rashkin et al., 2019), the experiment results show that CAiRE achieves\nstate-of-the-art performance on dialogue emotion detection and empathetic\nresponse generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12162v1",
            "date": "2019-07-28 23:41:53+00:00",
            "title": "Hybrid Code Networks using a convolutional neural network as an input layer achieves higher turn accuracy",
            "authors": [
                "Petr Marek"
            ],
            "abstract": "The dialogue management is a task of conversational artificial intelligence.\nThe goal of the dialogue manager is to select the appropriate response to the\nconversational partner conditioned by the input message and recent dialogue\nstate. Hybrid Code Networks is one of the models of dialogue managers, which\nuses an average of word embeddings and bag-of-words as input features. We\nperform experiments on Dialogue bAbI Task 6 and Alquist Conversational Dataset.\nThe experiments show that the convolutional neural network used as an input\nlayer of the Hybrid Code Network improves the model's turn accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12280v1",
            "date": "2019-07-29 08:54:02+00:00",
            "title": "Legal entity recognition in an agglutinating language and document connection network for EU Legislation and EU/Hungarian Case Law",
            "authors": [
                "Gy\u00f6rgy G\u00f6r\u00f6g",
                "P\u00e9ter Weisz"
            ],
            "abstract": "We have developed an application aiming at federated search for EU and\nHungarian legislation and jurisdiction. It now contains above 1 million\ndocuments, with daily updates. The database holds documents downloaded from the\nEU sources EUR-Lex and Curia Online as well as public jurisdiction documents\nfrom the Constitutional Court of Hungary and The National Office for The\nJudiciary. The application is termed Justeus. Justeus provides comprehensible\nsearch possibilities. Besides free text and metadata (dropdown list) searches,\nit features hierarchical data structures (concept hierarchy trees) of directory\ncodes and classification as well as subject terms. Justeus collects all links\nof a particular document to other documents (court judgements citing other case\nlaw documents as well as legislation, national court decisions referring to EU\nregulation etc.) as tables and directed graph networks. Choosing a document,\nits relations to other documents are visualized in real time as a network.\nNetwork graphs help in identifying key documents influencing or referred by\nmany other documents (legislative and/or jurisdictive) and sets of documents\npredominantly referring to each other (citation networks).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12316v1",
            "date": "2019-07-29 10:12:18+00:00",
            "title": "Hierarchical Multi-Label Dialog Act Recognition on Spanish Data",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "Dialog acts reveal the intention behind the uttered words. Thus, their\nautomatic recognition is important for a dialog system trying to understand its\nconversational partner. The study presented in this article approaches that\ntask on the DIHANA corpus, whose three-level dialog act annotation scheme poses\nproblems which have not been explored in recent studies. In addition to the\nhierarchical problem, the two lower levels pose multi-label classification\nproblems. Furthermore, each level in the hierarchy refers to a different aspect\nconcerning the intention of the speaker both in terms of the structure of the\ndialog and the task. Also, since its dialogs are in Spanish, it allows us to\nassess whether the state-of-the-art approaches on English data generalize to a\ndifferent language. More specifically, we compare the performance of different\nsegment representation approaches focusing on both sequences and patterns of\nwords and assess the importance of the dialog history and the relations between\nthe multiple levels of the hierarchy. Concerning the single-label\nclassification problem posed by the top level, we show that the conclusions\ndrawn on English data also hold on Spanish data. Furthermore, we show that the\napproaches can be adapted to multi-label scenarios. Finally, by hierarchically\ncombining the best classifiers for each level, we achieve the best results\nreported for this corpus.",
            "journal ref": "Linguam\\'atica 11(1) (2019) 17-40",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12412v2",
            "date": "2019-07-29 13:25:37+00:00",
            "title": "ERNIE 2.0: A Continual Pre-training Framework for Language Understanding",
            "authors": [
                "Yu Sun",
                "Shuohuan Wang",
                "Yukun Li",
                "Shikun Feng",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Recently, pre-trained models have achieved state-of-the-art results in\nvarious language understanding tasks, which indicates that pre-training on\nlarge-scale corpora may play a crucial role in natural language processing.\nCurrent pre-training procedures usually focus on training the model with\nseveral simple tasks to grasp the co-occurrence of words or sentences. However,\nbesides co-occurring, there exists other valuable lexical, syntactic and\nsemantic information in training corpora, such as named entity, semantic\ncloseness and discourse relations. In order to extract to the fullest extent,\nthe lexical, syntactic and semantic information from training corpora, we\npropose a continual pre-training framework named ERNIE 2.0 which builds and\nlearns incrementally pre-training tasks through constant multi-task learning.\nExperimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on\n16 tasks including English tasks on GLUE benchmarks and several common tasks in\nChinese. The source codes and pre-trained models have been released at\nhttps://github.com/PaddlePaddle/ERNIE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12437v1",
            "date": "2019-07-29 13:59:02+00:00",
            "title": "A Baseline Neural Machine Translation System for Indian Languages",
            "authors": [
                "Jerin Philip",
                "Vinay P. Namboodiri",
                "C. V. Jawahar"
            ],
            "abstract": "We present a simple, yet effective, Neural Machine Translation system for\nIndian languages. We demonstrate the feasibility for multiple language pairs,\nand establish a strong baseline for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12461v2",
            "date": "2019-07-29 14:42:30+00:00",
            "title": "Leveraging Pre-trained Checkpoints for Sequence Generation Tasks",
            "authors": [
                "Sascha Rothe",
                "Shashi Narayan",
                "Aliaksei Severyn"
            ],
            "abstract": "Unsupervised pre-training of large neural models has recently revolutionized\nNatural Language Processing. By warm-starting from the publicly released\ncheckpoints, NLP practitioners have pushed the state-of-the-art on multiple\nbenchmarks while saving significant amounts of compute time. So far the focus\nhas been mainly on the Natural Language Understanding tasks. In this paper, we\ndemonstrate the efficacy of pre-trained checkpoints for Sequence Generation. We\ndeveloped a Transformer-based sequence-to-sequence model that is compatible\nwith publicly available pre-trained BERT, GPT-2 and RoBERTa checkpoints and\nconducted an extensive empirical study on the utility of initializing our\nmodel, both encoder and decoder, with these checkpoints. Our models result in\nnew state-of-the-art results on Machine Translation, Text Summarization,\nSentence Splitting, and Sentence Fusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12524v2",
            "date": "2019-07-29 16:59:53+00:00",
            "title": "Neural Mention Detection",
            "authors": [
                "Juntao Yu",
                "Bernd Bohnet",
                "Massimo Poesio"
            ],
            "abstract": "Mention detection is an important preprocessing step for annotation and\ninterpretation in applications such as NER and coreference resolution, but few\nstand-alone neural models have been proposed able to handle the full range of\nmentions. In this work, we propose and compare three neural network-based\napproaches to mention detection. The first approach is based on the mention\ndetection part of a state of the art coreference resolution system; the second\nuses ELMO embeddings together with a bidirectional LSTM and a biaffine\nclassifier; the third approach uses the recently introduced BERT model. Our\nbest model (using a biaffine classifier) achieves gains of up to 1.8 percentage\npoints on mention recall when compared with a strong baseline in a HIGH RECALL\ncoreference annotation setting. The same model achieves improvements of up to\n5.3 and 6.2 p.p. when compared with the best-reported mention detection F1 on\nthe CONLL and CRAC coreference data sets respectively in a HIGH F1 annotation\nsetting. We then evaluate our models for coreference resolution by using\nmentions predicted by our best model in start-of-the-art coreference systems.\nThe enhanced model achieved absolute improvements of up to 1.7 and 0.7 p.p.\nwhen compared with our strong baseline systems (pipeline system and end-to-end\nsystem) respectively. For nested NER, the evaluation of our model on the GENIA\ncorpora shows that our model matches or outperforms state-of-the-art models\ndespite not being specifically designed for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12664v1",
            "date": "2019-07-29 21:44:50+00:00",
            "title": "CUNI Systems for the Unsupervised News Translation Task in WMT 2019",
            "authors": [
                "Ivana Kvapil\u00edkov\u00e1",
                "Dominik Mach\u00e1\u010dek",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "In this paper we describe the CUNI translation system used for the\nunsupervised news shared task of the ACL 2019 Fourth Conference on Machine\nTranslation (WMT19). We follow the strategy of Artexte et al. (2018b), creating\na seed phrase-based system where the phrase table is initialized from\ncross-lingual embedding mappings trained on monolingual data, followed by a\nneural machine translation system trained on synthetic parallel data. The\nsynthetic corpus was produced from a monolingual corpus by a tuned PBMT model\nrefined through iterative back-translation. We further focus on the handling of\nnamed entities, i.e. the part of vocabulary where the cross-lingual embedding\nmapping suffers most. Our system reaches a BLEU score of 15.3 on the\nGerman-Czech WMT19 shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12674v1",
            "date": "2019-07-29 22:26:51+00:00",
            "title": "One-to-X analogical reasoning on word embeddings: a case for diachronic armed conflict prediction from news texts",
            "authors": [
                "Andrey Kutuzov",
                "Erik Velldal",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "We extend the well-known word analogy task to a one-to-X formulation,\nincluding one-to-none cases, when no correct answer exists. The task is cast as\na relation discovery problem and applied to historical armed conflicts\ndatasets, attempting to predict new relations of type `location:armed-group'\nbased on data about past events. As the source of semantic information, we use\ndiachronic word embedding models trained on English news texts. A simple\ntechnique to improve diachronic performance in such task is demonstrated, using\na threshold based on a function of cosine distance to decrease the number of\nfalse positives; this approach is shown to be beneficial on two different\ncorpora. Finally, we publish a ready-to-use test set for one-to-X analogy\nevaluation on historical armed conflicts data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12679v1",
            "date": "2019-07-29 22:53:59+00:00",
            "title": "Machine Translation Evaluation with BERT Regressor",
            "authors": [
                "Hiroki Shimanaka",
                "Tomoyuki Kajiwara",
                "Mamoru Komachi"
            ],
            "abstract": "We introduce the metric using BERT (Bidirectional Encoder Representations\nfrom Transformers) (Devlin et al., 2019) for automatic machine translation\nevaluation. The experimental results of the WMT-2017 Metrics Shared Task\ndataset show that our metric achieves state-of-the-art performance in\nsegment-level metrics task for all to-English language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12750v1",
            "date": "2019-07-30 06:17:19+00:00",
            "title": "English-Czech Systems in WMT19: Document-Level Transformer",
            "authors": [
                "Martin Popel",
                "Dominik Mach\u00e1\u010dek",
                "Michal Auersperger",
                "Ond\u0159ej Bojar",
                "Pavel Pecina"
            ],
            "abstract": "We describe our NMT systems submitted to the WMT19 shared task in\nEnglish-Czech news translation. Our systems are based on the Transformer model\nimplemented in either Tensor2Tensor (T2T) or Marian framework.\n  We aimed at improving the adequacy and coherence of translated documents by\nenlarging the context of the source and target. Instead of translating each\nsentence independently, we split the document into possibly overlapping\nmulti-sentence segments. In case of the T2T implementation, this\n\"document-level\"-trained system achieves a $+0.6$ BLEU improvement ($p<0.05$)\nrelative to the same system applied on isolated sentences. To assess the\npotential effect document-level models might have on lexical coherence, we\nperformed a semi-automatic analysis, which revealed only a few sentences\nimproved in this aspect. Thus, we cannot draw any conclusions from this weak\nevidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12801v2",
            "date": "2019-07-30 09:33:13+00:00",
            "title": "IPRE: a Dataset for Inter-Personal Relationship Extraction",
            "authors": [
                "Haitao Wang",
                "Zhengqiu He",
                "Jin Ma",
                "Wenliang Chen",
                "Min Zhang"
            ],
            "abstract": "Inter-personal relationship is the basis of human society. In order to\nautomatically identify the relations between persons from texts, we need\nannotated data for training systems. However, there is a lack of a massive\namount of such data so far. To address this situation, we introduce IPRE, a new\ndataset for inter-personal relationship extraction which aims to facilitate\ninformation extraction and knowledge graph construction research. In total,\nIPRE has over 41,000 labeled sentences for 34 types of relations, including\nabout 9,000 sentences annotated by workers. Our data is the first dataset for\ninter-personal relationship extraction. Additionally, we define three\nevaluation tasks based on IPRE and provide the baseline systems for further\ncomparison in future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12850v1",
            "date": "2019-07-30 12:00:03+00:00",
            "title": "Confirmatory Aspect-based Opinion Mining Processes",
            "authors": [
                "Jongho Im",
                "Taikgun Song",
                "Youngsu Lee",
                "Jewoo Kim"
            ],
            "abstract": "A new opinion extraction method is proposed to summarize unstructured,\nuser-generated content (i.e., online customer reviews) in the fixed topic\ndomains. To differentiate the current approach from other opinion extraction\napproaches, which are often exposed to a sparsity problem and lack of sentiment\nscores, a confirmatory aspect-based opinion mining framework is introduced\nalong with its practical algorithm called DiSSBUS. In this procedure, 1) each\ncustomer review is disintegrated into a set of clauses; 2) each clause is\nsummarized to bi-terms-a topic word and an evaluation word-using a\npart-of-speech (POS) tagger; and 3) each bi-term is matched to a pre-specified\ntopic relevant to a specific domain. The proposed processes have two primary\nadvantages over existing methods: 1) they can decompose a single review into a\nset of bi-terms related to pre-specified topics in the domain of interest and,\ntherefore, 2) allow identification of the reviewer's opinions on the topics via\nevaluation words within the set of bi-terms. The proposed aspect-based opinion\nmining is applied to customer reviews of restaurants in Hawaii obtained from\nTripAdvisor, and the empirical findings validate the effectiveness of the\nmethod.\n  Keywords: Clause-based sentiment analysis, Customer review, Opinion mining,\nTopic modeling, User-generate-contents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12878v1",
            "date": "2019-07-30 13:16:18+00:00",
            "title": "Deep Retrieval-Based Dialogue Systems: A Short Review",
            "authors": [
                "Basma El Amel Boussaha",
                "Nicolas Hernandez",
                "Christine Jacquin",
                "Emmanuel Morin"
            ],
            "abstract": "Building dialogue systems that naturally converse with humans is being an\nattractive and an active research domain. Multiple systems are being designed\neveryday and several datasets are being available. For this reason, it is being\nhard to keep an up-to-date state-of-the-art. In this work, we present the\nlatest and most relevant retrieval-based dialogue systems and the available\ndatasets used to build and evaluate them. We discuss their limitations and\nprovide insights and guidelines for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12885v1",
            "date": "2019-07-30 13:23:01+00:00",
            "title": "Zero-shot transfer for implicit discourse relation classification",
            "authors": [
                "Murathan Kurfal\u0131",
                "Robert \u00d6stling"
            ],
            "abstract": "Automatically classifying the relation between sentences in a discourse is a\nchallenging task, in particular when there is no overt expression of the\nrelation. It becomes even more challenging by the fact that annotated training\ndata exists only for a small number of languages, such as English and Chinese.\nWe present a new system using zero-shot transfer learning for implicit\ndiscourse relation classification, where the only resource used for the target\nlanguage is unannotated parallel text. This system is evaluated on the\ndiscourse-annotated TED-MDB parallel corpus, where it obtains good results for\nall seven languages using only English training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12895v3",
            "date": "2019-07-30 13:31:49+00:00",
            "title": "MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible",
            "authors": [
                "Marcely Zanon Boito",
                "William N. Havard",
                "Mahault Garnerin",
                "\u00c9ric Le Ferrand",
                "Laurent Besacier"
            ],
            "abstract": "The CMU Wilderness Multilingual Speech Dataset (Black, 2019) is a newly\npublished multilingual speech dataset based on recorded readings of the New\nTestament. It provides data to build Automatic Speech Recognition (ASR) and\nText-to-Speech (TTS) models for potentially 700 languages. However, the fact\nthat the source content (the Bible) is the same for all the languages is not\nexploited to date.Therefore, this article proposes to add multilingual links\nbetween speech segments in different languages, and shares a large and clean\ndataset of 8,130 parallel spoken utterances across 8 languages (56 language\npairs). We name this corpus MaSS (Multilingual corpus of Sentence-aligned\nSpoken utterances). The covered languages (Basque, English, Finnish, French,\nHungarian, Romanian, Russian and Spanish) allow researches on speech-to-speech\nalignment as well as on translation for typologically different language pairs.\nThe quality of the final corpus is attested by human evaluation performed on a\ncorpus subset (100 utterances, 8 language pairs). Lastly, we showcase the\nusefulness of the final product on a bilingual speech retrieval task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12951v2",
            "date": "2019-07-30 14:00:03+00:00",
            "title": "Abstractive Document Summarization without Parallel Data",
            "authors": [
                "Nikola I. Nikolov",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "Abstractive summarization typically relies on large collections of paired\narticles and summaries. However, in many cases, parallel data is scarce and\ncostly to obtain. We develop an abstractive summarization system that relies\nonly on large collections of example summaries and non-matching articles. Our\napproach consists of an unsupervised sentence extractor that selects salient\nsentences to include in the final summary, as well as a sentence abstractor\nthat is trained on pseudo-parallel and synthetic data, that paraphrases each of\nthe extracted sentences. We perform an extensive evaluation of our method: on\nthe CNN/DailyMail benchmark, on which we compare our approach to fully\nsupervised baselines, as well as on the novel task of automatically generating\na press release from a scientific journal article, which is well suited for our\nsystem. We show promising performance on both tasks, without relying on any\narticle-summary pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12984v2",
            "date": "2019-07-30 14:35:06+00:00",
            "title": "DuTongChuan: Context-aware Translation Model for Simultaneous Interpreting",
            "authors": [
                "Hao Xiong",
                "Ruiqing Zhang",
                "Chuanqiang Zhang",
                "Zhongjun He",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "In this paper, we present DuTongChuan, a novel context-aware translation\nmodel for simultaneous interpreting. This model allows to constantly read\nstreaming text from the Automatic Speech Recognition (ASR) model and\nsimultaneously determine the boundaries of Information Units (IUs) one after\nanother. The detected IU is then translated into a fluent translation with two\nsimple yet effective decoding strategies: partial decoding and context-aware\ndecoding. In practice, by controlling the granularity of IUs and the size of\nthe context, we can get a good trade-off between latency and translation\nquality easily. Elaborate evaluation from human translators reveals that our\nsystem achieves promising translation quality (85.71% for Chinese-English, and\n86.36% for English-Chinese), specially in the sense of surprisingly good\ndiscourse coherence. According to an End-to-End (speech-to-speech simultaneous\ninterpreting) evaluation, this model presents impressive performance in\nreducing latency (to less than 3 seconds at most times). Furthermore, we\nsuccessfully deploy this model in a variety of Baidu's products which have\nhundreds of millions of users, and we release it as a service in our AI\nplatform.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13280v1",
            "date": "2019-07-31 01:37:58+00:00",
            "title": "Learning Question-Guided Video Representation for Multi-Turn Video Question Answering",
            "authors": [
                "Guan-Lin Chao",
                "Abhinav Rastogi",
                "Semih Yavuz",
                "Dilek Hakkani-T\u00fcr",
                "Jindong Chen",
                "Ian Lane"
            ],
            "abstract": "Understanding and conversing about dynamic scenes is one of the key\ncapabilities of AI agents that navigate the environment and convey useful\ninformation to humans. Video question answering is a specific scenario of such\nAI-human interaction where an agent generates a natural language response to a\nquestion regarding the video of a dynamic scene. Incorporating features from\nmultiple modalities, which often provide supplementary information, is one of\nthe challenging aspects of video question answering. Furthermore, a question\noften concerns only a small segment of the video, hence encoding the entire\nvideo sequence using a recurrent neural network is not computationally\nefficient. Our proposed question-guided video representation module efficiently\ngenerates the token-level video summary guided by each word in the question.\nThe learned representations are then fused with the question to generate the\nanswer. Through empirical evaluation on the Audio Visual Scene-aware Dialog\n(AVSD) dataset, our proposed models in single-turn and multi-turn question\nanswering achieve state-of-the-art performance on several automatic natural\nlanguage generation evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13356v2",
            "date": "2019-07-31 07:53:57+00:00",
            "title": "Normalyzing Numeronyms -- A NLP approach",
            "authors": [
                "Avishek Garain",
                "Sainik Kumar Mahata",
                "Subhabrata Dutta"
            ],
            "abstract": "This paper presents a method to apply Natural Language Processing for\nnormalizing numeronyms to make them understandable by humans. We approach the\nproblem through a two-step mechanism. We make use of the state of the art\nLevenshtein distance of words. We then apply Cosine Similarity for selection of\nthe normalized text and reach greater accuracy in solving the problem. Our\napproach garners accuracy figures of 71\\% and 72\\% for Bengali and English\nlanguage, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13362v1",
            "date": "2019-07-31 08:40:26+00:00",
            "title": "On conducting better validation studies of automatic metrics in natural language generation evaluation",
            "authors": [
                "Johnny Tian-Zheng Wei"
            ],
            "abstract": "Natural language generation (NLG) has received increasing attention, which\nhas highlighted evaluation as a central methodological concern. Since human\nevaluations for these systems are costly, automatic metrics have broad appeal\nin NLG. Research in language generation often finds situations where it is\nappropriate to apply existing metrics or propose new ones. The application of\nthese metrics are entirely dependent on validation studies - studies that\ndetermine a metric's correlation to human judgment. However, there are many\ndetails and considerations in conducting strong validation studies. This\ndocument is intended for those validating existing metrics or proposing new\nones in the broad context of NLG: we 1) begin with a write-up of best practices\nin validation studies, 2) outline how to adopt these practices, 3) conduct\nanalyses in the WMT'17 metrics shared task\\footnote{Our jupyter notebook\ncontaining the analyses is available at \\url{https://github.com}}, and 4)\nhighlight promising approaches to NLG metrics 5) conclude with our opinions on\nthe future of this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00059v2",
            "date": "2019-07-31 19:23:38+00:00",
            "title": "GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension",
            "authors": [
                "Yu Chen",
                "Lingfei Wu",
                "Mohammed J. Zaki"
            ],
            "abstract": "Conversational machine comprehension (MC) has proven significantly more\nchallenging compared to traditional MC since it requires better utilization of\nconversation history. However, most existing approaches do not effectively\ncapture conversation history and thus have trouble handling questions involving\ncoreference or ellipsis. Moreover, when reasoning over passage text, most of\nthem simply treat it as a word sequence without exploring rich semantic\nrelationships among words. In this paper, we first propose a simple yet\neffective graph structure learning technique to dynamically construct a\nquestion and conversation history aware context graph at each conversation\nturn. Then we propose a novel Recurrent Graph Neural Network, and based on\nthat, we introduce a flow mechanism to model the temporal dependencies in a\nsequence of context graphs. The proposed GraphFlow model can effectively\ncapture conversational flow in a dialog, and shows competitive performance\ncompared to existing state-of-the-art methods on CoQA, QuAC and DoQA\nbenchmarks. In addition, visualization experiments show that our proposed model\ncan offer good interpretability for the reasoning process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00321v1",
            "date": "2019-08-01 10:52:26+00:00",
            "title": "Sentiment Analysis at SEPLN (TASS)-2019: Sentiment Analysis at Tweet level using Deep Learning",
            "authors": [
                "Avishek Garain",
                "Sainik Kumar Mahata"
            ],
            "abstract": "This paper describes the system submitted to \"Sentiment Analysis at SEPLN\n(TASS)-2019\" shared task. The task includes sentiment analysis of Spanish\ntweets, where the tweets are in different dialects spoken in Spain, Peru, Costa\nRica, Uruguay and Mexico. The tweets are short (up to 240 characters) and the\nlanguage is informal, i.e., it contains misspellings, emojis, onomatopeias etc.\nSentiment analysis includes classification of the tweets into 4 classes, viz.,\nPositive, Negative, Neutral and None. For preparing the proposed system, we use\nDeep Learning networks like LSTMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00323v1",
            "date": "2019-08-01 10:57:31+00:00",
            "title": "JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to English",
            "authors": [
                "Sainik Kumar Mahata",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "In the current work, we present a description of the system submitted to WMT\n2018 News Translation Shared task. The system was created to translate news\ntext from Finnish to English. The system used a Character Based Neural Machine\nTranslation model to accomplish the given task. The current paper documents the\npreprocessing steps, the description of the submitted system and the results\nproduced using the same. Our system garnered a BLEU score of 12.9.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00588v1",
            "date": "2019-08-01 19:24:59+00:00",
            "title": "Visualizing RNN States with Predictive Semantic Encodings",
            "authors": [
                "Lindsey Sawatzky",
                "Steven Bergner",
                "Fred Popowich"
            ],
            "abstract": "Recurrent Neural Networks are an effective and prevalent tool used to model\nsequential data such as natural language text. However, their deep nature and\nmassive number of parameters pose a challenge for those intending to study\nprecisely how they work. We present a visual technique that gives a high level\nintuition behind the semantics of the hidden states within Recurrent Neural\nNetworks. This semantic encoding allows for hidden states to be compared\nthroughout the model independent of their internal details. The proposed\ntechnique is displayed in a proof of concept visualization tool which is\ndemonstrated to visualize the natural language processing task of language\nmodelling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00908v1",
            "date": "2019-08-02 15:30:06+00:00",
            "title": "Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language",
            "authors": [
                "Sandeep Nallan Chakravarthula",
                "Haoqi Li",
                "Shao-Yen Tseng",
                "Maija Reblin",
                "Panayiotis Georgiou"
            ],
            "abstract": "Cancer impacts the quality of life of those diagnosed as well as their spouse\ncaregivers, in addition to potentially influencing their day-to-day behaviors.\nThere is evidence that effective communication between spouses can improve\nwell-being related to cancer but it is difficult to efficiently evaluate the\nquality of daily life interactions using manual annotation frameworks.\nAutomated recognition of behaviors based on the interaction cues of speakers\ncan help analyze interactions in such couples and identify behaviors which are\nbeneficial for effective communication. In this paper, we present and detail a\ndataset of dyadic interactions in 85 real-life cancer-afflicted couples and a\nset of observational behavior codes pertaining to interpersonal communication\nattributes. We describe and employ neural network-based systems for classifying\nthese behaviors based on turn-level acoustic and lexical speech patterns.\nFurthermore, we investigate the effect of controlling for factors such as\ngender, patient/caregiver role and conversation content on behavior\nclassification. Analysis of our preliminary results indicates the challenges in\nthis task due to the nature of the targeted behaviors and suggests that\ntechniques incorporating contextual processing might be better suited to tackle\nthis problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00916v1",
            "date": "2019-08-02 15:39:31+00:00",
            "title": "A Speech Test Set of Practice Business Presentations with Additional Relevant Texts",
            "authors": [
                "Dominik Mach\u00e1\u010dek",
                "Jon\u00e1\u0161 Kratochv\u00edl",
                "Tereza Vojt\u011bchov\u00e1",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "We present a test corpus of audio recordings and transcriptions of\npresentations of students' enterprises together with their slides and\nweb-pages. The corpus is intended for evaluation of automatic speech\nrecognition (ASR) systems, especially in conditions where the prior\navailability of in-domain vocabulary and named entities is benefitable. The\ncorpus consists of 39 presentations in English, each up to 90 seconds long. The\nspeakers are high school students from European countries with English as their\nsecond language. We benchmark three baseline ASR systems on the corpus and show\ntheir imperfection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01192v1",
            "date": "2019-08-03 15:38:19+00:00",
            "title": "The TALP-UPC System for the WMT Similar Language Task: Statistical vs Neural Machine Translation",
            "authors": [
                "Magdalena Biesialska",
                "Lluis Guardia",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "Although the problem of similar language translation has been an area of\nresearch interest for many years, yet it is still far from being solved. In\nthis paper, we study the performance of two popular approaches: statistical and\nneural. We conclude that both methods yield similar results; however, the\nperformance varies depending on the language pair. While the statistical\napproach outperforms the neural one by a difference of 6 BLEU points for the\nSpanish-Portuguese language pair, the proposed neural model surpasses the\nstatistical one by a difference of 2 BLEU points for Czech-Polish. In the\nformer case, the language similarity (based on perplexity) is much higher than\nin the latter case. Additionally, we report negative results for the system\ncombination with back-translation. Our TALP-UPC system submission won 1st place\nfor Czech-to-Polish and 2nd place for Spanish-to-Portuguese in the official\nevaluation of the 1st WMT Similar Language Translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01294v2",
            "date": "2019-08-04 08:24:05+00:00",
            "title": "Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations",
            "authors": [
                "Chanatip Saetia",
                "Ekapol Chuangsuwanich",
                "Tawunrat Chalothorn",
                "Peerapon Vateekul"
            ],
            "abstract": "A sentence is typically treated as the minimal syntactic unit used for\nextracting valuable information from a longer piece of text. However, in\nwritten Thai, there are no explicit sentence markers. We proposed a deep\nlearning model for the task of sentence segmentation that includes three main\ncontributions. First, we integrate n-gram embedding as a local representation\nto capture word groups near sentence boundaries. Second, to focus on the\nkeywords of dependent clauses, we combine the model with a distant\nrepresentation obtained from self-attention modules. Finally, due to the\nscarcity of labeled data, for which annotation is difficult and time-consuming,\nwe also investigate and adapt Cross-View Training (CVT) as a semi-supervised\nlearning technique, allowing us to utilize unlabeled data to improve the model\nrepresentations. In the Thai sentence segmentation experiments, our model\nreduced the relative error by 7.4% and 10.5% compared with the baseline models\non the Orchid and UGWC datasets, respectively. We also applied our model to the\ntask of pronunciation recovery on the IWSLT English dataset. Our model\noutperformed the prior sequence tagging models, achieving a relative error\nreduction of 2.5%. Ablation studies revealed that utilizing n-gram\npresentations was the main contributing factor for Thai, while the\nsemi-supervised training helped the most for English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01349v1",
            "date": "2019-08-01 11:03:51+00:00",
            "title": "JUMT at WMT2019 News Translation Task: A Hybrid approach to Machine Translation for Lithuanian to English",
            "authors": [
                "Sainik Kumar Mahata",
                "Avishek Garain",
                "Adityar Rayala",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "In the current work, we present a description of the system submitted to WMT\n2019 News Translation Shared task. The system was created to translate news\ntext from Lithuanian to English. To accomplish the given task, our system used\na Word Embedding based Neural Machine Translation model to post edit the\noutputs generated by a Statistical Machine Translation model. The current paper\ndocuments the architecture of our model, descriptions of the various modules\nand the results produced using the same. Our system garnered a BLEU score of\n17.6.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01355v2",
            "date": "2019-08-04 14:46:35+00:00",
            "title": "Separating Argument Structure from Logical Structure in AMR",
            "authors": [
                "Johan Bos"
            ],
            "abstract": "The AMR (Abstract Meaning Representation) formalism for representing meaning\nof natural language sentences was not designed to deal with scope and\nquantifiers. By extending AMR with indices for contexts and formulating\nconstraints on these contexts, a formalism is derived that makes correct\nprediction for inferences involving negation and bound variables. The\nattractive core predicate-argument structure of AMR is preserved. The resulting\nframework is similar to that of Discourse Representation Theory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01665v2",
            "date": "2019-08-05 14:56:01+00:00",
            "title": "Predicting Actions to Help Predict Translations",
            "authors": [
                "Zixiu Wu",
                "Julia Ive",
                "Josiah Wang",
                "Pranava Madhyastha",
                "Lucia Specia"
            ],
            "abstract": "We address the task of text translation on the How2 dataset using a state of\nthe art transformer-based multimodal approach. The question we ask ourselves is\nwhether visual features can support the translation process, in particular,\ngiven that this is a dataset extracted from videos, we focus on the translation\nof actions, which we believe are poorly captured in current static image-text\ndatasets currently used for multimodal translation. For that purpose, we\nextract different types of action features from the videos and carefully\ninvestigate how helpful this visual information is by testing whether it can\nincrease translation quality when used in conjunction with (i) the original\ntext and (ii) the original text where action-related words (or all verbs) are\nmasked out. The latter is a simulation that helps us assess the utility of the\nimage in cases where the text does not provide enough context about the action,\nor in the presence of noise in the input text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01674v2",
            "date": "2019-08-05 15:05:48+00:00",
            "title": "Processamento de linguagem natural em Portugu\u00eas e aprendizagem profunda para o dom\u00ednio de \u00d3leo e G\u00e1s",
            "authors": [
                "Diogo Gomes",
                "Alexandre Evsukoff"
            ],
            "abstract": "Over the last few decades, institutions around the world have been challenged\nto deal with the sheer volume of information captured in unstructured formats,\nespecially in textual documents. The so called Digital Transformation age,\ncharacterized by important technological advances and the advent of disruptive\nmethods in Artificial Intelligence, offers opportunities to make better use of\nthis information. Recent techniques in Natural Language Processing (NLP) with\nDeep Learning approaches allow to efficiently process a large volume of data in\norder to obtain relevant information, to identify patterns, classify text,\namong other applications. In this context, the highly technical vocabulary of\nOil and Gas (O&G) domain represents a challenge for these NLP algorithms, in\nwhich terms can assume a very different meaning in relation to common sense\nunderstanding. The search for suitable mathematical representations and\nspecific models requires a large amount of representative corpora in the O&G\ndomain. However, public access to this material is scarce in the scientific\nliterature, especially considering the Portuguese language. This paper presents\na literature review about the main techniques for deep learning NLP and their\nmajor applications for O&G domain in Portuguese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01761v3",
            "date": "2019-07-26 11:29:37+00:00",
            "title": "Hybrid Neural Tagging Model for Open Relation Extraction",
            "authors": [
                "Shengbin Jia",
                "Yang Xiang"
            ],
            "abstract": "Open relation extraction (ORE) remains a challenge to obtain a semantic\nrepresentation by discovering arbitrary relation tuples from the unstructured\ntext. Conventional methods heavily depend on feature engineering or syntactic\nparsing, they are inefficient or error-cascading. Recently, leveraging\nsupervised deep learning structures to address the ORE task is an\nextraordinarily promising way. However, there are two main challenges: (1) The\nlack of enough labeled corpus to support supervised training; (2) The\nexploration of specific neural architecture that adapts to the characteristics\nof open relation extracting. In this paper, to overcome these difficulties, we\nbuild a large-scale, high-quality training corpus in a fully automated way, and\ndesign a tagging scheme to assist in transforming the ORE task into a sequence\ntagging processing. Furthermore, we propose a hybrid neural network model\n(HNN4ORT) for open relation tagging. The model employs the Ordered Neurons LSTM\nto encode potential syntactic information for capturing the associations among\nthe arguments and relations. It also emerges a novel Dual Aware Mechanism,\nincluding Local-aware Attention and Global-aware Convolution. The dual aware\nnesses complement each other so that the model can take the sentence-level\nsemantics as a global perspective, and at the same time implement salient local\nfeatures to achieve sparse annotation. Experimental results on various testing\nsets show that our model can achieve state-of-the-art performances compared to\nthe conventional methods or other neural models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02262v1",
            "date": "2019-08-06 17:19:46+00:00",
            "title": "Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations",
            "authors": [
                "Aarne Talman",
                "Antti Suni",
                "Hande Celikkanat",
                "Sofoklis Kakouros",
                "J\u00f6rg Tiedemann",
                "Martti Vainio"
            ],
            "abstract": "In this paper we introduce a new natural language processing dataset and\nbenchmark for predicting prosodic prominence from written text. To our\nknowledge this will be the largest publicly available dataset with prosodic\nlabels. We describe the dataset construction and the resulting benchmark\ndataset in detail and train a number of different models ranging from\nfeature-based classifiers to neural network systems for the prediction of\ndiscretized prosodic prominence. We show that pre-trained contextualized word\nrepresentations from BERT outperform the other models even with less than 10%\nof the training data. Finally we discuss the dataset in light of the results\nand point to future research and plans for further improving both the dataset\nand methods of predicting prosodic prominence from text. The dataset and the\ncode for the models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02285v1",
            "date": "2019-08-06 09:57:08+00:00",
            "title": "Text Summarization in the Biomedical Domain",
            "authors": [
                "Milad Moradi",
                "Nasser Ghadiri"
            ],
            "abstract": "This chapter gives an overview of recent advances in the field of biomedical\ntext summarization. Different types of challenges are introduced, and methods\nare discussed concerning the type of challenge that they address. Biomedical\nliterature summarization is explored as a leading trend in the field, and some\nfuture lines of work are pointed out. Underlying methods of recent\nsummarization systems are briefly explained and the most significant evaluation\nresults are mentioned. The primary purpose of this chapter is to review the\nmost significant research efforts made in the current decade toward new methods\nof biomedical text summarization. As the main parts of this chapter, current\ntrends are discussed and new challenges are introduced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02286v2",
            "date": "2019-08-06 10:18:20+00:00",
            "title": "Clustering of Deep Contextualized Representations for Summarization of Biomedical Texts",
            "authors": [
                "Milad Moradi",
                "Matthias Samwald"
            ],
            "abstract": "In recent years, summarizers that incorporate domain knowledge into the\nprocess of text summarization have outperformed generic methods, especially for\nsummarization of biomedical texts. However, construction and maintenance of\ndomain knowledge bases are resource-intense tasks requiring significant manual\nannotation. In this paper, we demonstrate that contextualized representations\nextracted from the pre-trained deep language model BERT, can be effectively\nused to measure the similarity between sentences and to quantify the\ninformative content. The results show that our BERT-based summarizer can\nimprove the performance of biomedical summarization. Although the summarizer\ndoes not use any sources of domain knowledge, it can capture the context of\nsentences more accurately than the comparison methods. The source code and data\nare available at https://github.com/BioTextSumm/BERT-based-Summ.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02322v1",
            "date": "2019-08-06 18:50:45+00:00",
            "title": "DpgMedia2019: A Dutch News Dataset for Partisanship Detection",
            "authors": [
                "Chia-Lun Yeh",
                "Babak Loni",
                "Mari\u00eblle Hendriks",
                "Henrike Reinhardt",
                "Anne Schuth"
            ],
            "abstract": "We present a new Dutch news dataset with labeled partisanship. The dataset\ncontains more than 100K articles that are labeled on the publisher level and\n776 articles that were crowdsourced using an internal survey platform and\nlabeled on the article level. In this paper, we document our original\nmotivation, the collection and annotation process, limitations, and\napplications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02404v1",
            "date": "2019-08-07 00:04:58+00:00",
            "title": "Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Merging",
            "authors": [
                "Binh Nguyen",
                "Vu Bao Hung Nguyen",
                "Hien Nguyen",
                "Pham Ngoc Phuong",
                "The-Loc Nguyen",
                "Quoc Truong Do",
                "Luong Chi Mai"
            ],
            "abstract": "In recent years, studies on automatic speech recognition (ASR) have shown\noutstanding results that reach human parity on short speech segments. However,\nthere are still difficulties in standardizing the output of ASR such as\ncapitalization and punctuation restoration for long-speech transcription. The\nproblems obstruct readers to understand the ASR output semantically and also\ncause difficulties for natural language processing models such as NER, POS and\nsemantic parsing. In this paper, we propose a method to restore the punctuation\nand capitalization for long-speech ASR transcription. The method is based on\nTransformer models and chunk merging that allows us to (1), build a single\nmodel that performs punctuation and capitalization in one go, and (2), perform\ndecoding in parallel while improving the prediction accuracy. Experiments on\nBritish National Corpus showed that the proposed approach outperforms existing\nmethods in both accuracy and decoding speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02477v3",
            "date": "2019-08-07 08:03:08+00:00",
            "title": "Ab Antiquo: Neural Proto-language Reconstruction",
            "authors": [
                "Carlo Meloni",
                "Shauli Ravfogel",
                "Yoav Goldberg"
            ],
            "abstract": "Historical linguists have identified regularities in the process of historic\nsound change. The comparative method utilizes those regularities to reconstruct\nproto-words based on observed forms in daughter languages. Can this process be\nefficiently automated? We address the task of proto-word reconstruction, in\nwhich the model is exposed to cognates in contemporary daughter languages, and\nhas to predict the proto word in the ancestor language. We provide a novel\ndataset for this task, encompassing over 8,000 comparative entries, and show\nthat neural sequence models outperform conventional methods applied to this\ntask so far. Error analysis reveals variability in the ability of neural model\nto capture different phonological changes, correlating with the complexity of\nthe changes. Analysis of learned embeddings reveals the models learn\nphonologically meaningful generalizations, corresponding to well-attested\nphonological shifts documented by historical linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02505v1",
            "date": "2019-08-07 09:44:31+00:00",
            "title": "Embedding-based system for the Text part of CALL v3 shared task",
            "authors": [
                "Volodymyr Sokhatskyi",
                "Olga Zvyeryeva",
                "Ievgen Karaulov",
                "Dmytro Tkanov"
            ],
            "abstract": "This paper presents a scoring system that has shown the top result on the\ntext subset of CALL v3 shared task. The presented system is based on text\nembeddings, namely NNLM~\\cite{nnlm} and BERT~\\cite{Bert}. The distinguishing\nfeature of the given approach is that it does not rely on the reference grammar\nfile for scoring. The model is compared against approaches that use the grammar\nfile and proves the possibility to achieve similar and even higher results\nwithout a predefined set of correct answers.\n  The paper describes the model itself and the data preparation process that\nplayed a crucial role in the model training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02579v2",
            "date": "2019-08-07 12:32:33+00:00",
            "title": "A Simple and Effective Approach for Fine Tuning Pre-trained Word Embeddings for Improved Text Classification",
            "authors": [
                "Amr Al-Khatib",
                "Samhaa R. El-Beltagy"
            ],
            "abstract": "This work presents a new and simple approach for fine-tuning pretrained word\nembeddings for text classification tasks. In this approach, the class in which\na term appears, acts as an additional contextual variable during the fine\ntuning process, and contributes to the final word vector for that term. As a\nresult, words that are used distinctively within a particular class, will bear\nvectors that are closer to each other in the embedding space and will be more\ndiscriminative towards that class. To validate this novel approach, it was\napplied to three Arabic and two English datasets that have been previously used\nfor text classification tasks such as sentiment analysis and emotion detection.\nIn the vast majority of cases, the results obtained using the proposed\napproach, improved considerably.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02895v1",
            "date": "2019-08-08 01:16:34+00:00",
            "title": "Neural Network based Deep Transfer Learning for Cross-domain Dependency Parsing",
            "authors": [
                "Zhentao Xia",
                "Likai Wang",
                "Weiguang Qu",
                "Junsheng Zhou",
                "Yanhui Gu"
            ],
            "abstract": "In this paper, we describe the details of the neural dependency parser\nsub-mitted by our team to the NLPCC 2019 Shared Task of Semi-supervised do-main\nadaptation subtask on Cross-domain Dependency Parsing. Our system is based on\nthe stack-pointer networks(STACKPTR). Considering the im-portance of context,\nwe utilize self-attention mechanism for the representa-tion vectors to capture\nthe meaning of words. In addition, to adapt three dif-ferent domains, we\nutilize neural network based deep transfer learning which transfers the\npre-trained partial network in the source domain to be a part of deep neural\nnetwork in the three target domains (product comments, product blogs and web\nfiction) respectively. Results on the three target domains demonstrate that our\nmodel performs competitively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02899v1",
            "date": "2019-08-08 01:41:16+00:00",
            "title": "Do Neural Language Representations Learn Physical Commonsense?",
            "authors": [
                "Maxwell Forbes",
                "Ari Holtzman",
                "Yejin Choi"
            ],
            "abstract": "Humans understand language based on the rich background knowledge about how\nthe physical world works, which in turn allows us to reason about the physical\nworld through language. In addition to the properties of objects (e.g., boats\nrequire fuel) and their affordances, i.e., the actions that are applicable to\nthem (e.g., boats can be driven), we can also reason about if-then inferences\nbetween what properties of objects imply the kind of actions that are\napplicable to them (e.g., that if we can drive something then it likely\nrequires fuel).\n  In this paper, we investigate the extent to which state-of-the-art neural\nlanguage representations, trained on a vast amount of natural language text,\ndemonstrate physical commonsense reasoning. While recent advancements of neural\nlanguage models have demonstrated strong performance on various types of\nnatural language inference tasks, our study based on a dataset of over 200k\nnewly collected annotations suggests that neural language representations still\nonly learn associations that are explicitly written down.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02914v1",
            "date": "2019-08-08 03:31:11+00:00",
            "title": "Mitigating Noisy Inputs for Question Answering",
            "authors": [
                "Denis Peskov",
                "Joe Barrow",
                "Pedro Rodriguez",
                "Graham Neubig",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Natural language processing systems are often downstream of unreliable\ninputs: machine translation, optical character recognition, or speech\nrecognition. For instance, virtual assistants can only answer your questions\nafter understanding your speech. We investigate and mitigate the effects of\nnoise from Automatic Speech Recognition systems on two factoid Question\nAnswering (QA) tasks. Integrating confidences into the model and forced\ndecoding of unknown words are empirically shown to improve the accuracy of\ndownstream neural QA systems. We create and train models on a synthetic corpus\nof over 500,000 noisy sentences and evaluate on two human corpora from Quizbowl\nand Jeopardy! competitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03043v1",
            "date": "2019-08-08 12:50:23+00:00",
            "title": "A Test Suite and Manual Evaluation of Document-Level NMT at WMT19",
            "authors": [
                "Kate\u0159ina Rysov\u00e1",
                "Magdal\u00e9na Rysov\u00e1",
                "Tom\u00e1\u0161 Musil",
                "Lucie Pol\u00e1kov\u00e1",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "As the quality of machine translation rises and neural machine translation\n(NMT) is moving from sentence to document level translations, it is becoming\nincreasingly difficult to evaluate the output of translation systems.\n  We provide a test suite for WMT19 aimed at assessing discourse phenomena of\nMT systems participating in the News Translation Task. We have manually checked\nthe outputs and identified types of translation errors that are relevant to\ndocument-level translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03067v1",
            "date": "2019-08-08 13:41:31+00:00",
            "title": "Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation",
            "authors": [
                "Shuming Ma",
                "Pengcheng Yang",
                "Tianyu Liu",
                "Peng Li",
                "Jie Zhou",
                "Xu Sun"
            ],
            "abstract": "Table-to-text generation aims to translate the structured data into the\nunstructured text. Most existing methods adopt the encoder-decoder framework to\nlearn the transformation, which requires large-scale training samples. However,\nthe lack of large parallel data is a major practical problem for many domains.\nIn this work, we consider the scenario of low resource table-to-text\ngeneration, where only limited parallel data is available. We propose a novel\nmodel to separate the generation into two stages: key fact prediction and\nsurface realization. It first predicts the key facts from the tables, and then\ngenerates the text with the key facts. The training of key fact prediction\nneeds much fewer annotated data, while surface realization can be trained with\npseudo parallel corpus. We evaluate our model on a biography generation\ndataset. Our model can achieve $27.34$ BLEU score with only $1,000$ parallel\ndata, while the baseline model only obtain the performance of $9.71$ BLEU\nscore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03402v1",
            "date": "2019-08-09 10:40:52+00:00",
            "title": "UdS Submission for the WMT 19 Automatic Post-Editing Task",
            "authors": [
                "Hongfei Xu",
                "Qiuhui Liu",
                "Josef van Genabith"
            ],
            "abstract": "In this paper, we describe our submission to the English-German APE shared\ntask at WMT 2019. We utilize and adapt an NMT architecture originally developed\nfor exploiting context information to APE, implement this in our own\ntransformer model and explore joint training of the APE task with a de-noising\nencoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03480v2",
            "date": "2019-08-09 14:43:36+00:00",
            "title": "Artificially Evolved Chunks for Morphosyntactic Analysis",
            "authors": [
                "Mark Anderson",
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We introduce a language-agnostic evolutionary technique for automatically\nextracting chunks from dependency treebanks. We evaluate these chunks on a\nnumber of morphosyntactic tasks, namely POS tagging, morphological feature\ntagging, and dependency parsing. We test the utility of these chunks in a host\nof different ways. We first learn chunking as one task in a shared multi-task\nframework together with POS and morphological feature tagging. The predictions\nfrom this network are then used as input to augment sequence-labelling\ndependency parsing. Finally, we investigate the impact chunks have on\ndependency parsing in a multi-task framework. Our results from these analyses\nshow that these chunks improve performance at different levels of syntactic\nabstraction on English UD treebanks and a small, diverse subset of non-English\nUD treebanks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03594v2",
            "date": "2019-08-09 18:51:30+00:00",
            "title": "Generating Information Extraction Patterns from Overlapping and Variable Length Annotations using Sequence Alignment",
            "authors": [
                "Frank Meng",
                "Craig A. Morioka",
                "Danne C. Elbers"
            ],
            "abstract": "Sequence alignments are used to capture patterns composed of elements\nrepresenting multiple conceptual levels through the alignment of sequences that\ncontain overlapping and variable length annotations. The alignments also\ndetermine the proper context window of words and phrases that most directly\nimpact the meaning of a given target within a sentence, eliminating the need to\npredefine a fixed context window of words surrounding the targets. We evaluated\nthe system using the CoNLL-2003 named entity recognition (NER) task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04212v1",
            "date": "2019-08-12 15:49:57+00:00",
            "title": "A Finnish News Corpus for Named Entity Recognition",
            "authors": [
                "Teemu Ruokolainen",
                "Pekka Kauppinen",
                "Miikka Silfverberg",
                "Krister Lind\u00e9n"
            ],
            "abstract": "We present a corpus of Finnish news articles with a manually prepared named\nentity annotation. The corpus consists of 953 articles (193,742 word tokens)\nwith six named entity classes (organization, location, person, product, event,\nand date). The articles are extracted from the archives of Digitoday, a Finnish\nonline technology news source. The corpus is available for research purposes.\nWe present baseline experiments on the corpus using a rule-based and two deep\nlearning systems on two, in-domain and out-of-domain, test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04485v1",
            "date": "2019-08-13 04:44:22+00:00",
            "title": "Understanding Spatial Language in Radiology: Representation Framework, Annotation, and Spatial Relation Extraction from Chest X-ray Reports using Deep Learning",
            "authors": [
                "Surabhi Datta",
                "Yuqi Si",
                "Laritza Rodriguez",
                "Sonya E Shooshan",
                "Dina Demner-Fushman",
                "Kirk Roberts"
            ],
            "abstract": "We define a representation framework for extracting spatial information from\nradiology reports (Rad-SpRL). We annotated a total of 2000 chest X-ray reports\nwith 4 spatial roles corresponding to the common radiology entities. Our focus\nis on extracting detailed information of a radiologist's interpretation\ncontaining a radiographic finding, its anatomical location, corresponding\nprobable diagnoses, as well as associated hedging terms. For this, we propose a\ndeep learning-based natural language processing (NLP) method involving both\nword and character-level encodings. Specifically, we utilize a bidirectional\nlong short-term memory (Bi-LSTM) conditional random field (CRF) model for\nextracting the spatial roles. The model achieved average F1 measures of 90.28\nand 94.61 for extracting the Trajector and Landmark roles respectively whereas\nthe performance was moderate for Diagnosis and Hedge roles with average F1 of\n71.47 and 73.27 respectively. The corpus will soon be made available upon\nrequest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04530v2",
            "date": "2019-08-13 08:28:05+00:00",
            "title": "Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning",
            "authors": [
                "Jiangnan Xia",
                "Chen Wu",
                "Ming Yan"
            ],
            "abstract": "This paper focuses on how to take advantage of external relational knowledge\nto improve machine reading comprehension (MRC) with multi-task learning. Most\nof the traditional methods in MRC assume that the knowledge used to get the\ncorrect answer generally exists in the given documents. However, in real-world\ntask, part of knowledge may not be mentioned and machines should be equipped\nwith the ability to leverage external knowledge. In this paper, we integrate\nrelational knowledge into MRC model for commonsense reasoning. Specifically,\nbased on a pre-trained language model (LM). We design two auxiliary\nrelation-aware tasks to predict if there exists any commonsense relation and\nwhat is the relation type between two words, in order to better model the\ninteractions between document and candidate answer option. We conduct\nexperiments on two multi-choice benchmark datasets: the SemEval-2018 Task 11\nand the Cloze Story Test. The experimental results demonstrate the\neffectiveness of the proposed method, which achieves superior performance\ncompared with the comparable baselines on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04531v1",
            "date": "2019-08-13 08:29:48+00:00",
            "title": "Offensive Language and Hate Speech Detection for Danish",
            "authors": [
                "Gudbjartur Ingi Sigurbergsson",
                "Leon Derczynski"
            ],
            "abstract": "The presence of offensive language on social media platforms and the\nimplications this poses is becoming a major concern in modern society. Given\nthe enormous amount of content created every day, automatic methods are\nrequired to detect and deal with this type of content. Until now, most of the\nresearch has focused on solving the problem for the English language, while the\nproblem is multilingual.\n  We construct a Danish dataset containing user-generated comments from\n\\textit{Reddit} and \\textit{Facebook}. It contains user generated comments from\nvarious social media platforms, and to our knowledge, it is the first of its\nkind. Our dataset is annotated to capture various types and target of offensive\nlanguage. We develop four automatic classification systems, each designed to\nwork for both the English and the Danish language. In the detection of\noffensive language in English, the best performing system achieves a macro\naveraged F1-score of $0.74$, and the best performing system for Danish achieves\na macro averaged F1-score of $0.70$. In the detection of whether or not an\noffensive post is targeted, the best performing system for English achieves a\nmacro averaged F1-score of $0.62$, while the best performing system for Danish\nachieves a macro averaged F1-score of $0.73$. Finally, in the detection of the\ntarget type in a targeted offensive post, the best performing system for\nEnglish achieves a macro averaged F1-score of $0.56$, and the best performing\nsystem for Danish achieves a macro averaged F1-score of $0.63$.\n  Our work for both the English and the Danish language captures the type and\ntargets of offensive language, and present automatic methods for detecting\ndifferent kinds of offensive language such as hate speech and cyberbullying.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04567v2",
            "date": "2019-08-13 10:17:11+00:00",
            "title": "EASSE: Easier Automatic Sentence Simplification Evaluation",
            "authors": [
                "Fernando Alva-Manchego",
                "Louis Martin",
                "Carolina Scarton",
                "Lucia Specia"
            ],
            "abstract": "We introduce EASSE, a Python package aiming to facilitate and standardise\nautomatic evaluation and comparison of Sentence Simplification (SS) systems.\nEASSE provides a single access point to a broad range of evaluation resources:\nstandard automatic metrics for assessing SS outputs (e.g. SARI), word-level\naccuracy scores for certain simplification transformations,\nreference-independent quality estimation features (e.g. compression ratio), and\nstandard test data for SS evaluation (e.g. TurkCorpus). Finally, EASSE\ngenerates easy-to-visualise reports on the various metrics and features above\nand on how a particular SS output fares against reference simplifications.\nThrough experiments, we show that these functionalities allow for better\ncomparison and understanding of the performance of SS systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04577v3",
            "date": "2019-08-13 11:12:58+00:00",
            "title": "StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding",
            "authors": [
                "Wei Wang",
                "Bin Bi",
                "Ming Yan",
                "Chen Wu",
                "Zuyi Bao",
                "Jiangnan Xia",
                "Liwei Peng",
                "Luo Si"
            ],
            "abstract": "Recently, the pre-trained language model, BERT (and its robustly optimized\nversion RoBERTa), has attracted a lot of attention in natural language\nunderstanding (NLU), and achieved state-of-the-art accuracy in various NLU\ntasks, such as sentiment classification, natural language inference, semantic\ntextual similarity and question answering. Inspired by the linearization\nexploration work of Elman [8], we extend BERT to a new model, StructBERT, by\nincorporating language structures into pre-training. Specifically, we pre-train\nStructBERT with two auxiliary tasks to make the most of the sequential order of\nwords and sentences, which leverage language structures at the word and\nsentence levels, respectively. As a result, the new model is adapted to\ndifferent levels of language understanding required by downstream tasks. The\nStructBERT with structural pre-training gives surprisingly good empirical\nresults on a variety of downstream tasks, including pushing the\nstate-of-the-art on the GLUE benchmark to 89.0 (outperforming all published\nmodels), the F1 score on SQuAD v1.1 question answering to 93.0, the accuracy on\nSNLI to 91.7.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04626v2",
            "date": "2019-08-13 13:15:04+00:00",
            "title": "Attention is not not Explanation",
            "authors": [
                "Sarah Wiegreffe",
                "Yuval Pinter"
            ],
            "abstract": "Attention mechanisms play a central role in NLP systems, especially within\nrecurrent neural network (RNN) models. Recently, there has been increasing\ninterest in whether or not the intermediate representations offered by these\nmodules may be used to explain the reasoning for a model's prediction, and\nconsequently reach insights regarding the model's decision-making process. A\nrecent paper claims that `Attention is not Explanation' (Jain and Wallace,\n2019). We challenge many of the assumptions underlying this work, arguing that\nsuch a claim depends on one's definition of explanation, and that testing it\nneeds to take into account all elements of the model, using a rigorous\nexperimental design. We propose four alternative tests to determine\nwhen/whether attention can be used as explanation: a simple uniform-weights\nbaseline; a variance calibration based on multiple random seed runs; a\ndiagnostic framework using frozen weights from pretrained models; and an\nend-to-end adversarial attention training protocol. Each allows for meaningful\ninterpretation of attention mechanisms in RNN models. We show that even when\nreliable adversarial distributions can be found, they don't perform well on the\nsimple diagnostic, indicating that prior work does not disprove the usefulness\nof attention mechanisms for explainability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04660v1",
            "date": "2019-08-13 14:31:05+00:00",
            "title": "Playing log(N)-Questions over Sentences",
            "authors": [
                "Peter Potash",
                "Kaheer Suleman"
            ],
            "abstract": "We propose a two-agent game wherein a questioner must be able to conjure\ndiscerning questions between sentences, incorporate responses from an answerer,\nand keep track of a hypothesis state. The questioner must be able to understand\nthe information required to make its final guess, while also being able to\nreason over the game's text environment based on the answerer's responses. We\nexperiment with an end-to-end model where both agents can learn simultaneously\nto play the game, showing that simultaneously achieving high game accuracy and\nproducing meaningful questions can be a difficult trade-off.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04664v4",
            "date": "2019-08-13 14:32:56+00:00",
            "title": "Neural Machine Translation with Noisy Lexical Constraints",
            "authors": [
                "Huayang Li",
                "Guoping Huang",
                "Deng Cai",
                "Lemao Liu"
            ],
            "abstract": "Lexically constrained decoding for machine translation has shown to be\nbeneficial in previous studies. Unfortunately, constraints provided by users\nmay contain mistakes in real-world situations. It is still an open question\nthat how to manipulate these noisy constraints in such practical scenarios. We\npresent a novel framework that treats constraints as external memories. In this\nsoft manner, a mistaken constraint can be corrected. Experiments demonstrate\nthat our approach can achieve substantial BLEU gains in handling noisy\nconstraints. These results motivate us to apply the proposed approach on a new\nscenario where constraints are generated without the help of users. Experiments\nshow that our approach can indeed improve the translation quality with the\nautomatically generated constraints.",
            "journal ref": "TASLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04728v1",
            "date": "2019-08-13 16:39:48+00:00",
            "title": "Improving Generalization in Coreference Resolution via Adversarial Training",
            "authors": [
                "Sanjay Subramanian",
                "Dan Roth"
            ],
            "abstract": "In order for coreference resolution systems to be useful in practice, they\nmust be able to generalize to new text. In this work, we demonstrate that the\nperformance of the state-of-the-art system decreases when the names of PER and\nGPE named entities in the CoNLL dataset are changed to names that do not occur\nin the training set. We use the technique of adversarial gradient-based\ntraining to retrain the state-of-the-art system and demonstrate that the\nretrained system achieves higher performance on the CoNLL dataset (both with\nand without the change of named entities) and the GAP dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04877v1",
            "date": "2019-08-13 22:21:25+00:00",
            "title": "Meta Reasoning over Knowledge Graphs",
            "authors": [
                "Hong Wang",
                "Wenhan Xiong",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Shiyu Chang",
                "William Yang Wang"
            ],
            "abstract": "The ability to reason over learned knowledge is an innate ability for humans\nand humans can easily master new reasoning rules with only a few\ndemonstrations. While most existing studies on knowledge graph (KG) reasoning\nassume enough training examples, we study the challenging and practical problem\nof few-shot knowledge graph reasoning under the paradigm of meta-learning. We\npropose a new meta learning framework that effectively utilizes the\ntask-specific meta information such as local graph neighbors and reasoning\npaths in KGs. Specifically, we design a meta-encoder that encodes the meta\ninformation into task-specific initialization parameters for different tasks.\nThis allows our reasoning module to have diverse starting points when learning\nto reason over different relations, which is expected to better fit the target\ntask. On two few-shot knowledge base completion benchmarks, we show that the\naugmented task-specific meta-encoder yields much better initial point than MAML\nand outperforms several few-shot learning baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04899v2",
            "date": "2019-08-14 00:38:13+00:00",
            "title": "Aspect and Opinion Terms Extraction Using Double Embeddings and Attention Mechanism for Indonesian Hotel Reviews",
            "authors": [
                "Jordhy Fernando",
                "Masayu Leylia Khodra",
                "Ali Akbar Septiandri"
            ],
            "abstract": "Aspect and opinion terms extraction from review texts is one of the key tasks\nin aspect-based sentiment analysis. In order to extract aspect and opinion\nterms for Indonesian hotel reviews, we adapt double embeddings feature and\nattention mechanism that outperform the best system at SemEval 2015 and 2016.\nWe conduct experiments using 4000 reviews to find the best configuration and\nshow the influences of double embeddings and attention mechanism toward model\nperformance. Using 1000 reviews for evaluation, we achieved F1-measure of 0.914\nand 0.90 for aspect and opinion terms extraction in token and entity (term)\nlevel respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04942v4",
            "date": "2019-08-14 03:40:04+00:00",
            "title": "Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation",
            "authors": [
                "Yu Chen",
                "Lingfei Wu",
                "Mohammed J. Zaki"
            ],
            "abstract": "Natural question generation (QG) aims to generate questions from a passage\nand an answer. Previous works on QG either (i) ignore the rich structure\ninformation hidden in text, (ii) solely rely on cross-entropy loss that leads\nto issues like exposure bias and inconsistency between train/test measurement,\nor (iii) fail to fully exploit the answer information. To address these\nlimitations, in this paper, we propose a reinforcement learning (RL) based\ngraph-to-sequence (Graph2Seq) model for QG. Our model consists of a Graph2Seq\ngenerator with a novel Bidirectional Gated Graph Neural Network based encoder\nto embed the passage, and a hybrid evaluator with a mixed objective combining\nboth cross-entropy and RL losses to ensure the generation of syntactically and\nsemantically valid text. We also introduce an effective Deep Alignment Network\nfor incorporating the answer information into the passage at both the word and\ncontextual levels. Our model is end-to-end trainable and achieves new\nstate-of-the-art scores, outperforming existing methods by a significant margin\non the standard SQuAD benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04943v4",
            "date": "2019-08-14 03:45:15+00:00",
            "title": "Establishing Strong Baselines for the New Decade: Sequence Tagging, Syntactic and Semantic Parsing with BERT",
            "authors": [
                "Han He",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents new state-of-the-art models for three tasks,\npart-of-speech tagging, syntactic parsing, and semantic parsing, using the\ncutting-edge contextualized embedding framework known as BERT. For each task,\nwe first replicate and simplify the current state-of-the-art approach to\nenhance its model efficiency. We then evaluate our simplified approaches on\nthose three tasks using token embeddings generated by BERT. 12 datasets in both\nEnglish and Chinese are used for our experiments. The BERT models outperform\nthe previously best-performing models by 2.5% on average (7.5% for the most\nsignificant case). Moreover, an in-depth analysis on the impact of BERT\nembeddings is provided using self-attention, which helps understanding in this\nrich yet representation. All models and source codes are available in public so\nthat researchers can improve upon and utilize them to establish strong\nbaselines for the next decade.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05009v1",
            "date": "2019-08-14 08:06:14+00:00",
            "title": "FlexNER: A Flexible LSTM-CNN Stack Framework for Named Entity Recognition",
            "authors": [
                "Hongyin Zhu",
                "Wenpeng Hu",
                "Yi Zeng"
            ],
            "abstract": "Named entity recognition (NER) is a foundational technology for information\nextraction. This paper presents a flexible NER framework compatible with\ndifferent languages and domains. Inspired by the idea of distant supervision\n(DS), this paper enhances the representation by increasing the entity-context\ndiversity without relying on external resources. We choose different layer\nstacks and sub-network combinations to construct the bilateral networks. This\nstrategy can generally improve model performance on different datasets. We\nconduct experiments on five languages, such as English, German, Spanish, Dutch\nand Chinese, and biomedical fields, such as identifying the chemicals and\ngene/protein terms from scientific works. Experimental results demonstrate the\ngood performance of this framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05111v2",
            "date": "2019-08-14 13:15:24+00:00",
            "title": "X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension",
            "authors": [
                "Mostafa Abdou",
                "Cezar Sas",
                "Rahul Aralikatte",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Although the vast majority of knowledge bases KBs are heavily biased towards\nEnglish, Wikipedias do cover very different topics in different languages.\nExploiting this, we introduce a new multilingual dataset (X-WikiRE), framing\nrelation extraction as a multilingual machine reading problem. We show that by\nleveraging this resource it is possible to robustly transfer models\ncross-lingually and that multilingual support significantly improves\n(zero-shot) relation extraction, enabling the population of low-resourced KBs\nfrom their well-populated counterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05147v3",
            "date": "2019-08-14 14:28:07+00:00",
            "title": "SG-Net: Syntax-Guided Machine Reading Comprehension",
            "authors": [
                "Zhuosheng Zhang",
                "Yuwei Wu",
                "Junru Zhou",
                "Sufeng Duan",
                "Hai Zhao",
                "Rui Wang"
            ],
            "abstract": "For machine reading comprehension, the capacity of effectively modeling the\nlinguistic knowledge from the detail-riddled and lengthy passages and getting\nride of the noises is essential to improve its performance. Traditional\nattentive models attend to all words without explicit constraint, which results\nin inaccurate concentration on some dispensable words. In this work, we propose\nusing syntax to guide the text modeling by incorporating explicit syntactic\nconstraints into attention mechanism for better linguistically motivated word\nrepresentations. In detail, for self-attention network (SAN) sponsored\nTransformer-based encoder, we introduce syntactic dependency of interest (SDOI)\ndesign into the SAN to form an SDOI-SAN with syntax-guided self-attention.\nSyntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the\nSAN from the original Transformer encoder through a dual contextual\narchitecture for better linguistics inspired representation. To verify its\neffectiveness, the proposed SG-Net is applied to typical pre-trained language\nmodel BERT which is right based on a Transformer encoder. Extensive experiments\non popular benchmarks including SQuAD 2.0 and RACE show that the proposed\nSG-Net design helps achieve substantial performance improvement over strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05204v2",
            "date": "2019-08-14 16:24:56+00:00",
            "title": "On The Evaluation of Machine Translation Systems Trained With Back-Translation",
            "authors": [
                "Sergey Edunov",
                "Myle Ott",
                "Marc'Aurelio Ranzato",
                "Michael Auli"
            ],
            "abstract": "Back-translation is a widely used data augmentation technique which leverages\ntarget monolingual data. However, its effectiveness has been challenged since\nautomatic metrics such as BLEU only show significant improvements for test\nexamples where the source itself is a translation, or translationese. This is\nbelieved to be due to translationese inputs better matching the back-translated\ntraining data. In this work, we show that this conjecture is not empirically\nsupported and that back-translation improves translation quality of both\nnaturally occurring text as well as translationese according to professional\nhuman translators. We provide empirical evidence to support the view that\nback-translation is preferred by humans because it produces more fluent\noutputs. BLEU cannot capture human preferences because references are\ntranslationese when source sentences are natural text. We recommend\ncomplementing BLEU with a language model score to measure fluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05253v3",
            "date": "2019-08-14 17:32:40+00:00",
            "title": "The lexical and grammatical sources of neg-raising inferences",
            "authors": [
                "Hannah Youngeun An",
                "Aaron Steven White"
            ],
            "abstract": "We investigate neg(ation)-raising inferences, wherein negation on a predicate\ncan be interpreted as though in that predicate's subordinate clause. To do\nthis, we collect a large-scale dataset of neg-raising judgments for effectively\nall English clause-embedding verbs and develop a model to jointly induce the\nsemantic types of verbs and their subordinate clauses and the relationship of\nthese types to neg-raising inferences. We find that some neg-raising inferences\nare attributable to properties of particular predicates, while others are\nattributable to subordinate clause structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05267v2",
            "date": "2019-08-14 17:47:02+00:00",
            "title": "Towards Debiasing Fact Verification Models",
            "authors": [
                "Tal Schuster",
                "Darsh J Shah",
                "Yun Jie Serene Yeo",
                "Daniel Filizzola",
                "Enrico Santus",
                "Regina Barzilay"
            ],
            "abstract": "Fact verification requires validating a claim in the context of evidence. We\nshow, however, that in the popular FEVER dataset this might not necessarily be\nthe case. Claim-only classifiers perform competitively with top evidence-aware\nmodels. In this paper, we investigate the cause of this phenomenon, identifying\nstrong cues for predicting labels solely based on the claim, without\nconsidering any evidence. We create an evaluation set that avoids those\nidiosyncrasies. The performance of FEVER-trained models significantly drops\nwhen evaluated on this test set. Therefore, we introduce a regularization\nmethod which alleviates the effect of bias in the training data, obtaining\nimprovements on the newly created test set. This work is a step towards a more\nsound evaluation of reasoning capabilities in fact verification models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05378v2",
            "date": "2019-08-15 00:22:38+00:00",
            "title": "Multi-Task Self-Supervised Learning for Disfluency Detection",
            "authors": [
                "Shaolei Wang",
                "Wanxiang Che",
                "Qi Liu",
                "Pengda Qin",
                "Ting Liu",
                "William Yang Wang"
            ],
            "abstract": "Most existing approaches to disfluency detection heavily rely on\nhuman-annotated data, which is expensive to obtain in practice. To tackle the\ntraining data bottleneck, we investigate methods for combining multiple\nself-supervised tasks-i.e., supervised tasks where data can be collected\nwithout manual labeling. First, we construct large-scale pseudo training data\nby randomly adding or deleting words from unlabeled news data, and propose two\nself-supervised pre-training tasks: (i) tagging task to detect the added noisy\nwords. (ii) sentence classification to distinguish original sentences from\ngrammatically-incorrect sentences. We then combine these two tasks to jointly\ntrain a network. The pre-trained network is then fine-tuned using\nhuman-annotated disfluency detection training data. Experimental results on the\ncommonly used English Switchboard test set show that our approach can achieve\ncompetitive performance compared to the previous systems (trained using the\nfull dataset) by using less than 1% (1000 sentences) of the training data. Our\nmethod trained on the full dataset significantly outperforms previous methods,\nreducing the error by 21% on English Switchboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05408v1",
            "date": "2019-08-15 03:50:53+00:00",
            "title": "Towards End-to-End Learning for Efficient Dialogue Agent by Modeling Looking-ahead Ability",
            "authors": [
                "Zhuoxuan Jiang",
                "Xian-Ling Mao",
                "Ziming Huang",
                "Jie Ma",
                "Shaochun Li"
            ],
            "abstract": "Learning an efficient manager of dialogue agent from data with little manual\nintervention is important, especially for goal-oriented dialogues. However,\nexisting methods either take too many manual efforts (e.g. reinforcement\nlearning methods) or cannot guarantee the dialogue efficiency (e.g.\nsequence-to-sequence methods). In this paper, we address this problem by\nproposing a novel end-to-end learning model to train a dialogue agent that can\nlook ahead for several future turns and generate an optimal response to make\nthe dialogue efficient. Our method is data-driven and does not require too much\nmanual work for intervention during system design. We evaluate our method on\ntwo datasets of different scenarios and the experimental results demonstrate\nthe efficiency of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05416v1",
            "date": "2019-08-15 04:40:27+00:00",
            "title": "XCMRC: Evaluating Cross-lingual Machine Reading Comprehension",
            "authors": [
                "Pengyuan Liu",
                "Yuning Deng",
                "Chenghao Zhu",
                "Han Hu"
            ],
            "abstract": "We present XCMRC, the first public cross-lingual language understanding (XLU)\nbenchmark which aims to test machines on their cross-lingual reading\ncomprehension ability. To be specific, XCMRC is a Cross-lingual Cloze-style\nMachine Reading Comprehension task which requires the reader to fill in a\nmissing word (we additionally provide ten noun candidates) in a sentence\nwritten in target language (English / Chinese) by reading a given passage\nwritten in source language (Chinese / English). Chinese and English are\nrich-resource language pairs, in order to study low-resource cross-lingual\nmachine reading comprehension (XMRC), besides defining the common XCMRC task\nwhich has no restrictions on use of external language resources, we also define\nthe pseudo low-resource XCMRC task by limiting the language resources to be\nused. In addition, we provide two baselines for common XCMRC task and two for\npseudo XCMRC task respectively. We also provide an upper bound baseline for\nboth tasks. We found that for common XCMRC task, translation-based method and\nmultilingual sentence encoder-based method can obtain reasonable performance\nbut still have much room for improvement. As for pseudo low-resource XCMRC\ntask, due to strict restrictions on the use of language resources, our two\napproaches are far below the upper bound so there are many challenges ahead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05453v1",
            "date": "2019-08-15 08:09:52+00:00",
            "title": "What's Wrong with Hebrew NLP? And How to Make it Right",
            "authors": [
                "Reut Tsarfaty",
                "Amit Seker",
                "Shoval Sadde",
                "Stav Klein"
            ],
            "abstract": "For languages with simple morphology, such as English, automatic annotation\npipelines such as spaCy or Stanford's CoreNLP successfully serve projects in\nacademia and the industry. For many morphologically-rich languages (MRLs),\nsimilar pipelines show sub-optimal performance that limits their applicability\nfor text analysis in research and the industry.The sub-optimal performance is\nmainly due to errors in early morphological disambiguation decisions, which\ncannot be recovered later in the pipeline, yielding incoherent annotations on\nthe whole. In this paper we describe the design and use of the Onlp suite, a\njoint morpho-syntactic parsing framework for processing Modern Hebrew texts.\nThe joint inference over morphology and syntax substantially limits error\npropagation, and leads to high accuracy. Onlp provides rich and expressive\noutput which already serves diverse academic and commercial needs. Its\naccompanying online demo further serves educational activities, introducing\nHebrew NLP intricacies to researchers and non-researchers alike.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05514v2",
            "date": "2019-08-15 12:32:52+00:00",
            "title": "A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning",
            "authors": [
                "Minghao Hu",
                "Yuxing Peng",
                "Zhen Huang",
                "Dongsheng Li"
            ],
            "abstract": "Rapid progress has been made in the field of reading comprehension and\nquestion answering, where several systems have achieved human parity in some\nsimplified settings. However, the performance of these models degrades\nsignificantly when they are applied to more realistic scenarios, such as\nanswers involve various types, multiple text strings are correct answers, or\ndiscrete reasoning abilities are required. In this paper, we introduce the\nMulti-Type Multi-Span Network (MTMSN), a neural reading comprehension model\nthat combines a multi-type answer predictor designed to support various answer\ntypes (e.g., span, count, negation, and arithmetic expression) with a\nmulti-span extraction method for dynamically producing one or multiple text\nspans. In addition, an arithmetic expression reranking mechanism is proposed to\nrank expression candidates for further confirming the prediction. Experiments\nshow that our model achieves 79.9 F1 on the DROP hidden test set, creating new\nstate-of-the-art results. Source\ncode\\footnote{\\url{https://github.com/huminghao16/MTMSN}} is released to\nfacilitate future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05691v1",
            "date": "2019-08-15 18:04:39+00:00",
            "title": "Improving Multi-Word Entity Recognition for Biomedical Texts",
            "authors": [
                "Hamada A. Nayel",
                "H. L. Shashirekha",
                "Hiroyuki Shindo",
                "Yuji Matsumoto"
            ],
            "abstract": "Biomedical Named Entity Recognition (BioNER) is a crucial step for analyzing\nBiomedical texts, which aims at extracting biomedical named entities from a\ngiven text. Different supervised machine learning algorithms have been applied\nfor BioNER by various researchers. The main requirement of these approaches is\nan annotated dataset used for learning the parameters of machine learning\nalgorithms. Segment Representation (SR) models comprise of different tag sets\nused for representing the annotated data, such as IOB2, IOE2 and IOBES. In this\npaper, we propose an extension of IOBES model to improve the performance of\nBioNER. The proposed SR model, FROBES, improves the representation of\nmulti-word entities. We used Bidirectional Long Short-Term Memory (BiLSTM)\nnetwork; an instance of Recurrent Neural Networks (RNN), to design a baseline\nsystem for BioNER and evaluated the new SR model on two datasets, i2b2/VA 2010\nchallenge dataset and JNLPBA 2004 shared task dataset. The proposed SR model\noutperforms other models for multi-word entities with length greater than two.\nFurther, the outputs of different SR models have been combined using majority\nvoting ensemble method which outperforms the baseline models performance.",
            "journal ref": "International Journal of Pure and Applied Mathematics, Volume 118\n  No. 16, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05731v1",
            "date": "2019-08-15 19:54:23+00:00",
            "title": "Simple and Effective Noisy Channel Modeling for Neural Machine Translation",
            "authors": [
                "Kyra Yee",
                "Nathan Ng",
                "Yann N. Dauphin",
                "Michael Auli"
            ],
            "abstract": "Previous work on neural noisy channel modeling relied on latent variable\nmodels that incrementally process the source and target sentence. This makes\ndecoding decisions based on partial source prefixes even though the full source\nis available. We pursue an alternative approach based on standard sequence to\nsequence models which utilize the entire source. These models perform\nremarkably well as channel models, even though they have neither been trained\non, nor designed to factor over incomplete target sentences. Experiments with\nneural language models trained on billions of words show that noisy channel\nmodels can outperform a direct model by up to 3.2 BLEU on WMT'17 German-English\ntranslation. We evaluate on four language-pairs and our channel models\nconsistently outperform strong alternatives such right-to-left reranking models\nand ensembles of direct models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05739v2",
            "date": "2019-08-15 20:03:10+00:00",
            "title": "Abductive Commonsense Reasoning",
            "authors": [
                "Chandra Bhagavatula",
                "Ronan Le Bras",
                "Chaitanya Malaviya",
                "Keisuke Sakaguchi",
                "Ari Holtzman",
                "Hannah Rashkin",
                "Doug Downey",
                "Scott Wen-tau Yih",
                "Yejin Choi"
            ],
            "abstract": "Abductive reasoning is inference to the most plausible explanation. For\nexample, if Jenny finds her house in a mess when she returns from work, and\nremembers that she left a window open, she can hypothesize that a thief broke\ninto her house and caused the mess, as the most plausible explanation. While\nabduction has long been considered to be at the core of how people interpret\nand read between the lines in natural language (Hobbs et al., 1988), there has\nbeen relatively little research in support of abductive natural language\ninference and generation. We present the first study that investigates the\nviability of language-based abductive reasoning. We introduce a challenge\ndataset, ART, that consists of over 20k commonsense narrative contexts and 200k\nexplanations. Based on this dataset, we conceptualize two new tasks -- (i)\nAbductive NLI: a multiple-choice question answering task for choosing the more\nlikely explanation, and (ii) Abductive NLG: a conditional generation task for\nexplaining given observations in natural language. On Abductive NLI, the best\nmodel achieves 68.9% accuracy, well below human performance of 91.4%. On\nAbductive NLG, the current best language generators struggle even more, as they\nlack reasoning capabilities that are trivial for humans. Our analysis leads to\nnew insights into the types of reasoning that deep pre-trained language models\nfail to perform--despite their strong performance on the related but more\nnarrowly defined task of entailment NLI--pointing to interesting avenues for\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05757v1",
            "date": "2019-08-14 17:37:31+00:00",
            "title": "Debiasing Personal Identities in Toxicity Classification",
            "authors": [
                "Apik Ashod Zorian",
                "Chandra Shekar Bikkanur"
            ],
            "abstract": "As Machine Learning models continue to be relied upon for making automated\ndecisions, the issue of model bias becomes more and more prevalent. In this\npaper, we approach training a text classifica-tion model and optimize on bias\nminimization by measuring not only the models performance on our dataset as a\nwhole, but also how it performs across different subgroups. This requires\nmeasuring per-formance independently for different demographic subgroups and\nmeasuring bias by comparing them to results from the rest of our data. We show\nhow unintended bias can be detected using these metrics and how removing bias\nfrom a dataset completely can result in worse results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05760v1",
            "date": "2019-08-13 13:55:48+00:00",
            "title": "BioFLAIR: Pretrained Pooled Contextualized Embeddings for Biomedical Sequence Labeling Tasks",
            "authors": [
                "Shreyas Sharma",
                "Ron Daniel Jr"
            ],
            "abstract": "Biomedical Named Entity Recognition (NER) is a challenging problem in\nbiomedical information processing due to the widespread ambiguity of out of\ncontext terms and extensive lexical variations. Performance on bioNER\nbenchmarks continues to improve due to advances like BERT, GPT, and XLNet.\nFLAIR (1) is an alternative embedding model which is less computationally\nintensive than the others mentioned. We test FLAIR and its pretrained PubMed\nembeddings (which we term BioFLAIR) on a variety of bio NER tasks and compare\nthose with results from BERT-type networks. We also investigate the effects of\na small amount of additional pretraining on PubMed content, and of combining\nFLAIR and ELMO models. We find that with the provided embeddings, FLAIR\nperforms on-par with the BERT networks - even establishing a new state of the\nart on one benchmark. Additional pretraining did not provide a clear benefit,\nalthough this might change with even more pretraining being done. Stacking the\nFLAIR embeddings with others typically does provide a boost in the benchmark\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05803v2",
            "date": "2019-08-16 00:59:44+00:00",
            "title": "Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning",
            "authors": [
                "Pradeep Dasigi",
                "Nelson F. Liu",
                "Ana Marasovi\u0107",
                "Noah A. Smith",
                "Matt Gardner"
            ],
            "abstract": "Machine comprehension of texts longer than a single sentence often requires\ncoreference resolution. However, most current reading comprehension benchmarks\ndo not contain complex coreferential phenomena and hence fail to evaluate the\nability of models to resolve coreference. We present a new crowdsourced dataset\ncontaining more than 24K span-selection questions that require resolving\ncoreference among entities in over 4.7K English paragraphs from Wikipedia.\nObtaining questions focused on such phenomena is challenging, because it is\nhard to avoid lexical cues that shortcut complex reasoning. We deal with this\nissue by using a strong baseline model as an adversary in the crowdsourcing\nloop, which helps crowdworkers avoid writing questions with exploitable surface\ncues. We show that state-of-the-art reading comprehension models perform\nsignificantly worse than humans on this benchmark---the best model performance\nis 70.5 F1, while the estimated human performance is 93.4 F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05828v1",
            "date": "2019-08-16 03:14:43+00:00",
            "title": "Named Entity Recognition for Nepali Language",
            "authors": [
                "Oyesh Mann Singh",
                "Ankur Padia",
                "Anupam Joshi"
            ],
            "abstract": "Named Entity Recognition have been studied for different languages like\nEnglish, German, Spanish and many others but no study have focused on Nepali\nlanguage. In this paper we propose a neural based Nepali NER using latest\nstate-of-the-art architecture based on grapheme-level which doesn't require any\nhand-crafted features and no data pre-processing. Our novel neural based model\ngained relative improvement of 33% to 50% compared to feature based SVM model\nand up to 10% improvement over state-of-the-art neural based model developed\nfor languages beside Nepali.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05838v2",
            "date": "2019-08-16 04:15:32+00:00",
            "title": "Pushing the Limits of Low-Resource Morphological Inflection",
            "authors": [
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "Recent years have seen exceptional strides in the task of automatic\nmorphological inflection generation. However, for a long tail of languages the\nnecessary resources are hard to come by, and state-of-the-art neural methods\nthat work well under higher resource settings perform poorly in the face of a\npaucity of data. In response, we propose a battery of improvements that greatly\nimprove performance under such low-resource conditions. First, we present a\nnovel two-step attention architecture for the inflection decoder. In addition,\nwe investigate the effects of cross-lingual transfer from single and multiple\nlanguages, as well as monolingual data hallucination. The macro-averaged\naccuracy of our models outperforms the state-of-the-art by 15 percentage\npoints. Also, we identify the crucial factors for success with cross-lingual\ntransfer for morphological inflection: typological similarity and a common\nrepresentation across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05848v2",
            "date": "2019-08-16 05:09:57+00:00",
            "title": "Sketch-Driven Regular Expression Generation from Natural Language and Examples",
            "authors": [
                "Xi Ye",
                "Qiaochu Chen",
                "Xinyu Wang",
                "Isil Dillig",
                "Greg Durrett"
            ],
            "abstract": "Recent systems for converting natural language descriptions into regular\nexpressions (regexes) have achieved some success, but typically deal with\nshort, formulaic text and can only produce simple regexes. Realworld regexes\nare complex, hard to describe with brief sentences, and sometimes require\nexamples to fully convey the user's intent. We present a framework for regex\nsynthesis in this setting where both natural language (NL) and examples are\navailable. First, a semantic parser (either grammar-based or neural) maps the\nnatural language description into an intermediate sketch, which is an\nincomplete regex containing holes to denote missing components. Then a program\nsynthesizer searches over the regex space defined by the sketch and finds a\nregex that is consistent with the given string examples. Our semantic parser\ncan be trained purely from weak supervision based on correctness of the\nsynthesized regex, or it can leverage heuristically-derived sketches. We\nevaluate on two prior datasets (Kushman and Barzilay, 2013; Locascio et al.,\n2016) and a real-world dataset from Stack Overflow. Our system achieves\nstate-of-the-art performance on the prior datasets and solves 57% of the\nreal-world dataset, which existing neural systems completely fail on.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05852v2",
            "date": "2019-08-16 05:36:04+00:00",
            "title": "Reasoning Over Paragraph Effects in Situations",
            "authors": [
                "Kevin Lin",
                "Oyvind Tafjord",
                "Peter Clark",
                "Matt Gardner"
            ],
            "abstract": "A key component of successfully reading a passage of text is the ability to\napply knowledge gained from the passage to a new situation. In order to\nfacilitate progress on this kind of reading, we present ROPES, a challenging\nbenchmark for reading comprehension targeting Reasoning Over Paragraph Effects\nin Situations. We target expository language describing causes and effects\n(e.g., \"animal pollinators increase efficiency of fertilization in flowers\"),\nas they have clear implications for new situations. A system is presented a\nbackground passage containing at least one of these relations, a novel\nsituation that uses this background, and questions that require reasoning about\neffects of the relationships in the background passage in the context of the\nsituation. We collect background passages from science textbooks and Wikipedia\nthat contain such phenomena, and ask crowd workers to author situations,\nquestions, and answers, resulting in a 14,322 question dataset. We analyze the\nchallenges of this task and evaluate the performance of state-of-the-art\nreading comprehension models. The best model performs only slightly better than\nrandomly guessing an answer of the correct type, at 61.6% F1, well below the\nhuman performance of 89.0%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05854v1",
            "date": "2019-08-16 05:48:41+00:00",
            "title": "Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach",
            "authors": [
                "Igor Shalyminov",
                "Sungjin Lee",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "Learning with minimal data is one of the key challenges in the development of\npractical, production-ready goal-oriented dialogue systems. In a real-world\nenterprise setting where dialogue systems are developed rapidly and are\nexpected to work robustly for an ever-growing variety of domains, products, and\nscenarios, efficient learning from a limited number of examples becomes\nindispensable.\n  In this paper, we introduce a technique to achieve state-of-the-art dialogue\ngeneration performance in a few-shot setup, without using any annotated data.\nWe do this by leveraging background knowledge from a larger, more highly\nrepresented dialogue source --- namely, the MetaLWOz dataset. We evaluate our\nmodel on the Stanford Multi-Domain Dialogue Dataset, consisting of human-human\ngoal-oriented dialogues in in-car navigation, appointment scheduling, and\nweather information domains.\n  We show that our few-shot approach achieves state-of-the art results on that\ndataset by consistently outperforming the previous best model in terms of BLEU\nand Entity F1 scores, while being more data-efficient by not requiring any data\nannotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05908v2",
            "date": "2019-08-16 09:29:43+00:00",
            "title": "BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction",
            "authors": [
                "Weipeng Huang",
                "Xingyi Cheng",
                "Taifeng Wang",
                "Wei Chu"
            ],
            "abstract": "In this paper, we report our method for the Information Extraction task in\n2019 Language and Intelligence Challenge. We incorporate BERT into the\nmulti-head selection framework for joint entity-relation extraction. This model\nextends existing approaches from three perspectives. First, BERT is adopted as\na feature extraction layer at the bottom of the multi-head selection framework.\nWe further optimize BERT by introducing a semantic-enhanced task during BERT\npre-training. Second, we introduce a large-scale Baidu Baike corpus for entity\nrecognition pre-training, which is of weekly supervised learning since there is\nno actual named entity label. Third, soft label embedding is proposed to\neffectively transmit information between entity recognition and relation\nextraction. Combining these three contributions, we enhance the information\nextracting ability of the multi-head selection model and achieve F1-score 0.876\non testset-1 with a single model. By ensembling four variants of our model, we\nfinally achieve F1 score 0.892 (1st place) on testset-1 and F1 score 0.8924\n(2nd place) on testset-2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05925v2",
            "date": "2019-08-16 10:44:25+00:00",
            "title": "Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring",
            "authors": [
                "Zihan Liu",
                "Yan Xu",
                "Genta Indra Winata",
                "Pascale Fung"
            ],
            "abstract": "This paper describes CAiRE's submission to the unsupervised machine\ntranslation track of the WMT'19 news shared task from German to Czech. We\nleverage a phrase-based statistical machine translation (PBSMT) model and a\npre-trained language model to combine word-level neural machine translation\n(NMT) and subword-level NMT models without using any parallel data. We propose\nto solve the morphological richness problem of languages by training byte-pair\nencoding (BPE) embeddings for German and Czech separately, and they are aligned\nusing MUSE (Conneau et al., 2018). To ensure the fluency and consistency of\ntranslations, a rescoring mechanism is proposed that reuses the pre-trained\nlanguage model to select the translation candidates generated through beam\nsearch. Moreover, a series of pre-processing and post-processing approaches are\napplied to improve the quality of final translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05947v1",
            "date": "2019-08-16 12:38:05+00:00",
            "title": "How Sequence-to-Sequence Models Perceive Language Styles?",
            "authors": [
                "Ruozi Huang",
                "Mi Zhang",
                "Xudong Pan",
                "Beina Sheng"
            ],
            "abstract": "Style is ubiquitous in our daily language uses, while what is language style\nto learning machines? In this paper, by exploiting the second-order statistics\nof semantic vectors of different corpora, we present a novel perspective on\nthis question via style matrix, i.e. the covariance matrix of semantic vectors,\nand explain for the first time how Sequence-to-Sequence models encode style\ninformation innately in its semantic vectors. As an application, we devise a\nlearning-free text style transfer algorithm, which explicitly constructs a pair\nof transfer operators from the style matrices for style transfer. Moreover, our\nalgorithm is also observed to be flexible enough to transfer out-of-domain\nsentences. Extensive experimental evidence justifies the informativeness of\nstyle matrix and the competitive performance of our proposed style transfer\nalgorithm with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05957v2",
            "date": "2019-08-16 12:58:16+00:00",
            "title": "Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning",
            "authors": [
                "Zhijiang Guo",
                "Yan Zhang",
                "Zhiyang Teng",
                "Wei Lu"
            ],
            "abstract": "We focus on graph-to-sequence learning, which can be framed as transducing\ngraph structures to sequences for text generation. To capture structural\ninformation associated with graphs, we investigate the problem of encoding\ngraphs using graph convolutional networks (GCNs). Unlike various existing\napproaches where shallow architectures were used for capturing local structural\ninformation only, we introduce a dense connection strategy, proposing a novel\nDensely Connected Graph Convolutional Networks (DCGCNs). Such a deep\narchitecture is able to integrate both local and non-local features to learn a\nbetter structural representation of a graph. Our model outperforms the\nstate-of-the-art neural models significantly on AMRto-text generation and\nsyntax-based neural machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05969v2",
            "date": "2019-08-16 13:35:24+00:00",
            "title": "Simplify the Usage of Lexicon in Chinese NER",
            "authors": [
                "Ruotian Ma",
                "Minlong Peng",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, many works have tried to augment the performance of Chinese named\nentity recognition (NER) using word lexicons. As a representative, Lattice-LSTM\n(Zhang and Yang, 2018) has achieved new benchmark results on several public\nChinese NER datasets. However, Lattice-LSTM has a complex model architecture.\nThis limits its application in many industrial areas where real-time NER\nresponses are needed.\n  In this work, we propose a simple but effective method for incorporating the\nword lexicon into the character representations. This method avoids designing a\ncomplicated sequence modeling architecture, and for any neural NER model, it\nrequires only subtle adjustment of the character representation layer to\nintroduce the lexicon information. Experimental studies on four benchmark\nChinese NER datasets show that our method achieves an inference speed up to\n6.15 times faster than those of state-ofthe-art methods, along with a better\nperformance. The experimental results also show that the proposed method can be\neasily incorporated with pre-trained models like BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06024v2",
            "date": "2019-08-13 10:59:06+00:00",
            "title": "Tackling Online Abuse: A Survey of Automated Abuse Detection Methods",
            "authors": [
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "Abuse on the Internet represents an important societal problem of our time.\nMillions of Internet users face harassment, racism, personal attacks, and other\ntypes of abuse on online platforms. The psychological effects of such abuse on\nindividuals can be profound and lasting. Consequently, over the past few years,\nthere has been a substantial research effort towards automated abuse detection\nin the field of natural language processing (NLP). In this paper, we present a\ncomprehensive survey of the methods that have been proposed to date, thus\nproviding a platform for further development of this area. We describe the\nexisting datasets and review the computational approaches to abuse detection,\nanalyzing their strengths and limitations. We discuss the main trends that\nemerge, highlight the challenges that remain, outline possible solutions, and\npropose guidelines for ethics and explainability",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06083v1",
            "date": "2019-08-17 18:34:11+00:00",
            "title": "Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack",
            "authors": [
                "Emily Dinan",
                "Samuel Humeau",
                "Bharath Chintagunta",
                "Jason Weston"
            ],
            "abstract": "The detection of offensive language in the context of a dialogue has become\nan increasingly important application of natural language processing. The\ndetection of trolls in public forums (Gal\\'an-Garc\\'ia et al., 2016), and the\ndeployment of chatbots in the public domain (Wolf et al., 2017) are two\nexamples that show the necessity of guarding against adversarially offensive\nbehavior on the part of humans. In this work, we develop a training scheme for\na model to become robust to such human attacks by an iterative build it, break\nit, fix it strategy with humans and models in the loop. In detailed experiments\nwe show this approach is considerably more robust than previous systems.\nFurther, we show that offensive language used within a conversation critically\ndepends on the dialogue context, and cannot be viewed as a single sentence\noffensive detection task as in most previous work. Our newly collected tasks\nand methods will be made open source and publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06136v2",
            "date": "2019-08-16 19:31:13+00:00",
            "title": "Transductive Auxiliary Task Self-Training for Neural Multi-Task Models",
            "authors": [
                "Johannes Bjerva",
                "Katharina Kann",
                "Isabelle Augenstein"
            ],
            "abstract": "Multi-task learning and self-training are two common ways to improve a\nmachine learning model's performance in settings with limited training data.\nDrawing heavily on ideas from those two approaches, we suggest transductive\nauxiliary task self-training: training a multi-task model on (i) a combination\nof main and auxiliary task training data, and (ii) test instances with\nauxiliary task labels which a single-task version of the model has previously\ngenerated. We perform extensive experiments on 86 combinations of languages and\ntasks. Our results are that, on average, transductive auxiliary task\nself-training improves absolute accuracy by up to 9.56% over the pure\nmulti-task model for dependency relation tagging and by up to 13.03% for\nsemantic tagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06138v1",
            "date": "2019-08-16 19:36:57+00:00",
            "title": "UDS--DFKI Submission to the WMT2019 Similar Language Translation Shared Task",
            "authors": [
                "Santanu Pal",
                "Marcos Zampieri",
                "Josef van Genabith"
            ],
            "abstract": "In this paper we present the UDS-DFKI system submitted to the Similar\nLanguage Translation shared task at WMT 2019. The first edition of this shared\ntask featured data from three pairs of similar languages: Czech and Polish,\nHindi and Nepali, and Portuguese and Spanish. Participants could choose to\nparticipate in any of these three tracks and submit system outputs in any\ntranslation direction. We report the results obtained by our system in\ntranslating from Czech to Polish and comment on the impact of out-of-domain\ntest data in the performance of our system. UDS-DFKI achieved competitive\nperformance ranking second among ten teams in Czech to Polish translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06140v1",
            "date": "2019-08-16 19:38:39+00:00",
            "title": "Improving CAT Tools in the Translation Workflow: New Approaches and Evaluation",
            "authors": [
                "Mihaela Vela",
                "Santanu Pal",
                "Marcos Zampieri",
                "Sudip Kumar Naskar",
                "Josef van Genabith"
            ],
            "abstract": "This paper describes strategies to improve an existing web-based\ncomputer-aided translation (CAT) tool entitled CATaLog Online. CATaLog Online\nprovides a post-editing environment with simple yet helpful project management\ntools. It offers translation suggestions from translation memories (TM),\nmachine translation (MT), and automatic post-editing (APE) and records detailed\nlogs of post-editing activities. To test the new approaches proposed in this\npaper, we carried out a user study on an English--German translation task using\nCATaLog Online. User feedback revealed that the users preferred using CATaLog\nOnline over existing CAT tools in some respects, especially by selecting the\noutput of the MT system and taking advantage of the color scheme for TM\nsuggestions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06151v2",
            "date": "2019-08-16 20:09:11+00:00",
            "title": "The Transference Architecture for Automatic Post-Editing",
            "authors": [
                "Santanu Pal",
                "Hongfei Xu",
                "Nico Herbig",
                "Sudip Kumar Naskar",
                "Antonio Krueger",
                "Josef van Genabith"
            ],
            "abstract": "In automatic post-editing (APE) it makes sense to condition post-editing (pe)\ndecisions on both the source (src) and the machine translated text (mt) as\ninput. This has led to multi-source encoder based APE approaches. A research\nchallenge now is the search for architectures that best support the capture,\npreparation and provision of src and mt information and its integration with pe\ndecisions. In this paper we present a new multi-source APE model, called\ntransference. Unlike previous approaches, it (i) uses a transformer encoder\nblock for src, (ii) followed by a decoder block, but without masking for\nself-attention on mt, which effectively acts as second encoder combining src ->\nmt, and (iii) feeds this representation into a final decoder block generating\npe. Our model outperforms the state-of-the-art by 1 BLEU point on the WMT 2016,\n2017, and 2018 English--German APE shared tasks (PBSMT and NMT). We further\ninvestigate the importance of our newly introduced second encoder and find that\na too small amount of layers does hurt the performance, while reducing the\nnumber of layers of the decoder does not matter much.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06203v3",
            "date": "2019-08-16 23:27:25+00:00",
            "title": "Learning Conceptual-Contextual Embeddings for Medical Text",
            "authors": [
                "Xiao Zhang",
                "Dejing Dou",
                "Ji Wu"
            ],
            "abstract": "External knowledge is often useful for natural language understanding tasks.\nWe introduce a contextual text representation model called\nConceptual-Contextual (CC) embeddings, which incorporates structured knowledge\ninto text representations. Unlike entity embedding methods, our approach\nencodes a knowledge graph into a context model. CC embeddings can be easily\nreused for a wide range of tasks just like pre-trained language models. Our\nmodel effectively encodes the huge UMLS database by leveraging semantic\ngeneralizability. Experiments on electronic health records (EHRs) and medical\ntext processing benchmarks showed our model gives a major boost to the\nperformance of supervised medical NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06216v1",
            "date": "2019-08-17 01:11:04+00:00",
            "title": "Generating an Overview Report over Many Documents",
            "authors": [
                "Jingwen Wang",
                "Hao Zhang",
                "Cheng Zhang",
                "Wenjing Yang",
                "Liqun Shao",
                "Jie Wang"
            ],
            "abstract": "How to efficiently generate an accurate, well-structured overview report\n(ORPT) over thousands of related documents is challenging. A well-structured\nORPT consists of sections of multiple levels (e.g., sections and subsections).\nNone of the existing multi-document summarization (MDS) algorithms is directed\ntoward this task. To overcome this obstacle, we present NDORGS (Numerous\nDocuments' Overview Report Generation Scheme) that integrates text filtering,\nkeyword scoring, single-document summarization (SDS), topic modeling, MDS, and\ntitle generation to generate a coherent, well-structured ORPT. We then devise a\nmulti-criteria evaluation method using techniques of text mining and\nmulti-attribute decision making on a combination of human judgments, running\ntime, information coverage, and topic diversity. We evaluate ORPTs generated by\nNDORGS on two large corpora of documents, where one is classified and the other\nunclassified. We show that, using Saaty's pairwise comparison 9-point scale and\nunder TOPSIS, the ORPTs generated on SDS's with the length of 20% of the\noriginal documents are the best overall on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06258v1",
            "date": "2019-08-17 08:01:05+00:00",
            "title": "Language Graph Distillation for Low-Resource Machine Translation",
            "authors": [
                "Tianyu He",
                "Jiale Chen",
                "Xu Tan",
                "Tao Qin"
            ],
            "abstract": "Neural machine translation on low-resource language is challenging due to the\nlack of bilingual sentence pairs. Previous works usually solve the low-resource\ntranslation problem with knowledge transfer in a multilingual setting. In this\npaper, we propose the concept of Language Graph and further design a novel\ngraph distillation algorithm that boosts the accuracy of low-resource\ntranslations in the graph with forward and backward knowledge distillation.\nPreliminary experiments on the TED talks multilingual dataset demonstrate the\neffectiveness of our proposed method. Specifically, we improve the low-resource\ntranslation pair by more than 3.13 points in terms of BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06259v1",
            "date": "2019-08-17 08:09:33+00:00",
            "title": "Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation",
            "authors": [
                "Tianyu He",
                "Xu Tan",
                "Tao Qin"
            ],
            "abstract": "Neural machine translation (NMT) typically adopts the encoder-decoder\nframework. A good understanding of the characteristics and functionalities of\nthe encoder and decoder can help to explain the pros and cons of the framework,\nand design better models for NMT. In this work, we conduct an empirical study\non the encoder and the decoder in NMT, taking Transformer as an example. We\nfind that 1) the decoder handles an easier task than the encoder in NMT, 2) the\ndecoder is more sensitive to the input noise than the encoder, and 3) the\npreceding words/tokens in the decoder provide strong conditional information,\nwhich accounts for the two observations above. We hope those observations can\nshed light on the characteristics of the encoder and decoder and inspire future\nresearch on NMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06264v1",
            "date": "2019-08-17 08:59:51+00:00",
            "title": "EmotionX-IDEA: Emotion BERT -- an Affectional Model for Conversation",
            "authors": [
                "Yen-Hao Huang",
                "Ssu-Rui Lee",
                "Mau-Yun Ma",
                "Yi-Hsin Chen",
                "Ya-Wen Yu",
                "Yi-Shin Chen"
            ],
            "abstract": "In this paper, we investigate the emotion recognition ability of the\npre-training language model, namely BERT. By the nature of the framework of\nBERT, a two-sentence structure, we adapt BERT to continues dialogue emotion\nprediction tasks, which rely heavily on the sentence-level context-aware\nunderstanding. The experiments show that by mapping the continues dialogue into\na causal utterance pair, which is constructed by the utterance and the reply\nutterance, models can better capture the emotions of the reply utterance. The\npresent method has achieved 0.815 and 0.885 micro F1 score in the testing\ndataset of Friends and EmotionPush, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06267v2",
            "date": "2019-08-17 09:18:47+00:00",
            "title": "Message Passing Attention Networks for Document Understanding",
            "authors": [
                "Giannis Nikolentzos",
                "Antoine J. -P. Tixier",
                "Michalis Vazirgiannis"
            ],
            "abstract": "Graph neural networks have recently emerged as a very effective framework for\nprocessing graph-structured data. These models have achieved state-of-the-art\nperformance in many tasks. Most graph neural networks can be described in terms\nof message passing, vertex update, and readout functions. In this paper, we\nrepresent documents as word co-occurrence networks and propose an application\nof the message passing framework to NLP, the Message Passing Attention network\nfor Document understanding (MPAD). We also propose several hierarchical\nvariants of MPAD. Experiments conducted on 10 standard text classification\ndatasets show that our architectures are competitive with the state-of-the-art.\nAblation studies reveal further insights about the impact of the different\ncomponents on performance. Code is publicly available at:\nhttps://github.com/giannisnik/mpad .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06288v2",
            "date": "2019-08-17 10:36:43+00:00",
            "title": "Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting",
            "authors": [
                "Sebastian Borgeaud",
                "Guy Emerson"
            ],
            "abstract": "We propose a method for natural language generation, choosing the most\nrepresentative output rather than the most likely output. By viewing the\nlanguage generation process from the voting theory perspective, we define\nrepresentativeness using range voting and a similarity measure. The proposed\nmethod can be applied when generating from any probabilistic language model,\nincluding n-gram models and neural network models. We evaluate different\nsimilarity measures on an image captioning task and a machine translation task,\nand show that our method generates longer and more diverse sentences, providing\na solution to the common problem of short outputs being preferred over longer\nand more informative ones. The generated sentences obtain higher BLEU scores,\nparticularly when the beam size is large. We also perform a human evaluation on\nboth tasks and find that the outputs generated using our method are rated\nhigher.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06361v1",
            "date": "2019-08-18 01:28:45+00:00",
            "title": "Understanding Undesirable Word Embedding Associations",
            "authors": [
                "Kawin Ethayarajh",
                "David Duvenaud",
                "Graeme Hirst"
            ],
            "abstract": "Word embeddings are often criticized for capturing undesirable word\nassociations such as gender stereotypes. However, methods for measuring and\nremoving such biases remain poorly understood. We show that for any embedding\nmodel that implicitly does matrix factorization, debiasing vectors post hoc\nusing subspace projection (Bolukbasi et al., 2016) is, under certain\nconditions, equivalent to training on an unbiased corpus. We also prove that\nWEAT, the most common association test for word embeddings, systematically\noverestimates bias. Given that the subspace projection method is provably\neffective, we use it to derive a new measure of association called the\n$\\textit{relational inner product association}$ (RIPA). Experiments with RIPA\nreveal that, on average, skipgram with negative sampling (SGNS) does not make\nmost words any more gendered than they are in the training corpus. However, for\ngender-stereotyped words, SGNS actually amplifies the gender association in the\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06606v2",
            "date": "2019-08-19 06:21:29+00:00",
            "title": "Question Answering based Clinical Text Structuring Using Pre-trained Language Model",
            "authors": [
                "Jiahui Qiu",
                "Yangming Zhou",
                "Zhiyuan Ma",
                "Tong Ruan",
                "Jinlin Liu",
                "Jing Sun"
            ],
            "abstract": "Clinical text structuring is a critical and fundamental task for clinical\nresearch. Traditional methods such as taskspecific end-to-end models and\npipeline models usually suffer from the lack of dataset and error propagation.\nIn this paper, we present a question answering based clinical text structuring\n(QA-CTS) task to unify different specific tasks and make dataset shareable. A\nnovel model that aims to introduce domain-specific features (e.g., clinical\nnamed entity information) into pre-trained language model is also proposed for\nQA-CTS task. Experimental results on Chinese pathology reports collected from\nRuijing Hospital demonstrate our presented QA-CTS task is very effective to\nimprove the performance on specific tasks. Our proposed model also competes\nfavorably with strong baseline models in specific tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06725v5",
            "date": "2019-08-19 12:08:27+00:00",
            "title": "Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models",
            "authors": [
                "Zhi-Xiu Ye",
                "Qian Chen",
                "Wen Wang",
                "Zhen-Hua Ling"
            ],
            "abstract": "The state-of-the-art pre-trained language representation models, such as\nBidirectional Encoder Representations from Transformers (BERT), rarely\nincorporate commonsense knowledge or other knowledge explicitly. We propose a\npre-training approach for incorporating commonsense knowledge into language\nrepresentation models. We construct a commonsense-related multi-choice question\nanswering dataset for pre-training a neural language representation model. The\ndataset is created automatically by our proposed \"align, mask, and select\"\n(AMS) method. We also investigate different pre-training tasks. Experimental\nresults demonstrate that pre-training models using the proposed approach\nfollowed by fine-tuning achieve significant improvements over previous\nstate-of-the-art models on two commonsense-related benchmarks, including\nCommonsenseQA and Winograd Schema Challenge. We also observe that fine-tuned\nmodels after the proposed pre-training approach maintain comparable performance\non other NLP tasks, such as sentence classification and natural language\ninference tasks, compared to the original BERT models. These results verify\nthat the proposed approach, while significantly improving commonsense-related\nNLP tasks, does not degrade the general language representation capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06785v1",
            "date": "2019-08-19 13:21:10+00:00",
            "title": "Fast End-to-End Wikification",
            "authors": [
                "Ilya Shnayderman",
                "Liat Ein-Dor",
                "Yosi Mass",
                "Alon Halfon",
                "Benjamin Sznajder",
                "Artem Spector",
                "Yoav Katz",
                "Dafna Sheinwald",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "Wikification of large corpora is beneficial for various NLP applications.\nExisting methods focus on quality performance rather than run-time, and are\ntherefore non-feasible for large data. Here, we introduce RedW, a run-time\noriented Wikification solution, based on Wikipedia redirects, that can Wikify\nmassive corpora with competitive performance. We further propose an efficient\nmethod for estimating RedW confidence, opening the door for applying more\ndemanding methods only on top of RedW lower-confidence results. Our\nexperimental results support the validity of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06870v1",
            "date": "2019-08-19 15:11:27+00:00",
            "title": "Fine-grained Sentiment Analysis with Faithful Attention",
            "authors": [
                "Ruiqi Zhong",
                "Steven Shao",
                "Kathleen McKeown"
            ],
            "abstract": "While the general task of textual sentiment classification has been widely\nstudied, much less research looks specifically at sentiment between a specified\nsource and target. To tackle this problem, we experimented with a\nstate-of-the-art relation extraction model. Surprisingly, we found that despite\nreasonable performance, the model's attention was often systematically\nmisaligned with the words that contribute to sentiment. Thus, we directly\ntrained the model's attention with human rationales and improved our model\nperformance by a robust 4~8 points on all tasks we defined on our data sets. We\nalso present a rigorous analysis of the model's attention, both trained and\nuntrained, using novel and intuitive metrics. Our results show that untrained\nattention does not provide faithful explanations; however, trained attention\nwith concisely annotated human rationales not only increases performance, but\nalso brings faithful explanations. Encouragingly, a small amount of annotated\nhuman rationales suffice to correct the attention in our task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06931v1",
            "date": "2019-08-19 17:03:03+00:00",
            "title": "UDPipe at SIGMORPHON 2019: Contextualized Embeddings, Regularization with Morphological Categories, Corpora Merging",
            "authors": [
                "Milan Straka",
                "Jana Strakov\u00e1",
                "Jan Haji\u010d"
            ],
            "abstract": "We present our contribution to the SIGMORPHON 2019 Shared Task:\nCrosslinguality and Context in Morphology, Task 2: contextual morphological\nanalysis and lemmatization.\n  We submitted a modification of the UDPipe 2.0, one of best-performing systems\nof the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal\nDependencies and an overall winner of the The 2018 Shared Task on Extrinsic\nParser Evaluation.\n  As our first improvement, we use the pretrained contextualized embeddings\n(BERT) as additional inputs to the network; secondly, we use individual\nmorphological features as regularization; and finally, we merge the selected\ncorpora of the same language.\n  In the lemmatization task, our system exceeds all the submitted systems by a\nwide margin with lemmatization accuracy 95.78 (second best was 95.00, third\n94.46). In the morphological analysis, our system placed tightly second: our\nmorphological analysis accuracy was 93.19, the winning system's 93.23.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06938v2",
            "date": "2019-08-19 17:22:58+00:00",
            "title": "Encoder-Agnostic Adaptation for Conditional Language Generation",
            "authors": [
                "Zachary M. Ziegler",
                "Luke Melas-Kyriazi",
                "Sebastian Gehrmann",
                "Alexander M. Rush"
            ],
            "abstract": "Large pretrained language models have changed the way researchers approach\ndiscriminative natural language understanding tasks, leading to the dominance\nof approaches that adapt a pretrained model for arbitrary downstream tasks.\nHowever it is an open-question how to use similar techniques for language\ngeneration. Early results in the encoder-agnostic setting have been mostly\nnegative. In this work we explore methods for adapting a pretrained language\nmodel to arbitrary conditional input. We observe that pretrained transformer\nmodels are sensitive to large parameter changes during tuning. We therefore\npropose an adaptation that directly injects arbitrary conditioning into self\nattention, an approach we call pseudo self attention. Through experiments on\nfour diverse conditional text generation tasks we show that this\nencoder-agnostic technique outperforms strong baselines, produces coherent\ngenerations, and is data efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06941v1",
            "date": "2019-08-19 17:26:13+00:00",
            "title": "Why So Down? The Role of Negative (and Positive) Pointwise Mutual Information in Distributional Semantics",
            "authors": [
                "Alexandre Salle",
                "Aline Villavicencio"
            ],
            "abstract": "In distributional semantics, the pointwise mutual information\n($\\mathit{PMI}$) weighting of the cooccurrence matrix performs far better than\nraw counts. There is, however, an issue with unobserved pair cooccurrences as\n$\\mathit{PMI}$ goes to negative infinity. This problem is aggravated by\nunreliable statistics from finite corpora which lead to a large number of such\npairs. A common practice is to clip negative $\\mathit{PMI}$\n($\\mathit{\\texttt{-} PMI}$) at $0$, also known as Positive $\\mathit{PMI}$\n($\\mathit{PPMI}$). In this paper, we investigate alternative ways of dealing\nwith $\\mathit{\\texttt{-} PMI}$ and, more importantly, study the role that\nnegative information plays in the performance of a low-rank, weighted\nfactorization of different $\\mathit{PMI}$ matrices. Using various semantic and\nsyntactic tasks as probes into models which use either negative or positive\n$\\mathit{PMI}$ (or both), we find that most of the encoded semantics and syntax\ncome from positive $\\mathit{PMI}$, in contrast to $\\mathit{\\texttt{-} PMI}$\nwhich contributes almost exclusively syntactic information. Our findings deepen\nour understanding of distributional semantics, while also introducing novel\n$PMI$ variants and grounding the popular $PPMI$ measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07218v5",
            "date": "2019-08-20 08:33:58+00:00",
            "title": "CA-EHN: Commonsense Analogy from E-HowNet",
            "authors": [
                "Peng-Hsuan Li",
                "Tsan-Yu Yang",
                "Wei-Yun Ma"
            ],
            "abstract": "Embedding commonsense knowledge is crucial for end-to-end models to\ngeneralize inference beyond training corpora. However, existing word analogy\ndatasets have tended to be handcrafted, involving permutations of hundreds of\nwords with only dozens of pre-defined relations, mostly morphological relations\nand named entities. In this work, we model commonsense knowledge down to\nword-level analogical reasoning by leveraging E-HowNet, an ontology that\nannotates 88K Chinese words with their structured sense definitions and English\ntranslations. We present CA-EHN, the first commonsense word analogy dataset\ncontaining 90,505 analogies covering 5,656 words and 763 relations. Experiments\nshow that CA-EHN stands out as a great indicator of how well word\nrepresentations embed commonsense knowledge. The dataset is publicly available\nat https://github.com/ckiplab/CA-EHN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07245v4",
            "date": "2019-08-20 09:37:42+00:00",
            "title": "GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge",
            "authors": [
                "Luyao Huang",
                "Chi Sun",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous\nword in a particular context. Traditional supervised methods rarely take into\nconsideration the lexical resources like WordNet, which are widely utilized in\nknowledge-based methods. Recent studies have shown the effectiveness of\nincorporating gloss (sense definition) into neural networks for WSD. However,\ncompared with traditional word expert supervised methods, they have not\nachieved much improvement. In this paper, we focus on how to better leverage\ngloss knowledge in a supervised neural WSD system. We construct context-gloss\npairs and propose three BERT-based models for WSD. We fine-tune the pre-trained\nBERT model on SemCor3.0 training corpus and the experimental results on several\nEnglish all-words WSD benchmark datasets show that our approach outperforms the\nstate-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07397v2",
            "date": "2019-08-20 14:36:57+00:00",
            "title": "Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisited",
            "authors": [
                "Artur Kulmizev",
                "Miryam de Lhoneux",
                "Johannes Gontrum",
                "Elena Fano",
                "Joakim Nivre"
            ],
            "abstract": "Transition-based and graph-based dependency parsers have previously been\nshown to have complementary strengths and weaknesses: transition-based parsers\nexploit rich structural features but suffer from error propagation, while\ngraph-based parsers benefit from global optimization but have restricted\nfeature scope. In this paper, we show that, even though some details of the\npicture have changed after the switch to neural networks and continuous\nrepresentations, the basic trade-off between rich features and global\noptimization remains essentially the same. Moreover, we show that deep\ncontextualized word embeddings, which allow parsers to pack information about\nglobal sentence structure into local feature representations, benefit\ntransition-based parsers more than graph-based parsers, making the two\napproaches virtually equivalent in terms of both accuracy and error profile. We\nargue that the reason is that these representations help prevent search errors\nand thereby allow transition-based parsers to better exploit their inherent\nstrength of making accurate local decisions. We support this explanation by an\nerror analysis of parsing experiments on 13 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07448v1",
            "date": "2019-08-20 15:52:59+00:00",
            "title": "Evaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing",
            "authors": [
                "Milan Straka",
                "Jana Strakov\u00e1",
                "Jan Haji\u010d"
            ],
            "abstract": "We present an extensive evaluation of three recently proposed methods for\ncontextualized embeddings on 89 corpora in 54 languages of the Universal\nDependencies 2.3 in three tasks: POS tagging, lemmatization, and dependency\nparsing. Employing the BERT, Flair and ELMo as pretrained embedding inputs in a\nstrong baseline of UDPipe 2.0, one of the best-performing systems of the CoNLL\n2018 Shared Task and an overall winner of the EPE 2018, we present a one-to-one\ncomparison of the three contextualized word embedding methods, as well as a\ncomparison with word2vec-like pretrained embeddings and with end-to-end\ncharacter-level word embeddings. We report state-of-the-art results in all\nthree tasks as compared to results on UD 2.2 in the CoNLL 2018 Shared Task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07491v1",
            "date": "2019-08-20 17:07:52+00:00",
            "title": "Controversy in Context",
            "authors": [
                "Benjamin Sznajder",
                "Ariel Gera",
                "Yonatan Bilu",
                "Dafna Sheinwald",
                "Ella Rabinovich",
                "Ranit Aharonov",
                "David Konopnicki",
                "Noam Slonim"
            ],
            "abstract": "With the growing interest in social applications of Natural Language\nProcessing and Computational Argumentation, a natural question is how\ncontroversial a given concept is. Prior works relied on Wikipedia's metadata\nand on content analysis of the articles pertaining to a concept in question.\nHere we show that the immediate textual context of a concept is strongly\nindicative of this property, and, using simple and language-independent\nmachine-learning tools, we leverage this observation to achieve\nstate-of-the-art results in controversiality prediction. In addition, we\nanalyze and make available a new dataset of concepts labeled for\ncontroversiality. It is significantly larger than existing datasets, and grades\nconcepts on a 0-10 scale, rather than treating controversiality as a binary\nlabel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07687v1",
            "date": "2019-08-21 03:02:56+00:00",
            "title": "MoEL: Mixture of Empathetic Listeners",
            "authors": [
                "Zhaojiang Lin",
                "Andrea Madotto",
                "Jamin Shin",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "Previous research on empathetic dialogue systems has mostly focused on\ngenerating responses given certain emotions. However, being empathetic not only\nrequires the ability of generating emotional responses, but more importantly,\nrequires the understanding of user emotions and replying appropriately. In this\npaper, we propose a novel end-to-end approach for modeling empathy in dialogue\nsystems: Mixture of Empathetic Listeners (MoEL). Our model first captures the\nuser emotions and outputs an emotion distribution. Based on this, MoEL will\nsoftly combine the output states of the appropriate Listener(s), which are each\noptimized to react to certain emotions, and generate an empathetic response.\nHuman evaluations on empathetic-dialogues (Rashkin et al., 2018) dataset\nconfirm that MoEL outperforms multitask training baseline in terms of empathy,\nrelevance, and fluency. Furthermore, the case study on generated responses of\ndifferent Listeners shows high interpretability of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07688v1",
            "date": "2019-08-21 03:03:12+00:00",
            "title": "Improving Neural Machine Translation with Pre-trained Representation",
            "authors": [
                "Rongxiang Weng",
                "Heng Yu",
                "Shujian Huang",
                "Weihua Luo",
                "Jiajun Chen"
            ],
            "abstract": "Monolingual data has been demonstrated to be helpful in improving the\ntranslation quality of neural machine translation (NMT). The current methods\nstay at the usage of word-level knowledge, such as generating synthetic\nparallel data or extracting information from word embedding. In contrast, the\npower of sentence-level contextual knowledge which is more complex and diverse,\nplaying an important role in natural language generation, has not been fully\nexploited. In this paper, we propose a novel structure which could leverage\nmonolingual data to acquire sentence-level contextual representations. Then, we\ndesign a framework for integrating both source and target sentence-level\nrepresentations into NMT model to improve the translation quality. Experimental\nresults on Chinese-English, German-English machine translation tasks show that\nour proposed model achieves improvement over strong Transformer baselines,\nwhile experiments on English-Turkish further demonstrate the effectiveness of\nour approach in the low-resource scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07690v1",
            "date": "2019-08-21 03:09:16+00:00",
            "title": "Latent Relation Language Models",
            "authors": [
                "Hiroaki Hayashi",
                "Zecong Hu",
                "Chenyan Xiong",
                "Graham Neubig"
            ],
            "abstract": "In this paper, we propose Latent Relation Language Models (LRLMs), a class of\nlanguage models that parameterizes the joint distribution over the words in a\ndocument and the entities that occur therein via knowledge graph relations.\nThis model has a number of attractive properties: it not only improves language\nmodeling performance, but is also able to annotate the posterior probability of\nentity spans for a given text through relations. Experiments demonstrate\nempirical improvements over both a word-based baseline language model and a\nprevious approach that incorporates knowledge graph information. Qualitative\nanalysis further demonstrates the proposed model's ability to learn to predict\nappropriate relations in context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07705v1",
            "date": "2019-08-21 04:05:54+00:00",
            "title": "Copy-Enhanced Heterogeneous Information Learning for Dialogue State Tracking",
            "authors": [
                "Qingbin Liu",
                "Shizhu He",
                "Kang Liu",
                "Shengping Liu",
                "Jun Zhao"
            ],
            "abstract": "Dialogue state tracking (DST) is an essential component in task-oriented\ndialogue systems, which estimates user goals at every dialogue turn. However,\nmost previous approaches usually suffer from the following problems. Many\ndiscriminative models, especially end-to-end (E2E) models, are difficult to\nextract unknown values that are not in the candidate ontology; previous\ngenerative models, which can extract unknown values from utterances, degrade\nthe performance due to ignoring the semantic information of pre-defined\nontology. Besides, previous generative models usually need a hand-crafted list\nto normalize the generated values. How to integrate the semantic information of\npre-defined ontology and dialogue text (heterogeneous texts) to generate\nunknown values and improve performance becomes a severe challenge. In this\npaper, we propose a Copy-Enhanced Heterogeneous Information Learning model with\nmultiple encoder-decoder for DST (CEDST), which can effectively generate all\npossible values including unknown values by copying values from heterogeneous\ntexts. Meanwhile, CEDST can effectively decompose the large state space into\nseveral small state spaces through multi-encoder, and employ multi-decoder to\nmake full use of the reduced spaces to generate values. Multi-encoder-decoder\narchitecture can significantly improve performance. Experiments show that CEDST\ncan achieve state-of-the-art results on two datasets and our constructed\ndatasets with many unknown values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07721v2",
            "date": "2019-08-21 06:56:08+00:00",
            "title": "Fine-tuning BERT for Joint Entity and Relation Extraction in Chinese Medical Text",
            "authors": [
                "Kui Xue",
                "Yangming Zhou",
                "Zhiyuan Ma",
                "Tong Ruan",
                "Huanhuan Zhang",
                "Ping He"
            ],
            "abstract": "Entity and relation extraction is the necessary step in structuring medical\ntext. However, the feature extraction ability of the bidirectional long short\nterm memory network in the existing model does not achieve the best effect. At\nthe same time, the language model has achieved excellent results in more and\nmore natural language processing tasks. In this paper, we present a focused\nattention model for the joint entity and relation extraction task. Our model\nintegrates well-known BERT language model into joint learning through dynamic\nrange attention mechanism, thus improving the feature representation ability of\nshared parameter layer. Experimental results on coronary angiography texts\ncollected from Shuguang Hospital show that the F1-score of named entity\nrecognition and relation classification tasks reach 96.89% and 88.51%, which\nare better than state-of-the-art methods 1.65% and 1.22%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07742v4",
            "date": "2019-08-21 08:01:13+00:00",
            "title": "On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning",
            "authors": [
                "Yerai Doval",
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "Cross-lingual word embeddings are vector representations of words in\ndifferent languages where words with similar meaning are represented by similar\nvectors, regardless of the language. Recent developments which construct these\nembeddings by aligning monolingual spaces have shown that accurate alignments\ncan be obtained with little or no supervision. However, the focus has been on a\nparticular controlled scenario for evaluation, and there is no strong evidence\non how current state-of-the-art systems would fare with noisy text or for\nlanguage pairs with major linguistic differences. In this paper we present an\nextensive evaluation over multiple cross-lingual embedding models, analyzing\ntheir strengths and limitations with respect to different variables such as\ntarget language, training corpora and amount of supervision. Our conclusions\nput in doubt the view that high-quality cross-lingual embeddings can always be\nlearned without much supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07761v1",
            "date": "2019-08-21 09:25:53+00:00",
            "title": "Predict Emoji Combination with Retrieval Strategy",
            "authors": [
                "Weitsung Lin",
                "Tinghsuan Chao",
                "Jianmin Wu",
                "Tianhuang Su"
            ],
            "abstract": "As emojis are widely used in social media, people not only use an emoji to\nexpress their emotions or mention things but also extend its usage to represent\ncomplicate emotions, concepts or activities by combining multiple emojis. In\nthis work, we study how emoji combination, a consecutive emoji sequence, is\nused like a new language. We propose a novel algorithm called Retrieval\nStrategy to predict what emoji combination follows given a short text as\ncontext. Our algorithm treats emoji combinations as phrase in language, ranking\nsets of emoji combinations like retrieving words from dictionary. We show that\nour algorithm largely improves the F1 score from 0.141 to 0.204 on emoji\ncombination prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07831v1",
            "date": "2019-08-19 05:40:42+00:00",
            "title": "Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase Generation Datasets",
            "authors": [
                "Hongren Mao",
                "Hung-yi Lee"
            ],
            "abstract": "Paraphrase generation is an interesting and challenging NLP task which has\nnumerous practical applications. In this paper, we analyze datasets commonly\nused for paraphrase generation research, and show that simply parroting input\nsentences surpasses state-of-the-art models in the literature when evaluated on\nstandard metrics. Our findings illustrate that a model could be seemingly adept\nat generating paraphrases, despite only making trivial changes to the input\nsentence or even none at all.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07836v1",
            "date": "2019-08-16 00:40:08+00:00",
            "title": "PubLayNet: largest dataset ever for document layout analysis",
            "authors": [
                "Xu Zhong",
                "Jianbin Tang",
                "Antonio Jimeno Yepes"
            ],
            "abstract": "Recognizing the layout of unstructured digital documents is an important step\nwhen parsing the documents into structured machine-readable format for\ndownstream applications. Deep neural networks that are developed for computer\nvision have been proven to be an effective method to analyze layout of document\nimages. However, document layout datasets that are currently publicly available\nare several magnitudes smaller than established computing vision datasets.\nModels have to be trained by transfer learning from a base model that is\npre-trained on a traditional computer vision dataset. In this paper, we develop\nthe PubLayNet dataset for document layout analysis by automatically matching\nthe XML representations and the content of over 1 million PDF articles that are\npublicly available on PubMed Central. The size of the dataset is comparable to\nestablished computer vision datasets, containing over 360 thousand document\nimages, where typical document layout elements are annotated. The experiments\ndemonstrate that deep neural networks trained on PubLayNet accurately recognize\nthe layout of scientific articles. The pre-trained models are also a more\neffective base mode for transfer learning on a different document domain. We\nrelease the dataset (https://github.com/ibm-aur-nlp/PubLayNet) to support\ndevelopment and evaluation of more advanced models for document layout\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07855v1",
            "date": "2019-08-20 15:11:06+00:00",
            "title": "GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level",
            "authors": [
                "Zixian Huang",
                "Yulin Shen",
                "Xiao Li",
                "Yuang Wei",
                "Gong Cheng",
                "Lin Zhou",
                "Xinyu Dai",
                "Yuzhong Qu"
            ],
            "abstract": "Scenario-based question answering (SQA) has attracted increasing research\nattention. It typically requires retrieving and integrating knowledge from\nmultiple sources, and applying general knowledge to a specific case described\nby a scenario. SQA widely exists in the medical, geography, and legal\ndomains---both in practice and in the exams. In this paper, we introduce the\nGeoSQA dataset. It consists of 1,981 scenarios and 4,110 multiple-choice\nquestions in the geography domain at high school level, where diagrams (e.g.,\nmaps, charts) have been manually annotated with natural language descriptions\nto benefit NLP research. Benchmark results on a variety of state-of-the-art\nmethods for question answering, textual entailment, and reading comprehension\ndemonstrate the unique challenges presented by SQA for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07888v1",
            "date": "2019-08-21 14:20:35+00:00",
            "title": "Towards Better Understanding of Spontaneous Conversations: Overcoming Automatic Speech Recognition Errors With Intent Recognition",
            "authors": [
                "Piotr \u017belasko",
                "Jan Mizgajski",
                "Miko\u0142aj Morzy",
                "Adrian Szymczak",
                "Piotr Szyma\u0144ski",
                "\u0141ukasz Augustyniak",
                "Yishay Carmiel"
            ],
            "abstract": "In this paper, we present a method for correcting automatic speech\nrecognition (ASR) errors using a finite state transducer (FST) intent\nrecognition framework. Intent recognition is a powerful technique for dialog\nflow management in turn-oriented, human-machine dialogs. This technique can\nalso be very useful in the context of human-human dialogs, though it serves a\ndifferent purpose of key insight extraction from conversations. We argue that\ncurrently available intent recognition techniques are not applicable to\nhuman-human dialogs due to the complex structure of turn-taking and various\ndisfluencies encountered in spontaneous conversations, exacerbated by speech\nrecognition errors and scarcity of domain-specific labeled data. Without\nefficient key insight extraction techniques, raw human-human dialog transcripts\nremain significantly unexploited.\n  Our contribution consists of a novel FST for intent indexing and an algorithm\nfor fuzzy intent search over the lattice - a compact graph encoding of ASR's\nhypotheses. We also develop a pruning strategy to constrain the fuzziness of\nthe FST index search. Extracted intents represent linguistic domain knowledge\nand help us improve (rescore) the original transcript. We compare our method\nwith a baseline, which uses only the most likely transcript hypothesis (best\npath), and find an increase in the total number of recognized intents by 25%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07898v2",
            "date": "2019-08-21 14:48:01+00:00",
            "title": "Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets",
            "authors": [
                "Mor Geva",
                "Yoav Goldberg",
                "Jonathan Berant"
            ],
            "abstract": "Crowdsourcing has been the prevalent paradigm for creating natural language\nunderstanding datasets in recent years. A common crowdsourcing practice is to\nrecruit a small number of high-quality workers, and have them massively\ngenerate examples. Having only a few workers generate the majority of examples\nraises concerns about data diversity, especially when workers freely generate\nsentences. In this paper, we perform a series of experiments showing these\nconcerns are evident in three recent NLP datasets. We show that model\nperformance improves when training with annotator identifiers as features, and\nthat models are able to recognize the most productive annotators. Moreover, we\nshow that often models do not generalize well to examples from annotators that\ndid not contribute to the training set. Our findings suggest that annotator\nbias should be monitored during dataset creation, and that test set annotators\nshould be disjoint from training set annotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08025v3",
            "date": "2019-08-21 17:42:03+00:00",
            "title": "WikiCREM: A Large Unsupervised Corpus for Coreference Resolution",
            "authors": [
                "Vid Kocijan",
                "Oana-Maria Camburu",
                "Ana-Maria Cretu",
                "Yordan Yordanov",
                "Phil Blunsom",
                "Thomas Lukasiewicz"
            ],
            "abstract": "Pronoun resolution is a major area of natural language understanding.\nHowever, large-scale training sets are still scarce, since manually labelling\ndata is costly. In this work, we introduce WikiCREM (Wikipedia CoREferences\nMasked) a large-scale, yet accurate dataset of pronoun disambiguation\ninstances. We use a language-model-based approach for pronoun resolution in\ncombination with our WikiCREM dataset. We compare a series of models on a\ncollection of diverse and challenging coreference resolution problems, where we\nmatch or outperform previous state-of-the-art approaches on 6 out of 7\ndatasets, such as GAP, DPR, WNLI, PDP, WinoBias, and WinoGender. We release our\nmodel to be used off-the-shelf for solving pronoun disambiguation.",
            "journal ref": "IJCNLP-EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08039v1",
            "date": "2019-08-21 23:12:36+00:00",
            "title": "\"Mask and Infill\" : Applying Masked Language Model to Sentiment Transfer",
            "authors": [
                "Xing Wu",
                "Tao Zhang",
                "Liangjun Zang",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "This paper focuses on the task of sentiment transfer on non-parallel text,\nwhich modifies sentiment attributes (e.g., positive or negative) of sentences\nwhile preserving their attribute-independent content. Due to the limited\ncapability of RNNbased encoder-decoder structure to capture deep and long-range\ndependencies among words, previous works can hardly generate satisfactory\nsentences from scratch. When humans convert the sentiment attribute of a\nsentence, a simple but effective approach is to only replace the original\nsentimental tokens in the sentence with target sentimental expressions, instead\nof building a new sentence from scratch. Such a process is very similar to the\ntask of Text Infilling or Cloze, which could be handled by a deep bidirectional\nMasked Language Model (e.g. BERT). So we propose a two step approach \"Mask and\nInfill\". In the mask step, we separate style from content by masking the\npositions of sentimental tokens. In the infill step, we retrofit MLM to\nAttribute Conditional MLM, to infill the masked positions by predicting words\nor phrases conditioned on the context1 and target sentiment. We evaluate our\nmodel on two review datasets with quantitative, qualitative, and human\nevaluations. Experimental results demonstrate that our models improve\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08104v2",
            "date": "2019-08-21 20:13:44+00:00",
            "title": "Populating Web Scale Knowledge Graphs using Distantly Supervised Relation Extraction and Validation",
            "authors": [
                "Sarthak Dash",
                "Michael R. Glass",
                "Alfio Gliozzo",
                "Mustafa Canim"
            ],
            "abstract": "In this paper, we propose a fully automated system to extend knowledge graphs\nusing external information from web-scale corpora. The designed system\nleverages a deep learning based technology for relation extraction that can be\ntrained by a distantly supervised approach. In addition to that, the system\nuses a deep learning approach for knowledge base completion by utilizing the\nglobal structure information of the induced KG to further refine the confidence\nof the newly discovered relations. The designed system does not require any\neffort for adaptation to new languages and domains as it does not use any\nhand-labeled data, NLP analytics and inference rules. Our experiments,\nperformed on a popular academic benchmark demonstrate that the suggested system\nboosts the performance of relation extraction by a wide margin, reporting error\nreductions of 50%, resulting in relative improvement of up to 100%. Also, a\nweb-scale experiment conducted to extend DBPedia with knowledge from Common\nCrawl shows that our system is not only scalable but also does not require any\nadaptation cost, while yielding substantial accuracy gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08113v1",
            "date": "2019-08-21 20:52:53+00:00",
            "title": "X-SQL: reinforce schema representation with context",
            "authors": [
                "Pengcheng He",
                "Yi Mao",
                "Kaushik Chakrabarti",
                "Weizhu Chen"
            ],
            "abstract": "In this work, we present X-SQL, a new network architecture for the problem of\nparsing natural language to SQL query. X-SQL proposes to enhance the structural\nschema representation with the contextual output from BERT-style pre-training\nmodel, and together with type information to learn a new schema representation\nfor down-stream tasks. We evaluated X-SQL on the WikiSQL dataset and show its\nnew state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08191v1",
            "date": "2019-08-22 03:53:36+00:00",
            "title": "Entropy-Enhanced Multimodal Attention Model for Scene-Aware Dialogue Generation",
            "authors": [
                "Kuan-Yen Lin",
                "Chao-Chun Hsu",
                "Yun-Nung Chen",
                "Lun-Wei Ku"
            ],
            "abstract": "With increasing information from social media, there are more and more videos\navailable. Therefore, the ability to reason on a video is important and\ndeserves to be discussed. TheDialog System Technology Challenge (DSTC7)\n(Yoshino et al. 2018) proposed an Audio Visual Scene-aware Dialog (AVSD) task,\nwhich contains five modalities including video, dialogue history, summary, and\ncaption, as a scene-aware environment. In this paper, we propose the\nentropy-enhanced dynamic memory network (DMN) to effectively model video\nmodality. The attention-based GRU in the proposed model can improve the model's\nability to comprehend and memorize sequential information. The entropy\nmechanism can control the attention distribution higher, so each to-be-answered\nquestion can focus more specifically on a small set of video segments. After\nthe entropy-enhanced DMN secures the video context, we apply an attention model\nthat in-corporates summary and caption to generate an accurate answer given the\nquestion about the video. In the official evaluation, our system can achieve\nimproved performance against the released baseline model for both subjective\nand objective evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08206v1",
            "date": "2019-08-22 05:26:25+00:00",
            "title": "Denoising based Sequence-to-Sequence Pre-training for Text Generation",
            "authors": [
                "Liang Wang",
                "Wei Zhao",
                "Ruoyu Jia",
                "Sujian Li",
                "Jingming Liu"
            ],
            "abstract": "This paper presents a new sequence-to-sequence (seq2seq) pre-training method\nPoDA (Pre-training of Denoising Autoencoders), which learns representations\nsuitable for text generation tasks. Unlike encoder-only (e.g., BERT) or\ndecoder-only (e.g., OpenAI GPT) pre-training approaches, PoDA jointly\npre-trains both the encoder and decoder by denoising the noise-corrupted text,\nand it also has the advantage of keeping the network architecture unchanged in\nthe subsequent fine-tuning stage. Meanwhile, we design a hybrid model of\nTransformer and pointer-generator networks as the backbone architecture for\nPoDA. We conduct experiments on two text generation tasks: abstractive\nsummarization, and grammatical error correction. Results on four datasets show\nthat PoDA can improve model performance over strong baselines without using any\ntask-specific techniques and significantly speed up convergence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08210v1",
            "date": "2019-08-22 05:45:30+00:00",
            "title": "Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs",
            "authors": [
                "Yuting Wu",
                "Xiao Liu",
                "Yansong Feng",
                "Zheng Wang",
                "Rui Yan",
                "Dongyan Zhao"
            ],
            "abstract": "Entity alignment is the task of linking entities with the same real-world\nidentity from different knowledge graphs (KGs), which has been recently\ndominated by embedding-based methods. Such approaches work by learning KG\nrepresentations so that entity alignment can be performed by measuring the\nsimilarities between entity embeddings. While promising, prior works in the\nfield often fail to properly capture complex relation information that commonly\nexists in multi-relational KGs, leaving much room for improvement. In this\npaper, we propose a novel Relation-aware Dual-Graph Convolutional Network\n(RDGCN) to incorporate relation information via attentive interactions between\nthe knowledge graph and its dual relation counterpart, and further capture\nneighboring structures to learn better entity representations. Experiments on\nthree real-world cross-lingual datasets show that our approach delivers better\nand more robust results over the state-of-the-art alignment methods by learning\nbetter KG representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08326v8",
            "date": "2019-08-22 11:44:12+00:00",
            "title": "Revisiting Semantic Representation and Tree Search for Similar Question Retrieval",
            "authors": [
                "Tong Guo",
                "Huilin Gao"
            ],
            "abstract": "This paper studies the performances of BERT combined with tree structure in\nshort sentence ranking task. In retrieval-based question answering system, we\nretrieve the most similar question of the query question by ranking all the\nquestions in datasets. If we want to rank all the sentences by neural rankers,\nwe need to score all the sentence pairs. However it consumes large amount of\ntime. So we design a specific tree for searching and combine deep model to\nsolve this problem. We fine-tune BERT on the training data to get semantic\nvector or sentence embeddings on the test data. We use all the sentence\nembeddings of test data to build our tree based on k-means and do beam search\nat predicting time when given a sentence as query. We do the experiments on the\nsemantic textual similarity dataset, Quora Question Pairs, and process the\ndataset for sentence ranking. Experimental results show that our methods\noutperform the strong baseline. Our tree accelerate the predicting speed by\n500%-1000% without losing too much ranking accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08336v1",
            "date": "2019-08-22 12:36:58+00:00",
            "title": "Argument Invention from First Principles",
            "authors": [
                "Yonatan Bilu",
                "Ariel Gera",
                "Daniel Hershcovich",
                "Benjamin Sznajder",
                "Dan Lahav",
                "Guy Moshkowich",
                "Anael Malet",
                "Assaf Gavron",
                "Noam Slonim"
            ],
            "abstract": "Competitive debaters often find themselves facing a challenging task -- how\nto debate a topic they know very little about, with only minutes to prepare,\nand without access to books or the Internet? What they often do is rely on\n\"first principles\", commonplace arguments which are relevant to many topics,\nand which they have refined in past debates.\n  In this work we aim to explicitly define a taxonomy of such principled\nrecurring arguments, and, given a controversial topic, to automatically\nidentify which of these arguments are relevant to the topic.\n  As far as we know, this is the first time that this approach to argument\ninvention is formalized and made explicit in the context of NLP.\n  The main goal of this work is to show that it is possible to define such a\ntaxonomy. While the taxonomy suggested here should be thought of as a \"first\nattempt\" it is nonetheless coherent, covers well the relevant topics and\ncoincides with what professional debaters actually argue in their speeches, and\nfacilitates automatic argument invention for new topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08399v2",
            "date": "2019-08-22 14:16:20+00:00",
            "title": "Controllable Dual Skew Divergence Loss for Neural Machine Translation",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Yingting Wu",
                "Fengshun Xiao",
                "Shu Jiang"
            ],
            "abstract": "In sequence prediction tasks like neural machine translation, training with\ncross-entropy loss often leads to models that overgeneralize and plunge into\nlocal optima. In this paper, we propose an extended loss function called\n\\emph{dual skew divergence} (DSD) that integrates two symmetric terms on KL\ndivergences with a balanced weight. We empirically discovered that such a\nbalanced weight plays a crucial role in applying the proposed DSD loss into\ndeep models. Thus we eventually develop a controllable DSD loss for\ngeneral-purpose scenarios. Our experiments indicate that switching to the DSD\nloss after the convergence of ML training helps models escape local optima and\nstimulates stable performance improvements. Our evaluations on the WMT 2014\nEnglish-German and English-French translation tasks demonstrate that the\nproposed loss as a general and convenient mean for NMT training indeed brings\nperformance improvement in comparison to strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08419v3",
            "date": "2019-08-22 14:51:44+00:00",
            "title": "NE-LP: Normalized Entropy and Loss Prediction based Sampling for Active Learning in Chinese Word Segmentation on EHRs",
            "authors": [
                "Tingting Cai",
                "Zhiyuan Ma",
                "Hong Zheng",
                "Yangming Zhou"
            ],
            "abstract": "Electronic Health Records (EHRs) in hospital information systems contain\npatients' diagnosis and treatments, so EHRs are essential to clinical data\nmining. Of all the tasks in the mining process, Chinese Word Segmentation (CWS)\nis a fundamental and important one, and most state-of-the-art methods greatly\nrely on large-scale of manually-annotated data. Since annotation is\ntime-consuming and expensive, efforts have been devoted to techniques, such as\nactive learning, to locate the most informative samples for modeling. In this\npaper, we follow the trend and present an active learning method for CWS in\nEHRs. Specically, a new sampling strategy combining Normalized Entropy with\nLoss Prediction (NE-LP) is proposed to select the most representative data.\nMeanwhile, to minimize the computational cost of learning, we propose a joint\nmodel including a word segmenter and a loss prediction model. Furthermore, to\ncapture interactions between adjacent characters, bigram features are also\napplied in the joint model. To illustrate the effectiveness of NE-LP, we\nconducted experiments on EHRs collected from the Shuguang Hospital Affiliated\nto Shanghai University of Traditional Chinese Medicine. The results demonstrate\nthat NE-LP consistently outperforms conventional uncertainty-based sampling\nstrategies for active learning in CWS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08486v2",
            "date": "2019-08-22 16:41:57+00:00",
            "title": "Dialogue Coherence Assessment Without Explicit Dialogue Act Labels",
            "authors": [
                "Mohsen Mesgar",
                "Sebastian B\u00fccker",
                "Iryna Gurevych"
            ],
            "abstract": "Recent dialogue coherence models use the coherence features designed for\nmonologue texts, e.g. nominal entities, to represent utterances and then\nexplicitly augment them with dialogue-relevant features, e.g., dialogue act\nlabels. It indicates two drawbacks, (a) semantics of utterances is limited to\nentity mentions, and (b) the performance of coherence models strongly relies on\nthe quality of the input dialogue act labels. We address these issues by\nintroducing a novel approach to dialogue coherence assessment. We use dialogue\nact prediction as an auxiliary task in a multi-task learning scenario to obtain\ninformative utterance representations for coherence assessment. Our approach\nalleviates the need for explicit dialogue act labels during evaluation. The\nresults of our experiments show that our model substantially (more than 20\naccuracy points) outperforms its strong competitors on the DailyDialogue\ncorpus, and performs on par with them on the SwitchBoard corpus for ranking\ndialogues concerning their coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08528v1",
            "date": "2019-08-22 17:58:55+00:00",
            "title": "Unsupervised Lemmatization as Embeddings-Based Word Clustering",
            "authors": [
                "Rudolf Rosa",
                "Zden\u011bk \u017dabokrtsk\u00fd"
            ],
            "abstract": "We focus on the task of unsupervised lemmatization, i.e. grouping together\ninflected forms of one word under one label (a lemma) without the use of\nannotated training data. We propose to perform agglomerative clustering of word\nforms with a novel distance measure. Our distance measure is based on the\nobservation that inflections of the same word tend to be similar both\nstring-wise and in meaning. We therefore combine word embedding cosine\nsimilarity, serving as a proxy to the meaning similarity, with Jaro-Winkler\nedit distance. Our experiments on 23 languages show our approach to be\npromising, surpassing the baseline on 23 of the 28 evaluation datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08566v1",
            "date": "2019-08-22 19:07:34+00:00",
            "title": "Unsupervised Text Summarization via Mixed Model Back-Translation",
            "authors": [
                "Yacine Jernite"
            ],
            "abstract": "Back-translation based approaches have recently lead to significant progress\nin unsupervised sequence-to-sequence tasks such as machine translation or style\ntransfer. In this work, we extend the paradigm to the problem of learning a\nsentence summarization system from unaligned data. We present several initial\nmodels which rely on the asymmetrical nature of the task to perform the first\nback-translation step, and demonstrate the value of combining the data created\nby these diverse initialization methods. Our system outperforms the current\nstate-of-the-art for unsupervised sentence summarization from fully unaligned\ndata by over 2 ROUGE, and matches the performance of recent semi-supervised\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08672v1",
            "date": "2019-08-23 05:36:45+00:00",
            "title": "Joint Extraction of Entities and Relations with a Hierarchical Multi-task Tagging Model",
            "authors": [
                "Zhepei Wei",
                "Yantao Jia",
                "Yuan Tian",
                "Mohammad Javad Hosseini",
                "Mark Steedman",
                "Yi Chang"
            ],
            "abstract": "Entity extraction and relation extraction are two indispensable building\nblocks for knowledge graph construction. Recent works on entity and relation\nextraction have shown the superiority of solving the two problems in a joint\nmanner, where entities and relations are extracted simultaneously to form\nrelational triples in a knowledge graph. However, existing methods ignore the\nhierarchical semantic interdependency between entity extraction (EE) and joint\nextraction (JE), which leaves much to be desired in real applications. In this\nwork, we propose a hierarchical multi-task tagging model, called HMT, which\ncaptures such interdependency and achieves better performance for joint\nextraction of entities and relations. Specifically, the EE task is organized at\nthe bottom layer and JE task at the top layer in a hierarchical structure.\nFurthermore, the learned semantic representation at the lower level can be\nshared by the upper level via multi-task learning. Experimental results\ndemonstrate the effectiveness of the proposed model for joint extraction in\ncomparison with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08676v3",
            "date": "2019-08-23 05:55:46+00:00",
            "title": "Hierarchically-Refined Label Attention Network for Sequence Labeling",
            "authors": [
                "Leyang Cui",
                "Yue Zhang"
            ],
            "abstract": "CRF has been used as a powerful model for statistical sequence labeling. For\nneural sequence labeling, however, BiLSTM-CRF does not always lead to better\nresults compared with BiLSTM-softmax local classification. This can be because\nthe simple Markov label transition model of CRF does not give much information\ngain over strong neural encoding. For better representing label sequences, we\ninvestigate a hierarchically-refined label attention network, which explicitly\nleverages label embeddings and captures potential long-term label dependency by\ngiving each word incrementally refined label distributions with hierarchical\nattention. Results on POS tagging, NER and CCG supertagging show that the\nproposed model not only improves the overall tagging accuracy with similar\nnumber of parameters, but also significantly speeds up the training and testing\ncompared to BiLSTM-CRF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08835v1",
            "date": "2019-08-23 14:09:37+00:00",
            "title": "Deep Learning Based Chatbot Models",
            "authors": [
                "Richard Csaky"
            ],
            "abstract": "A conversational agent (chatbot) is a piece of software that is able to\ncommunicate with humans using natural language. Modeling conversation is an\nimportant task in natural language processing and artificial intelligence.\nWhile chatbots can be used for various tasks, in general they have to\nunderstand users' utterances and provide responses that are relevant to the\nproblem at hand.\n  In my work, I conduct an in-depth survey of recent literature, examining over\n70 publications related to chatbots published in the last 3 years. Then, I\nproceed to make the argument that the very nature of the general conversation\ndomain demands approaches that are different from current state-of-of-the-art\narchitectures. Based on several examples from the literature I show why current\nchatbot models fail to take into account enough priors when generating\nresponses and how this affects the quality of the conversation. In the case of\nchatbots, these priors can be outside sources of information that the\nconversation is conditioned on like the persona or mood of the conversers. In\naddition to presenting the reasons behind this problem, I propose several ideas\non how it could be remedied.\n  The next section focuses on adapting the very recent Transformer model to the\nchatbot domain, which is currently state-of-the-art in neural machine\ntranslation. I first present experiments with the vanilla model, using\nconversations extracted from the Cornell Movie-Dialog Corpus. Secondly, I\naugment the model with some of my ideas regarding the issues of encoder-decoder\narchitectures. More specifically, I feed additional features into the model\nlike mood or persona together with the raw conversation data. Finally, I\nconduct a detailed analysis of how the vanilla model performs on conversational\ndata by comparing it to previous chatbot models and how the additional features\naffect the quality of the generated responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08861v2",
            "date": "2019-08-23 15:09:07+00:00",
            "title": "Neural Poetry: Learning to Generate Poems using Syllables",
            "authors": [
                "Andrea Zugarini",
                "Stefano Melacci",
                "Marco Maggini"
            ],
            "abstract": "Motivated by the recent progresses on machine learning-based models that\nlearn artistic styles, in this paper we focus on the problem of poem\ngeneration. This is a challenging task in which the machine has to capture the\nlinguistic features that strongly characterize a certain poet, as well as the\nsemantics of the poet's production, that are influenced by his personal\nexperiences and by his literary background. Since poetry is constructed using\nsyllables, that regulate the form and structure of poems, we propose a\nsyllable-based neural language model, and we describe a poem generation\nmechanism that is designed around the poet style, automatically selecting the\nmost representative generations. The poetic work of a target author is usually\nnot enough to successfully train modern deep neural networks, so we propose a\nmulti-stage procedure that exploits non-poetic works of the same author, and\nalso other publicly available huge corpora to learn syntax and grammar of the\ntarget language. We focus on the Italian poet Dante Alighieri, widely famous\nfor his Divine Comedy. A quantitative and qualitative experimental analysis of\nthe generated tercets is reported, where we included expert judges with strong\nbackground in humanistic studies. The generated tercets are frequently\nconsidered to be real by a generic population of judges, with relative\ndifference of 56.25\\% with respect to the ones really authored by Dante, and\nexpert judges perceived Dante's style and rhymes in the generated text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08960v1",
            "date": "2019-08-23 18:00:38+00:00",
            "title": "Neural Text Summarization: A Critical Evaluation",
            "authors": [
                "Wojciech Kry\u015bci\u0144ski",
                "Nitish Shirish Keskar",
                "Bryan McCann",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Text summarization aims at compressing long documents into a shorter form\nthat conveys the most important parts of the original document. Despite\nincreased interest in the community and notable research effort, progress on\nbenchmark datasets has stagnated. We critically evaluate key ingredients of the\ncurrent research setup: datasets, evaluation metrics, and models, and highlight\nthree primary shortcomings: 1) automatically collected datasets leave the task\nunderconstrained and may contain noise detrimental to training and evaluation,\n2) current evaluation protocol is weakly correlated with human judgment and\ndoes not account for important characteristics such as factual correctness, 3)\nmodels overfit to layout biases of current datasets and offer limited diversity\nin their outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08962v2",
            "date": "2019-08-23 18:02:05+00:00",
            "title": "Well-Read Students Learn Better: On the Importance of Pre-training Compact Models",
            "authors": [
                "Iulia Turc",
                "Ming-Wei Chang",
                "Kenton Lee",
                "Kristina Toutanova"
            ],
            "abstract": "Recent developments in natural language representations have been accompanied\nby large and expensive models that leverage vast amounts of general-domain text\nthrough self-supervised pre-training. Due to the cost of applying such models\nto down-stream tasks, several model compression techniques on pre-trained\nlanguage representations have been proposed (Sun et al., 2019; Sanh, 2019).\nHowever, surprisingly, the simple baseline of just pre-training and fine-tuning\ncompact models has been overlooked. In this paper, we first show that\npre-training remains important in the context of smaller architectures, and\nfine-tuning pre-trained compact models can be competitive to more elaborate\nmethods proposed in concurrent work. Starting with pre-trained compact models,\nwe then explore transferring task knowledge from large fine-tuned models\nthrough standard knowledge distillation. The resulting simple, yet effective\nand general algorithm, Pre-trained Distillation, brings further improvements.\nThrough extensive experiments, we more generally explore the interaction\nbetween pre-training and distillation under two variables that have been\nunder-studied: model size and properties of unlabeled task data. One surprising\nobservation is that they have a compound effect even when sequentially applied\non the same data. To accelerate future research, we will make our 24\npre-trained miniature BERT models publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08971v2",
            "date": "2019-08-23 18:31:35+00:00",
            "title": "Deploying Technology to Save Endangered Languages",
            "authors": [
                "Hilaria Cruz",
                "Joseph Waring"
            ],
            "abstract": "Computer scientists working on natural language processing, native speakers\nof endangered languages, and field linguists to discuss ways to harness\nAutomatic Speech Recognition, especially neural networks, to automate\nannotation, speech tagging, and text parsing on endangered languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08983v1",
            "date": "2019-08-23 19:15:07+00:00",
            "title": "A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers",
            "authors": [
                "Aditi Chaudhary",
                "Jiateng Xie",
                "Zaid Sheikh",
                "Graham Neubig",
                "Jaime G. Carbonell"
            ],
            "abstract": "Most state-of-the-art models for named entity recognition (NER) rely on the\navailability of large amounts of labeled data, making them challenging to\nextend to new, lower-resourced languages. However, there are now several\nproposed approaches involving either cross-lingual transfer learning, which\nlearns from other highly resourced languages, or active learning, which\nefficiently selects effective training data based on model predictions. This\npaper poses the question: given this recent progress, and limited human\nannotation, what is the most effective method for efficiently creating\nhigh-quality entity recognizers in under-resourced languages? Based on\nextensive experimentation using both simulated and real human annotation, we\nfind a dual-strategy approach best, starting with a cross-lingual transferred\nmodel, then performing targeted annotation of only uncertain entity spans in\nthe target language, minimizing annotator effort. Results demonstrate that\ncross-lingual transfer is a powerful tool when very little data can be\nannotated, but an entity-targeted annotation strategy can achieve competitive\naccuracy quickly, with just one-tenth of training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09022v2",
            "date": "2019-08-23 20:10:36+00:00",
            "title": "Neural data-to-text generation: A comparison between pipeline and end-to-end architectures",
            "authors": [
                "Thiago Castro Ferreira",
                "Chris van der Lee",
                "Emiel van Miltenburg",
                "Emiel Krahmer"
            ],
            "abstract": "Traditionally, most data-to-text applications have been designed using a\nmodular pipeline architecture, in which non-linguistic input data is converted\ninto natural language through several intermediate transformations. In\ncontrast, recent neural models for data-to-text generation have been proposed\nas end-to-end approaches, where the non-linguistic input is rendered in natural\nlanguage with much less explicit intermediate representations in-between. This\nstudy introduces a systematic comparison between neural pipeline and end-to-end\ndata-to-text approaches for the generation of text from RDF triples. Both\narchitectures were implemented making use of state-of-the art deep learning\nmethods as the encoder-decoder Gated-Recurrent Units (GRU) and Transformer.\nAutomatic and human evaluations together with a qualitative analysis suggest\nthat having explicit intermediate steps in the generation process results in\nbetter texts than the ones generated by end-to-end approaches. Moreover, the\npipeline models generalize better to unseen inputs. Data and code are publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09083v2",
            "date": "2019-08-24 03:27:43+00:00",
            "title": "Multi-view Story Characterization from Movie Plot Synopses and Reviews",
            "authors": [
                "Sudipta Kar",
                "Gustavo Aguilar",
                "Mirella Lapata",
                "Thamar Solorio"
            ],
            "abstract": "This paper considers the problem of characterizing stories by inferring\nproperties such as theme and style using written synopses and reviews of\nmovies. We experiment with a multi-label dataset of movie synopses and a tagset\nrepresenting various attributes of stories (e.g., genre, type of events). Our\nproposed multi-view model encodes the synopses and reviews using hierarchical\nattention and shows improvement over methods that only use synopses. Finally,\nwe demonstrate how can we take advantage of such a model to extract a\ncomplementary set of story-attributes from reviews without direct supervision.\nWe have made our dataset and source code publicly available at\nhttps://ritual.uh.edu/ multiview-tag-2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09091v4",
            "date": "2019-08-24 05:07:36+00:00",
            "title": "BERT for Coreference Resolution: Baselines and Analysis",
            "authors": [
                "Mandar Joshi",
                "Omer Levy",
                "Daniel S. Weld",
                "Luke Zettlemoyer"
            ],
            "abstract": "We apply BERT to coreference resolution, achieving strong improvements on the\nOntoNotes (+3.9 F1) and GAP (+11.5 F1) benchmarks. A qualitative analysis of\nmodel predictions indicates that, compared to ELMo and BERT-base, BERT-large is\nparticularly better at distinguishing between related but distinct entities\n(e.g., President and CEO). However, there is still room for improvement in\nmodeling document-level context, conversations, and mention paraphrasing. Our\ncode and models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09122v1",
            "date": "2019-08-24 10:50:23+00:00",
            "title": "Domain-Invariant Feature Distillation for Cross-Domain Sentiment Classification",
            "authors": [
                "Mengting Hu",
                "Yike Wu",
                "Shiwan Zhao",
                "Honglei Guo",
                "Renhong Cheng",
                "Zhong Su"
            ],
            "abstract": "Cross-domain sentiment classification has drawn much attention in recent\nyears. Most existing approaches focus on learning domain-invariant\nrepresentations in both the source and target domains, while few of them pay\nattention to the domain-specific information. Despite the non-transferability\nof the domain-specific information, simultaneously learning domain-dependent\nrepresentations can facilitate the learning of domain-invariant\nrepresentations. In this paper, we focus on aspect-level cross-domain sentiment\nclassification, and propose to distill the domain-invariant sentiment features\nwith the help of an orthogonal domain-dependent task, i.e. aspect detection,\nwhich is built on the aspects varying widely in different domains. We conduct\nextensive experiments on three public datasets and the experimental results\ndemonstrate the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09138v2",
            "date": "2019-08-24 13:42:57+00:00",
            "title": "Query-Based Named Entity Recognition",
            "authors": [
                "Yuxian Meng",
                "Xiaoya Li",
                "Zijun Sun",
                "Jiwei Li"
            ],
            "abstract": "In this paper, we propose a new strategy for the task of named entity\nrecognition (NER). We cast the task as a query-based machine reading\ncomprehension task: e.g., the task of extracting entities with PER is\nformalized as answering the question of \"which person is mentioned in the text\n?\". Such a strategy comes with the advantage that it solves the long-standing\nissue of handling overlapping or nested entities (the same token that\nparticipates in more than one entity categories) with sequence-labeling\ntechniques for NER. Additionally, since the query encodes informative prior\nknowledge, this strategy facilitates the process of entity extraction, leading\nto better performances. We experiment the proposed model on five widely used\nNER datasets on English and Chinese, including MSRA, Resume, OntoNotes, ACE04\nand ACE05. The proposed model sets new SOTA results on all of these datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09209v1",
            "date": "2019-08-24 21:08:26+00:00",
            "title": "Adversarial Domain Adaptation for Machine Reading Comprehension",
            "authors": [
                "Huazheng Wang",
                "Zhe Gan",
                "Xiaodong Liu",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Hongning Wang"
            ],
            "abstract": "In this paper, we focus on unsupervised domain adaptation for Machine Reading\nComprehension (MRC), where the source domain has a large amount of labeled\ndata, while only unlabeled passages are available in the target domain. To this\nend, we propose an Adversarial Domain Adaptation framework (AdaMRC), where\n($i$) pseudo questions are first generated for unlabeled passages in the target\ndomain, and then ($ii$) a domain classifier is incorporated into an MRC model\nto predict which domain a given passage-question pair comes from. The\nclassifier and the passage-question encoder are jointly trained using\nadversarial learning to enforce domain-invariant representation learning.\nComprehensive evaluations demonstrate that our approach ($i$) is generalizable\nto different MRC models and datasets, ($ii$) can be combined with pre-trained\nlarge-scale language models (such as ELMo and BERT), and ($iii$) can be\nextended to semi-supervised learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09283v4",
            "date": "2019-08-25 09:05:06+00:00",
            "title": "Multi-task Learning for Low-resource Second Language Acquisition Modeling",
            "authors": [
                "Yong Hu",
                "Heyan Huang",
                "Tian Lan",
                "Xiaochi Wei",
                "Yuxiang Nie",
                "Jiarui Qi",
                "Liner Yang",
                "Xian-Ling Mao"
            ],
            "abstract": "Second language acquisition (SLA) modeling is to predict whether second\nlanguage learners could correctly answer the questions according to what they\nhave learned. It is a fundamental building block of the personalized learning\nsystem and has attracted more and more attention recently. However, as far as\nwe know, almost all existing methods cannot work well in low-resource scenarios\ndue to lacking of training data. Fortunately, there are some latent common\npatterns among different language-learning tasks, which gives us an opportunity\nto solve the low-resource SLA modeling problem. Inspired by this idea, in this\npaper, we propose a novel SLA modeling method, which learns the latent common\npatterns among different language-learning datasets by multi-task learning and\nare further applied to improving the prediction performance in low-resource\nscenarios. Extensive experiments show that the proposed method performs much\nbetter than the state-of-the-art baselines in the low-resource scenario.\nMeanwhile, it also obtains improvement slightly in the non-low-resource\nscenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09395v1",
            "date": "2019-08-25 21:29:28+00:00",
            "title": "Domain Adaptive Text Style Transfer",
            "authors": [
                "Dianqi Li",
                "Yizhe Zhang",
                "Zhe Gan",
                "Yu Cheng",
                "Chris Brockett",
                "Ming-Ting Sun",
                "Bill Dolan"
            ],
            "abstract": "Text style transfer without parallel data has achieved some practical\nsuccess. However, in the scenario where less data is available, these methods\nmay yield poor performance. In this paper, we examine domain adaptation for\ntext style transfer to leverage massively available data from other domains.\nThese data may demonstrate domain shift, which impedes the benefits of\nutilizing such data for training. To address this challenge, we propose simple\nyet effective domain adaptive text style transfer models, enabling\ndomain-adaptive information exchange. The proposed models presumably learn from\nthe source domain to: (i) distinguish stylized information and generic content\ninformation; (ii) maximally preserve content information; and (iii) adaptively\ntransfer the styles in a domain-aware manner. We evaluate the proposed models\non two style transfer tasks (sentiment and formality) over multiple target\ndomains where only limited non-parallel data is available. Extensive\nexperiments demonstrate the effectiveness of the proposed model compared to the\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09532v3",
            "date": "2019-08-26 08:55:00+00:00",
            "title": "Transductive Data-Selection Algorithms for Fine-Tuning Neural Machine Translation",
            "authors": [
                "Alberto Poncelas",
                "Gideon Maillette de Buy Wenniger",
                "Andy Way"
            ],
            "abstract": "Machine Translation models are trained to translate a variety of documents\nfrom one language into another. However, models specifically trained for a\nparticular characteristics of the documents tend to perform better. Fine-tuning\nis a technique for adapting an NMT model to some domain. In this work, we want\nto use this technique to adapt the model to a given test set. In particular, we\nare using transductive data selection algorithms which take advantage the\ninformation of the test set to retrieve sentences from a larger parallel set.\n  In cases where the model is available at translation time (when the test set\nis provided), it can be adapted with a small subset of data, thereby achieving\nbetter performance than a generic model or a domain-adapted model.",
            "journal ref": "Proceedings of The 8th Workshop on Patent and Scientific\n  Literature Translation, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09590v1",
            "date": "2019-08-26 10:55:42+00:00",
            "title": "Rethinking Attribute Representation and Injection for Sentiment Classification",
            "authors": [
                "Reinald Kim Amplayo"
            ],
            "abstract": "Text attributes, such as user and product information in product reviews,\nhave been used to improve the performance of sentiment classification models.\nThe de facto standard method is to incorporate them as additional biases in the\nattention mechanism, and more performance gains are achieved by extending the\nmodel architecture. In this paper, we show that the above method is the least\neffective way to represent and inject attributes. To demonstrate this\nhypothesis, unlike previous models with complicated architectures, we limit our\nbase model to a simple BiLSTM with attention classifier, and instead focus on\nhow and where the attributes should be incorporated in the model. We propose to\nrepresent attributes as chunk-wise importance weight matrices and consider four\nlocations in the model (i.e., embedding, encoding, attention, classifier) to\ninject attributes. Experiments show that our proposed method achieves\nsignificant improvements over the standard approach and that attention\nmechanism is the worst location to inject attributes, contradicting prior work.\nWe also outperform the state-of-the-art despite our use of a simple base model.\nFinally, we show that these representations transfer well to other tasks. Model\nimplementation and datasets are released here:\nhttps://github.com/rktamplayo/CHIM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09591v2",
            "date": "2019-08-26 10:59:55+00:00",
            "title": "Measuring Patent Claim Generation by Span Relevancy",
            "authors": [
                "Jieh-Sheng Lee",
                "Jieh Hsiang"
            ],
            "abstract": "Our goal of patent claim generation is to realize \"augmented inventing\" for\ninventors by leveraging latest Deep Learning techniques. We envision the\npossibility of building an \"auto-complete\" function for inventors to conceive\nbetter inventions in the era of artificial intelligence. In order to generate\npatent claims with good quality, a fundamental question is how to measure it.\nWe tackle the problem from a perspective of claim span relevancy. Patent claim\nlanguage was rarely explored in the NLP field. It is unique in its own way and\ncontains rich explicit and implicit human annotations. In this work, we propose\na span-based approach and a generic framework to measure patent claim\ngeneration quantitatively. In order to study the effectiveness of patent claim\ngeneration, we define a metric to measure whether two consecutive spans in a\ngenerated patent claims are relevant. We treat such relevancy measurement as a\nspan-pair classification problem, following the concept of natural language\ninference. Technically, the span-pair classifier is implemented by fine-tuning\na pre-trained language model. The patent claim generation is implemented by\nfine-tuning the other pre-trained model. Specifically, we fine-tune a\npre-trained Google BERT model to measure the patent claim spans generated by a\nfine-tuned OpenAI GPT-2 model. In this way, we re-use two of the\nstate-of-the-art pre-trained models in the NLP field. Our result shows the\neffectiveness of the span-pair classifier after fine-tuning the pre-trained\nmodel. It further validates the quantitative metric of span relevancy in patent\nclaim generation. Particularly, we found that the span relevancy ratio measured\nby BERT becomes lower when the diversity in GPT-2 text generation becomes\nhigher.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09605v3",
            "date": "2019-08-26 11:36:16+00:00",
            "title": "Revisiting Simple Domain Adaptation Methods in Unsupervised Neural Machine Translation",
            "authors": [
                "Haipeng Sun",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Tiejun Zhao",
                "Chenhui Chu"
            ],
            "abstract": "Domain adaptation has been well-studied in supervised neural machine\ntranslation (SNMT). However, it has not been well-studied for unsupervised\nneural machine translation (UNMT), although UNMT has recently achieved\nremarkable results in several domain-specific language pairs. Besides the\ninconsistent domains between training data and test data for SNMT, there\nsometimes exists an inconsistent domain between two monolingual training data\nfor UNMT. In this work, we empirically show different scenarios for\nunsupervised neural machine translation. Based on these scenarios, we revisit\nthe effect of the existing domain adaptation methods including batch weighting\nand fine tuning methods in UNMT. Finally, we propose modified methods to\nimprove the performances of domain-specific UNMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09659v1",
            "date": "2019-08-26 13:09:37+00:00",
            "title": "Low-Resource Name Tagging Learned with Weakly Labeled Data",
            "authors": [
                "Yixin Cao",
                "Zikun Hu",
                "Tat-Seng Chua",
                "Zhiyuan Liu",
                "Heng Ji"
            ],
            "abstract": "Name tagging in low-resource languages or domains suffers from inadequate\ntraining data. Existing work heavily relies on additional information, while\nleaving those noisy annotations unexplored that extensively exist on the web.\nIn this paper, we propose a novel neural model for name tagging solely based on\nweakly labeled (WL) data, so that it can be applied in any low-resource\nsettings. To take the best advantage of all WL sentences, we split them into\nhigh-quality and noisy portions for two modules, respectively: (1) a\nclassification module focusing on the large portion of noisy data can\nefficiently and robustly pretrain the tag classifier by capturing textual\ncontext semantics; and (2) a costly sequence labeling module focusing on\nhigh-quality data utilizes Partial-CRFs with non-entity sampling to achieve\nglobal optimum. Two modules are combined via shared parameters. Extensive\nexperiments involving five low-resource languages and fine-grained food domain\ndemonstrate our superior performance (6% and 7.8% F1 gains on average) as well\nas efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09716v1",
            "date": "2019-08-26 14:55:03+00:00",
            "title": "uniblock: Scoring and Filtering Corpus with Unicode Block Information",
            "authors": [
                "Yingbo Gao",
                "Weiyue Wang",
                "Hermann Ney"
            ],
            "abstract": "The preprocessing pipelines in Natural Language Processing usually involve a\nstep of removing sentences consisted of illegal characters. The definition of\nillegal characters and the specific removal strategy depend on the task,\nlanguage, domain, etc, which often lead to tiresome and repetitive scripting of\nrules. In this paper, we introduce a simple statistical method, uniblock, to\novercome this problem. For each sentence, uniblock generates a fixed-size\nfeature vector using Unicode block information of the characters. A Gaussian\nmixture model is then estimated on some clean corpus using variational\ninference. The learned model can then be used to score sentences and filter\ncorpus. We present experimental results on Sentiment Analysis, Language\nModeling and Machine Translation, and show the simplicity and effectiveness of\nour method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09892v1",
            "date": "2019-08-26 19:56:08+00:00",
            "title": "Does BERT agree? Evaluating knowledge of structure dependence through agreement relations",
            "authors": [
                "Geoff Bacon",
                "Terry Regier"
            ],
            "abstract": "Learning representations that accurately model semantics is an important goal\nof natural language processing research. Many semantic phenomena depend on\nsyntactic structure. Recent work examines the extent to which state-of-the-art\nmodels for pre-training representations, such as BERT, capture such\nstructure-dependent phenomena, but is largely restricted to one phenomenon in\nEnglish: number agreement between subjects and verbs. We evaluate BERT's\nsensitivity to four types of structure-dependent agreement relations in a new\nsemi-automatically curated dataset across 26 languages. We show that both the\nsingle-language and multilingual BERT models capture syntax-sensitive agreement\npatterns well in general, but we also highlight the specific linguistic\ncontexts in which their performance degrades.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09898v1",
            "date": "2019-08-26 20:05:37+00:00",
            "title": "Multi-Channel Graph Neural Network for Entity Alignment",
            "authors": [
                "Yixin Cao",
                "Zhiyuan Liu",
                "Chengjiang Li",
                "Zhiyuan Liu",
                "Juanzi Li",
                "Tat-Seng Chua"
            ],
            "abstract": "Entity alignment typically suffers from the issues of structural\nheterogeneity and limited seed alignments. In this paper, we propose a novel\nMulti-channel Graph Neural Network model (MuGNN) to learn alignment-oriented\nknowledge graph (KG) embeddings by robustly encoding two KGs via multiple\nchannels. Each channel encodes KGs via different relation weighting schemes\nwith respect to self-attention towards KG completion and cross-KG attention for\npruning exclusive entities respectively, which are further combined via pooling\ntechniques. Moreover, we also infer and transfer rule knowledge for completing\ntwo KGs consistently. MuGNN is expected to reconcile the structural differences\nof two KGs, and thus make better use of seed alignments. Extensive experiments\non five publicly available datasets demonstrate our superior performance (5%\nHits@1 up on average).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09920v1",
            "date": "2019-08-23 08:58:49+00:00",
            "title": "Reference Network for Neural Machine Translation",
            "authors": [
                "Han Fu",
                "Chenghao Liu",
                "Jianling Sun"
            ],
            "abstract": "Neural Machine Translation (NMT) has achieved notable success in recent\nyears. Such a framework usually generates translations in isolation. In\ncontrast, human translators often refer to reference data, either rephrasing\nthe intricate sentence fragments with common terms in source language, or just\naccessing to the golden translation directly. In this paper, we propose a\nReference Network to incorporate referring process into translation decoding of\nNMT. To construct a \\emph{reference book}, an intuitive way is to store the\ndetailed translation history with extra memory, which is computationally\nexpensive. Instead, we employ Local Coordinates Coding (LCC) to obtain global\ncontext vectors containing monolingual and bilingual contextual information for\nNMT decoding. Experimental results on Chinese-English and English-German tasks\ndemonstrate that our proposed model is effective in improving the translation\nquality with lightweight computation cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09964v1",
            "date": "2019-08-27 00:25:06+00:00",
            "title": "Text Modeling with Syntax-Aware Variational Autoencoders",
            "authors": [
                "Yijun Xiao",
                "William Yang Wang"
            ],
            "abstract": "Syntactic information contains structures and rules about how text sentences\nare arranged. Incorporating syntax into text modeling methods can potentially\nbenefit both representation learning and generation. Variational autoencoders\n(VAEs) are deep generative models that provide a probabilistic way to describe\nobservations in the latent space. When applied to text data, the latent\nrepresentations are often unstructured. We propose syntax-aware variational\nautoencoders (SAVAEs) that dedicate a subspace in the latent dimensions dubbed\nsyntactic latent to represent syntactic structures of sentences. SAVAEs are\ntrained to infer syntactic latent from either text inputs or parsed syntax\nresults as well as reconstruct original text with inferred latent variables.\nExperiments show that SAVAEs are able to achieve lower reconstruction loss on\nfour different data sets. Furthermore, they are capable of generating examples\nwith modified target syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09982v1",
            "date": "2019-08-27 01:52:07+00:00",
            "title": "On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Jamin Shin",
                "Elham J. Barezi",
                "Pascale Fung"
            ],
            "abstract": "Despite their ubiquity in NLP tasks, Long Short-Term Memory (LSTM) networks\nsuffer from computational inefficiencies caused by inherent unparallelizable\nrecurrences, which further aggravates as LSTMs require more parameters for\nlarger memory capacity. In this paper, we propose to apply low-rank matrix\nfactorization (MF) algorithms to different recurrences in LSTMs, and explore\nthe effectiveness on different NLP tasks and model components. We discover that\nadditive recurrence is more important than multiplicative recurrence, and\nexplain this by identifying meaningful correlations between matrix norms and\ncompression performance. We compare our approach across two settings: 1)\ncompressing core LSTM recurrences in language models, 2) compressing biLSTM\nlayers of ELMo evaluated in three downstream NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10023v1",
            "date": "2019-08-27 04:36:31+00:00",
            "title": "MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations",
            "authors": [
                "Dian Yu",
                "Zhou Yu"
            ],
            "abstract": "Dialog act prediction is an essential language comprehension task for both\ndialog system building and discourse analysis. Previous dialog act schemes,\nsuch as SWBD-DAMSL, are designed for human-human conversations, in which\nconversation partners have perfect language understanding ability. In this\npaper, we design a dialog act annotation scheme, MIDAS (Machine Interaction\nDialog Act Scheme), targeted on open-domain human-machine conversations. MIDAS\nis designed to assist machines which have limited ability to understand their\nhuman partners. MIDAS has a hierarchical structure and supports multi-label\nannotations. We collected and annotated a large open-domain human-machine\nspoken conversation dataset (consists of 24K utterances). To show the\napplicability of the scheme, we leverage transfer learning methods to train a\nmulti-label dialog act prediction model and reach an F1 score of 0.79.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10084v1",
            "date": "2019-08-27 08:50:17+00:00",
            "title": "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new\nstate-of-the-art performance on sentence-pair regression tasks like semantic\ntextual similarity (STS). However, it requires that both sentences are fed into\nthe network, which causes a massive computational overhead: Finding the most\nsimilar pair in a collection of 10,000 sentences requires about 50 million\ninference computations (~65 hours) with BERT. The construction of BERT makes it\nunsuitable for semantic similarity search as well as for unsupervised tasks\nlike clustering.\n  In this publication, we present Sentence-BERT (SBERT), a modification of the\npretrained BERT network that use siamese and triplet network structures to\nderive semantically meaningful sentence embeddings that can be compared using\ncosine-similarity. This reduces the effort for finding the most similar pair\nfrom 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while\nmaintaining the accuracy from BERT.\n  We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning\ntasks, where it outperforms other state-of-the-art sentence embeddings methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10090v1",
            "date": "2019-08-27 09:08:12+00:00",
            "title": "On NMT Search Errors and Model Errors: Cat Got Your Tongue?",
            "authors": [
                "Felix Stahlberg",
                "Bill Byrne"
            ],
            "abstract": "We report on search errors and model errors in neural machine translation\n(NMT). We present an exact inference procedure for neural sequence models based\non a combination of beam search and depth-first search. We use our exact search\nto find the global best model scores under a Transformer base model for the\nentire WMT15 English-German test set. Surprisingly, beam search fails to find\nthese global best model scores in most cases, even with a very large beam size\nof 100. For more than 50% of the sentences, the model in fact assigns its\nglobal best score to the empty translation, revealing a massive failure of\nneural models in properly accounting for adequacy. We show by constraining\nsearch with a minimum translation length that at the root of the problem of\nempty translations lies an inherent bias towards shorter translations. We\nconclude that vanilla NMT in its current form requires just the right amount of\nbeam search errors, which, from a modelling perspective, is a highly\nunsatisfactory conclusion indeed, as the model often prefers an empty\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10261v1",
            "date": "2019-08-27 15:10:24+00:00",
            "title": "A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition",
            "authors": [
                "Lilia Simeonova",
                "Kiril Simov",
                "Petya Osenova",
                "Preslav Nakov"
            ],
            "abstract": "We propose a morphologically informed model for named entity recognition,\nwhich is based on LSTM-CRF architecture and combines word embeddings, Bi-LSTM\ncharacter embeddings, part-of-speech (POS) tags, and morphological information.\nWhile previous work has focused on learning from raw word input, using word and\ncharacter embeddings only, we show that for morphologically rich languages,\nsuch as Bulgarian, access to POS information contributes more to the\nperformance gains than the detailed morphological information. Thus, we show\nthat named entity recognition needs only coarse-grained POS tags, but at the\nsame time it can benefit from simultaneously using some POS information of\ndifferent granularity. Our evaluation results over a standard dataset show\nsizable improvements over the state-of-the-art for Bulgarian NER.",
            "journal ref": "RANLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10328v2",
            "date": "2019-08-27 16:59:18+00:00",
            "title": "Movie Plot Analysis via Turning Point Identification",
            "authors": [
                "Pinelopi Papalampidi",
                "Frank Keller",
                "Mirella Lapata"
            ],
            "abstract": "According to screenwriting theory, turning points (e.g., change of plans,\nmajor setback, climax) are crucial narrative moments within a screenplay: they\ndefine the plot structure, determine its progression and segment the screenplay\ninto thematic units (e.g., setup, complications, aftermath). We propose the\ntask of turning point identification in movies as a means of analyzing their\nnarrative structure. We argue that turning points and the segmentation they\nprovide can facilitate processing long, complex narratives, such as\nscreenplays, for summarization and question answering. We introduce a dataset\nconsisting of screenplays and plot synopses annotated with turning points and\npresent an end-to-end neural network model that identifies turning points in\nplot synopses and projects them onto scenes in screenplays. Our model\noutperforms strong baselines based on state-of-the-art sentence representations\nand the expected position of turning points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10423v1",
            "date": "2019-08-27 19:26:31+00:00",
            "title": "Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks",
            "authors": [
                "Zi-Yi Dou",
                "Keyi Yu",
                "Antonios Anastasopoulos"
            ],
            "abstract": "Learning general representations of text is a fundamental problem for many\nnatural language understanding (NLU) tasks. Previously, researchers have\nproposed to use language model pre-training and multi-task learning to learn\nrobust representations. However, these methods can achieve sub-optimal\nperformance in low-resource scenarios. Inspired by the recent success of\noptimization-based meta-learning algorithms, in this paper, we explore the\nmodel-agnostic meta-learning algorithm (MAML) and its variants for low-resource\nNLU tasks. We validate our methods on the GLUE benchmark and show that our\nproposed models can outperform several strong baselines. We further empirically\ndemonstrate that the learned representations can be adapted to new tasks\nefficiently and effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10430v1",
            "date": "2019-08-27 19:38:42+00:00",
            "title": "Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings",
            "authors": [
                "Zi-Yi Dou",
                "Junjie Hu",
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "The recent success of neural machine translation models relies on the\navailability of high quality, in-domain data. Domain adaptation is required\nwhen domain-specific data is scarce or nonexistent. Previous unsupervised\ndomain adaptation strategies include training the model with in-domain copied\nmonolingual or back-translated data. However, these methods use generic\nrepresentations for text regardless of domain shift, which makes it infeasible\nfor translation models to control outputs conditional on a specific domain. In\nthis work, we propose an approach that adapts models with domain-aware feature\nembeddings, which are learned via an auxiliary language modeling task. Our\napproach allows the model to assign domain-specific representations to words\nand output sentences in the desired domain. Our empirical results demonstrate\nthe effectiveness of the proposed strategy, achieving consistent improvements\nin multiple experimental settings. In addition, we show that combining our\nmethod with back translation can further improve the performance of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10461v1",
            "date": "2019-08-27 20:43:49+00:00",
            "title": "A survey of cross-lingual features for zero-shot cross-lingual semantic parsing",
            "authors": [
                "Jingfeng Yang",
                "Federico Fancellu",
                "Bonnie Webber"
            ],
            "abstract": "The availability of corpora to train semantic parsers in English has lead to\nsignificant advances in the field. Unfortunately, for languages other than\nEnglish, annotation is scarce and so are developed parsers. We then ask: could\na parser trained in English be applied to language that it hasn't been trained\non? To answer this question we explore zero-shot cross-lingual semantic parsing\nwhere we train an available coarse-to-fine semantic parser (Liu et al., 2018)\nusing cross-lingual word embeddings and universal dependencies in English and\ntest it on Italian, German and Dutch. Results on the Parallel Meaning Bank - a\nmultilingual semantic graphbank, show that Universal Dependency features\nsignificantly boost performance when used in conjunction with other lexical\nfeatures but modelling the UD structure directly when encoding the input does\nnot.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10703v1",
            "date": "2019-08-28 13:07:05+00:00",
            "title": "Emotion Detection with Neural Personal Discrimination",
            "authors": [
                "Xiabing Zhou",
                "Zhongqing Wang",
                "Shoushan Li",
                "Guodong Zhou",
                "Min Zhang"
            ],
            "abstract": "There have been a recent line of works to automatically predict the emotions\nof posts in social media. Existing approaches consider the posts individually\nand predict their emotions independently. Different from previous researches,\nwe explore the dependence among relevant posts via the authors' backgrounds,\nsince the authors with similar backgrounds, e.g., gender, location, tend to\nexpress similar emotions. However, such personal attributes are not easy to\nobtain in most social media websites, and it is hard to capture\nattributes-aware words to connect similar people. Accordingly, we propose a\nNeural Personal Discrimination (NPD) approach to address above challenges by\ndetermining personal attributes from posts, and connecting relevant posts with\nsimilar attributes to jointly learn their emotions. In particular, we employ\nadversarial discriminators to determine the personal attributes, with attention\nmechanisms to aggregate attributes-aware words. In this way, social\ncorrelationship among different posts can be better addressed. Experimental\nresults show the usefulness of personal attributes, and the effectiveness of\nour proposed NPD approach in capturing such personal attributes with\nsignificant gains over the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10763v2",
            "date": "2019-08-28 15:02:45+00:00",
            "title": "Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual",
            "authors": [
                "He He",
                "Sheng Zha",
                "Haohan Wang"
            ],
            "abstract": "Statistical natural language inference (NLI) models are susceptible to\nlearning dataset bias: superficial cues that happen to associate with the label\non a particular dataset, but are not useful in general, e.g., negation words\nindicate contradiction. As exposed by several recent challenge datasets, these\nmodels perform poorly when such association is absent, e.g., predicting that \"I\nlove dogs\" contradicts \"I don't love cats\". Our goal is to design learning\nalgorithms that guard against known dataset bias. We formalize the concept of\ndataset bias under the framework of distribution shift and present a simple\ndebiasing algorithm based on residual fitting, which we call DRiFt. We first\nlearn a biased model that only uses features that are known to relate to\ndataset bias. Then, we train a debiased model that fits to the residual of the\nbiased model, focusing on examples that cannot be predicted well by biased\nfeatures only. We use DRiFt to train three high-performing NLI models on two\nbenchmark datasets, SNLI and MNLI. Our debiased models achieve significant\ngains over baseline models on two challenge test sets, while maintaining\nreasonable performance on the original test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10917v1",
            "date": "2019-08-28 19:32:00+00:00",
            "title": "SpatialNLI: A Spatial Domain Natural Language Interface to Databases Using Spatial Comprehension",
            "authors": [
                "Jingjing Li",
                "Wenlu Wang",
                "Wei-Shinn Ku",
                "Yingtao Tian",
                "Haixun Wang"
            ],
            "abstract": "A natural language interface (NLI) to databases is an interface that\ntranslates a natural language question to a structured query that is executable\nby database management systems (DBMS). However, an NLI that is trained in the\ngeneral domain is hard to apply in the spatial domain due to the idiosyncrasy\nand expressiveness of the spatial questions. Inspired by the machine\ncomprehension model, we propose a spatial comprehension model that is able to\nrecognize the meaning of spatial entities based on the semantics of the\ncontext. The spatial semantics learned from the spatial comprehension model is\nthen injected to the natural language question to ease the burden of capturing\nthe spatial-specific semantics. With our spatial comprehension model and\ninformation injection, our NLI for the spatial domain, named SpatialNLI, is\nable to capture the semantic structure of the question and translate it to the\ncorresponding syntax of an executable query accurately. We also experimentally\nascertain that SpatialNLI outperforms state-of-the-art methods.",
            "journal ref": "27th ACM SIGSPATIAL International Conference on Advances in\n  Geographic Information Systems (SIGSPATIAL '19), November 5-8, 2019, Chicago,\n  IL, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11017v3",
            "date": "2019-08-29 02:00:37+00:00",
            "title": "A Joint Model for Aspect-Category Sentiment Analysis with Contextualized Aspect Embedding",
            "authors": [
                "Yuncong Li",
                "Cunxiang Yin",
                "Ting Wei",
                "Huiqiang Zhong",
                "Jinchang Luo",
                "Siqi Xu",
                "Xiaohui Wu"
            ],
            "abstract": "Aspect-category sentiment analysis (ACSA) aims to identify all the aspect\ncategories mentioned in the text and their corresponding sentiment polarities.\nSome joint models have been proposed to address this task. However, these joint\nmodels do not solve the following two problems well: mismatching between the\naspect categories and the sentiment words, and data deficiency of some aspect\ncategories. To solve them, we propose a novel joint model which contains a\ncontextualized aspect embedding layer and a shared sentiment prediction layer.\nThe contextualized aspect embedding layer extracts the aspect category related\ninformation, which is used to generate aspect-specific representations for\nsentiment classification like traditional context-independent aspect embedding\n(CIAE) and is therefore called contextualized aspect embedding (CAE). The CAE\ncan mitigate the mismatching problem because it is semantically more related to\nsentiment words than CIAE. The shared sentiment prediction layer transfers\nsentiment knowledge between aspect categories and alleviates the problem caused\nby data deficiency. Experiments conducted on SemEval 2016 Datasets show that\nour proposed model achieves state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11020v2",
            "date": "2019-08-29 02:20:26+00:00",
            "title": "Regularized Context Gates on Transformer for Machine Translation",
            "authors": [
                "Xintong Li",
                "Lemao Liu",
                "Rui Wang",
                "Guoping Huang",
                "Max Meng"
            ],
            "abstract": "Context gates are effective to control the contributions from the source and\ntarget contexts in the recurrent neural network (RNN) based neural machine\ntranslation (NMT). However, it is challenging to extend them into the advanced\nTransformer architecture, which is more complicated than RNN. This paper first\nprovides a method to identify source and target contexts and then introduce a\ngate mechanism to control the source and target contributions in Transformer.\nIn addition, to further reduce the bias problem in the gate mechanism, this\npaper proposes a regularization method to guide the learning of the gates with\nsupervision automatically generated using pointwise mutual information.\nExtensive experiments on 4 translation datasets demonstrate that the proposed\nmodel obtains an averaged gain of 1.0 BLEU score over a strong Transformer\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11046v3",
            "date": "2019-08-29 04:36:30+00:00",
            "title": "Why Attention? Analyze BiLSTM Deficiency and Its Remedies in the Case of NER",
            "authors": [
                "Peng-Hsuan Li",
                "Tsu-Jui Fu",
                "Wei-Yun Ma"
            ],
            "abstract": "BiLSTM has been prevalently used as a core module for NER in a\nsequence-labeling setup. State-of-the-art approaches use BiLSTM with additional\nresources such as gazetteers, language-modeling, or multi-task supervision to\nfurther improve NER. This paper instead takes a step back and focuses on\nanalyzing problems of BiLSTM itself and how exactly self-attention can bring\nimprovements. We formally show the limitation of (CRF-)BiLSTM in modeling\ncross-context patterns for each word -- the XOR limitation. Then, we show that\ntwo types of simple cross-structures -- self-attention and Cross-BiLSTM -- can\neffectively remedy the problem. We test the practical impacts of the deficiency\non real-world NER datasets, OntoNotes 5.0 and WNUT 2017, with clear and\nconsistent improvements over the baseline, up to 8.7% on some of the\nmulti-token entity mentions. We give in-depth analyses of the improvements\nacross several aspects of NER, especially the identification of multi-token\nmentions. This study should lay a sound foundation for future improvements on\nsequence-labeling NER. (Source codes:\nhttps://github.com/jacobvsdanniel/cross-ner)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11047v1",
            "date": "2019-08-29 04:45:38+00:00",
            "title": "Shallow Syntax in Deep Water",
            "authors": [
                "Swabha Swayamdipta",
                "Matthew Peters",
                "Brendan Roof",
                "Chris Dyer",
                "Noah A. Smith"
            ],
            "abstract": "Shallow syntax provides an approximation of phrase-syntactic structure of\nsentences; it can be produced with high accuracy, and is computationally cheap\nto obtain. We investigate the role of shallow syntax-aware representations for\nNLP tasks using two techniques. First, we enhance the ELMo architecture to\nallow pretraining on predicted shallow syntactic parses, instead of just raw\ntext, so that contextual embeddings make use of shallow syntactic context. Our\nsecond method involves shallow syntactic features obtained automatically on\ndownstream task data. Neither approach leads to a significant gain on any of\nthe four downstream tasks we considered relative to ELMo-only baselines.\nFurther analysis using black-box probes confirms that our shallow-syntax-aware\ncontextual embeddings do not transfer to linguistic tasks any more easily than\nELMo's embeddings. We take these findings as evidence that ELMo-style\npretraining discovers representations which make additional awareness of\nshallow syntax redundant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11049v1",
            "date": "2019-08-29 04:53:29+00:00",
            "title": "Multilingual and Multi-Aspect Hate Speech Analysis",
            "authors": [
                "Nedjma Ousidhoum",
                "Zizheng Lin",
                "Hongming Zhang",
                "Yangqiu Song",
                "Dit-Yan Yeung"
            ],
            "abstract": "Current research on hate speech analysis is typically oriented towards\nmonolingual and single classification tasks. In this paper, we present a new\nmultilingual multi-aspect hate speech analysis dataset and use it to test the\ncurrent state-of-the-art multilingual multitask learning approaches. We\nevaluate our dataset in various classification settings, then we discuss how to\nleverage our annotations in order to improve hate speech detection and\nclassification in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11125v1",
            "date": "2019-08-29 09:47:48+00:00",
            "title": "Probing Representations Learned by Multimodal Recurrent and Transformer Models",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Pranava Madhyastha"
            ],
            "abstract": "Recent literature shows that large-scale language modeling provides excellent\nreusable sentence representations with both recurrent and self-attentive\narchitectures. However, there has been less clarity on the commonalities and\ndifferences in the representational properties induced by the two\narchitectures. It also has been shown that visual information serves as one of\nthe means for grounding sentence representations. In this paper, we present a\nmeta-study assessing the representational quality of models where the training\nsignal is obtained from different modalities, in particular, language modeling,\nimage features prediction, and both textual and multimodal machine translation.\nWe evaluate textual and visual features of sentence representations obtained\nusing predominant approaches on image retrieval and semantic textual\nsimilarity. Our experiments reveal that on moderate-sized datasets, a sentence\ncounterpart in a target language or visual modality provides much stronger\ntraining signal for sentence representation than language modeling.\nImportantly, we observe that while the Transformer models achieve superior\nmachine translation quality, representations from the recurrent neural network\nbased models perform significantly better over tasks focused on semantic\nrelevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11141v3",
            "date": "2019-08-29 10:25:10+00:00",
            "title": "Ellipsis Resolution as Question Answering: An Evaluation",
            "authors": [
                "Rahul Aralikatte",
                "Matthew Lamm",
                "Daniel Hardt",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Most, if not all forms of ellipsis (e.g., so does Mary) are similar to\nreading comprehension questions (what does Mary do), in that in order to\nresolve them, we need to identify an appropriate text span in the preceding\ndiscourse. Following this observation, we present an alternative approach for\nEnglish ellipsis resolution relying on architectures developed for question\nanswering (QA). We present both single-task models, and joint models trained on\nauxiliary QA and coreference resolution datasets, clearly outperforming the\ncurrent state of the art for Sluice Ellipsis (from 70.00 to 86.01 F1) and Verb\nPhrase Ellipsis (from 72.89 to 78.66 F1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11152v1",
            "date": "2019-08-29 11:17:33+00:00",
            "title": "A Summarization System for Scientific Documents",
            "authors": [
                "Shai Erera",
                "Michal Shmueli-Scheuer",
                "Guy Feigenblat",
                "Ora Peled Nakash",
                "Odellia Boni",
                "Haggai Roitman",
                "Doron Cohen",
                "Bar Weiner",
                "Yosi Mass",
                "Or Rivlin",
                "Guy Lev",
                "Achiya Jerbi",
                "Jonathan Herzig",
                "Yufang Hou",
                "Charles Jochim",
                "Martin Gleize",
                "Francesca Bonin",
                "David Konopnicki"
            ],
            "abstract": "We present a novel system providing summaries for Computer Science\npublications. Through a qualitative user study, we identified the most valuable\nscenarios for discovery, exploration and understanding of scientific documents.\nBased on these findings, we built a system that retrieves and summarizes\nscientific documents for a given information need, either in form of a\nfree-text query or by choosing categorized values such as scientific tasks,\ndatasets and more. Our system ingested 270,000 papers, and its summarization\nmodule aims to generate concise yet detailed summaries. We validated our\napproach with human experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11214v1",
            "date": "2019-08-29 13:29:36+00:00",
            "title": "Global Reasoning over Database Structures for Text-to-SQL Parsing",
            "authors": [
                "Ben Bogin",
                "Matt Gardner",
                "Jonathan Berant"
            ],
            "abstract": "State-of-the-art semantic parsers rely on auto-regressive decoding, emitting\none symbol at a time. When tested against complex databases that are unobserved\nat training time (zero-shot), the parser often struggles to select the correct\nset of database constants in the new database, due to the local nature of\ndecoding. In this work, we propose a semantic parser that globally reasons\nabout the structure of the output query to make a more contextually-informed\nselection of database constants. We use message-passing through a graph neural\nnetwork to softly select a subset of database constants for the output query,\nconditioned on the question. Moreover, we train a model to rank queries based\non the global alignment of database constants to question words. We apply our\ntechniques to the current state-of-the-art model for Spider, a zero-shot\nsemantic parsing dataset with complex databases, increasing accuracy from 39.4%\nto 47.4%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11254v1",
            "date": "2019-08-29 14:25:23+00:00",
            "title": "FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning",
            "authors": [
                "Jonas Pfeiffer",
                "Christian M. Meyer",
                "Claudia Schulz",
                "Jan Kiesewetter",
                "Jan Zottmann",
                "Michael Sailer",
                "Elisabeth Bauer",
                "Frank Fischer",
                "Martin R. Fischer",
                "Iryna Gurevych"
            ],
            "abstract": "Our proposed system FAMULUS helps students learn to diagnose based on\nautomatic feedback in virtual patient simulations, and it supports instructors\nin labeling training data.\n  Diagnosing is an exceptionally difficult skill to obtain but vital for many\ndifferent professions (e.g., medical doctors, teachers).\n  Previous case simulation systems are limited to multiple-choice questions and\nthus cannot give constructive individualized feedback on a student's diagnostic\nreasoning process.\n  Given initially only limited data, we leverage a (replaceable) NLP model to\nboth support experts in their further data annotation with automatic\nsuggestions, and we provide automatic feedback for students.\n  We argue that because the central model consistently improves, our\ninteractive approach encourages both students and instructors to recurrently\nuse the tool, and thus accelerate the speed of data creation and annotation.\n  We show results from two user studies on diagnostic reasoning in medicine and\nteacher education and outline how our system can be extended to further use\ncases.",
            "journal ref": "EMNLP-IJCNLP 2019: System Demonstrations",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11279v1",
            "date": "2019-08-29 15:05:52+00:00",
            "title": "Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settings",
            "authors": [
                "David Schlangen"
            ],
            "abstract": "Where early work on dialogue in Computational Linguistics put much emphasis\non dialogue structure and its relation to the mental states of the dialogue\nparticipants (e.g., Allen 1979, Grosz & Sidner 1986), current work mostly\nreduces dialogue to the task of producing at any one time a next utterance;\ne.g. in neural chatbot or Visual Dialogue settings. As a methodological\ndecision, this is sound: Even the longest journey is a sequence of steps. It\nbecomes detrimental, however, when the tasks and datasets from which dialogue\nbehaviour is to be learned are tailored too much to this framing of the\nproblem. In this short note, we describe a family of settings which still allow\nto keep dialogues simple, but add a constraint that makes participants care\nabout reaching mutual understanding. In such agreement games, there is a\nsecondary, but explicit goal besides the task level goal, and that is to reach\nmutual understanding about whether the task level goal has been reached. As we\nargue, this naturally triggers meta-semantic interaction and mutual engagement,\nand hence leads to richer data from which to induce models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11337v1",
            "date": "2019-08-29 16:42:19+00:00",
            "title": "CCKS 2019 Shared Task on Inter-Personal Relationship Extraction",
            "authors": [
                "Haitao Wang",
                "Zhengqiu He",
                "Tong Zhu",
                "Hao Shao",
                "Wenliang Chen",
                "Min Zhang"
            ],
            "abstract": "The CCKS2019 shared task was devoted to inter-personal relationship\nextraction. Given two person entities and at least one sentence containing\nthese two entities, participating teams are asked to predict the relationship\nbetween the entities according to a given relation list. This year, 358 teams\nfrom various universities and organizations participated in this task. In this\npaper, we present the task definition, the description of data and the\nevaluation methodology used during this shared task. We also present a brief\noverview of the various methods adopted by the participating teams. Finally, we\npresent the evaluation results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11365v1",
            "date": "2019-08-29 17:50:55+00:00",
            "title": "Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention",
            "authors": [
                "Biao Zhang",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "The general trend in NLP is towards increasing model capacity and performance\nvia deeper neural networks. However, simply stacking more layers of the popular\nTransformer architecture for machine translation results in poor convergence\nand high computational overhead. Our empirical analysis suggests that\nconvergence is poor due to gradient vanishing caused by the interaction between\nresidual connections and layer normalization. We propose depth-scaled\ninitialization (DS-Init), which decreases parameter variance at the\ninitialization stage, and reduces output variance of residual connections so as\nto ease gradient back-propagation through normalization layers. To address\ncomputational cost, we propose a merged attention sublayer (MAtt) which\ncombines a simplified averagebased self-attention sublayer and the\nencoderdecoder attention sublayer on the decoder side. Results on WMT and IWSLT\ntranslation tasks with five translation directions show that deep Transformers\nwith DS-Init and MAtt can substantially outperform their base counterpart in\nterms of BLEU (+1.1 BLEU on average for 12-layer models), while matching the\ndecoding speed of the baseline model thanks to the efficiency improvements of\nMAtt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11421v1",
            "date": "2019-08-29 18:50:51+00:00",
            "title": "Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds",
            "authors": [
                "John P. Lalor",
                "Hao Wu",
                "Hong Yu"
            ],
            "abstract": "Incorporating Item Response Theory (IRT) into NLP tasks can provide valuable\ninformation about model performance and behavior. Traditionally, IRT models are\nlearned using human response pattern (RP) data, presenting a significant\nbottleneck for large data sets like those required for training deep neural\nnetworks (DNNs). In this work we propose learning IRT models using RPs\ngenerated from artificial crowds of DNN models. We demonstrate the\neffectiveness of learning IRT models using DNN-generated data through\nquantitative and qualitative analyses for two NLP tasks. Parameters learned\nfrom human and machine RPs for natural language inference and sentiment\nanalysis exhibit medium to large positive correlations. We demonstrate a\nuse-case for latent difficulty item parameters, namely training set filtering,\nand show that using difficulty to sample training data outperforms baseline\nmethods. Finally, we highlight cases where human expectation about item\ndifficulty does not match difficulty as estimated from the machine RPs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11425v2",
            "date": "2019-08-29 19:11:26+00:00",
            "title": "Cross-lingual topic prediction for speech using translations",
            "authors": [
                "Sameer Bansal",
                "Herman Kamper",
                "Adam Lopez",
                "Sharon Goldwater"
            ],
            "abstract": "Given a large amount of unannotated speech in a low-resource language, can we\nclassify the speech utterances by topic? We consider this question in the\nsetting where a small amount of speech in the low-resource language is paired\nwith text translations in a high-resource language. We develop an effective\ncross-lingual topic classifier by training on just 20 hours of translated\nspeech, using a recent model for direct speech-to-text translation. While the\ntranslations are poor, they are still good enough to correctly classify the\ntopic of 1-minute speech segments over 70% of the time - a 20% improvement over\na majority-class baseline. Such a system could be useful for humanitarian\napplications like crisis response, where incoming speech in a foreign\nlow-resource language must be quickly assessed for further action.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11439v1",
            "date": "2019-08-29 19:57:25+00:00",
            "title": "Feature2Vec: Distributional semantic modelling of human property knowledge",
            "authors": [
                "Steven Derby",
                "Paul Miller",
                "Barry Devereux"
            ],
            "abstract": "Feature norm datasets of human conceptual knowledge, collected in surveys of\nhuman volunteers, yield highly interpretable models of word meaning and play an\nimportant role in neurolinguistic research on semantic cognition. However,\nthese datasets are limited in size due to practical obstacles associated with\nexhaustively listing properties for a large number of words. In contrast, the\ndevelopment of distributional modelling techniques and the availability of vast\ntext corpora have allowed researchers to construct effective vector space\nmodels of word meaning over large lexicons. However, this comes at the cost of\ninterpretable, human-like information about word meaning. We propose a method\nfor mapping human property knowledge onto a distributional semantic space,\nwhich adapts the word2vec architecture to the task of modelling concept\nfeatures. Our approach gives a measure of concept and feature affinity in a\nsingle semantic space, which makes for easy and efficient ranking of candidate\nhuman-derived semantic properties for arbitrary words. We compare our model\nwith a previous approach, and show that it performs better on several\nevaluation tasks. Finally, we discuss how our method could be used to develop\nefficient sampling techniques to extend existing feature norm datasets in a\nreliable way.",
            "journal ref": "Proceedings of the 2019 Conference on Empirical Methods in Natural\n  Language Processing (EMNLP 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11443v1",
            "date": "2019-08-29 20:09:27+00:00",
            "title": "NarrativeTime: Dense High-Speed Temporal Annotation on a Timeline",
            "authors": [
                "Anna Rogers",
                "Gregory Smelkov",
                "Anna Rumshisky"
            ],
            "abstract": "We present NarrativeTime, a new timeline-based annotation scheme for temporal\norder of events in text, and a new densely annotated fiction corpus comparable\nto TimeBank-Dense. NarrativeTime is considerably faster than schemes based on\nevent pairs such as TimeML, and it produces more temporal links between events\nthan TimeBank-Dense, while maintaining comparable agreement on temporal links.\nThis is achieved through new strategies for encoding vagueness in temporal\nrelations and an annotation workflow that takes into account the annotators'\nchunking and commonsense reasoning strategies. NarrativeTime comes with new\nspecialized web-based tools for annotation and adjudication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11474v2",
            "date": "2019-08-29 22:44:41+00:00",
            "title": "Detecting and Reducing Bias in a High Stakes Domain",
            "authors": [
                "Ruiqi Zhong",
                "Yanda Chen",
                "Desmond Patton",
                "Charlotte Selous",
                "Kathy McKeown"
            ],
            "abstract": "Gang-involved youth in cities such as Chicago sometimes post on social media\nto express their aggression towards rival gangs and previous research has\ndemonstrated that a deep learning approach can predict aggression and loss in\nposts. To address the possibility of bias in this sensitive application, we\ndeveloped an approach to systematically interpret the state of the art model.\nWe found, surprisingly, that it frequently bases its predictions on stop words\nsuch as \"a\" or \"on\", an approach that could harm social media users who have no\naggressive intentions. To tackle this bias, domain experts annotated the\nrationales, highlighting words that explain why a tweet is labeled as\n\"aggression\". These new annotations enable us to quantitatively measure how\njustified the model predictions are, and build models that drastically reduce\nbias. Our study shows that in high stake scenarios, accuracy alone cannot\nguarantee a good system and we need new evaluation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11487v2",
            "date": "2019-08-30 00:08:06+00:00",
            "title": "Dialog Intent Induction with Deep Multi-View Clustering",
            "authors": [
                "Hugh Perkins",
                "Yi Yang"
            ],
            "abstract": "We introduce the dialog intent induction task and present a novel deep\nmulti-view clustering approach to tackle the problem. Dialog intent induction\naims at discovering user intents from user query utterances in human-human\nconversations such as dialogs between customer support agents and customers.\nMotivated by the intuition that a dialog intent is not only expressed in the\nuser query utterance but also captured in the rest of the dialog, we split a\nconversation into two independent views and exploit multi-view clustering\ntechniques for inducing the dialog intent. In particular, we propose\nalternating-view k-means (AV-KMEANS) for joint multi-view representation\nlearning and clustering analysis. The key innovation is that the instance-view\nrepresentations are updated iteratively by predicting the cluster assignment\nobtained from the alternative view, so that the multi-view representations of\nthe instances lead to similar cluster assignments. Experiments on two public\ndatasets show that AV-KMEANS can induce better dialog intent clusters than\nstate-of-the-art unsupervised representation learning methods and standard\nmulti-view clustering approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11511v4",
            "date": "2019-08-30 02:30:28+00:00",
            "title": "DCMN+: Dual Co-Matching Network for Multi-choice Reading Comprehension",
            "authors": [
                "Shuailiang Zhang",
                "Hai Zhao",
                "Yuwei Wu",
                "Zhuosheng Zhang",
                "Xi Zhou",
                "Xiang Zhou"
            ],
            "abstract": "Multi-choice reading comprehension is a challenging task to select an answer\nfrom a set of candidate options when given passage and question. Previous\napproaches usually only calculate question-aware passage representation and\nignore passage-aware question representation when modeling the relationship\nbetween passage and question, which obviously cannot take the best of\ninformation between passage and question. In this work, we propose dual\nco-matching network (DCMN) which models the relationship among passage,\nquestion and answer options bidirectionally. Besides, inspired by how human\nsolve multi-choice questions, we integrate two reading strategies into our\nmodel: (i) passage sentence selection that finds the most salient supporting\nsentences to answer the question, (ii) answer option interaction that encodes\nthe comparison information between answer options. DCMN integrated with the two\nstrategies (DCMN+) obtains state-of-the-art results on five multi-choice\nreading comprehension datasets which are from different domains: RACE,\nSemEval-2018 Task 11, ROCStories, COIN, MCTest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11536v1",
            "date": "2019-08-30 05:27:39+00:00",
            "title": "Learning to Infer Entities, Properties and their Relations from Clinical Conversations",
            "authors": [
                "Nan Du",
                "Mingqiu Wang",
                "Linh Tran",
                "Gang Li",
                "Izhak Shafran"
            ],
            "abstract": "Recently we proposed the Span Attribute Tagging (SAT) Model (Du et al., 2019)\nto infer clinical entities (e.g., symptoms) and their properties (e.g.,\nduration). It tackles the challenge of large label space and limited training\ndata using a hierarchical two-stage approach that identifies the span of\ninterest in a tagging step and assigns labels to the span in a classification\nstep.\n  We extend the SAT model to jointly infer not only entities and their\nproperties but also relations between them. Most relation extraction models\nrestrict inferring relations between tokens within a few neighboring sentences,\nmainly to avoid high computational complexity. In contrast, our proposed\nRelation-SAT (R-SAT) model is computationally efficient and can infer relations\nover the entire conversation, spanning an average duration of 10 minutes.\n  We evaluate our model on a corpus of clinical conversations. When the\nentities are given, the R-SAT outperforms baselines in identifying relations\nbetween symptoms and their properties by about 32% (0.82 vs 0.62 F-score) and\nby about 50% (0.60 vs 0.41 F-score) on medications and their properties. On the\nmore difficult task of jointly inferring entities and relations, the R-SAT\nmodel achieves a performance of 0.34 and 0.45 for symptoms and medications\nrespectively, which is significantly better than 0.18 and 0.35 for the baseline\nmodel. The contributions of different components of the model are quantified\nusing ablation analysis.",
            "journal ref": "Proc. Empirical Methods in Natural Language Processing, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11546v1",
            "date": "2019-08-30 06:15:35+00:00",
            "title": "Modeling Multi-Action Policy for Task-Oriented Dialogues",
            "authors": [
                "Lei Shu",
                "Hu Xu",
                "Bing Liu",
                "Piero Molino"
            ],
            "abstract": "Dialogue management (DM) plays a key role in the quality of the interaction\nwith the user in a task-oriented dialogue system. In most existing approaches,\nthe agent predicts only one DM policy action per turn. This significantly\nlimits the expressive power of the conversational agent and introduces unwanted\nturns of interactions that may challenge users' patience. Longer conversations\nalso lead to more errors and the system needs to be more robust to handle them.\nIn this paper, we compare the performance of several models on the task of\npredicting multiple acts for each turn. A novel policy model is proposed based\non a recurrent cell called gated Continue-Act-Slots (gCAS) that overcomes the\nlimitations of the existing models. Experimental results show that gCAS\noutperforms other approaches. The code is available at\nhttps://leishu02.github.io/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11561v1",
            "date": "2019-08-30 06:38:55+00:00",
            "title": "Detect Camouflaged Spam Content via StoneSkipping: Graph and Text Joint Embedding for Chinese Character Variation Representation",
            "authors": [
                "Zhuoren Jiang",
                "Zhe Gao",
                "Guoxiu He",
                "Yangyang Kang",
                "Changlong Sun",
                "Qiong Zhang",
                "Luo Si",
                "Xiaozhong Liu"
            ],
            "abstract": "The task of Chinese text spam detection is very challenging due to both glyph\nand phonetic variations of Chinese characters. This paper proposes a novel\nframework to jointly model Chinese variational, semantic, and contextualized\nrepresentations for Chinese text spam detection task. In particular, a\nVariation Family-enhanced Graph Embedding (VFGE) algorithm is designed based on\na Chinese character variation graph. The VFGE can learn both the graph\nembeddings of the Chinese characters (local) and the latent variation families\n(global). Furthermore, an enhanced bidirectional language model, with a\ncombination gate function and an aggregation learning function, is proposed to\nintegrate the graph and text information while capturing the sequential\ninformation. Extensive experiments have been conducted on both SMS and review\ndatasets, to show the proposed method outperforms a series of state-of-the-art\nmodels for Chinese spam detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11599v2",
            "date": "2019-08-30 08:58:11+00:00",
            "title": "Online influence, offline violence: Language Use on YouTube surrounding the 'Unite the Right' rally",
            "authors": [
                "Isabelle van der Vegt",
                "Maximilian Mozes",
                "Paul Gill",
                "Bennett Kleinberg"
            ],
            "abstract": "The media frequently describes the 2017 Charlottesville 'Unite the Right'\nrally as a turning point for the alt-right and white supremacist movements.\nSocial movement theory suggests that the media attention and public discourse\nconcerning the rally may have influenced the alt-right, but this has yet to be\nempirically tested. The current study investigates whether there are\ndifferences in language use between 7,142 alt-right and progressive YouTube\nchannels, in addition to measuring possible changes as a result of the rally.\nTo do so, we create structural topic models and measure bigram proportions in\nvideo transcripts, spanning eight weeks before to eight weeks after the rally.\nWe observe differences in topics between the two groups, with the 'alternative\ninfluencers' for example discussing topics related to race and free speech to\nan increasing and larger extent than progressive channels. We also observe\nstructural breakpoints in the use of bigrams at the time of the rally,\nsuggesting there are changes in language use within the two groups as a result\nof the rally. While most changes relate to mentions of the rally itself, the\nalternative group also shows an increase in promotion of their YouTube\nchannels. Results are discussed in light of social movement theory, followed by\na discussion of potential implications for understanding the alt-right and\ntheir language use on YouTube.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11610v1",
            "date": "2019-08-30 09:30:38+00:00",
            "title": "Cross-domain Aspect Category Transfer and Detection via Traceable Heterogeneous Graph Representation Learning",
            "authors": [
                "Zhuoren Jiang",
                "Jian Wang",
                "Lujun Zhao",
                "Changlong Sun",
                "Yao Lu",
                "Xiaozhong Liu"
            ],
            "abstract": "Aspect category detection is an essential task for sentiment analysis and\nopinion mining. However, the cost of categorical data labeling, e.g., label the\nreview aspect information for a large number of product domains, can be\ninevitable but unaffordable. In this study, we propose a novel problem,\ncross-domain aspect category transfer and detection, which faces three\nchallenges: various feature spaces, different data distributions, and diverse\noutput spaces. To address these problems, we propose an innovative solution,\nTraceable Heterogeneous Graph Representation Learning (THGRL). Unlike prior\ntext-based aspect detection works, THGRL explores latent domain aspect category\nconnections via massive user behavior information on a heterogeneous graph.\nMoreover, an innovative latent variable \"Walker Tracer\" is introduced to\ncharacterize the global semantic/aspect dependencies and capture the\ninformative vertexes on the random walk paths. By using THGRL, we project\ndifferent domains' feature spaces into a common one, while allowing data\ndistributions and output spaces stay differently. Experiment results show that\nthe proposed method outperforms a series of state-of-the-art baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11664v1",
            "date": "2019-08-30 11:40:14+00:00",
            "title": "Exploring Domain Shift in Extractive Text Summarization",
            "authors": [
                "Danqing Wang",
                "Pengfei Liu",
                "Ming Zhong",
                "Jie Fu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Although domain shift has been well explored in many NLP applications, it\nstill has received little attention in the domain of extractive text\nsummarization. As a result, the model is under-utilizing the nature of the\ntraining data due to ignoring the difference in the distribution of training\nsets and shows poor generalization on the unseen domain.\n  With the above limitation in mind, in this paper, we first extend the\nconventional definition of the domain from categories into data sources for the\ntext summarization task. Then we re-purpose a multi-domain summarization\ndataset and verify how the gap between different domains influences the\nperformance of neural summarization models.\n  Furthermore, we investigate four learning strategies and examine their\nabilities to deal with the domain shift problem.\n  Experimental results on three different settings show their different\ncharacteristics in our new testbed.\n  Our source code including \\textit{BERT-based}, \\textit{meta-learning} methods\nfor multi-domain summarization learning and the re-purposed dataset\n\\textsc{Multi-SUM} will be available on our project:\n\\url{http://pfliu.com/TransferSum/}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11723v1",
            "date": "2019-08-30 13:16:18+00:00",
            "title": "Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization",
            "authors": [
                "Taehee Jung",
                "Dongyeop Kang",
                "Lucas Mentch",
                "Eduard Hovy"
            ],
            "abstract": "Despite the recent developments on neural summarization systems, the\nunderlying logic behind the improvements from the systems and its\ncorpus-dependency remains largely unexplored. Position of sentences in the\noriginal text, for example, is a well known bias for news summarization.\nFollowing in the spirit of the claim that summarization is a combination of\nsub-functions, we define three sub-aspects of summarization: position,\nimportance, and diversity and conduct an extensive analysis of the biases of\neach sub-aspect with respect to the domain of nine different summarization\ncorpora (e.g., news, academic papers, meeting minutes, movie script, books,\nposts). We find that while position exhibits substantial bias in news articles,\nthis is not the case, for example, with academic papers and meeting minutes.\nFurthermore, our empirical study shows that different types of summarization\nsystems (e.g., neural-based) are composed of different degrees of the\nsub-aspects. Our study provides useful lessons regarding consideration of\nunderlying sub-aspects when collecting a new summarization dataset or\ndeveloping a new system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11771v2",
            "date": "2019-08-30 15:00:19+00:00",
            "title": "Encoders Help You Disambiguate Word Senses in Neural Machine Translation",
            "authors": [
                "Gongbo Tang",
                "Rico Sennrich",
                "Joakim Nivre"
            ],
            "abstract": "Neural machine translation (NMT) has achieved new state-of-the-art\nperformance in translating ambiguous words. However, it is still unclear which\ncomponent dominates the process of disambiguation. In this paper, we explore\nthe ability of NMT encoders and decoders to disambiguate word senses by\nevaluating hidden states and investigating the distributions of self-attention.\nWe train a classifier to predict whether a translation is correct given the\nrepresentation of an ambiguous noun. We find that encoder hidden states\noutperform word embeddings significantly which indicates that encoders\nadequately encode relevant information for disambiguation into hidden states.\nDecoders could provide further relevant information for disambiguation.\nMoreover, the attention weights and attention entropy show that self-attention\ncan detect ambiguous nouns and distribute more attention to the context. Note\nthat this is a revised version. The content related to decoder hidden states\nhas been updated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11787v1",
            "date": "2019-08-30 15:26:44+00:00",
            "title": "Answering Conversational Questions on Structured Data without Logical Forms",
            "authors": [
                "Thomas M\u00fcller",
                "Francesco Piccinno",
                "Massimo Nicosia",
                "Peter Shaw",
                "Yasemin Altun"
            ],
            "abstract": "We present a novel approach to answering sequential questions based on\nstructured objects such as knowledge bases or tables without using a logical\nform as an intermediate representation. We encode tables as graphs using a\ngraph neural network model based on the Transformer architecture. The answers\nare then selected from the encoded graph using a pointer network. This model is\nappropriate for processing conversations around structured data, where the\nattention mechanism that selects the answers to a question can also be used to\nresolve conversational references. We demonstrate the validity of this approach\nwith competitive results on the Sequential Question Answering (SQA) task (Iyyer\net al., 2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11813v1",
            "date": "2019-08-30 16:10:20+00:00",
            "title": "Multi-Task Learning with Language Modeling for Question Generation",
            "authors": [
                "Wenjie Zhou",
                "Minghua Zhang",
                "Yunfang Wu"
            ],
            "abstract": "This paper explores the task of answer-aware questions generation. Based on\nthe attention-based pointer generator model, we propose to incorporate an\nauxiliary task of language modeling to help question generation in a\nhierarchical multi-task learning structure. Our joint-learning model enables\nthe encoder to learn a better representation of the input sequence, which will\nguide the decoder to generate more coherent and fluent questions. On both SQuAD\nand MARCO datasets, our multi-task learning model boosts the performance,\nachieving state-of-the-art results. Moreover, human evaluation further proves\nthe high quality of our generated questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11828v1",
            "date": "2019-08-30 16:40:00+00:00",
            "title": "PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification",
            "authors": [
                "Yinfei Yang",
                "Yuan Zhang",
                "Chris Tar",
                "Jason Baldridge"
            ],
            "abstract": "Most existing work on adversarial data generation focuses on English. For\nexample, PAWS (Paraphrase Adversaries from Word Scrambling) consists of\nchallenging English paraphrase identification pairs from Wikipedia and Quora.\nWe remedy this gap with PAWS-X, a new dataset of 23,659 human translated PAWS\nevaluation pairs in six typologically distinct languages: French, Spanish,\nGerman, Chinese, Japanese, and Korean. We provide baseline numbers for three\nmodels with different capacity to capture non-local context and sentence\nstructure, and using different multilingual training and evaluation regimes.\nMultilingual BERT fine-tuned on PAWS English plus machine-translated data\nperforms the best, with a range of 83.1-90.8 accuracy across the non-English\nlanguages and an average accuracy gain of 23% over the next best model. PAWS-X\nshows the effectiveness of deep, multilingual pre-training while also leaving\nconsiderable headroom as a new challenge to drive multilingual research that\nbetter captures structure and contextual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11841v1",
            "date": "2019-08-30 17:12:32+00:00",
            "title": "CodeSwitch-Reddit: Exploration of Written Multilingual Discourse in Online Discussion Forums",
            "authors": [
                "Ella Rabinovich",
                "Masih Sultani",
                "Suzanne Stevenson"
            ],
            "abstract": "In contrast to many decades of research on oral code-switching, the study of\nwritten multilingual productions has only recently enjoyed a surge of interest.\nMany open questions remain regarding the sociolinguistic underpinnings of\nwritten code-switching, and progress has been limited by a lack of suitable\nresources. We introduce a novel, large, and diverse dataset of written\ncode-switched productions, curated from topical threads of multiple bilingual\ncommunities on the Reddit discussion platform, and explore questions that were\nmainly addressed in the context of spoken language thus far. We investigate\nwhether findings in oral code-switching concerning content and style, as well\nas speaker proficiency, are carried over into written code-switching in\ndiscussion forums. The released dataset can further facilitate a range of\nresearch and practical activities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11860v2",
            "date": "2019-08-30 17:44:30+00:00",
            "title": "Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification",
            "authors": [
                "Alexander Rietzler",
                "Sebastian Stabinger",
                "Paul Opitz",
                "Stefan Engl"
            ],
            "abstract": "Aspect-Target Sentiment Classification (ATSC) is a subtask of Aspect-Based\nSentiment Analysis (ABSA), which has many applications e.g. in e-commerce,\nwhere data and insights from reviews can be leveraged to create value for\nbusinesses and customers. Recently, deep transfer-learning methods have been\napplied successfully to a myriad of Natural Language Processing (NLP) tasks,\nincluding ATSC. Building on top of the prominent BERT language model, we\napproach ATSC using a two-step procedure: self-supervised domain-specific BERT\nlanguage model finetuning, followed by supervised task-specific finetuning. Our\nfindings on how to best exploit domain-specific language model finetuning\nenable us to produce new state-of-the-art performance on the SemEval 2014 Task\n4 restaurants dataset. In addition, to explore the real-world robustness of our\nmodels, we perform cross-domain evaluation. We show that a cross-domain adapted\nBERT language model performs significantly better than strong baseline models\nlike vanilla BERT-base and XLNet-base. Finally, we conduct a case study to\ninterpret model prediction errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00040v1",
            "date": "2019-08-30 19:07:56+00:00",
            "title": "Handling Syntactic Divergence in Low-resource Machine Translation",
            "authors": [
                "Chunting Zhou",
                "Xuezhe Ma",
                "Junjie Hu",
                "Graham Neubig"
            ],
            "abstract": "Despite impressive empirical successes of neural machine translation (NMT) on\nstandard benchmarks, limited parallel data impedes the application of NMT\nmodels to many language pairs. Data augmentation methods such as\nback-translation make it possible to use monolingual data to help alleviate\nthese issues, but back-translation itself fails in extreme low-resource\nscenarios, especially for syntactically divergent languages. In this paper, we\npropose a simple yet effective solution, whereby target-language sentences are\nre-ordered to match the order of the source and used as an additional source of\ntraining-time supervision. Experiments with simulated low-resource\nJapanese-to-English, and real low-resource Uyghur-to-English scenarios find\nsignificant improvements over other semi-supervised alternatives.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00080v2",
            "date": "2019-08-30 22:10:52+00:00",
            "title": "Sequential Learning of Convolutional Features for Effective Text Classification",
            "authors": [
                "Avinash Madasu",
                "Vijjini Anvesh Rao"
            ],
            "abstract": "Text classification has been one of the major problems in natural language\nprocessing. With the advent of deep learning, convolutional neural network\n(CNN) has been a popular solution to this task. However, CNNs which were first\nproposed for images, face many crucial challenges in the context of text\nprocessing, namely in their elementary blocks: convolution filters and max\npooling. These challenges have largely been overlooked by the most existing CNN\nmodels proposed for text classification. In this paper, we present an\nexperimental study on the fundamental blocks of CNNs in text categorization.\nBased on this critique, we propose Sequential Convolutional Attentive Recurrent\nNetwork (SCARN). The proposed SCARN model utilizes both the advantages of\nrecurrent and convolutional structures efficiently in comparison to previously\nproposed recurrent convolutional models. We test our model on different text\nclassification datasets across tasks like sentiment analysis and question\nclassification. Extensive experiments establish that SCARN outperforms other\nrecurrent convolutional architectures with significantly less parameters.\nFurthermore, SCARN achieves better performance compared to equally large\nvarious deep CNN and LSTM architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00091v1",
            "date": "2019-08-30 23:27:06+00:00",
            "title": "Automatically Inferring Gender Associations from Language",
            "authors": [
                "Serina Chang",
                "Kathleen McKeown"
            ],
            "abstract": "In this paper, we pose the question: do people talk about women and men in\ndifferent ways? We introduce two datasets and a novel integration of approaches\nfor automatically inferring gender associations from language, discovering\ncoherent word clusters, and labeling the clusters for the semantic concepts\nthey represent. The datasets allow us to compare how people write about women\nand men in two different settings - one set draws from celebrity news and the\nother from student reviews of computer science professors. We demonstrate that\nthere are large-scale differences in the ways that people talk about women and\nmen and that these differences vary across domains. Human evaluations show that\nour methods significantly outperform strong baselines.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00098v1",
            "date": "2019-08-31 00:33:32+00:00",
            "title": "(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas",
            "authors": [
                "Dongyeop Kang",
                "Varun Gangal",
                "Eduard Hovy"
            ],
            "abstract": "Stylistic variation in text needs to be studied with different aspects\nincluding the writer's personal traits, interpersonal relations, rhetoric, and\nmore. Despite recent attempts on computational modeling of the variation, the\nlack of parallel corpora of style language makes it difficult to systematically\ncontrol the stylistic change as well as evaluate such models. We release\nPASTEL, the parallel and annotated stylistic language dataset, that contains\n~41K parallel sentences (8.3K parallel stories) annotated across different\npersonas. Each persona has different styles in conjunction: gender, age,\ncountry, political view, education, ethnic, and time-of-writing. The dataset is\ncollected from human annotators with solid control of input denotation: not\nonly preserving original meaning between text, but promoting stylistic\ndiversity to annotators. We test the dataset on two interesting applications of\nstyle language, where PASTEL helps design appropriate experiment and\nevaluation. First, in predicting a target style (e.g., male or female in\ngender) given a text, multiple styles of PASTEL make other external style\nvariables controlled (or fixed), which is a more accurate experimental design.\nSecond, a simple supervised model with our parallel text outperforms the\nunsupervised models using nonparallel text in style transfer. Our dataset is\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00100v1",
            "date": "2019-08-31 00:39:12+00:00",
            "title": "Small and Practical BERT Models for Sequence Labeling",
            "authors": [
                "Henry Tsai",
                "Jason Riesa",
                "Melvin Johnson",
                "Naveen Arivazhagan",
                "Xin Li",
                "Amelia Archer"
            ],
            "abstract": "We propose a practical scheme to train a single multilingual sequence\nlabeling model that yields state of the art results and is small and fast\nenough to run on a single CPU. Starting from a public multilingual BERT\ncheckpoint, our final model is 6x smaller and 27x faster, and has higher\naccuracy than a state-of-the-art multilingual baseline. We show that our model\nespecially outperforms on low-resource languages, and works on codemixed input\ntext without being explicitly trained on codemixed examples. We showcase the\neffectiveness of our method by reporting on part-of-speech tagging and\nmorphological prediction on 70 treebanks and 48 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00107v1",
            "date": "2019-08-31 02:10:32+00:00",
            "title": "Behavior Gated Language Models",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Shao-Yen Tseng",
                "Panayiotis Georgiou",
                "Shrikanth Narayanan"
            ],
            "abstract": "Most current language modeling techniques only exploit co-occurrence,\nsemantic and syntactic information from the sequence of words. However, a range\nof information such as the state of the speaker and dynamics of the interaction\nmight be useful. In this work we derive motivation from psycholinguistics and\npropose the addition of behavioral information into the context of language\nmodeling. We propose the augmentation of language models with an additional\nmodule which analyzes the behavioral state of the current context. This\nbehavioral information is used to gate the outputs of the language model before\nthe final word prediction output. We show that the addition of behavioral\ncontext in language models achieves lower perplexities on behavior-rich\ndatasets. We also confirm the validity of the proposed models on a variety of\nmodel architectures and improve on previous state-of-the-art models with\ngeneric domain Penn Treebank Corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00111v1",
            "date": "2019-08-31 02:41:49+00:00",
            "title": "Quantity doesn't buy quality syntax with neural language models",
            "authors": [
                "Marten van Schijndel",
                "Aaron Mueller",
                "Tal Linzen"
            ],
            "abstract": "Recurrent neural networks can learn to predict upcoming words remarkably well\non average; in syntactically complex contexts, however, they often assign\nunexpectedly high probabilities to ungrammatical words. We investigate to what\nextent these shortcomings can be mitigated by increasing the size of the\nnetwork and the corpus on which it is trained. We find that gains from\nincreasing network size are minimal beyond a certain point. Likewise, expanding\nthe training corpus yields diminishing returns; we estimate that the training\ncorpus would need to be unrealistically large for the models to match human\nperformance. A comparison to GPT and BERT, Transformer-based models trained on\nbillions of words, reveals that these models perform even more poorly than our\nLSTMs in some constructions. Our results make the case for more data efficient\narchitectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00136v1",
            "date": "2019-08-31 05:45:20+00:00",
            "title": "Modeling Graph Structure in Transformer for Better AMR-to-Text Generation",
            "authors": [
                "Jie Zhu",
                "Junhui Li",
                "Muhua Zhu",
                "Longhua Qian",
                "Min Zhang",
                "Guodong Zhou"
            ],
            "abstract": "Recent studies on AMR-to-text generation often formalize the task as a\nsequence-to-sequence (seq2seq) learning problem by converting an Abstract\nMeaning Representation (AMR) graph into a word sequence. Graph structures are\nfurther modeled into the seq2seq framework in order to utilize the structural\ninformation in the AMR graphs. However, previous approaches only consider the\nrelations between directly connected concepts while ignoring the rich structure\nin AMR graphs. In this paper we eliminate such a strong limitation and propose\na novel structure-aware self-attention approach to better modeling the\nrelations between indirectly connected concepts in the state-of-the-art seq2seq\nmodel, i.e., the Transformer. In particular, a few different methods are\nexplored to learn structural representations between two concepts. Experimental\nresults on English AMR benchmark datasets show that our approach significantly\noutperforms the state of the art with 29.66 and 31.82 BLEU scores on LDC2015E86\nand LDC2017T10, respectively. To the best of our knowledge, these are the best\nresults achieved so far by supervised models on the benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00137v2",
            "date": "2019-08-31 06:02:11+00:00",
            "title": "EntEval: A Holistic Evaluation Benchmark for Entity Representations",
            "authors": [
                "Mingda Chen",
                "Zewei Chu",
                "Yang Chen",
                "Karl Stratos",
                "Kevin Gimpel"
            ],
            "abstract": "Rich entity representations are useful for a wide class of problems involving\nentities. Despite their importance, there is no standardized benchmark that\nevaluates the overall quality of entity representations. In this work, we\npropose EntEval: a test suite of diverse tasks that require nontrivial\nunderstanding of entities including entity typing, entity similarity, entity\nrelation prediction, and entity disambiguation. In addition, we develop\ntraining techniques for learning better entity representations by using natural\nhyperlink annotations in Wikipedia. We identify effective objectives for\nincorporating the contextual information in hyperlinks into state-of-the-art\npretrained language models and show that they improve strong baselines on\nmultiple EntEval tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00140v1",
            "date": "2019-08-31 06:11:31+00:00",
            "title": "Question-type Driven Question Generation",
            "authors": [
                "Wenjie Zhou",
                "Minghua Zhang",
                "Yunfang Wu"
            ],
            "abstract": "Question generation is a challenging task which aims to ask a question based\non an answer and relevant context. The existing works suffer from the\nmismatching between question type and answer, i.e. generating a question with\ntype $how$ while the answer is a personal name. We propose to automatically\npredict the question type based on the input answer and context. Then, the\nquestion type is fused into a seq2seq model to guide the question generation,\nso as to deal with the mismatching problem. We achieve significant improvement\non the accuracy of question type prediction and finally obtain state-of-the-art\nresults for question generation on both SQuAD and MARCO datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00142v2",
            "date": "2019-08-31 06:19:42+00:00",
            "title": "Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations",
            "authors": [
                "Mingda Chen",
                "Zewei Chu",
                "Kevin Gimpel"
            ],
            "abstract": "Prior work on pretrained sentence embeddings and benchmarks focus on the\ncapabilities of stand-alone sentences. We propose DiscoEval, a test suite of\ntasks to evaluate whether sentence representations include broader context\ninformation. We also propose a variety of training objectives that makes use of\nnatural annotations from Wikipedia to build sentence encoders capable of\nmodeling discourse. We benchmark sentence encoders pretrained with our proposed\ntraining objectives, as well as other popular pretrained sentence encoders on\nDiscoEval and other sentence evaluation tasks. Empirically, we show that these\ntraining objectives help to encode different aspects of information in document\nstructures. Moreover, BERT and ELMo demonstrate strong performances over\nDiscoEval with individual hidden layers showing different characteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00156v1",
            "date": "2019-08-31 07:24:48+00:00",
            "title": "NCLS: Neural Cross-Lingual Summarization",
            "authors": [
                "Junnan Zhu",
                "Qian Wang",
                "Yining Wang",
                "Yu Zhou",
                "Jiajun Zhang",
                "Shaonan Wang",
                "Chengqing Zong"
            ],
            "abstract": "Cross-lingual summarization (CLS) is the task to produce a summary in one\nparticular language for a source document in a different language. Existing\nmethods simply divide this task into two steps: summarization and translation,\nleading to the problem of error propagation. To handle that, we present an\nend-to-end CLS framework, which we refer to as Neural Cross-Lingual\nSummarization (NCLS), for the first time. Moreover, we propose to further\nimprove NCLS by incorporating two related tasks, monolingual summarization and\nmachine translation, into the training process of CLS under multi-task\nlearning. Due to the lack of supervised CLS data, we propose a round-trip\ntranslation strategy to acquire two high-quality large-scale CLS datasets based\non existing monolingual summarization datasets. Experimental results have shown\nthat our NCLS achieves remarkable improvement over traditional pipeline methods\non both English-to-Chinese and Chinese-to-English CLS human-corrected test\nsets. In addition, NCLS with multi-task learning can further significantly\nimprove the quality of generated summaries. We make our dataset and code\npublicly available here: http://www.nlpr.ia.ac.cn/cip/dataset.htm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00157v1",
            "date": "2019-08-31 07:35:36+00:00",
            "title": "Improving Back-Translation with Uncertainty-based Confidence Estimation",
            "authors": [
                "Shuo Wang",
                "Yang Liu",
                "Chao Wang",
                "Huanbo Luan",
                "Maosong Sun"
            ],
            "abstract": "While back-translation is simple and effective in exploiting abundant\nmonolingual corpora to improve low-resource neural machine translation (NMT),\nthe synthetic bilingual corpora generated by NMT models trained on limited\nauthentic bilingual data are inevitably noisy. In this work, we propose to\nquantify the confidence of NMT model predictions based on model uncertainty.\nWith word- and sentence-level confidence measures based on uncertainty, it is\npossible for back-translation to better cope with noise in synthetic bilingual\ncorpora. Experiments on Chinese-English and English-German translation tasks\nshow that uncertainty-based confidence estimation significantly improves the\nperformance of back-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00161v1",
            "date": "2019-08-31 07:42:11+00:00",
            "title": "Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach",
            "authors": [
                "Wenpeng Yin",
                "Jamaal Hay",
                "Dan Roth"
            ],
            "abstract": "Zero-shot text classification (0Shot-TC) is a challenging NLU problem to\nwhich little attention has been paid by the research community. 0Shot-TC aims\nto associate an appropriate label with a piece of text, irrespective of the\ntext domain and the aspect (e.g., topic, emotion, event, etc.) described by the\nlabel. And there are only a few articles studying 0Shot-TC, all focusing only\non topical categorization which, we argue, is just the tip of the iceberg in\n0Shot-TC. In addition, the chaotic experiments in literature make no uniform\ncomparison, which blurs the progress.\n  This work benchmarks the 0Shot-TC problem by providing unified datasets,\nstandardized evaluations, and state-of-the-art baselines. Our contributions\ninclude: i) The datasets we provide facilitate studying 0Shot-TC relative to\nconceptually different and diverse aspects: the ``topic'' aspect includes\n``sports'' and ``politics'' as labels; the ``emotion'' aspect includes ``joy''\nand ``anger''; the ``situation'' aspect includes ``medical assistance'' and\n``water shortage''. ii) We extend the existing evaluation setup\n(label-partially-unseen) -- given a dataset, train on some labels, test on all\nlabels -- to include a more challenging yet realistic evaluation\nlabel-fully-unseen 0Shot-TC (Chang et al., 2008), aiming at classifying text\nsnippets without seeing task specific training data at all. iii) We unify the\n0Shot-TC of diverse aspects within a textual entailment formulation and study\nit this way.\n  Code & Data: https://github.com/yinwenpeng/BenchmarkingZeroShot",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00175v1",
            "date": "2019-08-31 09:31:36+00:00",
            "title": "Joint Detection and Location of English Puns",
            "authors": [
                "Yanyan Zou",
                "Wei Lu"
            ],
            "abstract": "A pun is a form of wordplay for an intended humorous or rhetorical effect,\nwhere a word suggests two or more meanings by exploiting polysemy (homographic\npun) or phonological similarity to another word (heterographic pun). This paper\npresents an approach that addresses pun detection and pun location jointly from\na sequence labeling perspective. We employ a new tagging scheme such that the\nmodel is capable of performing such a joint task, where useful structural\ninformation can be properly captured. We show that our proposed model is\neffective in handling both homographic and heterographic puns. Empirical\nresults on the benchmark datasets demonstrate that our approach can achieve new\nstate-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00176v1",
            "date": "2019-08-31 09:33:04+00:00",
            "title": "Quantity Tagger: A Latent-Variable Sequence Labeling Approach to Solving Addition-Subtraction Word Problems",
            "authors": [
                "Yanyan Zou",
                "Wei Lu"
            ],
            "abstract": "An arithmetic word problem typically includes a textual description\ncontaining several constant quantities. The key to solving the problem is to\nreveal the underlying mathematical relations (such as addition and subtraction)\namong quantities, and then generate equations to find solutions. This work\npresents a novel approach, Quantity Tagger, that automatically discovers such\nhidden relations by tagging each quantity with a sign corresponding to one type\nof mathematical operation. For each quantity, we assume there exists a latent,\nvariable-sized quantity span surrounding the quantity token in the text, which\nconveys information useful for determining its sign. Empirical results show\nthat our method achieves 5 and 8 points of accuracy gains on two datasets\nrespectively, compared to prior approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00180v1",
            "date": "2019-08-31 09:58:57+00:00",
            "title": "Explicit Cross-lingual Pre-training for Unsupervised Machine Translation",
            "authors": [
                "Shuo Ren",
                "Yu Wu",
                "Shujie Liu",
                "Ming Zhou",
                "Shuai Ma"
            ],
            "abstract": "Pre-training has proven to be effective in unsupervised machine translation\ndue to its ability to model deep context information in cross-lingual\nscenarios. However, the cross-lingual information obtained from shared BPE\nspaces is inexplicit and limited. In this paper, we propose a novel\ncross-lingual pre-training method for unsupervised machine translation by\nincorporating explicit cross-lingual training signals. Specifically, we first\ncalculate cross-lingual n-gram embeddings and infer an n-gram translation table\nfrom them. With those n-gram translation pairs, we propose a new pre-training\nmodel called Cross-lingual Masked Language Model (CMLM), which randomly chooses\nsource n-grams in the input text stream and predicts their translation\ncandidates at each time step. Experiments show that our method can incorporate\nbeneficial cross-lingual information into pre-trained models. Taking\npre-trained CMLM models as the encoder and decoder, we significantly improve\nthe performance of unsupervised machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00187v1",
            "date": "2019-08-31 10:38:56+00:00",
            "title": "Deep Ordinal Regression for Pledge Specificity Prediction",
            "authors": [
                "Shivashankar Subramanian",
                "Trevor Cohn",
                "Timothy Baldwin"
            ],
            "abstract": "Many pledges are made in the course of an election campaign, forming\nimportant corpora for political analysis of campaign strategy and governmental\naccountability. At present, there are no publicly available annotated datasets\nof pledges, and most political analyses rely on manual analysis. In this paper\nwe collate a novel dataset of manifestos from eleven Australian federal\nelection cycles, with over 12,000 sentences annotated with specificity (e.g.,\nrhetorical vs.\\ detailed pledge) on a fine-grained scale. We propose deep\nordinal regression approaches for specificity prediction, under both supervised\nand semi-supervised settings, and provide empirical results demonstrating the\neffectiveness of the proposed techniques over several baseline approaches. We\nanalyze the utility of pledge specificity modeling across a spectrum of policy\nissues in performing ideology prediction, and further provide qualitative\nanalysis in terms of capturing party-specific issue salience across election\ncycles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00188v1",
            "date": "2019-08-31 10:43:06+00:00",
            "title": "Improving Multi-Head Attention with Capsule Networks",
            "authors": [
                "Shuhao Gu",
                "Yang Feng"
            ],
            "abstract": "Multi-head attention advances neural machine translation by working out\nmultiple versions of attention in different subspaces, but the neglect of\nsemantic overlapping between subspaces increases the difficulty of translation\nand consequently hinders the further improvement of translation performance. In\nthis paper, we employ capsule networks to comb the information from the\nmultiple heads of the attention so that similar information can be clustered\nand unique information can be reserved. To this end, we adopt two routing\nmechanisms of Dynamic Routing and EM Routing, to fulfill the clustering and\nseparating. We conducted experiments on Chinese-to-English and\nEnglish-to-German translation tasks and got consistent improvements over the\nstrong Transformer baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00204v2",
            "date": "2019-08-31 12:08:53+00:00",
            "title": "NEZHA: Neural Contextualized Representation for Chinese Language Understanding",
            "authors": [
                "Junqiu Wei",
                "Xiaozhe Ren",
                "Xiaoguang Li",
                "Wenyong Huang",
                "Yi Liao",
                "Yasheng Wang",
                "Jiashu Lin",
                "Xin Jiang",
                "Xiao Chen",
                "Qun Liu"
            ],
            "abstract": "The pre-trained language models have achieved great successes in various\nnatural language understanding (NLU) tasks due to its capacity to capture the\ndeep contextualized information in text by pre-training on large-scale corpora.\nIn this technical report, we present our practice of pre-training language\nmodels named NEZHA (NEural contextualiZed representation for CHinese lAnguage\nunderstanding) on Chinese corpora and finetuning for the Chinese NLU tasks. The\ncurrent version of NEZHA is based on BERT with a collection of proven\nimprovements, which include Functional Relative Positional Encoding as an\neffective positional encoding scheme, Whole Word Masking strategy, Mixed\nPrecision Training and the LAMB Optimizer in training the models. The\nexperimental results show that NEZHA achieves the state-of-the-art performances\nwhen finetuned on several representative Chinese tasks, including named entity\nrecognition (People's Daily NER), sentence matching (LCQMC), Chinese sentiment\nclassification (ChnSenti) and natural language inference (XNLI).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00228v1",
            "date": "2019-08-31 15:14:01+00:00",
            "title": "Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs",
            "authors": [
                "Fenia Christopoulou",
                "Makoto Miwa",
                "Sophia Ananiadou"
            ],
            "abstract": "Document-level relation extraction is a complex human process that requires\nlogical inference to extract relationships between named entities in text.\nExisting approaches use graph-based neural models with words as nodes and edges\nas relations between them, to encode relations across sentences. These models\nare node-based, i.e., they form pair representations based solely on the two\ntarget node representations. However, entity relations can be better expressed\nthrough unique edge representations formed as paths between nodes. We thus\npropose an edge-oriented graph neural model for document-level relation\nextraction. The model utilises different types of nodes and edges to create a\ndocument-level graph. An inference mechanism on the graph edges enables to\nlearn intra- and inter-sentence relations using multi-instance learning\ninternally. Experiments on two document-level biomedical datasets for\nchemical-disease and gene-disease associations show the usefulness of the\nproposed edge-oriented approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00301v1",
            "date": "2019-09-01 01:57:45+00:00",
            "title": "Phrase Grounding by Soft-Label Chain Conditional Random Field",
            "authors": [
                "Jiacheng Liu",
                "Julia Hockenmaier"
            ],
            "abstract": "The phrase grounding task aims to ground each entity mention in a given\ncaption of an image to a corresponding region in that image. Although there are\nclear dependencies between how different mentions of the same caption should be\ngrounded, previous structured prediction methods that aim to capture such\ndependencies need to resort to approximate inference or non-differentiable\nlosses. In this paper, we formulate phrase grounding as a sequence labeling\ntask where we treat candidate regions as potential labels, and use neural chain\nConditional Random Fields (CRFs) to model dependencies among regions for\nadjacent mentions. In contrast to standard sequence labeling tasks, the phrase\ngrounding task is defined such that there may be multiple correct candidate\nregions. To address this multiplicity of gold labels, we define so-called\nSoft-Label Chain CRFs, and present an algorithm that enables convenient\nend-to-end training. Our method establishes a new state-of-the-art on phrase\ngrounding on the Flickr30k Entities dataset. Analysis shows that our model\nbenefits both from the entity dependencies captured by the CRF and from the\nsoft-label training regime. Our code is available at\n\\url{github.com/liujch1998/SoftLabelCCRF}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00303v2",
            "date": "2019-09-01 02:13:12+00:00",
            "title": "Higher-order Comparisons of Sentence Encoder Representations",
            "authors": [
                "Mostafa Abdou",
                "Artur Kulmizev",
                "Felix Hill",
                "Daniel M. Low",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Representational Similarity Analysis (RSA) is a technique developed by\nneuroscientists for comparing activity patterns of different measurement\nmodalities (e.g., fMRI, electrophysiology, behavior). As a framework, RSA has\nseveral advantages over existing approaches to interpretation of language\nencoders based on probing or diagnostic classification: namely, it does not\nrequire large training samples, is not prone to overfitting, and it enables a\nmore transparent comparison between the representational geometries of\ndifferent models and modalities. We demonstrate the utility of RSA by\nestablishing a previously unknown correspondence between widely-employed\npretrained language encoders and human processing difficulty via eye-tracking\ndata, showcasing its potential in the interpretability toolbox for neural\nmodels",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00310v3",
            "date": "2019-09-01 02:48:43+00:00",
            "title": "Syntax-aware Multilingual Semantic Role Labeling",
            "authors": [
                "Shexia He",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "Recently, semantic role labeling (SRL) has earned a series of success with\neven higher performance improvements, which can be mainly attributed to\nsyntactic integration and enhanced word representation. However, most of these\nefforts focus on English, while SRL on multiple languages more than English has\nreceived relatively little attention so that is kept underdevelopment. Thus\nthis paper intends to fill the gap on multilingual SRL with special focus on\nthe impact of syntax and contextualized word representation. Unlike existing\nwork, we propose a novel method guided by syntactic rule to prune arguments,\nwhich enables us to integrate syntax into multilingual SRL model simply and\neffectively. We present a unified SRL model designed for multiple languages\ntogether with the proposed uniform syntax enhancement. Our model achieves new\nstate-of-the-art results on the CoNLL-2009 benchmarks of all seven languages.\nBesides, we pose a discussion on the syntactic role among different languages\nand verify the effectiveness of deep enhanced representation for multilingual\nSRL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00324v1",
            "date": "2019-09-01 05:22:30+00:00",
            "title": "A Novel Aspect-Guided Deep Transition Model for Aspect Based Sentiment Analysis",
            "authors": [
                "Yunlong Liang",
                "Fandong Meng",
                "Jinchao Zhang",
                "Jinan Xu",
                "Yufeng Chen",
                "Jie Zhou"
            ],
            "abstract": "Aspect based sentiment analysis (ABSA) aims to identify the sentiment\npolarity towards the given aspect in a sentence, while previous models\ntypically exploit an aspect-independent (weakly associative) encoder for\nsentence representation generation. In this paper, we propose a novel\nAspect-Guided Deep Transition model, named AGDT, which utilizes the given\naspect to guide the sentence encoding from scratch with the specially-designed\ndeep transition architecture. Furthermore, an aspect-oriented objective is\ndesigned to enforce AGDT to reconstruct the given aspect with the generated\nsentence representation. In doing so, our AGDT can accurately generate\naspect-specific sentence representation, and thus conduct more accurate\nsentiment predictions. Experimental results on multiple SemEval datasets\ndemonstrate the effectiveness of our proposed approach, which significantly\noutperforms the best reported results with the same setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00352v1",
            "date": "2019-09-01 08:22:38+00:00",
            "title": "Enhancing AMR-to-Text Generation with Dual Graph Representations",
            "authors": [
                "Leonardo F. R. Ribeiro",
                "Claire Gardent",
                "Iryna Gurevych"
            ],
            "abstract": "Generating text from graph-based data, such as Abstract Meaning\nRepresentation (AMR), is a challenging task due to the inherent difficulty in\nhow to properly encode the structure of a graph with labeled edges. To address\nthis difficulty, we propose a novel graph-to-sequence model that encodes\ndifferent but complementary perspectives of the structural information\ncontained in the AMR graph. The model learns parallel top-down and bottom-up\nrepresentations of nodes capturing contrasting views of the graph. We also\ninvestigate the use of different node message passing strategies, employing\ndifferent state-of-the-art graph encoders to compute node representations based\non incoming and outgoing perspectives. In our experiments, we demonstrate that\nthe dual graph representation leads to improvements in AMR-to-text generation,\nachieving state-of-the-art results on two AMR datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00369v1",
            "date": "2019-09-01 10:07:20+00:00",
            "title": "One Model to Learn Both: Zero Pronoun Prediction and Translation",
            "authors": [
                "Longyue Wang",
                "Zhaopeng Tu",
                "Xing Wang",
                "Shuming Shi"
            ],
            "abstract": "Zero pronouns (ZPs) are frequently omitted in pro-drop languages, but should\nbe recalled in non-pro-drop languages. This discourse phenomenon poses a\nsignificant challenge for machine translation (MT) when translating texts from\npro-drop to non-pro-drop languages. In this paper, we propose a unified and\ndiscourse-aware ZP translation approach for neural MT models. Specifically, we\njointly learn to predict and translate ZPs in an end-to-end manner, allowing\nboth components to interact with each other. In addition, we employ\nhierarchical neural networks to exploit discourse-level context, which is\nbeneficial for ZP prediction and thus translation. Experimental results on both\nChinese-English and Japanese-English data show that our approach significantly\nand accumulatively improves both translation performance and ZP prediction\naccuracy over not only baseline but also previous works using external ZP\nprediction models. Extensive analyses confirm that the performance improvement\ncomes from the alleviation of different kinds of errors especially caused by\nsubjective ZPs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00383v1",
            "date": "2019-09-01 11:34:32+00:00",
            "title": "Self-Attention with Structural Position Representations",
            "authors": [
                "Xing Wang",
                "Zhaopeng Tu",
                "Longyue Wang",
                "Shuming Shi"
            ],
            "abstract": "Although self-attention networks (SANs) have advanced the state-of-the-art on\nvarious NLP tasks, one criticism of SANs is their ability of encoding positions\nof input words (Shaw et al., 2018). In this work, we propose to augment SANs\nwith structural position representations to model the latent structure of the\ninput sentence, which is complementary to the standard sequential positional\nrepresentations. Specifically, we use dependency tree to represent the\ngrammatical structure of a sentence, and propose two strategies to encode the\npositional relationships among words in the dependency tree. Experimental\nresults on NIST Chinese-to-English and WMT14 English-to-German translation\ntasks show that the proposed approach consistently boosts performance over both\nthe absolute and relative sequential position representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00412v1",
            "date": "2019-09-01 14:48:04+00:00",
            "title": "You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLP",
            "authors": [
                "Marco Del Tredici",
                "Diego Marcheggiani",
                "Sabine Schulte im Walde",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "Information about individuals can help to better understand what they say,\nparticularly in social media where texts are short. Current approaches to\nmodelling social media users pay attention to their social connections, but\nexploit this information in a static way, treating all connections uniformly.\nThis ignores the fact, well known in sociolinguistics, that an individual may\nbe part of several communities which are not equally relevant in all\ncommunicative situations. We present a model based on Graph Attention Networks\nthat captures this observation. It dynamically explores the social graph of a\nuser, computes a user representation given the most relevant connections for a\ntarget task, and combines it with linguistic information to make a prediction.\nWe apply our model to three different tasks, evaluate it against alternative\nmodels, and analyse the results extensively, showing that it significantly\noutperforms other current methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00421v1",
            "date": "2019-09-01 15:47:03+00:00",
            "title": "What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues",
            "authors": [
                "Xintong Yu",
                "Hongming Zhang",
                "Yangqiu Song",
                "Yan Song",
                "Changshui Zhang"
            ],
            "abstract": "Grounding a pronoun to a visual object it refers to requires complex\nreasoning from various information sources, especially in conversational\nscenarios. For example, when people in a conversation talk about something all\nspeakers can see, they often directly use pronouns (e.g., it) to refer to it\nwithout previous introduction. This fact brings a huge challenge for modern\nnatural language understanding systems, particularly conventional context-based\npronoun coreference models. To tackle this challenge, in this paper, we\nformally define the task of visual-aware pronoun coreference resolution (PCR)\nand introduce VisPro, a large-scale dialogue PCR dataset, to investigate\nwhether and how the visual information can help resolve pronouns in dialogues.\nWe then propose a novel visual-aware PCR model, VisCoref, for this task and\nconduct comprehensive experiments and case studies on our dataset. Results\ndemonstrate the importance of the visual information in this PCR case and show\nthe effectiveness of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00429v1",
            "date": "2019-09-01 16:47:57+00:00",
            "title": "An Improved Neural Baseline for Temporal Relation Extraction",
            "authors": [
                "Qiang Ning",
                "Sanjay Subramanian",
                "Dan Roth"
            ],
            "abstract": "Determining temporal relations (e.g., before or after) between events has\nbeen a challenging natural language understanding task, partly due to the\ndifficulty to generate large amounts of high-quality training data.\nConsequently, neural approaches have not been widely used on it, or showed only\nmoderate improvements. This paper proposes a new neural system that achieves\nabout 10% absolute improvement in accuracy over the previous best system (25%\nerror reduction) on two benchmark datasets. The proposed system is trained on\nthe state-of-the-art MATRES dataset and applies contextualized word embeddings,\na Siamese encoder of a temporal common sense knowledge base, and global\ninference via integer linear programming (ILP). We suggest that the new\napproach could serve as a strong baseline for future research in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00437v1",
            "date": "2019-09-01 17:32:21+00:00",
            "title": "Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation",
            "authors": [
                "Aditya Siddhant",
                "Melvin Johnson",
                "Henry Tsai",
                "Naveen Arivazhagan",
                "Jason Riesa",
                "Ankur Bapna",
                "Orhan Firat",
                "Karthik Raman"
            ],
            "abstract": "The recently proposed massively multilingual neural machine translation (NMT)\nsystem has been shown to be capable of translating over 100 languages to and\nfrom English within a single model. Its improved translation performance on low\nresource languages hints at potential cross-lingual transfer capability for\ndownstream tasks. In this paper, we evaluate the cross-lingual effectiveness of\nrepresentations from the encoder of a massively multilingual NMT model on 5\ndownstream classification and sequence labeling tasks covering a diverse set of\nover 50 languages. We compare against a strong baseline, multilingual BERT\n(mBERT), in different cross-lingual transfer learning scenarios and show gains\nin zero-shot transfer in 4 out of these 5 tasks.",
            "journal ref": "AAAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00444v1",
            "date": "2019-09-01 18:37:32+00:00",
            "title": "A Discriminative Neural Model for Cross-Lingual Word Alignment",
            "authors": [
                "Elias Stengel-Eskin",
                "Tzu-Ray Su",
                "Matt Post",
                "Benjamin Van Durme"
            ],
            "abstract": "We introduce a novel discriminative word alignment model, which we integrate\ninto a Transformer-based machine translation model. In experiments based on a\nsmall number of labeled examples (~1.7K-5K sentences) we evaluate its\nperformance intrinsically on both English-Chinese and English-Arabic alignment,\nwhere we achieve major improvements over unsupervised baselines (11-27 F1). We\nevaluate the model extrinsically on data projection for Chinese NER, showing\nthat our alignments lead to higher performance when used to project NER tags\nfrom English to Chinese. Finally, we perform an ablation analysis and an\nannotation experiment that jointly support the utility and feasibility of\nfuture manual alignment elicitation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00502v1",
            "date": "2019-09-02 01:33:08+00:00",
            "title": "An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction",
            "authors": [
                "Shun Kiyono",
                "Jun Suzuki",
                "Masato Mita",
                "Tomoya Mizumoto",
                "Kentaro Inui"
            ],
            "abstract": "The incorporation of pseudo data in the training of grammatical error\ncorrection models has been one of the main factors in improving the performance\nof such models. However, consensus is lacking on experimental configurations,\nnamely, choosing how the pseudo data should be generated or used. In this\nstudy, these choices are investigated through extensive experiments, and\nstate-of-the-art performance is achieved on the CoNLL-2014 test set\n($F_{0.5}=65.0$) and the official test set of the BEA-2019 shared task\n($F_{0.5}=70.2$) without making any modifications to the model architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00504v2",
            "date": "2019-09-02 01:36:33+00:00",
            "title": "Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space",
            "authors": [
                "Kawin Ethayarajh"
            ],
            "abstract": "A notable property of word embeddings is that word relationships can exist as\nlinear substructures in the embedding space. For example, $\\textit{gender}$\ncorresponds to $\\vec{\\textit{woman}} - \\vec{\\textit{man}}$ and\n$\\vec{\\textit{queen}} - \\vec{\\textit{king}}$. This, in turn, allows word\nanalogies to be solved arithmetically: $\\vec{\\textit{king}} -\n\\vec{\\textit{man}} + \\vec{\\textit{woman}} \\approx \\vec{\\textit{queen}}$. This\nproperty is notable because it suggests that models trained on word embeddings\ncan easily learn such relationships as geometric translations. However, there\nis no evidence that models $\\textit{exclusively}$ represent relationships in\nthis manner. We document an alternative way in which downstream models might\nlearn these relationships: orthogonal and linear transformations. For example,\ngiven a translation vector for $\\textit{gender}$, we can find an orthogonal\nmatrix $R$, representing a rotation and reflection, such that\n$R(\\vec{\\textit{king}}) \\approx \\vec{\\textit{queen}}$ and\n$R(\\vec{\\textit{man}}) \\approx \\vec{\\textit{woman}}$. Analogical reasoning\nusing orthogonal transformations is almost as accurate as using vector\narithmetic; using linear transformations is more accurate than both. Our\nfindings suggest that these transformations can be as good a representation of\nword relationships as translation vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00512v1",
            "date": "2019-09-02 01:51:46+00:00",
            "title": "How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings",
            "authors": [
                "Kawin Ethayarajh"
            ],
            "abstract": "Replacing static word embeddings with contextualized word representations has\nyielded significant improvements on many NLP tasks. However, just how\ncontextual are the contextualized representations produced by models such as\nELMo and BERT? Are there infinitely many context-specific representations for\neach word, or are words essentially assigned one of a finite number of\nword-sense representations? For one, we find that the contextualized\nrepresentations of all words are not isotropic in any layer of the\ncontextualizing model. While representations of the same word in different\ncontexts still have a greater cosine similarity than those of two different\nwords, this self-similarity is much lower in upper layers. This suggests that\nupper layers of contextualizing models produce more context-specific\nrepresentations, much like how upper layers of LSTMs produce more task-specific\nrepresentations. In all layers of ELMo, BERT, and GPT-2, on average, less than\n5% of the variance in a word's contextualized representations can be explained\nby a static embedding for that word, providing some justification for the\nsuccess of contextualized representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00516v1",
            "date": "2019-09-02 02:27:19+00:00",
            "title": "Beyond The Wall Street Journal: Anchoring and Comparing Discourse Signals across Genres",
            "authors": [
                "Yang Liu"
            ],
            "abstract": "Recent research on discourse relations has found that they are cued not only\nby discourse markers (DMs) but also by other textual signals and that signaling\ninformation is indicative of genres. While several corpora exist with discourse\nrelation signaling information such as the Penn Discourse Treebank (PDTB,\nPrasad et al. 2008) and the Rhetorical Structure Theory Signalling Corpus\n(RST-SC, Das and Taboada 2018), they both annotate the Wall Street Journal\n(WSJ) section of the Penn Treebank (PTB, Marcus et al. 1993), which is limited\nto the news domain. Thus, this paper adapts the signal identification and\nanchoring scheme (Liu and Zeldes, 2019) to three more genres, examines the\ndistribution of signaling devices across relations and genres, and provides a\ntaxonomy of indicative signals found in this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00522v1",
            "date": "2019-09-02 03:14:17+00:00",
            "title": "All Roads Lead to UD: Converting Stanford and Penn Parses to English Universal Dependencies with Multilayer Annotations",
            "authors": [
                "Siyao Peng",
                "Amir Zeldes"
            ],
            "abstract": "We describe and evaluate different approaches to the conversion of gold\nstandard corpus data from Stanford Typed Dependencies (SD) and Penn-style\nconstituent trees to the latest English Universal Dependencies representation\n(UD 2.2). Our results indicate that pure SD to UD conversion is highly accurate\nacross multiple genres, resulting in around 1.5% errors, but can be improved\nfurther to fewer than 0.5% errors given access to annotations beyond the pure\nsyntax tree, such as entity types and coreference resolution, which are\nnecessary for correct generation of several UD relations. We show that\nconstituent-based conversion using CoreNLP (with automatic NER) performs\nsubstantially worse in all genres, including when using gold constituent trees,\nprimarily due to underspecification of phrasal grammatical functions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00531v1",
            "date": "2019-09-02 04:04:18+00:00",
            "title": "Improving Context-aware Neural Machine Translation with Target-side Context",
            "authors": [
                "Hayahide Yamagishi",
                "Mamoru Komachi"
            ],
            "abstract": "In recent years, several studies on neural machine translation (NMT) have\nattempted to use document-level context by using a multi-encoder and two\nattention mechanisms to read the current and previous sentences to incorporate\nthe context of the previous sentences. These studies concluded that the\ntarget-side context is less useful than the source-side context. However, we\nconsidered that the reason why the target-side context is less useful lies in\nthe architecture used to model these contexts.\n  Therefore, in this study, we investigate how the target-side context can\nimprove context-aware neural machine translation. We propose a weight sharing\nmethod wherein NMT saves decoder states and calculates an attention vector\nusing the saved states when translating a current sentence. Our experiments\nshow that the target-side context is also useful if we plug it into NMT as the\ndecoder state when translating a previous sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00542v1",
            "date": "2019-09-02 04:39:26+00:00",
            "title": "Classification Betters Regression in Query-based Multi-document Summarisation Techniques for Question Answering: Macquarie University at BioASQ7b",
            "authors": [
                "Diego Molla",
                "Christopher Jones"
            ],
            "abstract": "Task B Phase B of the 2019 BioASQ challenge focuses on biomedical question\nanswering. Macquarie University's participation applies query-based\nmulti-document extractive summarisation techniques to generate a multi-sentence\nanswer given the question and the set of relevant snippets. In past\nparticipation we explored the use of regression approaches using deep learning\narchitectures and a simple policy gradient architecture. For the 2019 challenge\nwe experiment with the use of classification approaches with and without\nreinforcement learning. In addition, we conduct a correlation analysis between\nvarious ROUGE metrics and the BioASQ human evaluation scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00556v1",
            "date": "2019-09-02 05:55:36+00:00",
            "title": "Phrase-Level Class based Language Model for Mandarin Smart Speaker Query Recognition",
            "authors": [
                "Yiheng Huang",
                "Liqiang He",
                "Lei Han",
                "Guangsen Wang",
                "Dan Su"
            ],
            "abstract": "The success of speech assistants requires precise recognition of a number of\nentities on particular contexts. A common solution is to train a class-based\nn-gram language model and then expand the classes into specific words or\nphrases. However, when the class has a huge list, e.g., more than 20 million\nsongs, a fully expansion will cause memory explosion. Worse still, the list\nitems in the class need to be updated frequently, which requires a dynamic\nmodel updating technique. In this work, we propose to train pruned language\nmodels for the word classes to replace the slots in the root n-gram. We further\npropose to use a novel technique, named Difference Language Model (DLM), to\ncorrect the bias from the pruned language models. Once the decoding graph is\nbuilt, we only need to recalculate the DLM when the entities in word classes\nare updated. Results show that the proposed method consistently and\nsignificantly outperforms the conventional approaches on all datasets, esp. for\nlarge lists, which the conventional approaches cannot handle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00564v2",
            "date": "2019-09-02 06:55:16+00:00",
            "title": "Enhancing Context Modeling with a Query-Guided Capsule Network for Document-level Translation",
            "authors": [
                "Zhengxin Yang",
                "Jinchao Zhang",
                "Fandong Meng",
                "Shuhao Gu",
                "Yang Feng",
                "Jie Zhou"
            ],
            "abstract": "Context modeling is essential to generate coherent and consistent translation\nfor Document-level Neural Machine Translations. The widely used method for\ndocument-level translation usually compresses the context information into a\nrepresentation via hierarchical attention networks. However, this method\nneither considers the relationship between context words nor distinguishes the\nroles of context words. To address this problem, we propose a query-guided\ncapsule networks to cluster context information into different perspectives\nfrom which the target translation may concern. Experiment results show that our\nmethod can significantly outperform strong baselines on multiple data sets of\ndifferent domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00574v2",
            "date": "2019-09-02 07:16:57+00:00",
            "title": "A Sketch-Based System for Semantic Parsing",
            "authors": [
                "Zechang Li",
                "Yuxuan Lai",
                "Yuxi Xie",
                "Yansong Feng",
                "Dongyan Zhao"
            ],
            "abstract": "This paper presents our semantic parsing system for the evaluation task of\nopen domain semantic parsing in NLPCC 2019. Many previous works formulate\nsemantic parsing as a sequence-to-sequence(seq2seq) problem. Instead, we treat\nthe task as a sketch-based problem in a coarse-to-fine(coarse2fine) fashion.\nThe sketch is a high-level structure of the logical form exclusive of low-level\ndetails such as entities and predicates. In this way, we are able to optimize\neach part individually. Specifically, we decompose the process into three\nstages: the sketch classification determines the high-level structure while the\nentity labeling and the matching network fill in missing details. Moreover, we\nadopt the seq2seq method to evaluate logical form candidates from an overall\nperspective. The co-occurrence relationship between predicates and entities\ncontribute to the reranking as well. Our submitted system achieves the exactly\nmatching accuracy of 82.53% on full test set and 47.83% on hard test subset,\nwhich is the 3rd place in NLPCC 2019 Shared Task 2. After optimizations for\nparameters, network structure and sampling, the accuracy reaches 84.47% on full\ntest set and 63.08% on hard test subset(Our code and data are available at\nhttps://github.com/zechagl/NLPCC2019-Semantic-Parsing).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00578v1",
            "date": "2019-09-02 07:30:53+00:00",
            "title": "SumQE: a BERT-based Summary Quality Estimation Model",
            "authors": [
                "Stratos Xenouleas",
                "Prodromos Malakasiotis",
                "Marianna Apidianaki",
                "Ion Androutsopoulos"
            ],
            "abstract": "We propose SumQE, a novel Quality Estimation model for summarization based on\nBERT. The model addresses linguistic quality aspects that are only indirectly\ncaptured by content-based approaches to summary evaluation, without involving\ncomparison with human references. SumQE achieves very high correlations with\nhuman ratings, outperforming simpler models addressing these linguistic\naspects. Predictions of the SumQE model can be used for system development, and\nto inform users of the quality of automatically produced summaries and other\ntypes of generated text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00599v1",
            "date": "2019-09-02 08:40:41+00:00",
            "title": "Subword Language Model for Query Auto-Completion",
            "authors": [
                "Gyuwan Kim"
            ],
            "abstract": "Current neural query auto-completion (QAC) systems rely on character-level\nlanguage models, but they slow down when queries are long. We present how to\nutilize subword language models for the fast and accurate generation of query\ncompletion candidates. Representing queries with subwords shorten a decoding\nlength significantly. To deal with issues coming from introducing subword\nlanguage model, we develop a retrace algorithm and a reranking method by\napproximate marginalization. As a result, our model achieves up to 2.5 times\nfaster while maintaining a similar quality of generated results compared to the\ncharacter-level baseline. Also, we propose a new evaluation metric, mean\nrecoverable length (MRL), measuring how many upcoming characters the model\ncould complete correctly. It provides more explicit meaning and eliminates the\nneed for prefix length sampling for existing rank-based metrics. Moreover, we\nperformed a comprehensive analysis with ablation study to figure out the\nimportance of each component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00615v1",
            "date": "2019-09-02 09:16:06+00:00",
            "title": "Enriching Medcial Terminology Knowledge Bases via Pre-trained Language Model and Graph Convolutional Network",
            "authors": [
                "Jiaying Zhang",
                "Zhixing Zhang",
                "Huanhuan Zhang",
                "Zhiyuan Ma",
                "Yangming Zhou",
                "Ping He"
            ],
            "abstract": "Enriching existing medical terminology knowledge bases (KBs) is an important\nand never-ending work for clinical research because new terminology alias may\nbe continually added and standard terminologies may be newly renamed. In this\npaper, we propose a novel automatic terminology enriching approach to\nsupplement a set of terminologies to KBs. Specifically, terminology and entity\ncharacters are first fed into pre-trained language model to obtain semantic\nembedding. The pre-trained model is used again to initialize the terminology\nand entity representations, then they are further embedded through graph\nconvolutional network to gain structure embedding. Afterwards, both semantic\nand structure embeddings are combined to measure the relevancy between the\nterminology and the entity. Finally, the optimal alignment is achieved based on\nthe order of relevancy between the terminology and all the entities in the KB.\nExperimental results on clinical indicator terminology KB, collected from 38\ntop-class hospitals of Shanghai Hospital Development Center, show that our\nproposed approach outperforms baseline methods and can effectively enrich the\nKB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00692v1",
            "date": "2019-09-02 12:41:45+00:00",
            "title": "Story-oriented Image Selection and Placement",
            "authors": [
                "Sreyasi Nag Chowdhury",
                "Simon Razniewski",
                "Gerhard Weikum"
            ],
            "abstract": "Multimodal contents have become commonplace on the Internet today, manifested\nas news articles, social media posts, and personal or business blog posts.\nAmong the various kinds of media (images, videos, graphics, icons, audio) used\nin such multimodal stories, images are the most popular. The selection of\nimages from a collection - either author's personal photo album, or web\nrepositories - and their meticulous placement within a text, builds a succinct\nmultimodal commentary for digital consumption. In this paper we present a\nsystem that automates the process of selecting relevant images for a story and\nplacing them at contextual paragraphs within the story for a multimodal\nnarration. We leverage automatic object recognition, user-provided tags, and\ncommonsense knowledge, and use an unsupervised combinatorial optimization to\nsolve the selection and placement problems seamlessly as a single unit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00694v2",
            "date": "2019-09-02 12:46:26+00:00",
            "title": "Minimally Supervised Learning of Affective Events Using Discourse Relations",
            "authors": [
                "Jun Saito",
                "Yugo Murawaki",
                "Sadao Kurohashi"
            ],
            "abstract": "Recognizing affective events that trigger positive or negative sentiment has\na wide range of natural language processing applications but remains a\nchallenging problem mainly because the polarity of an event is not necessarily\npredictable from its constituent words. In this paper, we propose to propagate\naffective polarity using discourse relations. Our method is simple and only\nrequires a very small seed lexicon and a large raw corpus. Our experiments\nusing Japanese data show that our method learns affective events effectively\nwithout manually labeled data. It also improves supervised learning results\nwhen labeled data are small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00734v1",
            "date": "2019-09-02 14:29:36+00:00",
            "title": "Sentence-Level Content Planning and Style Specification for Neural Text Generation",
            "authors": [
                "Xinyu Hua",
                "Lu Wang"
            ],
            "abstract": "Building effective text generation systems requires three critical\ncomponents: content selection, text planning, and surface realization, and\ntraditionally they are tackled as separate problems. Recent all-in-one style\nneural generation models have made impressive progress, yet they often produce\noutputs that are incoherent and unfaithful to the input. To address these\nissues, we present an end-to-end trained two-step generation model, where a\nsentence-level content planner first decides on the keyphrases to cover as well\nas a desired language style, followed by a surface realization decoder that\ngenerates relevant and coherent text. For experiments, we consider three tasks\nfrom domains with diverse topics and varying language styles: persuasive\nargument construction from Reddit, paragraph generation for normal and simple\nversions of Wikipedia, and abstract generation for scientific articles.\nAutomatic evaluation shows that our system can significantly outperform\ncompetitive comparisons. Human judges further rate our system generated text as\nmore fluent and correct, compared to the generations by its variants that do\nnot consider language style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00786v2",
            "date": "2019-09-02 16:24:57+00:00",
            "title": "Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions",
            "authors": [
                "Rui Zhang",
                "Tao Yu",
                "He Yang Er",
                "Sungrok Shim",
                "Eric Xue",
                "Xi Victoria Lin",
                "Tianze Shi",
                "Caiming Xiong",
                "Richard Socher",
                "Dragomir Radev"
            ],
            "abstract": "We focus on the cross-domain context-dependent text-to-SQL generation task.\nBased on the observation that adjacent natural language questions are often\nlinguistically dependent and their corresponding SQL queries tend to overlap,\nwe utilize the interaction history by editing the previous predicted query to\nimprove the generation quality. Our editing mechanism views SQL as sequences\nand reuses generation results at the token level in a simple manner. It is\nflexible to change individual tokens and robust to error propagation.\nFurthermore, to deal with complex table structures in different domains, we\nemploy an utterance-table encoder and a table-aware decoder to incorporate the\ncontext of the user utterance and the table schema. We evaluate our approach on\nthe SParC dataset and demonstrate the benefit of editing compared with the\nstate-of-the-art baselines which generate SQL from scratch. Our code is\navailable at https://github.com/ryanzhumich/sparc_atis_pytorch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00867v1",
            "date": "2019-09-02 21:04:39+00:00",
            "title": "Investigating the Relationship between Multi-Party Linguistic Entrainment, Team Characteristics, and the Perception of Team Social Outcomes",
            "authors": [
                "Mingzhi Yu",
                "Diane Litman",
                "Susannah Paletz"
            ],
            "abstract": "Multi-party linguistic entrainment refers to the phenomenon that speakers\ntend to speak more similarly during conversation. We first developed new\nmeasures of multi-party entrainment on features describing linguistic style,\nand then examined the relationship between entrainment and team characteristics\nin terms of gender composition, team size, and diversity. Next, we predicted\nthe perception of team social outcomes using multi-party linguistic entrainment\nand team characteristics with a hierarchical regression model. We found that\nteams with greater gender diversity had higher minimum convergence than teams\nwith less gender diversity. Entrainment contributed significantly to predicting\nperceived team social outcomes both alone and controlling for team\ncharacteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00876v1",
            "date": "2019-09-02 21:50:52+00:00",
            "title": "Identifying Personality Traits Using Overlap Dynamics in Multiparty Dialogue",
            "authors": [
                "Mingzhi Yu",
                "Emer Gilmartin",
                "Diane Litman"
            ],
            "abstract": "Research on human spoken language has shown that speech plays an important\nrole in identifying speaker personality traits. In this work, we propose an\napproach for identifying speaker personality traits using overlap dynamics in\nmultiparty spoken dialogues. We first define a set of novel features\nrepresenting the overlap dynamics of each speaker. We then investigate the\nimpact of speaker personality traits on these features using ANOVA tests. We\nfind that features of overlap dynamics significantly vary for speakers with\ndifferent levels of both Extraversion and Conscientiousness. Finally, we find\nthat classifiers using only overlap dynamics features outperform random\nguessing in identifying Extraversion and Agreeableness, and that the\nimprovements are statistically significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00930v1",
            "date": "2019-09-03 02:59:21+00:00",
            "title": "Combining Spans into Entities: A Neural Two-Stage Approach for Recognizing Discontiguous Entities",
            "authors": [
                "Bailin Wang",
                "Wei Lu"
            ],
            "abstract": "In medical documents, it is possible that an entity of interest not only\ncontains a discontiguous sequence of words but also overlaps with another\nentity. Entities of such structures are intrinsically hard to recognize due to\nthe large space of possible entity combinations. In this work, we propose a\nneural two-stage approach to recognize discontiguous and overlapping entities\nby decomposing this problem into two subtasks: 1) it first detects all the\noverlapping spans that either form entities on their own or present as segments\nof discontiguous entities, based on the representation of segmental hypergraph,\n2) next it learns to combine these segments into discontiguous entities with a\nclassifier, which filters out other incorrect combinations of segments. Two\nneural components are designed for these subtasks respectively and they are\nlearned jointly using a shared encoder for text. Our model achieves the\nstate-of-the-art performance in a standard dataset, even in the absence of\nexternal features that previous methods used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00945v1",
            "date": "2019-09-03 04:16:28+00:00",
            "title": "\"Can you say more about the location?\" The Development of a Pedagogical Reference Resolution Agent",
            "authors": [
                "Maike Paetzel",
                "Ramesh Manuvinakurike"
            ],
            "abstract": "In an increasingly globalized world, geographic literacy is crucial. In this\npaper, we present a collaborative two-player game to improve people's ability\nto locate countries on the world map. We discuss two implementations of the\ngame: First, we created a web-based version which can be played with the\nremote-controlled agent Nellie. With the knowledge we gained from a large\nonline data collection, we re-implemented the game so it can be played\nface-to-face with the Furhat robot Neil. Our analysis shows that participants\nfound the game not just engaging to play, they also believe they gained lasting\nknowledge about the world map.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00964v2",
            "date": "2019-09-03 06:11:44+00:00",
            "title": "Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks",
            "authors": [
                "Haoyang Huang",
                "Yaobo Liang",
                "Nan Duan",
                "Ming Gong",
                "Linjun Shou",
                "Daxin Jiang",
                "Ming Zhou"
            ],
            "abstract": "We present Unicoder, a universal language encoder that is insensitive to\ndifferent languages. Given an arbitrary NLP task, a model can be trained with\nUnicoder using training data in one language and directly applied to inputs of\nthe same task in other languages. Comparing to similar efforts such as\nMultilingual BERT and XLM, three new cross-lingual pre-training tasks are\nproposed, including cross-lingual word recovery, cross-lingual paraphrase\nclassification and cross-lingual masked language model. These tasks help\nUnicoder learn the mappings among different languages from more perspectives.\nWe also find that doing fine-tuning on multiple languages together can bring\nfurther improvement. Experiments are performed on two tasks: cross-lingual\nnatural language inference (XNLI) and cross-lingual question answering (XQA),\nwhere XLM is our baseline. On XNLI, 1.8% averaged accuracy improvement (on 15\nlanguages) is obtained. On XQA, which is a new cross-lingual dataset built by\nus, 5.5% averaged accuracy improvement (on French and German) is obtained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01007v1",
            "date": "2019-09-03 09:00:44+00:00",
            "title": "Automatic Argument Quality Assessment -- New Datasets and Methods",
            "authors": [
                "Assaf Toledo",
                "Shai Gretz",
                "Edo Cohen-Karlik",
                "Roni Friedman",
                "Elad Venezian",
                "Dan Lahav",
                "Michal Jacovi",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "We explore the task of automatic assessment of argument quality. To that end,\nwe actively collected 6.3k arguments, more than a factor of five compared to\npreviously examined data. Each argument was explicitly and carefully annotated\nfor its quality. In addition, 14k pairs of arguments were annotated\nindependently, identifying the higher quality argument in each pair. In spite\nof the inherent subjective nature of the task, both annotation schemes led to\nsurprisingly consistent results. We release the labeled datasets to the\ncommunity. Furthermore, we suggest neural methods based on a recently released\nlanguage model, for argument ranking as well as for argument-pair\nclassification. In the former task, our results are comparable to\nstate-of-the-art; in the latter task our results significantly outperform\nearlier methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01013v1",
            "date": "2019-09-03 09:22:11+00:00",
            "title": "Duality Regularization for Unsupervised Bilingual Lexicon Induction",
            "authors": [
                "Xuefeng Bai",
                "Yue Zhang",
                "Hailong Cao",
                "Tiejun Zhao"
            ],
            "abstract": "Unsupervised bilingual lexicon induction naturally exhibits duality, which\nresults from symmetry in back-translation. For example, EN-IT and IT-EN\ninduction can be mutually primal and dual problems. Current state-of-the-art\nmethods, however, consider the two tasks independently. In this paper, we\npropose to train primal and dual models jointly, using regularizers to\nencourage consistency in back translation cycles. Experiments across 6 language\npairs show that the proposed method significantly outperforms competitive\nbaselines, obtaining the best-published results on a standard benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01059v3",
            "date": "2019-09-03 10:58:01+00:00",
            "title": "Attention-based Pairwise Multi-Perspective Convolutional Neural Network for Answer Selection in Question Answering",
            "authors": [
                "Jamshid Mozafari",
                "Mohammad Ali Nematbakhsh",
                "Afsaneh Fatemi"
            ],
            "abstract": "Over the past few years, question answering and information retrieval systems\nhave become widely used. These systems attempt to find the answer of the asked\nquestions from raw text sources. A component of these systems is Answer\nSelection which selects the most relevant from candidate answers. Syntactic\nsimilarities were mostly used to compute the similarity, but in recent works,\ndeep neural networks have been used, making a significant improvement in this\nfield. In this research, a model is proposed to select the most relevant\nanswers to the factoid question from the candidate answers. The proposed model\nranks the candidate answers in terms of semantic and syntactic similarity to\nthe question, using convolutional neural networks. In this research, Attention\nmechanism and Sparse feature vector use the context-sensitive interactions\nbetween questions and answer sentence. Wide convolution increases the\nimportance of the interrogative word. Pairwise ranking is used to learn\ndifferentiable representations to distinguish positive and negative answers.\nOur model indicates strong performance on the TrecQA Raw beating previous\nstate-of-the-art systems by 1.4% in MAP and 1.1% in MRR while using the\nbenefits of no additional syntactic parsers and external tools. The results\nshow that using context-sensitive interactions between question and answer\nsentences can help to find the correct answer more accurately.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01065v1",
            "date": "2019-09-03 11:09:12+00:00",
            "title": "Modeling Named Entity Embedding Distribution into Hypersphere",
            "authors": [
                "Zhuosheng Zhang",
                "Bingjie Tang",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "This work models named entity distribution from a way of visualizing\ntopological structure of embedding space, so that we make an assumption that\nmost, if not all, named entities (NEs) for a language tend to aggregate\ntogether to be accommodated by a specific hypersphere in embedding space. Thus\nwe present a novel open definition for NE which alleviates the obvious drawback\nin previous closed NE definition with a limited NE dictionary. Then, we show\ntwo applications with introducing the proposed named entity hypersphere model.\nFirst, using a generative adversarial neural network to learn a transformation\nmatrix of two embedding spaces, which results in a convenient determination of\nnamed entity distribution in the target language, indicating the potential of\nfast named entity discovery only using isomorphic relation between embedding\nspaces. Second, the named entity hypersphere model is directly integrated with\nvarious named entity recognition models over sentences to achieve\nstate-of-the-art results. Only assuming that embeddings are available, we show\na prior knowledge free approach on effective named entity distribution\ndepiction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01066v2",
            "date": "2019-09-03 11:11:08+00:00",
            "title": "Language Models as Knowledge Bases?",
            "authors": [
                "Fabio Petroni",
                "Tim Rockt\u00e4schel",
                "Patrick Lewis",
                "Anton Bakhtin",
                "Yuxiang Wu",
                "Alexander H. Miller",
                "Sebastian Riedel"
            ],
            "abstract": "Recent progress in pretraining language models on large textual corpora led\nto a surge of improvements for downstream NLP tasks. Whilst learning linguistic\nknowledge, these models may also be storing relational knowledge present in the\ntraining data, and may be able to answer queries structured as\n\"fill-in-the-blank\" cloze statements. Language models have many advantages over\nstructured knowledge bases: they require no schema engineering, allow\npractitioners to query about an open class of relations, are easy to extend to\nmore data, and require no human supervision to train. We present an in-depth\nanalysis of the relational knowledge already present (without fine-tuning) in a\nwide range of state-of-the-art pretrained language models. We find that (i)\nwithout fine-tuning, BERT contains relational knowledge competitive with\ntraditional NLP methods that have some access to oracle knowledge, (ii) BERT\nalso does remarkably well on open-domain question answering against a\nsupervised baseline, and (iii) certain types of factual knowledge are learned\nmuch more readily than others by standard language model pretraining\napproaches. The surprisingly strong ability of these models to recall factual\nknowledge without any fine-tuning demonstrates their potential as unsupervised\nopen-domain QA systems. The code to reproduce our analysis is available at\nhttps://github.com/facebookresearch/LAMA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01101v1",
            "date": "2019-09-03 11:55:59+00:00",
            "title": "Multi-agent Learning for Neural Machine Translation",
            "authors": [
                "Tianchi Bi",
                "Hao Xiong",
                "Zhongjun He",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Conventional Neural Machine Translation (NMT) models benefit from the\ntraining with an additional agent, e.g., dual learning, and bidirectional\ndecoding with one agent decoding from left to right and the other decoding in\nthe opposite direction. In this paper, we extend the training framework to the\nmulti-agent scenario by introducing diverse agents in an interactive updating\nprocess. At training time, each agent learns advanced knowledge from others,\nand they work together to improve translation quality. Experimental results on\nNIST Chinese-English, IWSLT 2014 German-English, WMT 2014 English-German and\nlarge-scale Chinese-English translation tasks indicate that our approach\nachieves absolute improvements over the strong baseline systems and shows\ncompetitive performance on all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01187v1",
            "date": "2019-09-03 13:54:52+00:00",
            "title": "Encode, Tag, Realize: High-Precision Text Editing",
            "authors": [
                "Eric Malmi",
                "Sebastian Krause",
                "Sascha Rothe",
                "Daniil Mirylenka",
                "Aliaksei Severyn"
            ],
            "abstract": "We propose LaserTagger - a sequence tagging approach that casts text\ngeneration as a text editing task. Target texts are reconstructed from the\ninputs using three main edit operations: keeping a token, deleting it, and\nadding a phrase before the token. To predict the edit operations, we propose a\nnovel model, which combines a BERT encoder with an autoregressive Transformer\ndecoder. This approach is evaluated on English text on four tasks: sentence\nfusion, sentence splitting, abstractive summarization, and grammar correction.\nLaserTagger achieves new state-of-the-art results on three of these tasks,\nperforms comparably to a set of strong seq2seq baselines with a large number of\ntraining examples, and outperforms them when the number of examples is limited.\nFurthermore, we show that at inference time tagging can be more than two orders\nof magnitude faster than comparable seq2seq models, making it more attractive\nfor running in a live environment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01214v1",
            "date": "2019-09-03 14:30:13+00:00",
            "title": "Better Rewards Yield Better Summaries: Learning to Summarise Without References",
            "authors": [
                "Florian B\u00f6hm",
                "Yang Gao",
                "Christian M. Meyer",
                "Ori Shapira",
                "Ido Dagan",
                "Iryna Gurevych"
            ],
            "abstract": "Reinforcement Learning (RL) based document summarisation systems yield\nstate-of-the-art performance in terms of ROUGE scores, because they directly\nuse ROUGE as the rewards during training. However, summaries with high ROUGE\nscores often receive low human judgement. To find a better reward function that\ncan guide RL to generate human-appealing summaries, we learn a reward function\nfrom human ratings on 2,500 summaries. Our reward function only takes the\ndocument and system summary as input. Hence, once trained, it can be used to\ntrain RL-based summarisation systems without using any reference summaries. We\nshow that our learned rewards have significantly higher correlation with human\nratings than previous approaches. Human evaluation experiments show that,\ncompared to the state-of-the-art supervised-learning systems and\nROUGE-as-rewards RL summarisation systems, the RL systems using our learned\nrewards during training generate summarieswith higher human ratings. The\nlearned reward function and our source code are available at\nhttps://github.com/yg211/summary-reward-no-reference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01247v2",
            "date": "2019-09-03 15:20:44+00:00",
            "title": "Introducing RONEC -- the Romanian Named Entity Corpus",
            "authors": [
                "Stefan Daniel Dumitrescu",
                "Andrei-Marius Avram"
            ],
            "abstract": "We present RONEC - the Named Entity Corpus for the Romanian language. The\ncorpus contains over 26000 entities in ~5000 annotated sentences, belonging to\n16 distinct classes. The sentences have been extracted from a copy-right free\nnewspaper, covering several styles. This corpus represents the first initiative\nin the Romanian language space specifically targeted for named entity\nrecognition. It is available in BRAT and CoNLL-U Plus formats, and it is free\nto use and extend at github.com/dumitrescustefan/ronec .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01276v1",
            "date": "2019-09-03 16:16:00+00:00",
            "title": "Aspect Detection using Word and Char Embeddings with (Bi)LSTM and CRF",
            "authors": [
                "\u0141ukasz Augustyniak",
                "Tomasz Kajdanowicz",
                "Przemys\u0142aw Kazienko"
            ],
            "abstract": "We proposed a~new accurate aspect extraction method that makes use of both\nword and character-based embeddings. We have conducted experiments of various\nmodels of aspect extraction using LSTM and BiLSTM including CRF enhancement on\nfive different pre-trained word embeddings extended with character embeddings.\nThe results revealed that BiLSTM outperforms regular LSTM, but also word\nembedding coverage in train and test sets profoundly impacted aspect detection\nperformance. Moreover, the additional CRF layer consistently improves the\nresults across different models and text embeddings. Summing up, we obtained\nstate-of-the-art F-score results for SemEval Restaurants (85%) and Laptops\n(80%).",
            "journal ref": "2019 IEEE Second International Conference on Artificial\n  Intelligence and Knowledge Engineering (AIKE), Sardinia, Italy, 2019, pp.\n  43-50",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01296v1",
            "date": "2019-09-03 16:40:24+00:00",
            "title": "PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking",
            "authors": [
                "Matthew Henderson",
                "Ivan Vuli\u0107",
                "I\u00f1igo Casanueva",
                "Pawe\u0142 Budzianowski",
                "Daniela Gerz",
                "Sam Coope",
                "Georgios Spithourakis",
                "Tsung-Hsien Wen",
                "Nikola Mrk\u0161i\u0107",
                "Pei-Hao Su"
            ],
            "abstract": "We present PolyResponse, a conversational search engine that supports\ntask-oriented dialogue. It is a retrieval-based approach that bypasses the\ncomplex multi-component design of traditional task-oriented dialogue systems\nand the use of explicit semantics in the form of task-specific ontologies. The\nPolyResponse engine is trained on hundreds of millions of examples extracted\nfrom real conversations: it learns what responses are appropriate in different\nconversational contexts. It then ranks a large index of text and visual\nresponses according to their similarity to the given context, and narrows down\nthe list of relevant entities during the multi-turn conversation. We introduce\na restaurant search and booking system powered by the PolyResponse engine,\ncurrently available in 8 different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01380v1",
            "date": "2019-09-03 18:06:03+00:00",
            "title": "The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives",
            "authors": [
                "Elena Voita",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "We seek to understand how the representations of individual tokens and the\nstructure of the learned feature space evolve between layers in deep neural\nnetworks under different learning objectives. We focus on the Transformers for\nour analysis as they have been shown effective on various tasks, including\nmachine translation (MT), standard left-to-right language models (LM) and\nmasked language modeling (MLM). Previous work used black-box probing tasks to\nshow that the representations learned by the Transformer differ significantly\ndepending on the objective. In this work, we use canonical correlation analysis\nand mutual information estimators to study how information flows across\nTransformer layers and how this process depends on the choice of learning\nobjective. For example, as you go from bottom to top layers, information about\nthe past in left-to-right language models gets vanished and predictions about\nthe future get formed. In contrast, for MLM, representations initially acquire\ninformation about the context around the token, partially forgetting the token\nidentity and producing a more generalized token representation. The token\nidentity then gets recreated at the top MLM layers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01383v2",
            "date": "2019-09-03 18:12:36+00:00",
            "title": "Context-Aware Monolingual Repair for Neural Machine Translation",
            "authors": [
                "Elena Voita",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "Modern sentence-level NMT systems often produce plausible translations of\nisolated sentences. However, when put in context, these translations may end up\nbeing inconsistent with each other. We propose a monolingual DocRepair model to\ncorrect inconsistencies between sentence-level translations. DocRepair performs\nautomatic post-editing on a sequence of sentence-level translations, refining\ntranslations of sentences in context of each other. For training, the DocRepair\nmodel requires only monolingual document-level data in the target language. It\nis trained as a monolingual sequence-to-sequence model that maps inconsistent\ngroups of sentences into consistent ones. The consistent groups come from the\noriginal training data; the inconsistent groups are obtained by sampling\nround-trip translations for each isolated sentence. We show that this approach\nsuccessfully imitates inconsistencies we aim to fix: using contrastive\nevaluation, we show large improvements in the translation of several contextual\nphenomena in an English-Russian translation task, as well as improvements in\nthe BLEU score. We also conduct a human evaluation and show a strong preference\nof the annotators to corrected translations over the baseline ones. Moreover,\nwe analyze which discourse phenomena are hard to capture using monolingual data\nonly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01441v1",
            "date": "2019-09-03 20:34:34+00:00",
            "title": "CrossWeigh: Training Named Entity Tagger from Imperfect Annotations",
            "authors": [
                "Zihan Wang",
                "Jingbo Shang",
                "Liyuan Liu",
                "Lihao Lu",
                "Jiacheng Liu",
                "Jiawei Han"
            ],
            "abstract": "Everyone makes mistakes. So do human annotators when curating labels for\nnamed entity recognition (NER). Such label mistakes might hurt model training\nand interfere model comparison. In this study, we dive deep into one of the\nwidely-adopted NER benchmark datasets, CoNLL03 NER. We are able to identify\nlabel mistakes in about 5.38% test sentences, which is a significant ratio\nconsidering that the state-of-the-art test F1 score is already around 93%.\nTherefore, we manually correct these label mistakes and form a cleaner test\nset. Our re-evaluation of popular models on this corrected test set leads to\nmore accurate assessments, compared to those on the original test set. More\nimportantly, we propose a simple yet effective framework, CrossWeigh, to handle\nlabel mistakes during NER model training. Specifically, it partitions the\ntraining data into several folds and train independent NER models to identify\npotential mistakes in each fold. Then it adjusts the weights of training data\naccordingly to train the final NER model. Extensive experiments demonstrate\nsignificant improvements of plugging various NER models into our proposed\nframework on three datasets. All implementations and corrected test set are\navailable at our Github repo: https://github.com/ZihanWangKi/CrossWeigh.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01462v1",
            "date": "2019-09-03 21:24:18+00:00",
            "title": "Predicting Specificity in Classroom Discussion",
            "authors": [
                "Luca Lugini",
                "Diane Litman"
            ],
            "abstract": "High quality classroom discussion is important to student development,\nenhancing abilities to express claims, reason about other students' claims, and\nretain information for longer periods of time. Previous small-scale studies\nhave shown that one indicator of classroom discussion quality is specificity.\nIn this paper we tackle the problem of predicting specificity for classroom\ndiscussions. We propose several methods and feature sets capable of\noutperforming the state of the art in specificity prediction. Additionally, we\nprovide a set of meaningful, interpretable features that can be used to analyze\nclassroom discussions at a pedagogical level.",
            "journal ref": "Proceedings of the 12th Workshop on Innovative Use of NLP for\n  Building Educational Applications, September 2017, Copenhagen, Denmark",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01482v1",
            "date": "2019-09-03 22:34:42+00:00",
            "title": "Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing",
            "authors": [
                "Tao Meng",
                "Nanyun Peng",
                "Kai-Wei Chang"
            ],
            "abstract": "Prior work on cross-lingual dependency parsing often focuses on capturing the\ncommonalities between source and target languages and overlooks the potential\nof leveraging linguistic properties of the languages to facilitate the\ntransfer. In this paper, we show that weak supervisions of linguistic knowledge\nfor the target languages can improve a cross-lingual graph-based dependency\nparser substantially. Specifically, we explore several types of corpus\nlinguistic statistics and compile them into corpus-wise constraints to guide\nthe inference process during the test time. We adapt two techniques, Lagrangian\nrelaxation and posterior regularization, to conduct inference with\ncorpus-statistics constraints. Experiments show that the Lagrangian relaxation\nand posterior regularization inference improve the performances on 15 and 17\nout of 19 target languages, respectively. The improvements are especially\nsignificant for target languages that have different word order features from\nthe source language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01522v2",
            "date": "2019-09-04 02:20:54+00:00",
            "title": "Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set",
            "authors": [
                "Katharina Kann",
                "Kyunghyun Cho",
                "Samuel R. Bowman"
            ],
            "abstract": "Development sets are impractical to obtain for real low-resource languages,\nsince using all available data for training is often more effective. However,\ndevelopment sets are widely used in research papers that purport to deal with\nlow-resource natural language processing (NLP). Here, we aim to answer the\nfollowing questions: Does using a development set for early stopping in the\nlow-resource setting influence results as compared to a more realistic\nalternative, where the number of training epochs is tuned on development\nlanguages? And does it lead to overestimation or underestimation of\nperformance? We repeat multiple experiments from recent work on neural models\nfor low-resource NLP and compare results for models obtained by training with\nand without development sets. On average over languages, absolute accuracy\ndiffers by up to 1.4%. However, for some languages and tasks, differences are\nas big as 18.0% accuracy. Our results highlight the importance of realistic\nexperimental setups in the publication of low-resource NLP research results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01559v2",
            "date": "2019-09-04 05:37:36+00:00",
            "title": "Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation",
            "authors": [
                "Baigong Zheng",
                "Renjie Zheng",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "Simultaneous translation is widely useful but remains challenging. Previous\nwork falls into two main categories: (a) fixed-latency policies such as Ma et\nal. (2019) and (b) adaptive policies such as Gu et al. (2017). The former are\nsimple and effective, but have to aggressively predict future content due to\ndiverging source-target word order; the latter do not anticipate, but suffer\nfrom unstable and inefficient training. To combine the merits of both\napproaches, we propose a simple supervised-learning framework to learn an\nadaptive policy from oracle READ/WRITE sequences generated from parallel text.\nAt each step, such an oracle sequence chooses to WRITE the next target word if\nthe available source sentence context provides enough information to do so,\notherwise READ the next source word. Experiments on German<->English show that\nour method, without retraining the underlying NMT model, can learn flexible\npolicies with better BLEU scores and similar latencies compared to previous\nwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01562v2",
            "date": "2019-09-04 05:59:03+00:00",
            "title": "Towards Better Modeling Hierarchical Structure for Self-Attention with Ordered Neurons",
            "authors": [
                "Jie Hao",
                "Xing Wang",
                "Shuming Shi",
                "Jinfeng Zhang",
                "Zhaopeng Tu"
            ],
            "abstract": "Recent studies have shown that a hybrid of self-attention networks (SANs) and\nrecurrent neural networks (RNNs) outperforms both individual architectures,\nwhile not much is known about why the hybrid models work. With the belief that\nmodeling hierarchical structure is an essential complementary between SANs and\nRNNs, we propose to further enhance the strength of hybrid models with an\nadvanced variant of RNNs - Ordered Neurons LSTM (ON-LSTM), which introduces a\nsyntax-oriented inductive bias to perform tree-like composition. Experimental\nresults on the benchmark machine translation task show that the proposed\napproach outperforms both individual architectures and a standard hybrid model.\nFurther analyses on targeted linguistic evaluation and logical inference tasks\ndemonstrate that the proposed approach indeed benefits from a better modeling\nof hierarchical structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01568v2",
            "date": "2019-09-04 06:29:35+00:00",
            "title": "AMR Normalization for Fairer Evaluation",
            "authors": [
                "Michael Wayne Goodman"
            ],
            "abstract": "Meaning Representation (AMR; Banarescu et al., 2013) encodes the meaning of\nsentences as a directed graph and Smatch (Cai and Knight, 2013) is the primary\nmetric for evaluating AMR graphs. Smatch, however, is unaware of some\nmeaning-equivalent variations in graph structure allowed by the AMR\nSpecification and gives different scores for AMRs exhibiting these variations.\nIn this paper I propose four normalization methods for helping to ensure that\nconceptually equivalent AMRs are evaluated as equivalent. Equivalent AMRs with\nand without normalization can look quite different---comparing a gold corpus to\nitself with relation reification alone yields a difference of 25 Smatch points,\nsuggesting that the outputs of two systems may not be directly comparable\nwithout normalization. The algorithms described in this paper are implemented\non top of an existing open-source Python toolkit for AMR and will be released\nunder the same license.",
            "journal ref": "33rd Pacific Asia Conference on Language, Information and\n  Computation (PACLIC 33), pages 47-56, Hakodate, Japan, September 13-15, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01584v1",
            "date": "2019-09-04 07:23:06+00:00",
            "title": "Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity",
            "authors": [
                "Yu Shi",
                "Jiaming Shen",
                "Yuchen Li",
                "Naijing Zhang",
                "Xinwei He",
                "Zhengzhi Lou",
                "Qi Zhu",
                "Matthew Walker",
                "Myunghwan Kim",
                "Jiawei Han"
            ],
            "abstract": "Text-rich heterogeneous information networks (text-rich HINs) are ubiquitous\nin real-world applications. Hypernymy, also known as is-a relation or\nsubclass-of relation, lays in the core of many knowledge graphs and benefits\nmany downstream applications. Existing methods of hypernymy discovery either\nleverage textual patterns to extract explicitly mentioned hypernym-hyponym\npairs, or learn a distributional representation for each term of interest based\nits context. These approaches rely on statistical signals from the textual\ncorpus, and their effectiveness would therefore be hindered when the signals\nfrom the corpus are not sufficient for all terms of interest. In this work, we\npropose to discover hypernymy in text-rich HINs, which can introduce additional\nhigh-quality signals. We develop a new framework, named HyperMine, that\nexploits multi-granular contexts and combines signals from both text and\nnetwork without human labeled data. HyperMine extends the definition of context\nto the scenario of text-rich HIN. For example, we can define typed nodes and\ncommunities as contexts. These contexts encode signals of different\ngranularities and we feed them into a hypernymy inference model. HyperMine\nlearns this model using weak supervision acquired based on high-precision\ntextual patterns. Extensive experiments on two large real-world datasets\ndemonstrate the effectiveness of HyperMine and the utility of modeling context\ngranularity. We further show a case study that a high-quality taxonomy can be\ngenerated solely based on the hypernymy discovered by HyperMine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01638v1",
            "date": "2019-09-04 09:17:21+00:00",
            "title": "Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?",
            "authors": [
                "Ivan Vuli\u0107",
                "Goran Glava\u0161",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "Recent efforts in cross-lingual word embedding (CLWE) learning have\npredominantly focused on fully unsupervised approaches that project monolingual\nembeddings into a shared cross-lingual space without any cross-lingual signal.\nThe lack of any supervision makes such approaches conceptually attractive. Yet,\ntheir only core difference from (weakly) supervised projection-based CLWE\nmethods is in the way they obtain a seed dictionary used to initialize an\niterative self-learning procedure. The fully unsupervised methods have arguably\nbecome more robust, and their primary use case is CLWE induction for pairs of\nresource-poor and distant languages. In this paper, we question the ability of\neven the most robust unsupervised CLWE approaches to induce meaningful CLWEs in\nthese more challenging settings. A series of bilingual lexicon induction (BLI)\nexperiments with 15 diverse languages (210 language pairs) show that fully\nunsupervised CLWE methods still fail for a large number of language pairs\n(e.g., they yield zero BLI performance for 87/210 pairs). Even when they\nsucceed, they never surpass the performance of weakly supervised methods\n(seeded with 500-1,000 translation pairs) using the same self-learning\nprocedure in any BLI setup, and the gaps are often substantial. These findings\ncall for revisiting the main motivations behind fully unsupervised CLWE\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01642v1",
            "date": "2019-09-04 09:23:50+00:00",
            "title": "ParaQG: A System for Generating Questions and Answers from Paragraphs",
            "authors": [
                "Vishwajeet Kumar",
                "Sivaanandh Muneeswaran",
                "Ganesh Ramakrishnan",
                "Yuan-Fang Li"
            ],
            "abstract": "Generating syntactically and semantically valid and relevant questions from\nparagraphs is useful with many applications. Manual generation is a\nlabour-intensive task, as it requires the reading, parsing and understanding of\nlong passages of text. A number of question generation models based on\nsequence-to-sequence techniques have recently been proposed. Most of them\ngenerate questions from sentences only, and none of them is publicly available\nas an easy-to-use service. In this paper, we demonstrate ParaQG, a Web-based\nsystem for generating questions from sentences and paragraphs. ParaQG\nincorporates a number of novel functionalities to make the question generation\nprocess user-friendly. It provides an interactive interface for a user to\nselect answers with visual insights on generation of questions. It also employs\nvarious faceted views to group similar questions as well as filtering\ntechniques to eliminate unanswerable questions",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01701v1",
            "date": "2019-09-04 11:37:26+00:00",
            "title": "SAO WMT19 Test Suite: Machine Translation of Audit Reports",
            "authors": [
                "Tereza Vojt\u011bchov\u00e1",
                "Michal Nov\u00e1k",
                "Milo\u0161 Klou\u010dek",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "This paper describes a machine translation test set of documents from the\nauditing domain and its use as one of the \"test suites\" in the WMT19 News\nTranslation Task for translation directions involving Czech, English and\nGerman.\n  Our evaluation suggests that current MT systems optimized for the general\nnews domain can perform quite well even in the particular domain of audit\nreports. The detailed manual evaluation however indicates that deep factual\nknowledge of the domain is necessary. For the naked eye of a non-expert,\ntranslations by many systems seem almost perfect and automatic MT evaluation\nwith one reference is practically useless for considering these details.\n  Furthermore, we show on a sample document from the domain of agreements that\neven the best systems completely fail in preserving the semantics of the\nagreement, namely the identity of the parties.",
            "journal ref": "Vojt\\v{e}chov\\'a et al. (2019): SAO WMT19 Test Suite: Machine\n  Translation of Audit Reports. In: Fourth Conference on Machine Translation -\n  Proceedings of the Conference, pp. 680-692, ACL, ISBN 978-1-950737-27-7",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01792v2",
            "date": "2019-09-04 13:32:23+00:00",
            "title": "Mogrifier LSTM",
            "authors": [
                "G\u00e1bor Melis",
                "Tom\u00e1\u0161 Ko\u010disk\u00fd",
                "Phil Blunsom"
            ],
            "abstract": "Many advances in Natural Language Processing have been based upon more\nexpressive models for how inputs interact with the context in which they occur.\nRecurrent networks, which have enjoyed a modicum of success, still lack the\ngeneralization and systematicity ultimately required for modelling language. In\nthis work, we propose an extension to the venerable Long Short-Term Memory in\nthe form of mutual gating of the current input and the previous output. This\nmechanism affords the modelling of a richer space of interactions between\ninputs and their context. Equivalently, our model can be viewed as making the\ntransition function given by the LSTM context-dependent. Experiments\ndemonstrate markedly improved generalization on language modelling in the range\nof 3-4 perplexity points on Penn Treebank and Wikitext-2, and 0.01-0.05 bpc on\nfour character-based datasets. We establish a new state of the art on all\ndatasets with the exception of Enwik8, where we close a large gap between the\nLSTM and Transformer models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01800v1",
            "date": "2019-09-04 13:48:27+00:00",
            "title": "Extracting Aspects Hierarchies using Rhetorical Structure Theory",
            "authors": [
                "\u0141ukasz Augustyniak",
                "Tomasz Kajdanowicz",
                "Przemys\u0142aw Kazienko"
            ],
            "abstract": "We propose a novel approach to generate aspect hierarchies that proved to be\nconsistently correct compared with human-generated hierarchies. We present an\nunsupervised technique using Rhetorical Structure Theory and graph analysis. We\nevaluated our approach based on 100,000 reviews from Amazon and achieved an\nastonishing 80% coverage compared with human-generated hierarchies coded in\nConceptNet. The method could be easily extended with a sentiment analysis model\nand used to describe sentiment on different levels of aspect granularity.\nHence, besides the flat aspect structure, we can differentiate between aspects\nand describe if the charging aspect is related to battery or price.",
            "journal ref": "ACAI 2018 MLNLP",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01807v1",
            "date": "2019-09-04 13:56:52+00:00",
            "title": "ICDM 2019 Knowledge Graph Contest: Team UWA",
            "authors": [
                "Michael Stewart",
                "Majigsuren Enkhsaikhan",
                "Wei Liu"
            ],
            "abstract": "We present an overview of our triple extraction system for the ICDM 2019\nKnowledge Graph Contest. Our system uses a pipeline-based approach to extract a\nset of triples from a given document. It offers a simple and effective solution\nto the challenge of knowledge graph construction from domain-specific text. It\nalso provides the facility to visualise useful information about each triple\nsuch as the degree, betweenness, structured relation type(s), and named entity\ntypes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01863v1",
            "date": "2019-09-04 15:11:32+00:00",
            "title": "Empirical Study of Diachronic Word Embeddings for Scarce Data",
            "authors": [
                "Syrielle Montariol",
                "Alexandre Allauzen"
            ],
            "abstract": "Word meaning change can be inferred from drifts of time-varying word\nembeddings. However, temporal data may be too sparse to build robust word\nembeddings and to discriminate significant drifts from noise. In this paper, we\ncompare three models to learn diachronic word embeddings on scarce data:\nincremental updating of a Skip-Gram from Kim et al. (2014), dynamic filtering\nfrom Bamler and Mandt (2017), and dynamic Bernoulli embeddings from Rudolph and\nBlei (2018). In particular, we study the performance of different\ninitialisation schemes and emphasise what characteristics of each model are\nmore suitable to data scarcity, relying on the distribution of detected drifts.\nFinally, we regularise the loss of these models to better adapt to scarce data.",
            "journal ref": "RANLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01953v1",
            "date": "2019-09-04 17:23:54+00:00",
            "title": "Mixture Content Selection for Diverse Sequence Generation",
            "authors": [
                "Jaemin Cho",
                "Minjoon Seo",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Generating diverse sequences is important in many NLP applications such as\nquestion generation or summarization that exhibit semantically one-to-many\nrelationships between source and the target sequences. We present a method to\nexplicitly separate diversification from generation using a general\nplug-and-play module (called SELECTOR) that wraps around and guides an existing\nencoder-decoder model. The diversification stage uses a mixture of experts to\nsample different binary masks on the source sequence for diverse content\nselection. The generation stage uses a standard encoder-decoder model given\neach selected content from the source sequence. Due to the non-differentiable\nnature of discrete sampling and the lack of ground truth labels for binary\nmask, we leverage a proxy for ground truth mask and adopt stochastic hard-EM\nfor training. In question generation (SQuAD) and abstractive summarization\n(CNN-DM), our method demonstrates significant improvements in accuracy,\ndiversity and training efficiency, including state-of-the-art top-1 accuracy in\nboth datasets, 6% gain in top-5 accuracy, and 3.7 times faster training over a\nstate of the art model. Our code is publicly available at\nhttps://github.com/clovaai/FocusSeq2Seq.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02074v1",
            "date": "2019-09-04 19:54:06+00:00",
            "title": "Jointly Learning to Align and Translate with Transformer Models",
            "authors": [
                "Sarthak Garg",
                "Stephan Peitz",
                "Udhyakumar Nallasamy",
                "Matthias Paulik"
            ],
            "abstract": "The state of the art in machine translation (MT) is governed by neural\napproaches, which typically provide superior translation accuracy over\nstatistical approaches. However, on the closely related task of word alignment,\ntraditional statistical word alignment models often remain the go-to solution.\nIn this paper, we present an approach to train a Transformer model to produce\nboth accurate translations and alignments. We extract discrete alignments from\nthe attention probabilities learnt during regular neural machine translation\nmodel training and leverage them in a multi-task framework to optimize towards\ntranslation and alignment objectives. We demonstrate that our approach produces\ncompetitive results compared to GIZA++ trained IBM alignment models without\nsacrificing translation accuracy and outperforms previous attempts on\nTransformer model based word alignment. Finally, by incorporating IBM model\nalignments into our multi-task training, we report significantly better\nalignment accuracies compared to GIZA++ on three publicly available data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02134v1",
            "date": "2019-09-04 22:10:41+00:00",
            "title": "PaLM: A Hybrid Parser and Language Model",
            "authors": [
                "Hao Peng",
                "Roy Schwartz",
                "Noah A. Smith"
            ],
            "abstract": "We present PaLM, a hybrid parser and neural language model. Building on an\nRNN language model, PaLM adds an attention layer over text spans in the left\ncontext. An unsupervised constituency parser can be derived from its attention\nweights, using a greedy decoding algorithm. We evaluate PaLM on language\nmodeling, and empirically show that it outperforms strong baselines. If\nsyntactic annotations are available, the attention component can be trained in\na supervised manner, providing syntactically-informed representations of the\ncontext, and further improving language modeling performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02177v4",
            "date": "2019-09-05 01:50:14+00:00",
            "title": "NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction",
            "authors": [
                "Wenxuan Zhou",
                "Hongtao Lin",
                "Bill Yuchen Lin",
                "Ziqi Wang",
                "Junyi Du",
                "Leonardo Neves",
                "Xiang Ren"
            ],
            "abstract": "Deep neural models for relation extraction tend to be less reliable when\nperfectly labeled data is limited, despite their success in label-sufficient\nscenarios. Instead of seeking more instance-level labels from human annotators,\nhere we propose to annotate frequent surface patterns to form labeling rules.\nThese rules can be automatically mined from large text corpora and generalized\nvia a soft rule matching mechanism. Prior works use labeling rules in an exact\nmatching fashion, which inherently limits the coverage of sentence matching and\nresults in the low-recall issue. In this paper, we present a neural approach to\nground rules for RE, named NERO, which jointly learns a relation extraction\nmodule and a soft matching module. One can employ any neural relation\nextraction models as the instantiation for the RE module. The soft matching\nmodule learns to match rules with semantically similar sentences such that raw\ncorpora can be automatically labeled and leveraged by the RE module (in a much\nbetter coverage) as augmented supervision, in addition to the exactly matched\nsentences. Extensive experiments and analysis on two public and widely-used\ndatasets demonstrate the effectiveness of the proposed NERO framework,\ncomparing with both rule-based and semi-supervised methods. Through user\nstudies, we find that the time efficiency for a human to annotate rules and\nsentences are similar (0.30 vs. 0.35 min per label). In particular, NERO's\nperformance using 270 rules is comparable to the models trained using 3,000\nlabeled sentences, yielding a 9.5x speedup. Moreover, NERO can predict for\nunseen relations at test time and provide interpretable predictions. We release\nour code to the community for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02188v1",
            "date": "2019-09-05 02:41:56+00:00",
            "title": "A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding",
            "authors": [
                "Libo Qin",
                "Wanxiang Che",
                "Yangming Li",
                "Haoyang Wen",
                "Ting Liu"
            ],
            "abstract": "Intent detection and slot filling are two main tasks for building a spoken\nlanguage understanding (SLU) system. The two tasks are closely tied and the\nslots often highly depend on the intent. In this paper, we propose a novel\nframework for SLU to better incorporate the intent information, which further\nguides the slot filling. In our framework, we adopt a joint model with\nStack-Propagation which can directly use the intent information as input for\nslot filling, thus to capture the intent semantic knowledge. In addition, to\nfurther alleviate the error propagation, we perform the token-level intent\ndetection for the Stack-Propagation framework. Experiments on two publicly\ndatasets show that our model achieves the state-of-the-art performance and\noutperforms other previous methods by a large margin. Finally, we use the\nBidirectional Encoder Representation from Transformer (BERT) model in our\nframework, which further boost our performance in SLU task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02209v3",
            "date": "2019-09-05 04:47:10+00:00",
            "title": "Semantics-aware BERT for Language Understanding",
            "authors": [
                "Zhuosheng Zhang",
                "Yuwei Wu",
                "Hai Zhao",
                "Zuchao Li",
                "Shuailiang Zhang",
                "Xi Zhou",
                "Xiang Zhou"
            ],
            "abstract": "The latest work on language representations carefully integrates\ncontextualized features into language model training, which enables a series of\nsuccess especially in various machine reading comprehension and natural\nlanguage inference tasks. However, the existing language representation models\nincluding ELMo, GPT and BERT only exploit plain context-sensitive features such\nas character or word embeddings. They rarely consider incorporating structured\nsemantic information which can provide rich semantics for language\nrepresentation. To promote natural language understanding, we propose to\nincorporate explicit contextual semantics from pre-trained semantic role\nlabeling, and introduce an improved language representation model,\nSemantics-aware BERT (SemBERT), which is capable of explicitly absorbing\ncontextual semantics over a BERT backbone. SemBERT keeps the convenient\nusability of its BERT precursor in a light fine-tuning way without substantial\ntask-specific modifications. Compared with BERT, semantics-aware BERT is as\nsimple in concept but more powerful. It obtains new state-of-the-art or\nsubstantially improves results on ten reading comprehension and language\ninference tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02222v1",
            "date": "2019-09-05 06:16:23+00:00",
            "title": "Multi-Granularity Self-Attention for Neural Machine Translation",
            "authors": [
                "Jie Hao",
                "Xing Wang",
                "Shuming Shi",
                "Jinfeng Zhang",
                "Zhaopeng Tu"
            ],
            "abstract": "Current state-of-the-art neural machine translation (NMT) uses a deep\nmulti-head self-attention network with no explicit phrase information. However,\nprior work on statistical machine translation has shown that extending the\nbasic translation unit from words to phrases has produced substantial\nimprovements, suggesting the possibility of improving NMT performance from\nexplicit modeling of phrases. In this work, we present multi-granularity\nself-attention (Mg-Sa): a neural network that combines multi-head\nself-attention and phrase modeling. Specifically, we train several attention\nheads to attend to phrases in either n-gram or syntactic formalism. Moreover,\nwe exploit interactions among phrases to enhance the strength of structure\nmodeling - a commonly-cited weakness of self-attention. Experimental results on\nWMT14 English-to-German and NIST Chinese-to-English translation tasks show the\nproposed approach consistently improves performance. Targeted linguistic\nanalysis reveals that Mg-Sa indeed captures useful phrase information at\nvarious levels of granularities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02224v2",
            "date": "2019-09-05 06:20:43+00:00",
            "title": "Examining Gender Bias in Languages with Grammatical Gender",
            "authors": [
                "Pei Zhou",
                "Weijia Shi",
                "Jieyu Zhao",
                "Kuan-Hao Huang",
                "Muhao Chen",
                "Ryan Cotterell",
                "Kai-Wei Chang"
            ],
            "abstract": "Recent studies have shown that word embeddings exhibit gender bias inherited\nfrom the training corpora. However, most studies to date have focused on\nquantifying and mitigating such bias only in English. These analyses cannot be\ndirectly extended to languages that exhibit morphological agreement on gender,\nsuch as Spanish and French. In this paper, we propose new metrics for\nevaluating gender bias in word embeddings of these languages and further\ndemonstrate evidence of gender bias in bilingual embeddings which align these\nlanguages with English. Finally, we extend an existing approach to mitigate\ngender bias in word embeddings under both monolingual and bilingual settings.\nExperiments on modified Word Embedding Association Test, word similarity, word\ntranslation, and word pair translation tasks show that the proposed approaches\neffectively reduce the gender bias while preserving the utility of the\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02235v1",
            "date": "2019-09-05 07:10:44+00:00",
            "title": "Cross-Lingual Dependency Parsing Using Code-Mixed TreeBank",
            "authors": [
                "Zhang Meishan",
                "Zhang Yue",
                "Fu Guohong"
            ],
            "abstract": "Treebank translation is a promising method for cross-lingual transfer of\nsyntactic dependency knowledge. The basic idea is to map dependency arcs from a\nsource treebank to its target translation according to word alignments. This\nmethod, however, can suffer from imperfect alignment between source and target\nwords. To address this problem, we investigate syntactic transfer by code\nmixing, translating only confident words in a source treebank. Cross-lingual\nword embeddings are leveraged for transferring syntactic knowledge to the\ntarget from the resulting code-mixed treebank. Experiments on University\nDependency Treebanks show that code-mixed treebanks are more effective than\ntranslated treebanks, giving highly competitive performances among\ncross-lingual parsing methods.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02250v3",
            "date": "2019-09-05 07:56:45+00:00",
            "title": "Nested Named Entity Recognition via Second-best Sequence Learning and Decoding",
            "authors": [
                "Takashi Shibuya",
                "Eduard Hovy"
            ],
            "abstract": "When an entity name contains other names within it, the identification of all\ncombinations of names can become difficult and expensive. We propose a new\nmethod to recognize not only outermost named entities but also inner nested\nones. We design an objective function for training a neural model that treats\nthe tag sequence for nested entities as the second best path within the span of\ntheir parent entity. In addition, we provide the decoding method for inference\nthat extracts entities iteratively from outermost ones to inner ones in an\noutside-to-inside way. Our method has no additional hyperparameters to the\nconditional random field based model widely used for flat named entity\nrecognition tasks. Experiments demonstrate that our method performs better than\nor at least as well as existing methods capable of handling nested entities,\nachieving the F1-scores of 85.82%, 84.34%, and 77.36% on ACE-2004, ACE-2005,\nand GENIA datasets, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02265v1",
            "date": "2019-09-05 08:47:49+00:00",
            "title": "Towards Task-Oriented Dialogue in Mixed Domains",
            "authors": [
                "Tho Luong Chi",
                "Phuong Le-Hong"
            ],
            "abstract": "This work investigates the task-oriented dialogue problem in mixed-domain\nsettings. We study the effect of alternating between different domains in\nsequences of dialogue turns using two related state-of-the-art dialogue\nsystems. We first show that a specialized state tracking component in multiple\ndomains plays an important role and gives better results than an end-to-end\ntask-oriented dialogue system. We then propose a hybrid system which is able to\nimprove the belief tracking accuracy of about 28% of average absolute point on\na standard multi-domain dialogue dataset. These experimental results give some\nuseful insights for improving our commercial chatbot platform FPT.AI, which is\ncurrently deployed for many practical chatbot applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02273v1",
            "date": "2019-09-05 09:17:37+00:00",
            "title": "Source Dependency-Aware Transformer with Supervised Self-Attention",
            "authors": [
                "Chengyi Wang",
                "Shuangzhi Wu",
                "Shujie Liu"
            ],
            "abstract": "Recently, Transformer has achieved the state-of-the-art performance on many\nmachine translation tasks. However, without syntax knowledge explicitly\nconsidered in the encoder, incorrect context information that violates the\nsyntax structure may be integrated into source hidden states, leading to\nerroneous translations. In this paper, we propose a novel method to incorporate\nsource dependencies into the Transformer. Specifically, we adopt the source\ndependency tree and define two matrices to represent the dependency relations.\nBased on the matrices, two heads in the multi-head self-attention module are\ntrained in a supervised manner and two extra cross entropy losses are\nintroduced into the training objective function. Under this training objective,\nthe model is trained to learn the source dependency relations directly. Without\nrequiring pre-parsed input during inference, our model can generate better\ntranslations with the dependency-aware context information. Experiments on\nbi-directional Chinese-to-English, English-to-Japanese and English-to-German\ntranslation tasks show that our proposed method can significantly improve the\nTransformer baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02279v1",
            "date": "2019-09-05 09:22:25+00:00",
            "title": "Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network",
            "authors": [
                "Chengyi Wang",
                "Shuangzhi Wu",
                "Shujie Liu"
            ],
            "abstract": "Due to the highly parallelizable architecture, Transformer is faster to train\nthan RNN-based models and popularly used in machine translation tasks. However,\nat inference time, each output word requires all the hidden states of the\npreviously generated words, which limits the parallelization capability, and\nmakes it much slower than RNN-based ones. In this paper, we systematically\nanalyze the time cost of different components of both the Transformer and\nRNN-based model. Based on it, we propose a hybrid network of self-attention and\nRNN structures, in which, the highly parallelizable self-attention is utilized\nas the encoder, and the simpler RNN structure is used as the decoder. Our\nhybrid network can decode 4-times faster than the Transformer. In addition,\nwith the help of knowledge distillation, our hybrid network achieves comparable\ntranslation quality to the original Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02304v1",
            "date": "2019-09-05 10:25:34+00:00",
            "title": "Table-to-Text Generation with Effective Hierarchical Encoder on Three Dimensions (Row, Column and Time)",
            "authors": [
                "Heng Gong",
                "Xiaocheng Feng",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Although Seq2Seq models for table-to-text generation have achieved remarkable\nprogress, modeling table representation in one dimension is inadequate. This is\nbecause (1) the table consists of multiple rows and columns, which means that\nencoding a table should not depend only on one dimensional sequence or set of\nrecords and (2) most of the tables are time series data (e.g. NBA game data,\nstock market data), which means that the description of the current table may\nbe affected by its historical data. To address aforementioned problems, not\nonly do we model each table cell considering other records in the same row, we\nalso enrich table's representation by modeling each table cell in context of\nother cells in the same column or with historical (time dimension) data\nrespectively. In addition, we develop a table cell fusion gate to combine\nrepresentations from row, column and time dimension into one dense vector\naccording to the saliency of each dimension's representation. We evaluated our\nmethods on ROTOWIRE, a benchmark dataset of NBA basketball games. Both\nautomatic and human evaluation results demonstrate the effectiveness of our\nmodel with improvement of 2.66 in BLEU over the strong baseline and\noutperformance of state-of-the-art model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02322v2",
            "date": "2019-09-05 11:11:41+00:00",
            "title": "Informative and Controllable Opinion Summarization",
            "authors": [
                "Reinald Kim Amplayo",
                "Mirella Lapata"
            ],
            "abstract": "Opinion summarization is the task of automatically generating summaries for a\nset of reviews about a specific target (e.g., a movie or a product). Since the\nnumber of reviews for each target can be prohibitively large, neural\nnetwork-based methods follow a two-stage approach where an extractive step\nfirst pre-selects a subset of salient opinions and an abstractive step creates\nthe summary while conditioning on the extracted subset. However, the extractive\nmodel leads to loss of information which may be useful depending on user needs.\nIn this paper we propose a summarization framework that eliminates the need to\nrely only on pre-selected content and waste possibly useful information,\nespecially when customizing summaries. The framework enables the use of all\ninput reviews by first condensing them into multiple dense vectors which serve\nas input to an abstractive model. We showcase an effective instantiation of our\nframework which produces more informative summaries and also allows to take\nuser preferences into account using our zero-shot customization technique.\nExperimental results demonstrate that our model improves the state of the art\non the Rotten Tomatoes dataset and generates customized summaries effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02339v2",
            "date": "2019-09-05 11:49:40+00:00",
            "title": "Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity",
            "authors": [
                "Anne Lauscher",
                "Ivan Vuli\u0107",
                "Edoardo Maria Ponti",
                "Anna Korhonen",
                "Goran Glava\u0161"
            ],
            "abstract": "Unsupervised pretraining models have been shown to facilitate a wide range of\ndownstream NLP applications. These models, however, retain some of the\nlimitations of traditional static word embeddings. In particular, they encode\nonly the distributional knowledge available in raw text corpora, incorporated\nthrough language modeling objectives. In this work, we complement such\ndistributional knowledge with external lexical knowledge, that is, we integrate\nthe discrete knowledge on word-level semantic similarity into pretraining. To\nthis end, we generalize the standard BERT model to a multi-task learning\nsetting where we couple BERT's masked language modeling and next sentence\nprediction objectives with an auxiliary task of binary word relation\nclassification. Our experiments suggest that our \"Lexically Informed\" BERT\n(LIBERT), specialized for the word-level semantic similarity, yields better\nperformance than the lexically blind \"vanilla\" BERT on several language\nunderstanding tasks. Concretely, LIBERT outperforms BERT in 9 out of 10 tasks\nof the GLUE benchmark and is on a par with BERT in the remaining one. Moreover,\nwe show consistent gains on 3 benchmarks for lexical simplification, a task\nwhere knowledge about word-level semantic similarity is paramount.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02392v2",
            "date": "2019-09-05 13:29:26+00:00",
            "title": "Rewarding Coreference Resolvers for Being Consistent with World Knowledge",
            "authors": [
                "Rahul Aralikatte",
                "Heather Lent",
                "Ana Valeria Gonzalez",
                "Daniel Hershcovich",
                "Chen Qiu",
                "Anders Sandholm",
                "Michael Ringaard",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Unresolved coreference is a bottleneck for relation extraction, and\nhigh-quality coreference resolvers may produce an output that makes it a lot\neasier to extract knowledge triples. We show how to improve coreference\nresolvers by forwarding their input to a relation extraction system and reward\nthe resolvers for producing triples that are found in knowledge bases. Since\nrelation extraction systems can rely on different forms of supervision and be\nbiased in different ways, we obtain the best performance, improving over the\nstate of the art, using multi-task reinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02399v1",
            "date": "2019-09-05 13:33:11+00:00",
            "title": "Reading Comprehension Ability Test-A Turing Test for Reading Comprehension",
            "authors": [
                "Yuan Miao",
                "Gongqi Lin",
                "Yidan Hu",
                "Chunyan Miao"
            ],
            "abstract": "Reading comprehension is an important ability of human intelligence. Literacy\nand numeracy are two most essential foundation for people to succeed at study,\nat work and in life. Reading comprehension ability is a core component of\nliteracy. In most of the education systems, developing reading comprehension\nability is compulsory in the curriculum from year one to year 12. It is an\nindispensable ability in the dissemination of knowledge. With the emerging\nartificial intelligence, computers start to be able to read and understand like\npeople in some context. They can even read better than human beings for some\ntasks, but have little clue in other tasks. It will be very beneficial if we\ncan identify the levels of machine comprehension ability, which will direct us\non the further improvement. Turing test is a well-known test of the difference\nbetween computer intelligence and human intelligence. In order to be able to\ncompare the difference between people reading and machines reading, we proposed\na test called (reading) Comprehension Ability Test (CAT).CAT is similar to\nTuring test, passing of which means we cannot differentiate people from\nalgorithms in term of their comprehension ability. CAT has multiple levels\nshowing the different abilities in reading comprehension, from identifying\nbasic facts, performing inference, to understanding the intent and sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02476v1",
            "date": "2019-09-05 15:22:44+00:00",
            "title": "A Discussion on Influence of Newspaper Headlines on Social Media",
            "authors": [
                "Aneek Barman Roy",
                "Baolei Chen",
                "Siddharth Tiwari",
                "Zihan Huang"
            ],
            "abstract": "Newspaper headlines contribute severely and have an influence on the social\nmedia. This work studies the durability of impact of verbs and adjectives on\nheadlines and determine the factors which are responsible for its nature of\ninfluence on the social media. Each headline has been categorized into\npositive, negative or neutral based on its sentiment score. Initial results\nshow that intensity of a sentiment nature is positively correlated with the\nsocial media impression. Additionally, verbs and adjectives show a relation\nwith the sentiment scores",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02560v5",
            "date": "2019-09-05 17:59:15+00:00",
            "title": "Robustness to Modification with Shared Words in Paraphrase Identification",
            "authors": [
                "Zhouxing Shi",
                "Minlie Huang"
            ],
            "abstract": "Revealing the robustness issues of natural language processing models and\nimproving their robustness is important to their performance under difficult\nsituations. In this paper, we study the robustness of paraphrase identification\nmodels from a new perspective -- via modification with shared words, and we\nshow that the models have significant robustness issues when facing such\nmodifications. To modify an example consisting of a sentence pair, we either\nreplace some words shared by both sentences or introduce new shared words. We\naim to construct a valid new example such that a target model makes a wrong\nprediction. To find a modification solution, we use beam search constrained by\nheuristic rules, and we leverage a BERT masked language model for generating\nsubstitution words compatible with the context. Experiments show that the\nperformance of the target models has a dramatic drop on the modified examples,\nthereby revealing the robustness issue. We also show that adversarial training\ncan mitigate this issue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02597v2",
            "date": "2019-09-05 18:58:51+00:00",
            "title": "Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs",
            "authors": [
                "Alex Warstadt",
                "Yu Cao",
                "Ioana Grosu",
                "Wei Peng",
                "Hagen Blix",
                "Yining Nie",
                "Anna Alsop",
                "Shikha Bordia",
                "Haokun Liu",
                "Alicia Parrish",
                "Sheng-Fu Wang",
                "Jason Phang",
                "Anhad Mohananey",
                "Phu Mon Htut",
                "Paloma Jereti\u010d",
                "Samuel R. Bowman"
            ],
            "abstract": "Though state-of-the-art sentence representation models can perform tasks\nrequiring significant knowledge of grammar, it is an open question how best to\nevaluate their grammatical knowledge. We explore five experimental methods\ninspired by prior work evaluating pretrained sentence representation models. We\nuse a single linguistic phenomenon, negative polarity item (NPI) licensing in\nEnglish, as a case study for our experiments. NPIs like \"any\" are grammatical\nonly if they appear in a licensing environment like negation (\"Sue doesn't have\nany cats\" vs. \"Sue has any cats\"). This phenomenon is challenging because of\nthe variety of NPI licensing environments that exist. We introduce an\nartificially generated dataset that manipulates key features of NPI licensing\nfor the experiments. We find that BERT has significant knowledge of these\nfeatures, but its success varies widely across different experimental methods.\nWe conclude that a variety of methods is necessary to reveal all relevant\naspects of a model's grammatical knowledge in a given domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02606v1",
            "date": "2019-09-05 19:20:27+00:00",
            "title": "Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks",
            "authors": [
                "Binxuan Huang",
                "Kathleen M. Carley"
            ],
            "abstract": "Aspect level sentiment classification aims to identify the sentiment\nexpressed towards an aspect given a context sentence. Previous neural network\nbased methods largely ignore the syntax structure in one sentence. In this\npaper, we propose a novel target-dependent graph attention network (TD-GAT) for\naspect level sentiment classification, which explicitly utilizes the dependency\nrelationship among words. Using the dependency graph, it propagates sentiment\nfeatures directly from the syntactic context of an aspect target. In our\nexperiments, we show our method outperforms multiple baselines with GloVe\nembeddings. We also demonstrate that using BERT representations further\nsubstantially boosts the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02607v2",
            "date": "2019-09-05 19:21:27+00:00",
            "title": "Broad-Coverage Semantic Parsing as Transduction",
            "authors": [
                "Sheng Zhang",
                "Xutai Ma",
                "Kevin Duh",
                "Benjamin Van Durme"
            ],
            "abstract": "We unify different broad-coverage semantic parsing tasks under a transduction\nparadigm, and propose an attention-based neural framework that incrementally\nbuilds a meaning representation via a sequence of semantic relations. By\nleveraging multiple attention mechanisms, the transducer can be effectively\ntrained without relying on a pre-trained aligner. Experiments conducted on\nthree separate broad-coverage semantic parsing tasks -- AMR, SDP and UCCA --\ndemonstrate that our attention-based neural transducer improves the state of\nthe art on both AMR and UCCA, and is competitive with the state of the art on\nSDP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02621v1",
            "date": "2019-09-05 20:26:10+00:00",
            "title": "TEASPN: Framework and Protocol for Integrated Writing Assistance Environments",
            "authors": [
                "Masato Hagiwara",
                "Takumi Ito",
                "Tatsuki Kuribayashi",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Language technologies play a key role in assisting people with their writing.\nAlthough there has been steady progress in e.g., grammatical error correction\n(GEC), human writers are yet to benefit from this progress due to the high\ndevelopment cost of integrating with writing software. We propose TEASPN, a\nprotocol and an open-source framework for achieving integrated writing\nassistance environments. The protocol standardizes the way writing software\ncommunicates with servers that implement such technologies, allowing developers\nand researchers to integrate the latest developments in natural language\nprocessing (NLP) with low cost. As a result, users can enjoy the integrated\nexperience in their favorite writing software. The results from experiments\nwith human participants show that users use a wide range of technologies and\nrate their writing experience favorably, allowing them to write more fluent\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02622v2",
            "date": "2019-09-05 20:26:44+00:00",
            "title": "MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance",
            "authors": [
                "Wei Zhao",
                "Maxime Peyrard",
                "Fei Liu",
                "Yang Gao",
                "Christian M. Meyer",
                "Steffen Eger"
            ],
            "abstract": "A robust evaluation metric has a profound impact on the development of text\ngeneration systems. A desirable metric compares system output against\nreferences based on their semantics rather than surface forms. In this paper we\ninvestigate strategies to encode system and reference texts to devise a metric\nthat shows a high correlation with human judgment of text quality. We validate\nour new metric, namely MoverScore, on a number of text generation tasks\nincluding summarization, machine translation, image captioning, and\ndata-to-text generation, where the outputs are produced by a variety of neural\nand non-neural systems. Our findings suggest that metrics combining\ncontextualized representations with a distance measure perform the best. Such\nmetrics also demonstrate strong generalization capability across tasks. For\nease-of-use we make our metrics available as web service.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02635v1",
            "date": "2019-09-05 21:13:37+00:00",
            "title": "Effective Use of Transformer Networks for Entity Tracking",
            "authors": [
                "Aditya Gupta",
                "Greg Durrett"
            ],
            "abstract": "Tracking entities in procedural language requires understanding the\ntransformations arising from actions on entities as well as those entities'\ninteractions. While self-attention-based pre-trained language encoders like GPT\nand BERT have been successfully applied across a range of natural language\nunderstanding tasks, their ability to handle the nuances of procedural texts is\nstill untested. In this paper, we explore the use of pre-trained transformer\nnetworks for entity tracking tasks in procedural text. First, we test standard\nlightweight approaches for prediction with pre-trained transformers, and find\nthat these approaches underperform even simple baselines. We show that much\nstronger results can be attained by restructuring the input to guide the\ntransformer model to focus on a particular entity. Second, we assess the degree\nto which transformer networks capture the process dynamics, investigating such\nfactors as merged entities and oblique entity references. On two different\ntasks, ingredient detection in recipes and QA over scientific processes, we\nachieve state-of-the-art results, but our models still largely attend to\nshallow context clues and do not form complex representations of intermediate\nentity or process state.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02670v1",
            "date": "2019-09-05 23:20:29+00:00",
            "title": "In Plain Sight: Media Bias Through the Lens of Factual Reporting",
            "authors": [
                "Lisa Fan",
                "Marshall White",
                "Eva Sharma",
                "Ruisi Su",
                "Prafulla Kumar Choubey",
                "Ruihong Huang",
                "Lu Wang"
            ],
            "abstract": "The increasing prevalence of political bias in news media calls for greater\npublic awareness of it, as well as robust methods for its detection. While\nprior work in NLP has primarily focused on the lexical bias captured by\nlinguistic attributes such as word choice and syntax, other types of bias stem\nfrom the actual content selected for inclusion in the text. In this work, we\ninvestigate the effects of informational bias: factual content that can\nnevertheless be deployed to sway reader opinion. We first produce a new\ndataset, BASIL, of 300 news articles annotated with 1,727 bias spans and find\nevidence that informational bias appears in news articles more frequently than\nlexical bias. We further study our annotations to observe how informational\nbias surfaces in news articles by different media outlets. Lastly, a baseline\nmodel for informational bias prediction is presented by fine-tuning BERT on our\nlabeled data, indicating the challenges of the task and future directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02745v1",
            "date": "2019-09-06 07:20:17+00:00",
            "title": "Incorporating External Knowledge into Machine Reading for Generative Question Answering",
            "authors": [
                "Bin Bi",
                "Chen Wu",
                "Ming Yan",
                "Wei Wang",
                "Jiangnan Xia",
                "Chenliang Li"
            ],
            "abstract": "Commonsense and background knowledge is required for a QA model to answer\nmany nontrivial questions. Different from existing work on knowledge-aware QA,\nwe focus on a more challenging task of leveraging external knowledge to\ngenerate answers in natural language for a given question with context.\n  In this paper, we propose a new neural model, Knowledge-Enriched Answer\nGenerator (KEAG), which is able to compose a natural answer by exploiting and\naggregating evidence from all four information sources available: question,\npassage, vocabulary and knowledge. During the process of answer generation,\nKEAG adaptively determines when to utilize symbolic knowledge and which fact\nfrom the knowledge is useful. This allows the model to exploit external\nknowledge that is not explicitly stated in the given text, but that is relevant\nfor generating an answer. The empirical study on public benchmark of answer\ngeneration demonstrates that KEAG improves answer quality over models without\nknowledge and existing knowledge-aware models, confirming its effectiveness in\nleveraging knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02762v1",
            "date": "2019-09-06 08:22:28+00:00",
            "title": "Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering",
            "authors": [
                "Tao Shen",
                "Xiubo Geng",
                "Tao Qin",
                "Guodong Long",
                "Jing Jiang",
                "Daxin Jiang"
            ],
            "abstract": "Many algorithms for Knowledge-Based Question Answering (KBQA) depend on\nsemantic parsing, which translates a question to its logical form. When only\nweak supervision is provided, it is usually necessary to search valid logical\nforms for model training. However, a complex question typically involves a huge\nsearch space, which creates two main problems: 1) the solutions limited by\ncomputation time and memory usually reduce the success rate of the search, and\n2) spurious logical forms in the search results degrade the quality of training\ndata. These two problems lead to a poorly-trained semantic parsing model. In\nthis work, we propose an effective search method for weakly supervised KBQA\nbased on operator prediction for questions. With search space constrained by\npredicted operators, sufficient search paths can be explored, more valid\nlogical forms can be derived, and operators possibly causing spurious logical\nforms can be avoided. As a result, a larger proportion of questions in a weakly\nsupervised training set are equipped with logical forms, and fewer spurious\nlogical forms are generated. Such high-quality training data directly\ncontributes to a better semantic parsing model. Experimental results on one of\nthe largest KBQA datasets (i.e., CSQA) verify the effectiveness of our\napproach: improving the precision from 67% to 72% and the recall from 67% to\n72% in terms of the overall score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02766v1",
            "date": "2019-09-06 08:37:42+00:00",
            "title": "Giveme5W1H: A Universal System for Extracting Main Events from News Articles",
            "authors": [
                "Felix Hamborg",
                "Corinna Breitinger",
                "Bela Gipp"
            ],
            "abstract": "Event extraction from news articles is a commonly required prerequisite for\nvarious tasks, such as article summarization, article clustering, and news\naggregation. Due to the lack of universally applicable and publicly available\nmethods tailored to news datasets, many researchers redundantly implement event\nextraction methods for their own projects. The journalistic 5W1H questions are\ncapable of describing the main event of an article, i.e., by answering who did\nwhat, when, where, why, and how. We provide an in-depth description of an\nimproved version of Giveme5W1H, a system that uses syntactic and\ndomain-specific rules to automatically extract the relevant phrases from\nEnglish news articles to provide answers to these 5W1H questions. Given the\nanswers to these questions, the system determines an article's main event. In\nan expert evaluation with three assessors and 120 articles, we determined an\noverall precision of p=0.73, and p=0.82 for answering the first four W\nquestions, which alone can sufficiently summarize the main event reported on in\na news article. We recently made our system publicly available, and it remains\nthe only universal open-source 5W1H extractor capable of being applied to a\nwide range of use cases in news analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02855v2",
            "date": "2019-09-06 12:27:43+00:00",
            "title": "Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction",
            "authors": [
                "Paula Czarnowska",
                "Sebastian Ruder",
                "Edouard Grave",
                "Ryan Cotterell",
                "Ann Copestake"
            ],
            "abstract": "Human translators routinely have to translate rare inflections of words - due\nto the Zipfian distribution of words in a language. When translating from\nSpanish, a good translator would have no problem identifying the proper\ntranslation of a statistically rare inflection such as habl\\'aramos. Note the\nlexeme itself, hablar, is relatively common. In this work, we investigate\nwhether state-of-the-art bilingual lexicon inducers are capable of learning\nthis kind of generalization. We introduce 40 morphologically complete\ndictionaries in 10 languages and evaluate three of the state-of-the-art models\non the task of translation of less frequent morphological forms. We demonstrate\nthat the performance of state-of-the-art models drops considerably when\nevaluated on infrequent morphological inflections and then show that adding a\nsimple morphological constraint at training time improves the performance,\nproving that the bilingual lexicon inducers can benefit from better encoding of\nmorphology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02857v1",
            "date": "2019-09-06 12:32:52+00:00",
            "title": "A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages",
            "authors": [
                "Clara Vania",
                "Yova Kementchedjhieva",
                "Anders S\u00f8gaard",
                "Adam Lopez"
            ],
            "abstract": "Parsers are available for only a handful of the world's languages, since they\nrequire lots of training data. How far can we get with just a small amount of\ntraining data? We systematically compare a set of simple strategies for\nimproving low-resource parsers: data augmentation, which has not been tested\nbefore; cross-lingual training; and transliteration. Experimenting on three\ntypologically diverse low-resource languages---North S\\'ami, Galician, and\nKazah---We find that (1) when only the low-resource treebank is available, data\naugmentation is very helpful; (2) when a related high-resource treebank is\navailable, cross-lingual training is helpful and complements data augmentation;\nand (3) when the high-resource treebank uses a different writing system,\ntransliteration into a shared orthographic spaces is also very helpful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02955v2",
            "date": "2019-09-06 15:03:29+00:00",
            "title": "Extracting and Learning a Dependency-Enhanced Type Lexicon for Dutch",
            "authors": [
                "Konstantinos Kogkalidis"
            ],
            "abstract": "This thesis is concerned with type-logical grammars and their practical\napplicability as tools of reasoning about sentence syntax and semantics. The\nfocal point is narrowed to Dutch, a language exhibiting a large degree of word\norder variability. In order to overcome difficulties arising as a result of\nthat variability, the thesis explores and expands upon a type grammar based on\nMultiplicative Intuitionistic Linear Logic, agnostic to word order but enriched\nwith decorations that aim to reduce its proof-theoretic complexity. An\nalgorithm for the conversion of dependency-annotated sentences into type\nsequences is then implemented, populating the type logic with concrete,\ndata-driven lexical types. Two experiments are ran on the resulting grammar\ninstantiation. The first pertains to the learnability of the type-assignment\nprocess by a neural architecture. A novel application of a self-attentive\nsequence transduction model is proposed; contrary to established practices, it\nconstructs types inductively by internalizing the type-formation syntax, thus\nexhibiting generalizability beyond a pre-specified type vocabulary. The second\nrevolves around a deductive parsing system that can resolve structural\nambiguities by consulting both word and type information; preliminary results\nsuggest both excellent computational efficiency and performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03022v1",
            "date": "2019-09-06 17:17:06+00:00",
            "title": "Argument Component Classification for Classroom Discussions",
            "authors": [
                "Luca Lugini",
                "Diane Litman"
            ],
            "abstract": "This paper focuses on argument component classification for transcribed\nspoken classroom discussions, with the goal of automatically classifying\nstudent utterances into claims, evidence, and warrants. We show that an\nexisting method for argument component classification developed for another\neducationally-oriented domain performs poorly on our dataset. We then show that\nfeature sets from prior work on argument mining for student essays and online\ndialogues can be used to improve performance considerably. We also provide a\ncomparison between convolutional neural networks and recurrent neural networks\nwhen trained under different conditions to classify argument components in\nclassroom discussions. While neural network models are not always able to\noutperform a logistic regression model, we were able to gain some useful\ninsights: convolutional networks are more robust than recurrent networks both\nat the character and at the word level, and specificity information can help\nboost performance in multi-task training.",
            "journal ref": "Proceedings of the 5th Workshop on Argument Mining, November 2018,\n  Brussels, Belgium",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03023v1",
            "date": "2019-09-06 17:18:49+00:00",
            "title": "Annotating Student Talk in Text-based Classroom Discussions",
            "authors": [
                "Luca Lugini",
                "Diane Litman",
                "Amanda Godley",
                "Christopher Olshefski"
            ],
            "abstract": "Classroom discussions in English Language Arts have a positive effect on\nstudents' reading, writing and reasoning skills. Although prior work has\nlargely focused on teacher talk and student-teacher interactions, we focus on\nthree theoretically-motivated aspects of high-quality student talk:\nargumentation, specificity, and knowledge domain. We introduce an annotation\nscheme, then show that the scheme can be used to produce reliable annotations\nand that the annotations are predictive of discussion quality. We also\nhighlight opportunities provided by our scheme for education and natural\nlanguage processing research.",
            "journal ref": "Proceedings of the Thirteenth Workshop on Innovative Use of NLP\n  for Building Educational Applications, June 2018, New Orleans, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03042v2",
            "date": "2019-09-06 17:52:55+00:00",
            "title": "Uncertain Natural Language Inference",
            "authors": [
                "Tongfei Chen",
                "Zhengping Jiang",
                "Adam Poliak",
                "Keisuke Sakaguchi",
                "Benjamin Van Durme"
            ],
            "abstract": "We introduce Uncertain Natural Language Inference (UNLI), a refinement of\nNatural Language Inference (NLI) that shifts away from categorical labels,\ntargeting instead the direct prediction of subjective probability assessments.\nWe demonstrate the feasibility of collecting annotations for UNLI by relabeling\na portion of the SNLI dataset under a probabilistic scale, where items even\nwith the same categorical label differ in how likely people judge them to be\ntrue given a premise. We describe a direct scalar regression modeling approach,\nand find that existing categorically labeled NLI data can be used in\npre-training. Our best models approach human performance, demonstrating models\nmay be capable of more subtle inferences than the categorical bin assignment\nemployed in current NLI tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03065v1",
            "date": "2019-09-06 14:12:15+00:00",
            "title": "\"Going on a vacation\" takes longer than \"Going for a walk\": A Study of Temporal Commonsense Understanding",
            "authors": [
                "Ben Zhou",
                "Daniel Khashabi",
                "Qiang Ning",
                "Dan Roth"
            ],
            "abstract": "Understanding time is crucial for understanding events expressed in natural\nlanguage. Because people rarely say the obvious, it is often necessary to have\ncommonsense knowledge about various temporal aspects of events, such as\nduration, frequency, and temporal order. However, this important problem has so\nfar received limited attention. This paper systematically studies this temporal\ncommonsense problem. Specifically, we define five classes of temporal\ncommonsense, and use crowdsourcing to develop a new dataset, MCTACO, that\nserves as a test set for this task. We find that the best current methods used\non MCTACO are still far behind human performance, by about 20%, and discuss\nseveral directions for improvement. We hope that the new dataset and our study\nhere can foster more future research on this topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03084v1",
            "date": "2019-09-06 18:13:48+00:00",
            "title": "Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification",
            "authors": [
                "Yichao Zhou",
                "Jyun-Yu Jiang",
                "Kai-Wei Chang",
                "Wei Wang"
            ],
            "abstract": "Adversarial attacks against machine learning models have threatened various\nreal-world applications such as spam filtering and sentiment analysis. In this\npaper, we propose a novel framework, learning to DIScriminate Perturbations\n(DISP), to identify and adjust malicious perturbations, thereby blocking\nadversarial attacks for text classification models. To identify adversarial\nattacks, a perturbation discriminator validates how likely a token in the text\nis perturbed and provides a set of potential perturbations. For each potential\nperturbation, an embedding estimator learns to restore the embedding of the\noriginal word based on the context and a replacement token is chosen based on\napproximate kNN search. DISP can block adversarial attacks for any NLP model\nwithout modifying the model structure or training procedure. Extensive\nexperiments on two benchmark datasets demonstrate that DISP significantly\noutperforms baseline methods in blocking adversarial attacks for text\nclassification. In addition, in-depth analysis shows the robustness of DISP\nacross different situations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03087v1",
            "date": "2019-09-06 18:44:49+00:00",
            "title": "ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons",
            "authors": [
                "Margaret Li",
                "Jason Weston",
                "Stephen Roller"
            ],
            "abstract": "While dialogue remains an important end-goal of natural language research,\nthe difficulty of evaluation is an oft-quoted reason why it remains troublesome\nto make real progress towards its solution. Evaluation difficulties are\nactually two-fold: not only do automatic metrics not correlate well with human\njudgments, but also human judgments themselves are in fact difficult to\nmeasure. The two most used human judgment tests, single-turn pairwise\nevaluation and multi-turn Likert scores, both have serious flaws as we discuss\nin this work.\n  We instead provide a novel procedure involving comparing two full dialogues,\nwhere a human judge is asked to pay attention to only one speaker within each,\nand make a pairwise judgment. The questions themselves are optimized to\nmaximize the robustness of judgments across different annotators, resulting in\nbetter tests. We also show how these tests work in self-play model chat setups,\nresulting in faster, cheaper tests. We hope these tests become the de facto\nstandard, and will release open-source code to that end.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03100v1",
            "date": "2019-09-06 19:40:06+00:00",
            "title": "Attending the Emotions to Detect Online Abusive Language",
            "authors": [
                "Niloofar Safi Samghabadi",
                "Afsheen Hatami",
                "Mahsa Shafaei",
                "Sudipta Kar",
                "Thamar Solorio"
            ],
            "abstract": "In recent years, abusive behavior has become a serious issue in online social\nnetworks. In this paper, we present a new corpus from a semi-anonymous social\nmedia platform, which contains the instances of offensive and neutral classes.\nWe introduce a single deep neural architecture that considers both local and\nsequential information from the text in order to detect abusive language. Along\nwith this model, we introduce a new attention mechanism called emotion-aware\nattention. This mechanism utilizes the emotions behind the text to find the\nmost important words within that text. We experiment with this model on our\ndataset and later present the analysis. Additionally, we evaluate our proposed\nmethod on different corpora and show new state-of-the-art results with respect\nto offensive language detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03104v2",
            "date": "2019-09-06 19:44:48+00:00",
            "title": "Efficient Sentence Embedding using Discrete Cosine Transform",
            "authors": [
                "Nada Almarwani",
                "Hanan Aldarmaki",
                "Mona Diab"
            ],
            "abstract": "Vector averaging remains one of the most popular sentence embedding methods\nin spite of its obvious disregard for syntactic structure. While more complex\nsequential or convolutional networks potentially yield superior classification\nperformance, the improvements in classification accuracy are typically mediocre\ncompared to the simple vector averaging. As an efficient alternative, we\npropose the use of discrete cosine transform (DCT) to compress word sequences\nin an order-preserving manner. The lower order DCT coefficients represent the\noverall feature patterns in sentences, which results in suitable embeddings for\ntasks that could benefit from syntactic features. Our results in semantic\nprobing tasks demonstrate that DCT embeddings indeed preserve more syntactic\ninformation compared with vector averaging. With practically equivalent\ncomplexity, the model yields better overall performance in downstream\nclassification tasks that correlate with syntactic features, which illustrates\nthe capacity of DCT to preserve word order information.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03135v1",
            "date": "2019-09-06 21:49:47+00:00",
            "title": "To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguation",
            "authors": [
                "Andrey Kutuzov",
                "Elizaveta Kuzmenko"
            ],
            "abstract": "We critically evaluate the widespread assumption that deep learning NLP\nmodels do not require lemmatized input. To test this, we trained versions of\ncontextualised word embedding ELMo models on raw tokenized corpora and on the\ncorpora with word tokens replaced by their lemmas. Then, these models were\nevaluated on the word sense disambiguation task. This was done for the English\nand Russian languages.\n  The experiments showed that while lemmatization is indeed not necessary for\nEnglish, the situation is different for Russian. It seems that for\nrich-morphology languages, using lemmatized training and testing data yields\nsmall but consistent improvements: at least for word sense disambiguation. This\nmeans that the decisions about text pre-processing before training ELMo should\nconsider the linguistic nature of the language in question.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03149v3",
            "date": "2019-09-06 23:29:57+00:00",
            "title": "Enhancing Machine Translation with Dependency-Aware Self-Attention",
            "authors": [
                "Emanuele Bugliarello",
                "Naoaki Okazaki"
            ],
            "abstract": "Most neural machine translation models only rely on pairs of parallel\nsentences, assuming syntactic information is automatically learned by an\nattention mechanism. In this work, we investigate different approaches to\nincorporate syntactic knowledge in the Transformer model and also propose a\nnovel, parameter-free, dependency-aware self-attention mechanism that improves\nits translation quality, especially for long sentences and in low-resource\nscenarios. We show the efficacy of each approach on WMT English-German and\nEnglish-Turkish, and WAT English-Japanese translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03186v2",
            "date": "2019-09-07 04:33:26+00:00",
            "title": "On Extractive and Abstractive Neural Document Summarization with Transformer Language Models",
            "authors": [
                "Sandeep Subramanian",
                "Raymond Li",
                "Jonathan Pilault",
                "Christopher Pal"
            ],
            "abstract": "We present a method to produce abstractive summaries of long documents that\nexceed several thousand words via neural abstractive summarization. We perform\na simple extractive step before generating a summary, which is then used to\ncondition the transformer language model on relevant information before being\ntasked with generating a summary. We show that this extractive step\nsignificantly improves summarization results. We also show that this approach\nproduces more abstractive summaries compared to prior work that employs a copy\nmechanism while still achieving higher rouge scores. Note: The abstract above\nwas not written by the authors, it was generated by one of the models presented\nin this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03223v1",
            "date": "2019-09-07 09:14:18+00:00",
            "title": "Deleter: Leveraging BERT to Perform Unsupervised Successive Text Compression",
            "authors": [
                "Tong Niu",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Text compression has diverse applications such as Summarization, Reading\nComprehension and Text Editing. However, almost all existing approaches require\neither hand-crafted features, syntactic labels or parallel data. Even for one\nthat achieves this task in an unsupervised setting, its architecture\nnecessitates a task-specific autoencoder. Moreover, these models only generate\none compressed sentence for each source input, so that adapting to different\nstyle requirements (e.g. length) for the final output usually implies\nretraining the model from scratch. In this work, we propose a fully\nunsupervised model, Deleter, that is able to discover an \"optimal deletion\npath\" for an arbitrary sentence, where each intermediate sequence along the\npath is a coherent subsequence of the previous one. This approach relies\nexclusively on a pretrained bidirectional language model (BERT) to score each\ncandidate deletion based on the average Perplexity of the resulting sentence\nand performs progressive greedy lookahead search to select the best deletion\nfor each step. We apply Deleter to the task of extractive Sentence Compression,\nand found that our model is competitive with state-of-the-art supervised models\ntrained on 1.02 million in-domain examples with similar compression ratio.\nQualitative analysis, as well as automatic and human evaluations both verify\nthat our model produces high-quality compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03227v4",
            "date": "2019-09-07 09:40:20+00:00",
            "title": "A Novel Cascade Binary Tagging Framework for Relational Triple Extraction",
            "authors": [
                "Zhepei Wei",
                "Jianlin Su",
                "Yue Wang",
                "Yuan Tian",
                "Yi Chang"
            ],
            "abstract": "Extracting relational triples from unstructured text is crucial for\nlarge-scale knowledge graph construction. However, few existing works excel in\nsolving the overlapping triple problem where multiple relational triples in the\nsame sentence share the same entities. In this work, we introduce a fresh\nperspective to revisit the relational triple extraction task and propose a\nnovel cascade binary tagging framework (CasRel) derived from a principled\nproblem formulation. Instead of treating relations as discrete labels as in\nprevious works, our new framework models relations as functions that map\nsubjects to objects in a sentence, which naturally handles the overlapping\nproblem. Experiments show that the CasRel framework already outperforms\nstate-of-the-art methods even when its encoder module uses a randomly\ninitialized BERT encoder, showing the power of the new tagging framework. It\nenjoys further performance boost when employing a pre-trained BERT encoder,\noutperforming the strongest baseline by 17.5 and 30.2 absolute gain in F1-score\non two public datasets NYT and WebNLG, respectively. In-depth analysis on\ndifferent scenarios of overlapping triples shows that the method delivers\nconsistent performance gain across all these scenarios. The source code and\ndata are released online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03285v1",
            "date": "2019-09-07 15:23:37+00:00",
            "title": "Semantic Role Labeling with Iterative Structure Refinement",
            "authors": [
                "Chunchuan Lyu",
                "Shay B. Cohen",
                "Ivan Titov"
            ],
            "abstract": "Modern state-of-the-art Semantic Role Labeling (SRL) methods rely on\nexpressive sentence encoders (e.g., multi-layer LSTMs) but tend to model only\nlocal (if any) interactions between individual argument labeling decisions.\nThis contrasts with earlier work and also with the intuition that the labels of\nindividual arguments are strongly interdependent. We model interactions between\nargument labeling decisions through {\\it iterative refinement}. Starting with\nan output produced by a factorized model, we iteratively refine it using a\nrefinement network. Instead of modeling arbitrary interactions among roles and\nwords, we encode prior knowledge about the SRL problem by designing a\nrestricted network architecture capturing non-local interactions. This modeling\nchoice prevents overfitting and results in an effective model, outperforming\nstrong factorized baseline models on all 7 CoNLL-2009 languages, and achieving\nstate-of-the-art results on 5 of them, including English.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03341v2",
            "date": "2019-09-07 21:29:46+00:00",
            "title": "Neural Machine Translation with Byte-Level Subwords",
            "authors": [
                "Changhan Wang",
                "Kyunghyun Cho",
                "Jiatao Gu"
            ],
            "abstract": "Almost all existing machine translation models are built on top of\ncharacter-based vocabularies: characters, subwords or words. Rare characters\nfrom noisy text or character-rich languages such as Japanese and Chinese\nhowever can unnecessarily take up vocabulary slots and limit its compactness.\nRepresenting text at the level of bytes and using the 256 byte set as\nvocabulary is a potential solution to this issue. High computational cost has\nhowever prevented it from being widely deployed or used in practice. In this\npaper, we investigate byte-level subwords, specifically byte-level BPE (BBPE),\nwhich is compacter than character vocabulary and has no out-of-vocabulary\ntokens, but is more efficient than using pure bytes only is. We claim that\ncontextualizing BBPE embeddings is necessary, which can be implemented by a\nconvolutional or recurrent layer. Our experiments show that BBPE has comparable\nperformance to BPE while its size is only 1/8 of that for BPE. In the\nmultilingual setting, BBPE maximizes vocabulary sharing across many languages\nand achieves better translation quality. Moreover, we show that BBPE enables\ntransferring models between languages with non-overlapping character sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03343v4",
            "date": "2019-09-07 21:40:04+00:00",
            "title": "Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts",
            "authors": [
                "Jack Merullo",
                "Luke Yeh",
                "Abram Handler",
                "Alvin Grissom II",
                "Brendan O'Connor",
                "Mohit Iyyer"
            ],
            "abstract": "Sports broadcasters inject drama into play-by-play commentary by building\nteam and player narratives through subjective analyses and anecdotes. Prior\nstudies based on small datasets and manual coding show that such theatrics\nevince commentator bias in sports broadcasts. To examine this phenomenon, we\nassemble FOOTBALL, which contains 1,455 broadcast transcripts from American\nfootball games across six decades that are automatically annotated with 250K\nplayer mentions and linked with racial metadata. We identify major confounding\nfactors for researchers examining racial bias in FOOTBALL, and perform a\ncomputational analysis that supports conclusions from prior social science\nstudies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03368v1",
            "date": "2019-09-08 02:04:32+00:00",
            "title": "Designing and Interpreting Probes with Control Tasks",
            "authors": [
                "John Hewitt",
                "Percy Liang"
            ],
            "abstract": "Probes, supervised models trained to predict properties (like\nparts-of-speech) from representations (like ELMo), have achieved high accuracy\non a range of linguistic tasks. But does this mean that the representations\nencode linguistic structure or just that the probe has learned the linguistic\ntask? In this paper, we propose control tasks, which associate word types with\nrandom outputs, to complement linguistic tasks. By construction, these tasks\ncan only be learned by the probe itself. So a good probe, (one that reflects\nthe representation), should be selective, achieving high linguistic task\naccuracy and low control task accuracy. The selectivity of a probe puts\nlinguistic task accuracy in context with the probe's capacity to memorize from\nword types. We construct control tasks for English part-of-speech tagging and\ndependency edge prediction, and show that popular probes on ELMo\nrepresentations are not selective. We also find that dropout, commonly used to\ncontrol probe complexity, is ineffective for improving selectivity of MLPs, but\nthat other forms of regularization are effective. Finally, we find that while\nprobes on the first layer of ELMo yield slightly better part-of-speech tagging\naccuracy than the second, probes on the second layer are substantially more\nselective, which raises the question of which layer better represents\nparts-of-speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03405v2",
            "date": "2019-09-08 08:55:09+00:00",
            "title": "Symmetric Regularization based BERT for Pair-wise Semantic Reasoning",
            "authors": [
                "Weidi Xu",
                "Xingyi Cheng",
                "Kunlong Chen",
                "Wei Wang",
                "Bin Bi",
                "Ming Yan",
                "Chen Wu",
                "Luo Si",
                "Wei Chu",
                "Taifeng Wang"
            ],
            "abstract": "The ability of semantic reasoning over the sentence pair is essential for\nmany natural language understanding tasks, e.g., natural language inference and\nmachine reading comprehension. A recent significant improvement in these tasks\ncomes from BERT. As reported, the next sentence prediction (NSP) in BERT, which\nlearns the contextual relationship between two sentences, is of great\nsignificance for downstream problems with sentence-pair input. Despite the\neffectiveness of NSP, we suggest that NSP still lacks the essential signal to\ndistinguish between entailment and shallow correlation. To remedy this, we\npropose to augment the NSP task to a 3-class categorization task, which\nincludes a category for previous sentence prediction (PSP). The involvement of\nPSP encourages the model to focus on the informative semantics to determine the\nsentence order, thereby improves the ability of semantic understanding. This\nsimple modification yields remarkable improvement against vanilla BERT. To\nfurther incorporate the document-level information, the scope of NSP and PSP is\nexpanded into a broader range, i.e., NSP and PSP also include close but\nnonsuccessive sentences, the noise of which is mitigated by the label-smoothing\ntechnique. Both qualitative and quantitative experimental results demonstrate\nthe effectiveness of the proposed method. Our method consistently improves the\nperformance on the NLI and MRC benchmarks, including the challenging HANS\ndataset \\cite{hans}, suggesting that the document-level task is still promising\nfor the pre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03415v1",
            "date": "2019-09-08 09:47:56+00:00",
            "title": "Commonsense Knowledge + BERT for Level 2 Reading Comprehension Ability Test",
            "authors": [
                "Yidan Hu",
                "Gongqi Lin",
                "Yuan Miao",
                "Chunyan Miao"
            ],
            "abstract": "Commonsense knowledge plays an important role when we read. The performance\nof BERT on SQuAD dataset shows that the accuracy of BERT can be better than\nhuman users. However, it does not mean that computers can surpass the human\nbeing in reading comprehension. CommonsenseQA is a large-scale dataset which is\ndesigned based on commonsense knowledge. BERT only achieved an accuracy of\n55.9% on it. The result shows that computers cannot apply commonsense knowledge\nlike human beings to answer questions. Comprehension Ability Test (CAT) divided\nthe reading comprehension ability at four levels. We can achieve human like\ncomprehension ability level by level. BERT has performed well at level 1 which\ndoes not require common knowledge. In this research, we propose a system which\naims to allow computers to read articles and answer related questions with\ncommonsense knowledge like a human being for CAT level 2. This system consists\nof three parts. Firstly, we built a commonsense knowledge graph; and then\nautomatically constructed the commonsense knowledge question dataset according\nto it. Finally, BERT is combined with the commonsense knowledge to achieve the\nreading comprehension ability at CAT level 2. Experiments show that it can pass\nthe CAT as long as the required common knowledge is included in the knowledge\nbase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03453v1",
            "date": "2019-09-08 12:52:46+00:00",
            "title": "May I Check Again? -- A simple but efficient way to generate and use contextual dictionaries for Named Entity Recognition. Application to French Legal Texts",
            "authors": [
                "Valentin Barriere",
                "Amaury Fouret"
            ],
            "abstract": "In this paper we present a new method to learn a model robust to typos for a\nNamed Entity Recognition task. Our improvement over existing methods helps the\nmodel to take into account the context of the sentence inside a court decision\nin order to recognize an entity with a typo. We used state-of-the-art models\nand enriched the last layer of the neural network with high-level information\nlinked with the potential of the word to be a certain type of entity. More\nprecisely, we utilized the similarities between the word and the potential\nentity candidates in the tagged sentence context. The experiments on a dataset\nof French court decisions show a reduction of the relative F1-score error of\n32%, upgrading the score obtained with the most competitive fine-tuned\nstate-of-the-art system from 94.85% to 96.52%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03477v2",
            "date": "2019-09-08 14:21:54+00:00",
            "title": "Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks",
            "authors": [
                "Chen Zhang",
                "Qiuchi Li",
                "Dawei Song"
            ],
            "abstract": "Due to their inherent capability in semantic alignment of aspects and their\ncontext words, attention mechanism and Convolutional Neural Networks (CNNs) are\nwidely applied for aspect-based sentiment classification. However, these models\nlack a mechanism to account for relevant syntactical constraints and long-range\nword dependencies, and hence may mistakenly recognize syntactically irrelevant\ncontextual words as clues for judging aspect sentiment. To tackle this problem,\nwe propose to build a Graph Convolutional Network (GCN) over the dependency\ntree of a sentence to exploit syntactical information and word dependencies.\nBased on it, a novel aspect-specific sentiment classification framework is\nraised. Experiments on three benchmarking collections illustrate that our\nproposed model has comparable effectiveness to a range of state-of-the-art\nmodels, and further demonstrate that both syntactical information and\nlong-range word dependencies are properly captured by the graph convolution\nstructure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03524v2",
            "date": "2019-09-08 18:25:48+00:00",
            "title": "Evaluating Topic Quality with Posterior Variability",
            "authors": [
                "Linzi Xing",
                "Michael J. Paul",
                "Giuseppe Carenini"
            ],
            "abstract": "Probabilistic topic models such as latent Dirichlet allocation (LDA) are\npopularly used with Bayesian inference methods such as Gibbs sampling to learn\nposterior distributions over topic model parameters. We derive a novel measure\nof LDA topic quality using the variability of the posterior distributions.\nCompared to several existing baselines for automatic topic evaluation, the\nproposed metric achieves state-of-the-art correlations with human judgments of\ntopic quality in experiments on three corpora. We additionally demonstrate that\ntopic quality estimation can be further improved using a supervised estimator\nthat combines multiple metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03544v2",
            "date": "2019-09-08 21:00:05+00:00",
            "title": "Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER",
            "authors": [
                "Milan Straka",
                "Jana Strakov\u00e1",
                "Jan Haji\u010d"
            ],
            "abstract": "Contextualized embeddings, which capture appropriate word meaning depending\non context, have recently been proposed. We evaluate two meth ods for\nprecomputing such embeddings, BERT and Flair, on four Czech text processing\ntasks: part-of-speech (POS) tagging, lemmatization, dependency pars ing and\nnamed entity recognition (NER). The first three tasks, POS tagging,\nlemmatization and dependency parsing, are evaluated on two corpora: the Prague\nDependency Treebank 3.5 and the Universal Dependencies 2.3. The named entity\nrecognition (NER) is evaluated on the Czech Named Entity Corpus 1.1 and 2.0. We\nreport state-of-the-art results for the above mentioned tasks and corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03546v2",
            "date": "2019-09-08 21:19:09+00:00",
            "title": "Entity, Relation, and Event Extraction with Contextualized Span Representations",
            "authors": [
                "David Wadden",
                "Ulme Wennberg",
                "Yi Luan",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We examine the capabilities of a unified, multi-task framework for three\ninformation extraction tasks: named entity recognition, relation extraction,\nand event extraction. Our framework (called DyGIE++) accomplishes all tasks by\nenumerating, refining, and scoring text spans designed to capture local\n(within-sentence) and global (cross-sentence) context. Our framework achieves\nstate-of-the-art results across all tasks, on four datasets from a variety of\ndomains. We perform experiments comparing different techniques to construct\nspan representations. Contextualized embeddings like BERT perform well at\ncapturing relationships among entities in the same or adjacent sentences, while\ndynamic span graph updates model long-range cross-sentence relationships. For\ninstance, propagating span representations via predicted coreference links can\nenable the model to disambiguate challenging entity mentions. Our code is\npublicly available at https://github.com/dwadden/dygiepp and can be easily\nadapted for new tasks or datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03588v2",
            "date": "2019-09-09 01:59:31+00:00",
            "title": "Unsupervised Paraphrasing by Simulated Annealing",
            "authors": [
                "Xianggen Liu",
                "Lili Mou",
                "Fandong Meng",
                "Hao Zhou",
                "Jie Zhou",
                "Sen Song"
            ],
            "abstract": "Unsupervised paraphrase generation is a promising and important research\ntopic in natural language processing. We propose UPSA, a novel approach that\naccomplishes Unsupervised Paraphrasing by Simulated Annealing. We model\nparaphrase generation as an optimization problem and propose a sophisticated\nobjective function, involving semantic similarity, expression diversity, and\nlanguage fluency of paraphrases. Then, UPSA searches the sentence space towards\nthis objective by performing a sequence of local editing. Our method is\nunsupervised and does not require parallel corpora for training, so it could be\neasily applied to different domains. We evaluate our approach on a variety of\nbenchmark datasets, namely, Quora, Wikianswers, MSCOCO, and Twitter. Extensive\nresults show that UPSA achieves the state-of-the-art performance compared with\nprevious unsupervised methods in terms of both automatic and human evaluations.\nFurther, our approach outperforms most existing domain-adapted supervised\nmodels, showing the generalizability of UPSA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03590v3",
            "date": "2019-09-09 02:04:53+00:00",
            "title": "Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence",
            "authors": [
                "Rui Meng",
                "Xingdi Yuan",
                "Tong Wang",
                "Peter Brusilovsky",
                "Adam Trischler",
                "Daqing He"
            ],
            "abstract": "Recently, concatenating multiple keyphrases as a target sequence has been\nproposed as a new learning paradigm for keyphrase generation. Existing studies\nconcatenate target keyphrases in different orders but no study has examined the\neffects of ordering on models' behavior. In this paper, we propose several\norderings for concatenation and inspect the important factors for training a\nsuccessful keyphrase generation model. By running comprehensive comparisons, we\nobserve one preferable ordering and summarize a number of empirical findings\nand challenges, which can shed light on future research on this line of work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03598v1",
            "date": "2019-09-09 02:45:38+00:00",
            "title": "What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis",
            "authors": [
                "Xiaolei Huang",
                "Jonathan May",
                "Nanyun Peng"
            ],
            "abstract": "Building named entity recognition (NER) models for languages that do not have\nmuch training data is a challenging task. While recent work has shown promising\nresults on cross-lingual transfer from high-resource languages to low-resource\nlanguages, it is unclear what knowledge is transferred. In this paper, we first\npropose a simple and efficient neural architecture for cross-lingual NER.\nExperiments show that our model achieves competitive performance with the\nstate-of-the-art. We further analyze how transfer learning works for\ncross-lingual NER on two transferable factors: sequential order and\nmultilingual embeddings, and investigate how model performance varies across\nentity lengths. Finally, we conduct a case-study on a non-Latin language,\nBengali, which suggests that leveraging knowledge from Wikipedia will be a\npromising direction to further improve the model performances. Our results can\nshed light on future research for improving cross-lingual NER.",
            "journal ref": "published at EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03716v2",
            "date": "2019-09-09 09:26:42+00:00",
            "title": "Improving Neural Question Generation using World Knowledge",
            "authors": [
                "Deepak Gupta",
                "Kaheer Suleman",
                "Mahmoud Adada",
                "Andrew McNamara",
                "Justin Harris"
            ],
            "abstract": "In this paper, we propose a method for incorporating world knowledge (linked\nentities and fine-grained entity types) into a neural question generation\nmodel. This world knowledge helps to encode additional information related to\nthe entities present in the passage required to generate human-like questions.\nWe evaluate our models on both SQuAD and MS MARCO to demonstrate the usefulness\nof the world knowledge features. The proposed world knowledge enriched question\ngeneration model is able to outperform the vanilla neural question generation\nmodel by 1.37 and 1.59 absolute BLEU 4 score on SQuAD and MS MARCO test dataset\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03750v1",
            "date": "2019-09-09 10:45:05+00:00",
            "title": "Combining SMT and NMT Back-Translated Data for Efficient NMT",
            "authors": [
                "Alberto Poncelas",
                "Maja Popovic",
                "Dimitar Shterionov",
                "Gideon Maillette de Buy Wenniger",
                "Andy Way"
            ],
            "abstract": "Neural Machine Translation (NMT) models achieve their best performance when\nlarge sets of parallel data are used for training. Consequently, techniques for\naugmenting the training set have become popular recently. One of these methods\nis back-translation (Sennrich et al., 2016), which consists on generating\nsynthetic sentences by translating a set of monolingual, target-language\nsentences using a Machine Translation (MT) model.\n  Generally, NMT models are used for back-translation. In this work, we analyze\nthe performance of models when the training data is extended with synthetic\ndata using different MT approaches. In particular we investigate\nback-translated data generated not only by NMT but also by Statistical Machine\nTranslation (SMT) models and combinations of both. The results reveal that the\nmodels achieve the best performances when the training set is augmented with\nback-translated data created by merging different MT approaches.",
            "journal ref": "Proceedings of Recent Advances in Natural Language Processing\n  (RANLP 2019). pages 922--931",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03795v1",
            "date": "2019-09-09 12:24:06+00:00",
            "title": "Language learning using Speech to Image retrieval",
            "authors": [
                "Danny Merkx",
                "Stefan L. Frank",
                "Mirjam Ernestus"
            ],
            "abstract": "Humans learn language by interaction with their environment and listening to\nother humans. It should also be possible for computational models to learn\nlanguage directly from speech but so far most approaches require text. We\nimprove on existing neural network approaches to create visually grounded\nembeddings for spoken utterances. Using a combination of a multi-layer GRU,\nimportance sampling, cyclic learning rates, ensembling and vectorial\nself-attention our results show a remarkable increase in image-caption\nretrieval performance over previous work. Furthermore, we investigate which\nlayers in the model learn to recognise words in the input. We find that deeper\nnetwork layers are better at encoding word presence, although the final layer\nhas slightly lower performance. This shows that our visually grounded sentence\nencoder learns to recognise words from the input even though it is not\nexplicitly trained for word recognition.",
            "journal ref": "Proc. Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04002v1",
            "date": "2019-09-09 17:28:40+00:00",
            "title": "The Trumpiest Trump? Identifying a Subject's Most Characteristic Tweets",
            "authors": [
                "Charuta Pethe",
                "Steven Skiena"
            ],
            "abstract": "The sequence of documents produced by any given author varies in style and\ncontent, but some documents are more typical or representative of the source\nthan others. We quantify the extent to which a given short text is\ncharacteristic of a specific person, using a dataset of tweets from fifteen\ncelebrities. Such analysis is useful for generating excerpts of high-volume\nTwitter profiles, and understanding how representativeness relates to tweet\npopularity. We first consider the related task of binary author detection (is x\nthe author of text T?), and report a test accuracy of 90.37% for the best of\nfive approaches to this problem. We then use these models to compute\ncharacterization scores among all of an author's texts. A user study shows\nhuman evaluators agree with our characterization model for all 15 celebrities\nin our dataset, each with p-value < 0.05. We use these classifiers to show\nsurprisingly strong correlations between characterization scores and the\npopularity of the associated texts. Indeed, we demonstrate a statistically\nsignificant correlation between this score and tweet popularity\n(likes/replies/retweets) for 13 of the 15 celebrities in our study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04054v2",
            "date": "2019-09-09 18:00:05+00:00",
            "title": "Pretrained Language Models for Sequential Sentence Classification",
            "authors": [
                "Arman Cohan",
                "Iz Beltagy",
                "Daniel King",
                "Bhavana Dalvi",
                "Daniel S. Weld"
            ],
            "abstract": "As a step toward better document-level understanding, we explore\nclassification of a sequence of sentences into their corresponding categories,\na task that requires understanding sentences in context of the document. Recent\nsuccessful models for this task have used hierarchical models to contextualize\nsentence representations, and Conditional Random Fields (CRFs) to incorporate\ndependencies between subsequent labels. In this work, we show that pretrained\nlanguage models, BERT (Devlin et al., 2018) in particular, can be used for this\ntask to capture contextual dependencies without the need for hierarchical\nencoding nor a CRF. Specifically, we construct a joint sentence representation\nthat allows BERT Transformer layers to directly utilize contextual information\nfrom all words in all sentences. Our approach achieves state-of-the-art results\non four datasets, including a new dataset of structured scientific abstracts.",
            "journal ref": "Proceedings of the 2019 Conference on Empirical Methods in Natural\n  Language Processing and the 9th International Joint Conference on Natural\n  Language Processing (EMNLP-IJCNLP) (2019) 3693-3699",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04130v1",
            "date": "2019-09-09 20:01:51+00:00",
            "title": "Reverse Transfer Learning: Can Word Embeddings Trained for Different NLP Tasks Improve Neural Language Models?",
            "authors": [
                "Lyan Verwimp",
                "Jerome R. Bellegarda"
            ],
            "abstract": "Natural language processing (NLP) tasks tend to suffer from a paucity of\nsuitably annotated training data, hence the recent success of transfer learning\nacross a wide variety of them. The typical recipe involves: (i) training a\ndeep, possibly bidirectional, neural network with an objective related to\nlanguage modeling, for which training data is plentiful; and (ii) using the\ntrained network to derive contextual representations that are far richer than\nstandard linear word embeddings such as word2vec, and thus result in important\ngains. In this work, we wonder whether the opposite perspective is also true:\ncan contextual representations trained for different NLP tasks improve language\nmodeling itself? Since language models (LMs) are predominantly locally\noptimized, other NLP tasks may help them make better predictions based on the\nentire semantic fabric of a document. We test the performance of several types\nof pre-trained embeddings in neural LMs, and we investigate whether it is\npossible to make the LM more aware of global semantic information through\nembeddings pre-trained with a domain classification model. Initial experiments\nsuggest that as long as the proper objective criterion is used during training,\npre-trained embeddings are likely to be beneficial for neural language\nmodeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04164v2",
            "date": "2019-09-09 21:18:50+00:00",
            "title": "Knowledge Enhanced Contextual Word Representations",
            "authors": [
                "Matthew E. Peters",
                "Mark Neumann",
                "Robert L. Logan IV",
                "Roy Schwartz",
                "Vidur Joshi",
                "Sameer Singh",
                "Noah A. Smith"
            ],
            "abstract": "Contextual word representations, typically trained on unstructured, unlabeled\ntext, do not contain any explicit grounding to real world entities and are\noften unable to remember facts about those entities. We propose a general\nmethod to embed multiple knowledge bases (KBs) into large scale models, and\nthereby enhance their representations with structured, human-curated knowledge.\nFor each KB, we first use an integrated entity linker to retrieve relevant\nentity embeddings, then update contextual word representations via a form of\nword-to-entity attention. In contrast to previous approaches, the entity\nlinkers and self-supervised language modeling objective are jointly trained\nend-to-end in a multitask setting that combines a small amount of entity\nlinking supervision with a large amount of raw text. After integrating WordNet\nand a subset of Wikipedia into BERT, the knowledge enhanced BERT (KnowBert)\ndemonstrates improved perplexity, ability to recall facts as measured in a\nprobing task and downstream performance on relationship extraction, entity\ntyping, and word sense disambiguation. KnowBert's runtime is comparable to\nBERT's and it scales to large KBs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04165v1",
            "date": "2019-09-09 21:20:36+00:00",
            "title": "Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs",
            "authors": [
                "Bailin Wang",
                "Ivan Titov",
                "Mirella Lapata"
            ],
            "abstract": "Semantic parsing aims to map natural language utterances onto machine\ninterpretable meaning representations, aka programs whose execution against a\nreal-world environment produces a denotation. Weakly-supervised semantic\nparsers are trained on utterance-denotation pairs treating programs as latent.\nThe task is challenging due to the large search space and spuriousness of\nprograms which may execute to the correct answer but do not generalize to\nunseen examples. Our goal is to instill an inductive bias in the parser to help\nit distinguish between spurious and correct programs. We capitalize on the\nintuition that correct programs would likely respect certain structural\nconstraints were they to be aligned to the question (e.g., program fragments\nare unlikely to align to overlapping text spans) and propose to model\nalignments as structured latent variables. In order to make the\nlatent-alignment framework tractable, we decompose the parsing task into (1)\npredicting a partial \"abstract program\" and (2) refining it while modeling\nstructured alignments with differential dynamic programming. We obtain\nstate-of-the-art performance on the WIKITABLEQUESTIONS and WIKISQL datasets.\nWhen compared to a standard attention baseline, we observe that the proposed\nstructured-alignment mechanism is highly beneficial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04181v3",
            "date": "2019-09-09 22:08:12+00:00",
            "title": "BERT-Based Arabic Social Media Author Profiling",
            "authors": [
                "Chiyu Zhang",
                "Muhammad Abdul-Mageed"
            ],
            "abstract": "We report our models for detecting age, language variety, and gender from\nsocial media data in the context of the Arabic author profiling and deception\ndetection shared task (APDA). We build simple models based on pre-trained\nbidirectional encoders from transformers (BERT). We first fine-tune the\npre-trained BERT model on each of the three datasets with shared task released\ndata. Then we augment shared task data with in-house data for gender and\ndialect, showing the utility of augmenting training data. Our best models on\nthe shared task test data are acquired with a majority voting of various BERT\nmodels trained under different data conditions. We acquire 54.72% accuracy for\nage, 93.75% for dialect, 81.67% for gender, and 40.97% joint accuracy across\nthe three tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04225v4",
            "date": "2019-09-10 01:30:23+00:00",
            "title": "Improving the Explainability of Neural Sentiment Classifiers via Data Augmentation",
            "authors": [
                "Hanjie Chen",
                "Yangfeng Ji"
            ],
            "abstract": "Sentiment analysis has been widely used by businesses for social media\nopinion mining, especially in the financial services industry, where customers'\nfeedbacks are critical for companies. Recent progress of neural network models\nhas achieved remarkable performance on sentiment classification, while the lack\nof classification interpretation may raise the trustworthy and many other\nissues in practice. In this work, we study the problem of improving the\nexplainability of existing sentiment classifiers. We propose two data\naugmentation methods that create additional training examples to help improve\nmodel explainability: one method with a predefined sentiment word list as\nexternal knowledge and the other with adversarial examples. We test the\nproposed methods on both CNN and RNN classifiers with three benchmark sentiment\ndatasets. The model explainability is assessed by both human evaluators and a\nsimple automatic evaluation measurement. Experiments show the proposed data\naugmentation methods significantly improve the explainability of both neural\nclassifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04273v3",
            "date": "2019-09-10 04:08:10+00:00",
            "title": "Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy",
            "authors": [
                "Bowen Yu",
                "Zhenyu Zhang",
                "Xiaobo Shu",
                "Yubin Wang",
                "Tingwen Liu",
                "Bin Wang",
                "Sujian Li"
            ],
            "abstract": "Joint extraction of entities and relations aims to detect entity pairs along\nwith their relations using a single model. Prior work typically solves this\ntask in the extract-then-classify or unified labeling manner. However, these\nmethods either suffer from the redundant entity pairs, or ignore the important\ninner structure in the process of extracting entities and relations. To address\nthese limitations, in this paper, we first decompose the joint extraction task\ninto two interrelated subtasks, namely HE extraction and TER extraction. The\nformer subtask is to distinguish all head-entities that may be involved with\ntarget relations, and the latter is to identify corresponding tail-entities and\nrelations for each extracted head-entity. Next, these two subtasks are further\ndeconstructed into several sequence labeling problems based on our proposed\nspan-based tagging scheme, which are conveniently solved by a hierarchical\nboundary tagger and a multi-span decoding algorithm. Owing to the reasonable\ndecomposition strategy, our model can fully capture the semantic\ninterdependency between different steps, as well as reduce noise from\nirrelevant entity pairs. Experimental results show that our method outperforms\nprevious work by 5.2%, 5.9% and 21.5% (F1 score), achieving a new\nstate-of-the-art on three public datasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04315v1",
            "date": "2019-09-10 06:31:09+00:00",
            "title": "Fine-grained Knowledge Fusion for Sequence Labeling Domain Adaptation",
            "authors": [
                "Huiyun Yang",
                "Shujian Huang",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "In sequence labeling, previous domain adaptation methods focus on the\nadaptation from the source domain to the entire target domain without\nconsidering the diversity of individual target domain samples, which may lead\nto negative transfer results for certain samples. Besides, an important\ncharacteristic of sequence labeling tasks is that different elements within a\ngiven sample may also have diverse domain relevance, which requires further\nconsideration. To take the multi-level domain relevance discrepancy into\naccount, in this paper, we propose a fine-grained knowledge fusion model with\nthe domain relevance modeling scheme to control the balance between learning\nfrom the target domain data and learning from the source domain model.\nExperiments on three sequence labeling tasks show that our fine-grained\nknowledge fusion model outperforms strong baselines and other state-of-the-art\nsequence labeling domain adaptation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04448v1",
            "date": "2019-09-10 12:50:06+00:00",
            "title": "A Corpus-free State2Seq User Simulator for Task-oriented Dialogue",
            "authors": [
                "Yutai Hou",
                "Meng Fang",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "Recent reinforcement learning algorithms for task-oriented dialogue system\nabsorbs a lot of interest. However, an unavoidable obstacle for training such\nalgorithms is that annotated dialogue corpora are often unavailable. One of the\npopular approaches addressing this is to train a dialogue agent with a user\nsimulator. Traditional user simulators are built upon a set of dialogue rules\nand therefore lack response diversity. This severely limits the simulated cases\nfor agent training. Later data-driven user models work better in diversity but\nsuffer from data scarcity problem. To remedy this, we design a new corpus-free\nframework that taking advantage of their benefits. The framework builds a user\nsimulator by first generating diverse dialogue data from templates and then\nbuild a new State2Seq user simulator on the data. To enhance the performance,\nwe propose the State2Seq user simulator model to efficiently leverage dialogue\nstate and history. Experiment results on an open dataset show that our user\nsimulator helps agents achieve an improvement of 6.36% on success rate.\nState2Seq model outperforms the seq2seq baseline for 1.9 F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04453v1",
            "date": "2019-09-10 12:59:10+00:00",
            "title": "Select and Attend: Towards Controllable Content Selection in Text Generation",
            "authors": [
                "Xiaoyu Shen",
                "Jun Suzuki",
                "Kentaro Inui",
                "Hui Su",
                "Dietrich Klakow",
                "Satoshi Sekine"
            ],
            "abstract": "Many text generation tasks naturally contain two steps: content selection and\nsurface realization. Current neural encoder-decoder models conflate both steps\ninto a black-box architecture. As a result, the content to be described in the\ntext cannot be explicitly controlled. This paper tackles this problem by\ndecoupling content selection from the decoder. The decoupled content selection\nis human interpretable, whose value can be manually manipulated to control the\ncontent of generated text. The model can be trained end-to-end without human\nannotations by maximizing a lower bound of the marginal likelihood. We further\npropose an effective way to trade-off between performance and controllability\nwith a single adjustable hyperparameter. In both data-to-text and headline\ngeneration tasks, our model achieves promising results, paving the way for\ncontrollable content selection in text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04547v1",
            "date": "2019-09-10 15:00:23+00:00",
            "title": "What do Deep Networks Like to Read?",
            "authors": [
                "Jonas Pfeiffer",
                "Aishwarya Kamath",
                "Iryna Gurevych",
                "Sebastian Ruder"
            ],
            "abstract": "Recent research towards understanding neural networks probes models in a\ntop-down manner, but is only able to identify model tendencies that are known a\npriori. We propose Susceptibility Identification through Fine-Tuning (SIFT), a\nnovel abstractive method that uncovers a model's preferences without imposing\nany prior. By fine-tuning an autoencoder with the gradients from a fixed\nclassifier, we are able to extract propensities that characterize different\nkinds of classifiers in a bottom-up manner. We further leverage the SIFT\narchitecture to rephrase sentences in order to predict the opposing class of\nthe ground truth label, uncovering potential artifacts encoded in the fixed\nclassification model. We evaluate our method on three diverse tasks with four\ndifferent models. We contrast the propensities of the models as well as\nreproduce artifacts reported in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04556v1",
            "date": "2019-09-10 15:06:58+00:00",
            "title": "Human Languages in Source Code: Auto-Translation for Localized Instruction",
            "authors": [
                "Chris Piech",
                "Sami Abu-El-Haija"
            ],
            "abstract": "Computer science education has promised open access around the world, but\naccess is largely determined by what human language you speak. As younger\nstudents learn computer science it is less appropriate to assume that they\nshould learn English beforehand. To that end we present CodeInternational, the\nfirst tool to translate code between human languages. To develop a theory of\nnon-English code, and inform our translation decisions, we conduct a study of\npublic code repositories on GitHub. The study is to the best of our knowledge\nthe first on human-language in code and covers 2.9 million Java repositories.\nTo demonstrate CodeInternational's educational utility, we build an interactive\nversion of the popular English-language Karel reader and translate it into 100\nspoken languages. Our translations have already been used in classrooms around\nthe world, and represent a first step in an important open CS-education\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04758v3",
            "date": "2019-09-10 21:17:20+00:00",
            "title": "Scientific Discourse Tagging for Evidence Extraction",
            "authors": [
                "Xiangci Li",
                "Gully Burns",
                "Nanyun Peng"
            ],
            "abstract": "Evidence plays a crucial role in any biomedical research narrative, providing\njustification for some claims and refutation for others. We seek to build\nmodels of scientific argument using information extraction methods from\nfull-text papers. We present the capability of automatically extracting text\nfragments from primary research papers that describe the evidence presented in\nthat paper's figures, which arguably provides the raw material of any\nscientific argument made within the paper. We apply richly contextualized deep\nrepresentation learning pre-trained on biomedical domain corpus to the analysis\nof scientific discourse structures and the extraction of \"evidence fragments\"\n(i.e., the text in the results section describing data presented in a specified\nsubfigure) from a set of biomedical experimental research articles. We first\ndemonstrate our state-of-the-art scientific discourse tagger on two scientific\ndiscourse tagging datasets and its transferability to new datasets. We then\nshow the benefit of leveraging scientific discourse tags for downstream tasks\nsuch as claim-extraction and evidence fragment detection. Our work demonstrates\nthe potential of using evidence fragments derived from figure spans for\nimproving the quality of scientific claims by cataloging, indexing and reusing\nevidence fragments as independent documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04793v2",
            "date": "2019-09-10 23:43:05+00:00",
            "title": "Definition Frames: Using Definitions for Hybrid Concept Representations",
            "authors": [
                "Evangelia Spiliopoulou",
                "Artidoro Pagnoni",
                "Eduard Hovy"
            ],
            "abstract": "Advances in word representations have shown tremendous improvements in\ndownstream NLP tasks, but lack semantic interpretability. In this paper, we\nintroduce Definition Frames (DF), a matrix distributed representation extracted\nfrom definitions, where each dimension is semantically interpretable. DF\ndimensions correspond to the Qualia structure relations: a set of relations\nthat uniquely define a term. Our results show that DFs have competitive\nperformance with other distributional semantic approaches on word similarity\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04879v1",
            "date": "2019-09-11 07:14:58+00:00",
            "title": "Dynamic Fusion: Attentional Language Model for Neural Machine Translation",
            "authors": [
                "Michiki Kurosawa",
                "Mamoru Komachi"
            ],
            "abstract": "Neural Machine Translation (NMT) can be used to generate fluent output. As\nsuch, language models have been investigated for incorporation with NMT. In\nprior investigations, two models have been used: a translation model and a\nlanguage model. The translation model's predictions are weighted by the\nlanguage model with a hand-crafted ratio in advance. However, these approaches\nfail to adopt the language model weighting with regard to the translation\nhistory. In another line of approach, language model prediction is incorporated\ninto the translation model by jointly considering source and target\ninformation. However, this line of approach is limited because it largely\nignores the adequacy of the translation output.\n  Accordingly, this work employs two mechanisms, the translation model and the\nlanguage model, with an attentive architecture to the language model as an\nauxiliary element of the translation model. Compared with previous work in\nEnglish--Japanese machine translation using a language model, the experimental\nresults obtained with the proposed Dynamic Fusion mechanism improve BLEU and\nRank-based Intuitive Bilingual Evaluation Scores (RIBES) scores. Additionally,\nin the analyses of the attention and predictivity of the language model, the\nDynamic Fusion mechanism allows predictive language modeling that conforms to\nthe appropriate grammatical structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04917v2",
            "date": "2019-09-11 08:40:16+00:00",
            "title": "Comprehensive Analysis of Aspect Term Extraction Methods using Various Text Embeddings",
            "authors": [
                "\u0141ukasz Augustyniak",
                "Tomasz Kajdanowicz",
                "Przemys\u0142aw Kazienko"
            ],
            "abstract": "Recently, a variety of model designs and methods have blossomed in the\ncontext of the sentiment analysis domain. However, there is still a lack of\nwide and comprehensive studies of aspect-based sentiment analysis (ABSA). We\nwant to fill this gap and propose a comparison with ablation analysis of aspect\nterm extraction using various text embedding methods. We particularly focused\non architectures based on long short-term memory (LSTM) with optional\nconditional random field (CRF) enhancement using different pre-trained word\nembeddings. Moreover, we analyzed the influence on the performance of extending\nthe word vectorization step with character embedding. The experimental results\non SemEval datasets revealed that not only does bi-directional long short-term\nmemory (BiLSTM) outperform regular LSTM, but also word embedding coverage and\nits source highly affect aspect detection performance. An additional CRF layer\nconsistently improves the results as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05016v1",
            "date": "2019-09-11 12:53:01+00:00",
            "title": "Proposal Towards a Personalized Knowledge-powered Self-play Based Ensemble Dialog System",
            "authors": [
                "Richard Csaky"
            ],
            "abstract": "This is the application document for the 2019 Amazon Alexa competition. We\ngive an overall vision of our conversational experience, as well as a sample\nconversation that we would like our dialog system to achieve by the end of the\ncompetition. We believe personalization, knowledge, and self-play are important\ncomponents towards better chatbots. These are further highlighted by our\ndetailed system architecture proposal and novelty section. Finally, we describe\nhow we would ensure an engaging experience, how this research would impact the\nfield, and related work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05088v1",
            "date": "2019-09-11 14:44:27+00:00",
            "title": "Getting Gender Right in Neural Machine Translation",
            "authors": [
                "Eva Vanmassenhove",
                "Christian Hardmeier",
                "Andy Way"
            ],
            "abstract": "Speakers of different languages must attend to and encode strikingly\ndifferent aspects of the world in order to use their language correctly (Sapir,\n1921; Slobin, 1996). One such difference is related to the way gender is\nexpressed in a language. Saying \"I am happy\" in English, does not encode any\nadditional knowledge of the speaker that uttered the sentence. However, many\nother languages do have grammatical gender systems and so such knowledge would\nbe encoded. In order to correctly translate such a sentence into, say, French,\nthe inherent gender information needs to be retained/recovered. The same\nsentence would become either \"Je suis heureux\", for a male speaker or \"Je suis\nheureuse\" for a female one. Apart from morphological agreement, demographic\nfactors (gender, age, etc.) also influence our use of language in terms of word\nchoices or even on the level of syntactic constructions (Tannen, 1991;\nPennebaker et al., 2003). We integrate gender information into NMT systems. Our\ncontribution is two-fold: (1) the compilation of large datasets with speaker\ninformation for 20 language pairs, and (2) a simple set of experiments that\nincorporate gender information into NMT for multiple language pairs. Our\nexperiments show that adding a gender feature to an NMT system significantly\nimproves the translation quality for some language pairs.",
            "journal ref": "Proceedings of the 2018 Conference on Empirical Methods in Natural\n  Language Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05158v3",
            "date": "2019-09-11 15:53:21+00:00",
            "title": "From English to Code-Switching: Transfer Learning with Strong Morphological Clues",
            "authors": [
                "Gustavo Aguilar",
                "Thamar Solorio"
            ],
            "abstract": "Linguistic Code-switching (CS) is still an understudied phenomenon in natural\nlanguage processing. The NLP community has mostly focused on monolingual and\nmulti-lingual scenarios, but little attention has been given to CS in\nparticular. This is partly because of the lack of resources and annotated data,\ndespite its increasing occurrence in social media platforms. In this paper, we\naim at adapting monolingual models to code-switched text in various tasks.\nSpecifically, we transfer English knowledge from a pre-trained ELMo model to\ndifferent code-switched language pairs (i.e., Nepali-English, Spanish-English,\nand Hindi-English) using the task of language identification. Our method,\nCS-ELMo, is an extension of ELMo with a simple yet effective position-aware\nattention mechanism inside its character convolutions. We show the\neffectiveness of this transfer learning step by outperforming multilingual BERT\nand homologous CS-unaware ELMo models and establishing a new state of the art\nin CS tasks, such as NER and POS tagging. Our technique can be expanded to more\nEnglish-paired code-switched languages, providing more resources to the CS\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05166v1",
            "date": "2019-09-11 16:10:26+00:00",
            "title": "Dependency-Aware Named Entity Recognition with Relative and Global Attentions",
            "authors": [
                "Gustavo Aguilar",
                "Thamar Solorio"
            ],
            "abstract": "Named entity recognition is one of the core tasks in NLP. Although many\nimprovements have been made on this task during the last years, the\nstate-of-the-art systems do not explicitly take into account the recursive\nnature of language. Instead of only treating the text as a plain sequence of\nwords, we incorporate a linguistically-inspired way to recognize entities based\non syntax and tree structures. Our model exploits syntactic relationships among\nwords using a Tree-LSTM guided by dependency trees. Then, we enhance these\nfeatures by applying relative and global attention mechanisms. On the one hand,\nthe relative attention detects the most informative words in the sentence with\nrespect to the word being evaluated. On the other hand, the global attention\nspots the most relevant words in the sequence. Lastly, we linearly project the\nweighted vectors into the tagging space so that a conditional random field\nclassifier predicts the entity labels. Our findings show that the model detects\nwords that disclose the entity types based on their syntactic roles in a\nsentence (e.g., verbs such as speak and write are attended when the entity type\nis PERSON, whereas meet and travel strongly relate to LOCATION). We confirm our\nfindings and establish a new state of the art on two datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05286v1",
            "date": "2019-09-11 18:28:48+00:00",
            "title": "Frustratingly Easy Natural Question Answering",
            "authors": [
                "Lin Pan",
                "Rishav Chakravarti",
                "Anthony Ferritto",
                "Michael Glass",
                "Alfio Gliozzo",
                "Salim Roukos",
                "Radu Florian",
                "Avirup Sil"
            ],
            "abstract": "Existing literature on Question Answering (QA) mostly focuses on algorithmic\nnovelty, data augmentation, or increasingly large pre-trained language models\nlike XLNet and RoBERTa. Additionally, a lot of systems on the QA leaderboards\ndo not have associated research documentation in order to successfully\nreplicate their experiments. In this paper, we outline these algorithmic\ncomponents such as Attention-over-Attention, coupled with data augmentation and\nensembling strategies that have shown to yield state-of-the-art results on\nbenchmark datasets like SQuAD, even achieving super-human performance. Contrary\nto these prior results, when we evaluate on the recently proposed Natural\nQuestions benchmark dataset, we find that an incredibly simple approach of\ntransfer learning from BERT outperforms the previous state-of-the-art system\ntrained on 4 million more examples than ours by 1.9 F1 points. Adding\nensembling strategies further improves that number by 2.3 F1 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05308v1",
            "date": "2019-09-03 20:47:32+00:00",
            "title": "Identifying Editor Roles in Argumentative Writing from Student Revision Histories",
            "authors": [
                "Tazin Afrin",
                "Diane Litman"
            ],
            "abstract": "We present a method for identifying editor roles from students' revision\nbehaviors during argumentative writing. We first develop a method for applying\na topic modeling algorithm to identify a set of editor roles from a vocabulary\ncapturing three aspects of student revision behaviors: operation, purpose, and\nposition. We validate the identified roles by showing that modeling the editor\nroles that students take when revising a paper not only accounts for the\nvariance in revision purposes in our data, but also relates to writing\nimprovement.",
            "journal ref": "In: Artificial Intelligence in Education. AIED 2019. Lecture Notes\n  in Computer Science, vol 11626. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05309v1",
            "date": "2019-09-03 21:02:16+00:00",
            "title": "Annotation and Classification of Sentence-level Revision Improvement",
            "authors": [
                "Tazin Afrin",
                "Diane Litman"
            ],
            "abstract": "Studies of writing revisions rarely focus on revision quality. To address\nthis issue, we introduce a corpus of between-draft revisions of student\nargumentative essays, annotated as to whether each revision improves essay\nquality. We demonstrate a potential usage of our annotations by developing a\nmachine learning model to predict revision improvement. With the goal of\nexpanding training data, we also extract revisions from a dataset edited by\nexpert proofreaders. Our results indicate that blending expert and non-expert\nrevisions increases model performance, with expert data particularly important\nfor predicting low-quality revisions.",
            "journal ref": "In Proceedings of the Thirteenth Workshop on Innovative Use of NLP\n  for Building Educational Applications (pp. 240-246) 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05311v2",
            "date": "2019-09-09 12:43:12+00:00",
            "title": "Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering",
            "authors": [
                "Shangwen Lv",
                "Daya Guo",
                "Jingjing Xu",
                "Duyu Tang",
                "Nan Duan",
                "Ming Gong",
                "Linjun Shou",
                "Daxin Jiang",
                "Guihong Cao",
                "Songlin Hu"
            ],
            "abstract": "Commonsense question answering aims to answer questions which require\nbackground knowledge that is not explicitly expressed in the question. The key\nchallenge is how to obtain evidence from external knowledge and make\npredictions based on the evidence. Recent works either learn to generate\nevidence from human-annotated evidence which is expensive to collect, or\nextract evidence from either structured or unstructured knowledge bases which\nfails to take advantages of both sources. In this work, we propose to\nautomatically extract evidence from heterogeneous knowledge sources, and answer\nquestions based on the extracted evidence. Specifically, we extract evidence\nfrom both structured knowledge base (i.e. ConceptNet) and Wikipedia plain\ntexts. We construct graphs for both sources to obtain the relational structures\nof evidence. Based on these graphs, we propose a graph-based approach\nconsisting of a graph-based contextual word representation learning module and\na graph-based inference module. The first module utilizes graph structural\ninformation to re-define the distance between words for learning better\ncontextual word representations. The second module adopts graph convolutional\nnetwork to encode neighbor information into the representations of nodes, and\naggregates evidence with graph attention mechanism for predicting the final\nanswer. Experimental results on CommonsenseQA dataset illustrate that our\ngraph-based approach over both knowledge sources brings improvement over strong\nbaselines. Our approach achieves the state-of-the-art accuracy (75.3%) on the\nCommonsenseQA leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05316v2",
            "date": "2019-09-11 19:17:03+00:00",
            "title": "What Makes A Good Story? Designing Composite Rewards for Visual Storytelling",
            "authors": [
                "Junjie Hu",
                "Yu Cheng",
                "Zhe Gan",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Graham Neubig"
            ],
            "abstract": "Previous storytelling approaches mostly focused on optimizing traditional\nmetrics such as BLEU, ROUGE and CIDEr. In this paper, we re-examine this\nproblem from a different angle, by looking deep into what defines a\nrealistically-natural and topically-coherent story. To this end, we propose\nthree assessment criteria: relevance, coherence and expressiveness, which we\nobserve through empirical analysis could constitute a \"high-quality\" story to\nthe human eye. Following this quality guideline, we propose a reinforcement\nlearning framework, ReCo-RL, with reward functions designed to capture the\nessence of these quality criteria. Experiments on the Visual Storytelling\nDataset (VIST) with both automatic and human evaluations demonstrate that our\nReCo-RL model achieves better performance than state-of-the-art baselines on\nboth traditional metrics and the proposed new criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05421v1",
            "date": "2019-09-12 01:00:59+00:00",
            "title": "Speculative Beam Search for Simultaneous Translation",
            "authors": [
                "Renjie Zheng",
                "Mingbo Ma",
                "Baigong Zheng",
                "Liang Huang"
            ],
            "abstract": "Beam search is universally used in full-sentence translation but its\napplication to simultaneous translation remains non-trivial, where output words\nare committed on the fly. In particular, the recently proposed wait-k policy\n(Ma et al., 2019a) is a simple and effective method that (after an initial\nwait) commits one output word on receiving each input word, making beam search\nseemingly impossible. To address this challenge, we propose a speculative beam\nsearch algorithm that hallucinates several steps into the future in order to\nreach a more accurate decision, implicitly benefiting from a target language\nmodel. This makes beam search applicable for the first time to the generation\nof a single word in each step. Experiments over diverse language pairs show\nlarge improvements over previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05424v1",
            "date": "2019-09-12 01:16:27+00:00",
            "title": "VizSeq: A Visual Analysis Toolkit for Text Generation Tasks",
            "authors": [
                "Changhan Wang",
                "Anirudh Jain",
                "Danlu Chen",
                "Jiatao Gu"
            ],
            "abstract": "Automatic evaluation of text generation tasks (e.g. machine translation, text\nsummarization, image captioning and video description) usually relies heavily\non task-specific metrics, such as BLEU and ROUGE. They, however, are abstract\nnumbers and are not perfectly aligned with human assessment. This suggests\ninspecting detailed examples as a complement to identify system error patterns.\nIn this paper, we present VizSeq, a visual analysis toolkit for instance-level\nand corpus-level system evaluation on a wide variety of text generation tasks.\nIt supports multimodal sources and multiple text references, providing\nvisualization in Jupyter notebook or a web app interface. It can be used\nlocally or deployed onto public servers for centralized data hosting and\nbenchmarking. It covers most common n-gram based metrics accelerated with\nmultiprocessing, and also provides latest embedding-based metrics such as\nBERTScore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05438v1",
            "date": "2019-09-12 02:47:47+00:00",
            "title": "Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning",
            "authors": [
                "Yibo Sun",
                "Duyu Tang",
                "Nan Duan",
                "Yeyun Gong",
                "Xiaocheng Feng",
                "Bing Qin",
                "Daxin Jiang"
            ],
            "abstract": "Neural semantic parsing has achieved impressive results in recent years, yet\nits success relies on the availability of large amounts of supervised data. Our\ngoal is to learn a neural semantic parser when only prior knowledge about a\nlimited number of simple rules is available, without access to either annotated\nprograms or execution results. Our approach is initialized by rules, and\nimproved in a back-translation paradigm using generated question-program pairs\nfrom the semantic parser and the question generator. A phrase table with\nfrequent mapping patterns is automatically derived, also updated as training\nprogresses, to measure the quality of generated instances. We train the model\nwith model-agnostic meta-learning to guarantee the accuracy and stability on\nexamples covered by rules, and meanwhile acquire the versatility to generalize\nwell on examples uncovered by rules. Results on three benchmark datasets with\ndifferent domains and programs show that our approach incrementally improves\nthe accuracy. On WikiSQL, our best model is comparable to the SOTA system\nlearned from denotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05448v1",
            "date": "2019-09-12 04:20:51+00:00",
            "title": "Uncover the Ground-Truth Relations in Distant Supervision: A Neural Expectation-Maximization Framework",
            "authors": [
                "Junfan Chen",
                "Richong Zhang",
                "Yongyi Mao",
                "Hongyu Guo",
                "Jie Xu"
            ],
            "abstract": "Distant supervision for relation extraction enables one to effectively\nacquire structured relations out of very large text corpora with less human\nefforts. Nevertheless, most of the prior-art models for such tasks assume that\nthe given text can be noisy, but their corresponding labels are clean. Such\nunrealistic assumption is contradictory with the fact that the given labels are\noften noisy as well, thus leading to significant performance degradation of\nthose models on real-world data. To cope with this challenge, we propose a\nnovel label-denoising framework that combines neural network with probabilistic\nmodelling, which naturally takes into account the noisy labels during learning.\nWe empirically demonstrate that our approach significantly improves the current\nart in uncovering the ground-truth relation labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05449v1",
            "date": "2019-09-12 04:21:41+00:00",
            "title": "Visualizing Trends of Key Roles in News Articles",
            "authors": [
                "Chen Xia",
                "Haoxiang Zhang",
                "Jacob Moghtader",
                "Allen Wu",
                "Kai-Wei Chang"
            ],
            "abstract": "There are tons of news articles generated every day reflecting the activities\nof key roles such as people, organizations and political parties. Analyzing\nthese key roles allows us to understand the trends in news. In this paper, we\npresent a demonstration system that visualizes the trend of key roles in news\narticles based on natural language processing techniques. Specifically, we\napply a semantic role labeler and the dynamic word embedding technique to\nunderstand relationships between key roles in the news across different time\nperiods and visualize the trends of key role and news topics change over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05553v1",
            "date": "2019-09-12 10:31:25+00:00",
            "title": "CUNI System for the Building Educational Applications 2019 Shared Task: Grammatical Error Correction",
            "authors": [
                "Jakub N\u00e1plava",
                "Milan Straka"
            ],
            "abstract": "In this paper, we describe our systems submitted to the Building Educational\nApplications (BEA) 2019 Shared Task (Bryant et al., 2019). We participated in\nall three tracks. Our models are NMT systems based on the Transformer model,\nwhich we improve by incorporating several enhancements: applying dropout to\nwhole source and target words, weighting target subwords, averaging model\ncheckpoints, and using the trained model iteratively for correcting the\nintermediate translations. The system in the Restricted Track is trained on the\nprovided corpora with oversampled \"cleaner\" sentences and reaches 59.39 F0.5\nscore on the test set. The system in the Low-Resource Track is trained from\nWikipedia revision histories and reaches 44.13 F0.5 score. Finally, we finetune\nthe system from the Low-Resource Track on restricted data and achieve 64.55\nF0.5 score, placing third in the Unrestricted Track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05708v2",
            "date": "2019-09-12 14:21:54+00:00",
            "title": "Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction",
            "authors": [
                "Yova Kementchedjhieva",
                "Mareike Hartmann",
                "Anders S\u00f8gaard"
            ],
            "abstract": "The task of bilingual dictionary induction (BDI) is commonly used for\nintrinsic evaluation of cross-lingual word embeddings. The largest dataset for\nBDI was generated automatically, so its quality is dubious. We study the\ncomposition and quality of the test sets for five diverse languages from this\ndataset, with concerning findings: (1) a quarter of the data consists of proper\nnouns, which can be hardly indicative of BDI performance, and (2) there are\npervasive gaps in the gold-standard targets. These issues appear to affect the\nranking between cross-lingual embedding systems on individual languages, and\nthe overall degree to which the systems differ in performance. With proper\nnouns removed from the data, the margin between the top two systems included in\nthe study grows from 3.4% to 17.2%. Manual verification of the predictions, on\nthe other hand, reveals that gaps in the gold standard targets artificially\ninflate the margin between the two systems on English to Bulgarian BDI from\n0.1% to 6.7%. We thus suggest that future research either avoids drawing\nconclusions from quantitative results on this BDI dataset, or accompanies such\nevaluation with rigorous error analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05855v2",
            "date": "2019-09-12 17:57:57+00:00",
            "title": "Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset",
            "authors": [
                "Abhinav Rastogi",
                "Xiaoxue Zang",
                "Srinivas Sunkara",
                "Raghav Gupta",
                "Pranav Khaitan"
            ],
            "abstract": "Virtual assistants such as Google Assistant, Alexa and Siri provide a\nconversational interface to a large number of services and APIs spanning\nmultiple domains. Such systems need to support an ever-increasing number of\nservices with possibly overlapping functionality. Furthermore, some of these\nservices have little to no training data available. Existing public datasets\nfor task-oriented dialogue do not sufficiently capture these challenges since\nthey cover few domains and assume a single static ontology per domain. In this\nwork, we introduce the the Schema-Guided Dialogue (SGD) dataset, containing\nover 16k multi-domain conversations spanning 16 domains. Our dataset exceeds\nthe existing task-oriented dialogue corpora in scale, while also highlighting\nthe challenges associated with building large-scale virtual assistants. It\nprovides a challenging testbed for a number of tasks including language\nunderstanding, slot filling, dialogue state tracking and response generation.\nAlong the same lines, we present a schema-guided paradigm for task-oriented\ndialogue, in which predictions are made over a dynamic set of intents and\nslots, provided as input, using their natural language descriptions. This\nallows a single dialogue system to easily support a large number of services\nand facilitates simple integration of new services without requiring additional\ntraining data. Building upon the proposed paradigm, we release a model for\ndialogue state tracking capable of zero-shot generalization to new APIs, while\nremaining competitive in the regular setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05858v2",
            "date": "2019-09-11 17:57:18+00:00",
            "title": "CTRL: A Conditional Transformer Language Model for Controllable Generation",
            "authors": [
                "Nitish Shirish Keskar",
                "Bryan McCann",
                "Lav R. Varshney",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Large-scale language models show promising text generation capabilities, but\nusers cannot easily control particular aspects of the generated text. We\nrelease CTRL, a 1.63 billion-parameter conditional transformer language model,\ntrained to condition on control codes that govern style, content, and\ntask-specific behavior. Control codes were derived from structure that\nnaturally co-occurs with raw text, preserving the advantages of unsupervised\nlearning while providing more explicit control over text generation. These\ncodes also allow CTRL to predict which parts of the training data are most\nlikely given a sequence. This provides a potential method for analyzing large\namounts of data via model-based source attribution. We have released multiple\nfull-sized, pretrained versions of CTRL at https://github.com/salesforce/ctrl.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06002v2",
            "date": "2019-09-13 02:20:25+00:00",
            "title": "Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting",
            "authors": [
                "Yi Zhang",
                "Tao Ge",
                "Furu Wei",
                "Ming Zhou",
                "Xu Sun"
            ],
            "abstract": "We study sequence-to-sequence (seq2seq) pre-training with data augmentation\nfor sentence rewriting. Instead of training a seq2seq model with gold training\ndata and augmented data simultaneously, we separate them to train in different\nphases: pre-training with the augmented data and fine-tuning with the gold\ndata. We also introduce multiple data augmentation methods to help model\npre-training for sentence rewriting. We evaluate our approach in two typical\nwell-defined sentence rewriting tasks: Grammatical Error Correction (GEC) and\nFormality Style Transfer (FST). Experiments demonstrate our approach can better\nutilize augmented data without hurting the model's trust in gold data and\nfurther improve the model's performance with our proposed data augmentation\nmethods.\n  Our approach substantially advances the state-of-the-art results in\nwell-recognized sentence rewriting benchmarks over both GEC and FST.\nSpecifically, it pushes the CoNLL-2014 benchmark's $F_{0.5}$ score and JFLEG\nTest GLEU score to 62.61 and 63.54 in the restricted training setting, 66.77\nand 65.22 respectively in the unrestricted setting, and advances GYAFC\nbenchmark's BLEU to 74.24 (2.23 absolute improvement) in E&M domain and 77.97\n(2.64 absolute improvement) in F&R domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06007v1",
            "date": "2019-09-13 02:43:36+00:00",
            "title": "Leveraging 2-hop Distant Supervision from Table Entity Pairs for Relation Extraction",
            "authors": [
                "Xiang Deng",
                "Huan Sun"
            ],
            "abstract": "Distant supervision (DS) has been widely used to automatically construct\n(noisy) labeled data for relation extraction (RE). Given two entities, distant\nsupervision exploits sentences that directly mention them for predicting their\nsemantic relation. We refer to this strategy as 1-hop DS, which unfortunately\nmay not work well for long-tail entities with few supporting sentences. In this\npaper, we introduce a new strategy named 2-hop DS to enhance distantly\nsupervised RE, based on the observation that there exist a large number of\nrelational tables on the Web which contain entity pairs that share common\nrelations. We refer to such entity pairs as anchors for each other, and collect\nall sentences that mention the anchor entity pairs of a given target entity\npair to help relation prediction. We develop a new neural RE method REDS2 in\nthe multi-instance learning paradigm, which adopts a hierarchical model\nstructure to fuse information respectively from 1-hop DS and 2-hop DS.\nExtensive experimental results on a benchmark dataset show that REDS2 can\nconsistently outperform various baselines across different settings by a\nsubstantial margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06249v4",
            "date": "2019-09-13 14:20:30+00:00",
            "title": "Taxonomical hierarchy of canonicalized relations from multiple Knowledge Bases",
            "authors": [
                "Akshay Parekh",
                "Ashish Anand",
                "Amit Awekar"
            ],
            "abstract": "This work addresses two important questions pertinent to Relation Extraction\n(RE). First, what are all possible relations that could exist between any two\ngiven entity types? Second, how do we define an unambiguous taxonomical (is-a)\nhierarchy among the identified relations? To address the first question, we use\nthree resources Wikipedia Infobox, Wikidata, and DBpedia. This study focuses on\nrelations between person, organization and location entity types. We exploit\nWikidata and DBpedia in a data-driven manner, and Wikipedia Infobox templates\nmanually to generate lists of relations. Further, to address the second\nquestion, we canonicalize, filter, and combine the identified relations from\nthe three resources to construct a taxonomical hierarchy. This hierarchy\ncontains 623 canonical relations with highest contribution from Wikipedia\nInfobox followed by DBpedia and Wikidata. The generated relation list subsumes\nan average of 85% of relations from RE datasets when entity types are\nrestricted.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06276v1",
            "date": "2019-09-13 14:59:51+00:00",
            "title": "Parameterized Convolutional Neural Networks for Aspect Level Sentiment Classification",
            "authors": [
                "Binxuan Huang",
                "Kathleen M. Carley"
            ],
            "abstract": "We introduce a novel parameterized convolutional neural network for aspect\nlevel sentiment classification. Using parameterized filters and parameterized\ngates, we incorporate aspect information into convolutional neural networks\n(CNN). Experiments demonstrate that our parameterized filters and parameterized\ngates effectively capture the aspect-specific features, and our CNN-based\nmodels achieve excellent results on SemEval 2014 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06321v3",
            "date": "2019-09-13 16:41:13+00:00",
            "title": "End-to-End Bias Mitigation by Modelling Biases in Corpora",
            "authors": [
                "Rabeeh Karimi Mahabadi",
                "Yonatan Belinkov",
                "James Henderson"
            ],
            "abstract": "Several recent studies have shown that strong natural language understanding\n(NLU) models are prone to relying on unwanted dataset biases without learning\nthe underlying task, resulting in models that fail to generalize to\nout-of-domain datasets and are likely to perform poorly in real-world\nscenarios. We propose two learning strategies to train neural models, which are\nmore robust to such biases and transfer better to out-of-domain datasets. The\nbiases are specified in terms of one or more bias-only models, which learn to\nleverage the dataset biases. During training, the bias-only models' predictions\nare used to adjust the loss of the base model to reduce its reliance on biases\nby down-weighting the biased examples and focusing the training on the hard\nexamples. We experiment on large-scale natural language inference and fact\nverification benchmarks, evaluating on out-of-domain datasets that are\nspecifically designed to assess the robustness of models against known biases\nin the training data. Results show that our debiasing methods greatly improve\nrobustness in all settings and better transfer to other textual entailment\ndatasets. Our code and data are publicly available in\n\\url{https://github.com/rabeehk/robust-nli}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06502v2",
            "date": "2019-09-14 01:47:09+00:00",
            "title": "Multi-class Multilingual Classification of Wikipedia Articles Using Extended Named Entity Tag Set",
            "authors": [
                "Hassan S. Shavarani",
                "Satoshi Sekine"
            ],
            "abstract": "Wikipedia is a great source of general world knowledge which can guide NLP\nmodels better understand their motivation to make predictions. Structuring\nWikipedia is the initial step towards this goal which can facilitate fine-grain\nclassification of articles. In this work, we introduce the Shinra 5-Language\nCategorization Dataset (SHINRA-5LDS), a large multi-lingual and multi-labeled\nset of annotated Wikipedia articles in Japanese, English, French, German, and\nFarsi using Extended Named Entity (ENE) tag set. We evaluate the dataset using\nthe best models provided for ENE label set classification and show that the\ncurrently available classification models struggle with large datasets using\nfine-grained tag sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06516v2",
            "date": "2019-09-14 03:11:52+00:00",
            "title": "A Universal Parent Model for Low-Resource Neural Machine Translation Transfer",
            "authors": [
                "Mozhdeh Gheini",
                "Jonathan May"
            ],
            "abstract": "Transfer learning from a high-resource language pair `parent' has been proven\nto be an effective way to improve neural machine translation quality for\nlow-resource language pairs `children.' However, previous approaches build a\ncustom parent model or at least update an existing parent model's vocabulary\nfor each child language pair they wish to train, in an effort to align parent\nand child vocabularies. This is not a practical solution. It is wasteful to\ndevote the majority of training time for new language pairs to optimizing\nparameters on an unrelated data set. Further, this overhead reduces the utility\nof neural machine translation for deployment in humanitarian assistance\nscenarios, where extra time to deploy a new language pair can mean the\ndifference between life and death. In this work, we present a `universal'\npre-trained neural parent model with constant vocabulary that can be used as a\nstarting point for training practically any new low-resource language to a\nfixed target language. We demonstrate that our approach, which leverages\northography unification and a broad-coverage approach to subword\nidentification, generalizes well to several languages from a variety of\nfamilies, and that translation systems built with our approach can be built\nmore quickly than competing methods and with better quality as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06564v1",
            "date": "2019-09-14 09:18:44+00:00",
            "title": "ALTER: Auxiliary Text Rewriting Tool for Natural Language Generation",
            "authors": [
                "Qiongkai Xu",
                "Chenchen Xu",
                "Lizhen Qu"
            ],
            "abstract": "In this paper, we describe ALTER, an auxiliary text rewriting tool that\nfacilitates the rewriting process for natural language generation tasks, such\nas paraphrasing, text simplification, fairness-aware text rewriting, and text\nstyle transfer. Our tool is characterized by two features, i) recording of\nword-level revision histories and ii) flexible auxiliary edit support and\nfeedback to annotators. The text rewriting assist and traceable rewriting\nhistory are potentially beneficial to the future research of natural language\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06644v1",
            "date": "2019-09-14 18:03:57+00:00",
            "title": "Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia",
            "authors": [
                "Angus Addlesee",
                "Arash Eshghi",
                "Ioannis Konstas"
            ],
            "abstract": "Dialogue technologies such as Amazon's Alexa have the potential to transform\nthe healthcare industry. However, current systems are not yet naturally\ninteractive: they are often turn-based, have naive end-of-turn detection and\ncompletely ignore many types of verbal and visual feedback - such as\nbackchannels, hesitation markers, filled pauses, gaze, brow furrows and\ndisfluencies - that are crucial in guiding and managing the conversational\nprocess. This is especially important in the healthcare industry as target\nusers of Spoken Dialogue Systems (SDSs) are likely to be frail, older,\ndistracted or suffer from cognitive decline which impacts their ability to make\neffective use of current systems. In this paper, we outline some of the\nchallenges that are in urgent need of further research, including Incremental\nSpeech Recognition and a systematic study of the interactional patterns in\nconversation that are potentially diagnostic of dementia, and how these might\ninform research on and the design of the next generation of SDSs.",
            "journal ref": "Dialog for Good (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06694v1",
            "date": "2019-09-14 23:15:20+00:00",
            "title": "Beyond BLEU: Training Neural Machine Translation with Semantic Similarity",
            "authors": [
                "John Wieting",
                "Taylor Berg-Kirkpatrick",
                "Kevin Gimpel",
                "Graham Neubig"
            ],
            "abstract": "While most neural machine translation (NMT) systems are still trained using\nmaximum likelihood estimation, recent work has demonstrated that optimizing\nsystems to directly improve evaluation metrics such as BLEU can substantially\nimprove final translation accuracy. However, training with BLEU has some\nlimitations: it doesn't assign partial credit, it has a limited range of output\nvalues, and it can penalize semantically correct hypotheses if they differ\nlexically from the reference. In this paper, we introduce an alternative reward\nfunction for optimizing NMT systems that is based on recent work in semantic\nsimilarity. We evaluate on four disparate languages translated to English, and\nfind that training with our proposed metric results in better translations as\nevaluated by BLEU, semantic similarity, and human evaluation, and also that the\noptimization procedure converges faster. Analysis suggests that this is because\nthe proposed metric is more conducive to optimization, assigning partial credit\nand providing more diversity in scores than BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06762v2",
            "date": "2019-09-15 08:50:59+00:00",
            "title": "Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever",
            "authors": [
                "Libo Qin",
                "Yijia Liu",
                "Wanxiang Che",
                "Haoyang Wen",
                "Yangming Li",
                "Ting Liu"
            ],
            "abstract": "Querying the knowledge base (KB) has long been a challenge in the end-to-end\ntask-oriented dialogue system. Previous sequence-to-sequence (Seq2Seq) dialogue\ngeneration work treats the KB query as an attention over the entire KB, without\nthe guarantee that the generated entities are consistent with each other. In\nthis paper, we propose a novel framework which queries the KB in two steps to\nimprove the consistency of generated entities. In the first step, inspired by\nthe observation that a response can usually be supported by a single KB row, we\nintroduce a KB retrieval component which explicitly returns the most relevant\nKB row given a dialogue history. The retrieval result is further used to filter\nthe irrelevant entities in a Seq2Seq response generation model to improve the\nconsistency among the output entities. In the second step, we further perform\nthe attention mechanism to address the most correlated KB column. Two methods\nare proposed to make the training feasible without labeled retrieval data,\nwhich include distant supervision and Gumbel-Softmax technique. Experiments on\ntwo publicly available task oriented dialog datasets show the effectiveness of\nour model by outperforming the baseline systems and producing entity-consistent\nresponses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06775v1",
            "date": "2019-09-15 10:33:17+00:00",
            "title": "Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing",
            "authors": [
                "Yuxuan Wang",
                "Wanxiang Che",
                "Jiang Guo",
                "Yijia Liu",
                "Ting Liu"
            ],
            "abstract": "This paper investigates the problem of learning cross-lingual representations\nin a contextual space. We propose Cross-Lingual BERT Transformation (CLBT), a\nsimple and efficient approach to generate cross-lingual contextualized word\nembeddings based on publicly available pre-trained BERT models (Devlin et al.,\n2018). In this approach, a linear transformation is learned from contextual\nword alignments to align the contextualized embeddings independently trained in\ndifferent languages. We demonstrate the effectiveness of this approach on\nzero-shot cross-lingual transfer parsing. Experiments show that our embeddings\nsubstantially outperform the previous state-of-the-art that uses static\nembeddings. We further compare our approach with XLM (Lample and Conneau,\n2019), a recently proposed cross-lingual language model trained with massive\nparallel data, and achieve highly competitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06877v1",
            "date": "2019-09-15 20:25:22+00:00",
            "title": "Query-Focused Scenario Construction",
            "authors": [
                "Su Wang",
                "Greg Durrett",
                "Katrin Erk"
            ],
            "abstract": "The news coverage of events often contains not one but multiple incompatible\naccounts of what happened. We develop a query-based system that extracts\ncompatible sets of events (scenarios) from such data, formulated as one-class\nclustering. Our system incrementally evaluates each event's compatibility with\nalready selected events, taking order into account. We use synthetic data\nconsisting of article mixtures for scalable training and evaluate our model on\na new human-curated dataset of scenarios about real-world news topics. Stronger\nneural network models and harder synthetic training settings are both important\nto achieve high performance, and our final scenario construction system\nsubstantially outperforms baselines based on prior work.",
            "journal ref": "EMNLP-IJCNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06937v1",
            "date": "2019-09-16 02:10:58+00:00",
            "title": "CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding",
            "authors": [
                "Yijin Liu",
                "Fandong Meng",
                "Jinchao Zhang",
                "Jie Zhou",
                "Yufeng Chen",
                "Jinan Xu"
            ],
            "abstract": "Spoken Language Understanding (SLU) mainly involves two tasks, intent\ndetection and slot filling, which are generally modeled jointly in existing\nworks. However, most existing models fail to fully utilize co-occurrence\nrelations between slots and intents, which restricts their potential\nperformance. To address this issue, in this paper we propose a novel\nCollaborative Memory Network (CM-Net) based on the well-designed block, named\nCM-block. The CM-block firstly captures slot-specific and intent-specific\nfeatures from memories in a collaborative manner, and then uses these enriched\nfeatures to enhance local context representations, based on which the\nsequential information flow leads to more specific (slot and intent) global\nutterance representations. Through stacking multiple CM-blocks, our CM-Net is\nable to alternately perform information exchange among specific memories, local\ncontexts and the global utterance, and thus incrementally enriches each other.\nWe evaluate the CM-Net on two standard benchmarks (ATIS and SNIPS) and a\nself-collected corpus (CAIS). Experimental results show that the CM-Net\nachieves the state-of-the-art results on the ATIS and SNIPS in most of\ncriteria, and significantly outperforms the baseline models on the CAIS.\nAdditionally, we make the CAIS dataset publicly available for the research\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07005v2",
            "date": "2019-09-16 06:15:27+00:00",
            "title": "KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension",
            "authors": [
                "Seungyoung Lim",
                "Myungji Kim",
                "Jooyoul Lee"
            ],
            "abstract": "Machine Reading Comprehension (MRC) is a task that requires machine to\nunderstand natural language and answer questions by reading a document. It is\nthe core of automatic response technology such as chatbots and automatized\ncustomer supporting systems. We present Korean Question Answering\nDataset(KorQuAD), a large-scale Korean dataset for extractive machine reading\ncomprehension task. It consists of 70,000+ human generated question-answer\npairs on Korean Wikipedia articles. We release KorQuAD1.0 and launch a\nchallenge at https://KorQuAD.github.io to encourage the development of\nmultilingual natural language processing research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07009v2",
            "date": "2019-09-16 06:20:23+00:00",
            "title": "Bridging the domain gap in cross-lingual document classification",
            "authors": [
                "Guokun Lai",
                "Barlas Oguz",
                "Yiming Yang",
                "Veselin Stoyanov"
            ],
            "abstract": "The scarcity of labeled training data often prohibits the\ninternationalization of NLP models to multiple languages. Recent developments\nin cross-lingual understanding (XLU) has made progress in this area, trying to\nbridge the language barrier using language universal representations. However,\neven if the language problem was resolved, models trained in one language would\nnot transfer to another language perfectly due to the natural domain drift\nacross languages and cultures. We consider the setting of semi-supervised\ncross-lingual understanding, where labeled data is available in a source\nlanguage (English), but only unlabeled data is available in the target\nlanguage. We combine state-of-the-art cross-lingual methods with recently\nproposed methods for weakly supervised learning such as unsupervised\npre-training and unsupervised data augmentation to simultaneously close both\nthe language gap and the domain gap in XLU. We show that addressing the domain\ngap is crucial. We improve over strong baselines and achieve a new\nstate-of-the-art for cross-lingual document classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07101v1",
            "date": "2019-09-16 10:19:02+00:00",
            "title": "Domain Transfer in Dialogue Systems without Turn-Level Supervision",
            "authors": [
                "Joachim Bingel",
                "Victor Petr\u00e9n Bach Hansen",
                "Ana Valeria Gonzalez",
                "Pawe\u0142 Budzianowski",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Task oriented dialogue systems rely heavily on specialized dialogue state\ntracking (DST) modules for dynamically predicting user intent throughout the\nconversation. State-of-the-art DST models are typically trained in a supervised\nmanner from manual annotations at the turn level. However, these annotations\nare costly to obtain, which makes it difficult to create accurate dialogue\nsystems for new domains. To address these limitations, we propose a method,\nbased on reinforcement learning, for transferring DST models to new domains\nwithout turn-level supervision. Across several domains, our experiments show\nthat this method quickly adapts off-the-shelf models to new domains and\nperforms on par with models trained with turn-level supervision. We also show\nour method can improve models trained using turn-level supervision by\nsubsequent fine-tuning optimization toward dialog-level rewards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07285v1",
            "date": "2019-09-16 15:38:36+00:00",
            "title": "Fast transcription of speech in low-resource languages",
            "authors": [
                "Mark Hasegawa-Johnson",
                "Camille Goudeseune",
                "Gina-Anne Levow"
            ],
            "abstract": "We present software that, in only a few hours, transcribes forty hours of\nrecorded speech in a surprise language, using only a few tens of megabytes of\nnoisy text in that language, and a zero-resource grapheme to phoneme (G2P)\ntable. A pretrained acoustic model maps acoustic features to phonemes; a\nreversed G2P maps these to graphemes; then a language model maps these to a\nmost-likely grapheme sequence, i.e., a transcription. This software has worked\nsuccessfully with corpora in Arabic, Assam, Kinyarwanda, Russian, Sinhalese,\nSwahili, Tagalog, and Tamil.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07290v2",
            "date": "2019-09-16 15:42:36+00:00",
            "title": "Communication-based Evaluation for Natural Language Generation",
            "authors": [
                "Benjamin Newman",
                "Reuben Cohn-Gordon",
                "Christopher Potts"
            ],
            "abstract": "Natural language generation (NLG) systems are commonly evaluated using n-gram\noverlap measures (e.g. BLEU, ROUGE). These measures do not directly capture\nsemantics or speaker intentions, and so they often turn out to be misaligned\nwith our true goals for NLG. In this work, we argue instead for\ncommunication-based evaluations: assuming the purpose of an NLG system is to\nconvey information to a reader/listener, we can directly evaluate its\neffectiveness at this task using the Rational Speech Acts model of pragmatic\nlanguage use. We illustrate with a color reference dataset that contains\ndescriptions in pre-defined quality categories, showing that our method better\naligns with these quality categories than do any of the prominent n-gram\noverlap methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07342v1",
            "date": "2019-09-16 17:22:25+00:00",
            "title": "Multilingual Neural Machine Translation for Zero-Resource Languages",
            "authors": [
                "Surafel M. Lakew",
                "Marcello Federico",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "In recent years, Neural Machine Translation (NMT) has been shown to be more\neffective than phrase-based statistical methods, thus quickly becoming the\nstate of the art in machine translation (MT). However, NMT systems are limited\nin translating low-resourced languages, due to the significant amount of\nparallel data that is required to learn useful mappings between languages. In\nthis work, we show how the so-called multilingual NMT can help to tackle the\nchallenges associated with low-resourced language translation. The underlying\nprinciple of multilingual NMT is to force the creation of hidden\nrepresentations of words in a shared semantic space across multiple languages,\nthus enabling a positive parameter transfer across languages. Along this\ndirection, we present multilingual translation experiments with three languages\n(English, Italian, Romanian) covering six translation directions, utilizing\nboth recurrent neural networks and transformer (or self-attentive) neural\nnetworks. We then focus on the zero-shot translation problem, that is how to\nleverage multi-lingual data in order to learn translation directions that are\nnot covered by the available training material. To this aim, we introduce our\nrecently proposed iterative self-training method, which incrementally improves\na multilingual NMT on a zero-shot direction by just relying on monolingual\ndata. Our results on TED talks data show that multilingual NMT outperforms\nconventional bilingual NMT, that the transformer NMT outperforms recurrent NMT,\nand that zero-shot NMT outperforms conventional pivoting methods and even\nmatches the performance of a fully-trained bilingual system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07405v2",
            "date": "2019-09-16 18:00:24+00:00",
            "title": "BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle",
            "authors": [
                "Peter West",
                "Ari Holtzman",
                "Jan Buys",
                "Yejin Choi"
            ],
            "abstract": "The principle of the Information Bottleneck (Tishby et al. 1999) is to\nproduce a summary of information X optimized to predict some other relevant\ninformation Y. In this paper, we propose a novel approach to unsupervised\nsentence summarization by mapping the Information Bottleneck principle to a\nconditional language modelling objective: given a sentence, our approach seeks\na compressed sentence that can best predict the next sentence. Our iterative\nalgorithm under the Information Bottleneck objective searches gradually shorter\nsubsequences of the given sentence while maximizing the probability of the next\nsentence conditioned on the summary. Using only pretrained language models with\nno direct supervision, our approach can efficiently perform extractive sentence\nsummarization over a large corpus.\n  Building on our unsupervised extractive summarization (BottleSumEx), we then\npresent a new approach to self-supervised abstractive summarization\n(BottleSumSelf), where a transformer-based language model is trained on the\noutput summaries of our unsupervised method. Empirical results demonstrate that\nour extractive method outperforms other unsupervised models on multiple\nautomatic metrics. In addition, we find that our self-supervised abstractive\nmodel outperforms unsupervised baselines (including our own) by human\nevaluation along multiple attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07512v1",
            "date": "2019-09-16 22:51:29+00:00",
            "title": "Short-Text Classification Using Unsupervised Keyword Expansion",
            "authors": [
                "Duncan Cameron-Steinke"
            ],
            "abstract": "Short-text classification, like all data science, struggles to achieve high\nperformance using limited data. As a solution, a short sentence may be expanded\nwith new and relevant feature words to form an artificially enlarged dataset,\nand add new features to testing data. This paper applies a novel approach to\ntext expansion by generating new words directly for each input sentence, thus\nrequiring no additional datasets or previous training. In this unsupervised\napproach, new keywords are formed within the hidden states of a pre-trained\nlanguage model and then used to create extended pseudo documents. The word\ngeneration process was assessed by examining how well the predicted words\nmatched to topics of the input sentence. It was found that this method could\nproduce 3-10 relevant new words for each target topic, while generating just 1\nword related to each non-target topic. Generated words were then added to short\nnews headlines to create extended pseudo headlines. Experimental results have\nshown that models trained using the pseudo headlines can improve classification\naccuracy when limiting the number of training examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07521v2",
            "date": "2019-09-16 23:44:49+00:00",
            "title": "Probing Natural Language Inference Models through Semantic Fragments",
            "authors": [
                "Kyle Richardson",
                "Hai Hu",
                "Lawrence S. Moss",
                "Ashish Sabharwal"
            ],
            "abstract": "Do state-of-the-art models for language understanding already have, or can\nthey easily learn, abilities such as boolean coordination, quantification,\nconditionals, comparatives, and monotonicity reasoning (i.e., reasoning about\nword substitutions in sentential contexts)? While such phenomena are involved\nin natural language inference (NLI) and go beyond basic linguistic\nunderstanding, it is unclear the extent to which they are captured in existing\nNLI benchmarks and effectively learned by models. To investigate this, we\npropose the use of semantic fragments---systematically generated datasets that\neach target a different semantic phenomenon---for probing, and efficiently\nimproving, such capabilities of linguistic models. This approach to creating\nchallenge datasets allows direct control over the semantic diversity and\ncomplexity of the targeted linguistic phenomena, and results in a more precise\ncharacterization of a model's linguistic behavior. Our experiments, using a\nlibrary of 8 such semantic fragments, reveal two remarkable findings: (a)\nState-of-the-art models, including BERT, that are pre-trained on existing NLI\nbenchmark datasets perform poorly on these new fragments, even though the\nphenomena probed here are central to the NLI task. (b) On the other hand, with\nonly a few minutes of additional fine-tuning---with a carefully selected\nlearning rate and a novel variation of \"inoculation\"---a BERT-based model can\nmaster all of these logic and monotonicity fragments while retaining its\nperformance on established NLI benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07586v1",
            "date": "2019-09-17 04:49:25+00:00",
            "title": "Grounding learning of modifier dynamics: An application to color naming",
            "authors": [
                "Xudong Han",
                "Philip Schulz",
                "Trevor Cohn"
            ],
            "abstract": "Grounding is crucial for natural language understanding. An important subtask\nis to understand modified color expressions, such as 'dirty blue'. We present a\nmodel of color modifiers that, compared with previous additive models in RGB\nspace, learns more complex transformations. In addition, we present a model\nthat operates in the HSV color space. We show that certain adjectives are\nbetter modeled in that space. To account for all modifiers, we train a hard\nensemble model that selects a color space depending on the modifier color pair.\nExperimental results show significant and consistent improvements compared to\nthe state-of-the-art baseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07593v1",
            "date": "2019-09-17 05:03:43+00:00",
            "title": "Learning Explicit and Implicit Structures for Targeted Sentiment Analysis",
            "authors": [
                "Hao Li",
                "Wei Lu"
            ],
            "abstract": "Targeted sentiment analysis is the task of jointly predicting target entities\nand their associated sentiment information. Existing research efforts mostly\nregard this joint task as a sequence labeling problem, building models that can\ncapture explicit structures in the output space. However, the importance of\ncapturing implicit global structural information that resides in the input\nspace is largely unexplored. In this work, we argue that both types of\ninformation (implicit and explicit structural information) are crucial for\nbuilding a successful targeted sentiment analysis model. Our experimental\nresults show that properly capturing both information is able to lead to better\nperformance than competitive existing approaches. We also conduct extensive\nexperiments to investigate our model's effectiveness and robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07597v2",
            "date": "2019-09-17 05:15:05+00:00",
            "title": "Simple yet Effective Bridge Reasoning for Open-Domain Multi-Hop Question Answering",
            "authors": [
                "Wenhan Xiong",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Hong Wang",
                "Shiyu Chang",
                "Murray Campbell",
                "William Yang Wang"
            ],
            "abstract": "A key challenge of multi-hop question answering (QA) in the open-domain\nsetting is to accurately retrieve the supporting passages from a large corpus.\nExisting work on open-domain QA typically relies on off-the-shelf information\nretrieval (IR) techniques to retrieve \\textbf{answer passages}, i.e., the\npassages containing the groundtruth answers. However, IR-based approaches are\ninsufficient for multi-hop questions, as the topic of the second or further\nhops is not explicitly covered by the question. To resolve this issue, we\nintroduce a new sub-problem of open-domain multi-hop QA, which aims to\nrecognize the bridge (\\emph{i.e.}, the anchor that links to the answer passage)\nfrom the context of a set of start passages with a reading comprehension model.\nThis model, the \\textbf{bridge reasoner}, is trained with a weakly supervised\nsignal and produces the candidate answer passages for the \\textbf{passage\nreader} to extract the answer. On the full-wiki HotpotQA benchmark, we\nsignificantly improve the baseline method by 14 point F1. Without using any\nmemory-inefficient contextual embeddings, our result is also competitive with\nthe state-of-the-art that applies BERT in multiple modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07598v1",
            "date": "2019-09-17 05:23:50+00:00",
            "title": "Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering",
            "authors": [
                "Ameya Godbole",
                "Dilip Kavarthapu",
                "Rajarshi Das",
                "Zhiyu Gong",
                "Abhishek Singhal",
                "Hamed Zamani",
                "Mo Yu",
                "Tian Gao",
                "Xiaoxiao Guo",
                "Manzil Zaheer",
                "Andrew McCallum"
            ],
            "abstract": "Multi-hop question answering (QA) requires an information retrieval (IR)\nsystem that can find \\emph{multiple} supporting evidence needed to answer the\nquestion, making the retrieval process very challenging. This paper introduces\nan IR technique that uses information of entities present in the initially\nretrieved evidence to learn to `\\emph{hop}' to other relevant evidence. In a\nsetting, with more than \\textbf{5 million} Wikipedia paragraphs, our approach\nleads to significant boost in retrieval performance. The retrieved evidence\nalso increased the performance of an existing QA model (without any training)\non the \\hotpot benchmark by \\textbf{10.59} F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07734v2",
            "date": "2019-09-17 11:55:32+00:00",
            "title": "SocialNLP EmotionX 2019 Challenge Overview: Predicting Emotions in Spoken Dialogues and Chats",
            "authors": [
                "Boaz Shmueli",
                "Lun-Wei Ku"
            ],
            "abstract": "We present an overview of the EmotionX 2019 Challenge, held at the 7th\nInternational Workshop on Natural Language Processing for Social Media\n(SocialNLP), in conjunction with IJCAI 2019. The challenge entailed predicting\nemotions in spoken and chat-based dialogues using augmented EmotionLines\ndatasets. EmotionLines contains two distinct datasets: the first includes\nexcerpts from a US-based TV sitcom episode scripts (Friends) and the second\ncontains online chats (EmotionPush). A total of thirty-six teams registered to\nparticipate in the challenge. Eleven of the teams successfully submitted their\npredictions performance evaluation. The top-scoring team achieved a micro-F1\nscore of 81.5% for the spoken-based dialogues (Friends) and 79.5% for the\nchat-based dialogues (EmotionPush).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07907v1",
            "date": "2019-09-17 15:52:18+00:00",
            "title": "Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translation",
            "authors": [
                "Jetic G\u016b",
                "Hassan S. Shavarani",
                "Anoop Sarkar"
            ],
            "abstract": "Neural machine translation (NMT) systems require large amounts of high\nquality in-domain parallel corpora for training. State-of-the-art NMT systems\nstill face challenges related to out-of-vocabulary words and dealing with\nlow-resource language pairs. In this paper, we propose and compare several\nmodels for fusion of bilingual lexicons with an end-to-end trained\nsequence-to-sequence model for machine translation. The result is a fusion\nmodel with two information sources for the decoder: a neural conditional\nlanguage model and a bilingual lexicon. This fusion model learns how to combine\nboth sources of information in order to produce higher quality translation\noutput. Our experiments show that our proposed models work well in relatively\nlow-resource scenarios, and also effectively reduce the parameter size and\ntraining cost for NMT without sacrificing performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07928v1",
            "date": "2019-09-17 16:46:58+00:00",
            "title": "Say Anything: Automatic Semantic Infelicity Detection in L2 English Indefinite Pronouns",
            "authors": [
                "Ella Rabinovich",
                "Julia Watson",
                "Barend Beekhuizen",
                "Suzanne Stevenson"
            ],
            "abstract": "Computational research on error detection in second language speakers has\nmainly addressed clear grammatical anomalies typical to learners at the\nbeginner-to-intermediate level. We focus instead on acquisition of subtle\nsemantic nuances of English indefinite pronouns by non-native speakers at\nvarying levels of proficiency. We first lay out theoretical, linguistically\nmotivated hypotheses, and supporting empirical evidence on the nature of the\nchallenges posed by indefinite pronouns to English learners. We then suggest\nand evaluate an automatic approach for detection of atypical usage patterns,\ndemonstrating that deep learning architectures are promising for this task\ninvolving nuanced semantic anomalies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08053v4",
            "date": "2019-09-17 19:42:54+00:00",
            "title": "Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism",
            "authors": [
                "Mohammad Shoeybi",
                "Mostofa Patwary",
                "Raul Puri",
                "Patrick LeGresley",
                "Jared Casper",
                "Bryan Catanzaro"
            ],
            "abstract": "Recent work in language modeling demonstrates that training large transformer\nmodels advances the state of the art in Natural Language Processing\napplications. However, very large models can be quite difficult to train due to\nmemory constraints. In this work, we present our techniques for training very\nlarge transformer models and implement a simple, efficient intra-layer model\nparallel approach that enables training transformer models with billions of\nparameters. Our approach does not require a new compiler or library changes, is\northogonal and complimentary to pipeline model parallelism, and can be fully\nimplemented with the insertion of a few communication operations in native\nPyTorch. We illustrate this approach by converging transformer based models up\nto 8.3 billion parameters using 512 GPUs. We sustain 15.1 PetaFLOPs across the\nentire application with 76% scaling efficiency when compared to a strong single\nGPU baseline that sustains 39 TeraFLOPs, which is 30% of peak FLOPs. To\ndemonstrate that large language models can further advance the state of the art\n(SOTA), we train an 8.3 billion parameter transformer language model similar to\nGPT-2 and a 3.9 billion parameter model similar to BERT. We show that careful\nattention to the placement of layer normalization in BERT-like models is\ncritical to achieving increased performance as the model size grows. Using the\nGPT-2 model we achieve SOTA results on the WikiText103 (10.8 compared to SOTA\nperplexity of 15.8) and LAMBADA (66.5% compared to SOTA accuracy of 63.2%)\ndatasets. Our BERT model achieves SOTA results on the RACE dataset (90.9%\ncompared to SOTA accuracy of 89.4%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08089v1",
            "date": "2019-09-17 20:52:19+00:00",
            "title": "Extractive Summarization of Long Documents by Combining Global and Local Context",
            "authors": [
                "Wen Xiao",
                "Giuseppe Carenini"
            ],
            "abstract": "In this paper, we propose a novel neural single document extractive\nsummarization model for long documents, incorporating both the global context\nof the whole document and the local context within the current topic. We\nevaluate the model on two datasets of scientific papers, Pubmed and arXiv,\nwhere it outperforms previous work, both extractive and abstractive models, on\nROUGE-1, ROUGE-2 and METEOR scores. We also show that, consistently with our\ngoal, the benefits of our method become stronger as we apply it to longer\ndocuments. Rather surprisingly, an ablation study indicates that the benefits\nof our model seem to come exclusively from modeling the local context, even for\nthe longest documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08090v2",
            "date": "2019-09-17 20:56:50+00:00",
            "title": "DOVER: A Method for Combining Diarization Outputs",
            "authors": [
                "Andreas Stolcke",
                "Takuya Yoshioka"
            ],
            "abstract": "Speech recognition and other natural language tasks have long benefited from\nvoting-based algorithms as a method to aggregate outputs from several systems\nto achieve a higher accuracy than any of the individual systems. Diarization,\nthe task of segmenting an audio stream into speaker-homogeneous and co-indexed\nregions, has so far not seen the benefit of this strategy because the structure\nof the task does not lend itself to a simple voting approach. This paper\npresents DOVER (diarization output voting error reduction), an algorithm for\nweighted voting among diarization hypotheses, in the spirit of the ROVER\nalgorithm for combining speech recognition hypotheses. We evaluate the\nalgorithm for diarization of meeting recordings with multiple microphones, and\nfind that it consistently reduces diarization error rate over the average of\nresults from individual channels, and often improves on the single best channel\nchosen by an oracle.",
            "journal ref": "Proc. IEEE Automatic Speech Recognition and Understanding Workshop\n  2019, pp. 757-763",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08135v3",
            "date": "2019-09-17 22:54:10+00:00",
            "title": "SUPP.AI: Finding Evidence for Supplement-Drug Interactions",
            "authors": [
                "Lucy Lu Wang",
                "Oyvind Tafjord",
                "Arman Cohan",
                "Sarthak Jain",
                "Sam Skjonsberg",
                "Carissa Schoenick",
                "Nick Botner",
                "Waleed Ammar"
            ],
            "abstract": "Dietary supplements are used by a large portion of the population, but\ninformation on their pharmacologic interactions is incomplete. To address this\nchallenge, we present SUPP.AI, an application for browsing evidence of\nsupplement-drug interactions (SDIs) extracted from the biomedical literature.\nWe train a model to automatically extract supplement information and identify\nsuch interactions from the scientific literature. To address the lack of\nlabeled data for SDI identification, we use labels of the closely related task\nof identifying drug-drug interactions (DDIs) for supervision. We fine-tune the\ncontextualized word representations of the RoBERTa language model using labeled\nDDI data, and apply the fine-tuned model to identify supplement interactions.\nWe extract 195k evidence sentences from 22M articles (P=0.82, R=0.58, F1=0.68)\nfor 60k interactions. We create the SUPP.AI application for users to search\nevidence sentences extracted by our model. SUPP.AI is an attempt to close the\ninformation gap on dietary supplements by making up-to-date evidence on SDIs\nmore discoverable for researchers, clinicians, and consumers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08166v1",
            "date": "2019-09-18 01:54:53+00:00",
            "title": "Recursive Graphical Neural Networks for Text Classification",
            "authors": [
                "Wei Li",
                "Shuheng Li",
                "Shuming Ma",
                "Yancheng He",
                "Deli Chen",
                "Xu Sun"
            ],
            "abstract": "The complicated syntax structure of natural language is hard to be explicitly\nmodeled by sequence-based models. Graph is a natural structure to describe the\ncomplicated relation between tokens. The recent advance in Graph Neural\nNetworks (GNN) provides a powerful tool to model graph structure data, but\nsimple graph models such as Graph Convolutional Networks (GCN) suffer from\nover-smoothing problem, that is, when stacking multiple layers, all nodes will\nconverge to the same value. In this paper, we propose a novel Recursive\nGraphical Neural Networks model (ReGNN) to represent text organized in the form\nof graph. In our proposed model, LSTM is used to dynamically decide which part\nof the aggregated neighbor information should be transmitted to upper layers\nthus alleviating the over-smoothing problem. Furthermore, to encourage the\nexchange between the local and global information, a global graph-level node is\ndesigned. We conduct experiments on both single and multiple label text\nclassification tasks. Experiment results show that our ReGNN model surpasses\nthe strong baselines significantly in most of the datasets and greatly\nalleviates the over-smoothing problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08217v1",
            "date": "2019-09-18 05:54:32+00:00",
            "title": "Improving Natural Language Inference with a Pretrained Parser",
            "authors": [
                "Deric Pang",
                "Lucy H. Lin",
                "Noah A. Smith"
            ],
            "abstract": "We introduce a novel approach to incorporate syntax into natural language\ninference (NLI) models. Our method uses contextual token-level vector\nrepresentations from a pretrained dependency parser. Like other contextual\nembedders, our method is broadly applicable to any neural model. We experiment\nwith four strong NLI models (decomposable attention model, ESIM, BERT, and\nMT-DNN), and show consistent benefit to accuracy across three NLI benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08229v1",
            "date": "2019-09-18 06:53:30+00:00",
            "title": "Pre-trained Language Model for Biomedical Question Answering",
            "authors": [
                "Wonjin Yoon",
                "Jinhyuk Lee",
                "Donghyeon Kim",
                "Minbyul Jeong",
                "Jaewoo Kang"
            ],
            "abstract": "The recent success of question answering systems is largely attributed to\npre-trained language models. However, as language models are mostly pre-trained\non general domain corpora such as Wikipedia, they often have difficulty in\nunderstanding biomedical questions. In this paper, we investigate the\nperformance of BioBERT, a pre-trained biomedical language model, in answering\nbiomedical questions including factoid, list, and yes/no type questions.\nBioBERT uses almost the same structure across various question types and\nachieved the best performance in the 7th BioASQ Challenge (Task 7b, Phase B).\nBioBERT pre-trained on SQuAD or SQuAD 2.0 easily outperformed previous\nstate-of-the-art models. BioBERT obtains the best performance when it uses the\nappropriate pre-/post-processing strategies for questions, passages, and\nanswers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08306v1",
            "date": "2019-09-18 09:21:28+00:00",
            "title": "Text Length Adaptation in Sentiment Classification",
            "authors": [
                "Reinald Kim Amplayo",
                "Seonjae Lim",
                "Seung-won Hwang"
            ],
            "abstract": "Can a text classifier generalize well for datasets where the text length is\ndifferent? For example, when short reviews are sentiment-labeled, can these\ntransfer to predict the sentiment of long reviews (i.e., short to long\ntransfer), or vice versa? While unsupervised transfer learning has been\nwell-studied for cross domain/lingual transfer tasks, Cross Length Transfer\n(CLT) has not yet been explored. One reason is the assumption that length\ndifference is trivially transferable in classification. We show that it is not,\nbecause short/long texts differ in context richness and word intensity. We\ndevise new benchmark datasets from diverse domains and languages, and show that\nexisting models from similar tasks cannot deal with the unique challenge of\ntransferring across text lengths. We introduce a strong baseline model called\nBaggedCNN that treats long texts as bags containing short texts. We propose a\nstate-of-the-art CLT model called Length Transfer Networks (LeTraNets) that\nintroduces a two-way encoding scheme for short and long texts using multiple\ntraining mechanisms. We test our models and find that existing models perform\nworse than the BaggedCNN baseline, while LeTraNets outperforms all models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08357v1",
            "date": "2019-09-18 11:14:53+00:00",
            "title": "Subword ELMo",
            "authors": [
                "Jiangtong Li",
                "Hai Zhao",
                "Zuchao Li",
                "Wei Bi",
                "Xiaojiang Liu"
            ],
            "abstract": "Embedding from Language Models (ELMo) has shown to be effective for improving\nmany natural language processing (NLP) tasks, and ELMo takes character\ninformation to compose word representation to train language models.However,\nthe character is an insufficient and unnatural linguistic unit for word\nrepresentation.Thus we introduce Embedding from Subword-aware Language Models\n(ESuLMo) which learns word representation from subwords using unsupervised\nsegmentation over words.We show that ESuLMo can enhance four benchmark NLP\ntasks more effectively than ELMo, including syntactic dependency parsing,\nsemantic role labeling, implicit discourse relation recognition and textual\nentailment, which brings a meaningful improvement over ELMo.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08358v1",
            "date": "2019-09-18 11:15:22+00:00",
            "title": "Using BERT for Word Sense Disambiguation",
            "authors": [
                "Jiaju Du",
                "Fanchao Qi",
                "Maosong Sun"
            ],
            "abstract": "Word Sense Disambiguation (WSD), which aims to identify the correct sense of\na given polyseme, is a long-standing problem in NLP. In this paper, we propose\nto use BERT to extract better polyseme representations for WSD and explore\nseveral ways of combining BERT and the classifier. We also utilize sense\ndefinitions to train a unified classifier for all words, which enables the\nmodel to disambiguate unseen polysemes. Experiments show that our model\nachieves the state-of-the-art results on the standard English All-word WSD\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08504v1",
            "date": "2019-09-18 15:34:12+00:00",
            "title": "Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition",
            "authors": [
                "Genta Indra Winata",
                "Zhaojiang Lin",
                "Jamin Shin",
                "Zihan Liu",
                "Pascale Fung"
            ],
            "abstract": "In countries that speak multiple main languages, mixing up different\nlanguages within a conversation is commonly called code-switching. Previous\nworks addressing this challenge mainly focused on word-level aspects such as\nword embeddings. However, in many cases, languages share common subwords,\nespecially for closely related languages, but also for languages that are\nseemingly irrelevant. Therefore, we propose Hierarchical Meta-Embeddings (HME)\nthat learn to combine multiple monolingual word-level and subword-level\nembeddings to create language-agnostic lexical representations. On the task of\nNamed Entity Recognition for English-Spanish code-switching data, our model\nachieves the state-of-the-art performance in the multilingual settings. We also\nshow that, in cross-lingual settings, our model not only leverages closely\nrelated languages, but also learns from languages with different roots.\nFinally, we show that combining different subunits are crucial for capturing\ncode-switching entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08582v1",
            "date": "2019-09-18 17:11:41+00:00",
            "title": "Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences",
            "authors": [
                "Genta Indra Winata",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "Training code-switched language models is difficult due to lack of data and\ncomplexity in the grammatical structure. Linguistic constraint theories have\nbeen used for decades to generate artificial code-switching sentences to cope\nwith this issue. However, this require external word alignments or constituency\nparsers that create erroneous results on distant languages. We propose a\nsequence-to-sequence model using a copy mechanism to generate code-switching\ndata by leveraging parallel monolingual translations from a limited source of\ncode-switching data. The model learns how to combine words from parallel\nsentences and identifies when to switch one language to the other. Moreover, it\ncaptures code-switching constraints by attending and aligning the words in\ninputs, without requiring any external knowledge. Based on experimental\nresults, the language model trained with the generated sentences achieves\nstate-of-the-art performance and improves end-to-end automatic speech\nrecognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08681v1",
            "date": "2019-09-18 20:10:32+00:00",
            "title": "Cross-Lingual Contextual Word Embeddings Mapping With Multi-Sense Words In Mind",
            "authors": [
                "Zheng Zhang",
                "Ruiqing Yin",
                "Jun Zhu",
                "Pierre Zweigenbaum"
            ],
            "abstract": "Recent work in cross-lingual contextual word embedding learning cannot handle\nmulti-sense words well. In this work, we explore the characteristics of\ncontextual word embeddings and show the link between contextual word embeddings\nand word senses. We propose two improving solutions by considering contextual\nmulti-sense word embeddings as noise (removal) and by generating cluster level\naverage anchor embeddings for contextual multi-sense word embeddings\n(replacement). Experiments show that our solutions can improve the supervised\ncontextual word embeddings alignment for multi-sense words in a microscopic\nperspective without hurting the macroscopic performance on the bilingual\nlexicon induction task. For unsupervised alignment, our methods significantly\nimprove the performance on the bilingual lexicon induction task for more than\n10 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08686v1",
            "date": "2019-09-18 20:20:25+00:00",
            "title": "Sentiment-Aware Recommendation System for Healthcare using Social Media",
            "authors": [
                "Alan Aipe",
                "Mukuntha Narayanan Sundararaman",
                "Asif Ekbal"
            ],
            "abstract": "Over the last decade, health communities (known as forums) have evolved into\nplatforms where more and more users share their medical experiences, thereby\nseeking guidance and interacting with people of the community. The shared\ncontent, though informal and unstructured in nature, contains valuable medical\nand/or health-related information and can be leveraged to produce structured\nsuggestions to the common people. In this paper, at first we propose a stacked\ndeep learning model for sentiment analysis from the medical forum data. The\nstacked model comprises of Convolutional Neural Network (CNN) followed by a\nLong Short Term Memory (LSTM) and then by another CNN. For a blog classified\nwith positive sentiment, we retrieve the top-n similar posts. Thereafter, we\ndevelop a probabilistic model for suggesting the suitable treatments or\nprocedures for a particular disease or health condition. We believe that\nintegration of medical sentiment and suggestion would be beneficial to the\nusers for finding the relevant contents regarding medications and medical\nconditions, without having to manually stroll through a large amount of\nunstructured contents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08744v1",
            "date": "2019-09-19 00:23:09+00:00",
            "title": "Low-Resource Parsing with Crosslingual Contextualized Representations",
            "authors": [
                "Phoebe Mulcaire",
                "Jungo Kasai",
                "Noah A. Smith"
            ],
            "abstract": "Despite advances in dependency parsing, languages with small treebanks still\npresent challenges. We assess recent approaches to multilingual contextual word\nrepresentations (CWRs), and compare them for crosslingual transfer from a\nlanguage with a large treebank to a language with a small or nonexistent\ntreebank, by sharing parameters between languages in the parser itself. We\nexperiment with a diverse selection of languages in both simulated and truly\nlow-resource scenarios, and show that multilingual CWRs greatly facilitate\nlow-resource dependency parsing even without crosslingual supervision such as\ndictionaries or parallel text. Furthermore, we examine the non-contextual part\nof the learned language models (which we call a \"decontextual probe\") to\ndemonstrate that polyglot language models better encode crosslingual lexical\ncorrespondence compared to aligned monolingual language models. This analysis\nprovides further evidence that polyglot training is an effective approach to\ncrosslingual transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08796v1",
            "date": "2019-09-19 04:15:02+00:00",
            "title": "Made for Each Other: Broad-coverage Semantic Structures Meet Preposition Supersenses",
            "authors": [
                "Jakob Prange",
                "Nathan Schneider",
                "Omri Abend"
            ],
            "abstract": "Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013)\nis a typologically-informed, broad-coverage semantic annotation scheme that\ndescribes coarse-grained predicate-argument structure but currently lacks\nsemantic roles. We argue that lexicon-free annotation of the semantic roles\nmarked by prepositions, as formulated by Schneider et al. (2018b), is\ncomplementary and suitable for integration within UCCA. We show empirically for\nEnglish that the schemes, though annotated independently, are compatible and\ncan be combined in a single semantic graph. A comparison of several approaches\nto parsing the integrated representation lays the groundwork for future\nresearch on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08837v1",
            "date": "2019-09-19 07:36:54+00:00",
            "title": "How to Write Summaries with Patterns? Learning towards Abstractive Summarization through Prototype Editing",
            "authors": [
                "Shen Gao",
                "Xiuying Chen",
                "Piji Li",
                "Zhangming Chan",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Under special circumstances, summaries should conform to a particular style\nwith patterns, such as court judgments and abstracts in academic papers. To\nthis end, the prototype document-summary pairs can be utilized to generate\nbetter summaries. There are two main challenges in this task: (1) the model\nneeds to incorporate learned patterns from the prototype, but (2) should avoid\ncopying contents other than the patternized words---such as irrelevant\nfacts---into the generated summaries. To tackle these challenges, we design a\nmodel named Prototype Editing based Summary Generator (PESG). PESG first learns\nsummary patterns and prototype facts by analyzing the correlation between a\nprototype document and its summary. Prototype facts are then utilized to help\nextract facts from the input document. Next, an editing generator generates new\nsummary based on the summary pattern or extracted facts. Finally, to address\nthe second challenge, a fact checker is used to estimate mutual information\nbetween the input document and generated summary, providing an additional\nsignal for the generator. Extensive experiments conducted on a large-scale\nreal-world text summarization dataset show that PESG achieves the\nstate-of-the-art performance in terms of both automatic metrics and human\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08880v1",
            "date": "2019-09-19 09:20:32+00:00",
            "title": "An Edit-centric Approach for Wikipedia Article Quality Assessment",
            "authors": [
                "Edison Marrese-Taylor",
                "Pablo Loyola",
                "Yutaka Matsuo"
            ],
            "abstract": "We propose an edit-centric approach to assess Wikipedia article quality as a\ncomplementary alternative to current full document-based techniques. Our model\nconsists of a main classifier equipped with an auxiliary generative module\nwhich, for a given edit, jointly provides an estimation of its quality and\ngenerates a description in natural language. We performed an empirical study to\nassess the feasibility of the proposed model and its cost-effectiveness in\nterms of data and quality requirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08940v1",
            "date": "2019-09-19 12:38:22+00:00",
            "title": "Improving Generalization by Incorporating Coverage in Natural Language Inference",
            "authors": [
                "Nafise Sadat Moosavi",
                "Prasetya Ajie Utama",
                "Andreas R\u00fcckl\u00e9",
                "Iryna Gurevych"
            ],
            "abstract": "The task of natural language inference (NLI) is to identify the relation\nbetween the given premise and hypothesis. While recent NLI models achieve very\nhigh performance on individual datasets, they fail to generalize across similar\ndatasets. This indicates that they are solving NLI datasets instead of the task\nitself. In order to improve generalization, we propose to extend the input\nrepresentations with an abstract view of the relation between the hypothesis\nand the premise, i.e., how well the individual words, or word n-grams, of the\nhypothesis are covered by the premise. Our experiments show that the use of\nthis information considerably improves generalization across different NLI\ndatasets without requiring any external knowledge or additional data. Finally,\nwe show that using the coverage information is not only beneficial for\nimproving the performance across different datasets of the same task. The\nresulting generalization improves the performance across datasets that belong\nto similar but not the same tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08970v1",
            "date": "2019-09-19 13:21:05+00:00",
            "title": "RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation",
            "authors": [
                "Tzuf Paz-Argaman",
                "Reut Tsarfaty"
            ],
            "abstract": "Following navigation instructions in natural language requires a composition\nof language, action, and knowledge of the environment. Knowledge of the\nenvironment may be provided via visual sensors or as a symbolic world\nrepresentation referred to as a map. Here we introduce the Realistic Urban\nNavigation (RUN) task, aimed at interpreting navigation instructions based on a\nreal, dense, urban map. Using Amazon Mechanical Turk, we collected a dataset of\n2515 instructions aligned with actual routes over three regions of Manhattan.\nWe propose a strong baseline for the task and empirically investigate which\naspects of the neural architecture are important for the RUN success. Our\nresults empirically show that entity abstraction, attention over words and\nworlds, and a constantly updating world-state, significantly contribute to task\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09001v2",
            "date": "2019-09-19 13:57:17+00:00",
            "title": "CogniVal: A Framework for Cognitive Word Embedding Evaluation",
            "authors": [
                "Nora Hollenstein",
                "Antonio de la Torre",
                "Nicolas Langer",
                "Ce Zhang"
            ],
            "abstract": "An interesting method of evaluating word representations is by how much they\nreflect the semantic representations in the human brain. However, most, if not\nall, previous works only focus on small datasets and a single modality. In this\npaper, we present the first multi-modal framework for evaluating English word\nrepresentations based on cognitive lexical semantics. Six types of word\nembeddings are evaluated by fitting them to 15 datasets of eye-tracking, EEG\nand fMRI signals recorded during language processing. To achieve a global score\nover all evaluation hypotheses, we apply statistical significance testing\naccounting for the multiple comparisons problem. This framework is easily\nextensible and available to include other intrinsic and extrinsic evaluation\nmethods. We find strong correlations in the results between cognitive datasets,\nacross recording modalities and to their performance on extrinsic NLP tasks.",
            "journal ref": "Proceedings of the 23rd Conference on Computational Natural\n  Language Learning (CoNLL). 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09010v3",
            "date": "2019-09-19 14:11:36+00:00",
            "title": "A Random Gossip BMUF Process for Neural Language Modeling",
            "authors": [
                "Yiheng Huang",
                "Jinchuan Tian",
                "Lei Han",
                "Guangsen Wang",
                "Xingcheng Song",
                "Dan Su",
                "Dong Yu"
            ],
            "abstract": "Neural network language model (NNLM) is an essential component of industrial\nASR systems. One important challenge of training an NNLM is to leverage between\nscaling the learning process and handling big data. Conventional approaches\nsuch as block momentum provides a blockwise model update filtering (BMUF)\nprocess and achieves almost linear speedups with no performance degradation for\nspeech recognition. However, it needs to calculate the model average from all\ncomputing nodes (e.g., GPUs) and when the number of computing nodes is large,\nthe learning suffers from the severe communication latency. As a consequence,\nBMUF is not suitable under restricted network conditions. In this paper, we\npresent a decentralized BMUF process, in which the model is split into\ndifferent components, each of which is updated by communicating to some\nrandomly chosen neighbor nodes with the same component, followed by a BMUF-like\nprocess. We apply this method to several LSTM language modeling tasks.\nExperimental results show that our approach achieves consistently better\nperformance than conventional BMUF. In particular, we obtain a lower perplexity\nthan the single-GPU baseline on the wiki-text-103 benchmark using 4 GPUs. In\naddition, no performance degradation is observed when scaling to 8 and 16 GPUs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09031v1",
            "date": "2019-09-19 15:01:46+00:00",
            "title": "Argumentative Relation Classification as Plausibility Ranking",
            "authors": [
                "Juri Opitz"
            ],
            "abstract": "We formulate argumentative relation classification (support vs. attack) as a\ntext-plausibility ranking task. To this aim, we propose a simple reconstruction\ntrick which enables us to build minimal pairs of plausible and implausible\ntexts by simulating natural contexts in which two argumentative units are\nlikely or unlikely to appear. We show that this method is competitive with\nprevious work albeit it is considerably simpler. In a recently introduced\ncontent-based version of the task, where contextual discourse clues are hidden,\nthe approach offers a performance increase of more than 10% macro F1. With\nrespect to the scarce attack-class, the method achieves a large increase in\nprecision while the incurred loss in recall is small or even nonexistent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09067v1",
            "date": "2019-09-19 16:07:32+00:00",
            "title": "A Corpus for Automatic Readability Assessment and Text Simplification of German",
            "authors": [
                "Alessia Battisti",
                "Sarah Ebling"
            ],
            "abstract": "In this paper, we present a corpus for use in automatic readability\nassessment and automatic text simplification of German. The corpus is compiled\nfrom web sources and consists of approximately 211,000 sentences. As a novel\ncontribution, it contains information on text structure, typography, and\nimages, which can be exploited as part of machine learning approaches to\nreadability assessment and text simplification. The focus of this publication\nis on representing such information as an extension to an existing corpus\nstandard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09237v1",
            "date": "2019-09-19 21:16:29+00:00",
            "title": "Improved Variational Neural Machine Translation by Promoting Mutual Information",
            "authors": [
                "Arya D. McCarthy",
                "Xian Li",
                "Jiatao Gu",
                "Ning Dong"
            ],
            "abstract": "Posterior collapse plagues VAEs for text, especially for conditional text\ngeneration with strong autoregressive decoders. In this work, we address this\nproblem in variational neural machine translation by explicitly promoting\nmutual information between the latent variables and the data. Our model extends\nthe conditional variational autoencoder (CVAE) with two new ingredients: first,\nwe propose a modified evidence lower bound (ELBO) objective which explicitly\npromotes mutual information; second, we regularize the probabilities of the\ndecoder by mixing an auxiliary factorized distribution which is directly\npredicted by the latent variables. We present empirical results on the\nTransformer architecture and show the proposed model effectively addressed\nposterior collapse: latent variables are no longer ignored in the presence of\npowerful decoder. As a result, the proposed model yields improved translation\nquality while demonstrating superior performance in terms of data efficiency\nand robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09253v1",
            "date": "2019-09-19 22:49:04+00:00",
            "title": "What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering",
            "authors": [
                "Tushar Khot",
                "Ashish Sabharwal",
                "Peter Clark"
            ],
            "abstract": "Multi-hop textual question answering requires combining information from\nmultiple sentences. We focus on a natural setting where, unlike typical reading\ncomprehension, only partial information is provided with each question. The\nmodel must retrieve and use additional knowledge to correctly answer the\nquestion. To tackle this challenge, we develop a novel approach that explicitly\nidentifies the knowledge gap between a key span in the provided knowledge and\nthe answer choices. The model, GapQA, learns to fill this gap by determining\nthe relationship between the span and an answer choice, based on retrieved\nknowledge targeting this gap. We propose jointly training a model to\nsimultaneously fill this knowledge gap and compose it with the provided partial\nknowledge. On the OpenBookQA dataset, given partial knowledge, explicitly\nidentifying what's missing substantially outperforms previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09265v1",
            "date": "2019-09-20 00:12:39+00:00",
            "title": "Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages",
            "authors": [
                "Wasi Uddin Ahmad",
                "Zhisong Zhang",
                "Xuezhe Ma",
                "Kai-Wei Chang",
                "Nanyun Peng"
            ],
            "abstract": "Cross-lingual transfer learning has become an important weapon to battle the\nunavailability of annotated resources for low-resource languages. One of the\nfundamental techniques to transfer across languages is learning\n\\emph{language-agnostic} representations, in the form of word embeddings or\ncontextual encodings. In this work, we propose to leverage unannotated\nsentences from auxiliary languages to help learning language-agnostic\nrepresentations. Specifically, we explore adversarial training for learning\ncontextual encoders that produce invariant representations across languages to\nfacilitate cross-lingual transfer. We conduct experiments on cross-lingual\ndependency parsing where we train a dependency parser on a source language and\ntransfer it to a wide range of target languages. Experiments on 28 target\nlanguages demonstrate that adversarial training significantly improves the\noverall transfer performances under several different settings. We conduct a\ncareful analysis to evaluate the language-agnostic representations resulted\nfrom adversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09279v1",
            "date": "2019-09-20 01:07:11+00:00",
            "title": "Working Hard or Hardly Working: Challenges of Integrating Typology into Neural Dependency Parsers",
            "authors": [
                "Adam Fisch",
                "Jiang Guo",
                "Regina Barzilay"
            ],
            "abstract": "This paper explores the task of leveraging typology in the context of\ncross-lingual dependency parsing. While this linguistic information has shown\ngreat promise in pre-neural parsing, results for neural architectures have been\nmixed. The aim of our investigation is to better understand this\nstate-of-the-art. Our main findings are as follows: 1) The benefit of\ntypological information is derived from coarsely grouping languages into\nsyntactically-homogeneous clusters rather than from learning to leverage\nvariations along individual typological dimensions in a compositional manner;\n2) Typology consistent with the actual corpus statistics yields better transfer\nperformance; 3) Typological similarity is only a rough proxy of cross-lingual\ntransferability with respect to parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09317v1",
            "date": "2019-09-20 04:35:07+00:00",
            "title": "Jointly Learning Entity and Relation Representations for Entity Alignment",
            "authors": [
                "Yuting Wu",
                "Xiao Liu",
                "Yansong Feng",
                "Zheng Wang",
                "Dongyan Zhao"
            ],
            "abstract": "Entity alignment is a viable means for integrating heterogeneous knowledge\namong different knowledge graphs (KGs). Recent developments in the field often\ntake an embedding-based approach to model the structural information of KGs so\nthat entity alignment can be easily performed in the embedding space. However,\nmost existing works do not explicitly utilize useful relation representations\nto assist in entity alignment, which, as we will show in the paper, is a simple\nyet effective way for improving entity alignment. This paper presents a novel\njoint learning framework for entity alignment. At the core of our approach is a\nGraph Convolutional Network (GCN) based framework for learning both entity and\nrelation representations. Rather than relying on pre-aligned relation seeds to\nlearn relation representations, we first approximate them using entity\nembeddings learned by the GCN. We then incorporate the relation approximation\ninto entities to iteratively learn better representations for both. Experiments\nperformed on three real-world cross-lingual datasets show that our approach\nsubstantially outperforms state-of-the-art entity alignment methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09480v1",
            "date": "2019-09-19 11:37:12+00:00",
            "title": "Generating Philosophical Statements using Interpolated Markov Models and Dynamic Templates",
            "authors": [
                "Thomas Winters"
            ],
            "abstract": "Automatically imitating input text is a common task in natural language\ngeneration, often used to create humorous results. Classic algorithms for\nlearning to imitate text, e.g. simple Markov chains, usually have a trade-off\nbetween originality and syntactic correctness. We present two ways of\nautomatically parodying philosophical statements from examples overcoming this\nissue, and show how these can work in interactive systems as well. The first\nalgorithm uses interpolated Markov models with extensions to improve the\nquality of the generated texts. For the second algorithm, we propose\ndynamically extracting templates and filling these with new content. To\nillustrate these algorithms, we implemented TorfsBot, a Twitterbot imitating\nthe witty, semi-philosophical tweets of professor Rik Torfs, the previous KU\nLeuven rector. We found that users preferred generative models that focused on\nlocal coherent sentences, rather than those mimicking the global structure of a\nphilosophical statement. The proposed algorithms are thus valuable new tools\nfor automatic parody as well as template learning systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09483v1",
            "date": "2019-09-19 01:34:52+00:00",
            "title": "Multi-sense Definition Modeling using Word Sense Decompositions",
            "authors": [
                "Ruimin Zhu",
                "Thanapon Noraset",
                "Alisa Liu",
                "Wenxin Jiang",
                "Doug Downey"
            ],
            "abstract": "Word embeddings capture syntactic and semantic information about words.\nDefinition modeling aims to make the semantic content in each embedding\nexplicit, by outputting a natural language definition based on the embedding.\nHowever, existing definition models are limited in their ability to generate\naccurate definitions for different senses of the same word. In this paper, we\nintroduce a new method that enables definition modeling for multiple senses. We\nshow how a Gumble-Softmax approach outperforms baselines at matching\nsense-specific embeddings to definitions during training. In experiments, our\nmulti-sense definition model improves recall over a state-of-the-art\nsingle-sense definition model by a factor of three, without harming precision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09491v1",
            "date": "2019-09-15 18:18:35+00:00",
            "title": "A simple discriminative training method for machine translation with large-scale features",
            "authors": [
                "Tian Xia",
                "Shaodan Zhai",
                "Shaojun Wang"
            ],
            "abstract": "Margin infused relaxed algorithms (MIRAs) dominate model tuning in\nstatistical machine translation in the case of large scale features, but also\nthey are famous for the complexity in implementation. We introduce a new\nmethod, which regards an N-best list as a permutation and minimizes the\nPlackett-Luce loss of ground-truth permutations. Experiments with large-scale\nfeatures demonstrate that, the new method is more robust than MERT; though it\nis only matchable with MIRAs, it has a comparatively advantage, easier to\nimplement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09492v1",
            "date": "2019-09-17 08:57:07+00:00",
            "title": "Controllable Length Control Neural Encoder-Decoder via Reinforcement Learning",
            "authors": [
                "Junyi Bian",
                "Baojun Lin",
                "Ke Zhang",
                "Zhaohui Yan",
                "Hong Tang",
                "Yonghe Zhang"
            ],
            "abstract": "Controlling output length in neural language generation is valuable in many\nscenarios, especially for the tasks that have length constraints. A model with\nstronger length control capacity can produce sentences with more specific\nlength, however, it usually sacrifices semantic accuracy of the generated\nsentences. Here, we denote a concept of Controllable Length Control (CLC) for\nthe trade-off between length control capacity and semantic accuracy of the\nlanguage generation model. More specifically, CLC is to alter length control\ncapacity of the model so as to generate sentence with corresponding quality.\nThis is meaningful in real applications when length control capacity and\noutputs quality are requested with different priorities, or to overcome\nunstability of length control during model training. In this paper, we propose\ntwo reinforcement learning (RL) methods to adjust the trade-off between length\ncontrol capacity and semantic accuracy of length control models. Results show\nthat our RL methods improve scores across a wide range of target lengths and\nachieve the goal of CLC. Additionally, two models LenMC and LenLInit modified\non previous length-control models are proposed to obtain better performance in\nsummarization task while still maintain the ability to control length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09531v1",
            "date": "2019-09-20 14:37:05+00:00",
            "title": "Designing dialogue systems: A mean, grumpy, sarcastic chatbot in the browser",
            "authors": [
                "Suzana Ili\u0107",
                "Reiichiro Nakano",
                "Ivo Hajnal"
            ],
            "abstract": "In this work we explore a deep learning-based dialogue system that generates\nsarcastic and humorous responses from a conversation design perspective. We\ntrained a seq2seq model on a carefully curated dataset of 3000\nquestion-answering pairs, the core of our mean, grumpy, sarcastic chatbot. We\nshow that end-to-end systems learn patterns very quickly from small datasets\nand thus, are able to transfer simple linguistic structures representing\nabstract concepts to unseen settings. We also deploy our LSTM-based\nencoder-decoder model in the browser, where users can directly interact with\nthe chatbot. Human raters evaluated linguistic quality, creativity and\nhuman-like traits, revealing the system's strengths, limitations and potential\nfor future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09779v1",
            "date": "2019-09-21 06:16:52+00:00",
            "title": "Self-attention based end-to-end Hindi-English Neural Machine Translation",
            "authors": [
                "Siddhant Srivastava",
                "Ritu Tiwari"
            ],
            "abstract": "Machine Translation (MT) is a zone of concentrate in Natural Language\nprocessing which manages the programmed interpretation of human language,\nstarting with one language then onto the next by the PC. Having a rich research\nhistory spreading over about three decades, Machine interpretation is a\nstandout amongst the most looked for after region of research in the\ncomputational linguistics network. As a piece of this current ace's proposal,\nthe fundamental center examines the Deep-learning based strategies that have\ngained critical ground as of late and turning into the de facto strategy in MT.\nWe would like to point out the recent advances that have been put forward in\nthe field of Neural Translation models, different domains under which NMT has\nreplaced conventional SMT models and would also like to mention future avenues\nin the field. Consequently, we propose an end-to-end self-attention transformer\nnetwork for Neural Machine Translation, trained on Hindi-English parallel\ncorpus and compare the model's efficiency with other state of art models like\nencoder-decoder and attention-based encoder-decoder neural models on the basis\nof BLEU. We conclude this paper with a comparative analysis of the three\nproposed models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09855v1",
            "date": "2019-09-21 16:58:46+00:00",
            "title": "Low-Rank Approximation of Matrices for PMI-based Word Embeddings",
            "authors": [
                "Alena Sorokina",
                "Aidana Karipbayeva",
                "Zhenisbek Assylbekov"
            ],
            "abstract": "We perform an empirical evaluation of several methods of low-rank\napproximation in the problem of obtaining PMI-based word embeddings. All word\nvectors were trained on parts of a large corpus extracted from English\nWikipedia (enwik9) which was divided into two equal-sized datasets, from which\nPMI matrices were obtained. A repeated measures design was used in assigning a\nmethod of low-rank approximation (SVD, NMF, QR) and dimensionality of the\nvectors (250, 500) to each of the PMI matrix replicates. Our experiments show\nthat word vectors obtained from the truncated SVD achieve the best performance\non two downstream tasks, similarity and analogy, compare to the other two\nlow-rank approximation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09907v1",
            "date": "2019-09-21 21:57:38+00:00",
            "title": "Generating Timelines by Modeling Semantic Change",
            "authors": [
                "Guy D. Rosin",
                "Kira Radinsky"
            ],
            "abstract": "Though languages can evolve slowly, they can also react strongly to dramatic\nworld events. By studying the connection between words and events, it is\npossible to identify which events change our vocabulary and in what way. In\nthis work, we tackle the task of creating timelines - records of historical\n\"turning points\", represented by either words or events, to understand the\ndynamics of a target word. Our approach identifies these points by leveraging\nboth static and time-varying word embeddings to measure the influence of words\nand events. In addition to quantifying changes, we show how our technique can\nhelp isolate semantic changes. Our qualitative and quantitative evaluations\nshow that we are able to capture this semantic change and event influence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09986v1",
            "date": "2019-09-22 11:41:53+00:00",
            "title": "Improving Quality and Efficiency in Plan-based Neural Data-to-Text Generation",
            "authors": [
                "Amit Moryossef",
                "Ido Dagan",
                "Yoav Goldberg"
            ],
            "abstract": "We follow the step-by-step approach to neural data-to-text generation we\nproposed in Moryossef et al (2019), in which the generation process is divided\ninto a text-planning stage followed by a plan-realization stage. We suggest\nfour extensions to that framework: (1) we introduce a trainable neural planning\ncomponent that can generate effective plans several orders of magnitude faster\nthan the original planner; (2) we incorporate typing hints that improve the\nmodel's ability to deal with unseen relations and entities; (3) we introduce a\nverification-by-reranking stage that substantially improves the faithfulness of\nthe resulting texts; (4) we incorporate a simple but effective referring\nexpression generation module. These extensions result in a generation process\nthat is faster, more fluent, and more accurate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09993v1",
            "date": "2019-09-22 12:41:05+00:00",
            "title": "Improving OOV Detection and Resolution with External Language Models in Acoustic-to-Word ASR",
            "authors": [
                "Hirofumi Inaguma",
                "Masato Mimura",
                "Shinsuke Sakai",
                "Tatsuya Kawahara"
            ],
            "abstract": "Acoustic-to-word (A2W) end-to-end automatic speech recognition (ASR) systems\nhave attracted attention because of an extremely simplified architecture and\nfast decoding. To alleviate data sparseness issues due to infrequent words, the\ncombination with an acoustic-to-character (A2C) model is investigated.\nMoreover, the A2C model can be used to recover out-of-vocabulary (OOV) words\nthat are not covered by the A2W model, but this requires accurate detection of\nOOV words. A2W models learn contexts with both acoustic and transcripts;\ntherefore they tend to falsely recognize OOV words as words in the vocabulary.\nIn this paper, we tackle this problem by using external language models (LM),\nwhich are trained only with transcriptions and have better linguistic\ninformation to detect OOV words. The A2C model is used to resolve these OOV\nwords. Experimental evaluations show that external LMs have the effects of not\nonly reducing errors but also increasing the number of detected OOV words, and\nthe proposed method significantly improves performances in English\nconversational and Japanese lecture corpora, especially for out-of-domain\nscenario. We also investigate the impact of the vocabulary size of A2W models\nand the data size for training LMs. Moreover, our approach can reduce the\nvocabulary size several times with marginal performance degradation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10012v1",
            "date": "2019-09-22 14:17:50+00:00",
            "title": "Is change the only constant? Profile change perspective on #LokSabhaElections2019",
            "authors": [
                "Kumari Neha",
                "Shashank Srikanth",
                "Sonali Singhal",
                "Shwetanshu Singh",
                "Arun Balaji Buduru",
                "Ponnurangam Kumaraguru"
            ],
            "abstract": "Users on Twitter are identified with the help of their profile attributes\nthat consists of username, display name, profile image, to name a few. The\nprofile attributes that users adopt can reflect their interests, belief, or\nthematic inclinations. Literature has proposed the implications and\nsignificance of profile attribute change for a random population of users.\nHowever, the use of profile attribute for endorsements and to start a movement\nhave been under-explored. In this work, we consider #LokSabhaElections2019 as a\nmovement and perform a large-scale study of the profile of users who actively\nmade changes to profile attributes centered around #LokSabhaElections2019. We\ncollect the profile metadata for 49.4M users for a period of 2 months from\nApril 5, 2019 to June 5, 2019 amid #LokSabhaElections2019. We investigate how\nthe profile changes vary for the influential leaders and their followers over\nthe social movement. We further differentiate the organic and inorganic ways to\nshow the political inclination from the prism of profile changes. We report how\nthe addition of election campaign related keywords lead to spread of behavior\ncontagion and further investigate it with respect to \"Chowkidar Movement\" in\ndetail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10056v1",
            "date": "2019-09-22 18:01:36+00:00",
            "title": "Inducing Constituency Trees through Neural Machine Translation",
            "authors": [
                "Phu Mon Htut",
                "Kyunghyun Cho",
                "Samuel R. Bowman"
            ],
            "abstract": "Latent tree learning(LTL) methods learn to parse sentences using only\nindirect supervision from a downstream task. Recent advances in latent tree\nlearning have made it possible to recover moderately high quality tree\nstructures by training with language modeling or auto-encoding objectives. In\nthis work, we explore the hypothesis that decoding in machine translation, as a\nconditional language modeling task, will produce better tree structures since\nit offers a similar training signal as language modeling, but with more\nsemantic signal. We adapt two existing latent-tree language models--PRPN\nandON-LSTM--for use in translation. We find that they indeed recover trees that\nare better in F1 score than those seen in language modeling on WSJ test set,\nwhile maintaining strong translation quality. We observe that translation is a\nbetter objective than language modeling for inducing trees, marking the first\nsuccess at latent tree learning using a machine translation objective.\nAdditionally, our findings suggest that, although translation provides better\nsignal for inducing trees than language modeling, translation models can\nperform well without exploiting the latent tree structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10063v1",
            "date": "2019-09-22 18:24:41+00:00",
            "title": "Algorithms for certain classes of Tamil Spelling correction",
            "authors": [
                "Muthiah Annamalai",
                "T. Shrinivasan"
            ],
            "abstract": "Tamil language has an agglutinative, diglossic, alpha-syllabary structure\nwhich provides a significant combinatorial explosion of morphological forms all\nof which are effectively used in Tamil prose, poetry from antiquity to the\nmodern age in an unbroken chain of continuity. However, for the language\nunderstanding, spelling correction purposes some of these present challenges as\nout-of-dictionary words. In this paper the authors propose algorithmic\ntechniques to handle specific problems of conjoined-words (out-of-dictionary)\n(transliteration)[thendRalkattRu] = [thendRal]+[kattRu] when parts are alone\npresent in word-list in efficient way. Morphological structure of Tamil makes\nit necessary to depend on synthesis-analysis approach and dictionary lists will\nnever be sufficient to truly capture the language. In this paper we have\nattempted to make a summary of various known algorithms for specific classes of\nTamil spelling errors. We believe this collection of suggestions to improve\nfuture spelling checkers. We also note do not cover many important techniques\nlike affix removal and other such techniques of key importance in rule-based\nspell checkers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10094v2",
            "date": "2019-09-22 21:11:08+00:00",
            "title": "Deep Structured Neural Network for Event Temporal Relation Extraction",
            "authors": [
                "Rujun Han",
                "I-Hung Hsu",
                "Mu Yang",
                "Aram Galstyan",
                "Ralph Weischedel",
                "Nanyun Peng"
            ],
            "abstract": "We propose a novel deep structured learning framework for event temporal\nrelation extraction. The model consists of 1) a recurrent neural network (RNN)\nto learn scoring functions for pair-wise relations, and 2) a structured support\nvector machine (SSVM) to make joint predictions. The neural network\nautomatically learns representations that account for long-term contexts to\nprovide robust features for the structured model, while the SSVM incorporates\ndomain knowledge such as transitive closure of temporal relations as\nconstraints to make better globally consistent decisions. By jointly training\nthe two components, our model combines the benefits of both data-driven\nlearning and knowledge exploitation. Experimental results on three high-quality\nevent temporal relation datasets (TCR, MATRES, and TB-Dense) demonstrate that\nincorporated with pre-trained contextualized embeddings, the proposed model\nachieves significantly better performances than the state-of-the-art methods on\nall three datasets. We also provide thorough ablation studies to investigate\nour model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10122v1",
            "date": "2019-09-23 01:45:55+00:00",
            "title": "Towards Best Experiment Design for Evaluating Dialogue System Output",
            "authors": [
                "Sashank Santhanam",
                "Samira Shaikh"
            ],
            "abstract": "To overcome the limitations of automated metrics (e.g. BLEU, METEOR) for\nevaluating dialogue systems, researchers typically use human judgments to\nprovide convergent evidence. While it has been demonstrated that human\njudgments can suffer from the inconsistency of ratings, extant research has\nalso found that the design of the evaluation task affects the consistency and\nquality of human judgments. We conduct a between-subjects study to understand\nthe impact of four experiment conditions on human ratings of dialogue system\noutput. In addition to discrete and continuous scale ratings, we also\nexperiment with a novel application of Best-Worst scaling to dialogue\nevaluation. Through our systematic study with 40 crowdsourced workers in each\ntask, we find that using continuous scales achieves more consistent ratings\nthan Likert scale or ranking-based experiment design. Additionally, we find\nthat factors such as time taken to complete the task and no prior experience of\nparticipating in similar studies of rating dialogue system output positively\nimpact consistency and agreement amongst raters",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10148v1",
            "date": "2019-09-23 04:21:35+00:00",
            "title": "Dependency-Guided LSTM-CRF for Named Entity Recognition",
            "authors": [
                "Zhanming Jie",
                "Wei Lu"
            ],
            "abstract": "Dependency tree structures capture long-distance and syntactic relationships\nbetween words in a sentence. The syntactic relations (e.g., nominal subject,\nobject) can potentially infer the existence of certain named entities. In\naddition, the performance of a named entity recognizer could benefit from the\nlong-distance dependencies between the words in dependency trees. In this work,\nwe propose a simple yet effective dependency-guided LSTM-CRF model to encode\nthe complete dependency trees and capture the above properties for the task of\nnamed entity recognition (NER). The data statistics show strong correlations\nbetween the entity types and dependency relations. We conduct extensive\nexperiments on several standard datasets and demonstrate the effectiveness of\nthe proposed model in improving NER and achieving state-of-the-art performance.\nOur analysis reveals that the significant improvements mainly result from the\ndependency relations and long-distance interactions provided by dependency\ntrees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10171v1",
            "date": "2019-09-23 05:56:13+00:00",
            "title": "Syntax-Aware Aspect-Level Sentiment Classification with Proximity-Weighted Convolution Network",
            "authors": [
                "Chen Zhang",
                "Qiuchi Li",
                "Dawei Song"
            ],
            "abstract": "It has been widely accepted that Long Short-Term Memory (LSTM) network,\ncoupled with attention mechanism and memory module, is useful for aspect-level\nsentiment classification. However, existing approaches largely rely on the\nmodelling of semantic relatedness of an aspect with its context words, while to\nsome extent ignore their syntactic dependencies within sentences. Consequently,\nthis may lead to an undesirable result that the aspect attends on contextual\nwords that are descriptive of other aspects. In this paper, we propose a\nproximity-weighted convolution network to offer an aspect-specific syntax-aware\nrepresentation of contexts. In particular, two ways of determining proximity\nweight are explored, namely position proximity and dependency proximity. The\nrepresentation is primarily abstracted by a bidirectional LSTM architecture and\nfurther enhanced by a proximity-weighted convolution. Experiments conducted on\nthe SemEval 2014 benchmark demonstrate the effectiveness of our proposed\napproach compared with a range of state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10324v1",
            "date": "2019-09-23 12:27:04+00:00",
            "title": "Speech Replay Detection with x-Vector Attack Embeddings and Spectral Features",
            "authors": [
                "Jennifer Williams",
                "Joanna Rownicka"
            ],
            "abstract": "We present our system submission to the ASVspoof 2019 Challenge Physical\nAccess (PA) task. The objective for this challenge was to develop a\ncountermeasure that identifies speech audio as either bona fide or intercepted\nand replayed. The target prediction was a value indicating that a speech\nsegment was bona fide (positive values) or \"spoofed\" (negative values). Our\nsystem used convolutional neural networks (CNNs) and a representation of the\nspeech audio that combined x-vector attack embeddings with signal processing\nfeatures. The x-vector attack embeddings were created from mel-frequency\ncepstral coefficients (MFCCs) using a time-delay neural network (TDNN). These\nembeddings jointly modeled 27 different environments and 9 types of attacks\nfrom the labeled data. We also used sub-band spectral centroid magnitude\ncoefficients (SCMCs) as features. We included an additive Gaussian noise layer\nduring training as a way to augment the data to make our system more robust to\npreviously unseen attack examples. We report system performance using the\ntandem detection cost function (tDCF) and equal error rate (EER). Our approach\nperformed better that both of the challenge baselines. Our technique suggests\nthat our x-vector attack embeddings can help regularize the CNN predictions\neven when environments or attacks are more challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10368v1",
            "date": "2019-09-23 13:57:47+00:00",
            "title": "A Consolidated System for Robust Multi-Document Entity Risk Extraction and Taxonomy Augmentation",
            "authors": [
                "Berk Ekmekci",
                "Eleanor Hagerman",
                "Blake Howald"
            ],
            "abstract": "We introduce a hybrid human-automated system that provides scalable\nentity-risk relation extractions across large data sets. Given an\nexpert-defined keyword taxonomy, entities, and data sources, the system returns\ntext extractions based on bidirectional token distances between entities and\nkeywords and expands taxonomy coverage with word vector encodings. Our system\nrepresents a more simplified architecture compared to alerting focused systems\n- motivated by high coverage use cases in the risk mining space such as due\ndiligence activities and intelligence gathering. We provide an overview of the\nsystem and expert evaluations for a range of token distances. We demonstrate\nthat single and multi-sentence distance groups significantly outperform\nbaseline extractions with shorter, single sentences being preferred by\nanalysts. As the taxonomy expands, the amount of relevant information increases\nand multi-sentence extractions become more preferred, but this is tempered\nagainst entity-risk relations become more indirect. We discuss the implications\nof these observations on users, management of ambiguity and taxonomy expansion,\nand future system modifications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10390v1",
            "date": "2019-09-23 14:35:23+00:00",
            "title": "GNTeam at 2018 n2c2: Feature-augmented BiLSTM-CRF for drug-related entity recognition in hospital discharge summaries",
            "authors": [
                "Maksim Belousov",
                "Nikola Milosevic",
                "Ghada Alfattni",
                "Haifa Alrdahi",
                "Goran Nenadic"
            ],
            "abstract": "Monitoring the administration of drugs and adverse drug reactions are key\nparts of pharmacovigilance. In this paper, we explore the extraction of drug\nmentions and drug-related information (reason for taking a drug, route,\nfrequency, dosage, strength, form, duration, and adverse events) from hospital\ndischarge summaries through deep learning that relies on various\nrepresentations for clinical named entity recognition. This work was officially\npart of the 2018 n2c2 shared task, and we use the data supplied as part of the\ntask. We developed two deep learning architecture based on recurrent neural\nnetworks and pre-trained language models. We also explore the effect of\naugmenting word representations with semantic features for clinical named\nentity recognition. Our feature-augmented BiLSTM-CRF model performed with\nF1-score of 92.67% and ranked 4th for entity extraction sub-task among\nsubmitted systems to n2c2 challenge. The recurrent neural networks that use the\npre-trained domain-specific word embeddings and a CRF layer for label\noptimization perform drug, adverse event and related entities extraction with\nmicro-averaged F1-score of over 91%. The augmentation of word vectors with\nsemantic features extracted using available clinical NLP toolkits can further\nimprove the performance. Word embeddings that are pre-trained on a large\nunannotated corpus of relevant documents and further fine-tuned to the task\nperform rather well. However, the augmentation of word embeddings with semantic\nfeatures can help improve the performance (primarily by boosting precision) of\ndrug-related named entity recognition from electronic health records.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10393v1",
            "date": "2019-09-23 14:37:30+00:00",
            "title": "Specificity-Based Sentence Ordering for Multi-Document Extractive Risk Summarization",
            "authors": [
                "Berk Ekmekci",
                "Eleanor Hagerman",
                "Blake Howald"
            ],
            "abstract": "Risk mining technologies seek to find relevant textual extractions that\ncapture entity-risk relationships. However, when high volume data sets are\nprocessed, a multitude of relevant extractions can be returned, shifting the\nfocus to how best to present the results. We provide the details of a risk\nmining multi-document extractive summarization system that produces high\nquality output by modeling shifts in specificity that are characteristic of\nwell-formed discourses. In particular, we propose a novel selection algorithm\nthat alternates between extracts based on human curated or expanded autoencoded\nkey terms, which exhibit greater specificity or generality as it relates to an\nentity-risk relationship. Through this extract ordering, and without the need\nfor more complex discourse-aware NLP, we induce felicitous shifts in\nspecificity in the alternating summaries that outperform non-alternating\nsummaries on automatic ROUGE and BLEU scores, and manual understandability and\npreferences evaluations - achieving no statistically significant difference\nwhen compared to human authored summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10413v1",
            "date": "2019-09-23 15:12:45+00:00",
            "title": "Automated Chess Commentator Powered by Neural Chess Engine",
            "authors": [
                "Hongyu Zang",
                "Zhiwei Yu",
                "Xiaojun Wan"
            ],
            "abstract": "In this paper, we explore a new approach for automated chess commentary\ngeneration, which aims to generate chess commentary texts in different\ncategories (e.g., description, comparison, planning, etc.). We introduce a\nneural chess engine into text generation models to help with encoding boards,\npredicting moves, and analyzing situations. By jointly training the neural\nchess engine and the generation models for different categories, the models\nbecome more effective. We conduct experiments on 5 categories in a benchmark\nChess Commentary dataset and achieve inspiring results in both automatic and\nhuman evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10416v1",
            "date": "2019-09-23 15:14:56+00:00",
            "title": "Biomedical Mention Disambiguation using a Deep Learning Approach",
            "authors": [
                "Chih-Hsuan Wei",
                "Kyubum Lee",
                "Robert Leaman",
                "Zhiyong Lu"
            ],
            "abstract": "Automatically locating named entities in natural language text - named entity\nrecognition - is an important task in the biomedical domain. Many named entity\nmentions are ambiguous between several bioconcept types, however, causing text\nspans to be annotated as more than one type when simultaneously recognizing\nmultiple entity types. The straightforward solution is a rule-based approach\napplying a priority order based on the precision of each entity tagger (from\nhighest to lowest). While this method is straightforward and useful, imprecise\ndisambiguation remains a significant source of error. We address this issue by\ngenerating a partially labeled corpus of ambiguous concept mentions. We first\ncollect named entity mentions from multiple human-curated databases (e.g.\nCTDbase, gene2pubmed), then correlate them with the text mined span from\nPubTator to provide the context where the mention appears. Our corpus contains\nmore than 3 million concept mentions that ambiguous between one or more concept\ntypes in PubTator (about 3% of all mentions). We approached this task as a\nclassification problem and developed a deep learning-based method which uses\nthe semantics of the span being classified and the surrounding words to\nidentify the most likely bioconcept type. More specifically, we develop a\nconvolutional neural network (CNN) and along short-term memory (LSTM) network\nto respectively handle the semantic syntax features, then concatenate these\nwithin a fully connected layer for final classification. The priority ordering\nrule-based approach demonstrated F1-scores of 71.29% (micro-averaged) and\n41.19% (macro-averaged), while the new disambiguation method demonstrated\nF1-scores of 91.94% (micro-averaged) and 85.42% (macro-averaged), a very\nsubstantial increase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10430v2",
            "date": "2019-09-23 15:38:02+00:00",
            "title": "Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings",
            "authors": [
                "Gregor Wiedemann",
                "Steffen Remus",
                "Avi Chawla",
                "Chris Biemann"
            ],
            "abstract": "Contextualized word embeddings (CWE) such as provided by ELMo (Peters et al.,\n2018), Flair NLP (Akbik et al., 2018), or BERT (Devlin et al., 2019) are a\nmajor recent innovation in NLP. CWEs provide semantic vector representations of\nwords depending on their respective context. Their advantage over static word\nembeddings has been shown for a number of tasks, such as text classification,\nsequence tagging, or machine translation. Since vectors of the same word type\ncan vary depending on the respective context, they implicitly provide a model\nfor word sense disambiguation (WSD). We introduce a simple but effective\napproach to WSD using a nearest neighbor classification on CWEs. We compare the\nperformance of different CWE models for the task and can report improvements\nabove the current state of the art for two standard WSD benchmark datasets. We\nfurther show that the pre-trained BERT model is able to place polysemic words\ninto distinct 'sense' regions of the embedding space, while ELMo and Flair NLP\ndo not seem to possess this ability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10481v3",
            "date": "2019-09-23 16:59:28+00:00",
            "title": "Cross-Lingual Natural Language Generation via Pre-Training",
            "authors": [
                "Zewen Chi",
                "Li Dong",
                "Furu Wei",
                "Wenhui Wang",
                "Xian-Ling Mao",
                "Heyan Huang"
            ],
            "abstract": "In this work we focus on transferring supervision signals of natural language\ngeneration (NLG) tasks between multiple languages. We propose to pretrain the\nencoder and the decoder of a sequence-to-sequence model under both monolingual\nand cross-lingual settings. The pre-training objective encourages the model to\nrepresent different languages in the shared space, so that we can conduct\nzero-shot cross-lingual transfer. After the pre-training procedure, we use\nmonolingual data to fine-tune the pre-trained model on downstream NLG tasks.\nThen the sequence-to-sequence model trained in a single language can be\ndirectly evaluated beyond that language (i.e., accepting multi-lingual input\nand producing multi-lingual output). Experimental results on question\ngeneration and abstractive summarization show that our model outperforms the\nmachine-translation-based pipeline methods for zero-shot cross-lingual\ngeneration. Moreover, cross-lingual transfer improves NLG performance of\nlow-resource languages by leveraging rich-resource language data. Our\nimplementation and data are available at https://github.com/CZWin32768/xnlg.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10579v1",
            "date": "2019-09-23 19:13:33+00:00",
            "title": "Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models",
            "authors": [
                "Grusha Prasad",
                "Marten van Schijndel",
                "Tal Linzen"
            ],
            "abstract": "Neural language models (LMs) perform well on tasks that require sensitivity\nto syntactic structure. Drawing on the syntactic priming paradigm from\npsycholinguistics, we propose a novel technique to analyze the representations\nthat enable such success. By establishing a gradient similarity metric between\nstructures, this technique allows us to reconstruct the organization of the\nLMs' syntactic representational space. We use this technique to demonstrate\nthat LSTM LMs' representations of different types of sentences with relative\nclauses are organized hierarchically in a linguistically interpretable manner,\nsuggesting that the LMs track abstract properties of the sentence.",
            "journal ref": "CoNLL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10599v1",
            "date": "2019-09-23 20:10:56+00:00",
            "title": "Multi-stage Pretraining for Abstractive Summarization",
            "authors": [
                "Sebastian Goodman",
                "Zhenzhong Lan",
                "Radu Soricut"
            ],
            "abstract": "Neural models for abstractive summarization tend to achieve the best\nperformance in the presence of highly specialized, summarization specific\nmodeling add-ons such as pointer-generator, coverage-modeling, and\ninferencetime heuristics. We show here that pretraining can complement such\nmodeling advancements to yield improved results in both short-form and\nlong-form abstractive summarization using two key concepts: full-network\ninitialization and multi-stage pretraining. Our method allows the model to\ntransitively benefit from multiple pretraining tasks, from generic language\ntasks to a specialized summarization task to an even more specialized one such\nas bullet-based summarization. Using this approach, we demonstrate improvements\nof 1.05 ROUGE-L points on the Gigaword benchmark and 1.78 ROUGE-L points on the\nCNN/DailyMail benchmark, compared to a randomly-initialized baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10724v1",
            "date": "2019-09-24 06:03:35+00:00",
            "title": "Situating Sentence Embedders with Nearest Neighbor Overlap",
            "authors": [
                "Lucy H. Lin",
                "Noah A. Smith"
            ],
            "abstract": "As distributed approaches to natural language semantics have developed and\ndiversified, embedders for linguistic units larger than words have come to play\nan increasingly important role. To date, such embedders have been evaluated\nusing benchmark tasks (e.g., GLUE) and linguistic probes. We propose a\ncomparative approach, nearest neighbor overlap (N2O), that quantifies\nsimilarity between embedders in a task-agnostic manner. N2O requires only a\ncollection of examples and is simple to understand: two embedders are more\nsimilar if, for the same set of inputs, there is greater overlap between the\ninputs' nearest neighbors. Though applicable to embedders of texts of any size,\nwe focus on sentence embedders and use N2O to show the effects of different\ndesign choices and architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10743v2",
            "date": "2019-09-24 07:36:45+00:00",
            "title": "An Empirical Study of Content Understanding in Conversational Question Answering",
            "authors": [
                "Ting-Rui Chiang",
                "Hao-Tong Ye",
                "Yun-Nung Chen"
            ],
            "abstract": "With a lot of work about context-free question answering systems, there is an\nemerging trend of conversational question answering models in the natural\nlanguage processing field. Thanks to the recently collected datasets, including\nQuAC and CoQA, there has been more work on conversational question answering,\nand recent work has achieved competitive performance on both datasets. However,\nto best of our knowledge, two important questions for conversational\ncomprehension research have not been well studied: 1) How well can the\nbenchmark dataset reflect models' content understanding? 2) Do the models well\nutilize the conversation content when answering questions? To investigate these\nquestions, we design different training settings, testing settings, as well as\nan attack to verify the models' capability of content understanding on QuAC and\nCoQA. The experimental results indicate some potential hazards in the benchmark\ndatasets, QuAC and CoQA, for conversational comprehension research. Our\nanalysis also sheds light on both what models may learn and how datasets may\nbias the models. With deep investigation of the task, it is believed that this\nwork can benefit the future progress of conversation comprehension. The source\ncode is available at https://github.com/MiuLab/CQA-Study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10772v1",
            "date": "2019-09-24 09:26:24+00:00",
            "title": "Technical report on Conversational Question Answering",
            "authors": [
                "Ying Ju",
                "Fubang Zhao",
                "Shijie Chen",
                "Bowen Zheng",
                "Xuefeng Yang",
                "Yunfeng Liu"
            ],
            "abstract": "Conversational Question Answering is a challenging task since it requires\nunderstanding of conversational history. In this project, we propose a new\nsystem RoBERTa + AT +KD, which involves rationale tagging multi-task,\nadversarial training, knowledge distillation and a linguistic post-process\nstrategy. Our single model achieves 90.4(F1) on the CoQA test set without data\naugmentation, outperforming the current state-of-the-art single model by 2.6%\nF1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10852v2",
            "date": "2019-09-24 12:53:36+00:00",
            "title": "In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes",
            "authors": [
                "Lei Li",
                "Wei Liu",
                "Marina Litvak",
                "Natalia Vanetik",
                "Zuying Huang"
            ],
            "abstract": "Various Seq2Seq learning models designed for machine translation were applied\nfor abstractive summarization task recently. Despite these models provide high\nROUGE scores, they are limited to generate comprehensive summaries with a high\nlevel of abstraction due to its degenerated attention distribution. We\nintroduce Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) using\nDeterminantal Point Processes methods(Micro DPPs and Macro DPPs) to produce\nattention distribution considering both quality and diversity. Without breaking\nthe end to end architecture, DivCNN Seq2Seq achieves a higher level of\ncomprehensiveness compared to vanilla models and strong baselines. All the\nreproducible codes and datasets are available online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10892v1",
            "date": "2019-09-24 13:27:30+00:00",
            "title": "Code-switching Language Modeling With Bilingual Word Embeddings: A Case Study for Egyptian Arabic-English",
            "authors": [
                "Injy Hamed",
                "Moritz Zhu",
                "Mohamed Elmahdy",
                "Slim Abdennadher",
                "Ngoc Thang Vu"
            ],
            "abstract": "Code-switching (CS) is a widespread phenomenon among bilingual and\nmultilingual societies. The lack of CS resources hinders the performance of\nmany NLP tasks. In this work, we explore the potential use of bilingual word\nembeddings for code-switching (CS) language modeling (LM) in the low resource\nEgyptian Arabic-English language. We evaluate different state-of-the-art\nbilingual word embeddings approaches that require cross-lingual resources at\ndifferent levels and propose an innovative but simple approach that jointly\nlearns bilingual word representations without the use of any parallel data,\nrelying only on monolingual and a small amount of CS data. While all\nrepresentations improve CS LM, ours performs the best and improves perplexity\n33.5% relative over the baseline.",
            "journal ref": "Proceedings of the 21st International Conference on Speech and\n  Computer (SPECOM'19), Istanbul, Turkey, August 20-25, 2019\n  https://link.springer.com/book/10.1007/978-3-030-26061-3",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10955v2",
            "date": "2019-09-24 14:32:52+00:00",
            "title": "Efficiently Reusing Old Models Across Languages via Transfer Learning",
            "authors": [
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Recent progress in neural machine translation is directed towards larger\nneural networks trained on an increasing amount of hardware resources. As a\nresult, NMT models are costly to train, both financially, due to the\nelectricity and hardware cost, and environmentally, due to the carbon\nfootprint. It is especially true in transfer learning for its additional cost\nof training the \"parent\" model before transferring knowledge and training the\ndesired \"child\" model. In this paper, we propose a simple method of re-using an\nalready trained model for different language pairs where there is no need for\nmodifications in model architecture. Our approach does not need a separate\nparent model for each investigated language pair, as it is typical in NMT\ntransfer learning. To show the applicability of our method, we recycle a\nTransformer model trained by different researchers and use it to seed models\nfor different language pairs. We achieve better translation quality and shorter\nconvergence times than when training from random initialization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11042v1",
            "date": "2019-09-24 16:52:18+00:00",
            "title": "Assessing the Lexico-Semantic Relational Knowledge Captured by Word and Concept Embeddings",
            "authors": [
                "Ronald Denaux",
                "Jose Manuel Gomez-Perez"
            ],
            "abstract": "Deep learning currently dominates the benchmarks for various NLP tasks and,\nat the basis of such systems, words are frequently represented as embeddings\n--vectors in a low dimensional space-- learned from large text corpora and\nvarious algorithms have been proposed to learn both word and concept\nembeddings. One of the claimed benefits of such embeddings is that they capture\nknowledge about semantic relations. Such embeddings are most often evaluated\nthrough tasks such as predicting human-rated similarity and analogy which only\ntest a few, often ill-defined, relations. In this paper, we propose a method\nfor (i) reliably generating word and concept pair datasets for a wide number of\nrelations by using a knowledge graph and (ii) evaluating to what extent\npre-trained embeddings capture those relations. We evaluate the approach\nagainst a proprietary and a public knowledge graph and analyze the results,\nshowing which lexico-semantic relational knowledge is captured by current\nembedding learning approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11049v1",
            "date": "2019-09-24 17:02:32+00:00",
            "title": "Neural Generative Rhetorical Structure Parsing",
            "authors": [
                "Amandla Mabona",
                "Laura Rimell",
                "Stephen Clark",
                "Andreas Vlachos"
            ],
            "abstract": "Rhetorical structure trees have been shown to be useful for several\ndocument-level tasks including summarization and document classification.\nPrevious approaches to RST parsing have used discriminative models; however,\nthese are less sample efficient than generative models, and RST parsing\ndatasets are typically small. In this paper, we present the first generative\nmodel for RST parsing. Our model is a document-level RNN grammar (RNNG) with a\nbottom-up traversal order. We show that, for our parser's traversal order,\nprevious beam search algorithms for RNNGs have a left-branching bias which is\nill-suited for RST parsing. We develop a novel beam search algorithm that keeps\ntrack of both structure- and word-generating actions without exhibiting this\nbranching bias and results in absolute improvements of 6.8 and 2.9 on\nunlabelled and labelled F1 over previous algorithms. Overall, our generative\nmodel outperforms a discriminative model with the same features by 2.6 F1\npoints and achieves performance comparable to the state-of-the-art,\noutperforming all published parsers from a recent replication study that do not\nuse additional training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11291v1",
            "date": "2019-09-25 05:16:15+00:00",
            "title": "Question Answering is a Format; When is it Useful?",
            "authors": [
                "Matt Gardner",
                "Jonathan Berant",
                "Hannaneh Hajishirzi",
                "Alon Talmor",
                "Sewon Min"
            ],
            "abstract": "Recent years have seen a dramatic expansion of tasks and datasets posed as\nquestion answering, from reading comprehension, semantic role labeling, and\neven machine translation, to image and video understanding. With this\nexpansion, there are many differing views on the utility and definition of\n\"question answering\" itself. Some argue that its scope should be narrow, or\nbroad, or that it is overused in datasets today. In this opinion piece, we\nargue that question answering should be considered a format which is sometimes\nuseful for studying particular phenomena, not a phenomenon or task in itself.\nWe discuss when a task is correctly described as question answering, and when a\ntask is usefully posed as question answering, instead of using some other\nformat.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11297v1",
            "date": "2019-09-25 05:43:28+00:00",
            "title": "Learning to Detect Opinion Snippet for Aspect-Based Sentiment Analysis",
            "authors": [
                "Mengting Hu",
                "Shiwan Zhao",
                "Honglei Guo",
                "Renhong Cheng",
                "Zhong Su"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) is to predict the sentiment polarity\ntowards a particular aspect in a sentence. Recently, this task has been widely\naddressed by the neural attention mechanism, which computes attention weights\nto softly select words for generating aspect-specific sentence representations.\nThe attention is expected to concentrate on opinion words for accurate\nsentiment prediction. However, attention is prone to be distracted by noisy or\nmisleading words, or opinion words from other aspects. In this paper, we\npropose an alternative hard-selection approach, which determines the start and\nend positions of the opinion snippet, and selects the words between these two\npositions for sentiment prediction. Specifically, we learn deep associations\nbetween the sentence and aspect, and the long-term dependencies within the\nsentence by leveraging the pre-trained BERT model. We further detect the\nopinion snippet by self-critical reinforcement learning. Especially,\nexperimental results demonstrate the effectiveness of our method and prove that\nour hard-selection approach outperforms soft-selection approaches when handling\nmulti-aspect sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11359v1",
            "date": "2019-09-25 09:20:39+00:00",
            "title": "Tackling Long-Tailed Relations and Uncommon Entities in Knowledge Graph Completion",
            "authors": [
                "Zihao Wang",
                "Kwun Ping Lai",
                "Piji Li",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "For large-scale knowledge graphs (KGs), recent research has been focusing on\nthe large proportion of infrequent relations which have been ignored by\nprevious studies. For example few-shot learning paradigm for relations has been\ninvestigated. In this work, we further advocate that handling uncommon entities\nis inevitable when dealing with infrequent relations. Therefore, we propose a\nmeta-learning framework that aims at handling infrequent relations with\nfew-shot learning and uncommon entities by using textual descriptions. We\ndesign a novel model to better extract key information from textual\ndescriptions. Besides, we also develop a novel generative model in our\nframework to enhance the performance by generating extra triplets during the\ntraining stage. Experiments are conducted on two datasets from real-world KGs,\nand the results show that our framework outperforms previous methods when\ndealing with infrequent relations and their accompanying uncommon entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11467v1",
            "date": "2019-09-25 13:10:15+00:00",
            "title": "Developing a Fine-Grained Corpus for a Less-resourced Language: the case of Kurdish",
            "authors": [
                "Roshna Omer Abdulrahman",
                "Hossein Hassani",
                "Sina Ahmadi"
            ],
            "abstract": "Kurdish is a less-resourced language consisting of different dialects written\nin various scripts. Approximately 30 million people in different countries\nspeak the language. The lack of corpora is one of the main obstacles in Kurdish\nlanguage processing. In this paper, we present KTC-the Kurdish Textbooks\nCorpus, which is composed of 31 K-12 textbooks in Sorani dialect. The corpus is\nnormalized and categorized into 12 educational subjects containing 693,800\ntokens (110,297 types). Our resource is publicly available for non-commercial\nuse under the CC BY-NC-SA 4.0 license.",
            "journal ref": "In Proceedings of the 2019 Workshop on Widening NLP (pp. 106-109),\n  August 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11493v1",
            "date": "2019-09-25 13:46:29+00:00",
            "title": "Semi-supervised Text Style Transfer: Cross Projection in Latent Space",
            "authors": [
                "Mingyue Shang",
                "Piji Li",
                "Zhenxin Fu",
                "Lidong Bing",
                "Dongyan Zhao",
                "Shuming Shi",
                "Rui Yan"
            ],
            "abstract": "Text style transfer task requires the model to transfer a sentence of one\nstyle to another style while retaining its original content meaning, which is a\nchallenging problem that has long suffered from the shortage of parallel data.\nIn this paper, we first propose a semi-supervised text style transfer model\nthat combines the small-scale parallel data with the large-scale nonparallel\ndata. With these two types of training data, we introduce a projection function\nbetween the latent space of different styles and design two constraints to\ntrain it. We also introduce two other simple but effective semi-supervised\nmethods to compare with. To evaluate the performance of the proposed methods,\nwe build and release a novel style transfer dataset that alters sentences\nbetween the style of ancient Chinese poem and the modern Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11535v2",
            "date": "2019-09-25 14:53:57+00:00",
            "title": "Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation",
            "authors": [
                "Xiao Huang",
                "Li Dong",
                "Elizabeth Boschee",
                "Nanyun Peng"
            ],
            "abstract": "Named entity recognition (NER) identifies typed entity mentions in raw text.\nWhile the task is well-established, there is no universally used tagset: often,\ndatasets are annotated for use in downstream applications and accordingly only\ncover a small set of entity types relevant to a particular task. For instance,\nin the biomedical domain, one corpus might annotate genes, another chemicals,\nand another diseases---despite the texts in each corpus containing references\nto all three types of entities. In this paper, we propose a deep structured\nmodel to integrate these \"partially annotated\" datasets to jointly identify all\nentity types appearing in the training corpora. By leveraging multiple\ndatasets, the model can learn robust input representations; by building a joint\nstructured model, it avoids potential conflicts caused by combining several\nmodels' predictions at test time. Experiments show that the proposed model\nsignificantly outperforms strong multi-task learning baselines when training on\nmultiple, partially annotated datasets and testing on datasets that contain\ntags from more than one of the training corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11687v2",
            "date": "2019-09-25 18:07:35+00:00",
            "title": "Extremely Small BERT Models from Mixed-Vocabulary Training",
            "authors": [
                "Sanqiang Zhao",
                "Raghav Gupta",
                "Yang Song",
                "Denny Zhou"
            ],
            "abstract": "Pretrained language models like BERT have achieved good results on NLP tasks,\nbut are impractical on resource-limited devices due to memory footprint. A\nlarge fraction of this footprint comes from the input embeddings with large\ninput vocabulary and embedding dimensions. Existing knowledge distillation\nmethods used for model compression cannot be directly applied to train student\nmodels with reduced vocabulary sizes. To this end, we propose a distillation\nmethod to align the teacher and student embeddings via mixed-vocabulary\ntraining. Our method compresses BERT-LARGE to a task-agnostic model with\nsmaller vocabulary and hidden dimensions, which is an order of magnitude\nsmaller than other distilled BERT models and offers a better size-accuracy\ntrade-off on language understanding benchmarks as well as a practical dialogue\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11833v1",
            "date": "2019-09-26 00:51:58+00:00",
            "title": "SIM: A Slot-Independent Neural Model for Dialogue State Tracking",
            "authors": [
                "Chenguang Zhu",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "Dialogue state tracking is an important component in task-oriented dialogue\nsystems to identify users' goals and requests as a dialogue proceeds. However,\nas most previous models are dependent on dialogue slots, the model complexity\nsoars when the number of slots increases. In this paper, we put forward a\nslot-independent neural model (SIM) to track dialogue states while keeping the\nmodel complexity invariant to the number of dialogue slots. The model utilizes\nattention mechanisms between user utterance and system actions. SIM achieves\nstate-of-the-art results on WoZ and DSTC2 tasks, with only 20% of the model\nsize of previous models.",
            "journal ref": "SIGDial 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11861v3",
            "date": "2019-09-26 03:06:47+00:00",
            "title": "Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs",
            "authors": [
                "Yuxian Meng",
                "Xiangyuan Ren",
                "Zijun Sun",
                "Xiaoya Li",
                "Arianna Yuan",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "In this paper, we investigate the problem of training neural machine\ntranslation (NMT) systems with a dataset of more than 40 billion bilingual\nsentence pairs, which is larger than the largest dataset to date by orders of\nmagnitude. Unprecedented challenges emerge in this situation compared to\nprevious NMT work, including severe noise in the data and prohibitively long\ntraining time. We propose practical solutions to handle these issues and\ndemonstrate that large-scale pretraining significantly improves NMT\nperformance. We are able to push the BLEU score of WMT17 Chinese-English\ndataset to 32.3, with a significant performance boost of +3.2 over existing\nstate-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11879v5",
            "date": "2019-09-26 04:17:48+00:00",
            "title": "Aspect and Opinion Term Extraction for Hotel Reviews using Transfer Learning and Auxiliary Labels",
            "authors": [
                "Yosef Ardhito Winatmoko",
                "Ali Akbar Septiandri",
                "Arie Pratama Sutiono"
            ],
            "abstract": "Aspect and opinion term extraction is a critical step in Aspect-Based\nSentiment Analysis (ABSA). Our study focuses on evaluating transfer learning\nusing pre-trained BERT (Devlin et al., 2018) to classify tokens from hotel\nreviews in bahasa Indonesia. The primary challenge is the language informality\nof the review texts. By utilizing transfer learning from a multilingual model,\nwe achieved up to 2% difference on token level F1-score compared to the\nstate-of-the-art Bi-LSTM model with fewer training epochs (3 vs. 200 epochs).\nThe fine-tuned model clearly outperforms the Bi-LSTM model on the entity level.\nFurthermore, we propose a method to include CRF with auxiliary labels as an\noutput layer for the BERT-based models. The CRF addition further improves the\nF1-score for both token and entity level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11898v1",
            "date": "2019-09-26 05:25:10+00:00",
            "title": "Fine-tune Bert for DocRED with Two-step Process",
            "authors": [
                "Hong Wang",
                "Christfried Focke",
                "Rob Sylvester",
                "Nilesh Mishra",
                "William Wang"
            ],
            "abstract": "Modelling relations between multiple entities has attracted increasing\nattention recently, and a new dataset called DocRED has been collected in order\nto accelerate the research on the document-level relation extraction. Current\nbaselines for this task uses BiLSTM to encode the whole document and are\ntrained from scratch. We argue that such simple baselines are not strong enough\nto model to complex interaction between entities. In this paper, we further\napply a pre-trained language model (BERT) to provide a stronger baseline for\nthis task. We also find that solving this task in phases can further improve\nthe performance. The first step is to predict whether or not two entities have\na relation, the second step is to predict the specific relation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12016v1",
            "date": "2019-09-26 10:37:39+00:00",
            "title": "Selecting Artificially-Generated Sentences for Fine-Tuning Neural Machine Translation",
            "authors": [
                "Alberto Poncelas",
                "Andy Way"
            ],
            "abstract": "Neural Machine Translation (NMT) models tend to achieve best performance when\nlarger sets of parallel sentences are provided for training. For this reason,\naugmenting the training set with artificially-generated sentence pairs can\nboost performance.\n  Nonetheless, the performance can also be improved with a small number of\nsentences if they are in the same domain as the test set. Accordingly, we want\nto explore the use of artificially-generated sentences along with\ndata-selection algorithms to improve German-to-English NMT models trained\nsolely with authentic data.\n  In this work, we show how artificially-generated sentences can be more\nbeneficial than authentic pairs, and demonstrate their advantages when used in\ncombination with data-selection algorithms.",
            "journal ref": "Proceedings of the 12th International Conference on Natural\n  Language Generation (INLG 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12079v1",
            "date": "2019-09-26 13:20:10+00:00",
            "title": "Improving Fine-grained Entity Typing with Entity Linking",
            "authors": [
                "Hongliang Dai",
                "Donghong Du",
                "Xin Li",
                "Yangqiu Song"
            ],
            "abstract": "Fine-grained entity typing is a challenging problem since it usually involves\na relatively large tag set and may require to understand the context of the\nentity mention. In this paper, we use entity linking to help with the\nfine-grained entity type classification process. We propose a deep neural model\nthat makes predictions based on both the context and the information obtained\nfrom entity linking results. Experimental results on two commonly used datasets\ndemonstrates the effectiveness of our approach. On both datasets, it achieves\nmore than 5\\% absolute strict accuracy improvement over the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12131v1",
            "date": "2019-09-26 14:13:39+00:00",
            "title": "MinWikiSplit: A Sentence Splitting Corpus with Minimal Propositions",
            "authors": [
                "Christina Niklaus",
                "Andre Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We compiled a new sentence splitting corpus that is composed of 203K pairs of\naligned complex source and simplified target sentences. Contrary to previously\nproposed text simplification corpora, which contain only a small number of\nsplit examples, we present a dataset where each input sentence is broken down\ninto a set of minimal propositions, i.e. a sequence of sound, self-contained\nutterances with each of them presenting a minimal semantic unit that cannot be\nfurther decomposed into meaningful propositions. This corpus is useful for\ndeveloping sentence splitting approaches that learn how to transform sentences\nwith a complex linguistic structure into a fine-grained representation of short\nsentences that present a simple and more regular structure which is easier to\nprocess for downstream applications and thus facilitates and improves their\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12136v1",
            "date": "2019-09-26 14:18:09+00:00",
            "title": "Semantic Change and Emerging Tropes In a Large Corpus of New High German Poetry",
            "authors": [
                "Thomas Haider",
                "Steffen Eger"
            ],
            "abstract": "Due to its semantic succinctness and novelty of expression, poetry is a great\ntest bed for semantic change analysis. However, so far there is a scarcity of\nlarge diachronic corpora. Here, we provide a large corpus of German poetry\nwhich consists of about 75k poems with more than 11 million tokens, with poems\nranging from the 16th to early 20th century. We then track semantic change in\nthis corpus by investigating the rise of tropes (`love is magic') over time and\ndetecting change points of meaning, which we find to occur particularly within\nthe German Romantic period. Additionally, through self-similarity, we\nreconstruct literary periods and find evidence that the law of linear semantic\nchange also applies to poetry.",
            "journal ref": "Proceedings of the 1st International Workshop on Computational\n  Approaches to Historical Language Change (pp. 216-222). At ACL 2019,\n  Florence. https://www.aclweb.org/anthology/W19-4727",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12140v1",
            "date": "2019-09-26 14:21:33+00:00",
            "title": "DisSim: A Discourse-Aware Syntactic Text Simplification Frameworkfor English and German",
            "authors": [
                "Christina Niklaus",
                "Matthias Cetto",
                "Andre Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "We introduce DisSim, a discourse-aware sentence splitting framework for\nEnglish and German whose goal is to transform syntactically complex sentences\ninto an intermediate representation that presents a simple and more regular\nstructure which is easier to process for downstream semantic applications. For\nthis purpose, we turn input sentences into a two-layered semantic hierarchy in\nthe form of core facts and accompanying contexts, while identifying the\nrhetorical relations that hold between them. In that way, we preserve the\ncoherence structure of the input and, hence, its interpretability for\ndownstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12163v1",
            "date": "2019-09-26 14:46:58+00:00",
            "title": "DARTS: Dialectal Arabic Transcription System",
            "authors": [
                "Sameer Khurana",
                "Ahmed Ali",
                "James Glass"
            ],
            "abstract": "We present the speech to text transcription system, called DARTS, for low\nresource Egyptian Arabic dialect. We analyze the following; transfer learning\nfrom high resource broadcast domain to low-resource dialectal domain and\nsemi-supervised learning where we use in-domain unlabeled audio data collected\nfrom YouTube. Key features of our system are: A deep neural network acoustic\nmodel that consists of a front end Convolutional Neural Network (CNN) followed\nby several layers of Time Delayed Neural Network (TDNN) and Long-Short Term\nMemory Recurrent Neural Network (LSTM); sequence discriminative training of the\nacoustic model; n-gram and recurrent neural network language model for decoding\nand N-best list rescoring. We show that a simple transfer learning method can\nachieve good results. The results are further improved by using unlabeled data\nfrom YouTube in a semi-supervised setup. Various systems are combined to give\nthe final system that achieves the lowest word error on on the community\nstandard Egyptian-Arabic speech dataset (MGB-3).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12230v2",
            "date": "2019-09-22 20:19:29+00:00",
            "title": "KnowBias: Detecting Political Polarity in Long Text Content",
            "authors": [
                "Aditya Saligrama"
            ],
            "abstract": "We introduce a classification scheme for detecting political bias in long\ntext content such as newspaper opinion articles. Obtaining long text data and\nannotations at sufficient scale for training is difficult, but it is relatively\neasy to extract political polarity from tweets through their authorship. We\ntrain on tweets and perform inference on articles. Universal sentence encoders\nand other existing methods that aim to address this domain-adaptation scenario\ndeliver inaccurate and inconsistent predictions on articles, which we show is\ndue to a difference in opinion concentration between tweets and articles. We\npropose a two-step classification scheme that uses a neutral detector trained\non tweets to remove neutral sentences from articles in order to align opinion\nconcentration and therefore improve accuracy on that domain. Our implementation\nis available for public use at https://knowbias.ml.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12233v1",
            "date": "2019-09-13 13:39:59+00:00",
            "title": "Deep Ensemble Learning for News Stance Detection",
            "authors": [
                "Wenjun Liao",
                "Chenghua Lin"
            ],
            "abstract": "Stance detection in fake news is an important component in news veracity\nassessment because this process helps fact-checking by understanding stance to\na central claim from different information sources. The Fake News Challenge\nStage 1 (FNC-1) held in 2017 was setup for this purpose, which involves\nestimating the stance of a news article body relative to a given headline. This\nthesis starts from the error analysis for the three top-performing systems in\nFNC-1. Based on the analysis, a simple but tough-to-beat Multilayer Perceptron\nsystem is chosen as the baseline. Afterwards, three approaches are explored to\nimprove baseline.The first approach explores the possibility of improving the\nprediction accuracy by adding extra keywords features when training a model,\nwhere keywords are converted to an indicator vector and then concatenated to\nthe baseline features. A list of keywords is manually selected based on the\nerror analysis, which may best reflect some characteristics of fake news titles\nand bodies. To make this selection process automatically, three algorithms are\ncreated based on Mutual Information (MI) theory: keywords generator based on MI\nstance class, MI customised class, and Pointwise MI algorithm. The second\napproach is based on word embedding, where word2vec model is introduced and two\ndocument similarities calculation algorithms are implemented: wor2vec cosine\nsimilarity and WMD distance. The third approach is ensemble learning. Different\nmodels are configured together with two continuous outputs combining\nalgorithms. The 10-fold cross validation reveals that the ensemble of three\nneural network models trained from simple bag-of-words features gives the best\nperformance. It is therefore selected to compete in FNC-1. After\nhyperparameters fine tuning, the selected deep ensemble model beats the FNC-1\nwinner team by a remarkable 34.25 marks under FNC-1's evaluation metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12375v1",
            "date": "2019-09-26 20:26:51+00:00",
            "title": "On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages",
            "authors": [
                "Yi Zhu",
                "Benjamin Heinzerling",
                "Ivan Vuli\u0107",
                "Michael Strube",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "Recent work has validated the importance of subword information for word\nrepresentation learning. Since subwords increase parameter sharing ability in\nneural models, their value should be even more pronounced in low-data regimes.\nIn this work, we therefore provide a comprehensive analysis focused on the\nusefulness of subwords for word representation learning in truly low-resource\nscenarios and for three representative morphological tasks: fine-grained entity\ntyping, morphological tagging, and named entity recognition. We conduct a\nsystematic study that spans several dimensions of comparison: 1) type of data\nscarcity which can stem from the lack of task-specific training data, or even\nfrom the lack of unannotated data required to train word embeddings, or both;\n2) language type by working with a sample of 16 typologically diverse languages\nincluding some truly low-resource ones (e.g. Rusyn, Buryat, and Zulu); 3) the\nchoice of the subword-informed word representation method. Our main results\nshow that subword-informed models are universally useful across all language\ntypes, with large gains over subword-agnostic embeddings. They also suggest\nthat the effective use of subwords largely depends on the language (type) and\nthe task at hand, as well as on the amount of available data for training the\nembeddings and task-based models, where having sufficient in-task data is a\nmore critical requirement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12406v1",
            "date": "2019-09-26 21:32:50+00:00",
            "title": "Monotonic Multihead Attention",
            "authors": [
                "Xutai Ma",
                "Juan Pino",
                "James Cross",
                "Liezl Puzon",
                "Jiatao Gu"
            ],
            "abstract": "Simultaneous machine translation models start generating a target sequence\nbefore they have encoded or read the source sequence. Recent approaches for\nthis task either apply a fixed policy on a state-of-the art Transformer model,\nor a learnable monotonic attention on a weaker recurrent neural network-based\nstructure. In this paper, we propose a new attention mechanism, Monotonic\nMultihead Attention (MMA), which extends the monotonic attention mechanism to\nmultihead attention. We also introduce two novel and interpretable approaches\nfor latency control that are specifically designed for multiple attentions\nheads. We apply MMA to the simultaneous machine translation task and\ndemonstrate better latency-quality tradeoffs compared to MILk, the previous\nstate-of-the-art approach. We also analyze how the latency controls affect the\nattention span and we motivate the introduction of our model by analyzing the\neffect of the number of decoder layers and heads on quality and latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12411v1",
            "date": "2019-09-26 21:58:26+00:00",
            "title": "Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture",
            "authors": [
                "Ashok Thillaisundaram",
                "Theodosia Togia"
            ],
            "abstract": "This paper presents our participation in the AGAC Track from the 2019 BioNLP\nOpen Shared Tasks. We provide a solution for Task 3, which aims to extract\n\"gene - function change - disease\" triples, where \"gene\" and \"disease\" are\nmentions of particular genes and diseases respectively and \"function change\" is\none of four pre-defined relationship types. Our system extends BERT (Devlin et\nal., 2018), a state-of-the-art language model, which learns contextual language\nrepresentations from a large unlabelled corpus and whose parameters can be\nfine-tuned to solve specific tasks with minimal additional architecture. We\nencode the pair of mentions and their textual context as two consecutive\nsequences in BERT, separated by a special symbol. We then use a single linear\nlayer to classify their relationship into five classes (four pre-defined, as\nwell as 'no relation'). Despite considerable class imbalance, our system\nsignificantly outperforms a random baseline while relying on an extremely\nsimple setup with no specially engineered features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12440v1",
            "date": "2019-09-26 23:56:07+00:00",
            "title": "Improving Pre-Trained Multilingual Models with Vocabulary Expansion",
            "authors": [
                "Hai Wang",
                "Dian Yu",
                "Kai Sun",
                "Janshu Chen",
                "Dong Yu"
            ],
            "abstract": "Recently, pre-trained language models have achieved remarkable success in a\nbroad range of natural language processing tasks. However, in multilingual\nsetting, it is extremely resource-consuming to pre-train a deep language model\nover large-scale corpora for each language. Instead of exhaustively\npre-training monolingual language models independently, an alternative solution\nis to pre-train a powerful multilingual deep language model over large-scale\ncorpora in hundreds of languages. However, the vocabulary size for each\nlanguage in such a model is relatively small, especially for low-resource\nlanguages. This limitation inevitably hinders the performance of these\nmultilingual models on tasks such as sequence labeling, wherein in-depth\ntoken-level or sentence-level understanding is essential.\n  In this paper, inspired by previous methods designed for monolingual\nsettings, we investigate two approaches (i.e., joint mapping and mixture\nmapping) based on a pre-trained multilingual model BERT for addressing the\nout-of-vocabulary (OOV) problem on a variety of tasks, including part-of-speech\ntagging, named entity recognition, machine translation quality estimation, and\nmachine reading comprehension. Experimental results show that using mixture\nmapping is more promising. To the best of our knowledge, this is the first work\nthat attempts to address and discuss the OOV issue in multilingual settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12764v1",
            "date": "2019-09-27 16:10:06+00:00",
            "title": "Improving Semantic Parsing with Neural Generator-Reranker Architecture",
            "authors": [
                "Huseyin A. Inan",
                "Gaurav Singh Tomar",
                "Huapu Pan"
            ],
            "abstract": "Semantic parsing is the problem of deriving machine interpretable meaning\nrepresentations from natural language utterances. Neural models with\nencoder-decoder architectures have recently achieved substantial improvements\nover traditional methods. Although neural semantic parsers appear to have\nrelatively high recall using large beam sizes, there is room for improvement\nwith respect to one-best precision. In this work, we propose a\ngenerator-reranker architecture for semantic parsing. The generator produces a\nlist of potential candidates and the reranker, which consists of a\npre-processing step for the candidates followed by a novel critic network,\nreranks these candidates based on the similarity between each candidate and the\ninput sentence. We show the advantages of this approach along with how it\nimproves the parsing performance through extensive analysis. We experiment our\nmodel on three semantic parsing datasets (GEO, ATIS, and OVERNIGHT). The\noverall architecture achieves the state-of-the-art results in all three\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12928v1",
            "date": "2019-09-26 11:34:04+00:00",
            "title": "Decomposing Textual Information For Style Transfer",
            "authors": [
                "Ivan P. Yamshchikov",
                "Viacheslav Shibaev",
                "Aleksander Nagaev",
                "J\u00fcrgen Jost",
                "Alexey Tikhonov"
            ],
            "abstract": "This paper focuses on latent representations that could effectively decompose\ndifferent aspects of textual information. Using a framework of style transfer\nfor texts, we propose several empirical methods to assess information\ndecomposition quality. We validate these methods with several state-of-the-art\ntextual style transfer methods. Higher quality of information decomposition\ncorresponds to higher performance in terms of bilingual evaluation understudy\n(BLEU) between output and human-written reformulations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13006v2",
            "date": "2019-09-28 02:12:39+00:00",
            "title": "Part of speech tagging for code switched data",
            "authors": [
                "Fahad AlGhamdi",
                "Giovanni Molina",
                "Mona Diab",
                "Thamar Solorio",
                "Abdelati Hawwari",
                "Victor Soto",
                "Julia Hirschberg"
            ],
            "abstract": "We address the problem of Part of Speech tagging (POS) in the context of\nlinguistic code switching (CS). CS is the phenomenon where a speaker switches\nbetween two languages or variants of the same language within or across\nutterances, known as intra-sentential or inter-sentential CS, respectively.\nProcessing CS data is especially challenging in intra-sentential data given\nstate of the art monolingual NLP technology since such technology is geared\ntoward the processing of one language at a time. In this paper we explore\nmultiple strategies of applying state of the art POS taggers to CS data. We\ninvestigate the landscape in two CS language pairs, Spanish-English and Modern\nStandard Arabic-Arabic dialects. We compare the use of two POS taggers vs. a\nunified tagger trained on CS data. Our results show that applying a machine\nlearning framework using two state of the art POS taggers achieves better\nperformance compared to all other approaches that we investigate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13008v1",
            "date": "2019-09-28 02:37:23+00:00",
            "title": "WASA: A Web Application for Sequence Annotation",
            "authors": [
                "Fahad AlGhamdi",
                "Mona Diab"
            ],
            "abstract": "Data annotation is an important and necessary task for all NLP applications.\nDesigning and implementing a web-based application that enables many annotators\nto annotate and enter their input into one central database is not a trivial\ntask. These kinds of web-based applications require a consistent and robust\nbackup for the underlying database and support to enhance the efficiency and\nspeed of the annotation. Also, they need to ensure that the annotations are\nstored with a minimal amount of redundancy in order to take advantage of the\navailable resources(e.g, storage space). In this paper, we introduce WASA, a\nweb-based annotation system for managing large-scale multilingual Code\nSwitching (CS) data annotation. Although WASA has the ability to perform the\nannotation for any token sequence with arbitrary tag sets, we will focus on how\nWASA is used for CS annotation. The system supports concurrent annotation,\nhandles multiple encodings, allows for several levels of management control,\nand enables quality control measures while seamlessly reporting annotation\nstatistics from various perspectives and at different levels of granularity.\nMoreover, the system is integrated with a robust language specific date\nprepossessing tool to enhance the speed and efficiency of the annotation. We\ndescribe the annotation and the administration interfaces as well as the\nbackend engine.",
            "journal ref": "Proceedings of the Eleventh International Conference on Language\n  Resources and Evaluation, 2018 , LREC",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13009v1",
            "date": "2019-09-28 02:42:18+00:00",
            "title": "Creating a Large Multi-Layered Representational Repository of Linguistic Code Switched Arabic Data",
            "authors": [
                "Mona Diab",
                "Mahmoud Ghoneim",
                "Abdelati Hawwari",
                "Fahad AlGhamdi",
                "Nada AlMarwani",
                "Mohamed Al-Badrashiny"
            ],
            "abstract": "We present our effort to create a large Multi-Layered representational\nrepository of Linguistic Code-Switched Arabic data. The process involves\ndeveloping clear annotation standards and Guidelines, streamlining the\nannotation process, and implementing quality control measures. We used two main\nprotocols for annotation: in-lab gold annotations and crowd sourcing\nannotations. We developed a web-based annotation tool to facilitate the\nmanagement of the annotation process. The current version of the repository\ncontains a total of 886,252 tokens that are tagged into one of sixteen\ncode-switching tags. The data exhibits code switching between Modern Standard\nArabic and Egyptian Dialectal Arabic representing three data genres: Tweets,\ncommentaries, and discussion fora. The overall Inter-Annotator Agreement is\n93.1%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13016v1",
            "date": "2019-09-28 03:26:31+00:00",
            "title": "Overview for the Second Shared Task on Language Identification in Code-Switched Data",
            "authors": [
                "Giovanni Molina",
                "Fahad AlGhamdi",
                "Mahmoud Ghoneim",
                "Abdelati Hawwari",
                "Nicolas Rey-Villamizar",
                "Mona Diab",
                "Thamar Solorio"
            ],
            "abstract": "We present an overview of the second shared task on language identification\nin code-switched data. For the shared task, we had code-switched data from two\ndifferent language pairs: Modern Standard Arabic-Dialectal Arabic (MSA-DA) and\nSpanish-English (SPA-ENG). We had a total of nine participating teams, with all\nteams submitting a system for SPA-ENG and four submitting for MSA-DA. Through\nevaluation, we found that once again language identification is more difficult\nfor the language pair that is more closely related. We also found that this\nyear's systems performed better overall than the systems from the previous\nshared task indicating overall progress in the state of the art for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13078v1",
            "date": "2019-09-28 12:26:07+00:00",
            "title": "OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction",
            "authors": [
                "Xu Han",
                "Tianyu Gao",
                "Yuan Yao",
                "Demin Ye",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "OpenNRE is an open-source and extensible toolkit that provides a unified\nframework to implement neural models for relation extraction (RE).\nSpecifically, by implementing typical RE methods, OpenNRE not only allows\ndevelopers to train custom models to extract structured relational facts from\nthe plain text but also supports quick model validation for researchers.\nBesides, OpenNRE provides various functional RE modules based on both\nTensorFlow and PyTorch to maintain sufficient modularity and extensibility,\nmaking it becomes easy to incorporate new models into the framework. Besides\nthe toolkit, we also release an online system to meet real-time extraction\nwithout any training and deploying. Meanwhile, the online system can extract\nfacts in various scenarios as well as aligning the extracted facts to Wikidata,\nwhich may benefit various downstream knowledge-driven applications (e.g.,\ninformation retrieval and question answering). More details of the toolkit and\nonline system can be obtained from http://github.com/thunlp/OpenNRE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13128v1",
            "date": "2019-09-28 18:28:56+00:00",
            "title": "Integrated Triaging for Fast Reading Comprehension",
            "authors": [
                "Felix Wu",
                "Boyi Li",
                "Lequn Wang",
                "Ni Lao",
                "John Blitzer",
                "Kilian Q. Weinberger"
            ],
            "abstract": "Although according to several benchmarks automatic machine reading\ncomprehension (MRC) systems have recently reached super-human performance, less\nattention has been paid to their computational efficiency. However, efficiency\nis of crucial importance for training and deployment in real world\napplications. This paper introduces Integrated Triaging, a framework that\nprunes almost all context in early layers of a network, leaving the remaining\n(deep) layers to scan only a tiny fraction of the full corpus. This pruning\ndrastically increases the efficiency of MRC models and further prevents the\nlater layers from overfitting to prevalent short paragraphs in the training\nset. Our framework is extremely flexible and naturally applicable to a wide\nvariety of models. Our experiment on doc-SQuAD and TriviaQA tasks demonstrates\nits effectiveness in consistently improving both speed and quality of several\ndiverse MRC models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13151v2",
            "date": "2019-09-28 21:03:09+00:00",
            "title": "The Source-Target Domain Mismatch Problem in Machine Translation",
            "authors": [
                "Jiajun Shen",
                "Peng-Jen Chen",
                "Matt Le",
                "Junxian He",
                "Jiatao Gu",
                "Myle Ott",
                "Michael Auli",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "While we live in an increasingly interconnected world, different places still\nexhibit strikingly different cultures and many events we experience in our\nevery day life pertain only to the specific place we live in. As a result,\npeople often talk about different things in different parts of the world. In\nthis work we study the effect of local context in machine translation and\npostulate that particularly in low resource settings this causes the domains of\nthe source and target language to greatly mismatch, as the two languages are\noften spoken in further apart regions of the world with more distinctive\ncultural traits and unrelated local events. We first formalize the concept of\nsource-target domain mismatch, propose a metric to quantify it, and provide\nempirical evidence corroborating our intuition that organic text produced by\npeople speaking very different languages exhibits the most dramatic\ndifferences. We conclude with an empirical study of how source-target domain\nmismatch affects training of machine translation systems for low resource\nlanguage pairs. In particular, we find that it severely affects\nback-translation, but the degradation can be alleviated by combining\nback-translation with self-training and by increasing the relative amount of\ntarget side monolingual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13180v2",
            "date": "2019-09-29 01:36:19+00:00",
            "title": "Towards Zero-resource Cross-lingual Entity Linking",
            "authors": [
                "Shuyan Zhou",
                "Shruti Rijhwani",
                "Graham Neubig"
            ],
            "abstract": "Cross-lingual entity linking (XEL) grounds named entities in a source\nlanguage to an English Knowledge Base (KB), such as Wikipedia. XEL is\nchallenging for most languages because of limited availability of requisite\nresources. However, much previous work on XEL has been on simulated settings\nthat actually use significant resources (e.g. source language Wikipedia,\nbilingual entity maps, multilingual embeddings) that are unavailable in truly\nlow-resource languages. In this work, we first examine the effect of these\nresource assumptions and quantify how much the availability of these resource\naffects overall quality of existing XEL systems. Next, we propose three\nimprovements to both entity candidate generation and disambiguation that make\nbetter use of the limited data we do have in resource-scarce scenarios. With\nexperiments on four extremely low-resource languages, we show that our model\nresults in gains of 6-23% in end-to-end linking accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13293v2",
            "date": "2019-09-29 14:41:47+00:00",
            "title": "A Pilot Study for Chinese SQL Semantic Parsing",
            "authors": [
                "Qingkai Min",
                "Yuefeng Shi",
                "Yue Zhang"
            ],
            "abstract": "The task of semantic parsing is highly useful for dialogue and question\nanswering systems. Many datasets have been proposed to map natural language\ntext into SQL, among which the recent Spider dataset provides cross-domain\nsamples with multiple tables and complex queries. We build a Spider dataset for\nChinese, which is currently a low-resource language in this task area.\nInteresting research questions arise from the uniqueness of the language, which\nrequires word segmentation, and also from the fact that SQL keywords and\ncolumns of DB tables are typically written in English. We compare character-\nand word-based encoders for a semantic parser, and different embedding schemes.\nResults show that word-based semantic parser is subject to segmentation errors\nand cross-lingual word embeddings are useful for text-to-SQL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13302v3",
            "date": "2019-09-29 15:35:08+00:00",
            "title": "Controllable Data Synthesis Method for Grammatical Error Correction",
            "authors": [
                "Chencheng Wang",
                "Liner Yang",
                "Yun Chen",
                "Yongping Du",
                "Erhong Yang"
            ],
            "abstract": "Due to the lack of parallel data in current Grammatical Error Correction\n(GEC) task, models based on Sequence to Sequence framework cannot be adequately\ntrained to obtain higher performance. We propose two data synthesis methods\nwhich can control the error rate and the ratio of error types on synthetic\ndata. The first approach is to corrupt each word in the monolingual corpus with\na fixed probability, including replacement, insertion and deletion. Another\napproach is to train error generation models and further filtering the decoding\nresults of the models. The experiments on different synthetic data show that\nthe error rate is 40% and the ratio of error types is the same can improve the\nmodel performance better. Finally, we synthesize about 100 million data and\nachieve comparable performance as the state of the art, which uses twice as\nmuch data as we use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13362v1",
            "date": "2019-09-29 20:32:27+00:00",
            "title": "Language-Agnostic Syllabification with Neural Sequence Labeling",
            "authors": [
                "Jacob Krantz",
                "Maxwell Dulin",
                "Paul De Palma"
            ],
            "abstract": "The identification of syllables within phonetic sequences is known as\nsyllabification. This task is thought to play an important role in natural\nlanguage understanding, speech production, and the development of speech\nrecognition systems. The concept of the syllable is cross-linguistic, though\nformal definitions are rarely agreed upon, even within a language. In response,\ndata-driven syllabification methods have been developed to learn from\nsyllabified examples. These methods often employ classical machine learning\nsequence labeling models. In recent years, recurrence-based neural networks\nhave been shown to perform increasingly well for sequence labeling tasks such\nas named entity recognition (NER), part of speech (POS) tagging, and chunking.\nWe present a novel approach to the syllabification problem which leverages\nmodern neural network techniques. Our network is constructed with long\nshort-term memory (LSTM) cells, a convolutional component, and a conditional\nrandom field (CRF) output layer. Existing syllabification approaches are rarely\nevaluated across multiple language families. To demonstrate cross-linguistic\ngeneralizability, we show that the network is competitive with state of the art\nsystems in syllabifying English, Dutch, Italian, French, Manipuri, and Basque\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13375v4",
            "date": "2019-09-29 21:49:36+00:00",
            "title": "A Simple and Effective Model for Answering Multi-span Questions",
            "authors": [
                "Elad Segal",
                "Avia Efrat",
                "Mor Shoham",
                "Amir Globerson",
                "Jonathan Berant"
            ],
            "abstract": "Models for reading comprehension (RC) commonly restrict their output space to\nthe set of all single contiguous spans from the input, in order to alleviate\nthe learning problem and avoid the need for a model that generates text\nexplicitly. However, forcing an answer to be a single span can be restrictive,\nand some recent datasets also include multi-span questions, i.e., questions\nwhose answer is a set of non-contiguous spans in the text. Naturally, models\nthat return single spans cannot answer these questions. In this work, we\npropose a simple architecture for answering multi-span questions by casting the\ntask as a sequence tagging problem, namely, predicting for each input token\nwhether it should be part of the output or not. Our model substantially\nimproves performance on span extraction questions from DROP and Quoref by 9.9\nand 5.5 EM points respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13425v1",
            "date": "2019-09-30 02:08:47+00:00",
            "title": "Augmenting Non-Collaborative Dialog Systems with Explicit Semantic and Strategic Dialog History",
            "authors": [
                "Yiheng Zhou",
                "Yulia Tsvetkov",
                "Alan W Black",
                "Zhou Yu"
            ],
            "abstract": "We study non-collaborative dialogs, where two agents have a conflict of\ninterest but must strategically communicate to reach an agreement (e.g.,\nnegotiation). This setting poses new challenges for modeling dialog history\nbecause the dialog's outcome relies not only on the semantic intent, but also\non tactics that convey the intent. We propose to model both semantic and tactic\nhistory using finite state transducers (FSTs). Unlike RNN, FSTs can explicitly\nrepresent dialog history through all the states traversed, facilitating\ninterpretability of dialog structure. We train FSTs on a set of strategies and\ntactics used in negotiation dialogs. The trained FSTs show plausible tactic\nstructure and can be generalized to other non-collaborative domains (e.g.,\npersuasion). We evaluate the FSTs by incorporating them in an automated\nnegotiating system that attempts to sell products and a persuasion system that\npersuades people to donate to a charity. Experiments show that explicitly\nmodeling both semantic and tactic history is an effective way to improve both\ndialog policy planning and generation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13426v1",
            "date": "2019-09-30 02:15:29+00:00",
            "title": "A Dynamic Strategy Coach for Effective Negotiation",
            "authors": [
                "Yiheng Zhou",
                "He He",
                "Alan W Black",
                "Yulia Tsvetkov"
            ],
            "abstract": "Negotiation is a complex activity involving strategic reasoning, persuasion,\nand psychology. An average person is often far from an expert in negotiation.\nOur goal is to assist humans to become better negotiators through a\nmachine-in-the-loop approach that combines machine's advantage at data-driven\ndecision-making and human's language generation ability. We consider a\nbargaining scenario where a seller and a buyer negotiate the price of an item\nfor sale through a text-based dialog. Our negotiation coach monitors messages\nbetween them and recommends tactics in real time to the seller to get a better\ndeal (e.g., \"reject the proposal and propose a price\", \"talk about your\npersonal experience with the product\"). The best strategy and tactics largely\ndepend on the context (e.g., the current price, the buyer's attitude).\nTherefore, we first identify a set of negotiation tactics, then learn to\npredict the best strategy and tactics in a given dialog context from a set of\nhuman-human bargaining dialogs. Evaluation on human-human dialogs shows that\nour coach increases the profits of the seller by almost 60%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13466v1",
            "date": "2019-09-30 05:55:06+00:00",
            "title": "Regressing Word and Sentence Embeddings for Regularization of Neural Machine Translation",
            "authors": [
                "Inigo Jauregi Unanue",
                "Ehsan Zare Borzeshi",
                "Massimo Piccardi"
            ],
            "abstract": "In recent years, neural machine translation (NMT) has become the dominant\napproach in automated translation. However, like many other deep learning\napproaches, NMT suffers from overfitting when the amount of training data is\nlimited. This is a serious issue for low-resource language pairs and many\nspecialized translation domains that are inherently limited in the amount of\navailable supervised data. For this reason, in this paper we propose regressing\nword (ReWE) and sentence (ReSE) embeddings at training time as a way to\nregularize NMT models and improve their generalization. During training, our\nmodels are trained to jointly predict categorical (words in the vocabulary) and\ncontinuous (word and sentence embeddings) outputs. An extensive set of\nexperiments over four language pairs of variable training set size has showed\nthat ReWE and ReSE can outperform strong state-of-the-art baseline models, with\nan improvement that is larger for smaller training sets (e.g., up to +5:15 BLEU\npoints in Basque-English translation). Visualizations of the decoder's output\nspace show that the proposed regularizers improve the clustering of unique\nwords, facilitating correct predictions. In a final experiment on unsupervised\nNMT, we show that ReWE and ReSE are also able to improve the quality of machine\ntranslation when no parallel data are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13494v1",
            "date": "2019-09-30 07:38:25+00:00",
            "title": "A Critique of the Smooth Inverse Frequency Sentence Embeddings",
            "authors": [
                "Aidana Karipbayeva",
                "Alena Sorokina",
                "Zhenisbek Assylbekov"
            ],
            "abstract": "We critically review the smooth inverse frequency sentence embedding method\nof Arora, Liang, and Ma (2017), and show inconsistencies in its setup,\nderivation, and evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13705v1",
            "date": "2019-09-30 13:55:10+00:00",
            "title": "A Closer Look at Data Bias in Neural Extractive Summarization Models",
            "authors": [
                "Ming Zhong",
                "Danqing Wang",
                "Pengfei Liu",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we take stock of the current state of summarization datasets\nand explore how different factors of datasets influence the generalization\nbehaviour of neural extractive summarization models. Specifically, we first\npropose several properties of datasets, which matter for the generalization of\nsummarization models. Then we build the connection between priors residing in\ndatasets and model designs, analyzing how different properties of datasets\ninfluence the choices of model structure design and training methods. Finally,\nby taking a typical dataset as an example, we rethink the process of the model\ndesign based on the experience of the above analysis. We demonstrate that when\nwe have a deep understanding of the characteristics of datasets, a simple\napproach can bring significant improvements to the existing state-of-the-art\nmodel.A",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13838v2",
            "date": "2019-09-30 17:02:57+00:00",
            "title": "Automatic Fact-guided Sentence Modification",
            "authors": [
                "Darsh J Shah",
                "Tal Schuster",
                "Regina Barzilay"
            ],
            "abstract": "Online encyclopediae like Wikipedia contain large amounts of text that need\nfrequent corrections and updates. The new information may contradict existing\ncontent in encyclopediae. In this paper, we focus on rewriting such dynamically\nchanging articles. This is a challenging constrained generation task, as the\noutput must be consistent with the new information and fit into the rest of the\nexisting document. To this end, we propose a two-step solution: (1) We identify\nand remove the contradicting components in a target text for a given claim,\nusing a neutralizing stance model; (2) We expand the remaining text to be\nconsistent with the given claim, using a novel two-encoder sequence-to-sequence\nmodel with copy attention. Applied to a Wikipedia fact update dataset, our\nmethod successfully generates updated sentences for new claims, achieving the\nhighest SARI score. Furthermore, we demonstrate that generating synthetic data\nthrough such rewritten sentences can successfully augment the FEVER\nfact-checking training dataset, leading to a relative error reduction of 13%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13851v1",
            "date": "2019-09-30 17:15:57+00:00",
            "title": "The Universal Decompositional Semantics Dataset and Decomp Toolkit",
            "authors": [
                "Aaron Steven White",
                "Elias Stengel-Eskin",
                "Siddharth Vashishtha",
                "Venkata Govindarajan",
                "Dee Ann Reisinger",
                "Tim Vieira",
                "Keisuke Sakaguchi",
                "Sheng Zhang",
                "Francis Ferraro",
                "Rachel Rudinger",
                "Kyle Rawlins",
                "Benjamin Van Durme"
            ],
            "abstract": "We present the Universal Decompositional Semantics (UDS) dataset (v1.0),\nwhich is bundled with the Decomp toolkit (v0.1). UDS1.0 unifies five\nhigh-quality, decompositional semantics-aligned annotation sets within a single\nsemantic graph specification---with graph structures defined by the predicative\npatterns produced by the PredPatt tool and real-valued node and edge attributes\nconstructed using sophisticated normalization procedures. The Decomp toolkit\nprovides a suite of Python 3 tools for querying UDS graphs using SPARQL. Both\nUDS1.0 and Decomp0.1 are publicly available at http://decomp.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13872v1",
            "date": "2019-09-30 17:54:50+00:00",
            "title": "Simple and Effective Paraphrastic Similarity from Parallel Translations",
            "authors": [
                "John Wieting",
                "Kevin Gimpel",
                "Graham Neubig",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We present a model and methodology for learning paraphrastic sentence\nembeddings directly from bitext, removing the time-consuming intermediate step\nof creating paraphrase corpora. Further, we show that the resulting model can\nbe applied to cross-lingual tasks where it both outperforms and is orders of\nmagnitude faster than more complex state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00051v2",
            "date": "2019-09-30 18:39:08+00:00",
            "title": "Semantic Graph Parsing with Recurrent Neural Network DAG Grammars",
            "authors": [
                "Federico Fancellu",
                "Sorcha Gilroy",
                "Adam Lopez",
                "Mirella Lapata"
            ],
            "abstract": "Semantic parses are directed acyclic graphs (DAGs), so semantic parsing\nshould be modeled as graph prediction. But predicting graphs presents difficult\ntechnical challenges, so it is simpler and more common to predict the\nlinearized graphs found in semantic parsing datasets using well-understood\nsequence models. The cost of this simplicity is that the predicted strings may\nnot be well-formed graphs. We present recurrent neural network DAG grammars, a\ngraph-aware sequence model that ensures only well-formed graphs while\nsidestepping many difficulties in graph prediction. We test our model on the\nParallel Meaning Bank---a multilingual semantic graphbank. Our approach yields\ncompetitive results in English and establishes the first results for German,\nItalian and Dutch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00065v1",
            "date": "2019-09-30 19:34:45+00:00",
            "title": "Lexical Features Are More Vulnerable, Syntactic Features Have More Predictive Power",
            "authors": [
                "Jekaterina Novikova",
                "Aparna Balagopalan",
                "Ksenia Shkaruta",
                "Frank Rudzicz"
            ],
            "abstract": "Understanding the vulnerability of linguistic features extracted from noisy\ntext is important for both developing better health text classification models\nand for interpreting vulnerabilities of natural language models. In this paper,\nwe investigate how generic language characteristics, such as syntax or the\nlexicon, are impacted by artificial text alterations. The vulnerability of\nfeatures is analysed from two perspectives: (1) the level of feature value\nchange, and (2) the level of change of feature predictive power as a result of\ntext modifications. We show that lexical features are more sensitive to text\nmodifications than syntactic ones. However, we also demonstrate that these\nsmaller changes of syntactic features have a stronger influence on\nclassification performance downstream, compared to the impact of changes to\nlexical features. Results are validated across three datasets representing\ndifferent text-classification tasks, with different levels of lexical and\nsyntactic complexity of both conversational and written language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00139v1",
            "date": "2019-09-30 22:30:56+00:00",
            "title": "Interrogating the Explanatory Power of Attention in Neural Machine Translation",
            "authors": [
                "Pooya Moradi",
                "Nishant Kambhatla",
                "Anoop Sarkar"
            ],
            "abstract": "Attention models have become a crucial component in neural machine\ntranslation (NMT). They are often implicitly or explicitly used to justify the\nmodel's decision in generating a specific token but it has not yet been\nrigorously established to what extent attention is a reliable source of\ninformation in NMT. To evaluate the explanatory power of attention for NMT, we\nexamine the possibility of yielding the same prediction but with counterfactual\nattention models that modify crucial aspects of the trained attention model.\nUsing these counterfactual attention mechanisms we assess the extent to which\nthey still preserve the generation of function and content words in the\ntranslation process. Compared to a state of the art attention model, our\ncounterfactual attention models produce 68% of function words and 21% of\ncontent words in our German-English dataset. Our experiments demonstrate that\nattention models by themselves cannot reliably explain the decisions made by a\nNMT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00194v2",
            "date": "2019-10-01 04:11:33+00:00",
            "title": "Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations",
            "authors": [
                "Christian Hadiwinoto",
                "Hwee Tou Ng",
                "Wee Chung Gan"
            ],
            "abstract": "Contextualized word representations are able to give different\nrepresentations for the same word in different contexts, and they have been\nshown to be effective in downstream natural language processing tasks, such as\nquestion answering, named entity recognition, and sentiment analysis. However,\nevaluation on word sense disambiguation (WSD) in prior work shows that using\ncontextualized word representations does not outperform the state-of-the-art\napproach that makes use of non-contextualized word embeddings. In this paper,\nwe explore different strategies of integrating pre-trained contextualized word\nrepresentations and our best strategy achieves accuracies exceeding the best\nprior published accuracies by significant margins on multiple benchmark WSD\ndatasets. We make the source code available at\nhttps://github.com/nusnlp/contextemb-wsd.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00203v1",
            "date": "2019-10-01 05:22:19+00:00",
            "title": "Analyzing Sentence Fusion in Abstractive Summarization",
            "authors": [
                "Logan Lebanoff",
                "John Muchovej",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Seokhwan Kim",
                "Walter Chang",
                "Fei Liu"
            ],
            "abstract": "While recent work in abstractive summarization has resulted in higher scores\nin automatic metrics, there is little understanding on how these systems\ncombine information taken from multiple document sentences. In this paper, we\nanalyze the outputs of five state-of-the-art abstractive summarizers, focusing\non summary sentences that are formed by sentence fusion. We ask assessors to\njudge the grammaticality, faithfulness, and method of fusion for summary\nsentences. Our analysis reveals that system sentences are mostly grammatical,\nbut often fail to remain faithful to the original article.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00294v1",
            "date": "2019-10-01 10:40:26+00:00",
            "title": "When and Why is Document-level Context Useful in Neural Machine Translation?",
            "authors": [
                "Yunsu Kim",
                "Duc Thanh Tran",
                "Hermann Ney"
            ],
            "abstract": "Document-level context has received lots of attention for compensating neural\nmachine translation (NMT) of isolated sentences. However, recent advances in\ndocument-level NMT focus on sophisticated integration of the context,\nexplaining its improvement with only a few selected examples or targeted test\nsets. We extensively quantify the causes of improvements by a document-level\nmodel in general test sets, clarifying the limit of the usefulness of\ndocument-level context in NMT. We show that most of the improvements are not\ninterpretable as utilizing the context. We also show that a minimal encoding is\nsufficient for the context modeling and very long context is not helpful for\nNMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00353v3",
            "date": "2019-10-01 12:58:44+00:00",
            "title": "Grammatical Error Correction in Low-Resource Scenarios",
            "authors": [
                "Jakub N\u00e1plava",
                "Milan Straka"
            ],
            "abstract": "Grammatical error correction in English is a long studied problem with many\nexisting systems and datasets. However, there has been only a limited research\non error correction of other languages. In this paper, we present a new dataset\nAKCES-GEC on grammatical error correction for Czech. We then make experiments\non Czech, German and Russian and show that when utilizing synthetic parallel\ncorpus, Transformer neural machine translation model can reach new\nstate-of-the-art results on these datasets. AKCES-GEC is published under CC\nBY-NC-SA 4.0 license at https://hdl.handle.net/11234/1-3057 and the source code\nof the GEC model is available at\nhttps://github.com/ufal/low-resource-gec-wnut2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00368v1",
            "date": "2019-10-01 13:24:56+00:00",
            "title": "Application of Low-resource Machine Translation Techniques to Russian-Tatar Language Pair",
            "authors": [
                "Aidar Valeev",
                "Ilshat Gibadullin",
                "Albina Khusainova",
                "Adil Khan"
            ],
            "abstract": "Neural machine translation is the current state-of-the-art in machine\ntranslation. Although it is successful in a resource-rich setting, its\napplicability for low-resource language pairs is still debatable. In this\npaper, we explore the effect of different techniques to improve machine\ntranslation quality when a parallel corpus is as small as 324 000 sentences,\ntaking as an example previously unexplored Russian-Tatar language pair. We\napply such techniques as transfer learning and semi-supervised learning to the\nbase Transformer model, and empirically show that the resulting models improve\nRussian to Tatar and Tatar to Russian translation quality by +2.57 and +3.66\nBLEU, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00373v1",
            "date": "2019-10-01 13:29:53+00:00",
            "title": "A Survey of Methods to Leverage Monolingual Data in Low-resource Neural Machine Translation",
            "authors": [
                "Ilshat Gibadullin",
                "Aidar Valeev",
                "Albina Khusainova",
                "Adil Khan"
            ],
            "abstract": "Neural machine translation has become the state-of-the-art for language pairs\nwith large parallel corpora. However, the quality of machine translation for\nlow-resource languages leaves much to be desired. There are several approaches\nto mitigate this problem, such as transfer learning, semi-supervised and\nunsupervised learning techniques. In this paper, we review the existing\nmethods, where the main idea is to exploit the power of monolingual data,\nwhich, compared to parallel, is usually easier to obtain and significantly\ngreater in amount.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00478v1",
            "date": "2019-10-01 15:24:57+00:00",
            "title": "Machine Translation for Machines: the Sentiment Classification Use Case",
            "authors": [
                "Amirhossein Tebbifakhr",
                "Luisa Bentivogli",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "We propose a neural machine translation (NMT) approach that, instead of\npursuing adequacy and fluency (\"human-oriented\" quality criteria), aims to\ngenerate translations that are best suited as input to a natural language\nprocessing component designed for a specific downstream task (a\n\"machine-oriented\" criterion). Towards this objective, we present a\nreinforcement learning technique based on a new candidate sampling strategy,\nwhich exploits the results obtained on the downstream task as weak feedback.\nExperiments in sentiment classification of Twitter data in German and Italian\nshow that feeding an English classifier with machine-oriented translations\nsignificantly improves its performance. Classification results outperform those\nobtained with translations produced by general-purpose NMT models as well as by\nan approach based on reinforcement learning. Moreover, our results on both\nlanguages approximate the classification accuracy computed on gold standard\nEnglish tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00486v3",
            "date": "2019-10-01 15:36:27+00:00",
            "title": "Dialogue Transformers",
            "authors": [
                "Vladimir Vlasov",
                "Johannes E. M. Mosig",
                "Alan Nichol"
            ],
            "abstract": "We introduce a dialogue policy based on a transformer architecture, where the\nself-attention mechanism operates over the sequence of dialogue turns. Recent\nwork has used hierarchical recurrent neural networks to encode multiple\nutterances in a dialogue context, but we argue that a pure self-attention\nmechanism is more suitable. By default, an RNN assumes that every item in a\nsequence is relevant for producing an encoding of the full sequence, but a\nsingle conversation can consist of multiple overlapping discourse segments as\nspeakers interleave multiple topics. A transformer picks which turns to include\nin its encoding of the current dialogue state, and is naturally suited to\nselectively ignoring or attending to dialogue history. We compare the\nperformance of the Transformer Embedding Dialogue (TED) policy to an LSTM and\nto the REDP, which was specifically designed to overcome this limitation of\nRNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00523v2",
            "date": "2019-10-01 16:25:12+00:00",
            "title": "BillSum: A Corpus for Automatic Summarization of US Legislation",
            "authors": [
                "Anastassia Kornilova",
                "Vlad Eidelman"
            ],
            "abstract": "Automatic summarization methods have been studied on a variety of domains,\nincluding news and scientific articles. Yet, legislation has not previously\nbeen considered for this task, despite US Congress and state governments\nreleasing tens of thousands of bills every year. In this paper, we introduce\nBillSum, the first dataset for summarization of US Congressional and California\nstate bills (https://github.com/FiscalNote/BillSum). We explain the properties\nof the dataset that make it more challenging to process than other domains.\nThen, we benchmark extractive methods that consider neural sentence\nrepresentations and traditional contextual features. Finally, we demonstrate\nthat models built on Congressional bills can be used to summarize California\nbills, thus, showing that methods developed on this dataset can transfer to\nstates without human-written summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00546v1",
            "date": "2019-10-01 17:09:29+00:00",
            "title": "Type-aware Convolutional Neural Networks for Slot Filling",
            "authors": [
                "Heike Adel",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "The slot filling task aims at extracting answers for queries about entities\nfrom text, such as \"Who founded Apple\". In this paper, we focus on the relation\nclassification component of a slot filling system. We propose type-aware\nconvolutional neural networks to benefit from the mutual dependencies between\nentity and relation classification. In particular, we explore different ways of\nintegrating the named entity types of the relation arguments into a neural\nnetwork for relation classification, including a joint training and a\nstructured prediction approach. To the best of our knowledge, this is the first\nstudy on type-aware neural networks for slot filling. The type-aware models\nlead to the best results of our slot filling pipeline. Joint training performs\ncomparable to structured prediction. To understand the impact of the different\ncomponents of the slot filling pipeline, we perform a recall analysis, a manual\nerror analysis and several ablation studies. Such analyses are of particular\nimportance to other slot filling researchers since the official slot filling\nevaluations only assess pipeline outputs. The analyses show that especially\ncoreference resolution and our convolutional neural networks have a large\npositive impact on the final performance of the slot filling pipeline. The\npresented models, the source code of our system as well as our coreference\nresource is publicy available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00637v2",
            "date": "2019-10-01 19:51:57+00:00",
            "title": "Essentia: Mining Domain-Specific Paraphrases with Word-Alignment Graphs",
            "authors": [
                "Danni Ma",
                "Chen Chen",
                "Behzad Golshan",
                "Wang-Chiew Tan"
            ],
            "abstract": "Paraphrases are important linguistic resources for a wide variety of NLP\napplications. Many techniques for automatic paraphrase mining from general\ncorpora have been proposed. While these techniques are successful at\ndiscovering generic paraphrases, they often fail to identify domain-specific\nparaphrases (e.g., {staff, concierge} in the hospitality domain). This is\nbecause current techniques are often based on statistical methods, while\ndomain-specific corpora are too small to fit statistical methods. In this\npaper, we present an unsupervised graph-based technique to mine paraphrases\nfrom a small set of sentences that roughly share the same topic or intent. Our\nsystem, Essentia, relies on word-alignment techniques to create a\nword-alignment graph that merges and organizes tokens from input sentences. The\nresulting graph is then used to generate candidate paraphrases. We demonstrate\nthat our system obtains high-quality paraphrases, as evaluated by crowd\nworkers. We further show that the majority of the identified paraphrases are\ndomain-specific and thus complement existing paraphrase databases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00825v1",
            "date": "2019-10-02 08:22:03+00:00",
            "title": "Abstractive Dialog Summarization with Semantic Scaffolds",
            "authors": [
                "Lin Yuan",
                "Zhou Yu"
            ],
            "abstract": "The demand for abstractive dialog summary is growing in real-world\napplications. For example, customer service center or hospitals would like to\nsummarize customer service interaction and doctor-patient interaction. However,\nfew researchers explored abstractive summarization on dialogs due to the lack\nof suitable datasets. We propose an abstractive dialog summarization dataset\nbased on MultiWOZ. If we directly apply previous state-of-the-art document\nsummarization methods on dialogs, there are two significant drawbacks: the\ninformative entities such as restaurant names are difficult to preserve, and\nthe contents from different dialog domains are sometimes mismatched. To address\nthese two drawbacks, we propose Scaffold Pointer Network (SPNet)to utilize the\nexisting annotation on speaker role, semantic slot and dialog domain. SPNet\nincorporates these semantic scaffolds for dialog summarization. Since ROUGE\ncannot capture the two drawbacks mentioned, we also propose a new evaluation\nmetric that considers critical informative entities in the text. On MultiWOZ,\nour proposed SPNet outperforms state-of-the-art abstractive summarization\nmethods on all the automatic and human evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00856v1",
            "date": "2019-10-02 10:05:44+00:00",
            "title": "BookQA: Stories of Challenges and Opportunities",
            "authors": [
                "Stefanos Angelidis",
                "Lea Frermann",
                "Diego Marcheggiani",
                "Roi Blanco",
                "Llu\u00eds M\u00e0rquez"
            ],
            "abstract": "We present a system for answering questions based on the full text of books\n(BookQA), which first selects book passages given a question at hand, and then\nuses a memory network to reason and predict an answer. To improve\ngeneralization, we pretrain our memory network using artificial questions\ngenerated from book sentences. We experiment with the recently published\nNarrativeQA corpus, on the subset of Who questions, which expect book\ncharacters as answers. We experimentally show that BERT-based retrieval and\npretraining improve over baseline results significantly. At the same time, we\nconfirm that NarrativeQA is a highly challenging data set, and that there is\nneed for novel research in order to achieve high-precision BookQA results. We\nanalyze some of the bottlenecks of the current approach, and we argue that more\nresearch is needed on text representation, retrieval of relevant passages, and\nreasoning, including commonsense knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00883v2",
            "date": "2019-10-02 11:34:58+00:00",
            "title": "Exploiting BERT for End-to-End Aspect-based Sentiment Analysis",
            "authors": [
                "Xin Li",
                "Lidong Bing",
                "Wenxuan Zhang",
                "Wai Lam"
            ],
            "abstract": "In this paper, we investigate the modeling power of contextualized embeddings\nfrom pre-trained language models, e.g. BERT, on the E2E-ABSA task.\nSpecifically, we build a series of simple yet insightful neural baselines to\ndeal with E2E-ABSA. The experimental results show that even with a simple\nlinear classification layer, our BERT-based architecture can outperform\nstate-of-the-art works. Besides, we also standardize the comparative study by\nconsistently utilizing a hold-out validation dataset for model selection, which\nis largely ignored by previous works. Therefore, our work can serve as a\nBERT-based benchmark for E2E-ABSA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00912v1",
            "date": "2019-10-02 12:39:53+00:00",
            "title": "Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLU",
            "authors": [
                "Andrea Vanzo",
                "Emanuele Bastianelli",
                "Oliver Lemon"
            ],
            "abstract": "We present a new neural architecture for wide-coverage Natural Language\nUnderstanding in Spoken Dialogue Systems. We develop a hierarchical multi-task\narchitecture, which delivers a multi-layer representation of sentence meaning\n(i.e., Dialogue Acts and Frame-like structures). The architecture is a\nhierarchy of self-attention mechanisms and BiLSTM encoders followed by CRF\ntagging layers. We describe a variety of experiments, showing that our approach\nobtains promising results on a dataset annotated with Dialogue Acts and Frame\nSemantics. Moreover, we demonstrate its applicability to a different, publicly\navailable NLU dataset annotated with domain-specific intents and corresponding\nsemantic roles, providing overall performance higher than state-of-the-art\ntools such as RASA, Dialogflow, LUIS, and Watson. For example, we show an\naverage 4.45% improvement in entity tagging F-score over Rasa, Dialogflow and\nLUIS.",
            "journal ref": "SIGDial 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00930v1",
            "date": "2019-10-02 13:05:48+00:00",
            "title": "A CCG-based Compositional Semantics and Inference System for Comparatives",
            "authors": [
                "Izumi Haruta",
                "Koji Mineshima",
                "Daisuke Bekki"
            ],
            "abstract": "Comparative constructions play an important role in natural language\ninference. However, attempts to study semantic representations and logical\ninferences for comparatives from the computational perspective are not well\ndeveloped, due to the complexity of their syntactic structures and inference\npatterns. In this study, using a framework based on Combinatory Categorial\nGrammar (CCG), we present a compositional semantics that maps various\ncomparative constructions in English to semantic representations and introduces\nan inference system that effectively handles logical inference with\ncomparatives, including those involving numeral adjectives, antonyms, and\nquantification. We evaluate the performance of our system on the FraCaS test\nsuite and show that the system can handle a variety of complex logical\ninferences with comparatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01043v1",
            "date": "2019-10-02 16:00:42+00:00",
            "title": "Neural Word Decomposition Models for Abusive Language Detection",
            "authors": [
                "Sravan Babu Bodapati",
                "Spandana Gella",
                "Kasturi Bhattacharjee",
                "Yaser Al-Onaizan"
            ],
            "abstract": "User generated text on social media often suffers from a lot of undesired\ncharacteristics including hatespeech, abusive language, insults etc. that are\ntargeted to attack or abuse a specific group of people. Often such text is\nwritten differently compared to traditional text such as news involving either\nexplicit mention of abusive words, obfuscated words and typological errors or\nimplicit abuse i.e., indicating or targeting negative stereotypes. Thus,\nprocessing this text poses several robustness challenges when we apply natural\nlanguage processing techniques developed for traditional text. For example,\nusing word or token based models to process such text can treat two spelling\nvariants of a word as two different words. Following recent work, we analyze\nhow character, subword and byte pair encoding (BPE) models can be aid some of\nthe challenges posed by user generated text. In our work, we analyze the\neffectiveness of each of the above techniques, compare and contrast various\nword decomposition techniques when used in combination with others. We\nexperiment with finetuning large pretrained language models, and demonstrate\ntheir robustness to domain shift by studying Wikipedia attack, toxicity and\nTwitter hatespeech datasets",
            "journal ref": "https://www.aclweb.org/anthology/events/acl-2019/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01108v4",
            "date": "2019-10-02 17:56:28+00:00",
            "title": "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter",
            "authors": [
                "Victor Sanh",
                "Lysandre Debut",
                "Julien Chaumond",
                "Thomas Wolf"
            ],
            "abstract": "As Transfer Learning from large-scale pre-trained models becomes more\nprevalent in Natural Language Processing (NLP), operating these large models in\non-the-edge and/or under constrained computational training or inference\nbudgets remains challenging. In this work, we propose a method to pre-train a\nsmaller general-purpose language representation model, called DistilBERT, which\ncan then be fine-tuned with good performances on a wide range of tasks like its\nlarger counterparts. While most prior work investigated the use of distillation\nfor building task-specific models, we leverage knowledge distillation during\nthe pre-training phase and show that it is possible to reduce the size of a\nBERT model by 40%, while retaining 97% of its language understanding\ncapabilities and being 60% faster. To leverage the inductive biases learned by\nlarger models during pre-training, we introduce a triple loss combining\nlanguage modeling, distillation and cosine-distance losses. Our smaller, faster\nand lighter model is cheaper to pre-train and we demonstrate its capabilities\nfor on-device computations in a proof-of-concept experiment and a comparative\non-device study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01157v2",
            "date": "2019-10-02 18:35:40+00:00",
            "title": "Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations",
            "authors": [
                "Jeff Da",
                "Jungo Kasai"
            ],
            "abstract": "Pretrained deep contextual representations have advanced the state-of-the-art\non various commonsense NLP tasks, but we lack a concrete understanding of the\ncapability of these models. Thus, we investigate and challenge several aspects\nof BERT's commonsense representation abilities. First, we probe BERT's ability\nto classify various object attributes, demonstrating that BERT shows a strong\nability in encoding various commonsense features in its embedding space, but is\nstill deficient in many areas. Next, we show that, by augmenting BERT's\npretraining data with additional data related to the deficient attributes, we\nare able to improve performance on a downstream commonsense reasoning task\nwhile using a minimal amount of data. Finally, we develop a method of\nfine-tuning knowledge graphs embeddings alongside BERT and show the continued\nimportance of explicit knowledge graphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01302v1",
            "date": "2019-10-03 05:08:15+00:00",
            "title": "Data-Efficient Goal-Oriented Conversation with Dialogue Knowledge Transfer Networks",
            "authors": [
                "Igor Shalyminov",
                "Sungjin Lee",
                "Arash Eshghi",
                "Oliver Lemon"
            ],
            "abstract": "Goal-oriented dialogue systems are now being widely adopted in industry where\nit is of key importance to maintain a rapid prototyping cycle for new products\nand domains. Data-driven dialogue system development has to be adapted to meet\nthis requirement --- therefore, reducing the amount of data and annotations\nnecessary for training such systems is a central research problem.\n  In this paper, we present the Dialogue Knowledge Transfer Network (DiKTNet),\na state-of-the-art approach to goal-oriented dialogue generation which only\nuses a few example dialogues (i.e. few-shot learning), none of which has to be\nannotated. We achieve this by performing a 2-stage training. Firstly, we\nperform unsupervised dialogue representation pre-training on a large source of\ngoal-oriented dialogues in multiple domains, the MetaLWOz corpus. Secondly, at\nthe transfer stage, we train DiKTNet using this representation together with 2\nother textual knowledge sources with different levels of generality: ELMo\nencoder and the main dataset's source domains.\n  Our main dataset is the Stanford Multi-Domain dialogue corpus. We evaluate\nour model on it in terms of BLEU and Entity F1 scores, and show that our\napproach significantly and consistently improves upon a series of baseline\nmodels as well as over the previous state-of-the-art dialogue generation model,\nZSDG. The improvement upon the latter --- up to 10% in Entity F1 and the\naverage of 3% in BLEU score --- is achieved using only the equivalent of 10% of\nZSDG's in-domain training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01335v1",
            "date": "2019-10-03 07:48:32+00:00",
            "title": "Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations",
            "authors": [
                "Zhengyuan Liu",
                "Angela Ng",
                "Sheldon Lee",
                "Ai Ti Aw",
                "Nancy F. Chen"
            ],
            "abstract": "Due to the lack of publicly available resources, conversation summarization\nhas received far less attention than text summarization. As the purpose of\nconversations is to exchange information between at least two interlocutors,\nkey information about a certain topic is often scattered and spanned across\nmultiple utterances and turns from different speakers. This phenomenon is more\npronounced during spoken conversations, where speech characteristics such as\nbackchanneling and false-starts might interrupt the topical flow. Moreover,\ntopic diffusion and (intra-utterance) topic drift are also more common in\nhuman-to-human conversations. Such linguistic characteristics of dialogue\ntopics make sentence-level extractive summarization approaches used in spoken\ndocuments ill-suited for summarizing conversations. Pointer-generator networks\nhave effectively demonstrated its strength at integrating extractive and\nabstractive capabilities through neural modeling in text summarization. To the\nbest of our knowledge, to date no one has adopted it for summarizing\nconversations. In this work, we propose a topic-aware architecture to exploit\nthe inherent hierarchical structure in conversations to further adapt the\npointer-generator model. Our approach significantly outperforms competitive\nbaselines, achieves more efficient learning outcomes, and attains more robust\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01462v1",
            "date": "2019-10-03 13:35:00+00:00",
            "title": "Towards Understanding of Medical Randomized Controlled Trials by Conclusion Generation",
            "authors": [
                "Alexander Te-Wei Shieh",
                "Yung-Sung Chuang",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Randomized controlled trials (RCTs) represent the paramount evidence of\nclinical medicine. Using machines to interpret the massive amount of RCTs has\nthe potential of aiding clinical decision-making. We propose a RCT conclusion\ngeneration task from the PubMed 200k RCT sentence classification dataset to\nexamine the effectiveness of sequence-to-sequence models on understanding RCTs.\nWe first build a pointer-generator baseline model for conclusion generation.\nThen we fine-tune the state-of-the-art GPT-2 language model, which is\npre-trained with general domain data, for this new medical domain task. Both\nautomatic and human evaluation show that our GPT-2 fine-tuned models achieve\nimproved quality and correctness in the generated conclusions compared to the\nbaseline pointer-generator model. Further inspection points out the limitations\nof this current approach and future directions to explore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01531v1",
            "date": "2019-10-03 14:46:22+00:00",
            "title": "Modeling Color Terminology Across Thousands of Languages",
            "authors": [
                "Arya D. McCarthy",
                "Winston Wu",
                "Aaron Mueller",
                "Bill Watson",
                "David Yarowsky"
            ],
            "abstract": "There is an extensive history of scholarship into what constitutes a \"basic\"\ncolor term, as well as a broadly attested acquisition sequence of basic color\nterms across many languages, as articulated in the seminal work of Berlin and\nKay (1969). This paper employs a set of diverse measures on massively\ncross-linguistic data to operationalize and critique the Berlin and Kay color\nterm hypotheses. Collectively, the 14 empirically-grounded computational\nlinguistic metrics we design---as well as their aggregation---correlate\nstrongly with both the Berlin and Kay basic/secondary color term partition\n(gamma=0.96) and their hypothesized universal acquisition sequence. The\nmeasures and result provide further empirical evidence from computational\nlinguistics in support of their claims, as well as additional nuance: they\nsuggest treating the partition as a spectrum instead of a dichotomy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01818v2",
            "date": "2019-10-04 07:22:25+00:00",
            "title": "DialectGram: Detecting Dialectal Variation at Multiple Geographic Resolutions",
            "authors": [
                "Hang Jiang",
                "Haoshen Hong",
                "Yuxing Chen",
                "Vivek Kulkarni"
            ],
            "abstract": "Several computational models have been developed to detect and analyze\ndialect variation in recent years. Most of these models assume a predefined set\nof geographical regions over which they detect and analyze dialectal variation.\nHowever, dialect variation occurs at multiple levels of geographic resolution\nranging from cities within a state, states within a country, and between\ncountries across continents. In this work, we propose a model that enables\ndetection of dialectal variation at multiple levels of geographic resolution\nobviating the need for a-priori definition of the resolution level. Our method\nDialectGram, learns dialect-sensitive word embeddings while being agnostic of\nthe geographic resolution. Specifically it only requires one-time training and\nenables analysis of dialectal variation at a chosen resolution post-hoc -- a\nsignificant departure from prior models which need to be re-trained whenever\nthe pre-defined set of regions changes. Furthermore, DialectGram explicitly\nmodels senses thus enabling one to estimate the proportion of each sense usage\nin any given region. Finally, we quantitatively evaluate our model against\nother baselines on a new evaluation dataset DialectSim (in English) and show\nthat DialectGram can effectively model linguistic variation.",
            "journal ref": "Proceedings of the Society for Computation in Linguistics, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01822v1",
            "date": "2019-10-04 07:27:23+00:00",
            "title": "Multi-level Gated Recurrent Neural Network for Dialog Act Classification",
            "authors": [
                "Wei Li",
                "Yunfang Wu"
            ],
            "abstract": "In this paper we focus on the problem of dialog act (DA) labelling. This\nproblem has recently attracted a lot of attention as it is an important\nsub-part of an automatic question answering system, which is currently in great\ndemand. Traditional methods tend to see this problem as a sequence labelling\ntask and deals with it by applying classifiers with rich features. Most of the\ncurrent neural network models still omit the sequential information in the\nconversation. Henceforth, we apply a novel multi-level gated recurrent neural\nnetwork (GRNN) with non-textual information to predict the DA tag. Our model\nnot only utilizes textual information, but also makes use of non-textual and\ncontextual information. In comparison, our model has shown significant\nimprovement over previous works on Switchboard Dialog Act (SWDA) task by over\n6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01859v1",
            "date": "2019-10-04 10:30:36+00:00",
            "title": "Modeling Confidence in Sequence-to-Sequence Models",
            "authors": [
                "Jan Niehues",
                "Ngoc-Quan Pham"
            ],
            "abstract": "Recently, significant improvements have been achieved in various natural\nlanguage processing tasks using neural sequence-to-sequence models. While\naiming for the best generation quality is important, ultimately it is also\nnecessary to develop models that can assess the quality of their output.\n  In this work, we propose to use the similarity between training and test\nconditions as a measure for models' confidence. We investigate methods solely\nusing the similarity as well as methods combining it with the posterior\nprobability. While traditionally only target tokens are annotated with\nconfidence measures, we also investigate methods to annotate source tokens with\nconfidence. By learning an internal alignment model, we can significantly\nimprove confidence projection over using state-of-the-art external alignment\ntools. We evaluate the proposed methods on downstream confidence estimation for\nmachine translation (MT). We show improvements on segment-level confidence\nestimation as well as on confidence estimation for source tokens. In addition,\nwe show that the same methods can also be applied to other tasks using\nsequence-to-sequence models. On the automatic speech recognition (ASR) task, we\nare able to find 60% of the errors by looking at 20% of the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01863v1",
            "date": "2019-10-04 10:40:44+00:00",
            "title": "Template-free Data-to-Text Generation of Finnish Sports News",
            "authors": [
                "Jenna Kanerva",
                "Samuel R\u00f6nnqvist",
                "Riina Kekki",
                "Tapio Salakoski",
                "Filip Ginter"
            ],
            "abstract": "News articles such as sports game reports are often thought to closely follow\nthe underlying game statistics, but in practice they contain a notable amount\nof background knowledge, interpretation, insight into the game, and quotes that\nare not present in the official statistics. This poses a challenge for\nautomated data-to-text news generation with real-world news corpora as training\ndata. We report on the development of a corpus of Finnish ice hockey news,\nedited to be suitable for training of end-to-end news generation methods, as\nwell as demonstrate generation of text, which was judged by journalists to be\nrelatively close to a viable product. The new dataset and system source code\nare available for research purposes at\nhttps://github.com/scoopmatic/finnish-hockey-news-generation-paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02223v1",
            "date": "2019-10-05 06:48:23+00:00",
            "title": "A Machine Learning Analysis of the Features in Deceptive and Credible News",
            "authors": [
                "Qi Jia Sun"
            ],
            "abstract": "Fake news is a type of pervasive propaganda that spreads misinformation\nonline, taking advantage of social media's extensive reach to manipulate public\nperception. Over the past three years, fake news has become a focal discussion\npoint in the media due to its impact on the 2016 U.S. presidential election.\nFake news can have severe real-world implications: in 2016, a man walked into a\npizzeria carrying a rifle because he read that Hillary Clinton was harboring\nchildren as sex slaves. This project presents a high accuracy (87%) machine\nlearning classifier that determines the validity of news based on the word\ndistributions and specific linguistic and stylistic differences in the first\nfew sentences of an article. This can help readers identify the validity of an\narticle by looking for specific features in the opening lines aiding them in\nmaking informed decisions. Using a dataset of 2,107 articles from 30 different\nwebsites, this project establishes an understanding of the variations between\nfake and credible news by examining the model, dataset, and features. This\nclassifier appears to use the differences in word distribution, levels of tone\nauthenticity, and frequency of adverbs, adjectives, and nouns. The\ndifferentiation in the features of these articles can be used to improve future\nclassifiers. This classifier can also be further applied directly to browsers\nas a Google Chrome extension or as a filter for social media outlets or news\nwebsites to reduce the spread of misinformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02228v1",
            "date": "2019-10-05 07:49:12+00:00",
            "title": "On the Limits of Learning to Actively Learn Semantic Representations",
            "authors": [
                "Omri Koshorek",
                "Gabriel Stanovsky",
                "Yichu Zhou",
                "Vivek Srikumar",
                "Jonathan Berant"
            ],
            "abstract": "One of the goals of natural language understanding is to develop models that\nmap sentences into meaning representations. However, training such models\nrequires expensive annotation of complex structures, which hinders their\nadoption. Learning to actively-learn (LTAL) is a recent paradigm for reducing\nthe amount of labeled data by learning a policy that selects which samples\nshould be labeled. In this work, we examine LTAL for learning semantic\nrepresentations, such as QA-SRL. We show that even an oracle policy that is\nallowed to pick examples that maximize performance on the test set (and\nconstitutes an upper bound on the potential of LTAL), does not substantially\nimprove performance compared to a random policy. We investigate factors that\ncould explain this finding and show that a distinguishing characteristic of\nsuccessful applications of LTAL is the interaction between optimization and the\noracle policy selection process. In successful applications of LTAL, the\nexamples selected by the oracle policy do not substantially depend on the\noptimization procedure, while in our setup the stochastic nature of\noptimization strongly affects the examples selected by the oracle. We conclude\nthat the current applicability of LTAL for improving data efficiency in\nlearning semantic meaning representations is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02238v2",
            "date": "2019-10-05 09:16:44+00:00",
            "title": "How Transformer Revitalizes Character-based Neural Machine Translation: An Investigation on Japanese-Vietnamese Translation Systems",
            "authors": [
                "Thi-Vinh Ngo",
                "Thanh-Le Ha",
                "Phuong-Thai Nguyen",
                "Le-Minh Nguyen"
            ],
            "abstract": "While translating between East Asian languages, many works have discovered\nclear advantages of using characters as the translation unit. Unfortunately,\ntraditional recurrent neural machine translation systems hinder the practical\nusage of those character-based systems due to their architectural limitations.\nThey are unfavorable in handling extremely long sequences as well as highly\nrestricted in parallelizing the computations. In this paper, we demonstrate\nthat the new transformer architecture can perform character-based translation\nbetter than the recurrent one. We conduct experiments on a low-resource\nlanguage pair: Japanese-Vietnamese. Our models considerably outperform the\nstate-of-the-art systems which employ word-based recurrent architectures.",
            "journal ref": "16th International Workshop on Spoken Language Translation 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02267v1",
            "date": "2019-10-05 13:31:39+00:00",
            "title": "Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging",
            "authors": [
                "Nasser Zalmout",
                "Nizar Habash"
            ],
            "abstract": "Semitic languages can be highly ambiguous, having several interpretations of\nthe same surface forms, and morphologically rich, having many morphemes that\nrealize several morphological features. This is further exacerbated for\ndialectal content, which is more prone to noise and lacks a standard\northography. The morphological features can be lexicalized, like lemmas and\ndiacritized forms, or non-lexicalized, like gender, number, and part-of-speech\ntags, among others. Joint modeling of the lexicalized and non-lexicalized\nfeatures can identify more intricate morphological patterns, which provide\nbetter context modeling, and further disambiguate ambiguous lexical choices.\nHowever, the different modeling granularity can make joint modeling more\ndifficult. Our approach models the different features jointly, whether\nlexicalized (on the character-level), where we also model surface form\nnormalization, or non-lexicalized (on the word-level). We use Arabic as a test\ncase, and achieve state-of-the-art results for Modern Standard Arabic, with 20%\nrelative error reduction, and Egyptian Arabic (a dialectal variant of Arabic),\nwith 11% reduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02356v2",
            "date": "2019-10-06 02:38:25+00:00",
            "title": "Text Level Graph Neural Network for Text Classification",
            "authors": [
                "Lianzhe Huang",
                "Dehong Ma",
                "Sujian Li",
                "Xiaodong Zhang",
                "Houfeng WANG"
            ],
            "abstract": "Recently, researches have explored the graph neural network (GNN) techniques\non text classification, since GNN does well in handling complex structures and\npreserving global information. However, previous methods based on GNN are\nmainly faced with the practical problems of fixed corpus level graph structure\nwhich do not support online testing and high memory consumption. To tackle the\nproblems, we propose a new GNN based model that builds graphs for each input\ntext with global parameters sharing instead of a single graph for the whole\ncorpus. This method removes the burden of dependence between an individual text\nand entire corpus which support online testing, but still preserve global\ninformation. Besides, we build graphs by much smaller windows in the text,\nwhich not only extract more local features but also significantly reduce the\nedge numbers as well as memory consumption. Experiments show that our model\noutperforms existing models on several text classification datasets even with\nconsuming less memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02403v2",
            "date": "2019-10-06 09:35:52+00:00",
            "title": "Named Entity Recognition -- Is there a glass ceiling?",
            "authors": [
                "Tomasz Stanislawek",
                "Anna Wr\u00f3blewska",
                "Alicja W\u00f3jcicka",
                "Daniel Ziembicki",
                "Przemyslaw Biecek"
            ],
            "abstract": "Recent developments in Named Entity Recognition (NER) have resulted in better\nand better models. However, is there a glass ceiling? Do we know which types of\nerrors are still hard or even impossible to correct? In this paper, we present\na detailed analysis of the types of errors in state-of-the-art machine learning\n(ML) methods. Our study reveals the weak and strong points of the Stanford,\nCMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also\nintroduce new techniques for improving annotation, for training processes and\nfor checking a model's quality and stability. Presented results are based on\nthe CoNLL 2003 data set for the English language. A new enriched semantic\nannotation of errors for this data set and new diagnostic data sets are\nattached in the supplementary materials.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02555v1",
            "date": "2019-10-07 00:15:11+00:00",
            "title": "Domain Differential Adaptation for Neural Machine Translation",
            "authors": [
                "Zi-Yi Dou",
                "Xinyi Wang",
                "Junjie Hu",
                "Graham Neubig"
            ],
            "abstract": "Neural networks are known to be data hungry and domain sensitive, but it is\nnearly impossible to obtain large quantities of labeled data for every domain\nwe are interested in. This necessitates the use of domain adaptation\nstrategies. One common strategy encourages generalization by aligning the\nglobal distribution statistics between source and target domains, but one\ndrawback is that the statistics of different domains or tasks are inherently\ndivergent, and smoothing over these differences can lead to sub-optimal\nperformance. In this paper, we propose the framework of {\\it Domain\nDifferential Adaptation (DDA)}, where instead of smoothing over these\ndifferences we embrace them, directly modeling the difference between domains\nusing models in a related task. We then use these learned domain differentials\nto adapt models for the target task accordingly. Experimental results on domain\nadaptation for neural machine translation demonstrate the effectiveness of this\nstrategy, achieving consistent improvements over other alternative adaptation\nstrategies in multiple experimental settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02586v2",
            "date": "2019-10-07 03:03:08+00:00",
            "title": "Why Attention? Analyzing and Remedying BiLSTM Deficiency in Modeling Cross-Context for NER",
            "authors": [
                "Peng-Hsuan Li",
                "Tsu-Jui Fu",
                "Wei-Yun Ma"
            ],
            "abstract": "State-of-the-art approaches of NER have used sequence-labeling BiLSTM as a\ncore module. This paper formally shows the limitation of BiLSTM in modeling\ncross-context patterns. Two types of simple cross-structures -- self-attention\nand Cross-BiLSTM -- are shown to effectively remedy the problem. On both\nOntoNotes 5.0 and WNUT 2017, clear and consistent improvements are achieved\nover bare-bone models, up to 8.7% on some of the multi-token mentions. In-depth\nanalyses across several aspects of the improvements, especially the\nidentification of multi-token mentions, are further given.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02655v1",
            "date": "2019-10-07 07:58:26+00:00",
            "title": "BERT for Evidence Retrieval and Claim Verification",
            "authors": [
                "Amir Soleimani",
                "Christof Monz",
                "Marcel Worring"
            ],
            "abstract": "Motivated by the promising performance of pre-trained language models, we\ninvestigate BERT in an evidence retrieval and claim verification pipeline for\nthe FEVER fact extraction and verification challenge. To this end, we propose\nto use two BERT models, one for retrieving potential evidence sentences\nsupporting or rejecting claims, and another for verifying claims based on the\npredicted evidence sets. To train the BERT retrieval system, we use pointwise\nand pairwise loss functions, and examine the effect of hard negative mining. A\nsecond BERT model is trained to classify the samples as supported, refuted, and\nnot enough information. Our system achieves a new state of the art recall of\n87.1 for retrieving top five sentences out of the FEVER documents consisting of\n50K Wikipedia pages, and scores second in the official leaderboard with the\nFEVER score of 69.7.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02677v3",
            "date": "2019-10-07 09:00:26+00:00",
            "title": "Controllable Sentence Simplification",
            "authors": [
                "Louis Martin",
                "Beno\u00eet Sagot",
                "\u00c9ric de la Clergerie",
                "Antoine Bordes"
            ],
            "abstract": "Text simplification aims at making a text easier to read and understand by\nsimplifying grammar and structure while keeping the underlying information\nidentical. It is often considered an all-purpose generic task where the same\nsimplification is suitable for all; however multiple audiences can benefit from\nsimplified text in different ways. We adapt a discrete parametrization\nmechanism that provides explicit control on simplification systems based on\nSequence-to-Sequence models. As a result, users can condition the\nsimplifications returned by a model on attributes such as length, amount of\nparaphrasing, lexical complexity and syntactic complexity. We also show that\ncarefully chosen values of these attributes allow out-of-the-box\nSequence-to-Sequence models to outperform their standard counterparts on\nsimplification benchmarks. Our model, which we call ACCESS (as shorthand for\nAudienCe-CEntric Sentence Simplification), establishes the state of the art at\n41.87 SARI on the WikiLarge test set, a +1.42 improvement over the best\npreviously reported score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02733v1",
            "date": "2019-10-07 11:38:28+00:00",
            "title": "MaskParse@Deskin at SemEval-2019 Task 1: Cross-lingual UCCA Semantic Parsing using Recursive Masked Sequence Tagging",
            "authors": [
                "Gabriel Marzinotto",
                "Johannes Heinecke",
                "Geraldine Damnati"
            ],
            "abstract": "This paper describes our recursive system for SemEval-2019 \\textit{ Task 1:\nCross-lingual Semantic Parsing with UCCA}. Each recursive step consists of two\nparts. We first perform semantic parsing using a sequence tagger to estimate\nthe probabilities of the UCCA categories in the sentence. Then, we apply a\ndecoding policy which interprets these probabilities and builds the graph\nnodes. Parsing is done recursively, we perform a first inference on the\nsentence to extract the main scenes and links and then we recursively apply our\nmodel on the sentence using a masking feature that reflects the decisions made\nin previous steps. Process continues until the terminal nodes are reached. We\nchoose a standard neural tagger and we focused on our recursive parsing\nstrategy and on the cross lingual transfer problem to develop a robust model\nfor the French language, using only few training samples.",
            "journal ref": "Proceedings of the Thirteenth International Workshop on Semantic\n  Evaluation, Jun 2019, Minneapolis, United States",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02734v1",
            "date": "2019-10-07 11:41:14+00:00",
            "title": "Adapting a FrameNet Semantic Parser for Spoken Language Understanding Using Adversarial Learning",
            "authors": [
                "Gabriel Marzinotto",
                "Geraldine Damnati",
                "Fr\u00e9d\u00e9ric B\u00e9chet"
            ],
            "abstract": "This paper presents a new semantic frame parsing model, based on Berkeley\nFrameNet, adapted to process spoken documents in order to perform information\nextraction from broadcast contents. Building upon previous work that had shown\nthe effectiveness of adversarial learning for domain generalization in the\ncontext of semantic parsing of encyclopedic written documents, we propose to\nextend this approach to elocutionary style generalization. The underlying\nquestion throughout this study is whether adversarial learning can be used to\ncombine data from different sources and train models on a higher level of\nabstraction in order to increase their robustness to lexical and stylistic\nvariations as well as automatic speech recognition errors. The proposed\nstrategy is evaluated on a French corpus of encyclopedic written documents and\na smaller corpus of radio podcast transcriptions, both annotated with a\nFrameNet paradigm. We show that adversarial learning increases all models\ngeneralization capabilities both on manual and automatic speech transcription\nas well as on encyclopedic data.",
            "journal ref": "Interspeech 2019, Sep 2019, Graz, Austria. pp.799-803",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02766v1",
            "date": "2019-10-07 13:08:07+00:00",
            "title": "Adversarial reconstruction for Multi-modal Machine Translation",
            "authors": [
                "Jean-Benoit Delbrouck",
                "St\u00e9phane Dupont"
            ],
            "abstract": "Even with the growing interest in problems at the intersection of Computer\nVision and Natural Language, grounding (i.e. identifying) the components of a\nstructured description in an image still remains a challenging task. This\ncontribution aims to propose a model which learns grounding by reconstructing\nthe visual features for the Multi-modal translation task. Previous works have\npartially investigated standard approaches such as regression methods to\napproximate the reconstruction of a visual input. In this paper, we propose a\ndifferent and novel approach which learns grounding by adversarial feedback. To\ndo so, we modulate our network following the recent promising adversarial\narchitectures and evaluate how the adversarial response from a visual\nreconstruction as an auxiliary task helps the model in its learning. We report\nthe highest scores in term of BLEU and METEOR metrics on the different\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02930v1",
            "date": "2019-10-07 17:39:39+00:00",
            "title": "A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captions",
            "authors": [
                "Jack Hessel",
                "Bo Pang",
                "Zhenhai Zhu",
                "Radu Soricut"
            ],
            "abstract": "Instructional videos get high-traffic on video sharing platforms, and prior\nwork suggests that providing time-stamped, subtask annotations (e.g., \"heat the\noil in the pan\") improves user experiences. However, current automatic\nannotation methods based on visual features alone perform only slightly better\nthan constant prediction. Taking cues from prior work, we show that we can\nimprove performance significantly by considering automatic speech recognition\n(ASR) tokens as input. Furthermore, jointly modeling ASR tokens and visual\nfeatures results in higher performance compared to training individually on\neither modality. We find that unstated background information is better\nexplained by visual features, whereas fine-grained distinctions (e.g., \"add\noil\" vs. \"add olive oil\") are disambiguated more easily via ASR tokens.",
            "journal ref": "Published in The SIGNLL Conference on Computational Natural\n  Language Learning (CoNLL) 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03009v3",
            "date": "2019-10-07 18:55:32+00:00",
            "title": "Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back Translation",
            "authors": [
                "Zhenhao Li",
                "Lucia Specia"
            ],
            "abstract": "Neural Machine Translation (NMT) models have been proved strong when\ntranslating clean texts, but they are very sensitive to noise in the input.\nImproving NMT models robustness can be seen as a form of \"domain\" adaption to\nnoise. The recently created Machine Translation on Noisy Text task corpus\nprovides noisy-clean parallel data for a few language pairs, but this data is\nvery limited in size and diversity. The state-of-the-art approaches are heavily\ndependent on large volumes of back-translated data. This paper has two main\ncontributions: Firstly, we propose new data augmentation methods to extend\nlimited noisy data and further improve NMT robustness to noise while keeping\nthe models small. Secondly, we explore the effect of utilizing noise from\nexternal data in the form of speech transcripts and show that it could help\nrobustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03136v1",
            "date": "2019-10-07 23:46:49+00:00",
            "title": "Capturing Argument Interaction in Semantic Role Labeling with Capsule Networks",
            "authors": [
                "Xinchi Chen",
                "Chunchuan Lyu",
                "Ivan Titov"
            ],
            "abstract": "Semantic role labeling (SRL) involves extracting propositions (i.e.\npredicates and their typed arguments) from natural language sentences.\nState-of-the-art SRL models rely on powerful encoders (e.g., LSTMs) and do not\nmodel non-local interaction between arguments. We propose a new approach to\nmodeling these interactions while maintaining efficient inference.\nSpecifically, we use Capsule Networks: each proposition is encoded as a tuple\nof \\textit{capsules}, one capsule per argument type (i.e. role). These tuples\nserve as embeddings of entire propositions. In every network layer, the\ncapsules interact with each other and with representations of words in the\nsentence. Each iteration results in updated proposition embeddings and updated\npredictions about the SRL structure. Our model substantially outperforms the\nnon-refinement baseline model on all 7 CoNLL-2019 languages and achieves\nstate-of-the-art results on 5 languages (including English) for dependency SRL.\nWe analyze the types of mistakes corrected by the refinement procedure. For\nexample, each role is typically (but not always) filled with at most one\nargument. Whereas enforcing this approximate constraint is not useful with the\nmodern SRL system, iterative procedure corrects the mistakes by capturing this\nintuition in a flexible and context-sensitive way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03246v1",
            "date": "2019-10-08 07:18:58+00:00",
            "title": "Riposte! A Large Corpus of Counter-Arguments",
            "authors": [
                "Paul Reisert",
                "Benjamin Heinzerling",
                "Naoya Inoue",
                "Shun Kiyono",
                "Kentaro Inui"
            ],
            "abstract": "Constructive feedback is an effective method for improving critical thinking\nskills. Counter-arguments (CAs), one form of constructive feedback, have been\nproven to be useful for critical thinking skills. However, little work has been\ndone for constructing a large-scale corpus of them which can drive research on\nautomatic generation of CAs for fallacious micro-level arguments (i.e. a single\nclaim and premise pair). In this work, we cast providing constructive feedback\nas a natural language processing task and create Riposte!, a corpus of CAs,\ntowards this goal. Produced by crowdworkers, Riposte! contains over 18k CAs. We\ninstruct workers to first identify common fallacy types and produce a CA which\nidentifies the fallacy. We analyze how workers create CAs and construct a\nbaseline model based on our analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03355v1",
            "date": "2019-10-08 12:15:52+00:00",
            "title": "An Interactive Machine Translation Framework for Modernizing Historical Documents",
            "authors": [
                "Miguel Domingo",
                "Francisco Casacuberta"
            ],
            "abstract": "Due to the nature of human language, historical documents are hard to\ncomprehend by contemporary people. This limits their accessibility to scholars\nspecialized in the time period in which the documents were written.\nModernization aims at breaking this language barrier by generating a new\nversion of a historical document, written in the modern version of the\ndocument's original language. However, while it is able to increase the\ndocument's comprehension, modernization is still far from producing an\nerror-free version. In this work, we propose a collaborative framework in which\na scholar can work together with the machine to generate the new version. We\ntested our approach on a simulated environment, achieving significant\nreductions of the human effort needed to produce the modernized version of the\ndocument.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03375v1",
            "date": "2019-10-08 13:06:01+00:00",
            "title": "In Search for Linear Relations in Sentence Embedding Spaces",
            "authors": [
                "Petra Baran\u010d\u00edkov\u00e1",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "We present an introductory investigation into continuous-space vector\nrepresentations of sentences. We acquire pairs of very similar sentences\ndiffering only by a small alterations (such as change of a noun, adding an\nadjective, noun or punctuation) from datasets for natural language inference\nusing a simple pattern method. We look into how such a small change within the\nsentence text affects its representation in the continuous space and how such\nalterations are reflected by some of the popular sentence embedding models. We\nfound that vector differences of some embeddings actually reflect small changes\nwithin a sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03385v1",
            "date": "2019-10-08 13:33:48+00:00",
            "title": "Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in BioNLP-OST 2019",
            "authors": [
                "Usama Yaseen",
                "Pankaj Gupta",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Named Entity Recognition (NER) and Relation Extraction (RE) are essential\ntools in distilling knowledge from biomedical literature. This paper presents\nour findings from participating in BioNLP Shared Tasks 2019. We addressed Named\nEntity Recognition including nested entities extraction, Entity Normalization\nand Relation Extraction. Our proposed approach of Named Entities can be\ngeneralized to different languages and we have shown it's effectiveness for\nEnglish and Spanish text. We investigated linguistic features, hybrid loss\nincluding ranking and Conditional Random Fields (CRF), multi-task objective and\ntoken-level ensembling strategy to improve NER. We employed dictionary based\nfuzzy and semantic search to perform Entity Normalization. Finally, our RE\nsystem employed Support Vector Machine (SVM) with linguistic features.\n  Our NER submission (team:MIC-CIS) ranked first in BB-2019 norm+NER task with\nstandard error rate (SER) of 0.7159 and showed competitive performance on\nPharmaCo NER task with F1-score of 0.8662. Our RE system ranked first in the\nSeeDev-binary Relation Extraction Task with F1-score of 0.3738.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03401v1",
            "date": "2019-10-08 13:57:02+00:00",
            "title": "Generating Highly Relevant Questions",
            "authors": [
                "Jiazuo Qiu",
                "Deyi Xiong"
            ],
            "abstract": "The neural seq2seq based question generation (QG) is prone to generating\ngeneric and undiversified questions that are poorly relevant to the given\npassage and target answer. In this paper, we propose two methods to address the\nissue. (1) By a partial copy mechanism, we prioritize words that are\nmorphologically close to words in the input passage when generating questions;\n(2) By a QA-based reranker, from the n-best list of question candidates, we\nselect questions that are preferred by both the QA and QG model. Experiments\nand analyses demonstrate that the proposed two methods substantially improve\nthe relevance of generated questions to passages and answers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03475v1",
            "date": "2019-09-28 07:45:55+00:00",
            "title": "Named Entity Recognition System for Sindhi Language",
            "authors": [
                "Awais Khan Jumani",
                "Mashooque Ahmed Memon",
                "Fida Hussain Khoso",
                "Anwar Ali Sanjrani",
                "Safeeullah Soomro"
            ],
            "abstract": "Named Entity Recognition (NER) System aims to extract the existing\ninformation into the following categories such as: Persons Name, Organization,\nLocation, Date and Time, Term, Designation and Short forms. Now, it is\nconsidered to be important aspect for many natural languages processing (NLP)\ntasks such as: information retrieval system, machine translation system,\ninformation extraction system and question answering. Even at a surface level,\nthe understanding of the named entities involved in a document gives richer\nanalytical framework and cross referencing. It has been used for different\nArabic Script-Based languages like, Arabic, Persian and Urdu but, Sindhi could\nnot come into being yet. This paper explains the problem of NER in the\nframework of Sindhi Language and provides relevant solution. The system is\ndeveloped to tag ten different Named Entities. We have used Ruled based\napproach for NER system of Sindhi Language. For the training and testing, 936\nwords were used and calculated performance accuracy of 98.71%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03484v1",
            "date": "2019-09-29 11:37:18+00:00",
            "title": "Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models",
            "authors": [
                "Raheel Qader",
                "Fran\u00e7ois Portet",
                "Cyril Labb\u00e9"
            ],
            "abstract": "In Natural Language Generation (NLG), End-to-End (E2E) systems trained\nthrough deep learning have recently gained a strong interest. Such deep models\nneed a large amount of carefully annotated data to reach satisfactory\nperformance. However, acquiring such datasets for every new NLG application is\na tedious and time-consuming task. In this paper, we propose a semi-supervised\ndeep learning scheme that can learn from non-annotated data and annotated data\nwhen available. It uses an NLG and a Natural Language Understanding (NLU)\nsequence-to-sequence models which are learned jointly to compensate for the\nlack of annotation. Experiments on two benchmark datasets show that, with\nlimited amount of annotated data, the method can achieve very competitive\nresults while not using any pre-processing or re-scoring tricks. These findings\nopen the way to the exploitation of non-annotated datasets which is the current\nbottleneck for the E2E NLG system development to new applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03723v2",
            "date": "2019-10-08 23:56:27+00:00",
            "title": "Knowledge Distillation from Internal Representations",
            "authors": [
                "Gustavo Aguilar",
                "Yuan Ling",
                "Yu Zhang",
                "Benjamin Yao",
                "Xing Fan",
                "Chenlei Guo"
            ],
            "abstract": "Knowledge distillation is typically conducted by training a small model (the\nstudent) to mimic a large and cumbersome model (the teacher). The idea is to\ncompress the knowledge from the teacher by using its output probabilities as\nsoft-labels to optimize the student. However, when the teacher is considerably\nlarge, there is no guarantee that the internal knowledge of the teacher will be\ntransferred into the student; even if the student closely matches the\nsoft-labels, its internal representations may be considerably different. This\ninternal mismatch can undermine the generalization capabilities originally\nintended to be transferred from the teacher to the student. In this paper, we\npropose to distill the internal representations of a large model such as BERT\ninto a simplified version of it. We formulate two ways to distill such\nrepresentations and various algorithms to conduct the distillation. We\nexperiment with datasets from the GLUE benchmark and consistently show that\nadding knowledge distillation from internal representations is a more powerful\nmethod than only using soft-label distillation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03739v1",
            "date": "2019-10-09 01:33:29+00:00",
            "title": "Towards De-identification of Legal Texts",
            "authors": [
                "Diego Garat",
                "Dina Wonsever"
            ],
            "abstract": "In many countries, personal information that can be published or shared\nbetween organizations is regulated and, therefore, documents must undergo a\nprocess of de-identification to eliminate or obfuscate confidential data. Our\nwork focuses on the de-identification of legal texts, where the goal is to hide\nthe names of the actors involved in a lawsuit without losing the sense of the\nstory. We present a first evaluation on our corpus of NLP tools in tasks such\nas segmentation, tokenization and recognition of named entities, and we analyze\nseveral evaluation measures for our de-identification task. Results are meager:\n84% of the documents have at least one name not covered by NER tools, something\nthat might lead to the re-identification of involved names. We conclude that\ntools must be strongly adapted for processing texts of this particular domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03747v2",
            "date": "2019-10-09 01:54:47+00:00",
            "title": "The Daunting Task of Real-World Textual Style Transfer Auto-Evaluation",
            "authors": [
                "Richard Yuanzhe Pang"
            ],
            "abstract": "The difficulty of textual style transfer lies in the lack of parallel\ncorpora. Numerous advances have been proposed for the unsupervised generation.\nHowever, significant problems remain with the auto-evaluation of style transfer\ntasks. Based on the summary of Pang and Gimpel (2018) and Mir et al. (2019),\nstyle transfer evaluations rely on three criteria: style accuracy of\ntransferred sentences, content similarity between original and transferred\nsentences, and fluency of transferred sentences. We elucidate the problematic\ncurrent state of style transfer research. Given that current tasks do not\nrepresent real use cases of style transfer, current auto-evaluation approach is\nflawed. This discussion aims to bring researchers to think about the future of\nstyle transfer and style transfer evaluation research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03771v5",
            "date": "2019-10-09 03:23:22+00:00",
            "title": "HuggingFace's Transformers: State-of-the-art Natural Language Processing",
            "authors": [
                "Thomas Wolf",
                "Lysandre Debut",
                "Victor Sanh",
                "Julien Chaumond",
                "Clement Delangue",
                "Anthony Moi",
                "Pierric Cistac",
                "Tim Rault",
                "R\u00e9mi Louf",
                "Morgan Funtowicz",
                "Joe Davison",
                "Sam Shleifer",
                "Patrick von Platen",
                "Clara Ma",
                "Yacine Jernite",
                "Julien Plu",
                "Canwen Xu",
                "Teven Le Scao",
                "Sylvain Gugger",
                "Mariama Drame",
                "Quentin Lhoest",
                "Alexander M. Rush"
            ],
            "abstract": "Recent progress in natural language processing has been driven by advances in\nboth model architecture and model pretraining. Transformer architectures have\nfacilitated building higher-capacity models and pretraining has made it\npossible to effectively utilize this capacity for a wide variety of tasks.\n\\textit{Transformers} is an open-source library with the goal of opening up\nthese advances to the wider machine learning community. The library consists of\ncarefully engineered state-of-the art Transformer architectures under a unified\nAPI. Backing this library is a curated collection of pretrained models made by\nand available for the community. \\textit{Transformers} is designed to be\nextensible by researchers, simple for practitioners, and fast and robust in\nindustrial deployments. The library is available at\n\\url{https://github.com/huggingface/transformers}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04006v1",
            "date": "2019-10-09 14:10:47+00:00",
            "title": "Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction",
            "authors": [
                "Elena Alvarez-Mellado",
                "Eben Holderness",
                "Nicholas Miller",
                "Fyonn Dhang",
                "Philip Cawkwell",
                "Kirsten Bolton",
                "James Pustejovsky",
                "Mei-Hua Hall"
            ],
            "abstract": "Predicting which patients are more likely to be readmitted to a hospital\nwithin 30 days after discharge is a valuable piece of information in clinical\ndecision-making. Building a successful readmission risk classifier based on the\ncontent of Electronic Health Records (EHRs) has proved, however, to be a\nchallenging task. Previously explored features include mainly structured\ninformation, such as sociodemographic data, comorbidity codes and physiological\nvariables. In this paper we assess incorporating additional clinically\ninterpretable NLP-based features such as topic extraction and clinical\nsentiment analysis to predict early readmission risk in psychiatry patients.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04073v1",
            "date": "2019-10-09 15:42:25+00:00",
            "title": "BHAAV- A Text Corpus for Emotion Analysis from Hindi Stories",
            "authors": [
                "Yaman Kumar",
                "Debanjan Mahata",
                "Sagar Aggarwal",
                "Anmol Chugh",
                "Rajat Maheshwari",
                "Rajiv Ratn Shah"
            ],
            "abstract": "In this paper, we introduce the first and largest Hindi text corpus, named\nBHAAV, which means emotions in Hindi, for analyzing emotions that a writer\nexpresses through his characters in a story, as perceived by a narrator/reader.\nThe corpus consists of 20,304 sentences collected from 230 different short\nstories spanning across 18 genres such as Inspirational and Mystery. Each\nsentence has been annotated into one of the five emotion categories - anger,\njoy, suspense, sad, and neutral, by three native Hindi speakers with at least\nten years of formal education in Hindi. We also discuss challenges in the\nannotation of low resource languages such as Hindi, and discuss the scope of\nthe proposed corpus along with its possible uses. We also provide a detailed\nanalysis of the dataset and train strong baseline classifiers reporting their\nperformances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04196v1",
            "date": "2019-10-09 18:34:17+00:00",
            "title": "Efficient Semi-Supervised Learning for Natural Language Understanding by Optimizing Diversity",
            "authors": [
                "Eunah Cho",
                "He Xie",
                "John P. Lalor",
                "Varun Kumar",
                "William M. Campbell"
            ],
            "abstract": "Expanding new functionalities efficiently is an ongoing challenge for\nsingle-turn task-oriented dialogue systems. In this work, we explore\nfunctionality-specific semi-supervised learning via self-training. We consider\nmethods that augment training data automatically from unlabeled data sets in a\nfunctionality-targeted manner. In addition, we examine multiple techniques for\nefficient selection of augmented utterances to reduce training time and\nincrease diversity. First, we consider paraphrase detection methods that\nattempt to find utterance variants of labeled training data with good coverage.\nSecond, we explore sub-modular optimization based on n-grams features for\nutterance selection. Experiments show that functionality-specific self-training\nis very effective for improving system performance. In addition, methods\noptimizing diversity can reduce training data in many cases to 50% with little\nimpact on performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04210v1",
            "date": "2019-10-09 19:25:21+00:00",
            "title": "Perturbation Sensitivity Analysis to Detect Unintended Model Biases",
            "authors": [
                "Vinodkumar Prabhakaran",
                "Ben Hutchinson",
                "Margaret Mitchell"
            ],
            "abstract": "Data-driven statistical Natural Language Processing (NLP) techniques leverage\nlarge amounts of language data to build models that can understand language.\nHowever, most language data reflect the public discourse at the time the data\nwas produced, and hence NLP models are susceptible to learning incidental\nassociations around named referents at a particular point in time, in addition\nto general linguistic meaning. An NLP system designed to model notions such as\nsentiment and toxicity should ideally produce scores that are independent of\nthe identity of such entities mentioned in text and their social associations.\nFor example, in a general purpose sentiment analysis system, a phrase such as I\nhate Katy Perry should be interpreted as having the same sentiment as I hate\nTaylor Swift. Based on this idea, we propose a generic evaluation framework,\nPerturbation Sensitivity Analysis, which detects unintended model biases\nrelated to named entities, and requires no new annotations or corpora. We\ndemonstrate the utility of this analysis by employing it on two different NLP\nmodels --- a sentiment model and a toxicity model --- applied on online\ncomments in English language from four different genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04387v1",
            "date": "2019-10-10 06:37:36+00:00",
            "title": "Controllable Sentence Simplification: Employing Syntactic and Lexical Constraints",
            "authors": [
                "Jonathan Mallinson",
                "Mirella Lapata"
            ],
            "abstract": "Sentence simplification aims to make sentences easier to read and understand.\nRecent approaches have shown promising results with sequence-to-sequence models\nwhich have been developed assuming homogeneous target audiences. In this paper\nwe argue that different users have different simplification needs (e.g.\ndyslexics vs. non-native speakers), and propose CROSS, ContROllable Sentence\nSimplification model, which allows to control both the level of simplicity and\nthe type of the simplification. We achieve this by enriching a\nTransformer-based architecture with syntactic and lexical constraints (which\ncan be set or learned from data). Empirical results on two benchmark datasets\nshow that constraints are key to successful simplification, offering flexible\ngeneration output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04601v2",
            "date": "2019-10-10 14:28:56+00:00",
            "title": "R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason",
            "authors": [
                "Naoya Inoue",
                "Pontus Stenetorp",
                "Kentaro Inui"
            ],
            "abstract": "Recent studies have revealed that reading comprehension (RC) systems learn to\nexploit annotation artifacts and other biases in current datasets. This\nprevents the community from reliably measuring the progress of RC systems. To\naddress this issue, we introduce R4C, a new task for evaluating RC systems'\ninternal reasoning. R4C requires giving not only answers but also derivations:\nexplanations that justify predicted answers. We present a reliable,\ncrowdsourced framework for scalably annotating RC datasets with derivations. We\ncreate and publicly release the R4C dataset, the first, quality-assured dataset\nconsisting of 4.6k questions, each of which is annotated with 3 reference\nderivations (i.e. 13.8k derivations). Experiments show that our automatic\nevaluation metrics using multiple reference derivations are reliable, and that\nR4C assesses different skills from an existing benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04602v4",
            "date": "2019-10-10 14:28:57+00:00",
            "title": "Multi-label Categorization of Accounts of Sexism using a Neural Framework",
            "authors": [
                "Pulkit Parikh",
                "Harika Abburi",
                "Pinkesh Badjatiya",
                "Radhika Krishnan",
                "Niyati Chhaya",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Sexism, an injustice that subjects women and girls to enormous suffering,\nmanifests in blatant as well as subtle ways. In the wake of growing\ndocumentation of experiences of sexism on the web, the automatic categorization\nof accounts of sexism has the potential to assist social scientists and policy\nmakers in studying and countering sexism better. The existing work on sexism\nclassification, which is different from sexism detection, has certain\nlimitations in terms of the categories of sexism used and/or whether they can\nco-occur. To the best of our knowledge, this is the first work on the\nmulti-label classification of sexism of any kind(s), and we contribute the\nlargest dataset for sexism categorization. We develop a neural solution for\nthis multi-label classification that can combine sentence representations\nobtained using models such as BERT with distributional and linguistic word\nembeddings using a flexible, hierarchical architecture involving recurrent\ncomponents and optional convolutional ones. Further, we leverage unlabeled\naccounts of sexism to infuse domain-specific elements into our framework. The\nbest proposed method outperforms several deep learning as well as traditional\nmachine learning baselines by an appreciable margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04659v2",
            "date": "2019-10-10 15:44:29+00:00",
            "title": "Multilingual Question Answering from Formatted Text applied to Conversational Agents",
            "authors": [
                "Wissam Siblini",
                "Charlotte Pasqual",
                "Axel Lavielle",
                "Mohamed Challal",
                "Cyril Cauchois"
            ],
            "abstract": "Recent advances with language models (e.g. BERT, XLNet, ...), have allowed\nsurpassing human performance on complex NLP tasks such as Reading\nComprehension. However, labeled datasets for training are available mostly in\nEnglish which makes it difficult to acknowledge progress in other languages.\nFortunately, models are now pre-trained on unlabeled data from hundreds of\nlanguages and exhibit interesting transfer abilities from one language to\nanother. In this paper, we show that multilingual BERT is naturally capable of\nzero-shot transfer for an extractive Question Answering task (eQA) from English\nto other languages. More specifically, it outperforms the best previously known\nbaseline for transfer to Japanese and French. Moreover, using a recently\npublished large eQA French dataset, we are able to further show that (1)\nzero-shot transfer provides results really close to a direct training on the\ntarget language and (2) combination of transfer and training on target is the\nbest option overall. We finally present a practical application: a multilingual\nconversational agent called Kate which answers to HR-related questions in\nseveral languages directly from the content of intranet pages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04731v1",
            "date": "2019-10-10 17:43:31+00:00",
            "title": "Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Karin Sevegnani",
                "Ioannis Konstas",
                "Verena Rieser"
            ],
            "abstract": "We present a recurrent neural network based system for automatic quality\nestimation of natural language generation (NLG) outputs, which jointly learns\nto assign numerical ratings to individual outputs and to provide pairwise\nrankings of two different outputs. The latter is trained using pairwise hinge\nloss over scores from two copies of the rating network.\n  We use learning to rank and synthetic data to improve the quality of ratings\nassigned by our system: we synthesise training pairs of distorted system\noutputs and train the system to rank the less distorted one higher. This leads\nto a 12% increase in correlation with human ratings over the previous\nbenchmark. We also establish the state of the art on the dataset of relative\nrankings from the E2E NLG Challenge (Du\\v{s}ek et al., 2019), where synthetic\ndata lead to a 4% accuracy increase over the base model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04980v3",
            "date": "2019-10-11 05:39:08+00:00",
            "title": "Conversational Transfer Learning for Emotion Recognition",
            "authors": [
                "Devamanyu Hazarika",
                "Soujanya Poria",
                "Roger Zimmermann",
                "Rada Mihalcea"
            ],
            "abstract": "Recognizing emotions in conversations is a challenging task due to the\npresence of contextual dependencies governed by self- and inter-personal\ninfluences. Recent approaches have focused on modeling these dependencies\nprimarily via supervised learning. However, purely supervised strategies demand\nlarge amounts of annotated data, which is lacking in most of the available\ncorpora in this task. To tackle this challenge, we look at transfer learning\napproaches as a viable alternative. Given the large amount of available\nconversational data, we investigate whether generative conversational models\ncan be leveraged to transfer affective knowledge for detecting emotions in\ncontext. We propose an approach, TL-ERC, where we pre-train a hierarchical\ndialogue model on multi-turn conversations (source) and then transfer its\nparameters to a conversational emotion classifier (target). In addition to the\npopular practice of using pre-trained sentence encoders, our approach also\nincorporates recurrent parameters that model inter-sentential context across\nthe whole conversation. Based on this idea, we perform several experiments\nacross multiple datasets and find improvement in performance and robustness\nagainst limited training data. TL-ERC also achieves better validation\nperformances in significantly fewer epochs. Overall, we infer that knowledge\nacquired from dialogue generators can indeed help recognize emotions in\nconversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05032v1",
            "date": "2019-10-11 08:56:52+00:00",
            "title": "Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction",
            "authors": [
                "Deli Chen",
                "Shuming ma",
                "Keiko Harimoto",
                "Ruihan Bao",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Incorporating related text information has proven successful in stock market\nprediction. However, it is a huge challenge to utilize texts in the enormous\nforex (foreign currency exchange) market because the associated texts are too\nredundant. In this work, we propose a BERT-based Hierarchical Aggregation Model\nto summarize a large amount of finance news to predict forex movement. We\nfirstly group news from different aspects: time, topic and category. Then we\nextract the most crucial news in each group by the SOTA extractive\nsummarization method. Finally, we conduct interaction between the news and the\ntrade data with attention to predict the forex movement. The experimental\nresults show that the category based method performs best among three grouping\nmethods and outperforms all the baselines. Besides, we study the influence of\nessential news attributes (category and region) by statistical analysis and\nsummarize the influence patterns for different currency pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05069v1",
            "date": "2019-10-11 10:40:42+00:00",
            "title": "Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base",
            "authors": [
                "Tao Shen",
                "Xiubo Geng",
                "Tao Qin",
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Guodong Long",
                "Daxin Jiang"
            ],
            "abstract": "We consider the problem of conversational question answering over a\nlarge-scale knowledge base. To handle huge entity vocabulary of a large-scale\nknowledge base, recent neural semantic parsing based approaches usually\ndecompose the task into several subtasks and then solve them sequentially,\nwhich leads to following issues: 1) errors in earlier subtasks will be\npropagated and negatively affect downstream ones; and 2) each subtask cannot\nnaturally share supervision signals with others. To tackle these issues, we\npropose an innovative multi-task learning framework where a pointer-equipped\nsemantic parsing model is designed to resolve coreference in conversations, and\nnaturally empower joint learning with a novel type-aware entity detection\nmodel. The proposed framework thus enables shared supervisions and alleviates\nthe effect of error propagation. Experiments on a large-scale conversational\nquestion answering dataset containing 1.6M question answering pairs over 12.8M\nentities show that the proposed framework improves overall F1 score from 67% to\n79% compared with previous state-of-the-art work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05154v1",
            "date": "2019-10-11 13:04:31+00:00",
            "title": "How Does Language Influence Documentation Workflow? Unsupervised Word Discovery Using Translations in Multiple Languages",
            "authors": [
                "Marcely Zanon Boito",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "For language documentation initiatives, transcription is an expensive\nresource: one minute of audio is estimated to take one hour and a half on\naverage of a linguist's work (Austin and Sallabank, 2013). Recently, collecting\naligned translations in well-resourced languages became a popular solution for\nensuring posterior interpretability of the recordings (Adda et al. 2016). In\nthis paper we investigate language-related impact in automatic approaches for\ncomputational language documentation. We translate the bilingual Mboshi-French\nparallel corpus (Godard et al. 2017) into four other languages, and we perform\nbilingual-rooted unsupervised word discovery. Our results hint towards an\nimpact of the well-resourced language in the quality of the output. However, by\ncombining the information learned by different bilingual models, we are only\nable to marginally increase the quality of the segmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05298v1",
            "date": "2019-10-11 16:43:05+00:00",
            "title": "Neural Generation for Czech: Data and Baselines",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Filip Jur\u010d\u00ed\u010dek"
            ],
            "abstract": "We present the first dataset targeted at end-to-end NLG in Czech in the\nrestaurant domain, along with several strong baseline models using the\nsequence-to-sequence approach. While non-English NLG is under-explored in\ngeneral, Czech, as a morphologically rich language, makes the task even harder:\nSince Czech requires inflecting named entities, delexicalization or copy\nmechanisms do not work out-of-the-box and lexicalizing the generated outputs is\nnon-trivial.\n  In our experiments, we present two different approaches to this this problem:\n(1) using a neural language model to select the correct inflected form while\nlexicalizing, (2) a two-step generation setup: our sequence-to-sequence model\ngenerates an interleaved sequence of lemmas and morphological tags, which are\nthen inflected by a morphological generator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05456v1",
            "date": "2019-10-12 01:10:24+00:00",
            "title": "Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge",
            "authors": [
                "Katharina Kann"
            ],
            "abstract": "How does knowledge of one language's morphology influence learning of\ninflection rules in a second one? In order to investigate this question in\nartificial neural network models, we perform experiments with a\nsequence-to-sequence architecture, which we train on different combinations of\neight source and three target languages. A detailed analysis of the model\noutputs suggests the following conclusions: (i) if source and target language\nare closely related, acquisition of the target language's inflectional\nmorphology constitutes an easier task for the model; (ii) knowledge of a\nprefixing (resp. suffixing) language makes acquisition of a suffixing (resp.\nprefixing) language's morphology more challenging; and (iii) surprisingly, a\nsource language which exhibits an agglutinative morphology simplifies learning\nof a second language's inflectional morphology, independent of their\nrelatedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05479v1",
            "date": "2019-10-12 03:44:56+00:00",
            "title": "Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations",
            "authors": [
                "Ke Tran",
                "Arianna Bisazza"
            ],
            "abstract": "We investigate whether off-the-shelf deep bidirectional sentence\nrepresentations trained on a massively multilingual corpus (multilingual BERT)\nenable the development of an unsupervised universal dependency parser. This\napproach only leverages a mix of monolingual corpora in many languages and does\nnot require any translation data making it applicable to low-resource\nlanguages. In our experiments we outperform the best CoNLL 2018\nlanguage-specific systems in all of the shared task's six truly low-resource\nlanguages while using a single system. However, we also find that (i) parsing\naccuracy still varies dramatically when changing the training languages and\n(ii) in some target languages zero-shot transfer fails under all tested\nconditions, raising concerns on the 'universality' of the whole approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05535v1",
            "date": "2019-10-12 09:23:11+00:00",
            "title": "From the Paft to the Fiiture: a Fully Automatic NMT and Word Embeddings Method for OCR Post-Correction",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Simon Hengchen"
            ],
            "abstract": "A great deal of historical corpora suffer from errors introduced by the OCR\n(optical character recognition) methods used in the digitization process.\nCorrecting these errors manually is a time-consuming process and a great part\nof the automatic approaches have been relying on rules or supervised machine\nlearning. We present a fully automatic unsupervised way of extracting parallel\ndata for training a character-based sequence-to-sequence NMT (neural machine\ntranslation) model to conduct OCR error correction.",
            "journal ref": "Proceedings of Recent Advances in Natural Language Processing.\n  Angelova, G., Mitkov, R., Nikolova, I. & Temnikova, I. (eds.). Shoumen:\n  INCOMA, p. 432-437 6 p (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05598v1",
            "date": "2019-10-12 17:06:07+00:00",
            "title": "SmokEng: Towards Fine-grained Classification of Tobacco-related Social Media Text",
            "authors": [
                "Kartikey Pant",
                "Venkata Himakar Yanamandra",
                "Alok Debnath",
                "Radhika Mamidi"
            ],
            "abstract": "Contemporary datasets on tobacco consumption focus on one of two topics,\neither public health mentions and disease surveillance, or sentiment analysis\non topical tobacco products and services. However, two primary considerations\nare not accounted for, the language of the demographic affected and a\ncombination of the topics mentioned above in a fine-grained classification\nmechanism. In this paper, we create a dataset of 3144 tweets, which are\nselected based on the presence of colloquial slang related to smoking and\nanalyze it based on the semantics of the tweet. Each class is created and\nannotated based on the content of the tweets such that further hierarchical\nmethods can be easily applied.\n  Further, we prove the efficacy of standard text classification methods on\nthis dataset, by designing experiments which do both binary as well as\nmulti-class classification. Our experiments tackle the identification of either\na specific topic (such as tobacco product promotion), a general mention\n(cigarettes and related products) or a more fine-grained classification. This\nmethodology paves the way for further analysis, such as understanding sentiment\nor style, which makes this dataset a vital contribution to both disease\nsurveillance and tobacco use research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05608v1",
            "date": "2019-10-12 17:46:16+00:00",
            "title": "VAIS Hate Speech Detection System: A Deep Learning based Approach for System Combination",
            "authors": [
                "Thai Binh Nguyen",
                "Quang Minh Nguyen",
                "Thu Hien Nguyen",
                "Ngoc Phuong Pham",
                "The Loc Nguyen",
                "Quoc Truong Do"
            ],
            "abstract": "Nowadays, Social network sites (SNSs) such as Facebook, Twitter are common\nplaces where people show their opinions, sentiments and share information with\nothers. However, some people use SNSs to post abuse and harassment threats in\norder to prevent other SNSs users from expressing themselves as well as seeking\ndifferent opinions. To deal with this problem, SNSs have to use a lot of\nresources including people to clean the aforementioned content. In this paper,\nwe propose a supervised learning model based on the ensemble method to solve\nthe problem of detecting hate content on SNSs in order to make conversations on\nSNSs more effective. Our proposed model got the first place for public\ndashboard with 0.730 F1 macro-score and the third place with 0.584 F1\nmacro-score for private dashboard at the sixth international workshop on\nVietnamese Language and Speech Processing 2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05752v1",
            "date": "2019-10-13 13:54:31+00:00",
            "title": "VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning",
            "authors": [
                "Ziqi Zhang",
                "Yaya Shi",
                "Jiutong Wei",
                "Chunfeng Yuan",
                "Bing Li",
                "Weiming Hu"
            ],
            "abstract": "Multi-modal information is essential to describe what has happened in a\nvideo. In this work, we represent videos by various appearance, motion and\naudio information guided with video topic. By following multi-stage training\nstrategy, our experiments show steady and significant improvement on the VATEX\nbenchmark. This report presents an overview and comparative analysis of our\nsystem designed for both Chinese and English tracks on VATEX Captioning\nChallenge 2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05786v2",
            "date": "2019-10-13 16:54:21+00:00",
            "title": "Progress Notes Classification and Keyword Extraction using Attention-based Deep Learning Models with BERT",
            "authors": [
                "Matthew Tang",
                "Priyanka Gandhi",
                "Md Ahsanul Kabir",
                "Christopher Zou",
                "Jordyn Blakey",
                "Xiao Luo"
            ],
            "abstract": "Various deep learning algorithms have been developed to analyze different\ntypes of clinical data including clinical text classification and extracting\ninformation from 'free text' and so on. However, automate the keyword\nextraction from the clinical notes is still challenging. The challenges include\ndealing with noisy clinical notes which contain various abbreviations, possible\ntypos, and unstructured sentences. The objective of this research is to\ninvestigate the attention-based deep learning models to classify the\nde-identified clinical progress notes extracted from a real-world EHR system.\nThe attention-based deep learning models can be used to interpret the models\nand understand the critical words that drive the correct or incorrect\nclassification of the clinical progress notes. The attention-based models in\nthis research are capable of presenting the human interpretable text\nclassification models. The results show that the fine-tuned BERT with the\nattention layer can achieve a high classification accuracy of 97.6%, which is\nhigher than the baseline fine-tuned BERT classification model. In this\nresearch, we also demonstrate that the attention-based models can identify\nrelevant keywords that are strongly related to the clinical progress note\ncategories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06204v1",
            "date": "2019-10-14 15:20:30+00:00",
            "title": "Estimating post-editing effort: a study on human judgements, task-based and reference-based metrics of MT quality",
            "authors": [
                "Carolina Scarton",
                "Mikel L. Forcada",
                "Miquel Espl\u00e0-Gomis",
                "Lucia Specia"
            ],
            "abstract": "Devising metrics to assess translation quality has always been at the core of\nmachine translation (MT) research. Traditional automatic reference-based\nmetrics, such as BLEU, have shown correlations with human judgements of\nadequacy and fluency and have been paramount for the advancement of MT system\ndevelopment. Crowd-sourcing has popularised and enabled the scalability of\nmetrics based on human judgements, such as subjective direct assessments (DA)\nof adequacy, that are believed to be more reliable than reference-based\nautomatic metrics. Finally, task-based measurements, such as post-editing time,\nare expected to provide a more detailed evaluation of the usefulness of\ntranslations for a specific task. Therefore, while DA averages adequacy\njudgements to obtain an appraisal of (perceived) quality independently of the\ntask, and reference-based automatic metrics try to objectively estimate quality\nalso in a task-independent way, task-based metrics are measurements obtained\neither during or after performing a specific task. In this paper we argue that,\nalthough expensive, task-based measurements are the most reliable when\nestimating MT quality in a specific task; in our case, this task is\npost-editing. To that end, we report experiments on a dataset with\nnewly-collected post-editing indicators and show their usefulness when\nestimating post-editing effort. Our results show that task-based metrics\ncomparing machine-translated and post-edited versions are the best at tracking\npost-editing effort, as expected. These metrics are followed by DA, and then by\nmetrics comparing the machine-translated version and independent references. We\nsuggest that MT practitioners should be aware of these differences and\nacknowledge their implications when deciding how to evaluate MT for\npost-editing purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06241v2",
            "date": "2019-10-14 16:19:20+00:00",
            "title": "Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment",
            "authors": [
                "Piotr Bojanowski",
                "Onur Celebi",
                "Tomas Mikolov",
                "Edouard Grave",
                "Armand Joulin"
            ],
            "abstract": "In this paper, we focus on the problem of adapting word vector-based models\nto new textual data. Given a model pre-trained on large reference data, how can\nwe adapt it to a smaller piece of data with a slightly different language\ndistribution? We frame the adaptation problem as a monolingual word vector\nalignment problem, and simply average models after alignment. We align vectors\nusing the RCSLS criterion. Our formulation results in a simple and efficient\nalgorithm that allows adapting general-purpose models to changing word\ndistributions. In our evaluation, we consider applications to word embedding\nand text classification models. We show that the proposed approach yields good\nperformance in all setups and outperforms a baseline consisting in fine-tuning\nthe model on new data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06411v1",
            "date": "2019-10-14 20:32:41+00:00",
            "title": "Mapping Supervised Bilingual Word Embeddings from English to low-resource languages",
            "authors": [
                "Sourav Dutta"
            ],
            "abstract": "It is very challenging to work with low-resource languages due to the\ninadequate availability of data. Using a dictionary to map independently\ntrained word embeddings into a shared vector space has proved to be very useful\nin learning bilingual embeddings in the past. Here we have tried to map\nindividual embeddings of words in English and their corresponding translated\nwords in low-resource languages like Estonian, Slovenian, Slovakian, and\nHungarian. We have used a supervised learning approach. We report accuracy\nscores through various retrieval strategies which show that it is possible to\napproach challenging tasks in Natural Language Processing like machine\ntranslation for such languages, provided that we have at least some amount of\nproper bilingual data. We also conclude that we can follow an unsupervised\nlearning path on monolingual text data as that is more suitable for\nlow-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06431v1",
            "date": "2019-10-14 21:32:38+00:00",
            "title": "Whatcha lookin' at? DeepLIFTing BERT's Attention in Question Answering",
            "authors": [
                "Ekaterina Arkhangelskaia",
                "Sourav Dutta"
            ],
            "abstract": "There has been great success recently in tackling challenging NLP tasks by\nneural networks which have been pre-trained and fine-tuned on large amounts of\ntask data. In this paper, we investigate one such model, BERT for\nquestion-answering, with the aim to analyze why it is able to achieve\nsignificantly better results than other models. We run DeepLIFT on the model\npredictions and test the outcomes to monitor shift in the attention values for\ninput. We also cluster the results to analyze any possible patterns similar to\nhuman reasoning depending on the kind of input paragraph and question the model\nis trying to answer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06558v1",
            "date": "2019-10-15 06:54:28+00:00",
            "title": "Detecting Machine-Translated Text using Back Translation",
            "authors": [
                "Hoang-Quoc Nguyen-Son",
                "Tran Phuong Thao",
                "Seira Hidano",
                "Shinsaku Kiyomoto"
            ],
            "abstract": "Machine-translated text plays a crucial role in the communication of people\nusing different languages. However, adversaries can use such text for malicious\npurposes such as plagiarism and fake review. The existing methods detected a\nmachine-translated text only using the text's intrinsic content, but they are\nunsuitable for classifying the machine-translated and human-written texts with\nthe same meanings. We have proposed a method to extract features used to\ndistinguish machine/human text based on the similarity between the intrinsic\ntext and its back-translation. The evaluation of detecting translated sentences\nwith French shows that our method achieves 75.0% of both accuracy and F-score.\nIt outperforms the existing methods whose the best accuracy is 62.8% and the\nF-score is 62.7%. The proposed method even detects more efficiently the\nback-translated text with 83.4% of accuracy, which is higher than 66.7% of the\nbest previous accuracy. We also achieve similar results not only with F-score\nbut also with similar experiments related to Japanese. Moreover, we prove that\nour detector can recognize both machine-translated and machine-back-translated\ntexts without the language information which is used to generate these machine\ntexts. It demonstrates the persistence of our method in various applications in\nboth low- and rich-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06571v1",
            "date": "2019-10-15 07:44:54+00:00",
            "title": "Text2Math: End-to-end Parsing Text into Math Expressions",
            "authors": [
                "Yanyan Zou",
                "Wei Lu"
            ],
            "abstract": "We propose Text2Math, a model for semantically parsing text into math\nexpressions. The model can be used to solve different math related problems\nincluding arithmetic word problems and equation parsing problems. Unlike\nprevious approaches, we tackle the problem from an end-to-end structured\nprediction perspective where our algorithm aims to predict the complete math\nexpression at once as a tree structure, where minimal manual efforts are\ninvolved in the process. Empirical results on benchmark datasets demonstrate\nthe efficacy of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06575v1",
            "date": "2019-10-15 07:54:12+00:00",
            "title": "Aligning Cross-Lingual Entities with Multi-Aspect Information",
            "authors": [
                "Hsiu-Wei Yang",
                "Yanyan Zou",
                "Peng Shi",
                "Wei Lu",
                "Jimmy Lin",
                "Xu Sun"
            ],
            "abstract": "Multilingual knowledge graphs (KGs), such as YAGO and DBpedia, represent\nentities in different languages. The task of cross-lingual entity alignment is\nto match entities in a source language with their counterparts in target\nlanguages. In this work, we investigate embedding-based approaches to encode\nentities from multilingual KGs into the same vector space, where equivalent\nentities are close to each other. Specifically, we apply graph convolutional\nnetworks (GCNs) to combine multi-aspect information of entities, including\ntopological connections, relations, and attributes of entities, to learn entity\nembeddings. To exploit the literal descriptions of entities expressed in\ndifferent languages, we propose two uses of a pretrained multilingual BERT\nmodel to bridge cross-lingual gaps. We further propose two strategies to\nintegrate GCN-based and BERT-based modules to boost performance. Extensive\nexperiments on two benchmark datasets demonstrate that our method significantly\noutperforms existing systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06700v1",
            "date": "2019-10-01 09:23:35+00:00",
            "title": "Robust Semantic Parsing with Adversarial Learning for Domain Generalization",
            "authors": [
                "Gabriel Marzinotto",
                "Geraldine Damnati",
                "Fr\u00e9d\u00e9ric B\u00e9chet",
                "Benoit Favre"
            ],
            "abstract": "This paper addresses the issue of generalization for Semantic Parsing in an\nadversarial framework. Building models that are more robust to inter-document\nvariability is crucial for the integration of Semantic Parsing technologies in\nreal applications. The underlying question throughout this study is whether\nadversarial learning can be used to train models on a higher level of\nabstraction in order to increase their robustness to lexical and stylistic\nvariations.We propose to perform Semantic Parsing with a domain classification\nadversarial task without explicit knowledge of the domain. The strategy is\nfirst evaluated on a French corpus of encyclopedic documents, annotated with\nFrameNet, in an information retrieval perspective, then on PropBank Semantic\nRole Labeling task on the CoNLL-2005 benchmark. We show that adversarial\nlearning increases all models generalization capabilities both on in and\nout-of-domain data.",
            "journal ref": "Proceedings of the 2019 Conference of the North, Jun 2019,\n  Minneapolis - Minnesota, France. pp.166-173",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06701v1",
            "date": "2019-10-15 13:09:06+00:00",
            "title": "NumNet: Machine Reading Comprehension with Numerical Reasoning",
            "authors": [
                "Qiu Ran",
                "Yankai Lin",
                "Peng Li",
                "Jie Zhou",
                "Zhiyuan Liu"
            ],
            "abstract": "Numerical reasoning, such as addition, subtraction, sorting and counting is a\ncritical skill in human's reading comprehension, which has not been well\nconsidered in existing machine reading comprehension (MRC) systems. To address\nthis issue, we propose a numerical MRC model named as NumNet, which utilizes a\nnumerically-aware graph neural network to consider the comparing information\nand performs numerical reasoning over numbers in the question and passage. Our\nsystem achieves an EM-score of 64.56% on the DROP dataset, outperforming all\nexisting machine reading comprehension models by considering the numerical\nrelations among numbers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06753v2",
            "date": "2019-10-15 13:54:44+00:00",
            "title": "On the Importance of Word Boundaries in Character-level Neural Machine Translation",
            "authors": [
                "Duygu Ataman",
                "Orhan Firat",
                "Mattia A. Di Gangi",
                "Marcello Federico",
                "Alexandra Birch"
            ],
            "abstract": "Neural Machine Translation (NMT) models generally perform translation using a\nfixed-size lexical vocabulary, which is an important bottleneck on their\ngeneralization capability and overall translation quality. The standard\napproach to overcome this limitation is to segment words into subword units,\ntypically using some external tools with arbitrary heuristics, resulting in\nvocabulary units not optimized for the translation task. Recent studies have\nshown that the same approach can be extended to perform NMT directly at the\nlevel of characters, which can deliver translation accuracy on-par with\nsubword-based models, on the other hand, this requires relatively deeper\nnetworks. In this paper, we propose a more computationally-efficient solution\nfor character-level NMT which implements a hierarchical decoding architecture\nwhere translations are subsequently generated at the level of words and\ncharacters. We evaluate different methods for open-vocabulary NMT in the\nmachine translation task from English into five languages with distinct\nmorphological typology, and show that the hierarchical decoding model can reach\nhigher translation accuracy than the subword-level NMT model using\nsignificantly fewer parameters, while demonstrating better capacity in learning\nlonger-distance contextual and grammatical dependencies than the standard\ncharacter-level NMT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06848v1",
            "date": "2019-10-15 15:10:16+00:00",
            "title": "Facebook AI's WAT19 Myanmar-English Translation Task Submission",
            "authors": [
                "Peng-Jen Chen",
                "Jiajun Shen",
                "Matt Le",
                "Vishrav Chaudhary",
                "Ahmed El-Kishky",
                "Guillaume Wenzek",
                "Myle Ott",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "This paper describes Facebook AI's submission to the WAT 2019 Myanmar-English\ntranslation task. Our baseline systems are BPE-based transformer models. We\nexplore methods to leverage monolingual data to improve generalization,\nincluding self-training, back-translation and their combination. We further\nimprove results by using noisy channel re-ranking and ensembling. We\ndemonstrate that these techniques can significantly improve not only a system\ntrained with additional monolingual data, but even the baseline system trained\nexclusively on the provided small parallel dataset. Our system ranks first in\nboth directions according to human evaluation and BLEU, with a gain of over 8\nBLEU points above the second best system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07000v1",
            "date": "2019-10-15 18:44:47+00:00",
            "title": "Answering Complex Open-domain Questions Through Iterative Query Generation",
            "authors": [
                "Peng Qi",
                "Xiaowen Lin",
                "Leo Mehr",
                "Zijian Wang",
                "Christopher D. Manning"
            ],
            "abstract": "It is challenging for current one-step retrieve-and-read question answering\n(QA) systems to answer questions like \"Which novel by the author of 'Armada'\nwill be adapted as a feature film by Steven Spielberg?\" because the question\nseldom contains retrievable clues about the missing entity (here, the author).\nAnswering such a question requires multi-hop reasoning where one must gather\ninformation about the missing entity (or facts) to proceed with further\nreasoning. We present GoldEn (Gold Entity) Retriever, which iterates between\nreading context and retrieving more supporting documents to answer open-domain\nmulti-hop questions. Instead of using opaque and computationally expensive\nneural retrieval models, GoldEn Retriever generates natural language search\nqueries given the question and available context, and leverages off-the-shelf\ninformation retrieval systems to query for missing entities. This allows GoldEn\nRetriever to scale up efficiently for open-domain multi-hop reasoning while\nmaintaining interpretability. We evaluate GoldEn Retriever on the recently\nproposed open-domain multi-hop QA dataset, HotpotQA, and demonstrate that it\noutperforms the best previously published model despite not using pretrained\nlanguage models such as BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07060v1",
            "date": "2019-10-15 21:28:13+00:00",
            "title": "Iterative Delexicalization for Improved Spoken Language Understanding",
            "authors": [
                "Avik Ray",
                "Yilin Shen",
                "Hongxia Jin"
            ],
            "abstract": "Recurrent neural network (RNN) based joint intent classification and slot\ntagging models have achieved tremendous success in recent years for building\nspoken language understanding and dialog systems. However, these models suffer\nfrom poor performance for slots which often encounter large semantic\nvariability in slot values after deployment (e.g. message texts, partial\nmovie/artist names). While greedy delexicalization of slots in the input\nutterance via substring matching can partly improve performance, it often\nproduces incorrect input. Moreover, such techniques cannot delexicalize slots\nwith out-of-vocabulary slot values not seen at training. In this paper, we\npropose a novel iterative delexicalization algorithm, which can accurately\ndelexicalize the input, even with out-of-vocabulary slot values. Based on model\nconfidence of the current delexicalized input, our algorithm improves\ndelexicalization in every iteration to converge to the best input having the\nhighest confidence. We show on benchmark and in-house datasets that our\nalgorithm can greatly improve parsing performance for RNN based models,\nespecially for out-of-distribution slot values.",
            "journal ref": "Proc. Interspeech 2019 (2019): 1183-1187",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07124v1",
            "date": "2019-10-16 01:27:40+00:00",
            "title": "FewRel 2.0: Towards More Challenging Few-Shot Relation Classification",
            "authors": [
                "Tianyu Gao",
                "Xu Han",
                "Hao Zhu",
                "Zhiyuan Liu",
                "Peng Li",
                "Maosong Sun",
                "Jie Zhou"
            ],
            "abstract": "We present FewRel 2.0, a more challenging task to investigate two aspects of\nfew-shot relation classification models: (1) Can they adapt to a new domain\nwith only a handful of instances? (2) Can they detect none-of-the-above (NOTA)\nrelations? To construct FewRel 2.0, we build upon the FewRel dataset (Han et\nal., 2018) by adding a new test set in a quite different domain, and a NOTA\nrelation choice. With the new dataset and extensive experimental analysis, we\nfound (1) that the state-of-the-art few-shot relation classification models\nstruggle on these two aspects, and (2) that the commonly-used techniques for\ndomain adaptation and NOTA detection still cannot handle the two challenges\nwell. Our research calls for more attention and further efforts to these two\nreal-world issues. All details and resources about the dataset and baselines\nare released at https: //github.com/thunlp/fewrel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07134v1",
            "date": "2019-10-16 02:22:07+00:00",
            "title": "Efficiency through Auto-Sizing: Notre Dame NLP's Submission to the WNGT 2019 Efficiency Task",
            "authors": [
                "Kenton Murray",
                "Brian DuSell",
                "David Chiang"
            ],
            "abstract": "This paper describes the Notre Dame Natural Language Processing Group's\n(NDNLP) submission to the WNGT 2019 shared task (Hayashi et al., 2019). We\ninvestigated the impact of auto-sizing (Murray and Chiang, 2015; Murray et al.,\n2019) to the Transformer network (Vaswani et al., 2017) with the goal of\nsubstantially reducing the number of parameters in the model. Our method was\nable to eliminate more than 25% of the model's parameters while suffering a\ndecrease of only 1.1 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07154v1",
            "date": "2019-10-16 03:34:34+00:00",
            "title": "Unsupervised Question Answering for Fact-Checking",
            "authors": [
                "Mayank Jobanputra"
            ],
            "abstract": "Recent Deep Learning (DL) models have succeeded in achieving human-level\naccuracy on various natural language tasks such as question-answering, natural\nlanguage inference (NLI), and textual entailment. These tasks not only require\nthe contextual knowledge but also the reasoning abilities to be solved\nefficiently. In this paper, we propose an unsupervised question-answering based\napproach for a similar task, fact-checking. We transform the FEVER dataset into\na Cloze-task by masking named entities provided in the claims. To predict the\nanswer token, we utilize pre-trained Bidirectional Encoder Representations from\nTransformers (BERT). The classifier computes label based on the correctly\nanswered questions and a threshold. Currently, the classifier is able to\nclassify the claims as \"SUPPORTS\" and \"MANUAL_REVIEW\". This approach achieves a\nlabel accuracy of 80.2% on the development set and 80.25% on the test set of\nthe transformed dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07179v5",
            "date": "2019-10-16 05:50:35+00:00",
            "title": "Content Enhanced BERT-based Text-to-SQL Generation",
            "authors": [
                "Tong Guo",
                "Huilin Gao"
            ],
            "abstract": "We present a simple methods to leverage the table content for the BERT-based\nmodel to solve the text-to-SQL problem. Based on the observation that some of\nthe table content match some words in question string and some of the table\nheader also match some words in question string, we encode two addition feature\nvector for the deep model. Our methods also benefit the model inference in\ntesting time as the tables are almost the same in training and testing time. We\ntest our model on the WikiSQL dataset and outperform the BERT-based baseline by\n3.7% in logic form and 3.7% in execution accuracy and achieve state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07181v3",
            "date": "2019-10-16 06:14:42+00:00",
            "title": "BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Pretraining deep language models has led to large performance gains in NLP.\nDespite this success, Schick and Sch\\\"utze (2020) recently showed that these\nmodels struggle to understand rare words. For static word embeddings, this\nproblem has been addressed by separately learning representations for rare\nwords. In this work, we transfer this idea to pretrained language models: We\nintroduce BERTRAM, a powerful architecture based on BERT that is capable of\ninferring high-quality embeddings for rare words that are suitable as input\nrepresentations for deep language models. This is achieved by enabling the\nsurface form and contexts of a word to interact with each other in a deep\narchitecture. Integrating BERTRAM into BERT leads to large performance\nincreases due to improved representations of rare and medium frequency words on\nboth a rare word probing task and three downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07221v4",
            "date": "2019-10-16 08:59:31+00:00",
            "title": "Meemi: A Simple Method for Post-processing and Integrating Cross-lingual Word Embeddings",
            "authors": [
                "Yerai Doval",
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "Word embeddings have become a standard resource in the toolset of any Natural\nLanguage Processing practitioner. While monolingual word embeddings encode\ninformation about words in the context of a particular language, cross-lingual\nembeddings define a multilingual space where word embeddings from two or more\nlanguages are integrated together. Current state-of-the-art approaches learn\nthese embeddings by aligning two disjoint monolingual vector spaces through an\northogonal transformation which preserves the structure of the monolingual\ncounterparts. In this work, we propose to apply an additional transformation\nafter this initial alignment step, which aims to bring the vector\nrepresentations of a given word and its translations closer to their average.\nSince this additional transformation is non-orthogonal, it also affects the\nstructure of the monolingual spaces. We show that our approach both improves\nthe integration of the monolingual spaces as well as the quality of the\nmonolingual spaces themselves. Furthermore, because our transformation can be\napplied to an arbitrary number of languages, we are able to effectively obtain\na truly multilingual space. The resulting (monolingual and multilingual) spaces\nshow consistent gains over the current state-of-the-art in standard intrinsic\ntasks, namely dictionary induction and word similarity, as well as in extrinsic\ntasks such as cross-lingual hypernym discovery and cross-lingual natural\nlanguage inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07350v1",
            "date": "2019-10-16 13:56:27+00:00",
            "title": "Why can't memory networks read effectively?",
            "authors": [
                "Simon \u0160uster",
                "Madhumita Sushil",
                "Walter Daelemans"
            ],
            "abstract": "Memory networks have been a popular choice among neural architectures for\nmachine reading comprehension and question answering. While recent work\nrevealed that memory networks can't truly perform multi-hop reasoning, we show\nin the present paper that vanilla memory networks are ineffective even in\nsingle-hop reading comprehension. We analyze the reasons for this on two\ncloze-style datasets, one from the medical domain and another including\nchildren's fiction. We find that the output classification layer with\nentity-specific weights, and the aggregation of passage information with\nrelatively flat attention distributions are the most important contributors to\npoor results. We propose network adaptations that can serve as simple remedies.\nWe also find that the presence of unseen answers at test time can dramatically\naffect the reported results, so we suggest controlling for this factor during\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07357v1",
            "date": "2019-10-16 14:07:42+00:00",
            "title": "Generating Challenge Datasets for Task-Oriented Conversational Agents through Self-Play",
            "authors": [
                "Sourabh Majumdar",
                "Serra Sinem Tekiroglu",
                "Marco Guerini"
            ],
            "abstract": "End-to-end neural approaches are becoming increasingly common in\nconversational scenarios due to their promising performances when provided with\nsufficient amount of data. In this paper, we present a novel methodology to\naddress the interpretability of neural approaches in such scenarios by creating\nchallenge datasets using dialogue self-play over multiple tasks/intents.\nDialogue self-play allows generating large amount of synthetic data; by taking\nadvantage of the complete control over the generation process, we show how\nneural approaches can be evaluated in terms of unseen dialogue patterns. We\npropose several out-of-pattern test cases each of which introduces a natural\nand unexpected user utterance phenomenon. As a proof of concept, we built a\nsingle and a multiple memory network, and show that these two architectures\nhave diverse performances depending on the peculiar dialogue patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07370v1",
            "date": "2019-10-16 14:24:37+00:00",
            "title": "Evolution of transfer learning in natural language processing",
            "authors": [
                "Aditya Malte",
                "Pratik Ratadiya"
            ],
            "abstract": "In this paper, we present a study of the recent advancements which have\nhelped bring Transfer Learning to NLP through the use of semi-supervised\ntraining. We discuss cutting-edge methods and architectures such as BERT, GPT,\nELMo, ULMFit among others. Classically, tasks in natural language processing\nhave been performed through rule-based and statistical methodologies. However,\nowing to the vast nature of natural languages these methods do not generalise\nwell and failed to learn the nuances of language. Thus machine learning\nalgorithms such as Naive Bayes and decision trees coupled with traditional\nmodels such as Bag-of-Words and N-grams were used to usurp this problem.\nEventually, with the advent of advanced recurrent neural network architectures\nsuch as the LSTM, we were able to achieve state-of-the-art performance in\nseveral natural language processing tasks such as text classification and\nmachine translation. We talk about how Transfer Learning has brought about the\nwell-known ImageNet moment for NLP. Several advanced architectures such as the\nTransformer and its variants have allowed practitioners to leverage knowledge\ngained from unrelated task to drastically fasten convergence and provide better\nperformance on the target task. This survey represents an effort at providing a\nsuccinct yet complete understanding of the recent advances in natural language\nprocessing using deep learning in with a special focus on detailing transfer\nlearning and its potential advantages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07419v1",
            "date": "2019-10-15 16:38:55+00:00",
            "title": "Comprehend Medical: a Named Entity Recognition and Relationship Extraction Web Service",
            "authors": [
                "Parminder Bhatia",
                "Busra Celikkaya",
                "Mohammed Khalilia",
                "Selvan Senthivel"
            ],
            "abstract": "Comprehend Medical is a stateless and Health Insurance Portability and\nAccountability Act (HIPAA) eligible Named Entity Recognition (NER) and\nRelationship Extraction (RE) service launched under Amazon Web Services (AWS)\ntrained using state-of-the-art deep learning models. Contrary to many existing\nopen source tools, Comprehend Medical is scalable and does not require steep\nlearning curve, dependencies, pipeline configurations, or installations.\nCurrently, Comprehend Medical performs NER in five medical categories: Anatomy,\nMedical Condition, Medications, Protected Health Information (PHI) and\nTreatment, Test and Procedure (TTP). Additionally, the service provides\nrelationship extraction for the detected entities as well as contextual\ninformation such as negation and temporality in the form of traits. Comprehend\nMedical provides two Application Programming Interfaces (API): 1) the NERe API\nwhich returns all the extracted named entities, their traits and the\nrelationships between them and 2) the PHId API which returns just the protected\nhealth information contained in the text. Furthermore, Comprehend Medical is\naccessible through AWS Console, Java and Python Software Development Kit (SDK),\nmaking it easier for non-developers and developers to use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07429v1",
            "date": "2019-10-16 15:44:09+00:00",
            "title": "Bridging the Knowledge Gap: Enhancing Question Answering with World and Domain Knowledge",
            "authors": [
                "Travis R. Goodwin",
                "Dina Demner-Fushman"
            ],
            "abstract": "In this paper we present OSCAR (Ontology-based Semantic Composition Augmented\nRegularization), a method for injecting task-agnostic knowledge from an\nOntology or knowledge graph into a neural network during pretraining. We\nevaluated the impact of including OSCAR when pretraining BERT with Wikipedia\narticles by measuring the performance when fine-tuning on two question\nanswering tasks involving world knowledge and causal reasoning and one\nrequiring domain (healthcare) knowledge and obtained 33:3%, 18:6%, and 4%\nimproved accuracy compared to pretraining BERT without OSCAR and obtaining new\nstate-of-the-art results on two of the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07457v1",
            "date": "2019-10-16 16:27:26+00:00",
            "title": "Linguistic evaluation of German-English Machine Translation using a Test Suite",
            "authors": [
                "Eleftherios Avramidis",
                "Vivien Macketanz",
                "Ursula Strohriegel",
                "Hans Uszkoreit"
            ],
            "abstract": "We present the results of the application of a grammatical test suite for\nGerman$\\rightarrow$English MT on the systems submitted at WMT19, with a\ndetailed analysis for 107 phenomena organized in 14 categories. The systems\nstill translate wrong one out of four test items in average. Low performance is\nindicated for idioms, modals, pseudo-clefts, multi-word expressions and verb\nvalency. When compared to last year, there has been a improvement of function\nwords, non-verbal agreement and punctuation. More detailed conclusions about\nparticular systems and phenomena are also presented.",
            "journal ref": "Proceedings of the Fourth Conference on Machine Translation.\n  Conference on Machine Translation (WMT-2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07460v1",
            "date": "2019-10-16 16:36:07+00:00",
            "title": "Fine-grained evaluation of German-English Machine Translation based on a Test Suite",
            "authors": [
                "Vivien Macketanz",
                "Eleftherios Avramidis",
                "Aljoscha Burchardt",
                "Hans Uszkoreit"
            ],
            "abstract": "We present an analysis of 16 state-of-the-art MT systems on German-English\nbased on a linguistically-motivated test suite. The test suite has been devised\nmanually by a team of language professionals in order to cover a broad variety\nof linguistic phenomena that MT often fails to translate properly. It contains\n5,000 test sentences covering 106 linguistic phenomena in 14 categories, with\nan increased focus on verb tenses, aspects and moods. The MT outputs are\nevaluated in a semi-automatic way through regular expressions that focus only\non the part of the sentence that is relevant to each phenomenon. Through our\nanalysis, we are able to compare systems based on their performance on these\ncategories. Additionally, we reveal strengths and weaknesses of particular\nsystems and we identify grammatical phenomena where the overall performance of\nMT is relatively low.",
            "journal ref": "Proceedings of the Third Conference on Machine Translation\n  (WMT-2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07468v1",
            "date": "2019-10-16 16:49:28+00:00",
            "title": "Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically-motivated Test Suite",
            "authors": [
                "Avramidis Eleftherios",
                "Vivien Macketanz",
                "Arle Lommel",
                "Hans Uszkoreit"
            ],
            "abstract": "We present an alternative method of evaluating Quality Estimation systems,\nwhich is based on a linguistically-motivated Test Suite. We create a test-set\nconsisting of 14 linguistic error categories and we gather for each of them a\nset of samples with both correct and erroneous translations. Then, we measure\nthe performance of 5 Quality Estimation systems by checking their ability to\ndistinguish between the correct and the erroneous translations. The detailed\nresults are much more informative about the ability of each system. The fact\nthat different Quality Estimation systems perform differently at various\nphenomena confirms the usefulness of the Test Suite.",
            "journal ref": "Proceedings of the First Workshop on Translation Quality\n  Estimation and Automatic Post-Editing (QEAPE-2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07481v1",
            "date": "2019-10-16 17:15:35+00:00",
            "title": "Using Whole Document Context in Neural Machine Translation",
            "authors": [
                "Valentin Mac\u00e9",
                "Christophe Servan"
            ],
            "abstract": "In Machine Translation, considering the document as a whole can help to\nresolve ambiguities and inconsistencies. In this paper, we propose a simple yet\npromising approach to add contextual information in Neural Machine Translation.\nWe present a method to add source context that capture the whole document with\naccurate boundaries, taking every word into account. We provide this additional\ninformation to a Transformer model and study the impact of our method on three\nlanguage pairs. The proposed approach obtains promising results in the\nEnglish-German, English-French and French-English document-level translation\ntasks. We observe interesting cross-sentential behaviors where the model learns\nto use document-level information to improve translation coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07659v1",
            "date": "2019-10-17 00:20:11+00:00",
            "title": "Towards Annotating and Creating Sub-Sentence Summary Highlights",
            "authors": [
                "Kristjan Arumae",
                "Parminder Bhatia",
                "Fei Liu"
            ],
            "abstract": "Highlighting is a powerful tool to pick out important content and emphasize.\nCreating summary highlights at the sub-sentence level is particularly\ndesirable, because sub-sentences are more concise than whole sentences. They\nare also better suited than individual words and phrases that can potentially\nlead to disfluent, fragmented summaries. In this paper we seek to generate\nsummary highlights by annotating summary-worthy sub-sentences and teaching\nclassifiers to do the same. We frame the task as jointly selecting important\nsentences and identifying a single most informative textual unit from each\nsentence. This formulation dramatically reduces the task complexity involved in\nsentence compression. Our study provides new benchmarks and baselines for\ngenerating highlights at the sub-sentence level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07713v1",
            "date": "2019-10-17 05:19:29+00:00",
            "title": "BIG MOOD: Relating Transformers to Explicit Commonsense Knowledge",
            "authors": [
                "Jeff Da"
            ],
            "abstract": "We introduce a simple yet effective method of integrating contextual\nembeddings with commonsense graph embeddings, dubbed BERT Infused Graphs:\nMatching Over Other embeDdings. First, we introduce a preprocessing method to\nimprove the speed of querying knowledge bases. Then, we develop a method of\ncreating knowledge embeddings from each knowledge base. We introduce a method\nof aligning tokens between two misaligned tokenization methods. Finally, we\ncontribute a method of contextualizing BERT after combining with knowledge base\nembeddings. We also show BERTs tendency to correct lower accuracy question\ntypes. Our model achieves a higher accuracy than BERT, and we score fifth on\nthe official leaderboard of the shared task and score the highest without any\nadditional language model pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07834v1",
            "date": "2019-10-17 11:42:58+00:00",
            "title": "Using a KG-Copy Network for Non-Goal Oriented Dialogues",
            "authors": [
                "Debanjan Chaudhuri",
                "Md Rashad Al Hasan Rony",
                "Simon Jordan",
                "Jens Lehmann"
            ],
            "abstract": "Non-goal oriented, generative dialogue systems lack the ability to generate\nanswers with grounded facts. A knowledge graph can be considered an abstraction\nof the real world consisting of well-grounded facts. This paper addresses the\nproblem of generating well grounded responses by integrating knowledge graphs\ninto the dialogue systems response generation process, in an end-to-end manner.\nA dataset for nongoal oriented dialogues is proposed in this paper in the\ndomain of soccer, conversing on different clubs and national teams along with a\nknowledge graph for each of these teams. A novel neural network architecture is\nalso proposed as a baseline on this dataset, which can integrate knowledge\ngraphs into the response generation process, producing well articulated,\nknowledge grounded responses. Empirical evidence suggests that the proposed\nmodel performs better than other state-of-the-art models for knowledge graph\nintegrated dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07848v1",
            "date": "2019-10-17 12:09:11+00:00",
            "title": "Topical Keyphrase Extraction with Hierarchical Semantic Networks",
            "authors": [
                "Yoo yeon Sung",
                "Seoung Bum Kim"
            ],
            "abstract": "Topical keyphrase extraction is used to summarize large collections of text\ndocuments. However, traditional methods cannot properly reflect the intrinsic\nsemantics and relationships of keyphrases because they rely on a simple\nterm-frequency-based process. Consequently, these methods are not effective in\nobtaining significant contextual knowledge. To resolve this, we propose a\ntopical keyphrase extraction method based on a hierarchical semantic network\nand multiple centrality network measures that together reflect the hierarchical\nsemantics of keyphrases. We conduct experiments on real data to examine the\npracticality of the proposed method and to compare its performance with that of\nexisting topical keyphrase extraction methods. The results confirm that the\nproposed method outperforms state-of-the-art topical keyphrase extraction\nmethods in terms of the representativeness of the selected keyphrases for each\ntopic. The proposed method can effectively reflect intrinsic keyphrase\nsemantics and interrelationships.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07924v3",
            "date": "2019-10-17 14:01:57+00:00",
            "title": "LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognition",
            "authors": [
                "Benjamin Beilharz",
                "Xin Sun",
                "Sariya Karimova",
                "Stefan Riezler"
            ],
            "abstract": "We present a corpus of sentence-aligned triples of German audio, German text,\nand English translation, based on German audiobooks. The speech translation\ndata consist of 110 hours of audio material aligned to over 50k parallel\nsentences. An even larger dataset comprising 547 hours of German speech aligned\nto German text is available for speech recognition. The audio data is read\nspeech and thus low in disfluencies. The quality of audio and sentence\nalignments has been checked by a manual evaluation, showing that speech\nalignment quality is in general very high. The sentence alignment quality is\ncomparable to well-used parallel translation data and can be adjusted by\ncutoffs on the automatic alignment score. To our knowledge, this corpus is to\ndate the largest resource for German speech recognition and for end-to-end\nGerman-to-English speech translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07931v3",
            "date": "2019-10-17 14:09:42+00:00",
            "title": "PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable",
            "authors": [
                "Siqi Bao",
                "Huang He",
                "Fan Wang",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Pre-training models have been proved effective for a wide range of natural\nlanguage processing tasks. Inspired by this, we propose a novel dialogue\ngeneration pre-training framework to support various kinds of conversations,\nincluding chit-chat, knowledge grounded dialogues, and conversational question\nanswering. In this framework, we adopt flexible attention mechanisms to fully\nleverage the bi-directional context and the uni-directional characteristic of\nlanguage generation. We also introduce discrete latent variables to tackle the\ninherent one-to-many mapping problem in response generation. Two reciprocal\ntasks of response generation and latent act recognition are designed and\ncarried out simultaneously within a shared network. Comprehensive experiments\non three publicly available datasets verify the effectiveness and superiority\nof the proposed framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07938v1",
            "date": "2019-10-17 14:28:35+00:00",
            "title": "Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study",
            "authors": [
                "James Barry",
                "Joachim Wagner",
                "Jennifer Foster"
            ],
            "abstract": "Cross-lingual dependency parsing involves transferring syntactic knowledge\nfrom one language to another. It is a crucial component for inducing dependency\nparsers in low-resource scenarios where no training data for a language exists.\nUsing Faroese as the target language, we compare two approaches using\nannotation projection: first, projecting from multiple monolingual source\nmodels; second, projecting from a single polyglot model which is trained on the\ncombination of all source languages. Furthermore, we reproduce multi-source\nprojection (Tyers et al., 2018), in which dependency trees of multiple sources\nare combined. Finally, we apply multi-treebank modelling to the projected\ntreebanks, in addition to or alternatively to polyglot modelling on the source\nside. We find that polyglot training on the source languages produces an\noverall trend of better results on the target language but the single best\nresult for the target language is obtained by projecting from monolingual\nsource parsing models and then training multi-treebank POS tagging and parsing\nmodels on the target side.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08129v1",
            "date": "2019-10-17 19:45:18+00:00",
            "title": "Marpa, A practical general parser: the recognizer",
            "authors": [
                "Jeffrey Kegler"
            ],
            "abstract": "The Marpa recognizer is described. Marpa is a practical and fully implemented\nalgorithm for the recognition, parsing and evaluation of context-free grammars.\nThe Marpa recognizer is the first to unite the improvements to Earley's\nalgorithm found in Joop Leo's 1991 paper to those in Aycock and Horspool's 2002\npaper. Marpa tracks the full state of the parse, at it proceeds, in a form\nconvenient for the application. This greatly improves error detection and\nenables event-driven parsing. One such technique is \"Ruby Slippers\" parsing, in\nwhich the input is altered in response to the parser's expectations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08144v2",
            "date": "2019-10-17 20:18:23+00:00",
            "title": "Explainable Authorship Verification in Social Media via Attention-based Similarity Learning",
            "authors": [
                "Benedikt Boenninghoff",
                "Steffen Hessler",
                "Dorothea Kolossa",
                "Robert M. Nickel"
            ],
            "abstract": "Authorship verification is the task of analyzing the linguistic patterns of\ntwo or more texts to determine whether they were written by the same author or\nnot. The analysis is traditionally performed by experts who consider linguistic\nfeatures, which include spelling mistakes, grammatical inconsistencies, and\nstylistics for example. Machine learning algorithms, on the other hand, can be\ntrained to accomplish the same, but have traditionally relied on so-called\nstylometric features. The disadvantage of such features is that their\nreliability is greatly diminished for short and topically varied social media\ntexts. In this interdisciplinary work, we propose a substantial extension of a\nrecently published hierarchical Siamese neural network approach, with which it\nis feasible to learn neural features and to visualize the decision-making\nprocess. For this purpose, a new large-scale corpus of short Amazon reviews for\ntext comparison research is compiled and we show that the Siamese network\ntopologies outperform state-of-the-art approaches that were built up on\nstylometric features. Our linguistic analysis of the internal attention weights\nof the network shows that the proposed method is indeed able to latch on to\nsome traditional linguistic categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08192v1",
            "date": "2019-10-17 22:55:29+00:00",
            "title": "SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble",
            "authors": [
                "Jiaming Shen",
                "Zeqiu Wu",
                "Dongming Lei",
                "Jingbo Shang",
                "Xiang Ren",
                "Jiawei Han"
            ],
            "abstract": "Corpus-based set expansion (i.e., finding the \"complete\" set of entities\nbelonging to the same semantic class, based on a given corpus and a tiny set of\nseeds) is a critical task in knowledge discovery. It may facilitate numerous\ndownstream applications, such as information extraction, taxonomy induction,\nquestion answering, and web search. To discover new entities in an expanded\nset, previous approaches either make one-time entity ranking based on\ndistributional similarity, or resort to iterative pattern-based bootstrapping.\nThe core challenge for these methods is how to deal with noisy context features\nderived from free-text corpora, which may lead to entity intrusion and semantic\ndrifting. In this study, we propose a novel framework, SetExpan, which tackles\nthis problem, with two techniques: (1) a context feature selection method that\nselects clean context features for calculating entity-entity distributional\nsimilarity, and (2) a ranking-based unsupervised ensemble method for expanding\nentity set based on denoised context features. Experiments on three datasets\nshow that SetExpan is robust and outperforms previous state-of-the-art methods\nin terms of mean average precision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08381v1",
            "date": "2019-10-18 12:36:52+00:00",
            "title": "Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System",
            "authors": [
                "Ze Yang",
                "Linjun Shou",
                "Ming Gong",
                "Wutao Lin",
                "Daxin Jiang"
            ],
            "abstract": "Deep pre-training and fine-tuning models (such as BERT and OpenAI GPT) have\ndemonstrated excellent results in question answering areas. However, due to the\nsheer amount of model parameters, the inference speed of these models is very\nslow. How to apply these complex models to real business scenarios becomes a\nchallenging but practical problem. Previous model compression methods usually\nsuffer from information loss during the model compression procedure, leading to\ninferior models compared with the original one. To tackle this challenge, we\npropose a Two-stage Multi-teacher Knowledge Distillation (TMKD for short)\nmethod for web Question Answering system. We first develop a general Q\\&A\ndistillation task for student model pre-training, and further fine-tune this\npre-trained student model with multi-teacher knowledge distillation on\ndownstream tasks (like Web Q\\&A task, MNLI, SNLI, RTE tasks from GLUE), which\neffectively reduces the overfitting bias in individual teacher models, and\ntransfers more general knowledge to the student model. The experiment results\nshow that our method can significantly outperform the baseline methods and even\nachieve comparable results with the original teacher models, along with\nsubstantial speedup of model inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08418v1",
            "date": "2019-10-18 13:44:16+00:00",
            "title": "Controlling Utterance Length in NMT-based Word Segmentation with Attention",
            "authors": [
                "Pierre Godard",
                "Laurent Besacier",
                "Francois Yvon"
            ],
            "abstract": "One of the basic tasks of computational language documentation (CLD) is to\nidentify word boundaries in an unsegmented phonemic stream. While several\nunsupervised monolingual word segmentation algorithms exist in the literature,\nthey are challenged in real-world CLD settings by the small amount of available\ndata. A possible remedy is to take advantage of glosses or translation in a\nforeign, well-resourced, language, which often exist for such data. In this\npaper, we explore and compare ways to exploit neural machine translation models\nto perform unsupervised boundary detection with bilingual information, notably\nintroducing a new loss function for jointly learning alignment and\nsegmentation. We experiment with an actual under-resourced language, Mboshi,\nand show that these techniques can effectively control the output segmentation\nlength.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08435v1",
            "date": "2019-10-18 14:23:03+00:00",
            "title": "Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs",
            "authors": [
                "Angela Fan",
                "Claire Gardent",
                "Chloe Braud",
                "Antoine Bordes"
            ],
            "abstract": "Query-based open-domain NLP tasks require information synthesis from long and\ndiverse web results. Current approaches extractively select portions of web\ntext as input to Sequence-to-Sequence models using methods such as TF-IDF\nranking. We propose constructing a local graph structured knowledge base for\neach query, which compresses the web search information and reduces redundancy.\nWe show that by linearizing the graph into a structured input sequence, models\ncan encode the graph representations within a standard Sequence-to-Sequence\nsetting. For two generative tasks with very long text input, long-form question\nanswering and multi-document summarization, feeding graph representations as\ninput can achieve better performance than using retrieved text portions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08486v1",
            "date": "2019-10-18 16:11:31+00:00",
            "title": "Concept Pointer Network for Abstractive Summarization",
            "authors": [
                "Wang Wenbo",
                "Gao Yang",
                "Huang Heyan",
                "Zhou Yuxiang"
            ],
            "abstract": "A quality abstractive summary should not only copy salient source texts as\nsummaries but should also tend to generate new conceptual words to express\nconcrete details. Inspired by the popular pointer generator\nsequence-to-sequence model, this paper presents a concept pointer network for\nimproving these aspects of abstractive summarization. The network leverages\nknowledge-based, context-aware conceptualizations to derive an extended set of\ncandidate concepts. The model then points to the most appropriate choice using\nboth the concept set and original source text. This joint approach generates\nabstractive summaries with higher-level semantic concepts. The training model\nis also optimized in a way that adapts to different data, which is based on a\nnovel method of distantly-supervised learning guided by reference summaries and\ntesting set. Overall, the proposed approach provides statistically significant\nimprovements over several state-of-the-art models on both the DUC-2004 and\nGigaword datasets. A human evaluation of the model's abstractive abilities also\nsupports the quality of the summaries produced within this framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08592v1",
            "date": "2019-10-18 19:14:17+00:00",
            "title": "Automatic Post-Editing for Machine Translation",
            "authors": [
                "Rajen Chatterjee"
            ],
            "abstract": "Automatic Post-Editing (APE) aims to correct systematic errors in a machine\ntranslated text. This is primarily useful when the machine translation (MT)\nsystem is not accessible for improvement, leaving APE as a viable option to\nimprove translation quality as a downstream task - which is the focus of this\nthesis. This field has received less attention compared to MT due to several\nreasons, which include: the limited availability of data to perform a sound\nresearch, contrasting views reported by different researchers about the\neffectiveness of APE, and limited attention from the industry to use APE in\ncurrent production pipelines. In this thesis, we perform a thorough\ninvestigation of APE as a downstream task in order to: i) understand its\npotential to improve translation quality; ii) advance the core technology -\nstarting from classical methods to recent deep-learning based solutions; iii)\ncope with limited and sparse data; iv) better leverage multiple input sources;\nv) mitigate the task-specific problem of over-correction; vi) enhance neural\ndecoding to leverage external knowledge; and vii) establish an online learning\nframework to handle data diversity in real-time. All the above contributions\nare discussed across several chapters, and most of them are evaluated in the\nAPE shared task organized each year at the Conference on Machine Translation.\nOur efforts in improving the technology resulted in the best system at the 2017\nAPE shared task, and our work on online learning received a distinguished paper\naward at the Italian Conference on Computational Linguistics. Overall, outcomes\nand findings of our work have boost interest among researchers and attracted\nindustries to examine this technology to solve real-word problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08684v3",
            "date": "2019-10-19 03:00:46+00:00",
            "title": "Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation",
            "authors": [
                "Ran Tian",
                "Shashi Narayan",
                "Thibault Sellam",
                "Ankur P. Parikh"
            ],
            "abstract": "We address the issue of hallucination in data-to-text generation, i.e.,\nreducing the generation of text that is unsupported by the source. We\nconjecture that hallucination can be caused by an encoder-decoder model\ngenerating content phrases without attending to the source; so we propose a\nconfidence score to ensure that the model attends to the source whenever\nnecessary, as well as a variational Bayes training framework that can learn the\nscore from data. Experiments on the WikiBio (Lebretet al., 2016) dataset show\nthat our approach is more faithful to the source than existing state-of-the-art\napproaches, according to both PARENT score (Dhingra et al., 2019) and human\nevaluation. We also report strong results on the WebNLG (Gardent et al., 2017)\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08772v1",
            "date": "2019-10-19 13:45:30+00:00",
            "title": "MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity",
            "authors": [
                "Hai Hu",
                "Qi Chen",
                "Kyle Richardson",
                "Atreyee Mukherjee",
                "Lawrence S. Moss",
                "Sandra Kuebler"
            ],
            "abstract": "We present a new logic-based inference engine for natural language inference\n(NLI) called MonaLog, which is based on natural logic and the monotonicity\ncalculus. In contrast to existing logic-based approaches, our system is\nintentionally designed to be as lightweight as possible, and operates using a\nsmall set of well-known (surface-level) monotonicity facts about quantifiers,\nlexical items and tokenlevel polarity information. Despite its simplicity, we\nfind our approach to be competitive with other logic-based NLI models on the\nSICK benchmark. We also use MonaLog in combination with the current\nstate-of-the-art model BERT in a variety of settings, including for\ncompositional data augmentation. We show that MonaLog is capable of generating\nlarge amounts of high-quality training data for BERT, improving its accuracy on\nSICK.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08832v1",
            "date": "2019-10-19 20:05:44+00:00",
            "title": "Natural Question Generation with Reinforcement Learning Based Graph-to-Sequence Model",
            "authors": [
                "Yu Chen",
                "Lingfei Wu",
                "Mohammed J. Zaki"
            ],
            "abstract": "Natural question generation (QG) aims to generate questions from a passage\nand an answer. In this paper, we propose a novel reinforcement learning (RL)\nbased graph-to-sequence (Graph2Seq) model for QG. Our model consists of a\nGraph2Seq generator where a novel Bidirectional Gated Graph Neural Network is\nproposed to embed the passage, and a hybrid evaluator with a mixed objective\ncombining both cross-entropy and RL losses to ensure the generation of\nsyntactically and semantically valid text. The proposed model outperforms\nprevious state-of-the-art methods by a large margin on the SQuAD dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08840v1",
            "date": "2019-10-19 20:42:59+00:00",
            "title": "Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings",
            "authors": [
                "Dhruva Sahrawat",
                "Debanjan Mahata",
                "Mayank Kulkarni",
                "Haimin Zhang",
                "Rakesh Gosangi",
                "Amanda Stent",
                "Agniv Sharma",
                "Yaman Kumar",
                "Rajiv Ratn Shah",
                "Roger Zimmermann"
            ],
            "abstract": "In this paper, we formulate keyphrase extraction from scholarly articles as a\nsequence labeling task solved using a BiLSTM-CRF, where the words in the input\ntext are represented using deep contextualized embeddings. We evaluate the\nproposed architecture using both contextualized and fixed word embedding models\non three different benchmark datasets (Inspec, SemEval 2010, SemEval 2017) and\ncompare with existing popular unsupervised and supervised techniques. Our\nresults quantify the benefits of (a) using contextualized embeddings (e.g.\nBERT) over fixed word embeddings (e.g. Glove); (b) using a BiLSTM-CRF\narchitecture with contextualized word embeddings over fine-tuning the\ncontextualized word embedding model directly, and (c) using genre-specific\ncontextualized embeddings (SciBERT). Through error analysis, we also provide\nsome insights into why particular models work better than others. Lastly, we\npresent a case study where we analyze different self-attention layers of the\ntwo best models (BERT and SciBERT) to better understand the predictions made by\neach for the task of keyphrase extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08916v1",
            "date": "2019-10-20 07:12:04+00:00",
            "title": "PT-CoDE: Pre-trained Context-Dependent Encoder for Utterance-level Emotion Recognition",
            "authors": [
                "Wenxiang Jiao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Utterance-level emotion recognition (ULER) is a significant research topic\nfor understanding human behaviors and developing empathetic chatting machines\nin the artificial intelligence area. Unlike traditional text classification\nproblem, this task is supported by a limited number of datasets, among which\nmost contain inadequate conversations or speeches. Such a data scarcity issue\nlimits the possibility of training larger and more powerful models for this\ntask. Witnessing the success of transfer learning in natural language process\n(NLP), we propose to pre-train a context-dependent encoder (CoDE) for ULER by\nlearning from unlabeled conversation data. Essentially, CoDE is a hierarchical\narchitecture that contains an utterance encoder and a conversation encoder,\nmaking it different from those works that aim to pre-train a universal sentence\nencoder. Also, we propose a new pre-training task named \"conversation\ncompletion\" (CoCo), which attempts to select the correct answer from candidate\nanswers to fill a masked utterance in a question conversation. The CoCo task is\ncarried out on pure movie subtitles so that our CoDE can be pre-trained in an\nunsupervised fashion. Finally, the pre-trained CoDE (PT-CoDE) is fine-tuned for\nULER and boosts the model performance significantly on five datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09180v1",
            "date": "2019-10-21 07:26:07+00:00",
            "title": "Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistance",
            "authors": [
                "Takumi Ito",
                "Tatsuki Kuribayashi",
                "Hayato Kobayashi",
                "Ana Brassard",
                "Masato Hagiwara",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "The writing process consists of several stages such as drafting, revising,\nediting, and proofreading. Studies on writing assistance, such as grammatical\nerror correction (GEC), have mainly focused on sentence editing and\nproofreading, where surface-level issues such as typographical, spelling, or\ngrammatical errors should be corrected. We broaden this focus to include the\nearlier revising stage, where sentences require adjustment to the information\nincluded or major rewriting and propose Sentence-level Revision (SentRev) as a\nnew writing assistance task. Well-performing systems in this task can help\ninexperienced authors by producing fluent, complete sentences given their\nrough, incomplete drafts. We build a new freely available crowdsourced\nevaluation dataset consisting of incomplete sentences authored by non-native\nwriters paired with their final versions extracted from published academic\npapers for developing and evaluating SentRev models. We also establish baseline\nperformance on SentRev using our newly built evaluation dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09183v1",
            "date": "2019-10-21 07:35:46+00:00",
            "title": "Semantic Graph Convolutional Network for Implicit Discourse Relation Classification",
            "authors": [
                "Yingxue Zhang",
                "Ping Jian",
                "Fandong Meng",
                "Ruiying Geng",
                "Wei Cheng",
                "Jie Zhou"
            ],
            "abstract": "Implicit discourse relation classification is of great importance for\ndiscourse parsing, but remains a challenging problem due to the absence of\nexplicit discourse connectives communicating these relations. Modeling the\nsemantic interactions between the two arguments of a relation has proven useful\nfor detecting implicit discourse relations. However, most previous approaches\nmodel such semantic interactions from a shallow interactive level, which is\ninadequate on capturing enough semantic information. In this paper, we propose\na novel and effective Semantic Graph Convolutional Network (SGCN) to enhance\nthe modeling of inter-argument semantics on a deeper interaction level for\nimplicit discourse relation classification. We first build an interaction graph\nover representations of the two arguments, and then automatically extract\nin-depth semantic interactive information through graph convolution.\nExperimental results on the English corpus PDTB and the Chinese corpus CDTB\nboth demonstrate the superiority of our model to previous state-of-the-art\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09295v3",
            "date": "2019-10-21 12:28:00+00:00",
            "title": "Localization of Fake News Detection via Multitask Transfer Learning",
            "authors": [
                "Jan Christian Blaise Cruz",
                "Julianne Agatha Tan",
                "Charibeth Cheng"
            ],
            "abstract": "The use of the internet as a fast medium of spreading fake news reinforces\nthe need for computational tools that combat it. Techniques that train fake\nnews classifiers exist, but they all assume an abundance of resources including\nlarge labeled datasets and expert-curated corpora, which low-resource languages\nmay not have. In this work, we make two main contributions: First, we alleviate\nresource scarcity by constructing the first expertly-curated benchmark dataset\nfor fake news detection in Filipino, which we call \"Fake News Filipino.\"\nSecond, we benchmark Transfer Learning (TL) techniques and show that they can\nbe used to train robust fake news classifiers from little data, achieving 91%\naccuracy on our fake news dataset, reducing the error by 14% compared to\nestablished few-shot baselines. Furthermore, lifting ideas from multitask\nlearning, we show that augmenting transformer-based transfer techniques with\nauxiliary language modeling losses improves their performance by adapting to\nwriting style. Using this, we improve TL performance by 4-6%, achieving an\naccuracy of 96% on our best model. Lastly, we show that our method generalizes\nwell to different types of news articles, including political news,\nentertainment news, and opinion articles.",
            "journal ref": "In Proceedings of The 12th Language Resources and Evaluation\n  Conference, pp.2589-2597 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09302v1",
            "date": "2019-10-21 12:34:53+00:00",
            "title": "Diversify Your Datasets: Analyzing Generalization via Controlled Variance in Adversarial Datasets",
            "authors": [
                "Ohad Rozen",
                "Vered Shwartz",
                "Roee Aharoni",
                "Ido Dagan"
            ],
            "abstract": "Phenomenon-specific \"adversarial\" datasets have been recently designed to\nperform targeted stress-tests for particular inference types. Recent work (Liu\net al., 2019a) proposed that such datasets can be utilized for training NLI and\nother types of models, often allowing to learn the phenomenon in focus and\nimprove on the challenge dataset, indicating a \"blind spot\" in the original\ntraining data. Yet, although a model can improve in such a training process, it\nmight still be vulnerable to other challenge datasets targeting the same\nphenomenon but drawn from a different distribution, such as having a different\nsyntactic complexity level. In this work, we extend this method to drive\nconclusions about a model's ability to learn and generalize a target phenomenon\nrather than to \"learn\" a dataset, by controlling additional aspects in the\nadversarial datasets. We demonstrate our approach on two inference phenomena -\ndative alternation and numerical reasoning, elaborating, and in some cases\ncontradicting, the results of Liu et al.. Our methodology enables building\nbetter challenge datasets for creating more robust models, and may yield better\nmodel understanding and subsequent overarching improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09342v2",
            "date": "2019-10-21 13:11:21+00:00",
            "title": "Domain-agnostic Question-Answering with Adversarial Training",
            "authors": [
                "Seanie Lee",
                "Donggyu Kim",
                "Jangwon Park"
            ],
            "abstract": "Adapting models to new domain without finetuning is a challenging problem in\ndeep learning. In this paper, we utilize an adversarial training framework for\ndomain generalization in Question Answering (QA) task. Our model consists of a\nconventional QA model and a discriminator. The training is performed in the\nadversarial manner, where the two models constantly compete, so that QA model\ncan learn domain-invariant features. We apply this approach in MRQA Shared Task\n2019 and show better performance compared to the baseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09362v1",
            "date": "2019-10-21 13:34:12+00:00",
            "title": "Improving Word Representations: A Sub-sampled Unigram Distribution for Negative Sampling",
            "authors": [
                "Wenxiang Jiao",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "Word2Vec is the most popular model for word representation and has been\nwidely investigated in literature. However, its noise distribution for negative\nsampling is decided by empirical trials and the optimality has always been\nignored. We suggest that the distribution is a sub-optimal choice, and propose\nto use a sub-sampled unigram distribution for better negative sampling. Our\ncontributions include: (1) proposing the concept of semantics quantification\nand deriving a suitable sub-sampling rate for the proposed distribution\nadaptive to different training corpora; (2) demonstrating the advantages of our\napproach in both negative sampling and noise contrastive estimation by\nextensive evaluation tasks; and (3) proposing a semantics weighted model for\nthe MSR sentence completion task, resulting in considerable improvements. Our\nwork not only improves the quality of word vectors but also benefits current\nunderstanding of Word2Vec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09702v1",
            "date": "2019-10-22 00:06:52+00:00",
            "title": "Fine-Tuned Neural Models for Propaganda Detection at the Sentence and Fragment levels",
            "authors": [
                "Tariq Alhindi",
                "Jonas Pfeiffer",
                "Smaranda Muresan"
            ],
            "abstract": "This paper presents the CUNLP submission for the NLP4IF 2019 shared-task on\nFineGrained Propaganda Detection. Our system finished 5th out of 26 teams on\nthe sentence-level classification task and 5th out of 11 teams on the\nfragment-level classification task based on our scores on the blind test set.\nWe present our models, a discussion of our ablation studies and experiments,\nand an analysis of our performance on all eighteen propaganda techniques\npresent in the corpus of the shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09729v1",
            "date": "2019-10-22 02:16:47+00:00",
            "title": "Grammatical Gender, Neo-Whorfianism, and Word Embeddings: A Data-Driven Approach to Linguistic Relativity",
            "authors": [
                "Katharina Kann"
            ],
            "abstract": "The relation between language and thought has occupied linguists for at least\na century. Neo-Whorfianism, a weak version of the controversial Sapir-Whorf\nhypothesis, holds that our thoughts are subtly influenced by the grammatical\nstructures of our native language. One area of investigation in this vein\nfocuses on how the grammatical gender of nouns affects the way we perceive the\ncorresponding objects. For instance, does the fact that key is masculine in\nGerman (der Schl\\\"ussel), but feminine in Spanish (la llave) change the\nspeakers' views of those objects? Psycholinguistic evidence presented by\nBoroditsky et al. (2003, {\\S}4) suggested the answer might be yes: When asked\nto produce adjectives that best described a key, German and Spanish speakers\nnamed more stereotypically masculine and feminine ones, respectively. However,\nrecent attempts to replicate those experiments have failed (Mickan et al.,\n2014). In this work, we offer a computational analogue of Boroditsky et al.\n(2003, {\\S}4)'s experimental design on 9 languages, finding evidence against\nneo-Whorfianism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09753v2",
            "date": "2019-10-22 03:41:33+00:00",
            "title": "MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension",
            "authors": [
                "Adam Fisch",
                "Alon Talmor",
                "Robin Jia",
                "Minjoon Seo",
                "Eunsol Choi",
                "Danqi Chen"
            ],
            "abstract": "We present the results of the Machine Reading for Question Answering (MRQA)\n2019 shared task on evaluating the generalization capabilities of reading\ncomprehension systems. In this task, we adapted and unified 18 distinct\nquestion answering datasets into the same format. Among them, six datasets were\nmade available for training, six datasets were made available for development,\nand the final six were hidden for final evaluation. Ten teams submitted\nsystems, which explored various ideas including data sampling, multi-task\nlearning, adversarial training and ensembling. The best system achieved an\naverage F1 score of 72.5 on the 12 held-out datasets, 10.7 absolute points\nhigher than our initial baseline based on BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09796v3",
            "date": "2019-10-22 07:06:45+00:00",
            "title": "Fine-grained Fact Verification with Kernel Graph Attention Network",
            "authors": [
                "Zhenghao Liu",
                "Chenyan Xiong",
                "Maosong Sun",
                "Zhiyuan Liu"
            ],
            "abstract": "Fact Verification requires fine-grained natural language inference capability\nthat finds subtle clues to identify the syntactical and semantically correct\nbut not well-supported claims. This paper presents Kernel Graph Attention\nNetwork (KGAT), which conducts more fine-grained fact verification with\nkernel-based attentions. Given a claim and a set of potential evidence\nsentences that form an evidence graph, KGAT introduces node kernels, which\nbetter measure the importance of the evidence node, and edge kernels, which\nconduct fine-grained evidence propagation in the graph, into Graph Attention\nNetworks for more accurate fact verification. KGAT achieves a 70.38% FEVER\nscore and significantly outperforms existing fact verification models on FEVER,\na large-scale benchmark for fact verification. Our analyses illustrate that,\ncompared to dot-product attentions, the kernel-based attention concentrates\nmore on relevant evidence sentences and meaningful clues in the evidence graph,\nwhich is the main source of KGAT's effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09916v1",
            "date": "2019-10-22 12:16:00+00:00",
            "title": "Automatic Extraction of Personality from Text: Challenges and Opportunities",
            "authors": [
                "Nazar Akrami",
                "Johan Fernquist",
                "Tim Isbister",
                "Lisa Kaati",
                "Bj\u00f6rn Pelzer"
            ],
            "abstract": "In this study, we examined the possibility to extract personality traits from\na text. We created an extensive dataset by having experts annotate personality\ntraits in a large number of texts from multiple online sources. From these\nannotated texts, we selected a sample and made further annotations ending up in\na large low-reliability dataset and a small high-reliability dataset. We then\nused the two datasets to train and test several machine learning models to\nextract personality from text, including a language model. Finally, we\nevaluated our best models in the wild, on datasets from different domains. Our\nresults show that the models based on the small high-reliability dataset\nperformed better (in terms of $\\textrm{R}^2$) than models based on large\nlow-reliability dataset. Also, language model based on small high-reliability\ndataset performed better than the random baseline. Finally, and more\nimportantly, the results showed our best model did not perform better than the\nrandom baseline when tested in the wild. Taken together, our results show that\ndetermining personality traits from a text remains a challenge and that no firm\nconclusions can be made on model performance before testing in the wild.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09942v1",
            "date": "2019-10-22 13:01:00+00:00",
            "title": "Scalable Neural Dialogue State Tracking",
            "authors": [
                "Vevake Balaraman",
                "Bernardo Magnini"
            ],
            "abstract": "A Dialogue State Tracker (DST) is a key component in a dialogue system aiming\nat estimating the beliefs of possible user goals at each dialogue turn. Most of\nthe current DST trackers make use of recurrent neural networks and are based on\ncomplex architectures that manage several aspects of a dialogue, including the\nuser utterance, the system actions, and the slot-value pairs defined in a\ndomain ontology. However, the complexity of such neural architectures incurs\ninto a considerable latency in the dialogue state prediction, which limits the\ndeployments of the models in real-world applications, particularly when task\nscalability (i.e. amount of slots) is a crucial factor. In this paper, we\npropose an innovative neural model for dialogue state tracking, named Global\nencoder and Slot-Attentive decoders (G-SAT), which can predict the dialogue\nstate with a very low latency time, while maintaining high-level performance.\nWe report experiments on three different languages (English, Italian, and\nGerman) of the WoZ2.0 dataset, and show that the proposed approach provides\ncompetitive advantages over state-of-art DST systems, both in terms of accuracy\nand in terms of time complexity for predictions, being over 15 times faster\nthan the other systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10138v3",
            "date": "2019-10-22 17:41:48+00:00",
            "title": "Universal Decompositional Semantic Parsing",
            "authors": [
                "Elias Stengel-Eskin",
                "Aaron Steven White",
                "Sheng Zhang",
                "Benjamin Van Durme"
            ],
            "abstract": "We introduce a transductive model for parsing into Universal Decompositional\nSemantics (UDS) representations, which jointly learns to map natural language\nutterances into UDS graph structures and annotate the graph with\ndecompositional semantic attribute scores. We also introduce a strong pipeline\nmodel for parsing into the UDS graph structure, and show that our transductive\nparser performs comparably while additionally performing attribute prediction.\nBy analyzing the attribute prediction errors, we find the model captures\nnatural relationships between attribute groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10274v1",
            "date": "2019-10-22 23:19:04+00:00",
            "title": "Capturing Greater Context for Question Generation",
            "authors": [
                "Luu Anh Tuan",
                "Darsh J Shah",
                "Regina Barzilay"
            ],
            "abstract": "Automatic question generation can benefit many applications ranging from\ndialogue systems to reading comprehension. While questions are often asked with\nrespect to long documents, there are many challenges with modeling such long\ndocuments. Many existing techniques generate questions by effectively looking\nat one sentence at a time, leading to questions that are easy and not\nreflective of the human process of question generation. Our goal is to\nincorporate interactions across multiple sentences to generate realistic\nquestions for long documents. In order to link a broad document context to the\ntarget answer, we represent the relevant context via a multi-stage attention\nmechanism, which forms the foundation of a sequence to sequence model. We\noutperform state-of-the-art methods on question generation on three\nquestion-answering datasets -- SQuAD, MS MARCO and NewsQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10281v2",
            "date": "2019-10-22 23:41:53+00:00",
            "title": "A Search-based Neural Model for Biomedical Nested and Overlapping Event Detection",
            "authors": [
                "Kurt Espinosa",
                "Makoto Miwa",
                "Sophia Ananiadou"
            ],
            "abstract": "We tackle the nested and overlapping event detection task and propose a novel\nsearch-based neural network (SBNN) structured prediction model that treats the\ntask as a search problem on a relation graph of trigger-argument structures.\nUnlike existing structured prediction tasks such as dependency parsing, the\ntask targets to detect DAG structures, which constitute events, from the\nrelation graph. We define actions to construct events and use all the beams in\na beam search to detect all event structures that may be overlapping and\nnested. The search process constructs events in a bottom-up manner while\nmodelling the global properties for nested and overlapping structures\nsimultaneously using neural networks. We show that the model achieves\nperformance comparable to the state-of-the-art model Turku Event Extraction\nSystem (TEES) on the BioNLP Cancer Genetics (CG) Shared Task 2013 without the\nuse of any syntactic and hand-engineered features. Further analyses on the\ndevelopment set show that our model is more computationally efficient while\nyielding higher F1-score performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10408v2",
            "date": "2019-10-23 08:25:43+00:00",
            "title": "Controlling the Output Length of Neural Machine Translation",
            "authors": [
                "Surafel Melaku Lakew",
                "Mattia Di Gangi",
                "Marcello Federico"
            ],
            "abstract": "The recent advances introduced by neural machine translation (NMT) are\nrapidly expanding the application fields of machine translation, as well as\nreshaping the quality level to be targeted. In particular, if translations have\nto fit some given layout, quality should not only be measured in terms of\nadequacy and fluency, but also length. Exemplary cases are the translation of\ndocument files, subtitles, and scripts for dubbing, where the output length\nshould ideally be as close as possible to the length of the input text. This\npaper addresses for the first time, to the best of our knowledge, the problem\nof controlling the output length in NMT. We investigate two methods for biasing\nthe output length with a transformer architecture: i) conditioning the output\nto a given target-source length-ratio class and ii) enriching the transformer\npositional embedding with length information. Our experiments show that both\nmethods can induce the network to generate shorter translations, as well as\nacquiring interpretable linguistic skills.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10869v2",
            "date": "2019-10-24 01:18:24+00:00",
            "title": "Combining Acoustics, Content and Interaction Features to Find Hot Spots in Meetings",
            "authors": [
                "Dave Makhervaks",
                "William Hinthorn",
                "Dimitrios Dimitriadis",
                "Andreas Stolcke"
            ],
            "abstract": "Involvement hot spots have been proposed as a useful concept for meeting\nanalysis and studied off and on for over 15 years. These are regions of\nmeetings that are marked by high participant involvement, as judged by human\nannotators. However, prior work was either not conducted in a formal machine\nlearning setting, or focused on only a subset of possible meeting features or\ndownstream applications (such as summarization). In this paper we investigate\nto what extent various acoustic, linguistic and pragmatic aspects of the\nmeetings, both in isolation and jointly, can help detect hot spots. In this\ncontext, the openSMILE toolkit is to used to extract features based on\nacoustic-prosodic cues, BERT word embeddings are used for encoding the lexical\ncontent, and a variety of statistics based on speech activity are used to\ndescribe the verbal interaction among participants. In experiments on the\nannotated ICSI meeting corpus, we find that the lexical model is the most\ninformative, with incremental contributions from interaction and\nacoustic-prosodic model components.",
            "journal ref": "Proc. IEEE ICASSP, May 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10893v1",
            "date": "2019-10-24 03:00:53+00:00",
            "title": "Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations",
            "authors": [
                "Zuyi Bao",
                "Rui Huang",
                "Chen Li",
                "Kenny Q. Zhu"
            ],
            "abstract": "Previous work on cross-lingual sequence labeling tasks either requires\nparallel data or bridges the two languages through word-byword matching. Such\nrequirements and assumptions are infeasible for most languages, especially for\nlanguages with large linguistic distances, e.g., English and Chinese. In this\nwork, we propose a Multilingual Language Model with deep semantic Alignment\n(MLMA) to generate language-independent representations for cross-lingual\nsequence labeling. Our methods require only monolingual corpora with no\nbilingual resources at all and take advantage of deep contextualized\nrepresentations. Experimental results show that our approach achieves new\nstate-of-the-art NER and POS performance across European languages, and is also\neffective on distant language pairs such as English and Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10950v1",
            "date": "2019-10-24 07:26:36+00:00",
            "title": "Pun-GAN: Generative Adversarial Network for Pun Generation",
            "authors": [
                "Fuli Luo",
                "Shunyao Li",
                "Pengcheng Yang",
                "Lei li",
                "Baobao Chang",
                "Zhifang Sui",
                "Xu Sun"
            ],
            "abstract": "In this paper, we focus on the task of generating a pun sentence given a pair\nof word senses. A major challenge for pun generation is the lack of large-scale\npun corpus to guide the supervised learning. To remedy this, we propose an\nadversarial generative network for pun generation (Pun-GAN), which does not\nrequire any pun corpus. It consists of a generator to produce pun sentences,\nand a discriminator to distinguish between the generated pun sentences and the\nreal sentences with specific word senses. The output of the discriminator is\nthen used as a reward to train the generator via reinforcement learning,\nencouraging it to produce pun sentences that can support two word senses\nsimultaneously. Experiments show that the proposed Pun-GAN can generate\nsentences that are more ambiguous and diverse in both automatic and human\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11005v2",
            "date": "2019-10-24 10:04:12+00:00",
            "title": "Wasserstein distances for evaluating cross-lingual embeddings",
            "authors": [
                "Georgios Balikas",
                "Ioannis Partalas"
            ],
            "abstract": "Word embeddings are high dimensional vector representations of words that\ncapture their semantic similarity in the vector space. There exist several\nalgorithms for learning such embeddings both for a single language as well as\nfor several languages jointly. In this work we propose to evaluate collections\nof embeddings by adapting downstream natural language tasks to the optimal\ntransport framework. We show how the family of Wasserstein distances can be\nused to solve cross-lingual document retrieval and the cross-lingual document\nclassification problems. We argue on the advantages of this approach compared\nto more traditional evaluation methods of embeddings like bilingual lexical\ninduction. Our experimental results suggest that using Wasserstein distances on\nthese problems out-performs several strong baselines and performs on par with\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11204v1",
            "date": "2019-10-24 15:05:01+00:00",
            "title": "Syntax-Enhanced Self-Attention-Based Semantic Role Labeling",
            "authors": [
                "Yue Zhang",
                "Rui Wang",
                "Luo Si"
            ],
            "abstract": "As a fundamental NLP task, semantic role labeling (SRL) aims to discover the\nsemantic roles for each predicate within one sentence. This paper investigates\nhow to incorporate syntactic knowledge into the SRL task effectively. We\npresent different approaches of encoding the syntactic information derived from\ndependency trees of different quality and representations; we propose a\nsyntax-enhanced self-attention model and compare it with other two strong\nbaseline methods; and we conduct experiments with newly published deep\ncontextualized word representations as well. The experiment results demonstrate\nthat with proper incorporation of the high quality syntactic information, our\nmodel achieves a new state-of-the-art performance for the Chinese SRL task on\nthe CoNLL-2009 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11218v1",
            "date": "2019-10-24 15:23:08+00:00",
            "title": "Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed",
            "authors": [
                "Thuong-Hai Pham",
                "Dominik Mach\u00e1\u010dek",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "The utility of linguistic annotation in neural machine translation seemed to\nhad been established in past papers. The experiments were however limited to\nrecurrent sequence-to-sequence architectures and relatively small data\nsettings. We focus on the state-of-the-art Transformer model and use comparably\nlarger corpora. Specifically, we try to promote the knowledge of source-side\nsyntax using multi-task learning either through simple data manipulation\ntechniques or through a dedicated model component. In particular, we train one\nof Transformer attention heads to produce source-side dependency tree. Overall,\nour results cast some doubt on the utility of multi-task setups with linguistic\ninformation. The data manipulation techniques, recommended in previous works,\nprove ineffective in large data settings. The treatment of self-attention as\ndependencies seems much more promising: it helps in translation and reveals\nthat Transformer model can very easily grasp the syntactic structure. An\nimportant but curious result is, however, that identical gains are obtained by\nusing trivial \"linear trees\" instead of true dependencies. The reason for the\ngain thus may not be coming from the added linguistic knowledge but from some\nsimpler regularizing effect we induced on self-attention matrices.",
            "journal ref": "Computac\\'ion y Sistemas, Vol. 23, No. 3, 2019, pp. 923-934",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11236v1",
            "date": "2019-10-14 11:52:03+00:00",
            "title": "Interpretable Text Classification Using CNN and Max-pooling",
            "authors": [
                "Hao Cheng",
                "Xiaoqing Yang",
                "Zang Li",
                "Yanghua Xiao",
                "Yucheng Lin"
            ],
            "abstract": "Deep neural networks have been widely used in text classification. However,\nit is hard to interpret the neural models due to the complicate mechanisms. In\nthis work, we study the interpretability of a variant of the typical text\nclassification model which is based on convolutional operation and max-pooling\nlayer. Two mechanisms: convolution attribution and n-gram feature analysis are\nproposed to analyse the process procedure for the CNN model. The\ninterpretability of the model is reflected by providing posterior\ninterpretation for neural network predictions. Besides, a multi-sentence\nstrategy is proposed to enable the model to beused in multi-sentence situation\nwithout loss of performance and interpret ability. We evaluate the performance\nof the model on several classification tasks and justify the interpretable\nperformance with some case studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11295v1",
            "date": "2019-10-24 17:17:10+00:00",
            "title": "\u00daFAL MRPipe at MRP 2019: UDPipe Goes Semantic in the Meaning Representation Parsing Shared Task",
            "authors": [
                "Milan Straka",
                "Jana Strakov\u00e1"
            ],
            "abstract": "We present a system description of our contribution to the CoNLL 2019 shared\ntask, Cross-Framework Meaning Representation Parsing (MRP 2019). The proposed\narchitecture is our first attempt towards a semantic parsing extension of the\nUDPipe 2.0, a lemmatization, POS tagging and dependency parsing pipeline.\n  For the MRP 2019, which features five formally and linguistically different\napproaches to meaning representation (DM, PSD, EDS, UCCA and AMR), we propose a\nuniform, language and framework agnostic graph-to-graph neural network\narchitecture. Without any knowledge about the graph structure, and specifically\nwithout any linguistically or framework motivated features, our system\nimplicitly models the meaning representation graphs.\n  After fixing a human error (we used earlier incorrect version of provided\ntest set analyses), our submission would score third in the competition\nevaluation. The source code of our system is available at\nhttps://github.com/ufal/mrpipe-conll2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11399v1",
            "date": "2019-10-24 19:54:32+00:00",
            "title": "Comparison of Quality Indicators in User-generated Content Using Social Media and Scholarly Text",
            "authors": [
                "Manirupa Das",
                "Renhao Cui"
            ],
            "abstract": "Predicting the quality of a text document is a critical task when presented\nwith the problem of measuring the performance of a document before its release.\nIn this work, we evaluate various features including those extracted from the\ntext content (textual) and those describing higher-level characteristics of the\ntext (meta) features that are not directly available from the text, and show\nhow these features inform prediction of document quality in different ways.\nMoreover, we also compare our methods on both social user-generated data such\nas tweets, and scholarly user-generated data such as academic articles, showing\nhow the same features differently influence prediction of quality across these\ndisparate domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11411v1",
            "date": "2019-10-24 20:14:12+00:00",
            "title": "Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations",
            "authors": [
                "Sangwoo Cho",
                "Chen Li",
                "Dong Yu",
                "Hassan Foroosh",
                "Fei Liu"
            ],
            "abstract": "Emerged as one of the best performing techniques for extractive\nsummarization, determinantal point processes select the most probable set of\nsentences to form a summary according to a probability measure defined by\nmodeling sentence prominence and pairwise repulsion. Traditionally, these\naspects are modelled using shallow and linguistically informed features, but\nthe rise of deep contextualized representations raises an interesting question\nof whether, and to what extent, contextualized representations can be used to\nimprove DPP modeling. Our findings suggest that, despite the success of deep\nrepresentations, it remains necessary to combine them with surface indicators\nfor effective identification of summary sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11473v2",
            "date": "2019-10-25 01:02:18+00:00",
            "title": "QASC: A Dataset for Question Answering via Sentence Composition",
            "authors": [
                "Tushar Khot",
                "Peter Clark",
                "Michal Guerquin",
                "Peter Jansen",
                "Ashish Sabharwal"
            ],
            "abstract": "Composing knowledge from multiple pieces of texts is a key challenge in\nmulti-hop question answering. We present a multi-hop reasoning dataset,\nQuestion Answering via Sentence Composition(QASC), that requires retrieving\nfacts from a large corpus and composing them to answer a multiple-choice\nquestion. QASC is the first dataset to offer two desirable properties: (a) the\nfacts to be composed are annotated in a large corpus, and (b) the decomposition\ninto these facts is not evident from the question itself. The latter makes\nretrieval challenging as the system must introduce new concepts or relations in\norder to discover potential decompositions. Further, the reasoning model must\nthen learn to identify valid compositions of these retrieved facts using\ncommon-sense reasoning. To help address these challenges, we provide annotation\nfor supporting facts as well as their composition. Guided by these annotations,\nwe present a two-step approach to mitigate the retrieval challenges. We use\nother multiple-choice datasets as additional training data to strengthen the\nreasoning model. Our proposed approach improves over current state-of-the-art\nlanguage models by 11% (absolute). The reasoning and retrieval problems,\nhowever, remain unsolved as this model still lags by 20% behind human\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11476v6",
            "date": "2019-10-25 01:06:07+00:00",
            "title": "A Unified MRC Framework for Named Entity Recognition",
            "authors": [
                "Xiaoya Li",
                "Jingrong Feng",
                "Yuxian Meng",
                "Qinghong Han",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "The task of named entity recognition (NER) is normally divided into nested\nNER and flat NER depending on whether named entities are nested or not. Models\nare usually separately developed for the two tasks, since sequence labeling\nmodels, the most widely used backbone for flat NER, are only able to assign a\nsingle label to a particular token, which is unsuitable for nested NER where a\ntoken may be assigned several labels.\n  In this paper, we propose a unified framework that is capable of handling\nboth flat and nested NER tasks. Instead of treating the task of NER as a\nsequence labeling problem, we propose to formulate it as a machine reading\ncomprehension (MRC) task. For example, extracting entities with the\n\\textsc{per} label is formalized as extracting answer spans to the question\n\"{\\it which person is mentioned in the text?}\". This formulation naturally\ntackles the entity overlapping issue in nested NER: the extraction of two\noverlapping entities for different categories requires answering two\nindependent questions. Additionally, since the query encodes informative prior\nknowledge, this strategy facilitates the process of entity extraction, leading\nto better performances for not only nested NER, but flat NER.\n  We conduct experiments on both {\\em nested} and {\\em flat} NER datasets.\nExperimental results demonstrate the effectiveness of the proposed formulation.\nWe are able to achieve vast amount of performance boost over current SOTA\nmodels on nested NER datasets, i.e., +1.28, +2.55, +5.44, +6.37, respectively\non ACE04, ACE05, GENIA and KBP17, along with SOTA results on flat NER datasets,\ni.e.,+0.24, +1.95, +0.21, +1.49 respectively on English CoNLL 2003, English\nOntoNotes 5.0, Chinese MSRA, Chinese OntoNotes 4.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11483v1",
            "date": "2019-10-25 01:35:14+00:00",
            "title": "Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models",
            "authors": [
                "Woon Sang Cho",
                "Yizhe Zhang",
                "Sudha Rao",
                "Chris Brockett",
                "Sungjin Lee"
            ],
            "abstract": "Ambiguous user queries in search engines result in the retrieval of documents\nthat often span multiple topics. One potential solution is for the search\nengine to generate multiple refined queries, each of which relates to a subset\nof the documents spanning the same topic. A preliminary step towards this goal\nis to generate a question that captures common concepts of multiple documents.\nWe propose a new task of generating common question from multiple documents and\npresent simple variant of an existing multi-source encoder-decoder framework,\ncalled the Multi-Source Question Generator (MSQG). We first train an RNN-based\nsingle encoder-decoder generator from (single document, question) pairs. At\ntest time, given multiple documents, the 'Distribute' step of our MSQG model\npredicts target word distributions for each document using the trained model.\nThe 'Aggregate' step aggregates these distributions to generate a common\nquestion. This simple yet effective strategy significantly outperforms several\nexisting baseline models applied to the new task when evaluated using automated\nmetrics and human judgments on the MS-MARCO-QA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11491v1",
            "date": "2019-10-25 02:14:17+00:00",
            "title": "Attention Optimization for Abstractive Document Summarization",
            "authors": [
                "Min Gui",
                "Junfeng Tian",
                "Rui Wang",
                "Zhenglu Yang"
            ],
            "abstract": "Attention plays a key role in the improvement of sequence-to-sequence-based\ndocument summarization models. To obtain a powerful attention helping with\nreproducing the most salient information and avoiding repetitions, we augment\nthe vanilla attention model from both local and global aspects. We propose an\nattention refinement unit paired with local variance loss to impose supervision\non the attention model at each decoding step, and a global variance loss to\noptimize the attention distributions of all decoding steps from the global\nperspective. The performances on the CNN/Daily Mail dataset verify the\neffectiveness of our methods.",
            "journal ref": "Proceedings of the 2019 Conference on Empirical Methods in Natural\n  Language Processing and the 9th International Joint Conference on Natural\n  Language Processing (EMNLP-IJCNLP),",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11493v2",
            "date": "2019-10-25 02:20:06+00:00",
            "title": "The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection",
            "authors": [
                "Arya D. McCarthy",
                "Ekaterina Vylomova",
                "Shijie Wu",
                "Chaitanya Malaviya",
                "Lawrence Wolf-Sonkin",
                "Garrett Nicolai",
                "Christo Kirov",
                "Miikka Silfverberg",
                "Sabrina J. Mielke",
                "Jeffrey Heinz",
                "Ryan Cotterell",
                "Mans Hulden"
            ],
            "abstract": "The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual\nanalysis in morphology examined transfer learning of inflection between 100\nlanguage pairs, as well as contextual lemmatization and morphosyntactic\ndescription in 66 languages. The first task evolves past years' inflection\ntasks by examining transfer of morphological inflection knowledge from a\nhigh-resource language to a low-resource language. This year also presents a\nnew second challenge on lemmatization and morphological feature analysis in\ncontext. All submissions featured a neural component and built on either this\nyear's strong baselines or highly ranked systems from previous years' shared\ntasks. Every participating team improved in accuracy over the baselines for the\ninflection task (though not Levenshtein distance), and every team in the\ncontextual analysis task improved on both state-of-the-art neural and\nnon-neural baselines.",
            "journal ref": "Proceedings of the 16th Workshop on Computational Research in\n  Phonetics, Phonology, and Morphology (2019) 229-244",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11536v1",
            "date": "2019-10-25 05:35:36+00:00",
            "title": "Stem-driven Language Models for Morphologically Rich Languages",
            "authors": [
                "Yash Shah",
                "Ishan Tarunesh",
                "Harsh Deshpande",
                "Preethi Jyothi"
            ],
            "abstract": "Neural language models (LMs) have shown to benefit significantly from\nenhancing word vectors with subword-level information, especially for\nmorphologically rich languages. This has been mainly tackled by providing\nsubword-level information as an input; using subword units in the output layer\nhas been far less explored. In this work, we propose LMs that are cognizant of\nthe underlying stems in each word. We derive stems for words using a simple\nunsupervised technique for stem identification. We experiment with different\narchitectures involving multi-task learning and mixture models over words and\nstems. We focus on four morphologically complex languages -- Hindi, Tamil,\nKannada and Finnish -- and observe significant perplexity gains with using our\nstem-driven LMs when compared with other competitive baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11769v1",
            "date": "2019-10-25 14:40:14+00:00",
            "title": "DENS: A Dataset for Multi-class Emotion Analysis",
            "authors": [
                "Chen Liu",
                "Muhammad Osama",
                "Anderson de Andrade"
            ],
            "abstract": "We introduce a new dataset for multi-class emotion analysis from long-form\nnarratives in English. The Dataset for Emotions of Narrative Sequences (DENS)\nwas collected from both classic literature available on Project Gutenberg and\nmodern online narratives available on Wattpad, annotated using Amazon\nMechanical Turk. A number of statistics and baseline benchmarks are provided\nfor the dataset. Of the tested techniques, we find that the fine-tuning of a\npre-trained BERT model achieves the best results, with an average micro-F1\nscore of 60.4%. Our results show that the dataset provides a novel opportunity\nin emotion analysis that requires moving beyond existing sentence-level\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11790v1",
            "date": "2019-10-25 15:15:44+00:00",
            "title": "Measuring Conversational Fluidity in Automated Dialogue Agents",
            "authors": [
                "Keith Vella",
                "Massimo Poesio",
                "Michael Sigamani",
                "Cihan Dogan",
                "Aimore Dutra",
                "Dimitrios Dimakopoulos",
                "Alfredo Gemma",
                "Ella Walters"
            ],
            "abstract": "We present an automated evaluation method to measure fluidity in\nconversational dialogue systems. The method combines various state of the art\nNatural Language tools into a classifier, and human ratings on these dialogues\nto train an automated judgment model. Our experiments show that the results are\nan improvement on existing metrics for measuring fluidity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11834v2",
            "date": "2019-10-25 16:35:47+00:00",
            "title": "Evaluation of Sentence Representations in Polish",
            "authors": [
                "S\u0142awomir Dadas",
                "Micha\u0142 Pere\u0142kiewicz",
                "Rafa\u0142 Po\u015bwiata"
            ],
            "abstract": "Methods for learning sentence representations have been actively developed in\nrecent years. However, the lack of pre-trained models and datasets annotated at\nthe sentence level has been a problem for low-resource languages such as Polish\nwhich led to less interest in applying these methods to language-specific\ntasks. In this study, we introduce two new Polish datasets for evaluating\nsentence embeddings and provide a comprehensive evaluation of eight sentence\nrepresentation methods including Polish and multilingual models. We consider\nclassic word embedding models, recently developed contextual embeddings and\nmultilingual sentence encoders, showing strengths and weaknesses of specific\napproaches. We also examine different methods of aggregating word vectors into\na single sentence vector.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11932v1",
            "date": "2019-10-25 20:59:09+00:00",
            "title": "Exploring Author Context for Detecting Intended vs Perceived Sarcasm",
            "authors": [
                "Silviu Oprea",
                "Walid Magdy"
            ],
            "abstract": "We investigate the impact of using author context on textual sarcasm\ndetection. We define author context as the embedded representation of their\nhistorical posts on Twitter and suggest neural models that extract these\nrepresentations. We experiment with two tweet datasets, one labelled manually\nfor sarcasm, and the other via tag-based distant supervision. We achieve\nstate-of-the-art performance on the second dataset, but not on the one labelled\nmanually, indicating a difference between intended sarcasm, captured by distant\nsupervision, and perceived sarcasm, captured by manual labelling.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics, 2019, pages 2854-2859",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11959v1",
            "date": "2019-10-25 23:13:15+00:00",
            "title": "FineText: Text Classification via Attention-based Language Model Fine-tuning",
            "authors": [
                "Yunzhe Tao",
                "Saurabh Gupta",
                "Satyapriya Krishna",
                "Xiong Zhou",
                "Orchid Majumder",
                "Vineet Khare"
            ],
            "abstract": "Training deep neural networks from scratch on natural language processing\n(NLP) tasks requires significant amount of manually labeled text corpus and\nsubstantial time to converge, which usually cannot be satisfied by the\ncustomers. In this paper, we aim to develop an effective transfer learning\nalgorithm by fine-tuning a pre-trained language model. The goal is to provide\nexpressive and convenient-to-use feature extractors for downstream NLP tasks,\nand achieve improvement in terms of accuracy, data efficiency, and\ngeneralization to new domains. Therefore, we propose an attention-based\nfine-tuning algorithm that automatically selects relevant contextualized\nfeatures from the pre-trained language model and uses those features on\ndownstream text classification tasks. We test our methods on six widely-used\nbenchmarking datasets, and achieve new state-of-the-art performance on all of\nthem. Moreover, we then introduce an alternative multi-task learning approach,\nwhich is an end-to-end algorithm given the pre-trained model. By doing\nmulti-task learning, one can largely reduce the total training time by trading\noff some classification accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11966v1",
            "date": "2019-10-26 00:21:18+00:00",
            "title": "Yall should read this! Identifying Plurality in Second-Person Personal Pronouns in English Texts",
            "authors": [
                "Gabriel Stanovsky",
                "Ronen Tamari"
            ],
            "abstract": "Distinguishing between singular and plural \"you\" in English is a challenging\ntask which has potential for downstream applications, such as machine\ntranslation or coreference resolution. While formal written English does not\ndistinguish between these cases, other languages (such as Spanish), as well as\nother dialects of English (via phrases such as \"yall\"), do make this\ndistinction. We make use of this to obtain distantly-supervised labels for the\ntask on a large-scale in two domains. Following, we train a model to\ndistinguish between the single/plural you, finding that although in-domain\ntraining achieves reasonable accuracy (over 77%), there is still a lot of room\nfor improvement, especially in the domain-transfer scenario, which proves\nextremely challenging. Our code and data are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12038v1",
            "date": "2019-10-26 09:40:26+00:00",
            "title": "Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention",
            "authors": [
                "Lei Cao",
                "Huijun Zhang",
                "Ling Feng",
                "Zihan Wei",
                "Xin Wang",
                "Ningyun Li",
                "Xiaohao He"
            ],
            "abstract": "Despite detection of suicidal ideation on social media has made great\nprogress in recent years, people's implicitly and anti-real contrarily\nexpressed posts still remain as an obstacle, constraining the detectors to\nacquire higher satisfactory performance. Enlightened by the hidden \"tree holes\"\nphenomenon on microblog, where people at suicide risk tend to disclose their\ninner real feelings and thoughts to the microblog space whose authors have\ncommitted suicide, we explore the use of tree holes to enhance microblog-based\nsuicide risk detection from the following two perspectives. (1) We build\nsuicide-oriented word embeddings based on tree hole contents to strength the\nsensibility of suicide-related lexicons and context based on tree hole\ncontents. (2) A two-layered attention mechanism is deployed to grasp\nintermittently changing points from individual's open blog streams, revealing\none's inner emotional world more or less. Our experimental results show that\nwith suicide-oriented word embeddings and attention, microblog-based suicide\nrisk detection can achieve over 91\\% accuracy. A large-scale well-labelled\nsuicide data set is also reported in the paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12129v1",
            "date": "2019-10-26 20:18:59+00:00",
            "title": "ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversation",
            "authors": [
                "Juraj Juraska",
                "Kevin K. Bowden",
                "Marilyn Walker"
            ],
            "abstract": "The uptake of deep learning in natural language generation (NLG) led to the\nrelease of both small and relatively large parallel corpora for training neural\nmodels. The existing data-to-text datasets are, however, aimed at task-oriented\ndialogue systems, and often thus limited in diversity and versatility. They are\ntypically crowdsourced, with much of the noise left in them. Moreover, current\nneural NLG models do not take full advantage of large training data, and due to\ntheir strong generalizing properties produce sentences that look template-like\nregardless. We therefore present a new corpus of 7K samples, which (1) is clean\ndespite being crowdsourced, (2) has utterances of 9 generalizable and\nconversational dialogue act types, making it more suitable for open-domain\ndialogue systems, and (3) explores the domain of video games, which is new to\ndialogue systems despite having excellent potential for supporting rich\nconversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12368v1",
            "date": "2019-10-27 22:14:04+00:00",
            "title": "Multitask Learning For Different Subword Segmentations In Neural Machine Translation",
            "authors": [
                "Tejas Srinivasan",
                "Ramon Sanabria",
                "Florian Metze"
            ],
            "abstract": "In Neural Machine Translation (NMT) the usage of subwords and characters as\nsource and target units offers a simple and flexible solution for translation\nof rare and unseen words. However, selecting the optimal subword segmentation\ninvolves a trade-off between expressiveness and flexibility, and is language\nand dataset-dependent. We present Block Multitask Learning (BMTL), a novel NMT\narchitecture that predicts multiple targets of different granularities\nsimultaneously, removing the need to search for the optimal segmentation\nstrategy. Our multi-task model exhibits improvements of up to 1.7 BLEU points\non each decoder over single-task baseline models with the same number of\nparameters on datasets from two language pairs of IWSLT15 and one from IWSLT19.\nThe multiple hypotheses generated at different granularities can be combined as\na post-processing step to give better translations, which improves over\nhypothesis combination from baseline models while using substantially fewer\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12391v1",
            "date": "2019-10-28 00:50:55+00:00",
            "title": "What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?",
            "authors": [
                "Chenglei Si",
                "Shuohang Wang",
                "Min-Yen Kan",
                "Jing Jiang"
            ],
            "abstract": "Multiple-Choice Reading Comprehension (MCRC) requires the model to read the\npassage and question, and select the correct answer among the given options.\nRecent state-of-the-art models have achieved impressive performance on multiple\nMCRC datasets. However, such performance may not reflect the model's true\nability of language understanding and reasoning. In this work, we adopt two\napproaches to investigate what BERT learns from MCRC datasets: 1) an\nun-readable data attack, in which we add keywords to confuse BERT, leading to a\nsignificant performance drop; and 2) an un-answerable data training, in which\nwe train BERT on partial or shuffled input. Under un-answerable data training,\nBERT achieves unexpectedly high performance. Based on our experiments on the 5\nkey MCRC datasets - RACE, MCTest, MCScript, MCScript2.0, DREAM - we observe\nthat 1) fine-tuned BERT mainly learns how keywords lead to correct prediction,\ninstead of learning semantic understanding and reasoning; and 2) BERT does not\nneed correct syntactic information to solve the task; 3) there exists artifacts\nin these datasets such that they can be solved even without the full context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12419v2",
            "date": "2019-10-28 03:18:25+00:00",
            "title": "Attention-Gated Graph Convolutions for Extracting Drug Interaction Information from Drug Labels",
            "authors": [
                "Tung Tran",
                "Ramakanth Kavuluru",
                "Halil Kilicoglu"
            ],
            "abstract": "Preventable adverse events as a result of medical errors present a growing\nconcern in the healthcare system. As drug-drug interactions (DDIs) may lead to\npreventable adverse events, being able to extract DDIs from drug labels into a\nmachine-processable form is an important step toward effective dissemination of\ndrug safety information. In this study, we tackle the problem of jointly\nextracting drugs and their interactions, including interaction outcome, from\ndrug labels. Our deep learning approach entails composing various intermediate\nrepresentations including sequence and graph based context, where the latter is\nderived using graph convolutions (GC) with a novel attention-based gating\nmechanism (holistically called GCA). These representations are then composed in\nmeaningful ways to handle all subtasks jointly. To overcome scarcity in\ntraining data, we additionally propose transfer learning by pre-training on\nrelated DDI data. Our model is trained and evaluated on the 2018 TAC DDI\ncorpus. Our GCA model in conjunction with transfer learning performs at 39.20%\nF1 and 26.09% F1 on entity recognition (ER) and relation extraction (RE)\nrespectively on the first official test set and at 45.30% F1 and 27.87% F1 on\nER and RE respectively on the second official test set corresponding to an\nimprovement over our prior best results by up to 6 absolute F1 points. After\ncontrolling for available training data, our model exhibits state-of-the-art\nperformance by improving over the next comparable best outcome by roughly three\nF1 points in ER and 1.5 F1 points in RE evaluation across two official test\nsets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12477v1",
            "date": "2019-10-28 07:28:57+00:00",
            "title": "Multi-Module System for Open Domain Chinese Question Answering over Knowledge Base",
            "authors": [
                "Yiying Yang",
                "Xiahui He",
                "Kaijie Zhou",
                "Zhongyu Wei"
            ],
            "abstract": "For the task of open domain Knowledge Based Question Answering in CCKS2019,\nwe propose a method combining information retrieval and semantic parsing. This\nmulti-module system extracts the topic entity and the most related relation\npredicate from a question and transforms it into a Sparql query statement. Our\nmethod obtained the F1 score of 70.45% on the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12527v2",
            "date": "2019-10-28 10:03:24+00:00",
            "title": "RPM-Oriented Query Rewriting Framework for E-commerce Keyword-Based Sponsored Search",
            "authors": [
                "Xiuying Chen",
                "Daorui Xiao",
                "Shen Gao",
                "Guojun Liu",
                "Wei Lin",
                "Bo Zheng",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Sponsored search optimizes revenue and relevance, which is estimated by\nRevenue Per Mille (RPM). Existing sponsored search models are all based on\ntraditional statistical models, which have poor RPM performance when queries\nfollow a heavy-tailed distribution. Here, we propose an RPM-oriented Query\nRewriting Framework (RQRF) which outputs related bid keywords that can yield\nhigh RPM. RQRF embeds both queries and bid keywords to vectors in the same\nimplicit space, converting the rewriting probability between each query and\nkeyword to the distance between the two vectors. For label construction, we\npropose an RPM-oriented sample construction method, labeling keywords based on\nwhether or not they can lead to high RPM. Extensive experiments are conducted\nto evaluate performance of RQRF. In a one month large-scale real-world traffic\nof e-commerce sponsored search system, the proposed model significantly\noutperforms traditional baseline.",
            "journal ref": "AAAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12622v1",
            "date": "2019-10-21 02:02:59+00:00",
            "title": "Trouble with the Curve: Predicting Future MLB Players Using Scouting Reports",
            "authors": [
                "Jacob Danovitch"
            ],
            "abstract": "In baseball, a scouting report profiles a player's characteristics and\ntraits, usually intended for use in player valuation. This work presents a\nfirst-of-its-kind dataset of almost 10,000 scouting reports for minor league,\ninternational, and draft prospects. Compiled from articles posted to MLB.com\nand Fangraphs.com, each report consists of a written description of the player,\nnumerical grades for several skills, and unique IDs to reference their profiles\non popular resources like MLB.com, FanGraphs, and Baseball-Reference. With this\ndataset, we employ several deep neural networks to predict if minor league\nplayers will make the MLB given their scouting report. We open-source this data\nto share with the community, and present a web application demonstrating\nlanguage variations in the reports of successful and unsuccessful prospects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12698v1",
            "date": "2019-10-28 14:17:24+00:00",
            "title": "Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis",
            "authors": [
                "Shrey Desai",
                "Barea Sinno",
                "Alex Rosenfeld",
                "Junyi Jessy Li"
            ],
            "abstract": "Insightful findings in political science often require researchers to analyze\ndocuments of a certain subject or type, yet these documents are usually\ncontained in large corpora that do not distinguish between pertinent and\nnon-pertinent documents. In contrast, we can find corpora that label relevant\ndocuments but have limitations (e.g., from a single source or era), preventing\ntheir use for political science research. To bridge this gap, we present\n\\textit{adaptive ensembling}, an unsupervised domain adaptation framework,\nequipped with a novel text classification model and time-aware training to\nensure our methods work well with diachronic corpora. Experiments on an\nexpert-annotated dataset show that our framework outperforms strong benchmarks.\nFurther analysis indicates that our methods are more stable, learn better\nrepresentations, and extract cleaner corpora for fine-grained analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12840v1",
            "date": "2019-10-28 17:51:44+00:00",
            "title": "Evaluating the Factual Consistency of Abstractive Text Summarization",
            "authors": [
                "Wojciech Kry\u015bci\u0144ski",
                "Bryan McCann",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Currently used metrics for assessing summarization algorithms do not account\nfor whether summaries are factually consistent with source documents. We\npropose a weakly-supervised, model-based approach for verifying factual\nconsistency and identifying conflicts between source documents and a generated\nsummary. Training data is generated by applying a series of rule-based\ntransformations to the sentences of source documents. The factual consistency\nmodel is then trained jointly for three tasks: 1) identify whether sentences\nremain factually consistent after transformation, 2) extract a span in the\nsource documents to support the consistency prediction, 3) extract a span in\nthe summary sentence that is inconsistent if one exists. Transferring this\nmodel to summaries generated by several state-of-the art models reveals that\nthis highly scalable approach substantially outperforms previous models,\nincluding those trained with strong supervision using standard datasets for\nnatural language inference and fact checking. Additionally, human evaluation\nshows that the auxiliary span extraction tasks provide useful assistance in the\nprocess of verifying factual consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13008v1",
            "date": "2019-10-28 23:49:26+00:00",
            "title": "Sketch-Fill-A-R: A Persona-Grounded Chit-Chat Generation Framework",
            "authors": [
                "Michael Shum",
                "Stephan Zheng",
                "Wojciech Kry\u015bci\u0144ski",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "Human-like chit-chat conversation requires agents to generate responses that\nare fluent, engaging and consistent. We propose Sketch-Fill-A-R, a framework\nthat uses a persona-memory to generate chit-chat responses in three phases.\nFirst, it generates dynamic sketch responses with open slots. Second, it\ngenerates candidate responses by filling slots with parts of its stored persona\ntraits. Lastly, it ranks and selects the final response via a language model\nscore. Sketch-Fill-A-R outperforms a state-of-the-art baseline both\nquantitatively (10-point lower perplexity) and qualitatively (preferred by 55%\nheads-up in single-turn and 20% higher in consistency in multi-turn user\nstudies) on the Persona-Chat dataset. Finally, we extensively analyze\nSketch-Fill-A-R's responses and human feedback, and show it is more consistent\nand engaging by using more relevant responses and questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13106v1",
            "date": "2019-10-29 06:43:51+00:00",
            "title": "Incorporating Interlocutor-Aware Context into Response Generation on Multi-Party Chatbots",
            "authors": [
                "Cao Liu",
                "Kang Liu",
                "Shizhu He",
                "Zaiqing Nie",
                "Jun Zhao"
            ],
            "abstract": "Conventional chatbots focus on two-party response generation, which\nsimplifies the real dialogue scene. In this paper, we strive toward a novel\ntask of Response Generation on Multi-Party Chatbot (RGMPC), where the generated\nresponses heavily rely on the interlocutors' roles (e.g., speaker and\naddressee) and their utterances. Unfortunately, complex interactions among the\ninterlocutors' roles make it challenging to precisely capture conversational\ncontexts and interlocutors' information. Facing this challenge, we present a\nresponse generation model which incorporates Interlocutor-aware Contexts into\nRecurrent Encoder-Decoder frameworks (ICRED) for RGMPC. Specifically, we employ\ninteractive representations to capture dialogue contexts for different\ninterlocutors. Moreover, we leverage an addressee memory to enhance contextual\ninterlocutor information for the target addressee. Finally, we construct a\ncorpus for RGMPC based on an existing open-access dataset. Automatic and manual\nevaluations demonstrate that the ICRED remarkably outperforms strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13215v3",
            "date": "2019-10-29 11:56:12+00:00",
            "title": "Transformer-based Cascaded Multimodal Speech Translation",
            "authors": [
                "Zixiu Wu",
                "Ozan Caglayan",
                "Julia Ive",
                "Josiah Wang",
                "Lucia Specia"
            ],
            "abstract": "This paper describes the cascaded multimodal speech translation systems\ndeveloped by Imperial College London for the IWSLT 2019 evaluation campaign.\nThe architecture consists of an automatic speech recognition (ASR) system\nfollowed by a Transformer-based multimodal machine translation (MMT) system.\nWhile the ASR component is identical across the experiments, the MMT model\nvaries in terms of the way of integrating the visual context (simple\nconditioning vs. attention), the type of visual features exploited (pooled,\nconvolutional, action categories) and the underlying architecture. For the\nlatter, we explore both the canonical transformer and its deliberation version\nwith additive and cascade variants which differ in how they integrate the\ntextual attention. Upon conducting extensive experiments, we found that (i) the\nexplored visual integration schemes often harm the translation performance for\nthe transformer and additive deliberation, but considerably improve the cascade\ndeliberation; (ii) the transformer and cascade deliberation integrate the\nvisual modality better than the additive deliberation, as shown by the\nincongruence analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13267v2",
            "date": "2019-10-29 13:42:56+00:00",
            "title": "BPE-Dropout: Simple and Effective Subword Regularization",
            "authors": [
                "Ivan Provilkov",
                "Dmitrii Emelianenko",
                "Elena Voita"
            ],
            "abstract": "Subword segmentation is widely used to address the open vocabulary problem in\nmachine translation. The dominant approach to subword segmentation is Byte Pair\nEncoding (BPE), which keeps the most frequent words intact while splitting the\nrare ones into multiple tokens. While multiple segmentations are possible even\nwith the same vocabulary, BPE splits words into unique sequences; this may\nprevent a model from better learning the compositionality of words and being\nrobust to segmentation errors. So far, the only way to overcome this BPE\nimperfection, its deterministic nature, was to create another subword\nsegmentation algorithm (Kudo, 2018). In contrast, we show that BPE itself\nincorporates the ability to produce multiple segmentations of the same word. We\nintroduce BPE-dropout - simple and effective subword regularization method\nbased on and compatible with conventional BPE. It stochastically corrupts the\nsegmentation procedure of BPE, which leads to producing multiple segmentations\nwithin the same fixed BPE framework. Using BPE-dropout during training and the\nstandard BPE during inference improves translation quality up to 3 BLEU\ncompared to BPE and up to 0.9 BLEU compared to the previous subword\nregularization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13299v2",
            "date": "2019-10-29 14:41:19+00:00",
            "title": "Findings of the Third Workshop on Neural Generation and Translation",
            "authors": [
                "Hiroaki Hayashi",
                "Yusuke Oda",
                "Alexandra Birch",
                "Ioannis Konstas",
                "Andrew Finch",
                "Minh-Thang Luong",
                "Graham Neubig",
                "Katsuhito Sudoh"
            ],
            "abstract": "This document describes the findings of the Third Workshop on Neural\nGeneration and Translation, held in concert with the annual conference of the\nEmpirical Methods in Natural Language Processing (EMNLP 2019). First, we\nsummarize the research trends of papers presented in the proceedings. Second,\nwe describe the results of the two shared tasks 1) efficient neural machine\ntranslation (NMT) where participants were tasked with creating NMT systems that\nare both accurate and efficient, and 2) document-level generation and\ntranslation (DGT) where participants were tasked with developing systems that\ngenerate summaries from structured data, potentially with assistance from text\nin another language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13497v1",
            "date": "2019-10-29 19:44:17+00:00",
            "title": "Quantifying the Semantic Core of Gender Systems",
            "authors": [
                "Adina Williams",
                "Ryan Cotterell",
                "Lawrence Wolf-Sonkin",
                "Dami\u00e1n Blasi",
                "Hanna Wallach"
            ],
            "abstract": "Many of the world's languages employ grammatical gender on the lexeme. For\nexample, in Spanish, the word for 'house' (casa) is feminine, whereas the word\nfor 'paper' (papel) is masculine. To a speaker of a genderless language, this\nassignment seems to exist with neither rhyme nor reason. But is the assignment\nof inanimate nouns to grammatical genders truly arbitrary? We present the first\nlarge-scale investigation of the arbitrariness of noun-gender assignments. To\nthat end, we use canonical correlation analysis to correlate the grammatical\ngender of inanimate nouns with an externally grounded definition of their\nlexical semantics. We find that 18 languages exhibit a significant correlation\nbetween grammatical gender and lexical semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13664v2",
            "date": "2019-10-30 04:47:17+00:00",
            "title": "Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models",
            "authors": [
                "Andriy Mulyar",
                "Elliot Schumacher",
                "Masoud Rouhizadeh",
                "Mark Dredze"
            ],
            "abstract": "Clinical notes contain an extensive record of a patient's health status, such\nas smoking status or the presence of heart conditions. However, this detail is\nnot replicated within the structured data of electronic health systems.\nPhenotyping, the extraction of patient conditions from free clinical text, is a\ncritical task which supports avariety of downstream applications such as\ndecision support and secondary use of medical records. Previous work has\nresulted in systems which are high performing but require hand engineering,\noften of rules. Recent work in pretrained contextualized language models have\nenabled advances in representing text for a variety of tasks. We therefore\nexplore several architectures for modeling pheno-typing that rely solely on\nBERT representations of the clinical note, removing the need for manual\nengineering. We find these architectures are competitive with or outperform\nexisting state of the art methods on two phenotyping tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13732v1",
            "date": "2019-10-30 09:28:49+00:00",
            "title": "LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese",
            "authors": [
                "Binh Duc Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In Vietnamese dependency parsing, several methods have been proposed.\nDependency parser which uses deep neural network model has been reported that\nachieved state-of-the-art results. In this paper, we proposed a new method\nwhich applies LSTM easy-first dependency parsing with pre-trained word\nembeddings and character-level word embeddings. Our method achieves an accuracy\nof 80.91% of unlabeled attachment score and 72.98% of labeled attachment score\non the Vietnamese Dependency Treebank (VnDT).",
            "journal ref": "10th International Conference on Knowledge and Systems Engineering\n  (KSE). IEEE, 2018. p. 187-192",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13793v1",
            "date": "2019-10-30 12:11:36+00:00",
            "title": "Time to Take Emoji Seriously: They Vastly Improve Casual Conversational Models",
            "authors": [
                "Pieter Delobelle",
                "Bettina Berendt"
            ],
            "abstract": "Graphical emoji are ubiquitous in modern-day online conversations. So is a\nsingle thumbs-up emoji able to signify an agreement, without any words. We\nargue that the current state-of-the-art systems are ill-equipped to correctly\ninterpret these emoji, especially in a conversational context. However, in a\ncasual context, the benefits might be high: a better understanding of users'\nutterances and more natural, emoji-rich responses.\n  With this in mind, we modify BERT to fully support emoji, both from the\nUnicode Standard and custom emoji. This modified BERT is then trained on a\ncorpus of question-answer (QA) tuples with a high number of emoji, where we're\nable to increase the 1-of-100 accuracy from 12.7% for the current\nstate-of-the-art to 17.8% for our model with emoji support.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13794v1",
            "date": "2019-10-30 12:14:11+00:00",
            "title": "Let Me Know What to Ask: Interrogative-Word-Aware Question Generation",
            "authors": [
                "Junmo Kang",
                "Haritz Puerto San Roman",
                "Sung-Hyon Myaeng"
            ],
            "abstract": "Question Generation (QG) is a Natural Language Processing (NLP) task that\naids advances in Question Answering (QA) and conversational assistants.\nExisting models focus on generating a question based on a text and possibly the\nanswer to the generated question. They need to determine the type of\ninterrogative word to be generated while having to pay attention to the grammar\nand vocabulary of the question. In this work, we propose\nInterrogative-Word-Aware Question Generation (IWAQG), a pipelined system\ncomposed of two modules: an interrogative word classifier and a QG model. The\nfirst module predicts the interrogative word that is provided to the second\nmodule to create the question. Owing to an increased recall of deciding the\ninterrogative words to be used for the generated questions, the proposed model\nachieves new state-of-the-art results on the task of QG in SQuAD, improving\nfrom 46.58 to 47.69 in BLEU-1, 17.55 to 18.53 in BLEU-4, 21.24 to 22.33 in\nMETEOR, and from 44.53 to 46.94 in ROUGE-L.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13826v4",
            "date": "2019-10-30 13:06:23+00:00",
            "title": "A Framework for Building Closed-Domain Chat Dialogue Systems",
            "authors": [
                "Mikio Nakano",
                "Kazunori Komatani"
            ],
            "abstract": "This paper presents HRIChat, a framework for developing closed-domain chat\ndialogue systems. Being able to engage in chat dialogues has been found\neffective for improving communication between humans and dialogue systems. This\npaper focuses on closed-domain systems because they would be useful when\ncombined with task-oriented dialogue systems in the same domain. HRIChat\nenables domain-dependent language understanding so that it can deal well with\ndomain-specific utterances. In addition, HRIChat makes it possible to integrate\nstate transition network-based dialogue management and reaction-based dialogue\nmanagement. FoodChatbot, which is an application in the food and restaurant\ndomain, has been developed and evaluated through a user study. Its results\nsuggest that reasonably good systems can be developed with HRIChat. This paper\nalso reports lessons learned from the development and evaluation of\nFoodChatbot.",
            "journal ref": "Knowledge-Based Systems, Volume 204, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13890v3",
            "date": "2019-10-30 14:29:47+00:00",
            "title": "A Latent Morphology Model for Open-Vocabulary Neural Machine Translation",
            "authors": [
                "Duygu Ataman",
                "Wilker Aziz",
                "Alexandra Birch"
            ],
            "abstract": "Translation into morphologically-rich languages challenges neural machine\ntranslation (NMT) models with extremely sparse vocabularies where atomic\ntreatment of surface forms is unrealistic. This problem is typically addressed\nby either pre-processing words into subword units or performing translation\ndirectly at the level of characters. The former is based on word segmentation\nalgorithms optimized using corpus-level statistics with no regard to the\ntranslation task. The latter learns directly from translation data but requires\nrather deep architectures. In this paper, we propose to translate words by\nmodeling word formation through a hierarchical latent variable model which\nmimics the process of morphological inflection. Our model generates words one\ncharacter at a time by composing two latent representations: a continuous one,\naimed at capturing the lexical semantics, and a set of (approximately) discrete\nfeatures, aimed at capturing the morphosyntactic function, which are shared\namong different surface forms. Our model achieves better accuracy in\ntranslation into three morphologically-rich languages than conventional\nopen-vocabulary NMT methods, while also demonstrating a better generalization\ncapacity under low to mid-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13913v4",
            "date": "2019-10-30 14:59:56+00:00",
            "title": "Toward Gender-Inclusive Coreference Resolution",
            "authors": [
                "Yang Trista Cao",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "Correctly resolving textual mentions of people fundamentally entails making\ninferences about those people. Such inferences raise the risk of systemic\nbiases in coreference resolution systems, including biases that can harm binary\nand non-binary trans and cis stakeholders. To better understand such biases, we\nforeground nuanced conceptualizations of gender from sociology and\nsociolinguistics, and develop two new datasets for interrogating bias in crowd\nannotations and in existing coreference resolution systems. Through these\nstudies, conducted on English text, we confirm that without acknowledging and\nbuilding systems that recognize the complexity of gender, we build systems that\nlead to many potential harms.",
            "journal ref": "Association for Computational Linguistics. Proceedings of the 58th\n  Annual Meeting of the Association for Computational Linguistics (2020)\n  4568-4595",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13946v1",
            "date": "2019-10-30 16:00:54+00:00",
            "title": "Let's FACE it. Finnish Poetry Generation with Aesthetics and Framing",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Khalid Alnajjar"
            ],
            "abstract": "We present a creative poem generator for the morphologically rich Finnish\nlanguage. Our method falls into the master-apprentice paradigm, where a\ncomputationally creative genetic algorithm teaches a BRNN model to generate\npoetry. We model several parts of poetic aesthetics in the fitness function of\nthe genetic algorithm, such as sonic features, semantic coherence, imagery and\nmetaphor. Furthermore, we justify the creativity of our method based on the\nFACE theory on computational creativity and take additional care in evaluating\nour system by automatic metrics for concepts together with human evaluation for\naesthetics, framing and expressions.",
            "journal ref": "Proceedings of the 12th International Conference on Natural\n  Language Generation (INLG 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13998v1",
            "date": "2019-10-30 17:34:46+00:00",
            "title": "Adapting Multilingual Neural Machine Translation to Unseen Languages",
            "authors": [
                "Surafel M. Lakew",
                "Alina Karakanta",
                "Marcello Federico",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Multilingual Neural Machine Translation (MNMT) for low-resource languages\n(LRL) can be enhanced by the presence of related high-resource languages (HRL),\nbut the relatedness of HRL usually relies on predefined linguistic assumptions\nabout language similarity. Recently, adapting MNMT to a LRL has shown to\ngreatly improve performance. In this work, we explore the problem of adapting\nan MNMT model to an unseen LRL using data selection and model adaptation. In\norder to improve NMT for LRL, we employ perplexity to select HRL data that are\nmost similar to the LRL on the basis of language distance. We extensively\nexplore data selection in popular multilingual NMT settings, namely in\n(zero-shot) translation, and in adaptation from a multilingual pre-trained\nmodel, for both directions (LRL-en). We further show that dynamic adaptation of\nthe model's vocabulary results in a more favourable segmentation for the LRL in\ncomparison with direct adaptation. Experiments show reductions in training time\nand significant performance gains over LRL baselines, even with zero LRL data\n(+13.0 BLEU), up to +17.0 BLEU for pre-trained multilingual model dynamic\nadaptation with related data selection. Our method outperforms current\napproaches, such as massively multilingual models and data augmentation, on\nfour LRL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14075v1",
            "date": "2019-10-30 18:38:17+00:00",
            "title": "Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation",
            "authors": [
                "S\u00e9bastien Jean",
                "Ankur Bapna",
                "Orhan Firat"
            ],
            "abstract": "Most neural machine translation systems still translate sentences in\nisolation. To make further progress, a promising line of research additionally\nconsiders the surrounding context in order to provide the model potentially\nmissing source-side information, as well as to maintain a coherent output. One\ndifficulty in training such larger-context (i.e. document-level) machine\ntranslation systems is that context may be missing from many parallel examples.\nTo circumvent this issue, two-stage approaches, in which sentence-level\ntranslations are post-edited in context, have recently been proposed. In this\npaper, we instead consider the viability of filling in the missing context. In\nparticular, we consider three distinct approaches to generate the missing\ncontext: using random contexts, applying a copy heuristic or generating it with\na language model. In particular, the copy heuristic significantly helps with\nlexical coherence, while using completely random contexts hurts performance on\nmany long-distance linguistic phenomena. We also validate the usefulness of\ntagged back-translation. In addition to improving BLEU scores as expected,\nusing back-translated data helps larger-context machine translation systems to\nbetter capture long-range phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14076v1",
            "date": "2019-10-30 18:39:46+00:00",
            "title": "A Neural Topic-Attention Model for Medical Term Abbreviation Disambiguation",
            "authors": [
                "Irene Li",
                "Michihiro Yasunaga",
                "Muhammed Yavuz Nuzumlal\u0131",
                "Cesar Caraballo",
                "Shiwani Mahajan",
                "Harlan Krumholz",
                "Dragomir Radev"
            ],
            "abstract": "Automated analysis of clinical notes is attracting increasing attention.\nHowever, there has not been much work on medical term abbreviation\ndisambiguation. Such abbreviations are abundant, and highly ambiguous, in\nclinical documents. One of the main obstacles is the lack of large scale,\nbalance labeled data sets. To address the issue, we propose a few-shot learning\napproach to take advantage of limited labeled data. Specifically, a neural\ntopic-attention model is applied to learn improved contextualized sentence\nrepresentations for medical term abbreviation disambiguation. Another vital\nissue is that the existing scarce annotations are noisy and missing. We\nre-examine and correct an existing dataset for training and collect a test set\nto evaluate the models fairly especially for rare senses. We train our model on\nthe training set which contains 30 abbreviation terms as categories (on\naverage, 479 samples and 3.24 classes in each term) selected from a public\nabbreviation disambiguation dataset, and then test on a manually-created\nbalanced dataset (each class in each term has 15 samples). We show that\nenhancing the sentence representation with topic information improves the\nperformance on small-scale unbalanced training datasets by a large margin,\ncompared to a number of baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14087v1",
            "date": "2019-10-30 19:07:31+00:00",
            "title": "Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering",
            "authors": [
                "Kaixin Ma",
                "Jonathan Francis",
                "Quanyang Lu",
                "Eric Nyberg",
                "Alessandro Oltramari"
            ],
            "abstract": "Non-extractive commonsense QA remains a challenging AI task, as it requires\nsystems to reason about, synthesize, and gather disparate pieces of\ninformation, in order to generate responses to queries. Recent approaches on\nsuch tasks show increased performance, only when models are either pre-trained\nwith additional information or when domain-specific heuristics are used,\nwithout any special consideration regarding the knowledge resource type. In\nthis paper, we perform a survey of recent commonsense QA methods and we provide\na systematic analysis of popular knowledge resources and knowledge-integration\nmethods, across benchmarks from multiple commonsense datasets. Our results and\nanalysis show that attention-based injection seems to be a preferable choice\nfor knowledge integration and that the degree of domain overlap, between\nknowledge bases and datasets, plays a crucial role in determining model\nsuccess.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14142v2",
            "date": "2019-10-30 21:17:26+00:00",
            "title": "Discourse-Aware Neural Extractive Text Summarization",
            "authors": [
                "Jiacheng Xu",
                "Zhe Gan",
                "Yu Cheng",
                "Jingjing Liu"
            ],
            "abstract": "Recently BERT has been adopted for document encoding in state-of-the-art text\nsummarization models. However, sentence-based extractive models often result in\nredundant or uninformative phrases in the extracted summaries. Also, long-range\ndependencies throughout a document are not well captured by BERT, which is\npre-trained on sentence pairs instead of documents. To address these issues, we\npresent a discourse-aware neural summarization model - DiscoBert. DiscoBert\nextracts sub-sentential discourse units (instead of sentences) as candidates\nfor extractive selection on a finer granularity. To capture the long-range\ndependencies among discourse units, structural discourse graphs are constructed\nbased on RST trees and coreference mentions, encoded with Graph Convolutional\nNetworks. Experiments show that the proposed model outperforms state-of-the-art\nmethods by a significant margin on popular summarization benchmarks compared to\nother BERT-base models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14161v1",
            "date": "2019-10-30 22:28:14+00:00",
            "title": "How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?",
            "authors": [
                "Hila Gonen",
                "Yova Kementchedjhieva",
                "Yoav Goldberg"
            ],
            "abstract": "Many natural languages assign grammatical gender also to inanimate nouns in\nthe language. In such languages, words that relate to the gender-marked nouns\nare inflected to agree with the noun's gender. We show that this affects the\nword representations of inanimate nouns, resulting in nouns with the same\ngender being closer to each other than nouns with different gender. While\n\"embedding debiasing\" methods fail to remove the effect, we demonstrate that a\ncareful application of methods that neutralize grammatical gender signals from\nthe words' context when training word embeddings is effective in removing it.\nFixing the grammatical gender bias yields a positive effect on the quality of\nthe resulting word embeddings, both in monolingual and cross-lingual settings.\nWe note that successfully removing gender signals, while achievable, is not\ntrivial to do and that a language-specific morphological analyzer, together\nwith careful usage of it, are essential for achieving good results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14176v1",
            "date": "2019-10-30 23:15:24+00:00",
            "title": "Predicting Discourse Structure using Distant Supervision from Sentiment",
            "authors": [
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "Discourse parsing could not yet take full advantage of the neural NLP\nrevolution, mostly due to the lack of annotated datasets. We propose a novel\napproach that uses distant supervision on an auxiliary task (sentiment\nclassification), to generate abundant data for RST-style discourse structure\nprediction. Our approach combines a neural variant of multiple-instance\nlearning, using document-level supervision, with an optimal CKY-style tree\ngeneration algorithm. In a series of experiments, we train a discourse parser\n(for only structure prediction) on our automatically generated dataset and\ncompare it with parsers trained on human-annotated corpora (news domain RST-DT\nand Instructional domain). Results indicate that while our parser does not yet\nmatch the performance of a parser trained and tested on the same dataset\n(intra-domain), it does perform remarkably well on the much more difficult and\narguably more useful task of inter-domain discourse structure prediction, where\nthe parser is trained on one domain and tested/applied on another one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14192v1",
            "date": "2019-10-31 00:22:27+00:00",
            "title": "Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning",
            "authors": [
                "Zheng Li",
                "Xin Li",
                "Ying Wei",
                "Lidong Bing",
                "Yu Zhang",
                "Qiang Yang"
            ],
            "abstract": "Joint extraction of aspects and sentiments can be effectively formulated as a\nsequence labeling problem. However, such formulation hinders the effectiveness\nof supervised methods due to the lack of annotated sequence data in many\ndomains. To address this issue, we firstly explore an unsupervised domain\nadaptation setting for this task. Prior work can only use common syntactic\nrelations between aspect and opinion words to bridge the domain gaps, which\nhighly relies on external linguistic resources. To resolve it, we propose a\nnovel Selective Adversarial Learning (SAL) method to align the inferred\ncorrelation vectors that automatically capture their latent relations. The SAL\nmethod can dynamically learn an alignment weight for each word such that more\nimportant words can possess higher alignment weights to achieve fine-grained\n(word-level) adaptation. Empirically, extensive experiments demonstrate the\neffectiveness of the proposed SAL method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14254v2",
            "date": "2019-10-31 04:35:17+00:00",
            "title": "Harnessing the linguistic signal to predict scalar inferences",
            "authors": [
                "Sebastian Schuster",
                "Yuxing Chen",
                "Judith Degen"
            ],
            "abstract": "Pragmatic inferences often subtly depend on the presence or absence of\nlinguistic features. For example, the presence of a partitive construction (of\nthe) increases the strength of a so-called scalar inference: listeners perceive\nthe inference that Chris did not eat all of the cookies to be stronger after\nhearing \"Chris ate some of the cookies\" than after hearing the same utterance\nwithout a partitive, \"Chris ate some cookies.\" In this work, we explore to what\nextent neural network sentence encoders can learn to predict the strength of\nscalar inferences. We first show that an LSTM-based sentence encoder trained on\nan English dataset of human inference strength ratings is able to predict\nratings with high accuracy (r=0.78). We then probe the model's behavior using\nmanually constructed minimal sentence pairs and corpus data. We find that the\nmodel inferred previously established associations between linguistic features\nand inference strength, suggesting that the model learns to use linguistic\nfeatures to predict pragmatic inferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14286v1",
            "date": "2019-10-31 07:50:41+00:00",
            "title": "A neural document language modeling framework for spoken document retrieval",
            "authors": [
                "Li-Phen Yen",
                "Zhen-Yu Wu",
                "Kuan-Yu Chen"
            ],
            "abstract": "Recent developments in deep learning have led to a significant innovation in\nvarious classic and practical subjects, including speech recognition, computer\nvision, question answering, information retrieval and so on. In the context of\nnatural language processing (NLP), language representations have shown giant\nsuccesses in many downstream tasks, so the school of studies have become a\nmajor stream of research recently. Because the immenseness of multimedia data\nalong with speech have spread around the world in our daily life, spoken\ndocument retrieval (SDR) has become an important research subject in the past\ndecades. Targeting on enhancing the SDR performance, the paper concentrates on\nproposing a neural retrieval framework, which assembles the merits of using\nlanguage modeling (LM) mechanism in SDR and leveraging the abstractive\ninformation learned by the language representation models. Consequently, to our\nknowledge, this is a pioneer study on supervised training of a neural LM-based\nSDR framework, especially combined with the pretrained language representation\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14326v2",
            "date": "2019-10-31 09:25:34+00:00",
            "title": "Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks",
            "authors": [
                "Yiping Song",
                "Zequn Liu",
                "Wei Bi",
                "Rui Yan",
                "Ming Zhang"
            ],
            "abstract": "Training the generative models with minimal corpus is one of the critical\nchallenges for building open-domain dialogue systems. Existing methods tend to\nuse the meta-learning framework which pre-trains the parameters on all\nnon-target tasks then fine-tunes on the target task. However, fine-tuning\ndistinguishes tasks from the parameter perspective but ignores the\nmodel-structure perspective, resulting in similar dialogue models for different\ntasks. In this paper, we propose an algorithm that can customize a unique\ndialogue model for each task in the few-shot setting. In our approach, each\ndialogue model consists of a shared module, a gating module, and a private\nmodule. The first two modules are shared among all the tasks, while the third\none will differentiate into different network structures to better capture the\ncharacteristics of the corresponding task. The extensive experiments on two\ndatasets show that our method outperforms all the baselines in terms of task\nconsistency, response quality, and diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14520v1",
            "date": "2019-10-31 15:02:49+00:00",
            "title": "Do Multi-hop Readers Dream of Reasoning Chains?",
            "authors": [
                "Haoyu Wang",
                "Mo Yu",
                "Xiaoxiao Guo",
                "Rajarshi Das",
                "Wenhan Xiong",
                "Tian Gao"
            ],
            "abstract": "General Question Answering (QA) systems over texts require the multi-hop\nreasoning capability, i.e. the ability to reason with information collected\nfrom multiple passages to derive the answer. In this paper we conduct a\nsystematic analysis to assess such an ability of various existing models\nproposed for multi-hop QA tasks. Specifically, our analysis investigates that\nwhether providing the full reasoning chain of multiple passages, instead of\njust one final passage where the answer appears, could improve the performance\nof the existing QA models. Surprisingly, when using the additional evidence\npassages, the improvements of all the existing multi-hop reading approaches are\nrather limited, with the highest error reduction of 5.8% on F1 (corresponding\nto 1.3% absolute improvement) from the BERT model.\n  To better understand whether the reasoning chains could indeed help find\ncorrect answers, we further develop a co-matching-based method that leads to\n13.1% error reduction with passage chains when applied to two of our base\nreaders (including BERT). Our results demonstrate the existence of the\npotential improvement using explicit multi-hop reasoning and the necessity to\ndevelop models with better reasoning abilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14528v1",
            "date": "2019-10-31 15:14:54+00:00",
            "title": "Document-level Neural Machine Translation with Inter-Sentence Attention",
            "authors": [
                "Shu Jiang",
                "Rui Wang",
                "Zuchao Li",
                "Masao Utiyama",
                "Kehai Chen",
                "Eiichiro Sumita",
                "Hai Zhao",
                "Bao-liang Lu"
            ],
            "abstract": "Standard neural machine translation (NMT) is on the assumption of\ndocument-level context independent. Most existing document-level NMT methods\nonly focus on briefly introducing document-level information but fail to\nconcern about selecting the most related part inside document context. The\ncapacity of memory network for detecting the most relevant part of the current\nsentence from the memory provides a natural solution for the requirement of\nmodeling document-level context by document-level NMT. In this work, we propose\na Transformer NMT system with associated memory network (AMN) to both capture\nthe document-level context and select the most salient part related to the\nconcerned translation from the memory. Experiments on several tasks show that\nthe proposed method significantly improves the NMT performance over strong\nTransformer baselines and other related studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14537v3",
            "date": "2019-10-31 15:32:19+00:00",
            "title": "Attention Is All You Need for Chinese Word Segmentation",
            "authors": [
                "Sufeng Duan",
                "Hai Zhao"
            ],
            "abstract": "Taking greedy decoding algorithm as it should be, this work focuses on\nfurther strengthening the model itself for Chinese word segmentation (CWS),\nwhich results in an even more fast and more accurate CWS model. Our model\nconsists of an attention only stacked encoder and a light enough decoder for\nthe greedy segmentation plus two highway connections for smoother training, in\nwhich the encoder is composed of a newly proposed Transformer variant,\nGaussian-masked Directional (GD) Transformer, and a biaffine attention scorer.\nWith the effective encoder design, our model only needs to take unigram\nfeatures for scoring. Our model is evaluated on SIGHAN Bakeoff benchmark\ndatasets. The experimental results show that with the highest segmentation\nspeed, the proposed model achieves new state-of-the-art or comparable\nperformance against strong baselines in terms of strict closed test setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14539v1",
            "date": "2019-10-31 15:34:48+00:00",
            "title": "Naver Labs Europe's Systems for the Document-Level Generation and Translation Task at WNGT 2019",
            "authors": [
                "Fahimeh Saleh",
                "Alexandre B\u00e9rard",
                "Ioan Calapodescu",
                "Laurent Besacier"
            ],
            "abstract": "Recently, neural models led to significant improvements in both machine\ntranslation (MT) and natural language generation tasks (NLG). However,\ngeneration of long descriptive summaries conditioned on structured data remains\nan open challenge. Likewise, MT that goes beyond sentence-level context is\nstill an open issue (e.g., document-level MT or MT with metadata). To address\nthese challenges, we propose to leverage data from both tasks and do transfer\nlearning between MT, NLG, and MT with source-side metadata (MT+NLG). First, we\ntrain document-based MT systems with large amounts of parallel data. Then, we\nadapt these models to pure NLG and MT+NLG tasks by fine-tuning with smaller\namounts of domain-specific data. This end-to-end NLG approach, without data\nselection and planning, outperforms the previous state of the art on the\nRotowire NLG task. We participated to the \"Document Generation and Translation\"\ntask at WNGT 2019, and ranked first in all tracks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14589v1",
            "date": "2019-10-31 16:42:50+00:00",
            "title": "Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness",
            "authors": [
                "Alexandre B\u00e9rard",
                "Ioan Calapodescu",
                "Marc Dymetman",
                "Claude Roux",
                "Jean-Luc Meunier",
                "Vassilina Nikoulina"
            ],
            "abstract": "We share a French-English parallel corpus of Foursquare restaurant reviews\n(https://europe.naverlabs.com/research/natural-language-processing/machine-translation-of-restaurant-reviews),\nand define a new task to encourage research on Neural Machine Translation\nrobustness and domain adaptation, in a real-world scenario where better-quality\nMT would be greatly beneficial. We discuss the challenges of such\nuser-generated content, and train good baseline models that build upon the\nlatest techniques for MT robustness. We also perform an extensive evaluation\n(automatic and human) that shows significant improvements over existing online\nsystems. Finally, we propose task-specific metrics based on sentiment analysis\nor translation accuracy of domain-specific polysemous words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00133v1",
            "date": "2019-10-31 22:28:45+00:00",
            "title": "Dreaddit: A Reddit Dataset for Stress Analysis in Social Media",
            "authors": [
                "Elsbeth Turcan",
                "Kathleen McKeown"
            ],
            "abstract": "Stress is a nigh-universal human experience, particularly in the online\nworld. While stress can be a motivator, too much stress is associated with many\nnegative health outcomes, making its identification useful across a range of\ndomains. However, existing computational research typically only studies stress\nin domains such as speech, or in short genres such as Twitter. We present\nDreaddit, a new text corpus of lengthy multi-domain social media data for the\nidentification of stress. Our dataset consists of 190K posts from five\ndifferent categories of Reddit communities; we additionally label 3.5K total\nsegments taken from 3K posts using Amazon Mechanical Turk. We present\npreliminary supervised learning methods for identifying stress, both neural and\ntraditional, and analyze the complexity and diversity of the data and\ncharacteristics of each category.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00152v1",
            "date": "2019-10-31 23:54:41+00:00",
            "title": "Implementation of an Index Optimize Technology for Highly Specialized Terms based on the Phonetic Algorithm Metaphone",
            "authors": [
                "V. Buriachok",
                "M. Hadzhyiev",
                "V. Sokolov",
                "P. Skladannyi",
                "L. Kuzmenko"
            ],
            "abstract": "When compiling databases, for example to meet the needs of healthcare\nestablishments, there is quite a common problem with the introduction and\nfurther processing of names and last names of doctors and patients that are\nhighly specialized both in terms of pronunciation and writing. This is because\nnames and last names of people cannot be unique, their notation is not subject\nto any rules of phonetics, while their length in different languages may not\nmatch. With the advent of the Internet, this situation has become generally\ncritical and can lead to that multiple copies of e-mails are sent to one\naddress. It is possible to solve the specified problem by using phonetic\nalgorithms for comparing words Daitch-Mokotoff, Soundex, NYSIIS, Polyphone, and\nMetaphone, as well as the Levenshtein and Jaro algorithms, Q-gram-based\nalgorithms, which make it possible to find distances between words. The most\nwidespread among them are the Soundex and Metaphone algorithms, which are\ndesigned to index the words based on their sound, taking into consideration the\nrules of pronunciation. By applying the Metaphone algorithm, an attempt has\nbeen made to optimize the phonetic search processes for tasks of fuzzy\ncoincidence, for example, at data deduplication in various databases and\nregistries, in order to reduce the number of errors of incorrect input of last\nnames. An analysis of the most common last names reveals that some of them are\nof the Ukrainian or Russian origin. At the same time, the rules following which\nthe names are pronounced and written, for example in Ukrainian, differ\nradically from basic algorithms for English and differ quite significantly for\nthe Russian language. That is why a phonetic algorithm should take into\nconsideration first of all the peculiarities in the formation of Ukrainian last\nnames, which is of special relevance now.",
            "journal ref": "Eastern-European Journal of Enterprise Technologies (ISSN:\n  1729-4061). 2019. Vol. 5. No. 2(101)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00172v2",
            "date": "2019-11-01 01:09:53+00:00",
            "title": "Generalization through Memorization: Nearest Neighbor Language Models",
            "authors": [
                "Urvashi Khandelwal",
                "Omer Levy",
                "Dan Jurafsky",
                "Luke Zettlemoyer",
                "Mike Lewis"
            ],
            "abstract": "We introduce $k$NN-LMs, which extend a pre-trained neural language model (LM)\nby linearly interpolating it with a $k$-nearest neighbors ($k$NN) model. The\nnearest neighbors are computed according to distance in the pre-trained LM\nembedding space, and can be drawn from any text collection, including the\noriginal LM training data. Applying this augmentation to a strong Wikitext-103\nLM, with neighbors drawn from the original training set, our $k$NN-LM achieves\na new state-of-the-art perplexity of 15.79 - a 2.9 point improvement with no\nadditional training. We also show that this approach has implications for\nefficiently scaling up to larger training sets and allows for effective domain\nadaptation, by simply varying the nearest neighbor datastore, again without\nfurther training. Qualitatively, the model is particularly helpful in\npredicting rare patterns, such as factual knowledge. Together, these results\nstrongly suggest that learning similarity between sequences of text is easier\nthan predicting the next word, and that nearest neighbor search is an effective\napproach for language modeling in the long tail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00176v1",
            "date": "2019-11-01 01:36:24+00:00",
            "title": "Sequence Modeling with Unconstrained Generation Order",
            "authors": [
                "Dmitrii Emelianenko",
                "Elena Voita",
                "Pavel Serdyukov"
            ],
            "abstract": "The dominant approach to sequence generation is to produce a sequence in some\npredefined order, e.g. left to right. In contrast, we propose a more general\nmodel that can generate the output sequence by inserting tokens in any\narbitrary order. Our model learns decoding order as a result of its training\nprocedure. Our experiments show that this model is superior to fixed order\nmodels on a number of sequence generation tasks, such as Machine Translation,\nImage-to-LaTeX and Image Captioning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00225v1",
            "date": "2019-11-01 06:48:07+00:00",
            "title": "When Choosing Plausible Alternatives, Clever Hans can be Clever",
            "authors": [
                "Pride Kavumba",
                "Naoya Inoue",
                "Benjamin Heinzerling",
                "Keshav Singh",
                "Paul Reisert",
                "Kentaro Inui"
            ],
            "abstract": "Pretrained language models, such as BERT and RoBERTa, have shown large\nimprovements in the commonsense reasoning benchmark COPA. However, recent work\nfound that many improvements in benchmarks of natural language understanding\nare not due to models learning the task, but due to their increasing ability to\nexploit superficial cues, such as tokens that occur more often in the correct\nanswer than the wrong one. Are BERT's and RoBERTa's good performance on COPA\nalso caused by this? We find superficial cues in COPA, as well as evidence that\nBERT exploits these cues. To remedy this problem, we introduce Balanced COPA,\nan extension of COPA that does not suffer from easy-to-exploit single token\ncues. We analyze BERT's and RoBERTa's performance on original and Balanced\nCOPA, finding that BERT relies on superficial cues when they are present, but\nstill achieves comparable performance once they are made ineffective,\nsuggesting that BERT learns the task to a certain degree when forced to. In\ncontrast, RoBERTa does not appear to rely on superficial cues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00269v1",
            "date": "2019-11-01 09:08:58+00:00",
            "title": "A Robust Data-Driven Approach for Dialogue State Tracking of Unseen Slot Values",
            "authors": [
                "Vevake Balaraman",
                "Bernardo Magnini"
            ],
            "abstract": "A Dialogue State Tracker is a key component in dialogue systems which\nestimates the beliefs of possible user goals at each dialogue turn. Deep\nlearning approaches using recurrent neural networks have shown state-of-the-art\nperformance for the task of dialogue state tracking. Generally, these\napproaches assume a predefined candidate list and struggle to predict any new\ndialogue state values that are not seen during training. This makes extending\nthe candidate list for a slot without model retaining infeasible and also has\nlimitations in modelling for low resource domains where training data for slot\nvalues are expensive. In this paper, we propose a novel dialogue state tracker\nbased on copying mechanism that can effectively track such unseen slot values\nwithout compromising performance on slot values seen during training. The\nproposed model is also flexible in extending the candidate list without\nrequiring any retraining or change in the model. We evaluate the proposed model\non various benchmark datasets (DSTC2, DSTC3 and WoZ2.0) and show that our\napproach, outperform other end-to-end data-driven approaches in tracking unseen\nslot values and also provides significant advantages in modelling for DST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00317v1",
            "date": "2019-11-01 12:13:45+00:00",
            "title": "On the Linguistic Representational Power of Neural Machine Translation Models",
            "authors": [
                "Yonatan Belinkov",
                "Nadir Durrani",
                "Fahim Dalvi",
                "Hassan Sajjad",
                "James Glass"
            ],
            "abstract": "Despite the recent success of deep neural networks in natural language\nprocessing (NLP), their interpretability remains a challenge. We analyze the\nrepresentations learned by neural machine translation models at various levels\nof granularity and evaluate their quality through relevant extrinsic\nproperties. In particular, we seek answers to the following questions: (i) How\naccurately is word-structure captured within the learned representations, an\nimportant aspect in translating morphologically-rich languages? (ii) Do the\nrepresentations capture long-range dependencies, and effectively handle\nsyntactically divergent languages? (iii) Do the representations capture lexical\nsemantics? We conduct a thorough investigation along several parameters: (i)\nWhich layers in the architecture capture each of these linguistic phenomena;\n(ii) How does the choice of translation unit (word, character, or subword unit)\nimpact the linguistic properties captured by the underlying representations?\n(iii) Do the encoder and decoder learn differently and independently? (iv) Do\nthe representations learned by multilingual NMT models capture the same amount\nof linguistic information as their bilingual counterparts? Our data-driven,\nquantitative evaluation illuminates important aspects in NMT models and their\nability to capture various linguistic phenomena. We show that deep NMT models\nlearn a non-trivial amount of linguistic information. Notable findings include:\ni) Word morphology and part-of-speech information are captured at the lower\nlayers of the model; (ii) In contrast, lexical semantics or non-local syntactic\nand semantic dependencies are better represented at the higher layers; (iii)\nRepresentations learned using characters are more informed about wordmorphology\ncompared to those learned using subword units; and (iv) Representations learned\nby multilingual models are richer compared to bilingual models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00337v3",
            "date": "2019-10-30 19:18:14+00:00",
            "title": "Ensembling Strategies for Answering Natural Questions",
            "authors": [
                "Anthony Ferritto",
                "Lin Pan",
                "Rishav Chakravarti",
                "Salim Roukos",
                "Radu Florian",
                "J. William Murdock",
                "Avirup Sil"
            ],
            "abstract": "Many of the top question answering systems today utilize ensembling to\nimprove their performance on tasks such as the Stanford Question Answering\nDataset (SQuAD) and Natural Questions (NQ) challenges. Unfortunately most of\nthese systems do not publish their ensembling strategies used in their\nleaderboard submissions. In this work, we investigate a number of ensembling\ntechniques and demonstrate a strategy which improves our F1 score for short\nanswers on the dev set for NQ by 2.3 F1 points over our single model (which\noutperforms the previous SOTA by 1.9 F1 points).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00484v4",
            "date": "2019-11-01 17:48:38+00:00",
            "title": "Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents",
            "authors": [
                "Ming Tu",
                "Kevin Huang",
                "Guangtao Wang",
                "Jing Huang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Interpretable multi-hop reading comprehension (RC) over multiple documents is\na challenging problem because it demands reasoning over multiple information\nsources and explaining the answer prediction by providing supporting evidences.\nIn this paper, we propose an effective and interpretable Select, Answer and\nExplain (SAE) system to solve the multi-document RC problem. Our system first\nfilters out answer-unrelated documents and thus reduce the amount of\ndistraction information. This is achieved by a document classifier trained with\na novel pairwise learning-to-rank loss. The selected answer-related documents\nare then input to a model to jointly predict the answer and supporting\nsentences. The model is optimized with a multi-task learning objective on both\ntoken level for answer prediction and sentence level for supporting sentences\nprediction, together with an attention-based interaction between these two\ntasks. Evaluated on HotpotQA, a challenging multi-hop RC data set, the proposed\nSAE system achieves top competitive performance in distractor setting compared\nto other existing systems on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00547v1",
            "date": "2019-11-01 18:38:27+00:00",
            "title": "Uncover Sexual Harassment Patterns from Personal Stories by Joint Key Element Extraction and Categorization",
            "authors": [
                "Yingchi Liu",
                "Quanzhi Li",
                "Marika Cifor",
                "Xiaozhong Liu",
                "Qiong Zhang",
                "Luo Si"
            ],
            "abstract": "The number of personal stories about sexual harassment shared online has\nincreased exponentially in recent years. This is in part inspired by the\n\\#MeToo and \\#TimesUp movements. Safecity is an online forum for people who\nexperienced or witnessed sexual harassment to share their personal experiences.\nIt has collected \\textgreater 10,000 stories so far. Sexual harassment occurred\nin a variety of situations, and categorization of the stories and extraction of\ntheir key elements will provide great help for the related parties to\nunderstand and address sexual harassment. In this study, we manually annotated\nthose stories with labels in the dimensions of location, time, and harassers'\ncharacteristics, and marked the key elements related to these dimensions.\nFurthermore, we applied natural language processing technologies with joint\nlearning schemes to automatically categorize these stories in those dimensions\nand extract key elements at the same time. We also uncovered significant\npatterns from the categorized sexual harassment stories. We believe our\nannotated data set, proposed algorithms, and analysis will help people who have\nbeen harassed, authorities, researchers and other related parties in various\nways, such as automatically filling reports, enlightening the public in order\nto prevent future harassment, and enabling more effective, faster action to be\ntaken.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00650v2",
            "date": "2019-11-02 04:52:00+00:00",
            "title": "Automatic Detection of Generated Text is Easiest when Humans are Fooled",
            "authors": [
                "Daphne Ippolito",
                "Daniel Duckworth",
                "Chris Callison-Burch",
                "Douglas Eck"
            ],
            "abstract": "Recent advancements in neural language modelling make it possible to rapidly\ngenerate vast amounts of human-sounding text. The capabilities of humans and\nautomatic discriminators to detect machine-generated text have been a large\nsource of research interest, but humans and machines rely on different cues to\nmake their decisions. Here, we perform careful benchmarking and analysis of\nthree popular sampling-based decoding strategies---top-$k$, nucleus sampling,\nand untruncated random sampling---and show that improvements in decoding\nmethods have primarily optimized for fooling humans. This comes at the expense\nof introducing statistical abnormalities that make detection easy for automatic\nsystems. We also show that though both human and automatic detector performance\nimprove with longer excerpt length, even multi-sentence excerpts can fool\nexpert human raters over 30% of the time. Our findings reveal the importance of\nusing both human and automatic detectors to assess the humanness of text\ngeneration systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00681v1",
            "date": "2019-11-02 08:45:24+00:00",
            "title": "Machine Translation Evaluation using Bi-directional Entailment",
            "authors": [
                "Rakesh Khobragade",
                "Heaven Patel",
                "Anand Namdev",
                "Anish Mishra",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we propose a new metric for Machine Translation (MT)\nevaluation, based on bi-directional entailment. We show that machine generated\ntranslation can be evaluated by determining paraphrasing with a reference\ntranslation provided by a human translator. We hypothesize, and show through\nexperiments, that paraphrasing can be detected by evaluating entailment\nrelationship in the forward and backward direction. Unlike conventional\nmetrics, like BLEU or METEOR, our approach uses deep learning to determine the\nsemantic similarity between candidate and reference translation for generating\nscores rather than relying upon simple n-gram overlap. We use BERT's\npre-trained implementation of transformer networks, fine-tuned on MNLI corpus,\nfor natural language inferencing. We apply our evaluation metric on WMT'14 and\nWMT'17 dataset to evaluate systems participating in the translation task and\nfind that our metric has a better correlation with the human annotated score\ncompared to the other traditional metrics at system level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00720v1",
            "date": "2019-11-02 14:32:08+00:00",
            "title": "ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations",
            "authors": [
                "Shizhe Diao",
                "Jiaxin Bai",
                "Yan Song",
                "Tong Zhang",
                "Yonggang Wang"
            ],
            "abstract": "The pre-training of text encoders normally processes text as a sequence of\ntokens corresponding to small text units, such as word pieces in English and\ncharacters in Chinese. It omits information carried by larger text granularity,\nand thus the encoders cannot easily adapt to certain combinations of\ncharacters. This leads to a loss of important semantic information, which is\nespecially problematic for Chinese because the language does not have explicit\nword boundaries. In this paper, we propose ZEN, a BERT-based Chinese (Z) text\nencoder Enhanced by N-gram representations, where different combinations of\ncharacters are considered during training. As a result, potential word or phase\nboundaries are explicitly pre-trained and fine-tuned with the character encoder\n(BERT). Therefore ZEN incorporates the comprehensive information of both the\ncharacter sequence and words or phrases it contains. Experimental results\nillustrated the effectiveness of ZEN on a series of Chinese NLP tasks. We show\nthat ZEN, using less resource than other published encoders, can achieve\nstate-of-the-art performance on most tasks. Moreover, it is shown that\nreasonable performance can be obtained when ZEN is trained on a small corpus,\nwhich is important for applying pre-training techniques to scenarios with\nlimited data. The code and pre-trained models of ZEN are available at\nhttps://github.com/sinovation/zen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00811v1",
            "date": "2019-11-03 02:47:51+00:00",
            "title": "Posing Fair Generalization Tasks for Natural Language Inference",
            "authors": [
                "Atticus Geiger",
                "Ignacio Cases",
                "Lauri Karttunen",
                "Chris Potts"
            ],
            "abstract": "Deep learning models for semantics are generally evaluated using naturalistic\ncorpora. Adversarial methods, in which models are evaluated on new examples\nwith known semantic properties, have begun to reveal that good performance at\nthese naturalistic tasks can hide serious shortcomings. However, we should\ninsist that these evaluations be fair -that the models are given data\nsufficient to support the requisite kinds of generalization. In this paper, we\ndefine and motivate a formal notion of fairness in this sense. We then apply\nthese ideas to natural language inference by constructing very challenging but\nprovably fair artificial datasets and showing that standard neural models fail\nto generalize in the required ways; only task-specific models that jointly\ncompose the premise and hypothesis are able to achieve high performance, and\neven these models do not solve the task perfectly.",
            "journal ref": "Proceedings of the 2019 conference on Empircal Methods in Natural\n  Lanuage Processing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00835v1",
            "date": "2019-11-03 05:41:53+00:00",
            "title": "Controlling Text Complexity in Neural Machine Translation",
            "authors": [
                "Sweta Agrawal",
                "Marine Carpuat"
            ],
            "abstract": "This work introduces a machine translation task where the output is aimed at\naudiences of different levels of target language proficiency. We collect a high\nquality dataset of news articles available in English and Spanish, written for\ndiverse grade levels and propose a method to align segments across comparable\nbilingual articles. The resulting dataset makes it possible to train multi-task\nsequence-to-sequence models that translate Spanish into English targeted at an\neasier reading grade level than the original Spanish. We show that these\nmulti-task models outperform pipeline approaches that translate and simplify\ntext independently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00841v1",
            "date": "2019-11-03 06:25:17+00:00",
            "title": "Question Answering for Privacy Policies: Combining Computational and Legal Perspectives",
            "authors": [
                "Abhilasha Ravichander",
                "Alan W Black",
                "Shomir Wilson",
                "Thomas Norton",
                "Norman Sadeh"
            ],
            "abstract": "Privacy policies are long and complex documents that are difficult for users\nto read and understand, and yet, they have legal effects on how user data is\ncollected, managed and used. Ideally, we would like to empower users to inform\nthemselves about issues that matter to them, and enable them to selectively\nexplore those issues. We present PrivacyQA, a corpus consisting of 1750\nquestions about the privacy policies of mobile applications, and over 3500\nexpert annotations of relevant answers. We observe that a strong neural\nbaseline underperforms human performance by almost 0.3 F1 on PrivacyQA,\nsuggesting considerable room for improvement for future systems. Further, we\nuse this dataset to shed light on challenges to question answerability, with\ndomain-general implications for any question answering system. The PrivacyQA\ncorpus offers a challenging corpus for question answering, with genuine\nreal-world utility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00932v1",
            "date": "2019-11-03 17:24:48+00:00",
            "title": "Machine Translation in Pronunciation Space",
            "authors": [
                "Hairong Liu",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "The research in machine translation community focus on translation in text\nspace. However, humans are in fact also good at direct translation in\npronunciation space. Some existing translation systems, such as simultaneous\nmachine translation, are inherently more natural and thus potentially more\nrobust by directly translating in pronunciation space. In this paper, we\nconduct large scale experiments on a self-built dataset with about $20$M En-Zh\npairs of text sentences and corresponding pronunciation sentences. We proposed\nthree new categories of translations: $1)$ translating a pronunciation sentence\nin source language into a pronunciation sentence in target language (P2P-Tran),\n$2)$ translating a text sentence in source language into a pronunciation\nsentence in target language (T2P-Tran), and $3)$ translating a pronunciation\nsentence in source language into a text sentence in target language (P2T-Tran),\nand compare them with traditional text translation (T2T-Tran). Our experiments\nclearly show that all $4$ categories of translations have comparable\nperformances, with small and sometimes ignorable differences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01055v1",
            "date": "2019-11-04 07:15:05+00:00",
            "title": "On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning",
            "authors": [
                "Tuan Ngo Nguyen",
                "Franck Dernoncourt",
                "Thien Huu Nguyen"
            ],
            "abstract": "Deep learning models have achieved state-of-the-art performances on many\nrelation extraction datasets. A common element in these deep learning models\ninvolves the pooling mechanisms where a sequence of hidden vectors is\naggregated to generate a single representation vector, serving as the features\nto perform prediction for RE. Unfortunately, the models in the literature tend\nto employ different strategies to perform pooling for RE, leading to the\nchallenge to determine the best pooling mechanism for this problem, especially\nin the biomedical domain. In order to answer this question, in this work, we\nconduct a comprehensive study to evaluate the effectiveness of different\npooling mechanisms for the deep learning models in biomedical RE. The\nexperimental results suggest that dependency-based pooling is the best pooling\nstrategy for RE in the biomedical domain, yielding the state-of-the-art\nperformance on two benchmark datasets for this problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01098v1",
            "date": "2019-11-04 09:58:23+00:00",
            "title": "Emergence of Numeric Concepts in Multi-Agent Autonomous Communication",
            "authors": [
                "Shangmin Guo"
            ],
            "abstract": "With the rapid development of deep learning, most of current state-of-the-art\ntechniques in natural langauge processing are based on deep learning models\ntrained with argescaled static textual corpora. However, we human beings learn\nand understand in a different way. Thus, grounded language learning argues that\nmodels need to learn and understand language by the experience and perceptions\nobtained by interacting with enviroments, like how humans do. With the help of\ndeep reinforcement learning techniques, there are already lots of works\nfocusing on facilitating the emergence of communication protocols that have\ncompositionalities like natural languages among computational agents\npopulation. Unlike these works, we, on the other hand, focus on the numeric\nconcepts which correspond to abstractions in cognition and function words in\nnatural language. Based on a specifically designed language game, we verify\nthat computational agents are capable of transmitting numeric concepts during\nautonomous communication, and the emergent communication protocols can reflect\nthe underlying structure of meaning space. Although their encodeing method is\nnot compositional like natural languages from a perspective of human beings,\nthe emergent languages can be generalised to unseen inputs and, more\nimportantly, are easier for models to learn. Besides, iterated learning can\nhelp further improving the compositionality of the emergent languages, under\nthe measurement of topological similarity. Furthermore, we experiment another\nrepresentation method, i.e. directly encode numerals into concatenations of\none-hot vectors, and find that the emergent languages would become\ncompositional like human natural languages. Thus, we argue that there are 2\nimportant factors for the emergence of compositional languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01188v1",
            "date": "2019-11-04 13:25:26+00:00",
            "title": "Analysing Coreference in Transformer Outputs",
            "authors": [
                "Ekaterina Lapshinova-Koltunski",
                "Cristina Espa\u00f1a-Bonet",
                "Josef van Genabith"
            ],
            "abstract": "We analyse coreference phenomena in three neural machine translation systems\ntrained with different data settings with or without access to explicit intra-\nand cross-sentential anaphoric information. We compare system performance on\ntwo different genres: news and TED talks. To do this, we manually annotate (the\npossibly incorrect) coreference chains in the MT outputs and evaluate the\ncoreference chain translations. We define an error typology that aims to go\nfurther than pronoun translation adequacy and includes types such as incorrect\nword selection or missing words. The features of coreference chains in\nautomatic translations are also compared to those of the source texts and human\ntranslations. The analysis shows stronger potential translationese effects in\nmachine translated outputs than in human translations.",
            "journal ref": "Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01214v1",
            "date": "2019-10-29 16:07:12+00:00",
            "title": "A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking",
            "authors": [
                "Andreas Hanselowski",
                "Christian Stab",
                "Claudia Schulz",
                "Zile Li",
                "Iryna Gurevych"
            ],
            "abstract": "Automated fact-checking based on machine learning is a promising approach to\nidentify false information distributed on the web. In order to achieve\nsatisfactory performance, machine learning methods require a large corpus with\nreliable annotations for the different tasks in the fact-checking process.\nHaving analyzed existing fact-checking corpora, we found that none of them\nmeets these criteria in full. They are either too small in size, do not provide\ndetailed annotations, or are limited to a single domain. Motivated by this gap,\nwe present a new substantially sized mixed-domain corpus with annotations of\ngood quality for the core fact-checking tasks: document retrieval, evidence\nextraction, stance detection, and claim validation. To aid future corpus\nconstruction, we describe our methodology for corpus creation and annotation,\nand demonstrate that it results in substantial inter-annotator agreement. As\nbaselines for future research, we perform experiments on our corpus with a\nnumber of model architectures that reach high performance in similar problem\nsettings. Finally, to support the development of future models, we provide a\ndetailed error analysis for each of the tasks. Our results show that the\nrealistic, multi-domain setting defined by our data poses new challenges for\nthe existing models, providing opportunities for considerable improvement by\nfuture systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01352v3",
            "date": "2019-11-04 17:31:02+00:00",
            "title": "Learning from Explanations with Neural Execution Tree",
            "authors": [
                "Ziqi Wang",
                "Yujia Qin",
                "Wenxuan Zhou",
                "Jun Yan",
                "Qinyuan Ye",
                "Leonardo Neves",
                "Zhiyuan Liu",
                "Xiang Ren"
            ],
            "abstract": "While deep neural networks have achieved impressive performance on a range of\nNLP tasks, these data-hungry models heavily rely on labeled data, which\nrestricts their applications in scenarios where data annotation is expensive.\nNatural language (NL) explanations have been demonstrated very useful\nadditional supervision, which can provide sufficient domain knowledge for\ngenerating more labeled data over new instances, while the annotation time only\ndoubles. However, directly applying them for augmenting model learning\nencounters two challenges: (1) NL explanations are unstructured and inherently\ncompositional, which asks for a modularized model to represent their semantics,\n(2) NL explanations often have large numbers of linguistic variants, resulting\nin low recall and limited generalization ability. In this paper, we propose a\nnovel Neural Execution Tree (NExT) framework to augment training data for text\nclassification using NL explanations. After transforming NL explanations into\nexecutable logical forms by semantic parsing, NExT generalizes different types\nof actions specified by the logical forms for labeling data instances, which\nsubstantially increases the coverage of each NL explanation. Experiments on two\nNLP tasks (relation extraction and sentiment analysis) demonstrate its\nsuperiority over baseline methods. Its extension to multi-hop question\nanswering achieves performance gain with light annotation effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01456v2",
            "date": "2019-11-04 19:21:48+00:00",
            "title": "Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systems",
            "authors": [
                "Sarik Ghazarian",
                "Ralph Weischedel",
                "Aram Galstyan",
                "Nanyun Peng"
            ],
            "abstract": "User engagement is a critical metric for evaluating the quality of\nopen-domain dialogue systems. Prior work has focused on conversation-level\nengagement by using heuristically constructed features such as the number of\nturns and the total time of the conversation. In this paper, we investigate the\npossibility and efficacy of estimating utterance-level engagement and define a\nnovel metric, {\\em predictive engagement}, for automatic evaluation of\nopen-domain dialogue systems. Our experiments demonstrate that (1) human\nannotators have high agreement on assessing utterance-level engagement scores;\n(2) conversation-level engagement scores can be predicted from properly\naggregated utterance-level engagement scores. Furthermore, we show that the\nutterance-level engagement scores can be learned from data. These scores can\nimprove automatic evaluation metrics for open-domain dialogue systems, as shown\nby correlation with human judgements. This suggests that predictive engagement\ncan be used as a real-time feedback for training better dialogue models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01460v1",
            "date": "2019-11-04 19:27:50+00:00",
            "title": "A Failure of Aspect Sentiment Classifiers and an Adaptive Re-weighting Solution",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "Aspect-based sentiment classification (ASC) is an important task in\nfine-grained sentiment analysis.~Deep supervised ASC approaches typically model\nthis task as a pair-wise classification task that takes an aspect and a\nsentence containing the aspect and outputs the polarity of the aspect in that\nsentence. However, we discovered that many existing approaches fail to learn an\neffective ASC classifier but more like a sentence-level sentiment classifier\nbecause they have difficulty to handle sentences with different polarities for\ndifferent aspects.~This paper first demonstrates this problem using several\nstate-of-the-art ASC models. It then proposes a novel and general adaptive\nre-weighting (ARW) scheme to adjust the training to dramatically improve ASC\nfor such complex sentences. Experimental results show that the proposed\nframework is effective \\footnote{The dataset and code are available at\n\\url{https://github.com/howardhsu/ASC_failure}.}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01464v3",
            "date": "2019-11-04 19:41:13+00:00",
            "title": "Emerging Cross-lingual Structure in Pretrained Language Models",
            "authors": [
                "Shijie Wu",
                "Alexis Conneau",
                "Haoran Li",
                "Luke Zettlemoyer",
                "Veselin Stoyanov"
            ],
            "abstract": "We study the problem of multilingual masked language modeling, i.e. the\ntraining of a single model on concatenated text from multiple languages, and\npresent a detailed study of several factors that influence why these models are\nso effective for cross-lingual transfer. We show, contrary to what was\npreviously hypothesized, that transfer is possible even when there is no shared\nvocabulary across the monolingual corpora and also when the text comes from\nvery different domains. The only requirement is that there are some shared\nparameters in the top layers of the multi-lingual encoder. To better understand\nthis result, we also show that representations from independently trained\nmodels in different languages can be aligned post-hoc quite effectively,\nstrongly suggesting that, much like for non-contextual word embeddings, there\nare universal latent symmetries in the learned embedding spaces. For\nmultilingual masked language modeling, these symmetries seem to be\nautomatically discovered and aligned during the joint training process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01528v3",
            "date": "2019-11-04 23:16:47+00:00",
            "title": "BAS: An Answer Selection Method Using BERT Language Model",
            "authors": [
                "Jamshid Mozafari",
                "Afsaneh Fatemi",
                "Mohammad Ali Nematbakhsh"
            ],
            "abstract": "In recent years, Question Answering systems have become more popular and\nwidely used by users. Despite the increasing popularity of these systems, the\ntheir performance is not even sufficient for textual data and requires further\nresearch. These systems consist of several parts that one of them is the Answer\nSelection component. This component detects the most relevant answer from a\nlist of candidate answers. The methods presented in previous researches have\nattempted to provide an independent model to undertake the answer-selection\ntask. An independent model cannot comprehend the syntactic and semantic\nfeatures of questions and answers with a small training dataset. To fill this\ngap, language models can be employed in implementing the answer selection part.\nThis action enables the model to have a better understanding of the language in\norder to understand questions and answers better than previous works. In this\nresearch, we will present the \"BAS\" (BERT Answer Selection) that uses the BERT\nlanguage model to comprehend language. The empirical results of applying the\nmodel on the TrecQA Raw, TrecQA Clean, and WikiQA datasets demonstrate that\nusing a robust language model such as BERT can enhance the performance. Using a\nmore robust classifier also enhances the effect of the language model on the\nanswer selection component. The results demonstrate that language comprehension\nis an essential requirement in natural language processing tasks such as\nanswer-selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01597v1",
            "date": "2019-11-05 03:43:14+00:00",
            "title": "Improving Bidirectional Decoding with Dynamic Target Semantics in Neural Machine Translation",
            "authors": [
                "Yong Shan",
                "Yang Feng",
                "Jinchao Zhang",
                "Fandong Meng",
                "Wen Zhang"
            ],
            "abstract": "Generally, Neural Machine Translation models generate target words in a\nleft-to-right (L2R) manner and fail to exploit any future (right) semantics\ninformation, which usually produces an unbalanced translation. Recent works\nattempt to utilize the right-to-left (R2L) decoder in bidirectional decoding to\nalleviate this problem. In this paper, we propose a novel \\textbf{D}ynamic\n\\textbf{I}nteraction \\textbf{M}odule (\\textbf{DIM}) to dynamically exploit\ntarget semantics from R2L translation for enhancing the L2R translation\nquality. Different from other bidirectional decoding approaches, DIM firstly\nextracts helpful target information through addressing and reading operations,\nthen updates target semantics for tracking the interactive history.\nAdditionally, we further introduce an \\textbf{agreement regularization} term\ninto the training objective to narrow the gap between L2R and R2L translations.\nExperimental results on NIST Chinese$\\Rightarrow$English and WMT'16\nEnglish$\\Rightarrow$Romanian translation tasks show that our system achieves\nsignificant improvements over baseline systems, which also reaches comparable\nresults compared to the state-of-the-art Transformer model with much fewer\nparameters of it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01616v4",
            "date": "2019-11-05 04:36:52+00:00",
            "title": "Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis",
            "authors": [
                "Haiyun Peng",
                "Lu Xu",
                "Lidong Bing",
                "Fei Huang",
                "Wei Lu",
                "Luo Si"
            ],
            "abstract": "Target-based sentiment analysis or aspect-based sentiment analysis (ABSA)\nrefers to addressing various sentiment analysis tasks at a fine-grained level,\nwhich includes but is not limited to aspect extraction, aspect sentiment\nclassification, and opinion extraction. There exist many solvers of the above\nindividual subtasks or a combination of two subtasks, and they can work\ntogether to tell a complete story, i.e. the discussed aspect, the sentiment on\nit, and the cause of the sentiment. However, no previous ABSA research tried to\nprovide a complete solution in one shot. In this paper, we introduce a new\nsubtask under ABSA, named aspect sentiment triplet extraction (ASTE).\nParticularly, a solver of this task needs to extract triplets (What, How, Why)\nfrom the inputs, which show WHAT the targeted aspects are, HOW their sentiment\npolarities are and WHY they have such polarities (i.e. opinion reasons). For\ninstance, one triplet from \"Waiters are very friendly and the pasta is simply\naverage\" could be ('Waiters', positive, 'friendly'). We propose a two-stage\nframework to address this task. The first stage predicts what, how and why in a\nunified model, and then the second stage pairs up the predicted what (how) and\nwhy from the first stage to output triplets. In the experiments, our framework\nhas set a benchmark performance in this novel triplet extraction task.\nMeanwhile, it outperforms a few strong baselines adapted from state-of-the-art\nrelated methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01621v1",
            "date": "2019-11-05 05:08:08+00:00",
            "title": "Discrete Argument Representation Learning for Interactive Argument Pair Identification",
            "authors": [
                "Lu Ji",
                "Zhongyu Wei",
                "Jing Li",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we focus on extracting interactive argument pairs from two\nposts with opposite stances to a certain topic. Considering opinions are\nexchanged from different perspectives of the discussing topic, we study the\ndiscrete representations for arguments to capture varying aspects in\nargumentation languages (e.g., the debate focus and the participant behavior).\nMoreover, we utilize hierarchical structure to model post-wise information\nincorporating contextual knowledge. Experimental results on the large-scale\ndataset collected from CMV show that our proposed framework can significantly\noutperform the competitive baselines. Further analyses reveal why our model\nyields superior performance and prove the usefulness of our learned\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01622v4",
            "date": "2019-11-05 05:14:08+00:00",
            "title": "Adversarial Language Games for Advanced Natural Language Intelligence",
            "authors": [
                "Yuan Yao",
                "Haoxi Zhong",
                "Zhengyan Zhang",
                "Xu Han",
                "Xiaozhi Wang",
                "Chaojun Xiao",
                "Guoyang Zeng",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "We study the problem of adversarial language games, in which multiple agents\nwith conflicting goals compete with each other via natural language\ninteractions. While adversarial language games are ubiquitous in human\nactivities, little attention has been devoted to this field in natural language\nprocessing. In this work, we propose a challenging adversarial language game\ncalled Adversarial Taboo as an example, in which an attacker and a defender\ncompete around a target word. The attacker is tasked with inducing the defender\nto utter the target word invisible to the defender, while the defender is\ntasked with detecting the target word before being induced by the attacker. In\nAdversarial Taboo, a successful attacker must hide its intention and subtly\ninduce the defender, while a competitive defender must be cautious with its\nutterances and infer the intention of the attacker. Such language abilities can\nfacilitate many important downstream NLP tasks. To instantiate the game, we\ncreate a game environment and a competition platform. Comprehensive experiments\nand empirical studies on several baseline attack and defense strategies show\npromising and interesting results. Based on the analysis on the game and\nexperiments, we discuss multiple promising directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01623v3",
            "date": "2019-11-05 05:14:54+00:00",
            "title": "Incremental Sense Weight Training for the Interpretation of Contextualized Word Embeddings",
            "authors": [
                "Xinyi Jiang",
                "Zhengzhe Yang",
                "Jinho D. Choi"
            ],
            "abstract": "We present a novel online algorithm that learns the essence of each dimension\nin word embeddings by minimizing the within-group distance of contextualized\nembedding groups. Three state-of-the-art neural-based language models are used,\nFlair, ELMo, and BERT, to generate contextualized word embeddings such that\ndifferent embeddings are generated for the same word type, which are grouped by\ntheir senses manually annotated in the SemCor dataset. We hypothesize that not\nall dimensions are equally important for downstream tasks so that our algorithm\ncan detect unessential dimensions and discard them without hurting the\nperformance. To verify this hypothesis, we first mask dimensions determined\nunessential by our algorithm, apply the masked word embeddings to a word sense\ndisambiguation task (WSD), and compare its performance against the one achieved\nby the original embeddings. Several KNN approaches are experimented to\nestablish strong baselines for WSD. Our results show that the masked word\nembeddings do not hurt the performance and can improve it by 3%. Our work can\nbe used to conduct future research on the interpretability of contextualized\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01746v4",
            "date": "2019-11-05 12:44:36+00:00",
            "title": "Coreference Resolution as Query-based Span Prediction",
            "authors": [
                "Wei Wu",
                "Fei Wang",
                "Arianna Yuan",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "In this paper, we present an accurate and extensible approach for the\ncoreference resolution task. We formulate the problem as a span prediction\ntask, like in machine reading comprehension (MRC): A query is generated for\neach candidate mention using its surrounding context, and a span prediction\nmodule is employed to extract the text spans of the coreferences within the\ndocument using the generated query. This formulation comes with the following\nkey advantages: (1) The span prediction strategy provides the flexibility of\nretrieving mentions left out at the mention proposal stage; (2) In the MRC\nframework, encoding the mention and its context explicitly in a query makes it\npossible to have a deep and thorough examination of cues embedded in the\ncontext of coreferent mentions; and (3) A plethora of existing MRC datasets can\nbe used for data augmentation to improve the model's generalization capability.\nExperiments demonstrate significant performance boost over previous models,\nwith 87.5 (+2.5) F1 score on the GAP benchmark and 83.1 (+3.5) F1 score on the\nCoNLL-2012 benchmark.",
            "journal ref": "ACL2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01940v2",
            "date": "2019-11-05 16:59:50+00:00",
            "title": "Deepening Hidden Representations from Pre-trained Language Models",
            "authors": [
                "Junjie Yang",
                "Hai Zhao"
            ],
            "abstract": "Transformer-based pre-trained language models have proven to be effective for\nlearning contextualized language representation. However, current approaches\nonly take advantage of the output of the encoder's final layer when fine-tuning\nthe downstream tasks. We argue that only taking single layer's output restricts\nthe power of pre-trained representation. Thus we deepen the representation\nlearned by the model by fusing the hidden representation in terms of an\nexplicit HIdden Representation Extractor (HIRE), which automatically absorbs\nthe complementary representation with respect to the output from the final\nlayer. Utilizing RoBERTa as the backbone encoder, our proposed improvement over\nthe pre-trained models is shown effective on multiple natural language\nunderstanding tasks and help our model rival with the state-of-the-art models\non the GLUE benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02116v2",
            "date": "2019-11-05 22:42:00+00:00",
            "title": "Unsupervised Cross-lingual Representation Learning at Scale",
            "authors": [
                "Alexis Conneau",
                "Kartikay Khandelwal",
                "Naman Goyal",
                "Vishrav Chaudhary",
                "Guillaume Wenzek",
                "Francisco Guzm\u00e1n",
                "Edouard Grave",
                "Myle Ott",
                "Luke Zettlemoyer",
                "Veselin Stoyanov"
            ],
            "abstract": "This paper shows that pretraining multilingual language models at scale leads\nto significant performance gains for a wide range of cross-lingual transfer\ntasks. We train a Transformer-based masked language model on one hundred\nlanguages, using more than two terabytes of filtered CommonCrawl data. Our\nmodel, dubbed XLM-R, significantly outperforms multilingual BERT (mBERT) on a\nvariety of cross-lingual benchmarks, including +14.6% average accuracy on XNLI,\n+13% average F1 score on MLQA, and +2.4% F1 score on NER. XLM-R performs\nparticularly well on low-resource languages, improving 15.7% in XNLI accuracy\nfor Swahili and 11.4% for Urdu over previous XLM models. We also present a\ndetailed empirical analysis of the key factors that are required to achieve\nthese gains, including the trade-offs between (1) positive transfer and\ncapacity dilution and (2) the performance of high and low resource languages at\nscale. Finally, we show, for the first time, the possibility of multilingual\nmodeling without sacrificing per-language performance; XLM-R is very\ncompetitive with strong monolingual models on the GLUE and XNLI benchmarks. We\nwill make our code, data and models publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02170v1",
            "date": "2019-11-06 02:34:10+00:00",
            "title": "Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network",
            "authors": [
                "Deming Ye",
                "Yankai Lin",
                "Zhenghao Liu",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Multi-paragraph reasoning is indispensable for open-domain question answering\n(OpenQA), which receives less attention in the current OpenQA systems. In this\nwork, we propose a knowledge-enhanced graph neural network (KGNN), which\nperforms reasoning over multiple paragraphs with entities. To explicitly\ncapture the entities' relatedness, KGNN utilizes relational facts in knowledge\ngraph to build the entity graph. The experimental results show that KGNN\noutperforms in both distractor and full wiki settings than baselines methods on\nHotpotQA dataset. And our further analysis illustrates KGNN is effective and\nrobust with more retrieved paragraphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02215v2",
            "date": "2019-11-06 06:17:16+00:00",
            "title": "Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information",
            "authors": [
                "Qiu Ran",
                "Yankai Lin",
                "Peng Li",
                "Jie Zhou"
            ],
            "abstract": "Non-autoregressive neural machine translation (NAT) generates each target\nword in parallel and has achieved promising inference acceleration. However,\nexisting NAT models still have a big gap in translation quality compared to\nautoregressive neural machine translation models due to the enormous decoding\nspace. To address this problem, we propose a novel NAT framework named\nReorderNAT which explicitly models the reordering information in the decoding\nprocedure. We further introduce deterministic and non-deterministic decoding\nstrategies that utilize reordering information to narrow the decoding search\nspace in our proposed ReorderNAT. Experimental results on various widely-used\ndatasets show that our proposed model achieves better performance compared to\nexisting NAT models, and even achieves comparable translation quality as\nautoregressive translation models with a significant speedup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02290v1",
            "date": "2019-11-06 10:24:45+00:00",
            "title": "Enriching Conversation Context in Retrieval-based Chatbots",
            "authors": [
                "Amir Vakili Tahami",
                "Azadeh Shakery"
            ],
            "abstract": "Work on retrieval-based chatbots, like most sequence pair matching tasks, can\nbe divided into Cross-encoders that perform word matching over the pair, and\nBi-encoders that encode the pair separately. The latter has better performance,\nhowever since candidate responses cannot be encoded offline, it is also much\nslower. Lately, multi-layer transformer architectures pre-trained as language\nmodels have been used to great effect on a variety of natural language\nprocessing and information retrieval tasks. Recent work has shown that these\nlanguage models can be used in text-matching scenarios to create Bi-encoders\nthat perform almost as well as Cross-encoders while having a much faster\ninference speed. In this paper, we expand upon this work by developing a\nsequence matching architecture that %takes into account contexts in the\ntraining dataset at inference time. utilizes the entire training set as a\nmakeshift knowledge-base during inference. We perform detailed experiments\ndemonstrating that this architecture can be used to further improve Bi-encoders\nperformance while still maintaining a relatively high inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02390v2",
            "date": "2019-11-06 13:53:46+00:00",
            "title": "Guiding Variational Response Generator to Exploit Persona",
            "authors": [
                "Bowen Wu",
                "Mengyuan Li",
                "Zongsheng Wang",
                "Yifu Chen",
                "Derek Wong",
                "Qihang Feng",
                "Junhong Huang",
                "Baoxun Wang"
            ],
            "abstract": "Leveraging persona information of users in Neural Response Generators (NRG)\nto perform personalized conversations has been considered as an attractive and\nimportant topic in the research of conversational agents over the past few\nyears. Despite of the promising progresses achieved by recent studies in this\nfield, persona information tends to be incorporated into neural networks in the\nform of user embeddings, with the expectation that the persona can be involved\nvia the End-to-End learning. This paper proposes to adopt the\npersonality-related characteristics of human conversations into variational\nresponse generators, by designing a specific conditional variational\nautoencoder based deep model with two new regularization terms employed to the\nloss function, so as to guide the optimization towards the direction of\ngenerating both persona-aware and relevant responses. Besides, to reasonably\nevaluate the performances of various persona modeling approaches, this paper\nfurther presents three direct persona-oriented metrics from different\nperspectives. The experimental results have shown that our proposed methodology\ncan notably improve the performance of persona-aware response generation, and\nthe metrics are reasonable to evaluate the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02493v3",
            "date": "2019-11-06 17:05:26+00:00",
            "title": "SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge",
            "authors": [
                "Pei Ke",
                "Haozhe Ji",
                "Siyang Liu",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "Most of the existing pre-trained language representation models neglect to\nconsider the linguistic knowledge of texts, which can promote language\nunderstanding in NLP tasks. To benefit the downstream tasks in sentiment\nanalysis, we propose a novel language representation model called SentiLARE,\nwhich introduces word-level linguistic knowledge including part-of-speech tag\nand sentiment polarity (inferred from SentiWordNet) into pre-trained models. We\nfirst propose a context-aware sentiment attention mechanism to acquire the\nsentiment polarity of each word with its part-of-speech tag by querying\nSentiWordNet. Then, we devise a new pre-training task called label-aware masked\nlanguage model to construct knowledge-aware language representation.\nExperiments show that SentiLARE obtains new state-of-the-art performance on a\nvariety of sentiment analysis tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02541v3",
            "date": "2019-11-06 18:25:00+00:00",
            "title": "Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports",
            "authors": [
                "Yuhao Zhang",
                "Derek Merck",
                "Emily Bao Tsai",
                "Christopher D. Manning",
                "Curtis P. Langlotz"
            ],
            "abstract": "Neural abstractive summarization models are able to generate summaries which\nhave high overlap with human references. However, existing models are not\noptimized for factual correctness, a critical metric in real-world\napplications. In this work, we develop a general framework where we evaluate\nthe factual correctness of a generated summary by fact-checking it\nautomatically against its reference using an information extraction module. We\nfurther propose a training strategy which optimizes a neural summarization\nmodel with a factual correctness reward via reinforcement learning. We apply\nthe proposed method to the summarization of radiology reports, where factual\ncorrectness is a key requirement. On two separate datasets collected from\nhospitals, we show via both automatic and human evaluation that the proposed\napproach substantially improves the factual correctness and overall quality of\noutputs over a competitive neural summarization system, producing radiology\nsummaries that approach the quality of human-authored ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02655v1",
            "date": "2019-11-06 22:35:00+00:00",
            "title": "Towards Domain Adaptation from Limited Data for Question Answering Using Deep Neural Networks",
            "authors": [
                "Timothy J. Hazen",
                "Shehzaad Dhuliawala",
                "Daniel Boies"
            ],
            "abstract": "This paper explores domain adaptation for enabling question answering (QA)\nsystems to answer questions posed against documents in new specialized domains.\nCurrent QA systems using deep neural network (DNN) technology have proven\neffective for answering general purpose factoid-style questions. However,\ncurrent general purpose DNN models tend to be ineffective for use in new\nspecialized domains. This paper explores the effectiveness of transfer learning\ntechniques for this problem. In experiments on question answering in the\nautomobile manual domain we demonstrate that standard DNN transfer learning\ntechniques work surprisingly well in adapting DNN models to a new domain using\nlimited amounts of annotated training data in the new domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02709v3",
            "date": "2019-11-07 01:45:33+00:00",
            "title": "Using Interlinear Glosses as Pivot in Low-Resource Multilingual Machine Translation",
            "authors": [
                "Zhong Zhou",
                "Lori Levin",
                "David R. Mortensen",
                "Alex Waibel"
            ],
            "abstract": "We demonstrate a new approach to Neural Machine Translation (NMT) for\nlow-resource languages using a ubiquitous linguistic resource, Interlinear\nGlossed Text (IGT). IGT represents a non-English sentence as a sequence of\nEnglish lemmas and morpheme labels. As such, it can serve as a pivot or\ninterlingua for NMT. Our contribution is four-fold. Firstly, we pool IGT for\n1,497 languages in ODIN (54,545 glosses) and 70,918 glosses in Arapaho and\ntrain a gloss-to-target NMT system from IGT to English, with a BLEU score of\n25.94. We introduce a multilingual NMT model that tags all glossed text with\ngloss-source language tags and train a universal system with shared attention\nacross 1,497 languages. Secondly, we use the IGT gloss-to-target translation as\na key step in an English-Turkish MT system trained on only 865 lines from ODIN.\nThirdly, we we present five metrics for evaluating extremely low-resource\ntranslation when BLEU is no longer sufficient and evaluate the Turkish\nlow-resource system using BLEU and also using accuracy of matching nouns,\nverbs, agreement, tense, and spurious repetition, showing large improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02711v2",
            "date": "2019-11-07 01:46:54+00:00",
            "title": "Making the Best Use of Review Summary for Sentiment Analysis",
            "authors": [
                "Sen Yang",
                "Leyang Cui",
                "Jun Xie",
                "Yue Zhang"
            ],
            "abstract": "Sentiment analysis provides a useful overview of customer review contents.\nMany review websites allow a user to enter a summary in addition to a full\nreview. Intuitively, summary information may give additional benefit for review\nsentiment analysis. In this paper, we conduct a study to exploit methods for\nbetter use of summary information. We start by finding out that the sentimental\nsignal distribution of a review and that of its corresponding summary are in\nfact complementary to each other. We thus explore various architectures to\nbetter guide the interactions between the two and propose a\nhierarchically-refined review-centric attention model. Empirical results show\nthat our review-centric model can make better use of user-written summaries for\nreview sentiment analysis, and is also more effective compared to existing\nmethods when the user summary is replaced with summary generated by an\nautomatic summarization system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02727v3",
            "date": "2019-11-07 02:47:26+00:00",
            "title": "Understanding Knowledge Distillation in Non-autoregressive Machine Translation",
            "authors": [
                "Chunting Zhou",
                "Graham Neubig",
                "Jiatao Gu"
            ],
            "abstract": "Non-autoregressive machine translation (NAT) systems predict a sequence of\noutput tokens in parallel, achieving substantial improvements in generation\nspeed compared to autoregressive models. Existing NAT models usually rely on\nthe technique of knowledge distillation, which creates the training data from a\npretrained autoregressive model for better performance. Knowledge distillation\nis empirically useful, leading to large gains in accuracy for NAT models, but\nthe reason for this success has, as of yet, been unclear. In this paper, we\nfirst design systematic experiments to investigate why knowledge distillation\nis crucial to NAT training. We find that knowledge distillation can reduce the\ncomplexity of data sets and help NAT to model the variations in the output\ndata. Furthermore, a strong correlation is observed between the capacity of an\nNAT model and the optimal complexity of the distilled data for the best\ntranslation quality. Based on these findings, we further propose several\napproaches that can alter the complexity of data sets to improve the\nperformance of NAT models. We achieve the state-of-the-art performance for the\nNAT-based models, and close the gap with the autoregressive baseline on WMT14\nEn-De benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02737v1",
            "date": "2019-11-07 03:13:26+00:00",
            "title": "SubCharacter Chinese-English Neural Machine Translation with Wubi encoding",
            "authors": [
                "Wei Zhang",
                "Feifei Lin",
                "Xiaodong Wang",
                "Zhenshuang Liang",
                "Zhen Huang"
            ],
            "abstract": "Neural machine translation (NMT) is one of the best methods for understanding\nthe differences in semantic rules between two languages. Especially for\nIndo-European languages, subword-level models have achieved impressive results.\nHowever, when the translation task involves Chinese, semantic granularity\nremains at the word and character level, so there is still need more\nfine-grained translation model of Chinese. In this paper, we introduce a simple\nand effective method for Chinese translation at the sub-character level. Our\napproach uses the Wubi method to translate Chinese into English; byte-pair\nencoding (BPE) is then applied. Our method for Chinese-English translation\neliminates the need for a complicated word segmentation algorithm during\npreprocessing. Furthermore, our method allows for sub-character-level neural\ntranslation based on recurrent neural network (RNN) architecture, without\npreprocessing. The empirical results show that for Chinese-English translation\ntasks, our sub-character-level model has a comparable BLEU score to the subword\nmodel, despite having a much smaller vocabulary. Additionally, the small\nvocabulary is highly advantageous for NMT model compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02808v1",
            "date": "2019-11-07 08:58:02+00:00",
            "title": "Transition-Based Deep Input Linearization",
            "authors": [
                "Ratish Puduppully",
                "Yue Zhang",
                "Manish Shrivastava"
            ],
            "abstract": "Traditional methods for deep NLG adopt pipeline approaches comprising stages\nsuch as constructing syntactic input, predicting function words, linearizing\nthe syntactic input and generating the surface forms. Though easier to\nvisualize, pipeline approaches suffer from error propagation. In addition,\ninformation available across modules cannot be leveraged by all modules. We\nconstruct a transition-based model to jointly perform linearization, function\nword prediction and morphological generation, which considerably improves upon\nthe accuracy compared to a pipelined baseline system. On a standard deep input\nlinearization shared task, our system achieves the best results reported so\nfar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02821v2",
            "date": "2019-11-07 09:50:21+00:00",
            "title": "Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention",
            "authors": [
                "Yanzeng Li",
                "Bowen Yu",
                "Mengge Xue",
                "Tingwen Liu"
            ],
            "abstract": "Most Chinese pre-trained models take character as the basic unit and learn\nrepresentation according to character's external contexts, ignoring the\nsemantics expressed in the word, which is the smallest meaningful utterance in\nChinese. Hence, we propose a novel word-aligned attention to exploit explicit\nword information, which is complementary to various character-based Chinese\npre-trained language models. Specifically, we devise a pooling mechanism to\nalign the character-level attention to the word level and propose to alleviate\nthe potential issue of segmentation error propagation by multi-source\ninformation fusion. As a result, word and character information are explicitly\nintegrated at the fine-tuning procedure. Experimental results on five Chinese\nNLP benchmark tasks demonstrate that our model could bring another significant\ngain over several pre-trained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02825v2",
            "date": "2019-11-07 10:08:04+00:00",
            "title": "Improving Grammatical Error Correction with Machine Translation Pairs",
            "authors": [
                "Wangchunshu Zhou",
                "Tao Ge",
                "Chang Mu",
                "Ke Xu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "We propose a novel data synthesis method to generate diverse error-corrected\nsentence pairs for improving grammatical error correction, which is based on a\npair of machine translation models of different qualities (i.e., poor and\ngood). The poor translation model resembles the ESL (English as a second\nlanguage) learner and tends to generate translations of low quality in terms of\nfluency and grammatical correctness, while the good translation model generally\ngenerates fluent and grammatically correct translations. We build the poor and\ngood translation model with phrase-based statistical machine translation model\nwith decreased language model weight and neural machine translation model\nrespectively. By taking the pair of their translations of the same sentences in\na bridge language as error-corrected sentence pairs, we can construct unlimited\npseudo parallel data. Our approach is capable of generating diverse\nfluency-improving patterns without being limited by the pre-defined rule set\nand the seed error-corrected data. Experimental results demonstrate the\neffectiveness of our approach and show that it can be combined with other\nsynthetic data sources to yield further improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02847v1",
            "date": "2019-11-07 10:59:40+00:00",
            "title": "Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks",
            "authors": [
                "Yinan Zhang",
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "In English semantic similarity tasks, classic word embedding-based approaches\nexplicitly model pairwise \"interactions\" between the word representations of a\nsentence pair. Transformer-based pretrained language models disregard this\nnotion, instead modeling pairwise word interactions globally and implicitly\nthrough their self-attention mechanism. In this paper, we hypothesize that\nintroducing an explicit, constrained pairwise word interaction mechanism to\npretrained language models improves their effectiveness on semantic similarity\ntasks. We validate our hypothesis using BERT on four tasks in semantic textual\nsimilarity and answer sentence selection. We demonstrate consistent\nimprovements in quality by adding an explicit pairwise word interaction module\nto BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02851v2",
            "date": "2019-11-07 11:02:39+00:00",
            "title": "Dependency and Span, Cross-Style Semantic Role Labeling on PropBank and NomBank",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Junru Zhou",
                "Kevin Parnow",
                "Shexia He"
            ],
            "abstract": "The latest developments in neural semantic role labeling (SRL) have shown\ngreat performance improvements with both the dependency and span\nformalisms/styles. Although the two styles share many similarities in\nlinguistic meaning and computation, most previous studies focus on a single\nstyle. In this paper, we define a new cross-style semantic role label\nconvention and propose a new cross-style joint optimization model designed\naround the most basic linguistic meaning of a semantic role, providing a\nsolution to make the results of the two styles more comparable and allowing\nboth formalisms of SRL to benefit from their natural connections in both\nlinguistics and computation. Our model learns a general semantic argument\nstructure and is capable of outputting in either style. Additionally, we\npropose a syntax-aided method to uniformly enhance the learning of both\ndependency and span representations. Experiments show that the proposed methods\nare effective on both span and dependency SRL benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02855v3",
            "date": "2019-11-07 11:14:05+00:00",
            "title": "Dice Loss for Data-imbalanced NLP Tasks",
            "authors": [
                "Xiaoya Li",
                "Xiaofei Sun",
                "Yuxian Meng",
                "Junjun Liang",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Many NLP tasks such as tagging and machine reading comprehension are faced\nwith the severe data imbalance issue: negative examples significantly outnumber\npositive examples, and the huge number of background examples (or easy-negative\nexamples) overwhelms the training. The most commonly used cross entropy (CE)\ncriteria is actually an accuracy-oriented objective, and thus creates a\ndiscrepancy between training and test: at training time, each training instance\ncontributes equally to the objective function, while at test time F1 score\nconcerns more about positive examples. In this paper, we propose to use dice\nloss in replacement of the standard cross-entropy objective for data-imbalanced\nNLP tasks. Dice loss is based on the Sorensen-Dice coefficient or Tversky\nindex, which attaches similar importance to false positives and false\nnegatives, and is more immune to the data-imbalance issue. To further alleviate\nthe dominating influence from easy-negative examples in training, we propose to\nassociate training examples with dynamically adjusted weights to deemphasize\neasy-negative examples.Theoretical analysis shows that this strategy narrows\ndown the gap between the F1 score in evaluation and the dice loss in training.\nWith the proposed training objective, we observe significant performance boost\non a wide range of data imbalanced NLP tasks. Notably, we are able to achieve\nSOTA results on CTB5, CTB6 and UD1.4 for the part of speech tagging task; SOTA\nresults on CoNLL03, OntoNotes5.0, MSRA and OntoNotes4.0 for the named entity\nrecognition task; along with competitive results on the tasks of machine\nreading comprehension and paraphrase identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02898v1",
            "date": "2019-11-07 13:40:54+00:00",
            "title": "The LIG system for the English-Czech Text Translation Task of IWSLT 2019",
            "authors": [
                "Lo\u00efc Vial",
                "Benjamin Lecouteux",
                "Didier Schwab",
                "Hang Le",
                "Laurent Besacier"
            ],
            "abstract": "In this paper, we present our submission for the English to Czech Text\nTranslation Task of IWSLT 2019. Our system aims to study how pre-trained\nlanguage models, used as input embeddings, can improve a specialized machine\ntranslation system trained on few data. Therefore, we implemented a\nTransformer-based encoder-decoder neural system which is able to use the output\nof a pre-trained language model as input embeddings, and we compared its\nperformance under three configurations: 1) without any pre-trained language\nmodel (constrained), 2) using a language model trained on the monolingual parts\nof the allowed English-Czech data (constrained), and 3) using a language model\ntrained on a large quantity of external monolingual data (unconstrained). We\nused BERT as external pre-trained language model (configuration 3), and BERT\narchitecture for training our own language model (configuration 2). Regarding\nthe training data, we trained our MT system on a small quantity of parallel\ntext: one set only consists of the provided MuST-C corpus, and the other set\nconsists of the MuST-C corpus and the News Commentary corpus from WMT. We\nobserved that using the external pre-trained BERT improves the scores of our\nsystem by +0.8 to +1.5 of BLEU on our development set, and +0.97 to +1.94 of\nBLEU on the test set. However, using our own language model trained only on the\nallowed parallel data seems to improve the machine translation performances\nonly when the system is trained on the smallest dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02914v1",
            "date": "2019-11-07 14:06:36+00:00",
            "title": "Transformation of Dense and Sparse Text Representations",
            "authors": [
                "Wenpeng Hu",
                "Mengyu Wang",
                "Bing Liu",
                "Feng Ji",
                "Haiqing Chen",
                "Dongyan Zhao",
                "Jinwen Ma",
                "Rui Yan"
            ],
            "abstract": "Sparsity is regarded as a desirable property of representations, especially\nin terms of explanation. However, its usage has been limited due to the gap\nwith dense representations. Most NLP research progresses in recent years are\nbased on dense representations. Thus the desirable property of sparsity cannot\nbe leveraged. Inspired by Fourier Transformation, in this paper, we propose a\nnovel Semantic Transformation method to bridge the dense and sparse spaces,\nwhich can facilitate the NLP research to shift from dense space to sparse space\nor to jointly use both spaces. The key idea of the proposed approach is to use\na Forward Transformation to transform dense representations to sparse\nrepresentations. Then some useful operations in the sparse space can be\nperformed over the sparse representations, and the sparse representations can\nbe used directly to perform downstream tasks such as text classification and\nnatural language inference. Then, a Backward Transformation can also be carried\nout to transform those processed sparse representations to dense\nrepresentations. Experiments using classification tasks and natural language\ninference task show that the proposed Semantic Transformation is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02929v2",
            "date": "2019-11-07 14:23:05+00:00",
            "title": "How Can BERT Help Lexical Semantics Tasks?",
            "authors": [
                "Yile Wang",
                "Leyang Cui",
                "Yue Zhang"
            ],
            "abstract": "Contextualized embeddings such as BERT can serve as strong input\nrepresentations to NLP tasks, outperforming their static embeddings\ncounterparts such as skip-gram, CBOW and GloVe. However, such embeddings are\ndynamic, calculated according to a sentence-level context, which limits their\nuse in lexical semantics tasks. We address this issue by making use of dynamic\nembeddings as word representations in training static embeddings, thereby\nleveraging their strong representation power for disambiguating context\ninformation. Results show that this method leads to improvements over\ntraditional static embeddings on a range of lexical semantics tasks, obtaining\nthe best reported results on seven datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02969v2",
            "date": "2019-11-07 16:20:40+00:00",
            "title": "BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance",
            "authors": [
                "R. Thomas McCoy",
                "Junghyun Min",
                "Tal Linzen"
            ],
            "abstract": "If the same neural network architecture is trained multiple times on the same\ndataset, will it make similar linguistic generalizations across runs? To study\nthis question, we fine-tuned 100 instances of BERT on the Multi-genre Natural\nLanguage Inference (MNLI) dataset and evaluated them on the HANS dataset, which\nevaluates syntactic generalization in natural language inference. On the MNLI\ndevelopment set, the behavior of all instances was remarkably consistent, with\naccuracy ranging between 83.6% and 84.8%. In stark contrast, the same models\nvaried widely in their generalization performance. For example, on the simple\ncase of subject-object swap (e.g., determining that \"the doctor visited the\nlawyer\" does not entail \"the lawyer visited the doctor\"), accuracy ranged from\n0.00% to 66.2%. Such variation is likely due to the presence of many local\nminima that are equally attractive to a low-bias learner such as a neural\nnetwork; decreasing the variability may therefore require models with stronger\ninductive biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03024v1",
            "date": "2019-11-08 03:45:55+00:00",
            "title": "Why Do Masked Neural Language Models Still Need Common Sense Knowledge?",
            "authors": [
                "Sunjae Kwon",
                "Cheongwoong Kang",
                "Jiyeon Han",
                "Jaesik Choi"
            ],
            "abstract": "Currently, contextualized word representations are learned by intricate\nneural network models, such as masked neural language models (MNLMs). The new\nrepresentations significantly enhanced the performance in automated question\nanswering by reading paragraphs. However, identifying the detailed knowledge\ntrained in the MNLMs is difficult owing to numerous and intermingled\nparameters. This paper provides empirical but insightful analyses on the\npretrained MNLMs with respect to common sense knowledge. First, we propose a\ntest that measures what types of common sense knowledge do pretrained MNLMs\nunderstand. From the test, we observed that MNLMs partially understand various\ntypes of common sense knowledge but do not accurately understand the semantic\nmeaning of relations. In addition, based on the difficulty of the\nquestion-answering task problems, we observed that pretrained MLM-based models\nare still vulnerable to problems that require common sense knowledge. We also\nexperimentally demonstrated that we can elevate existing MNLM-based models by\ncombining knowledge from an external common sense repository.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03042v3",
            "date": "2019-11-08 04:31:59+00:00",
            "title": "Neural Graph Embedding Methods for Natural Language Processing",
            "authors": [
                "Shikhar Vashishth"
            ],
            "abstract": "Knowledge graphs are structured representations of facts in a graph, where\nnodes represent entities and edges represent relationships between them. Recent\nresearch has resulted in the development of several large KGs. However, all of\nthem tend to be sparse with very few facts per entity. In the first part of the\nthesis, we propose two solutions to alleviate this problem: (1) KG\nCanonicalization, i.e., identifying and merging duplicate entities in a KG, (2)\nRelation Extraction which involves automating the process of extracting\nsemantic relationships between entities from unstructured text. Traditional\nNeural Networks like CNNs and RNNs are constrained to handle Euclidean data.\nHowever, graphs in Natural Language Processing (NLP) are prominent. Recently,\nGraph Convolutional Networks (GCNs) have been proposed to address this\nshortcoming and have been successfully applied for several problems. In the\nsecond part of the thesis, we utilize GCNs for Document Timestamping problem\nand for learning word embeddings using dependency context of a word instead of\nsequential context. In this third part of the thesis, we address two\nlimitations of existing GCN models, i.e., (1) The standard neighborhood\naggregation scheme puts no constraints on the number of nodes that can\ninfluence the representation of a target node. This leads to a noisy\nrepresentation of hub-nodes which coves almost the entire graph in a few hops.\n(2) Most of the existing GCN models are limited to handle undirected graphs.\nHowever, a more general and pervasive class of graphs are relational graphs\nwhere each edge has a label and direction associated with it. Existing\napproaches to handle such graphs suffer from over-parameterization and are\nrestricted to learning representation of nodes only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03047v3",
            "date": "2019-11-08 04:45:55+00:00",
            "title": "Contrastive Multi-document Question Generation",
            "authors": [
                "Woon Sang Cho",
                "Yizhe Zhang",
                "Sudha Rao",
                "Asli Celikyilmaz",
                "Chenyan Xiong",
                "Jianfeng Gao",
                "Mengdi Wang",
                "Bill Dolan"
            ],
            "abstract": "Multi-document question generation focuses on generating a question that\ncovers the common aspect of multiple documents. Such a model is useful in\ngenerating clarifying options. However, a naive model trained only using the\ntargeted (\"positive\") document set may generate too generic questions that\ncover a larger scope than delineated by the document set. To address this\nchallenge, we introduce the contrastive learning strategy where given\n\"positive\" and \"negative\" sets of documents, we generate a question that is\nclosely related to the \"positive\" set but is far away from the \"negative\" set.\nThis setting allows generated questions to be more specific and related to the\ntarget document set. To generate such specific questions, we propose\nMulti-Source Coordinated Question Generator (MSCQG), a novel framework that\nincludes a supervised learning (SL) stage and a reinforcement learning (RL)\nstage. In the SL stage, a single-document question generator is trained. In the\nRL stage, a coordinator model is trained to find optimal attention weights to\nalign multiple single-document generators, by optimizing a reward designed to\npromote specificity of generated questions. We also develop an effective\nauxiliary objective, named Set-induced Contrastive Regularization (SCR) that\nimproves the coordinator's contrastive learning during the RL stage. We show\nthat our model significantly outperforms several strong baselines, as measured\nby automatic metrics and human evaluation. The source repository is publicly\navailable at \\url{www.github.com/woonsangcho/contrast_qgen}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03058v2",
            "date": "2019-11-08 05:29:57+00:00",
            "title": "Should All Cross-Lingual Embeddings Speak English?",
            "authors": [
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "Most of recent work in cross-lingual word embeddings is severely\nAnglocentric. The vast majority of lexicon induction evaluation dictionaries\nare between English and another language, and the English embedding space is\nselected by default as the hub when learning in a multilingual setting. With\nthis work, however, we challenge these practices. First, we show that the\nchoice of hub language can significantly impact downstream lexicon induction\nperformance. Second, we both expand the current evaluation dictionary\ncollection to include all language pairs using triangulation, and also create\nnew dictionaries for under-represented languages. Evaluating established\nmethods over all these language pairs sheds light into their suitability and\npresents new challenges for the field. Finally, in our analysis we identify\ngeneral guidelines for strong cross-lingual embeddings baselines, based on more\nthan just Anglocentric experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03059v2",
            "date": "2019-11-08 05:30:33+00:00",
            "title": "A Comprehensive Comparison of Machine Learning Based Methods Used in Bengali Question Classification",
            "authors": [
                "Afra Anika",
                "Md. Hasibur Rahman",
                "Salekul Islam",
                "Abu Shafin Mohammad Mahdee Jameel",
                "Chowdhury Rafeed Rahman"
            ],
            "abstract": "QA classification system maps questions asked by humans to an appropriate\nanswer category. A sound question classification (QC) system model is the\npre-requisite of a sound QA system. This work demonstrates phases of assembling\na QA type classification model. We present a comprehensive comparison\n(performance and computational complexity) among some machine learning based\napproaches used in QC for Bengali language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03090v1",
            "date": "2019-11-08 07:05:20+00:00",
            "title": "What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning",
            "authors": [
                "Jaejun Lee",
                "Raphael Tang",
                "Jimmy Lin"
            ],
            "abstract": "Pretrained transformer-based language models have achieved state of the art\nacross countless tasks in natural language processing. These models are highly\nexpressive, comprising at least a hundred million parameters and a dozen\nlayers. Recent evidence suggests that only a few of the final layers need to be\nfine-tuned for high quality on downstream tasks. Naturally, a subsequent\nresearch question is, \"how many of the last layers do we need to fine-tune?\" In\nthis paper, we precisely answer this question. We examine two recent pretrained\nlanguage models, BERT and RoBERTa, across standard tasks in textual entailment,\nsemantic similarity, sentiment analysis, and linguistic acceptability. We vary\nthe number of final layers that are fine-tuned, then study the resulting change\nin task-specific effectiveness. We show that only a fourth of the final layers\nneed to be fine-tuned to achieve 90% of the original quality. Surprisingly, we\nalso find that fine-tuning all layers does not always help.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03091v6",
            "date": "2019-11-08 07:05:25+00:00",
            "title": "Relation Adversarial Network for Low Resource Knowledge Graph Completion",
            "authors": [
                "Ningyu Zhang",
                "Shumin Deng",
                "Zhanlin Sun",
                "Jiaoayan Chen",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "Knowledge Graph Completion (KGC) has been proposed to improve Knowledge\nGraphs by filling in missing connections via link prediction or relation\nextraction. One of the main difficulties for KGC is a low resource problem.\nPrevious approaches assume sufficient training triples to learn versatile\nvectors for entities and relations, or a satisfactory number of labeled\nsentences to train a competent relation extraction model. However, low resource\nrelations are very common in KGs, and those newly added relations often do not\nhave many known samples for training. In this work, we aim at predicting new\nfacts under a challenging setting where only limited training instances are\navailable. We propose a general framework called Weighted Relation Adversarial\nNetwork, which utilizes an adversarial procedure to help adapt\nknowledge/features learned from high resource relations to different but\nrelated low resource relations. Specifically, the framework takes advantage of\na relation discriminator to distinguish between samples from different\nrelations, and help learn relation-invariant features more transferable from\nsource relations to target relations. Experimental results show that the\nproposed approach outperforms previous methods regarding low resource settings\nfor both link prediction and relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03109v2",
            "date": "2019-11-08 07:57:46+00:00",
            "title": "Domain Robustness in Neural Machine Translation",
            "authors": [
                "Mathias M\u00fcller",
                "Annette Rios",
                "Rico Sennrich"
            ],
            "abstract": "Translating text that diverges from the training domain is a key challenge\nfor machine translation. Domain robustness---the generalization of models to\nunseen test domains---is low for both statistical (SMT) and neural machine\ntranslation (NMT). In this paper, we study the performance of SMT and NMT\nmodels on out-of-domain test sets. We find that in unknown domains, SMT and NMT\nsuffer from very different problems: SMT systems are mostly adequate but not\nfluent, while NMT systems are mostly fluent, but not adequate. For NMT, we\nidentify such hallucinations (translations that are fluent but unrelated to the\nsource) as a key reason for low domain robustness. To mitigate this problem, we\nempirically compare methods that are reported to improve adequacy or in-domain\nrobustness in terms of their effectiveness at improving domain robustness. In\nexperiments on German to English OPUS data, and German to Romansh (a\nlow-resource setting) we find that several methods improve domain robustness.\nWhile those methods do lead to higher BLEU scores overall, they only slightly\nincrease the adequacy of translations compared to SMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03110v1",
            "date": "2019-11-08 08:00:45+00:00",
            "title": "Pretrained Language Models for Document-Level Neural Machine Translation",
            "authors": [
                "Liangyou Li",
                "Xin Jiang",
                "Qun Liu"
            ],
            "abstract": "Previous work on document-level NMT usually focuses on limited contexts\nbecause of degraded performance on larger contexts. In this paper, we\ninvestigate on using large contexts with three main contributions: (1)\nDifferent from previous work which pertrained models on large-scale\nsentence-level parallel corpora, we use pretrained language models,\nspecifically BERT, which are trained on monolingual documents; (2) We propose\ncontext manipulation methods to control the influence of large contexts, which\nlead to comparable results on systems using small and large contexts; (3) We\nintroduce a multi-task training for regularization to avoid models overfitting\nour training corpora, which further improves our systems together with a deeper\nencoder. Experiments are conducted on the widely used IWSLT data sets with\nthree language pairs, i.e., Chinese--English, French--English and\nSpanish--English. Results show that our systems are significantly better than\nthree previously reported document-level systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03123v2",
            "date": "2019-11-08 08:40:22+00:00",
            "title": "iSarcasm: A Dataset of Intended Sarcasm",
            "authors": [
                "Silviu Oprea",
                "Walid Magdy"
            ],
            "abstract": "We consider the distinction between intended and perceived sarcasm in the\ncontext of textual sarcasm detection. The former occurs when an utterance is\nsarcastic from the perspective of its author, while the latter occurs when the\nutterance is interpreted as sarcastic by the audience. We show the limitations\nof previous labelling methods in capturing intended sarcasm and introduce the\niSarcasm dataset of tweets labeled for sarcasm directly by their authors.\nExamining the state-of-the-art sarcasm detection models on our dataset showed\nlow performance compared to previously studied datasets, which indicates that\nthese datasets might be biased or obvious and sarcasm could be a phenomenon\nunder-studied computationally thus far. By providing the iSarcasm dataset, we\naim to encourage future NLP research to develop methods for detecting sarcasm\nin text as intended by the authors of the text, not as labeled under\nassumptions that we demonstrate to be sub-optimal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03154v2",
            "date": "2019-11-08 09:55:37+00:00",
            "title": "A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation",
            "authors": [
                "Yun Chen",
                "Liangyou Li",
                "Xin Jiang",
                "Xiao Chen",
                "Qun Liu"
            ],
            "abstract": "Despite the success of neural machine translation (NMT), simultaneous neural\nmachine translation (SNMT), the task of translating in real time before a full\nsentence has been observed, remains challenging due to the syntactic structure\ndifference and simultaneity requirements. In this paper, we propose a general\nframework for adapting neural machine translation to translate simultaneously.\nOur framework contains two parts: prefix translation that utilizes a\nconsecutive NMT model to translate source prefixes and a stopping criterion\nthat determines when to stop the prefix translation. Experiments on three\ntranslation corpora and two language pairs show the efficacy of the proposed\nframework on balancing the quality and latency in adapting NMT to perform\nsimultaneous translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03243v2",
            "date": "2019-11-08 13:22:12+00:00",
            "title": "Controlled Crowdsourcing for High-Quality QA-SRL Annotation",
            "authors": [
                "Paul Roit",
                "Ayal Klein",
                "Daniela Stepanov",
                "Jonathan Mamou",
                "Julian Michael",
                "Gabriel Stanovsky",
                "Luke Zettlemoyer",
                "Ido Dagan"
            ],
            "abstract": "Question-answer driven Semantic Role Labeling (QA-SRL) was proposed as an\nattractive open and natural flavour of SRL, potentially attainable from laymen.\nRecently, a large-scale crowdsourced QA-SRL corpus and a trained parser were\nreleased. Trying to replicate the QA-SRL annotation for new texts, we found\nthat the resulting annotations were lacking in quality, particularly in\ncoverage, making them insufficient for further research and evaluation. In this\npaper, we present an improved crowdsourcing protocol for complex semantic\nannotation, involving worker selection and training, and a data consolidation\nphase. Applying this protocol to QA-SRL yielded high-quality annotation with\ndrastically higher coverage, producing a new gold evaluation dataset. We\nbelieve that our annotation protocol and gold standard will facilitate future\nreplicable research of natural semantic annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03283v1",
            "date": "2019-11-08 14:33:06+00:00",
            "title": "Composing and Embedding the Words-as-Classifiers Model of Grounded Semantics",
            "authors": [
                "Daniele Moro",
                "Stacy Black",
                "Casey Kennington"
            ],
            "abstract": "The words-as-classifiers model of grounded lexical semantics learns a\nsemantic fitness score between physical entities and the words that are used to\ndenote those entities. In this paper, we explore how such a model can\nincrementally perform composition and how the model can be unified with a\ndistributional representation. For the latter, we leverage the classifier\ncoefficients as an embedding. For composition, we leverage the underlying\nmechanics of three different classifier types (i.e., logistic regression,\ndecision trees, and multi-layer perceptrons) to arrive at a several systematic\napproaches to composition unique to each classifier including both denotational\nand connotational methods of composition. We compare these approaches to each\nother and to prior work in a visual reference resolution task using the refCOCO\ndataset. Our results demonstrate the need to expand upon existing composition\nstrategies and bring together grounded and distributional representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03310v1",
            "date": "2019-11-08 15:12:36+00:00",
            "title": "How Language-Neutral is Multilingual BERT?",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Rudolf Rosa",
                "Alexander Fraser"
            ],
            "abstract": "Multilingual BERT (mBERT) provides sentence representations for 104\nlanguages, which are useful for many multi-lingual tasks. Previous work probed\nthe cross-linguality of mBERT using zero-shot transfer learning on\nmorphological and syntactic tasks. We instead focus on the semantic properties\nof mBERT. We show that mBERT representations can be split into a\nlanguage-specific component and a language-neutral component, and that the\nlanguage-neutral component is sufficiently general in terms of modeling\nsemantics to allow high-accuracy word-alignment and sentence retrieval but is\nnot yet good enough for the more difficult task of MT quality estimation. Our\nwork presents interesting challenges which must be solved to build better\nlanguage-neutral representations, particularly for tasks requiring linguistic\ntransfer of semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03324v1",
            "date": "2019-11-08 15:28:21+00:00",
            "title": "Transforming Wikipedia into Augmented Data for Query-Focused Summarization",
            "authors": [
                "Haichao Zhu",
                "Li Dong",
                "Furu Wei",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "The manual construction of a query-focused summarization corpus is costly and\ntimeconsuming. The limited size of existing datasets renders training\ndata-driven summarization models challenging. In this paper, we use Wikipedia\nto automatically collect a large query-focused summarization dataset (named as\nWIKIREF) of more than 280,000 examples, which can serve as a means of data\naugmentation. Moreover, we develop a query-focused summarization model based on\nBERT to extract summaries from the documents. Experimental results on three DUC\nbenchmarks show that the model pre-trained on WIKIREF has already achieved\nreasonable performance. After fine-tuning on the specific datasets, the model\nwith data augmentation outperforms the state of the art on the benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03343v3",
            "date": "2019-11-08 16:08:48+00:00",
            "title": "Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly",
            "authors": [
                "Nora Kassner",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Building on Petroni et al. (2019), we propose two new probing tasks analyzing\nfactual knowledge stored in Pretrained Language Models (PLMs). (1) Negation. We\nfind that PLMs do not distinguish between negated (\"Birds cannot [MASK]\") and\nnon-negated (\"Birds can [MASK]\") cloze questions. (2) Mispriming. Inspired by\npriming methods in human psychology, we add \"misprimes\" to cloze questions\n(\"Talk? Birds can [MASK]\"). We find that PLMs are easily distracted by\nmisprimes. These results suggest that PLMs still have a long way to go to\nadequately learn human-like factual knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03373v1",
            "date": "2019-11-08 16:54:58+00:00",
            "title": "A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models",
            "authors": [
                "Chris Kedzie",
                "Kathleen McKeown"
            ],
            "abstract": "Deep neural networks (DNN) are quickly becoming the de facto standard\nmodeling method for many natural language generation (NLG) tasks. In order for\nsuch models to truly be useful, they must be capable of correctly generating\nutterances for novel meaning representations (MRs) at test time. In practice,\neven sophisticated DNNs with various forms of semantic control frequently fail\nto generate utterances faithful to the input MR. In this paper, we propose an\narchitecture agnostic self-training method to sample novel MR/text utterance\npairs to augment the original training data. Remarkably, after training on the\naugmented data, even simple encoder-decoder models with greedy decoding are\ncapable of generating semantically correct utterances that are as good as\nstate-of-the-art outputs in both automatic and human evaluations of quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03385v1",
            "date": "2019-11-08 17:10:49+00:00",
            "title": "Low-Level Linguistic Controls for Style Transfer and Content Preservation",
            "authors": [
                "Katy Gero",
                "Chris Kedzie",
                "Jonathan Reeve",
                "Lydia Chilton"
            ],
            "abstract": "Despite the success of style transfer in image processing, it has seen\nlimited progress in natural language generation. Part of the problem is that\ncontent is not as easily decoupled from style in the text domain. Curiously, in\nthe field of stylometry, content does not figure prominently in practical\nmethods of discriminating stylistic elements, such as authorship and genre.\nRather, syntax and function words are the most salient features. Drawing on\nthis work, we model style as a suite of low-level linguistic controls, such as\nfrequency of pronouns, prepositions, and subordinate clause constructions. We\ntrain a neural encoder-decoder model to reconstruct reference sentences given\nonly content words and the setting of the controls. We perform style transfer\nby keeping the content words fixed while adjusting the controls to be\nindicative of another style. In experiments, we show that the model reliably\nresponds to the linguistic controls and perform both automatic and manual\nevaluations on style transfer. We find we can fool a style classifier 84% of\nthe time, and that our model produces highly diverse and stylistically\ndistinctive outputs. This work introduces a formal, extendable model of style\nthat can add control to any neural text generation system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03407v1",
            "date": "2019-11-08 17:49:08+00:00",
            "title": "Question Generation from Paragraphs: A Tale of Two Hierarchical Models",
            "authors": [
                "Vishwajeet Kumar",
                "Raktim Chaki",
                "Sai Teja Talluri",
                "Ganesh Ramakrishnan",
                "Yuan-Fang Li",
                "Gholamreza Haffari"
            ],
            "abstract": "Automatic question generation from paragraphs is an important and challenging\nproblem, particularly due to the long context from paragraphs. In this paper,\nwe propose and study two hierarchical models for the task of question\ngeneration from paragraphs. Specifically, we propose (a) a novel hierarchical\nBiLSTM model with selective attention and (b) a novel hierarchical Transformer\narchitecture, both of which learn hierarchical representations of paragraphs.\nWe model a paragraph in terms of its constituent sentences, and a sentence in\nterms of its constituent words. While the introduction of the attention\nmechanism benefits the hierarchical BiLSTM model, the hierarchical Transformer,\nwith its inherent attention and positional encoding mechanisms also performs\nbetter than flat transformer model. We conducted empirical evaluation on the\nwidely used SQuAD and MS MARCO datasets using standard metrics. The results\ndemonstrate the overall effectiveness of the hierarchical models over their\nflat counterparts. Qualitatively, our hierarchical models are able to generate\nfluent and relevant questions",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03514v1",
            "date": "2019-11-08 19:33:06+00:00",
            "title": "An Annotation Scheme of A Large-scale Multi-party Dialogues Dataset for Discourse Parsing and Machine Comprehension",
            "authors": [
                "Jiaqi Li",
                "Ming Liu",
                "Bing Qin",
                "Zihao Zheng",
                "Ting Liu"
            ],
            "abstract": "In this paper, we propose the scheme for annotating large-scale multi-party\nchat dialogues for discourse parsing and machine comprehension. The main goal\nof this project is to help understand multi-party dialogues. Our dataset is\nbased on the Ubuntu Chat Corpus. For each multi-party dialogue, we annotate the\ndiscourse structure and question-answer pairs for dialogues. As we know, this\nis the first large scale corpus for multi-party dialogues discourse parsing,\nand we firstly propose the task for multi-party dialogues machine reading\ncomprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03587v2",
            "date": "2019-11-09 00:16:03+00:00",
            "title": "How Decoding Strategies Affect the Verifiability of Generated Text",
            "authors": [
                "Luca Massarelli",
                "Fabio Petroni",
                "Aleksandra Piktus",
                "Myle Ott",
                "Tim Rockt\u00e4schel",
                "Vassilis Plachouras",
                "Fabrizio Silvestri",
                "Sebastian Riedel"
            ],
            "abstract": "Recent progress in pre-trained language models led to systems that are able\nto generate text of an increasingly high quality. While several works have\ninvestigated the fluency and grammatical correctness of such models, it is\nstill unclear to which extent the generated text is consistent with factual\nworld knowledge. Here, we go beyond fluency and also investigate the\nverifiability of text generated by state-of-the-art pre-trained language\nmodels. A generated sentence is verifiable if it can be corroborated or\ndisproved by Wikipedia, and we find that the verifiability of generated text\nstrongly depends on the decoding strategy. In particular, we discover a\ntradeoff between factuality (i.e., the ability of generating Wikipedia\ncorroborated text) and repetitiveness. While decoding strategies such as top-k\nand nucleus sampling lead to less repetitive generations, they also produce\nless verifiable text. Based on these finding, we introduce a simple and\neffective decoding strategy which, in comparison to previously used decoding\nstrategies, produces less repetitive and more verifiable text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03597v1",
            "date": "2019-11-09 02:49:31+00:00",
            "title": "Zero-Shot Paraphrase Generation with Multilingual Language Models",
            "authors": [
                "Yinpeng Guo",
                "Yi Liao",
                "Xin Jiang",
                "Qing Zhang",
                "Yibo Zhang",
                "Qun Liu"
            ],
            "abstract": "Leveraging multilingual parallel texts to automatically generate paraphrases\nhas drawn much attention as size of high-quality paraphrase corpus is limited.\nRound-trip translation, also known as the pivoting method, is a typical\napproach to this end. However, we notice that the pivoting process involves\nmultiple machine translation models and is likely to incur semantic drift\nduring the two-step translations. In this paper, inspired by the\nTransformer-based language models, we propose a simple and unified paraphrasing\nmodel, which is purely trained on multilingual parallel data and can conduct\nzero-shot paraphrase generation in one step. Compared with the pivoting\napproach, paraphrases generated by our model is more semantically similar to\nthe input sentence. Moreover, since our model shares the same architecture as\nGPT (Radford et al., 2018), we are able to pre-train the model on large-scale\nunparallel corpus, which further improves the fluency of the output sentences.\nIn addition, we introduce the mechanism of denoising auto-encoder (DAE) to\nimprove diversity and robustness of the model. Experimental results show that\nour model surpasses the pivoting method in terms of relevance, diversity,\nfluency and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03601v1",
            "date": "2019-11-09 03:18:02+00:00",
            "title": "Table-to-Text Natural Language Generation with Unseen Schemas",
            "authors": [
                "Tianyu Liu",
                "Wei Wei",
                "William Yang Wang"
            ],
            "abstract": "Traditional table-to-text natural language generation (NLG) tasks focus on\ngenerating text from schemas that are already seen in the training set. This\nlimitation curbs their generalizabilities towards real-world scenarios, where\nthe schemas of input tables are potentially infinite. In this paper, we propose\nthe new task of table-to-text NLG with unseen schemas, which specifically aims\nto test the generalization of NLG for input tables with attribute types that\nnever appear during training. To do this, we construct a new benchmark dataset\nfor this task. To deal with the problem of unseen attribute types, we propose a\nnew model that first aligns unseen table schemas to seen ones, and then\ngenerates text with updated table representations. Experimental evaluation on\nthe new benchmark demonstrates that our model outperforms baseline methods by a\nlarge margin. In addition, comparison with standard data-to-text settings shows\nthe challenges and uniqueness of our proposed task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03627v1",
            "date": "2019-11-09 06:58:29+00:00",
            "title": "Learning to Copy for Automatic Post-Editing",
            "authors": [
                "Xuancheng Huang",
                "Yang Liu",
                "Huanbo Luan",
                "Jingfang Xu",
                "Maosong Sun"
            ],
            "abstract": "Automatic post-editing (APE), which aims to correct errors in the output of\nmachine translation systems in a post-processing step, is an important task in\nnatural language processing. While recent work has achieved considerable\nperformance gains by using neural networks, how to model the copying mechanism\nfor APE remains a challenge. In this work, we propose a new method for modeling\ncopying for APE. To better identify translation errors, our method learns the\nrepresentations of source sentences and system outputs in an interactive way.\nThese representations are used to explicitly indicate which words in the system\noutputs should be copied, which is useful to help CopyNet (Gu et al., 2016)\nbetter generate post-edited translations. Experiments on the datasets of the\nWMT 2016-2017 APE shared tasks show that our approach outperforms all best\npublished results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03631v4",
            "date": "2019-11-09 07:18:47+00:00",
            "title": "Hierarchical Graph Network for Multi-hop Question Answering",
            "authors": [
                "Yuwei Fang",
                "Siqi Sun",
                "Zhe Gan",
                "Rohit Pillai",
                "Shuohang Wang",
                "Jingjing Liu"
            ],
            "abstract": "In this paper, we present Hierarchical Graph Network (HGN) for multi-hop\nquestion answering. To aggregate clues from scattered texts across multiple\nparagraphs, a hierarchical graph is created by constructing nodes on different\nlevels of granularity (questions, paragraphs, sentences, entities), the\nrepresentations of which are initialized with pre-trained contextual encoders.\nGiven this hierarchical graph, the initial node representations are updated\nthrough graph propagation, and multi-hop reasoning is performed via traversing\nthrough the graph edges for each subsequent sub-task (e.g., paragraph\nselection, supporting facts extraction, answer prediction). By weaving\nheterogeneous nodes into an integral unified graph, this hierarchical\ndifferentiation of node granularity enables HGN to support different question\nanswering sub-tasks simultaneously. Experiments on the HotpotQA benchmark\ndemonstrate that the proposed model achieves new state of the art,\noutperforming existing multi-hop QA approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03644v3",
            "date": "2019-11-09 09:12:58+00:00",
            "title": "Hate Speech Detection on Vietnamese Social Media Text using the Bi-GRU-LSTM-CNN Model",
            "authors": [
                "Tin Van Huynh",
                "Vu Duc Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen",
                "Anh Gia-Tuan Nguyen"
            ],
            "abstract": "In recent years, Hate Speech Detection has become one of the interesting\nfields in natural language processing or computational linguistics. In this\npaper, we present the description of our system to solve this problem at the\nVLSP shared task 2019: Hate Speech Detection on Social Networks with the corpus\nwhich contains 20,345 human-labeled comments/posts for training and 5,086 for\npublic-testing. We implement a deep learning method based on the\nBi-GRU-LSTM-CNN classifier into this task. Our result in this task is 70.576%\nof F1-score, ranking the 5th of performance on public-test set.",
            "journal ref": "VLSP Workshop 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03668v1",
            "date": "2019-11-09 11:21:48+00:00",
            "title": "Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective",
            "authors": [
                "Zhen Cheng",
                "Zaixiang Zheng",
                "Xin-Yu Dai",
                "Shujian Huang",
                "Jiajun Chen"
            ],
            "abstract": "Natural Language Inference (NLI) aims to determine the logic relationships\n(i.e., entailment, neutral and contradiction) between a pair of premise and\nhypothesis. Recently, the alignment mechanism effectively helps NLI by\ncapturing the aligned parts (i.e., the similar segments) in the sentence pairs,\nwhich imply the perspective of entailment and contradiction. However, these\naligned parts will sometimes mislead the judgment of neutral relations.\nIntuitively, NLI should rely more on multiple perspectives to form a holistic\nview to eliminate bias. In this paper, we propose the Multi-Perspective\nInferrer (MPI), a novel NLI model that reasons relationships from multiple\nperspectives associated with the three relationships. The MPI determines the\nperspectives of different parts of the sentences via a routing-by-agreement\npolicy and makes the final decision from a holistic view. Additionally, we\nintroduce an auxiliary supervised signal to ensure the MPI to learn the\nexpected perspectives. Experiments on SNLI and MultiNLI show that 1) the MPI\nachieves substantial improvements on the base model, which verifies the\nmotivation of multi-perspective inference; 2) visualized evidence verifies that\nthe MPI learns highly interpretable perspectives as expected; 3) more\nimportantly, the MPI is architecture-free and compatible with the powerful\nBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03681v2",
            "date": "2019-11-09 13:08:25+00:00",
            "title": "E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT",
            "authors": [
                "Nina Poerner",
                "Ulli Waltinger",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present a novel way of injecting factual knowledge about entities into the\npretrained BERT model (Devlin et al., 2019): We align Wikipedia2Vec entity\nvectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use\nthe aligned entity vectors as if they were wordpiece vectors. The resulting\nentity-enhanced version of BERT (called E-BERT) is similar in spirit to ERNIE\n(Zhang et al., 2019) and KnowBert (Peters et al., 2019), but it requires no\nexpensive further pretraining of the BERT encoder. We evaluate E-BERT on\nunsupervised question answering (QA), supervised relation classification (RC)\nand entity linking (EL). On all three tasks, E-BERT outperforms BERT and other\nbaselines. We also show quantitatively that the original BERT model is overly\nreliant on the surface form of entity names (e.g., guessing that someone with\nan Italian-sounding name speaks Italian), and that E-BERT mitigates this\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03688v2",
            "date": "2019-11-09 13:35:18+00:00",
            "title": "ConveRT: Efficient and Accurate Conversational Representations from Transformers",
            "authors": [
                "Matthew Henderson",
                "I\u00f1igo Casanueva",
                "Nikola Mrk\u0161i\u0107",
                "Pei-Hao Su",
                "Tsung-Hsien Wen",
                "Ivan Vuli\u0107"
            ],
            "abstract": "General-purpose pretrained sentence encoders such as BERT are not ideal for\nreal-world conversational AI applications; they are computationally heavy,\nslow, and expensive to train. We propose ConveRT (Conversational\nRepresentations from Transformers), a pretraining framework for conversational\ntasks satisfying all the following requirements: it is effective, affordable,\nand quick to train. We pretrain using a retrieval-based response selection\ntask, effectively leveraging quantization and subword-level parameterization in\nthe dual encoder to build a lightweight memory- and energy-efficient model. We\nshow that ConveRT achieves state-of-the-art performance across widely\nestablished response selection tasks. We also demonstrate that the use of\nextended dialog history as context yields further performance gains. Finally,\nwe show that pretrained representations from the proposed encoder can be\ntransferred to the intent classification task, yielding strong results across\nthree diverse data sets. ConveRT trains substantially faster than standard\nsentence encoders or previous state-of-the-art dual encoders. With its reduced\nsize and superior performance, we believe this model promises wider portability\nand scalability for Conversational AI applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03700v3",
            "date": "2019-11-09 14:31:03+00:00",
            "title": "Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity",
            "authors": [
                "Nina Poerner",
                "Ulli Waltinger",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We address the task of unsupervised Semantic Textual Similarity (STS) by\nensembling diverse pre-trained sentence encoders into sentence meta-embeddings.\nWe apply, extend and evaluate different meta-embedding methods from the word\nembedding literature at the sentence level, including dimensionality reduction\n(Yin and Sch\\\"utze, 2016), generalized Canonical Correlation Analysis (Rastogi\net al., 2015) and cross-view auto-encoders (Bollegala and Bao, 2018). Our\nsentence meta-embeddings set a new unsupervised State of The Art (SoTA) on the\nSTS Benchmark and on the STS12-STS16 datasets, with gains of between 3.7% and\n6.4% Pearson's r over single-source systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03724v1",
            "date": "2019-11-09 16:00:26+00:00",
            "title": "Error Analysis for Vietnamese Dependency Parsing",
            "authors": [
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Dependency parsing is needed in different applications of natural language\nprocessing. In this paper, we present a thorough error analysis for dependency\nparsing for the Vietnamese language, using two state-of-the-art parsers:\nMSTParser and MaltParser. The error analysis results provide us insights in\norder to improve the performance of dependency parsing for the Vietnamese\nlanguage.",
            "journal ref": "2015 Seventh International Conference on Knowledge and Systems\n  Engineering (KSE)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03726v1",
            "date": "2019-11-09 16:07:38+00:00",
            "title": "Vietnamese transition-based dependency parsing with supertag features",
            "authors": [
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In recent years, dependency parsing is a fascinating research topic and has a\nlot of applications in natural language processing. In this paper, we present\nan effective approach to improve dependency parsing by utilizing supertag\nfeatures. We performed experiments with the transition-based dependency parsing\napproach because it can take advantage of rich features. Empirical evaluation\non Vietnamese Dependency Treebank showed that, we achieved an improvement of\n18.92% in labeled attachment score with gold supertags and an improvement of\n3.57% with automatic supertags.",
            "journal ref": "In Proceeding of the 2016 Eighth International Conference on\n  Knowledge and Systems Engineering (KSE)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03766v3",
            "date": "2019-11-09 20:00:03+00:00",
            "title": "Multi-Sentence Argument Linking",
            "authors": [
                "Seth Ebner",
                "Patrick Xia",
                "Ryan Culkin",
                "Kyle Rawlins",
                "Benjamin Van Durme"
            ],
            "abstract": "We present a novel document-level model for finding argument spans that fill\nan event's roles, connecting related ideas in sentence-level semantic role\nlabeling and coreference resolution. Because existing datasets for\ncross-sentence linking are small, development of our neural model is supported\nthrough the creation of a new resource, Roles Across Multiple Sentences (RAMS),\nwhich contains 9,124 annotated events across 139 types. We demonstrate strong\nperformance of our model on RAMS and other event-related datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03772v2",
            "date": "2019-11-09 20:41:54+00:00",
            "title": "Code-Mixed to Monolingual Translation Framework",
            "authors": [
                "Sainik Kumar Mahata",
                "Soumil Mandal",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "The use of multilingualism in the new generation is widespread in the form of\ncode-mixed data on social media, and therefore a robust translation system is\nrequired for catering to the monolingual users, as well as for easier\ncomprehension by language processing models. In this work, we present a\ntranslation framework that uses a translation-transliteration strategy for\ntranslating code-mixed data into their equivalent monolingual instances. For\nconverting the output to a more fluent form, it is reordered using a target\nlanguage model. The most important advantage of the proposed framework is that\nit does not require a code-mixed to monolingual parallel corpus at any point.\nOn testing the framework, it achieved BLEU and TER scores of 16.47 and 55.45,\nrespectively. Since the proposed framework deals with various sub-modules, we\ndive deeper into the importance of each of them, analyze the errors and\nfinally, discuss some improvement strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03776v1",
            "date": "2019-11-09 20:59:40+00:00",
            "title": "Subjective Sentiment Analysis for Arabic Newswire Comments",
            "authors": [
                "Sadik Bessou",
                "Rania Aberkane"
            ],
            "abstract": "This paper presents an approach based on supervised machine learning methods\nto discriminate between positive, negative and neutral Arabic reviews in online\nnewswire. The corpus is labeled for subjectivity and sentiment analysis (SSA)\nat the sentence-level. The model uses both count and TF-IDF representations and\napply six machine learning algorithms; Multinomial Naive Bayes, Support Vector\nMachines (SVM), Random Forest, Logistic Regression, Multi-layer perceptron and\nk-nearest neighbors using uni-grams, bi-grams features. With the goal of\nextracting users sentiment from written text. Experimental results showed that\nn-gram features could substantially improve performance; and showed that the\nMultinomial Naive Bayes approach is the most accurate in predicting topic\npolarity. Best results were achieved using count vectors trained by combination\nof word-based uni-grams and bi-grams with an overall accuracy of 85.57% over\ntwo classes and 65.64% over three classes.",
            "journal ref": "Journal of Digital Information Management ISSN 0972-7272 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03785v2",
            "date": "2019-11-09 22:06:39+00:00",
            "title": "PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction",
            "authors": [
                "Yichun Yin",
                "Chenguang Wang",
                "Ming Zhang"
            ],
            "abstract": "Dependency context-based word embedding jointly learns the representations of\nword and dependency context, and has been proved effective in aspect term\nextraction. In this paper, we design the positional dependency-based word\nembedding (PoD) which considers both dependency context and positional context\nfor aspect term extraction. Specifically, the positional context is modeled via\nrelative position encoding. Besides, we enhance the dependency context by\nintegrating more lexical information (e.g., POS tags) along dependency paths.\nExperiments on SemEval 2014/2015/2016 datasets show that our approach\noutperforms other embedding methods in aspect term extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03814v3",
            "date": "2019-11-10 01:01:45+00:00",
            "title": "Scalable Zero-shot Entity Linking with Dense Entity Retrieval",
            "authors": [
                "Ledell Wu",
                "Fabio Petroni",
                "Martin Josifoski",
                "Sebastian Riedel",
                "Luke Zettlemoyer"
            ],
            "abstract": "This paper introduces a conceptually simple, scalable, and highly effective\nBERT-based entity linking model, along with an extensive evaluation of its\naccuracy-speed trade-off. We present a two-stage zero-shot linking algorithm,\nwhere each entity is defined only by a short textual description. The first\nstage does retrieval in a dense space defined by a bi-encoder that\nindependently embeds the mention context and the entity descriptions. Each\ncandidate is then re-ranked with a cross-encoder, that concatenates the mention\nand entity text. Experiments demonstrate that this approach is state of the art\non recent zero-shot benchmarks (6 point absolute gains) and also on more\nestablished non-zero-shot evaluations (e.g. TACKBP-2010), despite its relative\nsimplicity (e.g. no explicit entity embeddings or manually engineered mention\ntables). We also show that bi-encoder linking is very fast with nearest\nneighbour search (e.g. linking with 5.9 million candidates in 2 milliseconds),\nand that much of the accuracy gain from the more expensive cross-encoder can be\ntransferred to the bi-encoder via knowledge distillation. Our code and models\nare available at https://github.com/facebookresearch/BLINK.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03817v3",
            "date": "2019-11-10 01:32:02+00:00",
            "title": "Adversarial Learning on the Latent Space for Diverse Dialog Generation",
            "authors": [
                "Kashif Khan",
                "Gaurav Sahu",
                "Vikash Balasubramanian",
                "Lili Mou",
                "Olga Vechtomova"
            ],
            "abstract": "Generating relevant responses in a dialog is challenging, and requires not\nonly proper modeling of context in the conversation but also being able to\ngenerate fluent sentences during inference. In this paper, we propose a\ntwo-step framework based on generative adversarial nets for generating\nconditioned responses. Our model first learns a meaningful representation of\nsentences by autoencoding and then learns to map an input query to the response\nrepresentation, which is in turn decoded as a response sentence. Both\nquantitative and qualitative evaluations show that our model generates more\nfluent, relevant, and diverse responses than existing state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03822v2",
            "date": "2019-11-10 01:42:14+00:00",
            "title": "Generalizing Natural Language Analysis through Span-relation Representations",
            "authors": [
                "Zhengbao Jiang",
                "Wei Xu",
                "Jun Araki",
                "Graham Neubig"
            ],
            "abstract": "Natural language processing covers a wide variety of tasks predicting syntax,\nsemantics, and information content, and usually each type of output is\ngenerated with specially designed architectures. In this paper, we provide the\nsimple insight that a great variety of tasks can be represented in a single\nunified format consisting of labeling spans and relations between spans, thus a\nsingle task-independent model can be used across different tasks. We perform\nextensive experiments to test this insight on 10 disparate tasks spanning\ndependency parsing (syntax), semantic role labeling (semantics), relation\nextraction (information content), aspect based sentiment analysis (sentiment),\nand many others, achieving performance comparable to state-of-the-art\nspecialized models. We further demonstrate benefits of multi-task learning, and\nalso show that the proposed method makes it easy to analyze differences and\nsimilarities in how the model handles different tasks. Finally, we convert\nthese datasets into a unified format to build a benchmark, which provides a\nholistic testbed for evaluating future models for generalized natural language\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03823v2",
            "date": "2019-11-10 01:43:22+00:00",
            "title": "Translationese as a Language in \"Multilingual\" NMT",
            "authors": [
                "Parker Riley",
                "Isaac Caswell",
                "Markus Freitag",
                "David Grangier"
            ],
            "abstract": "Machine translation has an undesirable propensity to produce \"translationese\"\nartifacts, which can lead to higher BLEU scores while being liked less by human\nraters. Motivated by this, we model translationese and original (i.e. natural)\ntext as separate languages in a multilingual model, and pose the question: can\nwe perform zero-shot translation between original source text and original\ntarget text? There is no data with original source and original target, so we\ntrain sentence-level classifiers to distinguish translationese from original\ntarget text, and use this classifier to tag the training data for an NMT model.\nUsing this technique we bias the model to produce more natural outputs at test\ntime, yielding gains in human evaluation scores on both accuracy and fluency.\nAdditionally, we demonstrate that it is possible to bias the model to produce\ntranslationese and game the BLEU score, increasing it while decreasing\nhuman-rated quality. We analyze these models using metrics to measure the\ndegree of translationese in the output, and present an analysis of the\ncapriciousness of heuristically-based train-data tagging.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics (2020) 7737-7746",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03828v1",
            "date": "2019-11-10 02:06:23+00:00",
            "title": "Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior",
            "authors": [
                "Amirpasha Ghabussi",
                "Lili Mou",
                "Olga Vechtomova"
            ],
            "abstract": "Wasserstein autoencoders are effective for text generation. They do not\nhowever provide any control over the style and topic of the generated sentences\nif the dataset has multiple classes and includes different topics. In this\nwork, we present a semi-supervised approach for generating stylized sentences.\nOur model is trained on a multi-class dataset and learns the latent\nrepresentation of the sentences using a mixture of Gaussian prior without any\nadversarial losses. This allows us to generate sentences in the style of a\nspecified class or multiple classes by sampling from their corresponding prior\ndistributions. Moreover, we can train our model on relatively small datasets\nand learn the latent representation of a specified class by adding external\ndata with other styles/classes to our dataset. While a simple WAE or VAE cannot\ngenerate diverse sentences in this case, generated sentences with our approach\nare diverse, fluent, and preserve the style and the content of the desired\nclasses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03834v3",
            "date": "2019-11-10 02:26:19+00:00",
            "title": "Contextualized End-to-End Neural Entity Linking",
            "authors": [
                "Haotian Chen",
                "Andrej Zukov-Gregoric",
                "Xi David Li",
                "Sahil Wadhwa"
            ],
            "abstract": "We propose yet another entity linking model (YELM) which links words to\nentities instead of spans. This overcomes any difficulties associated with the\nselection of good candidate mention spans and makes the joint training of\nmention detection (MD) and entity disambiguation (ED) easily possible. Our\nmodel is based on BERT and produces contextualized word embeddings which are\ntrained against a joint MD and ED objective. We achieve state-of-the-art\nresults on several standard entity linking (EL) datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03842v2",
            "date": "2019-11-10 03:10:50+00:00",
            "title": "Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation",
            "authors": [
                "Emily Dinan",
                "Angela Fan",
                "Adina Williams",
                "Jack Urbanek",
                "Douwe Kiela",
                "Jason Weston"
            ],
            "abstract": "Models often easily learn biases present in the training data, and their\npredictions directly reflect this bias. We analyze gender bias in dialogue\ndata, and examine how this bias is actually amplified in subsequent generative\nchit-chat dialogue models. We measure gender bias in six existing dialogue\ndatasets, and focus on the most biased one, the multi-player text-based fantasy\nadventure dataset LIGHT, as a testbed for our bias mitigation techniques. The\nLIGHT dataset is highly imbalanced with respect to gender, containing\npredominantly male characters, likely because it is entirely collected by\ncrowdworkers and reflects common biases that exist in fantasy or medieval\nsettings. We consider three techniques to mitigate gender bias: counterfactual\ndata augmentation, targeted data collection, and bias controlled training. We\nshow that our proposed techniques mitigate gender bias in LIGHT by balancing\nthe genderedness of generated dialogue utterances and are particularly\neffective in combination. We quantify performance using various evaluation\nmethods---such as quantity of gendered words, a dialogue safety classifier, and\nhuman studies---all of which show that our models generate less gendered, but\nequally engaging chit-chat responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03860v2",
            "date": "2019-11-10 05:53:40+00:00",
            "title": "Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training",
            "authors": [
                "Margaret Li",
                "Stephen Roller",
                "Ilia Kulikov",
                "Sean Welleck",
                "Y-Lan Boureau",
                "Kyunghyun Cho",
                "Jason Weston"
            ],
            "abstract": "Generative dialogue models currently suffer from a number of problems which\nstandard maximum likelihood training does not address. They tend to produce\ngenerations that (i) rely too much on copying from the context, (ii) contain\nrepetitions within utterances, (iii) overuse frequent words, and (iv) at a\ndeeper level, contain logical flaws. In this work we show how all of these\nproblems can be addressed by extending the recently introduced unlikelihood\nloss (Welleck et al., 2019) to these cases. We show that appropriate loss\nfunctions which regularize generated outputs to match human distributions are\neffective for the first three issues. For the last important general issue, we\nshow applying unlikelihood to collected data of what a model should not do is\neffective for improving logical consistency, potentially paving the way to\ngenerative models with greater reasoning ability. We demonstrate the efficacy\nof our approach across several dialogue tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03862v1",
            "date": "2019-11-10 06:04:35+00:00",
            "title": "Unsupervised Annotation of Phenotypic Abnormalities via Semantic Latent Representations on Electronic Health Records",
            "authors": [
                "Jingqing Zhang",
                "Xiaoyu Zhang",
                "Kai Sun",
                "Xian Yang",
                "Chengliang Dai",
                "Yike Guo"
            ],
            "abstract": "The extraction of phenotype information which is naturally contained in\nelectronic health records (EHRs) has been found to be useful in various\nclinical informatics applications such as disease diagnosis. However, due to\nimprecise descriptions, lack of gold standards and the demand for efficiency,\nannotating phenotypic abnormalities on millions of EHR narratives is still\nchallenging. In this work, we propose a novel unsupervised deep learning\nframework to annotate the phenotypic abnormalities from EHRs via semantic\nlatent representations. The proposed framework takes the advantage of Human\nPhenotype Ontology (HPO), which is a knowledge base of phenotypic\nabnormalities, to standardize the annotation results. Experiments have been\nconducted on 52,722 EHRs from MIMIC-III dataset. Quantitative and qualitative\nanalysis have shown the proposed framework achieves state-of-the-art annotation\nperformance and computational efficiency compared with other methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03876v2",
            "date": "2019-11-10 08:20:20+00:00",
            "title": "Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering",
            "authors": [
                "Antoine Bosselut",
                "Ronan Le Bras",
                "Yejin Choi"
            ],
            "abstract": "Understanding narratives requires reasoning about implicit world knowledge\nrelated to the causes, effects, and states of situations described in text. At\nthe core of this challenge is how to access contextually relevant knowledge on\ndemand and reason over it.\n  In this paper, we present initial studies toward zero-shot commonsense\nquestion answering by formulating the task as inference over dynamically\ngenerated commonsense knowledge graphs. In contrast to previous studies for\nknowledge integration that rely on retrieval of existing knowledge from static\nknowledge graphs, our study requires commonsense knowledge integration where\ncontextually relevant knowledge is often not present in existing knowledge\nbases. Therefore, we present a novel approach that generates\ncontextually-relevant symbolic knowledge structures on demand using generative\nneural commonsense knowledge models.\n  Empirical results on two datasets demonstrate the efficacy of our\nneuro-symbolic approach for dynamically constructing knowledge graphs for\nreasoning. Our approach achieves significant performance boosts over pretrained\nlanguage models and vanilla knowledge models, all while providing interpretable\nreasoning paths for its predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03891v3",
            "date": "2019-11-10 10:38:27+00:00",
            "title": "Social Bias Frames: Reasoning about Social and Power Implications of Language",
            "authors": [
                "Maarten Sap",
                "Saadia Gabriel",
                "Lianhui Qin",
                "Dan Jurafsky",
                "Noah A. Smith",
                "Yejin Choi"
            ],
            "abstract": "Warning: this paper contains content that may be offensive or upsetting.\n  Language has the power to reinforce stereotypes and project social biases\nonto others. At the core of the challenge is that it is rarely what is stated\nexplicitly, but rather the implied meanings, that frame people's judgments\nabout others. For example, given a statement that \"we shouldn't lower our\nstandards to hire more women,\" most listeners will infer the implicature\nintended by the speaker -- that \"women (candidates) are less qualified.\" Most\nsemantic formalisms, to date, do not capture such pragmatic implications in\nwhich people express social biases and power differentials in language.\n  We introduce Social Bias Frames, a new conceptual formalism that aims to\nmodel the pragmatic frames in which people project social biases and\nstereotypes onto others. In addition, we introduce the Social Bias Inference\nCorpus to support large-scale modelling and evaluation with 150k structured\nannotations of social media posts, covering over 34k implications about a\nthousand demographic groups.\n  We then establish baseline approaches that learn to recover Social Bias\nFrames from unstructured text. We find that while state-of-the-art neural\nmodels are effective at high-level categorization of whether a given statement\nprojects unwanted social bias (80% F1), they are not effective at spelling out\nmore detailed explanations in terms of Social Bias Frames. Our study motivates\nfuture work that combines structured pragmatic inference with commonsense\nreasoning on social implications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03892v2",
            "date": "2019-11-10 10:41:52+00:00",
            "title": "INSET: Sentence Infilling with INter-SEntential Transformer",
            "authors": [
                "Yichen Huang",
                "Yizhe Zhang",
                "Oussama Elachqar",
                "Yu Cheng"
            ],
            "abstract": "Missing sentence generation (or sentence infilling) fosters a wide range of\napplications in natural language generation, such as document auto-completion\nand meeting note expansion. This task asks the model to generate intermediate\nmissing sentences that can syntactically and semantically bridge the\nsurrounding context. Solving the sentence infilling task requires techniques in\nnatural language processing ranging from understanding to discourse-level\nplanning to generation. In this paper, we propose a framework to decouple the\nchallenge and address these three aspects respectively, leveraging the power of\nexisting large-scale pre-trained models such as BERT and GPT-2. We empirically\ndemonstrate the effectiveness of our model in learning a sentence\nrepresentation for generation and further generating a missing sentence that\nfits the context.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics, pages 2502-2515 (long paper), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03894v3",
            "date": "2019-11-10 10:46:37+00:00",
            "title": "CamemBERT: a Tasty French Language Model",
            "authors": [
                "Louis Martin",
                "Benjamin Muller",
                "Pedro Javier Ortiz Su\u00e1rez",
                "Yoann Dupont",
                "Laurent Romary",
                "\u00c9ric Villemonte de la Clergerie",
                "Djam\u00e9 Seddah",
                "Beno\u00eet Sagot"
            ],
            "abstract": "Pretrained language models are now ubiquitous in Natural Language Processing.\nDespite their success, most available models have either been trained on\nEnglish data or on the concatenation of data in multiple languages. This makes\npractical use of such models --in all languages except English-- very limited.\nIn this paper, we investigate the feasibility of training monolingual\nTransformer-based language models for other languages, taking French as an\nexample and evaluating our language models on part-of-speech tagging,\ndependency parsing, named entity recognition and natural language inference\ntasks. We show that the use of web crawled data is preferable to the use of\nWikipedia data. More surprisingly, we show that a relatively small web crawled\ndataset (4GB) leads to results that are as good as those obtained using larger\ndatasets (130+GB). Our best performing model CamemBERT reaches or improves the\nstate of the art in all four downstream tasks.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics, July 2020, Online",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03897v1",
            "date": "2019-11-10 10:55:12+00:00",
            "title": "Two-Headed Monster And Crossed Co-Attention Networks",
            "authors": [
                "Yaoyiran Li",
                "Jing Jiang"
            ],
            "abstract": "This paper presents some preliminary investigations of a new co-attention\nmechanism in neural transduction models. We propose a paradigm, termed\nTwo-Headed Monster (THM), which consists of two symmetric encoder modules and\none decoder module connected with co-attention. As a specific and concrete\nimplementation of THM, Crossed Co-Attention Networks (CCNs) are designed based\non the Transformer model. We demonstrate CCNs on WMT 2014 EN-DE and WMT 2016\nEN-FI translation tasks and our model outperforms the strong Transformer\nbaseline by 0.51 (big) and 0.74 (base) BLEU points on EN-DE and by 0.17 (big)\nand 0.47 (base) BLEU points on EN-FI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03903v3",
            "date": "2019-11-10 11:19:08+00:00",
            "title": "A Re-evaluation of Knowledge Graph Completion Methods",
            "authors": [
                "Zhiqing Sun",
                "Shikhar Vashishth",
                "Soumya Sanyal",
                "Partha Talukdar",
                "Yiming Yang"
            ],
            "abstract": "Knowledge Graph Completion (KGC) aims at automatically predicting missing\nlinks for large-scale knowledge graphs. A vast number of state-of-the-art KGC\ntechniques have got published at top conferences in several research fields,\nincluding data mining, machine learning, and natural language processing.\nHowever, we notice that several recent papers report very high performance,\nwhich largely outperforms previous state-of-the-art methods. In this paper, we\nfind that this can be attributed to the inappropriate evaluation protocol used\nby them and propose a simple evaluation protocol to address this problem. The\nproposed protocol is robust to handle bias in the model, which can\nsubstantially affect the final results. We conduct extensive experiments and\nreport the performance of several existing methods using our protocol. The\nreproducible code has been made publicly available",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03905v1",
            "date": "2019-11-10 11:24:02+00:00",
            "title": "Semantic Noise Matters for Neural Natural Language Generation",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "David M. Howcroft",
                "Verena Rieser"
            ],
            "abstract": "Neural natural language generation (NNLG) systems are known for their\npathological outputs, i.e. generating text which is unrelated to the input\nspecification. In this paper, we show the impact of semantic noise on\nstate-of-the-art NNLG models which implement different semantic control\nmechanisms. We find that cleaned data can improve semantic correctness by up to\n97%, while maintaining fluency. We also find that the most common error is\nomitting information, rather than hallucination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03906v2",
            "date": "2019-11-10 11:27:53+00:00",
            "title": "Efficient Dialogue State Tracking by Selectively Overwriting Memory",
            "authors": [
                "Sungdong Kim",
                "Sohee Yang",
                "Gyuwan Kim",
                "Sang-Woo Lee"
            ],
            "abstract": "Recent works in dialogue state tracking (DST) focus on an open\nvocabulary-based setting to resolve scalability and generalization issues of\nthe predefined ontology-based approaches. However, they are inefficient in that\nthey predict the dialogue state at every turn from scratch. Here, we consider\ndialogue state as an explicit fixed-sized memory and propose a selectively\noverwriting mechanism for more efficient DST. This mechanism consists of two\nsteps: (1) predicting state operation on each of the memory slots, and (2)\noverwriting the memory with new values, of which only a few are generated\naccording to the predicted state operations. Our method decomposes DST into two\nsub-tasks and guides the decoder to focus only on one of the tasks, thus\nreducing the burden of the decoder. This enhances the effectiveness of training\nand DST performance. Our SOM-DST (Selectively Overwriting Memory for Dialogue\nState Tracking) model achieves state-of-the-art joint goal accuracy with 51.72%\nin MultiWOZ 2.0 and 53.01% in MultiWOZ 2.1 in an open vocabulary-based DST\nsetting. In addition, we analyze the accuracy gaps between the current and the\nground truth-given situations and suggest that it is a promising direction to\nimprove state operation prediction to boost the DST performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03911v2",
            "date": "2019-11-10 11:50:09+00:00",
            "title": "Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines",
            "authors": [
                "\u0141ukasz Borchmann",
                "Dawid Wi\u015bniewski",
                "Andrzej Gretkowski",
                "Izabela Kosmala",
                "Dawid Jurkiewicz",
                "\u0141ukasz Sza\u0142kiewicz",
                "Gabriela Pa\u0142ka",
                "Karol Kaczmarek",
                "Agnieszka Kaliska",
                "Filip Grali\u0144ski"
            ],
            "abstract": "We propose a new shared task of semantic retrieval from legal texts, in which\na so-called contract discovery is to be performed, where legal clauses are\nextracted from documents, given a few examples of similar clauses from other\nlegal acts. The task differs substantially from conventional NLI and shared\ntasks on legal information extraction (e.g., one has to identify text span\ninstead of a single document, page, or paragraph). The specification of the\nproposed task is followed by an evaluation of multiple solutions within the\nunified framework proposed for this branch of methods. It is shown that\nstate-of-the-art pretrained encoders fail to provide satisfactory results on\nthe task proposed. In contrast, Language Model-based solutions perform better,\nespecially when unsupervised fine-tuning is applied. Besides the ablation\nstudies, we addressed questions regarding detection accuracy for relevant text\nfragments depending on the number of examples available. In addition to the\ndataset and reference results, LMs specialized in the legal domain were made\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03913v1",
            "date": "2019-11-10 11:53:26+00:00",
            "title": "Can Monolingual Pretrained Models Help Cross-Lingual Classification?",
            "authors": [
                "Zewen Chi",
                "Li Dong",
                "Furu Wei",
                "Xian-Ling Mao",
                "Heyan Huang"
            ],
            "abstract": "Multilingual pretrained language models (such as multilingual BERT) have\nachieved impressive results for cross-lingual transfer. However, due to the\nconstant model capacity, multilingual pre-training usually lags behind the\nmonolingual competitors. In this work, we present two approaches to improve\nzero-shot cross-lingual classification, by transferring the knowledge from\nmonolingual pretrained models to multilingual ones. Experimental results on two\ncross-lingual classification benchmarks show that our methods outperform\nvanilla multilingual fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03914v1",
            "date": "2019-11-10 11:53:43+00:00",
            "title": "Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen Styles",
            "authors": [
                "Eric Michael Smith",
                "Diana Gonzalez-Rico",
                "Emily Dinan",
                "Y-Lan Boureau"
            ],
            "abstract": "Text style transfer is usually performed using attributes that can take a\nhandful of discrete values (e.g., positive to negative reviews). In this work,\nwe introduce an architecture that can leverage pre-trained consistent\ncontinuous distributed style representations and use them to transfer to an\nattribute unseen during training, without requiring any re-tuning of the style\ntransfer model. We demonstrate the method by training an architecture to\ntransfer text conveying one sentiment to another sentiment, using a\nfine-grained set of over 20 sentiment labels rather than the binary\npositive/negative often used in style transfer. Our experiments show that this\nmodel can then rewrite text to match a target sentiment that was unseen during\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03918v2",
            "date": "2019-11-10 12:17:35+00:00",
            "title": "Improving BERT Fine-tuning with Embedding Normalization",
            "authors": [
                "Wenxuan Zhou",
                "Junyi Du",
                "Xiang Ren"
            ],
            "abstract": "Large pre-trained sentence encoders like BERT start a new chapter in natural\nlanguage processing. A common practice to apply pre-trained BERT to sequence\nclassification tasks (e.g., classification of sentences or sentence pairs) is\nby feeding the embedding of [CLS] token (in the last layer) to a task-specific\nclassification layer, and then fine tune the model parameters of BERT and\nclassifier jointly. In this paper, we conduct systematic analysis over several\nsequence classification datasets to examine the embedding values of [CLS] token\nbefore the fine tuning phase, and present the biased embedding distribution\nissue---i.e., embedding values of [CLS] concentrate on a few dimensions and are\nnon-zero centered. Such biased embedding brings challenge to the optimization\nprocess during fine-tuning as gradients of [CLS] embedding may explode and\nresult in degraded model performance. We further propose several simple yet\neffective normalization methods to modify the [CLS] embedding during the\nfine-tuning. Compared with the previous practice, neural classification model\nwith the normalized embedding shows improvements on several text classification\ntasks, demonstrates the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03922v2",
            "date": "2019-11-10 12:53:38+00:00",
            "title": "Un systeme de lemmatisation pour les applications de TALN",
            "authors": [
                "Sadik Bessou",
                "Mohamed Louail",
                "Allaoua Refoufi",
                "Zehour Kadem",
                "Mohamed Touahria"
            ],
            "abstract": "This paper presents a method of stemming for the Arabian texts based on the\nlinguistic techniques of the natural language processing. This method leans on\nthe notion of scheme (one of the strong points of the morphology of the Arabian\nlanguage). The advantage of this approach is that it doesn't use a dictionary\nof inflexions but a smart dynamic recognition of the different words of the\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03937v1",
            "date": "2019-11-10 14:04:59+00:00",
            "title": "Language Model-Driven Unsupervised Neural Machine Translation",
            "authors": [
                "Wei Zhang",
                "Youyuan Lin",
                "Ruoran Ren",
                "Xiaodong Wang",
                "Zhenshuang Liang",
                "Zhen Huang"
            ],
            "abstract": "Unsupervised neural machine translation(NMT) is associated with noise and\nerrors in synthetic data when executing vanilla back-translations. Here, we\nexplicitly exploits language model(LM) to drive construction of an unsupervised\nNMT system. This features two steps. First, we initialize NMT models using\nsynthetic data generated via temporary statistical machine translation(SMT).\nSecond, unlike vanilla back-translation, we formulate a weight function, that\nscores synthetic data at each step of subsequent iterative training; this\nallows unsupervised training to an improved outcome. We present the detailed\nmathematical construction of our method. Experimental WMT2014 English-French,\nand WMT2016 English-German and English-Russian translation tasks revealed that\nour method outperforms the best prior systems by more than 3 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04015v2",
            "date": "2019-11-11 00:07:02+00:00",
            "title": "Word Sense Disambiguation using Knowledge-based Word Similarity",
            "authors": [
                "Sunjae Kwon",
                "Dongsuk Oh",
                "Youngjoong Ko"
            ],
            "abstract": "In natural language processing, word-sense disambiguation (WSD) is an open\nproblem concerned with identifying the correct sense of words in a particular\ncontext. To address this problem, we introduce a novel knowledge-based WSD\nsystem. We suggest the adoption of two methods in our system. First, we suggest\na novel method to encode the word vector representation by considering the\ngraphical semantic relationships from the lexical knowledge-base. Second, we\npropose a method for extracting the contextual words from the text for\nanalyzing an ambiguous word based on the similarity of word vector\nrepresentations. To validate the effectiveness of our WSD system, we conducted\nexperiments on the five benchmark English WSD corpora (Senseval-02,\nSenseval-03, SemEval-07, SemEval-13, and SemEval-15). The obtained results\ndemonstrated that the suggested methods significantly enhanced the WSD\nperformance. Furthermore, our system outperformed the existing knowledge-based\nWSD systems and showed a performance comparable to that of the state-of-the-art\nsupervised WSD systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04053v2",
            "date": "2019-11-11 03:15:22+00:00",
            "title": "Decompressing Knowledge Graph Representations for Link Prediction",
            "authors": [
                "Xiang Kong",
                "Xianyang Chen",
                "Eduard Hovy"
            ],
            "abstract": "This paper studies the problem of predicting missing relationships between\nentities in knowledge graphs through learning their representations. Currently,\nthe majority of existing link prediction models employ simple but intuitive\nscoring functions and relatively small embedding size so that they could be\napplied to large-scale knowledge graphs. However, these properties also\nrestrict the ability to learn more expressive and robust features. Therefore,\ndiverging from most of the prior works which focus on designing new objective\nfunctions, we propose, DeCom, a simple but effective mechanism to boost the\nperformance of existing link predictors such as DistMult, ComplEx, etc, through\nextracting more expressive features while preventing overfitting by adding just\na few extra parameters. Specifically, embeddings of entities and relationships\nare first decompressed to a more expressive and robust space by decompressing\nfunctions, then knowledge graph embedding models are trained in this new\nfeature space. Experimental results on several benchmark knowledge graphs and\nadvanced link prediction systems demonstrate the generalization and\neffectiveness of our method. Especially, RESCAL + DeCom achieves\nstate-of-the-art performance on the FB15k-237 benchmark across all evaluation\nmetrics. In addition, we also show that compared with DeCom, explicitly\nincreasing the embedding size significantly increase the number of parameters\nbut could not achieve promising performance improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04088v1",
            "date": "2019-11-11 05:43:45+00:00",
            "title": "DialogAct2Vec: Towards End-to-End Dialogue Agent by Multi-Task Representation Learning",
            "authors": [
                "Zhuoxuan Jiang",
                "Ziming Huang",
                "Dong Sheng Li",
                "Xian-Ling Mao"
            ],
            "abstract": "In end-to-end dialogue modeling and agent learning, it is important to (1)\neffectively learn knowledge from data, and (2) fully utilize heterogeneous\ninformation, e.g., dialogue act flow and utterances. However, the majority of\nexisting methods cannot simultaneously satisfy the two conditions. For example,\nrule definition and data labeling during system design take too much manual\nwork, and sequence-to-sequence methods only model one-side utterance\ninformation. In this paper, we propose a novel joint end-to-end model by\nmulti-task representation learning, which can capture the knowledge from\nheterogeneous information through automatically learning knowledgeable\nlow-dimensional embeddings from data, named with DialogAct2Vec. The model\nrequires little manual work for intervention in system design and we find that\nthe multi-task learning can greatly improve the effectiveness of representation\nlearning. Extensive experiments on a public dataset for restaurant reservation\nshow that the proposed method leads to significant improvements against the\nstate-of-the-art baselines on both the act prediction task and utterance\nprediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04115v2",
            "date": "2019-11-11 07:28:25+00:00",
            "title": "Text classification with pixel embedding",
            "authors": [
                "Bin Liu",
                "Guosheng Yin",
                "Wenbin Du"
            ],
            "abstract": "We propose a novel framework to understand the text by converting sentences\nor articles into video-like 3-dimensional tensors. Each frame, corresponding to\na slice of the tensor, is a word image that is rendered by the word's shape.\nThe length of the tensor equals to the number of words in the sentence or\narticle. The proposed transformation from the text to a 3-dimensional tensor\nmakes it very convenient to implement an $n$-gram model with convolutional\nneural networks for text analysis. Concretely, we impose a 3-dimensional\nconvolutional kernel on the 3-dimensional text tensor. The first two dimensions\nof the convolutional kernel size equal the size of the word image and the last\ndimension of the kernel size is $n$. That is, every time when we slide the\n3-dimensional kernel over a word sequence, the convolution covers $n$ word\nimages and outputs a scalar. By iterating this process continuously for each\n$n$-gram along with the sentence or article with multiple kernels, we obtain a\n2-dimensional feature map. A subsequent 1-dimensional max-over-time pooling is\napplied to this feature map, and three fully-connected layers are used for\nconducting text classification finally. Experiments of several text\nclassification datasets demonstrate surprisingly superior performances using\nthe proposed model in comparison with existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04118v2",
            "date": "2019-11-11 07:40:37+00:00",
            "title": "TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection",
            "authors": [
                "Siddhant Garg",
                "Thuy Vu",
                "Alessandro Moschitti"
            ],
            "abstract": "We propose TANDA, an effective technique for fine-tuning pre-trained\nTransformer models for natural language tasks. Specifically, we first transfer\na pre-trained model into a model for a general task by fine-tuning it with a\nlarge and high-quality dataset. We then perform a second fine-tuning step to\nadapt the transferred model to the target domain. We demonstrate the benefits\nof our approach for answer sentence selection, which is a well-known inference\ntask in Question Answering. We built a large scale dataset to enable the\ntransfer step, exploiting the Natural Questions dataset. Our approach\nestablishes the state of the art on two well-known benchmarks, WikiQA and\nTREC-QA, achieving MAP scores of 92% and 94.3%, respectively, which largely\noutperform the previous highest scores of 83.4% and 87.5%, obtained in very\nrecent work. We empirically show that TANDA generates more stable and robust\nmodels reducing the effort required for selecting optimal hyper-parameters.\nAdditionally, we show that the transfer step of TANDA makes the adaptation step\nmore robust to noise. This enables a more effective use of noisy datasets for\nfine-tuning. Finally, we also confirm the positive impact of TANDA in an\nindustrial setting, using domain specific datasets subject to different types\nof noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04123v2",
            "date": "2019-11-11 07:58:52+00:00",
            "title": "Leveraging Dependency Forest for Neural Medical Relation Extraction",
            "authors": [
                "Linfeng Song",
                "Yue Zhang",
                "Daniel Gildea",
                "Mo Yu",
                "Zhiguo Wang",
                "Jinsong Su"
            ],
            "abstract": "Medical relation extraction discovers relations between entity mentions in\ntext, such as research articles. For this task, dependency syntax has been\nrecognized as a crucial source of features. Yet in the medical domain, 1-best\nparse trees suffer from relatively low accuracies, diminishing their\nusefulness. We investigate a method to alleviate this problem by utilizing\ndependency forests. Forests contain many possible decisions and therefore have\nhigher recall but more noise compared with 1-best outputs. A graph neural\nnetwork is used to represent the forests, automatically distinguishing the\nuseful syntactic information from parsing noise. Results on two biomedical\nbenchmarks show that our method outperforms the standard tree-based methods,\ngiving the state-of-the-art results in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04128v2",
            "date": "2019-11-11 08:19:49+00:00",
            "title": "A hybrid text normalization system using multi-head self-attention for mandarin",
            "authors": [
                "Junhui Zhang",
                "Junjie Pan",
                "Xiang Yin",
                "Chen Li",
                "Shichao Liu",
                "Yang Zhang",
                "Yuxuan Wang",
                "Zejun Ma"
            ],
            "abstract": "In this paper, we propose a hybrid text normalization system using multi-head\nself-attention. The system combines the advantages of a rule-based model and a\nneural model for text preprocessing tasks. Previous studies in Mandarin text\nnormalization usually use a set of hand-written rules, which are hard to\nimprove on general cases. The idea of our proposed system is motivated by the\nneural models from recent studies and has a better performance on our internal\nnews corpus. This paper also includes different attempts to deal with\nimbalanced pattern distribution of the dataset. Overall, the performance of the\nsystem is improved by over 1.5% on sentence-level and it has a potential to\nimprove further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04211v4",
            "date": "2019-11-11 12:28:29+00:00",
            "title": "NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution",
            "authors": [
                "Aditya Khandelwal",
                "Suraj Sawant"
            ],
            "abstract": "Negation is an important characteristic of language, and a major component of\ninformation extraction from text. This subtask is of considerable importance to\nthe biomedical domain. Over the years, multiple approaches have been explored\nto address this problem: Rule-based systems, Machine Learning classifiers,\nConditional Random Field Models, CNNs and more recently BiLSTMs. In this paper,\nwe look at applying Transfer Learning to this problem. First, we extensively\nreview previous literature addressing Negation Detection and Scope Resolution\nacross the 3 datasets that have gained popularity over the years: the BioScope\nCorpus, the Sherlock dataset, and the SFU Review Corpus. We then explore the\ndecision choices involved with using BERT, a popular transfer learning model,\nfor this task, and report state-of-the-art results for scope resolution across\nall 3 datasets. Our model, referred to as NegBERT, achieves a token level F1\nscore on scope resolution of 92.36 on the Sherlock dataset, 95.68 on the\nBioScope Abstracts subcorpus, 91.24 on the BioScope Full Papers subcorpus,\n90.95 on the SFU Review Corpus, outperforming the previous state-of-the-art\nsystems by a significant margin. We also analyze the model's generalizability\nto datasets on which it is not trained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04361v1",
            "date": "2019-11-11 16:04:55+00:00",
            "title": "Attending to Entities for Better Text Understanding",
            "authors": [
                "Pengxiang Cheng",
                "Katrin Erk"
            ],
            "abstract": "Recent progress in NLP witnessed the development of large-scale pre-trained\nlanguage models (GPT, BERT, XLNet, etc.) based on Transformer (Vaswani et al.\n2017), and in a range of end tasks, such models have achieved state-of-the-art\nresults, approaching human performance. This demonstrates the power of the\nstacked self-attention architecture when paired with a sufficient number of\nlayers and a large amount of pre-training data. However, on tasks that require\ncomplex and long-distance reasoning where surface-level cues are not enough,\nthere is still a large gap between the pre-trained models and human\nperformance. Strubell et al. (2018) recently showed that it is possible to\ninject knowledge of syntactic structure into a model through supervised\nself-attention. We conjecture that a similar injection of semantic knowledge,\nin particular, coreference information, into an existing model would improve\nperformance on such complex problems. On the LAMBADA (Paperno et al. 2016)\ntask, we show that a model trained from scratch with coreference as auxiliary\nsupervision for self-attention outperforms the largest GPT-2 model, setting the\nnew state-of-the-art, while only containing a tiny fraction of parameters\ncompared to GPT-2. We also conduct a thorough analysis of different variants of\nmodel architectures and supervision configurations, suggesting future\ndirections on applying similar techniques to other problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04525v1",
            "date": "2019-11-11 19:16:01+00:00",
            "title": "Understanding BERT performance in propaganda analysis",
            "authors": [
                "Yiqing Hua"
            ],
            "abstract": "In this paper, we describe our system used in the shared task for\nfine-grained propaganda analysis at sentence level. Despite the challenging\nnature of the task, our pretrained BERT model (team YMJA) fine tuned on the\ntraining dataset provided by the shared task scored 0.62 F1 on the test set and\nranked third among 25 teams who participated in the contest. We present a set\nof illustrative experiments to better understand the performance of our BERT\nmodel on this shared task. Further, we explore beyond the given dataset for\nfalse-positive cases that likely to be produced by our system. We show that\ndespite the high performance on the given testset, our system may have the\ntendency of classifying opinion pieces as propaganda and cannot distinguish\nquotations of propaganda speech from actual usage of propaganda techniques.",
            "journal ref": "Proceedings of the Second Workshop on Natural Language Processing\n  for Internet Freedom: Censorship, Disinformation, and Propaganda (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04641v1",
            "date": "2019-11-12 02:49:38+00:00",
            "title": "A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling",
            "authors": [
                "Qingrong Xia",
                "Zhenghua Li",
                "Min Zhang"
            ],
            "abstract": "Semantic role labeling (SRL) aims to identify the predicate-argument\nstructure of a sentence. Inspired by the strong correlation between syntax and\nsemantics, previous works pay much attention to improve SRL performance on\nexploiting syntactic knowledge, achieving significant results. Pipeline methods\nbased on automatic syntactic trees and multi-task learning (MTL) approaches\nusing standard syntactic trees are two common research orientations. In this\npaper, we adopt a simple unified span-based model for both span-based and\nword-based Chinese SRL as a strong baseline. Besides, we present a MTL\nframework that includes the basic SRL module and a dependency parser module.\nDifferent from the commonly used hard parameter sharing strategy in MTL, the\nmain idea is to extract implicit syntactic representations from the dependency\nparser as external inputs for the basic SRL model. Experiments on the\nbenchmarks of Chinese Proposition Bank 1.0 and CoNLL-2009 Chinese datasets show\nthat our proposed framework can effectively improve the performance over the\nstrong baselines. With the external BERT representations, our framework\nachieves new state-of-the-art 87.54 and 88.5 F1 scores on the two test data of\nthe two benchmarks, respectively. In-depth analysis are conducted to gain more\ninsights on the proposed framework and the effectiveness of syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04916v2",
            "date": "2019-11-12 14:57:28+00:00",
            "title": "Morphological Segmentation Inside-Out",
            "authors": [
                "Ryan Cotterell",
                "Arun Kumar",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Morphological segmentation has traditionally been modeled with\nnon-hierarchical models, which yield flat segmentations as output. In many\ncases, however, proper morphological analysis requires hierarchical structure\n-- especially in the case of derivational morphology. In this work, we\nintroduce a discriminative, joint model of morphological segmentation along\nwith the orthographic changes that occur during word formation. To the best of\nour knowledge, this is the first attempt to approach discriminative\nsegmentation with a context-free model. Additionally, we release an annotated\ntreebank of 7454 English words with constituency parses, encouraging future\nresearch in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04944v2",
            "date": "2019-11-10 12:09:46+00:00",
            "title": "CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB",
            "authors": [
                "Holger Schwenk",
                "Guillaume Wenzek",
                "Sergey Edunov",
                "Edouard Grave",
                "Armand Joulin"
            ],
            "abstract": "We show that margin-based bitext mining in a multilingual sentence space can\nbe applied to monolingual corpora of billions of sentences. We are using ten\nsnapshots of a curated common crawl corpus (Wenzek et al., 2019) totalling 32.7\nbillion unique sentences. Using one unified approach for 38 languages, we were\nable to mine 4.5 billions parallel sentences, out of which 661 million are\naligned with English. 20 language pairs have more then 30 million parallel\nsentences, 112 more then 10 million, and most more than one million, including\ndirect alignments between many European or Asian languages.\n  To evaluate the quality of the mined bitexts, we train NMT systems for most\nof the language pairs and evaluate them on TED, WMT and WAT test sets. Using\nour mined bitexts only and no human translated parallel data, we achieve a new\nstate-of-the-art for a single system on the WMT'19 test set for translation\nbetween English and German, Russian and Chinese, as well as German/French. In\nparticular, our English/German system outperforms the best single one by close\nto 4 BLEU points and is almost on pair with best WMT'19 evaluation system which\nuses system combination and back-translation. We also achieve excellent results\nfor distant languages pairs like Russian/Japanese, outperforming the best\nsubmission at the 2019 workshop on Asian Translation (WAT).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04997v1",
            "date": "2019-11-12 16:32:38+00:00",
            "title": "Character-based NMT with Transformer",
            "authors": [
                "Rohit Gupta",
                "Laurent Besacier",
                "Marc Dymetman",
                "Matthias Gall\u00e9"
            ],
            "abstract": "Character-based translation has several appealing advantages, but its\nperformance is in general worse than a carefully tuned BPE baseline. In this\npaper we study the impact of character-based input and output with the\nTransformer architecture. In particular, our experiments on EN-DE show that\ncharacter-based Transformer models are more robust than their BPE counterpart,\nboth when translating noisy text, and when translating text from a different\ndomain. To obtain comparable BLEU scores in clean, in-domain data and close the\ngap with BPE-based models we use known techniques to train deeper Transformer\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05604v1",
            "date": "2019-11-13 16:43:22+00:00",
            "title": "Adapting and evaluating a deep learning language model for clinical why-question answering",
            "authors": [
                "Andrew Wen",
                "Mohamed Y. Elwazir",
                "Sungrim Moon",
                "Jungwei Fan"
            ],
            "abstract": "Objectives: To adapt and evaluate a deep learning language model for\nanswering why-questions based on patient-specific clinical text. Materials and\nMethods: Bidirectional encoder representations from transformers (BERT) models\nwere trained with varying data sources to perform SQuAD 2.0 style why-question\nanswering (why-QA) on clinical notes. The evaluation focused on: 1) comparing\nthe merits from different training data, 2) error analysis. Results: The best\nmodel achieved an accuracy of 0.707 (or 0.760 by partial match). Training\ntoward customization for the clinical language helped increase 6% in accuracy.\nDiscussion: The error analysis suggested that the model did not really perform\ndeep reasoning and that clinical why-QA might warrant more sophisticated\nsolutions. Conclusion: The BERT model achieved moderate accuracy in clinical\nwhy-QA and should benefit from the rapidly evolving technology. Despite the\nidentified limitations, it could serve as a competent proxy for question-driven\nclinical information extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05636v3",
            "date": "2019-11-13 17:12:40+00:00",
            "title": "Prevalence of code mixing in semi-formal patient communication in low resource languages of South Africa",
            "authors": [
                "Monika Obrocka",
                "Charles Copley",
                "Themba Gqaza",
                "Eli Grant"
            ],
            "abstract": "In this paper we address the problem of code-mixing in resource-poor language\nsettings. We examine data consisting of 182k unique questions generated by\nusers of the MomConnect helpdesk, part of a national scale public health\nplatform in South Africa. We show evidence of code-switching at the level of\napproximately 10% within this dataset -- a level that is likely to pose\nchallenges for future services. We use a natural language processing library\n(Polyglot) that supports detection of 196 languages and attempt to evaluate its\nperformance at identifying English, isiZulu and code-mixed questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05689v1",
            "date": "2019-11-13 18:13:07+00:00",
            "title": "Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text",
            "authors": [
                "Ian Porada",
                "Kaheer Suleman",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Modeling semantic plausibility requires commonsense knowledge about the world\nand has been used as a testbed for exploring various knowledge representations.\nPrevious work has focused specifically on modeling physical plausibility and\nshown that distributional methods fail when tested in a supervised setting. At\nthe same time, distributional models, namely large pretrained language models,\nhave led to improved results for many natural language understanding tasks. In\nthis work, we show that these pretrained language models are in fact effective\nat modeling physical plausibility in the supervised setting. We therefore\npresent the more difficult problem of learning to model physical plausibility\ndirectly from text. We create a training set by extracting attested events from\na large corpus, and we provide a baseline for training on these attested events\nin a self-supervised manner and testing on a physical plausibility task. We\nbelieve results could be further improved by injecting explicit commonsense\nknowledge into a distributional model.",
            "journal ref": "Proceedings of the First Workshop on Commonsense Inference in\n  Natural Language Processing. (2019) 123-129",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05715v1",
            "date": "2019-11-13 18:44:15+00:00",
            "title": "Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling",
            "authors": [
                "Timothee Mickus",
                "Denis Paperno",
                "Mathieu Constant"
            ],
            "abstract": "Defining words in a textual context is a useful task both for practical\npurposes and for gaining insight into distributed word representations.\nBuilding on the distributional hypothesis, we argue here that the most natural\nformalization of definition modeling is to treat it as a sequence-to-sequence\ntask, rather than a word-to-sequence task: given an input sequence with a\nhighlighted word, generate a contextually appropriate definition for it. We\nimplement this approach in a Transformer-based sequence-to-sequence model. Our\nproposal allows to train contextualization and definition generation in an\nend-to-end fashion, which is a conceptual improvement over earlier works. We\nachieve state-of-the-art results both in contextual and non-contextual\ndefinition modeling.",
            "journal ref": "Proceedings of the First NLPL Workshop on Deep Learning for\n  Natural Language Processing, 30 September, 2019, University of Turku, Turku,\n  Finland",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05758v2",
            "date": "2019-11-13 19:04:42+00:00",
            "title": "What do you mean, BERT? Assessing BERT as a Distributional Semantics Model",
            "authors": [
                "Timothee Mickus",
                "Denis Paperno",
                "Mathieu Constant",
                "Kees van Deemter"
            ],
            "abstract": "Contextualized word embeddings, i.e. vector representations for words in\ncontext, are naturally seen as an extension of previous noncontextual\ndistributional semantic models. In this work, we focus on BERT, a deep neural\nnetwork that produces contextualized embeddings and has set the\nstate-of-the-art in several semantic tasks, and study the semantic coherence of\nits embedding space. While showing a tendency towards coherence, BERT does not\nfully live up to the natural expectations for a semantic vector space. In\nparticular, we find that the position of the sentence in which a word occurs,\nwhile having no meaning correlates, leaves a noticeable trace on the word\nembeddings and disturbs similarity relationships.",
            "journal ref": "Proceedings of the Society for Computation in Linguistics: Vol. 3\n  (2020), Article 34",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05930v2",
            "date": "2019-11-14 04:18:55+00:00",
            "title": "FAQ-based Question Answering via Knowledge Anchors",
            "authors": [
                "Ruobing Xie",
                "Yanan Lu",
                "Fen Lin",
                "Leyu Lin"
            ],
            "abstract": "Question answering (QA) aims to understand questions and find appropriate\nanswers. In real-world QA systems, Frequently Asked Question (FAQ) based QA is\nusually a practical and effective solution, especially for some complicated\nquestions (e.g., How and Why). Recent years have witnessed the great successes\nof knowledge graphs (KGs) in KBQA systems, while there are still few works\nfocusing on making full use of KGs in FAQ-based QA. In this paper, we propose a\nnovel Knowledge Anchor based Question Answering (KAQA) framework for FAQ-based\nQA to better understand questions and retrieve more appropriate answers. More\nspecifically, KAQA mainly consists of three modules: knowledge graph\nconstruction, query anchoring and query-document matching. We consider entities\nand triples of KGs in texts as knowledge anchors to precisely capture the core\nsemantics, which brings in higher precision and better interpretability. The\nmulti-channel matching strategy also enables most sentence matching models to\nbe flexibly plugged in our KAQA framework to fit different real-world\ncomputation limitations. In experiments, we evaluate our models on both offline\nand online query-document matching tasks on a real-world FAQ-based QA system in\nWeChat Search, with detailed analysis, ablation tests and case studies. The\nsignificant improvements confirm the effectiveness and robustness of the KAQA\nframework in real-world FAQ-based QA.",
            "journal ref": "NLPCC-2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05960v1",
            "date": "2019-11-14 06:33:24+00:00",
            "title": "Contextual Recurrent Units for Cloze-style Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Wei-Nan Zhang",
                "Wanxiang Che",
                "Ting Liu",
                "Zhipeng Chen",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Recurrent Neural Networks (RNN) are known as powerful models for handling\nsequential data, and especially widely utilized in various natural language\nprocessing tasks. In this paper, we propose Contextual Recurrent Units (CRU)\nfor enhancing local contextual representations in neural networks. The proposed\nCRU injects convolutional neural networks (CNN) into the recurrent units to\nenhance the ability to model the local context and reducing word ambiguities\neven in bi-directional RNNs. We tested our CRU model on sentence-level and\ndocument-level modeling NLP tasks: sentiment classification and reading\ncomprehension. Experimental results show that the proposed CRU model could give\nsignificant improvements over traditional CNN or RNN models, including\nbidirectional conditions, as well as various state-of-the-art systems on both\ntasks, showing its promising future of extensibility to other NLP tasks as\nwell.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06003v2",
            "date": "2019-11-14 09:09:10+00:00",
            "title": "Training a code-switching language model with monolingual data",
            "authors": [
                "Shun-Po Chuang",
                "Tzu-Wei Sung",
                "Hung-Yi Lee"
            ],
            "abstract": "A lack of code-switching data complicates the training of code-switching (CS)\nlanguage models. We propose an approach to train such CS language models on\nmonolingual data only. By constraining and normalizing the output projection\nmatrix in RNN-based language models, we bring embeddings of different languages\ncloser to each other. Numerical and visualization results show that the\nproposed approaches remarkably improve the performance of CS language models\ntrained on monolingual data. The proposed approaches are comparable or even\nbetter than training CS language models with artificially generated CS data. We\nadditionally use unsupervised bilingual word translation to analyze whether\nsemantically equivalent words in different languages are mapped together.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06121v1",
            "date": "2019-11-13 15:51:23+00:00",
            "title": "Towards Supervised Extractive Text Summarization via RNN-based Sequence Classification",
            "authors": [
                "Eduardo Brito",
                "Max L\u00fcbbering",
                "David Biesner",
                "Lars Patrick Hillebrand",
                "Christian Bauckhage"
            ],
            "abstract": "This article briefly explains our submitted approach to the DocEng'19\ncompetition on extractive summarization. We implemented a recurrent neural\nnetwork based model that learns to classify whether an article's sentence\nbelongs to the corresponding extractive summary or not. We bypass the lack of\nlarge annotated news corpora for extractive summarization by generating\nextractive summaries from abstractive ones, which are available from the CNN\ncorpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06136v3",
            "date": "2019-11-13 05:21:45+00:00",
            "title": "KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation",
            "authors": [
                "Xiaozhi Wang",
                "Tianyu Gao",
                "Zhaocheng Zhu",
                "Zhengyan Zhang",
                "Zhiyuan Liu",
                "Juanzi Li",
                "Jian Tang"
            ],
            "abstract": "Pre-trained language representation models (PLMs) cannot well capture factual\nknowledge from text. In contrast, knowledge embedding (KE) methods can\neffectively represent the relational facts in knowledge graphs (KGs) with\ninformative entity embeddings, but conventional KE models cannot take full\nadvantage of the abundant textual information. In this paper, we propose a\nunified model for Knowledge Embedding and Pre-trained LanguagE Representation\n(KEPLER), which can not only better integrate factual knowledge into PLMs but\nalso produce effective text-enhanced KE with the strong PLMs. In KEPLER, we\nencode textual entity descriptions with a PLM as their embeddings, and then\njointly optimize the KE and language modeling objectives. Experimental results\nshow that KEPLER achieves state-of-the-art performances on various NLP tasks,\nand also works remarkably well as an inductive KE model on KG link prediction.\nFurthermore, for pre-training and evaluating KEPLER, we construct Wikidata5M, a\nlarge-scale KG dataset with aligned entity descriptions, and benchmark\nstate-of-the-art KE methods on it. It shall serve as a new KE benchmark and\nfacilitate the research on large KG, inductive KE, and KG with text. The source\ncode can be obtained from https://github.com/THU-KEG/KEPLER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06146v2",
            "date": "2019-11-11 23:10:15+00:00",
            "title": "Biomedical Evidence Generation Engine",
            "authors": [
                "Sendong Zhao",
                "Fei Wang"
            ],
            "abstract": "With the rapid development of precision medicine, a large amount of health\ndata (such as electronic health records, gene sequencing, medical images, etc.)\nhas been produced. It encourages more and more interest in data-driven insight\ndiscovery from these data. It is a reasonable way to verify the derived\ninsights in biomedical literature. However, manual verification is inefficient\nand not scalable. Therefore, an intelligent technique is necessary to solve\nthis problem. In this paper, we propose a task of biomedical evidence\ngeneration, which is very novel and different from existing NLP tasks.\nFurthermore, we developed a biomedical evidence generation engine for this task\nwith the pipeline of three components which are a literature retrieval module,\na skeleton information identification module, and a text summarization module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06161v2",
            "date": "2019-11-14 15:01:15+00:00",
            "title": "Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources",
            "authors": [
                "Qianhui Wu",
                "Zijia Lin",
                "Guoxin Wang",
                "Hui Chen",
                "B\u00f6rje F. Karlsson",
                "Biqing Huang",
                "Chin-Yew Lin"
            ],
            "abstract": "For languages with no annotated resources, transferring knowledge from\nrich-resource languages is an effective solution for named entity recognition\n(NER). While all existing methods directly transfer from source-learned model\nto a target language, in this paper, we propose to fine-tune the learned model\nwith a few similar examples given a test case, which could benefit the\nprediction by leveraging the structural and semantic information conveyed in\nsuch similar examples. To this end, we present a meta-learning algorithm to\nfind a good model parameter initialization that could fast adapt to the given\ntest case and propose to construct multiple pseudo-NER tasks for meta-training\nby computing sentence similarities. To further improve the model's\ngeneralization ability across different languages, we introduce a masking\nscheme and augment the loss function with an additional maximum term during\nmeta-training. We conduct extensive experiments on cross-lingual named entity\nrecognition with minimal resources over five target languages. The results show\nthat our approach significantly outperforms existing state-of-the-art methods\nacross the board.",
            "journal ref": "In AAAI, pages 9274-9281, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06171v1",
            "date": "2019-11-13 16:09:59+00:00",
            "title": "Unsupervised Pre-training for Natural Language Generation: A Literature Review",
            "authors": [
                "Yuanxin Liu",
                "Zheng Lin"
            ],
            "abstract": "Recently, unsupervised pre-training is gaining increasing popularity in the\nrealm of computational linguistics, thanks to its surprising success in\nadvancing natural language understanding (NLU) and the potential to effectively\nexploit large-scale unlabelled corpus. However, regardless of the success in\nNLU, the power of unsupervised pre-training is only partially excavated when it\ncomes to natural language generation (NLG). The major obstacle stems from an\nidiosyncratic nature of NLG: Texts are usually generated based on certain\ncontext, which may vary with the target applications. As a result, it is\nintractable to design a universal architecture for pre-training as in NLU\nscenarios. Moreover, retaining the knowledge learned from pre-training when\nlearning on the target task is also a non-trivial problem. This review\nsummarizes the recent efforts to enhance NLG systems with unsupervised\npre-training, with a special focus on the methods to catalyse the integration\nof pre-trained models into downstream tasks. They are classified into\narchitecture-based methods and strategy-based methods, based on their way of\nhandling the above obstacle. Discussions are also provided to give further\ninsights into the relationship between these two lines of work, some\ninformative empirical phenomenons, as well as some possible directions where\nfuture work can be devoted to.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06172v1",
            "date": "2019-11-13 14:42:55+00:00",
            "title": "Word-level Lexical Normalisation using Context-Dependent Embeddings",
            "authors": [
                "Michael Stewart",
                "Wei Liu",
                "Rachel Cardell-Oliver"
            ],
            "abstract": "Lexical normalisation (LN) is the process of correcting each word in a\ndataset to its canonical form so that it may be more easily and more accurately\nanalysed. Most lexical normalisation systems operate at the character-level,\nwhile word-level models are seldom used. Recent language models offer solutions\nto the drawbacks of word-level LN models, yet, to the best of our knowledge, no\nresearch has investigated their effectiveness on LN. In this paper we introduce\na word-level GRU-based LN model and investigate the effectiveness of recent\nembedding techniques on word-level LN. Our results show that our GRU-based\nword-level model produces greater results than character-level models, and\noutperforms existing deep-learning based LN techniques on Twitter data. We also\nfind that randomly-initialised embeddings are capable of outperforming\npre-trained embedding models in certain scenarios. Finally, we release a\nsubstantial lexical normalisation dataset to the community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06394v1",
            "date": "2019-11-14 21:42:48+00:00",
            "title": "The Eighth Dialog System Technology Challenge",
            "authors": [
                "Seokhwan Kim",
                "Michel Galley",
                "Chulaka Gunasekara",
                "Sungjin Lee",
                "Adam Atkinson",
                "Baolin Peng",
                "Hannes Schulz",
                "Jianfeng Gao",
                "Jinchao Li",
                "Mahmoud Adada",
                "Minlie Huang",
                "Luis Lastras",
                "Jonathan K. Kummerfeld",
                "Walter S. Lasecki",
                "Chiori Hori",
                "Anoop Cherian",
                "Tim K. Marks",
                "Abhinav Rastogi",
                "Xiaoxue Zang",
                "Srinivas Sunkara",
                "Raghav Gupta"
            ],
            "abstract": "This paper introduces the Eighth Dialog System Technology Challenge. In line\nwith recent challenges, the eighth edition focuses on applying end-to-end\ndialog technologies in a pragmatic way for multi-domain task-completion, noetic\nresponse selection, audio visual scene-aware dialog, and schema-guided dialog\nstate tracking tasks. This paper describes the task definition, provided\ndatasets, and evaluation set-up for each track. We also summarize the results\nof the submitted systems to highlight the overall trends of the\nstate-of-the-art technologies for the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06488v1",
            "date": "2019-11-15 06:30:25+00:00",
            "title": "Using natural language processing to extract health-related causality from Twitter messages",
            "authors": [
                "Son Doan",
                "Elly W Yang",
                "Sameer Tilak",
                "Manabu Torii"
            ],
            "abstract": "Twitter messages (tweets) contain various types of information, which include\nhealth-related information. Analysis of health-related tweets would help us\nunderstand health conditions and concerns encountered in our daily life. In\nthis work, we evaluated an approach to extracting causal relations from tweets\nusing natural language processing (NLP) techniques. We focused on three\nhealth-related topics: stress\", \"insomnia\", and \"headache\". We proposed a set\nof lexico-syntactic patterns based on dependency parser outputs to extract\ncausal information. A large dataset consisting of 24 million tweets were used.\nThe results show that our approach achieved an average precision between 74.59%\nand 92.27%. Analysis of extracted relations revealed interesting findings about\nhealth-related in Twitter.",
            "journal ref": "2018 IEEE International Conference on Healthcare Informatics\n  Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06673v1",
            "date": "2019-11-15 14:47:02+00:00",
            "title": "Bootstrapping NLU Models with Multi-task Learning",
            "authors": [
                "Shubham Kapoor",
                "Caglar Tirkaz"
            ],
            "abstract": "Bootstrapping natural language understanding (NLU) systems with minimal\ntraining data is a fundamental challenge of extending digital assistants like\nAlexa and Siri to a new language. A common approach that is adapted in digital\nassistants when responding to a user query is to process the input in a\npipeline manner where the first task is to predict the domain, followed by the\ninference of intent and slots. However, this cascaded approach instigates error\npropagation and prevents information sharing among these tasks. Further, the\nuse of words as the atomic units of meaning as done in many studies might lead\nto coverage problems for morphologically rich languages such as German and\nFrench when data is limited. We address these issues by introducing a\ncharacter-level unified neural architecture for joint modeling of the domain,\nintent, and slot classification. We compose word-embeddings from characters and\njointly optimize all classification tasks via multi-task learning. In our\nresults, we show that the proposed architecture is an optimal choice for\nbootstrapping NLU systems in low-resource settings thus saving time, cost and\nhuman effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06915v1",
            "date": "2019-11-15 23:37:41+00:00",
            "title": "Evaluating robustness of language models for chief complaint extraction from patient-generated text",
            "authors": [
                "Ilya Valmianski",
                "Caleb Goodwin",
                "Ian M. Finn",
                "Naqi Khan",
                "Daniel S. Zisook"
            ],
            "abstract": "Automated classification of chief complaints from patient-generated text is a\ncritical first step in developing scalable platforms to triage patients without\nhuman intervention. In this work, we evaluate several approaches to chief\ncomplaint classification using a novel Chief Complaint (CC) Dataset that\ncontains ~200,000 patient-generated reasons-for-visit entries mapped to a set\nof 795 discrete chief complaints. We examine the use of several fine-tuned\nbidirectional transformer (BERT) models trained on both unrelated texts as well\nas on the CC dataset. We contrast this performance with a TF-IDF baseline. Our\nevaluation has three components: (1) a random test hold-out from the original\ndataset; (2) a \"misspelling set,\" consisting of a hand-selected subset of the\ntest set, where every entry has at least one misspelling; (3) a separate\nexperimenter-generated free-text set. We find that the TF-IDF model performs\nsignificantly better than the strongest BERT-based model on the test (best BERT\nPR-AUC $0.3597 \\pm 0.0041$ vs TF-IDF PR-AUC $0.3878 \\pm 0.0148$, $p=7\\cdot\n10^{-5}$), and is statistically comparable to the misspelling sets (best BERT\nPR-AUC $0.2579 \\pm 0.0079$ vs TF-IDF PR-AUC $0.2733 \\pm 0.0130$, $p=0.06$).\nHowever, when examining model predictions on experimenter-generated queries,\nsome concerns arise about TF-IDF baseline's robustness. Our results suggest\nthat in certain tasks, simple language embedding baselines may be very\nperformant; however, truly understanding their robustness requires further\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06940v1",
            "date": "2019-11-16 02:47:03+00:00",
            "title": "Utterance-to-Utterance Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots",
            "authors": [
                "Jia-Chen Gu",
                "Zhen-Hua Ling",
                "Quan Liu"
            ],
            "abstract": "This paper proposes an utterance-to-utterance interactive matching network\n(U2U-IMN) for multi-turn response selection in retrieval-based chatbots.\nDifferent from previous methods following context-to-response matching or\nutterance-to-response matching frameworks, this model treats both contexts and\nresponses as sequences of utterances when calculating the matching degrees\nbetween them. For a context-response pair, the U2U-IMN model first encodes each\nutterance separately using recurrent and self-attention layers. Then, a global\nand bidirectional interaction between the context and the response is conducted\nusing the attention mechanism to collect the matching information between them.\nThe distances between context and response utterances are employed as a prior\ncomponent when calculating the attention weights. Finally, sentence-level\naggregation and context-response-level aggregation are executed in turn to\nobtain the feature vector for matching degree prediction. Experiments on four\npublic datasets showed that our proposed method outperformed baseline methods\non all metrics, achieving a new state-of-the-art performance and demonstrating\ncompatibility across domains for multi-turn response selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06948v1",
            "date": "2019-11-16 03:41:40+00:00",
            "title": "Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization",
            "authors": [
                "Mantong Zhou",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "In spite of great advancements of machine reading comprehension (RC),\nexisting RC models are still vulnerable and not robust to different types of\nadversarial examples. Neural models over-confidently predict wrong answers to\nsemantic different adversarial examples, while over-sensitively predict wrong\nanswers to semantic equivalent adversarial examples. Existing methods which\nimprove the robustness of such neural models merely mitigate one of the two\nissues but ignore the other. In this paper, we address the over-confidence\nissue and the over-sensitivity issue existing in current RC models\nsimultaneously with the help of external linguistic knowledge. We first\nincorporate external knowledge to impose different linguistic constraints\n(entity constraint, lexical constraint, and predicate constraint), and then\nregularize RC models through posterior regularization. Linguistic constraints\ninduce more reasonable predictions for both semantic different and semantic\nequivalent adversarial examples, and posterior regularization provides an\neffective mechanism to incorporate these constraints. Our method can be applied\nto any existing neural RC models including state-of-the-art BERT models.\nExtensive experiments show that our method remarkably improves the robustness\nof base RC models, and is better to cope with these two issues simultaneously.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07030v1",
            "date": "2019-11-16 13:34:16+00:00",
            "title": "Contribution au Niveau de l'Approche Indirecte \u00e0 Base de Transfert dans la Traduction Automatique",
            "authors": [
                "Sadik Bessou"
            ],
            "abstract": "In this thesis, we address several important issues concerning the\nmorphological analysis of Arabic language applied to textual data and machine\ntranslation. First, we provided an overview on machine translation, its history\nand its development, then we exposed human translation techniques for eventual\ninspiration in machine translation, and we exposed linguistic approaches and\nparticularly indirect transfer approaches. Finally, we presented our\ncontributions to the resolution of morphosyntactic problems in computer\nlinguistics as multilingual information retrieval and machine translation. As a\nfirst contribution, we developed a morphological analyzer for Arabic, and we\nhave exploited it in the bilingual information retrieval such as a computer\napplication of multilingual documentary. Results validation showed a\nstatistically significant performance. In a second contribution, we proposed a\nlist of morphosyntactic transfer rules from English to Arabic for translation\nin three phases: analysis, transfer, generation. We focused on the transfer\nphase without semantic distortion for an abstraction of English in a sufficient\nsubset of Arabic.",
            "journal ref": "Phd Thesis, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07176v2",
            "date": "2019-11-17 07:51:42+00:00",
            "title": "Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering",
            "authors": [
                "Vikas Yadav",
                "Steven Bethard",
                "Mihai Surdeanu"
            ],
            "abstract": "We propose an unsupervised strategy for the selection of justification\nsentences for multi-hop question answering (QA) that (a) maximizes the\nrelevance of the selected sentences, (b) minimizes the overlap between the\nselected facts, and (c) maximizes the coverage of both question and answer.\nThis unsupervised sentence selection method can be coupled with any supervised\nQA approach. We show that the sentences selected by our method improve the\nperformance of a state-of-the-art supervised QA model on two multi-hop QA\ndatasets: AI2's Reasoning Challenge (ARC) and Multi-Sentence Reading\nComprehension (MultiRC). We obtain new state-of-the-art performance on both\ndatasets among approaches that do not use external resources for training the\nQA system: 56.82% F1 on ARC (41.24% on Challenge and 64.49% on Easy) and 26.1%\nEM0 on MultiRC. Our justification sentences have higher quality than the\njustifications selected by a strong information retrieval baseline, e.g., by\n5.4% F1 in MultiRC. We also show that our unsupervised selection of\njustification sentences is more stable across domains than a state-of-the-art\nsupervised sentence selection method.",
            "journal ref": "EMNLP-IJCNLP, 2578--2589 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07184v1",
            "date": "2019-11-17 08:27:26+00:00",
            "title": "Multi-Zone Unit for Recurrent Neural Networks",
            "authors": [
                "Fandong Meng",
                "Jinchao Zhang",
                "Yang Liu",
                "Jie Zhou"
            ],
            "abstract": "Recurrent neural networks (RNNs) have been widely used to deal with sequence\nlearning problems. The input-dependent transition function, which folds new\nobservations into hidden states to sequentially construct fixed-length\nrepresentations of arbitrary-length sequences, plays a critical role in RNNs.\nBased on single space composition, transition functions in existing RNNs often\nhave difficulty in capturing complicated long-range dependencies. In this\npaper, we introduce a new Multi-zone Unit (MZU) for RNNs. The key idea is to\ndesign a transition function that is capable of modeling multiple space\ncomposition. The MZU consists of three components: zone generation, zone\ncomposition, and zone aggregation. Experimental results on multiple datasets of\nthe character-level language modeling task and the aspect-based sentiment\nanalysis task demonstrate the superiority of the MZU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07474v2",
            "date": "2019-11-18 07:49:13+00:00",
            "title": "Deep and Dense Sarcasm Detection",
            "authors": [
                "Devin Pelser",
                "Hugh Murrell"
            ],
            "abstract": "Recent work in automated sarcasm detection has placed a heavy focus on\ncontext and meta-data. Whilst certain utterances indeed require background\nknowledge and commonsense reasoning, previous works have only explored shallow\nmodels for capturing the lexical, syntactic and semantic cues present within a\ntext. In this paper, we propose a deep 56 layer network, implemented with dense\nconnectivity to model the isolated utterance and extract richer features\ntherein. We compare our approach against recent state-of-the-art architectures\nwhich make considerable use of extrinsic information, and demonstrate\ncompetitive results whilst using only the local features of the text. Further,\nwe provide an analysis of the dependency of prior convolution outputs in\ngenerating the final feature maps. Finally a case study is presented,\nsupporting that our approach accurately classifies additional uses of clear\nsarcasm, which a standard CNN misclassifies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07555v2",
            "date": "2019-11-18 11:34:38+00:00",
            "title": "Short Text Language Identification for Under Resourced Languages",
            "authors": [
                "Bernardt Duvenhage"
            ],
            "abstract": "The paper presents a hierarchical naive Bayesian and lexicon based classifier\nfor short text language identification (LID) useful for under resourced\nlanguages. The algorithm is evaluated on short pieces of text for the 11\nofficial South African languages some of which are similar languages. The\nalgorithm is compared to recent approaches using test sets from previous works\non South African languages as well as the Discriminating between Similar\nLanguages (DSL) shared tasks' datasets. Remaining research opportunities and\npressing concerns in evaluating and comparing LID approaches are also\ndiscussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08212v1",
            "date": "2019-11-19 11:41:02+00:00",
            "title": "Deep Poetry: A Chinese Classical Poetry Generation System",
            "authors": [
                "Yusen Liu",
                "Dayiheng Liu",
                "Jiancheng Lv"
            ],
            "abstract": "In this work, we demonstrate a Chinese classical poetry generation system\ncalled Deep Poetry. Existing systems for Chinese classical poetry generation\nare mostly template-based and very few of them can accept multi-modal input.\nUnlike previous systems, Deep Poetry uses neural networks that are trained on\nover 200 thousand poems and 3 million ancient Chinese prose. Our system can\naccept plain text, images or artistic conceptions as inputs to generate Chinese\nclassical poetry. More importantly, users are allowed to participate in the\nprocess of writing poetry by our system. For the user's convenience, we deploy\nthe system at the WeChat applet platform, users can use the system on the\nmobile device whenever and wherever possible. The demo video of this paper is\navailable at https://youtu.be/jD1R_u9TA3M.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08249v1",
            "date": "2019-11-15 20:17:06+00:00",
            "title": "An Accuracy-Enhanced Stemming Algorithm for Arabic Information Retrieval",
            "authors": [
                "Sadik Bessou",
                "Mohamed Touahria"
            ],
            "abstract": "This paper provides a method for indexing and retrieving Arabic texts, based\non natural language processing. Our approach exploits the notion of template in\nword stemming and replaces the words by their stems. This technique has proven\nto be effective since it has returned significant relevant retrieval results by\ndecreasing silence during the retrieval phase. Series of experiments have been\nconducted to test the performance of the proposed algorithm ESAIR (Enhanced\nStemmer for Arabic Information Retrieval). The results obtained indicate that\nthe algorithm extracts the exact root with an accuracy rate up to 96% and\nhence, improving information retrieval.",
            "journal ref": "Neural Network World, 24(2), p.117-128. 2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08648v1",
            "date": "2019-11-20 00:48:36+00:00",
            "title": "Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension",
            "authors": [
                "Xiaorui Zhou",
                "Senlin Luo",
                "Yunfang Wu"
            ],
            "abstract": "In reading comprehension, generating sentence-level distractors is a\nsignificant task, which requires a deep understanding of the article and\nquestion. The traditional entity-centered methods can only generate word-level\nor phrase-level distractors. Although recently proposed neural-based methods\nlike sequence-to-sequence (Seq2Seq) model show great potential in generating\ncreative text, the previous neural methods for distractor generation ignore two\nimportant aspects. First, they didn't model the interactions between the\narticle and question, making the generated distractors tend to be too general\nor not relevant to question context. Second, they didn't emphasize the\nrelationship between the distractor and article, making the generated\ndistractors not semantically relevant to the article and thus fail to form a\nset of meaningful options. To solve the first problem, we propose a\nco-attention enhanced hierarchical architecture to better capture the\ninteractions between the article and question, thus guide the decoder to\ngenerate more coherent distractors. To alleviate the second problem, we add an\nadditional semantic similarity loss to push the generated distractors more\nrelevant to the article. Experimental results show that our model outperforms\nseveral strong baselines on automatic metrics, achieving state-of-the-art\nperformance. Further human evaluation indicates that our generated distractors\nare more coherent and more educative compared with those distractors generated\nby baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08673v3",
            "date": "2019-11-20 02:57:53+00:00",
            "title": "Global Greedy Dependency Parsing",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Kevin Parnow"
            ],
            "abstract": "Most syntactic dependency parsing models may fall into one of two categories:\ntransition- and graph-based models. The former models enjoy high inference\nefficiency with linear time complexity, but they rely on the stacking or\nre-ranking of partially-built parse trees to build a complete parse tree and\nare stuck with slower training for the necessity of dynamic oracle training.\nThe latter, graph-based models, may boast better performance but are\nunfortunately marred by polynomial time inference. In this paper, we propose a\nnovel parsing order objective, resulting in a novel dependency parsing model\ncapable of both global (in sentence scope) feature extraction as in graph\nmodels and linear time inference as in transitional models. The proposed global\ngreedy parser only uses two arc-building actions, left and right arcs, for\nprojective parsing. When equipped with two extra non-projective arc-building\nactions, the proposed parser may also smoothly support non-projective parsing.\nUsing multiple benchmark treebanks, including the Penn Treebank (PTB), the\nCoNLL-X treebanks, and the Universal Dependency Treebanks, we evaluate our\nparser and demonstrate that the proposed novel parser achieves good performance\nwith faster training and decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08698v2",
            "date": "2019-11-20 04:25:20+00:00",
            "title": "EmpDG: Multiresolution Interactive Empathetic Dialogue Generation",
            "authors": [
                "Qintong Li",
                "Hongshen Chen",
                "Zhaochun Ren",
                "Pengjie Ren",
                "Zhaopeng Tu",
                "Zhumin Chen"
            ],
            "abstract": "A humanized dialogue system is expected to generate empathetic replies, which\nshould be sensitive to the users' expressed emotion. The task of empathetic\ndialogue generation is proposed to address this problem. The essential\nchallenges lie in accurately capturing the nuances of human emotion and\nconsidering the potential of user feedback, which are overlooked by the\nmajority of existing work. In response to this problem, we propose a\nmulti-resolution adversarial model -- EmpDG, to generate more empathetic\nresponses. EmpDG exploits both the coarse-grained dialogue-level and\nfine-grained token-level emotions, the latter of which helps to better capture\nthe nuances of user emotion. In addition, we introduce an interactive\nadversarial learning framework which exploits the user feedback, to identify\nwhether the generated responses evoke emotion perceptivity in dialogues.\nExperimental results show that the proposed approach significantly outperforms\nthe state-of-the-art baselines in both content quality and emotion\nperceptivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08706v2",
            "date": "2019-11-20 04:54:21+00:00",
            "title": "Controlling Neural Machine Translation Formality with Synthetic Supervision",
            "authors": [
                "Xing Niu",
                "Marine Carpuat"
            ],
            "abstract": "This work aims to produce translations that convey source language content at\na formality level that is appropriate for a particular audience. Framing this\nproblem as a neural sequence-to-sequence task ideally requires training\ntriplets consisting of a bilingual sentence pair labeled with target language\nformality. However, in practice, available training examples are limited to\nEnglish sentence pairs of different styles, and bilingual parallel sentences of\nunknown formality. We introduce a novel training scheme for multi-task models\nthat automatically generates synthetic training triplets by inferring the\nmissing element on the fly, thus enabling end-to-end training. Comprehensive\nautomatic and human assessments show that our best model outperforms existing\nmodels by producing translations that better match desired formality levels\nwhile preserving the source meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08782v1",
            "date": "2019-11-20 09:24:38+00:00",
            "title": "Joint Emotion Label Space Modelling for Affect Lexica",
            "authors": [
                "Luna De Bruyne",
                "Pepa Atanasova",
                "Isabelle Augenstein"
            ],
            "abstract": "Emotion lexica are commonly used resources to combat data poverty in\nautomatic emotion detection. However, methodological issues emerge when\nemploying them: lexica are often not very extensive, and the way they are\nconstructed can vary widely -- from lab conditions to crowdsourced approaches\nand distant supervision. Furthermore, both categorical frameworks and\ndimensional frameworks coexist, in which theorists provide many different sets\nof categorical labels or dimensional axes. The heterogenous nature of the\nresulting emotion detection resources results in a need for a unified approach\nto utilising them. This paper contributes to the field of emotion analysis in\nNLP by a) presenting the first study to unify existing emotion detection\nresources automatically and thus learn more about the relationships between\nthem; b) exploring the use of existing lexica for the above-mentioned task; c)\npresenting an approach to automatically combining emotion lexica, namely by a\nmulti-view variational auto-encoder (VAE), which facilitates the mapping of\ndatasets into a joint emotion label space. We test the utility of joint emotion\nlexica by using them as additional features in state-of-the art emotion\ndetection models. Our overall findings are that emotion lexica can offer\ncomplementary information to even extremely large pre-trained models such as\nBERT. The performance of our models is comparable to state-of-the art models\nthat are specifically engineered for certain datasets, and even outperform the\nstate-of-the art on four datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08794v1",
            "date": "2019-11-20 09:52:23+00:00",
            "title": "Natural Language Generation Challenges for Explainable AI",
            "authors": [
                "Ehud Reiter"
            ],
            "abstract": "Good quality explanations of artificial intelligence (XAI) reasoning must be\nwritten (and evaluated) for an explanatory purpose, targeted towards their\nreaders, have a good narrative and causal structure, and highlight where\nuncertainty and data quality affect the AI output. I discuss these challenges\nfrom a Natural Language Generation (NLG) perspective, and highlight four\nspecific NLG for XAI research challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08829v1",
            "date": "2019-11-20 11:15:47+00:00",
            "title": "Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions",
            "authors": [
                "Hessel Haagsma",
                "Malvina Nissim",
                "Johan Bos"
            ],
            "abstract": "Idiomatic expressions like `out of the woods' and `up the ante' present a\nrange of difficulties for natural language processing applications. We present\nwork on the annotation and extraction of what we term potentially idiomatic\nexpressions (PIEs), a subclass of multiword expressions covering both literal\nand non-literal uses of idiomatic expressions. Existing corpora of PIEs are\nsmall and have limited coverage of different PIE types, which hampers research.\nTo further progress on the extraction and disambiguation of potentially\nidiomatic expressions, larger corpora of PIEs are required. In addition, larger\ncorpora are a potential source for valuable linguistic insights into idiomatic\nexpressions and their variability. We propose automatic tools to facilitate the\nbuilding of larger PIE corpora, by investigating the feasibility of using\ndictionary-based extraction of PIEs as a pre-extraction tool for English. We do\nthis by assessing the reliability and coverage of idiom dictionaries, the\nannotation of a PIE corpus, and the automatic extraction of PIEs from a large\ncorpus. Results show that combinations of dictionaries are a reliable source of\nidiomatic expressions, that PIEs can be annotated with a high reliability\n(0.74-0.91 Fleiss' Kappa), and that parse-based PIE extraction yields highly\naccurate performance (88% F1-score). Combining complementary PIE extraction\nmethods increases reliability further, to over 92% F1-score. Moreover, the\nextraction method presented here could be extended to other types of multiword\nexpressions and to other languages, given that sufficient NLP tools are\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08836v1",
            "date": "2019-11-20 11:29:47+00:00",
            "title": "Table-Of-Contents generation on contemporary documents",
            "authors": [
                "Najah-Imane Bentabet",
                "R\u00e9mi Juge",
                "Sira Ferradans"
            ],
            "abstract": "The generation of precise and detailed Table-Of-Contents (TOC) from a\ndocument is a problem of major importance for document understanding and\ninformation extraction. Despite its importance, it is still a challenging task,\nespecially for non-standardized documents with rich layout information such as\ncommercial documents. In this paper, we present a new neural-based pipeline for\nTOC generation applicable to any searchable document. Unlike previous methods,\nwe do not use semantic labeling nor assume the presence of parsable TOC pages\nin the document. Moreover, we analyze the influence of using external knowledge\nencoded as a template. We empirically show that this approach is only useful in\na very low resource environment. Finally, we propose a new domain-specific data\nset that sheds some light on the difficulties of TOC generation in real-world\ndocuments. The proposed method shows better performance than the\nstate-of-the-art on a public data set and on the newly released data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08962v3",
            "date": "2019-11-20 15:23:59+00:00",
            "title": "CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain",
            "authors": [
                "Chaojun Xiao",
                "Haoxi Zhong",
                "Zhipeng Guo",
                "Cunchao Tu",
                "Zhiyuan Liu",
                "Maosong Sun",
                "Tianyang Zhang",
                "Xianpei Han",
                "Zhen Hu",
                "Heng Wang",
                "Jianfeng Xu"
            ],
            "abstract": "In this paper, we introduce CAIL2019-SCM, Chinese AI and Law 2019 Similar\nCase Matching dataset. CAIL2019-SCM contains 8,964 triplets of cases published\nby the Supreme People's Court of China. CAIL2019-SCM focuses on detecting\nsimilar cases, and the participants are required to check which two cases are\nmore similar in the triplets. There are 711 teams who participated in this\nyear's competition, and the best team has reached a score of 71.88. We have\nalso implemented several baselines to help researchers better understand this\ntask. The dataset and more details can be found from\nhttps://github.com/china-ai-law-challenge/CAIL2019/tree/master/scm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09075v1",
            "date": "2019-11-20 18:27:22+00:00",
            "title": "Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Network",
            "authors": [
                "Wenxiang Jiao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Real-time emotion recognition (RTER) in conversations is significant for\ndeveloping emotionally intelligent chatting machines. Without the future\ncontext in RTER, it becomes critical to build the memory bank carefully for\ncapturing historical context and summarize the memories appropriately to\nretrieve relevant information. We propose an Attention Gated Hierarchical\nMemory Network (AGHMN) to address the problems of prior work: (1) Commonly used\nconvolutional neural networks (CNNs) for utterance feature extraction are less\ncompatible in the memory modules; (2) Unidirectional gated recurrent units\n(GRUs) only allow each historical utterance to have context before it,\npreventing information propagation in the opposite direction; (3) The Soft\nAttention for summarizing loses the positional and ordering information of\nmemories, regardless of how the memory bank is built. Particularly, we propose\na Hierarchical Memory Network (HMN) with a bidirectional GRU (BiGRU) as the\nutterance reader and a BiGRU fusion layer for the interaction between\nhistorical utterances. For memory summarizing, we propose an Attention GRU\n(AGRU) where we utilize the attention weights to update the internal state of\nGRU. We further promote the AGRU to a bidirectional variant (BiAGRU) to balance\nthe contextual information from recent memories and that from distant memories.\nWe conduct experiments on two emotion conversation datasets with extensive\nanalysis, demonstrating the efficacy of our AGHMN models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09241v1",
            "date": "2019-11-21 02:04:52+00:00",
            "title": "Assessing the Benchmarking Capacity of Machine Reading Comprehension Datasets",
            "authors": [
                "Saku Sugawara",
                "Pontus Stenetorp",
                "Kentaro Inui",
                "Akiko Aizawa"
            ],
            "abstract": "Existing analysis work in machine reading comprehension (MRC) is largely\nconcerned with evaluating the capabilities of systems. However, the\ncapabilities of datasets are not assessed for benchmarking language\nunderstanding precisely. We propose a semi-automated, ablation-based\nmethodology for this challenge; By checking whether questions can be solved\neven after removing features associated with a skill requisite for language\nunderstanding, we evaluate to what degree the questions do not require the\nskill. Experiments on 10 datasets (e.g., CoQA, SQuAD v2.0, and RACE) with a\nstrong baseline model show that, for example, the relative scores of a baseline\nmodel provided with content words only and with shuffled sentence words in the\ncontext are on average 89.2% and 78.5% of the original score, respectively.\nThese results suggest that most of the questions already answered correctly by\nthe model do not necessarily require grammatical and complex reasoning. For\nprecise benchmarking, MRC datasets will need to take extra care in their design\nto ensure that questions can correctly evaluate the intended skills.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09247v1",
            "date": "2019-11-21 02:24:21+00:00",
            "title": "How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions",
            "authors": [
                "Zewei Chu",
                "Mingda Chen",
                "Jing Chen",
                "Miaosen Wang",
                "Kevin Gimpel",
                "Manaal Faruqui",
                "Xiance Si"
            ],
            "abstract": "We present a large-scale dataset for the task of rewriting an ill-formed\nnatural language question to a well-formed one. Our multi-domain question\nrewriting MQR dataset is constructed from human contributed Stack Exchange\nquestion edit histories. The dataset contains 427,719 question pairs which come\nfrom 303 domains. We provide human annotations for a subset of the dataset as a\nquality estimate. When moving from ill-formed to well-formed questions, the\nquestion quality improves by an average of 45 points across three aspects. We\ntrain sequence-to-sequence neural models on the constructed dataset and obtain\nan improvement of 13.2% in BLEU-4 over baseline methods built from other data\nresources. We release the MQR dataset to encourage research on the problem of\nquestion rewriting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09271v1",
            "date": "2019-11-21 03:48:46+00:00",
            "title": "Cantonese Automatic Speech Recognition Using Transfer Learning from Mandarin",
            "authors": [
                "Bryan Li",
                "Xinyue Wang",
                "Homayoon Beigi"
            ],
            "abstract": "We propose a system to develop a basic automatic speech recognizer(ASR) for\nCantonese, a low-resource language, through transfer learning of Mandarin, a\nhigh-resource language. We take a time-delayed neural network trained on\nMandarin, and perform weight transfer of several layers to a newly initialized\nmodel for Cantonese. We experiment with the number of layers transferred, their\nlearning rates, and pretraining i-vectors. Key findings are that this approach\nallows for quicker training time with less data. We find that for every epoch,\nlog-probability is smaller for transfer learning models compared to a\nCantonese-only model. The transfer learning models show slight improvement in\nCER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09319v1",
            "date": "2019-11-21 07:22:03+00:00",
            "title": "An Empirical Study of Sections in Classifying Disease Outbreak Reports",
            "authors": [
                "Son Doan",
                "Mike Conway",
                "Nigel Collier"
            ],
            "abstract": "Identifying articles that relate to infectious diseases is a necessary step\nfor any automatic bio-surveillance system that monitors news articles from the\nInternet. Unlike scientific articles which are available in a strongly\nstructured form, news articles are usually loosely structured. In this chapter,\nwe investigate the importance of each section and the effect of section\nweighting on performance of text classification. The experimental results show\nthat (1) classification models using the headline and leading sentence achieve\na high performance in terms of F-score compared to other parts of the article;\n(2) all section with bag-of-word representation (full text) achieves the\nhighest recall; and (3) section weighting information can help to improve\naccuracy.",
            "journal ref": "Web-Based Applications in Healthcare and Biomedicine, pp 47-58,\n  2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09333v1",
            "date": "2019-11-21 08:22:07+00:00",
            "title": "Generating Diverse Translation by Manipulating Multi-Head Attention",
            "authors": [
                "Zewei Sun",
                "Shujian Huang",
                "Hao-Ran Wei",
                "Xin-yu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Transformer model has been widely used on machine translation tasks and\nobtained state-of-the-art results. In this paper, we report an interesting\nphenomenon in its encoder-decoder multi-head attention: different attention\nheads of the final decoder layer align to different word translation\ncandidates. We empirically verify this discovery and propose a method to\ngenerate diverse translations by manipulating heads. Furthermore, we make use\nof these diverse translations with the back-translation technique for better\ndata augmentation. Experiment results show that our method generates diverse\ntranslations without severe drop in translation quality. Experiments also show\nthat back-translation with these diverse translations could bring significant\nimprovement on performance on translation tasks. An auxiliary experiment of\nconversation response generation task proves the effect of diversity as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09339v2",
            "date": "2019-11-21 08:39:21+00:00",
            "title": "Emotion Recognition for Vietnamese Social Media Text",
            "authors": [
                "Vong Anh Ho",
                "Duong Huynh-Cong Nguyen",
                "Danh Hoang Nguyen",
                "Linh Thi-Van Pham",
                "Duc-Vu Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Emotion recognition or emotion prediction is a higher approach or a special\ncase of sentiment analysis. In this task, the result is not produced in terms\nof either polarity: positive or negative or in the form of rating (from 1 to 5)\nbut of a more detailed level of analysis in which the results are depicted in\nmore expressions like sadness, enjoyment, anger, disgust, fear, and surprise.\nEmotion recognition plays a critical role in measuring the brand value of a\nproduct by recognizing specific emotions of customers' comments. In this study,\nwe have achieved two targets. First and foremost, we built a standard\nVietnamese Social Media Emotion Corpus (UIT-VSMEC) with exactly 6,927\nemotion-annotated sentences, contributing to emotion recognition research in\nVietnamese which is a low-resource language in natural language processing\n(NLP). Secondly, we assessed and measured machine learning and deep neural\nnetwork models on our UIT-VSMEC corpus. As a result, the CNN model achieved the\nhighest performance with the weighted F1-score of 59.74%. Our corpus is\navailable at our research website.",
            "journal ref": "In Proceeding of PACLING 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09487v2",
            "date": "2019-11-21 14:29:42+00:00",
            "title": "Chemical-protein Interaction Extraction via Gaussian Probability Distribution and External Biomedical Knowledge",
            "authors": [
                "Cong Sun",
                "Zhihao Yang",
                "Leilei Su",
                "Lei Wang",
                "Yin Zhang",
                "Hongfei Lin",
                "Jian Wang"
            ],
            "abstract": "Motivation: The biomedical literature contains a wealth of chemical-protein\ninteractions (CPIs). Automatically extracting CPIs described in biomedical\nliterature is essential for drug discovery, precision medicine, as well as\nbasic biomedical research. Most existing methods focus only on the sentence\nsequence to identify these CPIs. However, the local structure of sentences and\nexternal biomedical knowledge also contain valuable information. Effective use\nof such information may improve the performance of CPI extraction. Results: In\nthis paper, we propose a novel neural network-based approach to improve CPI\nextraction. Specifically, the approach first employs BERT to generate\nhigh-quality contextual representations of the title sequence, instance\nsequence, and knowledge sequence. Then, the Gaussian probability distribution\nis introduced to capture the local structure of the instance. Meanwhile, the\nattention mechanism is applied to fuse the title information and biomedical\nknowledge, respectively. Finally, the related representations are concatenated\nand fed into the softmax function to extract CPIs. We evaluate our proposed\nmodel on the CHEMPROT corpus. Our proposed model is superior in performance as\ncompared with other state-of-the-art models. The experimental results show that\nthe Gaussian probability distribution and external knowledge are complementary\nto each other. Integrating them can effectively improve the CPI extraction\nperformance. Furthermore, the Gaussian probability distribution can effectively\nimprove the extraction performance of sentences with overlapping relations in\nbiomedical relation extraction tasks. Availability: Data and code are available\nat https://github.com/CongSun-dlut/CPI_extraction. Contact: yangzh@dlut.edu.cn,\nwangleibihami@gmail.com Supplementary information: Supplementary data are\navailable at Bioinformatics online.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09515v3",
            "date": "2019-11-21 15:05:10+00:00",
            "title": "An analysis of observation length requirements for machine understanding of human behaviors from spoken language",
            "authors": [
                "Sandeep Nallan Chakravarthula",
                "Brian Baucom",
                "Shrikanth Narayanan",
                "Panayiotis Georgiou"
            ],
            "abstract": "The task of quantifying human behavior by observing interaction cues is an\nimportant and useful one across a range of domains in psychological research\nand practice. Machine learning-based approaches typically perform this task by\nfirst estimating behavior based on cues within an observation window, such as a\nfixed number of words, and then aggregating the behavior over all the windows\nin that interaction. The length of this window directly impacts the accuracy of\nestimation by controlling the amount of information being used. The exact link\nbetween window length and accuracy, however, has not been well studied,\nespecially in spoken language. In this paper, we investigate this link and\npresent an analysis framework that determines appropriate window lengths for\nthe task of behavior estimation. Our proposed framework utilizes a two-pronged\nevaluation approach: (a) extrinsic similarity between machine predictions and\nhuman expert annotations, and (b) intrinsic consistency between intra-machine\nand intra-human behavior relations. We apply our analysis to real-life\nconversations that are annotated for a large and diverse set of behavior codes\nand examine the relation between the nature of a behavior and how long it\nshould be observed. We find that behaviors describing negative and positive\naffect can be accurately estimated from short to medium-length expressions\nwhereas behaviors related to problem-solving and dysphoria require much longer\nobservations and are difficult to quantify from language alone. These findings\nare found to be generally consistent across different behavior modeling\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09532v2",
            "date": "2019-11-21 15:14:39+00:00",
            "title": "A Cluster Ranking Model for Full Anaphora Resolution",
            "authors": [
                "Juntao Yu",
                "Alexandra Uma",
                "Massimo Poesio"
            ],
            "abstract": "Anaphora resolution (coreference) systems designed for the CONLL 2012 dataset\ntypically cannot handle key aspects of the full anaphora resolution task such\nas the identification of singletons and of certain types of non-referring\nexpressions (e.g., expletives), as these aspects are not annotated in that\ncorpus. However, the recently released dataset for the CRAC 2018 Shared Task\ncan now be used for that purpose. In this paper, we introduce an architecture\nto simultaneously identify non-referring expressions (including expletives,\npredicative s, and other types) and build coreference chains, including\nsingletons. Our cluster-ranking system uses an attention mechanism to determine\nthe relative importance of the mentions in the same cluster. Additional\nclassifiers are used to identify singletons and non-referring markables. Our\ncontributions are as follows. First all, we report the first result on the CRAC\ndata using system mentions; our result is 5.8% better than the shared task\nbaseline system, which used gold mentions. Second, we demonstrate that the\navailability of singleton clusters and non-referring expressions can lead to\nsubstantially improved performance on non-singleton clusters as well. Third, we\nshow that despite our model not being designed specifically for the CONLL data,\nit achieves a score equivalent to that of the state-of-the-art system by Kantor\nand Globerson (2019) on that dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09789v1",
            "date": "2019-11-22 00:04:07+00:00",
            "title": "Learning Multi-level Dependencies for Robust Word Recognition",
            "authors": [
                "Zhiwei Wang",
                "Hui Liu",
                "Jiliang Tang",
                "Songfan Yang",
                "Gale Yan Huang",
                "Zitao Liu"
            ],
            "abstract": "Robust language processing systems are becoming increasingly important given\nthe recent awareness of dangerous situations where brittle machine learning\nmodels can be easily broken with the presence of noises. In this paper, we\nintroduce a robust word recognition framework that captures multi-level\nsequential dependencies in noised sentences. The proposed framework employs a\nsequence-to-sequence model over characters of each word, whose output is given\nto a word-level bi-directional recurrent neural network. We conduct extensive\nexperiments to verify the effectiveness of the framework. The results show that\nthe proposed framework outperforms state-of-the-art methods by a large margin\nand they also suggest that character-level dependencies can play an important\nrole in word recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09801v1",
            "date": "2019-11-22 01:14:57+00:00",
            "title": "Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answering",
            "authors": [
                "Yang Deng",
                "Wai Lam",
                "Yuexiang Xie",
                "Daoyuan Chen",
                "Yaliang Li",
                "Min Yang",
                "Ying Shen"
            ],
            "abstract": "Community question answering (CQA) gains increasing popularity in both\nacademy and industry recently. However, the redundancy and lengthiness issues\nof crowdsourced answers limit the performance of answer selection and lead to\nreading difficulties and misunderstandings for community users. To solve these\nproblems, we tackle the tasks of answer selection and answer summary generation\nin CQA with a novel joint learning model. Specifically, we design a\nquestion-driven pointer-generator network, which exploits the correlation\ninformation between question-answer pairs to aid in attending the essential\ninformation when generating answer summaries. Meanwhile, we leverage the answer\nsummaries to alleviate noise in original lengthy answers when ranking the\nrelevancy degrees of question-answer pairs. In addition, we construct a new\nlarge-scale CQA corpus, WikiHowQA, which contains long answers for answer\nselection as well as reference summaries for answer summarization. The\nexperimental results show that the joint learning method can effectively\naddress the answer redundancy issue in CQA and achieves state-of-the-art\nresults on both answer selection and text summarization tasks. Furthermore, the\nproposed model is shown to be of great transferring ability and applicability\nfor resource-poor CQA tasks, which lack of reference answer summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09844v1",
            "date": "2019-11-22 04:10:46+00:00",
            "title": "Weakly-Supervised Opinion Summarization by Leveraging External Information",
            "authors": [
                "Chao Zhao",
                "Snigdha Chaturvedi"
            ],
            "abstract": "Opinion summarization from online product reviews is a challenging task,\nwhich involves identifying opinions related to various aspects of the product\nbeing reviewed. While previous works require additional human effort to\nidentify relevant aspects, we instead apply domain knowledge from external\nsources to automatically achieve the same goal. This work proposes AspMem, a\ngenerative method that contains an array of memory cells to store\naspect-related knowledge. This explicit memory can help obtain a better opinion\nrepresentation and infer the aspect information more precisely. We evaluate\nthis method on both aspect identification and opinion summarization tasks. Our\nexperiments show that AspMem outperforms the state-of-the-art methods even\nthough, unlike the baselines, it does not rely on human supervision which is\ncarefully handcrafted for the given tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09845v1",
            "date": "2019-11-22 04:14:31+00:00",
            "title": "A Discrete CVAE for Response Generation on Short-Text Conversation",
            "authors": [
                "Jun Gao",
                "Wei Bi",
                "Xiaojiang Liu",
                "Junhui Li",
                "Guodong Zhou",
                "Shuming Shi"
            ],
            "abstract": "Neural conversation models such as encoder-decoder models are easy to\ngenerate bland and generic responses. Some researchers propose to use the\nconditional variational autoencoder(CVAE) which maximizes the lower bound on\nthe conditional log-likelihood on a continuous latent variable. With different\nsampled la-tent variables, the model is expected to generate diverse responses.\nAlthough the CVAE-based models have shown tremendous potential, their\nimprovement of generating high-quality responses is still unsatisfactory. In\nthis paper, we introduce a discrete latent variable with an explicit semantic\nmeaning to improve the CVAE on short-text conversation. A major advantage of\nour model is that we can exploit the semantic distance between the latent\nvariables to maintain good diversity between the sampled latent variables.\nAccordingly, we pro-pose a two-stage sampling approach to enable efficient\ndiverse variable selection from a large latent space assumed in the short-text\nconversation task. Experimental results indicate that our model outperforms\nvarious kinds of generation models under both automatic and human evaluations\nand generates more diverse and in-formative responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09883v1",
            "date": "2019-11-22 06:46:11+00:00",
            "title": "Classifying Vietnamese Disease Outbreak Reports with Important Sentences and Rich Features",
            "authors": [
                "Son Doan",
                "Nguyen Thi Ngoc Vinh",
                "Tu Minh Phuong"
            ],
            "abstract": "Text classification is an important field of research from mid 90s up to now.\nIt has many applications, one of them is in Web-based biosurveillance systems\nwhich identify and summarize online disease outbreak reports. In this paper we\nfocus on classifying Vietnamese disease outbreak reports. We investigate\nimportant properties of disease outbreak reports, e.g., sentences containing\nnames of outbreak disease, locations. Evaluation on 10-time 10- fold\ncross-validation using the Support Vector Machine algorithm shows that using\nsentences containing disease outbreak names with its preceding/following\nsentences in combination with location features achieve the best F-score with\n86.67% - an improvement of 0.38% in comparison to using all raw text. Our\nresults suggest that using important sentences and rich feature can improve\nperformance of Vietnamese disease outbreak text classification.",
            "journal ref": "Proc. of the Third Symposium on Information and Communication\n  Technology (SoICT), pages 260-265, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09896v2",
            "date": "2019-11-22 07:26:40+00:00",
            "title": "Continual adaptation for efficient machine communication",
            "authors": [
                "Robert D. Hawkins",
                "Minae Kwon",
                "Dorsa Sadigh",
                "Noah D. Goodman"
            ],
            "abstract": "To communicate with new partners in new contexts, humans rapidly form new\nlinguistic conventions. Recent neural language models are able to comprehend\nand produce the existing conventions present in their training data, but are\nnot able to flexibly and interactively adapt those conventions on the fly as\nhumans do. We introduce an interactive repeated reference task as a benchmark\nfor models of adaptation in communication and propose a regularized continual\nlearning framework that allows an artificial agent initialized with a generic\nlanguage model to more accurately and efficiently communicate with a partner\nover time. We evaluate this framework through simulations on COCO and in\nreal-time reference game experiments with human partners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09912v1",
            "date": "2019-11-22 08:15:52+00:00",
            "title": "Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks",
            "authors": [
                "Yong Wang",
                "Longyue Wang",
                "Shuming Shi",
                "Victor O. K. Li",
                "Zhaopeng Tu"
            ],
            "abstract": "The key challenge of multi-domain translation lies in simultaneously encoding\nboth the general knowledge shared across domains and the particular knowledge\ndistinctive to each domain in a unified model. Previous work shows that the\nstandard neural machine translation (NMT) model, trained on mixed-domain data,\ngenerally captures the general knowledge, but misses the domain-specific\nknowledge. In response to this problem, we augment NMT model with additional\ndomain transformation networks to transform the general representations to\ndomain-specific representations, which are subsequently fed to the NMT decoder.\nTo guarantee the knowledge transformation, we also propose two complementary\nsupervision signals by leveraging the power of knowledge distillation and\nadversarial learning. Experimental results on several language pairs, covering\nboth balanced and unbalanced multi-domain translation, demonstrate the\neffectiveness and universality of the proposed approach. Encouragingly, the\nproposed unified model achieves comparable results with the fine-tuning\napproach that requires multiple models to preserve the particular knowledge.\nFurther analyses reveal that the domain transformation networks successfully\ncapture the domain-specific knowledge as expected.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09919v1",
            "date": "2019-11-22 08:36:42+00:00",
            "title": "Resource production of written forms of Sign Languages by a user-centered editor, SWift (SignWriting improved fast transcriber)",
            "authors": [
                "Fabrizio Borgia",
                "Claudia S. Bianchini",
                "Patrice Dalle",
                "Maria de Marsico"
            ],
            "abstract": "The SignWriting improved fast transcriber (SWift), presented in this paper,\nis an advanced editor for computer-aided writing and transcribing of any Sign\nLanguage (SL) using SignWriting (SW). The application is an editor which allows\ncomposing and saving desired signs using the SW elementary components, called\n\"glyphs\". These make up a sort of alphabet, which does not depend on the\nnational Sign Language and which codes the basic components of any sign. The\nuser is guided through a fully-automated procedure, making the composition\nprocess fast and intuitive. SWift pursues the goal of helping to break down the\n\"electronic barriers\" that keep deaf people away from the web, and at the same\ntime to support linguistic research about Sign Languages features. For this\nreason it has been designed with a special attention to deaf user needs, and to\ngeneral usability issues. The editor has been developed in a modular way, so it\ncan be integrated everywhere the use of SW as an alternative to written\n\"verbal\" language may be advisable.",
            "journal ref": "Proceedings of the VIII International Language Resources\n  Evaluation Conference (LREC), ELRA - European Language Resources Association,\n  pp.3779-3784, 2012, 978-2-9517408-7-7",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10038v2",
            "date": "2019-11-22 13:39:06+00:00",
            "title": "Multilingual Culture-Independent Word Analogy Datasets",
            "authors": [
                "Matej Ul\u010dar",
                "Kristiina Vaik",
                "Jessica Lindstr\u00f6m",
                "Milda Dailid\u0117nait\u0117",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "In text processing, deep neural networks mostly use word embeddings as an\ninput. Embeddings have to ensure that relations between words are reflected\nthrough distances in a high-dimensional numeric space. To compare the quality\nof different text embeddings, typically, we use benchmark datasets. We present\na collection of such datasets for the word analogy task in nine languages:\nCroatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian,\nand Swedish. We redesigned the original monolingual analogy task to be much\nmore culturally independent and also constructed cross-lingual analogy datasets\nfor the involved languages. We present basic statistics of the created datasets\nand their initial evaluation using fastText embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10132v1",
            "date": "2019-11-22 16:48:10+00:00",
            "title": "CRUR: Coupled-Recurrent Unit for Unification, Conceptualization and Context Capture for Language Representation -- A Generalization of Bi Directional LSTM",
            "authors": [
                "Chiranjib Sur"
            ],
            "abstract": "In this work we have analyzed a novel concept of sequential binding based\nlearning capable network based on the coupling of recurrent units with Bayesian\nprior definition. The coupling structure encodes to generate efficient tensor\nrepresentations that can be decoded to generate efficient sentences and can\ndescribe certain events. These descriptions are derived from structural\nrepresentations of visual features of images and media. An elaborated study of\nthe different types of coupling recurrent structures are studied and some\ninsights of their performance are provided. Supervised learning performance for\nnatural language processing is judged based on statistical evaluations,\nhowever, the truth is perspective, and in this case the qualitative evaluations\nreveal the real capability of the different architectural strengths and\nvariations. Bayesian prior definition of different embedding helps in better\ncharacterization of the sentences based on the natural language structure\nrelated to parts of speech and other semantic level categorization in a form\nwhich is machine interpret-able and inherits the characteristics of the Tensor\nRepresentation binding and unbinding based on the mutually orthogonality. Our\napproach has surpassed some of the existing basic works related to image\ncaptioning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10180v1",
            "date": "2019-11-22 18:29:19+00:00",
            "title": "Topical Phrase Extraction from Clinical Reports by Incorporating both Local and Global Context",
            "authors": [
                "Gabriele Pergola",
                "Yulan He",
                "David Lowe"
            ],
            "abstract": "Making sense of words often requires to simultaneously examine the\nsurrounding context of a term as well as the global themes characterizing the\noverall corpus. Several topic models have already exploited word embeddings to\nrecognize local context, however, it has been weakly combined with the global\ncontext during the topic inference. This paper proposes to extract topical\nphrases corroborating the word embedding information with the global context\ndetected by Latent Semantic Analysis, and then combine them by means of the\nP\\'{o}lya urn model. To highlight the effectiveness of this combined approach\nthe model was assessed analyzing clinical reports, a challenging scenario\ncharacterized by technical jargon and a limited word statistics available.\nResults show it outperforms the state-of-the-art approaches in terms of both\ntopic coherence and computational cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10384v4",
            "date": "2019-11-23 16:23:21+00:00",
            "title": "Discourse Level Factors for Sentence Deletion in Text Simplification",
            "authors": [
                "Yang Zhong",
                "Chao Jiang",
                "Wei Xu",
                "Junyi Jessy Li"
            ],
            "abstract": "This paper presents a data-driven study focusing on analyzing and predicting\nsentence deletion -- a prevalent but understudied phenomenon in document\nsimplification -- on a large English text simplification corpus. We inspect\nvarious document and discourse factors associated with sentence deletion, using\na new manually annotated sentence alignment corpus we collected. We reveal that\nprofessional editors utilize different strategies to meet readability standards\nof elementary and middle schools. To predict whether a sentence will be deleted\nduring simplification to a certain level, we harness automatically aligned data\nto train a classification model. Evaluated on our manually annotated data, our\nbest models reached F1 scores of 65.2 and 59.7 for this task at the levels of\nelementary and middle school, respectively. We find that discourse level\nfactors contribute to the challenging task of predicting sentence deletion for\nsimplification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10389v1",
            "date": "2019-11-23 17:28:54+00:00",
            "title": "Joint Parsing and Generation for Abstractive Summarization",
            "authors": [
                "Kaiqiang Song",
                "Logan Lebanoff",
                "Qipeng Guo",
                "Xipeng Qiu",
                "Xiangyang Xue",
                "Chen Li",
                "Dong Yu",
                "Fei Liu"
            ],
            "abstract": "Sentences produced by abstractive summarization systems can be ungrammatical\nand fail to preserve the original meanings, despite being locally fluent. In\nthis paper we propose to remedy this problem by jointly generating a sentence\nand its syntactic dependency parse while performing abstraction. If generating\na word can introduce an erroneous relation to the summary, the behavior must be\ndiscouraged. The proposed method thus holds promise for producing grammatical\nsentences and encouraging the summary to stay true-to-original. Our\ncontributions of this work are twofold. First, we present a novel neural\narchitecture for abstractive summarization that combines a sequential decoder\nwith a tree-based decoder in a synchronized manner to generate a summary\nsentence and its syntactic parse. Secondly, we describe a novel human\nevaluation protocol to assess if, and to what extent, a summary remains true to\nits original meanings. We evaluate our method on a number of summarization\ndatasets and demonstrate competitive results against strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10390v1",
            "date": "2019-11-23 17:34:19+00:00",
            "title": "Controlling the Amount of Verbatim Copying in Abstractive Summarization",
            "authors": [
                "Kaiqiang Song",
                "Bingqing Wang",
                "Zhe Feng",
                "Liu Ren",
                "Fei Liu"
            ],
            "abstract": "An abstract must not change the meaning of the original text. A single most\neffective way to achieve that is to increase the amount of copying while still\nallowing for text abstraction. Human editors can usually exercise control over\ncopying, resulting in summaries that are more extractive than abstractive, or\nvice versa. However, it remains poorly understood whether modern neural\nabstractive summarizers can provide the same flexibility, i.e., learning from\nsingle reference summaries to generate multiple summary hypotheses with varying\ndegrees of copying. In this paper, we present a neural summarization model\nthat, by learning from single human abstracts, can produce a broad spectrum of\nsummaries ranging from purely extractive to highly generative ones. We frame\nthe task of summarization as language modeling and exploit alternative\nmechanisms to generate summary hypotheses. Our method allows for control over\ncopying during both training and decoding stages of a neural summarization\nmodel. Through extensive experiments we illustrate the significance of our\nproposed method on controlling the amount of verbatim copying and achieve\ncompetitive results over strong baselines. Our analysis further reveals\ninteresting and unobvious facts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10392v1",
            "date": "2019-11-23 17:41:02+00:00",
            "title": "When is ACL's Deadline? A Scientific Conversational Agent",
            "authors": [
                "Mohsen Mesgar",
                "Paul Youssef",
                "Lin Li",
                "Dominik Bierwirth",
                "Yihao Li",
                "Christian M. Meyer",
                "Iryna Gurevych"
            ],
            "abstract": "Our conversational agent UKP-ATHENA assists NLP researchers in finding and\nexploring scientific literature, identifying relevant authors, planning or\npost-processing conference visits, and preparing paper submissions using a\nunified interface based on natural language inputs and responses. UKP-ATHENA\nenables new access paths to our swiftly evolving research area with its massive\namounts of scientific information and high turnaround times. UKP-ATHENA's\nresponses connect information from multiple heterogeneous sources which\nresearchers currently have to explore manually one after another. Unlike a\nsearch engine, UKP-ATHENA maintains the context of a conversation to allow for\nefficient information access on papers, researchers, and conferences. Our\narchitecture consists of multiple components with reference implementations\nthat can be easily extended by new skills and domains. Our user-based\nevaluation shows that UKP-ATHENA already responds 45% of different formulations\nof defined intents with 37% information coverage rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10401v2",
            "date": "2019-11-23 18:37:48+00:00",
            "title": "A Transformer-based approach to Irony and Sarcasm detection",
            "authors": [
                "Rolandos Alexandros Potamias",
                "Georgios Siolas",
                "Andreas - Georgios Stafylopatis"
            ],
            "abstract": "Figurative Language (FL) seems ubiquitous in all social-media discussion\nforums and chats, posing extra challenges to sentiment analysis endeavors.\nIdentification of FL schemas in short texts remains largely an unresolved issue\nin the broader field of Natural Language Processing (NLP), mainly due to their\ncontradictory and metaphorical meaning content. The main FL expression forms\nare sarcasm, irony and metaphor. In the present paper we employ advanced Deep\nLearning (DL) methodologies to tackle the problem of identifying the\naforementioned FL forms. Significantly extending our previous work [71], we\npropose a neural network methodology that builds on a recently proposed\npre-trained transformer-based network architecture which, is further enhanced\nwith the employment and devise of a recurrent convolutional neural network\n(RCNN). With this set-up, data preprocessing is kept in minimum. The\nperformance of the devised hybrid neural architecture is tested on four\nbenchmark datasets, and contrasted with other relevant state of the art\nmethodologies and systems. Results demonstrate that the proposed methodology\nachieves state of the art performance under all benchmark datasets,\noutperforming, even by a large margin, all other methodologies and published\nstudies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10436v1",
            "date": "2019-11-24 00:08:09+00:00",
            "title": "ScienceExamCER: A High-Density Fine-Grained Science-Domain Corpus for Common Entity Recognition",
            "authors": [
                "Hannah Smith",
                "Zeyu Zhang",
                "John Culnan",
                "Peter Jansen"
            ],
            "abstract": "Named entity recognition identifies common classes of entities in text, but\nthese entity labels are generally sparse, limiting utility to downstream tasks.\nIn this work we present ScienceExamCER, a densely-labeled semantic\nclassification corpus of 133k mentions in the science exam domain where nearly\nall (96%) of content words have been annotated with one or more fine-grained\nsemantic class labels including taxonomic groups, meronym groups, verb/action\ngroups, properties and values, and synonyms. Semantic class labels are drawn\nfrom a manually-constructed fine-grained typology of 601 classes generated\nthrough a data-driven analysis of 4,239 science exam questions. We show an\noff-the-shelf BERT-based named entity recognition model modified for\nmulti-label classification achieves an accuracy of 0.85 F1 on this task,\nsuggesting strong utility for downstream tasks in science domain question\nanswering requiring densely-labeled semantic classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10439v3",
            "date": "2019-11-24 00:30:30+00:00",
            "title": "Enhancing Out-Of-Domain Utterance Detection with Data Augmentation Based on Word Embeddings",
            "authors": [
                "Yueqi Feng",
                "Jiali Lin"
            ],
            "abstract": "For most intelligent assistant systems, it is essential to have a mechanism\nthat detects out-of-domain (OOD) utterances automatically to handle noisy input\nproperly. One typical approach would be introducing a separate class that\ncontains OOD utterance examples combined with in-domain text samples into the\nclassifier. However, since OOD utterances are usually unseen to the training\ndatasets, the detection performance largely depends on the quality of the\nattached OOD text data with restricted sizes of samples due to computing\nlimits. In this paper, we study how augmented OOD data based on sampling impact\nOOD utterance detection with a small sample size. We hypothesize that OOD\nutterance samples chosen randomly can increase the coverage of unknown OOD\nutterance space and enhance detection accuracy if they are more dispersed.\nExperiments show that given the same dataset with the same OOD sample size, the\nOOD utterance detection performance improves when OOD samples are more\nspread-out.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10470v2",
            "date": "2019-11-24 08:27:42+00:00",
            "title": "Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering",
            "authors": [
                "Akari Asai",
                "Kazuma Hashimoto",
                "Hannaneh Hajishirzi",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Answering questions that require multi-hop reasoning at web-scale\nnecessitates retrieving multiple evidence documents, one of which often has\nlittle lexical or semantic relationship to the question. This paper introduces\na new graph-based recurrent retrieval approach that learns to retrieve\nreasoning paths over the Wikipedia graph to answer multi-hop open-domain\nquestions. Our retriever model trains a recurrent neural network that learns to\nsequentially retrieve evidence paragraphs in the reasoning path by conditioning\non the previously retrieved documents. Our reader model ranks the reasoning\npaths and extracts the answer span included in the best reasoning path.\nExperimental results show state-of-the-art results in three open-domain QA\ndatasets, showcasing the effectiveness and robustness of our method. Notably,\nour method achieves significant improvement in HotpotQA, outperforming the\nprevious best model by more than 14 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10666v1",
            "date": "2019-11-25 02:12:45+00:00",
            "title": "Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer",
            "authors": [
                "Henghui Zhu",
                "Feng Nan",
                "Zhiguo Wang",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "Conversation structure is useful for both understanding the nature of\nconversation dynamics and for providing features for many downstream\napplications such as summarization of conversations. In this work, we define\nthe problem of conversation structure modeling as identifying the parent\nutterance(s) to which each utterance in the conversation responds to. Previous\nwork usually took a pair of utterances to decide whether one utterance is the\nparent of the other. We believe the entire ancestral history is a very\nimportant information source to make accurate prediction. Therefore, we design\na novel masking mechanism to guide the ancestor flow, and leverage the\ntransformer model to aggregate all ancestors to predict parent utterances. Our\nexperiments are performed on the Reddit dataset (Zhang, Culbertson, and\nParitosh 2017) and the Ubuntu IRC dataset (Kummerfeld et al. 2019). In\naddition, we also report experiments on a new larger corpus from the Reddit\nplatform and release this dataset. We show that the proposed model, that takes\ninto account the ancestral history of the conversation, significantly\noutperforms several strong baselines including the BERT model on all datasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10668v2",
            "date": "2019-11-25 02:14:51+00:00",
            "title": "JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus",
            "authors": [
                "Makoto Morishita",
                "Jun Suzuki",
                "Masaaki Nagata"
            ],
            "abstract": "Recent machine translation algorithms mainly rely on parallel corpora.\nHowever, since the availability of parallel corpora remains limited, only some\nresource-rich language pairs can benefit from them. We constructed a parallel\ncorpus for English-Japanese, for which the amount of publicly available\nparallel corpora is still limited. We constructed the parallel corpus by\nbroadly crawling the web and automatically aligning parallel sentences. Our\ncollected corpus, called JParaCrawl, amassed over 8.7 million sentence pairs.\nWe show how it includes a broader range of domains and how a neural machine\ntranslation model trained with it works as a good pre-trained model for\nfine-tuning specific domains. The pre-training and fine-tuning approaches\nachieved or surpassed performance comparable to model training from the initial\nstate and reduced the training time. Additionally, we trained the model with an\nin-domain dataset and JParaCrawl to show how we achieved the best performance\nwith them. JParaCrawl and the pre-trained models are freely available online\nfor research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10704v1",
            "date": "2019-11-25 04:57:08+00:00",
            "title": "Conversational implicatures in English dialogue: Annotated dataset",
            "authors": [
                "Elizabeth Jasmi George",
                "Radhika Mamidi"
            ],
            "abstract": "Human dialogue often contains utterances having meanings entirely different\nfrom the sentences used and are clearly understood by the interlocutors. But in\nhuman-computer interactions, the machine fails to understand the implicated\nmeaning unless it is trained with a dataset containing the implicated meaning\nof an utterance along with the utterance and the context in which it is\nuttered. In linguistic terms, conversational implicatures are the meanings of\nthe speaker's utterance that are not part of what is explicitly said. In this\npaper, we introduce a dataset of dialogue snippets with three constituents,\nwhich are the context, the utterance, and the implicated meanings. These\nimplicated meanings are the conversational implicatures. The utterances are\ncollected by transcribing from listening comprehension sections of English\ntests like TOEFL (Test of English as a Foreign Language) as well as scraping\ndialogues from movie scripts available on IMSDb (Internet Movie Script\nDatabase). The utterances are manually annotated with implicatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10732v2",
            "date": "2019-11-25 07:22:47+00:00",
            "title": "Learning to Reuse Translations: Guiding Neural Machine Translation with Examples",
            "authors": [
                "Qian Cao",
                "Shaohui Kuang",
                "Deyi Xiong"
            ],
            "abstract": "In this paper, we study the problem of enabling neural machine translation\n(NMT) to reuse previous translations from similar examples in target\nprediction. Distinguishing reusable translations from noisy segments and\nlearning to reuse them in NMT are non-trivial. To solve these challenges, we\npropose an Example-Guided NMT (EGNMT) framework with two models: (1) a\nnoise-masked encoder model that masks out noisy words according to word\nalignments and encodes the noise-masked sentences with an additional example\nencoder and (2) an auxiliary decoder model that predicts reusable words via an\nauxiliary decoder sharing parameters with the primary decoder. We define and\nimplement the two models with the state-of-the-art Transformer. Experiments\nshow that the noise-masked encoder model allows NMT to learn useful information\nfrom examples with low fuzzy match scores (FMS) while the auxiliary decoder\nmodel is good for high-FMS examples. More experiments on Chinese-English,\nEnglish-German and English-Spanish translation demonstrate that the combination\nof the two EGNMT models can achieve improvements of up to +9 BLEU points over\nthe baseline system and +7 BLEU points over a two-encoder Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10750v1",
            "date": "2019-11-25 07:58:00+00:00",
            "title": "Chinese Spelling Error Detection Using a Fusion Lattice LSTM",
            "authors": [
                "Hao Wang",
                "Bing Wang",
                "Jianyong Duan",
                "Jiajun Zhang"
            ],
            "abstract": "Spelling error detection serves as a crucial preprocessing in many natural\nlanguage processing applications. Due to the characteristics of Chinese\nLanguage, Chinese spelling error detection is more challenging than error\ndetection in English. Existing methods are mainly under a pipeline framework,\nwhich artificially divides error detection process into two steps. Thus, these\nmethods bring error propagation and cannot always work well due to the\ncomplexity of the language environment. Besides existing methods only adopt\ncharacter or word information, and ignore the positive effect of fusing\ncharacter, word, pinyin1 information together. We propose an LF-LSTM-CRF model,\nwhich is an extension of the LSTMCRF with word lattices and\ncharacter-pinyin-fusion inputs. Our model takes advantage of the end-to-end\nframework to detect errors as a whole process, and dynamically integrates\ncharacter, word and pinyin information. Experiments on the SIGHAN data show\nthat our LF-LSTM-CRF outperforms existing methods with similar external\nresources consistently, and confirm the feasibility of adopting the end-to-end\nframework and the availability of integrating of character, word and pinyin\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10768v2",
            "date": "2019-11-25 08:40:34+00:00",
            "title": "Unsupervised Domain Adaptation of Language Models for Reading Comprehension",
            "authors": [
                "Kosuke Nishida",
                "Kyosuke Nishida",
                "Itsumi Saito",
                "Hisako Asano",
                "Junji Tomita"
            ],
            "abstract": "This study tackles unsupervised domain adaptation of reading comprehension\n(UDARC). Reading comprehension (RC) is a task to learn the capability for\nquestion answering with textual sources. State-of-the-art models on RC still do\nnot have general linguistic intelligence; i.e., their accuracy worsens for\nout-domain datasets that are not used in the training. We hypothesize that this\ndiscrepancy is caused by a lack of the language modeling (LM) capability for\nthe out-domain. The UDARC task allows models to use supervised RC training data\nin the source domain and only unlabeled passages in the target domain. To solve\nthe UDARC problem, we provide two domain adaptation models. The first one\nlearns the out-domain LM and in-domain RC task sequentially. The second one is\nthe proposed model that uses a multi-task learning approach of LM and RC. The\nmodels can retain both the RC capability acquired from the supervised data in\nthe source domain and the LM capability from the unlabeled data in the target\ndomain. We evaluated the models on UDARC with five datasets in different\ndomains. The models outperformed the model without domain adaptation. In\nparticular, the proposed model yielded an improvement of 4.3/4.2 points in\nEM/F1 in an unseen biomedical domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10783v1",
            "date": "2019-11-25 09:35:02+00:00",
            "title": "Financial Event Extraction Using Wikipedia-Based Weak Supervision",
            "authors": [
                "Liat Ein-Dor",
                "Ariel Gera",
                "Orith Toledo-Ronen",
                "Alon Halfon",
                "Benjamin Sznajder"
            ],
            "abstract": "Extraction of financial and economic events from text has previously been\ndone mostly using rule-based methods, with more recent works employing machine\nlearning techniques. This work is in line with this latter approach, leveraging\nrelevant Wikipedia sections to extract weak labels for sentences describing\neconomic events. Whereas previous weakly supervised approaches required a\nknowledge-base of such events, or corresponding financial figures, our approach\nrequires no such additional data, and can be employed to extract economic\nevents related to companies which are not even mentioned in the training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10835v2",
            "date": "2019-11-25 11:22:45+00:00",
            "title": "Outbound Translation User Interface Ptakopet: A Pilot Study",
            "authors": [
                "Vil\u00e9m Zouhar",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "It is not uncommon for Internet users to have to produce a text in a foreign\nlanguage they have very little knowledge of and are unable to verify the\ntranslation quality. We call the task \"outbound translation\" and explore it by\nintroducing an open-source modular system Ptakop\\v{e}t. Its main purpose is to\ninspect human interaction with MT systems enhanced with additional subsystems,\nsuch as backward translation and quality estimation. We follow up with an\nexperiment on (Czech) human annotators tasked to produce questions in a\nlanguage they do not speak (German), with the help of Ptakop\\v{e}t. We focus on\nthree real-world use cases (communication with IT support, describing\nadministrative issues and asking encyclopedic questions) from which we gain\ninsight into different strategies users take when faced with outbound\ntranslation tasks. Round trip translation is known to be unreliable for\nevaluating MT systems but our experimental evaluation documents that it works\nvery well for users, at least on MT systems of mid-range quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10876v4",
            "date": "2019-11-25 12:48:27+00:00",
            "title": "Towards robust word embeddings for noisy texts",
            "authors": [
                "Yerai Doval",
                "Jes\u00fas Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Research on word embeddings has mainly focused on improving their performance\non standard corpora, disregarding the difficulties posed by noisy texts in the\nform of tweets and other types of non-standard writing from social media. In\nthis work, we propose a simple extension to the skipgram model in which we\nintroduce the concept of bridge-words, which are artificial words added to the\nmodel to strengthen the similarity between standard words and their noisy\nvariants. Our new embeddings outperform baseline models on noisy texts on a\nwide range of evaluation tasks, both intrinsic and extrinsic, while retaining a\ngood performance on standard texts. To the best of our knowledge, this is the\nfirst explicit approach at dealing with this type of noisy texts at the word\nembedding level that goes beyond the support for out-of-vocabulary words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11008v1",
            "date": "2019-11-25 15:55:56+00:00",
            "title": "Korean-to-Chinese Machine Translation using Chinese Character as Pivot Clue",
            "authors": [
                "Jeonghyeok Park",
                "Hai Zhao"
            ],
            "abstract": "Korean-Chinese is a low resource language pair, but Korean and Chinese have a\nlot in common in terms of vocabulary. Sino-Korean words, which can be converted\ninto corresponding Chinese characters, account for more than fifty of the\nentire Korean vocabulary. Motivated by this, we propose a simple linguistically\nmotivated solution to improve the performance of the Korean-to-Chinese neural\nmachine translation model by using their common vocabulary. We adopt Chinese\ncharacters as a translation pivot by converting Sino-Korean words in Korean\nsentences to Chinese characters and then train the machine translation model\nwith the converted Korean sentences as source sentences. The experimental\nresults on Korean-to-Chinese translation demonstrate that the models with the\nproposed method improve translation quality up to 1.5 BLEU points in comparison\nto the baseline models.",
            "journal ref": "33rd Pacific Asia Conference on Language, Information and\n  Computation (PACLIC 33), pages 558-566, Hakodate, Japan, September 13-15,\n  2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11161v1",
            "date": "2019-11-25 19:01:36+00:00",
            "title": "Emotional Neural Language Generation Grounded in Situational Contexts",
            "authors": [
                "Sashank Santhanam",
                "Samira Shaikh"
            ],
            "abstract": "Emotional language generation is one of the keys to human-like artificial\nintelligence. Humans use different type of emotions depending on the situation\nof the conversation. Emotions also play an important role in mediating the\nengagement level with conversational partners. However, current conversational\nagents do not effectively account for emotional content in the language\ngeneration process. To address this problem, we develop a language modeling\napproach that generates affective content when the dialogue is situated in a\ngiven context. We use the recently released Empathetic-Dialogues corpus to\nbuild our models. Through detailed experiments, we find that our approach\noutperforms the state-of-the-art method on the perplexity metric by about 5\npoints and achieves a higher BLEU metric score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11404v1",
            "date": "2019-11-26 08:46:11+00:00",
            "title": "Natural Language Generation Using Reinforcement Learning with External Rewards",
            "authors": [
                "Vidhushini Srinivasan",
                "Sashank Santhanam",
                "Samira Shaikh"
            ],
            "abstract": "We propose an approach towards natural language generation using a\nbidirectional encoder-decoder which incorporates external rewards through\nreinforcement learning (RL). We use attention mechanism and maximum mutual\ninformation as an initial objective function using RL. Using a two-part\ntraining scheme, we train an external reward analyzer to predict the external\nrewards and then use the predicted rewards to maximize the expected rewards\n(both internal and external). We evaluate the system on two standard dialogue\ncorpora - Cornell Movie Dialog Corpus and Yelp Restaurant Review Corpus. We\nreport standard evaluation metrics including BLEU, ROUGE-L, and perplexity as\nwell as human evaluation to validate our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11408v1",
            "date": "2019-11-26 08:53:12+00:00",
            "title": "A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis",
            "authors": [
                "Shai Gretz",
                "Roni Friedman",
                "Edo Cohen-Karlik",
                "Assaf Toledo",
                "Dan Lahav",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "Identifying the quality of free-text arguments has become an important task\nin the rapidly expanding field of computational argumentation. In this work, we\nexplore the challenging task of argument quality ranking. To this end, we\ncreated a corpus of 30,497 arguments carefully annotated for point-wise\nquality, released as part of this work. To the best of our knowledge, this is\nthe largest dataset annotated for point-wise argument quality, larger by a\nfactor of five than previously released datasets. Moreover, we address the core\nissue of inducing a labeled score from crowd annotations by performing a\ncomprehensive evaluation of different approaches to this problem. In addition,\nwe analyze the quality dimensions that characterize this dataset. Finally, we\npresent a neural method for argument quality ranking, which outperforms several\nbaselines on our own dataset, as well as previous methods published for another\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11489v1",
            "date": "2019-11-26 12:17:59+00:00",
            "title": "Relevance-Promoting Language Model for Short-Text Conversation",
            "authors": [
                "Xin Li",
                "Piji Li",
                "Wei Bi",
                "Xiaojiang Liu",
                "Wai Lam"
            ],
            "abstract": "Despite the effectiveness of sequence-to-sequence framework on the task of\nShort-Text Conversation (STC), the issue of under-exploitation of training data\n(i.e., the supervision signals from query text is \\textit{ignored}) still\nremains unresolved. Also, the adopted \\textit{maximization}-based decoding\nstrategies, inclined to generating the generic responses or responses with\nrepetition, are unsuited to the STC task. In this paper, we propose to\nformulate the STC task as a language modeling problem and tailor-make a\ntraining strategy to adapt a language model for response generation. To enhance\ngeneration performance, we design a relevance-promoting transformer language\nmodel, which performs additional supervised source attention after the\nself-attention to increase the importance of informative query tokens in\ncalculating the token-level representation. The model further refines the query\nrepresentation with relevance clues inferred from its multiple references\nduring training. In testing, we adopt a\n\\textit{randomization-over-maximization} strategy to reduce the generation of\ngeneric responses. Experimental results on a large Chinese STC dataset\ndemonstrate the superiority of the proposed model on relevance metrics and\ndiversity metrics.\\footnote{Code available at\nhttps://ai.tencent.com/ailab/nlp/dialogue/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11493v1",
            "date": "2019-11-26 12:29:19+00:00",
            "title": "Integrating Relation Constraints with Neural Relation Extractors",
            "authors": [
                "Yuan Ye",
                "Yansong Feng",
                "Bingfeng Luo",
                "Yuxuan Lai",
                "Dongyan Zhao"
            ],
            "abstract": "Recent years have seen rapid progress in identifying predefined relationship\nbetween entity pairs using neural networks NNs. However, such models often make\npredictions for each entity pair individually, thus often fail to solve the\ninconsistency among different predictions, which can be characterized by\ndiscrete relation constraints. These constraints are often defined over\ncombinations of entity-relation-entity triples, since there often lack of\nexplicitly well-defined type and cardinality requirements for the relations. In\nthis paper, we propose a unified framework to integrate relation constraints\nwith NNs by introducing a new loss term, ConstraintLoss. Particularly, we\ndevelop two efficient methods to capture how well the local predictions from\nmultiple instance pairs satisfy the relation constraints. Experiments on both\nEnglish and Chinese datasets show that our approach can help NNs learn from\ndiscrete relation constraints to reduce inconsistency among local predictions,\nand outperform popular neural relation extraction NRE models even enhanced with\nextra post-processing. Our source code and datasets will be released at\nhttps://github.com/PKUYeYuan/Constraint-Loss-AAAI-2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11520v3",
            "date": "2019-11-26 13:28:59+00:00",
            "title": "Neural Machine Translation with Explicit Phrase Alignment",
            "authors": [
                "Jiacheng Zhang",
                "Huanbo Luan",
                "Maosong Sun",
                "FeiFei Zhai",
                "Jingfang Xu",
                "Yang Liu"
            ],
            "abstract": "While neural machine translation (NMT) has achieved state-of-the-art\ntranslation performance, it is unable to capture the alignment between the\ninput and output during the translation process. The lack of alignment in NMT\nmodels leads to three problems: it is hard to (1) interpret the translation\nprocess, (2) impose lexical constraints, and (3) impose structural constraints.\nTo alleviate these problems, we propose to introduce explicit phrase alignment\ninto the translation process of arbitrary NMT models. The key idea is to build\na search space similar to that of phrase-based statistical machine translation\nfor NMT where phrase alignment is readily available. We design a new decoding\nalgorithm that can easily impose lexical and structural constraints.\nExperiments show that our approach makes the translation process of NMT more\ninterpretable without sacrificing translation quality. In addition, our\napproach achieves significant improvements in lexically and structurally\nconstrained translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11522v1",
            "date": "2019-11-26 13:31:08+00:00",
            "title": "A Time Series Analysis of Emotional Loading in Central Bank Statements",
            "authors": [
                "Sven Buechel",
                "Simon Junker",
                "Thore Schlaak",
                "Claus Michelsen",
                "Udo Hahn"
            ],
            "abstract": "We examine the affective content of central bank press statements using\nemotion analysis. Our focus is on two major international players, the European\nCentral Bank (ECB) and the US Federal Reserve Bank (Fed), covering a time span\nfrom 1998 through 2019. We reveal characteristic patterns in the emotional\ndimensions of valence, arousal, and dominance and find---despite the commonly\nestablished attitude that emotional wording in central bank communication\nshould be avoided---a correlation between the state of the economy and\nparticularly the dominance dimension in the press releases under scrutiny and,\noverall, an impact of the president in office.",
            "journal ref": "Proceedings of the Second Workshop on Economics and Natural\n  Language Processing @ EMNLP 2019, Hong Kong, November 4, 2019, pages 16-21",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11547v1",
            "date": "2019-11-26 14:11:50+00:00",
            "title": "A Vietnamese Text-Based Conversational Agent",
            "authors": [
                "Dai Quoc Nguyen",
                "Dat Quoc Nguyen",
                "Son Bao Pham"
            ],
            "abstract": "This paper introduces a Vietnamese text-based conversational agent\narchitecture on specific knowledge domain which is integrated in a question\nanswering system. When the question answering system fails to provide answers\nto users' input, our conversational agent can step in to interact with users to\nprovide answers to users. Experimental results are promising where our\nVietnamese text-based conversational agent achieves positive feedback in a\nstudy conducted in the university academic regulation domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11672v1",
            "date": "2019-11-26 16:12:36+00:00",
            "title": "Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling",
            "authors": [
                "Bo-Hsiang Tseng",
                "Marek Rei",
                "Pawe\u0142 Budzianowski",
                "Richard E. Turner",
                "Bill Byrne",
                "Anna Korhonen"
            ],
            "abstract": "Dialogue systems benefit greatly from optimizing on detailed annotations,\nsuch as transcribed utterances, internal dialogue state representations and\ndialogue act labels. However, collecting these annotations is expensive and\ntime-consuming, holding back development in the area of dialogue modelling. In\nthis paper, we investigate semi-supervised learning methods that are able to\nreduce the amount of required intermediate labelling. We find that by\nleveraging un-annotated data instead, the amount of turn-level annotations of\ndialogue state can be significantly reduced when building a neural dialogue\nsystem. Our analysis on the MultiWOZ corpus, covering a range of domains and\ntopics, finds that annotations can be reduced by up to 30\\% while maintaining\nequivalent system performance. We also describe and evaluate the first\nend-to-end dialogue model created for the MultiWOZ corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11899v1",
            "date": "2019-11-27 00:55:12+00:00",
            "title": "Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction",
            "authors": [
                "Yang Li",
                "Guodong Long",
                "Tao Shen",
                "Tianyi Zhou",
                "Lina Yao",
                "Huan Huo",
                "Jing Jiang"
            ],
            "abstract": "Distantly supervised relation extraction intrinsically suffers from noisy\nlabels due to the strong assumption of distant supervision. Most prior works\nadopt a selective attention mechanism over sentences in a bag to denoise from\nwrongly labeled data, which however could be incompetent when there is only one\nsentence in a bag. In this paper, we propose a brand-new light-weight neural\nframework to address the distantly supervised relation extraction problem and\nalleviate the defects in previous selective attention framework. Specifically,\nin the proposed framework, 1) we use an entity-aware word embedding method to\nintegrate both relative position information and head/tail entity embeddings,\naiming to highlight the essence of entities for this task; 2) we develop a\nself-attention mechanism to capture the rich contextual dependencies as a\ncomplement for local dependencies captured by piecewise CNN; and 3) instead of\nusing selective attention, we design a pooling-equipped gate, which is based on\nrich contextual representations, as an aggregator to generate bag-level\nrepresentation for final relation classification. Compared to selective\nattention, one major advantage of the proposed gating mechanism is that, it\nperforms stably and promisingly even if only one sentence appears in a bag and\nthus keeps the consistency across all training examples. The experiments on NYT\ndataset demonstrate that our approach achieves a new state-of-the-art\nperformance in terms of both AUC and top-n precision metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11933v1",
            "date": "2019-11-27 03:38:36+00:00",
            "title": "Simultaneous Neural Machine Translation using Connectionist Temporal Classification",
            "authors": [
                "Katsuki Chousa",
                "Katsuhito Sudoh",
                "Satoshi Nakamura"
            ],
            "abstract": "Simultaneous machine translation is a variant of machine translation that\nstarts the translation process before the end of an input. This task faces a\ntrade-off between translation accuracy and latency. We have to determine when\nwe start the translation for observed inputs so far, to achieve good practical\nperformance. In this work, we propose a neural machine translation method to\ndetermine this timing in an adaptive manner. The proposed method introduces a\nspecial token '<wait>', which is generated when the translation model chooses\nto read the next input token instead of generating an output token. It also\nintroduces an objective function to handle the ambiguity in wait timings that\ncan be optimized using an algorithm called Connectionist Temporal\nClassification (CTC). The use of CTC enables the optimization to consider all\npossible output sequences including '<wait>' that are equivalent to the\nreference translations and to choose the best one adaptively. We apply the\nproposed method into simultaneous translation from English to Japanese and\ninvestigate its performance and remaining problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11951v1",
            "date": "2019-11-27 04:52:53+00:00",
            "title": "Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection",
            "authors": [
                "Chris Dulhanty",
                "Jason L. Deglint",
                "Ibrahim Ben Daya",
                "Alexander Wong"
            ],
            "abstract": "The exponential rise of social media and digital news in the past decade has\nhad the unfortunate consequence of escalating what the United Nations has\ncalled a global topic of concern: the growing prevalence of disinformation.\nGiven the complexity and time-consuming nature of combating disinformation\nthrough human assessment, one is motivated to explore harnessing AI solutions\nto automatically assess news articles for the presence of disinformation. A\nvaluable first step towards automatic identification of disinformation is\nstance detection, where given a claim and a news article, the aim is to predict\nif the article agrees, disagrees, takes no position, or is unrelated to the\nclaim. Existing approaches in literature have largely relied on hand-engineered\nfeatures or shallow learned representations (e.g., word embeddings) to encode\nthe claim-article pairs, which can limit the level of representational\nexpressiveness needed to tackle the high complexity of disinformation\nidentification. In this work, we explore the notion of harnessing large-scale\ndeep bidirectional transformer language models for encoding claim-article pairs\nin an effort to construct state-of-the-art stance detection geared for\nidentifying disinformation. Taking advantage of bidirectional cross-attention\nbetween claim-article pairs via pair encoding with self-attention, we construct\na large-scale language model for stance detection by performing transfer\nlearning on a RoBERTa deep bidirectional transformer language model, and were\nable to achieve state-of-the-art performance (weighted accuracy of 90.01%) on\nthe Fake News Challenge Stage 1 (FNC-I) benchmark. These promising results\nserve as motivation for harnessing such large-scale language models as powerful\nbuilding blocks for creating effective AI solutions to combat disinformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12011v1",
            "date": "2019-11-27 08:13:27+00:00",
            "title": "JEC-QA: A Legal-Domain Question Answering Dataset",
            "authors": [
                "Haoxi Zhong",
                "Chaojun Xiao",
                "Cunchao Tu",
                "Tianyang Zhang",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "We present JEC-QA, the largest question answering dataset in the legal\ndomain, collected from the National Judicial Examination of China. The\nexamination is a comprehensive evaluation of professional skills for legal\npractitioners. College students are required to pass the examination to be\ncertified as a lawyer or a judge. The dataset is challenging for existing\nquestion answering methods, because both retrieving relevant materials and\nanswering questions require the ability of logic reasoning. Due to the high\ndemand of multiple reasoning abilities to answer legal questions, the\nstate-of-the-art models can only achieve about 28% accuracy on JEC-QA, while\nskilled humans and unskilled humans can reach 81% and 64% accuracy\nrespectively, which indicates a huge gap between humans and machines on this\ntask. We will release JEC-QA and our baselines to help improve the reasoning\nability of machine comprehension models. You can access the dataset from\nhttp://jecqa.thunlp.org/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12014v1",
            "date": "2019-11-27 08:17:45+00:00",
            "title": "Zero-shot Chinese Discourse Dependency Parsing via Cross-lingual Mapping",
            "authors": [
                "Yi Cheng",
                "Sujian Li"
            ],
            "abstract": "Due to the absence of labeled data, discourse parsing still remains\nchallenging in some languages. In this paper, we present a simple and efficient\nmethod to conduct zero-shot Chinese text-level dependency parsing by leveraging\nEnglish discourse labeled data and parsing techniques. We first construct the\nChinese-English mapping from the level of sentence and elementary discourse\nunit (EDU), and then exploit the parsing results of the corresponding English\ntranslations to obtain the discourse trees for the Chinese text. This method\ncan automatically conduct Chinese discourse parsing, with no need of a large\nscale of Chinese labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12019v1",
            "date": "2019-11-27 08:37:32+00:00",
            "title": "word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairs",
            "authors": [
                "Yo Joong Choe",
                "Kyubyong Park",
                "Dongwoo Kim"
            ],
            "abstract": "We present word2word, a publicly available dataset and an open-source Python\npackage for cross-lingual word translations extracted from sentence-level\nparallel corpora. Our dataset provides top-k word translations in 3,564\n(directed) language pairs across 62 languages in OpenSubtitles2018 (Lison et\nal., 2018). To obtain this dataset, we use a count-based bilingual lexicon\nextraction model based on the observation that not only source and target words\nbut also source words themselves can be highly correlated. We illustrate that\nthe resulting bilingual lexicons have high coverage and attain competitive\ntranslation quality for several language pairs. We wrap our dataset and model\nin an easy-to-use Python library, which supports downloading and retrieving\ntop-k word translations in any of the supported language pairs as well as\ncomputing top-k word translations for custom parallel corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12022v1",
            "date": "2019-11-27 08:38:57+00:00",
            "title": "Sideways Transliteration: How to Transliterate Multicultural Person Names?",
            "authors": [
                "Raphael Cohen",
                "Michael Elhadad"
            ],
            "abstract": "In a global setting, texts contain transliterated names from many cultural\norigins. Correct transliteration depends not only on target and source\nlanguages but also, on the source language of the name. We introduce a novel\nmethodology for transliteration of names originating in different languages\nusing only monolingual resources. Our method is based on a step of noisy\ntransliteration and then ranking of the results based on origin specific letter\nmodels. The transliteration table used for noisy generation is learned in an\nunsupervised manner for each possible origin language. We present a solution\nfor gathering monolingual training data used by our method by mining of social\nmedia sites such as Facebook and Wikipedia. We present results in the context\nof transliterating from English to Hebrew and provide an online web service for\ntransliteration from English to Hebrew",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12071v1",
            "date": "2019-11-27 10:43:20+00:00",
            "title": "Jejueo Datasets for Machine Translation and Speech Synthesis",
            "authors": [
                "Kyubyong Park",
                "Yo Joong Choe",
                "Jiyeon Ham"
            ],
            "abstract": "Jejueo was classified as critically endangered by UNESCO in 2010. Although\ndiverse efforts to revitalize it have been made, there have been few\ncomputational approaches. Motivated by this, we construct two new Jejueo\ndatasets: Jejueo Interview Transcripts (JIT) and Jejueo Single Speaker Speech\n(JSS). The JIT dataset is a parallel corpus containing 170k+ Jejueo-Korean\nsentences, and the JSS dataset consists of 10k high-quality audio files\nrecorded by a native Jejueo speaker and a transcript file. Subsequently, we\nbuild neural systems of machine translation and speech synthesis using them.\nAll resources are publicly available via our GitHub repository. We hope that\nthese datasets will attract interest of both language and machine learning\ncommunities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12146v2",
            "date": "2019-11-27 13:30:36+00:00",
            "title": "NorNE: Annotating Named Entities for Norwegian",
            "authors": [
                "Fredrik J\u00f8rgensen",
                "Tobias Aasmoe",
                "Anne-Stine Ruud Husev\u00e5g",
                "Lilja \u00d8vrelid",
                "Erik Velldal"
            ],
            "abstract": "This paper presents NorNE, a manually annotated corpus of named entities\nwhich extends the annotation of the existing Norwegian Dependency Treebank.\nComprising both of the official standards of written Norwegian (Bokm{\\aa}l and\nNynorsk), the corpus contains around 600,000 tokens and annotates a rich set of\nentity types including persons, organizations, locations, geo-political\nentities, products, and events, in addition to a class corresponding to\nnominals derived from names. We here present details on the annotation effort,\nguidelines, inter-annotator agreement and an experimental analysis of the\ncorpus using a neural sequence labeling architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12237v2",
            "date": "2019-11-27 15:54:55+00:00",
            "title": "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization",
            "authors": [
                "Bogdan Gliwa",
                "Iwona Mochol",
                "Maciej Biesek",
                "Aleksander Wawer"
            ],
            "abstract": "This paper introduces the SAMSum Corpus, a new dataset with abstractive\ndialogue summaries. We investigate the challenges it poses for automated\nsummarization by testing several models and comparing their results with those\nobtained on a corpus of news articles. We show that model-generated summaries\nof dialogues achieve higher ROUGE scores than the model-generated summaries of\nnews -- in contrast with human evaluators' judgement. This suggests that a\nchallenging task of abstractive dialogue summarization requires dedicated\nmodels and non-standard quality measures. To our knowledge, our study is the\nfirst attempt to introduce a high-quality chat-dialogues corpus, manually\nannotated with abstractive summarizations, which can be used by the research\ncommunity for further studies.",
            "journal ref": "Proceedings of the 2nd Workshop on New Frontiers in Summarization,\n  Association for Computational Linguistics. November 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12246v1",
            "date": "2019-11-27 16:09:11+00:00",
            "title": "Do Attention Heads in BERT Track Syntactic Dependencies?",
            "authors": [
                "Phu Mon Htut",
                "Jason Phang",
                "Shikha Bordia",
                "Samuel R. Bowman"
            ],
            "abstract": "We investigate the extent to which individual attention heads in pretrained\ntransformer language models, such as BERT and RoBERTa, implicitly capture\nsyntactic dependency relations. We employ two methods---taking the maximum\nattention weight and computing the maximum spanning tree---to extract implicit\ndependency relations from the attention weights of each layer/head, and compare\nthem to the ground-truth Universal Dependency (UD) trees. We show that, for\nsome UD relation types, there exist heads that can recover the dependency type\nsignificantly better than baselines on parsed English text, suggesting that\nsome self-attention heads act as a proxy for syntactic structure. We also\nanalyze BERT fine-tuned on two datasets---the syntax-oriented CoLA and the\nsemantics-oriented MNLI---to investigate whether fine-tuning affects the\npatterns of their self-attention, but we do not observe substantial differences\nin the overall dependency relations extracted using our methods. Our results\nsuggest that these models have some specialist attention heads that track\nindividual dependency types, but no generalist head that performs holistic\nparsing significantly better than a trivial baseline, and that analyzing\nattention weights directly may not reveal much of the syntactic knowledge that\nBERT-style models are known to learn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12267v1",
            "date": "2019-11-26 11:44:55+00:00",
            "title": "A Vietnamese Question Answering System",
            "authors": [
                "Dai Quoc Nguyen",
                "Dat Quoc Nguyen",
                "Son Bao Pham"
            ],
            "abstract": "Question answering systems aim to produce exact answers to users' questions\ninstead of a list of related documents as used by current search engines. In\nthis paper, we propose an ontology-based Vietnamese question answering system\nthat allows users to express their questions in natural language. To the best\nof our knowledge, this is the first attempt to enable users to query an\nontological knowledge base using Vietnamese natural language. Experiments of\nour system on an organizational ontology show promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12391v1",
            "date": "2019-11-27 19:37:26+00:00",
            "title": "SimpleBooks: Long-term dependency book dataset with simplified English vocabulary for word-level language modeling",
            "authors": [
                "Huyen Nguyen"
            ],
            "abstract": "With language modeling becoming the popular base task for unsupervised\nrepresentation learning in Natural Language Processing, it is important to come\nup with new architectures and techniques for faster and better training of\nlanguage models. However, due to a peculiarity of languages -- the larger the\ndataset, the higher the average number of times a word appears in that dataset\n-- datasets of different sizes have very different properties. Architectures\nperforming well on small datasets might not perform well on larger ones. For\nexample, LSTM models perform well on WikiText-2 but poorly on WikiText-103,\nwhile Transformer models perform well on WikiText-103 but not on WikiText-2.\nFor setups like architectural search, this is a challenge since it is\nprohibitively costly to run a search on the full dataset but it is not\nindicative to experiment on smaller ones. In this paper, we introduce\nSimpleBooks, a small dataset with the average word frequency as high as that of\nmuch larger ones. Created from 1,573 Gutenberg books with the highest ratio of\nword-level book length to vocabulary size, SimpleBooks contains 92M word-level\ntokens, on par with WikiText-103 (103M tokens), but has the vocabulary of 98K,\na third of WikiText-103's. SimpleBooks can be downloaded from\nhttps://dldata-public.s3.us-east-2.amazonaws.com/simplebooks.zip.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12556v1",
            "date": "2019-11-28 06:53:14+00:00",
            "title": "Improving Neural Relation Extraction with Positive and Unlabeled Learning",
            "authors": [
                "Zhengqiu He",
                "Wenliang Chen",
                "Yuyi Wang",
                "Wei zhang",
                "Guanchun Wang",
                "Min Zhang"
            ],
            "abstract": "We present a novel approach to improve the performance of distant supervision\nrelation extraction with Positive and Unlabeled (PU) Learning. This approach\nfirst applies reinforcement learning to decide whether a sentence is positive\nto a given relation, and then positive and unlabeled bags are constructed. In\ncontrast to most previous studies, which mainly use selected positive instances\nonly, we make full use of unlabeled instances and propose two new\nrepresentations for positive and unlabeled bags. These two representations are\nthen combined in an appropriate way to make bag-level prediction. Experimental\nresults on a widely used real-world dataset demonstrate that this new approach\nindeed achieves significant and consistent improvements as compared to several\ncompetitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12569v1",
            "date": "2019-11-28 07:43:04+00:00",
            "title": "Emotion helps Sentiment: A Multi-task Model for Sentiment and Emotion Analysis",
            "authors": [
                "Abhishek Kumar",
                "Asif Ekbal",
                "Daisuke Kawahra",
                "Sadao Kurohashi"
            ],
            "abstract": "In this paper, we propose a two-layered multi-task attention based neural\nnetwork that performs sentiment analysis through emotion analysis. The proposed\napproach is based on Bidirectional Long Short-Term Memory and uses\nDistributional Thesaurus as a source of external knowledge to improve the\nsentiment and emotion prediction. The proposed system has two levels of\nattention to hierarchically build a meaningful representation. We evaluate our\nsystem on the benchmark dataset of SemEval 2016 Task 6 and also compare it with\nthe state-of-the-art systems on Stance Sentiment Emotion Corpus. Experimental\nresults show that the proposed system improves the performance of sentiment\nanalysis by 3.2 F-score points on SemEval 2016 Task 6 dataset. Our network also\nboosts the performance of emotion analysis by 5 F-score points on Stance\nSentiment Emotion Corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12579v3",
            "date": "2019-11-28 08:11:44+00:00",
            "title": "Word Embedding based New Corpus for Low-resourced Language: Sindhi",
            "authors": [
                "Wazir Ali",
                "Jay Kumar",
                "Junyu Lu",
                "Zenglin Xu"
            ],
            "abstract": "Representing words and phrases into dense vectors of real numbers which\nencode semantic and syntactic properties is a vital constituent in natural\nlanguage processing (NLP). The success of neural network (NN) models in NLP\nlargely rely on such dense word representations learned on the large unlabeled\ncorpus. Sindhi is one of the rich morphological language, spoken by large\npopulation in Pakistan and India lacks corpora which plays an essential role of\na test-bed for generating word embeddings and developing language independent\nNLP systems. In this paper, a large corpus of more than 61 million words is\ndeveloped for low-resourced Sindhi language for training neural word\nembeddings. The corpus is acquired from multiple web-resources using\nweb-scrappy. Due to the unavailability of open source preprocessing tools for\nSindhi, the prepossessing of such large corpus becomes a challenging problem\nspecially cleaning of noisy data extracted from web resources. Therefore, a\npreprocessing pipeline is employed for the filtration of noisy text.\nAfterwards, the cleaned vocabulary is utilized for training Sindhi word\nembeddings with state-of-the-art GloVe, Skip-Gram (SG), and Continuous Bag of\nWords (CBoW) word2vec algorithms. The intrinsic evaluation approach of cosine\nsimilarity matrix and WordSim-353 are employed for the evaluation of generated\nSindhi word embeddings. Moreover, we compare the proposed word embeddings with\nrecently revealed Sindhi fastText (SdfastText) word representations. Our\nintrinsic evaluation results demonstrate the high quality of our generated\nSindhi word embeddings using SG, CBoW, and GloVe as compare to SdfastText word\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12722v2",
            "date": "2019-11-28 14:09:50+00:00",
            "title": "A Fine-Grained Sentiment Dataset for Norwegian",
            "authors": [
                "Lilja \u00d8vrelid",
                "Petter M\u00e6hlum",
                "Jeremy Barnes",
                "Erik Velldal"
            ],
            "abstract": "We introduce NoReC_fine, a dataset for fine-grained sentiment analysis in\nNorwegian, annotated with respect to polar expressions, targets and holders of\nopinion. The underlying texts are taken from a corpus of professionally\nauthored reviews from multiple news-sources and across a wide variety of\ndomains, including literature, games, music, products, movies and more. We here\npresent a detailed description of this annotation effort. We provide an\noverview of the developed annotation guidelines, illustrated with examples, and\npresent an analysis of inter-annotator agreement. We also report the first\nexperimental results on the dataset, intended as a preliminary benchmark for\nfurther experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12798v1",
            "date": "2019-11-28 17:18:41+00:00",
            "title": "Multimodal Machine Translation through Visuals and Speech",
            "authors": [
                "Umut Sulubacak",
                "Ozan Caglayan",
                "Stig-Arne Gr\u00f6nroos",
                "Aku Rouhe",
                "Desmond Elliott",
                "Lucia Specia",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Multimodal machine translation involves drawing information from more than\none modality, based on the assumption that the additional modalities will\ncontain useful alternative views of the input data. The most prominent tasks in\nthis area are spoken language translation, image-guided translation, and\nvideo-guided translation, which exploit audio and visual modalities,\nrespectively. These tasks are distinguished from their monolingual counterparts\nof speech recognition, image captioning, and video captioning by the\nrequirement of models to generate outputs in a different language. This survey\nreviews the major data resources for these tasks, the evaluation campaigns\nconcentrated around them, the state of the art in end-to-end and pipeline\napproaches, and also the challenges in performance evaluation. The paper\nconcludes with a discussion of directions for future research in these areas:\nthe need for more expansive and challenging datasets, for targeted evaluations\nof model performance, and for multimodality in both the input and output space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12893v1",
            "date": "2019-11-28 22:57:45+00:00",
            "title": "GitHub Typo Corpus: A Large-Scale Multilingual Dataset of Misspellings and Grammatical Errors",
            "authors": [
                "Masato Hagiwara",
                "Masato Mita"
            ],
            "abstract": "The lack of large-scale datasets has been a major hindrance to the\ndevelopment of NLP tasks such as spelling correction and grammatical error\ncorrection (GEC). As a complementary new resource for these tasks, we present\nthe GitHub Typo Corpus, a large-scale, multilingual dataset of misspellings and\ngrammatical errors along with their corrections harvested from GitHub, a large\nand popular platform for hosting and sharing git repositories. The dataset,\nwhich we have made publicly available, contains more than 350k edits and 65M\ncharacters in more than 15 languages, making it the largest dataset of\nmisspellings to date. We also describe our process for filtering true typo\nedits based on learned classifiers on a small annotated subset, and demonstrate\nthat typo edits can be identified with F1 ~ 0.9 using a very simple classifier\nwith only three features. The detailed analyses of the dataset show that\nexisting spelling correctors merely achieve an F-measure of approx. 0.5,\nsuggesting that the dataset serves as a new, rich source of spelling errors\nthat complement existing datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12982v1",
            "date": "2019-11-29 07:22:01+00:00",
            "title": "Neural Chinese Word Segmentation as Sequence to Sequence Translation",
            "authors": [
                "Xuewen Shi",
                "Heyan Huang",
                "Ping Jian",
                "Yuhang Guo",
                "Xiaochi Wei",
                "Yi-Kun Tang"
            ],
            "abstract": "Recently, Chinese word segmentation (CWS) methods using neural networks have\nmade impressive progress. Most of them regard the CWS as a sequence labeling\nproblem which construct models based on local features rather than considering\nglobal information of input sequence. In this paper, we cast the CWS as a\nsequence translation problem and propose a novel sequence-to-sequence CWS model\nwith an attention-based encoder-decoder framework. The model captures the\nglobal information from the input and directly outputs the segmented sequence.\nIt can also tackle other NLP tasks with CWS jointly in an end-to-end mode.\nExperiments on Weibo, PKU and MSRA benchmark datasets show that our approach\nhas achieved competitive performances compared with state-of-the-art methods.\nMeanwhile, we successfully applied our proposed model to jointly learning CWS\nand Chinese spelling correction, which demonstrates its applicability of\nmulti-task fusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13062v1",
            "date": "2019-11-29 11:24:10+00:00",
            "title": "Sentiment Analysis of German Twitter",
            "authors": [
                "Wladimir Sidorenko"
            ],
            "abstract": "This thesis explores the ways by how people express their opinions on German\nTwitter, examines current approaches to automatic mining of these feelings, and\nproposes novel methods, which outperform state-of-the-art techniques. For this\npurpose, I introduce a new corpus of German tweets that have been manually\nannotated with sentiments, their targets and holders, as well as polar terms\nand their contextual modifiers. Using these data, I explore four major areas of\nsentiment research: (i) generation of sentiment lexicons, (ii) fine-grained\nopinion mining, (iii) message-level polarity classification, and (iv)\ndiscourse-aware sentiment analysis. In the first task, I compare three popular\ngroups of lexicon generation methods: dictionary-, corpus-, and\nword-embedding-based ones, finding that dictionary-based systems generally\nyield better lexicons than the last two groups. Apart from this, I propose a\nlinear projection algorithm, whose results surpass many existing automatic\nlexicons. Afterwords, in the second task, I examine two common approaches to\nautomatic prediction of sentiments, sources, and targets: conditional random\nfields and recurrent neural networks, obtaining higher scores with the former\nmodel and improving these results even further by redefining the structure of\nCRF graphs. When dealing with message-level polarity classification, I\njuxtapose three major sentiment paradigms: lexicon-, machine-learning-, and\ndeep-learning-based systems, and try to unite the first and last of these\ngroups by introducing a bidirectional neural network with lexicon-based\nattention. Finally, in order to make the new classifier aware of discourse\nstructure, I let it separately analyze the elementary discourse units of each\nmicroblog and infer the overall polarity of a message from the scores of its\nEDUs with the help of two new approaches: latent-marginalized CRFs and\nRecursive Dirichlet Process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13066v1",
            "date": "2019-11-29 11:35:13+00:00",
            "title": "A Multi-cascaded Deep Model for Bilingual SMS Classification",
            "authors": [
                "Muhammad Haroon Shakeel",
                "Asim Karim",
                "Imdadullah Khan"
            ],
            "abstract": "Most studies on text classification are focused on the English language.\nHowever, short texts such as SMS are influenced by regional languages. This\nmakes the automatic text classification task challenging due to the\nmultilingual, informal, and noisy nature of language in the text. In this work,\nwe propose a novel multi-cascaded deep learning model called McM for bilingual\nSMS classification. McM exploits $n$-gram level information as well as\nlong-term dependencies of text for learning. Our approach aims to learn a model\nwithout any code-switching indication, lexical normalization, language\ntranslation, or language transliteration. The model relies entirely upon the\ntext as no external knowledge base is utilized for learning. For this purpose,\na 12 class bilingual text dataset is developed from SMS feedbacks of citizens\non public services containing mixed Roman Urdu and English languages. Our model\nachieves high accuracy for classification on this dataset and outperforms the\nprevious model for multilingual text classification, highlighting language\nindependence of McM.",
            "journal ref": "Neural Information Processing. ICONIP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00113v1",
            "date": "2019-11-30 02:12:31+00:00",
            "title": "Tag Recommendation by Word-Level Tag Sequence Modeling",
            "authors": [
                "Xuewen Shi",
                "Heyan Huang",
                "Shuyang Zhao",
                "Ping Jian",
                "Yi-Kun Tang"
            ],
            "abstract": "In this paper, we transform tag recommendation into a word-based text\ngeneration problem and introduce a sequence-to-sequence model. The model\ninherits the advantages of LSTM-based encoder for sequential modeling and\nattention-based decoder with local positional encodings for learning relations\nglobally. Experimental results on Zhihu datasets illustrate the proposed model\noutperforms other state-of-the-art text classification based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00159v3",
            "date": "2019-11-30 08:42:25+00:00",
            "title": "Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss German",
            "authors": [
                "Lucy Linder",
                "Michael Jungo",
                "Jean Hennebert",
                "Claudiu Musat",
                "Andreas Fischer"
            ],
            "abstract": "This paper presents SwissCrawl, the largest Swiss German text corpus to date.\nComposed of more than half a million sentences, it was generated using a\ncustomized web scraping tool that could be applied to other low-resource\nlanguages as well. The approach demonstrates how freely available web pages can\nbe used to construct comprehensive text corpora, which are of fundamental\nimportance for natural language processing. In an experimental evaluation, we\nshow that using the new corpus leads to significant improvements for the task\nof language modeling. To capture new content, our approach will run\ncontinuously to keep increasing the corpus over time.",
            "journal ref": "Proceedings of The 12th Language Resources and Evaluation\n  Conference, LREC (2020) 2706-2711",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00239v2",
            "date": "2019-11-30 17:11:07+00:00",
            "title": "Neural language modeling of free word order argument structure",
            "authors": [
                "Charlotte Rochereau",
                "Beno\u00eet Sagot",
                "Emmanuel Dupoux"
            ],
            "abstract": "Neural language models trained with a predictive or masked objective have\nproven successful at capturing short and long distance syntactic dependencies.\nHere, we focus on verb argument structure in German, which has the interesting\nproperty that verb arguments may appear in a relatively free order in\nsubordinate clauses. Therefore, checking that the verb argument structure is\ncorrect cannot be done in a strictly sequential fashion, but rather requires to\nkeep track of the arguments' cases irrespective of their orders. We introduce a\nnew probing methodology based on minimal variation sets and show that both\nTransformers and LSTM achieve a score substantially better than chance on this\ntest. As humans, they also show graded judgments preferring canonical word\norders and plausible case assignments. However, we also found unexpected\ndiscrepancies in the strength of these effects, the LSTMs having difficulties\nrejecting ungrammatical sentences containing frequent argument structure types\n(double nominatives), and the Transformers tending to overgeneralize, accepting\nsome infrequent word orders or implausible sentences that humans barely accept.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00342v2",
            "date": "2019-12-01 07:08:19+00:00",
            "title": "Machines Getting with the Program: Understanding Intent Arguments of Non-Canonical Directives",
            "authors": [
                "Won Ik Cho",
                "Young Ki Moon",
                "Sangwhan Moon",
                "Seok Min Kim",
                "Nam Soo Kim"
            ],
            "abstract": "Modern dialog managers face the challenge of having to fulfill human-level\nconversational skills as part of common user expectations, including but not\nlimited to discourse with no clear objective. Along with these requirements,\nagents are expected to extrapolate intent from the user's dialogue even when\nsubjected to non-canonical forms of speech. This depends on the agent's\ncomprehension of paraphrased forms of such utterances. Especially in\nlow-resource languages, the lack of data is a bottleneck that prevents\nadvancements of the comprehension performance for these types of agents. In\nthis regard, here we demonstrate the necessity of extracting the intent\nargument of non-canonical directives in a natural language format, which may\nyield more accurate parsing, and suggest guidelines for building a parallel\ncorpus for this purpose. Following the guidelines, we construct a Korean corpus\nof 50K instances of question/command-intent pairs, including the labels for\nclassification of the utterance type. We also propose a method for mitigating\nclass imbalance, demonstrating the potential applications of the corpus\ngeneration method and its multilingual extensibility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00380v2",
            "date": "2019-12-01 10:41:42+00:00",
            "title": "HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation",
            "authors": [
                "Yiru Wang",
                "Pengda Si",
                "Zeyang Lei",
                "Guangxu Xun",
                "Yujiu Yang"
            ],
            "abstract": "The sequence-to-sequence (Seq2Seq) model generates target words iteratively\ngiven the previously observed words during decoding process, which results in\nthe loss of the holistic semantics in the target response and the complete\nsemantic relationship between responses and dialogue histories. In this paper,\nwe propose a generic diversity-promoting joint network, called Holistic\nSemantic Constraint Joint Network (HSCJN), enhancing the global sentence\ninformation, and then regularizing the objective function with penalizing the\nlow entropy output. Our network introduces more target information to improve\ndiversity, and captures direct semantic information to better constrain the\nrelevance simultaneously. Moreover, the proposed method can be easily applied\nto any Seq2Seq structure. Extensive experiments on several dialogue corpuses\nshow that our method effectively improves both semantic consistency and\ndiversity of generated responses, and achieves better performance than other\ncompetitive methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00567v1",
            "date": "2019-12-02 03:05:50+00:00",
            "title": "Merging External Bilingual Pairs into Neural Machine Translation",
            "authors": [
                "Tao Wang",
                "Shaohui Kuang",
                "Deyi Xiong",
                "Ant\u00f3nio Branco"
            ],
            "abstract": "As neural machine translation (NMT) is not easily amenable to explicit\ncorrection of errors, incorporating pre-specified translations into NMT is\nwidely regarded as a non-trivial challenge. In this paper, we propose and\nexplore three methods to endow NMT with pre-specified bilingual pairs. Instead,\nfor instance, of modifying the beam search algorithm during decoding or making\ncomplex modifications to the attention mechanism --- mainstream approaches to\ntackling this challenge ---, we experiment with the training data being\nappropriately pre-processed to add information about pre-specified\ntranslations. Extra embeddings are also used to distinguish pre-specified\ntokens from the other tokens. Extensive experimentation and analysis indicate\nthat over 99% of the pre-specified phrases are successfully translated (given a\n85% baseline) and that there is also a substantive improvement in translation\nquality with the methods explored here.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00582v3",
            "date": "2019-12-02 05:42:41+00:00",
            "title": "BLiMP: The Benchmark of Linguistic Minimal Pairs for English",
            "authors": [
                "Alex Warstadt",
                "Alicia Parrish",
                "Haokun Liu",
                "Anhad Mohananey",
                "Wei Peng",
                "Sheng-Fu Wang",
                "Samuel R. Bowman"
            ],
            "abstract": "We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLiMP),\na challenge set for evaluating what language models (LMs) know about major\ngrammatical phenomena in English. BLiMP consists of 67 sub-datasets, each\ncontaining 1000 minimal pairs isolating specific contrasts in syntax,\nmorphology, or semantics. The data is automatically generated according to\nexpert-crafted grammars, and aggregate human agreement with the labels is\n96.4%. We use it to evaluate n-gram, LSTM, and Transformer (GPT-2 and\nTransformer-XL) LMs. We find that state-of-the-art models identify\nmorphological contrasts reliably, but they struggle with semantic restrictions\non the distribution of quantifiers and negative polarity items and subtle\nsyntactic phenomena such as extraction islands.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00698v3",
            "date": "2019-12-02 11:51:57+00:00",
            "title": "Fiction Sentence Expansion and Enhancement via Focused Objective and Novelty Curve Sampling",
            "authors": [
                "Yuri Safovich",
                "Amos Azaria"
            ],
            "abstract": "We describe the task of sentence expansion and enhancement, in which a\nsentence provided by a human is expanded in some creative way. The expansion\nshould be understandable, believably grammatical, and optimally\nmeaning-preserving. Sentence expansion and enhancement may serve as an\nauthoring tool, or integrate in dynamic media, conversational agents, or\nvariegated advertising.\n  We implement a neural sentence expander trained on sentence compressions\ngenerated from a corpus of modern fiction. We modify an MLE objective to\nsupport the task by focusing on new words, and decode at test time with\ncontrolled curve-like novelty sampling. We run our sentence expander on\nsentences provided by human subjects and have humans evaluate these expansions.\nWe show that, although the generation methods are inferior to professional\nhuman writers, they are comparable to, and as well liked as, our subjects'\noriginal input sentences, and preferred over baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00839v1",
            "date": "2019-11-27 20:37:26+00:00",
            "title": "Automatic Generation of Headlines for Online Math Questions",
            "authors": [
                "Ke Yuan",
                "Dafang He",
                "Zhuoren Jiang",
                "Liangcai Gao",
                "Zhi Tang",
                "C. Lee Giles"
            ],
            "abstract": "Mathematical equations are an important part of dissemination and\ncommunication of scientific information. Students, however, often feel\nchallenged in reading and understanding math content and equations. With the\ndevelopment of the Web, students are posting their math questions online.\nNevertheless, constructing a concise math headline that gives a good\ndescription of the posted detailed math question is nontrivial. In this study,\nwe explore a novel summarization task denoted as geNerating A concise Math\nhEadline from a detailed math question (NAME). Compared to conventional\nsummarization tasks, this task has two extra and essential constraints: 1)\nDetailed math questions consist of text and math equations which require a\nunified framework to jointly model textual and mathematical information; 2)\nUnlike text, math equations contain semantic and structural features, and both\nof them should be captured together. To address these issues, we propose\nMathSum, a novel summarization model which utilizes a pointer mechanism\ncombined with a multi-head attention mechanism for mathematical representation\naugmentation. The pointer mechanism can either copy textual tokens or math\ntokens from source questions in order to generate math headlines. The\nmulti-head attention mechanism is designed to enrich the representation of math\nequations by modeling and integrating both its semantic and structural\nfeatures. For evaluation, we collect and make available two sets of real-world\ndetailed math questions along with human-written math headlines, namely\nEXEQ-300k and OFEQ-10k. Experimental results demonstrate that our model\n(MathSum) significantly outperforms state-of-the-art models for both the\nEXEQ-300k and OFEQ-10k datasets.",
            "journal ref": "AAA2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00991v1",
            "date": "2019-12-02 18:50:37+00:00",
            "title": "Morphological Tagging and Lemmatization of Albanian: A Manually Annotated Corpus and Neural Models",
            "authors": [
                "Nelda Kote",
                "Marenglen Biba",
                "Jenna Kanerva",
                "Samuel R\u00f6nnqvist",
                "Filip Ginter"
            ],
            "abstract": "In this paper, we present the first publicly available part-of-speech and\nmorphologically tagged corpus for the Albanian language, as well as a neural\nmorphological tagger and lemmatizer trained on it. There is currently a lack of\navailable NLP resources for Albanian, and its complex grammar and morphology\npresent challenges to their development. We have created an Albanian\npart-of-speech corpus based on the Universal Dependencies schema for\nmorphological annotation, containing about 118,000 tokens of naturally occuring\ntext collected from different text sources, with an addition of 67,000 tokens\nof artificially created simple sentences used only in training. On this corpus,\nwe subsequently train and evaluate segmentation, morphological tagging and\nlemmatization models, using the Turku Neural Parser Pipeline. On the held-out\nevaluation set, the model achieves 92.74% accuracy on part-of-speech tagging,\n85.31% on morphological tagging, and 89.95% on lemmatization. The manually\nannotated corpus, as well as the trained models are available under an open\nlicense.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01046v2",
            "date": "2019-12-02 19:17:57+00:00",
            "title": "TutorialVQA: Question Answering Dataset for Tutorial Videos",
            "authors": [
                "Anthony Colas",
                "Seokhwan Kim",
                "Franck Dernoncourt",
                "Siddhesh Gupte",
                "Daisy Zhe Wang",
                "Doo Soon Kim"
            ],
            "abstract": "Despite the number of currently available datasets on video question\nanswering, there still remains a need for a dataset involving multi-step and\nnon-factoid answers. Moreover, relying on video transcripts remains an\nunder-explored topic. To adequately address this, We propose a new question\nanswering task on instructional videos, because of their verbose and narrative\nnature. While previous studies on video question answering have focused on\ngenerating a short text as an answer, given a question and video clip, our task\naims to identify a span of a video segment as an answer which contains\ninstructional details with various granularities. This work focuses on\nscreencast tutorial videos pertaining to an image editing program. We introduce\na dataset, TutorialVQA, consisting of about 6,000manually collected triples of\n(video, question, answer span). We also provide experimental results with\nseveral baselines algorithms using the video transcripts. The results indicate\nthat the task is challenging and call for the investigation of new algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01072v2",
            "date": "2019-12-02 20:52:18+00:00",
            "title": "Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift",
            "authors": [
                "Matej Martinc",
                "Petra Kralj Novak",
                "Senja Pollak"
            ],
            "abstract": "We propose a new method that leverages contextual embeddings for the task of\ndiachronic semantic shift detection by generating time specific word\nrepresentations from BERT embeddings. The results of our experiments in the\ndomain specific LiverpoolFC corpus suggest that the proposed method has\nperformance comparable to the current state-of-the-art without requiring any\ntime consuming domain adaptation on large corpora. The results on the newly\ncreated Brexit news corpus suggest that the method can be successfully used for\nthe detection of a short-term yearly semantic shift. And lastly, the model also\nshows promising results in a multilingual settings, where the task was to\ndetect differences and similarities between diachronic semantic shifts in\ndifferent languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01114v1",
            "date": "2019-11-25 06:42:02+00:00",
            "title": "Importance-Aware Learning for Neural Headline Editing",
            "authors": [
                "Qingyang Wu",
                "Lei Li",
                "Hao Zhou",
                "Ying Zeng",
                "Zhou Yu"
            ],
            "abstract": "Many social media news writers are not professionally trained. Therefore,\nsocial media platforms have to hire professional editors to adjust amateur\nheadlines to attract more readers. We propose to automate this headline editing\nprocess through neural network models to provide more immediate writing support\nfor these social media news writers. To train such a neural headline editing\nmodel, we collected a dataset which contains articles with original headlines\nand professionally edited headlines. However, it is expensive to collect a\nlarge number of professionally edited headlines. To solve this low-resource\nproblem, we design an encoder-decoder model which leverages large scale\npre-trained language models. We further improve the pre-trained model's quality\nby introducing a headline generation task as an intermediate task before the\nheadline editing task. Also, we propose Self Importance-Aware (SIA) loss to\naddress the different levels of editing in the dataset by down-weighting the\nimportance of easily classified tokens and sentences. With the help of\nPre-training, Adaptation, and SIA, the model learns to generate headlines in\nthe professional editor's style. Experimental results show that our method\nsignificantly improves the quality of headline editing comparing against\nprevious methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01140v2",
            "date": "2019-12-03 00:58:01+00:00",
            "title": "An Annotated Dataset of Coreference in English Literature",
            "authors": [
                "David Bamman",
                "Olivia Lewke",
                "Anya Mansoor"
            ],
            "abstract": "We present in this work a new dataset of coreference annotations for works of\nliterature in English, covering 29,103 mentions in 210,532 tokens from 100\nworks of fiction. This dataset differs from previous coreference datasets in\ncontaining documents whose average length (2,105.3 words) is four times longer\nthan other benchmark datasets (463.7 for OntoNotes), and contains examples of\ndifficult coreference problems common in literature. This dataset allows for an\nevaluation of cross-domain performance for the task of coreference resolution,\nand analysis into the characteristics of long-distance within-document\ncoreference.",
            "journal ref": "Proceedings of the 12th Language Resources and Evaluation\n  Conference (LREC 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01156v1",
            "date": "2019-12-03 02:25:16+00:00",
            "title": "Unsupervised Inflection Generation Using Neural Language Modeling",
            "authors": [
                "Octavia-Maria Sulea",
                "Steve Young"
            ],
            "abstract": "The use of Deep Neural Network architectures for Language Modeling has\nrecently seen a tremendous increase in interest in the field of NLP with the\nadvent of transfer learning and the shift in focus from rule-based and\npredictive models (supervised learning) to generative or unsupervised models to\nsolve the long-standing problems in NLP like Information Extraction or Question\nAnswering. While this shift has worked greatly for languages lacking in\ninflectional morphology, such as English, challenges still arise when trying to\nbuild similar systems for morphologically-rich languages, since their\nindividual words shift forms in context more often. In this paper we\ninvestigate the extent to which these new unsupervised or generative techniques\ncan serve to alleviate the type-token ratio disparity in morphologically rich\nlanguages. We apply an off-the-shelf neural language modeling library to the\nnewly introduced task of unsupervised inflection generation in the nominal\ndomain of three morphologically rich languages: Romanian, German, and Finnish.\nWe show that this neural language model architecture can successfully generate\nthe full inflection table of nouns without needing any pre-training on large,\nwikipedia-sized corpora, as long as the model is shown enough inflection\nexamples. In fact, our experiments show that pre-training hinders the\ngeneration performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01214v1",
            "date": "2019-12-03 06:41:03+00:00",
            "title": "Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation",
            "authors": [
                "Baijun Ji",
                "Zhirui Zhang",
                "Xiangyu Duan",
                "Min Zhang",
                "Boxing Chen",
                "Weihua Luo"
            ],
            "abstract": "Transfer learning between different language pairs has shown its\neffectiveness for Neural Machine Translation (NMT) in low-resource scenario.\nHowever, existing transfer methods involving a common target language are far\nfrom success in the extreme scenario of zero-shot translation, due to the\nlanguage space mismatch problem between transferor (the parent model) and\ntransferee (the child model) on the source side. To address this challenge, we\npropose an effective transfer learning approach based on cross-lingual\npre-training. Our key idea is to make all source languages share the same\nfeature space and thus enable a smooth transition for zero-shot translation. To\nthis end, we introduce one monolingual pre-training method and two bilingual\npre-training methods to obtain a universal encoder for different languages.\nOnce the universal encoder is constructed, the parent model built on such\nencoder is trained with large-scale annotated data and then directly applied in\nzero-shot translation scenario. Experiments on two public datasets show that\nour approach significantly outperforms strong pivot-based baseline and various\nmultilingual NMT approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01496v1",
            "date": "2019-12-03 16:16:13+00:00",
            "title": "Knowledge-Enriched Visual Storytelling",
            "authors": [
                "Chao-Chun Hsu",
                "Zi-Yuan Chen",
                "Chi-Yang Hsu",
                "Chih-Chia Li",
                "Tzu-Yuan Lin",
                "Ting-Hao 'Kenneth' Huang",
                "Lun-Wei Ku"
            ],
            "abstract": "Stories are diverse and highly personalized, resulting in a large possible\noutput space for story generation. Existing end-to-end approaches produce\nmonotonous stories because they are limited to the vocabulary and knowledge in\na single training dataset. This paper introduces KG-Story, a three-stage\nframework that allows the story generation model to take advantage of external\nKnowledge Graphs to produce interesting stories. KG-Story distills a set of\nrepresentative words from the input prompts, enriches the word set by using\nexternal knowledge graphs, and finally generates stories based on the enriched\nword set. This distill-enrich-generate framework allows the use of external\nresources not only for the enrichment phase, but also for the distillation and\ngeneration phases. In this paper, we show the superiority of KG-Story for\nvisual storytelling, where the input prompt is a sequence of five photos and\nthe output is a short story. Per the human ranking evaluation, stories\ngenerated by KG-Story are on average ranked better than that of the\nstate-of-the-art systems. Our code and output stories are available at\nhttps://github.com/zychen423/KE-VIST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01586v2",
            "date": "2019-12-03 18:31:42+00:00",
            "title": "Reading the Manual: Event Extraction as Definition Comprehension",
            "authors": [
                "Yunmo Chen",
                "Tongfei Chen",
                "Seth Ebner",
                "Aaron Steven White",
                "Benjamin Van Durme"
            ],
            "abstract": "We ask whether text understanding has progressed to where we may extract\nevent information through incremental refinement of bleached statements derived\nfrom annotation manuals. Such a capability would allow for the trivial\nconstruction and extension of an extraction framework by intended end-users\nthrough declarations such as, \"Some person was born in some location at some\ntime.\" We introduce an example of a model that employs such statements, with\nexperiments illustrating we can extract events under closed ontologies and\ngeneralize to unseen event types simply by reading new definitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01673v2",
            "date": "2019-12-03 20:20:31+00:00",
            "title": "COSTRA 1.0: A Dataset of Complex Sentence Transformations",
            "authors": [
                "Petra Barancikova",
                "Ondrej Bojar"
            ],
            "abstract": "We present COSTRA 1.0, a dataset of complex sentence transformations. The\ndataset is intended for the study of sentence-level embeddings beyond simple\nword alternations or standard paraphrasing. This first version of the dataset\nis limited to sentences in Czech but the construction method is universal and\nwe plan to use it also for other languages. The dataset consist of 4,262 unique\nsentences with average length of 10 words, illustrating 15 types of\nmodifications such as simplification, generalization, or formal and informal\nlanguage variation. The hope is that with this dataset, we should be able to\ntest semantic properties of sentence embeddings and perhaps even to find some\ntopologically interesting 'skeleton' in the sentence embedding space. A\npreliminary analysis using LASER, multi-purpose multi-lingual sentence\nembeddings suggests that the LASER space does not exhibit the desired\nproperties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01682v1",
            "date": "2019-12-03 20:45:04+00:00",
            "title": "AMR-to-Text Generation with Cache Transition Systems",
            "authors": [
                "Lisa Jin",
                "Daniel Gildea"
            ],
            "abstract": "Text generation from AMR involves emitting sentences that reflect the meaning\nof their AMR annotations. Neural sequence-to-sequence models have successfully\nbeen used to decode strings from flattened graphs (e.g., using depth-first or\nrandom traversal). Such models often rely on attention-based decoders to map\nAMR node to English token sequences. Instead of linearizing AMR, we directly\nencode its graph structure and delegate traversal to the decoder. To enforce a\nsentence-aligned graph traversal and provide local graph context, we predict\ntransition-based parser actions in addition to English words. We present two\nmodel variants: one generates parser actions prior to words, while the other\ninterleaves actions with words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01731v1",
            "date": "2019-12-03 23:00:38+00:00",
            "title": "HAMNER: Headword Amplified Multi-span Distantly Supervised Method for Domain Specific Named Entity Recognition",
            "authors": [
                "Shifeng Liu",
                "Yifang Sun",
                "Bing Li",
                "Wei Wang",
                "Xiang Zhao"
            ],
            "abstract": "To tackle Named Entity Recognition (NER) tasks, supervised methods need to\nobtain sufficient cleanly annotated data, which is labor and time consuming. On\nthe contrary, distantly supervised methods acquire automatically annotated data\nusing dictionaries to alleviate this requirement. Unfortunately, dictionaries\nhinder the effectiveness of distantly supervised methods for NER due to its\nlimited coverage, especially in specific domains. In this paper, we aim at the\nlimitations of the dictionary usage and mention boundary detection. We\ngeneralize the distant supervision by extending the dictionary with headword\nbased non-exact matching. We apply a function to better weight the matched\nentity mentions. We propose a span-level model, which classifies all the\npossible spans then infers the selected spans with a proposed dynamic\nprogramming algorithm. Experiments on all three benchmark datasets demonstrate\nthat our method outperforms previous state-of-the-art distantly supervised\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01772v2",
            "date": "2019-12-04 02:26:39+00:00",
            "title": "A Resource for Computational Experiments on Mapudungun",
            "authors": [
                "Mingjun Duan",
                "Carlos Fasola",
                "Sai Krishna Rallabandi",
                "Rodolfo M. Vega",
                "Antonios Anastasopoulos",
                "Lori Levin",
                "Alan W Black"
            ],
            "abstract": "We present a resource for computational experiments on Mapudungun, a\npolysynthetic indigenous language spoken in Chile with upwards of 200 thousand\nspeakers. We provide 142 hours of culturally significant conversations in the\ndomain of medical treatment. The conversations are fully transcribed and\ntranslated into Spanish. The transcriptions also include annotations for\ncode-switching and non-standard pronunciations. We also provide baseline\nresults on three core NLP tasks: speech recognition, speech synthesis, and\nmachine translation between Spanish and Mapudungun. We further explore other\napplications for which the corpus will be suitable, including the study of\ncode-switching, historical orthography change, linguistic structure, and\nsociological and anthropological studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01774v1",
            "date": "2019-12-04 02:54:18+00:00",
            "title": "Acquiring Knowledge from Pre-trained Model to Neural Machine Translation",
            "authors": [
                "Rongxiang Weng",
                "Heng Yu",
                "Shujian Huang",
                "Shanbo Cheng",
                "Weihua Luo"
            ],
            "abstract": "Pre-training and fine-tuning have achieved great success in the natural\nlanguage process field. The standard paradigm of exploiting them includes two\nsteps: first, pre-training a model, e.g. BERT, with a large scale unlabeled\nmonolingual data. Then, fine-tuning the pre-trained model with labeled data\nfrom downstream tasks. However, in neural machine translation (NMT), we address\nthe problem that the training objective of the bilingual task is far different\nfrom the monolingual pre-trained model. This gap leads that only using\nfine-tuning in NMT can not fully utilize prior language knowledge. In this\npaper, we propose an APT framework for acquiring knowledge from the pre-trained\nmodel to NMT. The proposed approach includes two modules: 1). a dynamic fusion\nmechanism to fuse task-specific features adapted from general knowledge into\nNMT network, 2). a knowledge distillation paradigm to learn language knowledge\ncontinuously during the NMT training process. The proposed approach could\nintegrate suitable knowledge from pre-trained models to improve the NMT.\nExperimental results on WMT English to German, German to English and Chinese to\nEnglish machine translation tasks show that our model outperforms strong\nbaselines and the fine-tuning counterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01858v1",
            "date": "2019-12-04 09:16:36+00:00",
            "title": "Enhancing Relation Extraction Using Syntactic Indicators and Sentential Contexts",
            "authors": [
                "Qiongxing Tao",
                "Xiangfeng Luo",
                "Hao Wang"
            ],
            "abstract": "State-of-the-art methods for relation extraction consider the sentential\ncontext by modeling the entire sentence. However, syntactic indicators, certain\nphrases or words like prepositions that are more informative than other words\nand may be beneficial for identifying semantic relations. Other approaches\nusing fixed text triggers capture such information but ignore the lexical\ndiversity. To leverage both syntactic indicators and sentential contexts, we\npropose an indicator-aware approach for relation extraction. Firstly, we\nextract syntactic indicators under the guidance of syntactic knowledge. Then we\nconstruct a neural network to incorporate both syntactic indicators and the\nentire sentences into better relation representations. By this way, the\nproposed model alleviates the impact of noisy information from entire sentences\nand breaks the limit of text triggers. Experiments on the SemEval-2010 Task 8\nbenchmark dataset show that our model significantly outperforms the\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01957v1",
            "date": "2019-11-29 20:09:15+00:00",
            "title": "A probabilistic assessment of the Indo-Aryan Inner-Outer Hypothesis",
            "authors": [
                "Chundra A. Cathcart"
            ],
            "abstract": "This paper uses a novel data-driven probabilistic approach to address the\ncentury-old Inner-Outer hypothesis of Indo-Aryan. I develop a Bayesian\nhierarchical mixed-membership model to assess the validity of this hypothesis\nusing a large data set of automatically extracted sound changes operating\nbetween Old Indo-Aryan and Modern Indo-Aryan speech varieties. I employ\ndifferent prior distributions in order to model sound change, one of which, the\nlogistic normal distribution, has not received much attention in linguistics\noutside of Natural Language Processing, despite its many attractive features. I\nfind evidence for cohesive dialect groups that have made their imprint on\ncontemporary Indo-Aryan languages, and find that when a logistic normal prior\nis used, the distribution of dialect components across languages is largely\ncompatible with a core-periphery pattern similar to that proposed under the\nInner-Outer hypothesis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01982v1",
            "date": "2019-12-02 18:45:23+00:00",
            "title": "Neural Academic Paper Generation",
            "authors": [
                "Samet Demir",
                "Uras Mutlu",
                "\u00d6zgur \u00d6zdemir"
            ],
            "abstract": "In this work, we tackle the problem of structured text generation,\nspecifically academic paper generation in $\\LaTeX{}$, inspired by the\nsurprisingly good results of basic character-level language models. Our\nmotivation is using more recent and advanced methods of language modeling on a\nmore complex dataset of $\\LaTeX{}$ source files to generate realistic academic\npapers. Our first contribution is preparing a dataset with $\\LaTeX{}$ source\nfiles on recent open-source computer vision papers. Our second contribution is\nexperimenting with recent methods of language modeling and text generation such\nas Transformer and Transformer-XL to generate consistent $\\LaTeX{}$ code. We\nreport cross-entropy and bits-per-character (BPC) results of the trained\nmodels, and we also discuss interesting points on some examples of the\ngenerated $\\LaTeX{}$ code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02047v2",
            "date": "2019-12-04 15:16:03+00:00",
            "title": "Neural Machine Translation: A Review and Survey",
            "authors": [
                "Felix Stahlberg"
            ],
            "abstract": "The field of machine translation (MT), the automatic translation of written\ntext from one natural language into another, has experienced a major paradigm\nshift in recent years. Statistical MT, which mainly relies on various\ncount-based models and which used to dominate MT research for decades, has\nlargely been superseded by neural machine translation (NMT), which tackles\ntranslation with a single neural network. In this work we will trace back the\norigins of modern NMT architectures to word and sentence embeddings and earlier\nexamples of the encoder-decoder network family. We will conclude with a survey\nof recent trends in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02145v1",
            "date": "2019-12-04 17:48:58+00:00",
            "title": "An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering",
            "authors": [
                "Shayne Longpre",
                "Yi Lu",
                "Zhucheng Tu",
                "Chris DuBois"
            ],
            "abstract": "To produce a domain-agnostic question answering model for the Machine Reading\nQuestion Answering (MRQA) 2019 Shared Task, we investigate the relative\nbenefits of large pre-trained language models, various data sampling\nstrategies, as well as query and context paraphrases generated by\nback-translation. We find a simple negative sampling technique to be\nparticularly effective, even though it is typically used for datasets that\ninclude unanswerable questions, such as SQuAD 2.0. When applied in conjunction\nwith per-domain sampling, our XLNet (Yang et al., 2019)-based submission\nachieved the second best Exact Match and F1 in the MRQA leaderboard\ncompetition.",
            "journal ref": "In Proceedings of the 2nd Workshop on Machine Reading for Question\n  Answering, pp. 220-227. 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02206v1",
            "date": "2019-12-04 19:00:54+00:00",
            "title": "Cooperative Reasoning on Knowledge Graph and Corpus: A Multi-agentReinforcement Learning Approach",
            "authors": [
                "Yunan Zhang",
                "Xiang Cheng",
                "Heting Gao",
                "Chengxiang Zhai"
            ],
            "abstract": "Knowledge-graph-based reasoning has drawn a lot of attention due to its\ninterpretability. However, previous methods suffer from the incompleteness of\nthe knowledge graph, namely the interested link or entity that can be missing\nin the knowledge graph(explicit missing). Also, most previous models assume the\ndistance between the target and source entity is short, which is not true on a\nreal-world KG like Freebase(implicit missing). The sensitivity to the\nincompleteness of KG and the incapability to capture the long-distance link\nbetween entities have limited the performance of these models on large KG. In\nthis paper, we propose a model that leverages the text corpus to cure such\nlimitations, either the explicit or implicit missing links. We model the\nquestion answering on KG as a cooperative task between two agents, a knowledge\ngraph reasoning agent and an information extraction agent. Each agent learns\nits skill to complete its own task, hopping on KG or select knowledge from the\ncorpus, via maximizing the reward for correctly answering the question. The\nreasoning agent decides how to find an equivalent path for the given entity and\nrelation. The extraction agent provide shortcut for long-distance target entity\nor provide missing relations for explicit missing links with messages from the\nreasoning agent. Through such cooperative reward design, our model can augment\nthe incomplete KG strategically while not introduce much unnecessary noise that\ncould enlarge the search space and lower the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02367v2",
            "date": "2019-12-05 03:44:20+00:00",
            "title": "Easy-to-Hard: Leveraging Simple Questions for Complex Question Generation",
            "authors": [
                "Jie Zhao",
                "Xiang Deng",
                "Huan Sun"
            ],
            "abstract": "This paper makes one of the first efforts toward automatically generating\ncomplex questions from knowledge graphs. Particularly, we study how to leverage\nexisting simple question datasets for this task, under two separate scenarios:\nusing either sub-questions of the target complex questions, or distantly\nrelated pseudo sub-questions when the former are unavailable. First, a\ncompetitive base model named CoG2Q is designed to map complex query qraphs to\nnatural language questions. Afterwards, we propose two extension models, namely\nCoGSub2Q and CoGSub^m2Q, respectively for the above two scenarios. The former\nencodes and copies from a sub-question, while the latter further scores and\naggregates multiple pseudo sub-questions. Experiment results show that the\nextension models significantly outperform not only base CoG2Q, but also its\naugmented variant using simple questions as additional training examples. This\ndemonstrates the importance of instance-level connections between simple and\ncorresponding complex questions, which may be underexploited by straightforward\ndata augmentation of CoG2Q that builds model-level connections through learned\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02395v2",
            "date": "2019-12-05 05:32:41+00:00",
            "title": "Learning to Predict Explainable Plots for Neural Story Generation",
            "authors": [
                "Gang Chen",
                "Yang Liu",
                "Huanbo Luan",
                "Meng Zhang",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Story generation is an important natural language processing task that aims\nto generate coherent stories automatically. While the use of neural networks\nhas proven effective in improving story generation, how to learn to generate an\nexplainable high-level plot still remains a major challenge. In this work, we\npropose a latent variable model for neural story generation. The model treats\nan outline, which is a natural language sentence explainable to humans, as a\nlatent variable to represent a high-level plot that bridges the input and\noutput. We adopt an external summarization model to guide the latent variable\nmodel to learn how to generate outlines from training data. Experiments show\nthat our approach achieves significant improvements over state-of-the-art\nmethods in both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02477v2",
            "date": "2019-12-05 10:15:13+00:00",
            "title": "Love Me, Love Me, Say (and Write!) that You Love Me: Enriching the WASABI Song Corpus with Lyrics Annotations",
            "authors": [
                "Michael Fell",
                "Elena Cabrio",
                "Elmahdi Korfed",
                "Michel Buffa",
                "Fabien Gandon"
            ],
            "abstract": "We present the WASABI Song Corpus, a large corpus of songs enriched with\nmetadata extracted from music databases on the Web, and resulting from the\nprocessing of song lyrics and from audio analysis. More specifically, given\nthat lyrics encode an important part of the semantics of a song, we focus here\non the description of the methods we proposed to extract relevant information\nfrom the lyrics, such as their structure segmentation, their topics, the\nexplicitness of the lyrics content, the salient passages of a song and the\nemotions conveyed. The creation of the resource is still ongoing: so far, the\ncorpus contains 1.73M songs with lyrics (1.41M unique lyrics) annotated at\ndifferent levels with the output of the above mentioned methods. Such corpus\nlabels and the provided methods can be exploited by music search engines and\nmusic professionals (e.g. journalists, radio presenters) to better handle large\ncollections of lyrics, allowing an intelligent browsing, categorization and\nsegmentation recommendation of songs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02478v1",
            "date": "2019-12-05 10:16:32+00:00",
            "title": "Effective Data Augmentation Approaches to End-to-End Task-Oriented Dialogue",
            "authors": [
                "Jun Quan",
                "Deyi Xiong"
            ],
            "abstract": "The training of task-oriented dialogue systems is often confronted with the\nlack of annotated data. In contrast to previous work which augments training\ndata through expensive crowd-sourcing efforts, we propose four different\nautomatic approaches to data augmentation at both the word and sentence level\nfor end-to-end task-oriented dialogue and conduct an empirical study on their\nimpact. Experimental results on the CamRest676 and KVRET datasets demonstrate\nthat each of the four data augmentation approaches is able to obtain a\nsignificant improvement over a strong baseline in terms of Success F1 score and\nthat the ensemble of the four approaches achieves the state-of-the-art results\nin the two datasets. In-depth analyses further confirm that our methods\nadequately increase the diversity of user utterances, which enables the\nend-to-end model to learn features robustly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02481v2",
            "date": "2019-12-05 10:25:32+00:00",
            "title": "Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor\u00f9b\u00e1 and Twi",
            "authors": [
                "Jesujoba O. Alabi",
                "Kwabena Amponsah-Kaakyire",
                "David I. Adelani",
                "Cristina Espa\u00f1a-Bonet"
            ],
            "abstract": "The success of several architectures to learn semantic representations from\nunannotated text and the availability of these kind of texts in online\nmultilingual resources such as Wikipedia has facilitated the massive and\nautomatic creation of resources for multiple languages. The evaluation of such\nresources is usually done for the high-resourced languages, where one has a\nsmorgasbord of tasks and test sets to evaluate on. For low-resourced languages,\nthe evaluation is more difficult and normally ignored, with the hope that the\nimpressive capability of deep learning architectures to learn (multilingual)\nrepresentations in the high-resourced setting holds in the low-resourced\nsetting too. In this paper we focus on two African languages, Yor\\`ub\\'a and\nTwi, and compare the word embeddings obtained in this way, with word embeddings\nobtained from curated corpora and a language-dependent processing. We analyse\nthe noise in the publicly available corpora, collect high quality and noisy\ndata for the two languages and quantify the improvements that depend not only\non the amount of data but on the quality too. We also use different\narchitectures that learn word representations both from surface forms and\ncharacters to further exploit all the available information which showed to be\nimportant for these languages. For the evaluation, we manually translate the\nwordsim-353 word pairs dataset from English into Yor\\`ub\\'a and Twi. As output\nof the work, we provide corpora, embeddings and the test suits for both\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02545v1",
            "date": "2019-12-05 12:56:28+00:00",
            "title": "Fine-Grained Emotion Classification of Chinese Microblogs Based on Graph Convolution Networks",
            "authors": [
                "Yuni Lai",
                "Linfeng Zhang",
                "Donghong Han",
                "Rui Zhou",
                "Guoren Wang"
            ],
            "abstract": "Microblogs are widely used to express people's opinions and feelings in daily\nlife. Sentiment analysis (SA) can timely detect personal sentiment polarities\nthrough analyzing text. Deep learning approaches have been broadly used in SA\nbut still have not fully exploited syntax information. In this paper, we\npropose a syntax-based graph convolution network (GCN) model to enhance the\nunderstanding of diverse grammatical structures of Chinese microblogs. In\naddition, a pooling method based on percentile is proposed to improve the\naccuracy of the model. In experiments, for Chinese microblogs emotion\nclassification categories including happiness, sadness, like, anger, disgust,\nfear, and surprise, the F-measure of our model reaches 82.32% and exceeds the\nstate-of-the-art algorithm by 5.90%. The experimental results show that our\nmodel can effectively utilize the information of dependency parsing to improve\nthe performance of emotion detection. What is more, we annotate a new dataset\nfor Chinese emotion classification, which is open to other researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02761v2",
            "date": "2019-12-05 17:53:25+00:00",
            "title": "Measuring Social Bias in Knowledge Graph Embeddings",
            "authors": [
                "Joseph Fisher",
                "Dave Palfrey",
                "Christos Christodoulopoulos",
                "Arpit Mittal"
            ],
            "abstract": "It has recently been shown that word embeddings encode social biases, with a\nharmful impact on downstream tasks. However, to this point there has been no\nsimilar work done in the field of graph embeddings. We present the first study\non social bias in knowledge graph embeddings, and propose a new metric suitable\nfor measuring such bias. We conduct experiments on Wikidata and Freebase, and\nshow that, as with word embeddings, harmful social biases related to\nprofessions are encoded in the embeddings with respect to gender, religion,\nethnicity and nationality. For example, graph embeddings encode the information\nthat men are more likely to be bankers, and women more likely to be\nhomekeepers. As graph embeddings become increasingly utilized, we suggest that\nit is important the existence of such biases are understood and steps taken to\nmitigate their impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03334v1",
            "date": "2019-12-06 20:27:38+00:00",
            "title": "Explaining Sequence-Level Knowledge Distillation as Data-Augmentation for Neural Machine Translation",
            "authors": [
                "Mitchell A. Gordon",
                "Kevin Duh"
            ],
            "abstract": "Sequence-level knowledge distillation (SLKD) is a model compression technique\nthat leverages large, accurate teacher models to train smaller,\nunder-parameterized student models. Why does pre-processing MT data with SLKD\nhelp us train smaller models? We test the common hypothesis that SLKD addresses\na capacity deficiency in students by \"simplifying\" noisy data points and find\nit unlikely in our case. Models trained on concatenations of original and\n\"simplified\" datasets generalize just as well as baseline SLKD. We then propose\nan alternative hypothesis under the lens of data augmentation and\nregularization. We try various augmentation strategies and observe that dropout\nregularization can become unnecessary. Our methods achieve BLEU gains of\n0.7-1.2 on TED Talks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03441v1",
            "date": "2019-12-07 05:12:03+00:00",
            "title": "Adversarial Analysis of Natural Language Inference Systems",
            "authors": [
                "Tiffany Chien",
                "Jugal Kalita"
            ],
            "abstract": "The release of large natural language inference (NLI) datasets like SNLI and\nMNLI have led to rapid development and improvement of completely neural systems\nfor the task. Most recently, heavily pre-trained, Transformer-based models like\nBERT and MT-DNN have reached near-human performance on these datasets. However,\nthese standard datasets have been shown to contain many annotation artifacts,\nallowing models to shortcut understanding using simple fallible heuristics, and\nstill perform well on the test set. So it is no surprise that many adversarial\n(challenge) datasets have been created that cause models trained on standard\ndatasets to fail dramatically. Although extra training on this data generally\nimproves model performance on just that type of data, transferring that\nlearning to unseen examples is still partial at best. This work evaluates the\nfailures of state-of-the-art models on existing adversarial datasets that test\ndifferent linguistic phenomena, and find that even though the models perform\nsimilarly on MNLI, they differ greatly in their robustness to these attacks. In\nparticular, we find syntax-related attacks to be particularly effective across\nall models, so we provide a fine-grained analysis and comparison of model\nperformance on those examples. We draw conclusions about the value of model\nsize and multi-task learning (beyond comparing their standard test set\nperformance), and provide suggestions for more effective training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03502v2",
            "date": "2019-12-07 13:26:18+00:00",
            "title": "Personalized Patent Claim Generation and Measurement",
            "authors": [
                "Jieh-Sheng Lee"
            ],
            "abstract": "This work-in-progress paper proposes a framework to generate and measure\npersonalized patent claims. The objective is to help inventors conceive better\ninventions by learning from relevant inventors. Patent claim generation is a\nway of \"augmented inventing.\" for inventors. Such patent claim generation\nleverages the recent transfer learning in the Deep Learning field, particularly\nthe state-of-the-art Transformer-based models. In terms of system\nimplementa-tion, it is planned to build an \"auto-complete\" function for patent\nclaim drafting. The auto-complete function is analyzed from four different\nperspectives: extent of generation, generative direction, proximity of\ngeneration, and constraint in generation. Technically, the framework is\ncomposed of two Transformer models. One is for text generation and the other is\nfor quality measurement. Specifically, the patent claim generation is based on\nGPT-2 model and the measurement of personalization is based on BERT model. The\ntraining data is inventor-centric and comes from the Inventors Endpoint API\nprovided by the USPTO.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04479v1",
            "date": "2019-12-10 03:45:42+00:00",
            "title": "Homograph Disambiguation Through Selective Diacritic Restoration",
            "authors": [
                "Sawsan Alqahtani",
                "Hanan Aldarmaki",
                "Mona Diab"
            ],
            "abstract": "Lexical ambiguity, a challenging phenomenon in all natural languages, is\nparticularly prevalent for languages with diacritics that tend to be omitted in\nwriting, such as Arabic. Omitting diacritics leads to an increase in the number\nof homographs: different words with the same spelling. Diacritic restoration\ncould theoretically help disambiguate these words, but in practice, the\nincrease in overall sparsity leads to performance degradation in NLP\napplications. In this paper, we propose approaches for automatically marking a\nsubset of words for diacritic restoration, which leads to selective homograph\ndisambiguation. Compared to full or no diacritic restoration, these approaches\nyield selectively-diacritized datasets that balance sparsity and lexical\ndisambiguation. We evaluate the various selection strategies extrinsically on\nseveral downstream applications: neural machine translation, part-of-speech\ntagging, and semantic textual similarity. Our experiments on Arabic show\npromising results, where our devised strategies on selective diacritization\nlead to a more balanced and consistent performance in downstream applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04664v1",
            "date": "2019-12-10 12:27:45+00:00",
            "title": "How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning",
            "authors": [
                "Lu Li",
                "Zhongheng He",
                "Xiangyang Zhou",
                "Dianhai Yu"
            ],
            "abstract": "Automatic dialogue evaluation plays a crucial role in open-domain dialogue\nresearch. Previous works train neural networks with limited annotation for\nconducting automatic dialogue evaluation, which would naturally affect the\nevaluation fairness as dialogue systems close to the scope of training corpus\nwould have more preference than the other ones. In this paper, we study\nalleviating this problem from the perspective of continual learning: given an\nexisting neural dialogue evaluator and the next system to be evaluated, we\nfine-tune the learned neural evaluator by selectively forgetting/updating its\nparameters, to jointly fit dialogue systems have been and will be evaluated.\nOur motivation is to seek for a lifelong and low-cost automatic evaluation for\ndialogue systems, rather than to reconstruct the evaluator over and over again.\nExperimental results show that our continual evaluator achieves comparable\nperformance with reconstructing new evaluators, while requires significantly\nlower resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04778v1",
            "date": "2019-12-10 15:50:50+00:00",
            "title": "GeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia Biographies",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Pau Li Lin",
                "Cristina Espa\u00f1a-Bonet"
            ],
            "abstract": "We introduce GeBioToolkit, a tool for extracting multilingual parallel\ncorpora at sentence level, with document and gender information from Wikipedia\nbiographies. Despite thegender inequalitiespresent in Wikipedia, the toolkit\nhas been designed to extract corpus balanced in gender. While our toolkit is\ncustomizable to any number of languages (and different domains), in this work\nwe present a corpus of 2,000 sentences in English, Spanish and Catalan, which\nhas been post-edited by native speakers to become a high-quality dataset for\nmachinetranslation evaluation. While GeBioCorpus aims at being one of the first\nnon-synthetic gender-balanced test datasets, GeBioToolkit aims at paving the\npath to standardize procedures to produce gender-balanced datasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04965v2",
            "date": "2019-12-10 20:20:40+00:00",
            "title": "An Ensemble Method for Producing Word Representations focusing on the Greek Language",
            "authors": [
                "Michalis Lioudakis",
                "Stamatis Outsios",
                "Michalis Vazirgiannis"
            ],
            "abstract": "In this paper we present a new ensemble method, Continuous Bag-of-Skip-grams\n(CBOS), that produces high-quality word representations putting emphasis on the\nmodern Greek language. The CBOS method combines the pioneering approaches for\nlearning word representations: Continuous Bag-of-Words (CBOW) and Continuous\nSkip-gram. These methods are compared through intrinsic and extrinsic\nevaluation tasks on three different sources of data: the English Wikipedia\ncorpus, the modern Greek Wikipedia corpus, and the modern Greek Web Content\ncorpus. By comparing these methods across different tasks and datasets, it is\nevident that the CBOS method achieves state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04971v2",
            "date": "2019-12-10 20:36:07+00:00",
            "title": "Neural Module Networks for Reasoning over Text",
            "authors": [
                "Nitish Gupta",
                "Kevin Lin",
                "Dan Roth",
                "Sameer Singh",
                "Matt Gardner"
            ],
            "abstract": "Answering compositional questions that require multiple steps of reasoning\nagainst text is challenging, especially when they involve discrete, symbolic\noperations. Neural module networks (NMNs) learn to parse such questions as\nexecutable programs composed of learnable modules, performing well on synthetic\nvisual QA domains. However, we find that it is challenging to learn these\nmodels for non-synthetic questions on open-domain text, where a model needs to\ndeal with the diversity of natural language and perform a broader range of\nreasoning. We extend NMNs by: (a) introducing modules that reason over a\nparagraph of text, performing symbolic reasoning (such as arithmetic, sorting,\ncounting) over numbers and dates in a probabilistic and differentiable manner;\nand (b) proposing an unsupervised auxiliary loss to help extract arguments\nassociated with the events in text. Additionally, we show that a limited amount\nof heuristically-obtained question program and intermediate module output\nsupervision provides sufficient inductive bias for accurate learning. Our\nproposed model significantly outperforms state-of-the-art models on a subset of\nthe DROP dataset that poses a variety of reasoning challenges that are covered\nby our modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05134v1",
            "date": "2019-12-11 06:21:16+00:00",
            "title": "Unsupervised Neural Dialect Translation with Commonality and Diversity Modeling",
            "authors": [
                "Yu Wan",
                "Baosong Yang",
                "Derek F. Wong",
                "Lidia S. Chao",
                "Haihua Du",
                "Ben C. H. Ao"
            ],
            "abstract": "As a special machine translation task, dialect translation has two main\ncharacteristics: 1) lack of parallel training corpus; and 2) possessing similar\ngrammar between two sides of the translation. In this paper, we investigate how\nto exploit the commonality and diversity between dialects thus to build\nunsupervised translation models merely accessing to monolingual data.\nSpecifically, we leverage pivot-private embedding, layer coordination, as well\nas parameter sharing to sufficiently model commonality and diversity among\nsource and target, ranging from lexical, through syntactic, to semantic levels.\nIn order to examine the effectiveness of the proposed models, we collect 20\nmillion monolingual corpus for each of Mandarin and Cantonese, which are\nofficial language and the most widely used dialect in China. Experimental\nresults reveal that our methods outperform rule-based simplified and\ntraditional Chinese conversion and conventional unsupervised translation models\nover 12 BLEU scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05147v1",
            "date": "2019-12-11 07:29:59+00:00",
            "title": "Improving Neural Protein-Protein Interaction Extraction with Knowledge Selection",
            "authors": [
                "Huiwei Zhou",
                "Xuefei Li",
                "Weihong Yao",
                "Zhuang Liu",
                "Shixian Ning",
                "Chengkun Lang",
                "Lei Du"
            ],
            "abstract": "Protein-protein interaction (PPI) extraction from published scientific\nliterature provides additional support for precision medicine efforts.\nMeanwhile, knowledge bases (KBs) contain huge amounts of structured information\nof protein entities and their relations, which can be encoded in entity and\nrelation embeddings to help PPI extraction. However, the prior knowledge of\nprotein-protein pairs must be selectively used so that it is suitable for\ndifferent contexts. This paper proposes a Knowledge Selection Model (KSM) to\nfuse the selected prior knowledge and context information for PPI extraction.\nFirstly, two Transformers encode the context sequence of a protein pair\naccording to each protein embedding, respectively. Then, the two outputs are\nfed to a mutual attention to capture the important context features towards the\nprotein pair. Next, the context features are used to distill the relation\nembedding by a knowledge selector. Finally, the selected relation embedding and\nthe context features are concatenated for PPI extraction. Experiments on the\nBioCreative VI PPI dataset show that KSM achieves a new state-of-the-art\nperformance (38.08% F1-score) by adding knowledge selection.",
            "journal ref": "Computational Biology and Chemistry, 2019, 83: 107146",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05200v2",
            "date": "2019-12-11 09:33:21+00:00",
            "title": "Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering",
            "authors": [
                "Casimiro Pio Carrino",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa"
            ],
            "abstract": "Recently, multilingual question answering became a crucial research topic,\nand it is receiving increased interest in the NLP community. However, the\nunavailability of large-scale datasets makes it challenging to train\nmultilingual QA systems with performance comparable to the English ones. In\nthis work, we develop the Translate Align Retrieve (TAR) method to\nautomatically translate the Stanford Question Answering Dataset (SQuAD) v1.1 to\nSpanish. We then used this dataset to train Spanish QA systems by fine-tuning a\nMultilingual-BERT model. Finally, we evaluated our QA models with the recently\nproposed MLQA and XQuAD benchmarks for cross-lingual Extractive QA.\nExperimental results show that our models outperform the previous\nMultilingual-BERT baselines achieving the new state-of-the-art value of 68.1 F1\npoints on the Spanish MLQA corpus and 77.6 F1 and 61.8 Exact Match points on\nthe Spanish XQuAD corpus. The resulting, synthetically generated SQuAD-es v1.1\ncorpora, with almost 100% of data contained in the original English version, to\nthe best of our knowledge, is the first large-scale QA training resource for\nSpanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05320v3",
            "date": "2019-12-11 14:02:59+00:00",
            "title": "CoSimLex: A Resource for Evaluating Graded Word Similarity in Context",
            "authors": [
                "Carlos Santos Armendariz",
                "Matthew Purver",
                "Matej Ul\u010dar",
                "Senja Pollak",
                "Nikola Ljube\u0161i\u0107",
                "Marko Robnik-\u0160ikonja",
                "Mark Granroth-Wilding",
                "Kristiina Vaik"
            ],
            "abstract": "State of the art natural language processing tools are built on\ncontext-dependent word embeddings, but no direct method for evaluating these\nrepresentations currently exists. Standard tasks and datasets for intrinsic\nevaluation of embeddings are based on judgements of similarity, but ignore\ncontext; standard tasks for word sense disambiguation take account of context\nbut do not provide continuous measures of meaning similarity. This paper\ndescribes an effort to build a new dataset, CoSimLex, intended to fill this\ngap. Building on the standard pairwise similarity task of SimLex-999, it\nprovides context-dependent similarity measures; covers not only discrete\ndifferences in word sense but more subtle, graded changes in meaning; and\ncovers not only a well-resourced language (English) but a number of\nless-resourced languages. We define the task and evaluation metrics, outline\nthe dataset collection methodology, and describe the status of the dataset so\nfar.",
            "journal ref": "Proceedings of the 12th Language Resources and Evaluation\n  Conference (2020) 5878-5886",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05493v1",
            "date": "2019-12-11 17:47:29+00:00",
            "title": "Quality of syntactic implication of RL-based sentence summarization",
            "authors": [
                "Hoa T. Le",
                "Christophe Cerisara",
                "Claire Gardent"
            ],
            "abstract": "Work on summarization has explored both reinforcement learning (RL)\noptimization using ROUGE as a reward and syntax-aware models, such as models\nthose input is enriched with part-of-speech (POS)-tags and dependency\ninformation. However, it is not clear what is the respective impact of these\napproaches beyond the standard ROUGE evaluation metric. Especially, RL-based\nfor summarization is becoming more and more popular. In this paper, we provide\na detailed comparison of these two approaches and of their combination along\nseveral dimensions that relate to the perceived quality of the generated\nsummaries: number of repeated words, distribution of part-of-speech tags,\nimpact of sentence length, relevance and grammaticality. Using the standard\nGigaword sentence summarization task, we compare an RL self-critical sequence\ntraining (SCST) method with syntax-aware models that leverage POS tags and\nDependency information. We show that on all qualitative evaluations, the\ncombined model gives the best results, but also that only training with RL and\nwithout any syntactic information already gives nearly as good results as\nsyntax-aware models with less parameters and faster training convergence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05898v2",
            "date": "2019-12-12 12:45:34+00:00",
            "title": "Improving Interpretability of Word Embeddings by Generating Definition and Usage",
            "authors": [
                "Haitong Zhang",
                "Yongping Du",
                "Jiaxin Sun",
                "Qingxiao Li"
            ],
            "abstract": "Word embeddings are substantially successful in capturing semantic relations\namong words. However, these lexical semantics are difficult to be interpreted.\nDefinition modeling provides a more intuitive way to evaluate embeddings by\nutilizing them to generate natural language definitions of corresponding words.\nThis task is of great significance for practical application and in-depth\nunderstanding of word representations. We propose a novel framework for\ndefinition modeling, which can generate reasonable and understandable\ncontext-dependent definitions. Moreover, we introduce usage modeling and study\nwhether it is possible to utilize embeddings to generate example sentences of\nwords. These ways are a more direct and explicit expression of embedding's\nsemantics for better interpretability. We extend the single task model to\nmulti-task setting and investigate several joint multi-task models to combine\nusage modeling and definition modeling together. Experimental results on\nexisting Oxford dataset and a new collected Oxford-2019 dataset show that our\nsingle-task model achieves the state-of-the-art result in definition modeling\nand the multi-task learning methods are helpful for two tasks to improve the\nperformance.",
            "journal ref": "Expert Systems with Applications, Volume 160, 1 December 2020,\n  113633",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06214v3",
            "date": "2019-12-12 21:11:56+00:00",
            "title": "Encoding Knowledge Graph Entity Aliases in Attentive Neural Network for Wikidata Entity Linking",
            "authors": [
                "Isaiah Onando Mulang",
                "Kuldeep Singh",
                "Akhilesh Vyas",
                "Saeedeh Shekarpour",
                "Maria Esther Vidal",
                "Jens Lehmann",
                "Soren Auer"
            ],
            "abstract": "The collaborative knowledge graphs such as Wikidata excessively rely on the\ncrowd to author the information. Since the crowd is not bound to a standard\nprotocol for assigning entity titles, the knowledge graph is populated by\nnon-standard, noisy, long or even sometimes awkward titles. The issue of long,\nimplicit, and nonstandard entity representations is a challenge in Entity\nLinking (EL) approaches for gaining high precision and recall. Underlying KG,\nin general, is the source of target entities for EL approaches, however, it\noften contains other relevant information, such as aliases of entities (e.g.,\nObama and Barack Hussein Obama are aliases for the entity Barack Obama). EL\nmodels usually ignore such readily available entity attributes. In this paper,\nwe examine the role of knowledge graph context on an attentive neural network\napproach for entity linking on Wikidata. Our approach contributes by exploiting\nthe sufficient context from a KG as a source of background knowledge, which is\nthen fed into the neural network. This approach demonstrates merit to address\nchallenges associated with entity titles (multi-word, long, implicit,\ncase-sensitive). Our experimental study shows approx 8% improvements over the\nbaseline approach, and significantly outperform an end to end approach for\nWikidata entity linking.",
            "journal ref": "WISE 2020 (21st International Conference on Web Information\n  Systems Engineering)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06598v2",
            "date": "2019-12-13 16:49:16+00:00",
            "title": "Document Sub-structure in Neural Machine Translation",
            "authors": [
                "Radina Dobreva",
                "Jie Zhou",
                "Rachel Bawden"
            ],
            "abstract": "Current approaches to machine translation (MT) either translate sentences in\nisolation, disregarding the context they appear in, or model context at the\nlevel of the full document, without a notion of any internal structure the\ndocument may have. In this work we consider the fact that documents are rarely\nhomogeneous blocks of text, but rather consist of parts covering different\ntopics. Some documents, such as biographies and encyclopedia entries, have\nhighly predictable, regular structures in which sections are characterised by\ndifferent topics. We draw inspiration from Louis and Webber (2014) who use this\ninformation to improve statistical MT and transfer their proposal into the\nframework of neural MT. We compare two different methods of including\ninformation about the topic of the section within which each sentence is found:\none using side constraints and the other using a cache-based model. We create\nand release the data on which we run our experiments - parallel corpora for\nthree language pairs (Chinese-English, French-English, Bulgarian-English) from\nWikipedia biographies, which we extract automatically, preserving the\nboundaries of sections within the articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06872v1",
            "date": "2019-12-14 16:03:15+00:00",
            "title": "Towards Robust Toxic Content Classification",
            "authors": [
                "Keita Kurita",
                "Anna Belova",
                "Antonios Anastasopoulos"
            ],
            "abstract": "Toxic content detection aims to identify content that can offend or harm its\nrecipients. Automated classifiers of toxic content need to be robust against\nadversaries who deliberately try to bypass filters. We propose a method of\ngenerating realistic model-agnostic attacks using a lexicon of toxic tokens,\nwhich attempts to mislead toxicity classifiers by diluting the toxicity signal\neither by obfuscating toxic tokens through character-level perturbations, or by\ninjecting non-toxic distractor tokens. We show that these realistic attacks\nreduce the detection recall of state-of-the-art neural toxicity detectors,\nincluding those using ELMo and BERT, by more than 50% in some cases. We explore\ntwo approaches for defending against such attacks. First, we examine the effect\nof training on synthetically noised data. Second, we propose the Contextual\nDenoising Autoencoder (CDAE): a method for learning robust representations that\nuses character-level and contextual information to denoise perturbed tokens. We\nshow that the two approaches are complementary, improving robustness to both\ncharacter-level perturbations and distractors, recovering a considerable\nportion of the lost accuracy. Finally, we analyze the robustness\ncharacteristics of the most competitive methods and outline practical\nconsiderations for improving toxicity detectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06900v1",
            "date": "2019-12-14 18:28:02+00:00",
            "title": "Efficient Convolutional Neural Networks for Diacritic Restoration",
            "authors": [
                "Sawsan Alqahtani",
                "Ajay Mishra",
                "Mona Diab"
            ],
            "abstract": "Diacritic restoration has gained importance with the growing need for\nmachines to understand written texts. The task is typically modeled as a\nsequence labeling problem and currently Bidirectional Long Short Term Memory\n(BiLSTM) models provide state-of-the-art results. Recently, Bai et al. (2018)\nshow the advantages of Temporal Convolutional Neural Networks (TCN) over\nRecurrent Neural Networks (RNN) for sequence modeling in terms of performance\nand computational resources. As diacritic restoration benefits from both\nprevious as well as subsequent timesteps, we further apply and evaluate a\nvariant of TCN, Acausal TCN (A-TCN), which incorporates context from both\ndirections (previous and future) rather than strictly incorporating previous\ncontext as in the case of TCN. A-TCN yields significant improvement over TCN\nfor diacritization in three different languages: Arabic, Yoruba, and\nVietnamese. Furthermore, A-TCN and BiLSTM have comparable performance, making\nA-TCN an efficient alternative over BiLSTM since convolutions can be trained in\nparallel. A-TCN is significantly faster than BiLSTM at inference time\n(270%-334% improvement in the amount of text diacritized per minute).",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06905v1",
            "date": "2019-12-14 19:13:44+00:00",
            "title": "Long-length Legal Document Classification",
            "authors": [
                "Lulu Wan",
                "George Papageorgiou",
                "Michael Seddon",
                "Mirko Bernardoni"
            ],
            "abstract": "One of the principal tasks of machine learning with major applications is\ntext classification. This paper focuses on the legal domain and, in particular,\non the classification of lengthy legal documents. The main challenge that this\nstudy addresses is the limitation that current models impose on the length of\nthe input text. In addition, the present paper shows that dividing the text\ninto segments and later combining the resulting embeddings with a BiLSTM\narchitecture to form a single document embedding can improve results. These\nadvancements are achieved by utilising a simpler structure, rather than an\nincreasingly complex one, which is often the case in NLP research. The dataset\nused in this paper is obtained from an online public database containing\nlengthy legal documents with highly domain-specific vocabulary and thus, the\ncomparison of our results to the ones produced by models implemented on the\ncommonly used datasets would be unjustified. This work provides the foundation\nfor future work in document classification in the legal field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07076v1",
            "date": "2019-12-15 17:50:56+00:00",
            "title": "Multilingual is not enough: BERT for Finnish",
            "authors": [
                "Antti Virtanen",
                "Jenna Kanerva",
                "Rami Ilo",
                "Jouni Luoma",
                "Juhani Luotolahti",
                "Tapio Salakoski",
                "Filip Ginter",
                "Sampo Pyysalo"
            ],
            "abstract": "Deep learning-based language models pretrained on large unannotated text\ncorpora have been demonstrated to allow efficient transfer learning for natural\nlanguage processing, with recent approaches such as the transformer-based BERT\nmodel advancing the state of the art across a variety of tasks. While most work\non these models has focused on high-resource languages, in particular English,\na number of recent efforts have introduced multilingual models that can be\nfine-tuned to address tasks in a large number of different languages. However,\nwe still lack a thorough understanding of the capabilities of these models, in\nparticular for lower-resourced languages. In this paper, we focus on Finnish\nand thoroughly evaluate the multilingual BERT model on a range of tasks,\ncomparing it with a new Finnish BERT model trained from scratch. The new\nlanguage-specific model is shown to systematically and clearly outperform the\nmultilingual. While the multilingual model largely fails to reach the\nperformance of previously proposed methods, the custom Finnish BERT model\nestablishes new state-of-the-art results on all corpora for all reference\ntasks: part-of-speech tagging, named entity recognition, and dependency\nparsing. We release the model and all related resources created for this study\nwith open licenses at https://turkunlp.org/finbert .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07095v1",
            "date": "2019-12-15 19:33:25+00:00",
            "title": "Robust Named Entity Recognition with Truecasing Pretraining",
            "authors": [
                "Stephen Mayhew",
                "Nitish Gupta",
                "Dan Roth"
            ],
            "abstract": "Although modern named entity recognition (NER) systems show impressive\nperformance on standard datasets, they perform poorly when presented with noisy\ndata. In particular, capitalization is a strong signal for entities in many\nlanguages, and even state of the art models overfit to this feature, with\ndrastically lower performance on uncapitalized text. In this work, we address\nthe problem of robustness of NER systems in data with noisy or uncertain\ncasing, using a pretraining objective that predicts casing in text, or a\ntruecaser, leveraging unlabeled data. The pretrained truecaser is combined with\na standard BiLSTM-CRF model for NER by appending output distributions to\ncharacter embeddings. In experiments over several datasets of varying domain\nand casing quality, we show that our new model improves performance in uncased\ntext, even adding value to uncased BERT embeddings. Our method achieves a new\nstate of the art on the WNUT17 shared task dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07199v2",
            "date": "2019-12-16 05:25:07+00:00",
            "title": "Characterizing the dynamics of learning in repeated reference games",
            "authors": [
                "Robert D. Hawkins",
                "Michael C. Frank",
                "Noah D. Goodman"
            ],
            "abstract": "The language we use over the course of conversation changes as we establish\ncommon ground and learn what our partner finds meaningful. Here we draw upon\nrecent advances in natural language processing to provide a finer-grained\ncharacterization of the dynamics of this learning process. We release an open\ncorpus (>15,000 utterances) of extended dyadic interactions in a classic\nrepeated reference game task where pairs of participants had to coordinate on\nhow to refer to initially difficult-to-describe tangram stimuli. We find that\ndifferent pairs discover a wide variety of idiosyncratic but efficient and\nstable solutions to the problem of reference. Furthermore, these conventions\nare shaped by the communicative context: words that are more discriminative in\nthe initial context (i.e. that are used for one target more than others) are\nmore likely to persist through the final repetition. Finally, we find\nsystematic structure in how a speaker's referring expressions become more\nefficient over time: syntactic units drop out in clusters following positive\nfeedback from the listener, eventually leaving short labels containing\nopen-class parts of speech. These findings provide a higher resolution look at\nthe quantitative dynamics of ad hoc convention formation and support further\ndevelopment of computational models of learning in communication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07225v1",
            "date": "2019-12-16 07:23:21+00:00",
            "title": "Graph-based Neural Sentence Ordering",
            "authors": [
                "Yongjing Yin",
                "Linfeng Song",
                "Jinsong Su",
                "Jiali Zeng",
                "Chulun Zhou",
                "Jiebo Luo"
            ],
            "abstract": "Sentence ordering is to restore the original paragraph from a set of\nsentences. It involves capturing global dependencies among sentences regardless\nof their input order. In this paper, we propose a novel and flexible\ngraph-based neural sentence ordering model, which adopts graph recurrent\nnetwork \\cite{Zhang:acl18} to accurately learn semantic representations of the\nsentences. Instead of assuming connections between all pairs of input\nsentences, we use entities that are shared among multiple sentences to make\nmore expressive graph representations with less noise. Experimental results\nshow that our proposed model outperforms the existing state-of-the-art systems\non several benchmark datasets, demonstrating the effectiveness of our model. We\nalso conduct a thorough analysis on how entities help the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07239v1",
            "date": "2019-12-16 08:21:28+00:00",
            "title": "Iterative Dual Domain Adaptation for Neural Machine Translation",
            "authors": [
                "Jiali Zeng",
                "Yang Liu",
                "Jinsong Su",
                "Yubin Ge",
                "Yaojie Lu",
                "Yongjing Yin",
                "Jiebo Luo"
            ],
            "abstract": "Previous studies on the domain adaptation for neural machine translation\n(NMT) mainly focus on the one-pass transferring out-of-domain translation\nknowledge to in-domain NMT model. In this paper, we argue that such a strategy\nfails to fully extract the domain-shared translation knowledge, and repeatedly\nutilizing corpora of different domains can lead to better distillation of\ndomain-shared translation knowledge. To this end, we propose an iterative dual\ndomain adaptation framework for NMT. Specifically, we first pre-train in-domain\nand out-of-domain NMT models using their own training corpora respectively, and\nthen iteratively perform bidirectional translation knowledge transfer (from\nin-domain to out-of-domain and then vice versa) based on knowledge distillation\nuntil the in-domain NMT model convergences. Furthermore, we extend the proposed\nframework to the scenario of multiple out-of-domain training corpora, where the\nabove-mentioned transfer is performed sequentially between the in-domain and\neach out-of-domain NMT models in the ascending order of their domain\nsimilarities. Empirical results on Chinese-English and English-German\ntranslation tasks demonstrate the effectiveness of our framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07491v1",
            "date": "2019-12-16 16:39:01+00:00",
            "title": "Improving Knowledge-aware Dialogue Generation via Knowledge Base Question Answering",
            "authors": [
                "Jian Wang",
                "Junhao Liu",
                "Wei Bi",
                "Xiaojiang Liu",
                "Kejing He",
                "Ruifeng Xu",
                "Min Yang"
            ],
            "abstract": "Neural network models usually suffer from the challenge of incorporating\ncommonsense knowledge into the open-domain dialogue systems. In this paper, we\npropose a novel knowledge-aware dialogue generation model (called TransDG),\nwhich transfers question representation and knowledge matching abilities from\nknowledge base question answering (KBQA) task to facilitate the utterance\nunderstanding and factual knowledge selection for dialogue generation. In\naddition, we propose a response guiding attention and a multi-step decoding\nstrategy to steer our model to focus on relevant features for response\ngeneration. Experiments on two benchmark datasets demonstrate that our model\nhas robust superiority over compared methods in generating informative and\nfluent dialogues. Our code is available at https://github.com/siat-nlp/TransDG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07506v1",
            "date": "2019-12-16 17:12:00+00:00",
            "title": "Scale-dependent Relationships in Natural Language",
            "authors": [
                "Aakash Sarkar",
                "Marc Howard"
            ],
            "abstract": "Natural language exhibits statistical dependencies at a wide range of scales.\nFor instance, the mutual information between words in natural language decays\nlike a power law with the temporal lag between them. However, many statistical\nlearning models applied to language impose a sampling scale while extracting\nstatistical structure. For instance, Word2Vec constructs a vector embedding\nthat maximizes the prediction between a target word and the context words that\nappear nearby in the corpus. The size of the context is chosen by the user and\ndefines a strong scale; relationships over much larger temporal scales would be\ninvisible to the algorithm. This paper examines the family of Word2Vec\nembeddings generated while systematically manipulating the sampling scale used\nto define the context around each word. The primary result is that different\nlinguistic relationships are preferentially encoded at different scales.\nDifferent scales emphasize different syntactic and semantic relations between\nwords.Moreover, the neighborhoods of a given word in the embeddings change\nsignificantly depending on the scale. These results suggest that any individual\nscale can only identify a subset of the meaningful relationships a word might\nhave, and point toward the importance of developing scale-free models of\nsemantic meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07940v1",
            "date": "2019-12-17 11:43:17+00:00",
            "title": "To What Extent are Name Variants Used as Named Entities in Turkish Tweets?",
            "authors": [
                "Dilek K\u00fc\u00e7\u00fck"
            ],
            "abstract": "Social media texts differ from regular texts in various aspects. One of the\nmain differences is the common use of informal name variants instead of\nwell-formed named entities in social media compared to regular texts. These\nname variants may come in the form of abbreviations, nicknames, contractions,\nand hypocoristic uses, in addition to names distorted due to capitalization and\nwriting errors. In this paper, we present an analysis of the named entities in\na publicly-available tweet dataset in Turkish with respect to their being name\nvariants belonging to different categories. We also provide finer-grained\nannotations of the named entities as well-formed names and different categories\nof name variants, where these annotations are made publicly-available. The\nanalysis presented and the accompanying annotations will contribute to related\nresearch on the treatment of named entities in social media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08259v1",
            "date": "2019-12-17 20:15:34+00:00",
            "title": "Open Set Authorship Attribution toward Demystifying Victorian Periodicals",
            "authors": [
                "Sarkhan Badirli",
                "Mary Borgo Ton",
                "Abdulmecit Gungor",
                "Murat Dundar"
            ],
            "abstract": "Existing research in computational authorship attribution (AA) has primarily\nfocused on attribution tasks with a limited number of authors in a closed-set\nconfiguration. This restricted set-up is far from being realistic in dealing\nwith highly entangled real-world AA tasks that involve a large number of\ncandidate authors for attribution during test time. In this paper, we study AA\nin historical texts using anew data set compiled from the Victorian literature.\nWe investigate the predictive capacity of most common English words in\ndistinguishing writings of most prominent Victorian novelists. We challenged\nthe closed-set classification assumption and discussed the limitations of\nstandard machine learning techniques in dealing with the open set AA task. Our\nexperiments suggest that a linear classifier can achieve near perfect\nattribution accuracy under closed set assumption yet, the need for more robust\napproaches becomes evident once a large candidate pool has to be considered in\nthe open-set classification setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08282v2",
            "date": "2019-12-17 21:38:22+00:00",
            "title": "Chinese Named Entity Recognition Augmented with Lexicon Memory",
            "authors": [
                "Yi Zhou",
                "Xiaoqing Zheng",
                "Xuanjing Huang"
            ],
            "abstract": "Inspired by a concept of content-addressable retrieval from cognitive\nscience, we propose a novel fragment-based model augmented with a lexicon-based\nmemory for Chinese NER, in which both the character-level and word-level\nfeatures are combined to generate better feature representations for possible\nname candidates. It is observed that locating the boundary information of\nentity names is useful in order to classify them into pre-defined categories.\nPosition-dependent features, including prefix and suffix are introduced for NER\nin the form of distributed representation. The lexicon-based memory is used to\nhelp generate such position-dependent features and deal with the problem of\nout-of-vocabulary words. Experimental results showed that the proposed model,\ncalled LEMON, achieved state-of-the-art on four datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08290v1",
            "date": "2019-12-17 21:58:51+00:00",
            "title": "The performance evaluation of Multi-representation in the Deep Learning models for Relation Extraction Task",
            "authors": [
                "Jefferson A. Pe\u00f1a Torres",
                "Raul Ernesto Gutierrez",
                "Victor A. Bucheli",
                "Fabio A. Gonzalez O"
            ],
            "abstract": "Single implementing, concatenating, adding or replacing of the\nrepresentations has yielded significant improvements on many NLP tasks. Mainly\nin Relation Extraction where static, contextualized and others representations\nthat are capable of explaining word meanings through the linguistic features\nthat these incorporates. In this work addresses the question of how is improved\nthe relation extraction using different types of representations generated by\npretrained language representation models. We benchmarked our approach using\npopular word representation models, replacing and concatenating static,\ncontextualized and others representations of hand-extracted features. The\nexperiments show that representation is a crucial element to choose when DL\napproach is applied. Word embeddings from Flair and BERT can be well\ninterpreted by a deep learning model for RE task, and replacing static word\nembeddings with contextualized word representations could lead to significant\nimprovements. While, the hand-created representations requires is\ntime-consuming and not is ensure a improve in combination with others\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08360v1",
            "date": "2019-12-18 03:09:12+00:00",
            "title": "DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog",
            "authors": [
                "Feilong Chen",
                "Fandong Meng",
                "Jiaming Xu",
                "Peng Li",
                "Bo Xu",
                "Jie Zhou"
            ],
            "abstract": "Visual Dialog is a vision-language task that requires an AI agent to engage\nin a conversation with humans grounded in an image. It remains a challenging\ntask since it requires the agent to fully understand a given question before\nmaking an appropriate response not only from the textual dialog history, but\nalso from the visually-grounded information. While previous models typically\nleverage single-hop reasoning or single-channel reasoning to deal with this\ncomplex multimodal reasoning task, which is intuitively insufficient. In this\npaper, we thus propose a novel and more powerful Dual-channel Multi-hop\nReasoning Model for Visual Dialog, named DMRM. DMRM synchronously captures\ninformation from the dialog history and the image to enrich the semantic\nrepresentation of the question by exploiting dual-channel reasoning.\nSpecifically, DMRM maintains a dual channel to obtain the question- and\nhistory-aware image features and the question- and image-aware dialog history\nfeatures by a mulit-hop reasoning process in each channel. Additionally, we\nalso design an effective multimodal attention to further enhance the decoder to\ngenerate more accurate responses. Experimental results on the VisDial v0.9 and\nv1.0 datasets demonstrate that the proposed model is effective and outperforms\ncompared models by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08492v1",
            "date": "2019-12-18 10:05:49+00:00",
            "title": "Generating summaries tailored to target characteristics",
            "authors": [
                "Kushal Chawla",
                "Hrituraj Singh",
                "Arijit Pramanik",
                "Mithlesh Kumar",
                "Balaji Vasan Srinivasan"
            ],
            "abstract": "Recently, research efforts have gained pace to cater to varied user\npreferences while generating text summaries. While there have been attempts to\nincorporate a few handpicked characteristics such as length or entities, a\nholistic view around these preferences is missing and crucial insights on why\ncertain characteristics should be incorporated in a specific manner are absent.\nWith this objective, we provide a categorization around these characteristics\nrelevant to the task of text summarization: one, focusing on what content needs\nto be generated and second, focusing on the stylistic aspects of the output\nsummaries. We use our insights to provide guidelines on appropriate methods to\nincorporate various classes characteristics in sequence-to-sequence\nsummarization framework. Our experiments with incorporating topics, readability\nand simplicity indicate the viability of the proposed prescriptions",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08494v3",
            "date": "2019-12-18 10:07:20+00:00",
            "title": "A Survey on Document-level Neural Machine Translation: Methods and Evaluation",
            "authors": [
                "Sameen Maruf",
                "Fahimeh Saleh",
                "Gholamreza Haffari"
            ],
            "abstract": "Machine translation (MT) is an important task in natural language processing\n(NLP) as it automates the translation process and reduces the reliance on human\ntranslators. With the resurgence of neural networks, the translation quality\nsurpasses that of the translations obtained using statistical techniques for\nmost language-pairs. Up until a few years ago, almost all of the neural\ntranslation models translated sentences independently, without incorporating\nthe wider document-context and inter-dependencies among the sentences. The aim\nof this survey paper is to highlight the major works that have been undertaken\nin the space of document-level machine translation after the neural revolution,\nso that researchers can recognise the current state and future directions of\nthis field. We provide an organisation of the literature based on novelties in\nmodelling and architectures as well as training and decoding strategies. In\naddition, we cover evaluation strategies that have been introduced to account\nfor the improvements in document MT, including automatic metrics and\ndiscourse-targeted test sets. We conclude by presenting possible avenues for\nfuture exploration in this research field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08777v3",
            "date": "2019-12-18 18:16:20+00:00",
            "title": "PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization",
            "authors": [
                "Jingqing Zhang",
                "Yao Zhao",
                "Mohammad Saleh",
                "Peter J. Liu"
            ],
            "abstract": "Recent work pre-training Transformers with self-supervised objectives on\nlarge text corpora has shown great success when fine-tuned on downstream NLP\ntasks including text summarization. However, pre-training objectives tailored\nfor abstractive text summarization have not been explored. Furthermore there is\na lack of systematic evaluation across diverse domains. In this work, we\npropose pre-training large Transformer-based encoder-decoder models on massive\ntext corpora with a new self-supervised objective. In PEGASUS, important\nsentences are removed/masked from an input document and are generated together\nas one output sequence from the remaining sentences, similar to an extractive\nsummary. We evaluated our best PEGASUS model on 12 downstream summarization\ntasks spanning news, science, stories, instructions, emails, patents, and\nlegislative bills. Experiments demonstrate it achieves state-of-the-art\nperformance on all 12 downstream datasets measured by ROUGE scores. Our model\nalso shows surprising performance on low-resource summarization, surpassing\nprevious state-of-the-art results on 6 datasets with only 1000 examples.\nFinally we validated our results using human evaluation and show that our model\nsummaries achieve human performance on multiple datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08960v1",
            "date": "2019-12-19 00:27:40+00:00",
            "title": "Going Beneath the Surface: Evaluating Image Captioning for Grammaticality, Truthfulness and Diversity",
            "authors": [
                "Huiyuan Xie",
                "Tom Sherborne",
                "Alexander Kuhnle",
                "Ann Copestake"
            ],
            "abstract": "Image captioning as a multimodal task has drawn much interest in recent\nyears. However, evaluation for this task remains a challenging problem.\nExisting evaluation metrics focus on surface similarity between a candidate\ncaption and a set of reference captions, and do not check the actual relation\nbetween a caption and the underlying visual content. We introduce a new\ndiagnostic evaluation framework for the task of image captioning, with the goal\nof directly assessing models for grammaticality, truthfulness and diversity\n(GTD) of generated captions. We demonstrate the potential of our evaluation\nframework by evaluating existing image captioning models on a wide ranging set\nof synthetic datasets that we construct for diagnostic evaluation. We\nempirically show how the GTD evaluation framework, in combination with\ndiagnostic datasets, can provide insights into model capabilities and\nlimitations to supplement standard evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08981v1",
            "date": "2019-12-19 01:45:36+00:00",
            "title": "Identifying Adversarial Sentences by Analyzing Text Complexity",
            "authors": [
                "Hoang-Quoc Nguyen-Son",
                "Tran Phuong Thao",
                "Seira Hidano",
                "Shinsaku Kiyomoto"
            ],
            "abstract": "Attackers create adversarial text to deceive both human perception and the\ncurrent AI systems to perform malicious purposes such as spam product reviews\nand fake political posts. We investigate the difference between the adversarial\nand the original text to prevent the risk. We prove that the text written by a\nhuman is more coherent and fluent. Moreover, the human can express the idea\nthrough the flexible text with modern words while a machine focuses on\noptimizing the generated text by the simple and common words. We also suggest a\nmethod to identify the adversarial text by extracting the features related to\nour findings. The proposed method achieves high performance with 82.0% of\naccuracy and 18.4% of equal error rate, which is better than the existing\nmethods whose the best accuracy is 77.0% corresponding to the error rate 22.8%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09008v1",
            "date": "2019-12-19 03:48:05+00:00",
            "title": "Discriminative Sentence Modeling for Story Ending Prediction",
            "authors": [
                "Yiming Cui",
                "Wanxiang Che",
                "Wei-Nan Zhang",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Story Ending Prediction is a task that needs to select an appropriate ending\nfor the given story, which requires the machine to understand the story and\nsometimes needs commonsense knowledge. To tackle this task, we propose a new\nneural network called Diff-Net for better modeling the differences of each\nending in this task. The proposed model could discriminate two endings in three\nsemantic levels: contextual representation, story-aware representation, and\ndiscriminative representation. Experimental results on the Story Cloze Test\ndataset show that the proposed model siginificantly outperforms various systems\nby a large margin, and detailed ablation studies are given for better\nunderstanding our model. We also carefully examine the traditional and\nBERT-based models on both SCT v1.0 and v1.5 with interesting findings that may\npotentially help future studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09084v1",
            "date": "2019-12-19 09:40:19+00:00",
            "title": "Neural Simile Recognition with Cyclic Multitask Learning and Local Attention",
            "authors": [
                "Jiali Zeng",
                "Linfeng Song",
                "Jinsong Su",
                "Jun Xie",
                "Wei Song",
                "Jiebo Luo"
            ],
            "abstract": "Simile recognition is to detect simile sentences and to extract simile\ncomponents, i.e., tenors and vehicles. It involves two subtasks: {\\it simile\nsentence classification} and {\\it simile component extraction}. Recent work has\nshown that standard multitask learning is effective for Chinese simile\nrecognition, but it is still uncertain whether the mutual effects between the\nsubtasks have been well captured by simple parameter sharing. We propose a\nnovel cyclic multitask learning framework for neural simile recognition, which\nstacks the subtasks and makes them into a loop by connecting the last to the\nfirst. It iteratively performs each subtask, taking the outputs of the previous\nsubtask as additional inputs to the current one, so that the interdependence\nbetween the subtasks can be better explored. Extensive experiments show that\nour framework significantly outperforms the current state-of-the-art model and\nour carefully designed baselines, and the gains are still remarkable using\nBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09152v1",
            "date": "2019-12-19 12:05:36+00:00",
            "title": "Annotating and normalizing biomedical NEs with limited knowledge",
            "authors": [
                "Fernando S\u00e1nchez Le\u00f3n",
                "Ana Gonz\u00e1lez Ledesma"
            ],
            "abstract": "Named entity recognition (NER) is the very first step in the linguistic\nprocessing of any new domain. It is currently a common process in BioNLP on\nEnglish clinical text. However, it is still in its infancy in other major\nlanguages, as it is the case for Spanish. Presented under the umbrella of the\nPharmaCoNER shared task, this paper describes a very simple method for the\nannotation and normalization of pharmacological, chemical and, ultimately,\nbiomedical named entities in clinical cases. The system developed for the\nshared task is based on limited knowledge, collected, structured and munged in\na way that clearly outperforms scores obtained by similar dictionary-based\nsystems for English in the past. Along with this recovering of the\nknowledge-based methods for NER in subdomains, the paper also highlights the\nkey contribution of resource-based systems in the validation and consolidation\nof both the annotation guidelines and the human annotation practices. In this\nsense, some of the authors discoverings on the overall quality of human\nannotated datasets question the above-mentioned `official' results obtained by\nthis system, that ranked second (0.91 F1-score) and first (0.916 F1-score),\nrespectively, in the two PharmaCoNER subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09297v2",
            "date": "2019-12-19 15:32:19+00:00",
            "title": "An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classification",
            "authors": [
                "Yue Ma",
                "Zengfeng Zeng",
                "Dawei Zhu",
                "Xuan Li",
                "Yiying Yang",
                "Xiaoyuan Yao",
                "Kaijie Zhou",
                "Jianping Shen"
            ],
            "abstract": "This paper describes our approach in DSTC 8 Track 4: Schema-Guided Dialogue\nState Tracking. The goal of this task is to predict the intents and slots in\neach user turn to complete the dialogue state tracking (DST) based on the\ninformation provided by the task's schema. Different from traditional\nstage-wise DST, we propose an end-to-end DST system to avoid error accumulation\nbetween the dialogue turns. The DST system consists of a machine reading\ncomprehension (MRC) model for non-categorical slots and a Wide & Deep model for\ncategorical slots. As far as we know, this is the first time that MRC and Wide\n& Deep model are applied to DST problem in a fully end-to-end way. Experimental\nresults show that our framework achieves an excellent performance on the test\ndataset including 50% zero-shot services with a joint goal accuracy of 0.8652\nand a slot tagging F1-Score of 0.9835.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09558v2",
            "date": "2019-12-19 21:47:08+00:00",
            "title": "RIMAX: Ranking Semantic Rhymes by calculating Definition Similarity",
            "authors": [
                "Alfonso Medina-Urrea",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "This paper presents RIMAX, a new system for detecting semantic rhymes, using\na Comprehensive Mexican Spanish Dictionary (DEM) and its Rhyming Dictionary\n(REM). We use the Vector Space Model to calculate the similarity of the\ndefinition of a query with the definitions corresponding to the assonant and\nconsonant rhymes of the query. The preliminary results using a manual\nevaluation are very encouraging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09582v1",
            "date": "2019-12-19 22:59:26+00:00",
            "title": "BERTje: A Dutch BERT Model",
            "authors": [
                "Wietse de Vries",
                "Andreas van Cranenburgh",
                "Arianna Bisazza",
                "Tommaso Caselli",
                "Gertjan van Noord",
                "Malvina Nissim"
            ],
            "abstract": "The transformer-based pre-trained language model BERT has helped to improve\nstate-of-the-art performance on many natural language processing (NLP) tasks.\nUsing the same architecture and parameters, we developed and evaluated a\nmonolingual Dutch BERT model called BERTje. Compared to the multilingual BERT\nmodel, which includes Dutch but is only based on Wikipedia text, BERTje is\nbased on a large and diverse dataset of 2.4 billion tokens. BERTje consistently\noutperforms the equally-sized multilingual BERT model on downstream NLP tasks\n(part-of-speech tagging, named-entity recognition, semantic role labeling, and\nsentiment analysis). Our pre-trained Dutch BERT model is made available at\nhttps://github.com/wietsedv/bertje.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09637v1",
            "date": "2019-12-20 04:25:48+00:00",
            "title": "Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model",
            "authors": [
                "Wenhan Xiong",
                "Jingfei Du",
                "William Yang Wang",
                "Veselin Stoyanov"
            ],
            "abstract": "Recent breakthroughs of pretrained language models have shown the\neffectiveness of self-supervised learning for a wide range of natural language\nprocessing (NLP) tasks. In addition to standard syntactic and semantic NLP\ntasks, pretrained models achieve strong improvements on tasks that involve\nreal-world knowledge, suggesting that large-scale language modeling could be an\nimplicit method to capture knowledge. In this work, we further investigate the\nextent to which pretrained models such as BERT capture knowledge using a\nzero-shot fact completion task. Moreover, we propose a simple yet effective\nweakly supervised pretraining objective, which explicitly forces the model to\nincorporate knowledge about real-world entities. Models trained with our new\nobjective yield significant improvements on the fact completion task. When\napplied to downstream tasks, our model consistently outperforms BERT on four\nentity-related question answering datasets (i.e., WebQuestions, TriviaQA,\nSearchQA and Quasar-T) with an average 2.7 F1 improvements and a standard\nfine-grained entity typing dataset (i.e., FIGER) with 5.7 accuracy gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09723v3",
            "date": "2019-12-20 09:44:42+00:00",
            "title": "SberQuAD -- Russian Reading Comprehension Dataset: Description and Analysis",
            "authors": [
                "Pavel Efimov",
                "Andrey Chertok",
                "Leonid Boytsov",
                "Pavel Braslavski"
            ],
            "abstract": "SberQuAD -- a large scale analog of Stanford SQuAD in the Russian language -\nis a valuable resource that has not been properly presented to the scientific\ncommunity. We fill this gap by providing a description, a thorough analysis,\nand baseline experimental results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09879v1",
            "date": "2019-12-20 15:25:57+00:00",
            "title": "When to Talk: Chatbot Controls the Timing of Talking during Multi-turn Open-domain Dialogue Generation",
            "authors": [
                "Tian Lan",
                "Xianling Mao",
                "Heyan Huang",
                "Wei Wei"
            ],
            "abstract": "Despite the multi-turn open-domain dialogue systems have attracted more and\nmore attention and made great progress, the existing dialogue systems are still\nvery boring. Nearly all the existing dialogue models only provide a response\nwhen the user's utterance is accepted. But during daily conversations, humans\nalways decide whether to continue to utter an utterance based on the context.\nIntuitively, a dialogue model that can control the timing of talking\nautonomously based on the conversation context can chat with humans more\nnaturally. In this paper, we explore the dialogue system that automatically\ncontrols the timing of talking during the conversation. Specifically, we adopt\nthe decision module for the existing dialogue models. Furthermore, modeling\nconversation context effectively is very important for controlling the timing\nof talking. So we also adopt the graph neural networks to process the context\nwith the natural graph structure. Extensive experiments on two benchmarks show\nthat controlling the timing of talking can effectively improve the quality of\ndialogue generation, and the proposed methods significantly improve the\naccuracy of the timing of talking. In addition, we have publicly released the\ncodes of our proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09913v2",
            "date": "2019-12-20 16:24:23+00:00",
            "title": "Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks",
            "authors": [
                "Minh Nguyen",
                "Gia H. Ngo",
                "Nancy F. Chen"
            ],
            "abstract": "Logographs (Chinese characters) have recursive structures (i.e. hierarchies\nof sub-units in logographs) that contain phonological and semantic information,\nas developmental psychology literature suggests that native speakers leverage\non the structures to learn how to read. Exploiting these structures could\npotentially lead to better embeddings that can benefit many downstream tasks.\nWe propose building hierarchical logograph (character) embeddings from\nlogograph recursive structures using treeLSTM, a recursive neural network.\nUsing recursive neural network imposes a prior on the mapping from logographs\nto embeddings since the network must read in the sub-units in logographs\naccording to the order specified by the recursive structures. Based on human\nbehavior in language learning and reading, we hypothesize that modeling\nlogographs' structures using recursive neural network should be beneficial. To\nverify this claim, we consider two tasks (1) predicting logographs' Cantonese\npronunciation from logographic structures and (2) language modeling. Empirical\nresults show that the proposed hierarchical embeddings outperform baseline\napproaches. Diagnostic analysis suggests that hierarchical embeddings\nconstructed using treeLSTM is less sensitive to distractors, thus is more\nrobust, especially on complex logographs.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  vol. 28, pp. 461-473, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10130v1",
            "date": "2019-12-20 22:53:18+00:00",
            "title": "Modeling Intent, Dialog Policies and Response Adaptation for Goal-Oriented Interactions",
            "authors": [
                "Saurav Sahay",
                "Shachi H Kumar",
                "Eda Okur",
                "Haroon Syed",
                "Lama Nachman"
            ],
            "abstract": "Building a machine learning driven spoken dialog system for goal-oriented\ninteractions involves careful design of intents and data collection along with\ndevelopment of intent recognition models and dialog policy learning algorithms.\nThe models should be robust enough to handle various user distractions during\nthe interaction flow and should steer the user back into an engaging\ninteraction for successful completion of the interaction. In this work, we have\ndesigned a goal-oriented interaction system where children can engage with\nagents for a series of interactions involving `Meet \\& Greet' and `Simon Says'\ngame play. We have explored various feature extractors and models for improved\nintent recognition and looked at leveraging previous user and system\ninteractions in novel ways with attention models. We have also looked at dialog\nadaptation methods for entrained response selection. Our bootstrapped models\nfrom limited training data perform better than many baseline approaches we have\nlooked at for intent recognition and dialog action prediction.",
            "journal ref": "Proceedings of the 23rd Workshop on the Semantics and Pragmatics\n  of Dialogue (SEMDIAL), pp. 146-155, London, United Kingdom, September 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10132v1",
            "date": "2019-12-20 22:56:54+00:00",
            "title": "Exploring Context, Attention and Audio Features for Audio Visual Scene-Aware Dialog",
            "authors": [
                "Shachi H Kumar",
                "Eda Okur",
                "Saurav Sahay",
                "Jonathan Huang",
                "Lama Nachman"
            ],
            "abstract": "We are witnessing a confluence of vision, speech and dialog system\ntechnologies that are enabling the IVAs to learn audio-visual groundings of\nutterances and have conversations with users about the objects, activities and\nevents surrounding them. Recent progress in visual grounding techniques and\nAudio Understanding are enabling machines to understand shared semantic\nconcepts and listen to the various sensory events in the environment. With\naudio and visual grounding methods, end-to-end multimodal SDS are trained to\nmeaningfully communicate with us in natural language about the real dynamic\naudio-visual sensory world around us. In this work, we explore the role of\n`topics' as the context of the conversation along with multimodal attention\ninto such an end-to-end audio-visual scene-aware dialog system architecture. We\nalso incorporate an end-to-end audio classification ConvNet, AclNet, into our\nmodels. We develop and test our approaches on the Audio Visual Scene-Aware\nDialog (AVSD) dataset released as a part of the DSTC7. We present the analysis\nof our experiments and show that some of our model variations outperform the\nbaseline system released for AVSD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10161v1",
            "date": "2019-12-04 14:43:46+00:00",
            "title": "Implicit Knowledge in Argumentative Texts: An Annotated Corpus",
            "authors": [
                "Maria Becker",
                "Katharina Korfhage",
                "Anette Frank"
            ],
            "abstract": "When speaking or writing, people omit information that seems clear and\nevident, such that only part of the message is expressed in words. Especially\nin argumentative texts it is very common that (important) parts of the argument\nare implied and omitted. We hypothesize that for argument analysis it will be\nbeneficial to reconstruct this implied information. As a starting point for\nfilling such knowledge gaps, we build a corpus consisting of high-quality human\nannotations of missing and implied information in argumentative texts. To learn\nmore about the characteristics of both the argumentative texts and the added\ninformation, we further annotate the data with semantic clause types and\ncommonsense knowledge relations. The outcome of our work is a carefully\nde-signed and richly annotated dataset, for which we then provide an in-depth\nanalysis by investigating characteristic distributions and correlations of the\nassigned labels. We reveal interesting patterns and intersections between the\nannotation categories and properties of our dataset, which enable insights into\nthe characteristics of both argumentative texts and implicit knowledge in terms\nof structural features and semantic information. The results of our analysis\ncan help to assist automated argument analysis and can guide the process of\nrevealing implicit information in argumentative texts automatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10164v1",
            "date": "2019-12-06 09:36:15+00:00",
            "title": "Decomposing predictability: Semantic feature overlap between words and the dynamics of reading for meaning",
            "authors": [
                "Markus J. Hofmann",
                "Mareike A. Kleemann",
                "Andre Roelke",
                "Christian Vorstius",
                "Ralph Radach"
            ],
            "abstract": "The present study uses a computational approach to examine the role of\nsemantic constraints in normal reading. This methodology avoids confounds\ninherent in conventional measures of predictability, allowing for theoretically\ndeeper accounts of semantic processing. We start from a definition of\nassociations between words based on the significant log likelihood that two\nwords co-occur frequently together in the sentences of a large text corpus.\nDirect associations between stimulus words were controlled, and semantic\nfeature overlap between prime and target words was manipulated by their common\nassociates. The stimuli consisted of sentences of the form pronoun, verb,\narticle, adjective and noun, followed by a series of closed class words, e. g.\n\"She rides the grey elephant on one of her many exploratory voyages\". The\nresults showed that verb-noun overlap reduces single and first fixation\ndurations of the target noun and adjective-noun overlap reduces go-past\ndurations. A dynamic spreading of activation account suggests that associates\nof the prime words take some time to become activated: The verb can act on the\ntarget noun's early eye-movement measures presented three words later, while\nthe adjective is presented immediately prior to the target, which induces\nsentence re-examination after a difficult adjective-noun semantic integration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10165v1",
            "date": "2019-12-10 17:01:34+00:00",
            "title": "Zero-shot Text Classification With Generative Language Models",
            "authors": [
                "Raul Puri",
                "Bryan Catanzaro"
            ],
            "abstract": "This work investigates the use of natural language to enable zero-shot model\nadaptation to new tasks. We use text and metadata from social commenting\nplatforms as a source for a simple pretraining task. We then provide the\nlanguage model with natural language descriptions of classification tasks as\ninput and train it to generate the correct answer in natural language via a\nlanguage modeling objective. This allows the model to generalize to new\nclassification tasks without the need for multiple multitask classification\nheads. We show the zero-shot performance of these generative language models,\ntrained with weak supervision, on six benchmark text classification datasets\nfrom the torchtext library. Despite no access to training data, we achieve up\nto a 45% absolute improvement in classification accuracy over random or\nmajority class baselines. These results show that natural language can serve as\nsimple and powerful descriptors for task adaptation. We believe this points the\nway to new metalearning strategies for text problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10169v1",
            "date": "2019-12-15 11:42:32+00:00",
            "title": "A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings",
            "authors": [
                "Niels van der Heijden",
                "Samira Abnar",
                "Ekaterina Shutova"
            ],
            "abstract": "The lack of annotated data in many languages is a well-known challenge within\nthe field of multilingual natural language processing (NLP). Therefore, many\nrecent studies focus on zero-shot transfer learning and joint training across\nlanguages to overcome data scarcity for low-resource languages. In this work we\n(i) perform a comprehensive comparison of state-ofthe-art multilingual word and\nsentence encoders on the tasks of named entity recognition (NER) and part of\nspeech (POS) tagging; and (ii) propose a new method for creating multilingual\ncontextualized word embeddings, compare it to multiple baselines and show that\nit performs at or above state-of-theart level in zero-shot transfer settings.\nFinally, we show that our method allows for better knowledge sharing across\nlanguages in a joint training setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10590v1",
            "date": "2019-12-23 02:28:10+00:00",
            "title": "Knowledge-guided Convolutional Networks for Chemical-Disease Relation Extraction",
            "authors": [
                "Huiwei Zhou",
                "Chengkun Lang",
                "Zhuang Liu",
                "Shixian Ning",
                "Yingyu Lin",
                "Lei Du"
            ],
            "abstract": "Background: Automatic extraction of chemical-disease relations (CDR) from\nunstructured text is of essential importance for disease treatment and drug\ndevelopment. Meanwhile, biomedical experts have built many highly-structured\nknowledge bases (KBs), which contain prior knowledge about chemicals and\ndiseases. Prior knowledge provides strong support for CDR extraction. How to\nmake full use of it is worth studying. Results: This paper proposes a novel\nmodel called \"Knowledge-guided Convolutional Networks (KCN)\" to leverage prior\nknowledge for CDR extraction. The proposed model first learns knowledge\nrepresentations including entity embeddings and relation embeddings from KBs.\nThen, entity embeddings are used to control the propagation of context features\ntowards a chemical-disease pair with gated convolutions. After that, relation\nembeddings are employed to further capture the weighted context features by a\nshared attention pooling. Finally, the weighted context features containing\nadditional knowledge information are used for CDR extraction. Experiments on\nthe BioCreative V CDR dataset show that the proposed KCN achieves 71.28%\nF1-score, which outperforms most of the state-of-the-art systems. Conclusions:\nThis paper proposes a novel CDR extraction model KCN to make full use of prior\nknowledge. Experimental results demonstrate that KCN could effectively\nintegrate prior knowledge and contexts for the performance improvement.",
            "journal ref": "BMC Bioinformatics, 2019, 20(1):260",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10604v1",
            "date": "2019-12-23 03:34:13+00:00",
            "title": "Combining Context and Knowledge Representations for Chemical-Disease Relation Extraction",
            "authors": [
                "Huiwei Zhou",
                "Yunlong Yang",
                "Shixian Ning",
                "Zhuang Liu",
                "Chengkun Lang",
                "Yingyu Lin",
                "Degen Huang"
            ],
            "abstract": "Automatically extracting the relationships between chemicals and diseases is\nsignificantly important to various areas of biomedical research and health\ncare. Biomedical experts have built many large-scale knowledge bases (KBs) to\nadvance the development of biomedical research. KBs contain huge amounts of\nstructured information about entities and relationships, therefore plays a\npivotal role in chemical-disease relation (CDR) extraction. However, previous\nresearches pay less attention to the prior knowledge existing in KBs. This\npaper proposes a neural network-based attention model (NAM) for CDR extraction,\nwhich makes full use of context information in documents and prior knowledge in\nKBs. For a pair of entities in a document, an attention mechanism is employed\nto select important context words with respect to the relation representations\nlearned from KBs. Experiments on the BioCreative V CDR dataset show that\ncombining context and knowledge representations through the attention\nmechanism, could significantly improve the CDR extraction performance while\nachieve comparable results with state-of-the-art systems.",
            "journal ref": "IEEE/ACM TCBB,2018,16(6):1879-1889",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10616v3",
            "date": "2019-12-23 04:38:00+00:00",
            "title": "Siamese Networks for Large-Scale Author Identification",
            "authors": [
                "Chakaveh Saedi",
                "Mark Dras"
            ],
            "abstract": "Authorship attribution is the process of identifying the author of a text.\nApproaches to tackling it have been conventionally divided into\nclassification-based ones, which work well for small numbers of candidate\nauthors, and similarity-based methods, which are applicable for larger numbers\nof authors or for authors beyond the training set; these existing\nsimilarity-based methods have only embodied static notions of similarity. Deep\nlearning methods, which blur the boundaries between classification-based and\nsimilarity-based approaches, are promising in terms of ability to learn a\nnotion of similarity, but have previously only been used in a conventional\nsmall-closed-class classification setup.\n  Siamese networks have been used to develop learned notions of similarity in\none-shot image tasks, and also for tasks of mostly semantic relatedness in NLP.\nWe examine their application to the stylistic task of authorship attribution on\ndatasets with large numbers of authors, looking at multiple energy functions\nand neural network architectures, and show that they can substantially\noutperform previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11046v1",
            "date": "2019-12-24 03:34:58+00:00",
            "title": "Improving Abstractive Text Summarization with History Aggregation",
            "authors": [
                "Pengcheng Liao",
                "Chuang Zhang",
                "Xiaojun Chen",
                "Xiaofei Zhou"
            ],
            "abstract": "Recent neural sequence to sequence models have provided feasible solutions\nfor abstractive summarization. However, such models are still hard to tackle\nlong text dependency in the summarization task. A high-quality summarization\nsystem usually depends on strong encoder which can refine important information\nfrom long input texts so that the decoder can generate salient summaries from\nthe encoder's memory. In this paper, we propose an aggregation mechanism based\non the Transformer model to address the challenge of long text representation.\nOur model can review history information to make encoder hold more memory\ncapacity. Empirically, we apply our aggregation mechanism to the Transformer\nmodel and experiment on CNN/DailyMail dataset to achieve higher quality\nsummaries compared to several strong baseline models on the ROUGE metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11270v7",
            "date": "2019-12-24 09:53:27+00:00",
            "title": "Falcon 2.0: An Entity and Relation Linking Tool over Wikidata",
            "authors": [
                "Ahmad Sakor",
                "Kuldeep Singh",
                "Anery Patel",
                "Maria-Esther Vidal"
            ],
            "abstract": "The Natural Language Processing (NLP) community has significantly contributed\nto the solutions for entity and relation recognition from the text, and\npossibly linking them to proper matches in Knowledge Graphs (KGs). Considering\nWikidata as the background KG, still, there are limited tools to link knowledge\nwithin the text to Wikidata. In this paper, we present Falcon 2.0, first joint\nentity, and relation linking tool over Wikidata. It receives a short natural\nlanguage text in the English language and outputs a ranked list of entities and\nrelations annotated with the proper candidates in Wikidata. The candidates are\nrepresented by their Internationalized Resource Identifier (IRI) in Wikidata.\nFalcon 2.0 resorts to the English language model for the recognition task\n(e.g., N-Gram tiling and N-Gram splitting), and then an optimization approach\nfor linking task. We have empirically studied the performance of Falcon 2.0 on\nWikidata and concluded that it outperforms all the existing baselines. Falcon\n2.0 is public and can be reused by the community; all the required instructions\nof Falcon 2.0 are well-documented at our GitHub repository. We also demonstrate\nan online API, which can be run without any technical expertise. Falcon 2.0 and\nits background knowledge bases are available as resources at\nhttps://labs.tib.eu/falcon/falcon2/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11334v1",
            "date": "2019-12-08 10:19:40+00:00",
            "title": "Open-domain Event Extraction and Embedding for Natural Gas Market Prediction",
            "authors": [
                "Minh Triet Chau",
                "Diego Esteves",
                "Jens Lehmann"
            ],
            "abstract": "We propose an approach to predict the natural gas price in several days using\nhistorical price data and events extracted from news headlines. Most previous\nmethods treats price as an extrapolatable time series, those analyze the\nrelation between prices and news either trim their price data correspondingly\nto a public news dataset, manually annotate headlines or use off-the-shelf\ntools. In comparison to off-the-shelf tools, our event extraction method\ndetects not only the occurrence of phenomena but also the changes in\nattribution and characteristics from public sources. Instead of using sentence\nembedding as a feature, we use every word of the extracted events, encode and\norganize them before feeding to the learning models. Empirical results show\nfavorable results, in terms of prediction performance, money saved and\nscalability.",
            "journal ref": "CLEOPATRA 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11602v4",
            "date": "2019-12-25 06:05:44+00:00",
            "title": "Leveraging Lead Bias for Zero-shot Abstractive News Summarization",
            "authors": [
                "Chenguang Zhu",
                "Ziyi Yang",
                "Robert Gmyr",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "A typical journalistic convention in news articles is to deliver the most\nsalient information in the beginning, also known as the lead bias. While this\nphenomenon can be exploited in generating a summary, it has a detrimental\neffect on teaching a model to discriminate and extract important information in\ngeneral. We propose that this lead bias can be leveraged in our favor in a\nsimple and effective way to pre-train abstractive news summarization models on\nlarge-scale unlabeled news corpora: predicting the leading sentences using the\nrest of an article. We collect a massive news corpus and conduct data cleaning\nand filtering via statistical analysis. We then apply self-supervised\npre-training on this dataset to existing generation models BART and T5 for\ndomain adaptation. Via extensive experiments on six benchmark datasets, we show\nthat this approach can dramatically improve the summarization quality and\nachieve state-of-the-art results for zero-shot news summarization without any\nfine-tuning. For example, in the DUC2003 dataset, the ROUGE-1 score of BART\nincreases 13.7% after the lead-bias pre-training. We deploy the model in\nMicrosoft News and provide public APIs as well as a demo website for\nmulti-lingual news summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11625v1",
            "date": "2019-12-25 08:50:45+00:00",
            "title": "A Study of Multilingual Neural Machine Translation",
            "authors": [
                "Xu Tan",
                "Yichong Leng",
                "Jiale Chen",
                "Yi Ren",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Multilingual neural machine translation (NMT) has recently been investigated\nfrom different aspects (e.g., pivot translation, zero-shot translation,\nfine-tuning, or training from scratch) and in different settings (e.g., rich\nresource and low resource, one-to-many, and many-to-one translation). This\npaper concentrates on a deep understanding of multilingual NMT and conducts a\ncomprehensive study on a multilingual dataset with more than 20 languages. Our\nresults show that (1) low-resource language pairs benefit much from\nmultilingual training, while rich-resource language pairs may get hurt under\nlimited model capacity and training with similar languages benefits more than\ndissimilar languages; (2) fine-tuning performs better than training from\nscratch in the one-to-many setting while training from scratch performs better\nin the many-to-one setting; (3) the bottom layers of the encoder and top layers\nof the decoder capture more language-specific information, and just fine-tuning\nthese parts can achieve good accuracy for low-resource language pairs; (4)\ndirect translation is better than pivot translation when the source language is\nsimilar to the target language (e.g., in the same language branch), even when\nthe size of direct training data is much smaller; (5) given a fixed training\ndata budget, it is better to introduce more languages into multilingual\ntraining for zero-shot translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11973v2",
            "date": "2019-12-27 03:20:48+00:00",
            "title": "Language Independent Sentiment Analysis",
            "authors": [
                "Muhammad Haroon Shakeel",
                "Turki Alghamidi",
                "Safi Faizullah",
                "Imdadullah Khan"
            ],
            "abstract": "Social media platforms and online forums generate rapid and increasing amount\nof textual data. Businesses, government agencies, and media organizations seek\nto perform sentiment analysis on this rich text data. The results of these\nanalytics are used for adapting marketing strategies, customizing products,\nsecurity and various other decision makings. Sentiment analysis has been\nextensively studied and various methods have been developed for it with great\nsuccess. These methods, however apply to texts written in a specific language.\nThis limits applicability to a limited demographic and a specific geographic\nregion. In this paper we propose a general approach for sentiment analysis on\ndata containing texts from multiple languages. This enables all the\napplications to utilize the results of sentiment analysis in a language\noblivious or language-independent fashion.",
            "journal ref": "International Conference on Advances in the Emerging Computing\n  Technologies (AECT), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11980v1",
            "date": "2019-12-27 04:14:06+00:00",
            "title": "Explicit Sentence Compression for Neural Machine Translation",
            "authors": [
                "Zuchao Li",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "State-of-the-art Transformer-based neural machine translation (NMT) systems\nstill follow a standard encoder-decoder framework, in which source sentence\nrepresentation can be well done by an encoder with self-attention mechanism.\nThough Transformer-based encoder may effectively capture general information in\nits resulting source sentence representation, the backbone information, which\nstands for the gist of a sentence, is not specifically focused on. In this\npaper, we propose an explicit sentence compression method to enhance the source\nsentence representation for NMT. In practice, an explicit sentence compression\ngoal used to learn the backbone information in a sentence. We propose three\nways, including backbone source-side fusion, target-side fusion, and both-side\nfusion, to integrate the compressed sentence into NMT. Our empirical tests on\nthe WMT English-to-French and English-to-German translation tasks show that the\nproposed sentence compression method significantly improves the translation\nperformances over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12010v1",
            "date": "2019-12-27 07:28:11+00:00",
            "title": "Synthesising Expressiveness in Peking Opera via Duration Informed Attention Network",
            "authors": [
                "Yusong Wu",
                "Shengchen Li",
                "Chengzhu Yu",
                "Heng Lu",
                "Chao Weng",
                "Liqiang Zhang",
                "Dong Yu"
            ],
            "abstract": "This paper presents a method that generates expressive singing voice of\nPeking opera. The synthesis of expressive opera singing usually requires pitch\ncontours to be extracted as the training data, which relies on techniques and\nis not able to be manually labeled. With the Duration Informed Attention\nNetwork (DurIAN), this paper makes use of musical note instead of pitch\ncontours for expressive opera singing synthesis. The proposed method enables\nhuman annotation being combined with automatic extracted features to be used as\ntraining data thus the proposed method gives extra flexibility in data\ncollection for Peking opera singing synthesis. Comparing with the expressive\nsinging voice of Peking opera synthesised by pitch contour based system, the\nproposed musical note based system produces comparable singing voice in Peking\nopera with expressiveness in various aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12068v1",
            "date": "2019-12-27 12:10:10+00:00",
            "title": "A Multi-cascaded Model with Data Augmentation for Enhanced Paraphrase Detection in Short Texts",
            "authors": [
                "Muhammad Haroon Shakeel",
                "Asim Karim",
                "Imdadullah Khan"
            ],
            "abstract": "Paraphrase detection is an important task in text analytics with numerous\napplications such as plagiarism detection, duplicate question identification,\nand enhanced customer support helpdesks. Deep models have been proposed for\nrepresenting and classifying paraphrases. These models, however, require large\nquantities of human-labeled data, which is expensive to obtain. In this work,\nwe present a data augmentation strategy and a multi-cascaded model for improved\nparaphrase detection in short texts. Our data augmentation strategy considers\nthe notions of paraphrases and non-paraphrases as binary relations over the set\nof texts. Subsequently, it uses graph theoretic concepts to efficiently\ngenerate additional paraphrase and non-paraphrase pairs in a sound manner. Our\nmulti-cascaded model employs three supervised feature learners (cascades) based\non CNN and LSTM networks with and without soft-attention. The learned features,\ntogether with hand-crafted linguistic features, are then forwarded to a\ndiscriminator network for final classification. Our model is both wide and deep\nand provides greater robustness across clean and noisy short texts. We evaluate\nour approach on three benchmark datasets and show that it produces a comparable\nor state-of-the-art performance on all three.",
            "journal ref": "Information Processing & Management, Volume 57, Issue 3, May 2020,\n  102204",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12214v2",
            "date": "2019-12-27 16:13:43+00:00",
            "title": "Job Prediction: From Deep Neural Network Models to Applications",
            "authors": [
                "Tin Van Huynh",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen",
                "Anh Gia-Tuan Nguyen"
            ],
            "abstract": "Determining the job is suitable for a student or a person looking for work\nbased on their job's descriptions such as knowledge and skills that are\ndifficult, as well as how employers must find ways to choose the candidates\nthat match the job they require. In this paper, we focus on studying the job\nprediction using different deep neural network models including TextCNN,\nBi-GRU-LSTM-CNN, and Bi-GRU-CNN with various pre-trained word embeddings on the\nIT Job dataset. In addition, we also proposed a simple and effective ensemble\nmodel combining different deep neural network models. The experimental results\nillustrated that our proposed ensemble model achieved the highest result with\nan F1 score of 72.71%. Moreover, we analyze these experimental results to have\ninsights about this problem to find better solutions in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12380v1",
            "date": "2019-12-28 01:12:15+00:00",
            "title": "Knowledge-guided Text Structuring in Clinical Trials",
            "authors": [
                "Yingcheng Sun",
                "Kenneth Loparo"
            ],
            "abstract": "Clinical trial records are variable resources or the analysis of patients and\ndiseases. Information extraction from free text such as eligibility criteria\nand summary of results and conclusions in clinical trials would better support\ncomputer-based eligibility query formulation and electronic patient screening.\nPrevious research has focused on extracting information from eligibility\ncriteria, with usually a single pair of medical entity and attribute, but\nseldom considering other kinds of free text with multiple entities, attributes\nand relations that are more complex for parsing. In this paper, we propose a\nknowledge-guided text structuring framework with an automatically generated\nknowledge base as training corpus and word dependency relations as context\ninformation to transfer free text into formal, computer-interpretable\nrepresentations. Experimental results show that our method can achieve overall\nhigh precision and recall, demonstrating the effectiveness and efficiency of\nthe proposed method.",
            "journal ref": "2019 19th Industrial Conference on Data Mining",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12598v1",
            "date": "2019-12-29 07:27:23+00:00",
            "title": "ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension",
            "authors": [
                "Dheeru Dua",
                "Ananth Gottumukkala",
                "Alon Talmor",
                "Sameer Singh",
                "Matt Gardner"
            ],
            "abstract": "Reading comprehension is one of the crucial tasks for furthering research in\nnatural language understanding. A lot of diverse reading comprehension datasets\nhave recently been introduced to study various phenomena in natural language,\nranging from simple paraphrase matching and entity typing to entity tracking\nand understanding the implications of the context. Given the availability of\nmany such datasets, comprehensive and reliable evaluation is tedious and\ntime-consuming for researchers working on this problem. We present an\nevaluation server, ORB, that reports performance on seven diverse reading\ncomprehension datasets, encouraging and facilitating testing a single model's\ncapability in understanding a wide variety of reading phenomena. The evaluation\nserver places no restrictions on how models are trained, so it is a suitable\ntest bed for exploring training paradigms and representation learning for\ngeneral reading facility. As more suitable datasets are released, they will be\nadded to the evaluation server. We also collect and include synthetic\naugmentations for these datasets, testing how well models can handle\nout-of-domain questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12635v2",
            "date": "2019-12-29 11:31:11+00:00",
            "title": "\u00c6THEL: Automatically Extracted Typelogical Derivations for Dutch",
            "authors": [
                "Konstantinos Kogkalidis",
                "Michael Moortgat",
                "Richard Moot"
            ],
            "abstract": "We present {\\AE}THEL, a semantic compositionality dataset for written Dutch.\n{\\AE}THEL consists of two parts. First, it contains a lexicon of supertags for\nabout 900 000 words in context. The supertags correspond to types of the simply\ntyped linear lambda-calculus, enhanced with dependency decorations that capture\ngrammatical roles supplementary to function-argument structures. On the basis\nof these types, {\\AE}THEL further provides 72 192 validated derivations,\npresented in four formats: natural-deduction and sequent-style proofs, linear\nlogic proofnets and the associated programs (lambda terms) for meaning\ncomposition. {\\AE}THEL's types and derivations are obtained by means of an\nextraction algorithm applied to the syntactic analyses of LASSY Small, the gold\nstandard corpus of written Dutch. We discuss the extraction algorithm and show\nhow `virtual elements' in the original LASSY annotation of unbounded\ndependencies and coordination phenomena give rise to higher-order types. We\nsuggest some example usecases highlighting the benefits of a type-driven\napproach at the syntax semantics interface. The following resources are\nopen-sourced with {\\AE}THEL: the lexical mappings between words and types, a\nsubset of the dataset consisting of 7 924 semantic parses, and the Python code\nthat implements the extraction algorithm.",
            "journal ref": "Proceedings of The 12th Language Resources and Evaluation\n  Conference (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13194v1",
            "date": "2019-12-31 06:38:57+00:00",
            "title": "CASE: Context-Aware Semantic Expansion",
            "authors": [
                "Jialong Han",
                "Aixin Sun",
                "Haisong Zhang",
                "Chenliang Li",
                "Shuming Shi"
            ],
            "abstract": "In this paper, we define and study a new task called Context-Aware Semantic\nExpansion (CASE). Given a seed term in a sentential context, we aim to suggest\nother terms that well fit the context as the seed. CASE has many interesting\napplications such as query suggestion, computer-assisted writing, and word\nsense disambiguation, to name a few. Previous explorations, if any, only\ninvolve some similar tasks, and all require human annotations for evaluation.\nIn this study, we demonstrate that annotations for this task can be harvested\nat scale from existing corpora, in a fully automatic manner. On a dataset of\n1.8 million sentences thus derived, we propose a network architecture that\nencodes the context and seed term separately before suggesting alternative\nterms. The context encoder in this architecture can be easily extended by\nincorporating seed-aware attention. Our experiments demonstrate that\ncompetitive results are achieved with appropriate choices of context encoder\nand attention scoring function.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13318v5",
            "date": "2019-12-31 14:31:29+00:00",
            "title": "LayoutLM: Pre-training of Text and Layout for Document Image Understanding",
            "authors": [
                "Yiheng Xu",
                "Minghao Li",
                "Lei Cui",
                "Shaohan Huang",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Pre-training techniques have been verified successfully in a variety of NLP\ntasks in recent years. Despite the widespread use of pre-training models for\nNLP applications, they almost exclusively focus on text-level manipulation,\nwhile neglecting layout and style information that is vital for document image\nunderstanding. In this paper, we propose the \\textbf{LayoutLM} to jointly model\ninteractions between text and layout information across scanned document\nimages, which is beneficial for a great number of real-world document image\nunderstanding tasks such as information extraction from scanned documents.\nFurthermore, we also leverage image features to incorporate words' visual\ninformation into LayoutLM. To the best of our knowledge, this is the first time\nthat text and layout are jointly learned in a single framework for\ndocument-level pre-training. It achieves new state-of-the-art results in\nseveral downstream tasks, including form understanding (from 70.72 to 79.27),\nreceipt understanding (from 94.02 to 95.24) and document image classification\n(from 93.07 to 94.42). The code and pre-trained LayoutLM models are publicly\navailable at \\url{https://aka.ms/layoutlm}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13321v3",
            "date": "2019-12-31 14:36:45+00:00",
            "title": "OTEANN: Estimating the Transparency of Orthographies with an Artificial Neural Network",
            "authors": [
                "Xavier Marjou"
            ],
            "abstract": "To transcribe spoken language to written medium, most alphabets enable an\nunambiguous sound-to-letter rule. However, some writing systems have distanced\nthemselves from this simple concept and little work exists on measuring such\ndistance. In this study, we use an Artificial Neural Network (ANN) model to\nevaluate the transparency between written words and their pronunciation, hence\nits name Orthographic Transparency Estimation with an ANN (OTEANN). Based on\ndatasets derived from Wikimedia dictionaries, we trained and tested this model\nto score the percentage of false predictions in phoneme-to-grapheme and\ngrapheme-to-phoneme translation tasks. The scores obtained on 17 orthographies\nwere in line with the estimations of other studies. Interestingly, the model\nalso provided insight into typical mistakes made by learners who only consider\nthe phonemic rule in reading and writing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13337v2",
            "date": "2019-12-31 15:05:54+00:00",
            "title": "What Does My QA Model Know? Devising Controlled Probes using Expert Knowledge",
            "authors": [
                "Kyle Richardson",
                "Ashish Sabharwal"
            ],
            "abstract": "Open-domain question answering (QA) is known to involve several underlying\nknowledge and reasoning challenges, but are models actually learning such\nknowledge when trained on benchmark tasks? To investigate this, we introduce\nseveral new challenge tasks that probe whether state-of-the-art QA models have\ngeneral knowledge about word definitions and general taxonomic reasoning, both\nof which are fundamental to more complex forms of reasoning and are widespread\nin benchmark datasets. As an alternative to expensive crowd-sourcing, we\nintroduce a methodology for automatically building datasets from various types\nof expert knowledge (e.g., knowledge graphs and lexical taxonomies), allowing\nfor systematic control over the resulting probes and for a more comprehensive\nevaluation. We find automatically constructing probes to be vulnerable to\nannotation artifacts, which we carefully control for. Our evaluation confirms\nthat transformer-based QA models are already predisposed to recognize certain\ntypes of structural lexical knowledge. However, it also reveals a more nuanced\npicture: their performance degrades substantially with even a slight increase\nin the number of hops in the underlying taxonomic hierarchy, or as more\nchallenging distractor candidate answers are introduced. Further, even when\nthese models succeed at the standard instance-level evaluation, they leave much\nroom for improvement when assessed at the level of clusters of semantically\nconnected probes (e.g., all Isa questions about a concept).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13413v1",
            "date": "2019-12-23 18:01:32+00:00",
            "title": "Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings",
            "authors": [
                "Maxat Tezekbayev",
                "Zhenisbek Assylbekov",
                "Rustem Takhanov"
            ],
            "abstract": "We show that the skip-gram embedding of any word can be decomposed into two\nsubvectors which roughly correspond to semantic and syntactic roles of the\nword.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00003v3",
            "date": "2019-12-28 03:15:43+00:00",
            "title": "Learning Numeral Embeddings",
            "authors": [
                "Chengyue Jiang",
                "Zhonglin Nian",
                "Kaihao Guo",
                "Shanbo Chu",
                "Yinggong Zhao",
                "Libin Shen",
                "Kewei Tu"
            ],
            "abstract": "Word embedding is an essential building block for deep learning methods for\nnatural language processing. Although word embedding has been extensively\nstudied over the years, the problem of how to effectively embed numerals, a\nspecial subset of words, is still underexplored. Existing word embedding\nmethods do not learn numeral embeddings well because there are an infinite\nnumber of numerals and their individual appearances in training corpora are\nhighly scarce. In this paper, we propose two novel numeral embedding methods\nthat can handle the out-of-vocabulary (OOV) problem for numerals. We first\ninduce a finite set of prototype numerals using either a self-organizing map or\na Gaussian mixture model. We then represent the embedding of a numeral as a\nweighted average of the prototype number embeddings. Numeral embeddings\nrepresented in this manner can be plugged into existing word embedding learning\napproaches such as skip-gram for training. We evaluated our methods and showed\nits effectiveness on four intrinsic and extrinsic tasks: word similarity,\nembedding numeracy, numeral prediction, and sequence labeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00009v1",
            "date": "2019-12-30 01:32:42+00:00",
            "title": "Deep Reinforced Self-Attention Masks for Abstractive Summarization (DR.SAS)",
            "authors": [
                "Ankit Chadha",
                "Mohamed Masoud"
            ],
            "abstract": "We present a novel architectural scheme to tackle the abstractive\nsummarization problem based on the CNN/DMdataset which fuses Reinforcement\nLearning (RL) withUniLM, which is a pre-trained Deep Learning Model, to solve\nvarious natural language tasks. We have tested the limits of learning\nfine-grained attention in Transformers to improve the summarization quality.\nUniLM applies attention to the entire token space in a global fashion. We\npropose DR.SAS which applies the Actor-Critic (AC) algorithm to learn a dynamic\nself-attention distribution over the tokens to reduce redundancy and generate\nfactual and coherent summaries to improve the quality of summarization. After\nperforming hyperparameter tuning, we achievedbetter ROUGE results compared to\nthe baseline. Our model tends to be more extractive/factual yet coherent in\ndetail because of optimization over ROUGE rewards. We present detailed error\nanalysis with examples of the strengths and limitations of our model. Our\ncodebase will be publicly available on our GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00100v1",
            "date": "2019-12-31 22:40:30+00:00",
            "title": "Building chatbots from large scale domain-specific knowledge bases: challenges and opportunities",
            "authors": [
                "Walid Shalaby",
                "Adriano Arantes",
                "Teresa GonzalezDiaz",
                "Chetan Gupta"
            ],
            "abstract": "Popular conversational agents frameworks such as Alexa Skills Kit (ASK) and\nGoogle Actions (gActions) offer unprecedented opportunities for facilitating\nthe development and deployment of voice-enabled AI solutions in various\nverticals. Nevertheless, understanding user utterances with high accuracy\nremains a challenging task with these frameworks. Particularly, when building\nchatbots with large volume of domain-specific entities. In this paper, we\ndescribe the challenges and lessons learned from building a large scale virtual\nassistant for understanding and responding to equipment-related complaints. In\nthe process, we describe an alternative scalable framework for: 1) extracting\nthe knowledge about equipment components and their associated problem entities\nfrom short texts, and 2) learning to identify such entities in user utterances.\nWe show through evaluation on a real dataset that the proposed framework,\ncompared to off-the-shelf popular ones, scales better with large volume of\nentities being up to 30% more accurate, and is more effective in understanding\nuser utterances with domain-specific entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11327v2",
            "date": "2019-11-26 05:59:41+00:00",
            "title": "Iterative Batch Back-Translation for Neural Machine Translation: A Conceptual Model",
            "authors": [
                "Idris Abdulmumin",
                "Bashir Shehu Galadanci",
                "Abubakar Isa"
            ],
            "abstract": "An effective method to generate a large number of parallel sentences for\ntraining improved neural machine translation (NMT) systems is the use of\nback-translations of the target-side monolingual data. Recently, iterative\nback-translation has been shown to outperform standard back-translation albeit\non some language pairs. This work proposes the iterative batch back-translation\nthat is aimed at enhancing the standard iterative back-translation and enabling\nthe efficient utilization of more monolingual data. After each iteration,\nimproved back-translations of new sentences are added to the parallel data that\nwill be used to train the final forward model. The work presents a conceptual\nmodel of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00027v2",
            "date": "2019-01-31 19:00:07+00:00",
            "title": "Towards Machine-assisted Meta-Studies: The Hubble Constant",
            "authors": [
                "Tom Crossland",
                "Pontus Stenetorp",
                "Sebastian Riedel",
                "Daisuke Kawata",
                "Thomas D. Kitching",
                "Rupert A. C. Croft"
            ],
            "abstract": "We present an approach for automatic extraction of measured values from the\nastrophysical literature, using the Hubble constant for our pilot study. Our\nrules-based model -- a classical technique in natural language processing --\nhas successfully extracted 298 measurements of the Hubble constant, with\nuncertainties, from the 208,541 available arXiv astrophysics papers. We have\nalso created an artificial neural network classifier to identify papers in\narXiv which report novel measurements. From the analysis of our results we find\nthat reporting measurements with uncertainties and the correct units is\ncritical information when distinguishing novel measurements in free text. Our\nresults correctly highlight the current tension for measurements of the Hubble\nconstant and recover the $3.5\\sigma$ discrepancy -- demonstrating that the tool\npresented in this paper is useful for meta-studies of astrophysical\nmeasurements from a large number of publications.",
            "journal ref": "None",
            "category": "astro-ph.IM"
        },
        {
            "url": "http://arxiv.org/abs/1902.00154v2",
            "date": "2019-02-01 02:42:55+00:00",
            "title": "Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models",
            "authors": [
                "Dinghan Shen",
                "Asli Celikyilmaz",
                "Yizhe Zhang",
                "Liqun Chen",
                "Xin Wang",
                "Jianfeng Gao",
                "Lawrence Carin"
            ],
            "abstract": "Variational autoencoders (VAEs) have received much attention recently as an\nend-to-end architecture for text generation with latent variables. In this\npaper, we investigate several multi-level structures to learn a VAE model to\ngenerate long, and coherent text. In particular, we use a hierarchy of\nstochastic layers between the encoder and decoder networks to generate more\ninformative latent codes. We also investigate a multi-level decoder structure\nto learn a coherent long-term structure by generating intermediate sentence\nrepresentations as high-level plan vectors. Empirical results demonstrate that\na multi-level VAE model produces more coherent and less repetitive long text\ncompared to the standard VAE models and can further mitigate the\nposterior-collapse issue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00579v2",
            "date": "2019-02-01 22:48:26+00:00",
            "title": "Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog",
            "authors": [
                "Zhe Gan",
                "Yu Cheng",
                "Ahmed El Kholy",
                "Linjie Li",
                "Jingjing Liu",
                "Jianfeng Gao"
            ],
            "abstract": "This paper presents a new model for visual dialog, Recurrent Dual Attention\nNetwork (ReDAN), using multi-step reasoning to answer a series of questions\nabout an image. In each question-answering turn of a dialog, ReDAN infers the\nanswer progressively through multiple reasoning steps. In each step of the\nreasoning process, the semantic representation of the question is updated based\non the image and the previous dialog history, and the recurrently-refined\nrepresentation is used for further reasoning in the subsequent step. On the\nVisDial v1.0 dataset, the proposed ReDAN model achieves a new state-of-the-art\nof 64.47% NDCG score. Visualization on the reasoning process further\ndemonstrates that ReDAN can locate context-relevant visual and textual clues\nvia iterative refinement, which can lead to the correct answer step-by-step.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1902.00663v7",
            "date": "2019-02-02 07:44:41+00:00",
            "title": "A Multi-Resolution Word Embedding for Document Retrieval from Large Unstructured Knowledge Bases",
            "authors": [
                "Tolgahan Cakaloglu",
                "Xiaowei Xu"
            ],
            "abstract": "Deep language models learning a hierarchical representation proved to be a\npowerful tool for natural language processing, text mining and information\nretrieval. However, representations that perform well for retrieval must\ncapture semantic meaning at different levels of abstraction or context-scopes.\nIn this paper, we propose a new method to generate multi-resolution word\nembeddings that represent documents at multiple resolutions in terms of\ncontext-scopes. In order to investigate its performance,we use the Stanford\nQuestion Answering Dataset (SQuAD) and the Question Answering by Search And\nReading (QUASAR) in an open-domain question-answering setting, where the first\ntask is to find documents useful for answering a given question. To this end,\nwe first compare the quality of various text-embedding methods for retrieval\nperformance and give an extensive empirical comparison with the performance of\nvarious non-augmented base embeddings with and without multi-resolution\nrepresentation. We argue that multi-resolution word embeddings are consistently\nsuperior to the original counterparts and deep residual neural models\nspecifically trained for retrieval purposes can yield further significant gains\nwhen they are used for augmenting those embeddings.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.00672v1",
            "date": "2019-02-02 08:52:44+00:00",
            "title": "Query-oriented text summarization based on hypergraph transversals",
            "authors": [
                "Hadrien Van Lierde",
                "Tommy W. S. Chow"
            ],
            "abstract": "Existing graph- and hypergraph-based algorithms for document summarization\nrepresent the sentences of a corpus as the nodes of a graph or a hypergraph in\nwhich the edges represent relationships of lexical similarities between\nsentences. Each sentence of the corpus is then scored individually, using\npopular node ranking algorithms, and a summary is produced by extracting highly\nscored sentences. This approach fails to select a subset of jointly relevant\nsentences and it may produce redundant summaries that are missing important\ntopics of the corpus. To alleviate this issue, a new hypergraph-based\nsummarizer is proposed in this paper, in which each node is a sentence and each\nhyperedge is a theme, namely a group of sentences sharing a topic. Themes are\nweighted in terms of their prominence in the corpus and their relevance to a\nuser-defined query. It is further shown that the problem of identifying a\nsubset of sentences covering the relevant themes of the corpus is equivalent to\nthat of finding a hypergraph transversal in our theme-based hypergraph. Two\nextensions of the notion of hypergraph transversal are proposed for the purpose\nof summarization, and polynomial time algorithms building on the theory of\nsubmodular functions are proposed for solving the associated discrete\noptimization problems. The worst-case time complexity of the proposed\nalgorithms is squared in the number of terms, which makes it cheaper than the\nexisting hypergraph-based methods. A thorough comparative analysis with related\nmodels on DUC benchmark datasets demonstrates the effectiveness of our\napproach, which outperforms existing graph- or hypergraph-based methods by at\nleast 6% of ROUGE-SU4 score.",
            "journal ref": "Information Processing & Management, Volume 56, Issue 4, July\n  2019, Pages 1317-1338",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00750v2",
            "date": "2019-02-02 16:29:12+00:00",
            "title": "How to Write High-quality News on Social Network? Predicting News Quality by Mining Writing Style",
            "authors": [
                "Yuting Yang",
                "Juan Cao",
                "Mingyan Lu",
                "Jintao Li",
                "Chia-Wen Lin"
            ],
            "abstract": "Rapid development of Internet technologies promotes traditional newspapers to\nreport news on social networks. However, people on social networks may have\ndifferent needs which naturally arises the question: whether can we analyze the\ninfluence of writing style on news quality automatically and assist writers in\nimproving news quality? It's challenging due to writing style and 'quality' are\nhard to measure. First, we use 'popularity' as the measure of 'quality'. It is\nnatural on social networks but brings new problems: popularity are also\ninfluenced by event and publisher. So we design two methods to alleviate their\ninfluence. Then, we proposed eight types of linguistic features (53 features in\nall) according eight writing guidelines and analyze their relationship with\nnews quality. The experimental results show these linguistic features influence\ngreatly on news quality. Based on it, we design a news quality assessment model\non social network (SNQAM). SNQAM performs excellently on predicting quality,\npresenting interpretable quality score and giving accessible suggestions on how\nto improve it according to writing guidelines we referred to.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00753v1",
            "date": "2019-02-02 17:04:14+00:00",
            "title": "Word Embeddings for Sentiment Analysis: A Comprehensive Empirical Survey",
            "authors": [
                "Erion \u00c7ano",
                "Maurizio Morisio"
            ],
            "abstract": "This work investigates the role of factors like training method, training\ncorpus size and thematic relevance of texts in the performance of word\nembedding features on sentiment analysis of tweets, song lyrics, movie reviews\nand item reviews. We also explore specific training or post-processing methods\nthat can be used to enhance the performance of word embeddings in certain tasks\nor domains. Our empirical observations indicate that models trained with\nmultithematic texts that are large and rich in vocabulary are the best in\nanswering syntactic and semantic word analogy questions. We further observe\nthat influence of thematic relevance is stronger on movie and phone reviews,\nbut weaker on tweets and lyrics. These two later domains are more sensitive to\ncorpus size and training method, with Glove outperforming Word2vec. \"Injecting\"\nextra intelligence from lexicons or generating sentiment specific word\nembeddings are two prominent alternatives for increasing performance of word\nembedding features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01030v2",
            "date": "2019-02-04 04:42:08+00:00",
            "title": "Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers",
            "authors": [
                "Haoyu Wang",
                "Ming Tan",
                "Mo Yu",
                "Shiyu Chang",
                "Dakuo Wang",
                "Kun Xu",
                "Xiaoxiao Guo",
                "Saloni Potdar"
            ],
            "abstract": "Most approaches to extraction multiple relations from a paragraph require\nmultiple passes over the paragraph. In practice, multiple passes are\ncomputationally expensive and this makes difficult to scale to longer\nparagraphs and larger text corpora. In this work, we focus on the task of\nmultiple relation extraction by encoding the paragraph only once (one-pass). We\nbuild our solution on the pre-trained self-attentive (Transformer) models,\nwhere we first add a structured prediction layer to handle extraction between\nmultiple entity pairs, then enhance the paragraph embedding to capture multiple\nrelational information associated with each entity with an entity-aware\nattention technique. We show that our approach is not only scalable but can\nalso perform state-of-the-art on the standard benchmark ACE 2005.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01177v2",
            "date": "2019-02-04 13:47:18+00:00",
            "title": "Unsupervised Clinical Language Translation",
            "authors": [
                "Wei-Hung Weng",
                "Yu-An Chung",
                "Peter Szolovits"
            ],
            "abstract": "As patients' access to their doctors' clinical notes becomes common,\ntranslating professional, clinical jargon to layperson-understandable language\nis essential to improve patient-clinician communication. Such translation\nyields better clinical outcomes by enhancing patients' understanding of their\nown health conditions, and thus improving patients' involvement in their own\ncare. Existing research has used dictionary-based word replacement or\ndefinition insertion to approach the need. However, these methods are limited\nby expert curation, which is hard to scale and has trouble generalizing to\nunseen datasets that do not share an overlapping vocabulary. In contrast, we\napproach the clinical word and sentence translation problem in a completely\nunsupervised manner. We show that a framework using representation learning,\nbilingual dictionary induction and statistical machine translation yields the\nbest precision at 10 of 0.827 on professional-to-consumer word translation, and\nmean opinion scores of 4.10 and 4.28 out of 5 for clinical correctness and\nlayperson readability, respectively, on sentence translation. Our\nfully-unsupervised strategy overcomes the curation problem, and the clinically\nmeaningful evaluation reduces biases from inappropriate evaluators, which are\ncritical in clinical machine learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01370v3",
            "date": "2019-02-04 18:35:59+00:00",
            "title": "Insertion-based Decoding with automatically Inferred Generation Order",
            "authors": [
                "Jiatao Gu",
                "Qi Liu",
                "Kyunghyun Cho"
            ],
            "abstract": "Conventional neural autoregressive decoding commonly assumes a fixed\nleft-to-right generation order, which may be sub-optimal. In this work, we\npropose a novel decoding algorithm -- InDIGO -- which supports flexible\nsequence generation in arbitrary orders through insertion operations. We extend\nTransformer, a state-of-the-art sequence generation model, to efficiently\nimplement the proposed approach, enabling it to be trained with either a\npre-defined generation order or adaptive orders obtained from beam-search.\nExperiments on four real-world tasks, including word order recovery, machine\ntranslation, image caption and code generation, demonstrate that our algorithm\ncan generate sequences following arbitrary orders, while achieving competitive\nor even better performance compared to the conventional left-to-right\ngeneration. The generated sequences show that InDIGO adopts adaptive generation\norders based on input information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01474v2",
            "date": "2019-02-04 22:05:50+00:00",
            "title": "Optimized Compilation of Aggregated Instructions for Realistic Quantum Computers",
            "authors": [
                "Yunong Shi",
                "Nelson Leung",
                "Pranav Gokhale",
                "Zane Rossi",
                "David I. Schuster",
                "Henry Hoffman",
                "Fred T. Chong"
            ],
            "abstract": "Recent developments in engineering and algorithms have made real-world\napplications in quantum computing possible in the near future. Existing quantum\nprogramming languages and compilers use a quantum assembly language composed of\n1- and 2-qubit (quantum bit) gates. Quantum compiler frameworks translate this\nquantum assembly to electric signals (called control pulses) that implement the\nspecified computation on specific physical devices. However, there is a\nmismatch between the operations defined by the 1- and 2-qubit logical ISA and\ntheir underlying physical implementation, so the current practice of directly\ntranslating logical instructions into control pulses results in inefficient,\nhigh-latency programs. To address this inefficiency, we propose a universal\nquantum compilation methodology that aggregates multiple logical operations\ninto larger units that manipulate up to 10 qubits at a time. Our methodology\nthen optimizes these aggregates by (1) finding commutative intermediate\noperations that result in more efficient schedules and (2) creating custom\ncontrol pulses optimized for the aggregate (instead of individual 1- and\n2-qubit operations). Compared to the standard gate-based compilation, the\nproposed approach realizes a deeper vertical integration of high-level quantum\nsoftware and low-level, physical quantum hardware. We evaluate our approach on\nimportant near-term quantum applications on simulations of superconducting\nquantum architectures. Our proposed approach provides a mean speedup of\n$5\\times$, with a maximum of $10\\times$. Because latency directly affects the\nfeasibility of quantum computation, our results not only improve performance\nbut also have the potential to enable quantum computation sooner than otherwise\npossible.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1902.01529v1",
            "date": "2019-02-05 03:25:24+00:00",
            "title": "An Ensemble Dialogue System for Facts-Based Sentence Generation",
            "authors": [
                "Ryota Tanaka",
                "Akihide Ozeki",
                "Shugo Kato",
                "Akinobu Lee"
            ],
            "abstract": "This study aims to generate responses based on real-world facts by\nconditioning context and external facts extracted from information websites.\nOur system is an ensemble system that combines three modules: generated-based\nmodule, retrieval-based module, and reranking module. Therefore, this system\ncan return diverse and meaningful responses from various perspectives. The\nexperiments and evaluations are conducted with the sentence generation task in\nDialog System Technology Challenges 7 (DSTC7-Task2). As a result, the proposed\nsystem performed significantly better than sole modules, and worked fine at the\nDSTC7-Task2, specifically on the objective evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01541v2",
            "date": "2019-02-05 04:41:55+00:00",
            "title": "The Referential Reader: A Recurrent Entity Network for Anaphora Resolution",
            "authors": [
                "Fei Liu",
                "Luke Zettlemoyer",
                "Jacob Eisenstein"
            ],
            "abstract": "We present a new architecture for storing and accessing entity mentions\nduring online text processing. While reading the text, entity references are\nidentified, and may be stored by either updating or overwriting a cell in a\nfixed-length memory. The update operation implies coreference with the other\nmentions that are stored in the same cell; the overwrite operation causes these\nmentions to be forgotten. By encoding the memory operations as differentiable\ngates, it is possible to train the model end-to-end, using both a supervised\nanaphora resolution objective as well as a supplementary language modeling\nobjective. Evaluation on a dataset of pronoun-name anaphora demonstrates strong\nperformance with purely incremental text processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01718v2",
            "date": "2019-02-05 14:50:48+00:00",
            "title": "End-to-End Open-Domain Question Answering with BERTserini",
            "authors": [
                "Wei Yang",
                "Yuqing Xie",
                "Aileen Lin",
                "Xingyu Li",
                "Luchen Tan",
                "Kun Xiong",
                "Ming Li",
                "Jimmy Lin"
            ],
            "abstract": "We demonstrate an end-to-end question answering system that integrates BERT\nwith the open-source Anserini information retrieval toolkit. In contrast to\nmost question answering and reading comprehension models today, which operate\nover small amounts of input text, our system integrates best practices from IR\nwith a BERT-based reader to identify answers from a large corpus of Wikipedia\narticles in an end-to-end fashion. We report large improvements over previous\nresults on a standard benchmark test collection, showing that fine-tuning\npretrained BERT with SQuAD is sufficient to achieve high accuracy in\nidentifying answer spans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01918v1",
            "date": "2019-02-05 21:26:57+00:00",
            "title": "CMS Sematrix: A Tool to Aid the Development of Clinical Quality Measures (CQMs)",
            "authors": [
                "Michael A. Schwemmer",
                "Po-Hsu Chen",
                "Mithun Balakrishna",
                "Amy Leibrand",
                "Aaron Leonard",
                "Nancy J. McMillan",
                "Jeffrey J. Geppert"
            ],
            "abstract": "As part of the effort to improve quality and to reduce national healthcare\ncosts, the Centers for Medicare and Medicaid Services (CMS) are responsible for\ncreating and maintaining an array of clinical quality measures (CQMs) for\nassessing healthcare structure, process, outcome, and patient experience across\nvarious conditions, clinical specialties, and settings. The development and\nmaintenance of CQMs involves substantial and ongoing evaluation of the evidence\non the measure's properties: importance, reliability, validity, feasibility,\nand usability. As such, CMS conducts monthly environmental scans of the\npublished clinical and health service literature. Conducting time consuming,\nexhaustive evaluations of the ever-changing healthcare literature presents one\nof the largest challenges to an evidence-based approach to healthcare quality\nimprovement. Thus, it is imperative to leverage automated techniques to aid CMS\nin the identification of clinical and health services literature relevant to\nCQMs. Additionally, the estimated labor hours and related cost savings of using\nCMS Sematrix compared to a traditional literature review are roughly 818 hours\nand 122,000 dollars for a single monthly environmental scan.",
            "journal ref": "None",
            "category": "stat.OT"
        },
        {
            "url": "http://arxiv.org/abs/1902.02162v1",
            "date": "2019-01-19 17:40:08+00:00",
            "title": "Adaptive Artificial Intelligent Q&A Platform",
            "authors": [
                "M. R",
                "Akram",
                "C. P",
                "Singhabahu",
                "M. S. M Saad",
                "P",
                "Deleepa",
                "Anupiya",
                "Nugaliyadde",
                "Yashas",
                "Mallawarachchi"
            ],
            "abstract": "The paper presents an approach to build a question and answer system that is\ncapable of processing the information in a large dataset and allows the user to\ngain knowledge from this dataset by asking questions in natural language form.\nKey content of this research covers four dimensions which are; Corpus\nPreprocessing, Question Preprocessing, Deep Neural Network for Answer\nExtraction and Answer Generation. The system is capable of understanding the\nquestion, responds to the user's query in natural language form as well. The\ngoal is to make the user feel as if they were interacting with a person than a\nmachine.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02173v1",
            "date": "2019-01-28 21:39:23+00:00",
            "title": "Dise\u00f1o de un espacio sem\u00e1ntico sobre la base de la Wikipedia. Una propuesta de an\u00e1lisis de la sem\u00e1ntica latente para el idioma espa\u00f1ol",
            "authors": [
                "Dalina Aidee Villa",
                "Igor Barahona",
                "Luis Javier \u00c1lvarez"
            ],
            "abstract": "Latent Semantic Analysis (LSA) was initially conceived by the cognitive\npsychology at the 90s decade. Since its emergence, the LSA has been used to\nmodel cognitive processes, pointing out academic texts, compare literature\nworks and analyse political speeches, among other applications. Taking as\nstarting point multivariate method for dimensionality reduction, this paper\npropose a semantic space for Spanish language. Out results include a document\ntext matrix with dimensions 1.3 x10^6 and 5.9x10^6, which later is decomposed\ninto singular values. Those singular values are used to semantically words or\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02380v1",
            "date": "2019-02-06 19:49:22+00:00",
            "title": "Compression of Recurrent Neural Networks for Efficient Language Modeling",
            "authors": [
                "Artem M. Grachev",
                "Dmitry I. Ignatov",
                "Andrey V. Savchenko"
            ],
            "abstract": "Recurrent neural networks have proved to be an effective method for\nstatistical language modeling. However, in practice their memory and run-time\ncomplexity are usually too large to be implemented in real-time offline mobile\napplications. In this paper we consider several compression techniques for\nrecurrent neural networks including Long-Short Term Memory models. We make\nparticular attention to the high-dimensional output problem caused by the very\nlarge vocabulary size. We focus on effective compression methods in the context\nof their exploitation on devices: pruning, quantization, and matrix\ndecomposition approaches (low-rank factorization and tensor train\ndecomposition, in particular). For each model we investigate the trade-off\nbetween its size, suitability for fast inference and perplexity. We propose a\ngeneral pipeline for applying the most suitable methods to compress recurrent\nneural networks for language modeling. It has been shown in the experimental\nstudy with the Penn Treebank (PTB) dataset that the most efficient results in\nterms of speed and compression-perplexity balance are obtained by matrix\ndecomposition techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02507v3",
            "date": "2019-02-07 07:44:03+00:00",
            "title": "Towards Autoencoding Variational Inference for Aspect-based Opinion Summary",
            "authors": [
                "Tai Hoang",
                "Huy Le",
                "Tho Quan"
            ],
            "abstract": "Aspect-based Opinion Summary (AOS), consisting of aspect discovery and\nsentiment classification steps, has recently been emerging as one of the most\ncrucial data mining tasks in e-commerce systems. Along this direction, the\nLDA-based model is considered as a notably suitable approach, since this model\noffers both topic modeling and sentiment classification. However, unlike\ntraditional topic modeling, in the context of aspect discovery it is often\nrequired some initial seed words, whose prior knowledge is not easy to be\nincorporated into LDA models. Moreover, LDA approaches rely on sampling\nmethods, which need to load the whole corpus into memory, making them hardly\nscalable. In this research, we study an alternative approach for AOS problem,\nbased on Autoencoding Variational Inference (AVI). Firstly, we introduce the\nAutoencoding Variational Inference for Aspect Discovery (AVIAD) model, which\nextends the previous work of Autoencoding Variational Inference for Topic\nModels (AVITM) to embed prior knowledge of seed words. This work includes\nenhancement of the previous AVI architecture and also modification of the loss\nfunction. Ultimately, we present the Autoencoding Variational Inference for\nJoint Sentiment/Topic (AVIJST) model. In this model, we substantially extend\nthe AVI model to support the JST model, which performs topic modeling for\ncorresponding sentiment. The experimental results show that our proposed models\nenjoy higher topic coherent, faster convergence time and better accuracy on\nsentiment classification, as compared to their LDA-based counterparts.",
            "journal ref": "Applied Artificial Intelligence, 33 (2019) 796-816",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03455v1",
            "date": "2019-02-09 17:41:56+00:00",
            "title": "Contextual Recurrent Neural Networks",
            "authors": [
                "Sam Wenke",
                "Jim Fleming"
            ],
            "abstract": "There is an implicit assumption that by unfolding recurrent neural networks\n(RNN) in finite time, the misspecification of choosing a zero value for the\ninitial hidden state is mitigated by later time steps. This assumption has been\nshown to work in practice and alternative initialization may be suggested but\noften overlooked. In this paper, we propose a method of parameterizing the\ninitial hidden state of an RNN. The resulting architecture, referred to as a\nContextual RNN, can be trained end-to-end. The performance on an associative\nretrieval task is found to improve by conditioning the RNN initial hidden state\non contextual information from the input sequence. Furthermore, we propose a\nnovel method of conditionally generating sequences using the hidden state\nparameterization of Contextual RNN.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.04094v2",
            "date": "2019-02-11 19:02:27+00:00",
            "title": "BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model",
            "authors": [
                "Alex Wang",
                "Kyunghyun Cho"
            ],
            "abstract": "We show that BERT (Devlin et al., 2018) is a Markov random field language\nmodel. This formulation gives way to a natural procedure to sample sentences\nfrom BERT. We generate from BERT and find that it can produce high-quality,\nfluent generations. Compared to the generations of a traditional left-to-right\nlanguage model, BERT generates sentences that are more diverse but of slightly\nworse quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.04994v1",
            "date": "2019-02-13 16:37:32+00:00",
            "title": "Explainable Text-Driven Neural Network for Stock Prediction",
            "authors": [
                "Linyi Yang",
                "Zheng Zhang",
                "Su Xiong",
                "Lirui Wei",
                "James Ng",
                "Lina Xu",
                "Ruihai Dong"
            ],
            "abstract": "It has been shown that financial news leads to the fluctuation of stock\nprices. However, previous work on news-driven financial market prediction\nfocused only on predicting stock price movement without providing an\nexplanation. In this paper, we propose a dual-layer attention-based neural\nnetwork to address this issue. In the initial stage, we introduce a\nknowledge-based method to adaptively extract relevant financial news. Then, we\nuse input attention to pay more attention to the more influential news and\nconcatenate the day embeddings with the output of the news representation.\nFinally, we use an output attention mechanism to allocate different weights to\ndifferent days in terms of their contribution to stock price movement. Thorough\nempirical studies based upon historical prices of several individual stocks\ndemonstrate the superiority of our proposed method in stock price prediction\ncompared to state-of-the-art methods.",
            "journal ref": "2018 5th IEEE International Conference on Cloud Computing and\n  Intelligence Systems",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05715v1",
            "date": "2019-02-15 07:59:11+00:00",
            "title": "Generating Natural Language Explanations for Visual Question Answering using Scene Graphs and Visual Attention",
            "authors": [
                "Shalini Ghosh",
                "Giedrius Burachas",
                "Arijit Ray",
                "Avi Ziskind"
            ],
            "abstract": "In this paper, we present a novel approach for the task of eXplainable\nQuestion Answering (XQA), i.e., generating natural language (NL) explanations\nfor the Visual Question Answering (VQA) problem. We generate NL explanations\ncomprising of the evidence to support the answer to a question asked to an\nimage using two sources of information: (a) annotations of entities in an image\n(e.g., object labels, region descriptions, relation phrases) generated from the\nscene graph of the image, and (b) the attention map generated by a VQA model\nwhen answering the question. We show how combining the visual attention map\nwith the NL representation of relevant scene graph entities, carefully selected\nusing a language model, can give reasonable textual explanations without the\nneed of any additional collected data (explanation captions, etc). We run our\nalgorithms on the Visual Genome (VG) dataset and conduct internal user-studies\nto demonstrate the efficacy of our approach over a strong baseline. We have\nalso released a live web demo showcasing our VQA and textual explanation\ngeneration using scene graphs and visual attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.05770v1",
            "date": "2019-02-15 11:14:35+00:00",
            "title": "Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement",
            "authors": [
                "Zi-Yi Dou",
                "Zhaopeng Tu",
                "Xing Wang",
                "Longyue Wang",
                "Shuming Shi",
                "Tong Zhang"
            ],
            "abstract": "With the promising progress of deep neural networks, layer aggregation has\nbeen used to fuse information across layers in various fields, such as computer\nvision and machine translation. However, most of the previous methods combine\nlayers in a static fashion in that their aggregation strategy is independent of\nspecific hidden states. Inspired by recent progress on capsule networks, in\nthis paper we propose to use routing-by-agreement strategies to aggregate\nlayers dynamically. Specifically, the algorithm learns the probability of a\npart (individual layer representations) assigned to a whole (aggregated\nrepresentations) in an iterative way and combines parts accordingly. We\nimplement our algorithm on top of the state-of-the-art neural machine\ntranslation model TRANSFORMER and conduct experiments on the widely-used WMT14\nEnglish-German and WMT17 Chinese-English translation datasets. Experimental\nresults across language pairs show that the proposed approach consistently\noutperforms the strong baseline model and a representative static aggregation\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06000v1",
            "date": "2019-02-15 22:54:32+00:00",
            "title": "Improving Semantic Parsing for Task Oriented Dialog",
            "authors": [
                "Arash Einolghozati",
                "Panupong Pasupat",
                "Sonal Gupta",
                "Rushin Shah",
                "Mrinal Mohit",
                "Mike Lewis",
                "Luke Zettlemoyer"
            ],
            "abstract": "Semantic parsing using hierarchical representations has recently been\nproposed for task oriented dialog with promising results [Gupta et al 2018]. In\nthis paper, we present three different improvements to the model:\ncontextualized embeddings, ensembling, and pairwise re-ranking based on a\nlanguage model. We taxonomize the errors possible for the hierarchical\nrepresentation, such as wrong top intent, missing spans or split spans, and\nshow that the three approaches correct different kinds of errors. The best\nmodel combines the three techniques and gives 6.4% better exact match accuracy\nthan the state-of-the-art, with an error reduction of 33%, resulting in a new\nstate-of-the-art result on the Task Oriented Parsing (TOP) dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06024v1",
            "date": "2019-02-16 01:54:47+00:00",
            "title": "CruzAffect at AffCon 2019 Shared Task: A feature-rich approach to characterize happiness",
            "authors": [
                "Jiaqi Wu",
                "Ryan Compton",
                "Geetanjali Rakshit",
                "Marilyn Walker",
                "Pranav Anand",
                "Steve Whittaker"
            ],
            "abstract": "We present our system, CruzAffect, for the CL-Aff Shared Task 2019.\nCruzAffect consists of several types of robust and efficient models for\naffective classification tasks. We utilize both traditional classifiers, such\nas XGBoosted Forest, as well as a deep learning Convolutional Neural Networks\n(CNN) classifier. We explore rich feature sets such as syntactic features,\nemotional features, and profile features, and utilize several sentiment\nlexicons, to discover essential indicators of social involvement and control\nthat a subject might exercise in their happy moments, as described in textual\nsnippets from the HappyDB database. The data comes with a labeled set (10K),\nand a larger unlabeled set (70K). We therefore use supervised methods on the\n10K dataset, and a bootstrapped semi-supervised approach for the 70K. We\nevaluate these models for binary classification of agency and social labels\n(Task 1), as well as multi-class prediction for concepts labels (Task 2). We\nobtain promising results on the held-out data, suggesting that the proposed\nfeature sets effectively represent the data for affective classification tasks.\nWe also build concepts models that discover general themes recurring in happy\nmoments. Our results indicate that generic characteristics are shared between\nthe classes of agency, social and concepts, suggesting it should be possible to\nbuild general models for affective classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06193v1",
            "date": "2019-02-17 03:17:03+00:00",
            "title": "An Automated Testing Framework for Conversational Agents",
            "authors": [
                "Soodeh Atefi",
                "Mohammad Amin Alipour"
            ],
            "abstract": "Conversational agents are systems with a conversational interface that afford\ninteraction in spoken language. These systems are becoming prevalent and are\npreferred in various contexts and for many users. Despite their increasing\nsuccess, the automated testing infrastructure to support the effective and\nefficient development of such systems compared to traditional software systems\nis still limited. Automated testing framework for conversational systems can\nimprove the quality of these systems by assisting developers to write, execute,\nand maintain test cases. In this paper, we introduce our work-in-progress\nautomated testing framework, and its realization in the Python programming\nlanguage. We discuss some research problems in the development of such an\nautomated testing framework for conversational agents. In particular, we point\nout the problems of the specification of the expected behavior, known as test\noracles, and semantic comparison of utterances.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1902.06208v1",
            "date": "2019-02-17 06:09:39+00:00",
            "title": "Twitch Plays Pokemon, Machine Learns Twitch: Unsupervised Context-Aware Anomaly Detection for Identifying Trolls in Streaming Data",
            "authors": [
                "Albert Haque"
            ],
            "abstract": "With the increasing importance of online communities, discussion forums, and\ncustomer reviews, Internet \"trolls\" have proliferated thereby making it\ndifficult for information seekers to find relevant and correct information. In\nthis paper, we consider the problem of detecting and identifying Internet\ntrolls, almost all of which are human agents. Identifying a human agent among a\nhuman population presents significant challenges compared to detecting\nautomated spam or computerized robots. To learn a troll's behavior, we use\ncontextual anomaly detection to profile each chat user. Using clustering and\ndistance-based methods, we use contextual data such as the group's current\ngoal, the current time, and the username to classify each point as an anomaly.\nA user whose features significantly differ from the norm will be classified as\na troll. We collected 38 million data points from the viral Internet fad,\nTwitch Plays Pokemon. Using clustering and distance-based methods, we develop\nheuristics for identifying trolls. Using MapReduce techniques for preprocessing\nand user profiling, we are able to classify trolls based on 10 features\nextracted from a user's lifetime history.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06377v2",
            "date": "2019-02-18 02:00:23+00:00",
            "title": "SCEF: A Support-Confidence-aware Embedding Framework for Knowledge Graph Refinement",
            "authors": [
                "Yu Zhao",
                "Ji Liu"
            ],
            "abstract": "Knowledge graph (KG) refinement mainly aims at KG completion and correction\n(i.e., error detection). However, most conventional KG embedding models only\nfocus on KG completion with an unreasonable assumption that all facts in KG\nhold without noises, ignoring error detection which also should be significant\nand essential for KG refinement.In this paper, we propose a novel\nsupport-confidence-aware KG embedding framework (SCEF), which implements KG\ncompletion and correction simultaneously by learning knowledge representations\nwith both triple support and triple confidence. Specifically, we build model\nenergy function by incorporating conventional translation-based model with\nsupport and confidence. To make our triple support-confidence more sufficient\nand robust, we not only consider the internal structural information in KG,\nstudying the approximate relation entailment as triple confidence constraints,\nbut also the external textual evidence, proposing two kinds of triple supports\nwith entity types and descriptions respectively.Through extensive experiments\non real-world datasets, we demonstrate SCEF's effectiveness.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.06423v1",
            "date": "2019-02-18 06:54:14+00:00",
            "title": "CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model",
            "authors": [
                "Florian Mai",
                "Lukas Galke",
                "Ansgar Scherp"
            ],
            "abstract": "Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to\nits strong capabilities to encode word content, CBOW embeddings perform well on\na wide range of downstream tasks while being efficient to compute. However,\nCBOW is not capable of capturing the word order. The reason is that the\ncomputation of CBOW's word embeddings is commutative, i.e., embeddings of XYZ\nand ZYX are the same. In order to address this shortcoming, we propose a\nlearning algorithm for the Continuous Matrix Space Model, which we call\nContinual Multiplication of Words (CMOW). Our algorithm is an adaptation of\nword2vec, so that it can be trained on large quantities of unlabeled text. We\nempirically show that CMOW better captures linguistic properties, but it is\ninferior to CBOW in memorizing word content. Motivated by these findings, we\npropose a hybrid model that combines the strengths of CBOW and CMOW. Our\nresults show that the hybrid CBOW-CMOW-model retains CBOW's strong ability to\nmemorize word content while at the same time substantially improving its\nability to encode other linguistic information by 8%. As a result, the hybrid\nalso performs better on 8 out of 11 supervised downstream tasks with an average\nimprovement of 1.2%.",
            "journal ref": "In International Conference on Learning Representations 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06428v1",
            "date": "2019-02-18 07:21:21+00:00",
            "title": "\"The Michael Jordan of Greatness\": Extracting Vossian Antonomasia from Two Decades of the New York Times, 1987-2007",
            "authors": [
                "Frank Fischer",
                "Robert J\u00e4schke"
            ],
            "abstract": "Vossian Antonomasia is a prolific stylistic device, in use since antiquity.\nIt can compress the introduction or description of a person or another named\nentity into a terse, poignant formulation and can best be explained by an\nexample: When Norwegian world champion Magnus Carlsen is described as \"the\nMozart of chess\", it is Vossian Antonomasia we are dealing with. The pattern is\nsimple: A source (Mozart) is used to describe a target (Magnus Carlsen), the\ntransfer of meaning is reached via a modifier (\"of chess\"). This phenomenon has\nbeen discussed before (as 'metaphorical antonomasia' or, with special focus on\nthe source object, as 'paragons'), but no corpus-based approach has been\nundertaken as yet to explore its breadth and variety. We are looking into a\nfull-text newspaper corpus (The New York Times, 1987-2007) and describe a new\nmethod for the automatic extraction of Vossian Antonomasia based on Wikidata\nentities. Our analysis offers new insights into the occurrence of popular\nparagons and their distribution.",
            "journal ref": "Digital Scholarship in the Humanities (January 2019)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.06598v1",
            "date": "2019-02-09 15:53:14+00:00",
            "title": "Network connectivity dynamics affect the evolution of culturally transmitted variants",
            "authors": [
                "Jos\u00e9 Segovia Mart\u00edn",
                "Bradley Walker",
                "Nicolas Fay",
                "Monica Tamariz"
            ],
            "abstract": "The distribution of cultural variants in a population is shaped by both\nneutral evolutionary dynamics and by selection pressures, which include several\nindividual cognitive biases, demographic factors and social network structures.\nThe temporal dynamics of social network connectivity, i.e. the order in which\nindividuals in a population interact with each other, has been largely\nunexplored. In this paper we investigate how, in a fully connected social\nnetwork, connectivity dynamics, alone and in interaction with different\ncognitive biases, affect the evolution of cultural variants. Using agent-based\ncomputer simulations, we manipulate population connectivity dynamics (early,\nmiddle and late full-population connectivity); content bias, or a preference\nfor high-quality variants; coordination bias, or whether agents tend to use\nself-produced variants (egocentric bias), or to switch to variants observed in\nothers (allocentric bias); and memory size, or the number of items that agents\ncan store in their memory. We show that connectivity dynamics affect the\ntime-course of variant spread, with lower connectivity slowing down convergence\nof the population onto a single cultural variant. We also show that, compared\nto a neutral evolutionary model, content bias accelerates convergence and\namplifies the effects of connectivity dynamics, whilst larger memory size and\ncoordination bias, especially egocentric bias, slow down convergence.\nFurthermore, connectivity dynamics affect the frequency of high quality\nvariants (adaptiveness), with late connectivity populations showing bursts of\nrapid change in adaptiveness followed by periods of relatively slower change,\nand early connectivity populations following a single-peak evolutionary\ndynamic. In this way, we provide for the first time a direct connection between\nthe order of agents' interactions and punctuational evolution.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1902.06689v1",
            "date": "2019-02-02 23:42:42+00:00",
            "title": "A Corpus Linguistic Analysis of Public Reddit Blog Posts on Non-Suicidal Self-Injury",
            "authors": [
                "Mandy M. Greaves",
                "Cass Dykeman"
            ],
            "abstract": "While non-suicidal self-injury (NSSI) is not a new phenomenon, there is still\na limited yet little is still known about understanding of the behavior, the\nintent behind the behavior and what the individuals themselves say about their\nbehavior. This study collected pro-NSSI public blog posts from Reddit on\npro-NSSI and analyzed the content linguistically using LIWC software, in order\nto examine the use of NSSI specific words, linguistic properties and the\npsychological linguistic properties. were examined. The results inform current\ncounseling practices by dispelling myths and providing insight into the inner\nworld of people who engage in use NSSII to cope. The most frequently appearing\ncategory of For NSSI specific words categories, in the Reddit blogs was the\nreasons in which one engagesfor engaging in NSSI was the most frequently used\nin the Reddit blogs. The linguistic properties found in the analysis reflected\nthe predicted results; authors of pro-NSSI posts used demonstrated expected\nresults of first-person singular pronouns extensively, which indicatesing high\nlevels of mental health distress and isolation. The psychological linguistic\nproperties that could be observed of in these public Reddit posts were\ndominantly in a negative emotional tone which demonstrates youth and\nimpulsivity. The linguistic properties found when these posts were analyzed\nsupports the work of earlier studies that dispelled common myths about NSSI\nthat were circulating in the mental health community. These findings suggest\nthat the language of people who engage in NSSI supports research findings in\ndispelling common myths about NSSI.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1902.07248v1",
            "date": "2019-02-13 05:43:00+00:00",
            "title": "Sentence Compression via DC Programming Approach",
            "authors": [
                "Yi-Shuai Niu",
                "Xi-Wei Hu",
                "Yu You",
                "Faouzi Mohamed Benammour",
                "Hu Zhang"
            ],
            "abstract": "Sentence compression is an important problem in natural language processing.\nIn this paper, we firstly establish a new sentence compression model based on\nthe probability model and the parse tree model. Our sentence compression model\nis equivalent to an integer linear program (ILP) which can both guarantee the\nsyntax correctness of the compression and save the main meaning. We propose\nusing a DC (Difference of convex) programming approach (DCA) for finding local\noptimal solution of our model. Combing DCA with a parallel-branch-and-bound\nframework, we can find global optimal solution. Numerical results demonstrate\nthe good quality of our sentence compression model and the excellent\nperformance of our proposed solution algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07282v1",
            "date": "2019-02-19 21:03:35+00:00",
            "title": "Semantic Neural Machine Translation using AMR",
            "authors": [
                "Linfeng Song",
                "Daniel Gildea",
                "Yue Zhang",
                "Zhiguo Wang",
                "Jinsong Su"
            ],
            "abstract": "It is intuitive that semantic representations can be useful for machine\ntranslation, mainly because they can help in enforcing meaning preservation and\nhandling data sparsity (many sentences correspond to one meaning) of machine\ntranslation models. On the other hand, little work has been done on leveraging\nsemantics for neural machine translation (NMT). In this work, we study the\nusefulness of AMR (short for abstract meaning representation) on NMT.\nExperiments on a standard English-to-German dataset show that incorporating AMR\nas additional knowledge can significantly improve a strong attention-based\nsequence-to-sequence neural translation model.",
            "journal ref": "Transactions of the Association for Computational Linguistics, 7,\n  pages19-31, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07814v2",
            "date": "2019-02-20 23:56:21+00:00",
            "title": "Learning Dual Retrieval Module for Semi-supervised Relation Extraction",
            "authors": [
                "Hongtao Lin",
                "Jun Yan",
                "Meng Qu",
                "Xiang Ren"
            ],
            "abstract": "Relation extraction is an important task in structuring content of text data,\nand becomes especially challenging when learning with weak supervision---where\nonly a limited number of labeled sentences are given and a large number of\nunlabeled sentences are available. Most existing work exploits unlabeled data\nbased on the ideas of self-training (i.e., bootstrapping a model) and\nmulti-view learning (e.g., ensembling multiple model variants). However, these\nmethods either suffer from the issue of semantic drift, or do not fully capture\nthe problem characteristics of relation extraction. In this paper, we leverage\na key insight that retrieving sentences expressing a relation is a dual task of\npredicting relation label for a given sentence---two tasks are complementary to\neach other and can be optimized jointly for mutual enhancement. To model this\nintuition, we propose DualRE, a principled framework that introduces a\nretrieval module which is jointly trained with the original relation prediction\nmodule. In this way, high-quality samples selected by retrieval module from\nunlabeled data can be used to improve prediction module, and vice versa.\nExperimental results\\footnote{\\small Code and data can be found at\n\\url{https://github.com/INK-USC/DualRE}.} on two public datasets as well as\ncase studies demonstrate the effectiveness of the DualRE approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07816v2",
            "date": "2019-02-20 23:57:35+00:00",
            "title": "Mixture Models for Diverse Machine Translation: Tricks of the Trade",
            "authors": [
                "Tianxiao Shen",
                "Myle Ott",
                "Michael Auli",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Mixture models trained via EM are among the simplest, most widely used and\nwell understood latent variable models in the machine learning literature.\nSurprisingly, these models have been hardly explored in text generation\napplications such as machine translation. In principle, they provide a latent\nvariable to control generation and produce a diverse set of hypotheses. In\npractice, however, mixture models are prone to degeneracies---often only one\ncomponent gets trained or the latent variable is simply ignored. We find that\ndisabling dropout noise in responsibility computation is critical to successful\ntraining. In addition, the design choices of parameterization, prior\ndistribution, hard versus soft EM and online versus offline assignment can\ndramatically affect model performance. We develop an evaluation protocol to\nassess both quality and diversity of generations against multiple references,\nand provide an extensive empirical study of several mixture model variants. Our\nanalysis shows that certain types of mixture models are more robust and offer\nthe best trade-off between translation quality and diversity compared to\nvariational models and diverse decoding approaches.\\footnote{Code to reproduce\nthe results in this paper is available at\n\\url{https://github.com/pytorch/fairseq}}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07831v1",
            "date": "2019-02-21 01:12:07+00:00",
            "title": "Predicting ConceptNet Path Quality Using Crowdsourced Assessments of Naturalness",
            "authors": [
                "Yilun Zhou",
                "Steven Schockaert",
                "Julie A. Shah"
            ],
            "abstract": "In many applications, it is important to characterize the way in which two\nconcepts are semantically related. Knowledge graphs such as ConceptNet provide\na rich source of information for such characterizations by encoding relations\nbetween concepts as edges in a graph. When two concepts are not directly\nconnected by an edge, their relationship can still be described in terms of the\npaths that connect them. Unfortunately, many of these paths are uninformative\nand noisy, which means that the success of applications that use such path\nfeatures crucially relies on their ability to select high-quality paths. In\nexisting applications, this path selection process is based on relatively\nsimple heuristics. In this paper we instead propose to learn to predict path\nquality from crowdsourced human assessments. Since we are interested in a\ngeneric task-independent notion of quality, we simply ask human participants to\nrank paths according to their subjective assessment of the paths' naturalness,\nwithout attempting to define naturalness or steering the participants towards\nparticular indicators of quality. We show that a neural network model trained\non these assessments is able to predict human judgments on unseen paths with\nnear optimal performance. Most notably, we find that the resulting path\nselection method is substantially better than the current heuristic approaches\nat identifying meaningful paths.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08327v2",
            "date": "2019-02-22 01:26:05+00:00",
            "title": "On How Users Edit Computer-Generated Visual Stories",
            "authors": [
                "Ting-Yao Hsu",
                "Yen-Chia Hsu",
                "Ting-Hao 'Kenneth' Huang"
            ],
            "abstract": "A significant body of research in Artificial Intelligence (AI) has focused on\ngenerating stories automatically, either based on prior story plots or input\nimages. However, literature has little to say about how users would receive and\nuse these stories. Given the quality of stories generated by modern AI\nalgorithms, users will nearly inevitably have to edit these stories before\nputting them to real use. In this paper, we present the first analysis of how\nhuman users edit machine-generated stories. We obtained 962 short stories\ngenerated by one of the state-of-the-art visual storytelling models. For each\nstory, we recruited five crowd workers from Amazon Mechanical Turk to edit it.\nOur analysis of these edits shows that, on average, users (i) slightly\nshortened machine-generated stories, (ii) increased lexical diversity in these\nstories, and (iii) often replaced nouns and their determiners/articles with\npronouns. Our study provides a better understanding on how users receive and\nedit machine-generated stories,informing future researchers to create more\nusable and helpful story generation systems.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1902.08373v1",
            "date": "2019-02-22 06:28:36+00:00",
            "title": "Learning to Learn Semantic Parsers from Natural Language Supervision",
            "authors": [
                "Igor Labutov",
                "Bishan Yang",
                "Tom Mitchell"
            ],
            "abstract": "As humans, we often rely on language to learn language. For example, when\ncorrected in a conversation, we may learn from that correction, over time\nimproving our language fluency. Inspired by this observation, we propose a\nlearning algorithm for training semantic parsers from supervision (feedback)\nexpressed in natural language. Our algorithm learns a semantic parser from\nusers' corrections such as \"no, what I really meant was before his job, not\nafter\", by also simultaneously learning to parse this natural language feedback\nin order to leverage it as a form of supervision. Unlike supervision with\ngold-standard logical forms, our method does not require the user to be\nfamiliar with the underlying logical formalism, and unlike supervision from\ndenotation, it does not require the user to know the correct answer to their\nquery. This makes our learning algorithm naturally scalable in settings where\nexisting conversational logs are available and can be leveraged as training\ndata. We construct a novel dataset of natural language feedback in a\nconversational setting, and show that our method is effective at learning a\nsemantic parser from such natural language supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08558v1",
            "date": "2019-02-22 17:02:18+00:00",
            "title": "Topology and dynamics of narratives on Brexit propagated by UK press during 2016 and 2017",
            "authors": [
                "Jorge Lou\u00e7\u00e3",
                "Ant\u00f3nio Fonseca"
            ],
            "abstract": "This article identifies and characterises political narratives regarding\nEurope and broadcasted in UK press during 2016 and 2017. A new theoretical and\noperational framework is proposed for typifying discourse narratives propagated\nin the public opinion space, based on the social constructivism and structural\nlinguistics approaches, and the mathematical theory of hypernetworks, where\nelementary units are aggregated into high-level entities. In this line of\nthought, a narrative is understood as a social construct where a related and\ncoherent aggregate of terms within public discourse is repeated and propagated\non media until it can be identified as a communication pattern, embodying\nmeaning in a way that provides individuals some interpretation of their world.\nAn inclusive methodology, with state-of-the-art technologies on natural\nlanguage processing and network theory, implements this concept of narrative. A\ncorpus from the Observatorium database, including articles from six UK\nnewspapers and incorporating far-right, right-wing, and left-wing narratives,\nis analysed. The research revealed clear distinctions between narratives along\nthe political spectrum. In 2016 far-right was particularly focused on\nemigration and refugees. Namely, during the referendum campaign, Europe was\nrelated to attacks on women and children, sexual offences, and terrorism.\nRight-wing was manly focused on internal politics, while left-wing was\nremarkably mentioning a diversity of non-political topics, such as sports, side\nby side with economics. During 2017, in general terrorism was less mentioned,\nand negotiations with EU, namely regarding economics, finance, and Ireland,\nbecame central.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1902.08646v2",
            "date": "2019-02-22 19:27:45+00:00",
            "title": "OpenKiwi: An Open Source Framework for Quality Estimation",
            "authors": [
                "F\u00e1bio Kepler",
                "Jonay Tr\u00e9nous",
                "Marcos Treviso",
                "Miguel Vera",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "We introduce OpenKiwi, a PyTorch-based open source framework for translation\nquality estimation. OpenKiwi supports training and testing of word-level and\nsentence-level quality estimation systems, implementing the winning systems of\nthe WMT 2015-18 quality estimation campaigns. We benchmark OpenKiwi on two\ndatasets from WMT 2018 (English-German SMT and NMT), yielding state-of-the-art\nperformance on the word-level tasks and near state-of-the-art in the\nsentence-level tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08858v2",
            "date": "2019-02-23 22:27:45+00:00",
            "title": "Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models",
            "authors": [
                "Tiancheng Zhao",
                "Kaige Xie",
                "Maxine Eskenazi"
            ],
            "abstract": "Defining action spaces for conversational agents and optimizing their\ndecision-making process with reinforcement learning is an enduring challenge.\nCommon practice has been to use handcrafted dialog acts, or the output\nvocabulary, e.g. in neural encoder decoders, as the action spaces. Both have\ntheir own limitations. This paper proposes a novel latent action framework that\ntreats the action spaces of an end-to-end dialog agent as latent variables and\ndevelops unsupervised methods in order to induce its own action space from the\ndata. Comprehensive experiments are conducted examining both continuous and\ndiscrete action types and two different optimization methods based on\nstochastic variational inference. Results show that the proposed latent actions\nachieve superior empirical performance improvement over previous word-level\npolicy gradient methods on both DealOrNoDeal and MultiWoz dialogs. Our detailed\nanalysis also provides insights about various latent variable approaches for\npolicy learning and can serve as a foundation for developing better latent\nactions in future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08955v1",
            "date": "2019-02-24 14:44:07+00:00",
            "title": "Synchronous Bidirectional Inference for Neural Sequence Generation",
            "authors": [
                "Jiajun Zhang",
                "Long Zhou",
                "Yang Zhao",
                "Chengqing Zong"
            ],
            "abstract": "In sequence to sequence generation tasks (e.g. machine translation and\nabstractive summarization), inference is generally performed in a left-to-right\nmanner to produce the result token by token. The neural approaches, such as\nLSTM and self-attention networks, are now able to make full use of all the\npredicted history hypotheses from left side during inference, but cannot\nmeanwhile access any future (right side) information and usually generate\nunbalanced outputs in which left parts are much more accurate than right ones.\nIn this work, we propose a synchronous bidirectional inference model to\ngenerate outputs using both left-to-right and right-to-left decoding\nsimultaneously and interactively. First, we introduce a novel beam search\nalgorithm that facilitates synchronous bidirectional decoding. Then, we present\nthe core approach which enables left-to-right and right-to-left decoding to\ninteract with each other, so as to utilize both the history and future\npredictions simultaneously during inference. We apply the proposed model to\nboth LSTM and self-attention networks. In addition, we propose two strategies\nfor parameter optimization. The extensive experiments on machine translation\nand abstractive summarization demonstrate that our synchronous bidirectional\ninference model can achieve remarkable improvements over the strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09092v1",
            "date": "2019-02-25 05:04:11+00:00",
            "title": "Transfer Learning for Sequences via Learning to Collocate",
            "authors": [
                "Wanyun Cui",
                "Guangyu Zheng",
                "Zhiqiang Shen",
                "Sihang Jiang",
                "Wei Wang"
            ],
            "abstract": "Transfer learning aims to solve the data sparsity for a target domain by\napplying information of the source domain. Given a sequence (e.g. a natural\nlanguage sentence), the transfer learning, usually enabled by recurrent neural\nnetwork (RNN), represents the sequential information transfer. RNN uses a chain\nof repeating cells to model the sequence data. However, previous studies of\nneural network based transfer learning simply represents the whole sentence by\na single vector, which is unfeasible for seq2seq and sequence labeling.\nMeanwhile, such layer-wise transfer learning mechanisms lose the fine-grained\ncell-level information from the source domain.\n  In this paper, we proposed the aligned recurrent transfer, ART, to achieve\ncell-level information transfer. ART is under the pre-training framework. Each\ncell attentively accepts transferred information from a set of positions in the\nsource domain. Therefore, ART learns the cross-domain word collocations in a\nmore flexible way. We conducted extensive experiments on both sequence labeling\ntasks (POS tagging, NER) and sentence classification (sentiment analysis). ART\noutperforms the state-of-the-arts over all experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09093v1",
            "date": "2019-02-25 05:04:26+00:00",
            "title": "Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds",
            "authors": [
                "Igor Labutov",
                "Bishan Yang",
                "Anusha Prakash",
                "Amos Azaria"
            ],
            "abstract": "Question Answering (QA), as a research field, has primarily focused on either\nknowledge bases (KBs) or free text as a source of knowledge. These two sources\nhave historically shaped the kinds of questions that are asked over these\nsources, and the methods developed to answer them. In this work, we look\ntowards a practical use-case of QA over user-instructed knowledge that uniquely\ncombines elements of both structured QA over knowledge bases, and unstructured\nQA over narrative, introducing the task of multi-relational QA over personal\nnarrative. As a first step towards this goal, we make three key contributions:\n(i) we generate and release TextWorldsQA, a set of five diverse datasets, where\neach dataset contains dynamic narrative that describes entities and relations\nin a simulated world, paired with variably compositional questions over that\nknowledge, (ii) we perform a thorough evaluation and analysis of several\nstate-of-the-art QA models and their variants at this task, and (iii) we\nrelease a lightweight Python-based framework we call TextWorlds for easily\ngenerating arbitrary additional worlds and narrative, with the goal of allowing\nthe community to create and share a growing collection of diverse worlds as a\ntest-bed for this task.",
            "journal ref": "ACL 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09243v2",
            "date": "2019-02-25 13:07:32+00:00",
            "title": "Pretraining-Based Natural Language Generation for Text Summarization",
            "authors": [
                "Haoyu Zhang",
                "Jianjun Xu",
                "Ji Wang"
            ],
            "abstract": "In this paper, we propose a novel pretraining-based encoder-decoder\nframework, which can generate the output sequence based on the input sequence\nin a two-stage manner. For the encoder of our model, we encode the input\nsequence into context representations using BERT. For the decoder, there are\ntwo stages in our model, in the first stage, we use a Transformer-based decoder\nto generate a draft output sequence. In the second stage, we mask each word of\nthe draft sequence and feed it to BERT, then by combining the input sequence\nand the draft representation generated by BERT, we use a Transformer-based\ndecoder to predict the refined word for each masked position. To the best of\nour knowledge, our approach is the first method which applies the BERT into\ntext generation tasks. As the first step in this direction, we evaluate our\nproposed method on the text summarization task. Experimental results show that\nour model achieves new state-of-the-art on both CNN/Daily Mail and New York\nTimes datasets.",
            "journal ref": "CoNLL'2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09476v1",
            "date": "2019-02-25 17:53:20+00:00",
            "title": "MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts",
            "authors": [
                "Sunil Mohan",
                "Donghui Li"
            ],
            "abstract": "This paper presents the formal release of MedMentions, a new manually\nannotated resource for the recognition of biomedical concepts. What\ndistinguishes MedMentions from other annotated biomedical corpora is its size\n(over 4,000 abstracts and over 350,000 linked mentions), as well as the size of\nthe concept ontology (over 3 million concepts from UMLS 2017) and its broad\ncoverage of biomedical disciplines. In addition to the full corpus, a\nsub-corpus of MedMentions is also presented, comprising annotations for a\nsubset of UMLS 2017 targeted towards document retrieval. To encourage research\nin Biomedical Named Entity Recognition and Linking, data splits for training\nand testing are included in the release, and a baseline model and its metrics\nfor entity linking are also described.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09492v2",
            "date": "2019-02-25 18:14:11+00:00",
            "title": "Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing",
            "authors": [
                "Tal Schuster",
                "Ori Ram",
                "Regina Barzilay",
                "Amir Globerson"
            ],
            "abstract": "We introduce a novel method for multilingual transfer that utilizes deep\ncontextual embeddings, pretrained in an unsupervised fashion. While contextual\nembeddings have been shown to yield richer representations of meaning compared\nto their static counterparts, aligning them poses a challenge due to their\ndynamic nature. To this end, we construct context-independent variants of the\noriginal monolingual spaces and utilize their mapping to derive an alignment\nfor the context-dependent spaces. This mapping readily supports processing of a\ntarget language, improving transfer by context-aware embeddings. Our\nexperimental results demonstrate the effectiveness of this approach for\nzero-shot and few-shot learning of dependency parsing. Specifically, our method\nconsistently outperforms the previous state-of-the-art on 6 tested languages,\nyielding an improvement of 6.8 LAS points on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09508v2",
            "date": "2019-02-25 18:39:42+00:00",
            "title": "Improving Robustness of Machine Translation with Synthetic Noise",
            "authors": [
                "Vaibhav Vaibhav",
                "Sumeet Singh",
                "Craig Stewart",
                "Graham Neubig"
            ],
            "abstract": "Modern Machine Translation (MT) systems perform consistently well on clean,\nin-domain text. However most human generated text, particularly in the realm of\nsocial media, is full of typos, slang, dialect, idiolect and other noise which\ncan have a disastrous impact on the accuracy of output translation. In this\npaper we leverage the Machine Translation of Noisy Text (MTNT) dataset to\nenhance the robustness of MT systems by emulating naturally occurring noise in\notherwise clean data. Synthesizing noise in this manner we are ultimately able\nto make a vanilla MT system resilient to naturally occurring noise and\npartially mitigate loss in accuracy resulting therefrom.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09723v2",
            "date": "2019-02-26 04:32:42+00:00",
            "title": "Syntactic Recurrent Neural Network for Authorship Attribution",
            "authors": [
                "Fereshteh Jafariakinabad",
                "Sansiri Tarnpradab",
                "Kien A. Hua"
            ],
            "abstract": "Writing style is a combination of consistent decisions at different levels of\nlanguage production including lexical, syntactic, and structural associated to\na specific author (or author groups). While lexical-based models have been\nwidely explored in style-based text classification, relying on content makes\nthe model less scalable when dealing with heterogeneous data comprised of\nvarious topics. On the other hand, syntactic models which are\ncontent-independent, are more robust against topic variance. In this paper, we\nintroduce a syntactic recurrent neural network to encode the syntactic patterns\nof a document in a hierarchical structure. The model first learns the syntactic\nrepresentation of sentences from the sequence of part-of-speech tags. For this\npurpose, we exploit both convolutional filters and long short-term memories to\ninvestigate the short-term and long-term dependencies of part-of-speech tags in\nthe sentences. Subsequently, the syntactic representations of sentences are\naggregated into document representation using recurrent neural networks. Our\nexperimental results on PAN 2012 dataset for authorship attribution task shows\nthat syntactic recurrent neural network outperforms the lexical model with the\nidentical architecture by approximately 14% in terms of accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09774v1",
            "date": "2019-02-26 07:30:43+00:00",
            "title": "Image-Question-Answer Synergistic Network for Visual Dialog",
            "authors": [
                "Dalu Guo",
                "Chang Xu",
                "Dacheng Tao"
            ],
            "abstract": "The image, question (combined with the history for de-referencing), and the\ncorresponding answer are three vital components of visual dialog. Classical\nvisual dialog systems integrate the image, question, and history to search for\nor generate the best matched answer, and so, this approach significantly\nignores the role of the answer. In this paper, we devise a novel\nimage-question-answer synergistic network to value the role of the answer for\nprecise visual dialog. We extend the traditional one-stage solution to a\ntwo-stage solution. In the first stage, candidate answers are coarsely scored\naccording to their relevance to the image and question pair. Afterward, in the\nsecond stage, answers with high probability of being correct are re-ranked by\nsynergizing with image and question. On the Visual Dialog v1.0 dataset, the\nproposed synergistic network boosts the discriminative visual dialog model to\nachieve a new state-of-the-art of 57.88\\% normalized discounted cumulative\ngain. A generative visual dialog model equipped with the proposed technique\nalso shows promising improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10102v2",
            "date": "2019-02-26 18:30:50+00:00",
            "title": "A multimodal movie review corpus for fine-grained opinion mining",
            "authors": [
                "Alexandre Garcia",
                "Slim Essid",
                "Florence d'Alch\u00e9-Buc",
                "Chlo\u00e9 Clavel"
            ],
            "abstract": "In this paper, we introduce a set of opinion annotations for the POM movie\nreview dataset, composed of 1000 videos. The annotation campaign is motivated\nby the development of a hierarchical opinion prediction framework allowing one\nto predict the different components of the opinions (e.g. polarity and aspect)\nand to identify the corresponding textual spans. The resulting annotations have\nbeen gathered at two granularity levels: a coarse one (opinionated span) and a\nfiner one (span of opinion components). We introduce specific categories in\norder to make the annotation of opinions easier for movie reviews. For example,\nsome categories allow the discovery of user recommendation and preference in\nmovie reviews. We provide a quantitative analysis of the annotations and report\nthe inter-annotator agreement under the different levels of granularity. We\nprovide thus the first set of ground-truth annotations which can be used for\nthe task of fine-grained multimodal opinion prediction. We provide an analysis\nof the data gathered through an inter-annotator study and show that a linear\nstructured predictor learns meaningful features even for the prediction of\nscarce labels. Both the annotations and the baseline system are made publicly\navailable. https://github.com/eusip/POM/",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1902.10186v3",
            "date": "2019-02-26 19:59:15+00:00",
            "title": "Attention is not Explanation",
            "authors": [
                "Sarthak Jain",
                "Byron C. Wallace"
            ],
            "abstract": "Attention mechanisms have seen wide adoption in neural NLP models. In\naddition to improving predictive performance, these are often touted as\naffording transparency: models equipped with attention provide a distribution\nover attended-to input units, and this is often presented (at least implicitly)\nas communicating the relative importance of inputs. However, it is unclear what\nrelationship exists between attention weights and model outputs. In this work,\nwe perform extensive experiments across a variety of NLP tasks that aim to\nassess the degree to which attention weights provide meaningful `explanations'\nfor predictions. We find that they largely do not. For example, learned\nattention weights are frequently uncorrelated with gradient-based measures of\nfeature importance, and one can identify very different attention distributions\nthat nonetheless yield equivalent predictions. Our findings show that standard\nattention modules do not provide meaningful explanations and should not be\ntreated as though they do. Code for all experiments is available at\nhttps://github.com/successar/AttentionExplanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10339v4",
            "date": "2019-02-27 05:57:13+00:00",
            "title": "How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection",
            "authors": [
                "Wenhu Chen",
                "Yu Su",
                "Yilin Shen",
                "Zhiyu Chen",
                "Xifeng Yan",
                "William Wang"
            ],
            "abstract": "With the rapid development in deep learning, deep neural networks have been\nwidely adopted in many real-life natural language applications. Under deep\nneural networks, a pre-defined vocabulary is required to vectorize text inputs.\nThe canonical approach to select pre-defined vocabulary is based on the word\nfrequency, where a threshold is selected to cut off the long tail distribution.\nHowever, we observed that such simple approach could easily lead to under-sized\nvocabulary or over-sized vocabulary issues. Therefore, we are interested in\nunderstanding how the end-task classification accuracy is related to the\nvocabulary size and what is the minimum required vocabulary size to achieve a\nspecific performance. In this paper, we provide a more sophisticated\nvariational vocabulary dropout (VVD) based on variational dropout to perform\nvocabulary selection, which can intelligently select the subset of the\nvocabulary to achieve the required performance. To evaluate different\nalgorithms on the newly proposed vocabulary selection problem, we propose two\nnew metrics: Area Under Accuracy-Vocab Curve and Vocab Size under X\\% Accuracy\nDrop. Through extensive experiments on various NLP classification tasks, our\nvariational framework is shown to significantly outperform the frequency-based\nand other selection baselines on these metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10360v1",
            "date": "2019-02-27 07:04:47+00:00",
            "title": "An Editorial Network for Enhanced Document Summarization",
            "authors": [
                "Edward Moroshko",
                "Guy Feigenblat",
                "Haggai Roitman",
                "David Konopnicki"
            ],
            "abstract": "We suggest a new idea of Editorial Network - a mixed extractive-abstractive\nsummarization approach, which is applied as a post-processing step over a given\nsequence of extracted sentences. Our network tries to imitate the decision\nprocess of a human editor during summarization. Within such a process, each\nextracted sentence may be either kept untouched, rephrased or completely\nrejected. We further suggest an effective way for training the \"editor\" based\non a novel soft-labeling approach. Using the CNN/DailyMail dataset we\ndemonstrate the effectiveness of our approach compared to state-of-the-art\nextractive-only or abstractive-only baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10505v2",
            "date": "2019-02-27 13:08:27+00:00",
            "title": "Viable Dependency Parsing as Sequence Labeling",
            "authors": [
                "Michalina Strzyz",
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We recast dependency parsing as a sequence labeling problem, exploring\nseveral encodings of dependency trees as labels. While dependency parsing by\nmeans of sequence labeling had been attempted in existing work, results\nsuggested that the technique was impractical. We show instead that with a\nconventional BiLSTM-based model it is possible to obtain fast and accurate\nparsers. These parsers are conceptually simple, not needing traditional parsing\nalgorithms or auxiliary structures. However, experiments on the PTB and a\nsample of UD treebanks show that they provide a good speed-accuracy tradeoff,\nwith results competitive with more complex approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10547v3",
            "date": "2019-02-27 14:17:12+00:00",
            "title": "An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models",
            "authors": [
                "Alexandra Chronopoulou",
                "Christos Baziotis",
                "Alexandros Potamianos"
            ],
            "abstract": "A growing number of state-of-the-art transfer learning methods employ\nlanguage models pretrained on large generic corpora. In this paper we present a\nconceptually simple and effective transfer learning approach that addresses the\nproblem of catastrophic forgetting. Specifically, we combine the task-specific\noptimization function with an auxiliary language model objective, which is\nadjusted during the training process. This preserves language regularities\ncaptured by language models, while enabling sufficient adaptation for solving\nthe target task. Our method does not require pretraining or finetuning separate\ncomponents of the network and we train our models end-to-end in a single step.\nWe present results on a variety of challenging affective and text\nclassification tasks, surpassing well established transfer learning methods\nwith greater level of complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10667v2",
            "date": "2019-02-27 18:01:53+00:00",
            "title": "Bridging the Gap: Attending to Discontinuity in Identification of Multiword Expressions",
            "authors": [
                "Omid Rohanian",
                "Shiva Taslimipoor",
                "Samaneh Kouchaki",
                "Le An Ha",
                "Ruslan Mitkov"
            ],
            "abstract": "We introduce a new method to tag Multiword Expressions (MWEs) using a\nlinguistically interpretable language-independent deep learning architecture.\nWe specifically target discontinuity, an under-explored aspect that poses a\nsignificant challenge to computational treatment of MWEs. Two neural\narchitectures are explored: Graph Convolutional Network (GCN) and multi-head\nself-attention. GCN leverages dependency parse information, and self-attention\nattends to long-range relations. We finally propose a combined model that\nintegrates complementary information from both through a gating mechanism. The\nexperiments on a standard multilingual dataset for verbal MWEs show that our\nmodel outperforms the baselines not only in the case of discontinuous MWEs but\nalso in overall F-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10680v3",
            "date": "2019-02-27 18:45:09+00:00",
            "title": "Analyzing the Perceived Severity of Cybersecurity Threats Reported on Social Media",
            "authors": [
                "Shi Zong",
                "Alan Ritter",
                "Graham Mueller",
                "Evan Wright"
            ],
            "abstract": "Breaking cybersecurity events are shared across a range of websites,\nincluding security blogs (FireEye, Kaspersky, etc.), in addition to social\nmedia platforms such as Facebook and Twitter. In this paper, we investigate\nmethods to analyze the severity of cybersecurity threats based on the language\nthat is used to describe them online. A corpus of 6,000 tweets describing\nsoftware vulnerabilities is annotated with authors' opinions toward their\nseverity. We show that our corpus supports the development of automatic\nclassifiers with high precision for this task. Furthermore, we demonstrate the\nvalue of analyzing users' opinions about the severity of threats reported\nonline as an early indicator of important software vulnerabilities. We present\na simple, yet effective method for linking software vulnerabilities reported in\ntweets to Common Vulnerabilities and Exposures (CVEs) in the National\nVulnerability Database (NVD). Using our predicted severity scores, we show that\nit is possible to achieve a Precision@50 of 0.86 when forecasting high severity\nvulnerabilities, significantly outperforming a baseline that is based on tweet\nvolume. Finally we show how reports of severe vulnerabilities online are\npredictive of real-world exploits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11109v1",
            "date": "2019-02-14 04:37:49+00:00",
            "title": "Actions Generation from Captions",
            "authors": [
                "Xuan Liang",
                "Yida Xu"
            ],
            "abstract": "Sequence transduction models have been widely explored in many natural\nlanguage processing tasks. However, the target sequence usually consists of\ndiscrete tokens which represent word indices in a given vocabulary. We barely\nsee the case where target sequence is composed of continuous vectors, where\neach vector is an element of a time series taken successively in a temporal\ndomain. In this work, we introduce a new data set, named Action Generation Data\nSet (AGDS) which is specifically designed to carry out the task of\ncaption-to-action generation. This data set contains caption-action pairs. The\ncaption is comprised of a sequence of words describing the interactive movement\nbetween two people, and the action is a captured sequence of poses representing\nthe movement. This data set is introduced to study the ability of generating\ncontinuous sequences through sequence transduction models. We also propose a\nmodel to innovatively combine Multi-Head Attention (MHA) and Generative\nAdversarial Network (GAN) together. In our model, we have one generator to\ngenerate actions from captions and three discriminators where each of them is\ndesigned to carry out a unique functionality: caption-action consistency\ndiscriminator, pose discriminator and pose transition discriminator. This novel\ndesign allowed us to achieve plausible generation performance which is\ndemonstrated in the experiments.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1902.11205v3",
            "date": "2019-02-28 16:45:19+00:00",
            "title": "Jointly Optimizing Diversity and Relevance in Neural Response Generation",
            "authors": [
                "Xiang Gao",
                "Sungjin Lee",
                "Yizhe Zhang",
                "Chris Brockett",
                "Michel Galley",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "Although recent neural conversation models have shown great potential, they\noften generate bland and generic responses. While various approaches have been\nexplored to diversify the output of the conversation model, the improvement\noften comes at the cost of decreased relevance. In this paper, we propose a\nSpaceFusion model to jointly optimize diversity and relevance that essentially\nfuses the latent space of a sequence-to-sequence model and that of an\nautoencoder model by leveraging novel regularization terms. As a result, our\napproach induces a latent space in which the distance and direction from the\npredicted response vector roughly match the relevance and diversity,\nrespectively. This property also lends itself well to an intuitive\nvisualization of the latent space. Both automatic and human evaluation results\ndemonstrate that the proposed approach brings significant improvement compared\nto strong baselines in both diversity and relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11269v1",
            "date": "2019-02-28 18:19:14+00:00",
            "title": "Efficient Contextual Representation Learning Without Softmax Layer",
            "authors": [
                "Liunian Harold Li",
                "Patrick H. Chen",
                "Cho-Jui Hsieh",
                "Kai-Wei Chang"
            ],
            "abstract": "Contextual representation models have achieved great success in improving\nvarious downstream tasks. However, these language-model-based encoders are\ndifficult to train due to the large parameter sizes and high computational\ncomplexity. By carefully examining the training procedure, we find that the\nsoftmax layer (the output layer) causes significant inefficiency due to the\nlarge vocabulary size. Therefore, we redesign the learning objective and\npropose an efficient framework for training contextual representation models.\nSpecifically, the proposed approach bypasses the softmax layer by performing\nlanguage modeling with dimension reduction, and allows the models to leverage\npre-trained word embeddings. Our framework reduces the time spent on the output\nlayer to a negligible level, eliminates almost all the trainable parameters of\nthe softmax layer and performs language modeling without truncating the\nvocabulary. When applied to ELMo, our method achieves a 4 times speedup and\neliminates 80% trainable parameters while achieving competitive performance on\ndownstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00066v1",
            "date": "2019-01-01 00:15:45+00:00",
            "title": "Improving Tree-LSTM with Tree Attention",
            "authors": [
                "Mahtab Ahmed",
                "Muhammad Rifayat Samee",
                "Robert E. Mercer"
            ],
            "abstract": "In Natural Language Processing (NLP), we often need to extract information\nfrom tree topology. Sentence structure can be represented via a dependency tree\nor a constituency tree structure. For this reason, a variant of LSTMs, named\nTree-LSTM, was proposed to work on tree topology. In this paper, we design a\ngeneralized attention framework for both dependency and constituency trees by\nencoding variants of decomposable attention inside a Tree-LSTM cell. We\nevaluated our models on a semantic relatedness task and achieved notable\nresults compared to Tree-LSTM based methods with no attention as well as other\nneural and non-neural methods and good results compared to Tree-LSTM based\nmethods with attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00603v2",
            "date": "2019-01-03 03:55:49+00:00",
            "title": "Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering",
            "authors": [
                "Victor Zhong",
                "Caiming Xiong",
                "Nitish Shirish Keskar",
                "Richard Socher"
            ],
            "abstract": "End-to-end neural models have made significant progress in question\nanswering, however recent studies show that these models implicitly assume that\nthe answer and evidence appear close together in a single document. In this\nwork, we propose the Coarse-grain Fine-grain Coattention Network (CFC), a new\nquestion answering model that combines information from evidence across\nmultiple documents. The CFC consists of a coarse-grain module that interprets\ndocuments with respect to the query then finds a relevant answer, and a\nfine-grain module which scores each candidate answer by comparing its\noccurrences across all of the documents with the query. We design these modules\nusing hierarchies of coattention and self-attention, which learn to emphasize\ndifferent parts of the input. On the Qangaroo WikiHop multi-evidence question\nanswering task, the CFC obtains a new state-of-the-art result of 70.6% on the\nblind test set, outperforming the previous best by 3% accuracy despite not\nusing pretrained contextual encoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01590v1",
            "date": "2019-01-06 18:30:50+00:00",
            "title": "Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder",
            "authors": [
                "Yunsu Kim",
                "Jiahui Geng",
                "Hermann Ney"
            ],
            "abstract": "Unsupervised learning of cross-lingual word embedding offers elegant matching\nof words across languages, but has fundamental limitations in translating\nsentences. In this paper, we propose simple yet effective methods to improve\nword-by-word translation of cross-lingual embeddings, using only monolingual\ncorpora but without any back-translation. We integrate a language model for\ncontext-aware search, and use a novel denoising autoencoder to handle\nreordering. Our system surpasses state-of-the-art unsupervised neural\ntranslation systems without costly iterative training. We also analyze the\neffect of vocabulary size and denoising type on the translation performance,\nwhich provides better understanding of learning the cross-lingual word\nembedding and its usage in translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01695v1",
            "date": "2019-01-07 08:03:35+00:00",
            "title": "Vector representations of text data in deep learning",
            "authors": [
                "Karol Grzegorczyk"
            ],
            "abstract": "In this dissertation we report results of our research on dense distributed\nrepresentations of text data. We propose two novel neural models for learning\nsuch representations. The first model learns representations at the document\nlevel, while the second model learns word-level representations.\n  For document-level representations we propose Binary Paragraph Vector: a\nneural network models for learning binary representations of text documents,\nwhich can be used for fast document retrieval. We provide a thorough evaluation\nof these models and demonstrate that they outperform the seminal method in the\nfield in the information retrieval task. We also report strong results in\ntransfer learning settings, where our models are trained on a generic text\ncorpus and then used to infer codes for documents from a domain-specific\ndataset. In contrast to previously proposed approaches, Binary Paragraph Vector\nmodels learn embeddings directly from raw text data.\n  For word-level representations we propose Disambiguated Skip-gram: a neural\nnetwork model for learning multi-sense word embeddings. Representations learned\nby this model can be used in downstream tasks, like part-of-speech tagging or\nidentification of semantic relations. In the word sense induction task\nDisambiguated Skip-gram outperforms state-of-the-art models on three out of\nfour benchmarks datasets. Our model has an elegant probabilistic\ninterpretation. Furthermore, unlike previous models of this kind, it is\ndifferentiable with respect to all its parameters and can be trained with\nbackpropagation. In addition to quantitative results, we present qualitative\nevaluation of Disambiguated Skip-gram, including two-dimensional visualisations\nof selected word-sense embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02081v1",
            "date": "2019-01-07 21:49:51+00:00",
            "title": "Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents",
            "authors": [
                "Artur Nowak",
                "Pawe\u0142 Kunstman"
            ],
            "abstract": "We describe our entry for the Systematic Review Information Extraction track\nof the 2018 Text Analysis Conference. Our solution is an end-to-end, deep\nlearning, sequence tagging model based on the BI-LSTM-CRF architecture.\nHowever, we use interleaved, alternating LSTM layers with highway connections\ninstead of the more traditional approach, where last hidden states of both\ndirections are concatenated to create an input to the next layer. We also make\nextensive use of pre-trained word embeddings, namely GloVe and ELMo. Thanks to\na number of regularization techniques, we were able to achieve relatively large\ncapacity of the model (31.3M+ of trainable parameters) for the size of training\nset (100 documents, less than 200K tokens). The system's official score was\n60.9% (micro-F1) and it ranked first for the Task 1. Additionally, after\nrectifying an obvious mistake in the submission format, the system scored\n67.35%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02360v2",
            "date": "2019-01-05 23:34:29+00:00",
            "title": "Sum-of-square-of-rational-function based representations of positive semidefinite polynomial matrices",
            "authors": [
                "Thanh-Hieu Le",
                "Nhat-Thien Pham"
            ],
            "abstract": "The paper proves sum-of-square-of-rational-function based representations\n(shortly, sosrf-based representations) of polynomial matrices that are positive\nsemidefinite on some special sets: $\\mathbb{R}^n;$ $\\mathbb{R}$ and its\nintervals $[a,b]$, $[0,\\infty)$; and the strips $[a,b] \\times \\mathbb{R}\n\\subset \\mathbb{R}^2.$ A method for numerically computing such representations\nis also presented. The methodology is divided into two stages:\n  (S1) diagonalizing the initial polynomial matrix based on the Schm\\\"{u}dgen's\nprocedure \\cite{Schmudgen09};\n  (S2) for each diagonal element of the resulting matrix, find its low rank\nsosrf-representation satisfying the Artin's theorem solving the Hilbert's 17th\nproblem.\n  Some numerical tests and illustrations with \\textsf{OCTAVE} are also\npresented for each type of polynomial matrices.",
            "journal ref": "None",
            "category": "math.OC"
        },
        {
            "url": "http://arxiv.org/abs/1901.02522v3",
            "date": "2019-01-08 21:19:34+00:00",
            "title": "On the Possibilities and Limitations of Multi-hop Reasoning Under Linguistic Imperfections",
            "authors": [
                "Daniel Khashabi",
                "Erfan Sadeqi Azer",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Dan Roth"
            ],
            "abstract": "Systems for language understanding have become remarkably strong at\novercoming linguistic imperfections in tasks involving phrase matching or\nsimple reasoning. Yet, their accuracy drops dramatically as the number of\nreasoning steps increases. We present the first formal framework to study such\nempirical observations. It allows one to quantify the amount and effect of\nambiguity, redundancy, incompleteness, and inaccuracy that the use of language\nintroduces when representing a hidden conceptual space. The idea is to consider\ntwo interrelated spaces: a conceptual meaning space that is unambiguous and\ncomplete but hidden, and a linguistic space that captures a noisy grounding of\nthe meaning space in the words of a language---the level at which all systems,\nwhether neural or symbolic, operate. Applying this framework to a special class\nof multi-hop reasoning, namely the connectivity problem in graphs of\nrelationships between concepts, we derive rigorous intuitions and impossibility\nresults even under this simplified setting. For instance, if a query requires a\nmoderately large (logarithmic) number of hops in the meaning graph, no\nreasoning system operating over a noisy graph grounded in language is likely to\ncorrectly answer it. This highlights a fundamental barrier that extends to a\nbroader class of reasoning problems and systems, and suggests an alternative\npath forward: focusing on aligning the two spaces via richer representations,\nbefore investing in reasoning with many hops.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02539v1",
            "date": "2019-01-08 22:24:59+00:00",
            "title": "Supervised Transfer Learning for Product Information Question Answering",
            "authors": [
                "Tuan Manh Lai",
                "Trung Bui",
                "Nedim Lipka",
                "Sheng Li"
            ],
            "abstract": "Popular e-commerce websites such as Amazon offer community question answering\nsystems for users to pose product related questions and experienced customers\nmay provide answers voluntarily. In this paper, we show that the large volume\nof existing community question answering data can be beneficial when building a\nsystem for answering questions related to product facts and specifications. Our\nexperimental results demonstrate that the performance of a model for answering\nquestions related to products listed in the Home Depot website can be improved\nby a large margin via a simple transfer learning technique from an existing\nlarge-scale Amazon community question answering dataset. Transfer learning can\nresult in an increase of about 10% in accuracy in the experimental setting\nwhere we restrict the size of the data of the target task used for training. As\nan application of this work, we integrate the best performing model trained in\nthis work into a mobile-based shopping assistant and show its usefulness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02543v1",
            "date": "2019-01-08 22:35:03+00:00",
            "title": "Computational Register Analysis and Synthesis",
            "authors": [
                "Shlomo Engelson Argamon"
            ],
            "abstract": "The study of register in computational language research has historically\nbeen divided into register analysis, seeking to determine the registerial\ncharacter of a text or corpus, and register synthesis, seeking to generate a\ntext in a desired register. This article surveys the different approaches to\nthese disparate tasks. Register synthesis has tended to use more theoretically\narticulated notions of register and genre than analysis work, which often seeks\nto categorize on the basis of intuitive and somewhat incoherent notions of\nprelabeled 'text types'. I argue that an integration of computational register\nanalysis and synthesis will benefit register studies as a whole, by enabling a\nnew large-scale research program in register studies. It will enable\ncomprehensive global mapping of functional language varieties in multiple\nlanguages, including the relationships between them. Furthermore, computational\nmethods together with high coverage systematically collected and analyzed data\nwill thus enable rigorous empirical validation and refinement of different\ntheories of register, which will have also implications for our understanding\nof linguistic variation in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03141v1",
            "date": "2019-01-10 13:15:46+00:00",
            "title": "Emotion Detection using Data Driven Models",
            "authors": [
                "Naveenkumar K S",
                "Vinayakumar R",
                "Soman KP"
            ],
            "abstract": "Text is the major method that is used for communication now a days, each and\nevery day lots of text are created. In this paper the text data is used for the\nclassification of the emotions. Emotions are the way of expression of the\npersons feelings which has an high influence on the decision making tasks.\nDatasets are collected which are available publically and combined together\nbased on the three emotions that are considered here positive, negative and\nneutral. In this paper we have proposed the text representation method TFIDF\nand keras embedding and then given to the classical machine learning algorithms\nof which Logistics Regression gives the highest accuracy of about 75.6%, after\nwhich it is passed to the deep learning algorithm which is the CNN which gives\nthe state of art accuracy of about 45.25%. For the research purpose the\ndatasets that has been collected are released.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.03253v3",
            "date": "2019-01-10 16:25:53+00:00",
            "title": "Reverse-Engineering Satire, or \"Paper on Computational Humor Accepted Despite Making Serious Advances\"",
            "authors": [
                "Robert West",
                "Eric Horvitz"
            ],
            "abstract": "Humor is an essential human trait. Efforts to understand humor have called\nout links between humor and the foundations of cognition, as well as the\nimportance of humor in social engagement. As such, it is a promising and\nimportant subject of study, with relevance for artificial intelligence and\nhuman-computer interaction. Previous computational work on humor has mostly\noperated at a coarse level of granularity, e.g., predicting whether an entire\nsentence, paragraph, document, etc., is humorous. As a step toward deep\nunderstanding of humor, we seek fine-grained models of attributes that make a\ngiven text humorous. Starting from the observation that satirical news\nheadlines tend to resemble serious news headlines, we build and analyze a\ncorpus of satirical headlines paired with nearly identical but serious\nheadlines. The corpus is constructed via Unfun.me, an online game that\nincentivizes players to make minimal edits to satirical headlines with the goal\nof making other players believe the results are serious headlines. The edit\noperations used to successfully remove humor pinpoint the words and concepts\nthat play a key role in making the original, satirical headline funny. Our\nanalysis reveals that the humor tends to reside toward the end of headlines,\nand primarily in noun phrases, and that most satirical headlines follow a\ncertain logical pattern, which we term false analogy. Overall, this paper\ndeepens our understanding of the syntactic and semantic structure of satirical\nnews headlines and provides insights for building humor-producing systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1901.03489v1",
            "date": "2019-01-11 05:53:13+00:00",
            "title": "User Intent Prediction in Information-seeking Conversations",
            "authors": [
                "Chen Qu",
                "Liu Yang",
                "Bruce Croft",
                "Yongfeng Zhang",
                "Johanne R. Trippas",
                "Minghui Qiu"
            ],
            "abstract": "Conversational assistants are being progressively adopted by the general\npopulation. However, they are not capable of handling complicated\ninformation-seeking tasks that involve multiple turns of information exchange.\nDue to the limited communication bandwidth in conversational search, it is\nimportant for conversational assistants to accurately detect and predict user\nintent in information-seeking conversations. In this paper, we investigate two\naspects of user intent prediction in an information-seeking setting. First, we\nextract features based on the content, structural, and sentiment\ncharacteristics of a given utterance, and use classic machine learning methods\nto perform user intent prediction. We then conduct an in-depth feature\nimportance analysis to identify key features in this prediction task. We find\nthat structural features contribute most to the prediction performance. Given\nthis finding, we construct neural classifiers to incorporate context\ninformation and achieve better performance without feature engineering. Our\nfindings can provide insights into the important factors and effective methods\nof user intent prediction in information-seeking conversations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.03491v2",
            "date": "2019-01-11 06:02:22+00:00",
            "title": "Answer Interaction in Non-factoid Question Answering Systems",
            "authors": [
                "Chen Qu",
                "Liu Yang",
                "Bruce Croft",
                "Falk Scholer",
                "Yongfeng Zhang"
            ],
            "abstract": "Information retrieval systems are evolving from document retrieval to answer\nretrieval. Web search logs provide large amounts of data about how people\ninteract with ranked lists of documents, but very little is known about\ninteraction with answer texts. In this paper, we use Amazon Mechanical Turk to\ninvestigate three answer presentation and interaction approaches in a\nnon-factoid question answering setting. We find that people perceive and react\nto good and bad answers very differently, and can identify good answers\nrelatively quickly. Our results provide the basis for further investigation of\neffective answer interaction and feedback methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.03526v1",
            "date": "2019-01-11 09:39:55+00:00",
            "title": "On Event Causality Detection in Tweets",
            "authors": [
                "Humayun Kayesh",
                "Md. Saiful Islam",
                "Junhu Wang"
            ],
            "abstract": "Nowadays, Twitter has become a great source of user-generated information\nabout events. Very often people report causal relationships between events in\ntheir tweets. Automatic detection of causality information in these events\nmight play an important role in predictive event analytics. Existing approaches\ninclude both rule-based and data-driven supervised methods. However, it is\nchallenging to correctly identify event causality using only linguistic rules\ndue to the highly unstructured nature and grammatical incorrectness of social\nmedia short text such as tweets. Also, it is difficult to develop a data-driven\nsupervised method for event causality detection in tweets due to insufficient\ncontextual information. This paper proposes a novel event context word\nextension technique based on background knowledge. To demonstrate the\neffectiveness of our proposed event context word extension technique, we\ndevelop a feed-forward neural network based approach to detect event causality\nfrom tweets. Extensive experiments demonstrate the superiority of our approach.",
            "journal ref": "Griffith University Technical Report, 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.04216v1",
            "date": "2019-01-14 10:05:52+00:00",
            "title": "Albanian Language Identification in Text Documents",
            "authors": [
                "Klesti Hoxha",
                "Artur Baxhaku"
            ],
            "abstract": "In this work we investigate the accuracy of standard and state-of-the-art\nlanguage identification methods in identifying Albanian in written text\ndocuments. A dataset consisting of news articles written in Albanian has been\nconstructed for this purpose. We noticed a considerable decrease of accuracy\nwhen using test documents that miss the Albanian alphabet letters \" \\\"E \" and \"\n\\c{C} \" and created a custom training corpus that solved this problem by\nachieving an accuracy of more than 99%. Based on our experiments, the most\nperforming language identification methods for Albanian use a na\\\"ive Bayes\nclassifier and n-gram based classification features.",
            "journal ref": "Buletini i Shkencave te Natyres, Vol. 23, 2017",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.04299v1",
            "date": "2019-01-05 20:33:28+00:00",
            "title": "Quantum-theoretic Modeling in Computer Science A complex Hilbert space model for entangled concepts in corpuses of documents",
            "authors": [
                "Diederik Aerts",
                "Lester Beltran",
                "Suzette Geriente",
                "Sandro Sozzo"
            ],
            "abstract": "We work out a quantum-theoretic model in complex Hilbert space of a recently\nperformed test on co-occurrencies of two concepts and their combination in\nretrieval processes on specific corpuses of documents. The test violated the\nClauser-Horne-Shimony-Holt version of the Bell inequalities ('CHSH\ninequality'), thus indicating the presence of entanglement between the combined\nconcepts. We make use of a recently elaborated 'entanglement scheme' and\nrepresent the collected data in the tensor product of Hilbert spaces of the\nindividual concepts, showing that the identified violation is due to the\noccurrence of a strong form of entanglement, involving both states and\nmeasurements and reflecting the meaning connection between the component\nconcepts. These results provide a significant confirmation of the presence of\nquantum structures in corpuses of documents, like it is the case for the\nentanglement identified in human cognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04713v2",
            "date": "2019-01-15 08:55:53+00:00",
            "title": "Global-to-local Memory Pointer Networks for Task-Oriented Dialogue",
            "authors": [
                "Chien-Sheng Wu",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "End-to-end task-oriented dialogue is challenging since knowledge bases are\nusually large, dynamic and hard to incorporate into a learning framework. We\npropose the global-to-local memory pointer (GLMP) networks to address this\nissue. In our model, a global memory encoder and a local memory decoder are\nproposed to share external knowledge. The encoder encodes dialogue history,\nmodifies global contextual representation, and generates a global memory\npointer. The decoder first generates a sketch response with unfilled slots.\nNext, it passes the global memory pointer to filter the external knowledge for\nrelevant information, then instantiates the slots via the local memory\npointers. We empirically show that our model can improve copy accuracy and\nmitigate the common out-of-vocabulary problem. As a result, GLMP is able to\nimprove over the previous state-of-the-art models in both simulated bAbI\nDialogue dataset and human-human Stanford Multi-domain Dialogue dataset on\nautomatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04831v1",
            "date": "2019-01-15 14:10:25+00:00",
            "title": "Exploiting Synchronized Lyrics And Vocal Features For Music Emotion Detection",
            "authors": [
                "Loreto Parisi",
                "Simone Francia",
                "Silvio Olivastri",
                "Maria Stella Tavella"
            ],
            "abstract": "One of the key points in music recommendation is authoring engaging playlists\naccording to sentiment and emotions. While previous works were mostly based on\naudio for music discovery and playlists generation, we take advantage of our\nsynchronized lyrics dataset to combine text representations and music features\nin a novel way; we therefore introduce the Synchronized Lyrics Emotion Dataset.\nUnlike other approaches that randomly exploited the audio samples and the whole\ntext, our data is split according to the temporal information provided by the\nsynchronization between lyrics and audio. This work shows a comparison between\ntext-based and audio-based deep learning classification models using different\ntechniques from Natural Language Processing and Music Information Retrieval\ndomains. From the experiments on audio we conclude that using vocals only,\ninstead of the whole audio data improves the overall performances of the audio\nclassifier. In the lyrics experiments we exploit the state-of-the-art word\nrepresentations applied to the main Deep Learning architectures available in\nliterature. In our benchmarks the results show how the Bilinear LSTM classifier\nwith Attention based on fastText word embedding performs better than the CNN\napplied on audio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04936v1",
            "date": "2019-01-15 17:03:32+00:00",
            "title": "Incremental Reading for Question Answering",
            "authors": [
                "Samira Abnar",
                "Tania Bedrax-weiss",
                "Tom Kwiatkowski",
                "William W. Cohen"
            ],
            "abstract": "Any system which performs goal-directed continual learning must not only\nlearn incrementally but process and absorb information incrementally. Such a\nsystem also has to understand when its goals have been achieved. In this paper,\nwe consider these issues in the context of question answering. Current\nstate-of-the-art question answering models reason over an entire passage, not\nincrementally. As we will show, naive approaches to incremental reading, such\nas restriction to unidirectional language models in the model, perform poorly.\nWe present extensions to the DocQA [2] model to allow incremental reading\nwithout loss of accuracy. The model also jointly learns to provide the best\nanswer given the text that is seen so far and predict whether this best-so-far\nanswer is sufficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.05115v1",
            "date": "2019-01-16 02:37:59+00:00",
            "title": "Variable-sized input, character-level recurrent neural networks in lead generation: predicting close rates from raw user inputs",
            "authors": [
                "Giulio Giorcelli"
            ],
            "abstract": "Predicting lead close rates is one of the most problematic tasks in the lead\ngeneration industry. In most cases, the only available data on the prospect is\nthe self-reported information inputted by the user on the lead form and a few\nother data points publicly available through social media and search engine\nusage. All the major market niches for lead generation [1], such as insurance,\nhealth & medical and real estate, deal with life-altering decision making that\nno amount of data will be ever be able to describe or predict. This paper\nillustrates how character-level, deep long short-term memory networks can be\napplied to raw user inputs to help predict close rates. The output of the model\nis then used as an additional, highly predictive feature to significantly boost\nperformance of lead scoring models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06039v1",
            "date": "2019-01-18 00:09:24+00:00",
            "title": "Automatic Keyboard Layout Design for Low-Resource Latin-Script Languages",
            "authors": [
                "Theresa Breiner",
                "Chieu Nguyen",
                "Daan van Esch",
                "Jeremy O'Brien"
            ],
            "abstract": "We present our approach to automatically designing and implementing keyboard\nlayouts on mobile devices for typing low-resource languages written in the\nLatin script. For many speakers, one of the barriers in accessing and creating\ntext content on the web is the absence of input tools for their language. Ease\nin typing in these languages would lower technological barriers to online\ncommunication and collaboration, likely leading to the creation of more web\ncontent. Unfortunately, it can be time-consuming to develop layouts manually\neven for language communities that use a keyboard layout very similar to\nEnglish; starting from scratch requires many configuration files to describe\nmultiple possible behaviors for each key. With our approach, we only need a\nsmall amount of data in each language to generate keyboard layouts with very\nlittle human effort. This process can help serve speakers of low-resource\nlanguages in a scalable way, allowing us to develop input tools for more\nlanguages. Having input tools that reflect the linguistic diversity of the\nworld will let as many people as possible use technology to learn, communicate,\nand express themselves in their own native languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06168v1",
            "date": "2019-01-18 10:29:20+00:00",
            "title": "Identifying Unclear Questions in Community Question Answering Websites",
            "authors": [
                "Jan Trienes",
                "Krisztian Balog"
            ],
            "abstract": "Thousands of complex natural language questions are submitted to community\nquestion answering websites on a daily basis, rendering them as one of the most\nimportant information sources these days. However, oftentimes submitted\nquestions are unclear and cannot be answered without further clarification\nquestions by expert community members. This study is the first to investigate\nthe complex task of classifying a question as clear or unclear, i.e., if it\nrequires further clarification. We construct a novel dataset and propose a\nclassification approach that is based on the notion of similar questions. This\napproach is compared to state-of-the-art text classification baselines. Our\nmain finding is that the similar questions approach is a viable alternative\nthat can be used as a stepping stone towards the development of supportive user\ninterfaces for question formulation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.06613v2",
            "date": "2019-01-20 02:25:23+00:00",
            "title": "Beyond Turing: Intelligent Agents Centered on the User",
            "authors": [
                "Maxine Eskenazi",
                "Shikib Mehri",
                "Evgeniia Razumovskaia",
                "Tiancheng Zhao"
            ],
            "abstract": "Most research on intelligent agents centers on the agent and not on the user.\nWe look at the origins of agent-centric research for slot-filling, gaming and\nchatbot agents. We then argue that it is important to concentrate more on the\nuser. After reviewing relevant literature, some approaches for creating and\nassessing user-centric systems are proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07067v1",
            "date": "2019-01-21 20:21:05+00:00",
            "title": "Development of verification system of socio-demographic data of virtual community member",
            "authors": [
                "S. S. Fedushko"
            ],
            "abstract": "The important task of developing verification system of data of virtual\ncommunity member on the basis of computer-linguistic analysis of the content of\na large sample of Ukrainian virtual communities is solved. The subject of\nresearch is methods and tools for verification of web-members socio-demographic\ncharacteristics based on computer-linguistic analysis of their communicative\ninteraction results. The aim of paper is to verifying web-user personal data on\nthe basis of computer-linguistic analysis of web-members information tracks.\nThe structure of verification software for web-user profile is designed for a\npractical implementation of assigned tasks. The method of personal data\nverification of web-members by analyzing information track of virtual community\nmember is conducted. For the first time the method for checking the\nauthenticity of web members personal data, which helped to design of\nverification tool for socio-demographic characteristics of web-member is\ndeveloped. The verification system of data of web-members, which forms the\nverified socio-demographic profiles of web-members, is developed as a result of\nconducted experiments. Also the user interface of the developed verification\nsystem web-members data is presented. Effectiveness and efficiency of use of\nthe developed methods and means for solving tasks in web-communities\nadministration is proved by their approbation. The number of false results of\nverification system is 18%.",
            "journal ref": "Radio Electronics, Computer Science, Control. 2016",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1901.07696v2",
            "date": "2019-01-23 02:33:52+00:00",
            "title": "Product-Aware Answer Generation in E-Commerce Question-Answering",
            "authors": [
                "Shen Gao",
                "Zhaochun Ren",
                "Yihong Eric Zhao",
                "Dongyan Zhao",
                "Dawei Yin",
                "Rui Yan"
            ],
            "abstract": "In e-commerce portals, generating answers for product-related questions has\nbecome a crucial task. In this paper, we propose the task of product-aware\nanswer generation, which tends to generate an accurate and complete answer from\nlarge-scale unlabeled e-commerce reviews and product attributes. Unlike\nexisting question-answering problems, answer generation in e-commerce confronts\nthree main challenges: (1) Reviews are informal and noisy; (2) joint modeling\nof reviews and key-value product attributes is challenging; (3) traditional\nmethods easily generate meaningless answers. To tackle above challenges, we\npropose an adversarial learning based model, named PAAG, which is composed of\nthree components: a question-aware review representation module, a key-value\nmemory network encoding attributes, and a recurrent neural network as a\nsequence generator. Specifically, we employ a convolutional discriminator to\ndistinguish whether our generated answer matches the facts. To extract the\nsalience part of reviews, an attention-based review reader is proposed to\ncapture the most relevant words given the question. Conducted on a large-scale\nreal-world e-commerce dataset, our extensive experiments verify the\neffectiveness of each module in our proposed model. Moreover, our experiments\nshow that our model achieves the state-of-the-art performance in terms of both\nautomatic metrics and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07786v1",
            "date": "2019-01-23 09:39:45+00:00",
            "title": "Self-Attentive Model for Headline Generation",
            "authors": [
                "Daniil Gavrilov",
                "Pavel Kalaidin",
                "Valentin Malykh"
            ],
            "abstract": "Headline generation is a special type of text summarization task. While the\namount of available training data for this task is almost unlimited, it still\nremains challenging, as learning to generate headlines for news articles\nimplies that the model has strong reasoning about natural language. To overcome\nthis issue, we applied recent Universal Transformer architecture paired with\nbyte-pair encoding technique and achieved new state-of-the-art results on the\nNew York Times Annotated corpus with ROUGE-L F1-score 24.84 and ROUGE-2\nF1-score 13.48. We also present the new RIA corpus and reach ROUGE-L F1-score\n36.81 and ROUGE-2 F1-score 22.15 on it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07829v1",
            "date": "2019-01-23 11:38:15+00:00",
            "title": "AspeRa: Aspect-based Rating Prediction Model",
            "authors": [
                "Sergey I. Nikolenko",
                "Elena Tutubalina",
                "Valentin Malykh",
                "Ilya Shenbin",
                "Anton Alekseev"
            ],
            "abstract": "We propose a novel end-to-end Aspect-based Rating Prediction model (AspeRa)\nthat estimates user rating based on review texts for the items and at the same\ntime discovers coherent aspects of reviews that can be used to explain\npredictions or profile users. The AspeRa model uses max-margin losses for joint\nitem and user embedding learning and a dual-headed architecture; it\nsignificantly outperforms recently proposed state-of-the-art models such as\nDeepCoNN, HFT, NARRE, and TransRev on two real world data sets of user reviews.\nWith qualitative examination of the aspects and quantitative evaluation of\nrating prediction models based on these aspects, we show how aspect embeddings\ncan be used in a recommender system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07880v1",
            "date": "2019-01-23 13:53:24+00:00",
            "title": "Phonetic-enriched Text Representation for Chinese Sentiment Analysis with Reinforcement Learning",
            "authors": [
                "Haiyun Peng",
                "Yukun Ma",
                "Soujanya Poria",
                "Yang Li",
                "Erik Cambria"
            ],
            "abstract": "The Chinese pronunciation system offers two characteristics that distinguish\nit from other languages: deep phonemic orthography and intonation variations.\nWe are the first to argue that these two important properties can play a major\nrole in Chinese sentiment analysis. Particularly, we propose two effective\nfeatures to encode phonetic information. Next, we develop a Disambiguate\nIntonation for Sentiment Analysis (DISA) network using a reinforcement network.\nIt functions as disambiguating intonations for each Chinese character (pinyin).\nThus, a precise phonetic representation of Chinese is learned. Furthermore, we\nalso fuse phonetic features with textual and visual features in order to mimic\nthe way humans read and understand Chinese text. Experimental results on five\ndifferent Chinese sentiment analysis datasets show that the inclusion of\nphonetic features significantly and consistently improves the performance of\ntextual and visual representations and outshines the state-of-the-art Chinese\ncharacter level representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08158v1",
            "date": "2019-01-23 22:37:11+00:00",
            "title": "A Tool for Spatio-Temporal Analysis of Social Anxiety with Twitter Data",
            "authors": [
                "Joohong Lee",
                "Dongyoung Son",
                "Yong Suk Choi"
            ],
            "abstract": "In this paper, we present a tool for analyzing spatio-temporal distribution\nof social anxiety. Twitter, one of the most popular social network services,\nhas been chosen as data source for analysis of social anxiety. Tweets (posted\non the Twitter) contain various emotions and thus these individual emotions\nreflect social atmosphere and public opinion, which are often dependent on\nspatial and temporal factors. The reason why we choose anxiety among various\nemotions is that anxiety is very important emotion that is useful for observing\nand understanding social events of communities. We develop a machine learning\nbased tool to analyze the changes of social atmosphere spatially and\ntemporally. Our tool classifies whether each Tweet contains anxious content or\nnot, and also estimates degree of Tweet anxiety. Furthermore, it also\nvisualizes spatio-temporal distribution of anxiety as a form of web\napplication, which is incorporated with physical map, word cloud, search engine\nand chart viewer. Our tool is applied to a big tweet data in South Korea to\nillustrate its usefulness for exploring social atmosphere and public opinion\nspatio-temporally.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08259v1",
            "date": "2019-01-24 07:25:16+00:00",
            "title": "FANDA: A Novel Approach to Perform Follow-up Query Analysis",
            "authors": [
                "Qian Liu",
                "Bei Chen",
                "Jian-Guang Lou",
                "Ge Jin",
                "Dongmei Zhang"
            ],
            "abstract": "Recent work on Natural Language Interfaces to Databases (NLIDB) has attracted\nconsiderable attention. NLIDB allow users to search databases using natural\nlanguage instead of SQL-like query languages. While saving the users from\nhaving to learn query languages, multi-turn interaction with NLIDB usually\ninvolves multiple queries where contextual information is vital to understand\nthe users' query intents. In this paper, we address a typical contextual\nunderstanding problem, termed as follow-up query analysis. In spite of its\nubiquity, follow-up query analysis has not been well studied due to two primary\nobstacles: the multifarious nature of follow-up query scenarios and the lack of\nhigh-quality datasets. Our work summarizes typical follow-up query scenarios\nand provides a new FollowUp dataset with $1000$ query triples on 120 tables.\nMoreover, we propose a novel approach FANDA, which takes into account the\nstructures of queries and employs a ranking model with weakly supervised\nmax-margin learning. The experimental results on FollowUp demonstrate the\nsuperiority of FANDA over multiple baselines across multiple metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08351v1",
            "date": "2019-01-24 11:14:49+00:00",
            "title": "Extracting PICO elements from RCT abstracts using 1-2gram analysis and multitask classification",
            "authors": [
                "Xia Yuan",
                "Liao xiaoli",
                "Li Shilei",
                "Shi Qinwen",
                "Wu Jinfa",
                "Li Ke"
            ],
            "abstract": "The core of evidence-based medicine is to read and analyze numerous papers in\nthe medical literature on a specific clinical problem and summarize the\nauthoritative answers to that problem. Currently, to formulate a clear and\nfocused clinical problem, the popular PICO framework is usually adopted, in\nwhich each clinical problem is considered to consist of four parts:\npatient/problem (P), intervention (I), comparison (C) and outcome (O). In this\nstudy, we compared several classification models that are commonly used in\ntraditional machine learning. Next, we developed a multitask classification\nmodel based on a soft-margin SVM with a specialized feature engineering method\nthat combines 1-2gram analysis with TF-IDF analysis. Finally, we trained and\ntested several generic models on an open-source data set from BioNLP 2018. The\nresults show that the proposed multitask SVM classification model based on\n1-2gram TF-IDF features exhibits the best performance among the tested models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08407v1",
            "date": "2019-01-24 13:53:17+00:00",
            "title": "A model for a Lindenmayer reconstruction algorithm",
            "authors": [
                "Diego Gabriel Krivochen",
                "Beth Phillips"
            ],
            "abstract": "Given an input string s and a specific Lindenmayer system (the so-called\nFibonacci grammar), we define an automaton which is capable of (i) determining\nwhether s belongs to the set of strings that the Fibonacci grammar can generate\n(in other words, if s corresponds to a generation of the grammar) and, if so,\n(ii) reconstructing the previous generation.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08456v1",
            "date": "2019-01-24 15:31:11+00:00",
            "title": "Semantic Classification of Tabular Datasets via Character-Level Convolutional Neural Networks",
            "authors": [
                "Paul Azunre",
                "Craig Corcoran",
                "Numa Dhamani",
                "Jeffrey Gleason",
                "Garrett Honke",
                "David Sullivan",
                "Rebecca Ruppel",
                "Sandeep Verma",
                "Jonathon Morgan"
            ],
            "abstract": "A character-level convolutional neural network (CNN) motivated by\napplications in \"automated machine learning\" (AutoML) is proposed to\nsemantically classify columns in tabular data. Simulated data containing a set\nof base classes is first used to learn an initial set of weights. Hand-labeled\ndata from the CKAN repository is then used in a transfer-learning paradigm to\nadapt the initial weights to a more sophisticated representation of the problem\n(e.g., including more classes). In doing so, realistic data imperfections are\nlearned and the set of classes handled can be expanded from the base set with\nreduced labeled data and computing power requirements. Results show the\neffectiveness and flexibility of this approach in three diverse domains:\nsemantic classification of tabular data, age prediction from social media\nposts, and email spam classification. In addition to providing further evidence\nof the effectiveness of transfer learning in natural language processing (NLP),\nour experiments suggest that analyzing the semantic structure of language at\nthe character level without additional metadata---i.e., network structure,\nheaders, etc.---can produce competitive accuracy for type classification, spam\nclassification, and social media age prediction. We present our open-source\ntoolkit SIMON, an acronym for Semantic Inference for the Modeling of\nONtologies, which implements this approach in a user-friendly and\nscalable/parallelizable fashion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09037v1",
            "date": "2019-01-25 15:04:02+00:00",
            "title": "Comparing of Term Clustering Frameworks for Modular Ontology Learning",
            "authors": [
                "Ziwei Xu",
                "Mounira Harzallah",
                "Fabrice Guillet"
            ],
            "abstract": "This paper aims to use term clustering to build a modular ontology according\nto core ontology from domain-specific text. The acquisition of semantic\nknowledge focuses on noun phrase appearing with the same syntactic roles in\nrelation to a verb or its preposition combination in a sentence. The\nconstruction of this co-occurrence matrix from context helps to build feature\nspace of noun phrases, which is then transformed to several encoding\nrepresentations including feature selection and dimensionality reduction. In\naddition, the content has also been presented with the construction of word\nvectors. These representations are clustered respectively with K-Means and\nAffinity Propagation (AP) methods, which differentiate into the term clustering\nframeworks. Due to the randomness of K-Means, iteration efforts are adopted to\nfind the optimal parameter. The frameworks are evaluated extensively where AP\nshows dominant effectiveness for co-occurred terms and NMF encoding technique\nis salient by its promising facilities in feature compression.",
            "journal ref": "10th International Joint Conference on Knowledge Discovery,\n  Knowledge Engineering and Knowledge Management, Sep 2018, Seville, Spain.\n  SCITEPRESS - Science and Technology Publications; SCITEPRESS - Science and\n  Technology Publications, pp.128-135, 2018, Proceedings of the 10th\n  International Joint Conference on Knowledge Discovery, Knowledge Engineering\n  and Knowledge Management - Volume 2: KEOD",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.09066v1",
            "date": "2019-01-20 18:46:21+00:00",
            "title": "Visualizing Semantic Structures of Sequential Data by Learning Temporal Dependencies",
            "authors": [
                "Kyoung-Woon On",
                "Eun-Sol Kim",
                "Yu-Jung Heo",
                "Byoung-Tak Zhang"
            ],
            "abstract": "While conventional methods for sequential learning focus on interaction\nbetween consecutive inputs, we suggest a new method which captures composite\nsemantic flows with variable-length dependencies. In addition, the semantic\nstructures within given sequential data can be interpreted by visualizing\ntemporal dependencies learned from the method. The proposed method, called\nTemporal Dependency Network (TDN), represents a video as a temporal graph whose\nnode represents a frame of the video and whose edge represents the temporal\ndependency between two frames of a variable distance. The temporal dependency\nstructure of semantic is discovered by learning parameterized kernels of graph\nconvolutional methods. We evaluate the proposed method on the large-scale video\ndataset, Youtube-8M. By visualizing the temporal dependency structures as\nexperimental results, we show that the suggested method can find the temporal\ndependency structures of video semantic.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.09492v1",
            "date": "2019-01-28 03:00:06+00:00",
            "title": "Neural Related Work Summarization with a Joint Context-driven Attention Mechanism",
            "authors": [
                "Yongzhen Wang",
                "Xiaozhong Liu",
                "Zheng Gao"
            ],
            "abstract": "Conventional solutions to automatic related work summarization rely heavily\non human-engineered features. In this paper, we develop a neural data-driven\nsummarizer by leveraging the seq2seq paradigm, in which a joint context-driven\nattention mechanism is proposed to measure the contextual relevance within full\ntexts and a heterogeneous bibliography graph simultaneously. Our motivation is\nto maintain the topic coherency between a related work section and its target\ndocument, where both the textual and graphic contexts play a big role in\ncharacterizing the relationship among scientific publications accurately.\nExperimental results on a large dataset show that our approach achieves a\nconsiderable improvement over a typical seq2seq summarizer and five classical\nsummarization baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10133v1",
            "date": "2019-01-29 06:53:21+00:00",
            "title": "Structuring an unordered text document",
            "authors": [
                "Shashank Yadav",
                "Tejas Shimpi",
                "C. Ravindranath Chowdary",
                "Prashant Sharma",
                "Deepansh Agrawal",
                "Shivang Agarwal"
            ],
            "abstract": "Segmenting an unordered text document into different sections is a very\nuseful task in many text processing applications like multiple document\nsummarization, question answering, etc. This paper proposes structuring of an\nunordered text document based on the keywords in the document. We test our\napproach on Wikipedia documents using both statistical and predictive methods\nsuch as the TextRank algorithm and Google's USE (Universal Sentence Encoder).\nFrom our experimental results, we show that the proposed model can effectively\nstructure an unordered document into sections.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.10219v1",
            "date": "2019-01-29 11:12:58+00:00",
            "title": "Revised JNLPBA Corpus: A Revised Version of Biomedical NER Corpus for Relation Extraction Task",
            "authors": [
                "Ming-Siang Huang",
                "Po-Ting Lai",
                "Richard Tzong-Han Tsai",
                "Wen-Lian Hsu"
            ],
            "abstract": "The advancement of biomedical named entity recognition (BNER) and biomedical\nrelation extraction (BRE) researches promotes the development of text mining in\nbiological domains. As a cornerstone of BRE, robust BNER system is required to\nidentify the mentioned NEs in plain texts for further relation extraction\nstage. However, the current BNER corpora, which play important roles in these\ntasks, paid less attention to achieve the criteria for BRE task. In this study,\nwe present Revised JNLPBA corpus, the revision of JNLPBA corpus, to broaden the\napplicability of a NER corpus from BNER to BRE task. We preserve the original\nentity types including protein, DNA, RNA, cell line and cell type while all the\nabstracts in JNLPBA corpus are manually curated by domain experts again basis\non the new annotation guideline focusing on the specific NEs instead of general\nterms. Simultaneously, several imperfection issues in JNLPBA are pointed out\nand made up in the new corpus. To compare the adaptability of different NER\nsystems in Revised JNLPBA and JNLPBA corpora, the F1-measure was measured in\nthree open sources NER systems including BANNER, Gimli and NERSuite. In the\nsame circumstance, all the systems perform average 10% better in Revised JNLPBA\nthan in JNLPBA. Moreover, the cross-validation test is carried out which we\ntrain the NER systems on JNLPBA/Revised JNLPBA corpora and access the\nperformance in both protein-protein interaction extraction (PPIE) and\nbiomedical event extraction (BEE) corpora to confirm that the newly refined\nRevised JNLPBA is a competent NER corpus in biomedical relation application.\nThe revised JNLPBA corpus is freely available at\niasl-btm.iis.sinica.edu.tw/BNER/Content/Revised_JNLPBA.zip.",
            "journal ref": "Briefings in Bioinformatics, 2020, bbaa054",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.10408v2",
            "date": "2019-01-29 17:23:32+00:00",
            "title": "Guidelines for creating man-machine multimodal interfaces",
            "authors": [
                "Jo\u00e3o Ranhel",
                "Cacilda Vilela"
            ],
            "abstract": "Understanding details of human multimodal interaction can elucidate many\naspects of the type of information processing machines must perform to interact\nwith humans. This article gives an overview of recent findings from Linguistics\nregarding the organization of conversation in turns, adjacent pairs,\n(dis)preferred responses, (self)repairs, etc. Besides, we describe how multiple\nmodalities of signs interfere with each other modifying meanings. Then, we\npropose an abstract algorithm that describes how a machine can implement a\ndouble-feedback system that can reproduces a human-like face-to-face\ninteraction by processing various signs, such as verbal, prosodic, facial\nexpressions, gestures, etc. Multimodal face-to-face interactions enrich the\nexchange of information between agents, mainly because these agents are active\nall the time by emitting and interpreting signs simultaneously. This article is\nnot about an untested new computational model. Instead, it translates findings\nfrom Linguistics as guidelines for designs of multimodal man-machine\ninterfaces. An algorithm is presented. Brought from Linguistics, it is a\ndescription pointing out how human face-to-face interactions work. The\nlinguistic findings reported here are the first steps towards the integration\nof multimodal communication. Some developers involved on interface designs\ncarry on working on isolated models for interpreting text, grammar, gestures\nand facial expressions, neglecting the interwoven between these signs. In\ncontrast, for linguists working on the state-of-the-art multimodal integration,\nthe interpretation of separated modalities leads to an incomplete\ninterpretation, if not to a miscomprehension of information. The algorithm\nproposed herein intends to guide man-machine interface designers who want to\nintegrate multimodal components on face-to-face interactions as close as\npossible to those performed between humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10496v1",
            "date": "2019-01-29 19:00:21+00:00",
            "title": "Impact of Training Dataset Size on Neural Answer Selection Models",
            "authors": [
                "Trond Linjordet",
                "Krisztian Balog"
            ],
            "abstract": "It is held as a truism that deep neural networks require large datasets to\ntrain effective models. However, large datasets, especially with high-quality\nlabels, can be expensive to obtain. This study sets out to investigate (i) how\nlarge a dataset must be to train well-performing models, and (ii) what impact\ncan be shown from fractional changes to the dataset size. A practical method to\ninvestigate these questions is to train a collection of deep neural answer\nselection models using fractional subsets of varying sizes of an initial\ndataset. We observe that dataset size has a conspicuous lack of effect on the\ntraining of some of these models, bringing the underlying algorithms into\nquestion.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.10619v1",
            "date": "2019-01-30 00:01:14+00:00",
            "title": "Twitter Job/Employment Corpus: A Dataset of Job-Related Discourse Built with Humans in the Loop",
            "authors": [
                "Tong Liu",
                "Christopher M. Homan"
            ],
            "abstract": "We present the Twitter Job/Employment Corpus, a collection of tweets\nannotated by a humans-in-the-loop supervised learning framework that integrates\ncrowdsourcing contributions and expertise on the local community and employment\nenvironment. Previous computational studies of job-related phenomena have used\ncorpora collected from workplace social media that are hosted internally by the\nemployers, and so lacks independence from latent job-related coercion and the\nbroader context that an open domain, general-purpose medium such as Twitter\nprovides. Our new corpus promises to be a benchmark for the extraction of\njob-related topics and advanced analysis and modeling, and can potentially\nbenefit a wide range of research communities in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10787v2",
            "date": "2019-01-30 12:43:50+00:00",
            "title": "Tensorized Embedding Layers for Efficient Model Compression",
            "authors": [
                "Oleksii Hrinchuk",
                "Valentin Khrulkov",
                "Leyla Mirvakhabova",
                "Elena Orlova",
                "Ivan Oseledets"
            ],
            "abstract": "The embedding layers transforming input words into real vectors are the key\ncomponents of deep neural networks used in natural language processing.\nHowever, when the vocabulary is large, the corresponding weight matrices can be\nenormous, which precludes their deployment in a limited resource setting. We\nintroduce a novel way of parametrizing embedding layers based on the Tensor\nTrain (TT) decomposition, which allows compressing the model significantly at\nthe cost of a negligible drop or even a slight gain in performance. We evaluate\nour method on a wide range of benchmarks in natural language processing and\nanalyze the trade-off between performance and compression ratios for a wide\nrange of architectures, from MLPs to LSTMs and Transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11167v1",
            "date": "2019-01-31 01:46:35+00:00",
            "title": "A Generalized Language Model in Tensor Space",
            "authors": [
                "Lipeng Zhang",
                "Peng Zhang",
                "Xindian Ma",
                "Shuqin Gu",
                "Zhan Su",
                "Dawei Song"
            ],
            "abstract": "In the literature, tensors have been effectively used for capturing the\ncontext information in language models. However, the existing methods usually\nadopt relatively-low order tensors, which have limited expressive power in\nmodeling language. Developing a higher-order tensor representation is\nchallenging, in terms of deriving an effective solution and showing its\ngenerality. In this paper, we propose a language model named Tensor Space\nLanguage Model (TSLM), by utilizing tensor networks and tensor decomposition.\nIn TSLM, we build a high-dimensional semantic space constructed by the tensor\nproduct of word vectors. Theoretically, we prove that such tensor\nrepresentation is a generalization of the n-gram language model. We further\nshow that this high-order tensor representation can be decomposed to a\nrecursive calculation of conditional probability for language modeling. The\nexperimental results on Penn Tree Bank (PTB) dataset and WikiText benchmark\ndemonstrate the effectiveness of TSLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11344v1",
            "date": "2019-01-31 13:26:28+00:00",
            "title": "Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory",
            "authors": [
                "Ya Li",
                "Xinyu Liu",
                "Dan Liu",
                "Xueqiang Zhang",
                "Junhua Liu"
            ],
            "abstract": "Recent years has witnessed dramatic progress of neural machine translation\n(NMT), however, the method of manually guiding the translation procedure\nremains to be better explored. Previous works proposed to handle such problem\nthrough lexcially-constrained beam search in the decoding phase. Unfortunately,\nthese lexically-constrained beam search methods suffer two fatal disadvantages:\nhigh computational complexity and hard beam search which generates unexpected\ntranslations. In this paper, we propose to learn the ability of\nlexically-constrained translation with external memory, which can overcome the\nabove mentioned disadvantages. For the training process, automatically\nextracted phrase pairs are extracted from alignment and sentence parsing, then\nfurther be encoded into an external memory. This memory is then used to provide\nlexically-constrained information for training through a memory-attention\nmachanism. Various experiments are conducted on WMT Chinese to English and\nEnglish to German tasks. All the results can demonstrate the effectiveness of\nour method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11462v1",
            "date": "2019-01-31 16:40:26+00:00",
            "title": "Exploring the context of recurrent neural network based conversational agents",
            "authors": [
                "Raffaele Piccini",
                "Gerasimos Spanakis"
            ],
            "abstract": "Conversational agents have begun to rise both in the academic (in terms of\nresearch) and commercial (in terms of applications) world. This paper\ninvestigates the task of building a non-goal driven conversational agent, using\nneural network generative models and analyzes how the conversation context is\nhandled. It compares a simpler Encoder-Decoder with a Hierarchical Recurrent\nEncoder-Decoder architecture, which includes an additional module to model the\ncontext of the conversation using previous utterances information. We found\nthat the hierarchical model was able to extract relevant context information\nand include them in the generation of the output. However, it performed worse\n(35-40%) than the simple Encoder-Decoder model regarding both grammatically\ncorrect output and meaningful response. Despite these results, experiments\ndemonstrate how conversations about similar topics appear close to each other\nin the context space due to the increased frequency of specific topic-related\nwords, thus leaving promising directions for future research and how the\ncontext of a conversation can be exploited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11467v1",
            "date": "2019-01-31 16:51:49+00:00",
            "title": "Towards Controlled Transformation of Sentiment in Sentences",
            "authors": [
                "Wouter Leeftink",
                "Gerasimos Spanakis"
            ],
            "abstract": "An obstacle to the development of many natural language processing products\nis the vast amount of training examples necessary to get satisfactory results.\nThe generation of these examples is often a tedious and time-consuming task.\nThis paper this paper proposes a method to transform the sentiment of sentences\nin order to limit the work necessary to generate more training data. This means\nthat one sentence can be transformed to an opposite sentiment sentence and\nshould reduce by half the work required in the generation of text. The proposed\npipeline consists of a sentiment classifier with an attention mechanism to\nhighlight the short phrases that determine the sentiment of a sentence. Then,\nthese phrases are changed to phrases of the opposite sentiment using a baseline\nmodel and an autoencoder approach. Experiments are run on both the separate\nparts of the pipeline as well as on the end-to-end model. The sentiment\nclassifier is tested on its accuracy and is found to perform adequately. The\nautoencoder is tested on how well it is able to change the sentiment of an\nencoded phrase and it was found that such a task is possible. We use human\nevaluation to judge the performance of the full (end-to-end) pipeline and that\nreveals that a model using word vectors outperforms the encoder model.\nNumerical evaluation shows that a success rate of 54.7% is achieved on the\nsentiment change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00122v1",
            "date": "2019-03-01 01:43:11+00:00",
            "title": "Improving Grounded Natural Language Understanding through Human-Robot Dialog",
            "authors": [
                "Jesse Thomason",
                "Aishwarya Padmakumar",
                "Jivko Sinapov",
                "Nick Walker",
                "Yuqian Jiang",
                "Harel Yedidsion",
                "Justin Hart",
                "Peter Stone",
                "Raymond J. Mooney"
            ],
            "abstract": "Natural language understanding for robotics can require substantial domain-\nand platform-specific engineering. For example, for mobile robots to\npick-and-place objects in an environment to satisfy human commands, we can\nspecify the language humans use to issue such commands, and connect concept\nwords like red can to physical object properties. One way to alleviate this\nengineering for a new domain is to enable robots in human environments to adapt\ndynamically---continually learning new language constructions and perceptual\nconcepts. In this work, we present an end-to-end pipeline for translating\nnatural language commands to discrete robot actions, and use clarification\ndialogs to jointly improve language parsing and concept grounding. We train and\nevaluate this agent in a virtual setting on Amazon Mechanical Turk, and we\ntransfer the learned agent to a physical robot platform to demonstrate it in\nthe real world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00232v2",
            "date": "2019-03-01 10:10:36+00:00",
            "title": "A Framework for Detecting Event related Sentiments of a Community",
            "authors": [
                "Muhammad Aslam Jarwar"
            ],
            "abstract": "Social media has revolutionized human communication and styles of\ninteraction. Due to its easiness and effective medium, people share and\nexchange information, carry out discussion on various events, and express their\nopinions. For effective policy making and understanding the response of a\ncommunity on different events, we need to monitor and analyze the social media.\nIn social media, there are some users who are more influential, for example, a\nfamous politician may have more influence than a common person. These\ninfluential users belong to specific communities. The main object of this\nresearch is to know the sentiments of a specific community on various events.\nFor detecting the event based sentiments of a community we propose a generic\nframework. Our framework identifies the users of a specific community on\ntwitter. After identifying the users of a community, we fetch their tweets and\nidentify tweets belonging to specific events. The event based tweets are\npre-processed. Pre-processed tweets are then analyzed for detecting sentiments\nof a community for specific events. Qualitative and quantitative evaluation\nconfirms the effectiveness and usefulness of our proposed framework.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1903.00384v1",
            "date": "2019-03-01 16:04:27+00:00",
            "title": "Data-driven Approach for Quality Evaluation on Knowledge Sharing Platform",
            "authors": [
                "Lu Xu",
                "Jinhai Xiang",
                "Yating Wang",
                "Fuchuan Ni"
            ],
            "abstract": "In recent years, voice knowledge sharing and question answering (Q&A)\nplatforms have attracted much attention, which greatly facilitate the knowledge\nacquisition for people. However, little research has evaluated on the quality\nevaluation on voice knowledge sharing. This paper presents a data-driven\napproach to automatically evaluate the quality of a specific Q&A platform\n(Zhihu Live). Extensive experiments demonstrate the effectiveness of the\nproposed method. Furthermore, we introduce a dataset of Zhihu Live as an open\nresource for researchers in related areas. This dataset will facilitate the\ndevelopment of new methods on knowledge sharing services quality evaluation.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1903.00415v1",
            "date": "2019-03-01 17:14:33+00:00",
            "title": "Using natural language processing techniques to extract information on the properties and functionalities of energetic materials from large text corpora",
            "authors": [
                "Daniel C. Elton",
                "Dhruv Turakhia",
                "Nischal Reddy",
                "Zois Boukouvalas",
                "Mark D. Fuge",
                "Ruth M. Doherty",
                "Peter W. Chung"
            ],
            "abstract": "The number of scientific journal articles and reports being published about\nenergetic materials every year is growing exponentially, and therefore\nextracting relevant information and actionable insights from the latest\nresearch is becoming a considerable challenge. In this work we explore how\ntechniques from natural language processing and machine learning can be used to\nautomatically extract chemical insights from large collections of documents. We\nfirst describe how to download and process documents from a variety of sources\n- journal articles, conference proceedings (including NTREM), the US Patent &\nTrademark Office, and the Defense Technical Information Center archive on\narchive.org. We present a custom NLP pipeline which uses open source NLP tools\nto identify the names of chemical compounds and relates them to function words\n(\"underwater\", \"rocket\", \"pyrotechnic\") and property words (\"elastomer\",\n\"non-toxic\"). After explaining how word embeddings work we compare the utility\nof two popular word embeddings - word2vec and GloVe. Chemical-chemical and\nchemical-application relationships are obtained by doing computations with word\nvectors. We show that word embeddings capture latent information about\nenergetic materials, so that related materials appear close together in the\nword embedding space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00724v1",
            "date": "2019-03-02 15:32:39+00:00",
            "title": "Predicting and interpreting embeddings for out of vocabulary words in downstream tasks",
            "authors": [
                "Nicolas Garneau",
                "Jean-Samuel Leboeuf",
                "Luc Lamontagne"
            ],
            "abstract": "We propose a novel way to handle out of vocabulary (OOV) words in downstream\nnatural language processing (NLP) tasks. We implement a network that predicts\nuseful embeddings for OOV words based on their morphology and on the context in\nwhich they appear. Our model also incorporates an attention mechanism\nindicating the focus allocated to the left context words, the right context\nwords or the word's characters, hence making the prediction more interpretable.\nThe model is a ``drop-in'' module that is jointly trained with the downstream\ntask's neural network, thus producing embeddings specialized for the task at\nhand. When the task is mostly syntactical, we observe that our model aims most\nof its attention on surface form characters. On the other hand, for tasks more\nsemantical, the network allocates more attention to the surrounding words. In\nall our tests, the module helps the network to achieve better performances in\ncomparison to the use of simple random embeddings.",
            "journal ref": "Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and\n  Interpreting Neural Networks for NLP",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00830v2",
            "date": "2019-03-03 04:39:33+00:00",
            "title": "Predicting Algorithm Classes for Programming Word Problems",
            "authors": [
                "Vinayak Athavale",
                "Aayush Naik",
                "Rajas Vanjape",
                "Manish Shrivastava"
            ],
            "abstract": "We introduce the task of algorithm class prediction for programming word\nproblems. A programming word problem is a problem written in natural language,\nwhich can be solved using an algorithm or a program. We define classes of\nvarious programming word problems which correspond to the class of algorithms\nrequired to solve the problem. We present four new datasets for this task, two\nmulticlass datasets with 550 and 1159 problems each and two multilabel datasets\nhaving 3737 and 3960 problems each. We pose the problem as a text\nclassification problem and train neural network and non-neural network-based\nmodels on this task. Our best performing classifier gets an accuracy of 62.7\npercent for the multiclass case on the five class classification dataset,\nCodeforces Multiclass-5 (CFMC5). We also do some human-level analysis and\ncompare human performance with that of our text classification models. Our best\nclassifier has an accuracy only 9 percent lower than that of a human on this\ntask. To the best of our knowledge, these are the first reported results on\nsuch a task. We make our code and datasets publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00839v2",
            "date": "2019-03-03 05:55:15+00:00",
            "title": "Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing",
            "authors": [
                "Xihui Liu",
                "Zihao Wang",
                "Jing Shao",
                "Xiaogang Wang",
                "Hongsheng Li"
            ],
            "abstract": "Referring expression grounding aims at locating certain objects or persons in\nan image with a referring expression, where the key challenge is to comprehend\nand align various types of information from visual and textual domain, such as\nvisual attributes, location and interactions with surrounding regions. Although\nthe attention mechanism has been successfully applied for cross-modal\nalignments, previous attention models focus on only the most dominant features\nof both modalities, and neglect the fact that there could be multiple\ncomprehensive textual-visual correspondences between images and referring\nexpressions. To tackle this issue, we design a novel cross-modal\nattention-guided erasing approach, where we discard the most dominant\ninformation from either textual or visual domains to generate difficult\ntraining samples online, and to drive the model to discover complementary\ntextual-visual correspondences. Extensive experiments demonstrate the\neffectiveness of our proposed method, which achieves state-of-the-art\nperformance on three referring expression grounding datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1903.01275v1",
            "date": "2019-03-04 14:36:21+00:00",
            "title": "Using Word Embeddings for Visual Data Exploration with Ontodia and Wikidata",
            "authors": [
                "Gerhard Wohlgenannt",
                "Nikolay Klimov",
                "Dmitry Mouromtsev",
                "Daniil Razdyakonov",
                "Dmitry Pavlov",
                "Yury Emelyanov"
            ],
            "abstract": "One of the big challenges in Linked Data consumption is to create visual and\nnatural language interfaces to the data usable for non-technical users. Ontodia\nprovides support for diagrammatic data exploration, showcased in this\npublication in combination with the Wikidata dataset. We present improvements\nto the natural language interface regarding exploring and querying Linked Data\nentities. The method uses models of distributional semantics to find and rank\nentity properties related to user input in Ontodia. Various word embedding\ntypes and model settings are evaluated, and the results show that user\nexperience in visual data exploration benefits from the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.01306v1",
            "date": "2019-03-04 15:32:39+00:00",
            "title": "Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks",
            "authors": [
                "Ningyu Zhang",
                "Shumin Deng",
                "Zhanlin Sun",
                "Guanying Wang",
                "Xi Chen",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "We propose a distance supervised relation extraction approach for\nlong-tailed, imbalanced data which is prevalent in real-world settings. Here,\nthe challenge is to learn accurate \"few-shot\" models for classes existing at\nthe tail of the class distribution, for which little data is available.\nInspired by the rich semantic correlations between classes at the long tail and\nthose at the head, we take advantage of the knowledge from data-rich classes at\nthe head of the distribution to boost the performance of the data-poor classes\nat the tail. First, we propose to leverage implicit relational knowledge among\nclass labels from knowledge graph embeddings and learn explicit relational\nknowledge using graph convolution networks. Second, we integrate that\nrelational knowledge into relation extraction model by coarse-to-fine\nknowledge-aware attention mechanism. We demonstrate our results for a\nlarge-scale benchmark dataset which show that our approach significantly\noutperforms other baselines, especially for long-tail relations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.01728v4",
            "date": "2019-03-05 08:52:33+00:00",
            "title": "Mining Dual Emotion for Fake News Detection",
            "authors": [
                "Xueyao Zhang",
                "Juan Cao",
                "Xirong Li",
                "Qiang Sheng",
                "Lei Zhong",
                "Kai Shu"
            ],
            "abstract": "Emotion plays an important role in detecting fake news online. When\nleveraging emotional signals, the existing methods focus on exploiting the\nemotions of news contents that conveyed by the publishers (i.e., publisher\nemotion). However, fake news often evokes high-arousal or activating emotions\nof people, so the emotions of news comments aroused in the crowd (i.e., social\nemotion) should not be ignored. Furthermore, it remains to be explored whether\nthere exists a relationship between publisher emotion and social emotion (i.e.,\ndual emotion), and how the dual emotion appears in fake news. In this paper, we\nverify that dual emotion is distinctive between fake and real news and propose\nDual Emotion Features to represent dual emotion and the relationship between\nthem for fake news detection. Further, we exhibit that our proposed features\ncan be easily plugged into existing fake news detectors as an enhancement.\nExtensive experiments on three real-world datasets (one in English and the\nothers in Chinese) show that our proposed feature set: 1) outperforms the\nstate-of-the-art task-related emotional features; 2) can be well compatible\nwith existing fake news detectors and effectively improve the performance of\ndetecting fake news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02134v5",
            "date": "2019-03-06 01:37:51+00:00",
            "title": "Negative Training for Neural Dialogue Response Generation",
            "authors": [
                "Tianxing He",
                "James Glass"
            ],
            "abstract": "Although deep learning models have brought tremendous advancements to the\nfield of open-domain dialogue response generation, recent research results have\nrevealed that the trained models have undesirable generation behaviors, such as\nmalicious responses and generic (boring) responses. In this work, we propose a\nframework named \"Negative Training\" to minimize such behaviors. Given a trained\nmodel, the framework will first find generated samples that exhibit the\nundesirable behavior, and then use them to feed negative training signals for\nfine-tuning the model. Our experiments show that negative training can\nsignificantly reduce the hit rate of malicious responses, or discourage\nfrequent responses and improve response diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02163v2",
            "date": "2019-03-06 03:53:19+00:00",
            "title": "SNU_IDS at SemEval-2019 Task 3: Addressing Training-Test Class Distribution Mismatch in Conversational Classification",
            "authors": [
                "Sanghwan Bae",
                "Jihun Choi",
                "Sang-goo Lee"
            ],
            "abstract": "We present several techniques to tackle the mismatch in class distributions\nbetween training and test data in the Contextual Emotion Detection task of\nSemEval 2019, by extending the existing methods for class imbalance problem.\nReducing the distance between the distribution of prediction and ground truth,\nthey consistently show positive effects on the performance. Also we propose a\nnovel neural architecture which utilizes representation of overall context as\nwell as of each utterance. The combination of the methods and the models\nachieved micro F1 score of about 0.766 on the final evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02642v1",
            "date": "2019-03-06 22:48:21+00:00",
            "title": "A Character-Level Approach to the Text Normalization Problem Based on a New Causal Encoder",
            "authors": [
                "Adri\u00e1n Javaloy Born\u00e1s",
                "Gin\u00e9s Garc\u00eda Mateos"
            ],
            "abstract": "Text normalization is a ubiquitous process that appears as the first step of\nmany Natural Language Processing problems. However, previous Deep Learning\napproaches have suffered from so-called silly errors, which are undetectable on\nunsupervised frameworks, making those models unsuitable for deployment. In this\nwork, we make use of an attention-based encoder-decoder architecture that\novercomes these undetectable errors by using a fine-grained character-level\napproach rather than a word-level one. Furthermore, our new general-purpose\nencoder based on causal convolutions, called Causal Feature Extractor (CFE), is\nintroduced and compared to other common encoders. The experimental results show\nthe feasibility of this encoder, which leverages the attention mechanisms the\nmost and obtains better results in terms of accuracy, number of parameters and\nconvergence time. While our method results in a slightly worse initial accuracy\n(92.74%), errors can be automatically detected and, thus, more readily solved,\nobtaining a more robust model for deployment. Furthermore, there is still\nplenty of room for future improvements that will push even further these\nadvantages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02861v1",
            "date": "2019-03-07 12:12:17+00:00",
            "title": "Small-world networks for summarization of biomedical articles",
            "authors": [
                "Milad Moradi"
            ],
            "abstract": "In recent years, many methods have been developed to identify important\nportions of text documents. Summarization tools can utilize these methods to\nextract summaries from large volumes of textual information. However, to\nidentify concepts representing central ideas within a text document and to\nextract the most informative sentences that best convey those concepts still\nremain two crucial tasks in summarization methods. In this paper, we introduce\na graph-based method to address these two challenges in the context of\nbiomedical text summarization. We show that how a summarizer can discover\nmeaningful concepts within a biomedical text document using the Helmholtz\nprinciple. The summarizer considers the meaningful concepts as the main topics\nand constructs a graph based on the topics that the sentences share. The\nsummarizer can produce an informative summary by extracting those sentences\nhaving higher values of the degree. We assess the performance of our method for\nsummarization of biomedical articles using the Recall-Oriented Understudy for\nGisting Evaluation (ROUGE) toolkit. The results show that the degree can be a\nuseful centrality measure to identify important sentences in this type of\ngraph-based modelling. Our method can improve the performance of biomedical\ntext summarization compared to some state-of-the-art and publicly available\nsummarizers. Combining a concept-based modelling strategy and a graph-based\napproach to sentence extraction, our summarizer can produce summaries with the\nhighest scores of informativeness among the comparison methods. This research\nwork can be regarded as a start point to the study of small-world networks in\nsummarization of biomedical texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03094v1",
            "date": "2019-03-07 18:45:52+00:00",
            "title": "Learning to Speak and Act in a Fantasy Text Adventure Game",
            "authors": [
                "Jack Urbanek",
                "Angela Fan",
                "Siddharth Karamcheti",
                "Saachi Jain",
                "Samuel Humeau",
                "Emily Dinan",
                "Tim Rockt\u00e4schel",
                "Douwe Kiela",
                "Arthur Szlam",
                "Jason Weston"
            ],
            "abstract": "We introduce a large scale crowdsourced text adventure game as a research\nplatform for studying grounded dialogue. In it, agents can perceive, emote, and\nact whilst conducting dialogue with other agents. Models and humans can both\nact as characters within the game. We describe the results of training\nstate-of-the-art generative and retrieval models in this setting. We show that\nin addition to using past dialogue, these models are able to effectively use\nthe state of the underlying world to condition their predictions. In\nparticular, we show that grounding on the details of the local environment,\nincluding location descriptions, and the objects (and their affordances) and\ncharacters (and their previous actions) present within it allows better\npredictions of agent behavior and dialogue. We analyze the ingredients\nnecessary for successful grounding in this setting, and how each of these\nfactors relate to agents that can talk and act successfully.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03282v1",
            "date": "2019-03-08 04:44:59+00:00",
            "title": "Attribute Acquisition in Ontology based on Representation Learning of Hierarchical Classes and Attributes",
            "authors": [
                "Tianwen Jiang",
                "Ming Liu",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Attribute acquisition for classes is a key step in ontology construction,\nwhich is often achieved by community members manually. This paper investigates\nan attention-based automatic paradigm called TransATT for attribute\nacquisition, by learning the representation of hierarchical classes and\nattributes in Chinese ontology. The attributes of an entity can be acquired by\nmerely inspecting its classes, because the entity can be regard as the instance\nof its classes and inherit their attributes. For explicitly describing of the\nclass of an entity unambiguously, we propose class-path to represent the\nhierarchical classes in ontology, instead of the terminal class word of the\nhypernym-hyponym relation (i.e., is-a relation) based hierarchy. The high\nperformance of TransATT on attribute acquisition indicates the promising\nability of the learned representation of class-paths and attributes. Moreover,\nwe construct a dataset named \\textbf{BigCilin11k}. To the best of our\nknowledge, this is the first Chinese dataset with abundant hierarchical classes\nand entities with attributes.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.03289v1",
            "date": "2019-03-08 05:10:00+00:00",
            "title": "Towards Time-Aware Distant Supervision for Relation Extraction",
            "authors": [
                "Tianwen Jiang",
                "Sendong Zhao",
                "Jing Liu",
                "Jin-Ge Yao",
                "Ming Liu",
                "Bing Qin",
                "Ting Liu",
                "Chin-Yew Lin"
            ],
            "abstract": "Distant supervision for relation extraction heavily suffers from the wrong\nlabeling problem. To alleviate this issue in news data with the timestamp, we\ntake a new factor time into consideration and propose a novel time-aware\ndistant supervision framework (Time-DS). Time-DS is composed of a time series\ninstance-popularity and two strategies. Instance-popularity is to encode the\nstrong relevance of time and true relation mention. Therefore,\ninstance-popularity would be an effective clue to reduce the noises generated\nthrough distant supervision labeling. The two strategies, i.e., hard filter and\ncurriculum learning are both ways to implement instance-popularity for better\nrelation extraction in the manner of Time-DS. The curriculum learning is a more\nsophisticated and flexible way to exploit instance-popularity to eliminate the\nbad effects of noises, thus get better relation extraction performance.\nExperiments on our collected multi-source news corpus show that Time-DS\nachieves significant improvements for relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03762v1",
            "date": "2019-03-09 08:24:15+00:00",
            "title": "Mutual Clustering on Comparative Texts via Heterogeneous Information Networks",
            "authors": [
                "Jianping Cao",
                "Senzhang Wang",
                "Danyan Wen",
                "Zhaohui Peng",
                "Philip S. Yu",
                "Fei-yue Wang"
            ],
            "abstract": "Currently, many intelligence systems contain the texts from multi-sources,\ne.g., bulletin board system (BBS) posts, tweets and news. These texts can be\n``comparative'' since they may be semantically correlated and thus provide us\nwith different perspectives toward the same topics or events. To better\norganize the multi-sourced texts and obtain more comprehensive knowledge, we\npropose to study the novel problem of Mutual Clustering on Comparative Texts\n(MCCT), which aims to cluster the comparative texts simultaneously and\ncollaboratively. The MCCT problem is difficult to address because 1)\ncomparative texts usually present different data formats and structures and\nthus they are hard to organize, and 2) there lacks an effective method to\nconnect the semantically correlated comparative texts to facilitate clustering\nthem in an unified way. To this aim, in this paper we propose a Heterogeneous\nInformation Network-based Text clustering framework HINT. HINT first models\nmulti-sourced texts (e.g. news and tweets) as heterogeneous information\nnetworks by introducing the shared ``anchor texts'' to connect the comparative\ntexts. Next, two similarity matrices based on HINT as well as a transition\nmatrix for cross-text-source knowledge transfer are constructed. Comparative\ntexts clustering are then conducted by utilizing the constructed matrices.\nFinally, a mutual clustering algorithm is also proposed to further unify the\nseparate clustering results of the comparative texts by introducing a\nclustering consistency constraint. We conduct extensive experimental on three\ntweets-news datasets, and the results demonstrate the effectiveness and\nrobustness of the proposed method in addressing the MCCT problem.",
            "journal ref": "Knowledge and Information System, 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.03772v1",
            "date": "2019-03-09 10:01:12+00:00",
            "title": "Logic Rules Powered Knowledge Graph Embedding",
            "authors": [
                "Pengwei Wang",
                "Dejing Dou",
                "Fangzhao Wu",
                "Nisansa de Silva",
                "Lianwen Jin"
            ],
            "abstract": "Large scale knowledge graph embedding has attracted much attention from both\nacademia and industry in the field of Artificial Intelligence. However, most\nexisting methods concentrate solely on fact triples contained in the given\nknowledge graph. Inspired by the fact that logic rules can provide a flexible\nand declarative language for expressing rich background knowledge, it is\nnatural to integrate logic rules into knowledge graph embedding, to transfer\nhuman knowledge to entity and relation embedding, and strengthen the learning\nprocess. In this paper, we propose a novel logic rule-enhanced method which can\nbe easily integrated with any translation based knowledge graph embedding\nmodel, such as TransE . We first introduce a method to automatically mine the\nlogic rules and corresponding confidences from the triples. And then, to put\nboth triples and mined logic rules within the same semantic space, all triples\nin the knowledge graph are represented as first-order logic. Finally, we define\nseveral operations on the first-order logic and minimize a global loss over\nboth of the mined logic rules and the transformed first-order logics. We\nconduct extensive experiments for link prediction and triple classification on\nthree datasets: WN18, FB166, and FB15K. Experiments show that the rule-enhanced\nmethod can significantly improve the performance of several baselines. The\nhighlight of our model is that the filtered Hits@1, which is a pivotal\nevaluation in the knowledge inference task, has a significant improvement (up\nto 700% improvement).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.03985v2",
            "date": "2019-03-10 13:16:37+00:00",
            "title": "Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approaches",
            "authors": [
                "Philip John Gorinski",
                "Honghan Wu",
                "Claire Grover",
                "Richard Tobin",
                "Conn Talbot",
                "Heather Whalley",
                "Cathie Sudlow",
                "William Whiteley",
                "Beatrice Alex"
            ],
            "abstract": "This work investigates multiple approaches to Named Entity Recognition (NER)\nfor text in Electronic Health Record (EHR) data. In particular, we look into\nthe application of (i) rule-based, (ii) deep learning and (iii) transfer\nlearning systems for the task of NER on brain imaging reports with a focus on\nrecords from patients with stroke. We explore the strengths and weaknesses of\neach approach, develop rules and train on a common dataset, and evaluate each\nsystem's performance on common test sets of Scottish radiology reports from two\nsources (brain imaging reports in ESS -- Edinburgh Stroke Study data collected\nby NHS Lothian as well as radiology reports created in NHS Tayside). Our\ncomparison shows that a hand-crafted system is the most accurate way to\nautomatically label EHR, but machine learning approaches can provide a feasible\nalternative where resources for a manual system are not readily available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.03995v3",
            "date": "2019-03-10 14:05:08+00:00",
            "title": "Efficiently Reusing Natural Language Processing Models for Phenotype-Mention Identification in Free-text Electronic Medical Records: Methodology Study",
            "authors": [
                "Honghan Wu",
                "Karen Hodgson",
                "Sue Dyson",
                "Katherine I. Morley",
                "Zina M. Ibrahim",
                "Ehtesham Iqbal",
                "Robert Stewart",
                "Richard JB Dobson",
                "Cathie Sudlow"
            ],
            "abstract": "Background: Many efforts have been put into the use of automated approaches,\nsuch as natural language processing (NLP), to mine or extract data from\nfree-text medical records to construct comprehensive patient profiles for\ndelivering better health-care. Reusing NLP models in new settings, however,\nremains cumbersome - requiring validation and/or retraining on new data\niteratively to achieve convergent results.\n  Objective: The aim of this work is to minimize the effort involved in reusing\nNLP models on free-text medical records.\n  Methods: We formally define and analyse the model adaptation problem in\nphenotype-mention identification tasks. We identify \"duplicate waste\" and\n\"imbalance waste\", which collectively impede efficient model reuse. We propose\na phenotype embedding based approach to minimize these sources of waste without\nthe need for labelled data from new settings.\n  Results: We conduct experiments on data from a large mental health registry\nto reuse NLP models in four phenotype-mention identification tasks. The\nproposed approach can choose the best model for a new task, identifying up to\n76% (duplicate waste), i.e. phenotype mentions without the need for validation\nand model retraining, and with very good performance (93-97% accuracy). It can\nalso provide guidance for validating and retraining the selected model for\nnovel language patterns in new tasks, saving around 80% (imbalance waste), i.e.\nthe effort required in \"blind\" model-adaptation approaches.\n  Conclusions: Adapting pre-trained NLP models for new tasks can be more\nefficient and effective if the language pattern landscapes of old settings and\nnew settings can be made explicit and comparable. Our experiments show that the\nphenotype-mention embedding approach is an effective way to model language\npatterns for phenotype-mention identification tasks and that its use can guide\nefficient NLP model reuse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04081v1",
            "date": "2019-03-11 00:10:29+00:00",
            "title": "Redditors in Recovery: Text Mining Reddit to Investigate Transitions into Drug Addiction",
            "authors": [
                "John Lu",
                "Sumati Sridhar",
                "Ritika Pandey",
                "Mohammad Al Hasan",
                "George Mohler"
            ],
            "abstract": "Increasing rates of opioid drug abuse and heightened prevalence of online\nsupport communities underscore the necessity of employing data mining\ntechniques to better understand drug addiction using these rapidly developing\nonline resources. In this work, we obtain data from Reddit, an online\ncollection of forums, to gather insight into drug use/misuse using text data\nfrom users themselves. Specifically, using user posts, we trained 1) a binary\nclassifier which predicts transitions from casual drug discussion forums to\ndrug recovery forums and 2) a Cox regression model that outputs likelihoods of\nsuch transitions. In doing so, we found that utterances of select drugs and\ncertain linguistic features contained in one's posts can help predict these\ntransitions. Using unfiltered drug-related posts, our research delineates drugs\nthat are associated with higher rates of transitions from recreational drug\ndiscussion to support/recovery discussion, offers insight into modern drug\nculture, and provides tools with potential applications in combating the opioid\ncrisis.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1903.04146v1",
            "date": "2019-03-11 07:08:39+00:00",
            "title": "An Innovative Word Encoding Method For Text Classification Using Convolutional Neural Network",
            "authors": [
                "Amr Adel Helmy",
                "Yasser M. K. Omar",
                "Rania Hodhod"
            ],
            "abstract": "Text classification plays a vital role today especially with the intensive\nuse of social networking media. Recently, different architectures of\nconvolutional neural networks have been used for text classification in which\none-hot vector, and word embedding methods are commonly used. This paper\npresents a new language independent word encoding method for text\nclassification. The proposed model converts raw text data to low-level feature\ndimension with minimal or no preprocessing steps by using a new approach called\nbinary unique number of word \"BUNOW\". BUNOW allows each unique word to have an\ninteger ID in a dictionary that is represented as a k-dimensional vector of its\nbinary equivalent. The output vector of this encoding is fed into a\nconvolutional neural network (CNN) model for classification. Moreover, the\nproposed model reduces the neural network parameters, allows faster computation\nwith few network layers, where a word is atomic representation the document as\nin word level, and decrease memory consumption for character level\nrepresentation. The provided CNN model is able to work with other languages or\nmulti-lingual text without the need for any changes in the encoding method. The\nmodel outperforms the character level and very deep character level CNNs models\nin terms of accuracy, network parameters, and memory consumption; the results\nshow total classification accuracy 91.99% and error 8.01% using AG's News\ndataset compared to the state of art methods that have total classification\naccuracy 91.45% and error 8.55%, in addition to the reduction in input feature\nvector and neural network parameters by 62% and 34%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.04750v1",
            "date": "2019-03-12 07:12:46+00:00",
            "title": "Interaction Embeddings for Prediction and Explanation in Knowledge Graphs",
            "authors": [
                "Wen Zhang",
                "Bibek Paudel",
                "Wei Zhang",
                "Abraham Bernstein",
                "Huajun Chen"
            ],
            "abstract": "Knowledge graph embedding aims to learn distributed representations for\nentities and relations, and is proven to be effective in many applications.\nCrossover interactions --- bi-directional effects between entities and\nrelations --- help select related information when predicting a new triple, but\nhaven't been formally discussed before. In this paper, we propose CrossE, a\nnovel knowledge graph embedding which explicitly simulates crossover\ninteractions. It not only learns one general embedding for each entity and\nrelation as most previous methods do, but also generates multiple triple\nspecific embeddings for both of them, named interaction embeddings. We evaluate\nembeddings on typical link prediction tasks and find that CrossE achieves\nstate-of-the-art results on complex and more challenging datasets. Furthermore,\nwe evaluate embeddings from a new perspective --- giving explanations for\npredicted triples, which is important for real applications. In this work, an\nexplanation for a triple is regarded as a reliable closed-path between the head\nand the tail entity. Compared to other baselines, we show experimentally that\nCrossE, benefiting from interaction embeddings, is more capable of generating\nreliable explanations to support its predictions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.04870v2",
            "date": "2019-03-12 12:30:54+00:00",
            "title": "Few-Shot and Zero-Shot Learning for Historical Text Normalization",
            "authors": [
                "Marcel Bollmann",
                "Natalia Korchagina",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Historical text normalization often relies on small training datasets. Recent\nwork has shown that multi-task learning can lead to significant improvements by\nexploiting synergies with related datasets, but there has been no systematic\nstudy of different multi-task learning architectures. This paper evaluates\n63~multi-task learning configurations for sequence-to-sequence-based historical\ntext normalization across ten datasets from eight languages, using\nautoencoding, grapheme-to-phoneme mapping, and lemmatization as auxiliary\ntasks. We observe consistent, significant improvements across languages when\ntraining data for the target task is limited, but minimal or no improvements\nwhen training data is abundant. We also show that zero-shot learning\noutperforms the simple, but relatively strong, identity baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05181v1",
            "date": "2019-03-12 19:57:34+00:00",
            "title": "Topological Analysis of Syntactic Structures",
            "authors": [
                "Alexander Port",
                "Taelin Karidi",
                "Matilde Marcolli"
            ],
            "abstract": "We use the persistent homology method of topological data analysis and\ndimensional analysis techniques to study data of syntactic structures of world\nlanguages. We analyze relations between syntactic parameters in terms of\ndimensionality, of hierarchical clustering structures, and of non-trivial\nloops. We show there are relations that hold across language families and\nadditional relations that are family-specific. We then analyze the trees\ndescribing the merging structure of persistent connected components for\nlanguages in different language families and we show that they partly correlate\nto historical phylogenetic trees but with significant differences. We also show\nthe existence of interesting non-trivial persistent first homology groups in\nvarious language families. We give examples where explicit generators for the\npersistent first homology can be identified, some of which appear to correspond\nto homoplasy phenomena, while others may have an explanation in terms of\nhistorical linguistics, corresponding to known cases of syntactic borrowing\nacross different language subfamilies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05210v1",
            "date": "2019-03-12 20:55:09+00:00",
            "title": "\"Hang in There\": Lexical and Visual Analysis to Identify Posts Warranting Empathetic Responses",
            "authors": [
                "Mimansa Jaiswal",
                "Sairam Tabibu",
                "Erik Cambria"
            ],
            "abstract": "In the past few years, social media has risen as a platform where people\nexpress and share personal incidences about abuse, violence and mental health\nissues. There is a need to pinpoint such posts and learn the kind of response\nexpected. For this purpose, we understand the sentiment that a personal story\nelicits on different posts present on different social media sites, on the\ntopics of abuse or mental health. In this paper, we propose a method supported\nby hand-crafted features to judge if the post requires an empathetic response.\nThe model is trained upon posts from various web-pages and corresponding\ncomments, on both the captions and the images. We were able to obtain 80%\naccuracy in tagging posts requiring empathetic responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05280v3",
            "date": "2019-03-12 09:36:25+00:00",
            "title": "Offensive Language Analysis using Deep Learning Architecture",
            "authors": [
                "Ryan Ong"
            ],
            "abstract": "SemEval-2019 Task 6 (Zampieri et al., 2019b) requires us to identify and\ncategorise offensive language in social media. In this paper we will describe\nthe process we took to tackle this challenge. Our process is heavily inspired\nby Sosa (2017) where he proposed CNN-LSTM and LSTM-CNN models to conduct\ntwitter sentiment analysis. We decided to follow his approach as well as\nfurther his work by testing out different variations of RNN models with CNN.\nSpecifically, we have divided the challenge into two parts: data processing and\nsampling and choosing the optimal deep learning architecture. In preprocessing,\nwe experimented with two techniques, SMOTE and Class Weights to counter the\nimbalance between classes. Once we are happy with the quality of our input\ndata, we proceed to choosing the optimal deep learning architecture for this\ntask. Given the quality and quantity of data we have been given, we found that\nthe addition of CNN layer provides very little to no additional improvement to\nour model's performance and sometimes even lead to a decrease in our F1-score.\nIn the end, the deep learning architecture that gives us the highest macro\nF1-score is a simple BiLSTM-CNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05440v1",
            "date": "2019-03-13 12:19:45+00:00",
            "title": "Market Trend Prediction using Sentiment Analysis: Lessons Learned and Paths Forward",
            "authors": [
                "Andrius Mudinas",
                "Dell Zhang",
                "Mark Levene"
            ],
            "abstract": "Financial market forecasting is one of the most attractive practical\napplications of sentiment analysis. In this paper, we investigate the potential\nof using sentiment \\emph{attitudes} (positive vs negative) and also sentiment\n\\emph{emotions} (joy, sadness, etc.) extracted from financial news or tweets to\nhelp predict stock price movements. Our extensive experiments using the\n\\emph{Granger-causality} test have revealed that (i) in general sentiment\nattitudes do not seem to Granger-cause stock price changes; and (ii) while on\nsome specific occasions sentiment emotions do seem to Granger-cause stock price\nchanges, the exhibited pattern is not universal and must be looked at on a case\nby case basis. Furthermore, it has been observed that at least for certain\nstocks, integrating sentiment emotions as additional features into the machine\nlearning based market trend prediction model could improve its accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05485v1",
            "date": "2019-03-13 13:48:32+00:00",
            "title": "MMKG: Multi-Modal Knowledge Graphs",
            "authors": [
                "Ye Liu",
                "Hui Li",
                "Alberto Garcia-Duran",
                "Mathias Niepert",
                "Daniel Onoro-Rubio",
                "David S. Rosenblum"
            ],
            "abstract": "We present MMKG, a collection of three knowledge graphs that contain both\nnumerical features and (links to) images for all entities as well as entity\nalignments between pairs of KGs. Therefore, multi-relational link prediction\nand entity matching communities can benefit from this resource. We believe this\ndata set has the potential to facilitate the development of novel multi-modal\nlearning approaches for knowledge graphs.We validate the utility ofMMKG in the\nsameAs link prediction task with an extensive set of experiments. These\nexperiments show that the task at hand benefits from learning of multiple\nfeature types.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.05543v1",
            "date": "2019-03-13 15:29:22+00:00",
            "title": "Adversarial attacks against Fact Extraction and VERification",
            "authors": [
                "James Thorne",
                "Andreas Vlachos"
            ],
            "abstract": "This paper describes a baseline for the second iteration of the Fact\nExtraction and VERification shared task (FEVER2.0) which explores the\nresilience of systems through adversarial evaluation. We present a collection\nof simple adversarial attacks against systems that participated in the first\nFEVER shared task. FEVER modeled the assessment of truthfulness of written\nclaims as a joint information retrieval and natural language inference task\nusing evidence from Wikipedia. A large number of participants made use of deep\nneural networks in their submissions to the shared task. The extent as to\nwhether such models understand language has been the subject of a number of\nrecent investigations and discussion in literature. In this paper, we present a\nsimple method of generating entailment-preserving and entailment-altering\nperturbations of instances by common patterns within the training data. We find\nthat a number of systems are greatly affected with absolute losses in\nclassification accuracy of up to $29\\%$ on the newly perturbed instances. Using\nthese newly generated instances, we construct a sample submission for the\nFEVER2.0 shared task. Addressing these types of attacks will aid in building\nmore robust fact-checking models, as well as suggest directions to expand the\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05566v3",
            "date": "2019-03-13 16:08:46+00:00",
            "title": "Benchmarking Natural Language Understanding Services for building Conversational Agents",
            "authors": [
                "Xingkun Liu",
                "Arash Eshghi",
                "Pawel Swietojanski",
                "Verena Rieser"
            ],
            "abstract": "We have recently seen the emergence of several publicly available Natural\nLanguage Understanding (NLU) toolkits, which map user utterances to structured,\nbut more abstract, Dialogue Act (DA) or Intent specifications, while making\nthis process accessible to the lay developer. In this paper, we present the\nfirst wide coverage evaluation and comparison of some of the most popular NLU\nservices, on a large, multi-domain (21 domains) dataset of 25K user utterances\nthat we have collected and annotated with Intent and Entity Type specifications\nand which will be released as part of this submission. The results show that on\nIntent classification Watson significantly outperforms the other platforms,\nnamely, Dialogflow, LUIS and Rasa; though these also perform well.\nInterestingly, on Entity Type recognition, Watson performs significantly worse\ndue to its low Precision. Again, Dialogflow, LUIS and Rasa perform well on this\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05801v1",
            "date": "2019-03-14 03:23:15+00:00",
            "title": "Survey of Text-based Epidemic Intelligence: A Computational Linguistic Perspective",
            "authors": [
                "Aditya Joshi",
                "Sarvnaz Karimi",
                "Ross Sparks",
                "Cecile Paris",
                "C Raina MacIntyre"
            ],
            "abstract": "Epidemic intelligence deals with the detection of disease outbreaks using\nformal (such as hospital records) and informal sources (such as user-generated\ntext on the web) of information. In this survey, we discuss approaches for\nepidemic intelligence that use textual datasets, referring to it as `text-based\nepidemic intelligence'. We view past work in terms of two broad categories:\nhealth mention classification (selecting relevant text from a large volume) and\nhealth event detection (predicting epidemic events from a collection of\nrelevant text). The focus of our discussion is the underlying computational\nlinguistic techniques in the two categories. The survey also provides details\nof the state-of-the-art in annotation techniques, resources and evaluation\nstrategies for epidemic intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05823v4",
            "date": "2019-03-14 05:53:22+00:00",
            "title": "Deep Patent Landscaping Model Using Transformer and Graph Embedding",
            "authors": [
                "Seokkyu Choi",
                "Hyeonju Lee",
                "Eunjeong Lucy Park",
                "Sungchul Choi"
            ],
            "abstract": "Patent landscaping is a method used for searching related patents during a\nresearch and development (R&D) project. To avoid the risk of patent\ninfringement and to follow current trends in technology, patent landscaping is\na crucial task required during the early stages of an R&D project. As the\nprocess of patent landscaping requires advanced resources and can be tedious,\nthe demand for automated patent landscaping has been gradually increasing.\nHowever, a shortage of well-defined benchmark datasets and comparable models\nmakes it difficult to find related research studies. In this paper, we propose\nan automated patent landscaping model based on deep learning. To analyze the\ntext of patents, the proposed model uses a modified transformer structure. To\nanalyze the metadata of patents, we propose a graph embedding method that uses\na diffusion graph called Diff2Vec. Furthermore, we introduce four benchmark\ndatasets for comparing related research studies in patent landscaping. The\ndatasets are produced by querying Google BigQuery, based on a search formula\nfrom a Korean patent attorney. The obtained results indicate that the proposed\nmodel and datasets can attain state-of-the-art performance, as compared with\ncurrent patent landscaping models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05872v1",
            "date": "2019-03-14 09:37:53+00:00",
            "title": "Interactive Concept Mining on Personal Data -- Bootstrapping Semantic Services",
            "authors": [
                "Markus Schr\u00f6der",
                "Christian Jilek",
                "Andreas Dengel"
            ],
            "abstract": "Semantic services (e.g. Semantic Desktops) are still afflicted by a cold\nstart problem: in the beginning, the user's personal information sphere, i.e.\nfiles, mails, bookmarks, etc., is not represented by the system. Information\nextraction tools used to kick-start the system typically create 1:1\nrepresentations of the different information items. Higher level concepts, for\nexample found in file names, mail subjects or in the content body of these\nitems, are not extracted. Leaving these concepts out may lead to\nunderperformance, having to many of them (e.g. by making every found term a\nconcept) will clutter the arising knowledge graph with non-helpful relations.\nIn this paper, we present an interactive concept mining approach proposing\nconcept candidates gathered by exploiting given schemata of usual personal\ninformation management applications and analysing the personal information\nsphere using various metrics. To heed the subjective view of the user, a\ngraphical user interface allows to easily rank and give feedback on proposed\nconcept candidates, thus keeping only those actually considered relevant. A\nprototypical implementation demonstrates major steps of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05942v4",
            "date": "2019-03-14 12:36:01+00:00",
            "title": "Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning",
            "authors": [
                "Dong-Jin Kim",
                "Jinsoo Choi",
                "Tae-Hyun Oh",
                "In So Kweon"
            ],
            "abstract": "Our goal in this work is to train an image captioning model that generates\nmore dense and informative captions. We introduce \"relational captioning,\" a\nnovel image captioning task which aims to generate multiple captions with\nrespect to relational information between objects in an image. Relational\ncaptioning is a framework that is advantageous in both diversity and amount of\ninformation, leading to image understanding based on relationships. Part-of\nspeech (POS, i.e. subject-object-predicate categories) tags can be assigned to\nevery English word. We leverage the POS as a prior to guide the correct\nsequence of words in a caption. To this end, we propose a multi-task\ntriple-stream network (MTTSNet) which consists of three recurrent units for the\nrespective POS and jointly performs POS prediction and captioning. We\ndemonstrate more diverse and richer representations generated by the proposed\nmodel against several baselines and competing methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1903.05987v2",
            "date": "2019-03-14 13:32:31+00:00",
            "title": "To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks",
            "authors": [
                "Matthew E. Peters",
                "Sebastian Ruder",
                "Noah A. Smith"
            ],
            "abstract": "While most previous work has focused on different pretraining objectives and\narchitectures for transfer learning, we ask how to best adapt the pretrained\nmodel to a given target task. We focus on the two most common forms of\nadaptation, feature extraction (where the pretrained weights are frozen), and\ndirectly fine-tuning the pretrained model. Our empirical results across diverse\nNLP tasks with two state-of-the-art models show that the relative performance\nof fine-tuning vs. feature extraction depends on the similarity of the\npretraining and target tasks. We explore possible explanations for this finding\nand provide a set of adaptation guidelines for the NLP practitioner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06607v1",
            "date": "2019-03-15 15:45:34+00:00",
            "title": "Matching Entities Across Different Knowledge Graphs with Graph Embeddings",
            "authors": [
                "Michael Azmy",
                "Peng Shi",
                "Jimmy Lin",
                "Ihab F. Ilyas"
            ],
            "abstract": "This paper explores the problem of matching entities across different\nknowledge graphs. Given a query entity in one knowledge graph, we wish to find\nthe corresponding real-world entity in another knowledge graph. We formalize\nthis problem and present two large-scale datasets for this task based on\nexiting cross-ontology links between DBpedia and Wikidata, focused on several\nhundred thousand ambiguous entities. Using a classification-based approach, we\nfind that a simple multi-layered perceptron based on representations derived\nfrom RDF2Vec graph embeddings of entities in each knowledge graph is sufficient\nto achieve high accuracy, with only small amounts of training data. The\ncontributions of our work are datasets for examining this problem and strong\nbaselines on which future work can be based.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.06765v1",
            "date": "2019-02-27 07:21:44+00:00",
            "title": "A Machine Learning Approach to Comment Toxicity Classification",
            "authors": [
                "Navoneel Chakrabarty"
            ],
            "abstract": "Now-a-days, derogatory comments are often made by one another, not only in\noffline environment but also immensely in online environments like social\nnetworking websites and online communities. So, an Identification combined with\nPrevention System in all social networking websites and applications, including\nall the communities, existing in the digital world is a necessity. In such a\nsystem, the Identification Block should identify any negative online behaviour\nand should signal the Prevention Block to take action accordingly. This study\naims to analyse any piece of text and detecting different types of toxicity\nlike obscenity, threats, insults and identity-based hatred. The labelled\nWikipedia Comment Dataset prepared by Jigsaw is used for the purpose. A\n6-headed Machine Learning tf-idf Model has been made and trained separately,\nyielding a Mean Validation Accuracy of 98.08% and Absolute Validation Accuracy\nof 91.61%. Such an Automated System should be deployed for enhancing healthy\nonline conversation",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07398v1",
            "date": "2019-03-11 18:18:38+00:00",
            "title": "Deep Text-to-Speech System with Seq2Seq Model",
            "authors": [
                "Gary Wang"
            ],
            "abstract": "Recent trends in neural network based text-to-speech/speech synthesis\npipelines have employed recurrent Seq2seq architectures that can synthesize\nrealistic sounding speech directly from text characters. These systems however\nhave complex architectures and takes a substantial amount of time to train. We\nintroduce several modifications to these Seq2seq architectures that allow for\nfaster training time, and also allows us to reduce the complexity of the model\narchitecture at the same time. We show that our proposed model can achieve\nattention alignment much faster than previous architectures and that good audio\nquality can be achieved with a model that's much smaller in size. Sample audio\navailable at https://soundcloud.com/gary-wang-23/sets/tts-samples-for-cmpt-419.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07588v1",
            "date": "2019-03-18 17:31:14+00:00",
            "title": "A Multilingual Encoding Method for Text Classification and Dialect Identification Using Convolutional Neural Network",
            "authors": [
                "Amr Adel Helmy"
            ],
            "abstract": "This thesis presents a language-independent text classification model by\nintroduced two new encoding methods \"BUNOW\" and \"BUNOC\" used for feeding the\nraw text data into a new CNN spatial architecture with vertical and horizontal\nconvolutional process instead of commonly used methods like one hot vector or\nword representation (i.e. word2vec) with temporal CNN architecture. The\nproposed model can be classified as hybrid word-character model in its work\nmethodology because it consumes less memory space by using a fewer neural\nnetwork parameters as in character level representation, in addition to\nproviding much faster computations with fewer network layers depth, as in word\nlevel representation. A promising result achieved compared to state of art\nmodels in two different morphological benchmarked dataset one for Arabic\nlanguage and one for English language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07666v1",
            "date": "2019-03-18 18:44:07+00:00",
            "title": "An Updated Duet Model for Passage Re-ranking",
            "authors": [
                "Bhaskar Mitra",
                "Nick Craswell"
            ],
            "abstract": "We propose several small modifications to Duet---a deep neural ranking\nmodel---and evaluate the updated model on the MS MARCO passage ranking task. We\nreport significant improvements from the proposed changes based on an ablation\nstudy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.08206v2",
            "date": "2019-03-19 18:31:23+00:00",
            "title": "Aligning Biomedical Metadata with Ontologies Using Clustering and Embeddings",
            "authors": [
                "Rafael S. Gon\u00e7alves",
                "Maulik R. Kamdar",
                "Mark A. Musen"
            ],
            "abstract": "The metadata about scientific experiments published in online repositories\nhave been shown to suffer from a high degree of representational\nheterogeneity---there are often many ways to represent the same type of\ninformation, such as a geographical location via its latitude and longitude. To\nharness the potential that metadata have for discovering scientific data, it is\ncrucial that they be represented in a uniform way that can be queried\neffectively. One step toward uniformly-represented metadata is to normalize the\nmultiple, distinct field names used in metadata (e.g., lat lon, lat and long)\nto describe the same type of value. To that end, we present a new method based\non clustering and embeddings (i.e., vector representations of words) to align\nmetadata field names with ontology terms. We apply our method to biomedical\nmetadata by generating embeddings for terms in biomedical ontologies from the\nBioPortal repository. We carried out a comparative study between our method and\nthe NCBO Annotator, which revealed that our method yields more and\nsubstantially better alignments between metadata and ontology terms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08268v2",
            "date": "2019-03-19 21:58:24+00:00",
            "title": "Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems",
            "authors": [
                "Arshit Gupta",
                "John Hewitt",
                "Katrin Kirchhoff"
            ],
            "abstract": "With the advent of conversational assistants, like Amazon Alexa, Google Now,\netc., dialogue systems are gaining a lot of traction, especially in industrial\nsetting. These systems typically consist of Spoken Language understanding\ncomponent which, in turn, consists of two tasks - Intent Classification (IC)\nand Slot Labeling (SL). Generally, these two tasks are modeled together jointly\nto achieve best performance. However, this joint modeling adds to model\nobfuscation. In this work, we first design framework for a modularization of\njoint IC-SL task to enhance architecture transparency. Then, we explore a\nnumber of self-attention, convolutional, and recurrent models, contributing a\nlarge-scale analysis of modeling paradigms for IC+SL across two datasets.\nFinally, using this framework, we propose a class of 'label-recurrent' models\nthat otherwise non-recurrent, with a 10-dimensional representation of the label\nhistory, and show that our proposed systems are easy to interpret, highly\naccurate (achieving over 30% error reduction in SL over the state-of-the-art on\nthe Snips dataset), as well as fast, at 2x the inference and 2/3 to 1/2 the\ntraining time of comparable recurrent models, thus giving an edge in critical\nreal-world systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08389v1",
            "date": "2019-03-20 08:53:05+00:00",
            "title": "Contextual Compositionality Detection with External Knowledge Bases andWord Embeddings",
            "authors": [
                "Dongsheng Wang",
                "Quichi Li",
                "Lucas Chaves Lima",
                "Jakob grue Simonsen",
                "Christina Lioma"
            ],
            "abstract": "When the meaning of a phrase cannot be inferred from the individual meanings\nof its words (e.g., hot dog), that phrase is said to be non-compositional.\nAutomatic compositionality detection in multi-word phrases is critical in any\napplication of semantic processing, such as search engines; failing to detect\nnon-compositional phrases can hurt system effectiveness notably. Existing\nresearch treats phrases as either compositional or non-compositional in a\ndeterministic manner. In this paper, we operationalize the viewpoint that\ncompositionality is contextual rather than deterministic, i.e., that whether a\nphrase is compositional or non-compositional depends on its context. For\nexample, the phrase `green card' is compositional when referring to a green\ncolored card, whereas it is non-compositional when meaning permanent residence\nauthorization. We address the challenge of detecting this type of contextual\ncompositionality as follows: given a multi-word phrase, we enrich the word\nembedding representing its semantics with evidence about its global context\n(terms it often collocates with) as well as its local context (narratives where\nthat phrase is used, which we call usage scenarios). We further extend this\nrepresentation with information extracted from external knowledge bases. The\nresulting representation incorporates both localized context and more general\nusage of the phrase and allows to detect its compositionality in a\nnon-deterministic and contextual way. Empirical evaluation of our model on a\ndataset of phrase compositionality, manually collected by crowdsourcing\ncontextual compositionality assessments, shows that our model outperforms\nstate-of-the-art baselines notably on detecting phrase compositionality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08718v1",
            "date": "2019-03-18 11:35:10+00:00",
            "title": "CRAFT: A multifunction online platform for speech prosody visualisation",
            "authors": [
                "Dafydd Gibbon"
            ],
            "abstract": "There are many research tools which are also used for teaching the acoustic\nphonetics of speech rhythm and speech melody. But they were not\npurpose-designed for teaching-learning situations, and some have a steep\nlearning curve. CRAFT (Creation and Recovery of Amplitude and Frequency Tracks)\nis custom-designed as a novel flexible online tool for visualisation and\ncritical comparison of functions and transforms, with implementations of the\nReaper, RAPT, PyRapt, YAAPT, YIN and PySWIPE F0 estimators, three Praat\nconfigurations, and two purpose-built estimators, PyAMDF, S0FT. Visualisations\nof amplitude and frequency envelope spectra, spectral edge detection of rhythm\nzones, and a parametrised spectrogram are included. A selection of audio clips\nfrom tone and intonation languages is provided for demonstration purposes. The\nmain advantages of online tools are consistency (users have the same version\nand the same data selection), interoperability over different platforms, and\nease of maintenance. The code is available on GitHub.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1903.08948v1",
            "date": "2019-03-21 12:26:44+00:00",
            "title": "Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning",
            "authors": [
                "Wen Zhang",
                "Bibek Paudel",
                "Liang Wang",
                "Jiaoyan Chen",
                "Hai Zhu",
                "Wei Zhang",
                "Abraham Bernstein",
                "Huajun Chen"
            ],
            "abstract": "Reasoning is essential for the development of large knowledge graphs,\nespecially for completion, which aims to infer new triples based on existing\nones. Both rules and embeddings can be used for knowledge graph reasoning and\nthey have their own advantages and difficulties. Rule-based reasoning is\naccurate and explainable but rule learning with searching over the graph always\nsuffers from efficiency due to huge search space. Embedding-based reasoning is\nmore scalable and efficient as the reasoning is conducted via computation\nbetween embeddings, but it has difficulty learning good representations for\nsparse entities because a good embedding relies heavily on data richness. Based\non this observation, in this paper we explore how embedding and rule learning\ncan be combined together and complement each other's difficulties with their\nadvantages. We propose a novel framework IterE iteratively learning embeddings\nand rules, in which rules are learned from embeddings with proper pruning\nstrategy and embeddings are learned from existing triples and new triples\ninferred by rules. Evaluations on embedding qualities of IterE show that rules\nhelp improve the quality of sparse entity embeddings and their link prediction\nresults. We also evaluate the efficiency of rule learning and quality of rules\nfrom IterE compared with AMIE+, showing that IterE is capable of generating\nhigh quality rules more efficiently. Experiments show that iteratively learning\nembeddings and rules benefit each other during learning and prediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.09025v1",
            "date": "2019-03-21 14:30:55+00:00",
            "title": "Recent advances in conversational NLP : Towards the standardization of Chatbot building",
            "authors": [
                "Maali Mnasri"
            ],
            "abstract": "Dialogue systems have become recently essential in our life. Their use is\ngetting more and more fluid and easy throughout the time. This boils down to\nthe improvements made in NLP and AI fields. In this paper, we try to provide an\noverview to the current state of the art of dialogue systems, their categories\nand the different approaches to build them. We end up with a discussion that\ncompares all the techniques and analyzes the strengths and weaknesses of each.\nFinally, we present an opinion piece suggesting to orientate the research\ntowards the standardization of dialogue systems building.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09244v2",
            "date": "2019-03-21 21:40:09+00:00",
            "title": "Low Resource Text Classification with ULMFit and Backtranslation",
            "authors": [
                "Sam Shleifer"
            ],
            "abstract": "In computer vision, virtually every state-of-the-art deep learning system is\ntrained with data augmentation. In text classification, however, data\naugmentation is less widely practiced because it must be performed before\ntraining and risks introducing label noise. We augment the IMDB movie reviews\ndataset with examples generated by two families of techniques: random token\nperturbations introduced by Wei and Zou [2019] and backtranslation --\ntranslating to a second language then back to English. In low resource\nenvironments, backtranslation generates significant improvement on top of the\nstate of-the-art ULMFit model. A ULMFit model pretrained on wikitext103 and\nthen fine-tuned on only 50 IMDB examples and 500 synthetic examples generated\nby backtranslation achieves 80.6% accuracy, an 8.1% improvement over the\naugmentation-free baseline with only 9 minutes of additional training time.\nRandom token perturbations do not yield any improvements but incur equivalent\ncomputational cost. The benefits of training with backtranslated examples\ndecreases with the size of the available training data. On the full dataset,\nneither augmentation technique improves upon ULMFit's state of the art\nperformance. We address this by using backtranslations as a form of test time\naugmentation as well as ensembling ULMFit with other models, and achieve small\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09333v2",
            "date": "2019-03-22 03:06:36+00:00",
            "title": "A Type-coherent, Expressive Representation as an Initial Step to Language Understanding",
            "authors": [
                "Gene Louis Kim",
                "Lenhart Schubert"
            ],
            "abstract": "A growing interest in tasks involving language understanding by the NLP\ncommunity has led to the need for effective semantic parsing and inference.\nModern NLP systems use semantic representations that do not quite fulfill the\nnuanced needs for language understanding: adequately modeling language\nsemantics, enabling general inferences, and being accurately recoverable. This\ndocument describes underspecified logical forms (ULF) for Episodic Logic (EL),\nwhich is an initial form for a semantic representation that balances these\nneeds. ULFs fully resolve the semantic type structure while leaving issues such\nas quantifier scope, word sense, and anaphora unresolved; they provide a\nstarting point for further resolution into EL, and enable certain structural\ninferences without further resolution. This document also presents preliminary\nresults of creating a hand-annotated corpus of ULFs for the purpose of training\na precise ULF parser, showing a three-person pairwise interannotator agreement\nof 0.88 on confident annotations. We hypothesize that a divide-and-conquer\napproach to semantic parsing starting with derivation of ULFs will lead to\nsemantic analyses that do justice to subtle aspects of linguistic meaning, and\nwill enable construction of more accurate semantic parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09846v1",
            "date": "2019-03-23 16:40:17+00:00",
            "title": "Toward the Evaluation of Written Proficiency on a Collaborative Social Network for Learning Languages: Yask",
            "authors": [
                "Fabio N. Silva",
                "Sergio Jimenez",
                "George Due\u00f1as"
            ],
            "abstract": "Yask is an online social collaborative network for practicing languages in a\nframework that includes requests, answers, and votes. Since measuring\nlinguistic competence using current approaches is difficult, expensive and in\nmany cases imprecise, we present a new alternative approach based on social\nnetworks. Our method, called Proficiency Rank, extends the well-known Page Rank\nalgorithm to measure the reputation of users in a collaborative social graph.\nFirst, we extended Page Rank so that it not only considers positive links\n(votes) but also negative links. Second, in addition to using explicit links,\nwe also incorporate other 4 types of signals implicit in the social graph.\nThese extensions allow Proficiency Rank to produce proficiency rankings for\nalmost all users in the data set used, where only a minority contributes by\nanswering, while the majority contributes only by voting. This overcomes the\nintrinsic limitation of Page Rank of only being able to rank the nodes that\nhave incoming links. Our experimental validation showed that the\nreputation/importance of the users in Yask is significantly correlated with\ntheir language proficiency. In contrast, their written production was poorly\ncorrelated with the vocabulary profiles of the Common European Framework of\nReference. In addition, we found that negative signals (votes) are considerably\nmore informative than positive ones. We concluded that the use of this\ntechnology is a promising tool for measuring second language proficiency, even\nfor relatively small groups of people.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1903.09941v1",
            "date": "2019-03-24 07:54:57+00:00",
            "title": "Relation extraction between the clinical entities based on the shortest dependency path based LSTM",
            "authors": [
                "Dhanachandra Ningthoujam",
                "Shweta Yadav",
                "Pushpak Bhattacharyya",
                "Asif Ekbal"
            ],
            "abstract": "Owing to the exponential rise in the electronic medical records, information\nextraction in this domain is becoming an important area of research in recent\nyears. Relation extraction between the medical concepts such as medical\nproblem, treatment, and test etc. is also one of the most important tasks in\nthis area. In this paper, we present an efficient relation extraction system\nbased on the shortest dependency path (SDP) generated from the dependency\nparsed tree of the sentence. Instead of relying on many handcrafted features\nand the whole sequence of tokens present in a sentence, our system relies only\non the SDP between the target entities. For every pair of entities, the system\ntakes only the words in the SDP, their dependency labels, Part-of-Speech\ninformation and the types of the entities as the input. We develop a dependency\nparser for extracting dependency information. We perform our experiments on the\nbenchmark i2b2 dataset for clinical relation extraction challenge 2010.\nExperimental results show that our system outperforms the existing systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10118v1",
            "date": "2019-03-25 03:40:15+00:00",
            "title": "End-to-End Learning Using Cycle Consistency for Image-to-Caption Transformations",
            "authors": [
                "Keisuke Hagiwara",
                "Yusuke Mukuta",
                "Tatsuya Harada"
            ],
            "abstract": "So far, research to generate captions from images has been carried out from\nthe viewpoint that a caption holds sufficient information for an image. If it\nis possible to generate an image that is close to the input image from a\ngenerated caption, i.e., if it is possible to generate a natural language\ncaption containing sufficient information to reproduce the image, then the\ncaption is considered to be faithful to the image. To make such regeneration\npossible, learning using the cycle-consistency loss is effective. In this\nstudy, we propose a method of generating captions by learning end-to-end mutual\ntransformations between images and texts. To evaluate our method, we perform\ncomparative experiments with and without the cycle consistency. The results are\nevaluated by an automatic evaluation and crowdsourcing, demonstrating that our\nproposed method is effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10222v1",
            "date": "2019-03-25 10:21:43+00:00",
            "title": "Anxious Depression Prediction in Real-time Social Data",
            "authors": [
                "Akshi Kumar",
                "Aditi Sharma",
                "Anshika Arora"
            ],
            "abstract": "Mental well-being and social media have been closely related domains of\nstudy. In this research a novel model, AD prediction model, for anxious\ndepression prediction in real-time tweets is proposed. This mixed\nanxiety-depressive disorder is a predominantly associated with erratic thought\nprocess, restlessness and sleeplessness. Based on the linguistic cues and user\nposting patterns, the feature set is defined using a 5-tuple vector <word,\ntiming, frequency, sentiment, contrast>. An anxiety-related lexicon is built to\ndetect the presence of anxiety indicators. Time and frequency of tweet is\nanalyzed for irregularities and opinion polarity analytics is done to find\ninconsistencies in posting behaviour. The model is trained using three\nclassifiers (multinomial na\\\"ive bayes, gradient boosting, and random forest)\nand majority voting using an ensemble voting classifier is done. Preliminary\nresults are evaluated for tweets of sampled 100 users and the proposed model\nachieves a classification accuracy of 85.09%.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1903.10245v4",
            "date": "2019-03-25 11:23:17+00:00",
            "title": "Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs",
            "authors": [
                "Zhibin Liu",
                "Zheng-Yu Niu",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Two types of knowledge, triples from knowledge graphs and texts from\ndocuments, have been studied for knowledge aware open-domain conversation\ngeneration, in which graph paths can narrow down vertex candidates for\nknowledge selection decision, and texts can provide rich information for\nresponse generation. Fusion of a knowledge graph and texts might yield mutually\nreinforcing advantages, but there is less study on that. To address this\nchallenge, we propose a knowledge aware chatting machine with three components,\nan augmented knowledge graph with both triples and texts, knowledge selector,\nand knowledge aware response generator. For knowledge selection on the graph,\nwe formulate it as a problem of multi-hop graph reasoning to effectively\ncapture conversation flow, which is more explainable and flexible in comparison\nwith previous work. To fully leverage long text information that differentiates\nour graph from others, we improve a state of the art reasoning algorithm with\nmachine reading comprehension technology. We demonstrate the effectiveness of\nour system on two datasets in comparison with state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.10453v1",
            "date": "2019-03-25 16:41:20+00:00",
            "title": "dpUGC: Learn Differentially Private Representation for User Generated Contents",
            "authors": [
                "Xuan-Son Vu",
                "Son N. Tran",
                "Lili Jiang"
            ],
            "abstract": "This paper firstly proposes a simple yet efficient generalized approach to\napply differential privacy to text representation (i.e., word embedding). Based\non it, we propose a user-level approach to learn personalized differentially\nprivate word embedding model on user generated contents (UGC). To our best\nknowledge, this is the first work of learning user-level differentially private\nword embedding model from text for sharing. The proposed approaches protect the\nprivacy of the individual from re-identification, especially provide better\ntrade-off of privacy and data utility on UGC data for sharing. The experimental\nresults show that the trained embedding models are applicable for the classic\ntext analysis tasks (e.g., regression). Moreover, the proposed approaches of\nlearning differentially private embedding models are both framework- and data-\nindependent, which facilitates the deployment and sharing. The source code is\navailable at https://github.com/sonvx/dpText.",
            "journal ref": "Proceedings of the 20th International Conference on Computational\n  Linguistics and Intelligent Text Processing, La Rochelle, France, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10561v1",
            "date": "2019-03-25 19:30:21+00:00",
            "title": "On Measuring Social Biases in Sentence Encoders",
            "authors": [
                "Chandler May",
                "Alex Wang",
                "Shikha Bordia",
                "Samuel R. Bowman",
                "Rachel Rudinger"
            ],
            "abstract": "The Word Embedding Association Test shows that GloVe and word2vec word\nembeddings exhibit human-like implicit biases based on gender, race, and other\nsocial constructs (Caliskan et al., 2017). Meanwhile, research on learning\nreusable text representations has begun to explore sentence-level texts, with\nsome sentence encoders seeing enthusiastic adoption. Accordingly, we extend the\nWord Embedding Association Test to measure bias in sentence encoders. We then\ntest several sentence encoders, including state-of-the-art methods such as ELMo\nand BERT, for the social biases studied in prior work and two important biases\nthat are difficult or impossible to test at the word level. We observe mixed\nresults including suspicious patterns of sensitivity that suggest the test's\nassumptions may not hold in general. We conclude by proposing directions for\nfuture work on measuring bias in sentence encoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10716v4",
            "date": "2019-03-26 07:44:39+00:00",
            "title": "Domain Representation for Knowledge Graph Embedding",
            "authors": [
                "Cunxiang Wang",
                "Feiliang Ren",
                "Zhichao Lin",
                "Chenxv Zhao",
                "Tian Xie",
                "Yue Zhang"
            ],
            "abstract": "Embedding entities and relations into a continuous multi-dimensional vector\nspace have become the dominant method for knowledge graph embedding in\nrepresentation learning. However, most existing models ignore to represent\nhierarchical knowledge, such as the similarities and dissimilarities of\nentities in one domain. We proposed to learn a Domain Representations over\nexisting knowledge graph embedding models, such that entities that have similar\nattributes are organized into the same domain. Such hierarchical knowledge of\ndomains can give further evidence in link prediction. Experimental results show\nthat domain embeddings give a significant improvement over the most recent\nstate-of-art baseline knowledge graph embedding models.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.10735v1",
            "date": "2019-03-26 08:43:38+00:00",
            "title": "Interoperability and machine-to-machine translation model with mappings to machine learning tasks",
            "authors": [
                "Jacob Nilsson",
                "Fredrik Sandin",
                "Jerker Delsing"
            ],
            "abstract": "Modern large-scale automation systems integrate thousands to hundreds of\nthousands of physical sensors and actuators. Demands for more flexible\nreconfiguration of production systems and optimization across different\ninformation models, standards and legacy systems challenge current system\ninteroperability concepts. Automatic semantic translation across information\nmodels and standards is an increasingly important problem that needs to be\naddressed to fulfill these demands in a cost-efficient manner under constraints\nof human capacity and resources in relation to timing requirements and system\ncomplexity. Here we define a translator-based operational interoperability\nmodel for interacting cyber-physical systems in mathematical terms, which\nincludes system identification and ontology-based translation as special cases.\nWe present alternative mathematical definitions of the translator learning task\nand mappings to similar machine learning tasks and solutions based on recent\ndevelopments in machine learning. Possibilities to learn translators between\nartefacts without a common physical context, for example in simulations of\ndigital twins and across layers of the automation pyramid are briefly\ndiscussed.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.10963v1",
            "date": "2019-03-26 15:43:36+00:00",
            "title": "Extracting Success from IBM's 20-Qubit Machines Using Error-Aware Compilation",
            "authors": [
                "Shin Nishio",
                "Yulu Pan",
                "Takahiko Satoh",
                "Hideharu Amano",
                "Rodney Van Meter"
            ],
            "abstract": "NISQ (Noisy, Intermediate-Scale Quantum) computing requires error mitigation\nto achieve meaningful computation. Our compilation tool development focuses on\nthe fact that the error rates of individual qubits are not equal, with a goal\nof maximizing the success probability of real-world subroutines such as an\nadder circuit. We begin by establishing a metric for choosing among possible\npaths and circuit alternatives for executing gates between variables placed far\napart within the processor, and test our approach on two IBM 20-qubit systems\nnamed Tokyo and Poughkeepsie. We find that a single-number metric describing\nthe fidelity of individual gates is a useful but imperfect guide. Our compiler\nuses this subsystem and maps complete circuits onto the machine using a beam\nsearch-based heuristic that will scale as processor and program sizes grow. To\nevaluate the whole compilation process, we compiled and executed adder\ncircuits, then calculated the KL-divergence (a measure of the distance between\ntwo probability distributions). For a circuit within the capabilities of the\nhardware, our compilation increases estimated success probability and reduces\nKL-divergence relative to an error-oblivious placement.",
            "journal ref": "ACM Journal on Emerging Technologies in Computing Systems (May\n  2020) Vol.16, No.3",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1903.10972v1",
            "date": "2019-03-26 15:58:33+00:00",
            "title": "Simple Applications of BERT for Ad Hoc Document Retrieval",
            "authors": [
                "Wei Yang",
                "Haotian Zhang",
                "Jimmy Lin"
            ],
            "abstract": "Following recent successes in applying BERT to question answering, we explore\nsimple applications to ad hoc document retrieval. This required confronting the\nchallenge posed by documents that are typically longer than the length of input\nBERT was designed to handle. We address this issue by applying inference on\nsentences individually, and then aggregating sentence scores to produce\ndocument scores. Experiments on TREC microblog and newswire test collections\nshow that our approach is simple yet effective, as we report the highest\naverage precision on these datasets by neural approaches that we are aware of.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.11024v1",
            "date": "2019-03-26 17:10:07+00:00",
            "title": "Deep Learning and Word Embeddings for Tweet Classification for Crisis Response",
            "authors": [
                "Reem ALRashdi",
                "Simon O'Keefe"
            ],
            "abstract": "Tradition tweet classification models for crisis response focus on\nconvolutional layers and domain-specific word embeddings. In this paper, we\nstudy the application of different neural networks with general-purpose and\ndomain-specific word embeddings to investigate their ability to improve the\nperformance of tweet classification models. We evaluate four tweet\nclassification models on CrisisNLP dataset and obtain comparable results which\nindicates that general-purpose word embedding such as GloVe can be used instead\nof domain-specific word embedding especially with Bi-LSTM where results\nreported the highest performance of 62.04% F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11245v1",
            "date": "2019-03-27 04:25:57+00:00",
            "title": "On Attribution of Recurrent Neural Network Predictions via Additive Decomposition",
            "authors": [
                "Mengnan Du",
                "Ninghao Liu",
                "Fan Yang",
                "Shuiwang Ji",
                "Xia Hu"
            ],
            "abstract": "RNN models have achieved the state-of-the-art performance in a wide range of\ntext mining tasks. However, these models are often regarded as black-boxes and\nare criticized due to the lack of interpretability. In this paper, we enhance\nthe interpretability of RNNs by providing interpretable rationales for RNN\npredictions. Nevertheless, interpreting RNNs is a challenging problem. Firstly,\nunlike existing methods that rely on local approximation, we aim to provide\nrationales that are more faithful to the decision making process of RNN models.\nSecondly, a flexible interpretation method should be able to assign\ncontribution scores to text segments of varying lengths, instead of only to\nindividual words. To tackle these challenges, we propose a novel attribution\nmethod, called REAT, to provide interpretations to RNN predictions. REAT\ndecomposes the final prediction of a RNN into additive contribution of each\nword in the input text. This additive decomposition enables REAT to further\nobtain phrase-level attribution scores. In addition, REAT is generally\napplicable to various RNN architectures, including GRU, LSTM and their\nbidirectional versions. Experimental results demonstrate the faithfulness and\ninterpretability of the proposed attribution method. Comprehensive analysis\nshows that our attribution method could unveil the useful linguistic knowledge\ncaptured by RNNs. Some analysis further demonstrates our method could be\nutilized as a debugging tool to examine the vulnerability and failure reasons\nof RNNs, which may lead to several promising future directions to promote\ngeneralization ability of RNNs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11299v3",
            "date": "2019-03-27 09:02:41+00:00",
            "title": "Image search using multilingual texts: a cross-modal learning approach between image and text",
            "authors": [
                "Maxime Portaz",
                "Hicham Randrianarivo",
                "Adrien Nivaggioli",
                "Estelle Maudet",
                "Christophe Servan",
                "Sylvain Peyronnet"
            ],
            "abstract": "Multilingual (or cross-lingual) embeddings represent several languages in a\nunique vector space. Using a common embedding space enables for a shared\nsemantic between words from different languages. In this paper, we propose to\nembed images and texts into a unique distributional vector space, enabling to\nsearch images by using text queries expressing information needs related to the\n(visual) content of images, as well as using image similarity. Our framework\nforces the representation of an image to be similar to the representation of\nthe text that describes it. Moreover, by using multilingual embeddings we\nensure that words from two different languages have close descriptors and thus\nare attached to similar images. We provide experimental evidence of the\nefficiency of our approach by experimenting it on two datasets: Common Objects\nin COntext (COCO) [19] and Multi30K [7].",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1903.11367v2",
            "date": "2019-03-27 12:00:20+00:00",
            "title": "Does My Rebuttal Matter? Insights from a Major NLP Conference",
            "authors": [
                "Yang Gao",
                "Steffen Eger",
                "Ilia Kuznetsov",
                "Iryna Gurevych",
                "Yusuke Miyao"
            ],
            "abstract": "Peer review is a core element of the scientific process, particularly in\nconference-centered fields such as ML and NLP. However, only few studies have\nevaluated its properties empirically. Aiming to fill this gap, we present a\ncorpus that contains over 4k reviews and 1.2k author responses from ACL-2018.\nWe quantitatively and qualitatively assess the corpus. This includes a pilot\nstudy on paper weaknesses given by reviewers and on quality of author\nresponses. We then focus on the role of the rebuttal phase, and propose a novel\ntask to predict after-rebuttal (i.e., final) scores from initial reviews and\nauthor responses. Although author responses do have a marginal (and\nstatistically significant) influence on the final scores, especially for\nborderline papers, our results suggest that a reviewer's final score is largely\ndetermined by her initial score and the distance to the other reviewers'\ninitial scores. In this context, we discuss the conformity bias inherent to\npeer reviewing, a bias that has largely been overlooked in previous research.\nWe hope our analyses will help better assess the usefulness of the rebuttal\nphase in NLP conferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11919v1",
            "date": "2019-03-28 12:38:58+00:00",
            "title": "Imbalanced Sentiment Classification Enhanced with Discourse Marker",
            "authors": [
                "Tao Zhang",
                "Xing Wu",
                "Meng Lin",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Imbalanced data commonly exists in real world, espacially in\nsentiment-related corpus, making it difficult to train a classifier to\ndistinguish latent sentiment in text data. We observe that humans often express\ntransitional emotion between two adjacent discourses with discourse markers\nlike \"but\", \"though\", \"while\", etc, and the head discourse and the tail\ndiscourse 3 usually indicate opposite emotional tendencies. Based on this\nobservation, we propose a novel plug-and-play method, which first samples\ndiscourses according to transitional discourse markers and then validates\nsentimental polarities with the help of a pretrained attention-based model. Our\nmethod increases sample diversity in the first place, can serve as a upstream\npreprocessing part in data augmentation. We conduct experiments on three public\nsentiment datasets, with several frequently used algorithms. Results show that\nour method is found to be consistently effective, even in highly imbalanced\nscenario, and easily be integrated with oversampling method to boost the\nperformance on imbalanced sentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12008v1",
            "date": "2019-03-28 14:33:50+00:00",
            "title": "Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling",
            "authors": [
                "Debjit Paul",
                "Mittul Singh",
                "Michael A. Hedderich",
                "Dietrich Klakow"
            ],
            "abstract": "In this paper, we address the problem of effectively self-training neural\nnetworks in a low-resource setting. Self-training is frequently used to\nautomatically increase the amount of training data. However, in a low-resource\nscenario, it is less effective due to unreliable annotations created using\nself-labeling of unlabeled data. We propose to combine self-training with noise\nhandling on the self-labeled data. Directly estimating noise on the combined\nclean training set and self-labeled data can lead to corruption of the clean\ndata and hence, performs worse. Thus, we propose the Clean and Noisy Label\nNeural Network which trains on clean and noisy self-labeled data simultaneously\nby explicitly modelling clean and noisy labels separately. In our experiments\non Chunking and NER, this approach performs more robustly than the baselines.\nComplementary to this explicit approach, noise can also be handled implicitly\nwith the help of an auxiliary learning task. To such a complementary approach,\nour method is more beneficial than other baseline methods and together provides\nthe best performance overall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12017v1",
            "date": "2019-03-28 14:45:42+00:00",
            "title": "Train, Sort, Explain: Learning to Diagnose Translation Models",
            "authors": [
                "Robert Schwarzenberg",
                "David Harbecke",
                "Vivien Macketanz",
                "Eleftherios Avramidis",
                "Sebastian M\u00f6ller"
            ],
            "abstract": "Evaluating translation models is a trade-off between effort and detail. On\nthe one end of the spectrum there are automatic count-based methods such as\nBLEU, on the other end linguistic evaluations by humans, which arguably are\nmore informative but also require a disproportionately high effort. To narrow\nthe spectrum, we propose a general approach on how to automatically expose\nsystematic differences between human and machine translations to human experts.\nInspired by adversarial settings, we train a neural text classifier to\ndistinguish human from machine translations. A classifier that performs and\ngeneralizes well after training should recognize systematic differences between\nthe two classes, which we uncover with neural explainability methods. Our\nproof-of-concept implementation, DiaMaT, is open source. Applied to a dataset\ntranslated by a state-of-the-art neural Transformer model, DiaMaT achieves a\nclassification accuracy of 75% and exposes meaningful differences between\nhumans and the Transformer, amidst the current discussion about human parity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12136v1",
            "date": "2019-03-28 17:23:50+00:00",
            "title": "Distilling Task-Specific Knowledge from BERT into Simple Neural Networks",
            "authors": [
                "Raphael Tang",
                "Yao Lu",
                "Linqing Liu",
                "Lili Mou",
                "Olga Vechtomova",
                "Jimmy Lin"
            ],
            "abstract": "In the natural language processing literature, neural networks are becoming\nincreasingly deeper and complex. The recent poster child of this trend is the\ndeep language representation model, which includes BERT, ELMo, and GPT. These\ndevelopments have led to the conviction that previous-generation, shallower\nneural networks for language understanding are obsolete. In this paper,\nhowever, we demonstrate that rudimentary, lightweight neural networks can still\nbe made competitive without architecture changes, external training data, or\nadditional input features. We propose to distill knowledge from BERT, a\nstate-of-the-art language representation model, into a single-layer BiLSTM, as\nwell as its siamese counterpart for sentence-pair tasks. Across multiple\ndatasets in paraphrasing, natural language inference, and sentiment\nclassification, we achieve comparable results with ELMo, while using roughly\n100 times fewer parameters and 15 times less inference time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12354v2",
            "date": "2019-03-29 05:02:51+00:00",
            "title": "Training neural networks to encode symbols enables combinatorial generalization",
            "authors": [
                "Ivan Vankov",
                "Jeffrey Bowers"
            ],
            "abstract": "Combinatorial generalization - the ability to understand and produce novel\ncombinations of already familiar elements - is considered to be a core capacity\nof the human mind and a major challenge to neural network models. A significant\nbody of research suggests that conventional neural networks can't solve this\nproblem unless they are endowed with mechanisms specifically engineered for the\npurpose of representing symbols. In this paper we introduce a novel way of\nrepresenting symbolic structures in connectionist terms - the vectors approach\nto representing symbols (VARS), which allows training standard neural\narchitectures to encode symbolic knowledge explicitly at their output layers.\nIn two simulations, we show that neural networks not only can learn to produce\nVARS representations, but in doing so they achieve combinatorial generalization\nin their symbolic and non-symbolic output. This adds to other recent work that\nhas shown improved combinatorial generalization under specific training\nconditions, and raises the question of whether specific mechanisms or training\nroutines are needed to support symbolic processing.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.12356v1",
            "date": "2019-03-29 05:15:58+00:00",
            "title": "A General FOFE-net Framework for Simple and Effective Question Answering over Knowledge Bases",
            "authors": [
                "Dekun Wu",
                "Nana Nosirova",
                "Hui Jiang",
                "Mingbin Xu"
            ],
            "abstract": "Question answering over knowledge base (KB-QA) has recently become a popular\nresearch topic in NLP. One popular way to solve the KB-QA problem is to make\nuse of a pipeline of several NLP modules, including entity discovery and\nlinking (EDL) and relation detection. Recent success on KB-QA task usually\ninvolves complex network structures with sophisticated heuristics. Inspired by\na previous work that builds a strong KB-QA baseline, we propose a simple but\ngeneral neural model composed of fixed-size ordinally forgetting encoding\n(FOFE) and deep neural networks, called FOFE-net to solve KB-QA problem at\ndifferent stages. For evaluation, we use two popular KB-QA datasets,\nSimpleQuestions and WebQSP, and a newly created dataset, FreebaseQA. The\nexperimental results show that FOFE-net performs well on KB-QA subtasks, entity\ndiscovery and linking (EDL) and relation detection, and in turn pushing overall\nKB-QA system to achieve strong results on all datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12424v1",
            "date": "2019-03-29 09:57:45+00:00",
            "title": "Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech",
            "authors": [
                "Zixing Zhang",
                "Bingwen Wu",
                "Bjoern Schuller"
            ],
            "abstract": "Despite the increasing research interest in end-to-end learning systems for\nspeech emotion recognition, conventional systems either suffer from the\noverfitting due in part to the limited training data, or do not explicitly\nconsider the different contributions of automatically learnt representations\nfor a specific task. In this contribution, we propose a novel end-to-end\nframework which is enhanced by learning other auxiliary tasks and an attention\nmechanism. That is, we jointly train an end-to-end network with several\ndifferent but related emotion prediction tasks, i.e., arousal, valence, and\ndominance predictions, to extract more robust representations shared among\nvarious tasks than traditional systems with the hope that it is able to relieve\nthe overfitting problem. Meanwhile, an attention layer is implemented on top of\nthe layers for each task, with the aim to capture the contribution distribution\nof different segment parts for each individual task. To evaluate the\neffectiveness of the proposed system, we conducted a set of experiments on the\nwidely used database IEMOCAP. The empirical results show that the proposed\nsystems significantly outperform corresponding baseline systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.12520v1",
            "date": "2019-03-13 07:54:29+00:00",
            "title": "Multimodal Emotion Classification",
            "authors": [
                "Anurag Illendula",
                "Amit Sheth"
            ],
            "abstract": "Most NLP and Computer Vision tasks are limited to scarcity of labelled data.\nIn social media emotion classification and other related tasks, hashtags have\nbeen used as indicators to label data. With the rapid increase in emoji usage\nof social media, emojis are used as an additional feature for major social NLP\ntasks. However, this is less explored in case of multimedia posts on social\nmedia where posts are composed of both image and text. At the same time, w.e\nhave seen a surge in the interest to incorporate domain knowledge to improve\nmachine understanding of text. In this paper, we investigate whether domain\nknowledge for emoji can improve the accuracy of emotion classification task. We\nexploit the importance of different modalities from social media post for\nemotion classification task using state-of-the-art deep learning architectures.\nOur experiments demonstrate that the three modalities (text, emoji and images)\nencode different information to express emotion and therefore can complement\neach other. Our results also demonstrate that emoji sense depends on the\ntextual context, and emoji combined with text encodes better information than\nconsidered separately. The highest accuracy of 71.98\\% is achieved with a\ntraining data of 550k posts.",
            "journal ref": "Companion Proceedings of the 2019 World Wide Web Conference",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1903.12542v1",
            "date": "2019-03-29 14:32:02+00:00",
            "title": "Re-Ranking Words to Improve Interpretability of Automatically Generated Topics",
            "authors": [
                "Areej Alokaili",
                "Nikolaos Aletras",
                "Mark Stevenson"
            ],
            "abstract": "Topics models, such as LDA, are widely used in Natural Language Processing.\nMaking their output interpretable is an important area of research with\napplications to areas such as the enhancement of exploratory search interfaces\nand the development of interpretable machine learning models. Conventionally,\ntopics are represented by their n most probable words, however, these\nrepresentations are often difficult for humans to interpret. This paper\nexplores the re-ranking of topic words to generate more interpretable topic\nrepresentations. A range of approaches are compared and evaluated in two\nexperiments. The first uses crowdworkers to associate topics represented by\ndifferent word rankings with related documents. The second experiment is an\nautomatic approach based on a document retrieval task applied on multiple\ndomains. Results in both experiments demonstrate that re-ranking words improves\ntopic interpretability and that the most effective re-ranking schemes were\nthose which combine information about the importance of words both within\ntopics and their relative frequency in the entire corpus. In addition, close\ncorrelation between the results of the two evaluation approaches suggests that\nthe automatic method proposed here could be used to evaluate re-ranking methods\nwithout the need for human judgements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00143v1",
            "date": "2019-03-30 03:55:20+00:00",
            "title": "Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions",
            "authors": [
                "Zhi-Xiu Ye",
                "Zhen-Hua Ling"
            ],
            "abstract": "This paper presents a neural relation extraction method to deal with the\nnoisy training data generated by distant supervision. Previous studies mainly\nfocus on sentence-level de-noising by designing neural networks with intra-bag\nattentions. In this paper, both intra-bag and inter-bag attentions are\nconsidered in order to deal with the noise at sentence-level and bag-level\nrespectively. First, relation-aware bag representations are calculated by\nweighting sentence embeddings using intra-bag attentions. Here, each possible\nrelation is utilized as the query for attention calculation instead of only\nusing the target relation in conventional methods. Furthermore, the\nrepresentation of a group of bags in the training set which share the same\nrelation label is calculated by weighting bag representations using a\nsimilarity-based inter-bag attention module. Finally, a bag group is utilized\nas a training sample when building our relation extractor. Experimental results\non the New York Times dataset demonstrate the effectiveness of our proposed\nintra-bag and inter-bag attention modules. Our method also achieves better\nrelation extraction accuracy than state-of-the-art methods on this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00160v1",
            "date": "2019-03-30 07:13:38+00:00",
            "title": "Machine translation considering context information using Encoder-Decoder model",
            "authors": [
                "Tetsuto Takano",
                "Satoshi Yamane"
            ],
            "abstract": "In the task of machine translation, context information is one of the\nimportant factor. But considering the context information model dose not\nproposed. The paper propose a new model which can integrate context information\nand make translation. In this paper, we create a new model based Encoder\nDecoder model. When translating current sentence, the model integrates output\nfrom preceding encoder with current encoder. The model can consider context\ninformation and the result score is higher than existing model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00785v1",
            "date": "2019-03-25 18:59:32+00:00",
            "title": "Question Embeddings Based on Shannon Entropy: Solving intent classification task in goal-oriented dialogue system",
            "authors": [
                "Aleksandr Perevalov",
                "Daniil Kurushin",
                "Rustam Faizrakhmanov",
                "Farida Khabibrakhmanova"
            ],
            "abstract": "Question-answering systems and voice assistants are becoming major part of\nclient service departments of many organizations, helping them to reduce the\nlabor costs of staff. In many such systems, there is always natural language\nunderstanding module that solves intent classification task. This task is\ncomplicated because of its case-dependency - every subject area has its own\nsemantic kernel. The state of art approaches for intent classification are\ndifferent machine learning and deep learning methods that use text vector\nrepresentations as input. The basic vector representation models such as Bag of\nwords and TF-IDF generate sparse matrixes, which are becoming very big as the\namount of input data grows. Modern methods such as word2vec and FastText use\nneural networks to evaluate word embeddings with fixed dimension size. As we\nare developing a question-answering system for students and enrollees of the\nPerm National Research Polytechnic University, we have faced the problem of\nuser's intent detection. The subject area of our system is very specific, that\nis why there is a lack of training data. This aspect makes intent\nclassification task more challenging for using state of the art deep learning\nmethods. In this paper, we propose an approach of the questions embeddings\nrepresentation based on calculation of Shannon entropy.The goal of the approach\nis to produce low dimensional question vectors as neural approaches do and to\noutperform related methods, described above in condition of small dataset. We\nevaluate and compare our model with existing ones using logistic regression and\ndataset that contains questions asked by students and enrollees. The data is\nlabeled into six classes. Experimental comparison of proposed approach and\nother models revealed that proposed model performed better in the given task.",
            "journal ref": "2019/03/06, Volume 7, Issue 1, Koethen Germany, ISBN:\n  978-3-96057-086-8 (Online)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00812v1",
            "date": "2019-03-27 14:21:57+00:00",
            "title": "Polysemy and brevity versus frequency in language",
            "authors": [
                "Bernardino Casas",
                "Antoni Hern\u00e1ndez-Fern\u00e1ndez",
                "Neus Catal\u00e0",
                "Ramon Ferrer-i-Cancho",
                "Jaume Baixeries"
            ],
            "abstract": "The pioneering research of G. K. Zipf on the relationship between word\nfrequency and other word features led to the formulation of various linguistic\nlaws. The most popular is Zipf's law for word frequencies. Here we focus on two\nlaws that have been studied less intensively: the meaning-frequency law, i.e.\nthe tendency of more frequent words to be more polysemous, and the law of\nabbreviation, i.e. the tendency of more frequent words to be shorter. In a\nprevious work, we tested the robustness of these Zipfian laws for English,\nroughly measuring word length in number of characters and distinguishing adult\nfrom child speech. In the present article, we extend our study to other\nlanguages (Dutch and Spanish) and introduce two additional measures of length:\nsyllabic length and phonemic length. Our correlation analysis indicates that\nboth the meaning-frequency law and the law of abbreviation hold overall in all\nthe analyzed languages.",
            "journal ref": "Computer Speech and Language 58, 19-50 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01313v1",
            "date": "2019-04-02 10:00:58+00:00",
            "title": "Short Text Classification Improved by Feature Space Extension",
            "authors": [
                "Yanxuan Li"
            ],
            "abstract": "With the explosive development of mobile Internet, short text has been\napplied extensively. The difference between classifying short text and long\ndocuments is that short text is of shortness and sparsity. Thus, it is\nchallenging to deal with short text classification owing to its less semantic\ninformation. In this paper, we propose a novel topic-based convolutional neural\nnetwork (TB-CNN) based on Latent Dirichlet Allocation (LDA) model and\nconvolutional neural network. Comparing to traditional CNN methods, TB-CNN\ngenerates topic words with LDA model to reduce the sparseness and combines the\nembedding vectors of topic words and input words to extend feature space of\nshort text. The validation results on IMDB movie review dataset show the\nimprovement and effectiveness of TB-CNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01356v1",
            "date": "2019-04-02 11:57:40+00:00",
            "title": "Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition",
            "authors": [
                "Omer Arshad",
                "Ignazio Gallo",
                "Shah Nawaz",
                "Alessandro Calefati"
            ],
            "abstract": "With massive explosion of social media such as Twitter and Instagram, people\ndaily share billions of multimedia posts, containing images and text.\nTypically, text in these posts is short, informal and noisy, leading to\nambiguities which can be resolved using images. In this paper we explore\ntext-centric Named Entity Recognition task on these multimedia posts. We\npropose an end to end model which learns a joint representation of a text and\nan image. Our model extends multi-dimensional self attention technique, where\nnow image helps to enhance relationship between words. Experiments show that\nour model is capable of capturing both textual and visual contexts with greater\naccuracy, achieving state-of-the-art results on Twitter multimodal Named Entity\nRecognition dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.01451v1",
            "date": "2019-04-02 14:17:19+00:00",
            "title": "Using Multi-Sense Vector Embeddings for Reverse Dictionaries",
            "authors": [
                "Michael A. Hedderich",
                "Andrew Yates",
                "Dietrich Klakow",
                "Gerard de Melo"
            ],
            "abstract": "Popular word embedding methods such as word2vec and GloVe assign a single\nvector representation to each word, even if a word has multiple distinct\nmeanings. Multi-sense embeddings instead provide different vectors for each\nsense of a word. However, they typically cannot serve as a drop-in replacement\nfor conventional single-sense embeddings, because the correct sense vector\nneeds to be selected for each word. In this work, we study the effect of\nmulti-sense embeddings on the task of reverse dictionaries. We propose a\ntechnique to easily integrate them into an existing neural network architecture\nusing an attention mechanism. Our experiments demonstrate that large\nimprovements can be obtained when employing multi-sense embeddings both in the\ninput sequence as well as for the target representation. An analysis of the\nsense distributions and of the learned attention is provided as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01500v1",
            "date": "2019-04-02 15:39:27+00:00",
            "title": "Neural Vector Conceptualization for Word Vector Space Interpretation",
            "authors": [
                "Robert Schwarzenberg",
                "Lisa Raithel",
                "David Harbecke"
            ],
            "abstract": "Distributed word vector spaces are considered hard to interpret which hinders\nthe understanding of natural language processing (NLP) models. In this work, we\nintroduce a new method to interpret arbitrary samples from a word vector space.\nTo this end, we train a neural model to conceptualize word vectors, which means\nthat it activates higher order concepts it recognizes in a given vector.\nContrary to prior approaches, our model operates in the original vector space\nand is capable of learning non-linear relations between word vectors and\nconcepts. Furthermore, we show that it produces considerably less entropic\nconcept activation profiles than the popular cosine similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01548v1",
            "date": "2019-04-02 17:02:19+00:00",
            "title": "Understanding language-elicited EEG data by predicting it from a fine-tuned language model",
            "authors": [
                "Dan Schwartz",
                "Tom Mitchell"
            ],
            "abstract": "Electroencephalography (EEG) recordings of brain activity taken while\nparticipants read or listen to language are widely used within the cognitive\nneuroscience and psycholinguistics communities as a tool to study language\ncomprehension. Several time-locked stereotyped EEG responses to\nword-presentations -- known collectively as event-related potentials (ERPs) --\nare thought to be markers for semantic or syntactic processes that take place\nduring comprehension. However, the characterization of each individual ERP in\nterms of what features of a stream of language trigger the response remains\ncontroversial. Improving this characterization would make ERPs a more useful\ntool for studying language comprehension. We take a step towards better\nunderstanding the ERPs by fine-tuning a language model to predict them. This\nnew approach to analysis shows for the first time that all of the ERPs are\npredictable from embeddings of a stream of language. Prior work has only found\ntwo of the ERPs to be predictable. In addition to this analysis, we examine\nwhich ERPs benefit from sharing parameters during joint training. We find that\ntwo pairs of ERPs previously identified in the literature as being related to\neach other benefit from joint training, while several other pairs of ERPs that\nbenefit from joint training are suggestive of potential relationships.\nExtensions of this analysis that further examine what kinds of information in\nthe model embeddings relate to each ERP have the potential to elucidate the\nprocesses involved in human language comprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01628v1",
            "date": "2019-04-02 19:12:35+00:00",
            "title": "Identification, Interpretability, and Bayesian Word Embeddings",
            "authors": [
                "Adam M. Lauretig"
            ],
            "abstract": "Social scientists have recently turned to analyzing text using tools from\nnatural language processing like word embeddings to measure concepts like\nideology, bias, and affinity. However, word embeddings are difficult to use in\nthe regression framework familiar to social scientists: embeddings are are\nneither identified, nor directly interpretable. I offer two advances on\nstandard embedding models to remedy these problems. First, I develop Bayesian\nWord Embeddings with Automatic Relevance Determination priors, relaxing the\nassumption that all embedding dimensions have equal weight. Second, I apply\nwork identifying latent variable models to anchor the dimensions of the\nresulting embeddings, identifying them, and making them interpretable and\nusable in a regression. I then apply this model and anchoring approach to two\ncases, the shift in internationalist rhetoric in the American presidents'\ninaugural addresses, and the relationship between bellicosity in American\nforeign policy decision-makers' deliberations. I find that inaugural addresses\nbecame less internationalist after 1945, which goes against the conventional\nwisdom, and that an increase in bellicosity is associated with an increase in\nhostile actions by the United States, showing that elite deliberations are not\ncheap talk, and helping confirm the validity of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01655v1",
            "date": "2019-04-02 20:27:42+00:00",
            "title": "A frame semantic overview of NLP-based information extraction for cancer-related EHR notes",
            "authors": [
                "Surabhi Datta",
                "Elmer V Bernstam",
                "Kirk Roberts"
            ],
            "abstract": "Objective: There is a lot of information about cancer in Electronic Health\nRecord (EHR) notes that can be useful for biomedical research provided natural\nlanguage processing (NLP) methods are available to extract and structure this\ninformation. In this paper, we present a scoping review of existing clinical\nNLP literature for cancer. Methods: We identified studies describing an NLP\nmethod to extract specific cancer-related information from EHR sources from\nPubMed, Google Scholar, ACL Anthology, and existing reviews. Two exclusion\ncriteria were used in this study. We excluded articles where the extraction\ntechniques used were too broad to be represented as frames and also where very\nlow-level extraction methods were used. 79 articles were included in the final\nreview. We organized this information according to frame semantic principles to\nhelp identify common areas of overlap and potential gaps. Results: Frames were\ncreated from the reviewed articles pertaining to cancer information such as\ncancer diagnosis, tumor description, cancer procedure, breast cancer diagnosis,\nprostate cancer diagnosis and pain in prostate cancer patients. These frames\nincluded both a definition as well as specific frame elements (i.e. extractable\nattributes). We found that cancer diagnosis was the most common frame among the\nreviewed papers (36 out of 79), with recent work focusing on extracting\ninformation related to treatment and breast cancer diagnosis. Conclusion: The\nlist of common frames described in this paper identifies important\ncancer-related information extracted by existing NLP techniques and serves as a\nuseful resource for future researchers requiring cancer information extracted\nfrom EHR notes. We also argue, due to the heavy duplication of cancer NLP\nsystems, that a general purpose resource of annotated cancer frames and\ncorresponding NLP tools would be valuable.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1904.01689v1",
            "date": "2019-04-02 22:23:05+00:00",
            "title": "The Tower of Babel Meets Web 2.0: User-Generated Content and its Applications in a Multilingual Context",
            "authors": [
                "B. Hecht",
                "D. Gergle"
            ],
            "abstract": "This study explores language's fragmenting effect on user-generated content\nby examining the diversity of knowledge representations across 25 different\nWikipedia language editions. This diversity is measured at two levels: the\nconcepts that are included in each edition and the ways in which these concepts\nare described. We demonstrate that the diversity present is greater than has\nbeen presumed in the literature and has a significant influence on applications\nthat use Wikipedia as a source of world knowledge. We close by explicating how\nknowledge diversity can be beneficially leveraged to create \"culturally-aware\napplications\" and \"hyperlingual applications\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01873v1",
            "date": "2019-04-03 09:27:57+00:00",
            "title": "Modeling Vocabulary for Big Code Machine Learning",
            "authors": [
                "Hlib Babii",
                "Andrea Janes",
                "Romain Robbes"
            ],
            "abstract": "When building machine learning models that operate on source code, several\ndecisions have to be made to model source-code vocabulary. These decisions can\nhave a large impact: some can lead to not being able to train models at all,\nothers significantly affect performance, particularly for Neural Language\nModels. Yet, these decisions are not often fully described. This paper lists\nimportant modeling choices for source code vocabulary, and explores their\nimpact on the resulting vocabulary on a large-scale corpus of 14,436 projects.\nWe show that a subset of decisions have decisive characteristics, allowing to\ntrain accurate Neural Language Models quickly on a large corpus of 10,106\nprojects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01989v1",
            "date": "2019-04-03 13:08:12+00:00",
            "title": "Subword-Level Language Identification for Intra-Word Code-Switching",
            "authors": [
                "Manuel Mager",
                "\u00d6zlem \u00c7etino\u011flu",
                "Katharina Kann"
            ],
            "abstract": "Language identification for code-switching (CS), the phenomenon of\nalternating between two or more languages in conversations, has traditionally\nbeen approached under the assumption of a single language per token. However,\nif at least one language is morphologically rich, a large number of words can\nbe composed of morphemes from more than one language (intra-word CS). In this\npaper, we extend the language identification task to the subword-level, such\nthat it includes splitting mixed words while tagging each part with a language\nID. We further propose a model for this task, which is based on a segmental\nrecurrent neural network. In experiments on a new Spanish--Wixarika dataset and\non an adapted German--Turkish dataset, our proposed model performs slightly\nbetter than or roughly on par with our best baseline, respectively. Considering\nonly mixed words, however, it strongly outperforms all baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02032v1",
            "date": "2019-04-03 14:36:49+00:00",
            "title": "OpBerg: Discovering causal sentences using optimal alignments",
            "authors": [
                "Justin Wood",
                "Nicholas J. Matiasz",
                "Alcino J. Silva",
                "William Hsu",
                "Alexej Abyzov",
                "Wei Wang"
            ],
            "abstract": "The biological literature is rich with sentences that describe causal\nrelations. Methods that automatically extract such sentences can help\nbiologists to synthesize the literature and even discover latent relations that\nhad not been articulated explicitly. Current methods for extracting causal\nsentences are based on either machine learning or a predefined database of\ncausal terms. Machine learning approaches require a large set of labeled\ntraining data and can be susceptible to noise. Methods based on predefined\ndatabases are limited by the quality of their curation and are unable to\ncapture new concepts or mistakes in the input. We address these challenges by\nadapting and improving a method designed for a seemingly unrelated problem:\nfinding alignments between genomic sequences. This paper presents a novel and\noutperforming method for extracting causal relations from text by aligning the\npart-of-speech representations of an input set with that of known causal\nsentences. Our experiments show that when applied to the task of finding causal\nsentences in biological literature, our method improves on the accuracy of\nother methods in a computationally efficient manner.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.02037v1",
            "date": "2019-04-03 14:46:44+00:00",
            "title": "Automated Fact Checking in the News Room",
            "authors": [
                "Sebasti\u00e3o Miranda",
                "David Nogueira",
                "Afonso Mendes",
                "Andreas Vlachos",
                "Andrew Secker",
                "Rebecca Garrett",
                "Jeff Mitchel",
                "Zita Marinho"
            ],
            "abstract": "Fact checking is an essential task in journalism; its importance has been\nhighlighted due to recently increased concerns and efforts in combating\nmisinformation. In this paper, we present an automated fact-checking platform\nwhich given a claim, it retrieves relevant textual evidence from a document\ncollection, predicts whether each piece of evidence supports or refutes the\nclaim, and returns a final verdict. We describe the architecture of the system\nand the user interface, focusing on the choices made to improve its\nuser-friendliness and transparency. We conduct a user study of the\nfact-checking platform in a journalistic setting: we integrated it with a\ncollection of news articles and provide an evaluation of the platform using\nfeedback from journalists in their workflow. We found that the predictions of\nour platform were correct 58\\% of the time, and 59\\% of the returned evidence\nwas relevant.",
            "journal ref": "WEBCONF 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02099v3",
            "date": "2019-04-03 16:52:55+00:00",
            "title": "75 Languages, 1 Model: Parsing Universal Dependencies Universally",
            "authors": [
                "Dan Kondratyuk",
                "Milan Straka"
            ],
            "abstract": "We present UDify, a multilingual multi-task model capable of accurately\npredicting universal part-of-speech, morphological features, lemmas, and\ndependency trees simultaneously for all 124 Universal Dependencies treebanks\nacross 75 languages. By leveraging a multilingual BERT self-attention model\npretrained on 104 languages, we found that fine-tuning it on all datasets\nconcatenated together with simple softmax classifiers for each UD task can\nresult in state-of-the-art UPOS, UFeats, Lemmas, UAS, and LAS scores, without\nrequiring any recurrent or language-specific components. We evaluate UDify for\nmultilingual learning, showing that low-resource languages benefit the most\nfrom cross-linguistic annotations. We also evaluate for zero-shot learning,\nwith results suggesting that multilingual training provides strong UD\npredictions even for languages that neither UDify nor BERT have ever been\ntrained on. Code for UDify is available at\nhttps://github.com/hyperparticle/udify.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02210v1",
            "date": "2019-04-03 19:28:53+00:00",
            "title": "Massively Multilingual Adversarial Speech Recognition",
            "authors": [
                "Oliver Adams",
                "Matthew Wiesner",
                "Shinji Watanabe",
                "David Yarowsky"
            ],
            "abstract": "We report on adaptation of multilingual end-to-end speech recognition models\ntrained on as many as 100 languages. Our findings shed light on the relative\nimportance of similarity between the target and pretraining languages along the\ndimensions of phonetics, phonology, language family, geographical location, and\northography. In this context, experiments demonstrate the effectiveness of two\nadditional pretraining objectives in encouraging language-independent encoder\nrepresentations: a context-independent phoneme objective paired with a\nlanguage-adversarial classification objective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02343v3",
            "date": "2019-04-04 04:36:11+00:00",
            "title": "Density Matching for Bilingual Word Embedding",
            "authors": [
                "Chunting Zhou",
                "Xuezhe Ma",
                "Di Wang",
                "Graham Neubig"
            ],
            "abstract": "Recent approaches to cross-lingual word embedding have generally been based\non linear transformations between the sets of embedding vectors in the two\nlanguages. In this paper, we propose an approach that instead expresses the two\nmonolingual embedding spaces as probability densities defined by a Gaussian\nmixture model, and matches the two densities using a method called normalizing\nflow. The method requires no explicit supervision, and can be learned with only\na seed dictionary of words that have identical strings. We argue that this\nformulation has several intuitively attractive properties, particularly with\nthe respect to improving robustness and generalization to mappings between\ndifficult language pairs or word pairs. On a benchmark data set of bilingual\nlexicon induction and cross-lingual word similarity, our approach can achieve\ncompetitive or superior performance compared to state-of-the-art published\nresults, with particularly strong results being found on etymologically distant\nand/or morphologically rich languages.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02399v4",
            "date": "2019-04-04 08:13:42+00:00",
            "title": "Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling",
            "authors": [
                "Prince Zizhuang Wang",
                "William Yang Wang"
            ],
            "abstract": "Recurrent Variational Autoencoder has been widely used for language modeling\nand text generation tasks. These models often face a difficult optimization\nproblem, also known as the Kullback-Leibler (KL) term vanishing issue, where\nthe posterior easily collapses to the prior, and the model will ignore latent\ncodes in generative tasks. To address this problem, we introduce an improved\nWasserstein Variational Autoencoder (WAE) with Riemannian Normalizing Flow\n(RNF) for text modeling. The RNF transforms a latent variable into a space that\nrespects the geometric characteristics of input space, which makes posterior\nimpossible to collapse to the non-informative prior. The Wasserstein objective\nminimizes the distance between the marginal distribution and the prior directly\nand therefore does not force the posterior to match the prior. Empirical\nexperiments show that our model avoids KL vanishing over a range of datasets\nand has better performances in tasks such as language modeling, likelihood\napproximation, and text generation. Through a series of experiments and\nanalysis over latent space, we show that our model learns latent distributions\nthat respect latent space geometry and is able to generate sentences that are\nmore diverse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02496v2",
            "date": "2019-04-04 11:45:52+00:00",
            "title": "Multi-Context Term Embeddings: the Use Case of Corpus-based Term Set Expansion",
            "authors": [
                "Jonathan Mamou",
                "Oren Pereg",
                "Moshe Wasserblat",
                "Ido Dagan"
            ],
            "abstract": "In this paper, we present a novel algorithm that combines multi-context term\nembeddings using a neural classifier and we test this approach on the use case\nof corpus-based term set expansion. In addition, we present a novel and unique\ndataset for intrinsic evaluation of corpus-based term set expansion algorithms.\nWe show that, over this dataset, our algorithm provides up to 5 mean average\nprecision points over the best baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02633v2",
            "date": "2019-04-04 16:04:30+00:00",
            "title": "Clinically Accurate Chest X-Ray Report Generation",
            "authors": [
                "Guanxiong Liu",
                "Tzu-Ming Harry Hsu",
                "Matthew McDermott",
                "Willie Boag",
                "Wei-Hung Weng",
                "Peter Szolovits",
                "Marzyeh Ghassemi"
            ],
            "abstract": "The automatic generation of radiology reports given medical radiographs has\nsignificant potential to operationally and improve clinical patient care. A\nnumber of prior works have focused on this problem, employing advanced methods\nfrom computer vision and natural language generation to produce readable\nreports. However, these works often fail to account for the particular nuances\nof the radiology domain, and, in particular, the critical importance of\nclinical accuracy in the resulting generated reports. In this work, we present\na domain-aware automatic chest X-ray radiology report generation system which\nfirst predicts what topics will be discussed in the report, then conditionally\ngenerates sentences corresponding to these topics. The resulting system is\nfine-tuned using reinforcement learning, considering both readability and\nclinical accuracy, as assessed by the proposed Clinically Coherent Reward. We\nverify this system on two datasets, Open-I and MIMIC-CXR, and demonstrate that\nour model offers marked improvements on both language generation metrics and\nCheXpert assessed accuracy over a variety of competitive baselines.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.02734v1",
            "date": "2019-04-04 18:14:23+00:00",
            "title": "Neural Models of the Psychosemantics of `Most'",
            "authors": [
                "Lewis O'Sullivan",
                "Shane Steinert-Threlkeld"
            ],
            "abstract": "How are the meanings of linguistic expressions related to their use in\nconcrete cognitive tasks? Visual identification tasks show human speakers can\nexhibit considerable variation in their understanding, representation and\nverification of certain quantifiers. This paper initiates an investigation into\nneural models of these psycho-semantic tasks. We trained two types of network\n-- a convolutional neural network (CNN) model and a recurrent model of visual\nattention (RAM) -- on the \"most\" verification task from \\citet{Pietroski2009},\nmanipulating the visual scene and novel notions of task duration. Our results\nqualitatively mirror certain features of human performance (such as sensitivity\nto the ratio of set sizes, indicating a reliance on approximate number) while\ndiffering in interesting ways (such as exhibiting a subtly different pattern\nfor the effect of image type). We conclude by discussing the prospects for\nusing neural models as cognitive models of this and other psychosemantic tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02839v1",
            "date": "2019-04-05 01:03:31+00:00",
            "title": "Combining Sentiment Lexica with a Multi-View Variational Autoencoder",
            "authors": [
                "Alexander Hoyle",
                "Lawrence Wolf-Sonkin",
                "Hanna Wallach",
                "Ryan Cotterell",
                "Isabelle Augenstein"
            ],
            "abstract": "When assigning quantitative labels to a dataset, different methodologies may\nrely on different scales. In particular, when assigning polarities to words in\na sentiment lexicon, annotators may use binary, categorical, or continuous\nlabels. Naturally, it is of interest to unify these labels from disparate\nscales to both achieve maximal coverage over words and to create a single, more\nrobust sentiment lexicon while retaining scale coherence. We introduce a\ngenerative model of sentiment lexica to combine disparate scales into a common\nlatent representation. We realize this model with a novel multi-view\nvariational autoencoder (VAE), called SentiVAE. We evaluate our approach via a\ndownstream text classification task involving nine English-Language sentiment\nanalysis datasets; our representation outperforms six individual sentiment\nlexica, as well as a straightforward combination thereof.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02856v1",
            "date": "2019-04-05 03:17:00+00:00",
            "title": "Graph Pattern Entity Ranking Model for Knowledge Graph Completion",
            "authors": [
                "Takuma Ebisu",
                "Ryutaro Ichise"
            ],
            "abstract": "Knowledge graphs have evolved rapidly in recent years and their usefulness\nhas been demonstrated in many artificial intelligence tasks. However, knowledge\ngraphs often have lots of missing facts. To solve this problem, many knowledge\ngraph embedding models have been developed to populate knowledge graphs and\nthese have shown outstanding performance. However, knowledge graph embedding\nmodels are so-called black boxes, and the user does not know how the\ninformation in a knowledge graph is processed and the models can be difficult\nto interpret. In this paper, we utilize graph patterns in a knowledge graph to\novercome such problems. Our proposed model, the {\\it graph pattern entity\nranking model} (GRank), constructs an entity ranking system for each graph\npattern and evaluates them using a ranking measure. By doing so, we can find\ngraph patterns which are useful for predicting facts. Then, we perform link\nprediction tasks on standard datasets to evaluate our GRank method. We show\nthat our approach outperforms other state-of-the-art approaches such as ComplEx\nand TorusE for standard metrics such as HITS@{\\it n} and MRR. Moreover, our\nmodel is easily interpretable because the output facts are described by graph\npatterns.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1904.03092v1",
            "date": "2019-04-05 14:40:22+00:00",
            "title": "Modeling Recurrence for Transformer",
            "authors": [
                "Jie Hao",
                "Xing Wang",
                "Baosong Yang",
                "Longyue Wang",
                "Jinfeng Zhang",
                "Zhaopeng Tu"
            ],
            "abstract": "Recently, the Transformer model that is based solely on attention mechanisms,\nhas advanced the state-of-the-art on various machine translation tasks.\nHowever, recent studies reveal that the lack of recurrence hinders its further\nimprovement of translation capacity. In response to this problem, we propose to\ndirectly model recurrence for Transformer with an additional recurrence\nencoder. In addition to the standard recurrent neural network, we introduce a\nnovel attentive recurrent network to leverage the strengths of both attention\nand recurrent networks. Experimental results on the widely-used WMT14\nEnglish-German and WMT17 Chinese-English translation tasks demonstrate the\neffectiveness of the proposed approach. Our studies also reveal that the\nproposed model benefits from a short-cut that bridges the source and target\nsequences with a single recurrent layer, which outperforms its deep\ncounterpart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03100v1",
            "date": "2019-04-05 14:52:28+00:00",
            "title": "Information Aggregation for Multi-Head Attention with Routing-by-Agreement",
            "authors": [
                "Jian Li",
                "Baosong Yang",
                "Zi-Yi Dou",
                "Xing Wang",
                "Michael R. Lyu",
                "Zhaopeng Tu"
            ],
            "abstract": "Multi-head attention is appealing for its ability to jointly extract\ndifferent types of information from multiple representation subspaces.\nConcerning the information aggregation, a common practice is to use a\nconcatenation followed by a linear transformation, which may not fully exploit\nthe expressiveness of multi-head attention. In this work, we propose to improve\nthe information aggregation for multi-head attention with a more powerful\nrouting-by-agreement algorithm. Specifically, the routing algorithm iteratively\nupdates the proportion of how much a part (i.e. the distinct information\nlearned from a specific subspace) should be assigned to a whole (i.e. the final\noutput representation), based on the agreement between parts and wholes.\nExperimental results on linguistic probing tasks and machine translation tasks\nprove the superiority of the advanced information aggregation over the standard\nlinear transformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03107v1",
            "date": "2019-04-05 15:02:26+00:00",
            "title": "Convolutional Self-Attention Networks",
            "authors": [
                "Baosong Yang",
                "Longyue Wang",
                "Derek Wong",
                "Lidia S. Chao",
                "Zhaopeng Tu"
            ],
            "abstract": "Self-attention networks (SANs) have drawn increasing interest due to their\nhigh parallelization in computation and flexibility in modeling dependencies.\nSANs can be further enhanced with multi-head attention by allowing the model to\nattend to information from different representation subspaces. In this work, we\npropose novel convolutional self-attention networks, which offer SANs the\nabilities to 1) strengthen dependencies among neighboring elements, and 2)\nmodel the interaction between features extracted by multiple attention heads.\nExperimental results of machine translation on different language pairs and\nmodel settings show that our approach outperforms both the strong Transformer\nbaseline and other existing models on enhancing the locality of SANs. Comparing\nwith prior studies, the proposed model is parameter free in terms of\nintroducing no more parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03223v1",
            "date": "2019-04-05 18:31:06+00:00",
            "title": "NELEC at SemEval-2019 Task 3: Think Twice Before Going Deep",
            "authors": [
                "Parag Agrawal",
                "Anshuman Suri"
            ],
            "abstract": "Existing Machine Learning techniques yield close to human performance on\ntext-based classification tasks. However, the presence of multi-modal noise in\nchat data such as emoticons, slang, spelling mistakes, code-mixed data, etc.\nmakes existing deep-learning solutions perform poorly. The inability of\ndeep-learning systems to robustly capture these covariates puts a cap on their\nperformance. We propose NELEC: Neural and Lexical Combiner, a system which\nelegantly combines textual and deep-learning based methods for sentiment\nclassification. We evaluate our system as part of the third task of 'Contextual\nEmotion Detection in Text' as part of SemEval-2019. Our system performs\nsignificantly better than the baseline, as well as our deep-learning model\nbenchmarks. It achieved a micro-averaged F1 score of 0.7765, ranking 3rd on the\ntest-set leader-board. Our code is available at\nhttps://github.com/iamgroot42/nelec",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03244v1",
            "date": "2019-04-05 19:22:47+00:00",
            "title": "An Analysis of Attention over Clinical Notes for Predictive Tasks",
            "authors": [
                "Sarthak Jain",
                "Ramin Mohammadi",
                "Byron C. Wallace"
            ],
            "abstract": "The shift to electronic medical records (EMRs) has engendered research into\nmachine learning and natural language technologies to analyze patient records,\nand to predict from these clinical outcomes of interest. Two observations\nmotivate our aims here. First, unstructured notes contained within EMR often\ncontain key information, and hence should be exploited by models. Second, while\nstrong predictive performance is important, interpretability of models is\nperhaps equally so for applications in this domain. Together, these points\nsuggest that neural models for EMR may benefit from incorporation of attention\nover notes, which one may hope will both yield performance gains and afford\ntransparency in predictions. In this work we perform experiments to explore\nthis question using two EMR corpora and four different predictive tasks, that:\n(i) inclusion of attention mechanisms is critical for neural encoder modules\nthat operate over notes fields in order to yield competitive performance, but,\n(ii) unfortunately, while these boost predictive performance, it is decidedly\nless clear whether they provide meaningful support for predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03371v2",
            "date": "2019-04-06 06:06:11+00:00",
            "title": "Evaluating Coherence in Dialogue Systems using Entailment",
            "authors": [
                "Nouha Dziri",
                "Ehsan Kamalloo",
                "Kory W. Mathewson",
                "Osmar Zaiane"
            ],
            "abstract": "Evaluating open-domain dialogue systems is difficult due to the diversity of\npossible correct answers. Automatic metrics such as BLEU correlate weakly with\nhuman annotations, resulting in a significant bias across different models and\ndatasets. Some researchers resort to human judgment experimentation for\nassessing response quality, which is expensive, time consuming, and not\nscalable. Moreover, judges tend to evaluate a small number of dialogues,\nmeaning that minor differences in evaluation configuration may lead to\ndissimilar results. In this paper, we present interpretable metrics for\nevaluating topic coherence by making use of distributed sentence\nrepresentations. Furthermore, we introduce calculable approximations of human\njudgment based on conversational coherence by adopting state-of-the-art\nentailment techniques. Results show that our metrics can be used as a surrogate\nfor human judgment, making it easy to evaluate dialogue systems on large-scale\ndatasets and allowing an unbiased estimate for the quality of the responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03396v2",
            "date": "2019-04-06 09:25:32+00:00",
            "title": "Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation",
            "authors": [
                "Amit Moryossef",
                "Yoav Goldberg",
                "Ido Dagan"
            ],
            "abstract": "Data-to-text generation can be conceptually divided into two parts: ordering\nand structuring the information (planning), and generating fluent language\ndescribing the information (realization). Modern neural generation systems\nconflate these two steps into a single end-to-end differentiable system. We\npropose to split the generation process into a symbolic text-planning stage\nthat is faithful to the input, followed by a neural generation stage that\nfocuses only on realization. For training a plan-to-text generator, we present\na method for matching reference texts to their corresponding text plans. For\ninference time, we describe a method for selecting high-quality text plans for\nnew inputs. We implement and evaluate our approach on the WebNLG benchmark. Our\nresults demonstrate that decoupling text planning from neural realization\nindeed improves the system's reliability and adequacy while maintaining fluent\noutput. We observe improvements both in BLEU scores and in manual evaluations.\nAnother benefit of our approach is the ability to output diverse realizations\nof the same input, paving the way to explicit control over the generated text\nstructure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03713v1",
            "date": "2019-04-07 19:01:32+00:00",
            "title": "AI Meets Austen: Towards Human-Robot Discussions of Literary Metaphor",
            "authors": [
                "Natalie Parde",
                "Rodney D. Nielsen"
            ],
            "abstract": "Artificial intelligence is revolutionizing formal education, fueled by\ninnovations in learning assessment, content generation, and instructional\ndelivery. Informal, lifelong learning settings have been the subject of less\nattention. We provide a proof-of-concept for an embodied book discussion\ncompanion, designed to stimulate conversations with readers about particularly\ncreative metaphors in fiction literature. We collect ratings from 26\nparticipants, each of whom discuss Jane Austen's \"Pride and Prejudice\" with the\nrobot across one or more sessions, and find that participants rate their\ninteractions highly. This suggests that companion robots could be an\ninteresting entryway for the promotion of lifelong learning and cognitive\nexercise in future applications.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1904.03736v2",
            "date": "2019-04-07 20:28:47+00:00",
            "title": "Unsupervised Dialog Structure Learning",
            "authors": [
                "Weiyan Shi",
                "Tiancheng Zhao",
                "Zhou Yu"
            ],
            "abstract": "Learning a shared dialog structure from a set of task-oriented dialogs is an\nimportant challenge in computational linguistics. The learned dialog structure\ncan shed light on how to analyze human dialogs, and more importantly contribute\nto the design and evaluation of dialog systems. We propose to extract dialog\nstructures using a modified VRNN model with discrete latent vectors. Different\nfrom existing HMM-based models, our model is based on variational-autoencoder\n(VAE). Such model is able to capture more dynamics in dialogs beyond the\nsurface forms of the language. We find that qualitatively, our method extracts\nmeaningful dialog structure, and quantitatively, outperforms previous models on\nthe ability to predict unseen data. We further evaluate the model's\neffectiveness in a downstream task, the dialog system building task.\nExperiments show that, by integrating the learned dialog structure into the\nreward function design, the model converges faster and to a better outcome in a\nreinforcement learning setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03746v6",
            "date": "2019-04-07 21:14:43+00:00",
            "title": "Unsupervised Recurrent Neural Network Grammars",
            "authors": [
                "Yoon Kim",
                "Alexander M. Rush",
                "Lei Yu",
                "Adhiguna Kuncoro",
                "Chris Dyer",
                "G\u00e1bor Melis"
            ],
            "abstract": "Recurrent neural network grammars (RNNG) are generative models of language\nwhich jointly model syntax and surface structure by incrementally generating a\nsyntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs\nachieve strong language modeling and parsing performance, but require an\nannotated corpus of parse trees. In this work, we experiment with unsupervised\nlearning of RNNGs. Since directly marginalizing over the space of latent trees\nis intractable, we instead apply amortized variational inference. To maximize\nthe evidence lower bound, we develop an inference network parameterized as a\nneural CRF constituency parser. On language modeling, unsupervised RNNGs\nperform as well their supervised counterparts on benchmarks in English and\nChinese. On constituency grammar induction, they are competitive with recent\nneural language models that induce tree structures from words through attention\nmechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03969v2",
            "date": "2019-04-08 11:36:53+00:00",
            "title": "Issue Framing in Online Discussion Fora",
            "authors": [
                "Mareike Hartmann",
                "Tallulah Jansen",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "In online discussion fora, speakers often make arguments for or against\nsomething, say birth control, by highlighting certain aspects of the topic. In\nsocial science, this is referred to as issue framing. In this paper, we\nintroduce a new issue frame annotated corpus of online discussions. We explore\nto what extent models trained to detect issue frames in newswire and social\nmedia can be transferred to the domain of discussion fora, using a combination\nof multi-task and adversarial training, assuming only unlabeled training data\nin the target domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04063v1",
            "date": "2019-04-05 15:15:45+00:00",
            "title": "Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop",
            "authors": [
                "Afra Alishahi",
                "Grzegorz Chrupa\u0142a",
                "Tal Linzen"
            ],
            "abstract": "The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques\nspecifically developed for analyzing and understanding the inner-workings and\nrepresentations acquired by neural models of language. Approaches included:\nsystematic manipulation of input to neural networks and investigating the\nimpact on their performance, testing whether interpretable knowledge can be\ndecoded from intermediate representations acquired by neural networks,\nproposing modifications to neural network architectures to make their knowledge\nstate or generated output more explainable, and examining the performance of\nnetworks on simplified or formal languages. Here we review a number of\nrepresentative studies in each category.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04079v2",
            "date": "2019-04-04 04:48:07+00:00",
            "title": "Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation",
            "authors": [
                "Weijia Xu",
                "Xing Niu",
                "Marine Carpuat"
            ],
            "abstract": "Despite some empirical success at correcting exposure bias in machine\ntranslation, scheduled sampling algorithms suffer from a major drawback: they\nincorrectly assume that words in the reference translations and in sampled\nsequences are aligned at each time step. Our new differentiable sampling\nalgorithm addresses this issue by optimizing the probability that the reference\ncan be aligned with the sampled output, based on a soft alignment predicted by\nthe model itself. As a result, the output distribution at each time step is\nevaluated with respect to the whole predicted sequence. Experiments on IWSLT\ntranslation tasks show that our approach improves BLEU compared to maximum\nlikelihood and scheduled sampling baselines. In addition, our approach is\nsimpler to train with no need for sampling schedule and yields models that\nachieve larger improvements with smaller beam sizes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04365v4",
            "date": "2019-04-08 21:25:50+00:00",
            "title": "CODAH: An Adversarially Authored Question-Answer Dataset for Common Sense",
            "authors": [
                "Michael Chen",
                "Mike D'Arcy",
                "Alisa Liu",
                "Jared Fernandez",
                "Doug Downey"
            ],
            "abstract": "Commonsense reasoning is a critical AI capability, but it is difficult to\nconstruct challenging datasets that test common sense. Recent neural question\nanswering systems, based on large pre-trained models of language, have already\nachieved near-human-level performance on commonsense knowledge benchmarks.\nThese systems do not possess human-level common sense, but are able to exploit\nlimitations of the datasets to achieve human-level scores.\n  We introduce the CODAH dataset, an adversarially-constructed evaluation\ndataset for testing common sense. CODAH forms a challenging extension to the\nrecently-proposed SWAG dataset, which tests commonsense knowledge using\nsentence-completion questions that describe situations observed in video. To\nproduce a more difficult dataset, we introduce a novel procedure for question\nacquisition in which workers author questions designed to target weaknesses of\nstate-of-the-art neural question answering systems. Workers are rewarded for\nsubmissions that models fail to answer correctly both before and after\nfine-tuning (in cross-validation). We create 2.8k questions via this procedure\nand evaluate the performance of multiple state-of-the-art question answering\nsystems on our dataset. We observe a significant gap between human performance,\nwhich is 95.3%, and the performance of the best baseline accuracy of 67.5% by\nthe BERT-Large model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04388v1",
            "date": "2019-04-08 22:47:37+00:00",
            "title": "Giving Attention to the Unexpected: Using Prosody Innovations in Disfluency Detection",
            "authors": [
                "Vicky Zayats",
                "Mari Ostendorf"
            ],
            "abstract": "Disfluencies in spontaneous speech are known to be associated with prosodic\ndisruptions. However, most algorithms for disfluency detection use only word\ntranscripts. Integrating prosodic cues has proved difficult because of the many\nsources of variability affecting the acoustic correlates. This paper introduces\na new approach to extracting acoustic-prosodic cues using text-based\ndistributional prediction of acoustic cues to derive vector z-score features\n(innovations). We explore both early and late fusion techniques for integrating\ntext and prosody, showing gains over a high-accuracy text-only model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04460v1",
            "date": "2019-04-09 04:21:52+00:00",
            "title": "Attention-based Multi-instance Neural Network for Medical Diagnosis from Incomplete and Low Quality Data",
            "authors": [
                "Zeyuan Wang",
                "Josiah Poon",
                "Shiding Sun",
                "Simon Poon"
            ],
            "abstract": "One way to extract patterns from clinical records is to consider each patient\nrecord as a bag with various number of instances in the form of symptoms.\nMedical diagnosis is to discover informative ones first and then map them to\none or more diseases. In many cases, patients are represented as vectors in\nsome feature space and a classifier is applied after to generate diagnosis\nresults. However, in many real-world cases, data is often of low-quality due to\na variety of reasons, such as data consistency, integrity, completeness,\naccuracy, etc. In this paper, we propose a novel approach, attention based\nmulti-instance neural network (AMI-Net), to make the single disease\nclassification only based on the existing and valid information in the\nreal-world outpatient records. In the context of a patient, it takes a bag of\ninstances as input and output the bag label directly in end-to-end way.\nEmbedding layer is adopted at the beginning, mapping instances into an\nembedding space which represents the individual patient condition. The\ncorrelations among instances and their importance for the final classification\nare captured by multi-head attention transformer, instance-level multi-instance\npooling and bag-level multi-instance pooling. The proposed approach was test on\ntwo non-standardized and highly imbalanced datasets, one in the Traditional\nChinese Medicine (TCM) domain and the other in the Western Medicine (WM)\ndomain. Our preliminary results show that the proposed approach outperforms all\nbaselines results by a significant margin.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.04697v2",
            "date": "2019-04-09 14:25:17+00:00",
            "title": "A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing",
            "authors": [
                "Hang Yan",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Chinese word segmentation and dependency parsing are two fundamental tasks\nfor Chinese natural language processing. The dependency parsing is defined on\nword-level. Therefore word segmentation is the precondition of dependency\nparsing, which makes dependency parsing suffer from error propagation and\nunable to directly make use of the character-level pre-trained language model\n(such as BERT). In this paper, we propose a graph-based model to integrate\nChinese word segmentation and dependency parsing. Different from previous\ntransition-based joint models, our proposed model is more concise, which\nresults in fewer efforts of feature engineering. Our graph-based joint model\nachieves better performance than previous joint models and state-of-the-art\nresults in both Chinese word segmentation and dependency parsing. Besides, when\nBERT is combined, our model can substantially reduce the performance gap of\ndependency parsing between joint models and gold-segmented word-based models.\nOur code is publicly available at https://github.com/fastnlp/JointCwsParser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04733v3",
            "date": "2019-04-09 15:33:59+00:00",
            "title": "Seq2Biseq: Bidirectional Output-wise Recurrent Neural Networks for Sequence Modelling",
            "authors": [
                "Marco Dinarelli",
                "Lo\u00efc Grobol"
            ],
            "abstract": "During the last couple of years, Recurrent Neural Networks (RNN) have reached\nstate-of-the-art performances on most of the sequence modelling problems. In\nparticular, the \"sequence to sequence\" model and the neural CRF have proved to\nbe very effective in this domain. In this article, we propose a new RNN\narchitecture for sequence labelling, leveraging gated recurrent layers to take\narbitrarily long contexts into account, and using two decoders operating\nforward and backward. We compare several variants of the proposed solution and\ntheir performances to the state-of-the-art. Most of our results are better than\nthe state-of-the-art or very close to it and thanks to the use of recent\ntechnologies, our architecture can scale on corpora larger than those used in\nthis work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04742v2",
            "date": "2019-04-09 15:42:08+00:00",
            "title": "Bilingual-GAN: A Step Towards Parallel Text Generation",
            "authors": [
                "Ahmad Rashid",
                "Alan Do-Omri",
                "Md. Akmal Haidar",
                "Qun Liu",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "Latent space based GAN methods and attention based sequence to sequence\nmodels have achieved impressive results in text generation and unsupervised\nmachine translation respectively. Leveraging the two domains, we propose an\nadversarial latent space based model capable of generating parallel sentences\nin two languages concurrently and translating bidirectionally. The bilingual\ngeneration goal is achieved by sampling from the latent space that is shared\nbetween both languages. First two denoising autoencoders are trained, with\nshared encoders and back-translation to enforce a shared latent state between\nthe two languages. The decoder is shared for the two translation directions.\nNext, a GAN is trained to generate synthetic \"code\" mimicking the languages'\nshared latent space. This code is then fed into the decoder to generate text in\neither language. We perform our experiments on Europarl and Multi30k datasets,\non the English-French language pair, and document our performance using both\nsupervised and unsupervised machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04745v1",
            "date": "2019-04-09 15:51:07+00:00",
            "title": "Cross-Modal Self-Attention Network for Referring Image Segmentation",
            "authors": [
                "Linwei Ye",
                "Mrigank Rochan",
                "Zhi Liu",
                "Yang Wang"
            ],
            "abstract": "We consider the problem of referring image segmentation. Given an input image\nand a natural language expression, the goal is to segment the object referred\nby the language expression in the image. Existing works in this area treat the\nlanguage expression and the input image separately in their representations.\nThey do not sufficiently capture long-range correlations between these two\nmodalities. In this paper, we propose a cross-modal self-attention (CMSA)\nmodule that effectively captures the long-range dependencies between linguistic\nand visual features. Our model can adaptively focus on informative words in the\nreferring expression and important regions in the input image. In addition, we\npropose a gated multi-level fusion module to selectively integrate\nself-attentive cross-modal features corresponding to different levels in the\nimage. This module controls the information flow of features at different\nlevels. We validate the proposed approach on four evaluation datasets. Our\nproposed approach consistently outperforms existing state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.04866v1",
            "date": "2019-04-09 18:53:00+00:00",
            "title": "Characterizing the impact of geometric properties of word embeddings on task performance",
            "authors": [
                "Brendan Whitaker",
                "Denis Newman-Griffis",
                "Aparajita Haldar",
                "Hakan Ferhatosmanoglu",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Analysis of word embedding properties to inform their use in downstream NLP\ntasks has largely been studied by assessing nearest neighbors. However,\ngeometric properties of the continuous feature space contribute directly to the\nuse of embedding features in downstream models, and are largely unexplored. We\nconsider four properties of word embedding geometry, namely: position relative\nto the origin, distribution of features in the vector space, global pairwise\ndistances, and local pairwise distances. We define a sequence of\ntransformations to generate new embeddings that expose subsets of these\nproperties to downstream models and evaluate change in task performance to\nunderstand the contribution of each property to NLP models. We transform\npublicly available pretrained embeddings from three popular toolkits (word2vec,\nGloVe, and FastText) and evaluate on a variety of intrinsic tasks, which model\nlinguistic information in the vector space, and extrinsic tasks, which use\nvectors as input to machine learning models. We find that intrinsic evaluations\nare highly sensitive to absolute position, while extrinsic tasks rely primarily\non local similarity. Our findings suggest that future embedding models and\npost-processing techniques should focus primarily on similarity to nearby\npoints in vector space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05092v2",
            "date": "2019-04-10 09:43:06+00:00",
            "title": "Cross-lingual Visual Verb Sense Disambiguation",
            "authors": [
                "Spandana Gella",
                "Desmond Elliott",
                "Frank Keller"
            ],
            "abstract": "Recent work has shown that visual context improves cross-lingual sense\ndisambiguation for nouns. We extend this line of work to the more challenging\ntask of cross-lingual verb sense disambiguation, introducing the MultiSense\ndataset of 9,504 images annotated with English, German, and Spanish verbs. Each\nimage in MultiSense is annotated with an English verb and its translation in\nGerman or Spanish. We show that cross-lingual verb sense disambiguation models\nbenefit from visual context, compared to unimodal baselines. We also show that\nthe verb sense predicted by our best disambiguation model can improve the\nresults of a text-only machine translation system when used for a multimodal\ntranslation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05276v1",
            "date": "2019-04-10 16:26:51+00:00",
            "title": "Advances in Natural Language Question Answering: A Review",
            "authors": [
                "K. S. D. Ishwari",
                "A. K. R. R. Aneeze",
                "S. Sudheesan",
                "H. J. D. A. Karunaratne",
                "A. Nugaliyadde",
                "Y. Mallawarrachchi"
            ],
            "abstract": "Question Answering has recently received high attention from artificial\nintelligence communities due to the advancements in learning technologies.\nEarly question answering models used rule-based approaches and moved to the\nstatistical approach to address the vastly available information. However,\nstatistical approaches are shown to underperform in handling the dynamic nature\nand the variation of language. Therefore, learning models have shown the\ncapability of handling the dynamic nature and variations in language. Many deep\nlearning methods have been introduced to question answering. Most of the deep\nlearning approaches have shown to achieve higher results compared to machine\nlearning and statistical methods. The dynamic nature of language has profited\nfrom the nonlinear learning in deep learning. This has created prominent\nsuccess and a spike in work on question answering. This paper discusses the\nsuccesses and challenges in question answering question answering systems and\ntechniques that are used in these challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05342v3",
            "date": "2019-04-10 17:53:13+00:00",
            "title": "ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission",
            "authors": [
                "Kexin Huang",
                "Jaan Altosaar",
                "Rajesh Ranganath"
            ],
            "abstract": "Clinical notes contain information about patients that goes beyond structured\ndata like lab values and medications. However, clinical notes have been\nunderused relative to structured data, because notes are high-dimensional and\nsparse. This work develops and evaluates representations of clinical notes\nusing bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers\nhigh-quality relationships between medical concepts as judged by humans.\nClinicalBert outperforms baselines on 30-day hospital readmission prediction\nusing both discharge summaries and the first few days of notes in the intensive\ncare unit. Code and model parameters are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05584v1",
            "date": "2019-04-11 08:56:48+00:00",
            "title": "Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Study",
            "authors": [
                "Jorge A. Balazs",
                "Yutaka Matsuo"
            ],
            "abstract": "In this paper we study how different ways of combining character and\nword-level representations affect the quality of both final word and sentence\nrepresentations. We provide strong empirical evidence that modeling characters\nimproves the learned representations at the word and sentence levels, and that\ndoing so is particularly useful when representing less frequent words. We\nfurther show that a feature-wise sigmoid gating mechanism is a robust method\nfor creating representations that encode semantic similarity, as it performed\nreasonably well in several word similarity datasets. Finally, our findings\nsuggest that properly capturing semantic similarity at the word level does not\nconsistently yield improved performance in downstream sentence-level tasks. Our\ncode is available at https://github.com/jabalazs/gating",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05643v1",
            "date": "2019-04-11 11:46:07+00:00",
            "title": "Political Discussions in Homogeneous and Cross-Cutting Communication Spaces",
            "authors": [
                "Jisun An",
                "Haewoon Kwak",
                "Oliver Posegga",
                "Andreas Jungherr"
            ],
            "abstract": "Online platforms, such as Facebook, Twitter, and Reddit, provide users with a\nrich set of features for sharing and consuming political information,\nexpressing political opinions, and exchanging potentially contrary political\nviews. In such activities, two types of communication spaces naturally emerge:\nthose dominated by exchanges between politically homogeneous users and those\nthat allow and encourage cross-cutting exchanges in politically heterogeneous\ngroups. While research on political talk in online environments abounds, we\nknow surprisingly little about the potentially varying nature of discussions in\npolitically homogeneous spaces as compared to cross-cutting communication\nspaces. To fill this gap, we use Reddit to explore the nature of political\ndiscussions in homogeneous and cross-cutting communication spaces. In\nparticular, we develop an analytical template to study interaction and\nlinguistic patterns within and between politically homogeneous and\nheterogeneous communication spaces. Our analyses reveal different behavioral\npatterns in homogeneous and cross-cutting communications spaces. We discuss\ntheoretical and practical implications in the context of research on political\ntalk online.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1904.05674v1",
            "date": "2019-04-11 13:09:57+00:00",
            "title": "Cross-topic distributional semantic representations via unsupervised mappings",
            "authors": [
                "Eleftheria Briakou",
                "Nikos Athanasiou",
                "Alexandros Potamianos"
            ],
            "abstract": "In traditional Distributional Semantic Models (DSMs) the multiple senses of a\npolysemous word are conflated into a single vector space representation. In\nthis work, we propose a DSM that learns multiple distributional representations\nof a word based on different topics. First, a separate DSM is trained for each\ntopic and then each of the topic-based DSMs is aligned to a common vector\nspace. Our unsupervised mapping approach is motivated by the hypothesis that\nwords preserving their relative distances in different topic semantic\nsub-spaces constitute robust \\textit{semantic anchors} that define the mappings\nbetween them. Aligned cross-topic representations achieve state-of-the-art\nresults for the task of contextual word similarity. Furthermore, evaluation on\nNLP downstream tasks shows that multiple topic-based embeddings outperform\nsingle-prototype models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05780v1",
            "date": "2019-04-10 05:47:15+00:00",
            "title": "Corpora Generation for Grammatical Error Correction",
            "authors": [
                "Jared Lichtarge",
                "Chris Alberti",
                "Shankar Kumar",
                "Noam Shazeer",
                "Niki Parmar",
                "Simon Tong"
            ],
            "abstract": "Grammatical Error Correction (GEC) has been recently modeled using the\nsequence-to-sequence framework. However, unlike sequence transduction problems\nsuch as machine translation, GEC suffers from the lack of plentiful parallel\ndata. We describe two approaches for generating large parallel datasets for GEC\nusing publicly available Wikipedia data. The first method extracts\nsource-target pairs from Wikipedia edit histories with minimal filtration\nheuristics, while the second method introduces noise into Wikipedia sentences\nvia round-trip translation through bridge languages. Both strategies yield\nsimilar sized parallel corpora containing around 4B tokens. We employ an\niterative decoding strategy that is tailored to the loosely supervised nature\nof our constructed corpora. We demonstrate that neural GEC models trained using\neither type of corpora give similar performance. Fine-tuning these models on\nthe Lang-8 corpus and ensembling allows us to surpass the state of the art on\nboth the CoNLL-2014 benchmark and the JFLEG task. We provide systematic\nanalysis that compares the two approaches to data generation and highlights the\neffectiveness of ensembling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05829v1",
            "date": "2019-04-11 16:33:54+00:00",
            "title": "Adapting RNN Sequence Prediction Model to Multi-label Set Prediction",
            "authors": [
                "Kechen Qin",
                "Cheng Li",
                "Virgil Pavlu",
                "Javed A. Aslam"
            ],
            "abstract": "We present an adaptation of RNN sequence models to the problem of multi-label\nclassification for text, where the target is a set of labels, not a sequence.\nPrevious such RNN models define probabilities for sequences but not for sets;\nattempts to obtain a set probability are after-thoughts of the network design,\nincluding pre-specifying the label order, or relating the sequence probability\nto the set probability in ad hoc ways.\n  Our formulation is derived from a principled notion of set probability, as\nthe sum of probabilities of corresponding permutation sequences for the set. We\nprovide a new training objective that maximizes this set probability, and a new\nprediction objective that finds the most probable set on a test document. These\nnew objectives are theoretically appealing because they give the RNN model\nfreedom to discover the best label order, which often is the natural one (but\ndifferent among documents).\n  We develop efficient procedures to tackle the computation difficulties\ninvolved in training and prediction. Experiments on benchmark datasets\ndemonstrate that we outperform state-of-the-art methods for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06038v1",
            "date": "2019-04-12 05:18:35+00:00",
            "title": "Evaluating the Representational Hub of Language and Vision Models",
            "authors": [
                "Ravi Shekhar",
                "Ece Takmaz",
                "Raquel Fern\u00e1ndez",
                "Raffaella Bernardi"
            ],
            "abstract": "The multimodal models used in the emerging field at the intersection of\ncomputational linguistics and computer vision implement the bottom-up\nprocessing of the `Hub and Spoke' architecture proposed in cognitive science to\nrepresent how the brain processes and combines multi-sensory inputs. In\nparticular, the Hub is implemented as a neural network encoder. We investigate\nthe effect on this encoder of various vision-and-language tasks proposed in the\nliterature: visual question answering, visual reference resolution, and\nvisually grounded dialogue. To measure the quality of the representations\nlearned by the encoder, we use two kinds of analyses. First, we evaluate the\nencoder pre-trained on the different vision-and-language tasks on an existing\ndiagnostic task designed to assess multimodal semantic understanding. Second,\nwe carry out a battery of analyses aimed at studying how the encoder merges and\nexploits the two modalities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06475v1",
            "date": "2019-04-13 03:52:56+00:00",
            "title": "Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering",
            "authors": [
                "Bo Chen",
                "Xiaotao Gu",
                "Yufeng Hu",
                "Siliang Tang",
                "Guoping Hu",
                "Yueting Zhuang",
                "Xiang Ren"
            ],
            "abstract": "Recently, distant supervision has gained great success on Fine-grained Entity\nTyping (FET). Despite its efficiency in reducing manual labeling efforts, it\nalso brings the challenge of dealing with false entity type labels, as distant\nsupervision assigns labels in a context agnostic manner. Existing works\nalleviated this issue with partial-label loss, but usually suffer from\nconfirmation bias, which means the classifier fit a pseudo data distribution\ngiven by itself. In this work, we propose to regularize distantly supervised\nmodels with Compact Latent Space Clustering (CLSC) to bypass this problem and\neffectively utilize noisy data yet. Our proposed method first dynamically\nconstructs a similarity graph of different entity mentions; infer the labels of\nnoisy instances via label propagation. Based on the inferred labels, mention\nembeddings are updated accordingly to encourage entity mentions with close\nsemantics to form a compact cluster in the embedding space,thus leading to\nbetter classification performance. Extensive experiments on standard benchmarks\nshow that our CLSC model consistently outperforms state-of-the-art distantly\nsupervised entity typing systems by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06652v1",
            "date": "2019-04-14 08:17:06+00:00",
            "title": "Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering",
            "authors": [
                "Wei Yang",
                "Yuqing Xie",
                "Luchen Tan",
                "Kun Xiong",
                "Ming Li",
                "Jimmy Lin"
            ],
            "abstract": "Recently, a simple combination of passage retrieval using off-the-shelf IR\ntechniques and a BERT reader was found to be very effective for question\nanswering directly on Wikipedia, yielding a large improvement over the previous\nstate of the art on a standard benchmark dataset. In this paper, we present a\ndata augmentation technique using distant supervision that exploits positive as\nwell as negative examples. We apply a stage-wise approach to fine tuning BERT\non multiple datasets, starting with data that is \"furthest\" from the test data\nand ending with the \"closest\". Experimental results show large gains in\neffectiveness over previous approaches on English QA datasets, and we establish\nnew baselines on two recent Chinese QA datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06707v4",
            "date": "2019-04-14 15:26:52+00:00",
            "title": "Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Pretraining deep neural network architectures with a language modeling\nobjective has brought large improvements for many natural language processing\ntasks. Exemplified by BERT, a recently proposed such architecture, we\ndemonstrate that despite being trained on huge amounts of data, deep language\nmodels still struggle to understand rare words. To fix this problem, we adapt\nAttentive Mimicking, a method that was designed to explicitly learn embeddings\nfor rare words, to deep language models. In order to make this possible, we\nintroduce one-token approximation, a procedure that enables us to use Attentive\nMimicking even when the underlying language model uses subword-based\ntokenization, i.e., it does not assign embeddings to all words. To evaluate our\nmethod, we create a novel dataset that tests the ability of language models to\ncapture semantic properties of words without any task-specific fine-tuning.\nUsing this dataset, we show that adding our adapted version of Attentive\nMimicking to BERT does indeed substantially improve its understanding of rare\nwords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06725v1",
            "date": "2019-04-14 17:01:26+00:00",
            "title": "Distributed representation of multi-sense words: A loss-driven approach",
            "authors": [
                "Saurav Manchanda",
                "George Karypis"
            ],
            "abstract": "Word2Vec's Skip Gram model is the current state-of-the-art approach for\nestimating the distributed representation of words. However, it assumes a\nsingle vector per word, which is not well-suited for representing words that\nhave multiple senses. This work presents LDMI, a new model for estimating\ndistributional representations of words. LDMI relies on the idea that, if a\nword carries multiple senses, then having a different representation for each\nof its senses should lead to a lower loss associated with predicting its\nco-occurring words, as opposed to the case when a single vector representation\nis used for all the senses. After identifying the multi-sense words, LDMI\nclusters the occurrences of these words to assign a sense to each occurrence.\nExperiments on the contextual word similarity task show that LDMI leads to\nbetter performance than competing approaches.",
            "journal ref": "Advances in Knowledge Discovery and Data Mining. PAKDD 2018.\n  Lecture Notes in Computer Science, vol 10938. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06730v1",
            "date": "2019-04-14 17:32:44+00:00",
            "title": "Text segmentation on multilabel documents: A distant-supervised approach",
            "authors": [
                "Saurav Manchanda",
                "George Karypis"
            ],
            "abstract": "Segmenting text into semantically coherent segments is an important task with\napplications in information retrieval and text summarization. Developing\naccurate topical segmentation requires the availability of training data with\nground truth information at the segment level. However, generating such labeled\ndatasets, especially for applications in which the meaning of the labels is\nuser-defined, is expensive and time-consuming. In this paper, we develop an\napproach that instead of using segment-level ground truth information, it\ninstead uses the set of labels that are associated with a document and are\neasier to obtain as the training data essentially corresponds to a multilabel\ndataset. Our method, which can be thought of as an instance of distant\nsupervision, improves upon the previous approaches by exploiting the fact that\nconsecutive sentences in a document tend to talk about the same topic, and\nhence, probably belong to the same class. Experiments on the text segmentation\ntask on a variety of datasets show that the segmentation produced by our method\nbeats the competing approaches on four out of five datasets and performs at par\non the fifth dataset. On the multilabel text classification task, our method\nperforms at par with the competing approaches, while requiring significantly\nless time to estimate than the competing approaches.",
            "journal ref": "2018 IEEE International Conference on Data Mining (ICDM),\n  1170-1175",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06941v1",
            "date": "2019-04-15 10:06:47+00:00",
            "title": "A framework for streamlined statistical prediction using topic models",
            "authors": [
                "Vanessa Glenny",
                "Jonathan Tuke",
                "Nigel Bean",
                "Lewis Mitchell"
            ],
            "abstract": "In the Humanities and Social Sciences, there is increasing interest in\napproaches to information extraction, prediction, intelligent linkage, and\ndimension reduction applicable to large text corpora. With approaches in these\nfields being grounded in traditional statistical techniques, the need arises\nfor frameworks whereby advanced NLP techniques such as topic modelling may be\nincorporated within classical methodologies. This paper provides a classical,\nsupervised, statistical learning framework for prediction from text, using\ntopic models as a data reduction method and the topics themselves as\npredictors, alongside typical statistical tools for predictive modelling. We\napply this framework in a Social Sciences context (applied animal behaviour) as\nwell as a Humanities context (narrative analysis) as examples of this\nframework. The results show that topic regression models perform comparably to\ntheir much less efficient equivalents that use individual words as predictors.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1904.07094v3",
            "date": "2019-04-15 14:55:59+00:00",
            "title": "CEDR: Contextualized Embeddings for Document Ranking",
            "authors": [
                "Sean MacAvaney",
                "Andrew Yates",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "Although considerable attention has been given to neural ranking\narchitectures recently, far less attention has been paid to the term\nrepresentations that are used as input to these models. In this work, we\ninvestigate how two pretrained contextualized language models (ELMo and BERT)\ncan be utilized for ad-hoc document ranking. Through experiments on TREC\nbenchmarks, we find that several existing neural ranking architectures can\nbenefit from the additional context provided by contextualized language models.\nFurthermore, we propose a joint approach that incorporates BERT's\nclassification vector into existing neural models and show that it outperforms\nstate-of-the-art ad-hoc ranking baselines. We call this joint approach CEDR\n(Contextualized Embeddings for Document Ranking). We also address practical\nchallenges in using these models for ranking, including the maximum input\nlength imposed by BERT and runtime performance impacts of contextualized\nlanguage models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.07148v1",
            "date": "2019-04-09 23:46:52+00:00",
            "title": "Parallel Hardware for Faster Morphological Analysis",
            "authors": [
                "Issam Damaj",
                "Mahmoud Imdoukh",
                "Rached Zantout"
            ],
            "abstract": "Morphological analysis in the Arabic language is computationally intensive,\nhas numerous forms and rules, and is intrinsically parallel. The investigation\npresented in this paper confirms that the effective development of parallel\nalgorithms and the derivation of corresponding processors in hardware enable\nimplementations with appealing performance characteristics. The presented\ndevelopments of parallel hardware comprise the application of a variety of\nalgorithm modelling techniques, strategies for concurrent processing, and the\ncreation of pioneering hardware implementations that target modern programmable\ndevices. The investigation includes the creation of a linguistic-based stemmer\nfor Arabic verb root extraction with extended infix processing to attain\nhigh-levels of accuracy. The implementations comprise three versions, namely,\nsoftware, non-pipelined processor, and pipelined processor with high\nthroughput. The targeted systems are high-performance multi-core processors for\nsoftware implementations and high-end Field Programmable Gate Array systems for\nhardware implementations. The investigation includes a thorough evaluation of\nthe methodology, and performance and accuracy analyses of the developed\nsoftware and hardware implementations. The pipelined processor achieved a\nsignificant speedup of 5571.4 over the software implementation. The developed\nstemmer for verb root extraction with infix processing attained accuracies of\n87% and 90.7% for analyzing the texts of the Holy Quran and its Chapter 29 -\nSurat Al-Ankabut.",
            "journal ref": "Computer and Information Sciences, Elsevier. 30(2018) 531-546",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1904.07293v2",
            "date": "2019-04-15 19:07:49+00:00",
            "title": "Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation",
            "authors": [
                "Md. Akmal Haidar",
                "Mehdi Rezagholizadeh",
                "Alan Do-Omri",
                "Ahmad Rashid"
            ],
            "abstract": "Text generation with generative adversarial networks (GANs) can be divided\ninto the text-based and code-based categories according to the type of signals\nused for discrimination. In this work, we introduce a novel text-based approach\ncalled Soft-GAN to effectively exploit GAN setup for text generation. We\ndemonstrate how autoencoders (AEs) can be used for providing a continuous\nrepresentation of sentences, which we will refer to as soft-text. This soft\nrepresentation will be used in GAN discrimination to synthesize similar\nsoft-texts. We also propose hybrid latent code and text-based GAN (LATEXT-GAN)\napproaches with one or more discriminators, in which a combination of the\nlatent code and the soft-text is used for GAN discriminations. We perform a\nnumber of subjective and objective experiments on two well-known datasets (SNLI\nand Image COCO) to validate our techniques. We discuss the results using\nseveral evaluation metrics and show that the proposed techniques outperform the\ntraditional GAN-based text-generation methods.",
            "journal ref": "2019 Annual Conference of the North American Chapter of the\n  Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07318v1",
            "date": "2019-04-15 20:15:46+00:00",
            "title": "Natural Language Semantics With Pictures: Some Language & Vision Datasets and Potential Uses for Computational Semantics",
            "authors": [
                "David Schlangen"
            ],
            "abstract": "Propelling, and propelled by, the \"deep learning revolution\", recent years\nhave seen the introduction of ever larger corpora of images annotated with\nnatural language expressions. We survey some of these corpora, taking a\nperspective that reverses the usual directionality, as it were, by viewing the\nimages as semantic annotation of the natural language expressions. We discuss\ndatasets that can be derived from the corpora, and tasks of potential interest\nfor computational semanticists that can be defined on those. In this, we make\nuse of relations provided by the corpora (namely, the link between expression\nand image, and that between two expressions linked to the same image) and\nrelations that we can add (similarity relations between expressions, or between\nimages). Specifically, we show that in this way we can create data that can be\nused to learn and evaluate lexical and compositional grounded semantics, and we\nshow that the \"linked to same image\" relation tracks a semantic implication\nrelation that is recognisable to annotators even in the absence of the linking\nimage as evidence. Finally, as an example of possible benefits of this\napproach, we show that an exemplar-model-based approach to implication beats a\n(simple) distributional space-based one on some derived datasets, while lending\nitself to explainability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07342v1",
            "date": "2019-04-15 21:51:21+00:00",
            "title": "Learning Twitter User Sentiments on Climate Change with Limited Labeled Data",
            "authors": [
                "Allison Koenecke",
                "Jordi Feliu-Fab\u00e0"
            ],
            "abstract": "While it is well-documented that climate change accepters and deniers have\nbecome increasingly polarized in the United States over time, there has been no\nlarge-scale examination of whether these individuals are prone to changing\ntheir opinions as a result of natural external occurrences. On the\nsub-population of Twitter users, we examine whether climate change sentiment\nchanges in response to five separate natural disasters occurring in the U.S. in\n2018. We begin by showing that relevant tweets can be classified with over 75%\naccuracy as either accepting or denying climate change when using our\nmethodology to compensate for limited labeled data; results are robust across\nseveral machine learning models and yield geographic-level results in line with\nprior research. We then apply RNNs to conduct a cohort-level analysis showing\nthat the 2018 hurricanes yielded a statistically significant increase in\naverage tweet sentiment affirming climate change. However, this effect does not\nhold for the 2018 blizzard and wildfires studied, implying that Twitter users'\nopinions on climate change are fairly ingrained on this subset of natural\ndisasters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07531v4",
            "date": "2019-04-16 08:30:31+00:00",
            "title": "Understanding the Behaviors of BERT in Ranking",
            "authors": [
                "Yifan Qiao",
                "Chenyan Xiong",
                "Zhenghao Liu",
                "Zhiyuan Liu"
            ],
            "abstract": "This paper studies the performances and behaviors of BERT in ranking tasks.\nWe explore several different ways to leverage the pre-trained BERT and\nfine-tune it on two ranking tasks: MS MARCO passage reranking and TREC Web\nTrack ad hoc document ranking. Experimental results on MS MARCO demonstrate the\nstrong effectiveness of BERT in question-answering focused passage ranking\ntasks, as well as the fact that BERT is a strong interaction-based seq2seq\nmatching model. Experimental results on TREC show the gaps between the BERT\npre-trained on surrounding contexts and the needs of ad hoc document ranking.\nAnalyses illustrate how BERT allocates its attentions between query-document\ntokens in its Transformer layers, how it prefers semantic matches between\nparaphrase tokens, and how that differs with the soft match patterns learned by\na click-trained neural ranker.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.07535v2",
            "date": "2019-04-16 08:39:06+00:00",
            "title": "Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extraction",
            "authors": [
                "Shun Zheng",
                "Wei Cao",
                "Wei Xu",
                "Jiang Bian"
            ],
            "abstract": "Most existing event extraction (EE) methods merely extract event arguments\nwithin the sentence scope. However, such sentence-level EE methods struggle to\nhandle soaring amounts of documents from emerging applications, such as\nfinance, legislation, health, etc., where event arguments always scatter across\ndifferent sentences, and even multiple such event mentions frequently co-exist\nin the same document. To address these challenges, we propose a novel\nend-to-end model, Doc2EDAG, which can generate an entity-based directed acyclic\ngraph to fulfill the document-level EE (DEE) effectively. Moreover, we\nreformalize a DEE task with the no-trigger-words design to ease the\ndocument-level event labeling. To demonstrate the effectiveness of Doc2EDAG, we\nbuild a large-scale real-world dataset consisting of Chinese financial\nannouncements with the challenges mentioned above. Extensive experiments with\ncomprehensive analyses illustrate the superiority of Doc2EDAG over\nstate-of-the-art methods. Data and codes can be found at\nhttps://github.com/dolphin-zs/Doc2EDAG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07695v1",
            "date": "2019-04-13 09:08:46+00:00",
            "title": "Short Text Topic Modeling Techniques, Applications, and Performance: A Survey",
            "authors": [
                "Qiang Jipeng",
                "Qian Zhenyu",
                "Li Yun",
                "Yuan Yunhao",
                "Wu Xindong"
            ],
            "abstract": "Analyzing short texts infers discriminative and coherent latent topics that\nis a critical and fundamental task since many real-world applications require\nsemantic understanding of short texts. Traditional long text topic modeling\nalgorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this\nproblem very well since only very limited word co-occurrence information is\navailable in short texts. Therefore, short text topic modeling has already\nattracted much attention from the machine learning research community in recent\nyears, which aims at overcoming the problem of sparseness in short texts. In\nthis survey, we conduct a comprehensive review of various short text topic\nmodeling techniques proposed in the literature. We present three categories of\nmethods based on Dirichlet multinomial mixture, global word co-occurrences, and\nself-aggregation, with example of representative approaches in each category\nand analysis of their performance on various tasks. We develop the first\ncomprehensive open-source library, called STTM, for use in Java that integrates\nall surveyed algorithms within a unified interface, benchmark datasets, to\nfacilitate the expansion of new methods in this research field. Finally, we\nevaluate these state-of-the-art methods on many real-world datasets and compare\ntheir performance against one another and versus long text topic modeling\nalgorithm.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.07741v1",
            "date": "2019-04-16 14:50:09+00:00",
            "title": "Sameness Attracts, Novelty Disturbs, but Outliers Flourish in Fanfiction Online",
            "authors": [
                "Elise Jing",
                "Simon DeDeo",
                "Yong-Yeol Ahn"
            ],
            "abstract": "The nature of what people enjoy is not just a central question for the\ncreative industry, it is a driving force of cultural evolution. It is widely\nbelieved that successful cultural products balance novelty and conventionality:\nthey provide something familiar but at least somewhat divergent from what has\ncome before, and occupy a satisfying middle ground between \"more of the same\"\nand \"too strange\". We test this belief using a large dataset of over half a\nmillion works of fanfiction from the website Archive of Our Own (AO3), looking\nat how the recognition a work receives varies with its novelty. We quantify the\nnovelty through a term-based language model, and a topic model, in the context\nof existing works within the same fandom. Contrary to the balance theory, we\nfind that the lowest-novelty are the most popular and that popularity declines\nmonotonically with novelty. A few exceptions can be found: extremely popular\nworks that are among the highest novelty within the fandom. Taken together, our\nfindings not only challenge the traditional theory of the hedonic value of\nnovelty, they invert it: people prefer the least novel things, are repelled by\nthe middle ground, and have an occasional enthusiasm for extreme outliers. It\nsuggests that cultural evolution must work against inertia --- the appetite\npeople have to continually reconsume the familiar, and may resemble a\npunctuated equilibrium rather than a smooth evolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07826v2",
            "date": "2019-04-16 17:07:20+00:00",
            "title": "Unsupervised Discovery of Multimodal Links in Multi-image, Multi-sentence Documents",
            "authors": [
                "Jack Hessel",
                "Lillian Lee",
                "David Mimno"
            ],
            "abstract": "Images and text co-occur constantly on the web, but explicit links between\nimages and sentences (or other intra-document textual units) are often not\npresent. We present algorithms that discover image-sentence relationships\nwithout relying on explicit multimodal annotation in training. We experiment on\nseven datasets of varying difficulty, ranging from documents consisting of\ngroups of images captioned post hoc by crowdworkers to naturally-occurring\nuser-generated multimodal documents. We find that a structured training\nobjective based on identifying whether collections of images and sentences\nco-occur in documents can suffice to predict links between specific sentences\nand specific images within the same document at test time.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07982v1",
            "date": "2019-04-16 20:55:59+00:00",
            "title": "Query Expansion for Cross-Language Question Re-Ranking",
            "authors": [
                "Muhammad Mahbubur Rahman",
                "Sorami Hisamoto",
                "Kevin Duh"
            ],
            "abstract": "Community question-answering (CQA) platforms have become very popular forums\nfor asking and answering questions daily. While these forums are rich\nrepositories of community knowledge, they present challenges for finding\nrelevant answers and similar questions, due to the open-ended nature of\ninformal discussions. Further, if the platform allows questions and answers in\nmultiple languages, we are faced with the additional challenge of matching\ncross-lingual information. In this work, we focus on the cross-language\nquestion re-ranking shared task, which aims to find existing questions that may\nbe written in different languages. Our contribution is an exploration of query\nexpansion techniques for this problem. We investigate expansions based on Word\nEmbeddings, DBpedia concepts linking, and Hypernym, and show that they\noutperform existing state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.08051v1",
            "date": "2019-04-17 02:21:51+00:00",
            "title": "Posterior-regularized REINFORCE for Instance Selection in Distant Supervision",
            "authors": [
                "Qi Zhang",
                "Siliang Tang",
                "Xiang Ren",
                "Fei Wu",
                "Shiliang Pu",
                "Yueting Zhuang"
            ],
            "abstract": "This paper provides a new way to improve the efficiency of the REINFORCE\ntraining process. We apply it to the task of instance selection in distant\nsupervision. Modeling the instance selection in one bag as a sequential\ndecision process, a reinforcement learning agent is trained to determine\nwhether an instance is valuable or not and construct a new bag with less noisy\ninstances. However unbiased methods, such as REINFORCE, could usually take much\ntime to train. This paper adopts posterior regularization (PR) to integrate\nsome domain-specific rules in instance selection using REINFORCE. As the\nexperiment results show, this method remarkably improves the performance of the\nrelation classifier trained on cleaned distant supervision dataset as well as\nthe efficiency of the REINFORCE training.",
            "journal ref": "naacl 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08061v1",
            "date": "2019-04-17 03:01:16+00:00",
            "title": "Reinforcement Learning Based Emotional Editing Constraint Conversation Generation",
            "authors": [
                "Jia Li",
                "Xiao Sun",
                "Xing Wei",
                "Changliang Li",
                "Jianhua Tao"
            ],
            "abstract": "In recent years, the generation of conversation content based on deep neural\nnetworks has attracted many researchers. However, traditional neural language\nmodels tend to generate general replies, lacking logical and emotional factors.\nThis paper proposes a conversation content generation model that combines\nreinforcement learning with emotional editing constraints to generate more\nmeaningful and customizable emotional replies. The model divides the replies\ninto three clauses based on pre-generated keywords and uses the emotional\neditor to further optimize the final reply. The model combines multi-task\nlearning with multiple indicator rewards to comprehensively optimize the\nquality of replies. Experiments shows that our model can not only improve the\nfluency of the replies, but also significantly enhance the logical relevance\nand emotional relevance of the replies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08100v1",
            "date": "2019-04-17 06:20:53+00:00",
            "title": "Patent Analytics Based on Feature Vector Space Model: A Case of IoT",
            "authors": [
                "Lei Lei",
                "Jiaju Qi",
                "Kan Zheng"
            ],
            "abstract": "The number of approved patents worldwide increases rapidly each year, which\nrequires new patent analytics to efficiently mine the valuable information\nattached to these patents. Vector space model (VSM) represents documents as\nhigh-dimensional vectors, where each dimension corresponds to a unique term.\nWhile originally proposed for information retrieval systems, VSM has also seen\nwide applications in patent analytics, and used as a fundamental tool to map\npatent documents to structured data. However, VSM method suffers from several\nlimitations when applied to patent analysis tasks, such as loss of\nsentence-level semantics and curse-of-dimensionality problems. In order to\naddress the above limitations, we propose a patent analytics based on feature\nvector space model (FVSM), where the FVSM is constructed by mapping patent\ndocuments to feature vectors extracted by convolutional neural networks (CNN).\nThe applications of FVSM for three typical patent analysis tasks, i.e., patents\nsimilarity comparison, patent clustering, and patent map generation are\ndiscussed. A case study using patents related to Internet of Things (IoT)\ntechnology is illustrated to demonstrate the performance and effectiveness of\nFVSM. The proposed FVSM can be adopted by other patent analysis studies to\nreplace VSM, based on which various big data learning tasks can be performed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08109v1",
            "date": "2019-04-17 07:16:10+00:00",
            "title": "Contextual Aware Joint Probability Model Towards Question Answering System",
            "authors": [
                "Liu Yang",
                "Lijing Song"
            ],
            "abstract": "In this paper, we address the question answering challenge with the SQuAD 2.0\ndataset. We design a model architecture which leverages BERT's capability of\ncontext-aware word embeddings and BiDAF's context interactive exploration\nmechanism. By integrating these two state-of-the-art architectures, our system\ntries to extract the contextual word representation at word and character\nlevels, for better comprehension of both question and context and their\ncorrelations. We also propose our original joint posterior probability\npredictor module and its associated loss functions. Our best model so far\nobtains F1 score of 75.842% and EM score of 72.24% on the test PCE leaderboad.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08194v3",
            "date": "2019-04-17 11:24:58+00:00",
            "title": "Effective Estimation of Deep Generative Language Models",
            "authors": [
                "Tom Pelsmaeker",
                "Wilker Aziz"
            ],
            "abstract": "Advances in variational inference enable parameterisation of probabilistic\nmodels by deep neural networks. This combines the statistical transparency of\nthe probabilistic modelling framework with the representational power of deep\nlearning. Yet, due to a problem known as posterior collapse, it is difficult to\nestimate such models in the context of language modelling effectively. We\nconcentrate on one such model, the variational auto-encoder, which we argue is\nan important building block in hierarchical probabilistic models of language.\nThis paper contributes a sober view of the problem, a survey of techniques to\naddress it, novel techniques, and extensions to the model. To establish a\nranking of techniques, we perform a systematic comparison using Bayesian\noptimisation and find that many techniques perform reasonably similar, given\nenough resources. Still, a favourite can be named based on convenience. We also\nmake several empirical observations and recommendations of best practices that\nshould help researchers interested in this exciting field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08311v2",
            "date": "2019-04-17 15:18:23+00:00",
            "title": "Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation",
            "authors": [
                "Gakuto Kurata",
                "Kartik Audhkhasi"
            ],
            "abstract": "Conventional automatic speech recognition (ASR) systems trained from\nframe-level alignments can easily leverage posterior fusion to improve ASR\naccuracy and build a better single model with knowledge distillation.\nEnd-to-end ASR systems trained using the Connectionist Temporal Classification\n(CTC) loss do not require frame-level alignment and hence simplify model\ntraining. However, sparse and arbitrary posterior spike timings from CTC models\npose a new set of challenges in posterior fusion from multiple models and\nknowledge distillation between CTC models. We propose a method to train a CTC\nmodel so that its spike timings are guided to align with those of a pre-trained\nguiding CTC model. As a result, all models that share the same guiding model\nhave aligned spike timings. We show the advantage of our method in various\nscenarios including posterior fusion of CTC models and knowledge distillation\nbetween CTC models with different architectures. With the 300-hour Switchboard\ntraining data, the single word CTC model distilled from multiple models\nimproved the word error rates to 13.7%/23.1% from 14.9%/24.1% on the Hub5 2000\nSwitchboard/CallHome test sets without using any data augmentation, language\nmodel, or complex decoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08314v2",
            "date": "2019-04-17 15:21:33+00:00",
            "title": "MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction",
            "authors": [
                "Oscar Araque",
                "Lorenzo Gatti",
                "Kyriaki Kalimeri"
            ],
            "abstract": "Moral rhetoric plays a fundamental role in how we perceive and interpret the\ninformation we receive, greatly influencing our decision-making process.\nEspecially when it comes to controversial social and political issues, our\nopinions and attitudes are hardly ever based on evidence alone. The Moral\nFoundations Dictionary (MFD) was developed to operationalize moral values in\nthe text. In this study, we present MoralStrength, a lexicon of approximately\n1,000 lemmas, obtained as an extension of the Moral Foundations Dictionary,\nbased on WordNet synsets. Moreover, for each lemma it provides with a\ncrowdsourced numeric assessment of Moral Valence, indicating the strength with\nwhich a lemma is expressing the specific value. We evaluated the predictive\npotentials of this moral lexicon, defining three utilization approaches of\nincreased complexity, ranging from lemmas' statistical properties to a deep\nlearning approach of word embeddings based on semantic similarity. Logistic\nregression models trained on the features extracted from MoralStrength,\nsignificantly outperformed the current state-of-the-art, reaching an F1-score\nof 87.6% over the previous 62.4% (p-value<0.01), and an average F1-Score of\n86.25% over six different datasets. Such findings pave the way for further\nresearch, allowing for an in-depth understanding of moral narratives in text\nfor a wide range of social issues.",
            "journal ref": "Knowledge-Based Systems 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08524v1",
            "date": "2019-04-17 22:40:01+00:00",
            "title": "Towards Open Intent Discovery for Conversational Text",
            "authors": [
                "Nikhita Vedula",
                "Nedim Lipka",
                "Pranav Maneriker",
                "Srinivasan Parthasarathy"
            ],
            "abstract": "Detecting and identifying user intent from text, both written and spoken,\nplays an important role in modelling and understand dialogs. Existing research\nfor intent discovery model it as a classification task with a predefined set of\nknown categories. To generailze beyond these preexisting classes, we define a\nnew task of \\textit{open intent discovery}. We investigate how intent can be\ngeneralized to those not seen during training. To this end, we propose a\ntwo-stage approach to this task - predicting whether an utterance contains an\nintent, and then tagging the intent in the input utterance. Our model consists\nof a bidirectional LSTM with a CRF on top to capture contextual semantics,\nsubject to some constraints. Self-attention is used to learn long distance\ndependencies. Further, we adapt an adversarial training approach to improve\nrobustness and perforamce across domains. We also present a dataset of 25k\nreal-life utterances that have been labelled via crowd sourcing. Our\nexperiments across different domains and real-world datasets show the\neffectiveness of our approach, with less than 100 annotated examples needed per\nunique domain to recognize diverse intents. The approach outperforms\nstate-of-the-art baselines by 5-15% F1 score points.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.08587v1",
            "date": "2019-04-18 04:22:23+00:00",
            "title": "Creative Procedural-Knowledge Extraction From Web Design Tutorials",
            "authors": [
                "Longqi Yang",
                "Chen Fang",
                "Hailin Jin",
                "Walter Chang",
                "Deborah Estrin"
            ],
            "abstract": "Complex design tasks often require performing diverse actions in a specific\norder. To (semi-)autonomously accomplish these tasks, applications need to\nunderstand and learn a wide range of design procedures, i.e., Creative\nProcedural-Knowledge (CPK). Prior knowledge base construction and mining have\nnot typically addressed the creative fields, such as design and arts. In this\npaper, we formalize an ontology of CPK using five components: goal, workflow,\naction, command and usage; and extract components' values from online design\ntutorials. We scraped 19.6K tutorial-related webpages and built a web\napplication for professional designers to identify and summarize CPK\ncomponents. The annotated dataset consists of 819 unique commands, 47,491\nactions, and 2,022 workflows and goals. Based on this dataset, we propose a\ngeneral CPK extraction pipeline and demonstrate that existing text\nclassification and sequence-to-sequence models are limited in identifying,\npredicting and summarizing complex operations described in heterogeneous\nstyles. Through quantitative and qualitative error analysis, we discuss CPK\nextraction challenges that need to be addressed by future research.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.08637v1",
            "date": "2019-04-18 08:35:49+00:00",
            "title": "ConvLab: Multi-Domain End-to-End Dialog System Platform",
            "authors": [
                "Sungjin Lee",
                "Qi Zhu",
                "Ryuichi Takanobu",
                "Xiang Li",
                "Yaoqin Zhang",
                "Zheng Zhang",
                "Jinchao Li",
                "Baolin Peng",
                "Xiujun Li",
                "Minlie Huang",
                "Jianfeng Gao"
            ],
            "abstract": "We present ConvLab, an open-source multi-domain end-to-end dialog system\nplatform, that enables researchers to quickly set up experiments with reusable\ncomponents and compare a large set of different approaches, ranging from\nconventional pipeline systems to end-to-end neural models, in common\nenvironments. ConvLab offers a set of fully annotated datasets and associated\npre-trained reference models. As a showcase, we extend the MultiWOZ dataset\nwith user dialog act annotations to train all component models and demonstrate\nhow ConvLab makes it easy and effortless to conduct complicated experiments in\nmulti-domain end-to-end dialog settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08783v1",
            "date": "2019-04-18 13:47:00+00:00",
            "title": "Evaluating the Underlying Gender Bias in Contextualized Word Embeddings",
            "authors": [
                "Christine Basta",
                "Marta R. Costa-juss\u00e0",
                "Noe Casas"
            ],
            "abstract": "Gender bias is highly impacting natural language processing applications.\nWord embeddings have clearly been proven both to keep and amplify gender biases\nthat are present in current data sources. Recently, contextualized word\nembeddings have enhanced previous word embedding techniques by computing word\nvector representations dependent on the sentence they appear in.\n  In this paper, we study the impact of this conceptual change in the word\nembedding computation in relation with gender bias. Our analysis includes\ndifferent measures previously applied in the literature to standard word\nembeddings. Our findings suggest that contextualized word embeddings are less\nbiased than standard ones even when the latter are debiased.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08835v2",
            "date": "2019-04-18 15:20:45+00:00",
            "title": "Clause-Wise and Recursive Decoding for Complex and Cross-Domain Text-to-SQL Generation",
            "authors": [
                "Dongjun Lee"
            ],
            "abstract": "Most deep learning approaches for text-to-SQL generation are limited to the\nWikiSQL dataset, which only supports very simple queries over a single table.\nWe focus on the Spider dataset, a complex and cross-domain text-to-SQL task,\nwhich includes complex queries over multiple tables. In this paper, we propose\na SQL clause-wise decoding neural architecture with a self-attention based\ndatabase schema encoder to address the Spider task. Each of the clause-specific\ndecoders consists of a set of sub-modules, which is defined by the syntax of\neach clause. Additionally, our model works recursively to support nested\nqueries. When evaluated on the Spider dataset, our approach achieves 4.6\\% and\n9.8\\% accuracy gain in the test and dev sets, respectively. In addition, we\nshow that our model is significantly more effective at predicting complex and\nnested queries than previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08850v1",
            "date": "2019-04-17 14:21:43+00:00",
            "title": "True Parallel Graph Transformations: an Algebraic Approach Based on Weak Spans",
            "authors": [
                "Thierry Boy de la Tour",
                "Rachid Echahed"
            ],
            "abstract": "We address the problem of defining graph transformations by the simultaneous\napplication of direct transformations even when these cannot be applied\nindependently of each other. An algebraic approach is adopted, with production\nrules of the form $L\\xleftarrow{l}K \\xleftarrow{i} I \\xrightarrow{r} R$, called\nweak spans. A parallel coherent transformation is introduced and shown to be a\nconservative extension of the interleaving semantics of parallel independent\ndirect transformations. A categorical construction of finitely attributed\nstructures is proposed, in which parallel coherent transformations can be built\nin a natural way. These notions are introduced and illustrated on detailed\nexamples.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1904.08926v1",
            "date": "2019-04-15 13:25:38+00:00",
            "title": "Characterization of citizens using word2vec and latent topic analysis in a large set of tweets",
            "authors": [
                "Vargas-Calder\u00f3n Vladimir",
                "Camargo Jorge"
            ],
            "abstract": "With the increasing use of the Internet and mobile devices, social networks\nare becoming the most used media to communicate citizens' ideas and thoughts.\nThis information is very useful to identify communities with common ideas based\non what they publish in the network. This paper presents a method to\nautomatically detect city communities based on machine learning techniques\napplied to a set of tweets from Bogot\\'a's citizens. An analysis was performed\nin a collection of 2,634,176 tweets gathered from Twitter in a period of six\nmonths. Results show that the proposed method is an interesting tool to\ncharacterize a city population based on a machine learning methods and text\nanalytics.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1904.09068v2",
            "date": "2019-04-19 04:10:03+00:00",
            "title": "A Hybrid Retrieval-Generation Neural Conversation Model",
            "authors": [
                "Liu Yang",
                "Junjie Hu",
                "Minghui Qiu",
                "Chen Qu",
                "Jianfeng Gao",
                "W. Bruce Croft",
                "Xiaodong Liu",
                "Yelong Shen",
                "Jingjing Liu"
            ],
            "abstract": "Intelligent personal assistant systems that are able to have multi-turn\nconversations with human users are becoming increasingly popular. Most previous\nresearch has been focused on using either retrieval-based or generation-based\nmethods to develop such systems. Retrieval-based methods have the advantage of\nreturning fluent and informative responses with great diversity. However, the\nperformance of the methods is limited by the size of the response repository.\nOn the other hand, generation-based methods can produce highly coherent\nresponses on any topics. But the generated responses are often generic and not\ninformative due to the lack of grounding knowledge. In this paper, we propose a\nhybrid neural conversation model that combines the merits of both response\nretrieval and generation methods. Experimental results on Twitter and\nFoursquare data show that the proposed model outperforms both retrieval-based\nmethods and generation-based methods (including a recently proposed\nknowledge-grounded neural conversation model) under both automatic evaluation\nmetrics and human evaluation. We hope that the findings in this study provide\nnew insights on how to integrate text retrieval and text generation models for\nbuilding conversation systems.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.09122v1",
            "date": "2019-04-19 08:59:13+00:00",
            "title": "Zero-Shot Cross-Lingual Opinion Target Extraction",
            "authors": [
                "Soufian Jebbara",
                "Philipp Cimiano"
            ],
            "abstract": "Aspect-based sentiment analysis involves the recognition of so called opinion\ntarget expressions (OTEs). To automatically extract OTEs, supervised learning\nalgorithms are usually employed which are trained on manually annotated\ncorpora. The creation of these corpora is labor-intensive and sufficiently\nlarge datasets are therefore usually only available for a very narrow selection\nof languages and domains. In this work, we address the lack of available\nannotated data for specific languages by proposing a zero-shot cross-lingual\napproach for the extraction of opinion target expressions. We leverage\nmultilingual word embeddings that share a common vector space across various\nlanguages and incorporate these into a convolutional neural network\narchitecture for OTE extraction. Our experiments with 5 languages give\npromising results: We can successfully train a model on annotated data of a\nsource language and perform accurate prediction on a target language without\never using any annotated samples in that target language. Depending on the\nsource and target language pairs, we reach performances in a zero-shot regime\nof up to 77% of a model trained on target language data. Furthermore, we can\nincrease this performance up to 87% of a baseline model trained on target\nlanguage data by performing cross-lingual learning from multiple source\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09131v2",
            "date": "2019-04-19 09:44:22+00:00",
            "title": "OpenTapioca: Lightweight Entity Linking for Wikidata",
            "authors": [
                "Antonin Delpeuch"
            ],
            "abstract": "We propose a simple Named Entity Linking system that can be trained from\nWikidata only. This demonstrates the strengths and weaknesses of this data\nsource for this task and provides an easily reproducible baseline to compare\nother systems against. Our model is lightweight to train, to run and to keep\nsynchronous with Wikidata in real time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09331v2",
            "date": "2019-04-19 20:23:27+00:00",
            "title": "Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction",
            "authors": [
                "Qinyuan Ye",
                "Liyuan Liu",
                "Maosen Zhang",
                "Xiang Ren"
            ],
            "abstract": "In recent years there is a surge of interest in applying distant supervision\n(DS) to automatically generate training data for relation extraction (RE). In\nthis paper, we study the problem what limits the performance of DS-trained\nneural models, conduct thorough analyses, and identify a factor that can\ninfluence the performance greatly, shifted label distribution. Specifically, we\nfound this problem commonly exists in real-world DS datasets, and without\nspecial handing, typical DS-RE models cannot automatically adapt to this shift,\nthus achieving deteriorated performance. To further validate our intuition, we\ndevelop a simple yet effective adaptation method for DS-trained models, bias\nadjustment, which updates models learned over the source domain (i.e., DS\ntraining set) with a label distribution estimated on the target domain (i.e.,\ntest set). Experiments demonstrate that bias adjustment achieves consistent\nperformance gains on DS-trained models, especially on neural models, with an up\nto 23% relative F1 improvement, which verifies our assumptions. Our code and\ndata can be found at\n\\url{https://github.com/INK-USC/shifted-label-distribution}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09447v4",
            "date": "2019-04-20 13:46:36+00:00",
            "title": "An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing",
            "authors": [
                "Martin Schmitt",
                "Sahand Sharifzadeh",
                "Volker Tresp",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Knowledge graphs (KGs) can vary greatly from one domain to another. Therefore\nsupervised approaches to both graph-to-text generation and text-to-graph\nknowledge extraction (semantic parsing) will always suffer from a shortage of\ndomain-specific parallel graph-text data; at the same time, adapting a model\ntrained on a different domain is often impossible due to little or no overlap\nin entities and relations. This situation calls for an approach that (1) does\nnot need large amounts of annotated data and thus (2) does not need to rely on\ndomain adaptation techniques to work well in different domains. To this end, we\npresent the first approach to unsupervised text generation from KGs and show\nsimultaneously how it can be used for unsupervised semantic parsing. We\nevaluate our approach on WebNLG v2.1 and a new benchmark leveraging scene\ngraphs from Visual Genome. Our system outperforms strong baselines for both\ntext$\\leftrightarrow$graph conversion tasks without any manual adaptation from\none dataset to the other. In additional experiments, we investigate the impact\nof using different unsupervised objectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09491v2",
            "date": "2019-04-20 20:01:52+00:00",
            "title": "Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding",
            "authors": [
                "Guokan Shang",
                "Antoine Jean-Pierre Tixier",
                "Michalis Vazirgiannis",
                "Jean-Pierre Lorr\u00e9"
            ],
            "abstract": "Abstractive community detection is an important spoken language understanding\ntask, whose goal is to group utterances in a conversation according to whether\nthey can be jointly summarized by a common abstractive sentence. This paper\nprovides a novel approach to this task. We first introduce a neural contextual\nutterance encoder featuring three types of self-attention mechanisms. We then\ntrain it using the siamese and triplet energy-based meta-architectures.\nExperiments on the AMI corpus show that our system outperforms multiple\nenergy-based and non-energy based baselines from the state-of-the-art. Code and\ndata are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09537v1",
            "date": "2019-04-21 03:49:09+00:00",
            "title": "PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text",
            "authors": [
                "Haitian Sun",
                "Tania Bedrax-Weiss",
                "William W. Cohen"
            ],
            "abstract": "We consider open-domain queston answering (QA) where answers are drawn from\neither a corpus, a knowledge base (KB), or a combination of both of these. We\nfocus on a setting in which a corpus is supplemented with a large but\nincomplete KB, and on questions that require non-trivial (e.g., ``multi-hop'')\nreasoning. We describe PullNet, an integrated framework for (1) learning what\nto retrieve (from the KB and/or corpus) and (2) reasoning with this\nheterogeneous information to find the best answer. PullNet uses an {iterative}\nprocess to construct a question-specific subgraph that contains information\nrelevant to the question. In each iteration, a graph convolutional network\n(graph CNN) is used to identify subgraph nodes that should be expanded using\nretrieval (or ``pull'') operations on the corpus and/or KB. After the subgraph\nis complete, a similar graph CNN is used to extract the answer from the\nsubgraph. This retrieve-and-reason process allows us to answer multi-hop\nquestions using large KBs and corpora. PullNet is weakly supervised, requiring\nquestion-answer pairs but not gold inference paths. Experimentally PullNet\nimproves over the prior state-of-the art, and in the setting where a corpus is\nused with incomplete KB these improvements are often dramatic. PullNet is also\noften superior to prior systems in a KB-only setting or a text-only setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09557v1",
            "date": "2019-04-21 07:30:35+00:00",
            "title": "A Study on Agreement in PICO Span Annotations",
            "authors": [
                "Grace E. Lee",
                "Aixin Sun"
            ],
            "abstract": "In evidence-based medicine, relevance of medical literature is determined by\npredefined relevance conditions. The conditions are defined based on PICO\nelements, namely, Patient, Intervention, Comparator, and Outcome. Hence, PICO\nannotations in medical literature are essential for automatic relevant document\nfiltering. However, defining boundaries of text spans for PICO elements is not\nstraightforward. In this paper, we study the agreement of PICO annotations made\nby multiple human annotators, including both experts and non-experts.\nAgreements are estimated by a standard span agreement (i.e., matching both\nlabels and boundaries of text spans), and two types of relaxed span agreement\n(i.e., matching labels without guaranteeing matching boundaries of spans).\nBased on the analysis, we report two observations: (i) Boundaries of PICO span\nannotations by individual human annotators are very diverse. (ii) Despite the\ndisagreement in span boundaries, general areas of the span annotations are\nbroadly agreed by annotators. Our results suggest that applying a standard\nagreement alone may undermine the agreement of PICO spans, and adopting both a\nstandard and a relaxed agreements is more suitable for PICO span evaluation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.09705v1",
            "date": "2019-04-22 03:00:40+00:00",
            "title": "Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge",
            "authors": [
                "Yu-Ping Ruan",
                "Xiaodan Zhu",
                "Zhen-Hua Ling",
                "Zhan Shi",
                "Quan Liu",
                "Si Wei"
            ],
            "abstract": "Winograd Schema Challenge (WSC) was proposed as an AI-hard problem in testing\ncomputers' intelligence on common sense representation and reasoning. This\npaper presents the new state-of-theart on WSC, achieving an accuracy of 71.1%.\nWe demonstrate that the leading performance benefits from jointly modelling\nsentence structures, utilizing knowledge learned from cutting-edge pretraining\nmodels, and performing fine-tuning. We conduct detailed analyses, showing that\nfine-tuning is critical for achieving the performance, but it helps more on the\nsimpler associative problems. Modelling sentence dependency structures,\nhowever, consistently helps on the harder non-associative subset of WSC.\nAnalysis also shows that larger fine-tuning datasets yield better performances,\nsuggesting the potential benefit of future work on annotating more Winograd\nschema sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10151v2",
            "date": "2019-04-23 04:45:28+00:00",
            "title": "REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments",
            "authors": [
                "Yuankai Qi",
                "Qi Wu",
                "Peter Anderson",
                "Xin Wang",
                "William Yang Wang",
                "Chunhua Shen",
                "Anton van den Hengel"
            ],
            "abstract": "One of the long-term challenges of robotics is to enable robots to interact\nwith humans in the visual world via natural language, as humans are visual\nanimals that communicate through language. Overcoming this challenge requires\nthe ability to perform a wide variety of complex tasks in response to\nmultifarious instructions from humans. In the hope that it might drive progress\ntowards more flexible and powerful human interactions with robots, we propose a\ndataset of varied and complex robot tasks, described in natural language, in\nterms of objects visible in a large set of real images. Given an instruction,\nsuccess requires navigating through a previously-unseen environment to identify\nan object. This represents a practical challenge, but one that closely reflects\none of the core visual problems in robotics. Several state-of-the-art\nvision-and-language navigation, and referring-expression models are tested to\nverify the difficulty of this new task, but none of them show promising results\nbecause there are many fundamental differences between our task and previous\nones. A novel Interactive Navigator-Pointer model is also proposed that\nprovides a strong baseline on the task. The proposed model especially achieves\nthe best performance on the unseen test split, but still leaves substantial\nroom for improvement compared to the human performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.10610v1",
            "date": "2019-04-24 02:26:48+00:00",
            "title": "Condition-Transforming Variational AutoEncoder for Conversation Response Generation",
            "authors": [
                "Yu-Ping Ruan",
                "Zhen-Hua Ling",
                "Quan Liu",
                "Zhigang Chen",
                "Nitin Indurkhya"
            ],
            "abstract": "This paper proposes a new model, called condition-transforming variational\nautoencoder (CTVAE), to improve the performance of conversation response\ngeneration using conditional variational autoencoders (CVAEs). In conventional\nCVAEs , the prior distribution of latent variable z follows a multivariate\nGaussian distribution with mean and variance modulated by the input conditions.\nPrevious work found that this distribution tends to become condition\nindependent in practical application. In our proposed CTVAE model, the latent\nvariable z is sampled by performing a non-lineartransformation on the\ncombination of the input conditions and the samples from a\ncondition-independent prior distribution N (0; I). In our objective\nevaluations, the CTVAE model outperforms the CVAE model on fluency metrics and\nsurpasses a sequence-to-sequence (Seq2Seq) model on diversity metrics. In\nsubjective preference tests, our proposed CTVAE model performs significantly\nbetter than CVAE and Seq2Seq models on generating fluency, informative and\ntopic relevant responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10820v4",
            "date": "2019-04-24 13:55:42+00:00",
            "title": "Semantic Drift in Multilingual Representations",
            "authors": [
                "Lisa Beinborn",
                "Rochelle Choenni"
            ],
            "abstract": "Multilingual representations have mostly been evaluated based on their\nperformance on specific tasks. In this article, we look beyond engineering\ngoals and analyze the relations between languages in computational\nrepresentations. We introduce a methodology for comparing languages based on\ntheir organization of semantic concepts. We propose to conduct an adapted\nversion of representational similarity analysis of a selected set of concepts\nin computational multilingual representations. Using this analysis method, we\ncan reconstruct a phylogenetic tree that closely resembles those assumed by\nlinguistic experts. These results indicate that multilingual distributional\nrepresentations which are only trained on monolingual text and bilingual\ndictionaries preserve relations between languages without the need for any\netymological information. In addition, we propose a measure to identify\nsemantic drift between language families. We perform experiments on word-based\nand sentence-based multilingual models and provide both quantitative results\nand qualitative examples. Analyses of semantic drift in multilingual\nrepresentations can serve two purposes: they can indicate unwanted\ncharacteristics of the computational models and they provide a quantitative\nmeans to study linguistic phenomena across languages. The code is available at\nhttps://github.com/beinborn/SemanticDrift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11610v1",
            "date": "2019-04-25 22:12:43+00:00",
            "title": "Look Who's Talking: Inferring Speaker Attributes from Personal Longitudinal Dialog",
            "authors": [
                "Charles Welch",
                "Ver\u00f3nica P\u00e9rez-Rosas",
                "Jonathan K. Kummerfeld",
                "Rada Mihalcea"
            ],
            "abstract": "We examine a large dialog corpus obtained from the conversation history of a\nsingle individual with 104 conversation partners. The corpus consists of half a\nmillion instant messages, across several messaging platforms. We focus our\nanalyses on seven speaker attributes, each of which partitions the set of\nspeakers, namely: gender; relative age; family member; romantic partner;\nclassmate; co-worker; and native to the same country. In addition to the\ncontent of the messages, we examine conversational aspects such as the time\nmessages are sent, messaging frequency, psycholinguistic word categories,\nlinguistic mirroring, and graph-based features reflecting how people in the\ncorpus mention each other. We present two sets of experiments predicting each\nattribute using (1) short context windows; and (2) a larger set of messages. We\nfind that using all features leads to gains of 9-14% over using message text\nonly.",
            "journal ref": "Proceedings of the 20th International Conference on Computational\n  Linguistics and Intelligent Text Processing (CICLing 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11679v2",
            "date": "2019-04-26 05:52:05+00:00",
            "title": "Fake News Early Detection: An Interdisciplinary Study",
            "authors": [
                "Xinyi Zhou",
                "Atishay Jain",
                "Vir V. Phoha",
                "Reza Zafarani"
            ],
            "abstract": "Massive dissemination of fake news and its potential to erode democracy has\nincreased the demand for accurate fake news detection. Recent advancements in\nthis area have proposed novel techniques that aim to detect fake news by\nexploring how it propagates on social networks. Nevertheless, to detect fake\nnews at an early stage, i.e., when it is published on a news outlet but not yet\nspread on social media, one cannot rely on news propagation information as it\ndoes not exist. Hence, there is a strong need to develop approaches that can\ndetect fake news by focusing on news content. In this paper, a theory-driven\nmodel is proposed for fake news detection. The method investigates news content\nat various levels: lexicon-level, syntax-level, semantic-level and\ndiscourse-level. We represent news at each level, relying on well-established\ntheories in social and forensic psychology. Fake news detection is then\nconducted within a supervised machine learning framework. As an\ninterdisciplinary research, our work explores potential fake news patterns,\nenhances the interpretability in fake news feature engineering, and studies the\nrelationships among fake news, deception/disinformation, and clickbaits.\nExperiments conducted on two real-world datasets indicate the proposed method\ncan outperform the state-of-the-art and enable fake news early detection when\nthere is limited content information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11816v2",
            "date": "2019-04-26 12:57:25+00:00",
            "title": "Think Again Networks and the Delta Loss",
            "authors": [
                "Alexandre Salle",
                "Marcelo Prates"
            ],
            "abstract": "This short paper introduces an abstraction called Think Again Networks\n(ThinkNet) which can be applied to any state-dependent function (such as a\nrecurrent neural network).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12106v1",
            "date": "2019-04-27 04:36:57+00:00",
            "title": "Understanding Dataset Design Choices for Multi-hop Reasoning",
            "authors": [
                "Jifan Chen",
                "Greg Durrett"
            ],
            "abstract": "Learning multi-hop reasoning has been a key challenge for reading\ncomprehension models, leading to the design of datasets that explicitly focus\non it. Ideally, a model should not be able to perform well on a multi-hop\nquestion answering task without doing multi-hop reasoning. In this paper, we\ninvestigate two recently proposed datasets, WikiHop and HotpotQA. First, we\nexplore sentence-factored models for these tasks; by design, these models\ncannot do multi-hop reasoning, but they are still able to solve a large number\nof examples in both datasets. Furthermore, we find spurious correlations in the\nunmasked version of WikiHop, which make it easy to achieve high performance\nconsidering only the questions and answers. Finally, we investigate one key\ndifference between these datasets, namely span-based vs. multiple-choice\nformulations of the QA task. Multiple-choice versions of both datasets can be\neasily gamed, and two models we examine only marginally exceed a baseline in\nthis setting. Overall, while these datasets are useful testbeds,\nhigh-performing models may not be learning as much multi-hop reasoning as\npreviously thought.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12162v2",
            "date": "2019-04-27 14:46:34+00:00",
            "title": "Sentiment Classification using N-gram IDF and Automated Machine Learning",
            "authors": [
                "Rungroj Maipradit",
                "Hideaki Hata",
                "Kenichi Matsumoto"
            ],
            "abstract": "We propose a sentiment classification method with a general machine learning\nframework. For feature representation, n-gram IDF is used to extract\nsoftware-engineering-related, dataset-specific, positive, neutral, and negative\nn-gram expressions. For classifiers, an automated machine learning tool is\nused. In the comparison using publicly available datasets, our method achieved\nthe highest F1 values in positive and negative sentences on all datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.13264v1",
            "date": "2019-04-30 14:08:37+00:00",
            "title": "Don't Settle for Average, Go for the Max: Fuzzy Sets and Max-Pooled Word Vectors",
            "authors": [
                "Vitalii Zhelezniak",
                "Aleksandar Savkov",
                "April Shen",
                "Francesco Moramarco",
                "Jack Flann",
                "Nils Y. Hammerla"
            ],
            "abstract": "Recent literature suggests that averaged word vectors followed by simple\npost-processing outperform many deep learning methods on semantic textual\nsimilarity tasks. Furthermore, when averaged word vectors are trained\nsupervised on large corpora of paraphrases, they achieve state-of-the-art\nresults on standard STS benchmarks. Inspired by these insights, we push the\nlimits of word embeddings even further. We propose a novel fuzzy bag-of-words\n(FBoW) representation for text that contains all the words in the vocabulary\nsimultaneously but with different degrees of membership, which are derived from\nsimilarities between word vectors. We show that max-pooled word vectors are\nonly a special case of fuzzy BoW and should be compared via fuzzy Jaccard index\nrather than cosine similarity. Finally, we propose DynaMax, a completely\nunsupervised and non-parametric similarity measure that dynamically extracts\nand max-pools good features depending on the sentence pair. This method is both\nefficient and easy to implement, yet outperforms current baselines on STS tasks\nby a large margin and is even competitive with supervised word vectors trained\nto directly optimise cosine similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00079v1",
            "date": "2019-04-30 19:57:47+00:00",
            "title": "FastContext: an efficient and scalable implementation of the ConText algorithm",
            "authors": [
                "Jianlin Shi",
                "John F. Hurdle"
            ],
            "abstract": "Objective: To develop and evaluate FastContext, an efficient, scalable\nimplementation of the ConText algorithm suitable for very large-scale clinical\nnatural language processing. Background: The ConText algorithm performs with\nstate-of-art accuracy in detecting the experiencer, negation status, and\ntemporality of concept mentions in clinical narratives. However, the speed\nlimitation of its current implementations hinders its use in big data\nprocessing. Methods: We developed FastContext through hashing the ConText's\nrules, then compared its speed and accuracy with JavaConText and\nGeneralConText, two widely used Java implementations. Results: FastContext ran\ntwo orders of magnitude faster and was less decelerated by rule increase than\nthe other two implementations used in this study for comparison. Additionally,\nFastContext consistently gained accuracy improvement as the rules increased\n(the desired outcome of adding new rules), while the other two implementations\ndid not. Conclusions: FastContext is an efficient, scalable implementation of\nthe popular ConText algorithm, suitable for natural language applications on\nvery large clinical corpora.",
            "journal ref": "Journal of Biomedical Informatics, August 6, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00198v1",
            "date": "2019-05-01 06:29:02+00:00",
            "title": "Declarative Question Answering over Knowledge Bases containing Natural Language Text with Answer Set Programming",
            "authors": [
                "Arindam Mitra",
                "Peter Clark",
                "Oyvind Tafjord",
                "Chitta Baral"
            ],
            "abstract": "While in recent years machine learning (ML) based approaches have been the\npopular approach in developing end-to-end question answering systems, such\nsystems often struggle when additional knowledge is needed to correctly answer\nthe questions. Proposed alternatives involve translating the question and the\nnatural language text to a logical representation and then use logical\nreasoning. However, this alternative falters when the size of the text gets\nbigger. To address this we propose an approach that does logical reasoning over\npremises written in natural language text. The proposed method uses recent\nfeatures of Answer Set Programming (ASP) to call external NLP modules (which\nmay be based on ML) which perform simple textual entailment. To test our\napproach we develop a corpus based on the life cycle questions and showed that\nOur system achieves up to $18\\%$ performance gain when compared to standard MCQ\nsolvers.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.00245v1",
            "date": "2019-05-01 10:16:46+00:00",
            "title": "Context-Dependent Semantic Parsing over Temporally Structured Data",
            "authors": [
                "Charles Chen",
                "Razvan Bunescu"
            ],
            "abstract": "We describe a new semantic parsing setting that allows users to query the\nsystem using both natural language questions and actions within a graphical\nuser interface. Multiple time series belonging to an entity of interest are\nstored in a database and the user interacts with the system to obtain a better\nunderstanding of the entity's state and behavior, entailing sequences of\nactions and questions whose answers may depend on previous factual or\nnavigational interactions. We design an LSTM-based encoder-decoder architecture\nthat models context dependency through copying mechanisms and multiple levels\nof attention over inputs and previous outputs. When trained to predict tokens\nusing supervised learning, the proposed architecture substantially outperforms\nstandard sequence generation baselines. Training the architecture using policy\ngradient leads to further improvements in performance, reaching a\nsequence-level accuracy of 88.7% on artificial data and 74.8% on real data.",
            "journal ref": "Proceedings of the 2019 Conference of the North American Chapter\n  of the Association for Computational Linguistics: Human Language Technologies\n  (NAACL 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00270v3",
            "date": "2019-05-01 11:32:13+00:00",
            "title": "ASER: A Large-scale Eventuality Knowledge Graph",
            "authors": [
                "Hongming Zhang",
                "Xin Liu",
                "Haojie Pan",
                "Yangqiu Song",
                "Cane Wing-Ki Leung"
            ],
            "abstract": "Understanding human's language requires complex world knowledge. However,\nexisting large-scale knowledge graphs mainly focus on knowledge about entities\nwhile ignoring knowledge about activities, states, or events, which are used to\ndescribe how entities or things act in the real world. To fill this gap, we\ndevelop ASER (activities, states, events, and their relations), a large-scale\neventuality knowledge graph extracted from more than 11-billion-token\nunstructured textual data. ASER contains 15 relation types belonging to five\ncategories, 194-million unique eventualities, and 64-million unique edges among\nthem. Both intrinsic and extrinsic evaluations demonstrate the quality and\neffectiveness of ASER.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.00422v1",
            "date": "2019-05-01 16:04:12+00:00",
            "title": "Time-series Insights into the Process of Passing or Failing Online University Courses using Neural-Induced Interpretable Student States",
            "authors": [
                "Byungsoo Jeon",
                "Eyal Shafran",
                "Luke Breitfeller",
                "Jason Levin",
                "Carolyn P. Rose"
            ],
            "abstract": "This paper addresses a key challenge in Educational Data Mining, namely to\nmodel student behavioral trajectories in order to provide a means for\nidentifying students most at-risk, with the goal of providing supportive\ninterventions. While many forms of data including clickstream data or data from\nsensors have been used extensively in time series models for such purposes, in\nthis paper we explore the use of textual data, which is sometimes available in\nthe records of students at large, online universities. We propose a time series\nmodel that constructs an evolving student state representation using both\nclickstream data and a signal extracted from the textual notes recorded by\nhuman mentors assigned to each student. We explore how the addition of this\ntextual data improves both the predictive power of student states for the\npurpose of identifying students at risk for course failure as well as for\nproviding interpretable insights about student course engagement processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00470v1",
            "date": "2019-05-01 19:49:41+00:00",
            "title": "Semi-automatic System for Title Construction",
            "authors": [
                "Swagata Duari",
                "Vasudha Bhatnagar"
            ],
            "abstract": "In this paper, we propose a semi-automatic system for title construction from\nscientific abstracts. The system extracts and recommends impactful words from\nthe text, which the author can creatively use to construct an appropriate title\nfor the manuscript. The work is based on the hypothesis that keywords are good\ncandidates for title construction. We extract important words from the document\nby inducing a supervised keyword extraction model. The model is trained on\nnovel features extracted from graph-of-text representation of the document. We\nempirically show that these graph-based features are capable of discriminating\nkeywords from non-keywords. We further establish empirically that the proposed\napproach can be applied to any text irrespective of the training domain and\ncorpus. We evaluate the proposed system by computing the overlap between\nextracted keywords and the list of title-words for documents, and we observe a\nmacro-averaged precision of 82%.",
            "journal ref": "Gani A., Das P., Kharb L., Chahal D. (eds) Information,\n  Communication and Computing Technology. ICICCT 2019. Communications in\n  Computer and Information Science, vol 1025, 216-227. Springer, Singapore",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.00537v3",
            "date": "2019-05-02 00:41:50+00:00",
            "title": "SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems",
            "authors": [
                "Alex Wang",
                "Yada Pruksachatkun",
                "Nikita Nangia",
                "Amanpreet Singh",
                "Julian Michael",
                "Felix Hill",
                "Omer Levy",
                "Samuel R. Bowman"
            ],
            "abstract": "In the last year, new models and methods for pretraining and transfer\nlearning have driven striking performance improvements across a range of\nlanguage understanding tasks. The GLUE benchmark, introduced a little over one\nyear ago, offers a single-number metric that summarizes progress on a diverse\nset of such tasks, but performance on the benchmark has recently surpassed the\nlevel of non-expert humans, suggesting limited headroom for further research.\nIn this paper we present SuperGLUE, a new benchmark styled after GLUE with a\nnew set of more difficult language understanding tasks, a software toolkit, and\na public leaderboard. SuperGLUE is available at super.gluebenchmark.com.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00572v2",
            "date": "2019-05-02 05:04:03+00:00",
            "title": "Argument Identification in Public Comments from eRulemaking",
            "authors": [
                "Vlad Eidelman",
                "Brian Grom"
            ],
            "abstract": "Administrative agencies in the United States receive millions of comments\neach year concerning proposed agency actions during the eRulemaking process.\nThese comments represent a diversity of arguments in support and opposition of\nthe proposals. While agencies are required to identify and respond to\nsubstantive comments, they have struggled to keep pace with the volume of\ninformation. In this work we address the tasks of identifying argumentative\ntext, classifying the type of argument claims employed, and determining the\nstance of the comment. First, we propose a taxonomy of argument claims based on\nan analysis of thousands of rules and millions of comments. Second, we collect\nand semi-automatically bootstrap annotations to create a dataset of millions of\nsentences with argument claim type annotation at the sentence level. Third, we\nbuild a system for automatically determining argumentative spans and claim type\nusing our proposed taxonomy in a hierarchical classification model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00840v3",
            "date": "2019-05-02 16:39:55+00:00",
            "title": "Knowledge Authoring and Question Answering with KALM",
            "authors": [
                "Tiantian Gao"
            ],
            "abstract": "Knowledge representation and reasoning (KRR) is one of the key areas in\nartificial intelligence (AI) field. It is intended to represent the world\nknowledge in formal languages (e.g., Prolog, SPARQL) and then enhance the\nexpert systems to perform querying and inference tasks. Currently, constructing\nlarge scale knowledge bases (KBs) with high quality is prohibited by the fact\nthat the construction process requires many qualified knowledge engineers who\nnot only understand the domain-specific knowledge but also have sufficient\nskills in knowledge representation. Unfortunately, qualified knowledge\nengineers are in short supply. Therefore, it would be very useful to build a\ntool that allows the user to construct and query the KB simply via text.\nAlthough there is a number of systems developed for knowledge extraction and\nquestion answering, they mainly fail in that these system don't achieve high\nenough accuracy whereas KRR is highly sensitive to erroneous data. In this\nthesis proposal, I will present Knowledge Authoring Logic Machine (KALM), a\nrule-based system which allows the user to author knowledge and query the KB in\ntext. The experimental results show that KALM achieved superior accuracy in\nknowledge authoring and question answering as compared to the state-of-the-art\nsystems.",
            "journal ref": "EPTCS 306, 2019, pp. 389-395",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.01566v1",
            "date": "2019-05-04 23:22:51+00:00",
            "title": "Learning to Denoise Distantly-Labeled Data for Entity Typing",
            "authors": [
                "Yasumasa Onoe",
                "Greg Durrett"
            ],
            "abstract": "Distantly-labeled data can be used to scale up training of statistical\nmodels, but it is typically noisy and that noise can vary with the distant\nlabeling technique. In this work, we propose a two-stage procedure for handling\nthis type of data: denoise it with a learned model, then train our final model\non clean and denoised distant data with standard supervised training. Our\ndenoising approach consists of two parts. First, a filtering function discards\nexamples from the distantly labeled data that are wholly unusable. Second, a\nrelabeling function repairs noisy labels for the retained examples. Each of\nthese components is a model trained on synthetically-noised examples generated\nfrom a small manually-labeled set. We investigate this approach on the\nultra-fine entity typing task of Choi et al. (2018). Our baseline model is an\nextension of their model with pre-trained ELMo representations, which already\nachieves state-of-the-art performance. Adding distant data that has been\ndenoised with our learned models gives further performance gains over this base\nmodel, outperforming models trained on raw distant data or\nheuristically-denoised distant data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01712v1",
            "date": "2019-05-05 16:18:17+00:00",
            "title": "BVS Corpus: A Multilingual Parallel Corpus of Biomedical Scientific Texts",
            "authors": [
                "Felipe Soares",
                "Martin Krallinger"
            ],
            "abstract": "The BVS database (Health Virtual Library) is a centralized source of\nbiomedical information for Latin America and Carib, created in 1998 and\ncoordinated by BIREME (Biblioteca Regional de Medicina) in agreement with the\nPan American Health Organization (OPAS). Abstracts are available in English,\nSpanish, and Portuguese, with a subset in more than one language, thus being a\npossible source of parallel corpora. In this article, we present the\ndevelopment of parallel corpora from BVS in three languages: English,\nPortuguese, and Spanish. Sentences were automatically aligned using the\nHunalign algorithm for EN/ES and EN/PT language pairs, and for a subset of\ntrilingual articles also. We demonstrate the capabilities of our corpus by\ntraining a Neural Machine Translation (OpenNMT) system for each language pair,\nwhich outperformed related works on scientific biomedical articles. Sentence\nalignment was also manually evaluated, presenting an average 96% of correctly\naligned sentences across all languages. Our parallel corpus is freely\navailable, with complementary information regarding article metadata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01758v1",
            "date": "2019-05-05 22:16:08+00:00",
            "title": "Investigating the Successes and Failures of BERT for Passage Re-Ranking",
            "authors": [
                "Harshith Padigela",
                "Hamed Zamani",
                "W. Bruce Croft"
            ],
            "abstract": "The bidirectional encoder representations from transformers (BERT) model has\nrecently advanced the state-of-the-art in passage re-ranking. In this paper, we\nanalyze the results produced by a fine-tuned BERT model to better understand\nthe reasons behind such substantial improvements. To this aim, we focus on the\nMS MARCO passage re-ranking dataset and provide potential reasons for the\nsuccesses and failures of BERT for retrieval. In more detail, we empirically\nstudy a set of hypotheses and provide additional analysis to explain the\nsuccessful performance of BERT.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.01962v1",
            "date": "2019-04-10 17:43:51+00:00",
            "title": "Harvey Mudd College at SemEval-2019 Task 4: The Clint Buchanan Hyperpartisan News Detector",
            "authors": [
                "Mehdi Drissi",
                "Pedro Sandoval",
                "Vivaswat Ojha",
                "Julie Medero"
            ],
            "abstract": "We investigate the recently developed Bidirectional Encoder Representations\nfrom Transformers (BERT) model for the hyperpartisan news detection task. Using\na subset of hand-labeled articles from SemEval as a validation set, we test the\nperformance of different parameters for BERT models. We find that accuracy from\ntwo different BERT models using different proportions of the articles is\nconsistently high, with our best-performing model on the validation set\nachieving 85% accuracy and the best-performing model on the test set achieving\n77%. We further determined that our model exhibits strong consistency, labeling\nindependent slices of the same article identically. Finally, we find that\nrandomizing the order of word pieces dramatically reduces validation accuracy\n(to approximately 60%), but that shuffling groups of four or more word pieces\nmaintains an accuracy of about 80%, indicating the model mainly gains value\nfrom local context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01965v1",
            "date": "2019-04-25 12:29:06+00:00",
            "title": "Arabic Text Diacritization Using Deep Neural Networks",
            "authors": [
                "Ali Fadel",
                "Ibraheem Tuffaha",
                "Bara' Al-Jawarneh",
                "Mahmoud Al-Ayyoub"
            ],
            "abstract": "Diacritization of Arabic text is both an interesting and a challenging\nproblem at the same time with various applications ranging from speech\nsynthesis to helping students learning the Arabic language. Like many other\ntasks or problems in Arabic language processing, the weak efforts invested into\nthis problem and the lack of available (open-source) resources hinder the\nprogress towards solving this problem. This work provides a critical review for\nthe currently existing systems, measures and resources for Arabic text\ndiacritization. Moreover, it introduces a much-needed free-for-all cleaned\ndataset that can be easily used to benchmark any work on Arabic diacritization.\nExtracted from the Tashkeela Corpus, the dataset consists of 55K lines\ncontaining about 2.3M words. After constructing the dataset, existing tools and\nsystems are tested on it. The results of the experiments show that the neural\nShakkala system significantly outperforms traditional rule-based approaches and\nother closed-source tools with a Diacritic Error Rate (DER) of 2.88% compared\nwith 13.78%, which the best DER for the non-neural approach (obtained by the\nMishkal tool).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01966v2",
            "date": "2019-05-03 01:45:50+00:00",
            "title": "Question Relatedness on Stack Overflow: The Task, Dataset, and Corpus-inspired Models",
            "authors": [
                "Amirreza Shirani",
                "Bowen Xu",
                "David Lo",
                "Thamar Solorio",
                "Amin Alipour"
            ],
            "abstract": "Domain-specific community question answering is becoming an integral part of\nprofessions. Finding related questions and answers in these communities can\nsignificantly improve the effectiveness and efficiency of information seeking.\nStack Overflow is one of the most popular communities that is being used by\nmillions of programmers. In this paper, we analyze the problem of predicting\nknowledge unit (question thread) relatedness in Stack Overflow. In particular,\nwe formulate the question relatedness task as a multi-class classification\nproblem with four degrees of relatedness. We present a large-scale dataset with\nmore than 300K pairs. To the best of our knowledge, this dataset is the largest\ndomain-specific dataset for Question-Question relatedness. We present the steps\nthat we took to collect, clean, process, and assure the quality of the dataset.\nThe proposed dataset Stack Overflow is a useful resource to develop novel\nsolutions, specifically data-hungry neural network models, for the prediction\nof relatedness in technical community question-answering forums. We adopt a\nneural network architecture and a traditional model for this task that\neffectively utilize information from different parts of knowledge units to\ncompute the relatedness between them. These models can be used to benchmark\nnovel models, as they perform well in our task and in a closely similar task.",
            "journal ref": "AAAI 2019 Reasoning for Complex Question Answering Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01969v4",
            "date": "2019-04-22 02:18:00+00:00",
            "title": "Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring",
            "authors": [
                "Samuel Humeau",
                "Kurt Shuster",
                "Marie-Anne Lachaux",
                "Jason Weston"
            ],
            "abstract": "The use of deep pre-trained bidirectional transformers has led to remarkable\nprogress in a number of applications (Devlin et al., 2018). For tasks that make\npairwise comparisons between sequences, matching a given input with a\ncorresponding label, two approaches are common: Cross-encoders performing full\nself-attention over the pair and Bi-encoders encoding the pair separately. The\nformer often performs better, but is too slow for practical use. In this work,\nwe develop a new transformer architecture, the Poly-encoder, that learns global\nrather than token level self-attention features. We perform a detailed\ncomparison of all three approaches, including what pre-training and fine-tuning\nstrategies work best. We show our models achieve state-of-the-art results on\nthree existing tasks; that Poly-encoders are faster than Cross-encoders and\nmore accurate than Bi-encoders; and that the best results are obtained by\npre-training on large datasets similar to the downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01971v1",
            "date": "2019-04-19 18:11:54+00:00",
            "title": "An Evaluation of Transfer Learning for Classifying Sales Engagement Emails at Large Scale",
            "authors": [
                "Yong Liu",
                "Pavel Dmitriev",
                "Yifei Huang",
                "Andrew Brooks",
                "Li Dong"
            ],
            "abstract": "This paper conducts an empirical investigation to evaluate transfer learning\nfor classifying sales engagement emails arising from digital sales engagement\nplatforms. Given the complexity of content and context of sales engagement,\nlack of standardized large corpora and benchmarks, limited labeled examples and\nheterogenous context of intent, this real-world use case poses both a challenge\nand an opportunity for adopting a transfer learning approach. We propose an\nevaluation framework to assess a high performance transfer learning (HPTL)\napproach in three key areas in addition to commonly used accuracy metrics: 1)\neffective embeddings and pretrained language model usage, 2) minimum labeled\nsamples requirement and 3) transfer learning implementation strategies. We use\nin-house sales engagement email samples as the experiment dataset, which\nincludes over 3000 emails labeled as positive, objection, unsubscribe, or\nnot-sure. We discuss our findings on evaluating BERT, ELMo, Flair and GloVe\nembeddings with both feature-based and fine-tuning approaches and their\nscalability on a GPU cluster with increasingly larger labeled samples. Our\nresults show that fine-tuning of the BERT model outperforms with as few as 300\nlabeled samples, but underperforms with fewer than 300 labeled samples,\nrelative to all the feature-based approaches using different embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01978v1",
            "date": "2019-04-17 19:55:20+00:00",
            "title": "CraftAssist Instruction Parsing: Semantic Parsing for a Minecraft Assistant",
            "authors": [
                "Yacine Jernite",
                "Kavya Srinet",
                "Jonathan Gray",
                "Arthur Szlam"
            ],
            "abstract": "We propose a large scale semantic parsing dataset focused on\ninstruction-driven communication with an agent in Minecraft. We describe the\ndata collection process which yields additional 35K human generated\ninstructions with their semantic annotations. We report the performance of\nthree baseline models and find that while a dataset of this size helps us train\na usable instruction parser, it still poses interesting generalization\nchallenges which we hope will help develop better and more robust models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01987v2",
            "date": "2019-05-01 05:10:48+00:00",
            "title": "Disease Identification From Unstructured User Input",
            "authors": [
                "Fahim Faisal",
                "Shafkat Ahmed Bhuiyan",
                "Dr. Abu Raihan Mostofa Kamal"
            ],
            "abstract": "A method to identify probable diseases from the unstructured textual input\n(eg, health forum posts) by incorporating a lexicographic and semantic feature\nbased two-phase text classification module and a symptom-disease\ncorrelation-based similarity measurement module. One notable aspect of my\napproach was to develop a competent algorithm to extract all inherent features\nfrom the data source to make a better decision.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.01988v2",
            "date": "2019-04-30 23:56:54+00:00",
            "title": "Semi-Unsupervised Lifelong Learning for Sentiment Classification: Less Manual Data Annotation and More Self-Studying",
            "authors": [
                "Xianbin Hong",
                "Gautam Pal",
                "Sheng-Uei Guan",
                "Prudence Wong",
                "Dawei Liu",
                "Ka Lok Man",
                "Xin Huang"
            ],
            "abstract": "Lifelong machine learning is a novel machine learning paradigm which can\ncontinually accumulate knowledge during learning. The knowledge extracting and\nreusing abilities enable the lifelong machine learning to solve the related\nproblems. The traditional approaches like Na\\\"ive Bayes and some neural network\nbased approaches only aim to achieve the best performance upon a single task.\nUnlike them, the lifelong machine learning in this paper focuses on how to\naccumulate knowledge during learning and leverage them for further tasks.\nMeanwhile, the demand for labelled data for training also is significantly\ndecreased with the knowledge reusing. This paper suggests that the aim of the\nlifelong learning is to use less labelled data and computational cost to\nachieve the performance as well as or even better than the supervised learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01994v1",
            "date": "2019-04-27 01:57:28+00:00",
            "title": "Review-Driven Answer Generation for Product-Related Questions in E-Commerce",
            "authors": [
                "Shiqian Chen",
                "Chenliang Li",
                "Feng Ji",
                "Wei Zhou",
                "Haiqing Chen"
            ],
            "abstract": "The users often have many product-related questions before they make a\npurchase decision in E-commerce. However, it is often time-consuming to examine\neach user review to identify the desired information. In this paper, we propose\na novel review-driven framework for answer generation for product-related\nquestions in E-commerce, named RAGE. We develope RAGE on the basis of the\nmulti-layer convolutional architecture to facilitate speed-up of answer\ngeneration with the parallel computation. For each question, RAGE first\nextracts the relevant review snippets from the reviews of the corresponding\nproduct. Then, we devise a mechanism to identify the relevant information from\nthe noise-prone review snippets and incorporate this information to guide the\nanswer generation. The experiments on two real-world E-Commerce datasets show\nthat the proposed RAGE significantly outperforms the existing alternatives in\nproducing more accurate and informative answers in natural language. Moreover,\nRAGE takes much less time for both model training and answer generation than\nthe existing RNN based generation models.",
            "journal ref": "WSDM 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01995v1",
            "date": "2019-04-27 12:57:24+00:00",
            "title": "Using Context Information to Enhance Simple Question Answering",
            "authors": [
                "Lin Li",
                "Mengjing Zhang",
                "Zhaohui Chao",
                "Jianwen Xiang"
            ],
            "abstract": "With the rapid development of knowledge bases(KBs),question\nanswering(QA)based on KBs has become a hot research issue. In this paper,we\npropose two frameworks(i.e.,pipeline framework,an end-to-end framework)to focus\nanswering single-relation factoid question. In both of two frameworks,we study\nthe effect of context information on the quality of QA,such as the entity's\nnotable type,out-degree. In the end-to-end framework,we combine char-level\nencoding and self-attention mechanisms,using weight sharing and multi-task\nstrategies to enhance the accuracy of QA. Experimental results show that\ncontext information can get better results of simple QA whether it is the\npipeline framework or the end-to-end framework. In addition,we find that the\nend-to-end framework achieves results competitive with state-of-the-art\napproaches in terms of accuracy and take much shorter time than them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01996v1",
            "date": "2019-04-28 08:27:55+00:00",
            "title": "Neural Machine Translation with Recurrent Highway Networks",
            "authors": [
                "Maulik Parmar",
                "V. Susheela Devi"
            ],
            "abstract": "Recurrent Neural Networks have lately gained a lot of popularity in language\nmodelling tasks, especially in neural machine translation(NMT). Very recent NMT\nmodels are based on Encoder-Decoder, where a deep LSTM based encoder is used to\nproject the source sentence to a fixed dimensional vector and then another deep\nLSTM decodes the target sentence from the vector. However there has been very\nlittle work on exploring architectures that have more than one layer in\nspace(i.e. in each time step). This paper examines the effectiveness of the\nsimple Recurrent Highway Networks(RHN) in NMT tasks. The model uses Recurrent\nHighway Neural Network in encoder and decoder, with attention .We also explore\nthe reconstructor model to improve adequacy. We demonstrate the effectiveness\nof all three approaches on the IWSLT English-Vietnamese dataset. We see that\nRHN performs on par with LSTM based models and even better in some cases.We see\nthat deep RHN models are easy to train compared to deep LSTM based models\nbecause of highway connections. The paper also investigates the effects of\nincreasing recurrent depth in each time step.",
            "journal ref": "In: Groza A., Prasath R. (eds) Mining Intelligence and Knowledge\n  Exploration. MIKE 2018. Lecture Notes in Computer Science, vol 11308.\n  Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02623v1",
            "date": "2019-05-04 02:56:14+00:00",
            "title": "The method of automatic summarization from different sources",
            "authors": [
                "Nataliya Shakhovska",
                "Taras Cherna"
            ],
            "abstract": "In this article is analyzed technology of automatic text abstracting and\nannotation. The role of annotation in automatic search and classification for\ndifferent scientific articles is described. The algorithm of summarization of\nnatural language documents using the concept of importance coefficients is\ndeveloped. Such concept allows considering the peculiarity of subject areas and\ntopics that could be found in different kinds of documents. Method for\ngenerating abstracts of single document based on frequency analysis is\ndeveloped. The recognition elements for unstructured text analysis are given.\nThe method of pre-processing analysis of several documents is developed. This\ntechnique simultaneously considers both statistical approaches to abstracting\nand the importance of terms in a particular subject domain. The quality of\ngenerated abstract is evaluated. For the developed system there was conducted\nexperts evaluation. It was held only for texts in Ukrainian. The developed\nsystem concluding essay has higher aggregate score on all criteria. The\nsummarization system architecture is building. To build an information system\nmodel there is used CASE-tool AllFusion ERwin Data Modeler. The database scheme\nfor information saving was built. The system is designed to work primarily with\nUkrainian texts, which gives a significant advantage, since most modern systems\nstill oriented to English texts",
            "journal ref": "ECONTECHMOD. AN INTERNATIONAL QUARTERLY JOURNAL - 2016. Vol. 5.\n  No. 1. 103-109",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02712v1",
            "date": "2019-05-07 17:53:21+00:00",
            "title": "The Alt-Right and Global Information Warfare",
            "authors": [
                "Emmi Bevensee",
                "Alexander Reid Ross"
            ],
            "abstract": "The Alt-Right is a neo-fascist white supremacist movement that is involved in\nviolent extremism and shows signs of engagement in extensive disinformation\ncampaigns. Using social media data mining, this study develops a deeper\nunderstanding of such targeted disinformation campaigns and the ways they\nspread. It also adds to the available literature on the endogenous and\nexogenous influences within the US far right, as well as motivating factors\nthat drive disinformation campaigns, such as geopolitical strategy. This study\nis to be taken as a preliminary analysis to indicate future methods and\nfollow-on research that will help develop an integrated approach to\nunderstanding the strategies and associations of the modern fascist movement.",
            "journal ref": "2018 IEEE International Conference on Big Data (Big Data),\n  4393-4402",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.02851v2",
            "date": "2019-05-08 00:33:37+00:00",
            "title": "FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance",
            "authors": [
                "Wataru Sakata",
                "Tomohide Shibata",
                "Ribeka Tanaka",
                "Sadao Kurohashi"
            ],
            "abstract": "Frequently Asked Question (FAQ) retrieval is an important task where the\nobjective is to retrieve an appropriate Question-Answer (QA) pair from a\ndatabase based on a user's query. We propose a FAQ retrieval system that\nconsiders the similarity between a user's query and a question as well as the\nrelevance between the query and an answer. Although a common approach to FAQ\nretrieval is to construct labeled data for training, it takes annotation costs.\nTherefore, we use a traditional unsupervised information retrieval system to\ncalculate the similarity between the query and question. On the other hand, the\nrelevance between the query and answer can be learned by using QA pairs in a\nFAQ database. The recently-proposed BERT model is used for the relevance\ncalculation. Since the number of QA pairs in FAQ page is not enough to train a\nmodel, we cope with this issue by leveraging FAQ sets that are similar to the\none in question. We evaluate our approach on two datasets. The first one is\nlocalgovFAQ, a dataset we construct in a Japanese administrative municipality\ndomain. The second is StackExchange dataset, which is the public dataset in\nEnglish. We demonstrate that our proposed method outperforms baseline methods\non these datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.02925v1",
            "date": "2019-05-08 06:01:33+00:00",
            "title": "ShapeGlot: Learning Language for Shape Differentiation",
            "authors": [
                "Panos Achlioptas",
                "Judy Fan",
                "Robert X. D. Hawkins",
                "Noah D. Goodman",
                "Leonidas J. Guibas"
            ],
            "abstract": "In this work we explore how fine-grained differences between the shapes of\ncommon objects are expressed in language, grounded on images and 3D models of\nthe objects. We first build a large scale, carefully controlled dataset of\nhuman utterances that each refers to a 2D rendering of a 3D CAD model so as to\ndistinguish it from a set of shape-wise similar alternatives. Using this\ndataset, we develop neural language understanding (listening) and production\n(speaking) models that vary in their grounding (pure 3D forms via point-clouds\nvs. rendered 2D images), the degree of pragmatic reasoning captured (e.g.\nspeakers that reason about a listener or not), and the neural architecture\n(e.g. with or without attention). We find models that perform well with both\nsynthetic and human partners, and with held out utterances and objects. We also\nfind that these models are amenable to zero-shot transfer learning to novel\nobject classes (e.g. transfer from training on chairs to testing on lamps), as\nwell as to real-world images drawn from furniture catalogs. Lesion studies\nindicate that the neural listeners depend heavily on part-related words and\nassociate these words correctly with visual parts of objects (without any\nexplicit network training on object parts), and that transfer to novel classes\nis most successful when known part-words are available. This work illustrates a\npractical approach to language grounding, and provides a case study in the\nrelationship between object shape and linguistic structure when it comes to\nobject differentiation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02947v1",
            "date": "2019-05-08 07:46:30+00:00",
            "title": "Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances",
            "authors": [
                "Soujanya Poria",
                "Navonil Majumder",
                "Rada Mihalcea",
                "Eduard Hovy"
            ],
            "abstract": "Emotion is intrinsic to humans and consequently emotion understanding is a\nkey part of human-like artificial intelligence (AI). Emotion recognition in\nconversation (ERC) is becoming increasingly popular as a new research frontier\nin natural language processing (NLP) due to its ability to mine opinions from\nthe plethora of publicly available conversational data in platforms such as\nFacebook, Youtube, Reddit, Twitter, and others. Moreover, it has potential\napplications in health-care systems (as a tool for psychological analysis),\neducation (understanding student frustration) and more. Additionally, ERC is\nalso extremely important for generating emotion-aware dialogues that require an\nunderstanding of the user's emotions. Catering to these needs calls for\neffective and scalable conversational emotion-recognition algorithms. However,\nit is a strenuous problem to solve because of several research challenges. In\nthis paper, we discuss these challenges and shed light on the recent research\nin this field. We also describe the drawbacks of these approaches and discuss\nthe reasons why they fail to successfully overcome the research challenges in\nERC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.03640v1",
            "date": "2019-05-09 14:04:44+00:00",
            "title": "Transparency in Maintenance of Recruitment Chatbots",
            "authors": [
                "Kit Kuksenok",
                "Nina Pra\u00df"
            ],
            "abstract": "We report on experiences with implementing conversational agents in the\nrecruitment domain based on a machine learning (ML) system. Recruitment\nchatbots mediate communication between job-seekers and recruiters by exposing\nML data to recruiter teams. Errors are difficult to understand, communicate,\nand resolve because they may span and combine UX, ML, and software issues. In\nan effort to improve organizational and technical transparency, we came to rely\non a key contact role. Though effective for design and development, the\ncentralization of this role poses challenges for transparency in sustained\nmaintenance of this kind of ML-based mediating system.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.03968v3",
            "date": "2019-05-10 06:58:35+00:00",
            "title": "MobiVSR: A Visual Speech Recognition Solution for Mobile Devices",
            "authors": [
                "Nilay Shrivastava",
                "Astitwa Saxena",
                "Yaman Kumar",
                "Rajiv Ratn Shah",
                "Debanjan Mahata",
                "Amanda Stent"
            ],
            "abstract": "Visual speech recognition (VSR) is the task of recognizing spoken language\nfrom video input only, without any audio. VSR has many applications as an\nassistive technology, especially if it could be deployed in mobile devices and\nembedded systems. The need of intensive computational resources and large\nmemory footprint are two of the major obstacles in developing neural network\nmodels for VSR in a resource constrained environment. We propose a novel\nend-to-end deep neural network architecture for word level VSR called MobiVSR\nwith a design parameter that aids in balancing the model's accuracy and\nparameter count. We use depthwise-separable 3D convolution for the first time\nin the domain of VSR and show how it makes our model efficient. MobiVSR\nachieves an accuracy of 73\\% on a challenging Lip Reading in the Wild dataset\nwith 6 times fewer parameters and 20 times lesser memory footprint than the\ncurrent state of the art. MobiVSR can also be compressed to 6 MB by applying\npost training quantization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04226v2",
            "date": "2019-05-10 15:50:00+00:00",
            "title": "Language Modeling with Deep Transformers",
            "authors": [
                "Kazuki Irie",
                "Albert Zeyer",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "We explore deep autoregressive Transformer models in language modeling for\nspeech recognition. We focus on two aspects. First, we revisit Transformer\nmodel configurations specifically for language modeling. We show that well\nconfigured Transformer models outperform our baseline models based on the\nshallow stack of LSTM recurrent neural network layers. We carry out experiments\non the open-source LibriSpeech 960hr task, for both 200K vocabulary word-level\nand 10K byte-pair encoding subword-level language modeling. We apply our\nword-level models to conventional hybrid speech recognition by lattice\nrescoring, and the subword-level models to attention based encoder-decoder\nmodels by shallow fusion. Second, we show that deep Transformer language models\ndo not require positional encoding. The positional encoding is an essential\naugmentation for the self-attention mechanism which is invariant to sequence\nordering. However, in autoregressive setup, as is the case for language\nmodeling, the amount of information increases along the position dimension,\nwhich is a positional signal by its own. The analysis of attention weights\nshows that deep autoregressive self-attention models can automatically make use\nof such positional information. We find that removing the positional encoding\neven slightly improves the performance of these models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04422v1",
            "date": "2019-05-11 02:02:55+00:00",
            "title": "Controlled Natural Languages and Default Reasoning",
            "authors": [
                "Tiantian Gao"
            ],
            "abstract": "Controlled natural languages (CNLs) are effective languages for knowledge\nrepresentation and reasoning. They are designed based on certain natural\nlanguages with restricted lexicon and grammar. CNLs are unambiguous and simple\nas opposed to their base languages. They preserve the expressiveness and\ncoherence of natural languages. In this report, we focus on a class of CNLs,\ncalled machine-oriented CNLs, which have well-defined semantics that can be\ndeterministically translated into formal languages, such as Prolog, to do\nlogical reasoning. Over the past 20 years, a number of machine-oriented CNLs\nemerged and have been used in many application domains for problem solving and\nquestion answering. However, few of them support non-monotonic inference. In\nour work, we propose non-monotonic extensions of CNL to support defeasible\nreasoning.\n  In the first part of this report, we survey CNLs and compare three\ninfluential systems: Attempto Controlled English (ACE), Processable English\n(PENG), and Computer-processable English (CPL). We compare their language\ndesign, semantic interpretations, and reasoning services. In the second part of\nthis report, we first identify typical non-monotonicity in natural languages,\nsuch as defaults, exceptions and conversational implicatures. Then, we propose\ntheir representation in CNL and the corresponding formalizations in a form of\ndefeasible reasoning known as Logic Programming with Defaults and Argumentation\nTheory (LPDA).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.04705v1",
            "date": "2019-05-09 14:17:15+00:00",
            "title": "A joint text mining-rank size investigation of the rhetoric structures of the US Presidents' speeches",
            "authors": [
                "Valerio Ficcadenti",
                "Roy Cerqueti",
                "Marcel Ausloos"
            ],
            "abstract": "This work presents a text mining context and its use for a deep analysis of\nthe messages delivered by the politicians. Specifically, we deal with an expert\nsystems-based exploration of the rhetoric dynamics of a large collection of US\nPresidents' speeches, ranging from Washington to Trump. In particular, speeches\nare viewed as complex expert systems whose structures can be effectively\nanalyzed through rank-size laws. The methodological contribution of the paper\nis twofold. First, we develop a text mining-based procedure for the\nconstruction of the dataset by using a web scraping routine on the Miller\nCenter website -- the repository collecting the speeches. Second, we explore\nthe implicit structure of the discourse data by implementing a rank-size\nprocedure over the individual speeches, being the words of each speech ranked\nin terms of their frequencies. The scientific significance of the proposed\ncombination of text-mining and rank-size approaches can be found in its\nflexibility and generality, which let it be reproducible to a wide set of\nexpert systems and text mining contexts. The usefulness of the proposed method\nand the speech subsequent analysis is demonstrated by the findings themselves.\nIndeed, in terms of impact, it is worth noting that interesting conclusions of\nsocial, political and linguistic nature on how 45 United States Presidents,\nfrom April 30, 1789 till February 28, 2017 delivered political messages can be\ncarried out. Indeed, the proposed analysis shows some remarkable regularities,\nnot only inside a given speech, but also among different speeches. Moreover,\nunder a purely methodological perspective, the presented contribution suggests\npossible ways of generating a linguistic decision-making algorithm.",
            "journal ref": "Expert Systems With Applications 123 (2019) 127-142",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04799v1",
            "date": "2019-05-12 22:20:44+00:00",
            "title": "The Secret Lives of Names? Name Embeddings from Social Media",
            "authors": [
                "Junting Ye",
                "Steven Skiena"
            ],
            "abstract": "Your name tells a lot about you: your gender, ethnicity and so on. It has\nbeen shown that name embeddings are more effective in representing names than\ntraditional substring features. However, our previous name embedding model is\ntrained on private email data and are not publicly accessible. In this paper,\nwe explore learning name embeddings from public Twitter data. We argue that\nTwitter embeddings have two key advantages: \\textit{(i)} they can and will be\npublicly released to support research community. \\textit{(ii)} even with a\nsmaller training corpus, Twitter embeddings achieve similar performances on\nmultiple tasks comparing to email embeddings.\n  As a test case to show the power of name embeddings, we investigate the\nmodeling of lifespans. We find it interesting that adding name embeddings can\nfurther improve the performances of models using demographic features, which\nare traditionally used for lifespan modeling. Through residual analysis, we\nobserve that fine-grained groups (potentially reflecting socioeconomic status)\nare the latent contributing factors encoded in name embeddings. These were\npreviously hidden to demographic models, and may help to enhance the predictive\npower of a wide class of research studies.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.04981v1",
            "date": "2019-05-13 11:40:09+00:00",
            "title": "Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks",
            "authors": [
                "Maolin Li",
                "Arvid Fahlstr\u00f6m Myrman",
                "Tingting Mu",
                "Sophia Ananiadou"
            ],
            "abstract": "When constructing models that learn from noisy labels produced by multiple\nannotators, it is important to accurately estimate the reliability of\nannotators. Annotators may provide labels of inconsistent quality due to their\nvarying expertise and reliability in a domain. Previous studies have mostly\nfocused on estimating each annotator's overall reliability on the entire\nannotation task. However, in practice, the reliability of an annotator may\ndepend on each specific instance. Only a limited number of studies have\ninvestigated modelling per-instance reliability and these only considered\nbinary labels. In this paper, we propose an unsupervised model which can handle\nboth binary and multi-class labels. It can automatically estimate the\nper-instance reliability of each annotator and the correct label for each\ninstance. We specify our model as a probabilistic model which incorporates\nneural networks to model the dependency between latent variables and instances.\nFor evaluation, the proposed method is applied to both synthetic and real data,\nincluding two labelling tasks: text classification and textual entailment.\nExperimental results demonstrate our novel method can not only accurately\nestimate the reliability of annotators across different instances, but also\nachieve superior performance in predicting the correct labels and detecting the\nleast reliable annotators compared to state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05044v2",
            "date": "2019-05-13 14:01:08+00:00",
            "title": "A Review of Keyphrase Extraction",
            "authors": [
                "Eirini Papagiannopoulou",
                "Grigorios Tsoumakas"
            ],
            "abstract": "Keyphrase extraction is a textual information processing task concerned with\nthe automatic extraction of representative and characteristic phrases from a\ndocument that express all the key aspects of its content. Keyphrases constitute\na succinct conceptual summary of a document, which is very useful in digital\ninformation management systems for semantic indexing, faceted search, document\nclustering and classification. This article introduces keyphrase extraction,\nprovides a well-structured review of the existing work, offers interesting\ninsights on the different evaluation approaches, highlights open issues and\npresents a comparative experimental study of popular unsupervised techniques on\nfive datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05471v3",
            "date": "2019-05-14 09:07:30+00:00",
            "title": "Improving Neural Conversational Models with Entropy-Based Data Filtering",
            "authors": [
                "Richard Csaky",
                "Patrik Purgai",
                "Gabor Recski"
            ],
            "abstract": "Current neural network-based conversational models lack diversity and\ngenerate boring responses to open-ended utterances. Priors such as persona,\nemotion, or topic provide additional information to dialog models to aid\nresponse generation, but annotating a dataset with priors is expensive and such\nannotations are rarely available. While previous methods for improving the\nquality of open-domain response generation focused on either the underlying\nmodel or the training objective, we present a method of filtering dialog\ndatasets by removing generic utterances from training data using a simple\nentropy-based approach that does not require human supervision. We conduct\nextensive experiments with different variations of our method, and compare\ndialog models across 17 evaluation metrics to show that training on datasets\nfiltered this way results in better conversational quality as chatbots learn to\noutput more diverse responses.",
            "journal ref": "Proceedings of the 57th Conference of the ACL (2019) 5650-5669",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05475v2",
            "date": "2019-05-14 09:13:23+00:00",
            "title": "Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies",
            "authors": [
                "Yunsu Kim",
                "Yingbo Gao",
                "Hermann Ney"
            ],
            "abstract": "Transfer learning or multilingual model is essential for low-resource neural\nmachine translation (NMT), but the applicability is limited to cognate\nlanguages by sharing their vocabularies. This paper shows effective techniques\nto transfer a pre-trained NMT model to a new, unrelated language without shared\nvocabularies. We relieve the vocabulary mismatch by using cross-lingual word\nembedding, train a more language-agnostic encoder by injecting artificial\nnoises, and generate synthetic data easily from the pre-training data without\nback-translation. Our methods do not require restructuring the vocabulary or\nretraining the model. We improve plain NMT transfer by up to +5.1% BLEU in five\nlow-resource translation tasks, outperforming multilingual joint training by a\nlarge margin. We also provide extensive ablation studies on pre-trained\nembedding, synthetic data, vocabulary size, and parameter freezing for a better\nunderstanding of NMT transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05526v2",
            "date": "2019-05-14 11:39:43+00:00",
            "title": "Is Word Segmentation Necessary for Deep Learning of Chinese Representations?",
            "authors": [
                "Xiaoya Li",
                "Yuxian Meng",
                "Xiaofei Sun",
                "Qinghong Han",
                "Arianna Yuan",
                "Jiwei Li"
            ],
            "abstract": "Segmenting a chunk of text into words is usually the first step of processing\nChinese text, but its necessity has rarely been explored. In this paper, we ask\nthe fundamental question of whether Chinese word segmentation (CWS) is\nnecessary for deep learning-based Chinese Natural Language Processing. We\nbenchmark neural word-based models which rely on word segmentation against\nneural char-based models which do not involve word segmentation in four\nend-to-end NLP benchmark tasks: language modeling, machine translation,\nsentence matching/paraphrase and text classification. Through direct\ncomparisons between these two types of models, we find that char-based models\nconsistently outperform word-based models. Based on these observations, we\nconduct comprehensive experiments to study why word-based models underperform\nchar-based models in these deep learning-based NLP tasks. We show that it is\nbecause word-based models are more vulnerable to data sparsity and the presence\nof out-of-vocabulary (OOV) words, and thus more prone to overfitting. We hope\nthis paper could encourage researchers in the community to rethink the\nnecessity of word segmentation in deep learning-based Chinese Natural Language\nProcessing. \\footnote{Yuxian Meng and Xiaoya Li contributed equally to this\npaper.}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05532v2",
            "date": "2019-05-14 11:44:54+00:00",
            "title": "Atom Responding Machine for Dialog Generation",
            "authors": [
                "Ganbin Zhou",
                "Ping Luo",
                "Jingwu Chen",
                "Fen Lin",
                "Leyu Lin",
                "Qing He"
            ],
            "abstract": "Recently, improving the relevance and diversity of dialogue system has\nattracted wide attention. For a post x, the corresponding response y is usually\ndiverse in the real-world corpus, while the conventional encoder-decoder model\ntends to output the high-frequency (safe but trivial) responses and thus is\ndifficult to handle the large number of responding styles. To address these\nissues, we propose the Atom Responding Machine (ARM), which is based on a\nproposed encoder-composer-decoder network trained by a teacher-student\nframework. To enrich the generated responses, ARM introduces a large number of\nmolecule-mechanisms as various responding styles, which are conducted by taking\ndifferent combinations from a few atom-mechanisms. In other words, even a\nlittle of atom-mechanisms can make a mickle of molecule-mechanisms. The\nexperiments demonstrate diversity and quality of the responses generated by\nARM. We also present generating process to show underlying interpretability for\nthe result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05615v1",
            "date": "2019-05-13 03:18:38+00:00",
            "title": "Transfer Learning for Scientific Data Chain Extraction in Small Chemical Corpus with BERT-CRF Model",
            "authors": [
                "Na Pang",
                "Li Qian",
                "Weimin Lyu",
                "Jin-Dong Yang"
            ],
            "abstract": "Computational chemistry develops fast in recent years due to the rapid growth\nand breakthroughs in AI. Thanks for the progress in natural language\nprocessing, researchers can extract more fine-grained knowledge in publications\nto stimulate the development in computational chemistry. While the works and\ncorpora in chemical entity extraction have been restricted in the biomedicine\nor life science field instead of the chemistry field, we build a new corpus in\nchemical bond field annotated for 7 types of entities: compound, solvent,\nmethod, bond, reaction, pKa and pKa value. This paper presents a novel BERT-CRF\nmodel to build scientific chemical data chains by extracting 7 chemical\nentities and relations from publications. And we propose a joint model to\nextract the entities and relations simultaneously. Experimental results on our\nChemical Special Corpus demonstrate that we achieve state-of-art and\ncompetitive NER performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05644v1",
            "date": "2019-05-14 14:35:06+00:00",
            "title": "Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems",
            "authors": [
                "Fei Mi",
                "Minlie Huang",
                "Jiyong Zhang",
                "Boi Faltings"
            ],
            "abstract": "Natural language generation (NLG) is an essential component of task-oriented\ndialogue systems. Despite the recent success of neural approaches for NLG, they\nare typically developed for particular domains with rich annotated training\nexamples. In this paper, we study NLG in a low-resource setting to generate\nsentences in new scenarios with handful training examples. We formulate the\nproblem from a meta-learning perspective, and propose a generalized\noptimization-based approach (Meta-NLG) based on the well-recognized\nmodel-agnostic meta-learning (MAML) algorithm. Meta-NLG defines a set of meta\ntasks, and directly incorporates the objective of adapting to new low-resource\nNLG tasks into the meta-learning optimization process. Extensive experiments\nare conducted on a large multi-domain dataset (MultiWoz) with diverse\nlinguistic variations. We show that Meta-NLG significantly outperforms other\ntraining procedures in various low-resource configurations. We analyze the\nresults, and demonstrate that Meta-NLG adapts extremely fast and well to\nlow-resource situations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05682v2",
            "date": "2019-05-14 15:54:57+00:00",
            "title": "A Unified Linear-Time Framework for Sentence-Level Discourse Parsing",
            "authors": [
                "Xiang Lin",
                "Shafiq Joty",
                "Prathyusha Jwalapuram",
                "M Saiful Bari"
            ],
            "abstract": "We propose an efficient neural framework for sentence-level discourse\nanalysis in accordance with Rhetorical Structure Theory (RST). Our framework\ncomprises a discourse segmenter to identify the elementary discourse units\n(EDU) in a text, and a discourse parser that constructs a discourse tree in a\ntop-down fashion. Both the segmenter and the parser are based on Pointer\nNetworks and operate in linear time. Our segmenter yields an $F_1$ score of\n95.4, and our parser achieves an $F_1$ score of 81.7 on the aggregated labeled\n(relation) metric, surpassing previous approaches by a good margin and\napproaching human agreement on both tasks (98.3 and 83.0 $F_1$).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05699v1",
            "date": "2019-05-07 21:09:49+00:00",
            "title": "Development of Deep Learning Based Natural Language Processing Model for Turkish",
            "authors": [
                "Baris Baburoglu",
                "Adem Tekerek",
                "Mehmet Tekerek"
            ],
            "abstract": "Natural language is one of the most fundamental features that distinguish\npeople from other living things and enable people to communicate each other.\nLanguage is a tool that enables people to express their feelings and thoughts\nand to transfers cultures through generations. Texts and audio are examples of\nnatural language in daily life. In the natural language, many words disappear\nin time, on the other hand new words are derived. Therefore, while the process\nof natural language processing (NLP) is complex even for human, it is difficult\nto process in computer system. The area of linguistics examines how people use\nlanguage. NLP, which requires the collaboration of linguists and computer\nscientists, plays an important role in human computer interaction. Studies in\nNLP have increased with the use of artificial intelligence technologies in the\nfield of linguistics. With the deep learning methods which are one of the\nartificial intelligence study areas, platforms close to natural language are\nbeing developed. Developed platforms for language comprehension, machine\ntranslation and part of speech (POS) tagging benefit from deep learning\nmethods. Recurrent Neural Network (RNN), one of the deep learning\narchitectures, is preferred for processing sequential data such as text or\naudio data. In this study, Turkish POS tagging model has been proposed by using\nBidirectional Long-Short Term Memory (BLSTM) which is an RNN type. The proposed\nPOS tagging model is provided to natural language researchers with a platform\nthat allows them to perform and use their own analysis. In the development\nphase of the platform developed by using BLSTM, the error rate of the POS\ntagger has been reduced by taking feedback with expert opinion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05702v2",
            "date": "2019-05-14 16:21:34+00:00",
            "title": "Sparse Sequence-to-Sequence Models",
            "authors": [
                "Ben Peters",
                "Vlad Niculae",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Sequence-to-sequence models are a powerful workhorse of NLP. Most variants\nemploy a softmax transformation in both their attention mechanism and output\nlayer, leading to dense alignments and strictly positive output probabilities.\nThis density is wasteful, making models less interpretable and assigning\nprobability mass to many implausible outputs. In this paper, we propose sparse\nsequence-to-sequence models, rooted in a new family of $\\alpha$-entmax\ntransformations, which includes softmax and sparsemax as particular cases, and\nis sparse for any $\\alpha > 1$. We provide fast algorithms to evaluate these\ntransformations and their gradients, which scale well for large vocabulary\nsizes. Our models are able to produce sparse alignments and to assign nonzero\nprobability to a short list of plausible outputs, sometimes rendering beam\nsearch exact. Experiments on morphological inflection and machine translation\nreveal consistent gains over dense models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05708v1",
            "date": "2019-05-12 11:19:25+00:00",
            "title": "The relational processing limits of classic and contemporary neural network models of language processing",
            "authors": [
                "Guillermo Puebla",
                "Andrea E. Martin",
                "Leonidas A. A. Doumas"
            ],
            "abstract": "The ability of neural networks to capture relational knowledge is a matter of\nlong-standing controversy. Recently, some researchers in the PDP side of the\ndebate have argued that (1) classic PDP models can handle relational structure\n(Rogers & McClelland, 2008, 2014) and (2) the success of deep learning\napproaches to text processing suggests that structured representations are\nunnecessary to capture the gist of human language (Rabovsky et al., 2018). In\nthe present study we tested the Story Gestalt model (St. John, 1992), a classic\nPDP model of text comprehension, and a Sequence-to-Sequence with Attention\nmodel (Bahdanau et al., 2015), a contemporary deep learning architecture for\ntext processing. Both models were trained to answer questions about stories\nbased on the thematic roles that several concepts played on the stories. In\nthree critical test we varied the statistical structure of new stories while\nkeeping their relational structure constant with respect to the training data.\nEach model was susceptible to each statistical structure manipulation to a\ndifferent degree, with their performance failing below chance at least under\none manipulation. We argue that the failures of both models are due to the fact\nthat they cannotperform dynamic binding of independent roles and fillers.\nUltimately, these results cast doubts onthe suitability of traditional neural\nnetworks models for explaining phenomena based on relational reasoning,\nincluding language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05709v3",
            "date": "2019-05-13 02:46:28+00:00",
            "title": "Challenges in Building Intelligent Open-domain Dialog Systems",
            "authors": [
                "Minlie Huang",
                "Xiaoyan Zhu",
                "Jianfeng Gao"
            ],
            "abstract": "There is a resurgent interest in developing intelligent open-domain dialog\nsystems due to the availability of large amounts of conversational data and the\nrecent progress on neural approaches to conversational AI. Unlike traditional\ntask-oriented bots, an open-domain dialog system aims to establish long-term\nconnections with users by satisfying the human need for communication,\naffection, and social belonging. This paper reviews the recent works on neural\napproaches that are devoted to addressing three challenges in developing such\nsystems: semantics, consistency, and interactiveness. Semantics requires a\ndialog system to not only understand the content of the dialog but also\nidentify user's social needs during the conversation. Consistency requires the\nsystem to demonstrate a consistent personality to win users trust and gain\ntheir long-term confidence. Interactiveness refers to the system's ability to\ngenerate interpersonal responses to achieve particular social goals such as\nentertainment, conforming, and task completion. The works we select to present\nhere is based on our unique views and are by no means complete. Nevertheless,\nwe hope that the discussion will inspire new research in developing more\nintelligent dialog systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05733v1",
            "date": "2019-05-14 17:27:08+00:00",
            "title": "Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering",
            "authors": [
                "Rajarshi Das",
                "Shehzaad Dhuliawala",
                "Manzil Zaheer",
                "Andrew McCallum"
            ],
            "abstract": "This paper introduces a new framework for open-domain question answering in\nwhich the retriever and the reader iteratively interact with each other. The\nframework is agnostic to the architecture of the machine reading model, only\nrequiring access to the token-level hidden representations of the reader. The\nretriever uses fast nearest neighbor search to scale to corpora containing\nmillions of paragraphs. A gated recurrent unit updates the query at each step\nconditioned on the state of the reader and the reformulated query is used to\nre-rank the paragraphs by the retriever. We conduct analysis and show that\niterative interaction helps in retrieving informative paragraphs from the\ncorpus. Finally, we show that our multi-step-reasoning framework brings\nconsistent improvement when applied to two widely used reader architectures\nDrQA and BiDAF on various large open-domain datasets --- TriviaQA-unfiltered,\nQuasarT, SearchQA, and SQuAD-Open.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05818v1",
            "date": "2019-05-14 20:09:18+00:00",
            "title": "Ontology-Aware Clinical Abstractive Summarization",
            "authors": [
                "Sean MacAvaney",
                "Sajad Sotudeh",
                "Arman Cohan",
                "Nazli Goharian",
                "Ish Talati",
                "Ross W. Filice"
            ],
            "abstract": "Automatically generating accurate summaries from clinical reports could save\na clinician's time, improve summary coverage, and reduce errors. We propose a\nsequence-to-sequence abstractive summarization model augmented with\ndomain-specific ontological information to enhance content selection and\nsummary generation. We apply our method to a dataset of radiology reports and\nshow that it significantly outperforms the current state-of-the-art on this\ntask in terms of rouge scores. Extensive human evaluation conducted by a\nradiologist further indicates that this approach yields summaries that are less\nlikely to omit important details, without sacrificing readability or accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05910v2",
            "date": "2019-05-15 01:47:57+00:00",
            "title": "Passage Ranking with Weak Supervision",
            "authors": [
                "Peng Xu",
                "Xiaofei Ma",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "In this paper, we propose a \\textit{weak supervision} framework for neural\nranking tasks based on the data programming paradigm \\citep{Ratner2016}, which\nenables us to leverage multiple weak supervision signals from different\nsources. Empirically, we consider two sources of weak supervision signals,\nunsupervised ranking functions and semantic feature similarities. We train a\nBERT-based passage-ranking model (which achieves new state-of-the-art\nperformances on two benchmark datasets with full supervision) in our weak\nsupervision framework. Without using ground-truth training labels, BERT-PR\nmodels outperform BM25 baseline by a large margin on all three datasets and\neven beat the previous state-of-the-art results with full supervision on two of\nthe datasets.",
            "journal ref": "ICLR 2019 LLD workshop",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.06139v3",
            "date": "2019-05-15 12:39:49+00:00",
            "title": "Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations",
            "authors": [
                "Fenglin Liu",
                "Yuanxin Liu",
                "Xuancheng Ren",
                "Xiaodong He",
                "Xu Sun"
            ],
            "abstract": "In vision-and-language grounding problems, fine-grained representations of\nthe image are considered to be of paramount importance. Most of the current\nsystems incorporate visual features and textual concepts as a sketch of an\nimage. However, plainly inferred representations are usually undesirable in\nthat they are composed of separate components, the relations of which are\nelusive. In this work, we aim at representing an image with a set of integrated\nvisual regions and corresponding textual concepts, reflecting certain\nsemantics. To this end, we build the Mutual Iterative Attention (MIA) module,\nwhich integrates correlated visual features and textual concepts, respectively,\nby aligning the two modalities. We evaluate the proposed approach on two\nrepresentative vision-and-language grounding tasks, i.e., image captioning and\nvisual question answering. In both tasks, the semantic-grounded image\nrepresentations consistently boost the performance of the baseline models under\nall metrics across the board. The results demonstrate that our approach is\neffective and generalizes well to a wide range of models for image-related\napplications. (The code is available at https://github.com/fenglinliu98/MIA)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06221v4",
            "date": "2019-05-15 14:51:33+00:00",
            "title": "Selection Bias Explorations and Debias Methods for Natural Language Sentence Matching Datasets",
            "authors": [
                "Guanhua Zhang",
                "Bing Bai",
                "Jian Liang",
                "Kun Bai",
                "Shiyu Chang",
                "Mo Yu",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Natural Language Sentence Matching (NLSM) has gained substantial attention\nfrom both academics and the industry, and rich public datasets contribute a lot\nto this process. However, biased datasets can also hurt the generalization\nperformance of trained models and give untrustworthy evaluation results. For\nmany NLSM datasets, the providers select some pairs of sentences into the\ndatasets, and this sampling procedure can easily bring unintended pattern,\ni.e., selection bias. One example is the QuoraQP dataset, where some\ncontent-independent naive features are unreasonably predictive. Such features\nare the reflection of the selection bias and termed as the leakage features. In\nthis paper, we investigate the problem of selection bias on six NLSM datasets\nand find that four out of them are significantly biased. We further propose a\ntraining and evaluation framework to alleviate the bias. Experimental results\non QuoraQP suggest that the proposed framework can improve the generalization\nability of trained models, and give more trustworthy evaluation results for\nreal-world adoptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06401v2",
            "date": "2019-05-14 10:09:14+00:00",
            "title": "Correlating neural and symbolic representations of language",
            "authors": [
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi"
            ],
            "abstract": "Analysis methods which enable us to better understand the representations and\nfunctioning of neural models of language are increasingly needed as deep\nlearning becomes the dominant approach in NLP. Here we present two methods\nbased on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which\nallow us to directly quantify how strongly the information encoded in neural\nactivation patterns corresponds to information represented by symbolic\nstructures such as syntax trees. We first validate our methods on the case of a\nsimple synthetic language for arithmetic expressions with clearly defined\nsyntax and semantics, and show that they exhibit the expected pattern of\nresults. We then apply our methods to correlate neural representations of\nEnglish sentences with their constituency parse trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06407v2",
            "date": "2019-05-15 19:28:10+00:00",
            "title": "Controlled CNN-based Sequence Labeling for Aspect Extraction",
            "authors": [
                "Lei Shu",
                "Hu Xu",
                "Bing Liu"
            ],
            "abstract": "One key task of fine-grained sentiment analysis on reviews is to extract\naspects or features that users have expressed opinions on. This paper focuses\non supervised aspect extraction using a modified CNN called controlled CNN\n(Ctrl). The modified CNN has two types of control modules. Through asynchronous\nparameter updating, it prevents over-fitting and boosts CNN's performance\nsignificantly. This model achieves state-of-the-art results on standard aspect\nextraction datasets. To the best of our knowledge, this is the first paper to\napply control modules to aspect extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06566v1",
            "date": "2019-05-16 07:20:21+00:00",
            "title": "HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization",
            "authors": [
                "Xingxing Zhang",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Neural extractive summarization models usually employ a hierarchical encoder\nfor document encoding and they are trained using sentence-level labels, which\nare created heuristically using rule-based methods. Training the hierarchical\nencoder with these \\emph{inaccurate} labels is challenging. Inspired by the\nrecent work on pre-training transformer sentence encoders\n\\cite{devlin:2018:arxiv}, we propose {\\sc Hibert} (as shorthand for {\\bf\nHI}erachical {\\bf B}idirectional {\\bf E}ncoder {\\bf R}epresentations from {\\bf\nT}ransformers) for document encoding and a method to pre-train it using\nunlabeled data. We apply the pre-trained {\\sc Hibert} to our summarization\nmodel and it outperforms its randomly initialized counterpart by 1.25 ROUGE on\nthe CNN/Dailymail dataset and by 2.0 ROUGE on a version of New York Times\ndataset. We also achieve the state-of-the-art performance on these two\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06596v1",
            "date": "2019-05-16 08:35:12+00:00",
            "title": "Joint Source-Target Self Attention with Locality Constraints",
            "authors": [
                "Jos\u00e9 A. R. Fonollosa",
                "Noe Casas",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "The dominant neural machine translation models are based on the\nencoder-decoder structure, and many of them rely on an unconstrained receptive\nfield over source and target sequences. In this paper we study a new\narchitecture that breaks with both conventions. Our simplified architecture\nconsists in the decoder part of a transformer model, based on self-attention,\nbut with locality constraints applied on the attention receptive field. As\ninput for training, both source and target sentences are fed to the network,\nwhich is trained as a language model. At inference time, the target tokens are\npredicted autoregressively starting with the source sequence as previous\ntokens. The proposed model achieves a new state of the art of 35.7 BLEU on\nIWSLT'14 German-English and matches the best reported results in the literature\non the WMT'14 English-German and WMT'14 English-French translation benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06883v1",
            "date": "2019-05-16 16:15:01+00:00",
            "title": "TraceWalk: Semantic-based Process Graph Embedding for Consistency Checking",
            "authors": [
                "Chen Qian",
                "Lijie Wen",
                "Akhil Kumar"
            ],
            "abstract": "Process consistency checking (PCC), an interdiscipline of natural language\nprocessing (NLP) and business process management (BPM), aims to quantify the\ndegree of (in)consistencies between graphical and textual descriptions of a\nprocess. However, previous studies heavily depend on a great deal of complex\nexpert-defined knowledge such as alignment rules and assessment metrics, thus\nsuffer from the problems of low accuracy and poor adaptability when applied in\nopen-domain scenarios. To address the above issues, this paper makes the first\nattempt that uses deep learning to perform PCC. Specifically, we proposed\nTraceWalk, using semantic information of process graphs to learn latent node\nrepresentations, and integrates it into a convolutional neural network (CNN)\nbased model called TraceNet to predict consistencies. The theoretical proof\nformally provides the PCC's lower limit and experimental results demonstrate\nthat our approach performs more accurately than state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06939v2",
            "date": "2019-05-16 17:57:35+00:00",
            "title": "The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures",
            "authors": [
                "Sheshera Mysore",
                "Zach Jensen",
                "Edward Kim",
                "Kevin Huang",
                "Haw-Shiuan Chang",
                "Emma Strubell",
                "Jeffrey Flanigan",
                "Andrew McCallum",
                "Elsa Olivetti"
            ],
            "abstract": "Materials science literature contains millions of materials synthesis\nprocedures described in unstructured natural language text. Large-scale\nanalysis of these synthesis procedures would facilitate deeper scientific\nunderstanding of materials synthesis and enable automated synthesis planning.\nSuch analysis requires extracting structured representations of synthesis\nprocedures from the raw text as a first step. To facilitate the training and\nevaluation of synthesis extraction models, we introduce a dataset of 230\nsynthesis procedures annotated by domain experts with labeled graphs that\nexpress the semantics of the synthesis sentences. The nodes in this graph are\nsynthesis operations and their typed arguments, and labeled edges specify\nrelations between the nodes. We describe this new resource in detail and\nhighlight some specific challenges to annotating scientific text with shallow\nsemantic structure. We make the corpus available to the community to promote\nfurther research and development of scientific information extraction systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07098v2",
            "date": "2019-05-17 03:00:46+00:00",
            "title": "Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader",
            "authors": [
                "Wenhan Xiong",
                "Mo Yu",
                "Shiyu Chang",
                "Xiaoxiao Guo",
                "William Yang Wang"
            ],
            "abstract": "We propose a new end-to-end question answering model, which learns to\naggregate answer evidence from an incomplete knowledge base (KB) and a set of\nretrieved text snippets. Under the assumptions that the structured KB is easier\nto query and the acquired knowledge can help the understanding of unstructured\ntext, our model first accumulates knowledge of entities from a question-related\nKB subgraph; then reformulates the question in the latent space and reads the\ntexts with the accumulated entity knowledge at hand. The evidence from KB and\ntexts are finally aggregated to predict answers. On the widely-used KBQA\nbenchmark WebQSP, our model achieves consistent improvements across settings\nwith different extents of KB incompleteness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07185v1",
            "date": "2019-05-17 10:14:21+00:00",
            "title": "Plotting Markson's 'Mistress'",
            "authors": [
                "Kelleher Conor",
                "Mark T. Keane"
            ],
            "abstract": "The post-modern novel 'Wittgenstein's Mistress' by David Markson (1988)\npresents the reader with a very challenging non linear narrative, that itself\nappears to one of the novel's themes. We present a distant reading of this work\ndesigned to complement a close reading of it by David Foster Wallace (1990).\nUsing a combination of text analysis, entity recognition and networks, we plot\nrepetitive structures in the novel's narrative relating them to its critical\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07289v1",
            "date": "2019-05-17 14:25:27+00:00",
            "title": "Conversion Prediction Using Multi-task Conditional Attention Networks to Support the Creation of Effective Ad Creative",
            "authors": [
                "Shunsuke Kitada",
                "Hitoshi Iyatomi",
                "Yoshifumi Seki"
            ],
            "abstract": "Accurately predicting conversions in advertisements is generally a\nchallenging task, because such conversions do not occur frequently. In this\npaper, we propose a new framework to support creating high-performing ad\ncreatives, including the accurate prediction of ad creative text conversions\nbefore delivering to the consumer. The proposed framework includes three key\nideas: multi-task learning, conditional attention, and attention highlighting.\nMulti-task learning is an idea for improving the prediction accuracy of\nconversion, which predicts clicks and conversions simultaneously, to solve the\ndifficulty of data imbalance. Furthermore, conditional attention focuses\nattention of each ad creative with the consideration of its genre and target\ngender, thus improving conversion prediction accuracy. Attention highlighting\nvisualizes important words and/or phrases based on conditional attention. We\nevaluated the proposed framework with actual delivery history data (14,000\ncreatives displayed more than a certain number of times from Gunosy Inc.), and\nconfirmed that these ideas improve the prediction performance of conversions,\nand visualize noteworthy words according to the creatives' attributes.",
            "journal ref": "Proceedings of the 25th ACM SIGKDD International Conference on\n  Knowledge Discovery & Data Mining (KDD '19), August 4--8, 2019, Anchorage,\n  AK, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07358v3",
            "date": "2019-05-17 16:27:43+00:00",
            "title": "Learning Cross-lingual Embeddings from Twitter via Distant Supervision",
            "authors": [
                "Jose Camacho-Collados",
                "Yerai Doval",
                "Eugenio Mart\u00ednez-C\u00e1mara",
                "Luis Espinosa-Anke",
                "Francesco Barbieri",
                "Steven Schockaert"
            ],
            "abstract": "Cross-lingual embeddings represent the meaning of words from different\nlanguages in the same vector space. Recent work has shown that it is possible\nto construct such representations by aligning independently learned monolingual\nembedding spaces, and that accurate alignments can be obtained even without\nexternal bilingual data. In this paper we explore a research direction that has\nbeen surprisingly neglected in the literature: leveraging noisy user-generated\ntext to learn cross-lingual embeddings particularly tailored towards social\nmedia applications. While the noisiness and informal nature of the social media\ngenre poses additional challenges to cross-lingual embedding methods, we find\nthat it also provides key opportunities due to the abundance of code-switching\nand the existence of a shared vocabulary of emoji and named entities. Our\ncontribution consists of a very simple post-processing step that exploits these\nphenomena to significantly improve the performance of state-of-the-art\nalignment methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07504v2",
            "date": "2019-05-17 23:52:08+00:00",
            "title": "Story Ending Prediction by Transferable BERT",
            "authors": [
                "Zhongyang Li",
                "Xiao Ding",
                "Ting Liu"
            ],
            "abstract": "Recent advances, such as GPT and BERT, have shown success in incorporating a\npre-trained transformer language model and fine-tuning operation to improve\ndownstream NLP systems. However, this framework still has some fundamental\nproblems in effectively incorporating supervised knowledge from other related\ntasks. In this study, we investigate a transferable BERT (TransBERT) training\nframework, which can transfer not only general language knowledge from\nlarge-scale unlabeled data but also specific kinds of knowledge from various\nsemantically related supervised tasks, for a target task. Particularly, we\npropose utilizing three kinds of transfer tasks, including natural language\ninference, sentiment classification, and next action prediction, to further\ntrain BERT based on a pre-trained model. This enables the model to get a better\ninitialization for the target task. We take story ending prediction as the\ntarget task to conduct experiments. The final result, an accuracy of 91.8%,\ndramatically outperforms previous state-of-the-art baseline methods. Several\ncomparative experiments give some helpful suggestions on how to select transfer\ntasks. Error analysis shows what are the strength and weakness of BERT-based\nmodels for story ending prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07687v1",
            "date": "2019-05-19 04:00:08+00:00",
            "title": "Learning to Memorize in Neural Task-Oriented Dialogue Systems",
            "authors": [
                "Chien-Sheng Wu"
            ],
            "abstract": "In this thesis, we leverage the neural copy mechanism and memory-augmented\nneural networks (MANNs) to address existing challenge of neural task-oriented\ndialogue learning. We show the effectiveness of our strategy by achieving good\nperformance in multi-domain dialogue state tracking, retrieval-based dialogue\nsystems, and generation-based dialogue systems. We first propose a transferable\ndialogue state generator (TRADE) that leverages its copy mechanism to get rid\nof dialogue ontology and share knowledge between domains. We also evaluate\nunseen domain dialogue state tracking and show that TRADE enables zero-shot\ndialogue state tracking and can adapt to new few-shot domains without\nforgetting the previous domains. Second, we utilize MANNs to improve\nretrieval-based dialogue learning. They are able to capture dialogue sequential\ndependencies and memorize long-term information. We also propose a recorded\ndelexicalization copy strategy to replace real entity values with ordered\nentity types. Our models are shown to surpass other retrieval baselines,\nespecially when the conversation has a large number of turns. Lastly, we tackle\ngeneration-based dialogue learning with two proposed models, the\nmemory-to-sequence (Mem2Seq) and global-to-local memory pointer network (GLMP).\nMem2Seq is the first model to combine multi-hop memory attention with the idea\nof the copy mechanism. GLMP further introduces the concept of response\nsketching and double pointers copying. We show that GLMP achieves the\nstate-of-the-art performance on human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07689v1",
            "date": "2019-05-19 05:17:12+00:00",
            "title": "DivGraphPointer: A Graph Pointer Network for Extracting Diverse Keyphrases",
            "authors": [
                "Zhiqing Sun",
                "Jian Tang",
                "Pan Du",
                "Zhi-Hong Deng",
                "Jian-Yun Nie"
            ],
            "abstract": "Keyphrase extraction from documents is useful to a variety of applications\nsuch as information retrieval and document summarization. This paper presents\nan end-to-end method called DivGraphPointer for extracting a set of diversified\nkeyphrases from a document. DivGraphPointer combines the advantages of\ntraditional graph-based ranking methods and recent neural network-based\napproaches. Specifically, given a document, a word graph is constructed from\nthe document based on word proximity and is encoded with graph convolutional\nnetworks, which effectively capture document-level word salience by modeling\nlong-range dependency between words in the document and aggregating multiple\nappearances of identical words into one node. Furthermore, we propose a\ndiversified point network to generate a set of diverse keyphrases out of the\nword graph in the decoding process. Experimental results on five benchmark data\nsets show that our proposed method significantly outperforms the existing\nstate-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08454v1",
            "date": "2019-05-21 06:12:47+00:00",
            "title": "A Seq-to-Seq Transformer Premised Temporal Convolutional Network for Chinese Word Segmentation",
            "authors": [
                "Wei Jiang",
                "Yan Tang"
            ],
            "abstract": "The prevalent approaches of Chinese word segmentation task almost rely on the\nBi-LSTM neural network. However, the methods based the Bi-LSTM have some\ninherent drawbacks: hard to parallel computing, little efficient in applying\nthe Dropout method to inhibit the Overfitting and little efficient in capturing\nthe character information at the more distant site of a long sentence for the\nword segmentation task. In this work, we propose a sequence-to-sequence\ntransformer model for Chinese word segmentation, which is premised a type of\nconvolutional neural network named temporal convolutional network. The model\nuses the temporal convolutional network to construct an encoder, and uses one\nlayer of fully-connected neural network to build a decoder, and applies the\nDropout method to inhibit the Overfitting, and captures the character\ninformation at the distant site of a sentence by adding the layers of the\nencoder, and binds Conditional Random Fields model to train parameters, and\nuses the Viterbi algorithm to infer the final result of the Chinese word\nsegmentation. The experiments on traditional Chinese corpora and simplified\nChinese corpora show that the performance of Chinese word segmentation of the\nmodel is equivalent to the performance of the methods based the Bi-LSTM, and\nthe model has a tremendous growth in parallel computing than the models based\nthe Bi-LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08633v1",
            "date": "2019-05-16 20:38:05+00:00",
            "title": "Fonts-2-Handwriting: A Seed-Augment-Train framework for universal digit classification",
            "authors": [
                "Vinay Uday Prabhu",
                "Sanghyun Han",
                "Dian Ang Yap",
                "Mihail Douhaniaris",
                "Preethi Seshadri",
                "John Whaley"
            ],
            "abstract": "In this paper, we propose a Seed-Augment-Train/Transfer (SAT) framework that\ncontains a synthetic seed image dataset generation procedure for languages with\ndifferent numeral systems using freely available open font file datasets. This\nseed dataset of images is then augmented to create a purely synthetic training\ndataset, which is in turn used to train a deep neural network and test on\nheld-out real world handwritten digits dataset spanning five Indic scripts,\nKannada, Tamil, Gujarati, Malayalam, and Devanagari. We showcase the efficacy\nof this approach both qualitatively, by training a Boundary-seeking GAN (BGAN)\nthat generates realistic digit images in the five languages, and also\nquantitatively by testing a CNN trained on the synthetic data on the real-world\ndatasets. This establishes not only an interesting nexus between the\nfont-datasets-world and transfer learning but also provides a recipe for\nuniversal-digit classification in any script.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.08635v1",
            "date": "2019-05-21 13:41:47+00:00",
            "title": "Generic Multilayer Network Data Analysis with the Fusion of Content and Structure",
            "authors": [
                "Xuan-Son Vu",
                "Abhishek Santra",
                "Sharma Chakravarthy",
                "Lili Jiang"
            ],
            "abstract": "Multi-feature data analysis (e.g., on Facebook, LinkedIn) is challenging\nespecially if one wants to do it efficiently and retain the flexibility by\nchoosing features of interest for analysis. Features (e.g., age, gender,\nrelationship, political view etc.) can be explicitly given from datasets, but\nalso can be derived from content (e.g., political view based on Facebook\nposts). Analysis from multiple perspectives is needed to understand the\ndatasets (or subsets of it) and to infer meaningful knowledge. For example, the\ninfluence of age, location, and marital status on political views may need to\nbe inferred separately (or in combination). In this paper, we adapt multilayer\nnetwork (MLN) analysis, a nontraditional approach, to model the Facebook\ndatasets, integrate content analysis, and conduct analysis, which is driven by\na list of desired application based queries. Our experimental analysis shows\nthe flexibility and efficiency of the proposed approach when modeling and\nanalyzing datasets with multiple features.",
            "journal ref": "Proceedings of the 20th International Conference on Computational\n  Linguistics and Intelligent Text Processing, April, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08743v2",
            "date": "2019-05-21 16:43:54+00:00",
            "title": "Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems",
            "authors": [
                "Chien-Sheng Wu",
                "Andrea Madotto",
                "Ehsan Hosseini-Asl",
                "Caiming Xiong",
                "Richard Socher",
                "Pascale Fung"
            ],
            "abstract": "Over-dependence on domain ontology and lack of knowledge sharing across\ndomains are two practical and yet less studied problems of dialogue state\ntracking. Existing approaches generally fall short in tracking unknown slot\nvalues during inference and often have difficulties in adapting to new domains.\nIn this paper, we propose a Transferable Dialogue State Generator (TRADE) that\ngenerates dialogue states from utterances using a copy mechanism, facilitating\nknowledge transfer when predicting (domain, slot, value) triplets not\nencountered during training. Our model is composed of an utterance encoder, a\nslot gate, and a state generator, which are shared across domains. Empirical\nresults demonstrate that TRADE achieves state-of-the-art joint goal accuracy of\n48.62% for the five domains of MultiWOZ, a human-human dialogue dataset. In\naddition, we show its transferring ability by simulating zero-shot and few-shot\ndialogue state tracking for unseen domains. TRADE achieves 60.58% joint goal\naccuracy in one of the zero-shot domains, and is able to adapt to few-shot\ncases without forgetting already trained domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08941v1",
            "date": "2019-05-22 03:55:54+00:00",
            "title": "Augmenting Data with Mixup for Sentence Classification: An Empirical Study",
            "authors": [
                "Hongyu Guo",
                "Yongyi Mao",
                "Richong Zhang"
            ],
            "abstract": "Mixup, a recent proposed data augmentation method through linearly\ninterpolating inputs and modeling targets of random samples, has demonstrated\nits capability of significantly improving the predictive accuracy of the\nstate-of-the-art networks for image classification. However, how this technique\ncan be applied to and what is its effectiveness on natural language processing\n(NLP) tasks have not been investigated. In this paper, we propose two\nstrategies for the adaption of Mixup on sentence classification: one performs\ninterpolation on word embeddings and another on sentence embeddings. We conduct\nexperiments to evaluate our methods using several benchmark datasets. Our\nstudies show that such interpolation strategies serve as an effective, domain\nindependent data augmentation approach for sentence classification, and can\nresult in significant accuracy improvement for both CNN and LSTM models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08957v2",
            "date": "2019-05-22 05:32:03+00:00",
            "title": "ANTIQUE: A Non-Factoid Question Answering Benchmark",
            "authors": [
                "Helia Hashemi",
                "Mohammad Aliannejadi",
                "Hamed Zamani",
                "W. Bruce Croft"
            ],
            "abstract": "Considering the widespread use of mobile and voice search, answer passage\nretrieval for non-factoid questions plays a critical role in modern information\nretrieval systems. Despite the importance of the task, the community still\nfeels the significant lack of large-scale non-factoid question answering\ncollections with real questions and comprehensive relevance judgments. In this\npaper, we develop and release a collection of 2,626 open-domain non-factoid\nquestions from a diverse set of categories. The dataset, called ANTIQUE,\ncontains 34,011 manual relevance annotations. The questions were asked by real\nusers in a community question answering service, i.e., Yahoo! Answers.\nRelevance judgments for all the answers to each question were collected through\ncrowdsourcing. To facilitate further research, we also include a brief analysis\nof the data as well as baseline results on both classical and recently\ndeveloped neural IR models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.09052v1",
            "date": "2019-05-22 10:13:48+00:00",
            "title": "Retrieving Multi-Entity Associations: An Evaluation of Combination Modes for Word Embeddings",
            "authors": [
                "Gloria Feher",
                "Andreas Spitz",
                "Michael Gertz"
            ],
            "abstract": "Word embeddings have gained significant attention as learnable\nrepresentations of semantic relations between words, and have been shown to\nimprove upon the results of traditional word representations. However, little\neffort has been devoted to using embeddings for the retrieval of entity\nassociations beyond pairwise relations. In this paper, we use popular embedding\nmethods to train vector representations of an entity-annotated news corpus, and\nevaluate their performance for the task of predicting entity participation in\nnews events versus a traditional word cooccurrence network as a baseline. To\nsupport queries for events with multiple participating entities, we test a\nnumber of combination modes for the embedding vectors. While we find that even\nthe best combination modes for word embeddings do not quite reach the\nperformance of the full cooccurrence network, especially for rare entities, we\nobserve that different embedding methods model different types of relations,\nthereby indicating the potential for ensemble methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.09217v1",
            "date": "2019-05-22 16:11:47+00:00",
            "title": "Deeper Text Understanding for IR with Contextual Neural Language Modeling",
            "authors": [
                "Zhuyun Dai",
                "Jamie Callan"
            ],
            "abstract": "Neural networks provide new possibilities to automatically learn complex\nlanguage patterns and query-document relations. Neural IR models have achieved\npromising results in learning query-document relevance patterns, but few\nexplorations have been done on understanding the text content of a query or a\ndocument. This paper studies leveraging a recently-proposed contextual neural\nlanguage model, BERT, to provide deeper text understanding for IR. Experimental\nresults demonstrate that the contextual text representations from BERT are more\neffective than traditional word embeddings. Compared to bag-of-words retrieval\nmodels, the contextual language model can better leverage language structures,\nbringing large improvements on queries written in natural languages. Combining\nthe text understanding ability with search knowledge leads to an enhanced\npre-trained BERT model that can benefit related search tasks where training\ndata are limited.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.09557v1",
            "date": "2019-05-23 09:44:50+00:00",
            "title": "Knowledge Graph Embedding Bi-Vector Models for Symmetric Relation",
            "authors": [
                "Jinkui Yao",
                "Lianghua Xu"
            ],
            "abstract": "Knowledge graph embedding (KGE) models have been proposed to improve the\nperformance of knowledge graph reasoning. However, there is a general\nphenomenon in most of KGEs, as the training progresses, the symmetric relations\ntend to zero vector, if the symmetric triples ratio is high enough in the\ndataset. This phenomenon causes subsequent tasks, e.g. link prediction etc., of\nsymmetric relations to fail. The root cause of the problem is that KGEs do not\nutilize the semantic information of symmetric relations. We propose KGE\nbi-vector models, which represent the symmetric relations as vector pair,\nsignificantly increasing the processing capability of the symmetry relations.\nWe generate the benchmark datasets based on FB15k and WN18 by completing the\nsymmetric relation triples to verify models. The experiment results of our\nmodels clearly affirm the effectiveness and superiority of our models against\nbaseline.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.09642v3",
            "date": "2019-05-23 13:24:15+00:00",
            "title": "An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese",
            "authors": [
                "Enkhbold Bataa",
                "Joshua Wu"
            ],
            "abstract": "Text classification approaches have usually required task-specific model\narchitectures and huge labeled datasets. Recently, thanks to the rise of\ntext-based transfer learning techniques, it is possible to pre-train a language\nmodel in an unsupervised manner and leverage them to perform effective on\ndownstream tasks. In this work we focus on Japanese and show the potential use\nof transfer learning techniques in text classification. Specifically, we\nperform binary and multi-class sentiment classification on the Rakuten product\nreview and Yahoo movie review datasets. We show that transfer learning-based\napproaches perform better than task-specific models trained on 3 times as much\ndata. Furthermore, these approaches perform just as well for language modeling\npre-trained on only 1/30 of the data. We release our pre-trained models and\ncode as open source.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09755v1",
            "date": "2019-05-23 16:28:08+00:00",
            "title": "Misspelling Oblivious Word Embeddings",
            "authors": [
                "Bora Edizel",
                "Aleksandra Piktus",
                "Piotr Bojanowski",
                "Rui Ferreira",
                "Edouard Grave",
                "Fabrizio Silvestri"
            ],
            "abstract": "In this paper we present a method to learn word embeddings that are resilient\nto misspellings. Existing word embeddings have limited applicability to\nmalformed texts, which contain a non-negligible amount of out-of-vocabulary\nwords. We propose a method combining FastText with subwords and a supervised\ntask of learning misspelling patterns. In our method, misspellings of each word\nare embedded close to their correct variants. We train these embeddings on a\nnew dataset we are releasing publicly. Finally, we experimentally show the\nadvantages of this approach on both intrinsic and extrinsic NLP tasks using\npublic test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09998v3",
            "date": "2019-05-24 01:52:31+00:00",
            "title": "Self-Critical Reasoning for Robust Visual Question Answering",
            "authors": [
                "Jialin Wu",
                "Raymond J. Mooney"
            ],
            "abstract": "Visual Question Answering (VQA) deep-learning systems tend to capture\nsuperficial statistical correlations in the training data because of strong\nlanguage priors and fail to generalize to test data with a significantly\ndifferent question-answer (QA) distribution. To address this issue, we\nintroduce a self-critical training objective that ensures that visual\nexplanations of correct answers match the most influential image regions more\nthan other competitive answer candidates. The influential regions are either\ndetermined from human visual/textual explanations or automatically from just\nsignificant words in the question and answer. We evaluate our approach on the\nVQA generalization task using the VQA-CP dataset, achieving a new\nstate-of-the-art i.e., 49.5% using textual explanations and 48.5% using\nautomatically annotated regions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.10033v1",
            "date": "2019-05-24 05:01:14+00:00",
            "title": "Personalizing Dialogue Agents via Meta-Learning",
            "authors": [
                "Zhaojiang Lin",
                "Andrea Madotto",
                "Chien-Sheng Wu",
                "Pascale Fung"
            ],
            "abstract": "Existing personalized dialogue models use human designed persona descriptions\nto improve dialogue consistency. Collecting such descriptions from existing\ndialogues is expensive and requires hand-crafted feature designs. In this\npaper, we propose to extend Model-Agnostic Meta-Learning (MAML)(Finn et al.,\n2017) to personalized dialogue learning without using any persona descriptions.\nOur model learns to quickly adapt to new personas by leveraging only a few\ndialogue samples collected from the same user, which is fundamentally different\nfrom conditioning the response on the persona descriptions. Empirical results\non Persona-chat dataset (Zhang et al., 2018) indicate that our solution\noutperforms non-meta-learning baselines using automatic evaluation metrics, and\nin terms of human-evaluated fluency and consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10072v1",
            "date": "2019-05-24 07:32:37+00:00",
            "title": "mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation",
            "authors": [
                "Dayiheng Liu",
                "Xu Yang",
                "Feng He",
                "Yuanyuan Chen",
                "Jiancheng Lv"
            ],
            "abstract": "It has been previously observed that training Variational Recurrent\nAutoencoders (VRAE) for text generation suffers from serious uninformative\nlatent variables problem. The model would collapse into a plain language model\nthat totally ignore the latent variables and can only generate repeating and\ndull samples. In this paper, we explore the reason behind this issue and\npropose an effective regularizer based approach to address it. The proposed\nmethod directly injects extra constraints on the posteriors of latent variables\ninto the learning process of VRAE, which can flexibly and stably control the\ntrade-off between the KL term and the reconstruction term, making the model\nlearn dense and meaningful latent representations. The experimental results\nshow that the proposed method outperforms several strong baselines and can make\nthe model learn interpretable latent variables and generate diverse meaningful\nsentences. Furthermore, the proposed method can perform well without using\nother strategies, such as KL annealing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10077v3",
            "date": "2019-05-24 07:55:42+00:00",
            "title": "Controlling Risk of Web Question Answering",
            "authors": [
                "Lixin Su",
                "Jiafeng Guo",
                "Yixing Fan",
                "Yanyan Lan",
                "Xueqi Cheng"
            ],
            "abstract": "Web question answering (QA) has become an indispensable component in modern\nsearch systems, which can significantly improve users' search experience by\nproviding a direct answer to users' information need. This could be achieved by\napplying machine reading comprehension (MRC) models over the retrieved passages\nto extract answers with respect to the search query. With the development of\ndeep learning techniques, state-of-the-art MRC performances have been achieved\nby recent deep methods. However, existing studies on MRC seldom address the\npredictive uncertainty issue, i.e., how likely the prediction of an MRC model\nis wrong, leading to uncontrollable risks in real-world Web QA applications. In\nthis work, we first conduct an in-depth investigation over the risk of Web QA.\nWe then introduce a novel risk control framework, which consists of a qualify\nmodel for uncertainty estimation using the probe idea, and a decision model for\nselectively output. For evaluation, we introduce risk-related metrics, rather\nthan the traditional EM and F1 in MRC, for the evaluation of risk-aware Web QA.\nThe empirical results over both the real-world Web QA dataset and the academic\nMRC benchmark collection demonstrate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.10289v2",
            "date": "2019-05-24 15:34:54+00:00",
            "title": "MatchZoo: A Learning, Practicing, and Developing System for Neural Text Matching",
            "authors": [
                "Jiafeng Guo",
                "Yixing Fan",
                "Xiang Ji",
                "Xueqi Cheng"
            ],
            "abstract": "Text matching is the core problem in many natural language processing (NLP)\ntasks, such as information retrieval, question answering, and conversation.\nRecently, deep leaning technology has been widely adopted for text matching,\nmaking neural text matching a new and active research domain. With a large\nnumber of neural matching models emerging rapidly, it becomes more and more\ndifficult for researchers, especially those newcomers, to learn and understand\nthese new models. Moreover, it is usually difficult to try these models due to\nthe tedious data pre-processing, complicated parameter configuration, and\nmassive optimization tricks, not to mention the unavailability of public codes\nsometimes. Finally, for researchers who want to develop new models, it is also\nnot an easy task to implement a neural text matching model from scratch, and to\ncompare with a bunch of existing models. In this paper, therefore, we present a\nnovel system, namely MatchZoo, to facilitate the learning, practicing and\ndesigning of neural text matching models. The system consists of a powerful\nmatching library and a user-friendly and interactive studio, which can help\nresearchers: 1) to learn state-of-the-art neural text matching models\nsystematically, 2) to train, test and apply these models with simple\nconfigurable steps; and 3) to develop their own models with rich APIs and\nassistance.",
            "journal ref": "Proceedings of the 42nd International ACM SIGIR Conference on\n  Research and Development in Information Retrieval (SIGIR '19), July 21--25,\n  2019, Paris, France",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.10348v1",
            "date": "2019-04-20 15:55:13+00:00",
            "title": "Predicting Brazilian court decisions",
            "authors": [
                "Andr\u00e9 Lage-Freitas",
                "H\u00e9ctor Allende-Cid",
                "Orivaldo Santana",
                "L\u00edvia de Oliveira-Lage"
            ],
            "abstract": "Predicting case outcomes is useful but still an extremely hard task for\nattorneys and other Law professionals. It is not easy to search case\ninformation to extract valuable information as this requires dealing with huge\ndata sets and their complexity. For instance, the complexity of Brazil legal\nsystem along with the high litigation rates makes this problem even harder.\nThis paper introduces an approach for predicting Brazilian court decisions\nwhich is also able to predict whether the decision will be unanimous. We\ndeveloped a working prototype which performs 79% of accuracy (F1-score) on a\ndata set composed of 4,043 cases from a Brazilian court. To our knowledge, this\nis the first study to forecast judge decisions in Brazil.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.10425v3",
            "date": "2019-05-24 19:55:34+00:00",
            "title": "Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark",
            "authors": [
                "Nikita Nangia",
                "Samuel R. Bowman"
            ],
            "abstract": "The GLUE benchmark (Wang et al., 2019b) is a suite of language understanding\ntasks which has seen dramatic progress in the past year, with average\nperformance moving from 70.0 at launch to 83.9, state of the art at the time of\nwriting (May 24, 2019). Here, we measure human performance on the benchmark, in\norder to learn whether significant headroom remains for further progress. We\nprovide a conservative estimate of human performance on the benchmark through\ncrowdsourcing: Our annotators are non-experts who must learn each task from a\nbrief set of instructions and 20 examples. In spite of limited training, these\nannotators robustly outperform the state of the art on six of the nine GLUE\ntasks and achieve an average score of 87.1. Given the fast pace of progress\nhowever, the headroom we observe is quite limited. To reproduce the data-poor\nsetting that our annotators must learn in, we also train the BERT model (Devlin\net al., 2019) in limited-data regimes, and conclude that low-resource sentence\nclassification remains a challenge for modern neural network approaches to text\nunderstanding.",
            "journal ref": "ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10625v4",
            "date": "2019-05-25 16:06:42+00:00",
            "title": "ESA: Entity Summarization with Attention",
            "authors": [
                "Dongjun Wei",
                "Yaxin Liu",
                "Fuqing Zhu",
                "Liangjun Zang",
                "Wei Zhou",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Entity summarization aims at creating brief but informative descriptions of\nentities from knowledge graphs. While previous work mostly focused on\ntraditional techniques such as clustering algorithms and graph models, we ask\nhow to apply deep learning methods into this task. In this paper we propose\nESA, a neural network with supervised attention mechanisms for entity\nsummarization. Specifically, we calculate attention weights for facts in each\nentity, and rank facts to generate reliable summaries. We explore techniques to\nsolve difficult learning problems presented by the ESA, and demonstrate the\neffectiveness of our model in comparison with the state-of-the-art methods.\nExperimental results show that our model improves the quality of the entity\nsummaries in both F-measure and MAP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10718v1",
            "date": "2019-05-26 03:33:42+00:00",
            "title": "Hashing based Answer Selection",
            "authors": [
                "Dong Xu",
                "Wu-Jun Li"
            ],
            "abstract": "Answer selection is an important subtask of question answering (QA), where\ndeep models usually achieve better performance. Most deep models adopt\nquestion-answer interaction mechanisms, such as attention, to get vector\nrepresentations for answers. When these interaction based deep models are\ndeployed for online prediction, the representations of all answers need to be\nrecalculated for each question. This procedure is time-consuming for deep\nmodels with complex encoders like BERT which usually have better accuracy than\nsimple encoders. One possible solution is to store the matrix representation\n(encoder output) of each answer in memory to avoid recalculation. But this will\nbring large memory cost. In this paper, we propose a novel method, called\nhashing based answer selection (HAS), to tackle this problem. HAS adopts a\nhashing strategy to learn a binary matrix representation for each answer, which\ncan dramatically reduce the memory cost for storing the matrix representations\nof answers. Hence, HAS can adopt complex encoders like BERT in the model, but\nthe online prediction of HAS is still fast with a low memory cost. Experimental\nresults on three popular answer selection datasets show that HAS can outperform\nexisting models to achieve state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10720v1",
            "date": "2019-05-26 03:40:17+00:00",
            "title": "Gated Group Self-Attention for Answer Selection",
            "authors": [
                "Dong Xu",
                "Jianhui Ji",
                "Haikuan Huang",
                "Hongbo Deng",
                "Wu-Jun Li"
            ],
            "abstract": "Answer selection (answer ranking) is one of the key steps in many kinds of\nquestion answering (QA) applications, where deep models have achieved\nstate-of-the-art performance. Among these deep models, recurrent neural network\n(RNN) based models are most popular, typically with better performance than\nconvolutional neural network (CNN) based models. Nevertheless, it is difficult\nfor RNN based models to capture the information about long-range dependency\namong words in the sentences of questions and answers. In this paper, we\npropose a new deep model, called gated group self-attention (GGSA), for answer\nselection. GGSA is inspired by global self-attention which is originally\nproposed for machine translation and has not been explored in answer selection.\nGGSA tackles the problem of global self-attention that local and global\ninformation cannot be well distinguished. Furthermore, an interaction mechanism\nbetween questions and answers is also proposed to enhance GGSA by a residual\nstructure. Experimental results on two popular QA datasets show that GGSA can\noutperform existing answer selection models to achieve state-of-the-art\nperformance. Furthermore, GGSA can also achieve higher accuracy than global\nself-attention for the answer selection task, with a lower computation cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10851v1",
            "date": "2019-05-26 18:40:06+00:00",
            "title": "When to reply? Context Sensitive Models to Predict Instructor Interventions in MOOC Forums",
            "authors": [
                "Muthu Kumar Chandrasekaran",
                "Min-Yen Kan"
            ],
            "abstract": "Due to time constraints, course instructors often need to selectively\nparticipate in student discussion threads, due to their limited bandwidth and\nlopsided student--instructor ratio on online forums. We propose the first deep\nlearning models for this binary prediction problem. We propose novel attention\nbased models to infer the amount of latent context necessary to predict\ninstructor intervention. Such models also allow themselves to be tuned to\ninstructor's preference to intervene early or late. Our three proposed\nattentive model variants to infer the latent context improve over the\nstate-of-the-art by a significant, large margin of 11% in F1 and 10% in recall,\non average. Further, introspection of attention help us better understand what\naspects of a discussion post propagate through the discussion thread that\nprompts instructor intervention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10957v3",
            "date": "2019-05-27 03:35:30+00:00",
            "title": "Enhancing Item Response Theory for Cognitive Diagnosis",
            "authors": [
                "Song Cheng",
                "Qi Liu"
            ],
            "abstract": "Cognitive diagnosis is a fundamental and crucial task in many educational\napplications, e.g., computer adaptive test and cognitive assignments. Item\nResponse Theory (IRT) is a classical cognitive diagnosis method which can\nprovide interpretable parameters (i.e., student latent trait, question\ndiscrimination, and difficulty) for analyzing student performance. However,\ntraditional IRT ignores the rich information in question texts, cannot diagnose\nknowledge concept proficiency, and it is inaccurate to diagnose the parameters\nfor the questions which only appear several times. To this end, in this paper,\nwe propose a general Deep Item Response Theory (DIRT) framework to enhance\ntraditional IRT for cognitive diagnosis by exploiting semantic representation\nfrom question texts with deep learning. In DIRT, we first use a proficiency\nvector to represent students' proficiency in knowledge concepts and embed\nquestion texts and knowledge concepts to dense vectors by Word2Vec. Then, we\ndesign a deep diagnosis module to diagnose parameters in traditional IRT by\ndeep learning techniques. Finally, with the diagnosed parameters, we input them\ninto the logistic-like formula of IRT to predict student performance. Extensive\nexperimental results on real-world data clearly demonstrate the effectiveness\nand interpretation power of DIRT framework.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.11006v2",
            "date": "2019-05-27 07:08:12+00:00",
            "title": "Levenshtein Transformer",
            "authors": [
                "Jiatao Gu",
                "Changhan Wang",
                "Jake Zhao"
            ],
            "abstract": "Modern neural sequence generation models are built to either generate tokens\nstep-by-step from scratch or (iteratively) modify a sequence of tokens bounded\nby a fixed length. In this work, we develop Levenshtein Transformer, a new\npartially autoregressive model devised for more flexible and amenable sequence\ngeneration. Unlike previous approaches, the atomic operations of our model are\ninsertion and deletion. The combination of them facilitates not only generation\nbut also sequence refinement allowing dynamic length changes. We also propose a\nset of new training techniques dedicated at them, effectively exploiting one as\nthe other's learning signal thanks to their complementary nature. Experiments\napplying the proposed model achieve comparable performance but much-improved\nefficiency on both generation (e.g. machine translation, text summarization)\nand refinement tasks (e.g. automatic post-editing). We further confirm the\nflexibility of our model by showing a Levenshtein Transformer trained by\nmachine translation can straightforwardly be used for automatic post-editing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11499v1",
            "date": "2019-04-26 06:29:29+00:00",
            "title": "One-Shot Learning for Text-to-SQL Generation",
            "authors": [
                "Dongjun Lee",
                "Jaesik Yoon",
                "Jongyun Song",
                "Sanggil Lee",
                "Sungroh Yoon"
            ],
            "abstract": "Most deep learning approaches for text-to-SQL generation are limited to the\nWikiSQL dataset, which only supports very simple queries. Recently,\ntemplate-based and sequence-to-sequence approaches were proposed to support\ncomplex queries, which contain join queries, nested queries, and other types.\nHowever, Finegan-Dollak et al. (2018) demonstrated that both the approaches\nlack the ability to generate SQL of unseen templates. In this paper, we propose\na template-based one-shot learning model for the text-to-SQL generation so that\nthe model can generate SQL of an untrained template based on a single example.\nFirst, we classify the SQL template using the Matching Network that is\naugmented by our novel architecture Candidate Search Network. Then, we fill the\nvariable slots in the predicted template using the Pointer Network. We show\nthat our model outperforms state-of-the-art approaches for various text-to-SQL\ndatasets in two aspects: 1) the SQL generation accuracy for the trained\ntemplates, and 2) the adaptability to the unseen SQL templates based on a\nsingle example without any additional training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11531v1",
            "date": "2019-05-27 22:51:39+00:00",
            "title": "Compositional pre-training for neural semantic parsing",
            "authors": [
                "Amir Ziai"
            ],
            "abstract": "Semantic parsing is the process of translating natural language utterances\ninto logical forms, which has many important applications such as question\nanswering and instruction following. Sequence-to-sequence models have been very\nsuccessful across many NLP tasks. However, a lack of task-specific prior\nknowledge can be detrimental to the performance of these models. Prior work has\nused frameworks for inducing grammars over the training examples, which capture\nconditional independence properties that the model can leverage. Inspired by\nthe recent success stories such as BERT we set out to extend this augmentation\nframework into two stages. The first stage is to pre-train using a corpus of\naugmented examples in an unsupervised manner. The second stage is to fine-tune\nto a domain-specific task. In addition, since the pre-training stage is\nseparate from the training on the main task we also expand the universe of\npossible augmentations without causing catastrophic inference. We also propose\na novel data augmentation strategy that interchanges tokens that co-occur in\nsimilar contexts to produce new training pairs. We demonstrate that the\nproposed two-stage framework is beneficial for improving the parsing accuracy\nin a standard dataset called GeoQuery for the task of generating logical forms\nfrom a set of questions about the US geography.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11605v3",
            "date": "2019-05-28 04:37:49+00:00",
            "title": "Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network",
            "authors": [
                "Kun Xu",
                "Liwei Wang",
                "Mo Yu",
                "Yansong Feng",
                "Yan Song",
                "Zhiguo Wang",
                "Dong Yu"
            ],
            "abstract": "Previous cross-lingual knowledge graph (KG) alignment studies rely on entity\nembeddings derived only from monolingual KG structural information, which may\nfail at matching entities that have different facts in two KGs. In this paper,\nwe introduce the topic entity graph, a local sub-graph of an entity, to\nrepresent entities with their contextual information in KG. From this view, the\nKB-alignment task can be formulated as a graph matching problem; and we further\npropose a graph-attention based solution, which first matches all entities in\ntwo topic entity graphs, and then jointly model the local matching information\nto derive a graph-level matching vector. Experiments show that our model\noutperforms previous state-of-the-art methods by a large margin.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.11716v1",
            "date": "2019-05-28 10:07:01+00:00",
            "title": "Extracting adverse drug reactions and their context using sequence labelling ensembles in TAC2017",
            "authors": [
                "Maksim Belousov",
                "Nikola Milosevic",
                "William Dixon",
                "Goran Nenadic"
            ],
            "abstract": "Adverse drug reactions (ADRs) are unwanted or harmful effects experienced\nafter the administration of a certain drug or a combination of drugs,\npresenting a challenge for drug development and drug administration. In this\npaper, we present a set of taggers for extracting adverse drug reactions and\nrelated entities, including factors, severity, negations, drug class and\nanimal. The systems used a mix of rule-based, machine learning (CRF) and deep\nlearning (BLSTM with word2vec embeddings) methodologies in order to annotate\nthe data. The systems were submitted to adverse drug reaction shared task,\norganised during Text Analytics Conference in 2017 by National Institute for\nStandards and Technology, archiving F1-scores of 76.00 and 75.61 respectively.",
            "journal ref": "Text Analytics Conference 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11871v1",
            "date": "2019-05-28 15:10:30+00:00",
            "title": "Miss Tools and Mr Fruit: Emergent communication in agents learning about object affordances",
            "authors": [
                "Diane Bouchacourt",
                "Marco Baroni"
            ],
            "abstract": "Recent research studies communication emergence in communities of deep\nnetwork agents assigned a joint task, hoping to gain insights on human language\nevolution. We propose here a new task capturing crucial aspects of the human\nenvironment, such as natural object affordances, and of human conversation,\nsuch as full symmetry among the participants. By conducting a thorough\npragmatic and semantic analysis of the emergent protocol, we show that the\nagents solve the shared task through genuine bilateral, referential\ncommunication. However, the agents develop multiple idiolects, which makes us\nconclude that full symmetry is not a sufficient condition for a common language\nto emerge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11975v4",
            "date": "2019-05-28 17:49:47+00:00",
            "title": "On Variational Learning of Controllable Representations for Text without Supervision",
            "authors": [
                "Peng Xu",
                "Jackie Chi Kit Cheung",
                "Yanshuai Cao"
            ],
            "abstract": "The variational autoencoder (VAE) can learn the manifold of natural images on\ncertain datasets, as evidenced by meaningful interpolating or extrapolating in\nthe continuous latent space. However, on discrete data such as text, it is\nunclear if unsupervised learning can discover similar latent space that allows\ncontrollable manipulation. In this work, we find that sequence VAEs trained on\ntext fail to properly decode when the latent codes are manipulated, because the\nmodified codes often land in holes or vacant regions in the aggregated\nposterior latent space, where the decoding network fails to generalize. Both as\na validation of the explanation and as a fix to the problem, we propose to\nconstrain the posterior mean to a learned probability simplex, and performs\nmanipulation within this simplex. Our proposed method mitigates the latent\nvacancy problem and achieves the first success in unsupervised learning of\ncontrollable representations for text. Empirically, our method outperforms\nunsupervised baselines and strong supervised approaches on text style transfer,\nand is capable of performing more flexible fine-grained control over text\ngeneration than existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12099v1",
            "date": "2019-05-28 21:32:02+00:00",
            "title": "Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae",
            "authors": [
                "Piero Molino",
                "Yang Wang",
                "Jiawei Zhang"
            ],
            "abstract": "Embeddings are a fundamental component of many modern machine learning and\nnatural language processing models. Understanding them and visualizing them is\nessential for gathering insights about the information they capture and the\nbehavior of the models. State of the art in analyzing embeddings consists in\nprojecting them in two-dimensional planes without any interpretable semantics\nassociated to the axes of the projection, which makes detailed analyses and\ncomparison among multiple sets of embeddings challenging. In this work, we\npropose to use explicit axes defined as algebraic formulae over embeddings to\nproject them into a lower dimensional, but semantically meaningful subspace, as\na simple yet effective analysis and visualization methodology. This methodology\nassigns an interpretable semantics to the measures of variability and the axes\nof visualizations, allowing for both comparisons among different sets of\nembeddings and fine-grained inspection of the embedding spaces. We demonstrate\nthe power of the proposed methodology through a series of case studies that\nmake use of visualizations constructed around the underlying methodology and\nthrough a user study. The results show how the methodology is effective at\nproviding more profound insights than classical projection methods and how it\nis widely applicable to many other use cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12188v1",
            "date": "2019-05-29 02:50:50+00:00",
            "title": "Exploiting Persona Information for Diverse Generation of Conversational Responses",
            "authors": [
                "Haoyu Song",
                "Wei-Nan Zhang",
                "Yiming Cui",
                "Dong Wang",
                "Ting Liu"
            ],
            "abstract": "In human conversations, due to their personalities in mind, people can easily\ncarry out and maintain the conversations. Giving conversational context with\npersona information to a chatbot, how to exploit the information to generate\ndiverse and sustainable conversations is still a non-trivial task. Previous\nwork on persona-based conversational models successfully make use of predefined\npersona information and have shown great promise in delivering more realistic\nresponses. And they all learn with the assumption that given a source input,\nthere is only one target response. However, in human conversations, there are\nmassive appropriate responses to a given input message. In this paper, we\npropose a memory-augmented architecture to exploit persona information from\ncontext and incorporate a conditional variational autoencoder model together to\ngenerate diverse and sustainable conversations. We evaluate the proposed model\non a benchmark persona-chat dataset. Both automatic and human evaluations show\nthat our model can deliver more diverse and more engaging persona-based\nresponses than baseline approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.12255v3",
            "date": "2019-05-29 07:40:38+00:00",
            "title": "Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation",
            "authors": [
                "Vihan Jain",
                "Gabriel Magalhaes",
                "Alexander Ku",
                "Ashish Vaswani",
                "Eugene Ie",
                "Jason Baldridge"
            ],
            "abstract": "Advances in learning and representations have reinvigorated work that\nconnects language to other modalities. A particularly exciting direction is\nVision-and-Language Navigation(VLN), in which agents interpret natural language\ninstructions and visual scenes to move through environments and reach goals.\nDespite recent progress, current research leaves unclear how much of a role\nlanguage understanding plays in this task, especially because dominant\nevaluation metrics have focused on goal completion rather than the sequence of\nactions corresponding to the instructions. Here, we highlight shortcomings of\ncurrent metrics for the Room-to-Room dataset (Anderson et al.,2018b) and\npropose a new metric, Coverage weighted by Length Score (CLS). We also show\nthat the existing paths in the dataset are not ideal for evaluating instruction\nfollowing because they are direct-to-goal shortest paths. We join existing\nshort paths to form more challenging extended paths to create a new data set,\nRoom-for-Room (R4R). Using R4R and CLS, we show that agents that receive\nrewards for instruction fidelity outperform agents that focus on goal\ncompletion.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.12516v1",
            "date": "2019-05-29 15:12:58+00:00",
            "title": "Racial Bias in Hate Speech and Abusive Language Detection Datasets",
            "authors": [
                "Thomas Davidson",
                "Debasmita Bhattacharya",
                "Ingmar Weber"
            ],
            "abstract": "Technologies for abusive language detection are being developed and applied\nwith little consideration of their potential biases. We examine racial bias in\nfive different sets of Twitter data annotated for hate speech and abusive\nlanguage. We train classifiers on these datasets and compare the predictions of\nthese classifiers on tweets written in African-American English with those\nwritten in Standard American English. The results show evidence of systematic\nracial bias in all datasets, as classifiers trained on them tend to predict\nthat tweets written in African-American English are abusive at substantially\nhigher rates. If these abusive language detection systems are used in the field\nthey will therefore have a disproportionate negative impact on African-American\nsocial media users. Consequently, these systems may discriminate against the\ngroups who are often the targets of the abuse we are trying to detect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12595v1",
            "date": "2019-05-29 17:14:24+00:00",
            "title": "Predicting next shopping stage using Google Analytics data for E-commerce applications",
            "authors": [
                "Mihai Cristian P\u00eervu",
                "Alexandra Anghel"
            ],
            "abstract": "E-commerce web applications are almost ubiquitous in our day to day life,\nhowever as useful as they are, most of them have little to no adaptation to\nuser needs, which in turn can cause both lower conversion rates as well as\nunsatisfied customers. We propose a machine learning system which learns the\nuser behaviour from multiple previous sessions and predicts useful metrics for\nthe current session. In turn, these metrics can be used by the applications to\ncustomize and better target the customer, which can mean anything from offering\nbetter offers of specific products, targeted notifications or placing smart\nads. The data used for the learning algorithm is extracted from Google\nAnalytics Enhanced E-commerce, which is enabled by most e-commerce websites and\nthus the system can be used by any such merchant. In order to learn the user\npatterns, only its behaviour features were used, which don't include names,\ngender or any other personal information that could identify the user. The\nlearning model that was used is a double recurrent neural network which learns\nboth intra-session and inter-session features. The model predicts for each\nsession a probability score for each of the defined target classes.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.12616v3",
            "date": "2019-05-29 17:58:52+00:00",
            "title": "Defending Against Neural Fake News",
            "authors": [
                "Rowan Zellers",
                "Ari Holtzman",
                "Hannah Rashkin",
                "Yonatan Bisk",
                "Ali Farhadi",
                "Franziska Roesner",
                "Yejin Choi"
            ],
            "abstract": "Recent progress in natural language generation has raised dual-use concerns.\nWhile applications like summarization and translation are positive, the\nunderlying technology also might enable adversaries to generate neural fake\nnews: targeted propaganda that closely mimics the style of real news.\n  Modern computer security relies on careful threat modeling: identifying\npotential threats and vulnerabilities from an adversary's point of view, and\nexploring potential mitigations to these threats. Likewise, developing robust\ndefenses against neural fake news requires us first to carefully investigate\nand characterize the risks of these models. We thus present a model for\ncontrollable text generation called Grover. Given a headline like `Link Found\nBetween Vaccines and Autism,' Grover can generate the rest of the article;\nhumans find these generations to be more trustworthy than human-written\ndisinformation.\n  Developing robust verification techniques against generators like Grover is\ncritical. We find that best current discriminators can classify neural fake\nnews from real, human-written, news with 73% accuracy, assuming access to a\nmoderate level of training data. Counterintuitively, the best defense against\nGrover turns out to be Grover itself, with 92% accuracy, demonstrating the\nimportance of public release of strong generators. We investigate these results\nfurther, showing that exposure bias -- and sampling strategies that alleviate\nits effects -- both leave artifacts that similar discriminators can pick up on.\nWe conclude by discussing ethical issues regarding the technology, and plan to\nrelease Grover publicly, helping pave the way for better detection of neural\nfake news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12866v3",
            "date": "2019-05-30 05:57:27+00:00",
            "title": "Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention",
            "authors": [
                "Wenhu Chen",
                "Jianshu Chen",
                "Pengda Qin",
                "Xifeng Yan",
                "William Yang Wang"
            ],
            "abstract": "Semantically controlled neural response generation on limited-domain has\nachieved great performance. However, moving towards multi-domain large-scale\nscenarios are shown to be difficult because the possible combinations of\nsemantic inputs grow exponentially with the number of domains. To alleviate\nsuch scalability issue, we exploit the structure of dialog acts to build a\nmulti-layer hierarchical graph, where each act is represented as a root-to-leaf\nroute on the graph. Then, we incorporate such graph structure prior as an\ninductive bias to build a hierarchical disentangled self-attention network,\nwhere we disentangle attention heads to model designated nodes on the dialog\nact graph. By activating different (disentangled) heads at each layer,\ncombinatorially many dialog act semantics can be modeled to control the neural\nresponse generation. On the large-scale Multi-Domain-WOZ dataset, our model can\nyield a significant improvement over the baselines on various automatic and\nhuman evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12884v1",
            "date": "2019-05-30 07:07:52+00:00",
            "title": "M-GWAP: An Online and Multimodal Game With A Purpose in WordPress for Mental States Annotation",
            "authors": [
                "Fabio Paolizzo"
            ],
            "abstract": "M-GWAP is a multimodal game with a purpose of that leverages on the wisdom of\ncrowds phenomenon for the annotation of multimedia data in terms of mental\nstates. This game with a purpose is developed in WordPress to allow users\nimplementing the game without programming skills. The game adopts motivational\nstrategies for the player to remain engaged, such as a score system, text\nmotivators while playing, a ranking system to foster competition and mechanics\nfor identify building. The current version of the game was deployed after alpha\nand beta testing helped refining the game accordingly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12897v2",
            "date": "2019-05-30 07:44:34+00:00",
            "title": "A Compare-Aggregate Model with Latent Clustering for Answer Selection",
            "authors": [
                "Seunghyun Yoon",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Trung Bui",
                "Kyomin Jung"
            ],
            "abstract": "In this paper, we propose a novel method for a sentence-level\nanswer-selection task that is a fundamental problem in natural language\nprocessing. First, we explore the effect of additional information by adopting\na pretrained language model to compute the vector representation of the input\ntext and by applying transfer learning from a large-scale corpus. Second, we\nenhance the compare-aggregate model by proposing a novel latent clustering\nmethod to compute additional information within the target corpus and by\nchanging the objective function from listwise to pointwise. To evaluate the\nperformance of the proposed approaches, experiments are performed with the\nWikiQA and TREC-QA datasets. The empirical results demonstrate the superiority\nof our proposed approach, which achieve state-of-the-art performance for both\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12980v1",
            "date": "2019-05-30 11:47:04+00:00",
            "title": "Interactive-predictive neural multimodal systems",
            "authors": [
                "\u00c1lvaro Peris",
                "Francisco Casacuberta"
            ],
            "abstract": "Despite the advances achieved by neural models in sequence to sequence\nlearning, exploited in a variety of tasks, they still make errors. In many use\ncases, these are corrected by a human expert in a posterior revision process.\nThe interactive-predictive framework aims to minimize the human effort spent on\nthis process by considering partial corrections for iteratively refining the\nhypothesis. In this work, we generalize the interactive-predictive approach,\ntypically applied in to machine translation field, to tackle other multimodal\nproblems namely, image and video captioning. We study the application of this\nframework to multimodal neural sequence to sequence models. We show that,\nfollowing this framework, we approximately halve the effort spent for\ncorrecting the outputs generated by the automatic systems. Moreover, we deploy\nour systems in a publicly accessible demonstration, that allows to better\nunderstand the behavior of the interactive-predictive framework.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.13164v1",
            "date": "2019-05-30 16:49:11+00:00",
            "title": "Hierarchical Transformers for Multi-Document Summarization",
            "authors": [
                "Yang Liu",
                "Mirella Lapata"
            ],
            "abstract": "In this paper, we develop a neural summarization model which can effectively\nprocess multiple input documents and distill Transformer architecture with the\nability to encode documents in a hierarchical manner. We represent\ncross-document relationships via an attention mechanism which allows to share\ninformation as opposed to simply concatenating text spans and processing them\nas a flat sequence. Our model learns latent dependencies among textual units,\nbut can also take advantage of explicit graph representations focusing on\nsimilarity or discourse relations. Empirical results on the WikiSum dataset\ndemonstrate that the proposed architecture brings substantial improvements over\nseveral strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13358v1",
            "date": "2019-05-31 00:07:24+00:00",
            "title": "Multi-modal Discriminative Model for Vision-and-Language Navigation",
            "authors": [
                "Haoshuo Huang",
                "Vihan Jain",
                "Harsh Mehta",
                "Jason Baldridge",
                "Eugene Ie"
            ],
            "abstract": "Vision-and-Language Navigation (VLN) is a natural language grounding task\nwhere agents have to interpret natural language instructions in the context of\nvisual scenes in a dynamic environment to achieve prescribed navigation goals.\nSuccessful agents must have the ability to parse natural language of varying\nlinguistic styles, ground them in potentially unfamiliar scenes, plan and react\nwith ambiguous environmental feedback. Generalization ability is limited by the\namount of human annotated data. In particular, \\emph{paired} vision-language\nsequence data is expensive to collect. We develop a discriminator that\nevaluates how well an instruction explains a given path in VLN task using\nmulti-modal alignment. Our study reveals that only a small fraction of the\nhigh-quality augmented data from \\citet{Fried:2018:Speaker}, as scored by our\ndiscriminator, is useful for training VLN agents with similar performance on\npreviously unseen environments. We also show that a VLN agent warm-started with\npre-trained components from the discriminator outperforms the benchmark success\nrates of 35.5 by 10\\% relative measure on previously unseen environments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13418v1",
            "date": "2019-05-31 05:16:15+00:00",
            "title": "Constructive Type-Logical Supertagging with Self-Attention Networks",
            "authors": [
                "Konstantinos Kogkalidis",
                "Michael Moortgat",
                "Tejaswini Deoskar"
            ],
            "abstract": "We propose a novel application of self-attention networks towards grammar\ninduction. We present an attention-based supertagger for a refined type-logical\ngrammar, trained on constructing types inductively. In addition to achieving a\nhigh overall type accuracy, our model is able to learn the syntax of the\ngrammar's type system along with its denotational semantics. This lifts the\nclosed world assumption commonly made by lexicalized grammar supertaggers,\ngreatly enhancing its generalization potential. This is evidenced both by its\nadequate accuracy over sparse word types and its ability to correctly construct\ncomplex types never seen during training, which, to the best of our knowledge,\nwas as of yet unaccomplished.",
            "journal ref": "Proceedings of the 4th Workshop on Representation Learning for NLP\n  (RepL4NLP-2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13438v2",
            "date": "2019-05-31 06:36:19+00:00",
            "title": "Content Word-based Sentence Decoding and Evaluating for Open-domain Neural Response Generation",
            "authors": [
                "Tianyu Zhao",
                "Shinsuke Mori",
                "Tatsuya Kawahara"
            ],
            "abstract": "Various encoder-decoder models have been applied to response generation in\nopen-domain dialogs, but a majority of conventional models directly learn a\nmapping from lexical input to lexical output without explicitly modeling\nintermediate representations. Utilizing language hierarchy and modeling\nintermediate information have been shown to benefit many language understanding\nand generation tasks. Motivated by Broca's aphasia, we propose to use a content\nword sequence as an intermediate representation for open-domain response\ngeneration. Experimental results show that the proposed method improves content\nrelatedness of produced responses, and our models can often choose correct\ngrammar for generated content words. Meanwhile, instead of evaluating complete\nsentences, we propose to compute conventional metrics on content word\nsequences, which is a better indicator of content relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13497v1",
            "date": "2019-05-31 10:27:58+00:00",
            "title": "Attention Is (not) All You Need for Commonsense Reasoning",
            "authors": [
                "Tassilo Klein",
                "Moin Nabi"
            ],
            "abstract": "The recently introduced BERT model exhibits strong performance on several\nlanguage understanding benchmarks. In this paper, we describe a simple\nre-implementation of BERT for commonsense reasoning. We show that the\nattentions produced by BERT can be directly utilized for tasks such as the\nPronoun Disambiguation Problem and Winograd Schema Challenge. Our proposed\nattention-guided commonsense reasoning method is conceptually simple yet\nempirically powerful. Experimental analysis on multiple datasets demonstrates\nthat our proposed system performs remarkably well on all cases while\noutperforming the previously reported state of the art by a margin. While\nresults suggest that BERT seems to implicitly learn to establish complex\nrelationships between entities, solving commonsense reasoning tasks might\nrequire more than unsupervised models learned from huge text corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13601v1",
            "date": "2019-05-31 13:13:51+00:00",
            "title": "Using Natural Language Processing to Develop an Automated Orthodontic Diagnostic System",
            "authors": [
                "Tomoyuki Kajiwara",
                "Chihiro Tanikawa",
                "Yuujin Shimizu",
                "Chenhui Chu",
                "Takashi Yamashiro",
                "Hajime Nagahara"
            ],
            "abstract": "We work on the task of automatically designing a treatment plan from the\nfindings included in the medical certificate written by the dentist. To develop\nan artificial intelligence system that deals with free-form certificates\nwritten by dentists, we annotate the findings and utilized the natural language\nprocessing approach. As a result of the experiment using 990 certificates,\n0.585 F1-score was achieved for the task of extracting orthodontic problems\nfrom findings, and 0.584 correlation coefficient with the human ranking was\nachieved for the treatment prioritization task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13637v1",
            "date": "2019-05-31 14:30:49+00:00",
            "title": "GSN: A Graph-Structured Network for Multi-Party Dialogues",
            "authors": [
                "Wenpeng Hu",
                "Zhangming Chan",
                "Bing Liu",
                "Dongyan Zhao",
                "Jinwen Ma",
                "Rui Yan"
            ],
            "abstract": "Existing neural models for dialogue response generation assume that\nutterances are sequentially organized. However, many real-world dialogues\ninvolve multiple interlocutors (i.e., multi-party dialogues), where the\nassumption does not hold as utterances from different interlocutors can occur\n\"in parallel.\" This paper generalizes existing sequence-based models to a\nGraph-Structured neural Network (GSN) for dialogue modeling. The core of GSN is\na graph-based encoder that can model the information flow along the\ngraph-structured dialogues (two-party sequential dialogues are a special case).\nExperimental results show that GSN significantly outperforms existing\nsequence-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13649v6",
            "date": "2019-05-31 14:49:30+00:00",
            "title": "Spotting Collective Behaviour of Online Frauds in Customer Reviews",
            "authors": [
                "Sarthika Dhawan",
                "Siva Charan Reddy Gangireddy",
                "Shiv Kumar",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Online reviews play a crucial role in deciding the quality before purchasing\nany product. Unfortunately, spammers often take advantage of online review\nforums by writing fraud reviews to promote/demote certain products. It may turn\nout to be more detrimental when such spammers collude and collectively inject\nspam reviews as they can take complete control of users' sentiment due to the\nvolume of fraud reviews they inject. Group spam detection is thus more\nchallenging than individual-level fraud detection due to unclear definition of\na group, variation of inter-group dynamics, scarcity of labeled group-level\nspam data, etc. Here, we propose DeFrauder, an unsupervised method to detect\nonline fraud reviewer groups. It first detects candidate fraud groups by\nleveraging the underlying product review graph and incorporating several\nbehavioral signals which model multi-faceted collaboration among reviewers. It\nthen maps reviewers into an embedding space and assigns a spam score to each\ngroup such that groups comprising spammers with highly similar behavioral\ntraits achieve high spam score. While comparing with five baselines on four\nreal-world datasets (two of them were curated by us), DeFrauder shows superior\nperformance by outperforming the best baseline with 17.11% higher NDCG@50 (on\naverage) across datasets.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.13687v3",
            "date": "2019-05-31 15:54:41+00:00",
            "title": "Entropy Minimization In Emergent Languages",
            "authors": [
                "Eugene Kharitonov",
                "Rahma Chaabouni",
                "Diane Bouchacourt",
                "Marco Baroni"
            ],
            "abstract": "There is growing interest in studying the languages that emerge when neural\nagents are jointly trained to solve tasks requiring communication through a\ndiscrete channel. We investigate here the information-theoretic complexity of\nsuch languages, focusing on the basic two-agent, one-exchange setup. We find\nthat, under common training procedures, the emergent languages are subject to\nan entropy minimization pressure that has also been detected in human language,\nwhereby the mutual information between the communicating agent's inputs and the\nmessages is minimized, within the range afforded by the need for successful\ncommunication. That is, emergent languages are (nearly) as simple as the task\nthey are developed for allow them to be. This pressure is amplified as we\nincrease communication channel discreteness. Further, we observe that stronger\ndiscrete-channel-driven entropy minimization leads to representations with\nincreased robustness to overfitting and adversarial attacks. We conclude by\ndiscussing the implications of our findings for the study of natural and\nartificial communication systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00038v2",
            "date": "2019-05-31 19:12:55+00:00",
            "title": "Visual Understanding and Narration: A Deeper Understanding and Explanation of Visual Scenes",
            "authors": [
                "Stephanie M. Lukin",
                "Claire Bonial",
                "Clare R. Voss"
            ],
            "abstract": "We describe the task of Visual Understanding and Narration, in which a robot\n(or agent) generates text for the images that it collects when navigating its\nenvironment, by answering open-ended questions, such as 'what happens, or might\nhave happened, here?'",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00067v2",
            "date": "2019-05-31 20:29:01+00:00",
            "title": "OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge",
            "authors": [
                "Kenneth Marino",
                "Mohammad Rastegari",
                "Ali Farhadi",
                "Roozbeh Mottaghi"
            ],
            "abstract": "Visual Question Answering (VQA) in its ideal form lets us study reasoning in\nthe joint space of vision and language and serves as a proxy for the AI task of\nscene understanding. However, most VQA benchmarks to date are focused on\nquestions such as simple counting, visual attributes, and object detection that\ndo not require reasoning or knowledge beyond what is in the image. In this\npaper, we address the task of knowledge-based visual question answering and\nprovide a benchmark, called OK-VQA, where the image content is not sufficient\nto answer the questions, encouraging methods that rely on external knowledge\nresources. Our new dataset includes more than 14,000 questions that require\nexternal knowledge to answer. We show that the performance of the\nstate-of-the-art VQA models degrades drastically in this new setting. Our\nanalysis shows that our knowledge-based VQA task is diverse, difficult, and\nlarge compared to previous knowledge-based VQA datasets. We hope that this\ndataset enables researchers to open up new avenues for research in this domain.\nSee http://okvqa.allenai.org to download and browse the dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.00080v1",
            "date": "2019-05-17 07:58:44+00:00",
            "title": "Gmail Smart Compose: Real-Time Assisted Writing",
            "authors": [
                "Mia Xu Chen",
                "Benjamin N Lee",
                "Gagan Bansal",
                "Yuan Cao",
                "Shuyuan Zhang",
                "Justin Lu",
                "Jackie Tsay",
                "Yinan Wang",
                "Andrew M. Dai",
                "Zhifeng Chen",
                "Timothy Sohn",
                "Yonghui Wu"
            ],
            "abstract": "In this paper, we present Smart Compose, a novel system for generating\ninteractive, real-time suggestions in Gmail that assists users in writing mails\nby reducing repetitive typing. In the design and deployment of such a\nlarge-scale and complicated system, we faced several challenges including model\nselection, performance evaluation, serving and other practical issues. At the\ncore of Smart Compose is a large-scale neural language model. We leveraged\nstate-of-the-art machine learning techniques for language model training which\nenabled high-quality suggestion prediction, and constructed novel serving\ninfrastructure for high-throughput and real-time inference. Experimental\nresults show the effectiveness of our proposed system design and deployment\napproach. This system is currently being served in Gmail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00112v2",
            "date": "2019-05-31 22:46:03+00:00",
            "title": "Emotional Embeddings: Refining Word Embeddings to Capture Emotional Content of Words",
            "authors": [
                "Armin Seyeditabari",
                "Narges Tabari",
                "Shafie Gholizade",
                "Wlodek Zadrozny"
            ],
            "abstract": "Word embeddings are one of the most useful tools in any modern natural\nlanguage processing expert's toolkit. They contain various types of information\nabout each word which makes them the best way to represent the terms in any NLP\ntask. But there are some types of information that cannot be learned by these\nmodels. Emotional information of words are one of those. In this paper, we\npresent an approach to incorporate emotional information of words into these\nmodels. We accomplish this by adding a secondary training stage which uses an\nemotional lexicon and a psychological model of basic emotions. We show that\nfitting an emotional model into pre-trained word vectors can increase the\nperformance of these models in emotional similarity metrics. Retrained models\nperform better than their original counterparts from 13% improvement for\nWord2Vec model, to 29% for GloVe vectors. This is the first such model\npresented in the literature, and although preliminary, these emotion sensitive\nmodels can open the way to increase performance in variety of emotion detection\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00141v2",
            "date": "2019-06-01 03:31:26+00:00",
            "title": "Multi-Turn Beam Search for Neural Dialogue Modeling",
            "authors": [
                "Ilia Kulikov",
                "Jason Lee",
                "Kyunghyun Cho"
            ],
            "abstract": "In neural dialogue modeling, a neural network is trained to predict the next\nutterance, and at inference time, an approximate decoding algorithm is used to\ngenerate next utterances given previous ones. While this autoregressive\nframework allows us to model the whole conversation during training, inference\nis highly suboptimal, as a wrong utterance can affect future utterances. While\nbeam search yields better results than greedy search does, we argue that it is\nstill greedy in the context of the entire conversation, in that it does not\nconsider future utterances. We propose a novel approach for conversation-level\ninference by explicitly modeling the dialogue partner and running beam search\nacross multiple conversation turns. Given a set of candidates for next\nutterance, we unroll the conversation for a number of turns and identify the\ncandidate utterance in the initial hypothesis set that gives rise to the most\nlikely sequence of future utterances. We empirically validate our approach by\nconducting human evaluation using the Persona-Chat dataset, and find that our\nmulti-turn beam search generates significantly better dialogue responses. We\npropose three approximations to the partner model, and observe that more\ninformed partner models give better performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00156v2",
            "date": "2019-06-01 05:55:52+00:00",
            "title": "Promotion of Answer Value Measurement with Domain Effects in Community Question Answering Systems",
            "authors": [
                "Binbin Jin",
                "Enhong Chen",
                "Hongke Zhao",
                "Zhenya Huang",
                "Qi Liu",
                "Hengshu Zhu",
                "Shui Yu"
            ],
            "abstract": "In the area of community question answering (CQA), answer selection and\nanswer ranking are two tasks which are applied to help users quickly access\nvaluable answers. Existing solutions mainly exploit the syntactic or semantic\ncorrelation between a question and its related answers (Q&A), where the\nmulti-facet domain effects in CQA are still underexplored. In this paper, we\npropose a unified model, Enhanced Attentive Recurrent Neural Network (EARNN),\nfor both answer selection and answer ranking tasks by taking full advantages of\nboth Q&A semantics and multi-facet domain effects (i.e., topic effects and\ntimeliness). Specifically, we develop a serialized LSTM to learn the unified\nrepresentations of Q&A, where two attention mechanisms at either sentence-level\nor word-level are designed for capturing the deep effects of topics. Meanwhile,\nthe emphasis of Q&A can be automatically distinguished. Furthermore, we design\na time-sensitive ranking function to model the timeliness in CQA. To\neffectively train EARNN, a question-dependent pairwise learning strategy is\nalso developed. Finally, we conduct extensive experiments on a real-world\ndataset from Quora. Experimental results validate the effectiveness and\ninterpretability of our proposed EARNN model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00318v1",
            "date": "2019-06-02 00:29:05+00:00",
            "title": "Question Answering as an Automatic Evaluation Metric for News Article Summarization",
            "authors": [
                "Matan Eyal",
                "Tal Baumel",
                "Michael Elhadad"
            ],
            "abstract": "Recent work in the field of automatic summarization and headline generation\nfocuses on maximizing ROUGE scores for various news datasets. We present an\nalternative, extrinsic, evaluation metric for this task, Answering Performance\nfor Evaluation of Summaries. APES utilizes recent progress in the field of\nreading-comprehension to quantify the ability of a summary to answer a set of\nmanually created questions regarding central entities in the source article. We\nfirst analyze the strength of this metric by comparing it to known manual\nevaluation metrics. We then present an end-to-end neural abstractive model that\nmaximizes APES, while increasing ROUGE scores to competitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00363v2",
            "date": "2019-06-02 08:03:21+00:00",
            "title": "Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanation",
            "authors": [
                "Cunxiang Wang",
                "Shuailong Liang",
                "Yue Zhang",
                "Xiaonan Li",
                "Tian Gao"
            ],
            "abstract": "Introducing common sense to natural language understanding systems has\nreceived increasing research attention. It remains a fundamental question on\nhow to evaluate whether a system has a sense making capability. Existing\nbenchmarks measures commonsense knowledge indirectly and without explanation.\nIn this paper, we release a benchmark to directly test whether a system can\ndifferentiate natural language statements that make sense from those that do\nnot make sense. In addition, a system is asked to identify the most crucial\nreason why a statement does not make sense. We evaluate models trained over\nlarge-scale language modeling tasks as well as human performance, showing that\nthere are different challenges for system sense making.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.00378v1",
            "date": "2019-06-02 10:05:26+00:00",
            "title": "Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data",
            "authors": [
                "Shizhe Chen",
                "Qin Jin",
                "Alexander Hauptmann"
            ],
            "abstract": "Bilingual lexicon induction, translating words from the source language to\nthe target language, is a long-standing natural language processing task.\nRecent endeavors prove that it is promising to employ images as pivot to learn\nthe lexicon induction without reliance on parallel corpora. However, these\nvision-based approaches simply associate words with entire images, which are\nconstrained to translate concrete words and require object-centered images. We\nhumans can understand words better when they are within a sentence with\ncontext. Therefore, in this paper, we propose to utilize images and their\nassociated captions to address the limitations of previous approaches. We\npropose a multi-lingual caption model trained with different mono-lingual\nmultimodal data to map words in different languages into joint spaces. Two\ntypes of word representation are induced from the multi-lingual caption model:\nlinguistic features and localized visual features. The linguistic feature is\nlearned from the sentence contexts with visual semantic constraints, which is\nbeneficial to learn translation for words that are less visual-relevant. The\nlocalized visual feature is attended to the region in the image that correlates\nto the word, so that it alleviates the image restriction for salient visual\nrepresentation. The two types of features are complementary for word\ntranslation. Experimental results on multiple language pairs demonstrate the\neffectiveness of our proposed method, which substantially outperforms previous\nvision-based approaches without using any parallel sentences or supervision of\nseed word pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00411v4",
            "date": "2019-06-02 14:11:37+00:00",
            "title": "TechNet: Technology Semantic Network Based on Patent Data",
            "authors": [
                "Serhad Sarica",
                "Jianxi Luo",
                "Kristin L. Wood"
            ],
            "abstract": "The growing developments in general semantic networks, knowledge graphs and\nontology databases have motivated us to build a large-scale comprehensive\nsemantic network of technology-related data for engineering knowledge\ndiscovery, technology search and retrieval, and artificial intelligence for\nengineering design and innovation. Specially, we constructed a technology\nsemantic network (TechNet) that covers the elemental concepts in all domains of\ntechnology and their semantic associations by mining the complete U.S. patent\ndatabase from 1976. To derive the TechNet, natural language processing\ntechniques were utilized to extract terms from massive patent texts and recent\nword embedding algorithms were employed to vectorize such terms and establish\ntheir semantic relationships. We report and evaluate the TechNet for retrieving\nterms and their pairwise relevance that is meaningful from a technology and\nengineering design perspective. The TechNet may serve as an infrastructure to\nsupport a wide range of applications, e.g., technical text summaries, search\nquery predictions, relational knowledge discovery, and design ideation support,\nin the context of engineering and technology, and complement or enrich existing\nsemantic databases. To enable such applications, the TechNet is made public via\nan online interface and APIs for public users to retrieve technology-related\nterms and their relevancies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.00414v2",
            "date": "2019-06-02 14:57:25+00:00",
            "title": "Pretraining Methods for Dialog Context Representation Learning",
            "authors": [
                "Shikib Mehri",
                "Evgeniia Razumovskaia",
                "Tiancheng Zhao",
                "Maxine Eskenazi"
            ],
            "abstract": "This paper examines various unsupervised pretraining objectives for learning\ndialog context representations. Two novel methods of pretraining dialog context\nencoders are proposed, and a total of four methods are examined. Each\npretraining objective is fine-tuned and evaluated on a set of downstream dialog\ntasks using the MultiWoz dataset and strong performance improvement is\nobserved. Further evaluation shows that our pretraining objectives result in\nnot only better performance, but also better convergence, models that are less\ndata hungry and have better domain generalizability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00434v1",
            "date": "2019-06-02 16:14:46+00:00",
            "title": "Deep Unknown Intent Detection with Margin Loss",
            "authors": [
                "Ting-En Lin",
                "Hua Xu"
            ],
            "abstract": "Identifying the unknown (novel) user intents that have never appeared in the\ntraining set is a challenging task in the dialogue system. In this paper, we\npresent a two-stage method for detecting unknown intents. We use bidirectional\nlong short-term memory (BiLSTM) network with the margin loss as the feature\nextractor. With margin loss, we can learn discriminative deep features by\nforcing the network to maximize inter-class variance and to minimize\nintra-class variance. Then, we feed the feature vectors to the density-based\nnovelty detection algorithm, local outlier factor (LOF), to detect unknown\nintents. Experiments on two benchmark datasets show that our method can yield\nconsistent improvements compared with the baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00513v3",
            "date": "2019-06-03 00:42:08+00:00",
            "title": "Generating Question Relevant Captions to Aid Visual Question Answering",
            "authors": [
                "Jialin Wu",
                "Zeyuan Hu",
                "Raymond J. Mooney"
            ],
            "abstract": "Visual question answering (VQA) and image captioning require a shared body of\ngeneral knowledge connecting language and vision. We present a novel approach\nto improve VQA performance that exploits this connection by jointly generating\ncaptions that are targeted to help answer a specific visual question. The model\nis trained using an existing caption dataset by automatically determining\nquestion-relevant captions using an online gradient-based method. Experimental\nresults on the VQA v2 challenge demonstrates that our approach obtains\nstate-of-the-art VQA performance (e.g. 68.4% on the Test-standard set using a\nsingle model) by simultaneously generating question-relevant captions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.00534v2",
            "date": "2019-06-03 02:33:01+00:00",
            "title": "Sentiment Tagging with Partial Labels using Modular Architectures",
            "authors": [
                "Xiao Zhang",
                "Dan Goldwasser"
            ],
            "abstract": "Many NLP learning tasks can be decomposed into several distinct sub-tasks,\neach associated with a partial label. In this paper we focus on a popular class\nof learning problems, sequence prediction applied to several sentiment analysis\ntasks, and suggest a modular learning approach in which different sub-tasks are\nlearned using separate functional modules, combined to perform the final task\nwhile sharing information. Our experiments show this approach helps constrain\nthe learning process and can alleviate some of the supervision efforts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00592v1",
            "date": "2019-06-03 06:32:29+00:00",
            "title": "Assessing the Ability of Self-Attention Networks to Learn Word Order",
            "authors": [
                "Baosong Yang",
                "Longyue Wang",
                "Derek F. Wong",
                "Lidia S. Chao",
                "Zhaopeng Tu"
            ],
            "abstract": "Self-attention networks (SAN) have attracted a lot of interests due to their\nhigh parallelization and strong performance on a variety of NLP tasks, e.g.\nmachine translation. Due to the lack of recurrence structure such as recurrent\nneural networks (RNN), SAN is ascribed to be weak at learning positional\ninformation of words for sequence modeling. However, neither this speculation\nhas been empirically confirmed, nor explanations for their strong performances\non machine translation tasks when \"lacking positional information\" have been\nexplored. To this end, we propose a novel word reordering detection task to\nquantify how well the word order information learned by SAN and RNN.\nSpecifically, we randomly move one word to another position, and examine\nwhether a trained model can detect both the original and inserted positions.\nExperimental results reveal that: 1) SAN trained on word reordering detection\nindeed has difficulty learning the positional information even with the\nposition embedding; and 2) SAN trained on machine translation learns better\npositional information than its RNN counterpart, in which position embedding\nplays a critical role. Although recurrence structure make the model more\nuniversally-effective on learning word order, learning objectives matter more\nin the downstream tasks such as machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00638v1",
            "date": "2019-06-03 08:50:04+00:00",
            "title": "Federated Hierarchical Hybrid Networks for Clickbait Detection",
            "authors": [
                "Feng Liao",
                "Hankz Hankui Zhuo",
                "Xiaoling Huang",
                "Yu Zhang"
            ],
            "abstract": "Online media outlets adopt clickbait techniques to lure readers to click on\narticles in a bid to expand their reach and subsequently increase revenue\nthrough ad monetization. As the adverse effects of clickbait attract more and\nmore attention, researchers have started to explore machine learning techniques\nto automatically detect clickbaits. Previous work on clickbait detection\nassumes that all the training data is available locally during training. In\nmany real-world applications, however, training data is generally distributedly\nstored by different parties (e.g., different parties maintain data with\ndifferent feature spaces), and the parties cannot share their data with each\nother due to data privacy issues. It is challenging to build models of\nhigh-quality federally for detecting clickbaits effectively without data\nsharing. In this paper, we propose a federated training framework, which is\ncalled federated hierarchical hybrid networks, to build clickbait detection\nmodels, where the titles and contents are stored by different parties, whose\nrelationships must be exploited for clickbait detection. We empirically\ndemonstrate that our approach is effective by comparing our approach to the\nstate-of-the-art approaches using datasets from social media.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.00674v1",
            "date": "2019-06-03 09:52:47+00:00",
            "title": "Contextually Propagated Term Weights for Document Representation",
            "authors": [
                "Casper Hansen",
                "Christian Hansen",
                "Stephen Alstrup",
                "Jakob Grue Simonsen",
                "Christina Lioma"
            ],
            "abstract": "Word embeddings predict a word from its neighbours by learning small, dense\nembedding vectors. In practice, this prediction corresponds to a semantic score\ngiven to the predicted word (or term weight). We present a novel model that,\ngiven a target word, redistributes part of that word's weight (that has been\ncomputed with word embeddings) across words occurring in similar contexts as\nthe target word. Thus, our model aims to simulate how semantic meaning is\nshared by words occurring in similar contexts, which is incorporated into\nbag-of-words document representations. Experimental evaluation in an\nunsupervised setting against 8 state of the art baselines shows that our model\nyields the best micro and macro F1 scores across datasets of increasing\ndifficulty.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.00742v1",
            "date": "2019-06-03 12:26:25+00:00",
            "title": "Gender-preserving Debiasing for Pre-trained Word Embeddings",
            "authors": [
                "Masahiro Kaneko",
                "Danushka Bollegala"
            ],
            "abstract": "Word embeddings learnt from massive text collections have demonstrated\nsignificant levels of discriminative biases such as gender, racial or ethnic\nbiases, which in turn bias the down-stream NLP applications that use those word\nembeddings. Taking gender-bias as a working example, we propose a debiasing\nmethod that preserves non-discriminative gender-related information, while\nremoving stereotypical discriminative gender biases from pre-trained word\nembeddings. Specifically, we consider four types of information:\n\\emph{feminine}, \\emph{masculine}, \\emph{gender-neutral} and\n\\emph{stereotypical}, which represent the relationship between gender vs. bias,\nand propose a debiasing method that (a) preserves the gender-related\ninformation in feminine and masculine words, (b) preserves the neutrality in\ngender-neutral words, and (c) removes the biases from stereotypical words.\nExperimental results on several previously proposed benchmark datasets show\nthat our proposed method can debias pre-trained word embeddings better than\nexisting SoTA methods proposed for debiasing word embeddings while preserving\ngender-related but non-discriminative information.",
            "journal ref": "Association for Computational Linguistics (ACL-2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00744v5",
            "date": "2019-06-03 12:28:50+00:00",
            "title": "Hierarchical Decision Making by Generating and Following Natural Language Instructions",
            "authors": [
                "Hengyuan Hu",
                "Denis Yarats",
                "Qucheng Gong",
                "Yuandong Tian",
                "Mike Lewis"
            ],
            "abstract": "We explore using latent natural language instructions as an expressive and\ncompositional representation of complex actions for hierarchical decision\nmaking. Rather than directly selecting micro-actions, our agent first generates\na latent plan in natural language, which is then executed by a separate model.\nWe introduce a challenging real-time strategy game environment in which the\nactions of a large number of units must be coordinated across long time scales.\nWe gather a dataset of 76 thousand pairs of instructions and executions from\nhuman play, and train instructor and executor models. Experiments show that\nmodels using natural language as a latent variable significantly outperform\nmodels that directly imitate human actions. The compositional structure of\nlanguage proves crucial to its effectiveness for action representation. We also\nrelease our code, models and data.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.01130v1",
            "date": "2019-06-03 23:47:43+00:00",
            "title": "Dynamically Composing Domain-Data Selection with Clean-Data Selection by \"Co-Curricular Learning\" for Neural Machine Translation",
            "authors": [
                "Wei Wang",
                "Isaac Caswell",
                "Ciprian Chelba"
            ],
            "abstract": "Noise and domain are important aspects of data quality for neural machine\ntranslation. Existing research focus separately on domain-data selection,\nclean-data selection, or their static combination, leaving the dynamic\ninteraction across them not explicitly examined. This paper introduces a\n\"co-curricular learning\" method to compose dynamic domain-data selection with\ndynamic clean-data selection, for transfer learning across both capabilities.\nWe apply an EM-style optimization procedure to further refine the\n\"co-curriculum\". Experiment results and analysis with two domains demonstrate\nthe effectiveness of the method and the properties of data scheduled by the\nco-curriculum.",
            "journal ref": "The 57th Annual Meeting of the Association for Computational\n  Linguistics (ACL2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01231v1",
            "date": "2019-06-04 07:03:04+00:00",
            "title": "Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model",
            "authors": [
                "Wei Li",
                "Jingjing Xu",
                "Yancheng He",
                "Shengli Yan",
                "Yunfang Wu",
                "Xu sun"
            ],
            "abstract": "Automatic article commenting is helpful in encouraging user engagement and\ninteraction on online news platforms. However, the news documents are usually\ntoo long for traditional encoder-decoder based models, which often results in\ngeneral and irrelevant comments. In this paper, we propose to generate comments\nwith a graph-to-sequence model that models the input news as a topic\ninteraction graph. By organizing the article into graph structure, our model\ncan better understand the internal structure of the article and the connection\nbetween topics, which makes it better able to understand the story. We collect\nand release a large scale news-comment corpus from a popular Chinese online\nnews platform Tencent Kuaibao. Extensive experiment results show that our model\ncan generate much more coherent and informative comments compared with several\nstrong baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01234v2",
            "date": "2019-06-04 07:07:56+00:00",
            "title": "Transcoding compositionally: using attention to find more generalizable solutions",
            "authors": [
                "Kris Korrel",
                "Dieuwke Hupkes",
                "Verna Dankers",
                "Elia Bruni"
            ],
            "abstract": "While sequence-to-sequence models have shown remarkable generalization power\nacross several natural language tasks, their construct of solutions are argued\nto be less compositional than human-like generalization. In this paper, we\npresent seq2attn, a new architecture that is specifically designed to exploit\nattention to find compositional patterns in the input. In seq2attn, the two\nstandard components of an encoder-decoder model are connected via a transcoder,\nthat modulates the information flow between them. We show that seq2attn can\nsuccessfully generalize, without requiring any additional supervision, on two\ntasks which are specifically constructed to challenge the compositional skills\nof neural networks. The solutions found by the model are highly interpretable,\nallowing easy analysis of both the types of solutions that are found and\npotential causes for mistakes. We exploit this opportunity to introduce a new\nparadigm to test compositionality that studies the extent to which a model\novergeneralizes when confronted with exceptions. We show that seq2attn exhibits\nsuch overgeneralization to a larger degree than a standard sequence-to-sequence\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01393v1",
            "date": "2019-06-04 13:07:35+00:00",
            "title": "SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference",
            "authors": [
                "Martin Schmitt",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present SherLIiC, a testbed for lexical inference in context (LIiC),\nconsisting of 3985 manually annotated inference rule candidates (InfCands),\naccompanied by (i) ~960k unlabeled InfCands, and (ii) ~190k typed textual\nrelations between Freebase entities extracted from the large entity-linked\ncorpus ClueWeb09. Each InfCand consists of one of these relations, expressed as\na lemmatized dependency path, and two argument placeholders, each linked to one\nor more Freebase types. Due to our candidate selection process based on strong\ndistributional evidence, SherLIiC is much harder than existing testbeds because\ndistributional evidence is of little utility in the classification of InfCands.\nWe also show that, due to its construction, many of SherLIiC's correct InfCands\nare novel and missing from existing rule bases. We evaluate a number of strong\nbaselines on SherLIiC, ranging from semantic vector space models to state of\nthe art neural models of natural language inference (NLI). We show that\nSherLIiC poses a tough challenge to existing NLI systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01511v1",
            "date": "2019-05-29 14:16:28+00:00",
            "title": "Neural Review Rating Prediction with Hierarchical Attentions and Latent Factors",
            "authors": [
                "Xianchen Wang",
                "Hongtao Liu",
                "Peiyi Wang",
                "Fangzhao Wu",
                "Hongyan Xu",
                "Wenjun Wang",
                "Xing Xie"
            ],
            "abstract": "Text reviews can provide rich useful semantic information for modeling users\nand items, which can benefit rating prediction in recommendation. Different\nwords and reviews may have different informativeness for users or items.\nBesides, different users and items should be personalized. Most existing works\nregard all reviews equally or utilize a general attention mechanism. In this\npaper, we propose a hierarchical attention model fusing latent factor model for\nrating prediction with reviews, which can focus on important words and\ninformative reviews. Specially, we use the factor vectors of Latent Factor\nModel to guide the attention network and combine the factor vectors with\nfeature representation learned from reviews to predict the final ratings.\nExperiments on real-world datasets validate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.01675v2",
            "date": "2019-06-04 18:39:20+00:00",
            "title": "4-D Scene Alignment in Surveillance Video",
            "authors": [
                "Robert Wagner",
                "Daniel Crispell",
                "Patrick Feeney",
                "Joe Mundy"
            ],
            "abstract": "Designing robust activity detectors for fixed camera surveillance video\nrequires knowledge of the 3-D scene. This paper presents an automatic camera\ncalibration process that provides a mechanism to reason about the spatial\nproximity between objects at different times. It combines a CNN-based camera\npose estimator with a vertical scale provided by pedestrian observations to\nestablish the 4-D scene geometry. Unlike some previous methods, the people do\nnot need to be tracked nor do the head and feet need to be explicitly detected.\nIt is robust to individual height variations and camera parameter estimation\nerrors.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.01702v1",
            "date": "2019-06-04 19:58:05+00:00",
            "title": "Improving Neural Language Models by Segmenting, Attending, and Predicting the Future",
            "authors": [
                "Hongyin Luo",
                "Lan Jiang",
                "Yonatan Belinkov",
                "James Glass"
            ],
            "abstract": "Common language models typically predict the next word given the context. In\nthis work, we propose a method that improves language modeling by learning to\nalign the given context and the following phrase. The model does not require\nany linguistic annotation of phrase segmentation. Instead, we define syntactic\nheights and phrase segmentation rules, enabling the model to automatically\ninduce phrases, recognize their task-specific heads, and generate phrase\nembeddings in an unsupervised learning manner. Our method can easily be applied\nto language models with different network architectures since an independent\nmodule is used for phrase induction and context-phrase alignment, and no change\nis required in the underlying language modeling network. Experiments have shown\nthat our model outperformed several strong baseline models on different data\nsets. We achieved a new state-of-the-art performance of 17.4 perplexity on the\nWikitext-103 dataset. Additionally, visualizing the outputs of the phrase\ninduction module showed that our model is able to learn approximate\nphrase-level structural knowledge without any annotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01787v1",
            "date": "2019-06-05 02:24:12+00:00",
            "title": "Learning Deep Transformer Models for Machine Translation",
            "authors": [
                "Qiang Wang",
                "Bei Li",
                "Tong Xiao",
                "Jingbo Zhu",
                "Changliang Li",
                "Derek F. Wong",
                "Lidia S. Chao"
            ],
            "abstract": "Transformer is the state-of-the-art model in recent machine translation\nevaluations. Two strands of research are promising to improve models of this\nkind: the first uses wide networks (a.k.a. Transformer-Big) and has been the de\nfacto standard for the development of the Transformer system, and the other\nuses deeper language representation but faces the difficulty arising from\nlearning deep networks. Here, we continue the line of research on the latter.\nWe claim that a truly deep Transformer model can surpass the Transformer-Big\ncounterpart by 1) proper use of layer normalization and 2) a novel way of\npassing the combination of previous layers to the next. On WMT'16 English-\nGerman, NIST OpenMT'12 Chinese-English and larger WMT'18 Chinese-English tasks,\nour deep system (30/25-layer encoder) outperforms the shallow\nTransformer-Big/Base baseline (6-layer encoder) by 0.4-2.4 BLEU points. As\nanother bonus, the deep model is 1.6X smaller in size and 3X faster in training\nthan Transformer-Big.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01794v1",
            "date": "2019-06-05 02:45:06+00:00",
            "title": "DOER: Dual Cross-Shared RNN for Aspect Term-Polarity Co-Extraction",
            "authors": [
                "Huaishao Luo",
                "Tianrui Li",
                "Bing Liu",
                "Junbo Zhang"
            ],
            "abstract": "This paper focuses on two related subtasks of aspect-based sentiment\nanalysis, namely aspect term extraction and aspect sentiment classification,\nwhich we call aspect term-polarity co-extraction. The former task is to extract\naspects of a product or service from an opinion document, and the latter is to\nidentify the polarity expressed in the document about these extracted aspects.\nMost existing algorithms address them as two separate tasks and solve them one\nby one, or only perform one task, which can be complicated for real\napplications. In this paper, we treat these two tasks as two sequence labeling\nproblems and propose a novel Dual crOss-sharEd RNN framework (DOER) to generate\nall aspect term-polarity pairs of the input sentence simultaneously.\nSpecifically, DOER involves a dual recurrent neural network to extract the\nrespective representation of each task, and a cross-shared unit to consider the\nrelationship between them. Experimental results demonstrate that the proposed\nframework outperforms state-of-the-art baselines on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01815v1",
            "date": "2019-06-05 04:08:47+00:00",
            "title": "Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)",
            "authors": [
                "Santiago Castro",
                "Devamanyu Hazarika",
                "Ver\u00f3nica P\u00e9rez-Rosas",
                "Roger Zimmermann",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "Sarcasm is often expressed through several verbal and non-verbal cues, e.g.,\na change of tone, overemphasis in a word, a drawn-out syllable, or a straight\nlooking face. Most of the recent work in sarcasm detection has been carried out\non textual data. In this paper, we argue that incorporating multimodal cues can\nimprove the automatic classification of sarcasm. As a first step towards\nenabling the development of multimodal approaches for sarcasm detection, we\npropose a new sarcasm dataset, Multimodal Sarcasm Detection Dataset (MUStARD),\ncompiled from popular TV shows. MUStARD consists of audiovisual utterances\nannotated with sarcasm labels. Each utterance is accompanied by its context of\nhistorical utterances in the dialogue, which provides additional information on\nthe scenario where the utterance occurs. Our initial results show that the use\nof multimodal information can reduce the relative error rate of sarcasm\ndetection by up to 12.9% in F-score when compared to the use of individual\nmodalities. The full dataset is publicly available for use at\nhttps://github.com/soujanyaporia/MUStARD",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01833v1",
            "date": "2019-06-05 05:27:31+00:00",
            "title": "A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer",
            "authors": [
                "Chen Wu",
                "Xuancheng Ren",
                "Fuli Luo",
                "Xu Sun"
            ],
            "abstract": "Unsupervised text style transfer aims to alter text styles while preserving\nthe content, without aligned data for supervision. Existing seq2seq methods\nface three challenges: 1) the transfer is weakly interpretable, 2) generated\noutputs struggle in content preservation, and 3) the trade-off between content\nand style is intractable. To address these challenges, we propose a\nhierarchical reinforced sequence operation method, named Point-Then-Operate\n(PTO), which consists of a high-level agent that proposes operation positions\nand a low-level agent that alters the sentence. We provide comprehensive\ntraining objectives to control the fluency, style, and content of the outputs\nand a mask-based inference algorithm that allows for multi-step revision based\non the single-step trained agents. Experimental results on two text style\ntransfer datasets show that our method significantly outperforms recent methods\nand effectively addresses the aforementioned challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01840v1",
            "date": "2019-06-05 05:59:07+00:00",
            "title": "Improving Textual Network Embedding with Global Attention via Optimal Transport",
            "authors": [
                "Liqun Chen",
                "Guoyin Wang",
                "Chenyang Tao",
                "Dinghan Shen",
                "Pengyu Cheng",
                "Xinyuan Zhang",
                "Wenlin Wang",
                "Yizhe Zhang",
                "Lawrence Carin"
            ],
            "abstract": "Constituting highly informative network embeddings is an important tool for\nnetwork analysis. It encodes network topology, along with other useful side\ninformation, into low-dimensional node-based feature representations that can\nbe exploited by statistical modeling. This work focuses on learning\ncontext-aware network embeddings augmented with text data. We reformulate the\nnetwork-embedding problem, and present two novel strategies to improve over\ntraditional attention mechanisms: ($i$) a content-aware sparse attention module\nbased on optimal transport, and ($ii$) a high-level attention parsing module.\nOur approach yields naturally sparse and self-normalized relational inference.\nIt can capture long-term interactions between sequences, thus addressing the\nchallenges faced by existing textual network embedding schemes. Extensive\nexperiments are conducted to demonstrate our model can consistently outperform\nalternative state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01873v3",
            "date": "2019-06-05 08:11:12+00:00",
            "title": "Towards conceptual generalization in the embedding space",
            "authors": [
                "Luka Nenadovi\u0107",
                "Vladimir Prelovac"
            ],
            "abstract": "Humans are able to conceive physical reality by jointly learning different\nfacets thereof. To every pair of notions related to a perceived reality may\ncorrespond a mutual relation, which is a notion on its own, but one-level\nhigher. Thus, we may have a description of perceived reality on at least two\nlevels and the translation map between them is in general, due to their\ndifferent content corpus, one-to-many. Following success of the unsupervised\nneural machine translation models, which are essentially one-to-one mappings\ntrained separately on monolingual corpora, we examine further capabilities of\nthe unsupervised deep learning methods used there and apply some of these\nmethods to sets of notions of different level and measure. Using the graph and\nword embedding-like techniques, we build one-to-many map without parallel data\nin order to establish a unified vector representation of the outer world by\ncombining notions of different kind into a unique conceptual framework. Due to\ntheir latent similarity, by aligning the two embedding spaces in purely\nunsupervised way, one obtains a geometric relation between objects of cognition\non the two levels, making it possible to express a natural knowledge using one\ndescription in the context of the other.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.01942v1",
            "date": "2019-06-05 11:16:33+00:00",
            "title": "Learning Bilingual Sentence Embeddings via Autoencoding and Computing Similarities with a Multilayer Perceptron",
            "authors": [
                "Yunsu Kim",
                "Hendrik Rosendahl",
                "Nick Rossenbach",
                "Jan Rosendahl",
                "Shahram Khadivi",
                "Hermann Ney"
            ],
            "abstract": "We propose a novel model architecture and training algorithm to learn\nbilingual sentence embeddings from a combination of parallel and monolingual\ndata. Our method connects autoencoding and neural machine translation to force\nthe source and target sentence embeddings to share the same space without the\nhelp of a pivot language or an additional transformation. We train a multilayer\nperceptron on top of the sentence embeddings to extract good bilingual sentence\npairs from nonparallel or noisy parallel data. Our approach shows promising\nperformance on sentence alignment recovery and the WMT 2018 parallel corpus\nfiltering tasks with only a single model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01946v1",
            "date": "2019-06-05 11:23:14+00:00",
            "title": "Automated Speech Generation from UN General Assembly Statements: Mapping Risks in AI Generated Texts",
            "authors": [
                "Joseph Bullock",
                "Miguel Luengo-Oroz"
            ],
            "abstract": "Automated text generation has been applied broadly in many domains such as\nmarketing and robotics, and used to create chatbots, product reviews and write\npoetry. The ability to synthesize text, however, presents many potential risks,\nwhile access to the technology required to build generative models is becoming\nincreasingly easy. This work is aligned with the efforts of the United Nations\nand other civil society organisations to highlight potential political and\nsocietal risks arising through the malicious use of text generation software,\nand their potential impact on human rights. As a case study, we present the\nfindings of an experiment to generate remarks in the style of political leaders\nby fine-tuning a pretrained AWD- LSTM model on a dataset of speeches made at\nthe UN General Assembly. This work highlights the ease with which this can be\naccomplished, as well as the threats of combining these techniques with other\ntechnologies.",
            "journal ref": "International Conference on Machine Learning AI for Social Good\n  Workshop, Long Beach, United States, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01958v1",
            "date": "2019-06-05 11:53:38+00:00",
            "title": "From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions",
            "authors": [
                "David Mare\u010dek",
                "Rudolf Rosa"
            ],
            "abstract": "We inspect the multi-head self-attention in Transformer NMT encoders for\nthree source languages, looking for patterns that could have a syntactic\ninterpretation. In many of the attention heads, we frequently find sequences of\nconsecutive states attending to the same position, which resemble syntactic\nphrases. We propose a transparent deterministic method of quantifying the\namount of syntactic information present in the self-attentions, based on\nautomatically building and evaluating phrase-structure trees from the\nphrase-like sequences. We compare the resulting trees to existing constituency\ntreebanks, both manually and by computing precision and recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01965v1",
            "date": "2019-05-25 03:05:15+00:00",
            "title": "Triple-to-Text: Converting RDF Triples into High-Quality Natural Languages via Optimizing an Inverse KL Divergence",
            "authors": [
                "Yaoming Zhu",
                "Juncheng Wan",
                "Zhiming Zhou",
                "Liheng Chen",
                "Lin Qiu",
                "Weinan Zhang",
                "Xin Jiang",
                "Yong Yu"
            ],
            "abstract": "Knowledge base is one of the main forms to represent information in a\nstructured way. A knowledge base typically consists of Resource Description\nFrameworks (RDF) triples which describe the entities and their relations.\nGenerating natural language description of the knowledge base is an important\ntask in NLP, which has been formulated as a conditional language generation\ntask and tackled using the sequence-to-sequence framework. Current works mostly\ntrain the language models by maximum likelihood estimation, which tends to\ngenerate lousy sentences. In this paper, we argue that such a problem of\nmaximum likelihood estimation is intrinsic, which is generally irrevocable via\nchanging network structures. Accordingly, we propose a novel Triple-to-Text\n(T2T) framework, which approximately optimizes the inverse Kullback-Leibler\n(KL) divergence between the distributions of the real and generated sentences.\nDue to the nature that inverse KL imposes large penalty on fake-looking\nsamples, the proposed method can significantly reduce the probability of\ngenerating low-quality sentences. Our experiments on three real-world datasets\ndemonstrate that T2T can generate higher-quality sentences and outperform\nbaseline models in several evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02121v1",
            "date": "2019-05-13 19:43:54+00:00",
            "title": "Classifying Norm Conflicts using Learned Semantic Representations",
            "authors": [
                "Jo\u00e3o Paulo Aires",
                "Roger Granada",
                "Juarez Monteiro",
                "Rodrigo C. Barros",
                "Felipe Meneguzzi"
            ],
            "abstract": "While most social norms are informal, they are often formalized by companies\nin contracts to regulate trades of goods and services. When poorly written,\ncontracts may contain normative conflicts resulting from opposing deontic\nmeanings or contradict specifications. As contracts tend to be long and contain\nmany norms, manually identifying such conflicts requires human-effort, which is\ntime-consuming and error-prone. Automating such task benefits contract makers\nincreasing productivity and making conflict identification more reliable. To\naddress this problem, we introduce an approach to detect and classify norm\nconflicts in contracts by converting them into latent representations that\npreserve both syntactic and semantic information and training a model to\nclassify norm conflicts in four conflict types. Our results reach the new state\nof the art when compared to a previous approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02123v1",
            "date": "2019-05-14 08:32:39+00:00",
            "title": "SP-10K: A Large-scale Evaluation Set for Selectional Preference Acquisition",
            "authors": [
                "Hongming Zhang",
                "Hantian Ding",
                "Yangqiu Song"
            ],
            "abstract": "Selectional Preference (SP) is a commonly observed language phenomenon and\nproved to be useful in many natural language processing tasks. To provide a\nbetter evaluation method for SP models, we introduce SP-10K, a large-scale\nevaluation set that provides human ratings for the plausibility of 10,000 SP\npairs over five SP relations, covering 2,500 most frequent verbs, nouns, and\nadjectives in American English. Three representative SP acquisition methods\nbased on pseudo-disambiguation are evaluated with SP-10K. To demonstrate the\nimportance of our dataset, we investigate the relationship between SP-10K and\nthe commonsense knowledge in ConceptNet5 and show the potential of using SP to\nrepresent the commonsense knowledge. We also use the Winograd Schema Challenge\nto prove that the proposed new SP relations are essential for the hard pronoun\ncoreference resolution problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02126v1",
            "date": "2019-05-15 12:42:42+00:00",
            "title": "Extractive Summarization via Weighted Dissimilarity and Importance Aligned Key Iterative Algorithm",
            "authors": [
                "Ryohto Sawada"
            ],
            "abstract": "We present importance aligned key iterative algorithm for extractive\nsummarization that is faster than conventional algorithms keeping its accuracy.\nThe computational complexity of our algorithm is O($SNlogN$) to summarize\noriginal $N$ sentences into final $S$ sentences. Our algorithm maximizes the\nweighted dissimilarity defined by the product of importance and cosine\ndissimilarity so that the summary represents the document and at the same time\nthe sentences of the summary are not similar to each other. The weighted\ndissimilarity is heuristically maximized by iterative greedy search and binary\nsearch to the sentences ordered by importance. We finally show a benchmark\nscore based on summarization of customer reviews of products, which highlights\nthe quality of our algorithm comparable to human and existing algorithms. We\nprovide the source code of our algorithm on github\nhttps://github.com/qhapaq-49/imakita .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02127v3",
            "date": "2019-05-16 16:04:49+00:00",
            "title": "An Approach for Process Model Extraction By Multi-Grained Text Classification",
            "authors": [
                "Chen Qian",
                "Lijie Wen",
                "Akhil Kumar",
                "Leilei Lin",
                "Li Lin",
                "Zan Zong",
                "Shuang Li",
                "Jianmin Wang"
            ],
            "abstract": "Process model extraction (PME) is a recently emerged interdiscipline between\nnatural language processing (NLP) and business process management (BPM), which\naims to extract process models from textual descriptions. Previous process\nextractors heavily depend on manual features and ignore the potential relations\nbetween clues of different text granularities. In this paper, we formalize the\nPME task into the multi-grained text classification problem, and propose a\nhierarchical neural network to effectively model and extract multi-grained\ninformation without manually-defined procedural features. Under this structure,\nwe accordingly propose the coarse-to-fine (grained) learning mechanism,\ntraining multi-grained tasks in coarse-to-fine grained order to share the\nhigh-level knowledge for the low-level tasks. To evaluate our approach, we\nconstruct two multi-grained datasets from two different domains and conduct\nextensive experiments from different dimensions. The experimental results\ndemonstrate that our approach outperforms the state-of-the-art methods with\nstatistical significance and further investigations demonstrate its\neffectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02134v1",
            "date": "2019-05-23 08:50:15+00:00",
            "title": "Theme-aware generation model for chinese lyrics",
            "authors": [
                "Jie Wang",
                "Xinyan Zhao"
            ],
            "abstract": "With rapid development of neural networks, deep-learning has been extended to\nvarious natural language generation fields, such as machine translation,\ndialogue generation and even literature creation. In this paper, we propose a\ntheme-aware language generation model for Chinese music lyrics, which improves\nthe theme-connectivity and coherence of generated paragraphs greatly. A\nmulti-channel sequence-to-sequence (seq2seq) model encodes themes and previous\nsentences as global and local contextual information. Moreover, attention\nmechanism is incorporated for sequence decoding, enabling to fuse context into\npredicted next texts. To prepare appropriate train corpus, LDA (Latent\nDirichlet Allocation) is applied for theme extraction. Generated lyrics is\ngrammatically correct and semantically coherent with selected themes, which\noffers a valuable modelling method in other fields including multi-turn\nchatbots, long paragraph generation and etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02135v2",
            "date": "2019-05-23 09:12:59+00:00",
            "title": "Deep learning based mood tagging for Chinese song lyrics",
            "authors": [
                "Jie Wang",
                "Yilin Yang"
            ],
            "abstract": "Nowadays, listening music has been and will always be an indispensable part\nof our daily life. In recent years, sentiment analysis of music has been widely\nused in the information retrieval systems, personalized recommendation systems\nand so on. Due to the development of deep learning, this paper commits to find\nan effective approach for mood tagging of Chinese song lyrics. To achieve this\ngoal, both machine-learning and deep-learning models have been studied and\ncompared. Eventually, a CNN-based model with pre-trained word embedding has\nbeen demonstrated to effectively extract the distribution of emotional features\nof Chinese lyrics, with at least 15 percentage points higher than traditional\nmachine-learning methods (i.e. TF-IDF+SVM and LIWC+SVM), and 7 percentage\npoints higher than other deep-learning models (i.e. RNN, LSTM). In this paper,\nmore than 160,000 lyrics corpus has been leveraged for pre-training word\nembedding for mood tagging boost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02239v1",
            "date": "2019-06-05 18:34:16+00:00",
            "title": "Extracting Symptoms and their Status from Clinical Conversations",
            "authors": [
                "Nan Du",
                "Kai Chen",
                "Anjuli Kannan",
                "Linh Tran",
                "Yuhui Chen",
                "Izhak Shafran"
            ],
            "abstract": "This paper describes novel models tailored for a new application, that of\nextracting the symptoms mentioned in clinical conversations along with their\nstatus. Lack of any publicly available corpus in this privacy-sensitive domain\nled us to develop our own corpus, consisting of about 3K conversations\nannotated by professional medical scribes. We propose two novel deep learning\napproaches to infer the symptom names and their status: (1) a new hierarchical\nspan-attribute tagging (\\SAT) model, trained using curriculum learning, and (2)\na variant of sequence-to-sequence model which decodes the symptoms and their\nstatus from a few speaker turns within a sliding window over the conversation.\nThis task stems from a realistic application of assisting medical providers in\ncapturing symptoms mentioned by patients from their clinical conversations. To\nreflect this application, we define multiple metrics. From inter-rater\nagreement, we find that the task is inherently difficult. We conduct\ncomprehensive evaluations on several contrasting conditions and observe that\nthe performance of the models range from an F-score of 0.5 to 0.8 depending on\nthe condition. Our analysis not only reveals the inherent challenges of the\ntask, but also provides useful directions to improve the models.",
            "journal ref": "Proceedings of the Annual Meeting of the Association of\n  Computational Linguistics, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02242v1",
            "date": "2019-06-05 18:40:37+00:00",
            "title": "Variational Pretraining for Semi-supervised Text Classification",
            "authors": [
                "Suchin Gururangan",
                "Tam Dang",
                "Dallas Card",
                "Noah A. Smith"
            ],
            "abstract": "We introduce VAMPIRE, a lightweight pretraining framework for effective text\nclassification when data and computing resources are limited. We pretrain a\nunigram document model as a variational autoencoder on in-domain, unlabeled\ndata and use its internal states as features in a downstream classifier.\nEmpirically, we show the relative strength of VAMPIRE against computationally\nexpensive contextual embeddings and other popular semi-supervised baselines\nunder low resource settings. We also find that fine-tuning to in-domain data is\ncrucial to achieving decent performance from contextual embeddings when working\nwith limited supervision. We accompany this paper with code to pretrain and use\nVAMPIRE embeddings in downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02285v1",
            "date": "2019-06-05 20:05:18+00:00",
            "title": "SParC: Cross-Domain Semantic Parsing in Context",
            "authors": [
                "Tao Yu",
                "Rui Zhang",
                "Michihiro Yasunaga",
                "Yi Chern Tan",
                "Xi Victoria Lin",
                "Suyi Li",
                "Heyang Er",
                "Irene Li",
                "Bo Pang",
                "Tao Chen",
                "Emily Ji",
                "Shreya Dixit",
                "David Proctor",
                "Sungrok Shim",
                "Jonathan Kraft",
                "Vincent Zhang",
                "Caiming Xiong",
                "Richard Socher",
                "Dragomir Radev"
            ],
            "abstract": "We present SParC, a dataset for cross-domainSemanticParsing inContext that\nconsists of 4,298 coherent question sequences (12k+ individual questions\nannotated with SQL queries). It is obtained from controlled user interactions\nwith 200 complex databases over 138 domains. We provide an in-depth analysis of\nSParC and show that it introduces new challenges compared to existing datasets.\nSParC demonstrates complex contextual dependencies, (2) has greater semantic\ndiversity, and (3) requires generalization to unseen domains due to its\ncross-domain nature and the unseen databases at test time. We experiment with\ntwo state-of-the-art text-to-SQL models adapted to the context-dependent,\ncross-domain setup. The best model obtains an exact match accuracy of 20.2%\nover all questions and less than10% over all interaction sequences, indicating\nthat the cross-domain setting and the con-textual phenomena of the dataset\npresent significant challenges for future research. The dataset, baselines, and\nleaderboard are released at https://yale-lily.github.io/sparc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02376v1",
            "date": "2019-06-05 09:40:25+00:00",
            "title": "Training Temporal Word Embeddings with a Compass",
            "authors": [
                "Valerio Di Carlo",
                "Federico Bianchi",
                "Matteo Palmonari"
            ],
            "abstract": "Temporal word embeddings have been proposed to support the analysis of word\nmeaning shifts during time and to study the evolution of languages. Different\napproaches have been proposed to generate vector representations of words that\nembed their meaning during a specific time interval. However, the training\nprocess used in these approaches is complex, may be inefficient or it may\nrequire large text corpora. As a consequence, these approaches may be difficult\nto apply in resource-scarce domains or by scientists with limited in-depth\nknowledge of embedding models. In this paper, we propose a new heuristic to\ntrain temporal word embeddings based on the Word2vec model. The heuristic\nconsists in using atemporal vectors as a reference, i.e., as a compass, when\ntraining the representations specific to a given time interval. The use of the\ncompass simplifies the training process and makes it more efficient.\nExperiments conducted using state-of-the-art datasets and methodologies suggest\nthat our approach outperforms or equals comparable approaches while being more\nrobust in terms of the required corpus size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02564v1",
            "date": "2019-06-06 13:13:46+00:00",
            "title": "Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains",
            "authors": [
                "Claudia Schulz",
                "Christian M. Meyer",
                "Jan Kiesewetter",
                "Michael Sailer",
                "Elisabeth Bauer",
                "Martin R. Fischer",
                "Frank Fischer",
                "Iryna Gurevych"
            ],
            "abstract": "Many complex discourse-level tasks can aid domain experts in their work but\nrequire costly expert annotations for data creation. To speed up and ease\nannotations, we investigate the viability of automatically generated annotation\nsuggestions for such tasks. As an example, we choose a task that is\nparticularly hard for both humans and machines: the segmentation and\nclassification of epistemic activities in diagnostic reasoning texts. We create\nand publish a new dataset covering two domains and carefully analyse the\nsuggested annotations. We find that suggestions have positive effects on\nannotation speed and performance, while not introducing noteworthy biases.\nEnvisioning suggestion models that improve with newly annotated texts, we\ncontrast methods for continuous model adjustment and suggest the most effective\nsetup for suggestions in future expert tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02656v4",
            "date": "2019-06-06 15:46:17+00:00",
            "title": "Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections",
            "authors": [
                "Junxian He",
                "Zhisong Zhang",
                "Taylor Berg-Kirkpatrick",
                "Graham Neubig"
            ],
            "abstract": "Cross-lingual transfer is an effective way to build syntactic analysis tools\nin low-resource languages. However, transfer is difficult when transferring to\ntypologically distant languages, especially when neither annotated target data\nnor parallel corpora are available. In this paper, we focus on methods for\ncross-lingual transfer to distant languages and propose to learn a generative\nmodel with a structured prior that utilizes labeled source data and unlabeled\ntarget data jointly. The parameters of source model and target model are softly\nshared through a regularized log likelihood objective. An invertible projection\nis employed to learn a new interlingual latent embedding space that compensates\nfor imperfect cross-lingual word embedding input. We evaluate our method on two\nsyntactic tasks: part-of-speech (POS) tagging and dependency parsing. On the\nUniversal Dependency Treebanks, we use English as the only source corpus and\ntransfer to a wide range of target languages. On the 10 languages in this\ndataset that are distant from English, our method yields an average of 5.2%\nabsolute improvement on POS tagging and 8.3% absolute improvement on dependency\nparsing over a direct transfer method using state-of-the-art discriminative\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02850v1",
            "date": "2019-06-07 00:54:53+00:00",
            "title": "Figure Captioning with Reasoning and Sequence-Level Training",
            "authors": [
                "Charles Chen",
                "Ruiyi Zhang",
                "Eunyee Koh",
                "Sungchul Kim",
                "Scott Cohen",
                "Tong Yu",
                "Ryan Rossi",
                "Razvan Bunescu"
            ],
            "abstract": "Figures, such as bar charts, pie charts, and line plots, are widely used to\nconvey important information in a concise format. They are usually\nhuman-friendly but difficult for computers to process automatically. In this\nwork, we investigate the problem of figure captioning where the goal is to\nautomatically generate a natural language description of the figure. While\nnatural image captioning has been studied extensively, figure captioning has\nreceived relatively little attention and remains a challenging problem. First,\nwe introduce a new dataset for figure captioning, FigCAP, based on FigureQA.\nSecond, we propose two novel attention mechanisms. To achieve accurate\ngeneration of labels in figures, we propose Label Maps Attention. To model the\nrelations between figure labels, we propose Relation Maps Attention. Third, we\nuse sequence-level training with reinforcement learning in order to directly\noptimizes evaluation metrics, which alleviates the exposure bias issue and\nfurther improves the models in generating long captions. Extensive experiments\nshow that the proposed method outperforms the baselines, thus demonstrating a\nsignificant potential for the automatic captioning of vast repositories of\nfigures.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.02882v1",
            "date": "2019-06-07 03:20:46+00:00",
            "title": "Learning to Recommend Third-Party Library Migration Opportunities at the API Level",
            "authors": [
                "Hussein Alrubaye",
                "Mohamed Wiem Mkaouer",
                "Igor Khokhlov",
                "Leon Reznik",
                "Ali Ouni",
                "Jason Mcgoff"
            ],
            "abstract": "The manual migration between different third-party libraries represents a\nchallenge for software developers. Developers typically need to explore both\nlibraries Application Programming Interfaces, along with reading their\ndocumentation, in order to locate the suitable mappings between replacing and\nreplaced methods. In this paper, we introduce RAPIM, a novel machine learning\napproach that recommends mappings between methods from two different libraries.\nOur model learns from previous migrations, manually performed in mined software\nsystems, and extracts a set of features related to the similarity between\nmethod signatures and method textual documentation. We evaluate our model using\n8 popular migrations, collected from 57,447 open-source Java projects. Results\nshow that RAPIM is able to recommend relevant library API mappings with an\naverage accuracy score of 87%. Finally, we provide the community with an API\nrecommendation web service that could be used to support the migration process.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.02890v2",
            "date": "2019-06-07 04:03:53+00:00",
            "title": "Visually Grounded Neural Syntax Acquisition",
            "authors": [
                "Haoyue Shi",
                "Jiayuan Mao",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We present the Visually Grounded Neural Syntax Learner (VG-NSL), an approach\nfor learning syntactic representations and structures without any explicit\nsupervision. The model learns by looking at natural images and reading paired\ncaptions. VG-NSL generates constituency parse trees of texts, recursively\ncomposes representations for constituents, and matches them with images. We\ndefine concreteness of constituents by their matching scores with images, and\nuse it to guide the parsing of text. Experiments on the MSCOCO data set show\nthat VG-NSL outperforms various unsupervised parsing approaches that do not use\nvisual grounding, in terms of F1 scores against gold parse trees. We find that\nVGNSL is much more stable with respect to the choice of random initialization\nand the amount of training data. We also find that the concreteness acquired by\nVG-NSL correlates well with a similar measure defined by linguists. Finally, we\nalso apply VG-NSL to multiple languages in the Multi30K data set, showing that\nour model consistently outperforms prior unsupervised approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02897v1",
            "date": "2019-06-07 05:01:47+00:00",
            "title": "Semi-supervised Stochastic Multi-Domain Learning using Variational Inference",
            "authors": [
                "Yitong Li",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Supervised models of NLP rely on large collections of text which closely\nresemble the intended testing setting. Unfortunately matching text is often not\navailable in sufficient quantity, and moreover, within any domain of text, data\nis often highly heterogenous. In this paper we propose a method to distill the\nimportant domain signal as part of a multi-domain learning system, using a\nlatent variable model in which parts of a neural model are stochastically gated\nbased on the inferred domain. We compare the use of discrete versus continuous\nlatent variables, operating in a domain-supervised or a domain semi-supervised\nsetting, where the domain is known only for a subset of training inputs. We\nshow that our model leads to substantial performance improvements over\ncompetitive benchmark domain adaptation methods, including methods using\nadversarial learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02900v1",
            "date": "2019-06-07 05:10:15+00:00",
            "title": "Compositional Questions Do Not Necessitate Multi-hop Reasoning",
            "authors": [
                "Sewon Min",
                "Eric Wallace",
                "Sameer Singh",
                "Matt Gardner",
                "Hannaneh Hajishirzi",
                "Luke Zettlemoyer"
            ],
            "abstract": "Multi-hop reading comprehension (RC) questions are challenging because they\nrequire reading and reasoning over multiple paragraphs. We argue that it can be\ndifficult to construct large multi-hop RC datasets. For example, even highly\ncompositional questions can be answered with a single hop if they target\nspecific entity types, or the facts needed to answer them are redundant. Our\nanalysis is centered on HotpotQA, where we show that single-hop reasoning can\nsolve much more of the dataset than previously thought. We introduce a\nsingle-hop BERT-based RC model that achieves 67 F1---comparable to\nstate-of-the-art multi-hop models. We also design an evaluation setting where\nhumans are not shown all of the necessary paragraphs for the intended multi-hop\nreasoning but can still answer over 80% of questions. Together with detailed\nerror analysis, these results suggest there should be an increasing focus on\nthe role of evidence in multi-hop reasoning and possibly even a shift towards\ninformation retrieval style evaluations with large and diverse evidence\ncollections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02916v2",
            "date": "2019-06-07 06:22:17+00:00",
            "title": "Multi-hop Reading Comprehension through Question Decomposition and Rescoring",
            "authors": [
                "Sewon Min",
                "Victor Zhong",
                "Luke Zettlemoyer",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Multi-hop Reading Comprehension (RC) requires reasoning and aggregation\nacross several paragraphs. We propose a system for multi-hop RC that decomposes\na compositional question into simpler sub-questions that can be answered by\noff-the-shelf single-hop RC models. Since annotations for such decomposition\nare expensive, we recast sub-question generation as a span prediction problem\nand show that our method, trained using only 400 labeled examples, generates\nsub-questions that are as effective as human-authored sub-questions. We also\nintroduce a new global rescoring approach that considers each decomposition\n(i.e. the sub-questions and their answers) to select the best final answer,\ngreatly improving overall performance. Our experiments on HotpotQA show that\nthis approach achieves the state-of-the-art results, while providing\nexplainable evidence for its decision making in the form of sub-questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03100v1",
            "date": "2019-06-07 13:48:46+00:00",
            "title": "Shared-Private Bilingual Word Embeddings for Neural Machine Translation",
            "authors": [
                "Xuebo Liu",
                "Derek F. Wong",
                "Yang Liu",
                "Lidia S. Chao",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "Word embedding is central to neural machine translation (NMT), which has\nattracted intensive research interest in recent years. In NMT, the source\nembedding plays the role of the entrance while the target embedding acts as the\nterminal. These layers occupy most of the model parameters for representation\nlearning. Furthermore, they indirectly interface via a soft-attention\nmechanism, which makes them comparatively isolated. In this paper, we propose\nshared-private bilingual word embeddings, which give a closer relationship\nbetween the source and target embeddings, and which also reduce the number of\nmodel parameters. For similar source and target words, their embeddings tend to\nshare a part of the features and they cooperatively learn these common\nrepresentation units. Experiments on 5 language pairs belonging to 6 different\nlanguage families and written in 5 different alphabets demonstrate that the\nproposed model provides a significant performance boost over the strong\nbaselines with dramatically fewer model parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03129v1",
            "date": "2019-06-07 14:32:17+00:00",
            "title": "Word-based Domain Adaptation for Neural Machine Translation",
            "authors": [
                "Shen Yan",
                "Leonard Dahlmann",
                "Pavel Petrushkov",
                "Sanjika Hewavitharana",
                "Shahram Khadivi"
            ],
            "abstract": "In this paper, we empirically investigate applying word-level weights to\nadapt neural machine translation to e-commerce domains, where small e-commerce\ndatasets and large out-of-domain datasets are available. In order to mine\nin-domain like words in the out-of-domain datasets, we compute word weights by\nusing a domain-specific and a non-domain-specific language model followed by\nsmoothing and binary quantization. The baseline model is trained on mixed\nin-domain and out-of-domain datasets. Experimental results on English to\nChinese e-commerce domain translation show that compared to continuing training\nwithout word weights, it improves MT quality by up to 2.11% BLEU absolute and\n1.59% TER. We have also trained models using fine-tuning on the in-domain data.\nPre-training a model with word weights improves fine-tuning up to 1.24% BLEU\nabsolute and 1.64% TER, respectively.",
            "journal ref": "Proceedings of the 15th International Workshop on Spoken Language\n  Translation, Bruges, Belgium, October 29-30, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03158v1",
            "date": "2019-06-07 15:26:50+00:00",
            "title": "Matching the Blanks: Distributional Similarity for Relation Learning",
            "authors": [
                "Livio Baldini Soares",
                "Nicholas FitzGerald",
                "Jeffrey Ling",
                "Tom Kwiatkowski"
            ],
            "abstract": "General purpose relation extractors, which can model arbitrary relations, are\na core aspiration in information extraction. Efforts have been made to build\ngeneral purpose extractors that represent relations with their surface forms,\nor which jointly embed surface forms with relations from an existing knowledge\ngraph. However, both of these approaches are limited in their ability to\ngeneralize. In this paper, we build on extensions of Harris' distributional\nhypothesis to relations, as well as recent advances in learning text\nrepresentations (specifically, BERT), to build task agnostic relation\nrepresentations solely from entity-linked text. We show that these\nrepresentations significantly outperform previous work on exemplar based\nrelation extraction (FewRel) even without using any of that task's training\ndata. We also show that models initialized with our task agnostic\nrepresentations, and then tuned on supervised relation extraction datasets,\nsignificantly outperform the previous methods on SemEval 2010 Task 8, KBP37,\nand TACRED.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03293v1",
            "date": "2019-06-07 18:45:51+00:00",
            "title": "Assessing incrementality in sequence-to-sequence models",
            "authors": [
                "Dennis Ulmer",
                "Dieuwke Hupkes",
                "Elia Bruni"
            ],
            "abstract": "Since their inception, encoder-decoder models have successfully been applied\nto a wide array of problems in computational linguistics. The most recent\nsuccesses are predominantly due to the use of different variations of attention\nmechanisms, but their cognitive plausibility is questionable. In particular,\nbecause past representations can be revisited at any point in time,\nattention-centric methods seem to lack an incentive to build up incrementally\nmore informative representations of incoming sentences. This way of processing\nstands in stark contrast with the way in which humans are believed to process\nlanguage: continuously and rapidly integrating new information as it is\nencountered. In this work, we propose three novel metrics to assess the\nbehavior of RNNs with and without an attention mechanism and identify key\ndifferences in the way the different model types process sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03348v1",
            "date": "2019-06-07 22:26:29+00:00",
            "title": "Classifying the reported ability in clinical mobility descriptions",
            "authors": [
                "Denis Newman-Griffis",
                "Ayah Zirikly",
                "Guy Divita",
                "Bart Desmet"
            ],
            "abstract": "Assessing how individuals perform different activities is key information for\nmodeling health states of individuals and populations. Descriptions of activity\nperformance in clinical free text are complex, including syntactic negation and\nsimilarities to textual entailment tasks. We explore a variety of methods for\nthe novel task of classifying four types of assertions about activity\nperformance: Able, Unable, Unclear, and None (no information). We find that\nensembling an SVM trained with lexical features and a CNN achieves 77.9% macro\nF1 score on our task, and yields nearly 80% recall on the rare Unclear and\nUnable samples. Finally, we highlight several challenges in classifying\nperformance assertions, including capturing information about sources of\nassistance, incorporating syntactic structure and negation scope, and handling\nnew modalities at test time. Our findings establish a strong baseline for this\nnovel task, and identify intriguing areas for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03360v1",
            "date": "2019-06-08 00:01:08+00:00",
            "title": "Deep Contextualized Biomedical Abbreviation Expansion",
            "authors": [
                "Qiao Jin",
                "Jinling Liu",
                "Xinghua Lu"
            ],
            "abstract": "Automatic identification and expansion of ambiguous abbreviations are\nessential for biomedical natural language processing applications, such as\ninformation retrieval and question answering systems. In this paper, we present\nDEep Contextualized Biomedical. Abbreviation Expansion (DECBAE) model. DECBAE\nautomatically collects substantial and relatively clean annotated contexts for\n950 ambiguous abbreviations from PubMed abstracts using a simple heuristic.\nThen it utilizes BioELMo to extract the contextualized features of words, and\nfeed those features to abbreviation-specific bidirectional LSTMs, where the\nhidden states of the ambiguous abbreviations are used to assign the exact\ndefinitions. Our DECBAE model outperforms other baselines by large margins,\nachieving average accuracy of 0.961 and macro-F1 of 0.917 on the dataset. It\nalso surpasses human performance for expanding a sample abbreviation, and\nremains robust in imbalanced, low-resources and clinical settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03496v1",
            "date": "2019-06-08 18:19:50+00:00",
            "title": "Making Asynchronous Stochastic Gradient Descent Work for Transformers",
            "authors": [
                "Alham Fikri Aji",
                "Kenneth Heafield"
            ],
            "abstract": "Asynchronous stochastic gradient descent (SGD) is attractive from a speed\nperspective because workers do not wait for synchronization. However, the\nTransformer model converges poorly with asynchronous SGD, resulting in\nsubstantially lower quality compared to synchronous SGD. To investigate why\nthis is the case, we isolate differences between asynchronous and synchronous\nmethods to investigate batch size and staleness effects. We find that summing\nseveral asynchronous updates, rather than applying them immediately, restores\nconvergence behavior. With this hybrid method, Transformer training for neural\nmachine translation task reaches a near-convergence level 1.36x faster in\nsingle-node multi-GPU training with no impact on model quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03520v2",
            "date": "2019-06-08 20:54:02+00:00",
            "title": "Domain Adaptive Dialog Generation via Meta Learning",
            "authors": [
                "Kun Qian",
                "Zhou Yu"
            ],
            "abstract": "Domain adaptation is an essential task in dialog system building because\nthere are so many new dialog tasks created for different needs every day.\nCollecting and annotating training data for these new tasks is costly since it\ninvolves real user interactions. We propose a domain adaptive dialog generation\nmethod based on meta-learning (DAML). DAML is an end-to-end trainable dialog\nsystem model that learns from multiple rich-resource tasks and then adapts to\nnew domains with minimal training samples. We train a dialog system model using\nmultiple rich-resource single-domain dialog data by applying the model-agnostic\nmeta-learning algorithm to dialog domain. The model is capable of learning a\ncompetitive dialog system on a new domain with only a few training examples in\nan efficient manner. The two-step gradient updates in DAML enable the model to\nlearn general features across multiple tasks. We evaluate our method on a\nsimulated dialog dataset and achieve state-of-the-art performance, which is\ngeneralizable to new tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03591v2",
            "date": "2019-06-09 08:15:53+00:00",
            "title": "A Survey on Neural Network Language Models",
            "authors": [
                "Kun Jing",
                "Jungang Xu"
            ],
            "abstract": "As the core component of Natural Language Processing (NLP) system, Language\nModel (LM) can provide word representation and probability indication of word\nsequences. Neural Network Language Models (NNLMs) overcome the curse of\ndimensionality and improve the performance of traditional LMs. A survey on\nNNLMs is performed in this paper. The structure of classic NNLMs is described\nfirstly, and then some major improvements are introduced and analyzed. We\nsummarize and compare corpora and toolkits of NNLMs. Further, some research\ndirections of NNLMs are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03608v1",
            "date": "2019-06-09 09:45:24+00:00",
            "title": "Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Katharina Kann",
                "Timothy J. Hazen",
                "Eneko Agirre",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word embeddings typically represent different meanings of a word in a single\nconflated vector. Empirical analysis of embeddings of ambiguous words is\ncurrently limited by the small size of manually annotated resources and by the\nfact that word senses are treated as unrelated individual concepts. We present\na large dataset based on manual Wikipedia annotations and word senses, where\nword senses from different words are related by semantic classes. This is the\nbasis for novel diagnostic tests for an embedding's content: we probe word\nembeddings for semantic classes and analyze the embedding space by classifying\nembeddings into semantic classes. Our main findings are: (i) Information about\na sense is generally represented well in a single-vector embedding - if the\nsense is frequent. (ii) A classifier can accurately predict whether a word is\nsingle-sense or multi-sense, based only on its embedding. (iii) Although rare\nsenses are not well represented in single-vector embeddings, this does not have\nnegative impact on an NLP application whose performance depends on frequent\nsenses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03672v1",
            "date": "2019-06-09 16:56:31+00:00",
            "title": "Question Answering as Global Reasoning over Semantic Abstractions",
            "authors": [
                "Daniel Khashabi",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Dan Roth"
            ],
            "abstract": "We propose a novel method for exploiting the semantic structure of text to\nanswer multiple-choice questions. The approach is especially suitable for\ndomains that require reasoning over a diverse set of linguistic constructs but\nhave limited training data. To address these challenges, we present the first\nsystem, to the best of our knowledge, that reasons over a wide range of\nsemantic abstractions of the text, which are derived using off-the-shelf,\ngeneral-purpose, pre-trained natural language modules such as semantic role\nlabelers, coreference resolvers, and dependency parsers. Representing multiple\nabstractions as a family of graphs, we translate question answering (QA) into a\nsearch for an optimal subgraph that satisfies certain global and local\nproperties. This formulation generalizes several prior structured QA systems.\nOur system, SEMANTICILP, demonstrates strong performance on two domains\nsimultaneously. In particular, on a collection of challenging science QA\ndatasets, it outperforms various state-of-the-art approaches, including neural\nmodels, broad coverage information retrieval, and specialized techniques using\nstructured knowledge bases, by 2%-6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03695v1",
            "date": "2019-06-09 19:25:27+00:00",
            "title": "Gendered Pronoun Resolution using BERT and an extractive question answering formulation",
            "authors": [
                "Rakesh Chada"
            ],
            "abstract": "The resolution of ambiguous pronouns is a longstanding challenge in Natural\nLanguage Understanding. Recent studies have suggested gender bias among\nstate-of-the-art coreference resolution systems. As an example, Google AI\nLanguage team recently released a gender-balanced dataset and showed that\nperformance of these coreference resolvers is significantly limited on the\ndataset. In this paper, we propose an extractive question answering (QA)\nformulation of pronoun resolution task that overcomes this limitation and shows\nmuch lower gender bias (0.99) on their dataset. This system uses fine-tuned\nrepresentations from the pre-trained BERT model and outperforms the existing\nbaseline by a significant margin (22.2% absolute improvement in F1 score)\nwithout using any hand-engineered features. This QA framework is equally\nperformant even without the knowledge of the candidate antecedents of the\npronoun. An ensemble of QA and BERT-based multiple choice and sequence\nclassification models further improves the F1 (23.3% absolute improvement upon\nthe baseline). This ensemble model was submitted to the shared task for the 1st\nACL workshop on Gender Bias for Natural Language Processing. It ranked 9th on\nthe final official leaderboard. Source code is available at\nhttps://github.com/rakeshchada/corefqa",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03741v1",
            "date": "2019-06-10 00:24:26+00:00",
            "title": "BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization",
            "authors": [
                "Eva Sharma",
                "Chen Li",
                "Lu Wang"
            ],
            "abstract": "Most existing text summarization datasets are compiled from the news domain,\nwhere summaries have a flattened discourse structure. In such datasets,\nsummary-worthy content often appears in the beginning of input articles.\nMoreover, large segments from input articles are present verbatim in their\nrespective summaries. These issues impede the learning and evaluation of\nsystems that can understand an article's global content structure as well as\nproduce abstractive summaries with high compression ratio. In this work, we\npresent a novel dataset, BIGPATENT, consisting of 1.3 million records of U.S.\npatent documents along with human written abstractive summaries. Compared to\nexisting summarization datasets, BIGPATENT has the following properties: i)\nsummaries contain a richer discourse structure with more recurring entities,\nii) salient content is evenly distributed in the input, and iii) lesser and\nshorter extractive fragments are present in the summaries. Finally, we train\nand evaluate baselines and popular learning models on BIGPATENT to shed light\non new challenges and motivate future directions for summarization research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03824v1",
            "date": "2019-06-10 07:49:14+00:00",
            "title": "A Survey on Neural Machine Reading Comprehension",
            "authors": [
                "Boyu Qiu",
                "Xu Chen",
                "Jungang Xu",
                "Yingfei Sun"
            ],
            "abstract": "Enabling a machine to read and comprehend the natural language documents so\nthat it can answer some questions remains an elusive challenge. In recent\nyears, the popularity of deep learning and the establishment of large-scale\ndatasets have both promoted the prosperity of Machine Reading Comprehension.\nThis paper aims to present how to utilize the Neural Network to build a Reader\nand introduce some classic models, analyze what improvements they make.\nFurther, we also point out the defects of existing models and future research\ndirections",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03890v1",
            "date": "2019-06-10 10:44:23+00:00",
            "title": "Automatically Identifying Complaints in Social Media",
            "authors": [
                "Daniel Preotiuc-Pietro",
                "Mihaela Gaman",
                "Nikolaos Aletras"
            ],
            "abstract": "Complaining is a basic speech act regularly used in human and computer\nmediated communication to express a negative mismatch between reality and\nexpectations in a particular situation. Automatically identifying complaints in\nsocial media is of utmost importance for organizations or brands to improve the\ncustomer experience or in developing dialogue systems for handling and\nresponding to complaints. In this paper, we introduce the first systematic\nanalysis of complaints in computational linguistics. We collect a new annotated\ndata set of written complaints expressed in English on Twitter.\\footnote{Data\nand code is available here:\n\\url{https://github.com/danielpreotiuc/complaints-social-media}} We present an\nextensive linguistic analysis of complaining as a speech act in social media\nand train strong feature-based and neural models of complaints across nine\ndomains achieving a predictive performance of up to 79 F1 using distant\nsupervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03940v1",
            "date": "2019-05-21 00:38:33+00:00",
            "title": "Predicting TED Talk Ratings from Language and Prosody",
            "authors": [
                "Md Iftekhar Tanveer",
                "Md Kamrul Hassan",
                "Daniel Gildea",
                "M. Ehsan Hoque"
            ],
            "abstract": "We use the largest open repository of public speaking---TED Talks---to\npredict the ratings of the online viewers. Our dataset contains over 2200 TED\nTalk transcripts (includes over 200 thousand sentences), audio features and the\nassociated meta information including about 5.5 Million ratings from\nspontaneous visitors of the website. We propose three neural network\narchitectures and compare with statistical machine learning. Our experiments\nreveal that it is possible to predict all the 14 different ratings with an\naverage AUC of 0.83 using the transcripts and prosody features only. The\ndataset and the complete source code is available for further analysis.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1906.04102v1",
            "date": "2019-06-10 16:23:07+00:00",
            "title": "Detecting Everyday Scenarios in Narrative Texts",
            "authors": [
                "Lilian D. A. Wanzare",
                "Michael Roth",
                "Manfred Pinkal"
            ],
            "abstract": "Script knowledge consists of detailed information on everyday activities.\nSuch information is often taken for granted in text and needs to be inferred by\nreaders. Therefore, script knowledge is a central component to language\ncomprehension. Previous work on representing scripts is mostly based on\nextensive manual work or limited to scenarios that can be found with sufficient\nredundancy in large corpora. We introduce the task of scenario detection, in\nwhich we identify references to scripts. In this task, we address a wide range\nof different scripts (200 scenarios) and we attempt to identify all references\nto them in a collection of narrative texts. We present a first benchmark data\nset and a baseline model that tackles scenario detection using techniques from\ntopic segmentation and text classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04177v2",
            "date": "2019-06-10 16:39:16+00:00",
            "title": "Estimating Causal Effects of Tone in Online Debates",
            "authors": [
                "Dhanya Sridhar",
                "Lise Getoor"
            ],
            "abstract": "Statistical methods applied to social media posts shed light on the dynamics\nof online dialogue. For example, users' wording choices predict their\npersuasiveness and users adopt the language patterns of other dialogue\nparticipants. In this paper, we estimate the causal effect of reply tones in\ndebates on linguistic and sentiment changes in subsequent responses. The\nchallenge for this estimation is that a reply's tone and subsequent responses\nare confounded by the users' ideologies on the debate topic and their emotions.\nTo overcome this challenge, we learn representations of ideology using\ngenerative models of text. We study debates from 4Forums and compare annotated\ntones of replying such as emotional versus factual, or reasonable versus\nattacking. We show that our latent confounder representation reduces bias in\nATE estimation. Our results suggest that factual and asserting tones affect\ndialogue and provide a methodology for estimating causal effects from text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04236v1",
            "date": "2019-06-10 19:11:01+00:00",
            "title": "Identifying Visible Actions in Lifestyle Vlogs",
            "authors": [
                "Oana Ignat",
                "Laura Burdick",
                "Jia Deng",
                "Rada Mihalcea"
            ],
            "abstract": "We consider the task of identifying human actions visible in online videos.\nWe focus on the widely spread genre of lifestyle vlogs, which consist of videos\nof people performing actions while verbally describing them. Our goal is to\nidentify if actions mentioned in the speech description of a video are visually\npresent. We construct a dataset with crowdsourced manual annotations of visible\nactions, and introduce a multimodal algorithm that leverages information\nderived from visual and linguistic clues to automatically infer which actions\nare visible in a video. We demonstrate that our multimodal algorithm\noutperforms algorithms based only on one modality at a time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04329v1",
            "date": "2019-06-11 00:40:33+00:00",
            "title": "Federated Learning for Emoji Prediction in a Mobile Keyboard",
            "authors": [
                "Swaroop Ramaswamy",
                "Rajiv Mathews",
                "Kanishka Rao",
                "Fran\u00e7oise Beaufays"
            ],
            "abstract": "We show that a word-level recurrent neural network can predict emoji from\ntext typed on a mobile keyboard. We demonstrate the usefulness of transfer\nlearning for predicting emoji by pretraining the model using a language\nmodeling task. We also propose mechanisms to trigger emoji and tune the\ndiversity of candidates. The model is trained using a distributed on-device\nlearning framework called federated learning. The federated model is shown to\nachieve better performance than a server-trained model. This work demonstrates\nthe feasibility of using federated learning to train production-quality models\nfor natural language understanding tasks while keeping users' data on their\ndevices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04331v2",
            "date": "2019-06-11 00:43:38+00:00",
            "title": "Parallel Scheduled Sampling",
            "authors": [
                "Daniel Duckworth",
                "Arvind Neelakantan",
                "Ben Goodrich",
                "Lukasz Kaiser",
                "Samy Bengio"
            ],
            "abstract": "Auto-regressive models are widely used in sequence generation problems. The\noutput sequence is typically generated in a predetermined order, one discrete\nunit (pixel or word or character) at a time. The models are trained by\nteacher-forcing where ground-truth history is fed to the model as input, which\nat test time is replaced by the model prediction. Scheduled Sampling aims to\nmitigate this discrepancy between train and test time by randomly replacing\nsome discrete units in the history with the model's prediction. While\nteacher-forced training works well with ML accelerators as the computation can\nbe parallelized across time, Scheduled Sampling involves undesirable sequential\nprocessing. In this paper, we introduce a simple technique to parallelize\nScheduled Sampling across time. Experimentally, we find the proposed technique\nleads to equivalent or better performance on image generation, summarization,\ndialog generation, and translation compared to teacher-forced training. In\ndialog response generation task, Parallel Scheduled Sampling achieves 1.6 BLEU\nscore (11.5%) improvement over teacher-forcing while in image generation it\nachieves 20% and 13.8% improvement in Frechet Inception Distance (FID) and\nInception Score (IS) respectively. Further, we discuss the effects of different\nhyper-parameters associated with Scheduled Sampling on the model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04393v1",
            "date": "2019-06-11 04:56:17+00:00",
            "title": "Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks",
            "authors": [
                "Yi Tay",
                "Aston Zhang",
                "Luu Anh Tuan",
                "Jinfeng Rao",
                "Shuai Zhang",
                "Shuohang Wang",
                "Jie Fu",
                "Siu Cheung Hui"
            ],
            "abstract": "Many state-of-the-art neural models for NLP are heavily parameterized and\nthus memory inefficient. This paper proposes a series of lightweight and memory\nefficient neural architectures for a potpourri of natural language processing\n(NLP) tasks. To this end, our models exploit computation using Quaternion\nalgebra and hypercomplex spaces, enabling not only expressive inter-component\ninteractions but also significantly ($75\\%$) reduced parameter size due to\nlesser degrees of freedom in the Hamilton product. We propose Quaternion\nvariants of models, giving rise to new architectures such as the Quaternion\nattention Model and Quaternion Transformer. Extensive experiments on a battery\nof NLP tasks demonstrates the utility of proposed Quaternion-inspired models,\nenabling up to $75\\%$ reduction in parameter size without significant loss in\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04447v1",
            "date": "2019-06-11 08:54:23+00:00",
            "title": "Reinforcement Learning of Minimalist Numeral Grammars",
            "authors": [
                "Peter beim Graben",
                "Ronald R\u00f6mer",
                "Werner Meyer",
                "Markus Huber",
                "Matthias Wolff"
            ],
            "abstract": "Speech-controlled user interfaces facilitate the operation of devices and\nhousehold functions to laymen. State-of-the-art language technology scans the\nacoustically analyzed speech signal for relevant keywords that are subsequently\ninserted into semantic slots to interpret the user's intent. In order to\ndevelop proper cognitive information and communication technologies, simple\nslot-filling should be replaced by utterance meaning transducers (UMT) that are\nbased on semantic parsers and a \\emph{mental lexicon}, comprising syntactic,\nphonetic and semantic features of the language under consideration. This\nlexicon must be acquired by a cognitive agent during interaction with its\nusers. We outline a reinforcement learning algorithm for the acquisition of the\nsyntactic morphology and arithmetic semantics of English numerals, based on\nminimalist grammar (MG), a recent computational implementation of generative\nlinguistics. Number words are presented to the agent by a teacher in form of\nutterance meaning pairs (UMP) where the meanings are encoded as arithmetic\nterms from a suitable term algebra. Since MG encodes universal linguistic\ncompetence through inference rules, thereby separating innate linguistic\nknowledge from the contingently acquired lexicon, our approach unifies\ngenerative grammar and reinforcement learning, hence potentially resolving the\nstill pending Chomsky-Skinner controversy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04464v3",
            "date": "2019-06-11 09:47:26+00:00",
            "title": "Relationship-Embedded Representation Learning for Grounding Referring Expressions",
            "authors": [
                "Sibei Yang",
                "Guanbin Li",
                "Yizhou Yu"
            ],
            "abstract": "Grounding referring expressions in images aims to locate the object instance\nin an image described by a referring expression. It involves a joint\nunderstanding of natural language and image content, and is essential for a\nrange of visual tasks related to human-computer interaction. As a\nlanguage-to-vision matching task, the core of this problem is to not only\nextract all the necessary information (i.e., objects and the relationships\namong them) in both the image and referring expression, but also make full use\nof context information to align cross-modal semantic concepts in the extracted\ninformation. Unfortunately, existing work on grounding referring expressions\nfails to accurately extract multi-order relationships from the referring\nexpression and associate them with the objects and their related contexts in\nthe image. In this paper, we propose a Cross-Modal Relationship Extractor\n(CMRE) to adaptively highlight objects and relationships (spatial and semantic\nrelations) related to the given expression with a cross-modal attention\nmechanism, and represent the extracted information as a language-guided visual\nrelation graph. In addition, we propose a Gated Graph Convolutional Network\n(GGCN) to compute multimodal semantic contexts by fusing information from\ndifferent modes and propagating multimodal information in the structured\nrelation graph. Experimental results on three common benchmark datasets show\nthat our Cross-Modal Relationship Inference Network, which consists of CMRE and\nGGCN, significantly surpasses all existing state-of-the-art methods. Code is\navailable at https://github.com/sibeiyang/sgmn/tree/master/lib/cmrin_models",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.04501v1",
            "date": "2019-06-11 11:26:25+00:00",
            "title": "Modeling Sentiment Dependencies with Graph Convolutional Networks for Aspect-level Sentiment Classification",
            "authors": [
                "Pinlong Zhaoa",
                "Linlin Houb",
                "Ou Wua"
            ],
            "abstract": "Aspect-level sentiment classification aims to distinguish the sentiment\npolarities over one or more aspect terms in a sentence. Existing approaches\nmostly model different aspects in one sentence independently, which ignore the\nsentiment dependencies between different aspects. However, we find such\ndependency information between different aspects can bring additional valuable\ninformation. In this paper, we propose a novel aspect-level sentiment\nclassification model based on graph convolutional networks (GCN) which can\neffectively capture the sentiment dependencies between multi-aspects in one\nsentence. Our model firstly introduces bidirectional attention mechanism with\nposition encoding to model aspect-specific representations between each aspect\nand its context words, then employs GCN over the attention mechanism to capture\nthe sentiment dependencies between different aspects in one sentence. We\nevaluate the proposed approach on the SemEval 2014 datasets. Experiments show\nthat our model outperforms the state-of-the-art methods. We also conduct\nexperiments to evaluate the effectiveness of GCN module, which indicates that\nthe dependencies between different aspects is highly helpful in aspect-level\nsentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04684v1",
            "date": "2019-06-11 16:30:27+00:00",
            "title": "Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network",
            "authors": [
                "Sunil Kumar Sahu",
                "Fenia Christopoulou",
                "Makoto Miwa",
                "Sophia Ananiadou"
            ],
            "abstract": "Inter-sentence relation extraction deals with a number of complex semantic\nrelationships in documents, which require local, non-local, syntactic and\nsemantic dependencies. Existing methods do not fully exploit such dependencies.\nWe present a novel inter-sentence relation extraction model that builds a\nlabelled edge graph convolutional neural network model on a document-level\ngraph. The graph is constructed using various inter- and intra-sentence\ndependencies to capture local and non-local dependency information. In order to\npredict the relation of an entity pair, we utilise multi-instance learning with\nbi-affine pairwise scoring. Experimental results show that our model achieves\ncomparable performance to the state-of-the-art neural models on two\nbiochemistry datasets. Our analysis shows that all the types in the graph are\neffective for inter-sentence relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04706v1",
            "date": "2019-06-11 17:15:32+00:00",
            "title": "Using Structured Representation and Data: A Hybrid Model for Negation and Sentiment in Customer Service Conversations",
            "authors": [
                "Amita Misra",
                "Mansurul Bhuiyan",
                "Jalal Mahmud",
                "Saurabh Tripathy"
            ],
            "abstract": "Twitter customer service interactions have recently emerged as an effective\nplatform to respond and engage with customers. In this work, we explore the\nrole of negation in customer service interactions, particularly applied to\nsentiment analysis. We define rules to identify true negation cues and scope\nmore suited to conversational data than existing general review data. Using\nsemantic knowledge and syntactic structure from constituency parse trees, we\npropose an algorithm for scope detection that performs comparable to state of\nthe art BiLSTM. We further investigate the results of negation scope detection\nfor the sentiment prediction task on customer service conversation data using\nboth a traditional SVM and a Neural Network. We propose an antonym dictionary\nbased method for negation applied to a CNN-LSTM combination model for sentiment\nanalysis. Experimental results show that the antonym-based method outperforms\nthe previous lexicon-based and neural network methods.",
            "journal ref": "Proceedings of the 10th Workshop on Computational Approaches to\n  Subjectivity, Sentiment and Social Media Analysis, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04903v1",
            "date": "2019-06-12 02:48:57+00:00",
            "title": "Does BLEU Score Work for Code Migration?",
            "authors": [
                "Ngoc Tran",
                "Hieu Tran",
                "Son Nguyen",
                "Hoan Nguyen",
                "Tien N. Nguyen"
            ],
            "abstract": "Statistical machine translation (SMT) is a fast-growing sub-field of\ncomputational linguistics. Until now, the most popular automatic metric to\nmeasure the quality of SMT is BiLingual Evaluation Understudy (BLEU) score.\nLately, SMT along with the BLEU metric has been applied to a Software\nEngineering task named code migration. (In)Validating the use of BLEU score\ncould advance the research and development of SMT-based code migration tools.\nUnfortunately, there is no study to approve or disapprove the use of BLEU score\nfor source code. In this paper, we conducted an empirical study on BLEU score\nto (in)validate its suitability for the code migration task due to its\ninability to reflect the semantics of source code. In our work, we use human\njudgment as the ground truth to measure the semantic correctness of the\nmigrated code. Our empirical study demonstrates that BLEU does not reflect\ntranslation quality due to its weak correlation with the semantic correctness\nof translated code. We provided counter-examples to show that BLEU is\nineffective in comparing the translation quality between SMT-based models. Due\nto BLEU's ineffectiveness for code migration task, we propose an alternative\nmetric RUBY, which considers lexical, syntactical, and semantic representations\nof source code. We verified that RUBY achieves a higher correlation coefficient\nwith the semantic correctness of migrated code, 0.775 in comparison with 0.583\nof BLEU score. We also confirmed the effectiveness of RUBY in reflecting the\nchanges in translation quality of SMT-based translation models. With its\nadvantages, RUBY can be used to evaluate SMT-based code migration models.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1906.04940v1",
            "date": "2019-06-12 04:48:27+00:00",
            "title": "CogCompTime: A Tool for Understanding Time in Natural Language Text",
            "authors": [
                "Qiang Ning",
                "Ben Zhou",
                "Zhili Feng",
                "Haoruo Peng",
                "Dan Roth"
            ],
            "abstract": "Automatic extraction of temporal information in text is an important\ncomponent of natural language understanding. It involves two basic tasks: (1)\nUnderstanding time expressions that are mentioned explicitly in text (e.g.,\nFebruary 27, 1998 or tomorrow), and (2) Understanding temporal information that\nis conveyed implicitly via relations. In this paper, we introduce CogCompTime,\na system that has these two important functionalities. It incorporates the most\nrecent progress, achieves state-of-the-art performance, and is publicly\navailable.1 We believe that this demo will be useful for multiple time-aware\napplications and provide valuable insight for future research in temporal\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04943v1",
            "date": "2019-06-12 05:07:42+00:00",
            "title": "A Structured Learning Approach to Temporal Relation Extraction",
            "authors": [
                "Qiang Ning",
                "Zhili Feng",
                "Dan Roth"
            ],
            "abstract": "Identifying temporal relations between events is an essential step towards\nnatural language understanding. However, the temporal relation between two\nevents in a story depends on, and is often dictated by, relations among other\nevents. Consequently, effectively identifying temporal relations between events\nis a challenging problem even for human annotators. This paper suggests that it\nis important to take these dependencies into account while learning to identify\nthese relations and proposes a structured learning approach to address this\nchallenge. As a byproduct, this provides a new perspective on handling missing\nrelations, a known issue that hurts existing methods. As we show, the proposed\napproach results in significant improvements on the two commonly used data sets\nfor this problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05012v1",
            "date": "2019-06-12 09:03:26+00:00",
            "title": "BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization",
            "authors": [
                "Kai Wang",
                "Xiaojun Quan",
                "Rui Wang"
            ],
            "abstract": "The success of neural summarization models stems from the meticulous\nencodings of source articles. To overcome the impediments of limited and\nsometimes noisy training data, one promising direction is to make better use of\nthe available training data by applying filters during summarization. In this\npaper, we propose a novel Bi-directional Selective Encoding with Template\n(BiSET) model, which leverages template discovered from training data to softly\nselect key information from each source article to guide its summarization\nprocess. Extensive experiments on a standard summarization dataset were\nconducted and the results show that the template-equipped BiSET model manages\nto improve the summarization performance significantly with a new state of the\nart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05062v1",
            "date": "2019-06-12 11:27:38+00:00",
            "title": "Unified Semantic Parsing with Weak Supervision",
            "authors": [
                "Priyanka Agrawal",
                "Parag Jain",
                "Ayushi Dalmia",
                "Abhishek Bansal",
                "Ashish Mittal",
                "Karthik Sankaranarayanan"
            ],
            "abstract": "Semantic parsing over multiple knowledge bases enables a parser to exploit\nstructural similarities of programs across the multiple domains. However, the\nfundamental challenge lies in obtaining high-quality annotations of (utterance,\nprogram) pairs across various domains needed for training such models. To\novercome this, we propose a novel framework to build a unified multi-domain\nenabled semantic parser trained only with weak supervision (denotations).\nWeakly supervised training is particularly arduous as the program search space\ngrows exponentially in a multi-domain setting. To solve this, we incorporate a\nmulti-policy distillation mechanism in which we first train domain-specific\nsemantic parsers (teachers) using weak supervision in the absence of the ground\ntruth programs, followed by training a single unified parser (student) from the\ndomain specific policies obtained from these teachers. The resultant semantic\nparser is not only compact but also generalizes better, and generates more\naccurate programs. It further does not require the user to provide a domain\nlabel while querying. On the standard Overnight dataset (containing multiple\ndomains), we demonstrate that the proposed model improves performance by 20% in\nterms of denotation accuracy in comparison to baseline techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05190v3",
            "date": "2019-06-12 15:01:31+00:00",
            "title": "Vispi: Automatic Visual Perception and Interpretation of Chest X-rays",
            "authors": [
                "Xin Li",
                "Rui Cao",
                "Dongxiao Zhu"
            ],
            "abstract": "Medical imaging contains the essential information for rendering diagnostic\nand treatment decisions. Inspecting (visual perception) and interpreting image\nto generate a report are tedious clinical routines for a radiologist where\nautomation is expected to greatly reduce the workload. Despite rapid\ndevelopment of natural image captioning, computer-aided medical image visual\nperception and interpretation remain a challenging task, largely due to the\nlack of high-quality annotated image-report pairs and tailor-made generative\nmodels for sufficient extraction and exploitation of localized semantic\nfeatures, particularly those associated with abnormalities. To tackle these\nchallenges, we present Vispi, an automatic medical image interpretation system,\nwhich first annotates an image via classifying and localizing common thoracic\ndiseases with visual support and then followed by report generation from an\nattentive LSTM model. Analyzing an open IU X-ray dataset, we demonstrate a\nsuperior performance of Vispi in disease classification, localization and\nreport generation using automatic performance evaluation metrics ROUGE and\nCIDEr.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.05210v1",
            "date": "2019-06-12 15:26:59+00:00",
            "title": "Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension",
            "authors": [
                "Yichen Jiang",
                "Nitish Joshi",
                "Yen-Chun Chen",
                "Mohit Bansal"
            ],
            "abstract": "Multi-hop reading comprehension requires the model to explore and connect\nrelevant information from multiple sentences/documents in order to answer the\nquestion about the context. To achieve this, we propose an interpretable\n3-module system called Explore-Propose-Assemble reader (EPAr). First, the\nDocument Explorer iteratively selects relevant documents and represents\ndivergent reasoning chains in a tree structure so as to allow assimilating\ninformation from all chains. The Answer Proposer then proposes an answer from\nevery root-to-leaf path in the reasoning tree. Finally, the Evidence Assembler\nextracts a key sentence containing the proposed answer from every path and\ncombines them to predict the final answer. Intuitively, EPAr approximates the\ncoarse-to-fine-grained comprehension behavior of human readers when facing\nmultiple long documents. We jointly optimize our 3 modules by minimizing the\nsum of losses from each stage conditioned on the previous stage's output. On\ntwo multi-hop reading comprehension datasets WikiHop and MedHop, our EPAr model\nachieves significant improvements over the baseline and competitive results\ncompared to the state-of-the-art model. We also present multiple\nreasoning-chain-recovery tests and ablation studies to demonstrate our system's\nability to perform interpretable and accurate reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05275v2",
            "date": "2019-06-12 17:58:27+00:00",
            "title": "Keeping Notes: Conditional Natural Language Generation with a Scratchpad Mechanism",
            "authors": [
                "Ryan Y. Benmalek",
                "Madian Khabsa",
                "Suma Desu",
                "Claire Cardie",
                "Michele Banko"
            ],
            "abstract": "We introduce the Scratchpad Mechanism, a novel addition to the\nsequence-to-sequence (seq2seq) neural network architecture and demonstrate its\neffectiveness in improving the overall fluency of seq2seq models for natural\nlanguage generation tasks. By enabling the decoder at each time step to write\nto all of the encoder output layers, Scratchpad can employ the encoder as a\n\"scratchpad\" memory to keep track of what has been generated so far and thereby\nguide future generation. We evaluate Scratchpad in the context of three\nwell-studied natural language generation tasks --- Machine Translation,\nQuestion Generation, and Text Summarization --- and obtain state-of-the-art or\ncomparable performance on standard datasets for each task. Qualitative\nassessments in the form of human judgements (question generation), attention\nvisualization (MT), and sample output (summarization) provide further evidence\nof the ability of Scratchpad to generate fluent and expressive output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05317v2",
            "date": "2019-06-12 18:11:20+00:00",
            "title": "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction",
            "authors": [
                "Antoine Bosselut",
                "Hannah Rashkin",
                "Maarten Sap",
                "Chaitanya Malaviya",
                "Asli Celikyilmaz",
                "Yejin Choi"
            ],
            "abstract": "We present the first comprehensive study on automatic knowledge base\nconstruction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et\nal., 2019) and ConceptNet (Speer et al., 2017). Contrary to many conventional\nKBs that store knowledge with canonical templates, commonsense KBs only store\nloosely structured open-text descriptions of knowledge. We posit that an\nimportant step toward automatic commonsense completion is the development of\ngenerative models of commonsense knowledge, and propose COMmonsEnse\nTransformers (COMET) that learn to generate rich and diverse commonsense\ndescriptions in natural language. Despite the challenges of commonsense\nmodeling, our investigation reveals promising results when implicit knowledge\nfrom deep pre-trained language models is transferred to generate explicit\nknowledge in commonsense knowledge graphs. Empirical results demonstrate that\nCOMET is able to generate novel knowledge that humans rate as high quality,\nwith up to 77.5% (ATOMIC) and 91.7% (ConceptNet) precision at top 1, which\napproaches human performance for these resources. Our findings suggest that\nusing generative commonsense models for automatic commonsense KB completion\ncould soon be a plausible alternative to extractive methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05394v1",
            "date": "2019-06-12 21:49:06+00:00",
            "title": "Neural Arabic Question Answering",
            "authors": [
                "Hussein Mozannar",
                "Karl El Hajal",
                "Elie Maamary",
                "Hazem Hajj"
            ],
            "abstract": "This paper tackles the problem of open domain factual Arabic question\nanswering (QA) using Wikipedia as our knowledge source. This constrains the\nanswer of any question to be a span of text in Wikipedia. Open domain QA for\nArabic entails three challenges: annotated QA datasets in Arabic, large scale\nefficient information retrieval and machine reading comprehension. To deal with\nthe lack of Arabic QA datasets we present the Arabic Reading Comprehension\nDataset (ARCD) composed of 1,395 questions posed by crowdworkers on Wikipedia\narticles, and a machine translation of the Stanford Question Answering Dataset\n(Arabic-SQuAD). Our system for open domain question answering in Arabic (SOQAL)\nis based on two components: (1) a document retriever using a hierarchical\nTF-IDF approach and (2) a neural reading comprehension model using the\npre-trained bi-directional transformer BERT. Our experiments on ARCD indicate\nthe effectiveness of our approach with our BERT-based reader achieving a 61.3\nF1 score, and our open domain system SOQAL achieving a 27.6 F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05407v1",
            "date": "2019-06-12 22:19:26+00:00",
            "title": "Analyzing the Limitations of Cross-lingual Word Embedding Mappings",
            "authors": [
                "Aitor Ormazabal",
                "Mikel Artetxe",
                "Gorka Labaka",
                "Aitor Soroa",
                "Eneko Agirre"
            ],
            "abstract": "Recent research in cross-lingual word embeddings has almost exclusively\nfocused on offline methods, which independently train word embeddings in\ndifferent languages and map them to a shared space through linear\ntransformations. While several authors have questioned the underlying\nisomorphism assumption, which states that word embeddings in different\nlanguages have approximately the same structure, it is not clear whether this\nis an inherent limitation of mapping approaches or a more general issue when\nlearning cross-lingual embeddings. So as to answer this question, we experiment\nwith parallel corpora, which allows us to compare offline mapping to an\nextension of skip-gram that jointly learns both embedding spaces. We observe\nthat, under these ideal conditions, joint learning yields to more isomorphic\nembeddings, is less sensitive to hubness, and obtains stronger results in\nbilingual lexicon induction. We thus conclude that current mapping methods do\nhave strong limitations, calling for further research to jointly learn\ncross-lingual embeddings with a weaker cross-lingual signal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05466v2",
            "date": "2019-06-13 03:42:34+00:00",
            "title": "Figurative Usage Detection of Symptom Words to Improve Personal Health Mention Detection",
            "authors": [
                "Adith Iyer",
                "Aditya Joshi",
                "Sarvnaz Karimi",
                "Ross Sparks",
                "Cecile Paris"
            ],
            "abstract": "Personal health mention detection deals with predicting whether or not a\ngiven sentence is a report of a health condition. Past work mentions errors in\nthis prediction when symptom words, i.e. names of symptoms of interest, are\nused in a figurative sense. Therefore, we combine a state-of-the-art figurative\nusage detection with CNN-based personal health mention detection. To do so, we\npresent two methods: a pipeline-based approach and a feature augmentation-based\napproach. The introduction of figurative usage detection results in an average\nimprovement of 2.21% F-score of personal health mention detection, in the case\nof the feature augmentation-based approach. This paper demonstrates the promise\nof using figurative usage detection to improve personal health mention\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05468v1",
            "date": "2019-06-13 03:48:34+00:00",
            "title": "A Comparison of Word-based and Context-based Representations for Classification Problems in Health Informatics",
            "authors": [
                "Aditya Joshi",
                "Sarvnaz Karimi",
                "Ross Sparks",
                "Cecile Paris",
                "C Raina MacIntyre"
            ],
            "abstract": "Distributed representations of text can be used as features when training a\nstatistical classifier. These representations may be created as a composition\nof word vectors or as context-based sentence vectors. We compare the two kinds\nof representations (word versus context) for three classification problems:\ninfluenza infection classification, drug usage classification and personal\nhealth mention classification. For statistical classifiers trained for each of\nthese problems, context-based representations based on ELMo, Universal Sentence\nEncoder, Neural-Net Language Model and FLAIR are better than Word2Vec, GloVe\nand the two adapted using the MESH ontology. There is an improvement of 2-4% in\nthe accuracy when these context-based representations are used instead of\nword-based representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05483v1",
            "date": "2019-06-13 05:15:25+00:00",
            "title": "Enriching Neural Models with Targeted Features for Dementia Detection",
            "authors": [
                "Flavio Di Palo",
                "Natalie Parde"
            ],
            "abstract": "Alzheimer's disease (AD) is an irreversible brain disease that can\ndramatically reduce quality of life, most commonly manifesting in older adults\nand eventually leading to the need for full-time care. Early detection is\nfundamental to slowing its progression; however, diagnosis can be expensive,\ntime-consuming, and invasive. In this work we develop a neural model based on a\nCNN-LSTM architecture that learns to detect AD and related dementias using\ntargeted and implicitly-learned features from conversational transcripts. Our\napproach establishes the new state of the art on the DementiaBank dataset,\nachieving an F1 score of 0.929 when classifying participants into AD and\ncontrol groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05659v1",
            "date": "2019-06-10 21:19:41+00:00",
            "title": "Deep Two-path Semi-supervised Learning for Fake News Detection",
            "authors": [
                "Xishuang Dong",
                "Uboho Victor",
                "Shanta Chowdhury",
                "Lijun Qian"
            ],
            "abstract": "News in social media such as Twitter has been generated in high volume and\nspeed. However, very few of them can be labeled (as fake or true news) in a\nshort time. In order to achieve timely detection of fake news in social media,\na novel deep two-path semi-supervised learning model is proposed, where one\npath is for supervised learning and the other is for unsupervised learning.\nThese two paths implemented with convolutional neural networks are jointly\noptimized to enhance detection performance. In addition, we build a shared\nconvolutional neural networks between these two paths to share the low level\nfeatures. Experimental results using Twitter datasets show that the proposed\nmodel can recognize fake news effectively with very few labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05670v1",
            "date": "2019-06-13 13:33:22+00:00",
            "title": "KCAT: A Knowledge-Constraint Typing Annotation Tool",
            "authors": [
                "Sheng Lin",
                "Luye Zheng",
                "Bo Chen",
                "Siliang Tang",
                "Yueting Zhuang",
                "Fei Wu",
                "Zhigang Chen",
                "Guoping Hu",
                "Xiang Ren"
            ],
            "abstract": "Fine-grained Entity Typing is a tough task which suffers from noise samples\nextracted from distant supervision. Thousands of manually annotated samples can\nachieve greater performance than millions of samples generated by the previous\ndistant supervision method. Whereas, it's hard for human beings to\ndifferentiate and memorize thousands of types, thus making large-scale human\nlabeling hardly possible. In this paper, we introduce a Knowledge-Constraint\nTyping Annotation Tool (KCAT), which is efficient for fine-grained entity\ntyping annotation. KCAT reduces the size of candidate types to an acceptable\nrange for human beings through entity linking and provides a Multi-step Typing\nscheme to revise the entity linking result. Moreover, KCAT provides an\nefficient Annotator Client to accelerate the annotation process and a\ncomprehensive Manager Module to analyse crowdsourcing annotations. Experiment\nshows that KCAT can significantly improve annotation efficiency, the time\nconsumption increases slowly as the size of type set expands.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.05760v1",
            "date": "2019-06-13 15:40:50+00:00",
            "title": "Semantic Change and Semantic Stability: Variation is Key",
            "authors": [
                "Claire Bowern"
            ],
            "abstract": "I survey some recent approaches to studying change in the lexicon,\nparticularly change in meaning across phylogenies. I briefly sketch an\nevolutionary approach to language change and point out some issues in recent\napproaches to studying semantic change that rely on temporally stratified word\nembeddings. I draw illustrations from lexical cognate models in Pama-Nyungan to\nidentify meaning classes most appropriate for lexical phylogenetic inference,\nparticularly highlighting the importance of variation in studying change over\ntime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05833v2",
            "date": "2019-06-09 12:42:23+00:00",
            "title": "There is no Artificial General Intelligence",
            "authors": [
                "J. Landgrebe",
                "B. Smith"
            ],
            "abstract": "The goal of creating Artificial General Intelligence (AGI) -- or in other\nwords of creating Turing machines (modern computers) that can behave in a way\nthat mimics human intelligence -- has occupied AI researchers ever since the\nidea of AI was first proposed. One common theme in these discussions is the\nthesis that the ability of a machine to conduct convincing dialogues with human\nbeings can serve as at least a sufficient criterion of AGI. We argue that this\nvery ability should be accepted also as a necessary condition of AGI, and we\nprovide a description of the nature of human dialogue in particular and of\nhuman language in general against this background. We then argue that it is for\nmathematical reasons impossible to program a machine in such a way that it\ncould master human dialogue behaviour in its full generality. This is (1)\nbecause there are no traditional explicitly designed mathematical models that\ncould be used as a starting point for creating such programs; and (2) because\neven the sorts of automated models generated by using machine learning, which\nhave been used successfully in areas such as machine translation, cannot be\nextended to cope with human dialogue. If this is so, then we can conclude that\na Turing machine also cannot possess AGI, because it fails to fulfil a\nnecessary condition thereof. At the same time, however, we acknowledge the\npotential of Turing machines to master dialogue behaviour in highly restricted\ncontexts, where what is called ``narrow'' AI can still be of considerable\nutility.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.05939v2",
            "date": "2019-06-13 21:40:15+00:00",
            "title": "Embedding Biomedical Ontologies by Jointly Encoding Network Structure and Textual Node Descriptors",
            "authors": [
                "Sotiris Kotitsas",
                "Dimitris Pappas",
                "Ion Androutsopoulos",
                "Ryan McDonald",
                "Marianna Apidianaki"
            ],
            "abstract": "Network Embedding (NE) methods, which map network nodes to low-dimensional\nfeature vectors, have wide applications in network analysis and bioinformatics.\nMany existing NE methods rely only on network structure, overlooking other\ninformation associated with the nodes, e.g., text describing the nodes. Recent\nattempts to combine the two sources of information only consider local network\nstructure. We extend NODE2VEC, a well-known NE method that considers broader\nnetwork structure, to also consider textual node descriptors using recurrent\nneural encoders. Our method is evaluated on link prediction in two networks\nderived from UMLS. Experimental results demonstrate the effectiveness of the\nproposed approach compared to previous work.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.05963v2",
            "date": "2019-06-14 00:00:29+00:00",
            "title": "Image Captioning: Transforming Objects into Words",
            "authors": [
                "Simao Herdade",
                "Armin Kappeler",
                "Kofi Boakye",
                "Joao Soares"
            ],
            "abstract": "Image captioning models typically follow an encoder-decoder architecture\nwhich uses abstract image feature vectors as input to the encoder. One of the\nmost successful algorithms uses feature vectors extracted from the region\nproposals obtained from an object detector. In this work we introduce the\nObject Relation Transformer, that builds upon this approach by explicitly\nincorporating information about the spatial relationship between input detected\nobjects through geometric attention. Quantitative and qualitative results\ndemonstrate the importance of such geometric attention for image captioning,\nleading to improvements on all common captioning metrics on the MS-COCO\ndataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.06187v1",
            "date": "2019-06-14 13:05:08+00:00",
            "title": "NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language",
            "authors": [
                "Leon Weber",
                "Pasquale Minervini",
                "Jannes M\u00fcnchmeyer",
                "Ulf Leser",
                "Tim Rockt\u00e4schel"
            ],
            "abstract": "Rule-based models are attractive for various tasks because they inherently\nlead to interpretable and explainable decisions and can easily incorporate\nprior knowledge. However, such systems are difficult to apply to problems\ninvolving natural language, due to its linguistic variability. In contrast,\nneural models can cope very well with ambiguity by learning distributed\nrepresentations of words and their composition from data, but lead to models\nthat are difficult to interpret. In this paper, we describe a model combining\nneural networks with logic programming in a novel manner for solving multi-hop\nreasoning tasks over natural language. Specifically, we propose to use a Prolog\nprover which we extend to utilize a similarity function over pretrained\nsentence encoders. We fine-tune the representations for the similarity function\nvia backpropagation. This leads to a system that can apply rule-based reasoning\nto natural language, and induce domain-specific rules from training data. We\nevaluate the proposed system on two different question answering tasks, showing\nthat it outperforms two baselines -- BIDAF (Seo et al., 2016a) and FAST QA\n(Weissenborn et al., 2017b) on a subset of the WikiHop corpus and achieves\ncompetitive results on the MedHop data set (Welbl et al., 2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06207v1",
            "date": "2019-06-14 13:55:12+00:00",
            "title": "Cumulative Adaptation for BLSTM Acoustic Models",
            "authors": [
                "Markus Kitza",
                "Pavel Golik",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "This paper addresses the robust speech recognition problem as an adaptation\ntask. Specifically, we investigate the cumulative application of adaptation\nmethods. A bidirectional Long Short-Term Memory (BLSTM) based neural network,\ncapable of learning temporal relationships and translation invariant\nrepresentations, is used for robust acoustic modelling. Further, i-vectors were\nused as an input to the neural network to perform instantaneous speaker and\nenvironment adaptation, providing 8\\% relative improvement in word error rate\non the NIST Hub5 2000 evaluation test set. By enhancing the first-pass i-vector\nbased adaptation with a second-pass adaptation using speaker and environment\ndependent transformations within the network, a further relative improvement of\n5\\% in word error rate was achieved. We have reevaluated the features used to\nestimate i-vectors and their normalization to achieve the best performance in a\nmodern large scale automatic speech recognition system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06253v1",
            "date": "2019-06-14 15:36:22+00:00",
            "title": "A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning",
            "authors": [
                "Gon\u00e7alo M. Correia",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Automatic post-editing (APE) seeks to automatically refine the output of a\nblack-box machine translation (MT) system through human post-edits. APE systems\nare usually trained by complementing human post-edited data with large,\nartificial data generated through back-translations, a time-consuming process\noften no easier than training an MT system from scratch. In this paper, we\npropose an alternative where we fine-tune pre-trained BERT models on both the\nencoder and decoder of an APE system, exploring several parameter sharing\nstrategies. By only training on a dataset of 23K sentences for 3 hours on a\nsingle GPU, we obtain results that are competitive with systems that were\ntrained on 5M artificial sentences. When we add this artificial data, our\nmethod obtains state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06332v1",
            "date": "2019-06-14 06:38:33+00:00",
            "title": "IITP at MEDIQA 2019: Systems Report for Natural Language Inference, Question Entailment and Question Answering",
            "authors": [
                "Dibyanayan Bandyopadhyay",
                "Baban Gain",
                "Tanik Saikh",
                "Asif Ekbal"
            ],
            "abstract": "This paper presents the experiments accomplished as a part of our\nparticipation in the MEDIQA challenge, an (Abacha et al., 2019) shared task. We\nparticipated in all the three tasks defined in this particular shared task. The\ntasks are viz. i. Natural Language Inference (NLI) ii. Recognizing Question\nEntailment(RQE) and their application in medical Question Answering (QA). We\nsubmitted runs using multiple deep learning based systems (runs) for each of\nthese three tasks. We submitted five system results in each of the NLI and RQE\ntasks, and four system results for the QA task. The systems yield encouraging\nresults in all three tasks. The highest performance obtained in NLI, RQE and QA\ntasks are 81.8%, 53.2%, and 71.7%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06349v2",
            "date": "2019-06-14 18:04:39+00:00",
            "title": "On the Computational Power of RNNs",
            "authors": [
                "Samuel A. Korsky",
                "Robert C. Berwick"
            ],
            "abstract": "Recent neural network architectures such as the basic recurrent neural\nnetwork (RNN) and Gated Recurrent Unit (GRU) have gained prominence as\nend-to-end learning architectures for natural language processing tasks. But\nwhat is the computational power of such systems? We prove that finite precision\nRNNs with one hidden layer and ReLU activation and finite precision GRUs are\nexactly as computationally powerful as deterministic finite automata. Allowing\narbitrary precision, we prove that RNNs with one hidden layer and ReLU\nactivation are at least as computationally powerful as pushdown automata. If we\nalso allow infinite precision, infinite edge weights, and nonlinear output\nactivation functions, we prove that GRUs are at least as computationally\npowerful as pushdown automata. All results are shown constructively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06425v1",
            "date": "2019-06-14 22:52:21+00:00",
            "title": "Principled Frameworks for Evaluating Ethics in NLP Systems",
            "authors": [
                "Shrimai Prabhumoye",
                "Elijah Mayfield",
                "Alan W Black"
            ],
            "abstract": "We critique recent work on ethics in natural language processing. Those\ndiscussions have focused on data collection, experimental design, and\ninterventions in modeling. But we argue that we ought to first understand the\nframeworks of ethics that are being used to evaluate the fairness and justice\nof algorithmic systems. Here, we begin that discussion by outlining\ndeontological ethics, and envision a research agenda prioritized by it.",
            "journal ref": "Widening NLP Workshop at ACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06438v1",
            "date": "2019-06-14 23:42:08+00:00",
            "title": "Scalable Syntax-Aware Language Models Using Knowledge Distillation",
            "authors": [
                "Adhiguna Kuncoro",
                "Chris Dyer",
                "Laura Rimell",
                "Stephen Clark",
                "Phil Blunsom"
            ],
            "abstract": "Prior work has shown that, on small amounts of training data, syntactic\nneural language models learn structurally sensitive generalisations more\nsuccessfully than sequential language models. However, their computational\ncomplexity renders scaling difficult, and it remains an open question whether\nstructural biases are still necessary when sequential models have access to\never larger amounts of training data. To answer this question, we introduce an\nefficient knowledge distillation (KD) technique that transfers knowledge from a\nsyntactic language model trained on a small corpus to an LSTM language model,\nhence enabling the LSTM to develop a more structurally sensitive representation\nof the larger training data it learns from. On targeted syntactic evaluations,\nwe find that, while sequential LSTMs perform much better than previously\nreported, our proposed technique substantially improves on this baseline,\nyielding a new state of the art. Our findings and analysis affirm the\nimportance of structural biases, even in models that learn from large amounts\nof data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06442v1",
            "date": "2019-06-15 00:36:41+00:00",
            "title": "Tagged Back-Translation",
            "authors": [
                "Isaac Caswell",
                "Ciprian Chelba",
                "David Grangier"
            ],
            "abstract": "Recent work in Neural Machine Translation (NMT) has shown significant quality\ngains from noised-beam decoding during back-translation, a method to generate\nsynthetic parallel data. We show that the main role of such synthetic noise is\nnot to diversify the source side, as previously suggested, but simply to\nindicate to the model that the given source is synthetic. We propose a simpler\nalternative to noising techniques, consisting of tagging back-translated source\nsentences with an extra token. Our results on WMT outperform noised\nback-translation in English-Romanian and match performance on English-German,\nre-defining state-of-the-art in the former.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06464v1",
            "date": "2019-06-12 00:03:33+00:00",
            "title": "Action-Sensitive Phonological Dependencies",
            "authors": [
                "Yiding Hao",
                "Dustin Bowers"
            ],
            "abstract": "This paper defines a subregular class of functions called the tier-based\nsynchronized strictly local (TSSL) functions. These functions are similar to\nthe the tier-based input-output strictly local (TIOSL) functions, except that\nthe locality condition is enforced not on the input and output streams, but on\nthe computation history of the minimal subsequential finite-state transducer.\nWe show that TSSL functions naturally describe rhythmic syncope while TIOSL\nfunctions cannot, and we argue that TSSL functions provide a more restricted\ncharacterization of rhythmic syncope than existing treatments within Optimality\nTheory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06481v1",
            "date": "2019-06-15 06:58:42+00:00",
            "title": "A Hierarchical Attention Based Seq2seq Model for Chinese Lyrics Generation",
            "authors": [
                "Haoshen Fan",
                "Jie Wang",
                "Bojin Zhuang",
                "Shaojun Wang",
                "Jing Xiao"
            ],
            "abstract": "In this paper, we comprehensively study on context-aware generation of\nChinese song lyrics. Conventional text generative models generate a sequence or\nsentence word by word, failing to consider the contextual relationship between\nsentences. Taking account into the characteristics of lyrics, a hierarchical\nattention based Seq2Seq (Sequence-to-Sequence) model is proposed for Chinese\nlyrics generation. With encoding of word-level and sentence-level contextual\ninformation, this model promotes the topic relevance and consistency of\ngeneration. A large Chinese lyrics corpus is also leveraged for model training.\nEventually, results of automatic and human evaluations demonstrate that our\nmodel is able to compose complete Chinese lyrics with one united topic\nconstraint.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06581v2",
            "date": "2019-06-15 15:55:43+00:00",
            "title": "Practical User Feedback-driven Internal Search Using Online Learning to Rank",
            "authors": [
                "Rajhans Samdani",
                "Pierre Rappolt",
                "Ankit Goyal",
                "Pratyus Patnaik"
            ],
            "abstract": "We present a system, Spoke, for creating and searching internal knowledge\nbase (KB) articles for organizations. Spoke is available as a SaaS\n(Software-as-a-Service) product deployed across hundreds of organizations with\na diverse set of domains. Spoke continually improves search quality using\nconversational user feedback which allows it to provide better search\nexperience than standard information retrieval systems without encoding any\nexplicit domain knowledge. We achieve this by using a real-time online\nlearning-to-rank (L2R) algorithm that automatically customizes relevance\nscoring for each organization deploying Spoke by using a query similarity\nkernel.\n  The focus of this paper is on incorporating practical considerations into our\nrelevance scoring function and algorithm that make Spoke easy to deploy and\nsuitable for handling events that naturally happen over the life-cycle of any\nKB deployment. We show that Spoke outperforms competitive baselines by up to\n41% in offline F1 comparisons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06593v2",
            "date": "2019-06-15 17:29:06+00:00",
            "title": "Context is Key: Grammatical Error Detection with Contextual Word Representations",
            "authors": [
                "Samuel Bell",
                "Helen Yannakoudakis",
                "Marek Rei"
            ],
            "abstract": "Grammatical error detection (GED) in non-native writing requires systems to\nidentify a wide range of errors in text written by language learners. Error\ndetection as a purely supervised task can be challenging, as GED datasets are\nlimited in size and the label distributions are highly imbalanced.\nContextualized word representations offer a possible solution, as they can\nefficiently capture compositional information in language and can be optimized\non large amounts of unsupervised data. In this paper, we perform a systematic\ncomparison of ELMo, BERT and Flair embeddings (Peters et al., 2017; Devlin et\nal., 2018; Akbik et al., 2018) on a range of public GED datasets, and propose\nan approach to effectively integrate such representations in current methods,\nachieving a new state of the art on GED. We further analyze the strengths and\nweaknesses of different contextual embeddings for the task at hand, and present\ndetailed analyses of their impact on different types of errors.",
            "journal ref": "Proceedings of the Fourteenth Workshop on Innovative Use of NLP\n  for Building Educational Applications (BEA 2019), pp. 103-115",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06606v1",
            "date": "2019-06-15 19:17:10+00:00",
            "title": "Multi-Hop Paragraph Retrieval for Open-Domain Question Answering",
            "authors": [
                "Yair Feldman",
                "Ran El-Yaniv"
            ],
            "abstract": "This paper is concerned with the task of multi-hop open-domain Question\nAnswering (QA). This task is particularly challenging since it requires the\nsimultaneous performance of textual reasoning and efficient searching. We\npresent a method for retrieving multiple supporting paragraphs, nested amidst a\nlarge knowledge base, which contain the necessary evidence to answer a given\nquestion. Our method iteratively retrieves supporting paragraphs by forming a\njoint vector representation of both a question and a paragraph. The retrieval\nis performed by considering contextualized sentence-level representations of\nthe paragraphs in the knowledge source. Our method achieves state-of-the-art\nperformance over two well-known datasets, SQuAD-Open and HotpotQA, which serve\nas our single- and multi-hop open-domain QA benchmarks, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06685v1",
            "date": "2019-06-16 13:33:55+00:00",
            "title": "Improving Background Based Conversation with Context-aware Knowledge Pre-selection",
            "authors": [
                "Yangjun Zhang",
                "Pengjie Ren",
                "Maarten de Rijke"
            ],
            "abstract": "Background Based Conversations (BBCs) have been developed to make dialogue\nsystems generate more informative and natural responses by leveraging\nbackground knowledge. Existing methods for BBCs can be grouped into two\ncategories: extraction-based methods and generation-based methods. The former\nextract spans frombackground material as responses that are not necessarily\nnatural. The latter generate responses thatare natural but not necessarily\neffective in leveraging background knowledge. In this paper, we focus on\ngeneration-based methods and propose a model, namely Context-aware Knowledge\nPre-selection (CaKe), which introduces a pre-selection process that uses\ndynamic bi-directional attention to improve knowledge selection by using the\nutterance history context as prior information to select the most relevant\nbackground material. Experimental results show that our model is superior to\ncurrent state-of-the-art baselines, indicating that it benefits from the\npre-selection process, thus improving in-formativeness and fluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06849v1",
            "date": "2019-06-17 05:37:27+00:00",
            "title": "A Multi-Task Architecture on Relevance-based Neural Query Translation",
            "authors": [
                "Sheikh Muhammad Sarwar",
                "Hamed Bonab",
                "James Allan"
            ],
            "abstract": "We describe a multi-task learning approach to train a Neural Machine\nTranslation (NMT) model with a Relevance-based Auxiliary Task (RAT) for search\nquery translation. The translation process for Cross-lingual Information\nRetrieval (CLIR) task is usually treated as a black box and it is performed as\nan independent step. However, an NMT model trained on sentence-level parallel\ndata is not aware of the vocabulary distribution of the retrieval corpus. We\naddress this problem with our multi-task learning architecture that achieves\n16% improvement over a strong NMT baseline on Italian-English query-document\ndataset. We show using both quantitative and qualitative analysis that our\nmodel generates balanced and precise translations with the regularization\neffect it achieves from multi-task learning paradigm.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.06917v1",
            "date": "2019-06-17 09:33:30+00:00",
            "title": "Recursive Style Breach Detection with Multifaceted Ensemble Learning",
            "authors": [
                "Daniel Kopev",
                "Dimitrina Zlatkova",
                "Kristiyan Mitov",
                "Atanas Atanasov",
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "We present a supervised approach for style change detection, which aims at\npredicting whether there are changes in the style in a given text document, as\nwell as at finding the exact positions where such changes occur. In particular,\nwe combine a TF.IDF representation of the document with features specifically\nengineered for the task, and we make predictions via an ensemble of diverse\nclassifiers including SVM, Random Forest, AdaBoost, MLP, and LightGBM. Whenever\nthe model detects that style change is present, we apply it recursively,\nlooking to find the specific positions of the change. Our approach powered the\nwinning system for the PAN@CLEF 2018 task on Style Change Detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06945v1",
            "date": "2019-06-17 10:59:50+00:00",
            "title": "Context-aware Embedding for Targeted Aspect-based Sentiment Analysis",
            "authors": [
                "Bin Liang",
                "Jiachen Du",
                "Ruifeng Xu",
                "Binyang Li",
                "Hejiao Huang"
            ],
            "abstract": "Attention-based neural models were employed to detect the different aspects\nand sentiment polarities of the same target in targeted aspect-based sentiment\nanalysis (TABSA). However, existing methods do not specifically pre-train\nreasonable embeddings for targets and aspects in TABSA. This may result in\ntargets or aspects having the same vector representations in different contexts\nand losing the context-dependent information. To address this problem, we\npropose a novel method to refine the embeddings of targets and aspects. Such\npivotal embedding refinement utilizes a sparse coefficient vector to adjust the\nembeddings of target and aspect from the context. Hence the embeddings of\ntargets and aspects can be refined from the highly correlative words instead of\nusing context-independent or randomly initialized vectors. Experiment results\non two benchmark datasets show that our approach yields the state-of-the-art\nperformance in TABSA task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07004v1",
            "date": "2019-06-14 06:45:08+00:00",
            "title": "Improving Multi-turn Dialogue Modelling with Utterance ReWriter",
            "authors": [
                "Hui Su",
                "Xiaoyu Shen",
                "Rongzhi Zhang",
                "Fei Sun",
                "Pengwei Hu",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Recent research has made impressive progress in single-turn dialogue\nmodelling. In the multi-turn setting, however, current models are still far\nfrom satisfactory. One major challenge is the frequently occurred coreference\nand information omission in our daily conversation, making it hard for machines\nto understand the real intention. In this paper, we propose rewriting the human\nutterance as a pre-process to help multi-turn dialgoue modelling. Each\nutterance is first rewritten to recover all coreferred and omitted information.\nThe next processing steps are then performed based on the rewritten utterance.\nTo properly train the utterance rewriter, we collect a new dataset with human\nannotations and introduce a Transformer-based utterance rewriting architecture\nusing the pointer network. We show the proposed architecture achieves\nremarkably good performance on the utterance rewriting task. The trained\nutterance rewriter can be easily integrated into online chatbots and brings\ngeneral improvement over different domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07132v1",
            "date": "2019-06-17 17:03:57+00:00",
            "title": "Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA",
            "authors": [
                "Yichen Jiang",
                "Mohit Bansal"
            ],
            "abstract": "Multi-hop question answering requires a model to connect multiple pieces of\nevidence scattered in a long context to answer the question. In this paper, we\nshow that in the multi-hop HotpotQA (Yang et al., 2018) dataset, the examples\noften contain reasoning shortcuts through which models can directly locate the\nanswer by word-matching the question with a sentence in the context. We\ndemonstrate this issue by constructing adversarial documents that create\ncontradicting answers to the shortcut but do not affect the validity of the\noriginal answer. The performance of strong baseline models drops significantly\non our adversarial evaluation, indicating that they are indeed exploiting the\nshortcuts rather than performing multi-hop reasoning. After adversarial\ntraining, the baseline's performance improves but is still limited on the\nadversarial evaluation. Hence, we use a control unit that dynamically attends\nto the question at different reasoning hops to guide the model's multi-hop\nreasoning. We show that this 2-hop model trained on the regular data is more\nrobust to the adversaries than the baseline model. After adversarial training,\nthis 2-hop model not only achieves improvements over its counterpart trained on\nregular data, but also outperforms the adversarially-trained 1-hop baseline. We\nhope that these insights and initial improvements will motivate the development\nof new models that combine explicit compositional reasoning with adversarial\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07194v1",
            "date": "2019-06-17 18:00:04+00:00",
            "title": "Finding Your Voice: The Linguistic Development of Mental Health Counselors",
            "authors": [
                "Justine Zhang",
                "Robert Filbin",
                "Christine Morrison",
                "Jaclyn Weiser",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Mental health counseling is an enterprise with profound societal importance\nwhere conversations play a primary role. In order to acquire the conversational\nskills needed to face a challenging range of situations, mental health\ncounselors must rely on training and on continued experience with actual\nclients. However, in the absence of large scale longitudinal studies, the\nnature and significance of this developmental process remain unclear. For\nexample, prior literature suggests that experience might not translate into\nconsequential changes in counselor behavior. This has led some to even argue\nthat counseling is a profession without expertise.\n  In this work, we develop a computational framework to quantify the extent to\nwhich individuals change their linguistic behavior with experience and to study\nthe nature of this evolution. We use our framework to conduct a large\nlongitudinal study of mental health counseling conversations, tracking over\n3,400 counselors across their tenure. We reveal that overall, counselors do\nindeed change their conversational behavior to become more diverse across\ninteractions, developing an individual voice that distinguishes them from other\ncounselors. Furthermore, a finer-grained investigation shows that the rate and\nnature of this diversification vary across functionally different\nconversational components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07220v1",
            "date": "2019-06-17 18:54:51+00:00",
            "title": "Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue",
            "authors": [
                "Anusha Balakrishnan",
                "Jinfeng Rao",
                "Kartikeya Upasani",
                "Michael White",
                "Rajen Subba"
            ],
            "abstract": "Generating fluent natural language responses from structured semantic\nrepresentations is a critical step in task-oriented conversational systems.\nAvenues like the E2E NLG Challenge have encouraged the development of neural\napproaches, particularly sequence-to-sequence (Seq2Seq) models for this\nproblem. The semantic representations used, however, are often underspecified,\nwhich places a higher burden on the generation model for sentence planning, and\nalso limits the extent to which generated responses can be controlled in a live\nsystem. In this paper, we (1) propose using tree-structured semantic\nrepresentations, like those used in traditional rule-based NLG systems, for\nbetter discourse-level structuring and sentence-level planning; (2) introduce a\nchallenging dataset using this representation for the weather domain; (3)\nintroduce a constrained decoding approach for Seq2Seq models that leverages\nthis representation to improve semantic correctness; and (4) demonstrate\npromising results on our dataset and the E2E dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07286v1",
            "date": "2019-06-17 22:13:24+00:00",
            "title": "Generalizing Back-Translation in Neural Machine Translation",
            "authors": [
                "Miguel Gra\u00e7a",
                "Yunsu Kim",
                "Julian Schamper",
                "Shahram Khadivi",
                "Hermann Ney"
            ],
            "abstract": "Back-translation - data augmentation by translating target monolingual data -\nis a crucial component in modern neural machine translation (NMT). In this\nwork, we reformulate back-translation in the scope of cross-entropy\noptimization of an NMT model, clarifying its underlying mathematical\nassumptions and approximations beyond its heuristic usage. Our formulation\ncovers broader synthetic data generation schemes, including sampling from a\ntarget-to-source NMT model. With this formulation, we point out fundamental\nproblems of the sampling-based approaches and propose to remedy them by (i)\ndisabling label smoothing for the target-to-source model and (ii) sampling from\na restricted search space. Our statements are investigated on the WMT 2018\nGerman - English news translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07348v1",
            "date": "2019-06-18 02:36:39+00:00",
            "title": "Zero-Shot Entity Linking by Reading Entity Descriptions",
            "authors": [
                "Lajanugen Logeswaran",
                "Ming-Wei Chang",
                "Kenton Lee",
                "Kristina Toutanova",
                "Jacob Devlin",
                "Honglak Lee"
            ],
            "abstract": "We present the zero-shot entity linking task, where mentions must be linked\nto unseen entities without in-domain labeled data. The goal is to enable robust\ntransfer to highly specialized domains, and so no metadata or alias tables are\nassumed. In this setting, entities are only identified by text descriptions,\nand models must rely strictly on language understanding to resolve the new\nentities. First, we show that strong reading comprehension models pre-trained\non large unlabeled data can be used to generalize to unseen entities. Second,\nwe propose a simple and effective adaptive pre-training strategy, which we term\ndomain-adaptive pre-training (DAP), to address the domain shift problem\nassociated with linking unseen entities in a new domain. We present experiments\non a new dataset that we construct for this task and show that DAP improves\nover strong pre-training baselines, including BERT. The data and code are\navailable at https://github.com/lajanugen/zeshel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07510v8",
            "date": "2019-06-18 11:55:16+00:00",
            "title": "Attention Guided Graph Convolutional Networks for Relation Extraction",
            "authors": [
                "Zhijiang Guo",
                "Yan Zhang",
                "Wei Lu"
            ],
            "abstract": "Dependency trees convey rich structural information that is proven useful for\nextracting relations among entities in text. However, how to effectively make\nuse of relevant information while ignoring irrelevant information from the\ndependency trees remains a challenging research question. Existing approaches\nemploying rule based hard-pruning strategies for selecting relevant partial\ndependency structures may not always yield optimal results. In this work, we\npropose Attention Guided Graph Convolutional Networks (AGGCNs), a novel model\nwhich directly takes full dependency trees as inputs. Our model can be\nunderstood as a soft-pruning approach that automatically learns how to\nselectively attend to the relevant sub-structures useful for the relation\nextraction task. Extensive results on various tasks including cross-sentence\nn-ary relation extraction and large-scale sentence-level relation extraction\nshow that our model is able to better leverage the structural information of\nthe full dependency trees, giving significantly better results than previous\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07591v1",
            "date": "2019-06-18 14:03:14+00:00",
            "title": "Query Generation for Patent Retrieval with Keyword Extraction based on Syntactic Features",
            "authors": [
                "Julien Rossi",
                "Matthias Wirth",
                "Evangelos Kanoulas"
            ],
            "abstract": "This paper describes a new method to extract relevant keywords from patent\nclaims, as part of the task of retrieving other patents with similar claims\n(search for prior art). The method combines a qualitative analysis of the\nwriting style of the claims with NLP methods to parse text, in order to\nrepresent a legal text as a specialization arborescence of terms. In this\nsetting, the set of extracted keywords are yielding better search results than\nkeywords extracted with traditional methods such as tf-idf. The performance is\nmeasured on the search results of a query consisting of the extracted keywords.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.07839v1",
            "date": "2019-06-18 23:04:09+00:00",
            "title": "The Second DIHARD Diarization Challenge: Dataset, task, and baselines",
            "authors": [
                "Neville Ryant",
                "Kenneth Church",
                "Christopher Cieri",
                "Alejandrina Cristia",
                "Jun Du",
                "Sriram Ganapathy",
                "Mark Liberman"
            ],
            "abstract": "This paper introduces the second DIHARD challenge, the second in a series of\nspeaker diarization challenges intended to improve the robustness of\ndiarization systems to variation in recording equipment, noise conditions, and\nconversational domain. The challenge comprises four tracks evaluating\ndiarization performance under two input conditions (single channel vs.\nmulti-channel) and two segmentation conditions (diarization from a reference\nspeech segmentation vs. diarization from scratch). In order to prevent\nparticipants from overtuning to a particular combination of recording\nconditions and conversational domain, recordings are drawn from a variety of\nsources ranging from read audiobooks to meeting speech, to child language\nacquisition recordings, to dinner parties, to web video. We describe the task\nand metrics, challenge design, datasets, and baseline systems for speech\nenhancement, speech activity detection, and diarization.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.07880v3",
            "date": "2019-06-19 02:05:39+00:00",
            "title": "Second-Order Semantic Dependency Parsing with End-to-End Neural Networks",
            "authors": [
                "Xinyu Wang",
                "Jingxian Huang",
                "Kewei Tu"
            ],
            "abstract": "Semantic dependency parsing aims to identify semantic relationships between\nwords in a sentence that form a graph. In this paper, we propose a second-order\nsemantic dependency parser, which takes into consideration not only individual\ndependency edges but also interactions between pairs of edges. We show that\nsecond-order parsing can be approximated using mean field (MF) variational\ninference or loopy belief propagation (LBP). We can unfold both algorithms as\nrecurrent layers of a neural network and therefore can train the parser in an\nend-to-end manner. Our experiments show that our approach achieves\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08101v2",
            "date": "2019-06-19 13:54:25+00:00",
            "title": "Pre-Training with Whole Word Masking for Chinese BERT",
            "authors": [
                "Yiming Cui",
                "Wanxiang Che",
                "Ting Liu",
                "Bing Qin",
                "Ziqing Yang",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Bidirectional Encoder Representations from Transformers (BERT) has shown\nmarvelous improvements across various NLP tasks. Recently, an upgraded version\nof BERT has been released with Whole Word Masking (WWM), which mitigate the\ndrawbacks of masking partial WordPiece tokens in pre-training BERT. In this\ntechnical report, we adapt whole word masking in Chinese text, that masking the\nwhole word instead of masking Chinese characters, which could bring another\nchallenge in Masked Language Model (MLM) pre-training task. The proposed models\nare verified on various NLP tasks, across sentence-level to document-level,\nincluding machine reading comprehension (CMRC 2018, DRCD, CJRC), natural\nlanguage inference (XNLI), sentiment classification (ChnSentiCorp), sentence\npair matching (LCQMC, BQ Corpus), and document classification (THUCNews).\nExperimental results on these datasets show that the whole word masking could\nbring another significant gain. Moreover, we also examine the effectiveness of\nthe Chinese pre-trained models: BERT, ERNIE, BERT-wwm, BERT-wwm-ext,\nRoBERTa-wwm-ext, and RoBERTa-wwm-ext-large. We release all the pre-trained\nmodels: \\url{https://github.com/ymcui/Chinese-BERT-wwm",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08237v2",
            "date": "2019-06-19 17:35:48+00:00",
            "title": "XLNet: Generalized Autoregressive Pretraining for Language Understanding",
            "authors": [
                "Zhilin Yang",
                "Zihang Dai",
                "Yiming Yang",
                "Jaime Carbonell",
                "Ruslan Salakhutdinov",
                "Quoc V. Le"
            ],
            "abstract": "With the capability of modeling bidirectional contexts, denoising\nautoencoding based pretraining like BERT achieves better performance than\npretraining approaches based on autoregressive language modeling. However,\nrelying on corrupting the input with masks, BERT neglects dependency between\nthe masked positions and suffers from a pretrain-finetune discrepancy. In light\nof these pros and cons, we propose XLNet, a generalized autoregressive\npretraining method that (1) enables learning bidirectional contexts by\nmaximizing the expected likelihood over all permutations of the factorization\norder and (2) overcomes the limitations of BERT thanks to its autoregressive\nformulation. Furthermore, XLNet integrates ideas from Transformer-XL, the\nstate-of-the-art autoregressive model, into pretraining. Empirically, under\ncomparable experiment settings, XLNet outperforms BERT on 20 tasks, often by a\nlarge margin, including question answering, natural language inference,\nsentiment analysis, and document ranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08340v1",
            "date": "2019-06-19 20:29:31+00:00",
            "title": "Learning Compressed Sentence Representations for On-Device Text Processing",
            "authors": [
                "Dinghan Shen",
                "Pengyu Cheng",
                "Dhanasekar Sundararaman",
                "Xinyuan Zhang",
                "Qian Yang",
                "Meng Tang",
                "Asli Celikyilmaz",
                "Lawrence Carin"
            ],
            "abstract": "Vector representations of sentences, trained on massive text corpora, are\nwidely used as generic sentence embeddings across a variety of NLP problems.\nThe learned representations are generally assumed to be continuous and\nreal-valued, giving rise to a large memory footprint and slow retrieval speed,\nwhich hinders their applicability to low-resource (memory and computation)\nplatforms, such as mobile devices. In this paper, we propose four different\nstrategies to transform continuous and generic sentence embeddings into a\nbinarized form, while preserving their rich semantic information. The\nintroduced methods are evaluated across a wide range of downstream tasks, where\nthe binarized sentence embeddings are demonstrated to degrade performance by\nonly about 2% relative to their continuous counterparts, while reducing the\nstorage requirement by over 98%. Moreover, with the learned binary\nrepresentations, the semantic relatedness of two sentences can be evaluated by\nsimply calculating their Hamming distance, which is more computational\nefficient compared with the inner product operation between continuous\nembeddings. Detailed analysis and case study further validate the effectiveness\nof proposed methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08382v1",
            "date": "2019-06-19 22:23:20+00:00",
            "title": "An Open-World Extension to Knowledge Graph Completion Models",
            "authors": [
                "Haseeb Shah",
                "Johannes Villmow",
                "Adrian Ulges",
                "Ulrich Schwanecke",
                "Faisal Shafait"
            ],
            "abstract": "We present a novel extension to embedding-based knowledge graph completion\nmodels which enables them to perform open-world link prediction, i.e. to\npredict facts for entities unseen in training based on their textual\ndescription. Our model combines a regular link prediction model learned from a\nknowledge graph with word embeddings learned from a textual corpus. After\ntraining both independently, we learn a transformation to map the embeddings of\nan entity's name and description to the graph-based embedding space. In\nexperiments on several datasets including FB20k, DBPedia50k and our new dataset\nFB15k-237-OWE, we demonstrate competitive results. Particularly, our approach\nexploits the full knowledge graph structure even when textual descriptions are\nscarce, does not require a joint training on graph and text, and can be applied\nto any embedding-based link prediction model, such as TransE, ComplEx and\nDistMult.",
            "journal ref": "AAAI-19 Vol 33 (2019) 3044-3051",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.08487v1",
            "date": "2019-06-20 08:03:58+00:00",
            "title": "HappyBot: Generating Empathetic Dialogue Responses by Improving User Experience Look-ahead",
            "authors": [
                "Jamin Shin",
                "Peng Xu",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Recent neural conversation models that attempted to incorporate emotion and\ngenerate empathetic responses either focused on conditioning the output to a\ngiven emotion, or incorporating the current user emotional state. While these\napproaches have been successful to some extent in generating more diverse and\nseemingly engaging utterances, they do not factor in how the user would feel\ntowards the generated dialogue response. Hence, in this paper, we advocate such\nlook-ahead of user emotion as the key to modeling and generating empathetic\ndialogue responses. We thus train a Sentiment Predictor to estimate the user\nsentiment look-ahead towards the generated system responses, which is then used\nas the reward function for generating more empathetic responses. Human\nevaluation results show that our model outperforms other baselines in empathy,\nrelevance, and fluency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08711v3",
            "date": "2019-06-20 15:57:26+00:00",
            "title": "Few-Shot Sequence Labeling with Label Dependency Transfer and Pair-wise Embedding",
            "authors": [
                "Yutai Hou",
                "Zhihan Zhou",
                "Yijia Liu",
                "Ning Wang",
                "Wanxiang Che",
                "Han Liu",
                "Ting Liu"
            ],
            "abstract": "While few-shot classification has been widely explored with similarity based\nmethods, few-shot sequence labeling poses a unique challenge as it also calls\nfor modeling the label dependencies. To consider both the item similarity and\nlabel dependency, we propose to leverage the conditional random fields (CRFs)\nin few-shot sequence labeling. It calculates emission score with similarity\nbased methods and obtains transition score with a specially designed transfer\nmechanism. When applying CRF in the few-shot scenarios, the discrepancy of\nlabel sets among different domains makes it hard to use the label dependency\nlearned in prior domains. To tackle this, we introduce the dependency transfer\nmechanism that transfers abstract label transition patterns. In addition, the\nsimilarity methods rely on the high quality sample representation, which is\nchallenging for sequence labeling, because sense of a word is different when\nmeasuring its similarity to words in different sentences. To remedy this, we\ntake advantage of recent contextual embedding technique, and further propose a\npair-wise embedder. It provides additional certainty for word sense by\nembedding query and support sentence pairwisely. Experimental results on slot\ntagging and named entity recognition show that our model significantly\noutperforms the strongest few-shot learning baseline by 11.76 (21.2%) and 12.18\n(97.7%) F1 scores respectively in the one-shot setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08733v1",
            "date": "2019-06-20 16:25:47+00:00",
            "title": "Autonomous Haiku Generation",
            "authors": [
                "Rui Aguiar",
                "Kevin Liao"
            ],
            "abstract": "Artificial Intelligence is an excellent tool to improve efficiency and lower\ncost in many quantitative real world applications, but what if the task is not\neasily defined? What if the task is generating creativity? Poetry is a creative\nendeavor that is highly difficult to both grasp and achieve with any level of\ncompetence. As Rita Dove, a famous American poet and author states, \"Poetry is\nlanguage at its most distilled and most powerful.\" Taking Doves quote as an\ninspiration, our task was to generate high quality haikus using artificial\nintelligence and deep learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08876v1",
            "date": "2019-06-20 21:51:48+00:00",
            "title": "Informative Image Captioning with External Sources of Information",
            "authors": [
                "Sanqiang Zhao",
                "Piyush Sharma",
                "Tomer Levinboim",
                "Radu Soricut"
            ],
            "abstract": "An image caption should fluently present the essential information in a given\nimage, including informative, fine-grained entity mentions and the manner in\nwhich these entities interact. However, current captioning models are usually\ntrained to generate captions that only contain common object names, thus\nfalling short on an important \"informativeness\" dimension. We present a\nmechanism for integrating image information together with fine-grained labels\n(assumed to be generated by some upstream models) into a caption that describes\nthe image in a fluent and informative manner. We introduce a multimodal,\nmulti-encoder model based on Transformer that ingests both image features and\nmultiple sources of entity labels. We demonstrate that we can learn to control\nthe appearance of these entity labels in the output, resulting in captions that\nare both fluent and informative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08942v1",
            "date": "2019-06-21 04:29:22+00:00",
            "title": "Be Consistent! Improving Procedural Text Comprehension using Label Consistency",
            "authors": [
                "Xinya Du",
                "Bhavana Dalvi Mishra",
                "Niket Tandon",
                "Antoine Bosselut",
                "Wen-tau Yih",
                "Peter Clark",
                "Claire Cardie"
            ],
            "abstract": "Our goal is procedural text comprehension, namely tracking how the properties\nof entities (e.g., their location) change with time given a procedural text\n(e.g., a paragraph about photosynthesis, a recipe). This task is challenging as\nthe world is changing throughout the text, and despite recent advances, current\nsystems still struggle with this task. Our approach is to leverage the fact\nthat, for many procedural texts, multiple independent descriptions are readily\navailable, and that predictions from them should be consistent (label\nconsistency). We present a new learning framework that leverages label\nconsistency during training, allowing consistency bias to be built into the\nmodel. Evaluation on a standard benchmark dataset for procedural text, ProPara\n(Dalvi et al., 2018), shows that our approach significantly improves prediction\nperformance (F1) over prior state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09266v1",
            "date": "2019-06-21 17:25:45+00:00",
            "title": "A Multitask Network for Localization and Recognition of Text in Images",
            "authors": [
                "Mohammad Reza Sarshogh",
                "Keegan E. Hines"
            ],
            "abstract": "We present an end-to-end trainable multi-task network that addresses the\nproblem of lexicon-free text extraction from complex documents. This network\nsimultaneously solves the problems of text localization and text recognition\nand text segments are identified with no post-processing, cropping, or word\ngrouping. A convolutional backbone and Feature Pyramid Network are combined to\nprovide a shared representation that benefits each of three model heads: text\nlocalization, classification, and text recognition. To improve recognition\naccuracy, we describe a dynamic pooling mechanism that retains high-resolution\ninformation across all RoIs. For text recognition, we propose a convolutional\nmechanism with attention which out-performs more common recurrent\narchitectures. Our model is evaluated against benchmark datasets and comparable\nmethods and achieves high performance in challenging regimes of non-traditional\nOCR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09317v1",
            "date": "2019-06-21 20:55:57+00:00",
            "title": "Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction",
            "authors": [
                "Yufang Hou",
                "Charles Jochim",
                "Martin Gleize",
                "Francesca Bonin",
                "Debasis Ganguly"
            ],
            "abstract": "While the fast-paced inception of novel tasks and new datasets helps foster\nactive research in a community towards interesting directions, keeping track of\nthe abundance of research activity in different areas on different datasets is\nlikely to become increasingly difficult. The community could greatly benefit\nfrom an automatic system able to summarize scientific results, e.g., in the\nform of a leaderboard. In this paper we build two datasets and develop a\nframework (TDMS-IE) aimed at automatically extracting task, dataset, metric and\nscore from NLP papers, towards the automatic construction of leaderboards.\nExperiments show that our model outperforms several baselines by a large\nmargin. Our model is a first step towards automatic leaderboard construction,\ne.g., in the NLP domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09321v1",
            "date": "2019-06-15 06:47:16+00:00",
            "title": "Automatic Acrostic Couplet Generation with Three-Stage Neural Network Pipelines",
            "authors": [
                "Haoshen Fan",
                "Jie Wang",
                "Bojin Zhuang",
                "Shaojun Wang",
                "Jing Xiao"
            ],
            "abstract": "As one of the quintessence of Chinese traditional culture, couplet\ncompromises two syntactically symmetric clauses equal in length, namely, an\nantecedent and subsequent clause. Moreover, corresponding characters and\nphrases at the same position of the two clauses are paired with each other\nunder certain constraints of semantic and/or syntactic relatedness. Automatic\ncouplet generation is recognized as a challenging problem even in the\nArtificial Intelligence field. In this paper, we comprehensively study on\nautomatic generation of acrostic couplet with the first characters defined by\nusers. The complete couplet generation is mainly divided into three stages,\nthat is, antecedent clause generation pipeline, subsequent clause generation\npipeline and clause re-ranker. To realize semantic and/or syntactic relatedness\nbetween two clauses, attention-based Sequence-to-Sequence (S2S) neural network\nis employed. Moreover, to provide diverse couplet candidates for re-ranking, a\ncluster-based beam search approach is incorporated into the S2S network. Both\nBLEU metrics and human judgments have demonstrated the effectiveness of our\nproposed method. Eventually, a mini-program based on this generation system is\ndeveloped and deployed on Wechat for real users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09404v2",
            "date": "2019-06-22 07:32:15+00:00",
            "title": "RLTM: An Efficient Neural IR Framework for Long Documents",
            "authors": [
                "Chen Zheng",
                "Yu Sun",
                "Shengxian Wan",
                "Dianhai Yu"
            ],
            "abstract": "Deep neural networks have achieved significant improvements in information\nretrieval (IR). However, most existing models are computational costly and can\nnot efficiently scale to long documents. This paper proposes a novel End-to-End\nneural ranking framework called Reinforced Long Text Matching (RLTM) which\nmatches a query with long documents efficiently and effectively. The core idea\nbehind the framework can be analogous to the human judgment process which\nfirstly locates the relevance parts quickly from the whole document and then\nmatches these parts with the query carefully to obtain the final label.\nFirstly, we select relevant sentences from the long documents by a coarse and\nefficient matching model. Secondly, we generate a relevance score by a more\nsophisticated matching model based on the sentence selected. The whole model is\ntrained jointly with reinforcement learning in a pairwise manner by maximizing\nthe expected score gaps between positive and negative examples. Experimental\nresults demonstrate that RLTM has greatly improved the efficiency and\neffectiveness of the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09445v1",
            "date": "2019-06-22 13:28:32+00:00",
            "title": "Learning with fuzzy hypergraphs: a topical approach to query-oriented text summarization",
            "authors": [
                "Hadrien Van Lierde",
                "Tommy W. S. Chow"
            ],
            "abstract": "Existing graph-based methods for extractive document summarization represent\nsentences of a corpus as the nodes of a graph or a hypergraph in which edges\ndepict relationships of lexical similarity between sentences. Such approaches\nfail to capture semantic similarities between sentences when they express a\nsimilar information but have few words in common and are thus lexically\ndissimilar. To overcome this issue, we propose to extract semantic similarities\nbased on topical representations of sentences. Inspired by the Hierarchical\nDirichlet Process, we propose a probabilistic topic model in order to infer\ntopic distributions of sentences. As each topic defines a semantic connection\namong a group of sentences with a certain degree of membership for each\nsentence, we propose a fuzzy hypergraph model in which nodes are sentences and\nfuzzy hyperedges are topics. To produce an informative summary, we extract a\nset of sentences from the corpus by simultaneously maximizing their relevance\nto a user-defined query, their centrality in the fuzzy hypergraph and their\ncoverage of topics present in the corpus. We formulate a polynomial time\nalgorithm building on the theory of submodular functions to solve the\nassociated optimization problem. A thorough comparative analysis with other\ngraph-based summarization systems is included in the paper. Our obtained\nresults show the superiority of our method in terms of content coverage of the\nsummaries.",
            "journal ref": "Information Sciences, 496 (2019), 212-224",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09450v1",
            "date": "2019-06-22 14:02:14+00:00",
            "title": "Semantically Driven Auto-completion",
            "authors": [
                "Konstantine Arkoudas",
                "Mohamed Yahya"
            ],
            "abstract": "The Bloomberg Terminal has been a leading source of financial data and\nanalytics for over 30 years. Through its thousands of functions, the Terminal\nallows its users to query and run analytics over a large array of data sources,\nincluding structured, semi-structured, and unstructured data; as well as plot\ncharts, set up event-driven alerts and triggers, create interactive maps,\nexchange information via instant and email-style messages, and so on. To\nimprove user experience, we have been building question answering systems that\ncan understand a wide range of natural language constructions for various\ndomains that are of fundamental interest to our users. Such natural language\ninterfaces, while exceedingly helpful to users, introduce a number of usability\nchallenges of their own. We tackle some of these challenges through\nauto-completion for query formulation. A distinguishing mark of our\nauto-complete systems is that they are based on and guided by corresponding\nsemantic parsing systems. We describe the auto-complete problem as it arises in\nthis setting, the novel algorithms that we use to solve it, and report on the\nquality of the results and the efficiency of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09543v1",
            "date": "2019-06-23 02:56:02+00:00",
            "title": "Cross-lingual Data Transformation and Combination for Text Classification",
            "authors": [
                "Jun Jiang",
                "Shumao Pang",
                "Xia Zhao",
                "Liwei Wang",
                "Andrew Wen",
                "Hongfang Liu",
                "Qianjin Feng"
            ],
            "abstract": "Text classification is a fundamental task for text data mining. In order to\ntrain a generalizable model, a large volume of text must be collected. To\naddress data insufficiency, cross-lingual data may occasionally be necessary.\nCross-lingual data sources may however suffer from data incompatibility, as\ntext written in different languages can hold distinct word sequences and\nsemantic patterns. Machine translation and word embedding alignment provide an\neffective way to transform and combine data for cross-lingual data training. To\nthe best of our knowledge, there has been little work done on evaluating how\nthe methodology used to conduct semantic space transformation and data\ncombination affects the performance of classification models trained from\ncross-lingual resources. In this paper, we systematically evaluated the\nperformance of two commonly used CNN (Convolutional Neural Network) and RNN\n(Recurrent Neural Network) text classifiers with differing data transformation\nand combination strategies. Monolingual models were trained from English and\nFrench alongside their translated and aligned embeddings. Our results suggested\nthat semantic space transformation may conditionally promote the performance of\nmonolingual models. Bilingual models were trained from a combination of both\nEnglish and French. Our results indicate that a cross-lingual classification\nmodel can significantly benefit from cross-lingual data by learning from\ntranslated or aligned embedding spaces.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.09635v1",
            "date": "2019-06-23 19:38:53+00:00",
            "title": "Investigating Biases in Textual Entailment Datasets",
            "authors": [
                "Shawn Tan",
                "Yikang Shen",
                "Chin-wei Huang",
                "Aaron Courville"
            ],
            "abstract": "The ability to understand logical relationships between sentences is an\nimportant task in language understanding. To aid in progress for this task,\nresearchers have collected datasets for machine learning and evaluation of\ncurrent systems. However, like in the crowdsourced Visual Question Answering\n(VQA) task, some biases in the data inevitably occur. In our experiments, we\nfind that performing classification on just the hypotheses on the SNLI dataset\nyields an accuracy of 64%. We analyze the bias extent in the SNLI and the\nMultiNLI dataset, discuss its implication, and propose a simple method to\nreduce the biases in the datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09694v1",
            "date": "2019-06-24 02:59:22+00:00",
            "title": "Business Taxonomy Construction Using Concept-Level Hierarchical Clustering",
            "authors": [
                "Haodong Bai",
                "Frank Z. Xing",
                "Erik Cambria",
                "Win-Bin Huang"
            ],
            "abstract": "Business taxonomies are indispensable tools for investors to do equity\nresearch and make professional decisions. However, to identify the structure of\nindustry sectors in an emerging market is challenging for two reasons. First,\nexisting taxonomies are designed for mature markets, which may not be the\nappropriate classification for small companies with innovative business models.\nSecond, emerging markets are fast-developing, thus the static business\ntaxonomies cannot promptly reflect the new features. In this article, we\npropose a new method to construct business taxonomies automatically from the\ncontent of corporate annual reports. Extracted concepts are hierarchically\nclustered using greedy affinity propagation. Our method requires less\nsupervision and is able to discover new terms. Experiments and evaluation on\nthe Chinese National Equities Exchange and Quotations (NEEQ) market show\nseveral advantages of the business taxonomy we build. Our results provide an\neffective tool for understanding and investing in the new growth companies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09777v3",
            "date": "2019-06-24 08:28:37+00:00",
            "title": "A Tensorized Transformer for Language Modeling",
            "authors": [
                "Xindian Ma",
                "Peng Zhang",
                "Shuai Zhang",
                "Nan Duan",
                "Yuexian Hou",
                "Dawei Song",
                "Ming Zhou"
            ],
            "abstract": "Latest development of neural models has connected the encoder and decoder\nthrough a self-attention mechanism. In particular, Transformer, which is solely\nbased on self-attention, has led to breakthroughs in Natural Language\nProcessing (NLP) tasks. However, the multi-head attention mechanism, as a key\ncomponent of Transformer, limits the effective deployment of the model to a\nresource-limited setting. In this paper, based on the ideas of tensor\ndecomposition and parameters sharing, we propose a novel self-attention model\n(namely Multi-linear attention) with Block-Term Tensor Decomposition (BTD). We\ntest and verify the proposed attention method on three language modeling tasks\n(i.e., PTB, WikiText-103 and One-billion) and a neural machine translation task\n(i.e., WMT-2016 English-German). Multi-linear attention can not only largely\ncompress the model parameters but also obtain performance improvements,\ncompared with a number of language modeling approaches, such as Transformer,\nTransformer-XL, and Transformer with tensor train decomposition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09833v1",
            "date": "2019-06-24 10:14:42+00:00",
            "title": "Translationese in Machine Translation Evaluation",
            "authors": [
                "Yvette Graham",
                "Barry Haddow",
                "Philipp Koehn"
            ],
            "abstract": "The term translationese has been used to describe the presence of unusual\nfeatures of translated text. In this paper, we provide a detailed analysis of\nthe adverse effects of translationese on machine translation evaluation\nresults. Our analysis shows evidence to support differences in text originally\nwritten in a given language relative to translated text and this can\npotentially negatively impact the accuracy of machine translation evaluations.\nFor this reason we recommend that reverse-created test data be omitted from\nfuture machine translation test sets. In addition, we provide a re-evaluation\nof a past high-profile machine translation evaluation claiming human-parity of\nMT, as well as analysis of the since re-evaluations of it. We find potential\nways of improving the reliability of all three past evaluations. One important\nissue not previously considered is the statistical power of significance tests\napplied in past evaluations that aim to investigate human-parity of MT. Since\nthe very aim of such evaluations is to reveal legitimate ties between human and\nMT systems, power analysis is of particular importance, where low power could\nresult in claims of human parity that in fact simply correspond to Type II\nerror. We therefore provide a detailed power analysis of tests used in such\nevaluations to provide an indication of a suitable minimum sample size of\ntranslations for such studies. Subsequently, since no past evaluation that\naimed to investigate claims of human parity ticks all boxes in terms of\naccuracy and reliability, we rerun the evaluation of the systems claiming human\nparity. Finally, we provide a comprehensive check-list for future machine\ntranslation evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09992v1",
            "date": "2019-06-24 14:29:12+00:00",
            "title": "Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming",
            "authors": [
                "Caio Corro",
                "Ivan Titov"
            ],
            "abstract": "We treat projective dependency trees as latent variables in our probabilistic\nmodel and induce them in such a way as to be beneficial for a downstream task,\nwithout relying on any direct tree supervision. Our approach relies on Gumbel\nperturbations and differentiable dynamic programming. Unlike previous\napproaches to latent tree learning, we stochastically sample global structures\nand our parser is fully differentiable. We illustrate its effectiveness on\nsentiment analysis and natural language inference tasks. We also study its\nproperties on a synthetic structure induction task. Ablation studies emphasize\nthe importance of both stochasticity and constraining latent structures to be\nprojective trees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10002v1",
            "date": "2019-06-24 14:49:05+00:00",
            "title": "LIAAD at SemDeep-5 Challenge: Word-in-Context (WiC)",
            "authors": [
                "Daniel Loureiro",
                "Alipio Jorge"
            ],
            "abstract": "This paper describes the LIAAD system that was ranked second place in the\nWord-in-Context challenge (WiC) featured in SemDeep-5. Our solution is based on\na novel system for Word Sense Disambiguation (WSD) using contextual embeddings\nand full-inventory sense embeddings. We adapt this WSD system, in a\nstraightforward manner, for the present task of detecting whether the same\nsense occurs in a pair of sentences. Additionally, we show that our solution is\nable to achieve competitive performance even without using the provided\ntraining or development sets, mitigating potential concerns related to task\noverfitting",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10068v1",
            "date": "2019-06-24 16:40:47+00:00",
            "title": "Is It Worth the Attention? A Comparative Evaluation of Attention Layers for Argument Unit Segmentation",
            "authors": [
                "Maximilian Splieth\u00f6ver",
                "Jonas Klaff",
                "Hendrik Heuer"
            ],
            "abstract": "Attention mechanisms have seen some success for natural language processing\ndownstream tasks in recent years and generated new State-of-the-Art results. A\nthorough evaluation of the attention mechanism for the task of Argumentation\nMining is missing, though. With this paper, we report a comparative evaluation\nof attention layers in combination with a bidirectional long short-term memory\nnetwork, which is the current state-of-the-art approach to the unit\nsegmentation task. We also compare sentence-level contextualized word\nembeddings to pre-generated ones. Our findings suggest that for this task the\nadditional attention layer does not improve upon a less complex approach. In\nmost cases, the contextualized embeddings do also not show an improvement on\nthe baseline score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10197v3",
            "date": "2019-06-24 19:47:05+00:00",
            "title": "Mutual exclusivity as a challenge for deep neural networks",
            "authors": [
                "Kanishk Gandhi",
                "Brenden M. Lake"
            ],
            "abstract": "Strong inductive biases allow children to learn in fast and adaptable ways.\nChildren use the mutual exclusivity (ME) bias to help disambiguate how words\nmap to referents, assuming that if an object has one label then it does not\nneed another. In this paper, we investigate whether or not standard neural\narchitectures have an ME bias, demonstrating that they lack this learning\nassumption. Moreover, we show that their inductive biases are poorly matched to\nlifelong learning formulations of classification and translation. We\ndemonstrate that there is a compelling case for designing neural networks that\nreason by mutual exclusivity, which remains an open challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10225v9",
            "date": "2019-06-24 20:45:50+00:00",
            "title": "Compound Probabilistic Context-Free Grammars for Grammar Induction",
            "authors": [
                "Yoon Kim",
                "Chris Dyer",
                "Alexander M. Rush"
            ],
            "abstract": "We study a formalization of the grammar induction problem that models\nsentences as being generated by a compound probabilistic context-free grammar.\nIn contrast to traditional formulations which learn a single stochastic\ngrammar, our grammar's rule probabilities are modulated by a per-sentence\ncontinuous latent variable, which induces marginal dependencies beyond the\ntraditional context-free assumptions. Inference in this grammar is performed by\ncollapsed variational inference, in which an amortized variational posterior is\nplaced on the continuous variable, and the latent trees are marginalized out\nwith dynamic programming. Experiments on English and Chinese show the\neffectiveness of our approach compared to recent state-of-the-art methods when\nevaluated on unsupervised parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10793v2",
            "date": "2019-06-26 00:31:46+00:00",
            "title": "Reconfigurable Interaction for MAS Modelling",
            "authors": [
                "Yehia Abd Alrahman",
                "Giuseppe Perelli",
                "Nir Piterman"
            ],
            "abstract": "We propose a formalism to model and reason about multi-agent systems. We\nallow agents to interact and communicate in different modes so that they can\npursue joint tasks; agents may dynamically synchronize, exchange data, adapt\ntheir behaviour, and reconfigure their communication interfaces. The formalism\ndefines a local behaviour based on shared variables and a global one based on\nmessage passing. We extend LTL to be able to reason explicitly about the\nintentions of the different agents and their interaction protocols. We also\nstudy the complexity of satisfiability and model-checking of this extension.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1906.11126v2",
            "date": "2019-06-26 14:33:18+00:00",
            "title": "On the Coherence of Fake News Articles",
            "authors": [
                "Iknoor Singh",
                "Deepak P",
                "Anoop K"
            ],
            "abstract": "The generation and spread of fake news within new and online media sources is\nemerging as a phenomenon of high societal significance. Combating them using\ndata-driven analytics has been attracting much recent scholarly interest. In\nthis study, we analyze the textual coherence of fake news articles vis-a-vis\nlegitimate ones. We develop three computational formulations of textual\ncoherence drawing upon the state-of-the-art methods in natural language\nprocessing and data science. Two real-world datasets from widely different\ndomains which have fake/legitimate article labellings are then analyzed with\nrespect to textual coherence. We observe apparent differences in textual\ncoherence across fake and legitimate news articles, with fake news articles\nconsistently scoring lower on coherence as compared to legitimate news ones.\nWhile the relative coherence shortfall of fake news articles as compared to\nlegitimate ones form the main observation from our study, we analyze several\naspects of the differences and outline potential avenues of further inquiry.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1906.11180v1",
            "date": "2019-06-26 15:53:59+00:00",
            "title": "Canonicalizing Knowledge Base Literals",
            "authors": [
                "Jiaoyan Chen",
                "Ernesto Jimenez-Ruiz",
                "Ian Horrocks"
            ],
            "abstract": "Ontology-based knowledge bases (KBs) like DBpedia are very valuable\nresources, but their usefulness and usability is limited by various quality\nissues. One such issue is the use of string literals instead of semantically\ntyped entities. In this paper we study the automated canonicalization of such\nliterals, i.e., replacing the literal with an existing entity from the KB or\nwith a new entity that is typed using classes from the KB. We propose a\nframework that combines both reasoning and machine learning in order to predict\nthe relevant entities and types, and we evaluate this framework against\nstate-of-the-art baselines for both semantic typing and entity matching.",
            "journal ref": "International Semantic Web Conference (ISWC) 2019",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.11298v1",
            "date": "2019-06-26 19:03:17+00:00",
            "title": "A Generative Model for Punctuation in Dependency Trees",
            "authors": [
                "Xiang Lisa Li",
                "Dingquan Wang",
                "Jason Eisner"
            ],
            "abstract": "Treebanks traditionally treat punctuation marks as ordinary words, but\nlinguists have suggested that a tree's \"true\" punctuation marks are not\nobserved (Nunberg, 1990). These latent \"underlying\" marks serve to delimit or\nseparate constituents in the syntax tree. When the tree's yield is rendered as\na written sentence, a string rewriting mechanism transduces the underlying\nmarks into \"surface\" marks, which are part of the observed (surface) string but\nshould not be regarded as part of the tree. We formalize this idea in a\ngenerative model of punctuation that admits efficient dynamic programming. We\ntrain it without observing the underlying marks, by locally maximizing the\nincomplete data likelihood (similarly to EM). When we use the trained model to\nreconstruct the tree's underlying punctuation, the results appear plausible\nacross 5 languages, and in particular, are consistent with Nunberg's analysis\nof English. We show that our generative model can be used to beat baselines on\npunctuation restoration. Also, our reconstruction of a sentence's underlying\npunctuation lets us appropriately render the surface punctuation (via our\ntrained underlying-to-surface mechanism) when we syntactically transform the\nsentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11301v1",
            "date": "2019-06-26 19:15:41+00:00",
            "title": "Exploring the Role of Prior Beliefs for Argument Persuasion",
            "authors": [
                "Esin Durmus",
                "Claire Cardie"
            ],
            "abstract": "Public debate forums provide a common platform for exchanging opinions on a\ntopic of interest. While recent studies in natural language processing (NLP)\nhave provided empirical evidence that the language of the debaters and their\npatterns of interaction play a key role in changing the mind of a reader,\nresearch in psychology has shown that prior beliefs can affect our\ninterpretation of an argument and could therefore constitute a competing\nalternative explanation for resistance to changing one's stance. To study the\nactual effect of language use vs. prior beliefs on persuasion, we provide a new\ndataset and propose a controlled setting that takes into consideration two\nreader level factors: political and religious ideology. We find that prior\nbeliefs affected by these reader level factors play a more important role than\nlanguage use effects and argue that it is important to account for them in NLP\nstudies of persuasion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11310v1",
            "date": "2019-06-26 19:38:02+00:00",
            "title": "A Corpus for Modeling User and Language Effects in Argumentation on Online Debating",
            "authors": [
                "Esin Durmus",
                "Claire Cardie"
            ],
            "abstract": "Existing argumentation datasets have succeeded in allowing researchers to\ndevelop computational methods for analyzing the content, structure and\nlinguistic features of argumentative text. They have been much less successful\nin fostering studies of the effect of \"user\" traits -- characteristics and\nbeliefs of the participants -- on the debate/argument outcome as this type of\nuser information is generally not available. This paper presents a dataset of\n78, 376 debates generated over a 10-year period along with surprisingly\ncomprehensive participant profiles. We also complete an example study using the\ndataset to analyze the effect of selected user traits on the debate outcome in\ncomparison to the linguistic features typically employed in studies of this\nkind.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11313v1",
            "date": "2019-06-26 19:45:42+00:00",
            "title": "Determining Relative Argument Specificity and Stance for Complex Argumentative Structures",
            "authors": [
                "Esin Durmus",
                "Faisal Ladhak",
                "Claire Cardie"
            ],
            "abstract": "Systems for automatic argument generation and debate require the ability to\n(1) determine the stance of any claims employed in the argument and (2) assess\nthe specificity of each claim relative to the argument context. Existing work\non understanding claim specificity and stance, however, has been limited to the\nstudy of argumentative structures that are relatively shallow, most often\nconsisting of a single claim that directly supports or opposes the argument\nthesis. In this paper, we tackle these tasks in the context of complex\narguments on a diverse set of topics. In particular, our dataset consists of\nmanually curated argument trees for 741 controversial topics covering 95,312\nunique claims; lines of argument are generally of depth 2 to 6. We find that as\nthe distance between a pair of claims increases along the argument path,\ndetermining the relative specificity of a pair of claims becomes easier and\ndetermining their relative stance becomes harder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11315v2",
            "date": "2019-06-26 19:48:25+00:00",
            "title": "Generalization to Novel Objects using Prior Relational Knowledge",
            "authors": [
                "Varun Kumar Vijay",
                "Abhinav Ganesh",
                "Hanlin Tang",
                "Arjun Bansal"
            ],
            "abstract": "To solve tasks in new environments involving objects unseen during training,\nagents must reason over prior information about those objects and their\nrelations. We introduce the Prior Knowledge Graph network, an architecture for\ncombining prior information, structured as a knowledge graph, with a symbolic\nparsing of the visual scene, and demonstrate that this approach is able to\napply learned relations to novel objects whereas the baseline algorithms fail.\nAblation experiments show that the agents ground the knowledge graph relations\nto semantically-relevant behaviors. In both a Sokoban game and the more complex\nPacman environment, our network is also more sample efficient than the\nbaselines, reaching the same performance in 5-10x fewer episodes. Once the\nagents are trained with our approach, we can manipulate agent behavior by\nmodifying the knowledge graph in semantically meaningful ways. These results\nsuggest that our network provides a framework for agents to reason over\nstructured knowledge graphs while still leveraging gradient based learning\napproaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.11752v1",
            "date": "2019-06-27 15:54:57+00:00",
            "title": "Semantic expressive capacity with bounded memory",
            "authors": [
                "Antoine Venant",
                "Alexander Koller"
            ],
            "abstract": "We investigate the capacity of mechanisms for compositional semantic parsing\nto describe relations between sentences and semantic representations.\n  We prove that in order to represent certain relations, mechanisms which are\nsyntactically projective must be able to remember an unbounded number of\nlocations in the semantic representations, where nonprojective mechanisms need\nnot.\n  This is the first result of this kind, and has consequences both for\ngrammar-based and for neural systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11894v2",
            "date": "2019-06-11 11:06:43+00:00",
            "title": "Labeling, Cutting, Grouping: an Efficient Text Line Segmentation Method for Medieval Manuscripts",
            "authors": [
                "Michele Alberti",
                "Lars V\u00f6gtlin",
                "Vinaychandran Pondenkandath",
                "Mathias Seuret",
                "Rolf Ingold",
                "Marcus Liwicki"
            ],
            "abstract": "This paper introduces a new way for text-line extraction by integrating\ndeep-learning based pre-classification and state-of-the-art segmentation\nmethods. Text-line extraction in complex handwritten documents poses a\nsignificant challenge, even to the most modern computer vision algorithms.\nHistorical manuscripts are a particularly hard class of documents as they\npresent several forms of noise, such as degradation, bleed-through, interlinear\nglosses, and elaborated scripts. In this work, we propose a novel method which\nuses semantic segmentation at pixel level as intermediate task, followed by a\ntext-line extraction step. We measured the performance of our method on a\nrecent dataset of challenging medieval manuscripts and surpassed\nstate-of-the-art results by reducing the error by 80.7%. Furthermore, we\ndemonstrate the effectiveness of our approach on various other datasets written\nin different scripts. Hence, our contribution is two-fold. First, we\ndemonstrate that semantic pixel segmentation can be used as strong denoising\npre-processing step before performing text line extraction. Second, we\nintroduce a novel, simple and robust algorithm that leverages the high-quality\nsemantic segmentation to achieve a text-line extraction performance of 99.42%\nline IU on a challenging dataset.",
            "journal ref": "2019 15th IAPR International Conference on Document Analysis and\n  Recognition (ICDAR), Sydney, Australia",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.12035v2",
            "date": "2019-06-28 04:08:15+00:00",
            "title": "A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder",
            "authors": [
                "Xipeng Qiu",
                "Hengzhi Pei",
                "Hang Yan",
                "Xuanjing Huang"
            ],
            "abstract": "Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the\nrelations among the multiple heterogeneous segmentation criteria and further\nimprove the performance of each single criterion. Previous work usually regards\nMCCWS as different tasks, which are learned together under the multi-task\nlearning framework. In this paper, we propose a concise but effective unified\nmodel for MCCWS, which is fully-shared for all the criteria. By leveraging the\npowerful ability of the Transformer encoder, the proposed unified model can\nsegment Chinese text according to a unique criterion-token indicating the\noutput criterion. Besides, the proposed unified model can segment both\nsimplified and traditional Chinese and has an excellent transfer capability.\nExperiments on eight datasets with different criteria show that our model\noutperforms our single-criterion baseline model and other multi-criteria\nmodels. Source codes of this paper are available on Github\nhttps://github.com/acphile/MCCWS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.12039v1",
            "date": "2019-06-28 04:41:06+00:00",
            "title": "Supervised Contextual Embeddings for Transfer Learning in Natural Language Processing Tasks",
            "authors": [
                "Mihir Kale",
                "Aditya Siddhant",
                "Sreyashi Nag",
                "Radhika Parik",
                "Matthias Grabmair",
                "Anthony Tomasic"
            ],
            "abstract": "Pre-trained word embeddings are the primary method for transfer learning in\nseveral Natural Language Processing (NLP) tasks. Recent works have focused on\nusing unsupervised techniques such as language modeling to obtain these\nembeddings. In contrast, this work focuses on extracting representations from\nmultiple pre-trained supervised models, which enriches word embeddings with\ntask and domain specific knowledge. Experiments performed in cross-task,\ncross-domain and cross-lingual settings indicate that such supervised\nembeddings are helpful, especially in the low-resource setting, but the extent\nof gains is dependent on the nature of the task and domain. We make our code\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.12068v1",
            "date": "2019-06-28 07:31:33+00:00",
            "title": "Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation",
            "authors": [
                "Eva Vanmassenhove",
                "Dimitar Shterionov",
                "Andy Way"
            ],
            "abstract": "This work presents an empirical approach to quantifying the loss of lexical\nrichness in Machine Translation (MT) systems compared to Human Translation\n(HT). Our experiments show how current MT systems indeed fail to render the\nlexical diversity of human generated or translated text. The inability of MT\nsystems to generate diverse outputs and its tendency to exacerbate already\nfrequent patterns while ignoring less frequent ones, might be the underlying\ncause for, among others, the currently heavily debated issues related to gender\nbiased output. Can we indeed, aside from biased data, talk about an algorithm\nthat exacerbates seen biases?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.12284v1",
            "date": "2019-06-28 16:14:06+00:00",
            "title": "Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts",
            "authors": [
                "Denis Emelin",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "The transformer is a state-of-the-art neural translation model that uses\nattention to iteratively refine lexical representations with information drawn\nfrom the surrounding context. Lexical features are fed into the first layer and\npropagated through a deep network of hidden layers. We argue that the need to\nrepresent and propagate lexical features in each layer limits the model's\ncapacity for learning and representing other information relevant to the task.\nTo alleviate this bottleneck, we introduce gated shortcut connections between\nthe embedding layer and each subsequent layer within the encoder and decoder.\nThis enables the model to access relevant lexical content dynamically, without\nexpending limited resources on storing it within intermediate states. We show\nthat the proposed modification yields consistent improvements over a baseline\ntransformer on standard WMT translation tasks in 5 translation directions (0.9\nBLEU on average) and reduces the amount of lexical information passed along the\nhidden layers. We furthermore evaluate different ways to integrate lexical\nconnections into the transformer architecture and present ablation experiments\nexploring the effect of proposed shortcuts on model behavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00151v5",
            "date": "2019-06-29 06:04:48+00:00",
            "title": "GPT-based Generation for Classical Chinese Poetry",
            "authors": [
                "Yi Liao",
                "Yasheng Wang",
                "Qun Liu",
                "Xin Jiang"
            ],
            "abstract": "We present a simple yet effective method for generating high quality\nclassical Chinese poetry with Generative Pre-trained Language Model (GPT). The\nmethod adopts a simple GPT model, without using any human crafted rules or\nfeatures, or designing any additional neural components. While the proposed\nmodel learns to generate various forms of classical Chinese poems, including\nJueju, L\\\"{u}shi, various Cipai and Couples, the generated poems are of very\nhigh quality. We also propose and implement a method to fine-tune the model to\ngenerate acrostic poetry. To the best of our knowledge, this is the first to\nemploy GPT in developing a poetry generation system. We have released an online\nmini demonstration program on Wechat to show the generation capability of the\nproposed method for classical Chinese poetry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00321v1",
            "date": "2019-06-30 05:29:38+00:00",
            "title": "Mechanisms of Artistic Creativity in Deep Learning Neural Networks",
            "authors": [
                "Lonce Wyse"
            ],
            "abstract": "The generative capabilities of deep learning neural networks (DNNs) have been\nattracting increasing attention for both the remarkable artifacts they produce,\nbut also because of the vast conceptual difference between how they are\nprogrammed and what they do. DNNs are 'black boxes' where high-level behavior\nis not explicitly programmed, but emerges from the complex interactions of\nthousands or millions of simple computational elements. Their behavior is often\ndescribed in anthropomorphic terms that can be misleading, seem magical, or\nstoke fears of an imminent singularity in which machines become 'more' than\nhuman. In this paper, we examine 5 distinct behavioral characteristics\nassociated with creativity, and provide an example of a mechanisms from\ngenerative deep learning architectures that give rise to each these\ncharacteristics. All 5 emerge from machinery built for purposes other than the\ncreative characteristics they exhibit, mostly classification. These mechanisms\nof creative generative capabilities thus demonstrate a deep kinship to\ncomputational perceptual processes. By understanding how these different\nbehaviors arise, we hope to on one hand take the magic out of anthropomorphic\ndescriptions, but on the other, to build a deeper appreciation of machinic\nforms of creativity on their own terms that will allow us to nurture their\nfurther development.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.00462v1",
            "date": "2019-06-30 21:09:48+00:00",
            "title": "Inter and Intra Document Attention for Depression Risk Assessment",
            "authors": [
                "Diego Maupom\u00e9",
                "Marc Queudot",
                "Marie-Jean Meurs"
            ],
            "abstract": "We take interest in the early assessment of risk for depression in social\nmedia users. We focus on the eRisk 2018 dataset, which represents users as a\nsequence of their written online contributions. We implement four RNN-based\nsystems to classify the users. We explore several aggregations methods to\ncombine predictions on individual posts. Our best model reads through all\nwritings of a user in parallel but uses an attention mechanism to prioritize\nthe most important ones at each timestep.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00464v1",
            "date": "2019-06-30 21:12:14+00:00",
            "title": "Merge and Label: A novel neural network architecture for nested NER",
            "authors": [
                "Joseph Fisher",
                "Andreas Vlachos"
            ],
            "abstract": "Named entity recognition (NER) is one of the best studied tasks in natural\nlanguage processing. However, most approaches are not capable of handling\nnested structures which are common in many applications. In this paper we\nintroduce a novel neural network architecture that first merges tokens and/or\nentities into entities forming nested structures, and then labels each of them\nindependently. Unlike previous work, our merge and label approach predicts\nreal-valued instead of discrete segmentation structures, which allow it to\ncombine word and nested entity embeddings while maintaining differentiability.\n%which smoothly groups entities into single vectors across multiple levels. We\nevaluate our approach using the ACE 2005 Corpus, where it achieves\nstate-of-the-art F1 of 74.6, further improved with contextual embeddings (BERT)\nto 82.4, an overall improvement of close to 8 F1 points over previous\napproaches trained on the same data. Additionally we compare it against\nBiLSTM-CRFs, the dominant approach for flat NER structures, demonstrating that\nits ability to predict nested structures does not impact performance in simpler\ncases.",
            "journal ref": "ACL 2019 P19-1585",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00494v1",
            "date": "2019-06-30 22:55:27+00:00",
            "title": "The University of Sydney's Machine Translation System for WMT19",
            "authors": [
                "Liang Ding",
                "Dacheng Tao"
            ],
            "abstract": "This paper describes the University of Sydney's submission of the WMT 2019\nshared news translation task. We participated in the\nFinnish$\\rightarrow$English direction and got the best BLEU(33.0) score among\nall the participants. Our system is based on the self-attentional Transformer\nnetworks, into which we integrated the most recent effective strategies from\nacademic research (e.g., BPE, back translation, multi-features data selection,\ndata augmentation, greedy model ensemble, reranking, ConMBR system combination,\nand post-processing). Furthermore, we propose a novel augmentation method\n$Cycle Translation$ and a data mixture strategy $Big$/$Small$ parallel\nconstruction to entirely exploit the synthetic corpus. Extensive experiments\nshow that adding the above techniques can make continuous improvements of the\nBLEU scores, and the best result outperforms the baseline (Transformer ensemble\nmodel trained with the original parallel corpus) by approximately 5.3 BLEU\nscore, achieving the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00607v1",
            "date": "2019-07-01 08:44:30+00:00",
            "title": "Weak Supervision Enhanced Generative Network for Question Generation",
            "authors": [
                "Yutong Wang",
                "Jiyuan Zheng",
                "Qijiong Liu",
                "Zhou Zhao",
                "Jun Xiao",
                "Yueting Zhuang"
            ],
            "abstract": "Automatic question generation according to an answer within the given passage\nis useful for many applications, such as question answering system, dialogue\nsystem, etc. Current neural-based methods mostly take two steps which extract\nseveral important sentences based on the candidate answer through manual rules\nor supervised neural networks and then use an encoder-decoder framework to\ngenerate questions about these sentences. These approaches neglect the semantic\nrelations between the answer and the context of the whole passage which is\nsometimes necessary for answering the question. To address this problem, we\npropose the Weak Supervision Enhanced Generative Network (WeGen) which\nautomatically discovers relevant features of the passage given the answer span\nin a weakly supervised manner to improve the quality of generated questions.\nMore specifically, we devise a discriminator, Relation Guider, to capture the\nrelations between the whole passage and the associated answer and then the\nMulti-Interaction mechanism is deployed to transfer the knowledge dynamically\nfor our question generation system. Experiments show the effectiveness of our\nmethod in both automatic evaluations and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00692v1",
            "date": "2019-06-24 16:24:46+00:00",
            "title": "Event extraction based on open information extraction and ontology",
            "authors": [
                "Sihem Sahnoun"
            ],
            "abstract": "The work presented in this master thesis consists of extracting a set of\nevents from texts written in natural language. For this purpose, we have based\nourselves on the basic notions of the information extraction as well as the\nopen information extraction. First, we applied an open information\nextraction(OIE) system for the relationship extraction, to highlight the\nimportance of OIEs in event extraction, and we used the ontology to the event\nmodeling. We tested the results of our approach with test metrics. As a result,\nthe two-level event extraction approach has shown good performance results but\nrequires a lot of expert intervention in the construction of classifiers and\nthis will take time. In this context we have proposed an approach that reduces\nthe expert intervention in the relation extraction, the recognition of entities\nand the reasoning which are automatic and based on techniques of adaptation and\ncorrespondence. Finally, to prove the relevance of the extracted results, we\nconducted a set of experiments using different test metrics as well as a\ncomparative study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00720v1",
            "date": "2019-06-26 14:50:39+00:00",
            "title": "Constructing Information-Lossless Biological Knowledge Graphs from Conditional Statements",
            "authors": [
                "Tianwen Jiang",
                "Tong Zhao",
                "Bing Qin",
                "Ting Liu",
                "Nitesh V. Chawla",
                "Meng Jiang"
            ],
            "abstract": "Conditions are essential in the statements of biological literature. Without\nthe conditions (e.g., environment, equipment) that were precisely specified,\nthe facts (e.g., observations) in the statements may no longer be valid. One\nbiological statement has one or multiple fact(s) and/or condition(s). Their\nsubject and object can be either a concept or a concept's attribute. Existing\ninformation extraction methods do not consider the role of condition in the\nbiological statement nor the role of attribute in the subject/object. In this\nwork, we design a new tag schema and propose a deep sequence tagging framework\nto structure conditional statement into fact and condition tuples from\nbiological text. Experiments demonstrate that our method yields a\ninformation-lossless structure of the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00852v2",
            "date": "2019-07-01 15:20:01+00:00",
            "title": "EGG: a toolkit for research on Emergence of lanGuage in Games",
            "authors": [
                "Eugene Kharitonov",
                "Rahma Chaabouni",
                "Diane Bouchacourt",
                "Marco Baroni"
            ],
            "abstract": "There is renewed interest in simulating language emergence among deep neural\nagents that communicate to jointly solve a task, spurred by the practical aim\nto develop language-enabled interactive AIs, as well as by theoretical\nquestions about the evolution of human language. However, optimizing deep\narchitectures connected by a discrete communication channel (such as that in\nwhich language emerges) is technically challenging. We introduce EGG, a toolkit\nthat greatly simplifies the implementation of emergent-language communication\ngames. EGG's modular design provides a set of building blocks that the user can\ncombine to create new games, easily navigating the optimization and\narchitecture space. We hope that the tool will lower the technical barrier, and\nencourage researchers from various backgrounds to do original work in this\nexciting area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00883v1",
            "date": "2019-07-01 15:55:36+00:00",
            "title": "HyST: A Hybrid Approach for Flexible and Accurate Dialogue State Tracking",
            "authors": [
                "Rahul Goel",
                "Shachi Paul",
                "Dilek Hakkani-T\u00fcr"
            ],
            "abstract": "Recent works on end-to-end trainable neural network based approaches have\ndemonstrated state-of-the-art results on dialogue state tracking. The best\nperforming approaches estimate a probability distribution over all possible\nslot values. However, these approaches do not scale for large value sets\ncommonly present in real-life applications and are not ideal for tracking slot\nvalues that were not observed in the training set. To tackle these issues,\ncandidate-generation-based approaches have been proposed. These approaches\nestimate a set of values that are possible at each turn based on the\nconversation history and/or language understanding outputs, and hence enable\nstate tracking over unseen values and large value sets however, they fall short\nin terms of performance in comparison to the first group. In this work, we\nanalyze the performance of these two alternative dialogue state tracking\nmethods, and present a hybrid approach (HyST) which learns the appropriate\nmethod for each slot type. To demonstrate the effectiveness of HyST on a\nrich-set of slot types, we experiment with the recently released MultiWOZ-2.0\nmulti-domain, task-oriented dialogue-dataset. Our experiments show that HyST\nscales to multi-domain applications. Our best performing model results in a\nrelative improvement of 24% and 10% over the previous SOTA and our best\nbaseline respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00937v1",
            "date": "2019-07-01 17:20:02+00:00",
            "title": "Semantic Product Search",
            "authors": [
                "Priyanka Nigam",
                "Yiwei Song",
                "Vijai Mohan",
                "Vihan Lakshman",
                "Weitian",
                "Ding",
                "Ankit Shingavi",
                "Choon Hui Teo",
                "Hao Gu",
                "Bing Yin"
            ],
            "abstract": "We study the problem of semantic matching in product search, that is, given a\ncustomer query, retrieve all semantically related products from the catalog.\nPure lexical matching via an inverted index falls short in this respect due to\nseveral factors: a) lack of understanding of hypernyms, synonyms, and antonyms,\nb) fragility to morphological variants (e.g. \"woman\" vs. \"women\"), and c)\nsensitivity to spelling errors. To address these issues, we train a deep\nlearning model for semantic matching using customer behavior data. Much of the\nrecent work on large-scale semantic search using deep learning focuses on\nranking for web search. In contrast, semantic matching for product search\npresents several novel challenges, which we elucidate in this paper. We address\nthese challenges by a) developing a new loss function that has an inbuilt\nthreshold to differentiate between random negative examples, impressed but not\npurchased examples, and positive examples (purchased items), b) using average\npooling in conjunction with n-grams to capture short-range linguistic patterns,\nc) using hashing to handle out of vocabulary tokens, and d) using a model\nparallel training architecture to scale across 8 GPUs. We present compelling\noffline results that demonstrate at least 4.7% improvement in Recall@100 and\n14.5% improvement in mean average precision (MAP) over baseline\nstate-of-the-art semantic search methods using the same tokenization method.\nMoreover, we present results and discuss learnings from online A/B tests which\ndemonstrate the efficacy of our method.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01041v1",
            "date": "2019-07-01 19:48:34+00:00",
            "title": "Natural Language Understanding with the Quora Question Pairs Dataset",
            "authors": [
                "Lakshay Sharma",
                "Laura Graesser",
                "Nikita Nangia",
                "Utku Evci"
            ],
            "abstract": "This paper explores the task Natural Language Understanding (NLU) by looking\nat duplicate question detection in the Quora dataset. We conducted extensive\nexploration of the dataset and used various machine learning models, including\nlinear and tree-based models. Our final finding was that a simple Continuous\nBag of Words neural network model had the best performance, outdoing more\ncomplicated recurrent and attention based models. We also conducted error\nanalysis and found some subjectivity in the labeling of the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01055v2",
            "date": "2019-07-01 20:17:59+00:00",
            "title": "Is artificial data useful for biomedical Natural Language Processing algorithms?",
            "authors": [
                "Zixu Wang",
                "Julia Ive",
                "Sumithra Velupillai",
                "Lucia Specia"
            ],
            "abstract": "A major obstacle to the development of Natural Language Processing (NLP)\nmethods in the biomedical domain is data accessibility. This problem can be\naddressed by generating medical data artificially. Most previous studies have\nfocused on the generation of short clinical text, and evaluation of the data\nutility has been limited. We propose a generic methodology to guide the\ngeneration of clinical text with key phrases. We use the artificial data as\nadditional training data in two key biomedical NLP tasks: text classification\nand temporal relation extraction. We show that artificially generated training\ndata used in conjunction with real training data can lead to performance boosts\nfor data-greedy neural network algorithms. We also demonstrate the usefulness\nof the generated data for NLP setups where it fully replaces real training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01108v2",
            "date": "2019-07-02 00:38:44+00:00",
            "title": "Language2Pose: Natural Language Grounded Pose Forecasting",
            "authors": [
                "Chaitanya Ahuja",
                "Louis-Philippe Morency"
            ],
            "abstract": "Generating animations from natural language sentences finds its applications\nin a a number of domains such as movie script visualization, virtual human\nanimation and, robot motion planning. These sentences can describe different\nkinds of actions, speeds and direction of these actions, and possibly a target\ndestination. The core modeling challenge in this language-to-pose application\nis how to map linguistic concepts to motion animations.\n  In this paper, we address this multimodal problem by introducing a neural\narchitecture called Joint Language to Pose (or JL2P), which learns a joint\nembedding of language and pose. This joint embedding space is learned\nend-to-end using a curriculum learning approach which emphasizes shorter and\neasier sequences first before moving to longer and harder ones. We evaluate our\nproposed model on a publicly available corpus of 3D pose data and\nhuman-annotated sentences. Both objective metrics and human judgment evaluation\nconfirm that our proposed approach is able to generate more accurate animations\nand are deemed visually more representative by humans than other data driven\napproaches.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.01115v1",
            "date": "2019-07-02 01:09:35+00:00",
            "title": "Neural Semantic Parsing with Anonymization for Command Understanding in General-Purpose Service Robots",
            "authors": [
                "Nick Walker",
                "Yu-Tang Peng",
                "Maya Cakmak"
            ],
            "abstract": "Service robots are envisioned to undertake a wide range of tasks at the\nrequest of users. Semantic parsing is one way to convert natural language\ncommands given to these robots into executable representations. Methods for\ncreating semantic parsers, however, rely either on large amounts of data or on\nengineered lexical features and parsing rules, which has limited their\napplication in robotics. To address this challenge, we propose an approach that\nleverages neural semantic parsing methods in combination with contextual word\nembeddings to enable the training of a semantic parser with little data and\nwithout domain specific parser engineering. Key to our approach is the use of\nan anonymized target representation which is more easily learned by the parser.\nIn most cases, this simplified representation can trivially be transformed into\nan executable format, and in others the parse can be completed through further\ninteraction with the user. We evaluate this approach in the context of the\nRoboCup@Home General Purpose Service Robot task, where we have collected a\ncorpus of paraphrased versions of commands from the standardized command\ngenerator. Our results show that neural semantic parsers can predict the\nlogical form of unseen commands with 89% accuracy. We release our data and the\ndetails of our models to encourage further development from the RoboCup and\nservice robotics communities.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1907.01256v1",
            "date": "2019-07-02 09:33:36+00:00",
            "title": "A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning",
            "authors": [
                "Yo Joong Choe",
                "Jiyeon Ham",
                "Kyubyong Park",
                "Yeoil Yoon"
            ],
            "abstract": "Grammatical error correction can be viewed as a low-resource\nsequence-to-sequence task, because publicly available parallel corpora are\nlimited. To tackle this challenge, we first generate erroneous versions of\nlarge unannotated corpora using a realistic noising function. The resulting\nparallel corpora are subsequently used to pre-train Transformer models. Then,\nby sequentially applying transfer learning, we adapt these models to the domain\nand style of the test set. Combined with a context-aware neural spellchecker,\nour system achieves competitive results in both restricted and low resource\ntracks in ACL 2019 BEA Shared Task. We release all of our code and materials\nfor reproducibility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01304v1",
            "date": "2019-07-02 11:44:31+00:00",
            "title": "Danish Stance Classification and Rumour Resolution",
            "authors": [
                "Anders Edelbo Lillie",
                "Emil Refsgaard Middelboe"
            ],
            "abstract": "The Internet is rife with flourishing rumours that spread through microblogs\nand social media. Recent work has shown that analysing the stance of the crowd\ntowards a rumour is a good indicator for its veracity. One state-of-the-art\nsystem uses an LSTM neural network to automatically classify stance for posts\non Twitter by considering the context of a whole branch, while another, more\nsimple Decision Tree classifier, performs at least as well by performing\ncareful feature engineering. One approach to predict the veracity of a rumour\nis to use stance as the only feature for a Hidden Markov Model (HMM). This\nthesis generates a stance-annotated Reddit dataset for the Danish language, and\nimplements various models for stance classification. Out of these, a Linear\nSupport Vector Machine provides the best results with an accuracy of 0.76 and\nmacro F1 score of 0.42. Furthermore, experiments show that stance labels can be\nused across languages and platforms with a HMM to predict the veracity of\nrumours, achieving an accuracy of 0.82 and F1 score of 0.67. Even higher scores\nare achieved by relying only on the Danish dataset. In this case veracity\nprediction scores an accuracy of 0.83 and an F1 of 0.68. Finally, when using\nautomatic stance labels for the HMM, only a small drop in performance is\nobserved, showing that the implemented system can have practical applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01339v3",
            "date": "2019-07-02 13:13:13+00:00",
            "title": "Sequence Labeling Parsing by Learning Across Representations",
            "authors": [
                "Michalina Strzyz",
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We use parsing as sequence labeling as a common framework to learn across\nconstituency and dependency syntactic abstractions. To do so, we cast the\nproblem as multitask learning (MTL). First, we show that adding a parsing\nparadigm as an auxiliary loss consistently improves the performance on the\nother paradigm. Secondly, we explore an MTL sequence labeling model that parses\nboth representations, at almost no cost in terms of performance and speed. The\nresults across the board show that on average MTL models with auxiliary losses\nfor constituency parsing outperform single-task ones by 1.14 F1 points, and for\ndependency parsing by 0.62 UAS points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01611v2",
            "date": "2019-07-02 20:00:29+00:00",
            "title": "CS563-QA: A Collection for Evaluating Question Answering Systems",
            "authors": [
                "Katerina Papantoniou",
                "Yannis Tzitzikas"
            ],
            "abstract": "Question Answering (QA) is a challenging topic since it requires tackling the\nvarious difficulties of natural language understanding. Since evaluation is\nimportant not only for identifying the strong and weak points of the various\ntechniques for QA, but also for facilitating the inception of new methods and\ntechniques, in this paper we present a collection for evaluating QA methods\nover free text that we have created. Although it is a small collection, it\ncontains cases of increasing difficulty, therefore it has an educational value\nand it can be used for rapid evaluation of QA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01638v1",
            "date": "2019-06-22 02:42:44+00:00",
            "title": "An Online Topic Modeling Framework with Topics Automatically Labeled",
            "authors": [
                "Fenglei Jin",
                "Cuiyun Gao",
                "Michael R. Lyu"
            ],
            "abstract": "In this paper, we propose a novel online topic tracking framework, named\nIEDL, for tracking the topic changes related to deep learning techniques on\nStack Exchange and automatically interpreting each identified topic. The\nproposed framework combines the prior topic distributions in a time window\nduring inferring the topics in current time slice, and introduces a new ranking\nscheme to select most representative phrases and sentences for the inferred\ntopics in each time slice. Experiments on 7,076 Stack Exchange posts show the\neffectiveness of IEDL in tracking topic changes and labeling topics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01669v4",
            "date": "2019-07-02 22:30:31+00:00",
            "title": "MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines",
            "authors": [
                "Mihail Eric",
                "Rahul Goel",
                "Shachi Paul",
                "Adarsh Kumar",
                "Abhishek Sethi",
                "Peter Ku",
                "Anuj Kumar Goyal",
                "Sanchit Agarwal",
                "Shuyang Gao",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "MultiWOZ 2.0 (Budzianowski et al., 2018) is a recently released multi-domain\ndialogue dataset spanning 7 distinct domains and containing over 10,000\ndialogues. Though immensely useful and one of the largest resources of its kind\nto-date, MultiWOZ 2.0 has a few shortcomings. Firstly, there is substantial\nnoise in the dialogue state annotations and dialogue utterances which\nnegatively impact the performance of state-tracking models. Secondly, follow-up\nwork (Lee et al., 2019) has augmented the original dataset with user dialogue\nacts. This leads to multiple co-existent versions of the same dataset with\nminor modifications. In this work we tackle the aforementioned issues by\nintroducing MultiWOZ 2.1. To fix the noisy state annotations, we use\ncrowdsourced workers to re-annotate state and utterances based on the original\nutterances in the dataset. This correction process results in changes to over\n32% of state annotations across 40% of the dialogue turns. In addition, we fix\n146 dialogue utterances by canonicalizing slot values in the utterances to the\nvalues in the dataset ontology. To address the second problem, we combined the\ncontributions of the follow-up works into MultiWOZ 2.1. Hence, our dataset also\nincludes user dialogue acts as well as multiple slot descriptions per dialogue\nstate slot. We then benchmark a number of state-of-the-art dialogue state\ntracking models on the MultiWOZ 2.1 dataset and show the joint state tracking\nperformance on the corrected state annotations. We are publicly releasing\nMultiWOZ 2.1 to the community, hoping that this dataset resource will allow for\nmore effective models across various dialogue subproblems to be built in the\nfuture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01677v1",
            "date": "2019-07-02 23:28:52+00:00",
            "title": "Scalable Multi Corpora Neural Language Models for ASR",
            "authors": [
                "Anirudh Raju",
                "Denis Filimonov",
                "Gautam Tiwari",
                "Guitang Lan",
                "Ariya Rastrow"
            ],
            "abstract": "Neural language models (NLM) have been shown to outperform conventional\nn-gram language models by a substantial margin in Automatic Speech Recognition\n(ASR) and other tasks. There are, however, a number of challenges that need to\nbe addressed for an NLM to be used in a practical large-scale ASR system. In\nthis paper, we present solutions to some of the challenges, including training\nNLM from heterogenous corpora, limiting latency impact and handling\npersonalized bias in the second-pass rescorer. Overall, we show that we can\nachieve a 6.2% relative WER reduction using neural LM in a second-pass n-best\nrescoring framework with a minimal increase in latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02202v1",
            "date": "2019-07-04 03:38:41+00:00",
            "title": "SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering",
            "authors": [
                "Zhenpeng Chen",
                "Yanbin Cao",
                "Xuan Lu",
                "Qiaozhu Mei",
                "Xuanzhe Liu"
            ],
            "abstract": "Sentiment analysis has various application scenarios in software engineering\n(SE), such as detecting developers' emotions in commit messages and identifying\ntheir opinions on Q&A forums. However, commonly used out-of-the-box sentiment\nanalysis tools cannot obtain reliable results on SE tasks and the\nmisunderstanding of technical jargon is demonstrated to be the main reason.\nThen, researchers have to utilize labeled SE-related texts to customize\nsentiment analysis for SE tasks via a variety of algorithms. However, the\nscarce labeled data can cover only very limited expressions and thus cannot\nguarantee the analysis quality. To address such a problem, we turn to the\neasily available emoji usage data for help. More specifically, we employ\nemotional emojis as noisy labels of sentiments and propose a representation\nlearning approach that uses both Tweets and GitHub posts containing emojis to\nlearn sentiment-aware representations for SE-related texts. These emoji-labeled\nposts can not only supply the technical jargon, but also incorporate more\ngeneral sentiment patterns shared across domains. They as well as labeled data\nare used to learn the final sentiment classifier. Compared to the existing\nsentiment analysis methods used in SE, the proposed approach can achieve\nsignificant improvement on representative benchmark datasets. By further\ncontrast experiments, we find that the Tweets make a key contribution to the\npower of our approach. This finding informs future research not to unilaterally\npursue the domain-specific resource, but try to transform knowledge from the\nopen domain through ubiquitous signals such as emojis.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1907.02205v1",
            "date": "2019-07-04 03:50:21+00:00",
            "title": "An External Knowledge Enhanced Multi-label Charge Prediction Approach with Label Number Learning",
            "authors": [
                "Duan Wei",
                "Li Lin"
            ],
            "abstract": "Multi-label charge prediction is a task to predict the corresponding\naccusations for legal cases, and recently becomes a hot topic. However, current\nstudies use rough methods to deal with the label number. These methods manually\nset parameters to select label numbers, which has an effect in final prediction\nquality. We propose an external knowledge enhanced multi-label charge\nprediction approach that has two phases. One is charge label prediction phase\nwith external knowledge from law provisions, the other one is number learning\nphase with a number learning network (NLN) designed. Our approach enhanced by\nexternal knowledge can automatically adjust the threshold to get label number\nof law cases. It combines the output probabilities of samples and their\ncorresponding label numbers to get final prediction results. In experiments,\nour approach is connected to some state of-the art deep learning models. By\ntesting on the biggest published Chinese law dataset, we find that our approach\nhas improvements on these models. We future conduct experiments on multi-label\nsamples from the dataset. In items of macro-F1, the improvement of baselines\nwith our approach is 3%-5%; In items of micro-F1, the significant improvement\nof our approach is 5%-15%. The experiment results show the effectiveness our\napproach for multi-label charge prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02427v2",
            "date": "2019-07-04 14:40:22+00:00",
            "title": "Multi-Task Learning for Coherence Modeling",
            "authors": [
                "Youmna Farag",
                "Helen Yannakoudakis"
            ],
            "abstract": "We address the task of assessing discourse coherence, an aspect of text\nquality that is essential for many NLP tasks, such as summarization and\nlanguage assessment. We propose a hierarchical neural network trained in a\nmulti-task fashion that learns to predict a document-level coherence score (at\nthe network's top layers) along with word-level grammatical roles (at the\nbottom layers), taking advantage of inductive transfer between the two tasks.\nWe assess the extent to which our framework generalizes to different domains\nand prediction tasks, and demonstrate its effectiveness not only on standard\nbinary evaluation coherence tasks, but also on real-world tasks involving the\nprediction of varying degrees of coherence, achieving a new state of the art.",
            "journal ref": "THE 57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL\n  LINGUISTICS (ACL 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02479v1",
            "date": "2019-07-04 16:20:42+00:00",
            "title": "Fine-grained robust prosody transfer for single-speaker neural text-to-speech",
            "authors": [
                "Viacheslav Klimkov",
                "Srikanth Ronanki",
                "Jonas Rohnke",
                "Thomas Drugman"
            ],
            "abstract": "We present a neural text-to-speech system for fine-grained prosody transfer\nfrom one speaker to another. Conventional approaches for end-to-end prosody\ntransfer typically use either fixed-dimensional or variable-length prosody\nembedding via a secondary attention to encode the reference signal. However,\nwhen trained on a single-speaker dataset, the conventional prosody transfer\nsystems are not robust enough to speaker variability, especially in the case of\na reference signal coming from an unseen speaker. Therefore, we propose\ndecoupling of the reference signal alignment from the overall system. For this\npurpose, we pre-compute phoneme-level time stamps and use them to aggregate\nprosodic features per phoneme, injecting them into a sequence-to-sequence\ntext-to-speech system. We incorporate a variational auto-encoder to further\nenhance the latent representation of prosody embeddings. We show that our\nproposed approach is significantly more stable and achieves reliable prosody\ntransplantation from an unseen speaker. We also propose a solution to the use\ncase in which the transcription of the reference signal is absent. We evaluate\nall our proposed methods using both objective and subjective listening tests.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.02636v2",
            "date": "2019-07-04 02:54:02+00:00",
            "title": "Collecting Indicators of Compromise from Unstructured Text of Cybersecurity Articles using Neural-Based Sequence Labelling",
            "authors": [
                "Zi Long",
                "Lianzhi Tan",
                "Shengping Zhou",
                "Chaoyang He",
                "Xin Liu"
            ],
            "abstract": "Indicators of Compromise (IOCs) are artifacts observed on a network or in an\noperating system that can be utilized to indicate a computer intrusion and\ndetect cyber-attacks in an early stage. Thus, they exert an important role in\nthe field of cybersecurity. However, state-of-the-art IOCs detection systems\nrely heavily on hand-crafted features with expert knowledge of cybersecurity,\nand require large-scale manually annotated corpora to train an IOC classifier.\nIn this paper, we propose using an end-to-end neural-based sequence labelling\nmodel to identify IOCs automatically from cybersecurity articles without expert\nknowledge of cybersecurity. By using a multi-head self-attention module and\ncontextual features, we find that the proposed model is capable of gathering\ncontextual information from texts of cybersecurity articles and performs better\nin the task of IOC identification. Experiments show that the proposed model\noutperforms other sequence labelling models, achieving the average F1-score of\n89.0% on English cybersecurity article test set, and approximately the average\nF1-score of 81.8% on Chinese test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02848v2",
            "date": "2019-07-05 14:19:15+00:00",
            "title": "Deep Reinforcement Learning For Modeling Chit-Chat Dialog With Discrete Attributes",
            "authors": [
                "Chinnadhurai Sankar",
                "Sujith Ravi"
            ],
            "abstract": "Open domain dialog systems face the challenge of being repetitive and\nproducing generic responses. In this paper, we demonstrate that by conditioning\nthe response generation on interpretable discrete dialog attributes and\ncomposed attributes, it helps improve the model perplexity and results in\ndiverse and interesting non-redundant responses. We propose to formulate the\ndialog attribute prediction as a reinforcement learning (RL) problem and use\npolicy gradients methods to optimize utterance generation using long-term\nrewards. Unlike existing RL approaches which formulate the token prediction as\na policy, our method reduces the complexity of the policy optimization by\nlimiting the action space to dialog attributes, thereby making the policy\noptimization more practical and sample efficient. We demonstrate this with\nexperimental and human evaluations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.02884v1",
            "date": "2019-07-05 15:11:29+00:00",
            "title": "Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model",
            "authors": [
                "Giuseppe Castellucci",
                "Valentina Bellomaria",
                "Andrea Favalli",
                "Raniero Romagnoli"
            ],
            "abstract": "Intent Detection and Slot Filling are two pillar tasks in Spoken Natural\nLanguage Understanding. Common approaches adopt joint Deep Learning\narchitectures in attention-based recurrent frameworks. In this work, we aim at\nexploiting the success of \"recurrence-less\" models for these tasks. We\nintroduce Bert-Joint, i.e., a multi-lingual joint text classification and\nsequence labeling framework. The experimental evaluation over two well-known\nEnglish benchmarks demonstrates the strong performances that can be obtained\nwith this model, even when few annotated data is available. Moreover, we\nannotated a new dataset for the Italian language, and we observed similar\nperformances without the need for changing the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02964v1",
            "date": "2019-07-04 21:42:26+00:00",
            "title": "Un Modelo Ontol\u00f3gico para el Gobierno Electr\u00f3nico",
            "authors": [
                "Carlos Roberto Brys",
                "Jos\u00e9 F. Aldana-Montes",
                "David Luis La Red Mart\u00ednez"
            ],
            "abstract": "Decision making often requires information that must be Provided with the\nrich data format. Addressing these new requirements appropriately makes it\nnecessary for government agencies to orchestrate large amounts of information\nfrom different sources and formats, to be efficiently delivered through the\ndevices commonly used by people, such as computers, netbooks, tablets and\nsmartphones. To overcome these problems, a model is proposed for the conceptual\nrepresentation of the State's organizational units, seen as georeferenced\nentities of Electronic Government, based on ontologies designed under the\nprinciples of Linked Open Data, which allows the automatic extraction of\ninformation through the machines, which supports the process of governmental\ndecision making and gives citizens full access to find and process through\nmobile technologies.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02997v2",
            "date": "2019-07-05 18:53:20+00:00",
            "title": "MigrationMiner: An Automated Detection Tool of Third-Party Java Library Migration at the Method Level",
            "authors": [
                "Hussein Alrubaye",
                "Mohamed Wiem Mkaouer",
                "Ali Ouni"
            ],
            "abstract": "In this paper we introduce, MigrationMiner, an automated tool that detects\ncode migrations performed between Java third-party library. Given a list of\nopen source projects, the tool detects potential library migration code changes\nand collects the specific code fragments in which the developer replaces\nmethods from the retired library with methods from the new library. To support\nthe migration process, MigrationMiner collects the library documentation that\nis associated with every method involved in the migration. We evaluate our tool\non a benchmark of manually validated library migrations. Results show that\nMigrationMiner achieves an accuracy of 100%. A demo video of MigrationMiner is\navailable at https://youtu.be/sAlR1HNetXc.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1907.03020v1",
            "date": "2019-07-05 20:43:30+00:00",
            "title": "Towards Universal Dialogue Act Tagging for Task-Oriented Dialogues",
            "authors": [
                "Shachi Paul",
                "Rahul Goel",
                "Dilek Hakkani-T\u00fcr"
            ],
            "abstract": "Machine learning approaches for building task-oriented dialogue systems\nrequire large conversational datasets with labels to train on. We are\ninterested in building task-oriented dialogue systems from human-human\nconversations, which may be available in ample amounts in existing customer\ncare center logs or can be collected from crowd workers. Annotating these\ndatasets can be prohibitively expensive. Recently multiple annotated\ntask-oriented human-machine dialogue datasets have been released, however their\nannotation schema varies across different collections, even for well-defined\ncategories such as dialogue acts (DAs). We propose a Universal DA schema for\ntask-oriented dialogues and align existing annotated datasets with our schema.\nOur aim is to train a Universal DA tagger (U-DAT) for task-oriented dialogues\nand use it for tagging human-human conversations. We investigate multiple\ndatasets, propose manual and automated approaches for aligning the different\nschema, and present results on a target corpus of human-human dialogues. In\nunsupervised learning experiments we achieve an F1 score of 54.1% on system\nturns in human-human dialogues. In a semi-supervised setup, the F1 score\nincreases to 57.7% which would otherwise require at least 1.7K manually\nannotated turns. For new domains, we show further improvements when unlabeled\nor labeled target domain data is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03049v3",
            "date": "2019-07-05 23:47:04+00:00",
            "title": "Video Question Generation via Cross-Modal Self-Attention Networks Learning",
            "authors": [
                "Yu-Siang Wang",
                "Hung-Ting Su",
                "Chen-Hsi Chang",
                "Zhe-Yu Liu",
                "Winston H. Hsu"
            ],
            "abstract": "We introduce a novel task, Video Question Generation (Video QG). A Video QG\nmodel automatically generates questions given a video clip and its\ncorresponding dialogues. Video QG requires a range of skills -- sentence\ncomprehension, temporal relation, the interplay between vision and language,\nand the ability to ask meaningful questions. To address this, we propose a\nnovel semantic rich cross-modal self-attention (SRCMSA) network to aggregate\nthe multi-modal and diverse features. To be more precise, we enhance the video\nframes semantic by integrating the object-level information, and we jointly\nconsider the cross-modal attention for the video question generation task.\nExcitingly, our proposed model remarkably improves the baseline from 7.58 to\n14.48 in the BLEU-4 score on the TVQA dataset. Most of all, we arguably pave a\nnovel path toward understanding the challenging video input and we provide\ndetailed analysis in terms of diversity, which ushers the avenues for future\ninvestigations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.03110v1",
            "date": "2019-07-06 10:37:18+00:00",
            "title": "ANETAC: Arabic Named Entity Transliteration and Classification Dataset",
            "authors": [
                "Mohamed Seghir Hadj Ameur",
                "Farid Meziane",
                "Ahmed Guessoum"
            ],
            "abstract": "In this paper, we make freely accessible ANETAC our English-Arabic named\nentity transliteration and classification dataset that we built from freely\navailable parallel translation corpora. The dataset contains 79,924 instances,\neach instance is a triplet (e, a, c), where e is the English named entity, a is\nits Arabic transliteration and c is its class that can be either a Person, a\nLocation, or an Organization. The ANETAC dataset is mainly aimed for the\nresearchers that are working on Arabic named entity transliteration, but it can\nalso be used for named entity classification purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03202v1",
            "date": "2019-07-06 22:51:28+00:00",
            "title": "Evolutionary Algorithm for Sinhala to English Translation",
            "authors": [
                "J. K. Joseph",
                "W. M. T. Chathurika",
                "A. Nugaliyadde",
                "Y. Mallawarachchi"
            ],
            "abstract": "Machine Translation (MT) is an area in natural language processing, which\nfocus on translating from one language to another. Many approaches ranging from\nstatistical methods to deep learning approaches are used in order to achieve\nMT. However, these methods either require a large number of data or a clear\nunderstanding about the language. Sinhala language has less digital text which\ncould be used to train a deep neural network. Furthermore, Sinhala has complex\nrules therefore, it is harder to create statistical rules in order to apply\nstatistical methods in MT. This research focuses on Sinhala to English\ntranslation using an Evolutionary Algorithm (EA). EA is used to identifying the\ncorrect meaning of Sinhala text and to translate it to English. The Sinhala\ntext is passed to identify the meaning in order to get the correct meaning of\nthe sentence. With the use of the EA the translation is carried out. The\ntranslated text is passed on to grammatically correct the sentence. This has\nshown to achieve accurate results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03240v2",
            "date": "2019-07-07 07:41:59+00:00",
            "title": "Informative Visual Storytelling with Cross-modal Rules",
            "authors": [
                "Jiacheng Li",
                "Haizhou Shi",
                "Siliang Tang",
                "Fei Wu",
                "Yueting Zhuang"
            ],
            "abstract": "Existing methods in the Visual Storytelling field often suffer from the\nproblem of generating general descriptions, while the image contains a lot of\nmeaningful contents remaining unnoticed. The failure of informative story\ngeneration can be concluded to the model's incompetence of capturing enough\nmeaningful concepts. The categories of these concepts include entities,\nattributes, actions, and events, which are in some cases crucial to grounded\nstorytelling. To solve this problem, we propose a method to mine the\ncross-modal rules to help the model infer these informative concepts given\ncertain visual input. We first build the multimodal transactions by\nconcatenating the CNN activations and the word indices. Then we use the\nassociation rule mining algorithm to mine the cross-modal rules, which will be\nused for the concept inference. With the help of the cross-modal rules, the\ngenerated stories are more grounded and informative. Besides, our proposed\nmethod holds the advantages of interpretation, expandability, and\ntransferability, indicating potential for wider application. Finally, we\nleverage these concepts in our encoder-decoder framework with the attention\nmechanism. We conduct several experiments on the VIsual StoryTelling~(VIST)\ndataset, the results of which demonstrate the effectiveness of our approach in\nterms of both automatic metrics and human evaluation. Additional experiments\nare also conducted showing that our mined cross-modal rules as additional\nknowledge helps the model gain better performance when trained on a small\ndataset.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1907.03399v1",
            "date": "2019-07-08 04:19:17+00:00",
            "title": "A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context",
            "authors": [
                "Takuma Udagawa",
                "Akiko Aizawa"
            ],
            "abstract": "Common grounding is the process of creating, repairing and updating mutual\nunderstandings, which is a critical aspect of sophisticated human\ncommunication. However, traditional dialogue systems have limited capability of\nestablishing common ground, and we also lack task formulations which introduce\nnatural difficulty in terms of common grounding while enabling easy evaluation\nand analysis of complex models. In this paper, we propose a minimal dialogue\ntask which requires advanced skills of common grounding under continuous and\npartially-observable context. Based on this task formulation, we collected a\nlargescale dataset of 6,760 dialogues which fulfills essential requirements of\nnatural language corpora. Our analysis of the dataset revealed important\nphenomena related to common grounding that need to be considered. Finally, we\nevaluate and analyze baseline neural models on a simple subtask that requires\nrecognition of the created common ground. We show that simple baseline models\nperform decently but leave room for further improvement. Overall, we show that\nour proposed task will be a fundamental testbed where we can train, evaluate,\nand analyze dialogue system's ability for sophisticated common grounding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03590v2",
            "date": "2019-07-08 13:16:07+00:00",
            "title": "Multiple Generative Models Ensemble for Knowledge-Driven Proactive Human-Computer Dialogue Agent",
            "authors": [
                "Zelin Dai",
                "Weitang Liu",
                "Guanhua Zhan"
            ],
            "abstract": "Multiple sequence to sequence models were used to establish an end-to-end\nmulti-turns proactive dialogue generation agent, with the aid of data\naugmentation techniques and variant encoder-decoder structure designs. A\nrank-based ensemble approach was developed for boosting performance. Results\nindicate that our single model, in average, makes an obvious improvement in the\nterms of F1-score and BLEU over the baseline by 18.67% on the DuConv dataset.\nIn particular, the ensemble methods further significantly outperform the\nbaseline by 35.85%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03920v1",
            "date": "2019-07-09 00:44:23+00:00",
            "title": "Hahahahaha, Duuuuude, Yeeessss!: A two-parameter characterization of stretchable words and the dynamics of mistypings and misspellings",
            "authors": [
                "Tyler J. Gray",
                "Christopher M. Danforth",
                "Peter Sheridan Dodds"
            ],
            "abstract": "Stretched words like `heellllp' or `heyyyyy' are a regular feature of spoken\nlanguage, often used to emphasize or exaggerate the underlying meaning of the\nroot word. While stretched words are rarely found in formal written language\nand dictionaries, they are prevalent within social media. In this paper, we\nexamine the frequency distributions of `stretchable words' found in roughly 100\nbillion tweets authored over an 8 year period. We introduce two central\nparameters, `balance' and `stretch', that capture their main characteristics,\nand explore their dynamics by creating visual tools we call `balance plots' and\n`spelling trees'. We discuss how the tools and methods we develop here could be\nused to study the statistical patterns of mistypings and misspellings, along\nwith the potential applications in augmenting dictionaries, improving language\nprocessing, and in any area where sequence construction matters, such as\ngenetics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03969v2",
            "date": "2019-07-09 03:40:13+00:00",
            "title": "Systematic quantitative analyses reveal the folk-zoological knowledge embedded in folktales",
            "authors": [
                "Yo Nakawake",
                "Kosuke Sato"
            ],
            "abstract": "Cultural learning is a unique human capacity essential for a wide range of\nadaptations. Researchers have argued that folktales have the pedagogical\nfunction of transmitting the essential information for the environment. The\nmost important knowledge for foraging and pastoral society is folk-zoological\nknowledge, such as the predator-prey relationship among wild animals, or\nbetween wild and domesticated animals. Here, we analysed the descriptions of\nthe 382 animal folktales using the natural language processing method and\ndescriptive statistics listed in a worldwide tale-type index\n(Aarne-Thompson-Uther type index). Our analyses suggested that first, the\npredator-prey relationship frequently appeared in a co-occurrent animal pair\nwithin a folktale (e.g., cat and mouse or wolf and pig), and second, the motif\nof 'deception', describing the antagonistic behaviour among animals, appeared\nrelatively higher in 'wild and domestic animals' and 'wild animals' than other\ntypes. Furthermore, the motif of 'deception' appeared more frequently in pairs,\ncorresponding to the predator-prey relationship. These results corresponded\nwith the hypothesis that the combination of animal characters and what happens\nin stories represented relationships in the real world. The present study\ndemonstrated that the combination of quantitative methods and qualitative data\nbroaden our understanding of the evolutionary aspects of human cultures.",
            "journal ref": "Palgrave Commun 5, 161 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04105v1",
            "date": "2019-07-09 12:01:35+00:00",
            "title": "On the Semantic Interpretability of Artificial Intelligence Models",
            "authors": [
                "Vivian S. Silva",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "Artificial Intelligence models are becoming increasingly more powerful and\naccurate, supporting or even replacing humans' decision making. But with\nincreased power and accuracy also comes higher complexity, making it hard for\nusers to understand how the model works and what the reasons behind its\npredictions are. Humans must explain and justify their decisions, and so do the\nAI models supporting them in this process, making semantic interpretability an\nemerging field of study. In this work, we look at interpretability from a\nbroader point of view, going beyond the machine learning scope and covering\ndifferent AI fields such as distributional semantics and fuzzy logic, among\nothers. We examine and classify the models according to their nature and also\nbased on how they introduce interpretability features, analyzing how each\napproach affects the final users and pointing to gaps that still need to be\naddressed to provide more human-centered interpretability solutions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.04149v1",
            "date": "2019-07-04 17:25:28+00:00",
            "title": "Answer Extraction for Why Arabic Questions Answering Systems: EWAQ",
            "authors": [
                "Fatima T. AL-Khawaldeh"
            ],
            "abstract": "With the increasing amount of web information, questions answering systems\nbecomes very important to allow users to access to direct answers for their\nrequests. This paper presents an Arabic Questions Answering Systems based on\nentailment metrics. The type of questions which this paper focuses on is why\nquestions. There are many reasons lead us to develop this system: generally,\nthe lack of Arabic Questions Answering Systems and scarcity Arabic Questions\nAnswering Systems which focus on why questions. The goal of the proposed system\nin this research is to extract answers from re-ranked retrieved passages which\nare retrieved by search engines. This system extracts the answer only to why\nquestions. This system is called by EWAQ: Entailment based Why Arabic Questions\nAnswering. Each answer is scored with entailment metrics and ranked according\nto their scores in order to determine the most possible correct answer. EWAQ is\ncompared with search engines: yahoo, google and ask.com, the well-established\nweb-based Questions Answering systems, using manual test set. In EWAQ\nexperiments, it is showed that the accuracy is increased by implementing the\ntextual entailment in re-raking the retrieved relevant passages by search\nengines and deciding the correct answer. The obtained results show that using\nentailment based similarity can help significantly to tackle the why Answer\nExtraction module in Arabic language.",
            "journal ref": "World of Computer Science and Information Technology Journal\n  (WCSIT) ISSN: 2221-0741 Vol. 5, No. 5, 82-86, 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04265v1",
            "date": "2019-07-09 16:07:38+00:00",
            "title": "Translating neural signals to text using a Brain-Machine Interface",
            "authors": [
                "Janaki Sheth",
                "Ariel Tankus",
                "Michelle Tran",
                "Nader Pouratian",
                "Itzhak Fried",
                "William Speier"
            ],
            "abstract": "Brain-Computer Interfaces (BCI) help patients with faltering communication\nabilities due to neurodegenerative diseases produce text or speech output by\ndirect neural processing. However, practical implementation of such a system\nhas proven difficult due to limitations in speed, accuracy, and\ngeneralizability of the existing interfaces. To this end, we aim to create a\nBCI system that decodes text directly from neural signals. We implement a\nframework that initially isolates frequency bands in the input signal\nencapsulating differential information regarding production of various phonemic\nclasses. These bands then form a feature set that feeds into an LSTM which\ndiscerns at each time point probability distributions across all phonemes\nuttered by a subject. Finally, these probabilities are fed into a particle\nfiltering algorithm which incorporates prior knowledge of the English language\nto output text corresponding to the decoded word. Performance of this model on\ndata obtained from six patients shows encouragingly high levels of accuracy at\nspeeds and bit rates significantly higher than existing BCI communication\nsystems. Further, in producing an output, our network abstains from\nconstraining the reconstructed word to be from a given bag-of-words, unlike\nprevious studies. The success of our proposed approach, offers promise for the\nemployment of a BCI interface by patients in unfettered, naturalistic\nenvironments.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1907.04407v2",
            "date": "2019-07-09 20:46:37+00:00",
            "title": "Sentiment Analysis Challenges in Persian Language",
            "authors": [
                "Mohammad Heydari"
            ],
            "abstract": "The rapid growth in data on the internet requires a data mining process to\nreach a decision to support insight. The Persian language has strong potential\nfor deep research in any aspect of natural language processing, especially\nsentimental analysis approach. Thousands of websites and blogs updates and\nmodifies by Persian users around the world that contains millions of Persian\ncontext. This range of application requires a comprehensive structured\nframework to extract beneficial information for helping enterprises to enhance\ntheir business and initiate a customer-centric management process by producing\neffective recommender systems. Sentimental analysis is an intelligent approach\nfor extracting useful information from huge amounts of data to help an\nenterprise for smart management process. In this road, machine learning and\ndeep learning techniques will become very helpful but there is the number of\nchallenges which are face to them. This paper tried to present and assert the\nmost important challenges of sentimental analysis in the Persian language. This\nlanguage is an Indo-European language which spoken by over 110 million people\naround the world and is an official language in Iran, Tajikistan, and\nAfghanistan. Its also widely used in Uzbekistan, Pakistan and Turkish by order.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.04613v1",
            "date": "2019-07-10 10:50:22+00:00",
            "title": "Neural Networks as Explicit Word-Based Rules",
            "authors": [
                "Jind\u0159ich Libovick\u00fd"
            ],
            "abstract": "Filters of convolutional networks used in computer vision are often\nvisualized as image patches that maximize the response of the filter. We use\nthe same approach to interpret weight matrices in simple architectures for\nnatural language processing tasks. We interpret a convolutional network for\nsentiment classification as word-based rules. Using the rule, we recover the\nperformance of the original model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04944v2",
            "date": "2019-07-10 22:13:48+00:00",
            "title": "Can Unconditional Language Models Recover Arbitrary Sentences?",
            "authors": [
                "Nishant Subramani",
                "Samuel R. Bowman",
                "Kyunghyun Cho"
            ],
            "abstract": "Neural network-based generative language models like ELMo and BERT can work\neffectively as general purpose sentence encoders in text classification without\nfurther fine-tuning. Is it possible to adapt them in a similar way for use as\ngeneral-purpose decoders? For this to be possible, it would need to be the case\nthat for any target sentence of interest, there is some continuous\nrepresentation that can be passed to the language model to cause it to\nreproduce that sentence. We set aside the difficult problem of designing an\nencoder that can produce such representations and, instead, ask directly\nwhether such representations exist at all. To do this, we introduce a pair of\neffective, complementary methods for feeding representations into pretrained\nunconditional language models and a corresponding set of methods to map\nsentences into and out of this representation space, the reparametrized\nsentence space. We then investigate the conditions under which a language model\ncan be made to generate a sentence through the identification of a point in\nsuch a space and find that it is possible to recover arbitrary sentences nearly\nperfectly with language models and representations of moderate size without\nmodifying any model parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05019v1",
            "date": "2019-07-11 06:47:30+00:00",
            "title": "Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges",
            "authors": [
                "Naveen Arivazhagan",
                "Ankur Bapna",
                "Orhan Firat",
                "Dmitry Lepikhin",
                "Melvin Johnson",
                "Maxim Krikun",
                "Mia Xu Chen",
                "Yuan Cao",
                "George Foster",
                "Colin Cherry",
                "Wolfgang Macherey",
                "Zhifeng Chen",
                "Yonghui Wu"
            ],
            "abstract": "We introduce our efforts towards building a universal neural machine\ntranslation (NMT) system capable of translating between any language pair. We\nset a milestone towards this goal by building a single massively multilingual\nNMT model handling 103 languages trained on over 25 billion examples. Our\nsystem demonstrates effective transfer learning ability, significantly\nimproving translation quality of low-resource languages, while keeping\nhigh-resource language translation quality on-par with competitive bilingual\nbaselines. We provide in-depth analysis of various aspects of model building\nthat are crucial to achieving quality and practicality in universal NMT. While\nwe prototype a high-quality universal translation system, our extensive\nempirical analysis exposes issues that need to be further addressed, and we\nsuggest directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05084v1",
            "date": "2019-07-11 10:06:20+00:00",
            "title": "MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment",
            "authors": [
                "Nikolai Ilinykh",
                "Sina Zarrie\u00df",
                "David Schlangen"
            ],
            "abstract": "Building computer systems that can converse about their visual environment is\none of the oldest concerns of research in Artificial Intelligence and\nComputational Linguistics (see, for example, Winograd's 1972 SHRDLU system).\nOnly recently, however, have methods from computer vision and natural language\nprocessing become powerful enough to make this vision seem more attainable.\nPushed especially by developments in computer vision, many data sets and\ncollection environments have recently been published that bring together verbal\ninteraction and visual processing. Here, we argue that these datasets tend to\noversimplify the dialogue part, and we propose a task---MeetUp!---that requires\nboth visual and conversational grounding, and that makes stronger demands on\nrepresentations of the discourse. MeetUp! is a two-player coordination game\nwhere players move in a visual environment, with the objective of finding each\nother. To do so, they must talk about what they see, and achieve mutual\nunderstanding. We describe a data collection and show that the resulting\ndialogues indeed exhibit the dialogue phenomena of interest, while also\nchallenging the language & vision aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05190v2",
            "date": "2019-07-11 13:42:46+00:00",
            "title": "Self-Regulated Interactive Sequence-to-Sequence Learning",
            "authors": [
                "Julia Kreutzer",
                "Stefan Riezler"
            ],
            "abstract": "Not all types of supervision signals are created equal: Different types of\nfeedback have different costs and effects on learning. We show how\nself-regulation strategies that decide when to ask for which kind of feedback\nfrom a teacher (or from oneself) can be cast as a learning-to-learn problem\nleading to improved cost-aware sequence-to-sequence learning. In experiments on\ninteractive neural machine translation, we find that the self-regulator\ndiscovers an $\\epsilon$-greedy strategy for the optimal cost-quality trade-off\nby mixing different feedback types including corrections, error markups, and\nself-supervision. Furthermore, we demonstrate its robustness under domain shift\nand identify it as a promising alternative to active learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05242v2",
            "date": "2019-07-10 14:52:12+00:00",
            "title": "Large Memory Layers with Product Keys",
            "authors": [
                "Guillaume Lample",
                "Alexandre Sablayrolles",
                "Marc'Aurelio Ranzato",
                "Ludovic Denoyer",
                "Herv\u00e9 J\u00e9gou"
            ],
            "abstract": "This paper introduces a structured memory which can be easily integrated into\na neural network. The memory is very large by design and significantly\nincreases the capacity of the architecture, by up to a billion parameters with\na negligible computational overhead. Its design and access pattern is based on\nproduct keys, which enable fast and exact nearest neighbor search. The ability\nto increase the number of parameters while keeping the same computational\nbudget lets the overall system strike a better trade-off between prediction\naccuracy and computation efficiency both at training and test time. This memory\nlayer allows us to tackle very large scale language modeling tasks. In our\nexperiments we consider a dataset with up to 30 billion words, and we plug our\nmemory layer in a state-of-the-art transformer-based architecture. In\nparticular, we found that a memory augmented model with only 12 layers\noutperforms a baseline transformer model with 24 layers, while being twice\nfaster at inference time. We release our code for reproducibility purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05333v1",
            "date": "2019-07-08 02:16:11+00:00",
            "title": "Improving Neural Relation Extraction with Implicit Mutual Relations",
            "authors": [
                "Jun Kuang",
                "Yixin Cao",
                "Jianbing Zheng",
                "Xiangnan He",
                "Ming Gao",
                "Aoying Zhou"
            ],
            "abstract": "Relation extraction (RE) aims at extracting the relation between two entities\nfrom the text corpora. It is a crucial task for Knowledge Graph (KG)\nconstruction. Most existing methods predict the relation between an entity pair\nby learning the relation from the training sentences, which contain the\ntargeted entity pair. In contrast to existing distant supervision approaches\nthat suffer from insufficient training corpora to extract relations, our\nproposal of mining implicit mutual relation from the massive unlabeled corpora\ntransfers the semantic information of entity pairs into the RE model, which is\nmore expressive and semantically plausible. After constructing an entity\nproximity graph based on the implicit mutual relations, we preserve the\nsemantic relations of entity pairs via embedding each vertex of the graph into\na low-dimensional space. As a result, we can easily and flexibly integrate the\nimplicit mutual relations and other entity information, such as entity types,\ninto the existing RE methods.\n  Our experimental results on a New York Times and another Google Distant\nSupervision datasets suggest that our proposed neural RE framework provides a\npromising improvement for the RE task, and significantly outperforms the\nstate-of-the-art methods. Moreover, the component for mining implicit mutual\nrelations is so flexible that can help to improve the performance of both\nCNN-based and RNN-based RE models significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05338v1",
            "date": "2019-07-09 04:46:31+00:00",
            "title": "To Tune or Not To Tune? How About the Best of Both Worlds?",
            "authors": [
                "Ran Wang",
                "Haibo Su",
                "Chunye Wang",
                "Kailin Ji",
                "Jupeng Ding"
            ],
            "abstract": "The introduction of pre-trained language models has revolutionized natural\nlanguage research communities. However, researchers still know relatively\nlittle regarding their theoretical and empirical properties. In this regard,\nPeters et al. perform several experiments which demonstrate that it is better\nto adapt BERT with a light-weight task-specific head, rather than building a\ncomplex one on top of the pre-trained language model, and freeze the parameters\nin the said language model. However, there is another option to adopt. In this\npaper, we propose a new adaptation method which we first train the task model\nwith the BERT parameters frozen and then fine-tune the entire model together.\nOur experimental results show that our model adaptation method can achieve 4.7%\naccuracy improvement in semantic similarity task, 0.99% accuracy improvement in\nsequence labeling task and 0.72% accuracy improvement in the text\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05339v1",
            "date": "2019-07-09 04:11:15+00:00",
            "title": "ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation",
            "authors": [
                "Hainan Zhang",
                "Yanyan Lan",
                "Liang Pang",
                "Jiafeng Guo",
                "Xueqi Cheng"
            ],
            "abstract": "In multi-turn dialogue generation, response is usually related with only a\nfew contexts. Therefore, an ideal model should be able to detect these relevant\ncontexts and produce a suitable response accordingly. However, the widely used\nhierarchical recurrent encoderdecoder models just treat all the contexts\nindiscriminately, which may hurt the following response generation process.\nSome researchers try to use the cosine similarity or the traditional attention\nmechanism to find the relevant contexts, but they suffer from either\ninsufficient relevance assumption or position bias problem. In this paper, we\npropose a new model, named ReCoSa, to tackle this problem. Firstly, a word\nlevel LSTM encoder is conducted to obtain the initial representation of each\ncontext. Then, the self-attention mechanism is utilized to update both the\ncontext and masked response representation. Finally, the attention weights\nbetween each context and response representations are computed and used in the\nfurther decoding process. Experimental results on both Chinese customer\nservices dataset and English Ubuntu dialogue dataset show that ReCoSa\nsignificantly outperforms baseline models, in terms of both metric-based and\nhuman evaluations. Further analysis on attention shows that the detected\nrelevant contexts by ReCoSa are highly coherent with human's understanding,\nvalidating the correctness and interpretability of ReCoSa.",
            "journal ref": "ACL2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05343v2",
            "date": "2019-07-10 04:51:44+00:00",
            "title": "Semantic Parsing with Dual Learning",
            "authors": [
                "Ruisheng Cao",
                "Su Zhu",
                "Chen Liu",
                "Jieyu Li",
                "Kai Yu"
            ],
            "abstract": "Semantic parsing converts natural language queries into structured logical\nforms. The paucity of annotated training samples is a fundamental challenge in\nthis field. In this work, we develop a semantic parsing framework with the dual\nlearning algorithm, which enables a semantic parser to make full use of data\n(labeled and even unlabeled) through a dual-learning game. This game between a\nprimal model (semantic parsing) and a dual model (logical form to query) forces\nthem to regularize each other, and can achieve feedback signals from some\nprior-knowledge. By utilizing the prior-knowledge of logical form structures,\nwe propose a novel reward signal at the surface and semantic levels which tends\nto generate complete and reasonable logical forms. Experimental results show\nthat our approach achieves new state-of-the-art performance on ATIS dataset and\ngets competitive performance on Overnight dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05507v2",
            "date": "2019-07-11 22:05:48+00:00",
            "title": "Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learning",
            "authors": [
                "Alexandros Papangelis",
                "Yi-Chia Wang",
                "Piero Molino",
                "Gokhan Tur"
            ],
            "abstract": "We present the first complete attempt at concurrently training conversational\nagents that communicate only via self-generated language. Using DSTC2 as seed\ndata, we trained natural language understanding (NLU) and generation (NLG)\nnetworks for each agent and let the agents interact online. We model the\ninteraction as a stochastic collaborative game where each agent (player) has a\nrole (\"assistant\", \"tourist\", \"eater\", etc.) and their own objectives, and can\nonly interact via natural language they generate. Each agent, therefore, needs\nto learn to operate optimally in an environment with multiple sources of\nuncertainty (its own NLU and NLG, the other agent's NLU, Policy, and NLG). In\nour evaluation, we show that the stochastic-game agents outperform deep\nlearning based supervised baselines.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1907.05545v2",
            "date": "2019-07-12 01:55:36+00:00",
            "title": "The Dynamic Embedded Topic Model",
            "authors": [
                "Adji B. Dieng",
                "Francisco J. R. Ruiz",
                "David M. Blei"
            ],
            "abstract": "Topic modeling analyzes documents to learn meaningful patterns of words. For\ndocuments collected in sequence, dynamic topic models capture how these\npatterns vary over time. We develop the dynamic embedded topic model (D-ETM), a\ngenerative model of documents that combines dynamic latent Dirichlet allocation\n(D-LDA) and word embeddings. The D-ETM models each word with a categorical\ndistribution parameterized by the inner product between the word embedding and\na per-time-step embedding representation of its assigned topic. The D-ETM\nlearns smooth topic trajectories by defining a random walk prior over the\nembedding representations of the topics. We fit the D-ETM using structured\namortized variational inference with a recurrent neural network. On three\ndifferent corpora---a collection of United Nations debates, a set of ACL\nabstracts, and a dataset of Science Magazine articles---we found that the D-ETM\noutperforms D-LDA on a document completion task. We further found that the\nD-ETM learns more diverse and coherent topics than D-LDA while requiring\nsignificantly less time to fit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05559v1",
            "date": "2019-07-12 03:11:14+00:00",
            "title": "NPA: Neural News Recommendation with Personalized Attention",
            "authors": [
                "Chuhan Wu",
                "Fangzhao Wu",
                "Mingxiao An",
                "Jianqiang Huang",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "News recommendation is very important to help users find interested news and\nalleviate information overload. Different users usually have different\ninterests and the same user may have various interests. Thus, different users\nmay click the same news article with attention on different aspects. In this\npaper, we propose a neural news recommendation model with personalized\nattention (NPA). The core of our approach is a news representation model and a\nuser representation model. In the news representation model we use a CNN\nnetwork to learn hidden representations of news articles based on their titles.\nIn the user representation model we learn the representations of users based on\nthe representations of their clicked news articles. Since different words and\ndifferent news articles may have different informativeness for representing\nnews and users, we propose to apply both word- and news-level attention\nmechanism to help our model attend to important words and news articles. In\naddition, the same news article and the same word may have different\ninformativeness for different users. Thus, we propose a personalized attention\nnetwork which exploits the embedding of user ID to generate the query vector\nfor the word- and news-level attentions. Extensive experiments are conducted on\na real-world news recommendation dataset collected from MSN news, and the\nresults validate the effectiveness of our approach on news recommendation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.05562v2",
            "date": "2019-07-12 03:34:06+00:00",
            "title": "ScenarioSA: A Large Scale Conversational Database for Interactive Sentiment Analysis",
            "authors": [
                "Yazhou Zhang",
                "Lingling Song",
                "Dawei Song",
                "Peng Guo",
                "Junwei Zhang",
                "Peng Zhang"
            ],
            "abstract": "Interactive sentiment analysis is an emerging, yet challenging, subtask of\nthe sentiment analysis problem. It aims to discover the affective state and\nsentimental change of each person in a conversation. Existing sentiment\nanalysis approaches are insufficient in modelling the interactions among\npeople. However, the development of new approaches are critically limited by\nthe lack of labelled interactive sentiment datasets. In this paper, we present\na new conversational emotion database that we have created and made publically\navailable, namely ScenarioSA. We manually label 2,214 multi-turn English\nconversations collected from natural contexts. In comparison with existing\nsentiment datasets, ScenarioSA (1) covers a wide range of scenarios; (2)\ndescribes the interactions between two speakers; and (3) reflects the\nsentimental evolution of each speaker over the course of a conversation.\nFinally, we evaluate various state-of-the-art algorithms on ScenarioSA,\ndemonstrating the need of novel interactive sentiment analysis models and the\npotential of ScenarioSA to facilitate the development of such models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.05664v1",
            "date": "2019-07-12 10:28:00+00:00",
            "title": "Saliency Maps Generation for Automatic Text Summarization",
            "authors": [
                "David Tuckey",
                "Krysia Broda",
                "Alessandra Russo"
            ],
            "abstract": "Saliency map generation techniques are at the forefront of explainable AI\nliterature for a broad range of machine learning applications. Our goal is to\nquestion the limits of these approaches on more complex tasks. In this paper we\napply Layer-Wise Relevance Propagation (LRP) to a sequence-to-sequence\nattention model trained on a text summarization dataset. We obtain unexpected\nsaliency maps and discuss the rightfulness of these \"explanations\". We argue\nthat we need a quantitative way of testing the counterfactual case to judge the\ntruthfulness of the saliency maps. We suggest a protocol to check the validity\nof the importance attributed to the input and show that the saliency maps\nobtained sometimes capture the real use of the input features by the network,\nand sometimes do not. We use this example to discuss how careful we need to be\nwhen accepting them as explanation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.05839v1",
            "date": "2019-07-12 17:02:14+00:00",
            "title": "Equiprobable mappings in weighted constraint grammars",
            "authors": [
                "Arto Anttila",
                "Scott Borgeson",
                "Giorgio Magri"
            ],
            "abstract": "We show that MaxEnt is so rich that it can distinguish between any two\ndifferent mappings: there always exists a nonnegative weight vector which\nassigns them different MaxEnt probabilities. Stochastic HG instead does admit\nequiprobable mappings and we give a complete formal characterization of them.\nWe compare these different predictions of the two frameworks on a test case of\nFinnish stress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05955v3",
            "date": "2019-07-12 20:54:59+00:00",
            "title": "PyKaldi2: Yet another speech toolkit based on Kaldi and PyTorch",
            "authors": [
                "Liang Lu",
                "Xiong Xiao",
                "Zhuo Chen",
                "Yifan Gong"
            ],
            "abstract": "We introduce PyKaldi2 speech recognition toolkit implemented based on Kaldi\nand PyTorch. While similar toolkits are available built on top of the two, a\nkey feature of PyKaldi2 is sequence training with criteria such as MMI, sMBR\nand MPE. In particular, we implemented the sequence training module with\non-the-fly lattice generation during model training in order to simplify the\ntraining pipeline. To address the challenging acoustic environments in real\napplications, PyKaldi2 also supports on-the-fly noise and reverberation\nsimulation to improve the model robustness. With this feature, it is possible\nto backpropogate the gradients from the sequence-level loss to the front-end\nfeature extraction module, which, hopefully, can foster more research in the\ndirection of joint front-end and backend learning. We performed benchmark\nexperiments on Librispeech, and show that PyKaldi2 can achieve reasonable\nrecognition accuracy. The toolkit is released under the MIT license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06017v1",
            "date": "2019-07-13 06:27:24+00:00",
            "title": "Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition",
            "authors": [
                "Ye Bai",
                "Jiangyan Yi",
                "Jianhua Tao",
                "Zhengkun Tian",
                "Zhengqi Wen"
            ],
            "abstract": "Integrating an external language model into a sequence-to-sequence speech\nrecognition system is non-trivial. Previous works utilize linear interpolation\nor a fusion network to integrate external language models. However, these\napproaches introduce external components, and increase decoding computation. In\nthis paper, we instead propose a knowledge distillation based training approach\nto integrating external language models into a sequence-to-sequence model. A\nrecurrent neural network language model, which is trained on large scale\nexternal text, generates soft labels to guide the sequence-to-sequence model\ntraining. Thus, the language model plays the role of the teacher. This approach\ndoes not add any external component to the sequence-to-sequence model during\ntesting. And this approach is flexible to be combined with shallow fusion\ntechnique together for decoding. The experiments are conducted on public\nChinese datasets AISHELL-1 and CLMAD. Our approach achieves a character error\nrate of 9.3%, which is relatively reduced by 18.42% compared with the vanilla\nsequence-to-sequence model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.06142v1",
            "date": "2019-07-13 22:48:31+00:00",
            "title": "Tackling Graphical NLP problems with Graph Recurrent Networks",
            "authors": [
                "Linfeng Song"
            ],
            "abstract": "How to properly model graphs is a long-existing and important problem in NLP\narea, where several popular types of graphs are knowledge graphs, semantic\ngraphs and dependency graphs. Comparing with other data structures, such as\nsequences and trees, graphs are generally more powerful in representing complex\ncorrelations among entities. For example, a knowledge graph stores real-word\nentities (such as \"Barack_Obama\" and \"U.S.\") and their relations (such as\n\"live_in\" and \"lead_by\"). Properly encoding a knowledge graph is beneficial to\nuser applications, such as question answering and knowledge discovery. Modeling\ngraphs is also very challenging, probably because graphs usually contain\nmassive and cyclic relations.\n  Recent years have witnessed the success of deep learning, especially\nRNN-based models, on many NLP problems. Besides, RNNs and their variations have\nbeen extensively studied on several graph problems and showed preliminary\nsuccesses. Despite the successes that have been achieved, RNN-based models\nsuffer from several major drawbacks on graphs. First, they can only consume\nsequential data, thus linearization is required to serialize input graphs,\nresulting in the loss of important structural information. Second, the\nserialization results are usually very long, so it takes a long time for RNNs\nto encode them.\n  In this thesis, we propose a novel graph neural network, named graph\nrecurrent network (GRN). We study our GRN model on 4 very different tasks, such\nas machine reading comprehension, relation extraction and machine translation.\nSome take undirected graphs without edge labels, while the others have directed\nones with edge labels. To consider these important differences, we gradually\nenhance our GRN model, such as further considering edge labels and adding an\nRNN decoder. Carefully designed experiments show the effectiveness of GRN on\nall these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06385v1",
            "date": "2019-07-15 09:23:49+00:00",
            "title": "GLOSS: Generative Latent Optimization of Sentence Representations",
            "authors": [
                "Sidak Pal Singh",
                "Angela Fan",
                "Michael Auli"
            ],
            "abstract": "We propose a method to learn unsupervised sentence representations in a\nnon-compositional manner based on Generative Latent Optimization. Our approach\ndoes not impose any assumptions on how words are to be combined into a sentence\nrepresentation. We discuss a simple Bag of Words model as well as a variant\nthat models word positions. Both are trained to reconstruct the sentence based\non a latent code and our model can be used to generate text. Experiments show\nlarge improvements over the related Paragraph Vectors. Compared to uSIF, we\nachieve a relative improvement of 5% when trained on the same data and our\nmethod performs competitively to Sent2vec while trained on 30 times less data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06407v3",
            "date": "2019-07-15 10:07:07+00:00",
            "title": "Investigation on N-gram Approximated RNNLMs for Recognition of Morphologically Rich Speech",
            "authors": [
                "Bal\u00e1zs Tarj\u00e1n",
                "Gy\u00f6rgy Szasz\u00e1k",
                "Tibor Fegy\u00f3",
                "P\u00e9ter Mihajlik"
            ],
            "abstract": "Recognition of Hungarian conversational telephone speech is challenging due\nto the informal style and morphological richness of the language. Recurrent\nNeural Network Language Model (RNNLM) can provide remedy for the high\nperplexity of the task; however, two-pass decoding introduces a considerable\nprocessing delay. In order to eliminate this delay we investigate approaches\naiming at the complexity reduction of RNNLM, while preserving its accuracy. We\ncompare the performance of conventional back-off n-gram language models (BNLM),\nBNLM approximation of RNNLMs (RNN-BNLM) and RNN n-grams in terms of perplexity\nand word error rate (WER). Morphological richness is often addressed by using\nstatistically derived subwords - morphs - in the language models, hence our\ninvestigations are extended to morph-based models, as well. We found that using\nRNN-BNLMs 40% of the RNNLM perplexity reduction can be recovered, which is\nroughly equal to the performance of a RNN 4-gram model. Combining morph-based\nmodeling and approximation of RNNLM, we were able to achieve 8% relative WER\nreduction and preserve real-time operation of our conversational telephone\nspeech recognition system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06585v1",
            "date": "2019-07-08 13:08:12+00:00",
            "title": "Parallelism Theorem and Derived Rules for Parallel Coherent Transformations",
            "authors": [
                "Thierry Boy de la Tour"
            ],
            "abstract": "An Independent Parallelism Theorem is proven in the theory of adhesive HLR\ncategories. It shows the bijective correspondence between sequential\nindependent and parallel independent direct derivations in the Weak\nDouble-Pushout framework, see [2]. The parallel derivations are expressed by\nmeans of Parallel Coherent Transformations (PCTs), hence without assuming the\nexistence of coproducts compatible with M as in the standard Parallelism\nTheorem. It is aslo shown that a derived rule can be extracted from any PCT, in\nthe sense that to any direct derivation of this rule corresponds a valid PCT.",
            "journal ref": "None",
            "category": "math.CT"
        },
        {
            "url": "http://arxiv.org/abs/1907.06848v1",
            "date": "2019-07-16 05:31:35+00:00",
            "title": "Modeling competitive evolution of multiple languages",
            "authors": [
                "Zejie Zhou",
                "Boleslaw K. Szymanski",
                "Jianxi Gao"
            ],
            "abstract": "Increasing evidence demonstrates that in many places language coexistence has\nbecome ubiquitous and essential for supporting language and cultural diversity\nand associated with its financial and economic benefits. The competitive\nevolution among multiple languages determines the evolution outcome, either\ncoexistence, decline, or extinction. Here, we extend the Abrams-Strogatz model\nof language competition to multiple languages and then validate it by analyzing\nthe behavioral transitions of language usage over the recent several decades in\nSingapore and Hong Kong. In each case, we estimate from data the model\nparameters that measure each language utility for its speakers and the strength\nof two biases, the majority preference for their language, and the minority\naversion to it. The values of these two biases decide which language is the\nfastest growing in the competition and what would be the stable state of the\nsystem. We also study the system convergence time to stable states and discover\nthe existence of tipping points with multiple attractors. Moreover, the\ncritical slowdown of convergence to the stable fractions of language users\nappears near and peaks at the tipping points, signaling when the system\napproaches them. Our analysis furthers our understanding of multiple language\nevolution and the role of tipping points in behavioral transitions. These\ninsights may help to protect languages from extinction and retain the language\nand cultural diversity.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1907.06860v1",
            "date": "2019-07-16 06:43:34+00:00",
            "title": "A generic rule-based system for clinical trial patient selection",
            "authors": [
                "Jianlin Shi",
                "Kevin Graves",
                "John F. Hurdle"
            ],
            "abstract": "The n2c2 2018 Challenge task 1 aimed to identify patients who meet lists of\nheterogeneous inclusion/exclusion criteria for a hypothetical clinical trial.\nWe demonstrate a generic rule-based natural language pipeline can support this\ntask with decent performance (the average F1 score on the test set is 0.89,\nranked the 8th out of 45 teams ).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06950v2",
            "date": "2019-07-16 11:52:46+00:00",
            "title": "Abstract categorial grammars with island constraints and effective decidability",
            "authors": [
                "Sergey Slavnov"
            ],
            "abstract": "A well-known approach to treating syntactic island constraints in the setting\nof Lambek grammars consists in adding specific bracket modalities to the logic.\nWe adapt this approach to abstract categorial grammars (ACG). Thus we define\nbracketed (implicational) linear logic, bracketed lambda-calculus, and,\neventually, bracketed ACG based on bracketed $\\lambda$-calculus. This allows us\nmodeling at least simplest island constraints, typically, in the context of\nrelativization. Next we identify specific safely bracketed ACG which, just like\nordinary (bracket-free) second order ACG generate effectively decidable\nlanguages, but are sufficiently flexible to model some higher order phenomena\nlike relativization and correctly deal with syntactic islands, at least in\nsimple toy examples.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1907.07033v1",
            "date": "2019-07-16 14:32:33+00:00",
            "title": "Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection",
            "authors": [
                "Sooji Han",
                "Jie Gao",
                "Fabio Ciravegna"
            ],
            "abstract": "The scarcity and class imbalance of training data are known issues in current\nrumor detection tasks. We propose a straight-forward and general-purpose data\naugmentation technique which is beneficial to early rumor detection relying on\nevent propagation patterns. The key idea is to exploit massive unlabeled event\ndata sets on social media to augment limited labeled rumor source tweets. This\nwork is based on rumor spreading patterns revealed by recent rumor studies and\nsemantic relatedness between labeled and unlabeled data. A state-of-the-art\nneural language model (NLM) and large credibility-focused Twitter corpora are\nemployed to learn context-sensitive representations of rumor tweets. Six\ndifferent real-world events based on three publicly available rumor datasets\nare employed in our experiments to provide a comparative evaluation of the\neffectiveness of the method. The results show that our method can expand the\nsize of an existing rumor data set nearly by 200% and corresponding social\ncontext (i.e., conversational threads) by 100% with reasonable quality.\nPreliminary experiments with a state-of-the-art deep learning-based rumor\ndetection model show that augmented data can alleviate over-fitting and class\nimbalance caused by limited train data and can help to train complex neural\nnetworks (NNs). With augmented data, the performance of rumor detection can be\nimproved by 12.1% in terms of F-score. Our experiments also indicate that\naugmented training data can help to generalize rumor detection models on unseen\nrumors.",
            "journal ref": "The 2019 IEEE/ACM International Conference on Advances in Social\n  Networks Analysis and Mining",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07366v1",
            "date": "2019-07-17 07:37:13+00:00",
            "title": "Leveraging Linguistic Characteristics for Bipolar Disorder Recognition with Gender Differences",
            "authors": [
                "Yen-Hao Huang",
                "Yi-Hsin Chen",
                "Fernando Henrique Calderon Alvarado",
                "Ssu-Rui Lee",
                "Shu-I Wu",
                "Yuwen Lai",
                "Yi-Shin Chen"
            ],
            "abstract": "Most previous studies on automatic recognition model for bipolar disorder\n(BD) were based on both social media and linguistic features. The present study\ninvestigates the possibility of adopting only language-based features, namely\nthe syntax and morpheme collocation. We also examine the effect of gender on\nthe results considering gender has long been recognized as an important\nmodulating factor for mental disorders, yet it received little attention in\nprevious linguistic models. The present study collects Twitter posts 3 months\nprior to the self-disclosure by 349 BD users (231 female, 118 male). We\nconstruct a set of syntactic patterns in terms of the word usage based on graph\npattern construction and pattern attention mechanism. The factors examined are\ngender differences, syntactic patterns, and bipolar recognition performance.\nThe performance indicates our F1 scores reach over 91% and outperform several\nbaselines, including those using TF-IDF, LIWC and pre-trained language models\n(ELMO and BERT). The contributions of the present study are: (1) The features\nare contextualized, domain-agnostic, and purely linguistic. (2) The performance\nof BD recognition is improved by gender-enriched linguistic pattern features,\nwhich are constructed with gender differences in language usage.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.07421v1",
            "date": "2019-07-17 10:03:38+00:00",
            "title": "SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking",
            "authors": [
                "Hwaran Lee",
                "Jinsik Lee",
                "Tae-Yoon Kim"
            ],
            "abstract": "In goal-oriented dialog systems, belief trackers estimate the probability\ndistribution of slot-values at every dialog turn. Previous neural approaches\nhave modeled domain- and slot-dependent belief trackers, and have difficulty in\nadding new slot-values, resulting in lack of flexibility of domain ontology\nconfigurations. In this paper, we propose a new approach to universal and\nscalable belief tracker, called slot-utterance matching belief tracker (SUMBT).\nThe model learns the relations between domain-slot-types and slot-values\nappearing in utterances through attention mechanisms based on contextual\nsemantic vectors. Furthermore, the model predicts slot-value labels in a\nnon-parametric way. From our experiments on two dialog corpora, WOZ 2.0 and\nMultiWOZ, the proposed model showed performance improvement in comparison with\nslot-dependent methods and achieved the state-of-the-art joint accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07638v1",
            "date": "2019-07-17 16:50:16+00:00",
            "title": "Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use",
            "authors": [
                "Janarthanan Rajendran",
                "Jatin Ganhotra",
                "Lazaros Polymenakos"
            ],
            "abstract": "Neural end-to-end goal-oriented dialog systems showed promise to reduce the\nworkload of human agents for customer service, as well as reduce wait time for\nusers. However, their inability to handle new user behavior at deployment has\nlimited their usage in real world. In this work, we propose an end-to-end\ntrainable method for neural goal-oriented dialog systems which handles new user\nbehaviors at deployment by transferring the dialog to a human agent\nintelligently. The proposed method has three goals: 1) maximize user's task\nsuccess by transferring to human agents, 2) minimize the load on the human\nagents by transferring to them only when it is essential and 3) learn online\nfrom the human agent's responses to reduce human agents load further. We\nevaluate our proposed method on a modified-bAbI dialog task that simulates the\nscenario of new user behaviors occurring at test time. Experimental results\nshow that our proposed method is effective in achieving the desired goals.",
            "journal ref": "Transactions of the Association for Computational Linguistics 2019\n  Vol. 7, 375-386",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07653v1",
            "date": "2019-07-17 17:33:13+00:00",
            "title": "Gated Recurrent Neural Network Approach for Multilabel Emotion Detection in Microblogs",
            "authors": [
                "Prabod Rathnayaka",
                "Supun Abeysinghe",
                "Chamod Samarajeewa",
                "Isura Manchanayake",
                "Malaka J. Walpola",
                "Rashmika Nawaratne",
                "Tharindu Bandaragoda",
                "Damminda Alahakoon"
            ],
            "abstract": "People express their opinions and emotions freely in social media posts and\nonline reviews that contain valuable feedback for multiple stakeholders such as\nbusinesses and political campaigns. Manually extracting opinions and emotions\nfrom large volumes of such posts is an impossible task. Therefore, automated\nprocessing of these posts to extract opinions and emotions is an important\nresearch problem. However, human emotion detection is a challenging task due to\nthe complexity and nuanced nature. To overcome these barriers, researchers have\nextensively used techniques such as deep learning, distant supervision, and\ntransfer learning. In this paper, we propose a novel Pyramid Attention Network\n(PAN) based model for emotion detection in microblogs. The main advantage of\nour approach is that PAN has the capability to evaluate sentences in different\nperspectives to capture multiple emotions existing in a single text. The\nproposed model was evaluated on a recently released dataset and the results\nachieved the state-of-the-art accuracy of 58.9%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07818v1",
            "date": "2019-07-17 23:57:46+00:00",
            "title": "Decoding the Style and Bias of Song Lyrics",
            "authors": [
                "Manash Pratim Barman",
                "Amit Awekar",
                "Sambhav Kothari"
            ],
            "abstract": "The central idea of this paper is to gain a deeper understanding of song\nlyrics computationally. We focus on two aspects: style and biases of song\nlyrics. All prior works to understand these two aspects are limited to manual\nanalysis of a small corpus of song lyrics. In contrast, we analyzed more than\nhalf a million songs spread over five decades. We characterize the lyrics style\nin terms of vocabulary, length, repetitiveness, speed, and readability. We have\nobserved that the style of popular songs significantly differs from other\nsongs. We have used distributed representation methods and WEAT test to measure\nvarious gender and racial biases in the song lyrics. We have observed that\nbiases in song lyrics correlate with prior results on human subjects. This\ncorrelation indicates that song lyrics reflect the biases that exist in\nsociety. Increasing consumption of music and the effect of lyrics on human\nemotions makes this analysis important.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.07826v1",
            "date": "2019-07-18 01:00:42+00:00",
            "title": "Comparison of Classical Machine Learning Approaches on Bangla Textual Emotion Analysis",
            "authors": [
                "Md. Ataur Rahman",
                "Md. Hanif Seddiqui"
            ],
            "abstract": "Detecting emotions from text is an extension of simple sentiment polarity\ndetection. Instead of considering only positive or negative sentiments,\nemotions are conveyed using more tangible manner; thus, they can be expressed\nas many shades of gray. This paper manifests the results of our experimentation\nfor fine-grained emotion analysis on Bangla text. We gathered and annotated a\ntext corpus consisting of user comments from several Facebook groups regarding\nsocio-economic and political issues, and we made efforts to extract the basic\nemotions (sadness, happiness, disgust, surprise, fear, anger) conveyed through\nthese comments. Finally, we compared the results of the five most popular\nclassical machine learning techniques namely Naive Bayes, Decision Tree,\nk-Nearest Neighbor (k-NN), Support Vector Machine (SVM) and K-Means Clustering\nwith several combinations of features. Our best model (SVM with a non-linear\nradial-basis function (RBF) kernel) achieved an overall average accuracy score\nof 52.98% and an F1 score (macro) of 0.3324",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08015v2",
            "date": "2019-07-18 12:39:12+00:00",
            "title": "ELG: An Event Logic Graph",
            "authors": [
                "Xiao Ding",
                "Zhongyang Li",
                "Ting Liu",
                "Kuo Liao"
            ],
            "abstract": "The evolution and development of events have their own basic principles,\nwhich make events happen sequentially. Therefore, the discovery of such\nevolutionary patterns among events are of great value for event prediction,\ndecision-making and scenario design of dialog systems. However, conventional\nknowledge graph mainly focuses on the entities and their relations, which\nneglects the real world events. In this paper, we present a novel type of\nknowledge base - Event Logic Graph (ELG), which can reveal evolutionary\npatterns and development logics of real world events. Specifically, ELG is a\ndirected cyclic graph, whose nodes are events, and edges stand for the\nsequential, causal, conditional or hypernym-hyponym (is-a) relations between\nevents. We constructed two domain ELG: financial domain ELG, which consists of\nmore than 1.5 million of event nodes and more than 1.8 million of directed\nedges, and travel domain ELG, which consists of about 30 thousand of event\nnodes and more than 234 thousand of directed edges. Experimental results show\nthat ELG is effective for the task of script event prediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.08167v1",
            "date": "2019-07-18 17:19:30+00:00",
            "title": "OCC: A Smart Reply System for Efficient In-App Communications",
            "authors": [
                "Yue Weng",
                "Huaixiu Zheng",
                "Franziska Bell",
                "Gokhan Tur"
            ],
            "abstract": "Smart reply systems have been developed for various messaging platforms. In\nthis paper, we introduce Uber's smart reply system: one-click-chat (OCC), which\nis a key enhanced feature on top of the Uber in-app chat system. It enables\ndriver-partners to quickly respond to rider messages using smart replies. The\nsmart replies are dynamically selected according to conversation content using\nmachine learning algorithms. Our system consists of two major components:\nintent detection and reply retrieval, which are very different from standard\nsmart reply systems where the task is to directly predict a reply. It is\ndesigned specifically for mobile applications with short and non-canonical\nmessages. Reply retrieval utilizes pairings between intent and reply based on\ntheir popularity in chat messages as derived from historical data. For intent\ndetection, a set of embedding and classification techniques are experimented\nwith, and we choose to deploy a solution using unsupervised distributed\nembedding and nearest-neighbor classifier. It has the advantage of only\nrequiring a small amount of labeled training data, simplicity in developing and\ndeploying to production, and fast inference during serving and hence highly\nscalable. At the same time, it performs comparably with deep learning\narchitectures such as word-level convolutional neural network. Overall, the\nsystem achieves a high accuracy of 76% on intent detection. Currently, the\nsystem is deployed in production for English-speaking countries and 71% of\nin-app communications between riders and driver-partners adopted the smart\nreplies to speedup the communication process.",
            "journal ref": "KDD 19, August 4-8, 2019, Anchorage, AK, USA",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08176v1",
            "date": "2019-07-18 17:37:13+00:00",
            "title": "Querying Knowledge via Multi-Hop English Questions",
            "authors": [
                "Tiantian Gao",
                "Paul Fodor",
                "Michael Kifer"
            ],
            "abstract": "The inherent difficulty of knowledge specification and the lack of trained\nspecialists are some of the key obstacles on the way to making intelligent\nsystems based on the knowledge representation and reasoning (KRR) paradigm\ncommonplace. Knowledge and query authoring using natural language, especially\ncontrolled natural language (CNL), is one of the promising approaches that\ncould enable domain experts, who are not trained logicians, to both create\nformal knowledge and query it. In previous work, we introduced the KALM system\n(Knowledge Authoring Logic Machine) that supports knowledge authoring (and\nsimple querying) with very high accuracy that at present is unachievable via\nmachine learning approaches. The present paper expands on the question\nanswering aspect of KALM and introduces KALM-QA (KALM for Question Answering)\nthat is capable of answering much more complex English questions. We show that\nKALM-QA achieves 100% accuracy on an extensive suite of movie-related\nquestions, called MetaQA, which contains almost 29,000 test questions and over\n260,000 training questions. We contrast this with a published machine learning\napproach, which falls far short of this high mark.",
            "journal ref": "Theory and Practice of Logic Programming 19 (2019) 636-653",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08326v1",
            "date": "2019-07-19 00:40:28+00:00",
            "title": "I Stand With You: Using Emojis to Study Solidarity in Crisis Events",
            "authors": [
                "Sashank Santhanam",
                "Vidhushini Srinivasan",
                "Shaina Glass",
                "Samira Shaikh"
            ],
            "abstract": "We study how emojis are used to express solidarity in social media in the\ncontext of two major crisis events - a natural disaster, Hurricane Irma in 2017\nand terrorist attacks that occurred on November 2015 in Paris. Using annotated\ncorpora, we first train a recurrent neural network model to classify\nexpressions of solidarity in text. Next, we use these expressions of solidarity\nto characterize human behavior in online social networks, through the temporal\nand geospatial diffusion of emojis. Our analysis reveals that emojis are a\npowerful indicator of sociolinguistic behaviors (solidarity) that are exhibited\non social media as the crisis events unfold.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.08501v1",
            "date": "2019-07-19 13:09:32+00:00",
            "title": "A Comparative Evaluation of Visual and Natural Language Question Answering Over Linked Data",
            "authors": [
                "Gerhard Wohlgenannt",
                "Dmitry Mouromtsev",
                "Dmitry Pavlov",
                "Yury Emelyanov",
                "Alexey Morozov"
            ],
            "abstract": "With the growing number and size of Linked Data datasets, it is crucial to\nmake the data accessible and useful for users without knowledge of formal query\nlanguages. Two approaches towards this goal are knowledge graph visualization\nand natural language interfaces. Here, we investigate specifically question\nanswering (QA) over Linked Data by comparing a diagrammatic visual approach\nwith existing natural language-based systems. Given a QA benchmark (QALD7), we\nevaluate a visual method which is based on iteratively creating diagrams until\nthe answer is found, against four QA systems that have natural language queries\nas input. Besides other benefits, the visual approach provides higher\nperformance, but also requires more manual input. The results indicate that the\nmethods can be used complementary, and that such a combination has a large\npositive impact on QA performance, and also facilitates additional features\nsuch as data exploration.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.08532v1",
            "date": "2019-07-19 14:43:51+00:00",
            "title": "Multi-Granular Text Encoding for Self-Explaining Categorization",
            "authors": [
                "Zhiguo Wang",
                "Yue Zhang",
                "Mo Yu",
                "Wei Zhang",
                "Lin Pan",
                "Linfeng Song",
                "Kun Xu",
                "Yousef El-Kurdi"
            ],
            "abstract": "Self-explaining text categorization requires a classifier to make a\nprediction along with supporting evidence. A popular type of evidence is\nsub-sequences extracted from the input text which are sufficient for the\nclassifier to make the prediction. In this work, we define multi-granular\nngrams as basic units for explanation, and organize all ngrams into a\nhierarchical structure, so that shorter ngrams can be reused while computing\nlonger ngrams. We leverage a tree-structured LSTM to learn a\ncontext-independent representation for each unit via parameter sharing.\nExperiments on medical disease classification show that our model is more\naccurate, efficient and compact than BiLSTM and CNN baselines. More\nimportantly, our model can extract intuitive multi-granular evidence to support\nits predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08710v3",
            "date": "2019-07-19 22:20:01+00:00",
            "title": "Structure-Invariant Testing for Machine Translation",
            "authors": [
                "Pinjia He",
                "Clara Meister",
                "Zhendong Su"
            ],
            "abstract": "In recent years, machine translation software has increasingly been\nintegrated into our daily lives. People routinely use machine translation for\nvarious applications, such as describing symptoms to a foreign doctor and\nreading political news in a foreign language. However, the complexity and\nintractability of neural machine translation (NMT) models that power modern\nmachine translation make the robustness of these systems difficult to even\nassess, much less guarantee. Machine translation systems can return inferior\nresults that lead to misunderstanding, medical misdiagnoses, threats to\npersonal safety, or political conflicts. Despite its apparent importance,\nvalidating the robustness of machine translation systems is very difficult and\nhas, therefore, been much under-explored.\n  To tackle this challenge, we introduce structure-invariant testing (SIT), a\nnovel metamorphic testing approach for validating machine translation software.\nOur key insight is that the translation results of \"similar\" source sentences\nshould typically exhibit similar sentence structures. Specifically, SIT (1)\ngenerates similar source sentences by substituting one word in a given sentence\nwith semantically similar, syntactically equivalent words; (2) represents\nsentence structure by syntax parse trees (obtained via constituency or\ndependency parsing); (3) reports sentence pairs whose structures differ\nquantitatively by more than some threshold. To evaluate SIT, we use it to test\nGoogle Translate and Bing Microsoft Translator with 200 source sentences as\ninput, which led to 64 and 70 buggy issues with 69.5\\% and 70\\% top-1 accuracy,\nrespectively. The translation errors are diverse, including under-translation,\nover-translation, incorrect modification, word/phrase mistranslation, and\nunclear logic.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1907.09038v1",
            "date": "2019-07-21 21:27:44+00:00",
            "title": "Augmenting a BiLSTM tagger with a Morphological Lexicon and a Lexical Category Identification Step",
            "authors": [
                "Stein\u00fe\u00f3r Steingr\u00edmsson",
                "\u00d6rvar K\u00e1rason",
                "Hrafn Loftsson"
            ],
            "abstract": "Previous work on using BiLSTM models for PoS tagging has primarily focused on\nsmall tagsets. We evaluate BiLSTM models for tagging Icelandic, a\nmorphologically rich language, using a relatively large tagset. Our baseline\nBiLSTM model achieves higher accuracy than any previously published tagger not\ntaking advantage of a morphological lexicon. When we extend the model by\nincorporating such data, we outperform previous state-of-the-art results by a\nsignificant margin. We also report on work in progress that attempts to address\nthe problem of data sparsity inherent in morphologically detailed, fine-grained\ntagsets. We experiment with training a separate model on only the lexical\ncategory and using the coarse-grained output tag as an input for the main\nmodel. This method further increases the accuracy and reduces the tagging\nerrors by 21.3% compared to previous state-of-the-art results. Finally, we\ntrain and test our tagger on a new gold standard for Icelandic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09169v1",
            "date": "2019-07-22 07:44:09+00:00",
            "title": "Learning dynamic word embeddings with drift regularisation",
            "authors": [
                "Syrielle Montariol",
                "Alexandre Allauzen"
            ],
            "abstract": "Word usage, meaning and connotation change throughout time. Diachronic word\nembeddings are used to grasp these changes in an unsupervised way. In this\npaper, we use variants of the Dynamic Bernoulli Embeddings model to learn\ndynamic word embeddings, in order to identify notable properties of the model.\nThe comparison is made on the New York Times Annotated Corpus in English and a\nset of articles from the French newspaper Le Monde covering the same period.\nThis allows us to define a pipeline to analyse the evolution of words use\nacross two languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09273v2",
            "date": "2019-07-22 12:32:15+00:00",
            "title": "Why Build an Assistant in Minecraft?",
            "authors": [
                "Arthur Szlam",
                "Jonathan Gray",
                "Kavya Srinet",
                "Yacine Jernite",
                "Armand Joulin",
                "Gabriel Synnaeve",
                "Douwe Kiela",
                "Haonan Yu",
                "Zhuoyuan Chen",
                "Siddharth Goyal",
                "Demi Guo",
                "Danielle Rothermel",
                "C. Lawrence Zitnick",
                "Jason Weston"
            ],
            "abstract": "In this document we describe a rationale for a research program aimed at\nbuilding an open \"assistant\" in the game Minecraft, in order to make progress\non the problems of natural language understanding and learning from dialogue.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.10036v1",
            "date": "2019-07-23 17:46:02+00:00",
            "title": "Happiness Entailment: Automating Suggestions for Well-Being",
            "authors": [
                "Sara Evensen",
                "Yoshihiko Suhara",
                "Alon Halevy",
                "Vivian Li",
                "Wang-Chiew Tan",
                "Saran Mumick"
            ],
            "abstract": "Understanding what makes people happy is a central topic in psychology. Prior\nwork has mostly focused on developing self-reporting assessment tools for\nindividuals and relies on experts to analyze the periodic reported assessments.\nOne of the goals of the analysis is to understand what actions are necessary to\nencourage modifications in the behaviors of the individuals to improve their\noverall well-being. In this paper, we outline a complementary approach; on the\nassumption that the user journals her happy moments as short texts, a system\ncan analyze these texts and propose sustainable suggestions for the user that\nmay lead to an overall improvement in her well-being. We prototype one\nnecessary component of such a system, the Happiness Entailment Recognition\n(HER) module, which takes as input a short text describing an event, a\ncandidate suggestion, and outputs a determination about whether the suggestion\nis more likely to be good for this user based on the event described. This\ncomponent is implemented as a neural network model with two encoders, one for\nthe user input and one for the candidate actionable suggestion, with additional\nlayers to capture psychologically significant features in the happy moment and\nsuggestion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10529v3",
            "date": "2019-07-24 15:43:40+00:00",
            "title": "SpanBERT: Improving Pre-training by Representing and Predicting Spans",
            "authors": [
                "Mandar Joshi",
                "Danqi Chen",
                "Yinhan Liu",
                "Daniel S. Weld",
                "Luke Zettlemoyer",
                "Omer Levy"
            ],
            "abstract": "We present SpanBERT, a pre-training method that is designed to better\nrepresent and predict spans of text. Our approach extends BERT by (1) masking\ncontiguous random spans, rather than random tokens, and (2) training the span\nboundary representations to predict the entire content of the masked span,\nwithout relying on the individual token representations within it. SpanBERT\nconsistently outperforms BERT and our better-tuned baselines, with substantial\ngains on span selection tasks such as question answering and coreference\nresolution. In particular, with the same training data and model size as\nBERT-large, our single model obtains 94.6% and 88.7% F1 on SQuAD 1.1 and 2.0,\nrespectively. We also achieve a new state of the art on the OntoNotes\ncoreference resolution task (79.6\\% F1), strong performance on the TACRED\nrelation extraction benchmark, and even show gains on GLUE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10658v1",
            "date": "2019-07-22 21:16:58+00:00",
            "title": "SlugBot: Developing a Computational Model andFramework of a Novel Dialogue Genre",
            "authors": [
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Wen Cui",
                "Juraj Juraska",
                "Vrindavan Harrison",
                "Brian Schwarzmann",
                "Nick Santer",
                "Marilyn Walker"
            ],
            "abstract": "One of the most interesting aspects of the Amazon Alexa Prize competition is\nthat the framing of the competition requires the development of new\ncomputational models of dialogue and its structure. Traditional computational\nmodels of dialogue are of two types: (1) task-oriented dialogue, supported by\nAI planning models,or simplified planning models consisting of frames with\nslots to be filled; or (2)search-oriented dialogue where every user turn is\ntreated as a search query that may elaborate and extend current search results.\nAlexa Prize dialogue systems such as SlugBot must support conversational\ncapabilities that go beyond what these traditional models can do. Moreover,\nwhile traditional dialogue systems rely on theoretical computational models,\nthere are no existing computational theories that circumscribe the expected\nsystem and user behaviors in the intended conversational genre of the Alexa\nPrize Bots. This paper describes how UCSC's SlugBot team has combined the\ndevelopment of a novel computational theoretical model, Discourse Relation\nDialogue Model, with its implementation in a modular system in order to test\nand refine it. We highlight how our novel dialogue model has led us to create a\nnovel ontological resource, UniSlug, and how the structure of UniSlug determine\nshow we curate and structure content so that our dialogue manager implements\nand tests our novel computational dialogue model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10710v1",
            "date": "2019-07-24 20:40:19+00:00",
            "title": "Generic Intent Representation in Web Search",
            "authors": [
                "Hongfei Zhang",
                "Xia Song",
                "Chenyan Xiong",
                "Corby Rosset",
                "Paul N. Bennett",
                "Nick Craswell",
                "Saurabh Tiwary"
            ],
            "abstract": "This paper presents GEneric iNtent Encoder (GEN Encoder) which learns a\ndistributed representation space for user intent in search. Leveraging large\nscale user clicks from Bing search logs as weak supervision of user intent, GEN\nEncoder learns to map queries with shared clicks into similar embeddings\nend-to-end and then finetunes on multiple paraphrase tasks. Experimental\nresults on an intrinsic evaluation task - query intent similarity modeling -\ndemonstrate GEN Encoder's robust and significant advantages over previous\nrepresentation methods. Ablation studies reveal the crucial role of learning\nfrom implicit user feedback in representing user intent and the contributions\nof multi-task learning in representation generality. We also demonstrate that\nGEN Encoder alleviates the sparsity of tail search traffic and cuts down half\nof the unseen queries by using an efficient approximate nearest neighbor search\nto effectively identify previous queries with the same search intent. Finally,\nwe demonstrate distances between GEN encodings reflect certain information\nseeking behaviors in search sessions.",
            "journal ref": "SIGIR 2019: Proceedings of the 42nd International ACM SIGIR\n  Conference on Research and Development in Information Retrieval",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.10781v1",
            "date": "2019-07-25 01:01:27+00:00",
            "title": "INS: An Interactive Chinese News Synthesis System",
            "authors": [
                "Hui Liu",
                "Wentao Qin",
                "Xiaojun Wan"
            ],
            "abstract": "Nowadays, we are surrounded by more and more online news articles. Tens or\nhundreds of news articles need to be read if we wish to explore a hot news\nevent or topic. So it is of vital importance to automatically synthesize a\nbatch of news articles related to the event or topic into a new synthesis\narticle (or overview article) for reader's convenience. It is so challenging to\nmake news synthesis fully automatic that there is no successful solution by\nnow. In this paper, we put forward a novel Interactive News Synthesis system\n(i.e. INS), which can help generate news overview articles automatically or by\ninteracting with users. More importantly, INS can serve as a tool for editors\nto help them finish their jobs. In our experiments, INS performs well on both\ntopic representation and synthesis article generation. A user study also\ndemonstrates the usefulness and users' satisfaction with the INS tool. A demo\nvideo is available at \\url{https://youtu.be/7ItteKW3GEk}.",
            "journal ref": "Proceedings of NAACL-HLT 2019: Demonstrations, pages 18-23",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10885v2",
            "date": "2019-07-25 07:58:08+00:00",
            "title": "Adaptive Noise Injection: A Structure-Expanding Regularization for RNN",
            "authors": [
                "Rui Li",
                "Kai Shuang",
                "Mengyu Gu",
                "Sen Su"
            ],
            "abstract": "The vanilla LSTM has become one of the most potential architectures in\nword-level language modeling, like other recurrent neural networks, overfitting\nis always a key barrier for its effectiveness. The existing noise-injected\nregularizations introduce the random noises of fixation intensity, which\ninhibits the learning of the RNN throughout the training process. In this\npaper, we propose a new structure-expanding regularization method called\nAdjective Noise Injection (ANI), which considers the output of an extra RNN\nbranch as a kind of adaptive noises and injects it into the main-branch RNN\noutput. Due to the adaptive noises can be improved as the training processes,\nits negative effects can be weakened and even transformed into a positive\neffect to further improve the expressiveness of the main-branch RNN. As a\nresult, ANI can regularize the RNN in the early stage of training and further\npromoting its training performance in the later stage. We conduct experiments\non three widely-used corpora: PTB, WT2, and WT103, whose results verify both\nthe regularization and promoting the training performance functions of ANI.\nFurthermore, we design a series simulation experiments to explore the reasons\nthat may lead to the regularization effect of ANI, and we find that in training\nprocess, the robustness against the parameter update errors can be strengthened\nwhen the LSTM equipped with ANI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11112v1",
            "date": "2019-07-25 14:45:04+00:00",
            "title": "Using Answer Set Programming for Commonsense Reasoning in the Winograd Schema Challenge",
            "authors": [
                "Arpit Sharma"
            ],
            "abstract": "The Winograd Schema Challenge (WSC) is a natural language understanding task\nproposed as an alternative to the Turing test in 2011. In this work we attempt\nto solve WSC problems by reasoning with additional knowledge. By using an\napproach built on top of graph-subgraph isomorphism encoded using Answer Set\nProgramming (ASP) we were able to handle 240 out of 291 WSC problems. The ASP\nencoding allows us to add additional constraints in an elaboration tolerant\nmanner. In the process we present a graph based representation of WSC problems\nas well as relevant commonsense knowledge. This paper is under consideration\nfor acceptance in TPLP.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.11656v1",
            "date": "2019-07-26 16:07:20+00:00",
            "title": "Vocal Interactivity in Crowds, Flocks and Swarms: Implications for Voice User Interfaces",
            "authors": [
                "Roger K. Moore"
            ],
            "abstract": "Recent years have seen an explosion in the availability of Voice User\nInterfaces. However, user surveys suggest that there are issues with respect to\nusability, and it has been hypothesised that contemporary voice-enabled systems\nare missing crucial behaviours relating to user engagement and vocal\ninteractivity. However, it is well established that such ostensive behaviours\nare ubiquitous in the animal kingdom, and that vocalisation provides a means\nthrough which interaction may be coordinated and managed between individuals\nand within groups. Hence, this paper reports results from a study aimed at\nidentifying generic mechanisms that might underpin coordinated collective vocal\nbehaviour with a particular focus on closed-loop negative-feedback control as a\npowerful regulatory process. A computer-based real-time simulation of vocal\ninteractivity is described which has provided a number of insights, including\nthe enumeration of a number of key control variables that may be worthy of\nfurther investigation.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1907.12133v1",
            "date": "2019-07-28 19:59:20+00:00",
            "title": "An Empirical Study on Leveraging Scene Graphs for Visual Question Answering",
            "authors": [
                "Cheng Zhang",
                "Wei-Lun Chao",
                "Dong Xuan"
            ],
            "abstract": "Visual question answering (Visual QA) has attracted significant attention\nthese years. While a variety of algorithms have been proposed, most of them are\nbuilt upon different combinations of image and language features as well as\nmulti-modal attention and fusion. In this paper, we investigate an alternative\napproach inspired by conventional QA systems that operate on knowledge graphs.\nSpecifically, we investigate the use of scene graphs derived from images for\nVisual QA: an image is abstractly represented by a graph with nodes\ncorresponding to object entities and edges to object relationships. We adapt\nthe recently proposed graph network (GN) to encode the scene graph and perform\nstructured reasoning according to the input question. Our empirical studies\ndemonstrate that scene graphs can already capture essential information of\nimages and graph networks have the potential to outperform state-of-the-art\nVisual QA algorithms but with a much cleaner architecture. By analyzing the\nfeatures generated by GNs we can further interpret the reasoning process,\nsuggesting a promising direction towards explainable Visual QA.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.12413v1",
            "date": "2019-07-29 13:26:03+00:00",
            "title": "VIANA: Visual Interactive Annotation of Argumentation",
            "authors": [
                "Fabian Sperrle",
                "Rita Sevastjanova",
                "Rebecca Kehlbeck",
                "Mennatallah El-Assady"
            ],
            "abstract": "Argumentation Mining addresses the challenging tasks of identifying\nboundaries of argumentative text fragments and extracting their relationships.\nFully automated solutions do not reach satisfactory accuracy due to their\ninsufficient incorporation of semantics and domain knowledge. Therefore,\nexperts currently rely on time-consuming manual annotations. In this paper, we\npresent a visual analytics system that augments the manual annotation process\nby automatically suggesting which text fragments to annotate next. The accuracy\nof those suggestions is improved over time by incorporating linguistic\nknowledge and language modeling to learn a measure of argument similarity from\nuser interactions. Based on a long-term collaboration with domain experts, we\nidentify and model five high-level analysis tasks. We enable close reading and\nnote-taking, annotation of arguments, argument reconstruction, extraction of\nargument relations, and exploration of argument graphs. To avoid context\nswitches, we transition between all views through seamless morphing, visually\nanchoring all text- and graph-based layers. We evaluate our system with a\ntwo-stage expert user study based on a corpus of presidential debates. The\nresults show that experts prefer our system over existing solutions due to the\nspeedup provided by the automatic suggestions and the tight integration between\ntext and graph views.",
            "journal ref": "2019 IEEE Conference on Visual Analytics Science and Technology\n  (VAST)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12484v3",
            "date": "2019-07-29 15:35:13+00:00",
            "title": "Joey NMT: A Minimalist NMT Toolkit for Novices",
            "authors": [
                "Julia Kreutzer",
                "Jasmijn Bastings",
                "Stefan Riezler"
            ],
            "abstract": "We present Joey NMT, a minimalist neural machine translation toolkit based on\nPyTorch that is specifically designed for novices. Joey NMT provides many\npopular NMT features in a small and simple code base, so that novices can\neasily and quickly learn to use it and adapt it to their needs. Despite its\nfocus on simplicity, Joey NMT supports classic architectures (RNNs,\ntransformers), fast beam search, weight tying, and more, and achieves\nperformance comparable to more complex toolkits on standard benchmarks. We\nevaluate the accessibility of our toolkit in a user study where novices with\ngeneral knowledge about Pytorch and NMT and experts work through a\nself-contained Joey NMT tutorial, showing that novices perform almost as well\nas experts in a subsequent code quiz. Joey NMT is available at\nhttps://github.com/joeynmt/joeynmt .",
            "journal ref": "EMNLP-IJCNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12667v1",
            "date": "2019-07-29 21:56:35+00:00",
            "title": "Reinforced Dynamic Reasoning for Conversational Question Generation",
            "authors": [
                "Boyuan Pan",
                "Hao Li",
                "Ziyu Yao",
                "Deng Cai",
                "Huan Sun"
            ],
            "abstract": "This paper investigates a new task named Conversational Question Generation\n(CQG) which is to generate a question based on a passage and a conversation\nhistory (i.e., previous turns of question-answer pairs). CQG is a crucial task\nfor developing intelligent agents that can drive question-answering style\nconversations or test user understanding of a given passage. Towards that end,\nwe propose a new approach named Reinforced Dynamic Reasoning (ReDR) network,\nwhich is based on the general encoder-decoder framework but incorporates a\nreasoning procedure in a dynamic manner to better understand what has been\nasked and what to ask next about the passage. To encourage producing meaningful\nquestions, we leverage a popular question answering (QA) model to provide\nfeedback and fine-tune the question generator using a reinforcement learning\nmechanism. Empirical results on the recently released CoQA dataset demonstrate\nthe effectiveness of our method in comparison with various baselines and model\nvariants. Moreover, to show the applicability of our method, we also apply it\nto create multi-turn question-answering conversations for passages in SQuAD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12763v1",
            "date": "2019-07-30 07:31:02+00:00",
            "title": "Temporal Localization of Moments in Video Collections with Natural Language",
            "authors": [
                "Victor Escorcia",
                "Mattia Soldan",
                "Josef Sivic",
                "Bernard Ghanem",
                "Bryan Russell"
            ],
            "abstract": "In this paper, we introduce the task of retrieving relevant video moments\nfrom a large corpus of untrimmed, unsegmented videos given a natural language\nquery. Our task poses unique challenges as a system must efficiently identify\nboth the relevant videos and localize the relevant moments in the videos. This\ntask is in contrast to prior work that localizes relevant moments in a single\nvideo or searches a large collection of already-segmented videos. For our task,\nwe introduce Clip Alignment with Language (CAL), a model that aligns features\nfor a natural language query to a sequence of short video clips that compose a\ncandidate moment in a video. Our approach goes beyond prior work that\naggregates video features over a candidate moment by allowing for finer clip\nalignment. Moreover, our approach is amenable to efficient indexing of the\nresulting clip-level representations, which makes it suitable for moment\nlocalization in large video collections. We evaluate our approach on three\nrecently proposed datasets for temporal localization of moments in video with\nnatural language extended to our video corpus moment retrieval setting: DiDeMo,\nCharades-STA, and ActivityNet-captions. We show that our CAL model outperforms\nthe recently proposed Moment Context Network (MCN) on all criteria across all\ndatasets on our proposed task, obtaining an 8%-85% and 11%-47% boost for\naverage recall and median rank, respectively, and achieves 5x faster retrieval\nand 8x smaller index size with a 500K video corpus.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.12894v1",
            "date": "2019-07-30 13:31:07+00:00",
            "title": "Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation",
            "authors": [
                "Yang Gao",
                "Christian M. Meyer",
                "Mohsen Mesgar",
                "Iryna Gurevych"
            ],
            "abstract": "Document summarisation can be formulated as a sequential decision-making\nproblem, which can be solved by Reinforcement Learning (RL) algorithms. The\npredominant RL paradigm for summarisation learns a cross-input policy, which\nrequires considerable time, data and parameter tuning due to the huge search\nspaces and the delayed rewards. Learning input-specific RL policies is a more\nefficient alternative but so far depends on handcrafted rewards, which are\ndifficult to design and yield poor performance. We propose RELIS, a novel RL\nparadigm that learns a reward function with Learning-to-Rank (L2R) algorithms\nat training time and uses this reward function to train an input-specific RL\npolicy at test time. We prove that RELIS guarantees to generate near-optimal\nsummaries with appropriate L2R and RL algorithms. Empirically, we evaluate our\napproach on extractive multi-document summarisation. We show that RELIS reduces\nthe training time by two orders of magnitude compared to the state-of-the-art\nmodels while performing on par with them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13237v1",
            "date": "2019-07-30 21:38:16+00:00",
            "title": "SenseFitting: Sense Level Semantic Specialization of Word Embeddings for Word Sense Disambiguation",
            "authors": [
                "Manuel Stoeckel",
                "Sajawel Ahmed",
                "Alexander Mehler"
            ],
            "abstract": "We introduce a neural network-based system of Word Sense Disambiguation (WSD)\nfor German that is based on SenseFitting, a novel method for optimizing WSD. We\noutperform knowledge-based WSD methods by up to 25% F1-score and produce a new\nstate-of-the-art on the German sense-annotated dataset WebCAGe. Our method uses\nthree feature vectors consisting of a) sense, b) gloss, and c) relational\nvectors to represent target senses and to compare them with the vector\ncentroids of sample contexts. Utilizing widely available word embeddings and\nlexical resources, we are able to compensate for the lower resource\navailability of German. SenseFitting builds upon the recently introduced\nsemantic specialization procedure Attract-Repel, and leverages sense level\nsemantic constraints from lexical-semantic networks (e.g. GermaNet) or online\nsocial dictionaries (e.g. Wiktionary) to produce high-quality sense embeddings\nfrom pre-trained word embeddings. We evaluate our sense embeddings with a new\nSimLex-999 based similarity dataset, called SimSense, that we developed for\nthis work. We achieve results that outperform current lemma-based\nspecialization methods for German, making them comparable to results achieved\nfor English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13337v1",
            "date": "2019-07-31 07:11:59+00:00",
            "title": "Simple Unsupervised Summarization by Contextual Matching",
            "authors": [
                "Jiawei Zhou",
                "Alexander M. Rush"
            ],
            "abstract": "We propose an unsupervised method for sentence summarization using only\nlanguage modeling. The approach employs two language models, one that is\ngeneric (i.e. pretrained), and the other that is specific to the target domain.\nWe show that by using a product-of-experts criteria these are enough for\nmaintaining continuous contextual matching while maintaining output fluency.\nExperiments on both abstractive and extractive sentence summarization data sets\nshow promising results of our method without being exposed to any paired data.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13528v2",
            "date": "2019-07-31 14:37:32+00:00",
            "title": "What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models",
            "authors": [
                "Allyson Ettinger"
            ],
            "abstract": "Pre-training by language modeling has become a popular and successful\napproach to NLP tasks, but we have yet to understand exactly what linguistic\ncapacities these pre-training processes confer upon models. In this paper we\nintroduce a suite of diagnostics drawn from human language experiments, which\nallow us to ask targeted questions about the information used by language\nmodels for generating predictions in context. As a case study, we apply these\ndiagnostics to the popular BERT model, finding that it can generally\ndistinguish good from bad completions involving shared category or role\nreversal, albeit with less sensitivity than humans, and it robustly retrieves\nnoun hypernyms, but it struggles with challenging inferences and role-based\nevent prediction -- and in particular, it shows clear insensitivity to the\ncontextual impacts of negation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00153v2",
            "date": "2019-08-01 00:28:57+00:00",
            "title": "Hateful People or Hateful Bots? Detection and Characterization of Bots Spreading Religious Hatred in Arabic Social Media",
            "authors": [
                "Nuha Albadi",
                "Maram Kurdi",
                "Shivakant Mishra"
            ],
            "abstract": "Arabic Twitter space is crawling with bots that fuel political feuds, spread\nmisinformation, and proliferate sectarian rhetoric. While efforts have long\nexisted to analyze and detect English bots, Arabic bot detection and\ncharacterization remains largely understudied. In this work, we contribute new\ninsights into the role of bots in spreading religious hatred on Arabic Twitter\nand introduce a novel regression model that can accurately identify Arabic\nlanguage bots. Our assessment shows that existing tools that are highly\naccurate in detecting English bots don't perform as well on Arabic bots. We\nidentify the possible reasons for this poor performance, perform a thorough\nanalysis of linguistic, content, behavioral and network features, and report on\nthe most informative features that distinguish Arabic bots from humans as well\nas the differences between Arabic and English bots. Our results mark an\nimportant step toward understanding the behavior of malicious bots on Arabic\nTwitter and pave the way for a more effective Arabic bot detection tools.",
            "journal ref": "Proc. ACM Hum.-Comput. Interact. 3, CSCW: Article 61 (2019)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.00249v1",
            "date": "2019-08-01 07:58:50+00:00",
            "title": "Convolutional Auto-encoding of Sentence Topics for Image Paragraph Generation",
            "authors": [
                "Jing Wang",
                "Yingwei Pan",
                "Ting Yao",
                "Jinhui Tang",
                "Tao Mei"
            ],
            "abstract": "Image paragraph generation is the task of producing a coherent story (usually\na paragraph) that describes the visual content of an image. The problem\nnevertheless is not trivial especially when there are multiple descriptive and\ndiverse gists to be considered for paragraph generation, which often happens in\nreal images. A valid question is how to encapsulate such gists/topics that are\nworthy of mention from an image, and then describe the image from one topic to\nanother but holistically with a coherent structure. In this paper, we present a\nnew design --- Convolutional Auto-Encoding (CAE) that purely employs\nconvolutional and deconvolutional auto-encoding framework for topic modeling on\nthe region-level features of an image. Furthermore, we propose an architecture,\nnamely CAE plus Long Short-Term Memory (dubbed as CAE-LSTM), that novelly\nintegrates the learnt topics in support of paragraph generation. Technically,\nCAE-LSTM capitalizes on a two-level LSTM-based paragraph generation framework\nwith attention mechanism. The paragraph-level LSTM captures the inter-sentence\ndependency in a paragraph, while sentence-level LSTM is to generate one\nsentence which is conditioned on each learnt topic. Extensive experiments are\nconducted on Stanford image paragraph dataset, and superior results are\nreported when comparing to state-of-the-art approaches. More remarkably,\nCAE-LSTM increases CIDEr performance from 20.93% to 25.15%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.00300v1",
            "date": "2019-08-01 10:07:07+00:00",
            "title": "Simple and Effective Text Matching with Richer Alignment Features",
            "authors": [
                "Runqi Yang",
                "Jianhai Zhang",
                "Xing Gao",
                "Feng Ji",
                "Haiqing Chen"
            ],
            "abstract": "In this paper, we present a fast and strong neural approach for general\npurpose text matching applications. We explore what is sufficient to build a\nfast and well-performed text matching model and propose to keep three key\nfeatures available for inter-sequence alignment: original point-wise features,\nprevious aligned features, and contextual features while simplifying all the\nremaining components. We conduct experiments on four well-studied benchmark\ndatasets across tasks of natural language inference, paraphrase identification\nand answer selection. The performance of our model is on par with the\nstate-of-the-art on all datasets with much fewer parameters and the inference\nspeed is at least 6 times faster compared with similarly performed ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00308v1",
            "date": "2019-08-01 10:27:29+00:00",
            "title": "MSnet: A BERT-based Network for Gendered Pronoun Resolution",
            "authors": [
                "Zili Wang"
            ],
            "abstract": "The pre-trained BERT model achieves a remarkable state of the art across a\nwide range of tasks in natural language processing. For solving the gender bias\nin gendered pronoun resolution task, I propose a novel neural network model\nbased on the pre-trained BERT. This model is a type of mention score classifier\nand uses an attention mechanism with no parameters to compute the contextual\nrepresentation of entity span, and a vector to represent the triple-wise\nsemantic similarity among the pronoun and the entities. In stage 1 of the\ngendered pronoun resolution task, a variant of this model, trained in the\nfine-tuning approach, reduced the multi-class logarithmic loss to 0.3033 in the\n5-fold cross-validation of training set and 0.2795 in testing set. Besides,\nthis variant won the 2nd place with a score at 0.17289 in stage 2 of the task.\nThe code in this paper is available at:\nhttps://github.com/ziliwang/MSnet-for-Gendered-PronounResolution",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00358v3",
            "date": "2019-08-01 12:31:49+00:00",
            "title": "Dolphin: A Spoken Language Proficiency Assessment System for Elementary Education",
            "authors": [
                "Wenbiao Ding",
                "Guowei Xu",
                "Tianqiao Liu",
                "Weiping Fu",
                "Yujia Song",
                "Chaoyou Guo",
                "Cong Kong",
                "Songfan Yang",
                "Gale Yan Huang",
                "Zitao Liu"
            ],
            "abstract": "Spoken language proficiency is critically important for children's growth and\npersonal development. Due to the limited and imbalanced educational resources\nin China, elementary students barely have chances to improve their oral\nlanguage skills in classes. Verbal fluency tasks (VFTs) were invented to let\nthe students practice their spoken language proficiency after school. VFTs are\nsimple but concrete math related questions that ask students to not only report\nanswers but speak out the entire thinking process. In spite of the great\nsuccess of VFTs, they bring a heavy grading burden to elementary teachers. To\nalleviate this problem, we develop Dolphin, a spoken language proficiency\nassessment system for Chinese elementary education. Dolphin is able to\nautomatically evaluate both phonological fluency and semantic relevance of\nstudents' VFT answers. We conduct a wide range of offline and online\nexperiments to demonstrate the effectiveness of Dolphin. In our offline\nexperiments, we show that Dolphin improves both phonological fluency and\nsemantic relevance evaluation performance when compared to state-of-the-art\nbaselines on real-world educational data sets. In our online A/B experiments,\nwe test Dolphin with 183 teachers from 2 major cities (Hangzhou and Xi'an) in\nChina for 10 weeks and the results show that VFT assignments grading coverage\nis improved by 22\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01328v1",
            "date": "2019-08-04 12:40:28+00:00",
            "title": "Automatic Fact-Checking Using Context and Discourse Information",
            "authors": [
                "Pepa Atanasova",
                "Preslav Nakov",
                "Llu\u00eds M\u00e0rquez",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Georgi Karadzhov",
                "Tsvetomila Mihaylova",
                "Mitra Mohtarami",
                "James Glass"
            ],
            "abstract": "We study the problem of automatic fact-checking, paying special attention to\nthe impact of contextual and discourse information. We address two related\ntasks: (i) detecting check-worthy claims, and (ii) fact-checking claims. We\ndevelop supervised systems based on neural networks, kernel-based support\nvector machines, and combinations thereof, which make use of rich input\nrepresentations in terms of discourse cues and contextual features. For the\ncheck-worthiness estimation task, we focus on political debates, and we model\nthe target claim in the context of the full intervention of a participant and\nthe previous and the following turns in the debate, taking into account\ncontextual meta information. For the fact-checking task, we focus on answer\nverification in a community forum, and we model the veracity of the answer with\nrespect to the entire question--answer thread in which it occurs as well as\nwith respect to other related posts from the entire forum. We develop annotated\ndatasets for both tasks and we run extensive experimental evaluation,\nconfirming that both types of information ---but especially contextual\nfeatures--- play an important role.",
            "journal ref": "J. Data and Information Quality, Volume 11 Issue 3, July 2019,\n  Article No. 12",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01519v2",
            "date": "2019-08-05 08:45:20+00:00",
            "title": "Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian",
            "authors": [
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Recently, reading comprehension models achieved near-human performance on\nlarge-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely\ndue to the release of pre-trained contextualized representations such as BERT\nand ELMo, which can be fine-tuned for the target task. Despite those advances\nand the creation of more challenging datasets, most of the work is still done\nfor English. Here, we study the effectiveness of multilingual BERT fine-tuned\non large-scale English datasets for reading comprehension (e.g., for RACE), and\nwe apply it to Bulgarian multiple-choice reading comprehension. We propose a\nnew dataset containing 2,221 questions from matriculation exams for twelfth\ngrade in various subjects -history, biology, geography and philosophy-, and 412\nadditional questions from online quizzes in history. While the quiz authors\ngave no relevant context, we incorporate knowledge from Wikipedia, retrieving\ndocuments matching the combination of question + each answer option. Moreover,\nwe experiment with different indexing and pre-training strategies. The\nevaluation results show accuracy of 42.23%, which is well above the baseline of\n24.89%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01699v1",
            "date": "2019-08-05 15:45:39+00:00",
            "title": "Thoth: Improved Rapid Serial Visual Presentation using Natural Language Processing",
            "authors": [
                "David Awad"
            ],
            "abstract": "Thoth is a tool designed to combine many different types of speed reading\ntechnology. The largest insight is using natural language parsing for more\noptimal rapid serial visual presentation and more effective reading\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01765v1",
            "date": "2019-08-03 23:48:05+00:00",
            "title": "Sentiment Analysis of Typhoon Related Tweets using Standard and Bidirectional Recurrent Neural Networks",
            "authors": [
                "Joseph Marvin Imperial",
                "Jeyrome Orosco",
                "Shiela Mae Mazo",
                "Lany Maceda"
            ],
            "abstract": "The Philippines is a common ground to natural calamities like typhoons,\nfloods, volcanic eruptions and earthquakes. With Twitter as one of the most\nused social media platform in the Philippines, a total of 39,867 preprocessed\ntweets were obtained given a time frame starting from November 1, 2013 to\nJanuary 31, 2014. Sentiment analysis determines the underlying emotion given a\nseries of words. The main purpose of this study is to identify the sentiments\nexpressed in the tweets sent by the Filipino people before, during, and after\nTyphoon Yolanda using two variations of Recurrent Neural Networks; standard and\nbidirectional. The best generated models after training with various\nhyperparameters achieved a high accuracy of 81.79% for fine-grained\nclassification using standard RNN and 87.69% for binary classification using\nbidirectional RNN. Findings revealed that 51.1% of the tweets sent were\npositive expressing support, love, and words of courage to the victims; 19.8%\nwere negative stating sadness and despair for the loss of lives and hate for\ncorrupt officials; while the other 29% were neutral tweets from local news\nstations, announcements of relief operations, donation drives, and observations\nby citizens.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1908.01816v1",
            "date": "2019-07-23 04:38:09+00:00",
            "title": "MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models",
            "authors": [
                "Boyuan Pan",
                "Yazheng Yang",
                "Hao Li",
                "Zhou Zhao",
                "Yueting Zhuang",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "Machine Comprehension (MC) is one of the core problems in natural language\nprocessing, requiring both understanding of the natural language and knowledge\nabout the world. Rapid progress has been made since the release of several\nbenchmark datasets, and recently the state-of-the-art models even surpass human\nperformance on the well-known SQuAD evaluation. In this paper, we transfer\nknowledge learned from machine comprehension to the sequence-to-sequence tasks\nto deepen the understanding of the text. We propose MacNet: a novel\nencoder-decoder supplementary architecture to the widely used attention-based\nsequence-to-sequence models. Experiments on neural machine translation (NMT)\nand abstractive text summarization show that our proposed framework can\nsignificantly improve the performance of the baseline models, and our method\nfor the abstractive text summarization achieves the state-of-the-art results on\nthe Gigaword dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01853v1",
            "date": "2019-08-02 01:13:50+00:00",
            "title": "DELTA: A DEep learning based Language Technology plAtform",
            "authors": [
                "Kun Han",
                "Junwen Chen",
                "Hui Zhang",
                "Haiyang Xu",
                "Yiping Peng",
                "Yun Wang",
                "Ning Ding",
                "Hui Deng",
                "Yonghu Gao",
                "Tingwei Guo",
                "Yi Zhang",
                "Yahao He",
                "Baochang Ma",
                "Yulong Zhou",
                "Kangli Zhang",
                "Chao Liu",
                "Ying Lyu",
                "Chenxi Wang",
                "Cheng Gong",
                "Yunbo Wang",
                "Wei Zou",
                "Hui Song",
                "Xiangang Li"
            ],
            "abstract": "In this paper we present DELTA, a deep learning based language technology\nplatform. DELTA is an end-to-end platform designed to solve industry level\nnatural language and speech processing problems. It integrates most popular\nneural network models for training as well as comprehensive deployment tools\nfor production. DELTA aims to provide easy and fast experiences for using,\ndeploying, and developing natural language processing and speech models for\nboth academia and industry use cases. We demonstrate the reliable performance\nwith DELTA on several natural language processing and speech tasks, including\ntext classification, named entity recognition, natural language inference,\nspeech recognition, speaker verification, etc. DELTA has been used for\ndeveloping several state-of-the-art algorithms for publications and delivering\nreal production to serve millions of users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01946v3",
            "date": "2019-08-06 04:01:42+00:00",
            "title": "Dialog State Tracking: A Neural Reading Comprehension Approach",
            "authors": [
                "Shuyang Gao",
                "Abhishek Sethi",
                "Sanchit Agarwal",
                "Tagyoung Chung",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Dialog state tracking is used to estimate the current belief state of a\ndialog given all the preceding conversation. Machine reading comprehension, on\nthe other hand, focuses on building systems that read passages of text and\nanswer questions that require some understanding of passages. We formulate\ndialog state tracking as a reading comprehension task to answer the question\n$what\\ is\\ the\\ state\\ of\\ the\\ current\\ dialog?$ after reading conversational\ncontext. In contrast to traditional state tracking methods where the dialog\nstate is often predicted as a distribution over a closed set of all the\npossible slot values within an ontology, our method uses a simple\nattention-based neural network to point to the slot values within the\nconversation. Experiments on MultiWOZ-2.0 cross-domain dialog dataset show that\nour simple system can obtain similar accuracies compared to the previous more\ncomplex methods. By exploiting recent advances in contextual word embeddings,\nadding a model that explicitly tracks whether a slot value should be carried\nover to the next turn, and combining our method with a traditional joint state\ntracking method that relies on closed set vocabulary, we can obtain a\njoint-goal accuracy of $47.33\\%$ on the standard test split, exceeding current\nstate-of-the-art by $11.75\\%$**.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01968v1",
            "date": "2019-08-06 05:57:22+00:00",
            "title": "Self-Balanced Dropout",
            "authors": [
                "Shen Li",
                "Chenhao Su",
                "Renfen Hu",
                "Zhengdong Lu"
            ],
            "abstract": "Dropout is known as an effective way to reduce overfitting via preventing\nco-adaptations of units. In this paper, we theoretically prove that the\nco-adaptation problem still exists after using dropout due to the correlations\namong the inputs. Based on the proof, we further propose Self-Balanced Dropout,\na novel dropout method which uses a trainable variable to balance the influence\nof the input correlation on parameter update. We evaluate Self-Balanced Dropout\non a range of tasks with both simple and complex models. The experimental\nresults show that the mechanism can effectively solve the co-adaption problem\nto some extent and significantly improve the performance on all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01969v1",
            "date": "2019-08-06 05:58:06+00:00",
            "title": "Word Embedding for Response-To-Text Assessment of Evidence",
            "authors": [
                "Haoran Zhang",
                "Diane Litman"
            ],
            "abstract": "Manually grading the Response to Text Assessment (RTA) is labor intensive.\nTherefore, an automatic method is being developed for scoring analytical\nwriting when the RTA is administered in large numbers of classrooms. Our\nlong-term goal is to also use this scoring method to provide formative feedback\nto students and teachers about students' writing quality. As a first step\ntowards this goal, interpretable features for automatically scoring the\nevidence rubric of the RTA have been developed. In this paper, we present a\nsimple but promising method for improving evidence scoring by employing the\nword embedding model. We evaluate our method on corpora of responses written by\nupper elementary students.",
            "journal ref": "Proceedings of ACL 2017, Student Research Workshop (2017) 75-81",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01992v1",
            "date": "2019-08-06 07:24:14+00:00",
            "title": "eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing",
            "authors": [
                "Haoran Zhang",
                "Ahmed Magooda",
                "Diane Litman",
                "Richard Correnti",
                "Elaine Wang",
                "Lindsay Clare Matsumura",
                "Emily Howe",
                "Rafael Quintana"
            ],
            "abstract": "Writing a good essay typically involves students revising an initial paper\ndraft after receiving feedback. We present eRevise, a web-based writing and\nrevising environment that uses natural language processing features generated\nfor rubric-based essay scoring to trigger formative feedback messages regarding\nstudents' use of evidence in response-to-text writing. By helping students\nunderstand the criteria for using text evidence during writing, eRevise\nempowers students to better revise their paper drafts. In a pilot deployment of\neRevise in 7 classrooms spanning grades 5 and 6, the quality of text evidence\nusage in writing improved after students received formative feedback then\nengaged in paper revision.",
            "journal ref": "Proceedings of the AAAI Conference on Artificial Intelligence\n  (2019) vol. 33, 9619-9625",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01993v1",
            "date": "2019-08-06 07:28:43+00:00",
            "title": "Co-Attention Based Neural Network for Source-Dependent Essay Scoring",
            "authors": [
                "Haoran Zhang",
                "Diane Litman"
            ],
            "abstract": "This paper presents an investigation of using a co-attention based neural\nnetwork for source-dependent essay scoring. We use a co-attention mechanism to\nhelp the model learn the importance of each part of the essay more accurately.\nAlso, this paper shows that the co-attention based neural network model\nprovides reliable score prediction of source-dependent responses. We evaluate\nour model on two source-dependent response corpora. Results show that our model\noutperforms the baseline on both corpora. We also show that the attention of\nthe model is similar to the expert opinions with examples.",
            "journal ref": "Proceedings of the Thirteenth Workshop on Innovative Use of NLP\n  for Building Educational Applications (2018) 399-409",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02265v1",
            "date": "2019-08-06 17:33:52+00:00",
            "title": "ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks",
            "authors": [
                "Jiasen Lu",
                "Dhruv Batra",
                "Devi Parikh",
                "Stefan Lee"
            ],
            "abstract": "We present ViLBERT (short for Vision-and-Language BERT), a model for learning\ntask-agnostic joint representations of image content and natural language. We\nextend the popular BERT architecture to a multi-modal two-stream model,\npro-cessing both visual and textual inputs in separate streams that interact\nthrough co-attentional transformer layers. We pretrain our model through two\nproxy tasks on the large, automatically collected Conceptual Captions dataset\nand then transfer it to multiple established vision-and-language tasks --\nvisual question answering, visual commonsense reasoning, referring expressions,\nand caption-based image retrieval -- by making only minor additions to the base\narchitecture. We observe significant improvements across tasks compared to\nexisting task-specific models -- achieving state-of-the-art on all four tasks.\nOur work represents a shift away from learning groundings between vision and\nlanguage only as part of task training and towards treating visual grounding as\na pretrainable and transferable capability.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.02425v1",
            "date": "2019-08-07 02:58:24+00:00",
            "title": "Text mining policy: Classifying forest and landscape restoration policy agenda with neural information retrieval",
            "authors": [
                "John Brandt"
            ],
            "abstract": "Dozens of countries have committed to restoring the ecological functionality\nof 350 million hectares of land by 2030. In order to achieve such wide-scale\nimplementation of restoration, the values and priorities of multi-sectoral\nstakeholders must be aligned and integrated with national level commitments and\nother development agenda. Although misalignment across scales of policy and\nbetween stakeholders are well known barriers to implementing restoration,\nfast-paced policy making in multi-stakeholder environments complicates the\nmonitoring and analysis of governance and policy. In this work, we assess the\npotential of machine learning to identify restoration policy agenda across\ndiverse policy documents. An unsupervised neural information retrieval\narchitecture is introduced that leverages transfer learning and word embeddings\nto create high-dimensional representations of paragraphs. Policy agenda labels\nare recast as information retrieval queries in order to classify policies with\na cosine similarity threshold between paragraphs and query embeddings. This\napproach achieves a 0.83 F1-score measured across 14 policy agenda in 31 policy\ndocuments in Malawi, Kenya, and Rwanda, indicating that automated text mining\ncan provide reliable, generalizable, and efficient analyses of restoration\npolicy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.02451v1",
            "date": "2019-08-07 06:02:17+00:00",
            "title": "TinySearch -- Semantics based Search Engine using Bert Embeddings",
            "authors": [
                "Manish Patel"
            ],
            "abstract": "Existing search engines use keyword matching or tf-idf based matching to map\nthe query to the web-documents and rank them. They also consider other factors\nsuch as page rank, hubs-and-authority scores, knowledge graphs to make the\nresults more meaningful. However, the existing search engines fail to capture\nthe meaning of query when it becomes large and complex. BERT, introduced by\nGoogle in 2018, provides embeddings for words as well as sentences. In this\npaper, I have developed a semantics-oriented search engine using neural\nnetworks and BERT embeddings that can search for query and rank the documents\nin the order of the most meaningful to least meaningful. The results shows\nimprovement over one existing search engine for complex queries for given set\nof documents.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.02923v3",
            "date": "2019-08-08 04:07:39+00:00",
            "title": "Image Captioning using Facial Expression and Attention",
            "authors": [
                "Omid Mohamad Nezami",
                "Mark Dras",
                "Stephen Wan",
                "Cecile Paris"
            ],
            "abstract": "Benefiting from advances in machine vision and natural language processing\ntechniques, current image captioning systems are able to generate detailed\nvisual descriptions. For the most part, these descriptions represent an\nobjective characterisation of the image, although some models do incorporate\nsubjective aspects related to the observer's view of the image, such as\nsentiment; current models, however, usually do not consider the emotional\ncontent of images during the caption generation process. This paper addresses\nthis issue by proposing novel image captioning models which use facial\nexpression features to generate image captions. The models generate image\ncaptions using long short-term memory networks applying facial features in\naddition to other visual features at different time steps. We compare a\ncomprehensive collection of image captioning models with and without facial\nfeatures using all standard evaluation metrics. The evaluation metrics indicate\nthat applying facial features with an attention mechanism achieves the best\nperformance, showing more expressive and more correlated image captions, on an\nimage caption dataset extracted from the standard Flickr 30K dataset,\nconsisting of around 11K images containing faces. An analysis of the generated\ncaptions finds that, perhaps unexpectedly, the improvement in caption quality\nappears to come not from the addition of adjectives linked to emotional aspects\nof the images, but from more variety in the actions described in the captions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.02943v1",
            "date": "2019-08-08 06:25:38+00:00",
            "title": "Towards Generating Stylized Image Captions via Adversarial Training",
            "authors": [
                "Omid Mohamad Nezami",
                "Mark Dras",
                "Stephen Wan",
                "Cecile Paris",
                "Len Hamey"
            ],
            "abstract": "While most image captioning aims to generate objective descriptions of\nimages, the last few years have seen work on generating visually grounded image\ncaptions which have a specific style (e.g., incorporating positive or negative\nsentiment). However, because the stylistic component is typically the last part\nof training, current models usually pay more attention to the style at the\nexpense of accurate content description. In addition, there is a lack of\nvariability in terms of the stylistic aspects. To address these issues, we\npropose an image captioning model called ATTEND-GAN which has two core\ncomponents: first, an attention-based caption generator to strongly correlate\ndifferent parts of an image with different parts of a caption; and second, an\nadversarial training mechanism to assist the caption generator to add diverse\nstylistic components to the generated captions. Because of these components,\nATTEND-GAN can generate correlated captions as well as more human-like\nvariability of stylistic patterns. Our system outperforms the state-of-the-art\nas well as a collection of our baseline models. A linguistic analysis of the\ngenerated captions demonstrates that captions generated using ATTEND-GAN have a\nwider range of stylistic adjectives and adjective-noun pairs.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.03146v1",
            "date": "2019-08-08 16:18:30+00:00",
            "title": "Your Stance is Exposed! Analysing Possible Factors for Stance Detection on Social Media",
            "authors": [
                "Abeer Aldayel",
                "Walid Magdy"
            ],
            "abstract": "To what extent user's stance towards a given topic could be inferred? Most of\nthe studies on stance detection have focused on analysing user's posts on a\ngiven topic to predict the stance. However, the stance in social media can be\ninferred from a mixture of signals that might reflect user's beliefs including\nposts and online interactions. This paper examines various online features of\nusers to detect their stance towards different topics. We compare multiple set\nof features, including on-topic content, network interactions, user's\npreferences, and online network connections. Our objective is to understand the\nonline signals that can reveal the users' stance. Experimentation is applied on\ntweets dataset from the SemEval stance detection task, which covers five\ntopics. Results show that stance of a user can be detected with multiple\nsignals of user's online activity, including their posts on the topic, the\nnetwork they interact with or follow, the websites they visit, and the content\nthey like. The performance of the stance modelling using different network\nfeatures are comparable with the state-of-the-art reported model that used\ntextual content only. In addition, combining network and content features leads\nto the highest reported performance to date on the SemEval dataset with\nF-measure of 72.49%. We further present an extensive analysis to show how these\ndifferent set of features can reveal stance. Our findings have distinct privacy\nimplications, where they highlight that stance is strongly embedded in user's\nonline social network that, in principle, individuals can be profiled from\ntheir interactions and connections even when they do not post about the topic.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.03181v1",
            "date": "2019-08-08 17:21:16+00:00",
            "title": "Assessing Sentiment of the Expressed Stance on Social Media",
            "authors": [
                "Abeer Aldayel",
                "Walid Magdy"
            ],
            "abstract": "Stance detection is the task of inferring viewpoint towards a given topic or\nentity either being supportive or opposing. One may express a viewpoint towards\na topic by using positive or negative language. This paper examines how the\nstance is being expressed in social media according to the sentiment polarity.\nThere has been a noticeable misconception of the similarity between the stance\nand sentiment when it comes to viewpoint discovery, where negative sentiment is\nassumed to mean against stance, and positive sentiment means in-favour stance.\nTo analyze the relation between stance and sentiment, we construct a new\ndataset with four topics and examine how people express their viewpoint with\nregards these topics. We validate our results by carrying a further analysis of\nthe popular stance benchmark SemEval stance dataset. Our analyses reveal that\nsentiment and stance are not highly aligned, and hence the simple sentiment\npolarity cannot be used solely to denote a stance toward a given topic.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.03313v1",
            "date": "2019-08-09 04:44:12+00:00",
            "title": "Using Semantic Role Knowledge for Relevance Ranking of Key Phrases in Documents: An Unsupervised Approach",
            "authors": [
                "Prateeti Mohapatra",
                "Neelamadhav Gantayat",
                "Gargi B. Dasgupta"
            ],
            "abstract": "In this paper, we investigate the integration of sentence position and\nsemantic role of words in a PageRank system to build a key phrase ranking\nmethod. We present the evaluation results of our approach on three scientific\narticles. We show that semantic role information, when integrated with a\nPageRank system, can become a new lexical feature. Our approach had an overall\nimprovement on all the data sets over the state-of-art baseline approaches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03538v2",
            "date": "2019-08-09 16:57:04+00:00",
            "title": "Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling",
            "authors": [
                "Siyuan Feng",
                "Tan Lee"
            ],
            "abstract": "This research addresses the problem of acoustic modeling of low-resource\nlanguages for which transcribed training data is absent. The goal is to learn\nrobust frame-level feature representations that can be used to identify and\ndistinguish subword-level speech units. The proposed feature representations\ncomprise various types of multilingual bottleneck features (BNFs) that are\nobtained via multi-task learning of deep neural networks (MTL-DNN). One of the\nkey problems is how to acquire high-quality frame labels for untranscribed\ntraining data to facilitate supervised DNN training. It is shown that learning\nof robust BNF representations can be achieved by effectively leveraging\ntranscribed speech data and well-trained automatic speech recognition (ASR)\nsystems from one or more out-of-domain (resource-rich) languages. Out-of-domain\nASR systems can be applied to perform speaker adaptation with untranscribed\ntraining data of the target language, and to decode the training speech into\nframe-level labels for DNN training. It is also found that better frame labels\ncan be generated by considering temporal dependency in speech when performing\nframe clustering. The proposed methods of feature learning are evaluated on the\nstandard task of unsupervised subword modeling in Track 1 of the ZeroSpeech\n2017 Challenge. The best performance achieved by our system is $9.7\\%$ in terms\nof across-speaker triphone minimal-pair ABX error rate, which is comparable to\nthe best systems reported recently. Lastly, our investigation reveals that the\ncloseness between target languages and out-of-domain languages and the amount\nof available training data for individual target languages could have\nsignificant impact on the goodness of learned features.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.03645v1",
            "date": "2019-08-09 22:06:06+00:00",
            "title": "A Generate-Validate Approach to Answering Questions about Qualitative Relationships",
            "authors": [
                "Arindam Mitra",
                "Chitta Baral",
                "Aurgho Bhattacharjee",
                "Ishan Shrivastava"
            ],
            "abstract": "Qualitative relationships describe how increasing or decreasing one property\n(e.g. altitude) affects another (e.g. temperature). They are an important\naspect of natural language question answering and are crucial for building\nchatbots or voice agents where one may enquire about qualitative relationships.\nRecently a dataset about question answering involving qualitative relationships\nhas been proposed, and a few approaches to answer such questions have been\nexplored, in the heart of which lies a semantic parser that converts the\nnatural language input to a suitable logical form. A problem with existing\nsemantic parsers is that they try to directly convert the input sentences to a\nlogical form. Since the output language varies with each application, it forces\nthe semantic parser to learn almost everything from scratch. In this paper, we\nshow that instead of using a semantic parser to produce the logical form, if we\napply the generate-validate framework i.e. generate a natural language\ndescription of the logical form and validate if the natural language\ndescription is followed from the input text, we get a better scope for transfer\nlearning and our method outperforms the state-of-the-art by a large margin of\n7.93%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03650v4",
            "date": "2019-08-09 22:41:20+00:00",
            "title": "TEQUILA: Temporal Question Answering over Knowledge Bases",
            "authors": [
                "Zhen Jia",
                "Abdalghani Abujabal",
                "Rishiraj Saha Roy",
                "Jannik Stroetgen",
                "Gerhard Weikum"
            ],
            "abstract": "Question answering over knowledge bases (KB-QA) poses challenges in handling\ncomplex questions that need to be decomposed into sub-questions. An important\ncase, addressed here, is that of temporal questions, where cues for temporal\nrelations need to be discovered and handled. We present TEQUILA, an enabler\nmethod for temporal QA that can run on top of any KB-QA engine. TEQUILA has\nfour stages. It detects if a question has temporal intent. It decomposes and\nrewrites the question into non-temporal sub-questions and temporal constraints.\nAnswers to sub-questions are then retrieved from the underlying KB-QA engine.\nFinally, TEQUILA uses constraint reasoning on temporal intervals to compute\nfinal answers to the full question. Comparisons against state-of-the-art\nbaselines show the viability of our method.",
            "journal ref": "CIKM 2018",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03734v1",
            "date": "2019-08-10 11:39:17+00:00",
            "title": "Unsupervised Stemming based Language Model for Telugu Broadcast News Transcription",
            "authors": [
                "Mythili Sharan Pala",
                "Parayitam Laxminarayana",
                "A. V. Ramana"
            ],
            "abstract": "In Indian Languages , native speakers are able to understand new words formed\nby either combining or modifying root words with tense and / or gender. Due to\ndata insufficiency, Automatic Speech Recognition system (ASR) may not\naccommodate all the words in the language model irrespective of the size of the\ntext corpus. It also becomes computationally challenging if the volume of the\ndata increases exponentially due to morphological changes to the root word. In\nthis paper a new unsupervised method is proposed for a Indian language: Telugu,\nbased on the unsupervised method for Hindi, to generate the Out of Vocabulary\n(OOV) words in the language model. By using techniques like smoothing and\ninterpolation of pre-processed data with supervised and unsupervised stemming,\ndifferent issues in language model for Indian language: Telugu has been\naddressed. We observe that the smoothing techniques Witten-Bell and Kneser-Ney\nperform well when compared to other techniques on pre-processed data from\nsupervised learning. The ASRs accuracy is improved by 0.76% and 0.94% with\nsupervised and unsupervised stemming respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04042v1",
            "date": "2019-08-12 08:04:42+00:00",
            "title": "Evaluating Tag Recommendations for E-Book Annotation Using a Semantic Similarity Metric",
            "authors": [
                "Emanuel Lacic",
                "Dominik Kowald",
                "Dieter Theiler",
                "Matthias Traub",
                "Lucky Kuffer",
                "Stefanie Lindstaedt",
                "Elisabeth Lex"
            ],
            "abstract": "In this paper, we present our work to support publishers and editors in\nfinding descriptive tags for e-books through tag recommendations. We propose a\nhybrid tag recommendation system for e-books, which leverages search query\nterms from Amazon users and e-book metadata, which is assigned by publishers\nand editors. Our idea is to mimic the vocabulary of users in Amazon, who search\nfor and review e-books, and to combine these search terms with editor tags in a\nhybrid tag recommendation approach. In total, we evaluate 19 tag recommendation\nalgorithms on the review content of Amazon users, which reflects the readers'\nvocabulary. Our results show that we can improve the performance of tag\nrecommender systems for e-books both concerning tag recommendation accuracy,\ndiversity as well as a novel semantic similarity metric, which we also propose\nin this paper.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.04092v1",
            "date": "2019-08-12 11:20:29+00:00",
            "title": "Active Annotation: bootstrapping annotation lexicon and guidelines for supervised NLU learning",
            "authors": [
                "Federico Marinelli",
                "Alessandra Cervone",
                "Giuliano Tortoreto",
                "Evgeny A. Stepanov",
                "Giuseppe Di Fabbrizio",
                "Giuseppe Riccardi"
            ],
            "abstract": "Natural Language Understanding (NLU) models are typically trained in a\nsupervised learning framework. In the case of intent classification, the\npredicted labels are predefined and based on the designed annotation schema\nwhile the labelling process is based on a laborious task where annotators\nmanually inspect each utterance and assign the corresponding label. We propose\nan Active Annotation (AA) approach where we combine an unsupervised learning\nmethod in the embedding space, a human-in-the-loop verification process, and\nlinguistic insights to create lexicons that can be open categories and adapted\nover time. In particular, annotators define the y-label space on-the-fly during\nthe annotation using an iterative process and without the need for prior\nknowledge about the input data. We evaluate the proposed annotation paradigm in\na real use-case NLU scenario. Results show that our Active Annotation paradigm\nachieves accurate and higher quality training data, with an annotation speed of\nan order of magnitude higher with respect to the traditional human-only driven\nbaseline annotation methodology.",
            "journal ref": "INTERSPEECH 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04211v4",
            "date": "2019-08-12 15:48:34+00:00",
            "title": "On Identifiability in Transformers",
            "authors": [
                "Gino Brunner",
                "Yang Liu",
                "Dami\u00e1n Pascual",
                "Oliver Richter",
                "Massimiliano Ciaramita",
                "Roger Wattenhofer"
            ],
            "abstract": "In this paper we delve deep in the Transformer architecture by investigating\ntwo of its core components: self-attention and contextual embeddings. In\nparticular, we study the identifiability of attention weights and token\nembeddings, and the aggregation of context into hidden tokens. We show that,\nfor sequences longer than the attention head dimension, attention weights are\nnot identifiable. We propose effective attention as a complementary tool for\nimproving explanatory interpretations based on attention. Furthermore, we show\nthat input tokens retain to a large degree their identity across the model. We\nalso find evidence suggesting that identity information is mainly encoded in\nthe angle of the embeddings and gradually decreases with depth. Finally, we\ndemonstrate strong mixing of input information in the generation of contextual\nembeddings by means of a novel quantification method based on gradient\nattribution. Overall, we show that self-attention distributions are not\ndirectly interpretable and present tools to better understand and further\ninvestigate Transformer models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04332v1",
            "date": "2019-08-12 18:39:10+00:00",
            "title": "LSTM vs. GRU vs. Bidirectional RNN for script generation",
            "authors": [
                "Sanidhya Mangal",
                "Poorva Joshi",
                "Rahul Modak"
            ],
            "abstract": "Scripts are an important part of any TV series. They narrate movements,\nactions and expressions of characters. In this paper, a case study is presented\non how different sequence to sequence deep learning models perform in the task\nof generating new conversations between characters as well as new scenarios on\nthe basis of a script (previous conversations). A comprehensive comparison\nbetween these models, namely, LSTM, GRU and Bidirectional RNN is presented. All\nthe models are designed to learn the sequence of recurring characters from the\ninput sequence. Each input sequence will contain, say \"n\" characters, and the\ncorresponding targets will contain the same number of characters, except, they\nwill be shifted one character to the right. In this manner, input and output\nsequences are generated and used to train the models. A closer analysis of\nexplored models performance and efficiency is delineated with the help of graph\nplots and generated texts by taking some input string. These graphs describe\nboth, intraneural performance and interneural model performance for each model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04364v2",
            "date": "2019-08-12 20:18:50+00:00",
            "title": "AmazonQA: A Review-Based Question Answering Task",
            "authors": [
                "Mansi Gupta",
                "Nitish Kulkarni",
                "Raghuveer Chanda",
                "Anirudha Rayasam",
                "Zachary C Lipton"
            ],
            "abstract": "Every day, thousands of customers post questions on Amazon product pages.\nAfter some time, if they are fortunate, a knowledgeable customer might answer\ntheir question. Observing that many questions can be answered based upon the\navailable product reviews, we propose the task of review-based QA. Given a\ncorpus of reviews and a question, the QA system synthesizes an answer. To this\nend, we introduce a new dataset and propose a method that combines information\nretrieval techniques for selecting relevant reviews (given a question) and\n\"reading comprehension\" models for synthesizing an answer (given a question and\nreview). Our dataset consists of 923k questions, 3.6M answers and 14M reviews\nacross 156k products. Building on the well-known Amazon dataset, we collect\nadditional annotations, marking each question as either answerable or\nunanswerable based on the available reviews. A deployed system could first\nclassify a question as answerable and then attempt to generate an answer.\nNotably, unlike many popular QA datasets, here, the questions, passages, and\nanswers are all extracted from real human interactions. We evaluate numerous\nmodels for answer generation and propose strong baselines, demonstrating the\nchallenging nature of this new task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04621v1",
            "date": "2019-08-13 13:03:58+00:00",
            "title": "Getting To Know You: User Attribute Extraction from Dialogues",
            "authors": [
                "Chien-Sheng Wu",
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "User attributes provide rich and useful information for user understanding,\nyet structured and easy-to-use attributes are often sparsely populated. In this\npaper, we leverage dialogues with conversational agents, which contain strong\nsuggestions of user information, to automatically extract user attributes.\nSince no existing dataset is available for this purpose, we apply distant\nsupervision to train our proposed two-stage attribute extractor, which\nsurpasses several retrieval and generation baselines on human evaluation.\nMeanwhile, we discuss potential applications (e.g., personalized recommendation\nand dialogue systems) of such extracted user attributes, and point out current\nlimitations to cast light on future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04755v1",
            "date": "2019-08-13 17:20:51+00:00",
            "title": "Fine-grained Information Status Classification Using Discourse Context-Aware Self-Attention",
            "authors": [
                "Yufang Hou"
            ],
            "abstract": "Previous work on bridging anaphora recognition (Hou et al., 2013a) casts the\nproblem as a subtask of learning fine-grained information status (IS). However,\nthese systems heavily depend on many hand-crafted linguistic features. In this\npaper, we propose a discourse context-aware self-attention neural network model\nfor fine-grained IS classification. On the ISNotes corpus (Markert et al.,\n2012), our model with the contextually-encoded word representations (BERT)\n(Devlin et al., 2018) achieves new state-of-the-art performances on\nfine-grained IS classification, obtaining a 4.1% absolute overall accuracy\nimprovement compared to Hou et al. (2013a). More importantly, we also show an\nimprovement of 3.9% F1 for bridging anaphora recognition without using any\ncomplex hand-crafted semantic features designed for capturing the bridging\nphenomenon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04777v3",
            "date": "2019-08-13 17:48:10+00:00",
            "title": "Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Games",
            "authors": [
                "Xusen Yin",
                "Jonathan May"
            ],
            "abstract": "We consider the task of learning to play families of text-based computer\nadventure games, i.e., fully textual environments with a common theme (e.g.\ncooking) and goal (e.g. prepare a meal from a recipe) but with different\nspecifics; new instances of such games are relatively straightforward for\nhumans to master after a brief exposure to the genre but have been curiously\ndifficult for computer agents to learn. We find that the deep Q-learning\nstrategies that have been successfully leveraged for superhuman performance in\nsingle-instance action video games can be applied to learn families of text\nvideo games when adopting simple strategies that correlate with human-like\nlearning behavior. Specifically, we build agents that learn to tackle simple\nscenarios before more complex ones using curriculum learning, that familiarize\nthemselves in an unfamiliar environment by navigating before acting, and that\nexplore uncertain environments more thoroughly using contextual multi-armed\nbandit decision policies. We demonstrate improved task completion rates over\nreasonable baselines when evaluating on never-before-seen games of that theme.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04812v2",
            "date": "2019-08-13 18:24:29+00:00",
            "title": "An Effective Domain Adaptive Post-Training Method for BERT in Response Selection",
            "authors": [
                "Taesun Whang",
                "Dongyub Lee",
                "Chanhee Lee",
                "Kisu Yang",
                "Dongsuk Oh",
                "HeuiSeok Lim"
            ],
            "abstract": "We focus on multi-turn response selection in a retrieval-based dialog system.\nIn this paper, we utilize the powerful pre-trained language model\nBi-directional Encoder Representations from Transformer (BERT) for a multi-turn\ndialog system and propose a highly effective post-training method on\ndomain-specific corpus. Although BERT is easily adopted to various NLP tasks\nand outperforms previous baselines of each task, it still has limitations if a\ntask corpus is too focused on a certain domain. Post-training on\ndomain-specific corpus (e.g., Ubuntu Corpus) helps the model to train\ncontextualized representations and words that do not appear in general corpus\n(e.g., English Wikipedia). Experimental results show that our approach achieves\nnew state-of-the-art on two response selection benchmarks (i.e., Ubuntu Corpus\nV1, Advising Corpus) performance improvement by 5.9% and 6% on R@1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04832v1",
            "date": "2019-08-13 19:10:36+00:00",
            "title": "Entertaining and Opinionated but Too Controlling: A Large-Scale User Study of an Open Domain Alexa Prize System",
            "authors": [
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Wen Cui",
                "Juraj Juraska",
                "Vrindavan Harrison",
                "Brian Schwarzmann",
                "Nicholas Santer",
                "Steve Whittaker",
                "Marilyn Walker"
            ],
            "abstract": "Conversational systems typically focus on functional tasks such as scheduling\nappointments or creating todo lists. Instead we design and evaluate SlugBot\n(SB), one of 8 semifinalists in the 2018 AlexaPrize, whose goal is to support\ncasual open-domain social inter-action. This novel application requires both\nbroad topic coverage and engaging interactive skills. We developed a new\ntechnical approach to meet this demanding situation by crowd-sourcing novel\ncontent and introducing playful conversational strategies based on storytelling\nand games. We collected over 10,000 conversations during August 2018 as part of\nthe Alexa Prize competition. We also conducted an in-lab follow-up qualitative\nevaluation. Over-all users found SB moderately engaging; conversations averaged\n3.6 minutes and involved 26 user turns. However, users reacted very differently\nto different conversation subtypes. Storytelling and games were evaluated\npositively; these were seen as entertaining with predictable interactive\nstructure. They also led users to impute personality and intelligence to SB. In\ncontrast, search and general Chit-Chat induced coverage problems; here users\nfound it hard to infer what topics SB could understand, with these\nconversations seen as being too system-driven. Theoretical and design\nimplications suggest a move away from conversational systems that simply\nprovide factual information. Future systems should be designed to have their\nown opinions with personal stories to share, and SB provides an example of how\nwe might achieve this.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04895v2",
            "date": "2019-08-14 00:24:54+00:00",
            "title": "HyperKG: Hyperbolic Knowledge Graph Embeddings for Knowledge Base Completion",
            "authors": [
                "Prodromos Kolyvakis",
                "Alexandros Kalousis",
                "Dimitris Kiritsis"
            ],
            "abstract": "Learning embeddings of entities and relations existing in knowledge bases\nallows the discovery of hidden patterns in data. In this work, we examine the\ngeometrical space's contribution to the task of knowledge base completion. We\nfocus on the family of translational models, whose performance has been\nlagging, and propose a model, dubbed HyperKG, which exploits the hyperbolic\nspace in order to better reflect the topological properties of knowledge bases.\nWe investigate the type of regularities that our model can capture and we show\nthat it is a prominent candidate for effectively representing a subset of\nDatalog rules. We empirically show, using a variety of link prediction\ndatasets, that hyperbolic space allows to narrow down significantly the\nperformance gap between translational and bilinear models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04917v2",
            "date": "2019-08-14 01:49:32+00:00",
            "title": "A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading",
            "authors": [
                "Ya Zhao",
                "Rui Xu",
                "Mingli Song"
            ],
            "abstract": "Lip reading aims at decoding texts from the movement of a speaker's mouth. In\nrecent years, lip reading methods have made great progress for English, at both\nword-level and sentence-level. Unlike English, however, Chinese Mandarin is a\ntone-based language and relies on pitches to distinguish lexical or grammatical\nmeaning, which significantly increases the ambiguity for the lip reading task.\nIn this paper, we propose a Cascade Sequence-to-Sequence Model for Chinese\nMandarin (CSSMCM) lip reading, which explicitly models tones when predicting\nsentence. Tones are modeled based on visual information and syntactic\nstructure, and are used to predict sentence along with visual information and\nsyntactic structure. In order to evaluate CSSMCM, a dataset called CMLR\n(Chinese Mandarin Lip Reading) is collected and released, consisting of over\n100,000 natural sentences from China Network Television website. When trained\non CMLR dataset, the proposed CSSMCM surpasses the performance of\nstate-of-the-art lip reading frameworks, which confirms the effectiveness of\nexplicit modeling of tones for Chinese Mandarin lip reading.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.04919v1",
            "date": "2019-08-14 01:51:49+00:00",
            "title": "Towards Diverse and Accurate Image Captions via Reinforcing Determinantal Point Process",
            "authors": [
                "Qingzhong Wang",
                "Antoni B. Chan"
            ],
            "abstract": "Although significant progress has been made in the field of automatic image\ncaptioning, it is still a challenging task. Previous works normally pay much\nattention to improving the quality of the generated captions but ignore the\ndiversity of captions. In this paper, we combine determinantal point process\n(DPP) and reinforcement learning (RL) and propose a novel reinforcing DPP\n(R-DPP) approach to generate a set of captions with high quality and diversity\nfor an image. We show that R-DPP performs better on accuracy and diversity than\nusing noise as a control signal (GANs, VAEs). Moreover, R-DPP is able to\npreserve the modes of the learned distribution. Hence, beam search algorithm\ncan be applied to generate a single accurate caption, which performs better\nthan other RL-based models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.04926v1",
            "date": "2019-08-14 02:07:02+00:00",
            "title": "Reasoning-Driven Question-Answering for Natural Language Understanding",
            "authors": [
                "Daniel Khashabi"
            ],
            "abstract": "Natural language understanding (NLU) of text is a fundamental challenge in\nAI, and it has received significant attention throughout the history of NLP\nresearch. This primary goal has been studied under different tasks, such as\nQuestion Answering (QA) and Textual Entailment (TE). In this thesis, we\ninvestigate the NLU problem through the QA task and focus on the aspects that\nmake it a challenge for the current state-of-the-art technology. This thesis is\norganized into three main parts:\n  In the first part, we explore multiple formalisms to improve existing machine\ncomprehension systems. We propose a formulation for abductive reasoning in\nnatural language and show its effectiveness, especially in domains with limited\ntraining data. Additionally, to help reasoning systems cope with irrelevant or\nredundant information, we create a supervised approach to learn and detect the\nessential terms in questions.\n  In the second part, we propose two new challenge datasets. In particular, we\ncreate two datasets of natural language questions where (i) the first one\nrequires reasoning over multiple sentences; (ii) the second one requires\ntemporal common sense reasoning. We hope that the two proposed datasets will\nmotivate the field to address more complex problems.\n  In the final part, we present the first formal framework for multi-step\nreasoning algorithms, in the presence of a few important properties of language\nuse, such as incompleteness, ambiguity, etc. We apply this framework to prove\nfundamental limitations for reasoning algorithms. These theoretical results\nprovide extra intuition into the existing empirical evidence in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05067v1",
            "date": "2019-08-14 10:58:14+00:00",
            "title": "Reactive Multi-Stage Feature Fusion for Multimodal Dialogue Modeling",
            "authors": [
                "Yi-Ting Yeh",
                "Tzu-Chuan Lin",
                "Hsiao-Hua Cheng",
                "Yu-Hsuan Deng",
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Visual question answering and visual dialogue tasks have been increasingly\nstudied in the multimodal field towards more practical real-world scenarios. A\nmore challenging task, audio visual scene-aware dialogue (AVSD), is proposed to\nfurther advance the technologies that connect audio, vision, and language,\nwhich introduces temporal video information and dialogue interactions between a\nquestioner and an answerer. This paper proposes an intuitive mechanism that\nfuses features and attention in multiple stages in order to well integrate\nmultimodal features, and the results demonstrate its capability in the\nexperiments. Also, we apply several state-of-the-art models in other tasks to\nthe AVSD task, and further analyze their generalization across different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05098v1",
            "date": "2019-08-14 12:42:48+00:00",
            "title": "Towards Optimisation of Collaborative Question Answering over Knowledge Graphs",
            "authors": [
                "Kuldeep Singh",
                "Mohamad Yaser Jaradeh",
                "Saeedeh Shekarpour",
                "Akash Kulkarni",
                "Arun Sethupat Radhakrishna",
                "Ioanna Lytra",
                "Maria-Esther Vidal",
                "Jens Lehmann"
            ],
            "abstract": "Collaborative Question Answering (CQA) frameworks for knowledge graphs aim at\nintegrating existing question answering (QA) components for implementing\nsequences of QA tasks (i.e. QA pipelines). The research community has paid\nsubstantial attention to CQAs since they support reusability and scalability of\nthe available components in addition to the flexibility of pipelines. CQA\nframeworks attempt to build such pipelines automatically by solving two\noptimisation problems: 1) local collective performance of QA components per QA\ntask and 2) global performance of QA pipelines. In spite offering several\nadvantages over monolithic QA systems, the effectiveness and efficiency of CQA\nframeworks in answering questions is limited. In this paper, we tackle the\nproblem of local optimisation of CQA frameworks and propose a three fold\napproach, which applies feature selection techniques with supervised machine\nlearning approaches in order to identify the best performing components\nefficiently. We have empirically evaluated our approach over existing\nbenchmarks and compared to existing automatic CQA frameworks. The observed\nresults provide evidence that our approach answers a higher number of questions\nthan the state of the art while reducing: i) the number of used features by 50%\nand ii) the number of components used by 76%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05117v3",
            "date": "2019-08-14 13:34:40+00:00",
            "title": "FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension",
            "authors": [
                "Yi-Ting Yeh",
                "Yun-Nung Chen"
            ],
            "abstract": "Conversational machine comprehension requires deep understanding of the\ndialogue flow, and the prior work proposed FlowQA to implicitly model the\ncontext representations in reasoning for better understanding. This paper\nproposes to explicitly model the information gain through dialogue reasoning in\norder to allow the model to focus on more informative cues. The proposed model\nachieves state-of-the-art performance in a conversational QA dataset QuAC and\nsequential instruction understanding dataset SCONE, which shows the\neffectiveness of the proposed mechanism and demonstrates its capability of\ngeneralization to different QA models and tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05138v1",
            "date": "2019-08-14 14:15:24+00:00",
            "title": "MemeFaceGenerator: Adversarial Synthesis of Chinese Meme-face from Natural Sentences",
            "authors": [
                "Yifu Chen",
                "Zongsheng Wang",
                "Bowen Wu",
                "Mengyuan Li",
                "Huan Zhang",
                "Lin Ma",
                "Feng Liu",
                "Qihang Feng",
                "Baoxun Wang"
            ],
            "abstract": "Chinese meme-face is a special kind of internet subculture widely spread in\nChinese Social Community Networks. It usually consists of a template image\nmodified by some amusing details and a text caption. In this paper, we present\nMemeFaceGenerator, a Generative Adversarial Network with the attention module\nand template information as supplementary signals, to automatically generate\nmeme-faces from text inputs. We also develop a web service as system\ndemonstration of meme-face synthesis. MemeFaceGenerator has been shown to be\ncapable of generating high-quality meme-faces from random text inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05344v1",
            "date": "2019-08-14 20:50:14+00:00",
            "title": "Raw-to-End Name Entity Recognition in Social Media",
            "authors": [
                "Liyuan Liu",
                "Zihan Wang",
                "Jingbo Shang",
                "Dandong Yin",
                "Heng Ji",
                "Xiang Ren",
                "Shaowen Wang",
                "Jiawei Han"
            ],
            "abstract": "Taking word sequences as the input, typical named entity recognition (NER)\nmodels neglect errors from pre-processing (e.g., tokenization). However, these\nerrors can influence the model performance greatly, especially for noisy texts\nlike tweets. Here, we introduce Neural-Char-CRF, a raw-to-end framework that is\nmore robust to pre-processing errors. It takes raw character sequences as\ninputs and makes end-to-end predictions. Word embedding and contextualized\nrepresentation models are further tailored to capture textual signals for each\ncharacter instead of each word. Our model neither requires the conversion from\ncharacter sequences to word sequences, nor assumes tokenizer can correctly\ndetect all word boundaries. Moreover, we observe our model performance remains\nunchanged after replacing tokenization with string matching, which demonstrates\nits potential to be tokenization-free. Extensive experimental results on two\npublic datasets demonstrate the superiority of our proposed method over the\nstate of the art. The implementations and datasets are made available at:\nhttps://github.com/LiyuanLucasLiu/Raw-to-End.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05407v1",
            "date": "2019-08-15 03:50:18+00:00",
            "title": "Unpaired Cross-lingual Image Caption Generation with Self-Supervised Rewards",
            "authors": [
                "Yuqing Song",
                "Shizhe Chen",
                "Yida Zhao",
                "Qin Jin"
            ],
            "abstract": "Generating image descriptions in different languages is essential to satisfy\nusers worldwide. However, it is prohibitively expensive to collect large-scale\npaired image-caption dataset for every target language which is critical for\ntraining descent image captioning models. Previous works tackle the unpaired\ncross-lingual image captioning problem through a pivot language, which is with\nthe help of paired image-caption data in the pivot language and pivot-to-target\nmachine translation models. However, such language-pivoted approach suffers\nfrom inaccuracy brought by the pivot-to-target translation, including\ndisfluency and visual irrelevancy errors. In this paper, we propose to generate\ncross-lingual image captions with self-supervised rewards in the reinforcement\nlearning framework to alleviate these two types of errors. We employ\nself-supervision from mono-lingual corpus in the target language to provide\nfluency reward, and propose a multi-level visual semantic matching model to\nprovide both sentence-level and concept-level visual relevancy rewards. We\nconduct extensive experiments for unpaired cross-lingual image captioning in\nboth English and Chinese respectively on two widely used image caption corpora.\nThe proposed approach achieves significant performance improvement over\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.05441v1",
            "date": "2019-08-15 07:00:16+00:00",
            "title": "Multi-class Hierarchical Question Classification for Multiple Choice Science Exams",
            "authors": [
                "Dongfang Xu",
                "Peter Jansen",
                "Jaycie Martin",
                "Zhengnan Xie",
                "Vikas Yadav",
                "Harish Tayyar Madabushi",
                "Oyvind Tafjord",
                "Peter Clark"
            ],
            "abstract": "Prior work has demonstrated that question classification (QC), recognizing\nthe problem domain of a question, can help answer it more accurately. However,\ndeveloping strong QC algorithms has been hindered by the limited size and\ncomplexity of annotated data available. To address this, we present the largest\nchallenge dataset for QC, containing 7,787 science exam questions paired with\ndetailed classification labels from a fine-grained hierarchical taxonomy of 406\nproblem domains. We then show that a BERT-based model trained on this dataset\nachieves a large (+0.12 MAP) gain compared with previous methods, while also\nachieving state-of-the-art performance on benchmark open-domain and biomedical\nQC datasets. Finally, we show that using this model's predictions of question\ntopic significantly improves the accuracy of a question answering system by\n+1.7% P@1, with substantial future gains possible as QC performance improves.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05490v1",
            "date": "2019-08-15 10:58:57+00:00",
            "title": "A Multivariate Model for Representing Semantic Non-compositionality",
            "authors": [
                "Meghdad Farahmand"
            ],
            "abstract": "Semantically non-compositional phrases constitute an intriguing research\ntopic in Natural Language Processing. Semantic non-compositionality --the\nsituation when the meaning of a phrase cannot be derived from the meaning of\nits components, is the main characteristic of such phrases, however, they bear\nother characteristics such as high statistical association and\nnon-substitutability. In this work, we present a model for identifying\nnon-compositional phrases that takes into account all of these characteristics.\nWe show that the presented model remarkably outperforms the existing models of\nidentifying non-compositional phrases that mostly focus only on one of these\ncharacteristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05528v4",
            "date": "2019-08-15 13:26:44+00:00",
            "title": "Vector spaces as Kripke frames",
            "authors": [
                "Giuseppe Greco",
                "Fei Liang",
                "Michael Moortgat",
                "Alessandra Palmigiano",
                "Apostolos Tzimoulis"
            ],
            "abstract": "In recent years, the compositional distributional approach in computational\nlinguistics has opened the way for an integration of the \\emph{lexical} aspects\nof meaning into Lambek's type-logical grammar program. This approach is based\non the observation that a sound semantics for the associative, commutative and\nunital Lambek calculus can be based on vector spaces by interpreting fusion as\nthe tensor product of vector spaces.\n  In this paper, we build on this observation and extend it to a `vector space\nsemantics' for the \\emph{general} Lambek calculus, based on \\emph{algebras over\na field} $\\mathbb{K}$ (or $\\mathbb{K}$-algebras), i.e. vector spaces endowed\nwith a bilinear binary product. Such structures are well known in algebraic\ngeometry and algebraic topology, since they are important instances of Lie\nalgebras and Hopf algebras. Applying results and insights from duality and\nrepresentation theory for the algebraic semantics of nonclassical logics, we\nregard $\\mathbb{K}$-algebras as `Kripke frames' the complex algebras of which\nare complete residuated lattices.\n  This perspective makes it possible to establish a systematic connection\nbetween vector space semantics and the standard Routley-Meyer semantics of\n(modal) substructural logics.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1908.05620v1",
            "date": "2019-08-15 16:11:45+00:00",
            "title": "Visualizing and Understanding the Effectiveness of BERT",
            "authors": [
                "Yaru Hao",
                "Li Dong",
                "Furu Wei",
                "Ke Xu"
            ],
            "abstract": "Language model pre-training, such as BERT, has achieved remarkable results in\nmany NLP tasks. However, it is unclear why the pre-training-then-fine-tuning\nparadigm can improve performance and generalization capability across different\ntasks. In this paper, we propose to visualize loss landscapes and optimization\ntrajectories of fine-tuning BERT on specific datasets. First, we find that\npre-training reaches a good initial point across downstream tasks, which leads\nto wider optima and easier optimization compared with training from scratch. We\nalso demonstrate that the fine-tuning procedure is robust to overfitting, even\nthough BERT is highly over-parameterized for downstream tasks. Second, the\nvisualization results indicate that fine-tuning BERT tends to generalize better\nbecause of the flat and wide optima, and the consistency between the training\nloss surface and the generalization error surface. Third, the lower layers of\nBERT are more invariant during fine-tuning, which suggests that the layers that\nare close to input learn more transferable representations of language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05646v2",
            "date": "2019-08-15 17:20:20+00:00",
            "title": "SenseBERT: Driving Some Sense into BERT",
            "authors": [
                "Yoav Levine",
                "Barak Lenz",
                "Or Dagan",
                "Ori Ram",
                "Dan Padnos",
                "Or Sharir",
                "Shai Shalev-Shwartz",
                "Amnon Shashua",
                "Yoav Shoham"
            ],
            "abstract": "The ability to learn from large unlabeled corpora has allowed neural language\nmodels to advance the frontier in natural language understanding. However,\nexisting self-supervision techniques operate at the word form level, which\nserves as a surrogate for the underlying semantic content. This paper proposes\na method to employ weak-supervision directly at the word sense level. Our\nmodel, named SenseBERT, is pre-trained to predict not only the masked words but\nalso their WordNet supersenses. Accordingly, we attain a lexical-semantic level\nlanguage model, without the use of human annotation. SenseBERT achieves\nsignificantly improved lexical understanding, as we demonstrate by\nexperimenting on SemEval Word Sense Disambiguation, and by attaining a state of\nthe art result on the Word in Context task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05672v4",
            "date": "2019-08-15 03:33:50+00:00",
            "title": "Towards Making the Most of BERT in Neural Machine Translation",
            "authors": [
                "Jiacheng Yang",
                "Mingxuan Wang",
                "Hao Zhou",
                "Chengqi Zhao",
                "Yong Yu",
                "Weinan Zhang",
                "Lei Li"
            ],
            "abstract": "GPT-2 and BERT demonstrate the effectiveness of using pre-trained language\nmodels (LMs) on various natural language processing tasks. However, LM\nfine-tuning often suffers from catastrophic forgetting when applied to\nresource-rich tasks. In this work, we introduce a concerted training framework\n(\\method) that is the key to integrate the pre-trained LMs to neural machine\ntranslation (NMT). Our proposed Cnmt consists of three techniques: a)\nasymptotic distillation to ensure that the NMT model can retain the previous\npre-trained knowledge; b) a dynamic switching gate to avoid catastrophic\nforgetting of pre-trained knowledge; and c) a strategy to adjust the learning\npaces according to a scheduled policy. Our experiments in machine translation\nshow \\method gains of up to 3 BLEU score on the WMT14 English-German language\npair which even surpasses the previous state-of-the-art pre-training aided NMT\nby 1.4 BLEU score. While for the large WMT14 English-French task with 40\nmillions of sentence-pairs, our base model still significantly improves upon\nthe state-of-the-art Transformer big model by more than 1 BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05679v1",
            "date": "2019-08-15 14:08:24+00:00",
            "title": "Transformer-based Automatic Post-Editing with a Context-Aware Encoding Approach for Multi-Source Inputs",
            "authors": [
                "WonKee Lee",
                "Junsu Park",
                "Byung-Hyun Go",
                "Jong-Hyeok Lee"
            ],
            "abstract": "Recent approaches to the Automatic Post-Editing (APE) research have shown\nthat better results are obtained by multi-source models, which jointly encode\nboth source (src) and machine translation output (mt) to produce post-edited\nsentence (pe). Along this trend, we present a new multi-source APE model based\non the Transformer. To construct effective joint representations, our model\ninternally learns to incorporate src context into mt representation. With this\napproach, we achieve a significant improvement over baseline systems, as well\nas the state-of-the-art multi-source APE model. Moreover, to demonstrate the\ncapability of our model to incorporate src context, we show that the word\nalignment of the unknown MT system is successfully captured in our encoding\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05758v1",
            "date": "2019-08-13 03:47:03+00:00",
            "title": "Building a Massive Corpus for Named Entity Recognition using Free Open Data Sources",
            "authors": [
                "Daniel Specht Menezes",
                "Pedro Savarese",
                "Ruy Luiz Milidi\u00fa"
            ],
            "abstract": "With the recent progress in machine learning, boosted by techniques such as\ndeep learning, many tasks can be successfully solved once a large enough\ndataset is available for training. Nonetheless, human-annotated datasets are\noften expensive to produce, especially when labels are fine-grained, as is the\ncase of Named Entity Recognition (NER), a task that operates with labels on a\nword-level.\n  In this paper, we propose a method to automatically generate labeled datasets\nfor NER from public data sources by exploiting links and structured data from\nDBpedia and Wikipedia. Due to the massive size of these data sources, the\nresulting dataset -- SESAME Available at https://sesame-pt.github.io -- is\ncomposed of millions of labeled sentences. We detail the method to generate the\ndataset, report relevant statistics, and design a baseline using a neural\nnetwork, showing that our dataset helps building better NER predictors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05763v3",
            "date": "2019-08-14 14:48:07+00:00",
            "title": "On-Device Text Representations Robust To Misspellings via Projections",
            "authors": [
                "Chinnadhurai Sankar",
                "Sujith Ravi",
                "Zornitsa Kozareva"
            ],
            "abstract": "Recently, there has been a strong interest in developing natural language\napplications that live on personal devices such as mobile phones, watches and\nIoT with the objective to preserve user privacy and have low memory. Advances\nin Locality-Sensitive Hashing (LSH)-based projection networks have demonstrated\nstate-of-the-art performance in various classification tasks without explicit\nword (or word-piece) embedding lookup tables by computing on-the-fly text\nrepresentations. In this paper, we show that the projection based neural\nclassifiers are inherently robust to misspellings and perturbations of the\ninput text. We empirically demonstrate that the LSH projection based\nclassifiers are more robust to common misspellings compared to BiLSTMs (with\nboth word-piece & word-only tokenization) and fine-tuned BERT based methods.\nWhen subject to misspelling attacks, LSH projection based classifiers had a\nsmall average accuracy drop of 2.94% across multiple classifications tasks,\nwhile the fine-tuned BERT model accuracy had a significant drop of 11.44%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05859v3",
            "date": "2019-08-16 06:15:18+00:00",
            "title": "Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbots",
            "authors": [
                "Jia-Chen Gu",
                "Zhen-Hua Ling",
                "Xiaodan Zhu",
                "Quan Liu"
            ],
            "abstract": "This paper proposes a dually interactive matching network (DIM) for\npresenting the personalities of dialogue agents in retrieval-based chatbots.\nThis model develops from the interactive matching network (IMN) which models\nthe matching degree between a context composed of multiple utterances and a\nresponse candidate. Compared with previous persona fusion approaches which\nenhance the representation of a context by calculating its similarity with a\ngiven persona, the DIM model adopts a dual matching architecture, which\nperforms interactive matching between responses and contexts and between\nresponses and personas respectively for ranking response candidates.\nExperimental results on PERSONA-CHAT dataset show that the DIM model\noutperforms its baseline model, i.e., IMN with persona fusion, by a margin of\n14.5% and outperforms the current state-of-the-art model by a margin of 27.7%\nin terms of top-1 accuracy hits@1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06006v1",
            "date": "2019-08-16 15:20:04+00:00",
            "title": "Bidirectional Context-Aware Hierarchical Attention Network for Document Understanding",
            "authors": [
                "Jean-Baptiste Remy",
                "Antoine Jean-Pierre Tixier",
                "Michalis Vazirgiannis"
            ],
            "abstract": "The Hierarchical Attention Network (HAN) has made great strides, but it\nsuffers a major limitation: at level 1, each sentence is encoded in complete\nisolation. In this work, we propose and compare several modifications of HAN in\nwhich the sentence encoder is able to make context-aware attentional decisions\n(CAHAN). Furthermore, we propose a bidirectional document encoder that\nprocesses the document forwards and backwards, using the preceding and\nfollowing sentences as context. Experiments on three large-scale sentiment and\ntopic classification datasets show that the bidirectional version of CAHAN\noutperforms HAN everywhere, with only a modest increase in computation time.\nWhile results are promising, we expect the superiority of CAHAN to be even more\nevident on tasks requiring a deeper understanding of the input documents, such\nas abstractive summarization. Code is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06015v1",
            "date": "2019-08-16 15:21:34+00:00",
            "title": "Automatically Identifying Comparator Groups on Twitter for Digital Epidemiology of Pregnancy Outcomes",
            "authors": [
                "Ari Z. Klein",
                "Abeselom Gebreyesus",
                "Graciela Gonzalez-Hernandez"
            ],
            "abstract": "Despite the prevalence of adverse pregnancy outcomes such as miscarriage,\nstillbirth, birth defects, and preterm birth, their causes are largely unknown.\nWe seek to advance the use of social media for observational studies of\npregnancy outcomes by developing a natural language processing pipeline for\nautomatically identifying users from which to select comparator groups on\nTwitter. We annotated 2361 tweets by users who have announced their pregnancy\non Twitter, which were used to train and evaluate supervised machine learning\nalgorithms as a basis for automatically detecting women who have reported that\ntheir pregnancy had reached term and their baby was born at a normal weight.\nUpon further processing the tweet-level predictions of a majority voting-based\nensemble classifier, the pipeline achieved a user-level F1-score of 0.933, with\na precision of 0.947 and a recall of 0.920. Our pipeline will be deployed to\nidentify large comparator groups for studying pregnancy outcomes on Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06039v3",
            "date": "2019-08-16 15:46:14+00:00",
            "title": "Few-shot Text Classification with Distributional Signatures",
            "authors": [
                "Yujia Bao",
                "Menghua Wu",
                "Shiyu Chang",
                "Regina Barzilay"
            ],
            "abstract": "In this paper, we explore meta-learning for few-shot text classification.\nMeta-learning has shown strong performance in computer vision, where low-level\npatterns are transferable across learning tasks. However, directly applying\nthis approach to text is challenging--lexical features highly informative for\none task may be insignificant for another. Thus, rather than learning solely\nfrom words, our model also leverages their distributional signatures, which\nencode pertinent word occurrence patterns. Our model is trained within a\nmeta-learning framework to map these signatures into attention scores, which\nare then used to weight the lexical representations of words. We demonstrate\nthat our model consistently outperforms prototypical networks learned on\nlexical knowledge (Snell et al., 2017) in both few-shot text classification and\nrelation classification by a significant margin across six benchmark datasets\n(20.0% on average in 1-shot classification).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06121v3",
            "date": "2019-08-16 18:19:59+00:00",
            "title": "CFO: A Framework for Building Production NLP Systems",
            "authors": [
                "Rishav Chakravarti",
                "Cezar Pendus",
                "Andrzej Sakrajda",
                "Anthony Ferritto",
                "Lin Pan",
                "Michael Glass",
                "Vittorio Castelli",
                "J. William Murdock",
                "Radu Florian",
                "Salim Roukos",
                "Avirup Sil"
            ],
            "abstract": "This paper introduces a novel orchestration framework, called CFO\n(COMPUTATION FLOW ORCHESTRATOR), for building, experimenting with, and\ndeploying interactive NLP (Natural Language Processing) and IR (Information\nRetrieval) systems to production environments. We then demonstrate a question\nanswering system built using this framework which incorporates state-of-the-art\nBERT based MRC (Machine Reading Comprehension) with IR components to enable\nend-to-end answer retrieval. Results from the demo system are shown to be high\nquality in both academic and industry domain specific settings. Finally, we\ndiscuss best practices when (pre-)training BERT based MRC models for production\nsystems.",
            "journal ref": "Proceedings of the 2019 Conference on Empirical Methods in Natural\n  Language Processing and the 9th International Joint Conference on Natural\n  Language Processing (EMNLP-IJCNLP): System Demonstrations",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06263v3",
            "date": "2019-08-17 08:40:18+00:00",
            "title": "A Sensitivity Analysis of Attention-Gated Convolutional Neural Networks for Sentence Classification",
            "authors": [
                "Yang Liu",
                "Jianpeng Zhang",
                "Chao Gao",
                "Jinghua Qu",
                "Lixin Ji"
            ],
            "abstract": "In this paper, we investigate the effect of different hyperparameters as well\nas different combinations of hyperparameters settings on the performance of the\nAttention-Gated Convolutional Neural Networks (AGCNNs), e.g., the kernel window\nsize, the number of feature maps, the keep rate of the dropout layer, and the\nactivation function. We draw practical advice from a wide range of empirical\nresults. Through the sensitivity analysis, we further improve the\nhyperparameters settings of AGCNNs. Experiments show that our proposals could\nachieve an average of 0.81% and 0.67% improvements on AGCNN-NLReLU-rand and\nAGCNN-SELU-rand, respectively; and an average of 0.47% and 0.45% improvements\non AGCNN-NLReLU-static and AGCNN-SELU-static, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06327v1",
            "date": "2019-08-17 18:01:27+00:00",
            "title": "Language Features Matter: Effective Language Representations for Vision-Language Tasks",
            "authors": [
                "Andrea Burns",
                "Reuben Tan",
                "Kate Saenko",
                "Stan Sclaroff",
                "Bryan A. Plummer"
            ],
            "abstract": "Shouldn't language and vision features be treated equally in vision-language\n(VL) tasks? Many VL approaches treat the language component as an afterthought,\nusing simple language models that are either built upon fixed word embeddings\ntrained on text-only data or are learned from scratch. We believe that language\nfeatures deserve more attention, and conduct experiments which compare\ndifferent word embeddings, language models, and embedding augmentation steps on\nfive common VL tasks: image-sentence retrieval, image captioning, visual\nquestion answering, phrase grounding, and text-to-clip retrieval. Our\nexperiments provide some striking results; an average embedding language model\noutperforms an LSTM on retrieval-style tasks; state-of-the-art representations\nsuch as BERT perform relatively poorly on vision-language tasks. From this\ncomprehensive set of experiments we propose a set of best practices for\nincorporating the language component of VL tasks. To further elevate language\nfeatures, we also show that knowledge in vision-language problems can be\ntransferred across tasks to gain performance with multi-task training. This\nmulti-task training is applied to a new Graph Oriented Vision-Language\nEmbedding (GrOVLE), which we adapt from Word2Vec using WordNet and an original\nvisual-language graph built from Visual Genome, providing a ready-to-use\nvision-language embedding: http://ai.bu.edu/grovle.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.06379v2",
            "date": "2019-08-18 05:10:59+00:00",
            "title": "Concurrent Parsing of Constituency and Dependency",
            "authors": [
                "Junru Zhou",
                "Shuailiang Zhang",
                "Hai Zhao"
            ],
            "abstract": "Constituent and dependency representation for syntactic structure share a lot\nof linguistic and computational characteristics, this paper thus makes the\nfirst attempt by introducing a new model that is capable of parsing constituent\nand dependency at the same time, so that lets either of the parsers enhance\neach other. Especially, we evaluate the effect of different shared network\ncomponents and empirically verify that dependency parsing may be much more\nbeneficial from constituent parsing structure.\n  The proposed parser achieves new state-of-the-art performance for both\nparsing tasks, constituent and dependency on PTB and CTB benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06435v1",
            "date": "2019-08-18 12:50:47+00:00",
            "title": "TDAM: a Topic-Dependent Attention Model for Sentiment Analysis",
            "authors": [
                "Gabriele Pergola",
                "Lin Gui",
                "Yulan He"
            ],
            "abstract": "We propose a topic-dependent attention model for sentiment classification and\ntopic extraction. Our model assumes that a global topic embedding is shared\nacross documents and employs an attention mechanism to derive local topic\nembedding for words and sentences. These are subsequently incorporated in a\nmodified Gated Recurrent Unit (GRU) for sentiment classification and extraction\nof topics bearing different sentiment polarities. Those topics emerge from the\nwords' local topic embeddings learned by the internal attention of the GRU\ncells in the context of a multi-task learning framework. In this paper, we\npresent the hierarchical architecture, the new GRU unit and the experiments\nconducted on users' reviews which demonstrate classification performance on a\npar with the state-of-the-art methodologies for sentiment classification and\ntopic coherence outperforming the current approaches for supervised topic\nextraction. In addition, our model is able to extract coherent aspect-sentiment\nclusters despite using no aspect-level annotations for training.",
            "journal ref": "Information Processing & Management, 56 (6), 102084, July 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06493v1",
            "date": "2019-08-18 18:09:19+00:00",
            "title": "TwistBytes -- Hierarchical Classification at GermEval 2019: walking the fine line (of recall and precision)",
            "authors": [
                "Fernando Benites"
            ],
            "abstract": "We present here our approach to the GermEval 2019 Task 1 - Shared Task on\nhierarchical classification of German blurbs. We achieved first place in the\nhierarchical subtask B and second place on the root node, flat classification\nsubtask A. In subtask A, we applied a simple multi-feature TF-IDF extraction\nmethod using different n-gram range and stopword removal, on each feature\nextraction module. The classifier on top was a standard linear SVM. For the\nhierarchical classification, we used a local approach, which was more\nlight-weighted but was similar to the one used in subtask A. The key point of\nour approach was the application of a post-processing to cope with the\nmulti-label aspect of the task, increasing the recall but not surpassing the\nprecision measure score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06520v3",
            "date": "2019-08-18 21:46:19+00:00",
            "title": "Modeling Islamist Extremist Communications on Social Media using Contextual Dimensions: Religion, Ideology, and Hate",
            "authors": [
                "Ugur Kursuncu",
                "Manas Gaur",
                "Carlos Castillo",
                "Amanuel Alambo",
                "K. Thirunarayan",
                "Valerie Shalin",
                "Dilshod Achilov",
                "I. Budak Arpinar",
                "Amit Sheth"
            ],
            "abstract": "Terror attacks have been linked in part to online extremist content. Although\ntens of thousands of Islamist extremism supporters consume such content, they\nare a small fraction relative to peaceful Muslims. The efforts to contain the\never-evolving extremism on social media platforms have remained inadequate and\nmostly ineffective. Divergent extremist and mainstream contexts challenge\nmachine interpretation, with a particular threat to the precision of\nclassification algorithms. Our context-aware computational approach to the\nanalysis of extremist content on Twitter breaks down this persuasion process\ninto building blocks that acknowledge inherent ambiguity and sparsity that\nlikely challenge both manual and automated classification. We model this\nprocess using a combination of three contextual dimensions -- religion,\nideology, and hate -- each elucidating a degree of radicalization and\nhighlighting independent features to render them computationally accessible. We\nutilize domain-specific knowledge resources for each of these contextual\ndimensions such as Qur'an for religion, the books of extremist ideologues and\npreachers for political ideology and a social media hate speech corpus for\nhate. Our study makes three contributions to reliable analysis: (i) Development\nof a computational approach rooted in the contextual dimensions of religion,\nideology, and hate that reflects strategies employed by online Islamist\nextremist groups, (ii) An in-depth analysis of relevant tweet datasets with\nrespect to these dimensions to exclude likely mislabeled users, and (iii) A\nframework for understanding online radicalization as a process to assist\ncounter-programming. Given the potentially significant social impact, we\nevaluate the performance of our algorithms to minimize mislabeling, where our\napproach outperforms a competitive baseline by 10.2% in precision.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.06559v1",
            "date": "2019-08-19 02:10:39+00:00",
            "title": "Recurrent Graph Syntax Encoder for Neural Machine Translation",
            "authors": [
                "Liang Ding",
                "Dacheng Tao"
            ],
            "abstract": "Syntax-incorporated machine translation models have been proven successful in\nimproving the model's reasoning and meaning preservation ability. In this\npaper, we propose a simple yet effective graph-structured encoder, the\nRecurrent Graph Syntax Encoder, dubbed \\textbf{RGSE}, which enhances the\nability to capture useful syntactic information. The RGSE is done over a\nstandard encoder (recurrent or self-attention encoder), regarding recurrent\nnetwork units as graph nodes and injects syntactic dependencies as edges, such\nthat RGSE models syntactic dependencies and sequential information\n(\\textit{i.e.}, word order) simultaneously. Our approach achieves considerable\nimprovements over several syntax-aware NMT models in English$\\Rightarrow$German\nand English$\\Rightarrow$Czech translation tasks. And RGSE-equipped big model\nobtains competitive result compared with the state-of-the-art model in WMT14\nEn-De task. Extensive analysis further verifies that RGSE could benefit long\nsentence modeling, and produces better translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06576v1",
            "date": "2019-08-19 03:31:48+00:00",
            "title": "A Co-analysis Framework for Exploring Multivariate Scientific Data",
            "authors": [
                "Xiangyang He",
                "Yubo Tao",
                "Qirui Wang",
                "Hai Lin"
            ],
            "abstract": "In complex multivariate data sets, different features usually include diverse\nassociations with different variables, and different variables are associated\nwithin different regions. Therefore, exploring the associations between\nvariables and voxels locally becomes necessary to better understand the\nunderlying phenomena. In this paper, we propose a co-analysis framework based\non biclusters, which are two subsets of variables and voxels with close\nscalar-value relationships, to guide the process of visually exploring\nmultivariate data. We first automatically extract all meaningful biclusters,\neach of which only contains voxels with a similar scalar-value pattern over a\nsubset of variables. These biclusters are organized according to their variable\nsets, and biclusters in each variable set are further grouped by a similarity\nmetric to reduce redundancy and support diversity during visual exploration.\nBiclusters are visually represented in coordinated views to facilitate\ninteractive exploration of multivariate data based on the similarity between\nbiclusters and the correlation of scalar values with different variables.\nExperiments on several representative multivariate scientific data sets\ndemonstrate the effectiveness of our framework in exploring local relationships\namong variables, biclusters and scalar values in the data.",
            "journal ref": "Visual Informatics Volume 2, Issue 4, December 2018, Pages 254-263",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.06605v2",
            "date": "2019-08-19 06:20:38+00:00",
            "title": "Long and Diverse Text Generation with Planning-based Hierarchical Variational Model",
            "authors": [
                "Zhihong Shao",
                "Minlie Huang",
                "Jiangtao Wen",
                "Wenfei Xu",
                "Xiaoyan Zhu"
            ],
            "abstract": "Existing neural methods for data-to-text generation are still struggling to\nproduce long and diverse texts: they are insufficient to model input data\ndynamically during generation, to capture inter-sentence coherence, or to\ngenerate diversified expressions. To address these issues, we propose a\nPlanning-based Hierarchical Variational Model (PHVM). Our model first plans a\nsequence of groups (each group is a subset of input items to be covered by a\nsentence) and then realizes each sentence conditioned on the planning result\nand the previously generated context, thereby decomposing long text generation\ninto dependent sentence generation sub-tasks. To capture expression diversity,\nwe devise a hierarchical latent structure where a global planning latent\nvariable models the diversity of reasonable planning and a sequence of local\nlatent variables controls sentence realization. Experiments show that our model\noutperforms state-of-the-art baselines in long and diverse text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06625v1",
            "date": "2019-08-19 07:36:19+00:00",
            "title": "Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces",
            "authors": [
                "Barun Patra",
                "Joel Ruben Antony Moniz",
                "Sarthak Garg",
                "Matthew R. Gormley",
                "Graham Neubig"
            ],
            "abstract": "Recent work on bilingual lexicon induction (BLI) has frequently depended\neither on aligned bilingual lexicons or on distribution matching, often with an\nassumption about the isometry of the two spaces. We propose a technique to\nquantitatively estimate this assumption of the isometry between two embedding\nspaces and empirically show that this assumption weakens as the languages in\nquestion become increasingly etymologically distant. We then propose Bilingual\nLexicon Induction with Semi-Supervision (BLISS) --- a semi-supervised approach\nthat relaxes the isometric assumption while leveraging both limited aligned\nbilingual lexicons and a larger set of unaligned word embeddings, as well as a\nnovel hubness filtering technique. Our proposed method obtains state of the art\nresults on 15 of 18 language pairs on the MUSE dataset, and does particularly\nwell when the embedding spaces don't appear to be isometric. In addition, we\nalso show that adding supervision stabilizes the learning procedure, and is\neffective even with minimal supervision.",
            "journal ref": "Proceedings of the 57th Conference of the Association for\n  Computational Linguistics (2019) 184-193",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06629v2",
            "date": "2019-08-19 07:56:46+00:00",
            "title": "Memory limitations are hidden in grammar",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Morten H. Christiansen",
                "Ramon Ferrer-i-Cancho"
            ],
            "abstract": "The ability to produce and understand an unlimited number of different\nsentences is a hallmark of human language. Linguists have sought to define the\nessence of this generative capacity using formal grammars that describe the\nsyntactic dependencies between constituents, independent of the computational\nlimitations of the human brain. Here, we evaluate this independence assumption\nby sampling sentences uniformly from the space of possible syntactic\nstructures. We find that the average dependency distance between syntactically\nrelated words, a proxy for memory limitations, is less than expected by chance\nin a collection of state-of-the-art classes of dependency grammars. Our\nfindings indicate that memory limitations have permeated grammatical\ndescriptions, suggesting that it may be impossible to build a parsimonious\ntheory of human linguistic productivity independent of non-linguistic cognitive\nconstraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06738v1",
            "date": "2019-08-15 12:50:51+00:00",
            "title": "Semantic Source Code Search: A Study of the Past and a Glimpse at the Future",
            "authors": [
                "Muhammad Khalifa"
            ],
            "abstract": "With the recent explosion in the size and complexity of source codebases and\nsoftware projects, the need for efficient source code search engines has\nincreased dramatically. Unfortunately, existing information retrieval-based\nmethods fail to capture the query semantics and perform well only when the\nquery contains syntax-based keywords. Consequently, such methods will perform\npoorly when given high-level natural language queries. In this paper, we review\nexisting methods for building code search engines. We also outline the open\nresearch directions and the various obstacles that stand in the way of having a\nuniversal source code search engine.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1908.06748v2",
            "date": "2019-08-14 05:17:04+00:00",
            "title": "Adabot: Fault-Tolerant Java Decompiler",
            "authors": [
                "Zhiming Li",
                "Qing Wu",
                "Kun Qian"
            ],
            "abstract": "Reverse Engineering(RE) has been a fundamental task in software engineering.\nHowever, most of the traditional Java reverse engineering tools are strictly\nrule defined, thus are not fault-tolerant, which pose serious problem when\nnoise and interference were introduced into the system. In this paper, we view\nreverse engineering as a statistical machine translation task instead of\nrule-based task, and propose a fault-tolerant Java decompiler based on machine\ntranslation models. Our model is based on attention-based Neural Machine\nTranslation (NMT) and Transformer architectures. First, we measure the\ntranslation quality on both the redundant and purified datasets. Next, we\nevaluate the fault-tolerance(anti-noise ability) of our framework on test sets\nwith different unit error probability (UEP). In addition, we compare the\nsuitability of different word segmentation algorithms for decompilation task.\nExperimental results demonstrate that our model is more robust and\nfault-tolerant compared to traditional Abstract Syntax Tree (AST) based\ndecompilers. Specifically, in terms of BLEU-4 and Word Error Rate (WER), our\nperformance has reached 94.50% and 2.65% on the redundant test set; 92.30% and\n3.48% on the purified test set.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1908.06780v1",
            "date": "2019-08-19 13:14:02+00:00",
            "title": "A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints",
            "authors": [
                "Yosi Mass",
                "Haggai Roitman",
                "Shai Erera",
                "Or Rivlin",
                "Bar Weiner",
                "David Konopnicki"
            ],
            "abstract": "We study the use of BERT for non-factoid question-answering, focusing on the\npassage re-ranking task under varying passage lengths. To this end, we explore\nthe fine-tuning of BERT in different learning-to-rank setups, comprising both\npoint-wise and pair-wise methods, resulting in substantial improvements over\nthe state-of-the-art. We then analyze the effectiveness of BERT for different\npassage lengths and suggest how to cope with large passages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.06809v2",
            "date": "2019-08-19 14:01:18+00:00",
            "title": "Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites",
            "authors": [
                "Alexey Tikhonov",
                "Viacheslav Shibaev",
                "Aleksander Nagaev",
                "Aigul Nugmanova",
                "Ivan P. Yamshchikov"
            ],
            "abstract": "This paper shows that standard assessment methodology for style transfer has\nseveral significant problems. First, the standard metrics for style accuracy\nand semantics preservation vary significantly on different re-runs. Therefore\none has to report error margins for the obtained results. Second, starting with\ncertain values of bilingual evaluation understudy (BLEU) between input and\noutput and accuracy of the sentiment transfer the optimization of these two\nstandard metrics diverge from the intuitive goal of the style transfer task.\nFinally, due to the nature of the task itself, there is a specific dependence\nbetween these two metrics that could be easily manipulated. Under these\ncircumstances, we suggest taking BLEU between input and human-written\nreformulations into consideration for benchmarks. We also propose three new\narchitectures that outperform state of the art in terms of this metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06820v1",
            "date": "2019-08-19 14:13:24+00:00",
            "title": "Are You for Real? Detecting Identity Fraud via Dialogue Interactions",
            "authors": [
                "Weikang Wang",
                "Jiajun Zhang",
                "Qian Li",
                "Chengqing Zong",
                "Zhifei Li"
            ],
            "abstract": "Identity fraud detection is of great importance in many real-world scenarios\nsuch as the financial industry. However, few studies addressed this problem\nbefore. In this paper, we focus on identity fraud detection in loan\napplications and propose to solve this problem with a novel interactive\ndialogue system which consists of two modules. One is the knowledge graph (KG)\nconstructor organizing the personal information for each loan applicant. The\nother is structured dialogue management that can dynamically generate a series\nof questions based on the personal KG to ask the applicants and determine their\nidentity states. We also present a heuristic user simulator based on problem\nanalysis to evaluate our method. Experiments have shown that the trainable\ndialogue system can effectively detect fraudsters, and achieve higher\nrecognition accuracy compared with rule-based systems. Furthermore, our learned\ndialogue strategies are interpretable and flexible, which can help promote\nreal-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06893v1",
            "date": "2019-08-19 15:52:36+00:00",
            "title": "Automated email Generation for Targeted Attacks using Natural Language",
            "authors": [
                "Avisha Das",
                "Rakesh Verma"
            ],
            "abstract": "With an increasing number of malicious attacks, the number of people and\norganizations falling prey to social engineering attacks is proliferating.\nDespite considerable research in mitigation systems, attackers continually\nimprove their modus operandi by using sophisticated machine learning, natural\nlanguage processing techniques with an intent to launch successful targeted\nattacks aimed at deceiving detection mechanisms as well as the victims. We\npropose a system for advanced email masquerading attacks using Natural Language\nGeneration (NLG) techniques. Using legitimate as well as an influx of varying\nmalicious content, the proposed deep learning system generates \\textit{fake}\nemails with malicious content, customized depending on the attacker's intent.\nThe system leverages Recurrent Neural Networks (RNNs) for automated text\ngeneration. We also focus on the performance of the generated emails in\ndefeating statistical detectors, and compare and analyze the emails using a\nproposed baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06926v1",
            "date": "2019-08-19 16:54:53+00:00",
            "title": "Neural Architectures for Nested NER through Linearization",
            "authors": [
                "Jana Strakov\u00e1",
                "Milan Straka",
                "Jan Haji\u010d"
            ],
            "abstract": "We propose two neural network architectures for nested named entity\nrecognition (NER), a setting in which named entities may overlap and also be\nlabeled with more than one label. We encode the nested labels using a\nlinearized scheme. In our first proposed approach, the nested labels are\nmodeled as multilabels corresponding to the Cartesian product of the nested\nlabels in a standard LSTM-CRF architecture. In the second one, the nested NER\nis viewed as a sequence-to-sequence problem, in which the input sequence\nconsists of the tokens and output sequence of the labels, using hard attention\non the word whose label is being predicted. The proposed methods outperform the\nnested NER state of the art on four corpora: ACE-2004, ACE-2005, GENIA and\nCzech CNEC. We also enrich our architectures with the recently published\ncontextual embeddings: ELMo, BERT and Flair, reaching further improvements for\nthe four nested entity corpora. In addition, we report flat NER\nstate-of-the-art results for CoNLL-2002 Dutch and Spanish and for CoNLL-2003\nEnglish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07013v1",
            "date": "2019-08-19 18:28:59+00:00",
            "title": "The Natural Selection of Words: Finding the Features of Fitness",
            "authors": [
                "Peter D. Turney",
                "Saif M. Mohammad"
            ],
            "abstract": "We introduce a dataset for studying the evolution of words, constructed from\nWordNet and the Google Books Ngram Corpus. The dataset tracks the evolution of\n4,000 synonym sets (synsets), containing 9,000 English words, from 1800 AD to\n2000 AD. We present a supervised learning algorithm that is able to predict the\nfuture leader of a synset: the word in the synset that will have the highest\nfrequency. The algorithm uses features based on a word's length, the characters\nin the word, and the historical frequencies of the word. It can predict change\nof leadership (including the identity of the new leader) fifty years in the\nfuture, with an F-score considerably above random guessing. Analysis of the\nlearned models provides insight into the causes of change in the leader of a\nsynset. The algorithm confirms observations linguists have made, such as the\ntrend to replace the -ise suffix with -ize, the rivalry between the -ity and\n-ness suffixes, and the struggle between economy (shorter words are easier to\nremember and to write) and clarity (longer words are more distinctive and less\nlikely to be confused with one another). The results indicate that integration\nof the Google Books Ngram Corpus with WordNet has significant potential for\nimproving our understanding of how language evolves.",
            "journal ref": "Published in PLOS ONE, 14(1), e0211512, January 28, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07125v3",
            "date": "2019-08-20 01:51:40+00:00",
            "title": "Universal Adversarial Triggers for Attacking and Analyzing NLP",
            "authors": [
                "Eric Wallace",
                "Shi Feng",
                "Nikhil Kandpal",
                "Matt Gardner",
                "Sameer Singh"
            ],
            "abstract": "Adversarial examples highlight model vulnerabilities and are useful for\nevaluation and interpretation. We define universal adversarial triggers:\ninput-agnostic sequences of tokens that trigger a model to produce a specific\nprediction when concatenated to any input from a dataset. We propose a\ngradient-guided search over tokens which finds short trigger sequences (e.g.,\none word for classification and four words for language modeling) that\nsuccessfully trigger the target prediction. For example, triggers cause SNLI\nentailment accuracy to drop from 89.94% to 0.55%, 72% of \"why\" questions in\nSQuAD to be answered \"to kill american people\", and the GPT-2 language model to\nspew racist output even when conditioned on non-racial contexts. Furthermore,\nalthough the triggers are optimized using white-box access to a specific model,\nthey transfer to other models for all tasks we consider. Finally, since\ntriggers are input-agnostic, they provide an analysis of global model behavior.\nFor instance, they confirm that SNLI models exploit dataset biases and help to\ndiagnose heuristics learned by reading comprehension models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07129v1",
            "date": "2019-08-20 02:07:14+00:00",
            "title": "Zero-Shot Grounding of Objects from Natural Language Queries",
            "authors": [
                "Arka Sadhu",
                "Kan Chen",
                "Ram Nevatia"
            ],
            "abstract": "A phrase grounding system localizes a particular object in an image referred\nto by a natural language query. In previous work, the phrases were restricted\nto have nouns that were encountered in training, we extend the task to\nZero-Shot Grounding(ZSG) which can include novel, \"unseen\" nouns. Current\nphrase grounding systems use an explicit object detection network in a 2-stage\nframework where one stage generates sparse proposals and the other stage\nevaluates them. In the ZSG setting, generating appropriate proposals itself\nbecomes an obstacle as the proposal generator is trained on the entities common\nin the detection and grounding datasets. We propose a new single-stage model\ncalled ZSGNet which combines the detector network and the grounding system and\npredicts classification scores and regression parameters. Evaluation of ZSG\nsystem brings additional subtleties due to the influence of the relationship\nbetween the query and learned categories; we define four distinct conditions\nthat incorporate different levels of difficulty. We also introduce new\ndatasets, sub-sampled from Flickr30k Entities and Visual Genome, that enable\nevaluations for the four conditions. Our experiments show that ZSGNet achieves\nstate-of-the-art performance on Flickr30k and ReferIt under the usual \"seen\"\nsettings and performs significantly better than baseline in the zero-shot\nsetting.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.07137v2",
            "date": "2019-08-20 02:59:37+00:00",
            "title": "Teacher-Student Framework Enhanced Multi-domain Dialogue Generation",
            "authors": [
                "Shuke Peng",
                "Xinjing Huang",
                "Zehao Lin",
                "Feng Ji",
                "Haiqing Chen",
                "Yin Zhang"
            ],
            "abstract": "Dialogue systems dealing with multi-domain tasks are highly required. How to\nrecord the state remains a key problem in a task-oriented dialogue system.\nNormally we use human-defined features as dialogue states and apply a state\ntracker to extract these features. However, the performance of such a system is\nlimited by the error propagation of a state tracker. In this paper, we propose\na dialogue generation model that needs no external state trackers and still\nbenefits from human-labeled semantic data. By using a teacher-student\nframework, several teacher models are firstly trained in their individual\ndomains, learn dialogue policies from labeled states. And then the learned\nknowledge and experience are merged and transferred to a universal student\nmodel, which takes raw utterance as its input. Experiments show that the\ndialogue system trained under our framework outperforms the one uses a belief\ntracker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07141v1",
            "date": "2019-08-20 03:12:13+00:00",
            "title": "LogicENN: A Neural Based Knowledge Graphs Embedding Model with Logical Rules",
            "authors": [
                "Mojtaba Nayyeri",
                "Chengjin Xu",
                "Jens Lehmann",
                "Hamed Shariat Yazdi"
            ],
            "abstract": "Knowledge graph embedding models have gained significant attention in AI\nresearch. Recent works have shown that the inclusion of background knowledge,\nsuch as logical rules, can improve the performance of embeddings in downstream\nmachine learning tasks. However, so far, most existing models do not allow the\ninclusion of rules. We address the challenge of including rules and present a\nnew neural based embedding model (LogicENN). We prove that LogicENN can learn\nevery ground truth of encoded rules in a knowledge graph. To the best of our\nknowledge, this has not been proved so far for the neural based family of\nembedding models. Moreover, we derive formulae for the inclusion of various\nrules, including (anti-)symmetric, inverse, irreflexive and transitive,\nimplication, composition, equivalence and negation. Our formulation allows to\navoid grounding for implication and equivalence relations. Our experiments show\nthat LogicENN outperforms the state-of-the-art models in link prediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1908.07147v2",
            "date": "2019-08-20 03:40:39+00:00",
            "title": "CBOWRA: A Representation Learning Approach for Medication Anomaly Detection",
            "authors": [
                "Liang Zhao",
                "Zhiyuan Ma",
                "Yangming Zhou",
                "Kai Wang",
                "Shengping Liu",
                "Ju Gao"
            ],
            "abstract": "Electronic health record is an important source for clinical researches and\napplications, and errors inevitably occur in the data, which could lead to\nsevere damages to both patients and hospital services. One of such error is the\nmismatches between diagnoses and prescriptions, which we address as 'medication\nanomaly' in the paper, and clinicians used to manually identify and correct\nthem. With the development of machine learning techniques, researchers are able\nto train specific model for the task, but the process still requires expert\nknowledge to construct proper features, and few semantic relations are\nconsidered. In this paper, we propose a simple, yet effective detection method\nthat tackles the problem by detecting the semantic inconsistency between\ndiagnoses and prescriptions. Unlike traditional outlier or anomaly detection,\nthe scheme uses continuous bag of words to construct the semantic connection\nbetween specific central words and their surrounding context. The detection of\nmedication anomaly is transformed into identifying the least possible central\nword based on given context. To help distinguish the anomaly from normal\ncontext, we also incorporate a ranking accumulation strategy. The experiments\nwere conducted on two real hospital electronic medical records, and the topN\naccuracy of the proposed method increased by 3.91 to 10.91% and 0.68 to 2.13%\non the datasets, respectively, which is highly competitive to other traditional\nmachine learning-based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07162v2",
            "date": "2019-08-20 04:32:30+00:00",
            "title": "Discriminative Topic Mining via Category-Name Guided Text Embedding",
            "authors": [
                "Yu Meng",
                "Jiaxin Huang",
                "Guangyuan Wang",
                "Zihan Wang",
                "Chao Zhang",
                "Yu Zhang",
                "Jiawei Han"
            ],
            "abstract": "Mining a set of meaningful and distinctive topics automatically from massive\ntext corpora has broad applications. Existing topic models, however, typically\nwork in a purely unsupervised way, which often generate topics that do not fit\nusers' particular needs and yield suboptimal performance on downstream tasks.\nWe propose a new task, discriminative topic mining, which leverages a set of\nuser-provided category names to mine discriminative topics from text corpora.\nThis new task not only helps a user understand clearly and distinctively the\ntopics he/she is most interested in, but also benefits directly keyword-driven\nclassification tasks. We develop CatE, a novel category-name guided text\nembedding method for discriminative topic mining, which effectively leverages\nminimal user guidance to learn a discriminative embedding space and discover\ncategory representative terms in an iterative manner. We conduct a\ncomprehensive set of experiments to show that CatE mines high-quality set of\ntopics guided by category names only, and benefits a variety of downstream\napplications including weakly-supervised classification and lexical entailment\ndirection identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07181v5",
            "date": "2019-08-20 06:14:18+00:00",
            "title": "Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior",
            "authors": [
                "Raphael Shu",
                "Jason Lee",
                "Hideki Nakayama",
                "Kyunghyun Cho"
            ],
            "abstract": "Although neural machine translation models reached high translation quality,\nthe autoregressive nature makes inference difficult to parallelize and leads to\nhigh translation latency. Inspired by recent refinement-based approaches, we\npropose LaNMT, a latent-variable non-autoregressive model with continuous\nlatent variables and deterministic inference procedure. In contrast to existing\napproaches, we use a deterministic inference algorithm to find the target\nsequence that maximizes the lowerbound to the log-probability. During\ninference, the length of translation automatically adapts itself. Our\nexperiments show that the lowerbound can be greatly increased by running the\ninference algorithm, resulting in significantly improved translation quality.\nOur proposed model closes the performance gap between non-autoregressive and\nautoregressive approaches on ASPEC Ja-En dataset with 8.6x faster decoding. On\nWMT'14 En-De dataset, our model narrows the gap with autoregressive baseline to\n2.0 BLEU points with 12.5x speedup. By decoding multiple initial latent\nvariables in parallel and rescore using a teacher model, the proposed model\nfurther brings the gap down to 1.0 BLEU point on WMT'14 En-De task with 6.8x\nspeedup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07195v1",
            "date": "2019-08-20 07:25:14+00:00",
            "title": "ARAML: A Stable Adversarial Training Framework for Text Generation",
            "authors": [
                "Pei Ke",
                "Fei Huang",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Most of the existing generative adversarial networks (GAN) for text\ngeneration suffer from the instability of reinforcement learning training\nalgorithms such as policy gradient, leading to unstable performance. To tackle\nthis problem, we propose a novel framework called Adversarial Reward Augmented\nMaximum Likelihood (ARAML). During adversarial training, the discriminator\nassigns rewards to samples which are acquired from a stationary distribution\nnear the data rather than the generator's distribution. The generator is\noptimized with maximum likelihood estimation augmented by the discriminator's\nrewards instead of policy gradient. Experiments show that our model can\noutperform state-of-the-art text GANs with a more stable training process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07347v3",
            "date": "2019-08-20 13:49:30+00:00",
            "title": "Density Matrices with Metric for Derivational Ambiguity",
            "authors": [
                "Adriana D. Correia",
                "Michael Moortgat",
                "Henk T. C. Stoof"
            ],
            "abstract": "Recent work on vector-based compositional natural language semantics has\nproposed the use of density matrices to model lexical ambiguity and (graded)\nentailment (e.g. Piedeleu et al 2015, Bankova et al 2019, Sadrzadeh et al\n2018). Ambiguous word meanings, in this work, are represented as mixed states,\nand the compositional interpretation of phrases out of their constituent parts\ntakes the form of a strongly monoidal functor sending the derivational\nmorphisms of a pregroup syntax to linear maps in FdHilb. Our aims in this paper\nare threefold. Firstly, we replace the pregroup front end by a Lambek\ncategorial grammar with directional implications expressing a word's\nselectional requirements. By the Curry-Howard correspondence, the derivations\nof the grammar's type logic are associated with terms of the (ordered) linear\nlambda calculus; these terms can be read as programs for compositional meaning\nassembly with density matrices as the target semantic spaces. Secondly, we\nextend on the existing literature and introduce a symmetric, nondegenerate\nbilinear form called a \"metric\" that defines a canonical isomorphism between a\nvector space and its dual, allowing us to keep a distinction between left and\nright implication. Thirdly, we use this metric to define density matrix spaces\nin a directional form, modeling the ubiquitous derivational ambiguity of\nnatural language syntax, and show how this alows an integrated treatment of\nlexical and derivational forms of ambiguity controlled at the level of the\ninterpretation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07599v3",
            "date": "2019-08-20 20:31:51+00:00",
            "title": "Learning document embeddings along with their uncertainties",
            "authors": [
                "Santosh Kesiraju",
                "Old\u0159ich Plchot",
                "Luk\u00e1\u0161 Burget",
                "Suryakanth V Gangashetty"
            ],
            "abstract": "Majority of the text modelling techniques yield only point-estimates of\ndocument embeddings and lack in capturing the uncertainty of the estimates.\nThese uncertainties give a notion of how well the embeddings represent a\ndocument. We present Bayesian subspace multinomial model (Bayesian SMM), a\ngenerative log-linear model that learns to represent documents in the form of\nGaussian distributions, thereby encoding the uncertainty in its co-variance.\nAdditionally, in the proposed Bayesian SMM, we address a commonly encountered\nproblem of intractability that appears during variational inference in\nmixed-logit models. We also present a generative Gaussian linear classifier for\ntopic identification that exploits the uncertainty in document embeddings. Our\nintrinsic evaluation using perplexity measure shows that the proposed Bayesian\nSMM fits the data better as compared to the state-of-the-art neural variational\ndocument model on Fisher speech and 20Newsgroups text corpora. Our topic\nidentification experiments show that the proposed systems are robust to\nover-fitting on unseen test data. The topic ID results show that the proposed\nmodel is outperforms state-of-the-art unsupervised topic models and achieve\ncomparable results to the state-of-the-art fully supervised discriminative\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07724v4",
            "date": "2019-08-21 07:12:31+00:00",
            "title": "Restricted Recurrent Neural Networks",
            "authors": [
                "Enmao Diao",
                "Jie Ding",
                "Vahid Tarokh"
            ],
            "abstract": "Recurrent Neural Network (RNN) and its variations such as Long Short-Term\nMemory (LSTM) and Gated Recurrent Unit (GRU), have become standard building\nblocks for learning online data of sequential nature in many research areas,\nincluding natural language processing and speech data analysis. In this paper,\nwe present a new methodology to significantly reduce the number of parameters\nin RNNs while maintaining performance that is comparable or even better than\nclassical RNNs. The new proposal, referred to as Restricted Recurrent Neural\nNetwork (RRNN), restricts the weight matrices corresponding to the input data\nand hidden states at each time step to share a large proportion of parameters.\nThe new architecture can be regarded as a compression of its classical\ncounterpart, but it does not require pre-training or sophisticated parameter\nfine-tuning, both of which are major issues in most existing compression\ntechniques. Experiments on natural language modeling show that compared with\nits classical counterpart, the restricted recurrent architecture generally\nproduces comparable results at about 50\\% compression rate. In particular, the\nRestricted LSTM can outperform classical RNN with even less number of\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07810v1",
            "date": "2019-08-21 12:15:35+00:00",
            "title": "Improving Captioning for Low-Resource Languages by Cycle Consistency",
            "authors": [
                "Yike Wu",
                "Shiwan Zhao",
                "Jia Chen",
                "Ying Zhang",
                "Xiaojie Yuan",
                "Zhong Su"
            ],
            "abstract": "Improving the captioning performance on low-resource languages by leveraging\nEnglish caption datasets has received increasing research interest in recent\nyears. Existing works mainly fall into two categories: translation-based and\nalignment-based approaches. In this paper, we propose to combine the merits of\nboth approaches in one unified architecture. Specifically, we use a pre-trained\nEnglish caption model to generate high-quality English captions, and then take\nboth the image and generated English captions to generate low-resource language\ncaptions. We improve the captioning performance by adding the cycle consistency\nconstraint on the cycle of image regions, English words, and low-resource\nlanguage words. Moreover, our architecture has a flexible design which enables\nit to benefit from large monolingual English caption datasets. Experimental\nresults demonstrate that our approach outperforms the state-of-the-art methods\non common evaluation metrics. The attention visualization also shows that the\nproposed approach really improves the fine-grained alignment between words and\nimage regions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07819v2",
            "date": "2019-08-21 15:18:10+00:00",
            "title": "Rating for Parents: Predicting Children Suitability Rating for Movies Based on Language of the Movies",
            "authors": [
                "Mahsa Shafaei",
                "Niloofar Safi Samghabadi",
                "Sudipta Kar",
                "Thamar Solorio"
            ],
            "abstract": "The film culture has grown tremendously in recent years. The large number of\nstreaming services put films as one of the most convenient forms of\nentertainment in today's world. Films can help us learn and inspire societal\nchange. But they can also negatively affect viewers. In this paper, our goal is\nto predict the suitability of the movie content for children and young adults\nbased on scripts. The criterion that we use to measure suitability is the MPAA\nrating that is specifically designed for this purpose. We propose an RNN based\narchitecture with attention that jointly models the genre and the emotions in\nthe script to predict the MPAA rating. We achieve 78% weighted F1-score for the\nclassification model that outperforms the traditional machine learning method\nby 6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07820v2",
            "date": "2019-08-16 03:16:40+00:00",
            "title": "Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing",
            "authors": [
                "Jianquan Li",
                "Xiaokang Liu",
                "Wenpeng Yin",
                "Min Yang",
                "Liqun Ma",
                "Yaohong Jin"
            ],
            "abstract": "Multi-Task Learning (MTL) aims at boosting the overall performance of each\nindividual task by leveraging useful information contained in multiple related\ntasks. It has shown great success in natural language processing (NLP).\nCurrently, a number of MLT architectures and learning mechanisms have been\nproposed for various NLP tasks. However, there is no systematic exploration and\ncomparison of different MLT architectures and learning mechanisms for their\nstrong performance in-depth. In this paper, we conduct a thorough examination\nof typical MTL methods on a broad range of representative NLP tasks. Our\nprimary goal is to understand the merits and demerits of existing MTL methods\nin NLP tasks, thus devising new hybrid architectures intended to combine their\nstrengths.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07912v1",
            "date": "2019-08-19 19:52:50+00:00",
            "title": "It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Prediction",
            "authors": [
                "Slavena Vasileva",
                "Pepa Atanasova",
                "Llu\u00eds M\u00e0rquez",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Preslav Nakov"
            ],
            "abstract": "We propose a multi-task deep-learning approach for estimating the\ncheck-worthiness of claims in political debates. Given a political debate, such\nas the 2016 US Presidential and Vice-Presidential ones, the task is to predict\nwhich statements in the debate should be prioritized for fact-checking. While\ndifferent fact-checking organizations would naturally make different choices\nwhen analyzing the same debate, we show that it pays to learn from multiple\nsources simultaneously (PolitiFact, FactCheck, ABC, CNN, NPR, NYT, Chicago\nTribune, The Guardian, and Washington Post) in a multi-task learning setup,\neven when a particular source is chosen as a target to imitate. Our evaluation\nshows state-of-the-art results on a standard dataset for the task of\ncheck-worthiness prediction.",
            "journal ref": "RANLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08167v2",
            "date": "2019-08-22 02:00:53+00:00",
            "title": "Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering",
            "authors": [
                "Zhiguo Wang",
                "Patrick Ng",
                "Xiaofei Ma",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "BERT model has been successfully applied to open-domain QA tasks. However,\nprevious work trains BERT by viewing passages corresponding to the same\nquestion as independent training instances, which may cause incomparable scores\nfor answers from different passages. To tackle this issue, we propose a\nmulti-passage BERT model to globally normalize answer scores across all\npassages of the same question, and this change enables our QA model find better\nanswers by utilizing more passages. In addition, we find that splitting\narticles into passages with the length of 100 words by sliding window improves\nperformance by 4%. By leveraging a passage ranker to select high-quality\npassages, multi-passage BERT gains additional 2%. Experiments on four standard\nbenchmarks showed that our multi-passage BERT outperforms all state-of-the-art\nmodels on all benchmarks. In particular, on the OpenSQuAD dataset, our model\ngains 21.4% EM and 21.5% $F_1$ over all non-BERT models, and 5.8% EM and 6.5%\n$F_1$ over BERT-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08345v2",
            "date": "2019-08-22 12:59:40+00:00",
            "title": "Text Summarization with Pretrained Encoders",
            "authors": [
                "Yang Liu",
                "Mirella Lapata"
            ],
            "abstract": "Bidirectional Encoder Representations from Transformers (BERT) represents the\nlatest incarnation of pretrained language models which have recently advanced a\nwide range of natural language processing tasks. In this paper, we showcase how\nBERT can be usefully applied in text summarization and propose a general\nframework for both extractive and abstractive models. We introduce a novel\ndocument-level encoder based on BERT which is able to express the semantics of\na document and obtain representations for its sentences. Our extractive model\nis built on top of this encoder by stacking several inter-sentence Transformer\nlayers. For abstractive summarization, we propose a new fine-tuning schedule\nwhich adopts different optimizers for the encoder and the decoder as a means of\nalleviating the mismatch between the two (the former is pretrained while the\nlatter is not). We also demonstrate that a two-staged fine-tuning approach can\nfurther boost the quality of the generated summaries. Experiments on three\ndatasets show that our model achieves state-of-the-art results across the board\nin both extractive and abstractive settings. Our code is available at\nhttps://github.com/nlpyang/PreSumm",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08527v1",
            "date": "2019-08-22 17:58:52+00:00",
            "title": "ViCo: Word Embeddings from Visual Co-occurrences",
            "authors": [
                "Tanmay Gupta",
                "Alexander Schwing",
                "Derek Hoiem"
            ],
            "abstract": "We propose to learn word embeddings from visual co-occurrences. Two words\nco-occur visually if both words apply to the same image or image region.\nSpecifically, we extract four types of visual co-occurrences between object and\nattribute words from large-scale, textually-annotated visual databases like\nVisualGenome and ImageNet. We then train a multi-task log-bilinear model that\ncompactly encodes word \"meanings\" represented by each co-occurrence type into a\nsingle visual word-vector. Through unsupervised clustering, supervised\npartitioning, and a zero-shot-like generalization analysis we show that our\nword embeddings complement text-only embeddings like GloVe by better\nrepresenting similarities and differences between visual concepts that are\ndifficult to obtain from text corpora alone. We further evaluate our embeddings\non five downstream applications, four of which are vision-language tasks.\nAugmenting GloVe with our embeddings yields gains on all tasks. We also find\nthat random embeddings perform comparably to learned embeddings on all\nsupervised vision-language tasks, contrary to conventional wisdom.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08594v3",
            "date": "2019-08-23 00:58:21+00:00",
            "title": "Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model",
            "authors": [
                "Matthias von Davier"
            ],
            "abstract": "This article describes new results of an application using transformer-based\nlanguage models to automated item generation (AIG), an area of ongoing interest\nin the domain of certification testing as well as in educational measurement\nand psychological testing. OpenAI's gpt2 pre-trained 345M parameter language\nmodel was retrained using the public domain text mining set of PubMed articles\nand subsequently used to generate item stems (case vignettes) as well as\ndistractor proposals for multiple-choice items. This case study shows promise\nand produces draft text that can be used by human item writers as input for\nauthoring. Future experiments with more recent transformer models (such as\nGrover, TransformerXL) using existing item pools are expected to improve\nresults further and to facilitate the development of assessment materials.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08788v2",
            "date": "2019-08-22 17:23:29+00:00",
            "title": "When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification",
            "authors": [
                "Shumin Deng",
                "Ningyu Zhang",
                "Zhanlin Sun",
                "Jiaoyan Chen",
                "Huajun Chen"
            ],
            "abstract": "Text classification tends to be difficult when data are deficient or when it\nis required to adapt to unseen classes. In such challenging scenarios, recent\nstudies have often used meta-learning to simulate the few-shot task, thus\nnegating implicit common linguistic features across tasks. This paper addresses\nsuch problems using meta-learning and unsupervised language models. Our\napproach is based on the insight that having a good generalization from a few\nexamples relies on both a generic model initialization and an effective\nstrategy for adapting this model to newly arising tasks. We show that our\napproach is not only simple but also produces a state-of-the-art performance on\na well-studied sentiment classification dataset. It can thus be further\nsuggested that pretraining could be a promising solution for few-shot learning\nof many other NLP tasks. The code and the dataset to replicate the experiments\nare made available at https://github.com/zxlzr/FewShotNLP.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.08917v1",
            "date": "2019-08-20 05:27:34+00:00",
            "title": "A Lost Croatian Cybernetic Machine Translation Program",
            "authors": [
                "Sandro Skansi",
                "Leo Mr\u0161i\u0107",
                "Ines Skelac"
            ],
            "abstract": "We are exploring the historical significance of research in the field of\nmachine translation conducted by Bulcsu Laszlo, Croatian linguist, who was a\npioneer in machine translation in Yugoslavia during the 1950s. We are focused\non two important seminal papers written by members of his research group from\n1959 and 1962, as well as their legacy in establishing a Croatian machine\ntranslation program based around the Faculty of Humanities and Social Sciences\nof the University of Zagreb in the late 1950s and early 1960s. We are exploring\ntheir work in connection with the beginnings of machine translation in the USA\nand USSR, motivated by the Cold War and the intelligence needs of the period.\nWe also present the approach to machine translation advocated by the Croatian\ngroup in Yugoslavia, which is different from the usual logical approaches of\nthe period, and his advocacy of cybernetic methods, which would be adopted as a\ncanon by the mainstream AI community only decades later.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1908.09080v5",
            "date": "2019-08-24 03:10:38+00:00",
            "title": "DAST Model: Deciding About Semantic Complexity of a Text",
            "authors": [
                "MohammadReza Besharati",
                "Mohammad Izadi"
            ],
            "abstract": "Measuring text complexity is an essential task in several fields and\napplications (such as NLP, semantic web, smart education, etc.). The semantic\nlayer of text is more tacit than its syntactic structure and, as a result,\ncalculation of semantic complexity is more difficult than syntactic complexity.\nWhile there are famous and powerful academic and commercial syntactic\ncomplexity measures, the problem of measuring semantic complexity is still a\nchallenging one. In this paper, we introduce the DAST model, which stands for\nDeciding About Semantic Complexity of a Text. DAST proposes an intuitionistic\napproach to semantics that lets us have a well-defined model for the semantics\nof a text and its complexity: semantic is considered as a lattice of intuitions\nand, as a result, semantic complexity is defined as the result of a calculation\non this lattice. A set theoretic formal definition of semantic complexity, as a\n6-tuple formal system, is provided. By using this formal system, a method for\nmeasuring semantic complexity is presented. The evaluation of the proposed\napproach is done by a set of three human-judgment experiments. The results show\nthat DAST model is capable of deciding about semantic complexity of text.\nFurthermore, the analysis of the results leads us to introduce a Markovian\nmodel for the process of common-sense, multiple-steps and semantic-complexity\nreasoning in people. The results of Experiments demonstrate that our method\noutperforms the random baseline with improvement in better precision and\ncompetes with other methods by less error percentage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09119v1",
            "date": "2019-08-24 10:05:40+00:00",
            "title": "Automatic Text Summarization of Legal Cases: A Hybrid Approach",
            "authors": [
                "Varun Pandya"
            ],
            "abstract": "Manual Summarization of large bodies of text involves a lot of human effort\nand time, especially in the legal domain. Lawyers spend a lot of time preparing\nlegal briefs of their clients' case files. Automatic Text summarization is a\nconstantly evolving field of Natural Language Processing(NLP), which is a\nsubdiscipline of the Artificial Intelligence Field. In this paper a hybrid\nmethod for automatic text summarization of legal cases using k-means clustering\ntechnique and tf-idf(term frequency-inverse document frequency) word vectorizer\nis proposed. The summary generated by the proposed method is compared using\nROGUE evaluation parameters with the case summary as prepared by the lawyer for\nappeal in court. Further, suggestions for improving the proposed method are\nalso presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09128v1",
            "date": "2019-08-24 11:40:08+00:00",
            "title": "Enhancing Neural Sequence Labeling with Position-Aware Self-Attention",
            "authors": [
                "Wei Wei",
                "Zanbo Wang",
                "Xianling Mao",
                "Guangyou Zhou",
                "Pan Zhou",
                "Sheng Jiang"
            ],
            "abstract": "Sequence labeling is a fundamental task in natural language processing and\nhas been widely studied. Recently, RNN-based sequence labeling models have\nincreasingly gained attentions. Despite superior performance achieved by\nlearning the long short-term (i.e., successive) dependencies, the way of\nsequentially processing inputs might limit the ability to capture the\nnon-continuous relations over tokens within a sentence. To tackle the problem,\nwe focus on how to effectively model successive and discrete dependencies of\neach token for enhancing the sequence labeling performance. Specifically, we\npropose an innovative and well-designed attention-based model (called\nposition-aware self-attention, i.e., PSA) within a neural network architecture,\nto explore the positional information of an input sequence for capturing the\nlatent relations among tokens. Extensive experiments on three classical tasks\nin sequence labeling domain, i.e., part-of-speech (POS) tagging, named entity\nrecognition (NER) and phrase chunking, demonstrate our proposed model\noutperforms the state-of-the-arts without any external knowledge, in terms of\nvarious metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09156v1",
            "date": "2019-08-24 15:52:57+00:00",
            "title": "A framework for anomaly detection using language modeling, and its applications to finance",
            "authors": [
                "Armineh Nourbakhsh",
                "Grace Bang"
            ],
            "abstract": "In the finance sector, studies focused on anomaly detection are often\nassociated with time-series and transactional data analytics. In this paper, we\nlay out the opportunities for applying anomaly and deviation detection methods\nto text corpora and challenges associated with them. We argue that language\nmodels that use distributional semantics can play a significant role in\nadvancing these studies in novel directions, with new applications in risk\nidentification, predictive modeling, and trend analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09282v3",
            "date": "2019-08-25 08:57:50+00:00",
            "title": "Don't Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja",
            "authors": [
                "Kang Min Yoo",
                "Taeuk Kim",
                "Sang-goo Lee"
            ],
            "abstract": "We propose a simple yet effective approach for improving Korean word\nrepresentations using additional linguistic annotation (i.e. Hanja). We employ\ncross-lingual transfer learning in training word representations by leveraging\nthe fact that Hanja is closely related to Chinese. We evaluate the intrinsic\nquality of representations learned through our approach using the word analogy\nand similarity tests. In addition, we demonstrate their effectiveness on\nseveral downstream tasks, including a novel Korean news headline generation\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09324v1",
            "date": "2019-08-25 13:27:57+00:00",
            "title": "Multilingual Neural Machine Translation with Language Clustering",
            "authors": [
                "Xu Tan",
                "Jiale Chen",
                "Di He",
                "Yingce Xia",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Multilingual neural machine translation (NMT), which translates multiple\nlanguages using a single model, is of great practical importance due to its\nadvantages in simplifying the training process, reducing online maintenance\ncosts, and enhancing low-resource and zero-shot translation. Given there are\nthousands of languages in the world and some of them are very different, it is\nextremely burdensome to handle them all in a single model or use a separate\nmodel for each language pair. Therefore, given a fixed resource budget, e.g.,\nthe number of models, how to determine which languages should be supported by\none model is critical to multilingual NMT, which, unfortunately, has been\nignored by previous work. In this work, we develop a framework that clusters\nlanguages into different groups and trains one multilingual model for each\ncluster. We study two methods for language clustering: (1) using prior\nknowledge, where we cluster languages according to language family, and (2)\nusing language embedding, in which we represent each language by an embedding\nvector and cluster them in the embedding space. In particular, we obtain the\nembedding vectors of all the languages by training a universal neural machine\ntranslation model. Our experiments on 23 languages show that the first\nclustering method is simple and easy to understand but leading to suboptimal\ntranslation accuracy, while the second method sufficiently captures the\nrelationship among languages well and improves the translation accuracy for\nalmost all the languages over baseline methods",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09329v1",
            "date": "2019-08-25 13:59:03+00:00",
            "title": "Efficient Bidirectional Neural Machine Translation",
            "authors": [
                "Xu Tan",
                "Yingce Xia",
                "Lijun Wu",
                "Tao Qin"
            ],
            "abstract": "The encoder-decoder based neural machine translation usually generates a\ntarget sequence token by token from left to right. Due to error propagation,\nthe tokens in the right side of the generated sequence are usually of poorer\nquality than those in the left side. In this paper, we propose an efficient\nmethod to generate a sequence in both left-to-right and right-to-left manners\nusing a single encoder and decoder, combining the advantages of both generation\ndirections. Experiments on three translation tasks show that our method\nachieves significant improvements over conventional unidirectional approach.\nCompared with ensemble methods that train and combine two models with different\ngeneration directions, our method saves 50% model parameters and about 40%\ntraining time, and also improve inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09355v1",
            "date": "2019-08-25 16:13:24+00:00",
            "title": "Patient Knowledge Distillation for BERT Model Compression",
            "authors": [
                "Siqi Sun",
                "Yu Cheng",
                "Zhe Gan",
                "Jingjing Liu"
            ],
            "abstract": "Pre-trained language models such as BERT have proven to be highly effective\nfor natural language processing (NLP) tasks. However, the high demand for\ncomputing resources in training such models hinders their application in\npractice. In order to alleviate this resource hunger in large-scale model\ntraining, we propose a Patient Knowledge Distillation approach to compress an\noriginal large model (teacher) into an equally-effective lightweight shallow\nnetwork (student). Different from previous knowledge distillation methods,\nwhich only use the output from the last layer of the teacher network for\ndistillation, our student model patiently learns from multiple intermediate\nlayers of the teacher model for incremental knowledge extraction, following two\nstrategies: ($i$) PKD-Last: learning from the last $k$ layers; and ($ii$)\nPKD-Skip: learning from every $k$ layers. These two patient distillation\nschemes enable the exploitation of rich information in the teacher's hidden\nlayers, and encourage the student model to patiently learn from and imitate the\nteacher through a multi-layer distillation process. Empirically, this\ntranslates into improved results on multiple NLP tasks with significant gain in\ntraining efficiency, without sacrificing model accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09368v1",
            "date": "2019-08-25 17:34:40+00:00",
            "title": "Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer",
            "authors": [
                "Akhilesh Sudhakar",
                "Bhargav Upadhyay",
                "Arjun Maheswaran"
            ],
            "abstract": "Text style transfer is the task of transferring the style of text having\ncertain stylistic attributes, while preserving non-stylistic or content\ninformation. In this work we introduce the Generative Style Transformer (GST) -\na new approach to rewriting sentences to a target style in the absence of\nparallel style corpora. GST leverages the power of both, large unsupervised\npre-trained language models as well as the Transformer. GST is a part of a\nlarger `Delete Retrieve Generate' framework, in which we also propose a novel\nmethod of deleting style attributes from the source sentence by exploiting the\ninner workings of the Transformer. Our models outperform state-of-art systems\nacross 5 datasets on sentiment, gender and political slant transfer. We also\npropose the use of the GLEU metric as an automatic metric of evaluation of\nstyle transfer, which we found to compare better with human ratings than the\npredominantly used BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09369v3",
            "date": "2019-08-25 17:50:18+00:00",
            "title": "On Measuring and Mitigating Biased Inferences of Word Embeddings",
            "authors": [
                "Sunipa Dev",
                "Tao Li",
                "Jeff Phillips",
                "Vivek Srikumar"
            ],
            "abstract": "Word embeddings carry stereotypical connotations from the text they are\ntrained on, which can lead to invalid inferences in downstream models that rely\non them. We use this observation to design a mechanism for measuring\nstereotypes using the task of natural language inference. We demonstrate a\nreduction in invalid inferences via bias mitigation strategies on static word\nembeddings (GloVe). Further, we show that for gender bias, these techniques\nextend to contextualized embeddings when applied selectively only to the static\ncomponents of contextualized embeddings (ELMo, BERT).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09507v1",
            "date": "2019-08-26 07:40:33+00:00",
            "title": "Partially-supervised Mention Detection",
            "authors": [
                "Lesly Miculicich",
                "James Henderson"
            ],
            "abstract": "Learning to detect entity mentions without using syntactic information can be\nuseful for integration and joint optimization with other tasks. However, it is\ncommon to have partially annotated data for this problem. Here, we investigate\ntwo approaches to deal with partial annotation of mentions: weighted loss and\nsoft-target classification. We also propose two neural mention detection\napproaches: a sequence tagging, and an exhaustive search. We evaluate our\nmethods with coreference resolution as a downstream task, using multitask\nlearning. The results show that the recall and F1 score improve for all\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09528v2",
            "date": "2019-08-26 08:52:33+00:00",
            "title": "Thinking Globally, Acting Locally: Distantly Supervised Global-to-Local Knowledge Selection for Background Based Conversation",
            "authors": [
                "Pengjie Ren",
                "Zhumin Chen",
                "Christof Monz",
                "Jun Ma",
                "Maarten de Rijke"
            ],
            "abstract": "Background Based Conversations (BBCs) have been introduced to help\nconversational systems avoid generating overly generic responses. In a BBC, the\nconversation is grounded in a knowledge source. A key challenge in BBCs is\nKnowledge Selection (KS): given a conversational context, try to find the\nappropriate background knowledge (a text fragment containing related facts or\ncomments, etc.) based on which to generate the next response. Previous work\naddresses KS by employing attention and/or pointer mechanisms. These mechanisms\nuse a local perspective, i.e., they select a token at a time based solely on\nthe current decoding state. We argue for the adoption of a global perspective,\ni.e., pre-selecting some text fragments from the background knowledge that\ncould help determine the topic of the next response. We enhance KS in BBCs by\nintroducing a Global-to-Local Knowledge Selection (GLKS) mechanism. Given a\nconversational context and background knowledge, we first learn a topic\ntransition vector to encode the most likely text fragments to be used in the\nnext response, which is then used to guide the local KS at each decoding\ntimestamp. In order to effectively learn the topic transition vector, we\npropose a distantly supervised learning schema. Experimental results show that\nthe GLKS model significantly outperforms state-of-the-art methods in terms of\nboth automatic and human evaluation. More importantly, GLKS achieves this\nwithout requiring any extra annotations, which demonstrates its high degree of\nscalability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09785v1",
            "date": "2019-08-26 16:37:03+00:00",
            "title": "Detecting Toxicity in News Articles: Application to Bulgarian",
            "authors": [
                "Yoan Dinkov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Online media aim for reaching ever bigger audience and for attracting ever\nlonger attention span. This competition creates an environment that rewards\nsensational, fake, and toxic news. To help limit their spread and impact, we\npropose and develop a news toxicity detector that can recognize various types\nof toxic content. While previous research primarily focused on English, here we\ntarget Bulgarian. We created a new dataset by crawling a website that for five\nyears has been collecting Bulgarian news articles that were manually\ncategorized into eight toxicity groups. Then we trained a multi-class\nclassifier with nine categories: eight toxic and one non-toxic. We experimented\nwith different representations based on ElMo, BERT, and XLM, as well as with a\nvariety of domain-specific features. Due to the small size of our dataset, we\ncreated a separate model for each feature type, and we ultimately combined\nthese models into a meta-classifier. The evaluation results show an accuracy of\n59.0% and a macro-F1 score of 39.7%, which represent sizable improvements over\nthe majority-class baseline (Acc=30.3%, macro-F1=5.2%).",
            "journal ref": "RANLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09805v2",
            "date": "2019-08-26 17:23:22+00:00",
            "title": "The Limitations of Stylometry for Detecting Machine-Generated Fake News",
            "authors": [
                "Tal Schuster",
                "Roei Schuster",
                "Darsh J Shah",
                "Regina Barzilay"
            ],
            "abstract": "Recent developments in neural language models (LMs) have raised concerns\nabout their potential misuse for automatically spreading misinformation. In\nlight of these concerns, several studies have proposed to detect\nmachine-generated fake news by capturing their stylistic differences from\nhuman-written text. These approaches, broadly termed stylometry, have found\nsuccess in source attribution and misinformation detection in human-written\ntexts. However, in this work, we show that stylometry is limited against\nmachine-generated misinformation. While humans speak differently when trying to\ndeceive, LMs generate stylistically consistent text, regardless of underlying\nmotive. Thus, though stylometry can successfully prevent impersonation by\nidentifying text provenance, it fails to distinguish legitimate LM applications\nfrom those that introduce false information. We create two benchmarks\ndemonstrating the stylistic similarity between malicious and legitimate uses of\nLMs, employed in auto-completion and editing-assistance settings. Our findings\nhighlight the need for non-stylometry approaches in detecting machine-generated\nmisinformation, and open up the discussion on the desired evaluation\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09921v1",
            "date": "2019-08-23 11:04:48+00:00",
            "title": "Toward Dialogue Modeling: A Semantic Annotation Scheme for Questions and Answers",
            "authors": [
                "Maria-Andrea Cruz-Bland\u00f3n",
                "Gosse Minnema",
                "Aria Nourbakhsh",
                "Maria Boritchev",
                "Maxime Amblard"
            ],
            "abstract": "The present study proposes an annotation scheme for classifying the content\nand discourse contribution of question-answer pairs. We propose detailed\nguidelines for using the scheme and apply them to dialogues in English,\nSpanish, and Dutch. Finally, we report on initial machine learning experiments\nfor automatic annotation.",
            "journal ref": "LAW XIII 2019 - Linguistic Annotation Workshop - ACL Workshop, Jul\n  2019, Florence, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09940v2",
            "date": "2019-08-26 22:15:55+00:00",
            "title": "Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing",
            "authors": [
                "Jonathan Herzig",
                "Jonathan Berant"
            ],
            "abstract": "A major hurdle on the road to conversational interfaces is the difficulty in\ncollecting data that maps language utterances to logical forms. One prominent\napproach for data collection has been to automatically generate pseudo-language\npaired with logical forms, and paraphrase the pseudo-language to natural\nlanguage through crowdsourcing (Wang et al., 2015). However, this data\ncollection procedure often leads to low performance on real data, due to a\nmismatch between the true distribution of examples and the distribution induced\nby the data collection procedure. In this paper, we thoroughly analyze two\nsources of mismatch in this process: the mismatch in logical form distribution\nand the mismatch in language distribution between the true and induced\ndistributions. We quantify the effects of these mismatches, and propose a new\ndata collection approach that mitigates them. Assuming access to unlabeled\nutterances from the true distribution, we combine crowdsourcing with a\nparaphrase model to detect correct logical forms for the unlabeled utterances.\nOn two datasets, our method leads to 70.6 accuracy on average on the true\ndistribution, compared to 51.3 in paraphrasing-based data collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10063v1",
            "date": "2019-08-27 07:40:48+00:00",
            "title": "FinBERT: Financial Sentiment Analysis with Pre-trained Language Models",
            "authors": [
                "Dogu Araci"
            ],
            "abstract": "Financial sentiment analysis is a challenging task due to the specialized\nlanguage and lack of labeled data in that domain. General-purpose models are\nnot effective enough because of the specialized language used in a financial\ncontext. We hypothesize that pre-trained language models can help with this\nproblem because they require fewer labeled examples and they can be further\ntrained on domain-specific corpora. We introduce FinBERT, a language model\nbased on BERT, to tackle NLP tasks in the financial domain. Our results show\nimprovement in every measured metric on current state-of-the-art results for\ntwo financial sentiment analysis datasets. We find that even with a smaller\ntraining set and fine-tuning only a part of the model, FinBERT outperforms\nstate-of-the-art machine learning methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10118v2",
            "date": "2019-08-27 10:17:24+00:00",
            "title": "Multi-Layer Softmaxing during Training Neural Machine Translation for Flexible Decoding with Fewer Layers",
            "authors": [
                "Raj Dabre",
                "Atsushi Fujita"
            ],
            "abstract": "This paper proposes a novel procedure for training an encoder-decoder based\ndeep neural network which compresses NxM models into a single model enabling us\nto dynamically choose the number of encoder and decoder layers for decoding.\nUsually, the output of the last layer of the N-layer encoder is fed to the\nM-layer decoder, and the output of the last decoder layer is used to compute\nsoftmax loss. Instead, our method computes a single loss consisting of NxM\nlosses: the softmax loss for the output of each of the M decoder layers derived\nusing the output of each of the N encoder layers. A single model trained by our\nmethod can be used for decoding with an arbitrary fewer number of encoder and\ndecoder layers. In practical scenarios, this (a) enables faster decoding with\ninsignificant losses in translation quality and (b) alleviates the need to\ntrain NxM models, thereby saving space. We take a case study of neural machine\ntranslation and show the advantage and give a cost-benefit analysis of our\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10383v2",
            "date": "2019-08-27 18:03:12+00:00",
            "title": "Facet-Aware Evaluation for Extractive Summarization",
            "authors": [
                "Yuning Mao",
                "Liyuan Liu",
                "Qi Zhu",
                "Xiang Ren",
                "Jiawei Han"
            ],
            "abstract": "Commonly adopted metrics for extractive summarization focus on lexical\noverlap at the token level. In this paper, we present a facet-aware evaluation\nsetup for better assessment of the information coverage in extracted summaries.\nSpecifically, we treat each sentence in the reference summary as a\n\\textit{facet}, identify the sentences in the document that express the\nsemantics of each facet as \\textit{support sentences} of the facet, and\nautomatically evaluate extractive summarization methods by comparing the\nindices of extracted sentences and support sentences of all the facets in the\nreference summary. To facilitate this new evaluation setup, we construct an\nextractive version of the CNN/Daily Mail dataset and perform a thorough\nquantitative investigation, through which we demonstrate that facet-aware\nevaluation manifests better correlation with human judgment than ROUGE, enables\nfine-grained evaluation as well as comparative analysis, and reveals valuable\ninsights of state-of-the-art summarization methods. Data can be found at\nhttps://github.com/morningmoni/FAR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10546v1",
            "date": "2019-08-28 04:52:32+00:00",
            "title": "Fingerspelling recognition in the wild with iterative visual attention",
            "authors": [
                "Bowen Shi",
                "Aurora Martinez Del Rio",
                "Jonathan Keane",
                "Diane Brentari",
                "Greg Shakhnarovich",
                "Karen Livescu"
            ],
            "abstract": "Sign language recognition is a challenging gesture sequence recognition\nproblem, characterized by quick and highly coarticulated motion. In this paper\nwe focus on recognition of fingerspelling sequences in American Sign Language\n(ASL) videos collected in the wild, mainly from YouTube and Deaf social media.\nMost previous work on sign language recognition has focused on controlled\nsettings where the data is recorded in a studio environment and the number of\nsigners is limited. Our work aims to address the challenges of real-life data,\nreducing the need for detection or segmentation modules commonly used in this\ndomain. We propose an end-to-end model based on an iterative attention\nmechanism, without explicit hand detection or segmentation. Our approach\ndynamically focuses on increasingly high-resolution regions of interest. It\noutperforms prior work by a large margin. We also introduce a newly collected\ndata set of crowdsourced annotations of fingerspelling in the wild, and show\nthat performance can be further improved with this additional data set.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.10606v1",
            "date": "2019-08-28 09:33:37+00:00",
            "title": "Classical Chinese Sentence Segmentation for Tomb Biographies of Tang Dynasty",
            "authors": [
                "Chao-Lin Liu",
                "Yi Chang"
            ],
            "abstract": "Tomb biographies of the Tang dynasty provide invaluable information about\nChinese history. The original biographies are classical Chinese texts which\ncontain neither word boundaries nor sentence boundaries. Relying on three\npublished books of tomb biographies of the Tang dynasty, we investigated the\neffectiveness of employing machine-learning methods for algorithmically\nidentifying the pauses and terminals of sentences in the biographies.\n  We consider the segmentation task as a classification problem. Chinese\ncharacters that are and are not followed by a punctuation mark are classified\ninto two categories. We applied a machine-learning-based mechanism, the\nconditional random fields (CRF), to classify the characters (and words) in the\ntexts, and we studied the contributions of selected types of lexical\ninformation to the resulting quality of the segmentation recommendations.\n  This proposal presented at the DH 2018 conference discussed some of the basic\nexperiments and their evaluations. By considering the contextual information\nand employing the heuristics provided by experts of Chinese literature, we\nachieved F1 measures that were better than 80%. More complex experiments that\nemploy deep neural networks helped us further improve the results in recent\nwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10657v1",
            "date": "2019-08-28 11:47:39+00:00",
            "title": "Exploiting Multiple Embeddings for Chinese Named Entity Recognition",
            "authors": [
                "Canwen Xu",
                "Feiyang Wang",
                "Jialong Han",
                "Chenliang Li"
            ],
            "abstract": "Identifying the named entities mentioned in text would enrich many semantic\napplications at the downstream level. However, due to the predominant usage of\ncolloquial language in microblogs, the named entity recognition (NER) in\nChinese microblogs experience significant performance deterioration, compared\nwith performing NER in formal Chinese corpus. In this paper, we propose a\nsimple yet effective neural framework to derive the character-level embeddings\nfor NER in Chinese text, named ME-CNER. A character embedding is derived with\nrich semantic information harnessed at multiple granularities, ranging from\nradical, character to word levels. The experimental results demonstrate that\nthe proposed approach achieves a large performance improvement on Weibo dataset\nand comparable performance on MSRA news dataset with lower computational cost\nagainst the existing state-of-the-art alternatives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10678v2",
            "date": "2019-08-21 19:08:56+00:00",
            "title": "How Good is Artificial Intelligence at Automatically Answering Consumer Questions Related to Alzheimer's Disease?",
            "authors": [
                "Krishna B. Soundararajan",
                "Sunyang Fu",
                "Luke A. Carlson",
                "Rebecca A. Smith",
                "David S. Knopman",
                "Hongfang Liu",
                "Yanshan Wang"
            ],
            "abstract": "Alzheimer's Disease (AD) is the most common type of dementia, comprising\n60-80% of cases. There were an estimated 5.8 million Americans living with\nAlzheimer's dementia in 2019, and this number will almost double every 20\nyears. The total lifetime cost of care for someone with dementia is estimated\nto be $350,174 in 2018, 70% of which is associated with family-provided care.\nMost family caregivers face emotional, financial and physical difficulties. As\na medium to relieve this burden, online communities in social media websites\nsuch as Twitter, Reddit, and Yahoo! Answers provide potential venues for\ncaregivers to search relevant questions and answers, or post questions and seek\nanswers from other members. However, there are often a limited number of\nrelevant questions and responses to search from, and posted questions are\nrarely answered immediately. Due to recent advancement in Artificial\nIntelligence (AI), particularly Natural Language Processing (NLP), we propose\nto utilize AI to automatically generate answers to AD-related consumer\nquestions posted by caregivers and evaluate how good AI is at answering those\nquestions. To the best of our knowledge, this is the first study in the\nliterature applying and evaluating AI models designed to automatically answer\nconsumer questions related to AD.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.10719v1",
            "date": "2019-08-28 13:36:25+00:00",
            "title": "Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog",
            "authors": [
                "Ryuichi Takanobu",
                "Hanlin Zhu",
                "Minlie Huang"
            ],
            "abstract": "Dialog policy decides what and how a task-oriented dialog system will\nrespond, and plays a vital role in delivering effective conversations. Many\nstudies apply Reinforcement Learning to learn a dialog policy with the reward\nfunction which requires elaborate design and pre-specified user goals. With the\ngrowing needs to handle complex goals across multiple domains, such manually\ndesigned reward functions are not affordable to deal with the complexity of\nreal-world tasks. To this end, we propose Guided Dialog Policy Learning, a\nnovel algorithm based on Adversarial Inverse Reinforcement Learning for joint\nreward estimation and policy optimization in multi-domain task-oriented dialog.\nThe proposed approach estimates the reward signal and infers the user goal in\nthe dialog sessions. The reward estimator evaluates the state-action pairs so\nthat it can guide the dialog policy at each dialog turn. Extensive experiments\non a multi-domain dialog dataset show that the dialog policy guided by the\nlearned reward function achieves remarkably higher task success than\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10721v1",
            "date": "2019-08-28 13:40:43+00:00",
            "title": "Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension",
            "authors": [
                "Todor Mihaylov",
                "Anette Frank"
            ],
            "abstract": "In this work, we propose to use linguistic annotations as a basis for a\n\\textit{Discourse-Aware Semantic Self-Attention} encoder that we employ for\nreading comprehension on long narrative texts. We extract relations between\ndiscourse units, events and their arguments as well as coreferring mentions,\nusing available annotation tools. Our empirical evaluation shows that the\ninvestigated structures improve the overall performance, especially\nintra-sentential and cross-sentential discourse relations, sentence-internal\nsemantic role relations, and long-distance coreference relations. We show that\ndedicating self-attention heads to intra-sentential relations and relations\nconnecting neighboring sentences is beneficial for finding answers to questions\nin longer contexts. Our findings encourage the use of discourse-semantic\nannotations to enhance the generalization capacity of self-attention models for\nreading comprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10747v1",
            "date": "2019-08-28 14:29:13+00:00",
            "title": "Language Tasks and Language Games: On Methodology in Current Natural Language Processing Research",
            "authors": [
                "David Schlangen"
            ],
            "abstract": "\"This paper introduces a new task and a new dataset\", \"we improve the state\nof the art in X by Y\" -- it is rare to find a current natural language\nprocessing paper (or AI paper more generally) that does not contain such\nstatements. What is mostly left implicit, however, is the assumption that this\nnecessarily constitutes progress, and what it constitutes progress towards.\nHere, we make more precise the normally impressionistically used notions of\nlanguage task and language game and ask how a research programme built on these\nmight make progress towards the goal of modelling general language competence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10770v1",
            "date": "2019-08-28 15:17:33+00:00",
            "title": "Data Augmentation with Atomic Templates for Spoken Language Understanding",
            "authors": [
                "Zijian Zhao",
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "Spoken Language Understanding (SLU) converts user utterances into structured\nsemantic representations. Data sparsity is one of the main obstacles of SLU due\nto the high cost of human annotation, especially when domain changes or a new\ndomain comes. In this work, we propose a data augmentation method with atomic\ntemplates for SLU, which involves minimum human efforts. The atomic templates\nproduce exemplars for fine-grained constituents of semantic representations. We\npropose an encoder-decoder model to generate the whole utterance from atomic\nexemplars. Moreover, the generator could be transferred from source domains to\nhelp a new domain which has little data. Experimental results show that our\nmethod achieves significant improvements on DSTC 2\\&3 dataset which is a domain\nadaptation setting of SLU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10835v1",
            "date": "2019-08-28 17:10:06+00:00",
            "title": "An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation",
            "authors": [
                "Wanyu Du",
                "Yangfeng Ji"
            ],
            "abstract": "Generating paraphrases from given sentences involves decoding words step by\nstep from a large vocabulary. To learn a decoder, supervised learning which\nmaximizes the likelihood of tokens always suffers from the exposure bias.\nAlthough both reinforcement learning (RL) and imitation learning (IL) have been\nwidely used to alleviate the bias, the lack of direct comparison leads to only\na partial image on their benefits. In this work, we present an empirical study\non how RL and IL can help boost the performance of generating paraphrases, with\nthe pointer-generator as a base model. Experiments on the benchmark datasets\nshow that (1) imitation learning is constantly better than reinforcement\nlearning; and (2) the pointer-generator models with imitation learning\noutperform the state-of-the-art methods with a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10896v1",
            "date": "2019-08-28 18:22:26+00:00",
            "title": "Analyzing Customer Feedback for Product Fit Prediction",
            "authors": [
                "Stephan Baier"
            ],
            "abstract": "One of the biggest hurdles for customers when purchasing fashion online, is\nthe difficulty of finding products with the right fit. In order to provide a\nbetter online shopping experience, platforms need to find ways to recommend the\nright product sizes and the best fitting products to their customers. These\nrecommendation systems, however, require customer feedback in order to estimate\nthe most suitable sizing options. Such feedback is rare and often only\navailable as natural text. In this paper, we examine the extraction of product\nfit feedback from customer reviews using natural language processing\ntechniques. In particular, we compare traditional methods with more recent\ntransfer learning techniques for text classification, and analyze their\nresults. Our evaluation shows, that the transfer learning approach ULMFit is\nnot only comparatively fast to train, but also achieves highest accuracy on\nthis task. The integration of the extracted information with actual size\nrecommendation systems is left for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10909v1",
            "date": "2019-08-28 19:10:08+00:00",
            "title": "Interactive Language Learning by Question Answering",
            "authors": [
                "Xingdi Yuan",
                "Marc-Alexandre Cote",
                "Jie Fu",
                "Zhouhan Lin",
                "Christopher Pal",
                "Yoshua Bengio",
                "Adam Trischler"
            ],
            "abstract": "Humans observe and interact with the world to acquire knowledge. However,\nmost existing machine reading comprehension (MRC) tasks miss the interactive,\ninformation-seeking component of comprehension. Such tasks present models with\nstatic documents that contain all necessary information, usually concentrated\nin a single short substring. Thus, models can achieve strong performance\nthrough simple word- and phrase-based pattern matching. We address this problem\nby formulating a novel text-based question answering task: Question Answering\nwith Interactive Text (QAit). In QAit, an agent must interact with a partially\nobservable text-based environment to gather information required to answer\nquestions. QAit poses questions about the existence, location, and attributes\nof objects found in the environment. The data is built using a text-based game\ngenerator that defines the underlying dynamics of interaction with the\nenvironment. We propose and evaluate a set of baseline models for the QAit task\nthat includes deep reinforcement learning agents. Experiments show that the\ntask presents a major challenge for machine reading systems, while humans solve\nit with relative ease.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10924v1",
            "date": "2019-08-28 19:42:37+00:00",
            "title": "Solving Math Word Problems with Double-Decoder Transformer",
            "authors": [
                "Yuanliang Meng",
                "Anna Rumshisky"
            ],
            "abstract": "This paper proposes a Transformer-based model to generate equations for math\nword problems. It achieves much better results than RNN models when copy and\nalign mechanisms are not used, and can outperform complex copy and align RNN\nmodels. We also show that training a Transformer jointly in a generation task\nwith two decoders, left-to-right and right-to-left, is beneficial. Such a\nTransformer performs better than the one with just one decoder not only because\nof the ensemble effect, but also because it improves the encoder training\nprocedure. We also experiment with adding reinforcement learning to our model,\nshowing improved performance compared to MLE training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.10940v2",
            "date": "2019-08-28 20:48:05+00:00",
            "title": "Learning a Multi-Domain Curriculum for Neural Machine Translation",
            "authors": [
                "Wei Wang",
                "Ye Tian",
                "Jiquan Ngiam",
                "Yinfei Yang",
                "Isaac Caswell",
                "Zarana Parekh"
            ],
            "abstract": "Most data selection research in machine translation focuses on improving a\nsingle domain. We perform data selection for multiple domains at once. This is\nachieved by carefully introducing instance-level domain-relevance features and\nautomatically constructing a training curriculum to gradually concentrate on\nmulti-domain relevant and noise-reduced data batches. Both the choice of\nfeatures and the use of curriculum are crucial for balancing and improving all\ndomains, including out-of-domain. In large-scale experiments, the multi-domain\ncurriculum simultaneously reaches or outperforms the individual performance and\nbrings solid gains over no-curriculum training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10970v1",
            "date": "2019-08-28 22:18:03+00:00",
            "title": "Leveraging Structural and Semantic Correspondence for Attribute-Oriented Aspect Sentiment Discovery",
            "authors": [
                "Zhe Zhang",
                "Munindar P. Singh"
            ],
            "abstract": "Opinionated text often involves attributes such as authorship and location\nthat influence the sentiments expressed for different aspects. We posit that\nstructural and semantic correspondence is both prevalent in opinionated text,\nespecially when associated with attributes, and crucial in accurately revealing\nits latent aspect and sentiment structure. However, it is not recognized by\nexisting approaches.\n  We propose Trait, an unsupervised probabilistic model that discovers aspects\nand sentiments from text and associates them with different attributes. To this\nend, Trait infers and leverages structural and semantic correspondence using a\nMarkov Random Field. We show empirically that by incorporating attributes\nexplicitly Trait significantly outperforms state-of-the-art baselines both by\ngenerating attribute profiles that accord with our intuitions, as shown via\nvisualization, and yielding topics of greater semantic cohesion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11007v2",
            "date": "2019-08-29 01:25:52+00:00",
            "title": "Neural Snowball for Few-Shot Relation Learning",
            "authors": [
                "Tianyu Gao",
                "Xu Han",
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Fen Lin",
                "Leyu Lin",
                "Maosong Sun"
            ],
            "abstract": "Knowledge graphs typically undergo open-ended growth of new relations. This\ncannot be well handled by relation extraction that focuses on pre-defined\nrelations with sufficient training data. To address new relations with few-shot\ninstances, we propose a novel bootstrapping approach, Neural Snowball, to learn\nnew relations by transferring semantic knowledge about existing relations. More\nspecifically, we use Relational Siamese Networks (RSN) to learn the metric of\nrelational similarities between instances based on existing relations and their\nlabeled data. Afterwards, given a new relation and its few-shot instances, we\nuse RSN to accumulate reliable instances from unlabeled corpora; these\ninstances are used to train a relation classifier, which can further identify\nnew facts of the new relation. The process is conducted iteratively like a\nsnowball. Experiments show that our model can gather high-quality instances for\nbetter few-shot relation learning and achieves significant improvement compared\nto baselines. Codes and datasets are released on\nhttps://github.com/thunlp/Neural-Snowball.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11030v1",
            "date": "2019-08-29 03:18:21+00:00",
            "title": "Towards Ethical Content-Based Detection of Online Influence Campaigns",
            "authors": [
                "Evan Crothers",
                "Nathalie Japkowicz",
                "Herna Viktor"
            ],
            "abstract": "The detection of clandestine efforts to influence users in online communities\nis a challenging problem with significant active development. We demonstrate\nthat features derived from the text of user comments are useful for identifying\nsuspect activity, but lead to increased erroneous identifications when keywords\nover-represented in past influence campaigns are present. Drawing on research\nin native language identification (NLI), we use \"named entity masking\" (NEM) to\ncreate sentence features robust to this shortcoming, while maintaining\ncomparable classification accuracy. We demonstrate that while NEM consistently\nreduces false positives when key named entities are mentioned, both masked and\nunmasked models exhibit increased false positive rates on English sentences by\nRussian native speakers, raising ethical considerations that should be\naddressed in future research.",
            "journal ref": "2019 IEEE 29th International Workshop on Machine Learning for\n  Signal Processing (MLSP), Pittsburgh, PA, USA, 2019, pp. 1-6",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1908.11052v1",
            "date": "2019-08-29 05:01:39+00:00",
            "title": "Zero-shot Text-to-SQL Learning with Auxiliary Task",
            "authors": [
                "Shuaichen Chang",
                "Pengfei Liu",
                "Yun Tang",
                "Jing Huang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Recent years have seen great success in the use of neural seq2seq models on\nthe text-to-SQL task. However, little work has paid attention to how these\nmodels generalize to realistic unseen data, which naturally raises a question:\ndoes this impressive performance signify a perfect generalization model, or are\nthere still some limitations?\n  In this paper, we first diagnose the bottleneck of text-to-SQL task by\nproviding a new testbed, in which we observe that existing models present poor\ngeneralization ability on rarely-seen data. The above analysis encourages us to\ndesign a simple but effective auxiliary task, which serves as a supportive\nmodel as well as a regularization term to the generation task to increase the\nmodels generalization. Experimentally, We evaluate our models on a large\ntext-to-SQL dataset WikiSQL. Compared to a strong baseline coarse-to-fine\nmodel, our models improve over the baseline by more than 3% absolute in\naccuracy on the whole dataset. More interestingly, on a zero-shot subset test\nof WikiSQL, our models achieve 5% absolute accuracy gain over the baseline,\nclearly demonstrating its superior generalizability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11053v1",
            "date": "2019-08-29 05:03:57+00:00",
            "title": "Leveraging Frequent Query Substructures to Generate Formal Queries for Complex Question Answering",
            "authors": [
                "Jiwei Ding",
                "Wei Hu",
                "Qixin Xu",
                "Yuzhong Qu"
            ],
            "abstract": "Formal query generation aims to generate correct executable queries for\nquestion answering over knowledge bases (KBs), given entity and relation\nlinking results. Current approaches build universal paraphrasing or ranking\nmodels for the whole questions, which are likely to fail in generating queries\nfor complex, long-tail questions. In this paper, we propose SubQG, a new query\ngeneration approach based on frequent query substructures, which helps rank the\nexisting (but nonsignificant) query structures or build new query structures.\nOur experiments on two benchmark datasets show that our approach significantly\noutperforms the existing ones, especially for complex questions. Also, it\nachieves promising performance with limited training data and noisy\nentity/relation linking results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11078v1",
            "date": "2019-08-29 07:29:28+00:00",
            "title": "Document Hashing with Mixture-Prior Generative Models",
            "authors": [
                "Wei Dong",
                "Qinliang Su",
                "Dinghan Shen",
                "Changyou Chen"
            ],
            "abstract": "Hashing is promising for large-scale information retrieval tasks thanks to\nthe efficiency of distance evaluation between binary codes. Generative hashing\nis often used to generate hashing codes in an unsupervised way. However,\nexisting generative hashing methods only considered the use of simple priors,\nlike Gaussian and Bernoulli priors, which limits these methods to further\nimprove their performance. In this paper, two mixture-prior generative models\nare proposed, under the objective to produce high-quality hashing codes for\ndocuments. Specifically, a Gaussian mixture prior is first imposed onto the\nvariational auto-encoder (VAE), followed by a separate step to cast the\ncontinuous latent representation of VAE into binary code. To avoid the\nperformance loss caused by the separate casting, a model using a Bernoulli\nmixture prior is further developed, in which an end-to-end training is admitted\nby resorting to the straight-through (ST) discrete gradient estimator.\nExperimental results on several benchmark datasets demonstrate that the\nproposed methods, especially the one using Bernoulli mixture priors,\nconsistently outperform existing ones by a substantial margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11135v1",
            "date": "2019-08-29 10:14:40+00:00",
            "title": "KBSET -- Knowledge-Based Support for Scholarly Editing and Text Processing",
            "authors": [
                "Jana Kittelmann",
                "Christoph Wernhard"
            ],
            "abstract": "KBSET supports a practical workflow for scholarly editing, based on using\nLaTeX with dedicated commands for semantics-oriented markup and a\nProlog-implemented core system. Prolog plays there various roles: as query\nlanguage and access mechanism for large Semantic Web fact bases, as data\nrepresentation of structured documents and as a workflow model for advanced\napplication tasks. The core system includes a LaTeX parser and a facility for\nthe identification of named entities. We also sketch future perspectives of\nthis approach to scholarly editing based on techniques of computational logic.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1908.11310v1",
            "date": "2019-08-29 15:50:28+00:00",
            "title": "Aesthetic Image Captioning From Weakly-Labelled Photographs",
            "authors": [
                "Koustav Ghosal",
                "Aakanksha Rana",
                "Aljosa Smolic"
            ],
            "abstract": "Aesthetic image captioning (AIC) refers to the multi-modal task of generating\ncritical textual feedbacks for photographs. While in natural image captioning\n(NIC), deep models are trained in an end-to-end manner using large curated\ndatasets such as MS-COCO, no such large-scale, clean dataset exists for AIC.\nTowards this goal, we propose an automatic cleaning strategy to create a\nbenchmarking AIC dataset, by exploiting the images and noisy comments easily\navailable from photography websites. We propose a probabilistic\ncaption-filtering method for cleaning the noisy web-data, and compile a\nlarge-scale, clean dataset \"AVA-Captions\", (230, 000 images with 5 captions per\nimage). Additionally, by exploiting the latent associations between aesthetic\nattributes, we propose a strategy for training the convolutional neural network\n(CNN) based visual feature extractor, the first component of the AIC framework.\nThe strategy is weakly supervised and can be effectively used to learn rich\naesthetic representations, without requiring expensive ground-truth\nannotations. We finally show-case a thorough analysis of the proposed\ncontributions using automatic metrics and subjective evaluations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.11326v1",
            "date": "2019-08-29 16:17:53+00:00",
            "title": "Translate and Label! An Encoder-Decoder Approach for Cross-lingual Semantic Role Labeling",
            "authors": [
                "Angel Daza",
                "Anette Frank"
            ],
            "abstract": "We propose a Cross-lingual Encoder-Decoder model that simultaneously\ntranslates and generates sentences with Semantic Role Labeling annotations in a\nresource-poor target language. Unlike annotation projection techniques, our\nmodel does not need parallel data during inference time. Our approach can be\napplied in monolingual, multilingual and cross-lingual settings and is able to\nproduce dependency-based and span-based SRL annotations. We benchmark the\nlabeling performance of our model in different monolingual and multilingual\nsettings using well-known SRL datasets. We then train our model in a\ncross-lingual setting to generate new SRL labeled data. Finally, we measure the\neffectiveness of our method by using the generated data to augment the training\nbasis for resource-poor languages and perform manual evaluation to show that it\nproduces high-quality sentences and assigns accurate semantic role annotations.\nOur proposed architecture offers a flexible method for leveraging SRL data in\nmultiple languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11522v3",
            "date": "2019-08-30 03:49:19+00:00",
            "title": "Parsing All: Syntax and Semantics, Dependencies and Spans",
            "authors": [
                "Junru Zhou",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "Both syntactic and semantic structures are key linguistic contextual clues,\nin which parsing the latter has been well shown beneficial from parsing the\nformer. However, few works ever made an attempt to let semantic parsing help\nsyntactic parsing. As linguistic representation formalisms, both syntax and\nsemantics may be represented in either span (constituent/phrase) or dependency,\non both of which joint learning was also seldom explored. In this paper, we\npropose a novel joint model of syntactic and semantic parsing on both span and\ndependency representations, which incorporates syntactic information\neffectively in the encoder of neural network and benefits from two\nrepresentation formalisms in a uniform way. The experiments show that semantics\nand syntax can benefit each other by optimizing joint objectives. Our single\nmodel achieves new state-of-the-art or competitive results on both span and\ndependency semantic parsing on Propbank benchmarks and both dependency and\nconstituent syntactic parsing on Penn Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11540v1",
            "date": "2019-08-30 05:44:24+00:00",
            "title": "DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation",
            "authors": [
                "Deepanway Ghosal",
                "Navonil Majumder",
                "Soujanya Poria",
                "Niyati Chhaya",
                "Alexander Gelbukh"
            ],
            "abstract": "Emotion recognition in conversation (ERC) has received much attention,\nlately, from researchers due to its potential widespread applications in\ndiverse areas, such as health-care, education, and human resources. In this\npaper, we present Dialogue Graph Convolutional Network (DialogueGCN), a graph\nneural network based approach to ERC. We leverage self and inter-speaker\ndependency of the interlocutors to model conversational context for emotion\nrecognition. Through the graph network, DialogueGCN addresses context\npropagation issues present in the current RNN-based methods. We empirically\nshow that this method alleviates such issues, while outperforming the current\nstate of the art on a number of benchmark emotion classification datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11571v1",
            "date": "2019-08-30 07:22:43+00:00",
            "title": "Hierarchical Pointer Net Parsing",
            "authors": [
                "Linlin Liu",
                "Xiang Lin",
                "Shafiq Joty",
                "Simeng Han",
                "Lidong Bing"
            ],
            "abstract": "Transition-based top-down parsing with pointer networks has achieved\nstate-of-the-art results in multiple parsing tasks, while having a linear time\ncomplexity. However, the decoder of these parsers has a sequential structure,\nwhich does not yield the most appropriate inductive bias for deriving tree\nstructures. In this paper, we propose hierarchical pointer network parsers, and\napply them to dependency and sentence-level discourse parsing tasks. Our\nresults on standard benchmark datasets demonstrate the effectiveness of our\napproach, outperforming existing methods and setting a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11782v1",
            "date": "2019-08-30 15:21:28+00:00",
            "title": "Latent Part-of-Speech Sequences for Neural Machine Translation",
            "authors": [
                "Xuewen Yang",
                "Yingru Liu",
                "Dongliang Xie",
                "Xin Wang",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Learning target side syntactic structure has been shown to improve Neural\nMachine Translation (NMT). However, incorporating syntax through latent\nvariables introduces additional complexity in inference, as the models need to\nmarginalize over the latent syntactic structures. To avoid this, models often\nresort to greedy search which only allows them to explore a limited portion of\nthe latent space. In this work, we introduce a new latent variable model,\nLaSyn, that captures the co-dependence between syntax and semantics, while\nallowing for effective and efficient inference over the latent space. LaSyn\ndecouples direct dependence between successive latent variables, which allows\nits decoder to exhaustively search through the latent syntactic choices, while\nkeeping decoding speed proportional to the size of the latent variable\nvocabulary. We implement LaSyn by modifying a transformer-based NMT system and\ndesign a neural expectation maximization algorithm that we regularize with\npart-of-speech information as the latent sequences. Evaluations on four\ndifferent MT tasks show that incorporating target side syntax with LaSyn\nimproves both translation quality, and also provides an opportunity to improve\ndiversity.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1908.11790v1",
            "date": "2019-08-30 15:30:11+00:00",
            "title": "Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs",
            "authors": [
                "Dongyeop Kang",
                "Hiroaki Hayashi",
                "Alan W Black",
                "Eduard Hovy"
            ],
            "abstract": "Generating a long, coherent text such as a paragraph requires a high-level\ncontrol of different levels of relations between sentences (e.g., tense,\ncoreference). We call such a logical connection between sentences as a\n(paragraph) flow. In order to produce a coherent flow of text, we explore two\nforms of intersentential relations in a paragraph: one is a human-created\nlinguistical relation that forms a structure (e.g., discourse tree) and the\nother is a relation from latent representation learned from the sentences\nthemselves. Our two proposed models incorporate each form of relations into\ndocument-level language models: the former is a supervised model that jointly\nlearns a language model as well as discourse relation prediction, and the\nlatter is an unsupervised model that is hierarchically conditioned by a\nrecurrent neural network (RNN) over the latent information. Our proposed models\nwith both forms of relations outperform the baselines in partially conditioned\nparagraph generation task. Our codes and data are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00015v2",
            "date": "2019-08-30 18:06:14+00:00",
            "title": "Adaptively Sparse Transformers",
            "authors": [
                "Gon\u00e7alo M. Correia",
                "Vlad Niculae",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Attention mechanisms have become ubiquitous in NLP. Recent architectures,\nnotably the Transformer, learn powerful context-aware word representations\nthrough layered, multi-headed attention. The multiple heads learn diverse types\nof word relationships. However, with standard softmax attention, all attention\nheads are dense, assigning a non-zero weight to all context words. In this\nwork, we introduce the adaptively sparse Transformer, wherein attention heads\nhave flexible, context-dependent sparsity patterns. This sparsity is\naccomplished by replacing softmax with $\\alpha$-entmax: a differentiable\ngeneralization of softmax that allows low-scoring words to receive precisely\nzero weight. Moreover, we derive a method to automatically learn the $\\alpha$\nparameter -- which controls the shape and sparsity of $\\alpha$-entmax --\nallowing attention heads to choose between focused or spread-out behavior. Our\nadaptively sparse Transformer improves interpretability and head diversity when\ncompared to softmax Transformers on machine translation datasets. Findings of\nthe quantitative and qualitative analysis of our approach include that heads in\ndifferent layers learn different sparsity preferences and tend to be more\ndiverse in their attention distributions than softmax Transformers.\nFurthermore, at no cost in accuracy, sparsity in attention heads helps to\nuncover different head specializations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00109v2",
            "date": "2019-08-31 02:30:24+00:00",
            "title": "Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension",
            "authors": [
                "Daniel Andor",
                "Luheng He",
                "Kenton Lee",
                "Emily Pitler"
            ],
            "abstract": "Reading comprehension models have been successfully applied to extractive\ntext answers, but it is unclear how best to generalize these models to\nabstractive numerical answers. We enable a BERT-based reading comprehension\nmodel to perform lightweight numerical reasoning. We augment the model with a\npredefined set of executable 'programs' which encompass simple arithmetic as\nwell as extraction. Rather than having to learn to manipulate numbers directly,\nthe model can pick a program and execute it. On the recent Discrete Reasoning\nOver Passages (DROP) dataset, designed to challenge reading comprehension\nmodels, we show a 33% absolute improvement by adding shallow programs. The\nmodel can learn to predict new operations when appropriate in a math word\nproblem setting (Roy and Roth, 2015) with very few training examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00121v3",
            "date": "2019-08-31 04:01:38+00:00",
            "title": "A Semantics-Assisted Video Captioning Model Trained with Scheduled Sampling",
            "authors": [
                "Haoran Chen",
                "Ke Lin",
                "Alexander Maye",
                "Jianming Li",
                "Xiaolin Hu"
            ],
            "abstract": "Given the features of a video, recurrent neural networks can be used to\nautomatically generate a caption for the video. Existing methods for video\ncaptioning have at least three limitations. First, semantic information has\nbeen widely applied to boost the performance of video captioning models, but\nexisting networks often fail to provide meaningful semantic features. Second,\nthe Teacher Forcing algorithm is often utilized to optimize video captioning\nmodels, but during training and inference, different strategies are applied to\nguide word generation, leading to poor performance. Third, current video\ncaptioning models are prone to generate relatively short captions that express\nvideo contents inappropriately. Toward resolving these three problems, we\nsuggest three corresponding improvements. First of all, we propose a metric to\ncompare the quality of semantic features, and utilize appropriate features as\ninput for a semantic detection network (SDN) with adequate complexity in order\nto generate meaningful semantic features for videos. Then, we apply a scheduled\nsampling strategy that gradually transfers the training phase from a\nteacher-guided manner toward a more self-teaching manner. Finally, the ordinary\nlogarithm probability loss function is leveraged by sentence length so that the\ninclination of generating short sentences is alleviated. Our model achieves\nbetter results than previous models on the YouTube2Text dataset and is\ncompetitive with the previous best model on the MSR-VTT dataset.",
            "journal ref": "Front. Robot. AI 7:475767 (2020)",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.00124v1",
            "date": "2019-08-31 04:18:50+00:00",
            "title": "Learning with Noisy Labels for Sentence-level Sentiment Classification",
            "authors": [
                "Hao Wang",
                "Bing Liu",
                "Chaozhuo Li",
                "Yan Yang",
                "Tianrui Li"
            ],
            "abstract": "Deep neural networks (DNNs) can fit (or even over-fit) the training data very\nwell. If a DNN model is trained using data with noisy labels and tested on data\nwith clean labels, the model may perform poorly. This paper studies the problem\nof learning with noisy labels for sentence-level sentiment classification. We\npropose a novel DNN model called NetAb (as shorthand for convolutional neural\nNetworks with Ab-networks) to handle noisy labels during training. NetAb\nconsists of two convolutional neural networks, one with a noise transition\nlayer for dealing with the input noisy labels and the other for predicting\n'clean' labels. We train the two networks using their respective loss functions\nin a mutual reinforcement manner. Experimental results demonstrate the\neffectiveness of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00153v3",
            "date": "2019-08-31 06:59:46+00:00",
            "title": "Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER",
            "authors": [
                "Phillip Keung",
                "Yichao Lu",
                "Vikas Bhardwaj"
            ],
            "abstract": "Contextual word embeddings (e.g. GPT, BERT, ELMo, etc.) have demonstrated\nstate-of-the-art performance on various NLP tasks. Recent work with the\nmultilingual version of BERT has shown that the model performs very well in\nzero-shot and zero-resource cross-lingual settings, where only labeled English\ndata is used to finetune the model. We improve upon multilingual BERT's\nzero-resource cross-lingual performance via adversarial learning. We report the\nmagnitude of the improvement on the multilingual MLDoc text classification and\nCoNLL 2002/2003 named entity recognition tasks. Furthermore, we show that\nlanguage-adversarial training encourages BERT to align the embeddings of\nEnglish documents and their translations, which may be the cause of the\nobserved performance gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00164v2",
            "date": "2019-08-31 08:22:13+00:00",
            "title": "Named Entity Recognition Only from Word Embeddings",
            "authors": [
                "Ying Luo",
                "Hai Zhao",
                "Junlang Zhan"
            ],
            "abstract": "Deep neural network models have helped named entity (NE) recognition achieve\namazing performance without handcrafting features. However, existing systems\nrequire large amounts of human annotated training data. Efforts have been made\nto replace human annotations with external knowledge (e.g., NE dictionary,\npart-of-speech tags), while it is another challenge to obtain such effective\nresources. In this work, we propose a fully unsupervised NE recognition model\nwhich only needs to take informative clues from pre-trained word embeddings. We\nfirst apply Gaussian Hidden Markov Model and Deep Autoencoding Gaussian Mixture\nModel on word embeddings for entity span detection and type prediction, and\nthen further design an instance selector based on reinforcement learning to\ndistinguish positive sentences from noisy sentences and refine these\ncoarse-grained annotations through neural networks. Extensive experiments on\nCoNLL benchmark datasets demonstrate that our proposed light NE recognition\nmodel achieves remarkable performance without using any annotated lexicon or\ncorpus.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.00170v2",
            "date": "2019-08-31 08:56:46+00:00",
            "title": "Open Named Entity Modeling from Embedding Distribution",
            "authors": [
                "Ying Luo",
                "Hai Zhao",
                "Zhuosheng Zhang",
                "Bingjie Tang"
            ],
            "abstract": "In this paper, we report our discovery on named entity distribution in a\ngeneral word embedding space, which helps an open definition on multilingual\nnamed entity definition rather than previous closed and constraint definition\non named entities through a named entity dictionary, which is usually derived\nfrom human labor and replies on schedule update. Our initial visualization of\nmonolingual word embeddings indicates named entities tend to gather together\ndespite of named entity types and language difference, which enable us to model\nall named entities using a specific geometric structure inside embedding space,\nnamely, the named entity hypersphere. For monolingual cases, the proposed named\nentity model gives an open description of diverse named entity types and\ndifferent languages. For cross-lingual cases, mapping the proposed named entity\nmodel provides a novel way to build a named entity dataset for resource-poor\nlanguages. At last, the proposed named entity model may be shown as a handy\nclue to enhance state-of-the-art named entity recognition systems generally.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00252v1",
            "date": "2019-08-31 18:01:29+00:00",
            "title": "Humor Detection: A Transformer Gets the Last Laugh",
            "authors": [
                "Orion Weller",
                "Kevin Seppi"
            ],
            "abstract": "Much previous work has been done in attempting to identify humor in text. In\nthis paper we extend that capability by proposing a new task: assessing whether\nor not a joke is humorous. We present a novel way of approaching this problem\nby building a model that learns to identify humorous jokes based on ratings\ngleaned from Reddit pages, consisting of almost 16,000 labeled instances. Using\nthese ratings to determine the level of humor, we then employ a Transformer\narchitecture for its advantages in learning from sentence context. We\ndemonstrate the effectiveness of this approach and show results that are\ncomparable to human performance. We further demonstrate our model's increased\ncapabilities on humor identification problems, such as the previously created\ndatasets for short jokes and puns. These experiments show that this method\noutperforms all previous work done on these tasks, with an F-measure of 93.1%\nfor the Puns dataset and 98.6% on the Short Jokes dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00277v2",
            "date": "2019-08-31 19:55:44+00:00",
            "title": "Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning",
            "authors": [
                "Lifu Huang",
                "Ronan Le Bras",
                "Chandra Bhagavatula",
                "Yejin Choi"
            ],
            "abstract": "Understanding narratives requires reading between the lines, which in turn,\nrequires interpreting the likely causes and effects of events, even when they\nare not mentioned explicitly. In this paper, we introduce Cosmos QA, a\nlarge-scale dataset of 35,600 problems that require commonsense-based reading\ncomprehension, formulated as multiple-choice questions. In stark contrast to\nmost existing reading comprehension datasets where the questions focus on\nfactual and literal understanding of the context paragraph, our dataset focuses\non reading between the lines over a diverse collection of people's everyday\nnarratives, asking such questions as \"what might be the possible reason of\n...?\", or \"what would have happened if ...\" that require reasoning beyond the\nexact text spans in the context. To establish baseline performances on Cosmos\nQA, we experiment with several state-of-the-art neural architectures for\nreading comprehension, and also propose a new architecture that improves over\nthe competitive baselines. Experimental results demonstrate a significant gap\nbetween machine (68.4%) and human performance (94%), pointing to avenues for\nfuture research on commonsense machine comprehension. Dataset, code and\nleaderboard is publicly available at https://wilburone.github.io/cosmos.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00325v1",
            "date": "2019-09-01 05:26:30+00:00",
            "title": "Repurposing Decoder-Transformer Language Models for Abstractive Summarization",
            "authors": [
                "Luke de Oliveira",
                "Alfredo L\u00e1inez Rodrigo"
            ],
            "abstract": "Neural network models have shown excellent fluency and performance when\napplied to abstractive summarization. Many approaches to neural abstractive\nsummarization involve the introduction of significant inductive bias,\nexemplified through the use of components such as pointer-generator\narchitectures, coverage, and partially extractive procedures, designed to mimic\nthe process by which humans summarize documents. We show that it is possible to\nattain competitive performance by instead directly viewing summarization as a\nlanguage modeling problem and effectively leveraging transfer learning. We\nintroduce a simple procedure built upon decoder-transformers to obtain highly\ncompetitive ROUGE scores for summarization performance using a language\nmodeling loss alone, with no beam-search or other decoding-time optimization,\nand instead relying on efficient nucleus sampling and greedy decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00326v2",
            "date": "2019-09-01 06:04:48+00:00",
            "title": "Towards Understanding Neural Machine Translation with Word Importance",
            "authors": [
                "Shilin He",
                "Zhaopeng Tu",
                "Xing Wang",
                "Longyue Wang",
                "Michael R. Lyu",
                "Shuming Shi"
            ],
            "abstract": "Although neural machine translation (NMT) has advanced the state-of-the-art\non various language pairs, the interpretability of NMT remains unsatisfactory.\nIn this work, we propose to address this gap by focusing on understanding the\ninput-output behavior of NMT models. Specifically, we measure the word\nimportance by attributing the NMT output to every input word through a\ngradient-based method. We validate the approach on a couple of perturbation\noperations, language pairs, and model architectures, demonstrating its\nsuperiority on identifying input words with higher influence on translation\nperformance. Encouragingly, the calculated importance can serve as indicators\nof input words that are under-translated by NMT models. Furthermore, our\nanalysis reveals that words of certain syntactic categories have higher\nimportance while the categories vary across language pairs, which can inspire\nbetter design principles of NMT architectures for multi-lingual translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00338v1",
            "date": "2019-09-01 07:00:26+00:00",
            "title": "Monitoring stance towards vaccination in Twitter messages",
            "authors": [
                "Florian Kunneman",
                "Mattijs Lambooij",
                "Albert Wong",
                "Antal van den Bosch",
                "Liesbeth Mollema"
            ],
            "abstract": "We developed a system to automatically classify stance towards vaccination in\nTwitter messages, with a focus on messages with a negative stance. Such a\nsystem makes it possible to monitor the ongoing stream of messages on social\nmedia, offering actionable insights into public hesitance with respect to\nvaccination. For Dutch Twitter messages that mention vaccination-related key\nterms, we annotated their stance and feeling in relation to vaccination\n(provided that they referred to this topic). Subsequently, we used these coded\ndata to train and test different machine learning set-ups. With the aim to best\nidentify messages with a negative stance towards vaccination, we compared\nset-ups at an increasing dataset size and decreasing reliability, at an\nincreasing number of categories to distinguish, and with different\nclassification algorithms. We found that Support Vector Machines trained on a\ncombination of strictly and laxly labeled data with a more fine-grained\nlabeling yielded the best result, at an F1-score of 0.36 and an Area under the\nROC curve of 0.66, outperforming a rule-based sentiment analysis baseline that\nyielded an F1-score of 0.25 and an Area under the ROC curve of 0.57. The\noutcomes of our study indicate that stance prediction by a computerized system\nonly is a challenging task. Our analysis of the data and behavior of our system\nsuggests that an approach is needed in which the use of a larger training\ndataset is combined with a setting in which a human-in-the-loop provides the\nsystem with feedback on its predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00426v2",
            "date": "2019-09-01 16:29:53+00:00",
            "title": "Global Entity Disambiguation with Pretrained Contextualized Embeddings of Words and Entities",
            "authors": [
                "Ikuya Yamada",
                "Koki Washio",
                "Hiroyuki Shindo",
                "Yuji Matsumoto"
            ],
            "abstract": "We propose a new global entity disambiguation (ED) model based on\ncontextualized embeddings of words and entities. Our model is based on a\nbidirectional transformer encoder (i.e., BERT) and produces contextualized\nembeddings for words and entities in the input text. The model is trained using\na new masked entity prediction task that aims to train the model by predicting\nrandomly masked entities in entity-annotated texts obtained from Wikipedia. We\nfurther extend the model by solving ED as a sequential decision task to capture\nglobal contextual information. We evaluate our model using six standard ED\ndatasets and achieve new state-of-the-art results on all but one dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00519v2",
            "date": "2019-09-02 03:10:14+00:00",
            "title": "Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding",
            "authors": [
                "Mojtaba Nayyeri",
                "Chengjin Xu",
                "Yadollah Yaghoobzadeh",
                "Hamed Shariat Yazdi",
                "Jens Lehmann"
            ],
            "abstract": "Knowledge graphs (KGs) represent world's facts in structured forms. KG\ncompletion exploits the existing facts in a KG to discover new ones.\nTranslation-based embedding model (TransE) is a prominent formulation to do KG\ncompletion. Despite the efficiency of TransE in memory and time, it suffers\nfrom several limitations in encoding relation patterns such as symmetric,\nreflexive etc. To resolve this problem, most of the attempts have circled\naround the revision of the score function of TransE i.e., proposing a more\ncomplicated score function such as Trans(A, D, G, H, R, etc) to mitigate the\nlimitations. In this paper, we tackle this problem from a different\nperspective. We show that existing theories corresponding to the limitations of\nTransE are inaccurate because they ignore the effect of loss function.\nAccordingly, we pose theoretical investigations of the main limitations of\nTransE in the light of loss function. To the best of our knowledge, this has\nnot been investigated so far comprehensively. We show that by a proper\nselection of the loss function for training the TransE model, the main\nlimitations of the model are mitigated. This is explained by setting\nupper-bound for the scores of positive samples, showing the region of truth\n(i.e., the region that a triple is considered positive by the model). Our\ntheoretical proofs with experimental results fill the gap between the\ncapability of translation-based class of embedding models and the loss\nfunction. The theories emphasise the importance of the selection of the loss\nfunctions for training the models. Our experimental evaluations on different\nloss functions used for training the models justify our theoretical proofs and\nconfirm the importance of the loss functions on the performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.00596v2",
            "date": "2019-09-02 08:36:32+00:00",
            "title": "Answering questions by learning to rank -- Learning to rank by answering questions",
            "authors": [
                "George-Sebastian P\u00eertoac\u0103",
                "Traian Rebedea",
                "Stefan Ruseti"
            ],
            "abstract": "Answering multiple-choice questions in a setting in which no supporting\ndocuments are explicitly provided continues to stand as a core problem in\nnatural language processing. The contribution of this article is two-fold.\nFirst, it describes a method which can be used to semantically rank documents\nextracted from Wikipedia or similar natural language corpora. Second, we\npropose a model employing the semantic ranking that holds the first place in\ntwo of the most popular leaderboards for answering multiple-choice questions:\nARC Easy and Challenge. To achieve this, we introduce a self-attention based\nneural network that latently learns to rank documents by their importance\nrelated to a given question, whilst optimizing the objective of predicting the\ncorrect answer. These documents are considered relevant contexts for the\nunderlying question. We have published the ranked documents so that they can be\nused off-the-shelf to improve downstream decision models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00672v2",
            "date": "2019-09-02 11:28:35+00:00",
            "title": "A Method to Learn Embedding of a Probabilistic Medical Knowledge Graph: Algorithm Development",
            "authors": [
                "Linfeng Li",
                "Peng Wang",
                "Yao Wang",
                "Jinpeng Jiang",
                "Buzhou Tang",
                "Jun Yan",
                "Shenghui Wang",
                "Yuting Liu"
            ],
            "abstract": "This paper proposes an algorithm named as PrTransH to learn embedding vectors\nfrom real world EMR data based medical knowledge. The unique challenge in\nembedding medical knowledge graph from real world EMR data is that the\nuncertainty of knowledge triplets blurs the border between \"correct triplet\"\nand \"wrong triplet\", changing the fundamental assumption of many existing\nalgorithms. To address the challenge, some enhancements are made to existing\nTransH algorithm, including: 1) involve probability of medical knowledge\ntriplet into training objective; 2) replace the margin-based ranking loss with\nunified loss calculation considering both valid and corrupted triplets; 3)\naugment training data set with medical background knowledge. Verifications on\nreal world EMR data based medical knowledge graph prove that PrTransH\noutperforms TransH in link prediction task. To the best of our survey, this\npaper is the first one to learn and verify knowledge embedding on probabilistic\nknowledge graphs.",
            "journal ref": "JMIR Med Inform 2020;8(5):e17645",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.00754v2",
            "date": "2019-09-02 15:00:08+00:00",
            "title": "Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation",
            "authors": [
                "Liliang Ren",
                "Jianmo Ni",
                "Julian McAuley"
            ],
            "abstract": "Existing approaches to dialogue state tracking rely on pre-defined ontologies\nconsisting of a set of all possible slot types and values. Though such\napproaches exhibit promising performance on single-domain benchmarks, they\nsuffer from computational complexity that increases proportionally to the\nnumber of pre-defined slots that need tracking. This issue becomes more severe\nwhen it comes to multi-domain dialogues which include larger numbers of slots.\nIn this paper, we investigate how to approach DST using a generation framework\nwithout the pre-defined ontology list. Given each turn of user utterance and\nsystem response, we directly generate a sequence of belief states by applying a\nhierarchical encoder-decoder structure. In this way, the computational\ncomplexity of our model will be a constant regardless of the number of\npre-defined slots. Experiments on both the multi-domain and the single domain\ndialogue state tracking dataset show that our model not only scales easily with\nthe increasing number of pre-defined domains and slots but also reaches the\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.00764v1",
            "date": "2019-09-02 15:23:55+00:00",
            "title": "The CL-SciSumm Shared Task 2018: Results and Key Insights",
            "authors": [
                "Kokil Jaidka",
                "Michihiro Yasunaga",
                "Muthu Kumar Chandrasekaran",
                "Dragomir Radev",
                "Min-Yen Kan"
            ],
            "abstract": "This overview describes the official results of the CL-SciSumm Shared Task\n2018 -- the first medium-scale shared task on scientific document summarization\nin the computational linguistics (CL) domain. This year, the dataset comprised\n60 annotated sets of citing and reference papers from the open access research\npapers in the CL domain. The Shared Task was organized as a part of the 41st\nAnnual Conference of the Special Interest Group in Information Retrieval\n(SIGIR), held in Ann Arbor, USA in July 2018. We compare the participating\nsystems in terms of two evaluation metrics. The annotated dataset and\nevaluation scripts can be accessed and used by the community from:\n\\url{https://github.com/WING-NUS/scisumm-corpus}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00871v3",
            "date": "2019-09-02 21:33:03+00:00",
            "title": "It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution",
            "authors": [
                "Rowan Hall Maudslay",
                "Hila Gonen",
                "Ryan Cotterell",
                "Simone Teufel"
            ],
            "abstract": "This paper treats gender bias latent in word embeddings. Previous mitigation\nattempts rely on the operationalisation of gender bias as a projection over a\nlinear subspace. An alternative approach is Counterfactual Data Augmentation\n(CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by\nswapping all inherently-gendered words in the copy. We perform an empirical\ncomparison of these approaches on the English Gigaword and Wikipedia, and find\nthat whilst both successfully reduce direct bias and perform well in tasks\nwhich quantify embedding quality, CDA variants outperform projection-based\nmethods at the task of drawing non-biased gender analogies by an average of 19%\nacross both corpora. We propose two improvements to CDA: Counterfactual Data\nSubstitution (CDS), a variant of CDA in which potentially biased text is\nrandomly substituted to avoid duplication, and the Names Intervention, a novel\nname-pairing technique that vastly increases the number of words being treated.\nCDA/S with the Names Intervention is the only approach which is able to\nmitigate indirect gender bias: following debiasing, previously biased words are\nsignificantly less clustered according to gender (cluster purity is reduced by\n49%), thus improving on the state-of-the-art for bias mitigation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00923v1",
            "date": "2019-09-03 02:31:47+00:00",
            "title": "Attributed Rhetorical Structure Grammar for Domain Text Summarization",
            "authors": [
                "Ruqian Lu",
                "Shengluan Hou",
                "Chuanqing Wang",
                "Yu Huang",
                "Chaoqun Fei",
                "Songmao Zhang"
            ],
            "abstract": "This paper presents a new approach of automatic text summarization which\ncombines domain oriented text analysis (DoTA) and rhetorical structure theory\n(RST) in a grammar form: the attributed rhetorical structure grammar (ARSG),\nwhere the non-terminal symbols are domain keywords, called domain relations,\nwhile the rhetorical relations serve as attributes. We developed machine\nlearning algorithms for learning such a grammar from a corpus of sample domain\ntexts, as well as parsing algorithms for the learned grammar, together with\nadjustable text summarization algorithms for generating domain specific\nsummaries. Our practical experiments have shown that with support of domain\nknowledge the drawback of missing very large training data set can be\neffectively compensated. We have also shown that the knowledge based approach\nmay be made more powerful by introducing grammar parsing and RST as inference\nengine. For checking the feasibility of model transfer, we introduced a\ntechnique for mapping a grammar from one domain to others with acceptable cost.\nWe have also made a comprehensive comparison of our approach with some others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00925v2",
            "date": "2019-09-03 02:45:28+00:00",
            "title": "Adversarial Bootstrapping for Dialogue Model Training",
            "authors": [
                "Oluwatobi Olabiyi",
                "Erik T. Mueller",
                "Christopher Larson",
                "Tarek Lahlou"
            ],
            "abstract": "Open domain neural dialogue models, despite their successes, are known to\nproduce responses that lack relevance, diversity, and in many cases coherence.\nThese shortcomings stem from the limited ability of common training objectives\nto directly express these properties as well as their interplay with training\ndatasets and model architectures. Toward addressing these problems, this paper\nproposes bootstrapping a dialogue response generator with an adversarially\ntrained discriminator. The method involves training a neural generator in both\nautoregressive and traditional teacher-forcing modes, with the maximum\nlikelihood loss of the auto-regressive outputs weighted by the score from a\nmetric-based discriminator model. The discriminator input is a mixture of\nground truth labels, the teacher-forcing outputs of the generator, and\ndistractors sampled from the dataset, thereby allowing for richer feedback on\nthe autoregressive outputs of the generator. To improve the calibration of the\ndiscriminator output, we also bootstrap the discriminator with the matching of\nthe intermediate features of the ground truth and the generator's\nautoregressive output. We explore different sampling and adversarial policy\noptimization strategies during training in order to understand how to encourage\nresponse diversity without sacrificing relevance. Our experiments shows that\nadversarial bootstrapping is effective at addressing exposure bias, leading to\nimprovement in response relevance and coherence. The improvement is\ndemonstrated with the state-of-the-art results on the Movie and Ubuntu dialogue\ndatasets with respect to human evaluations and BLUE, ROGUE, and distinct n-gram\nscores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00931v1",
            "date": "2019-09-03 03:06:46+00:00",
            "title": "Transfer Fine-Tuning: A BERT Case Study",
            "authors": [
                "Yuki Arase",
                "Junichi Tsujii"
            ],
            "abstract": "A semantic equivalence assessment is defined as a task that assesses semantic\nequivalence in a sentence pair by binary judgment (i.e., paraphrase\nidentification) or grading (i.e., semantic textual similarity measurement). It\nconstitutes a set of tasks crucial for research on natural language\nunderstanding. Recently, BERT realized a breakthrough in sentence\nrepresentation learning (Devlin et al., 2019), which is broadly transferable to\nvarious NLP tasks. While BERT's performance improves by increasing its model\nsize, the required computational power is an obstacle preventing practical\napplications from adopting the technology. Herein, we propose to inject phrasal\nparaphrase relations into BERT in order to generate suitable representations\nfor semantic equivalence assessment instead of increasing the model size.\nExperiments on standard natural language understanding tasks confirm that our\nmethod effectively improves a smaller BERT model while maintaining the model\nsize. The generated model exhibits superior performance compared to a larger\nBERT model on semantic equivalence assessment tasks. Furthermore, it achieves\nlarger performance gains on tasks with limited training datasets for\nfine-tuning, which is a property desirable for transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00986v1",
            "date": "2019-09-03 07:29:48+00:00",
            "title": "Certified Robustness to Adversarial Word Substitutions",
            "authors": [
                "Robin Jia",
                "Aditi Raghunathan",
                "Kerem G\u00f6ksel",
                "Percy Liang"
            ],
            "abstract": "State-of-the-art NLP models can often be fooled by adversaries that apply\nseemingly innocuous label-preserving transformations (e.g., paraphrasing) to\ninput text. The number of possible transformations scales exponentially with\ntext length, so data augmentation cannot cover all transformations of an input.\nThis paper considers one exponentially large family of label-preserving\ntransformations, in which every word in the input can be replaced with a\nsimilar word. We train the first models that are provably robust to all word\nsubstitutions in this family. Our training procedure uses Interval Bound\nPropagation (IBP) to minimize an upper bound on the worst-case loss that any\ncombination of word substitutions can induce. To evaluate models' robustness to\nthese transformations, we measure accuracy on adversarially chosen word\nsubstitutions applied to test examples. Our IBP-trained models attain $75\\%$\nadversarial accuracy on both sentiment analysis on IMDB and natural language\ninference on SNLI. In comparison, on IMDB, models trained normally and ones\ntrained with data augmentation achieve adversarial accuracy of only $8\\%$ and\n$35\\%$, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01053v1",
            "date": "2019-09-03 10:42:35+00:00",
            "title": "Towards Making a Dependency Parser See",
            "authors": [
                "Michalina Strzyz",
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We explore whether it is possible to leverage eye-tracking data in an RNN\ndependency parser (for English) when such information is only available during\ntraining, i.e., no aggregated or token-level gaze features are used at\ninference time. To do so, we train a multitask learning model that parses\nsentences as sequence labeling and leverages gaze features as auxiliary tasks.\nOur method also learns to train from disjoint datasets, i.e. it can be used to\ntest whether already collected gaze features are useful to improve the\nperformance on new non-gazed annotated treebanks. Accuracy gains are modest but\npositive, showing the feasibility of the approach. It can serve as a first step\ntowards architectures that can better leverage eye-tracking data or other\ncomplementary information available only for training sentences, possibly\nleading to improvements in syntactic parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01063v2",
            "date": "2019-09-03 11:04:19+00:00",
            "title": "A Smart Sliding Chinese Pinyin Input Method Editor on Touchscreen",
            "authors": [
                "Zhuosheng Zhang",
                "Zhen Meng",
                "Hai Zhao"
            ],
            "abstract": "This paper presents a smart sliding Chinese pinyin Input Method Editor (IME)\nfor touchscreen devices which allows user finger sliding from one key to\nanother on the touchscreen instead of tapping keys one by one, while the target\nChinese character sequence will be predicted during the sliding process to help\nuser input Chinese characters efficiently. Moreover, the layout of the virtual\nkeyboard of our IME adapts to user sliding for more efficient inputting. The\nlayout adaption process is utilized with Recurrent Neural Networks (RNN) and\ndeep reinforcement learning. The pinyin-to-character converter is implemented\nwith a sequence-to-sequence (Seq2Seq) model to predict the target Chinese\nsequence. A sliding simulator is built to automatically produce sliding samples\nfor model training and virtual keyboard test. The key advantage of our proposed\nIME is that nearly all its built-in tactics can be optimized automatically with\ndeep learning algorithms only following user behavior. Empirical studies verify\nthe effectiveness of the proposed model and show a better user input\nefficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01146v1",
            "date": "2019-08-30 03:16:10+00:00",
            "title": "Bilingual is At Least Monolingual (BALM): A Novel Translation Algorithm that Encodes Monolingual Priors",
            "authors": [
                "Jeffrey Cheng",
                "Chris Callison-Burch"
            ],
            "abstract": "State-of-the-art machine translation (MT) models do not use knowledge of any\nsingle language's structure; this is the equivalent of asking someone to\ntranslate from English to German while knowing neither language. BALM is a\nframework incorporates monolingual priors into an MT pipeline; by casting input\nand output languages into embedded space using BERT, we can solve machine\ntranslation with much simpler models. We find that English-to-German\ntranslation on the Multi30k dataset can be solved with a simple feedforward\nnetwork under the BALM framework with near-SOTA BLEU scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01259v2",
            "date": "2019-09-03 15:50:34+00:00",
            "title": "Neural Attentive Bag-of-Entities Model for Text Classification",
            "authors": [
                "Ikuya Yamada",
                "Hiroyuki Shindo"
            ],
            "abstract": "This study proposes a Neural Attentive Bag-of-Entities model, which is a\nneural network model that performs text classification using entities in a\nknowledge base. Entities provide unambiguous and relevant semantic signals that\nare beneficial for capturing semantics in texts. We combine simple high-recall\nentity detection based on a dictionary, to detect entities in a document, with\na novel neural attention mechanism that enables the model to focus on a small\nnumber of unambiguous and relevant entities. We tested the effectiveness of our\nmodel using two standard text classification datasets (i.e., the 20 Newsgroups\nand R8 datasets) and a popular factoid question answering dataset based on a\ntrivia quiz game. As a result, our model achieved state-of-the-art results on\nall datasets. The source code of the proposed model is available online at\nhttps://github.com/wikipedia2vec/wikipedia2vec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01322v1",
            "date": "2019-09-03 17:35:27+00:00",
            "title": "CMU GetGoing: An Understandable and Memorable Dialog System for Seniors",
            "authors": [
                "Shikib Mehri",
                "Alan W Black",
                "Maxine Eskenazi"
            ],
            "abstract": "Voice-based technologies are typically developed for the average user, and\nthus generally not tailored to the specific needs of any subgroup of the\npopulation, like seniors. This paper presents CMU GetGoing, an accessible trip\nplanning dialog system designed for senior users. The GetGoing system design is\ndescribed in detail, with particular attention to the senior-tailored features.\nA user study is presented, demonstrating that the senior-tailored features\nsignificantly improve comprehension and retention of information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01326v2",
            "date": "2019-09-03 17:50:44+00:00",
            "title": "The Woman Worked as a Babysitter: On Biases in Language Generation",
            "authors": [
                "Emily Sheng",
                "Kai-Wei Chang",
                "Premkumar Natarajan",
                "Nanyun Peng"
            ],
            "abstract": "We present a systematic study of biases in natural language generation (NLG)\nby analyzing text generated from prompts that contain mentions of different\ndemographic groups. In this work, we introduce the notion of the regard towards\na demographic, use the varying levels of regard towards different demographics\nas a defining metric for bias in NLG, and analyze the extent to which sentiment\nscores are a relevant proxy metric for regard. To this end, we collect\nstrategically-generated text from language models and manually annotate the\ntext with both sentiment and regard scores. Additionally, we build an automatic\nregard classifier through transfer learning, so that we can analyze biases in\nunseen text. Together, these methods reveal the extent of the biased nature of\nlanguage model generations. Our analysis provides a study of biases in NLG,\nbias metrics and correlated human judgments, and empirical evidence on the\nusefulness of our annotated dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01388v1",
            "date": "2019-09-03 18:22:24+00:00",
            "title": "How to Build User Simulators to Train RL-based Dialog Systems",
            "authors": [
                "Weiyan Shi",
                "Kun Qian",
                "Xuewei Wang",
                "Zhou Yu"
            ],
            "abstract": "User simulators are essential for training reinforcement learning (RL) based\ndialog models. The performance of the simulator directly impacts the RL policy.\nHowever, building a good user simulator that models real user behaviors is\nchallenging. We propose a method of standardizing user simulator building that\ncan be used by the community to compare dialog system quality using the same\nset of user simulators fairly. We present implementations of six user\nsimulators trained with different dialog planning and generation methods. We\nthen calculate a set of automatic metrics to evaluate the quality of these\nsimulators both directly and indirectly. We also ask human users to assess the\nsimulators directly and indirectly by rating the simulated dialogs and\ninteracting with the trained systems. This paper presents a comprehensive\nevaluation framework for user simulator study and provides a better\nunderstanding of the pros and cons of different user simulators, as well as\ntheir impacts on the trained systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01474v1",
            "date": "2019-09-03 22:20:24+00:00",
            "title": "Effective Strategies for Using Hashtags in Online Communication",
            "authors": [
                "Solomiia Fedushko",
                "Sofia Kolos"
            ],
            "abstract": "The features of use of hashtags among students of Lviv were investigated. The\nlist of optimal strategies for using these communicative tools for personal\nbranding is determined. The effective strategy for using hashtags in online\ncommunication for the personal and company branding is considered. The results\nof calculation of effectiveness of hashtags related to #education is\ncalculated. The reports of using hashtag #education in social networks is\npresented.",
            "journal ref": "International Journal of Computing and Related Technologies,\n  Volume 2, Issue 2, 2019",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1909.01515v1",
            "date": "2019-09-04 01:35:47+00:00",
            "title": "Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs",
            "authors": [
                "Mingyang Chen",
                "Wen Zhang",
                "Wei Zhang",
                "Qiang Chen",
                "Huajun Chen"
            ],
            "abstract": "Link prediction is an important way to complete knowledge graphs (KGs), while\nembedding-based methods, effective for link prediction in KGs, perform poorly\non relations that only have a few associative triples. In this work, we propose\na Meta Relational Learning (MetaR) framework to do the common but challenging\nfew-shot link prediction in KGs, namely predicting new triples about a relation\nby only observing a few associative triples. We solve few-shot link prediction\nby focusing on transferring relation-specific meta information to make model\nlearn the most important knowledge and learn faster, corresponding to relation\nmeta and gradient meta respectively in MetaR. Empirically, our model achieves\nstate-of-the-art results on few-shot link prediction KG benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01528v1",
            "date": "2019-09-04 02:42:07+00:00",
            "title": "Referring Expression Generation Using Entity Profiles",
            "authors": [
                "Meng Cao",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Referring Expression Generation (REG) is the task of generating contextually\nappropriate references to entities. A limitation of existing REG systems is\nthat they rely on entity-specific supervised training, which means that they\ncannot handle entities not seen during training. In this study, we address this\nin two ways. First, we propose task setups in which we specifically test a REG\nsystem's ability to generalize to entities not seen during training. Second, we\npropose a profile-based deep neural network model, ProfileREG, which encodes\nboth the local context and an external profile of the entity to generate\nreference realizations. Our model generates tokens by learning to choose\nbetween generating pronouns, generating from a fixed vocabulary, or copying a\nword from the profile. We evaluate our model on three different splits of the\nWebNLG dataset, and show that it outperforms competitive baselines in all\nsettings according to automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01567v1",
            "date": "2019-09-04 06:26:05+00:00",
            "title": "A Non-commutative Bilinear Model for Answering Path Queries in Knowledge Graphs",
            "authors": [
                "Katsuhiko Hayashi",
                "Masashi Shimbo"
            ],
            "abstract": "Bilinear diagonal models for knowledge graph embedding (KGE), such as\nDistMult and ComplEx, balance expressiveness and computational efficiency by\nrepresenting relations as diagonal matrices. Although they perform well in\npredicting atomic relations, composite relations (relation paths) cannot be\nmodeled naturally by the product of relation matrices, as the product of\ndiagonal matrices is commutative and hence invariant with the order of\nrelations. In this paper, we propose a new bilinear KGE model, called\nBlockHolE, based on block circulant matrices. In BlockHolE, relation matrices\ncan be non-commutative, allowing composite relations to be modeled by matrix\nproduct. The model is parameterized in a way that covers a spectrum ranging\nfrom diagonal to full relation matrices. A fast computation technique is\ndeveloped on the basis of the duality of the Fourier transform of circulant\nmatrices.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.01720v1",
            "date": "2019-09-04 12:15:02+00:00",
            "title": "Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection",
            "authors": [
                "Lianwei Wu",
                "Yuan Rao",
                "Haolin Jin",
                "Ambreen Nazir",
                "Ling Sun"
            ],
            "abstract": "Recently, neural networks based on multi-task learning have achieved\npromising performance on fake news detection, which focus on learning shared\nfeatures among tasks as complementary features to serve different tasks.\nHowever, in most of the existing approaches, the shared features are completely\nassigned to different tasks without selection, which may lead to some useless\nand even adverse features integrated into specific tasks. In this paper, we\ndesign a sifted multi-task learning method with a selected sharing layer for\nfake news detection. The selected sharing layer adopts gate mechanism and\nattention mechanism to filter and select shared feature flows between tasks.\nExperiments on two public and widely used competition datasets, i.e. RumourEval\nand PHEME, demonstrate that our proposed method achieves the state-of-the-art\nperformance and boosts the F1-score by more than 0.87%, 1.31%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01761v1",
            "date": "2019-08-30 23:19:41+00:00",
            "title": "Single Training Dimension Selection for Word Embedding with PCA",
            "authors": [
                "Yu Wang"
            ],
            "abstract": "In this paper, we present a fast and reliable method based on PCA to select\nthe number of dimensions for word embeddings. First, we train one embedding\nwith a generous upper bound (e.g. 1,000) of dimensions. Then we transform the\nembeddings using PCA and incrementally remove the lesser dimensions one at a\ntime while recording the embeddings' performance on language tasks. Lastly, we\nselect the number of dimensions while balancing model size and accuracy.\nExperiments using various datasets and language tasks demonstrate that we are\nable to train 10 times fewer sets of embeddings while retaining optimal\nperformance. Researchers interested in training the best-performing embeddings\nfor downstream tasks, such as sentiment analysis, question answering and\nhypernym extraction, as well as those interested in embedding compression\nshould find the method helpful.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01772v1",
            "date": "2019-09-04 13:12:30+00:00",
            "title": "Affect Enriched Word Embeddings for News Information Retrieval",
            "authors": [
                "Tommaso Teofili",
                "Niyati Chhaya"
            ],
            "abstract": "Distributed representations of words have shown to be useful to improve the\neffectiveness of IR systems in many sub-tasks like query expansion, retrieval\nand ranking. Algorithms like word2vec, GloVe and others are also key factors in\nmany improvements in different NLP tasks. One common issue with such embedding\nmodels is that words like happy and sad appear in similar contexts and hence\nare wrongly clustered close in the embedding space. In this paper we leverage\nAff2Vec, a set of word embeddings models which include affect information, in\norder to better capture the affect aspect in news text to achieve better\nresults in information retrieval tasks, also such embeddings are less hit by\nthe synonym/antonym issue. We evaluate their effectiveness on two IR related\ntasks (query expansion and ranking) over the New York Times dataset (TREC-core\n'17) comparing them against other word embeddings based models and classic\nranking models.",
            "journal ref": "NewsIR@SIGIR 2019: 63-68",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.01958v3",
            "date": "2019-09-04 17:33:42+00:00",
            "title": "From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project",
            "authors": [
                "Peter Clark",
                "Oren Etzioni",
                "Daniel Khashabi",
                "Tushar Khot",
                "Bhavana Dalvi Mishra",
                "Kyle Richardson",
                "Ashish Sabharwal",
                "Carissa Schoenick",
                "Oyvind Tafjord",
                "Niket Tandon",
                "Sumithra Bhakthavatsalam",
                "Dirk Groeneveld",
                "Michal Guerquin",
                "Michael Schmitz"
            ],
            "abstract": "AI has achieved remarkable mastery over games such as Chess, Go, and Poker,\nand even Jeopardy, but the rich variety of standardized exams has remained a\nlandmark challenge. Even in 2016, the best AI system achieved merely 59.3% on\nan 8th Grade science exam challenge. This paper reports unprecedented success\non the Grade 8 New York Regents Science Exam, where for the first time a system\nscores more than 90% on the exam's non-diagram, multiple choice (NDMC)\nquestions. In addition, our Aristo system, building upon the success of recent\nlanguage models, exceeded 83% on the corresponding Grade 12 Science Exam NDMC\nquestions. The results, on unseen test questions, are robust across different\ntest years and different variations of this kind of test. They demonstrate that\nmodern NLP methods can result in mastery on this task. While not a full\nsolution to general question-answering (the questions are multiple choice, and\nthe domain is restricted to 8th Grade science), it represents a significant\nmilestone for the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02050v1",
            "date": "2019-09-04 18:43:04+00:00",
            "title": "TIGEr: Text-to-Image Grounding for Image Caption Evaluation",
            "authors": [
                "Ming Jiang",
                "Qiuyuan Huang",
                "Lei Zhang",
                "Xin Wang",
                "Pengchuan Zhang",
                "Zhe Gan",
                "Jana Diesner",
                "Jianfeng Gao"
            ],
            "abstract": "This paper presents a new metric called TIGEr for the automatic evaluation of\nimage captioning systems. Popular metrics, such as BLEU and CIDEr, are based\nsolely on text matching between reference captions and machine-generated\ncaptions, potentially leading to biased evaluations because references may not\nfully cover the image content and natural language is inherently ambiguous.\nBuilding upon a machine-learned text-image grounding model, TIGEr allows to\nevaluate caption quality not only based on how well a caption represents image\ncontent, but also on how well machine-generated captions match human-generated\ncaptions. Our empirical tests show that TIGEr has a higher consistency with\nhuman judgments than alternative existing metrics. We also comprehensively\nassess the metric's effectiveness in caption evaluation by measuring the\ncorrelation between human judgments and metric scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02097v1",
            "date": "2019-09-04 20:37:30+00:00",
            "title": "Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering",
            "authors": [
                "Soravit Changpinyo",
                "Bo Pang",
                "Piyush Sharma",
                "Radu Soricut"
            ],
            "abstract": "Object detection plays an important role in current solutions to vision and\nlanguage tasks like image captioning and visual question answering. However,\npopular models like Faster R-CNN rely on a costly process of annotating\nground-truths for both the bounding boxes and their corresponding semantic\nlabels, making it less amenable as a primitive task for transfer learning. In\nthis paper, we examine the effect of decoupling box proposal and featurization\nfor down-stream tasks. The key insight is that this allows us to leverage a\nlarge amount of labeled annotations that were previously unavailable for\nstandard object detection benchmarks. Empirically, we demonstrate that this\nleads to effective transfer learning and improved image captioning and visual\nquestion answering models, as measured on publicly available benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02117v1",
            "date": "2019-09-04 21:13:23+00:00",
            "title": "Learning Dynamic Context Augmentation for Global Entity Linking",
            "authors": [
                "Xiyuan Yang",
                "Xiaotao Gu",
                "Sheng Lin",
                "Siliang Tang",
                "Yueting Zhuang",
                "Fei Wu",
                "Zhigang Chen",
                "Guoping Hu",
                "Xiang Ren"
            ],
            "abstract": "Despite of the recent success of collective entity linking (EL) methods,\nthese \"global\" inference methods may yield sub-optimal results when the\n\"all-mention coherence\" assumption breaks, and often suffer from high\ncomputational cost at the inference stage, due to the complex search space. In\nthis paper, we propose a simple yet effective solution, called Dynamic Context\nAugmentation (DCA), for collective EL, which requires only one pass through the\nmentions in a document. DCA sequentially accumulates context information to\nmake efficient, collective inference, and can cope with different local EL\nmodels as a plug-and-enhance module. We explore both supervised and\nreinforcement learning strategies for learning the DCA model. Extensive\nexperiments show the effectiveness of our model with different learning\nsettings, base models, decision orders and attention mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02126v1",
            "date": "2019-09-04 21:45:51+00:00",
            "title": "Reporting the Unreported: Event Extraction for Analyzing the Local Representation of Hate Crimes",
            "authors": [
                "Aida Mostafazadeh Davani",
                "Leigh Yeh",
                "Mohammad Atari",
                "Brendan Kennedy",
                "Gwenyth Portillo-Wightman",
                "Elaine Gonzalez",
                "Natalie Delong",
                "Rhea Bhatia",
                "Arineh Mirinjian",
                "Xiang Ren",
                "Morteza Dehghani"
            ],
            "abstract": "Official reports of hate crimes in the US are under-reported relative to the\nactual number of such incidents. Further, despite statistical approximations,\nthere are no official reports from a large number of US cities regarding\nincidents of hate. Here, we first demonstrate that event extraction and\nmulti-instance learning, applied to a corpus of local news articles, can be\nused to predict instances of hate crime. We then use the trained model to\ndetect incidents of hate in cities for which the FBI lacks statistics. Lastly,\nwe train models on predicting homicide and kidnapping, compare the predictions\nto FBI reports, and establish that incidents of hate are indeed under-reported,\ncompared to other types of crimes, in local press.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02151v1",
            "date": "2019-09-04 23:37:25+00:00",
            "title": "KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning",
            "authors": [
                "Bill Yuchen Lin",
                "Xinyue Chen",
                "Jamin Chen",
                "Xiang Ren"
            ],
            "abstract": "Commonsense reasoning aims to empower machines with the human ability to make\npresumptions about ordinary situations in our daily life. In this paper, we\npropose a textual inference framework for answering commonsense questions,\nwhich effectively utilizes external, structured commonsense knowledge graphs to\nperform explainable inferences. The framework first grounds a question-answer\npair from the semantic space to the knowledge-based symbolic space as a schema\ngraph, a related sub-graph of external knowledge graphs. It represents schema\ngraphs with a novel knowledge-aware graph network module named KagNet, and\nfinally scores answers with graph representations. Our model is based on graph\nconvolutional networks and LSTMs, with a hierarchical path-based attention\nmechanism. The intermediate attention scores make it transparent and\ninterpretable, which thus produce trustworthy inferences. Using ConceptNet as\nthe only external resource for Bert-based models, we achieved state-of-the-art\nperformance on the CommonsenseQA, a large-scale dataset for commonsense\nreasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02164v5",
            "date": "2019-09-05 00:25:17+00:00",
            "title": "TabFact: A Large-scale Dataset for Table-based Fact Verification",
            "authors": [
                "Wenhu Chen",
                "Hongmin Wang",
                "Jianshu Chen",
                "Yunkai Zhang",
                "Hong Wang",
                "Shiyang Li",
                "Xiyou Zhou",
                "William Yang Wang"
            ],
            "abstract": "The problem of verifying whether a textual hypothesis holds based on the\ngiven evidence, also known as fact verification, plays an important role in the\nstudy of natural language understanding and semantic representation. However,\nexisting studies are mainly restricted to dealing with unstructured evidence\n(e.g., natural language sentences and documents, news, etc), while verification\nunder structured evidence, such as tables, graphs, and databases, remains\nunder-explored. This paper specifically aims to study the fact verification\ngiven semi-structured data as evidence. To this end, we construct a large-scale\ndataset called TabFact with 16k Wikipedia tables as the evidence for 118k\nhuman-annotated natural language statements, which are labeled as either\nENTAILED or REFUTED. TabFact is challenging since it involves both soft\nlinguistic reasoning and hard symbolic reasoning. To address these reasoning\nchallenges, we design two different models: Table-BERT and Latent Program\nAlgorithm (LPA). Table-BERT leverages the state-of-the-art pre-trained language\nmodel to encode the linearized tables and statements into continuous vectors\nfor verification. LPA parses statements into programs and executes them against\nthe tables to obtain the returned binary value for verification. Both methods\nachieve similar accuracy but still lag far behind human performance. We also\nperform a comprehensive analysis to demonstrate great future opportunities. The\ndata and code of the dataset are provided in\n\\url{https://github.com/wenhuchen/Table-Fact-Checking}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02195v2",
            "date": "2019-09-05 03:30:26+00:00",
            "title": "Automated Let's Play Commentary",
            "authors": [
                "Shukan Shah",
                "Matthew Guzdial",
                "Mark O. Riedl"
            ],
            "abstract": "Let's Plays of video games represent a relatively unexplored area for\nexperimental AI in games. In this short paper, we discuss an approach to\ngenerate automated commentary for Let's Play videos, drawing on convolutional\ndeep neural networks. We focus on Let's Plays of the popular game Minecraft. We\ncompare our approach and a prior approach and demonstrate the generation of\nautomated, artificial commentary.",
            "journal ref": "Proceedings of the 2019 Experimental AI in Games Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02197v2",
            "date": "2019-09-05 03:32:48+00:00",
            "title": "Investigating Multilingual NMT Representations at Scale",
            "authors": [
                "Sneha Reddy Kudugunta",
                "Ankur Bapna",
                "Isaac Caswell",
                "Naveen Arivazhagan",
                "Orhan Firat"
            ],
            "abstract": "Multilingual Neural Machine Translation (NMT) models have yielded large\nempirical success in transfer learning settings. However, these black-box\nrepresentations are poorly understood, and their mode of transfer remains\nelusive. In this work, we attempt to understand massively multilingual NMT\nrepresentations (with 103 languages) using Singular Value Canonical Correlation\nAnalysis (SVCCA), a representation similarity framework that allows us to\ncompare representations across different languages, layers and models. Our\nanalysis validates several empirical results and long-standing intuitions, and\nunveils new observations regarding how representations evolve in a multilingual\ntranslation model. We draw three major conclusions from our analysis, with\nimplications on cross-lingual transfer learning: (i) Encoder representations of\ndifferent languages cluster based on linguistic similarity, (ii)\nRepresentations of a source language learned by the encoder are dependent on\nthe target language, and vice-versa, and (iii) Representations of high resource\nand/or linguistically similar languages are more robust when fine-tuning on an\narbitrary language pair, which is critical to determining how much\ncross-lingual transfer can be expected in a zero or few-shot setting. We\nfurther connect our findings with existing empirical observations in\nmultilingual NMT and transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02201v2",
            "date": "2019-09-05 04:16:48+00:00",
            "title": "Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach",
            "authors": [
                "Dong-Jin Kim",
                "Jinsoo Choi",
                "Tae-Hyun Oh",
                "In So Kweon"
            ],
            "abstract": "Constructing an organized dataset comprised of a large number of images and\nseveral captions for each image is a laborious task, which requires vast human\neffort. On the other hand, collecting a large number of images and sentences\nseparately may be immensely easier. In this paper, we develop a novel\ndata-efficient semi-supervised framework for training an image captioning\nmodel. We leverage massive unpaired image and caption data by learning to\nassociate them. To this end, our proposed semi-supervised learning method\nassigns pseudo-labels to unpaired samples via Generative Adversarial Networks\nto learn the joint distribution of image and caption. To evaluate, we construct\nscarcely-paired COCO dataset, a modified version of MS COCO caption dataset.\nThe empirical results show the effectiveness of our method compared to several\nstrong baselines, especially when the amount of the paired samples are scarce.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.02217v1",
            "date": "2019-09-05 05:44:46+00:00",
            "title": "REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning",
            "authors": [
                "Ming Jiang",
                "Junjie Hu",
                "Qiuyuan Huang",
                "Lei Zhang",
                "Jana Diesner",
                "Jianfeng Gao"
            ],
            "abstract": "Popular metrics used for evaluating image captioning systems, such as BLEU\nand CIDEr, provide a single score to gauge the system's overall effectiveness.\nThis score is often not informative enough to indicate what specific errors are\nmade by a given system. In this study, we present a fine-grained evaluation\nmethod REO for automatically measuring the performance of image captioning\nsystems. REO assesses the quality of captions from three perspectives: 1)\nRelevance to the ground truth, 2) Extraness of the content that is irrelevant\nto the ground truth, and 3) Omission of the elements in the images and human\nreferences. Experiments on three benchmark datasets demonstrate that our method\nachieves a higher consistency with human judgments and provides more intuitive\nevaluation results than alternative metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02314v2",
            "date": "2019-09-05 10:54:03+00:00",
            "title": "Commonsense Reasoning Using WordNet and SUMO: a Detailed Analysis",
            "authors": [
                "Javier \u00c1lvez",
                "Itziar Gonzalez-Dios",
                "German Rigau"
            ],
            "abstract": "We describe a detailed analysis of a sample of large benchmark of commonsense\nreasoning problems that has been automatically obtained from WordNet, SUMO and\ntheir mapping. The objective is to provide a better assessment of the quality\nof both the benchmark and the involved knowledge resources for advanced\ncommonsense reasoning tasks. By means of this analysis, we are able to detect\nsome knowledge misalignments, mapping errors and lack of knowledge and\nresources. Our final objective is the extraction of some guidelines towards a\nbetter exploitation of this commonsense knowledge framework by the improvement\nof the included resources.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.02480v3",
            "date": "2019-09-05 15:32:34+00:00",
            "title": "FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow",
            "authors": [
                "Xuezhe Ma",
                "Chunting Zhou",
                "Xian Li",
                "Graham Neubig",
                "Eduard Hovy"
            ],
            "abstract": "Most sequence-to-sequence (seq2seq) models are autoregressive; they generate\neach token by conditioning on previously generated tokens. In contrast,\nnon-autoregressive seq2seq models generate all tokens in one pass, which leads\nto increased efficiency through parallel processing on hardware such as GPUs.\nHowever, directly modeling the joint distribution of all tokens simultaneously\nis challenging, and even with increasingly complex model structures accuracy\nlags significantly behind autoregressive models. In this paper, we propose a\nsimple, efficient, and effective model for non-autoregressive sequence\ngeneration using latent variable models. Specifically, we turn to generative\nflow, an elegant technique to model complex distributions using neural\nnetworks, and design several layers of flow tailored for modeling the\nconditional density of sequential latent variables. We evaluate this model on\nthree neural machine translation (NMT) benchmark datasets, achieving comparable\nperformance with state-of-the-art non-autoregressive NMT models and almost\nconstant decoding time w.r.t the sequence length.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02489v1",
            "date": "2019-09-05 15:41:53+00:00",
            "title": "Stack-VS: Stacked Visual-Semantic Attention for Image Caption Generation",
            "authors": [
                "Wei Wei",
                "Ling Cheng",
                "Xianling Mao",
                "Guangyou Zhou",
                "Feida Zhu"
            ],
            "abstract": "Recently, automatic image caption generation has been an important focus of\nthe work on multimodal translation task. Existing approaches can be roughly\ncategorized into two classes, i.e., top-down and bottom-up, the former\ntransfers the image information (called as visual-level feature) directly into\na caption, and the later uses the extracted words (called as semanticlevel\nattribute) to generate a description. However, previous methods either are\ntypically based one-stage decoder or partially utilize part of visual-level or\nsemantic-level information for image caption generation. In this paper, we\naddress the problem and propose an innovative multi-stage architecture (called\nas Stack-VS) for rich fine-gained image caption generation, via combining\nbottom-up and top-down attention models to effectively handle both visual-level\nand semantic-level information of an input image. Specifically, we also propose\na novel well-designed stack decoder model, which is constituted by a sequence\nof decoder cells, each of which contains two LSTM-layers work interactively to\nre-optimize attention weights on both visual-level feature vectors and\nsemantic-level attribute embeddings for generating a fine-gained image caption.\nExtensive experiments on the popular benchmark dataset MSCOCO show the\nsignificant improvements on different evaluation metrics, i.e., the\nimprovements on BLEU-4/CIDEr/SPICE scores are 0.372, 1.226 and 0.216,\nrespectively, as compared to the state-of-the-arts.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.02776v2",
            "date": "2019-09-06 09:04:14+00:00",
            "title": "Features in Extractive Supervised Single-document Summarization: Case of Persian News",
            "authors": [
                "Hosein Rezaei",
                "Seyed Amid Moeinzadeh",
                "Azar Shahgholian",
                "Mohamad Saraee"
            ],
            "abstract": "Text summarization has been one of the most challenging areas of research in\nNLP. Much effort has been made to overcome this challenge by using either the\nabstractive or extractive methods. Extractive methods are more popular, due to\ntheir simplicity compared with the more elaborate abstractive methods. In\nextractive approaches, the system will not generate sentences. Instead, it\nlearns how to score sentences within the text by using some textual features\nand subsequently selecting those with the highest-rank. Therefore, the core\nobjective is ranking and it highly depends on the document. This dependency has\nbeen unnoticed by many state-of-the-art solutions. In this work, the features\nof the document are integrated into vectors of every sentence. In this way, the\nsystem becomes informed about the context, increases the precision of the\nlearned model and consequently produces comprehensive and brief summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02809v1",
            "date": "2019-09-06 10:36:33+00:00",
            "title": "#MeTooMaastricht: Building a chatbot to assist survivors of sexual harassment",
            "authors": [
                "Tobias Bauer",
                "Emre Devrim",
                "Misha Glazunov",
                "William Lopez Jaramillo",
                "Balaganesh Mohan",
                "Gerasimos Spanakis"
            ],
            "abstract": "Inspired by the recent social movement of #MeToo, we are building a chatbot\nto assist survivors of sexual harassment cases (designed for the city of\nMaastricht but can easily be extended). The motivation behind this work is\ntwofold: properly assist survivors of such events by directing them to\nappropriate institutions that can offer them help and increase the incident\ndocumentation so as to gather more data about harassment cases which are\ncurrently under reported. We break down the problem into three data\nscience/machine learning components: harassment type identification (treated as\na classification problem), spatio-temporal information extraction (treated as\nNamed Entity Recognition problem) and dialogue with the users (treated as a\nslot-filling based chatbot). We are able to achieve a success rate of more than\n98% for the identification of a harassment-or-not case and around 80% for the\nspecific type harassment identification. Locations and dates are identified\nwith more than 90% accuracy and time occurrences prove more challenging with\nalmost 80%. Finally, initial validation of the chatbot shows great potential\nfor the further development and deployment of such a beneficial for the whole\nsociety tool.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02965v1",
            "date": "2019-09-06 15:10:37+00:00",
            "title": "User Evaluation of a Multi-dimensional Statistical Dialogue System",
            "authors": [
                "Simon Keizer",
                "Ond\u0159ej Du\u0161ek",
                "Xingkun Liu",
                "Verena Rieser"
            ],
            "abstract": "We present the first complete spoken dialogue system driven by a\nmulti-dimensional statistical dialogue manager. This framework has been shown\nto substantially reduce data needs by leveraging domain-independent dimensions,\nsuch as social obligations or feedback, which (as we show) can be transferred\nbetween domains. In this paper, we conduct a user study and show that the\nperformance of a multi-dimensional system, which can be adapted from a source\ndomain, is equivalent to that of a one-dimensional baseline, which can only be\ntrained from scratch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03099v2",
            "date": "2019-09-06 19:39:37+00:00",
            "title": "Abductive Reasoning as Self-Supervision for Common Sense Question Answering",
            "authors": [
                "Sathyanarayanan N. Aakur",
                "Sudeep Sarkar"
            ],
            "abstract": "Question answering has seen significant advances in recent times, especially\nwith the introduction of increasingly bigger transformer-based models\npre-trained on massive amounts of data. While achieving impressive results on\nmany benchmarks, their performances appear to be proportional to the amount of\ntraining data available in the target domain. In this work, we explore the\nability of current question-answering models to generalize - to both other\ndomains as well as with restricted training data. We find that large amounts of\ntraining data are necessary, both for pre-training as well as fine-tuning to a\ntask, for the models to perform well on the designated task. We introduce a\nnovel abductive reasoning approach based on Grenander's Pattern Theory\nframework to provide self-supervised domain adaptation cues or \"pseudo-labels,\"\nwhich can be used instead of expensive human annotations. The proposed\nself-supervised training regimen allows for effective domain adaptation without\nlosing performance compared to fully supervised baselines. Extensive\nexperiments on two publicly available benchmarks show the efficacy of the\nproposed approach. We show that neural networks models trained using\nself-labeled data can retain up to $75\\%$ of the performance of models trained\non large amounts of human-annotated training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03193v2",
            "date": "2019-09-07 06:09:25+00:00",
            "title": "KG-BERT: BERT for Knowledge Graph Completion",
            "authors": [
                "Liang Yao",
                "Chengsheng Mao",
                "Yuan Luo"
            ],
            "abstract": "Knowledge graphs are important resources for many artificial intelligence\ntasks but often suffer from incompleteness. In this work, we propose to use\npre-trained language models for knowledge graph completion. We treat triples in\nknowledge graphs as textual sequences and propose a novel framework named\nKnowledge Graph Bidirectional Encoder Representations from Transformer\n(KG-BERT) to model these triples. Our method takes entity and relation\ndescriptions of a triple as input and computes scoring function of the triple\nwith the KG-BERT language model. Experimental results on multiple benchmark\nknowledge graphs show that our method can achieve state-of-the-art performance\nin triple classification, link prediction and relation prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03329v2",
            "date": "2019-09-07 20:17:34+00:00",
            "title": "LAMOL: LAnguage MOdeling for Lifelong Language Learning",
            "authors": [
                "Fan-Keng Sun",
                "Cheng-Hao Ho",
                "Hung-Yi Lee"
            ],
            "abstract": "Most research on lifelong learning applies to images or games, but not\nlanguage. We present LAMOL, a simple yet effective method for lifelong language\nlearning (LLL) based on language modeling. LAMOL replays pseudo-samples of\nprevious tasks while requiring no extra memory or model capacity. Specifically,\nLAMOL is a language model that simultaneously learns to solve the tasks and\ngenerate training samples. When the model is trained for a new task, it\ngenerates pseudo-samples of previous tasks for training alongside data for the\nnew task. The results show that LAMOL prevents catastrophic forgetting without\nany sign of intransigence and can perform five very different language tasks\nsequentially with only one model. Overall, LAMOL outperforms previous methods\nby a considerable margin and is only 2-3% worse than multitasking, which is\nusually considered the LLL upper bound. The source code is available at\nhttps://github.com/jojotenya/LAMOL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03396v2",
            "date": "2019-09-08 06:55:53+00:00",
            "title": "Quality Estimation for Image Captions Based on Large-scale Human Evaluations",
            "authors": [
                "Tomer Levinboim",
                "Ashish V. Thapliyal",
                "Piyush Sharma",
                "Radu Soricut"
            ],
            "abstract": "Automatic image captioning has improved significantly over the last few\nyears, but the problem is far from being solved, with state of the art models\nstill often producing low quality captions when used in the wild. In this\npaper, we focus on the task of Quality Estimation (QE) for image captions,\nwhich attempts to model the caption quality from a human perspective and\nwithout access to ground-truth references, so that it can be applied at\nprediction time to detect low-quality captions produced on previously unseen\nimages. For this task, we develop a human evaluation process that collects\ncoarse-grained caption annotations from crowdsourced users, which is then used\nto collect a large scale dataset spanning more than 600k caption quality\nratings. We then carefully validate the quality of the collected ratings and\nestablish baseline models for this new QE task. Finally, we further collect\nfine-grained caption quality annotations from trained raters, and use them to\ndemonstrate that QE models trained over the coarse ratings can effectively\ndetect and filter out low-quality image captions, thereby improving the user\nexperience from captioning systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03464v3",
            "date": "2019-09-08 13:35:12+00:00",
            "title": "Back to the Future -- Sequential Alignment of Text Representations",
            "authors": [
                "Johannes Bjerva",
                "Wouter Kouw",
                "Isabelle Augenstein"
            ],
            "abstract": "Language evolves over time in many ways relevant to natural language\nprocessing tasks. For example, recent occurrences of tokens 'BERT' and 'ELMO'\nin publications refer to neural network architectures rather than persons. This\ntype of temporal signal is typically overlooked, but is important if one aims\nto deploy a machine learning model over an extended period of time. In\nparticular, language evolution causes data drift between time-steps in\nsequential decision-making tasks. Examples of such tasks include prediction of\npaper acceptance for yearly conferences (regular intervals) or author stance\nprediction for rumours on Twitter (irregular intervals). Inspired by successes\nin computer vision, we tackle data drift by sequentially aligning learned\nrepresentations. We evaluate on three challenging tasks varying in terms of\ntime-scales, linguistic units, and domains. These tasks show our method\noutperforming several strong baselines, including using all available data. We\nargue that, due to its low computational expense, sequential alignment is a\npractical solution to dealing with language evolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03493v2",
            "date": "2019-09-08 16:08:04+00:00",
            "title": "MULE: Multimodal Universal Language Embedding",
            "authors": [
                "Donghyun Kim",
                "Kuniaki Saito",
                "Kate Saenko",
                "Stan Sclaroff",
                "Bryan A. Plummer"
            ],
            "abstract": "Existing vision-language methods typically support two languages at a time at\nmost. In this paper, we present a modular approach which can easily be\nincorporated into existing vision-language methods in order to support many\nlanguages. We accomplish this by learning a single shared Multimodal Universal\nLanguage Embedding (MULE) which has been visually-semantically aligned across\nall languages. Then we learn to relate MULE to visual data as if it were a\nsingle language. Our method is not architecture specific, unlike prior work\nwhich typically learned separate branches for each language, enabling our\napproach to easily be adapted to many vision-language methods and tasks. Since\nMULE learns a single language branch in the multimodal model, we can also scale\nto support many languages, and languages with fewer annotations can take\nadvantage of the good representation learned from other (more abundant)\nlanguage data. We demonstrate the effectiveness of MULE on the bidirectional\nimage-sentence retrieval task, supporting up to four languages in a single\nmodel. In addition, we show that Machine Translation can be used for data\naugmentation in multilingual learning, which, combined with MULE, improves mean\nrecall by up to 21.9% on a single-language compared to prior work, with the\nmost significant gains seen on languages with relatively few annotations. Our\ncode is publicly available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.03526v3",
            "date": "2019-09-08 18:31:42+00:00",
            "title": "Multi-Task Bidirectional Transformer Representations for Irony Detection",
            "authors": [
                "Chiyu Zhang",
                "Muhammad Abdul-Mageed"
            ],
            "abstract": "Supervised deep learning requires large amounts of training data. In the\ncontext of the FIRE2019 Arabic irony detection shared task (IDAT@FIRE2019), we\nshow how we mitigate this need by fine-tuning the pre-trained bidirectional\nencoders from transformers (BERT) on gold data in a multi-task setting. We\nfurther improve our models by by further pre-training BERT on `in-domain' data,\nthus alleviating an issue of dialect mismatch in the Google-released BERT\nmodel. Our best model acquires 82.4 macro F1 score, and has the unique\nadvantage of being feature-engineering free (i.e., based exclusively on deep\nlearning).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03553v1",
            "date": "2019-09-08 22:05:19+00:00",
            "title": "QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions",
            "authors": [
                "Oyvind Tafjord",
                "Matt Gardner",
                "Kevin Lin",
                "Peter Clark"
            ],
            "abstract": "We introduce the first open-domain dataset, called QuaRTz, for reasoning\nabout textual qualitative relationships. QuaRTz contains general qualitative\nstatements, e.g., \"A sunscreen with a higher SPF protects the skin longer.\",\ntwinned with 3864 crowdsourced situated questions, e.g., \"Billy is wearing\nsunscreen with a lower SPF than Lucy. Who will be best protected from the\nsun?\", plus annotations of the properties being compared. Unlike previous\ndatasets, the general knowledge is textual and not tied to a fixed set of\nrelationships, and tests a system's ability to comprehend and apply textual\nqualitative knowledge in a novel setting. We find state-of-the-art results are\nsubstantially (20%) below human performance, presenting an open challenge to\nthe NLP community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03582v1",
            "date": "2019-09-09 01:33:01+00:00",
            "title": "Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning",
            "authors": [
                "Peng Xu",
                "Chien-Sheng Wu",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Sensational headlines are headlines that capture people's attention and\ngenerate reader interest. Conventional abstractive headline generation methods,\nunlike human writers, do not optimize for maximal reader attention. In this\npaper, we propose a model that generates sensational headlines without labeled\ndata. We first train a sensationalism scorer by classifying online headlines\nwith many comments (\"clickbait\") against a baseline of headlines generated from\na summarization model. The score from the sensationalism scorer is used as the\nreward for a reinforcement learner. However, maximizing the noisy\nsensationalism reward will generate unnatural phrases instead of sensational\nheadlines. To effectively leverage this noisy reward, we propose a novel loss\nfunction, Auto-tuned Reinforcement Learning (ARL), to dynamically balance\nreinforcement learning (RL) with maximum likelihood estimation (MLE). Human\nevaluation shows that 60.8% of samples generated by our model are sensational,\nwhich is significantly better than the Pointer-Gen baseline and other RL\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03745v3",
            "date": "2019-09-09 10:34:09+00:00",
            "title": "Reasoning Over Semantic-Level Graph for Fact Checking",
            "authors": [
                "Wanjun Zhong",
                "Jingjing Xu",
                "Duyu Tang",
                "Zenan Xu",
                "Nan Duan",
                "Ming Zhou",
                "Jiahai Wang",
                "Jian Yin"
            ],
            "abstract": "Fact checking is a challenging task because verifying the truthfulness of a\nclaim requires reasoning about multiple retrievable evidence. In this work, we\npresent a method suitable for reasoning about the semantic-level structure of\nevidence. Unlike most previous works, which typically represent evidence\nsentences with either string concatenation or fusing the features of isolated\nevidence sentences, our approach operates on rich semantic structures of\nevidence obtained by semantic role labeling. We propose two mechanisms to\nexploit the structure of evidence while leveraging the advances of pre-trained\nmodels like BERT, GPT or XLNet. Specifically, using XLNet as the backbone, we\nfirst utilize the graph structure to re-define the relative distances of words,\nwith the intuition that semantically related words should have short distances.\nThen, we adopt graph convolutional network and graph attention network to\npropagate and aggregate information from neighboring nodes on the graph. We\nevaluate our system on FEVER, a benchmark dataset for fact checking, and find\nthat rich structural information is helpful and both our graph-based mechanisms\nimprove the accuracy. Our model is the state-of-the-art system in terms of both\nofficial evaluation metrics, namely claim verification accuracy and FEVER\nscore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03759v2",
            "date": "2019-09-09 11:05:15+00:00",
            "title": "Neural Conversational QA: Learning to Reason v.s. Exploiting Patterns",
            "authors": [
                "Nikhil Verma",
                "Abhishek Sharma",
                "Dhiraj Madan",
                "Danish Contractor",
                "Harshit Kumar",
                "Sachindra Joshi"
            ],
            "abstract": "Neural Conversational QA tasks like ShARC require systems to answer questions\nbased on the contents of a given passage. On studying recent state-of-the-art\nmodels on the ShARCQA task, we found indications that the models learn spurious\nclues/patterns in the dataset. Furthermore, we show that a heuristic-based\nprogram designed to exploit these patterns can have performance comparable to\nthat of the neural models. In this paper we share our findings about four types\nof patterns found in the ShARC corpus and describe how neural models exploit\nthem. Motivated by the aforementioned findings, we create and share a modified\ndataset that has fewer spurious patterns, consequently allowing models to learn\nbetter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03821v2",
            "date": "2019-09-09 12:58:16+00:00",
            "title": "Combination of Unified Embedding Model and Observed Features for Knowledge Graph Completion",
            "authors": [
                "Takuma Ebisu",
                "Ryutaro Ichise"
            ],
            "abstract": "Knowledge graphs are useful for many artificial intelligence tasks but often\nhave missing data. Hence, a method for completing knowledge graphs is required.\nExisting approaches include embedding models, the Path Ranking Algorithm, and\nrule evaluation models. However, these approaches have limitations. For\nexample, all the information is mixed and difficult to interpret in embedding\nmodels, and traditional rule evaluation models are basically slow. In this\npaper, we provide an integrated view of various approaches and combine them to\ncompensate for their limitations. We first unify state-of-the-art embedding\nmodels, such as ComplEx and TorusE, reinterpreting them as a variant of\ntranslation-based models. Then, we show that these models utilize paths for\nlink prediction and propose a method for evaluating rules based on this idea.\nFinally, we combine an embedding model and observed feature models to predict\nmissing triples. This is possible because all of these models utilize paths. We\nalso conduct experiments, including link prediction tasks, with standard\ndatasets to evaluate our method and framework. The experiments show that our\nmethod can evaluate rules faster than traditional methods and that our\nframework outperforms state-of-the-art models in terms of link prediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.03918v2",
            "date": "2019-09-09 15:18:21+00:00",
            "title": "Hierarchy Parsing for Image Captioning",
            "authors": [
                "Ting Yao",
                "Yingwei Pan",
                "Yehao Li",
                "Tao Mei"
            ],
            "abstract": "It is always well believed that parsing an image into constituent visual\npatterns would be helpful for understanding and representing an image.\nNevertheless, there has not been evidence in support of the idea on describing\nan image with a natural-language utterance. In this paper, we introduce a new\ndesign to model a hierarchy from instance level (segmentation), region level\n(detection) to the whole image to delve into a thorough image understanding for\ncaptioning. Specifically, we present a HIerarchy Parsing (HIP) architecture\nthat novelly integrates hierarchical structure into image encoder. Technically,\nan image decomposes into a set of regions and some of the regions are resolved\ninto finer ones. Each region then regresses to an instance, i.e., foreground of\nthe region. Such process naturally builds a hierarchal tree. A tree-structured\nLong Short-Term Memory (Tree-LSTM) network is then employed to interpret the\nhierarchal structure and enhance all the instance-level, region-level and\nimage-level features. Our HIP is appealing in view that it is pluggable to any\nneural captioning models. Extensive experiments on COCO image captioning\ndataset demonstrate the superiority of HIP. More remarkably, HIP plus a\ntop-down attention-based LSTM decoder increases CIDEr-D performance from 120.1%\nto 127.2% on COCO Karpathy test split. When further endowing instance-level and\nregion-level features from HIP with semantic relation learnt through Graph\nConvolutional Networks (GCN), CIDEr-D is boosted up to 130.6%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.03922v1",
            "date": "2019-09-09 15:19:56+00:00",
            "title": "Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue",
            "authors": [
                "Dongyeop Kang",
                "Anusha Balakrishnan",
                "Pararth Shah",
                "Paul Crook",
                "Y-Lan Boureau",
                "Jason Weston"
            ],
            "abstract": "Traditional recommendation systems produce static rather than interactive\nrecommendations invariant to a user's specific requests, clarifications, or\ncurrent mood, and can suffer from the cold-start problem if their tastes are\nunknown. These issues can be alleviated by treating recommendation as an\ninteractive dialogue task instead, where an expert recommender can sequentially\nask about someone's preferences, react to their requests, and recommend more\nappropriate items. In this work, we collect a goal-driven recommendation\ndialogue dataset (GoRecDial), which consists of 9,125 dialogue games and 81,260\nconversation turns between pairs of human workers recommending movies to each\nother. The task is specifically designed as a cooperative game between two\nplayers working towards a quantifiable common goal. We leverage the dataset to\ndevelop an end-to-end dialogue system that can simultaneously converse and\nrecommend. Models are first trained to imitate the behavior of human players\nwithout considering the task goal itself (supervised training). We then\nfinetune our models on simulated bot-bot conversations between two paired\npre-trained models (bot-play), in order to achieve the dialogue goal. Our\nexperiments show that models finetuned with bot-play learn improved dialogue\nstrategies, reach the dialogue goal more often when paired with a human, and\nare rated as more consistent by humans compared to models trained without\nbot-play. The dataset and code are publicly available through the ParlAI\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04028v1",
            "date": "2019-09-08 05:50:18+00:00",
            "title": "Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses",
            "authors": [
                "Matt Grenander",
                "Yue Dong",
                "Jackie Chi Kit Cheung",
                "Annie Louis"
            ],
            "abstract": "Sentence position is a strong feature for news summarization, since the lead\noften (but not always) summarizes the key points of the article. In this paper,\nwe show that recent neural systems excessively exploit this trend, which\nalthough powerful for many inputs, is also detrimental when summarizing\ndocuments where important content should be extracted from later parts of the\narticle. We propose two techniques to make systems sensitive to the importance\nof content in different parts of the article. The first technique employs\n'unbiased' data; i.e., randomly shuffled sentences of the source document, to\npretrain the model. The second technique uses an auxiliary ROUGE-based loss\nthat encourages the model to distribute importance scores throughout a document\nby mimicking sentence-level ROUGE scores on the training data. We show that\nthese techniques significantly improve the performance of a competitive\nreinforcement learning based extractive system, with the auxiliary loss being\nmore powerful than pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04076v2",
            "date": "2019-09-09 18:08:35+00:00",
            "title": "Counterfactual Story Reasoning and Generation",
            "authors": [
                "Lianhui Qin",
                "Antoine Bosselut",
                "Ari Holtzman",
                "Chandra Bhagavatula",
                "Elizabeth Clark",
                "Yejin Choi"
            ],
            "abstract": "Counterfactual reasoning requires predicting how alternative events, contrary\nto what actually happened, might have resulted in different outcomes. Despite\nbeing considered a necessary component of AI-complete systems, few resources\nhave been developed for evaluating counterfactual reasoning in narratives.\n  In this paper, we propose Counterfactual Story Rewriting: given an original\nstory and an intervening counterfactual event, the task is to minimally revise\nthe story to make it compatible with the given counterfactual event. Solving\nthis task will require deep understanding of causal narrative chains and\ncounterfactual invariance, and integration of such story reasoning capabilities\ninto conditional language generation models.\n  We present TimeTravel, a new dataset of 29,849 counterfactual rewritings,\neach with the original story, a counterfactual event, and human-generated\nrevision of the original story compatible with the counterfactual event.\nAdditionally, we include 80,115 counterfactual \"branches\" without a rewritten\nstoryline to support future work on semi- or un-supervised approaches to\ncounterfactual story rewriting.\n  Finally, we evaluate the counterfactual rewriting capacities of several\ncompetitive baselines based on pretrained language models, and assess whether\ncommon overlap and model-based automatic metrics for text generation correlate\nwell with human scores for counterfactual rewriting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04101v3",
            "date": "2019-09-09 18:54:40+00:00",
            "title": "Neural Naturalist: Generating Fine-Grained Image Comparisons",
            "authors": [
                "Maxwell Forbes",
                "Christine Kaeser-Chen",
                "Piyush Sharma",
                "Serge Belongie"
            ],
            "abstract": "We introduce the new Birds-to-Words dataset of 41k sentences describing\nfine-grained differences between photographs of birds. The language collected\nis highly detailed, while remaining understandable to the everyday observer\n(e.g., \"heart-shaped face,\" \"squat body\"). Paragraph-length descriptions\nnaturally adapt to varying levels of taxonomic and visual distance---drawn from\na novel stratified sampling approach---with the appropriate level of detail. We\npropose a new model called Neural Naturalist that uses a joint image encoding\nand comparative module to generate comparative language, and evaluate the\nresults with humans who must use the descriptions to distinguish real images.\n  Our results indicate promising potential for neural models to explain\ndifferences in visual embedding space using natural language, as well as a\nconcrete path for machine learning to aid citizen scientists in their effort to\npreserve biodiversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04176v1",
            "date": "2019-09-09 22:00:39+00:00",
            "title": "Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification",
            "authors": [
                "Jiawei Wu",
                "Wenhan Xiong",
                "William Yang Wang"
            ],
            "abstract": "Many tasks in natural language processing can be viewed as multi-label\nclassification problems. However, most of the existing models are trained with\nthe standard cross-entropy loss function and use a fixed prediction policy\n(e.g., a threshold of 0.5) for all the labels, which completely ignores the\ncomplexity and dependencies among different labels. In this paper, we propose a\nmeta-learning method to capture these complex label dependencies. More\nspecifically, our method utilizes a meta-learner to jointly learn the training\npolicies and prediction policies for different labels. The training policies\nare then used to train the classifier with the cross-entropy loss function, and\nthe prediction policies are further implemented for prediction. Experimental\nresults on fine-grained entity typing and text classification demonstrate that\nour proposed method can obtain more accurate multi-label classification\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04242v2",
            "date": "2019-09-10 02:35:34+00:00",
            "title": "Mitigating Annotation Artifacts in Natural Language Inference Datasets to Improve Cross-dataset Generalization Ability",
            "authors": [
                "Guanhua Zhang",
                "Bing Bai",
                "Junqi Zhang",
                "Kun Bai",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Natural language inference (NLI) aims at predicting the relationship between\na given pair of premise and hypothesis. However, several works have found that\nthere widely exists a bias pattern called annotation artifacts in NLI datasets,\nmaking it possible to identify the label only by looking at the hypothesis.\nThis irregularity makes the evaluation results over-estimated and affects\nmodels' generalization ability. In this paper, we consider a more trust-worthy\nsetting, i.e., cross-dataset evaluation. We explore the impacts of annotation\nartifacts in cross-dataset testing. Furthermore, we propose a training\nframework to mitigate the impacts of the bias pattern. Experimental results\ndemonstrate that our methods can alleviate the negative effect of the artifacts\nand improve the generalization ability of models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04302v1",
            "date": "2019-09-10 05:46:39+00:00",
            "title": "Multimodal Embeddings from Language Models",
            "authors": [
                "Shao-Yen Tseng",
                "Panayiotis Georgiou",
                "Shrikanth Narayanan"
            ],
            "abstract": "Word embeddings such as ELMo have recently been shown to model word semantics\nwith greater efficacy through contextualized learning on large-scale language\ncorpora, resulting in significant improvement in state of the art across many\nnatural language tasks. In this work we integrate acoustic information into\ncontextualized lexical embeddings through the addition of multimodal inputs to\na pretrained bidirectional language model. The language model is trained on\nspoken language that includes text and audio modalities. The resulting\nrepresentations from this model are multimodal and contain paralinguistic\ninformation which can modify word meanings and provide affective information.\nWe show that these multimodal embeddings can be used to improve over previous\nstate of the art multimodal models in emotion recognition on the CMU-MOSEI\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04303v2",
            "date": "2019-09-10 05:51:12+00:00",
            "title": "Core Semantic First: A Top-down Approach for AMR Parsing",
            "authors": [
                "Deng Cai",
                "Wai Lam"
            ],
            "abstract": "We introduce a novel scheme for parsing a piece of text into its Abstract\nMeaning Representation (AMR): Graph Spanning based Parsing (GSP). One novel\ncharacteristic of GSP is that it constructs a parse graph incrementally in a\ntop-down fashion. Starting from the root, at each step, a new node and its\nconnections to existing nodes will be jointly predicted. The output graph spans\nthe nodes by the distance to the root, following the intuition of first\ngrasping the main ideas then digging into more details. The \\textit{core\nsemantic first} principle emphasizes capturing the main ideas of a sentence,\nwhich is of great interest. We evaluate our model on the latest AMR sembank and\nachieve the state-of-the-art performance in the sense that no heuristic graph\nre-categorization is adopted. More importantly, the experiments show that our\nparser is especially good at obtaining the core semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04367v1",
            "date": "2019-09-10 09:35:23+00:00",
            "title": "Competing Topic Naming Conventions in Quora: Predicting Appropriate Topic Merges and Winning Topics from Millions of Topic Pairs",
            "authors": [
                "Binny Mathew",
                "Suman Kalyan Maity",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "Quora is a popular Q&A site which provides users with the ability to tag\nquestions with multiple relevant topics which helps to attract quality answers.\nThese topics are not predefined but user-defined conventions and it is not so\nrare to have multiple such conventions present in the Quora ecosystem\ndescribing exactly the same concept. In almost all such cases, users (or Quora\nmoderators) manually merge the topic pair into one of the either topics, thus\nselecting one of the competing conventions. An important application for the\nsite therefore is to identify such competing conventions early enough that\nshould merge in future. In this paper, we propose a two-step approach that\nuniquely combines the anomaly detection and the supervised classification\nframeworks to predict whether two topics from among millions of topic pairs are\nindeed competing conventions, and should merge, achieving an F-score of 0.711.\nWe also develop a model to predict the direction of the topic merge, i.e., the\nwinning convention, achieving an F-score of 0.898. Our system is also able to\npredict ~ 25% of the correct case of merges within the first month of the merge\nand ~ 40% of the cases within a year. This is an encouraging result since Quora\nusers on average take 936 days to identify such a correct merge. Human judgment\nexperiments show that our system is able to predict almost all the correct\ncases that humans can predict plus 37.24% correct cases which the humans are\nnot able to identify at all.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1909.04386v1",
            "date": "2019-09-10 10:12:01+00:00",
            "title": "Attesting Biases and Discrimination using Language Semantics",
            "authors": [
                "Xavier Ferrer Aran",
                "Jose M. Such",
                "Natalia Criado"
            ],
            "abstract": "AI agents are increasingly deployed and used to make automated decisions that\naffect our lives on a daily basis. It is imperative to ensure that these\nsystems embed ethical principles and respect human values. We focus on how we\ncan attest to whether AI agents treat users fairly without discriminating\nagainst particular individuals or groups through biases in language. In\nparticular, we discuss human unconscious biases, how they are embedded in\nlanguage, and how AI systems inherit those biases by learning from and\nprocessing human language. Then, we outline a roadmap for future research to\nbetter understand and attest problematic AI biases derived from language.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.04387v1",
            "date": "2019-09-10 10:12:59+00:00",
            "title": "A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents",
            "authors": [
                "Amanda Cercas Curry",
                "Verena Rieser"
            ],
            "abstract": "How should conversational agents respond to verbal abuse through the user? To\nanswer this question, we conduct a large-scale crowd-sourced evaluation of\nabuse response strategies employed by current state-of-the-art systems. Our\nresults show that some strategies, such as \"polite refusal\" score highly across\nthe board, while for other strategies demographic factors, such as age, as well\nas the severity of the preceding abuse influence the user's perception of which\nresponse is appropriate. In addition, we find that most data-driven models lag\nbehind rule-based or commercial systems in terms of their perceived\nappropriateness.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1909.04393v1",
            "date": "2019-09-10 10:32:49+00:00",
            "title": "Extending the Service Composition Formalism with Relational Parameters",
            "authors": [
                "Paul Diac",
                "Liana Tucar",
                "Radu Mereuta"
            ],
            "abstract": "Web Service Composition deals with the (re)use of Web Services to provide\ncomplex functionality, inexistent in any single service. Over the\nstate-of-the-art, we introduce a new type of modeling, based on ontologies and\nrelations between objects, which allows us to extend the expressiveness of\nproblems that can be solved automatically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04455v1",
            "date": "2019-09-10 13:01:27+00:00",
            "title": "Learning review representations from user and product level information for spam detection",
            "authors": [
                "Chunyuan Yuan",
                "Wei Zhou",
                "Qianwen Ma",
                "Shangwen Lv",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Opinion spam has become a widespread problem in social media, where hired\nspammers write deceptive reviews to promote or demote products to mislead the\nconsumers for profit or fame. Existing works mainly focus on manually designing\ndiscrete textual or behavior features, which cannot capture complex semantics\nof reviews. Although recent works apply deep learning methods to learn\nreview-level semantic features, their models ignore the impact of the\nuser-level and product-level information on learning review semantics and the\ninherent user-review-product relationship information. In this paper, we\npropose a Hierarchical Fusion Attention Network (HFAN) to automatically learn\nthe semantics of reviews from the user and product level. Specifically, we\ndesign a multi-attention unit to extract user(product)-related review\ninformation. Then, we use orthogonal decomposition and fusion attention to\nlearn a user, review, and product representation from the review information.\nFinally, we take the review as a relation between user and product entity and\napply TransH to jointly encode this relationship into review representation.\nExperimental results obtained more than 10\\% absolute precision improvement\nover the state-of-the-art performances on four real-world datasets, which show\nthe effectiveness and versatility of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04499v1",
            "date": "2019-09-10 14:05:28+00:00",
            "title": "Countering Language Drift via Visual Grounding",
            "authors": [
                "Jason Lee",
                "Kyunghyun Cho",
                "Douwe Kiela"
            ],
            "abstract": "Emergent multi-agent communication protocols are very different from natural\nlanguage and not easily interpretable by humans. We find that agents that were\ninitially pretrained to produce natural language can also experience\ndetrimental language drift: when a non-linguistic reward is used in a\ngoal-based task, e.g. some scalar success metric, the communication protocol\nmay easily and radically diverge from natural language. We recast translation\nas a multi-agent communication game and examine auxiliary training constraints\nfor their effectiveness in mitigating language drift. We show that a\ncombination of syntactic (language model likelihood) and semantic (visual\ngrounding) constraints gives the best communication performance, allowing\npre-trained agents to retain English syntax while learning to accurately convey\nthe intended meaning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04739v1",
            "date": "2019-09-10 20:37:39+00:00",
            "title": "WIQA: A dataset for \"What if...\" reasoning over procedural text",
            "authors": [
                "Niket Tandon",
                "Bhavana Dalvi Mishra",
                "Keisuke Sakaguchi",
                "Antoine Bosselut",
                "Peter Clark"
            ],
            "abstract": "We introduce WIQA, the first large-scale dataset of \"What if...\" questions\nover procedural text. WIQA contains three parts: a collection of paragraphs\neach describing a process, e.g., beach erosion; a set of crowdsourced influence\ngraphs for each paragraph, describing how one change affects another; and a\nlarge (40k) collection of \"What if...?\" multiple-choice questions derived from\nthe graphs. For example, given a paragraph about beach erosion, would stormy\nweather result in more or less erosion (or have no effect)? The task is to\nanswer the questions, given their associated paragraph. WIQA contains three\nkinds of questions: perturbations to steps mentioned in the paragraph; external\n(out-of-paragraph) perturbations requiring commonsense knowledge; and\nirrelevant (no effect) perturbations. We find that state-of-the-art models\nachieve 73.8% accuracy, well below the human performance of 96.3%. We analyze\nthe challenges, in particular tracking chains of influences, and present the\ndataset as an open challenge to the community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04745v2",
            "date": "2019-09-10 20:46:56+00:00",
            "title": "Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text",
            "authors": [
                "Bhavana Dalvi Mishra",
                "Niket Tandon",
                "Antoine Bosselut",
                "Wen-tau Yih",
                "Peter Clark"
            ],
            "abstract": "Our goal is to better comprehend procedural text, e.g., a paragraph about\nphotosynthesis, by not only predicting what happens, but why some actions need\nto happen before others. Our approach builds on a prior process comprehension\nframework for predicting actions' effects, to also identify subsequent steps\nthat those effects enable. We present our new model (XPAD) that biases effect\npredictions towards those that (1) explain more of the actions in the paragraph\nand (2) are more plausible with respect to background knowledge. We also extend\nan existing benchmark dataset for procedural text comprehension, ProPara, by\nadding the new task of explaining actions by predicting their dependencies. We\nfind that XPAD significantly outperforms prior systems on this task, while\nmaintaining the performance on the original task in ProPara. The dataset is\navailable at http://data.allenai.org/propara",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04761v2",
            "date": "2019-09-10 21:30:54+00:00",
            "title": "MultiFiT: Efficient Multi-lingual Language Model Fine-tuning",
            "authors": [
                "Julian Martin Eisenschlos",
                "Sebastian Ruder",
                "Piotr Czapla",
                "Marcin Kardas",
                "Sylvain Gugger",
                "Jeremy Howard"
            ],
            "abstract": "Pretrained language models are promising particularly for low-resource\nlanguages as they only require unlabelled data. However, training existing\nmodels requires huge amounts of compute, while pretrained cross-lingual models\noften underperform on low-resource languages. We propose Multi-lingual language\nmodel Fine-Tuning (MultiFiT) to enable practitioners to train and fine-tune\nlanguage models efficiently in their own language. In addition, we propose a\nzero-shot method using an existing pretrained cross-lingual model. We evaluate\nour methods on two widely used cross-lingual classification datasets where they\noutperform models pretrained on orders of magnitude more data and compute. We\nrelease all models and code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04849v1",
            "date": "2019-09-11 04:47:36+00:00",
            "title": "A Discrete Hard EM Approach for Weakly Supervised Question Answering",
            "authors": [
                "Sewon Min",
                "Danqi Chen",
                "Hannaneh Hajishirzi",
                "Luke Zettlemoyer"
            ],
            "abstract": "Many question answering (QA) tasks only provide weak supervision for how the\nanswer should be computed. For example, TriviaQA answers are entities that can\nbe mentioned multiple times in supporting documents, while DROP answers can be\ncomputed by deriving many different equations from numbers in the reference\ntext. In this paper, we show it is possible to convert such tasks into discrete\nlatent variable learning problems with a precomputed, task-specific set of\npossible \"solutions\" (e.g. different mentions or equations) that contains one\ncorrect option. We then develop a hard EM learning scheme that computes\ngradients relative to the most likely solution at each update. Despite its\nsimplicity, we show that this approach significantly outperforms previous\nmethods on six QA tasks, including absolute gains of 2--10%, and achieves the\nstate-of-the-art on five of them. Using hard updates instead of maximizing\nmarginal likelihood is key to these results as it encourages the model to find\nthe one correct answer, which we show through detailed qualitative analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04925v1",
            "date": "2019-09-11 08:55:09+00:00",
            "title": "How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations",
            "authors": [
                "Betty van Aken",
                "Benjamin Winter",
                "Alexander L\u00f6ser",
                "Felix A. Gers"
            ],
            "abstract": "Bidirectional Encoder Representations from Transformers (BERT) reach\nstate-of-the-art results in a variety of Natural Language Processing tasks.\nHowever, understanding of their internal functioning is still insufficient and\nunsatisfactory. In order to better understand BERT and other Transformer-based\nmodels, we present a layer-wise analysis of BERT's hidden states. Unlike\nprevious research, which mainly focuses on explaining Transformer models by\ntheir attention weights, we argue that hidden states contain equally valuable\ninformation. Specifically, our analysis focuses on models fine-tuned on the\ntask of Question Answering (QA) as an example of a complex downstream task. We\ninspect how QA models transform token vectors in order to find the correct\nanswer. To this end, we apply a set of general and QA-specific probing tasks\nthat reveal the information stored in each representation layer. Our\nqualitative analysis of hidden state visualizations provides additional\ninsights into BERT's reasoning process. Our results show that the\ntransformations within BERT go through phases that are related to traditional\npipeline tasks. The system can therefore implicitly incorporate task-specific\ninformation into its token representations. Furthermore, our analysis reveals\nthat fine-tuning has little impact on the models' semantic abilities and that\nprediction errors can be recognized in the vector representations of even early\nlayers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05192v3",
            "date": "2019-09-10 15:19:51+00:00",
            "title": "The Longer the Better? The Interplay Between Review Length and Line of Argumentation in Online Consumer Reviews",
            "authors": [
                "Bernhard Lutz",
                "Nicolas Pr\u00f6llochs",
                "Dirk Neumann"
            ],
            "abstract": "Review helpfulness serves as focal point in understanding customers' purchase\ndecision-making process on online retailer platforms. An overwhelming majority\nof previous works find longer reviews to be more helpful than short reviews. In\nthis paper, we propose that longer reviews should not be assumed to be\nuniformly more helpful; instead, we argue that the effect depends on the line\nof argumentation in the review text. To test this idea, we use a large dataset\nof customer reviews from Amazon in combination with a state-of-the-art approach\nfrom natural language processing that allows us to study argumentation lines at\nsentence level. Our empirical analysis suggests that the frequency of\nargumentation changes moderates the effect of review length on helpfulness.\nAltogether, we disprove the prevailing narrative that longer reviews are\nuniformly perceived as more helpful. Our findings allow retailer platforms to\nimprove their customer feedback systems and to feature more useful product\nreviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05355v1",
            "date": "2019-08-31 04:03:26+00:00",
            "title": "Let's Ask Again: Refine Network for Automatic Question Generation",
            "authors": [
                "Preksha Nema",
                "Akash Kumar Mohankumar",
                "Mitesh M. Khapra",
                "Balaji Vasan Srinivasan",
                "Balaraman Ravindran"
            ],
            "abstract": "In this work, we focus on the task of Automatic Question Generation (AQG)\nwhere given a passage and an answer the task is to generate the corresponding\nquestion. It is desired that the generated question should be (i) grammatically\ncorrect (ii) answerable from the passage and (iii) specific to the given\nanswer. An analysis of existing AQG models shows that they produce questions\nwhich do not adhere to one or more of {the above-mentioned qualities}. In\nparticular, the generated questions look like an incomplete draft of the\ndesired question with a clear scope for refinement. {To alleviate this\nshortcoming}, we propose a method which tries to mimic the human process of\ngenerating questions by first creating an initial draft and then refining it.\nMore specifically, we propose Refine Network (RefNet) which contains two\ndecoders. The second decoder uses a dual attention network which pays attention\nto both (i) the original passage and (ii) the question (initial draft)\ngenerated by the first decoder. In effect, it refines the question generated by\nthe first decoder, thereby making it more correct and complete. We evaluate\nRefNet on three datasets, \\textit{viz.}, SQuAD, HOTPOT-QA, and DROP, and show\nthat it outperforms existing state-of-the-art methods by 7-16\\% on all of these\ndatasets. Lastly, we show that we can improve the quality of the second decoder\non specific metrics, such as, fluency and answerability by explicitly rewarding\nrevisions that improve on the corresponding metric during training. The code\nhas been made publicly available\n\\footnote{https://github.com/PrekshaNema25/RefNet-QG}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05359v1",
            "date": "2019-09-03 08:27:37+00:00",
            "title": "From Textual Information Sources to Linked Data in the Agatha Project",
            "authors": [
                "Paulo Quaresma",
                "Vitor Beires Nogueira",
                "Kashyap Raiyani",
                "Roy Bayot",
                "Teresa Gon\u00e7alves"
            ],
            "abstract": "Automatic reasoning about textual information is a challenging task in modern\nNatural Language Processing (NLP) systems. In this work we describe our\nproposal for representing and reasoning about Portuguese documents by means of\nLinked Data like ontologies and thesauri. Our approach resorts to a specialized\npipeline of natural language processing (part-of-speech tagger, named entity\nrecognition, semantic role labeling) to populate an ontology for the domain of\ncriminal investigations. The provided architecture and ontology are language\nindependent. Although some of the NLP modules are language dependent, they can\nbe built using adequate AI methodologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05360v2",
            "date": "2019-09-02 18:00:43+00:00",
            "title": "Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction",
            "authors": [
                "Rujun Han",
                "Qiang Ning",
                "Nanyun Peng"
            ],
            "abstract": "We propose a joint event and temporal relation extraction model with shared\nrepresentation learning and structured prediction. The proposed method has two\nadvantages over existing work. First, it improves event representation by\nallowing the event and relation modules to share the same contextualized\nembeddings and neural representation learner. Second, it avoids error\npropagation in the conventional pipeline systems by leveraging structured\ninference and learning methods to assign both the event labels and the temporal\nrelation labels jointly. Experiments show that the proposed method can improve\nboth event extraction and temporal relation extraction over state-of-the-art\nsystems, with the end-to-end F1 improved by 10% and 6.8% on two benchmark\ndatasets respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05361v1",
            "date": "2019-09-03 18:11:58+00:00",
            "title": "Structuring Latent Spaces for Stylized Response Generation",
            "authors": [
                "Xiang Gao",
                "Yizhe Zhang",
                "Sungjin Lee",
                "Michel Galley",
                "Chris Brockett",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "Generating responses in a targeted style is a useful yet challenging task,\nespecially in the absence of parallel data. With limited data, existing methods\ntend to generate responses that are either less stylized or less\ncontext-relevant. We propose StyleFusion, which bridges conversation modeling\nand non-parallel style transfer by sharing a structured latent space. This\nstructure allows the system to generate stylized relevant responses by sampling\nin the neighborhood of the conversation model prediction, and continuously\ncontrol the style level. We demonstrate this method using dialogues from Reddit\ndata and two sets of sentences with distinct styles (arXiv and Sherlock Holmes\nnovels). Automatic and human evaluation show that, without sacrificing\nappropriateness, the system generates responses of the targeted style and\noutperforms competitive baselines.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05363v1",
            "date": "2019-09-05 01:13:41+00:00",
            "title": "Identifying and Explaining Discriminative Attributes",
            "authors": [
                "Armins Stepanjans",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "Identifying what is at the center of the meaning of a word and what\ndiscriminates it from other words is a fundamental natural language inference\ntask. This paper describes an explicit word vector representation model (WVM)\nto support the identification of discriminative attributes. A core contribution\nof the paper is a quantitative and qualitative comparative analysis of\ndifferent types of data sources and Knowledge Bases in the construction of\nexplainable and explicit WVMs: (i) knowledge graphs built from dictionary\ndefinitions, (ii) entity-attribute-relationships graphs derived from images and\n(iii) commonsense knowledge graphs. Using a detailed quantitative and\nqualitative analysis, we demonstrate that these data sources have complementary\nsemantic aspects, supporting the creation of explicit semantic vector spaces.\nThe explicit vector spaces are evaluated using the task of discriminative\nattribute identification, showing comparable performance to the\nstate-of-the-art systems in the task (F1-score = 0.69), while delivering full\nmodel transparency and explainability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05364v3",
            "date": "2019-09-05 14:03:35+00:00",
            "title": "TransSent: Towards Generation of Structured Sentences with Discourse Marker",
            "authors": [
                "Xing Wu",
                "Dongjun Wei",
                "Liangjun Zang",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Structured sentences are important expressions in human writings and\ndialogues. Previous works on neural text generation fused semantic and\nstructural information by encoding the entire sentence into a mixed hidden\nrepresentation. However, when a generated sentence becomes complicated, the\nstructure is difficult to be properly maintained. To alleviate this problem, we\nexplicitly separate the modeling process of semantic and structural\ninformation. Intuitively, humans generate structured sentences by directly\nconnecting discourses with discourse markers (such as and, but, etc.).\nTherefore, we propose a task that mimics this process, called discourse\ntransfer. This task represents a structured sentence as (head discourse,\ndiscourse marker, tail discourse), and aims at tail discourse generation based\non head discourse and discourse marker. We also propose a corresponding model\ncalled TransSent, which interprets the relationship between two discourses as a\ntranslation1 from the head discourse to the tail discourse in the embedding\nspace. We experiment TransSent not only in discourse transfer task but also in\nfree text generation and dialogue generation tasks. Automatic and human\nevaluation results show that TransSent can generate structured sentences with\nhigh quality, and has certain scalability in different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05378v1",
            "date": "2019-09-11 21:15:47+00:00",
            "title": "CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases",
            "authors": [
                "Tao Yu",
                "Rui Zhang",
                "He Yang Er",
                "Suyi Li",
                "Eric Xue",
                "Bo Pang",
                "Xi Victoria Lin",
                "Yi Chern Tan",
                "Tianze Shi",
                "Zihan Li",
                "Youxuan Jiang",
                "Michihiro Yasunaga",
                "Sungrok Shim",
                "Tao Chen",
                "Alexander Fabbri",
                "Zifan Li",
                "Luyao Chen",
                "Yuwen Zhang",
                "Shreya Dixit",
                "Vincent Zhang",
                "Caiming Xiong",
                "Richard Socher",
                "Walter S Lasecki",
                "Dragomir Radev"
            ],
            "abstract": "We present CoSQL, a corpus for building cross-domain, general-purpose\ndatabase (DB) querying dialogue systems. It consists of 30k+ turns plus 10k+\nannotated SQL queries, obtained from a Wizard-of-Oz (WOZ) collection of 3k\ndialogues querying 200 complex DBs spanning 138 domains. Each dialogue\nsimulates a real-world DB query scenario with a crowd worker as a user\nexploring the DB and a SQL expert retrieving answers with SQL, clarifying\nambiguous questions, or otherwise informing of unanswerable questions. When\nuser questions are answerable by SQL, the expert describes the SQL and\nexecution results to the user, hence maintaining a natural interaction flow.\nCoSQL introduces new challenges compared to existing task-oriented dialogue\ndatasets:(1) the dialogue states are grounded in SQL, a domain-independent\nexecutable representation, instead of domain-specific slot-value pairs, and (2)\nbecause testing is done on unseen databases, success requires generalizing to\nnew domains. CoSQL includes three tasks: SQL-grounded dialogue state tracking,\nresponse generation from query results, and user dialogue act prediction. We\nevaluate a set of strong baselines for each task and show that CoSQL presents\nsignificant challenges for future research. The dataset, baselines, and\nleaderboard will be released at https://yale-lily.github.io/cosql.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05398v3",
            "date": "2019-09-11 22:41:00+00:00",
            "title": "Interactive Fiction Games: A Colossal Adventure",
            "authors": [
                "Matthew Hausknecht",
                "Prithviraj Ammanabrolu",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Xingdi Yuan"
            ],
            "abstract": "A hallmark of human intelligence is the ability to understand and communicate\nwith language. Interactive Fiction games are fully text-based simulation\nenvironments where a player issues text commands to effect change in the\nenvironment and progress through the story. We argue that IF games are an\nexcellent testbed for studying language-based autonomous agents. In particular,\nIF games combine challenges of combinatorial action spaces, language\nunderstanding, and commonsense reasoning. To facilitate rapid development of\nlanguage-based agents, we introduce Jericho, a learning environment for\nman-made IF games and conduct a comprehensive study of text-agents across a\nrich set of games, highlighting directions in which agents can improve.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.05528v1",
            "date": "2019-09-12 09:27:37+00:00",
            "title": "MOSS: End-to-End Dialog System Framework with Modular Supervision",
            "authors": [
                "Weixin Liang",
                "Youzhi Tian",
                "Chengcai Chen",
                "Zhou Yu"
            ],
            "abstract": "A major bottleneck in training end-to-end task-oriented dialog system is the\nlack of data. To utilize limited training data more efficiently, we propose\nModular Supervision Network (MOSS), an encoder-decoder training framework that\ncould incorporate supervision from various intermediate dialog system modules\nincluding natural language understanding, dialog state tracking, dialog policy\nlearning, and natural language generation. With only 60% of the training data,\nMOSS-all (i.e., MOSS with supervision from all four dialog modules) outperforms\nstate-of-the-art models on CamRest676. Moreover, introducing modular\nsupervision has even bigger benefits when the dialog task has a more complex\ndialog state and action space. With only 40% of the training data, MOSS-all\noutperforms the state-of-the-art model on a complex laptop network\ntroubleshooting dataset, LaptopNetwork, that we introduced. LaptopNetwork\nconsists of conversations between real customers and customer service agents in\nChinese. Moreover, MOSS framework can accommodate dialogs that have supervision\nfrom different dialog modules at both the framework level and model level.\nTherefore, MOSS is extremely flexible to update in a real-world deployment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.05608v1",
            "date": "2019-09-12 12:50:34+00:00",
            "title": "ABSApp: A Portable Weakly-Supervised Aspect-Based Sentiment Extraction System",
            "authors": [
                "Oren Pereg",
                "Daniel Korat",
                "Moshe Wasserblat",
                "Jonathan Mamou",
                "Ido Dagan"
            ],
            "abstract": "We present ABSApp, a portable system for weakly-supervised aspect-based\nsentiment extraction. The system is interpretable and user friendly and does\nnot require labeled training data, hence can be rapidly and cost-effectively\nused across different domains in applied setups. The system flow includes three\nstages: First, it generates domain-specific aspect and opinion lexicons based\non an unlabeled dataset; second, it enables the user to view and edit those\nlexicons (weak supervision); and finally, it enables the user to select an\nunlabeled target dataset from the same domain, classify it, and generate an\naspect-based sentiment report. ABSApp has been successfully used in a number of\nreal-life use cases, among them movie review analysis and convention impact\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05658v1",
            "date": "2019-09-12 13:46:58+00:00",
            "title": "UER: An Open-Source Toolkit for Pre-training Models",
            "authors": [
                "Zhe Zhao",
                "Hui Chen",
                "Jinbin Zhang",
                "Xin Zhao",
                "Tao Liu",
                "Wei Lu",
                "Xi Chen",
                "Haotang Deng",
                "Qi Ju",
                "Xiaoyong Du"
            ],
            "abstract": "Existing works, including ELMO and BERT, have revealed the importance of\npre-training for NLP tasks. While there does not exist a single pre-training\nmodel that works best in all cases, it is of necessity to develop a framework\nthat is able to deploy various pre-training models efficiently. For this\npurpose, we propose an assemble-on-demand pre-training toolkit, namely\nUniversal Encoder Representations (UER). UER is loosely coupled, and\nencapsulated with rich modules. By assembling modules on demand, users can\neither reproduce a state-of-the-art pre-training model or develop a\npre-training model that remains unexplored. With UER, we have built a model\nzoo, which contains pre-trained models based on different corpora, encoders,\nand targets (objectives). With proper pre-trained models, we could achieve new\nstate-of-the-art results on a range of downstream datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05725v1",
            "date": "2019-09-12 14:43:38+00:00",
            "title": "InstructableCrowd: Creating IF-THEN Rules for Smartphones via Conversations with the Crowd",
            "authors": [
                "Ting-Hao 'Kenneth' Huang",
                "Amos Azaria",
                "Oscar J. Romero",
                "Jeffrey P. Bigham"
            ],
            "abstract": "Natural language interfaces have become a common part of modern digital life.\nChatbots utilize text-based conversations to communicate with users; personal\nassistants on smartphones such as Google Assistant take direct speech commands\nfrom their users; and speech-controlled devices such as Amazon Echo use voice\nas their only input mode. In this paper, we introduce InstructableCrowd, a\ncrowd-powered system that allows users to program their devices via\nconversation. The user verbally expresses a problem to the system, in which a\ngroup of crowd workers collectively respond and program relevant multi-part\nIF-THEN rules to help the user. The IF-THEN rules generated by\nInstructableCrowd connect relevant sensor combinations (e.g., location,\nweather, device acceleration, etc.) to useful effectors (e.g., text messages,\ndevice alarms, etc.). Our study showed that non-programmers can use the\nconversational interface of InstructableCrowd to create IF-THEN rules that have\nsimilar quality compared with the rules created manually. InstructableCrowd\ngenerally illustrates how users may converse with their devices, not only to\ntrigger simple voice commands, but also to personalize their increasingly\npowerful and complicated devices.",
            "journal ref": "Human Computation (2019) 6:1:113-146",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1909.05840v2",
            "date": "2019-09-12 17:45:59+00:00",
            "title": "Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT",
            "authors": [
                "Sheng Shen",
                "Zhen Dong",
                "Jiayu Ye",
                "Linjian Ma",
                "Zhewei Yao",
                "Amir Gholami",
                "Michael W. Mahoney",
                "Kurt Keutzer"
            ],
            "abstract": "Transformer based architectures have become de-facto models used for a range\nof Natural Language Processing tasks. In particular, the BERT based models\nachieved significant accuracy gain for GLUE tasks, CoNLL-03 and SQuAD. However,\nBERT based models have a prohibitive memory footprint and latency. As a result,\ndeploying BERT based models in resource constrained environments has become a\nchallenging task. In this work, we perform an extensive analysis of fine-tuned\nBERT models using second order Hessian information, and we use our results to\npropose a novel method for quantizing BERT models to ultra low precision. In\nparticular, we propose a new group-wise quantization scheme, and we use a\nHessian based mix-precision method to compress the model further. We\nextensively test our proposed method on BERT downstream tasks of SST-2, MNLI,\nCoNLL-03, and SQuAD. We can achieve comparable performance to baseline with at\nmost $2.3\\%$ performance degradation, even with ultra-low precision\nquantization down to 2 bits, corresponding up to $13\\times$ compression of the\nmodel parameters, and up to $4\\times$ compression of the embedding table as\nwell as activations. Among all tasks, we observed the highest performance loss\nfor BERT fine-tuned on SQuAD. By probing into the Hessian based analysis as\nwell as visualization, we show that this is related to the fact that current\ntraining/fine-tuning strategy of BERT does not converge for SQuAD.",
            "journal ref": "AAAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06091v2",
            "date": "2019-09-13 08:55:08+00:00",
            "title": "Neural Machine Translation with 4-Bit Precision and Beyond",
            "authors": [
                "Alham Fikri Aji",
                "Kenneth Heafield"
            ],
            "abstract": "Neural Machine Translation (NMT) is resource intensive. We design a\nquantization procedure to compress NMT models better for devices with limited\nhardware capability. Because most neural network parameters are near zero, we\nemploy logarithmic quantization in lieu of fixed-point quantization. However,\nwe find bias terms are less amenable to log quantization but note they comprise\na tiny fraction of the model, so we leave them uncompressed. We also propose to\nuse an error-feedback mechanism during retraining, to preserve the compressed\nmodel as a stale gradient. We empirically show that NMT models based on\nTransformer or RNN architecture can be compressed up to 4-bit precision without\nany noticeable quality degradation. Models can be compressed up to binary\nprecision, albeit with lower quality. The RNN architecture seems to be more\nrobust to quantization, compared to the Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06092v2",
            "date": "2019-09-13 08:57:14+00:00",
            "title": "A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces",
            "authors": [
                "Anne Lauscher",
                "Goran Glava\u0161",
                "Simone Paolo Ponzetto",
                "Ivan Vuli\u0107"
            ],
            "abstract": "Distributional word vectors have recently been shown to encode many of the\nhuman biases, most notably gender and racial biases, and models for attenuating\nsuch biases have consequently been proposed. However, existing models and\nstudies (1) operate on under-specified and mutually differing bias definitions,\n(2) are tailored for a particular bias (e.g., gender bias) and (3) have been\nevaluated inconsistently and non-rigorously. In this work, we introduce a\ngeneral framework for debiasing word embeddings. We operationalize the\ndefinition of a bias by discerning two types of bias specification: explicit\nand implicit. We then propose three debiasing models that operate on explicit\nor implicit bias specifications and that can be composed towards more robust\ndebiasing. Finally, we devise a full-fledged evaluation framework in which we\ncouple existing bias metrics with newly proposed ones. Experimental findings\nacross three embedding methods suggest that the proposed debiasing models are\nrobust and widely applicable: they often completely remove the bias both\nimplicitly and explicitly without degradation of semantic information encoded\nin any of the input distributional spaces. Moreover, we successfully transfer\ndebiasing models, by means of cross-lingual embedding spaces, and remove or\nattenuate biases in distributional word vector spaces of languages that lack\nreadily available bias specifications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06194v1",
            "date": "2019-09-12 16:25:05+00:00",
            "title": "Style-aware Neural Model with Application in Authorship Attribution",
            "authors": [
                "Fereshteh Jafariakinabad",
                "Kien A. Hua"
            ],
            "abstract": "Writing style is a combination of consistent decisions associated with a\nspecific author at different levels of language production, including lexical,\nsyntactic, and structural. In this paper, we introduce a style-aware neural\nmodel to encode document information from three stylistic levels and evaluate\nit in the domain of authorship attribution. First, we propose a simple way to\njointly encode syntactic and lexical representations of sentences.\nSubsequently, we employ an attention-based hierarchical neural network to\nencode the syntactic and semantic structure of sentences in documents while\nrewarding the sentences which contribute more to capturing the writing style.\nOur experimental results, based on four benchmark datasets, reveal the benefits\nof encoding document information from all three stylistic levels when compared\nto the baseline methods in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06283v4",
            "date": "2019-09-13 15:10:06+00:00",
            "title": "Toward Automated Quest Generation in Text-Adventure Games",
            "authors": [
                "Prithviraj Ammanabrolu",
                "William Broniec",
                "Alex Mueller",
                "Jeremy Paul",
                "Mark O. Riedl"
            ],
            "abstract": "Interactive fictions, or text-adventures, are games in which a player\ninteracts with a world entirely through textual descriptions and text actions.\nText-adventure games are typically structured as puzzles or quests wherein the\nplayer must execute certain actions in a certain order to succeed. In this\npaper, we consider the problem of procedurally generating a quest, defined as a\nseries of actions required to progress towards a goal, in a text-adventure\ngame. Quest generation in text environments is challenging because they must be\nsemantically coherent. We present and evaluate two quest generation techniques:\n(1) a Markov model, and (2) a neural generative model. We specifically look at\ngenerating quests about cooking and train our models on recipe data. We\nevaluate our techniques with human participant studies looking at perceived\ncreativity and coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06639v2",
            "date": "2019-09-14 17:49:37+00:00",
            "title": "Tree Transformer: Integrating Tree Structures into Self-Attention",
            "authors": [
                "Yau-Shian Wang",
                "Hung-Yi Lee",
                "Yun-Nung Chen"
            ],
            "abstract": "Pre-training Transformer from large-scale raw texts and fine-tuning on the\ndesired task have achieved state-of-the-art results on diverse NLP tasks.\nHowever, it is unclear what the learned attention captures. The attention\ncomputed by attention heads seems not to match human intuitions about\nhierarchical structures. This paper proposes Tree Transformer, which adds an\nextra constraint to attention heads of the bidirectional Transformer encoder in\norder to encourage the attention heads to follow tree structures. The tree\nstructures can be automatically induced from raw texts by our proposed\n\"Constituent Attention\" module, which is simply implemented by self-attention\nbetween two adjacent words. With the same training procedure identical to BERT,\nthe experiments demonstrate the effectiveness of Tree Transformer in terms of\ninducing tree structures, better language modeling, and further learning more\nexplainable attention scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06670v1",
            "date": "2019-09-14 20:31:34+00:00",
            "title": "Delivering Cognitive Behavioral Therapy Using A Conversational SocialRobot",
            "authors": [
                "Francesca Dino",
                "Rohola Zandie",
                "Hojjat Abdollahi",
                "Sarah Schoeder",
                "Mohammad H. Mahoor"
            ],
            "abstract": "Social robots are becoming an integrated part of our daily life due to their\nability to provide companionship and entertainment. A subfield of robotics,\nSocially Assistive Robotics (SAR), is particularly suitable for expanding these\nbenefits into the healthcare setting because of its unique ability to provide\ncognitive, social, and emotional support. This paper presents our recent\nresearch on developing SAR by evaluating the ability of a life-like\nconversational social robot, called Ryan, to administer internet-delivered\ncognitive behavioral therapy (iCBT) to older adults with depression. For Ryan\nto administer the therapy, we developed a dialogue-management system, called\nProgram-R. Using an accredited CBT manual for the treatment of depression, we\ncreated seven hour-long iCBT dialogues and integrated them into Program-R using\nArtificial Intelligence Markup Language (AIML). To assess the effectiveness of\nRobot-based iCBT and users' likability of our approach, we conducted an HRI\nstudy with a cohort of elderly people with mild-to-moderate depression over a\nperiod of four weeks. Quantitative analyses of participant's spoken responses\n(e.g. word count and sentiment analysis), face-scale mood scores, and exit\nsurveys, strongly support the notion robot-based iCBT is a viable alternative\nto traditional human-delivered therapy.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1909.06723v3",
            "date": "2019-09-15 03:35:18+00:00",
            "title": "Natural Language Adversarial Attacks and Defenses in Word Level",
            "authors": [
                "Xiaosen Wang",
                "Hao Jin",
                "Kun He"
            ],
            "abstract": "In recent years, inspired by a mass of researches on adversarial examples for\ncomputer vision, there has been a growing interest in designing adversarial\nattacks for Natural Language Processing (NLP) tasks, followed by very few works\nof adversarial defenses for NLP. To our knowledge, there exists no defense\nmethod against the successful synonym substitution based attacks that aim to\nsatisfy all the lexical, grammatical, semantic constraints and thus are hard to\nbe perceived by humans. We contribute to fill this gap and propose a novel\nadversarial defense method called \\textit{Synonym Encoding Method} (SEM), which\ninserts an encoder before the input layer of the model and then trains the\nmodel to eliminate adversarial perturbations. Extensive experiments demonstrate\nthat SEM can efficiently defend current best synonym substitution based\nadversarial attacks with little decay on the accuracy for benign examples. To\nbetter evaluate SEM, we also design a strong attack method called Improved\nGenetic Algorithm (IGA) that adopts the genetic metaheuristic for synonym\nsubstitution based attacks. Compared with the first genetic based adversarial\nattack proposed in 2018, IGA can achieve higher attack success rate with lower\nword substitution rate, at the same time maintain the transferability of\nadversarial examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06731v2",
            "date": "2019-09-15 04:31:21+00:00",
            "title": "Emu: Enhancing Multilingual Sentence Embeddings with Semantic Specialization",
            "authors": [
                "Wataru Hirota",
                "Yoshihiko Suhara",
                "Behzad Golshan",
                "Wang-Chiew Tan"
            ],
            "abstract": "We present Emu, a system that semantically enhances multilingual sentence\nembeddings. Our framework fine-tunes pre-trained multilingual sentence\nembeddings using two main components: a semantic classifier and a language\ndiscriminator. The semantic classifier improves the semantic similarity of\nrelated sentences, whereas the language discriminator enhances the\nmultilinguality of the embeddings via multilingual adversarial training. Our\nexperimental results based on several language pairs show that our specialized\nembeddings outperform the state-of-the-art multilingual sentence embedding\nmodel on the task of cross-lingual intent classification using only monolingual\nlabeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06743v1",
            "date": "2019-09-15 05:58:09+00:00",
            "title": "Learning Rhyming Constraints using Structured Adversaries",
            "authors": [
                "Harsh Jhamtani",
                "Sanket Vaibhav Mehta",
                "Jaime Carbonell",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Existing recurrent neural language models often fail to capture higher-level\nstructure present in text: for example, rhyming patterns present in poetry.\nMuch prior work on poetry generation uses manually defined constraints which\nare satisfied during decoding using either specialized decoding procedures or\nrejection sampling. The rhyming constraints themselves are typically not\nlearned by the generator. We propose an alternate approach that uses a\nstructured discriminator to learn a poetry generator that directly captures\nrhyming constraints in a generative adversarial setup. By causing the\ndiscriminator to compare poems based only on a learned similarity matrix of\npairs of line ending words, the proposed approach is able to successfully learn\nrhyming patterns in two different English poetry datasets (Sonnet and Limerick)\nwithout explicitly being provided with any phonetic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06805v4",
            "date": "2019-09-15 14:03:40+00:00",
            "title": "Many-to-Many Voice Conversion using Cycle-Consistent Variational Autoencoder with Multiple Decoders",
            "authors": [
                "Keonnyeong Lee",
                "In-Chul Yoo",
                "Dongsuk Yook"
            ],
            "abstract": "One of the obstacles in many-to-many voice conversion is the requirement of\nthe parallel training data, which contain pairs of utterances with the same\nlinguistic content spoken by different speakers. Since collecting such parallel\ndata is a highly expensive task, many works attempted to use non-parallel\ntraining data for many-to-many voice conversion. One of such approaches is\nusing the variational autoencoder (VAE). Though it can handle many-to-many\nvoice conversion without the parallel training, the VAE based voice conversion\nmethods suffer from low sound qualities of the converted speech. One of the\nmajor reasons is because the VAE learns only the self-reconstruction path. The\nconversion path is not trained at all. In this paper, we propose a cycle\nconsistency loss for VAE to explicitly learn the conversion path. In addition,\nwe propose to use multiple decoders to further improve the sound qualities of\nthe conventional VAE based voice conversion methods. The effectiveness of the\nproposed method is validated using objective and the subjective evaluations.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06814v4",
            "date": "2019-09-15 15:21:20+00:00",
            "title": "Automatically Extracting Challenge Sets for Non local Phenomena in Neural Machine Translation",
            "authors": [
                "Leshem Choshen",
                "Omri Abend"
            ],
            "abstract": "We show that the state of the art Transformer Machine Translation (MT) model\nis not biased towards monotonic reordering (unlike previous recurrent neural\nnetwork models), but that nevertheless, long-distance dependencies remain a\nchallenge for the model. Since most dependencies are short-distance, common\nevaluation metrics will be little influenced by how well systems perform on\nthem. We, therefore, propose an automatic approach for extracting challenge\nsets replete with long-distance dependencies and argue that evaluation using\nthis methodology provides a complementary perspective on system performance. To\nsupport our claim, we compile challenge sets for English-German and\nGerman-English, which are much larger than any previously released challenge\nset for MT. The extracted sets are large enough to allow reliable automatic\nevaluation, which makes the proposed approach a scalable and practical solution\nfor evaluating MT performance on the long-tail of syntactic phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06886v1",
            "date": "2019-09-15 21:20:09+00:00",
            "title": "Temporal Self-Attention Network for Medical Concept Embedding",
            "authors": [
                "Xueping Peng",
                "Guodong Long",
                "Tao Shen",
                "Sen Wang",
                "Jing Jiang",
                "Michael Blumenstein"
            ],
            "abstract": "In longitudinal electronic health records (EHRs), the event records of a\npatient are distributed over a long period of time and the temporal relations\nbetween the events reflect sufficient domain knowledge to benefit prediction\ntasks such as the rate of inpatient mortality. Medical concept embedding as a\nfeature extraction method that transforms a set of medical concepts with a\nspecific time stamp into a vector, which will be fed into a supervised learning\nalgorithm. The quality of the embedding significantly determines the learning\nperformance over the medical data. In this paper, we propose a medical concept\nembedding method based on applying a self-attention mechanism to represent each\nmedical concept. We propose a novel attention mechanism which captures the\ncontextual information and temporal relationships between medical concepts. A\nlight-weight neural net, \"Temporal Self-Attention Network (TeSAN)\", is then\nproposed to learn medical concept embedding based solely on the proposed\nattention mechanism. To test the effectiveness of our proposed methods, we have\nconducted clustering and prediction tasks on two public EHRs datasets comparing\nTeSAN against five state-of-the-art embedding methods. The experimental results\ndemonstrate that the proposed TeSAN model is superior to all the compared\nmethods. To the best of our knowledge, this work is the first to exploit\ntemporal self-attentive relations between medical events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07054v1",
            "date": "2019-09-16 08:31:20+00:00",
            "title": "Automatic detection of surgical site infections from a clinical data warehouse",
            "authors": [
                "Marine Qu\u00e9rou\u00e9",
                "Agn\u00e8s Lash\u00e9ras-Bauduin",
                "Vianney Jouhet",
                "Frantz Thiessard",
                "Jean-Marc Vital",
                "Anne-Marie Rogues",
                "S\u00e9bastien Cossin"
            ],
            "abstract": "Reducing the incidence of surgical site infections (SSIs) is one of the\nobjectives of the French nosocomial infection control program. Manual\nmonitoring of SSIs is carried out each year by the hospital hygiene team and\nsurgeons at the University Hospital of Bordeaux. Our goal was to develop an\nautomatic detection algorithm based on hospital information system data. Three\nyears (2015, 2016 and 2017) of manual spine surgery monitoring have been used\nas a gold standard to extract features and train machine learning algorithms.\nThe dataset contained 22 SSIs out of 2133 spine surgeries. Two different\napproaches were compared. The first used several data sources and achieved the\nbest performance but is difficult to generalize to other institutions. The\nsecond was based on free text only with semiautomatic extraction of\ndiscriminant terms. The algorithms managed to identify all the SSIs with 20 and\n26 false positives respectively on the dataset. Another evaluation is underway.\nThese results are encouraging for the development of semi-automated\nsurveillance methods.",
            "journal ref": "TALMED 2019, Aug 2019, Lyon, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07575v3",
            "date": "2019-09-17 03:47:25+00:00",
            "title": "Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation",
            "authors": [
                "Chengyi Wang",
                "Yu Wu",
                "Shujie Liu",
                "Zhenglu Yang",
                "Ming Zhou"
            ],
            "abstract": "End-to-end speech translation, a hot topic in recent years, aims to translate\na segment of audio into a specific language with an end-to-end model.\nConventional approaches employ multi-task learning and pre-training methods for\nthis task, but they suffer from the huge gap between pre-training and\nfine-tuning. To address these issues, we propose a Tandem Connectionist\nEncoding Network (TCEN) which bridges the gap by reusing all subnets in\nfine-tuning, keeping the roles of subnets consistent, and pre-training the\nattention module. Furthermore, we propose two simple but effective methods to\nguarantee the speech encoder outputs and the MT encoder inputs are consistent\nin terms of semantic representation and sequence length. Experimental results\nshow that our model outperforms baselines 2.2 BLEU on a large benchmark\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07606v1",
            "date": "2019-09-17 06:16:04+00:00",
            "title": "K-BERT: Enabling Language Representation with Knowledge Graph",
            "authors": [
                "Weijie Liu",
                "Peng Zhou",
                "Zhe Zhao",
                "Zhiruo Wang",
                "Qi Ju",
                "Haotang Deng",
                "Ping Wang"
            ],
            "abstract": "Pre-trained language representation models, such as BERT, capture a general\nlanguage representation from large-scale corpora, but lack domain-specific\nknowledge. When reading a domain text, experts make inferences with relevant\nknowledge. For machines to achieve this capability, we propose a\nknowledge-enabled language representation model (K-BERT) with knowledge graphs\n(KGs), in which triples are injected into the sentences as domain knowledge.\nHowever, too much knowledge incorporation may divert the sentence from its\ncorrect meaning, which is called knowledge noise (KN) issue. To overcome KN,\nK-BERT introduces soft-position and visible matrix to limit the impact of\nknowledge. K-BERT can easily inject domain knowledge into the models by\nequipped with a KG without pre-training by-self because it is capable of\nloading model parameters from the pre-trained BERT. Our investigation reveals\npromising results in twelve NLP tasks. Especially in domain-specific tasks\n(including finance, law, and medicine), K-BERT significantly outperforms BERT,\nwhich demonstrates that K-BERT is an excellent choice for solving the\nknowledge-driven problems that require experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07739v1",
            "date": "2019-09-17 12:07:22+00:00",
            "title": "Course Concept Expansion in MOOCs with External Knowledge and Interactive Game",
            "authors": [
                "Jifan Yu",
                "Chenyu Wang",
                "Gan Luo",
                "Lei Hou",
                "Juanzi Li",
                "Jie Tang",
                "Zhiyuan Liu"
            ],
            "abstract": "As Massive Open Online Courses (MOOCs) become increasingly popular, it is\npromising to automatically provide extracurricular knowledge for MOOC users.\nSuffering from semantic drifts and lack of knowledge guidance, existing methods\ncan not effectively expand course concepts in complex MOOC environments. In\nthis paper, we first build a novel boundary during searching for new concepts\nvia external knowledge base and then utilize heterogeneous features to verify\nthe high-quality results. In addition, to involve human efforts in our model,\nwe design an interactive optimization mechanism based on a game. Our\nexperiments on the four datasets from Coursera and XuetangX show that the\nproposed method achieves significant improvements(+0.19 by MAP) over existing\nmethods. The source code and datasets have been published.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07755v3",
            "date": "2019-09-17 13:01:12+00:00",
            "title": "Span-based Joint Entity and Relation Extraction with Transformer Pre-training",
            "authors": [
                "Markus Eberts",
                "Adrian Ulges"
            ],
            "abstract": "We introduce SpERT, an attention model for span-based joint entity and\nrelation extraction. Our key contribution is a light-weight reasoning on BERT\nembeddings, which features entity recognition and filtering, as well as\nrelation classification with a localized, marker-free context representation.\nThe model is trained using strong within-sentence negative samples, which are\nefficiently extracted in a single BERT pass. These aspects facilitate a search\nover all spans in the sentence.\n  In ablation studies, we demonstrate the benefits of pre-training, strong\nnegative sampling and localized context. Our model outperforms prior work by up\nto 2.6% F1 score on several datasets for joint entity and relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07863v3",
            "date": "2019-09-17 14:51:59+00:00",
            "title": "Character-Centric Storytelling",
            "authors": [
                "Aditya Surikuchi",
                "Jorma Laaksonen"
            ],
            "abstract": "Sequential vision-to-language or visual storytelling has recently been one of\nthe areas of focus in computer vision and language modeling domains. Though\nexisting models generate narratives that read subjectively well, there could be\ncases when these models miss out on generating stories that account and address\nall prospective human and animal characters in the image sequences. Considering\nthis scenario, we propose a model that implicitly learns relationships between\nprovided characters and thereby generates stories with respective characters in\nscope. We use the VIST dataset for this purpose and report numerous statistics\non the dataset. Eventually, we describe the model, explain the experiment and\ndiscuss our current status and future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07913v2",
            "date": "2019-09-17 16:10:30+00:00",
            "title": "Learning to Deceive with Attention-Based Explanations",
            "authors": [
                "Danish Pruthi",
                "Mansi Gupta",
                "Bhuwan Dhingra",
                "Graham Neubig",
                "Zachary C. Lipton"
            ],
            "abstract": "Attention mechanisms are ubiquitous components in neural architectures\napplied to natural language processing. In addition to yielding gains in\npredictive accuracy, attention weights are often claimed to confer\ninterpretability, purportedly useful both for providing insights to\npractitioners and for explaining why a model makes its decisions to\nstakeholders. We call the latter use of attention mechanisms into question by\ndemonstrating a simple method for training models to produce deceptive\nattention masks. Our method diminishes the total weight assigned to designated\nimpermissible tokens, even when the models can be shown to nevertheless rely on\nthese features to drive predictions. Across multiple models and tasks, our\napproach manipulates attention weights while paying surprisingly little cost in\naccuracy. Through a human study, we show that our manipulated attention-based\nexplanations deceive people into thinking that predictions from a model biased\nagainst gender minorities do not rely on the gender. Consequently, our results\ncast doubt on attention's reliability as a tool for auditing algorithms in the\ncontext of fairness and accountability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07940v2",
            "date": "2019-09-17 17:24:37+00:00",
            "title": "Do NLP Models Know Numbers? Probing Numeracy in Embeddings",
            "authors": [
                "Eric Wallace",
                "Yizhong Wang",
                "Sujian Li",
                "Sameer Singh",
                "Matt Gardner"
            ],
            "abstract": "The ability to understand and work with numbers (numeracy) is critical for\nmany complex reasoning tasks. Currently, most NLP models treat numbers in text\nin the same way as other tokens---they embed them as distributed vectors. Is\nthis enough to capture numeracy? We begin by investigating the numerical\nreasoning capabilities of a state-of-the-art question answering model on the\nDROP dataset. We find this model excels on questions that require numerical\nreasoning, i.e., it already captures numeracy. To understand how this\ncapability emerges, we probe token embedding methods (e.g., BERT, GloVe) on\nsynthetic list maximum, number decoding, and addition tasks. A surprising\ndegree of numeracy is naturally present in standard embeddings. For example,\nGloVe and word2vec accurately encode magnitude for numbers up to 1,000.\nFurthermore, character-level embeddings are even more precise---ELMo captures\nnumeracy the best for all pre-trained methods---but BERT, which uses sub-word\nunits, is less exact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07950v2",
            "date": "2019-09-17 17:31:37+00:00",
            "title": "Semantic Relatedness Based Re-ranker for Text Spotting",
            "authors": [
                "Ahmed Sabir",
                "Francesc Moreno-Noguer",
                "Llu\u00eds Padr\u00f3"
            ],
            "abstract": "Applications such as textual entailment, plagiarism detection or document\nclustering rely on the notion of semantic similarity, and are usually\napproached with dimension reduction techniques like LDA or with embedding-based\nneural approaches. We present a scenario where semantic similarity is not\nenough, and we devise a neural approach to learn semantic relatedness. The\nscenario is text spotting in the wild, where a text in an image (e.g. street\nsign, advertisement or bus destination) must be identified and recognized. Our\ngoal is to improve the performance of vision systems by leveraging semantic\ninformation. Our rationale is that the text to be spotted is often related to\nthe image context in which it appears (word pairs such as Delta-airplane, or\nquarters-parking are not similar, but are clearly related). We show how\nlearning a word-to-word or word-to-sentence relatedness score can improve the\nperformance of text spotting systems up to 2.9 points, outperforming other\nmeasures in a benchmark dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08211v1",
            "date": "2019-09-18 05:02:56+00:00",
            "title": "Modeling Conversation Structure and Temporal Dynamics for Jointly Predicting Rumor Stance and Veracity",
            "authors": [
                "Penghui Wei",
                "Nan Xu",
                "Wenji Mao"
            ],
            "abstract": "Automatically verifying rumorous information has become an important and\nchallenging task in natural language processing and social media analytics.\nPrevious studies reveal that people's stances towards rumorous messages can\nprovide indicative clues for identifying the veracity of rumors, and thus\ndetermining the stances of public reactions is a crucial preceding step for\nrumor veracity prediction. In this paper, we propose a hierarchical multi-task\nlearning framework for jointly predicting rumor stance and veracity on Twitter,\nwhich consists of two components. The bottom component of our framework\nclassifies the stances of tweets in a conversation discussing a rumor via\nmodeling the structural property based on a novel graph convolutional network.\nThe top component predicts the rumor veracity by exploiting the temporal\ndynamics of stance evolution. Experimental results on two benchmark datasets\nshow that our method outperforms previous methods in both rumor stance\nclassification and veracity prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08250v1",
            "date": "2019-09-18 07:09:07+00:00",
            "title": "Natural Language Generation for Non-Expert Users",
            "authors": [
                "Van Duc Nguyen",
                "Tran Cao Son",
                "Enrico Pontelli"
            ],
            "abstract": "Motivated by the difficulty in presenting computational results, especially\nwhen the results are a collection of atoms in a logical language, to users, who\nare not proficient in computer programming and/or the logical representation of\nthe results, we propose a system for automatic generation of natural language\ndescriptions for applications targeting mainstream users. Differently from many\nearlier systems with the same aim, the proposed system does not employ\ntemplates for the generation task. It assumes that there exist some natural\nlanguage sentences in the application domain and uses this repository for the\nnatural language description. It does not require, however, a large corpus as\nit is often required in machine learning approaches. The systems consist of two\nmain components. The first one aims at analyzing the sentences and constructs a\nGrammatical Framework (GF) for given sentences and is implemented using the\nStanford parser and an answer set program. The second component is for sentence\nconstruction and relies on GF Library. The paper includes two use cases to\ndemostrate the capability of the system. As the sentence construction is done\nvia GF, the paper includes a use case evaluation showing that the proposed\nsystem could also be utilized in addressing a challenge to create an abstract\nWikipedia, which is recently discussed in the BlueSky session of the 2018\nInternational Semantic Web Conference.",
            "journal ref": "EPTCS 306, 2019, pp. 280-294",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.08349v1",
            "date": "2019-09-18 10:55:46+00:00",
            "title": "A Lexical, Syntactic, and Semantic Perspective for Understanding Style in Text",
            "authors": [
                "Gaurav Verma",
                "Balaji Vasan Srinivasan"
            ],
            "abstract": "With a growing interest in modeling inherent subjectivity in natural\nlanguage, we present a linguistically-motivated process to understand and\nanalyze the writing style of individuals from three perspectives: lexical,\nsyntactic, and semantic. We discuss the stylistically expressive elements\nwithin each of these levels and use existing methods to quantify the linguistic\nintuitions related to some of these elements. We show that such a multi-level\nanalysis is useful for developing a well-knit understanding of style - which is\nindependent of the natural language task at hand, and also demonstrate its\nvalue in solving three downstream tasks: authors' style analysis, authorship\nattribution, and emotion prediction. We conduct experiments on a variety of\ndatasets, comprising texts from social networking sites, user reviews, legal\ndocuments, literary books, and newswire. The results on the aforementioned\ntasks and datasets illustrate that such a multi-level understanding of style,\nwhich has been largely ignored in recent works, models style-related\nsubjectivity in text and can be leveraged to improve performance on multiple\ndownstream tasks both qualitatively and quantitatively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08478v1",
            "date": "2019-09-18 14:38:34+00:00",
            "title": "Simple, Scalable Adaptation for Neural Machine Translation",
            "authors": [
                "Ankur Bapna",
                "Naveen Arivazhagan",
                "Orhan Firat"
            ],
            "abstract": "Fine-tuning pre-trained Neural Machine Translation (NMT) models is the\ndominant approach for adapting to new languages and domains. However,\nfine-tuning requires adapting and maintaining a separate model for each target\ntask. We propose a simple yet efficient approach for adaptation in NMT. Our\nproposed approach consists of injecting tiny task specific adapter layers into\na pre-trained model. These lightweight adapters, with just a small fraction of\nthe original model size, adapt the model to multiple individual tasks\nsimultaneously. We evaluate our approach on two tasks: (i) Domain Adaptation\nand (ii) Massively Multilingual NMT. Experiments on domain adaptation\ndemonstrate that our proposed approach is on par with full fine-tuning on\nvarious domains, dataset sizes and model capacities. On a massively\nmultilingual dataset of 103 languages, our adaptation approach bridges the gap\nbetween individual bilingual models and one massively multilingual model for\nmost language pairs, paving the way towards universal machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08491v1",
            "date": "2019-09-18 15:00:56+00:00",
            "title": "Word Recognition, Competition, and Activation in a Model of Visually Grounded Speech",
            "authors": [
                "William N. Havard",
                "Jean-Pierre Chevrot",
                "Laurent Besacier"
            ],
            "abstract": "In this paper, we study how word-like units are represented and activated in\na recurrent neural model of visually grounded speech. The model used in our\nexperiments is trained to project an image and its spoken description in a\ncommon representation space. We show that a recurrent model trained on spoken\nsentences implicitly segments its input into word-like units and reliably maps\nthem to their correct visual referents. We introduce a methodology originating\nfrom linguistics to analyse the representation learned by neural networks --\nthe gating paradigm -- and show that the correct representation of a word is\nonly activated if the network has access to first phoneme of the target word,\nsuggesting that the network does not rely on a global acoustic pattern.\nFurthermore, we find out that not all speech frames (MFCC vectors in our case)\nplay an equal role in the final encoded representation of a given word, but\nthat some frames have a crucial effect on it. Finally, we suggest that word\nrepresentation could be activated through a process of lexical competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08700v1",
            "date": "2019-09-18 20:50:51+00:00",
            "title": "Alleviating Sequence Information Loss with Data Overlapping and Prime Batch Sizes",
            "authors": [
                "No\u00e9mien Kocher",
                "Christian Scuito",
                "Lorenzo Tarantino",
                "Alexandros Lazaridis",
                "Andreas Fischer",
                "Claudiu Musat"
            ],
            "abstract": "In sequence modeling tasks the token order matters, but this information can\nbe partially lost due to the discretization of the sequence into data points.\nIn this paper, we study the imbalance between the way certain token pairs are\nincluded in data points and others are not. We denote this a token order\nimbalance (TOI) and we link the partial sequence information loss to a\ndiminished performance of the system as a whole, both in text and speech\nprocessing tasks. We then provide a mechanism to leverage the full token order\ninformation -Alleviated TOI- by iteratively overlapping the token composition\nof data points. For recurrent networks, we use prime numbers for the batch size\nto avoid redundancies when building batches from overlapped data points. The\nproposed method achieved state of the art performance in both text and speech\nrelated tasks.",
            "journal ref": "Proceedings of the 23rd Conference on Computational Natural\n  Language Learning (CoNLL), Hong Kong, China, 2019, p. 890-899",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08705v1",
            "date": "2019-09-18 21:00:04+00:00",
            "title": "CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots",
            "authors": [
                "Arshit Gupta",
                "Peng Zhang",
                "Garima Lalwani",
                "Mona Diab"
            ],
            "abstract": "Natural Language Understanding (NLU) is a core component of dialog systems.\nIt typically involves two tasks - intent classification (IC) and slot labeling\n(SL), which are then followed by a dialogue management (DM) component. Such NLU\nsystems cater to utterances in isolation, thus pushing the problem of context\nmanagement to DM. However, contextual information is critical to the correct\nprediction of intents and slots in a conversation. Prior work on contextual NLU\nhas been limited in terms of the types of contextual signals used and the\nunderstanding of their impact on the model. In this work, we propose a\ncontext-aware self-attentive NLU (CASA-NLU) model that uses multiple signals,\nsuch as previous intents, slots, dialog acts and utterances over a variable\ncontext window, in addition to the current user utterance. CASA-NLU outperforms\na recurrent contextual NLU baseline on two conversational datasets, yielding a\ngain of up to 7% on the IC task for one of the datasets. Moreover, a\nnon-contextual variant of CASA-NLU achieves state-of-the-art performance for IC\ntask on standard public datasets - Snips and ATIS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08784v3",
            "date": "2019-09-19 02:56:51+00:00",
            "title": "Characterizing Collective Attention via Descriptor Context: A Case Study of Public Discussions of Crisis Events",
            "authors": [
                "Ian Stewart",
                "Diyi Yang",
                "Jacob Eisenstein"
            ],
            "abstract": "Social media datasets make it possible to rapidly quantify collective\nattention to emerging topics and breaking news, such as crisis events.\nCollective attention is typically measured by aggregate counts, such as the\nnumber of posts that mention a name or hashtag. But according to rationalist\nmodels of natural language communication, the collective salience of each\nentity will be expressed not only in how often it is mentioned, but in the form\nthat those mentions take. This is because natural language communication is\npremised on (and customized to) the expectations that speakers and writers have\nabout how their messages will be interpreted by the intended audience. We test\nthis idea by conducting a large-scale analysis of public online discussions of\nbreaking news events on Facebook and Twitter, focusing on five recent crisis\nevents. We examine how people refer to locations, focusing specifically on\ncontextual descriptors, such as \"San Juan\" versus \"San Juan, Puerto Rico.\"\nRationalist accounts of natural language communication predict that such\ndescriptors will be unnecessary (and therefore omitted) when the named entity\nis expected to have high prior salience to the reader. We find that the use of\ncontextual descriptors is indeed associated with proxies for social and\ninformational expectations, including macro-level factors like the location's\nglobal salience and micro-level factors like audience engagement. We also find\na consistent decrease in descriptor context use over the lifespan of each\ncrisis event. These findings provide evidence about how social media users\ncommunicate with their audiences, and point towards more fine-grained models of\ncollective attention that may help researchers and crisis response\norganizations to better understand public perception of unfolding crisis\nevents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08859v1",
            "date": "2019-09-19 08:39:00+00:00",
            "title": "Procedural Reasoning Networks for Understanding Multimodal Procedures",
            "authors": [
                "Mustafa Sercan Amac",
                "Semih Yagcioglu",
                "Aykut Erdem",
                "Erkut Erdem"
            ],
            "abstract": "This paper addresses the problem of comprehending procedural commonsense\nknowledge. This is a challenging task as it requires identifying key entities,\nkeeping track of their state changes, and understanding temporal and causal\nrelations. Contrary to most of the previous work, in this study, we do not rely\non strong inductive bias and explore the question of how multimodality can be\nexploited to provide a complementary semantic signal. Towards this end, we\nintroduce a new entity-aware neural comprehension model augmented with external\nrelational memory units. Our model learns to dynamically update entity states\nin relation to each other while reading the text instructions. Our experimental\nanalysis on the visual reasoning tasks in the recently proposed RecipeQA\ndataset reveals that our approach improves the accuracy of the previously\nreported models by a large margin. Moreover, we find that our model learns\neffective dynamic representations of entities even though we do not use any\nsupervision at the level of entity states.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08905v1",
            "date": "2019-09-19 10:16:21+00:00",
            "title": "A Split-and-Recombine Approach for Follow-up Query Analysis",
            "authors": [
                "Qian Liu",
                "Bei Chen",
                "Haoyan Liu",
                "Lei Fang",
                "Jian-Guang Lou",
                "Bin Zhou",
                "Dongmei Zhang"
            ],
            "abstract": "Context-dependent semantic parsing has proven to be an important yet\nchallenging task. To leverage the advances in context-independent semantic\nparsing, we propose to perform follow-up query analysis, aiming to restate\ncontext-dependent natural language queries with contextual information. To\naccomplish the task, we propose STAR, a novel approach with a well-designed\ntwo-phase process. It is parser-independent and able to handle multifarious\nfollow-up scenarios in different domains. Experiments on the FollowUp dataset\nshow that STAR outperforms the state-of-the-art baseline by a large margin of\nnearly 8%. The superiority on parsing results verifies the feasibility of\nfollow-up query analysis. We also explore the extensibility of STAR on the SQA\ndataset, which is very promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08927v1",
            "date": "2019-09-19 11:54:55+00:00",
            "title": "Extracting Conceptual Knowledge from Natural Language Text Using Maximum Likelihood Principle",
            "authors": [
                "Shipra Sharma",
                "Balwinder Sodhi"
            ],
            "abstract": "Domain-specific knowledge graphs constructed from natural language text are\nubiquitous in today's world. In many such scenarios the base text, from which\nthe knowledge graph is constructed, concerns itself with practical, on-hand,\nactual or ground-reality information about the domain. Product documentation in\nsoftware engineering domain are one example of such base texts. Other examples\ninclude blogs and texts related to digital artifacts, reports on emerging\nmarkets and business models, patient medical records, etc. Though the above\nsources contain a wealth of knowledge about their respective domains, the\nconceptual knowledge on which they are based is often missing or unclear.\nAccess to this conceptual knowledge can enormously increase the utility of\navailable data and assist in several tasks such as knowledge graph completion,\ngrounding, querying, etc.\n  Our contributions in this paper are twofold. First, we propose a novel\nMarkovian stochastic model for document generation from conceptual knowledge.\nThe uniqueness of our approach lies in the fact that the conceptual knowledge\nin the writer's mind forms a component of the parameter set of our stochastic\nmodel. Secondly, we solve the inverse problem of learning the best conceptual\nknowledge from a given document, by finding model parameters which maximize the\nlikelihood of generating the specific document over all possible parameter\nvalues. This likelihood maximization is done using an application of Baum-Welch\nalgorithm, which is a known special case of Expectation-Maximization (EM)\nalgorithm. We run our conceptualization algorithm on several well-known natural\nlanguage sources and obtain very encouraging results. The results of our\nextensive experiments concur with the hypothesis that the information contained\nin these sources has a well-defined and rigorous underlying conceptual\nstructure, which can be discovered using our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09060v3",
            "date": "2019-09-19 15:59:33+00:00",
            "title": "Adaptively Aligned Image Captioning via Adaptive Attention Time",
            "authors": [
                "Lun Huang",
                "Wenmin Wang",
                "Yaxian Xia",
                "Jie Chen"
            ],
            "abstract": "Recent neural models for image captioning usually employ an encoder-decoder\nframework with an attention mechanism. However, the attention mechanism in such\na framework aligns one single (attended) image feature vector to one caption\nword, assuming one-to-one mapping from source image regions and target caption\nwords, which is never possible. In this paper, we propose a novel attention\nmodel, namely Adaptive Attention Time (AAT), to align the source and the target\nadaptively for image captioning. AAT allows the framework to learn how many\nattention steps to take to output a caption word at each decoding step. With\nAAT, an image region can be mapped to an arbitrary number of caption words\nwhile a caption word can also attend to an arbitrary number of image regions.\nAAT is deterministic and differentiable, and doesn't introduce any noise to the\nparameter gradients. In this paper, we empirically show that AAT improves over\nstate-of-the-art methods on the task of image captioning. Code is available at\nhttps://github.com/husthuaan/AAT.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.09220v1",
            "date": "2019-09-19 20:12:10+00:00",
            "title": "Goal-Embedded Dual Hierarchical Model for Task-Oriented Dialogue Generation",
            "authors": [
                "Yi-An Lai",
                "Arshit Gupta",
                "Yi Zhang"
            ],
            "abstract": "Hierarchical neural networks are often used to model inherent structures\nwithin dialogues. For goal-oriented dialogues, these models miss a mechanism\nadhering to the goals and neglect the distinct conversational patterns between\ntwo interlocutors. In this work, we propose Goal-Embedded Dual Hierarchical\nAttentional Encoder-Decoder (G-DuHA) able to center around goals and capture\ninterlocutor-level disparity while modeling goal-oriented dialogues.\nExperiments on dialogue generation, response generation, and human evaluations\ndemonstrate that the proposed model successfully generates higher-quality, more\ndiverse and goal-centric dialogues. Moreover, we apply data augmentation via\ngoal-oriented dialogue generation for task-oriented dialog systems with better\nperformance achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09251v1",
            "date": "2019-09-19 22:35:36+00:00",
            "title": "AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models",
            "authors": [
                "Eric Wallace",
                "Jens Tuyls",
                "Junlin Wang",
                "Sanjay Subramanian",
                "Matt Gardner",
                "Sameer Singh"
            ],
            "abstract": "Neural NLP models are increasingly accurate but are imperfect and\nopaque---they break in counterintuitive ways and leave end users puzzled at\ntheir behavior. Model interpretation methods ameliorate this opacity by\nproviding explanations for specific model predictions. Unfortunately, existing\ninterpretation codebases make it difficult to apply these methods to new models\nand tasks, which hinders adoption for practitioners and burdens\ninterpretability researchers. We introduce AllenNLP Interpret, a flexible\nframework for interpreting NLP models. The toolkit provides interpretation\nprimitives (e.g., input gradients) for any AllenNLP model and task, a suite of\nbuilt-in interpretation methods, and a library of front-end visualization\ncomponents. We demonstrate the toolkit's flexibility and utility by\nimplementing live demos for five interpretation methods (e.g., saliency maps\nand adversarial attacks) on a variety of models and tasks (e.g., masked\nlanguage modeling using BERT and reading comprehension using BiDAF). These\ndemos, alongside our code and tutorials, are available at\nhttps://allennlp.org/interpret .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09270v1",
            "date": "2019-09-20 00:39:07+00:00",
            "title": "Named Entity Recognition with Partially Annotated Training Data",
            "authors": [
                "Stephen Mayhew",
                "Snigdha Chaturvedi",
                "Chen-Tse Tsai",
                "Dan Roth"
            ],
            "abstract": "Supervised machine learning assumes the availability of fully-labeled data,\nbut in many cases, such as low-resource languages, the only data available is\npartially annotated. We study the problem of Named Entity Recognition (NER)\nwith partially annotated training data in which a fraction of the named\nentities are labeled, and all other tokens, entities or otherwise, are labeled\nas non-entity by default. In order to train on this noisy dataset, we need to\ndistinguish between the true and false negatives. To this end, we introduce a\nconstraint-driven iterative algorithm that learns to detect false negatives in\nthe noisy set and downweigh them, resulting in a weighted training set. With\nthis set, we train a weighted NER model. We evaluate our algorithm with\nweighted variants of neural and non-neural NER models on data in 8 languages\nfrom several language and script families, showing strong ability to learn from\npartial data. Finally, to show real-world efficacy, we evaluate on a Bengali\nNER corpus annotated by non-speakers, outperforming the prior state-of-the-art\nby over 5 points F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09389v1",
            "date": "2019-09-20 09:35:20+00:00",
            "title": "Sampling Bias in Deep Active Classification: An Empirical Study",
            "authors": [
                "Ameya Prabhu",
                "Charles Dognin",
                "Maneesh Singh"
            ],
            "abstract": "The exploding cost and time needed for data labeling and model training are\nbottlenecks for training DNN models on large datasets. Identifying smaller\nrepresentative data samples with strategies like active learning can help\nmitigate such bottlenecks. Previous works on active learning in NLP identify\nthe problem of sampling bias in the samples acquired by uncertainty-based\nquerying and develop costly approaches to address it. Using a large empirical\nstudy, we demonstrate that active set selection using the posterior entropy of\ndeep models like FastText.zip (FTZ) is robust to sampling biases and to various\nalgorithmic choices (query size and strategies) unlike that suggested by\ntraditional literature. We also show that FTZ based query strategy produces\nsample sets similar to those from more sophisticated approaches (e.g ensemble\nnetworks). Finally, we show the effectiveness of the selected samples by\ncreating tiny high-quality datasets, and utilizing them for fast and cheap\ntraining of large models. Based on the above, we propose a simple baseline for\ndeep active text classification that outperforms the state-of-the-art. We\nexpect the presented work to be useful and informative for dataset compression\nand for problems involving active, semi-supervised or online learning\nscenarios. Code and models are available at:\nhttps://github.com/drimpossible/Sampling-Bias-Active-Learning",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09428v1",
            "date": "2019-09-20 11:09:52+00:00",
            "title": "A Critical Analysis of Biased Parsers in Unsupervised Parsing",
            "authors": [
                "Chris Dyer",
                "G\u00e1bor Melis",
                "Phil Blunsom"
            ],
            "abstract": "A series of recent papers has used a parsing algorithm due to Shen et al.\n(2018) to recover phrase-structure trees based on proxies for \"syntactic\ndepth.\" These proxy depths are obtained from the representations learned by\nrecurrent language models augmented with mechanisms that encourage the\n(unsupervised) discovery of hierarchical structure latent in natural language\nsentences. Using the same parser, we show that proxies derived from a\nconventional LSTM language model produce trees comparably well to the\nspecialized architectures used in previous work. However, we also provide a\ndetailed analysis of the parsing algorithm, showing (1) that it is\nincomplete---that is, it can recover only a fraction of possible trees---and\n(2) that it has a marked bias for right-branching structures which results in\ninflated performance in right-branching languages like English. Our analysis\nshows that evaluating with biased parsing algorithms can inflate the apparent\nstructural competence of language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09484v1",
            "date": "2019-09-17 15:50:56+00:00",
            "title": "Generative Dialog Policy for Task-oriented Dialog Systems",
            "authors": [
                "Tian Lan",
                "Xianling Mao",
                "Heyan Huang"
            ],
            "abstract": "There is an increasing demand for task-oriented dialogue systems which can\nassist users in various activities such as booking tickets and restaurant\nreservations. In order to complete dialogues effectively, dialogue policy plays\na key role in task-oriented dialogue systems. As far as we know, the existing\ntask-oriented dialogue systems obtain the dialogue policy through\nclassification, which can assign either a dialogue act and its corresponding\nparameters or multiple dialogue acts without their corresponding parameters for\na dialogue action. In fact, a good dialogue policy should construct multiple\ndialogue acts and their corresponding parameters at the same time. However,\nit's hard for existing classification-based methods to achieve this goal. Thus,\nto address the issue above, we propose a novel generative dialogue policy\nlearning method. Specifically, the proposed method uses attention mechanism to\nfind relevant segments of given dialogue context and input utterance and then\nconstructs the dialogue policy by a seq2seq way for task-oriented dialogue\nsystems. Extensive experiments on two benchmark datasets show that the proposed\nmodel significantly outperforms the state-of-the-art baselines. In addition, we\nhave publicly released our codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09524v1",
            "date": "2019-09-20 14:16:27+00:00",
            "title": "Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages",
            "authors": [
                "Yunsu Kim",
                "Petre Petrov",
                "Pavel Petrushkov",
                "Shahram Khadivi",
                "Hermann Ney"
            ],
            "abstract": "We present effective pre-training strategies for neural machine translation\n(NMT) using parallel corpora involving a pivot language, i.e., source-pivot and\npivot-target, leading to a significant improvement in source-target\ntranslation. We propose three methods to increase the relation among source,\npivot, and target languages in the pre-training: 1) step-wise training of a\nsingle model for different language pairs, 2) additional adapter component to\nsmoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder\ntraining via autoencoding of the pivot language. Our methods greatly outperform\nmultilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech\ntasks. We show that our improvements are valid also in zero-shot/zero-resource\nscenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09534v1",
            "date": "2019-09-20 14:40:18+00:00",
            "title": "Creative GANs for generating poems, lyrics, and metaphors",
            "authors": [
                "Asir Saeed",
                "Suzana Ili\u0107",
                "Eva Zangerle"
            ],
            "abstract": "Generative models for text have substantially contributed to tasks like\nmachine translation and language modeling, using maximum likelihood\noptimization (MLE). However, for creative text generation, where multiple\noutputs are possible and originality and uniqueness are encouraged, MLE falls\nshort. Methods optimized for MLE lead to outputs that can be generic,\nrepetitive and incoherent. In this work, we use a Generative Adversarial\nNetwork framework to alleviate this problem. We evaluate our framework on\npoetry, lyrics and metaphor datasets, each with widely different\ncharacteristics, and report better performance of our objective function over\nother generative models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09551v1",
            "date": "2019-09-20 15:08:51+00:00",
            "title": "Natural Language Processing via LDA Topic Model in Recommendation Systems",
            "authors": [
                "Hamed Jelodar",
                "Yongli Wang",
                "Mahdi Rabbani",
                "SeyedValyAllah Ayobi"
            ],
            "abstract": "Today, Internet is one of the widest available media worldwide.\nRecommendation systems are increasingly being used in various applications such\nas movie recommendation, mobile recommendation, article recommendation and etc.\nCollaborative Filtering (CF) and Content-Based (CB) are Well-known techniques\nfor building recommendation systems. Topic modeling based on LDA, is a powerful\ntechnique for semantic mining and perform topic extraction. In the past few\nyears, many articles have been published based on LDA technique for building\nrecommendation systems. In this paper, we present taxonomy of recommendation\nsystems and applications based on LDA. In addition, we utilize LDA and Gibbs\nsampling algorithms to evaluate ISWC and WWW conference publications in\ncomputer science. Our study suggest that the recommendation systems based on\nLDA could be effective in building smart recommendation system in online\ncommunities.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.09696v1",
            "date": "2019-09-13 17:56:58+00:00",
            "title": "A Gated Self-attention Memory Network for Answer Selection",
            "authors": [
                "Tuan Lai",
                "Quan Hung Tran",
                "Trung Bui",
                "Daisuke Kihara"
            ],
            "abstract": "Answer selection is an important research problem, with applications in many\nareas. Previous deep learning based approaches for the task mainly adopt the\nCompare-Aggregate architecture that performs word-level comparison followed by\naggregation. In this work, we take a departure from the popular\nCompare-Aggregate architecture, and instead, propose a new gated self-attention\nmemory network for the task. Combined with a simple transfer learning technique\nfrom a large-scale online corpus, our model outperforms previous methods by a\nlarge margin, achieving new state-of-the-art results on two standard answer\nselection datasets: TrecQA and WikiQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09700v1",
            "date": "2019-09-12 22:35:53+00:00",
            "title": "Retrofitting Contextualized Word Embeddings with Paraphrases",
            "authors": [
                "Weijia Shi",
                "Muhao Chen",
                "Pei Zhou",
                "Kai-Wei Chang"
            ],
            "abstract": "Contextualized word embedding models, such as ELMo, generate meaningful\nrepresentations of words and their context. These models have been shown to\nhave a great impact on downstream applications. However, in many cases, the\ncontextualized embedding of a word changes drastically when the context is\nparaphrased. As a result, the downstream model is not robust to paraphrasing\nand other linguistic variations. To enhance the stability of contextualized\nword embedding models, we propose an approach to retrofitting contextualized\nembedding models with paraphrase contexts. Our method learns an orthogonal\ntransformation on the input space, which seeks to minimize the variance of word\nrepresentations on paraphrased contexts. Experiments show that the retrofitted\nmodel significantly outperforms the original ELMo on various sentence\nclassification and language inference tasks.",
            "journal ref": "EMNLP-IJCNLP2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09814v3",
            "date": "2019-09-21 11:37:23+00:00",
            "title": "Graph Convolutions over Constituent Trees for Syntax-Aware Semantic Role Labeling",
            "authors": [
                "Diego Marcheggiani",
                "Ivan Titov"
            ],
            "abstract": "Semantic role labeling (SRL) is the task of identifying predicates and\nlabeling argument spans with semantic roles. Even though most semantic-role\nformalisms are built upon constituent syntax and only syntactic constituents\ncan be labeled as arguments (e.g., FrameNet and PropBank), all the recent work\non syntax-aware SRL relies on dependency representations of syntax. In\ncontrast, we show how graph convolutional networks (GCNs) can be used to encode\nconstituent structures and inform an SRL system. Nodes in our SpanGCN\ncorrespond to constituents. The computation is done in 3 stages. First, initial\nnode representations are produced by `composing' word representations of the\nfirst and the last word in the constituent. Second, graph convolutions relying\non the constituent tree are performed, yielding syntactically-informed\nconstituent representations. Finally, the constituent representations are\n`decomposed' back into word representations which in turn are used as input to\nthe SRL classifier. We evaluate SpanGCN against alternatives, including a model\nusing GCNs over dependency trees, and show its effectiveness on standard\nCoNLL-2005, CoNLL-2012, and FrameNet benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09962v3",
            "date": "2019-09-22 08:13:28+00:00",
            "title": "Adapting Language Models for Non-Parallel Author-Stylized Rewriting",
            "authors": [
                "Bakhtiyar Syed",
                "Gaurav Verma",
                "Balaji Vasan Srinivasan",
                "Anandhavelu Natarajan",
                "Vasudeva Varma"
            ],
            "abstract": "Given the recent progress in language modeling using Transformer-based neural\nmodels and an active interest in generating stylized text, we present an\napproach to leverage the generalization capabilities of a language model to\nrewrite an input text in a target author's style. Our proposed approach adapts\na pre-trained language model to generate author-stylized text by fine-tuning on\nthe author-specific corpus using a denoising autoencoder (DAE) loss in a\ncascaded encoder-decoder framework. Optimizing over DAE loss allows our model\nto learn the nuances of an author's style without relying on parallel data,\nwhich has been a severe limitation of the previous related works in this space.\nTo evaluate the efficacy of our approach, we propose a linguistically-motivated\nframework to quantify stylistic alignment of the generated text to the target\nauthor at lexical, syntactic and surface levels. The evaluation framework is\nboth interpretable as it leads to several insights about the model, and\nself-contained as it does not rely on external classifiers, e.g. sentiment or\nformality classifiers. Qualitative and quantitative assessment indicates that\nthe proposed approach rewrites the input text with better alignment to the\ntarget style while preserving the original content better than state-of-the-art\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10166v1",
            "date": "2019-09-23 05:29:04+00:00",
            "title": "Automatic Short Answer Grading via Multiway Attention Networks",
            "authors": [
                "Tiaoqiao Liu",
                "Wenbiao Ding",
                "Zhiwei Wang",
                "Jiliang Tang",
                "Gale Yan Huang",
                "Zitao Liu"
            ],
            "abstract": "Automatic short answer grading (ASAG), which autonomously score student\nanswers according to reference answers, provides a cost-effective and\nconsistent approach to teaching professionals and can reduce their monotonous\nand tedious grading workloads. However, ASAG is a very challenging task due to\ntwo reasons: (1) student answers are made up of free text which requires a deep\nsemantic understanding; and (2) the questions are usually open-ended and across\nmany domains in K-12 scenarios. In this paper, we propose a generalized\nend-to-end ASAG learning framework which aims to (1) autonomously extract\nlinguistic information from both student and reference answers; and (2)\naccurately model the semantic relations between free-text student and reference\nanswers in open-ended domain. The proposed ASAG model is evaluated on a large\nreal-world K-12 dataset and can outperform the state-of-the-art baselines in\nterms of various evaluation metrics.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.10309v1",
            "date": "2019-09-10 19:07:15+00:00",
            "title": "An Evalutation of Programming Language Models' performance on Software Defect Detection",
            "authors": [
                "Kailun Wang"
            ],
            "abstract": "This dissertation presents an evaluation of several language models on\nsoftware defect datasets. A language Model (LM) \"can provide word\nrepresentation and probability indication of word sequences as the core\ncomponent of an NLP system.\" Language models for source code are specified for\ntasks in the software engineering field. While some models are directly the NLP\nones, others contain structural information that is uniquely owned by source\ncode. Software defects are defects in the source code that lead to unexpected\nbehaviours and malfunctions at all levels. This study provides an original\nattempt to detect these defects at three different levels (syntactical,\nalgorithmic and general) We also provide a tool chain that researchers can use\nto reproduce the experiments. We have tested the different models against\ndifferent datasets, and performed an analysis over the results. Our original\nattempt to deploy bert, the state-of-the-art model for multitasks, leveled or\noutscored all other models compared.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1909.10411v1",
            "date": "2019-09-23 15:10:41+00:00",
            "title": "NLVR2 Visual Bias Analysis",
            "authors": [
                "Alane Suhr",
                "Yoav Artzi"
            ],
            "abstract": "NLVR2 (Suhr et al., 2019) was designed to be robust for language bias through\na data collection process that resulted in each natural language sentence\nappearing with both true and false labels. The process did not provide a\nsimilar measure of control for visual bias. This technical report analyzes the\npotential for visual bias in NLVR2. We show that some amount of visual bias\nlikely exists. Finally, we identify a subset of the test data that allows to\ntest for model performance in a way that is robust to such potential biases. We\nshow that the performance of existing models (Li et al., 2019; Tan and Bansal\n2019) is relatively robust to this potential bias. We propose to add the\nevaluation on this subset of the data to the NLVR2 evaluation protocol, and\nupdate the official release to include it. A notebook including an\nimplementation of the code used to replicate this analysis is available at\nhttp://nlvr.ai/NLVR2BiasAnalysis.html.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10642v1",
            "date": "2019-09-23 22:20:03+00:00",
            "title": "Data Ordering Patterns for Neural Machine Translation: An Empirical Study",
            "authors": [
                "Siddhant Garg"
            ],
            "abstract": "Recent works show that ordering of the training data affects the model\nperformance for Neural Machine Translation. Several approaches involving\ndynamic data ordering and data sharding based on curriculum learning have been\nanalysed for the their performance gains and faster convergence. In this work\nwe propose to empirically study several ordering approaches for the training\ndata based on different metrics and evaluate their impact on the model\nperformance. Results from our study show that pre-fixing the ordering of the\ntraining data based on perplexity scores from a pre-trained model performs the\nbest and outperforms the default approach of randomly shuffling the training\ndata every epoch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10666v2",
            "date": "2019-09-24 00:45:32+00:00",
            "title": "TripleNet: Triple Attention Network for Multi-Turn Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Wentao Ma",
                "Yiming Cui",
                "Nan Shao",
                "Su He",
                "Wei-Nan Zhang",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "We consider the importance of different utterances in the context for\nselecting the response usually depends on the current query. In this paper, we\npropose the model TripleNet to fully model the task with the triple <context,\nquery, response> instead of <context, response> in previous works. The heart of\nTripleNet is a novel attention mechanism named triple attention to model the\nrelationships within the triple at four levels. The new mechanism updates the\nrepresentation for each element based on the attention with the other two\nconcurrently and symmetrically. We match the triple <C, Q, R> centered on the\nresponse from char to context level for prediction. Experimental results on two\nlarge-scale multi-turn response selection datasets show that the proposed model\ncan significantly outperform the state-of-the-art methods. TripleNet source\ncode is available at https://github.com/wtma/TripleNet",
            "journal ref": "CoNLL 2019 737-746",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10911v1",
            "date": "2019-09-24 13:37:02+00:00",
            "title": "Layerwise Relevance Visualization in Convolutional Text Graph Classifiers",
            "authors": [
                "Robert Schwarzenberg",
                "Marc H\u00fcbner",
                "David Harbecke",
                "Christoph Alt",
                "Leonhard Hennig"
            ],
            "abstract": "Representations in the hidden layers of Deep Neural Networks (DNN) are often\nhard to interpret since it is difficult to project them into an interpretable\ndomain. Graph Convolutional Networks (GCN) allow this projection, but existing\nexplainability methods do not exploit this fact, i.e. do not focus their\nexplanations on intermediate states. In this work, we present a novel method\nthat traces and visualizes features that contribute to a classification\ndecision in the visible and hidden layers of a GCN. Our method exposes hidden\ncross-layer dynamics in the input graph structure. We experimentally\ndemonstrate that it yields meaningful layerwise explanations for a GCN sentence\nclassifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11189v1",
            "date": "2019-09-24 21:19:01+00:00",
            "title": "Diachronic Topics in New High German Poetry",
            "authors": [
                "Thomas N. Haider"
            ],
            "abstract": "Statistical topic models are increasingly and popularly used by Digital\nHumanities scholars to perform distant reading tasks on literary data. It\nallows us to estimate what people talk about. Especially Latent Dirichlet\nAllocation (LDA) has shown its usefulness, as it is unsupervised, robust, easy\nto use, scalable, and it offers interpretable results. In a preliminary study,\nwe apply LDA to a corpus of New High German poetry (textgrid, with 51k poems,\n8m token), and use the distribution of topics over documents for a\nclassification of poems into time periods and for authorship attribution.",
            "journal ref": "In Proceedings of the International Digital Humanities Conference\n  DH2019, Utrecht, Link: https://dev.clariah.nl/files/dh2019/boa/1031.html",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11218v1",
            "date": "2019-09-24 22:58:44+00:00",
            "title": "Attention Interpretability Across NLP Tasks",
            "authors": [
                "Shikhar Vashishth",
                "Shyam Upadhyay",
                "Gaurav Singh Tomar",
                "Manaal Faruqui"
            ],
            "abstract": "The attention layer in a neural network model provides insights into the\nmodel's reasoning behind its prediction, which are usually criticized for being\nopaque. Recently, seemingly contradictory viewpoints have emerged about the\ninterpretability of attention weights (Jain & Wallace, 2019; Vig & Belinkov,\n2019). Amid such confusion arises the need to understand attention mechanism\nmore systematically. In this work, we attempt to fill this gap by giving a\ncomprehensive explanation which justifies both kinds of observations (i.e.,\nwhen is attention interpretable and when it is not). Through a series of\nexperiments on diverse NLP tasks, we validate our observations and reinforce\nour claim of interpretability of attention through manual evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11241v1",
            "date": "2019-09-25 00:28:50+00:00",
            "title": "Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysis",
            "authors": [
                "Franco M. Luque"
            ],
            "abstract": "In this article we describe our participation in TASS 2019, a shared task\naimed at the detection of sentiment polarity of Spanish tweets. We combined\ndifferent representations such as bag-of-words, bag-of-characters, and tweet\nembeddings. In particular, we trained robust subword-aware word embeddings and\ncomputed tweet representations using a weighted-averaging strategy. We also\nused two data augmentation techniques to deal with data scarcity: two-way\ntranslation augmentation, and instance crossover augmentation, a novel\ntechnique that generates new instances by combining halves of tweets. In\nexperiments, we trained linear classifiers and ensemble models, obtaining\nhighly competitive results despite the simplicity of our approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11272v1",
            "date": "2019-09-25 03:39:00+00:00",
            "title": "TalkDown: A Corpus for Condescension Detection in Context",
            "authors": [
                "Zijian Wang",
                "Christopher Potts"
            ],
            "abstract": "Condescending language use is caustic; it can bring dialogues to an end and\nbifurcate communities. Thus, systems for condescension detection could have a\nlarge positive impact. A challenge here is that condescension is often\nimpossible to detect from isolated utterances, as it depends on the discourse\nand social context. To address this, we present TalkDown, a new labeled dataset\nof condescending linguistic acts in context. We show that extending a\nlanguage-only model with representations of the discourse improves performance,\nand we motivate techniques for dealing with the low rates of condescension\noverall. We also use our model to estimate condescension rates in various\nonline communities and relate these differences to differing community norms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11287v1",
            "date": "2019-09-25 04:40:27+00:00",
            "title": "Task-Oriented Conversation Generation Using Heterogeneous Memory Networks",
            "authors": [
                "Zehao Lin",
                "Xinjing Huang",
                "Feng Ji",
                "Haiqing Chen",
                "Ying Zhang"
            ],
            "abstract": "How to incorporate external knowledge into a neural dialogue model is\ncritically important for dialogue systems to behave like real humans. To handle\nthis problem, memory networks are usually a great choice and a promising way.\nHowever, existing memory networks do not perform well when leveraging\nheterogeneous information from different sources. In this paper, we propose a\nnovel and versatile external memory networks called Heterogeneous Memory\nNetworks (HMNs), to simultaneously utilize user utterances, dialogue history\nand background knowledge tuples. In our method, historical sequential dialogues\nare encoded and stored into the context-aware memory enhanced by gating\nmechanism while grounding knowledge tuples are encoded and stored into the\ncontext-free memory. During decoding, the decoder augmented with HMNs\nrecurrently selects each word in one response utterance from these two memories\nand a general vocabulary. Experimental results on multiple real-world datasets\nshow that HMNs significantly outperform the state-of-the-art data-driven\ntask-oriented dialogue models in most domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11386v4",
            "date": "2019-09-25 10:26:36+00:00",
            "title": "Multi-Dimensional Explanation of Target Variables from Documents",
            "authors": [
                "Diego Antognini",
                "Claudiu Musat",
                "Boi Faltings"
            ],
            "abstract": "Automated predictions require explanations to be interpretable by humans.\nPast work used attention and rationale mechanisms to find words that predict\nthe target variable of a document. Often though, they result in a tradeoff\nbetween noisy explanations or a drop in accuracy. Furthermore, rationale\nmethods cannot capture the multi-faceted nature of justifications for multiple\ntargets, because of the non-probabilistic nature of the mask. In this paper, we\npropose the Multi-Target Masker (MTM) to address these shortcomings. The\nnovelty lies in the soft multi-dimensional mask that models a relevance\nprobability distribution over the set of target variables to handle\nambiguities. Additionally, two regularizers guide MTM to induce long,\nmeaningful explanations. We evaluate MTM on two datasets and show, using\nstandard metrics and human annotations, that the resulting masks are more\naccurate and coherent than those generated by the state-of-the-art methods.\nMoreover, MTM is the first to also achieve the highest F1 scores for all the\ntarget variables simultaneously.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11430v3",
            "date": "2019-09-25 12:13:43+00:00",
            "title": "Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Training",
            "authors": [
                "Qiao Cheng",
                "Meiyuan Fang",
                "Yaqian Han",
                "Jin Huang",
                "Yitao Duan"
            ],
            "abstract": "In a pipeline speech translation system, automatic speech recognition (ASR)\nsystem will transmit errors in recognition to the downstream machine\ntranslation (MT) system. A standard machine translation system is usually\ntrained on parallel corpus composed of clean text and will perform poorly on\ntext with recognition noise, a gap well known in speech translation community.\nIn this paper, we propose a training architecture which aims at making a neural\nmachine translation model more robust against speech recognition errors. Our\napproach addresses the encoder and the decoder simultaneously using adversarial\nlearning and data augmentation, respectively. Experimental results on IWSLT2018\nspeech translation task show that our approach can bridge the gap between the\nASR output and the MT input, outperforms the baseline by up to 2.83 BLEU on\nnoisy ASR output, while maintaining close performance on clean text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11764v5",
            "date": "2019-09-25 20:50:32+00:00",
            "title": "FreeLB: Enhanced Adversarial Training for Natural Language Understanding",
            "authors": [
                "Chen Zhu",
                "Yu Cheng",
                "Zhe Gan",
                "Siqi Sun",
                "Tom Goldstein",
                "Jingjing Liu"
            ],
            "abstract": "Adversarial training, which minimizes the maximal risk for label-preserving\ninput perturbations, has proved to be effective for improving the\ngeneralization of language models. In this work, we propose a novel adversarial\ntraining algorithm, FreeLB, that promotes higher invariance in the embedding\nspace, by adding adversarial perturbations to word embeddings and minimizing\nthe resultant adversarial risk inside different regions around input samples.\nTo validate the effectiveness of the proposed approach, we apply it to\nTransformer-based models for natural language understanding and commonsense\nreasoning tasks. Experiments on the GLUE benchmark show that when applied only\nto the finetuning stage, it is able to improve the overall test scores of\nBERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8.\nIn addition, the proposed approach achieves state-of-the-art single-model test\naccuracies of 85.44\\% and 67.75\\% on ARC-Easy and ARC-Challenge. Experiments on\nCommonsenseQA benchmark further demonstrate that FreeLB can be generalized and\nboost the performance of RoBERTa-large model on other tasks as well. Code is\navailable at \\url{https://github.com/zhuchen03/FreeLB .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11824v1",
            "date": "2019-09-26 00:14:22+00:00",
            "title": "Pre-train, Interact, Fine-tune: A Novel Interaction Representation for Text Classification",
            "authors": [
                "Jianming Zheng",
                "Fei Cai",
                "Honghui Chen",
                "Maarten de Rijke"
            ],
            "abstract": "Text representation can aid machines in understanding text. Previous work on\ntext representation often focuses on the so-called forward implication, i.e.,\npreceding words are taken as the context of later words for creating\nrepresentations, thus ignoring the fact that the semantics of a text segment is\na product of the mutual implication of words in the text: later words\ncontribute to the meaning of preceding words. We introduce the concept of\ninteraction and propose a two-perspective interaction representation, that\nencapsulates a local and a global interaction representation. Here, a local\ninteraction representation is one that interacts among words with\nparent-children relationships on the syntactic trees and a global interaction\ninterpretation is one that interacts among all the words in a sentence. We\ncombine the two interaction representations to develop a Hybrid Interaction\nRepresentation (HIR).\n  Inspired by existing feature-based and fine-tuning-based pretrain-finetuning\napproaches to language models, we integrate the advantages of feature-based and\nfine-tuning-based methods to propose the Pre-train, Interact, Fine-tune (PIF)\narchitecture.\n  We evaluate our proposed models on five widely-used datasets for text\nclassification tasks. Our ensemble method, outperforms state-of-the-art\nbaselines with improvements ranging from 2.03% to 3.15% in terms of error rate.\nIn addition, we find that, the improvements of PIF against most\nstate-of-the-art methods is not affected by increasing of the length of the\ntext.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.11942v6",
            "date": "2019-09-26 07:06:13+00:00",
            "title": "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations",
            "authors": [
                "Zhenzhong Lan",
                "Mingda Chen",
                "Sebastian Goodman",
                "Kevin Gimpel",
                "Piyush Sharma",
                "Radu Soricut"
            ],
            "abstract": "Increasing model size when pretraining natural language representations often\nresults in improved performance on downstream tasks. However, at some point\nfurther model increases become harder due to GPU/TPU memory limitations and\nlonger training times. To address these problems, we present two\nparameter-reduction techniques to lower memory consumption and increase the\ntraining speed of BERT. Comprehensive empirical evidence shows that our\nproposed methods lead to models that scale much better compared to the original\nBERT. We also use a self-supervised loss that focuses on modeling\ninter-sentence coherence, and show it consistently helps downstream tasks with\nmulti-sentence inputs. As a result, our best model establishes new\nstate-of-the-art results on the GLUE, RACE, and \\squad benchmarks while having\nfewer parameters compared to BERT-large. The code and the pretrained models are\navailable at https://github.com/google-research/ALBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11968v3",
            "date": "2019-09-26 08:19:34+00:00",
            "title": "Low-Resource Response Generation with Template Prior",
            "authors": [
                "Ze Yang",
                "Wei Wu",
                "Jian Yang",
                "Can Xu",
                "Zhoujun Li"
            ],
            "abstract": "We study open domain response generation with limited message-response pairs.\nThe problem exists in real-world applications but is less explored by the\nexisting work. Since the paired data now is no longer enough to train a neural\ngeneration model, we consider leveraging the large scale of unpaired data that\nare much easier to obtain, and propose response generation with both paired and\nunpaired data. The generation model is defined by an encoder-decoder\narchitecture with templates as prior, where the templates are estimated from\nthe unpaired data as a neural hidden semi-markov model. By this means, response\ngeneration learned from the small paired data can be aided by the semantic and\nsyntactic knowledge in the large unpaired data. To balance the effect of the\nprior and the input message to response generation, we propose learning the\nwhole generation model with an adversarial approach. Empirical studies on\nquestion response generation and sentiment response generation indicate that\nwhen only a few pairs are available, our model can significantly outperform\nseveral state-of-the-art response generation models in terms of both automatic\nand human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12208v1",
            "date": "2019-09-26 15:59:37+00:00",
            "title": "An Investigation into the Effectiveness of Enhancement in ASR Training and Test for CHiME-5 Dinner Party Transcription",
            "authors": [
                "Catalin Zorila",
                "Christoph Boeddeker",
                "Rama Doddipatla",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "Despite the strong modeling power of neural network acoustic models, speech\nenhancement has been shown to deliver additional word error rate improvements\nif multi-channel data is available. However, there has been a longstanding\ndebate whether enhancement should also be carried out on the ASR training data.\nIn an extensive experimental evaluation on the acoustically very challenging\nCHiME-5 dinner party data we show that: (i) cleaning up the training data can\nlead to substantial error rate reductions, and (ii) enhancement in training is\nadvisable as long as enhancement in test is at least as strong as in training.\nThis approach stands in contrast and delivers larger gains than the common\nstrategy reported in the literature to augment the training database with\nadditional artificially degraded speech. Together with an acoustic model\ntopology consisting of initial CNN layers followed by factorized TDNN layers we\nachieve with 41.6% and 43.2% WER on the DEV and EVAL test sets, respectively, a\nnew single-system state-of-the-art result on the CHiME-5 data. This is a 8%\nrelative improvement compared to the best word error rate published so far for\na speech recognizer without system combination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12335v1",
            "date": "2019-09-26 18:59:53+00:00",
            "title": "Rethinking Text Attribute Transfer: A Lexical Analysis",
            "authors": [
                "Yao Fu",
                "Hao Zhou",
                "Jiaze Chen",
                "Lei Li"
            ],
            "abstract": "Text attribute transfer is modifying certain linguistic attributes (e.g.\nsentiment, style, authorship, etc.) of a sentence and transforming them from\none type to another. In this paper, we aim to analyze and interpret what is\nchanged during the transfer process. We start from the observation that in many\nexisting models and datasets, certain words within a sentence play important\nroles in determining the sentence attribute class. These words are referred to\nas \\textit{the Pivot Words}. Based on these pivot words, we propose a lexical\nanalysis framework, \\textit{the Pivot Analysis}, to quantitatively analyze the\neffects of these words in text attribute classification and transfer. We apply\nthis framework to existing datasets and models and show that: (1) the pivot\nwords are strong features for the classification of sentence attributes; (2) to\nchange the attribute of a sentence, many datasets only requires to change\ncertain pivot words; (3) consequently, many transfer models only perform the\nlexical-level modification, while leaving higher-level sentence structures\nunchanged. Our work provides an in-depth understanding of linguistic attribute\ntransfer and further identifies the future requirements and challenges of this\ntask\\footnote{Our code can be found at\nhttps://github.com/FranxYao/pivot_analysis}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12339v1",
            "date": "2019-09-26 19:03:54+00:00",
            "title": "Coin_flipper at eHealth-KD Challenge 2019: Voting LSTMs for Key Phrases and Semantic Relation Identification Applied to Spanish eHealth Texts",
            "authors": [
                "Neus Catal\u00e0",
                "Mario Martin"
            ],
            "abstract": "This paper describes our approach presented for the eHealth-KD 2019\nchallenge. Our participation was aimed at testing how far we could go using\ngeneric tools for Text-Processing but, at the same time, using common\noptimization techniques in the field of Data Mining. The architecture proposed\nfor both tasks of the challenge is a standard stacked 2-layer bi-LSTM. The main\nparticularities of our approach are: (a) The use of a surrogate function of F1\nas loss function to close the gap between the minimization function and the\nevaluation metric, and (b) The generation of an ensemble of models for\ngenerating predictions by majority vote. Our system ranked second with an F1\nscore of 62.18% in the main task by a narrow margin with the winner that scored\n63.94%.",
            "journal ref": "Proceedings of the Iberian Languages Evaluation Forum (IberLEF\n  2019) co-located with 35th Conference of the Spanish Society for Natural\n  Language Processing (SEPLN 2019) http://ceur-ws.org/Vol-2421/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12415v1",
            "date": "2019-09-26 22:09:09+00:00",
            "title": "Improving RNN Transducer Modeling for End-to-End Speech Recognition",
            "authors": [
                "Jinyu Li",
                "Rui Zhao",
                "Hu Hu",
                "Yifan Gong"
            ],
            "abstract": "In the last few years, an emerging trend in automatic speech recognition\nresearch is the study of end-to-end (E2E) systems. Connectionist Temporal\nClassification (CTC), Attention Encoder-Decoder (AED), and RNN Transducer\n(RNN-T) are the most popular three methods. Among these three methods, RNN-T\nhas the advantages to do online streaming which is challenging to AED and it\ndoesn't have CTC's frame-independence assumption. In this paper, we improve the\nRNN-T training in two aspects. First, we optimize the training algorithm of\nRNN-T to reduce the memory consumption so that we can have larger training\nminibatch for faster training speed. Second, we propose better model structures\nso that we obtain RNN-T models with the very good accuracy but small footprint.\nTrained with 30 thousand hours anonymized and transcribed Microsoft production\ndata, the best RNN-T model with even smaller model size (216 Megabytes)\nachieves up-to 11.8% relative word error rate (WER) reduction from the baseline\nRNN-T model. This best RNN-T model is significantly better than the device\nhybrid model with similar size by achieving up-to 15.0% relative WER reduction,\nand obtains similar WERs as the server hybrid model of 5120 Megabytes in size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12642v1",
            "date": "2019-09-27 12:15:58+00:00",
            "title": "HateMonitors: Language Agnostic Abuse Detection in Social Media",
            "authors": [
                "Punyajoy Saha",
                "Binny Mathew",
                "Pawan Goyal",
                "Animesh Mukherjee"
            ],
            "abstract": "Reducing hateful and offensive content in online social media pose a dual\nproblem for the moderators. On the one hand, rigid censorship on social media\ncannot be imposed. On the other, the free flow of such content cannot be\nallowed. Hence, we require efficient abusive language detection system to\ndetect such harmful content in social media. In this paper, we present our\nmachine learning model, HateMonitor, developed for Hate Speech and Offensive\nContent Identification in Indo-European Languages (HASOC), a shared task at\nFIRE 2019. We have used a Gradient Boosting model, along with BERT and LASER\nembeddings, to make the system language agnostic. Our model came at First\nposition for the German sub-task A. We have also made our model public at\nhttps://github.com/punyajoy/HateMonitors-HASOC .",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1909.12681v2",
            "date": "2019-09-27 13:38:20+00:00",
            "title": "End-to-End Code-Switching ASR for Low-Resourced Language Pairs",
            "authors": [
                "Xianghu Yue",
                "Grandee Lee",
                "Emre Y\u0131lmaz",
                "Fang Deng",
                "Haizhou Li"
            ],
            "abstract": "Despite the significant progress in end-to-end (E2E) automatic speech\nrecognition (ASR), E2E ASR for low resourced code-switching (CS) speech has not\nbeen well studied. In this work, we describe an E2E ASR pipeline for the\nrecognition of CS speech in which a low-resourced language is mixed with a high\nresourced language. Low-resourcedness in acoustic data hinders the performance\nof E2E ASR systems more severely than the conventional ASR systems.~To mitigate\nthis problem in the transcription of archives with code-switching Frisian-Dutch\nspeech, we integrate a designated decoding scheme and perform rescoring with\nneural network-based language models to enable better utilization of the\navailable textual resources. We first incorporate a multi-graph decoding\napproach which creates parallel search spaces for each monolingual and mixed\nrecognition tasks to maximize the utilization of the textual resources from\neach language. Further, language model rescoring is performed using a recurrent\nneural network pre-trained with cross-lingual embedding and further adapted\nwith the limited amount of in-domain CS text. The ASR experiments demonstrate\nthe effectiveness of the described techniques in improving the recognition\nperformance of an E2E CS ASR system in a low-resourced scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12744v1",
            "date": "2019-09-27 15:23:17+00:00",
            "title": "On the use of BERT for Neural Machine Translation",
            "authors": [
                "St\u00e9phane Clinchant",
                "Kweon Woo Jung",
                "Vassilina Nikoulina"
            ],
            "abstract": "Exploiting large pretrained models for various NMT tasks have gained a lot of\nvisibility recently. In this work we study how BERT pretrained models could be\nexploited for supervised Neural Machine Translation. We compare various ways to\nintegrate pretrained BERT model with NMT model and study the impact of the\nmonolingual data used for BERT training on the final translation quality. We\nuse WMT-14 English-German, IWSLT15 English-German and IWSLT14 English-Russian\ndatasets for these experiments. In addition to standard task test set\nevaluation, we perform evaluation on out-of-domain test sets and noise injected\ntest sets, in order to assess how BERT pretrained representations affect model\nrobustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13154v1",
            "date": "2019-09-28 21:32:55+00:00",
            "title": "Generalized Zero-shot ICD Coding",
            "authors": [
                "Congzheng Song",
                "Shanghang Zhang",
                "Najmeh Sadoughi",
                "Pengtao Xie",
                "Eric Xing"
            ],
            "abstract": "The International Classification of Diseases (ICD) is a list of\nclassification codes for the diagnoses. Automatic ICD coding is in high demand\nas the manual coding can be labor-intensive and error-prone. It is a\nmulti-label text classification task with extremely long-tailed label\ndistribution, making it difficult to perform fine-grained classification on\nboth frequent and zero-shot codes at the same time. In this paper, we propose a\nlatent feature generation framework for generalized zero-shot ICD coding, where\nwe aim to improve the prediction on codes that have no labeled data without\ncompromising the performance on seen codes. Our framework generates pseudo\nfeatures conditioned on the ICD code descriptions and exploits the ICD code\nhierarchical structure. To guarantee the semantic consistency between the\ngenerated features and real features, we reconstruct the keywords in the input\ndocuments that are related to the conditioned ICD codes. To the best of our\nknowledge, this works represents the first one that proposes an adversarial\ngenerative model for the generalized zero-shot learning on multi-label text\nclassification. Extensive experiments demonstrate the effectiveness of our\napproach. On the public MIMIC-III dataset, our methods improve the F1 score\nfrom nearly 0 to 20.91% for the zero-shot codes, and increase the AUC score by\n3% (absolute improvement) from previous state of the art. We also show that the\nframework improves the performance on few-shot codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13162v1",
            "date": "2019-09-28 22:11:03+00:00",
            "title": "Translation, Sentiment and Voices: A Computational Model to Translate and Analyse Voices from Real-Time Video Calling",
            "authors": [
                "Aneek Barman Roy"
            ],
            "abstract": "With internet quickly becoming an easy access to many, voice calling over\ninternet is slowly gaining momentum. Individuals has been engaging in video\ncommunication across the world in different languages. The decade saw the\nemergence of language translation using neural networks as well. With more data\nbeing generated in audio and visual forms, there has become a need and a\nchallenge to analyse such information for many researchers from academia and\nindustry. The availability of video chat corpora is limited as organizations\nprotect user privacy and ensure data security. For this reason, an audio-visual\ncommunication system (VidALL) has been developed and audio-speeches were\nextracted. To understand human nature while answering a video call, an analysis\nwas conducted where polarity and vocal intensity were considered as parameters.\nSimultaneously, a translation model using a neural approach was developed to\ntranslate English sentences to French. Simple RNN-based and Embedded-RNN based\nmodels were used for the translation model. BLEU score and target sentence\ncomparators were used to check sentence correctness. Embedded-RNN showed an\naccuracy of 88.71 percentage and predicted correct sentences. A key finding\nsuggest that polarity is a good estimator to understand human emotion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13184v1",
            "date": "2019-09-29 01:58:15+00:00",
            "title": "Towards Automatic Bot Detection in Twitter for Health-related Tasks",
            "authors": [
                "Anahita Davoudi",
                "Ari Z. Klein",
                "Abeed Sarker",
                "Graciela Gonzalez-Hernandez"
            ],
            "abstract": "With the increasing use of social media data for health-related research, the\ncredibility of the information from this source has been questioned as the\nposts may originate from automated accounts or \"bots\". While automatic bot\ndetection approaches have been proposed, there are none that have been\nevaluated on users posting health-related information. In this paper, we extend\nan existing bot detection system and customize it for health-related research.\nUsing a dataset of Twitter users, we first show that the system, which was\ndesigned for political bot detection, underperforms when applied to\nhealth-related Twitter users. We then incorporate additional features and a\nstatistical machine learning classifier to significantly improve bot detection\nperformance. Our approach obtains F_1 scores of 0.7 for the \"bot\" class,\nrepresenting improvements of 0.339. Our approach is customizable and\ngeneralizable for bot detection in other health-related social media cohorts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13332v1",
            "date": "2019-09-29 18:01:00+00:00",
            "title": "Recent Advances in End-to-End Spoken Language Understanding",
            "authors": [
                "Natalia Tomashenko",
                "Antoine Caubriere",
                "Yannick Esteve",
                "Antoine Laurent",
                "Emmanuel Morin"
            ],
            "abstract": "This work investigates spoken language understanding (SLU) systems in the\nscenario when the semantic information is extracted directly from the speech\nsignal by means of a single end-to-end neural network model. Two SLU tasks are\nconsidered: named entity recognition (NER) and semantic slot filling (SF). For\nthese tasks, in order to improve the model performance, we explore various\ntechniques including speaker adaptation, a modification of the connectionist\ntemporal classification (CTC) training criterion, and sequential pretraining.",
            "journal ref": "Statistical Language and Speech Processing. SLSP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13434v2",
            "date": "2019-09-30 02:40:48+00:00",
            "title": "Generating Diverse Story Continuations with Controllable Semantics",
            "authors": [
                "Lifu Tu",
                "Xiaoan Ding",
                "Dong Yu",
                "Kevin Gimpel"
            ],
            "abstract": "We propose a simple and effective modeling framework for controlled\ngeneration of multiple, diverse outputs. We focus on the setting of generating\nthe next sentence of a story given its context. As controllable dimensions, we\nconsider several sentence attributes, including sentiment, length, predicates,\nframes, and automatically-induced clusters. Our empirical results demonstrate:\n(1) our framework is accurate in terms of generating outputs that match the\ntarget control values; (2) our model yields increased maximum metric scores\ncompared to standard n-best list generation via beam search; (3) controlling\ngeneration with semantic frames leads to a stronger combination of diversity\nand quality than other control variables as measured by automatic metrics. We\nalso conduct a human evaluation to assess the utility of providing multiple\nsuggestions for creative writing, demonstrating promising results for the\npotential of controllable, diverse generation in a collaborative writing\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13568v1",
            "date": "2019-09-30 10:29:45+00:00",
            "title": "A Hybrid Persian Sentiment Analysis Framework: Integrating Dependency Grammar Based Rules and Deep Neural Networks",
            "authors": [
                "Kia Dashtipour",
                "Mandar Gogate",
                "Jingpeng Li",
                "Fengling Jiang",
                "Bin Kong",
                "Amir Hussain"
            ],
            "abstract": "Social media hold valuable, vast and unstructured information on public\nopinion that can be utilized to improve products and services. The automatic\nanalysis of such data, however, requires a deep understanding of natural\nlanguage. Current sentiment analysis approaches are mainly based on word\nco-occurrence frequencies, which are inadequate in most practical cases. In\nthis work, we propose a novel hybrid framework for concept-level sentiment\nanalysis in Persian language, that integrates linguistic rules and deep\nlearning to optimize polarity detection. When a pattern is triggered, the\nframework allows sentiments to flow from words to concepts based on symbolic\ndependency relations. When no pattern is triggered, the framework switches to\nits subsymbolic counterpart and leverages deep neural networks (DNN) to perform\nthe classification. The proposed framework outperforms state-of-the-art\napproaches (including support vector machine, and logistic regression) and DNN\nclassifiers (long short-term memory, and Convolutional Neural Networks) with a\nmargin of 10-15% and 3-4% respectively, using benchmark Persian product and\nhotel reviews corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13668v1",
            "date": "2019-09-30 13:05:55+00:00",
            "title": "On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation",
            "authors": [
                "Victor Prokhorov",
                "Ehsan Shareghi",
                "Yingzhen Li",
                "Mohammad Taher Pilehvar",
                "Nigel Collier"
            ],
            "abstract": "Variational Autoencoders (VAEs) are known to suffer from learning\nuninformative latent representation of the input due to issues such as\napproximated posterior collapse, or entanglement of the latent space. We impose\nan explicit constraint on the Kullback-Leibler (KL) divergence term inside the\nVAE objective function. While the explicit constraint naturally avoids\nposterior collapse, we use it to further understand the significance of the KL\nterm in controlling the information transmitted through the VAE channel. Within\nthis framework, we explore different properties of the estimated posterior\ndistribution, and highlight the trade-off between the amount of information\nencoded in a latent code during training, and the generative capacity of the\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13717v1",
            "date": "2019-09-30 14:02:53+00:00",
            "title": "Retrieval-based Goal-Oriented Dialogue Generation",
            "authors": [
                "Ana Valeria Gonzalez",
                "Isabelle Augenstein",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Most research on dialogue has focused either on dialogue generation for\nopenended chit chat or on state tracking for goal-directed dialogue. In this\nwork, we explore a hybrid approach to goal-oriented dialogue generation that\ncombines retrieval from past history with a hierarchical, neural\nencoder-decoder architecture. We evaluate this approach in the customer support\ndomain using the Multiwoz dataset (Budzianowski et al., 2018). We show that\nadding this retrieval step to a hierarchical, neural encoder-decoder\narchitecture leads to significant improvements, including responses that are\nrated more appropriate and fluent by human evaluators. Finally, we compare our\nretrieval-based model to various semantically conditioned models explicitly\nusing past dialog act information, and find that our proposed model is\ncompetitive with the current state of the art (Chen et al., 2019), while not\nrequiring explicit labels about past machine acts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13827v1",
            "date": "2019-09-30 16:40:15+00:00",
            "title": "Towards Diverse Paraphrase Generation Using Multi-Class Wasserstein GAN",
            "authors": [
                "Zhecheng An",
                "Sicong Liu"
            ],
            "abstract": "Paraphrase generation is an important and challenging natural language\nprocessing (NLP) task. In this work, we propose a deep generative model to\ngenerate paraphrase with diversity. Our model is based on an encoder-decoder\narchitecture. An additional transcoder is used to convert a sentence into its\nparaphrasing latent code. The transcoder takes an explicit pattern embedding\nvariable as condition, so diverse paraphrase can be generated by sampling on\nthe pattern embedding variable. We use a Wasserstein GAN to align the\ndistributions of the real and generated paraphrase samples. We propose a\nmulti-class extension to the Wasserstein GAN, which allows our generative model\nto learn from both positive and negative samples. The generated paraphrase\ndistribution is forced to get closer to the positive real distribution, and be\npushed away from the negative distribution in Wasserstein distance. We test our\nmodel in two datasets with both automatic metrics and human evaluation. Results\nshow that our model can generate fluent and reliable paraphrase samples that\noutperform the state-of-art results, while also provides reasonable variability\nand diversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00058v1",
            "date": "2019-09-30 18:58:03+00:00",
            "title": "Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations",
            "authors": [
                "Po-Yao Huang",
                "Xiaojun Chang",
                "Alexander Hauptmann"
            ],
            "abstract": "With the aim of promoting and understanding the multilingual version of image\nsearch, we leverage visual object detection and propose a model with diverse\nmulti-head attention to learn grounded multilingual multimodal representations.\nSpecifically, our model attends to different types of textual semantics in two\nlanguages and visual objects for fine-grained alignments between sentences and\nimages. We introduce a new objective function which explicitly encourages\nattention diversity to learn an improved visual-semantic embedding space. We\nevaluate our model in the German-Image and English-Image matching tasks on the\nMulti30K dataset, and in the Semantic Textual Similarity task with the English\ndescriptions of visual content. Results show that our model yields a\nsignificant performance gain over other methods in all of the three tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00163v1",
            "date": "2019-10-01 00:47:31+00:00",
            "title": "Specializing Word Embeddings (for Parsing) by Information Bottleneck",
            "authors": [
                "Xiang Lisa Li",
                "Jason Eisner"
            ],
            "abstract": "Pre-trained word embeddings like ELMo and BERT contain rich syntactic and\nsemantic information, resulting in state-of-the-art performance on various\ntasks. We propose a very fast variational information bottleneck (VIB) method\nto nonlinearly compress these embeddings, keeping only the information that\nhelps a discriminative parser. We compress each word embedding to either a\ndiscrete tag or a continuous vector. In the discrete version, our automatically\ncompressed tags form an alternative tag set: we show experimentally that our\ntags capture most of the information in traditional POS tag annotations, but\nour tag sequences can be parsed more accurately at the same level of tag\ngranularity. In the continuous version, we show experimentally that moderately\ncompressing the word embeddings by our method yields a more accurate parser in\n8 of 9 languages, unlike simple dimensionality reduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00192v1",
            "date": "2019-10-01 04:07:15+00:00",
            "title": "Writing habits and telltale neighbors: analyzing clinical concept usage patterns with sublanguage embeddings",
            "authors": [
                "Denis Newman-Griffis",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Natural language processing techniques are being applied to increasingly\ndiverse types of electronic health records, and can benefit from in-depth\nunderstanding of the distinguishing characteristics of medical document types.\nWe present a method for characterizing the usage patterns of clinical concepts\namong different document types, in order to capture semantic differences beyond\nthe lexical level. By training concept embeddings on clinical documents of\ndifferent types and measuring the differences in their nearest neighborhood\nstructures, we are able to measure divergences in concept usage while\ncorrecting for noise in embedding learning. Experiments on the MIMIC-III corpus\ndemonstrate that our approach captures clinically-relevant differences in\nconcept usage and provides an intuitive way to explore semantic characteristics\nof clinical document collections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00254v2",
            "date": "2019-10-01 08:38:26+00:00",
            "title": "Multilingual End-to-End Speech Translation",
            "authors": [
                "Hirofumi Inaguma",
                "Kevin Duh",
                "Tatsuya Kawahara",
                "Shinji Watanabe"
            ],
            "abstract": "In this paper, we propose a simple yet effective framework for multilingual\nend-to-end speech translation (ST), in which speech utterances in source\nlanguages are directly translated to the desired target languages with a\nuniversal sequence-to-sequence architecture. While multilingual models have\nshown to be useful for automatic speech recognition (ASR) and machine\ntranslation (MT), this is the first time they are applied to the end-to-end ST\nproblem. We show the effectiveness of multilingual end-to-end ST in two\nscenarios: one-to-many and many-to-many translations with publicly available\ndata. We experimentally confirm that multilingual end-to-end ST models\nsignificantly outperform bilingual ones in both scenarios. The generalization\nof multilingual training is also evaluated in a transfer learning scenario to a\nvery low-resource language pair. All of our codes and the database are publicly\navailable to encourage further research in this emergent multilingual ST topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00275v1",
            "date": "2019-10-01 09:39:07+00:00",
            "title": "Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in Distributional Semantic Models",
            "authors": [
                "Jeroen Van Hautte",
                "Guy Emerson",
                "Marek Rei"
            ],
            "abstract": "Word embeddings are an essential component in a wide range of natural\nlanguage processing applications. However, distributional semantic models are\nknown to struggle when only a small number of context sentences are available.\nSeveral methods have been proposed to obtain higher-quality vectors for these\nwords, leveraging both this context information and sometimes the word forms\nthemselves through a hybrid approach. We show that the current tasks do not\nsuffice to evaluate models that use word-form information, as such models can\neasily leverage word forms in the training data that are related to word forms\nin the test data. We introduce 3 new tasks, allowing for a more balanced\ncomparison between models. Furthermore, we show that hyperparameters that have\nlargely been ignored in previous work can consistently improve the performance\nof both baseline and advanced models, achieving a new state of the art on 4 out\nof 6 tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00337v1",
            "date": "2019-10-01 12:25:44+00:00",
            "title": "TMLab: Generative Enhanced Model (GEM) for adversarial attacks",
            "authors": [
                "Piotr Niewinski",
                "Maria Pszona",
                "Maria Janicka"
            ],
            "abstract": "We present our Generative Enhanced Model (GEM) that we used to create samples\nawarded the first prize on the FEVER 2.0 Breakers Task. GEM is the extended\nlanguage model developed upon GPT-2 architecture. The addition of novel target\nvocabulary input to the already existing context input enabled controlled text\ngeneration. The training procedure resulted in creating a model that inherited\nthe knowledge of pretrained GPT-2, and therefore was ready to generate\nnatural-like English sentences in the task domain with some additional control.\nAs a result, GEM generated malicious claims that mixed facts from various\narticles, so it became difficult to classify their truthfulness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00340v1",
            "date": "2019-10-01 12:33:16+00:00",
            "title": "VOnDA: A Framework for Ontology-Based Dialogue Management",
            "authors": [
                "Bernd Kiefer",
                "Anna Welker",
                "Christophe Biwer"
            ],
            "abstract": "We present VOnDA, a framework to implement the dialogue management\nfunctionality in dialogue systems. Although domain-independent, VOnDA is\ntailored towards dialogue systems with a focus on social communication, which\nimplies the need of long-term memory and high user adaptivity. For these\nsystems, which are used in health environments or elderly care, margin of error\nis very low and control over the dialogue process is of topmost importance. The\nsame holds for commercial applications, where customer trust is at risk.\nVOnDA's specification and memory layer relies upon (extended) RDF/OWL, which\nprovides a universal and uniform representation, and facilitates\ninteroperability with external data sources, e.g., from physical sensors.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1910.00382v1",
            "date": "2019-10-01 13:45:56+00:00",
            "title": "Latent-Variable Generative Models for Data-Efficient Text Classification",
            "authors": [
                "Xiaoan Ding",
                "Kevin Gimpel"
            ],
            "abstract": "Generative classifiers offer potential advantages over their discriminative\ncounterparts, namely in the areas of data efficiency, robustness to data shift\nand adversarial examples, and zero-shot learning (Ng and Jordan,2002; Yogatama\net al., 2017; Lewis and Fan,2019). In this paper, we improve generative text\nclassifiers by introducing discrete latent variables into the generative story,\nand explore several graphical model configurations. We parameterize the\ndistributions using standard neural architectures used in conditional language\nmodeling and perform learning by directly maximizing the log marginal\nlikelihood via gradient-based optimization, which avoids the need to do\nexpectation-maximization. We empirically characterize the performance of our\nmodels on six text classification datasets. The choice of where to include the\nlatent variable has a significant impact on performance, with the strongest\nresults obtained when using the latent variable as an auxiliary conditioning\nvariable in the generation of the textual input. This model consistently\noutperforms both the generative and discriminative classifiers in small-data\nsettings. We analyze our model by using it for controlled generation, finding\nthat the latent variable captures interpretable properties of the data, even\nwith very small training sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00458v2",
            "date": "2019-10-01 14:43:37+00:00",
            "title": "MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension",
            "authors": [
                "Di Jin",
                "Shuyang Gao",
                "Jiun-Yu Kao",
                "Tagyoung Chung",
                "Dilek Hakkani-tur"
            ],
            "abstract": "Machine Reading Comprehension (MRC) for question answering (QA), which aims\nto answer a question given the relevant context passages, is an important way\nto test the ability of intelligence systems to understand human language.\nMultiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it\noften requires more advanced reading comprehension skills such as logical\nreasoning, summarization, and arithmetic operations, compared to the extractive\ncounterpart where answers are usually spans of text within given passages.\nMoreover, most existing MCQA datasets are small in size, making the learning\ntask even harder. We introduce MMM, a Multi-stage Multi-task learning framework\nfor Multi-choice reading comprehension. Our method involves two sequential\nstages: coarse-tuning stage using out-of-domain datasets and multi-task\nlearning stage using a larger in-domain dataset to help model generalize better\nwith limited data. Furthermore, we propose a novel multi-step attention network\n(MAN) as the top-level classifier for this task. We demonstrate MMM\nsignificantly advances the state-of-the-art on four representative MCQA\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00515v1",
            "date": "2019-10-01 16:06:44+00:00",
            "title": "Detecting Alzheimer's Disease by estimating attention and elicitation path through the alignment of spoken picture descriptions with the picture prompt",
            "authors": [
                "Bahman Mirheidari",
                "Yilin Pan",
                "Traci Walker",
                "Markus Reuber",
                "Annalena Venneri",
                "Daniel Blackburn",
                "Heidi Christensen"
            ],
            "abstract": "Cognitive decline is a sign of Alzheimer's disease (AD), and there is\nevidence that tracking a person's eye movement, using eye tracking devices, can\nbe used for the automatic identification of early signs of cognitive decline.\nHowever, such devices are expensive and may not be easy-to-use for people with\ncognitive problems. In this paper, we present a new way of capturing similar\nvisual features, by using the speech of people describing the Cookie Theft\npicture - a common cognitive testing task - to identify regions in the picture\nprompt that will have caught the speaker's attention and elicited their speech.\nAfter aligning the automatically recognised words with different regions of the\npicture prompt, we extract information inspired by eye tracking metrics such as\ncoordinates of the area of interests (AOI)s, time spent in AOI, time to reach\nthe AOI, and the number of AOI visits. Using the DementiaBank dataset we train\na binary classifier (AD vs. healthy control) using 10-fold cross-validation and\nachieve an 80% F1-score using the timing information from the forced alignments\nof the automatic speech recogniser (ASR); this achieved around 72% using the\ntiming information from the ASR outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00553v2",
            "date": "2019-10-01 17:30:56+00:00",
            "title": "Better Document-Level Machine Translation with Bayes' Rule",
            "authors": [
                "Lei Yu",
                "Laurent Sartran",
                "Wojciech Stokowiec",
                "Wang Ling",
                "Lingpeng Kong",
                "Phil Blunsom",
                "Chris Dyer"
            ],
            "abstract": "We show that Bayes' rule provides an effective mechanism for creating\ndocument translation models that can be learned from only parallel sentences\nand monolingual documents---a compelling benefit as parallel documents are not\nalways available. In our formulation, the posterior probability of a candidate\ntranslation is the product of the unconditional (prior) probability of the\ncandidate output document and the \"reverse translation probability\" of\ntranslating the candidate output back into the source language. Our proposed\nmodel uses a powerful autoregressive language model as the prior on target\nlanguage documents, but it assumes that each sentence is translated\nindependently from the target to the source language. Crucially, at test time,\nwhen a source document is observed, the document language model prior induces\ndependencies between the translations of the source sentences in the posterior.\nThe model's independence assumption not only enables efficient use of available\ndata, but it additionally admits a practical left-to-right beam-search\nalgorithm for carrying out inference. Experiments show that our model benefits\nfrom using cross-sentence context in the language model, and it outperforms\nexisting document translation approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00610v1",
            "date": "2019-10-01 18:29:08+00:00",
            "title": "DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs",
            "authors": [
                "Yi-Lin Tuan",
                "Yun-Nung Chen",
                "Hung-yi Lee"
            ],
            "abstract": "Data-driven, knowledge-grounded neural conversation models are capable of\ngenerating more informative responses. However, these models have not yet\ndemonstrated that they can zero-shot adapt to updated, unseen knowledge graphs.\nThis paper proposes a new task about how to apply dynamic knowledge graphs in\nneural conversation model and presents a novel TV series conversation corpus\n(DyKgChat) for the task. Our new task and corpus aids in understanding the\ninfluence of dynamic knowledge graphs on responses generation. Also, we propose\na preliminary model that selects an output from two networks at each time step:\na sequence-to-sequence model (Seq2Seq) and a multi-hop reasoning model, in\norder to support dynamic knowledge graphs. To benchmark this new task and\nevaluate the capability of adaptation, we introduce several evaluation metrics\nand the experiments show that our proposed approach outperforms previous\nknowledge-grounded conversation models. The proposed corpus and model can\nmotivate the future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00673v1",
            "date": "2019-10-01 21:20:33+00:00",
            "title": "Learning to estimate label uncertainty for automatic radiology report parsing",
            "authors": [
                "Tobi Olatunji",
                "Li Yao"
            ],
            "abstract": "Bootstrapping labels from radiology reports has become the scalable\nalternative to provide inexpensive ground truth for medical imaging. Because of\nthe domain specific nature, state-of-the-art report labeling tools are\npredominantly rule-based. These tools, however, typically yield a binary 0 or 1\nprediction that indicates the presence or absence of abnormalities. These hard\ntargets are then used as ground truth to train image models in the downstream,\nforcing models to express high degree of certainty even on cases where\nspecificity is low. This could negatively impact the statistical efficiency of\nimage models. We address such an issue by training a Bidirectional Long-Short\nTerm Memory Network to augment heuristic-based discrete labels of X-ray reports\nfrom all body regions and achieve performance comparable or better than\ndomain-specific NLP, but with additional uncertainty estimates which enable\nfiner downstream image model training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00861v1",
            "date": "2019-10-02 10:17:28+00:00",
            "title": "Clinical Text Generation through Leveraging Medical Concept and Relations",
            "authors": [
                "Wangjin Lee",
                "Hyeryun Park",
                "Jooyoung Yoon",
                "Kyeongmo Kim",
                "Jinwook Choi"
            ],
            "abstract": "With a neural sequence generation model, this study aims to develop a method\nof writing the patient clinical texts given a brief medical history. As a\nproof-of-a-concept, we have demonstrated that it can be workable to use medical\nconcept embedding in clinical text generation. Our model was based on the\nSequence-to-Sequence architecture and trained with a large set of de-identified\nclinical text data. The quantitative result shows that our concept embedding\nmethod decreased the perplexity of the baseline architecture. Also, we discuss\nthe analyzed results from a human evaluation performed by medical doctors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00896v1",
            "date": "2019-10-02 12:02:46+00:00",
            "title": "The merits of Universal Language Model Fine-tuning for Small Datasets -- a case with Dutch book reviews",
            "authors": [
                "Benjamin van der Burgh",
                "Suzan Verberne"
            ],
            "abstract": "We evaluated the effectiveness of using language models, that were\npre-trained in one domain, as the basis for a classification model in another\ndomain: Dutch book reviews. Pre-trained language models have opened up new\npossibilities for classification tasks with limited labelled data, because\nrepresentation can be learned in an unsupervised fashion. In our experiments we\nhave studied the effects of training set size (100-1600 items) on the\nprediction accuracy of a ULMFiT classifier, based on a language models that we\npre-trained on the Dutch Wikipedia. We also compared ULMFiT to Support Vector\nMachines, which is traditionally considered suitable for small collections. We\nfound that ULMFiT outperforms SVM for all training set sizes and that\nsatisfactory results (~90%) can be achieved using training sets that can be\nmanually annotated within a few hours. We deliver both our new benchmark\ncollection of Dutch book reviews for sentiment classification as well as the\npre-trained Dutch language model to the community.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.01160v2",
            "date": "2019-10-02 18:47:17+00:00",
            "title": "Identifying Nuances in Fake News vs. Satire: Using Semantic and Linguistic Cues",
            "authors": [
                "Or Levi",
                "Pedram Hosseini",
                "Mona Diab",
                "David A. Broniatowski"
            ],
            "abstract": "The blurry line between nefarious fake news and protected-speech satire has\nbeen a notorious struggle for social media platforms. Further to the efforts of\nreducing exposure to misinformation on social media, purveyors of fake news\nhave begun to masquerade as satire sites to avoid being demoted. In this work,\nwe address the challenge of automatically classifying fake news versus satire.\nPrevious work have studied whether fake news and satire can be distinguished\nbased on language differences. Contrary to fake news, satire stories are\nusually humorous and carry some political or social message. We hypothesize\nthat these nuances could be identified using semantic and linguistic cues.\nConsequently, we train a machine learning method using semantic representation,\nwith a state-of-the-art contextual language model, and with linguistic features\nbased on textual coherence metrics. Empirical evaluation attests to the merits\nof our approach compared to the language-based baseline and sheds light on the\nnuances between fake news and satire. As avenues for future work, we consider\nstudying additional linguistic features related to the humor aspect, and\nenriching the data with current news events, to help identify a political or\nsocial message.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01244v1",
            "date": "2019-10-02 22:36:51+00:00",
            "title": "Linking artificial and human neural representations of language",
            "authors": [
                "Jon Gauthier",
                "Roger Levy"
            ],
            "abstract": "What information from an act of sentence understanding is robustly\nrepresented in the human brain? We investigate this question by comparing\nsentence encoding models on a brain decoding task, where the sentence that an\nexperimental participant has seen must be predicted from the fMRI signal evoked\nby the sentence. We take a pre-trained BERT architecture as a baseline sentence\nencoding model and fine-tune it on a variety of natural language understanding\n(NLU) tasks, asking which lead to improvements in brain-decoding performance.\n  We find that none of the sentence encoding tasks tested yield significant\nincreases in brain decoding performance. Through further task ablations and\nrepresentational analyses, we find that tasks which produce syntax-light\nrepresentations yield significant improvements in brain decoding performance.\nOur results constrain the space of NLU models that could best account for human\nneural representations of language, but also suggest limits on the possibility\nof decoding fine-grained syntactic information from fMRI human neuroimaging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01274v1",
            "date": "2019-10-03 01:51:17+00:00",
            "title": "Extracting UMLS Concepts from Medical Text Using General and Domain-Specific Deep Learning Models",
            "authors": [
                "Kathleen C. Fraser",
                "Isar Nejadgholi",
                "Berry De Bruijn",
                "Muqun Li",
                "Astha LaPlante",
                "Khaldoun Zine El Abidine"
            ],
            "abstract": "Entity recognition is a critical first step to a number of clinical NLP\napplications, such as entity linking and relation extraction. We present the\nfirst attempt to apply state-of-the-art entity recognition approaches on a\nnewly released dataset, MedMentions. This dataset contains over 4000 biomedical\nabstracts, annotated for UMLS semantic types. In comparison to existing\ndatasets, MedMentions contains a far greater number of entity types, and thus\nrepresents a more challenging but realistic scenario in a real-world setting.\nWe explore a number of relevant dimensions, including the use of contextual\nversus non-contextual word embeddings, general versus domain-specific\nunsupervised pre-training, and different deep learning architectures. We\ncontrast our results against the well-known i2b2 2010 entity recognition\ndataset, and propose a new method to combine general and domain-specific\ninformation. While producing a state-of-the-art result for the i2b2 2010 task\n(F1 = 0.90), our results on MedMentions are significantly lower (F1 = 0.63),\nsuggesting there is still plenty of opportunity for improvement on this new\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01299v3",
            "date": "2019-10-03 04:27:49+00:00",
            "title": "Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing",
            "authors": [
                "Yuta Koreeda",
                "Gaku Morio",
                "Terufumi Morishita",
                "Hiroaki Ozaki",
                "Kohsuke Yanai"
            ],
            "abstract": "This paper describes the proposed system of the Hitachi team for the\nCross-Framework Meaning Representation Parsing (MRP 2019) shared task. In this\nshared task, the participating systems were asked to predict nodes, edges and\ntheir attributes for five frameworks, each with different order of\n\"abstraction\" from input tokens. We proposed a unified encoder-to-biaffine\nnetwork for all five frameworks, which effectively incorporates a shared\nencoder to extract rich input features, decoder networks to generate anchorless\nnodes in UCCA and AMR, and biaffine networks to predict edges. Our system was\nranked fifth with the macro-averaged MRP F1 score of 0.7604, and outperformed\nthe baseline unified transition-based MRP. Furthermore, post-evaluation\nexperiments showed that we can boost the performance of the proposed system by\nincorporating multi-task learning, whereas the baseline could not. These imply\nefficacy of incorporating the biaffine network to the shared architecture for\nMRP and that learning heterogeneous meaning representations at once can boost\nthe system performance.",
            "journal ref": "in Proceedings of the Shared Task on Cross-Framework Meaning\n  Representation Parsing at the 2019 Conference on Natural Language Learning",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01363v1",
            "date": "2019-10-03 09:00:58+00:00",
            "title": "Mapping (Dis-)Information Flow about the MH17 Plane Crash",
            "authors": [
                "Mareike Hartmann",
                "Yevgeniy Golovchenko",
                "Isabelle Augenstein"
            ],
            "abstract": "Digital media enables not only fast sharing of information, but also\ndisinformation. One prominent case of an event leading to circulation of\ndisinformation on social media is the MH17 plane crash. Studies analysing the\nspread of information about this event on Twitter have focused on small,\nmanually annotated datasets, or used proxys for data annotation. In this work,\nwe examine to what extent text classifiers can be used to label data for\nsubsequent content analysis, in particular we focus on predicting pro-Russian\nand pro-Ukrainian Twitter content related to the MH17 plane crash. Even though\nwe find that a neural classifier improves over a hashtag based baseline,\nlabeling pro-Russian and pro-Ukrainian content with high precision remains a\nchallenging problem. We provide an error analysis underlining the difficulty of\nthe task and identify factors that might help improve classification in future\nwork. Finally, we show how the classifier can facilitate the annotation task\nfor human annotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01441v1",
            "date": "2019-08-31 17:25:00+00:00",
            "title": "Can Sentiment Analysis Reveal Structure in a Plotless Novel?",
            "authors": [
                "Katherine Elkins",
                "Jon Chun"
            ],
            "abstract": "Modernist novels are thought to break with traditional plot structure. In\nthis paper, we test this theory by applying Sentiment Analysis to one of the\nmost famous modernist novels, To the Lighthouse by Virginia Woolf. We first\nassess Sentiment Analysis in light of the critique that it cannot adequately\naccount for literary language: we use a unique statistical comparison to\ndemonstrate that even simple lexical approaches to Sentiment Analysis are\nsurprisingly effective. We then use the Syuzhet.R package to explore\nsimilarities and differences across modeling methods. This comparative\napproach, when paired with literary close reading, can offer interpretive\nclues. To our knowledge, we are the first to undertake a hybrid model that\nfully leverages the strengths of both computational analysis and close reading.\nThis hybrid model raises new questions for the literary critic, such as how to\ninterpret relative versus absolute emotional valence and how to take into\naccount subjective identification. Our finding is that while To the Lighthouse\ndoes not replicate a plot centered around a traditional hero, it does reveal an\nunderlying emotional structure distributed between characters - what we term a\ndistributed heroine model. This finding is innovative in the field of modernist\nand narrative studies and demonstrates that a hybrid method can yield\nsignificant discoveries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01489v1",
            "date": "2019-10-03 14:12:38+00:00",
            "title": "Complex networks based word embeddings",
            "authors": [
                "Nicolas Dugu\u00e9",
                "Victor Connes"
            ],
            "abstract": "Most of the time, the first step to learn word embeddings is to build a word\nco-occurrence matrix. As such matrices are equivalent to graphs, complex\nnetworks theory can naturally be used to deal with such data. In this paper, we\nconsider applying community detection, a main tool of this field, to the\nco-occurrence matrix corresponding to a huge corpus. Community structure is\nused as a way to reduce the dimensionality of the initial space. Using this\ncommunity structure, we propose a method to extract word embeddings that are\ncomparable to the state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01761v1",
            "date": "2019-10-03 23:39:31+00:00",
            "title": "Character Feature Engineering for Japanese Word Segmentation",
            "authors": [
                "Mike Tian-Jian Jiang"
            ],
            "abstract": "On word segmentation problems, machine learning architecture engineering\noften draws attention. The problem representation itself, however, has remained\nalmost static as either word lattice ranking or character sequence tagging, for\nat least two decades. The latter of-ten shows stronger predictive power than\nthe former for out-of-vocabulary (OOV) issue. When the issue escalating to\nrapid adaptation, which is a common scenario for industrial applications,\nactive learning of partial annotations or re-training with additional lexical\nre-sources is usually applied, however, from a somewhat word-based perspective.\nNot only it is uneasy for end-users to comply with linguistically consistent\nword boundary decisions, but also the risk/cost of forking models permanently\nwith estimated weights is seldom affordable. To overcome the obstacle, this\nwork provides an alternative, which uses linguistic intuition about character\ncompositions, such that a sophisticated feature set and its derived scheme can\nenable dynamic lexicon expansion with the model remaining intact. Experiment\nresults suggest that the proposed solution, with or without external lexemes,\nperforms competitively in terms of F1 score and OOV recall across various\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02028v1",
            "date": "2019-10-04 16:43:49+00:00",
            "title": "Tanbih: Get To Know What You Are Reading",
            "authors": [
                "Yifan Zhang",
                "Giovanni Da San Martino",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Salvatore Romeo",
                "Jisun An",
                "Haewoon Kwak",
                "Todor Staykovski",
                "Israa Jaradat",
                "Georgi Karadzhov",
                "Ramy Baly",
                "Kareem Darwish",
                "James Glass",
                "Preslav Nakov"
            ],
            "abstract": "We introduce Tanbih, a news aggregator with intelligent analysis tools to\nhelp readers understanding what's behind a news story. Our system displays news\ngrouped into events and generates media profiles that show the general\nfactuality of reporting, the degree of propagandistic content,\nhyper-partisanship, leading political ideology, general frame of reporting, and\nstance with respect to various claims and topics of a news outlet. In addition,\nwe automatically analyse each article to detect whether it is propagandistic\nand to determine its stance with respect to a number of controversial topics.",
            "journal ref": "EMNLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02065v3",
            "date": "2019-10-04 17:44:36+00:00",
            "title": "Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods",
            "authors": [
                "Oana-Maria Camburu",
                "Eleonora Giunchiglia",
                "Jakob Foerster",
                "Thomas Lukasiewicz",
                "Phil Blunsom"
            ],
            "abstract": "For AI systems to garner widespread public acceptance, we must develop\nmethods capable of explaining the decisions of black-box models such as neural\nnetworks. In this work, we identify two issues of current explanatory methods.\nFirst, we show that two prevalent perspectives on explanations ---\nfeature-additivity and feature-selection --- lead to fundamentally different\ninstance-wise explanations. In the literature, explainers from different\nperspectives are currently being directly compared, despite their distinct\nexplanation goals. The second issue is that current post-hoc explainers are\neither validated under simplistic scenarios (on simple models such as linear\nregression, or on models trained on syntactic datasets), or, when applied to\nreal-world neural networks, explainers are commonly validated under the\nassumption that the learned models behave reasonably. However, neural networks\noften rely on unreasonable correlations, even when producing correct decisions.\nWe introduce a verification framework for explanatory methods under the\nfeature-selection perspective. Our framework is based on a non-trivial neural\nnetwork architecture trained on a real-world task, and for which we are able to\nprovide guarantees on its inner workings. We validate the efficacy of our\nevaluation by showing the failure modes of current explainers. We aim for this\nframework to provide a publicly available, off-the-shelf evaluation when the\nfeature-selection perspective on explanations is needed.",
            "journal ref": "NeurIPS 2019 Workshop on Safety and Robustness in Decision Making,\n  Vancouver, Canada",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02076v1",
            "date": "2019-10-04 16:01:23+00:00",
            "title": "Contrastive Language Adaptation for Cross-Lingual Stance Detection",
            "authors": [
                "Mitra Mohtarami",
                "James Glass",
                "Preslav Nakov"
            ],
            "abstract": "We study cross-lingual stance detection, which aims to leverage labeled data\nin one language to identify the relative perspective (or stance) of a given\ndocument with respect to a claim in a different target language. In particular,\nwe introduce a novel contrastive language adaptation approach applied to memory\nnetworks, which ensures accurate alignment of stances in the source and target\nlanguages, and can effectively deal with the challenge of limited labeled data\nin the target language. The evaluation results on public benchmark datasets and\ncomparison against current state-of-the-art approaches demonstrate the\neffectiveness of our approach.",
            "journal ref": "EMNLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02211v2",
            "date": "2019-10-05 05:03:30+00:00",
            "title": "On Dimensional Linguistic Properties of the Word Embedding Space",
            "authors": [
                "Vikas Raunak",
                "Vaibhav Kumar",
                "Vivek Gupta",
                "Florian Metze"
            ],
            "abstract": "Word embeddings have become a staple of several natural language processing\ntasks, yet much remains to be understood about their properties. In this work,\nwe analyze word embeddings in terms of their principal components and arrive at\na number of novel and counterintuitive observations. In particular, we\ncharacterize the utility of variance explained by the principal components as a\nproxy for downstream performance. Furthermore, through syntactic probing of the\nprincipal embedding space, we show that the syntactic information captured by a\nprincipal component does not correlate with the amount of variance it explains.\nConsequently, we investigate the limitations of variance based embedding\npost-processing and demonstrate that such post-processing is counter-productive\nin sentence classification and machine translation tasks. Finally, we offer a\nfew precautionary guidelines on applying variance based embedding\npost-processing and explain why non-isotropic geometry might be integral to\nword embedding performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02292v1",
            "date": "2019-10-05 16:30:49+00:00",
            "title": "Keyword Spotter Model for Crop Pest and Disease Monitoring from Community Radio Data",
            "authors": [
                "Benjamin Akera",
                "Joyce Nakatumba-Nabende",
                "Jonathan Mukiibi",
                "Ali Hussein",
                "Nathan Baleeta",
                "Daniel Ssendiwala",
                "Samiiha Nalwooga"
            ],
            "abstract": "In societies with well developed internet infrastructure, social media is the\nleading medium of communication for various social issues especially for\nbreaking news situations. In rural Uganda however, public community radio is\nstill a dominant means for news dissemination. Community radio gives audience\nto the general public especially to individuals living in rural areas, and thus\nplays an important role in giving a voice to those living in the broadcast\narea. It is an avenue for participatory communication and a tool relevant in\nboth economic and social development.This is supported by the rise to ubiquity\nof mobile phones providing access to phone-in or text-in talk shows. In this\npaper, we describe an approach to analysing the readily available community\nradio data with machine learning-based speech keyword spotting techniques. We\nidentify the keywords of interest related to agriculture and build models to\nautomatically identify these keywords from audio streams. Our contribution\nthrough these techniques is a cost-efficient and effective way to monitor food\nsecurity concerns particularly in rural areas. Through keyword spotting and\nradio talk show analysis, issues such as crop diseases, pests, drought and\nfamine can be captured and fed into an early warning system for stakeholders\nand policy makers.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1910.02339v3",
            "date": "2019-10-05 22:57:04+00:00",
            "title": "Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations",
            "authors": [
                "Kezhen Chen",
                "Qiuyuan Huang",
                "Hamid Palangi",
                "Paul Smolensky",
                "Kenneth D. Forbus",
                "Jianfeng Gao"
            ],
            "abstract": "Generating formal-language programs represented by relational tuples, such as\nLisp programs or mathematical operations, to solve problems stated in natural\nlanguage is a challenging task because it requires explicitly capturing\ndiscrete symbolic structural information implicit in the input. However, most\ngeneral neural sequence models do not explicitly capture such structural\ninformation, limiting their performance on these tasks. In this paper, we\npropose a new encoder-decoder model based on a structured neural\nrepresentation, Tensor Product Representations (TPRs), for mapping\nNatural-language problems to Formal-language solutions, called TP-N2F. The\nencoder of TP-N2F employs TPR `binding' to encode natural-language symbolic\nstructure in vector space and the decoder uses TPR `unbinding' to generate, in\nsymbolic space, a sequential program represented by relational tuples, each\nconsisting of a relation (or operation) and a number of arguments. TP-N2F\nconsiderably outperforms LSTM-based seq2seq models on two benchmarks and\ncreates new state-of-the-art results. Ablation studies show that improvements\ncan be attributed to the use of structured TPRs explicitly in both the encoder\nand decoder. Analysis of the learned structures shows how TPRs enhance the\ninterpretability of TP-N2F.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02365v1",
            "date": "2019-10-06 04:02:55+00:00",
            "title": "Multilingual Dialogue Generation with Shared-Private Memory",
            "authors": [
                "Chen Chen",
                "Lisong Qiu",
                "Zhenxin Fu",
                "Dongyan Zhao",
                "Junfei Liu",
                "Rui Yan"
            ],
            "abstract": "Existing dialog systems are all monolingual, where features shared among\ndifferent languages are rarely explored. In this paper, we introduce a novel\nmultilingual dialogue system. Specifically, we augment the sequence to sequence\nframework with improved shared-private memory. The shared memory learns common\nfeatures among different languages and facilitates a cross-lingual transfer to\nboost dialogue systems, while the private memory is owned by each separate\nlanguage to capture its unique feature. Experiments conducted on Chinese and\nEnglish conversation corpora of different scales show that our proposed\narchitecture outperforms the individually learned model with the help of the\nother language, where the improvement is particularly distinct when the\ntraining data is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02375v1",
            "date": "2019-10-06 05:06:38+00:00",
            "title": "Design and Use of Loop-Transformation Pragmas",
            "authors": [
                "Michael Kruse",
                "Hal Finkel"
            ],
            "abstract": "Adding a pragma directive into the source code is arguably easier than\nrewriting it, for instance for loop unrolling. Moreover, if the application is\nmaintained for multiple platforms, their difference in performance\ncharacteristics may require different code transformations. Code transformation\ndirectives allow replacing the directives depending on the platform, i.e.\nseparation of code semantics and its performance optimization.\n  In this paper, we explore the design space (syntax and semantics) of adding\nsuch directive into a future OpenMP specification. Using a prototype\nimplementation in Clang, we demonstrate the usefulness of such directives on a\nfew benchmarks.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02610v1",
            "date": "2019-10-07 04:58:43+00:00",
            "title": "Multi-hop Question Answering via Reasoning Chains",
            "authors": [
                "Jifan Chen",
                "Shih-ting Lin",
                "Greg Durrett"
            ],
            "abstract": "Multi-hop question answering requires models to gather information from\ndifferent parts of a text to answer a question. Most current approaches learn\nto address this task in an end-to-end way with neural networks, without\nmaintaining an explicit representation of the reasoning process. We propose a\nmethod to extract a discrete reasoning chain over the text, which consists of a\nseries of sentences leading to the answer. We then feed the extracted chains to\na BERT-based QA model to do final answer prediction. Critically, we do not rely\non gold annotated chains or \"supporting facts:\" at training time, we derive\npseudogold reasoning chains using heuristics based on named entity recognition\nand coreference resolution. Nor do we rely on these annotations at test time,\nas our model learns to extract chains from raw text alone. We test our approach\non two recently proposed large multi-hop question answering datasets: WikiHop\nand HotpotQA, and achieve state-of-art performance on WikiHop and strong\nperformance on HotpotQA. Our analysis shows the properties of chains that are\ncrucial for high performance: in particular, modeling extraction sequentially\nis important, as is dealing with each candidate sentence in a context-aware\nway. Furthermore, human evaluation shows that our extracted chains allow humans\nto give answers with high confidence, indicating that these are a strong\nintermediate abstraction for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02612v1",
            "date": "2019-10-07 05:27:27+00:00",
            "title": "Compositional Generalization for Primitive Substitutions",
            "authors": [
                "Yuanpeng Li",
                "Liang Zhao",
                "Jianyu Wang",
                "Joel Hestness"
            ],
            "abstract": "Compositional generalization is a basic mechanism in human language learning,\nbut current neural networks lack such ability. In this paper, we conduct\nfundamental research for encoding compositionality in neural networks.\nConventional methods use a single representation for the input sentence, making\nit hard to apply prior knowledge of compositionality. In contrast, our approach\nleverages such knowledge with two representations, one generating attention\nmaps, and the other mapping attended input words to output symbols. We reduce\nthe entropy in each representation to improve generalization. Our experiments\ndemonstrate significant improvements over the conventional methods in five NLP\ntasks including instruction learning and machine translation. In the SCAN\ndomain, it boosts accuracies from 14.0% to 98.8% in Jump task, and from 92.0%\nto 99.7% in TurnLeft task. It also beats human performance on a few-shot\nlearning task. We hope the proposed approach can help ease future research\ntowards human-level compositional language learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02724v2",
            "date": "2019-10-07 11:08:24+00:00",
            "title": "Improving Relation Extraction with Knowledge-attention",
            "authors": [
                "Pengfei Li",
                "Kezhi Mao",
                "Xuefeng Yang",
                "Qi Li"
            ],
            "abstract": "While attention mechanisms have been proven to be effective in many NLP\ntasks, majority of them are data-driven. We propose a novel knowledge-attention\nencoder which incorporates prior knowledge from external lexical resources into\ndeep neural networks for relation extraction task. Furthermore, we present\nthree effective ways of integrating knowledge-attention with self-attention to\nmaximize the utilization of both knowledge and data. The proposed relation\nextraction system is end-to-end and fully attention-based. Experiment results\nshow that the proposed knowledge-attention mechanism has complementary\nstrengths with self-attention, and our integrated models outperform existing\nCNN, RNN, and self-attention based models. State-of-the-art performance is\nachieved on TACRED, a complex and large-scale relation extraction dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02754v1",
            "date": "2019-10-07 12:42:09+00:00",
            "title": "On Leveraging the Visual Modality for Neural Machine Translation",
            "authors": [
                "Vikas Raunak",
                "Sang Keun Choe",
                "Quanyang Lu",
                "Yi Xu",
                "Florian Metze"
            ],
            "abstract": "Leveraging the visual modality effectively for Neural Machine Translation\n(NMT) remains an open problem in computational linguistics. Recently, Caglayan\net al. posit that the observed gains are limited mainly due to the very simple,\nshort, repetitive sentences of the Multi30k dataset (the only multimodal MT\ndataset available at the time), which renders the source text sufficient for\ncontext. In this work, we further investigate this hypothesis on a new large\nscale multimodal Machine Translation (MMT) dataset, How2, which has 1.57 times\nlonger mean sentence length than Multi30k and no repetition. We propose and\nevaluate three novel fusion techniques, each of which is designed to ensure the\nutilization of visual context at different stages of the Sequence-to-Sequence\ntransduction pipeline, even under full linguistic context. However, we still\nobtain only marginal gains under full linguistic context and posit that visual\nembeddings extracted from deep vision models (ResNet for Multi30k, ResNext for\nHow2) do not lend themselves to increasing the discriminativeness between the\nvocabulary elements at token level prediction in NMT. We demonstrate this\nqualitatively by analyzing attention distribution and quantitatively through\nPrincipal Component Analysis, arriving at the conclusion that it is the quality\nof the visual embeddings rather than the length of sentences, which need to be\nimproved in existing MMT datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02893v2",
            "date": "2019-10-07 16:37:31+00:00",
            "title": "Parallel Iterative Edit Models for Local Sequence Transduction",
            "authors": [
                "Abhijeet Awasthi",
                "Sunita Sarawagi",
                "Rasna Goyal",
                "Sabyasachi Ghosh",
                "Vihari Piratla"
            ],
            "abstract": "We present a Parallel Iterative Edit (PIE) model for the problem of local\nsequence transduction arising in tasks like Grammatical error correction (GEC).\nRecent approaches are based on the popular encoder-decoder (ED) model for\nsequence to sequence learning. The ED model auto-regressively captures full\ndependency among output tokens but is slow due to sequential decoding. The PIE\nmodel does parallel decoding, giving up the advantage of modelling full\ndependency in the output, yet it achieves accuracy competitive with the ED\nmodel for four reasons: 1.~predicting edits instead of tokens, 2.~labeling\nsequences instead of generating sequences, 3.~iteratively refining predictions\nto capture dependencies, and 4.~factorizing logits over edits and their token\nargument to harness pre-trained language models like BERT. Experiments on tasks\nspanning GEC, OCR correction and spell correction demonstrate that the PIE\nmodel is an accurate and significantly faster alternative for local sequence\ntransduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02902v1",
            "date": "2019-10-07 16:44:32+00:00",
            "title": "Correlations between Word Vector Sets",
            "authors": [
                "Vitalii Zhelezniak",
                "April Shen",
                "Daniel Busbridge",
                "Aleksandar Savkov",
                "Nils Hammerla"
            ],
            "abstract": "Similarity measures based purely on word embeddings are comfortably competing\nwith much more sophisticated deep learning and expert-engineered systems on\nunsupervised semantic textual similarity (STS) tasks. In contrast to commonly\nused geometric approaches, we treat a single word embedding as e.g. 300\nobservations from a scalar random variable. Using this paradigm, we first\nillustrate that similarities derived from elementary pooling operations and\nclassic correlation coefficients yield excellent results on standard STS\nbenchmarks, outperforming many recently proposed methods while being much\nfaster and trivial to implement. Next, we demonstrate how to avoid pooling\noperations altogether and compare sets of word embeddings directly via\ncorrelation operators between reproducing kernel Hilbert spaces. Just like\ncosine similarity is used to compare individual word vectors, we introduce a\nnovel application of the centered kernel alignment (CKA) as a natural\ngeneralisation of squared cosine similarity for sets of word vectors. Likewise,\nCKA is very easy to implement and enjoys very strong empirical results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02915v2",
            "date": "2019-10-07 17:16:04+00:00",
            "title": "Commonsense Knowledge Base Completion with Structural and Semantic Context",
            "authors": [
                "Chaitanya Malaviya",
                "Chandra Bhagavatula",
                "Antoine Bosselut",
                "Yejin Choi"
            ],
            "abstract": "Automatic KB completion for commonsense knowledge graphs (e.g., ATOMIC and\nConceptNet) poses unique challenges compared to the much studied conventional\nknowledge bases (e.g., Freebase). Commonsense knowledge graphs use free-form\ntext to represent nodes, resulting in orders of magnitude more nodes compared\nto conventional KBs (18x more nodes in ATOMIC compared to Freebase\n(FB15K-237)). Importantly, this implies significantly sparser graph structures\n- a major challenge for existing KB completion methods that assume densely\nconnected graphs over a relatively smaller set of nodes. In this paper, we\npresent novel KB completion models that can address these challenges by\nexploiting the structural and semantic context of nodes. Specifically, we\ninvestigate two key ideas: (1) learning from local graph structure, using graph\nconvolutional networks and automatic graph densification and (2) transfer\nlearning from pre-trained language models to knowledge graphs for enhanced\ncontextual representation of knowledge. We describe our method to incorporate\ninformation from both these sources in a joint model and provide the first\nempirical results for KB completion on ATOMIC and evaluation with ranking\nmetrics on ConceptNet. Our results demonstrate the effectiveness of language\nmodel representations in boosting link prediction performance and the\nadvantages of learning from local graph structure (+1.5 points in MRR for\nConceptNet) when training on subgraphs for computational efficiency. Further\nanalysis on model predictions shines light on the types of commonsense\nknowledge that language models capture well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03065v3",
            "date": "2019-10-07 20:14:23+00:00",
            "title": "Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations",
            "authors": [
                "Oana-Maria Camburu",
                "Brendan Shillingford",
                "Pasquale Minervini",
                "Thomas Lukasiewicz",
                "Phil Blunsom"
            ],
            "abstract": "To increase trust in artificial intelligence systems, a promising research\ndirection consists of designing neural models capable of generating natural\nlanguage explanations for their predictions. In this work, we show that such\nmodels are nonetheless prone to generating mutually inconsistent explanations,\nsuch as \"Because there is a dog in the image\" and \"Because there is no dog in\nthe [same] image\", exposing flaws in either the decision-making process of the\nmodel or in the generation of the explanations. We introduce a simple yet\neffective adversarial framework for sanity checking models against the\ngeneration of inconsistent natural language explanations. Moreover, as part of\nthe framework, we address the problem of adversarial attacks with full target\nsequences, a scenario that was not previously addressed in sequence-to-sequence\nattacks. Finally, we apply our framework on a state-of-the-art neural natural\nlanguage inference model that provides natural language explanations for its\npredictions. Our framework shows that this model is capable of generating a\nsignificant number of inconsistent explanations.",
            "journal ref": "Short Paper at ACL, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03176v1",
            "date": "2019-10-08 02:31:35+00:00",
            "title": "SesameBERT: Attention for Anywhere",
            "authors": [
                "Ta-Chun Su",
                "Hsiang-Chih Cheng"
            ],
            "abstract": "Fine-tuning with pre-trained models has achieved exceptional results for many\nlanguage tasks. In this study, we focused on one such self-attention network\nmodel, namely BERT, which has performed well in terms of stacking layers across\ndiverse language-understanding benchmarks. However, in many downstream tasks,\ninformation between layers is ignored by BERT for fine-tuning. In addition,\nalthough self-attention networks are well-known for their ability to capture\nglobal dependencies, room for improvement remains in terms of emphasizing the\nimportance of local contexts. In light of these advantages and disadvantages,\nthis paper proposes SesameBERT, a generalized fine-tuning method that (1)\nenables the extraction of global information among all layers through Squeeze\nand Excitation and (2) enriches local information by capturing neighboring\ncontexts via Gaussian blurring. Furthermore, we demonstrated the effectiveness\nof our approach in the HANS dataset, which is used to determine whether models\nhave adopted shallow heuristics instead of learning underlying generalizations.\nThe experiments revealed that SesameBERT outperformed BERT with respect to GLUE\nbenchmark and the HANS evaluation set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03262v3",
            "date": "2019-10-08 07:57:48+00:00",
            "title": "Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion",
            "authors": [
                "Philipp Christmann",
                "Rishiraj Saha Roy",
                "Abdalghani Abujabal",
                "Jyotsna Singh",
                "Gerhard Weikum"
            ],
            "abstract": "Fact-centric information needs are rarely one-shot; users typically ask\nfollow-up questions to explore a topic. In such a conversational setting, the\nuser's inputs are often incomplete, with entities or predicates left out, and\nungrammatical phrases. This poses a huge challenge to question answering (QA)\nsystems that typically rely on cues in full-fledged interrogative sentences. As\na solution, we develop CONVEX: an unsupervised method that can answer\nincomplete questions over a knowledge graph (KG) by maintaining conversation\ncontext using entities and predicates seen so far and automatically inferring\nmissing or ambiguous pieces for follow-up questions. The core of our method is\na graph exploration algorithm that judiciously expands a frontier to find\ncandidate answers for the current question. To evaluate CONVEX, we release\nConvQuestions, a crowdsourced benchmark with 11,200 distinct conversations from\nfive different domains. We show that CONVEX: (i) adds conversational support to\nany stand-alone QA system, and (ii) outperforms state-of-the-art baselines and\nquestion completion strategies.",
            "journal ref": "CIKM 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.03270v1",
            "date": "2019-10-08 08:33:56+00:00",
            "title": "CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech",
            "authors": [
                "Y. L. Chung",
                "E. Kuzmenko",
                "S. S. Tekiroglu",
                "M. Guerini"
            ],
            "abstract": "Although there is an unprecedented effort to provide adequate responses in\nterms of laws and policies to hate content on social media platforms, dealing\nwith hatred online is still a tough problem. Tackling hate speech in the\nstandard way of content deletion or user suspension may be charged with\ncensorship and overblocking. One alternate strategy, that has received little\nattention so far by the research community, is to actually oppose hate content\nwith counter-narratives (i.e. informed textual responses). In this paper, we\ndescribe the creation of the first large-scale, multilingual, expert-based\ndataset of hate speech/counter-narrative pairs. This dataset has been built\nwith the effort of more than 100 operators from three different NGOs that\napplied their training and expertise to the task. Together with the collected\ndata we also provide additional annotations about expert demographics, hate and\nresponse type, and data augmentation through translation and paraphrasing.\nFinally, we provide initial experiments to assess the quality of our data.",
            "journal ref": "In Proceedings of ACL 2019 (pp. 2819-2829)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03320v1",
            "date": "2019-10-08 10:29:09+00:00",
            "title": "One-To-Many Multilingual End-to-end Speech Translation",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Nowadays, training end-to-end neural models for spoken language translation\n(SLT) still has to confront with extreme data scarcity conditions. The existing\nSLT parallel corpora are indeed orders of magnitude smaller than those\navailable for the closely related tasks of automatic speech recognition (ASR)\nand machine translation (MT), which usually comprise tens of millions of\ninstances. To cope with data paucity, in this paper we explore the\neffectiveness of transfer learning in end-to-end SLT by presenting a\nmultilingual approach to the task. Multilingual solutions are widely studied in\nMT and usually rely on ``\\textit{target forcing}'', in which multilingual\nparallel data are combined to train a single model by prepending to the input\nsequences a language token that specifies the target language. However, when\ntested in speech translation, our experiments show that MT-like \\textit{target\nforcing}, used as is, is not effective in discriminating among the target\nlanguages. Thus, we propose a variant that uses target-language embeddings to\nshift the input representations in different portions of the space according to\nthe language, so to better support the production of output in the desired\ntarget language. Our experiments on end-to-end SLT from English into six\nlanguages show important improvements when translating into similar languages,\nespecially when these are supported by scarce data. Further improvements are\nobtained when using English ASR data as an additional language (up to $+2.5$\nBLEU points).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03387v1",
            "date": "2019-10-08 13:38:39+00:00",
            "title": "When Specialization Helps: Using Pooled Contextualized Embeddings to Detect Chemical and Biomedical Entities in Spanish",
            "authors": [
                "Manuel Stoeckel",
                "Wahed Hemati",
                "Alexander Mehler"
            ],
            "abstract": "The recognition of pharmacological substances, compounds and proteins is an\nessential preliminary work for the recognition of relations between chemicals\nand other biomedically relevant units. In this paper, we describe an approach\nto Task 1 of the PharmaCoNER Challenge, which involves the recognition of\nmentions of chemicals and drugs in Spanish medical texts. We train a\nstate-of-the-art BiLSTM-CRF sequence tagger with stacked Pooled Contextualized\nEmbeddings, word and sub-word embeddings using the open-source framework FLAIR.\nWe present a new corpus composed of articles and papers from Spanish health\nscience journals, termed the Spanish Health Corpus, and use it to train\ndomain-specific embeddings which we incorporate in our model training. We\nachieve a result of 89.76% F1-score using pre-trained embeddings and are able\nto improve these results to 90.52% F1-score using specialized embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03432v1",
            "date": "2019-10-08 14:48:43+00:00",
            "title": "Federated Learning of N-gram Language Models",
            "authors": [
                "Mingqing Chen",
                "Ananda Theertha Suresh",
                "Rajiv Mathews",
                "Adeline Wong",
                "Cyril Allauzen",
                "Fran\u00e7oise Beaufays",
                "Michael Riley"
            ],
            "abstract": "We propose algorithms to train production-quality n-gram language models\nusing federated learning. Federated learning is a distributed computation\nplatform that can be used to train global models for portable devices such as\nsmart phones. Federated learning is especially relevant for applications\nhandling privacy-sensitive data, such as virtual keyboards, because training is\nperformed without the users' data ever leaving their devices. While the\nprinciples of federated learning are fairly generic, its methodology assumes\nthat the underlying models are neural networks. However, virtual keyboards are\ntypically powered by n-gram language models for latency reasons.\n  We propose to train a recurrent neural network language model using the\ndecentralized FederatedAveraging algorithm and to approximate this federated\nmodel server-side with an n-gram model that can be deployed to devices for fast\ninference. Our technical contributions include ways of handling large\nvocabularies, algorithms to correct capitalization errors in user data, and\nefficient finite state transducer algorithms to convert word language models to\nword-piece language models and vice versa. The n-gram language models trained\nwith federated learning are compared to n-grams trained with traditional\nserver-based algorithms using A/B tests on tens of millions of users of virtual\nkeyboard. Results are presented for two languages, American English and\nBrazilian Portuguese. This work demonstrates that high-quality n-gram language\nmodels can be trained directly on client mobile devices without sensitive\ntraining data ever leaving the devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03476v3",
            "date": "2019-10-04 21:04:20+00:00",
            "title": "Classification As Decoder: Trading Flexibility For Control In Neural Dialogue",
            "authors": [
                "Sam Shleifer",
                "Manish Chablani",
                "Namit Katariya",
                "Anitha Kannan",
                "Xavier Amatriain"
            ],
            "abstract": "Generative seq2seq dialogue systems are trained to predict the next word in\ndialogues that have already occurred. They can learn from large unlabeled\nconversation datasets, build a deep understanding of conversational context,\nand generate a wide variety of responses. This flexibility comes at the cost of\ncontrol. Undesirable responses in the training data will be reproduced by the\nmodel at inference time, and longer generations often don't make sense. Instead\nof generating responses one word at a time, we train a classifier to choose\nfrom a predefined list of full responses. The classifier is trained on\n(conversation context, response class) pairs, where each response class is a\nnoisily labeled group of interchangeable responses. At inference, we generate\nthe exemplar response associated with the predicted response class. Experts can\nedit and improve these exemplar responses over time without retraining the\nclassifier or invalidating old training data. Human evaluation of 775 unseen\ndoctor/patient conversations shows that this tradeoff improves responses. Only\n12% of our discriminative approach's responses are worse than the doctor's\nresponse in the same conversational context, compared to 18% for the generative\nmodel. A discriminative model trained without any manual labeling of response\nclasses achieves equal performance to the generative model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03496v2",
            "date": "2019-09-29 17:45:48+00:00",
            "title": "Fake news detection using Deep Learning",
            "authors": [
                "\u00c1lvaro Ibrain Rodr\u00edguez",
                "Lara Lloret Iglesias"
            ],
            "abstract": "The evolution of the information and communication technologies has\ndramatically increased the number of people with access to the Internet, which\nhas changed the way the information is consumed. As a consequence of the above,\nfake news have become one of the major concerns because its potential to\ndestabilize governments, which makes them a potential danger to modern society.\nAn example of this can be found in the US. electoral campaign, where the term\n\"fake news\" gained great notoriety due to the influence of the hoaxes in the\nfinal result of these. In this work the feasibility of applying deep learning\ntechniques to discriminate fake news on the Internet using only their text is\nstudied. In order to accomplish that, three different neural network\narchitectures are proposed, one of them based on BERT, a modern language model\ncreated by Google which achieves state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03544v4",
            "date": "2019-10-08 17:08:39+00:00",
            "title": "Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking",
            "authors": [
                "Jian-Guo Zhang",
                "Kazuma Hashimoto",
                "Chien-Sheng Wu",
                "Yao Wan",
                "Philip S. Yu",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Dialog state tracking (DST) is a core component in task-oriented dialog\nsystems. Existing approaches for DST mainly fall into one of two categories,\nnamely, ontology-based and ontology-free methods. An ontology-based method\nselects a value from a candidate-value list for each target slot, while an\nontology-free method extracts spans from dialog contexts. Recent work\nintroduced a BERT-based model to strike a balance between the two methods by\npre-defining categorical and non-categorical slots. However, it is not clear\nenough which slots are better handled by either of the two slot types, and the\nway to use the pre-trained model has not been well investigated. In this paper,\nwe propose a simple yet effective dual-strategy model for DST, by adapting a\nsingle BERT-style reading comprehension model to jointly handle both the\ncategorical and non-categorical slots. Our experiments on the MultiWOZ datasets\nshow that our method significantly outperforms the BERT-based counterpart,\nfinding that the key is a deep interaction between the domain-slot and context\ninformation. When evaluated on noisy (MultiWOZ 2.0) and cleaner (MultiWOZ 2.1)\nsettings, our method performs competitively and robustly across the two\ndifferent settings. Our method sets the new state of the art in the noisy\nsetting, while performing more robustly than the best model in the cleaner\nsetting. We also conduct a comprehensive error analysis on the dataset,\nincluding the effects of the dual strategy for each slot, to facilitate future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03756v3",
            "date": "2019-10-09 02:31:37+00:00",
            "title": "Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models",
            "authors": [
                "Qingyang Wu",
                "Yichi Zhang",
                "Yu Li",
                "Zhou Yu"
            ],
            "abstract": "Existing dialog system models require extensive human annotations and are\ndifficult to generalize to different tasks. The recent success of large\npre-trained language models such as BERT and GPT-2 (Devlin et al., 2019;\nRadford et al., 2019) have suggested the effectiveness of incorporating\nlanguage priors in down-stream NLP tasks. However, how much pre-trained\nlanguage models can help dialog response generation is still under exploration.\nIn this paper, we propose a simple, general, and effective framework:\nAlternating Roles Dialog Model (ARDM). ARDM models each speaker separately and\ntakes advantage of the large pre-trained language model. It requires no\nsupervision from human annotations such as belief states or dialog acts to\nachieve effective conversations. ARDM outperforms or is on par with\nstate-of-the-art methods on two popular task-oriented dialog datasets:\nCamRest676 and MultiWOZ. Moreover, we can generalize ARDM to more challenging,\nnon-collaborative tasks such as persuasion. In persuasion tasks, ARDM is\ncapable of generating human-like responses to persuade people to donate to a\ncharity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03806v1",
            "date": "2019-10-09 06:35:59+00:00",
            "title": "Is Multilingual BERT Fluent in Language Generation?",
            "authors": [
                "Samuel R\u00f6nnqvist",
                "Jenna Kanerva",
                "Tapio Salakoski",
                "Filip Ginter"
            ],
            "abstract": "The multilingual BERT model is trained on 104 languages and meant to serve as\na universal language model and tool for encoding sentences. We explore how well\nthe model performs on several languages across several tasks: a diagnostic\nclassification probing the embeddings for a particular syntactic property, a\ncloze task testing the language modelling ability to fill in gaps in a\nsentence, and a natural language generation task testing for the ability to\nproduce coherent text fitting a given context. We find that the currently\navailable multilingual BERT model is clearly inferior to the monolingual\ncounterparts, and cannot in many cases serve as a substitute for a well-trained\nmonolingual model. We find that the English and German models perform well at\ngeneration, whereas the multilingual model is lacking, in particular, for\nNordic languages.",
            "journal ref": "In proceedings of the First NLPL Workshop on Deep Learning for\n  Natural Language Processing (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03814v1",
            "date": "2019-10-09 06:53:39+00:00",
            "title": "Exploring Hate Speech Detection in Multimodal Publications",
            "authors": [
                "Raul Gomez",
                "Jaume Gibert",
                "Lluis Gomez",
                "Dimosthenis Karatzas"
            ],
            "abstract": "In this work we target the problem of hate speech detection in multimodal\npublications formed by a text and an image. We gather and annotate a large\nscale dataset from Twitter, MMHS150K, and propose different models that jointly\nanalyze textual and visual information for hate speech detection, comparing\nthem with unimodal detection. We provide quantitative and qualitative results\nand analyze the challenges of the proposed task. We find that, even though\nimages are useful for the hate speech detection task, current multimodal models\ncannot outperform models analyzing only text. We discuss why and open the field\nand the dataset for further research.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.03833v2",
            "date": "2019-10-09 08:14:48+00:00",
            "title": "Word Embedding Visualization Via Dictionary Learning",
            "authors": [
                "Juexiao Zhang",
                "Yubei Chen",
                "Brian Cheung",
                "Bruno A Olshausen"
            ],
            "abstract": "Co-occurrence statistics based word embedding techniques have proved to be\nvery useful in extracting the semantic and syntactic representation of words as\nlow dimensional continuous vectors. In this work, we discovered that dictionary\nlearning can open up these word vectors as a linear combination of more\nelementary word factors. We demonstrate many of the learned factors have\nsurprisingly strong semantic or syntactic meaning corresponding to the factors\npreviously identified manually by human inspection. Thus dictionary learning\nprovides a powerful visualization tool for understanding word embedding\nrepresentations. Furthermore, we show that the word factors can help in\nidentifying key semantic and syntactic differences in word analogy tasks and\nimprove upon the state-of-the-art word embedding techniques in these tasks by a\nlarge margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03891v2",
            "date": "2019-10-09 10:33:59+00:00",
            "title": "Learning High-order Structural and Attribute information by Knowledge Graph Attention Networks for Enhancing Knowledge Graph Embedding",
            "authors": [
                "Wenqiang Liu",
                "Hongyun Cai",
                "Xu Cheng",
                "Sifa Xie",
                "Yipeng Yu",
                "Hanyu Zhang"
            ],
            "abstract": "The goal of representation learning of knowledge graph is to encode both\nentities and relations into a low-dimensional embedding spaces. Many recent\nworks have demonstrated the benefits of knowledge graph embedding on knowledge\ngraph completion task, such as relation extraction. However, we observe that:\n1) existing method just take direct relations between entities into\nconsideration and fails to express high-order structural relationship between\nentities; 2) these methods just leverage relation triples of KGs while ignoring\na large number of attribute triples that encoding rich semantic information. To\novercome these limitations, this paper propose a novel knowledge graph\nembedding method, named KANE, which is inspired by the recent developments of\ngraph convolutional networks (GCN). KANE can capture both high-order structural\nand attribute information of KGs in an efficient, explicit and unified manner\nunder the graph convolutional networks framework. Empirical results on three\ndatasets show that KANE significantly outperforms seven state-of-arts methods.\nFurther analysis verify the efficiency of our method and the benefits brought\nby the attention mechanism.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.03912v1",
            "date": "2019-10-09 11:45:07+00:00",
            "title": "Novel Applications of Factored Neural Machine Translation",
            "authors": [
                "Patrick Wilken",
                "Evgeny Matusov"
            ],
            "abstract": "In this work, we explore the usefulness of target factors in neural machine\ntranslation (NMT) beyond their original purpose of predicting word lemmas and\ntheir inflections, as proposed by Garc\\`ia-Mart\\`inez et al., 2016. For this,\nwe introduce three novel applications of the factored output architecture: In\nthe first one, we use a factor to explicitly predict the word case separately\nfrom the target word itself. This allows for information to be shared between\ndifferent casing variants of a word. In a second task, we use a factor to\npredict when two consecutive subwords have to be joined, eliminating the need\nfor target subword joining markers. The third task is the prediction of special\ntokens of the operation sequence NMT model (OSNMT) of Stahlberg et al., 2018.\nAutomatic evaluation on English-to-German and English-to-Turkish tasks showed\nthat integration of such auxiliary prediction tasks into NMT is at least as\ngood as the standard NMT approach. For the OSNMT, we observed a significant\nimprovement in BLEU over the baseline OSNMT implementation due to a reduced\noutput sequence length that resulted from the introduction of the target\nfactors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03940v1",
            "date": "2019-10-06 09:21:21+00:00",
            "title": "Measuring Sentences Similarity: A Survey",
            "authors": [
                "Mamdouh Farouk"
            ],
            "abstract": "This study is to review the approaches used for measuring sentences\nsimilarity. Measuring similarity between natural language sentences is a\ncrucial task for many Natural Language Processing applications such as text\nclassification, information retrieval, question answering, and plagiarism\ndetection. This survey classifies approaches of calculating sentences\nsimilarity based on the adopted methodology into three categories. Word-to-word\nbased, structure based, and vector-based are the most widely used approaches to\nfind sentences similarity. Each approach measures relatedness between short\ntexts based on a specific perspective. In addition, datasets that are mostly\nused as benchmarks for evaluating techniques in this field are introduced to\nprovide a complete view on this issue. The approaches that combine more than\none perspective give better results. Moreover, structure based similarity that\nmeasures similarity between sentences structures needs more investigation.",
            "journal ref": "Indian Journal of Science and Technology, Vol 12(25), July 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04269v1",
            "date": "2019-10-09 21:43:36+00:00",
            "title": "Spoken Language Identification using ConvNets",
            "authors": [
                "Sarthak",
                "Shikhar Shukla",
                "Govind Mittal"
            ],
            "abstract": "Language Identification (LI) is an important first step in several speech\nprocessing systems. With a growing number of voice-based assistants, speech LI\nhas emerged as a widely researched field. To approach the problem of\nidentifying languages, we can either adopt an implicit approach where only the\nspeech for a language is present or an explicit one where text is available\nwith its corresponding transcript. This paper focuses on an implicit approach\ndue to the absence of transcriptive data. This paper benchmarks existing models\nand proposes a new attention based model for language identification which uses\nlog-Mel spectrogram images as input. We also present the effectiveness of raw\nwaveforms as features to neural network models for LI tasks. For training and\nevaluation of models, we classified six languages (English, French, German,\nSpanish, Russian and Italian) with an accuracy of 95.4% and four languages\n(English, French, German, Spanish) with an accuracy of 96.3% obtained from the\nVoxForge dataset. This approach can further be scaled to incorporate more\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04289v2",
            "date": "2019-10-09 22:54:43+00:00",
            "title": "Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling",
            "authors": [
                "Ouyu Lan",
                "Xiao Huang",
                "Bill Yuchen Lin",
                "He Jiang",
                "Liyuan Liu",
                "Xiang Ren"
            ],
            "abstract": "Sequence labeling is a fundamental framework for various natural language\nprocessing problems. Its performance is largely influenced by the annotation\nquality and quantity in supervised learning scenarios, and obtaining ground\ntruth labels is often costly. In many cases, ground truth labels do not exist,\nbut noisy annotations or annotations from different domains are accessible. In\nthis paper, we propose a novel framework Consensus Network (ConNet) that can be\ntrained on annotations from multiple sources (e.g., crowd annotation,\ncross-domain data...). It learns individual representation for every source and\ndynamically aggregates source-specific knowledge by a context-aware attention\nmodule. Finally, it leads to a model reflecting the agreement (consensus) among\nmultiple sources. We evaluate the proposed framework in two practical settings\nof multi-source learning: learning with crowd annotations and unsupervised\ncross-domain model adaptation. Extensive experimental results show that our\nmodel achieves significant improvements over existing methods in both settings.\nWe also demonstrate that the method can apply to various tasks and cope with\ndifferent encoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04345v3",
            "date": "2019-10-10 03:06:46+00:00",
            "title": "FUSE: Multi-Faceted Set Expansion by Coherent Clustering of Skip-grams",
            "authors": [
                "Wanzheng Zhu",
                "Hongyu Gong",
                "Jiaming Shen",
                "Chao Zhang",
                "Jingbo Shang",
                "Suma Bhat",
                "Jiawei Han"
            ],
            "abstract": "Set expansion aims to expand a small set of seed entities into a complete set\nof relevant entities. Most existing approaches assume the input seed set is\nunambiguous and completely ignore the multi-faceted semantics of seed entities.\nAs a result, given the seed set {\"Canon\", \"Sony\", \"Nikon\"}, previous models\nreturn one mixed set of entities that are either Camera Brands or Japanese\nCompanies. In this paper, we study the task of multi-faceted set expansion,\nwhich aims to capture all semantic facets in the seed set and return multiple\nsets of entities, one for each semantic facet. We propose an unsupervised\nframework, FUSE, which consists of three major components: (1) facet discovery\nmodule: identifies all semantic facets of each seed entity by extracting and\nclustering its skip-grams, and (2) facet fusion module: discovers shared\nsemantic facets of the entire seed set by an optimization formulation, and (3)\nentity expansion module: expands each semantic facet by utilizing a masked\nlanguage model with pre-trained BERT models. Extensive experiments demonstrate\nthat FUSE can accurately identify multiple semantic facets of the seed set and\ngenerate quality entities for each facet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04708v4",
            "date": "2019-10-10 17:04:30+00:00",
            "title": "Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework",
            "authors": [
                "Zirui Wang",
                "Jiateng Xie",
                "Ruochen Xu",
                "Yiming Yang",
                "Graham Neubig",
                "Jaime Carbonell"
            ],
            "abstract": "Learning multilingual representations of text has proven a successful method\nfor many cross-lingual transfer learning tasks. There are two main paradigms\nfor learning such representations: (1) alignment, which maps different\nindependently trained monolingual representations into a shared space, and (2)\njoint training, which directly learns unified multilingual representations\nusing monolingual and cross-lingual objectives jointly. In this paper, we first\nconduct direct comparisons of representations learned using both of these\nmethods across diverse cross-lingual tasks. Our empirical results reveal a set\nof pros and cons for both methods, and show that the relative performance of\nalignment versus joint training is task-dependent. Stemming from this analysis,\nwe propose a simple and novel framework that combines these two previously\nmutually-exclusive approaches. Extensive experiments demonstrate that our\nproposed framework alleviates limitations of both approaches, and outperforms\nexisting methods on the MUSE bilingual lexicon induction (BLI) benchmark. We\nfurther show that this framework can generalize to contextualized\nrepresentations such as Multilingual BERT, and produces state-of-the-art\nresults on the CoNLL cross-lingual NER benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05014v1",
            "date": "2019-10-11 07:54:38+00:00",
            "title": "Automatic segmentation of texts into units of meaning for reading assistance",
            "authors": [
                "Jean-Claude Houbart",
                "Solen Quiniou",
                "Marion Berthaut",
                "B\u00e9atrice Daille",
                "Claire Salom\u00e9"
            ],
            "abstract": "The emergence of the digital book is a major step forward in providing access\nto reading, and therefore often to the common culture and the labour market. By\nallowing the enrichment of texts with cognitive crutches, EPub 3 compatible\naccessibility formats such as FROG have proven their effectiveness in\nalleviating but also reducing dyslexic disorders. In this paper, we show how\nArtificial Intelligence and particularly Transfer Learning with Google BERT can\nautomate the division into units of meaning, and thus facilitate the creation\nof enriched digital books at a moderate cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05040v1",
            "date": "2019-10-11 09:16:29+00:00",
            "title": "BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels",
            "authors": [
                "Yimin Jing",
                "Deyi Xiong",
                "Yan Zhen"
            ],
            "abstract": "This paper presents BiPaR, a bilingual parallel novel-style machine reading\ncomprehension (MRC) dataset, developed to support multilingual and\ncross-lingual reading comprehension. The biggest difference between BiPaR and\nexisting reading comprehension datasets is that each triple (Passage, Question,\nAnswer) in BiPaR is written parallelly in two languages. We collect 3,667\nbilingual parallel paragraphs from Chinese and English novels, from which we\nconstruct 14,668 parallel question-answer pairs via crowdsourced workers\nfollowing a strict quality control procedure. We analyze BiPaR in depth and\nfind that BiPaR offers good diversification in prefixes of questions, answer\ntypes and relationships between questions and passages. We also observe that\nanswering questions of novels requires reading comprehension skills of\ncoreference resolution, multi-sentence reasoning, and understanding of implicit\ncausality, etc. With BiPaR, we build monolingual, multilingual, and\ncross-lingual MRC baseline models. Even for the relatively simple monolingual\nMRC on this dataset, experiments show that a strong BERT baseline is over 30\npoints behind human in terms of both EM and F1 score, indicating that BiPaR\nprovides a challenging testbed for monolingual, multilingual and cross-lingual\nMRC on novels. The dataset is available at https://multinlp.github.io/BiPaR/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05059v1",
            "date": "2019-10-11 10:03:46+00:00",
            "title": "Keyphrase Generation: A Multi-Aspect Survey",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Extractive keyphrase generation research has been around since the nineties,\nbut the more advanced abstractive approach based on the encoder-decoder\nframework and sequence-to-sequence learning has been explored only recently. In\nfact, more than a dozen of abstractive methods have been proposed in the last\nthree years, producing meaningful keyphrases and achieving state-of-the-art\nscores. In this survey, we examine various aspects of the extractive keyphrase\ngeneration methods and focus mostly on the more recent abstractive methods that\nare based on neural networks. We pay particular attention to the mechanisms\nthat have driven the perfection of the later. A huge collection of scientific\narticle metadata and the corresponding keyphrases is created and released for\nthe research community. We also present various keyphrase generation and text\nsummarization research patterns and trends of the last two decades.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05276v1",
            "date": "2019-10-11 16:10:55+00:00",
            "title": "exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models",
            "authors": [
                "Benjamin Hoover",
                "Hendrik Strobelt",
                "Sebastian Gehrmann"
            ],
            "abstract": "Large language models can produce powerful contextual representations that\nlead to improvements across many NLP tasks. Since these models are typically\nguided by a sequence of learned self attention mechanisms and may comprise\nundesired inductive biases, it is paramount to be able to explore what the\nattention has learned. While static analyses of these models lead to targeted\ninsights, interactive tools are more dynamic and can help humans better gain an\nintuition for the model-internal reasoning process. We present exBERT, an\ninteractive tool named after the popular BERT language model, that provides\ninsights into the meaning of the contextual representations by matching a\nhuman-specified input to similar contexts in a large annotated dataset. By\naggregating the annotations of the matching similar contexts, exBERT helps\nintuitively explain what each attention-head has learned.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05315v1",
            "date": "2019-10-11 17:22:19+00:00",
            "title": "Learning Analogy-Preserving Sentence Embeddings for Answer Selection",
            "authors": [
                "Aissatou Diallo",
                "Markus Zopf",
                "Johannes F\u00fcrnkranz"
            ],
            "abstract": "Answer selection aims at identifying the correct answer for a given question\nfrom a set of potentially correct answers. Contrary to previous works, which\ntypically focus on the semantic similarity between a question and its answer,\nour hypothesis is that question-answer pairs are often in analogical relation\nto each other. Using analogical inference as our use case, we propose a\nframework and a neural network architecture for learning dedicated sentence\nembeddings that preserve analogical properties in the semantic space. We\nevaluate the proposed method on benchmark datasets for answer selection and\ndemonstrate that our sentence embeddings indeed capture analogical properties\nbetter than conventional embeddings, and that analogy-based question answering\noutperforms a comparable similarity-based technique.",
            "journal ref": "Proc. CoNLL 2019: 910-919",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05389v1",
            "date": "2019-10-11 19:56:47+00:00",
            "title": "Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study",
            "authors": [
                "Ziyu Yao",
                "Yu Su",
                "Huan Sun",
                "Wen-tau Yih"
            ],
            "abstract": "As a promising paradigm, interactive semantic parsing has shown to improve\nboth semantic parsing accuracy and user confidence in the results. In this\npaper, we propose a new, unified formulation of the interactive semantic\nparsing problem, where the goal is to design a model-based intelligent agent.\nThe agent maintains its own state as the current predicted semantic parse,\ndecides whether and where human intervention is needed, and generates a\nclarification question in natural language. A key part of the agent is a world\nmodel: it takes a percept (either an initial question or subsequent feedback\nfrom the user) and transitions to a new state. We then propose a simple yet\nremarkably effective instantiation of our framework, demonstrated on two\ntext-to-SQL datasets (WikiSQL and Spider) with different state-of-the-art base\nsemantic parsers. Compared to an existing interactive semantic parsing approach\nthat treats the base parser as a black box, our approach solicits less user\nfeedback but yields higher run-time accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05453v3",
            "date": "2019-10-12 00:55:06+00:00",
            "title": "vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations",
            "authors": [
                "Alexei Baevski",
                "Steffen Schneider",
                "Michael Auli"
            ],
            "abstract": "We propose vq-wav2vec to learn discrete representations of audio segments\nthrough a wav2vec-style self-supervised context prediction task. The algorithm\nuses either a gumbel softmax or online k-means clustering to quantize the dense\nrepresentations. Discretization enables the direct application of algorithms\nfrom the NLP community which require discrete inputs. Experiments show that\nBERT pre-training achieves a new state of the art on TIMIT phoneme\nclassification and WSJ speech recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05577v4",
            "date": "2019-10-12 15:30:18+00:00",
            "title": "Context-Gated Convolution",
            "authors": [
                "Xudong Lin",
                "Lin Ma",
                "Wei Liu",
                "Shih-Fu Chang"
            ],
            "abstract": "As the basic building block of Convolutional Neural Networks (CNNs), the\nconvolutional layer is designed to extract local patterns and lacks the ability\nto model global context in its nature. Many efforts have been recently devoted\nto complementing CNNs with the global modeling ability, especially by a family\nof works on global feature interaction. In these works, the global context\ninformation is incorporated into local features before they are fed into\nconvolutional layers. However, research on neuroscience reveals that the\nneurons' ability of modifying their functions dynamically according to context\nis essential for the perceptual tasks, which has been overlooked in most of\nCNNs. Motivated by this, we propose one novel Context-Gated Convolution (CGC)\nto explicitly modify the weights of convolutional layers adaptively under the\nguidance of global context. As such, being aware of the global context, the\nmodulated convolution kernel of our proposed CGC can better extract\nrepresentative local patterns and compose discriminative features. Moreover,\nour proposed CGC is lightweight and applicable with modern CNN architectures,\nand consistently improves the performance of CNNs according to extensive\nexperiments on image classification, action recognition, and machine\ntranslation. Our code of this paper is available at\nhttps://github.com/XudongLinthu/context-gated-convolution.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.05915v1",
            "date": "2019-10-14 04:48:16+00:00",
            "title": "Knowledge-guided Unsupervised Rhetorical Parsing for Text Summarization",
            "authors": [
                "Shengluan Hou",
                "Ruqian Lu"
            ],
            "abstract": "Automatic text summarization (ATS) has recently achieved impressive\nperformance thanks to recent advances in deep learning and the availability of\nlarge-scale corpora. To make the summarization results more faithful, this\npaper presents an unsupervised approach that combines rhetorical structure\ntheory, deep neural model and domain knowledge concern for ATS. This\narchitecture mainly contains three components: domain knowledge base\nconstruction based on representation learning, attentional encoder-decoder\nmodel for rhetorical parsing and subroutine-based model for text summarization.\nDomain knowledge can be effectively used for unsupervised rhetorical parsing\nthus rhetorical structure trees for each document can be derived. In the\nunsupervised rhetorical parsing module, the idea of translation was adopted to\nalleviate the problem of data scarcity. The subroutine-based summarization\nmodel purely depends on the derived rhetorical structure trees and can generate\ncontent-balanced results. To evaluate the summary results without golden\nstandard, we proposed an unsupervised evaluation metric, whose hyper-parameters\nwere tuned by supervised learning. Experimental results show that, on a\nlarge-scale Chinese dataset, our proposed approach can obtain comparable\nperformances compared with existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06036v2",
            "date": "2019-10-14 11:03:55+00:00",
            "title": "Improving Question Generation With to the Point Context",
            "authors": [
                "Jingjing Li",
                "Yifan Gao",
                "Lidong Bing",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "Question generation (QG) is the task of generating a question from a\nreference sentence and a specified answer within the sentence. A major\nchallenge in QG is to identify answer-relevant context words to finish the\ndeclarative-to-interrogative sentence transformation. Existing\nsequence-to-sequence neural models achieve this goal by proximity-based answer\nposition encoding under the intuition that neighboring words of answers are of\nhigh possibility to be answer-relevant. However, such intuition may not apply\nto all cases especially for sentences with complex answer-relevant relations.\nConsequently, the performance of these models drops sharply when the relative\ndistance between the answer fragment and other non-stop sentence words that\nalso appear in the ground truth question increases. To address this issue, we\npropose a method to jointly model the unstructured sentence and the structured\nanswer-relevant relation (extracted from the sentence in advance) for question\ngeneration. Specifically, the structured answer-relevant relation acts as the\nto the point context and it thus naturally helps keep the generated question to\nthe point, while the unstructured sentence provides the full information.\nExtensive experiments show that to the point context helps our question\ngeneration model achieve significant improvements on several automatic\nevaluation metrics. Furthermore, our model is capable of generating diverse\nquestions for a sentence which conveys multiple relations of its answer\nfragment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06061v2",
            "date": "2019-10-14 12:03:07+00:00",
            "title": "Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labels",
            "authors": [
                "Lukas Lange",
                "Michael A. Hedderich",
                "Dietrich Klakow"
            ],
            "abstract": "In low-resource settings, the performance of supervised labeling models can\nbe improved with automatically annotated or distantly supervised data, which is\ncheap to create but often noisy. Previous works have shown that significant\nimprovements can be reached by injecting information about the confusion\nbetween clean and noisy labels in this additional training data into the\nclassifier training. However, for noise estimation, these approaches either do\nnot take the input features (in our case word embeddings) into account, or they\nneed to learn the noise modeling from scratch which can be difficult in a\nlow-resource setting. We propose to cluster the training data using the input\nfeatures and then compute different confusion matrices for each cluster. To the\nbest of our knowledge, our approach is the first to leverage feature-dependent\nnoise modeling with pre-initialized confusion matrices. We evaluate on\nlow-resource named entity recognition settings in several languages, showing\nthat our methods improve upon other confusion-matrix based methods by up to 9%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06188v2",
            "date": "2019-10-14 14:55:19+00:00",
            "title": "Q8BERT: Quantized 8Bit BERT",
            "authors": [
                "Ofir Zafrir",
                "Guy Boudoukh",
                "Peter Izsak",
                "Moshe Wasserblat"
            ],
            "abstract": "Recently, pre-trained Transformer based language models such as BERT and GPT,\nhave shown great improvement in many Natural Language Processing (NLP) tasks.\nHowever, these models contain a large amount of parameters. The emergence of\neven larger and more accurate models such as GPT2 and Megatron, suggest a trend\nof large pre-trained Transformer models. However, using these large models in\nproduction environments is a complex task requiring a large amount of compute,\nmemory and power resources. In this work we show how to perform\nquantization-aware training during the fine-tuning phase of BERT in order to\ncompress BERT by $4\\times$ with minimal accuracy loss. Furthermore, the\nproduced quantized model can accelerate inference speed if it is optimized for\n8bit Integer supporting hardware.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06262v1",
            "date": "2019-10-14 16:43:00+00:00",
            "title": "Restoring ancient text using deep learning: a case study on Greek epigraphy",
            "authors": [
                "Yannis Assael",
                "Thea Sommerschield",
                "Jonathan Prag"
            ],
            "abstract": "Ancient history relies on disciplines such as epigraphy, the study of ancient\ninscribed texts, for evidence of the recorded past. However, these texts,\n\"inscriptions\", are often damaged over the centuries, and illegible parts of\nthe text must be restored by specialists, known as epigraphists. This work\npresents Pythia, the first ancient text restoration model that recovers missing\ncharacters from a damaged text input using deep neural networks. Its\narchitecture is carefully designed to handle long-term context information, and\ndeal efficiently with missing or corrupted character and word representations.\nTo train it, we wrote a non-trivial pipeline to convert PHI, the largest\ndigital corpus of ancient Greek inscriptions, to machine actionable text, which\nwe call PHI-ML. On PHI-ML, Pythia's predictions achieve a 30.1% character error\nrate, compared to the 57.3% of human epigraphists. Moreover, in 73.5% of cases\nthe ground-truth sequence was among the Top-20 hypotheses of Pythia, which\neffectively demonstrates the impact of this assistive method on the field of\ndigital epigraphy, and sets the state-of-the-art in ancient text restoration.",
            "journal ref": "Empirical Methods in Natural Language Processing (EMNLP) 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06294v2",
            "date": "2019-10-14 17:22:37+00:00",
            "title": "Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models",
            "authors": [
                "Peter Izsak",
                "Shira Guskin",
                "Moshe Wasserblat"
            ],
            "abstract": "Training models on low-resource named entity recognition tasks has been shown\nto be a challenge, especially in industrial applications where deploying\nupdated models is a continuous effort and crucial for business operations. In\nsuch cases there is often an abundance of unlabeled data, while labeled data is\nscarce or unavailable. Pre-trained language models trained to extract\ncontextual features from text were shown to improve many natural language\nprocessing (NLP) tasks, including scarcely labeled tasks, by leveraging\ntransfer learning. However, such models impose a heavy memory and computational\nburden, making it a challenge to train and deploy such models for inference\nuse. In this work-in-progress we combined the effectiveness of transfer\nlearning provided by pre-trained masked language models with a semi-supervised\napproach to train a fast and compact model using labeled and unlabeled\nexamples. Preliminary evaluations show that the compact models can achieve\ncompetitive accuracy with 36x compression rate when compared with a\nstate-of-the-art pre-trained language model, and run significantly faster in\ninference, allowing deployment of such models in production environments or on\nedge devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06360v3",
            "date": "2019-10-14 18:12:30+00:00",
            "title": "Structured Pruning of a BERT-based Question Answering Model",
            "authors": [
                "J. S. McCarley",
                "Rishav Chakravarti",
                "Avirup Sil"
            ],
            "abstract": "The recent trend in industry-setting Natural Language Processing (NLP)\nresearch has been to operate large %scale pretrained language models like BERT\nunder strict computational limits. While most model compression work has\nfocused on \"distilling\" a general-purpose language representation using\nexpensive pretraining distillation, less attention has been paid to creating\nsmaller task-specific language representations which, arguably, are more useful\nin an industry setting. In this paper, we investigate compressing BERT- and\nRoBERTa-based question answering systems by structured pruning of parameters\nfrom the underlying transformer model. We find that an inexpensive combination\nof task-specific structured pruning and task-specific distillation, without the\nexpense of pretraining distillation, yields highly-performing models across a\nrange of speed/accuracy tradeoff operating points. We start from existing\nfull-size models trained for SQuAD 2.0 or Natural Questions and introduce gates\nthat allow selected parts of transformers to be individually eliminated.\nSpecifically, we investigate (1) structured pruning to reduce the number of\nparameters in each transformer layer, (2) applicability to both BERT- and\nRoBERTa-based models, (3) applicability to both SQuAD 2.0 and Natural\nQuestions, and (4) combining structured pruning with distillation. We achieve a\nnear-doubling of inference speed with less than a 0.5 F1-point loss in short\nanswer accuracy on Natural Questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06393v2",
            "date": "2019-09-27 08:58:55+00:00",
            "title": "In-training Matrix Factorization for Parameter-frugal Neural Machine Translation",
            "authors": [
                "Zachary Kaden",
                "Teven Le Scao",
                "Raphael Olivier"
            ],
            "abstract": "In this paper, we propose the use of in-training matrix factorization to\nreduce the model size for neural machine translation. Using in-training matrix\nfactorization, parameter matrices may be decomposed into the products of\nsmaller matrices, which can compress large machine translation architectures by\nvastly reducing the number of learnable parameters. We apply in-training matrix\nfactorization to different layers of standard neural architectures and show\nthat in-training factorization is capable of reducing nearly 50% of learnable\nparameters without any associated loss in BLEU score. Further, we find that\nin-training matrix factorization is especially powerful on embedding layers,\nproviding a simple and effective method to curtail the number of parameters\nwith minimal impact on model performance, and, at times, an increase in\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06426v1",
            "date": "2019-10-14 21:21:03+00:00",
            "title": "Tell-the-difference: Fine-grained Visual Descriptor via a Discriminating Referee",
            "authors": [
                "Shuangjie Xu",
                "Feng Xu",
                "Yu Cheng",
                "Pan Zhou"
            ],
            "abstract": "In this paper, we investigate a novel problem of telling the difference\nbetween image pairs in natural language. Compared to previous approaches for\nsingle image captioning, it is challenging to fetch linguistic representation\nfrom two independent visual information. To this end, we have proposed an\neffective encoder-decoder caption framework based on Hyper Convolution Net. In\naddition, a series of novel feature fusing techniques for pairwise visual\ninformation fusing are introduced and a discriminating referee is proposed to\nevaluate the pipeline. Because of the lack of appropriate datasets to support\nthis task, we have collected and annotated a large new dataset with Amazon\nMechanical Turk (AMT) for generating captions in a pairwise manner (with 14764\nimages and 26710 image pairs in total). The dataset is the first one on the\nrelative difference caption task that provides descriptions in free language.\nWe evaluate the effectiveness of our model on two datasets in the field and it\noutperforms the state-of-the-art approach by a large margin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.06492v1",
            "date": "2019-10-15 02:40:29+00:00",
            "title": "Hierarchical Semantic Correspondence Learning for Post-Discharge Patient Mortality Prediction",
            "authors": [
                "Shaika Chowdhury",
                "Chenwei Zhang",
                "Philip S. Yu",
                "Yuan Luo"
            ],
            "abstract": "Predicting patient mortality is an important and challenging problem in the\nhealthcare domain, especially for intensive care unit (ICU) patients.\nElectronic health notes serve as a rich source for learning patient\nrepresentations, that can facilitate effective risk assessment. However, a\nlarge portion of clinical notes are unstructured and also contain domain\nspecific terminologies, from which we need to extract structured information.\nIn this paper, we introduce an embedding framework to learn\nsemantically-plausible distributed representations of clinical notes that\nexploits the semantic correspondence between the unstructured texts and their\ncorresponding structured knowledge, known as semantic frame, in a hierarchical\nfashion. Our approach integrates text modeling and semantic correspondence\nlearning into a single model that comprises 1) an unstructured embedding module\nthat makes use of self-similarity matrix representations in order to inject\nstructural regularities of different segments inherent in clinical texts to\npromote local coherence, 2) a structured embedding module to embed the semantic\nframes (e.g., UMLS semantic types) with deep ConvNet and 3) a hierarchical\nsemantic correspondence module that embeds by enhancing the interactions\nbetween text-semantic frame embedding pairs at multiple levels (i.e., words,\nsentence, note). Evaluations on multiple embedding benchmarks on post discharge\nintensive care patient mortality prediction tasks demonstrate its effectiveness\ncompared to approaches that do not exploit the semantic interactions between\nstructured and unstructured information present in clinical notes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06592v1",
            "date": "2019-10-15 08:43:47+00:00",
            "title": "FacTweet: Profiling Fake News Twitter Accounts",
            "authors": [
                "Bilal Ghanem",
                "Simone Paolo Ponzetto",
                "Paolo Rosso"
            ],
            "abstract": "We present an approach to detect fake news in Twitter at the account level\nusing a neural recurrent model and a variety of different semantic and\nstylistic features. Our method extracts a set of features from the timelines of\nnews Twitter accounts by reading their posts as chunks, rather than dealing\nwith each tweet independently. We show the experimental benefits of modeling\nlatent stylistic signatures of mixed fake and real news with a sequential model\nover a wide range of strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06710v1",
            "date": "2019-10-08 17:28:16+00:00",
            "title": "Knowledge-based Biomedical Data Science 2019",
            "authors": [
                "Tiffany J. Callahan",
                "Harrison Pielke-Lombardo",
                "Ignacio J. Tripodi",
                "Lawrence E. Hunter"
            ],
            "abstract": "Knowledge-based biomedical data science (KBDS) involves the design and\nimplementation of computer systems that act as if they knew about biomedicine.\nSuch systems depend on formally represented knowledge in computer systems,\noften in the form of knowledge graphs. Here we survey the progress in the last\nyear in systems that use formally represented knowledge to address data science\nproblems in both clinical and biological domains, as well as on approaches for\ncreating knowledge graphs. Major themes include the relationships between\nknowledge graphs and machine learning, the use of natural language processing,\nand the expansion of knowledge-based approaches to novel domains, such as\nChinese Traditional Medicine and biodiversity.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.06719v1",
            "date": "2019-10-02 14:27:11+00:00",
            "title": "Tree-Structured Semantic Encoder with Knowledge Sharing for Domain Adaptation in Natural Language Generation",
            "authors": [
                "Bo-Hsiang Tseng",
                "Pawe\u0142 Budzianowski",
                "Yen-Chen Wu",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Domain adaptation in natural language generation (NLG) remains challenging\nbecause of the high complexity of input semantics across domains and limited\ndata of a target domain. This is particularly the case for dialogue systems,\nwhere we want to be able to seamlessly include new domains into the\nconversation. Therefore, it is crucial for generation models to share knowledge\nacross domains for the effective adaptation from one domain to another. In this\nstudy, we exploit a tree-structured semantic encoder to capture the internal\nstructure of complex semantic representations required for multi-domain\ndialogues in order to facilitate knowledge sharing across domains. In addition,\na layer-wise attention mechanism between the tree encoder and the decoder is\nadopted to further improve the model's capability. The automatic evaluation\nresults show that our model outperforms previous methods in terms of the BLEU\nscore and the slot error rate, in particular when the adaptation data is\nlimited. In subjective evaluation, human judges tend to prefer the sentences\ngenerated by our model, rating them more highly on informativeness and\nnaturalness than other systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06720v2",
            "date": "2019-10-02 16:40:03+00:00",
            "title": "Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition",
            "authors": [
                "Vasileios Lioutas",
                "Ahmad Rashid",
                "Krtin Kumar",
                "Md Akmal Haidar",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "Word-embeddings are vital components of Natural Language Processing (NLP)\nmodels and have been extensively explored. However, they consume a lot of\nmemory which poses a challenge for edge deployment. Embedding matrices,\ntypically, contain most of the parameters for language models and about a third\nfor machine translation systems. In this paper, we propose Distilled Embedding,\nan (input/output) embedding compression method based on low-rank matrix\ndecomposition and knowledge distillation. First, we initialize the weights of\nour decomposed matrices by learning to reconstruct the full pre-trained\nword-embedding and then fine-tune end-to-end, employing knowledge distillation\non the factorized embedding. We conduct extensive experiments with various\ncompression rates on machine translation and language modeling, using different\ndata-sets with a shared word-embedding matrix for both embedding and vocabulary\nprojection matrices. We show that the proposed technique is simple to\nreplicate, with one fixed parameter controlling compression size, has higher\nBLEU score on translation and lower perplexity on language modeling compared to\ncomplex, difficult to tune state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06748v1",
            "date": "2019-10-15 13:51:24+00:00",
            "title": "Language Identification on Massive Datasets of Short Message using an Attention Mechanism CNN",
            "authors": [
                "Duy Tin Vo",
                "Richard Khoury"
            ],
            "abstract": "Language Identification (LID) is a challenging task, especially when the\ninput texts are short and noisy such as posts and statuses on social media or\nchat logs on gaming forums. The task has been tackled by either designing a\nfeature set for a traditional classifier (e.g. Naive Bayes) or applying a deep\nneural network classifier (e.g. Bi-directional Gated Recurrent Unit,\nEncoder-Decoder). These methods are usually trained and tested on a huge amount\nof private data, then used and evaluated as off-the-shelf packages by other\nresearchers using their own datasets, and consequently the various results\npublished are not directly comparable. In this paper, we first create a new\nmassive labelled dataset based on one year of Twitter data. We use this dataset\nto test several existing language identification systems, in order to obtain a\nset of coherent benchmarks, and we make our dataset publicly available so that\nothers can add to this set of benchmarks. Finally, we propose a shallow but\nefficient neural LID system, which is a ngram-regional convolution neural\nnetwork enhanced with an attention mechanism. Experimental results show that\nour architecture is able to predict tens of thousands of samples per second and\nsurpasses all state-of-the-art systems with an improvement of 5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07204v1",
            "date": "2019-10-16 08:04:07+00:00",
            "title": "Transformer ASR with Contextual Block Processing",
            "authors": [
                "Emiru Tsunoo",
                "Yosuke Kashiwagi",
                "Toshiyuki Kumakura",
                "Shinji Watanabe"
            ],
            "abstract": "The Transformer self-attention network has recently shown promising\nperformance as an alternative to recurrent neural networks (RNNs) in end-to-end\n(E2E) automatic speech recognition (ASR) systems. However, the Transformer has\na drawback in that the entire input sequence is required to compute\nself-attention. In this paper, we propose a new block processing method for the\nTransformer encoder by introducing a context-aware inheritance mechanism. An\nadditional context embedding vector handed over from the previously processed\nblock helps to encode not only local acoustic information but also global\nlinguistic, channel, and speaker attributes. We introduce a novel mask\ntechnique to implement the context inheritance to train the model efficiently.\nEvaluations of the Wall Street Journal (WSJ), Librispeech, VoxForge Italian,\nand AISHELL-1 Mandarin speech recognition datasets show that our proposed\ncontextual block processing method outperforms naive block processing\nconsistently. Furthermore, the attention weight tendency of each layer is\nanalyzed to clarify how the added contextual inheritance mechanism models the\nglobal information.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.07333v2",
            "date": "2019-10-16 13:24:54+00:00",
            "title": "A Probabilistic Framework for Learning Domain Specific Hierarchical Word Embeddings",
            "authors": [
                "Lahari Poddar",
                "Gyorgy Szarvas",
                "Lea Frermann"
            ],
            "abstract": "The meaning of a word often varies depending on its usage in different\ndomains. The standard word embedding models struggle to represent this\nvariation, as they learn a single global representation for a word. We propose\na method to learn domain-specific word embeddings, from text organized into\nhierarchical domains, such as reviews in an e-commerce website, where products\nfollow a taxonomy. Our structured probabilistic model allows vector\nrepresentations for the same word to drift away from each other for distant\ndomains in the taxonomy, to accommodate its domain-specific meanings. By\nlearning sets of domain-specific word representations jointly, our model can\nleverage domain relationships, and it scales well with the number of domains.\nUsing large real-world review datasets, we demonstrate the effectiveness of our\nmodel compared to state-of-the-art approaches, in learning domain-specific word\nembeddings that are both intuitive to humans and benefit downstream NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07482v1",
            "date": "2019-10-16 17:22:25+00:00",
            "title": "Imperial College London Submission to VATEX Video Captioning Task",
            "authors": [
                "Ozan Caglayan",
                "Zixiu Wu",
                "Pranava Madhyastha",
                "Josiah Wang",
                "Lucia Specia"
            ],
            "abstract": "This paper describes the Imperial College London team's submission to the\n2019' VATEX video captioning challenge, where we first explore two\nsequence-to-sequence models, namely a recurrent (GRU) model and a transformer\nmodel, which generate captions from the I3D action features. We then\ninvestigate the effect of dropping the encoder and the attention mechanism and\ninstead conditioning the GRU decoder over two different vectorial\nrepresentations: (i) a max-pooled action feature vector and (ii) the output of\na multi-label classifier trained to predict visual entities from the action\nfeatures. Our baselines achieved scores comparable to the official baseline.\nConditioning over entity predictions performed substantially better than\nconditioning on the max-pooled feature vector, and only marginally worse than\nthe GRU-based sequence-to-sequence baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07518v1",
            "date": "2019-10-16 00:54:17+00:00",
            "title": "Right-wing German Hate Speech on Twitter: Analysis and Automatic Detection",
            "authors": [
                "Sylvia Jaki",
                "Tom De Smedt"
            ],
            "abstract": "Discussion about the social network Twitter often concerns its role in\npolitical discourse, involving the question of when an expression of opinion\nbecomes offensive, immoral, and/or illegal, and how to deal with it. Given the\ngrowing amount of offensive communication on the internet, there is a demand\nfor new technology that can automatically detect hate speech, to assist content\nmoderation by humans. This comes with new challenges, such as defining exactly\nwhat is free speech and what is illegal in a specific country, and knowing\nexactly what the linguistic characteristics of hate speech are. To shed light\non the German situation, we analyzed over 50,000 right-wing German hate tweets\nposted between August 2017 and April 2018, at the time of the 2017 German\nfederal elections, using both quantitative and qualitative methods. In this\npaper, we discuss the results of the analysis and demonstrate how the insights\ncan be employed for the development of automatic detection systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07973v2",
            "date": "2019-10-17 15:33:26+00:00",
            "title": "Universal Text Representation from BERT: An Empirical Study",
            "authors": [
                "Xiaofei Ma",
                "Zhiguo Wang",
                "Patrick Ng",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "We present a systematic investigation of layer-wise BERT activations for\ngeneral-purpose text representations to understand what linguistic information\nthey capture and how transferable they are across different tasks.\nSentence-level embeddings are evaluated against two state-of-the-art models on\ndownstream and probing tasks from SentEval, while passage-level embeddings are\nevaluated on four question-answering (QA) datasets under a learning-to-rank\nproblem setting. Embeddings from the pre-trained BERT model perform poorly in\nsemantic similarity and sentence surface information probing tasks. Fine-tuning\nBERT on natural language inference data greatly improves the quality of the\nembeddings. Combining embeddings from different BERT layers can further boost\nperformance. BERT embeddings outperform BM25 baseline significantly on factoid\nQA datasets at the passage level, but fail to perform better than BM25 on\nnon-factoid datasets. For all QA datasets, there is a gap between\nembedding-based method and in-domain fine-tuned BERT (we report new\nstate-of-the-art results on two datasets), which suggests deep interactions\nbetween question and answer pairs are critical for those hard tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08194v1",
            "date": "2019-10-17 23:02:34+00:00",
            "title": "HiExpan: Task-Guided Taxonomy Construction by Hierarchical Tree Expansion",
            "authors": [
                "Jiaming Shen",
                "Zeqiu Wu",
                "Dongming Lei",
                "Chao Zhang",
                "Xiang Ren",
                "Michelle T. Vanni",
                "Brian M. Sadler",
                "Jiawei Han"
            ],
            "abstract": "Taxonomies are of great value to many knowledge-rich applications. As the\nmanual taxonomy curation costs enormous human effects, automatic taxonomy\nconstruction is in great demand. However, most existing automatic taxonomy\nconstruction methods can only build hypernymy taxonomies wherein each edge is\nlimited to expressing the \"is-a\" relation. Such a restriction limits their\napplicability to more diverse real-world tasks where the parent-child may carry\ndifferent relations. In this paper, we aim to construct a task-guided taxonomy\nfrom a domain-specific corpus and allow users to input a \"seed\" taxonomy,\nserving as the task guidance. We propose an expansion-based taxonomy\nconstruction framework, namely HiExpan, which automatically generates key term\nlist from the corpus and iteratively grows the seed taxonomy. Specifically,\nHiExpan views all children under each taxonomy node forming a coherent set and\nbuilds the taxonomy by recursively expanding all these sets. Furthermore,\nHiExpan incorporates a weakly-supervised relation extraction module to extract\nthe initial children of a newly-expanded node and adjusts the taxonomy tree by\noptimizing its global structure. Our experiments on three real datasets from\ndifferent domains demonstrate the effectiveness of HiExpan for building\ntask-guided taxonomies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08249v1",
            "date": "2019-10-18 03:54:58+00:00",
            "title": "Relational Graph Representation Learning for Open-Domain Question Answering",
            "authors": [
                "Salvatore Vivona",
                "Kaveh Hassani"
            ],
            "abstract": "We introduce a relational graph neural network with bi-directional attention\nmechanism and hierarchical representation learning for open-domain question\nanswering task. Our model can learn contextual representation by jointly\nlearning and updating the query, knowledge graph, and document representations.\nThe experiments suggest that our model achieves state-of-the-art on the\nWebQuestionsSP benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08252v2",
            "date": "2019-10-18 04:02:33+00:00",
            "title": "Entity Summarization: State of the Art and Future Challenges",
            "authors": [
                "Qingxia Liu",
                "Gong Cheng",
                "Kalpa Gunaratna",
                "Yuzhong Qu"
            ],
            "abstract": "The increasing availability of semantic data has substantially enhanced Web\napplications. Semantic data such as RDF data is commonly represented as\nentity-property-value triples. The magnitude of semantic data, in particular\nthe large number of triples describing an entity, could overload users with\nexcessive amounts of information. This has motivated fruitful research on\nautomated generation of summaries for entity descriptions to satisfy users'\ninformation needs efficiently and effectively. We focus on this prominent topic\nof entity summarization, and our research objective is to present the first\ncomprehensive survey of entity summarization research. Rather than separately\nreviewing each method, our contributions include (1) identifying and\nclassifying technical features of existing methods to form a high-level\noverview, (2) identifying and classifying frameworks for combining multiple\ntechnical features adopted by existing methods, (3) collecting known benchmarks\nfor intrinsic evaluation and efforts for extrinsic evaluation, and (4)\nsuggesting research directions for future work. By investigating the\nliterature, we synthesized two hierarchies of techniques. The first hierarchy\ncategories generic technical features into several perspectives: frequency and\ncentrality, informativeness, and diversity and coverage. In the second\nhierarchy we present domain-specific and task-specific technical features,\nincluding the use of domain knowledge, context awareness, and personalization.\nOur review demonstrated that existing methods are mainly unsupervised and they\ncombine multiple technical features using various frameworks: random surfer\nmodels, similarity-based grouping, MMR-like re-ranking, or combinatorial\noptimization. We also found a few deep learning based methods in recent\nresearch.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.08294v1",
            "date": "2019-10-18 07:56:08+00:00",
            "title": "Towards Computing Inferences from English News Headlines",
            "authors": [
                "Elizabeth Jasmi George",
                "Radhika Mamidi"
            ],
            "abstract": "Newspapers are a popular form of written discourse, read by many people,\nthanks to the novelty of the information provided by the news content in it. A\nheadline is the most widely read part of any newspaper due to its appearance in\na bigger font and sometimes in colour print. In this paper, we suggest and\nimplement a method for computing inferences from English news headlines,\nexcluding the information from the context in which the headlines appear. This\nmethod attempts to generate the possible assumptions a reader formulates in\nmind upon reading a fresh headline. The generated inferences could be useful\nfor assessing the impact of the news headline on readers including children.\nThe understandability of the current state of social affairs depends greatly on\nthe assimilation of the headlines. As the inferences that are independent of\nthe context depend mainly on the syntax of the headline, dependency trees of\nheadlines are used in this approach, to find the syntactical structure of the\nheadlines and to compute inferences out of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08350v2",
            "date": "2019-10-18 11:47:24+00:00",
            "title": "A Mutual Information Maximization Perspective of Language Representation Learning",
            "authors": [
                "Lingpeng Kong",
                "Cyprien de Masson d'Autume",
                "Wang Ling",
                "Lei Yu",
                "Zihang Dai",
                "Dani Yogatama"
            ],
            "abstract": "We show state-of-the-art word representation learning methods maximize an\nobjective function that is a lower bound on the mutual information between\ndifferent parts of a word sequence (i.e., a sentence). Our formulation provides\nan alternative perspective that unifies classical word embedding models (e.g.,\nSkip-gram) and modern contextual embeddings (e.g., BERT, XLNet). In addition to\nenhancing our theoretical understanding of these methods, our derivation leads\nto a principled framework that can be used to construct new self-supervised\ntasks. We provide an example by drawing inspirations from related methods based\non mutual information maximization that have been successful in computer\nvision, and introduce a simple self-supervised objective that maximizes the\nmutual information between a global sentence representation and n-grams in the\nsentence. Our analysis offers a holistic view of representation learning\nmethods to transfer knowledge and translate progress across multiple domains\n(e.g., natural language processing, computer vision, audio processing).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08502v2",
            "date": "2019-10-18 16:52:01+00:00",
            "title": "End-to-End Speech Recognition: A review for the French Language",
            "authors": [
                "Florian Boyer",
                "Jean-Luc Rouas"
            ],
            "abstract": "Recently, end-to-end ASR based either on sequence-to-sequence networks or on\nthe CTC objective function gained a lot of interest from the community,\nachieving competitive results over traditional systems using robust but complex\npipelines. One of the main features of end-to-end systems, in addition to the\nability to free themselves from extra linguistic resources such as dictionaries\nor language models, is the capacity to model acoustic units such as characters,\nsubwords or directly words; opening up the capacity to directly translate\nspeech with different representations or levels of knowledge depending on the\ntarget language. In this paper we propose a review of the existing end-to-end\nASR approaches for the French language. We compare results to conventional\nstate-of-the-art ASR systems and discuss which units are more suited to model\nthe French language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08692v1",
            "date": "2019-10-19 03:32:16+00:00",
            "title": "An Improved Historical Embedding without Alignment",
            "authors": [
                "Xiaofei Xu",
                "Ke Deng",
                "Fei Hu",
                "Li Li"
            ],
            "abstract": "Many words have evolved in meaning as a result of cultural and social change.\nUnderstanding such changes is crucial for modelling language and cultural\nevolution. Low-dimensional embedding methods have shown promise in detecting\nwords' meaning change by encoding them into dense vectors. However, when\nexploring semantic change of words over time, these methods require the\nalignment of word embeddings across different time periods. This process is\ncomputationally expensive, prohibitively time consuming and suffering from\ncontextual variability. In this paper, we propose a new and scalable method for\nencoding words from different time periods into one dense vector space. This\ncan greatly improve performance when it comes to identifying words that have\nchanged in meaning over time. We evaluated our method on dataset from Google\nBooks N-gram. Our method outperformed three other popular methods in terms of\nthe number of words correctly identified to have changed in meaning.\nAdditionally, we provide an intuitive visualization of the semantic evolution\nof some words extracted by our method",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08716v1",
            "date": "2019-10-19 07:28:39+00:00",
            "title": "Adversarial Attacks on Spoofing Countermeasures of automatic speaker verification",
            "authors": [
                "Songxiang Liu",
                "Haibin Wu",
                "Hung-yi Lee",
                "Helen Meng"
            ],
            "abstract": "High-performance spoofing countermeasure systems for automatic speaker\nverification (ASV) have been proposed in the ASVspoof 2019 challenge. However,\nthe robustness of such systems under adversarial attacks has not been studied\nyet. In this paper, we investigate the vulnerability of spoofing\ncountermeasures for ASV under both white-box and black-box adversarial attacks\nwith the fast gradient sign method (FGSM) and the projected gradient descent\n(PGD) method. We implement high-performing countermeasure models in the\nASVspoof 2019 challenge and conduct adversarial attacks on them. We compare\nperformance of black-box attacks across spoofing countermeasure models with\ndifferent network architectures and different amount of model parameters. The\nexperimental results show that all implemented countermeasure models are\nvulnerable to FGSM and PGD attacks under the scenario of white-box attack. The\nmore dangerous black-box attacks also prove to be effective by the experimental\nresults.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.08962v2",
            "date": "2019-10-20 12:32:20+00:00",
            "title": "Byte-Pair Encoding for Text-to-SQL Generation",
            "authors": [
                "Samuel M\u00fcller",
                "Andreas Vlachos"
            ],
            "abstract": "Neural sequence-to-sequence models provide a competitive approach to the task\nof mapping a question in natural language to an SQL query, also referred to as\ntext-to-SQL generation. The Byte-Pair Encoding algorithm (BPE) has previously\nbeen used to improve machine translation (MT) between natural languages. In\nthis work, we adapt BPE for text-to-SQL generation. As the datasets for this\ntask are rather small compared to MT, we present a novel stopping criterion\nthat prevents overfitting the BPE encoding to the training set. Additionally,\nwe present AST BPE, which is a version of BPE that uses the Abstract Syntax\nTree (AST) of the SQL statement to guide BPE merges and therefore produce BPE\nencodings that generalize better. We improved the accuracy of a strong\nattentive seq2seq baseline on five out of six English text-to-SQL tasks while\nreducing training time by more than 50% on four of them due to the shortened\ntargets. Finally, on two of these tasks we exceeded previously reported\naccuracies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09134v2",
            "date": "2019-10-21 03:32:17+00:00",
            "title": "Good, Better, Best: Textual Distractors Generation for Multi-Choice VQA via Policy Gradient",
            "authors": [
                "Jiaying Lu",
                "Xin Ye",
                "Yi Ren",
                "Yezhou Yang"
            ],
            "abstract": "An increasing amount of studies have investigated the decision-making process\nof VQA models. Many of these studies focus on the reason behind the correct\nanswer chosen by a model. Yet, the reason why the distracting answer chose by a\nmodel has rarely been studied. To this end, we introduce a novel task called\n\\textit{textual Distractors Generation for VQA} (DG-VQA) that explaining the\ndecision boundaries of existing VQA models. The goal of DG-VQA is to generate\nthe most confusing set of textual distractors in multi-choice VQA tasks which\nexpose the vulnerability of existing models (i.e. to generate distractors that\nlure existing models to fail). We show that DG-VQA can be formulated as a\nMarkov Decision Process, and present a reinforcement learning solution to come\nup with distractors in an unsupervised manner. The solution addresses the lack\nof large annotated corpus issues in previous distractor generation methods. Our\nproposed model receives reward signals from fully-trained multi-choice VQA\nmodels and updates its parameters via policy gradient. The empirical results\nshow that the generated textual distractors can successfully attack several\npopular VQA models with an average $20\\%$ accuracy drop from $64\\%$.\nFurthermore, we conduct adversarial training to improve the robustness of VQA\nmodels by incorporating the generated distractors. Empirical results validate\nthe effectiveness of adversarial training by showing a performance improvement\nof $27\\%$ for the multi-choice VQA task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.09260v1",
            "date": "2019-10-21 10:55:46+00:00",
            "title": "Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning",
            "authors": [
                "Jingjing Wang",
                "Changlong Sun",
                "Shoushan Li",
                "Jiancheng Wang",
                "Luo Si",
                "Min Zhang",
                "Xiaozhong Liu",
                "Guodong Zhou"
            ],
            "abstract": "Recently, neural networks have shown promising results on Document-level\nAspect Sentiment Classification (DASC). However, these approaches often offer\nlittle transparency w.r.t. their inner working mechanisms and lack\ninterpretability. In this paper, to simulating the steps of analyzing aspect\nsentiment in a document by human beings, we propose a new Hierarchical\nReinforcement Learning (HRL) approach to DASC. This approach incorporates\nclause selection and word selection strategies to tackle the data noise problem\nin the task of DASC. First, a high-level policy is proposed to select\naspect-relevant clauses and discard noisy clauses. Then, a low-level policy is\nproposed to select sentiment-relevant words and discard noisy words inside the\nselected clauses. Finally, a sentiment rating predictor is designed to provide\nreward signals to guide both clause and word selection. Experimental results\ndemonstrate the impressive effectiveness of the proposed approach to DASC over\nthe state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09275v3",
            "date": "2019-10-21 11:53:07+00:00",
            "title": "Text Matters but Speech Influences: A Computational Analysis of Syntactic Ambiguity Resolution",
            "authors": [
                "Won Ik Cho",
                "Jeonghwa Cho",
                "Woo Hyun Kang",
                "Nam Soo Kim"
            ],
            "abstract": "Analyzing how human beings resolve syntactic ambiguity has long been an issue\nof interest in the field of linguistics. It is, at the same time, one of the\nmost challenging issues for spoken language understanding (SLU) systems as\nwell. As syntactic ambiguity is intertwined with issues regarding prosody and\nsemantics, the computational approach toward speech intention identification is\nexpected to benefit from the observations of the human language processing\nmechanism. In this regard, we address the task with attentive recurrent neural\nnetworks that exploit acoustic and textual features simultaneously and reveal\nhow the modalities interact with each other to derive sentence meaning.\nUtilizing a speech corpus recorded on Korean scripts of syntactically ambiguous\nutterances, we revealed that co-attention frameworks, namely multi-hop\nattention and cross-attention, show significantly superior performance in\ndisambiguating speech intention. With further analysis, we demonstrate that the\ncomputational models reflect the internal relationship between auditory and\nlinguistic processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09292v1",
            "date": "2019-10-21 12:23:48+00:00",
            "title": "On Semi-Supervised Multiple Representation Behavior Learning",
            "authors": [
                "Ruqian Lu",
                "Shengluan Hou"
            ],
            "abstract": "We propose a novel paradigm of semi-supervised learning (SSL)--the\nsemi-supervised multiple representation behavior learning (SSMRBL). SSMRBL aims\nto tackle the difficulty of learning a grammar for natural language parsing\nwhere the data are natural language texts and the 'labels' for marking data are\nparsing trees and/or grammar rule pieces. We call such 'labels' as compound\nstructured labels which require a hard work for training. SSMRBL is an\nincremental learning process that can learn more than one representation, which\nis an appropriate solution for dealing with the scarce of labeled training data\nin the age of big data and with the heavy workload of learning compound\nstructured labels. We also present a typical example of SSMRBL, regarding\nbehavior learning in form of a grammatical approach towards domain-based\nmultiple text summarization (DBMTS). DBMTS works under the framework of\nrhetorical structure theory (RST). SSMRBL includes two representations: text\nembedding (for representing information contained in the texts) and grammar\nmodel (for representing parsing as a behavior). The first representation was\nlearned as embedded digital vectors called impacts in a low dimensional space.\nThe grammar model was learned in an iterative way. Then an automatic\ndomain-oriented multi-text summarization approach was proposed based on the two\nrepresentations discussed above. Experimental results on large-scale Chinese\ndataset SogouCA indicate that the proposed method brings a good performance\neven if only few labeled texts are used for training with respect to our\ndefined automated metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09329v2",
            "date": "2019-10-21 12:59:32+00:00",
            "title": "A Neural Entity Coreference Resolution Review",
            "authors": [
                "Nikolaos Stylianou",
                "Ioannis Vlahavas"
            ],
            "abstract": "Entity Coreference Resolution is the task of resolving all mentions in a\ndocument that refer to the same real world entity and is considered as one of\nthe most difficult tasks in natural language understanding. It is of great\nimportance for downstream natural language processing tasks such as entity\nlinking, machine translation, summarization, chatbots, etc. This work aims to\ngive a detailed review of current progress on solving Coreference Resolution\nusing neural-based approaches. It also provides a detailed appraisal of the\ndatasets and evaluation metrics in the field, as well as the subtask of Pronoun\nResolution that has seen various improvements in the recent years. We highlight\nthe advantages and disadvantages of the approaches, the challenges of the task,\nthe lack of agreed-upon standards in the task and propose a way to further\nexpand the boundaries of the field.",
            "journal ref": "Expert Systems with Applications, Volume 168, 15 April 2021,\n  114466",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09513v1",
            "date": "2019-10-21 17:06:38+00:00",
            "title": "The Czech Court Decisions Corpus (CzCDC): Availability as the First Step",
            "authors": [
                "Tereza Novotn\u00e1",
                "Jakub Hara\u0161ta"
            ],
            "abstract": "In this paper, we describe the Czech Court Decision Corpus (CzCDC). CzCDC is\na dataset of 237,723 decisions published by the Czech apex (or top-tier)\ncourts, namely the Supreme Court, the Supreme Administrative Court and the\nConstitutional Court. All the decisions were published between 1st January 1993\nand 30th September 2018.\n  Court decisions are available on the webpages of the respective courts or via\ncommercial databases of legal information. This often leads researchers\ninterested in these decisions to reach either to respective court or to\ncommercial provider. This leads to delays and additional costs. These are\nfurther exacerbated by a lack of inter-court standard in the terms of the data\nformat in which courts provide their decisions. Additionally, courts' databases\noften lack proper documentation.\n  Our goal is to make the dataset of court decisions freely available online in\nconsistent (plain) format to lower the cost associated with obtaining data for\nfuture research. We believe that simplified access to court decisions through\nthe CzCDC could benefit other researchers.\n  In this paper, we describe the processing of decisions before their inclusion\ninto CzCDC and basic statistics of the dataset. This dataset contains plain\ntexts of court decisions and these texts are not annotated for any grammatical\nor syntactical features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09532v3",
            "date": "2019-10-21 17:55:25+00:00",
            "title": "Building Dynamic Knowledge Graphs from Text-based Games",
            "authors": [
                "Mikul\u00e1\u0161 Zelinka",
                "Xingdi Yuan",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Romain Laroche",
                "Adam Trischler"
            ],
            "abstract": "We are interested in learning how to update Knowledge Graphs (KG) from text.\nIn this preliminary work, we propose a novel Sequence-to-Sequence (Seq2Seq)\narchitecture to generate elementary KG operations. Furthermore, we introduce a\nnew dataset for KG extraction built upon text-based game transitions (over 300k\ndata points). We conduct experiments and discuss the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09621v3",
            "date": "2019-10-21 19:29:06+00:00",
            "title": "On Automating Conversations",
            "authors": [
                "Ting-Hao 'Kenneth' Huang"
            ],
            "abstract": "From 2016 to 2018, we developed and deployed Chorus, a system that blends\nreal-time human computation with artificial intelligence (AI) and has\nreal-world, open conversations with users. We took a top-down approach that\nstarted with a working crowd-powered system, Chorus, and then created a\nframework, Evorus, that enables Chorus to automate itself over time. Over our\ntwo-year deployment, more than 420 users talked with Chorus, having over 2,200\nconversation sessions. This line of work demonstrated how a crowd-powered\nconversational assistant can be automated over time, and more importantly, how\nsuch a system can be deployed to talk with real users to help them with their\neveryday tasks. This position paper discusses two sets of challenges that we\nexplored during the development and deployment of Chorus and Evorus: the\nchallenges that come from being an \"agent\" and those that arise from the subset\nof conversations that are more difficult to automate.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1910.09760v2",
            "date": "2019-10-22 04:21:06+00:00",
            "title": "Question Answering over Knowledge Graphs via Structural Query Patterns",
            "authors": [
                "Weiguo Zheng",
                "Mei Zhang"
            ],
            "abstract": "Natural language question answering over knowledge graphs is an important and\ninteresting task as it enables common users to gain accurate answers in an easy\nand intuitive manner. However, it remains a challenge to bridge the gap between\nunstructured questions and structured knowledge graphs. To address the problem,\na natural discipline is building a structured query to represent the input\nquestion. Searching the structured query over the knowledge graph can produce\nanswers to the question. Distinct from the existing methods that are based on\nsemantic parsing or templates, we propose an effective approach powered by a\nnovel notion, structural query pattern, in this paper. Given an input question,\nwe first generate its query sketch that is compatible with the underlying\nstructure of the knowledge graph. Then, we complete the query graph by labeling\nthe nodes and edges under the guidance of the structural query pattern.\nFinally, answers can be retrieved by executing the constructed query graph over\nthe knowledge graph. Evaluations on three question answering benchmarks show\nthat our proposed approach outperforms state-of-the-art methods significantly.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.09799v2",
            "date": "2019-10-22 07:20:02+00:00",
            "title": "Transformer-based Acoustic Modeling for Hybrid Speech Recognition",
            "authors": [
                "Yongqiang Wang",
                "Abdelrahman Mohamed",
                "Duc Le",
                "Chunxi Liu",
                "Alex Xiao",
                "Jay Mahadeokar",
                "Hongzhao Huang",
                "Andros Tjandra",
                "Xiaohui Zhang",
                "Frank Zhang",
                "Christian Fuegen",
                "Geoffrey Zweig",
                "Michael L. Seltzer"
            ],
            "abstract": "We propose and evaluate transformer-based acoustic models (AMs) for hybrid\nspeech recognition. Several modeling choices are discussed in this work,\nincluding various positional embedding methods and an iterated loss to enable\ntraining deep transformers. We also present a preliminary study of using\nlimited right context in transformer models, which makes it possible for\nstreaming applications. We demonstrate that on the widely used Librispeech\nbenchmark, our transformer-based AM outperforms the best published hybrid\nresult by 19% to 26% relative when the standard n-gram language model (LM) is\nused. Combined with neural network LM for rescoring, our proposed approach\nachieves state-of-the-art results on Librispeech. Our findings are also\nconfirmed on a much larger internal dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09879v2",
            "date": "2019-10-22 10:35:41+00:00",
            "title": "Towards Combinational Relation Linking over Knowledge Graphs",
            "authors": [
                "Weiguo Zheng",
                "Mei Zhang"
            ],
            "abstract": "Given a natural language phrase, relation linking aims to find a relation\n(predicate or property) from the underlying knowledge graph to match the\nphrase. It is very useful in many applications, such as natural language\nquestion answering, personalized recommendation and text summarization.\nHowever, the previous relation linking algorithms usually produce a single\nrelation for the input phrase and pay little attention to a more general and\nchallenging problem, i.e., combinational relation linking that extracts a\nsubgraph pattern to match the compound phrase (e.g. mother-in-law). In this\npaper, we focus on the task of combinational relation linking over knowledge\ngraphs. To resolve the problem, we design a systematic method based on the\ndata-driven relation assembly technique, which is performed under the guidance\nof meta patterns. We also introduce external knowledge to enhance the system\nunderstanding ability. Finally, we conduct extensive experiments over the real\nknowledge graph to study the performance of the proposed method.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.09982v1",
            "date": "2019-10-20 10:53:18+00:00",
            "title": "Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection",
            "authors": [
                "Giovanni Da San Martino",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Preslav Nakov"
            ],
            "abstract": "We present the shared task on Fine-Grained Propaganda Detection, which was\norganized as part of the NLP4IF workshop at EMNLP-IJCNLP 2019. There were two\nsubtasks. FLC is a fragment-level task that asks for the identification of\npropagandist text fragments in a news article and also for the prediction of\nthe specific propaganda technique used in each such fragment (18-way\nclassification task). SLC is a sentence-level binary classification task asking\nto detect the sentences that contain propaganda. A total of 12 teams submitted\nsystems for the FLC task, 25 teams did so for the SLC task, and 14 teams\neventually submitted a system description paper. For both subtasks, most\nsystems managed to beat the baseline by a sizable margin. The leaderboard and\nthe data from the competition are available at\nhttp://propaganda.qcri.org/nlp4if-shared-task/.",
            "journal ref": "NLP4IF@EMNLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10032v2",
            "date": "2019-10-22 15:08:49+00:00",
            "title": "GPU-Accelerated Viterbi Exact Lattice Decoder for Batched Online and Offline Speech Recognition",
            "authors": [
                "Hugo Braun",
                "Justin Luitjens",
                "Ryan Leary",
                "Tim Kaldewey",
                "Daniel Povey"
            ],
            "abstract": "We present an optimized weighted finite-state transducer (WFST) decoder\ncapable of online streaming and offline batch processing of audio using\nGraphics Processing Units (GPUs). The decoder is efficient in memory\nutilization, input/output (I/O) bandwidth, and uses a novel Viterbi\nimplementation designed to maximize parallelism. The reduced memory footprint\nallows the decoder to process significantly larger graphs than previously\npossible, while optimizing I/O increases the number of simultaneous streams\nsupported. GPU preprocessing of lattice segments enables intermediate lattice\nresults to be returned to the requestor during streaming inference.\nCollectively, the proposed algorithm yields up to a 240x speedup over single\ncore CPU decoding, and up to 40x faster decoding than the current\nstate-of-the-art GPU decoder, while returning equivalent results. This decoder\ndesign enables deployment of production-grade ASR models on a large spectrum of\nsystems, ranging from large data center servers to low-power edge devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10037v1",
            "date": "2019-10-22 15:15:27+00:00",
            "title": "One-Shot Template Matching for Automatic Document Data Capture",
            "authors": [
                "Pranjal Dhakal",
                "Manish Munikar",
                "Bikram Dahal"
            ],
            "abstract": "In this paper, we propose a novel one-shot template-matching algorithm to\nautomatically capture data from business documents with an aim to minimize\nmanual data entry. Given one annotated document, our algorithm can\nautomatically extract similar data from other documents having the same format.\nBased on a set of engineered visual and textual features, our method is\ninvariant to changes in position and value. Experiments on a dataset of 595\nreal invoices demonstrate 86.4% accuracy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.10073v4",
            "date": "2019-10-22 16:15:58+00:00",
            "title": "Depth-Adaptive Transformer",
            "authors": [
                "Maha Elbayad",
                "Jiatao Gu",
                "Edouard Grave",
                "Michael Auli"
            ],
            "abstract": "State of the art sequence-to-sequence models for large scale tasks perform a\nfixed number of computations for each input sequence regardless of whether it\nis easy or hard to process. In this paper, we train Transformer models which\ncan make output predictions at different stages of the network and we\ninvestigate different ways to predict how much computation is required for a\nparticular sequence. Unlike dynamic computation in Universal Transformers,\nwhich applies the same set of layers iteratively, we apply different layers at\nevery step to adjust both the amount of computation as well as the model\ncapacity. On IWSLT German-English translation our approach matches the accuracy\nof a well tuned baseline Transformer while using less than a quarter of the\ndecoder layers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10082v1",
            "date": "2019-10-22 16:20:35+00:00",
            "title": "Toward estimating personal well-being using voice",
            "authors": [
                "Samuel Kim",
                "Namhee Kwon",
                "Henry O'Connell"
            ],
            "abstract": "Estimating personal well-being draws increasing attention particularly from\nhealthcare and pharmaceutical industries. We propose an approach to estimate\npersonal well-being in terms of various measurements such as anxiety, sleep\nquality and mood using voice. With clinically validated questionnaires to score\nthose measurements in a self-assessed way, we extract salient features from\nvoice and train regression models with deep neural networks. Experiments with\nthe collected database of 219 subjects show promising results in predicting the\nwell-being related measurements; concordance correlation coefficients (CCC)\nbetween self-assessed scores and predicted scores are 0.41 for anxiety, 0.44\nfor sleep quality and 0.38 for mood.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10238v1",
            "date": "2019-10-22 21:24:24+00:00",
            "title": "Robust Neural Machine Translation for Clean and Noisy Speech Transcripts",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Robert Enyedi",
                "Alessandra Brusadin",
                "Marcello Federico"
            ],
            "abstract": "Neural machine translation models have shown to achieve high quality when\ntrained and fed with well structured and punctuated input texts. Unfortunately,\nthe latter condition is not met in spoken language translation, where the input\nis generated by an automatic speech recognition (ASR) system. In this paper, we\nstudy how to adapt a strong NMT system to make it robust to typical ASR errors.\nAs in our application scenarios transcripts might be post-edited by human\nexperts, we propose adaptation strategies to train a single system that can\ntranslate either clean or noisy input with no supervision on the input type.\nOur experimental results on a public speech translation data set show that\nadapting a model on a significant amount of parallel data including ASR\ntranscripts is beneficial with test data of the same type, but produces a small\ndegradation when translating clean text. Adapting on both clean and noisy\nvariants of the same data leads to the best results on both input types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10387v2",
            "date": "2019-10-23 07:08:03+00:00",
            "title": "Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks",
            "authors": [
                "Xingchen Song",
                "Guangsen Wang",
                "Zhiyong Wu",
                "Yiheng Huang",
                "Dan Su",
                "Dong Yu",
                "Helen Meng"
            ],
            "abstract": "Self-attention network (SAN) can benefit significantly from the\nbi-directional representation learning through unsupervised pretraining\nparadigms such as BERT and XLNet. In this paper, we present an XLNet-like\npretraining scheme \"Speech-XLNet\" for unsupervised acoustic model pretraining\nto learn speech representations with SAN. The pretrained SAN is finetuned under\nthe hybrid SAN/HMM framework. We conjecture that by shuffling the speech frame\norders, the permutation in Speech-XLNet serves as a strong regularizer to\nencourage the SAN to make inferences by focusing on global structures through\nits attention weights. In addition, Speech-XLNet also allows the model to\nexplore the bi-directional contexts for effective speech representation\nlearning. Experiments on TIMIT and WSJ demonstrate that Speech-XLNet greatly\nimproves the SAN/HMM performance in terms of both convergence speed and\nrecognition accuracy compared to the one trained from randomly initialized\nweights. Our best systems achieve a relative improvement of 11.9% and 8.3% on\nthe TIMIT and WSJ tasks respectively. In particular, the best system achieves a\nphone error rate (PER) of 13.3% on the TIMIT test set, which to our best\nknowledge, is the lowest PER obtained from a single system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10486v3",
            "date": "2019-10-16 22:17:02+00:00",
            "title": "Does Gender Matter? Towards Fairness in Dialogue Systems",
            "authors": [
                "Haochen Liu",
                "Jamell Dacon",
                "Wenqi Fan",
                "Hui Liu",
                "Zitao Liu",
                "Jiliang Tang"
            ],
            "abstract": "Recently there are increasing concerns about the fairness of Artificial\nIntelligence (AI) in real-world applications such as computer vision and\nrecommendations. For example, recognition algorithms in computer vision are\nunfair to black people such as poorly detecting their faces and inappropriately\nidentifying them as \"gorillas\". As one crucial application of AI, dialogue\nsystems have been extensively applied in our society. They are usually built\nwith real human conversational data; thus they could inherit some fairness\nissues which are held in the real world. However, the fairness of dialogue\nsystems has not been well investigated. In this paper, we perform a pioneering\nstudy about the fairness issues in dialogue systems. In particular, we\nconstruct a benchmark dataset and propose quantitative measures to understand\nfairness in dialogue models. Our studies demonstrate that popular dialogue\nmodels show significant prejudice towards different genders and races. Besides,\nto mitigate the bias in dialogue systems, we propose two simple but effective\ndebiasing methods. Experiments show that our methods can reduce the bias in\ndialogue systems significantly. The dataset and the implementation are released\nto foster fairness research in dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10490v1",
            "date": "2019-10-16 13:55:02+00:00",
            "title": "Automated Text Summarization for the Enhancement of Public Services",
            "authors": [
                "Xingbang Liu",
                "Janyl Jumadinova"
            ],
            "abstract": "Natural language processing and machine learning algorithms have been shown\nto be effective in a variety of applications. In this work, we contribute to\nthe area of AI adoption in the public sector. We present an automated system\nthat was used to process textual information, generate important keywords, and\nautomatically summarize key elements of the Meadville community statements. We\nalso describe the process of collaboration with My Meadville administrators\nduring the development of our system. My Meadville, a community initiative,\nsupported by the city of Meadville conducted a large number of interviews with\nthe residents of Meadville during the community events and transcribed these\ninterviews into textual data files. Their goal was to uncover the issues of\nimportance to the Meadville residents in an attempt to enhance public services.\nOur AI system cleans and pre-processes the interview data, then using machine\nlearning algorithms it finds important keywords and key excerpts from each\ninterview. It also provides searching functionality to find excerpts from\nrelevant interviews based on specific keywords. Our automated system allowed\nthe city to save over 300 hours of human labor that would have taken to read\nall interviews and highlight important points. Our findings are being used by\nMy Meadville initiative to locate important information from the collected data\nset for ongoing community enhancement projects, to highlight relevant community\nassets, and to assist in identifying the steps to be taken based on the\nconcerns and areas of improvement identified by the community members.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10492v2",
            "date": "2019-10-17 23:00:22+00:00",
            "title": "Question Classification with Deep Contextualized Transformer",
            "authors": [
                "Haozheng Luo",
                "Ningwei Liu",
                "Charles Feng"
            ],
            "abstract": "The latest work for Question and Answer problems is to use the Stanford Parse\nTree. We build on prior work and develop a new method to handle the Question\nand Answer problem with the Deep Contextualized Transformer to manage some\naberrant expressions. We also conduct extensive evaluations of the SQuAD and\nSwDA dataset and show significant improvement over QA problem classification of\nindustry needs. We also investigate the impact of different models for the\naccuracy and efficiency of the problem answers. It shows that our new method is\nmore effective for solving QA problems with higher accuracy",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10502v1",
            "date": "2019-10-21 09:33:09+00:00",
            "title": "Opinion aspect extraction in Dutch childrens diary entries",
            "authors": [
                "Hella Haanstra",
                "Maaike H. T. de Boer"
            ],
            "abstract": "Aspect extraction can be used in dialogue systems to understand the topic of\nopinionated text. Expressing an empathetic reaction to an opinion can\nstrengthen the bond between a human and, for example, a robot. The aim of this\nstudy is three-fold: 1. create a new annotated dataset for both aspect\nextraction and opinion words for Dutch childrens language, 2. acquire aspect\nextraction results for this task and 3. improve current results for aspect\nextraction in Dutch reviews. This was done by training a deep learning Gated\nRecurrent Unit (GRU) model, originally developed for an English review dataset,\non Dutch restaurant review data to classify both opinion words and their\nrespective aspects. We obtained state-of-the-art performance on the Dutch\nrestaurant review dataset. Additionally, we acquired aspect extraction results\nfor the Dutch childrens dataset. Since the model was trained on standardised\nlanguage, these results are quite promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10663v1",
            "date": "2019-10-23 16:39:21+00:00",
            "title": "Instance-Based Model Adaptation For Direct Speech Translation",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Viet-Nhat Nguyen",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Despite recent technology advancements, the effectiveness of neural\napproaches to end-to-end speech-to-text translation is still limited by the\npaucity of publicly available training corpora. We tackle this limitation with\na method to improve data exploitation and boost the system's performance at\ninference time. Our approach allows us to customize \"on the fly\" an existing\nmodel to each incoming translation request. At its core, it exploits an\ninstance selection procedure to retrieve, from a given pool of data, a small\nset of samples similar to the input query in terms of latent properties of its\naudio signal. The retrieved samples are then used for an instance-specific\nfine-tuning of the model. We evaluate our approach in three different\nscenarios. In all data conditions (different languages, in/out-of-domain\nadaptation), our instance-based adaptation yields coherent performance gains\nover static models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10670v1",
            "date": "2019-10-23 17:10:26+00:00",
            "title": "Efficient Dynamic WFST Decoding for Personalized Language Models",
            "authors": [
                "Jun Liu",
                "Jiedan Zhu",
                "Vishal Kathuria",
                "Fuchun Peng"
            ],
            "abstract": "We propose a two-layer cache mechanism to speed up dynamic WFST decoding with\npersonalized language models. The first layer is a public cache that stores\nmost of the static part of the graph. This is shared globally among all users.\nA second layer is a private cache that caches the graph that represents the\npersonalized language model, which is only shared by the utterances from a\nparticular user. We also propose two simple yet effective pre-initialization\nmethods, one based on breadth-first search, and another based on a data-driven\nexploration of decoder states using previous utterances. Experiments with a\ncalling speech recognition task using a personalized contact list demonstrate\nthat the proposed public cache reduces decoding time by factor of three\ncompared to decoding without pre-initialization. Using the private cache\nprovides additional efficiency gains, reducing the decoding time by a factor of\nfive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10706v3",
            "date": "2019-10-23 01:44:12+00:00",
            "title": "KnowIT VQA: Answering Knowledge-Based Questions about Videos",
            "authors": [
                "Noa Garcia",
                "Mayu Otani",
                "Chenhui Chu",
                "Yuta Nakashima"
            ],
            "abstract": "We propose a novel video understanding task by fusing knowledge-based and\nvideo question answering. First, we introduce KnowIT VQA, a video dataset with\n24,282 human-generated question-answer pairs about a popular sitcom. The\ndataset combines visual, textual and temporal coherence reasoning together with\nknowledge-based questions, which need of the experience obtained from the\nviewing of the series to be answered. Second, we propose a video understanding\nmodel by combining the visual and textual video content with specific knowledge\nabout the show. Our main findings are: (i) the incorporation of knowledge\nproduces outstanding improvements for VQA in video, and (ii) the performance on\nKnowIT VQA still lags well behind human accuracy, indicating its usefulness for\nstudying current video modelling limitations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.10762v2",
            "date": "2019-10-23 18:37:56+00:00",
            "title": "Analyzing ASR pretraining for low-resource speech-to-text translation",
            "authors": [
                "Mihaela C. Stoian",
                "Sameer Bansal",
                "Sharon Goldwater"
            ],
            "abstract": "Previous work has shown that for low-resource source languages, automatic\nspeech-to-text translation (AST) can be improved by pretraining an end-to-end\nmodel on automatic speech recognition (ASR) data from a high-resource language.\nHowever, it is not clear what factors --e.g., language relatedness or size of\nthe pretraining data-- yield the biggest improvements, or whether pretraining\ncan be effectively combined with other methods such as data augmentation. Here,\nwe experiment with pretraining on datasets of varying sizes, including\nlanguages related and unrelated to the AST source language. We find that the\nbest predictor of final AST performance is the word error rate of the\npretrained ASR model, and that differences in ASR/AST performance correlate\nwith how phonetic information is encoded in the later RNN layers of our model.\nWe also show that pretraining and data augmentation yield complementary\nbenefits for AST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10832v1",
            "date": "2019-10-23 23:01:10+00:00",
            "title": "Emergent Properties of Finetuned Language Representation Models",
            "authors": [
                "Alexandre Matton",
                "Luke de Oliveira"
            ],
            "abstract": "Large, self-supervised transformer-based language representation models have\nrecently received significant amounts of attention, and have produced\nstate-of-the-art results across a variety of tasks simply by scaling up\npre-training on larger and larger corpora. Such models usually produce high\ndimensional vectors, on top of which additional task-specific layers and\narchitectural modifications are added to adapt them to specific downstream\ntasks. Though there exists ample evidence that such models work well, we aim to\nunderstand what happens when they work well. We analyze the redundancy and\nlocation of information contained in output vectors for one such language\nrepresentation model -- BERT. We show empirical evidence that the [CLS]\nembedding in BERT contains highly redundant information, and can be compressed\nwith minimal loss of accuracy, especially for finetuned models, dovetailing\ninto open threads in the field about the role of over-parameterization in\nlearning. We also shed light on the existence of specific output dimensions\nwhich alone give very competitive results when compared to using all dimensions\nof output vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10849v1",
            "date": "2019-10-24 00:22:43+00:00",
            "title": "GF + MMT = GLF -- From Language to Semantics through LF",
            "authors": [
                "Michael Kohlhase",
                "Jan Frederik Schaefer"
            ],
            "abstract": "These days, vast amounts of knowledge are available online, most of it in\nwritten form. Search engines help us access this knowledge, but aggregating,\nrelating and reasoning with it is still a predominantly human effort. One of\nthe key challenges for automated reasoning based on natural-language texts is\nthe need to extract meaning (semantics) from texts. Natural language\nunderstanding (NLU) systems describe the conversion from a set of natural\nlanguage utterances to terms in a particular logic. Tools for the\nco-development of grammar and target logic are currently largely missing.\n  We will describe the Grammatical Logical Framework (GLF), a combination of\ntwo existing frameworks, in which large parts of a symbolic, rule-based NLU\nsystem can be developed and implemented: the Grammatical Framework (GF) and\nMMT. GF is a tool for syntactic analysis, generation, and translation with\ncomplex natural language grammars and MMT can be used to specify logical\nsystems and to represent knowledge in them. Combining these tools is possible,\nbecause they are based on compatible logical frameworks: Martin-L\\\"of type\ntheory and LF. The flexibility of logical frameworks is needed, as NLU research\nhas not settled on a particular target logic for meaning representation.\nInstead, new logics are developed all the time to handle various language\nphenomena. GLF allows users to develop the logic and the language parsing\ncomponents in parallel, and to connect them for experimentation with the entire\npipeline.",
            "journal ref": "EPTCS 307, 2019, pp. 24-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10857v4",
            "date": "2019-10-24 00:33:39+00:00",
            "title": "Selective Attention Based Graph Convolutional Networks for Aspect-Level Sentiment Classification",
            "authors": [
                "Xiaochen Hou",
                "Jing Huang",
                "Guangtao Wang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Aspect-level sentiment classification aims to identify the sentiment polarity\ntowards a specific aspect term in a sentence. Most current approaches mainly\nconsider the semantic information by utilizing attention mechanisms to capture\nthe interactions between the context and the aspect term. In this paper, we\npropose to employ graph convolutional networks (GCNs) on the dependency tree to\nlearn syntax-aware representations of aspect terms. GCNs often show the best\nperformance with two layers, and deeper GCNs do not bring additional gain due\nto over-smoothing problem. However, in some cases, important context words\ncannot be reached within two hops on the dependency tree. Therefore we design a\nselective attention based GCN block (SA-GCN) to find the most important context\nwords, and directly aggregate these information into the aspect-term\nrepresentation. We conduct experiments on the SemEval 2014 Task 4 datasets. Our\nexperimental results show that our model outperforms the current\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10872v1",
            "date": "2019-10-24 01:32:24+00:00",
            "title": "Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition",
            "authors": [
                "Ninareh Mehrabi",
                "Thamme Gowda",
                "Fred Morstatter",
                "Nanyun Peng",
                "Aram Galstyan"
            ],
            "abstract": "We study the bias in several state-of-the-art named entity recognition (NER)\nmodels---specifically, a difference in the ability to recognize male and female\nnames as PERSON entity types. We evaluate NER models on a dataset containing\n139 years of U.S. census baby names and find that relatively more female names,\nas opposed to male names, are not recognized as PERSON entities. We study the\nextent of this bias in several NER systems that are used prominently in\nindustry and academia. In addition, we also report a bias in the datasets on\nwhich these models were trained. The result of this analysis yields a new\nbenchmark for gender bias evaluation in named entity recognition systems. The\ndata and code for the application of this benchmark will be publicly available\nfor researchers to use.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.10909v2",
            "date": "2019-10-24 04:24:27+00:00",
            "title": "ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit",
            "authors": [
                "Tomoki Hayashi",
                "Ryuichi Yamamoto",
                "Katsuki Inoue",
                "Takenori Yoshimura",
                "Shinji Watanabe",
                "Tomoki Toda",
                "Kazuya Takeda",
                "Yu Zhang",
                "Xu Tan"
            ],
            "abstract": "This paper introduces a new end-to-end text-to-speech (E2E-TTS) toolkit named\nESPnet-TTS, which is an extension of the open-source speech processing toolkit\nESPnet. The toolkit supports state-of-the-art E2E-TTS models, including\nTacotron~2, Transformer TTS, and FastSpeech, and also provides recipes inspired\nby the Kaldi automatic speech recognition (ASR) toolkit. The recipes are based\non the design unified with the ESPnet ASR recipe, providing high\nreproducibility. The toolkit also provides pre-trained models and samples of\nall of the recipes so that users can use it as a baseline. Furthermore, the\nunified design enables the integration of ASR functions with TTS, e.g.,\nASR-based objective evaluation and semi-supervised learning with both ASR and\nTTS models. This paper describes the design of the toolkit and experimental\nevaluation in comparison with other toolkits. The experimental results show\nthat our models can achieve state-of-the-art performance comparable to the\nother latest toolkits, resulting in a mean opinion score (MOS) of 4.25 on the\nLJSpeech dataset. The toolkit is publicly available at\nhttps://github.com/espnet/espnet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11161v1",
            "date": "2019-10-24 14:18:55+00:00",
            "title": "Diversifying Topic-Coherent Response Generation for Natural Multi-turn Conversations",
            "authors": [
                "Fei Hu",
                "Wei Liu",
                "Ajmal Saeed Mian",
                "Li Li"
            ],
            "abstract": "Although response generation (RG) diversification for single-turn dialogs has\nbeen well developed, it is less investigated for natural multi-turn\nconversations. Besides, past work focused on diversifying responses without\nconsidering topic coherence to the context, producing uninformative replies. In\nthis paper, we propose the Topic-coherent Hierarchical Recurrent\nEncoder-Decoder model (THRED) to diversify the generated responses without\ndeviating the contextual topics for multi-turn conversations. In overall, we\nbuild a sequence-to-sequence net (Seq2Seq) to model multi-turn conversations.\nAnd then we resort to the latent Variable Hierarchical Recurrent\nEncoder-Decoder model (VHRED) to learn global contextual distribution of\ndialogs. Besides, we construct a dense topic matrix which implies word-level\ncorrelations of the conversation corpora. The topic matrix is used to learn\nlocal topic distribution of the contextual utterances. By incorporating both\nthe global contextual distribution and the local topic distribution, THRED\nproduces both diversified and topic-coherent replies. In addition, we propose\nan explicit metric (\\emph{TopicDiv}) to measure the topic divergence between\nthe post and generated response, and we also propose an overall metric\ncombining the diversification metric (\\emph{Distinct}) and \\emph{TopicDiv}. We\nevaluate our model comparing with three baselines (Seq2Seq, HRED and VHRED) on\ntwo real-world corpora, respectively, and demonstrate its outstanding\nperformance in both diversification and topic coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11235v2",
            "date": "2019-10-13 23:34:04+00:00",
            "title": "Rethinking Exposure Bias In Language Modeling",
            "authors": [
                "Yifan Xu",
                "Kening Zhang",
                "Haoyu Dong",
                "Yuezhou Sun",
                "Wenlong Zhao",
                "Zhuowen Tu"
            ],
            "abstract": "Exposure bias describes the phenomenon that a language model trained under\nthe teacher forcing schema may perform poorly at the inference stage when its\npredictions are conditioned on its previous predictions unseen from the\ntraining corpus. Recently, several generative adversarial networks (GANs) and\nreinforcement learning (RL) methods have been introduced to alleviate this\nproblem. Nonetheless, a common issue in RL and GANs training is the sparsity of\nreward signals. In this paper, we adopt two simple strategies, multi-range\nreinforcing, and multi-entropy sampling, to amplify and denoise the reward\nsignal. Our model produces an improvement over competing models with regards to\nBLEU scores and road exam, a new metric we designed to measure the robustness\nagainst exposure bias in language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11377v4",
            "date": "2019-10-24 18:54:25+00:00",
            "title": "Clinical Concept Extraction: a Methodology Review",
            "authors": [
                "Sunyang Fu",
                "David Chen",
                "Huan He",
                "Sijia Liu",
                "Sungrim Moon",
                "Kevin J Peterson",
                "Feichen Shen",
                "Liwei Wang",
                "Yanshan Wang",
                "Andrew Wen",
                "Yiqing Zhao",
                "Sunghwan Sohn",
                "Hongfang Liu"
            ],
            "abstract": "Background Concept extraction, a subdomain of natural language processing\n(NLP) with a focus on extracting concepts of interest, has been adopted to\ncomputationally extract clinical information from text for a wide range of\napplications ranging from clinical decision support to care quality\nimprovement.\n  Objectives In this literature review, we provide a methodology review of\nclinical concept extraction, aiming to catalog development processes, available\nmethods and tools, and specific considerations when developing clinical concept\nextraction applications.\n  Methods Based on the Preferred Reporting Items for Systematic Reviews and\nMeta-Analyses (PRISMA) guidelines, a literature search was conducted for\nretrieving EHR-based information extraction articles written in English and\npublished from January 2009 through June 2019 from Ovid MEDLINE In-Process &\nOther Non-Indexed Citations, Ovid MEDLINE, Ovid EMBASE, Scopus, Web of Science,\nand the ACM Digital Library.\n  Results A total of 6,686 publications were retrieved. After title and\nabstract screening, 228 publications were selected. The methods used for\ndeveloping clinical concept extraction applications were discussed in this\nreview.",
            "journal ref": "Journal of Biomedical Informatics (2020): 103526",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.11450v1",
            "date": "2019-10-24 23:00:12+00:00",
            "title": "An Empirical Study of Efficient ASR Rescoring with Transformers",
            "authors": [
                "Hongzhao Huang",
                "Fuchun Peng"
            ],
            "abstract": "Neural language models (LMs) have been proved to significantly outperform\nclassical n-gram LMs for language modeling due to their superior abilities to\nmodel long-range dependencies in text and handle data sparsity problems. And\nrecently, well configured deep Transformers have exhibited superior performance\nover shallow stack of recurrent neural network layers for language modeling.\nHowever, these state-of-the-art deep Transformer models were mostly engineered\nto be deep with high model capacity, which makes it computationally inefficient\nand challenging to be deployed into large-scale real-world applications.\nTherefore, it is important to develop Transformer LMs that have relatively\nsmall model sizes, while still retaining good performance of those much larger\nmodels. In this paper, we aim to conduct empirical study on training\nTransformers with small parameter sizes in the context of ASR rescoring. By\ncombining techniques including subword units, adaptive softmax, large-scale\nmodel pre-training, and knowledge distillation, we show that we are able to\nsuccessfully train small Transformer LMs with significant relative word error\nrate reductions (WERR) through n-best rescoring. In particular, our experiments\non a video speech recognition dataset show that we are able to achieve WERRs\nranging from 6.46% to 7.17% while only with 5.5% to 11.9% parameter sizes of\nthe well-known large GPT model [1], whose WERR with rescoring on the same\ndataset is 7.58%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11470v1",
            "date": "2019-10-25 00:45:48+00:00",
            "title": "A Survey on Recent Advances in Named Entity Recognition from Deep Learning models",
            "authors": [
                "Vikas Yadav",
                "Steven Bethard"
            ],
            "abstract": "Named Entity Recognition (NER) is a key component in NLP systems for question\nanswering, information retrieval, relation extraction, etc. NER systems have\nbeen studied and developed widely for decades, but accurate systems using deep\nneural networks (NN) have only been introduced in the last few years. We\npresent a comprehensive survey of deep neural network architectures for NER,\nand contrast them with previous approaches to NER based on feature engineering\nand other supervised or semi-supervised learning algorithms. Our results\nhighlight the improvements achieved by neural networks, and show how\nincorporating some of the lessons learned from past work on feature-based NER\nsystems can yield further improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11621v2",
            "date": "2019-10-25 11:14:33+00:00",
            "title": "Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection",
            "authors": [
                "Shumin Deng",
                "Ningyu Zhang",
                "Jiaojian Kang",
                "Yichi Zhang",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "Event detection (ED), a sub-task of event extraction, involves identifying\ntriggers and categorizing event mentions. Existing methods primarily rely upon\nsupervised learning and require large-scale labeled event datasets which are\nunfortunately not readily available in many real-life applications. In this\npaper, we consider and reformulate the ED task with limited labeled data as a\nFew-Shot Learning problem. We propose a Dynamic-Memory-Based Prototypical\nNetwork (DMB-PN), which exploits Dynamic Memory Network (DMN) to not only learn\nbetter prototypes for event types, but also produce more robust sentence\nencodings for event mentions. Differing from vanilla prototypical networks\nsimply computing event prototypes by averaging, which only consume event\nmentions once, our model is more robust and is capable of distilling contextual\ninformation from event mentions for multiple times due to the multi-hop\nmechanism of DMNs. The experiments show that DMB-PN not only deals with sample\nscarcity better than a series of baseline models but also performs more\nrobustly when the variety of event types is relatively large and the instance\nquantity is extremely small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12197v1",
            "date": "2019-10-27 06:56:43+00:00",
            "title": "Look-up and Adapt: A One-shot Semantic Parser",
            "authors": [
                "Zhichu Lu",
                "Forough Arabshahi",
                "Igor Labutov",
                "Tom Mitchell"
            ],
            "abstract": "Computing devices have recently become capable of interacting with their end\nusers via natural language. However, they can only operate within a limited\n\"supported\" domain of discourse and fail drastically when faced with an\nout-of-domain utterance, mainly due to the limitations of their semantic\nparser. In this paper, we propose a semantic parser that generalizes to\nout-of-domain examples by learning a general strategy for parsing an unseen\nutterance through adapting the logical forms of seen utterances, instead of\nlearning to generate a logical form from scratch. Our parser maintains a memory\nconsisting of a representative subset of the seen utterances paired with their\nlogical forms. Given an unseen utterance, our parser works by looking up a\nsimilar utterance from the memory and adapting its logical form until it fits\nthe unseen utterance. Moreover, we present a data generation strategy for\nconstructing utterance-logical form pairs from different domains. Our results\nshow an improvement of up to 68.8% on one-shot parsing under two different\nevaluation settings compared to the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12279v2",
            "date": "2019-10-27 15:13:26+00:00",
            "title": "Memeify: A Large-Scale Meme Generation System",
            "authors": [
                "Suryatej Reddy Vyalla",
                "Vishaal Udandarao",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Interest in the research areas related to meme propagation and generation has\nbeen increasing rapidly in the last couple of years. Meme datasets available\nonline are either specific to a context or contain no class information. Here,\nwe prepare a large-scale dataset of memes with captions and class labels. The\ndataset consists of 1.1 million meme captions from 128 classes. We also provide\nreasoning for the existence of broad categories, called \"themes\" across the\nmeme dataset; each theme consists of multiple meme classes. Our generation\nsystem uses a trained state-of-the-art transformer-based model for caption\ngeneration by employing an encoder-decoder architecture. We develop a web\ninterface, called Memeify for users to generate memes of their choice, and\nexplain in detail, the working of individual components of the system. We also\nperform a qualitative evaluation of the generated memes by conducting a user\nstudy. A link to the demonstration of the Memeify system is\nhttps://youtu.be/P_Tfs0X-czs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12507v2",
            "date": "2019-10-28 09:06:00+00:00",
            "title": "A Survey on Knowledge Graph Embeddings with Literals: Which model links better Literal-ly?",
            "authors": [
                "Genet Asefa Gesese",
                "Russa Biswas",
                "Mehwish Alam",
                "Harald Sack"
            ],
            "abstract": "Knowledge Graphs (KGs) are composed of structured information about a\nparticular domain in the form of entities and relations. In addition to the\nstructured information KGs help in facilitating interconnectivity and\ninteroperability between different resources represented in the Linked Data\nCloud. KGs have been used in a variety of applications such as entity linking,\nquestion answering, recommender systems, etc. However, KG applications suffer\nfrom high computational and storage costs. Hence, there arises the necessity\nfor a representation able to map the high dimensional KGs into low dimensional\nspaces, i.e., embedding space, preserving structural as well as relational\ninformation. This paper conducts a survey of KG embedding models which not only\nconsider the structured information contained in the form of entities and\nrelations in a KG but also the unstructured information represented as literals\nsuch as text, numerical values, images, etc. Along with a theoretical analysis\nand comparison of the methods proposed so far for generating KG embeddings with\nliterals, an empirical evaluation of the different methods under identical\nsettings has been performed for the general task of link prediction.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.12619v1",
            "date": "2019-10-25 12:26:32+00:00",
            "title": "Is it a Fruit, an Apple or a Granny Smith? Predicting the Basic Level in a Concept Hierarchy",
            "authors": [
                "Laura Hollink",
                "Aysenur Bilgin",
                "Jacco van Ossenbruggen"
            ],
            "abstract": "The \"basic level\", according to experiments in cognitive psychology, is the\nlevel of abstraction in a hierarchy of concepts at which humans perform tasks\nquicker and with greater accuracy than at other levels. We argue that\napplications that use concept hierarchies - such as knowledge graphs,\nontologies or taxonomies - could significantly improve their user interfaces if\nthey `knew' which concepts are the basic level concepts. This paper examines to\nwhat extent the basic level can be learned from data. We test the utility of\nthree types of concept features, that were inspired by the basic level theory:\nlexical features, structural features and frequency features. We evaluate our\napproach on WordNet, and create a training set of manually labelled examples\nthat includes concepts from different domains. Our findings include that the\nbasic level concepts can be accurately identified within one domain. Concepts\nthat are difficult to label for humans are also harder to classify\nautomatically. Our experiments provide insight into how classification\nperformance across domains could be improved, which is necessary for\nidentification of basic level concepts on a larger scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12674v1",
            "date": "2019-10-28 13:46:42+00:00",
            "title": "A Comparison of Neural Network Training Methods for Text Classification",
            "authors": [
                "Anderson de Andrade"
            ],
            "abstract": "We study the impact of neural networks in text classification. Our focus is\non training deep neural networks with proper weight initialization and greedy\nlayer-wise pretraining. Results are compared with 1-layer neural networks and\nSupport Vector Machines. We work with a dataset of labeled messages from the\nTwitter microblogging service and aim to predict weather conditions. A feature\nextraction procedure specific for the task is proposed, which applies\ndimensionality reduction using Latent Semantic Analysis. Our results show that\nneural networks outperform Support Vector Machines with Gaussian kernels,\nnoticing performance gains from introducing additional hidden layers with\nnonlinearities. The impact of using Nesterov's Accelerated Gradient in\nbackpropagation is also studied. We conclude that deep neural networks are a\nreasonable approach for text classification and propose further ideas to\nimprove performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12702v1",
            "date": "2019-10-28 14:22:56+00:00",
            "title": "Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling",
            "authors": [
                "Nasser Zalmout",
                "Nizar Habash"
            ],
            "abstract": "Morphological tagging is challenging for morphologically rich languages due\nto the large target space and the need for more training data to minimize model\nsparsity. Dialectal variants of morphologically rich languages suffer more as\nthey tend to be more noisy and have less resources. In this paper we explore\nthe use of multitask learning and adversarial training to address morphological\nrichness and dialectal variations in the context of full morphological tagging.\nWe use multitask learning for joint morphological modeling for the features\nwithin two dialects, and as a knowledge-transfer scheme for cross-dialectal\nmodeling. We use adversarial training to learn dialect invariant features that\ncan help the knowledge-transfer scheme from the high to low-resource variants.\nWe work with two dialectal variants: Modern Standard Arabic (high-resource\n\"dialect\") and Egyptian Arabic (low-resource dialect) as a case study. Our\nmodels achieve state-of-the-art results for both. Furthermore, adversarial\ntraining provides more significant improvement when using smaller training\ndatasets in particular.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12708v1",
            "date": "2019-10-28 14:29:28+00:00",
            "title": "Evaluating Lottery Tickets Under Distributional Shifts",
            "authors": [
                "Shrey Desai",
                "Hongyuan Zhan",
                "Ahmed Aly"
            ],
            "abstract": "The Lottery Ticket Hypothesis suggests large, over-parameterized neural\nnetworks consist of small, sparse subnetworks that can be trained in isolation\nto reach a similar (or better) test accuracy. However, the initialization and\ngeneralizability of the obtained sparse subnetworks have been recently called\ninto question. Our work focuses on evaluating the initialization of sparse\nsubnetworks under distributional shifts. Specifically, we investigate the\nextent to which a sparse subnetwork obtained in a source domain can be\nre-trained in isolation in a dissimilar, target domain. In addition, we examine\nthe effects of different initialization strategies at transfer-time. Our\nexperiments show that sparse subnetworks obtained through lottery ticket\ntraining do not simply overfit to particular domains, but rather reflect an\ninductive bias of deep neural networks that can be exploited in multiple\ndomains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12941v1",
            "date": "2019-10-28 19:57:02+00:00",
            "title": "A Hierarchical Location Prediction Neural Network for Twitter User Geolocation",
            "authors": [
                "Binxuan Huang",
                "Kathleen M. Carley"
            ],
            "abstract": "Accurate estimation of user location is important for many online services.\nPrevious neural network based methods largely ignore the hierarchical structure\namong locations. In this paper, we propose a hierarchical location prediction\nneural network for Twitter user geolocation. Our model first predicts the home\ncountry for a user, then uses the country result to guide the city-level\nprediction. In addition, we employ a character-aware word embedding layer to\novercome the noisy information in tweets. With the feature fusion layer, our\nmodel can accommodate various feature combinations and achieves\nstate-of-the-art results over three commonly used benchmarks under different\nfeature settings. It not only improves the prediction accuracy but also greatly\nreduces the mean error distance.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1910.12995v3",
            "date": "2019-10-28 22:41:55+00:00",
            "title": "A Simple but Effective BERT Model for Dialog State Tracking on Resource-Limited Systems",
            "authors": [
                "Tuan Manh Lai",
                "Quan Hung Tran",
                "Trung Bui",
                "Daisuke Kihara"
            ],
            "abstract": "In a task-oriented dialog system, the goal of dialog state tracking (DST) is\nto monitor the state of the conversation from the dialog history. Recently,\nmany deep learning based methods have been proposed for the task. Despite their\nimpressive performance, current neural architectures for DST are typically\nheavily-engineered and conceptually complex, making it difficult to implement,\ndebug, and maintain them in a production setting. In this work, we propose a\nsimple but effective DST model based on BERT. In addition to its simplicity,\nour approach also has a number of other advantages: (a) the number of\nparameters does not grow with the ontology size (b) the model can operate in\nsituations where the domain ontology may change dynamically. Experimental\nresults demonstrate that our BERT-based model outperforms previous methods by a\nlarge margin, achieving new state-of-the-art results on the standard WoZ 2.0\ndataset. Finally, to make the model small and fast enough for\nresource-restricted systems, we apply the knowledge distillation method to\ncompress our model. The final compressed model achieves comparable results with\nthe original model while being 8x smaller and 7x faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13034v1",
            "date": "2019-10-29 01:38:24+00:00",
            "title": "Big Bidirectional Insertion Representations for Documents",
            "authors": [
                "Lala Li",
                "William Chan"
            ],
            "abstract": "The Insertion Transformer is well suited for long form text generation due to\nits parallel generation capabilities, requiring $O(\\log_2 n)$ generation steps\nto generate $n$ tokens. However, modeling long sequences is difficult, as there\nis more ambiguity captured in the attention mechanism. This work proposes the\nBig Bidirectional Insertion Representations for Documents (Big BIRD), an\ninsertion-based model for document-level translation tasks. We scale up the\ninsertion-based models to long form documents. Our key contribution is\nintroducing sentence alignment via sentence-positional embeddings between the\nsource and target document. We show an improvement of +4.3 BLEU on the WMT'19\nEnglish$\\rightarrow$German document-level translation task compared with the\nInsertion Transformer baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13105v2",
            "date": "2019-10-29 06:41:30+00:00",
            "title": "JarKA: Modeling Attribute Interactions for Cross-lingual Knowledge Alignment",
            "authors": [
                "Bo Chen",
                "Jing Zhang",
                "Xiaobin Tang",
                "Hong Chen",
                "Cuiping Li"
            ],
            "abstract": "Abstract. Cross-lingual knowledge alignment is the cornerstone in building a\ncomprehensive knowledge graph (KG), which can benefit various knowledge-driven\napplications. As the structures of KGs are usually sparse, attributes of\nentities may play an important role in aligning the entities. However, the\nheterogeneity of the attributes across KGs prevents from accurately embedding\nand comparing entities. To deal with the issue, we propose to model the\ninteractions between attributes, instead of globally embedding an entity with\nall the attributes. We further propose a joint framework to merge the\nalignments inferred from the attributes and the structures. Experimental\nresults show that the proposed model outperforms the state-of-art baselines by\nup to 38.48% HitRatio@1. The results also demonstrate that our model can infer\nthe alignments between attributes, relationships and values, in addition to\nentities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13108v1",
            "date": "2019-10-29 06:45:24+00:00",
            "title": "Generating Questions for Knowledge Bases via Incorporating Diversified Contexts and Answer-Aware Loss",
            "authors": [
                "Cao Liu",
                "Kang Liu",
                "Shizhu He",
                "Zaiqing Nie",
                "Jun Zhao"
            ],
            "abstract": "We tackle the task of question generation over knowledge bases. Conventional\nmethods for this task neglect two crucial research issues: 1) the given\npredicate needs to be expressed; 2) the answer to the generated question needs\nto be definitive. In this paper, we strive toward the above two issues via\nincorporating diversified contexts and answer-aware loss. Specifically, we\npropose a neural encoder-decoder model with multi-level copy mechanisms to\ngenerate such questions. Furthermore, the answer aware loss is introduced to\nmake generated questions corresponding to more definitive answers. Experiments\ndemonstrate that our model achieves state-of-the-art performance. Meanwhile,\nsuch generated question can express the given predicate and correspond to a\ndefinitive answer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13114v2",
            "date": "2019-10-29 06:56:46+00:00",
            "title": "Contrastive Attention Mechanism for Abstractive Sentence Summarization",
            "authors": [
                "Xiangyu Duan",
                "Hoongfei Yu",
                "Mingming Yin",
                "Min Zhang",
                "Weihua Luo",
                "Yue Zhang"
            ],
            "abstract": "We propose a contrastive attention mechanism to extend the\nsequence-to-sequence framework for abstractive sentence summarization task,\nwhich aims to generate a brief summary of a given source sentence. The proposed\ncontrastive attention mechanism accommodates two categories of attention: one\nis the conventional attention that attends to relevant parts of the source\nsentence, the other is the opponent attention that attends to irrelevant or\nless relevant parts of the source sentence. Both attentions are trained in an\nopposite way so that the contribution from the conventional attention is\nencouraged and the contribution from the opponent attention is discouraged\nthrough a novel softmax and softmin functionality. Experiments on benchmark\ndatasets show that, the proposed contrastive attention mechanism is more\nfocused on the relevant parts for the summary than the conventional attention\nmechanism, and greatly advances the state-of-the-art performance on the\nabstractive sentence summarization task. We release the code at\nhttps://github.com/travel-go/Abstractive-Text-Summarization",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13291v1",
            "date": "2019-10-29 14:28:34+00:00",
            "title": "Sentence Embeddings for Russian NLU",
            "authors": [
                "Dmitry Popov",
                "Alexander Pugachev",
                "Polina Svyatokum",
                "Elizaveta Svitanko",
                "Ekaterina Artemova"
            ],
            "abstract": "We investigate the performance of sentence embeddings models on several tasks\nfor the Russian language. In our comparison, we include such tasks as multiple\nchoice question answering, next sentence prediction, and paraphrase\nidentification. We employ FastText embeddings as a baseline and compare it to\nELMo and BERT embeddings. We conduct two series of experiments, using both\nunsupervised (i.e., based on similarity measure only) and supervised approaches\nfor the tasks. Finally, we present datasets for multiple choice question\nanswering and next sentence prediction in Russian.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13294v2",
            "date": "2019-10-29 14:32:54+00:00",
            "title": "Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control",
            "authors": [
                "Mo Yu",
                "Shiyu Chang",
                "Yang Zhang",
                "Tommi S. Jaakkola"
            ],
            "abstract": "Selective rationalization has become a common mechanism to ensure that\npredictive models reveal how they use any available features. The selection may\nbe soft or hard, and identifies a subset of input features relevant for\nprediction. The setup can be viewed as a co-operate game between the selector\n(aka rationale generator) and the predictor making use of only the selected\nfeatures. The co-operative setting may, however, be compromised for two\nreasons. First, the generator typically has no direct access to the outcome it\naims to justify, resulting in poor performance. Second, there's typically no\ncontrol exerted on the information left outside the selection. We revise the\noverall co-operative framework to address these challenges. We introduce an\nintrospective model which explicitly predicts and incorporates the outcome into\nthe selection process. Moreover, we explicitly control the rationale complement\nvia an adversary so as not to leave any useful information out of the\nselection. We show that the two complementary mechanisms maintain both high\npredictive accuracy and lead to comprehensive rationales.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13339v2",
            "date": "2019-10-29 15:56:33+00:00",
            "title": "Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning",
            "authors": [
                "Alon Jacovi",
                "Gang Niu",
                "Yoav Goldberg",
                "Masashi Sugiyama"
            ],
            "abstract": "We consider the situation in which a user has collected a small set of\ndocuments on a cohesive topic, and they want to retrieve additional documents\non this topic from a large collection. Information Retrieval (IR) solutions\ntreat the document set as a query, and look for similar documents in the\ncollection. We propose to extend the IR approach by treating the problem as an\ninstance of positive-unlabeled (PU) learning -- i.e., learning binary\nclassifiers from only positive and unlabeled data, where the positive data\ncorresponds to the query documents, and the unlabeled data is the results\nreturned by the IR engine. Utilizing PU learning for text with big neural\nnetworks is a largely unexplored field. We discuss various challenges in\napplying PU learning to the setting, including an unknown class prior,\nextremely imbalanced data and large-scale accurate evaluation of models, and we\npropose solutions and empirically validate them. We demonstrate the\neffectiveness of the method using a series of experiments of retrieving PubMed\nabstracts adhering to fine-grained topics. We demonstrate improvements over the\nbase IR solution and other baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13437v1",
            "date": "2019-10-29 17:54:36+00:00",
            "title": "An Empirical Study of Generation Order for Machine Translation",
            "authors": [
                "William Chan",
                "Mitchell Stern",
                "Jamie Kiros",
                "Jakob Uszkoreit"
            ],
            "abstract": "In this work, we present an empirical study of generation order for machine\ntranslation. Building on recent advances in insertion-based modeling, we first\nintroduce a soft order-reward framework that enables us to train models to\nfollow arbitrary oracle generation policies. We then make use of this framework\nto explore a large variety of generation orders, including uninformed orders,\nlocation-based orders, frequency-based orders, content-based orders, and\nmodel-based orders. Curiously, we find that for the WMT'14 English $\\to$ German\ntranslation task, order does not have a substantial impact on output quality,\nwith unintuitive orderings such as alphabetical and shortest-first matching the\nperformance of a standard Transformer. This demonstrates that traditional\nleft-to-right generation is not strictly necessary to achieve high performance.\nOn the other hand, results on the WMT'18 English $\\to$ Chinese task tend to\nvary more widely, suggesting that translation for less well-aligned language\npairs may be more sensitive to generation order.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13466v2",
            "date": "2019-10-29 18:14:14+00:00",
            "title": "Ordered Memory",
            "authors": [
                "Yikang Shen",
                "Shawn Tan",
                "Arian Hosseini",
                "Zhouhan Lin",
                "Alessandro Sordoni",
                "Aaron Courville"
            ],
            "abstract": "Stack-augmented recurrent neural networks (RNNs) have been of interest to the\ndeep learning community for some time. However, the difficulty of training\nmemory models remains a problem obstructing the widespread use of such models.\nIn this paper, we propose the Ordered Memory architecture. Inspired by Ordered\nNeurons (Shen et al., 2018), we introduce a new attention-based mechanism and\nuse its cumulative probability to control the writing and erasing operation of\nthe memory. We also introduce a new Gated Recursive Cell to compose lower-level\nrepresentations into higher-level representation. We demonstrate that our model\nachieves strong performance on the logical inference task (Bowman et al.,\n2015)and the ListOps (Nangia and Bowman, 2018) task. We can also interpret the\nmodel to retrieve the induced tree structure, and find that these induced\nstructures align with the ground truth. Finally, we evaluate our model on the\nStanford SentimentTreebank tasks (Socher et al., 2013), and find that it\nperforms comparatively with the state-of-the-art methods in the literature.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.13634v1",
            "date": "2019-10-30 02:46:04+00:00",
            "title": "An Augmented Transformer Architecture for Natural Language Generation Tasks",
            "authors": [
                "Hailiang Li",
                "Adele Y. C. Wang",
                "Yang Liu",
                "Du Tang",
                "Zhibin Lei",
                "Wenye Li"
            ],
            "abstract": "The Transformer based neural networks have been showing significant\nadvantages on most evaluations of various natural language processing and other\nsequence-to-sequence tasks due to its inherent architecture based\nsuperiorities. Although the main architecture of the Transformer has been\ncontinuously being explored, little attention was paid to the positional\nencoding module. In this paper, we enhance the sinusoidal positional encoding\nalgorithm by maximizing the variances between encoded consecutive positions to\nobtain additional promotion. Furthermore, we propose an augmented Transformer\narchitecture encoded with additional linguistic knowledge, such as the\nPart-of-Speech (POS) tagging, to boost the performance on some natural language\ngeneration tasks, e.g., the automatic translation and summarization tasks.\nExperiments show that the proposed architecture attains constantly superior\nresults compared to the vanilla Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14080v1",
            "date": "2019-10-30 18:47:37+00:00",
            "title": "Contextual Text Denoising with Masked Language Models",
            "authors": [
                "Yifu Sun",
                "Haoming Jiang"
            ],
            "abstract": "Recently, with the help of deep learning models, significant advances have\nbeen made in different Natural Language Processing (NLP) tasks. Unfortunately,\nstate-of-the-art models are vulnerable to noisy texts. We propose a new\ncontextual text denoising algorithm based on the ready-to-use masked language\nmodel. The proposed algorithm does not require retraining of the model and can\nbe integrated into any NLP system without additional training on paired\ncleaning training data. We evaluate our method under synthetic noise and\nnatural noise and show that the proposed algorithm can use context information\nto correct noise text and improve the performance of noisy inputs in several\ndownstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14208v2",
            "date": "2019-10-31 01:56:33+00:00",
            "title": "Hidden State Guidance: Improving Image Captioning using An Image Conditioned Autoencoder",
            "authors": [
                "Jialin Wu",
                "Raymond J. Mooney"
            ],
            "abstract": "Most RNN-based image captioning models receive supervision on the output\nwords to mimic human captions. Therefore, the hidden states can only receive\nnoisy gradient signals via layers of back-propagation through time, leading to\nless accurate generated captions. Consequently, we propose a novel framework,\nHidden State Guidance (HSG), that matches the hidden states in the caption\ndecoder to those in a teacher decoder trained on an easier task of autoencoding\nthe captions conditioned on the image. During training with the REINFORCE\nalgorithm, the conventional rewards are sentence-based evaluation metrics\nequally distributed to each generated word, no matter their relevance. HSG\nprovides a word-level reward that helps the model learn better hidden\nrepresentations. Experimental results demonstrate that HSG clearly outperforms\nvarious state-of-the-art caption decoders using either raw images or detected\nobjects as inputs.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.14229v1",
            "date": "2019-10-31 03:08:10+00:00",
            "title": "Cascaded LSTMs based Deep Reinforcement Learning for Goal-driven Dialogue",
            "authors": [
                "Yue Ma",
                "Xiaojie Wang",
                "Zhenjiang Dong",
                "Hong Chen"
            ],
            "abstract": "This paper proposes a deep neural network model for joint modeling Natural\nLanguage Understanding (NLU) and Dialogue Management (DM) in goal-driven\ndialogue systems. There are three parts in this model. A Long Short-Term Memory\n(LSTM) at the bottom of the network encodes utterances in each dialogue turn\ninto a turn embedding. Dialogue embeddings are learned by a LSTM at the middle\nof the network, and updated by the feeding of all turn embeddings. The top part\nis a forward Deep Neural Network which converts dialogue embeddings into the\nQ-values of different dialogue actions. The cascaded LSTMs based reinforcement\nlearning network is jointly optimized by making use of the rewards received at\neach dialogue turn as the only supervision information. There is no explicit\nNLU and dialogue states in the network. Experimental results show that our\nmodel outperforms both traditional Markov Decision Process (MDP) model and\nsingle LSTM with Deep Q-Network on meeting room booking tasks. Visualization of\ndialogue embeddings illustrates that the model can learn the representation of\ndialogue states.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14243v1",
            "date": "2019-10-31 03:56:32+00:00",
            "title": "DiaNet: BERT and Hierarchical Attention Multi-Task Learning of Fine-Grained Dialect",
            "authors": [
                "Muhammad Abdul-Mageed",
                "Chiyu Zhang",
                "AbdelRahim Elmadany",
                "Arun Rajendran",
                "Lyle Ungar"
            ],
            "abstract": "Prediction of language varieties and dialects is an important language\nprocessing task, with a wide range of applications. For Arabic, the native\ntongue of ~ 300 million people, most varieties remain unsupported. To ease this\nbottleneck, we present a very large scale dataset covering 319 cities from all\n21 Arab countries. We introduce a hierarchical attention multi-task learning\n(HA-MTL) approach for dialect identification exploiting our data at the city,\nstate, and country levels. We also evaluate use of BERT on the three tasks,\ncomparing it to the MTL approach. We benchmark and release our data and models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14270v2",
            "date": "2019-10-31 06:06:56+00:00",
            "title": "Parameter Sharing Decoder Pair for Auto Composing",
            "authors": [
                "Xu Zhao"
            ],
            "abstract": "Auto Composing is an active and appealing research area in the past few\nyears, and lots of efforts have been put into inventing more robust models to\nsolve this problem. With the fast evolution of deep learning techniques, some\ndeep neural network-based language models are becoming dominant. Notably, the\ntransformer structure has been proven to be very efficient and promising in\nmodeling texts. However, the transformer-based language models usually contain\nhuge number of parameters and the size of the model is usually too large to put\nin production for some storage limited applications. In this paper, we propose\na parameter sharing decoder pair (PSDP), which reduces the number of parameters\ndramatically and at the same time maintains the capability of generating\nunderstandable and reasonable compositions. Works created by the proposed model\nare presented to demonstrate the effectiveness of the model.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.14296v2",
            "date": "2019-10-31 08:14:51+00:00",
            "title": "LIMIT-BERT : Linguistic Informed Multi-Task BERT",
            "authors": [
                "Junru Zhou",
                "Zhuosheng Zhang",
                "Hai Zhao",
                "Shuailiang Zhang"
            ],
            "abstract": "In this paper, we present a Linguistic Informed Multi-Task BERT (LIMIT-BERT)\nfor learning language representations across multiple linguistic tasks by\nMulti-Task Learning (MTL). LIMIT-BERT includes five key linguistic syntax and\nsemantics tasks: Part-Of-Speech (POS) tags, constituent and dependency\nsyntactic parsing, span and dependency semantic role labeling (SRL). Besides,\nLIMIT-BERT adopts linguistics mask strategy: Syntactic and Semantic Phrase\nMasking which mask all of the tokens corresponding to a syntactic/semantic\nphrase. Different from recent Multi-Task Deep Neural Networks (MT-DNN) (Liu et\nal., 2019), our LIMIT-BERT is linguistically motivated and learning in a\nsemi-supervised method which provides large amounts of linguistic-task data as\nsame as BERT learning corpus. As a result, LIMIT-BERT not only improves\nlinguistic tasks performance but also benefits from a regularization effect and\nlinguistic information that leads to more general representations to help adapt\nto new tasks and domains. LIMIT-BERT obtains new state-of-the-art or\ncompetitive results on both span and dependency semantic parsing on Propbank\nbenchmarks and both dependency and constituent syntactic parsing on Penn\nTreebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14395v1",
            "date": "2019-10-31 11:49:32+00:00",
            "title": "Great New Design: How Do We Talk about Media Architecture in Social Media",
            "authors": [
                "Selena Savic"
            ],
            "abstract": "In social media, we communicate through pictures, videos, short codes, links,\npartial phrases. It is a rich, and digitally documented communication channel\nthat relies on a multitude of media and forms. These channels are sorted by\nalgorithms as organizers of discourse, mostly with the goal of channeling\nattention. In this research, we used Twitter to study the way Media\nArchitecture is discussed within the community of architects, designers,\nresearchers and policy makers. We look at the way they spontaneously share\nopinions on their engagement with digital infrastructures, networked places and\nhybrid public spaces. What can we do with all those opinions? We propose here\nthe use of text-mining and machine learning techniques to identify important\nconcepts and patterns in this prolific communication stream. We discuss how\nsuch techniques could inform the practice and emergence of future trends.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14497v2",
            "date": "2019-10-31 14:34:14+00:00",
            "title": "Probabilistic Bias Mitigation in Word Embeddings",
            "authors": [
                "Hailey James",
                "David Alvarez-Melis"
            ],
            "abstract": "It has been shown that word embeddings derived from large corpora tend to\nincorporate biases present in their training data. Various methods for\nmitigating these biases have been proposed, but recent work has demonstrated\nthat these methods hide but fail to truly remove the biases, which can still be\nobserved in word nearest-neighbor statistics. In this work we propose a\nprobabilistic view of word embedding bias. We leverage this framework to\npresent a novel method for mitigating bias which relies on probabilistic\nobservations to yield a more robust bias mitigation algorithm. We demonstrate\nthat this method effectively reduces bias according to three separate measures\nof bias while maintaining embedding quality across various popular benchmark\nsemantic tasks",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14549v1",
            "date": "2019-10-31 15:51:04+00:00",
            "title": "Positional Attention-based Frame Identification with BERT: A Deep Learning Approach to Target Disambiguation and Semantic Frame Selection",
            "authors": [
                "Sang-Sang Tan",
                "Jin-Cheon Na"
            ],
            "abstract": "Semantic parsing is the task of transforming sentences from natural language\ninto formal representations of predicate-argument structures. Under this\nresearch area, frame-semantic parsing has attracted much interest. This parsing\napproach leverages the lexical information defined in FrameNet to associate\nmarked predicates or targets with semantic frames, thereby assigning semantic\nroles to sentence components based on pre-specified frame elements in FrameNet.\nIn this paper, a deep neural network architecture known as Positional\nAttention-based Frame Identification with BERT (PAFIBERT) is presented as a\nsolution to the frame identification subtask in frame-semantic parsing.\nAlthough the importance of this subtask is well-established, prior research has\nyet to find a robust solution that works satisfactorily for both in-domain and\nout-of-domain data. This study thus set out to improve frame identification in\nlight of recent advancements of language modeling and transfer learning in\nnatural language processing. The proposed method is partially empowered by\nBERT, a pre-trained language model that excels at capturing contextual\ninformation in texts. By combining the language representation power of BERT\nwith a position-based attention mechanism, PAFIBERT is able to attend to\ntarget-specific contexts in sentences for disambiguating targets and\nassociating them with the most suitable semantic frames. Under various\nexperimental settings, PAFIBERT outperformed existing solutions by a\nsignificant margin, achieving new state-of-the-art results for both in-domain\nand out-of-domain benchmark test sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14599v2",
            "date": "2019-10-31 16:50:43+00:00",
            "title": "Adversarial NLI: A New Benchmark for Natural Language Understanding",
            "authors": [
                "Yixin Nie",
                "Adina Williams",
                "Emily Dinan",
                "Mohit Bansal",
                "Jason Weston",
                "Douwe Kiela"
            ],
            "abstract": "We introduce a new large-scale NLI benchmark dataset, collected via an\niterative, adversarial human-and-model-in-the-loop procedure. We show that\ntraining models on this new dataset leads to state-of-the-art performance on a\nvariety of popular NLI benchmarks, while posing a more difficult challenge with\nits new test set. Our analysis sheds light on the shortcomings of current\nstate-of-the-art models, and shows that non-expert annotators are successful at\nfinding their weaknesses. The data collection method can be applied in a\nnever-ending learning scenario, becoming a moving target for NLU, rather than a\nstatic benchmark that will quickly saturate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00202v3",
            "date": "2019-11-01 05:07:06+00:00",
            "title": "Forget Me Not: Reducing Catastrophic Forgetting for Domain Adaptation in Reading Comprehension",
            "authors": [
                "Y. Xu",
                "X. Zhong",
                "A. J. J. Yepes",
                "J. H. Lau"
            ],
            "abstract": "The creation of large-scale open domain reading comprehension data sets in\nrecent years has enabled the development of end-to-end neural comprehension\nmodels with promising results. To use these models for domains with limited\ntraining data, one of the most effective approach is to first pretrain them on\nlarge out-of-domain source data and then fine-tune them with the limited target\ndata. The caveat of this is that after fine-tuning the comprehension models\ntend to perform poorly in the source domain, a phenomenon known as catastrophic\nforgetting. In this paper, we explore methods that overcome catastrophic\nforgetting during fine-tuning without assuming access to data from the source\ndomain. We introduce new auxiliary penalty terms and observe the best\nperformance when a combination of auxiliary penalty terms is used to regularise\nthe fine-tuning process for adapting comprehension models. To test our methods,\nwe develop and release 6 narrow domain data sets that could potentially be used\nas reading comprehension benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00203v2",
            "date": "2019-11-01 05:16:12+00:00",
            "title": "Improving Generalization of Transformer for Speech Recognition with Parallel Schedule Sampling and Relative Positional Embedding",
            "authors": [
                "Pan Zhou",
                "Ruchao Fan",
                "Wei Chen",
                "Jia Jia"
            ],
            "abstract": "Transformer has shown promising results in many sequence to sequence\ntransformation tasks recently. It utilizes a number of feed-forward\nself-attention layers to replace the recurrent neural networks (RNN) in\nattention-based encoder decoder (AED) architecture. Self-attention layer learns\ntemporal dependence by incorporating sinusoidal positional embedding of tokens\nin a sequence for parallel computing. Quicker iteration speed in training than\nsequential operation of RNN can be obtained. Deeper layers of the transformer\nalso make it perform better than RNN-based AED. However, this parallelization\nability is lost when applying scheduled sampling training. Self-attention with\nsinusoidal positional embedding may cause performance degradations for longer\nsequences that have similar acoustic or semantic information at different\npositions as well. To address these problems, we propose to use parallel\nscheduled sampling (PSS) and relative positional embedding (RPE) to help the\ntransformer generalize to unseen data. Our proposed methods achieve a 7%\nrelative improvement for short utterances and a 70% relative gain for long\nutterances on a 10,000-hour Mandarin ASR task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00226v1",
            "date": "2019-11-01 06:53:12+00:00",
            "title": "Generating Justifications for Norm-Related Agent Decisions",
            "authors": [
                "Daniel Kasenberg",
                "Antonio Roque",
                "Ravenna Thielstrom",
                "Meia Chita-Tegmark",
                "Matthias Scheutz"
            ],
            "abstract": "We present an approach to generating natural language justifications of\ndecisions derived from norm-based reasoning. Assuming an agent which maximally\nsatisfies a set of rules specified in an object-oriented temporal logic, the\nuser can ask factual questions (about the agent's rules, actions, and the\nextent to which the agent violated the rules) as well as \"why\" questions that\nrequire the agent comparing actual behavior to counterfactual trajectories with\nrespect to these rules. To produce natural-sounding explanations, we focus on\nthe subproblem of producing natural language clauses from statements in a\nfragment of temporal logic, and then describe how to embed these clauses into\nexplanatory sentences. We use a human judgment evaluation on a testbed task to\ncompare our approach to variants in terms of intelligibility, mental model and\nperceived trust.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00229v1",
            "date": "2019-11-01 07:01:52+00:00",
            "title": "Engaging in Dialogue about an Agent's Norms and Behaviors",
            "authors": [
                "Daniel Kasenberg",
                "Antonio Roque",
                "Ravenna Thielstrom",
                "Matthias Scheutz"
            ],
            "abstract": "We present a set of capabilities allowing an agent planning with moral and\nsocial norms represented in temporal logic to respond to queries about its\nnorms and behaviors in natural language, and for the human user to add and\nremove norms directly in natural language. The user may also pose hypothetical\nmodifications to the agent's norms and inquire about their effects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00274v1",
            "date": "2019-11-01 09:20:14+00:00",
            "title": "Kernelized Bayesian Softmax for Text Generation",
            "authors": [
                "Ning Miao",
                "Hao Zhou",
                "Chengqi Zhao",
                "Wenxian Shi",
                "Lei Li"
            ],
            "abstract": "Neural models for text generation require a softmax layer with proper token\nembeddings during the decoding phase. Most existing approaches adopt single\npoint embedding for each token. However, a word may have multiple senses\naccording to different context, some of which might be distinct. In this paper,\nwe propose KerBS, a novel approach for learning better embeddings for text\ngeneration. KerBS embodies two advantages: (a) it employs a Bayesian\ncomposition of embeddings for words with multiple senses; (b) it is adaptive to\nsemantic variances of words and robust to rare sentence context by imposing\nlearned kernels to capture the closeness of words (senses) in the embedding\nspace. Empirical studies show that KerBS significantly boosts the performance\nof several text generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00288v2",
            "date": "2019-11-01 10:20:05+00:00",
            "title": "Efficient Feature Selection techniques for Sentiment Analysis",
            "authors": [
                "Avinash Madasu",
                "Sivasankar E"
            ],
            "abstract": "Sentiment analysis is a domain of study that focuses on identifying and\nclassifying the ideas expressed in the form of text into positive, negative and\nneutral polarities. Feature selection is a crucial process in machine learning.\nIn this paper, we aim to study the performance of different feature selection\ntechniques for sentiment analysis. Term Frequency Inverse Document Frequency\n(TF-IDF) is used as the feature extraction technique for creating feature\nvocabulary. Various Feature Selection (FS) techniques are experimented to\nselect the best set of features from feature vocabulary. The selected features\nare trained using different machine learning classifiers Logistic Regression\n(LR), Support Vector Machines (SVM), Decision Tree (DT) and Naive Bayes (NB).\nEnsemble techniques Bagging and Random Subspace are applied on classifiers to\nenhance the performance on sentiment analysis. We show that, when the best FS\ntechniques are trained using ensemble methods achieve remarkable results on\nsentiment analysis. We also compare the performance of FS methods trained using\nBagging, Random Subspace with varied neural network architectures. We show that\nFS techniques trained using ensemble classifiers outperform neural networks\nrequiring significantly less training time and parameters thereby eliminating\nthe need for extensive hyper-parameter tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00461v1",
            "date": "2019-11-01 16:52:02+00:00",
            "title": "On the Unintended Social Bias of Training Language Generation Models with Data from Local Media",
            "authors": [
                "Omar U. Florez"
            ],
            "abstract": "There are concerns that neural language models may preserve some of the\nstereotypes of the underlying societies that generate the large corpora needed\nto train these models. For example, gender bias is a significant problem when\ngenerating text, and its unintended memorization could impact the user\nexperience of many applications (e.g., the smart-compose feature in Gmail).\n  In this paper, we introduce a novel architecture that decouples the\nrepresentation learning of a neural model from its memory management role. This\narchitecture allows us to update a memory module with an equal ratio across\ngender types addressing biased correlations directly in the latent space. We\nexperimentally show that our approach can mitigate the gender bias\namplification in the automatic generation of articles news while providing\nsimilar perplexity values when extending the Sequence2Sequence architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00492v1",
            "date": "2019-11-01 17:59:38+00:00",
            "title": "Reasoning Over Paths via Knowledge Base Completion",
            "authors": [
                "Saatviga Sudhahar",
                "Ian Roberts",
                "Andrea Pierleoni"
            ],
            "abstract": "Reasoning over paths in large scale knowledge graphs is an important problem\nfor many applications. In this paper we discuss a simple approach to\nautomatically build and rank paths between a source and target entity pair with\nlearned embeddings using a knowledge base completion model (KBC). We assembled\na knowledge graph by mining the available biomedical scientific literature and\nextracted a set of high frequency paths to use for validation. We demonstrate\nthat our method is able to effectively rank a list of known paths between a\npair of entities and also come up with plausible paths that are not present in\nthe knowledge graph. For a given entity pair we are able to reconstruct the\nhighest ranking path 60% of the time within the the top 10 ranked paths and\nachieve 49% mean average precision. Our approach is compositional since any KBC\nmodel that can produce vector representations of entities can be used.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.00536v3",
            "date": "2019-11-01 18:16:54+00:00",
            "title": "DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation",
            "authors": [
                "Yizhe Zhang",
                "Siqi Sun",
                "Michel Galley",
                "Yen-Chun Chen",
                "Chris Brockett",
                "Xiang Gao",
                "Jianfeng Gao",
                "Jingjing Liu",
                "Bill Dolan"
            ],
            "abstract": "We present a large, tunable neural conversational response generation model,\nDialoGPT (dialogue generative pre-trained transformer). Trained on 147M\nconversation-like exchanges extracted from Reddit comment chains over a period\nspanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch\ntransformer to attain a performance close to human both in terms of automatic\nand human evaluation in single-turn dialogue settings. We show that\nconversational systems that leverage DialoGPT generate more relevant,\ncontentful and context-consistent responses than strong baseline systems. The\npre-trained model and training pipeline are publicly released to facilitate\nresearch into neural response generation and the development of more\nintelligent open-domain dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00637v1",
            "date": "2019-11-02 03:39:19+00:00",
            "title": "Sentence-Level BERT and Multi-Task Learning of Age and Gender in Social Media",
            "authors": [
                "Muhammad Abdul-Mageed",
                "Chiyu Zhang",
                "Arun Rajendran",
                "AbdelRahim Elmadany",
                "Michael Przystupa",
                "Lyle Ungar"
            ],
            "abstract": "Social media currently provide a window on our lives, making it possible to\nlearn how people from different places, with different backgrounds, ages, and\ngenders use language. In this work we exploit a newly-created Arabic dataset\nwith ground truth age and gender labels to learn these attributes both\nindividually and in a multi-task setting at the sentence level. Our models are\nbased on variations of deep bidirectional neural networks. More specifically,\nwe build models with gated recurrent units and bidirectional encoder\nrepresentations from transformers (BERT). We show the utility of multi-task\nlearning (MTL) on the two tasks and identify task-specific attention as a\nsuperior choice in this context. We also find that a single-task BERT model\noutperform our best MTL models on the two tasks. We report tweet-level accuracy\nof 51.43% for the age task (three-way) and 65.30% on the gender task (binary),\nboth of which outperforms our baselines with a large margin. Our models are\nlanguage-agnostic, and so can be applied to other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00643v1",
            "date": "2019-11-02 04:06:30+00:00",
            "title": "Credibility-based Fake News Detection",
            "authors": [
                "Niraj Sitaula",
                "Chilukuri K. Mohan",
                "Jennifer Grygiel",
                "Xinyi Zhou",
                "Reza Zafarani"
            ],
            "abstract": "Fake news can significantly misinform people who often rely on online sources\nand social media for their information. Current research on fake news detection\nhas mostly focused on analyzing fake news content and how it propagates on a\nnetwork of users. In this paper, we emphasize the detection of fake news by\nassessing its credibility. By analyzing public fake news data, we show that\ninformation on news sources (and authors) can be a strong indicator of\ncredibility. Our findings suggest that an author's history of association with\nfake news, and the number of authors of a news article, can play a significant\nrole in detecting fake news. Our approach can help improve traditional fake\nnews detection methods, wherein content features are often used to detect fake\nnews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00712v1",
            "date": "2019-11-02 13:25:15+00:00",
            "title": "How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering",
            "authors": [
                "Sanjay Kamath",
                "Brigitte Grau",
                "Yue Ma"
            ],
            "abstract": "Using deep learning models on small scale datasets would result in\noverfitting. To overcome this problem, the process of pre-training a model and\nfine-tuning it to the small scale dataset has been used extensively in domains\nsuch as image processing. Similarly for question answering, pre-training and\nfine-tuning can be done in several ways. Commonly reading comprehension models\nare used for pre-training, but we show that other types of pre-training can\nwork better. We compare two pre-training models based on reading comprehension\nand open domain question answering models and determine the performance when\nfine-tuned and tested over BIOASQ question answering dataset. We find open\ndomain question answering model to be a better fit for this task rather than\nreading comprehension model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00845v1",
            "date": "2019-11-03 07:11:58+00:00",
            "title": "Low-dimensional Semantic Space: from Text to Word Embedding",
            "authors": [
                "Xiaolei Lu",
                "Bin Ni"
            ],
            "abstract": "This article focuses on the study of Word Embedding, a feature-learning\ntechnique in Natural Language Processing that maps words or phrases to\nlow-dimensional vectors. Beginning with the linguistic theories concerning\ncontextual similarities - \"Distributional Hypothesis\" and \"Context of\nSituation\", this article introduces two ways of numerical representation of\ntext: One-hot and Distributed Representation. In addition, this article\npresents statistical-based Language Models(such as Co-occurrence Matrix and\nSingular Value Decomposition) as well as Neural Network Language Models (NNLM,\nsuch as Continuous Bag-of-Words and Skip-Gram). This article also analyzes how\nWord Embedding can be applied to the study of word-sense disambiguation and\ndiachronic linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00964v1",
            "date": "2019-11-03 20:38:38+00:00",
            "title": "MRNN: A Multi-Resolution Neural Network with Duplex Attention for Document Retrieval in the Context of Question Answering",
            "authors": [
                "Tolgahan Cakaloglu",
                "Xiaowei Xu"
            ],
            "abstract": "The primary goal of ad-hoc retrieval (document retrieval in the context of\nquestion answering) is to find relevant documents satisfied the information\nneed posted in a natural language query. It requires a good understanding of\nthe query and all the documents in a corpus, which is difficult because the\nmeaning of natural language texts depends on the context, syntax,and semantics.\nRecently deep neural networks have been used to rank search results in response\nto a query. In this paper, we devise a multi-resolution neural network(MRNN) to\nleverage the whole hierarchy of representations for document retrieval. The\nproposed MRNN model is capable of deriving a representation that integrates\nrepresentations of different levels of abstraction from all the layers of the\nlearned hierarchical representation.Moreover, a duplex attention component is\ndesigned to refinethe multi-resolution representation so that an optimal\ncontextfor matching the query and document can be determined. More\nspecifically, the first attention mechanism determines optimal context from the\nlearned multi-resolution representation for the query and document. The latter\nattention mechanism aims to fine-tune the representation so that the query and\nthe relevant document are closer in proximity. The empirical study shows that\nMRNN with the duplex attention is significantly superior to existing models\nused for ad-hoc retrieval on benchmark datasets including SQuAD, WikiQA,\nQUASAR, and TrecQA.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.01248v1",
            "date": "2019-11-04 14:35:59+00:00",
            "title": "A Holistic Natural Language Generation Framework for the Semantic Web",
            "authors": [
                "Axel-Cyrille Ngonga Ngomo",
                "Diego Moussallem",
                "Lorenz B\u00fchmann"
            ],
            "abstract": "With the ever-growing generation of data for the Semantic Web comes an\nincreasing demand for this data to be made available to non-semantic Web\nexperts. One way of achieving this goal is to translate the languages of the\nSemantic Web into natural language. We present LD2NL, a framework for\nverbalizing the three key languages of the Semantic Web, i.e., RDF, OWL, and\nSPARQL. Our framework is based on a bottom-up approach to verbalization. We\nevaluated LD2NL in an open survey with 86 persons. Our results suggest that our\nframework can generate verbalizations that are close to natural languages and\nthat can be easily understood by non-experts. Therewith, it enables non-domain\nexperts to interpret Semantic Web data with more than 91\\% of the accuracy of\ndomain experts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01256v2",
            "date": "2019-11-04 14:47:33+00:00",
            "title": "A Novel Approach to Enhance the Performance of Semantic Search in Bengali using Neural Net and other Classification Techniques",
            "authors": [
                "Arijit Das",
                "Diganta Saha"
            ],
            "abstract": "Search has for a long time been an important tool for users to retrieve\ninformation. Syntactic search is matching documents or objects containing\nspecific keywords like user-history, location, preference etc. to improve the\nresults. However, it is often possible that the query and the best answer have\nno term or very less number of terms in common and syntactic search can not\nperform properly in such cases. Semantic search, on the other hand, resolves\nthese issues but suffers from lack of annotation, absence of WordNet in case of\nlow resource languages. In this work, we have demonstrated an end to end\nprocedure to improve the performance of semantic search using semi-supervised\nand unsupervised learning algorithms. An available Bengali repository was\nchosen to have seven types of semantic properties primarily to develop the\nsystem. Performance has been tested using Support Vector Machine, Naive Bayes,\nDecision Tree and Artificial Neural Network (ANN). Our system has achieved the\nefficiency to predict the correct semantics using knowledge base over the time\nof learning. A repository containing around a million sentences, a product of\nTDIL project of Govt. of India, was used to test our system at first instance.\nThen the testing has been done for other languages. Being a cognitive system it\nmay be very useful for improving user satisfaction in e-Governance or\nm-Governance in the multilingual environment and also for other applications.",
            "journal ref": "IJEAT Vol 9 Issue 3 year 2020 ISSN 2249-8958",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01324v1",
            "date": "2019-11-04 16:39:21+00:00",
            "title": "Examining UK drill music through sentiment trajectory analysis",
            "authors": [
                "Bennett Kleinberg",
                "Paul McFarlane"
            ],
            "abstract": "This paper presents how techniques from natural language processing can be\nused to examine the sentiment trajectories of gang-related drill music in the\nUnited Kingdom (UK). This work is important because key public figures are\nloosely making controversial linkages between drill music and recent\nescalations in youth violence in London. Thus, this paper examines the dynamic\nuse of sentiment in gang-related drill music lyrics. The findings suggest two\ndistinct sentiment use patterns and statistical analyses revealed that lyrics\nwith a markedly positive tone attract more views and engagement on YouTube than\nnegative ones. Our work provides the first empirical insights into the language\nuse of London drill music, and it can, therefore, be used in future studies and\nby policymakers to help understand the alleged drill-gang nexus.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.01421v1",
            "date": "2019-11-05 05:47:22+00:00",
            "title": "A Deep Learning approach for Hindi Named Entity Recognition",
            "authors": [
                "Bansi Shah",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Named Entity Recognition is one of the most important text processing\nrequirement in many NLP tasks. In this paper we use a deep architecture to\naccomplish the task of recognizing named entities in a given Hindi text\nsentence. Bidirectional Long Short Term Memory (BiLSTM) based techniques have\nbeen used for NER task in literature. In this paper, we first tune BiLSTM\nlow-resource scenario to work for Hindi NER and propose two enhancements namely\n(a) de-noising auto-encoder (DAE) LSTM and (b) conditioning LSTM which show\nimprovement in NER task compared to the BiLSTM approach. We use pre-trained\nword embedding to represent the words in the corpus, and the NER tags of the\nwords are as defined by the used annotated corpora. Experiments have been\nperformed to analyze the performance of different word embeddings and batch\nsizes which is essential for training deep models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01497v3",
            "date": "2019-11-04 21:31:36+00:00",
            "title": "On Compositionality in Neural Machine Translation",
            "authors": [
                "Vikas Raunak",
                "Vaibhav Kumar",
                "Florian Metze"
            ],
            "abstract": "We investigate two specific manifestations of compositionality in Neural\nMachine Translation (NMT) : (1) Productivity - the ability of the model to\nextend its predictions beyond the observed length in training data and (2)\nSystematicity - the ability of the model to systematically recombine known\nparts and rules. We evaluate a standard Sequence to Sequence model on tests\ndesigned to assess these two properties in NMT. We quantitatively demonstrate\nthat inadequate temporal processing, in the form of poor encoder\nrepresentations is a bottleneck for both Productivity and Systematicity. We\npropose a simple pre-training mechanism which alleviates model performance on\nthe two properties and leads to a significant improvement in BLEU scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01542v2",
            "date": "2019-11-05 00:06:41+00:00",
            "title": "embComp: Visual Interactive Comparison of Vector Embeddings",
            "authors": [
                "Florian Heimerl",
                "Christoph Kralj",
                "Torsten M\u00f6ller",
                "Michael Gleicher"
            ],
            "abstract": "This paper introduces embComp, a novel approach for comparing two embeddings\nthat capture the similarity between objects, such as word and document\nembeddings. We survey scenarios where comparing these embedding spaces is\nuseful. From those scenarios, we derive common tasks, introduce visual analysis\nmethods that support these tasks, and combine them into a comprehensive system.\nOne of embComp's central features are overview visualizations that are based on\nmetrics for measuring differences in the local structure around objects.\nSummarizing these local metrics over the embeddings provides global overviews\nof similarities and differences. Detail views allow comparison of the local\nstructure around selected objects and relating this local information to the\nglobal views. Integrating and connecting all of these components, embComp\nsupports a range of analysis workflows that help understand similarities and\ndifferences between embedding spaces. We assess our approach by applying it in\nseveral use cases, including understanding corpora differences via word vector\nembeddings, and understanding algorithmic differences in generating embeddings.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.01556v2",
            "date": "2019-11-05 01:19:50+00:00",
            "title": "Review-based Question Generation with Adaptive Instance Transfer and Augmentation",
            "authors": [
                "Qian Yu",
                "Lidong Bing",
                "Qiong Zhang",
                "Wai Lam",
                "Luo Si"
            ],
            "abstract": "Online reviews provide rich information about products and service, while it\nremains inefficient for potential consumers to exploit the reviews for\nfulfilling their specific information need. We propose to explore question\ngeneration as a new way of exploiting review information. One major challenge\nof this task is the lack of review-question pairs for training a neural\ngeneration model. We propose an iterative learning framework for handling this\nchallenge via adaptive transfer and augmentation of the training instances with\nthe help of the available user-posed question-answer data. To capture the\naspect characteristics in reviews, the augmentation and generation procedures\nincorporate related features extracted via unsupervised learning. Experiments\non data from 10 categories of a popular E-commerce site demonstrate the\neffectiveness of the framework, as well as the usefulness of the new task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.01625v1",
            "date": "2019-11-05 05:28:05+00:00",
            "title": "Sparse Lifting of Dense Vectors: Unifying Word and Sentence Representations",
            "authors": [
                "Wenye Li",
                "Senyue Hao"
            ],
            "abstract": "As the first step in automated natural language processing, representing\nwords and sentences is of central importance and has attracted significant\nresearch attention. Different approaches, from the early one-hot and\nbag-of-words representation to more recent distributional dense and sparse\nrepresentations, were proposed. Despite the successful results that have been\nachieved, such vectors tend to consist of uninterpretable components and face\nnontrivial challenge in both memory and computational requirement in practical\napplications. In this paper, we designed a novel representation model that\nprojects dense word vectors into a higher dimensional space and favors a highly\nsparse and binary representation of word vectors with potentially interpretable\ncomponents, while trying to maintain pairwise inner products between original\nvectors as much as possible. Computationally, our model is relaxed as a\nsymmetric non-negative matrix factorization problem which admits a fast yet\neffective solution. In a series of empirical evaluations, the proposed model\nexhibited consistent improvement and high potential in practical applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01678v4",
            "date": "2019-11-05 09:23:58+00:00",
            "title": "A Joint Model for Definition Extraction with Syntactic Connection and Semantic Consistency",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Franck Dernoncourt",
                "Dejing Dou",
                "Thien Huu Nguyen"
            ],
            "abstract": "Definition Extraction (DE) is one of the well-known topics in Information\nExtraction that aims to identify terms and their corresponding definitions in\nunstructured texts. This task can be formalized either as a sentence\nclassification task (i.e., containing term-definition pairs or not) or a\nsequential labeling task (i.e., identifying the boundaries of the terms and\ndefinitions). The previous works for DE have only focused on one of the two\napproaches, failing to model the inter-dependencies between the two tasks. In\nthis work, we propose a novel model for DE that simultaneously performs the two\ntasks in a single framework to benefit from their inter-dependencies. Our model\nfeatures deep learning architectures to exploit the global structures of the\ninput sentences as well as the semantic consistencies between the terms and the\ndefinitions, thereby improving the quality of the representation vectors for\nDE. Besides the joint inference between sentence classification and sequential\nlabeling, the proposed model is fundamentally different from the prior work for\nDE in that the prior work has only employed the local structures of the input\nsentences (i.e., word-to-word relations), and not yet considered the semantic\nconsistencies between terms and definitions. In order to implement these novel\nideas, our model presents a multi-task learning framework that employs graph\nconvolutional neural networks and predicts the dependency paths between the\nterms and the definitions. We also seek to enforce the consistency between the\nrepresentations of the terms and definitions both globally (i.e., increasing\nsemantic consistency between the representations of the entire sentences and\nthe terms/definitions) and locally (i.e., promoting the similarity between the\nrepresentations of the terms and the definitions).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01680v2",
            "date": "2019-11-05 09:29:07+00:00",
            "title": "Improving Slot Filling by Utilizing Contextual Information",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Franck Dernoncourt",
                "Thien Huu Nguyen"
            ],
            "abstract": "Slot Filling (SF) is one of the sub-tasks of Spoken Language Understanding\n(SLU) which aims to extract semantic constituents from a given natural language\nutterance. It is formulated as a sequence labeling task. Recently, it has been\nshown that contextual information is vital for this task. However, existing\nmodels employ contextual information in a restricted manner, e.g., using\nself-attention. Such methods fail to distinguish the effects of the context on\nthe word representation and the word label. To address this issue, in this\npaper, we propose a novel method to incorporate the contextual information in\ntwo different levels, i.e., representation level and task-specific (i.e.,\nlabel) level. Our extensive experiments on three benchmark datasets on SF show\nthe effectiveness of our model leading to new state-of-the-art results on all\nthree benchmark datasets for the task of SF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01770v1",
            "date": "2019-11-05 13:42:30+00:00",
            "title": "Self-Attention and Ingredient-Attention Based Model for Recipe Retrieval from Image Queries",
            "authors": [
                "Matthias Fontanellaz",
                "Stergios Christodoulidis",
                "Stavroula Mougiakakou"
            ],
            "abstract": "Direct computer vision based-nutrient content estimation is a demanding task,\ndue to deformation and occlusions of ingredients, as well as high intra-class\nand low inter-class variability between meal classes. In order to tackle these\nissues, we propose a system for recipe retrieval from images. The recipe\ninformation can subsequently be used to estimate the nutrient content of the\nmeal. In this study, we utilize the multi-modal Recipe1M dataset, which\ncontains over 1 million recipes accompanied by over 13 million images. The\nproposed model can operate as a first step in an automatic pipeline for the\nestimation of nutrition content by supporting hints related to ingredient and\ninstruction. Through self-attention, our model can directly process raw recipe\ntext, making the upstream instruction sentence embedding process redundant and\nthus reducing training time, while providing desirable retrieval results.\nFurthermore, we propose the use of an ingredient attention mechanism, in order\nto gain insight into which instructions, parts of instructions or single\ninstruction words are of importance for processing a single ingredient within a\ncertain recipe. Attention-based recipe text encoding contributes to solving the\nissue of high intra-class/low inter-class variability by focusing on\npreparation steps specific to the meal. The experimental results demonstrate\nthe potential of such a system for recipe retrieval from images. A comparison\nwith respect to two baseline methods is also presented.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.01892v1",
            "date": "2019-11-05 15:55:19+00:00",
            "title": "Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game",
            "authors": [
                "Roberto Dess\u00ec",
                "Diane Bouchacourt",
                "Davide Crepaldi",
                "Marco Baroni"
            ],
            "abstract": "Research in multi-agent cooperation has shown that artificial agents are able\nto learn to play a simple referential game while developing a shared lexicon.\nThis lexicon is not easy to analyze, as it does not show many properties of a\nnatural language. In a simple referential game with two neural network-based\nagents, we analyze the object-symbol mapping trying to understand what kind of\nstrategy was used to develop the emergent language. We see that, when the\nenvironment is uniformly distributed, the agents rely on a random subset of\nfeatures to describe the objects. When we modify the objects making one feature\nnon-uniformly distributed,the agents realize it is less informative and start\nto ignore it, and, surprisingly, they make a better use of the remaining\nfeatures. This interesting result suggests that more natural, less uniformly\ndistributed environments might aid in spurring the emergence of better-behaved\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01917v1",
            "date": "2019-11-05 16:23:01+00:00",
            "title": "Scenarios and Recommendations for Ethical Interpretive AI",
            "authors": [
                "John Licato",
                "Zaid Marji",
                "Sophia Abraham"
            ],
            "abstract": "Artificially intelligent systems, given a set of non-trivial ethical rules to\nfollow, will inevitably be faced with scenarios which call into question the\nscope of those rules. In such cases, human reasoners typically will engage in\ninterpretive reasoning, where interpretive arguments are used to support or\nattack claims that some rule should be understood a certain way. Artificially\nintelligent reasoners, however, currently lack the ability to carry out\nhuman-like interpretive reasoning, and we argue that bridging this gulf is of\ntremendous importance to human-centered AI. In order to better understand how\nfuture artificial reasoners capable of human-like interpretive reasoning must\nbe developed, we have collected a dataset of ethical rules, scenarios designed\nto invoke interpretive reasoning, and interpretations of those scenarios. We\nperform a qualitative analysis of our dataset, and summarize our findings in\nthe form of practical recommendations.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.01986v4",
            "date": "2019-11-05 18:25:42+00:00",
            "title": "Data Diversification: A Simple Strategy For Neural Machine Translation",
            "authors": [
                "Xuan-Phi Nguyen",
                "Shafiq Joty",
                "Wu Kui",
                "Ai Ti Aw"
            ],
            "abstract": "We introduce Data Diversification: a simple but effective strategy to boost\nneural machine translation (NMT) performance. It diversifies the training data\nby using the predictions of multiple forward and backward models and then\nmerging them with the original dataset on which the final NMT model is trained.\nOur method is applicable to all NMT models. It does not require extra\nmonolingual data like back-translation, nor does it add more computations and\nparameters like ensembles of models. Our method achieves state-of-the-art BLEU\nscores of 30.7 and 43.7 in the WMT'14 English-German and English-French\ntranslation tasks, respectively. It also substantially improves on 8 other\ntranslation tasks: 4 IWSLT tasks (English-German and English-French) and 4\nlow-resource translation tasks (English-Nepali and English-Sinhala). We\ndemonstrate that our method is more effective than knowledge distillation and\ndual learning, it exhibits strong correlation with ensembles of models, and it\ntrades perplexity off for better BLEU score. We have released our source code\nat https://github.com/nxphi47/data_diversification",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02002v1",
            "date": "2019-11-05 18:57:50+00:00",
            "title": "Language coverage and generalization in RNN-based continuous sentence embeddings for interacting agents",
            "authors": [
                "Luca Celotti",
                "Simon Brodeur",
                "Jean Rouat"
            ],
            "abstract": "Continuous sentence embeddings using recurrent neural networks (RNNs), where\nvariable-length sentences are encoded into fixed-dimensional vectors, are often\nthe main building blocks of architectures applied to language tasks such as\ndialogue generation. While it is known that those embeddings are able to learn\nsome structures of language (e.g. grammar) in a purely data-driven manner,\nthere is very little work on the objective evaluation of their ability to cover\nthe whole language space and to generalize to sentences outside the language\nbias of the training data. Using a manually designed context-free grammar (CFG)\nto generate a large-scale dataset of sentences related to the content of\nrealistic 3D indoor scenes, we evaluate the language coverage and\ngeneralization abilities of the most common continuous sentence embeddings\nbased on RNNs. We also propose a new embedding method based on arithmetic\ncoding, AriEL, that is not data-driven and that efficiently encodes in\ncontinuous space any sentence from the CFG. We find that RNN-based embeddings\nunderfit the training data and cover only a small subset of the language\ndefined by the CFG. They also fail to learn the underlying CFG and generalize\nto unbiased sentences from that same CFG. We found that AriEL provides an\ninsightful baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02060v2",
            "date": "2019-11-05 19:52:34+00:00",
            "title": "Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks",
            "authors": [
                "Pavan Kapanipathi",
                "Veronika Thost",
                "Siva Sankalp Patel",
                "Spencer Whitehead",
                "Ibrahim Abdelaziz",
                "Avinash Balakrishnan",
                "Maria Chang",
                "Kshitij Fadnis",
                "Chulaka Gunasekara",
                "Bassem Makni",
                "Nicholas Mattei",
                "Kartik Talamadupula",
                "Achille Fokoue"
            ],
            "abstract": "Textual entailment is a fundamental task in natural language processing. Most\napproaches for solving the problem use only the textual content present in\ntraining data. A few approaches have shown that information from external\nknowledge sources like knowledge graphs (KGs) can add value, in addition to the\ntextual content, by providing background knowledge that may be critical for a\ntask. However, the proposed models do not fully exploit the information in the\nusually large and noisy KGs, and it is not clear how it can be effectively\nencoded to be useful for entailment. We present an approach that complements\ntext-based entailment models with information from KGs by (1) using\nPersonalized PageR- ank to generate contextual subgraphs with reduced noise and\n(2) encoding these subgraphs using graph convolutional networks to capture KG\nstructure. Our technique extends the capability of text models exploiting\nstructural and semantic information found in KGs. We evaluate our approach on\nmultiple textual entailment datasets and show that the use of external\nknowledge helps improve prediction accuracy. This is particularly evident in\nthe challenging BreakingNLI dataset, where we see an absolute improvement of\n5-20% over multiple text-based entailment models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02085v2",
            "date": "2019-11-05 21:06:04+00:00",
            "title": "Path-Based Contextualization of Knowledge Graphs for Textual Entailment",
            "authors": [
                "Kshitij Fadnis",
                "Kartik Talamadupula",
                "Pavan Kapanipathi",
                "Haque Ishfaq",
                "Salim Roukos",
                "Achille Fokoue"
            ],
            "abstract": "In this paper, we introduce the problem of knowledge graph contextualization\n-- that is, given a specific NLP task, the problem of extracting meaningful and\nrelevant sub-graphs from a given knowledge graph. The task in the case of this\npaper is the textual entailment problem, and the context is a relevant\nsub-graph for an instance of the textual entailment problem -- where given two\nsentences p and h, the entailment relationship between them has to be predicted\nautomatically. We base our methodology on finding paths in a cost-customized\nexternal knowledge graph, and building the most relevant sub-graph that\nconnects p and h. We show that our path selection mechanism to generate\nsub-graphs not only reduces noise, but also retrieves meaningful information\nfrom large knowledge graphs. Our evaluation shows that using information on\nentities as well as the relationships between them improves on the performance\nof purely text-based systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.02147v2",
            "date": "2019-11-06 00:08:19+00:00",
            "title": "Seq2Emo for Multi-label Emotion Classification Based on Latent Variable Chains Transformation",
            "authors": [
                "Chenyang Huang",
                "Amine Trabelsi",
                "Xuebin Qin",
                "Nawshad Farruque",
                "Osmar R. Za\u00efane"
            ],
            "abstract": "Emotion detection in text is an important task in NLP and is essential in\nmany applications. Most of the existing methods treat this task as a problem of\nsingle-label multi-class text classification. To predict multiple emotions for\none instance, most of the existing works regard it as a general Multi-label\nClassification (MLC) problem, where they usually either apply a manually\ndetermined threshold on the last output layer of their neural network models or\ntrain multiple binary classifiers and make predictions in the fashion of\none-vs-all. However, compared to labels in the general MLC datasets, the number\nof emotion categories are much fewer (less than 10). Additionally, emotions\ntend to have more correlations with each other. For example, the human usually\ndoes not express \"joy\" and \"anger\" at the same time, but it is very likely to\nhave \"joy\" and \"love\" expressed together. Given this intuition, in this paper,\nwe propose a Latent Variable Chain (LVC) transformation and a tailored model --\nSeq2Emo model that not only naturally predicts multiple emotion labels but also\ntakes into consideration their correlations. We perform the experiments on the\nexisting multi-label emotion datasets as well as on our newly collected\ndatasets. The results show that our model compares favorably with existing\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02168v2",
            "date": "2019-11-06 02:27:39+00:00",
            "title": "CoKE: Contextualized Knowledge Graph Embedding",
            "authors": [
                "Quan Wang",
                "Pingping Huang",
                "Haifeng Wang",
                "Songtai Dai",
                "Wenbin Jiang",
                "Jing Liu",
                "Yajuan Lyu",
                "Yong Zhu",
                "Hua Wu"
            ],
            "abstract": "Knowledge graph embedding, which projects symbolic entities and relations\ninto continuous vector spaces, is gaining increasing attention. Previous\nmethods allow a single static embedding for each entity or relation, ignoring\ntheir intrinsic contextual nature, i.e., entities and relations may appear in\ndifferent graph contexts, and accordingly, exhibit different properties. This\nwork presents Contextualized Knowledge Graph Embedding (CoKE), a novel paradigm\nthat takes into account such contextual nature, and learns dynamic, flexible,\nand fully contextualized entity and relation embeddings. Two types of graph\ncontexts are studied: edges and paths, both formulated as sequences of entities\nand relations. CoKE takes a sequence as input and uses a Transformer encoder to\nobtain contextualized representations. These representations are hence\nnaturally adaptive to the input, capturing contextual meanings of entities and\nrelations therein. Evaluation on a wide variety of public benchmarks verifies\nthe superiority of CoKE in link prediction and path query answering. It\nperforms consistently better than, or at least equally well as current\nstate-of-the-art in almost every case, in particular offering an absolute\nimprovement of 21.0% in H@10 on path query answering. Our code is available at\n\\url{https://github.com/PaddlePaddle/Research/tree/master/KG/CoKE}.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.02257v2",
            "date": "2019-11-06 08:52:52+00:00",
            "title": "Hierarchical Contextualized Representation for Named Entity Recognition",
            "authors": [
                "Ying Luo",
                "Fengshun Xiao",
                "Hai Zhao"
            ],
            "abstract": "Named entity recognition (NER) models are typically based on the architecture\nof Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the\nmodeling of single input prevent the full utilization of global information\nfrom larger scope, not only in the entire sentence, but also in the entire\ndocument (dataset). In this paper, we address these two deficiencies and\npropose a model augmented with hierarchical contextualized representation:\nsentence-level representation and document-level representation. In\nsentence-level, we take different contributions of words in a single sentence\ninto consideration to enhance the sentence representation learned from an\nindependent BiLSTM via label embedding attention mechanism. In document-level,\nthe key-value memory network is adopted to record the document-aware\ninformation for each unique word which is sensitive to similarity of context\ninformation. Our two-level hierarchical contextualized representations are\nfused with each input token embedding and corresponding hidden state of BiLSTM,\nrespectively. The experimental results on three benchmark NER datasets\n(CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset)\nshow that we establish new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02562v2",
            "date": "2019-11-06 10:57:38+00:00",
            "title": "Gextext: Disease Network Extraction from Biomedical Literature",
            "authors": [
                "Robert O'Shea"
            ],
            "abstract": "PURPOSE: We propose a fully unsupervised method to learn latent disease\nnetworks directly from unstructured biomedical text corpora. This method\naddresses current challenges in unsupervised knowledge extraction, such as the\ndetection of long-range dependencies and requirements for large training\ncorpora. METHODS: Let C be a corpus of n text chunks. Let V be a set of p\ndisease terms occurring in the corpus. Let X indicate the occurrence of V in C.\nGextext identifies disease similarities by positively correlated occurrence\npatterns. This information is combined to generate a graph on which geodesic\ndistance describes dissimilarity. Diseasomes were learned by Gextext and GloVE\non corpora of 100-1000 PubMed abstracts. Similarity matrix estimates were\nvalidated against biomedical semantic similarity metrics and gene profile\nsimilarity. RESULTS: Geodesic distance on Gextext-inferred diseasomes\ncorrelated inversely with external measures of semantic similarity. Gene\nprofile similarity also correlated significant with proximity on the inferred\ngraph. Gextext outperformed GloVE in our experiments. The information contained\non the Gextext graph exceeded the explicit information content within the text.\nCONCLUSIONS: Gextext extracts latent relationships from unstructured text,\nenabling fully unsupervised modelling of diseasome graphs from PubMed\nabstracts.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02639v1",
            "date": "2019-11-06 21:40:34+00:00",
            "title": "Word Embedding Algorithms as Generalized Low Rank Models and their Canonical Form",
            "authors": [
                "Kian Kenyon-Dean"
            ],
            "abstract": "Word embedding algorithms produce very reliable feature representations of\nwords that are used by neural network models across a constantly growing\nmultitude of NLP tasks. As such, it is imperative for NLP practitioners to\nunderstand how their word representations are produced, and why they are so\nimpactful.\n  The present work presents the Simple Embedder framework, generalizing the\nstate-of-the-art existing word embedding algorithms (including Word2vec (SGNS)\nand GloVe) under the umbrella of generalized low rank models. We derive that\nboth of these algorithms attempt to produce embedding inner products that\napproximate pointwise mutual information (PMI) statistics in the corpus. Once\ncast as Simple Embedders, comparison of these models reveals that these\nsuccessful embedders all resemble a straightforward maximum likelihood estimate\n(MLE) of the PMI parametrized by the inner product (between embeddings). This\nMLE induces our proposed novel word embedding model, Hilbert-MLE, as the\ncanonical representative of the Simple Embedder framework.\n  We empirically compare these algorithms with evaluations on 17 different\ndatasets. Hilbert-MLE consistently observes second-best performance on every\nextrinsic evaluation (news classification, sentiment analysis, POS-tagging, and\nsupersense tagging), while the first-best model depends varying on the task.\nMoreover, Hilbert-MLE consistently observes the least variance in results with\nrespect to the random initialization of the weights in bidirectional LSTMs. Our\nempirical results demonstrate that Hilbert-MLE is a very consistent word\nembedding algorithm that can be reliably integrated into existing NLP systems\nto obtain high-quality results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02645v1",
            "date": "2019-11-06 22:01:18+00:00",
            "title": "Unsupervised Domain Adaptation of Contextual Embeddings for Low-Resource Duplicate Question Detection",
            "authors": [
                "Alexandre Rochette",
                "Yadollah Yaghoobzadeh",
                "Timothy J. Hazen"
            ],
            "abstract": "Answering questions is a primary goal of many conversational systems or\nsearch products. While most current systems have focused on answering questions\nagainst structured databases or curated knowledge graphs, on-line community\nforums or frequently asked questions (FAQ) lists offer an alternative source of\ninformation for question answering systems. Automatic duplicate question\ndetection (DQD) is the key technology need for question answering systems to\nutilize existing online forums like StackExchange. Existing annotations of\nduplicate questions in such forums are community-driven, making them sparse or\neven completely missing for many domains. Therefore, it is important to\ntransfer knowledge from related domains and tasks. Recently, contextual\nembedding models such as BERT have been outperforming many baselines by\ntransferring self-supervised information to downstream tasks. In this paper, we\napply BERT to DQD and advance it by unsupervised adaptation to StackExchange\ndomains using self-supervised learning. We show the effectiveness of this\nadaptation for low-resource settings, where little or no training data is\navailable from the target domain. Our analysis reveals that unsupervised BERT\ndomain adaptation on even small amounts of data boosts the performance of BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02671v1",
            "date": "2019-11-06 23:12:56+00:00",
            "title": "Open Domain Web Keyphrase Extraction Beyond Language Modeling",
            "authors": [
                "Lee Xiong",
                "Chuan Hu",
                "Chenyan Xiong",
                "Daniel Campos",
                "Arnold Overwijk"
            ],
            "abstract": "This paper studies keyphrase extraction in real-world scenarios where\ndocuments are from diverse domains and have variant content quality. We curate\nand release OpenKP, a large scale open domain keyphrase extraction dataset with\nnear one hundred thousand web documents and expert keyphrase annotations. To\nhandle the variations of domain and content quality, we develop BLING-KPE, a\nneural keyphrase extraction model that goes beyond language understanding using\nvisual presentations of documents and weak supervision from search queries.\nExperimental results on OpenKP confirm the effectiveness of BLING-KPE and the\ncontributions of its neural architecture, visual features, and search log weak\nsupervision. Zero-shot evaluations on DUC-2001 demonstrate the improved\ngeneralization ability of learning from the open domain data compared to a\nspecific domain.",
            "journal ref": "EMNLP-IJCNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02683v2",
            "date": "2019-11-06 23:47:32+00:00",
            "title": "Shaping Visual Representations with Language for Few-shot Classification",
            "authors": [
                "Jesse Mu",
                "Percy Liang",
                "Noah Goodman"
            ],
            "abstract": "By describing the features and abstractions of our world, language is a\ncrucial tool for human learning and a promising source of supervision for\nmachine learning models. We use language to improve few-shot visual\nclassification in the underexplored scenario where natural language task\ndescriptions are available during training, but unavailable for novel tasks at\ntest time. Existing models for this setting sample new descriptions at test\ntime and use those to classify images. Instead, we propose language-shaped\nlearning (LSL), an end-to-end model that regularizes visual representations to\npredict language. LSL is conceptually simpler, more data efficient, and\noutperforms baselines in two challenging few-shot domains.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.02690v2",
            "date": "2019-11-07 00:52:38+00:00",
            "title": "SIMMC: Situated Interactive Multi-Modal Conversational Data Collection And Evaluation Platform",
            "authors": [
                "Paul A. Crook",
                "Shivani Poddar",
                "Ankita De",
                "Semir Shafi",
                "David Whitney",
                "Alborz Geramifard",
                "Rajen Subba"
            ],
            "abstract": "As digital virtual assistants become ubiquitous, it becomes increasingly\nimportant to understand the situated behaviour of users as they interact with\nthese assistants. To this end, we introduce SIMMC, an extension to ParlAI for\nmulti-modal conversational data collection and system evaluation. SIMMC\nsimulates an immersive setup, where crowd workers are able to interact with\nenvironments constructed in AI Habitat or Unity while engaging in a\nconversation. The assistant in SIMMC can be a crowd worker or Artificial\nIntelligent (AI) agent. This enables both (i) a multi-player / Wizard of Oz\nsetting for data collection, or (ii) a single player mode for model / system\nevaluation. We plan to open-source a situated conversational data-set collected\non this platform for the Conversational AI research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02692v3",
            "date": "2019-11-07 00:54:06+00:00",
            "title": "Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing",
            "authors": [
                "Haoming Jiang",
                "Chen Liang",
                "Chong Wang",
                "Tuo Zhao"
            ],
            "abstract": "Many multi-domain neural machine translation (NMT) models achieve knowledge\ntransfer by enforcing one encoder to learn shared embedding across domains.\nHowever, this design lacks adaptation to individual domains. To overcome this\nlimitation, we propose a novel multi-domain NMT model using individual modules\nfor each domain, on which we apply word-level, adaptive and layer-wise domain\nmixing. We first observe that words in a sentence are often related to multiple\ndomains. Hence, we assume each word has a domain proportion, which indicates\nits domain preference. Then word representations are obtained by mixing their\nembedding in individual domains based on their domain proportions. We show this\ncan be achieved by carefully designing multi-head dot-product attention modules\nfor different domains, and eventually taking weighted averages of their\nparameters by word-level layer-wise domain proportions. Through this, we can\nachieve effective domain knowledge sharing, and capture fine-grained\ndomain-specific knowledge as well. Our experiments show that our proposed model\noutperforms existing ones in several NMT tasks.",
            "journal ref": "The 58th annual meeting of the Association for Computational\n  Linguistics (ACL 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02707v3",
            "date": "2019-11-07 01:40:39+00:00",
            "title": "Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs",
            "authors": [
                "Houyu Zhang",
                "Zhenghao Liu",
                "Chenyan Xiong",
                "Zhiyuan Liu"
            ],
            "abstract": "Human conversations naturally evolve around related concepts and scatter to\nmulti-hop concepts. This paper presents a new conversation generation model,\nConceptFlow, which leverages commonsense knowledge graphs to explicitly model\nconversation flows. By grounding conversations to the concept space,\nConceptFlow represents the potential conversation flow as traverses in the\nconcept space along commonsense relations. The traverse is guided by graph\nattentions in the concept graph, moving towards more meaningful directions in\nthe concept space, in order to generate more semantic and informative\nresponses. Experiments on Reddit conversations demonstrate ConceptFlow's\neffectiveness over previous knowledge-aware conversation models and GPT-2 based\nmodels while using 70% fewer parameters, confirming the advantage of explicit\nmodeling conversation structures. All source codes of this work are available\nat https://github.com/thunlp/ConceptFlow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02733v3",
            "date": "2019-11-07 02:58:17+00:00",
            "title": "Porous Lattice-based Transformer Encoder for Chinese NER",
            "authors": [
                "Xue Mengge",
                "Yu Bowen",
                "Liu Tingwen",
                "Zhang Yue",
                "Meng Erli",
                "Wang Bin"
            ],
            "abstract": "Incorporating lattices into character-level Chinese named entity recognition\nis an effective method to exploit explicit word information. Recent works\nextend recurrent and convolutional neural networks to model lattice inputs.\nHowever, due to the DAG structure or the variable-sized potential word set for\nlattice inputs, these models prevent the convenient use of batched computation,\nresulting in serious inefficient. In this paper, we propose a porous\nlattice-based transformer encoder for Chinese named entity recognition, which\nis capable to better exploit the GPU parallelism and batch the computation\nowing to the mask mechanism in transformer. We first investigate the\nlattice-aware self-attention coupled with relative position representations to\nexplore effective word information in the lattice structure. Besides, to\nstrengthen the local dependencies among neighboring tokens, we propose a novel\nporous structure during self-attentional computation processing, in which every\ntwo non-neighboring tokens are connected through a shared pivot node.\nExperimental results on four datasets show that our model performs up to 9.47\ntimes faster than state-of-the-art models, while is roughly on a par with its\nperformance. The source code of this paper can be obtained from\nhttps://github.com/xxx/xxx.",
            "journal ref": "COLING 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02747v1",
            "date": "2019-11-07 04:11:03+00:00",
            "title": "Query-bag Matching with Mutual Coverage for Information-seeking Conversations in E-commerce",
            "authors": [
                "Zhenxin Fu",
                "Feng Ji",
                "Wenpeng Hu",
                "Wei Zhou",
                "Dongyan Zhao",
                "Haiqing Chen",
                "Rui Yan"
            ],
            "abstract": "Information-seeking conversation system aims at satisfying the information\nneeds of users through conversations. Text matching between a user query and a\npre-collected question is an important part of the information-seeking\nconversation in E-commerce. In the practical scenario, a sort of questions\nalways correspond to a same answer. Naturally, these questions can form a bag.\nLearning the matching between user query and bag directly may improve the\nconversation performance, denoted as query-bag matching. Inspired by such\nopinion, we propose a query-bag matching model which mainly utilizes the mutual\ncoverage between query and bag and measures the degree of the content in the\nquery mentioned by the bag, and vice verse. In addition, the learned bag\nrepresentation in word level helps find the main points of a bag in a fine\ngrade and promotes the query-bag matching performance. Experiments on two\ndatasets show the effectiveness of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02782v3",
            "date": "2019-11-07 07:34:43+00:00",
            "title": "S2ORC: The Semantic Scholar Open Research Corpus",
            "authors": [
                "Kyle Lo",
                "Lucy Lu Wang",
                "Mark Neumann",
                "Rodney Kinney",
                "Dan S. Weld"
            ],
            "abstract": "We introduce S2ORC, a large corpus of 81.1M English-language academic papers\nspanning many academic disciplines. The corpus consists of rich metadata, paper\nabstracts, resolved bibliographic references, as well as structured full text\nfor 8.1M open access papers. Full text is annotated with automatically-detected\ninline mentions of citations, figures, and tables, each linked to their\ncorresponding paper objects. In S2ORC, we aggregate papers from hundreds of\nacademic publishers and digital archives into a unified source, and create the\nlargest publicly-available collection of machine-readable academic text to\ndate. We hope this resource will facilitate research and development of tools\nand tasks for text mining over academic text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02850v3",
            "date": "2019-11-07 11:02:21+00:00",
            "title": "CROWN: Conversational Passage Ranking by Reasoning over Word Networks",
            "authors": [
                "Magdalena Kaiser",
                "Rishiraj Saha Roy",
                "Gerhard Weikum"
            ],
            "abstract": "Information needs around a topic cannot be satisfied in a single turn; users\ntypically ask follow-up questions referring to the same theme and a system must\nbe capable of understanding the conversational context of a request to retrieve\ncorrect answers. In this paper, we present our submission to the TREC\nConversational Assistance Track 2019, in which such a conversational setting is\nexplored. We propose a simple unsupervised method for conversational passage\nranking by formulating the passage score for a query as a combination of\nsimilarity and coherence. To be specific, passages are preferred that contain\nwords semantically similar to the words used in the question, and where such\nwords appear close by. We built a word-proximity network (WPN) from a large\ncorpus, where words are nodes and there is an edge between two nodes if they\nco-occur in the same passages in a statistically significant way, within a\ncontext window. Our approach, named CROWN, improved nDCG scores over a provided\nIndri baseline on the CAsT training data. On the evaluation data for CAsT, our\nbest run submission achieved above-average performance with respect to AP@5 and\nnDCG@1000.",
            "journal ref": "TREC 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.02891v2",
            "date": "2019-11-07 13:26:07+00:00",
            "title": "Improving Joint Training of Inference Networks and Structured Prediction Energy Networks",
            "authors": [
                "Lifu Tu",
                "Richard Yuanzhe Pang",
                "Kevin Gimpel"
            ],
            "abstract": "Deep energy-based models are powerful, but pose challenges for learning and\ninference (Belanger and McCallum, 2016). Tu and Gimpel (2018) developed an\nefficient framework for energy-based models by training \"inference networks\" to\napproximate structured inference instead of using gradient descent. However,\ntheir alternating optimization approach suffers from instabilities during\ntraining, requiring additional loss terms and careful hyperparameter tuning. In\nthis paper, we contribute several strategies to stabilize and improve this\njoint training of energy functions and inference networks for structured\nprediction. We design a compound objective to jointly train both cost-augmented\nand test-time inference networks along with the energy function. We propose\njoint parameterizations for the inference networks that encourage them to\ncapture complementary functionality during learning. We empirically validate\nour strategies on two sequence labeling tasks, showing easier paths to strong\nperformance than prior work, as well as further improvements with global energy\nterms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02896v2",
            "date": "2019-11-07 13:34:54+00:00",
            "title": "Contextualized Sparse Representations for Real-Time Open-Domain Question Answering",
            "authors": [
                "Jinhyuk Lee",
                "Minjoon Seo",
                "Hannaneh Hajishirzi",
                "Jaewoo Kang"
            ],
            "abstract": "Open-domain question answering can be formulated as a phrase retrieval\nproblem, in which we can expect huge scalability and speed benefit but often\nsuffer from low accuracy due to the limitation of existing phrase\nrepresentation models. In this paper, we aim to improve the quality of each\nphrase embedding by augmenting it with a contextualized sparse representation\n(Sparc). Unlike previous sparse vectors that are term-frequency-based (e.g.,\ntf-idf) or directly learned (only few thousand dimensions), we leverage\nrectified self-attention to indirectly learn sparse vectors in n-gram\nvocabulary space. By augmenting the previous phrase retrieval model (Seo et\nal., 2019) with Sparc, we show 4%+ improvement in CuratedTREC and SQuAD-Open.\nOur CuratedTREC score is even better than the best known retrieve & read model\nwith at least 45x faster inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02972v2",
            "date": "2019-11-07 16:35:53+00:00",
            "title": "Blockwise Self-Attention for Long Document Understanding",
            "authors": [
                "Jiezhong Qiu",
                "Hao Ma",
                "Omer Levy",
                "Scott Wen-tau Yih",
                "Sinong Wang",
                "Jie Tang"
            ],
            "abstract": "We present BlockBERT, a lightweight and efficient BERT model for better\nmodeling long-distance dependencies. Our model extends BERT by introducing\nsparse block structures into the attention matrix to reduce both memory\nconsumption and training/inference time, which also enables attention heads to\ncapture either short- or long-range contextual information. We conduct\nexperiments on language model pre-training and several benchmark question\nanswering datasets with various paragraph lengths. BlockBERT uses 18.7-36.1%\nless memory and 12.0-25.1% less time to learn the model. During testing,\nBlockBERT saves 27.8% inference time, while having comparable and sometimes\nbetter prediction accuracy, compared to an advanced BERT-based model, RoBERTa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02984v1",
            "date": "2019-11-08 02:04:39+00:00",
            "title": "The TechQA Dataset",
            "authors": [
                "Vittorio Castelli",
                "Rishav Chakravarti",
                "Saswati Dana",
                "Anthony Ferritto",
                "Radu Florian",
                "Martin Franz",
                "Dinesh Garg",
                "Dinesh Khandelwal",
                "Scott McCarley",
                "Mike McCawley",
                "Mohamed Nasr",
                "Lin Pan",
                "Cezar Pendus",
                "John Pitrelli",
                "Saurabh Pujar",
                "Salim Roukos",
                "Andrzej Sakrajda",
                "Avirup Sil",
                "Rosario Uceda-Sosa",
                "Todd Ward",
                "Rong Zhang"
            ],
            "abstract": "We introduce TechQA, a domain-adaptation question answering dataset for the\ntechnical support domain. The TechQA corpus highlights two real-world issues\nfrom the automated customer support domain. First, it contains actual questions\nposed by users on a technical forum, rather than questions generated\nspecifically for a competition or a task. Second, it has a real-world size --\n600 training, 310 dev, and 490 evaluation question/answer pairs -- thus\nreflecting the cost of creating large labeled datasets with actual data.\nConsequently, TechQA is meant to stimulate research in domain adaptation rather\nthan being a resource to build QA systems from scratch. The dataset was\nobtained by crawling the IBM Developer and IBM DeveloperWorks forums for\nquestions with accepted answers that appear in a published IBM Technote---a\ntechnical document that addresses a specific technical issue. We also release a\ncollection of the 801,998 publicly available Technotes as of April 4, 2019 as a\ncompanion resource that might be used for pretraining, to learn representations\nof the IT domain language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02989v1",
            "date": "2019-11-08 02:19:52+00:00",
            "title": "Cross-Lingual Relevance Transfer for Document Retrieval",
            "authors": [
                "Peng Shi",
                "Jimmy Lin"
            ],
            "abstract": "Recent work has shown the surprising ability of multi-lingual BERT to serve\nas a zero-shot cross-lingual transfer model for a number of language processing\ntasks. We combine this finding with a similarly-recently proposal on\nsentence-level relevance modeling for document retrieval to demonstrate the\nability of multi-lingual BERT to transfer models of relevance across languages.\nExperiments on test collections in five different languages from diverse\nlanguage families (Chinese, Arabic, French, Hindi, and Bengali) show that\nmodels trained with English data improve ranking quality, without any special\nprocessing, both for (non-English) mono-lingual retrieval as well as\ncross-lingual retrieval.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.03014v3",
            "date": "2019-11-08 03:17:54+00:00",
            "title": "Resurrecting Submodularity for Neural Text Generation",
            "authors": [
                "Simeng Han",
                "Xiang Lin",
                "Shafiq Joty"
            ],
            "abstract": "Submodularity is desirable for a variety of objectives in content selection\nwhere the current neural encoder-decoder framework is inadequate. However, it\nhas so far not been explored in the neural encoder-decoder system for text\ngeneration. In this work, we define diminishing attentions with submodular\nfunctions and in turn, prove the submodularity of the effective neural\ncoverage. The greedy algorithm approximating the solution to the submodular\nmaximization problem is not suited to attention score optimization in\nauto-regressive generation. Therefore instead of following how submodular\nfunction has been widely used, we propose a simplified yet principled solution.\nThe resulting attention module offers an architecturally simple and empirically\neffective method to improve the coverage of neural text generation. We run\nexperiments on three directed text generation tasks with different levels of\nrecovering rate, across two modalities, three different neural model\narchitectures and two training strategy variations. The results and analyses\ndemonstrate that our method generalizes well across these settings, produces\ntexts of good quality and outperforms state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03070v4",
            "date": "2019-11-08 06:07:25+00:00",
            "title": "Interactive Refinement of Cross-Lingual Word Embeddings",
            "authors": [
                "Michelle Yuan",
                "Mozhi Zhang",
                "Benjamin Van Durme",
                "Leah Findlater",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Cross-lingual word embeddings transfer knowledge between languages: models\ntrained on high-resource languages can predict in low-resource languages. We\nintroduce CLIME, an interactive system to quickly refine cross-lingual word\nembeddings for a given classification problem. First, CLIME ranks words by\ntheir salience to the downstream task. Then, users mark similarity between\nkeywords and their nearest neighbors in the embedding space. Finally, CLIME\nupdates the embeddings using the annotations. We evaluate CLIME on identifying\nhealth-related text in four low-resource languages: Ilocano, Sinhalese,\nTigrinya, and Uyghur. Embeddings refined by CLIME capture more nuanced word\nsemantics and have higher test accuracy than the original embeddings. CLIME\noften improves accuracy faster than an active learning baseline and can be\neasily combined with active learning to improve results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03083v1",
            "date": "2019-11-08 06:49:45+00:00",
            "title": "Are we asking the right questions in MovieQA?",
            "authors": [
                "Bhavan Jasani",
                "Rohit Girdhar",
                "Deva Ramanan"
            ],
            "abstract": "Joint vision and language tasks like visual question answering are\nfascinating because they explore high-level understanding, but at the same\ntime, can be more prone to language biases. In this paper, we explore the\nbiases in the MovieQA dataset and propose a strikingly simple model which can\nexploit them. We find that using the right word embedding is of utmost\nimportance. By using an appropriately trained word embedding, about half the\nQuestion-Answers (QAs) can be answered by looking at the questions and answers\nalone, completely ignoring narrative context from video clips, subtitles, and\nmovie scripts. Compared to the best published papers on the leaderboard, our\nsimple question + answer only model improves accuracy by 5% for video +\nsubtitle category, 5% for subtitle, 15% for DVS and 6% higher for scripts.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.03118v2",
            "date": "2019-11-08 08:30:22+00:00",
            "title": "Not Enough Data? Deep Learning to the Rescue!",
            "authors": [
                "Ateret Anaby-Tavor",
                "Boaz Carmeli",
                "Esther Goldbraich",
                "Amir Kantor",
                "George Kour",
                "Segev Shlomov",
                "Naama Tepper",
                "Naama Zwerdling"
            ],
            "abstract": "Based on recent advances in natural language modeling and those in text\ngeneration capabilities, we propose a novel data augmentation method for text\nclassification tasks. We use a powerful pre-trained neural network model to\nartificially synthesize new labeled data for supervised learning. We mainly\nfocus on cases with scarce labeled data. Our method, referred to as\nlanguage-model-based data augmentation (LAMBADA), involves fine-tuning a\nstate-of-the-art language generator to a specific task through an initial\ntraining phase on the existing (usually small) labeled data. Using the\nfine-tuned model and given a class label, new sentences for the class are\ngenerated. Our process then filters these new sentences by using a classifier\ntrained on the original data. In a series of experiments, we show that LAMBADA\nimproves classifiers' performance on a variety of datasets. Moreover, LAMBADA\nsignificantly improves upon the state-of-the-art techniques for data\naugmentation, specifically those applicable to text classification tasks with\nlittle data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03179v2",
            "date": "2019-11-08 10:52:43+00:00",
            "title": "Lipschitz Constrained Parameter Initialization for Deep Transformers",
            "authors": [
                "Hongfei Xu",
                "Qiuhui Liu",
                "Josef van Genabith",
                "Deyi Xiong",
                "Jingyi Zhang"
            ],
            "abstract": "The Transformer translation model employs residual connection and layer\nnormalization to ease the optimization difficulties caused by its multi-layer\nencoder/decoder structure. Previous research shows that even with residual\nconnection and layer normalization, deep Transformers still have difficulty in\ntraining, and particularly Transformer models with more than 12 encoder/decoder\nlayers fail to converge. In this paper, we first empirically demonstrate that a\nsimple modification made in the official implementation, which changes the\ncomputation order of residual connection and layer normalization, can\nsignificantly ease the optimization of deep Transformers. We then compare the\nsubtle differences in computation order in considerable detail, and present a\nparameter initialization method that leverages the Lipschitz constraint on the\ninitialization of Transformer parameters that effectively ensures training\nconvergence. In contrast to findings in previous research we further\ndemonstrate that with Lipschitz parameter initialization, deep Transformers\nwith the original computation order can converge, and obtain significant BLEU\nimprovements with up to 24 layers. In contrast to previous research which\nfocuses on deep encoders, our approach additionally enables Transformers to\nalso benefit from deep decoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03270v1",
            "date": "2019-11-08 14:03:55+00:00",
            "title": "Char-RNN and Active Learning for Hashtag Segmentation",
            "authors": [
                "Taisiya Glushkova",
                "Ekaterina Artemova"
            ],
            "abstract": "We explore the abilities of character recurrent neural network (char-RNN) for\nhashtag segmentation. Our approach to the task is the following: we generate\nsynthetic training dataset according to frequent n-grams that satisfy\npredefined morpho-syntactic patterns to avoid any manual annotation. The active\nlearning strategy limits the training dataset and selects informative training\nsubset. The approach does not require any language-specific settings and is\ncompared for two languages, which differ in inflection degree.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03350v1",
            "date": "2019-11-08 16:17:40+00:00",
            "title": "Ask to Learn: A Study on Curiosity-driven Question Generation",
            "authors": [
                "Thomas Scialom",
                "Jacopo Staiano"
            ],
            "abstract": "We propose a novel text generation task, namely Curiosity-driven Question\nGeneration. We start from the observation that the Question Generation task has\ntraditionally been considered as the dual problem of Question Answering, hence\ntackling the problem of generating a question given the text that contains its\nanswer. Such questions can be used to evaluate machine reading comprehension.\nHowever, in real life, and especially in conversational settings, humans tend\nto ask questions with the goal of enriching their knowledge and/or clarifying\naspects of previously gathered information. We refer to these inquisitive\nquestions as Curiosity-driven: these questions are generated with the goal of\nobtaining new information (the answer) which is not present in the input text.\nIn this work, we experiment on this new task using a conversational Question\nAnswering (QA) dataset; further, since the majority of QA dataset are not built\nin a conversational manner, we describe a methodology to derive data for this\nnovel task from non-conversational QA data. We investigate several automated\nmetrics to measure the different properties of Curious Questions, and\nexperiment different approaches on the Curiosity-driven Question Generation\ntask, including model pre-training and reinforcement learning. Finally, we\nreport a qualitative evaluation of the generated outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03353v2",
            "date": "2019-11-08 16:19:26+00:00",
            "title": "SEPT: Improving Scientific Named Entity Recognition with Span Representation",
            "authors": [
                "Tan Yan",
                "Heyan Huang",
                "Xian-Ling Mao"
            ],
            "abstract": "We introduce a new scientific named entity recognizer called SEPT, which\nstands for Span Extractor with Pre-trained Transformers. In recent papers, span\nextractors have been demonstrated to be a powerful model compared with sequence\nlabeling models. However, we discover that with the development of pre-trained\nlanguage models, the performance of span extractors appears to become similar\nto sequence labeling models. To keep the advantages of span representation, we\nmodified the model by under-sampling to balance the positive and negative\nsamples and reduce the search space. Furthermore, we simplify the origin\nnetwork architecture to combine the span extractor with BERT. Experiments\ndemonstrate that even simplified architecture achieves the same performance and\nSEPT achieves a new state of the art result in scientific named entity\nrecognition even without relation information involved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03531v1",
            "date": "2019-11-08 20:52:12+00:00",
            "title": "Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation",
            "authors": [
                "Ali Fadel",
                "Ibraheem Tuffaha",
                "Bara' Al-Jawarneh",
                "Mahmoud Al-Ayyoub"
            ],
            "abstract": "In this work, we present several deep learning models for the automatic\ndiacritization of Arabic text. Our models are built using two main approaches,\nviz. Feed-Forward Neural Network (FFNN) and Recurrent Neural Network (RNN),\nwith several enhancements such as 100-hot encoding, embeddings, Conditional\nRandom Field (CRF) and Block-Normalized Gradient (BNG). The models are tested\non the only freely available benchmark dataset and the results show that our\nmodels are either better or on par with other models, which require\nlanguage-dependent post-processing steps, unlike ours. Moreover, we show that\ndiacritics in Arabic can be used to enhance the models of NLP tasks such as\nMachine Translation (MT) by proposing the Translation over Diacritization (ToD)\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03561v4",
            "date": "2019-11-08 22:14:35+00:00",
            "title": "Graph-to-Graph Transformer for Transition-based Dependency Parsing",
            "authors": [
                "Alireza Mohammadshahi",
                "James Henderson"
            ],
            "abstract": "We propose the Graph2Graph Transformer architecture for conditioning on and\npredicting arbitrary graphs, and apply it to the challenging task of\ntransition-based dependency parsing. After proposing two novel Transformer\nmodels of transition-based dependency parsing as strong baselines, we show that\nadding the proposed mechanisms for conditioning on and predicting graphs of\nGraph2Graph Transformer results in significant improvements, both with and\nwithout BERT pre-training. The novel baselines and their integration with\nGraph2Graph Transformer significantly outperform the state-of-the-art in\ntraditional transition-based dependency parsing on both English Penn Treebank,\nand 13 languages of Universal Dependencies Treebanks. Graph2Graph Transformer\ncan be integrated with many previous structured prediction methods, making it\neasy to apply to a wide range of NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03562v1",
            "date": "2019-11-08 22:15:32+00:00",
            "title": "The State of NLP Literature: A Diachronic Analysis of the ACL Anthology",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "The ACL Anthology (AA) is a digital repository of tens of thousands of\narticles on Natural Language Processing (NLP). This paper examines the\nliterature as a whole to identify broad trends in productivity, focus, and\nimpact. It presents the analyses in a sequence of questions and answers. The\ngoal is to record the state of the AA literature: who and how many of us are\npublishing? what are we publishing on? where and in what form are we\npublishing? and what is the impact of our publications? The answers are usually\nin the form of numbers, graphs, and inter-connected visualizations. Special\nemphasis is laid on the demographics and inclusiveness of NLP publishing.\nNotably, we find that only about 30% of first authors are female, and that this\npercentage has not improved since the year 2000. We also show that, on average,\nfemale first authors are cited less than male first authors, even when\ncontrolling for experience. We hope that recording citation and participation\ngaps across demographic groups will encourage more inclusiveness and fairness\nin research.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03588v2",
            "date": "2019-11-09 00:22:05+00:00",
            "title": "MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models",
            "authors": [
                "Linqing Liu",
                "Huan Wang",
                "Jimmy Lin",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Pretrained language models have led to significant performance gains in many\nNLP tasks. However, the intensive computing resources to train such models\nremain an issue. Knowledge distillation alleviates this problem by learning a\nlight-weight student model. So far the distillation approaches are all\ntask-specific. In this paper, we explore knowledge distillation under the\nmulti-task learning setting. The student is jointly distilled across different\ntasks. It acquires more general representation capacity through multi-tasking\ndistillation and can be further fine-tuned to improve the model in the target\ndomain. Unlike other BERT distillation methods which specifically designed for\nTransformer-based architectures, we provide a general learning framework. Our\napproach is model agnostic and can be easily applied on different future\nteacher model architectures. We evaluate our approach on a Transformer-based\nand LSTM based student model. Compared to a strong, similarly LSTM-based\napproach, we achieve better quality under the same computational constraints.\nCompared to the present state of the art, we reach comparable results with much\nfaster inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03648v1",
            "date": "2019-11-09 09:33:42+00:00",
            "title": "Hate Speech Detection on Vietnamese Social Media Text using the Bidirectional-LSTM Model",
            "authors": [
                "Hang Thi-Thuy Do",
                "Huy Duc Huynh",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen",
                "Anh Gia-Tuan Nguyen"
            ],
            "abstract": "In this paper, we describe our system which participates in the shared task\nof Hate Speech Detection on Social Networks of VLSP 2019 evaluation campaign.\nWe are provided with the pre-labeled dataset and an unlabeled dataset for\nsocial media comments or posts. Our mission is to pre-process and build machine\nlearning models to classify comments/posts. In this report, we use\nBidirectional Long Short-Term Memory to build the model that can predict labels\nfor social media text according to Clean, Offensive, Hate. With this system, we\nachieve comparative results with 71.43% on the public standard test set of VLSP\n2019.",
            "journal ref": "VLSP Workshop 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03677v2",
            "date": "2019-11-09 12:33:47+00:00",
            "title": "A Reinforced Generation of Adversarial Examples for Neural Machine Translation",
            "authors": [
                "Wei Zou",
                "Shujian Huang",
                "Jun Xie",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Neural machine translation systems tend to fail on less decent inputs despite\nits significant efficacy, which may significantly harm the credibility of this\nsystems-fathoming how and when neural-based systems fail in such cases is\ncritical for industrial maintenance. Instead of collecting and analyzing bad\ncases using limited handcrafted error features, here we investigate this issue\nby generating adversarial examples via a new paradigm based on reinforcement\nlearning. Our paradigm could expose pitfalls for a given performance metric,\ne.g., BLEU, and could target any given neural machine translation architecture.\nWe conduct experiments of adversarial attacks on two mainstream neural machine\ntranslation architectures, RNN-search, and Transformer. The results show that\nour method efficiently produces stable attacks with meaning-preserving\nadversarial examples. We also present a qualitative and quantitative analysis\nfor the preference pattern of the attack, demonstrating its capability of\npitfall exposure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03678v1",
            "date": "2019-11-09 12:34:01+00:00",
            "title": "Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning",
            "authors": [
                "\u00c1kos K\u00e1d\u00e1r",
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi",
                "Desmond Elliott"
            ],
            "abstract": "Recent work has highlighted the advantage of jointly learning grounded\nsentence representations from multiple languages. However, the data used in\nthese studies has been limited to an aligned scenario: the same images\nannotated with sentences in multiple languages. We focus on the more realistic\ndisjoint scenario in which there is no overlap between the images in\nmultilingual image--caption datasets. We confirm that training with aligned\ndata results in better grounded sentence representations than training with\ndisjoint data, as measured by image--sentence retrieval performance. In order\nto close this gap in performance, we propose a pseudopairing method to generate\nsynthetically aligned English--German--image triplets from the disjoint sets.\nThe method works by first training a model on the disjoint data, and then\ncreating new triples across datasets using sentence similarity under the\nlearned model. Experiments show that pseudopairs improve image--sentence\nretrieval performance compared to disjoint training, despite requiring no\nexternal data or models. However, we do find that using an external machine\ntranslation model to generate the synthetic data sets results in better\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03738v1",
            "date": "2019-11-09 17:07:23+00:00",
            "title": "On Architectures for Including Visual Information in Neural Language Models for Image Description",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Kenneth P. Camilleri"
            ],
            "abstract": "A neural language model can be conditioned into generating descriptions for\nimages by providing visual information apart from the sentence prefix. This\nvisual information can be included into the language model through different\npoints of entry resulting in different neural architectures. We identify four\nmain architectures which we call init-inject, pre-inject, par-inject, and\nmerge.\n  We analyse these four architectures and conclude that the best performing one\nis init-inject, which is when the visual information is injected into the\ninitial state of the recurrent neural network. We confirm this using both\nautomatic evaluation measures and human annotation.\n  We then analyse how much influence the images have on each architecture. This\nis done by measuring how different the output probabilities of a model are when\na partial sentence is combined with a completely different image from the one\nit is meant to be combined with. We find that init-inject tends to quickly\nbecome less influenced by the image as more words are generated. A different\narchitecture called merge, which is when the visual information is merged with\nthe recurrent neural network's hidden state vector prior to output, loses\nvisual influence much more slowly, suggesting that it would work better for\ngenerating longer sentences.\n  We also observe that the merge architecture can have its recurrent neural\nnetwork pre-trained in a text-only language model (transfer learning) rather\nthan be initialised randomly as usual. This results in even better performance\nthan the other architectures, provided that the source language model is not\ntoo good at language modelling or it will overspecialise and be less effective\nat image description generation.\n  Our work opens up new avenues of research in neural architectures,\nexplainable AI, and transfer learning.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1911.03768v2",
            "date": "2019-11-09 20:05:06+00:00",
            "title": "The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents",
            "authors": [
                "Kurt Shuster",
                "Da Ju",
                "Stephen Roller",
                "Emily Dinan",
                "Y-Lan Boureau",
                "Jason Weston"
            ],
            "abstract": "We introduce dodecaDialogue: a set of 12 tasks that measures if a\nconversational agent can communicate engagingly with personality and empathy,\nask questions, answer questions by utilizing knowledge resources, discuss\ntopics and situations, and perceive and converse about images. By multi-tasking\non such a broad large-scale set of data, we hope to both move towards and\nmeasure progress in producing a single unified agent that can perceive, reason\nand converse with humans in an open-domain setting. We show that such\nmulti-tasking improves over a BERT pre-trained baseline, largely due to\nmulti-tasking with very large dialogue datasets in a similar domain, and that\nthe multi-tasking in general provides gains to both text and image-based tasks\nusing several metrics in both the fine-tune and task transfer settings. We\nobtain state-of-the-art results on many of the tasks, providing a strong\nbaseline for this challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03782v1",
            "date": "2019-11-09 21:36:28+00:00",
            "title": "Enforcing Encoder-Decoder Modularity in Sequence-to-Sequence Models",
            "authors": [
                "Siddharth Dalmia",
                "Abdelrahman Mohamed",
                "Mike Lewis",
                "Florian Metze",
                "Luke Zettlemoyer"
            ],
            "abstract": "Inspired by modular software design principles of independence,\ninterchangeability, and clarity of interface, we introduce a method for\nenforcing encoder-decoder modularity in seq2seq models without sacrificing the\noverall model quality or its full differentiability. We discretize the encoder\noutput units into a predefined interpretable vocabulary space using the\nConnectionist Temporal Classification (CTC) loss. Our modular systems achieve\nnear SOTA performance on the 300h Switchboard benchmark, with WER of 8.3% and\n17.6% on the SWB and CH subsets, using seq2seq models with encoder and decoder\nmodules which are independent and interchangeable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03829v3",
            "date": "2019-11-10 02:12:38+00:00",
            "title": "Distilling Knowledge Learned in BERT for Text Generation",
            "authors": [
                "Yen-Chun Chen",
                "Zhe Gan",
                "Yu Cheng",
                "Jingzhou Liu",
                "Jingjing Liu"
            ],
            "abstract": "Large-scale pre-trained language model such as BERT has achieved great\nsuccess in language understanding tasks. However, it remains an open question\nhow to utilize BERT for language generation. In this paper, we present a novel\napproach, Conditional Masked Language Modeling (C-MLM), to enable the\nfinetuning of BERT on target generation tasks. The finetuned BERT (teacher) is\nexploited as extra supervision to improve conventional Seq2Seq models (student)\nfor better text generation performance. By leveraging BERT's idiosyncratic\nbidirectional nature, distilling knowledge learned in BERT can encourage\nauto-regressive Seq2Seq models to plan ahead, imposing global sequence-level\nsupervision for coherent text generation. Experiments show that the proposed\napproach significantly outperforms strong Transformer baselines on multiple\nlanguage generation tasks such as machine translation and text summarization.\nOur proposed model also achieves new state of the art on IWSLT German-English\nand English-Vietnamese MT datasets. Code is available at\nhttps://github.com/ChenRocks/Distill-BERT-Textgen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03850v3",
            "date": "2019-11-10 04:41:31+00:00",
            "title": "Not All Claims are Created Equal: Choosing the Right Statistical Approach to Assess Hypotheses",
            "authors": [
                "Erfan Sadeqi Azer",
                "Daniel Khashabi",
                "Ashish Sabharwal",
                "Dan Roth"
            ],
            "abstract": "Empirical research in Natural Language Processing (NLP) has adopted a narrow\nset of principles for assessing hypotheses, relying mainly on p-value\ncomputation, which suffers from several known issues. While alternative\nproposals have been well-debated and adopted in other fields, they remain\nrarely discussed or used within the NLP community. We address this gap by\ncontrasting various hypothesis assessment techniques, especially those not\ncommonly used in the field (such as evaluations based on Bayesian inference).\nSince these statistical techniques differ in the hypotheses they can support,\nwe argue that practitioners should first decide their target hypothesis before\nchoosing an assessment method. This is crucial because common fallacies,\nmisconceptions, and misinterpretation surrounding hypothesis assessment methods\noften stem from a discrepancy between what one would like to claim versus what\nthe method used actually assesses. Our survey reveals that these issues are\nomnipresent in the NLP research community. As a step forward, we provide best\npractices and guidelines tailored to NLP research, as well as an easy-to-use\npackage called 'HyBayes' for Bayesian assessment of hypotheses, complementing\nexisting tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03861v2",
            "date": "2019-11-10 05:56:41+00:00",
            "title": "Increasing Robustness to Spurious Correlations using Forgettable Examples",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Soroush Mehri",
                "Remi Tachet",
                "T. J. Hazen",
                "Alessandro Sordoni"
            ],
            "abstract": "Neural NLP models tend to rely on spurious correlations between labels and\ninput features to perform their tasks. Minority examples, i.e., examples that\ncontradict the spurious correlations present in the majority of data points,\nhave been shown to increase the out-of-distribution generalization of\npre-trained language models. In this paper, we first propose using example\nforgetting to find minority examples without prior knowledge of the spurious\ncorrelations present in the dataset. Forgettable examples are instances either\nlearned and then forgotten during training or never learned. We empirically\nshow how these examples are related to minorities in our training sets. Then,\nwe introduce a new approach to robustify models by fine-tuning our models\ntwice, first on the full training data and second on the minorities only. We\nobtain substantial improvements in out-of-distribution generalization when\napplying our approach to the MNLI, QQP, and FEVER datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03863v3",
            "date": "2019-11-10 06:10:47+00:00",
            "title": "Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks",
            "authors": [
                "Trapit Bansal",
                "Rishikesh Jha",
                "Andrew McCallum"
            ],
            "abstract": "Self-supervised pre-training of transformer models has shown enormous success\nin improving performance on a number of downstream tasks. However, fine-tuning\non a new task still requires large amounts of task-specific labelled data to\nachieve good performance. We consider this problem of learning to generalize to\nnew tasks with few examples as a meta-learning problem. While meta-learning has\nshown tremendous progress in recent years, its application is still limited to\nsimulated problems or problems with limited diversity across tasks. We develop\na novel method, LEOPARD, which enables optimization-based meta-learning across\ntasks with different number of classes, and evaluate different methods on\ngeneralization to diverse NLP classification tasks. LEOPARD is trained with the\nstate-of-the-art transformer architecture and shows better generalization to\ntasks not seen at all during training, with as few as 4 examples per label.\nAcross 17 NLP tasks, including diverse domains of entity typing, natural\nlanguage inference, sentiment analysis, and several other text classification\ntasks, we show that LEOPARD learns better initial parameters for few-shot\nlearning than self-supervised pre-training or multi-task training,\noutperforming many strong baselines, for example, yielding 14.5% average\nrelative gain in accuracy on unseen tasks with only 4 examples per label.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03864v2",
            "date": "2019-11-10 06:14:15+00:00",
            "title": "Improving Transformer Models by Reordering their Sublayers",
            "authors": [
                "Ofir Press",
                "Noah A. Smith",
                "Omer Levy"
            ],
            "abstract": "Multilayer transformer networks consist of interleaved self-attention and\nfeedforward sublayers. Could ordering the sublayers in a different pattern lead\nto better performance? We generate randomly ordered transformers and train them\nwith the language modeling objective. We observe that some of these models are\nable to achieve better performance than the interleaved baseline, and that\nthose successful variants tend to have more self-attention at the bottom and\nmore feedforward sublayers at the top. We propose a new transformer pattern\nthat adheres to this property, the sandwich transformer, and show that it\nimproves perplexity on multiple word-level and character-level language\nmodeling benchmarks, at no cost in parameters, memory, or training time.\nHowever, the sandwich reordering pattern does not guarantee performance gains\nacross every task, as we demonstrate on machine translation models. Instead, we\nsuggest that further exploration of task-specific sublayer reorderings is\nneeded in order to unlock additional gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03868v2",
            "date": "2019-11-10 06:58:44+00:00",
            "title": "Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering",
            "authors": [
                "Sewon Min",
                "Danqi Chen",
                "Luke Zettlemoyer",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce an approach for open-domain question answering (QA) that\nretrieves and reads a passage graph, where vertices are passages of text and\nedges represent relationships that are derived from an external knowledge base\nor co-occurrence in the same article. Our goals are to boost coverage by using\nknowledge-guided retrieval to find more relevant passages than text-matching\nmethods, and to improve accuracy by allowing for better knowledge-guided fusion\nof information across related passages. Our graph retrieval method expands a\nset of seed keyword-retrieved passages by traversing the graph structure of the\nknowledge base. Our reader extends a BERT-based architecture and updates\npassage representations by propagating information from related passages and\ntheir relations, instead of reading each passage in isolation. Experiments on\nthree open-domain QA datasets, WebQuestions, Natural Questions and TriviaQA,\nshow improved performance over non-graph baselines by 2-11% absolute. Our\napproach also matches or exceeds the state-of-the-art in every case, without\nusing an expensive end-to-end training regime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03875v3",
            "date": "2019-11-10 08:17:11+00:00",
            "title": "Rethinking Self-Attention: Towards Interpretability in Neural Parsing",
            "authors": [
                "Khalil Mrini",
                "Franck Dernoncourt",
                "Quan Tran",
                "Trung Bui",
                "Walter Chang",
                "Ndapa Nakashole"
            ],
            "abstract": "Attention mechanisms have improved the performance of NLP tasks while\nallowing models to remain explainable. Self-attention is currently widely used,\nhowever interpretability is difficult due to the numerous attention\ndistributions. Recent work has shown that model representations can benefit\nfrom label-specific information, while facilitating interpretation of\npredictions. We introduce the Label Attention Layer: a new form of\nself-attention where attention heads represent labels. We test our novel layer\nby running constituency and dependency parsing experiments and show our new\nmodel obtains new state-of-the-art results for both tasks on both the Penn\nTreebank (PTB) and Chinese Treebank. Additionally, our model requires fewer\nself-attention layers compared to existing work. Finally, we find that the\nLabel Attention heads learn relations between syntactic categories and show\npathways to analyze errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03895v2",
            "date": "2019-11-10 10:48:09+00:00",
            "title": "A Bilingual Generative Transformer for Semantic Sentence Embedding",
            "authors": [
                "John Wieting",
                "Graham Neubig",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Semantic sentence embedding models encode natural language sentences into\nvectors, such that closeness in embedding space indicates closeness in the\nsemantics between the sentences. Bilingual data offers a useful signal for\nlearning such embeddings: properties shared by both sentences in a translation\npair are likely semantic, while divergent properties are likely stylistic or\nlanguage-specific. We propose a deep latent variable model that attempts to\nperform source separation on parallel sentences, isolating what they have in\ncommon in a latent semantic vector, and explaining what is left over with\nlanguage-specific latent vectors. Our proposed approach differs from past work\non semantic sentence encoding in two ways. First, by using a variational\nprobabilistic framework, we introduce priors that encourage source separation,\nand can use our model's posterior to predict sentence embeddings for\nmonolingual data at test time. Second, we use high-capacity transformers as\nboth data generating distributions and inference networks -- contrasting with\nmost past work on sentence embeddings. In experiments, our approach\nsubstantially outperforms the state-of-the-art on a standard suite of\nunsupervised semantic similarity evaluations. Further, we demonstrate that our\napproach yields the largest gains on more difficult subsets of these\nevaluations where simple word overlap is not a good indicator of similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03898v1",
            "date": "2019-11-10 10:56:10+00:00",
            "title": "Understanding Multi-Head Attention in Abstractive Summarization",
            "authors": [
                "Joris Baan",
                "Maartje ter Hoeve",
                "Marlies van der Wees",
                "Anne Schuth",
                "Maarten de Rijke"
            ],
            "abstract": "Attention mechanisms in deep learning architectures have often been used as a\nmeans of transparency and, as such, to shed light on the inner workings of the\narchitectures. Recently, there has been a growing interest in whether or not\nthis assumption is correct. In this paper we investigate the interpretability\nof multi-head attention in abstractive summarization, a sequence-to-sequence\ntask for which attention does not have an intuitive alignment role, such as in\nmachine translation. We first introduce three metrics to gain insight in the\nfocus of attention heads and observe that these heads specialize towards\nrelative positions, specific part-of-speech tags, and named entities. However,\nwe also find that ablating and pruning these heads does not lead to a\nsignificant drop in performance, indicating redundancy. By replacing the\nsoftmax activation functions with sparsemax activation functions, we find that\nattention heads behave seemingly more transparent: we can ablate fewer heads\nand heads score higher on our interpretability metrics. However, if we apply\npruning to the sparsemax model we find that we can prune even more heads,\nraising the question whether enforced sparsity actually improves transparency.\nFinally, we find that relative positions heads seem integral to summarization\nperformance and persistently remain after pruning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03912v3",
            "date": "2019-11-10 11:50:14+00:00",
            "title": "Effectiveness of self-supervised pre-training for speech recognition",
            "authors": [
                "Alexei Baevski",
                "Michael Auli",
                "Abdelrahman Mohamed"
            ],
            "abstract": "We compare self-supervised representation learning algorithms which either\nexplicitly quantize the audio data or learn representations without\nquantization. We find the former to be more accurate since it builds a good\nvocabulary of the data through vq-wav2vec [1] to enable learning of effective\nrepresentations in subsequent BERT training. Different to previous work, we\ndirectly fine-tune the pre-trained BERT models on transcribed speech using a\nConnectionist Temporal Classification (CTC) loss instead of feeding the\nrepresentations into a task-specific model. We also propose a BERT-style model\nlearning directly from the continuous audio data and compare pre-training on\nraw audio to spectral features. Fine-tuning a BERT model on 10 hour of labeled\nLibrispeech data with a vq-wav2vec vocabulary is almost as good as the best\nknown reported system trained on 100 hours of labeled data on testclean, while\nachieving a 25% WER reduction on test-other. When using only 10 minutes of\nlabeled data, WER is 25.2 on test-other and 16.3 on test-clean. This\ndemonstrates that self-supervision can enable speech recognition systems\ntrained on a near-zero amount of transcribed data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04065v2",
            "date": "2019-11-11 04:06:46+00:00",
            "title": "Learning to Order Sub-questions for Complex Question Answering",
            "authors": [
                "Yunan Zhang",
                "Xiang Cheng",
                "Yufeng Zhang",
                "Zihan Wang",
                "Zhengqi Fang",
                "Xiaoyan Wang",
                "Zhenya Huang",
                "Chengxiang Zhai"
            ],
            "abstract": "Answering complex questions involving multiple entities and relations is a\nchallenging task. Logically, the answer to a complex question should be derived\nby decomposing the complex question into multiple simple sub-questions and then\nanswering those sub-questions. Existing work has followed this strategy but has\nnot attempted to optimize the order of how those sub-questions are answered. As\na result, the sub-questions are answered in an arbitrary order, leading to\nlarger search space and a higher risk of missing an answer. In this paper, we\npropose a novel reinforcement learning(RL) approach to answering complex\nquestions that can learn a policy to dynamically decide which sub-question\nshould be answered at each stage of reasoning. We lever-age the expected\nvalue-variance criterion to enable the learned policy to balance between the\nrisk and utility of answering a sub-question. Experiment results show that the\nRL approach can substantially improve the optimality of ordering the\nsub-questions, leading to improved accuracy of question answering. The proposed\nmethod for learning to order sub-questions is general and can thus be\npotentially combined with many existing ideas for answering complex questions\nto enhance their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04070v1",
            "date": "2019-11-11 04:31:23+00:00",
            "title": "BP-Transformer: Modelling Long-Range Context via Binary Partitioning",
            "authors": [
                "Zihao Ye",
                "Qipeng Guo",
                "Quan Gan",
                "Xipeng Qiu",
                "Zheng Zhang"
            ],
            "abstract": "The Transformer model is widely successful on many natural language\nprocessing tasks. However, the quadratic complexity of self-attention limit its\napplication on long text. In this paper, adopting a fine-to-coarse attention\nmechanism on multi-scale spans via binary partitioning (BP), we propose\nBP-Transformer (BPT for short). BPT yields $O(k\\cdot n\\log (n/k))$ connections\nwhere $k$ is a hyperparameter to control the density of attention. BPT has a\ngood balance between computation complexity and model capacity. A series of\nexperiments on text classification, machine translation and language modeling\nshows BPT has a superior performance for long text than previous self-attention\nmodels. Our code, hyperparameters and CUDA kernels for sparse attention are\navailable in PyTorch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04081v1",
            "date": "2019-11-11 05:22:26+00:00",
            "title": "Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables",
            "authors": [
                "Zihan Liu",
                "Jamin Shin",
                "Yan Xu",
                "Genta Indra Winata",
                "Peng Xu",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Despite the surging demands for multilingual task-oriented dialog systems\n(e.g., Alexa, Google Home), there has been less research done in multilingual\nor cross-lingual scenarios. Hence, we propose a zero-shot adaptation of\ntask-oriented dialogue system to low-resource languages. To tackle this\nchallenge, we first use a set of very few parallel word pairs to refine the\naligned cross-lingual word-level representations. We then employ a latent\nvariable model to cope with the variance of similar sentences across different\nlanguages, which is induced by imperfect cross-lingual alignments and inherent\ndifferences in languages. Finally, the experimental results show that even\nthough we utilize much less external resources, our model achieves better\nadaptation performance for natural language understanding task (i.e., the\nintent detection and slot filling) compared to the current state-of-the-art\nmodel in the zero-shot scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04156v2",
            "date": "2019-11-11 10:07:57+00:00",
            "title": "Meta Answering for Machine Reading",
            "authors": [
                "Benjamin Borschinger",
                "Jordan Boyd-Graber",
                "Christian Buck",
                "Jannis Bulian",
                "Massimiliano Ciaramita",
                "Michelle Chen Huebscher",
                "Wojciech Gajewski",
                "Yannic Kilcher",
                "Rodrigo Nogueira",
                "Lierni Sestorain Saralegu"
            ],
            "abstract": "We investigate a framework for machine reading, inspired by real world\ninformation-seeking problems, where a meta question answering system interacts\nwith a black box environment. The environment encapsulates a competitive\nmachine reader based on BERT, providing candidate answers to questions, and\npossibly some context. To validate the realism of our formulation, we ask\nhumans to play the role of a meta-answerer. With just a small snippet of text\naround an answer, humans can outperform the machine reader, improving recall.\nSimilarly, a simple machine meta-answerer outperforms the environment,\nimproving both precision and recall on the Natural Questions dataset. The\nsystem relies on joint training of answer scoring and the selection of\nconditioning information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04286v1",
            "date": "2019-11-11 14:07:46+00:00",
            "title": "Deep Contextualized Self-training for Low Resource Dependency Parsing",
            "authors": [
                "Guy Rotman",
                "Roi Reichart"
            ],
            "abstract": "Neural dependency parsing has proven very effective, achieving\nstate-of-the-art results on numerous domains and languages. Unfortunately, it\nrequires large amounts of labeled data, that is costly and laborious to create.\nIn this paper we propose a self-training algorithm that alleviates this\nannotation bottleneck by training a parser on its own output. Our Deep\nContextualized Self-training (DCST) algorithm utilizes representation models\ntrained on sequence labeling tasks that are derived from the parser's output\nwhen applied to unlabeled data, and integrates these models with the base\nparser through a gating mechanism. We conduct experiments across multiple\nlanguages, both in low resource in-domain and in cross-domain setups, and\ndemonstrate that DCST substantially outperforms traditional self-training as\nwell as recent semi-supervised training methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04474v3",
            "date": "2019-11-10 15:05:48+00:00",
            "title": "TENER: Adapting Transformer Encoder for Named Entity Recognition",
            "authors": [
                "Hang Yan",
                "Bocao Deng",
                "Xiaonan Li",
                "Xipeng Qiu"
            ],
            "abstract": "The Bidirectional long short-term memory networks (BiLSTM) have been widely\nused as an encoder in models solving the named entity recognition (NER) task.\nRecently, the Transformer is broadly adopted in various Natural Language\nProcessing (NLP) tasks owing to its parallelism and advantageous performance.\nNevertheless, the performance of the Transformer in NER is not as good as it is\nin other NLP tasks. In this paper, we propose TENER, a NER architecture\nadopting adapted Transformer Encoder to model the character-level features and\nword-level features. By incorporating the direction and relative distance aware\nattention and the un-scaled attention, we prove the Transformer-like encoder is\njust as effective for NER as other NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04879v1",
            "date": "2019-11-12 14:25:53+00:00",
            "title": "A Survey on Why-Type Question Answering Systems",
            "authors": [
                "Manvi Breja",
                "Sanjay Kumar Jain"
            ],
            "abstract": "Search engines such as Google, Yahoo and Baidu yield information in the form\nof a relevant set of web pages according to the need of the user. Question\nAnswering Systems reduce the time taken to get an answer, to a query asked in\nnatural language by providing the one most relevant answer. To the best of our\nknowledge, major research in Why-type questions began in early 2000's and our\nwork on Why-type questions can help explore newer avenues for fact-finding and\nanalysis. The paper presents a survey on Why-type Question Answering System,\ndetails the architecture, the processes involved in the system and suggests\nfurther areas of research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04913v1",
            "date": "2019-11-12 14:53:34+00:00",
            "title": "Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?",
            "authors": [
                "Brij Mohan Lal Srivastava",
                "Aur\u00e9lien Bellet",
                "Marc Tommasi",
                "Emmanuel Vincent"
            ],
            "abstract": "Automatic speech recognition (ASR) is a key technology in many services and\napplications. This typically requires user devices to send their speech data to\nthe cloud for ASR decoding. As the speech signal carries a lot of information\nabout the speaker, this raises serious privacy concerns. As a solution, an\nencoder may reside on each user device which performs local computations to\nanonymize the representation. In this paper, we focus on the protection of\nspeaker identity and study the extent to which users can be recognized based on\nthe encoded representation of their speech as obtained by a deep\nencoder-decoder architecture trained for ASR. Through speaker identification\nand verification experiments on the Librispeech corpus with open and closed\nsets of speakers, we show that the representations obtained from a standard\narchitecture still carry a lot of information about speaker identity. We then\npropose to use adversarial training to learn representations that perform well\nin ASR while hiding speaker identity. Our results demonstrate that adversarial\ntraining dramatically reduces the closed-set classification accuracy, but this\ndoes not translate into increased open-set verification error hence into\nincreased protection of the speaker identity in practice. We suggest several\npossible reasons behind this negative result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04942v4",
            "date": "2019-11-10 09:09:13+00:00",
            "title": "RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers",
            "authors": [
                "Bailin Wang",
                "Richard Shin",
                "Xiaodong Liu",
                "Oleksandr Polozov",
                "Matthew Richardson"
            ],
            "abstract": "When translating natural language questions into SQL queries to answer\nquestions from a database, contemporary semantic parsing models struggle to\ngeneralize to unseen database schemas. The generalization challenge lies in (a)\nencoding the database relations in an accessible way for the semantic parser,\nand (b) modeling alignment between database columns and their mentions in a\ngiven query. We present a unified framework, based on the relation-aware\nself-attention mechanism, to address schema encoding, schema linking, and\nfeature representation within a text-to-SQL encoder. On the challenging Spider\ndataset this framework boosts the exact match accuracy to 57.2%, surpassing its\nbest counterparts by 8.7% absolute improvement. Further augmented with BERT, it\nachieves the new state-of-the-art performance of 65.6% on the Spider\nleaderboard. In addition, we observe qualitative improvements in the model's\nunderstanding of schema linking and alignment. Our implementation will be\nopen-sourced at https://github.com/Microsoft/rat-sql.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05034v2",
            "date": "2019-11-12 17:50:35+00:00",
            "title": "Learning Sparse Sharing Architectures for Multiple Tasks",
            "authors": [
                "Tianxiang Sun",
                "Yunfan Shao",
                "Xiaonan Li",
                "Pengfei Liu",
                "Hang Yan",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Most existing deep multi-task learning models are based on parameter sharing,\nsuch as hard sharing, hierarchical sharing, and soft sharing. How choosing a\nsuitable sharing mechanism depends on the relations among the tasks, which is\nnot easy since it is difficult to understand the underlying shared factors\namong these tasks. In this paper, we propose a novel parameter sharing\nmechanism, named \\emph{Sparse Sharing}. Given multiple tasks, our approach\nautomatically finds a sparse sharing structure. We start with an\nover-parameterized base network, from which each task extracts a subnetwork.\nThe subnetworks of multiple tasks are partially overlapped and trained in\nparallel. We show that both hard sharing and hierarchical sharing can be\nformulated as particular instances of the sparse sharing framework. We conduct\nextensive experiments on three sequence labeling tasks. Compared with\nsingle-task models and three typical multi-task learning baselines, our\nproposed approach achieves consistent improvement while requiring fewer\nparameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05153v1",
            "date": "2019-11-12 21:34:15+00:00",
            "title": "Improving Robustness of Task Oriented Dialog Systems",
            "authors": [
                "Arash Einolghozati",
                "Sonal Gupta",
                "Mrinal Mohit",
                "Rushin Shah"
            ],
            "abstract": "Task oriented language understanding in dialog systems is often modeled using\nintents (task of a query) and slots (parameters for that task). Intent\ndetection and slot tagging are, in turn, modeled using sentence classification\nand word tagging techniques respectively. Similar to adversarial attack\nproblems with computer vision models discussed in existing literature, these\nintent-slot tagging models are often over-sensitive to small variations in\ninput -- predicting different and often incorrect labels when small changes are\nmade to a query, thus reducing their accuracy and reliability. However,\nevaluating a model's robustness to these changes is harder for language since\nwords are discrete and an automated change (e.g. adding `noise') to a query\nsometimes changes the meaning and thus labels of a query. In this paper, we\nfirst describe how to create an adversarial test set to measure the robustness\nof these models. Furthermore, we introduce and adapt adversarial training\nmethods as well as data augmentation using back-translation to mitigate these\nissues. Our experiments show that both techniques improve the robustness of the\nsystem substantially and can be combined to yield the best results.",
            "journal ref": "3rd Conversational AI Workshop at 33rd Conference on Neural\n  Information Processing Systems (NeurIPS 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05202v1",
            "date": "2019-11-12 23:31:12+00:00",
            "title": "Creating Auxiliary Representations from Charge Definitions for Criminal Charge Prediction",
            "authors": [
                "Liangyi Kang",
                "Jie Liu",
                "Lingqiao Liu",
                "Qinfeng Shi",
                "Dan Ye"
            ],
            "abstract": "Charge prediction, determining charges for criminal cases by analyzing the\ntextual fact descriptions, is a promising technology in legal assistant\nsystems. In practice, the fact descriptions could exhibit a significant\nintra-class variation due to factors like non-normative use of language, which\nmakes the prediction task very challenging, especially for charge classes with\ntoo few samples to cover the expression variation. In this work, we explore to\nuse the charge definitions from criminal law to alleviate this issue. The key\nidea is that the expressions in a fact description should have corresponding\nformal terms in charge definitions, and those terms are shared across classes\nand could account for the diversity in the fact descriptions. Thus, we propose\nto create auxiliary fact representations from charge definitions to augment\nfact descriptions representation. The generated auxiliary representations are\ncreated through the interaction of fact description with the relevant charge\ndefinitions and terms in those definitions by integrated sentence- and\nword-level attention scheme. Experimental results on two datasets show that our\nmodel achieves significant improvement than baselines, especially for classes\nwith few samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05241v1",
            "date": "2019-11-13 01:52:27+00:00",
            "title": "Robustness to Capitalization Errors in Named Entity Recognition",
            "authors": [
                "Sravan Bodapati",
                "Hyokun Yun",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Robustness to capitalization errors is a highly desirable characteristic of\nnamed entity recognizers, yet we find standard models for the task are\nsurprisingly brittle to such noise. Existing methods to improve robustness to\nthe noise completely discard given orthographic information, mwhich\nsignificantly degrades their performance on well-formed text. We propose a\nsimple alternative approach based on data augmentation, which allows the model\nto \\emph{learn} to utilize or ignore orthographic information depending on its\nusefulness in the context. It achieves competitive robustness to capitalization\nerrors while making negligible compromise to its performance on well-formed\ntext and significantly improving generalization power on noisy user-generated\ntext. Our experiments clearly and consistently validate our claim across\ndifferent types of machine learning models, languages, and dataset sizes.",
            "journal ref": "http://noisy-text.github.io/2019/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05263v1",
            "date": "2019-11-13 02:48:33+00:00",
            "title": "LexiPers: An ontology based sentiment lexicon for Persian",
            "authors": [
                "Behnam Sabeti",
                "Pedram Hosseini",
                "Gholamreza Ghassem-Sani",
                "Seyed Abolghasem Mirroshandel"
            ],
            "abstract": "Sentiment analysis refers to the use of natural language processing to\nidentify and extract subjective information from textual resources. One\napproach for sentiment extraction is using a sentiment lexicon. A sentiment\nlexicon is a set of words associated with the sentiment orientation that they\nexpress. In this paper, we describe the process of generating a general purpose\nsentiment lexicon for Persian. A new graph-based method is introduced for seed\nselection and expansion based on an ontology. Sentiment lexicon generation is\nthen mapped to a document classification problem. We used the K-nearest\nneighbors and nearest centroid methods for classification. These classifiers\nhave been evaluated based on a set of hand labeled synsets. The final sentiment\nlexicon has been generated by the best classifier. The results show an\nacceptable performance in terms of accuracy and F-measure in the generated\nsentiment lexicon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05343v1",
            "date": "2019-11-13 08:11:42+00:00",
            "title": "A Stable Variational Autoencoder for Text Modelling",
            "authors": [
                "Ruizhe Li",
                "Xiao Li",
                "Chenghua Lin",
                "Matthew Collinson",
                "Rui Mao"
            ],
            "abstract": "Variational Autoencoder (VAE) is a powerful method for learning\nrepresentations of high-dimensional data. However, VAEs can suffer from an\nissue known as latent variable collapse (or KL loss vanishing), where the\nposterior collapses to the prior and the model will ignore the latent codes in\ngenerative tasks. Such an issue is particularly prevalent when employing\nVAE-RNN architectures for text modelling (Bowman et al., 2016). In this paper,\nwe present a simple architecture called holistic regularisation VAE (HR-VAE),\nwhich can effectively avoid latent variable collapse. Compared to existing\nVAE-RNN architectures, we show that our model can achieve much more stable\ntraining process and can generate text with significantly better quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05483v1",
            "date": "2019-10-26 08:38:19+00:00",
            "title": "Sentiment Analysis for Arabic in Social Media Network: A Systematic Mapping Study",
            "authors": [
                "Mohamed Elhag M. Abo",
                "Ram Gopal Raj",
                "Atika Qazi",
                "Abubakar Zakari"
            ],
            "abstract": "With the expansion in tenders on the Internet and social media, Arabic\nSentiment Analysis (ASA) has assumed a significant position in the field of\ntext mining study and has since remained used to explore the sentiments of\nusers about services, various products or topics conversed over the Internet.\nThis mapping paper designs to comprehensively investigate the papers\ndemographics, fertility, and directions of the ASA research domain.\nFurthermore, plans to analyze current ASA techniques and find movements in the\nresearch. This paper describes a systematic mapping study (SMS) of 51 primary\nselected studies (PSS) is handled with the approval of an evidence-based\nsystematic method to ensure handling of all related papers. The analyzed\nresults showed the increase of both the ASA research area and numbers of\npublications per year since 2015. Three main research facets were found, i.e.\nvalidation, solution, and evaluation research, with solution research becoming\nmore treatment than another research type. Therefore numerous contribution\nfacets were singled out. In totality, the general demographics of the ASA\nresearch field were highlighted and discussed",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.05594v2",
            "date": "2019-11-13 16:38:30+00:00",
            "title": "Neural Duplicate Question Detection without Labeled Training Data",
            "authors": [
                "Andreas R\u00fcckl\u00e9",
                "Nafise Sadat Moosavi",
                "Iryna Gurevych"
            ],
            "abstract": "Supervised training of neural models to duplicate question detection in\ncommunity Question Answering (cQA) requires large amounts of labeled question\npairs, which are costly to obtain. To minimize this cost, recent works thus\noften used alternative methods, e.g., adversarial domain adaptation. In this\nwork, we propose two novel methods: (1) the automatic generation of duplicate\nquestions, and (2) weak supervision using the title and body of a question. We\nshow that both can achieve improved performances even though they do not\nrequire any labeled data. We provide comprehensive comparisons of popular\ntraining strategies, which provides important insights on how to best train\nmodels in different scenarios. We show that our proposed approaches are more\neffective in many cases because they can utilize larger amounts of unlabeled\ndata from cQA forums. Finally, we also show that our proposed approach for weak\nsupervision with question title and body information is also an effective\nmethod to train cQA answer selection models without direct answer supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05889v4",
            "date": "2019-11-14 01:47:53+00:00",
            "title": "Generating Persona Consistent Dialogues by Exploiting Natural Language Inference",
            "authors": [
                "Haoyu Song",
                "Wei-Nan Zhang",
                "Jingwen Hu",
                "Ting Liu"
            ],
            "abstract": "Consistency is one of the major challenges faced by dialogue agents. A\nhuman-like dialogue agent should not only respond naturally, but also maintain\na consistent persona. In this paper, we exploit the advantages of natural\nlanguage inference (NLI) technique to address the issue of generating persona\nconsistent dialogues. Different from existing work that re-ranks the retrieved\nresponses through an NLI model, we cast the task as a reinforcement learning\nproblem and propose to exploit the NLI signals from response-persona pairs as\nrewards for the process of dialogue generation. Specifically, our generator\nemploys an attention-based encoder-decoder to generate persona-based responses.\nOur evaluator consists of two components: an adversarially trained naturalness\nmodule and an NLI based consistency module. Moreover, we use another\nwell-performed NLI model in the evaluation of persona-consistency. Experimental\nresults on both human and automatic metrics, including the model-based\nconsistency evaluation, demonstrate that the proposed approach outperforms\nstrong generative baselines, especially in the persona-consistency of generated\nresponses.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.05953v1",
            "date": "2019-11-14 05:55:17+00:00",
            "title": "Ethanos: Lightweight Bootstrapping for Ethereum",
            "authors": [
                "Jae-Yun Kim",
                "Jun-Mo Lee",
                "Yeon-Jae Koo",
                "Sang-Hyeon Park",
                "Soo-Mook Moon"
            ],
            "abstract": "As ethereum blockchain has become popular, the number of users and\ntransactions has skyrocketed, causing an explosive increase of its data size.\nAs a result, ordinary clients using PCs or smartphones cannot easily bootstrap\nas a full node, but rely on other full nodes such as the miners to run or\nverify transactions. This may affect the security of ethereum, so light\nbootstrapping techniques such as fast sync has been proposed to download only\nparts of full data, yet the space overhead is still too high. One of the\nbiggest space overhead that cannot easily be reduced is caused by saving the\nstate of all accounts in the block's state trie. Fortunately, we found that\nmore than 90% of accounts are inactive and old transactions are hard to be\nmanipulated. Based on these observations, this paper propose a novel\noptimization technique called ethanos that can reduce bootstrapping cost by\nsweeping inactive accounts periodically and by not downloading old\ntransactions. If an inactive account becomes active, ethanos restore its state\nby running a restoration transaction. Also, ethanos gives incentives for\narchive nodes to maintain the old transactions for possible re-verification. We\nimplemented ethanos by instrumenting the go-ethereum (geth) client and\nevaluated with the real 113 million transactions from 14 million accounts\nbetween 7M-th and 8M-th blocks in ethereum. Our experimental result shows that\nethanos can reduce the size of the account state by half, which, if combined\nwith removing old transactions, may reduce the storage size for bootstrapping\nto around 1GB. This would be reasonable enough for ordinary clients to\nbootstrap on their personal devices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06137v5",
            "date": "2019-11-13 00:54:39+00:00",
            "title": "Unsupervised Domain Adaptation on Reading Comprehension",
            "authors": [
                "Yu Cao",
                "Meng Fang",
                "Baosheng Yu",
                "Joey Tianyi Zhou"
            ],
            "abstract": "Reading comprehension (RC) has been studied in a variety of datasets with the\nboosted performance brought by deep neural networks. However, the\ngeneralization capability of these models across different domains remains\nunclear. To alleviate this issue, we are going to investigate unsupervised\ndomain adaptation on RC, wherein a model is trained on labeled source domain\nand to be applied to the target domain with only unlabeled samples. We first\nshow that even with the powerful BERT contextual representation, the\nperformance is still unsatisfactory when the model trained on one dataset is\ndirectly applied to another target dataset. To solve this, we provide a novel\nconditional adversarial self-training method (CASe). Specifically, our approach\nleverages a BERT model fine-tuned on the source dataset along with the\nconfidence filtering to generate reliable pseudo-labeled samples in the target\ndomain for self-training. On the other hand, it further reduces domain\ndistribution discrepancy through conditional adversarial learning across\ndomains. Extensive experiments show our approach achieves comparable accuracy\nto supervised models on multiple large-scale benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06155v2",
            "date": "2019-11-11 05:30:53+00:00",
            "title": "RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems",
            "authors": [
                "Jianmin Guo",
                "Yue Zhao",
                "Quan Zhang",
                "Yu Jiang"
            ],
            "abstract": "While massive efforts have been investigated in adversarial testing of\nconvolutional neural networks (CNN), testing for recurrent neural networks\n(RNN) is still limited and leaves threats for vast sequential application\ndomains. In this paper, we propose an adversarial testing framework RNN-Test\nfor RNN systems, focusing on the main sequential domains, not only\nclassification tasks. First, we design a novel search methodology customized\nfor RNN models by maximizing the inconsistency of RNN states to produce\nadversarial inputs. Next, we introduce two state-based coverage metrics\naccording to the distinctive structure of RNNs to explore more inference\nlogics. Finally, RNN-Test solves the joint optimization problem to maximize\nstate inconsistency and state coverage, and crafts adversarial inputs for\nvarious tasks of different kinds of inputs.\n  For evaluations, we apply RNN-Test on three sequential models of common RNN\nstructures. On the tested models, the RNN-Test approach is demonstrated to be\ncompetitive in generating adversarial inputs, outperforming FGSM-based and\nDLFuzz-based methods to reduce the model performance more sharply with 2.78% to\n32.5% higher success (or generation) rate. RNN-Test could also achieve 52.65%\nto 66.45% higher adversary rate on MNIST-LSTM model than relevant work testRNN.\nCompared with the neuron coverage, the proposed state coverage metrics as\nguidance excel with 4.17% to 97.22% higher success (or generation) rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06241v1",
            "date": "2019-11-03 07:21:41+00:00",
            "title": "BERT-CNN: a Hierarchical Patent Classifier Based on a Pre-Trained Language Model",
            "authors": [
                "Xiaolei Lu",
                "Bin Ni"
            ],
            "abstract": "The automatic classification is a process of automatically assigning text\ndocuments to predefined categories. An accurate automatic patent classifier is\ncrucial to patent inventors and patent examiners in terms of intellectual\nproperty protection, patent management, and patent information retrieval. We\npresent BERT-CNN, a hierarchical patent classifier based on pre-trained\nlanguage model by training the national patent application documents collected\nfrom the State Information Center, China. The experimental results show that\nBERT-CNN achieves 84.3% accuracy, which is far better than the two compared\nbaseline methods, Convolutional Neural Networks and Recurrent Neural Networks.\nWe didn't apply our model to the third and fourth hierarchical level of the\nInternational Patent Classification - \"subclass\" and \"group\".The visualization\nof the Attention Mechanism shows that BERT-CNN obtains new state-of-the-art\nresults in representing vocabularies and semantics. This article demonstrates\nthe practicality and effectiveness of BERT-CNN in the field of automatic patent\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06258v3",
            "date": "2019-11-14 17:32:10+00:00",
            "title": "Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA",
            "authors": [
                "Ronghang Hu",
                "Amanpreet Singh",
                "Trevor Darrell",
                "Marcus Rohrbach"
            ],
            "abstract": "Many visual scenes contain text that carries crucial information, and it is\nthus essential to understand text in images for downstream reasoning tasks. For\nexample, a deep water label on a warning sign warns people about the danger in\nthe scene. Recent work has explored the TextVQA task that requires reading and\nunderstanding text in images to answer a question. However, existing approaches\nfor TextVQA are mostly based on custom pairwise fusion mechanisms between a\npair of two modalities and are restricted to a single prediction step by\ncasting TextVQA as a classification task. In this work, we propose a novel\nmodel for the TextVQA task based on a multimodal transformer architecture\naccompanied by a rich representation for text in images. Our model naturally\nfuses different modalities homogeneously by embedding them into a common\nsemantic space where self-attention is applied to model inter- and intra-\nmodality context. Furthermore, it enables iterative answer decoding with a\ndynamic pointer network, allowing the model to form an answer through\nmulti-step prediction instead of one-step classification. Our model outperforms\nexisting approaches on three benchmark datasets for the TextVQA task by a large\nmargin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.06352v1",
            "date": "2019-11-14 19:37:33+00:00",
            "title": "Question-Conditioned Counterfactual Image Generation for VQA",
            "authors": [
                "Jingjing Pan",
                "Yash Goyal",
                "Stefan Lee"
            ],
            "abstract": "While Visual Question Answering (VQA) models continue to push the\nstate-of-the-art forward, they largely remain black-boxes - failing to provide\ninsight into how or why an answer is generated. In this ongoing work, we\npropose addressing this shortcoming by learning to generate counterfactual\nimages for a VQA model - i.e. given a question-image pair, we wish to generate\na new image such that i) the VQA model outputs a different answer, ii) the new\nimage is minimally different from the original, and iii) the new image is\nrealistic. Our hope is that providing such counterfactual examples allows users\nto investigate and understand the VQA model's internal mechanisms.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.06415v1",
            "date": "2019-11-14 23:31:02+00:00",
            "title": "Sparse associative memory based on contextual code learning for disambiguating word senses",
            "authors": [
                "Max Raphael Sobroza",
                "Tales Marra",
                "Deok-Hee Kim-Dufor",
                "Claude Berrou"
            ],
            "abstract": "In recent literature, contextual pretrained Language Models (LMs)\ndemonstrated their potential in generalizing the knowledge to several Natural\nLanguage Processing (NLP) tasks including supervised Word Sense Disambiguation\n(WSD), a challenging problem in the field of Natural Language Understanding\n(NLU). However, word representations from these models are still very dense,\ncostly in terms of memory footprint, as well as minimally interpretable. In\norder to address such issues, we propose a new supervised biologically inspired\ntechnique for transferring large pre-trained language model representations\ninto a compressed representation, for the case of WSD. Our produced\nrepresentation contributes to increase the general interpretability of the\nframework and to decrease memory footprint, while enhancing performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06489v2",
            "date": "2019-11-15 06:31:13+00:00",
            "title": "Improving Distant Supervised Relation Extraction by Dynamic Neural Network",
            "authors": [
                "Yanjie Gou",
                "Yinjie Lei",
                "Lingqiao Liu",
                "Pingping Zhang",
                "Xi Peng"
            ],
            "abstract": "Distant Supervised Relation Extraction (DSRE) is usually formulated as a\nproblem of classifying a bag of sentences that contain two query entities, into\nthe predefined relation classes. Most existing methods consider those relation\nclasses as distinct semantic categories while ignoring their potential\nconnection to query entities. In this paper, we propose to leverage this\nconnection to improve the relation extraction accuracy. Our key ideas are\ntwofold: (1) For sentences belonging to the same relation class, the expression\nstyle, i.e. words choice, can vary according to the query entities. To account\nfor this style shift, the model should adjust its parameters in accordance with\nentity types. (2) Some relation classes are semantically similar, and the\nentity types appear in one relation may also appear in others. Therefore, it\ncan be trained cross different relation classes and further enhance those\nclasses with few samples, i.e., long-tail classes. To unify these two\narguments, we developed a novel Dynamic Neural Network for Relation Extraction\n(DNNRE). The network adopts a novel dynamic parameter generator that\ndynamically generates the network parameters according to the query entity\ntypes and relation classes. By using this mechanism, the network can\nsimultaneously handle the style shift problem and enhance the prediction\naccuracy for long-tail classes. Through our experimental study, we demonstrate\nthe effectiveness of the proposed method and show that it can achieve superior\nperformance over the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06815v1",
            "date": "2019-11-15 07:14:35+00:00",
            "title": "Experiments in Detecting Persuasion Techniques in the News",
            "authors": [
                "Seunghak Yu",
                "Giovanni Da San Martino",
                "Preslav Nakov"
            ],
            "abstract": "Many recent political events, like the 2016 US Presidential elections or the\n2018 Brazilian elections have raised the attention of institutions and of the\ngeneral public on the role of Internet and social media in influencing the\noutcome of these events. We argue that a safe democracy is one in which\ncitizens have tools to make them aware of propaganda campaigns. We propose a\nnovel task: performing fine-grained analysis of texts by detecting all\nfragments that contain propaganda techniques as well as their type. We further\ndesign a novel multi-granularity neural network, and we show that it\noutperforms several strong BERT-based baselines.",
            "journal ref": "NeurIPS-2019 workshop on AI for Social Good",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06848v1",
            "date": "2019-11-15 19:40:57+00:00",
            "title": "Assigning Medical Codes at the Encounter Level by Paying Attention to Documents",
            "authors": [
                "Han-Chin Shing",
                "Guoli Wang",
                "Philip Resnik"
            ],
            "abstract": "The vast majority of research in computer assisted medical coding focuses on\ncoding at the document level, but a substantial proportion of medical coding in\nthe real world involves coding at the level of clinical encounters, each of\nwhich is typically represented by a potentially large set of documents. We\nintroduce encounter-level document attention networks, which use hierarchical\nattention to explicitly take the hierarchical structure of encounter\ndocumentation into account. Experimental evaluation demonstrates improvements\nin coding accuracy as well as facilitation of human reviewers in their ability\nto identify which documents within an encounter play a role in determining the\nencounter level codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06919v1",
            "date": "2019-11-16 00:07:09+00:00",
            "title": "Improved Document Modelling with a Neural Discourse Parser",
            "authors": [
                "Fajri Koto",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "Despite the success of attention-based neural models for natural language\ngeneration and classification tasks, they are unable to capture the discourse\nstructure of larger documents. We hypothesize that explicit discourse\nrepresentations have utility for NLP tasks over longer documents or document\nsequences, which sequence-to-sequence models are unable to capture. For\nabstractive summarization, for instance, conventional neural models simply\nmatch source documents and the summary in a latent space without explicit\nrepresentation of text structure or relations. In this paper, we propose to use\nneural discourse representations obtained from a rhetorical structure theory\n(RST) parser to enhance document representations. Specifically, document\nrepresentations are generated for discourse spans, known as the elementary\ndiscourse units (EDUs). We empirically investigate the benefit of the proposed\napproach on two different tasks: abstractive summarization and popularity\nprediction of online petitions. We find that the proposed approach leads to\nimprovements in all cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06961v1",
            "date": "2019-11-16 05:25:01+00:00",
            "title": "Towards Automated Sexual Violence Report Tracking",
            "authors": [
                "Naeemul Hassan",
                "Amrit Poudel",
                "Jason Hale",
                "Claire Hubacek",
                "Khandakar Tasnim Huq",
                "Shubhra Kanti Karmaker Santu",
                "Syed Ishtiaque Ahmed"
            ],
            "abstract": "Tracking sexual violence is a challenging task. In this paper, we present a\nsupervised learning-based automated sexual violence report tracking model that\nis more scalable, and reliable than its crowdsource based counterparts. We\ndefine the sexual violence report tracking problem by considering victim,\nperpetrator contexts and the nature of the violence. We find that our model\ncould identify sexual violence reports with a precision and recall of 80.4% and\n83.4%, respectively. Moreover, we also applied the model during and after the\n\\#MeToo movement. Several interesting findings are discovered which are not\neasily identifiable from a shallow analysis.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.06964v1",
            "date": "2019-11-16 05:34:47+00:00",
            "title": "Learning Autocomplete Systems as a Communication Game",
            "authors": [
                "Mina Lee",
                "Tatsunori B. Hashimoto",
                "Percy Liang"
            ],
            "abstract": "We study textual autocomplete---the task of predicting a full sentence from a\npartial sentence---as a human-machine communication game. Specifically, we\nconsider three competing goals for effective communication: use as few tokens\nas possible (efficiency), transmit sentences faithfully (accuracy), and be\nlearnable to humans (interpretability). We propose an unsupervised approach\nwhich tackles all three desiderata by constraining the communication scheme to\nkeywords extracted from a source sentence for interpretability and optimizing\nthe efficiency-accuracy tradeoff. Our experiments show that this approach\nresults in an autocomplete system that is 52% more accurate at a given\nefficiency level compared to baselines, is robust to user variations, and saves\ntime by nearly 50% compared to typing full sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07056v1",
            "date": "2019-11-16 16:27:44+00:00",
            "title": "AttaCut: A Fast and Accurate Neural Thai Word Segmenter",
            "authors": [
                "Pattarawat Chormai",
                "Ponrawee Prasertsom",
                "Attapol Rutherford"
            ],
            "abstract": "Word segmentation is a fundamental pre-processing step for Thai Natural\nLanguage Processing. The current off-the-shelf solutions are not benchmarked\nconsistently, so it is difficult to compare their trade-offs. We conducted a\nspeed and accuracy comparison of the popular systems on three different domains\nand found that the state-of-the-art deep learning system is slow and moreover\ndoes not use sub-word structures to guide the model. Here, we propose a fast\nand accurate neural Thai Word Segmenter that uses dilated CNN filters to\ncapture the environment of each character and uses syllable embeddings as\nfeatures. Our system runs at least 5.6x faster and outperforms the previous\nstate-of-the-art system on some domains. In addition, we develop the first\nML-based Thai orthographical syllable segmenter, which yields syllable\nembeddings to be used as features by the word segmenter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07156v1",
            "date": "2019-11-17 05:40:17+00:00",
            "title": "Mining Unfollow Behavior in Large-Scale Online Social Networks via Spatial-Temporal Interaction",
            "authors": [
                "Haozhe Wu",
                "Zhiyuan Hu",
                "Jia Jia",
                "Yaohua Bu",
                "Xiangnan He",
                "Tat-Seng Chua"
            ],
            "abstract": "Online Social Networks (OSNs) evolve through two pervasive behaviors: follow\nand unfollow, which respectively signify relationship creation and relationship\ndissolution. Researches on social network evolution mainly focus on the follow\nbehavior, while the unfollow behavior has largely been ignored. Mining unfollow\nbehavior is challenging because user's decision on unfollow is not only\naffected by the simple combination of user's attributes like informativeness\nand reciprocity, but also affected by the complex interaction among them.\nMeanwhile, prior datasets seldom contain sufficient records for inferring such\ncomplex interaction. To address these issues, we first construct a large-scale\nreal-world Weibo dataset, which records detailed post content and relationship\ndynamics of 1.8 million Chinese users. Next, we define user's attributes as two\ncategories: spatial attributes (e.g., social role of user) and temporal\nattributes (e.g., post content of user). Leveraging the constructed dataset, we\nsystematically study how the interaction effects between user's spatial and\ntemporal attributes contribute to the unfollow behavior. Afterwards, we propose\na novel unified model with heterogeneous information (UMHI) for unfollow\nprediction. Specifically, our UMHI model: 1) captures user's spatial attributes\nthrough social network structure; 2) infers user's temporal attributes through\nuser-posted content and unfollow history; and 3) models the interaction between\nspatial and temporal attributes by the nonlinear MLP layers. Comprehensive\nevaluations on the constructed dataset demonstrate that the proposed UMHI model\noutperforms baseline methods by 16.44% on average in terms of precision. In\naddition, factor analyses verify that both spatial attributes and temporal\nattributes are essential for mining unfollow behavior.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.07251v1",
            "date": "2019-11-17 14:58:17+00:00",
            "title": "DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue",
            "authors": [
                "Xiaoze Jiang",
                "Jing Yu",
                "Zengchang Qin",
                "Yingying Zhuang",
                "Xingxing Zhang",
                "Yue Hu",
                "Qi Wu"
            ],
            "abstract": "Different from Visual Question Answering task that requires to answer only\none question about an image, Visual Dialogue involves multiple questions which\ncover a broad range of visual content that could be related to any objects,\nrelationships or semantics. The key challenge in Visual Dialogue task is thus\nto learn a more comprehensive and semantic-rich image representation which may\nhave adaptive attentions on the image for variant questions. In this research,\nwe propose a novel model to depict an image from both visual and semantic\nperspectives. Specifically, the visual view helps capture the appearance-level\ninformation, including objects and their relationships, while the semantic view\nenables the agent to understand high-level visual semantics from the whole\nimage to the local regions. Futhermore, on top of such multi-view image\nfeatures, we propose a feature selection framework which is able to adaptively\ncapture question-relevant information hierarchically in fine-grained level. The\nproposed method achieved state-of-the-art results on benchmark Visual Dialogue\ndatasets. More importantly, we can tell which modality (visual or semantic) has\nmore contribution in answering the current question by visualizing the gate\nvalues. It gives us insights in understanding of human cognition in Visual\nDialogue.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.07470v2",
            "date": "2019-11-18 07:45:19+00:00",
            "title": "Graph Transformer for Graph-to-Sequence Learning",
            "authors": [
                "Deng Cai",
                "Wai Lam"
            ],
            "abstract": "The dominant graph-to-sequence transduction models employ graph neural\nnetworks for graph representation learning, where the structural information is\nreflected by the receptive field of neurons. Unlike graph neural networks that\nrestrict the information exchange between immediate neighborhood, we propose a\nnew model, known as Graph Transformer, that uses explicit relation encoding and\nallows direct communication between two distant nodes. It provides a more\nefficient way for global graph structure modeling. Experiments on the\napplications of text generation from Abstract Meaning Representation (AMR) and\nsyntax-based neural machine translation show the superiority of our proposed\nmodel. Specifically, our model achieves 27.4 BLEU on LDC2015E86 and 29.7 BLEU\non LDC2017T10 for AMR-to-text generation, outperforming the state-of-the-art\nresults by up to 2.2 points. On the syntax-based translation tasks, our model\nestablishes new single-model state-of-the-art BLEU scores, 21.3 for\nEnglish-to-German and 14.1 for English-to-Czech, improving over the existing\nbest results, including ensembles, by over 1 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07588v1",
            "date": "2019-11-18 12:41:25+00:00",
            "title": "An Annotated Corpus of Reference Resolution for Interpreting Common Grounding",
            "authors": [
                "Takuma Udagawa",
                "Akiko Aizawa"
            ],
            "abstract": "Common grounding is the process of creating, repairing and updating mutual\nunderstandings, which is a fundamental aspect of natural language conversation.\nHowever, interpreting the process of common grounding is a challenging task,\nespecially under continuous and partially-observable context where complex\nambiguity, uncertainty, partial understandings and misunderstandings are\nintroduced. Interpretation becomes even more challenging when we deal with\ndialogue systems which still have limited capability of natural language\nunderstanding and generation. To address this problem, we consider reference\nresolution as the central subtask of common grounding and propose a new\nresource to study its intermediate process. Based on a simple and general\nannotation schema, we collected a total of 40,172 referring expressions in\n5,191 dialogues curated from an existing corpus, along with multiple judgements\nof referent interpretations. We show that our annotation is highly reliable,\ncaptures the complexity of common grounding through a natural degree of\nreasonable disagreements, and allows for more detailed and quantitative\nanalyses of common grounding strategies. Finally, we demonstrate the advantages\nof our annotation for interpreting, analyzing and improving common grounding in\nbaseline dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07613v1",
            "date": "2019-11-15 08:22:33+00:00",
            "title": "A Subword Level Language Model for Bangla Language",
            "authors": [
                "Aisha Khatun",
                "Anisur Rahman",
                "Hemayet Ahmed Chowdhury",
                "Md. Saiful Islam",
                "Ayesha Tasnim"
            ],
            "abstract": "Language models are at the core of natural language processing. The ability\nto represent natural language gives rise to its applications in numerous NLP\ntasks including text classification, summarization, and translation. Research\nin this area is very limited in Bangla due to the scarcity of resources, except\nfor some count-based models and very recent neural language models being\nproposed, which are all based on words and limited in practical tasks due to\ntheir high perplexity. This paper attempts to approach this issue of perplexity\nand proposes a subword level neural language model with the AWD-LSTM\narchitecture and various other techniques suitable for training in Bangla\nlanguage. The model is trained on a corpus of Bangla newspaper articles of an\nappreciable size consisting of more than 28.5 million word tokens. The\nperformance comparison with various other models depicts the significant\nreduction in perplexity the proposed model provides, reaching as low as 39.84,\nin just 20 epochs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07620v1",
            "date": "2019-11-15 03:16:12+00:00",
            "title": "Exploiting Token and Path-based Representations of Code for Identifying Security-Relevant Commits",
            "authors": [
                "Achyudh Ram",
                "Ji Xin",
                "Meiyappan Nagappan",
                "Yaoliang Yu",
                "Roc\u00edo Cabrera Lozoya",
                "Antonino Sabetta",
                "Jimmy Lin"
            ],
            "abstract": "Public vulnerability databases such as CVE and NVD account for only 60% of\nsecurity vulnerabilities present in open-source projects, and are known to\nsuffer from inconsistent quality. Over the last two years, there has been\nconsiderable growth in the number of known vulnerabilities across projects\navailable in various repositories such as NPM and Maven Central. Such an\nincreasing risk calls for a mechanism to infer the presence of security threats\nin a timely manner. We propose novel hierarchical deep learning models for the\nidentification of security-relevant commits from either the commit diff or the\nsource code for the Java classes. By comparing the performance of our model\nagainst code2vec, a state-of-the-art model that learns from path-based\nrepresentations of code, and a logistic regression baseline, we show that deep\nlearning models show promising results in identifying security-related commits.\nWe also conduct a comparative analysis of how various deep learning models\nlearn across different input representations and the effect of regularization\non the generalization of our models.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1911.08117v1",
            "date": "2019-11-19 06:50:59+00:00",
            "title": "A Hybrid Morpheme-Word Representation for Machine Translation of Morphologically Rich Languages",
            "authors": [
                "Minh-Thang Luong",
                "Preslav Nakov",
                "Min-Yen Kan"
            ],
            "abstract": "We propose a language-independent approach for improving statistical machine\ntranslation for morphologically rich languages using a hybrid morpheme-word\nrepresentation where the basic unit of translation is the morpheme, but word\nboundaries are respected at all stages of the translation process. Our model\nextends the classic phrase-based model by means of (1) word boundary-aware\nmorpheme-level phrase extraction, (2) minimum error-rate training for a\nmorpheme-level translation model using word-level BLEU, and (3) joint scoring\nwith morpheme- and word-level language models. Further improvements are\nachieved by combining our model with the classic one. The evaluation on English\nto Finnish using Europarl (714K sentence pairs; 15.5M English words) shows\nstatistically significant improvements over the classic model based on BLEU and\nhuman judgments.",
            "journal ref": "EMNLP-2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08370v1",
            "date": "2019-11-19 15:58:14+00:00",
            "title": "Event detection in Colombian security Twitter news using fine-grained latent topic analysis",
            "authors": [
                "Vladimir Vargas-Calder\u00f3n",
                "Nicol\u00e1s Parra-A.",
                "Jorge E. Camargo",
                "Herbert Vinck-Posada"
            ],
            "abstract": "Cultural and social dynamics are important concepts that must be understood\nin order to grasp what a community cares about. To that end, an excellent\nsource of information on what occurs in a community is the news, especially in\nrecent years, when mass media giants use social networks to communicate and\ninteract with their audience. In this work, we use a method to discover latent\ntopics in tweets from Colombian Twitter news accounts in order to identify the\nmost prominent events in the country. We pay particular attention to security,\nviolence and crime-related tweets because of the violent environment that\nsurrounds Colombian society. The latent topic discovery method that we use\nbuilds vector representations of the tweets by using FastText and finds\nclusters of tweets through the K-means clustering algorithm. The number of\nclusters is found by measuring the $C_V$ coherence for a range of number of\ntopics of the Latent Dirichlet Allocation (LDA) model. We finally use Uniform\nManifold Approximation and Projection (UMAP) for dimensionality reduction to\nvisualise the tweets vectors. Once the clusters related to security, violence\nand crime are identified, we proceed to apply the same method within each\ncluster to perform a fine-grained analysis in which specific events mentioned\nin the news are grouped together. Our method is able to discover event-specific\nsets of news, which is the baseline to perform an extensive analysis of how\npeople engage in Twitter threads on the different types of news, with an\nemphasis on security, violence and crime-related tweets.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.08373v1",
            "date": "2019-11-19 16:09:02+00:00",
            "title": "Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition",
            "authors": [
                "Jibin Wu",
                "Emre Yilmaz",
                "Malu Zhang",
                "Haizhou Li",
                "Kay Chen Tan"
            ],
            "abstract": "Artificial neural networks (ANN) have become the mainstream acoustic modeling\ntechnique for large vocabulary automatic speech recognition (ASR). A\nconventional ANN features a multi-layer architecture that requires massive\namounts of computation. The brain-inspired spiking neural networks (SNN)\nclosely mimic the biological neural networks and can operate on low-power\nneuromorphic hardware with spike-based computation. Motivated by their\nunprecedented energyefficiency and rapid information processing capability, we\nexplore the use of SNNs for speech recognition. In this work, we use SNNs for\nacoustic modeling and evaluate their performance on several large vocabulary\nrecognition scenarios. The experimental results demonstrate competitive ASR\naccuracies to their ANN counterparts, while require significantly reduced\ncomputational cost and inference time. Integrating the algorithmic power of\ndeep SNNs with energy-efficient neuromorphic hardware, therefore, offer an\nattractive solution for ASR applications running locally on mobile and embedded\ndevices.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1911.08827v1",
            "date": "2019-11-20 11:14:07+00:00",
            "title": "Zero-Shot Semantic Parsing for Instructions",
            "authors": [
                "Ofer Givoli",
                "Roi Reichart"
            ],
            "abstract": "We consider a zero-shot semantic parsing task: parsing instructions into\ncompositional logical forms, in domains that were not seen during training. We\npresent a new dataset with 1,390 examples from 7 application domains (e.g. a\ncalendar or a file manager), each example consisting of a triplet: (a) the\napplication's initial state, (b) an instruction, to be carried out in the\ncontext of that state, and (c) the state of the application after carrying out\nthe instruction. We introduce a new training algorithm that aims to train a\nsemantic parser on examples from a set of source domains, so that it can\neffectively parse instructions from an unknown target domain. We integrate our\nalgorithm into the floating parser of Pasupat and Liang (2015), and further\naugment the parser with features and a logical form candidate filtering logic,\nto support zero-shot adaptation. Our experiments with various zero-shot\nadaptation setups demonstrate substantial performance gains over a non-adapted\nparser.",
            "journal ref": "In Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics, pages 4454-4464 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08891v1",
            "date": "2019-11-20 13:26:43+00:00",
            "title": "Discovering New Intents via Constrained Deep Adaptive Clustering with Cluster Refinement",
            "authors": [
                "Ting-En Lin",
                "Hua Xu",
                "Hanlei Zhang"
            ],
            "abstract": "Identifying new user intents is an essential task in the dialogue system.\nHowever, it is hard to get satisfying clustering results since the definition\nof intents is strongly guided by prior knowledge. Existing methods incorporate\nprior knowledge by intensive feature engineering, which not only leads to\noverfitting but also makes it sensitive to the number of clusters. In this\npaper, we propose constrained deep adaptive clustering with cluster refinement\n(CDAC+), an end-to-end clustering method that can naturally incorporate\npairwise constraints as prior knowledge to guide the clustering process.\nMoreover, we refine the clusters by forcing the model to learn from the high\nconfidence assignments. After eliminating low confidence assignments, our\napproach is surprisingly insensitive to the number of clusters. Experimental\nresults on the three benchmark datasets show that our method can yield\nsignificant improvements over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08915v2",
            "date": "2019-11-18 21:39:19+00:00",
            "title": "Universal and non-universal text statistics: Clustering coefficient for language identification",
            "authors": [
                "Diego Espitia",
                "Hern\u00e1n Larralde"
            ],
            "abstract": "In this work we analyze statistical properties of 91 relatively small texts\nin 7 different languages (Spanish, English, French, German, Turkish, Russian,\nIcelandic) as well as texts with randomly inserted spaces. Despite the size\n(around 11260 different words), the well known universal statistical laws --\nnamely Zipf and Herdan-Heap's laws -- are confirmed, and are in close agreement\nwith results obtained elsewhere. We also construct a word co-occurrence network\nof each text. While the degree distribution is again universal, we note that\nthe distribution of Clustering Coefficients, which depend strongly on the local\nstructure of networks, can be used to differentiate between languages, as well\nas to distinguish natural languages from random texts.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1911.08946v1",
            "date": "2019-11-14 19:49:50+00:00",
            "title": "Understanding Troll Writing as a Linguistic Phenomenon",
            "authors": [
                "Sergei Monakhov"
            ],
            "abstract": "The current study yielded a number of important findings. We managed to build\na neural network that achieved an accuracy score of 91 per cent in classifying\ntroll and genuine tweets. By means of regression analysis, we identified a\nnumber of features that make a tweet more susceptible to correct labelling and\nfound that they are inherently present in troll tweets as a special type of\ndiscourse. We hypothesised that those features are grounded in the\nsociolinguistic limitations of troll writing, which can be best described as a\ncombination of two factors: speaking with a purpose and trying to mask the\npurpose of speaking. Next, we contended that the orthogonal nature of these\nfactors must necessarily result in the skewed distribution of many different\nlanguage parameters of troll messages. Having chosen as an example distribution\nof the topics and vocabulary associated with those topics, we showed some very\npronounced distributional anomalies, thus confirming our prediction.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.09273v1",
            "date": "2019-11-21 03:52:50+00:00",
            "title": "Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systems",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Zhaojiang Lin",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "Recently, data-driven task-oriented dialogue systems have achieved promising\nperformance in English. However, developing dialogue systems that support\nlow-resource languages remains a long-standing challenge due to the absence of\nhigh-quality data. In order to circumvent the expensive and time-consuming data\ncollection, we introduce Attention-Informed Mixed-Language Training (MLT), a\nnovel zero-shot adaptation method for cross-lingual task-oriented dialogue\nsystems. It leverages very few task-related parallel word pairs to generate\ncode-switching sentences for learning the inter-lingual semantics across\nlanguages. Instead of manually selecting the word pairs, we propose to extract\nsource words based on the scores computed by the attention layer of a trained\nEnglish task-related model and then generate word pairs using existing\nbilingual dictionaries. Furthermore, intensive experiments with different\ncross-lingual embeddings demonstrate the effectiveness of our approach.\nFinally, with very few word pairs, our model achieves significant zero-shot\nadaptation performance improvements in both cross-lingual dialogue state\ntracking and natural language understanding (i.e., intent detection and slot\nfilling) tasks compared to the current state-of-the-art approaches, which\nutilize a much larger amount of bilingual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09304v1",
            "date": "2019-11-21 06:14:05+00:00",
            "title": "Automatic Text-based Personality Recognition on Monologues and Multiparty Dialogues Using Attentive Networks and Contextual Embeddings",
            "authors": [
                "Hang Jiang",
                "Xianzhe Zhang",
                "Jinho D. Choi"
            ],
            "abstract": "Previous works related to automatic personality recognition focus on using\ntraditional classification models with linguistic features. However, attentive\nneural networks with contextual embeddings, which have achieved huge success in\ntext classification, are rarely explored for this task. In this project, we\nhave two major contributions. First, we create the first dialogue-based\npersonality dataset, FriendsPersona, by annotating 5 personality traits of\nspeakers from Friends TV Show through crowdsourcing. Second, we present a novel\napproach to automatic personality recognition using pre-trained contextual\nembeddings (BERT and RoBERTa) and attentive neural networks. Our models largely\nimprove the state-of-art results on the monologue Essays dataset by 2.49%, and\nestablish a solid benchmark on our FriendsPersona. By comparing results in two\ndatasets, we demonstrate the challenges of modeling personality in multi-party\ndialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09320v1",
            "date": "2019-11-21 07:26:05+00:00",
            "title": "Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Chenze Shao",
                "Jinchao Zhang",
                "Yang Feng",
                "Fandong Meng",
                "Jie Zhou"
            ],
            "abstract": "Non-Autoregressive Neural Machine Translation (NAT) achieves significant\ndecoding speedup through generating target words independently and\nsimultaneously. However, in the context of non-autoregressive translation, the\nword-level cross-entropy loss cannot model the target-side sequential\ndependency properly, leading to its weak correlation with the translation\nquality. As a result, NAT tends to generate influent translations with\nover-translation and under-translation errors. In this paper, we propose to\ntrain NAT to minimize the Bag-of-Ngrams (BoN) difference between the model\noutput and the reference sentence. The bag-of-ngrams training objective is\ndifferentiable and can be efficiently calculated, which encourages NAT to\ncapture the target-side sequential dependency and correlates well with the\ntranslation quality. We validate our approach on three translation tasks and\nshow that our approach largely outperforms the NAT baseline by about 5.0 BLEU\nscores on WMT14 En$\\leftrightarrow$De and about 2.5 BLEU scores on WMT16\nEn$\\leftrightarrow$Ro.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09334v1",
            "date": "2019-11-21 08:22:37+00:00",
            "title": "Incorporating Textual Evidence in Visual Storytelling",
            "authors": [
                "Tianyi Li",
                "Sujian Li"
            ],
            "abstract": "Previous work on visual storytelling mainly focused on exploring image\nsequence as evidence for storytelling and neglected textual evidence for\nguiding story generation. Motivated by human storytelling process which recalls\nstories for familiar images, we exploit textual evidence from similar images to\nhelp generate coherent and meaningful stories. To pick the images which may\nprovide textual experience, we propose a two-step ranking method based on image\nobject recognition techniques. To utilize textual information, we design an\nextended Seq2Seq model with two-channel encoder and attention. Experiments on\nthe VIST dataset show that our method outperforms state-of-the-art baseline\nmodels without heavy engineering.",
            "journal ref": "In Proceeding of DSNNLG 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09478v1",
            "date": "2019-11-21 14:09:33+00:00",
            "title": "What Do You Mean `Why?': Resolving Sluices in Conversations",
            "authors": [
                "Victor Petr\u00e9n Bach Hansen",
                "Anders S\u00f8gaard"
            ],
            "abstract": "In conversation, we often ask one-word questions such as `Why?' or `Who?'.\nSuch questions are typically easy for humans to answer, but can be hard for\ncomputers, because their resolution requires retrieving both the right semantic\nframes and the right arguments from context. This paper introduces the novel\nellipsis resolution task of resolving such one-word questions, referred to as\nsluices in linguistics. We present a crowd-sourced dataset containing\nannotations of sluices from over 4,000 dialogues collected from conversational\nQA datasets, as well as a series of strong baseline architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09483v1",
            "date": "2019-11-17 09:36:07+00:00",
            "title": "MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning",
            "authors": [
                "Guangxiang Zhao",
                "Xu Sun",
                "Jingjing Xu",
                "Zhiyuan Zhang",
                "Liangchen Luo"
            ],
            "abstract": "In sequence to sequence learning, the self-attention mechanism proves to be\nhighly effective, and achieves significant improvements in many tasks. However,\nthe self-attention mechanism is not without its own flaws. Although\nself-attention can model extremely long dependencies, the attention in deep\nlayers tends to overconcentrate on a single token, leading to insufficient use\nof local information and difficultly in representing long sequences. In this\nwork, we explore parallel multi-scale representation learning on sequence data,\nstriving to capture both long-range and short-range language structures. To\nthis end, we propose the Parallel MUlti-Scale attEntion (MUSE) and MUSE-simple.\nMUSE-simple contains the basic idea of parallel multi-scale sequence\nrepresentation learning, and it encodes the sequence in parallel, in terms of\ndifferent scales with the help from self-attention, and pointwise\ntransformation. MUSE builds on MUSE-simple and explores combining convolution\nand self-attention for learning sequence representations from more different\nscales. We focus on machine translation and the proposed approach achieves\nsubstantial performance improvements over Transformer, especially on long\nsequences. More importantly, we find that although conceptually simple, its\nsuccess in practice requires intricate considerations, and the multi-scale\nattention must build on unified semantic space. Under common setting, the\nproposed model achieves substantial performance and outperforms all previous\nmodels on three main machine translation tasks. In addition, MUSE has potential\nfor accelerating inference due to its parallelism. Code will be available at\nhttps://github.com/lancopku/MUSE",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09661v1",
            "date": "2019-11-21 18:45:54+00:00",
            "title": "Paraphrasing with Large Language Models",
            "authors": [
                "Sam Witteveen",
                "Martin Andrews"
            ],
            "abstract": "Recently, large language models such as GPT-2 have shown themselves to be\nextremely adept at text generation and have also been able to achieve\nhigh-quality results in many downstream NLP tasks such as text classification,\nsentiment analysis and question answering with the aid of fine-tuning. We\npresent a useful technique for using a large language model to perform the task\nof paraphrasing on a variety of texts and subjects. Our approach is\ndemonstrated to be capable of generating paraphrases not only at a sentence\nlevel but also for longer spans of text such as paragraphs without needing to\nbreak the text into smaller chunks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09709v3",
            "date": "2019-11-21 19:15:03+00:00",
            "title": "Automatically Neutralizing Subjective Bias in Text",
            "authors": [
                "Reid Pryzant",
                "Richard Diehl Martinez",
                "Nathan Dass",
                "Sadao Kurohashi",
                "Dan Jurafsky",
                "Diyi Yang"
            ],
            "abstract": "Texts like news, encyclopedias, and some social media strive for objectivity.\nYet bias in the form of inappropriate subjectivity - introducing attitudes via\nframing, presupposing truth, and casting doubt - remains ubiquitous. This kind\nof bias erodes our collective trust and fuels social conflict. To address this\nissue, we introduce a novel testbed for natural language generation:\nautomatically bringing inappropriately subjective text into a neutral point of\nview (\"neutralizing\" biased text). We also offer the first parallel corpus of\nbiased language. The corpus contains 180,000 sentence pairs and originates from\nWikipedia edits that removed various framings, presuppositions, and attitudes\nfrom biased sentences. Last, we propose two strong encoder-decoder baselines\nfor the task. A straightforward yet opaque CONCURRENT system uses a BERT\nencoder to identify subjective words as part of the generation process. An\ninterpretable and controllable MODULAR algorithm separates these steps, using\n(1) a BERT-based classifier to identify problematic words and (2) a novel join\nembedding through which the classifier can edit the hidden states of the\nencoder. Large-scale human evaluation across four domains (encyclopedias, news\nheadlines, books, and political speeches) suggests that these algorithms are a\nfirst step towards the automatic identification and reduction of bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09728v1",
            "date": "2019-11-21 20:01:46+00:00",
            "title": "Improving Conditioning in Context-Aware Sequence to Sequence Models",
            "authors": [
                "Xinyi Wang",
                "Jason Weston",
                "Michael Auli",
                "Yacine Jernite"
            ],
            "abstract": "Neural sequence to sequence models are well established for applications\nwhich can be cast as mapping a single input sequence into a single output\nsequence. In this work, we focus on cases where generation is conditioned on\nboth a short query and a long context, such as abstractive question answering\nor document-level translation. We modify the standard sequence-to-sequence\napproach to make better use of both the query and the context by expanding the\nconditioning mechanism to intertwine query and context attention. We also\nintroduce a simple and efficient data augmentation method for the proposed\nmodel. Experiments on three different tasks show that both changes lead to\nconsistent improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09753v1",
            "date": "2019-11-21 21:26:28+00:00",
            "title": "Reinforcing an Image Caption Generator Using Off-Line Human Feedback",
            "authors": [
                "Paul Hongsuck Seo",
                "Piyush Sharma",
                "Tomer Levinboim",
                "Bohyung Han",
                "Radu Soricut"
            ],
            "abstract": "Human ratings are currently the most accurate way to assess the quality of an\nimage captioning model, yet most often the only used outcome of an expensive\nhuman rating evaluation is a few overall statistics over the evaluation\ndataset. In this paper, we show that the signal from instance-level human\ncaption ratings can be leveraged to improve captioning models, even when the\namount of caption ratings is several orders of magnitude less than the caption\ntraining data. We employ a policy gradient method to maximize the human ratings\nas rewards in an off-policy reinforcement learning setting, where policy\ngradients are estimated by samples from a distribution that focuses on the\ncaptions in a caption ratings dataset. Our empirical evidence indicates that\nthe proposed method learns to generalize the human raters' judgments to a\npreviously unseen set of images, as judged by a different set of human judges,\nand additionally on a different, multi-dimensional side-by-side human\nevaluation procedure.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.09788v1",
            "date": "2019-11-22 00:00:31+00:00",
            "title": "Are Noisy Sentences Useless for Distant Supervised Relation Extraction?",
            "authors": [
                "Yuming Shang"
            ],
            "abstract": "The noisy labeling problem has been one of the major obstacles for distant\nsupervised relation extraction. Existing approaches usually consider that the\nnoisy sentences are useless and will harm the model's performance. Therefore,\nthey mainly alleviate this problem by reducing the influence of noisy\nsentences, such as applying bag-level selective attention or removing noisy\nsentences from sentence-bags. However, the underlying cause of the noisy\nlabeling problem is not the lack of useful information, but the missing\nrelation labels. Intuitively, if we can allocate credible labels for noisy\nsentences, they will be transformed into useful training data and benefit the\nmodel's performance. Thus, in this paper, we propose a novel method for distant\nsupervised relation extraction, which employs unsupervised deep clustering to\ngenerate reliable labels for noisy sentences. Specifically, our model contains\nthree modules: a sentence encoder, a noise detector and a label generator. The\nsentence encoder is used to obtain feature representations. The noise detector\ndetects noisy sentences from sentence-bags, and the label generator produces\nhigh-confidence relation labels for noisy sentences. Extensive experimental\nresults demonstrate that our model outperforms the state-of-the-art baselines\non a popular benchmark dataset, and can indeed alleviate the noisy labeling\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09812v1",
            "date": "2019-11-22 02:09:08+00:00",
            "title": "Zero-Resource Cross-Lingual Named Entity Recognition",
            "authors": [
                "M Saiful Bari",
                "Shafiq Joty",
                "Prathyusha Jwalapuram"
            ],
            "abstract": "Recently, neural methods have achieved state-of-the-art (SOTA) results in\nNamed Entity Recognition (NER) tasks for many languages without the need for\nmanually crafted features. However, these models still require manually\nannotated training data, which is not available for many languages. In this\npaper, we propose an unsupervised cross-lingual NER model that can transfer NER\nknowledge from one language to another in a completely unsupervised way without\nrelying on any bilingual dictionary or parallel data. Our model achieves this\nthrough word-level adversarial learning and augmented fine-tuning with\nparameter sharing and feature augmentation. Experiments on five different\nlanguages demonstrate the effectiveness of our approach, outperforming existing\nmodels by a good margin and setting a new SOTA for each language pair.",
            "journal ref": "Proceedings of the 34th AAAI Conference on Artificial Intelligence\n  (AAAI-2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09877v1",
            "date": "2019-11-22 06:38:42+00:00",
            "title": "Neuron Interaction Based Representation Composition for Neural Machine Translation",
            "authors": [
                "Jian Li",
                "Xing Wang",
                "Baosong Yang",
                "Shuming Shi",
                "Michael R. Lyu",
                "Zhaopeng Tu"
            ],
            "abstract": "Recent NLP studies reveal that substantial linguistic information can be\nattributed to single neurons, i.e., individual dimensions of the representation\nvectors. We hypothesize that modeling strong interactions among neurons helps\nto better capture complex information by composing the linguistic properties\nembedded in individual neurons. Starting from this intuition, we propose a\nnovel approach to compose representations learned by different components in\nneural machine translation (e.g., multi-layer networks or multi-head\nattention), based on modeling strong interactions among neurons in the\nrepresentation vectors. Specifically, we leverage bilinear pooling to model\npairwise multiplicative interactions among individual neurons, and a low-rank\napproximation to make the model computationally feasible. We further propose\nextended bilinear pooling to incorporate first-order representations.\nExperiments on WMT14 English-German and English-French translation tasks show\nthat our model consistently improves performances over the SOTA Transformer\nbaseline. Further analyses demonstrate that our approach indeed captures more\nsyntactic and semantic information as expected.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09886v1",
            "date": "2019-11-22 06:52:21+00:00",
            "title": "Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction",
            "authors": [
                "Tapas Nayak",
                "Hwee Tou Ng"
            ],
            "abstract": "A relation tuple consists of two entities and the relation between them, and\noften such tuples are found in unstructured text. There may be multiple\nrelation tuples present in a text and they may share one or both entities among\nthem. Extracting such relation tuples from a sentence is a difficult task and\nsharing of entities or overlapping entities among the tuples makes it more\nchallenging. Most prior work adopted a pipeline approach where entities were\nidentified first followed by finding the relations among them, thus missing the\ninteraction among the relation tuples in a sentence. In this paper, we propose\ntwo approaches to use encoder-decoder architecture for jointly extracting\nentities and relations. In the first approach, we propose a representation\nscheme for relation tuples which enables the decoder to generate one word at a\ntime like machine translation models and still finds all the tuples present in\na sentence with full entity names of different length and with overlapping\nentities. Next, we propose a pointer network-based decoding approach where an\nentire tuple is generated at every time step. Experiments on the publicly\navailable New York Times corpus show that our proposed approaches outperform\nprevious work and achieve significantly higher F1 scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09994v1",
            "date": "2019-11-22 12:20:44+00:00",
            "title": "Anaphora Resolution in Dialogue Systems for South Asian Languages",
            "authors": [
                "Vinay Annam",
                "Nikhil Koditala",
                "Radhika Mamidi"
            ],
            "abstract": "Anaphora resolution is a challenging task which has been the interest of NLP\nresearchers for a long time. Traditional resolution techniques like eliminative\nconstraints and weighted preferences were successful in many languages.\nHowever, they are ineffective in free word order languages like most SouthAsian\nlanguages.Heuristic and rule-based techniques were typical in these languages,\nwhich are constrained to context and domain.In this paper, we venture a new\nstrategy us-ing neural networks for resolving anaphora in human-human\ndialogues. The architecture chiefly consists of three components, a shallow\nparser for extracting features, a feature vector generator which produces the\nword embed-dings, and a neural network model which will predict the antecedent\nmention of an anaphora.The system has been trained and tested on Telugu\nconversation corpus we generated. Given the advantage of the semantic\ninformation in word embeddings and appending actor, gender, number, person and\npart of plural features the model has reached an F1-score of 86.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10049v2",
            "date": "2019-11-22 14:06:21+00:00",
            "title": "High Quality ELMo Embeddings for Seven Less-Resourced Languages",
            "authors": [
                "Matej Ul\u010dar",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "Recent results show that deep neural networks using contextual embeddings\nsignificantly outperform non-contextual embeddings on a majority of text\nclassification task. We offer precomputed embeddings from popular contextual\nELMo model for seven languages: Croatian, Estonian, Finnish, Latvian,\nLithuanian, Slovenian, and Swedish. We demonstrate that the quality of\nembeddings strongly depends on the size of training set and show that existing\npublicly available ELMo embeddings for listed languages shall be improved. We\ntrain new ELMo embeddings on much larger training sets and show their advantage\nover baseline non-contextual FastText embeddings. In evaluation, we use two\nbenchmarks, the analogy task and the NER task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10082v2",
            "date": "2019-11-22 15:22:34+00:00",
            "title": "Injecting Prior Knowledge into Image Caption Generation",
            "authors": [
                "Arushi Goel",
                "Basura Fernando",
                "Thanh-Son Nguyen",
                "Hakan Bilen"
            ],
            "abstract": "Automatically generating natural language descriptions from an image is a\nchallenging problem in artificial intelligence that requires a good\nunderstanding of the visual and textual signals and the correlations between\nthem. The state-of-the-art methods in image captioning struggles to approach\nhuman level performance, especially when data is limited. In this paper, we\npropose to improve the performance of the state-of-the-art image captioning\nmodels by incorporating two sources of prior knowledge: (i) a conditional\nlatent topic attention, that uses a set of latent variables (topics) as an\nanchor to generate highly probable words and, (ii) a regularization technique\nthat exploits the inductive biases in syntactic and semantic structure of\ncaptions and improves the generalization of image captioning models. Our\nexperiments validate that our method produces more human interpretable captions\nand also leads to significant improvements on the MSCOCO dataset in both the\nfull and low data regimes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10130v1",
            "date": "2019-11-22 16:35:37+00:00",
            "title": "A Data Set of Internet Claims and Comparison of their Sentiments with Credibility",
            "authors": [
                "Amey Parundekar",
                "Susan Elias",
                "Ashwin Ashok"
            ],
            "abstract": "In this modern era, communication has become faster and easier. This means\nfallacious information can spread as fast as reality. Considering the damage\nthat fake news kindles on the psychology of people and the fact that such news\nproliferates faster than truth, we need to study the phenomenon that helps\nspread fake news. An unbiased data set that depends on reality for rating news\nis necessary to construct predictive models for its classification. This paper\ndescribes the methodology to create such a data set. We collect our data from\nsnopes.com which is a fact-checking organization. Furthermore, we intend to\ncreate this data set not only for classification of the news but also to find\npatterns that reason the intent behind misinformation. We also formally define\nan Internet Claim, its credibility, and the sentiment behind such a claim. We\ntry to realize the relationship between the sentiment of a claim with its\ncredibility. This relationship pours light on the bigger picture behind the\npropagation of misinformation. We pave the way for further research based on\nthe methodology described in this paper to create the data set and usage of\npredictive modeling along with research-based on psychology/mentality of people\nto understand why fake news spreads much faster than reality.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.10154v1",
            "date": "2019-11-22 17:25:32+00:00",
            "title": "Moral Dilemmas for Artificial Intelligence: a position paper on an application of Compositional Quantum Cognition",
            "authors": [
                "Camilo M. Signorelli",
                "Xerxes D. Arsiwalla"
            ],
            "abstract": "Traditionally, the way one evaluates the performance of an Artificial\nIntelligence (AI) system is via a comparison to human performance in specific\ntasks, treating humans as a reference for high-level cognition. However, these\ncomparisons leave out important features of human intelligence: the capability\nto transfer knowledge and make complex decisions based on emotional and\nrational reasoning. These decisions are influenced by current inferences as\nwell as prior experiences, making the decision process strongly subjective and\napparently biased. In this context, a definition of compositional intelligence\nis necessary to incorporate these features in future AI tests. Here, a concrete\nimplementation of this will be suggested, using recent developments in quantum\ncognition, natural language and compositional meaning of sentences, thanks to\ncategorical compositional models of meaning.",
            "journal ref": "Quantum Interaction. QI 2018. Lecture Notes in Computer Science,\n  vol 11690",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.10183v3",
            "date": "2019-11-22 18:31:53+00:00",
            "title": "Interactive Text Ranking with Bayesian Optimisation: A Case Study on Community QA and Summarisation",
            "authors": [
                "Edwin Simpson",
                "Yang Gao",
                "Iryna Gurevych"
            ],
            "abstract": "For many NLP applications, such as question answering and summarisation, the\ngoal is to select the best solution from a large space of candidates to meet a\nparticular user's needs. To address the lack of user-specific training data, we\npropose an interactive text ranking approach that actively selects pairs of\ncandidates, from which the user selects the best. Unlike previous strategies,\nwhich attempt to learn a ranking across the whole candidate space, our method\nemploys Bayesian optimisation to focus the user's labelling effort on high\nquality candidates and integrates prior knowledge in a Bayesian manner to cope\nbetter with small data scenarios. We apply our method to community question\nanswering (cQA) and extractive summarisation, finding that it significantly\noutperforms existing interactive approaches. We also show that the ranking\nfunction learned by our method is an effective reward function for\nreinforcement learning, which improves the state of the art for interactive\nsummarisation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10235v1",
            "date": "2019-11-22 20:11:40+00:00",
            "title": "Improving N-gram Language Models with Pre-trained Deep Transformer",
            "authors": [
                "Yiren Wang",
                "Hongzhao Huang",
                "Zhe Liu",
                "Yutong Pang",
                "Yongqiang Wang",
                "ChengXiang Zhai",
                "Fuchun Peng"
            ],
            "abstract": "Although n-gram language models (LMs) have been outperformed by the\nstate-of-the-art neural LMs, they are still widely used in speech recognition\ndue to its high efficiency in inference. In this paper, we demonstrate that\nn-gram LM can be improved by neural LMs through a text generation based data\naugmentation method. In contrast to previous approaches, we employ a\nlarge-scale general domain pre-training followed by in-domain fine-tuning\nstrategy to construct deep Transformer based neural LMs. Large amount of\nin-domain text data is generated with the well trained deep Transformer to\nconstruct new n-gram LMs, which are then interpolated with baseline n-gram\nsystems. Empirical studies on different speech recognition tasks show that the\nproposed approach can effectively improve recognition accuracy. In particular,\nour proposed approach brings significant relative word error rate reduction up\nto 6.0% for domains with limited in-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10438v2",
            "date": "2019-11-24 00:24:32+00:00",
            "title": "CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning",
            "authors": [
                "Daojian Zeng",
                "Ranran Haoran Zhang",
                "Qianying Liu"
            ],
            "abstract": "Joint extraction of entities and relations has received significant attention\ndue to its potential of providing higher performance for both tasks. Among\nexisting methods, CopyRE is effective and novel, which uses a\nsequence-to-sequence framework and copy mechanism to directly generate the\nrelation triplets. However, it suffers from two fatal problems. The model is\nextremely weak at differing the head and tail entity, resulting in inaccurate\nentity extraction. It also cannot predict multi-token entities (e.g.\n\\textit{Steven Jobs}). To address these problems, we give a detailed analysis\nof the reasons behind the inaccurate entity extraction problem, and then\npropose a simple but extremely effective model structure to solve this problem.\nIn addition, we propose a multi-task learning framework equipped with copy\nmechanism, called CopyMTL, to allow the model to predict multi-token entities.\nExperiments reveal the problems of CopyRE and show that our model achieves\nsignificant improvement over the current state-of-the-art method by 9% in NYT\nand 16% in WebNLG (F1 score). Our code is available at\nhttps://github.com/WindChimeRan/CopyMTL",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10484v2",
            "date": "2019-11-24 09:32:55+00:00",
            "title": "Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context",
            "authors": [
                "Yichi Zhang",
                "Zhijian Ou",
                "Zhou Yu"
            ],
            "abstract": "Conversations have an intrinsic one-to-many property, which means that\nmultiple responses can be appropriate for the same dialog context. In\ntask-oriented dialogs, this property leads to different valid dialog policies\ntowards task completion. However, none of the existing task-oriented dialog\ngeneration approaches takes this property into account. We propose a\nMulti-Action Data Augmentation (MADA) framework to utilize the one-to-many\nproperty to generate diverse appropriate dialog responses. Specifically, we\nfirst use dialog states to summarize the dialog history, and then discover all\npossible mappings from every dialog state to its different valid system\nactions. During dialog system training, we enable the current dialog state to\nmap to all valid system actions discovered in the previous process to create\nadditional state-action pairs. By incorporating these additional pairs, the\ndialog policy learns a balanced action distribution, which further guides the\ndialog model to generate diverse responses. Experimental results show that the\nproposed framework consistently improves dialog policy diversity, and results\nin improved response diversity and appropriateness. Our model obtains\nstate-of-the-art results on MultiWOZ.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10496v2",
            "date": "2019-11-24 10:35:35+00:00",
            "title": "Two Causal Principles for Improving Visual Dialog",
            "authors": [
                "Jiaxin Qi",
                "Yulei Niu",
                "Jianqiang Huang",
                "Hanwang Zhang"
            ],
            "abstract": "This paper unravels the design tricks adopted by us, the champion team\nMReaL-BDAI, for Visual Dialog Challenge 2019: two causal principles for\nimproving Visual Dialog (VisDial). By \"improving\", we mean that they can\npromote almost every existing VisDial model to the state-of-the-art performance\non the leader-board. Such a major improvement is only due to our careful\ninspection on the causality behind the model and data, finding that the\ncommunity has overlooked two causalities in VisDial. Intuitively, Principle 1\nsuggests: we should remove the direct input of the dialog history to the answer\nmodel, otherwise a harmful shortcut bias will be introduced; Principle 2 says:\nthere is an unobserved confounder for history, question, and answer, leading to\nspurious correlations from training data. In particular, to remove the\nconfounder suggested in Principle 2, we propose several causal intervention\nalgorithms, which make the training fundamentally different from the\ntraditional likelihood estimation. Note that the two principles are\nmodel-agnostic, so they are applicable in any VisDial model. The code is\navailable at https://github.com/simpleshinobu/visdial-principles.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.10524v1",
            "date": "2019-11-24 13:06:19+00:00",
            "title": "Causally Denoise Word Embeddings Using Half-Sibling Regression",
            "authors": [
                "Zekun Yang",
                "Tianlin Liu"
            ],
            "abstract": "Distributional representations of words, also known as word vectors, have\nbecome crucial for modern natural language processing tasks due to their wide\napplications. Recently, a growing body of word vector postprocessing algorithm\nhas emerged, aiming to render off-the-shelf word vectors even stronger. In line\nwith these investigations, we introduce a novel word vector postprocessing\nscheme under a causal inference framework. Concretely, the postprocessing\npipeline is realized by Half-Sibling Regression (HSR), which allows us to\nidentify and remove confounding noise contained in word vectors. Compared to\nprevious work, our proposed method has the advantages of interpretability and\ntransparency due to its causal inference grounding. Evaluated on a battery of\nstandard lexical-level evaluation tasks and downstream sentiment analysis\ntasks, our method reaches state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10677v1",
            "date": "2019-11-25 03:08:42+00:00",
            "title": "Non-autoregressive Transformer by Position Learning",
            "authors": [
                "Yu Bao",
                "Hao Zhou",
                "Jiangtao Feng",
                "Mingxuan Wang",
                "Shujian Huang",
                "Jiajun Chen",
                "Lei LI"
            ],
            "abstract": "Non-autoregressive models are promising on various text generation tasks.\nPrevious work hardly considers to explicitly model the positions of generated\nwords. However, position modeling is an essential problem in non-autoregressive\ntext generation. In this study, we propose PNAT, which incorporates positions\nas a latent variable into the text generative process. Experimental results\nshow that PNAT achieves top results on machine translation and paraphrase\ngeneration tasks, outperforming several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10776v1",
            "date": "2019-11-25 09:21:17+00:00",
            "title": "Filling Conversation Ellipsis for Better Social Dialog Understanding",
            "authors": [
                "Xiyuan Zhang",
                "Chengxi Li",
                "Dian Yu",
                "Samuel Davidson",
                "Zhou Yu"
            ],
            "abstract": "The phenomenon of ellipsis is prevalent in social conversations. Ellipsis\nincreases the difficulty of a series of downstream language understanding\ntasks, such as dialog act prediction and semantic role labeling. We propose to\nresolve ellipsis through automatic sentence completion to improve language\nunderstanding. However, automatic ellipsis completion can result in output\nwhich does not accurately reflect user intent. To address this issue, we\npropose a method which considers both the original utterance that has ellipsis\nand the automatically completed utterance in dialog act and semantic role\nlabeling tasks. Specifically, we first complete user utterances to resolve\nellipsis using an end-to-end pointer network model. We then train a prediction\nmodel using both utterances containing ellipsis and our automatically completed\nutterances. Finally, we combine the prediction results from these two\nutterances using a selection model that is guided by expert knowledge. Our\napproach improves dialog act prediction and semantic role labeling by 1.3% and\n2.5% in F1 score respectively in social conversations. We also present an\nopen-domain human-machine conversation dataset with manually completed user\nutterances and annotated semantic role labeling after manual completion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10787v1",
            "date": "2019-11-25 09:47:11+00:00",
            "title": "A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations",
            "authors": [
                "Zekun Yang",
                "Juan Feng"
            ],
            "abstract": "Word embedding has become essential for natural language processing as it\nboosts empirical performances of various tasks. However, recent research\ndiscovers that gender bias is incorporated in neural word embeddings, and\ndownstream tasks that rely on these biased word vectors also produce\ngender-biased results. While some word-embedding gender-debiasing methods have\nbeen developed, these methods mainly focus on reducing gender bias associated\nwith gender direction and fail to reduce the gender bias presented in word\nembedding relations. In this paper, we design a causal and simple approach for\nmitigating gender bias in word vector relation by utilizing the statistical\ndependency between gender-definition word embeddings and gender-biased word\nembeddings. Our method attains state-of-the-art results on gender-debiasing\ntasks, lexical- and sentence-level evaluation tasks, and downstream coreference\nresolution tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10882v1",
            "date": "2019-11-25 12:54:58+00:00",
            "title": "SWift -- A SignWriting improved fast transcriber",
            "authors": [
                "Claudia S. Bianchini",
                "Fabrizio Borgia",
                "Paolo Bottoni",
                "Maria de Marsico"
            ],
            "abstract": "We present SWift (SignWriting improved fast transcriber), an advanced editor\nfor computer-aided writing and transcribing using SignWriting (SW). SW is\ndevised to allow deaf people and linguists alike to exploit an easy-to-grasp\nwritten form of (any) sign language. Similarly, SWift has been developed for\neveryone who masters SW, and is not exclusively deaf-oriented. Using SWift, it\nis possible to compose and save any sign, using elementary components called\nglyphs. A guided procedure facilitates the composition process. SWift is aimed\nat helping to break down the \"electronic\" barriers that keep the deaf community\naway from Information and Communication Technology (ICT). The editor has been\ndeveloped modularly and can be integrated everywhere the use of SW, as an\nalternative to written vocal language, may be advisable.",
            "journal ref": "Proceedings of the International Working Conference on Advanced\n  Visual Interfaces, ACM - Association for Computer Machinery, pp.390 - 393,\n  2012, ACM AVI2012, 978-1-4503-1287-5",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10953v1",
            "date": "2019-11-25 14:55:11+00:00",
            "title": "FLATM: A Fuzzy Logic Approach Topic Model for Medical Documents",
            "authors": [
                "Amir Karami",
                "Aryya Gangopadhyay",
                "Bin Zhou",
                "Hadi Kharrazi"
            ],
            "abstract": "One of the challenges for text analysis in medical domains is analyzing\nlarge-scale medical documents. As a consequence, finding relevant documents has\nbecome more difficult. One of the popular methods to retrieve information based\non discovering the themes in the documents is topic modeling. The themes in the\ndocuments help to retrieve documents on the same topic with and without a\nquery. In this paper, we present a novel approach to topic modeling using fuzzy\nclustering. To evaluate our model, we experiment with two text datasets of\nmedical documents. The evaluation metrics carried out through document\nclassification and document modeling show that our model produces better\nperformance than LDA, indicating that fuzzy set theory can improve the\nperformance of topic models in medical domains.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.11324v2",
            "date": "2019-11-26 03:57:15+00:00",
            "title": "Tracing State-Level Obesity Prevalence from Sentence Embeddings of Tweets: A Feasibility Study",
            "authors": [
                "Xiaoyi Zhang",
                "Rodoniki Athanasiadou",
                "Narges Razavian"
            ],
            "abstract": "Twitter data has been shown broadly applicable for public health\nsurveillance. Previous public health studies based on Twitter data have largely\nrelied on keyword-matching or topic models for clustering relevant tweets.\nHowever, both methods suffer from the short-length of texts and unpredictable\nnoise that naturally occurs in user-generated contexts. In response, we\nintroduce a deep learning approach that uses hashtags as a form of supervision\nand learns tweet embeddings for extracting informative textual features. In\nthis case study, we address the specific task of estimating state-level obesity\nfrom dietary-related textual features. Our approach yields an estimation that\nstrongly correlates the textual features to government data and outperforms the\nkeyword-matching baseline. The results also demonstrate the potential of\ndiscovering risk factors using the textual features. This method is\ngeneral-purpose and can be applied to a wide range of Twitter-based public\nhealth studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11503v1",
            "date": "2019-11-26 13:05:33+00:00",
            "title": "Feature-Rich Part-of-speech Tagging for Morphologically Complex Languages: Application to Bulgarian",
            "authors": [
                "Georgi Georgiev",
                "Valentin Zhikov",
                "Petya Osenova",
                "Kiril Simov",
                "Preslav Nakov"
            ],
            "abstract": "We present experiments with part-of-speech tagging for Bulgarian, a Slavic\nlanguage with rich inflectional and derivational morphology. Unlike most\nprevious work, which has used a small number of grammatical categories, we work\nwith 680 morpho-syntactic tags. We combine a large morphological lexicon with\nprior linguistic knowledge and guided learning from a POS-annotated corpus,\nachieving accuracy of 97.98%, which is a significant improvement over the\nstate-of-the-art for Bulgarian.",
            "journal ref": "EACL-2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11698v1",
            "date": "2019-11-26 17:06:02+00:00",
            "title": "Doc2Vec on the PubMed corpus: study of a new approach to generate related articles",
            "authors": [
                "Emeric Dynomant",
                "St\u00e9fan J. Darmoni",
                "\u00c9meline Lejeune",
                "Ga\u00ebtan Kerdelhu\u00e9",
                "Jean-Philippe Leroy",
                "Vincent Lequertier",
                "St\u00e9phane Canu",
                "Julien Grosjean"
            ],
            "abstract": "PubMed is the biggest and most used bibliographic database worldwide, hosting\nmore than 26M biomedical publications. One of its useful features is the\n\"similar articles\" section, allowing the end-user to find scientific articles\nlinked to the consulted document in term of context. The aim of this study is\nto analyze whether it is possible to replace the statistic model PubMed Related\nArticles (pmra) with a document embedding method. Doc2Vec algorithm was used to\ntrain models allowing to vectorize documents. Six of its parameters were\noptimised by following a grid-search strategy to train more than 1,900 models.\nParameters combination leading to the best accuracy was used to train models on\nabstracts from the PubMed database. Four evaluations tasks were defined to\ndetermine what does or does not influence the proximity between documents for\nboth Doc2Vec and pmra. The two different Doc2Vec architectures have different\nabilities to link documents about a common context. The terminological\nindexing, words and stems contents of linked documents are highly similar\nbetween pmra and Doc2Vec PV-DBOW architecture. These algorithms are also more\nlikely to bring closer documents having a similar size. In contrary, the manual\nevaluation shows much better results for the pmra algorithm. While the pmra\nalgorithm links documents by explicitly using terminological indexing in its\nformula, Doc2Vec does not need a prior indexing. It can infer relations between\ndocuments sharing a similar indexing, without any knowledge about them,\nparticularly regarding the PV-DBOW architecture. In contrary, the human\nevaluation, without any clear agreement between evaluators, implies future\nstudies to better understand this difference between PV-DBOW and pmra\nalgorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11931v2",
            "date": "2019-11-27 03:22:40+00:00",
            "title": "Evaluating Commonsense in Pre-trained Language Models",
            "authors": [
                "Xuhui Zhou",
                "Yue Zhang",
                "Leyang Cui",
                "Dandan Huang"
            ],
            "abstract": "Contextualized representations trained over large raw text data have given\nremarkable improvements for NLP tasks including question answering and reading\ncomprehension. There have been works showing that syntactic, semantic and word\nsense knowledge are contained in such representations, which explains why they\nbenefit such tasks. However, relatively little work has been done investigating\ncommonsense knowledge contained in contextualized representations, which is\ncrucial for human question answering and reading comprehension. We study the\ncommonsense ability of GPT, BERT, XLNet, and RoBERTa by testing them on seven\nchallenging benchmarks, finding that language modeling and its variants are\neffective objectives for promoting models' commonsense ability while\nbi-directional context and larger training set are bonuses. We additionally\nfind that current models do poorly on tasks require more necessary inference\nsteps. Finally, we test the robustness of models by making dual test cases,\nwhich are correlated so that the correct prediction of one sample should lead\nto correct prediction of the other. Interestingly, the models show confusion on\nthese test cases, which suggests that they learn commonsense at the surface\nrather than the deep level. We release a test set, named CATs publicly, for\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12385v2",
            "date": "2019-11-27 19:09:41+00:00",
            "title": "DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling",
            "authors": [
                "Sachin Mehta",
                "Rik Koncel-Kedziorski",
                "Mohammad Rastegari",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "For sequence models with large vocabularies, a majority of network parameters\nlie in the input and output layers. In this work, we describe a new method,\nDeFINE, for learning deep token representations efficiently. Our architecture\nuses a hierarchical structure with novel skip-connections which allows for the\nuse of low dimensional input and output layers, reducing total parameters and\ntraining time while delivering similar or better performance versus existing\nmethods. DeFINE can be incorporated easily in new or existing sequence models.\nCompared to state-of-the-art methods including adaptive input representations,\nthis technique results in a 6% to 20% drop in perplexity. On WikiText-103,\nDeFINE reduces the total parameters of Transformer-XL by half with minimal\nimpact on performance. On the Penn Treebank, DeFINE improves AWD-LSTM by 4\npoints with a 17% reduction in parameters, achieving comparable performance to\nstate-of-the-art methods with fewer parameters. For machine translation, DeFINE\nimproves the efficiency of the Transformer model by about 1.4 times while\ndelivering similar performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12543v2",
            "date": "2019-11-28 05:55:42+00:00",
            "title": "How Can We Know What Language Models Know?",
            "authors": [
                "Zhengbao Jiang",
                "Frank F. Xu",
                "Jun Araki",
                "Graham Neubig"
            ],
            "abstract": "Recent work has presented intriguing results examining the knowledge\ncontained in language models (LM) by having the LM fill in the blanks of\nprompts such as \"Obama is a _ by profession\". These prompts are usually\nmanually created, and quite possibly sub-optimal; another prompt such as \"Obama\nworked as a _\" may result in more accurately predicting the correct profession.\nBecause of this, given an inappropriate prompt, we might fail to retrieve facts\nthat the LM does know, and thus any given prompt only provides a lower bound\nestimate of the knowledge contained in an LM. In this paper, we attempt to more\naccurately estimate the knowledge contained in LMs by automatically discovering\nbetter prompts to use in this querying process. Specifically, we propose\nmining-based and paraphrasing-based methods to automatically generate\nhigh-quality and diverse prompts, as well as ensemble methods to combine\nanswers from different prompts. Extensive experiments on the LAMA benchmark for\nextracting relational knowledge from LMs demonstrate that our methods can\nimprove accuracy from 31.1% to 39.6%, providing a tighter lower bound on what\nLMs know. We have released the code and the resulting LM Prompt And Query\nArchive (LPAQA) at https://github.com/jzbjyb/LPAQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12547v1",
            "date": "2019-11-28 06:05:12+00:00",
            "title": "DiscoTK: Using Discourse Structure for Machine Translation Evaluation",
            "authors": [
                "Shafiq Joty",
                "Francisco Guzman",
                "Lluis Marquez",
                "Preslav Nakov"
            ],
            "abstract": "We present novel automatic metrics for machine translation evaluation that\nuse discourse structure and convolution kernels to compare the discourse tree\nof an automatic translation with that of the human reference. We experiment\nwith five transformations and augmentations of a base discourse tree\nrepresentation based on the rhetorical structure theory, and we combine the\nkernel scores for each of them into a single score. Finally, we add other\nmetrics from the ASIYA MT evaluation toolkit, and we tune the weights of the\ncombination on actual human judgments. Experiments on the WMT12 and WMT13\nmetrics shared task datasets show correlation with human judgments that\noutperforms what the best systems that participated in these years achieved,\nboth at the segment and at the system level.",
            "journal ref": "WMT-2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12559v1",
            "date": "2019-11-28 07:12:30+00:00",
            "title": "KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents",
            "authors": [
                "Ygor Gallina",
                "Florian Boudin",
                "B\u00e9atrice Daille"
            ],
            "abstract": "Keyphrase generation is the task of predicting a set of lexical units that\nconveys the main content of a source text. Existing datasets for keyphrase\ngeneration are only readily available for the scholarly domain and include\nnon-expert annotations. In this paper we present KPTimes, a large-scale dataset\nof news texts paired with editor-curated keyphrases. Exploring the dataset, we\nshow how editors tag documents, and how their annotations differ from those\nfound in existing datasets. We also train and evaluate state-of-the-art neural\nkeyphrase generation models on KPTimes to gain insights on how well they\nperform on the news domain. The dataset is available online at\nhttps://github.com/ygorg/KPTimes .",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.12753v1",
            "date": "2019-11-28 15:38:53+00:00",
            "title": "Inducing Relational Knowledge from BERT",
            "authors": [
                "Zied Bouraoui",
                "Jose Camacho-Collados",
                "Steven Schockaert"
            ],
            "abstract": "One of the most remarkable properties of word embeddings is the fact that\nthey capture certain types of semantic and syntactic relationships. Recently,\npre-trained language models such as BERT have achieved groundbreaking results\nacross a wide range of Natural Language Processing tasks. However, it is\nunclear to what extent such models capture relational knowledge beyond what is\nalready captured by standard word embeddings. To explore this question, we\npropose a methodology for distilling relational knowledge from a pre-trained\nlanguage model. Starting from a few seed instances of a given relation, we\nfirst use a large text corpus to find sentences that are likely to express this\nrelation. We then use a subset of these extracted sentences as templates.\nFinally, we fine-tune a language model to predict whether a given word pair is\nlikely to be an instance of some relation, when given an instantiated template\nfor that relation as input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12986v1",
            "date": "2019-11-29 07:48:19+00:00",
            "title": "Merging Weak and Active Supervision for Semantic Parsing",
            "authors": [
                "Ansong Ni",
                "Pengcheng Yin",
                "Graham Neubig"
            ],
            "abstract": "A semantic parser maps natural language commands (NLs) from the users to\nexecutable meaning representations (MRs), which are later executed in certain\nenvironment to obtain user-desired results. The fully-supervised training of\nsuch parser requires NL/MR pairs, annotated by domain experts, which makes them\nexpensive to collect. However, weakly-supervised semantic parsers are learnt\nonly from pairs of NL and expected execution results, leaving the MRs latent.\nWhile weak supervision is cheaper to acquire, learning from this input poses\ndifficulties. It demands that parsers search a large space with a very weak\nlearning signal and it is hard to avoid spurious MRs that achieve the correct\nanswer in the wrong way. These factors lead to a performance gap between\nparsers trained in weakly- and fully-supervised setting. To bridge this gap, we\nexamine the intersection between weak supervision and active learning, which\nallows the learner to actively select examples and query for manual annotations\nas extra supervision to improve the model trained under weak supervision. We\nstudy different active learning heuristics for selecting examples to query, and\nvarious forms of extra supervision for such queries. We evaluate the\neffectiveness of our method on two different datasets. Experiments on the\nWikiSQL show that by annotating only 1.8% of examples, we improve over a\nstate-of-the-art weakly-supervised baseline by 6.4%, achieving an accuracy of\n79.0%, which is only 1.3% away from the model trained with full supervision.\nExperiments on WikiTableQuestions with human annotators show that our method\ncan improve the performance with only 100 active queries, especially for\nweakly-supervised parsers learnt from a cold start.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13182v1",
            "date": "2019-11-29 16:34:00+00:00",
            "title": "An Iterative Polishing Framework based on Quality Aware Masked Language Model for Chinese Poetry Generation",
            "authors": [
                "Liming Deng",
                "Jie Wang",
                "Hangming Liang",
                "Hui Chen",
                "Zhiqiang Xie",
                "Bojin Zhuang",
                "Shaojun Wang",
                "Jing Xiao"
            ],
            "abstract": "Owing to its unique literal and aesthetical characteristics, automatic\ngeneration of Chinese poetry is still challenging in Artificial Intelligence,\nwhich can hardly be straightforwardly realized by end-to-end methods. In this\npaper, we propose a novel iterative polishing framework for highly qualified\nChinese poetry generation. In the first stage, an encoder-decoder structure is\nutilized to generate a poem draft. Afterwards, our proposed Quality-Aware\nMasked Language Model (QAMLM) is employed to polish the draft towards higher\nquality in terms of linguistics and literalness. Based on a multi-task learning\nscheme, QA-MLM is able to determine whether polishing is needed based on the\npoem draft. Furthermore, QAMLM is able to localize improper characters of the\npoem draft and substitute with newly predicted ones accordingly. Benefited from\nthe masked language model structure, QAMLM incorporates global context\ninformation into the polishing process, which can obtain more appropriate\npolishing results than the unidirectional sequential decoding. Moreover, the\niterative polishing process will be terminated automatically when QA-MLM\nregards the processed poem as a qualified one. Both human and automatic\nevaluation have been conducted, and the results demonstrate that our approach\nis effective to improve the performance of encoder-decoder structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13207v1",
            "date": "2019-11-27 15:01:57+00:00",
            "title": "A concrete example of inclusive design: deaf-oriented accessibility",
            "authors": [
                "Claudia Bianchini",
                "Fabrizio Borgia",
                "Maria de Marsico"
            ],
            "abstract": "One of the continuing challenges of Human Computer Interaction research is\nthe full inclusion of people with special needs into the digital world. In\nparticular, this crucial category includes people that experiences some kind of\nlimitation in exploiting traditional information communication channels. One\nimmediately thinks about blind people, and several researches aim at addressing\ntheir needs. On the contrary, limitations suffered by deaf people are often\nunderestimated. This often the result of a kind of ignorance or\nmisunderstanding of the real nature of their communication difficulties. This\nchapter aims at both increasing the awareness of deaf problems in the digital\nworld, and at proposing the project of a comprehensive solution for their\nbetter inclusion. As for the former goal, we will provide a bird's-eye\npresentation of history and evolution of understanding of deafness issues, and\nof strategies to address them. As for the latter, we will present the design,\nimplementation and evaluation of the first nucleus of a comprehensive digital\nframework to facilitate the access of deaf people into the digital world.",
            "journal ref": "The Wiley Handbook of Human Computer Interaction, 2 (chapter 33),\n  Wiley (Blackwell), pp.731-756, 2018, 978-1-118-97613-5",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.13231v1",
            "date": "2019-11-27 14:55:16+00:00",
            "title": "Towards improving the e-learning experience for deaf students: e-LUX",
            "authors": [
                "Fabrizio Borgia",
                "Claudia Bianchini",
                "Maria de Marsico"
            ],
            "abstract": "Deaf people are more heavily affected by the digital divide than many would\nexpect. Moreover, most accessibility guidelines addressing their needs just\ndeal with captioning and audio-content transcription. However, this approach to\nthe problem does not consider that deaf people have big troubles with vocal\nlanguages, even in their written form. At present, only a few organizations,\nlike W3C, produced guidelines dealing with one of their most distinctive\nexpressions: Sign Language (SL). SL is, in fact, the visual-gestural language\nused by many deaf people to communicate with each other. The present work aims\nat supporting e-learning user experience (e-LUX) for these specific users by\nenhancing the accessibility of content and container services. In particular,\nwe propose preliminary solutions to tailor activities which can be more\nfruitful when performed in one's own \"native\" language, which for most deaf\npeople, especially younger ones, is represented by national SL.",
            "journal ref": "Lecture Notes in Computer Science, Springer, 2014, Universal\n  access in human-computer interaction: universal access to information and\n  knowledge, 8514 (2), pp.221-232",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.13232v1",
            "date": "2019-11-26 22:18:40+00:00",
            "title": "CONAN: Complementary Pattern Augmentation for Rare Disease Detection",
            "authors": [
                "Limeng Cui",
                "Siddharth Biswal",
                "Lucas M. Glass",
                "Greg Lever",
                "Jimeng Sun",
                "Cao Xiao"
            ],
            "abstract": "Rare diseases affect hundreds of millions of people worldwide but are hard to\ndetect since they have extremely low prevalence rates (varying from 1/1,000 to\n1/200,000 patients) and are massively underdiagnosed. How do we reliably detect\nrare diseases with such low prevalence rates? How to further leverage patients\nwith possibly uncertain diagnosis to improve detection? In this paper, we\npropose a Complementary pattern Augmentation (CONAN) framework for rare disease\ndetection. CONAN combines ideas from both adversarial training and max-margin\nclassification. It first learns self-attentive and hierarchical embedding for\npatient pattern characterization. Then, we develop a complementary generative\nadversarial networks (GAN) model to generate candidate positive and negative\nsamples from the uncertain patients by encouraging a max-margin between\nclasses. In addition, CONAN has a disease detector that serves as the\ndiscriminator during the adversarial training for identifying rare diseases. We\nevaluated CONAN on two disease detection tasks. For low prevalence inflammatory\nbowel disease (IBD) detection, CONAN achieved .96 precision recall area under\nthe curve (PR-AUC) and 50.1% relative improvement over best baseline. For rare\ndisease idiopathic pulmonary fibrosis (IPF) detection, CONAN achieves .22\nPR-AUC with 41.3% relative improvement over the best baseline.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.13280v1",
            "date": "2019-11-29 18:27:36+00:00",
            "title": "Deconstructing and reconstructing word embedding algorithms",
            "authors": [
                "Edward Newell",
                "Kian Kenyon-Dean",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Uncontextualized word embeddings are reliable feature representations of\nwords used to obtain high quality results for various NLP applications. Given\nthe historical success of word embeddings in NLP, we propose a retrospective on\nsome of the most well-known word embedding algorithms. In this work, we\ndeconstruct Word2vec, GloVe, and others, into a common form, unveiling some of\nthe necessary and sufficient conditions required for making performant word\nembeddings. We find that each algorithm: (1) fits vector-covector dot products\nto approximate pointwise mutual information (PMI); and, (2) modulates the loss\ngradient to balance weak and strong signals. We demonstrate that these two\nalgorithmic features are sufficient conditions to construct a novel word\nembedding algorithm, Hilbert-MLE. We find that its embeddings obtain equivalent\nor better performance against other algorithms across 17 intrinsic and\nextrinsic datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00147v2",
            "date": "2019-11-30 07:13:25+00:00",
            "title": "Integrating Graph Contextualized Knowledge into Pre-trained Language Models",
            "authors": [
                "Bin He",
                "Di Zhou",
                "Jinghui Xiao",
                "Xin jiang",
                "Qun Liu",
                "Nicholas Jing Yuan",
                "Tong Xu"
            ],
            "abstract": "Complex node interactions are common in knowledge graphs, and these\ninteractions also contain rich knowledge information. However, traditional\nmethods usually treat a triple as a training unit during the knowledge\nrepresentation learning (KRL) procedure, neglecting contextualized information\nof the nodes in knowledge graphs (KGs). We generalize the modeling object to a\nvery general form, which theoretically supports any subgraph extracted from the\nknowledge graph, and these subgraphs are fed into a novel transformer-based\nmodel to learn the knowledge embeddings. To broaden usage scenarios of\nknowledge, pre-trained language models are utilized to build a model that\nincorporates the learned knowledge representations. Experimental results\ndemonstrate that our model achieves the state-of-the-art performance on several\nmedical NLP tasks, and improvement above TransE indicates that our KRL method\ncaptures the graph contextualized information effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00178v1",
            "date": "2019-11-30 10:30:46+00:00",
            "title": "Modeling Fluency and Faithfulness for Diverse Neural Machine Translation",
            "authors": [
                "Yang Feng",
                "Wanying Xie",
                "Shuhao Gu",
                "Chenze Shao",
                "Wen Zhang",
                "Zhengxin Yang",
                "Dong Yu"
            ],
            "abstract": "Neural machine translation models usually adopt the teacher forcing strategy\nfor training which requires the predicted sequence matches ground truth word by\nword and forces the probability of each prediction to approach a 0-1\ndistribution. However, the strategy casts all the portion of the distribution\nto the ground truth word and ignores other words in the target vocabulary even\nwhen the ground truth word cannot dominate the distribution. To address the\nproblem of teacher forcing, we propose a method to introduce an evaluation\nmodule to guide the distribution of the prediction. The evaluation module\naccesses each prediction from the perspectives of fluency and faithfulness to\nencourage the model to generate the word which has a fluent connection with its\npast and future translation and meanwhile tends to form a translation\nequivalent in meaning to the source. The experiments on multiple translation\ntasks show that our method can achieve significant improvements over strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00336v2",
            "date": "2019-12-01 06:53:23+00:00",
            "title": "Semi-supervised Visual Feature Integration for Pre-trained Language Models",
            "authors": [
                "Lisai Zhang",
                "Qingcai Chen",
                "Dongfang Li",
                "Buzhou Tang"
            ],
            "abstract": "Integrating visual features has been proved useful for natural language\nunderstanding tasks. Nevertheless, in most existing multimodal language models,\nthe alignment of visual and textual data is expensive. In this paper, we\npropose a novel semi-supervised visual integration framework for pre-trained\nlanguage models. In the framework, the visual features are obtained through a\nvisualization and fusion mechanism. The uniqueness includes: 1) the integration\nis conducted via a semi-supervised approach, which does not require aligned\nimages for every sentences 2) the visual features are integrated as an external\ncomponent and can be directly used by pre-trained language models. To verify\nthe efficacy of the proposed framework, we conduct the experiments on both\nnatural language inference and reading comprehension tasks. The results\ndemonstrate that our mechanism brings improvement to two strong baseline\nmodels. Considering that our framework only requires an image database, and no\nnot requires further alignments, it provides an efficient and feasible way for\nmultimodal language learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00423v1",
            "date": "2019-12-01 15:06:44+00:00",
            "title": "Semantic Enrichment of Streaming Healthcare Data",
            "authors": [
                "Daniel Cotter",
                "V. K. Cody Bumgardner"
            ],
            "abstract": "In the past decade, the healthcare industry has made significant advances in\nthe digitization of patient information. However, a lack of interoperability\namong healthcare systems still imposes a high cost to patients, hospitals, and\ninsurers. Currently, most systems pass messages using idiosyncratic messaging\nstandards that require specialized knowledge to interpret. This increases the\ncost of systems integration and often puts more advanced uses of data out of\nreach. In this project, we demonstrate how two open standards, FHIR and RDF,\ncan be combined both to integrate data from disparate sources in real-time and\nmake that data queryable and susceptible to automated inference. To validate\nthe effectiveness of the semantic engine, we perform simulations of real-time\ndata feeds and demonstrate how they can be combined and used by client-side\napplications with no knowledge of the underlying sources.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.00463v1",
            "date": "2019-12-01 18:16:18+00:00",
            "title": "Generalizable prediction of academic performance from short texts on social media",
            "authors": [
                "Ivan Smirnov"
            ],
            "abstract": "It has already been established that digital traces can be used to predict\nvarious human attributes. In most cases, however, predictive models rely on\nfeatures that are specific to a particular source of digital trace data. In\ncontrast, short texts written by users $-$ tweets, posts, or comments $-$ are\nubiquitous across multiple platforms. In this paper, we explore the predictive\npower of short texts with respect to the academic performance of their authors.\nWe use data from a representative panel of Russian students that includes\ninformation about their educational outcomes and activity on a popular\nnetworking site, VK. We build a model to predict academic performance from\nusers' posts on VK and then apply it to a different context. In particular, we\nshow that the model could reproduce rankings of schools and universities from\nthe posts of their students on social media. We also find that the same model\ncould predict academic performance from tweets as well as from VK posts. The\ngeneralizability of a model trained on a relatively small data set could be\nexplained by the use of continuous word representations trained on a much\nlarger corpus of social media posts. This also allows for greater\ninterpretability of model predictions.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1912.00509v2",
            "date": "2019-12-01 22:08:32+00:00",
            "title": "Speeding up Word Mover's Distance and its variants via properties of distances between embeddings",
            "authors": [
                "Matheus Werner",
                "Eduardo Laber"
            ],
            "abstract": "The Word Mover's Distance (WMD) proposed by Kusner et al. is a distance\nbetween documents that takes advantage of semantic relations among words that\nare captured by their embeddings. This distance proved to be quite effective,\nobtaining state-of-art error rates for classification tasks, but is also\nimpracticable for large collections/documents due to its computational\ncomplexity. For circumventing this problem, variants of WMD have been proposed.\nAmong them, Relaxed Word Mover's Distance (RWMD) is one of the most successful\ndue to its simplicity, effectiveness, and also because of its fast\nimplementations.\n  Relying on assumptions that are supported by empirical properties of the\ndistances between embeddings, we propose an approach to speed up both WMD and\nRWMD. Experiments over 10 datasets suggest that our approach leads to a\nsignificant speed-up in document classification tasks while maintaining the\nsame error rates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00544v1",
            "date": "2019-12-02 02:08:00+00:00",
            "title": "Multi-Scale Self-Attention for Text Classification",
            "authors": [
                "Qipeng Guo",
                "Xipeng Qiu",
                "Pengfei Liu",
                "Xiangyang Xue",
                "Zheng Zhang"
            ],
            "abstract": "In this paper, we introduce the prior knowledge, multi-scale structure, into\nself-attention modules. We propose a Multi-Scale Transformer which uses\nmulti-scale multi-head self-attention to capture features from different\nscales. Based on the linguistic perspective and the analysis of pre-trained\nTransformer (BERT) on a huge corpus, we further design a strategy to control\nthe scale distribution for each layer. Results of three different kinds of\ntasks (21 datasets) show our Multi-Scale Transformer outperforms the standard\nTransformer consistently and significantly on small and moderate size datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00569v1",
            "date": "2019-12-02 03:14:23+00:00",
            "title": "Abstract Reasoning with Distracting Features",
            "authors": [
                "Kecheng Zheng",
                "Zheng-jun Zha",
                "Wei Wei"
            ],
            "abstract": "Abstraction reasoning is a long-standing challenge in artificial\nintelligence. Recent studies suggest that many of the deep architectures that\nhave triumphed over other domains failed to work well in abstract reasoning. In\nthis paper, we first illustrate that one of the main challenges in such a\nreasoning task is the presence of distracting features, which requires the\nlearning algorithm to leverage counterevidence and to reject any of the false\nhypotheses in order to learn the true patterns. We later show that carefully\ndesigned learning trajectory over different categories of training data can\neffectively boost learning performance by mitigating the impacts of distracting\nfeatures. Inspired by this fact, we propose feature robust abstract reasoning\n(FRAR) model, which consists of a reinforcement learning based teacher network\nto determine the sequence of training and a student network for predictions.\nExperimental results demonstrated strong improvements over baseline algorithms\nand we are able to beat the state-of-the-art models by 18.7% in the RAVEN\ndataset and 13.3% in the PGM dataset.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.00609v1",
            "date": "2019-12-02 07:41:25+00:00",
            "title": "GANCoder: An Automatic Natural Language-to-Programming Language Translation Approach based on GAN",
            "authors": [
                "Yabing Zhu",
                "Yanfeng Zhang",
                "Huili Yang",
                "Fangjing Wang"
            ],
            "abstract": "We propose GANCoder, an automatic programming approach based on Generative\nAdversarial Networks (GAN), which can generate the same functional and logical\nprogramming language codes conditioned on the given natural language\nutterances. The adversarial training between generator and discriminator helps\ngenerator learn distribution of dataset and improve code generation quality.\nOur experimental results show that GANCoder can achieve comparable accuracy\nwith the state-of-the-art methods and is more stable when programming\nlanguages.",
            "journal ref": "NLPCC 2019: Natural Language Processing and Chinese Computing",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00835v2",
            "date": "2019-11-26 16:01:51+00:00",
            "title": "Low Rank Factorization for Compact Multi-Head Self-Attention",
            "authors": [
                "Sneha Mehta",
                "Huzefa Rangwala",
                "Naren Ramakrishnan"
            ],
            "abstract": "Effective representation learning from text has been an active area of\nresearch in the fields of NLP and text mining. Attention mechanisms have been\nat the forefront in order to learn contextual sentence representations. Current\nstate-of-the-art approaches for many NLP tasks use large pre-trained language\nmodels such as BERT, XLNet and so on for learning representations. These models\nare based on the Transformer architecture that involves recurrent blocks of\ncomputation consisting of multi-head self-attention and feedforward networks.\nOne of the major bottlenecks largely contributing to the computational\ncomplexity of the Transformer models is the self-attention layer, that is both\ncomputationally expensive and parameter intensive. In this work, we introduce a\nnovel multi-head self-attention mechanism operating on GRUs that is shown to be\ncomputationally cheaper and more parameter efficient than self-attention\nmechanism proposed in Transformers for text classification tasks. The\nefficiency of our approach mainly stems from two optimizations; 1) we use\nlow-rank matrix factorization of the affinity matrix to efficiently get\nmultiple attention distributions instead of having separate parameters for each\nhead 2) attention scores are obtained by querying a global context vector\ninstead of densely querying all the words in the sentence. We evaluate the\nperformance of the proposed model on tasks such as sentiment analysis from\nmovie reviews, predicting business ratings from reviews and classifying news\narticles into topics. We find that the proposed approach matches or outperforms\na series of strong baselines and is more parameter efficient than comparable\nmulti-head approaches. We also perform qualitative analyses to verify that the\nproposed approach is interpretable and captures context-dependent word\nimportance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00862v1",
            "date": "2019-11-25 11:23:04+00:00",
            "title": "ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network",
            "authors": [
                "Fei Li",
                "Hong Yu"
            ],
            "abstract": "Automated ICD coding, which assigns the International Classification of\nDisease codes to patient visits, has attracted much research attention since it\ncan save time and labor for billing. The previous state-of-the-art model\nutilized one convolutional layer to build document representations for\npredicting ICD codes. However, the lengths and grammar of text fragments, which\nare closely related to ICD coding, vary a lot in different documents.\nTherefore, a flat and fixed-length convolutional architecture may not be\ncapable of learning good document representations. In this paper, we proposed a\nMulti-Filter Residual Convolutional Neural Network (MultiResCNN) for ICD\ncoding. The innovations of our model are two-folds: it utilizes a multi-filter\nconvolutional layer to capture various text patterns with different lengths and\na residual convolutional layer to enlarge the receptive field. We evaluated the\neffectiveness of our model on the widely-used MIMIC dataset. On the full code\nset of MIMIC-III, our model outperformed the state-of-the-art model in 4 out of\n6 evaluation metrics. On the top-50 code set of MIMIC-III and the full code set\nof MIMIC-II, our model outperformed all the existing and state-of-the-art\nmodels in all evaluation metrics. The code is available at\nhttps://github.com/foxlf823/Multi-Filter-Residual-Convolutional-Neural-Network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00863v1",
            "date": "2019-11-25 07:35:16+00:00",
            "title": "Independent language modeling architecture for end-to-end ASR",
            "authors": [
                "Van Tung Pham",
                "Haihua Xu",
                "Yerbolat Khassanov",
                "Zhiping Zeng",
                "Eng Siong Chng",
                "Chongjia Ni",
                "Bin Ma",
                "Haizhou Li"
            ],
            "abstract": "The attention-based end-to-end (E2E) automatic speech recognition (ASR)\narchitecture allows for joint optimization of acoustic and language models\nwithin a single network. However, in a vanilla E2E ASR architecture, the\ndecoder sub-network (subnet), which incorporates the role of the language model\n(LM), is conditioned on the encoder output. This means that the acoustic\nencoder and the language model are entangled that doesn't allow language model\nto be trained separately from external text data. To address this problem, in\nthis work, we propose a new architecture that separates the decoder subnet from\nthe encoder output. In this way, the decoupled subnet becomes an independently\ntrainable LM subnet, which can easily be updated using the external text data.\nWe study two strategies for updating the new architecture. Experimental results\nshow that, 1) the independent LM architecture benefits from external text data,\nachieving 9.3% and 22.8% relative character and word error rate reduction on\nMandarin HKUST and English NSC datasets respectively; 2)the proposed\narchitecture works well with external LM and can be generalized to different\namount of labelled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00903v3",
            "date": "2019-12-02 16:30:46+00:00",
            "title": "ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation",
            "authors": [
                "Nora Hollenstein",
                "Marius Troendle",
                "Ce Zhang",
                "Nicolas Langer"
            ],
            "abstract": "We recorded and preprocessed ZuCo 2.0, a new dataset of simultaneous\neye-tracking and electroencephalography during natural reading and during\nannotation. This corpus contains gaze and brain activity data of 739 sentences,\n349 in a normal reading paradigm and 390 in a task-specific paradigm, in which\nthe 18 participants actively search for a semantic relation type in the given\nsentences as a linguistic annotation task. This new dataset complements ZuCo\n1.0 by providing experiments designed to analyze the differences in cognitive\nprocessing between natural reading and annotation. The data is freely available\nhere: https://osf.io/2urht/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00958v1",
            "date": "2019-12-02 17:42:58+00:00",
            "title": "Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition",
            "authors": [
                "Surabhi Punjabi",
                "Harish Arsikere",
                "Sri Garimella"
            ],
            "abstract": "Building conversational speech recognition systems for new languages is\nconstrained by the availability of utterances that capture user-device\ninteractions. Data collection is both expensive and limited by the speed of\nmanual transcription. In order to address this, we advocate the use of neural\nmachine translation as a data augmentation technique for bootstrapping language\nmodels. Machine translation (MT) offers a systematic way of incorporating\ncollections from mature, resource-rich conversational systems that may be\navailable for a different language. However, ingesting raw translations from a\ngeneral purpose MT system may not be effective owing to the presence of named\nentities, intra sentential code-switching and the domain mismatch between the\nconversational data being translated and the parallel text used for MT\ntraining. To circumvent this, we explore the following domain adaptation\ntechniques: (a) sentence embedding based data selection for MT training, (b)\nmodel finetuning, and (c) rescoring and filtering translated hypotheses. Using\nHindi as the experimental testbed, we translate US English utterances to\nsupplement the transcribed collections. We observe a relative word error rate\nreduction of 7.8-15.6%, depending on the bootstrapping phase. Fine grained\nanalysis reveals that translation particularly aids the interaction scenarios\nwhich are underrepresented in the transcribed data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01079v2",
            "date": "2019-12-02 21:19:22+00:00",
            "title": "Learning Word Ratings for Empathy and Distress from Document-Level User Responses",
            "authors": [
                "Jo\u00e3o Sedoc",
                "Sven Buechel",
                "Yehonathan Nachmany",
                "Anneke Buffone",
                "Lyle Ungar"
            ],
            "abstract": "Despite the excellent performance of black box approaches to modeling\nsentiment and emotion, lexica (sets of informative words and associated\nweights) that characterize different emotions are indispensable to the NLP\ncommunity because they allow for interpretable and robust predictions. Emotion\nanalysis of text is increasing in popularity in NLP; however, manually creating\nlexica for psychological constructs such as empathy has proven difficult. This\npaper automatically creates empathy word ratings from document-level ratings.\nThe underlying problem of learning word ratings from higher-level supervision\nhas to date only been addressed in an ad hoc fashion and has not used deep\nlearning methods. We systematically compare a number of approaches to learning\nword ratings from higher-level supervision against a Mixed-Level Feed Forward\nNetwork (MLFFN), which we find performs best, and use the MLFFN to create the\nfirst-ever empathy lexicon. We then use Signed Spectral Clustering to gain\ninsights into the resulting words. The empathy and distress lexica are publicly\navailable at: http://www.wwbp.org/lexica.html.",
            "journal ref": "Proceedings of The 12th Language Resources and Evaluation\n  Conference (LREC 2020). Pages 1657-1666",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01113v1",
            "date": "2019-11-23 21:33:31+00:00",
            "title": "Using the Web as an Implicit Training Set: Application to Noun Compound Syntax and Semantics",
            "authors": [
                "Preslav Nakov"
            ],
            "abstract": "An important characteristic of English written text is the abundance of noun\ncompounds - sequences of nouns acting as a single noun, e.g., colon cancer\ntumor suppressor protein. While eventually mastered by domain experts, their\ninterpretation poses a major challenge for automated analysis. Understanding\nnoun compounds' syntax and semantics is important for many natural language\napplications, including question answering, machine translation, information\nretrieval, and information extraction. I address the problem of noun compounds\nsyntax by means of novel, highly accurate unsupervised and lightly supervised\nalgorithms using the Web as a corpus and search engines as interfaces to that\ncorpus. Traditionally the Web has been viewed as a source of page hit counts,\nused as an estimate for n-gram word frequencies. I extend this approach by\nintroducing novel surface features and paraphrases, which yield\nstate-of-the-art results for the task of noun compound bracketing. I also show\nhow these kinds of features can be applied to other structural ambiguity\nproblems, like prepositional phrase attachment and noun phrase coordination. I\naddress noun compound semantics by automatically generating paraphrasing verbs\nand prepositions that make explicit the hidden semantic relations between the\nnouns in a noun compound. I also demonstrate how these paraphrasing verbs can\nbe used to solve various relational similarity problems, and how paraphrasing\nnoun compounds can improve machine translation.",
            "journal ref": "PhD Thesis, University of California at Berkeley, 2007",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01119v2",
            "date": "2019-12-02 23:09:16+00:00",
            "title": "Deep Bayesian Active Learning for Multiple Correct Outputs",
            "authors": [
                "Khaled Jedoui",
                "Ranjay Krishna",
                "Michael Bernstein",
                "Li Fei-Fei"
            ],
            "abstract": "Typical active learning strategies are designed for tasks, such as\nclassification, with the assumption that the output space is mutually\nexclusive. The assumption that these tasks always have exactly one correct\nanswer has resulted in the creation of numerous uncertainty-based measurements,\nsuch as entropy and least confidence, which operate over a model's outputs.\nUnfortunately, many real-world vision tasks, like visual question answering and\nimage captioning, have multiple correct answers, causing these measurements to\noverestimate uncertainty and sometimes perform worse than a random sampling\nbaseline. In this paper, we propose a new paradigm that estimates uncertainty\nin the model's internal hidden space instead of the model's output space. We\nspecifically study a manifestation of this problem for visual question answer\ngeneration (VQA), where the aim is not to classify the correct answer but to\nproduce a natural language answer, given an image and a question. Our method\novercomes the paraphrastic nature of language. It requires a semantic space\nthat structures the model's output concepts and that enables the usage of\ntechniques like dropout-based Bayesian uncertainty. We build a visual-semantic\nspace that embeds paraphrases close together for any existing VQA model. We\nempirically show state-of-art active learning results on the task of VQA on two\ndatasets, being 5 times more cost-efficient on Visual Genome and 3 times more\ncost-efficient on VQA 2.0.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.01131v2",
            "date": "2019-12-03 00:12:09+00:00",
            "title": "See and Read: Detecting Depression Symptoms in Higher Education Students Using Multimodal Social Media Data",
            "authors": [
                "Paulo Mann",
                "Aline Paes",
                "Elton H. Matsushima"
            ],
            "abstract": "Mental disorders such as depression and anxiety have been increasing at\nalarming rates in the worldwide population. Notably, the major depressive\ndisorder has become a common problem among higher education students,\naggravated, and maybe even occasioned, by the academic pressures they must\nface. While the reasons for this alarming situation remain unclear (although\nwidely investigated), the student already facing this problem must receive\ntreatment. To that, it is first necessary to screen the symptoms. The\ntraditional way for that is relying on clinical consultations or answering\nquestionnaires. However, nowadays, the data shared at social media is a\nubiquitous source that can be used to detect the depression symptoms even when\nthe student is not able to afford or search for professional care. Previous\nworks have already relied on social media data to detect depression on the\ngeneral population, usually focusing on either posted images or texts or\nrelying on metadata. In this work, we focus on detecting the severity of the\ndepression symptoms in higher education students, by comparing deep learning to\nfeature engineering models induced from both the pictures and their captions\nposted on Instagram. The experimental results show that students presenting a\nBDI score higher or equal than 20 can be detected with 0.92 of recall and 0.69\nof precision in the best case, reached by a fusion model. Our findings show the\npotential of large-scale depression screening, which could shed light upon\nstudents at-risk.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01218v1",
            "date": "2019-12-03 06:56:15+00:00",
            "title": "Writing Across the World's Languages: Deep Internationalization for Gboard, the Google Keyboard",
            "authors": [
                "Daan van Esch",
                "Elnaz Sarbar",
                "Tamar Lucassen",
                "Jeremy O'Brien",
                "Theresa Breiner",
                "Manasa Prasad",
                "Evan Crew",
                "Chieu Nguyen",
                "Fran\u00e7oise Beaufays"
            ],
            "abstract": "This technical report describes our deep internationalization program for\nGboard, the Google Keyboard. Today, Gboard supports 900+ language varieties\nacross 70+ writing systems, and this report describes how and why we have been\nadding support for hundreds of language varieties from around the globe. Many\nlanguages of the world are increasingly used in writing on an everyday basis,\nand we describe the trends we see. We cover technological and logistical\nchallenges in scaling up a language technology product like Gboard to hundreds\nof language varieties, and describe how we built systems and processes to\noperate at scale. Finally, we summarize the key take-aways from user studies we\nran with speakers of hundreds of languages from around the world.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1912.01220v1",
            "date": "2019-12-03 07:02:38+00:00",
            "title": "Modelling Semantic Categories using Conceptual Neighborhood",
            "authors": [
                "Zied Bouraoui",
                "Jose Camacho-Collados",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "While many methods for learning vector space embeddings have been proposed in\nthe field of Natural Language Processing, these methods typically do not\ndistinguish between categories and individuals. Intuitively, if individuals are\nrepresented as vectors, we can think of categories as (soft) regions in the\nembedding space. Unfortunately, meaningful regions can be difficult to\nestimate, especially since we often have few examples of individuals that\nbelong to a given category. To address this issue, we rely on the fact that\ndifferent categories are often highly interdependent. In particular, categories\noften have conceptual neighbors, which are disjoint from but closely related to\nthe given category (e.g.\\ fruit and vegetable). Our hypothesis is that more\naccurate category representations can be learned by relying on the assumption\nthat the regions representing such conceptual neighbors should be adjacent in\nthe embedding space. We propose a simple method for identifying conceptual\nneighbors and then show that incorporating these conceptual neighbors indeed\nleads to more accurate region based representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01385v1",
            "date": "2019-12-03 14:19:20+00:00",
            "title": "TU Wien @ TREC Deep Learning '19 -- Simple Contextualization for Re-ranking",
            "authors": [
                "Sebastian Hofst\u00e4tter",
                "Markus Zlabinger",
                "Allan Hanbury"
            ],
            "abstract": "The usage of neural network models puts multiple objectives in conflict with\neach other: Ideally we would like to create a neural model that is effective,\nefficient, and interpretable at the same time. However, in most instances we\nhave to choose which property is most important to us. We used the opportunity\nof the TREC 2019 Deep Learning track to evaluate the effectiveness of a\nbalanced neural re-ranking approach. We submitted results of the TK\n(Transformer-Kernel) model: a neural re-ranking model for ad-hoc search using\nan efficient contextualization mechanism. TK employs a very small number of\nlightweight Transformer layers to contextualize query and document word\nembeddings. To score individual term interactions, we use a document-length\nenhanced kernel-pooling, which enables users to gain insight into the model.\nOur best result for the passage ranking task is: 0.420 MAP, 0.671 nDCG, 0.598\nP@10 (TUW19-p3 full). Our best result for the document ranking task is: 0.271\nMAP, 0.465 nDCG, 0.730 P@10 (TUW19-d3 re-ranking).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.01452v1",
            "date": "2019-11-30 09:32:38+00:00",
            "title": "Assessing the Robustness of Visual Question Answering",
            "authors": [
                "Jia-Hong Huang",
                "Modar Alfadly",
                "Bernard Ghanem",
                "Marcel Worring"
            ],
            "abstract": "Deep neural networks have been playing an essential role in the task of\nVisual Question Answering (VQA). Until recently, their accuracy has been the\nmain focus of research. Now there is a trend toward assessing the robustness of\nthese models against adversarial attacks by evaluating the accuracy of these\nmodels under increasing levels of noisiness in the inputs of VQA models. In\nVQA, the attack can target the image and/or the proposed query question, dubbed\nmain question, and yet there is a lack of proper analysis of this aspect of\nVQA. In this work, we propose a new method that uses semantically related\nquestions, dubbed basic questions, acting as noise to evaluate the robustness\nof VQA models. We hypothesize that as the similarity of a basic question to the\nmain question decreases, the level of noise increases. To generate a reasonable\nnoise level for a given main question, we rank a pool of basic questions based\non their similarity with this main question. We cast this ranking problem as a\nLASSO optimization problem. We also propose a novel robustness measure Rscore\nand two large-scale basic question datasets in order to standardize robustness\nanalysis of VQA models. The experimental results demonstrate that the proposed\nevaluation method is able to effectively analyze the robustness of VQA models.\nTo foster the VQA research, we will publish our proposed datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.01795v1",
            "date": "2019-12-04 04:39:32+00:00",
            "title": "Towards Building a Multilingual Sememe Knowledge Base: Predicting Sememes for BabelNet Synsets",
            "authors": [
                "Fanchao Qi",
                "Liang Chang",
                "Maosong Sun",
                "Sicong Ouyang",
                "Zhiyuan Liu"
            ],
            "abstract": "A sememe is defined as the minimum semantic unit of human languages. Sememe\nknowledge bases (KBs), which contain words annotated with sememes, have been\nsuccessfully applied to many NLP tasks. However, existing sememe KBs are built\non only a few languages, which hinders their widespread utilization. To address\nthe issue, we propose to build a unified sememe KB for multiple languages based\non BabelNet, a multilingual encyclopedic dictionary. We first build a dataset\nserving as the seed of the multilingual sememe KB. It manually annotates\nsememes for over $15$ thousand synsets (the entries of BabelNet). Then, we\npresent a novel task of automatic sememe prediction for synsets, aiming to\nexpand the seed dataset into a usable KB. We also propose two simple and\neffective models, which exploit different information of synsets. Finally, we\nconduct quantitative and qualitative analyses to explore important factors and\ndifficulties in the task. All the source code and data of this work can be\nobtained on https://github.com/thunlp/BabelNet-Sememe-Prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01972v1",
            "date": "2019-12-03 06:30:34+00:00",
            "title": "SemEval-2016 Task 3: Community Question Answering",
            "authors": [
                "Preslav Nakov",
                "Llu\u00eds M\u00e0rquez",
                "Alessandro Moschitti",
                "Walid Magdy",
                "Hamdy Mubarak",
                "Abed Alhakim Freihat",
                "James Glass",
                "Bilal Randeree"
            ],
            "abstract": "This paper describes the SemEval--2016 Task 3 on Community Question\nAnswering, which we offered in English and Arabic. For English, we had three\nsubtasks: Question--Comment Similarity (subtask A), Question--Question\nSimilarity (B), and Question--External Comment Similarity (C). For Arabic, we\nhad another subtask: Rerank the correct answers for a new question (D).\nEighteen teams participated in the task, submitting a total of 95 runs (38\nprimary and 57 contrastive) for the four subtasks. A variety of approaches and\nfeatures were used by the participating systems to address the different\nsubtasks, which are summarized in this paper. The best systems achieved an\nofficial score (MAP) of 79.19, 76.70, 55.41, and 45.83 in subtasks A, B, C, and\nD, respectively. These scores are significantly better than those for the\nbaselines that we provided. For subtask A, the best system improved over the\n2015 winner by 3 points absolute in terms of Accuracy.",
            "journal ref": "SemEval-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01973v1",
            "date": "2019-12-03 06:46:20+00:00",
            "title": "SemEval-2016 Task 4: Sentiment Analysis in Twitter",
            "authors": [
                "Preslav Nakov",
                "Alan Ritter",
                "Sara Rosenthal",
                "Fabrizio Sebastiani",
                "Veselin Stoyanov"
            ],
            "abstract": "This paper discusses the fourth year of the ``Sentiment Analysis in Twitter\nTask''. SemEval-2016 Task 4 comprises five subtasks, three of which represent a\nsignificant departure from previous editions. The first two subtasks are reruns\nfrom prior years and ask to predict the overall sentiment, and the sentiment\ntowards a topic in a tweet. The three new subtasks focus on two variants of the\nbasic ``sentiment classification in Twitter'' task. The first variant adopts a\nfive-point scale, which confers an ordinal character to the classification\ntask. The second variant focuses on the correct estimation of the prevalence of\neach class of interest, a task which has been called quantification in the\nsupervised learning literature. The task continues to be very popular,\nattracting a total of 43 teams.",
            "journal ref": "SemEval-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02077v1",
            "date": "2019-12-04 16:07:25+00:00",
            "title": "PDC -- a probabilistic distributional clustering algorithm: a case study on suicide articles in PubMed",
            "authors": [
                "Rezarta Islamaj",
                "Lana Yeganova",
                "Won Kim",
                "Natalie Xie",
                "W. John Wilbur",
                "Zhiyong Lu"
            ],
            "abstract": "The need to organize a large collection in a manner that facilitates human\ncomprehension is crucial given the ever-increasing volumes of information. In\nthis work, we present PDC (probabilistic distributional clustering), a novel\nalgorithm that, given a document collection, computes disjoint term sets\nrepresenting topics in the collection. The algorithm relies on probabilities of\nword co-occurrences to partition the set of terms appearing in the collection\nof documents into disjoint groups of related terms. In this work, we also\npresent an environment to visualize the computed topics in the term space and\nretrieve the most related PubMed articles for each group of terms. We\nillustrate the algorithm by applying it to PubMed documents on the topic of\nsuicide. Suicide is a major public health problem identified as the tenth\nleading cause of death in the US. In this application, our goal is to provide a\nglobal view of the mental health literature pertaining to the subject of\nsuicide, and through this, to help create a rich environment of multifaceted\ndata to guide health care researchers in their endeavor to better understand\nthe breadth, depth and scope of the problem. We demonstrate the usefulness of\nthe proposed algorithm by providing a web portal that allows mental health\nresearchers to peruse the suicide-related literature in PubMed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02114v1",
            "date": "2019-12-04 16:59:40+00:00",
            "title": "Keyword Aware Influential Community Search in Large Attributed Graphs",
            "authors": [
                "Md. Saiful Islam",
                "Mohammed Eunus Ali",
                "Yong-Bin Kang",
                "Timos Sellis",
                "Farhana M. Choudhury"
            ],
            "abstract": "We introduce a novel keyword-aware influential community query KICQ that\nfinds the most influential communities from an attributed graph, where an\ninfluential community is defined as a closely connected group of vertices\nhaving some dominance over other groups of vertices with the expertise (a set\nof keywords) matching with the query terms (words or phrases). We first design\nthe KICQ that facilitates users to issue an influential CS query intuitively by\nusing a set of query terms, and predicates (AND or OR). In this context, we\npropose a novel word-embedding based similarity model that enables semantic\ncommunity search, which substantially alleviates the limitations of exact\nkeyword based community search. Next, we propose a new influence measure for a\ncommunity that considers both the cohesiveness and influence of the community\nand eliminates the need for specifying values of internal parameters of a\nnetwork. Finally, we propose two efficient algorithms for searching influential\ncommunities in large attributed graphs. We present detailed experiments and a\ncase study to demonstrate the effectiveness and efficiency of the proposed\napproaches.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1912.02646v1",
            "date": "2019-12-05 15:28:05+00:00",
            "title": "Complete Variable-Length Codes: An Excursion into Word Edit Operations",
            "authors": [
                "Jean N\u00e9raud"
            ],
            "abstract": "Given an alphabet A and a binary relation $\\tau$ $\\subseteq$ A * x A * , a\nlanguage X $\\subseteq$ A * is $\\tau$-independent if $\\tau$ (X) $\\cap$ X =\n$\\emptyset$; X is $\\tau$-closed if $\\tau$ (X) $\\subseteq$ X. The language X is\ncomplete if any word over A is a factor of some concatenation of words in X.\nGiven a family of languages F containing X, X is maximal in F if no other set\nof F can stricly contain X. A language X $\\subseteq$ A * is a variable-length\ncode if any equation among the words of X is necessarily trivial. The study\ndiscusses the relationship between maximality and completeness in the case of\n$\\tau$-independent or $\\tau$-closed variable-length codes. We focus to the\nbinary relations by which the images of words are computed by deleting,\ninserting, or substituting some characters.",
            "journal ref": "LATA 2020, Mar 2020, Milan, Italy",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03223v1",
            "date": "2019-12-06 16:45:52+00:00",
            "title": "A limited-size ensemble of homogeneous CNN/LSTMs for high-performance word classification",
            "authors": [
                "Mahya Ameryan",
                "Lambert Schomaker"
            ],
            "abstract": "In recent years, long short-term memory neural networks (LSTMs) have been\napplied quite successfully to problems in handwritten text recognition.\nHowever, their strength is more located in handling sequences of variable\nlength than in handling geometric variability of the image patterns.\nFurthermore, the best results for LSTMs are often based on large-scale training\nof an ensemble of network instances. In this paper, an end-to-end convolutional\nLSTM Neural Network is used to handle both geometric variation and sequence\nvariability. We show that high performances can be reached on a common\nbenchmark set by using proper data augmentation for just five such networks\nusing a proper coding scheme and a proper voting scheme. The networks have\nsimilar architectures (Convolutional Neural Network (CNN): five layers,\nbidirectional LSTM (BiLSTM): three layers followed by a connectionist temporal\nclassification (CTC) processing step). The approach assumes differently-scaled\ninput images and different feature map sizes. Two datasets are used for\nevaluation of the performance of our algorithm: A standard benchmark RIMES\ndataset (French), and a historical handwritten dataset KdK (Dutch). Final\nperformance obtained for the word-recognition test of RIMES was 96.6%, a clear\nimprovement over other state-of-the-art approaches. On the KdK dataset, our\napproach also shows good results. The proposed approach is deployed in the Monk\nsearch engine for historical-handwriting collections.",
            "journal ref": "Neural Computing and Applications(2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03366v2",
            "date": "2019-12-06 22:11:37+00:00",
            "title": "Med2Meta: Learning Representations of Medical Concepts with Meta-Embeddings",
            "authors": [
                "Shaika Chowdhury",
                "Chenwei Zhang",
                "Philip S. Yu",
                "Yuan Luo"
            ],
            "abstract": "Distributed representations of medical concepts have been used to support\ndownstream clinical tasks recently. Electronic Health Records (EHR) capture\ndifferent aspects of patients' hospital encounters and serve as a rich source\nfor augmenting clinical decision making by learning robust medical concept\nembeddings. However, the same medical concept can be recorded in different\nmodalities (e.g., clinical notes, lab results)-with each capturing salient\ninformation unique to that modality-and a holistic representation calls for\nrelevant feature ensemble from all information sources. We hypothesize that\nrepresentations learned from heterogeneous data types would lead to performance\nenhancement on various clinical informatics and predictive modeling tasks. To\nthis end, our proposed approach makes use of meta-embeddings, embeddings\naggregated from learned embeddings. Firstly, modality-specific embeddings for\neach medical concept is learned with graph autoencoders. The ensemble of all\nthe embeddings is then modeled as a meta-embedding learning problem to\nincorporate their correlating and complementary information through a joint\nreconstruction. Empirical results of our model on both quantitative and\nqualitative clinical evaluations have shown improvements over state-of-the-art\nembedding models, thus validating our hypothesis.",
            "journal ref": "HEALTHINF 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03444v1",
            "date": "2019-12-07 05:30:09+00:00",
            "title": "PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English",
            "authors": [
                "Kelechi Ogueji",
                "Orevaoghene Ahia"
            ],
            "abstract": "Over 800 languages are spoken across West Africa. Despite the obvious\ndiversity among people who speak these languages, one language significantly\nunifies them all - West African Pidgin English. There are at least 80 million\nspeakers of West African Pidgin English. However, there is no known natural\nlanguage processing (NLP) work on this language. In this work, we perform the\nfirst NLP work on the most popular variant of the language, providing three\nmajor contributions. First, the provision of a Pidgin corpus of over 56000\nsentences, which is the largest we know of. Secondly, the training of the first\never cross-lingual embedding between Pidgin and English. This aligned embedding\nwill be helpful in the performance of various downstream tasks between English\nand Pidgin. Thirdly, the training of an Unsupervised Neural Machine Translation\nmodel between Pidgin and English which achieves BLEU scores of 7.93 from Pidgin\nto English, and 5.18 from English to Pidgin. In all, this work greatly reduces\nthe barrier of entry for future NLP works on West African Pidgin English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03457v1",
            "date": "2019-12-07 07:45:43+00:00",
            "title": "Unsung Challenges of Building and Deploying Language Technologies for Low Resource Language Communities",
            "authors": [
                "Pratik Joshi",
                "Christain Barnes",
                "Sebastin Santy",
                "Simran Khanuja",
                "Sanket Shah",
                "Anirudh Srinivasan",
                "Satwik Bhattamishra",
                "Sunayana Sitaram",
                "Monojit Choudhury",
                "Kalika Bali"
            ],
            "abstract": "In this paper, we examine and analyze the challenges associated with\ndeveloping and introducing language technologies to low-resource language\ncommunities. While doing so, we bring to light the successes and failures of\npast work in this area, challenges being faced in doing so, and what they have\nachieved. Throughout this paper, we take a problem-facing approach and describe\nessential factors which the success of such technologies hinges upon. We\npresent the various aspects in a manner which clarify and lay out the different\ntasks involved, which can aid organizations looking to make an impact in this\narea. We take the example of Gondi, an extremely-low resource Indian language,\nto reinforce and complement our discussion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03804v1",
            "date": "2019-12-09 01:11:59+00:00",
            "title": "Women in ISIS Propaganda: A Natural Language Processing Analysis of Topics and Emotions in a Comparison with Mainstream Religious Group",
            "authors": [
                "Mojtaba Heidarysafa",
                "Kamran Kowsari",
                "Tolu Odukoya",
                "Philip Potter",
                "Laura E. Barnes",
                "Donald E. Brown"
            ],
            "abstract": "Online propaganda is central to the recruitment strategies of extremist\ngroups and in recent years these efforts have increasingly extended to women.\nTo investigate ISIS' approach to targeting women in their online propaganda and\nuncover implications for counterterrorism, we rely on text mining and natural\nlanguage processing (NLP). Specifically, we extract articles published in Dabiq\nand Rumiyah (ISIS's online English language publications) to identify prominent\ntopics. To identify similarities or differences between these texts and those\nproduced by non-violent religious groups, we extend the analysis to articles\nfrom a Catholic forum dedicated to women. We also perform an emotional analysis\nof both of these resources to better understand the emotional components of\npropaganda. We rely on Depechemood (a lexical-base emotion analysis method) to\ndetect emotions most likely to be evoked in readers of these materials. The\nfindings indicate that the emotional appeal of ISIS and Catholic materials are\nsimilar",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03832v1",
            "date": "2019-12-09 03:38:16+00:00",
            "title": "Effective Attention Modeling for Neural Relation Extraction",
            "authors": [
                "Tapas Nayak",
                "Hwee Tou Ng"
            ],
            "abstract": "Relation extraction is the task of determining the relation between two\nentities in a sentence. Distantly-supervised models are popular for this task.\nHowever, sentences can be long and two entities can be located far from each\nother in a sentence. The pieces of evidence supporting the presence of a\nrelation between two entities may not be very direct, since the entities may be\nconnected via some indirect links such as a third entity or via co-reference.\nRelation extraction in such scenarios becomes more challenging as we need to\ncapture the long-distance interactions among the entities and other words in\nthe sentence. Also, the words in a sentence do not contribute equally in\nidentifying the relation between the two entities. To address this issue, we\npropose a novel and effective attention model which incorporates syntactic\ninformation of the sentence and a multi-factor attention mechanism. Experiments\non the New York Times corpus show that our proposed model outperforms prior\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03879v2",
            "date": "2019-12-09 07:22:54+00:00",
            "title": "AI2D-RST: A multimodal corpus of 1000 primary school science diagrams",
            "authors": [
                "Tuomo Hiippala",
                "Malihe Alikhani",
                "Jonas Haverinen",
                "Timo Kalliokoski",
                "Evanfiya Logacheva",
                "Serafina Orekhova",
                "Aino Tuomainen",
                "Matthew Stone",
                "John A. Bateman"
            ],
            "abstract": "This article introduces AI2D-RST, a multimodal corpus of 1000\nEnglish-language diagrams that represent topics in primary school natural\nsciences, such as food webs, life cycles, moon phases and human physiology. The\ncorpus is based on the Allen Institute for Artificial Intelligence Diagrams\n(AI2D) dataset, a collection of diagrams with crowd-sourced descriptions, which\nwas originally developed to support research on automatic diagram understanding\nand visual question answering. Building on the segmentation of diagram layouts\nin AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that\nprovides a rich description of their multimodal structure. Annotated by trained\nexperts, the layers describe (1) the grouping of diagram elements into\nperceptual units, (2) the connections set up by diagrammatic elements such as\narrows and lines, and (3) the discourse relations between diagram elements,\nwhich are described using Rhetorical Structure Theory (RST). Each annotation\nlayer in AI2D-RST is represented using a graph. The corpus is freely available\nfor research and teaching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04050v1",
            "date": "2019-12-05 04:52:24+00:00",
            "title": "PhoneBit: Efficient GPU-Accelerated Binary Neural Network Inference Engine for Mobile Phones",
            "authors": [
                "Gang Chen",
                "Shengyu He",
                "Haitao Meng",
                "Kai Huang"
            ],
            "abstract": "Over the last years, a great success of deep neural networks (DNNs) has been\nwitnessed in computer vision and other fields. However, performance and power\nconstraints make it still challenging to deploy DNNs on mobile devices due to\ntheir high computational complexity. Binary neural networks (BNNs) have been\ndemonstrated as a promising solution to achieve this goal by using bit-wise\noperations to replace most arithmetic operations. Currently, existing\nGPU-accelerated implementations of BNNs are only tailored for desktop\nplatforms. Due to architecture differences, mere porting of such\nimplementations to mobile devices yields suboptimal performance or is\nimpossible in some cases. In this paper, we propose PhoneBit, a GPU-accelerated\nBNN inference engine for Android-based mobile devices that fully exploits the\ncomputing power of BNNs on mobile GPUs. PhoneBit provides a set of\noperator-level optimizations including locality-friendly data layout, bit\npacking with vectorization and layers integration for efficient binary\nconvolution. We also provide a detailed implementation and parallelization\noptimization for PhoneBit to optimally utilize the memory bandwidth and\ncomputing power of mobile GPUs. We evaluate PhoneBit with AlexNet, YOLOv2 Tiny\nand VGG16 with their binary version. Our experiment results show that PhoneBit\ncan achieve significant speedup and energy efficiency compared with\nstate-of-the-art frameworks for mobile devices.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1912.04419v1",
            "date": "2019-12-09 23:18:13+00:00",
            "title": "Analysis of the Ethiopic Twitter Dataset for Abusive Speech in Amharic",
            "authors": [
                "Seid Muhie Yimam",
                "Abinew Ali Ayele",
                "Chris Biemann"
            ],
            "abstract": "In this paper, we present an analysis of the first Ethiopic Twitter Dataset\nfor the Amharic language targeted for recognizing abusive speech. The dataset\nhas been collected since 2014 that is written in Fidel script. Since several\nlanguages can be written using the Fidel script, we have used the existing\nAmharic, Tigrinya and Ge'ez corpora to retain only the Amharic tweets. We have\nanalyzed the tweets for abusive speech content with the following targets:\nAnalyze the distribution and tendency of abusive speech content over time and\ncompare the abusive speech content between a Twitter and general reference\nAmharic corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04748v1",
            "date": "2019-12-10 15:07:48+00:00",
            "title": "Fraud detection in telephone conversations for financial services using linguistic features",
            "authors": [
                "Nikesh Bajaj",
                "Tracy Goodluck Constance",
                "Marvin Rajwadi",
                "Julie Wall",
                "Mansour Moniri",
                "Cornelius Glackin",
                "Nigel Cannings",
                "Chris Woodruff",
                "James Laird"
            ],
            "abstract": "Detecting the elements of deception in a conversation is one of the most\nchallenging problems for the AI community. It becomes even more difficult to\ndesign a transparent system, which is fully explainable and satisfies the need\nfor financial and legal services to be deployed. This paper presents an\napproach for fraud detection in transcribed telephone conversations using\nlinguistic features. The proposed approach exploits the syntactic and semantic\ninformation of the transcription to extract both the linguistic markers and the\nsentiment of the customer's response. We demonstrate the results on real-world\nfinancial services data using simple, robust and explainable classifiers such\nas Naive Bayes, Decision Tree, Nearest Neighbours, and Support Vector Machines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05082v3",
            "date": "2019-12-11 02:03:31+00:00",
            "title": "A Collaborative Ecosystem for Digital Coptic Studies",
            "authors": [
                "Caroline T. Schroeder",
                "Amir Zeldes"
            ],
            "abstract": "Scholarship on underresourced languages bring with them a variety of\nchallenges which make access to the full spectrum of source materials and their\nevaluation difficult. For Coptic in particular, large scale analyses and any\nkind of quantitative work become difficult due to the fragmentation of\nmanuscripts, the highly fusional nature of an incorporational morphology, and\nthe complications of dealing with influences from Hellenistic era Greek, among\nother concerns. Many of these challenges, however, can be addressed using\nDigital Humanities tools and standards. In this paper, we outline some of the\nlatest developments in Coptic Scriptorium, a DH project dedicated to bringing\nCoptic resources online in uniform, machine readable, and openly available\nformats. Collaborative web-based tools create online 'virtual departments' in\nwhich scholars dispersed sparsely across the globe can collaborate, and natural\nlanguage processing tools counterbalance the scarcity of trained editors by\nenabling machine processing of Coptic text to produce searchable, annotated\ncorpora.",
            "journal ref": "Journal of Data Mining & Digital Humanities, Special Issue on\n  Collecting, Preserving, and Disseminating Endangered Cultural Heritage for\n  New Understandings through Multilingual Approaches (September 23, 2020)\n  jdmdh:6797",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05274v1",
            "date": "2019-12-11 12:50:48+00:00",
            "title": "Two Birds with One Stone: Investigating Invertible Neural Networks for Inverse Problems in Morphology",
            "authors": [
                "G\u00f6zde G\u00fcl \u015eahin",
                "Iryna Gurevych"
            ],
            "abstract": "Most problems in natural language processing can be approximated as inverse\nproblems such as analysis and generation at variety of levels from\nmorphological (e.g., cat+Plural <-> cats) to semantic (e.g., (call + 1 2) <->\n\"Calculate one plus two.\"). Although the tasks in both directions are closely\nrelated, general approach in the field has been to design separate models\nspecific for each task. However, having one shared model for both tasks, would\nhelp the researchers exploit the common knowledge among these problems with\nreduced time and memory requirements. We investigate a specific class of neural\nnetworks, called Invertible Neural Networks (INNs) (Ardizzone et al. 2019) that\nenable simultaneous optimization in both directions, hence allow addressing of\ninverse problems via a single model. In this study, we investigate INNs on\nmorphological problems casted as inverse problems. We apply INNs to various\nmorphological tasks with varying ambiguity and show that they provide\ncompetitive performance in both directions. We show that they are able to\nrecover the morphological input parameters, i.e., predicting the lemma (e.g.,\ncat) or the morphological tags (e.g., Plural) when run in the reverse\ndirection, without any significant performance drop in the forward direction,\ni.e., predicting the surface form (e.g., cats).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05372v4",
            "date": "2019-12-11 14:59:32+00:00",
            "title": "FlauBERT: Unsupervised Language Model Pre-training for French",
            "authors": [
                "Hang Le",
                "Lo\u00efc Vial",
                "Jibril Frej",
                "Vincent Segonne",
                "Maximin Coavoux",
                "Benjamin Lecouteux",
                "Alexandre Allauzen",
                "Beno\u00eet Crabb\u00e9",
                "Laurent Besacier",
                "Didier Schwab"
            ],
            "abstract": "Language models have become a key step to achieve state-of-the art results in\nmany different Natural Language Processing (NLP) tasks. Leveraging the huge\namount of unlabeled texts nowadays available, they provide an efficient way to\npre-train continuous word representations that can be fine-tuned for a\ndownstream task, along with their contextualization at the sentence level. This\nhas been widely demonstrated for English using contextualized representations\n(Dai and Le, 2015; Peters et al., 2018; Howard and Ruder, 2018; Radford et al.,\n2018; Devlin et al., 2019; Yang et al., 2019b). In this paper, we introduce and\nshare FlauBERT, a model learned on a very large and heterogeneous French\ncorpus. Models of different sizes are trained using the new CNRS (French\nNational Centre for Scientific Research) Jean Zay supercomputer. We apply our\nFrench language models to diverse NLP tasks (text classification, paraphrasing,\nnatural language inference, parsing, word sense disambiguation) and show that\nmost of the time they outperform other pre-training approaches. Different\nversions of FlauBERT as well as a unified evaluation protocol for the\ndownstream tasks, called FLUE (French Language Understanding Evaluation), are\nshared to the research community for further reproducible experiments in French\nNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05467v1",
            "date": "2019-12-11 17:05:18+00:00",
            "title": "MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation",
            "authors": [
                "Rumeng Li",
                "Xun Wang",
                "Hong Yu"
            ],
            "abstract": "Manipulating training data leads to robust neural models for MT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05728v1",
            "date": "2019-12-12 02:04:44+00:00",
            "title": "AliMe KBQA: Question Answering over Structured Knowledge for E-commerce Customer Service",
            "authors": [
                "Feng-Lin Li",
                "Weijia Chen",
                "Qi Huang",
                "Yikun Guo"
            ],
            "abstract": "With the rise of knowledge graph (KG), question answering over knowledge base\n(KBQA) has attracted increasing attention in recent years. Despite much\nresearch has been conducted on this topic, it is still challenging to apply\nKBQA technology in industry because business knowledge and real-world questions\ncan be rather complicated. In this paper, we present AliMe-KBQA, a bold attempt\nto apply KBQA in the E-commerce customer service field. To handle real\nknowledge and questions, we extend the classic \"subject-predicate-object (SPO)\"\nstructure with property hierarchy, key-value structure and compound value type\n(CVT), and enhance traditional KBQA with constraints recognition and reasoning\nability. We launch AliMe-KBQA in the Marketing Promotion scenario for merchants\nduring the \"Double 11\" period in 2018 and other such promotional events\nafterwards. Online results suggest that AliMe-KBQA is not only able to gain\nbetter resolution and improve customer satisfaction, but also becomes the\npreferred knowledge management method by business knowledge staffs since it\noffers a more convenient and efficient management experience.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.05877v2",
            "date": "2019-12-12 11:02:30+00:00",
            "title": "Extending Machine Language Models toward Human-Level Language Understanding",
            "authors": [
                "James L. McClelland",
                "Felix Hill",
                "Maja Rudolph",
                "Jason Baldridge",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Language is crucial for human intelligence, but what exactly is its role? We\ntake language to be a part of a system for understanding and communicating\nabout situations. The human ability to understand and communicate about\nsituations emerges gradually from experience and depends on domain-general\nprinciples of biological neural networks: connection-based learning,\ndistributed representation, and context-sensitive, mutual constraint\nsatisfaction-based processing. Current artificial language processing systems\nrely on the same domain general principles, embodied in artificial neural\nnetworks. Indeed, recent progress in this field depends on \\emph{query-based\nattention}, which extends the ability of these systems to exploit context and\nhas contributed to remarkable breakthroughs. Nevertheless, most current models\nfocus exclusively on language-internal tasks, limiting their ability to perform\ntasks that depend on understanding situations. These systems also lack memory\nfor the contents of prior situations outside of a fixed contextual span. We\ndescribe the organization of the brain's distributed understanding system,\nwhich includes a fast learning system that addresses the memory problem. We\nsketch a framework for future models of understanding drawing equally on\ncognitive neuroscience and artificial intelligence and exploiting query-based\nattention. We highlight relevant current directions and consider further\ndevelopments needed to fully capture human-level language understanding in a\ncomputational system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05957v2",
            "date": "2019-12-12 13:54:09+00:00",
            "title": "Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model",
            "authors": [
                "Hamid Mohammadi",
                "Seyed Hossein Khasteh"
            ],
            "abstract": "Evaluating the readability of a text can significantly facilitate the precise\nexpression of information in a written form. The formulation of text\nreadability assessment demands the identification of meaningful properties of\nthe text and correct conversion of features to the right readability level.\nSophisticated features and models are being used to evaluate the\ncomprehensibility of texts accurately. Still, these models are challenging to\nimplement, heavily language-dependent, and do not perform well on short texts.\nDeep reinforcement learning models are demonstrated to be helpful in further\nimprovement of state-of-the-art text readability assessment models. The main\ncontributions of the proposed approach are the automation of feature\nextraction, loosening the tight language dependency of text readability\nassessment task, and efficient use of text by finding the minimum portion of a\ntext required to assess its readability. The experiments on Weebit, Cambridge\nExams, and Persian readability datasets display the model's state-of-the-art\nprecision, efficiency, and the capability to be applied to other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06208v1",
            "date": "2019-12-12 20:56:59+00:00",
            "title": "Shaping representations through communication: community size effect in artificial learning systems",
            "authors": [
                "Olivier Tieleman",
                "Angeliki Lazaridou",
                "Shibl Mourad",
                "Charles Blundell",
                "Doina Precup"
            ],
            "abstract": "Motivated by theories of language and communication that explain why\ncommunities with large numbers of speakers have, on average, simpler languages\nwith more regularity, we cast the representation learning problem in terms of\nlearning to communicate. Our starting point sees the traditional autoencoder\nsetup as a single encoder with a fixed decoder partner that must learn to\ncommunicate. Generalizing from there, we introduce community-based autoencoders\nin which multiple encoders and decoders collectively learn representations by\nbeing randomly paired up on successive training iterations. We find that\nincreasing community sizes reduce idiosyncrasies in the learned codes,\nresulting in representations that better encode concept categories and\ncorrelate with human feature norms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06638v2",
            "date": "2019-12-13 18:15:37+00:00",
            "title": "WaLDORf: Wasteless Language-model Distillation On Reading-comprehension",
            "authors": [
                "James Yi Tian",
                "Alexander P. Kreuzer",
                "Pai-Hung Chen",
                "Hans-Martin Will"
            ],
            "abstract": "Transformer based Very Large Language Models (VLLMs) like BERT, XLNet and\nRoBERTa, have recently shown tremendous performance on a large variety of\nNatural Language Understanding (NLU) tasks. However, due to their size, these\nVLLMs are extremely resource intensive and cumbersome to deploy at production\ntime. Several recent publications have looked into various ways to distil\nknowledge from a transformer based VLLM (most commonly BERT-Base) into a\nsmaller model which can run much faster at inference time. Here, we propose a\nnovel set of techniques which together produce a task-specific hybrid\nconvolutional and transformer model, WaLDORf, that achieves state-of-the-art\ninference speed while still being more accurate than previous distilled models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.06670v2",
            "date": "2019-12-13 19:22:44+00:00",
            "title": "Common Voice: A Massively-Multilingual Speech Corpus",
            "authors": [
                "Rosana Ardila",
                "Megan Branson",
                "Kelly Davis",
                "Michael Henretty",
                "Michael Kohler",
                "Josh Meyer",
                "Reuben Morais",
                "Lindsay Saunders",
                "Francis M. Tyers",
                "Gregor Weber"
            ],
            "abstract": "The Common Voice corpus is a massively-multilingual collection of transcribed\nspeech intended for speech technology research and development. Common Voice is\ndesigned for Automatic Speech Recognition purposes but can be useful in other\ndomains (e.g. language identification). To achieve scale and sustainability,\nthe Common Voice project employs crowdsourcing for both data collection and\ndata validation. The most recent release includes 29 languages, and as of\nNovember 2019 there are a total of 38 languages collecting data. Over 50,000\nindividuals have participated so far, resulting in 2,500 hours of collected\naudio. To our knowledge this is the largest audio corpus in the public domain\nfor speech recognition, both in terms of number of hours and number of\nlanguages. As an example use case for Common Voice, we present speech\nrecognition experiments using Mozilla's DeepSpeech Speech-to-Text toolkit. By\napplying transfer learning from a source English model, we find an average\nCharacter Error Rate improvement of 5.99 +/- 5.48 for twelve target languages\n(German, French, Italian, Turkish, Catalan, Slovenian, Welsh, Irish, Breton,\nTatar, Chuvash, and Kabyle). For most of these languages, these are the first\never published results on end-to-end Automatic Speech Recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06719v2",
            "date": "2019-12-13 21:41:39+00:00",
            "title": "Neural Network Surgery with Sets",
            "authors": [
                "Jonathan Raiman",
                "Susan Zhang",
                "Christy Dennison"
            ],
            "abstract": "The cost to train machine learning models has been increasing exponentially,\nmaking exploration and research into the correct features and architecture a\ncostly or intractable endeavor at scale. However, using a technique named\n\"surgery\" OpenAI Five was continuously trained to play the game DotA 2 over the\ncourse of 10 months through 20 major changes in features and architecture.\nSurgery transfers trained weights from one network to another after a selection\nprocess to determine which sections of the model are unchanged and which must\nbe re-initialized. In the past, the selection process relied on heuristics,\nmanual labor, or pre-existing boundaries in the structure of the model,\nlimiting the ability to salvage experiments after modifications of the feature\nset or input reorderings.\n  We propose a solution to automatically determine which components of a neural\nnetwork model should be salvaged and which require retraining. We achieve this\nby allowing the model to operate over discrete sets of features and use\nset-based operations to determine the exact relationship between inputs and\noutputs, and how they change across tweaks in model architecture. In this\npaper, we introduce the methodology for enabling neural networks to operate on\nsets, derive two methods for detecting feature-parameter interaction maps, and\nshow their equivalence. We empirically validate that we can surgery weights\nacross feature and architecture changes to the OpenAI Five model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06721v1",
            "date": "2019-12-13 21:49:30+00:00",
            "title": "Long-Term Planning and Situational Awareness in OpenAI Five",
            "authors": [
                "Jonathan Raiman",
                "Susan Zhang",
                "Filip Wolski"
            ],
            "abstract": "Understanding how knowledge about the world is represented within model-free\ndeep reinforcement learning methods is a major challenge given the black box\nnature of its learning process within high-dimensional observation and action\nspaces. AlphaStar and OpenAI Five have shown that agents can be trained without\nany explicit hierarchical macro-actions to reach superhuman skill in games that\nrequire taking thousands of actions before reaching the final goal. Assessing\nthe agent's plans and game understanding becomes challenging given the lack of\nhierarchy or explicit representations of macro-actions in these models, coupled\nwith the incomprehensible nature of the internal representations.\n  In this paper, we study the distributed representations learned by OpenAI\nFive to investigate how game knowledge is gradually obtained over the course of\ntraining. We also introduce a general technique for learning a model from the\nagent's hidden states to identify the formation of plans and subgoals. We show\nthat the agent can learn situational similarity across actions, and find\nevidence of planning towards accomplishing subgoals minutes before they are\nexecuted. We perform a qualitative analysis of these predictions during the\ngames against the DotA 2 world champions OG in April 2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06825v2",
            "date": "2019-12-14 11:02:17+00:00",
            "title": "Knowledge forest: a novel model to organize knowledge fragments",
            "authors": [
                "Qinghua Zheng",
                "Jun Liu",
                "Hongwei Zeng",
                "Zhaotong Guo",
                "Bei Wu",
                "Bifan Wei"
            ],
            "abstract": "With the rapid growth of knowledge, it shows a steady trend of knowledge\nfragmentization. Knowledge fragmentization manifests as that the knowledge\nrelated to a specific topic in a course is scattered in isolated and autonomous\nknowledge sources. We term the knowledge of a facet in a specific topic as a\nknowledge fragment. The problem of knowledge fragmentization brings two\nchallenges: First, knowledge is scattered in various knowledge sources, which\nexerts users' considerable efforts to search for the knowledge of their\ninterested topics, thereby leading to information overload. Second, learning\ndependencies which refer to the precedence relationships between topics in the\nlearning process are concealed by the isolation and autonomy of knowledge\nsources, thus causing learning disorientation. To solve the knowledge\nfragmentization problem, we propose a novel knowledge organization model,\nknowledge forest, which consists of facet trees and learning dependencies.\nFacet trees can organize knowledge fragments with facet hyponymy to alleviate\ninformation overload. Learning dependencies can organize disordered topics to\ncope with learning disorientation. We conduct extensive experiments on three\nmanually constructed datasets from the Data Structure, Data Mining, and\nComputer Network courses, and the experimental results show that knowledge\nforest can effectively organize knowledge fragments, and alleviate information\noverload and learning disorientation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.06858v1",
            "date": "2019-12-14 14:55:59+00:00",
            "title": "LScDC-new large scientific dictionary",
            "authors": [
                "Neslihan Suzen",
                "Evgeny M. Mirkes",
                "Alexander N. Gorban"
            ],
            "abstract": "In this paper, we present a scientific corpus of abstracts of academic papers\nin English -- Leicester Scientific Corpus (LSC). The LSC contains 1,673,824\nabstracts of research articles and proceeding papers indexed by Web of Science\n(WoS) in which publication year is 2014. Each abstract is assigned to at least\none of 252 subject categories. Paper metadata include these categories and the\nnumber of citations. We then develop scientific dictionaries named Leicester\nScientific Dictionary (LScD) and Leicester Scientific Dictionary-Core (LScDC),\nwhere words are extracted from the LSC. The LScD is a list of 974,238 unique\nwords (lemmas). The LScDC is a core list (sub-list) of the LScD with 104,223\nlemmas. It was created by removing LScD words appearing in not greater than 10\ntexts in the LSC. LScD and LScDC are available online. Both the corpus and\ndictionaries are developed to be later used for quantification of meaning in\nacademic texts.\n  Finally, the core list LScDC was analysed by comparing its words and word\nfrequencies with a classic academic word list 'New Academic Word List (NAWL)'\ncontaining 963 word families, which is also sampled from an academic corpus.\nThe major sources of the corpus where NAWL is extracted are Cambridge English\nCorpus (CEC), oral sources and textbooks. We investigate whether two\ndictionaries are similar in terms of common words and ranking of words. Our\ncomparison leads us to main conclusion: most of words of NAWL (99.6%) are\npresent in the LScDC but two lists differ in word ranking. This difference is\nmeasured.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06859v1",
            "date": "2019-12-14 14:59:38+00:00",
            "title": "Knowledge-based Conversational Search",
            "authors": [
                "Svitlana Vakulenko"
            ],
            "abstract": "Conversational interfaces that allow for intuitive and comprehensive access\nto digitally stored information remain an ambitious goal. In this thesis, we\nlay foundations for designing conversational search systems by analyzing the\nrequirements and proposing concrete solutions for automating some of the basic\ncomponents and tasks that such systems should support. We describe several\ninterdependent studies that were conducted to analyse the design requirements\nfor more advanced conversational search systems able to support complex\nhuman-like dialogue interactions and provide access to vast knowledge\nrepositories. In the first two research chapters, we focus on analyzing the\nstructures common to information-seeking dialogues by capturing recurrent\npatterns in terms of both domain-independent functional relations between\nutterances as well as domain-specific implicit semantic relations from shared\nbackground knowledge.\n  Our results show that question answering is one of the key components\nrequired for efficient information access but it is not the only type of\ndialogue interactions that a conversational search system should support. In\nthe third research chapter, we propose a novel approach for complex question\nanswering from a knowledge graph that surpasses the current state-of-the-art\nresults in terms of both efficacy and efficiency. In the last research chapter,\nwe turn our attention towards an alternative interaction mode, which we termed\nconversational browsing, in which, unlike question answering, the\nconversational system plays a more pro-active role in the course of a dialogue\ninteraction. We show that this approach helps users to discover relevant items\nthat are difficult to retrieve using only question answering due to the\nvocabulary mismatch problem.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.06889v2",
            "date": "2019-12-14 17:38:46+00:00",
            "title": "Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting",
            "authors": [
                "Aakash Srinivasan",
                "Harshavardhan Kamarthi",
                "Devi Ganesan",
                "Sutanu Chakraborti"
            ],
            "abstract": "Neural network based word embeddings, such as Word2Vec and GloVe, are purely\ndata driven in that they capture the distributional information about words\nfrom the training corpus. Past works have attempted to improve these embeddings\nby incorporating semantic knowledge from lexical resources like WordNet. Some\ntechniques like retrofitting modify word embeddings in the post-processing\nstage while some others use a joint learning approach by modifying the\nobjective function of neural networks. In this paper, we discuss two novel\napproaches for incorporating semantic knowledge into word embeddings. In the\nfirst approach, we take advantage of Levy et al's work which showed that using\nSVD based methods on co-occurrence matrix provide similar performance to neural\nnetwork based embeddings. We propose a 'sprinkling' technique to add semantic\nrelations to the co-occurrence matrix directly before factorization. In the\nsecond approach, WordNet similarity scores are used to improve the retrofitting\nmethod. We evaluate the proposed methods in both intrinsic and extrinsic tasks\nand observe significant improvements over the baselines in many of the\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06927v2",
            "date": "2019-12-14 20:57:29+00:00",
            "title": "#MeTooMA: Multi-Aspect Annotations of Tweets Related to the MeToo Movement",
            "authors": [
                "Akash Gautam",
                "Puneet Mathur",
                "Rakesh Gosangi",
                "Debanjan Mahata",
                "Ramit Sawhney",
                "Rajiv Ratn Shah"
            ],
            "abstract": "In this paper, we present a dataset containing 9,973 tweets related to the\nMeToo movement that were manually annotated for five different linguistic\naspects: relevance, stance, hate speech, sarcasm, and dialogue acts. We present\na detailed account of the data collection and annotation processes. The\nannotations have a very high inter-annotator agreement (0.79 to 0.93 k-alpha)\ndue to the domain expertise of the annotators and clear annotation\ninstructions. We analyze the data in terms of geographical distribution, label\ncorrelations, and keywords. Lastly, we present some potential use cases of this\ndataset. We expect this dataset would be of great interest to psycholinguists,\nsocio-linguists, and computational linguists to study the discursive space of\ndigitally mobilized social movements on sensitive issues like sexual\nharassment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07025v1",
            "date": "2019-12-15 11:42:27+00:00",
            "title": "Indiscapes: Instance Segmentation Networks for Layout Parsing of Historical Indic Manuscripts",
            "authors": [
                "Abhishek Prusty",
                "Sowmya Aitha",
                "Abhishek Trivedi",
                "Ravi Kiran Sarvadevabhatla"
            ],
            "abstract": "Historical palm-leaf manuscript and early paper documents from Indian\nsubcontinent form an important part of the world's literary and cultural\nheritage. Despite their importance, large-scale annotated Indic manuscript\nimage datasets do not exist. To address this deficiency, we introduce\nIndiscapes, the first ever dataset with multi-regional layout annotations for\nhistorical Indic manuscripts. To address the challenge of large diversity in\nscripts and presence of dense, irregular layout elements (e.g. text lines,\npictures, multiple documents per image), we adapt a Fully Convolutional Deep\nNeural Network architecture for fully automatic, instance-level spatial layout\nparsing of manuscript images. We demonstrate the effectiveness of proposed\narchitecture on images from the Indiscapes dataset. For annotation flexibility\nand keeping the non-technical nature of domain experts in mind, we also\ncontribute a custom, web-based GUI annotation tool and a dashboard-style\nanalytics portal. Overall, our contributions set the stage for enabling\ndownstream applications such as OCR and word-spotting in historical Indic\nmanuscripts at scale.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.07240v1",
            "date": "2019-12-16 08:22:43+00:00",
            "title": "Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding",
            "authors": [
                "Yuchen Liu",
                "Jiajun Zhang",
                "Hao Xiong",
                "Long Zhou",
                "Zhongjun He",
                "Hua Wu",
                "Haifeng Wang",
                "Chengqing Zong"
            ],
            "abstract": "Speech-to-text translation (ST), which translates source language speech into\ntarget language text, has attracted intensive attention in recent years.\nCompared to the traditional pipeline system, the end-to-end ST model has\npotential benefits of lower latency, smaller model size, and less error\npropagation. However, it is notoriously difficult to implement such a model\nwithout transcriptions as intermediate. Existing works generally apply\nmulti-task learning to improve translation quality by jointly training\nend-to-end ST along with automatic speech recognition (ASR). However, different\ntasks in this method cannot utilize information from each other, which limits\nthe improvement. Other works propose a two-stage model where the second model\ncan use the hidden state from the first one, but its cascade manner greatly\naffects the efficiency of training and inference process. In this paper, we\npropose a novel interactive attention mechanism which enables ASR and ST to\nperform synchronously and interactively in a single model. Specifically, the\ngeneration of transcriptions and translations not only relies on its previous\noutputs but also the outputs predicted in the other task. Experiments on TED\nspeech translation corpora have shown that our proposed model can outperform\nstrong baselines on the quality of speech translation and achieve better speech\nrecognition performances as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07419v1",
            "date": "2019-12-16 14:43:11+00:00",
            "title": "Optimized Tracking of Topic Evolution",
            "authors": [
                "Patrick Kiss",
                "Elaheh Momeni"
            ],
            "abstract": "Topic evolution modeling has been researched for a long time and has gained\nconsiderable interest. A state-of-the-art method has been recently using word\nmodeling algorithms in combination with community detection mechanisms to\nachieve better results in a more effective way. We analyse results of this\napproach and discuss the two major challenges that this approach still faces.\nAlthough the topics that have resulted from the recent algorithm are good in\ngeneral, they are very noisy due to many topics that are very unimportant\nbecause of their size, words, or ambiguity. Additionally, the number of words\ndefining each topic is too large, making it difficult to analyse them in their\nunsorted state. In this paper, we propose approaches to tackle these challenges\nby adding topic filtering and network analysis metrics to define the importance\nof a topic. We test different combinations of these metrics to see which\ncombination yields the best results. Furthermore, we add word filtering and\nranking to each topic to identify the words with the highest novelty\nautomatically. We evaluate our enhancement methods in two ways: human\nqualitative evaluation and automatic quantitative evaluation. Moreover, we\ncreated two case studies to test the quality of the clusters and words. In the\nquantitative evaluation, we use the pairwise mutual information score to test\nthe coherency of topics. The quantitative evaluation also includes an analysis\nof execution times for each part of the program. The results of the\nexperimental evaluations show that the two evaluation methods agree on the\npositive feasibility of the algorithm. We then show possible extensions in the\nform of usability and future improvements to the algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07575v2",
            "date": "2019-12-16 18:41:49+00:00",
            "title": "Predicting detection filters for small footprint open-vocabulary keyword spotting",
            "authors": [
                "Theodore Bluche",
                "Thibault Gisselbrecht"
            ],
            "abstract": "In this paper, we propose a fully-neural approach to open-vocabulary keyword\nspotting, that allows the users to include a customizable voice interface to\ntheir device and that does not require task-specific data. We present a keyword\ndetection neural network weighing less than 250KB, in which the topmost layer\nperforming keyword detection is predicted by an auxiliary network, that may be\nrun offline to generate a detector for any keyword. We show that the proposed\nmodel outperforms acoustic keyword spotting baselines by a large margin on two\ntasks of detecting keywords in utterances and three tasks of detecting isolated\nspeech commands. We also propose a method to fine-tune the model when specific\ntraining data is available for some keywords, which yields a performance\nsimilar to a standard speech command neural network while keeping the ability\nof the model to be applied to new keywords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07976v3",
            "date": "2019-12-17 12:47:33+00:00",
            "title": "A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction",
            "authors": [
                "Heng Yang",
                "Biqing Zeng",
                "JianHao Yang",
                "Youwei Song",
                "Ruyang Xu"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) task is a multi-grained task of\nnatural language processing and consists of two subtasks: aspect term\nextraction (ATE) and aspect polarity classification (APC). Most of the existing\nwork focuses on the subtask of aspect term polarity inferring and ignores the\nsignificance of aspect term extraction. Besides, the existing researches do not\npay attention to the research of the Chinese-oriented ABSA task. Based on the\nlocal context focus (LCF) mechanism, this paper firstly proposes a multi-task\nlearning model for Chinese-oriented aspect-based sentiment analysis, namely\nLCF-ATEPC. Compared with existing models, this model equips the capability of\nextracting aspect term and inferring aspect term polarity synchronously,\nmoreover, this model is effective to analyze both Chinese and English comments\nsimultaneously and the experiment on a multilingual mixed dataset proved its\navailability. By integrating the domain-adapted BERT model, the LCF-ATEPC model\nachieved the state-of-the-art performance of aspect term extraction and aspect\npolarity classification in four Chinese review datasets. Besides, the\nexperimental results on the most commonly used SemEval-2014 task4 Restaurant\nand Laptop datasets outperform the state-of-the-art performance on the ATE and\nAPC subtask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08226v2",
            "date": "2019-12-17 19:03:23+00:00",
            "title": "Meshed-Memory Transformer for Image Captioning",
            "authors": [
                "Marcella Cornia",
                "Matteo Stefanini",
                "Lorenzo Baraldi",
                "Rita Cucchiara"
            ],
            "abstract": "Transformer-based architectures represent the state of the art in sequence\nmodeling tasks like machine translation and language understanding. Their\napplicability to multi-modal contexts like image captioning, however, is still\nlargely under-explored. With the aim of filling this gap, we present M$^2$ - a\nMeshed Transformer with Memory for Image Captioning. The architecture improves\nboth the image encoding and the language generation steps: it learns a\nmulti-level representation of the relationships between image regions\nintegrating learned a priori knowledge, and uses a mesh-like connectivity at\ndecoding stage to exploit low- and high-level features. Experimentally, we\ninvestigate the performance of the M$^2$ Transformer and different\nfully-attentive models in comparison with recurrent ones. When tested on COCO,\nour proposal achieves a new state of the art in single-model and ensemble\nconfigurations on the \"Karpathy\" test split and on the online test server. We\nalso assess its performances when describing objects unseen in the training\nset. Trained models and code for reproducing the experiments are publicly\navailable at: https://github.com/aimagelab/meshed-memory-transformer.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.08374v3",
            "date": "2019-12-18 04:32:18+00:00",
            "title": "Uncovering Relations for Marketing Knowledge Representation",
            "authors": [
                "Somak Aditya",
                "Atanu Sinha"
            ],
            "abstract": "Online behaviors of consumers and marketers generate massive marketing data,\nwhich ever more sophisticated models attempt to turn into insights and aid\ndecisions by marketers. Yet, in making decisions human managers bring to bear\nmarketing knowledge which reside outside of data and models. Thus, it behooves\ncreation of an automated marketing knowledge base that can interact with data\nand models. Currently, marketing knowledge is dispersed in large corpora, but\nno definitive knowledge base for marketing exists. Out of the two broad aspects\nof marketing knowledge - representation and reasoning - this treatise focuses\non the former. Specifically, we focus on creation of marketing knowledge graph\nfrom corpora, which requires identification of entities and relations. The\nrelation identification task is particularly challenging in marketing, because\nof the non-factoid nature of much marketing knowledge, and the difficulty of\nforming rules that govern relations. Specifically, we define a set of relations\nto capture marketing knowledge, propose a pipeline for creating the knowledge\ngraph from text and propose a rule-guided semi-supervised relation prediction\nalgorithm to extract relations between marketing entities from sentences.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.08582v1",
            "date": "2019-12-18 13:13:29+00:00",
            "title": "Towards an automatic recognition of mixed languages: The Ukrainian-Russian hybrid language Surzhyk",
            "authors": [
                "Nataliya Sira",
                "Giorgio Maria Di Nunzio",
                "Viviana Nosilia"
            ],
            "abstract": "Language interference is common in today's multilingual societies where more\nlanguages are being in contact and as a global final result leads to the\ncreation of hybrid languages. These, together with doubts on their right to be\nofficially recognised made emerge in the area of computational linguistics the\nproblem of their automatic identification and further elaboration. In this\npaper, we propose a first attempt to identify the elements of a\nUkrainian-Russian hybrid language, Surzhyk, through the adoption of the\nexample-based rules created with the instruments of programming language R. Our\nexample-based study consists of: 1) analysis of spoken samples of Surzhyk\nregistered by Del Gaudio (2010) in Kyiv area and creation of the written\ncorpus; 2) production of specific rules on the identification of Surzhyk\npatterns and their implementation; 3) testing the code and analysing the\neffectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09156v1",
            "date": "2019-12-19 12:17:38+00:00",
            "title": "CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension",
            "authors": [
                "Xingyi Duan",
                "Baoxin Wang",
                "Ziyue Wang",
                "Wentao Ma",
                "Yiming Cui",
                "Dayong Wu",
                "Shijin Wang",
                "Ting Liu",
                "Tianxiang Huo",
                "Zhen Hu",
                "Heng Wang",
                "Zhiyuan Liu"
            ],
            "abstract": "We present a Chinese judicial reading comprehension (CJRC) dataset which\ncontains approximately 10K documents and almost 50K questions with answers. The\ndocuments come from judgment documents and the questions are annotated by law\nexperts. The CJRC dataset can help researchers extract elements by reading\ncomprehension technology. Element extraction is an important task in the legal\nfield. However, it is difficult to predefine the element types completely due\nto the diversity of document types and causes of action. By contrast, machine\nreading comprehension technology can quickly extract elements by answering\nvarious questions from the long document. We build two strong baseline models\nbased on BERT and BiDAF. The experimental results show that there is enough\nspace for improvement compared to human annotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09253v3",
            "date": "2019-12-19 15:04:24+00:00",
            "title": "Towards a Philological Metric through a Topological Data Analysis Approach",
            "authors": [
                "Eduardo Paluzo-Hidalgo",
                "Rocio Gonzalez-Diaz",
                "Miguel A. Guti\u00e9rrez-Naranjo"
            ],
            "abstract": "The canon of the baroque Spanish literature has been thoroughly studied with\nphilological techniques. The major representatives of the poetry of this epoch\nare Francisco de Quevedo and Luis de G\\'ongora y Argote. They are commonly\nclassified by the literary experts in two different streams: Quevedo belongs to\nthe Conceptismo and G\\'ongora to the Culteranismo. Besides, traditionally, even\nif Quevedo is considered the most representative of the Conceptismo, Lope de\nVega is also considered to be, at least, closely related to this literary\ntrend. In this paper, we use Topological Data Analysis techniques to provide a\nfirst approach to a metric distance between the literary style of these poets.\nAs a consequence, we reach results that are under the literary experts'\ncriteria, locating the literary style of Lope de Vega, closer to the one of\nQuevedo than to the one of G\\'ongora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10163v1",
            "date": "2019-12-06 09:57:22+00:00",
            "title": "Can AI Generate Love Advice?: Toward Neural Answer Generation for Non-Factoid Questions",
            "authors": [
                "Makoto Nakatsuji"
            ],
            "abstract": "Deep learning methods that extract answers for non-factoid questions from QA\nsites are seen as critical since they can assist users in reaching their next\ndecisions through conversations with AI systems. The current methods, however,\nhave the following two problems: (1) They can not understand the ambiguous use\nof words in the questions as word usage can strongly depend on the context. As\na result, the accuracies of their answer selections are not good enough. (2)\nThe current methods can only select from among the answers held by QA sites and\ncan not generate new ones. Thus, they can not answer the questions that are\nsomewhat different with those stored in QA sites. Our solution, Neural Answer\nConstruction Model, tackles these problems as it: (1) Incorporates the biases\nof semantics behind questions into word embeddings while also computing them\nregardless of the semantics. As a result, it can extract answers that suit the\ncontexts of words used in the question as well as following the common usage of\nwords across semantics. This improves the accuracy of answer selection. (2)\nUses biLSTM to compute the embeddings of questions as well as those of the\nsentences often used to form answers. It then simultaneously learns the optimum\ncombination of those sentences as well as the closeness between the question\nand those sentences. As a result, our model can construct an answer that\ncorresponds to the situation that underlies the question; it fills the gap\nbetween answer selection and generation and is the first model to move beyond\nthe current simple answer selection model for non-factoid QAs. Evaluations\nusing datasets created for love advice stored in the Japanese QA site, Oshiete\ngoo, indicate that our model achieves 20% higher accuracy in answer creation\nthan the strong baselines. Our model is practical and has already been applied\nto the love advice service in Oshiete goo.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10167v2",
            "date": "2019-12-10 19:50:35+00:00",
            "title": "Machine Translation with Cross-lingual Word Embeddings",
            "authors": [
                "Marco Berlot",
                "Evan Kaplan"
            ],
            "abstract": "Learning word embeddings using distributional information is a task that has\nbeen studied by many researchers, and a lot of studies are reported in the\nliterature. On the contrary, less studies were done for the case of multiple\nlanguages. The idea is to focus on a single representation for a pair of\nlanguages such that semantically similar words are closer to one another in the\ninduced representation irrespective of the language. In this way, when data are\nmissing for a particular language, classifiers from another language can be\nused.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10308v1",
            "date": "2019-12-21 18:14:32+00:00",
            "title": "Candidate Fusion: Integrating Language Modelling into a Sequence-to-Sequence Handwritten Word Recognition Architecture",
            "authors": [
                "Lei Kang",
                "Pau Riba",
                "Mauricio Villegas",
                "Alicia Forn\u00e9s",
                "Mar\u00e7al Rusi\u00f1ol"
            ],
            "abstract": "Sequence-to-sequence models have recently become very popular for tackling\nhandwritten word recognition problems. However, how to effectively integrate an\nexternal language model into such recognizer is still a challenging problem.\nThe main challenge faced when training a language model is to deal with the\nlanguage model corpus which is usually different to the one used for training\nthe handwritten word recognition system. Thus, the bias between both word\ncorpora leads to incorrectness on the transcriptions, providing similar or even\nworse performances on the recognition task. In this work, we introduce\nCandidate Fusion, a novel way to integrate an external language model to a\nsequence-to-sequence architecture. Moreover, it provides suggestions from an\nexternal language knowledge, as a new input to the sequence-to-sequence\nrecognizer. Hence, Candidate Fusion provides two improvements. On the one hand,\nthe sequence-to-sequence recognizer has the flexibility not only to combine the\ninformation from itself and the language model, but also to choose the\nimportance of the information provided by the language model. On the other\nhand, the external language model has the ability to adapt itself to the\ntraining corpus and even learn the most commonly errors produced from the\nrecognizer. Finally, by conducting comprehensive experiments, the Candidate\nFusion proves to outperform the state-of-the-art language models for\nhandwritten word recognition tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.10375v2",
            "date": "2019-12-22 03:02:42+00:00",
            "title": "T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack",
            "authors": [
                "Boxin Wang",
                "Hengzhi Pei",
                "Boyuan Pan",
                "Qian Chen",
                "Shuohang Wang",
                "Bo Li"
            ],
            "abstract": "Adversarial attacks against natural language processing systems, which\nperform seemingly innocuous modifications to inputs, can induce arbitrary\nmistakes to the target models. Though raised great concerns, such adversarial\nattacks can be leveraged to estimate the robustness of NLP models. Compared\nwith the adversarial example generation in continuous data domain (e.g.,\nimage), generating adversarial text that preserves the original meaning is\nchallenging since the text space is discrete and non-differentiable. To handle\nthese challenges, we propose a target-controllable adversarial attack framework\nT3, which is applicable to a range of NLP tasks. In particular, we propose a\ntree-based autoencoder to embed the discrete text data into a continuous\nrepresentation space, upon which we optimize the adversarial perturbation. A\nnovel tree-based decoder is then applied to regularize the syntactic\ncorrectness of the generated text and manipulate it on either sentence\n(T3(Sent)) or word (T3(Word)) level. We consider two most representative NLP\ntasks: sentiment analysis and question answering (QA). Extensive experimental\nresults and human studies show that T3 generated adversarial texts can\nsuccessfully manipulate the NLP models to output the targeted incorrect answer\nwithout misleading the human. Moreover, we show that the generated adversarial\ntexts have high transferability which enables the black-box attacks in\npractice. Our work sheds light on an effective and general way to examine the\nrobustness of NLP models. Our code is publicly available at\nhttps://github.com/AI-secure/T3/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10434v1",
            "date": "2019-12-17 09:01:30+00:00",
            "title": "Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings",
            "authors": [
                "Andreas Hanselowski",
                "Iryna Gurevych"
            ],
            "abstract": "Word embeddings are rich word representations, which in combination with deep\nneural networks, lead to large performance gains for many NLP tasks. However,\nword embeddings are represented by dense, real-valued vectors and they are\ntherefore not directly interpretable. Thus, computational operations based on\nthem are also not well understood. In this paper, we present an approach for\nanalyzing structures in the semantic vector space to get a better understanding\nof the underlying semantic encoding principles. We present a framework for\ndecomposing word embeddings into smaller meaningful units which we call\nsub-vectors. The framework opens up a wide range of possibilities analyzing\nphenomena in vector space semantics, as well as solving concrete NLP problems:\nWe introduce the category completion task and show that a sub-vector based\napproach is superior to supervised techniques; We present a sub-vector based\nmethod for solving the word analogy task, which substantially outperforms\ndifferent variants of the traditional vector-offset method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10435v1",
            "date": "2019-12-14 06:44:12+00:00",
            "title": "BERTQA -- Attention on Steroids",
            "authors": [
                "Ankit Chadha",
                "Rewa Sood"
            ],
            "abstract": "In this work, we extend the Bidirectional Encoder Representations from\nTransformers (BERT) with an emphasis on directed coattention to obtain an\nimproved F1 performance on the SQUAD2.0 dataset. The Transformer architecture\non which BERT is based places hierarchical global attention on the\nconcatenation of the context and query. Our additions to the BERT architecture\naugment this attention with a more focused context to query (C2Q) and query to\ncontext (Q2C) attention via a set of modified Transformer encoder units. In\naddition, we explore adding convolution-based feature extraction within the\ncoattention architecture to add localized information to self-attention. We\nfound that coattention significantly improves the no answer F1 by 4 points in\nthe base and 1 point in the large architecture. After adding skip connections\nthe no answer F1 improved further without causing an additional loss in has\nanswer F1. The addition of localized feature extraction added to attention\nproduced an overall dev F1 of 77.03 in the base architecture. We applied our\nfindings to the large BERT model which contains twice as many layers and\nfurther used our own augmented version of the SQUAD 2.0 dataset created by back\ntranslation, which we have named SQUAD 2.Q. Finally, we performed\nhyperparameter tuning and ensembled our best models for a final F1/EM of\n82.317/79.442 (Attention on Steroids, PCE Test Leaderboard).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10546v1",
            "date": "2019-12-22 21:47:05+00:00",
            "title": "Hybrid Machine Learning Models of Classifying Residential Requests for Smart Dispatching",
            "authors": [
                "T. Chen",
                "J. Sun",
                "H. Lin",
                "Y. Liu"
            ],
            "abstract": "This paper presents a hybrid machine learning method of classifying\nresidential requests in natural language to responsible departments that\nprovide timely responses back to residents under the vision of digital\ngovernment services in smart cities. Residential requests in natural language\ndescriptions cover almost every aspect of a city's daily operation. Hence the\nresponsible departments are fine-grained to even the level of local\ncommunities. There are no specific general categories or labels for each\nrequest sample. This causes two issues for supervised classification solutions,\nnamely (1) the request sample data is unbalanced and (2) lack of specific\nlabels for training. To solve these issues, we investigate a hybrid machine\nlearning method that generates meta-class labels by means of unsupervised\nclustering algorithms; applies two-word embedding methods with three\nclassifiers (including two hierarchical classifiers and one residual\nconvolutional neural network); and selects the best performing classifier as\nthe classification result. We demonstrate our approach performing better\nclassification tasks compared to two benchmarking machine learning models,\nNaive Bayes classifier and a Multiple Layer Perceptron (MLP). In addition, the\nhierarchical classification method provides insights into the source of\nclassification errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10785v2",
            "date": "2019-12-23 13:14:40+00:00",
            "title": "Discovering Protagonist of Sentiment with Aspect Reconstructed Capsule Network",
            "authors": [
                "Chi Xu",
                "Hao Feng",
                "Guoxin Yu",
                "Min Yang",
                "Xiting Wang",
                "Xiang Ao"
            ],
            "abstract": "Most recent existing aspect-term level sentiment analysis (ATSA) approaches\ncombined various neural network models with delicately carved attention\nmechanisms built upon given aspect and context to generate refined sentence\nrepresentations for better predictions. In these methods, aspect terms are\nalways provided in both training and testing process which may degrade\naspect-level analysis into sentence-level prediction. However, the annotated\naspect term might be unavailable in real-world scenarios which may challenge\nthe applicability of the existing methods. In this paper, we aim to improve\nATSA by discovering the potential aspect terms of the predicted sentiment\npolarity when the aspect terms of a test sentence are unknown. We access this\ngoal by proposing a capsule network based model named CAPSAR. In CAPSAR,\nsentiment categories are denoted by capsules and aspect term information is\ninjected into sentiment capsules through a sentiment-aspect reconstruction\nprocedure during the training. As a result, coherent patterns between aspects\nand sentimental expressions are encapsulated by these sentiment capsules.\nExperiments on three widely used benchmarks demonstrate these patterns have\npotential in exploring aspect terms from test sentence when only feeding the\nsentence to the model. Meanwhile, the proposed CAPSAR can clearly outperform\nSOTA methods in standard ATSA tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10806v1",
            "date": "2019-12-20 02:52:27+00:00",
            "title": "DP-LSTM: Differential Privacy-inspired LSTM for Stock Prediction Using Financial News",
            "authors": [
                "Xinyi Li",
                "Yinchuan Li",
                "Hongyang Yang",
                "Liuqing Yang",
                "Xiao-Yang Liu"
            ],
            "abstract": "Stock price prediction is important for value investments in the stock\nmarket. In particular, short-term prediction that exploits financial news\narticles is promising in recent years. In this paper, we propose a novel deep\nneural network DP-LSTM for stock price prediction, which incorporates the news\narticles as hidden information and integrates difference news sources through\nthe differential privacy mechanism. First, based on the autoregressive moving\naverage model (ARMA), a sentiment-ARMA is formulated by taking into\nconsideration the information of financial news articles in the model. Then, an\nLSTM-based deep neural network is designed, which consists of three components:\nLSTM, VADER model and differential privacy (DP) mechanism. The proposed DP-LSTM\nscheme can reduce prediction errors and increase the robustness. Extensive\nexperiments on S&P 500 stocks show that (i) the proposed DP-LSTM achieves 0.32%\nimprovement in mean MPA of prediction result, and (ii) for the prediction of\nthe market index S&P 500, we achieve up to 65.79% improvement in MSE.",
            "journal ref": "None",
            "category": "q-fin.ST"
        },
        {
            "url": "http://arxiv.org/abs/1912.11612v1",
            "date": "2019-12-25 07:31:44+00:00",
            "title": "N-gram Statistical Stemmer for Bangla Corpus",
            "authors": [
                "Rabeya Sadia",
                "Md Ataur Rahman",
                "Md Hanif Seddiqui"
            ],
            "abstract": "Stemming is a process that can be utilized to trim inflected words to stem or\nroot form. It is useful for enhancing the retrieval effectiveness, especially\nfor text search in order to solve the mismatch problems. Previous research on\nBangla stemming mostly relied on eliminating multiple suffixes from a solitary\nword through a recursive rule based procedure to recover progressively\napplicable relative root. Our proposed system has enhanced the aforementioned\nexploration by actualizing one of the stemming algorithms called N-gram\nstemming. By utilizing an affiliation measure called dice coefficient, related\nsets of words are clustered depending on their character structure. The\nsmallest word in one cluster may be considered as the stem. We additionally\nanalyzed Affinity Propagation clustering algorithms with coefficient similarity\nas well as with median similarity. Our result indicates N-gram stemming\ntechniques to be effective in general which gave us around 87% accurate\nclusters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11637v1",
            "date": "2019-12-25 10:59:31+00:00",
            "title": "Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection",
            "authors": [
                "Guangxiang Zhao",
                "Junyang Lin",
                "Zhiyuan Zhang",
                "Xuancheng Ren",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Self-attention based Transformer has demonstrated the state-of-the-art\nperformances in a number of natural language processing tasks. Self-attention\nis able to model long-term dependencies, but it may suffer from the extraction\nof irrelevant information in the context. To tackle the problem, we propose a\nnovel model called \\textbf{Explicit Sparse Transformer}. Explicit Sparse\nTransformer is able to improve the concentration of attention on the global\ncontext through an explicit selection of the most relevant segments. Extensive\nexperimental results on a series of natural language processing and computer\nvision tasks, including neural machine translation, image captioning, and\nlanguage modeling, all demonstrate the advantages of Explicit Sparse\nTransformer in model performance. We also show that our proposed sparse\nattention method achieves comparable or better results than the previous sparse\nattention method, but significantly reduces training and testing time. For\nexample, the inference speed is twice that of sparsemax in Transformer model.\nCode will be available at\n\\url{https://github.com/lancopku/Explicit-Sparse-Transformer}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11668v1",
            "date": "2019-12-25 13:43:44+00:00",
            "title": "Learning to Answer Ambiguous Questions with Knowledge Graph",
            "authors": [
                "Yikai Zhu",
                "Jianhao Shen",
                "Ming Zhang"
            ],
            "abstract": "In the task of factoid question answering over knowledge base, many questions\nhave more than one plausible interpretation. Previous works on SimpleQuestions\nassume only one interpretation as the ground truth for each question, so they\nlack the ability to answer ambiguous questions correctly. In this paper, we\npresent a new way to utilize the dataset that takes into account the existence\nof ambiguous questions. Then we introduce a simple and effective model which\ncombines local knowledge subgraph with attention mechanism. Our experimental\nresults show that our approach achieves outstanding performance in this task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.11720v1",
            "date": "2019-12-25 22:01:59+00:00",
            "title": "Convolutional Quantum-Like Language Model with Mutual-Attention for Product Rating Prediction",
            "authors": [
                "Qing Ping",
                "Chaomei Chen"
            ],
            "abstract": "Recommender systems are designed to help mitigate information overload users\nexperience during online shopping. Recent work explores neural language models\nto learn user and item representations from user reviews and combines such\nrepresentations with rating information. Most existing convolutional-based\nneural models take pooling immediately after convolution and loses the\ninteraction information between the latent dimension of convolutional feature\nvectors along the way. Moreover, these models usually take all feature vectors\nat higher levels as equal and do not take into consideration that some features\nare more relevant to this specific user-item context. To bridge these gaps,\nthis paper proposes a convolutional quantum-like language model with\nmutual-attention for rating prediction (ConQAR). By introducing a quantum-like\ndensity matrix layer, interactions between latent dimensions of convolutional\nfeature vectors are well captured. With the attention weights learned from the\nmutual-attention layer, final representations of a user and an item absorb\ninformation from both itself and its counterparts for making rating prediction.\nExperiments on two large datasets show that our model outperforms multiple\nstate-of-the-art CNN-based models. We also perform an ablation test to analyze\nthe independent effects of the two components of our model. Moreover, we\nconduct a case study and present visualizations of the quantum probabilistic\ndistributions in one user and one item review document to show that the learned\ndistributions capture meaningful information about this user and item, and can\nbe potentially used as textual profiling of the user and item.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.11975v1",
            "date": "2019-12-27 03:40:31+00:00",
            "title": "Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilation",
            "authors": [
                "Kexin Huang",
                "Abhishek Singh",
                "Sitong Chen",
                "Edward T. Moseley",
                "Chih-ying Deng",
                "Naomi George",
                "Charlotta Lindvall"
            ],
            "abstract": "Clinical notes contain rich data, which is unexploited in predictive modeling\ncompared to structured data. In this work, we developed a new text\nrepresentation Clinical XLNet for clinical notes which also leverages the\ntemporal information of the sequence of the notes. We evaluated our models on\nprolonged mechanical ventilation prediction problem and our experiments\ndemonstrated that Clinical XLNet outperforms the best baselines consistently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12014v1",
            "date": "2019-12-27 07:46:29+00:00",
            "title": "Visual Agreement Regularized Training for Multi-Modal Machine Translation",
            "authors": [
                "Pengcheng Yang",
                "Boxing Chen",
                "Pei Zhang",
                "Xu Sun"
            ],
            "abstract": "Multi-modal machine translation aims at translating the source sentence into\na different language in the presence of the paired image. Previous work\nsuggests that additional visual information only provides dispensable help to\ntranslation, which is needed in several very special cases such as translating\nambiguous words. To make better use of visual information, this work presents\nvisual agreement regularized training. The proposed approach jointly trains the\nsource-to-target and target-to-source translation models and encourages them to\nshare the same focus on the visual information when generating semantically\nequivalent visual words (e.g. \"ball\" in English and \"ballon\" in French).\nBesides, a simple yet effective multi-head co-attention model is also\nintroduced to capture interactions between visual and textual features. The\nresults show that our approaches can outperform competitive baselines by a\nlarge margin on the Multi30k dataset. Further analysis demonstrates that the\nproposed regularized training can effectively improve the agreement of\nattention on the image, leading to better use of visual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12333v2",
            "date": "2019-12-27 20:59:38+00:00",
            "title": "Encoding word order in complex embeddings",
            "authors": [
                "Benyou Wang",
                "Donghao Zhao",
                "Christina Lioma",
                "Qiuchi Li",
                "Peng Zhang",
                "Jakob Grue Simonsen"
            ],
            "abstract": "Sequential word order is important when processing text. Currently, neural\nnetworks (NNs) address this by modeling word position using position\nembeddings. The problem is that position embeddings capture the position of\nindividual words, but not the ordered relationship (e.g., adjacency or\nprecedence) between individual word positions. We present a novel and\nprincipled solution for modeling both the global absolute positions of words\nand their order relationships. Our solution generalizes word embeddings,\npreviously defined as independent vectors, to continuous word functions over a\nvariable (position). The benefit of continuous functions over variable\npositions is that word representations shift smoothly with increasing\npositions. Hence, word representations in different positions can correlate\nwith each other in a continuous function. The general solution of these\nfunctions is extended to complex-valued domain due to richer representations.\nWe extend CNN, RNN and Transformer NNs to complex-valued versions to\nincorporate our complex embedding (we make all code available). Experiments on\ntext classification, machine translation and language modeling show gains over\nboth classical word embeddings and position-enriched word embeddings. To our\nknowledge, this is the first work in NLP to link imaginary numbers in\ncomplex-valued representations to concrete meanings (i.e., word order).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12397v1",
            "date": "2019-12-28 04:05:15+00:00",
            "title": "Natural language processing of MIMIC-III clinical notes for identifying diagnosis and procedures with neural networks",
            "authors": [
                "Siddhartha Nuthakki",
                "Sunil Neela",
                "Judy W. Gichoya",
                "Saptarshi Purkayastha"
            ],
            "abstract": "Coding diagnosis and procedures in medical records is a crucial process in\nthe healthcare industry, which includes the creation of accurate billings,\nreceiving reimbursements from payers, and creating standardized patient care\nrecords. In the United States, Billing and Insurance related activities cost\naround $471 billion in 2012 which constitutes about 25% of all the U.S hospital\nspending. In this paper, we report the performance of a natural language\nprocessing model that can map clinical notes to medical codes, and predict\nfinal diagnosis from unstructured entries of history of present illness,\nsymptoms at the time of admission, etc. Previous studies have demonstrated that\ndeep learning models perform better at such mapping when compared to\nconventional machine learning models. Therefore, we employed state-of-the-art\ndeep learning method, ULMFiT on the largest emergency department clinical notes\ndataset MIMIC III which has 1.2M clinical notes to select for the top-10 and\ntop-50 diagnosis and procedure codes. Our models were able to predict the\ntop-10 diagnoses and procedures with 80.3% and 80.5% accuracy, whereas the\ntop-50 ICD-9 codes of diagnosis and procedures are predicted with 70.7% and\n63.9% accuracy. Prediction of diagnosis and procedures from unstructured\nclinical notes benefit human coders to save time, eliminate errors and minimize\ncosts. With promising scores from our present model, the next step would be to\ndeploy this on a small-scale real-world scenario and compare it with human\ncoders as the gold standard. We believe that further research of this approach\ncan create highly accurate predictions that can ease the workflow in a clinical\nsetting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12514v1",
            "date": "2019-12-28 20:11:33+00:00",
            "title": "Tha3aroon at NSURL-2019 Task 8: Semantic Question Similarity in Arabic",
            "authors": [
                "Ali Fadel",
                "Ibraheem Tuffaha",
                "Mahmoud Al-Ayyoub"
            ],
            "abstract": "In this paper, we describe our team's effort on the semantic text question\nsimilarity task of NSURL 2019. Our top performing system utilizes several\ninnovative data augmentation techniques to enlarge the training data. Then, it\ntakes ELMo pre-trained contextual embeddings of the data and feeds them into an\nON-LSTM network with self-attention. This results in sequence representation\nvectors that are used to predict the relation between the question pairs. The\nmodel is ranked in the 1st place with 96.499 F1-score (same as the second place\nF1-score) and the 2nd place with 94.848 F1-score (differs by 1.076 F1-score\nfrom the first place) on the public and private leaderboards, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12800v1",
            "date": "2019-12-30 03:31:17+00:00",
            "title": "Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog",
            "authors": [
                "Varun Gangal",
                "Abhinav Arora",
                "Arash Einolghozati",
                "Sonal Gupta"
            ],
            "abstract": "The task of identifying out-of-domain (OOD) input examples directly at\ntest-time has seen renewed interest recently due to increased real world\ndeployment of models. In this work, we focus on OOD detection for natural\nlanguage sentence inputs to task-based dialog systems. Our findings are\nthree-fold: First, we curate and release ROSTD (Real Out-of-Domain Sentences\nFrom Task-oriented Dialog) - a dataset of 4K OOD examples for the publicly\navailable dataset from (Schuster et al. 2019). In contrast to existing settings\nwhich synthesize OOD examples by holding out a subset of classes, our examples\nwere authored by annotators with apriori instructions to be out-of-domain with\nrespect to the sentences in an existing dataset. Second, we explore likelihood\nratio based approaches as an alternative to currently prevalent paradigms.\nSpecifically, we reformulate and apply these approaches to natural language\ninputs. We find that they match or outperform the latter on all datasets, with\nlarger improvements on non-artificial OOD benchmarks such as our dataset. Our\nablations validate that specifically using likelihood ratios rather than plain\nlikelihood is necessary to discriminate well between OOD and in-domain data.\nThird, we propose learning a generative classifier and computing a marginal\nlikelihood (ratio) for OOD detection. This allows us to use a principled\nlikelihood while at the same time exploiting training-time labels. We find that\nthis approach outperforms both simple likelihood (ratio) based and other prior\napproaches. We are hitherto the first to investigate the use of generative\nclassifiers for OOD detection at test-time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13082v2",
            "date": "2019-12-30 21:03:59+00:00",
            "title": "The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries",
            "authors": [
                "Atef Chaudhury",
                "Makarand Tapaswi",
                "Seung Wook Kim",
                "Sanja Fidler"
            ],
            "abstract": "Understanding stories is a challenging reading comprehension problem for\nmachines as it requires reading a large volume of text and following long-range\ndependencies. In this paper, we introduce the Shmoop Corpus: a dataset of 231\nstories that are paired with detailed multi-paragraph summaries for each\nindividual chapter (7,234 chapters), where the summary is chronologically\naligned with respect to the story chapter. From the corpus, we construct a set\nof common NLP tasks, including Cloze-form question answering and a simplified\nform of abstractive summarization, as benchmarks for reading comprehension on\nstories. We then show that the chronological alignment provides a strong\nsupervisory signal that learning-based methods can exploit leading to\nsignificant improvements on these tasks. We believe that the unique structure\nof this corpus provides an important foothold towards making machine story\ncomprehension more approachable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13106v1",
            "date": "2019-12-30 22:41:57+00:00",
            "title": "An Empirical Study of Factors Affecting Language-Independent Models",
            "authors": [
                "Xiaotong Liu",
                "Yingbei Tong",
                "Anbang Xu",
                "Rama Akkiraju"
            ],
            "abstract": "Scaling existing applications and solutions to multiple human languages has\ntraditionally proven to be difficult, mainly due to the language-dependent\nnature of preprocessing and feature engineering techniques employed in\ntraditional approaches. In this work, we empirically investigate the factors\naffecting language-independent models built with multilingual representations,\nincluding task type, language set and data resource. On two most representative\nNLP tasks -- sentence classification and sequence labeling, we show that\nlanguage-independent models can be comparable to or even outperforms the models\ntrained using monolingual data, and they are generally more effective on\nsentence classification. We experiment language-independent models with many\ndifferent languages and show that they are more suitable for typologically\nsimilar languages. We also explore the effects of different data sizes when\ntraining and testing language-independent models, and demonstrate that they are\nnot only suitable for high-resource languages, but also very effective in\nlow-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13149v2",
            "date": "2019-12-31 02:46:29+00:00",
            "title": "Revisiting Paraphrase Question Generator using Pairwise Discriminator",
            "authors": [
                "Badri N. Patro",
                "Dev Chauhan",
                "Vinod K. Kurmi",
                "Vinay P. Namboodiri"
            ],
            "abstract": "In this paper, we propose a method for obtaining sentence-level embeddings.\nWhile the problem of securing word-level embeddings is very well studied, we\npropose a novel method for obtaining sentence-level embeddings. This is\nobtained by a simple method in the context of solving the paraphrase generation\ntask. If we use a sequential encoder-decoder model for generating paraphrase,\nwe would like the generated paraphrase to be semantically close to the original\nsentence. One way to ensure this is by adding constraints for true paraphrase\nembeddings to be close and unrelated paraphrase candidate sentence embeddings\nto be far. This is ensured by using a sequential pair-wise discriminator that\nshares weights with the encoder that is trained with a suitable loss function.\nOur loss function penalizes paraphrase sentence embedding distances from being\ntoo large. This loss is used in combination with a sequential encoder-decoder\nnetwork. We also validated our method by evaluating the obtained embeddings for\na sentiment analysis task. The proposed method results in semantic embeddings\nand outperforms the state-of-the-art on the paraphrase generation and sentiment\nanalysis task on standard datasets. These results are also shown to be\nstatistically significant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13161v1",
            "date": "2019-12-26 15:41:35+00:00",
            "title": "Amharic-Arabic Neural Machine Translation",
            "authors": [
                "Ibrahim Gashaw",
                "H L Shashirekha"
            ],
            "abstract": "Many automatic translation works have been addressed between major European\nlanguage pairs, by taking advantage of large scale parallel corpora, but very\nfew research works are conducted on the Amharic-Arabic language pair due to its\nparallel data scarcity. Two Long Short-Term Memory (LSTM) and Gated Recurrent\nUnits (GRU) based Neural Machine Translation (NMT) models are developed using\nAttention-based Encoder-Decoder architecture which is adapted from the\nopen-source OpenNMT system. In order to perform the experiment, a small\nparallel Quranic text corpus is constructed by modifying the existing\nmonolingual Arabic text and its equivalent translation of Amharic language text\ncorpora available on Tanzile. LSTM and GRU based NMT models and Google\nTranslation system are compared and found that LSTM based OpenNMT outperforms\nGRU based OpenNMT and Google Translation system, with a BLEU score of 12%, 11%,\nand 6% respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13415v1",
            "date": "2019-12-20 19:47:56+00:00",
            "title": "End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models",
            "authors": [
                "John Giorgi",
                "Xindi Wang",
                "Nicola Sahar",
                "Won Young Shin",
                "Gary D. Bader",
                "Bo Wang"
            ],
            "abstract": "Named entity recognition (NER) and relation extraction (RE) are two important\ntasks in information extraction and retrieval (IE \\& IR). Recent work has\ndemonstrated that it is beneficial to learn these tasks jointly, which avoids\nthe propagation of error inherent in pipeline-based systems and improves\nperformance. However, state-of-the-art joint models typically rely on external\nnatural language processing (NLP) tools, such as dependency parsers, limiting\ntheir usefulness to domains (e.g. news) where those tools perform well. The few\nneural, end-to-end models that have been proposed are trained almost completely\nfrom scratch. In this paper, we propose a neural, end-to-end model for jointly\nextracting entities and their relations which does not rely on external NLP\ntools and which integrates a large, pre-trained language model. Because the\nbulk of our model's parameters are pre-trained and we eschew recurrence for\nself-attention, our model is fast to train. On 5 datasets across 3 domains, our\nmodel matches or exceeds state-of-the-art performance, sometimes by a large\nmargin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04346v1",
            "date": "2019-12-18 23:48:42+00:00",
            "title": "Asymmetrical Hierarchical Networks with Attentive Interactions for Interpretable Review-Based Recommendation",
            "authors": [
                "Xin Dong",
                "Jingchao Ni",
                "Wei Cheng",
                "Zhengzhang Chen",
                "Bo Zong",
                "Dongjin Song",
                "Yanchi Liu",
                "Haifeng Chen",
                "Gerard de Melo"
            ],
            "abstract": "Recently, recommender systems have been able to emit substantially improved\nrecommendations by leveraging user-provided reviews. Existing methods typically\nmerge all reviews of a given user or item into a long document, and then\nprocess user and item documents in the same manner. In practice, however, these\ntwo sets of reviews are notably different: users' reviews reflect a variety of\nitems that they have bought and are hence very heterogeneous in their topics,\nwhile an item's reviews pertain only to that single item and are thus topically\nhomogeneous. In this work, we develop a novel neural network model that\nproperly accounts for this important difference by means of asymmetric\nattentive modules. The user module learns to attend to only those signals that\nare relevant with respect to the target item, whereas the item module learns to\nextract the most salient contents with regard to properties of the item. Our\nmulti-hierarchical paradigm accounts for the fact that neither are all reviews\nequally useful, nor are all sentences within each review equally pertinent.\nExtensive experimental results on a variety of real datasets demonstrate the\neffectiveness of our method.",
            "journal ref": "AAAI 2020",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.00098v1",
            "date": "2019-01-31 22:14:34+00:00",
            "title": "The Second Conversational Intelligence Challenge (ConvAI2)",
            "authors": [
                "Emily Dinan",
                "Varvara Logacheva",
                "Valentin Malykh",
                "Alexander Miller",
                "Kurt Shuster",
                "Jack Urbanek",
                "Douwe Kiela",
                "Arthur Szlam",
                "Iulian Serban",
                "Ryan Lowe",
                "Shrimai Prabhumoye",
                "Alan W Black",
                "Alexander Rudnicky",
                "Jason Williams",
                "Joelle Pineau",
                "Mikhail Burtsev",
                "Jason Weston"
            ],
            "abstract": "We describe the setting and results of the ConvAI2 NeurIPS competition that\naims to further the state-of-the-art in open-domain chatbots. Some key\ntakeaways from the competition are: (i) pretrained Transformer variants are\ncurrently the best performing models on this task, (ii) but to improve\nperformance on multi-turn conversations with humans, future systems must go\nbeyond single word metrics like perplexity to measure the performance across\nsequences of utterances (conversations) -- in terms of repetition, consistency\nand balance of dialogue acts (e.g. how many questions asked vs. answered).",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.00175v1",
            "date": "2019-02-01 04:30:42+00:00",
            "title": "Dating Documents using Graph Convolution Networks",
            "authors": [
                "Shikhar Vashishth",
                "Shib Sankar Dasgupta",
                "Swayambhu Nath Ray",
                "Partha Talukdar"
            ],
            "abstract": "Document date is essential for many important tasks, such as document\nretrieval, summarization, event detection, etc. While existing approaches for\nthese tasks assume accurate knowledge of the document date, this is not always\navailable, especially for arbitrary documents from the Web. Document Dating is\na challenging problem which requires inference over the temporal structure of\nthe document. Prior document dating systems have largely relied on handcrafted\nfeatures while ignoring such document internal structures. In this paper, we\npropose NeuralDater, a Graph Convolutional Network (GCN) based document dating\napproach which jointly exploits syntactic and temporal graph structures of\ndocument in a principled way. To the best of our knowledge, this is the first\napplication of deep learning for the problem of document dating. Through\nextensive experiments on real-world datasets, we find that NeuralDater\nsignificantly outperforms state-of-the-art baseline by 19% absolute (45%\nrelative) accuracy points.",
            "journal ref": "Proceedings of the 56th Annual Meeting of the Association for\n  Computational Linguistics 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.00570v1",
            "date": "2019-02-01 21:48:45+00:00",
            "title": "Exploring attention mechanism for acoustic-based classification of speech utterances into system-directed and non-system-directed",
            "authors": [
                "Atta Norouzian",
                "Bogdan Mazoure",
                "Dermot Connolly",
                "Daniel Willett"
            ],
            "abstract": "Voice controlled virtual assistants (VAs) are now available in smartphones,\ncars, and standalone devices in homes. In most cases, the user needs to first\n\"wake-up\" the VA by saying a particular word/phrase every time he or she wants\nthe VA to do something. Eliminating the need for saying the wake-up word for\nevery interaction could improve the user experience. This would require the VA\nto have the capability to detect the speech that is being directed at it and\nrespond accordingly. In other words, the challenge is to distinguish between\nsystem-directed and non-system-directed speech utterances. In this paper, we\npresent a number of neural network architectures for tackling this\nclassification problem based on using only acoustic features. These\narchitectures are based on using convolutional, recurrent and feed-forward\nlayers. In addition, we investigate the use of an attention mechanism applied\nto the output of the convolutional and the recurrent layers. It is shown that\nincorporating the proposed attention mechanism into the models always leads to\nsignificant improvement in classification accuracy. The best model achieved\nequal error rates of 16.25 and 15.62 percents on two distinct realistic\ndatasets.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1902.00613v1",
            "date": "2019-02-02 01:34:56+00:00",
            "title": "Understanding Composition of Word Embeddings via Tensor Decomposition",
            "authors": [
                "Abraham Frandsen",
                "Rong Ge"
            ],
            "abstract": "Word embedding is a powerful tool in natural language processing. In this\npaper we consider the problem of word embedding composition \\--- given vector\nrepresentations of two words, compute a vector for the entire phrase. We give a\ngenerative model that can capture specific syntactic relations between words.\nUnder our model, we prove that the correlations between three words (measured\nby their PMI) form a tensor that has an approximate low rank Tucker\ndecomposition. The result of the Tucker decomposition gives the word embeddings\nas well as a core tensor, which can be used to produce better compositions of\nthe word embeddings. We also complement our theoretical results with\nexperiments that verify our assumptions, and demonstrate the effectiveness of\nthe new composition method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.00751v2",
            "date": "2019-02-02 16:29:47+00:00",
            "title": "Parameter-Efficient Transfer Learning for NLP",
            "authors": [
                "Neil Houlsby",
                "Andrei Giurgiu",
                "Stanislaw Jastrzebski",
                "Bruna Morrone",
                "Quentin de Laroussilhe",
                "Andrea Gesmundo",
                "Mona Attariyan",
                "Sylvain Gelly"
            ],
            "abstract": "Fine-tuning large pre-trained models is an effective transfer mechanism in\nNLP. However, in the presence of many downstream tasks, fine-tuning is\nparameter inefficient: an entire new model is required for every task. As an\nalternative, we propose transfer with adapter modules. Adapter modules yield a\ncompact and extensible model; they add only a few trainable parameters per\ntask, and new tasks can be added without revisiting previous ones. The\nparameters of the original network remain fixed, yielding a high degree of\nparameter sharing. To demonstrate adapter's effectiveness, we transfer the\nrecently proposed BERT Transformer model to 26 diverse text classification\ntasks, including the GLUE benchmark. Adapters attain near state-of-the-art\nperformance, whilst adding only a few parameters per task. On GLUE, we attain\nwithin 0.4% of the performance of full fine-tuning, adding only 3.6% parameters\nper task. By contrast, fine-tuning trains 100% of the parameters per task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.01119v2",
            "date": "2019-02-04 10:46:53+00:00",
            "title": "The Natural Language of Actions",
            "authors": [
                "Guy Tennenholtz",
                "Shie Mannor"
            ],
            "abstract": "We introduce Act2Vec, a general framework for learning context-based action\nrepresentation for Reinforcement Learning. Representing actions in a vector\nspace help reinforcement learning algorithms achieve better performance by\ngrouping similar actions and utilizing relations between different actions. We\nshow how prior knowledge of an environment can be extracted from demonstrations\nand injected into action vector representations that encode natural compatible\nbehavior. We then use these for augmenting state representations as well as\nimproving function approximation of Q-values. We visualize and test action\nembeddings in three domains including a drawing task, a high dimensional\nnavigation task, and the large action space domain of StarCraft II.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.01267v2",
            "date": "2019-01-31 20:49:17+00:00",
            "title": "Rhythm Zone Theory: Speech Rhythms are Physical after all",
            "authors": [
                "Dafydd Gibbon",
                "Xuewei Lin"
            ],
            "abstract": "Speech rhythms have been dealt with in three main ways: from the\nintrospective analyses of rhythm as a correlate of syllable and foot timing in\nlinguistics and applied linguistics, through analyses of durations of segments\nof utterances associated with consonantal and vocalic properties, syllables,\nfeet and words, to models of rhythms in speech production and perception as\nphysical oscillations. The present study avoids introspection and\nhuman-filtered annotation methods and extends the signal processing paradigm of\namplitude envelope spectrum analysis by adding an additional analytic step of\nedge detection, and postulating the co-existence of multiple speech rhythms in\nrhythm zones marked by identifiable edges (Rhythm Zone Theory, RZT). An\nexploratory investigation of the utility of RZT is conducted, suggesting that\nnative and non-native readings of the same text are distinct sub-genres of read\nspeech: a reading by a US native speaker and non-native readings by relatively\nlow-performing Cantonese adult learners of English. The study concludes by\nnoting that with the methods used, RZT can distinguish between the speech\nrhythms of well-defined sub-genres of native speaker reading vs. non-native\nlearner reading, but needs further refinement in order to be applied to the\nparadoxically more complex speech of low-performing language learners, whose\nspeech rhythms are co-determined by non-fluency and disfluency factors in\naddition to well-known linguistic factors of grammar, vocabulary and discourse\nconstraints.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1902.01313v2",
            "date": "2019-02-04 17:08:32+00:00",
            "title": "An Effective Approach to Unsupervised Machine Translation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre"
            ],
            "abstract": "While machine translation has traditionally relied on large amounts of\nparallel corpora, a recent research line has managed to train both Neural\nMachine Translation (NMT) and Statistical Machine Translation (SMT) systems\nusing monolingual corpora only. In this paper, we identify and address several\ndeficiencies of existing unsupervised SMT approaches by exploiting subword\ninformation, developing a theoretically well founded unsupervised tuning\nmethod, and incorporating a joint refinement procedure. Moreover, we use our\nimproved SMT system to initialize a dual NMT model, which is further fine-tuned\nthrough on-the-fly back-translation. Together, we obtain large improvements\nover the previous state-of-the-art in unsupervised machine translation. For\ninstance, we get 22.5 BLEU points in English-to-German WMT 2014, 5.5 points\nmore than the previous best unsupervised system, and 0.5 points more than the\n(supervised) shared task winner back in 2014.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01509v1",
            "date": "2019-02-05 01:17:07+00:00",
            "title": "Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation",
            "authors": [
                "Vladimir Karpukhin",
                "Omer Levy",
                "Jacob Eisenstein",
                "Marjan Ghazvininejad"
            ],
            "abstract": "We consider the problem of making machine translation more robust to\ncharacter-level variation at the source side, such as typos. Existing methods\nachieve greater coverage by applying subword models such as byte-pair encoding\n(BPE) and character-level encoders, but these methods are highly sensitive to\nspelling mistakes. We show how training on a mild amount of random synthetic\nnoise can dramatically improve robustness to these variations, without\ndiminishing performance on clean text. We focus on translation performance on\nnatural noise, as captured by frequent corrections in Wikipedia edit logs, and\nshow that robustness to such noise can be achieved using a balanced diet of\nsimple synthetic noises at training time, without access to the natural noise\ndata or distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.01951v2",
            "date": "2019-02-02 07:33:48+00:00",
            "title": "Using multi-task learning to improve the performance of acoustic-to-word and conventional hybrid models",
            "authors": [
                "Thai-Son Nguyen",
                "Sebastian Stueker",
                "Alex Waibel"
            ],
            "abstract": "Acoustic-to-word (A2W) models that allow direct mapping from acoustic signals\nto word sequences are an appealing approach to end-to-end automatic speech\nrecognition due to their simplicity. However, prior works have shown that\nmodelling A2W typically encounters issues of data sparsity that prevent\ntraining such a model directly. So far, pre-training initialization is the only\napproach proposed to deal with this issue. In this work, we propose to build a\nshared neural network and optimize A2W and conventional hybrid models in a\nmulti-task manner. Our results show that training an A2W model is much more\nstable with our multi-task model without pre-training initialization, and\nresults in a significant improvement compared to a baseline model. Experiments\nalso reveal that the performance of a hybrid acoustic model can be further\nimproved when jointly training with a sequence-level optimization criterion\nsuch as acoustic-to-word.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1902.02113v1",
            "date": "2019-02-06 11:15:08+00:00",
            "title": "Latent Space Cartography: Generalised Metric-Inspired Measures and Measure-Based Transformations for Generative Models",
            "authors": [
                "Max F. Frenzel",
                "Bogdan Teleaga",
                "Asahi Ushio"
            ],
            "abstract": "Deep generative models are universal tools for learning data distributions on\nhigh dimensional data spaces via a mapping to lower dimensional latent spaces.\nWe provide a study of latent space geometries and extend and build upon\nprevious results on Riemannian metrics. We show how a class of heuristic\nmeasures gives more flexibility in finding meaningful, problem-specific\ndistances, and how it can be applied to diverse generator types such as\nautoregressive generators commonly used in e.g. language and other sequence\nmodeling. We further demonstrate how a diffusion-inspired transformation\npreviously studied in cartography can be used to smooth out latent spaces,\nstretching them according to a chosen measure. In addition to providing more\nmeaningful distances directly in latent space, this also provides a unique tool\nfor novel kinds of data visualizations. We believe that the proposed methods\ncan be a valuable tool for studying the structure of latent spaces and learned\ndata distributions of generative models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.02169v1",
            "date": "2019-01-31 17:18:42+00:00",
            "title": "Learning Taxonomies of Concepts and not Words using Contextualized Word Representations: A Position Paper",
            "authors": [
                "Lukas Schmelzeisen",
                "Steffen Staab"
            ],
            "abstract": "Taxonomies are semantic hierarchies of concepts. One limitation of current\ntaxonomy learning systems is that they define concepts as single words. This\nposition paper argues that contextualized word representations, which recently\nachieved state-of-the-art results on many competitive NLP tasks, are a\npromising method to address this limitation. We outline a novel approach for\ntaxonomy learning that (1) defines concepts as synsets, (2) learns\ndensity-based approximations of contextualized word representations, and (3)\ncan measure similarity and hypernymy among them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02181v3",
            "date": "2019-02-04 17:14:13+00:00",
            "title": "Attention in Natural Language Processing",
            "authors": [
                "Andrea Galassi",
                "Marco Lippi",
                "Paolo Torroni"
            ],
            "abstract": "Attention is an increasingly popular mechanism used in a wide range of neural\narchitectures. The mechanism itself has been realized in a variety of formats.\nHowever, because of the fast-paced advances in this domain, a systematic\noverview of attention is still missing. In this article, we define a unified\nmodel for attention architectures in natural language processing, with a focus\non those designed to work with vector representations of the textual data. We\npropose a taxonomy of attention models according to four dimensions: the\nrepresentation of the input, the compatibility function, the distribution\nfunction, and the multiplicity of the input and/or output. We present the\nexamples of how prior information can be exploited in attention models and\ndiscuss ongoing research efforts and open challenges in the area, providing the\nfirst extensive categorization of the vast body of literature in this exciting\ndomain.",
            "journal ref": "IEEE Transactions on Neural Networks and Learning Systems (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02192v3",
            "date": "2019-02-05 14:02:45+00:00",
            "title": "Non-Monotonic Sequential Text Generation",
            "authors": [
                "Sean Welleck",
                "Kiant\u00e9 Brantley",
                "Hal Daum\u00e9 III",
                "Kyunghyun Cho"
            ],
            "abstract": "Standard sequential generation methods assume a pre-specified generation\norder, such as text generation methods which generate words from left to right.\nIn this work, we propose a framework for training models of text generation\nthat operate in non-monotonic orders; the model directly learns good orders,\nwithout any additional annotation. Our framework operates by generating a word\nat an arbitrary position, and then recursively generating words to its left and\nthen words to its right, yielding a binary tree. Learning is framed as\nimitation learning, including a coaching method which moves from imitating an\noracle to reinforcing the policy's own preferences. Experimental results\ndemonstrate that using the proposed method, it is possible to learn policies\nwhich generate text without pre-specifying a generation order, while achieving\ncompetitive performance with conventional left-to-right generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02383v1",
            "date": "2019-02-06 19:50:23+00:00",
            "title": "End-to-end Anchored Speech Recognition",
            "authors": [
                "Yiming Wang",
                "Xing Fan",
                "I-Fan Chen",
                "Yuzong Liu",
                "Tongfei Chen",
                "Bj\u00f6rn Hoffmeister"
            ],
            "abstract": "Voice-controlled house-hold devices, like Amazon Echo or Google Home, face\nthe problem of performing speech recognition of device-directed speech in the\npresence of interfering background speech, i.e., background noise and\ninterfering speech from another person or media device in proximity need to be\nignored. We propose two end-to-end models to tackle this problem with\ninformation extracted from the \"anchored segment\". The anchored segment refers\nto the wake-up word part of an audio stream, which contains valuable speaker\ninformation that can be used to suppress interfering speech and background\nnoise. The first method is called \"Multi-source Attention\" where the attention\nmechanism takes both the speaker information and decoder state into\nconsideration. The second method directly learns a frame-level mask on top of\nthe encoder output. We also explore a multi-task learning setup where we use\nthe ground truth of the mask to guide the learner. Given that audio data with\ninterfering speech is rare in our training data set, we also propose a way to\nsynthesize \"noisy\" speech from \"clean\" speech to mitigate the mismatch between\ntraining and test data. Our proposed methods show up to 15% relative reduction\nin WER for Amazon Alexa live data with interfering background speech without\nsignificantly degrading on clean speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.02671v2",
            "date": "2019-02-07 15:05:46+00:00",
            "title": "BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning",
            "authors": [
                "Asa Cooper Stickland",
                "Iain Murray"
            ],
            "abstract": "Multi-task learning shares information between related tasks, sometimes\nreducing the number of parameters required. State-of-the-art results across\nmultiple natural language understanding tasks in the GLUE benchmark have\npreviously used transfer from a single large task: unsupervised pre-training\nwith BERT, where a separate BERT model was fine-tuned for each task. We explore\nmulti-task approaches that share a single BERT model with a small number of\nadditional task-specific parameters. Using new adaptation modules, PALs or\n`projected attention layers', we match the performance of separately fine-tuned\nmodels on the GLUE benchmark with roughly 7 times fewer parameters, and obtain\nstate-of-the-art results on the Recognizing Textual Entailment dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.02783v3",
            "date": "2019-02-08 14:36:43+00:00",
            "title": "Humor in Word Embeddings: Cockamamie Gobbledegook for Nincompoops",
            "authors": [
                "Limor Gultchin",
                "Genevieve Patterson",
                "Nancy Baym",
                "Nathaniel Swinger",
                "Adam Tauman Kalai"
            ],
            "abstract": "While humor is often thought to be beyond the reach of Natural Language\nProcessing, we show that several aspects of single-word humor correlate with\nsimple linear directions in Word Embeddings. In particular: (a) the word\nvectors capture multiple aspects discussed in humor theories from various\ndisciplines; (b) each individual's sense of humor can be represented by a\nvector, which can predict differences in people's senses of humor on new,\nunrated, words; and (c) upon clustering humor ratings of multiple demographic\ngroups, different humor preferences emerge across the different groups. Humor\nratings are taken from the work of Engelthaler and Hills (2017) as well as from\nan original crowdsourcing study of 120,000 words. Our dataset further includes\nannotations for the theoretically-motivated humor features we identify.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03089v1",
            "date": "2019-01-28 19:58:14+00:00",
            "title": "How is Your Mood When Writing Sexist tweets? Detecting the Emotion Type and Intensity of Emotion Using Natural Language Processing Techniques",
            "authors": [
                "Sima Sharifirad",
                "Borna Jafarpour",
                "Stan Matwin"
            ],
            "abstract": "Online social platforms have been the battlefield of users with different\nemotions and attitudes toward each other in recent years. While sexism has been\nconsidered as a category of hateful speech in the literature, there is no\ncomprehensive definition and category of sexism attracting natural language\nprocessing techniques. Categorizing sexism as either benevolent or hostile\nsexism is so broad that it easily ignores the other categories of sexism on\nsocial media. Sharifirad S and Matwin S 2018 proposed a well-defined category\nof sexism including indirect harassment, information threat, sexual harassment\nand physical harassment, inspired from social science for the purpose of\nnatural language processing techniques. In this article, we take advantage of a\nnewly released dataset in SemEval-2018 task1: Affect in tweets, to show the\ntype of emotion and intensity of emotion in each category. We train, test and\nevaluate different classification methods on the SemEval- 2018 dataset and\nchoose the classifier with highest accuracy for testing on each category of\nsexist tweets to know the mental state and the affectual state of the user who\ntweets in each category. It is a nice avenue to explore because not all the\ntweets are directly sexist and they carry different emotions from the users.\nThis is the first work experimenting on affect detection this in depth on\nsexist tweets. Based on our best knowledge they are all new contributions to\nthe field; we are the first to demonstrate the power of such in-depth sentiment\nanalysis on the sexist tweets.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1902.03190v1",
            "date": "2019-02-08 16:54:51+00:00",
            "title": "Speaker diarisation using 2D self-attentive combination of embeddings",
            "authors": [
                "Guangzhi Sun",
                "Chao Zhang",
                "Phil Woodland"
            ],
            "abstract": "Speaker diarisation systems often cluster audio segments using speaker\nembeddings such as i-vectors and d-vectors. Since different types of embeddings\nare often complementary, this paper proposes a generic framework to improve\nperformance by combining them into a single embedding, referred to as a\nc-vector. This combination uses a 2-dimensional (2D) self-attentive structure,\nwhich extends the standard self-attentive layer by averaging not only across\ntime but also across different types of embeddings. Two types of 2D\nself-attentive structure in this paper are the simultaneous combination and the\nconsecutive combination, adopting a single and multiple self-attentive layers\nrespectively. The penalty term in the original self-attentive layer which is\njointly minimised with the objective function to encourage diversity of\nannotation vectors is also modified to obtain not only different local peaks\nbut also the overall trends in the multiple annotation vectors. Experiments on\nthe AMI meeting corpus show that our modified penalty term improves the d-\nvector relative speaker error rate (SER) by 6% and 21% for d-vector systems,\nand a 10% further relative SER reduction can be obtained using the c-vector\nfrom our best 2D self-attentive structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03249v1",
            "date": "2019-02-08 19:00:04+00:00",
            "title": "Insertion Transformer: Flexible Sequence Generation via Insertion Operations",
            "authors": [
                "Mitchell Stern",
                "William Chan",
                "Jamie Kiros",
                "Jakob Uszkoreit"
            ],
            "abstract": "We present the Insertion Transformer, an iterative, partially autoregressive\nmodel for sequence generation based on insertion operations. Unlike typical\nautoregressive models which rely on a fixed, often left-to-right ordering of\nthe output, our approach accommodates arbitrary orderings by allowing for\ntokens to be inserted anywhere in the sequence during decoding. This\nflexibility confers a number of advantages: for instance, not only can our\nmodel be trained to follow specific orderings such as left-to-right generation\nor a binary tree traversal, but it can also be trained to maximize entropy over\nall valid insertions for robustness. In addition, our model seamlessly\naccommodates both fully autoregressive generation (one insertion at a time) and\npartially autoregressive generation (simultaneous insertions at multiple\nlocations). We validate our approach by analyzing its performance on the WMT\n2014 English-German machine translation task under various settings for\ntraining and decoding. We find that the Insertion Transformer outperforms many\nprior non-autoregressive approaches to translation at comparable or better\nlevels of parallelism, and successfully recovers the performance of the\noriginal Transformer while requiring only logarithmically many iterations\nduring decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.03570v1",
            "date": "2019-02-10 10:34:54+00:00",
            "title": "EvalAI: Towards Better Evaluation Systems for AI Agents",
            "authors": [
                "Deshraj Yadav",
                "Rishabh Jain",
                "Harsh Agrawal",
                "Prithvijit Chattopadhyay",
                "Taranjeet Singh",
                "Akash Jain",
                "Shiv Baran Singh",
                "Stefan Lee",
                "Dhruv Batra"
            ],
            "abstract": "We introduce EvalAI, an open source platform for evaluating and comparing\nmachine learning (ML) and artificial intelligence algorithms (AI) at scale.\nEvalAI is built to provide a scalable solution to the research community to\nfulfill the critical need of evaluating machine learning models and agents\nacting in an environment against annotations or with a human-in-the-loop. This\nwill help researchers, students, and data scientists to create, collaborate,\nand participate in AI challenges organized around the globe. By simplifying and\nstandardizing the process of benchmarking these models, EvalAI seeks to lower\nthe barrier to entry for participating in the global scientific effort to push\nthe frontiers of machine learning and artificial intelligence, thereby\nincreasing the rate of measurable progress in this domain.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.04187v1",
            "date": "2019-02-11 23:58:22+00:00",
            "title": "LS-Tree: Model Interpretation When the Data Are Linguistic",
            "authors": [
                "Jianbo Chen",
                "Michael I. Jordan"
            ],
            "abstract": "We study the problem of interpreting trained classification models in the\nsetting of linguistic data sets. Leveraging a parse tree, we propose to assign\nleast-squares based importance scores to each word of an instance by exploiting\nsyntactic constituency structure. We establish an axiomatic characterization of\nthese importance scores by relating them to the Banzhaf value in coalitional\ngame theory. Based on these importance scores, we develop a principled method\nfor detecting and quantifying interactions between words in a sentence. We\ndemonstrate that the proposed method can aid in interpretability and\ndiagnostics for several widely-used language models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.04247v2",
            "date": "2019-02-12 05:49:34+00:00",
            "title": "PAC-Bayes Analysis of Sentence Representation",
            "authors": [
                "Kento Nozawa",
                "Issei Sato"
            ],
            "abstract": "Learning sentence vectors from an unlabeled corpus has attracted attention\nbecause such vectors can represent sentences in a lower dimensional and\ncontinuous space. Simple heuristics using pre-trained word vectors are widely\napplied to machine learning tasks. However, they are not well understood from a\ntheoretical perspective. We analyze learning sentence vectors from a transfer\nlearning perspective by using a PAC-Bayes bound that enables us to understand\nexisting heuristics. We show that simple heuristics such as averaging and\ninverse document frequency weighted averaging are derived by our formulation.\nMoreover, we propose novel sentence vector learning algorithms on the basis of\nour PAC-Bayes analysis.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.05070v1",
            "date": "2019-02-13 11:17:43+00:00",
            "title": "Optimization problems with low SWaP tactical Computing",
            "authors": [
                "Mee Seong Im",
                "Venkat R. Dasari",
                "Lubjana Beshaj",
                "Dale Shires"
            ],
            "abstract": "In a resource-constrained, contested environment, computing resources need to\nbe aware of possible size, weight, and power (SWaP) restrictions. SWaP-aware\ncomputational efficiency depends upon optimization of computational resources\nand intelligent time versus efficiency tradeoffs in decision making. In this\npaper we address the complexity of various optimization strategies related to\nlow SWaP computing. Due to these restrictions, only a small subset of less\ncomplicated and fast computable algorithms can be used for tactical, adaptive\ncomputing.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.06034v3",
            "date": "2019-02-16 03:39:51+00:00",
            "title": "TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts",
            "authors": [
                "Michihiro Yasunaga",
                "John Lafferty"
            ],
            "abstract": "Scientific documents rely on both mathematics and text to communicate ideas.\nInspired by the topical correspondence between mathematical equations and word\ncontexts observed in scientific texts, we propose a novel topic model that\njointly generates mathematical equations and their surrounding text (TopicEq).\nUsing an extension of the correlated topic model, the context is generated from\na mixture of latent topics, and the equation is generated by an RNN that\ndepends on the latent topic activations. To experiment with this model, we\ncreate a corpus of 400K equation-context pairs extracted from a range of\nscientific articles from arXiv, and fit the model using a variational\nautoencoder approach. Experimental results show that this joint model\nsignificantly outperforms existing topic models and equation models for\nscientific texts. Moreover, we qualitatively show that the model effectively\ncaptures the relationship between topics and mathematics, enabling novel\napplications such as topic-aware equation generation, equation topic inference,\nand topic-aware alignment of mathematical symbols and words.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.06050v2",
            "date": "2019-02-16 06:03:57+00:00",
            "title": "Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social Media",
            "authors": [
                "Khuong Vo",
                "Tri Nguyen",
                "Dang Pham",
                "Mao Nguyen",
                "Minh Truong",
                "Trung Mai",
                "Tho Quan"
            ],
            "abstract": "Sentiment analysis has been emerging recently as one of the major natural\nlanguage processing (NLP) tasks in many applications. Especially, as social\nmedia channels (e.g. social networks or forums) have become significant sources\nfor brands to observe user opinions about their products, this task is thus\nincreasingly crucial. However, when applied with real data obtained from social\nmedia, we notice that there is a high volume of short and informal messages\nposted by users on those channels. This kind of data makes the existing works\nsuffer from many difficulties to handle, especially ones using deep learning\napproaches. In this paper, we propose an approach to handle this problem. This\nwork is extended from our previous work, in which we proposed to combine the\ntypical deep learning technique of Convolutional Neural Networks with domain\nknowledge. The combination is used for acquiring additional training data\naugmentation and a more reasonable loss function. In this work, we further\nimprove our architecture by various substantial enhancements, including\nnegation-based data augmentation, transfer learning for word embeddings, the\ncombination of word-level embeddings and character-level embeddings, and using\nmultitask learning technique for attaching domain knowledge rules in the\nlearning process. Those enhancements, specifically aiming to handle short and\ninformal messages, help us to enjoy significant improvement in performance once\nexperimenting on real datasets.",
            "journal ref": "International Journal of Computational Vision and Robotics, 2019\n  Vol.9 No.5, pp.458 - 485",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06450v1",
            "date": "2019-02-18 08:17:24+00:00",
            "title": "Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping",
            "authors": [
                "Linhao Dong",
                "Feng Wang",
                "Bo Xu"
            ],
            "abstract": "Self-attention network, an attention-based feedforward neural network, has\nrecently shown the potential to replace recurrent neural networks (RNNs) in a\nvariety of NLP tasks. However, it is not clear if the self-attention network\ncould be a good alternative of RNNs in automatic speech recognition (ASR),\nwhich processes the longer speech sequences and may have online recognition\nrequirements. In this paper, we present a RNN-free end-to-end model:\nself-attention aligner (SAA), which applies the self-attention networks to a\nsimplified recurrent neural aligner (RNA) framework. We also propose a\nchunk-hopping mechanism, which enables the SAA model to encode on segmented\nframe chunks one after another to support online recognition. Experiments on\ntwo Mandarin ASR datasets show the replacement of RNNs by the self-attention\nnetworks yields a 8.4%-10.2% relative character error rate (CER) reduction. In\naddition, the chunk-hopping mechanism allows the SAA to have only a 2.5%\nrelative CER degradation with a 320ms latency. After jointly training with a\nself-attention network language model, our SAA model obtains further error rate\nreduction on multiple datasets. Especially, it achieves 24.12% CER on the\nMandarin ASR benchmark (HKUST), exceeding the best end-to-end model by over 2%\nabsolute CER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06833v1",
            "date": "2019-02-18 23:06:56+00:00",
            "title": "Learned In Speech Recognition: Contextual Acoustic Word Embeddings",
            "authors": [
                "Shruti Palaskar",
                "Vikas Raunak",
                "Florian Metze"
            ],
            "abstract": "End-to-end acoustic-to-word speech recognition models have recently gained\npopularity because they are easy to train, scale well to large amounts of\ntraining data, and do not require a lexicon. In addition, word models may also\nbe easier to integrate with downstream tasks such as spoken language\nunderstanding, because inference (search) is much simplified compared to\nphoneme, character or any other sort of sub-word units. In this paper, we\ndescribe methods to construct contextual acoustic word embeddings directly from\na supervised sequence-to-sequence acoustic-to-word speech recognition model\nusing the learned attention distribution. On a suite of 16 standard sentence\nevaluation tasks, our embeddings show competitive performance against a\nword2vec model trained on the speech transcriptions. In addition, we evaluate\nthese embeddings on a spoken language understanding task, and observe that our\nembeddings match the performance of text-based embeddings in a pipeline of\nfirst performing speech recognition and then constructing word embeddings from\ntranscriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.06843v1",
            "date": "2019-02-19 00:10:08+00:00",
            "title": "Fusing Visual, Textual and Connectivity Clues for Studying Mental Health",
            "authors": [
                "Amir Hossein Yazdavar",
                "Mohammad Saeid Mahdavinejad",
                "Goonmeet Bajaj",
                "William Romine",
                "Amirhassan Monadjemi",
                "Krishnaprasad Thirunarayan",
                "Amit Sheth",
                "Jyotishman Pathak"
            ],
            "abstract": "With ubiquity of social media platforms, millions of people are sharing their\nonline persona by expressing their thoughts, moods, emotions, feelings, and\neven their daily struggles with mental health issues voluntarily and publicly\non social media. Unlike the most existing efforts which study depression by\nanalyzing textual content, we examine and exploit multimodal big data to\ndiscern depressive behavior using a wide variety of features including\nindividual-level demographics. By developing a multimodal framework and\nemploying statistical techniques for fusing heterogeneous sets of features\nobtained by processing visual, textual and user interaction data, we\nsignificantly enhance the current state-of-the-art approaches for identifying\ndepressed individuals on Twitter (improving the average F1-Score by 5 percent)\nas well as facilitate demographic inference from social media for broader\napplications. Besides providing insights into the relationship between\ndemographics and mental health, our research assists in the design of a new\nbreed of demographic-aware health interventions.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1902.06897v2",
            "date": "2019-02-19 05:14:14+00:00",
            "title": "Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks",
            "authors": [
                "Shubham Gupta",
                "Ambedkar Dukkipati"
            ],
            "abstract": "Humans use language to collectively execute abstract strategies besides using\nit as a referential tool for identifying physical entities. Recently, multiple\nattempts at replicating the process of emergence of language in artificial\nagents have been made. While existing approaches study emergent languages as\nreferential tools, in this paper, we study their role in discovering and\nimplementing strategies. We formulate the problem using a voting game where two\ncandidate agents contest in an election with the goal of convincing population\nmembers (other agents), that are connected to each other via an underlying\nnetwork, to vote for them. To achieve this goal, agents are only allowed to\nexchange messages in the form of sequences of discrete symbols to spread their\npropaganda. We use neural networks with Gumbel-Softmax relaxation for sampling\ncategorical random variables to parameterize the policies followed by all\nagents. Using our proposed framework, we provide concrete answers to the\nfollowing questions: (i) Do the agents learn to communicate in a meaningful way\nand does the emergent communication play a role in deciding the winner? (ii)\nDoes the system evolve as expected under various reward structures? (iii) How\nis the emergent language affected by the community structure in the network? To\nthe best of our knowledge, we are the first to explore emergence of\ncommunication for discovering and implementing strategies in a setting where\nagents communicate over a network.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/1902.07068v1",
            "date": "2019-02-18 15:47:35+00:00",
            "title": "Classifying textual data: shallow, deep and ensemble methods",
            "authors": [
                "Laura Anderlucci",
                "Lucia Guastadisegni",
                "Cinzia Viroli"
            ],
            "abstract": "This paper focuses on a comparative evaluation of the most common and modern\nmethods for text classification, including the recent deep learning strategies\nand ensemble methods. The study is motivated by a challenging real data\nproblem, characterized by high-dimensional and extremely sparse data, deriving\nfrom incoming calls to the customer care of an Italian phone company. We will\nshow that deep learning outperforms many classical (shallow) strategies but the\ncombination of shallow and deep learning methods in a unique ensemble\nclassifier may improve the robustness and the accuracy of \"single\"\nclassification methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07087v2",
            "date": "2019-02-13 20:29:00+00:00",
            "title": "Predicting US State-Level Agricultural Sentiment as a Measure of Food Security with Tweets from Farming Communities",
            "authors": [
                "Jared Dunnmon",
                "Swetava Ganguli",
                "Darren Hau",
                "Brooke Husic"
            ],
            "abstract": "The ability to obtain accurate food security metrics in developing areas\nwhere relevant data can be sparse is critically important for policy makers\ntasked with implementing food aid programs. As a result, a great deal of work\nhas been dedicated to predicting important food security metrics such as annual\ncrop yields using a variety of methods including simulation, remote sensing,\nweather models, and human expert input. As a complement to existing techniques\nin crop yield prediction, this work develops neural network models for\npredicting the sentiment of Twitter feeds from farming communities.\nSpecifically, we investigate the potential of both direct learning on a small\ndataset of agriculturally-relevant tweets and transfer learning from larger,\nwell-labeled sentiment datasets from other domains (e.g.~politics) to\naccurately predict agricultural sentiment, which we hope would ultimately serve\nas a useful crop yield predictor. We find that direct learning from small,\nrelevant datasets outperforms transfer learning from large, fully-labeled\ndatasets, that convolutional neural networks broadly outperform recurrent\nneural networks on Twitter sentiment classification, and that these models\nperform substantially less well on ternary sentiment problems characteristic of\npractical settings than on binary problems often found in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07181v2",
            "date": "2019-02-19 18:27:12+00:00",
            "title": "Measuring Compositionality in Representation Learning",
            "authors": [
                "Jacob Andreas"
            ],
            "abstract": "Many machine learning algorithms represent input data with vector embeddings\nor discrete codes. When inputs exhibit compositional structure (e.g. objects\nbuilt from parts or procedures from subroutines), it is natural to ask whether\nthis compositional structure is reflected in the the inputs' learned\nrepresentations. While the assessment of compositionality in languages has\nreceived significant attention in linguistics and adjacent fields, the machine\nlearning literature lacks general-purpose tools for producing graded\nmeasurements of compositional structure in more general (e.g. vector-valued)\nrepresentation spaces. We describe a procedure for evaluating compositionality\nby measuring how well the true representation-producing model can be\napproximated by a model that explicitly composes a collection of inferred\nrepresentational primitives. We use the procedure to provide formal and\nempirical characterizations of compositional structure in a variety of\nsettings, exploring the relationship between compositionality and learning\ndynamics, human judgments, representational similarity, and generalization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.07198v4",
            "date": "2019-02-19 18:51:10+00:00",
            "title": "Learning to Generalize from Sparse and Underspecified Rewards",
            "authors": [
                "Rishabh Agarwal",
                "Chen Liang",
                "Dale Schuurmans",
                "Mohammad Norouzi"
            ],
            "abstract": "We consider the problem of learning from sparse and underspecified rewards,\nwhere an agent receives a complex input, such as a natural language\ninstruction, and needs to generate a complex response, such as an action\nsequence, while only receiving binary success-failure feedback. Such\nsuccess-failure rewards are often underspecified: they do not distinguish\nbetween purposeful and accidental success. Generalization from underspecified\nrewards hinges on discounting spurious trajectories that attain accidental\nsuccess, while learning from sparse feedback requires effective exploration. We\naddress exploration by using a mode covering direction of KL divergence to\ncollect a diverse set of successful trajectories, followed by a mode seeking KL\ndivergence to train a robust policy. We propose Meta Reward Learning (MeRL) to\nconstruct an auxiliary reward function that provides more refined feedback for\nlearning. The parameters of the auxiliary reward function are optimized with\nrespect to the validation performance of a trained policy. The MeRL approach\noutperforms our alternative reward learning technique based on Bayesian\nOptimization, and achieves the state-of-the-art on weakly-supervised semantic\nparsing. It improves previous work by 1.2% and 2.4% on WikiTableQuestions and\nWikiSQL datasets respectively.",
            "journal ref": "Proceedings of the 36th International Conference on Machine\n  Learning, PMLR 97:130-140, 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.07249v2",
            "date": "2019-02-18 06:19:14+00:00",
            "title": "Discovery of Natural Language Concepts in Individual Units of CNNs",
            "authors": [
                "Seil Na",
                "Yo Joong Choe",
                "Dong-Hyun Lee",
                "Gunhee Kim"
            ],
            "abstract": "Although deep convolutional networks have achieved improved performance in\nmany natural language tasks, they have been treated as black boxes because they\nare difficult to interpret. Especially, little is known about how they\nrepresent language in their intermediate layers. In an attempt to understand\nthe representations of deep convolutional networks trained on language tasks,\nwe show that individual units are selectively responsive to specific morphemes,\nwords, and phrases, rather than responding to arbitrary and uninterpretable\npatterns. In order to quantitatively analyze such an intriguing phenomenon, we\npropose a concept alignment method based on how units respond to the replicated\ntext. We conduct analyses with different architectures on multiple datasets for\nclassification and translation tasks and provide new insights into how deep\nmodels understand natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07285v6",
            "date": "2019-02-12 02:42:54+00:00",
            "title": "Towards a Robust Deep Neural Network in Texts: A Survey",
            "authors": [
                "Wenqi Wang",
                "Run Wang",
                "Lina Wang",
                "Zhibo Wang",
                "Aoshuang Ye"
            ],
            "abstract": "Deep neural networks (DNNs) have achieved remarkable success in various tasks\n(e.g., image classification, speech recognition, and natural language\nprocessing (NLP)). However, researchers have demonstrated that DNN-based models\nare vulnerable to adversarial examples, which cause erroneous predictions by\nadding imperceptible perturbations into legitimate inputs. Recently, studies\nhave revealed adversarial examples in the text domain, which could effectively\nevade various DNN-based text analyzers and further bring the threats of the\nproliferation of disinformation. In this paper, we give a comprehensive survey\non the existing studies of adversarial techniques for generating adversarial\ntexts written by both English and Chinese characters and the corresponding\ndefense methods. More importantly, we hope that our work could inspire future\nstudies to develop more robust DNN-based text analyzers against known and\nunknown adversarial techniques.\n  We classify the existing adversarial techniques for crafting adversarial\ntexts based on the perturbation units, helping to better understand the\ngeneration of adversarial texts and build robust models for defense. In\npresenting the taxonomy of adversarial attacks and defenses in the text domain,\nwe introduce the adversarial techniques from the perspective of different NLP\ntasks. Finally, we discuss the existing challenges of adversarial attacks and\ndefenses in texts and present the future research directions in this emerging\nand challenging field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07516v2",
            "date": "2019-02-20 11:30:17+00:00",
            "title": "Emergence of order in random languages",
            "authors": [
                "E. DeGiuli"
            ],
            "abstract": "We consider languages generated by weighted context-free grammars. It is\nshown that the behaviour of large texts is controlled by saddle-point equations\nfor an appropriate generating function. We then consider ensembles of grammars,\nin particular the Random Language Model of E. DeGiuli, Phys. Rev. Lett., 122,\n128301, 2019. This model is solved in the replica-symmetric ansatz, which is\nvalid in the high-temperature, disordered phase. It is shown that in the phase\nin which languages carry information, the replica symmetry must be broken.",
            "journal ref": "None",
            "category": "cond-mat.dis-nn"
        },
        {
            "url": "http://arxiv.org/abs/1902.07817v1",
            "date": "2019-02-20 23:58:29+00:00",
            "title": "Audio-Linguistic Embeddings for Spoken Sentences",
            "authors": [
                "Albert Haque",
                "Michelle Guo",
                "Prateek Verma",
                "Li Fei-Fei"
            ],
            "abstract": "We propose spoken sentence embeddings which capture both acoustic and\nlinguistic content. While existing works operate at the character, phoneme, or\nword level, our method learns long-term dependencies by modeling speech at the\nsentence level. Formulated as an audio-linguistic multitask learning problem,\nour encoder-decoder model simultaneously reconstructs acoustic and natural\nlanguage features from audio. Our results show that spoken sentence embeddings\noutperform phoneme and word-level baselines on speech recognition and emotion\nrecognition tasks. Ablation studies show that our embeddings can better model\nhigh-level acoustic concepts while retaining linguistic content. Overall, our\nwork illustrates the viability of generic, multi-modal sentence embeddings for\nspoken language understanding.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1902.07821v1",
            "date": "2019-02-21 00:36:24+00:00",
            "title": "Deep Speaker Embedding Learning with Multi-Level Pooling for Text-Independent Speaker Verification",
            "authors": [
                "Yun Tang",
                "Guohong Ding",
                "Jing Huang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "This paper aims to improve the widely used deep speaker embedding x-vector\nmodel. We propose the following improvements: (1) a hybrid neural network\nstructure using both time delay neural network (TDNN) and long short-term\nmemory neural networks (LSTM) to generate complementary speaker information at\ndifferent levels; (2) a multi-level pooling strategy to collect speaker\ninformation from both TDNN and LSTM layers; (3) a regularization scheme on the\nspeaker embedding extraction layer to make the extracted embeddings suitable\nfor the following fusion step. The synergy of these improvements are shown on\nthe NIST SRE 2016 eval test (with a 19% EER reduction) and SRE 2018 dev test\n(with a 9% EER reduction), as well as more than 10% DCF scores reduction on\nthese two test sets over the x-vector baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.07867v2",
            "date": "2019-02-21 05:16:45+00:00",
            "title": "ntuer at SemEval-2019 Task 3: Emotion Classification with Word and Sentence Representations in RCNN",
            "authors": [
                "Peixiang Zhong",
                "Chunyan Miao"
            ],
            "abstract": "In this paper we present our model on the task of emotion detection in\ntextual conversations in SemEval-2019. Our model extends the Recurrent\nConvolutional Neural Network (RCNN) by using external fine-tuned word\nrepresentations and DeepMoji sentence representations. We also explored several\nother competitive pre-trained word and sentence representations including ELMo,\nBERT and InferSent but found inferior performance. In addition, we conducted\nextensive sensitivity analysis, which empirically shows that our model is\nrelatively robust to hyper-parameters. Our model requires no handcrafted\nfeatures or emotion lexicons but achieved good performance with a micro-F1\nscore of 0.7463.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08213v1",
            "date": "2019-02-21 19:00:30+00:00",
            "title": "Towards Visually Grounded Sub-Word Speech Unit Discovery",
            "authors": [
                "David Harwath",
                "James Glass"
            ],
            "abstract": "In this paper, we investigate the manner in which interpretable sub-word\nspeech units emerge within a convolutional neural network model trained to\nassociate raw speech waveforms with semantically related natural image scenes.\nWe show how diphone boundaries can be superficially extracted from the\nactivation patterns of intermediate layers of the model, suggesting that the\nmodel may be leveraging these events for the purpose of word recognition. We\npresent a series of experiments investigating the information encoded by these\nevents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08355v1",
            "date": "2019-02-22 03:46:53+00:00",
            "title": "Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation",
            "authors": [
                "Sang-Woo Lee",
                "Tong Gao",
                "Sohee Yang",
                "Jaejun Yoo",
                "Jung-Woo Ha"
            ],
            "abstract": "Answerer in Questioner's Mind (AQM) is an information-theoretic framework\nthat has been recently proposed for task-oriented dialog systems. AQM benefits\nfrom asking a question that would maximize the information gain when it is\nasked. However, due to its intrinsic nature of explicitly calculating the\ninformation gain, AQM has a limitation when the solution space is very large.\nTo address this, we propose AQM+ that can deal with a large-scale problem and\nask a question that is more coherent to the current context of the dialog. We\nevaluate our method on GuessWhich, a challenging task-oriented visual dialog\nproblem, where the number of candidate classes is near 10K. Our experimental\nresults and ablation studies show that AQM+ outperforms the state-of-the-art\nmodels by a remarkable margin with a reasonable approximation. In particular,\nthe proposed AQM+ reduces more than 60% of error as the dialog proceeds, while\nthe comparative algorithms diminish the error by less than 6%. Based on our\nresults, we argue that AQM+ is a general task-oriented dialog algorithm that\ncan be applied for non-yes-or-no responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.08628v1",
            "date": "2019-02-22 19:00:10+00:00",
            "title": "Trajectories of Blocked Community Members: Redemption, Recidivism and Departure",
            "authors": [
                "Jonathan P. Chang",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Community norm violations can impair constructive communication and\ncollaboration online. As a defense mechanism, community moderators often\naddress such transgressions by temporarily blocking the perpetrator. Such\nactions, however, come with the cost of potentially alienating community\nmembers. Given this tradeoff, it is essential to understand to what extent, and\nin which situations, this common moderation practice is effective in\nreinforcing community rules.\n  In this work, we introduce a computational framework for studying the future\nbehavior of blocked users on Wikipedia. After their block expires, they can\ntake several distinct paths: they can reform and adhere to the rules, but they\ncan also recidivate, or straight-out abandon the community. We reveal that\nthese trajectories are tied to factors rooted both in the characteristics of\nthe blocked individual and in whether they perceived the block to be fair and\njustified. Based on these insights, we formulate a series of prediction tasks\naiming to determine which of these paths a user is likely to take after being\nblocked for their first offense, and demonstrate the feasibility of these new\ntasks. Overall, this work builds towards a more nuanced approach to moderation\nby highlighting the tradeoffs that are in play.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1902.08649v3",
            "date": "2019-02-22 19:38:36+00:00",
            "title": "Saliency Learning: Teaching the Model Where to Pay Attention",
            "authors": [
                "Reza Ghaeini",
                "Xiaoli Z. Fern",
                "Hamed Shahbazi",
                "Prasad Tadepalli"
            ],
            "abstract": "Deep learning has emerged as a compelling solution to many NLP tasks with\nremarkable performances. However, due to their opacity, such models are hard to\ninterpret and trust. Recent work on explaining deep models has introduced\napproaches to provide insights toward the model's behaviour and predictions,\nwhich are helpful for assessing the reliability of the model's predictions.\nHowever, such methods do not improve the model's reliability. In this paper, we\naim to teach the model to make the right prediction for the right reason by\nproviding explanation training and ensuring the alignment of the model's\nexplanation with the ground truth explanation. Our experimental results on\nmultiple tasks and datasets demonstrate the effectiveness of the proposed\nmethod, which produces more reliable predictions while delivering better\nresults compared to traditionally trained models.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09006v3",
            "date": "2019-02-24 20:05:07+00:00",
            "title": "Learning to Perform Role-Filler Binding with Schematic Knowledge",
            "authors": [
                "Catherine Chen",
                "Qihong Lu",
                "Andre Beukers",
                "Christopher Baldassano",
                "Kenneth A. Norman"
            ],
            "abstract": "Through specific experiences, humans learn relationships underlying the\nstructure of events in the world. Schema theory suggests that we organize this\ninformation in mental frameworks called \"schemata,\" which represent our\nknowledge of the structure of the world. Generalizing knowledge of structural\nrelationships to new situations requires role-filler binding, the ability to\nassociate specific \"fillers\" with abstract \"roles.\" For instance, when we hear\nthe sentence \"Alice ordered a tea from Bob,\" the role-filler bindings\n\"Alice:customer,\" \"tea:drink,\" and \"Bob:barista\" allow us to understand and\nmake inferences about the sentence. We can perform these bindings for arbitrary\nfillers -- we understand this sentence even if we have never heard the names\n\"Alice,\" \"tea,\" or \"Bob\" before. In this work, we define a model as capable of\nperforming role-filler binding if it can recall arbitrary fillers corresponding\nto a specified role, even when these pairings violate correlations seen during\ntraining. Previous work found that models can learn this ability when\nexplicitly told what the roles and fillers are, or when given fillers seen\nduring training. We show that networks with external memory can learn these\nrelationships with fillers not seen during training and without explicitly\nlabeled role-filler bindings, and show that analyses inspired by neural\ndecoding can provide a means of understanding what the networks have learned.",
            "journal ref": "PeerJ 9:e11046 (2021)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1902.09091v1",
            "date": "2019-02-25 05:04:00+00:00",
            "title": "Leveraging Knowledge Bases in LSTMs for Improving Machine Reading",
            "authors": [
                "Bishan Yang",
                "Tom Mitchell"
            ],
            "abstract": "This paper focuses on how to take advantage of external knowledge bases (KBs)\nto improve recurrent neural networks for machine reading. Traditional methods\nthat exploit knowledge from KBs encode knowledge as discrete indicator\nfeatures. Not only do these features generalize poorly, but they require\ntask-specific feature engineering to achieve good performance. We propose\nKBLSTM, a novel neural model that leverages continuous representations of KBs\nto enhance the learning of recurrent neural networks for machine reading. To\neffectively integrate background knowledge with information from the currently\nprocessed text, our model employs an attention mechanism with a sentinel to\nadaptively decide whether to attend to background knowledge and which\ninformation from KBs is useful. Experimental results show that our model\nachieves accuracies that surpass the previous state-of-the-art results for both\nentity extraction and event extraction on the widely used ACE2005 dataset.",
            "journal ref": "ACL 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09191v1",
            "date": "2019-02-25 10:53:29+00:00",
            "title": "Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss",
            "authors": [
                "Shaojie Jiang",
                "Pengjie Ren",
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "Sequence-to-Sequence (Seq2Seq) models have achieved encouraging performance\non the dialogue response generation task. However, existing Seq2Seq-based\nresponse generation methods suffer from a low-diversity problem: they\nfrequently generate generic responses, which make the conversation less\ninteresting. In this paper, we address the low-diversity problem by\ninvestigating its connection with model over-confidence reflected in predicted\ndistributions. Specifically, we first analyze the influence of the commonly\nused Cross-Entropy (CE) loss function, and find that the CE loss function\nprefers high-frequency tokens, which results in low-diversity responses. We\nthen propose a Frequency-Aware Cross-Entropy (FACE) loss function that improves\nover the CE loss function by incorporating a weighting mechanism conditioned on\ntoken frequency. Extensive experiments on benchmark datasets show that the FACE\nloss function is able to substantially improve the diversity of existing\nstate-of-the-art Seq2Seq response generation methods, in terms of both\nautomatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.09254v4",
            "date": "2019-02-25 13:27:13+00:00",
            "title": "Audio Caption: Listen and Tell",
            "authors": [
                "Mengyue Wu",
                "Heinrich Dinkel",
                "Kai Yu"
            ],
            "abstract": "Increasing amount of research has shed light on machine perception of audio\nevents, most of which concerns detection and classification tasks. However,\nhuman-like perception of audio scenes involves not only detecting and\nclassifying audio sounds, but also summarizing the relationship between\ndifferent audio events. Comparable research such as image caption has been\nconducted, yet the audio field is still quite barren. This paper introduces a\nmanually-annotated dataset for audio caption. The purpose is to automatically\ngenerate natural sentences for audio scene description and to bridge the gap\nbetween machine perception of audio and image. The whole dataset is labelled in\nMandarin and we also include translated English annotations. A baseline\nencoder-decoder model is provided for both English and Mandarin. Similar BLEU\nscores are derived for both languages: our model can generate understandable\nand data-related captions based on the dataset.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1902.09393v2",
            "date": "2019-02-25 15:56:34+00:00",
            "title": "Cooperative Learning of Disjoint Syntax and Semantics",
            "authors": [
                "Serhii Havrylov",
                "Germ\u00e1n Kruszewski",
                "Armand Joulin"
            ],
            "abstract": "There has been considerable attention devoted to models that learn to jointly\ninfer an expression's syntactic structure and its semantics. Yet,\n\\citet{NangiaB18} has recently shown that the current best systems fail to\nlearn the correct parsing strategy on mathematical expressions generated from a\nsimple context-free grammar. In this work, we present a recursive model\ninspired by \\newcite{ChoiYL18} that reaches near perfect accuracy on this task.\nOur model is composed of two separated modules for syntax and semantics. They\nare cooperatively trained with standard continuous and discrete optimization\nschemes. Our model does not require any linguistic structure for supervision\nand its recursive nature allows for out-of-domain generalization with little\nloss in performance. Additionally, our approach performs competitively on\nseveral natural language tasks, such as Natural Language Inference or Sentiment\nAnalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09487v1",
            "date": "2019-02-25 18:04:05+00:00",
            "title": "MUREL: Multimodal Relational Reasoning for Visual Question Answering",
            "authors": [
                "Remi Cadene",
                "Hedi Ben-younes",
                "Matthieu Cord",
                "Nicolas Thome"
            ],
            "abstract": "Multimodal attentional networks are currently state-of-the-art models for\nVisual Question Answering (VQA) tasks involving real images. Although attention\nallows to focus on the visual content relevant to the question, this simple\nmechanism is arguably insufficient to model complex reasoning features required\nfor VQA or other high-level tasks.\n  In this paper, we propose MuRel, a multimodal relational network which is\nlearned end-to-end to reason over real images. Our first contribution is the\nintroduction of the MuRel cell, an atomic reasoning primitive representing\ninteractions between question and image regions by a rich vectorial\nrepresentation, and modeling region relations with pairwise combinations.\nSecondly, we incorporate the cell into a full MuRel network, which\nprogressively refines visual and question interactions, and can be leveraged to\ndefine visualization schemes finer than mere attention maps.\n  We validate the relevance of our approach with various ablation studies, and\nshow its superiority to attention-based methods on three datasets: VQA 2.0,\nVQA-CP v2 and TDIUC. Our final MuRel network is competitive to or outperforms\nstate-of-the-art results in this challenging context.\n  Our code is available: https://github.com/Cadene/murel.bootstrap.pytorch",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1902.09506v3",
            "date": "2019-02-25 18:37:49+00:00",
            "title": "GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering",
            "authors": [
                "Drew A. Hudson",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce GQA, a new dataset for real-world visual reasoning and\ncompositional question answering, seeking to address key shortcomings of\nprevious VQA datasets. We have developed a strong and robust question engine\nthat leverages scene graph structures to create 22M diverse reasoning\nquestions, all come with functional programs that represent their semantics. We\nuse the programs to gain tight control over the answer distribution and present\na new tunable smoothing technique to mitigate question biases. Accompanying the\ndataset is a suite of new metrics that evaluate essential qualities such as\nconsistency, grounding and plausibility. An extensive analysis is performed for\nbaselines as well as state-of-the-art models, providing fine-grained results\nfor different question types and topologies. Whereas a blind LSTM obtains mere\n42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%,\noffering ample opportunity for new research to explore. We strongly hope GQA\nwill provide an enabling resource for the next generation of models with\nenhanced robustness, improved consistency, and deeper semantic understanding\nfor images and language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09713v2",
            "date": "2019-02-26 02:54:03+00:00",
            "title": "Interpretable Structure-aware Document Encoders with Hierarchical Attention",
            "authors": [
                "Khalil Mrini",
                "Claudiu Musat",
                "Michael Baeriswyl",
                "Martin Jaggi"
            ],
            "abstract": "We propose a method to create document representations that reflect their\ninternal structure. We modify Tree-LSTMs to hierarchically merge basic elements\nsuch as words and sentences into blocks of increasing complexity. Our Structure\nTree-LSTM implements a hierarchical attention mechanism over individual\ncomponents and combinations thereof. We thus emphasize the usefulness of\nTree-LSTMs for texts larger than a sentence. We show that structure-aware\nencoders can be used to improve the performance of document classification. We\ndemonstrate that our method is resilient to changes to the basic building\nblocks, as it performs well with both sentence and word embeddings. The\nStructure Tree-LSTM outperforms all the baselines on two datasets by leveraging\nstructural clues. We show our model's interpretability by visualizing how our\nmodel distributes attention inside a document. On a third dataset from the\nmedical domain, our model achieves competitive performance with the state of\nthe art. This result shows the Structure Tree-LSTM can leverage dependency\nrelations other than text structure, such as a set of reports on the same\npatient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.09859v1",
            "date": "2019-02-26 11:09:29+00:00",
            "title": "Context Vectors are Reflections of Word Vectors in Half the Dimensions",
            "authors": [
                "Zhenisbek Assylbekov",
                "Rustem Takhanov"
            ],
            "abstract": "This paper takes a step towards theoretical analysis of the relationship\nbetween word embeddings and context embeddings in models such as word2vec. We\nstart from basic probabilistic assumptions on the nature of word vectors,\ncontext vectors, and text generation. These assumptions are well supported\neither empirically or theoretically by the existing literature. Next, we show\nthat under these assumptions the widely-used word-word PMI matrix is\napproximately a random symmetric Gaussian ensemble. This, in turn, implies that\ncontext vectors are reflections of word vectors in approximately half the\ndimensions. As a direct application of our result, we suggest a theoretically\ngrounded way of tying weights in the SGNS model.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1902.09969v1",
            "date": "2019-02-25 18:40:25+00:00",
            "title": "Using Deep Object Features for Image Descriptions",
            "authors": [
                "Ashutosh Mishra",
                "Marcus Liwicki"
            ],
            "abstract": "Inspired by recent advances in leveraging multiple modalities in machine\ntranslation, we introduce an encoder-decoder pipeline that uses (1) specific\nobjects within an image and their object labels, (2) a language model for\ndecoding joint embedding of object features and the object labels. Our pipeline\nmerges prior detected objects from the image and their object labels and then\nlearns the sequences of captions describing the particular image. The decoder\nmodel learns to extract descriptions for the image from scratch by decoding the\njoint representation of the object visual features and their object classes\nconditioned by the encoder component. The idea of the model is to concentrate\nonly on the specific objects of the image and their labels for generating\ndescriptions of the image rather than visual feature of the entire image. The\nmodel needs to be calibrated more by adjusting the parameters and settings to\nresult in better accuracy and performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1902.10031v1",
            "date": "2019-02-26 16:22:15+00:00",
            "title": "A framework for information extraction from tables in biomedical literature",
            "authors": [
                "Nikola Milosevic",
                "Cassie Gregson",
                "Robert Hernandez",
                "Goran Nenadic"
            ],
            "abstract": "The scientific literature is growing exponentially, and professionals are no\nmore able to cope with the current amount of publications. Text mining provided\nin the past methods to retrieve and extract information from text; however,\nmost of these approaches ignored tables and figures. The research done in\nmining table data still does not have an integrated approach for mining that\nwould consider all complexities and challenges of a table. Our research is\nexamining the methods for extracting numerical (number of patients, age, gender\ndistribution) and textual (adverse reactions) information from tables in the\nclinical literature. We present a requirement analysis template and an integral\nmethodology for information extraction from tables in clinical domain that\ncontains 7 steps: (1) table detection, (2) functional processing, (3)\nstructural processing, (4) semantic tagging, (5) pragmatic processing, (6) cell\nselection and (7) syntactic processing and extraction. Our approach performed\nwith the F-measure ranged between 82 and 92%, depending on the variable, task\nand its complexity.",
            "journal ref": "2019, International Journal on Document Analysis and Recognition\n  (IJDAR)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10126v2",
            "date": "2019-02-25 19:53:01+00:00",
            "title": "BUT-FIT at SemEval-2019 Task 7: Determining the Rumour Stance with Pre-Trained Deep Bidirectional Transformers",
            "authors": [
                "Martin Fajcik",
                "Luk\u00e1\u0161 Burget",
                "Pavel Smrz"
            ],
            "abstract": "This paper describes our system submitted to SemEval 2019 Task 7: RumourEval\n2019: Determining Rumour Veracity and Support for Rumours, Subtask A (Gorrell\net al., 2019). The challenge focused on classifying whether posts from Twitter\nand Reddit support, deny, query, or comment a hidden rumour, truthfulness of\nwhich is the topic of an underlying discussion thread. We formulate the problem\nas a stance classification, determining the rumour stance of a post with\nrespect to the previous thread post and the source thread post. The recent BERT\narchitecture was employed to build an end-to-end system which has reached the\nF1 score of 61.67% on the provided test data. It finished at the 2nd place in\nthe competition, without any hand-crafted features, only 0.2% behind the\nwinner.",
            "journal ref": "Proceedings of the 13th International Workshop on Semantic\n  Evaluation 13 (2019) 1097-1104",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10197v1",
            "date": "2019-02-26 20:15:09+00:00",
            "title": "RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space",
            "authors": [
                "Zhiqing Sun",
                "Zhi-Hong Deng",
                "Jian-Yun Nie",
                "Jian Tang"
            ],
            "abstract": "We study the problem of learning representations of entities and relations in\nknowledge graphs for predicting missing links. The success of such a task\nheavily relies on the ability of modeling and inferring the patterns of (or\nbetween) the relations. In this paper, we present a new approach for knowledge\ngraph embedding called RotatE, which is able to model and infer various\nrelation patterns including: symmetry/antisymmetry, inversion, and composition.\nSpecifically, the RotatE model defines each relation as a rotation from the\nsource entity to the target entity in the complex vector space. In addition, we\npropose a novel self-adversarial negative sampling technique for efficiently\nand effectively training the RotatE model. Experimental results on multiple\nbenchmark knowledge graphs show that the proposed RotatE model is not only\nscalable, but also able to infer and model various relation patterns and\nsignificantly outperform existing state-of-the-art models for link prediction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.10245v1",
            "date": "2019-02-22 02:37:15+00:00",
            "title": "Non-Autoregressive Machine Translation with Auxiliary Regularization",
            "authors": [
                "Yiren Wang",
                "Fei Tian",
                "Di He",
                "Tao Qin",
                "ChengXiang Zhai",
                "Tie-Yan Liu"
            ],
            "abstract": "As a new neural machine translation approach, Non-Autoregressive machine\nTranslation (NAT) has attracted attention recently due to its high efficiency\nin inference. However, the high efficiency has come at the cost of not\ncapturing the sequential dependency on the target side of translation, which\ncauses NAT to suffer from two kinds of translation errors: 1) repeated\ntranslations (due to indistinguishable adjacent decoder hidden states), and 2)\nincomplete translations (due to incomplete transfer of source side information\nvia the decoder hidden states).\n  In this paper, we propose to address these two problems by improving the\nquality of decoder hidden representations via two auxiliary regularization\nterms in the training process of an NAT model. First, to make the hidden states\nmore distinguishable, we regularize the similarity between consecutive hidden\nstates based on the corresponding target tokens. Second, to force the hidden\nstates to contain all the information in the source sentence, we leverage the\ndual nature of translation tasks (e.g., English to German and German to\nEnglish) and minimize a backward reconstruction error to ensure that the hidden\nstates of the NAT decoder are able to recover the source side sentence.\nExtensive experiments conducted on several benchmark datasets show that both\nregularization strategies are effective and can alleviate the issues of\nrepeated translations and incomplete translations in NAT models. The accuracy\nof NAT models is therefore improved significantly over the state-of-the-art NAT\nmodels with even better efficiency for inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10246v1",
            "date": "2019-02-23 00:22:58+00:00",
            "title": "Fixed-Size Ordinally Forgetting Encoding Based Word Sense Disambiguation",
            "authors": [
                "Xi Zhu",
                "Mingbin Xu",
                "Hui Jiang"
            ],
            "abstract": "In this paper, we present our method of using fixed-size ordinally forgetting\nencoding (FOFE) to solve the word sense disambiguation (WSD) problem. FOFE\nenables us to encode variable-length sequence of words into a theoretically\nunique fixed-size representation that can be fed into a feed forward neural\nnetwork (FFNN), while keeping the positional information between words. In our\nmethod, a FOFE-based FFNN is used to train a pseudo language model over\nunlabelled corpus, then the pre-trained language model is capable of\nabstracting the surrounding context of polyseme instances in labelled corpus\ninto context embeddings. Next, we take advantage of these context embeddings\ntowards WSD classification. We conducted experiments on several WSD data sets,\nwhich demonstrates that our proposed method can achieve comparable performance\nto that of the state-of-the-art approach at the expense of much lower\ncomputational cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10247v3",
            "date": "2019-02-23 16:38:35+00:00",
            "title": "Leveraging Deep Graph-Based Text Representation for Sentiment Polarity Applications",
            "authors": [
                "Kayvan Bijari",
                "Hadi Zare",
                "Emad Kebriaei",
                "Hadi Veisi"
            ],
            "abstract": "Over the last few years, machine learning over graph structures has\nmanifested a significant enhancement in text mining applications such as event\ndetection, opinion mining, and news recommendation. One of the primary\nchallenges in this regard is structuring a graph that encodes and encompasses\nthe features of textual data for the effective machine learning algorithm.\nBesides, exploration and exploiting of semantic relations is regarded as a\nprincipal step in text mining applications. However, most of the traditional\ntext mining methods perform somewhat poor in terms of employing such relations.\nIn this paper, we propose a sentence-level graph-based text representation\nwhich includes stop words to consider semantic and term relations. Then, we\nemploy a representation learning approach on the combined graphs of sentences\nto extract the latent and continuous features of the documents. Eventually, the\nlearned features of the documents are fed into a deep neural network for the\nsentiment classification task. The experimental results demonstrate that the\nproposed method substantially outperforms the related sentiment analysis\napproaches based on several benchmark datasets. Furthermore, our method can be\ngeneralized on different datasets without any dependency on pre-trained word\nembeddings.",
            "journal ref": "Expert Systems with Applications Volume 144, 15 April 2020, 113090",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.10525v2",
            "date": "2019-02-22 12:33:38+00:00",
            "title": "Fast Multi-language LSTM-based Online Handwriting Recognition",
            "authors": [
                "Victor Carbune",
                "Pedro Gonnet",
                "Thomas Deselaers",
                "Henry A. Rowley",
                "Alexander Daryin",
                "Marcos Calvo",
                "Li-Lun Wang",
                "Daniel Keysers",
                "Sandro Feuz",
                "Philippe Gervais"
            ],
            "abstract": "We describe an online handwriting system that is able to support 102\nlanguages using a deep neural network architecture. This new system has\ncompletely replaced our previous Segment-and-Decode-based system and reduced\nthe error rate by 20%-40% relative for most languages. Further, we report new\nstate-of-the-art results on IAM-OnDB for both the open and closed dataset\nsetting. The system combines methods from sequence recognition with a new input\nencoding using B\\'ezier curves. This leads to up to 10x faster recognition\ntimes compared to our previous system. Through a series of experiments we\ndetermine the optimal configuration of our models and report the results of our\nsetup on a number of additional public datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11004v1",
            "date": "2019-02-28 10:46:54+00:00",
            "title": "Global Vectors for Node Representations",
            "authors": [
                "Robin Brochier",
                "Adrien Guille",
                "Julien Velcin"
            ],
            "abstract": "Most network embedding algorithms consist in measuring co-occurrences of\nnodes via random walks then learning the embeddings using Skip-Gram with\nNegative Sampling. While it has proven to be a relevant choice, there are\nalternatives, such as GloVe, which has not been investigated yet for network\nembedding. Even though SGNS better handles non co-occurrence than GloVe, it has\na worse time-complexity. In this paper, we propose a matrix factorization\napproach for network embedding, inspired by GloVe, that better handles non\nco-occurrence with a competitive time-complexity. We also show how to extend\nthis model to deal with networks where nodes are documents, by simultaneously\nlearning word, node and document representations. Quantitative evaluations show\nthat our model achieves state-of-the-art performance, while not being so\nsensitive to the choice of hyper-parameters. Qualitatively speaking, we show\nhow our model helps exploring a network of documents by generating\ncomplementary network-oriented and content-oriented keywords.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11054v2",
            "date": "2019-02-28 12:45:42+00:00",
            "title": "Link Prediction with Mutual Attention for Text-Attributed Networks",
            "authors": [
                "Robin Brochier",
                "Adrien Guille",
                "Julien Velcin"
            ],
            "abstract": "In this extended abstract, we present an algorithm that learns a similarity\nmeasure between documents from the network topology of a structured corpus. We\nleverage the Scaled Dot-Product Attention, a recently proposed attention\nmechanism, to design a mutual attention mechanism between pairs of documents.\nTo train its parameters, we use the network links as supervision. We provide\npreliminary experiment results with a citation dataset on two prediction tasks,\ndemonstrating the capacity of our model to learn a meaningful textual\nsimilarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11058v1",
            "date": "2019-02-28 12:53:38+00:00",
            "title": "Representation Learning for Recommender Systems with Application to the Scientific Literature",
            "authors": [
                "Robin Brochier"
            ],
            "abstract": "The scientific literature is a large information network linking various\nactors (laboratories, companies, institutions, etc.). The vast amount of data\ngenerated by this network constitutes a dynamic heterogeneous attributed\nnetwork (HAN), in which new information is constantly produced and from which\nit is increasingly difficult to extract content of interest. In this article, I\npresent my first thesis works in partnership with an industrial company,\nDigital Scientific Research Technology. This later offers a scientific watch\ntool, Peerus, addressing various issues, such as the real time recommendation\nof newly published papers or the search for active experts to start new\ncollaborations. To tackle this diversity of applications, a common approach\nconsists in learning representations of the nodes and attributes of this HAN\nand use them as features for a variety of recommendation tasks. However, most\nworks on attributed network embedding pay too little attention to textual\nattributes and do not fully take advantage of recent natural language\nprocessing techniques. Moreover, proposed methods that jointly learn node and\ndocument representations do not provide a way to effectively infer\nrepresentations for new documents for which network information is missing,\nwhich happens to be crucial in real time recommender systems. Finally, the\ninterplay between textual and graph data in text-attributed heterogeneous\nnetworks remains an open research direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1902.11116v1",
            "date": "2019-02-28 14:50:59+00:00",
            "title": "Citation Needed: A Taxonomy and Algorithmic Assessment of Wikipedia's Verifiability",
            "authors": [
                "Miriam Redi",
                "Besnik Fetahu",
                "Jonathan Morgan",
                "Dario Taraborelli"
            ],
            "abstract": "Wikipedia is playing an increasingly central role on the web,and the policies\nits contributors follow when sourcing and fact-checking content affect million\nof readers. Among these core guiding principles, verifiability policies have a\nparticularly important role. Verifiability requires that information included\nin a Wikipedia article be corroborated against reliable secondary sources.\nBecause of the manual labor needed to curate and fact-check Wikipedia at scale,\nhowever, its contents do not always evenly comply with these policies.\nCitations (i.e. reference to external sources) may not conform to verifiability\nrequirements or may be missing altogether, potentially weakening the\nreliability of specific topic areas of the free encyclopedia. In this paper, we\naim to provide an empirical characterization of the reasons why and how\nWikipedia cites external sources to comply with its own verifiability\nguidelines. First, we construct a taxonomy of reasons why inline citations are\nrequired by collecting labeled data from editors of multiple Wikipedia language\neditions. We then collect a large-scale crowdsourced dataset of Wikipedia\nsentences annotated with categories derived from this taxonomy. Finally, we\ndesign and evaluate algorithmic models to determine if a statement requires a\ncitation, and to predict the citation reason based on our taxonomy. We evaluate\nthe robustness of such models across different classes of Wikipedia articles of\nvarying quality, as well as on an additional dataset of claims annotated for\nfact-checking purposes.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1902.11245v1",
            "date": "2019-02-28 17:48:20+00:00",
            "title": "Incorporating End-to-End Speech Recognition Models for Sentiment Analysis",
            "authors": [
                "Egor Lakomkin",
                "Mohammad Ali Zamani",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Previous work on emotion recognition demonstrated a synergistic effect of\ncombining several modalities such as auditory, visual, and transcribed text to\nestimate the affective state of a speaker. Among these, the linguistic modality\nis crucial for the evaluation of an expressed emotion. However, manually\ntranscribed spoken text cannot be given as input to a system practically. We\nargue that using ground-truth transcriptions during training and evaluation\nphases leads to a significant discrepancy in performance compared to real-world\nconditions, as the spoken text has to be recognized on the fly and can contain\nspeech recognition mistakes. In this paper, we propose a method of integrating\nan automatic speech recognition (ASR) output with a character-level recurrent\nneural network for sentiment recognition. In addition, we conduct several\nexperiments investigating sentiment recognition for human-robot interaction in\na noise-realistic scenario which is challenging for the ASR systems. We\nquantify the improvement compared to using only the acoustic modality in\nsentiment recognition. We demonstrate the effectiveness of this approach on the\nMultimodal Corpus of Sentiment Intensity (MOSI) by achieving 73,6% accuracy in\na binary sentiment classification task, exceeding previously reported results\nthat use only acoustic input. In addition, we set a new state-of-the-art\nperformance on the MOSI dataset (80.4% accuracy, 2% absolute improvement).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00072v1",
            "date": "2019-01-01 01:17:26+00:00",
            "title": "Exploring spectro-temporal features in end-to-end convolutional neural networks",
            "authors": [
                "Sean Robertson",
                "Gerald Penn",
                "Yingxue Wang"
            ],
            "abstract": "Triangular, overlapping Mel-scaled filters (\"f-banks\") are the current\nstandard input for acoustic models that exploit their input's time-frequency\ngeometry, because they provide a psycho-acoustically motivated time-frequency\ngeometry for a speech signal. F-bank coefficients are provably robust to small\ndeformations in the scale. In this paper, we explore two ways in which filter\nbanks can be adjusted for the purposes of speech recognition. First, triangular\nfilters can be replaced with Gabor filters, a compactly supported filter that\nbetter localizes events in time, or Gammatone filters, a\npsychoacoustically-motivated filter. Second, by rearranging the order of\noperations in computing filter bank features, features can be integrated over\nsmaller time scales while simultaneously providing better frequency resolution.\nWe make all feature implementations available online through open-source\nrepositories. Initial experimentation with a modern end-to-end CNN phone\nrecognizer yielded no significant improvements to phone error rate due to\neither modification. The result, and its ramifications with respect to learned\nfilter banks, is discussed.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.00158v2",
            "date": "2019-01-01 14:41:17+00:00",
            "title": "Text Infilling",
            "authors": [
                "Wanrong Zhu",
                "Zhiting Hu",
                "Eric Xing"
            ],
            "abstract": "Recent years have seen remarkable progress of text generation in different\ncontexts, such as the most common setting of generating text from scratch, and\nthe emerging paradigm of retrieval-and-rewriting. Text infilling, which fills\nmissing text portions of a sentence or paragraph, is also of numerous use in\nreal life, yet is under-explored. Previous work has focused on restricted\nsettings by either assuming single word per missing portion or limiting to a\nsingle missing portion to the end of the text. This paper studies the general\ntask of text infilling, where the input text can have an arbitrary number of\nportions to be filled, each of which may require an arbitrary unknown number of\ntokens. We study various approaches for the task, including a self-attention\nmodel with segment-aware position encoding and bidirectional context modeling.\nWe create extensive supervised data by masking out text with varying\nstrategies. Experiments show the self-attention model greatly outperforms\nothers, creating a strong baseline for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00398v2",
            "date": "2019-01-02 14:45:02+00:00",
            "title": "Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation",
            "authors": [
                "Cristina Garbacea",
                "Samuel Carton",
                "Shiyan Yan",
                "Qiaozhu Mei"
            ],
            "abstract": "We conduct a large-scale, systematic study to evaluate the existing\nevaluation methods for natural language generation in the context of generating\nonline product reviews. We compare human-based evaluators with a variety of\nautomated evaluation procedures, including discriminative evaluators that\nmeasure how well machine-generated text can be distinguished from human-written\ntext, as well as word overlap metrics that assess how similar the generated\ntext compares to human-written references. We determine to what extent these\ndifferent evaluators agree on the ranking of a dozen of state-of-the-art\ngenerators for online product reviews. We find that human evaluators do not\ncorrelate well with discriminative evaluators, leaving a bigger question of\nwhether adversarial accuracy is the correct objective for natural language\ngeneration. In general, distinguishing machine-generated text is challenging\neven for human evaluators, and human decisions correlate better with lexical\noverlaps. We find lexical diversity an intriguing metric that is indicative of\nthe assessments of different evaluators. A post-experiment survey of\nparticipants provides insights into how to evaluate and improve the quality of\nnatural language generation systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.00570v1",
            "date": "2019-01-03 01:06:55+00:00",
            "title": "Event detection in Twitter: A keyword volume approach",
            "authors": [
                "Ahmad Hany Hossny",
                "Lewis Mitchell"
            ],
            "abstract": "Event detection using social media streams needs a set of informative\nfeatures with strong signals that need minimal preprocessing and are highly\nassociated with events of interest. Identifying these informative features as\nkeywords from Twitter is challenging, as people use informal language to\nexpress their thoughts and feelings. This informality includes acronyms,\nmisspelled words, synonyms, transliteration and ambiguous terms. In this paper,\nwe propose an efficient method to select the keywords frequently used in\nTwitter that are mostly associated with events of interest such as protests.\nThe volume of these keywords is tracked in real time to identify the events of\ninterest in a binary classification scheme. We use keywords within word-pairs\nto capture the context. The proposed method is to binarize vectors of daily\ncounts for each word-pair by applying a spike detection temporal filter, then\nuse the Jaccard metric to measure the similarity of the binary vector for each\nword-pair with the binary vector describing event occurrence. The top n\nword-pairs are used as features to classify any day to be an event or non-event\nday. The selected features are tested using multiple classifiers such as Naive\nBayes, SVM, Logistic Regression, KNN and decision trees. They all produced AUC\nROC scores up to 0.91 and F1 scores up to 0.79. The experiment is performed\nusing the English language in multiple cities such as Melbourne, Sydney and\nBrisbane as well as the Indonesian language in Jakarta. The two experiments,\ncomprising different languages and locations, yielded similar results.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1901.00707v2",
            "date": "2019-01-03 13:15:19+00:00",
            "title": "Feature reinforcement with word embedding and parsing information in neural TTS",
            "authors": [
                "Huaiping Ming",
                "Lei He",
                "Haohan Guo",
                "Frank K. Soong"
            ],
            "abstract": "In this paper, we propose a feature reinforcement method under the\nsequence-to-sequence neural text-to-speech (TTS) synthesis framework. The\nproposed method utilizes the multiple input encoder to take three levels of\ntext information, i.e., phoneme sequence, pre-trained word embedding, and\ngrammatical structure of sentences from parser as the input feature for the\nneural TTS system. The added word and sentence level information can be viewed\nas the feature based pre-training strategy, which clearly enhances the model\ngeneralization ability. The proposed method not only improves the system\nrobustness significantly but also improves the synthesized speech to near\nrecording quality in our experiments for out-of-domain text.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1901.00850v2",
            "date": "2019-01-03 18:58:06+00:00",
            "title": "CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions",
            "authors": [
                "Runtao Liu",
                "Chenxi Liu",
                "Yutong Bai",
                "Alan Yuille"
            ],
            "abstract": "Referring object detection and referring image segmentation are important\ntasks that require joint understanding of visual information and natural\nlanguage. Yet there has been evidence that current benchmark datasets suffer\nfrom bias, and current state-of-the-art models cannot be easily evaluated on\ntheir intermediate reasoning process. To address these issues and complement\nsimilar efforts in visual question answering, we build CLEVR-Ref+, a synthetic\ndiagnostic dataset for referring expression comprehension. The precise\nlocations and attributes of the objects are readily available, and the\nreferring expressions are automatically associated with functional programs.\nThe synthetic nature allows control over dataset bias (through sampling\nstrategy), and the modular programs enable intermediate reasoning ground truth\nwithout human annotators.\n  In addition to evaluating several state-of-the-art models on CLEVR-Ref+, we\nalso propose IEP-Ref, a module network approach that significantly outperforms\nother models on our dataset. In particular, we present two interesting and\nimportant findings using IEP-Ref: (1) the module trained to transform feature\nmaps into segmentation masks can be attached to any intermediate module to\nreveal the entire reasoning process step-by-step; (2) even if all training data\nhas at least one object referred, IEP-Ref can correctly predict no-foreground\nwhen presented with false-premise referring expressions. To the best of our\nknowledge, this is the first direct and quantitative proof that neural modules\nbehave in the way they are intended.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1901.01010v2",
            "date": "2019-01-04 08:05:56+00:00",
            "title": "A Joint Model for Multimodal Document Quality Assessment",
            "authors": [
                "Aili Shen",
                "Bahar Salehi",
                "Timothy Baldwin",
                "Jianzhong Qi"
            ],
            "abstract": "The quality of a document is affected by various factors, including\ngrammaticality, readability, stylistics, and expertise depth, making the task\nof document quality assessment a complex one. In this paper, we explore this\ntask in the context of assessing the quality of Wikipedia articles and academic\npapers. Observing that the visual rendering of a document can capture implicit\nquality indicators that are not present in the document text --- such as\nimages, font choices, and visual layout --- we propose a joint model that\ncombines the text content with a visual rendering of the document for document\nquality assessment. Experimental results over two datasets reveal that textual\nand visual features are complementary, achieving state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01216v1",
            "date": "2019-01-01 20:23:40+00:00",
            "title": "Transfer learning from language models to image caption generators: Better models may not transfer better",
            "authors": [
                "Marc Tanti",
                "Albert Gatt",
                "Kenneth P. Camilleri"
            ],
            "abstract": "When designing a neural caption generator, a convolutional neural network can\nbe used to extract image features. Is it possible to also use a neural language\nmodel to extract sentence prefix features? We answer this question by trying\ndifferent ways to transfer the recurrent neural network and embedding layer\nfrom a neural language model to an image caption generator. We find that image\ncaption generators with transferred parameters perform better than those\ntrained from scratch, even when simply pre-training them on the text of the\nsame captions dataset it will later be trained on. We also find that the best\nlanguage models (in terms of perplexity) do not result in the best caption\ngenerators after transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.01592v2",
            "date": "2019-01-06 18:53:12+00:00",
            "title": "Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networks",
            "authors": [
                "Luka Gligic",
                "Andrey Kormilitzin",
                "Paul Goldberg",
                "Alejo Nevado-Holgado"
            ],
            "abstract": "Neural networks (NNs) have become the state of the art in many machine\nlearning applications, especially in image and sound processing [1]. The same,\nalthough to a lesser extent [2,3], could be said in natural language processing\n(NLP) tasks, such as named entity recognition. However, the success of NNs\nremains dependent on the availability of large labelled datasets, which is a\nsignificant hurdle in many important applications. One such case are electronic\nhealth records (EHRs), which are arguably the largest source of medical data,\nmost of which lies hidden in natural text [4,5]. Data access is difficult due\nto data privacy concerns, and therefore annotated datasets are scarce. With\nscarce data, NNs will likely not be able to extract this hidden information\nwith practical accuracy. In our study, we develop an approach that solves these\nproblems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009\nMedical Extraction Challenge [6], 4.3 above the architecture that won the\ncompetition. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on\nextracting relationships between medical terms. To reach this state-of-the-art\naccuracy, our approach applies transfer learning to leverage on datasets\nannotated for other I2B2 tasks, and designs and trains embeddings that\nspecially benefit from such transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02671v1",
            "date": "2019-01-09 10:45:20+00:00",
            "title": "Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks",
            "authors": [
                "Steffen Eger",
                "Paul Youssef",
                "Iryna Gurevych"
            ],
            "abstract": "Activation functions play a crucial role in neural networks because they are\nthe nonlinearities which have been attributed to the success story of deep\nlearning. One of the currently most popular activation functions is ReLU, but\nseveral competitors have recently been proposed or 'discovered', including\nLReLU functions and swish. While most works compare newly proposed activation\nfunctions on few tasks (usually from image classification) and against few\ncompetitors (usually ReLU), we perform the first large-scale comparison of 21\nactivation functions across eight different NLP tasks. We find that a largely\nunknown activation function performs most stably across all tasks, the\nso-called penalized tanh function. We also show that it can successfully\nreplace the sigmoid and tanh gates in LSTM cells, leading to a 2 percentage\npoint (pp) improvement over the standard choices on a challenging NLP task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02780v2",
            "date": "2019-01-09 15:30:39+00:00",
            "title": "Sentiment Analysis of Czech Texts: An Algorithmic Survey",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "In the area of online communication, commerce and transactions, analyzing\nsentiment polarity of texts written in various natural languages has become\ncrucial. While there have been a lot of contributions in resources and studies\nfor the English language, \"smaller\" languages like Czech have not received much\nattention. In this survey, we explore the effectiveness of many existing\nmachine learning algorithms for sentiment analysis of Czech Facebook posts and\nproduct reviews. We report the sets of optimal parameter values for each\nalgorithm and the scores in both datasets. We finally observe that support\nvector machines are the best classifier and efforts to increase performance\neven more with bagging, boosting or voting ensemble schemes fail to do so.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.02860v3",
            "date": "2019-01-09 18:28:19+00:00",
            "title": "Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context",
            "authors": [
                "Zihang Dai",
                "Zhilin Yang",
                "Yiming Yang",
                "Jaime Carbonell",
                "Quoc V. Le",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Transformers have a potential of learning longer-term dependency, but are\nlimited by a fixed-length context in the setting of language modeling. We\npropose a novel neural architecture Transformer-XL that enables learning\ndependency beyond a fixed length without disrupting temporal coherence. It\nconsists of a segment-level recurrence mechanism and a novel positional\nencoding scheme. Our method not only enables capturing longer-term dependency,\nbut also resolves the context fragmentation problem. As a result,\nTransformer-XL learns dependency that is 80% longer than RNNs and 450% longer\nthan vanilla Transformers, achieves better performance on both short and long\nsequences, and is up to 1,800+ times faster than vanilla Transformers during\nevaluation. Notably, we improve the state-of-the-art results of bpc/perplexity\nto 0.99 on enwiki8, 1.08 on text8, 18.3 on WikiText-103, 21.8 on One Billion\nWord, and 54.5 on Penn Treebank (without finetuning). When trained only on\nWikiText-103, Transformer-XL manages to generate reasonably coherent, novel\ntext articles with thousands of tokens. Our code, pretrained models, and\nhyperparameters are available in both Tensorflow and PyTorch.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.03035v1",
            "date": "2019-01-10 06:46:50+00:00",
            "title": "Self-Monitoring Navigation Agent via Auxiliary Progress Estimation",
            "authors": [
                "Chih-Yao Ma",
                "Jiasen Lu",
                "Zuxuan Wu",
                "Ghassan AlRegib",
                "Zsolt Kira",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "The Vision-and-Language Navigation (VLN) task entails an agent following\nnavigational instruction in photo-realistic unknown environments. This\nchallenging task demands that the agent be aware of which instruction was\ncompleted, which instruction is needed next, which way to go, and its\nnavigation progress towards the goal. In this paper, we introduce a\nself-monitoring agent with two complementary components: (1) visual-textual\nco-grounding module to locate the instruction completed in the past, the\ninstruction required for the next action, and the next moving direction from\nsurrounding images and (2) progress monitor to ensure the grounded instruction\ncorrectly reflects the navigation progress. We test our self-monitoring agent\non a standard benchmark and analyze our proposed approach through a series of\nablation studies that elucidate the contributions of the primary components.\nUsing our proposed method, we set the new state of the art by a significant\nmargin (8% absolute increase in success rate on the unseen test set). Code is\navailable at https://github.com/chihyaoma/selfmonitoring-agent .",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1901.03904v4",
            "date": "2019-01-12 21:54:23+00:00",
            "title": "A Speech Act Classifier for Persian Texts and its Application in Identifying Rumors",
            "authors": [
                "Zoleikha Jahanbakhsh-Nagadeh",
                "Mohammad-Reza Feizi-Derakhshi",
                "Arash Sharifi"
            ],
            "abstract": "Speech Acts (SAs) are one of the important areas of pragmatics, which give us\na better understanding of the state of mind of the people and convey an\nintended language function. Knowledge of the SA of a text can be helpful in\nanalyzing that text in natural language processing applications. This study\npresents a dictionary-based statistical technique for Persian SA recognition.\nThe proposed technique classifies a text into seven classes of SA based on four\ncriteria: lexical, syntactic, semantic, and surface features. WordNet as the\ntool for extracting synonym and enriching features dictionary is utilized. To\nevaluate the proposed technique, we utilized four classification methods\nincluding Random Forest (RF), Support Vector Machine (SVM), Naive Bayes (NB),\nand K-Nearest Neighbors (KNN). The experimental results demonstrate that the\nproposed method using RF and SVM as the best classifiers achieved a\nstate-of-the-art performance with an accuracy of 0.95 for classification of\nPersian SAs. Our original vision of this work is introducing an application of\nSA recognition on social media content, especially the common SA in rumors.\nTherefore, the proposed system utilized to determine the common SAs in rumors.\nThe results showed that Persian rumors are often expressed in three SA classes\nincluding narrative, question, and threat, and in some cases with the request\nSA.",
            "journal ref": "Journal of Soft Computing and Information Technology, 9, 1, 1399\n  (2020), 18-27",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04085v5",
            "date": "2019-01-13 23:27:58+00:00",
            "title": "Passage Re-ranking with BERT",
            "authors": [
                "Rodrigo Nogueira",
                "Kyunghyun Cho"
            ],
            "abstract": "Recently, neural models pretrained on a language modeling task, such as ELMo\n(Peters et al., 2017), OpenAI GPT (Radford et al., 2018), and BERT (Devlin et\nal., 2018), have achieved impressive results on various natural language\nprocessing tasks such as question-answering and natural language inference. In\nthis paper, we describe a simple re-implementation of BERT for query-based\npassage re-ranking. Our system is the state of the art on the TREC-CAR dataset\nand the top entry in the leaderboard of the MS MARCO passage retrieval task,\noutperforming the previous state of the art by 27% (relative) in MRR@10. The\ncode to reproduce our results is available at\nhttps://github.com/nyu-dl/dl4marco-bert",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.04140v1",
            "date": "2019-01-14 05:42:51+00:00",
            "title": "Image Based Review Text Generation with Emotional Guidance",
            "authors": [
                "Xuehui Sun",
                "Zihan Zhou",
                "Yuda Fan"
            ],
            "abstract": "In the current field of computer vision, automatically generating texts from\ngiven images has been a fully worked technique. Up till now, most works of this\narea focus on image content describing, namely image-captioning. However, rare\nresearches focus on generating product review texts, which is ubiquitous in the\nonline shopping malls and is crucial for online shopping selection and\nevaluation. Different from content describing, review texts include more\nsubjective information of customers, which may bring difference to the results.\nTherefore, we aimed at a new field concerning generating review text from\ncustomers based on images together with the ratings of online shopping\nproducts, which appear as non-image attributes. We made several adjustments to\nthe existing image-captioning model to fit our task, in which we should also\ntake non-image features into consideration. We also did experiments based on\nour model and get effective primary results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.04276v1",
            "date": "2019-01-14 13:05:48+00:00",
            "title": "Exploring Transfer Learning for Low Resource Emotional TTS",
            "authors": [
                "No\u00e9 Tits",
                "Kevin El Haddad",
                "Thierry Dutoit"
            ],
            "abstract": "During the last few years, spoken language technologies have known a big\nimprovement thanks to Deep Learning. However Deep Learning-based algorithms\nrequire amounts of data that are often difficult and costly to gather.\nParticularly, modeling the variability in speech of different speakers,\ndifferent styles or different emotions with few data remains challenging. In\nthis paper, we investigate how to leverage fine-tuning on a pre-trained Deep\nLearning-based TTS model to synthesize speech with a small dataset of another\nspeaker. Then we investigate the possibility to adapt this model to have\nemotional TTS by fine-tuning the neutral TTS model with a small emotional\ndataset.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1901.05531v1",
            "date": "2019-01-16 21:27:57+00:00",
            "title": "Response to \"Visual Dialogue without Vision or Dialogue\" (Massiceti et al., 2018)",
            "authors": [
                "Abhishek Das",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "In a recent workshop paper, Massiceti et al. presented a baseline model and\nsubsequent critique of Visual Dialog (Das et al., CVPR 2017) that raises what\nwe believe to be unfounded concerns about the dataset and evaluation. This\narticle intends to rebut the critique and clarify potential confusions for\npractitioners and future participants in the Visual Dialog challenge.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1901.06486v1",
            "date": "2019-01-19 09:11:03+00:00",
            "title": "Towards Universal End-to-End Affect Recognition from Multilingual Speech by ConvNets",
            "authors": [
                "Dario Bertero",
                "Onno Kampman",
                "Pascale Fung"
            ],
            "abstract": "We propose an end-to-end affect recognition approach using a Convolutional\nNeural Network (CNN) that handles multiple languages, with applications to\nemotion and personality recognition from speech. We lay the foundation of a\nuniversal model that is trained on multiple languages at once. As affect is\nshared across all languages, we are able to leverage shared information between\nlanguages and improve the overall performance for each one. We obtained an\naverage improvement of 12.8% on emotion and 10.1% on personality when compared\nwith the same model trained on each language only. It is end-to-end because we\ndirectly take narrow-band raw waveforms as input. This allows us to accept as\ninput audio recorded from any source and to avoid the overhead and information\nloss of feature extraction. It outperforms a similar CNN using spectrograms as\ninput by 12.8% for emotion and 6.3% for personality, based on F-scores.\nAnalysis of the network parameters and layers activation shows that the network\nlearns and extracts significant features in the first layer, in particular\npitch, energy and contour variations. Subsequent convolutional layers instead\ncapture language-specific representations through the analysis of\nsupra-segmental features. Our model represents an important step for the\ndevelopment of a fully universal affect recognizer, able to recognize\nadditional descriptors, such as stress, and for the future implementation into\naffective interactive systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.06595v2",
            "date": "2019-01-19 22:12:01+00:00",
            "title": "Evaluating Text-to-Image Matching using Binary Image Selection (BISON)",
            "authors": [
                "Hexiang Hu",
                "Ishan Misra",
                "Laurens van der Maaten"
            ],
            "abstract": "Providing systems the ability to relate linguistic and visual content is one\nof the hallmarks of computer vision. Tasks such as text-based image retrieval\nand image captioning were designed to test this ability but come with\nevaluation measures that have a high variance or are difficult to interpret. We\nstudy an alternative task for systems that match text and images: given a text\nquery, the system is asked to select the image that best matches the query from\na pair of semantically similar images. The system's accuracy on this Binary\nImage SelectiON (BISON) task is interpretable, eliminates the reliability\nproblems of retrieval evaluations, and focuses on the system's ability to\nunderstand fine-grained visual structure. We gather a BISON dataset that\ncomplements the COCO dataset and use it to evaluate modern text-based image\nretrieval and image captioning systems. Our results provide novel insights into\nthe performance of these systems. The COCO-BISON dataset and corresponding\nevaluation code are publicly available from \\url{http://hexianghu.com/bison/}.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1901.06610v2",
            "date": "2019-01-20 01:48:43+00:00",
            "title": "Hierarchical Attentional Hybrid Neural Networks for Document Classification",
            "authors": [
                "Jader Abreu",
                "Luis Fred",
                "David Mac\u00eado",
                "Cleber Zanchettin"
            ],
            "abstract": "Document classification is a challenging task with important applications.\nThe deep learning approaches to the problem have gained much attention\nrecently. Despite the progress, the proposed models do not incorporate the\nknowledge of the document structure in the architecture efficiently and not\ntake into account the contexting importance of words and sentences. In this\npaper, we propose a new approach based on a combination of convolutional neural\nnetworks, gated recurrent units, and attention mechanisms for document\nclassification tasks. The main contribution of this work is the use of\nconvolution layers to extract more meaningful, generalizable and abstract\nfeatures by the hierarchical representation. The proposed method in this paper\nimproves the results of the current attention-based approaches for document\nclassification.",
            "journal ref": "2019 International Conference on Artificial Neural Networks\n  (ICANN)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07002v2",
            "date": "2019-01-21 17:17:06+00:00",
            "title": "Error-Correcting Neural Sequence Prediction",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "We propose a novel neural sequence prediction method based on\n\\textit{error-correcting output codes} that avoids exact softmax normalization\nand allows for a tradeoff between speed and performance. Instead of minimizing\nmeasures between the predicted probability distribution and true distribution,\nwe use error-correcting codes to represent both predictions and outputs.\nSecondly, we propose multiple ways to improve accuracy and convergence rates by\nmaximizing the separability between codes that correspond to classes\nproportional to word embedding similarities. Lastly, we introduce our main\ncontribution called \\textit{Latent Variable Mixture Sampling}, a technique that\nis used to mitigate exposure bias, which can be integrated into training latent\nvariable-based neural sequence predictors such as ECOC. This involves mixing\nthe latent codes of past predictions and past targets in one of two ways: (1)\naccording to a predefined sampling schedule or (2) a differentiable sampling\nprocedure whereby the mixing probability is learned throughout training by\nreplacing the greedy argmax operation with a smooth approximation. ECOC-NSP\nleads to consistent improvements on language modelling datasets and the\nproposed Latent Variable mixture sampling methods are found to perform well for\ntext generation tasks such as image captioning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.07005v3",
            "date": "2019-01-21 17:36:19+00:00",
            "title": "DLocRL: A Deep Learning Pipeline for Fine-Grained Location Recognition and Linking in Tweets",
            "authors": [
                "Canwen Xu",
                "Jing Li",
                "Xiangyang Luo",
                "Jiaxin Pei",
                "Chenliang Li",
                "Donghong Ji"
            ],
            "abstract": "In recent years, with the prevalence of social media and smart devices,\npeople causally reveal their locations such as shops, hotels, and restaurants\nin their tweets. Recognizing and linking such fine-grained location mentions to\nwell-defined location profiles are beneficial for retrieval and recommendation\nsystems. In this paper, we propose DLocRL, a new deep learning pipeline for\nfine-grained location recognition and linking in tweets, and verify its\neffectiveness on a real-world Twitter dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07132v2",
            "date": "2019-01-22 00:54:30+00:00",
            "title": "Universal Rules for Fooling Deep Neural Networks based Text Classification",
            "authors": [
                "Di Li",
                "Danilo Vasconcellos Vargas",
                "Sakurai Kouichi"
            ],
            "abstract": "Recently, deep learning based natural language processing techniques are\nbeing extensively used to deal with spam mail, censorship evaluation in social\nnetworks, among others. However, there is only a couple of works evaluating the\nvulnerabilities of such deep neural networks. Here, we go beyond attacks to\ninvestigate, for the first time, universal rules, i.e., rules that are sample\nagnostic and therefore could turn any text sample in an adversarial one. In\nfact, the universal rules do not use any information from the method itself (no\ninformation from the method, gradient information or training dataset\ninformation is used), making them black-box universal attacks. In other words,\nthe universal rules are sample and method agnostic. By proposing a\ncoevolutionary optimization algorithm we show that it is possible to create\nuniversal rules that can automatically craft imperceptible adversarial samples\n(only less than five perturbations which are close to misspelling are inserted\nin the text sample). A comparison with a random search algorithm further\njustifies the strength of the method. Thus, universal rules for fooling\nnetworks are here shown to exist. Hopefully, the results from this work will\nimpact the development of yet more sample and model agnostic attacks as well as\ntheir defenses, culminating in perhaps a new age for artificial intelligence.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.07910v3",
            "date": "2019-01-23 14:18:26+00:00",
            "title": "NLSC: Unrestricted Natural Language-based Service Composition through Sentence Embeddings",
            "authors": [
                "Oscar J. Romero",
                "Ankit Dangi",
                "Sushma A. Akoju"
            ],
            "abstract": "Current approaches for service composition (assemblies of atomic services)\nrequire developers to use: (a) domain-specific semantics to formalize services\nthat restrict the vocabulary for their descriptions, and (b) translation\nmechanisms for service retrieval to convert unstructured user requests to\nstrongly-typed semantic representations. In our work, we argue that effort to\ndeveloping service descriptions, request translations, and matching mechanisms\ncould be reduced using unrestricted natural language; allowing both: (1)\nend-users to intuitively express their needs using natural language, and (2)\nservice developers to develop services without relying on syntactic/semantic\ndescription languages. Although there are some natural language-based service\ncomposition approaches, they restrict service retrieval to syntactic/semantic\nmatching. With recent developments in Machine learning and Natural Language\nProcessing, we motivate the use of Sentence Embeddings by leveraging richer\nsemantic representations of sentences for service description, matching and\nretrieval. Experimental results show that service composition development\neffort may be reduced by more than 44\\% while keeping a high precision/recall\nwhen matching high-level user requests with low-level service method\ninvocations.",
            "journal ref": "2019 IEEE International Conference on Services Computing (SCC)",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1901.08079v1",
            "date": "2019-01-23 19:02:27+00:00",
            "title": "A Question-Entailment Approach to Question Answering",
            "authors": [
                "Asma Ben Abacha",
                "Dina Demner-Fushman"
            ],
            "abstract": "One of the challenges in large-scale information retrieval (IR) is to develop\nfine-grained and domain-specific methods to answer natural language questions.\nDespite the availability of numerous sources and datasets for answer retrieval,\nQuestion Answering (QA) remains a challenging problem due to the difficulty of\nthe question understanding and answer extraction tasks. One of the promising\ntracks investigated in QA is to map new questions to formerly answered\nquestions that are `similar'. In this paper, we propose a novel QA approach\nbased on Recognizing Question Entailment (RQE) and we describe the QA system\nand resources that we built and evaluated on real medical questions. First, we\ncompare machine learning and deep learning methods for RQE using different\nkinds of datasets, including textual inference, question similarity and\nentailment in both the open and clinical domains. Second, we combine IR models\nwith the best RQE method to select entailed questions and rank the retrieved\nanswers. To study the end-to-end QA approach, we built the MedQuAD collection\nof 47,457 question-answer pairs from trusted medical sources, that we introduce\nand share in the scope of this paper. Following the evaluation process used in\nTREC 2017 LiveQA, we find that our approach exceeds the best results of the\nmedical task with a 29.8% increase over the best official score. The evaluation\nresults also support the relevance of question entailment for QA and highlight\nthe effectiveness of combining IR and RQE for future QA efforts. Our findings\nalso show that relying on a restricted set of reliable answer sources can bring\na substantial improvement in medical QA.",
            "journal ref": "BMC Bioinformatics 20, 511 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08241v1",
            "date": "2019-01-24 05:54:13+00:00",
            "title": "Location reference identification from tweets during emergencies: A deep learning approach",
            "authors": [
                "Abhinav Kumar",
                "Jyoti Prakash Singh"
            ],
            "abstract": "Twitter is recently being used during crises to communicate with officials\nand provide rescue and relief operation in real time. The geographical location\ninformation of the event, as well as users, are vitally important in such\nscenarios. The identification of geographic location is one of the challenging\ntasks as the location information fields, such as user location and place name\nof tweets are not reliable. The extraction of location information from tweet\ntext is difficult as it contains a lot of non-standard English, grammatical\nerrors, spelling mistakes, non-standard abbreviations, and so on. This research\naims to extract location words used in the tweet using a Convolutional Neural\nNetwork (CNN) based model. We achieved the exact matching score of 0.929,\nHamming loss of 0.002, and $F_1$-score of 0.96 for the tweets related to the\nearthquake. Our model was able to extract even three- to four-word long\nlocation references which is also evident from the exact matching score of over\n92\\%. The findings of this paper can help in early event localization,\nemergency situations, real-time road traffic management, localized\nadvertisement, and in various location-based services.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.08422v1",
            "date": "2019-01-24 14:20:11+00:00",
            "title": "Securing Tag-based recommender systems against profile injection attacks: A comparative study. (Extended Report)",
            "authors": [
                "Georgios K. Pitsilis",
                "Heri Ramampiaro",
                "Helge Langseth"
            ],
            "abstract": "This work addresses the challenges related to attacks on collaborative\ntagging systems, which often comes in a form of malicious annotations or\nprofile injection attacks. In particular, we study various countermeasures\nagainst two types of such attacks for social tagging systems, the Overload\nattack and the Piggyback attack. The countermeasure schemes studied here\ninclude baseline classifiers such as, Naive Bayes filter and Support Vector\nMachine, as well as a Deep Learning approach. Our evaluation performed over\nsynthetic spam data generated from del.icio.us dataset, shows that in most\ncases, Deep Learning can outperform the classical solutions, providing\nhigh-level protection against threats.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.08706v2",
            "date": "2019-01-25 01:18:04+00:00",
            "title": "Emergent Linguistic Phenomena in Multi-Agent Communication Games",
            "authors": [
                "Laura Graesser",
                "Kyunghyun Cho",
                "Douwe Kiela"
            ],
            "abstract": "In this work, we propose a computational framework in which agents equipped\nwith communication capabilities simultaneously play a series of referential\ngames, where agents are trained using deep reinforcement learning. We\ndemonstrate that the framework mirrors linguistic phenomena observed in natural\nlanguage: i) the outcome of contact between communities is a function of inter-\nand intra-group connectivity; ii) linguistic contact either converges to the\nmajority protocol, or in balanced cases leads to novel creole languages of\nlower complexity; and iii) a linguistic continuum emerges where neighboring\nlanguages are more mutually intelligible than farther removed languages. We\nconclude that intricate properties of language evolution need not depend on\ncomplex evolved linguistic capabilities, but can emerge from simple social\nexchanges between perceptually-enabled agents playing communication games.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.08759v1",
            "date": "2019-01-25 07:09:14+00:00",
            "title": "Misleading Metadata Detection on YouTube",
            "authors": [
                "Priyank Palod",
                "Ayush Patwari",
                "Sudhanshu Bahety",
                "Saurabh Bagchi",
                "Pawan Goyal"
            ],
            "abstract": "YouTube is the leading social media platform for sharing videos. As a result,\nit is plagued with misleading content that includes staged videos presented as\nreal footages from an incident, videos with misrepresented context and videos\nwhere audio/video content is morphed. We tackle the problem of detecting such\nmisleading videos as a supervised classification task. We develop UCNet - a\ndeep network to detect fake videos and perform our experiments on two datasets\n- VAVD created by us and publicly available FVC [8]. We achieve a macro\naveraged F-score of 0.82 while training and testing on a 70:30 split of FVC,\nwhile the baseline model scores 0.36. We find that the proposed model\ngeneralizes well when trained on one dataset and tested on the other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09069v1",
            "date": "2019-01-25 20:31:02+00:00",
            "title": "Word Embeddings: A Survey",
            "authors": [
                "Felipe Almeida",
                "Geraldo Xex\u00e9o"
            ],
            "abstract": "This work lists and describes the main recent strategies for building\nfixed-length, dense and distributed representations for words, based on the\ndistributional hypothesis. These representations are now commonly called word\nembeddings and, in addition to encoding surprisingly good syntactic and\nsemantic information, have been proven useful as extra features in many\ndownstream NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09102v1",
            "date": "2019-01-25 22:12:39+00:00",
            "title": "On Learning Meaningful Code Changes via Neural Machine Translation",
            "authors": [
                "Michele Tufano",
                "Jevgenija Pantiuchina",
                "Cody Watson",
                "Gabriele Bavota",
                "Denys Poshyvanyk"
            ],
            "abstract": "Recent years have seen the rise of Deep Learning (DL) techniques applied to\nsource code. Researchers have exploited DL to automate several development and\nmaintenance tasks, such as writing commit messages, generating comments and\ndetecting vulnerabilities among others. One of the long lasting dreams of\napplying DL to source code is the possibility to automate non-trivial coding\nactivities. While some steps in this direction have been taken (e.g., learning\nhow to fix bugs), there is still a glaring lack of empirical evidence on the\ntypes of code changes that can be learned and automatically applied by DL. Our\ngoal is to make this first important step by quantitatively and qualitatively\ninvestigating the ability of a Neural Machine Translation (NMT) model to learn\nhow to automatically apply code changes implemented by developers during pull\nrequests. We train and experiment with the NMT model on a set of 236k pairs of\ncode components before and after the implementation of the changes provided in\nthe pull requests. We show that, when applied in a narrow enough context (i.e.,\nsmall/medium-sized pairs of methods before/after the pull request changes), NMT\ncan automatically replicate the changes implemented by developers during pull\nrequests in up to 36% of the cases. Moreover, our qualitative analysis shows\nthat the model is capable of learning and replicating a wide variety of\nmeaningful code changes, especially refactorings and bug-fixing activities. Our\nresults pave the way for novel research in the area of DL on code, such as the\nautomatic learning and applications of refactoring.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1901.09118v2",
            "date": "2019-01-26 00:01:27+00:00",
            "title": "Stopping Active Learning based on Predicted Change of F Measure for Text Classification",
            "authors": [
                "Michael Altschuler",
                "Michael Bloodgood"
            ],
            "abstract": "During active learning, an effective stopping method allows users to limit\nthe number of annotations, which is cost effective. In this paper, a new\nstopping method called Predicted Change of F Measure will be introduced that\nattempts to provide the users an estimate of how much performance of the model\nis changing at each iteration. This stopping method can be applied with any\nbase learner. This method is useful for reducing the data annotation bottleneck\nencountered when building text classification systems.",
            "journal ref": "In Proceedings of the 2019 IEEE 13th International Conference on\n  Semantic Computing (ICSC), pages 47-54, Newport Beach, CA, USA, January 2019.\n  IEEE",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.09126v2",
            "date": "2019-01-26 00:27:02+00:00",
            "title": "The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification",
            "authors": [
                "Garrett Beatty",
                "Ethan Kochis",
                "Michael Bloodgood"
            ],
            "abstract": "Annotation of training data is the major bottleneck in the creation of text\nclassification systems. Active learning is a commonly used technique to reduce\nthe amount of training data one needs to label. A crucial aspect of active\nlearning is determining when to stop labeling data. Three potential sources for\ninforming when to stop active learning are an additional labeled set of data,\nan unlabeled set of data, and the training data that is labeled during the\nprocess of active learning. To date, no one has compared and contrasted the\nadvantages and disadvantages of stopping methods based on these three\ninformation sources. We find that stopping methods that use unlabeled data are\nmore effective than methods that use labeled data.",
            "journal ref": "In Proceedings of the 2019 IEEE 13th International Conference on\n  Semantic Computing (ICSC), pages 287-294, Newport Beach, CA, USA, January\n  2019. IEEE",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.09501v3",
            "date": "2019-01-28 03:38:08+00:00",
            "title": "Data-to-Text Generation with Style Imitation",
            "authors": [
                "Shuai Lin",
                "Wentao Wang",
                "Zichao Yang",
                "Xiaodan Liang",
                "Frank F. Xu",
                "Eric Xing",
                "Zhiting Hu"
            ],
            "abstract": "Recent neural approaches to data-to-text generation have mostly focused on\nimproving content fidelity while lacking explicit control over writing styles\n(e.g., word choices, sentence structures). More traditional systems use\ntemplates to determine the realization of text. Yet manual or automatic\nconstruction of high-quality templates is difficult, and a template acting as\nhard constraints could harm content fidelity when it does not match the record\nperfectly. We study a new way of stylistic control by using existing sentences\nas soft templates. That is, the model learns to imitate the writing style of\nany given exemplar sentence, with automatic adaptions to faithfully describe\nthe content record. The problem is challenging due to the lack of parallel\ndata. We develop a neural approach that includes a hybrid attention-copy\nmechanism, learns with weak supervisions, and is enhanced with a new content\ncoverage constraint. We conduct experiments in restaurants and sports domains.\nResults show our approach achieves stronger performance than a range of\ncomparison methods. Our approach balances well between content fidelity and\nstyle control given exemplars that match the records to varying degrees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09657v1",
            "date": "2019-01-05 11:56:13+00:00",
            "title": "Fake News Detection via NLP is Vulnerable to Adversarial Attacks",
            "authors": [
                "Zhixuan Zhou",
                "Huankang Guan",
                "Meghana Moorthy Bhat",
                "Justin Hsu"
            ],
            "abstract": "News plays a significant role in shaping people's beliefs and opinions. Fake\nnews has always been a problem, which wasn't exposed to the mass public until\nthe past election cycle for the 45th President of the United States. While\nquite a few detection methods have been proposed to combat fake news since\n2015, they focus mainly on linguistic aspects of an article without any fact\nchecking. In this paper, we argue that these models have the potential to\nmisclassify fact-tampering fake news as well as under-written real news.\nThrough experiments on Fakebox, a state-of-the-art fake news detector, we show\nthat fact tampering attacks can be effective. To address these weaknesses, we\nargue that fact checking should be adopted in conjunction with linguistic\ncharacteristics analysis, so as to truly separate fake news from real news. A\ncrowdsourced knowledge graph is proposed as a straw man solution to collecting\ntimely facts about news events.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1901.09813v2",
            "date": "2019-01-28 17:04:25+00:00",
            "title": "Analogies Explained: Towards Understanding Word Embeddings",
            "authors": [
                "Carl Allen",
                "Timothy Hospedales"
            ],
            "abstract": "Word embeddings generated by neural network methods such as word2vec (W2V)\nare well known to exhibit seemingly linear behaviour, e.g. the embeddings of\nanalogy \"woman is to queen as man is to king\" approximately describe a\nparallelogram. This property is particularly intriguing since the embeddings\nare not trained to achieve it. Several explanations have been proposed, but\neach introduces assumptions that do not hold in practice. We derive a\nprobabilistically grounded definition of paraphrasing that we re-interpret as\nword transformation, a mathematical description of \"$w_x$ is to $w_y$\". From\nthese concepts we prove existence of linear relationships between W2V-type\nembeddings that underlie the analogical phenomenon, identifying explicit error\nterms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.09821v1",
            "date": "2019-01-28 17:14:12+00:00",
            "title": "Squeezed Very Deep Convolutional Neural Networks for Text Classification",
            "authors": [
                "Andr\u00e9a B. Duque",
                "Lu\u00e3 L\u00e1zaro J. Santos",
                "David Mac\u00eado",
                "Cleber Zanchettin"
            ],
            "abstract": "Most of the research in convolutional neural networks has focused on\nincreasing network depth to improve accuracy, resulting in a massive number of\nparameters which restricts the trained network to platforms with memory and\nprocessing constraints. We propose to modify the structure of the Very Deep\nConvolutional Neural Networks (VDCNN) model to fit mobile platforms constraints\nand keep performance. In this paper, we evaluate the impact of Temporal\nDepthwise Separable Convolutions and Global Average Pooling in the network\nparameters, storage size, and latency. The squeezed model (SVDCNN) is between\n10x and 20x smaller, depending on the network depth, maintaining a maximum size\nof 6MB. Regarding accuracy, the network experiences a loss between 0.4% and\n1.3% and obtains lower latencies compared to the baseline model.",
            "journal ref": "2019 International Conference on Artificial Neural Networks\n  (ICANN)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.09848v1",
            "date": "2019-01-28 17:41:19+00:00",
            "title": "A new evaluation framework for topic modeling algorithms based on synthetic corpora",
            "authors": [
                "Hanyu Shi",
                "Martin Gerlach",
                "Isabel Diersen",
                "Doug Downey",
                "Luis A. N. Amaral"
            ],
            "abstract": "Topic models are in widespread use in natural language processing and beyond.\nHere, we propose a new framework for the evaluation of probabilistic topic\nmodeling algorithms based on synthetic corpora containing an unambiguously\ndefined ground truth topic structure. The major innovation of our approach is\nthe ability to quantify the agreement between the planted and inferred topic\nstructures by comparing the assigned topic labels at the level of the tokens.\nIn experiments, our approach yields novel insights about the relative strengths\nof topic models as corpus characteristics vary, and the first evidence of an\n\"undetectable phase\" for topic models when the planted structure is weak. We\nalso establish the practical relevance of the insights gained for synthetic\ncorpora by predicting the performance of topic modeling algorithms in\nclassification tasks in real-world corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10055v2",
            "date": "2019-01-22 21:37:07+00:00",
            "title": "Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition",
            "authors": [
                "Julian Salazar",
                "Katrin Kirchhoff",
                "Zhiheng Huang"
            ],
            "abstract": "The success of self-attention in NLP has led to recent applications in\nend-to-end encoder-decoder architectures for speech recognition. Separately,\nconnectionist temporal classification (CTC) has matured as an alignment-free,\nnon-autoregressive approach to sequence transduction, either by itself or in\nvarious multitask and decoding frameworks. We propose SAN-CTC, a deep, fully\nself-attentional network for CTC, and show it is tractable and competitive for\nend-to-end speech recognition. SAN-CTC trains quickly and outperforms existing\nCTC models and most encoder-decoder models, with character error rates (CERs)\nof 4.7% in 1 day on WSJ eval92 and 2.8% in 1 week on LibriSpeech test-clean,\nwith a fixed architecture and one GPU. Similar improvements hold for WERs after\nLM decoding. We motivate the architecture for speech, evaluate position and\ndownsampling approaches, and explore how label alphabets (character, phoneme,\nsubword) affect attention heads and performance.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1901.10125v5",
            "date": "2019-01-29 06:15:36+00:00",
            "title": "Glyce: Glyph-vectors for Chinese Character Representations",
            "authors": [
                "Yuxian Meng",
                "Wei Wu",
                "Fei Wang",
                "Xiaoya Li",
                "Ping Nie",
                "Fan Yin",
                "Muyu Li",
                "Qinghong Han",
                "Xiaofei Sun",
                "Jiwei Li"
            ],
            "abstract": "It is intuitive that NLP tasks for logographic languages like Chinese should\nbenefit from the use of the glyph information in those languages. However, due\nto the lack of rich pictographic evidence in glyphs and the weak generalization\nability of standard computer vision models on character data, an effective way\nto utilize the glyph information remains to be found. In this paper, we address\nthis gap by presenting Glyce, the glyph-vectors for Chinese character\nrepresentations. We make three major innovations: (1) We use historical Chinese\nscripts (e.g., bronzeware script, seal script, traditional Chinese, etc) to\nenrich the pictographic evidence in characters; (2) We design CNN structures\n(called tianzege-CNN) tailored to Chinese character image processing; and (3)\nWe use image-classification as an auxiliary task in a multi-task learning setup\nto increase the model's ability to generalize. We show that glyph-based models\nare able to consistently outperform word/char ID-based models in a wide range\nof Chinese NLP tasks. We are able to set new state-of-the-art results for a\nvariety of Chinese NLP tasks, including tagging (NER, CWS, POS), sentence pair\nclassification, single sentence classification tasks, dependency parsing, and\nsemantic role labeling. For example, the proposed model achieves an F1 score of\n80.6 on the OntoNotes dataset of NER, +1.5 over BERT; it achieves an almost\nperfect accuracy of 99.8\\% on the Fudan corpus for text classification. Code\nfound at https://github.com/ShannonAI/glyce.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10263v1",
            "date": "2019-01-29 13:07:13+00:00",
            "title": "TiFi: Taxonomy Induction for Fictional Domains [Extended version]",
            "authors": [
                "Cuong Xuan Chu",
                "Simon Razniewski",
                "Gerhard Weikum"
            ],
            "abstract": "Taxonomies are important building blocks of structured knowledge bases, and\ntheir construction from text sources and Wikipedia has received much attention.\nIn this paper we focus on the construction of taxonomies for fictional domains,\nusing noisy category systems from fan wikis or text extraction as input. Such\nfictional domains are archetypes of entity universes that are poorly covered by\nWikipedia, such as also enterprise-specific knowledge bases or highly\nspecialized verticals. Our fiction-targeted approach, called TiFi, consists of\nthree phases: (i) category cleaning, by identifying candidate categories that\ntruly represent classes in the domain of interest, (ii) edge cleaning, by\nselecting subcategory relationships that correspond to class subsumption, and\n(iii) top-level construction, by mapping classes onto a subset of high-level\nWordNet categories. A comprehensive evaluation shows that TiFi is able to\nconstruct taxonomies for a diverse range of fictional domains such as Lord of\nthe Rings, The Simpsons or Greek Mythology with very high precision and that it\noutperforms state-of-the-art baselines for taxonomy induction by a substantial\nmargin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10265v3",
            "date": "2019-01-29 13:13:16+00:00",
            "title": "Implicit Diversity in Image Summarization",
            "authors": [
                "L. Elisa Celis",
                "Vijay Keswani"
            ],
            "abstract": "Studies have shown that the people depicted in image search results tend to\nbe of majority groups with respect to socially salient attributes. This skew\ngoes beyond that which already exists in the world - e.g., Kay et al. showed\nthat although 28% of CEOs in US are women, only 10% of the top 100 results for\nCEO in Google Image Search are women. Most existing approaches to correct for\nthis kind of bias assume that the images of people include socially salient\nattribute labels. However, such labels are often unknown. Further, using\nautomated techniques to infer these labels may often not be possible within\nacceptable accuracy ranges, and may not be desirable due to the additional\nbiases this process could incur. We develop a novel approach that takes as\ninput a visibly diverse control set of images and uses this set to select a set\nof images of people in response to a query. The goal is to have a resulting set\nthat is more visibly diverse in a manner that emulates the diversity depicted\nin the control set. Importantly, this approach does not require images to be\nlabelled at any point; effectively, it gives a way to implicitly diversify the\nset of images selected. We provide two variants of our approach: the first is a\nmodification of the MMR algorithm to incorporate the diversity scores, and\nsecond is a more efficient variant that does not consider within-list\nredundancy. We evaluate these approaches empirically on two datasets 1) a new\ndataset containing top Google image results for 96 occupations, for which we\nevaluate gender and skin-tone diversity with respect to occupations and 2) the\nCelebA dataset for which we evaluate gender diversity with respect to facial\nfeatures. Our approaches produce image sets that significantly improve the\nvisible diversity of the results, compared to current Google search and other\ndiverse image summarization algorithms, at a minimal cost to accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.10548v4",
            "date": "2019-01-29 21:05:46+00:00",
            "title": "Latent Normalizing Flows for Discrete Sequences",
            "authors": [
                "Zachary M. Ziegler",
                "Alexander M. Rush"
            ],
            "abstract": "Normalizing flows are a powerful class of generative models for continuous\nrandom variables, showing both strong model flexibility and the potential for\nnon-autoregressive generation. These benefits are also desired when modeling\ndiscrete random variables such as text, but directly applying normalizing flows\nto discrete sequences poses significant additional challenges. We propose a\nVAE-based generative model which jointly learns a normalizing flow-based\ndistribution in the latent space and a stochastic mapping to an observed\ndiscrete space. In this setting, we find that it is crucial for the flow-based\ndistribution to be highly multimodal. To capture this property, we propose\nseveral normalizing flow architectures to maximize model flexibility.\nExperiments consider common discrete sequence tasks of character-level language\nmodeling and polyphonic music generation. Our results indicate that an\nautoregressive flow-based model can match the performance of a comparable\nautoregressive baseline, and a non-autoregressive flow-based model can improve\ngeneration speed with a penalty to performance.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1901.10723v1",
            "date": "2019-01-30 09:32:51+00:00",
            "title": "Compositionality for Recursive Neural Networks",
            "authors": [
                "Martha Lewis"
            ],
            "abstract": "Modelling compositionality has been a longstanding area of research in the\nfield of vector space semantics. The categorical approach to compositionality\nmaps grammar onto vector spaces in a principled way, but comes under fire for\nrequiring the formation of very high-dimensional matrices and tensors, and\ntherefore being computationally infeasible. In this paper I show how a linear\nsimplification of recursive neural tensor network models can be mapped directly\nonto the categorical approach, giving a way of computing the required matrices\nand tensors. This mapping suggests a number of lines of research for both\ncategorical compositional vector space models of meaning and for recursive\nneural network models of compositionality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11117v4",
            "date": "2019-01-30 22:03:01+00:00",
            "title": "The Evolved Transformer",
            "authors": [
                "David R. So",
                "Chen Liang",
                "Quoc V. Le"
            ],
            "abstract": "Recent works have highlighted the strength of the Transformer architecture on\nsequence tasks while, at the same time, neural architecture search (NAS) has\nbegun to outperform human-designed models. Our goal is to apply NAS to search\nfor a better alternative to the Transformer. We first construct a large search\nspace inspired by the recent advances in feed-forward sequence models and then\nrun evolutionary architecture search with warm starting by seeding our initial\npopulation with the Transformer. To directly search on the computationally\nexpensive WMT 2014 English-German translation task, we develop the Progressive\nDynamic Hurdles method, which allows us to dynamically allocate more resources\nto more promising candidate models. The architecture found in our experiments\n-- the Evolved Transformer -- demonstrates consistent improvement over the\nTransformer on four well-established language tasks: WMT 2014 English-German,\nWMT 2014 English-French, WMT 2014 English-Czech and LM1B. At a big model size,\nthe Evolved Transformer establishes a new state-of-the-art BLEU score of 29.8\non WMT'14 English-German; at smaller sizes, it achieves the same quality as the\noriginal \"big\" Transformer with 37.6% less parameters and outperforms the\nTransformer by 0.7 BLEU at a mobile-friendly model size of 7M parameters.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.11281v1",
            "date": "2019-01-31 09:23:57+00:00",
            "title": "Conversational Networks for Automatic Online Moderation",
            "authors": [
                "Etienne Papegnies",
                "Vincent Labatut",
                "Richard Dufour",
                "Georges Linares"
            ],
            "abstract": "Moderation of user-generated content in an online community is a challenge\nthat has great socio-economical ramifications. However, the costs incurred by\ndelegating this work to human agents are high. For this reason, an automatic\nsystem able to detect abuse in user-generated content is of great interest.\nThere are a number of ways to tackle this problem, but the most commonly seen\nin practice are word filtering or regular expression matching. The main\nlimitations are their vulnerability to intentional obfuscation on the part of\nthe users, and their context-insensitive nature. Moreover, they are\nlanguage-dependent and may require appropriate corpora for training. In this\npaper, we propose a system for automatic abuse detection that completely\ndisregards message content. We first extract a conversational network from raw\nchat logs and characterize it through topological measures. We then use these\nas features to train a classifier on our abuse detection task. We thoroughly\nassess our system on a dataset of user comments originating from a French\nMassively Multiplayer Online Game. We identify the most appropriate network\nextraction parameters and discuss the discriminative power of our features,\nrelatively to their topological and temporal nature. Our method reaches an\nF-measure of 83.89 when using the full feature set, improving on existing\napproaches. With a selection of the most discriminative features, we\ndramatically cut computing time while retaining most of the performance\n(82.65).",
            "journal ref": "IEEE Transactions on Computational Social Systems, 2019,\n  https://ieeexplore.ieee.org/document/8629298",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1901.11333v4",
            "date": "2019-01-31 12:41:57+00:00",
            "title": "IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation",
            "authors": [
                "Zhijing Jin",
                "Di Jin",
                "Jonas Mueller",
                "Nicholas Matthews",
                "Enrico Santus"
            ],
            "abstract": "Text attribute transfer aims to automatically rewrite sentences such that\nthey possess certain linguistic attributes, while simultaneously preserving\ntheir semantic content. This task remains challenging due to a lack of\nsupervised parallel data. Existing approaches try to explicitly disentangle\ncontent and attribute information, but this is difficult and often results in\npoor content-preservation and ungrammaticality. In contrast, we propose a\nsimpler approach, Iterative Matching and Translation (IMaT), which: (1)\nconstructs a pseudo-parallel corpus by aligning a subset of semantically\nsimilar sentences from the source and the target corpora; (2) applies a\nstandard sequence-to-sequence model to learn the attribute transfer; (3)\niteratively improves the learned transfer function by refining imperfections in\nthe alignment. In sentiment modification and formality transfer tasks, our\nmethod outperforms complex state-of-the-art systems by a large margin. As an\nauxiliary contribution, we produce a publicly-available test set with\nhuman-generated transfer references.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.11373v1",
            "date": "2019-01-31 14:29:35+00:00",
            "title": "Learning and Evaluating General Linguistic Intelligence",
            "authors": [
                "Dani Yogatama",
                "Cyprien de Masson d'Autume",
                "Jerome Connor",
                "Tomas Kocisky",
                "Mike Chrzanowski",
                "Lingpeng Kong",
                "Angeliki Lazaridou",
                "Wang Ling",
                "Lei Yu",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "We define general linguistic intelligence as the ability to reuse previously\nacquired knowledge about a language's lexicon, syntax, semantics, and pragmatic\nconventions to adapt to new tasks quickly. Using this definition, we analyze\nstate-of-the-art natural language understanding models and conduct an extensive\nempirical investigation to evaluate them against these criteria through a\nseries of experiments that assess the task-independence of the knowledge being\nacquired by the learning process. In addition to task performance, we propose a\nnew evaluation metric based on an online encoding of the test data that\nquantifies how quickly an existing agent (model) learns a new task. Our results\nshow that while the field has made impressive progress in terms of model\narchitectures that generalize to many tasks, these models still require a lot\nof in-domain training examples (e.g., for fine tuning, training task-specific\nmodules), and are prone to catastrophic forgetting. Moreover, we find that far\nfrom solving general tasks (e.g., document question answering), our models are\noverfitting to the quirks of particular datasets (e.g., SQuAD). We discuss\nmissing components and conjecture on how to make progress toward general\nlinguistic intelligence.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.11492v1",
            "date": "2019-01-20 18:50:58+00:00",
            "title": "Improving generation quality of pointer networks via guided attention",
            "authors": [
                "Kushal Chawla",
                "Kundan Krishna",
                "Balaji Vasan Srinivasan"
            ],
            "abstract": "Pointer generator networks have been used successfully for abstractive\nsummarization. Along with the capability to generate novel words, it also\nallows the model to copy from the input text to handle out-of-vocabulary words.\nIn this paper, we point out two key shortcomings of the summaries generated\nwith this framework via manual inspection, statistical analysis and human\nevaluation. The first shortcoming is the extractive nature of the generated\nsummaries, since the network eventually learns to copy from the input article\nmost of the times, affecting the abstractive nature of the generated summaries.\nThe second shortcoming is the factual inaccuracies in the generated text\ndespite grammatical correctness. Our analysis indicates that this arises due to\nincorrect attention transition between different parts of the article. We\npropose an initial attempt towards addressing both these shortcomings by\nexternally appending traditional linguistic information parsed from the input\ntext, thereby teaching networks on the structure of the underlying text.\nResults indicate feasibility and potential of such additional cues for improved\ngeneration.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1901.11528v1",
            "date": "2019-01-31 18:48:19+00:00",
            "title": "Shaping the Narrative Arc: An Information-Theoretic Approach to Collaborative Dialogue",
            "authors": [
                "Kory W. Mathewson",
                "Pablo Samuel Castro",
                "Colin Cherry",
                "George Foster",
                "Marc G. Bellemare"
            ],
            "abstract": "We consider the problem of designing an artificial agent capable of\ninteracting with humans in collaborative dialogue to produce creative, engaging\nnarratives. In this task, the goal is to establish universe details, and to\ncollaborate on an interesting story in that universe, through a series of\nnatural dialogue exchanges. Our model can augment any probabilistic\nconversational agent by allowing it to reason about universe information\nestablished and what potential next utterances might reveal. Ideally, with each\nutterance, agents would reveal just enough information to add specificity and\nreduce ambiguity without limiting the conversation. We empirically show that\nour model allows control over the rate at which the agent reveals information\nand that doing so significantly improves accuracy in predicting the next line\nof dialogues from movies. We close with a case-study with four professional\ntheatre performers, who preferred interactions with our model-augmented agent\nover an unaugmented agent.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1903.00058v2",
            "date": "2019-02-28 20:33:00+00:00",
            "title": "Non-Parametric Adaptation for Neural Machine Translation",
            "authors": [
                "Ankur Bapna",
                "Orhan Firat"
            ],
            "abstract": "Neural Networks trained with gradient descent are known to be susceptible to\ncatastrophic forgetting caused by parameter shift during the training process.\nIn the context of Neural Machine Translation (NMT) this results in poor\nperformance on heterogeneous datasets and on sub-tasks like rare phrase\ntranslation. On the other hand, non-parametric approaches are immune to\nforgetting, perfectly complementing the generalization ability of NMT. However,\nattempts to combine non-parametric or retrieval based approaches with NMT have\nonly been successful on narrow domains, possibly due to over-reliance on\nsentence level retrieval. We propose a novel n-gram level retrieval approach\nthat relies on local phrase level similarities, allowing us to retrieve\nneighbors that are useful for translation even when overall sentence similarity\nis low. We complement this with an expressive neural network, allowing our\nmodel to extract information from the noisy retrieved context. We evaluate our\nsemi-parametric NMT approach on a heterogeneous dataset composed of WMT, IWSLT,\nJRC-Acquis and OpenSubtitles, and demonstrate gains on all 4 evaluation sets.\nThe semi-parametric nature of our approach opens the door for non-parametric\ndomain adaptation, demonstrating strong inference-time adaptation performance\non new domains without the need for any parameter updates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00216v1",
            "date": "2019-03-01 09:14:50+00:00",
            "title": "KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos",
            "authors": [
                "Egor Lakomkin",
                "Sven Magg",
                "Cornelius Weber",
                "Stefan Wermter"
            ],
            "abstract": "In this paper, we describe KT-Speech-Crawler: an approach for automatic\ndataset construction for speech recognition by crawling YouTube videos. We\noutline several filtering and post-processing steps, which extract samples that\ncan be used for training end-to-end neural speech recognition systems. In our\nexperiments, we demonstrate that a single-core version of the crawler can\nobtain around 150 hours of transcribed speech within a day, containing an\nestimated 3.5% word error rate in the transcriptions. Automatically collected\nsamples contain reading and spontaneous speech recorded in various conditions\nincluding background noise and music, distant microphone recordings, and a\nvariety of accents and reverberation. When training a deep neural network on\nspeech recognition, we observed around 40\\% word error rate reduction on the\nWall Street Journal dataset by integrating 200 hours of the collected samples\ninto the training set. The demo (http://emnlp-demo.lakomkin.me/) and the\ncrawler code (https://github.com/EgorLakomkin/KTSpeechCrawler) are publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.00401v2",
            "date": "2019-03-01 16:50:02+00:00",
            "title": "Learning To Follow Directions in Street View",
            "authors": [
                "Karl Moritz Hermann",
                "Mateusz Malinowski",
                "Piotr Mirowski",
                "Andras Banki-Horvath",
                "Keith Anderson",
                "Raia Hadsell"
            ],
            "abstract": "Navigating and understanding the real world remains a key challenge in\nmachine learning and inspires a great variety of research in areas such as\nlanguage grounding, planning, navigation and computer vision. We propose an\ninstruction-following task that requires all of the above, and which combines\nthe practicality of simulated environments with the challenges of ambiguous,\nnoisy real world data. StreetNav is built on top of Google Street View and\nprovides visually accurate environments representing real places. Agents are\ngiven driving instructions which they must learn to interpret in order to\nsuccessfully navigate in this environment. Since humans equipped with driving\ninstructions can readily navigate in previously unseen cities, we set a high\nbar and test our trained agents for similar cognitive capabilities. Although\ndeep reinforcement learning (RL) methods are frequently evaluated only on data\nthat closely follow the training distribution, our dataset extends to multiple\ncities and has a clean train/test separation. This allows for thorough testing\nof generalisation ability. This paper presents the StreetNav environment and\ntasks, models that establish strong baselines, and extensive analysis of the\ntask and the trained agents.",
            "journal ref": "AAAI 2020",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.00802v1",
            "date": "2019-03-03 01:08:47+00:00",
            "title": "Calibration of Encoder Decoder Models for Neural Machine Translation",
            "authors": [
                "Aviral Kumar",
                "Sunita Sarawagi"
            ],
            "abstract": "We study the calibration of several state of the art neural machine\ntranslation(NMT) systems built on attention-based encoder-decoder models. For\nstructured outputs like in NMT, calibration is important not just for reliable\nconfidence with predictions, but also for proper functioning of beam-search\ninference. We show that most modern NMT models are surprisingly miscalibrated\neven when conditioned on the true previous tokens. Our investigation leads to\ntwo main reasons -- severe miscalibration of EOS (end of sequence marker) and\nsuppression of attention uncertainty. We design recalibration methods based on\nthese signals and demonstrate improved accuracy, better sequence-level\ncalibration, and more intuitive results from beam-search.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.01290v1",
            "date": "2019-03-04 14:53:26+00:00",
            "title": "Traditional Machine Learning for Pitch Detection",
            "authors": [
                "Thomas Drugman",
                "Goeric Huybrechts",
                "Viacheslav Klimkov",
                "Alexis Moinet"
            ],
            "abstract": "Pitch detection is a fundamental problem in speech processing as F0 is used\nin a large number of applications. Recent articles have proposed deep learning\nfor robust pitch tracking. In this paper, we consider voicing detection as a\nclassification problem and F0 contour estimation as a regression problem. For\nboth tasks, acoustic features from multiple domains and traditional machine\nlearning methods are used. The discrimination power of existing and proposed\nfeatures is assessed through mutual information. Multiple supervised and\nunsupervised approaches are compared. A significant relative reduction of\nvoicing errors over the best baseline is obtained: 20% with the best clustering\nmethod (K-means) and 45% with a Multi-Layer Perceptron. For F0 contour\nestimation, the benefits of regression techniques are limited though. We\ninvestigate whether those objective gains translate in a parametric synthesis\ntask. Clear perceptual preferences are observed for the proposed approach over\ntwo widely-used baselines (RAPT and DIO).",
            "journal ref": "IEEE Signal Processing Letters, Vol. 25, Issue 11, pp. 1745-1749,\n  2018",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1903.02156v2",
            "date": "2019-03-06 03:36:29+00:00",
            "title": "Persona-Aware Tips Generation",
            "authors": [
                "Piji Li",
                "Zihao Wang",
                "Lidong Bing",
                "Wai Lam"
            ],
            "abstract": "Tips, as a compacted and concise form of reviews, were paid less attention by\nresearchers. In this paper, we investigate the task of tips generation by\nconsidering the `persona' information which captures the intrinsic language\nstyle of the users or the different characteristics of the product items. In\norder to exploit the persona information, we propose a framework based on\nadversarial variational auto-encoders (aVAE) for persona modeling from the\nhistorical tips and reviews of users and items. The latent variables from aVAE\nare regarded as persona embeddings. Besides representing persona using the\nlatent embeddings, we design a persona memory for storing the persona related\nwords for users and items. Pointer Network is used to retrieve persona wordings\nfrom the memory when generating tips. Moreover, the persona embeddings are used\nas latent factors by a rating prediction component to predict the sentiment of\na user over an item. Finally, the persona embeddings and the sentiment\ninformation are incorporated into a recurrent neural networks based tips\ngeneration component. Extensive experimental results are reported and discussed\nto elaborate the peculiarities of our framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02831v1",
            "date": "2019-03-07 11:06:10+00:00",
            "title": "Predicting Research Trends From Arxiv",
            "authors": [
                "Steffen Eger",
                "Chao Li",
                "Florian Netzer",
                "Iryna Gurevych"
            ],
            "abstract": "We perform trend detection on two datasets of Arxiv papers, derived from its\nmachine learning (cs.LG) and natural language processing (cs.CL) categories.\nOur approach is bottom-up: we first rank papers by their normalized citation\ncounts, then group top-ranked papers into different categories based on the\ntasks that they pursue and the methods they use. We then analyze these\nresulting topics. We find that the dominating paradigm in cs.CL revolves around\nnatural language generation problems and those in cs.LG revolve around\nreinforcement learning and adversarial principles. By extrapolation, we predict\nthat these topics will remain lead problems/approaches in their fields in the\nshort- and mid-term.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.02978v1",
            "date": "2019-03-07 15:14:42+00:00",
            "title": "Integrating Artificial and Human Intelligence for Efficient Translation",
            "authors": [
                "Nico Herbig",
                "Santanu Pal",
                "Josef van Genabith",
                "Antonio Kr\u00fcger"
            ],
            "abstract": "Current advances in machine translation increase the need for translators to\nswitch from traditional translation to post-editing of machine-translated text,\na process that saves time and improves quality. Human and artificial\nintelligence need to be integrated in an efficient way to leverage the\nadvantages of both for the translation task. This paper outlines approaches at\nthis boundary of AI and HCI and discusses open research questions to further\nadvance the field.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1903.03166v2",
            "date": "2019-03-07 20:18:39+00:00",
            "title": "CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog",
            "authors": [
                "Satwik Kottur",
                "Jos\u00e9 M. F. Moura",
                "Devi Parikh",
                "Dhruv Batra",
                "Marcus Rohrbach"
            ],
            "abstract": "Visual Dialog is a multimodal task of answering a sequence of questions\ngrounded in an image, using the conversation history as context. It entails\nchallenges in vision, language, reasoning, and grounding. However, studying\nthese subtasks in isolation on large, real datasets is infeasible as it\nrequires prohibitively-expensive complete annotation of the 'state' of all\nimages and dialogs.\n  We develop CLEVR-Dialog, a large diagnostic dataset for studying multi-round\nreasoning in visual dialog. Specifically, we construct a dialog grammar that is\ngrounded in the scene graphs of the images from the CLEVR dataset. This\ncombination results in a dataset where all aspects of the visual dialog are\nfully annotated. In total, CLEVR-Dialog contains 5 instances of 10-round\ndialogs for about 85k CLEVR images, totaling to 4.25M question-answer pairs.\n  We use CLEVR-Dialog to benchmark performance of standard visual dialog\nmodels; in particular, on visual coreference resolution (as a function of the\ncoreference distance). This is the first analysis of its kind for visual dialog\nmodels that was not possible without this dataset. We hope the findings from\nCLEVR-Dialog will help inform the development of future models for visual\ndialog. Our dataset and code are publicly available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1903.04329v1",
            "date": "2019-03-11 14:41:07+00:00",
            "title": "Scaling in Words on Twitter",
            "authors": [
                "Eszter Bok\u00e1nyi",
                "D\u00e1niel Kondor",
                "G\u00e1bor Vattay"
            ],
            "abstract": "Scaling properties of language are a useful tool for understanding generative\nprocesses in texts. We investigate the scaling relations in citywise Twitter\ncorpora coming from the Metropolitan and Micropolitan Statistical Areas of the\nUnited States. We observe a slightly superlinear urban scaling with the city\npopulation for the total volume of the tweets and words created in a city. We\nthen find that a certain core vocabulary follows the scaling relationship of\nthat of the bulk text, but most words are sensitive to city size, exhibiting a\nsuper- or a sublinear urban scaling. For both regimes we can offer a plausible\nexplanation based on the meaning of the words. We also show that the parameters\nfor Zipf's law and Heaps law differ on Twitter from that of other texts, and\nthat the exponent of Zipf's law changes with city size.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/1903.04561v2",
            "date": "2019-03-11 19:45:54+00:00",
            "title": "Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification",
            "authors": [
                "Daniel Borkan",
                "Lucas Dixon",
                "Jeffrey Sorensen",
                "Nithum Thain",
                "Lucy Vasserman"
            ],
            "abstract": "Unintended bias in Machine Learning can manifest as systemic differences in\nperformance for different demographic groups, potentially compounding existing\nchallenges to fairness in society at large. In this paper, we introduce a suite\nof threshold-agnostic metrics that provide a nuanced view of this unintended\nbias, by considering the various ways that a classifier's score distribution\ncan vary across designated groups. We also introduce a large new test set of\nonline comments with crowd-sourced annotations for identity references. We use\nthis to show how our metrics can be used to find new and potentially subtle\nunintended bias in existing public models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.04567v2",
            "date": "2019-03-11 19:51:34+00:00",
            "title": "Bridging the Gap Between Monaural Speech Enhancement and Recognition with Distortion-Independent Acoustic Modeling",
            "authors": [
                "Peidong Wang",
                "Ke Tan",
                "DeLiang Wang"
            ],
            "abstract": "Monaural speech enhancement has made dramatic advances since the introduction\nof deep learning a few years ago. Although enhanced speech has been\ndemonstrated to have better intelligibility and quality for human listeners,\nfeeding it directly to automatic speech recognition (ASR) systems trained with\nnoisy speech has not produced expected improvements in ASR performance. The\nlack of an enhancement benefit on recognition, or the gap between monaural\nspeech enhancement and recognition, is often attributed to speech distortions\nintroduced in the enhancement process. In this study, we analyze the distortion\nproblem, compare different acoustic models, and investigate a\ndistortion-independent training scheme for monaural speech recognition.\nExperimental results suggest that distortion-independent acoustic modeling is\nable to overcome the distortion problem. Such an acoustic model can also work\nwith speech enhancement models different from the one used during training.\nMoreover, the models investigated in this paper outperform the previous best\nsystem on the CHiME-2 corpus.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1903.05168v1",
            "date": "2019-03-12 19:33:49+00:00",
            "title": "On the Pitfalls of Measuring Emergent Communication",
            "authors": [
                "Ryan Lowe",
                "Jakob Foerster",
                "Y-Lan Boureau",
                "Joelle Pineau",
                "Yann Dauphin"
            ],
            "abstract": "How do we know if communication is emerging in a multi-agent system? The vast\nmajority of recent papers on emergent communication show that adding a\ncommunication channel leads to an increase in reward or task success. This is a\nuseful indicator, but provides only a coarse measure of the agent's learned\ncommunication abilities. As we move towards more complex environments, it\nbecomes imperative to have a set of finer tools that allow qualitative and\nquantitative insights into the emergence of communication. This may be\nespecially useful to allow humans to monitor agents' behaviour, whether for\nfault detection, assessing performance, or even building trust. In this paper,\nwe examine a few intuitive existing metrics for measuring communication, and\nshow that they can be misleading. Specifically, by training deep reinforcement\nlearning agents to play simple matrix games augmented with a communication\nchannel, we find a scenario where agents appear to communicate (their messages\nprovide information about their subsequent action), and yet the messages do not\nimpact the environment or other agent in any way. We explain this phenomenon\nusing ablation studies and by visualizing the representations of the learned\npolicies. We also survey some commonly used metrics for measuring emergent\ncommunication, and provide recommendations as to when these metrics should be\nused.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.05538v2",
            "date": "2019-03-13 15:13:57+00:00",
            "title": "SciLens: Evaluating the Quality of Scientific News Articles Using Social Media and Scientific Literature Indicators",
            "authors": [
                "Panayiotis Smeros",
                "Carlos Castillo",
                "Karl Aberer"
            ],
            "abstract": "This paper describes, develops, and validates SciLens, a method to evaluate\nthe quality of scientific news articles. The starting point for our work are\nstructured methodologies that define a series of quality aspects for manually\nevaluating news. Based on these aspects, we describe a series of indicators of\nnews quality. According to our experiments, these indicators help non-experts\nevaluate more accurately the quality of a scientific news article, compared to\nnon-experts that do not have access to these indicators. Furthermore, SciLens\ncan also be used to produce a completely automated quality score for an\narticle, which agrees more with expert evaluators than manual evaluations done\nby non-experts. One of the main elements of SciLens is the focus on both\ncontent and context of articles, where context is provided by (1) explicit and\nimplicit references on the article to scientific literature, and (2) reactions\nin social media referencing the article. We show that both contextual elements\ncan be valuable sources of information for determining article quality. The\nvalidation of SciLens, done through a combination of expert and non-expert\nannotation, demonstrates its effectiveness for both semi-automatic and\nautomatic quality evaluation of scientific news.",
            "journal ref": "Proceedings of the 28th International Conference on World Wide Web\n  (WWW '19), San Francisco, CA, USA, May 13-17, 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.05587v2",
            "date": "2019-03-13 17:32:51+00:00",
            "title": "GASC: Genre-Aware Semantic Change for Ancient Greek",
            "authors": [
                "Valerio Perrone",
                "Marco Palma",
                "Simon Hengchen",
                "Alessandro Vatri",
                "Jim Q. Smith",
                "Barbara McGillivray"
            ],
            "abstract": "Word meaning changes over time, depending on linguistic and extra-linguistic\nfactors. Associating a word's correct meaning in its historical context is a\ncentral challenge in diachronic research, and is relevant to a range of NLP\ntasks, including information retrieval and semantic search in historical texts.\nBayesian models for semantic change have emerged as a powerful tool to address\nthis challenge, providing explicit and interpretable representations of\nsemantic change phenomena. However, while corpora typically come with rich\nmetadata, existing models are limited by their inability to exploit contextual\ninformation (such as text genre) beyond the document time-stamp. This is\nparticularly critical in the case of ancient languages, where lack of data and\nlong diachronic span make it harder to draw a clear distinction between\npolysemy (the fact that a word has several senses) and semantic change (the\nprocess of acquiring, losing, or changing senses), and current systems perform\npoorly on these languages. We develop GASC, a dynamic semantic change model\nthat leverages categorical metadata about the texts' genre to boost inference\nand uncover the evolution of meanings in Ancient Greek corpora. In a new\nevaluation framework, our model achieves improved predictive performance\ncompared to the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05854v1",
            "date": "2019-03-14 08:31:05+00:00",
            "title": "MirrorGAN: Learning Text-to-image Generation by Redescription",
            "authors": [
                "Tingting Qiao",
                "Jing Zhang",
                "Duanqing Xu",
                "Dacheng Tao"
            ],
            "abstract": "Generating an image from a given text description has two goals: visual\nrealism and semantic consistency. Although significant progress has been made\nin generating high-quality and visually realistic images using generative\nadversarial networks, guaranteeing semantic consistency between the text\ndescription and visual content remains very challenging. In this paper, we\naddress this problem by proposing a novel global-local attentive and\nsemantic-preserving text-to-image-to-text framework called MirrorGAN. MirrorGAN\nexploits the idea of learning text-to-image generation by redescription and\nconsists of three modules: a semantic text embedding module (STEM), a\nglobal-local collaborative attentive module for cascaded image generation\n(GLAM), and a semantic text regeneration and alignment module (STREAM). STEM\ngenerates word- and sentence-level embeddings. GLAM has a cascaded architecture\nfor generating target images from coarse to fine scales, leveraging both local\nword attention and global sentence attention to progressively enhance the\ndiversity and semantic consistency of the generated images. STREAM seeks to\nregenerate the text description from the generated image, which semantically\naligns with the given text description. Thorough experiments on two public\nbenchmark datasets demonstrate the superiority of MirrorGAN over other\nrepresentative state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.05955v1",
            "date": "2019-03-14 12:53:45+00:00",
            "title": "Generative adversarial network-based glottal waveform model for statistical parametric speech synthesis",
            "authors": [
                "Bajibabu Bollepalli",
                "Lauri Juvela",
                "Paavo Alku"
            ],
            "abstract": "Recent studies have shown that text-to-speech synthesis quality can be\nimproved by using glottal vocoding. This refers to vocoders that parameterize\nspeech into two parts, the glottal excitation and vocal tract, that occur in\nthe human speech production apparatus. Current glottal vocoders generate the\nglottal excitation waveform by using deep neural networks (DNNs). However, the\nsquared error-based training of the present glottal excitation models is\nlimited to generating conditional average waveforms, which fails to capture the\nstochastic variation of the waveforms. As a result, shaped noise is added as\npost-processing. In this study, we propose a new method for predicting glottal\nwaveforms by generative adversarial networks (GANs). GANs are generative models\nthat aim to embed the data distribution in a latent space, enabling generation\nof new instances very similar to the original by randomly sampling the latent\ndistribution. The glottal pulses generated by GANs show a stochastic component\nsimilar to natural glottal pulses. In our experiments, we compare synthetic\nspeech generated using glottal waveforms produced by both DNNs and GANs. The\nresults show that the newly proposed GANs achieve synthesis quality comparable\nto that of widely-used DNNs, without using an additive noise component.",
            "journal ref": "Interspeech-2017",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1903.06164v3",
            "date": "2019-03-14 14:00:56+00:00",
            "title": "Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data",
            "authors": [
                "Moonsu Han",
                "Minki Kang",
                "Hyunwoo Jung",
                "Sung Ju Hwang"
            ],
            "abstract": "We consider a novel question answering (QA) task where the machine needs to\nread from large streaming data (long documents or videos) without knowing when\nthe questions will be given, which is difficult to solve with existing QA\nmethods due to their lack of scalability. To tackle this problem, we propose a\nnovel end-to-end deep network model for reading comprehension, which we refer\nto as Episodic Memory Reader (EMR) that sequentially reads the input contexts\ninto an external memory, while replacing memories that are less important for\nanswering \\emph{unseen} questions. Specifically, we train an RL agent to\nreplace a memory entry when the memory is full, in order to maximize its QA\naccuracy at a future timepoint, while encoding the external memory using either\nthe GRU or the Transformer architecture to learn representations that considers\nrelative importance between the memory entries. We validate our model on a\nsynthetic dataset (bAbI) as well as real-world large-scale textual QA\n(TriviaQA) and video QA (TVQA) datasets, on which it achieves significant\nimprovements over rule-based memory scheduling policies or an RL-based baseline\nthat independently learns the query-specific importance of each memory.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.06464v1",
            "date": "2019-03-15 11:13:22+00:00",
            "title": "A Context-Aware Citation Recommendation Model with BERT and Graph Convolutional Networks",
            "authors": [
                "Chanwoo Jeong",
                "Sion Jang",
                "Hyuna Shin",
                "Eunjeong Park",
                "Sungchul Choi"
            ],
            "abstract": "With the tremendous growth in the number of scientific papers being\npublished, searching for references while writing a scientific paper is a\ntime-consuming process. A technique that could add a reference citation at the\nappropriate place in a sentence will be beneficial. In this perspective,\ncontext-aware citation recommendation has been researched upon for around two\ndecades. Many researchers have utilized the text data called the context\nsentence, which surrounds the citation tag, and the metadata of the target\npaper to find the appropriate cited research. However, the lack of\nwell-organized benchmarking datasets and no model that can attain high\nperformance has made the research difficult.\n  In this paper, we propose a deep learning based model and well-organized\ndataset for context-aware paper citation recommendation. Our model comprises a\ndocument encoder and a context encoder, which uses Graph Convolutional Networks\n(GCN) layer and Bidirectional Encoder Representations from Transformers (BERT),\nwhich is a pre-trained model of textual data. By modifying the related PeerRead\ndataset, we propose a new dataset called FullTextPeerRead containing context\nsentences to cited references and paper metadata. To the best of our knowledge,\nThis dataset is the first well-organized dataset for context-aware paper\nrecommendation. The results indicate that the proposed model with the proposed\ndatasets can attain state-of-the-art performance and achieve a more than 28%\nimprovement in mean average precision (MAP) and recall@k.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07091v1",
            "date": "2019-03-17 14:01:53+00:00",
            "title": "The Missing Ingredient in Zero-Shot Neural Machine Translation",
            "authors": [
                "Naveen Arivazhagan",
                "Ankur Bapna",
                "Orhan Firat",
                "Roee Aharoni",
                "Melvin Johnson",
                "Wolfgang Macherey"
            ],
            "abstract": "Multilingual Neural Machine Translation (NMT) models are capable of\ntranslating between multiple source and target languages. Despite various\napproaches to train such models, they have difficulty with zero-shot\ntranslation: translating between language pairs that were not together seen\nduring training. In this paper we first diagnose why state-of-the-art\nmultilingual NMT models that rely purely on parameter sharing, fail to\ngeneralize to unseen language pairs. We then propose auxiliary losses on the\nNMT encoder that impose representational invariance across languages. Our\nsimple approach vastly improves zero-shot translation quality without\nregressing on supervised directions. For the first time, on WMT14\nEnglish-FrenchGerman, we achieve zero-shot performance that is on par with\npivoting. We also demonstrate the easy scalability of our approach to multiple\nlanguages on the IWSLT 2017 shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07288v1",
            "date": "2019-03-18 07:52:59+00:00",
            "title": "Effects of padding on LSTMs and CNNs",
            "authors": [
                "Mahidhar Dwarampudi",
                "N V Subba Reddy"
            ],
            "abstract": "Long Short-Term Memory (LSTM) Networks and Convolutional Neural Networks\n(CNN) have become very common and are used in many fields as they were\neffective in solving many problems where the general neural networks were\ninefficient. They were applied to various problems mostly related to images and\nsequences. Since LSTMs and CNNs take inputs of the same length and dimension,\ninput images and sequences are padded to maximum length while testing and\ntraining. This padding can affect the way the networks function and can make a\ngreat deal when it comes to performance and accuracies. This paper studies this\nand suggests the best way to pad an input sequence. This paper uses a simple\nsentiment analysis task for this purpose. We use the same dataset on both the\nnetworks with various padding to show the difference. This paper also discusses\nsome preprocessing techniques done on the data to ensure effective analysis of\nthe data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.07389v6",
            "date": "2019-02-27 00:03:17+00:00",
            "title": "Learning Hierarchical Discourse-level Structure for Fake News Detection",
            "authors": [
                "Hamid Karimi",
                "Jiliang Tang"
            ],
            "abstract": "On the one hand, nowadays, fake news articles are easily propagated through\nvarious online media platforms and have become a grand threat to the\ntrustworthiness of information. On the other hand, our understanding of the\nlanguage of fake news is still minimal. Incorporating hierarchical\ndiscourse-level structure of fake and real news articles is one crucial step\ntoward a better understanding of how these articles are structured.\nNevertheless, this has rarely been investigated in the fake news detection\ndomain and faces tremendous challenges. First, existing methods for capturing\ndiscourse-level structure rely on annotated corpora which are not available for\nfake news datasets. Second, how to extract out useful information from such\ndiscovered structures is another challenge. To address these challenges, we\npropose Hierarchical Discourse-level Structure for Fake news detection. HDSF\nlearns and constructs a discourse-level structure for fake/real news articles\nin an automated and data-driven manner. Moreover, we identify insightful\nstructure-related properties, which can explain the discovered structures and\nboost our understating of fake news. Conducted experiments show the\neffectiveness of the proposed approach. Further structural analysis suggests\nthat real and fake news present substantial differences in the hierarchical\ndiscourse-level structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07406v1",
            "date": "2019-03-05 09:11:53+00:00",
            "title": "Automatic Classification of Pathology Reports using TF-IDF Features",
            "authors": [
                "Shivam Kalra",
                "Larry Li",
                "Hamid R. Tizhoosh"
            ],
            "abstract": "A Pathology report is arguably one of the most important documents in\nmedicine containing interpretive information about the visual findings from the\npatient's biopsy sample. Each pathology report has a retention period of up to\n20 years after the treatment of a patient. Cancer registries process and encode\nhigh volumes of free-text pathology reports for surveillance of cancer and\ntumor diseases all across the world. In spite of their extremely valuable\ninformation they hold, pathology reports are not used in any systematic way to\nfacilitate computational pathology. Therefore, in this study, we investigate\nautomated machine-learning techniques to identify/predict the primary diagnosis\n(based on ICD-O code) from pathology reports. We performed experiments by\nextracting the TF-IDF features from the reports and classifying them using\nthree different methods---SVM, XGBoost, and Logistic Regression. We constructed\na new dataset with 1,949 pathology reports arranged into 37 ICD-O categories,\ncollected from four different primary sites, namely lung, kidney, thymus, and\ntestis. The reports were manually transcribed into text format after collecting\nthem as PDF files from NCI Genomic Data Commons public dataset. We subsequently\npre-processed the reports by removing irrelevant textual artifacts produced by\nOCR software. The highest classification accuracy we achieved was 92\\% using\nXGBoost classifier on TF-IDF feature vectors, the linear SVM scored 87\\%\naccuracy. Furthermore, the study shows that TF-IDF vectors are suitable for\nhighlighting the important keywords within a report which can be helpful for\nthe cancer research and diagnostic workflow. The results are encouraging in\ndemonstrating the potential of machine learning methods for classification and\nencoding of pathology reports.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07445v2",
            "date": "2019-03-15 13:05:23+00:00",
            "title": "An Exploration of State-of-the-art Methods for Offensive Language Detection",
            "authors": [
                "Harrison Uglow",
                "Martin Zlocha",
                "Szymon Zmy\u015blony"
            ],
            "abstract": "We provide a comprehensive investigation of different custom and\noff-the-shelf architectures as well as different approaches to generating\nfeature vectors for offensive language detection. We also show that these\napproaches work well on small and noisy datasets such as on the Offensive\nLanguage Identification Dataset (OLID), so it should be possible to use them\nfor other applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07507v1",
            "date": "2019-03-18 15:27:50+00:00",
            "title": "An Effective Label Noise Model for DNN Text Classification",
            "authors": [
                "Ishan Jindal",
                "Daniel Pressel",
                "Brian Lester",
                "Matthew Nokleby"
            ],
            "abstract": "Because large, human-annotated datasets suffer from labeling errors, it is\ncrucial to be able to train deep neural networks in the presence of label\nnoise. While training image classification models with label noise have\nreceived much attention, training text classification models have not. In this\npaper, we propose an approach to training deep networks that is robust to label\nnoise. This approach introduces a non-linear processing layer (noise model)\nthat models the statistics of the label noise into a convolutional neural\nnetwork (CNN) architecture. The noise model and the CNN weights are learned\njointly from noisy training data, which prevents the model from overfitting to\nerroneous labels. Through extensive experiments on several text classification\ndatasets, we show that this approach enables the CNN to learn better sentence\nrepresentations and is robust even to extreme label noise. We find that proper\ninitialization and regularization of this noise model is critical. Further, by\ncontrast to results focusing on large batch sizes for mitigating label noise\nfor image classification, we find that altering the batch size does not have\nmuch effect on classification performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.07766v3",
            "date": "2019-03-18 23:35:31+00:00",
            "title": "Lemotif: An Affective Visual Journal Using Deep Neural Networks",
            "authors": [
                "X. Alice Li",
                "Devi Parikh"
            ],
            "abstract": "We present Lemotif, an integrated natural language processing and image\ngeneration system that uses machine learning to (1) parse a text-based input\njournal entry describing the user's day for salient themes and emotions and (2)\nvisualize the detected themes and emotions in creative and appealing image\nmotifs. Synthesizing approaches from artificial intelligence and psychology,\nLemotif acts as an affective visual journal, encouraging users to regularly\nwrite and reflect on their daily experiences through visual reinforcement. By\nmaking patterns in emotions and their sources more apparent, Lemotif aims to\nhelp users better understand their emotional lives, identify opportunities for\naction, and track the effectiveness of behavioral changes over time. We verify\nvia human studies that prospective users prefer motifs generated by Lemotif\nover corresponding baselines, find the motifs representative of their journal\nentries, and think they would be more likely to journal regularly using a\nLemotif-based app.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1903.07860v1",
            "date": "2019-03-19 07:05:59+00:00",
            "title": "Personalized Neural Embeddings for Collaborative Filtering with Text",
            "authors": [
                "Guangneng Hu"
            ],
            "abstract": "Collaborative filtering (CF) is a core technique for recommender systems.\nTraditional CF approaches exploit user-item relations (e.g., clicks, likes, and\nviews) only and hence they suffer from the data sparsity issue. Items are\nusually associated with unstructured text such as article abstracts and product\nreviews. We develop a Personalized Neural Embedding (PNE) framework to exploit\nboth interactions and words seamlessly. We learn such embeddings of users,\nitems, and words jointly, and predict user preferences on items based on these\nlearned representations. PNE estimates the probability that a user will like an\nitem by two terms---behavior factors and semantic factors. On two real-world\ndatasets, PNE shows better performance than four state-of-the-art baselines in\nterms of three metrics. We also show that PNE learns meaningful word embeddings\nby visualization.",
            "journal ref": "NAACL 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.08309v1",
            "date": "2019-03-20 01:52:37+00:00",
            "title": "Prospection: Interpretable Plans From Language By Predicting the Future",
            "authors": [
                "Chris Paxton",
                "Yonatan Bisk",
                "Jesse Thomason",
                "Arunkumar Byravan",
                "Dieter Fox"
            ],
            "abstract": "High-level human instructions often correspond to behaviors with multiple\nimplicit steps. In order for robots to be useful in the real world, they must\nbe able to to reason over both motions and intermediate goals implied by human\ninstructions. In this work, we propose a framework for learning representations\nthat convert from a natural-language command to a sequence of intermediate\ngoals for execution on a robot. A key feature of this framework is prospection,\ntraining an agent not just to correctly execute the prescribed command, but to\npredict a horizon of consequences of an action before taking it. We demonstrate\nthe fidelity of plans generated by our framework when interpreting real,\ncrowd-sourced natural language commands for a robot in simulated scenes.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.08404v1",
            "date": "2019-03-20 09:40:19+00:00",
            "title": "Neural Check-Worthiness Ranking with Weak Supervision: Finding Sentences for Fact-Checking",
            "authors": [
                "Casper Hansen",
                "Christian Hansen",
                "Stephen Alstrup",
                "Jakob Grue Simonsen",
                "Christina Lioma"
            ],
            "abstract": "Automatic fact-checking systems detect misinformation, such as fake news, by\n(i) selecting check-worthy sentences for fact-checking, (ii) gathering related\ninformation to the sentences, and (iii) inferring the factuality of the\nsentences. Most prior research on (i) uses hand-crafted features to select\ncheck-worthy sentences, and does not explicitly account for the recent finding\nthat the top weighted terms in both check-worthy and non-check-worthy sentences\nare actually overlapping [15]. Motivated by this, we present a neural\ncheck-worthiness sentence ranking model that represents each word in a sentence\nby \\textit{both} its embedding (aiming to capture its semantics) and its\nsyntactic dependencies (aiming to capture its role in modifying the semantics\nof other terms in the sentence). Our model is an end-to-end trainable neural\nnetwork for check-worthiness ranking, which is trained on large amounts of\nunlabelled data through weak supervision. Thorough experimental evaluation\nagainst state of the art baselines, with and without weak supervision, shows\nour model to be superior at all times (+13% in MAP and +28% at various\nPrecision cut-offs from the best baseline with statistical significance).\nEmpirical analysis of the use of weak supervision, word embedding pretraining\non domain-specific data, and the use of syntactic dependencies of our model\nreveals that check-worthy sentences contain notably more identical syntactic\ndependencies than non-check-worthy sentences.",
            "journal ref": "In Companion Proceedings of the 2019 World Wide Web Conference",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.08734v2",
            "date": "2019-03-16 11:19:38+00:00",
            "title": "Combination of multiple Deep Learning architectures for Offensive Language Detection in Tweets",
            "authors": [
                "Nicol\u00f2 Frisiani",
                "Alexis Laignelet",
                "Batuhan G\u00fcler"
            ],
            "abstract": "This report contains the details regarding our submission to the OffensEval\n2019 (SemEval 2019 - Task 6). The competition was based on the Offensive\nLanguage Identification Dataset. We first discuss the details of the classifier\nimplemented and the type of input data used and pre-processing performed. We\nthen move onto critically evaluating our performance. We have achieved a\nmacro-average F1-score of 0.76, 0.68, 0.54, respectively for Task a, Task b,\nand Task c, which we believe reflects on the level of sophistication of the\nmodels implemented. Finally, we will be discussing the difficulties encountered\nand possible improvements for the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08739v1",
            "date": "2019-03-04 14:18:48+00:00",
            "title": "Russian Language Datasets in the Digitial Humanities Domain and Their Evaluation with Word Embeddings",
            "authors": [
                "Gerhard Wohlgenannt",
                "Artemii Babushkin",
                "Denis Romashov",
                "Igor Ukrainets",
                "Anton Maskaykin",
                "Ilya Shutov"
            ],
            "abstract": "In this paper, we present Russian language datasets in the digital humanities\ndomain for the evaluation of word embedding techniques or similar language\nmodeling and feature learning algorithms. The datasets are split into two task\ntypes, word intrusion and word analogy, and contain 31362 task units in total.\nThe characteristics of the tasks and datasets are that they build upon small,\ndomain-specific corpora, and that the datasets contain a high number of named\nentities. The datasets were created manually for two fantasy novel book series\n(\"A Song of Ice and Fire\" and \"Harry Potter\"). We provide baseline evaluations\nwith popular word embedding models trained on the book corpora for the given\ntasks, both for the Russian and English language versions of the datasets.\nFinally, we compare and analyze the results and discuss specifics of Russian\nlanguage with regards to the problem setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09243v1",
            "date": "2019-03-21 21:38:20+00:00",
            "title": "Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions",
            "authors": [
                "Siddharth Patki",
                "Andrea F. Daniele",
                "Matthew R. Walter",
                "Thomas M. Howard"
            ],
            "abstract": "The speed and accuracy with which robots are able to interpret natural\nlanguage is fundamental to realizing effective human-robot interaction. A great\ndeal of attention has been paid to developing models and approximate inference\nalgorithms that improve the efficiency of language understanding. However,\nexisting methods still attempt to reason over a representation of the\nenvironment that is flat and unnecessarily detailed, which limits scalability.\nAn open problem is then to develop methods capable of producing the most\ncompact environment model sufficient for accurate and efficient natural\nlanguage understanding. We propose a model that leverages environment-related\ninformation encoded within instructions to identify the subset of observations\nand perceptual classifiers necessary to perceive a succinct,\ninstruction-specific environment representation. The framework uses three\nprobabilistic graphical models trained from a corpus of annotated instructions\nto infer salient scene semantics, perceptual classifiers, and grounded symbols.\nExperimental results on two robots operating in different environments\ndemonstrate that by exploiting the content and the structure of the\ninstructions, our method learns compact environment representations that\nsignificantly improve the efficiency of natural language symbol grounding.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1903.09848v2",
            "date": "2019-03-23 17:33:38+00:00",
            "title": "Competence-based Curriculum Learning for Neural Machine Translation",
            "authors": [
                "Emmanouil Antonios Platanios",
                "Otilia Stretcu",
                "Graham Neubig",
                "Barnabas Poczos",
                "Tom M. Mitchell"
            ],
            "abstract": "Current state-of-the-art NMT systems use large neural networks that are not\nonly slow to train, but also often require many heuristics and optimization\ntricks, such as specialized learning rate schedules and large batch sizes. This\nis undesirable as it requires extensive hyperparameter tuning. In this paper,\nwe propose a curriculum learning framework for NMT that reduces training time,\nreduces the need for specialized heuristics or large batch sizes, and results\nin overall better performance. Our framework consists of a principled way of\ndeciding which training samples are shown to the model at different times\nduring training, based on the estimated difficulty of a sample and the current\ncompetence of the model. Filtering training samples in this manner prevents the\nmodel from getting stuck in bad local optima, making it converge faster and\nreach a better solution than the common approach of uniformly sampling training\nexamples. Furthermore, the proposed method can be easily applied to existing\nNMT models by simply modifying their input data pipelines. We show that our\nframework can help improve the training time and the performance of both\nrecurrent neural network models and Transformers, achieving up to a 70%\ndecrease in training time, while at the same time obtaining accuracy\nimprovements of up to 2.2 BLEU.",
            "journal ref": "NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.09942v2",
            "date": "2019-03-24 08:11:58+00:00",
            "title": "Deep recommender engine based on efficient product embeddings neural pipeline",
            "authors": [
                "Laurentiu Piciu",
                "Andrei Damian",
                "Nicolae Tapus",
                "Andrei Simion-Constantinescu",
                "Bogdan Dumitrescu"
            ],
            "abstract": "Predictive analytics systems are currently one of the most important areas of\nresearch and development within the Artificial Intelligence domain and\nparticularly in Machine Learning. One of the \"holy grails\" of predictive\nanalytics is the research and development of the \"perfect\" recommendation\nsystem. In our paper, we propose an advanced pipeline model for the multi-task\nobjective of determining product complementarity, similarity and sales\nprediction using deep neural models applied to big-data sequential transaction\nsystems. Our highly parallelized hybrid model pipeline consists of both\nunsupervised and supervised models, used for the objectives of generating\nsemantic product embeddings and predicting sales, respectively. Our\nexperimentation and benchmarking processes have been done using pharma industry\nretail real-life transactional Big-Data streams.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.09952v1",
            "date": "2019-03-24 09:25:26+00:00",
            "title": "Optimization of Speaker Extraction Neural Network with Magnitude and Temporal Spectrum Approximation Loss",
            "authors": [
                "Chenglin Xu",
                "Wei Rao",
                "Eng Siong Chng",
                "Haizhou Li"
            ],
            "abstract": "The SpeakerBeam-FE (SBF) method is proposed for speaker extraction. It\nattempts to overcome the problem of unknown number of speakers in an audio\nrecording during source separation. The mask approximation loss of SBF is\nsub-optimal, which doesn't calculate direct signal reconstruction error and\nconsider the speech context. To address these problems, this paper proposes a\nmagnitude and temporal spectrum approximation loss to estimate a phase\nsensitive mask for the target speaker with the speaker characteristics.\nMoreover, this paper explores a concatenation framework instead of the context\nadaptive deep neural network in the SBF method to encode a speaker embedding\ninto the mask estimation network. Experimental results under open evaluation\ncondition show that the proposed method achieves 70.4% and 17.7% relative\nimprovement over the SBF baseline on signal-to-distortion ratio (SDR) and\nperceptual evaluation of speech quality (PESQ), respectively. A further\nanalysis demonstrates 69.1% and 72.3% relative SDR improvements obtained by the\nproposed method for different and same gender mixtures.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1903.10246v1",
            "date": "2019-03-25 11:28:36+00:00",
            "title": "Computational and Robotic Models of Early Language Development: A Review",
            "authors": [
                "Pierre-Yves Oudeyer",
                "George Kachergis",
                "William Schueller"
            ],
            "abstract": "We review computational and robotics models of early language learning and\ndevelopment. We first explain why and how these models are used to understand\nbetter how children learn language. We argue that they provide concrete\ntheories of language learning as a complex dynamic system, complementing\ntraditional methods in psychology and linguistics. We review different modeling\nformalisms, grounded in techniques from machine learning and artificial\nintelligence such as Bayesian and neural network approaches. We then discuss\ntheir role in understanding several key mechanisms of language development:\ncross-situational statistical learning, embodiment, situated social\ninteraction, intrinsically motivated learning, and cultural evolution. We\nconclude by discussing future challenges for research, including modeling of\nlarge-scale empirical data about language acquisition in real-world\nenvironments.\n  Keywords: Early language learning, Computational and robotic models, machine\nlearning, development, embodiment, social interaction, intrinsic motivation,\nself-organization, dynamical systems, complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10421v3",
            "date": "2019-03-25 16:05:47+00:00",
            "title": "A linear bound on the k-rendezvous time for primitive sets of NZ matrices",
            "authors": [
                "Costanza Catalano",
                "Umer Azfar",
                "Ludovic Charlier",
                "Rapha\u00ebl Jungers"
            ],
            "abstract": "A set of nonnegative matrices is called primitive if there exists a product\nof these matrices that is entrywise positive. Motivated by recent results\nrelating synchronizing automata and primitive sets, we study the length of the\nshortest product of a primitive set having a column or a row with k positive\nentries, called its k-rendezvous time (k-RT}), in the case of sets of matrices\nhaving no zero rows and no zero columns. We prove that the k-RT is at most\nlinear w.r.t. the matrix size n for small k, while the problem is still open\nfor synchronizing automata. We provide two upper bounds on the k-RT: the second\nis an improvement of the first one, although the latter can be written in\nclosed form. We then report numerical results comparing our upper bounds on the\nk-RT with heuristic approximation methods.",
            "journal ref": "None",
            "category": "cs.DM"
        },
        {
            "url": "http://arxiv.org/abs/1903.10630v1",
            "date": "2019-03-25 23:12:56+00:00",
            "title": "Diversifying Reply Suggestions using a Matching-Conditional Variational Autoencoder",
            "authors": [
                "Budhaditya Deb",
                "Peter Bailey",
                "Milad Shokouhi"
            ],
            "abstract": "We consider the problem of diversifying automated reply suggestions for a\ncommercial instant-messaging (IM) system (Skype). Our conversation model is a\nstandard matching based information retrieval architecture, which consists of\ntwo parallel encoders to project messages and replies into a common feature\nrepresentation. During inference, we select replies from a fixed response set\nusing nearest neighbors in the feature space. To diversify responses, we\nformulate the model as a generative latent variable model with Conditional\nVariational Auto-Encoder (M-CVAE). We propose a constrained-sampling approach\nto make the variational inference in M-CVAE efficient for our production\nsystem. In offline experiments, M-CVAE consistently increased diversity by\n~30-40% without significant impact on relevance. This translated to a 5% gain\nin click-rate in our online production system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.10842v1",
            "date": "2019-03-26 12:53:26+00:00",
            "title": "Improve Diverse Text Generation by Self Labeling Conditional Variational Auto Encoder",
            "authors": [
                "Yuchi Zhang",
                "Yongliang Wang",
                "Liping Zhang",
                "Zhiqiang Zhang",
                "Kun Gai"
            ],
            "abstract": "Diversity plays a vital role in many text generating applications. In recent\nyears, Conditional Variational Auto Encoders (CVAE) have shown promising\nperformances for this task. However, they often encounter the so called\nKL-Vanishing problem. Previous works mitigated such problem by heuristic\nmethods such as strengthening the encoder or weakening the decoder while\noptimizing the CVAE objective function. Nevertheless, the optimizing direction\nof these methods are implicit and it is hard to find an appropriate degree to\nwhich these methods should be applied. In this paper, we propose an explicit\noptimizing objective to complement the CVAE to directly pull away from\nKL-vanishing. In fact, this objective term guides the encoder towards the \"best\nencoder\" of the decoder to enhance the expressiveness. A labeling network is\nintroduced to estimate the \"best encoder\". It provides a continuous label in\nthe latent space of CVAE to help build a close connection between latent\nvariables and targets. The whole proposed method is named Self Labeling\nCVAE~(SLCVAE). To accelerate the research of diverse text generation, we also\npropose a large native one-to-many dataset. Extensive experiments are conducted\non two tasks, which show that our method largely improves the generating\ndiversity while achieving comparable accuracy compared with state-of-art\nalgorithms.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1903.11112v1",
            "date": "2019-03-26 18:48:43+00:00",
            "title": "Privacy-preserving Active Learning on Sensitive Data for User Intent Classification",
            "authors": [
                "Oluwaseyi Feyisetan",
                "Thomas Drake",
                "Borja Balle",
                "Tom Diethe"
            ],
            "abstract": "Active learning holds promise of significantly reducing data annotation costs\nwhile maintaining reasonable model performance. However, it requires sending\ndata to annotators for labeling. This presents a possible privacy leak when the\ntraining set includes sensitive user data. In this paper, we describe an\napproach for carrying out privacy preserving active learning with quantifiable\nguarantees. We evaluate our approach by showing the tradeoff between privacy,\nutility and annotation budget on a binary classification task in a active\nlearning setting.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.11508v2",
            "date": "2019-03-27 16:01:18+00:00",
            "title": "Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems",
            "authors": [
                "Steffen Eger",
                "G\u00f6zde G\u00fcl \u015eahin",
                "Andreas R\u00fcckl\u00e9",
                "Ji-Ung Lee",
                "Claudia Schulz",
                "Mohsen Mesgar",
                "Krishnkant Swarnkar",
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "Visual modifications to text are often used to obfuscate offensive comments\nin social media (e.g., \"!d10t\") or as a writing style (\"1337\" in \"leet speak\"),\namong other scenarios. We consider this as a new type of adversarial attack in\nNLP, a setting to which humans are very robust, as our experiments with both\nsimple and more difficult visual input perturbations demonstrate. We then\ninvestigate the impact of visual adversarial attacks on current NLP systems on\ncharacter-, word-, and sentence-level tasks, showing that both neural and\nnon-neural models are, in contrast to humans, extremely sensitive to such\nattacks, suffering performance decreases of up to 82\\%. We then explore three\nshielding methods---visual character embeddings, adversarial training, and\nrule-based recovery---which substantially improve the robustness of the models.\nHowever, the shielding methods still fall behind performances achieved in\nnon-attack scenarios, which demonstrates the difficulty of dealing with visual\nattacks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.11983v1",
            "date": "2019-03-18 09:25:10+00:00",
            "title": "Sentiment Analysis on IMDB Movie Comments and Twitter Data by Machine Learning and Vector Space Techniques",
            "authors": [
                "\u0130lhan Tar\u0131mer",
                "Adil \u00c7oban",
                "Arif Emre Kocaman"
            ],
            "abstract": "This study's goal is to create a model of sentiment analysis on a 2000 rows\nIMDB movie comments and 3200 Twitter data by using machine learning and vector\nspace techniques; positive or negative preliminary information about the text\nis to provide. In the study, a vector space was created in the KNIME Analytics\nplatform, and a classification study was performed on this vector space by\nDecision Trees, Na\\\"ive Bayes and Support Vector Machines classification\nalgorithms. The conclusions obtained were compared in terms of each algorithms.\nThe classification results for IMDB movie comments are obtained as 94,00%,\n73,20%, and 85,50% by Decision Tree, Naive Bayes and SVM algorithms. The\nclassification results for Twitter data set are presented as 82,76%, 75,44% and\n72,50% by Decision Tree, Naive Bayes SVM algorithms as well. It is seen that\nthe best classification results presented in both data sets are which\ncalculated by SVM algorithm.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.12389v2",
            "date": "2019-03-29 08:26:13+00:00",
            "title": "Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet",
            "authors": [
                "Mingyang Zhang",
                "Xin Wang",
                "Fuming Fang",
                "Haizhou Li",
                "Junichi Yamagishi"
            ],
            "abstract": "We investigated the training of a shared model for both text-to-speech (TTS)\nand voice conversion (VC) tasks. We propose using an extended model\narchitecture of Tacotron, that is a multi-source sequence-to-sequence model\nwith a dual attention mechanism as the shared model for both the TTS and VC\ntasks. This model can accomplish these two different tasks respectively\naccording to the type of input. An end-to-end speech synthesis task is\nconducted when the model is given text as the input while a\nsequence-to-sequence voice conversion task is conducted when it is given the\nspeech of a source speaker as the input. Waveform signals are generated by\nusing WaveNet, which is conditioned by using a predicted mel-spectrogram. We\npropose jointly training a shared model as a decoder for a target speaker that\nsupports multiple sources. Listening experiments show that our proposed\nmulti-source encoder-decoder model can efficiently achieve both the TTS and VC\ntasks.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1903.12392v2",
            "date": "2019-03-29 08:36:06+00:00",
            "title": "Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform",
            "authors": [
                "Shinji Takaki",
                "Hirokazu Kameoka",
                "Junichi Yamagishi"
            ],
            "abstract": "Recently, we proposed short-time Fourier transform (STFT)-based loss\nfunctions for training a neural speech waveform model. In this paper, we\ngeneralize the above framework and propose a training scheme for such models\nbased on spectral amplitude and phase losses obtained by either STFT or\ncontinuous wavelet transform (CWT), or both of them. Since CWT is capable of\nhaving time and frequency resolutions different from those of STFT and is cable\nof considering those closer to human auditory scales, the proposed loss\nfunctions could provide complementary information on speech signals.\nExperimental results showed that it is possible to train a high-quality model\nby using the proposed CWT spectral loss and is as good as one using STFT-based\nloss.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1903.12495v1",
            "date": "2019-03-28 15:13:20+00:00",
            "title": "Crowd Sourced Data Analysis: Mapping of Programming Concepts to Syntactical Patterns",
            "authors": [
                "Deepak Thukral",
                "Darvesh Punia"
            ],
            "abstract": "Since programming concepts do not match their syntactic representations, code\nsearch is a very tedious task. For instance in Java or C, array doesn't match\n[], so using \"array\" as a query, one cannot find what they are looking for.\nOften developers have to search code whether to understand any code, or to\nreuse some part of that code, or just to read it, without natural language\nsearching, developers have to often scroll back and forth or use variable names\nas their queries. In our work, we have used Stackoverflow (SO) question and\nanswers to make a mapping of programming concepts with their respective natural\nlanguage keywords, and then tag these natural language terms to every line of\ncode, which can further we used in searching using natural language keywords.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.00110v2",
            "date": "2019-03-29 22:43:26+00:00",
            "title": "Keyphrase Generation: A Text Summarization Struggle",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Authors' keyphrases assigned to scientific articles are essential for\nrecognizing content and topic aspects. Most of the proposed supervised and\nunsupervised methods for keyphrase generation are unable to produce terms that\nare valuable but do not appear in the text. In this paper, we explore the\npossibility of considering the keyphrase string as an abstractive summary of\nthe title and the abstract. First, we collect, process and release a large\ndataset of scientific paper metadata that contains 2.2 million records. Then we\nexperiment with popular text summarization neural architectures. Despite using\nadvanced deep learning models, large quantities of data and many days of\ncomputation, our systematic evaluation on four test datasets reveals that the\nexplored text summarization methods could not produce better keyphrases than\nthe simpler unsupervised methods, or the existing supervised ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00132v2",
            "date": "2019-03-30 01:51:24+00:00",
            "title": "ANA at SemEval-2019 Task 3: Contextual Emotion detection in Conversations through hierarchical LSTMs and BERT",
            "authors": [
                "Chenyang Huang",
                "Amine Trabelsi",
                "Osmar R. Za\u00efane"
            ],
            "abstract": "This paper describes the system submitted by ANA Team for the SemEval-2019\nTask 3: EmoContext. We propose a novel Hierarchical LSTMs for Contextual\nEmotion Detection (HRLCE) model. It classifies the emotion of an utterance\ngiven its conversational context. The results show that, in this task, our\nHRCLE outperforms the most recent state-of-the-art text classification\nframework: BERT. We combine the results generated by BERT and HRCLE to achieve\nan overall score of 0.7709 which ranked 5th on the final leader board of the\ncompetition among 165 Teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00350v1",
            "date": "2019-03-31 07:30:47+00:00",
            "title": "Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues",
            "authors": [
                "Sungjoon Park",
                "Donghyun Kim",
                "Alice Oh"
            ],
            "abstract": "The recent surge of text-based online counseling applications enables us to\ncollect and analyze interactions between counselors and clients. A dataset of\nthose interactions can be used to learn to automatically classify the client\nutterances into categories that help counselors in diagnosing client status and\npredicting counseling outcome. With proper anonymization, we collect\ncounselor-client dialogues, define meaningful categories of client utterances\nwith professional counselors, and develop a novel neural network model for\nclassifying the client utterances. The central idea of our model, ConvMFiT, is\na pre-trained conversation model which consists of a general language model\nbuilt from an out-of-domain corpus and two role-specific language models built\nfrom unlabeled in-domain dialogues. The classification result shows that\nConvMFiT outperforms state-of-the-art comparison models. Further, the attention\nweights in the learned model confirm that the model finds expected linguistic\npatterns for each category.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00720v1",
            "date": "2019-03-13 19:22:22+00:00",
            "title": "CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning",
            "authors": [
                "Ziyu Yao",
                "Jayavardhan Reddy Peddamail",
                "Huan Sun"
            ],
            "abstract": "To accelerate software development, much research has been performed to help\npeople understand and reuse the huge amount of available code resources. Two\nimportant tasks have been widely studied: code retrieval, which aims to\nretrieve code snippets relevant to a given natural language query from a code\nbase, and code annotation, where the goal is to annotate a code snippet with a\nnatural language description. Despite their advancement in recent years, the\ntwo tasks are mostly explored separately. In this work, we investigate a novel\nperspective of Code annotation for Code retrieval (hence called `CoaCor'),\nwhere a code annotation model is trained to generate a natural language\nannotation that can represent the semantic meaning of a given code snippet and\ncan be leveraged by a code retrieval model to better distinguish relevant code\nsnippets from others. To this end, we propose an effective framework based on\nreinforcement learning, which explicitly encourages the code annotation model\nto generate annotations that can be used for the retrieval task. Through\nextensive experiments, we show that code annotations generated by our framework\nare much more detailed and more useful for code retrieval, and they can further\nimprove the performance of existing code retrieval models significantly.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1904.00761v2",
            "date": "2019-03-20 12:01:46+00:00",
            "title": "Neural Speed Reading with Structural-Jump-LSTM",
            "authors": [
                "Christian Hansen",
                "Casper Hansen",
                "Stephen Alstrup",
                "Jakob Grue Simonsen",
                "Christina Lioma"
            ],
            "abstract": "Recurrent neural networks (RNNs) can model natural language by sequentially\n'reading' input tokens and outputting a distributed representation of each\ntoken. Due to the sequential nature of RNNs, inference time is linearly\ndependent on the input length, and all inputs are read regardless of their\nimportance. Efforts to speed up this inference, known as 'neural speed\nreading', either ignore or skim over part of the input. We present\nStructural-Jump-LSTM: the first neural speed reading model to both skip and\njump text during inference. The model consists of a standard LSTM and two\nagents: one capable of skipping single words when reading, and one capable of\nexploiting punctuation structure (sub-sentence separators (,:), sentence end\nsymbols (.!?), or end of text markers) to jump ahead after reading a word. A\ncomprehensive experimental evaluation of our model against all five\nstate-of-the-art neural reading models shows that Structural-Jump-LSTM achieves\nthe best overall floating point operations (FLOP) reduction (hence is faster),\nwhile keeping the same accuracy or even improving it compared to a vanilla LSTM\nthat reads the whole text.",
            "journal ref": "7th International Conference on Learning Representations (ICLR)\n  2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00762v1",
            "date": "2019-03-20 11:10:29+00:00",
            "title": "Affect in Tweets Using Experts Model",
            "authors": [
                "Subba Reddy Oota",
                "Adithya Avvaru",
                "Mounika Marreddy",
                "Radhika Mamidi"
            ],
            "abstract": "Estimating the intensity of emotion has gained significance as modern textual\ninputs in potential applications like social media, e-retail markets,\npsychology, advertisements etc., carry a lot of emotions, feelings, expressions\nalong with its meaning. However, the approaches of traditional sentiment\nanalysis primarily focuses on classifying the sentiment in general (positive or\nnegative) or at an aspect level(very positive, low negative, etc.) and cannot\nexploit the intensity information. Moreover, automatically identifying emotions\nlike anger, fear, joy, sadness, disgust etc., from text introduces challenging\nscenarios where single tweet may contain multiple emotions with different\nintensities and some emotions may even co-occur in some of the tweets. In this\npaper, we propose an architecture, Experts Model, inspired from the standard\nMixture of Experts (MoE) model. The key idea here is each expert learns\ndifferent sets of features from the feature vector which helps in better\nemotion detection from the tweet. We compared the results of our Experts Model\nwith both baseline results and top five performers of SemEval-2018 Task-1,\nAffect in Tweets (AIT). The experimental results show that our proposed\napproach deals with the emotion detection problem and stands at top-5 results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.00771v2",
            "date": "2019-04-01 12:39:05+00:00",
            "title": "Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corpora",
            "authors": [
                "Hieu-Thi Luong",
                "Xin Wang",
                "Junichi Yamagishi",
                "Nobuyuki Nishizawa"
            ],
            "abstract": "When the available data of a target speaker is insufficient to train a high\nquality speaker-dependent neural text-to-speech (TTS) system, we can combine\ndata from multiple speakers and train a multi-speaker TTS model instead. Many\nstudies have shown that neural multi-speaker TTS model trained with a small\namount data from multiple speakers combined can generate synthetic speech with\nbetter quality and stability than a speaker-dependent one. However when the\namount of data from each speaker is highly unbalanced, the best approach to\nmake use of the excessive data remains unknown. Our experiments showed that\nsimply combining all available data from every speaker to train a multi-speaker\nmodel produces better than or at least similar performance to its\nspeaker-dependent counterpart. Moreover by using an ensemble multi-speaker\nmodel, in which each subsystem is trained on a subset of available data, we can\nfurther improve the quality of the synthetic speech especially for\nunderrepresented speakers whose training data is limited.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.00784v3",
            "date": "2019-03-25 14:36:50+00:00",
            "title": "A Survey of Code-switched Speech and Language Processing",
            "authors": [
                "Sunayana Sitaram",
                "Khyathi Raghavi Chandu",
                "Sai Krishna Rallabandi",
                "Alan W Black"
            ],
            "abstract": "Code-switching, the alternation of languages within a conversation or\nutterance, is a common communicative phenomenon that occurs in multilingual\ncommunities across the world. This survey reviews computational approaches for\ncode-switched Speech and Natural Language Processing. We motivate why\nprocessing code-switched text and speech is essential for building intelligent\nagents and systems that interact with users in multilingual communities. As\ncode-switching data and resources are scarce, we list what is available in\nvarious code-switched language pairs with the language processing tasks they\ncan be used for. We review code-switching research in various Speech and NLP\napplications, including language processing tools and end-to-end systems. We\nconclude with future directions and open problems in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00788v2",
            "date": "2019-03-24 07:27:51+00:00",
            "title": "Neural Abstractive Text Summarization and Fake News Detection",
            "authors": [
                "Soheil Esmaeilzadeh",
                "Gao Xian Peh",
                "Angela Xu"
            ],
            "abstract": "In this work, we study abstractive text summarization by exploring different\nmodels such as LSTM-encoder-decoder with attention, pointer-generator networks,\ncoverage mechanisms, and transformers. Upon extensive and careful\nhyperparameter tuning we compare the proposed architectures against each other\nfor the abstractive text summarization task. Finally, as an extension of our\nwork, we apply our text summarization model as a feature extractor for a fake\nnews detection task where the news articles prior to classification will be\nsummarized and the results are compared against the classification using only\nthe original news text.\n  keywords: LSTM, encoder-deconder, abstractive text summarization,\npointer-generator, coverage mechanism, transformers, fake news detection",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00805v1",
            "date": "2019-03-29 15:53:28+00:00",
            "title": "A Convolutional Neural Network for Language-Agnostic Source Code Summarization",
            "authors": [
                "Jessica Moore",
                "Ben Gelman",
                "David Slater"
            ],
            "abstract": "Descriptive comments play a crucial role in the software engineering process.\nThey decrease development time, enable better bug detection, and facilitate the\nreuse of previously written code. However, comments are commonly the last of a\nsoftware developer's priorities and are thus either insufficient or missing\nentirely. Automatic source code summarization may therefore have the ability to\nsignificantly improve the software development process. We introduce a novel\nencoder-decoder model that summarizes source code, effectively writing a\ncomment to describe the code's functionality. We make two primary innovations\nbeyond current source code summarization models. First, our encoder is fully\nlanguage-agnostic and requires no complex input preprocessing. Second, our\ndecoder has an open vocabulary, enabling it to predict any word, even ones not\nseen in training. We demonstrate results comparable to state-of-the-art methods\non a single-language data set and provide the first results on a data set\nconsisting of multiple programming languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.00962v5",
            "date": "2019-04-01 16:53:35+00:00",
            "title": "Large Batch Optimization for Deep Learning: Training BERT in 76 minutes",
            "authors": [
                "Yang You",
                "Jing Li",
                "Sashank Reddi",
                "Jonathan Hseu",
                "Sanjiv Kumar",
                "Srinadh Bhojanapalli",
                "Xiaodan Song",
                "James Demmel",
                "Kurt Keutzer",
                "Cho-Jui Hsieh"
            ],
            "abstract": "Training large deep neural networks on massive datasets is computationally\nvery challenging. There has been recent surge in interest in using large batch\nstochastic optimization methods to tackle this issue. The most prominent\nalgorithm in this line of research is LARS, which by employing layerwise\nadaptive learning rates trains ResNet on ImageNet in a few minutes. However,\nLARS performs poorly for attention models like BERT, indicating that its\nperformance gains are not consistent across tasks. In this paper, we first\nstudy a principled layerwise adaptation strategy to accelerate training of deep\nneural networks using large mini-batches. Using this strategy, we develop a new\nlayerwise adaptive large batch optimization technique called LAMB; we then\nprovide convergence analysis of LAMB as well as LARS, showing convergence to a\nstationary point in general nonconvex settings. Our empirical results\ndemonstrate the superior performance of LAMB across various tasks such as BERT\nand ResNet-50 training with very little hyperparameter tuning. In particular,\nfor BERT training, our optimizer enables use of very large batch sizes of 32868\nwithout any degradation of performance. By increasing the batch size to the\nmemory limit of a TPUv3 Pod, BERT training time can be reduced from 3 days to\njust 76 minutes (Table 1). The LAMB implementation is available at\nhttps://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.00977v2",
            "date": "2019-04-01 17:28:35+00:00",
            "title": "Sentiment analysis with genetically evolved Gaussian kernels",
            "authors": [
                "Ibai Roman",
                "Alexander Mendiburu",
                "Roberto Santana",
                "Jose A. Lozano"
            ],
            "abstract": "Sentiment analysis consists of evaluating opinions or statements from the\nanalysis of text. Among the methods used to estimate the degree in which a text\nexpresses a given sentiment, are those based on Gaussian Processes. However,\ntraditional Gaussian Processes methods use a predefined kernel with\nhyperparameters that can be tuned but whose structure can not be adapted. In\nthis paper, we propose the application of Genetic Programming for evolving\nGaussian Process kernels that are more precise for sentiment analysis. We use\nuse a very flexible representation of kernels combined with a multi-objective\napproach that simultaneously considers two quality metrics and the\ncomputational time spent by the kernels. Our results show that the algorithm\ncan outperform Gaussian Processes with traditional kernels for some of the\nsentiment analysis tasks considered.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01120v1",
            "date": "2019-04-01 21:47:00+00:00",
            "title": "ASSERT: Anti-Spoofing with Squeeze-Excitation and Residual neTworks",
            "authors": [
                "Cheng-I Lai",
                "Nanxin Chen",
                "Jes\u00fas Villalba",
                "Najim Dehak"
            ],
            "abstract": "We present JHU's system submission to the ASVspoof 2019 Challenge:\nAnti-Spoofing with Squeeze-Excitation and Residual neTworks (ASSERT).\nAnti-spoofing has gathered more and more attention since the inauguration of\nthe ASVspoof Challenges, and ASVspoof 2019 dedicates to address attacks from\nall three major types: text-to-speech, voice conversion, and replay. Built upon\nprevious research work on Deep Neural Network (DNN), ASSERT is a pipeline for\nDNN-based approach to anti-spoofing. ASSERT has four components: feature\nengineering, DNN models, network optimization and system combination, where the\nDNN models are variants of squeeze-excitation and residual networks. We\nconducted an ablation study of the effectiveness of each component on the\nASVspoof 2019 corpus, and experimental results showed that ASSERT obtained more\nthan 93% and 17% relative improvements over the baseline systems in the two\nsub-challenges in ASVspooof 2019, ranking ASSERT one of the top performing\nsystems. Code and pretrained models will be made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01138v2",
            "date": "2019-04-01 23:08:05+00:00",
            "title": "Benchmarking Approximate Inference Methods for Neural Structured Prediction",
            "authors": [
                "Lifu Tu",
                "Kevin Gimpel"
            ],
            "abstract": "Exact structured inference with neural network scoring functions is\ncomputationally challenging but several methods have been proposed for\napproximating inference. One approach is to perform gradient descent with\nrespect to the output structure directly (Belanger and McCallum, 2016). Another\napproach, proposed recently, is to train a neural network (an \"inference\nnetwork\") to perform inference (Tu and Gimpel, 2018). In this paper, we compare\nthese two families of inference methods on three sequence labeling datasets. We\nchoose sequence labeling because it permits us to use exact inference as a\nbenchmark in terms of speed, accuracy, and search error. Across datasets, we\ndemonstrate that inference networks achieve a better speed/accuracy/search\nerror trade-off than gradient descent, while also being faster than exact\ninference at similar accuracy levels. We find further benefit by combining\ninference networks and gradient descent, using the former to provide a warm\nstart for the latter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01575v1",
            "date": "2019-04-01 23:54:08+00:00",
            "title": "Contrastive Predictive Coding Based Feature for Automatic Speaker Verification",
            "authors": [
                "Cheng-I Lai"
            ],
            "abstract": "This thesis describes our ongoing work on Contrastive Predictive Coding (CPC)\nfeatures for speaker verification. CPC is a recently proposed representation\nlearning framework based on predictive coding and noise contrastive estimation.\nWe focus on incorporating CPC features into the standard automatic speaker\nverification systems, and we present our methods, experiments, and analysis.\nThis thesis also details necessary background knowledge in past and recent work\non automatic speaker verification systems, conventional speech features, and\nthe motivation and techniques behind CPC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01650v2",
            "date": "2019-04-02 20:18:52+00:00",
            "title": "Improving Robot Success Detection using Static Object Data",
            "authors": [
                "Rosario Scalise",
                "Jesse Thomason",
                "Yonatan Bisk",
                "Siddhartha Srinivasa"
            ],
            "abstract": "We use static object data to improve success detection for stacking objects\non and nesting objects in one another. Such actions are necessary for certain\nrobotics tasks, e.g., clearing a dining table or packing a warehouse bin.\nHowever, using an RGB-D camera to detect success can be insufficient:\nsame-colored objects can be difficult to differentiate, and reflective\nsilverware cause noisy depth camera perception. We show that adding static data\nabout the objects themselves improves the performance of an end-to-end pipeline\nfor classifying action outcomes. Images of the objects, and language\nexpressions describing them, encode prior geometry, shape, and size information\nthat refine classification accuracy. We collect over 13 hours of egocentric\nmanipulation data for training a model to reason about whether a robot\nsuccessfully placed unseen objects in or on one another. The model achieves up\nto a 57% absolute gain over the task baseline on pairs of previously unseen\nobjects.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1904.01664v1",
            "date": "2019-04-02 20:51:27+00:00",
            "title": "Mirroring to Build Trust in Digital Assistants",
            "authors": [
                "Katherine Metcalf",
                "Barry-John Theobald",
                "Garrett Weinberg",
                "Robert Lee",
                "Ing-Marie Jonsson",
                "Russ Webb",
                "Nicholas Apostoloff"
            ],
            "abstract": "We describe experiments towards building a conversational digital assistant\nthat considers the preferred conversational style of the user. In particular,\nthese experiments are designed to measure whether users prefer and trust an\nassistant whose conversational style matches their own. To this end we\nconducted a user study where subjects interacted with a digital assistant that\nresponded in a way that either matched their conversational style, or did not.\nUsing self-reported personality attributes and subjects' feedback on the\ninteractions, we built models that can reliably predict a user's preferred\nconversational style.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1904.02020v2",
            "date": "2019-04-03 14:24:04+00:00",
            "title": "Jointly Extracting and Compressing Documents with Summary State Representations",
            "authors": [
                "Afonso Mendes",
                "Shashi Narayan",
                "Sebasti\u00e3o Miranda",
                "Zita Marinho",
                "Andr\u00e9 F. T. Martins",
                "Shay B. Cohen"
            ],
            "abstract": "We present a new neural model for text summarization that first extracts\nsentences from a document and then compresses them. The proposed model offers a\nbalance that sidesteps the difficulties in abstractive methods while generating\nmore concise summaries than extractive methods. In addition, our model\ndynamically determines the length of the output summary based on the gold\nsummaries it observes during training and does not require length constraints\ntypical to extractive summarization. The model achieves state-of-the-art\nresults on the CNN/DailyMail and Newsroom datasets, improving over current\nextractive and abstractive methods. Human evaluations demonstrate that our\nmodel generates concise and informative summaries. We also make available a new\ndataset of oracle compressive summaries derived automatically from the\nCNN/DailyMail reference summaries.",
            "journal ref": "NAACL 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.02293v1",
            "date": "2019-04-04 01:17:29+00:00",
            "title": "Generative Adversarial Networks for text using word2vec intermediaries",
            "authors": [
                "Akshay Budhkar",
                "Krishnapriya Vishnubhotla",
                "Safwan Hossain",
                "Frank Rudzicz"
            ],
            "abstract": "Generative adversarial networks (GANs) have shown considerable success,\nespecially in the realistic generation of images. In this work, we apply\nsimilar techniques for the generation of text. We propose a novel approach to\nhandle the discrete nature of text, during training, using word embeddings. Our\nmethod is agnostic to vocabulary size and achieves competitive results relative\nto methods with various discrete gradient estimators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02338v2",
            "date": "2019-04-04 03:49:05+00:00",
            "title": "Consistency by Agreement in Zero-shot Neural Machine Translation",
            "authors": [
                "Maruan Al-Shedivat",
                "Ankur P. Parikh"
            ],
            "abstract": "Generalization and reliability of multilingual translation often highly\ndepend on the amount of available parallel data for each language pair of\ninterest. In this paper, we focus on zero-shot generalization---a challenging\nsetup that tests models on translation directions they have not been optimized\nfor at training time. To solve the problem, we (i) reformulate multilingual\ntranslation as probabilistic inference, (ii) define the notion of zero-shot\nconsistency and show why standard training often results in models unsuitable\nfor zero-shot tasks, and (iii) introduce a consistent agreement-based training\nmethod that encourages the model to produce equivalent translations of parallel\nsentences in auxiliary languages. We test our multilingual NMT models on\nmultiple public zero-shot translation benchmarks (IWSLT17, UN corpus, Europarl)\nand show that agreement-based learning often results in 2-3 BLEU zero-shot\nimprovement over strong baselines without any loss in performance on supervised\ntranslation directions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.02418v1",
            "date": "2019-04-04 09:11:24+00:00",
            "title": "Learning to Decipher Hate Symbols",
            "authors": [
                "Jing Qian",
                "Mai ElSherief",
                "Elizabeth Belding",
                "William Yang Wang"
            ],
            "abstract": "Existing computational models to understand hate speech typically frame the\nproblem as a simple classification task, bypassing the understanding of hate\nsymbols (e.g., 14 words, kigy) and their secret connotations. In this paper, we\npropose a novel task of deciphering hate symbols. To do this, we leverage the\nUrban Dictionary and collected a new, symbol-rich Twitter corpus of hate\nspeech. We investigate neural network latent context models for deciphering\nhate symbols. More specifically, we study Sequence-to-Sequence models and show\nhow they are able to crack the ciphers based on context. Furthermore, we\npropose a novel Variational Decipher and show how it can generalize better to\nunseen hate symbols in a more challenging testing setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02547v1",
            "date": "2019-04-04 13:34:18+00:00",
            "title": "Robust Evaluation of Language-Brain Encoding Experiments",
            "authors": [
                "Lisa Beinborn",
                "Samira Abnar",
                "Rochelle Choenni"
            ],
            "abstract": "Language-brain encoding experiments evaluate the ability of language models\nto predict brain responses elicited by language stimuli. The evaluation\nscenarios for this task have not yet been standardized which makes it difficult\nto compare and interpret results. We perform a series of evaluation experiments\nwith a consistent encoding setup and compute the results for multiple fMRI\ndatasets. In addition, we test the sensitivity of the evaluation measures to\nrandomized data and analyze the effect of voxel selection methods. Our\nexperimental framework is publicly available to make modelling decisions more\ntransparent and support reproducibility for future comparisons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02790v1",
            "date": "2019-04-04 20:59:20+00:00",
            "title": "In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Data",
            "authors": [
                "Nishant Prateek",
                "Mateusz \u0141ajszczak",
                "Roberto Barra-Chicote",
                "Thomas Drugman",
                "Jaime Lorenzo-Trueba",
                "Thomas Merritt",
                "Srikanth Ronanki",
                "Trevor Wood"
            ],
            "abstract": "Neural text-to-speech synthesis (NTTS) models have shown significant progress\nin generating high-quality speech, however they require a large quantity of\ntraining data. This makes creating models for multiple styles expensive and\ntime-consuming. In this paper different styles of speech are analysed based on\nprosodic variations, from this a model is proposed to synthesise speech in the\nstyle of a newscaster, with just a few hours of supplementary data. We pose the\nproblem of synthesising in a target style using limited data as that of\ncreating a bi-style model that can synthesise both neutral-style and\nnewscaster-style speech via a one-hot vector which factorises the two styles.\nWe also propose conditioning the model on contextual word embeddings, and\nextensively evaluate it against neutral NTTS, and neutral concatenative-based\nsynthesis. This model closes the gap in perceived style-appropriateness between\nnatural recordings for newscaster-style of speech, and neutral speech synthesis\nby approximately two-thirds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02792v1",
            "date": "2019-04-04 21:03:34+00:00",
            "title": "Unifying Human and Statistical Evaluation for Natural Language Generation",
            "authors": [
                "Tatsunori B. Hashimoto",
                "Hugh Zhang",
                "Percy Liang"
            ],
            "abstract": "How can we measure whether a natural language generation system produces both\nhigh quality and diverse outputs? Human evaluation captures quality but not\ndiversity, as it does not catch models that simply plagiarize from the training\nset. On the other hand, statistical evaluation (i.e., perplexity) captures\ndiversity but not quality, as models that occasionally emit low quality samples\nwould be insufficiently penalized. In this paper, we propose a unified\nframework which evaluates both diversity and quality, based on the optimal\nerror rate of predicting whether a sentence is human- or machine-generated. We\ndemonstrate that this error rate can be efficiently estimated by combining\nhuman and statistical evaluation, using an evaluation metric which we call\nHUSE. On summarization and chit-chat dialogue, we show that (i) HUSE detects\ndiversity defects which fool pure human evaluation and that (ii) techniques\nsuch as annealing for improving quality actually decrease HUSE due to decreased\ndiversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02793v1",
            "date": "2019-04-04 21:05:13+00:00",
            "title": "Affect-Driven Dialog Generation",
            "authors": [
                "Pierre Colombo",
                "Wojciech Witon",
                "Ashutosh Modi",
                "James Kennedy",
                "Mubbasir Kapadia"
            ],
            "abstract": "The majority of current systems for end-to-end dialog generation focus on\nresponse quality without an explicit control over the affective content of the\nresponses. In this paper, we present an affect-driven dialog system, which\ngenerates emotional responses in a controlled manner using a continuous\nrepresentation of emotions. The system achieves this by modeling emotions at a\nword and sequence level using: (1) a vector representation of the desired\nemotion, (2) an affect regularizer, which penalizes neutral words, and (3) an\naffect sampling method, which forces the neural network to generate diverse\nwords that are emotionally relevant. During inference, we use a reranking\nprocedure that aims to extract the most emotionally relevant responses using a\nhuman-in-the-loop optimization process. We study the performance of our system\nin terms of both quantitative (BLEU score and response diversity), and\nqualitative (emotional appropriateness) measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02815v1",
            "date": "2019-04-04 22:54:57+00:00",
            "title": "Topic Spotting using Hierarchical Networks with Self Attention",
            "authors": [
                "Pooja Chitkara",
                "Ashutosh Modi",
                "Pravalika Avvaru",
                "Sepehr Janghorbani",
                "Mubbasir Kapadia"
            ],
            "abstract": "Success of deep learning techniques have renewed the interest in development\nof dialogue systems. However, current systems struggle to have consistent long\nterm conversations with the users and fail to build rapport. Topic spotting,\nthe task of automatically inferring the topic of a conversation, has been shown\nto be helpful in making a dialog system more engaging and efficient. We propose\na hierarchical model with self attention for topic spotting. Experiments on the\nSwitchboard corpus show the superior performance of our model over previously\nproposed techniques for topic spotting and deep models for text classification.\nAdditionally, in contrast to offline processing of dialog, we also analyze the\nperformance of our model in a more realistic setting i.e. in an online setting\nwhere the topic is identified in real time as the dialog progresses. Results\nshow that our model is able to generalize even with limited information in the\nonline setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02817v2",
            "date": "2019-04-04 23:05:45+00:00",
            "title": "Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling",
            "authors": [
                "Xiaochuang Han",
                "Jacob Eisenstein"
            ],
            "abstract": "Contextualized word embeddings such as ELMo and BERT provide a foundation for\nstrong performance across a wide range of natural language processing tasks by\npretraining on large corpora of unlabeled text. However, the applicability of\nthis approach is unknown when the target domain varies substantially from the\npretraining corpus. We are specifically interested in the scenario in which\nlabeled data is available in only a canonical source domain such as newstext,\nand the target domain is distinct from both the labeled and pretraining texts.\nTo address this scenario, we propose domain-adaptive fine-tuning, in which the\ncontextualized embeddings are adapted by masked language modeling on text from\nthe target domain. We test this approach on sequence labeling in two\nchallenging domains: Early Modern English and Twitter. Both domains differ\nsubstantially from existing pretraining corpora, and domain-adaptive\nfine-tuning yields substantial improvements over strong BERT baselines, with\nparticularly impressive results on out-of-vocabulary words. We conclude that\ndomain-adaptive fine-tuning offers a simple and effective approach for the\nunsupervised adaptation of sequence labeling to difficult new domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.02818v1",
            "date": "2019-04-04 23:06:09+00:00",
            "title": "Neural Networks for Modeling Source Code Edits",
            "authors": [
                "Rui Zhao",
                "David Bieber",
                "Kevin Swersky",
                "Daniel Tarlow"
            ],
            "abstract": "Programming languages are emerging as a challenging and interesting domain\nfor machine learning. A core task, which has received significant attention in\nrecent years, is building generative models of source code. However, to our\nknowledge, previous generative models have always been framed in terms of\ngenerating static snapshots of code. In this work, we instead treat source code\nas a dynamic object and tackle the problem of modeling the edits that software\ndevelopers make to source code files. This requires extracting intent from\nprevious edits and leveraging it to generate subsequent edits. We develop\nseveral neural networks and use synthetic data to test their ability to learn\nchallenging edit patterns that require strong generalization. We then collect\nand train our models on a large-scale dataset of Google source code, consisting\nof millions of fine-grained edits from thousands of Python developers. From the\nmodeling perspective, our main conclusion is that a new composition of\nattentional and pointer network components provides the best overall\nperformance and scalability. From the application perspective, our results\nprovide preliminary evidence of the feasibility of developing tools that learn\nto predict future edits.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.03084v1",
            "date": "2019-04-05 14:25:25+00:00",
            "title": "CLEARumor at SemEval-2019 Task 7: ConvoLving ELMo Against Rumors",
            "authors": [
                "Ipek Baris",
                "Lukas Schmelzeisen",
                "Steffen Staab"
            ],
            "abstract": "This paper describes our submission to SemEval-2019 Task 7: RumourEval:\nDetermining Rumor Veracity and Support for Rumors. We participated in both\nsubtasks. The goal of subtask A is to classify the type of interaction between\na rumorous social media post and a reply post as support, query, deny, or\ncomment. The goal of subtask B is to predict the veracity of a given rumor. For\nsubtask A, we implement a CNN-based neural architecture using ELMo embeddings\nof post text combined with auxiliary features and achieve a F1-score of 44.6%.\nFor subtask B, we employ a MLP neural network leveraging our estimates for\nsubtask A and achieve a F1-score of 30.1% (second place in the competition). We\nprovide results and analysis of our system performance and present ablation\nexperiments.",
            "journal ref": "SemEval@NAACL-HLT (2019) 1105-1109",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03152v1",
            "date": "2019-04-05 16:42:44+00:00",
            "title": "Data-driven Modelling of Dynamical Systems Using Tree Adjoining Grammar and Genetic Programming",
            "authors": [
                "Dhruv Khandelwal",
                "Maarten Schoukens",
                "Roland T\u00f3th"
            ],
            "abstract": "State-of-the-art methods for data-driven modelling of non-linear dynamical\nsystems typically involve interactions with an expert user. In order to\npartially automate the process of modelling physical systems from data, many\nEA-based approaches have been proposed for model-structure selection, with\nspecial focus on non-linear systems. Recently, an approach for data-driven\nmodelling of non-linear dynamical systems using Genetic Programming (GP) was\nproposed. The novelty of the method was the modelling of noise and the use of\nTree Adjoining Grammar to shape the search-space explored by GP. In this paper,\nwe report results achieved by the proposed method on three case studies. Each\nof the case studies considered here is based on real physical systems. The case\nstudies pose a variety of challenges. In particular, these challenges range\nover varying amounts of prior knowledge of the true system, amount of data\navailable, the complexity of the dynamics of the system, and the nature of\nnon-linearities in the system. Based on the results achieved for the case\nstudies, we critically analyse the performance of the proposed method.",
            "journal ref": "None",
            "category": "cs.SY"
        },
        {
            "url": "http://arxiv.org/abs/1904.03172v1",
            "date": "2019-04-05 17:44:12+00:00",
            "title": "Improving Scientific Article Visibility by Neural Title Simplification",
            "authors": [
                "Alexander Shvets"
            ],
            "abstract": "The rapidly growing amount of data that scientific content providers should\ndeliver to a user makes them create effective recommendation tools. A title of\nan article is often the only shown element to attract people's attention. We\noffer an approach to automatic generating titles with various levels of\ninformativeness to benefit from different categories of users. Statistics from\nResearchGate used to bias train datasets and specially designed post-processing\nstep applied to neural sequence-to-sequence models allow reaching the desired\nvariety of simplified titles to gain a trade-off between the attractiveness and\ntransparency of recommendation.",
            "journal ref": "Proceedings of the 8th International Workshop on\n  Bibliometric-enhanced Information Retrieval (BIR) co-located with ECIR 2019,\n  Cologne, Germany (pp. 140-147)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.03240v2",
            "date": "2019-04-05 19:04:19+00:00",
            "title": "An Unsupervised Autoregressive Model for Speech Representation Learning",
            "authors": [
                "Yu-An Chung",
                "Wei-Ning Hsu",
                "Hao Tang",
                "James Glass"
            ],
            "abstract": "This paper proposes a novel unsupervised autoregressive neural model for\nlearning generic speech representations. In contrast to other speech\nrepresentation learning methods that aim to remove noise or speaker\nvariabilities, ours is designed to preserve information for a wide range of\ndownstream tasks. In addition, the proposed model does not require any phonetic\nor word boundary labels, allowing the model to benefit from large quantities of\nunlabeled data. Speech representations learned by our model significantly\nimprove performance on both phone classification and speaker verification over\nthe surface features and other supervised and unsupervised approaches. Further\nanalysis shows that different levels of speech information are captured by our\nmodel at different layers. In particular, the lower layers tend to be more\ndiscriminative for speakers, while the upper layers provide more phonetic\ncontent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03266v1",
            "date": "2019-04-05 20:27:26+00:00",
            "title": "Domain Authoring Assistant for Intelligent Virtual Agents",
            "authors": [
                "Sepehr Janghorbani",
                "Ashutosh Modi",
                "Jakob Buhmann",
                "Mubbasir Kapadia"
            ],
            "abstract": "Developing intelligent virtual characters has attracted a lot of attention in\nthe recent years. The process of creating such characters often involves a team\nof creative authors who describe different aspects of the characters in natural\nlanguage, and planning experts that translate this description into a planning\ndomain. This can be quite challenging as the team of creative authors should\ndiligently define every aspect of the character especially if it contains\ncomplex human-like behavior. Also a team of engineers has to manually translate\nthe natural language description of a character's personality into the planning\ndomain knowledge. This can be extremely time and resource demanding and can be\nan obstacle to author's creativity. The goal of this paper is to introduce an\nauthoring assistant tool to automate the process of domain generation from\nnatural language description of virtual characters, thus bridging between the\ncreative authoring team and the planning domain experts. Moreover, the proposed\ntool also identifies possible missing information in the domain description and\niteratively makes suggestions to the author.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1904.03288v3",
            "date": "2019-04-05 21:35:44+00:00",
            "title": "Jasper: An End-to-End Convolutional Neural Acoustic Model",
            "authors": [
                "Jason Li",
                "Vitaly Lavrukhin",
                "Boris Ginsburg",
                "Ryan Leary",
                "Oleksii Kuchaiev",
                "Jonathan M. Cohen",
                "Huyen Nguyen",
                "Ravi Teja Gadde"
            ],
            "abstract": "In this paper, we report state-of-the-art results on LibriSpeech among\nend-to-end speech recognition models without any external training data. Our\nmodel, Jasper, uses only 1D convolutions, batch normalization, ReLU, dropout,\nand residual connections. To improve training, we further introduce a new\nlayer-wise optimizer called NovoGrad. Through experiments, we demonstrate that\nthe proposed deep architecture performs as well or better than more complex\nchoices. Our deepest Jasper variant uses 54 convolutional layers. With this\narchitecture, we achieve 2.95% WER using a beam-search decoder with an external\nneural language model and 3.86% WER with a greedy decoder on LibriSpeech\ntest-clean. We also report competitive results on the Wall Street Journal and\nthe Hub5'00 conversational evaluation datasets.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.03446v3",
            "date": "2019-04-06 13:49:16+00:00",
            "title": "Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion",
            "authors": [
                "Hao Sun",
                "Xu Tan",
                "Jun-Wei Gan",
                "Hongzhi Liu",
                "Sheng Zhao",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Grapheme-to-phoneme (G2P) conversion is an important task in automatic speech\nrecognition and text-to-speech systems. Recently, G2P conversion is viewed as a\nsequence to sequence task and modeled by RNN or CNN based encoder-decoder\nframework. However, previous works do not consider the practical issues when\ndeploying G2P model in the production system, such as how to leverage\nadditional unlabeled data to boost the accuracy, as well as reduce model size\nfor online deployment. In this work, we propose token-level ensemble\ndistillation for G2P conversion, which can (1) boost the accuracy by distilling\nthe knowledge from additional unlabeled data, and (2) reduce the model size but\nmaintain the high accuracy, both of which are very practical and helpful in the\nonline production system. We use token-level knowledge distillation, which\nresults in better accuracy than the sequence-level counterpart. What is more,\nwe adopt the Transformer instead of RNN or CNN based models to further boost\nthe accuracy of G2P conversion. Experiments on the publicly available CMUDict\ndataset and an internal English dataset demonstrate the effectiveness of our\nproposed method. Particularly, our method achieves 19.88% WER on CMUDict\ndataset, outperforming the previous works by more than 4.22% WER, and setting\nthe new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03461v1",
            "date": "2019-04-06 14:50:11+00:00",
            "title": "Embodied Question Answering in Photorealistic Environments with Point Cloud Perception",
            "authors": [
                "Erik Wijmans",
                "Samyak Datta",
                "Oleksandr Maksymets",
                "Abhishek Das",
                "Georgia Gkioxari",
                "Stefan Lee",
                "Irfan Essa",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "To help bridge the gap between internet vision-style problems and the goal of\nvision for embodied perception we instantiate a large-scale navigation task --\nEmbodied Question Answering [1] in photo-realistic environments (Matterport\n3D). We thoroughly study navigation policies that utilize 3D point clouds, RGB\nimages, or their combination. Our analysis of these models reveals several key\nfindings. We find that two seemingly naive navigation baselines, forward-only\nand random, are strong navigators and challenging to outperform, due to the\nspecific choice of the evaluation setting presented by [1]. We find a novel\nloss-weighting scheme we call Inflection Weighting to be important when\ntraining recurrent models for navigation with behavior cloning and are able to\nout perform the baselines with this technique. We find that point clouds\nprovide a richer signal than RGB images for learning obstacle avoidance,\nmotivating the use (and continued study) of 3D deep learning models for\nembodied navigation.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.03478v2",
            "date": "2019-04-06 15:47:13+00:00",
            "title": "The Mathematics of Text Structure",
            "authors": [
                "Bob Coecke"
            ],
            "abstract": "In previous work we gave a mathematical foundation, referred to as DisCoCat,\nfor how words interact in a sentence in order to produce the meaning of that\nsentence. To do so, we exploited the perfect structural match of grammar and\ncategories of meaning spaces. Here, we give a mathematical foundation, referred\nto as DisCoCirc, for how sentences interact in texts in order to produce the\nmeaning of that text. First we revisit DisCoCat. While in DisCoCat all meanings\nare fixed as states (i.e. have no input), in DisCoCirc word meanings correspond\nto a type, or system, and the states of this system can evolve. Sentences are\ngates within a circuit which update the variable meanings of those words. Like\nin DisCoCat, word meanings can live in a variety of spaces e.g. propositional,\nvectorial, or cognitive. The compositional structure are string diagrams\nrepresenting information flows, and an entire text yields a single string\ndiagram in which word meanings lift to the meaning of an entire text. While the\ndevelopments in this paper are independent of a physical embodiment (cf.\nclassical vs. quantum computing), both the compositional formalism and\nsuggested meaning model are highly quantum-inspired, and implementation on a\nquantum computer would come with a range of benefits. We also praise Jim Lambek\nfor his role in mathematical linguistics in general, and the development of the\nDisCo program more specifically.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03493v3",
            "date": "2019-04-06 16:50:31+00:00",
            "title": "VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research",
            "authors": [
                "Xin Wang",
                "Jiawei Wu",
                "Junkun Chen",
                "Lei Li",
                "Yuan-Fang Wang",
                "William Yang Wang"
            ],
            "abstract": "We present a new large-scale multilingual video description dataset, VATEX,\nwhich contains over 41,250 videos and 825,000 captions in both English and\nChinese. Among the captions, there are over 206,000 English-Chinese parallel\ntranslation pairs. Compared to the widely-used MSR-VTT dataset, VATEX is\nmultilingual, larger, linguistically complex, and more diverse in terms of both\nvideo and natural language descriptions. We also introduce two tasks for\nvideo-and-language research based on VATEX: (1) Multilingual Video Captioning,\naimed at describing a video in various languages with a compact unified\ncaptioning model, and (2) Video-guided Machine Translation, to translate a\nsource language description into the target language using the video\ninformation as additional spatiotemporal context. Extensive experiments on the\nVATEX dataset show that, first, the unified multilingual model can not only\nproduce both English and Chinese descriptions for a video more efficiently, but\nalso offer improved performance over the monolingual models. Furthermore, we\ndemonstrate that the spatiotemporal video context can be effectively utilized\nto align source and target languages and thus assist machine translation. In\nthe end, we discuss the potentials of using VATEX for other video-and-language\nresearch.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.03513v1",
            "date": "2019-04-06 19:04:29+00:00",
            "title": "Team QCRI-MIT at SemEval-2019 Task 4: Propaganda Analysis Meets Hyperpartisan News Detection",
            "authors": [
                "Abdelrhman Saleh",
                "Ramy Baly",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Giovanni Da San Martino",
                "Mitra Mohtarami",
                "Preslav Nakov",
                "James Glass"
            ],
            "abstract": "In this paper, we describe our submission to SemEval-2019 Task 4 on\nHyperpartisan News Detection. Our system relies on a variety of engineered\nfeatures originally used to detect propaganda. This is based on the assumption\nthat biased messages are propagandistic in the sense that they promote a\nparticular political cause or viewpoint. We trained a logistic regression model\nwith features ranging from simple bag-of-words to vocabulary richness and text\nreadability features. Our system achieved 72.9% accuracy on the test data that\nis annotated manually and 60.8% on the test data that is annotated with distant\nsupervision. Additional experiments showed that significant performance\nimprovements can be achieved with better feature pre-processing.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.03576v3",
            "date": "2019-04-07 03:18:44+00:00",
            "title": "Spoken Language Intent Detection using Confusion2Vec",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Mu Yang",
                "Panayiotis Georgiou"
            ],
            "abstract": "Decoding speaker's intent is a crucial part of spoken language understanding\n(SLU). The presence of noise or errors in the text transcriptions, in real life\nscenarios make the task more challenging. In this paper, we address the spoken\nlanguage intent detection under noisy conditions imposed by automatic speech\nrecognition (ASR) systems. We propose to employ confusion2vec word feature\nrepresentation to compensate for the errors made by ASR and to increase the\nrobustness of the SLU system. The confusion2vec, motivated from human speech\nproduction and perception, models acoustic relationships between words in\naddition to the semantic and syntactic relations of words in human language. We\nhypothesize that ASR often makes errors relating to acoustically similar words,\nand the confusion2vec with inherent model of acoustic relationships between\nwords is able to compensate for the errors. We demonstrate through experiments\non the ATIS benchmark dataset, the robustness of the proposed model to achieve\nstate-of-the-art results under noisy ASR conditions. Our system reduces\nclassification error rate (CER) by 20.84% and improves robustness by 37.48%\n(lower CER degradation) relative to the previous state-of-the-art going from\nclean to noisy transcripts. Improvements are also demonstrated when training\nthe intent detection models on noisy transcripts.",
            "journal ref": "Proceedings of Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03595v1",
            "date": "2019-04-07 06:46:36+00:00",
            "title": "Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging",
            "authors": [
                "Sara Meftah",
                "Youssef Tamaazousti",
                "Nasredine Semmar",
                "Hassane Essafi",
                "Fatiha Sadat"
            ],
            "abstract": "Fine-tuning neural networks is widely used to transfer valuable knowledge\nfrom high-resource to low-resource domains. In a standard fine-tuning scheme,\nsource and target problems are trained using the same architecture. Although\ncapable of adapting to new domains, pre-trained units struggle with learning\nuncommon target-specific patterns. In this paper, we propose to augment the\ntarget-network with normalised, weighted and randomly initialised units that\nbeget a better adaptation while maintaining the valuable source knowledge. Our\nexperiments on POS tagging of social media texts (Tweets domain) demonstrate\nthat our method achieves state-of-the-art performances on 3 commonly used\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03670v2",
            "date": "2019-04-07 15:24:32+00:00",
            "title": "Speech Model Pre-training for End-to-End Spoken Language Understanding",
            "authors": [
                "Loren Lugosch",
                "Mirco Ravanelli",
                "Patrick Ignoto",
                "Vikrant Singh Tomar",
                "Yoshua Bengio"
            ],
            "abstract": "Whereas conventional spoken language understanding (SLU) systems map speech\nto text, and then text to intent, end-to-end SLU systems map speech directly to\nintent through a single trainable model. Achieving high accuracy with these\nend-to-end models without a large amount of training data is difficult. We\npropose a method to reduce the data requirements of end-to-end SLU in which the\nmodel is first pre-trained to predict words and phonemes, thus learning good\nfeatures for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and\nshow that our method improves performance both when the full dataset is used\nfor training and when only a small subset is used. We also describe preliminary\nexperiments to gauge the model's ability to generalize to new phrases not heard\nduring training.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.03885v1",
            "date": "2019-04-08 08:28:54+00:00",
            "title": "Referring to Objects in Videos using Spatio-Temporal Identifying Descriptions",
            "authors": [
                "Peratham Wiriyathammabhum",
                "Abhinav Shrivastava",
                "Vlad I. Morariu",
                "Larry S. Davis"
            ],
            "abstract": "This paper presents a new task, the grounding of spatio-temporal identifying\ndescriptions in videos. Previous work suggests potential bias in existing\ndatasets and emphasizes the need for a new data creation schema to better model\nlinguistic structure. We introduce a new data collection scheme based on\ngrammatical constraints for surface realization to enable us to investigate the\nproblem of grounding spatio-temporal identifying descriptions in videos. We\nthen propose a two-stream modular attention network that learns and grounds\nspatio-temporal identifying descriptions based on appearance and motion. We\nshow that motion modules help to ground motion-related words and also help to\nlearn in appearance modules because modular neural networks resolve task\ninterference between modules. Finally, we propose a future challenge and a need\nfor a robust system arising from replacing ground truth visual annotations with\nautomatic video object detector and temporal event localization.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.03898v1",
            "date": "2019-04-08 09:07:30+00:00",
            "title": "Semi-Supervised Few-Shot Learning for Dual Question-Answer Extraction",
            "authors": [
                "Jue Wang",
                "Ke Chen",
                "Lidan Shou",
                "Sai Wu",
                "Sharad Mehrotra"
            ],
            "abstract": "This paper addresses the problem of key phrase extraction from sentences.\nExisting state-of-the-art supervised methods require large amounts of annotated\ndata to achieve good performance and generalization. Collecting labeled data\nis, however, often expensive. In this paper, we redefine the problem as\nquestion-answer extraction, and present SAMIE: Self-Asking Model for\nInformation Ixtraction, a semi-supervised model which dually learns to ask and\nto answer questions by itself. Briefly, given a sentence $s$ and an answer $a$,\nthe model needs to choose the most appropriate question $\\hat q$; meanwhile,\nfor the given sentence $s$ and same question $\\hat q$ selected in the previous\nstep, the model will predict an answer $\\hat a$. The model can support few-shot\nlearning with very limited supervision. It can also be used to perform\nclustering analysis when no supervision is provided. Experimental results show\nthat the proposed method outperforms typical supervised methods especially when\ngiven little labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.03971v2",
            "date": "2019-04-08 11:44:41+00:00",
            "title": "Jointly Measuring Diversity and Quality in Text Generation Models",
            "authors": [
                "Ehsan Montahaei",
                "Danial Alihosseini",
                "Mahdieh Soleymani Baghshah"
            ],
            "abstract": "Text generation is an important Natural Language Processing task with various\napplications. Although several metrics have already been introduced to evaluate\nthe text generation methods, each of them has its own shortcomings. The most\nwidely used metrics such as BLEU only consider the quality of generated\nsentences and neglect their diversity. For example, repeatedly generation of\nonly one high quality sentence would result in a high BLEU score. On the other\nhand, the more recent metric introduced to evaluate the diversity of generated\ntexts known as Self-BLEU ignores the quality of generated texts. In this paper,\nwe propose metrics to evaluate both the quality and diversity simultaneously by\napproximating the distance of the learned generative model and the real data\ndistribution. For this purpose, we first introduce a metric that approximates\nthis distance using n-gram based measures. Then, a feature-based measure which\nis based on a recent highly deep model trained on a large text corpus called\nBERT is introduced. Finally, for oracle training mode in which the generator's\ndensity can also be calculated, we propose to use the distance measures between\nthe corresponding explicit distributions. Eventually, the most popular and\nrecent text generation models are evaluated using both the existing and the\nproposed metrics and the preferences of the proposed metrics are determined.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.03991v1",
            "date": "2019-03-08 14:45:53+00:00",
            "title": "Source codes in human communication",
            "authors": [
                "Michael Ramscar"
            ],
            "abstract": "Although information theoretic characterizations of human communication have\nbecome increasingly popular in linguistics, to date they have largely involved\ngrafting probabilistic constructs onto older ideas about grammar. Similarities\nbetween human and digital communication have been strongly emphasized, and\ndifferences largely ignored. However, some of these differences matter:\ncommunication systems are based on predefined codes shared by every\nsender-receiver, whereas the distributions of words in natural languages\nguarantee that no speaker-hearer ever has access to an entire linguistic code,\nwhich seemingly undermines the idea that natural languages are probabilistic\nsystems in any meaningful sense. This paper describes how the distributional\nproperties of languages meet the various challenges arising from the\ndifferences between information systems and natural languages, along with the\nvery different view of human communication these properties suggest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04019v4",
            "date": "2019-04-02 15:49:02+00:00",
            "title": "Effectiveness of Data-Driven Induction of Semantic Spaces and Traditional Classifiers for Sarcasm Detection",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Giosu\u00e9 Lo Bosco",
                "Giovanni Pilato"
            ],
            "abstract": "Irony and sarcasm are two complex linguistic phenomena that are widely used\nin everyday language and especially over the social media, but they represent\ntwo serious issues for automated text understanding. Many labeled corpora have\nbeen extracted from several sources to accomplish this task, and it seems that\nsarcasm is conveyed in different ways for different domains. Nonetheless, very\nlittle work has been done for comparing different methods among the available\ncorpora. Furthermore, usually, each author collects and uses their own datasets\nto evaluate his own method. In this paper, we show that sarcasm detection can\nbe tackled by applying classical machine learning algorithms to input texts\nsub-symbolically represented in a Latent Semantic space. The main consequence\nis that our studies establish both reference datasets and baselines for the\nsarcasm detection problem that could serve the scientific community to test\nnewly proposed methods.",
            "journal ref": "Natural Language Engineering, 25(2), 257-285 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04021v1",
            "date": "2019-04-01 04:57:25+00:00",
            "title": "Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation",
            "authors": [
                "Tasnim Mohiuddin",
                "Thanh-Tung Nguyen",
                "Shafiq Joty"
            ],
            "abstract": "We address the problem of speech act recognition (SAR) in asynchronous\nconversations (forums, emails). Unlike synchronous conversations (e.g.,\nmeetings, phone), asynchronous domains lack large labeled datasets to train an\neffective SAR model. In this paper, we propose methods to effectively leverage\nabundant unlabeled conversational data and the available labeled data from\nsynchronous domains. We carry out our research in three main steps. First, we\nintroduce a neural architecture based on hierarchical LSTMs and conditional\nrandom fields (CRF) for SAR, and show that our method outperforms existing\nmethods when trained on in-domain data only. Second, we improve our initial SAR\nmodels by semi-supervised learning in the form of pretrained word embeddings\nlearned from a large unlabeled conversational corpus. Finally, we employ\nadversarial training to improve the results further by leveraging the labeled\ndata from synchronous domains and by explicitly modeling the distributional\nshift in two domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04047v3",
            "date": "2019-04-03 22:17:27+00:00",
            "title": "Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings",
            "authors": [
                "Thomas Manzini",
                "Yao Chong Lim",
                "Yulia Tsvetkov",
                "Alan W Black"
            ],
            "abstract": "Online texts -- across genres, registers, domains, and styles -- are riddled\nwith human stereotypes, expressed in overt or subtle ways. Word embeddings,\ntrained on these texts, perpetuate and amplify these stereotypes, and propagate\nbiases to machine learning models that use word embeddings as features. In this\nwork, we propose a method to debias word embeddings in multiclass settings such\nas race and religion, extending the work of (Bolukbasi et al., 2016) from the\nbinary setting, such as binary gender. Next, we propose a novel methodology for\nthe evaluation of multiclass debiasing. We demonstrate that our multiclass\ndebiasing is robust and maintains the efficacy in standard NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04049v1",
            "date": "2019-04-04 02:20:50+00:00",
            "title": "Simple Question Answering with Subgraph Ranking and Joint-Scoring",
            "authors": [
                "Wenbo Zhao",
                "Tagyoung Chung",
                "Anuj Goyal",
                "Angeliki Metallinou"
            ],
            "abstract": "Knowledge graph based simple question answering (KBSQA) is a major area of\nresearch within question answering. Although only dealing with simple\nquestions, i.e., questions that can be answered through a single knowledge base\n(KB) fact, this task is neither simple nor close to being solved. Targeting on\nthe two main steps, subgraph selection and fact selection, the research\ncommunity has developed sophisticated approaches. However, the importance of\nsubgraph ranking and leveraging the subject--relation dependency of a KB fact\nhave not been sufficiently explored. Motivated by this, we present a unified\nframework to describe and analyze existing approaches. Using this framework as\na starting point, we focus on two aspects: improving subgraph selection through\na novel ranking method and leveraging the subject--relation dependency by\nproposing a joint scoring CNN model with a novel loss function that enforces\nthe well-order of scores. Our methods achieve a new state of the art (85.44% in\naccuracy) on the SimpleQuestions dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04055v1",
            "date": "2019-04-03 14:47:30+00:00",
            "title": "Evaluating KGR10 Polish word embeddings in the recognition of temporal expressions using BiLSTM-CRF",
            "authors": [
                "Jan Koco\u0144",
                "Micha\u0142 Gawor"
            ],
            "abstract": "The article introduces a new set of Polish word embeddings, built using KGR10\ncorpus, which contains more than 4 billion words. These embeddings are\nevaluated in the problem of recognition of temporal expressions (timexes) for\nthe Polish language. We described the process of KGR10 corpus creation and a\nnew approach to the recognition problem using Bidirectional Long-Short Term\nMemory (BiLSTM) network with additional CRF layer, where specific embeddings\nare essential. We presented experiments and conclusions drawn from them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04096v1",
            "date": "2019-04-04 21:34:45+00:00",
            "title": "Deep Learning Sentiment Analysis of Amazon.com Reviews and Ratings",
            "authors": [
                "Nishit Shrestha",
                "Fatma Nasoz"
            ],
            "abstract": "Our study employs sentiment analysis to evaluate the compatibility of\nAmazon.com reviews with their corresponding ratings. Sentiment analysis is the\ntask of identifying and classifying the sentiment expressed in a piece of text\nas being positive or negative. On e-commerce websites such as Amazon.com,\nconsumers can submit their reviews along with a specific polarity rating. In\nsome instances, there is a mismatch between the review and the rating. To\nidentify the reviews with mismatched ratings we performed sentiment analysis\nusing deep learning on Amazon.com product review data. Product reviews were\nconverted to vectors using paragraph vector, which then was used to train a\nrecurrent neural network with gated recurrent unit. Our model incorporated both\nsemantic relationship of review text and product information. We also developed\na web service application that predicts the rating score for a submitted review\nusing the trained model and if there is a mismatch between predicted rating\nscore and submitted rating score, it provides feedback to the reviewer.",
            "journal ref": "International Journal on Soft Computing, Artificial Intelligence\n  and Applications (IJSCAI), Vol.8, No.1, February 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.04100v3",
            "date": "2019-04-08 14:47:45+00:00",
            "title": "Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Models",
            "authors": [
                "Kuan-Yu Chen",
                "Che-Ping Tsai",
                "Da-Rong Liu",
                "Hung-Yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Producing a large annotated speech corpus for training ASR systems remains\ndifficult for more than 95% of languages all over the world which are\nlow-resourced, but collecting a relatively big unlabeled data set for such\nlanguages is more achievable. This is why some initial effort have been\nreported on completely unsupervised speech recognition learned from unlabeled\ndata only, although with relatively high error rates. In this paper, we develop\na Generative Adversarial Network (GAN) to achieve this purpose, in which a\nGenerator and a Discriminator learn from each other iteratively to improve the\nperformance. We further use a set of Hidden Markov Models (HMMs) iteratively\nrefined from the machine generated labels to work in harmony with the GAN. The\ninitial experiments on TIMIT data set achieve an phone error rate of 33.1%,\nwhich is 8.5% lower than the previous state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04116v1",
            "date": "2019-04-04 12:46:07+00:00",
            "title": "Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training",
            "authors": [
                "Tasnim Mohiuddin",
                "Shafiq Joty"
            ],
            "abstract": "Adversarial training has shown impressive success in learning bilingual\ndictionary without any parallel data by mapping monolingual embeddings to a\nshared space. However, recent work has shown superior performance for\nnon-adversarial methods in more challenging language pairs. In this work, we\nrevisit adversarial autoencoder for unsupervised word translation and propose\ntwo novel extensions to it that yield more stable training and improved\nresults. Our method includes regularization terms to enforce cycle consistency\nand input reconstruction, and puts the target encoders as an adversary against\nthe corresponding discriminator. Extensive experimentations with European,\nnon-European and low-resource languages show that our method is more robust and\nachieves better performance than recently proposed adversarial and\nnon-adversarial approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04153v1",
            "date": "2019-04-08 16:05:43+00:00",
            "title": "AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning",
            "authors": [
                "Han Guo",
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Multi-task learning (MTL) has achieved success over a wide range of problems,\nwhere the goal is to improve the performance of a primary task using a set of\nrelevant auxiliary tasks. However, when the usefulness of the auxiliary tasks\nw.r.t. the primary task is not known a priori, the success of MTL models\ndepends on the correct choice of these auxiliary tasks and also a balanced\nmixing ratio of these tasks during alternate training. These two problems could\nbe resolved via manual intuition or hyper-parameter tuning over all\ncombinatorial task choices, but this introduces inductive bias or is not\nscalable when the number of candidate auxiliary tasks is very large. To address\nthese issues, we present AutoSeM, a two-stage MTL pipeline, where the first\nstage automatically selects the most useful auxiliary tasks via a\nBeta-Bernoulli multi-armed bandit with Thompson Sampling, and the second stage\nlearns the training mixing ratio of these selected auxiliary tasks via a\nGaussian Process based Bayesian optimization framework. We conduct several MTL\nexperiments on the GLUE language understanding tasks, and show that our AutoSeM\nframework can successfully find relevant auxiliary tasks and automatically\nlearn their mixing ratio, achieving significant performance boosts on several\nprimary tasks. Finally, we present ablations for each stage of AutoSeM and\nanalyze the learned auxiliary task choices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04195v1",
            "date": "2019-04-08 17:14:52+00:00",
            "title": "Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout",
            "authors": [
                "Hao Tan",
                "Licheng Yu",
                "Mohit Bansal"
            ],
            "abstract": "A grand goal in AI is to build a robot that can accurately navigate based on\nnatural language instructions, which requires the agent to perceive the scene,\nunderstand and ground language, and act in the real-world environment. One key\nchallenge here is to learn to navigate in new environments that are unseen\nduring training. Most of the existing approaches perform dramatically worse in\nunseen environments as compared to seen ones. In this paper, we present a\ngeneralizable navigational agent. Our agent is trained in two stages. The first\nstage is training via mixed imitation and reinforcement learning, combining the\nbenefits from both off-policy and on-policy optimization. The second stage is\nfine-tuning via newly-introduced 'unseen' triplets (environment, path,\ninstruction). To generate these unseen triplets, we propose a simple but\neffective 'environmental dropout' method to mimic unseen environments, which\novercomes the problem of limited seen environment variability. Next, we apply\nsemi-supervised learning (via back-translation) on these dropped-out\nenvironments to generate new paths and instructions. Empirically, we show that\nour agent is substantially better at generalizability when fine-tuned with\nthese triplets, outperforming the state-of-art approaches by a large margin on\nthe private unseen test set of the Room-to-Room task, and achieving the top\nrank on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04206v1",
            "date": "2019-04-08 17:26:20+00:00",
            "title": "Deep-Sentiment: Sentiment Analysis Using Ensemble of CNN and Bi-LSTM Models",
            "authors": [
                "Shervin Minaee",
                "Elham Azimi",
                "AmirAli Abdolrashidi"
            ],
            "abstract": "With the popularity of social networks, and e-commerce websites, sentiment\nanalysis has become a more active area of research in the past few years. On a\nhigh level, sentiment analysis tries to understand the public opinion about a\nspecific product or topic, or trends from reviews or tweets. Sentiment analysis\nplays an important role in better understanding customer/user opinion, and also\nextracting social/political trends. There has been a lot of previous works for\nsentiment analysis, some based on hand-engineering relevant textual features,\nand others based on different neural network architectures. In this work, we\npresent a model based on an ensemble of long-short-term-memory (LSTM), and\nconvolutional neural network (CNN), one to capture the temporal information of\nthe data, and the other one to extract the local structure thereof. Through\nexperimental results, we show that using this ensemble model we can outperform\nboth individual models. We are also able to achieve a very high accuracy rate\ncompared to the previous works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04294v2",
            "date": "2019-04-08 18:48:45+00:00",
            "title": "Exploring Methods for the Automatic Detection of Errors in Manual Transcription",
            "authors": [
                "Xiaofei Wang",
                "Jinyi Yang",
                "Ruizhi Li",
                "Samik Sadhu",
                "Hynek Hermansky"
            ],
            "abstract": "Quality of data plays an important role in most deep learning tasks. In the\nspeech community, transcription of speech recording is indispensable. Since the\ntranscription is usually generated artificially, automatically finding errors\nin manual transcriptions not only saves time and labors but benefits the\nperformance of tasks that need the training process. Inspired by the success of\nhybrid automatic speech recognition using both language model and acoustic\nmodel, two approaches of automatic error detection in the transcriptions have\nbeen explored in this work. Previous study using a biased language model\napproach, relying on a strong transcription-dependent language model, has been\nreviewed. In this work, we propose a novel acoustic model based approach,\nfocusing on the phonetic sequence of speech. Both methods have been evaluated\non a completely real dataset, which was originally transcribed with errors and\nstrictly corrected manually afterwards.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04364v2",
            "date": "2019-04-08 21:24:31+00:00",
            "title": "Audio Classification of Bit-Representation Waveform",
            "authors": [
                "Masaki Okawa",
                "Takuya Saito",
                "Naoki Sawada",
                "Hiromitsu Nishizaki"
            ],
            "abstract": "This study investigated the waveform representation for audio signal\nclassification. Recently, many studies on audio waveform classification such as\nacoustic event detection and music genre classification have been published.\nMost studies on audio waveform classification have proposed the use of a deep\nlearning (neural network) framework. Generally, a frequency analysis method\nsuch as Fourier transform is applied to extract the frequency or spectral\ninformation from the input audio waveform before inputting the raw audio\nwaveform into the neural network. In contrast to these previous studies, in\nthis paper, we propose a novel waveform representation method, in which audio\nwaveforms are represented as a bit sequence, for audio classification. In our\nexperiment, we compare the proposed bit representation waveform, which is\ndirectly given to a neural network, to other representations of audio waveforms\nsuch as a raw audio waveform and a power spectrum with two classification\ntasks: one is an acoustic event classification task and the other is a\nsound/music classification task. The experimental results showed that the bit\nrepresentation waveform achieved the best classification performance for both\nthe tasks.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.04458v2",
            "date": "2019-04-09 04:09:45+00:00",
            "title": "Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition",
            "authors": [
                "Angli Liu",
                "Jingfei Du",
                "Veselin Stoyanov"
            ],
            "abstract": "Traditional language models are unable to efficiently model entity names\nobserved in text. All but the most popular named entities appear infrequently\nin text providing insufficient context. Recent efforts have recognized that\ncontext can be generalized between entity names that share the same type (e.g.,\n\\emph{person} or \\emph{location}) and have equipped language models with access\nto an external knowledge base (KB). Our Knowledge-Augmented Language Model\n(KALM) continues this line of work by augmenting a traditional model with a KB.\nUnlike previous methods, however, we train with an end-to-end predictive\nobjective optimizing the perplexity of text. We do not require any additional\ninformation such as named entity tags. In addition to improving language\nmodeling performance, KALM learns to recognize named entities in an entirely\nunsupervised way by using entity type information latent in the model. On a\nNamed Entity Recognition (NER) task, KALM achieves performance comparable with\nstate-of-the-art supervised models. Our work demonstrates that named entities\n(and possibly other types of world knowledge) can be modeled successfully using\npredictive learning and training on large corpora of text without any\nadditional information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04764v1",
            "date": "2019-04-09 16:20:52+00:00",
            "title": "Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS",
            "authors": [
                "Haohan Guo",
                "Frank K. Soong",
                "Lei He",
                "Lei Xie"
            ],
            "abstract": "The end-to-end TTS, which can predict speech directly from a given sequence\nof graphemes or phonemes, has shown improved performance over the conventional\nTTS. However, its predicting capability is still limited by the\nacoustic/phonetic coverage of the training data, usually constrained by the\ntraining set size. To further improve the TTS quality in pronunciation, prosody\nand perceived naturalness, we propose to exploit the information embedded in a\nsyntactically parsed tree where the inter-phrase/word information of a sentence\nis organized in a multilevel tree structure. Specifically, two key features:\nphrase structure and relations between adjacent words are investigated.\nExperimental results in subjective listening, measured on three test sets, show\nthat the proposed approach is effective to improve the pronunciation clarity,\nprosody and naturalness of the synthesized speech of the baseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04775v1",
            "date": "2019-04-09 16:37:35+00:00",
            "title": "A New GAN-based End-to-End TTS Training Algorithm",
            "authors": [
                "Haohan Guo",
                "Frank K. Soong",
                "Lei He",
                "Lei Xie"
            ],
            "abstract": "End-to-end, autoregressive model-based TTS has shown significant performance\nimprovements over the conventional one. However, the autoregressive module\ntraining is affected by the exposure bias, or the mismatch between the\ndifferent distributions of real and predicted data. While real data is\navailable in training, but in testing, only predicted data is available to feed\nthe autoregressive module. By introducing both real and generated data\nsequences in training, we can alleviate the effects of the exposure bias. We\npropose to use Generative Adversarial Network (GAN) along with the key idea of\nProfessor Forcing in training. A discriminator in GAN is jointly trained to\nequalize the difference between real and predicted data. In AB subjective\nlistening test, the results show that the new approach is preferred over the\nstandard transfer learning with a CMOS improvement of 0.1. Sentence level\nintelligibility tests show significant improvement in a pathological test set.\nThe GAN-trained new model is also more stable than the baseline to produce\nbetter alignments for the Tacotron output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04969v1",
            "date": "2019-04-10 01:40:08+00:00",
            "title": "BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering",
            "authors": [
                "Yu Cao",
                "Meng Fang",
                "Dacheng Tao"
            ],
            "abstract": "Multi-hop reasoning question answering requires deep comprehension of\nrelationships between various documents and queries. We propose a\nBi-directional Attention Entity Graph Convolutional Network (BAG), leveraging\nrelationships between nodes in an entity graph and attention information\nbetween a query and the entity graph, to solve this task. Graph convolutional\nnetworks are used to obtain a relation-aware representation of nodes for entity\ngraphs built from documents with multi-level features. Bidirectional attention\nis then applied on graphs and queries to generate a query-aware nodes\nrepresentation, which will be used for the final prediction. Experimental\nevaluation shows BAG achieves state-of-the-art accuracy performance on the\nQAngaroo WIKIHOP dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.04995v1",
            "date": "2019-04-10 03:49:37+00:00",
            "title": "AMRec: An Intelligent System for Academic Method Recommendation",
            "authors": [
                "Shanshan Huang",
                "Xiaojun Wan",
                "Xuewei Tang"
            ],
            "abstract": "Finding new academic Methods for research problems is the key task in a\nresearcher's research career. It is usually very difficult for new researchers\nto find good Methods for their research problems since they lack of research\nexperiences. In order to help researchers carry out their researches in a more\nconvenient way, we describe a novel recommendation system called AMRec to\nrecommend new academic Methods for research problems in this paper. Our\nproposed system first extracts academic concepts (Tasks and Methods) and their\nrelations from academic literatures, and then leverages the regularized matrix\nfactorization Method for academic Method recommendation. Preliminary evaluation\nresults verify the effectiveness of our proposed system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.05033v1",
            "date": "2019-04-10 07:44:06+00:00",
            "title": "Better Word Embeddings by Disentangling Contextual n-Gram Information",
            "authors": [
                "Prakhar Gupta",
                "Matteo Pagliardini",
                "Martin Jaggi"
            ],
            "abstract": "Pre-trained word vectors are ubiquitous in Natural Language Processing\napplications. In this paper, we show how training word embeddings jointly with\nbigram and even trigram embeddings, results in improved unigram embeddings. We\nclaim that training word embeddings along with higher n-gram embeddings helps\nin the removal of the contextual information from the unigrams, resulting in\nbetter stand-alone word embeddings. We empirically show the validity of our\nhypothesis by outperforming other competing word representation models by a\nsignificant margin on a wide variety of tasks. We make our models publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05054v2",
            "date": "2019-04-10 08:23:31+00:00",
            "title": "Detecting Cybersecurity Events from Noisy Short Text",
            "authors": [
                "Semih Yagcioglu",
                "Mehmet Saygin Seyfioglu",
                "Begum Citamak",
                "Batuhan Bardak",
                "Seren Guldamlasioglu",
                "Azmi Yuksel",
                "Emin Islam Tatli"
            ],
            "abstract": "It is very critical to analyze messages shared over social networks for cyber\nthreat intelligence and cyber-crime prevention. In this study, we propose a\nmethod that leverages both domain-specific word embeddings and task-specific\nfeatures to detect cyber security events from tweets. Our model employs a\nconvolutional neural network (CNN) and a long short-term memory (LSTM)\nrecurrent neural network which takes word level meta-embeddings as inputs and\nincorporates contextual embeddings to classify noisy short text. We collected a\nnew dataset of cyber security related tweets from Twitter and manually\nannotated a subset of 2K of them. We experimented with this dataset and\nconcluded that the proposed model outperforms both traditional and neural\nbaselines. The results suggest that our method works well for detecting cyber\nsecurity events from noisy short text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05078v1",
            "date": "2019-04-10 09:16:24+00:00",
            "title": "From Semi-supervised to Almost-unsupervised Speech Recognition with Very-low Resource by Jointly Learning Phonetic Structures from Audio and Text Embeddings",
            "authors": [
                "Yi-Chen Chen",
                "Sung-Feng Huang",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "Producing a large amount of annotated speech data for training ASR systems\nremains difficult for more than 95% of languages all over the world which are\nlow-resourced. However, we note human babies start to learn the language by the\nsounds (or phonetic structures) of a small number of exemplar words, and\n\"generalize\" such knowledge to other words without hearing a large amount of\ndata. We initiate some preliminary work in this direction. Audio Word2Vec is\nused to learn the phonetic structures from spoken words (signal segments),\nwhile another autoencoder is used to learn the phonetic structures from text\nwords. The relationships among the above two can be learned jointly, or\nseparately after the above two are well trained. This relationship can be used\nin speech recognition with very low resource. In the initial experiments on the\nTIMIT dataset, only 2.1 hours of speech data (in which 2500 spoken words were\nannotated and the rest unlabeled) gave a word error rate of 44.6%, and this\nnumber can be reduced to 34.2% if 4.1 hr of speech data (in which 20000 spoken\nwords were annotated) were given. These results are not satisfactory, but a\ngood starting point.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05154v3",
            "date": "2019-04-08 03:47:15+00:00",
            "title": "Text-based depression detection on sparse data",
            "authors": [
                "Heinrich Dinkel",
                "Mengyue Wu",
                "Kai Yu"
            ],
            "abstract": "Previous text-based depression detection is commonly based on large\nuser-generated data. Sparse scenarios like clinical conversations are less\ninvestigated. This work proposes a text-based multi-task BGRU network with\npretrained word embeddings to model patients' responses during clinical\ninterviews. Our main approach uses a novel multi-task loss function, aiming at\nmodeling both depression severity and binary health state. We independently\ninvestigate word- and sentence-level word-embeddings as well as the use of\nlarge-data pretraining for depression detection. To strengthen our findings, we\nreport mean-averaged results for a multitude of independent runs on sparse\ndata. First, we show that pretraining is helpful for word-level text-based\ndepression detection. Second, our results demonstrate that sentence-level\nword-embeddings should be mostly preferred over word-level ones. While the\nchoice of pooling function is less crucial, mean and attention pooling should\nbe preferred over last-timestep pooling. Our method outputs depression presence\nresults as well as predicted severity score, culminating a macro F1 score of\n0.84 and MAE of 3.48 on the DAIC-WOZ development set.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.05233v1",
            "date": "2019-04-10 15:10:37+00:00",
            "title": "What's in a Name? Reducing Bias in Bios without Access to Protected Attributes",
            "authors": [
                "Alexey Romanov",
                "Maria De-Arteaga",
                "Hanna Wallach",
                "Jennifer Chayes",
                "Christian Borgs",
                "Alexandra Chouldechova",
                "Sahin Geyik",
                "Krishnaram Kenthapadi",
                "Anna Rumshisky",
                "Adam Tauman Kalai"
            ],
            "abstract": "There is a growing body of work that proposes methods for mitigating bias in\nmachine learning systems. These methods typically rely on access to protected\nattributes such as race, gender, or age. However, this raises two significant\nchallenges: (1) protected attributes may not be available or it may not be\nlegal to use them, and (2) it is often desirable to simultaneously consider\nmultiple protected attributes, as well as their intersections. In the context\nof mitigating bias in occupation classification, we propose a method for\ndiscouraging correlation between the predicted probability of an individual's\ntrue occupation and a word embedding of their name. This method leverages the\nsocietal biases that are encoded in word embeddings, eliminating the need for\naccess to protected attributes. Crucially, it only requires access to\nindividuals' names at training time and not at deployment time. We evaluate two\nvariations of our proposed method using a large-scale dataset of online\nbiographies. We find that both variations simultaneously reduce race and gender\nbiases, with almost no reduction in the classifier's overall true positive\nrate.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.05308v2",
            "date": "2019-04-10 17:18:17+00:00",
            "title": "Deep Neural Networks Ensemble for Detecting Medication Mentions in Tweets",
            "authors": [
                "Davy Weissenbacher",
                "Abeed Sarker",
                "Ari Klein",
                "Karen O'Connor",
                "Arjun Magge Ranganatha",
                "Graciela Gonzalez-Hernandez"
            ],
            "abstract": "Objective: After years of research, Twitter posts are now recognized as an\nimportant source of patient-generated data, providing unique insights into\npopulation health. A fundamental step to incorporating Twitter data in\npharmacoepidemiological research is to automatically recognize medication\nmentions in tweets. Given that lexical searches for medication names may fail\ndue to misspellings or ambiguity with common words, we propose a more advanced\nmethod to recognize them. Methods: We present Kusuri, an Ensemble Learning\nclassifier, able to identify tweets mentioning drug products and dietary\nsupplements. Kusuri (\"medication\" in Japanese) is composed of two modules.\nFirst, four different classifiers (lexicon-based, spelling-variant-based,\npattern-based and one based on a weakly-trained neural network) are applied in\nparallel to discover tweets potentially containing medication names. Second, an\nensemble of deep neural networks encoding morphological, semantical and\nlong-range dependencies of important words in the tweets discovered is used to\nmake the final decision. Results: On a balanced (50-50) corpus of 15,005\ntweets, Kusuri demonstrated performances close to human annotators with 93.7%\nF1-score, the best score achieved thus far on this corpus. On a corpus made of\nall tweets posted by 113 Twitter users (98,959 tweets, with only 0.26%\nmentioning medications), Kusuri obtained 76.3% F1-score. There is not a prior\ndrug extraction system that compares running on such an extremely unbalanced\ndataset. Conclusion: The system identifies tweets mentioning drug names with\nperformance high enough to ensure its usefulness and ready to be integrated in\nlarger natural language processing systems.",
            "journal ref": "Journal of the American Medical Informatics Association, ocz156,\n  2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05426v1",
            "date": "2019-04-10 20:22:31+00:00",
            "title": "A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages",
            "authors": [
                "Ronald Cardenas",
                "Ying Lin",
                "Heng Ji",
                "Jonathan May"
            ],
            "abstract": "Unsupervised part of speech (POS) tagging is often framed as a clustering\nproblem, but practical taggers need to \\textit{ground} their clusters as well.\nGrounding generally requires reference labeled data, a luxury a low-resource\nlanguage might not have. In this work, we describe an approach for low-resource\nunsupervised POS tagging that yields fully grounded output and requires no\nlabeled training data. We find the classic method of Brown et al. (1992)\nclusters well in our use case and employ a decipherment-based approach to\ngrounding. This approach presumes a sequence of cluster IDs is a `ciphertext'\nand seeks a POS tag-to-cluster ID mapping that will reveal the POS sequence. We\nshow intrinsically that, despite the difficulty of the task, we obtain\nreasonable performance across a variety of languages. We also show\nextrinsically that incorporating our POS tagger into a name tagger leads to\nstate-of-the-art tagging performance in Sinhalese and Kinyarwanda, two\nlanguages with nearly no labeled POS data available. We further demonstrate our\ntagger's utility by incorporating it into a true `zero-resource' variant of the\nMalopa (Ammar et al., 2016) dependency parser model that removes the current\nreliance on multilingual resources and gold POS tags for new languages.\nExperiments show that including our tagger makes up much of the accuracy lost\nwhen gold POS tags are unavailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05439v3",
            "date": "2019-04-08 18:30:36+00:00",
            "title": "Event-based Access to Historical Italian War Memoirs",
            "authors": [
                "Marco Rovera",
                "Federico Nanni",
                "Simone Paolo Ponzetto"
            ],
            "abstract": "The progressive digitization of historical archives provides new, often\ndomain specific, textual resources that report on facts and events which have\nhappened in the past; among these, memoirs are a very common type of primary\nsource. In this paper, we present an approach for extracting information from\nItalian historical war memoirs and turning it into structured knowledge. This\nis based on the semantic notions of events, participants and roles. We evaluate\nquantitatively each of the key-steps of our approach and provide a graph-based\nrepresentation of the extracted knowledge, which allows to move between a Close\nand a Distant Reading of the collection.",
            "journal ref": "J. Comput. Cult. Herit. 14, 1, Article 2 (February 2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05506v2",
            "date": "2019-04-11 02:53:21+00:00",
            "title": "Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?",
            "authors": [
                "Sorami Hisamoto",
                "Matt Post",
                "Kevin Duh"
            ],
            "abstract": "Data privacy is an important issue for \"machine learning as a service\"\nproviders. We focus on the problem of membership inference attacks: given a\ndata sample and black-box access to a model's API, determine whether the sample\nexisted in the model's training data. Our contribution is an investigation of\nthis problem in the context of sequence-to-sequence models, which are important\nin applications such as machine translation and video captioning. We define the\nmembership inference problem for sequence generation, provide an open dataset\nbased on state-of-the-art machine translation models, and report initial\nresults on whether these models leak private information against several kinds\nof membership inference attacks.",
            "journal ref": "Tansactions of the Association for Computational Linguistics\n  (TACL) Volume 8, 2020 p.49-63",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.05521v2",
            "date": "2019-04-11 04:04:06+00:00",
            "title": "UniVSE: Robust Visual Semantic Embeddings via Structured Semantic Representations",
            "authors": [
                "Hao Wu",
                "Jiayuan Mao",
                "Yufeng Zhang",
                "Yuning Jiang",
                "Lei Li",
                "Weiwei Sun",
                "Wei-Ying Ma"
            ],
            "abstract": "We propose Unified Visual-Semantic Embeddings (UniVSE) for learning a joint\nspace of visual and textual concepts. The space unifies the concepts at\ndifferent levels, including objects, attributes, relations, and full scenes. A\ncontrastive learning approach is proposed for the fine-grained alignment from\nonly image-caption pairs. Moreover, we present an effective approach for\nenforcing the coverage of semantic components that appear in the sentence. We\ndemonstrate the robustness of Unified VSE in defending text-domain adversarial\nattacks on cross-modal retrieval tasks. Such robustness also empowers the use\nof visual cues to resolve word dependencies in novel sentences.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.05530v4",
            "date": "2019-04-11 04:45:42+00:00",
            "title": "Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs",
            "authors": [
                "Woojeong Jin",
                "Meng Qu",
                "Xisen Jin",
                "Xiang Ren"
            ],
            "abstract": "Knowledge graph reasoning is a critical task in natural language processing.\nThe task becomes more challenging on temporal knowledge graphs, where each fact\nis associated with a timestamp. Most existing methods focus on reasoning at\npast timestamps and they are not able to predict facts happening in the future.\nThis paper proposes Recurrent Event Network (RE-NET), a novel autoregressive\narchitecture for predicting future interactions. The occurrence of a fact\n(event) is modeled as a probability distribution conditioned on temporal\nsequences of past knowledge graphs. Specifically, our RE-NET employs a\nrecurrent event encoder to encode past facts and uses a neighborhood aggregator\nto model the connection of facts at the same timestamp. Future facts can then\nbe inferred in a sequential manner based on the two modules. We evaluate our\nproposed method via link prediction at future times on five public datasets.\nThrough extensive experiments, we demonstrate the strength of RENET, especially\non multi-step inference over future timestamps, and achieve state-of-the-art\nperformance on all five datasets. Code and data can be found at\nhttps://github.com/INK-USC/RE-Net.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.05544v2",
            "date": "2019-04-11 06:16:17+00:00",
            "title": "FrameRank: A Text Processing Approach to Video Summarization",
            "authors": [
                "Zhuo Lei",
                "Chao Zhang",
                "Qian Zhang",
                "Guoping Qiu"
            ],
            "abstract": "Video summarization has been extensively studied in the past decades.\nHowever, user-generated video summarization is much less explored since there\nlack large-scale video datasets within which human-generated video summaries\nare unambiguously defined and annotated. Toward this end, we propose a\nuser-generated video summarization dataset - UGSum52 - that consists of 52\nvideos (207 minutes). In constructing the dataset, because of the subjectivity\nof user-generated video summarization, we manually annotate 25 summaries for\neach video, which are in total 1300 summaries. To the best of our knowledge, it\nis currently the largest dataset for user-generated video summarization.\n  Based on this dataset, we present FrameRank, an unsupervised video\nsummarization method that employs a frame-to-frame level affinity graph to\nidentify coherent and informative frames to summarize a video. We use the\nKullback-Leibler(KL)-divergence-based graph to rank temporal segments according\nto the amount of semantic information contained in their frames. We illustrate\nthe effectiveness of our method by applying it to three datasets SumMe, TVSum\nand UGSum52 and show it achieves state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05548v2",
            "date": "2019-04-11 06:46:15+00:00",
            "title": "Reasoning Visual Dialogs with Structural and Partial Observations",
            "authors": [
                "Zilong Zheng",
                "Wenguan Wang",
                "Siyuan Qi",
                "Song-Chun Zhu"
            ],
            "abstract": "We propose a novel model to address the task of Visual Dialog which exhibits\ncomplex dialog structures. To obtain a reasonable answer based on the current\nquestion and the dialog history, the underlying semantic dependencies between\ndialog entities are essential. In this paper, we explicitly formalize this task\nas inference in a graphical model with partially observed nodes and unknown\ngraph structures (relations in dialog). The given dialog entities are viewed as\nthe observed nodes. The answer to a given question is represented by a node\nwith missing value. We first introduce an Expectation Maximization algorithm to\ninfer both the underlying dialog structures and the missing node values\n(desired answers). Based on this, we proceed to propose a differentiable graph\nneural network (GNN) solution that approximates this process. Experiment\nresults on the VisDial and VisDial-Q datasets show that our model outperforms\ncomparative methods. It is also observed that our method can infer the\nunderlying dialog structure for better dialog reasoning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.05606v1",
            "date": "2019-04-11 09:55:41+00:00",
            "title": "Multi-lingual Dialogue Act Recognition with Deep Learning Methods",
            "authors": [
                "Ji\u0159\u00ed Mart\u00ednek",
                "Pavel Kr\u00e1l",
                "Ladislav Lenc",
                "Christophe Cerisara"
            ],
            "abstract": "This paper deals with multi-lingual dialogue act (DA) recognition. The\nproposed approaches are based on deep neural networks and use word2vec\nembeddings for word representation. Two multi-lingual models are proposed for\nthis task. The first approach uses one general model trained on the embeddings\nfrom all available languages. The second method trains the model on a single\npivot language and a linear transformation method is used to project other\nlanguages onto the pivot language. The popular convolutional neural network and\nLSTM architectures with different set-ups are used as classifiers. To the best\nof our knowledge this is the first attempt at multi-lingual DA recognition\nusing neural networks. The multi-lingual models are validated experimentally on\ntwo languages from the Verbmobil corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05873v1",
            "date": "2019-04-11 17:58:37+00:00",
            "title": "An Empirical Study of Spatial Attention Mechanisms in Deep Networks",
            "authors": [
                "Xizhou Zhu",
                "Dazhi Cheng",
                "Zheng Zhang",
                "Stephen Lin",
                "Jifeng Dai"
            ],
            "abstract": "Attention mechanisms have become a popular component in deep neural networks,\nyet there has been little examination of how different influencing factors and\nmethods for computing attention from these factors affect performance. Toward a\nbetter general understanding of attention mechanisms, we present an empirical\nstudy that ablates various spatial attention elements within a generalized\nattention formulation, encompassing the dominant Transformer attention as well\nas the prevalent deformable convolution and dynamic convolution modules.\nConducted on a variety of applications, the study yields significant findings\nabout spatial attention in deep networks, some of which run counter to\nconventional understanding. For example, we find that the query and key content\ncomparison in Transformer attention is negligible for self-attention, but vital\nfor encoder-decoder attention. A proper combination of deformable convolution\nwith key content only saliency achieves the best accuracy-efficiency tradeoff\nin self-attention. Our results suggest that there exists much room for\nimprovement in the design of attention mechanisms.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.06022v1",
            "date": "2019-04-12 03:22:13+00:00",
            "title": "Multimodal Speech Emotion Recognition and Ambiguity Resolution",
            "authors": [
                "Gaurav Sahu"
            ],
            "abstract": "Identifying emotion from speech is a non-trivial task pertaining to the\nambiguous definition of emotion itself. In this work, we adopt a\nfeature-engineering based approach to tackle the task of speech emotion\nrecognition. Formalizing our problem as a multi-class classification problem,\nwe compare the performance of two categories of models. For both, we extract\neight hand-crafted features from the audio signal. In the first approach, the\nextracted features are used to train six traditional machine learning\nclassifiers, whereas the second approach is based on deep learning wherein a\nbaseline feed-forward neural network and an LSTM-based classifier are trained\nover the same features. In order to resolve ambiguity in communication, we also\ninclude features from the text domain. We report accuracy, f-score, precision,\nand recall for the different experiment settings we evaluated our models in.\nOverall, we show that lighter machine learning based models trained over a few\nhand-crafted features are able to achieve performance comparable to the current\ndeep learning based state-of-the-art method for emotion recognition.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.06037v2",
            "date": "2019-04-12 05:15:31+00:00",
            "title": "Direct speech-to-speech translation with a sequence-to-sequence model",
            "authors": [
                "Ye Jia",
                "Ron J. Weiss",
                "Fadi Biadsy",
                "Wolfgang Macherey",
                "Melvin Johnson",
                "Zhifeng Chen",
                "Yonghui Wu"
            ],
            "abstract": "We present an attention-based sequence-to-sequence neural network which can\ndirectly translate speech from one language into speech in another language,\nwithout relying on an intermediate text representation. The network is trained\nend-to-end, learning to map speech spectrograms into target spectrograms in\nanother language, corresponding to the translated content (in a different\ncanonical voice). We further demonstrate the ability to synthesize translated\nspeech using the voice of the source speaker. We conduct experiments on two\nSpanish-to-English speech translation datasets, and find that the proposed\nmodel slightly underperforms a baseline cascade of a direct speech-to-text\ntranslation model and a text-to-speech synthesis model, demonstrating the\nfeasibility of the approach on this very challenging task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06063v2",
            "date": "2019-04-12 06:54:34+00:00",
            "title": "Building a mixed-lingual neural TTS system with only monolingual data",
            "authors": [
                "Liumeng Xue",
                "Wei Song",
                "Guanghui Xu",
                "Lei Xie",
                "Zhizheng Wu"
            ],
            "abstract": "When deploying a Chinese neural text-to-speech (TTS) synthesis system, one of\nthe challenges is to synthesize Chinese utterances with English phrases or\nwords embedded. This paper looks into the problem in the encoder-decoder\nframework when only monolingual data from a target speaker is available.\nSpecifically, we view the problem from two aspects: speaker consistency within\nan utterance and naturalness. We start the investigation with an Average Voice\nModel which is built from multi-speaker monolingual data, i.e. Mandarin and\nEnglish data. On the basis of that, we look into speaker embedding for speaker\nconsistency within an utterance and phoneme embedding for naturalness and\nintelligibility and study the choice of data for model training. We report the\nfindings and discuss the challenges to build a mixed-lingual TTS system with\nonly monolingual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06086v1",
            "date": "2019-04-12 07:55:53+00:00",
            "title": "Unsupervised Speech Domain Adaptation Based on Disentangled Representation Learning for Robust Speech Recognition",
            "authors": [
                "Jong-Hyeon Park",
                "Myungwoo Oh",
                "Hyung-Min Park"
            ],
            "abstract": "In general, the performance of automatic speech recognition (ASR) systems is\nsignificantly degraded due to the mismatch between training and test\nenvironments. Recently, a deep-learning-based image-to-image translation\ntechnique to translate an image from a source domain to a desired domain was\npresented, and cycle-consistent adversarial network (CycleGAN) was applied to\nlearn a mapping for speech-to-speech conversion from a speaker to a target\nspeaker. However, this method might not be adequate to remove corrupting noise\ncomponents for robust ASR because it was designed to convert speech itself. In\nthis paper, we propose a domain adaptation method based on generative\nadversarial nets (GANs) with disentangled representation learning to achieve\nrobustness in ASR systems. In particular, two separated encoders, context and\ndomain encoders, are introduced to learn distinct latent variables. The latent\nvariables allow us to convert the domain of speech according to its context and\ndomain representation. We improved word accuracies by 6.55~15.70\\% for the\nCHiME4 challenge corpus by applying a noisy-to-clean environment adaptation for\nrobust ASR. In addition, similar to the method based on the CycleGAN, this\nmethod can be used for gender adaptation in gender-mismatched recognition.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.06093v1",
            "date": "2019-04-12 08:23:26+00:00",
            "title": "STC Speaker Recognition Systems for the VOiCES From a Distance Challenge",
            "authors": [
                "Sergey Novoselov",
                "Aleksei Gusev",
                "Artem Ivanov",
                "Timur Pekhovsky",
                "Andrey Shulipa",
                "Galina Lavrentyeva",
                "Vladimir Volokhov",
                "Alexandr Kozlov"
            ],
            "abstract": "This paper presents the Speech Technology Center (STC) speaker recognition\n(SR) systems submitted to the VOiCES From a Distance challenge 2019. The\nchallenge's SR task is focused on the problem of speaker recognition in single\nchannel distant/far-field audio under noisy conditions. In this work we\ninvestigate different deep neural networks architectures for speaker embedding\nextraction to solve the task. We show that deep networks with residual frame\nlevel connections outperform more shallow architectures. Simple energy based\nspeech activity detector (SAD) and automatic speech recognition (ASR) based SAD\nare investigated in this work. We also address the problem of data preparation\nfor robust embedding extractors training. The reverberation for the data\naugmentation was performed using automatic room impulse response generator. In\nour systems we used discriminatively trained cosine similarity metric learning\nmodel as embedding backend. Scores normalization procedure was applied for each\nindividual subsystem we used. Our final submitted systems were based on the\nfusion of different subsystems. The results obtained on the VOiCES development\nand evaluation sets demonstrate effectiveness and robustness of the proposed\nsystems when dealing with distant/far-field audio under noisy conditions.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.06100v1",
            "date": "2019-04-12 08:45:43+00:00",
            "title": "Adapting Sequence to Sequence models for Text Normalization in Social Media",
            "authors": [
                "Ismini Lourentzou",
                "Kabir Manghnani",
                "ChengXiang Zhai"
            ],
            "abstract": "Social media offer an abundant source of valuable raw data, however informal\nwriting can quickly become a bottleneck for many natural language processing\n(NLP) tasks. Off-the-shelf tools are usually trained on formal text and cannot\nexplicitly handle noise found in short online posts. Moreover, the variety of\nfrequently occurring linguistic variations presents several challenges, even\nfor humans who might not be able to comprehend the meaning of such posts,\nespecially when they contain slang and abbreviations. Text Normalization aims\nto transform online user-generated text to a canonical form. Current text\nnormalization systems rely on string or phonetic similarity and classification\nmodels that work on a local fashion. We argue that processing contextual\ninformation is crucial for this task and introduce a social media text\nnormalization hybrid word-character attention-based encoder-decoder model that\ncan serve as a pre-processing step for NLP applications to adapt to noisy text\nin social media. Our character-based component is trained on synthetic\nadversarial examples that are designed to capture errors commonly found in\nonline user-generated text. Experiments show that our model surpasses neural\narchitectures designed for text normalization and achieves comparable\nperformance with state-of-the-art related work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06478v1",
            "date": "2019-04-13 04:24:12+00:00",
            "title": "Low-Latency Speaker-Independent Continuous Speech Separation",
            "authors": [
                "Takuya Yoshioka",
                "Zhuo Chen",
                "Changliang Liu",
                "Xiong Xiao",
                "Hakan Erdogan",
                "Dimitrios Dimitriadis"
            ],
            "abstract": "Speaker independent continuous speech separation (SI-CSS) is a task of\nconverting a continuous audio stream, which may contain overlapping voices of\nunknown speakers, into a fixed number of continuous signals each of which\ncontains no overlapping speech segment. A separated, or cleaned, version of\neach utterance is generated from one of SI-CSS's output channels\nnondeterministically without being split up and distributed to multiple\nchannels. A typical application scenario is transcribing multi-party\nconversations, such as meetings, recorded with microphone arrays. The output\nsignals can be simply sent to a speech recognition engine because they do not\ninclude speech overlaps. The previous SI-CSS method uses a neural network\ntrained with permutation invariant training and a data-driven beamformer and\nthus requires much processing latency. This paper proposes a low-latency SI-CSS\nmethod whose performance is comparable to that of the previous method in a\nmicrophone array-based meeting transcription task.This is achieved (1) by using\na new speech separation network architecture combined with a double buffering\nscheme and (2) by performing enhancement with a set of fixed beamformers\nfollowed by a neural post-filter.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.06508v2",
            "date": "2019-04-13 08:51:11+00:00",
            "title": "End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learning",
            "authors": [
                "Tao Tu",
                "Yuan-Jui Chen",
                "Cheng-chieh Yeh",
                "Hung-yi Lee"
            ],
            "abstract": "End-to-end text-to-speech (TTS) has shown great success on large quantities\nof paired text plus speech data. However, laborious data collection remains\ndifficult for at least 95% of the languages over the world, which hinders the\ndevelopment of TTS in different languages. In this paper, we aim to build TTS\nsystems for such low-resource (target) languages where only very limited paired\ndata are available. We show such TTS can be effectively constructed by\ntransferring knowledge from a high-resource (source) language. Since the model\ntrained on source language cannot be directly applied to target language due to\ninput space mismatch, we propose a method to learn a mapping between source and\ntarget linguistic symbols. Benefiting from this learned mapping, pronunciation\ninformation can be preserved throughout the transferring procedure. Preliminary\nexperiments show that we only need around 15 minutes of paired data to obtain a\nrelatively good TTS system. Furthermore, analytic studies demonstrated that the\nautomatically discovered mapping correlate well with the phonetic expertise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.06618v1",
            "date": "2019-04-14 03:15:38+00:00",
            "title": "UR-FUNNY: A Multimodal Language Dataset for Understanding Humor",
            "authors": [
                "Md Kamrul Hasan",
                "Wasifur Rahman",
                "Amir Zadeh",
                "Jianyuan Zhong",
                "Md Iftekhar Tanveer",
                "Louis-Philippe Morency",
                "Mohammed",
                "Hoque"
            ],
            "abstract": "Humor is a unique and creative communicative behavior displayed during social\ninteractions. It is produced in a multimodal manner, through the usage of words\n(text), gestures (vision) and prosodic cues (acoustic). Understanding humor\nfrom these three modalities falls within boundaries of multimodal language; a\nrecent research trend in natural language processing that models natural\nlanguage as it happens in face-to-face communication. Although humor detection\nis an established research area in NLP, in a multimodal context it is an\nunderstudied area. This paper presents a diverse multimodal dataset, called\nUR-FUNNY, to open the door to understanding multimodal language used in\nexpressing humor. The dataset and accompanying studies, present a framework in\nmultimodal humor detection for the natural language processing community.\nUR-FUNNY is publicly available for research.",
            "journal ref": "EMNLP-IJCNLP, 2019, 2046-2056",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.06834v1",
            "date": "2019-04-15 04:17:13+00:00",
            "title": "An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search",
            "authors": [
                "Kartik Goyal",
                "Chris Dyer",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Globally normalized neural sequence models are considered superior to their\nlocally normalized equivalents because they may ameliorate the effects of label\nbias. However, when considering high-capacity neural parametrizations that\ncondition on the whole input sequence, both model classes are theoretically\nequivalent in terms of the distributions they are capable of representing.\nThus, the practical advantage of global normalization in the context of modern\nneural methods remains unclear. In this paper, we attempt to shed light on this\nproblem through an empirical study. We extend an approach for search-aware\ntraining via a continuous relaxation of beam search (Goyal et al., 2017b) in\norder to enable training of globally normalized recurrent sequence models\nthrough simple backpropagation. We then use this technique to conduct an\nempirical study of the interaction between global normalization, high-capacity\nencoders, and search-aware optimization. We observe that in the context of\ninexact search, globally normalized neural models are still more effective than\ntheir locally normalized counterparts. Further, since our training approach is\nsensitive to warm-starting with pre-trained models, we also propose a novel\ninitialization strategy based on self-normalization for pre-training globally\nnormalized models. We perform analysis of our approach on two tasks: CCG\nsupertagging and Machine Translation, and demonstrate the importance of global\nnormalization under different conditions while using search-aware training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.06861v1",
            "date": "2019-04-15 05:47:23+00:00",
            "title": "Self-critical n-step Training for Image Captioning",
            "authors": [
                "Junlong Gao",
                "Shiqi Wang",
                "Shanshe Wang",
                "Siwei Ma",
                "Wen Gao"
            ],
            "abstract": "Existing methods for image captioning are usually trained by cross entropy\nloss, which leads to exposure bias and the inconsistency between the optimizing\nfunction and evaluation metrics. Recently it has been shown that these two\nissues can be addressed by incorporating techniques from reinforcement\nlearning, where one of the popular techniques is the advantage actor-critic\nalgorithm that calculates per-token advantage by estimating state value with a\nparametrized estimator at the cost of introducing estimation bias. In this\npaper, we estimate state value without using a parametrized value estimator.\nWith the properties of image captioning, namely, the deterministic state\ntransition function and the sparse reward, state value is equivalent to its\npreceding state-action value, and we reformulate advantage function by simply\nreplacing the former with the latter. Moreover, the reformulated advantage is\nextended to n-step, which can generally increase the absolute value of the mean\nof reformulated advantage while lowering variance. Then two kinds of rollout\nare adopted to estimate state-action value, which we call self-critical n-step\ntraining. Empirically we find that our method can obtain better performance\ncompared to the state-of-the-art methods that use the sequence level advantage\nand parametrized estimator respectively on the widely used MSCOCO benchmark.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.07078v1",
            "date": "2019-04-15 14:35:40+00:00",
            "title": "Semantic query-by-example speech search using visual grounding",
            "authors": [
                "Herman Kamper",
                "Aristotelis Anastassiou",
                "Karen Livescu"
            ],
            "abstract": "A number of recent studies have started to investigate how speech systems can\nbe trained on untranscribed speech by leveraging accompanying images at\ntraining time. Examples of tasks include keyword prediction and within- and\nacross-mode retrieval. Here we consider how such models can be used for\nquery-by-example (QbE) search, the task of retrieving utterances relevant to a\ngiven spoken query. We are particularly interested in semantic QbE, where the\ntask is not only to retrieve utterances containing exact instances of the\nquery, but also utterances whose meaning is relevant to the query. We follow a\nsegmental QbE approach where variable-duration speech segments (queries, search\nutterances) are mapped to fixed-dimensional embedding vectors. We show that a\nQbE system using an embedding function trained on visually grounded speech data\noutperforms a purely acoustic QbE system in terms of both exact and semantic\nretrieval performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07307v1",
            "date": "2019-04-15 19:49:06+00:00",
            "title": "Tracing Forum Posts to MOOC Content using Topic Analysis",
            "authors": [
                "Alexander William Wong",
                "Ken Wong",
                "Abram Hindle"
            ],
            "abstract": "Massive Open Online Courses are educational programs that are open and\naccessible to a large number of people through the internet. To facilitate\nlearning, MOOC discussion forums exist where students and instructors\ncommunicate questions, answers, and thoughts related to the course.\n  The primary objective of this paper is to investigate tracing discussion\nforum posts back to course lecture videos and readings using topic analysis. We\nutilize both unsupervised and supervised variants of Latent Dirichlet\nAllocation (LDA) to extract topics from course material and classify forum\nposts. We validate our approach on posts bootstrapped from five Coursera\ncourses and determine that topic models can be used to map student discussion\nposts back to the underlying course lecture or reading. Labeled LDA outperforms\nunsupervised Hierarchical Dirichlet Process LDA and base LDA for our\ntraceability task. This research is useful as it provides an automated approach\nfor clustering student discussions by course material, enabling instructors to\nquickly evaluate student misunderstanding of content and clarify materials\naccordingly.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.07372v1",
            "date": "2019-04-15 23:56:25+00:00",
            "title": "Something's Brewing! Early Prediction of Controversy-causing Posts from Discussion Features",
            "authors": [
                "Jack Hessel",
                "Lillian Lee"
            ],
            "abstract": "Controversial posts are those that split the preferences of a community,\nreceiving both significant positive and significant negative feedback. Our\ninclusion of the word \"community\" here is deliberate: what is controversial to\nsome audiences may not be so to others. Using data from several different\ncommunities on reddit.com, we predict the ultimate controversiality of posts,\nleveraging features drawn from both the textual content and the tree structure\nof the early comments that initiate the discussion. We find that even when only\na handful of comments are available, e.g., the first 5 comments made within 15\nminutes of the original post, discussion features often add predictive capacity\nto strong content-and-rate only baselines. Additional experiments on domain\ntransfer suggest that conversation-structure features often generalize to other\ncommunities better than conversation-content features do.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07386v1",
            "date": "2019-04-16 00:55:20+00:00",
            "title": "I4U Submission to NIST SRE 2018: Leveraging from a Decade of Shared Experiences",
            "authors": [
                "Kong Aik Lee",
                "Ville Hautamaki",
                "Tomi Kinnunen",
                "Hitoshi Yamamoto",
                "Koji Okabe",
                "Ville Vestman",
                "Jing Huang",
                "Guohong Ding",
                "Hanwu Sun",
                "Anthony Larcher",
                "Rohan Kumar Das",
                "Haizhou Li",
                "Mickael Rouvier",
                "Pierre-Michel Bousquet",
                "Wei Rao",
                "Qing Wang",
                "Chunlei Zhang",
                "Fahimeh Bahmaninezhad",
                "Hector Delgado",
                "Jose Patino",
                "Qiongqiong Wang",
                "Ling Guo",
                "Takafumi Koshinaka",
                "Jiacen Zhang",
                "Koichi Shinoda",
                "Trung Ngo Trong",
                "Md Sahidullah",
                "Fan Lu",
                "Yun Tang",
                "Ming Tu",
                "Kah Kuan Teh",
                "Huy Dat Tran",
                "Kuruvachan K. George",
                "Ivan Kukanov",
                "Florent Desnous",
                "Jichen Yang",
                "Emre Yilmaz",
                "Longting Xu",
                "Jean-Francois Bonastre",
                "Chenglin Xu",
                "Zhi Hao Lim",
                "Eng Siong Chng",
                "Shivesh Ranjan",
                "John H. L. Hansen",
                "Massimiliano Todisco",
                "Nicholas Evans"
            ],
            "abstract": "The I4U consortium was established to facilitate a joint entry to NIST\nspeaker recognition evaluations (SRE). The latest edition of such joint\nsubmission was in SRE 2018, in which the I4U submission was among the\nbest-performing systems. SRE'18 also marks the 10-year anniversary of I4U\nconsortium into NIST SRE series of evaluation. The primary objective of the\ncurrent paper is to summarize the results and lessons learned based on the\ntwelve sub-systems and their fusion submitted to SRE'18. It is also our\nintention to present a shared view on the advancements, progresses, and major\nparadigm shifts that we have witnessed as an SRE participant in the past decade\nfrom SRE'08 to SRE'18. In this regard, we have seen, among others, a paradigm\nshift from supervector representation to deep speaker embedding, and a switch\nof research challenge from channel compensation to domain adaptation.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.07556v2",
            "date": "2019-04-16 09:38:01+00:00",
            "title": "Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks",
            "authors": [
                "Ryan Eloff",
                "Andr\u00e9 Nortje",
                "Benjamin van Niekerk",
                "Avashna Govender",
                "Leanne Nortje",
                "Arnu Pretorius",
                "Elan van Biljon",
                "Ewald van der Westhuizen",
                "Lisa van Staden",
                "Herman Kamper"
            ],
            "abstract": "For our submission to the ZeroSpeech 2019 challenge, we apply discrete\nlatent-variable neural networks to unlabelled speech and use the discovered\nunits for speech synthesis. Unsupervised discrete subword modelling could be\nuseful for studies of phonetic category learning in infants or in low-resource\nspeech technology requiring symbolic input. We use an autoencoder (AE)\narchitecture with intermediate discretisation. We decouple acoustic unit\ndiscovery from speaker modelling by conditioning the AE's decoder on the\ntraining speaker identity. At test time, unit discovery is performed on speech\nfrom an unseen speaker, followed by unit decoding conditioned on a known target\nspeaker to obtain reconstructed filterbanks. This output is fed to a neural\nvocoder to synthesise speech in the target speaker's voice. For discretisation,\ncategorical variational autoencoders (CatVAEs), vector-quantised VAEs (VQ-VAEs)\nand straight-through estimation are compared at different compression levels on\ntwo languages. Our final model uses convolutional encoding, VQ-VAE\ndiscretisation, deconvolutional decoding and an FFTNet vocoder. We show that\ndecoupled speaker conditioning intrinsically improves discrete acoustic\nrepresentations, yielding competitive synthesis quality compared to the\nchallenge baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.07629v6",
            "date": "2019-04-16 12:54:00+00:00",
            "title": "Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddings",
            "authors": [
                "Zhaoning Li",
                "Qi Li",
                "Xiaotian Zou",
                "Jiangtao Ren"
            ],
            "abstract": "Causality extraction from natural language texts is a challenging open\nproblem in artificial intelligence. Existing methods utilize patterns,\nconstraints, and machine learning techniques to extract causality, heavily\ndepending on domain knowledge and requiring considerable human effort and time\nfor feature engineering. In this paper, we formulate causality extraction as a\nsequence labeling problem based on a novel causality tagging scheme. On this\nbasis, we propose a neural causality extractor with the BiLSTM-CRF model as the\nbackbone, named SCITE (Self-attentive BiLSTM-CRF wIth Transferred Embeddings),\nwhich can directly extract cause and effect without extracting candidate causal\npairs and identifying their relations separately. To address the problem of\ndata insufficiency, we transfer contextual string embeddings, also known as\nFlair embeddings, which are trained on a large corpus in our task. In addition,\nto improve the performance of causality extraction, we introduce a multihead\nself-attention mechanism into SCITE to learn the dependencies between causal\nwords. We evaluate our method on a public dataset, and experimental results\ndemonstrate that our method achieves significant and consistent improvement\ncompared to baselines.",
            "journal ref": "Neurocomputing, Volume 423, 2021, Pages 207-219",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08138v5",
            "date": "2019-04-17 08:46:53+00:00",
            "title": "Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis",
            "authors": [
                "Feiyang Chen",
                "Ziqian Luo",
                "Yanyan Xu",
                "Dengfeng Ke"
            ],
            "abstract": "Sentiment analysis, mostly based on text, has been rapidly developing in the\nlast decade and has attracted widespread attention in both academia and\nindustry. However, the information in the real world usually comes from\nmultiple modalities, such as audio and text. Therefore, in this paper, based on\naudio and text, we consider the task of multimodal sentiment analysis and\npropose a novel fusion strategy including both multi-feature fusion and\nmulti-modality fusion to improve the accuracy of audio-text sentiment analysis.\nWe call it the DFF-ATMF (Deep Feature Fusion - Audio and Text Modality Fusion)\nmodel, which consists of two parallel branches, the audio modality based branch\nand the text modality based branch. Its core mechanisms are the fusion of\nmultiple feature vectors and multiple modality attention. Experiments on the\nCMU-MOSI dataset and the recently released CMU-MOSEI dataset, both collected\nfrom YouTube for sentiment analysis, show the very competitive results of our\nDFF-ATMF model. Furthermore, by virtue of attention weight distribution\nheatmaps, we also demonstrate the deep features learned by using DFF-ATMF are\ncomplementary to each other and robust. Surprisingly, DFF-ATMF also achieves\nnew state-of-the-art results on the IEMOCAP dataset, indicating that the\nproposed fusion strategy also has a good generalization ability for multimodal\nemotion recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08248v2",
            "date": "2019-04-16 14:43:19+00:00",
            "title": "An Analysis of Speech Enhancement and Recognition Losses in Limited Resources Multi-talker Single Channel Audio-Visual ASR",
            "authors": [
                "Luca Pasa",
                "Giovanni Morrone",
                "Leonardo Badino"
            ],
            "abstract": "In this paper, we analyzed how audio-visual speech enhancement can help to\nperform the ASR task in a cocktail party scenario. Therefore we considered two\nsimple end-to-end LSTM-based models that perform single-channel audio-visual\nspeech enhancement and phone recognition respectively. Then, we studied how the\ntwo models interact, and how to train them jointly affects the final result. We\nanalyzed different training strategies that reveal some interesting and\nunexpected behaviors. The experiments show that during optimization of the ASR\ntask the speech enhancement capability of the model significantly decreases and\nvice-versa. Nevertheless the joint optimization of the two tasks shows a\nremarkable drop of the Phone Error Rate (PER) compared to the audio-visual\nbaseline models trained only to perform phone recognition. We analyzed the\nbehaviors of the proposed models by using two limited-size datasets, and in\nparticular we used the mixed-speech versions of GRID and TCD-TIMIT.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.08386v1",
            "date": "2019-04-17 17:37:33+00:00",
            "title": "Casting Light on Invisible Cities: Computationally Engaging with Literary Criticism",
            "authors": [
                "Shufan Wang",
                "Mohit Iyyer"
            ],
            "abstract": "Literary critics often attempt to uncover meaning in a single work of\nliterature through careful reading and analysis. Applying natural language\nprocessing methods to aid in such literary analyses remains a challenge in\ndigital humanities. While most previous work focuses on \"distant reading\" by\nalgorithmically discovering high-level patterns from large collections of\nliterary works, here we sharpen the focus of our methods to a single literary\ntheory about Italo Calvino's postmodern novel Invisible Cities, which consists\nof 55 short descriptions of imaginary cities. Calvino has provided a\nclassification of these cities into eleven thematic groups, but literary\nscholars disagree as to how trustworthy his categorization is. Due to the\nunique structure of this novel, we can computationally weigh in on this debate:\nwe leverage pretrained contextualized representations to embed each city's\ndescription and use unsupervised methods to cluster these embeddings.\nAdditionally, we compare results of our computational approach to similarity\njudgments generated by human readers. Our work is a first step towards\nincorporating natural language processing into literary criticism.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.08709v1",
            "date": "2019-04-18 11:50:20+00:00",
            "title": "Knowledge-rich Image Gist Understanding Beyond Literal Meaning",
            "authors": [
                "Lydia Weiland",
                "Ioana Hulpus",
                "Simone Paolo Ponzetto",
                "Wolfgang Effelsberg",
                "Laura Dietz"
            ],
            "abstract": "We investigate the problem of understanding the message (gist) conveyed by\nimages and their captions as found, for instance, on websites or news articles.\nTo this end, we propose a methodology to capture the meaning of image-caption\npairs on the basis of large amounts of machine-readable knowledge that has\npreviously been shown to be highly effective for text understanding. Our method\nidentifies the connotation of objects beyond their denotation: where most\napproaches to image understanding focus on the denotation of objects, i.e.,\ntheir literal meaning, our work addresses the identification of connotations,\ni.e., iconic meanings of objects, to understand the message of images. We view\nimage understanding as the task of representing an image-caption pair on the\nbasis of a wide-coverage vocabulary of concepts such as the one provided by\nWikipedia, and cast gist detection as a concept-ranking problem with\nimage-caption pairs as queries. To enable a thorough investigation of the\nproblem of gist understanding, we produce a gold standard of over 300\nimage-caption pairs and over 8,000 gist annotations covering a wide variety of\ntopics at different levels of abstraction. We use this dataset to\nexperimentally benchmark the contribution of signals from heterogeneous\nsources, namely image and text. The best result with a Mean Average Precision\n(MAP) of 0.69 indicate that by combining both dimensions we are able to better\nunderstand the meaning of our image-caption pairs than when using language or\nvision information alone. We test the robustness of our gist detection approach\nwhen receiving automatically generated input, i.e., using automatically\ngenerated image tags or generated captions, and prove the feasibility of an\nend-to-end automated process.",
            "journal ref": "Data & Knowledge Engineering, Volume 117, September 2018, Pages\n  114-132",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.08721v1",
            "date": "2019-04-18 12:19:09+00:00",
            "title": "Societal Controversies in Wikipedia Articles",
            "authors": [
                "Erik Borra",
                "Andreas Kaltenbrunner",
                "Michele Mauri",
                "Esther Weltevrede",
                "David Laniado",
                "Richard Rogers",
                "Paolo Ciuccarelli",
                "Giovanni Magni",
                "Tommaso Venturini"
            ],
            "abstract": "Collaborative content creation inevitably reaches situations where different\npoints of view lead to conflict. We focus on Wikipedia, the free encyclopedia\nanyone may edit, where disputes about content in controversial articles often\nreflect larger societal debates. While Wikipedia has a public edit history and\ndiscussion section for every article, the substance of these sections is\ndifficult to phantom for Wikipedia users interested in the development of an\narticle and in locating which topics were most controversial. In this paper we\npresent Contropedia, a tool that augments Wikipedia articles and gives insight\ninto the development of controversial topics. Contropedia uses an efficient\nlanguage agnostic measure based on the edit history that focuses on wiki links\nto easily identify which topics within a Wikipedia article have been most\ncontroversial and when.",
            "journal ref": "the 33rd Annual ACM Conference, Apr 2015, Seoul, France.\n  pp.193-196",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08770v1",
            "date": "2019-04-16 17:10:52+00:00",
            "title": "An Empirical Evaluation of Text Representation Schemes on Multilingual Social Web to Filter the Textual Aggression",
            "authors": [
                "Sandip Modha",
                "Prasenjit Majumder"
            ],
            "abstract": "This paper attempt to study the effectiveness of text representation schemes\non two tasks namely: User Aggression and Fact Detection from the social media\ncontents. In User Aggression detection, The aim is to identify the level of\naggression from the contents generated in the Social media and written in the\nEnglish, Devanagari Hindi and Romanized Hindi. Aggression levels are\ncategorized into three predefined classes namely: `Non-aggressive`, `Overtly\nAggressive`, and `Covertly Aggressive`. During the disaster-related incident,\nSocial media like, Twitter is flooded with millions of posts. In such emergency\nsituations, identification of factual posts is important for organizations\ninvolved in the relief operation. We anticipated this problem as a combination\nof classification and Ranking problem. This paper presents a comparison of\nvarious text representation scheme based on BoW techniques, distributed\nword/sentence representation, transfer learning on classifiers. Weighted $F_1$\nscore is used as a primary evaluation metric. Results show that text\nrepresentation using BoW performs better than word embedding on machine\nlearning classifiers. While pre-trained Word embedding techniques perform\nbetter on classifiers based on deep neural net. Recent transfer learning model\nlike ELMO, ULMFiT are fine-tuned for the Aggression classification task.\nHowever, results are not at par with pre-trained word embedding model. Overall,\nword embedding using fastText produce best weighted $F_1$-score than Word2Vec\nand Glove. Results are further improved using pre-trained vector model.\nStatistical significance tests are employed to ensure the significance of the\nclassification results. In the case of lexically different test Dataset, other\nthan training Dataset, deep neural models are more robust and perform\nsubstantially better than machine learning classifiers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.08920v2",
            "date": "2019-04-18 17:55:37+00:00",
            "title": "Towards VQA Models That Can Read",
            "authors": [
                "Amanpreet Singh",
                "Vivek Natarajan",
                "Meet Shah",
                "Yu Jiang",
                "Xinlei Chen",
                "Dhruv Batra",
                "Devi Parikh",
                "Marcus Rohrbach"
            ],
            "abstract": "Studies have shown that a dominant class of questions asked by visually\nimpaired users on images of their surroundings involves reading text in the\nimage. But today's VQA models can not read! Our paper takes a first step\ntowards addressing this problem. First, we introduce a new \"TextVQA\" dataset to\nfacilitate progress on this important problem. Existing datasets either have a\nsmall proportion of questions about text (e.g., the VQA dataset) or are too\nsmall (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408\nimages that require reasoning about text to answer. Second, we introduce a\nnovel model architecture that reads text in the image, reasons about it in the\ncontext of the image and the question, and predicts an answer which might be a\ndeduction based on the text and the image or composed of the strings found in\nthe image. Consequently, we call our approach Look, Read, Reason & Answer\n(LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on\nour TextVQA dataset. We find that the gap between human performance and machine\nperformance is significantly larger on TextVQA than on VQA 2.0, suggesting that\nTextVQA is well-suited to benchmark progress along directions complementary to\nVQA 2.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08936v1",
            "date": "2019-04-18 09:19:25+00:00",
            "title": "Language Modeling through Long Term Memory Network",
            "authors": [
                "Anupiya Nugaliyadde",
                "Kok Wai Wong",
                "Ferdous Sohel",
                "Hong Xie"
            ],
            "abstract": "Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and\nMemory Networks which contain memory are popularly used to learn patterns in\nsequential data. Sequential data has long sequences that hold relationships.\nRNN can handle long sequences but suffers from the vanishing and exploding\ngradient problems. While LSTM and other memory networks address this problem,\nthey are not capable of handling long sequences (50 or more data points long\nsequence patterns). Language modelling requiring learning from longer sequences\nare affected by the need for more information in memory. This paper introduces\nLong Term Memory network (LTM), which can tackle the exploding and vanishing\ngradient problems and handles long sequences without forgetting. LTM is\ndesigned to scale data in the memory and gives a higher weight to the input in\nthe sequence. LTM avoid overfitting by scaling the cell state after achieving\nthe optimal results. The LTM is tested on Penn treebank dataset, and Text8\ndataset and LTM achieves test perplexities of 83 and 82 respectively. 650 LTM\ncells achieved a test perplexity of 67 for Penn treebank, and 600 cells\nachieved a test perplexity of 77 for Text8. LTM achieves state of the art\nresults by only using ten hidden LTM cells for both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.08950v1",
            "date": "2019-04-18 18:00:30+00:00",
            "title": "No Permanent Friends or Enemies: Tracking Relationships between Nations from News",
            "authors": [
                "Xiaochuang Han",
                "Eunsol Choi",
                "Chenhao Tan"
            ],
            "abstract": "Understanding the dynamics of international politics is important yet\nchallenging for civilians. In this work, we explore unsupervised neural models\nto infer relations between nations from news articles. We extend existing\nmodels by incorporating shallow linguistics information and propose a new\nautomatic evaluation metric that aligns relationship dynamics with manually\nannotated key events. As understanding international relations requires\ncarefully analyzing complex relationships, we conduct in-person human\nevaluations with three groups of participants. Overall, humans prefer the\noutputs of our model and give insightful feedback that suggests future\ndirections for human-centered models. Furthermore, our model reveals\ninteresting regional differences in news coverage. For instance, with respect\nto US-China relations, Singaporean media focus more on \"strengthening\" and\n\"purchasing\", while US media focus more on \"criticizing\" and \"denouncing\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09037v1",
            "date": "2019-04-18 23:17:07+00:00",
            "title": "ProductNet: a Collection of High-Quality Datasets for Product Representation Learning",
            "authors": [
                "Chu Wang",
                "Lei Tang",
                "Yang Lu",
                "Shujun Bian",
                "Hirohisa Fujita",
                "Da Zhang",
                "Zuohua Zhang",
                "Yongning Wu"
            ],
            "abstract": "ProductNet is a collection of high-quality product datasets for better\nproduct understanding. Motivated by ImageNet, ProductNet aims at supporting\nproduct representation learning by curating product datasets of high quality\nwith properly chosen taxonomy. In this paper, the two goals of building\nhigh-quality product datasets and learning product representation support each\nother in an iterative fashion: the product embedding is obtained via a\nmulti-modal deep neural network (master model) designed to leverage product\nimage and catalog information; and in return, the embedding is utilized via\nactive learning (local model) to vastly accelerate the annotation process. For\nthe labeled data, the proposed master model yields high categorization accuracy\n(94.7% top-1 accuracy for 1240 classes), which can be used as search indices,\npartition keys, and input features for machine learning models. The product\nembedding, as well as the fined-tuned master model for a specific business\ntask, can also be used for various transfer learning tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.09049v3",
            "date": "2019-04-19 01:36:37+00:00",
            "title": "An Investigation of End-to-End Multichannel Speech Recognition for Reverberant and Mismatch Conditions",
            "authors": [
                "Aswin Shanmugam Subramanian",
                "Xiaofei Wang",
                "Shinji Watanabe",
                "Toru Taniguchi",
                "Dung Tran",
                "Yuya Fujita"
            ],
            "abstract": "Sequence-to-sequence (S2S) modeling is becoming a popular paradigm for\nautomatic speech recognition (ASR) because of its ability to jointly optimize\nall the conventional ASR components in an end-to-end (E2E) fashion. This report\ninvestigates the ability of E2E ASR from standard close-talk to far-field\napplications by encompassing entire multichannel speech enhancement and ASR\ncomponents within the S2S model. There have been previous studies on jointly\noptimizing neural beamforming alongside E2E ASR for denoising. It is clear from\nboth recent challenge outcomes and successful products that far-field systems\nwould be incomplete without solving both denoising and dereverberation\nsimultaneously. This report uses a recently developed architecture for\nfar-field ASR by composing neural extensions of dereverberation and beamforming\nmodules with the S2S ASR module as a single differentiable neural network and\nalso clearly defining the role of each subnetwork. The original implementation\nof this architecture was successfully applied to the noisy speech recognition\ntask (CHiME-4), while we applied this implementation to noisy reverberant tasks\n(DIRHA and REVERB). Our investigation shows that the method achieves better\nperformance than conventional pipeline methods on the DIRHA English dataset and\ncomparable performance on the REVERB dataset. It also has additional advantages\nof being neither iterative nor requiring parallel noisy and clean speech data.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.09067v2",
            "date": "2019-04-19 04:09:12+00:00",
            "title": "Emergence of Compositional Language with Deep Generational Transmission",
            "authors": [
                "Michael Cogswell",
                "Jiasen Lu",
                "Stefan Lee",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "Recent work has studied the emergence of language among deep reinforcement\nlearning agents that must collaborate to solve a task. Of particular interest\nare the factors that cause language to be compositional -- i.e., express\nmeaning by combining words which themselves have meaning. Evolutionary\nlinguists have found that in addition to structural priors like those already\nstudied in deep learning, the dynamics of transmitting language from generation\nto generation contribute significantly to the emergence of compositionality. In\nthis paper, we introduce these cultural evolutionary dynamics into language\nemergence by periodically replacing agents in a population to create a\nknowledge gap, implicitly inducing cultural transmission of language. We show\nthat this implicit cultural transmission encourages the resulting languages to\nexhibit better compositional generalization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.09234v1",
            "date": "2019-04-02 08:25:40+00:00",
            "title": "Software Tools for Big Data Resources in Family Names Dictionaries",
            "authors": [
                "Adam Rambousek",
                "Harry Parkin",
                "Ales Horak"
            ],
            "abstract": "This paper describes the design and development of specific software tools\nused during the creation of Family Names in Britain and Ireland (FaNBI)\nresearch project, started by the University of the West of England in 2010 and\nfinished successfully in 2016. First, the overview of the project and\nmethodology is provided. Next section contains the description of dictionary\nmanagement tools and software tools to combine input data resources.",
            "journal ref": "Names, 66:4, 246-255 (2018)",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09324v2",
            "date": "2019-04-19 19:53:01+00:00",
            "title": "Mask-Predict: Parallel Decoding of Conditional Masked Language Models",
            "authors": [
                "Marjan Ghazvininejad",
                "Omer Levy",
                "Yinhan Liu",
                "Luke Zettlemoyer"
            ],
            "abstract": "Most machine translation systems generate text autoregressively from left to\nright. We, instead, use a masked language modeling objective to train a model\nto predict any subset of the target words, conditioned on both the input text\nand a partially masked target translation. This approach allows for efficient\niterative decoding, where we first predict all of the target words\nnon-autoregressively, and then repeatedly mask out and regenerate the subset of\nwords that the model is least confident about. By applying this strategy for a\nconstant number of iterations, our model improves state-of-the-art performance\nlevels for non-autoregressive and parallel decoding translation models by over\n4 BLEU on average. It is also able to reach within about 1 BLEU point of a\ntypical left-to-right transformer model, while decoding significantly faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09380v1",
            "date": "2019-04-20 00:30:26+00:00",
            "title": "Repurposing Entailment for Multi-Hop Question Answering Tasks",
            "authors": [
                "Harsh Trivedi",
                "Heeyoung Kwon",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Question Answering (QA) naturally reduces to an entailment problem, namely,\nverifying whether some text entails the answer to a question. However, for\nmulti-hop QA tasks, which require reasoning with multiple sentences, it remains\nunclear how best to utilize entailment models pre-trained on large scale\ndatasets such as SNLI, which are based on sentence pairs. We introduce Multee,\na general architecture that can effectively use entailment models for multi-hop\nQA tasks. Multee uses (i) a local module that helps locate important sentences,\nthereby avoiding distracting information, and (ii) a global module that\naggregates information by effectively incorporating importance weights.\nImportantly, we show that both modules can use entailment functions pre-trained\non a large scale NLI datasets. We evaluate performance on MultiRC and\nOpenBookQA, two multihop QA datasets. When using an entailment function\npre-trained on NLI datasets, Multee outperforms QA models trained only on the\ntarget QA datasets and the OpenAI transformer models. The code is available at\nhttps://github.com/StonyBrookNLP/multee.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09408v2",
            "date": "2019-04-20 06:43:14+00:00",
            "title": "Language Models with Transformers",
            "authors": [
                "Chenguang Wang",
                "Mu Li",
                "Alexander J. Smola"
            ],
            "abstract": "The Transformer architecture is superior to RNN-based models in computational\nefficiency. Recently, GPT and BERT demonstrate the efficacy of Transformer\nmodels on various NLP tasks using pre-trained language models on large-scale\ncorpora. Surprisingly, these Transformer architectures are suboptimal for\nlanguage model itself. Neither self-attention nor the positional encoding in\nthe Transformer is able to efficiently incorporate the word-level sequential\ncontext crucial to language modeling.\n  In this paper, we explore effective Transformer architectures for language\nmodel, including adding additional LSTM layers to better capture the sequential\ncontext while still keeping the computation efficient. We propose Coordinate\nArchitecture Search (CAS) to find an effective architecture through iterative\nrefinement of the model. Experimental results on the PTB, WikiText-2, and\nWikiText-103 show that CAS achieves perplexities between 20.42 and 34.11 on all\nproblems, i.e. on average an improvement of 12.0 perplexity units compared to\nstate-of-the-art LSTMs. The source code is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09585v2",
            "date": "2019-04-21 12:09:39+00:00",
            "title": "Obfuscation for Privacy-preserving Syntactic Parsing",
            "authors": [
                "Zhifeng Hu",
                "Serhii Havrylov",
                "Ivan Titov",
                "Shay B. Cohen"
            ],
            "abstract": "The goal of homomorphic encryption is to encrypt data such that another party\ncan operate on it without being explicitly exposed to the content of the\noriginal data. We introduce an idea for a privacy-preserving transformation on\nnatural language data, inspired by homomorphic encryption. Our primary tool is\n{\\em obfuscation}, relying on the properties of natural language. Specifically,\na given English text is obfuscated using a neural model that aims to preserve\nthe syntactic relationships of the original sentence so that the obfuscated\nsentence can be parsed instead of the original one. The model works at the word\nlevel, and learns to obfuscate each word separately by changing it into a new\nword that has a similar syntactic role. The text obfuscated by our model leads\nto better performance on three syntactic parsers (two dependency and one\nconstituency parsers) in comparison to an upper-bound random substitution\nbaseline. More specifically, the results demonstrate that as more terms are\nobfuscated (by their part of speech), the substitution upper bound\nsignificantly degrades, while the neural model maintains a relatively high\nperforming parser. All of this is done without much sacrifice of privacy\ncompared to the random substitution upper bound. We also further analyze the\nresults, and discover that the substituted words have similar syntactic\nproperties, but different semantic content, compared to the original words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.09708v3",
            "date": "2019-04-22 03:12:09+00:00",
            "title": "Compositional generalization in a deep seq2seq model by separating syntax and semantics",
            "authors": [
                "Jake Russin",
                "Jason Jo",
                "Randall C. O'Reilly",
                "Yoshua Bengio"
            ],
            "abstract": "Standard methods in deep learning for natural language processing fail to\ncapture the compositional structure of human language that allows for\nsystematic generalization outside of the training distribution. However, human\nlearners readily generalize in this way, e.g. by applying known grammatical\nrules to novel words. Inspired by work in neuroscience suggesting separate\nbrain systems for syntactic and semantic processing, we implement a\nmodification to standard approaches in neural machine translation, imposing an\nanalogous separation. The novel model, which we call Syntactic Attention,\nsubstantially outperforms standard methods in deep learning on the SCAN\ndataset, a compositional generalization task, without any hand-engineered\nfeatures or additional supervision. Our work suggests that separating syntactic\nfrom semantic learning may be a useful heuristic for capturing compositional\nstructure.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10281v3",
            "date": "2019-04-23 12:36:59+00:00",
            "title": "Quaternion Knowledge Graph Embeddings",
            "authors": [
                "Shuai Zhang",
                "Yi Tay",
                "Lina Yao",
                "Qi Liu"
            ],
            "abstract": "In this work, we move beyond the traditional complex-valued representations,\nintroducing more expressive hypercomplex representations to model entities and\nrelations for knowledge graph embeddings. More specifically, quaternion\nembeddings, hypercomplex-valued embeddings with three imaginary components, are\nutilized to represent entities. Relations are modelled as rotations in the\nquaternion space. The advantages of the proposed approach are: (1) Latent\ninter-dependencies (between all components) are aptly captured with Hamilton\nproduct, encouraging a more compact interaction between entities and relations;\n(2) Quaternions enable expressive rotation in four-dimensional space and have\nmore degree of freedom than rotation in complex plane; (3) The proposed\nframework is a generalization of ComplEx on hypercomplex space while offering\nbetter geometrical interpretations, concurrently satisfying the key desiderata\nof relational representation learning (i.e., modeling symmetry, anti-symmetry\nand inversion). Experimental results demonstrate that our method achieves\nstate-of-the-art performance on four well-established knowledge graph\ncompletion benchmarks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10294v2",
            "date": "2019-04-23 13:11:40+00:00",
            "title": "Wasserstein-Fisher-Rao Document Distance",
            "authors": [
                "Zihao Wang",
                "Datong Zhou",
                "Yong Zhang",
                "Hao Wu",
                "Chenglong Bao"
            ],
            "abstract": "As a fundamental problem of natural language processing, it is important to\nmeasure the distance between different documents. Among the existing methods,\nthe Word Mover's Distance (WMD) has shown remarkable success in document\nsemantic matching for its clear physical insight as a parameter-free model.\nHowever, WMD is essentially based on the classical Wasserstein metric, thus it\noften fails to robustly represent the semantic similarity between texts of\ndifferent lengths. In this paper, we apply the newly developed\nWasserstein-Fisher-Rao (WFR) metric from unbalanced optimal transport theory to\nmeasure the distance between different documents. The proposed WFR document\ndistance maintains the great interpretability and simplicity as WMD. We\ndemonstrate that the WFR document distance has significant advantages when\ncomparing the texts of different lengths. In addition, an accelerated Sinkhorn\nbased algorithm with GPU implementation has been developed for the fast\ncomputation of WFR distances. The KNN classification results on eight datasets\nhave shown its clear improvement over WMD.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10503v1",
            "date": "2019-04-23 19:18:26+00:00",
            "title": "Fine-Grained Named Entity Recognition using ELMo and Wikidata",
            "authors": [
                "Cihan Dogan",
                "Aimore Dutra",
                "Adam Gara",
                "Alfredo Gemma",
                "Lei Shi",
                "Michael Sigamani",
                "Ella Walters"
            ],
            "abstract": "Fine-grained Named Entity Recognition is a task whereby we detect and\nclassify entity mentions to a large set of types. These types can span diverse\ndomains such as finance, healthcare, and politics. We observe that when the\ntype set spans several domains the accuracy of the entity detection becomes a\nlimitation for supervised learning models. The primary reason being the lack of\ndatasets where entity boundaries are properly annotated, whilst covering a\nlarge spectrum of entity types. Furthermore, many named entity systems suffer\nwhen considering the categorization of fine grained entity types. Our work\nattempts to address these issues, in part, by combining state-of-the-art deep\nlearning models (ELMo) with an expansive knowledge base (Wikidata). Using our\nframework, we cross-validate our model on the 112 fine-grained entity types\nbased on the hierarchy given from the Wiki(gold) dataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.10717v1",
            "date": "2019-04-24 09:41:14+00:00",
            "title": "Generating Token-Level Explanations for Natural Language Inference",
            "authors": [
                "James Thorne",
                "Andreas Vlachos",
                "Christos Christodoulopoulos",
                "Arpit Mittal"
            ],
            "abstract": "The task of Natural Language Inference (NLI) is widely modeled as supervised\nsentence pair classification. While there has been a lot of work recently on\ngenerating explanations of the predictions of classifiers on a single piece of\ntext, there have been no attempts to generate explanations of classifiers\noperating on pairs of sentences. In this paper, we show that it is possible to\ngenerate token-level explanations for NLI without the need for training data\nexplicitly annotated for this purpose. We use a simple LSTM architecture and\nevaluate both LIME and Anchor explanations for this task. We compare these to a\nMultiple Instance Learning (MIL) method that uses thresholded attention make\ntoken-level predictions. The approach we present in this paper is a novel\nextension of zero-shot single-sentence tagging to sentence pairs for NLI. We\nconduct our experiments on the well-studied SNLI dataset that was recently\naugmented with manually annotation of the tokens that explain the entailment\nrelation. We find that our white-box MIL-based method, while orders of\nmagnitude faster, does not reach the same accuracy as the black-box methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10743v1",
            "date": "2019-04-24 11:06:54+00:00",
            "title": "A bag-of-concepts model improves relation extraction in a narrow knowledge domain with limited data",
            "authors": [
                "Jiyu Chen",
                "Karin Verspoor",
                "Zenan Zhai"
            ],
            "abstract": "This paper focuses on a traditional relation extraction task in the context\nof limited annotated data and a narrow knowledge domain. We explore this task\nwith a clinical corpus consisting of 200 breast cancer follow-up treatment\nletters in which 16 distinct types of relations are annotated. We experiment\nwith an approach to extracting typed relations called window-bounded\nco-occurrence (WBC), which uses an adjustable context window around entity\nmentions of a relevant type, and compare its performance with a more typical\nintra-sentential co-occurrence baseline. We further introduce a new\nbag-of-concepts (BoC) approach to feature engineering based on the\nstate-of-the-art word embeddings and word synonyms. We demonstrate the\ncompetitiveness of BoC by comparing with methods of higher complexity, and\nexplore its effectiveness on this small dataset.",
            "journal ref": "In Proceedings of the Student Research Workshop at North American\n  Association for Computational Linguistics (NAACL) 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10760v1",
            "date": "2019-04-23 05:27:12+00:00",
            "title": "End-to-End Spoken Language Translation",
            "authors": [
                "Michelle Guo",
                "Albert Haque",
                "Prateek Verma"
            ],
            "abstract": "In this paper, we address the task of spoken language understanding. We\npresent a method for translating spoken sentences from one language into spoken\nsentences in another language. Given spectrogram-spectrogram pairs, our model\ncan be trained completely from scratch to translate unseen sentences. Our\nmethod consists of a pyramidal-bidirectional recurrent network combined with a\nconvolutional network to output sentence-level spectrograms in the target\nlanguage. Empirically, our model achieves competitive performance with\nstate-of-the-art methods on multiple languages and can generalize to unseen\nspeakers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10876v1",
            "date": "2019-04-24 15:40:14+00:00",
            "title": "Integrating Social Media into a Pan-European Flood Awareness System: A Multilingual Approach",
            "authors": [
                "V. Lorini",
                "C. Castillo",
                "F. Dottori",
                "M. Kalas",
                "D. Nappo",
                "P. Salamon"
            ],
            "abstract": "This paper describes a prototype system that integrates social media analysis\ninto the European Flood Awareness System (EFAS). This integration allows the\ncollection of social media data to be automatically triggered by flood risk\nwarnings determined by a hydro-meteorological model. Then, we adopt a\nmulti-lingual approach to find flood-related messages by employing two\nstate-of-the-art methodologies: language-agnostic word embeddings and\nlanguage-aligned word embeddings. Both approaches can be used to bootstrap a\nclassifier of social media messages for a new language with little or no\nlabeled data. Finally, we describe a method for selecting relevant and\nrepresentative messages and displaying them back in the interface of EFAS.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.10947v2",
            "date": "2019-04-24 17:44:06+00:00",
            "title": "On the Contributions of Visual and Textual Supervision in Low-Resource Semantic Speech Retrieval",
            "authors": [
                "Ankita Pasad",
                "Bowen Shi",
                "Herman Kamper",
                "Karen Livescu"
            ],
            "abstract": "Recent work has shown that speech paired with images can be used to learn\nsemantically meaningful speech representations even without any textual\nsupervision. In real-world low-resource settings, however, we often have access\nto some transcribed speech. We study whether and how visual grounding is useful\nin the presence of varying amounts of textual supervision. In particular, we\nconsider the task of semantic speech retrieval in a low-resource setting. We\nuse a previously studied data set and task, where models are trained on images\nwith spoken captions and evaluated on human judgments of semantic relevance. We\npropose a multitask learning approach to leverage both visual and textual\nmodalities, with visual supervision in the form of keyword probabilities from\nan external tagger. We find that visual grounding is helpful even in the\npresence of textual supervision, and we analyze this effect over a range of\nsizes of transcribed data sets. With ~5 hours of transcribed speech, we obtain\n23% higher average precision when also using visual supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11074v1",
            "date": "2019-04-24 21:27:09+00:00",
            "title": "An Attentional Neural Network Architecture for Folk Song Classification",
            "authors": [
                "Aitor Arronte-Alvarez",
                "Francisco Gomez-Martin"
            ],
            "abstract": "In this paper we present an attentional neural network for folk song\nclassification. We introduce the concept of musical motif embedding, and show\nhow using melodic local context we are able to model monophonic folk song\nmotifs using the skipgram version of the word2vec algorithm. We use the motif\nembeddings to represent folk songs from Germany, China, and Sweden, and\nclassify them using an attentional neural network that is able to discern\nrelevant motifs in a song. The results show how the network obtains state of\nthe art accuracy in a completely unsupervised manner, and how motif embeddings\nproduce high quality motif representations from folk songs. We conjecture on\nthe advantages of this type of representation in large symbolic music corpora,\nand how it can be helpful in the musicological analysis of folk song\ncollections from different cultures and geographical areas.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.11469v2",
            "date": "2019-04-25 17:29:16+00:00",
            "title": "The Zero Resource Speech Challenge 2019: TTS without T",
            "authors": [
                "Ewan Dunbar",
                "Robin Algayres",
                "Julien Karadayi",
                "Mathieu Bernard",
                "Juan Benjumea",
                "Xuan-Nga Cao",
                "Lucie Miskic",
                "Charlotte Dugrain",
                "Lucas Ondel",
                "Alan W. Black",
                "Laurent Besacier",
                "Sakriani Sakti",
                "Emmanuel Dupoux"
            ],
            "abstract": "We present the Zero Resource Speech Challenge 2019, which proposes to build a\nspeech synthesizer without any text or phonetic labels: hence, TTS without T\n(text-to-speech without text). We provide raw audio for a target voice in an\nunknown language (the Voice dataset), but no alignment, text or labels.\nParticipants must discover subword units in an unsupervised way (using the Unit\nDiscovery dataset) and align them to the voice recordings in a way that works\nbest for the purpose of synthesizing novel utterances from novel speakers,\nsimilar to the target speaker's voice. We describe the metrics used for\nevaluation, a baseline system consisting of unsupervised subword unit discovery\nplus a standard TTS system, and a topline TTS using gold phoneme\ntranscriptions. We present an overview of the 19 submitted systems from 10\nteams and discuss the main results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11475v1",
            "date": "2019-04-25 17:39:01+00:00",
            "title": "Importance of Copying Mechanism for News Headline Generation",
            "authors": [
                "Ilya Gusev"
            ],
            "abstract": "News headline generation is an essential problem of text summarization\nbecause it is constrained, well-defined, and is still hard to solve. Models\nwith a limited vocabulary can not solve it well, as new named entities can\nappear regularly in the news and these entities often should be in the\nheadline. News articles in morphologically rich languages such as Russian\nrequire model modifications due to a large number of possible word forms. This\nstudy aims to validate that models with a possibility of copying words from the\noriginal article performs better than models without such an option. The\nproposed model achieves a mean ROUGE score of 23 on the provided test dataset,\nwhich is 8 points greater than the result of a similar model without a copying\nmechanism. Moreover, the resulting model performs better than any known model\non the new dataset of Russian news.",
            "journal ref": "Computational Linguistics and Intellectual Technologies, Papers\n  from the Annual International Conference \"Dialogue\" (2019) Issue 18, 229-236",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.11574v2",
            "date": "2019-04-25 20:37:26+00:00",
            "title": "TVQA+: Spatio-Temporal Grounding for Video Question Answering",
            "authors": [
                "Jie Lei",
                "Licheng Yu",
                "Tamara L. Berg",
                "Mohit Bansal"
            ],
            "abstract": "We present the task of Spatio-Temporal Video Question Answering, which\nrequires intelligent systems to simultaneously retrieve relevant moments and\ndetect referenced visual concepts (people and objects) to answer natural\nlanguage questions about videos. We first augment the TVQA dataset with 310.8K\nbounding boxes, linking depicted objects to visual concepts in questions and\nanswers. We name this augmented version as TVQA+. We then propose\nSpatio-Temporal Answerer with Grounded Evidence (STAGE), a unified framework\nthat grounds evidence in both spatial and temporal domains to answer questions\nabout videos. Comprehensive experiments and analyses demonstrate the\neffectiveness of our framework and how the rich annotations in our TVQA+\ndataset can contribute to the question answering task. Moreover, by performing\nthis joint task, our model is able to produce insightful and interpretable\nspatio-temporal attention visualizations. Dataset and code are publicly\navailable at: http: //tvqa.cs.unc.edu, https://github.com/jayleicn/TVQAplus",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.11641v1",
            "date": "2019-04-26 01:42:41+00:00",
            "title": "Speaker Sincerity Detection based on Covariance Feature Vectors and Ensemble Methods",
            "authors": [
                "Mohammed Senoussaoui",
                "Patrick Cardinal",
                "Najim Dehak",
                "Alessandro Lameiras Koerich"
            ],
            "abstract": "Automatic measuring of speaker sincerity degree is a novel research problem\nin computational paralinguistics. This paper proposes covariance-based feature\nvectors to model speech and ensembles of support vector regressors to estimate\nthe degree of sincerity of a speaker. The elements of each covariance vector\nare pairwise statistics between the short-term feature components. These\nfeatures are used alone as well as in combination with the ComParE acoustic\nfeature set. The experimental results on the development set of the Sincerity\nSpeech Corpus using a cross-validation procedure have shown an 8.1% relative\nimprovement in the Spearman's correlation coefficient over the baseline system.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.11838v4",
            "date": "2019-04-26 13:33:56+00:00",
            "title": "Copy mechanism and tailored training for character-based data-to-text generation",
            "authors": [
                "Marco Roberti",
                "Giovanni Bonetta",
                "Rossella Cancelliere",
                "Patrick Gallinari"
            ],
            "abstract": "In the last few years, many different methods have been focusing on using\ndeep recurrent neural networks for natural language generation. The most widely\nused sequence-to-sequence neural methods are word-based: as such, they need a\npre-processing step called delexicalization (conversely, relexicalization) to\ndeal with uncommon or unknown words. These forms of processing, however, give\nrise to models that depend on the vocabulary used and are not completely\nneural.\n  In this work, we present an end-to-end sequence-to-sequence model with\nattention mechanism which reads and generates at a character level, no longer\nrequiring delexicalization, tokenization, nor even lowercasing. Moreover, since\ncharacters constitute the common \"building blocks\" of every text, it also\nallows a more general approach to text generation, enabling the possibility to\nexploit transfer learning for training. These skills are obtained thanks to two\nmajor features: (i) the possibility to alternate between the standard\ngeneration mechanism and a copy one, which allows to directly copy input facts\nto produce outputs, and (ii) the use of an original training pipeline that\nfurther improves the quality of the generated texts.\n  We also introduce a new dataset called E2E+, designed to highlight the\ncopying capabilities of character-based models, that is a modified version of\nthe well-known E2E dataset used in the E2E Challenge. We tested our model\naccording to five broadly accepted metrics (including the widely used BLEU),\nshowing that it yields competitive performance with respect to both\ncharacter-based and word-based approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.11961v1",
            "date": "2019-04-26 17:44:04+00:00",
            "title": "CoachAI: A Conversational Agent Assisted Health Coaching Platform",
            "authors": [
                "Ahmed Fadhil",
                "Gianluca Schiavo",
                "Yunlong Wang"
            ],
            "abstract": "Poor lifestyle represents a health risk factor and is the leading cause of\nmorbidity and chronic conditions. The impact of poor lifestyle can be\nsignificantly altered by individual behavior change. Although the current shift\nin healthcare towards a long lasting modifiable behavior, however, with\nincreasing caregiver workload and individuals' continuous needs of care, there\nis a need to ease caregiver's work while ensuring continuous interaction with\nusers. This paper describes the design and validation of CoachAI, a\nconversational agent assisted health coaching system to support health\nintervention delivery to individuals and groups. CoachAI instantiates a text\nbased healthcare chatbot system that bridges the remote human coach and the\nusers. This research provides three main contributions to the preventive\nhealthcare and healthy lifestyle promotion: (1) it presents the conversational\nagent to aid the caregiver; (2) it aims to decrease caregiver's workload and\nenhance care given to users, by handling (automating) repetitive caregiver\ntasks; and (3) it presents a domain independent mobile health conversational\nagent for health intervention delivery. We will discuss our approach and\nanalyze the results of a one month validation study on physical activity,\nhealthy diet and stress management.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1904.12580v1",
            "date": "2019-04-01 17:46:54+00:00",
            "title": "Twitter Sentiment Analysis using Distributed Word and Sentence Representation",
            "authors": [
                "Dwarampudi Mahidhar Reddy",
                "Dr. N V Subba Reddy",
                "Dr. N V Subba Reddy"
            ],
            "abstract": "An important part of the information gathering and data analysis is to find\nout what people think about, either a product or an entity. Twitter is an\nopinion rich social networking site. The posts or tweets from this data can be\nused for mining people's opinions. The recent surge of activity in this area\ncan be attributed to the computational treatment of data, which made opinion\nextraction and sentiment analysis easier. This paper classifies tweets into\npositive and negative sentiments, but instead of using traditional methods or\npreprocessing text data here we use the distributed representations of words\nand sentences to classify the tweets. We use Long Short Term Memory (LSTM)\nNetworks, Convolutional Neural Networks (CNNs) and Artificial Neural Networks.\nThe first two are used on Distributed Representation of words while the latter\nis used on the distributed representation of sentences. This paper achieves\naccuracies as high as 81%. It also suggests the best and optimal ways for\ncreating distributed representations of words for sentiment analysis, out of\nthe available methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.12584v1",
            "date": "2019-04-26 06:50:54+00:00",
            "title": "The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision",
            "authors": [
                "Jiayuan Mao",
                "Chuang Gan",
                "Pushmeet Kohli",
                "Joshua B. Tenenbaum",
                "Jiajun Wu"
            ],
            "abstract": "We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns\nvisual concepts, words, and semantic parsing of sentences without explicit\nsupervision on any of them; instead, our model learns by simply looking at\nimages and reading paired questions and answers. Our model builds an\nobject-based scene representation and translates sentences into executable,\nsymbolic programs. To bridge the learning of two modules, we use a\nneuro-symbolic reasoning module that executes these programs on the latent\nscene representation. Analogical to human concept learning, the perception\nmodule learns visual concepts based on the language description of the object\nbeing referred to. Meanwhile, the learned visual concepts facilitate learning\nnew words and parsing new sentences. We use curriculum learning to guide the\nsearching over the large compositional space of images and language. Extensive\nexperiments demonstrate the accuracy and efficiency of our model on learning\nvisual concepts, word representations, and semantic parsing of sentences.\nFurther, our method allows easy generalization to new object attributes,\ncompositions, language concepts, scenes and questions, and even new program\ndomains. It also empowers applications including visual question answering and\nbidirectional image-text retrieval.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.12624v1",
            "date": "2019-04-18 13:38:57+00:00",
            "title": "BowTie - A deep learning feedforward neural network for sentiment analysis",
            "authors": [
                "Apostol Vassilev"
            ],
            "abstract": "How to model and encode the semantics of human-written text and select the\ntype of neural network to process it are not settled issues in sentiment\nanalysis. Accuracy and transferability are critical issues in machine learning\nin general. These properties are closely related to the loss estimates for the\ntrained model. I present a computationally-efficient and accurate feedforward\nneural network for sentiment prediction capable of maintaining low losses. When\ncoupled with an effective semantics model of the text, it provides highly\naccurate models with low losses. Experimental results on representative\nbenchmark datasets and comparisons to other methods show the advantages of the\nnew approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1904.12638v2",
            "date": "2019-04-24 08:50:05+00:00",
            "title": "Context-Aware Zero-Shot Learning for Object Recognition",
            "authors": [
                "Eloi Zablocki",
                "Patrick Bordes",
                "Benjamin Piwowarski",
                "Laure Soulier",
                "Patrick Gallinari"
            ],
            "abstract": "Zero-Shot Learning (ZSL) aims at classifying unlabeled objects by leveraging\nauxiliary knowledge, such as semantic representations. A limitation of previous\napproaches is that only intrinsic properties of objects, e.g. their visual\nappearance, are taken into account while their context, e.g. the surrounding\nobjects in the image, is ignored. Following the intuitive principle that\nobjects tend to be found in certain contexts but not others, we propose a new\nand challenging approach, context-aware ZSL, that leverages semantic\nrepresentations in a new way to model the conditional likelihood of an object\nto appear in a given context. Finally, through extensive experiments conducted\non Visual Genome, we show that contextual information can substantially improve\nthe standard ZSL approach and is robust to unbalanced classes.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.12907v2",
            "date": "2019-04-29 18:59:59+00:00",
            "title": "Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning",
            "authors": [
                "Haonan Chen",
                "Hao Tan",
                "Alan Kuntz",
                "Mohit Bansal",
                "Ron Alterovitz"
            ],
            "abstract": "Enabling robots to understand instructions provided via spoken natural\nlanguage would facilitate interaction between robots and people in a variety of\nsettings in homes and workplaces. However, natural language instructions are\noften missing information that would be obvious to a human based on\nenvironmental context and common sense, and hence does not need to be\nexplicitly stated. In this paper, we introduce Language-Model-based Commonsense\nReasoning (LMCR), a new method which enables a robot to listen to a natural\nlanguage instruction from a human, observe the environment around it, and\nautomatically fill in information missing from the instruction using\nenvironmental context and a new commonsense reasoning approach. Our approach\nfirst converts an instruction provided as unconstrained natural language into a\nform that a robot can understand by parsing it into verb frames. Our approach\nthen fills in missing information in the instruction by observing objects in\nits vicinity and leveraging commonsense reasoning. To learn commonsense\nreasoning automatically, our approach distills knowledge from large\nunstructured textual corpora by training a language model. Our results show the\nfeasibility of a robot learning commonsense knowledge automatically from\nweb-based textual corpora, and the power of learned commonsense reasoning\nmodels in enabling a robot to autonomously perform tasks based on incomplete\nnatural language instructions.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1904.12973v2",
            "date": "2019-04-29 22:15:22+00:00",
            "title": "Unsupervised Extraction of Phenotypes from Cancer Clinical Notes for Association Studies",
            "authors": [
                "Stefan G. Stark",
                "Stephanie L. Hyland",
                "Melanie F. Pradier",
                "Kjong Lehmann",
                "Andreas Wicki",
                "Fernando Perez Cruz",
                "Julia E. Vogt",
                "Gunnar R\u00e4tsch"
            ],
            "abstract": "The recent adoption of Electronic Health Records (EHRs) by health care\nproviders has introduced an important source of data that provides detailed and\nhighly specific insights into patient phenotypes over large cohorts. These\ndatasets, in combination with machine learning and statistical approaches,\ngenerate new opportunities for research and clinical care. However, many\nmethods require the patient representations to be in structured formats, while\nthe information in the EHR is often locked in unstructured texts designed for\nhuman readability. In this work, we develop the methodology to automatically\nextract clinical features from clinical narratives from large EHR corpora\nwithout the need for prior knowledge. We consider medical terms and sentences\nappearing in clinical narratives as atomic information units. We propose an\nefficient clustering strategy suitable for the analysis of large text corpora\nand to utilize the clusters to represent information about the patient\ncompactly. To demonstrate the utility of our approach, we perform an\nassociation study of clinical features with somatic mutation profiles from\n4,007 cancer patients and their tumors. We apply the proposed algorithm to a\ndataset consisting of about 65 thousand documents with a total of about 3.2\nmillion sentences. We identify 341 significant statistical associations between\nthe presence of somatic mutations and clinical features. We annotated these\nassociations according to their novelty, and report several known associations.\nWe also propose 32 testable hypotheses where the underlying biological\nmechanism does not appear to be known but plausible. These results illustrate\nthat the automated discovery of clinical features is possible and the joint\nanalysis of clinical and genetic datasets can generate appealing new\nhypotheses.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.13015v4",
            "date": "2019-04-30 02:03:05+00:00",
            "title": "Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators",
            "authors": [
                "Sanghyun Yi",
                "Rahul Goel",
                "Chandra Khatri",
                "Alessandra Cervone",
                "Tagyoung Chung",
                "Behnam Hedayatnia",
                "Anu Venkatesh",
                "Raefer Gabriel",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Encoder-decoder based neural architectures serve as the basis of\nstate-of-the-art approaches in end-to-end open domain dialog systems. Since\nmost of such systems are trained with a maximum likelihood~(MLE) objective they\nsuffer from issues such as lack of generalizability and the generic response\nproblem, i.e., a system response that can be an answer to a large number of\nuser utterances, e.g., \"Maybe, I don't know.\" Having explicit feedback on the\nrelevance and interestingness of a system response at each turn can be a useful\nsignal for mitigating such issues and improving system quality by selecting\nresponses from different approaches. Towards this goal, we present a system\nthat evaluates chatbot responses at each dialog turn for coherence and\nengagement. Our system provides explicit turn-level dialog quality feedback,\nwhich we show to be highly correlated with human evaluation. To show that\nincorporating this feedback in the neural response generation models improves\ndialog quality, we present two different and complementary mechanisms to\nincorporate explicit feedback into a neural response generation model:\nreranking and direct modification of the loss function during training. Our\nstudies show that a response generation model that incorporates these combined\nfeedback mechanisms produce more engaging and coherent responses in an\nopen-domain spoken dialog setting, significantly improving the response quality\nusing both automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.13258v1",
            "date": "2019-04-30 13:59:18+00:00",
            "title": "English Broadcast News Speech Recognition by Humans and Machines",
            "authors": [
                "Samuel Thomas",
                "Masayuki Suzuki",
                "Yinghui Huang",
                "Gakuto Kurata",
                "Zoltan Tuske",
                "George Saon",
                "Brian Kingsbury",
                "Michael Picheny",
                "Tom Dibert",
                "Alice Kaiser-Schatzlein",
                "Bern Samko"
            ],
            "abstract": "With recent advances in deep learning, considerable attention has been given\nto achieving automatic speech recognition performance close to human\nperformance on tasks like conversational telephone speech (CTS) recognition. In\nthis paper we evaluate the usefulness of these proposed techniques on broadcast\nnews (BN), a similar challenging task. We also perform a set of recognition\nmeasurements to understand how close the achieved automatic speech recognition\nresults are to human performance on this task. On two publicly available BN\ntest sets, DEV04F and RT04, our speech recognition system using LSTM and\nresidual network based acoustic models with a combination of n-gram and neural\nnetwork language models performs at 6.5% and 5.9% word error rate. By achieving\nnew performance milestones on these test sets, our experiments show that\ntechniques developed on other related tasks, like CTS, can be transferred to\nachieve similar performance. In contrast, the best measured human recognition\nperformance on these test sets is much lower, at 3.6% and 2.8% respectively,\nindicating that there is still room for new techniques and improvements in this\nspace, to reach human performance levels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.13323v2",
            "date": "2019-04-30 15:37:16+00:00",
            "title": "Model Comparison for Semantic Grouping",
            "authors": [
                "Francisco Vargas",
                "Kamen Brestnichki",
                "Nils Hammerla"
            ],
            "abstract": "We introduce a probabilistic framework for quantifying the semantic\nsimilarity between two groups of embeddings. We formulate the task of semantic\nsimilarity as a model comparison task in which we contrast a generative model\nwhich jointly models two sentences versus one that does not. We illustrate how\nthis framework can be used for the Semantic Textual Similarity tasks using\nclear assumptions about how the embeddings of words are generated. We apply\nmodel comparison that utilises information criteria to address some of the\nshortcomings of Bayesian model comparison, whilst still penalising model\ncomplexity. We achieve competitive results by applying the proposed framework\nwith an appropriate choice of likelihood on the STS datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.13377v2",
            "date": "2019-04-30 17:20:32+00:00",
            "title": "Very Deep Self-Attention Networks for End-to-End Speech Recognition",
            "authors": [
                "Ngoc-Quan Pham",
                "Thai-Son Nguyen",
                "Jan Niehues",
                "Markus M\u00fcller",
                "Sebastian St\u00fcker",
                "Alexander Waibel"
            ],
            "abstract": "Recently, end-to-end sequence-to-sequence models for speech recognition have\ngained significant interest in the research community. While previous\narchitecture choices revolve around time-delay neural networks (TDNN) and long\nshort-term memory (LSTM) recurrent neural networks, we propose to use\nself-attention via the Transformer architecture as an alternative. Our analysis\nshows that deep Transformer networks with high learning capacity are able to\nexceed performance from previous end-to-end approaches and even match the\nconventional hybrid systems. Moreover, we trained very deep models with up to\n48 Transformer layers for both encoder and decoders combined with stochastic\nresidual connections, which greatly improve generalizability and training\nefficiency. The resulting models outperform all previous end-to-end ASR\napproaches on the Switchboard benchmark. An ensemble of these models achieve\n9.9% and 17.7% WER on Switchboard and CallHome test sets respectively. This\nfinding brings our end-to-end models to competitive levels with previous hybrid\nsystems. Further, with model ensembling the Transformers can outperform certain\nhybrid systems, which are more complicated in terms of both structure and\ntraining procedure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.00052v1",
            "date": "2019-04-30 18:29:28+00:00",
            "title": "Personalized Ranking in eCommerce Search",
            "authors": [
                "Grigor Aslanyan",
                "Aritra Mandal",
                "Prathyusha Senthil Kumar",
                "Amit Jaiswal",
                "Manojkumar Rangasamy Kannadasan"
            ],
            "abstract": "We address the problem of personalization in the context of eCommerce search.\nSpecifically, we develop personalization ranking features that use in-session\ncontext to augment a generic ranker optimized for conversion and relevance. We\nuse a combination of latent features learned from item co-clicks in historic\nsessions and content-based features that use item title and price.\nPersonalization in search has been discussed extensively in the existing\nliterature. The novelty of our work is combining and comparing content-based\nand content-agnostic features and showing that they complement each other to\nresult in a significant improvement of the ranker. Moreover, our technique does\nnot require an explicit re-ranking step, does not rely on learning user\nprofiles from long term search behavior, and does not involve complex modeling\nof query-item-user features. Our approach captures item co-click propensity\nusing lightweight item embeddings. We experimentally show that our technique\nsignificantly outperforms a generic ranker in terms of Mean Reciprocal Rank\n(MRR). We also provide anecdotal evidence for the semantic similarity captured\nby the item embeddings on the eBay search engine.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.00563v1",
            "date": "2019-05-02 03:30:17+00:00",
            "title": "Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications",
            "authors": [
                "Pouya Pezeshkpour",
                "Yifan Tian",
                "Sameer Singh"
            ],
            "abstract": "Representing entities and relations in an embedding space is a well-studied\napproach for machine learning on relational data. Existing approaches, however,\nprimarily focus on improving accuracy and overlook other aspects such as\nrobustness and interpretability. In this paper, we propose adversarial\nmodifications for link prediction models: identifying the fact to add into or\nremove from the knowledge graph that changes the prediction for a target fact\nafter the model is retrained. Using these single modifications of the graph, we\nidentify the most influential fact for a predicted link and evaluate the\nsensitivity of the model to the addition of fake facts. We introduce an\nefficient approach to estimate the effect of such modifications by\napproximating the change in the embeddings when the knowledge graph changes. To\navoid the combinatorial search over all possible facts, we train a network to\ndecode embeddings to their corresponding graph components, allowing the use of\ngradient-based optimization to identify the adversarial modification. We use\nthese techniques to evaluate the robustness of link prediction models (by\nmeasuring sensitivity to additional facts), study interpretability through the\nfacts most responsible for predictions (by identifying the most influential\nneighbors), and detect incorrect facts in the knowledge base.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.00787v4",
            "date": "2019-05-01 16:51:32+00:00",
            "title": "Computer Science and Metaphysics: A Cross-Fertilization",
            "authors": [
                "Daniel Kirchner",
                "Christoph Benzm\u00fcller",
                "Edward N. Zalta"
            ],
            "abstract": "Computational philosophy is the use of mechanized computational techniques to\nunearth philosophical insights that are either difficult or impossible to find\nusing traditional philosophical methods. Computational metaphysics is\ncomputational philosophy with a focus on metaphysics. In this paper, we (a)\ndevelop results in modal metaphysics whose discovery was computer assisted, and\n(b) conclude that these results work not only to the obvious benefit of\nphilosophy but also, less obviously, to the benefit of computer science, since\nthe new computational techniques that led to these results may be more broadly\napplicable within computer science. The paper includes a description of our\nbackground methodology and how it evolved, and a discussion of our new results.",
            "journal ref": "Open Philosophy, 2019",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1905.00855v1",
            "date": "2019-05-02 17:07:38+00:00",
            "title": "Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training",
            "authors": [
                "Bowen Shi",
                "Ming Sun",
                "Chieh-Chi Kao",
                "Viktor Rozgic",
                "Spyros Matsoukas",
                "Chao Wang"
            ],
            "abstract": "In this paper, we present a compression approach based on the combination of\nlow-rank matrix factorization and quantization training, to reduce complexity\nfor neural network based acoustic event detection (AED) models. Our\nexperimental results show this combined compression approach is very effective.\nFor a three-layer long short-term memory (LSTM) based AED model, the original\nmodel size can be reduced to 1% with negligible loss of accuracy. Our approach\nenables the feasibility of deploying AED for resource-constraint applications.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.00924v1",
            "date": "2019-05-02 18:23:47+00:00",
            "title": "Locale-agnostic Universal Domain Classification Model in Spoken Language Understanding",
            "authors": [
                "Jihwan Lee",
                "Ruhi Sarikaya",
                "Young-Bum Kim"
            ],
            "abstract": "In this paper, we introduce an approach for leveraging available data across\nmultiple locales sharing the same language to 1) improve domain classification\nmodel accuracy in Spoken Language Understanding and user experience even if new\nlocales do not have sufficient data and 2) reduce the cost of scaling the\ndomain classifier to a large number of locales. We propose a locale-agnostic\nuniversal domain classification model based on selective multi-task learning\nthat learns a joint representation of an utterance over locales with different\nsets of domains and allows locales to share knowledge selectively depending on\nthe domains. The experimental results demonstrate the effectiveness of our\napproach on domain classification task in the scenario of multiple locales with\nimbalanced data and disparate domain sets. The proposed approach outperforms\nother baselines models especially when classifying locale-specific domains and\nalso low-resourced domains.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.00957v1",
            "date": "2019-05-02 20:50:22+00:00",
            "title": "A Topic-Agnostic Approach for Identifying Fake News Pages",
            "authors": [
                "Sonia Castelo",
                "Thais Almeida",
                "Anas Elghafari",
                "A\u00e9cio Santos",
                "Kien Pham",
                "Eduardo Nakamura",
                "Juliana Freire"
            ],
            "abstract": "Fake news and misinformation have been increasingly used to manipulate\npopular opinion and influence political processes. To better understand fake\nnews, how they are propagated, and how to counter their effect, it is necessary\nto first identify them. Recently, approaches have been proposed to\nautomatically classify articles as fake based on their content. An important\nchallenge for these approaches comes from the dynamic nature of news: as new\npolitical events are covered, topics and discourse constantly change and thus,\na classifier trained using content from articles published at a given time is\nlikely to become ineffective in the future. To address this challenge, we\npropose a topic-agnostic (TAG) classification strategy that uses linguistic and\nweb-markup features to identify fake news pages. We report experimental results\nusing multiple data sets which show that our approach attains high accuracy in\nthe identification of fake news, even as topics evolve over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01347v2",
            "date": "2019-05-03 19:33:02+00:00",
            "title": "Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasets",
            "authors": [
                "Chris Dulhanty",
                "Alexander Wong"
            ],
            "abstract": "The ImageNet dataset ushered in a flood of academic and industry interest in\ndeep learning for computer vision applications. Despite its significant impact,\nthere has not been a comprehensive investigation into the demographic\nattributes of images contained within the dataset. Such a study could lead to\nnew insights on inherent biases within ImageNet, particularly important given\nit is frequently used to pretrain models for a wide variety of computer vision\ntasks. In this work, we introduce a model-driven framework for the automatic\nannotation of apparent age and gender attributes in large-scale image datasets.\nUsing this framework, we conduct the first demographic audit of the 2012\nImageNet Large Scale Visual Recognition Challenge (ILSVRC) subset of ImageNet\nand the \"person\" hierarchical category of ImageNet. We find that 41.62% of\nfaces in ILSVRC appear as female, 1.71% appear as individuals above the age of\n60, and males aged 15 to 29 account for the largest subgroup with 27.11%. We\nnote that the presented model-driven framework is not fair for all\nintersectional groups, so annotation are subject to bias. We present this work\nas the starting point for future development of unbiased annotation models and\nfor the study of downstream effects of imbalances in the demographics of\nImageNet. Code and annotations are available at:\nhttp://bit.ly/ImageNetDemoAudit",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.01386v1",
            "date": "2019-05-03 23:28:18+00:00",
            "title": "Personalized Query Auto-Completion Through a Lightweight Representation of the User Context",
            "authors": [
                "Manojkumar Rangasamy Kannadasan",
                "Grigor Aslanyan"
            ],
            "abstract": "Query Auto-Completion (QAC) is a widely used feature in many domains,\nincluding web and eCommerce search, suggesting full queries based on a prefix\ntyped by the user. QAC has been extensively studied in the literature in the\nrecent years, and it has been consistently shown that adding personalization\nfeatures can significantly improve the performance of QAC. In this work we\npropose a novel method for personalized QAC that uses lightweight embeddings\nlearnt through fastText. We construct an embedding for the user context\nqueries, which are the last few queries issued by the user. We also use the\nsame model to get the embedding for the candidate queries to be ranked. We\nintroduce ranking features that compute the distance between the candidate\nqueries and the context queries in the embedding space. These features are then\ncombined with other commonly used QAC ranking features to learn a ranking\nmodel. We apply our method to a large eCommerce search engine (eBay) and show\nthat the ranker with our proposed feature significantly outperforms the\nbaselines on all of the offline metrics measured, which includes Mean\nReciprocal Rank (MRR), Success Rate (SR), Mean Average Precision (MAP), and\nNormalized Discounted Cumulative Gain (NDCG). Our baselines include the Most\nPopular Completion (MPC) model as well as a ranking model without our proposed\nfeatures. The ranking model with the proposed features results in a $20-30\\%$\nimprovement over the MPC model on all metrics. We obtain up to a $5\\%$\nimprovement over the baseline ranking model for all the sessions, which goes up\nto about $10\\%$ when we restrict to sessions that contain the user context.\nMoreover, our proposed features also significantly outperform text based\npersonalization features studied in the literature before, and adding text\nbased features on top of our proposed embedding based features results only in\nminor improvements.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.01647v1",
            "date": "2019-05-05 10:30:03+00:00",
            "title": "A Typedriven Vector Semantics for Ellipsis with Anaphora using Lambek Calculus with Limited Contraction",
            "authors": [
                "Gijs Wijnholds",
                "Mehrnoosh Sadrzadeh"
            ],
            "abstract": "We develop a vector space semantics for verb phrase ellipsis with anaphora\nusing type-driven compositional distributional semantics based on the Lambek\ncalculus with limited contraction (LCC) of J\\\"ager (2006). Distributional\nsemantics has a lot to say about the statistical collocation-based meanings of\ncontent words, but provides little guidance on how to treat function words.\nFormal semantics on the other hand, has powerful mechanisms for dealing with\nrelative pronouns, coordinators, and the like. Type-driven compositional\ndistributional semantics brings these two models together. We review previous\ncompositional distributional models of relative pronouns, coordination and a\nrestricted account of ellipsis in the DisCoCat framework of Coecke et al.\n(2010, 2013). We show how DisCoCat cannot deal with general forms of ellipsis,\nwhich rely on copying of information, and develop a novel way of connecting\ntypelogical grammar to distributional semantics by assigning vector\ninterpretable lambda terms to derivations of LCC in the style of Muskens &\nSadrzadeh (2016). What follows is an account of (verb phrase) ellipsis in which\nword meanings can be copied: the meaning of a sentence is now a program with\nnon-linear access to individual word embeddings. We present the theoretical\nsetting, work out examples, and demonstrate our results on a toy distributional\nmodel motivated by data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01780v2",
            "date": "2019-05-06 01:16:33+00:00",
            "title": "Anonymized BERT: An Augmentation Approach to the Gendered Pronoun Resolution Challenge",
            "authors": [
                "Bo Liu"
            ],
            "abstract": "We present our 7th place solution to the Gendered Pronoun Resolution\nchallenge, which uses BERT without fine-tuning and a novel augmentation\nstrategy designed for contextual embedding token-level tasks. Our method\nanonymizes the referent by replacing candidate names with a set of common\nplaceholder names. Besides the usual benefits of effectively increasing\ntraining data size, this approach diversifies idiosyncratic information\nembedded in names. Using same set of common first names can also help the model\nrecognize names better, shorten token length, and remove gender and regional\nbiases associated with names. The system scored 0.1947 log loss in stage 2,\nwhere the augmentation contributed to an improvements of 0.04. Post-competition\nanalysis shows that, when using different embedding layers, the system scores\n0.1799 which would be third place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01957v1",
            "date": "2019-04-13 20:04:22+00:00",
            "title": "M2H-GAN: A GAN-based Mapping from Machine to Human Transcripts for Speech Understanding",
            "authors": [
                "Titouan Parcollet",
                "Mohamed Morchid",
                "Xavier Bost",
                "Georges Linar\u00e8s"
            ],
            "abstract": "Deep learning is at the core of recent spoken language understanding (SLU)\nrelated tasks. More precisely, deep neural networks (DNNs) drastically\nincreased the performances of SLU systems, and numerous architectures have been\nproposed. In the real-life context of theme identification of telephone\nconversations, it is common to hold both a human, manual (TRS) and an\nautomatically transcribed (ASR) versions of the conversations. Nonetheless, and\ndue to production constraints, only the ASR transcripts are considered to build\nautomatic classifiers. TRS transcripts are only used to measure the\nperformances of ASR systems. Moreover, the recent performances in term of\nclassification accuracy, obtained by DNN related systems are close to the\nperformances reached by humans, and it becomes difficult to further increase\nthe performances by only considering the ASR transcripts. This paper proposes\nto distillates the TRS knowledge available during the training phase within the\nASR representation, by using a new generative adversarial network called\nM2H-GAN to generate a TRS-like version of an ASR document, to improve the theme\nidentification performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01958v1",
            "date": "2019-04-11 17:31:23+00:00",
            "title": "Text2Node: a Cross-Domain System for Mapping Arbitrary Phrases to a Taxonomy",
            "authors": [
                "Rohollah Soltani",
                "Alexandre Tomberg"
            ],
            "abstract": "Electronic health record (EHR) systems are used extensively throughout the\nhealthcare domain. However, data interchangeability between EHR systems is\nlimited due to the use of different coding standards across systems. Existing\nmethods of mapping coding standards based on manual human experts mapping,\ndictionary mapping, symbolic NLP and classification are unscalable and cannot\naccommodate large scale EHR datasets.\n  In this work, we present Text2Node, a cross-domain mapping system capable of\nmapping medical phrases to concepts in a large taxonomy (such as SNOMED CT).\nThe system is designed to generalize from a limited set of training samples and\nmap phrases to elements of the taxonomy that are not covered by training data.\nAs a result, our system is scalable, robust to wording variants between coding\nsystems and can output highly relevant concepts when no exact concept exists in\nthe target taxonomy. Text2Node operates in three main stages: first, the\nlexicon is mapped to word embeddings; second, the taxonomy is vectorized using\nnode embeddings; and finally, the mapping function is trained to connect the\ntwo embedding spaces. We compared multiple algorithms and architectures for\neach stage of the training, including GloVe and FastText word embeddings, CNN\nand Bi-LSTM mapping functions, and node2vec for node embeddings. We confirmed\nthe robustness and generalisation properties of Text2Node by mapping ICD-9-CM\nDiagnosis phrases to SNOMED CT and by zero-shot training at comparable\naccuracy.\n  This system is a novel methodological contribution to the task of normalizing\nand linking phrases to a taxonomy, advancing data interchangeability in\nhealthcare. When applied, the system can use electronic health records to\ngenerate an embedding that incorporates taxonomical medical knowledge to\nimprove clinical predictive models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01959v1",
            "date": "2019-04-19 02:30:59+00:00",
            "title": "Relation Discovery with Out-of-Relation Knowledge Base as Supervision",
            "authors": [
                "Yan Liang",
                "Xin Liu",
                "Jianwen Zhang",
                "Yangqiu Song"
            ],
            "abstract": "Unsupervised relation discovery aims to discover new relations from a given\ntext corpus without annotated data. However, it does not consider existing\nhuman annotated knowledge bases even when they are relevant to the relations to\nbe discovered. In this paper, we study the problem of how to use\nout-of-relation knowledge bases to supervise the discovery of unseen relations,\nwhere out-of-relation means that relations to discover from the text corpus and\nthose in knowledge bases are not overlapped. We construct a set of constraints\nbetween entity pairs based on the knowledge base embedding and then incorporate\nconstraints into the relation discovery by a variational auto-encoder based\nalgorithm. Experiments show that our new approach can improve the\nstate-of-the-art relation discovery performance by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01963v1",
            "date": "2019-04-26 08:21:08+00:00",
            "title": "Neural Chinese Word Segmentation with Lexicon and Unlabeled Data via Posterior Regularization",
            "authors": [
                "Junxin Liu",
                "Fangzhao Wu",
                "Chuhan Wu",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "Existing methods for CWS usually rely on a large number of labeled sentences\nto train word segmentation models, which are expensive and time-consuming to\nannotate. Luckily, the unlabeled data is usually easy to collect and many\nhigh-quality Chinese lexicons are off-the-shelf, both of which can provide\nuseful information for CWS. In this paper, we propose a neural approach for\nChinese word segmentation which can exploit both lexicon and unlabeled data.\nOur approach is based on a variant of posterior regularization algorithm, and\nthe unlabeled data and lexicon are incorporated into model training as indirect\nsupervision by regularizing the prediction space of CWS models. Extensive\nexperiments on multiple benchmark datasets in both in-domain and cross-domain\nscenarios validate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01964v1",
            "date": "2019-04-26 08:09:56+00:00",
            "title": "Neural Chinese Named Entity Recognition via CNN-LSTM-CRF and Joint Training with Word Segmentation",
            "authors": [
                "Fangzhao Wu",
                "Junxin Liu",
                "Chuhan Wu",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "Chinese named entity recognition (CNER) is an important task in Chinese\nnatural language processing field. However, CNER is very challenging since\nChinese entity names are highly context-dependent. In addition, Chinese texts\nlack delimiters to separate words, making it difficult to identify the boundary\nof entities. Besides, the training data for CNER in many domains is usually\ninsufficient, and annotating enough training data for CNER is very expensive\nand time-consuming. In this paper, we propose a neural approach for CNER.\nFirst, we introduce a CNN-LSTM-CRF neural architecture to capture both local\nand long-distance contexts for CNER. Second, we propose a unified framework to\njointly train CNER and word segmentation models in order to enhance the ability\nof CNER model in identifying entity boundaries. Third, we introduce an\nautomatic method to generate pseudo labeled samples from existing labeled data\nwhich can enrich the training data. Experiments on two benchmark datasets show\nthat our approach can effectively improve the performance of Chinese named\nentity recognition, especially when training data is insufficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01972v1",
            "date": "2019-04-24 09:46:58+00:00",
            "title": "A Self-Attentive Emotion Recognition Network",
            "authors": [
                "Harris Partaourides",
                "Kostantinos Papadamou",
                "Nicolas Kourtellis",
                "Ilias Leontiadis",
                "Sotirios Chatzis"
            ],
            "abstract": "Modern deep learning approaches have achieved groundbreaking performance in\nmodeling and classifying sequential data. Specifically, attention networks\nconstitute the state-of-the-art paradigm for capturing long temporal dynamics.\nThis paper examines the efficacy of this paradigm in the challenging task of\nemotion recognition in dyadic conversations. In contrast to existing\napproaches, our work introduces a novel attention mechanism capable of\ninferring the immensity of the effect of each past utterance on the current\nspeaker emotional state. The proposed attention mechanism performs this\ninference procedure without the need of a decoder network; this is achieved by\nmeans of innovative self-attention arguments. Our self-attention networks\ncapture the correlation patterns among consecutive encoder network states, thus\nallowing to robustly and effectively model temporal dynamics over arbitrary\nlong temporal horizons. Thus, we enable capturing strong affective patterns\nover the course of long discussions. We exhibit the effectiveness of our\napproach considering the challenging IEMOCAP benchmark. As we show, our devised\nmethodology outperforms state-of-the-art alternatives and commonly used\napproaches, giving rise to promising new research directions in the context of\nOnline Social Network (OSN) analysis tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01973v1",
            "date": "2019-04-19 10:13:07+00:00",
            "title": "Who wrote this book? A challenge for e-commerce",
            "authors": [
                "B\u00e9ranger Dumont",
                "Simona Maggio",
                "Ghiles Sidi Said",
                "Quoc-Tien Au"
            ],
            "abstract": "Modern e-commerce catalogs contain millions of references, associated with\ntextual and visual information that is of paramount importance for the products\nto be found via search or browsing. Of particular significance is the book\ncategory, where the author name(s) field poses a significant challenge. Indeed,\nbooks written by a given author (such as F. Scott Fitzgerald) might be listed\nwith different authors' names in a catalog due to abbreviations and spelling\nvariants and mistakes, among others. To solve this problem at scale, we design\na composite system involving open data sources for books as well as machine\nlearning components leveraging deep learning-based techniques for natural\nlanguage processing. In particular, we use Siamese neural networks for an\napproximate match with known author names, and direct correction of the\nprovided author's name using sequence-to-sequence learning with neural\nnetworks. We evaluate this approach on product data from the e-commerce website\nRakuten France, and find that the top proposal of the system is the normalized\nauthor name with 72% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01984v1",
            "date": "2019-05-01 23:26:38+00:00",
            "title": "AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions",
            "authors": [
                "Qiuyun Zhang",
                "Bin Guo",
                "Hao Wang",
                "Yunji Liang",
                "Shaoyang Hao",
                "Zhiwen Yu"
            ],
            "abstract": "In the last two decades, the landscape of text generation has undergone\ntremendous changes and is being reshaped by the success of deep learning. New\ntechnologies for text generation ranging from template-based methods to neural\nnetwork-based methods emerged. Meanwhile, the research objectives have also\nchanged from generating smooth and coherent sentences to infusing personalized\ntraits to enrich the diversification of newly generated content. With the rapid\ndevelopment of text generation solutions, one comprehensive survey is urgent to\nsummarize the achievements and track the state of the arts. In this survey\npaper, we present the general systematical framework, illustrate the widely\nutilized models and summarize the classic applications of text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01992v2",
            "date": "2019-04-29 15:21:13+00:00",
            "title": "An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model",
            "authors": [
                "Oluwatobi Olabiyi",
                "Anish Khazane",
                "Alan Salimov",
                "Erik T. Mueller"
            ],
            "abstract": "In this paper, we extend the persona-based sequence-to-sequence (Seq2Seq)\nneural network conversation model to a multi-turn dialogue scenario by\nmodifying the state-of-the-art hredGAN architecture to simultaneously capture\nutterance attributes such as speaker identity, dialogue topic, speaker\nsentiments and so on. The proposed system, phredGAN has a persona-based HRED\ngenerator (PHRED) and a conditional discriminator. We also explore two\napproaches to accomplish the conditional discriminator: (1) phredGAN_a, a\nsystem that passes the attribute representation as an additional input into a\ntraditional adversarial discriminator, and (2) phredGAN_d, a dual discriminator\nsystem which in addition to the adversarial discriminator, collaboratively\npredicts the attribute(s) that generated the input utterance. To demonstrate\nthe superior performance of phredGAN over the persona Seq2Seq model, we\nexperiment with two conversational datasets, the Ubuntu Dialogue Corpus (UDC)\nand TV series transcripts from the Big Bang Theory and Friends. Performance\ncomparison is made with respect to a variety of quantitative measures as well\nas crowd-sourced human evaluation. We also explore the trade-offs from using\neither variant of phredGAN on datasets with many but weak attribute modalities\n(such as with Big Bang Theory and Friends) and ones with few but strong\nattribute modalities (customer-agent interactions in Ubuntu dataset).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.01998v1",
            "date": "2019-04-29 15:09:34+00:00",
            "title": "A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework",
            "authors": [
                "Oluwatobi O. Olabiyi",
                "Anish Khazane",
                "Erik T. Mueller"
            ],
            "abstract": "In this paper, we extend the persona-based sequence-to-sequence (Seq2Seq)\nneural network conversation model to multi-turn dialogue by modifying the\nstate-of-the-art hredGAN architecture. To achieve this, we introduce an\nadditional input modality into the encoder and decoder of hredGAN to capture\nother attributes such as speaker identity, location, sub-topics, and other\nexternal attributes that might be available from the corpus of human-to-human\ninteractions. The resulting persona hredGAN ($phredGAN$) shows better\nperformance than both the existing persona-based Seq2Seq and hredGAN models\nwhen those external attributes are available in a multi-turn dialogue corpus.\nThis superiority is demonstrated on TV drama series with character consistency\n(such as Big Bang Theory and Friends) and customer service interaction datasets\nsuch as Ubuntu dialogue corpus in terms of perplexity, BLEU, ROUGE, and\nDistinct n-gram scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02019v1",
            "date": "2019-05-02 01:07:20+00:00",
            "title": "Conditioning LSTM Decoder and Bi-directional Attention Based Question Answering System",
            "authors": [
                "Heguang Liu"
            ],
            "abstract": "Applying neural-networks on Question Answering has gained increasing\npopularity in recent years. In this paper, I implemented a model with\nBi-directional attention flow layer, connected with a Multi-layer LSTM encoder,\nconnected with one start-index decoder and one conditioning end-index decoder.\nI introduce a new end-index decoder layer, conditioning on start-index output.\nThe Experiment shows this has increased model performance by 15.16%. For\nprediction, I proposed a new smart-span equation, rewarding both short answer\nlength and high probability in start-index and end-index, which further\nimproved the prediction accuracy. The best single model achieves an F1 score of\n73.97% and EM score of 64.95% on test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02283v1",
            "date": "2019-05-06 22:38:18+00:00",
            "title": "Caveats in Generating Medical Imaging Labels from Radiology Reports",
            "authors": [
                "Tobi Olatunji",
                "Li Yao",
                "Ben Covington",
                "Alexander Rhodes",
                "Anthony Upton"
            ],
            "abstract": "Acquiring high-quality annotations in medical imaging is usually a costly\nprocess. Automatic label extraction with natural language processing (NLP) has\nemerged as a promising workaround to bypass the need of expert annotation.\nDespite the convenience, the limitation of such an approximation has not been\ncarefully examined and is not well understood. With a challenging set of 1,000\nchest X-ray studies and their corresponding radiology reports, we show that\nthere exists a surprisingly large discrepancy between what radiologists\nvisually perceive and what they clinically report. Furthermore, with inherently\nflawed report as ground truth, the state-of-the-art medical NLP fails to\nproduce high-fidelity labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02430v1",
            "date": "2019-05-07 09:23:12+00:00",
            "title": "Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings",
            "authors": [
                "Iva Gornishka",
                "Stevan Rudinac",
                "Marcel Worring"
            ],
            "abstract": "In this paper we present a novel interactive multimodal learning system,\nwhich facilitates search and exploration in large networks of social multimedia\nusers. It allows the analyst to identify and select users of interest, and to\nfind similar users in an interactive learning setting. Our approach is based on\nnovel multimodal representations of users, words and concepts, which we\nsimultaneously learn by deploying a general-purpose neural embedding model. We\nshow these representations to be useful not only for categorizing users, but\nalso for automatically generating user and community profiles. Inspired by\ntraditional summarization approaches, we create the profiles by selecting\ndiverse and representative content from all available modalities, i.e. the\ntext, image and user modality. The usefulness of the approach is evaluated\nusing artificial actors, which simulate user behavior in a relevance feedback\nscenario. Multiple experiments were conducted in order to evaluate the quality\nof our multimodal representations, to compare different embedding strategies,\nand to determine the importance of different modalities. We demonstrate the\ncapabilities of the proposed approach on two different multimedia collections\noriginating from the violent online extremism forum Stormfront and the\nmicroblogging platform Twitter, which are particularly interesting due to the\nhigh semantic level of the discussions they feature.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.02450v5",
            "date": "2019-05-07 10:13:04+00:00",
            "title": "MASS: Masked Sequence to Sequence Pre-training for Language Generation",
            "authors": [
                "Kaitao Song",
                "Xu Tan",
                "Tao Qin",
                "Jianfeng Lu",
                "Tie-Yan Liu"
            ],
            "abstract": "Pre-training and fine-tuning, e.g., BERT, have achieved great success in\nlanguage understanding by transferring knowledge from rich-resource\npre-training task to the low/zero-resource downstream tasks. Inspired by the\nsuccess of BERT, we propose MAsked Sequence to Sequence pre-training (MASS) for\nthe encoder-decoder based language generation tasks. MASS adopts the\nencoder-decoder framework to reconstruct a sentence fragment given the\nremaining part of the sentence: its encoder takes a sentence with randomly\nmasked fragment (several consecutive tokens) as input, and its decoder tries to\npredict this masked fragment. In this way, MASS can jointly train the encoder\nand decoder to develop the capability of representation extraction and language\nmodeling. By further fine-tuning on a variety of zero/low-resource language\ngeneration tasks, including neural machine translation, text summarization and\nconversational response generation (3 tasks and totally 8 datasets), MASS\nachieves significant improvements over the baselines without pre-training or\nwith other pre-training methods. Specially, we achieve the state-of-the-art\naccuracy (37.5 in terms of BLEU score) on the unsupervised English-French\ntranslation, even beating the early attention-based supervised model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02497v2",
            "date": "2019-05-07 12:30:55+00:00",
            "title": "RelExt: Relation Extraction using Deep Learning approaches for Cybersecurity Knowledge Graph Improvement",
            "authors": [
                "Aditya Pingle",
                "Aritran Piplai",
                "Sudip Mittal",
                "Anupam Joshi",
                "James Holt",
                "Richard Zak"
            ],
            "abstract": "Security Analysts that work in a `Security Operations Center' (SoC) play a\nmajor role in ensuring the security of the organization. The amount of\nbackground knowledge they have about the evolving and new attacks makes a\nsignificant difference in their ability to detect attacks. Open source threat\nintelligence sources, like text descriptions about cyber-attacks, can be stored\nin a structured fashion in a cybersecurity knowledge graph. A cybersecurity\nknowledge graph can be paramount in aiding a security analyst to detect cyber\nthreats because it stores a vast range of cyber threat information in the form\nof semantic triples which can be queried. A semantic triple contains two\ncybersecurity entities with a relationship between them. In this work, we\npropose a system to create semantic triples over cybersecurity text, using deep\nlearning approaches to extract possible relationships. We use the set of\nsemantic triples generated through our system to assert in a cybersecurity\nknowledge graph. Security Analysts can retrieve this data from the knowledge\ngraph, and use this information to form a decision about a cyber-attack.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.02525v1",
            "date": "2019-04-30 15:35:25+00:00",
            "title": "Many-to-Many Voice Conversion with Out-of-Dataset Speaker Support",
            "authors": [
                "Gokce Keskin",
                "Tyler Lee",
                "Cory Stephenson",
                "Oguz H. Elibol"
            ],
            "abstract": "We present a Cycle-GAN based many-to-many voice conversion method that can\nconvert between speakers that are not in the training set. This property is\nenabled through speaker embeddings generated by a neural network that is\njointly trained with the Cycle-GAN. In contrast to prior work in this domain,\nour method enables conversion between an out-of-dataset speaker and a target\nspeaker in either direction and does not require re-training. Out-of-dataset\nspeaker conversion quality is evaluated using an independently trained speaker\nidentification model, and shows good style conversion characteristics for\npreviously unheard speakers. Subjective tests on human listeners show style\nconversion quality for in-dataset speakers is comparable to the\nstate-of-the-art baseline model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.02545v2",
            "date": "2019-05-03 10:30:17+00:00",
            "title": "Meeting Transcription Using Virtual Microphone Arrays",
            "authors": [
                "Takuya Yoshioka",
                "Zhuo Chen",
                "Dimitrios Dimitriadis",
                "William Hinthorn",
                "Xuedong Huang",
                "Andreas Stolcke",
                "Michael Zeng"
            ],
            "abstract": "We describe a system that generates speaker-annotated transcripts of meetings\nby using a virtual microphone array, a set of spatially distributed\nasynchronous recording devices such as laptops and mobile phones. The system is\ncomposed of continuous audio stream alignment, blind beamforming, speech\nrecognition, speaker diarization using prior speaker information, and system\ncombination. When utilizing seven input audio streams, our system achieves a\nword error rate (WER) of 22.3% and comes within 3% of the close-talking\nmicrophone WER on the non-overlapping speech segments. The speaker-attributed\nWER (SAWER) is 26.7%. The relative gains in SAWER over the single-device system\nare 14.8%, 20.3%, and 22.4% for three, five, and seven microphones,\nrespectively. The presented system achieves a 13.6% diarization error rate when\n10% of the speech duration contains more than one speaker. The contribution of\neach component to the overall performance is also investigated, and we validate\nthe system with experiments on the NIST RT-07 conference meeting test set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.02895v1",
            "date": "2019-05-07 12:15:32+00:00",
            "title": "Cyber-All-Intel: An AI for Security related Threat Intelligence",
            "authors": [
                "Sudip Mittal",
                "Anupam Joshi",
                "Tim Finin"
            ],
            "abstract": "Keeping up with threat intelligence is a must for a security analyst today.\nThere is a volume of information present in `the wild' that affects an\norganization. We need to develop an artificial intelligence system that scours\nthe intelligence sources, to keep the analyst updated about various threats\nthat pose a risk to her organization. A security analyst who is better `tapped\nin' can be more effective.\n  In this paper we present, Cyber-All-Intel an artificial intelligence system\nto aid a security analyst. It is a system for knowledge extraction,\nrepresentation and analytics in an end-to-end pipeline grounded in the\ncybersecurity informatics domain. It uses multiple knowledge representations\nlike, vector spaces and knowledge graphs in a 'VKG structure' to store incoming\nintelligence. The system also uses neural network models to pro-actively\nimprove its knowledge. We have also created a query engine and an alert system\nthat can be used by an analyst to find actionable cybersecurity insights.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.03042v1",
            "date": "2019-04-18 14:13:03+00:00",
            "title": "Rumour Detection via News Propagation Dynamics and User Representation Learning",
            "authors": [
                "Tien Huu Do",
                "Xiao Luo",
                "Duc Minh Nguyen",
                "Nikos Deligiannis"
            ],
            "abstract": "Rumours have existed for a long time and have been known for serious\nconsequences. The rapid growth of social media platforms has multiplied the\nnegative impact of rumours; it thus becomes important to early detect them.\nMany methods have been introduced to detect rumours using the content or the\nsocial context of news. However, most existing methods ignore or do not explore\neffectively the propagation pattern of news in social media, including the\nsequence of interactions of social media users with news across time. In this\nwork, we propose a novel method for rumour detection based on deep learning.\nOur method leverages the propagation process of the news by learning the users'\nrepresentation and the temporal interrelation of users' responses. Experiments\nconducted on Twitter and Weibo datasets demonstrate the state-of-the-art\nperformance of the proposed method.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.03072v3",
            "date": "2019-05-08 13:57:28+00:00",
            "title": "RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentation",
            "authors": [
                "Christoph L\u00fcscher",
                "Eugen Beck",
                "Kazuki Irie",
                "Markus Kitza",
                "Wilfried Michel",
                "Albert Zeyer",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "We present state-of-the-art automatic speech recognition (ASR) systems\nemploying a standard hybrid DNN/HMM architecture compared to an attention-based\nencoder-decoder design for the LibriSpeech task. Detailed descriptions of the\nsystem development, including model design, pretraining schemes, training\nschedules, and optimization approaches are provided for both system\narchitectures. Both hybrid DNN/HMM and attention-based systems employ\nbi-directional LSTMs for acoustic modeling/encoding. For language modeling, we\nemploy both LSTM and Transformer based architectures. All our systems are built\nusing RWTHs open-source toolkits RASR and RETURNN. To the best knowledge of the\nauthors, the results obtained when training on the full LibriSpeech training\nset, are the best published currently, both for the hybrid DNN/HMM and the\nattention-based systems. Our single hybrid system even outperforms previous\nresults obtained from combining eight single systems. Our comparison shows that\non the LibriSpeech 960h task, the hybrid DNN/HMM system outperforms the\nattention-based system by 15% relative on the clean and 40% relative on the\nother test sets in terms of word error rate. Moreover, experiments on a reduced\n100h-subset of the LibriSpeech training corpus even show a more pronounced\nmargin between the hybrid DNN/HMM and attention-based architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.03500v2",
            "date": "2019-05-09 09:22:40+00:00",
            "title": "Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech",
            "authors": [
                "Tobias Menne",
                "Ilya Sklyar",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "Significant performance degradation of automatic speech recognition (ASR)\nsystems is observed when the audio signal contains cross-talk. One of the\nrecently proposed approaches to solve the problem of multi-speaker ASR is the\ndeep clustering (DPCL) approach. Combining DPCL with a state-of-the-art hybrid\nacoustic model, we obtain a word error rate (WER) of 16.5 % on the commonly\nused wsj0-2mix dataset, which is the best performance reported thus far to the\nbest of our knowledge. The wsj0-2mix dataset contains simulated cross-talk\nwhere the speech of multiple speakers overlaps for almost the entire utterance.\nIn a more realistic ASR scenario the audio signal contains significant portions\nof single-speaker speech and only part of the signal contains speech of\nmultiple competing speakers. This paper investigates obstacles of applying DPCL\nas a preprocessing method for ASR in such a scenario of sparsely overlapping\nspeech. To this end we present a data simulation approach, closely related to\nthe wsj0-2mix dataset, generating sparsely overlapping speech datasets of\narbitrary overlap ratio. The analysis of applying DPCL to sparsely overlapping\nspeech is an important interim step between the fully overlapping datasets like\nwsj0-2mix and more realistic ASR datasets, such as CHiME-5 or AMI.",
            "journal ref": "Proceedings of INTERSPEECH 2019",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1905.03638v2",
            "date": "2019-05-09 13:51:46+00:00",
            "title": "Mappa Mundi: An Interactive Artistic Mind Map Generator with Artificial Imagination",
            "authors": [
                "Ruixue Liu",
                "Baoyang Chen",
                "Meng Chen",
                "Youzheng Wu",
                "Zhijie Qiu",
                "Xiaodong He"
            ],
            "abstract": "We present a novel real-time, collaborative, and interactive AI painting\nsystem, Mappa Mundi, for artistic Mind Map creation. The system consists of a\nvoice-based input interface, an automatic topic expansion module, and an image\nprojection module. The key innovation is to inject Artificial Imagination into\npainting creation by considering lexical and phonological similarities of\nlanguage, learning and inheriting artist's original painting style, and\napplying the principles of Dadaism and impossibility of improvisation. Our\nsystem indicates that AI and artist can collaborate seamlessly to create\nimaginative artistic painting and Mappa Mundi has been applied in art\nexhibition in UCCA, Beijing",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.03813v4",
            "date": "2019-05-09 18:47:38+00:00",
            "title": "When Deep Learning Met Code Search",
            "authors": [
                "Jose Cambronero",
                "Hongyu Li",
                "Seohyun Kim",
                "Koushik Sen",
                "Satish Chandra"
            ],
            "abstract": "There have been multiple recent proposals on using deep neural networks for\ncode search using natural language. Common across these proposals is the idea\nof $\\mathit{embedding}$ code and natural language queries, into real vectors\nand then using vector distance to approximate semantic correlation between code\nand the query. Multiple approaches exist for learning these embeddings,\nincluding $\\mathit{unsupervised}$ techniques, which rely only on a corpus of\ncode examples, and $\\mathit{supervised}$ techniques, which use an\n$\\mathit{aligned}$ corpus of paired code and natural language descriptions. The\ngoal of this supervision is to produce embeddings that are more similar for a\nquery and the corresponding desired code snippet. Clearly, there are choices in\nwhether to use supervised techniques at all, and if one does, what sort of\nnetwork and training to use for supervision. This paper is the first to\nevaluate these choices systematically. To this end, we assembled\nimplementations of state-of-the-art techniques to run on a common platform,\ntraining and evaluation corpora. To explore the design space in network\ncomplexity, we also introduced a new design point that is a $\\mathit{minimal}$\nsupervision extension to an existing unsupervised technique. Our evaluation\nshows that: 1. adding supervision to an existing unsupervised technique can\nimprove performance, though not necessarily by much; 2. simple networks for\nsupervision can be more effective that more sophisticated sequence-based\nnetworks for code search; 3. while it is common to use docstrings to carry out\nsupervision, there is a sizeable gap between the effectiveness of docstrings\nand a more query-appropriate supervision corpus.\n  The evaluation dataset is now available at arXiv:1908.09804",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1905.04035v1",
            "date": "2019-05-10 09:44:35+00:00",
            "title": "Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models",
            "authors": [
                "Derya Cavdar",
                "Valeriu Codreanu",
                "Can Karakus",
                "John A. Lockman III",
                "Damian Podareanu",
                "Vikram Saletore",
                "Alexander Sergeev",
                "Don D. Smith II",
                "Victor Suthichai",
                "Quy Ta",
                "Srinivas Varadharajan",
                "Lucas A. Wilson",
                "Rengan Xu",
                "Pei Yang"
            ],
            "abstract": "Neural machine translation - using neural networks to translate human\nlanguage - is an area of active research exploring new neuron types and network\ntopologies with the goal of dramatically improving machine translation\nperformance. Current state-of-the-art approaches, such as the multi-head\nattention-based transformer, require very large translation corpuses and many\nepochs to produce models of reasonable quality. Recent attempts to parallelize\nthe official TensorFlow \"Transformer\" model across multiple nodes have hit\nroadblocks due to excessive memory use and resulting out of memory errors when\nperforming MPI collectives. This paper describes modifications made to the\nHorovod MPI-based distributed training framework to reduce memory usage for\ntransformer models by converting assumed-sparse tensors to dense tensors, and\nsubsequently replacing sparse gradient gather with dense gradient reduction.\nThe result is a dramatic increase in scale-out capability, with CPU-only\nscaling tests achieving 91% weak scaling efficiency up to 1200 MPI processes\n(300 nodes), and up to 65% strong scaling efficiency up to 400 MPI processes\n(200 nodes) using the Stampede2 supercomputer.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.04071v2",
            "date": "2019-05-10 11:14:12+00:00",
            "title": "Survey on Evaluation Methods for Dialogue Systems",
            "authors": [
                "Jan Deriu",
                "Alvaro Rodrigo",
                "Arantxa Otegi",
                "Guillermo Echegoyen",
                "Sophie Rosset",
                "Eneko Agirre",
                "Mark Cieliebak"
            ],
            "abstract": "In this paper we survey the methods and concepts developed for the evaluation\nof dialogue systems. Evaluation is a crucial part during the development\nprocess. Often, dialogue systems are evaluated by means of human evaluations\nand questionnaires. However, this tends to be very cost and time intensive.\nThus, much work has been put into finding methods, which allow to reduce the\ninvolvement of human labour. In this survey, we present the main concepts and\nmethods. For this, we differentiate between the various classes of dialogue\nsystems (task-oriented dialogue systems, conversational dialogue systems, and\nquestion-answering dialogue systems). We cover each class by introducing the\nmain technologies developed for the dialogue systems and then by presenting the\nevaluation methods regarding this class.",
            "journal ref": "Artificial Intelligence Review, June 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04260v1",
            "date": "2019-05-10 17:00:40+00:00",
            "title": "Check-It: A Plugin for Detecting and Reducing the Spread of Fake News and Misinformation on the Web",
            "authors": [
                "Demetris Paschalides",
                "Alexandros Kornilakis",
                "Chrysovalantis Christodoulou",
                "Rafael Andreou",
                "George Pallis",
                "Marios D. Dikaiakos",
                "Evangelos Markatos"
            ],
            "abstract": "Over the past few years, we have been witnessing the rise of misinformation\non the Web. People fall victims of fake news during their daily lives and\nassist their further propagation knowingly and inadvertently. There have been\nmany initiatives that are trying to mitigate the damage caused by fake news,\nfocusing on signals from either domain flag-lists, online social networks or\nartificial intelligence. In this work, we present Check-It, a system that\ncombines, in an intelligent way, a variety of signals into a pipeline for fake\nnews identification. Check-It is developed as a web browser plugin with the\nobjective of efficient and timely fake news detection, respecting the user's\nprivacy. Experimental results show that Check-It is able to outperform the\nstate-of-the-art methods. On a dataset, consisting of 9 millions of articles\nlabeled as fake and real, Check-It obtains classification accuracies that\nexceed 99%.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.04554v1",
            "date": "2019-05-11 17:20:19+00:00",
            "title": "Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification",
            "authors": [
                "Achintya kr. Sarkar",
                "Zheng-Hua Tan",
                "Hao Tang",
                "Suwon Shon",
                "James Glass"
            ],
            "abstract": "There are a number of studies about extraction of bottleneck (BN) features\nfrom deep neural networks (DNNs)trained to discriminate speakers, pass-phrases\nand triphone states for improving the performance of text-dependent speaker\nverification (TD-SV). However, a moderate success has been achieved. A recent\nstudy [1] presented a time contrastive learning (TCL) concept to explore the\nnon-stationarity of brain signals for classification of brain states. Speech\nsignals have similar non-stationarity property, and TCL further has the\nadvantage of having no need for labeled data. We therefore present a TCL based\nBN feature extraction method. The method uniformly partitions each speech\nutterance in a training dataset into a predefined number of multi-frame\nsegments. Each segment in an utterance corresponds to one class, and class\nlabels are shared across utterances. DNNs are then trained to discriminate all\nspeech frames among the classes to exploit the temporal structure of speech. In\naddition, we propose a segment-based unsupervised clustering algorithm to\nre-assign class labels to the segments. TD-SV experiments were conducted on the\nRedDots challenge database. The TCL-DNNs were trained using speech data of\nfixed pass-phrases that were excluded from the TD-SV evaluation set, so the\nlearned features can be considered phrase-independent. We compare the\nperformance of the proposed TCL bottleneck (BN) feature with those of\nshort-time cepstral features and BN features extracted from DNNs discriminating\nspeakers, pass-phrases, speaker+pass-phrase, as well as monophones whose labels\nand boundaries are generated by three different automatic speech recognition\n(ASR) systems. Experimental results show that the proposed TCL-BN outperforms\ncepstral features and speaker+pass-phrase discriminant BN features, and its\nperformance is on par with those of ASR derived BN features. Moreover,....",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing,\n  2019",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1905.04655v1",
            "date": "2019-05-12 06:11:30+00:00",
            "title": "Improving Natural Language Interaction with Robots Using Advice",
            "authors": [
                "Nikhil Mehta",
                "Dan Goldwasser"
            ],
            "abstract": "Over the last few years, there has been growing interest in learning models\nfor physically grounded language understanding tasks, such as the popular\nblocks world domain. These works typically view this problem as a single-step\nprocess, in which a human operator gives an instruction and an automated agent\nis evaluated on its ability to execute it. In this paper we take the first step\ntowards increasing the bandwidth of this interaction, and suggest a protocol\nfor including advice, high-level observations about the task, which can help\nconstrain the agent's prediction. We evaluate our approach on the blocks world\ntask, and show that even simple advice can help lead to significant performance\nimprovements. To help reduce the effort involved in supplying the advice, we\nalso explore model self-generated advice which can still improve results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04749v2",
            "date": "2019-05-12 17:15:11+00:00",
            "title": "A Benchmark Study of Machine Learning Models for Online Fake News Detection",
            "authors": [
                "Junaed Younus Khan",
                "Md. Tawkat Islam Khondaker",
                "Sadia Afroz",
                "Gias Uddin",
                "Anindya Iqbal"
            ],
            "abstract": "The proliferation of fake news and its propagation on social media has become\na major concern due to its ability to create devastating impacts. Different\nmachine learning approaches have been suggested to detect fake news. However,\nmost of those focused on a specific type of news (such as political) which\nleads us to the question of dataset-bias of the models used. In this research,\nwe conducted a benchmark study to assess the performance of different\napplicable machine learning approaches on three different datasets where we\naccumulated the largest and most diversified one. We explored a number of\nadvanced pre-trained language models for fake news detection along with the\ntraditional and deep learning ones and compared their performances from\ndifferent aspects for the first time to the best of our knowledge. We find that\nBERT and similar pre-trained models perform the best for fake news detection,\nespecially with very small dataset. Hence, these models are significantly\nbetter option for languages with limited electronic contents, i.e., training\ndata. We also carried out several analysis based on the models' performance,\narticle's topic, article's length, and discussed different lessons learned from\nthem. We believe that this benchmark study will help the research community to\nexplore further and news sites/blogs to select the most appropriate fake news\ndetection method.",
            "journal ref": "Machine Learning with Applications: 4(2021).100032",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04847v1",
            "date": "2019-05-13 03:34:14+00:00",
            "title": "Synchronous Bidirectional Neural Machine Translation",
            "authors": [
                "Long Zhou",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Existing approaches to neural machine translation (NMT) generate the target\nlanguage sequence token by token from left to right. However, this kind of\nunidirectional decoding framework cannot make full use of the target-side\nfuture contexts which can be produced in a right-to-left decoding direction,\nand thus suffers from the issue of unbalanced outputs. In this paper, we\nintroduce a synchronous bidirectional neural machine translation (SB-NMT) that\npredicts its outputs using left-to-right and right-to-left decoding\nsimultaneously and interactively, in order to leverage both of the history and\nfuture information at the same time. Specifically, we first propose a new\nalgorithm that enables synchronous bidirectional decoding in a single model.\nThen, we present an interactive decoding model in which left-to-right\n(right-to-left) generation does not only depend on its previously generated\noutputs, but also relies on future contexts predicted by right-to-left\n(left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on\nlarge-scale NIST Chinese-English, WMT14 English-German, and WMT18\nRussian-English translation tasks. Experimental results demonstrate that our\nmodel achieves significant improvements over the strong Transformer model by\n3.92, 1.49 and 1.04 BLEU points respectively, and obtains the state-of-the-art\nperformance on Chinese-English and English-German translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.04877v1",
            "date": "2019-05-13 06:31:33+00:00",
            "title": "Quantifying and Alleviating the Language Prior Problem in Visual Question Answering",
            "authors": [
                "Yangyang Guo",
                "Zhiyong Cheng",
                "Liqiang Nie",
                "Yibing Liu",
                "Yinglong Wang",
                "Mohan Kankanhalli"
            ],
            "abstract": "Benefiting from the advancement of computer vision, natural language\nprocessing and information retrieval techniques, visual question answering\n(VQA), which aims to answer questions about an image or a video, has received\nlots of attentions over the past few years. Although some progress has been\nachieved so far, several studies have pointed out that current VQA models are\nheavily affected by the \\emph{language prior problem}, which means they tend to\nanswer questions based on the co-occurrence patterns of question keywords\n(e.g., how many) and answers (e.g., 2) instead of understanding images and\nquestions. Existing methods attempt to solve this problem by either balancing\nthe biased datasets or forcing models to better understand images. However,\nonly marginal effects and even performance deterioration are observed for the\nfirst and second solution, respectively. In addition, another important issue\nis the lack of measurement to quantitatively measure the extent of the language\nprior effect, which severely hinders the advancement of related techniques.\n  In this paper, we make contributions to solve the above problems from two\nperspectives. Firstly, we design a metric to quantitatively measure the\nlanguage prior effect of VQA models. The proposed metric has been demonstrated\nto be effective in our empirical studies. Secondly, we propose a regularization\nmethod (i.e., score regularization module) to enhance current VQA models by\nalleviating the language prior problem as well as boosting the backbone model\nperformance. The proposed score regularization module adopts a pair-wise\nlearning strategy, which makes the VQA models answer the question based on the\nreasoning of the image (upon this question) instead of basing on\nquestion-answer patterns observed in the biased training set. The score\nregularization module is flexible to be integrated into various VQA models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.04914v1",
            "date": "2019-05-13 08:53:31+00:00",
            "title": "Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs",
            "authors": [
                "Lingbing Guo",
                "Zequn Sun",
                "Wei Hu"
            ],
            "abstract": "We study the problem of knowledge graph (KG) embedding. A widely-established\nassumption to this problem is that similar entities are likely to have similar\nrelational roles. However, existing related methods derive KG embeddings mainly\nbased on triple-level learning, which lack the capability of capturing\nlong-term relational dependencies of entities. Moreover, triple-level learning\nis insufficient for the propagation of semantic information among entities,\nespecially for the case of cross-KG embedding. In this paper, we propose\nrecurrent skipping networks (RSNs), which employ a skipping mechanism to bridge\nthe gaps between entities. RSNs integrate recurrent neural networks (RNNs) with\nresidual learning to efficiently capture the long-term relational dependencies\nwithin and between KGs. We design an end-to-end framework to support RSNs on\ndifferent tasks. Our experimental results showed that RSNs outperformed\nstate-of-the-art embedding-based methods for entity alignment and achieved\ncompetitive performance for KG completion.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.05547v2",
            "date": "2019-05-14 12:22:39+00:00",
            "title": "Multilingual Factor Analysis",
            "authors": [
                "Francisco Vargas",
                "Kamen Brestnichki",
                "Alex Papadopoulos-Korfiatis",
                "Nils Hammerla"
            ],
            "abstract": "In this work we approach the task of learning multilingual word\nrepresentations in an offline manner by fitting a generative latent variable\nmodel to a multilingual dictionary. We model equivalent words in different\nlanguages as different views of the same word generated by a common latent\nvariable representing their latent lexical meaning. We explore the task of\nalignment by querying the fitted model for multilingual embeddings achieving\ncompetitive results across a variety of tasks. The proposed model is robust to\nnoise in the embedding space making it a suitable method for distributed\nrepresentations learned from noisy corpora.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.05598v1",
            "date": "2019-05-08 19:27:22+00:00",
            "title": "Confirmatory Factor Analysis -- A Case study",
            "authors": [
                "Rui Portocarrero Sarmento",
                "Vera Costa"
            ],
            "abstract": "Confirmatory Factor Analysis (CFA) is a particular form of factor analysis,\nmost commonly used in social research. In confirmatory factor analysis, the\nresearcher first develops a hypothesis about what factors they believe are\nunderlying the used measures and may impose constraints on the model based on\nthese a priori hypotheses. For example, if two factors are accounting for the\ncovariance in the measures, and these factors are unrelated to one another, we\ncan create a model where the correlation between factor X and factor Y is set\nto zero. Measures could then be obtained to assess how well the fitted model\ncaptured the covariance between all the items or measures in the model. Thus,\nif the results of statistical tests of the model fit indicate a poor fit, the\nmodel will be rejected. If the fit is weak, it may be due to a variety of\nreasons. We propose to introduce state of the art techniques to do CFA in R\nlanguage. Then, we propose to do some examples of CFA with R and some datasets,\nrevealing several scenarios where CFA is relevant.",
            "journal ref": "None",
            "category": "stat.AP"
        },
        {
            "url": "http://arxiv.org/abs/1905.05700v1",
            "date": "2019-05-07 21:14:03+00:00",
            "title": "Learning meters of Arabic and English poems with Recurrent Neural Networks: a step forward for language understanding and synthesis",
            "authors": [
                "Waleed A. Yousef",
                "Omar M. Ibrahime",
                "Taha M. Madbouly",
                "Moustafa A. Mahmoud"
            ],
            "abstract": "Recognizing a piece of writing as a poem or prose is usually easy for the\nmajority of people; however, only specialists can determine which meter a poem\nbelongs to. In this paper, we build Recurrent Neural Network (RNN) models that\ncan classify poems according to their meters from plain text. The input text is\nencoded at the character level and directly fed to the models without feature\nhandcrafting. This is a step forward for machine understanding and synthesis of\nlanguages in general, and Arabic language in particular. Among the 16 poem\nmeters of Arabic and the 4 meters of English the networks were able to\ncorrectly classify poem with an overall accuracy of 96.38\\% and 82.31\\%\nrespectively. The poem datasets used to conduct this research were massive,\nover 1.5 million of verses, and were crawled from different nontechnical\nsources, almost Arabic and English literature sites, and in different\nheterogeneous and unstructured formats. These datasets are now made publicly\navailable in clean, structured, and documented format for other future\nresearch. To the best of the authors' knowledge, this research is the first to\naddress classifying poem meters in a machine learning approach, in general, and\nin RNN featureless based approach, in particular. In addition, the dataset is\nthe first publicly available dataset ready for the purpose of future\ncomputational research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05701v2",
            "date": "2019-05-09 20:57:14+00:00",
            "title": "Modeling user context for valence prediction from narratives",
            "authors": [
                "Aniruddha Tammewar",
                "Alessandra Cervone",
                "Eva-Maria Messner",
                "Giuseppe Riccardi"
            ],
            "abstract": "Automated prediction of valence, one key feature of a person's emotional\nstate, from individuals' personal narratives may provide crucial information\nfor mental healthcare (e.g. early diagnosis of mental diseases, supervision of\ndisease course, etc.). In the Interspeech 2018 ComParE Self-Assessed Affect\nchallenge, the task of valence prediction was framed as a three-class\nclassification problem using 8 seconds fragments from individuals' narratives.\nAs such, the task did not allow for exploring contextual information of the\nnarratives. In this work, we investigate the intrinsic information from\nmultiple narratives recounted by the same individual in order to predict their\ncurrent state-of-mind. Furthermore, with generalizability in mind, we decided\nto focus our experiments exclusively on textual information as the public\navailability of audio narratives is limited compared to text. Our hypothesis\nis, that context modeling might provide insights about emotion triggering\nconcepts (e.g. events, people, places) mentioned in the narratives that are\nlinked to an individual's state of mind. We explore multiple machine learning\ntechniques to model narratives. We find that the models are able to capture\ninter-individual differences, leading to more accurate predictions of an\nindividual's emotional state, as compared to single narratives.",
            "journal ref": "Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05704v5",
            "date": "2019-05-10 19:39:55+00:00",
            "title": "A logical-based corpus for cross-lingual evaluation",
            "authors": [
                "Felipe Salvatore",
                "Marcelo Finger",
                "Roberto Hirata Jr"
            ],
            "abstract": "At present, different deep learning models are presenting high accuracy on\npopular inference datasets such as SNLI, MNLI, and SciTail. However, there are\ndifferent indicators that those datasets can be exploited by using some simple\nlinguistic patterns. This fact poses difficulties to our understanding of the\nactual capacity of machine learning models to solve the complex task of textual\ninference. We propose a new set of syntactic tasks focused on contradiction\ndetection that require specific capacities over linguistic logical forms such\nas: Boolean coordination, quantifiers, definite description, and counting\noperators. We evaluate two kinds of deep learning models that implicitly\nexploit language structure: recurrent models and the Transformer network BERT.\nWe show that although BERT is clearly more efficient to generalize over most\nlogical forms, there is space for improvement when dealing with counting\noperators. Since the syntactic tasks can be implemented in different languages,\nwe show a successful case of cross-lingual transfer learning between English\nand Portuguese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.05778v3",
            "date": "2019-05-14 18:01:41+00:00",
            "title": "Misleading Failures of Partial-input Baselines",
            "authors": [
                "Shi Feng",
                "Eric Wallace",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Recent work establishes dataset difficulty and removes annotation artifacts\nvia partial-input baselines (e.g., hypothesis-only models for SNLI or\nquestion-only models for VQA). When a partial-input baseline gets high\naccuracy, a dataset is cheatable. However, the converse is not necessarily\ntrue: the failure of a partial-input baseline does not mean a dataset is free\nof artifacts. To illustrate this, we first design artificial datasets which\ncontain trivial patterns in the full input that are undetectable by any\npartial-input model. Next, we identify such artifacts in the SNLI dataset - a\nhypothesis-only model augmented with trivial patterns in the premise can solve\n15% of the examples that are previously considered \"hard\". Our work provides a\ncaveat for the use of partial-input baselines for dataset verification and\ncreation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.05888v1",
            "date": "2019-05-14 23:53:57+00:00",
            "title": "Generative Design in Minecraft: Chronicle Challenge",
            "authors": [
                "Christoph Salge",
                "Christian Guckelsberger",
                "Michael Cerny Green",
                "Rodrigo Canaan",
                "Julian Togelius"
            ],
            "abstract": "We introduce the Chronicle Challenge as an optional addition to the\nSettlement Generation Challenge in Minecraft. One of the foci of the overall\ncompetition is adaptive procedural content generation (PCG), an arguably\nunder-explored problem in computational creativity. In the base challenge,\nparticipants must generate new settlements that respond to and ideally interact\nwith existing content in the world, such as the landscape or climate. The goal\nis to understand the underlying creative process, and to design better PCG\nsystems. The Chronicle Challenge in particular focuses on the generation of a\nnarrative based on the history of a generated settlement, expressed in natural\nlanguage. We discuss the unique features of the Chronicle Challenge in\ncomparison to other competitions, clarify the characteristics of a chronicle\neligible for submission and describe the evaluation criteria. We furthermore\ndraw on simulation-based approaches in computational storytelling as examples\nto how this challenge could be approached.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.05961v1",
            "date": "2019-05-15 06:15:16+00:00",
            "title": "Demographic Inference and Representative Population Estimates from Multilingual Social Media Data",
            "authors": [
                "Zijian Wang",
                "Scott A. Hale",
                "David Adelani",
                "Przemyslaw A. Grabowicz",
                "Timo Hartmann",
                "Fabian Fl\u00f6ck",
                "David Jurgens"
            ],
            "abstract": "Social media provide access to behavioural data at an unprecedented scale and\ngranularity. However, using these data to understand phenomena in a broader\npopulation is difficult due to their non-representativeness and the bias of\nstatistical inference tools towards dominant languages and groups. While\ndemographic attribute inference could be used to mitigate such bias, current\ntechniques are almost entirely monolingual and fail to work in a global\nenvironment. We address these challenges by combining multilingual demographic\ninference with post-stratification to create a more representative population\nsample. To learn demographic attributes, we create a new multimodal deep neural\narchitecture for joint classification of age, gender, and organization-status\nof social media users that operates in 32 languages. This method substantially\noutperforms current state of the art while also reducing algorithmic bias. To\ncorrect for sampling biases, we propose fully interpretable multilevel\nregression methods that estimate inclusion probabilities from inferred joint\npopulation counts and ground-truth population counts. In a large experiment\nover multilingual heterogeneous European regions, we show that our demographic\ninference and bias correction together allow for more accurate estimates of\npopulations and make a significant step towards representative social sensing\nin downstream applications with multilingual social media.",
            "journal ref": "Proceedings of the 2019 World Wide Web Conference (WWW '19), May\n  13--17, 2019, San Francisco, CA, USA",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1905.06109v1",
            "date": "2019-05-10 12:16:10+00:00",
            "title": "A New Anchor Word Selection Method for the Separable Topic Discovery",
            "authors": [
                "Kun He",
                "Wu Wang",
                "Xiaosen Wang",
                "John E. Hopcroft"
            ],
            "abstract": "Separable Non-negative Matrix Factorization (SNMF) is an important method for\ntopic modeling, where \"separable\" assumes every topic contains at least one\nanchor word, defined as a word that has non-zero probability only on that\ntopic. SNMF focuses on the word co-occurrence patterns to reveal topics by two\nsteps: anchor word selection and topic recovery. The quality of the anchor\nwords strongly influences the quality of the extracted topics. Existing anchor\nword selection algorithm is to greedily find an approximate convex hull in a\nhigh-dimensional word co-occurrence space. In this work, we propose a new\nmethod for the anchor word selection by associating the word co-occurrence\nprobability with the words similarity and assuming that the most different\nwords on semantic are potential candidates for the anchor words. Therefore, if\nthe similarity of a word-pair is very low, then the two words are very likely\nto be the anchor words. According to the statistical information of text\ncorpora, we can get the similarity of all word-pairs. We build the word\nsimilarity graph where the nodes correspond to words and weights on edges stand\nfor the word-pair similarity. Following this way, we design a greedy method to\nfind a minimum edge-weight anchor clique of a given size in the graph for the\nanchor word selection. Extensive experiments on real-world corpus demonstrate\nthe effectiveness of the proposed anchor word selection method that outperforms\nthe common convex hull-based methods on the revealed topic quality. Meanwhile,\nour method is much faster than typical SNMF based method.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.06175v1",
            "date": "2019-05-15 13:37:58+00:00",
            "title": "TSXplain: Demystification of DNN Decisions for Time-Series using Natural Language and Statistical Features",
            "authors": [
                "Mohsin Munir",
                "Shoaib Ahmed Siddiqui",
                "Ferdinand K\u00fcsters",
                "Dominique Mercier",
                "Andreas Dengel",
                "Sheraz Ahmed"
            ],
            "abstract": "Neural networks (NN) are considered as black-boxes due to the lack of\nexplainability and transparency of their decisions. This significantly hampers\ntheir deployment in environments where explainability is essential along with\nthe accuracy of the system. Recently, significant efforts have been made for\nthe interpretability of these deep networks with the aim to open up the\nblack-box. However, most of these approaches are specifically developed for\nvisual modalities. In addition, the interpretations provided by these systems\nrequire expert knowledge and understanding for intelligibility. This indicates\na vital gap between the explainability provided by the systems and the novice\nuser. To bridge this gap, we present a novel framework i.e. Time-Series\neXplanation (TSXplain) system which produces a natural language based\nexplanation of the decision taken by a NN. It uses the extracted statistical\nfeatures to describe the decision of a NN, merging the deep learning world with\nthat of statistics. The two-level explanation provides ample description of the\ndecision made by the network to aid an expert as well as a novice user alike.\nOur survey and reliability assessment test confirm that the generated\nexplanations are meaningful and correct. We believe that generating natural\nlanguage based descriptions of the network's decisions is a big step towards\nopening up the black-box.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.06233v2",
            "date": "2019-05-14 14:06:09+00:00",
            "title": "Generic Encodings of Constructor Rewriting Systems",
            "authors": [
                "Horatiu Cirstea",
                "Pierre-Etienne Moreau"
            ],
            "abstract": "Rewriting is a formalism widely used in computer science and mathematical\nlogic. The classical formalism has been extended, in the context of functional\nlanguages, with an order over the rules and, in the context of rewrite based\nlanguages, with the negation over patterns. We propose in this paper a concise\nand clear algorithm computing the difference over patterns which can be used to\ndefine generic encodings of constructor term rewriting systems with negation\nand order into classical term rewriting systems. As a direct consequence,\nestablished methods used for term rewriting systems can be applied to analyze\nproperties of the extended systems. The approach can also be seen as a generic\ncompiler which targets any language providing basic pattern matching\nprimitives. The formalism provides also a new method for deciding if a set of\npatterns subsumes a given pattern and thus, for checking the presence of\nuseless patterns or the completeness of a set of patterns.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1905.06533v2",
            "date": "2019-05-16 05:40:18+00:00",
            "title": "Articulatory and bottleneck features for speaker-independent ASR of dysarthric speech",
            "authors": [
                "Emre Y\u0131lmaz",
                "Vikramjit Mitra",
                "Ganesh Sivaraman",
                "Horacio Franco"
            ],
            "abstract": "The rapid population aging has stimulated the development of assistive\ndevices that provide personalized medical support to the needies suffering from\nvarious etiologies. One prominent clinical application is a computer-assisted\nspeech training system which enables personalized speech therapy to patients\nimpaired by communicative disorders in the patient's home environment. Such a\nsystem relies on the robust automatic speech recognition (ASR) technology to be\nable to provide accurate articulation feedback. With the long-term aim of\ndeveloping off-the-shelf ASR systems that can be incorporated in clinical\ncontext without prior speaker information, we compare the ASR performance of\nspeaker-independent bottleneck and articulatory features on dysarthric speech\nused in conjunction with dedicated neural network-based acoustic models that\nhave been shown to be robust against spectrotemporal deviations. We report ASR\nperformance of these systems on two dysarthric speech datasets of different\ncharacteristics to quantify the achieved performance gains. Despite the\nremaining performance gap between the dysarthric and normal speech, significant\nimprovements have been reported on both datasets using speaker-independent ASR\narchitectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06597v1",
            "date": "2019-05-16 08:40:47+00:00",
            "title": "A Simple Dual-decoder Model for Generating Response with Sentiment",
            "authors": [
                "Xiuyu Wu",
                "Yunfang Wu"
            ],
            "abstract": "How to generate human like response is one of the most challenging tasks for\nartificial intelligence. In a real application, after reading the same post\ndifferent people might write responses with positive or negative sentiment\naccording to their own experiences and attitudes. To simulate this procedure,\nwe propose a simple but effective dual-decoder model to generate response with\na particular sentiment, by connecting two sentiment decoders to one encoder. To\nsupport this model training, we construct a new conversation dataset with the\nform of (post, resp1, resp2) where two responses contain opposite sentiment.\nExperiment results show that our dual-decoder model can generate diverse\nresponses with target sentiment, which obtains significant performance gain in\nsentiment accuracy and word diversity over the traditional single-decoder\nmodel. We will make our data and code publicly available for further study.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.06638v1",
            "date": "2019-05-16 10:21:51+00:00",
            "title": "Latent Universal Task-Specific BERT",
            "authors": [
                "Alon Rozental",
                "Zohar Kelrich",
                "Daniel Fleischer"
            ],
            "abstract": "This paper describes a language representation model which combines the\nBidirectional Encoder Representations from Transformers (BERT) learning\nmechanism described in Devlin et al. (2018) with a generalization of the\nUniversal Transformer model described in Dehghani et al. (2018). We further\nimprove this model by adding a latent variable that represents the persona and\ntopics of interests of the writer for each training example. We also describe a\nsimple method to improve the usefulness of our language representation for\nsolving problems in a specific domain at the expense of its ability to\ngeneralize to other fields. Finally, we release a pre-trained language\nrepresentation model for social texts that was trained on 100 million tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06655v1",
            "date": "2019-05-16 11:00:49+00:00",
            "title": "Effective Sentence Scoring Method using Bidirectional Language Model for Speech Recognition",
            "authors": [
                "Joongbo Shin",
                "Yoonhyung Lee",
                "Kyomin Jung"
            ],
            "abstract": "In automatic speech recognition, many studies have shown performance\nimprovements using language models (LMs). Recent studies have tried to use\nbidirectional LMs (biLMs) instead of conventional unidirectional LMs (uniLMs)\nfor rescoring the $N$-best list decoded from the acoustic model. In spite of\ntheir theoretical benefits, the biLMs have not given notable improvements\ncompared to the uniLMs in their experiments. This is because their biLMs do not\nconsider the interaction between the two directions. In this paper, we propose\na novel sentence scoring method considering the interaction between the past\nand the future words on the biLM. Our experimental results on the LibriSpeech\ncorpus show that the biLM with the proposed sentence scoring outperforms the\nuniLM for the $N$-best list rescoring, consistently and significantly in all\nexperimental conditions. The analysis of WERs by word position demonstrates\nthat the biLM is more robust than the uniLM especially when a recognized\nsentence is short or a misrecognized word is at the beginning of the sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.06777v1",
            "date": "2019-05-15 09:55:10+00:00",
            "title": "Towards Comparing Programming Paradigms",
            "authors": [
                "Igor Ivkic",
                "Alexander W\u00f6hrer",
                "Markus Tauber"
            ],
            "abstract": "Rapid technological progress in computer sciences finds solutions and at the\nsame time creates ever more complex requirements. Due to an evolving complexity\ntodays programming languages provide powerful frameworks which offer standard\nsolutions for recurring tasks to assist the programmer and to avoid the\nre-invention of the wheel with so-called out-of-the-box-features. In this\npaper, we propose a way of comparing different programming paradigms on a\ntheoretical, technical and practical level. Furthermore, the paper presents the\nresults of an initial comparison of two representative programming approaches,\nboth in the closed SAP environment.",
            "journal ref": "2017 12th International Conference for Internet Technology and\n  Secured Transactions (ICITST), Cambridge, UK",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1905.06791v3",
            "date": "2019-05-13 13:20:57+00:00",
            "title": "Almost Unsupervised Text to Speech and Automatic Speech Recognition",
            "authors": [
                "Yi Ren",
                "Xu Tan",
                "Tao Qin",
                "Sheng Zhao",
                "Zhou Zhao",
                "Tie-Yan Liu"
            ],
            "abstract": "Text to speech (TTS) and automatic speech recognition (ASR) are two dual\ntasks in speech processing and both achieve impressive performance thanks to\nthe recent advance in deep learning and large amount of aligned speech and text\ndata. However, the lack of aligned data poses a major practical problem for TTS\nand ASR on low-resource languages. In this paper, by leveraging the dual nature\nof the two tasks, we propose an almost unsupervised learning method that only\nleverages few hundreds of paired data and extra unpaired data for TTS and ASR.\nOur method consists of the following components: (1) a denoising auto-encoder,\nwhich reconstructs speech and text sequences respectively to develop the\ncapability of language modeling both in speech and text domain; (2) dual\ntransformation, where the TTS model transforms the text $y$ into speech\n$\\hat{x}$, and the ASR model leverages the transformed pair $(\\hat{x},y)$ for\ntraining, and vice versa, to boost the accuracy of the two tasks; (3)\nbidirectional sequence modeling, which addresses error propagation especially\nin the long speech and text sequence when training with few paired data; (4) a\nunified model structure, which combines all the above components for TTS and\nASR based on Transformer model. Our method achieves 99.84% in terms of word\nlevel intelligible rate and 2.68 MOS for TTS, and 11.7% PER for ASR on LJSpeech\ndataset, by leveraging only 200 paired speech and text data (about 20 minutes\naudio), together with extra unpaired speech and text data.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.07075v3",
            "date": "2019-05-17 01:16:15+00:00",
            "title": "Deep Unified Multimodal Embeddings for Understanding both Content and Users in Social Media Networks",
            "authors": [
                "Karan Sikka",
                "Lucas Van Bramer",
                "Ajay Divakaran"
            ],
            "abstract": "There has been an explosion of multimodal content generated on social media\nnetworks in the last few years, which has necessitated a deeper understanding\nof social media content and user behavior. We present a novel\ncontent-independent content-user-reaction model for social multimedia content\nanalysis. Compared to prior works that generally tackle semantic content\nunderstanding and user behavior modeling in isolation, we propose a generalized\nsolution to these problems within a unified framework. We embed users, images\nand text drawn from open social media in a common multimodal geometric space,\nusing a novel loss function designed to cope with distant and disparate\nmodalities, and thereby enable seamless three-way retrieval. Our model not only\noutperforms unimodal embedding based methods on cross-modal retrieval tasks but\nalso shows improvements stemming from jointly solving the two tasks on Twitter\ndata. We also show that the user embeddings learned within our joint multimodal\nembedding model are better at predicting user interests compared to those\nlearned with unimodal content on Instagram data. Our framework thus goes beyond\nthe prior practice of using explicit leader-follower link information to\nestablish affiliations by extracting implicit content-centric affiliations from\nisolated users. We provide qualitative results to show that the user clusters\nemerging from learned embeddings have consistent semantics and the ability of\nour model to discover fine-grained semantics from noisy and unstructured data.\nOur work reveals that social multimodal content is inherently multimodal and\npossesses a consistent structure because in social networks meaning is created\nthrough interactions between users and content.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.07149v3",
            "date": "2019-05-17 07:49:33+00:00",
            "title": "End-to-end Adaptation with Backpropagation through WFST for On-device Speech Recognition System",
            "authors": [
                "Emiru Tsunoo",
                "Yosuke Kashiwagi",
                "Satoshi Asakawa",
                "Toshiyuki Kumakura"
            ],
            "abstract": "An on-device DNN-HMM speech recognition system efficiently works with a\nlimited vocabulary in the presence of a variety of predictable noise. In such a\ncase, vocabulary and environment adaptation is highly effective. In this paper,\nwe propose a novel method of end-to-end (E2E) adaptation, which adjusts not\nonly an acoustic model (AM) but also a weighted finite-state transducer (WFST).\nWe convert a pretrained WFST to a trainable neural network and adapt the system\nto target environments/vocabulary by E2E joint training with an AM. We\nreplicate Viterbi decoding with forward--backward neural network computation,\nwhich is similar to recurrent neural networks (RNNs). By pooling output score\nsequences, a vocabulary posterior for each utterance is obtained and used for\ndiscriminative loss computation. Experiments using 2--10 hours of\nEnglish/Japanese adaptation datasets indicate that the fine-tuning of only\nWFSTs and that of only AMs are both comparable to a state-of-the-art adaptation\nmethod, and E2E joint training of the two components achieves the best\nrecognition performance. We also adapt each language system to the other\nlanguage using the adaptation data, and the results show that the proposed\nmethod also works well for language adaptations.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.07189v2",
            "date": "2019-05-17 10:49:47+00:00",
            "title": "Distant Learning for Entity Linking with Automatic Noise Detection",
            "authors": [
                "Phong Le",
                "Ivan Titov"
            ],
            "abstract": "Accurate entity linkers have been produced for domains and languages where\nannotated data (i.e., texts linked to a knowledge base) is available. However,\nlittle progress has been made for the settings where no or very limited amounts\nof labeled data are present (e.g., legal or most scientific domains). In this\nwork, we show how we can learn to link mentions without having any labeled\nexamples, only a knowledge base and a collection of unannotated texts from the\ncorresponding domain. In order to achieve this, we frame the task as a\nmulti-instance learning problem and rely on surface matching to create initial\nnoisy labels. As the learning signal is weak and our surrogate labels are\nnoisy, we introduce a noise detection component in our model: it lets the model\ndetect and disregard examples which are likely to be noisy. Our method, jointly\nlearning to detect noise and link entities, greatly outperforms the surface\nmatching baseline. For a subset of entity categories, it even approaches the\nperformance of supervised learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07195v2",
            "date": "2019-05-17 11:03:58+00:00",
            "title": "CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Network",
            "authors": [
                "Vincent Wan",
                "Chun-an Chan",
                "Tom Kenter",
                "Jakub Vit",
                "Rob Clark"
            ],
            "abstract": "The prosodic aspects of speech signals produced by current text-to-speech\nsystems are typically averaged over training material, and as such lack the\nvariety and liveliness found in natural speech. To avoid monotony and averaged\nprosody contours, it is desirable to have a way of modeling the variation in\nthe prosodic aspects of speech, so audio signals can be synthesized in multiple\nways for a given text. We present a new, hierarchically structured conditional\nvariational autoencoder to generate prosodic features (fundamental frequency,\nenergy and duration) suitable for use with a vocoder or a generative model like\nWaveNet. At inference time, an embedding representing the prosody of a sentence\nmay be sampled from the variational layer to allow for prosodic variation. To\nefficiently capture the hierarchical nature of the linguistic input (words,\nsyllables and phones), both the encoder and decoder parts of the auto-encoder\nare hierarchical, in line with the linguistic structure, with layers being\nclocked dynamically at the respective rates. We show in our experiments that\nour dynamic hierarchical network outperforms a non-hierarchical\nstate-of-the-art baseline, and, additionally, that prosody transfer across\nsentences is possible by employing the prosody embedding of one sentence to\ngenerate the speech signal of another.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07356v1",
            "date": "2019-05-17 16:26:05+00:00",
            "title": "Don't Blame Distributional Semantics if it can't do Entailment",
            "authors": [
                "Matthijs Westera",
                "Gemma Boleda"
            ],
            "abstract": "Distributional semantics has had enormous empirical success in Computational\nLinguistics and Cognitive Science in modeling various semantic phenomena, such\nas semantic similarity, and distributional models are widely used in\nstate-of-the-art Natural Language Processing systems. However, the theoretical\nstatus of distributional semantics within a broader theory of language and\ncognition is still unclear: What does distributional semantics model? Can it\nbe, on its own, a fully adequate model of the meanings of linguistic\nexpressions? The standard answer is that distributional semantics is not fully\nadequate in this regard, because it falls short on some of the central aspects\nof formal semantic approaches: truth conditions, entailment, reference, and\ncertain aspects of compositionality. We argue that this standard answer rests\non a misconception: These aspects do not belong in a theory of expression\nmeaning, they are instead aspects of speaker meaning, i.e., communicative\nintentions in a particular context. In a slogan: words do not refer, speakers\ndo. Clearing this up enables us to argue that distributional semantics on its\nown is an adequate model of expression meaning. Our proposal sheds light on the\nrole of distributional semantics in a broader theory of language and cognition,\nits relationship to formal semantics, and its place in computational models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07408v3",
            "date": "2019-05-17 15:23:39+00:00",
            "title": "Functorial Question Answering",
            "authors": [
                "Giovanni de Felice",
                "Konstantinos Meichanetzidis",
                "Alexis Toumi"
            ],
            "abstract": "Distributional compositional (DisCo) models are functors that compute the\nmeaning of a sentence from the meaning of its words. We show that DisCo models\nin the category of sets and relations correspond precisely to relational\ndatabases. As a consequence, we get complexity-theoretic reductions from\nsemantics and entailment of a fragment of natural language to evaluation and\ncontainment of conjunctive queries, respectively. Finally, we define question\nanswering as an NP-complete problem.",
            "journal ref": "EPTCS 323, 2020, pp. 84-94",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07471v1",
            "date": "2019-05-15 19:02:35+00:00",
            "title": "Learning Open Information Extraction of Implicit Relations from Reading Comprehension Datasets",
            "authors": [
                "Jacob Beckerman",
                "Theodore Christakis"
            ],
            "abstract": "The relationship between two entities in a sentence is often implied by word\norder and common sense, rather than an explicit predicate. For example, it is\nevident that \"Fed chair Powell indicates rate hike\" implies (Powell, is a, Fed\nchair) and (Powell, works for, Fed). These tuples are just as significant as\nthe explicit-predicate tuple (Powell, indicates, rate hike), but have much\nlower recall under traditional Open Information Extraction (OpenIE) systems.\nImplicit tuples are our term for this type of extraction where the relation is\nnot present in the input sentence. There is very little OpenIE training data\navailable relative to other NLP tasks and none focused on implicit relations.\nWe develop an open source, parse-based tool for converting large reading\ncomprehension datasets to OpenIE datasets and release a dataset 35x larger than\npreviously available by sentence count. A baseline neural model trained on this\ndata outperforms previous methods on the implicit extraction task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.07508v1",
            "date": "2019-05-18 00:28:37+00:00",
            "title": "Cross-referencing using Fine-grained Topic Modeling",
            "authors": [
                "Jeffrey Lund",
                "Piper Armstrong",
                "Wilson Fearn",
                "Stephen Cowley",
                "Emily Hales",
                "Kevin Seppi"
            ],
            "abstract": "Cross-referencing, which links passages of text to other related passages,\ncan be a valuable study aid for facilitating comprehension of a text. However,\ncross-referencing requires first, a comprehensive thematic knowledge of the\nentire corpus, and second, a focused search through the corpus specifically to\nfind such useful connections. Due to this, cross-reference resources are\nprohibitively expensive and exist only for the most well-studied texts (e.g.\nreligious texts). We develop a topic-based system for automatically producing\ncandidate cross-references which can be easily verified by human annotators.\nOur system utilizes fine-grained topic modeling with thousands of highly\nnuanced and specific topics to identify verse pairs which are topically\nrelated. We demonstrate that our system can be cost effective compared to\nhaving annotators acquire the expertise necessary to produce cross-reference\nresources unaided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07562v2",
            "date": "2019-05-18 09:23:00+00:00",
            "title": "Human-like machine thinking: Language guided imagination",
            "authors": [
                "Feng Qi",
                "Wenchuan Wu"
            ],
            "abstract": "Human thinking requires the brain to understand the meaning of language\nexpression and to properly organize the thoughts flow using the language.\nHowever, current natural language processing models are primarily limited in\nthe word probability estimation. Here, we proposed a Language guided\nimagination (LGI) network to incrementally learn the meaning and usage of\nnumerous words and syntaxes, aiming to form a human-like machine thinking\nprocess. LGI contains three subsystems: (1) vision system that contains an\nencoder to disentangle the input or imagined scenarios into abstract population\nrepresentations, and an imagination decoder to reconstruct imagined scenario\nfrom higher level representations; (2) Language system, that contains a\nbinarizer to transfer symbol texts into binary vectors, an IPS (mimicking the\nhuman IntraParietal Sulcus, implemented by an LSTM) to extract the quantity\ninformation from the input texts, and a textizer to convert binary vectors into\ntext symbols; (3) a PFC (mimicking the human PreFrontal Cortex, implemented by\nan LSTM) to combine inputs of both language and vision representations, and\npredict text symbols and manipulated images accordingly. LGI has incrementally\nlearned eight different syntaxes (or tasks), with which a machine thinking loop\nhas been formed and validated by the proper interaction between language and\nvision system. The paper provides a new architecture to let the machine learn,\nunderstand and use language in a human-like way that could ultimately enable a\nmachine to construct fictitious 'mental' scenario and possess intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07790v1",
            "date": "2019-05-19 18:23:14+00:00",
            "title": "Correlation Coefficients and Semantic Textual Similarity",
            "authors": [
                "Vitalii Zhelezniak",
                "Aleksandar Savkov",
                "April Shen",
                "Nils Y. Hammerla"
            ],
            "abstract": "A large body of research into semantic textual similarity has focused on\nconstructing state-of-the-art embeddings using sophisticated modelling, careful\nchoice of learning signals and many clever tricks. By contrast, little\nattention has been devoted to similarity measures between these embeddings,\nwith cosine similarity being used unquestionably in the majority of cases. In\nthis work, we illustrate that for all common word vectors, cosine similarity is\nessentially equivalent to the Pearson correlation coefficient, which provides\nsome justification for its use. We thoroughly characterise cases where Pearson\ncorrelation (and thus cosine similarity) is unfit as similarity measure.\nImportantly, we show that Pearson correlation is appropriate for some word\nvectors but not others. When it is not appropriate, we illustrate how common\nnon-parametric rank correlation coefficients can be used instead to\nsignificantly improve performance. We support our analysis with a series of\nevaluations on word-level and sentence-level semantic textual similarity\nbenchmarks. On the latter, we show that even the simplest averaged word vectors\ncompared by rank correlation easily rival the strongest deep representations\ncompared by cosine similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07870v4",
            "date": "2019-05-20 04:41:10+00:00",
            "title": "PaperRobot: Incremental Draft Generation of Scientific Ideas",
            "authors": [
                "Qingyun Wang",
                "Lifu Huang",
                "Zhiying Jiang",
                "Kevin Knight",
                "Heng Ji",
                "Mohit Bansal",
                "Yi Luan"
            ],
            "abstract": "We present a PaperRobot who performs as an automatic research assistant by\n(1) conducting deep understanding of a large collection of human-written papers\nin a target domain and constructing comprehensive background knowledge graphs\n(KGs); (2) creating new ideas by predicting links from the background KGs, by\ncombining graph attention and contextual text attention; (3) incrementally\nwriting some key elements of a new paper based on memory-attention networks:\nfrom the input title along with predicted related entities to generate a paper\nabstract, from the abstract to generate conclusion and future work, and finally\nfrom future work to generate a title for a follow-on paper. Turing Tests, where\na biomedical domain expert is asked to compare a system output and a\nhuman-authored string, show PaperRobot generated abstracts, conclusion and\nfuture work sections, and new titles are chosen over human-written ones up to\n30%, 24% and 12% of the time, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.07894v1",
            "date": "2019-05-20 06:15:07+00:00",
            "title": "Abusive Language Detection in Online Conversations by Combining Content-and Graph-based Features",
            "authors": [
                "No\u00e9 Cecillon",
                "Vincent Labatut",
                "Richard Dufour",
                "Georges Linar\u00e8s"
            ],
            "abstract": "In recent years, online social networks have allowed worldwide users to meet\nand discuss. As guarantors of these communities, the administrators of these\nplatforms must prevent users from adopting inappropriate behaviors. This\nverification task, mainly done by humans, is more and more difficult due to the\never growing amount of messages to check. Methods have been proposed to\nautomatize this moderation process, mainly by providing approaches based on the\ntextual content of the exchanged messages. Recent work has also shown that\ncharacteristics derived from the structure of conversations, in the form of\nconversational graphs, can help detecting these abusive messages. In this\npaper, we propose to take advantage of both sources of information by proposing\nfusion methods integrating content-and graph-based features. Our experiments on\nraw chat logs show that the content of the messages, but also of their dynamics\nwithin a conversation contain partially complementary information, allowing\nperformance improvements on an abusive message classification task with a final\nF-measure of 93.26%.",
            "journal ref": "ICWSM International Workshop on Modeling and mining\n  Social-Media-driven Complex Networks (Soc2Net) 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.08063v1",
            "date": "2019-05-17 10:14:07+00:00",
            "title": "The Unexpected Unexpected and the Expected Unexpected: How People's Conception of the Unexpected is Not That Unexpected",
            "authors": [
                "Molly S Quinn",
                "Kathleen Campbell",
                "Mark T Keane"
            ],
            "abstract": "The answers people give when asked to 'think of the unexpected' for everyday\nevent scenarios appear to be more expected than unexpected. There are expected\nunexpected outcomes that closely adhere to the given information in a scenario,\nbased on familiar disruptions and common plan-failures. There are also\nunexpected unexpected outcomes that are more inventive, that depart from given\ninformation, adding new concepts/actions. However, people seem to tend to\nconceive of the unexpected as the former more than the latter. Study 1 tests\nthese proposals by analysing the object-concepts people mention in their\nreports of the unexpected and the agreement between their answers. Study 2\nshows that object-choices are weakly influenced by recency, the order of\nsentences in the scenario. The implications of these results for ideas in\nphilosophy, psychology and computing is discussed",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08110v1",
            "date": "2019-05-20 13:43:52+00:00",
            "title": "Image Captioning based on Deep Learning Methods: A Survey",
            "authors": [
                "Yiyu Wang",
                "Jungang Xu",
                "Yingfei Sun",
                "Ben He"
            ],
            "abstract": "Image captioning is a challenging task and attracting more and more attention\nin the field of Artificial Intelligence, and which can be applied to efficient\nimage retrieval, intelligent blind guidance and human-computer interaction,\netc. In this paper, we present a survey on advances in image captioning based\non Deep Learning methods, including Encoder-Decoder structure, improved methods\nin Encoder, improved methods in Decoder, and other improvements. Furthermore,\nwe discussed future research directions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.08300v1",
            "date": "2019-05-20 19:05:12+00:00",
            "title": "A Neural Network Architecture for Learning Word-Referent Associations in Multiple Contexts",
            "authors": [
                "Hansenclever F. Bassani",
                "Aluizio F. R. Araujo"
            ],
            "abstract": "This article proposes a biologically inspired neurocomputational architecture\nwhich learns associations between words and referents in different contexts,\nconsidering evidence collected from the literature of Psycholinguistics and\nNeurolinguistics. The multi-layered architecture takes as input raw images of\nobjects (referents) and streams of word's phonemes (labels), builds an adequate\nrepresentation, recognizes the current context, and associates label with\nreferents incrementally, by employing a Self-Organizing Map which creates new\nassociation nodes (prototypes) as required, adjusts the existing prototypes to\nbetter represent the input stimuli and removes prototypes that become\nobsolete/unused. The model takes into account the current context to retrieve\nthe correct meaning of words with multiple meanings. Simulations show that the\nmodel can reach up to 78% of word-referent association accuracy in ambiguous\nsituations and approximates well the learning rates of humans as reported by\nthree different authors in five Cross-Situational Word Learning experiments,\nalso displaying similar learning patterns in the different learning conditions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.08392v1",
            "date": "2019-05-21 00:32:21+00:00",
            "title": "A Causality-Guided Prediction of the TED Talk Ratings from the Speech-Transcripts using Neural Networks",
            "authors": [
                "Md Iftekhar Tanveer",
                "Md Kamrul Hasan",
                "Daniel Gildea",
                "M. Ehsan Hoque"
            ],
            "abstract": "Automated prediction of public speaking performance enables novel systems for\ntutoring public speaking skills. We use the largest open repository---TED\nTalks---to predict the ratings provided by the online viewers. The dataset\ncontains over 2200 talk transcripts and the associated meta information\nincluding over 5.5 million ratings from spontaneous visitors to the website. We\ncarefully removed the bias present in the dataset (e.g., the speakers'\nreputations, popularity gained by publicity, etc.) by modeling the data\ngenerating process using a causal diagram. We use a word sequence based\nrecurrent architecture and a dependency tree based recursive architecture as\nthe neural networks for predicting the TED talk ratings. Our neural network\nmodels can predict the ratings with an average F-score of 0.77 which largely\noutperforms the competitive baseline method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.08459v3",
            "date": "2019-05-21 06:36:15+00:00",
            "title": "Non-Autoregressive Neural Text-to-Speech",
            "authors": [
                "Kainan Peng",
                "Wei Ping",
                "Zhao Song",
                "Kexin Zhao"
            ],
            "abstract": "In this work, we propose ParaNet, a non-autoregressive seq2seq model that\nconverts text to spectrogram. It is fully convolutional and brings 46.7 times\nspeed-up over the lightweight Deep Voice 3 at synthesis, while obtaining\nreasonably good speech quality. ParaNet also produces stable alignment between\ntext and speech on the challenging test sentences by iteratively improving the\nattention in a layer-by-layer manner. Furthermore, we build the parallel\ntext-to-speech system and test various parallel neural vocoders, which can\nsynthesize speech from text through a single feed-forward pass. We also explore\na novel VAE-based approach to train the inverse autoregressive flow (IAF) based\nparallel vocoder from scratch, which avoids the need for distillation from a\nseparately trained WaveNet as previous work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08527v1",
            "date": "2019-05-21 10:14:12+00:00",
            "title": "CNNs found to jump around more skillfully than RNNs: Compositional generalization in seq2seq convolutional networks",
            "authors": [
                "Roberto Dess\u00ec",
                "Marco Baroni"
            ],
            "abstract": "Lake and Baroni (2018) introduced the SCAN dataset probing the ability of\nseq2seq models to capture compositional generalizations, such as inferring the\nmeaning of \"jump around\" 0-shot from the component words. Recurrent networks\n(RNNs) were found to completely fail the most challenging generalization cases.\nWe test here a convolutional network (CNN) on these tasks, reporting hugely\nimproved performance with respect to RNNs. Despite the big improvement, the CNN\nhas however not induced systematic rules, suggesting that the difference\nbetween compositional and non-compositional behaviour is not clear-cut.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08622v3",
            "date": "2019-05-18 13:58:12+00:00",
            "title": "Variational Hetero-Encoder Randomized GANs for Joint Image-Text Modeling",
            "authors": [
                "Hao Zhang",
                "Bo Chen",
                "Long Tian",
                "Zhengjue Wang",
                "Mingyuan Zhou"
            ],
            "abstract": "For bidirectional joint image-text modeling, we develop variational\nhetero-encoder (VHE) randomized generative adversarial network (GAN), a\nversatile deep generative model that integrates a probabilistic text decoder,\nprobabilistic image encoder, and GAN into a coherent end-to-end multi-modality\nlearning framework. VHE randomized GAN (VHE-GAN) encodes an image to decode its\nassociated text, and feeds the variational posterior as the source of\nrandomness into the GAN image generator. We plug three off-the-shelf modules,\nincluding a deep topic model, a ladder-structured image encoder, and\nStackGAN++, into VHE-GAN, which already achieves competitive performance. This\nfurther motivates the development of VHE-raster-scan-GAN that generates\nphoto-realistic images in not only a multi-scale low-to-high-resolution manner,\nbut also a hierarchical-semantic coarse-to-fine fashion. By capturing and\nrelating hierarchical semantic and visual concepts with end-to-end training,\nVHE-raster-scan-GAN achieves state-of-the-art performance in a wide variety of\nimage-text multi-modality learning and generation tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.08701v3",
            "date": "2019-05-21 15:36:54+00:00",
            "title": "Approximating probabilistic models as weighted finite automata",
            "authors": [
                "Ananda Theertha Suresh",
                "Brian Roark",
                "Michael Riley",
                "Vlad Schogol"
            ],
            "abstract": "Weighted finite automata (WFA) are often used to represent probabilistic\nmodels, such as $n$-gram language models, since they are efficient for\nrecognition tasks in time and space. The probabilistic source to be represented\nas a WFA, however, may come in many forms. Given a generic probabilistic model\nover sequences, we propose an algorithm to approximate it as a weighted finite\nautomaton such that the Kullback-Leiber divergence between the source model and\nthe WFA target model is minimized. The proposed algorithm involves a counting\nstep and a difference of convex optimization step, both of which can be\nperformed efficiently. We demonstrate the usefulness of our approach on various\ntasks, including distilling $n$-gram models from neural models, building\ncompact language models, and building open-vocabulary character models. The\nalgorithms used for these experiments are available in an open-source software\nlibrary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08732v1",
            "date": "2019-05-21 16:22:47+00:00",
            "title": "A realistic and robust model for Chinese word segmentation",
            "authors": [
                "Chu-Ren Huang",
                "Ting-Shuo Yo",
                "Petr Simon",
                "Shu-Kai Hsieh"
            ],
            "abstract": "A realistic Chinese word segmentation tool must adapt to textual variations\nwith minimal training input and yet robust enough to yield reliable\nsegmentation result for all variants. Various lexicon-driven approaches to\nChinese segmentation, e.g. [1,16], achieve high f-scores yet require massive\ntraining for any variation. Text-driven approach, e.g. [12], can be easily\nadapted for domain and genre changes yet has difficulty matching the high\nf-scores of the lexicon-driven approaches. In this paper, we refine and\nimplement an innovative text-driven word boundary decision (WBD) segmentation\nmodel proposed in [15]. The WBD model treats word segmentation simply and\nefficiently as a binary decision on whether to realize the natural textual\nbreak between two adjacent characters as a word boundary. The WBD model allows\nsimple and quick training data preparation converting characters as contextual\nvectors for learning the word boundary decision. Machine learning experiments\nwith four different classifiers show that training with 1,000 vectors and 1\nmillion vectors achieve comparable and reliable results. In addition, when\napplied to SigHAN Bakeoff 3 competition data, the WBD model produces OOV recall\nrates that are higher than all published results. Unlike all previous work, our\nOOV recall rate is comparable to our own F-score. Both experiments support the\nclaim that the WBD model is a realistic model for Chinese word segmentation as\nit can be easily adapted for new variants with the robust result. In\nconclusion, we will discuss linguistic ramifications as well as future\nimplications for the WBD approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08760v1",
            "date": "2019-05-21 17:26:39+00:00",
            "title": "Sampling from Stochastic Finite Automata with Applications to CTC Decoding",
            "authors": [
                "Martin Jansche",
                "Alexander Gutkin"
            ],
            "abstract": "Stochastic finite automata arise naturally in many language and speech\nprocessing tasks. They include stochastic acceptors, which represent certain\nprobability distributions over random strings. We consider the problem of\nefficient sampling: drawing random string variates from the probability\ndistribution represented by stochastic automata and transformations of those.\nWe show that path-sampling is effective and can be efficient if the\nepsilon-graph of a finite automaton is acyclic. We provide an algorithm that\nensures this by conflating epsilon-cycles within strongly connected components.\nSampling is also effective in the presence of non-injective transformations of\nstrings. We illustrate this in the context of decoding for Connectionist\nTemporal Classification (CTC), where the predictive probabilities yield\nauxiliary sequences which are transformed into shorter labeling strings. We can\nsample efficiently from the transformed labeling distribution and use this in\ntwo different strategies for finding the most probable CTC labeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08796v1",
            "date": "2019-05-21 15:44:03+00:00",
            "title": "Acoustic-to-Word Models with Conversational Context Information",
            "authors": [
                "Suyoun Kim",
                "Florian Metze"
            ],
            "abstract": "Conversational context information, higher-level knowledge that spans across\nsentences, can help to recognize a long conversation. However, existing speech\nrecognition models are typically built at a sentence level, and thus it may not\ncapture important conversational context information. The recent progress in\nend-to-end speech recognition enables integrating context with other available\ninformation (e.g., acoustic, linguistic resources) and directly recognizing\nwords from speech. In this work, we present a direct acoustic-to-word,\nend-to-end speech recognition model capable of utilizing the conversational\ncontext to better process long conversations. We evaluate our proposed approach\non the Switchboard conversational speech corpus and show that our system\noutperforms a standard end-to-end speech recognition system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08920v1",
            "date": "2019-05-21 10:33:06+00:00",
            "title": "Domain adaptation for part-of-speech tagging of noisy user-generated text",
            "authors": [
                "Luisa M\u00e4rz",
                "Dietrich Trautmann",
                "Benjamin Roth"
            ],
            "abstract": "The performance of a Part-of-speech (POS) tagger is highly dependent on the\ndomain ofthe processed text, and for many domains there is no or only very\nlittle training data available. This work addresses the problem of POS tagging\nnoisy user-generated text using a neural network. We propose an architecture\nthat trains an out-of-domain model on a large newswire corpus, and transfers\nthose weights by using them as a prior for a model trained on the target domain\n(a data-set of German Tweets) for which there is very little an-notations\navailable. The neural network has two standard bidirectional LSTMs at its core.\nHowever, we find it crucial to also encode a set of task-specific features, and\nto obtain reliable (source-domain and target-domain) word representations.\nExperiments with different regularization techniques such as early stopping,\ndropout and fine-tuning the domain adaptation prior weights are conducted. Our\nbest model uses external weights from the out-of-domain model, as well as\nfeature embeddings, pre-trained word and sub-word embeddings and achieves a\ntagging accuracy of slightly over 90%, improving on the previous state of the\nart for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.08937v2",
            "date": "2019-05-22 03:34:59+00:00",
            "title": "Can a Humanoid Robot be part of the Organizational Workforce? A User Study Leveraging Sentiment Analysis",
            "authors": [
                "Nidhi Mishra",
                "Manoj Ramanathan",
                "Ranjan Satapathy",
                "Erik Cambria",
                "Nadia Magnenat-Thalmann"
            ],
            "abstract": "Hiring robots for the workplaces is a challenging task as robots have to\ncater to customer demands, follow organizational protocols and behave with\nsocial etiquette. In this study, we propose to have a humanoid social robot,\nNadine, as a customer service agent in an open social work environment. The\nobjective of this study is to analyze the effects of humanoid robots on\ncustomers at work environment, and see if it can handle social scenarios. We\npropose to evaluate these objectives through two modes, namely, survey\nquestionnaire and customer feedback. We also propose a novel approach to\nanalyze customer feedback data (text) using sentic computing methods.\nSpecifically, we employ aspect extraction and sentiment analysis to analyze the\ndata. From our framework, we detect sentiment associated to the aspects that\nmainly concerned the customers during their interaction. This allows us to\nunderstand customers expectations and current limitations of robots as\nemployees.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1905.09086v1",
            "date": "2019-05-22 11:49:37+00:00",
            "title": "From web crawled text to project descriptions: automatic summarizing of social innovation projects",
            "authors": [
                "Nikola Milosevic",
                "Dimitar Marinov",
                "Abdullah Gok",
                "Goran Nenadic"
            ],
            "abstract": "In the past decade, social innovation projects have gained the attention of\npolicy makers, as they address important social issues in an innovative manner.\nA database of social innovation is an important source of information that can\nexpand collaboration between social innovators, drive policy and serve as an\nimportant resource for research. Such a database needs to have projects\ndescribed and summarized. In this paper, we propose and compare several methods\n(e.g. SVM-based, recurrent neural network based, ensambled) for describing\nprojects based on the text that is available on project websites. We also\naddress and propose a new metric for automated evaluation of summaries based on\ntopic modelling.",
            "journal ref": "Preceeding of 24th International Conference on Applications of\n  Natural Language to Information Systems (NLDB2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09153v1",
            "date": "2019-05-22 14:11:30+00:00",
            "title": "Simplified Neural Unsupervised Domain Adaptation",
            "authors": [
                "Timothy A Miller"
            ],
            "abstract": "Unsupervised domain adaptation (UDA) is the task of modifying a statistical\nmodel trained on labeled data from a source domain to achieve better\nperformance on data from a target domain, with access to only unlabeled data in\nthe target domain. Existing state-of-the-art UDA approaches use neural networks\nto learn representations that can predict the values of subset of important\nfeatures called \"pivot features.\" In this work, we show that it is possible to\nimprove on these methods by jointly training the representation learner with\nthe task learner, and examine the importance of existing pivot selection\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09263v5",
            "date": "2019-05-22 17:50:21+00:00",
            "title": "FastSpeech: Fast, Robust and Controllable Text to Speech",
            "authors": [
                "Yi Ren",
                "Yangjun Ruan",
                "Xu Tan",
                "Tao Qin",
                "Sheng Zhao",
                "Zhou Zhao",
                "Tie-Yan Liu"
            ],
            "abstract": "Neural network based end-to-end text to speech (TTS) has significantly\nimproved the quality of synthesized speech. Prominent methods (e.g., Tacotron\n2) usually first generate mel-spectrogram from text, and then synthesize speech\nfrom the mel-spectrogram using vocoder such as WaveNet. Compared with\ntraditional concatenative and statistical parametric approaches, neural network\nbased end-to-end models suffer from slow inference speed, and the synthesized\nspeech is usually not robust (i.e., some words are skipped or repeated) and\nlack of controllability (voice speed or prosody control). In this work, we\npropose a novel feed-forward network based on Transformer to generate\nmel-spectrogram in parallel for TTS. Specifically, we extract attention\nalignments from an encoder-decoder based teacher model for phoneme duration\nprediction, which is used by a length regulator to expand the source phoneme\nsequence to match the length of the target mel-spectrogram sequence for\nparallel mel-spectrogram generation. Experiments on the LJSpeech dataset show\nthat our parallel model matches autoregressive models in terms of speech\nquality, nearly eliminates the problem of word skipping and repeating in\nparticularly hard cases, and can adjust voice speed smoothly. Most importantly,\ncompared with autoregressive Transformer TTS, our model speeds up\nmel-spectrogram generation by 270x and the end-to-end speech synthesis by 38x.\nTherefore, we call our model FastSpeech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09438v1",
            "date": "2019-05-23 02:32:34+00:00",
            "title": "Multi-hop Reading Comprehension via Deep Reinforcement Learning based Document Traversal",
            "authors": [
                "Alex Long",
                "Joel Mason",
                "Alan Blair",
                "Wei Wang"
            ],
            "abstract": "Reading Comprehension has received significant attention in recent years as\nhigh quality Question Answering (QA) datasets have become available. Despite\nstate-of-the-art methods achieving strong overall accuracy, Multi-Hop (MH)\nreasoning remains particularly challenging. To address MH-QA specifically, we\npropose a Deep Reinforcement Learning based method capable of learning\nsequential reasoning across large collections of documents so as to pass a\nquery-aware, fixed-size context subset to existing models for answer\nextraction. Our method is comprised of two stages: a linker, which decomposes\nthe provided support documents into a graph of sentences, and an extractor,\nwhich learns where to look based on the current question and already-visited\nsentences. The result of the linker is a novel graph structure at the sentence\nlevel that preserves logical flow while still allowing rapid movement between\ndocuments. Importantly, we demonstrate that the sparsity of the resultant graph\nis invariant to context size. This translates to fewer decisions required from\nthe Deep-RL trained extractor, allowing the system to scale effectively to\nlarge collections of documents.\n  The importance of sequential decision making in the document traversal step\nis demonstrated by comparison to standard IE methods, and we additionally\nintroduce a BM25-based IR baseline that retrieves documents relevant to the\nquery only. We examine the integration of our method with existing models on\nthe recently proposed QAngaroo benchmark and achieve consistent increases in\naccuracy across the board, as well as a 2-3x reduction in training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09761v1",
            "date": "2019-05-23 16:33:30+00:00",
            "title": "An Efficient Approach for Super and Nested Term Indexing and Retrieval",
            "authors": [
                "Md Faisal Mahbub Chowdhury",
                "Robert Farrell"
            ],
            "abstract": "This paper describes a new approach, called Terminological Bucket Indexing\n(TBI), for efficient indexing and retrieval of both nested and super terms\nusing a single method. We propose a hybrid data structure for facilitating\nfaster indexing building. An evaluation of our approach with respect to widely\nused existing approaches on several publicly available dataset is provided.\nCompared to Trie based approaches, TBI provides comparable performance on\nnested term retrieval and far superior performance on super term retrieval.\nCompared to traditional hash table, TBI needs 80\\% less time for indexing.",
            "journal ref": "None",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/1905.09856v1",
            "date": "2019-05-23 18:25:35+00:00",
            "title": "Copy this Sentence",
            "authors": [
                "Vasileios Lioutas",
                "Andriy Drozdyuk"
            ],
            "abstract": "Attention is an operation that selects some largest element from some set,\nwhere the notion of largest is defined elsewhere. Applying this operation to\nsequence to sequence mapping results in significant improvements to the task at\nhand. In this paper we provide the mathematical definition of attention and\nexamine its application to sequence to sequence models. We highlight the exact\ncorrespondences between machine learning implementations of attention and our\nmathematical definition. We provide clear evidence of effectiveness of\nattention mechanisms evaluating models with varying degrees of attention on a\nvery simple task: copying a sentence. We find that models that make greater use\nof attention perform much better on sequence to sequence mapping tasks,\nconverge faster and are more stable.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.09864v2",
            "date": "2019-05-23 18:40:57+00:00",
            "title": "Why Didn't You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models",
            "authors": [
                "Varun Kumar",
                "Alison Smith-Renner",
                "Leah Findlater",
                "Kevin Seppi",
                "Jordan Boyd-Graber"
            ],
            "abstract": "To address the lack of comparative evaluation of Human-in-the-Loop Topic\nModeling (HLTM) systems, we implement and evaluate three contrasting HLTM\nmodeling approaches using simulation experiments. These approaches extend\npreviously proposed frameworks, including constraints and informed prior-based\nmethods. Users should have a sense of control in HLTM systems, so we propose a\ncontrol metric to measure whether refinement operations' results match users'\nexpectations. Informed prior-based methods provide better control than\nconstraints, but constraints yield higher quality topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.09922v2",
            "date": "2019-05-23 21:01:24+00:00",
            "title": "Training language GANs from Scratch",
            "authors": [
                "Cyprien de Masson d'Autume",
                "Mihaela Rosca",
                "Jack Rae",
                "Shakir Mohamed"
            ],
            "abstract": "Generative Adversarial Networks (GANs) enjoy great success at image\ngeneration, but have proven difficult to train in the domain of natural\nlanguage. Challenges with gradient estimation, optimization instability, and\nmode collapse have lead practitioners to resort to maximum likelihood\npre-training, followed by small amounts of adversarial fine-tuning. The\nbenefits of GAN fine-tuning for language generation are unclear, as the\nresulting models produce comparable or worse samples than traditional language\nmodels. We show it is in fact possible to train a language GAN from scratch --\nwithout maximum likelihood pre-training. We combine existing techniques such as\nlarge batch sizes, dense rewards and discriminator regularization to stabilize\nand improve language GANs. The resulting model, ScratchGAN, performs comparably\nto maximum likelihood training on EMNLP2017 News and WikiText-103 corpora\naccording to quality and diversity metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10412v1",
            "date": "2019-05-24 19:10:18+00:00",
            "title": "Using Deep Networks and Transfer Learning to Address Disinformation",
            "authors": [
                "Numa Dhamani",
                "Paul Azunre",
                "Jeffrey L. Gleason",
                "Craig Corcoran",
                "Garrett Honke",
                "Steve Kramer",
                "Jonathon Morgan"
            ],
            "abstract": "We apply an ensemble pipeline composed of a character-level convolutional\nneural network (CNN) and a long short-term memory (LSTM) as a general tool for\naddressing a range of disinformation problems. We also demonstrate the ability\nto use this architecture to transfer knowledge from labeled data in one domain\nto related (supervised and unsupervised) tasks. Character-level neural networks\nand transfer learning are particularly valuable tools in the disinformation\nspace because of the messy nature of social media, lack of labeled data, and\nthe multi-channel tactics of influence campaigns. We demonstrate their\neffectiveness in several tasks relevant for detecting disinformation: spam\nemails, review bombing, political sentiment, and conversation clustering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10417v1",
            "date": "2019-05-24 19:20:03+00:00",
            "title": "Differentiable Representations For Multihop Inference Rules",
            "authors": [
                "William W. Cohen",
                "Haitian Sun",
                "R. Alex Hofer",
                "Matthew Siegler"
            ],
            "abstract": "We present efficient differentiable implementations of second-order multi-hop\nreasoning using a large symbolic knowledge base (KB). We introduce a new\noperation which can be used to compositionally construct second-order multi-hop\ntemplates in a neural model, and evaluate a number of alternative\nimplementations, with different time and memory trade offs. These techniques\nscale to KBs with millions of entities and tens of millions of triples, and\nlead to simple models with competitive performance on several learning tasks\nrequiring multi-hop reasoning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10617v8",
            "date": "2019-05-25 15:34:43+00:00",
            "title": "Quantifying Exposure Bias for Open-ended Language Generation",
            "authors": [
                "Tianxing He",
                "Jingzhao Zhang",
                "Zhiming Zhou",
                "James Glass"
            ],
            "abstract": "The exposure bias problem refers to the incrementally distorted generation\ninduced by the training-generation discrepancy, in teacher-forcing training for\nauto-regressive neural network language models (LM). It has been regarded as a\ncentral problem for LMs trained for open-ended language generation. Although a\nlot of algorithms have been proposed to avoid teacher forcing and therefore\nalleviate exposure bias, there is little work showing how serious the exposure\nbias problem actually is. In this work, we propose novel metrics to quantify\nthe impact of exposure bias in the generation of MLE-trained LMs. Our key\nintuition is that if we feed ground-truth data prefixes (instead of prefixes\ngenerated by the model itself) into the model and ask it to continue the\ngeneration, the performance should become much better because the\ntraining-generation discrepancy in the prefix is removed. We conduct both\nautomatic and human evaluation in our experiments, and our observations are\ntwo-fold: (1) We confirm that the prefix discrepancy indeed induces some level\nof performance loss. (2) However, the induced distortion seems to be limited,\nand is not incremental during the generation, which contradicts the claim of\nexposure bias.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10702v8",
            "date": "2019-05-25 23:48:00+00:00",
            "title": "MDE: Multiple Distance Embeddings for Link Prediction in Knowledge Graphs",
            "authors": [
                "Afshin Sadeghi",
                "Damien Graux",
                "Hamed Shariat Yazdi",
                "Jens Lehmann"
            ],
            "abstract": "Over the past decade, knowledge graphs became popular for capturing\nstructured domain knowledge. Relational learning models enable the prediction\nof missing links inside knowledge graphs. More specifically, latent distance\napproaches model the relationships among entities via a distance between latent\nrepresentations. Translating embedding models (e.g., TransE) are among the most\npopular latent distance approaches which use one distance function to learn\nmultiple relation patterns. However, they are mostly inefficient in capturing\nsymmetric relations since the representation vector norm for all the symmetric\nrelations becomes equal to zero. They also lose information when learning\nrelations with reflexive patterns since they become symmetric and transitive.\nWe propose the Multiple Distance Embedding model (MDE) that addresses these\nlimitations and a framework to collaboratively combine variant latent\ndistance-based terms. Our solution is based on two principles: 1) we use a\nlimit-based loss instead of a margin ranking loss and, 2) by learning\nindependent embedding vectors for each of the terms we can collectively train\nand predict using contradicting distance terms. We further demonstrate that MDE\nallows modeling relations with (anti)symmetry, inversion, and composition\npatterns. We propose MDE as a neural network model that allows us to map\nnon-linear relations between the embedding vectors and the expected output of\nthe score function. Our empirical results show that MDE performs competitively\nto state-of-the-art embedding models on several benchmark datasets.",
            "journal ref": "24th European Conference on Artificial Intelligence (ECAI), 2020",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.10799v3",
            "date": "2019-05-26 12:57:47+00:00",
            "title": "Path Ranking with Attention to Type Hierarchies",
            "authors": [
                "Weiyu Liu",
                "Angel Daruna",
                "Zsolt Kira",
                "Sonia Chernova"
            ],
            "abstract": "The objective of the knowledge base completion problem is to infer missing\ninformation from existing facts in a knowledge base. Prior work has\ndemonstrated the effectiveness of path-ranking based methods, which solve the\nproblem by discovering observable patterns in knowledge graphs, consisting of\nnodes representing entities and edges representing relations. However, these\npatterns either lack accuracy because they rely solely on relations or cannot\neasily generalize due to the direct use of specific entity information. We\nintroduce Attentive Path Ranking, a novel path pattern representation that\nleverages type hierarchies of entities to both avoid ambiguity and maintain\ngeneralization. Then, we present an end-to-end trained attention-based RNN\nmodel to discover the new path patterns from data. Experiments conducted on\nbenchmark knowledge base completion datasets WN18RR and FB15k-237 demonstrate\nthat the proposed model outperforms existing methods on the fact prediction\ntask by statistically significant margins of 26% and 10%, respectively.\nFurthermore, quantitative and qualitative analyses show that the path patterns\nbalance between generalization and discrimination.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1905.10802v2",
            "date": "2019-05-26 13:20:11+00:00",
            "title": "Hyperbolic Interaction Model For Hierarchical Multi-Label Classification",
            "authors": [
                "Boli Chen",
                "Xin Huang",
                "Lin Xiao",
                "Zixin Cai",
                "Liping Jing"
            ],
            "abstract": "Different from the traditional classification tasks which assume mutual\nexclusion of labels, hierarchical multi-label classification (HMLC) aims to\nassign multiple labels to every instance with the labels organized under\nhierarchical relations. Besides the labels, since linguistic ontologies are\nintrinsic hierarchies, the conceptual relations between words can also form\nhierarchical structures. Thus it can be a challenge to learn mappings from word\nhierarchies to label hierarchies. We propose to model the word and label\nhierarchies by embedding them jointly in the hyperbolic space. The main reason\nis that the tree-likeness of the hyperbolic space matches the complexity of\nsymbolic data with hierarchical structures. A new Hyperbolic Interaction Model\n(HyperIM) is designed to learn the label-aware document representations and\nmake predictions for HMLC. Extensive experiments are conducted on three\nbenchmark datasets. The results have demonstrated that the new model can\nrealistically capture the complex data structures and further improve the\nperformance for HMLC comparing with the state-of-the-art methods. To facilitate\nfuture research, our code is publicly available.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10847v1",
            "date": "2019-05-26 17:56:11+00:00",
            "title": "Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives",
            "authors": [
                "Yi Tay",
                "Shuohang Wang",
                "Luu Anh Tuan",
                "Jie Fu",
                "Minh C. Phan",
                "Xingdi Yuan",
                "Jinfeng Rao",
                "Siu Cheung Hui",
                "Aston Zhang"
            ],
            "abstract": "This paper tackles the problem of reading comprehension over long narratives\nwhere documents easily span over thousands of tokens. We propose a curriculum\nlearning (CL) based Pointer-Generator framework for reading/sampling over large\ndocuments, enabling diverse training of the neural model based on the notion of\nalternating contextual difficulty. This can be interpreted as a form of domain\nrandomization and/or generative pretraining during training. To this end, the\nusage of the Pointer-Generator softens the requirement of having the answer\nwithin the context, enabling us to construct diverse training samples for\nlearning. Additionally, we propose a new Introspective Alignment Layer (IAL),\nwhich reasons over decomposed alignments using block-based self-attention. We\nevaluate our proposed method on the NarrativeQA reading comprehension\nbenchmark, achieving state-of-the-art performance, improving existing baselines\nby $51\\%$ relative improvement on BLEU-4 and $17\\%$ relative improvement on\nRouge-L. Extensive ablations confirm the effectiveness of our proposed IAL and\nCL components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.10930v2",
            "date": "2019-05-27 01:42:30+00:00",
            "title": "Sequential Graph Dependency Parser",
            "authors": [
                "Sean Welleck",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose a method for non-projective dependency parsing by incrementally\npredicting a set of edges. Since the edges do not have a pre-specified order,\nwe propose a set-based learning method. Our method blends graph, transition,\nand easy-first parsing, including a prior state of the parser as a special\ncase. The proposed transition-based method successfully parses near the state\nof the art on both projective and non-projective languages, without assuming a\ncertain parsing order.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10949v1",
            "date": "2019-05-27 03:08:17+00:00",
            "title": "QuesNet: A Unified Representation for Heterogeneous Test Questions",
            "authors": [
                "Yu Yin",
                "Qi Liu",
                "Zhenya Huang",
                "Enhong Chen",
                "Wei Tong",
                "Shijin Wang",
                "Yu Su"
            ],
            "abstract": "Understanding learning materials (e.g. test questions) is a crucial issue in\nonline learning systems, which can promote many applications in education\ndomain. Unfortunately, many supervised approaches suffer from the problem of\nscarce human labeled data, whereas abundant unlabeled resources are highly\nunderutilized. To alleviate this problem, an effective solution is to use\npre-trained representations for question understanding. However, existing\npre-training methods in NLP area are infeasible to learn test question\nrepresentations due to several domain-specific characteristics in education.\nFirst, questions usually comprise of heterogeneous data including content text,\nimages and side information. Second, there exists both basic linguistic\ninformation as well as domain logic and knowledge. To this end, in this paper,\nwe propose a novel pre-training method, namely QuesNet, for comprehensively\nlearning question representations. Specifically, we first design a unified\nframework to aggregate question information with its heterogeneous inputs into\na comprehensive vector. Then we propose a two-level hierarchical pre-training\nalgorithm to learn better understanding of test questions in an unsupervised\nway. Here, a novel holed language model objective is developed to extract\nlow-level linguistic features, and a domain-oriented objective is proposed to\nlearn high-level logic and knowledge. Moreover, we show that QuesNet has good\ncapability of being fine-tuned in many question-based tasks. We conduct\nextensive experiments on large-scale real-world question data, where the\nexperimental results clearly demonstrate the effectiveness of QuesNet for\nquestion understanding as well as its superior applicability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10971v3",
            "date": "2019-05-27 04:49:45+00:00",
            "title": "An Empirical Study on Post-processing Methods for Word Embeddings",
            "authors": [
                "Shuai Tang",
                "Mahta Mousavi",
                "Virginia R. de Sa"
            ],
            "abstract": "Word embeddings learnt from large corpora have been adopted in various\napplications in natural language processing and served as the general input\nrepresentations to learning systems. Recently, a series of post-processing\nmethods have been proposed to boost the performance of word embeddings on\nsimilarity comparison and analogy retrieval tasks, and some have been adapted\nto compose sentence representations. The general hypothesis behind these\nmethods is that by enforcing the embedding space to be more isotropic, the\nsimilarity between words can be better expressed. We view these methods as an\napproach to shrink the covariance/gram matrix, which is estimated by learning\nword vectors, towards a scaled identity matrix. By optimising an objective in\nthe semi-Riemannian manifold with Centralised Kernel Alignment (CKA), we are\nable to search for the optimal shrinkage parameter, and provide a\npost-processing method to smooth the spectrum of learnt word vectors which\nyields improved performance on downstream tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.10989v4",
            "date": "2019-05-27 06:12:56+00:00",
            "title": "Commonsense Properties from Query Logs and Question Answering Forums",
            "authors": [
                "Julien Romero",
                "Simon Razniewski",
                "Koninika Pal",
                "Jeff Z. Pan",
                "Archit Sakhadeo",
                "Gerhard Weikum"
            ],
            "abstract": "Commonsense knowledge about object properties, human behavior and general\nconcepts is crucial for robust AI applications. However, automatic acquisition\nof this knowledge is challenging because of sparseness and bias in online\nsources. This paper presents Quasimodo, a methodology and tool suite for\ndistilling commonsense properties from non-standard web sources. We devise\nnovel ways of tapping into search-engine query logs and QA forums, and\ncombining the resulting candidate assertions with statistical cues from\nencyclopedias, books and image tags in a corroboration step. Unlike prior work\non commonsense knowledge bases, Quasimodo focuses on salient properties that\nare typically associated with certain objects or concepts. Extensive\nevaluations, including extrinsic use-case studies, show that Quasimodo provides\nbetter coverage than state-of-the-art baselines with comparable quality.",
            "journal ref": "CIKM 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11173v3",
            "date": "2019-05-27 12:41:36+00:00",
            "title": "ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks",
            "authors": [
                "Xiaoqi Jia",
                "Jianwei Tai",
                "Hang Zhou",
                "Yakai Li",
                "Weijuan Zhang",
                "Haichao Du",
                "Qingjia Huang"
            ],
            "abstract": "Despite the remarkable progress made in synthesizing emotional speech from\ntext, it is still challenging to provide emotion information to existing speech\nsegments. Previous methods mainly rely on parallel data, and few works have\nstudied the generalization ability for one model to transfer emotion\ninformation across different languages. To cope with such problems, we propose\nan emotion transfer system named ET-GAN, for learning language-independent\nemotion transfer from one emotion to another without parallel training samples.\nBased on cycle-consistent generative adversarial network, our method ensures\nthe transfer of only emotion information across speeches with simple loss\ndesigns. Besides, we introduce an approach for migrating emotion information\nacross different languages by using transfer learning. The experiment results\nshow that our method can efficiently generate high-quality emotional speech for\nany given emotion category, without aligned speech pairs.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1905.11240v2",
            "date": "2019-05-24 10:22:16+00:00",
            "title": "Bridging Dialogue Generation and Facial Expression Synthesis",
            "authors": [
                "Shang-Yu Su",
                "Yun-Nung Chen"
            ],
            "abstract": "Spoken dialogue systems that assist users to solve complex tasks such as\nmovie ticket booking have become an emerging research topic in artificial\nintelligence and natural language processing areas. With a well-designed\ndialogue system as an intelligent personal assistant, people can accomplish\ncertain tasks more easily via natural language interactions. Today there are\nseveral virtual intelligent assistants in the market; however, most systems\nonly focus on single modality, such as textual or vocal interaction. A\nmultimodal interface has various advantages: (1) allowing human to communicate\nwith machines in a natural and concise form using the mixture of modalities\nthat most precisely convey the intention to satisfy communication needs, and\n(2) providing more engaging experience by natural and human-like feedback. This\npaper explores a brand new research direction, which aims at bridging dialogue\ngeneration and facial expression synthesis for better multimodal interaction.\nThe goal is to generate dialogue responses and simultaneously synthesize\ncorresponding visual expressions on faces, which is also an ultimate step\ntoward more human-like virtual assistants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11259v1",
            "date": "2019-05-27 14:27:13+00:00",
            "title": "AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning",
            "authors": [
                "Lu Chen",
                "Zhi Chen",
                "Bowen Tan",
                "Sishan Long",
                "Milica Gasic",
                "Kai Yu"
            ],
            "abstract": "Dialogue policy plays an important role in task-oriented spoken dialogue\nsystems. It determines how to respond to users. The recently proposed deep\nreinforcement learning (DRL) approaches have been used for policy optimization.\nHowever, these deep models are still challenging for two reasons: 1) Many\nDRL-based policies are not sample-efficient. 2) Most models don't have the\ncapability of policy transfer between different domains. In this paper, we\npropose a universal framework, AgentGraph, to tackle these two problems. The\nproposed AgentGraph is the combination of GNN-based architecture and DRL-based\nalgorithm. It can be regarded as one of the multi-agent reinforcement learning\napproaches. Each agent corresponds to a node in a graph, which is defined\naccording to the dialogue domain ontology. When making a decision, each agent\ncan communicate with its neighbors on the graph. Under AgentGraph framework, we\nfurther propose Dual GNN-based dialogue policy, which implicitly decomposes the\ndecision in each turn into a high-level global decision and a low-level local\ndecision. Experiments show that AgentGraph models significantly outperform\ntraditional reinforcement learning approaches on most of the 18 tasks of the\nPyDial benchmark. Moreover, when transferred from the source task to a target\ntask, these models not only have acceptable initial performance but also\nconverge much faster on the target task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11268v2",
            "date": "2019-05-27 14:35:35+00:00",
            "title": "Combating Adversarial Misspellings with Robust Word Recognition",
            "authors": [
                "Danish Pruthi",
                "Bhuwan Dhingra",
                "Zachary C. Lipton"
            ],
            "abstract": "To combat adversarial spelling mistakes, we propose placing a word\nrecognition model in front of the downstream classifier. Our word recognition\nmodels build upon the RNN semi-character architecture, introducing several new\nbackoff strategies for handling rare and unseen words. Trained to recognize\nwords corrupted by random adds, drops, swaps, and keyboard mistakes, our method\nachieves 32% relative (and 3.3% absolute) error reduction over the vanilla\nsemi-character model. Notably, our pipeline confers robustness on the\ndownstream classifier, outperforming both adversarial training and\noff-the-shelf spell checkers. Against a BERT model fine-tuned for sentiment\nanalysis, a single adversarially-chosen character attack lowers accuracy from\n90.3% to 45.8%. Our defense restores accuracy to 75%. Surprisingly, better word\nrecognition does not always entail greater robustness. Our analysis reveals\nthat robustness also depends upon a quantity that we denote the sensitivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11391v2",
            "date": "2019-05-27 09:33:29+00:00",
            "title": "Using Neural Networks for Relation Extraction from Biomedical Literature",
            "authors": [
                "Diana Sousa",
                "Andre Lamurias",
                "Francisco M. Couto"
            ],
            "abstract": "Using different sources of information to support automated extracting of\nrelations between biomedical concepts contributes to the development of our\nunderstanding of biological systems. The primary comprehensive source of these\nrelations is biomedical literature. Several relation extraction approaches have\nbeen proposed to identify relations between concepts in biomedical literature,\nnamely, using neural networks algorithms. The use of multichannel architectures\ncomposed of multiple data representations, as in deep neural networks, is\nleading to state-of-the-art results. The right combination of data\nrepresentations can eventually lead us to even higher evaluation scores in\nrelation extraction tasks. Thus, biomedical ontologies play a fundamental role\nby providing semantic and ancestry information about an entity. The\nincorporation of biomedical ontologies has already been proved to enhance\nprevious state-of-the-art results.",
            "journal ref": "Artificial Neural Networks. Methods in Molecular Biology, vol\n  2190. Humana, New York, NY. 2020. pp. 289-305",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11393v1",
            "date": "2019-05-27 10:22:20+00:00",
            "title": "A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications",
            "authors": [
                "Mengyang Chen",
                "Jin Zeng",
                "Jie Lou"
            ],
            "abstract": "Spoken language understanding (SLU) acts as a critical component in\ngoal-oriented dialog systems. It typically involves identifying the speakers\nintent and extracting semantic slots from user utterances, which are known as\nintent detection (ID) and slot filling (SF). SLU problem has been intensively\ninvestigated in recent years. However, these methods just constrain SF results\ngrammatically, solve ID and SF independently, or do not fully utilize the\nmutual impact of the two tasks. This paper proposes a multi-head self-attention\njoint model with a conditional random field (CRF) layer and a prior mask. The\nexperiments show the effectiveness of our model, as compared with\nstate-of-the-art models. Meanwhile, online education in China has made great\nprogress in the last few years. But there are few intelligent educational\ndialog applications for students to learn foreign languages. Hence, we design\nan intelligent dialog robot equipped with different scenario settings to help\nstudents learn communication skills.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11449v2",
            "date": "2019-05-27 18:59:54+00:00",
            "title": "VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019",
            "authors": [
                "Andros Tjandra",
                "Berrak Sisman",
                "Mingyang Zhang",
                "Sakriani Sakti",
                "Haizhou Li",
                "Satoshi Nakamura"
            ],
            "abstract": "We describe our submitted system for the ZeroSpeech Challenge 2019. The\ncurrent challenge theme addresses the difficulty of constructing a speech\nsynthesizer without any text or phonetic labels and requires a system that can\n(1) discover subword units in an unsupervised way, and (2) synthesize the\nspeech with a target speaker's voice. Moreover, the system should also balance\nthe discrimination score ABX, the bit-rate compression rate, and the\nnaturalness and the intelligibility of the constructed voice. To tackle these\nproblems and achieve the best trade-off, we utilize a vector quantized\nvariational autoencoder (VQ-VAE) and a multi-scale codebook-to-spectrogram\n(Code2Spec) inverter trained by mean square error and adversarial loss. The\nVQ-VAE extracts the speech to a latent space, forces itself to map it into the\nnearest codebook and produces compressed representation. Next, the inverter\ngenerates a magnitude spectrogram to the target voice, given the codebook\nvectors from VQ-VAE. In our experiments, we also investigated several other\nclustering algorithms, including K-Means and GMM, and compared them with the\nVQ-VAE result on ABX scores and bit rates. Our proposed approach significantly\nimproved the intelligibility (in CER), the MOS, and discrimination ABX scores\ncompared to the official ZeroSpeech 2019 baseline or even the topline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11471v1",
            "date": "2019-05-27 19:44:33+00:00",
            "title": "XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering",
            "authors": [
                "Jasdeep Singh",
                "Bryan McCann",
                "Nitish Shirish Keskar",
                "Caiming Xiong",
                "Richard Socher"
            ],
            "abstract": "While natural language processing systems often focus on a single language,\nmultilingual transfer learning has the potential to improve performance,\nespecially for low-resource languages. We introduce XLDA, cross-lingual data\naugmentation, a method that replaces a segment of the input text with its\ntranslation in another language. XLDA enhances performance of all 14 tested\nlanguages of the cross-lingual natural language inference (XNLI) benchmark.\nWith improvements of up to $4.8\\%$, training with XLDA achieves\nstate-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast\nto, and performs markedly better than, a more naive approach that aggregates\nexamples in various languages in a way that each example is solely in one\nlanguage. On the SQuAD question answering task, we see that XLDA provides a\n$1.0\\%$ performance increase on the English evaluation set. Comprehensive\nexperiments suggest that most languages are effective as cross-lingual\naugmentors, that XLDA is robust to a wide range of translation quality, and\nthat XLDA is even more effective for randomly initialized models than for\npretrained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11553v2",
            "date": "2019-05-28 00:55:25+00:00",
            "title": "Target-Guided Open-Domain Conversation",
            "authors": [
                "Jianheng Tang",
                "Tiancheng Zhao",
                "Chenyan Xiong",
                "Xiaodan Liang",
                "Eric P. Xing",
                "Zhiting Hu"
            ],
            "abstract": "Many real-world open-domain conversation applications have specific goals to\nachieve during open-ended chats, such as recommendation, psychotherapy,\neducation, etc. We study the problem of imposing conversational goals on\nopen-domain chat agents. In particular, we want a conversational system to chat\nnaturally with human and proactively guide the conversation to a designated\ntarget subject. The problem is challenging as no public data is available for\nlearning such a target-guided strategy. We propose a structured approach that\nintroduces coarse-grained keywords to control the intended content of system\nresponses. We then attain smooth conversation transition through turn-level\nsupervised learning, and drive the conversation towards the target with\ndiscourse-level constraints. We further derive a keyword-augmented conversation\ndataset for the study. Quantitative and human evaluations show our system can\nproduce meaningful and effective conversations, significantly improving over\nother approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11563v3",
            "date": "2019-05-28 01:36:31+00:00",
            "title": "Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion",
            "authors": [
                "Andy T. Liu",
                "Po-chun Hsu",
                "Hung-yi Lee"
            ],
            "abstract": "We present an unsupervised end-to-end training scheme where we discover\ndiscrete subword units from speech without using any labels. The discrete\nsubword units are learned under an ASR-TTS autoencoder reconstruction setting,\nwhere an ASR-Encoder is trained to discover a set of common linguistic units\ngiven a variety of speakers, and a TTS-Decoder trained to project the\ndiscovered units back to the designated speech. We propose a discrete encoding\nmethod, Multilabel-Binary Vectors (MBV), to make the ASR-TTS autoencoder\ndifferentiable. We found that the proposed encoding method offers automatic\nextraction of speech content from speaker style, and is sufficient to cover\nfull linguistic content in a given language. Therefore, the TTS-Decoder can\nsynthesize speech with the same content as the input of ASR-Encoder but with\ndifferent speaker characteristics, which achieves voice conversion (VC). We\nfurther improve the quality of VC using adversarial training, where we train a\nTTS-Patcher that augments the output of TTS-Decoder. Objective and subjective\nevaluations show that the proposed approach offers strong VC results as it\neliminates speaker identity while preserving content within speech. In the\nZeroSpeech 2019 Challenge, we achieved outstanding performance in terms of low\nbitrate.",
            "journal ref": "Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11833v4",
            "date": "2019-05-28 14:13:09+00:00",
            "title": "Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)",
            "authors": [
                "Mariya Toneva",
                "Leila Wehbe"
            ],
            "abstract": "Neural networks models for NLP are typically implemented without the explicit\nencoding of language rules and yet they are able to break one performance\nrecord after another. This has generated a lot of research interest in\ninterpreting the representations learned by these networks. We propose here a\nnovel interpretation approach that relies on the only processing system we have\nthat does understand language: the human brain. We use brain imaging recordings\nof subjects reading complex natural text to interpret word and sequence\nembeddings from 4 recent NLP models - ELMo, USE, BERT and Transformer-XL. We\nstudy how their representations differ across layer depth, context length, and\nattention type. Our results reveal differences in the context-related\nrepresentations across these models. Further, in the transformer models, we\nfind an interaction between layer depth and context length, and between layer\ndepth and attention type. We finally hypothesize that altering BERT to better\nalign with brain recordings would enable it to also better understand language.\nProbing the altered BERT using syntactic NLP tasks reveals that the model with\nincreased brain-alignment outperforms the original model. Cognitive\nneuroscientists have already begun using NLP networks to study the brain, and\nthis work closes the loop to allow the interaction between NLP and cognitive\nneuroscience to be a true cross-pollination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.11978v2",
            "date": "2019-05-28 17:55:32+00:00",
            "title": "Better Long-Range Dependency By Bootstrapping A Mutual Information Regularizer",
            "authors": [
                "Yanshuai Cao",
                "Peng Xu"
            ],
            "abstract": "In this work, we develop a novel regularizer to improve the learning of\nlong-range dependency of sequence data. Applied on language modelling, our\nregularizer expresses the inductive bias that sequence variables should have\nhigh mutual information even though the model might not see abundant\nobservations for complex long-range dependency. We show how the `next sentence\nprediction (classification)' heuristic can be derived in a principled way from\nour mutual information estimation framework, and be further extended to\nmaximize the mutual information of sequence variables. The proposed approach\nnot only is effective at increasing the mutual information of segments under\nthe learned model but more importantly, leads to a higher likelihood on holdout\ndata, and improved generation quality. Code is released at\nhttps://github.com/BorealisAI/BMI.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12008v1",
            "date": "2019-05-28 18:15:52+00:00",
            "title": "Leveraging Medical Visual Question Answering with Supporting Facts",
            "authors": [
                "Tomasz Kornuta",
                "Deepta Rajan",
                "Chaitanya Shivade",
                "Alexis Asseman",
                "Ahmet S. Ozcan"
            ],
            "abstract": "In this working notes paper, we describe IBM Research AI (Almaden) team's\nparticipation in the ImageCLEF 2019 VQA-Med competition. The challenge consists\nof four question-answering tasks based on radiology images. The diversity of\nimaging modalities, organs and disease types combined with a small imbalanced\ntraining set made this a highly complex problem. To overcome these\ndifficulties, we implemented a modular pipeline architecture that utilized\ntransfer learning and multi-task learning. Our findings led to the development\nof a novel model called Supporting Facts Network (SFN). The main idea behind\nSFN is to cross-utilize information from upstream tasks to improve the accuracy\non harder downstream ones. This approach significantly improved the scores\nachieved in the validation set (18 point improvement in F-1 score). Finally, we\nsubmitted four runs to the competition and were ranked seventh.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1905.12230v2",
            "date": "2019-05-29 05:50:35+00:00",
            "title": "Guided Source Separation Meets a Strong ASR Backend: Hitachi/Paderborn University Joint Investigation for Dinner Party ASR",
            "authors": [
                "Naoyuki Kanda",
                "Christoph Boeddeker",
                "Jens Heitkaemper",
                "Yusuke Fujita",
                "Shota Horiguchi",
                "Kenji Nagamatsu",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "In this paper, we present Hitachi and Paderborn University's joint effort for\nautomatic speech recognition (ASR) in a dinner party scenario. The main\nchallenges of ASR systems for dinner party recordings obtained by multiple\nmicrophone arrays are (1) heavy speech overlaps, (2) severe noise and\nreverberation, (3) very natural conversational content, and possibly (4)\ninsufficient training data. As an example of a dinner party scenario, we have\nchosen the data presented during the CHiME-5 speech recognition challenge,\nwhere the baseline ASR had a 73.3% word error rate (WER), and even the best\nperforming system at the CHiME-5 challenge had a 46.1% WER. We extensively\ninvestigated a combination of the guided source separation-based speech\nenhancement technique and an already proposed strong ASR backend and found that\na tight combination of these techniques provided substantial accuracy\nimprovements. Our final system achieved WERs of 39.94% and 41.64% for the\ndevelopment and evaluation data, respectively, both of which are the best\npublished results for the dataset. We also investigated with additional\ntraining data on the official small data in the CHiME-5 corpus to assess the\nintrinsic difficulty of this ASR task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12260v1",
            "date": "2019-05-29 07:55:17+00:00",
            "title": "Learning Multilingual Word Embeddings Using Image-Text Data",
            "authors": [
                "Karan Singhal",
                "Karthik Raman",
                "Balder ten Cate"
            ],
            "abstract": "There has been significant interest recently in learning multilingual word\nembeddings -- in which semantically similar words across languages have similar\nembeddings. State-of-the-art approaches have relied on expensive labeled data,\nwhich is unavailable for low-resource languages, or have involved post-hoc\nunification of monolingual embeddings. In the present paper, we investigate the\nefficacy of multilingual embeddings learned from weakly-supervised image-text\ndata. In particular, we propose methods for learning multilingual embeddings\nusing image-text data, by enforcing similarity between the representations of\nthe image and that of the text. Our experiments reveal that even without using\nany expensive labeled data, a bag-of-words-based embedding model trained on\nimage-text data achieves performance comparable to the state-of-the-art on\ncrosslingual semantic similarity tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12330v3",
            "date": "2019-05-29 11:17:59+00:00",
            "title": "Word-order biases in deep-agent emergent communication",
            "authors": [
                "Rahma Chaabouni",
                "Eugene Kharitonov",
                "Alessandro Lazaric",
                "Emmanuel Dupoux",
                "Marco Baroni"
            ],
            "abstract": "Sequence-processing neural networks led to remarkable progress on many NLP\ntasks. As a consequence, there has been increasing interest in understanding to\nwhat extent they process language as humans do. We aim here to uncover which\nbiases such models display with respect to \"natural\" word-order constraints. We\ntrain models to communicate about paths in a simple gridworld, using miniature\nlanguages that reflect or violate various natural language trends, such as the\ntendency to avoid redundancy or to minimize long-distance dependencies. We\nstudy how the controlled characteristics of our miniature languages affect\nindividual learning and their stability across multiple network generations.\nThe results draw a mixed picture. On the one hand, neural networks show a\nstrong tendency to avoid long-distance dependencies. On the other hand, there\nis no clear preference for the efficient, non-redundant encoding of information\nthat is widely attested in natural language. We thus suggest inoculating a\nnotion of \"effort\" into neural networks, as a possible way to make their\nlinguistic behavior more human-like.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12356v3",
            "date": "2019-05-29 12:05:31+00:00",
            "title": "SECRET: Semantically Enhanced Classification of Real-world Tasks",
            "authors": [
                "Ayten Ozge Akmandor",
                "Jorge Ortiz",
                "Irene Manotas",
                "Bongjun Ko",
                "Niraj K. Jha"
            ],
            "abstract": "Supervised machine learning (ML) algorithms are aimed at maximizing\nclassification performance under available energy and storage constraints. They\ntry to map the training data to the corresponding labels while ensuring\ngeneralizability to unseen data. However, they do not integrate meaning-based\nrelationships among labels in the decision process. On the other hand, natural\nlanguage processing (NLP) algorithms emphasize the importance of semantic\ninformation. In this paper, we synthesize the complementary advantages of\nsupervised ML and NLP algorithms into one method that we refer to as SECRET\n(Semantically Enhanced Classification of REal-world Tasks). SECRET performs\nclassifications by fusing the semantic information of the labels with the\navailable data: it combines the feature space of the supervised algorithms with\nthe semantic space of the NLP algorithms and predicts labels based on this\njoint space. Experimental results indicate that, compared to traditional\nsupervised learning, SECRET achieves up to 14.0% accuracy and 13.1% F1 score\nimprovements. Moreover, compared to ensemble methods, SECRET achieves up to\n12.7% accuracy and 13.3% F1 score improvements. This points to a new research\ndirection for supervised classification based on incorporation of semantic\ninformation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12480v1",
            "date": "2019-05-29 14:16:17+00:00",
            "title": "NRPA: Neural Recommendation with Personalized Attention",
            "authors": [
                "Hongtao Liu",
                "Fangzhao Wu",
                "Wenjun Wang",
                "Xianchen Wang",
                "Pengfei Jiao",
                "Chuhan Wu",
                "Xing Xie"
            ],
            "abstract": "Existing review-based recommendation methods usually use the same model to\nlearn the representations of all users/items from reviews posted by users\ntowards items. However, different users have different preference and different\nitems have different characteristics. Thus, the same word or similar reviews\nmay have different informativeness for different users and items. In this paper\nwe propose a neural recommendation approach with personalized attention to\nlearn personalized representations of users and items from reviews. We use a\nreview encoder to learn representations of reviews from words, and a user/item\nencoder to learn representations of users or items from reviews. We propose a\npersonalized attention model, and apply it to both review and user/item\nencoders to select different important words and reviews for different\nusers/items. Experiments on five datasets validate our approach can effectively\nimprove the performance of neural recommendation.",
            "journal ref": "sigir 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.12561v4",
            "date": "2019-05-29 16:14:24+00:00",
            "title": "Anti-efficient encoding in emergent communication",
            "authors": [
                "Rahma Chaabouni",
                "Eugene Kharitonov",
                "Emmanuel Dupoux",
                "Marco Baroni"
            ],
            "abstract": "Despite renewed interest in emergent language simulations with neural\nnetworks, little is known about the basic properties of the induced code, and\nhow they compare to human language. One fundamental characteristic of the\nlatter, known as Zipf's Law of Abbreviation (ZLA), is that more frequent words\nare efficiently associated to shorter strings. We study whether the same\npattern emerges when two neural networks, a \"speaker\" and a \"listener\", are\ntrained to play a signaling game. Surprisingly, we find that networks develop\nan \\emph{anti-efficient} encoding scheme, in which the most frequent inputs are\nassociated to the longest messages, and messages in general are skewed towards\nthe maximum length threshold. This anti-efficient code appears easier to\ndiscriminate for the listener, and, unlike in human communication, the speaker\ndoes not impose a contrasting least-effort pressure towards brevity. Indeed,\nwhen the cost function includes a penalty for longer messages, the resulting\nmessage distribution starts respecting ZLA. Our analysis stresses the\nimportance of studying the basic features of emergent communication in a highly\ncontrolled setup, to ensure the latter will not strand too far from human\nlanguage. Moreover, we present a concrete illustration of how different\nfunctional pressures can lead to successful communication codes that lack basic\nproperties of human language, thus highlighting the role such pressures play in\nthe latter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12730v2",
            "date": "2019-05-29 21:10:58+00:00",
            "title": "Recursive Sketches for Modular Deep Learning",
            "authors": [
                "Badih Ghazi",
                "Rina Panigrahy",
                "Joshua R. Wang"
            ],
            "abstract": "We present a mechanism to compute a sketch (succinct summary) of how a\ncomplex modular deep network processes its inputs. The sketch summarizes\nessential information about the inputs and outputs of the network and can be\nused to quickly identify key components and summary statistics of the inputs.\nFurthermore, the sketch is recursive and can be unrolled to identify\nsub-components of these components and so forth, capturing a potentially\ncomplicated DAG structure. These sketches erase gracefully; even if we erase a\nfraction of the sketch at random, the remainder still retains the `high-weight'\ninformation present in the original sketch. The sketches can also be organized\nin a repository to implicitly form a `knowledge graph'; it is possible to\nquickly retrieve sketches in the repository that are related to a sketch of\ninterest; arranged in this fashion, the sketches can also be used to learn\nemerging concepts by looking for new clusters in sketch space. Finally, in the\nscenario where we want to learn a ground truth deep network, we show that\naugmenting input/output pairs with these sketches can theoretically make it\neasier to do so.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12741v2",
            "date": "2019-05-29 21:29:37+00:00",
            "title": "Adapting Text Embeddings for Causal Inference",
            "authors": [
                "Victor Veitch",
                "Dhanya Sridhar",
                "David M. Blei"
            ],
            "abstract": "Does adding a theorem to a paper affect its chance of acceptance? Does\nlabeling a post with the author's gender affect the post popularity? This paper\ndevelops a method to estimate such causal effects from observational text data,\nadjusting for confounding features of the text such as the subject or writing\nquality. We assume that the text suffices for causal adjustment but that, in\npractice, it is prohibitively high-dimensional. To address this challenge, we\ndevelop causally sufficient embeddings, low-dimensional document\nrepresentations that preserve sufficient information for causal identification\nand allow for efficient estimation of causal effects. Causally sufficient\nembeddings combine two ideas. The first is supervised dimensionality reduction:\ncausal adjustment requires only the aspects of text that are predictive of both\nthe treatment and outcome. The second is efficient language modeling:\nrepresentations of text are designed to dispose of linguistically irrelevant\ninformation, and this information is also causally irrelevant. Our method\nadapts language models (specifically, word embeddings and topic models) to\nlearn document embeddings that are able to predict both treatment and outcome.\nWe study causally sufficient embeddings with semi-synthetic datasets and find\nthat they improve causal estimation over related embedding methods. We\nillustrate the methods by answering the two motivating questions---the effect\nof a theorem on paper acceptance and the effect of a gender label on post\npopularity. Code and data available at\nhttps://github.com/vveitch/causal-text-embeddings-tf2}{github.com/vveitch/causal-text-embeddings-tf2",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12752v1",
            "date": "2019-05-29 22:15:38+00:00",
            "title": "Unsupervised Paraphrasing without Translation",
            "authors": [
                "Aurko Roy",
                "David Grangier"
            ],
            "abstract": "Paraphrasing exemplifies the ability to abstract semantic content from\nsurface forms. Recent work on automatic paraphrasing is dominated by methods\nleveraging Machine Translation (MT) as an intermediate step. This contrasts\nwith humans, who can paraphrase without being bilingual. This work proposes to\nlearn paraphrasing models from an unlabeled monolingual corpus only. To that\nend, we propose a residual variant of vector-quantized variational\nauto-encoder.\n  We compare with MT-based approaches on paraphrase identification, generation,\nand training augmentation. Monolingual paraphrasing outperforms unsupervised\ntranslation in all settings. Comparisons with supervised translation are more\nmixed: monolingual paraphrasing is interesting for identification and\naugmentation; supervised translation is superior for generation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12777v3",
            "date": "2019-05-29 23:22:56+00:00",
            "title": "Educating Text Autoencoders: Latent Representation Guidance via Denoising",
            "authors": [
                "Tianxiao Shen",
                "Jonas Mueller",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "Generative autoencoders offer a promising approach for controllable text\ngeneration by leveraging their latent sentence representations. However,\ncurrent models struggle to maintain coherent latent spaces required to perform\nmeaningful text manipulations via latent vector operations. Specifically, we\ndemonstrate by example that neural encoders do not necessarily map similar\nsentences to nearby latent vectors. A theoretical explanation for this\nphenomenon establishes that high capacity autoencoders can learn an arbitrary\nmapping between sequences and associated latent representations. To remedy this\nissue, we augment adversarial autoencoders with a denoising objective where\noriginal sentences are reconstructed from perturbed versions (referred to as\nDAAE). We prove that this simple modification guides the latent space geometry\nof the resulting model by encouraging the encoder to map similar texts to\nsimilar latent representations. In empirical comparisons with various types of\nautoencoders, our model provides the best trade-off between generation quality\nand reconstruction capacity. Moreover, the improved geometry of the DAAE latent\nspace enables zero-shot text style transfer via simple latent vector\narithmetic.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12786v1",
            "date": "2019-05-29 23:40:54+00:00",
            "title": "Large Scale Question Paraphrase Retrieval with Smoothed Deep Metric Learning",
            "authors": [
                "Daniele Bonadiman",
                "Anjishnu Kumar",
                "Arpit Mittal"
            ],
            "abstract": "The goal of a Question Paraphrase Retrieval (QPR) system is to retrieve\nequivalent questions that result in the same answer as the original question.\nSuch a system can be used to understand and answer rare and noisy\nreformulations of common questions by mapping them to a set of canonical forms.\nThis has large-scale applications for community Question Answering (cQA) and\nopen-domain spoken language question answering systems. In this paper we\ndescribe a new QPR system implemented as a Neural Information Retrieval (NIR)\nsystem consisting of a neural network sentence encoder and an approximate\nk-Nearest Neighbour index for efficient vector retrieval. We also describe our\nmechanism to generate an annotated dataset for question paraphrase retrieval\nexperiments automatically from question-answer logs via distant supervision. We\nshow that the standard loss function in NIR, triplet loss, does not perform\nwell with noisy labels. We propose smoothed deep metric loss (SDML) and with\nour experiments on two QPR datasets we show that it significantly outperforms\ntriplet loss in the noisy label setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.12790v2",
            "date": "2019-05-29 23:47:17+00:00",
            "title": "A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models",
            "authors": [
                "Elman Mansimov",
                "Alex Wang",
                "Sean Welleck",
                "Kyunghyun Cho"
            ],
            "abstract": "Undirected neural sequence models such as BERT (Devlin et al., 2019) have\nreceived renewed interest due to their success on discriminative natural\nlanguage understanding tasks such as question-answering and natural language\ninference. The problem of generating sequences directly from these models has\nreceived relatively little attention, in part because generating from\nundirected models departs significantly from conventional monotonic generation\nin directed sequence models. We investigate this problem by proposing a\ngeneralized model of sequence generation that unifies decoding in directed and\nundirected models. The proposed framework models the process of generation\nrather than the resulting sequence, and under this framework, we derive various\nneural sequence models as special cases, such as autoregressive,\nsemi-autoregressive, and refinement-based non-autoregressive models. This\nunification enables us to adapt decoding algorithms originally developed for\ndirected sequence models to undirected sequence models. We demonstrate this by\nevaluating various handcrafted and learned decoding strategies on a BERT-like\nmachine translation model (Lample & Conneau, 2019). The proposed approach\nachieves constant-time translation results on par with linear-time translation\nresults from the same undirected sequence model, while both are competitive\nwith the state-of-the-art on WMT'14 English-German translation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.12864v1",
            "date": "2019-05-30 05:55:58+00:00",
            "title": "Interpretable Adversarial Training for Text",
            "authors": [
                "Samuel Barham",
                "Soheil Feizi"
            ],
            "abstract": "Generating high-quality and interpretable adversarial examples in the text\ndomain is a much more daunting task than it is in the image domain. This is due\npartly to the discrete nature of text, partly to the problem of ensuring that\nthe adversarial examples are still probable and interpretable, and partly to\nthe problem of maintaining label invariance under input perturbations. In order\nto address some of these challenges, we introduce sparse projected gradient\ndescent (SPGD), a new approach to crafting interpretable adversarial examples\nfor text. SPGD imposes a directional regularization constraint on input\nperturbations by projecting them onto the directions to nearby word embeddings\nwith highest cosine similarities. This constraint ensures that perturbations\nmove each word embedding in an interpretable direction (i.e., towards another\nnearby word embedding). Moreover, SPGD imposes a sparsity constraint on\nperturbations at the sentence level by ignoring word-embedding perturbations\nwhose norms are below a certain threshold. This constraint ensures that our\nmethod changes only a few words per sequence, leading to higher quality\nadversarial examples. Our experiments with the IMDB movie review dataset show\nthat the proposed SPGD method improves adversarial example interpretability and\nlikelihood (evaluated by average per-word perplexity) compared to\nstate-of-the-art methods, while suffering little to no loss in training\nperformance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.13126v1",
            "date": "2019-05-18 00:44:47+00:00",
            "title": "Automatic Evaluation of Local Topic Quality",
            "authors": [
                "Jeffrey Lund",
                "Piper Armstrong",
                "Wilson Fearn",
                "Stephen Cowley",
                "Courtni Byun",
                "Jordan Boyd-Graber",
                "Kevin Seppi"
            ],
            "abstract": "Topic models are typically evaluated with respect to the global topic\ndistributions that they generate, using metrics such as coherence, but without\nregard to local (token-level) topic assignments. Token-level assignments are\nimportant for downstream tasks such as classification. Even recent models,\nwhich aim to improve the quality of these token-level topic assignments, have\nbeen evaluated only with respect to global metrics. We propose a task designed\nto elicit human judgments of token-level topic assignments. We use a variety of\ntopic model types and parameters and discover that global metrics agree poorly\nwith human assignments.\n  Since human evaluation is expensive we propose a variety of automated metrics\nto evaluate topic models at a local level. Finally, we correlate our proposed\nmetrics with human judgments from the task on several datasets. We show that an\nevaluation based on the percent of topic switches correlates most strongly with\nhuman judgment of local topic quality. We suggest that this new metric, which\nwe call consistency, be adopted alongside global metrics such as topic\ncoherence when evaluating new topic models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.13150v2",
            "date": "2019-05-30 16:12:26+00:00",
            "title": "Lattice-based lightly-supervised acoustic model training",
            "authors": [
                "Joachim Fainberg",
                "Ond\u0159ej Klejch",
                "Steve Renals",
                "Peter Bell"
            ],
            "abstract": "In the broadcast domain there is an abundance of related text data and\npartial transcriptions, such as closed captions and subtitles. This text data\ncan be used for lightly supervised training, in which text matching the audio\nis selected using an existing speech recognition model. Current approaches to\nlight supervision typically filter the data based on matching error rates\nbetween the transcriptions and biased decoding hypotheses. In contrast,\nsemi-supervised training does not require matching text data, instead\ngenerating a hypothesis using a background language model. State-of-the-art\nsemi-supervised training uses lattice-based supervision with the lattice-free\nMMI (LF-MMI) objective function. We propose a technique to combine inaccurate\ntranscriptions with the lattices generated for semi-supervised training, thus\npreserving uncertainty in the lattice where appropriate. We demonstrate that\nthis combined approach reduces the expected error rates over the lattices, and\nreduces the word error rate (WER) on a broadcast task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13153v2",
            "date": "2019-05-30 16:15:36+00:00",
            "title": "Grounding Language Attributes to Objects using Bayesian Eigenobjects",
            "authors": [
                "Vanya Cohen",
                "Benjamin Burchfiel",
                "Thao Nguyen",
                "Nakul Gopalan",
                "Stefanie Tellex",
                "George Konidaris"
            ],
            "abstract": "We develop a system to disambiguate object instances within the same class\nbased on simple physical descriptions. The system takes as input a natural\nlanguage phrase and a depth image containing a segmented object and predicts\nhow similar the observed object is to the object described by the phrase. Our\nsystem is designed to learn from only a small amount of human-labeled language\ndata and generalize to viewpoints not represented in the language-annotated\ndepth image training set. By decoupling 3D shape representation from language\nrepresentation, this method is able to ground language to novel objects using a\nsmall amount of language-annotated depth-data and a larger corpus of unlabeled\n3D object meshes, even when these objects are partially observed from unusual\nviewpoints. Our system is able to disambiguate between novel objects, observed\nvia depth images, based on natural language descriptions. Our method also\nenables view-point transfer; trained on human-annotated data on a small set of\ndepth images captured from frontal viewpoints, our system successfully\npredicted object attributes from rear views despite having no such depth images\nin its training set. Finally, we demonstrate our approach on a Baxter robot,\nenabling it to pick specific objects based on human-provided natural language\ndescriptions.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1905.13350v1",
            "date": "2019-05-30 23:17:26+00:00",
            "title": "Threshold-Based Retrieval and Textual Entailment Detection on Legal Bar Exam Questions",
            "authors": [
                "Sabine Wehnert",
                "Sayed Anisul Hoque",
                "Wolfram Fenske",
                "Gunter Saake"
            ],
            "abstract": "Getting an overview over the legal domain has become challenging, especially\nin a broad, international context. Legal question answering systems have the\npotential to alleviate this task by automatically retrieving relevant legal\ntexts for a specific statement and checking whether the meaning of the\nstatement can be inferred from the found documents. We investigate a\ncombination of the BM25 scoring method of Elasticsearch with word embeddings\ntrained on English translations of the German and Japanese civil law. For this,\nwe define criteria which select a dynamic number of relevant documents\naccording to threshold scores. Exploiting two deep learning classifiers and\ntheir respective prediction bias with a threshold-based answer inclusion\ncriterion has shown to be beneficial for the textual entailment task, when\ncompared to the baseline.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1905.13364v1",
            "date": "2019-05-31 00:36:21+00:00",
            "title": "Can We Derive Explicit and Implicit Bias from Corpus?",
            "authors": [
                "Bo Wang",
                "Baixiang Xue",
                "Anthony G. Greenwald"
            ],
            "abstract": "Language is a popular resource to mine speakers' attitude bias, supposing\nthat speakers' statements represent their bias on concepts. However, psychology\nstudies show that people's explicit bias in statements can be different from\ntheir implicit bias in mind. Although both explicit and implicit bias are\nuseful for different applications, current automatic techniques do not\ndistinguish them. Inspired by psychological measurements of explicit and\nimplicit bias, we develop an automatic language-based technique to reproduce\npsychological measurements on large population. By connecting each\npsychological measurement with the statements containing the certain\ncombination of special words, we derive explicit and implicit bias by\nunderstanding the sentiment of corresponding category of statements. Extensive\nexperiments on English and Chinese serious media (Wikipedia) and non-serious\nmedia (social media) show that our method successfully reproduce the\nsmall-scale psychological observations on large population and achieve new\nfindings.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.13448v2",
            "date": "2019-05-31 07:30:15+00:00",
            "title": "Audio Caption in a Car Setting with a Sentence-Level Loss",
            "authors": [
                "Xuenan Xu",
                "Heinrich Dinkel",
                "Mengyue Wu",
                "Kai Yu"
            ],
            "abstract": "Captioning has attracted much attention in image and video understanding\nwhile a small amount of work examines audio captioning. This paper contributes\na Mandarin-annotated dataset for audio captioning within a car scene. A\nsentence-level loss is proposed to be used in tandem with a GRU encoder-decoder\nmodel to generate captions with higher semantic similarity to human\nannotations. We evaluate the model on the newly-proposed Car dataset, a\npreviously published Mandarin Hospital dataset and the Joint dataset,\nindicating its generalization capability across different scenes. An\nimprovement in all metrics can be observed, including classical natural\nlanguage generation (NLG) metrics, sentence richness and human evaluation\nratings. However, though detailed audio captions can now be automatically\ngenerated, human annotations still outperform model captions on many aspects.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1905.13453v1",
            "date": "2019-05-31 08:05:31+00:00",
            "title": "MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension",
            "authors": [
                "Alon Talmor",
                "Jonathan Berant"
            ],
            "abstract": "A large number of reading comprehension (RC) datasets has been created\nrecently, but little analysis has been done on whether they generalize to one\nanother, and the extent to which existing datasets can be leveraged for\nimproving performance on new ones. In this paper, we conduct such an\ninvestigation over ten RC datasets, training on one or more source RC datasets,\nand evaluating generalization, as well as transfer to a target RC dataset. We\nanalyze the factors that contribute to generalization, and show that training\non a source RC dataset and transferring to a target dataset substantially\nimproves performance, even in the presence of powerful contextual\nrepresentations from BERT (Devlin et al., 2019). We also find that training on\nmultiple source RC datasets leads to robust generalization and transfer, and\ncan reduce the cost of example collection for a new RC dataset. Following our\nanalysis, we propose MultiQA, a BERT-based model, trained on multiple RC\ndatasets, which leads to state-of-the-art performance on five RC datasets. We\nshare our infrastructure for the benefit of the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13618v1",
            "date": "2019-05-31 13:54:54+00:00",
            "title": "Crowdsourcing and Validating Event-focused Emotion Corpora for German and English",
            "authors": [
                "Enrica Troiano",
                "Sebastian Pad\u00f3",
                "Roman Klinger"
            ],
            "abstract": "Sentiment analysis has a range of corpora available across multiple\nlanguages. For emotion analysis, the situation is more limited, which hinders\npotential research on cross-lingual modeling and the development of predictive\nmodels for other languages. In this paper, we fill this gap for German by\nconstructing deISEAR, a corpus designed in analogy to the well-established\nEnglish ISEAR emotion dataset. Motivated by Scherer's appraisal theory, we\nimplement a crowdsourcing experiment which consists of two steps. In step 1,\nparticipants create descriptions of emotional events for a given emotion. In\nstep 2, five annotators assess the emotion expressed by the texts. We show that\ntransferring an emotion classification model from the original English ISEAR to\nthe German crowdsourced deISEAR via machine translation does not, on average,\ncause a performance drop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00041v1",
            "date": "2019-05-31 19:22:29+00:00",
            "title": "Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval",
            "authors": [
                "Li Deng",
                "Shuo Zhang",
                "Krisztian Balog"
            ],
            "abstract": "Tables contain valuable knowledge in a structured form. We employ neural\nlanguage modeling approaches to embed tabular data into vector spaces.\nSpecifically, we consider different table elements, such caption, column\nheadings, and cells, for training word and entity embeddings. These embeddings\nare then utilized in three particular table-related tasks, row population,\ncolumn population, and table retrieval, by incorporating them into existing\nretrieval models as additional semantic similarity signals. Evaluation results\nshow that table embeddings can significantly improve upon the performance of\nstate-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.00180v1",
            "date": "2019-06-01 08:17:42+00:00",
            "title": "Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization",
            "authors": [
                "Mathijs Mul",
                "Willem Zuidema"
            ],
            "abstract": "Can neural nets learn logic? We approach this classic question with current\nmethods, and demonstrate that recurrent neural networks can learn to recognize\nfirst order logical entailment relations between expressions. We define an\nartificial language in first-order predicate logic, generate a large dataset of\nsample 'sentences', and use an automatic theorem prover to infer the relation\nbetween random pairs of such sentences. We describe a Siamese neural\narchitecture trained to predict the logical relation, and experiment with\nrecurrent and recursive networks. Siamese Recurrent Networks are surprisingly\nsuccessful at the entailment recognition task, reaching near perfect\nperformance on novel sentences (consisting of known words), and even\noutperforming recursive networks. We report a series of experiments to test the\nability of the models to perform compositional generalization. In particular,\nwe study how they deal with sentences of unseen length, and sentences\ncontaining unseen words. We show that set-ups using LSTMs and GRUs obtain high\nscores on these tests, demonstrating a form of compositionality.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.00282v1",
            "date": "2019-06-01 20:07:11+00:00",
            "title": "Biomedical Named Entity Recognition via Reference-Set Augmented Bootstrapping",
            "authors": [
                "Joel Mathew",
                "Shobeir Fakhraei",
                "Jos\u00e9 Luis Ambite"
            ],
            "abstract": "We present a weakly-supervised data augmentation approach to improve Named\nEntity Recognition (NER) in a challenging domain: extracting biomedical\nentities (e.g., proteins) from the scientific literature. First, we train a\nneural NER (NNER) model over a small seed of fully-labeled examples. Second, we\nuse a reference set of entity names (e.g., proteins in UniProt) to identify\nentity mentions with high precision, but low recall, on an unlabeled corpus.\nThird, we use the NNER model to assign weak labels to the corpus. Finally, we\nretrain our NNER model iteratively over the augmented training set, including\nthe seed, the reference-set examples, and the weakly-labeled examples, which\nimproves model performance. We show empirically that this augmented\nbootstrapping process significantly improves NER performance, and discuss the\nfactors impacting the efficacy of the approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.00283v3",
            "date": "2019-06-01 20:21:24+00:00",
            "title": "Learning to Generate Grounded Visual Captions without Localization Supervision",
            "authors": [
                "Chih-Yao Ma",
                "Yannis Kalantidis",
                "Ghassan AlRegib",
                "Peter Vajda",
                "Marcus Rohrbach",
                "Zsolt Kira"
            ],
            "abstract": "When automatically generating a sentence description for an image or video,\nit often remains unclear how well the generated caption is grounded, that is\nwhether the model uses the correct image regions to output particular words, or\nif the model is hallucinating based on priors in the dataset and/or the\nlanguage model. The most common way of relating image regions with words in\ncaption models is through an attention mechanism over the regions that are used\nas input to predict the next word. The model must therefore learn to predict\nthe attentional weights without knowing the word it should localize. This is\ndifficult to train without grounding supervision since recurrent models can\npropagate past information and there is no explicit signal to force the\ncaptioning model to properly ground the individual decoded words. In this work,\nwe help the model to achieve this via a novel cyclical training regimen that\nforces the model to localize each word in the image after the sentence decoder\ngenerates it, and then reconstruct the sentence from the localized image\nregion(s) to match the ground-truth. Our proposed framework only requires\nlearning one extra fully-connected layer (the localizer), a layer that can be\nremoved at test time. We show that our model significantly improves grounding\naccuracy without relying on grounding supervision or introducing extra\ncomputation during inference, for both image and video captioning tasks. Code\nis available at https://github.com/chihyaoma/cyclical-visual-captioning .",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.00346v2",
            "date": "2019-06-02 05:11:38+00:00",
            "title": "Pre-training of Graph Augmented Transformers for Medication Recommendation",
            "authors": [
                "Junyuan Shang",
                "Tengfei Ma",
                "Cao Xiao",
                "Jimeng Sun"
            ],
            "abstract": "Medication recommendation is an important healthcare application. It is\ncommonly formulated as a temporal prediction task. Hence, most existing works\nonly utilize longitudinal electronic health records (EHRs) from a small number\nof patients with multiple visits ignoring a large number of patients with a\nsingle visit (selection bias). Moreover, important hierarchical knowledge such\nas diagnosis hierarchy is not leveraged in the representation learning process.\nTo address these challenges, we propose G-BERT, a new model to combine the\npower of Graph Neural Networks (GNNs) and BERT (Bidirectional Encoder\nRepresentations from Transformers) for medical code representation and\nmedication recommendation. We use GNNs to represent the internal hierarchical\nstructures of medical codes. Then we integrate the GNN representation into a\ntransformer-based visit encoder and pre-train it on EHR data from patients only\nwith a single visit. The pre-trained visit encoder and representation are then\nfine-tuned for downstream predictive tasks on longitudinal EHRs from patients\nwith multiple visits. G-BERT is the first to bring the language model\npre-training schema into the healthcare domain and it achieved state-of-the-art\nperformance on the medication recommendation task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.00499v1",
            "date": "2019-06-02 22:53:33+00:00",
            "title": "Budgeted Policy Learning for Task-Oriented Dialogue Systems",
            "authors": [
                "Zhirui Zhang",
                "Xiujun Li",
                "Jianfeng Gao",
                "Enhong Chen"
            ],
            "abstract": "This paper presents a new approach that extends Deep Dyna-Q (DDQ) by\nincorporating a Budget-Conscious Scheduling (BCS) to best utilize a fixed,\nsmall amount of user interactions (budget) for learning task-oriented dialogue\nagents. BCS consists of (1) a Poisson-based global scheduler to allocate budget\nover different stages of training; (2) a controller to decide at each training\nstep whether the agent is trained using real or simulated experiences; (3) a\nuser goal sampling module to generate the experiences that are most effective\nfor policy learning. Experiments on a movie-ticket booking task with simulated\nand real users show that our approach leads to significant improvements in\nsuccess rate over the state-of-the-art baselines given the fixed budget.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00579v3",
            "date": "2019-06-03 05:25:42+00:00",
            "title": "Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain",
            "authors": [
                "Johanes Effendi",
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "Previously, a machine speech chain, which is based on sequence-to-sequence\ndeep learning, was proposed to mimic speech perception and production behavior.\nSuch chains separately processed listening and speaking by automatic speech\nrecognition (ASR) and text-to-speech synthesis (TTS) and simultaneously enabled\nthem to teach each other in semi-supervised learning when they received\nunpaired data. Unfortunately, this speech chain study is limited to speech and\ntextual modalities. In fact, natural communication is actually multimodal and\ninvolves both auditory and visual sensory systems. Although the said speech\nchain reduces the requirement of having a full amount of paired data, in this\ncase we still need a large amount of unpaired data. In this research, we take a\nfurther step and construct a multimodal chain and design a closely knit chain\narchitecture that combines ASR, TTS, image captioning, and image production\nmodels into a single framework. The framework allows the training of each\ncomponent without requiring a large number of parallel multimodal data. Our\nexperimental results also show that an ASR can be further trained without\nspeech and text data and cross-modal data augmentation remains possible through\nour proposed chain, which improves the ASR performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00580v1",
            "date": "2019-06-03 05:27:05+00:00",
            "title": "Massive Styles Transfer with Limited Labeled Data",
            "authors": [
                "Hongyu Zang",
                "Xiaojun Wan"
            ],
            "abstract": "Language style transfer has attracted more and more attention in the past few\nyears. Recent researches focus on improving neural models targeting at\ntransferring from one style to the other with labeled data. However,\ntransferring across multiple styles is often very useful in real-life\napplications. Previous researches of language style transfer have two main\ndeficiencies: dependency on massive labeled data and neglect of mutual\ninfluence among different style transfer tasks. In this paper, we propose a\nmulti-agent style transfer system (MAST) for addressing multiple style transfer\ntasks with limited labeled data, by leveraging abundant unlabeled data and the\nmutual benefit among the multiple styles. A style transfer agent in our system\nnot only learns from unlabeled data by using techniques like denoising\nauto-encoder and back-translation, but also learns to cooperate with other\nstyle transfer agents in a self-organization manner. We conduct our experiments\nby simulating a set of real-world style transfer tasks with multiple versions\nof the Bible. Our model significantly outperforms the other competitive\nmethods. Extensive results and analysis further verify the efficacy of our\nproposed system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00671v1",
            "date": "2019-06-03 09:52:17+00:00",
            "title": "Unsupervised Neural Generative Semantic Hashing",
            "authors": [
                "Casper Hansen",
                "Christian Hansen",
                "Jakob Grue Simonsen",
                "Stephen Alstrup",
                "Christina Lioma"
            ],
            "abstract": "Fast similarity search is a key component in large-scale information\nretrieval, where semantic hashing has become a popular strategy for\nrepresenting documents as binary hash codes. Recent advances in this area have\nbeen obtained through neural network based models: generative models trained by\nlearning to reconstruct the original documents. We present a novel unsupervised\ngenerative semantic hashing approach, \\textit{Ranking based Semantic Hashing}\n(RBSH) that consists of both a variational and a ranking based component.\nSimilarly to variational autoencoders, the variational component is trained to\nreconstruct the original document conditioned on its generated hash code, and\nas in prior work, it only considers documents individually. The ranking\ncomponent solves this limitation by incorporating inter-document similarity\ninto the hash code generation, modelling document ranking through a hinge loss.\nTo circumvent the need for labelled data to compute the hinge loss, we use a\nweak labeller and thus keep the approach fully unsupervised.\n  Extensive experimental evaluation on four publicly available datasets against\ntraditional baselines and recent state-of-the-art methods for semantic hashing\nshows that RBSH significantly outperforms all other methods across all\nevaluated hash code lengths. In fact, RBSH hash codes are able to perform\nsimilarly to state-of-the-art hash codes while using 2-4x fewer bits.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.00672v3",
            "date": "2019-06-03 09:52:19+00:00",
            "title": "Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS",
            "authors": [
                "Mutian He",
                "Yan Deng",
                "Lei He"
            ],
            "abstract": "Neural TTS has demonstrated strong capabilities to generate human-like speech\nwith high quality and naturalness, while its generalization to out-of-domain\ntexts is still a challenging task, with regard to the design of attention-based\nsequence-to-sequence acoustic modeling. Various errors occur in those inputs\nwith unseen context, including attention collapse, skipping, repeating, etc.,\nwhich limits the broader applications. In this paper, we propose a novel\nstepwise monotonic attention method in sequence-to-sequence acoustic modeling\nto improve the robustness on out-of-domain inputs. The method utilizes the\nstrict monotonic property in TTS with constraints on monotonic hard attention\nthat the alignments between inputs and outputs sequence must be not only\nmonotonic but allowing no skipping on inputs. Soft attention could be used to\nevade mismatch between training and inference. The experimental results show\nthat the proposed method could achieve significant improvements in robustness\non out-of-domain scenarios for phoneme-based models, without any regression on\nthe in-domain naturalness test.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00687v1",
            "date": "2019-06-03 10:13:32+00:00",
            "title": "Relation Embedding with Dihedral Group in Knowledge Graph",
            "authors": [
                "Canran Xu",
                "Ruijiang Li"
            ],
            "abstract": "Link prediction is critical for the application of incomplete knowledge graph\n(KG) in the downstream tasks. As a family of effective approaches for link\npredictions, embedding methods try to learn low-rank representations for both\nentities and relations such that the bilinear form defined therein is a\nwell-behaved scoring function. Despite of their successful performances,\nexisting bilinear forms overlook the modeling of relation compositions,\nresulting in lacks of interpretability for reasoning on KG. To fulfill this\ngap, we propose a new model called DihEdral, named after dihedral symmetry\ngroup. This new model learns knowledge graph embeddings that can capture\nrelation compositions by nature. Furthermore, our approach models the relation\nembeddings parametrized by discrete values, thereby decrease the solution space\ndrastically. Our experiments show that DihEdral is able to capture all desired\nproperties such as (skew-) symmetry, inversion and (non-) Abelian composition,\nand outperforms existing bilinear form based approach and is comparable to or\nbetter than deep learning models such as ConvE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00800v1",
            "date": "2019-06-03 13:38:20+00:00",
            "title": "Neural Network-based Object Classification by Known and Unknown Features (Based on Text Queries)",
            "authors": [
                "A. Artemov",
                "I. Bolokhov",
                "D. Kem",
                "I. Khasenevich"
            ],
            "abstract": "The article presents a method that improves the quality of classification of\nobjects described by a combination of known and unknown features. The method is\nbased on modernized Informational Neurobayesian Approach with consideration of\nunknown features. The proposed method was developed and trained on 1500 text\nqueries of Promobot users in Russian to classify them into 20 categories\n(classes). As a result, the use of the method allowed to completely solve the\nproblem of misclassification for queries with combining known and unknown\nfeatures of the model. The theoretical substantiation of the method is\npresented by the formulated and proved theorem On the Model with Limited\nKnowledge. It states, that in conditions of limited data, an equal number of\nequally unknown features of an object cannot have different significance for\nthe classification problem.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.00872v1",
            "date": "2019-06-03 15:28:48+00:00",
            "title": "From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots",
            "authors": [
                "Shizhe Chen",
                "Qin Jin",
                "Jianlong Fu"
            ],
            "abstract": "The neural machine translation model has suffered from the lack of\nlarge-scale parallel corpora. In contrast, we humans can learn multi-lingual\ntranslations even without parallel texts by referring our languages to the\nexternal world. To mimic such human learning behavior, we employ images as\npivots to enable zero-resource translation learning. However, a picture tells a\nthousand words, which makes multi-lingual sentences pivoted by the same image\nnoisy as mutual translations and thus hinders the translation model learning.\nIn this work, we propose a progressive learning approach for image-pivoted\nzero-resource machine translation. Since words are less diverse when grounded\nin the image, we first learn word-level translation with image pivots, and then\nprogress to learn the sentence-level translation by utilizing the learned word\ntranslation to suppress noises in image-pivoted multi-lingual sentences.\nExperimental results on two widely used image-pivot translation datasets,\nIAPR-TC12 and Multi30k, show that the proposed approach significantly\noutperforms other state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00908v1",
            "date": "2019-06-03 16:20:04+00:00",
            "title": "Phase-based Minimalist Parsing and complexity in non-local dependencies",
            "authors": [
                "Cristiano Chesi"
            ],
            "abstract": "A cognitively plausible parsing algorithm should perform like the human\nparser in critical contexts. Here I propose an adaptation of Earley's parsing\nalgorithm, suitable for Phase-based Minimalist Grammars (PMG, Chesi 2012), that\nis able to predict complexity effects in performance. Focusing on self-paced\nreading experiments of object clefts sentences (Warren & Gibson 2005) I will\nassociate to parsing a complexity metric based on cued features to be retrieved\nat the verb segment (Feature Retrieval & Encoding Cost, FREC). FREC is\ncrucially based on the usage of memory predicted by the discussed parsing\nalgorithm and it correctly fits with the reading time revealed.",
            "journal ref": "Proceedings of CLiC-it 2017. CEUR WORKSHOP PROCEEDINGS, ROMA:CEUR\n  Workshop Proceedings, ISBN: 9788899982768, ISSN: 1613-0073, Rome, 11-13\n  December 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.00923v2",
            "date": "2019-05-26 07:06:58+00:00",
            "title": "TACAM: Topic And Context Aware Argument Mining",
            "authors": [
                "Michael Fromm",
                "Evgeniy Faerman",
                "Thomas Seidl"
            ],
            "abstract": "In this work we address the problem of argument search. The purpose of\nargument search is the distillation of pro and contra arguments for requested\ntopics from large text corpora. In previous works, the usual approach is to use\na standard search engine to extract text parts which are relevant to the given\ntopic and subsequently use an argument recognition algorithm to select\narguments from them. The main challenge in the argument recognition task, which\nis also known as argument mining, is that often sentences containing arguments\nare structurally similar to purely informative sentences without any stance\nabout the topic. In fact, they only differ semantically. Most approaches use\ntopic or search term information only for the first search step and therefore\nassume that arguments can be classified independently of a topic. We argue that\ntopic information is crucial for argument mining, since the topic defines the\nsemantic context of an argument. Precisely, we propose different models for the\nclassification of arguments, which take information about a topic of an\nargument into account. Moreover, to enrich the context of a topic and to let\nmodels understand the context of the potential argument better, we integrate\ninformation from different external sources such as Knowledge Graphs or\npre-trained NLP models. Our evaluation shows that considering topic\ninformation, especially in connection with external information, provides a\nsignificant performance boost for the argument mining task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01032v1",
            "date": "2019-06-03 19:21:42+00:00",
            "title": "A Language-Agnostic Model for Semantic Source Code Labeling",
            "authors": [
                "Ben Gelman",
                "Bryan Hoyle",
                "Jessica Moore",
                "Joshua Saxe",
                "David Slater"
            ],
            "abstract": "Code search and comprehension have become more difficult in recent years due\nto the rapid expansion of available source code. Current tools lack a way to\nlabel arbitrary code at scale while maintaining up-to-date representations of\nnew programming languages, libraries, and functionalities. Comprehensive\nlabeling of source code enables users to search for documents of interest and\nobtain a high-level understanding of their contents. We use Stack Overflow code\nsnippets and their tags to train a language-agnostic, deep convolutional neural\nnetwork to automatically predict semantic labels for source code documents. On\nStack Overflow code snippets, we demonstrate a mean area under ROC of 0.957\nover a long-tailed list of 4,508 tags. We also manually validate the model\noutputs on a diverse set of unlabeled source code documents retrieved from\nGithub, and we obtain a top-1 accuracy of 86.6%. This strongly indicates that\nthe model successfully transfers its knowledge from Stack Overflow snippets to\narbitrary source code documents.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.01076v3",
            "date": "2019-06-03 20:50:58+00:00",
            "title": "Episodic Memory in Lifelong Language Learning",
            "authors": [
                "Cyprien de Masson d'Autume",
                "Sebastian Ruder",
                "Lingpeng Kong",
                "Dani Yogatama"
            ],
            "abstract": "We introduce a lifelong language learning setup where a model needs to learn\nfrom a stream of text examples without any dataset identifier. We propose an\nepisodic memory model that performs sparse experience replay and local\nadaptation to mitigate catastrophic forgetting in this setup. Experiments on\ntext classification and question answering demonstrate the complementary\nbenefits of sparse experience replay and local adaptation to allow the model to\ncontinuously learn from new datasets. We also show that the space complexity of\nthe episodic memory module can be reduced significantly (~50-90%) by randomly\nchoosing which examples to store in memory with a minimal decrease in\nperformance. We consider an episodic memory component as a crucial building\nblock of general linguistic intelligence and see our model as a first step in\nthat direction.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.01105v2",
            "date": "2019-06-03 22:33:22+00:00",
            "title": "Training Neural Machine Translation To Apply Terminology Constraints",
            "authors": [
                "Georgiana Dinu",
                "Prashant Mathur",
                "Marcello Federico",
                "Yaser Al-Onaizan"
            ],
            "abstract": "This paper proposes a novel method to inject custom terminology into neural\nmachine translation at run time. Previous works have mainly proposed\nmodifications to the decoding algorithm in order to constrain the output to\ninclude run-time-provided target terms. While being effective, these\nconstrained decoding methods add, however, significant computational overhead\nto the inference step, and, as we show in this paper, can be brittle when\ntested in realistic conditions. In this paper we approach the problem by\ntraining a neural MT system to learn how to use custom terminology when\nprovided with the input. Comparative experiments show that our method is not\nonly more effective than a state-of-the-art implementation of constrained\ndecoding, but is also as fast as constraint-free decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01155v3",
            "date": "2019-06-04 01:58:58+00:00",
            "title": "ShEMO -- A Large-Scale Validated Database for Persian Speech Emotion Detection",
            "authors": [
                "Omid Mohamad Nezami",
                "Paria Jamshid Lou",
                "Mansoureh Karami"
            ],
            "abstract": "This paper introduces a large-scale, validated database for Persian called\nSharif Emotional Speech Database (ShEMO). The database includes 3000\nsemi-natural utterances, equivalent to 3 hours and 25 minutes of speech data\nextracted from online radio plays. The ShEMO covers speech samples of 87\nnative-Persian speakers for five basic emotions including anger, fear,\nhappiness, sadness and surprise, as well as neutral state. Twelve annotators\nlabel the underlying emotional state of utterances and majority voting is used\nto decide on the final labels. According to the kappa measure, the\ninter-annotator agreement is 64% which is interpreted as \"substantial\nagreement\". We also present benchmark results based on common classification\nmethods in speech emotion detection task. According to the experiments, support\nvector machine achieves the best results for both gender-independent (58.2%)\nand gender-dependent models (female=59.4%, male=57.6%). The ShEMO is available\nfor academic purposes free of charge to provide a baseline for further research\non Persian emotional speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01157v2",
            "date": "2019-06-04 02:17:18+00:00",
            "title": "A Review of Automated Speech and Language Features for Assessment of Cognitive and Thought Disorders",
            "authors": [
                "Rohit Voleti",
                "Julie M. Liss",
                "Visar Berisha"
            ],
            "abstract": "It is widely accepted that information derived from analyzing speech (the\nacoustic signal) and language production (words and sentences) serves as a\nuseful window into the health of an individual's cognitive ability. In fact,\nmost neuropsychological testing batteries have a component related to speech\nand language where clinicians elicit speech from patients for subjective\nevaluation across a broad set of dimensions. With advances in speech signal\nprocessing and natural language processing, there has been recent interest in\ndeveloping tools to detect more subtle changes in cognitive-linguistic\nfunction. This work relies on extracting a set of features from recorded and\ntranscribed speech for objective assessments of speech and language, early\ndiagnosis of neurological disease, and tracking of disease after diagnosis.\nWith an emphasis on cognitive and thought disorders, in this paper we provide a\nreview of existing speech and language features used in this domain, discuss\ntheir clinical application, and highlight their advantages and disadvantages.\nBroadly speaking, the review is split into two categories: language features\nbased on natural language processing and speech features based on speech signal\nprocessing. Within each category, we consider features that aim to measure\ncomplementary dimensions of cognitive-linguistics, including language\ndiversity, syntactic complexity, semantic coherence, and timing. We conclude\nthe review with a proposal of new research directions to further advance the\nfield.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01161v2",
            "date": "2019-06-03 11:10:10+00:00",
            "title": "Resolving Gendered Ambiguous Pronouns with BERT",
            "authors": [
                "Matei Ionita",
                "Yury Kashnitsky",
                "Ken Krige",
                "Vladimir Larin",
                "Denis Logvinenko",
                "Atanas Atanasov"
            ],
            "abstract": "Pronoun resolution is part of coreference resolution, the task of pairing an\nexpression to its referring entity. This is an important task for natural\nlanguage understanding and a necessary component of machine translation\nsystems, chat bots and assistants. Neural machine learning systems perform far\nfrom ideally in this task, reaching as low as 73% F1 scores on modern benchmark\ndatasets. Moreover, they tend to perform better for masculine pronouns than for\nfeminine ones. Thus, the problem is both challenging and important for NLP\nresearchers and practitioners. In this project, we describe our BERT-based\napproach to solving the problem of gender-balanced pronoun resolution. We are\nable to reach 92% F1 score and a much lower gender bias on the benchmark\ndataset shared by Google AI Language team.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01195v1",
            "date": "2019-06-04 04:59:08+00:00",
            "title": "Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs",
            "authors": [
                "Deepak Nathani",
                "Jatin Chauhan",
                "Charu Sharma",
                "Manohar Kaul"
            ],
            "abstract": "The recent proliferation of knowledge graphs (KGs) coupled with incomplete or\npartial information, in the form of missing relations (links) between entities,\nhas fueled a lot of research on knowledge base completion (also known as\nrelation prediction). Several recent works suggest that convolutional neural\nnetwork (CNN) based models generate richer and more expressive feature\nembeddings and hence also perform well on relation prediction. However, we\nobserve that these KG embeddings treat triples independently and thus fail to\ncover the complex and hidden information that is inherently implicit in the\nlocal neighborhood surrounding a triple. To this effect, our paper proposes a\nnovel attention based feature embedding that captures both entity and relation\nfeatures in any given entity's neighborhood. Additionally, we also encapsulate\nrelation clusters and multihop relations in our model. Our empirical study\noffers insights into the efficacy of our attention based model and we show\nmarked performance gains in comparison to state of the art methods on all\ndatasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.01199v1",
            "date": "2019-06-04 05:12:31+00:00",
            "title": "Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation",
            "authors": [
                "Elizabeth Salesky",
                "Matthias Sperber",
                "Alan W Black"
            ],
            "abstract": "Previous work on end-to-end translation from speech has primarily used\nframe-level features as speech representations, which creates longer, sparser\nsequences than text. We show that a naive method to create compressed\nphoneme-like speech representations is far more effective and efficient for\ntranslation than traditional frame-level speech features. Specifically, we\ngenerate phoneme labels for speech frames and average consecutive frames with\nthe same label to create shorter, higher-level source sequences for\ntranslation. We see improvements of up to 5 BLEU on both our high and low\nresource language pairs, with a reduction in training time of 60%. Our\nimprovements hold across multiple data sizes and two language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01205v1",
            "date": "2019-06-04 05:42:58+00:00",
            "title": "A Strong and Robust Baseline for Text-Image Matching",
            "authors": [
                "Fangyu Liu",
                "Rongtian Ye"
            ],
            "abstract": "We review the current schemes of text-image matching models and propose\nimprovements for both training and inference. First, we empirically show\nlimitations of two popular loss (sum and max-margin loss) widely used in\ntraining text-image embeddings and propose a trade-off: a kNN-margin loss which\n1) utilizes information from hard negatives and 2) is robust to noise as all\n$K$-most hardest samples are taken into account, tolerating \\emph{pseudo}\nnegatives and outliers. Second, we advocate the use of Inverted Softmax\n(\\textsc{Is}) and Cross-modal Local Scaling (\\textsc{Csls}) during inference to\nmitigate the so-called hubness problem in high-dimensional embedding space,\nenhancing scores of all metrics by a large margin.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.01250v1",
            "date": "2019-06-04 07:49:46+00:00",
            "title": "Boosting Entity Linking Performance by Leveraging Unlabeled Documents",
            "authors": [
                "Phong Le",
                "Ivan Titov"
            ],
            "abstract": "Modern entity linking systems rely on large collections of documents\nspecifically annotated for the task (e.g., AIDA CoNLL). In contrast, we propose\nan approach which exploits only naturally occurring information: unlabeled\ndocuments and Wikipedia. Our approach consists of two stages. First, we\nconstruct a high recall list of candidate entities for each mention in an\nunlabeled document. Second, we use the candidate lists as weak supervision to\nconstrain our document-level entity linking model. The model treats entities as\nlatent variables and, when estimated on a collection of unlabelled texts,\nlearns to choose entities relying both on local context of each mention and on\ncoherence with other entities in the document. The resulting approach rivals\nfully-supervised state-of-the-art systems on standard test sets. It also\napproaches their performance in the very challenging setting: when tested on a\ntest set sampled from the data used to estimate the supervised systems. By\ncomparing to Wikipedia-only training of our model, we demonstrate that modeling\nunlabeled documents is beneficial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01440v1",
            "date": "2019-06-04 13:54:47+00:00",
            "title": "Tracing Antisemitic Language Through Diachronic Embedding Projections: France 1789-1914",
            "authors": [
                "Rocco Tripodi",
                "Massimo Warglien",
                "Simon Levis Sullam",
                "Deborah Paci"
            ],
            "abstract": "We investigate some aspects of the history of antisemitism in France, one of\nthe cradles of modern antisemitism, using diachronic word embeddings. We\nconstructed a large corpus of French books and periodicals issues that contain\na keyword related to Jews and performed a diachronic word embedding over the\n1789-1914 period. We studied the changes over time in the semantic spaces of 4\ntarget words and performed embedding projections over 6 streams of antisemitic\ndiscourse. This allowed us to track the evolution of antisemitic bias in the\nreligious, economic, socio-politic, racial, ethic and conspiratorial domains.\nProjections show a trend of growing antisemitism, especially in the years\nstarting in the mid-80s and culminating in the Dreyfus affair. Our analysis\nalso allows us to highlight the peculiar adverse bias towards Judaism in the\nbroader context of other religions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01496v1",
            "date": "2019-05-29 13:27:11+00:00",
            "title": "Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains",
            "authors": [
                "Navid Rekabsaz",
                "Nikolaos Pappas",
                "James Henderson",
                "Banriskhem K. Khonglah",
                "Srikanth Madikeri"
            ],
            "abstract": "Neural language modeling (LM) has led to significant improvements in several\napplications, including Automatic Speech Recognition. However, they typically\nrequire large amounts of training data, which is not available for many domains\nand languages. In this study, we propose a multilingual neural language model\narchitecture, trained jointly on the domain-specific data of several\nlow-resource languages. The proposed multilingual LM consists of language\nspecific word embeddings in the encoder and decoder, and one language specific\nLSTM layer, plus two LSTM layers with shared parameters across the languages.\nThis multilingual LM model facilitates transfer learning across the languages,\nacting as an extra regularizer in very low-resource scenarios. We integrate our\nproposed multilingual approach with a state-of-the-art highly-regularized\nneural LM, and evaluate on the conversational data domain for four languages\nover a range of training data sizes. Compared to monolingual LMs, the results\nshow significant improvements of our proposed multilingual LM when the amount\nof available training data is limited, indicating the advantages of\ncross-lingual parameter sharing in very low-resource language modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01498v1",
            "date": "2019-05-31 20:25:06+00:00",
            "title": "Multimodal Ensemble Approach to Incorporate Various Types of Clinical Notes for Predicting Readmission",
            "authors": [
                "Bonggun Shin",
                "Julien Hogan",
                "Andrew B. Adams",
                "Raymond J. Lynch",
                "Rachel E. Patzer",
                "Jinho D. Choi"
            ],
            "abstract": "Electronic Health Records (EHRs) have been heavily used to predict various\ndownstream clinical tasks such as readmission or mortality. One of the\nmodalities in EHRs, clinical notes, has not been fully explored for these tasks\ndue to its unstructured and inexplicable nature. Although recent advances in\ndeep learning (DL) enables models to extract interpretable features from\nunstructured data, they often require a large amount of training data. However,\nmany tasks in medical domains inherently consist of small sample data with\nlengthy documents; for a kidney transplant as an example, data from only a few\nthousand of patients are available and each patient's document consists of a\ncouple of millions of words in major hospitals. Thus, complex DL methods cannot\nbe applied to these kinds of domains. In this paper, we present a comprehensive\nensemble model using vector space modeling and topic modeling. Our proposed\nmodel is evaluated on the readmission task of kidney transplant patients and\nimproves 0.0211 in terms of c-statistics from the previous state-of-the-art\napproach using structured data, while typical DL methods fail to beat this\napproach. The proposed architecture provides the interpretable score for each\nfeature from both modalities, structured and unstructured data, which is shown\nto be meaningful through a physician's evaluation.",
            "journal ref": "Proceedings of the IEEE-EMBS International Conference on\n  Biomedical and Health Informatics, 2019 (BHI'19)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01502v1",
            "date": "2019-06-04 15:12:47+00:00",
            "title": "How multilingual is Multilingual BERT?",
            "authors": [
                "Telmo Pires",
                "Eva Schlinger",
                "Dan Garrette"
            ],
            "abstract": "In this paper, we show that Multilingual BERT (M-BERT), released by Devlin et\nal. (2018) as a single language model pre-trained from monolingual corpora in\n104 languages, is surprisingly good at zero-shot cross-lingual model transfer,\nin which task-specific annotations in one language are used to fine-tune the\nmodel for evaluation in another language. To understand why, we present a large\nnumber of probing experiments, showing that transfer is possible even to\nlanguages in different scripts, that transfer works best between typologically\nsimilar languages, that monolingual corpora can train models for\ncode-switching, and that the model can find translation pairs. From these\nresults, we can conclude that M-BERT does create multilingual representations,\nbut that these representations exhibit systematic deficiencies affecting\ncertain language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01515v1",
            "date": "2019-05-29 09:31:26+00:00",
            "title": "TMLab SRPOL at SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums",
            "authors": [
                "Piotr Niewinski",
                "Aleksander Wawer",
                "Maria Pszona",
                "Maria Janicka"
            ],
            "abstract": "The article describes our submission to SemEval 2019 Task 8 on Fact-Checking\nin Community Forums. The systems under discussion participated in Subtask A:\ndecide whether a question asks for factual information, opinion/advice or is\njust socializing. Our primary submission was ranked as the second one among all\nparticipants in the official evaluation phase. The article presents our primary\nsolution: Deeply Regularized Residual Neural Network (DRR NN) with Universal\nSentence Encoder embeddings. This is followed by a description of two\ncontrastive solutions based on ensemble methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01530v2",
            "date": "2019-06-04 15:41:32+00:00",
            "title": "The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue",
            "authors": [
                "Janosch Haber",
                "Tim Baumg\u00e4rtner",
                "Ece Takmaz",
                "Lieke Gelderloos",
                "Elia Bruni",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "This paper introduces the PhotoBook dataset, a large-scale collection of\nvisually-grounded, task-oriented dialogues in English designed to investigate\nshared dialogue history accumulating during conversation. Taking inspiration\nfrom seminal work on dialogue analysis, we propose a data-collection task\nformulated as a collaborative game prompting two online participants to refer\nto images utilising both their visual context as well as previously established\nreferring expressions. We provide a detailed description of the task setup and\na thorough analysis of the 2,500 dialogues collected. To further illustrate the\nnovel features of the dataset, we propose a baseline model for reference\nresolution which uses a simple method to take into account shared information\naccumulated in a reference chain. Our results show that this information is\nparticularly important to resolve later descriptions and underline the need to\ndevelop more sophisticated models of common ground in dialogue interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01539v2",
            "date": "2019-06-04 15:52:46+00:00",
            "title": "Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains",
            "authors": [
                "Samira Abnar",
                "Lisa Beinborn",
                "Rochelle Choenni",
                "Willem Zuidema"
            ],
            "abstract": "In this paper, we define and apply representational stability analysis\n(ReStA), an intuitive way of analyzing neural language models. ReStA is a\nvariant of the popular representational similarity analysis (RSA) in cognitive\nneuroscience. While RSA can be used to compare representations in models, model\ncomponents, and human brains, ReStA compares instances of the same model, while\nsystematically varying single model parameter. Using ReStA, we study four\nrecent and successful neural language models, and evaluate how sensitive their\ninternal representations are to the amount of prior context. Using RSA, we\nperform a systematic study of how similar the representational spaces in the\nfirst and second (or higher) layers of these models are to each other and to\npatterns of activation in the human brain. Our results reveal surprisingly\nstrong differences between language models, and give insights into where the\ndeep linguistic processing, that integrates information over multiple\nsentences, is happening in these models. The combination of ReStA and RSA on\nmodels and brains allows us to start addressing the important question of what\nkind of linguistic processes we can hope to observe in fMRI brain imaging data.\nIn particular, our results suggest that the data on story reading from Wehbe et\nal. (2014) contains a signal of shallow linguistic processing, but show no\nevidence on the more interesting deep linguistic processing.",
            "journal ref": "2nd BlackBoxNLP workshop @ACL2019",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.01545v4",
            "date": "2019-06-04 16:03:18+00:00",
            "title": "Optimal coding and the origins of Zipfian laws",
            "authors": [
                "Ramon Ferrer-i-Cancho",
                "Christian Bentz",
                "Caio Seguin"
            ],
            "abstract": "The problem of compression in standard information theory consists of\nassigning codes as short as possible to numbers. Here we consider the problem\nof optimal coding -- under an arbitrary coding scheme -- and show that it\npredicts Zipf's law of abbreviation, namely a tendency in natural languages for\nmore frequent words to be shorter. We apply this result to investigate optimal\ncoding also under so-called non-singular coding, a scheme where unique\nsegmentation is not warranted but codes stand for a distinct number. Optimal\nnon-singular coding predicts that the length of a word should grow\napproximately as the logarithm of its frequency rank, which is again consistent\nwith Zipf's law of abbreviation. Optimal non-singular coding in combination\nwith the maximum entropy principle also predicts Zipf's rank-frequency\ndistribution. Furthermore, our findings on optimal non-singular coding\nchallenge common beliefs about random typing. It turns out that random typing\nis in fact an optimal coding process, in stark contrast with the common\nassumption that it is detached from cost cutting considerations. Finally, we\ndiscuss the implications of optimal coding for the construction of a compact\ntheory of Zipfian laws and other linguistic laws.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01546v3",
            "date": "2019-06-04 16:04:04+00:00",
            "title": "Task-Guided Pair Embedding in Heterogeneous Network",
            "authors": [
                "Chanyoung Park",
                "Donghyun Kim",
                "Qi Zhu",
                "Jiawei Han",
                "Hwanjo Yu"
            ],
            "abstract": "Many real-world tasks solved by heterogeneous network embedding methods can\nbe cast as modeling the likelihood of pairwise relationship between two nodes.\nFor example, the goal of author identification task is to model the likelihood\nof a paper being written by an author (paper-author pairwise relationship).\nExisting task-guided embedding methods are node-centric in that they simply\nmeasure the similarity between the node embeddings to compute the likelihood of\na pairwise relationship between two nodes. However, we claim that for\ntask-guided embeddings, it is crucial to focus on directly modeling the\npairwise relationship. In this paper, we propose a novel task-guided pair\nembedding framework in heterogeneous network, called TaPEm, that directly\nmodels the relationship between a pair of nodes that are related to a specific\ntask (e.g., paper-author relationship in author identification). To this end,\nwe 1) propose to learn a pair embedding under the guidance of its associated\ncontext path, i.e., a sequence of nodes between the pair, and 2) devise the\npair validity classifier to distinguish whether the pair is valid with respect\nto the specific task at hand. By introducing pair embeddings that capture the\nsemantics behind the pairwise relationships, we are able to learn the\nfine-grained pairwise relationship between two nodes, which is paramount for\ntask-guided embedding methods. Extensive experiments on author identification\ntask demonstrate that TaPEm outperforms the state-of-the-art methods,\nespecially for authors with few publication records.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01594v1",
            "date": "2019-06-04 17:14:26+00:00",
            "title": "Finding Syntactic Representations in Neural Stacks",
            "authors": [
                "William Merrill",
                "Lenny Khazan",
                "Noah Amsel",
                "Yiding Hao",
                "Simon Mendelsohn",
                "Robert Frank"
            ],
            "abstract": "Neural network architectures have been augmented with differentiable stacks\nin order to introduce a bias toward learning hierarchy-sensitive regularities.\nIt has, however, proven difficult to assess the degree to which such a bias is\neffective, as the operation of the differentiable stack is not always\ninterpretable. In this paper, we attempt to detect the presence of latent\nrepresentations of hierarchical structure through an exploration of the\nunsupervised learning of constituency structure. Using a technique due to Shen\net al. (2018a,b), we extract syntactic trees from the pushing behavior of stack\nRNNs trained on language modeling and classification objectives. We find that\nour models produce parses that reflect natural language syntactic\nconstituencies, demonstrating that stack RNNs do indeed infer linguistically\nrelevant hierarchical structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01603v2",
            "date": "2019-06-04 17:32:35+00:00",
            "title": "Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study",
            "authors": [
                "Chinnadhurai Sankar",
                "Sandeep Subramanian",
                "Christopher Pal",
                "Sarath Chandar",
                "Yoshua Bengio"
            ],
            "abstract": "Neural generative models have been become increasingly popular when building\nconversational agents. They offer flexibility, can be easily adapted to new\ndomains, and require minimal domain engineering. A common criticism of these\nsystems is that they seldom understand or use the available dialog history\neffectively. In this paper, we take an empirical approach to understanding how\nthese models use the available dialog history by studying the sensitivity of\nthe models to artificially introduced unnatural changes or perturbations to\ntheir context at test time. We experiment with 10 different types of\nperturbations on 4 multi-turn dialog datasets and find that commonly used\nneural dialog architectures like recurrent and transformer-based seq2seq models\nare rarely sensitive to most perturbations such as missing or reordering\nutterances, shuffling words, etc. Also, by open-sourcing our code, we believe\nthat it will serve as a useful diagnostic tool for evaluating dialog systems in\nthe future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01604v1",
            "date": "2019-06-04 17:35:35+00:00",
            "title": "KERMIT: Generative Insertion-Based Modeling for Sequences",
            "authors": [
                "William Chan",
                "Nikita Kitaev",
                "Kelvin Guu",
                "Mitchell Stern",
                "Jakob Uszkoreit"
            ],
            "abstract": "We present KERMIT, a simple insertion-based approach to generative modeling\nfor sequences and sequence pairs. KERMIT models the joint distribution and its\ndecompositions (i.e., marginals and conditionals) using a single neural network\nand, unlike much prior work, does not rely on a prespecified factorization of\nthe data distribution. During training, one can feed KERMIT paired data $(x,\ny)$ to learn the joint distribution $p(x, y)$, and optionally mix in unpaired\ndata $x$ or $y$ to refine the marginals $p(x)$ or $p(y)$. During inference, we\nhave access to the conditionals $p(x \\mid y)$ and $p(y \\mid x)$ in both\ndirections. We can also sample from the joint distribution or the marginals.\nThe model supports both serial fully autoregressive decoding and parallel\npartially autoregressive decoding, with the latter exhibiting an empirically\nlogarithmic runtime. We demonstrate through experiments in machine translation,\nrepresentation learning, and zero-shot cloze question answering that our\nunified approach is capable of matching or exceeding the performance of\ndedicated state-of-the-art systems across a wide range of tasks without the\nneed for problem-specific architectural adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01605v1",
            "date": "2019-06-04 17:39:52+00:00",
            "title": "Transferable Neural Projection Representations",
            "authors": [
                "Chinnadhurai Sankar",
                "Sujith Ravi",
                "Zornitsa Kozareva"
            ],
            "abstract": "Neural word representations are at the core of many state-of-the-art natural\nlanguage processing models. A widely used approach is to pre-train, store and\nlook up word or character embedding matrices. While useful, such\nrepresentations occupy huge memory making it hard to deploy on-device and often\ndo not generalize to unknown words due to vocabulary pruning.\n  In this paper, we propose a skip-gram based architecture coupled with\nLocality-Sensitive Hashing (LSH) projections to learn efficient dynamically\ncomputable representations. Our model does not need to store lookup tables as\nrepresentations are computed on-the-fly and require low memory footprint. The\nrepresentations can be trained in an unsupervised fashion and can be easily\ntransferred to other NLP tasks. For qualitative evaluation, we analyze the\nnearest neighbors of the word representations and discover semantically similar\nwords even with misspellings. For quantitative evaluation, we plug our\ntransferable projections into a simple LSTM and run it on multiple NLP tasks\nand show how our transferable projections achieve better performance compared\nto prior work.",
            "journal ref": "Proc. of NAACL 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01615v3",
            "date": "2019-06-04 17:47:34+00:00",
            "title": "Sequential Neural Networks as Automata",
            "authors": [
                "William Merrill"
            ],
            "abstract": "This work attempts to explain the types of computation that neural networks\ncan perform by relating them to automata. We first define what it means for a\nreal-time network with bounded precision to accept a language. A measure of\nnetwork memory follows from this definition. We then characterize the classes\nof languages acceptable by various recurrent networks, attention, and\nconvolutional networks. We find that LSTMs function like counter machines and\nrelate convolutional networks to the subregular hierarchy. Overall, this work\nattempts to increase our understanding and ability to interpret neural networks\nthrough the lens of theory. These theoretical insights help explain neural\ncomputation, as well as the relationship between neural networks and natural\nlanguage grammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01622v3",
            "date": "2019-06-04 17:56:22+00:00",
            "title": "Are Girls Neko or Sh\u014djo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization",
            "authors": [
                "Mozhi Zhang",
                "Keyulu Xu",
                "Ken-ichi Kawarabayashi",
                "Stefanie Jegelka",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Cross-lingual word embeddings (CLWE) underlie many multilingual natural\nlanguage processing systems, often through orthogonal transformations of\npre-trained monolingual embeddings. However, orthogonal mapping only works on\nlanguage pairs whose embeddings are naturally isomorphic. For non-isomorphic\npairs, our method (Iterative Normalization) transforms monolingual embeddings\nto make orthogonal alignment easier by simultaneously enforcing that (1)\nindividual word vectors are unit length, and (2) each language's average vector\nis zero. Iterative Normalization consistently improves word translation\naccuracy of three CLWE methods, with the largest improvement observed on\nEnglish-Japanese (from 2% to 44% test accuracy).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01634v2",
            "date": "2019-06-04 07:30:48+00:00",
            "title": "On the Realization of Compositionality in Neural Networks",
            "authors": [
                "Joris Baan",
                "Jana Leible",
                "Mitja Nikolaus",
                "David Rau",
                "Dennis Ulmer",
                "Tim Baumg\u00e4rtner",
                "Dieuwke Hupkes",
                "Elia Bruni"
            ],
            "abstract": "We present a detailed comparison of two types of sequence to sequence models\ntrained to conduct a compositional task. The models are architecturally\nidentical at inference time, but differ in the way that they are trained: our\nbaseline model is trained with a task-success signal only, while the other\nmodel receives additional supervision on its attention mechanism (Attentive\nGuidance), which has shown to be an effective method for encouraging more\ncompositional solutions (Hupkes et al.,2019). We first confirm that the models\nwith attentive guidance indeed infer more compositional solutions than the\nbaseline, by training them on the lookup table task presented by Li\\v{s}ka et\nal. (2019). We then do an in-depth analysis of the structural differences\nbetween the two model types, focusing in particular on the organisation of the\nparameter space and the hidden layer activations and find noticeable\ndifferences in both these aspects. Guided networks focus more on the components\nof the input rather than the sequence as a whole and develop small functional\ngroups of neurons with specific purposes that use their gates more selectively.\nResults from parameter heat maps, component swapping and graph analysis also\nindicate that guided networks exhibit a more modular structure with a small\nnumber of specialized, strongly connected neurons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01635v1",
            "date": "2019-06-04 13:03:38+00:00",
            "title": "Detecting Ghostwriters in High Schools",
            "authors": [
                "Magnus Stavngaard",
                "August S\u00f8rensen",
                "Stephan Lorenzen",
                "Niklas Hjuler",
                "Stephen Alstrup"
            ],
            "abstract": "Students hiring ghostwriters to write their assignments is an increasing\nproblem in educational institutions all over the world, with companies selling\nthese services as a product. In this work, we develop automatic techniques with\nspecial focus on detecting such ghostwriting in high school assignments. This\nis done by training deep neural networks on an unprecedented large amount of\ndata supplied by the Danish company MaCom, which covers 90% of Danish high\nschools. We achieve an accuracy of 0.875 and a AUC score of 0.947 on an evenly\nsplit data set.",
            "journal ref": "Proceedings. ESANN 2019: 27th European Symposium on Artificial\n  Neural Networks, Computational Intelligence and Machine Learning. ed. Michel\n  Verleysen. 2019. p 197-202",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01726v1",
            "date": "2019-06-03 08:06:39+00:00",
            "title": "An Introduction to a New Text Classification and Visualization for Natural Language Processing Using Topological Data Analysis",
            "authors": [
                "Naiereh Elyasi",
                "Mehdi Hosseini Moghadam"
            ],
            "abstract": "Topological Data Analysis (TDA) is a novel new and fast growing field of data\nscience providing a set of new topological and geometric tools to derive\nrelevant features out of complex high-dimensional data. In this paper we apply\ntwo of best methods in topological data analysis, \"Persistent Homology\" and\n\"Mapper\", in order to classify persian poems which has been composed by two of\nthe best Iranian poets namely \"Ferdowsi\" and \"Hafez\". This article has two main\nparts, in the first part we explain the mathematics behind these two methods\nwhich is easy to understand for general audience and in the second part we\ndescribe our models and the results of applying TDA tools to NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01727v1",
            "date": "2019-05-25 16:46:49+00:00",
            "title": "SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums",
            "authors": [
                "Tsvetomila Mihaylova",
                "Georgi Karadjov",
                "Pepa Atanasova",
                "Ramy Baly",
                "Mitra Mohtarami",
                "Preslav Nakov"
            ],
            "abstract": "We present SemEval-2019 Task 8 on Fact Checking in Community Question\nAnswering Forums, which features two subtasks. Subtask A is about deciding\nwhether a question asks for factual information vs. an opinion/advice vs. just\nsocializing. Subtask B asks to predict whether an answer to a factual question\nis true, false or not a proper answer. We received 17 official submissions for\nsubtask A and 11 official submissions for Subtask B. For subtask A, all systems\nimproved over the majority class baseline. For Subtask B, all systems were\nbelow a majority class baseline, but several systems were very close to it. The\nleaderboard and the data from the competition can be found at\nhttp://competitions.codalab.org/competitions/20022",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01733v1",
            "date": "2019-06-04 21:28:31+00:00",
            "title": "The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction",
            "authors": [
                "Dimitrios Alikaniotis",
                "Vipul Raheja"
            ],
            "abstract": "Recent work on Grammatical Error Correction (GEC) has highlighted the\nimportance of language modeling in that it is certainly possible to achieve\ngood performance by comparing the probabilities of the proposed edits. At the\nsame time, advancements in language modeling have managed to generate\nlinguistic output, which is almost indistinguishable from that of\nhuman-generated text. In this paper, we up the ante by exploring the potential\nof more sophisticated language models in GEC and offer some key insights on\ntheir strengths and weaknesses. We show that, in line with recent results in\nother NLP tasks, Transformer architectures achieve consistently high\nperformance and provide a competitive baseline for future machine learning\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01738v2",
            "date": "2019-06-04 21:54:08+00:00",
            "title": "A Just and Comprehensive Strategy for Using NLP to Address Online Abuse",
            "authors": [
                "David Jurgens",
                "Eshwar Chandrasekharan",
                "Libby Hemphill"
            ],
            "abstract": "Online abusive behavior affects millions and the NLP community has attempted\nto mitigate this problem by developing technologies to detect abuse. However,\ncurrent methods have largely focused on a narrow definition of abuse to\ndetriment of victims who seek both validation and solutions. In this position\npaper, we argue that the community needs to make three substantive changes: (1)\nexpanding our scope of problems to tackle both more subtle and more serious\nforms of abuse, (2) developing proactive technologies that counter or inhibit\nabuse before it harms, and (3) reframing our effort within a framework of\njustice to promote healthy communities.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1906.01764v1",
            "date": "2019-06-05 00:33:47+00:00",
            "title": "Visual Story Post-Editing",
            "authors": [
                "Ting-Yao Hsu",
                "Chieh-Yang Huang",
                "Yen-Chia Hsu",
                "Ting-Hao 'Kenneth' Huang"
            ],
            "abstract": "We introduce the first dataset for human edits of machine-generated visual\nstories and explore how these collected edits may be used for the visual story\npost-editing task. The dataset, VIST-Edit, includes 14,905 human edited\nversions of 2,981 machine-generated visual stories. The stories were generated\nby two state-of-the-art visual storytelling models, each aligned to 5\nhuman-edited versions. We establish baselines for the task, showing how a\nrelatively small set of human edits can be leveraged to boost the performance\nof large visual storytelling models. We also discuss the weak correlation\nbetween automatic evaluation scores and human ratings, motivating the need for\nnew automatic metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01830v1",
            "date": "2019-05-25 13:31:52+00:00",
            "title": "ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets",
            "authors": [
                "Ramy Baly",
                "Alaa Khaddaj",
                "Hazem Hajj",
                "Wassim El-Hajj",
                "Khaled Bashir Shaban"
            ],
            "abstract": "Sentiment analysis is a highly subjective and challenging task. Its\ncomplexity further increases when applied to the Arabic language, mainly\nbecause of the large variety of dialects that are unstandardized and widely\nused in the Web, especially in social media. While many datasets have been\nreleased to train sentiment classifiers in Arabic, most of these datasets\ncontain shallow annotation, only marking the sentiment of the text unit, as a\nword, a sentence or a document. In this paper, we present the Arabic Sentiment\nTwitter Dataset for the Levantine dialect (ArSenTD-LEV). Based on findings from\nanalyzing tweets from the Levant region, we created a dataset of 4,000 tweets\nwith the following annotations: the overall sentiment of the tweet, the target\nto which the sentiment was expressed, how the sentiment was expressed, and the\ntopic of the tweet. Results confirm the importance of these annotations at\nimproving the performance of a baseline sentiment classifier. They also confirm\nthe gap of training in a certain domain, and testing in another domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01874v2",
            "date": "2019-06-05 08:16:42+00:00",
            "title": "Terminology-based Text Embedding for Computing Document Similarities on Technical Content",
            "authors": [
                "Hamid Mirisaee",
                "Eric Gaussier",
                "Cedric Lagnier",
                "Agnes Guerraz"
            ],
            "abstract": "We propose in this paper a new, hybrid document embedding approach in order\nto address the problem of document similarities with respect to the technical\ncontent. To do so, we employ a state-of-the-art graph techniques to first\nextract the keyphrases (composite keywords) of documents and, then, use them to\nscore the sentences. Using the ranked sentences, we propose two approaches to\nembed documents and show their performances with respect to two baselines. With\ndomain expert annotations, we illustrate that the proposed methods can find\nmore relevant documents and outperform the baselines up to 27% in terms of\nNDCG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.01910v1",
            "date": "2019-06-05 09:52:22+00:00",
            "title": "Evaluation and Improvement of Chatbot Text Classification Data Quality Using Plausible Negative Examples",
            "authors": [
                "Kit Kuksenok",
                "Andriy Martyniv"
            ],
            "abstract": "We describe and validate a metric for estimating multi-class classifier\nperformance based on cross-validation and adapted for improvement of small,\nunbalanced natural-language datasets used in chatbot design. Our experiences\ndraw upon building recruitment chatbots that mediate communication between\njob-seekers and recruiters by exposing the ML/NLP dataset to the recruiting\nteam. Evaluation approaches must be understandable to various stakeholders, and\nuseful for improving chatbot performance. The metric, nex-cv, uses negative\nexamples in the evaluation of text classification, and fulfils three\nrequirements. First, it is actionable: it can be used by non-developer staff.\nSecond, it is not overly optimistic compared to human ratings, making it a fast\nmethod for comparing classifiers. Third, it allows model-agnostic comparison,\nmaking it useful for comparing systems despite implementation differences. We\nvalidate the metric based on seven recruitment-domain datasets in English and\nGerman over the course of one year.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.02057v1",
            "date": "2019-06-05 14:51:31+00:00",
            "title": "The Language of Dialogue Is Complex",
            "authors": [
                "Alexander Robertson",
                "Luca Maria Aiello",
                "Daniele Quercia"
            ],
            "abstract": "Integrative Complexity (IC) is a psychometric that measures the ability of a\nperson to recognize multiple perspectives and connect them, thus identifying\npaths for conflict resolution. IC has been linked to a wide variety of\npolitical, social and personal outcomes but evaluating it is a time-consuming\nprocess requiring skilled professionals to manually score texts, a fact which\naccounts for the limited exploration of IC at scale on social media.We combine\nnatural language processing and machine learning to train an IC classification\nmodel that achieves state-of-the-art performance on unseen data and more\nclosely adheres to the established structure of the IC coding process than\nprevious automated approaches. When applied to the content of 400k+ comments\nfrom online fora about depression and knowledge exchange, our model was capable\nof replicating key findings of prior work, thus providing the first example of\nusing IC tools for large-scale social media analytics.",
            "journal ref": "In proceedings of the 13th International Conference on Web and\n  Social Media (ICWSM). Munich, 2019",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1906.02124v2",
            "date": "2019-05-14 10:33:08+00:00",
            "title": "PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model",
            "authors": [
                "Jieh-Sheng Lee",
                "Jieh Hsiang"
            ],
            "abstract": "In this work we focus on fine-tuning a pre-trained BERT model and applying it\nto patent classification. When applied to large datasets of over two millions\npatents, our approach outperforms the state of the art by an approach using CNN\nwith word embeddings. In addition, we focus on patent claims without other\nparts in patent documents. Our contributions include: (1) a new\nstate-of-the-art method based on pre-trained BERT model and fine-tuning for\npatent classification, (2) a large dataset USPTO-3M at the CPC subclass level\nwith SQL statements that can be used by future researchers, (3) showing that\npatent claims alone are sufficient for classification task, in contrast to\nconventional wisdom.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02132v2",
            "date": "2019-05-24 15:26:18+00:00",
            "title": "Ex-Twit: Explainable Twitter Mining on Health Data",
            "authors": [
                "Tunazzina Islam"
            ],
            "abstract": "Since most machine learning models provide no explanations for the\npredictions, their predictions are obscure for the human. The ability to\nexplain a model's prediction has become a necessity in many applications\nincluding Twitter mining. In this work, we propose a method called Explainable\nTwitter Mining (Ex-Twit) combining Topic Modeling and Local Interpretable\nModel-agnostic Explanation (LIME) to predict the topic and explain the model\npredictions. We demonstrate the effectiveness of Ex-Twit on Twitter\nhealth-related data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02181v1",
            "date": "2019-06-05 22:48:08+00:00",
            "title": "Syntax-Infused Variational Autoencoder for Text Generation",
            "authors": [
                "Xinyuan Zhang",
                "Yi Yang",
                "Siyang Yuan",
                "Dinghan Shen",
                "Lawrence Carin"
            ],
            "abstract": "We present a syntax-infused variational autoencoder (SIVAE), that integrates\nsentences with their syntactic trees to improve the grammar of generated\nsentences. Distinct from existing VAE-based text generative models, SIVAE\ncontains two separate latent spaces, for sentences and syntactic trees. The\nevidence lower bound objective is redesigned correspondingly, by optimizing a\njoint distribution that accommodates two encoders and two decoders. SIVAE works\nwith long short-term memory architectures to simultaneously generate sentences\nand syntactic trees. Two versions of SIVAE are proposed: one captures the\ndependencies between the latent variables through a conditional prior network,\nand the other treats the latent variables independently such that\nsyntactically-controlled sentence generation can be performed. Experimental\nresults demonstrate the generative superiority of SIVAE on both reconstruction\nand targeted syntactic evaluations. Finally, we show that the proposed models\ncan be used for unsupervised paraphrasing given different syntactic tree\ntemplates.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1906.02390v1",
            "date": "2019-06-06 02:52:12+00:00",
            "title": "Multi-view Knowledge Graph Embedding for Entity Alignment",
            "authors": [
                "Qingheng Zhang",
                "Zequn Sun",
                "Wei Hu",
                "Muhao Chen",
                "Lingbing Guo",
                "Yuzhong Qu"
            ],
            "abstract": "We study the problem of embedding-based entity alignment between knowledge\ngraphs (KGs). Previous works mainly focus on the relational structure of\nentities. Some further incorporate another type of features, such as\nattributes, for refinement. However, a vast of entity features are still\nunexplored or not equally treated together, which impairs the accuracy and\nrobustness of embedding-based entity alignment. In this paper, we propose a\nnovel framework that unifies multiple views of entities to learn embeddings for\nentity alignment. Specifically, we embed entities based on the views of entity\nnames, relations and attributes, with several combination strategies.\nFurthermore, we design some cross-KG inference methods to enhance the alignment\nbetween two KGs. Our experiments on real-world datasets show that the proposed\nframework significantly outperforms the state-of-the-art embedding-based entity\nalignment methods. The selected views, cross-KG inference and combination\nstrategies all contribute to the performance improvement.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.02403v2",
            "date": "2019-06-06 03:59:37+00:00",
            "title": "Ease-of-Teaching and Language Structure from Emergent Communication",
            "authors": [
                "Fushan Li",
                "Michael Bowling"
            ],
            "abstract": "Artificial agents have been shown to learn to communicate when needed to\ncomplete a cooperative task. Some level of language structure (e.g.,\ncompositionality) has been found in the learned communication protocols. This\nobserved structure is often the result of specific environmental pressures\nduring training. By introducing new agents periodically to replace old ones,\nsequentially and within a population, we explore such a new pressure -- ease of\nteaching -- and show its impact on the structure of the resulting language.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.02416v2",
            "date": "2019-06-06 05:04:08+00:00",
            "title": "Sparse Parallel Training of Hierarchical Dirichlet Process Topic Models",
            "authors": [
                "Alexander Terenin",
                "M\u00e5ns Magnusson",
                "Leif Jonsson"
            ],
            "abstract": "To scale non-parametric extensions of probabilistic topic models such as\nLatent Dirichlet allocation to larger data sets, practitioners rely\nincreasingly on parallel and distributed systems. In this work, we study\ndata-parallel training for the hierarchical Dirichlet process (HDP) topic\nmodel. Based upon a representation of certain conditional distributions within\nan HDP, we propose a doubly sparse data-parallel sampler for the HDP topic\nmodel. This sampler utilizes all available sources of sparsity found in natural\nlanguage - an important way to make computation efficient. We benchmark our\nmethod on a well-known corpus (PubMed) with 8m documents and 768m tokens, using\na single multi-core machine in under four days.",
            "journal ref": "Conference on Empirical Methods in Natural Language Processing,\n  2020",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1906.02448v2",
            "date": "2019-06-06 07:15:52+00:00",
            "title": "Bridging the Gap between Training and Inference for Neural Machine Translation",
            "authors": [
                "Wen Zhang",
                "Yang Feng",
                "Fandong Meng",
                "Di You",
                "Qun Liu"
            ],
            "abstract": "Neural Machine Translation (NMT) generates target words sequentially in the\nway of predicting the next word conditioned on the context words. At training\ntime, it predicts with the ground truth words as context while at inference it\nhas to generate the entire sequence from scratch. This discrepancy of the fed\ncontext leads to error accumulation among the way. Furthermore, word-level\ntraining requires strict matching between the generated sequence and the ground\ntruth sequence which leads to overcorrection over different but reasonable\ntranslations. In this paper, we address these issues by sampling context words\nnot only from the ground truth sequence but also from the predicted sequence by\nthe model during training, where the predicted sequence is selected with a\nsentence-level optimum. Experiment results on Chinese->English and WMT'14\nEnglish->German translation tasks demonstrate that our approach can achieve\nsignificant improvements on multiple datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02461v3",
            "date": "2019-06-06 07:48:36+00:00",
            "title": "Unsupervised Pivot Translation for Distant Languages",
            "authors": [
                "Yichong Leng",
                "Xu Tan",
                "Tao Qin",
                "Xiang-Yang Li",
                "Tie-Yan Liu"
            ],
            "abstract": "Unsupervised neural machine translation (NMT) has attracted a lot of\nattention recently. While state-of-the-art methods for unsupervised translation\nusually perform well between similar languages (e.g., English-German\ntranslation), they perform poorly between distant languages, because\nunsupervised alignment does not work well for distant languages. In this work,\nwe introduce unsupervised pivot translation for distant languages, which\ntranslates a language to a distant language through multiple hops, and the\nunsupervised translation on each hop is relatively easier than the original\ndirect translation. We propose a learning to route (LTR) method to choose the\ntranslation path between the source and target languages. LTR is trained on\nlanguage pairs whose best translation path is available and is applied on the\nunseen language pairs for path selection. Experiments on 20 languages and 294\ndistant language pairs demonstrate the advantages of the unsupervised pivot\ntranslation for distant languages, as well as the effectiveness of the proposed\nLTR for path selection. Specifically, in the best case, LTR achieves an\nimprovement of 5.58 BLEU points over the conventional direct unsupervised\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02688v2",
            "date": "2019-06-05 07:15:06+00:00",
            "title": "Topic Sensitive Attention on Generic Corpora Corrects Sense Bias in Pretrained Embeddings",
            "authors": [
                "Vihari Piratla",
                "Sunita Sarawagi",
                "Soumen Chakrabarti"
            ],
            "abstract": "Given a small corpus $\\mathcal D_T$ pertaining to a limited set of focused\ntopics, our goal is to train embeddings that accurately capture the sense of\nwords in the topic in spite of the limited size of $\\mathcal D_T$. These\nembeddings may be used in various tasks involving $\\mathcal D_T$. A popular\nstrategy in limited data settings is to adapt pre-trained embeddings $\\mathcal\nE$ trained on a large corpus. To correct for sense drift, fine-tuning,\nregularization, projection, and pivoting have been proposed recently. Among\nthese, regularization informed by a word's corpus frequency performed well, but\nwe improve upon it using a new regularizer based on the stability of its\ncooccurrence with other words. However, a thorough comparison across ten\ntopics, spanning three tasks, with standardized settings of hyper-parameters,\nreveals that even the best embedding adaptation strategies provide small gains\nbeyond well-tuned baselines, which many earlier comparisons ignored. In a bold\ndeparture from adapting pretrained embeddings, we propose using $\\mathcal D_T$\nto probe, attend to, and borrow fragments from any large, topic-rich source\ncorpus (such as Wikipedia), which need not be the corpus used to pretrain\nembeddings. This step is made scalable and practical by suitable indexing. We\nreach the surprising conclusion that even limited corpus augmentation is more\nuseful than adapting embeddings, which suggests that non-dominant sense\ninformation may be irrevocably obliterated from pretrained embeddings and\ncannot be salvaged by adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02715v2",
            "date": "2019-06-06 17:33:22+00:00",
            "title": "Visualizing and Measuring the Geometry of BERT",
            "authors": [
                "Andy Coenen",
                "Emily Reif",
                "Ann Yuan",
                "Been Kim",
                "Adam Pearce",
                "Fernanda Vi\u00e9gas",
                "Martin Wattenberg"
            ],
            "abstract": "Transformer architectures show significant promise for natural language\nprocessing. Given that a single pretrained model can be fine-tuned to perform\nwell on many different tasks, these networks appear to extract generally useful\nlinguistic features. A natural question is how such networks represent this\ninformation internally. This paper describes qualitative and quantitative\ninvestigations of one particularly effective model, BERT. At a high level,\nlinguistic features seem to be represented in separate semantic and syntactic\nsubspaces. We find evidence of a fine-grained geometric representation of word\nsenses. We also present empirical descriptions of syntactic representations in\nboth attention matrices and individual word embeddings, as well as a\nmathematical argument to explain the geometry of these representations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.02738v2",
            "date": "2019-06-06 17:55:37+00:00",
            "title": "Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading",
            "authors": [
                "Lianhui Qin",
                "Michel Galley",
                "Chris Brockett",
                "Xiaodong Liu",
                "Xiang Gao",
                "Bill Dolan",
                "Yejin Choi",
                "Jianfeng Gao"
            ],
            "abstract": "Although neural conversation models are effective in learning how to produce\nfluent responses, their primary challenge lies in knowing what to say to make\nthe conversation contentful and non-vacuous. We present a new end-to-end\napproach to contentful neural conversation that jointly models response\ngeneration and on-demand machine reading. The key idea is to provide the\nconversation model with relevant long-form text on the fly as a source of\nexternal knowledge. The model performs QA-style reading comprehension on this\ntext in response to each conversational turn, thereby allowing for more focused\nintegration of external knowledge than has been possible in prior approaches.\nTo support further research on knowledge-grounded conversation, we introduce a\nnew large-scale conversation dataset grounded in external web pages (2.8M\nturns, 7.4M sentences of grounding). Both human evaluation and automated\nmetrics show that our approach results in more contentful responses compared to\na variety of previous methods, improving both the informativeness and diversity\nof generated output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02762v1",
            "date": "2019-06-06 18:10:08+00:00",
            "title": "Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View",
            "authors": [
                "Yiping Lu",
                "Zhuohan Li",
                "Di He",
                "Zhiqing Sun",
                "Bin Dong",
                "Tao Qin",
                "Liwei Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "The Transformer architecture is widely used in natural language processing.\nDespite its success, the design principle of the Transformer remains elusive.\nIn this paper, we provide a novel perspective towards understanding the\narchitecture: we show that the Transformer can be mathematically interpreted as\na numerical Ordinary Differential Equation (ODE) solver for a\nconvection-diffusion equation in a multi-particle dynamic system. In\nparticular, how words in a sentence are abstracted into contexts by passing\nthrough the layers of the Transformer can be interpreted as approximating\nmultiple particles' movement in the space using the Lie-Trotter splitting\nscheme and the Euler's method. Given this ODE's perspective, the rich\nliterature of numerical analysis can be brought to guide us in designing\neffective structures beyond the Transformer. As an example, we propose to\nreplace the Lie-Trotter splitting scheme by the Strang-Marchuk splitting\nscheme, a scheme that is more commonly used and with much lower local\ntruncation errors. The Strang-Marchuk splitting scheme suggests that the\nself-attention and position-wise feed-forward network (FFN) sub-layers should\nnot be treated equally. Instead, in each layer, two position-wise FFN\nsub-layers should be used, and the self-attention sub-layer is placed in\nbetween. This leads to a brand new architecture. Such an FFN-attention-FFN\nlayer is \"Macaron-like\", and thus we call the network with this new\narchitecture the Macaron Net. Through extensive experiments, we show that the\nMacaron Net is superior to the Transformer on both supervised and unsupervised\nlearning tasks. The reproducible codes and pretrained models can be found at\nhttps://github.com/zhuohan123/macaron-net",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.03072v1",
            "date": "2019-06-04 13:20:40+00:00",
            "title": "Investigating Writing Style Development in High School",
            "authors": [
                "Stephan Lorenzen",
                "Niklas Hjuler",
                "Stephen Alstrup"
            ],
            "abstract": "In this paper we do the first large scale analysis of writing style\ndevelopment among Danish high school students. More than 10K students with more\nthan 100K essays are analyzed. Writing style itself is often studied in the\nnatural language processing community, but usually with the goal of verifying\nauthorship, assessing quality or popularity, or other kinds of predictions.\n  In this work, we analyze writing style changes over time, with the goal of\ndetecting global development trends among students, and identifying at-risk\nstudents. We train a Siamese neural network to compute the similarity between\ntwo texts. Using this similarity measure, a student's newer essays are compared\nto their first essays, and a writing style development profile is constructed\nfor the student. We cluster these student profiles and analyze the resulting\nclusters in order to detect general development patterns. We evaluate clusters\nwith respect to writing style quality indicators, and identify optimal\nclusters, showing significant improvement in writing style, while also\nobserving suboptimal clusters, exhibiting periods of limited development and\neven setbacks.\n  Furthermore, we identify general development trends between high school\nstudents, showing that as students progress through high school, their writing\nstyle deviates, leaving students less similar when they finish high school,\nthan when they start.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1906.03183v1",
            "date": "2019-05-23 17:38:06+00:00",
            "title": "Exploring Diseases and Syndromes in Neurology Case Reports from 1955 to 2017 with Text Mining",
            "authors": [
                "Amir Karami",
                "Mehdi Ghasemi",
                "Souvik Sen",
                "Marcos Moraes",
                "Vishal Shah"
            ],
            "abstract": "Background: A large number of neurology case reports have been published, but\nit is a challenging task for human medical experts to explore all of these\npublications. Text mining offers a computational approach to investigate\nneurology literature and capture meaningful patterns. The overarching goal of\nthis study is to provide a new perspective on case reports of neurological\ndisease and syndrome analysis over the last six decades using text mining.\n  Methods: We extracted diseases and syndromes (DsSs) from more than 65,000\nneurology case reports from 66 journals in PubMed over the last six decades\nfrom 1955 to 2017. Text mining was applied to reports on the detected DsSs to\ninvestigate high-frequency DsSs, categorize them, and explore the linear trends\nover the 63-year time frame.\n  Results: The text mining methods explored high-frequency neurologic DsSs and\ntheir trends and the relationships between them from 1955 to 2017. We detected\nmore than 18,000 unique DsSs and found 10 categories of neurologic DsSs. While\nthe trend analysis showed the increasing trends in the case reports for top-10\nhigh-frequency DsSs, the categories had mixed trends.\n  Conclusion: Our study provided new insights into the application of text\nmining methods to investigate DsSs in a large number of medical case reports\nthat occur over several decades. The proposed approach can be used to provide a\nmacro level analysis of medical literature by discovering interesting patterns\nand tracking them over several years to help physicians explore these case\nreports more efficiently.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1906.03351v2",
            "date": "2019-06-07 22:45:33+00:00",
            "title": "Real or Fake? Learning to Discriminate Machine from Human Generated Text",
            "authors": [
                "Anton Bakhtin",
                "Sam Gross",
                "Myle Ott",
                "Yuntian Deng",
                "Marc'Aurelio Ranzato",
                "Arthur Szlam"
            ],
            "abstract": "Energy-based models (EBMs), a.k.a. un-normalized models, have had recent\nsuccesses in continuous spaces. However, they have not been successfully\napplied to model text sequences. While decreasing the energy at training\nsamples is straightforward, mining (negative) samples where the energy should\nbe increased is difficult. In part, this is because standard gradient-based\nmethods are not readily applicable when the input is high-dimensional and\ndiscrete. Here, we side-step this issue by generating negatives using\npre-trained auto-regressive language models. The EBM then works in the residual\nof the language model; and is trained to discriminate real text from text\ngenerated by the auto-regressive models. We investigate the generalization\nability of residual EBMs, a pre-requisite for using them in other applications.\nWe extensively analyze generalization for the task of classifying whether an\ninput is machine or human generated, a natural task given the training loss and\nhow we mine negatives. Overall, we observe that EBMs can generalize remarkably\nwell to changes in the architecture of the generators producing negatives.\nHowever, EBMs exhibit more sensitivity to the training set used by such\ngenerators.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.03392v1",
            "date": "2019-06-08 05:10:28+00:00",
            "title": "Detection and Prediction of Users Attitude Based on Real-Time and Batch Sentiment Analysis of Facebook Comments",
            "authors": [
                "Hieu Tran",
                "Maxim Shcherbakov"
            ],
            "abstract": "The most of the people have their account on social networks (e.g. Facebook,\nVkontakte) where they express their attitude to different situations and\nevents. Facebook provides only the positive mark as a like button and share.\nHowever, it is important to know the position of a certain user on posts even\nthough the opinion is negative. Positive, negative and neutral attitude can be\nextracted from the comments of users. Overall information about positive,\nnegative and neutral opinion can bring the understanding of how people react in\na position. Moreover, it is important to know how attitude is changing during\nthe time period. The contribution of the paper is a new method based on\nsentiment text analysis for detection and prediction negative and positive\npatterns for Facebook comments which combines (i) real-time sentiment text\nanalysis for pattern discovery and (ii) batch data processing for creating\nopinion forecasting algorithm. To perform forecast we propose two-steps\nalgorithm where: (i) patterns are clustered using unsupervised clustering\ntechniques and (ii) trend prediction is performed based on finding the nearest\npattern from the certain cluster. Case studies show the efficiency and accuracy\n(Avg. MAE = 0.008) of the proposed method and its practical applicability.\nAlso, we discovered three types of users attitude patterns and described them.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1906.03402v3",
            "date": "2019-06-08 06:59:56+00:00",
            "title": "Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis",
            "authors": [
                "Eric Battenberg",
                "Soroosh Mariooryad",
                "Daisy Stanton",
                "RJ Skerry-Ryan",
                "Matt Shannon",
                "David Kao",
                "Tom Bagby"
            ],
            "abstract": "Recent work has explored sequence-to-sequence latent variable models for\nexpressive speech synthesis (supporting control and transfer of prosody and\nstyle), but has not presented a coherent framework for understanding the\ntrade-offs between the competing methods. In this paper, we propose embedding\ncapacity (the amount of information the embedding contains about the data) as a\nunified method of analyzing the behavior of latent variable models of speech,\ncomparing existing heuristic (non-variational) methods to variational methods\nthat are able to explicitly constrain capacity using an upper bound on\nrepresentational mutual information. In our proposed model (Capacitron), we\nshow that by adding conditional dependencies to the variational posterior such\nthat it matches the form of the true posterior, the same model can be used for\nhigh-precision prosody transfer, text-agnostic style transfer, and generation\nof natural-sounding prior samples. For multi-speaker models, Capacitron is able\nto preserve target speaker identity during inter-speaker prosody transfer and\nwhen drawing samples from the latent prior. Lastly, we introduce a method for\ndecomposing embedding capacity hierarchically across two sets of latents,\nallowing a portion of the latent variability to be specified and the remaining\nvariability sampled from a learned prior. Audio examples are available on the\nweb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03423v1",
            "date": "2019-06-08 08:56:02+00:00",
            "title": "News Labeling as Early as Possible: Real or Fake?",
            "authors": [
                "Maryam Ramezani",
                "Mina Rafiei",
                "Soroush Omranpour",
                "Hamid R. Rabiee"
            ],
            "abstract": "Making disguise between real and fake news propagation through online social\nnetworks is an important issue in many applications. The time gap between the\nnews release time and detection of its label is a significant step towards\nbroadcasting the real information and avoiding the fake. Therefore, one of the\nchallenging tasks in this area is to identify fake and real news in early\nstages of propagation. However, there is a trade-off between minimizing the\ntime gap and maximizing accuracy. Despite recent efforts in detection of fake\nnews, there has been no significant work that explicitly incorporates early\ndetection in its model. In this paper, we focus on accurate early labeling of\nnews, and propose a model by considering earliness both in modeling and\nprediction. The proposed method utilizes recurrent neural networks with a novel\nloss function, and a new stopping rule. Given the context of news, we first\nembed it with a class-specific text representation. Then, we utilize the\navailable public profile of users, and speed of news diffusion, for early\nlabeling of the news. Experiments on real datasets demonstrate the\neffectiveness of our model both in terms of early labelling and accuracy,\ncompared to the state of the art baseline and models.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1906.03506v1",
            "date": "2019-06-08 19:18:18+00:00",
            "title": "Forward and Backward Knowledge Transfer for Sentiment Classification",
            "authors": [
                "Hao Wang",
                "Bing Liu",
                "Shuai Wang",
                "Nianzu Ma",
                "Yan Yang"
            ],
            "abstract": "This paper studies the problem of learning a sequence of sentiment\nclassification tasks. The learned knowledge from each task is retained and used\nto help future or subsequent task learning. This learning paradigm is called\nLifelong Learning (LL). However, existing LL methods either only transfer\nknowledge forward to help future learning and do not go back to improve the\nmodel of a previous task or require the training data of the previous task to\nretrain its model to exploit backward/reverse knowledge transfer. This paper\nstudies reverse knowledge transfer of LL in the context of naive Bayesian (NB)\nclassification. It aims to improve the model of a previous task by leveraging\nfuture knowledge without retraining using its training data. This is done by\nexploiting a key characteristic of the generative model of NB. That is, it is\npossible to improve the NB classifier for a task by improving its model\nparameters directly by using the retained knowledge from other tasks.\nExperimental results show that the proposed method markedly outperforms\nexisting LL baselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.03588v1",
            "date": "2019-06-09 07:51:23+00:00",
            "title": "rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method",
            "authors": [
                "Zheng-Hua Tan",
                "Achintya kr. Sarkar",
                "Najim Dehak"
            ],
            "abstract": "This paper presents an unsupervised segment-based method for robust voice\nactivity detection (rVAD). The method consists of two passes of denoising\nfollowed by a voice activity detection (VAD) stage. In the first pass,\nhigh-energy segments in a speech signal are detected by using a posteriori\nsignal-to-noise ratio (SNR) weighted energy difference and if no pitch is\ndetected within a segment, the segment is considered as a high-energy noise\nsegment and set to zero. In the second pass, the speech signal is denoised by a\nspeech enhancement method, for which several methods are explored. Next,\nneighbouring frames with pitch are grouped together to form pitch segments, and\nbased on speech statistics, the pitch segments are further extended from both\nends in order to include both voiced and unvoiced sounds and likely non-speech\nparts as well. In the end, a posteriori SNR weighted energy difference is\napplied to the extended pitch segments of the denoised speech signal for\ndetecting voice activity. We evaluate the VAD performance of the proposed\nmethod using two databases, RATS and Aurora-2, which contain a large variety of\nnoise conditions. The rVAD method is further evaluated, in terms of speaker\nverification performance, on the RedDots 2016 challenge database and its\nnoise-corrupted versions. Experiment results show that rVAD is compared\nfavourably with a number of existing methods. In addition, we present a\nmodified version of rVAD where computationally intensive pitch extraction is\nreplaced by computationally efficient spectral flatness calculation. The\nmodified version significantly reduces the computational complexity at the cost\nof moderately inferior VAD performance, which is an advantage when processing a\nlarge amount of data and running on low resource devices. The source code of\nrVAD is made publicly available.",
            "journal ref": "Computer Speech & Language, volume 59, January 2020, Pages 1-21",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1906.03648v1",
            "date": "2019-06-09 14:30:00+00:00",
            "title": "LSTM Networks Can Perform Dynamic Counting",
            "authors": [
                "Mirac Suzgun",
                "Sebastian Gehrmann",
                "Yonatan Belinkov",
                "Stuart M. Shieber"
            ],
            "abstract": "In this paper, we systematically assess the ability of standard recurrent\nnetworks to perform dynamic counting and to encode hierarchical\nrepresentations. All the neural models in our experiments are designed to be\nsmall-sized networks both to prevent them from memorizing the training sets and\nto visualize and interpret their behaviour at test time. Our results\ndemonstrate that the Long Short-Term Memory (LSTM) networks can learn to\nrecognize the well-balanced parenthesis language (Dyck-$1$) and the shuffles of\nmultiple Dyck-$1$ languages, each defined over different parenthesis-pairs, by\nemulating simple real-time $k$-counter machines. To the best of our knowledge,\nthis work is the first study to introduce the shuffle languages to analyze the\ncomputational power of neural networks. We also show that a single-layer LSTM\nwith only one hidden unit is practically sufficient for recognizing the\nDyck-$1$ language. However, none of our recurrent networks was able to yield a\ngood performance on the Dyck-$2$ language learning task, which requires a model\nto have a stack-like mechanism for recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03674v1",
            "date": "2019-06-09 17:06:28+00:00",
            "title": "Attention-based Conditioning Methods for External Knowledge Integration",
            "authors": [
                "Katerina Margatina",
                "Christos Baziotis",
                "Alexandros Potamianos"
            ],
            "abstract": "In this paper, we present a novel approach for incorporating external\nknowledge in Recurrent Neural Networks (RNNs). We propose the integration of\nlexicon features into the self-attention mechanism of RNN-based architectures.\nThis form of conditioning on the attention distribution, enforces the\ncontribution of the most salient words for the task at hand. We introduce three\nmethods, namely attentional concatenation, feature-based gating and affine\ntransformation. Experiments on six benchmark datasets show the effectiveness of\nour methods. Attentional feature-based gating yields consistent performance\nimprovement across tasks. Our approach is implemented as a simple add-on module\nfor RNN-based models with minimal computational overhead and can be adapted to\nany deep neural architecture.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.03805v2",
            "date": "2019-06-10 05:55:08+00:00",
            "title": "Improving Neural Language Modeling via Adversarial Training",
            "authors": [
                "Dilin Wang",
                "Chengyue Gong",
                "Qiang Liu"
            ],
            "abstract": "Recently, substantial progress has been made in language modeling by using\ndeep neural networks. However, in practice, large scale neural language models\nhave been shown to be prone to overfitting. In this paper, we present a simple\nyet highly effective adversarial training mechanism for regularizing neural\nlanguage models. The idea is to introduce adversarial noise to the output\nembedding layer while training the models. We show that the optimal adversarial\nnoise yields a simple closed-form solution, thus allowing us to develop a\nsimple and time efficient algorithm. Theoretically, we show that our\nadversarial mechanism effectively encourages the diversity of the embedding\nvectors, helping to increase the robustness of models. Empirically, we show\nthat our method improves on the single model state-of-the-art results for\nlanguage modeling on Penn Treebank (PTB) and Wikitext-2, achieving test\nperplexity scores of 46.01 and 38.07, respectively. When applied to machine\ntranslation, our method improves over various transformer-based translation\nbaselines in BLEU scores on the WMT14 English-German and IWSLT14 German-English\ntasks.",
            "journal ref": "International Conference on Machine Learning 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.03897v1",
            "date": "2019-06-10 10:57:47+00:00",
            "title": "Learning to combine Grammatical Error Corrections",
            "authors": [
                "Yoav Kantor",
                "Yoav Katz",
                "Leshem Choshen",
                "Edo Cohen-Karlik",
                "Naftali Liberman",
                "Assaf Toledo",
                "Amir Menczel",
                "Noam Slonim"
            ],
            "abstract": "The field of Grammatical Error Correction (GEC) has produced various systems\nto deal with focused phenomena or general text editing. We propose an automatic\nway to combine black-box systems. Our method automatically detects the strength\nof a system or the combination of several systems per error type, improving\nprecision and recall while optimizing $F$ score directly. We show consistent\nimprovement over the best standalone system in all the configurations tested.\nThis approach also outperforms average ensembling of different RNN models with\nrandom initializations.\n  In addition, we analyze the use of BERT for GEC - reporting promising results\non this end. We also present a spellchecker created for this task which\noutperforms standard spellcheckers tested on the task of spellchecking.\n  This paper describes a system submission to Building Educational Applications\n2019 Shared Task: Grammatical Error Correction.\n  Combining the output of top BEA 2019 shared task systems using our approach,\ncurrently holds the highest reported score in the open phase of the BEA 2019\nshared task, improving F0.5 by 3.7 points over the best result reported.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.03926v1",
            "date": "2019-06-10 12:17:45+00:00",
            "title": "A Survey of Reinforcement Learning Informed by Natural Language",
            "authors": [
                "Jelena Luketina",
                "Nantas Nardelli",
                "Gregory Farquhar",
                "Jakob Foerster",
                "Jacob Andreas",
                "Edward Grefenstette",
                "Shimon Whiteson",
                "Tim Rockt\u00e4schel"
            ],
            "abstract": "To be successful in real-world tasks, Reinforcement Learning (RL) needs to\nexploit the compositional, relational, and hierarchical structure of the world,\nand learn to transfer it to the task at hand. Recent advances in representation\nlearning for language make it possible to build models that acquire world\nknowledge from text corpora and integrate this knowledge into downstream\ndecision making problems. We thus argue that the time is right to investigate a\ntight integration of natural language understanding into RL in particular. We\nsurvey the state of the field, including work on instruction following, text\ngames, and learning from textual domain knowledge. Finally, we call for the\ndevelopment of new environments as well as further investigation into the\npotential uses of recent Natural Language Processing (NLP) techniques for such\ntasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.04043v1",
            "date": "2019-06-10 14:52:41+00:00",
            "title": "GLTR: Statistical Detection and Visualization of Generated Text",
            "authors": [
                "Sebastian Gehrmann",
                "Hendrik Strobelt",
                "Alexander M. Rush"
            ],
            "abstract": "The rapid improvement of language models has raised the specter of abuse of\ntext generation systems. This progress motivates the development of simple\nmethods for detecting generated text that can be used by and explained to\nnon-experts. We develop GLTR, a tool to support humans in detecting whether a\ntext was generated by a model. GLTR applies a suite of baseline statistical\nmethods that can detect generation artifacts across common sampling schemes. In\na human-subjects study, we show that the annotation scheme provided by GLTR\nimproves the human detection-rate of fake text from 54% to 72% without any\nprior training. GLTR is open-source and publicly deployed, and has already been\nwidely used to detect generated outputs",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04164v1",
            "date": "2019-06-07 18:49:38+00:00",
            "title": "FAKTA: An Automatic End-to-End Fact Checking System",
            "authors": [
                "Moin Nadeem",
                "Wei Fang",
                "Brian Xu",
                "Mitra Mohtarami",
                "James Glass"
            ],
            "abstract": "We present FAKTA which is a unified framework that integrates various\ncomponents of a fact checking process: document retrieval from media sources\nwith various types of reliability, stance detection of documents with respect\nto given claims, evidence extraction, and linguistic analysis. FAKTA predicts\nthe factuality of given claims and provides evidence at the document and\nsentence level to explain its predictions",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04284v2",
            "date": "2019-06-07 13:58:49+00:00",
            "title": "Analyzing the Structure of Attention in a Transformer Language Model",
            "authors": [
                "Jesse Vig",
                "Yonatan Belinkov"
            ],
            "abstract": "The Transformer is a fully attention-based alternative to recurrent networks\nthat has achieved state-of-the-art results across a range of NLP tasks. In this\npaper, we analyze the structure of attention in a Transformer language model,\nthe GPT-2 small pretrained model. We visualize attention for individual\ninstances and analyze the interaction between attention and syntax over a large\ncorpus. We find that attention targets different parts of speech at different\nlayer depths within the model, and that attention aligns with dependency\nrelations most strongly in the middle layers. We also find that the deepest\nlayers of the model capture the most distant relationships. Finally, we extract\nexemplar sentences that reveal highly specific patterns targeted by particular\nattention heads.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04323v2",
            "date": "2019-06-10 23:31:56+00:00",
            "title": "Word-level Speech Recognition with a Letter to Word Encoder",
            "authors": [
                "Ronan Collobert",
                "Awni Hannun",
                "Gabriel Synnaeve"
            ],
            "abstract": "We propose a direct-to-word sequence model which uses a word network to learn\nword embeddings from letters. The word network can be integrated seamlessly\nwith arbitrary sequence models including Connectionist Temporal Classification\nand encoder-decoder models with attention. We show our direct-to-word model can\nachieve word error rate gains over sub-word level models for speech\nrecognition. We also show that our direct-to-word approach retains the ability\nto predict words not seen at training time without any retraining. Finally, we\ndemonstrate that a word-level model can use a larger stride than a sub-word\nlevel model while maintaining accuracy. This makes the model more efficient\nboth for training and inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04580v1",
            "date": "2019-06-09 07:08:20+00:00",
            "title": "Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks",
            "authors": [
                "Hao Peng",
                "Jianxin Li",
                "Qiran Gong",
                "Yangqiu Song",
                "Yuanxing Ning",
                "Kunfeng Lai",
                "Philip S. Yu"
            ],
            "abstract": "Events are happening in real-world and real-time, which can be planned and\norganized occasions involving multiple people and objects. Social media\nplatforms publish a lot of text messages containing public events with\ncomprehensive topics. However, mining social events is challenging due to the\nheterogeneous event elements in texts and explicit and implicit social network\nstructures. In this paper, we design an event meta-schema to characterize the\nsemantic relatedness of social events and build an event-based heterogeneous\ninformation network (HIN) integrating information from external knowledge base,\nand propose a novel Pair-wise Popularity Graph Convolutional Network (PP-GCN)\nbased fine-grained social event categorization model. We propose a\nKnowledgeable meta-paths Instances based social Event Similarity (KIES) between\nevents and build a weighted adjacent matrix as input to the PP-GCN model.\nComprehensive experiments on real data collections are conducted to compare\nvarious social event detection and clustering tasks. Experimental results\ndemonstrate that our proposed framework outperforms other alternative social\nevent categorization techniques.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1906.04908v1",
            "date": "2019-06-12 03:13:18+00:00",
            "title": "SPoC: Search-based Pseudocode to Code",
            "authors": [
                "Sumith Kulal",
                "Panupong Pasupat",
                "Kartik Chandra",
                "Mina Lee",
                "Oded Padon",
                "Alex Aiken",
                "Percy Liang"
            ],
            "abstract": "We consider the task of mapping pseudocode to long programs that are\nfunctionally correct. Given test cases as a mechanism to validate programs, we\nsearch over the space of possible translations of the pseudocode to find a\nprogram that passes the validation. However, without proper credit assignment\nto localize the sources of program failures, it is difficult to guide search\ntoward more promising programs. We propose to perform credit assignment based\non signals from compilation errors, which constitute 88.7% of program failures.\nConcretely, we treat the translation of each pseudocode line as a discrete\nportion of the program, and whenever a synthesized program fails to compile, an\nerror localization method tries to identify the portion of the program\nresponsible for the failure. We then focus search over alternative translations\nof the pseudocode for those portions. For evaluation, we collected the SPoC\ndataset (Search-based Pseudocode to Code) containing 18,356 programs with\nhuman-authored pseudocode and test cases. Under a budget of 100 program\ncompilations, performing search improves the synthesis success rate over using\nthe top-one translation of the pseudocode from 25.6% to 44.7%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.04914v1",
            "date": "2019-06-11 17:38:28+00:00",
            "title": "From Fully Supervised to Zero Shot Settings for Twitter Hashtag Recommendation",
            "authors": [
                "Abhay Kumar",
                "Nishant Jain",
                "Suraj Tripathi",
                "Chirag Singh"
            ],
            "abstract": "We propose a comprehensive end-to-end pipeline for Twitter hashtags\nrecommendation system including data collection, supervised training setting\nand zero shot training setting. In the supervised training setting, we have\nproposed and compared the performance of various deep learning architectures,\nnamely Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and\nTransformer Network. However, it is not feasible to collect data for all\npossible hashtag labels and train a classifier model on them. To overcome this\nlimitation, we propose a Zero Shot Learning (ZSL) paradigm for predicting\nunseen hashtag labels by learning the relationship between the semantic space\nof tweets and the embedding space of hashtag labels. We evaluated various\nstate-of-the-art ZSL methods like Convex combination of Semantic Embedding\n(ConSE), Embarrassingly Simple Zero-Shot Learning (ESZSL) and Deep Embedding\nModel for Zero-Shot Learning (DEM-ZSL) for the hashtag recommendation task. We\ndemonstrate the effectiveness and scalability of ZSL methods for the\nrecommendation of unseen hashtags. To the best of our knowledge, this is the\nfirst quantitative evaluation of ZSL methods to date for unseen hashtags\nrecommendations from tweet text.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.04937v1",
            "date": "2019-06-12 04:35:11+00:00",
            "title": "Partial Or Complete, That's The Question",
            "authors": [
                "Qiang Ning",
                "Hangfeng He",
                "Chuchu Fan",
                "Dan Roth"
            ],
            "abstract": "For many structured learning tasks, the data annotation process is complex\nand costly. Existing annotation schemes usually aim at acquiring completely\nannotated structures, under the common perception that partial structures are\nof low quality and could hurt the learning process. This paper questions this\ncommon perception, motivated by the fact that structures consist of\ninterdependent sets of variables. Thus, given a fixed budget, partly annotating\neach structure may provide the same level of supervision, while allowing for\nmore structures to be annotated. We provide an information theoretic\nformulation for this perspective and use it, in the context of three diverse\nstructured learning tasks, to show that learning from partial structures can\nsometimes outperform learning from complete ones. Our findings may provide\nimportant insights into structured data annotation schemes and could support\nprogress in learning protocols for structured tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.04941v1",
            "date": "2019-06-12 04:58:51+00:00",
            "title": "Joint Reasoning for Temporal and Causal Relations",
            "authors": [
                "Qiang Ning",
                "Zhili Feng",
                "Hao Wu",
                "Dan Roth"
            ],
            "abstract": "Understanding temporal and causal relations between events is a fundamental\nnatural language understanding task. Because a cause must be before its effect\nin time, temporal and causal relations are closely related and one relation\neven dictates the other one in many cases. However, limited attention has been\npaid to studying these two relations jointly. This paper presents a joint\ninference framework for them using constrained conditional models (CCMs).\nSpecifically, we formulate the joint problem as an integer linear programming\n(ILP) problem, enforcing constraints inherently in the nature of time and\ncausality. We show that the joint inference framework results in statistically\nsignificant improvement in the extraction of both temporal and causal relations\nfrom text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04980v2",
            "date": "2019-06-12 07:30:32+00:00",
            "title": "Unsupervised Question Answering by Cloze Translation",
            "authors": [
                "Patrick Lewis",
                "Ludovic Denoyer",
                "Sebastian Riedel"
            ],
            "abstract": "Obtaining training data for Question Answering (QA) is time-consuming and\nresource-intensive, and existing QA datasets are only available for limited\ndomains and languages. In this work, we explore to what extent high quality\ntraining data is actually required for Extractive QA, and investigate the\npossibility of unsupervised Extractive QA. We approach this problem by first\nlearning to generate context, question and answer triples in an unsupervised\nmanner, which we then use to synthesize Extractive QA training data\nautomatically. To generate such triples, we first sample random context\nparagraphs from a large corpus of documents and then random noun phrases or\nnamed entity mentions from these paragraphs as answers. Next we convert answers\nin context to \"fill-in-the-blank\" cloze questions and finally translate them\ninto natural questions. We propose and compare various unsupervised ways to\nperform cloze-to-natural question translation, including training an\nunsupervised NMT model using non-aligned corpora of natural questions and cloze\nquestions as well as a rule-based approach. We find that modern QA models can\nlearn to answer human questions surprisingly well using only synthetic training\ndata. We demonstrate that, without using the SQuAD training data at all, our\napproach achieves 56.4 F1 on SQuAD v1 (64.5 F1 when the answer is a Named\nentity mention), outperforming early supervised models.",
            "journal ref": "Proceedings of the 57th Annual Meeting of the Association for\n  Computational Linguistics, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05226v1",
            "date": "2019-06-12 16:01:35+00:00",
            "title": "Continual and Multi-Task Architecture Search",
            "authors": [
                "Ramakanth Pasunuru",
                "Mohit Bansal"
            ],
            "abstract": "Architecture search is the process of automatically learning the neural model\nor cell structure that best suits the given task. Recently, this approach has\nshown promising performance improvements (on language modeling and image\nclassification) with reasonable training speed, using a weight sharing strategy\ncalled Efficient Neural Architecture Search (ENAS). In our work, we first\nintroduce a novel continual architecture search (CAS) approach, so as to\ncontinually evolve the model parameters during the sequential training of\nseveral tasks, without losing performance on previously learned tasks (via\nblock-sparsity and orthogonality constraints), thus enabling life-long\nlearning. Next, we explore a multi-task architecture search (MAS) approach over\nENAS for finding a unified, single cell structure that performs well across\nmultiple tasks (via joint controller rewards), and hence allows more\ngeneralizable transfer of the cell structure knowledge to an unseen new task.\nWe empirically show the effectiveness of our sequential continual learning and\nparallel multi-task learning based architecture search approaches on diverse\nsentence-pair classification tasks (GLUE) and multimodal-generation based video\ncaptioning tasks. Further, we present several ablations and analyses on the\nlearned cell structures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05373v2",
            "date": "2019-06-12 20:49:48+00:00",
            "title": "E3: Entailment-driven Extracting and Editing for Conversational Machine Reading",
            "authors": [
                "Victor Zhong",
                "Luke Zettlemoyer"
            ],
            "abstract": "Conversational machine reading systems help users answer high-level questions\n(e.g. determine if they qualify for particular government benefits) when they\ndo not know the exact rules by which the determination is made(e.g. whether\nthey need certain income levels or veteran status). The key challenge is that\nthese rules are only provided in the form of a procedural text (e.g. guidelines\nfrom government website) which the system must read to figure out what to ask\nthe user. We present a new conversational machine reading model that jointly\nextracts a set of decision rules from the procedural text while reasoning about\nwhich are entailed by the conversational history and which still need to be\nedited to create questions for the user. On the recently introduced ShARC\nconversational machine reading dataset, our Entailment-driven Extract and Edit\nnetwork (E3) achieves a new state-of-the-art, outperforming existing systems as\nwell as a new BERT-based baseline. In addition, by explicitly highlighting\nwhich information still needs to be gathered, E3 provides a more explainable\nalternative to prior work. We release source code for our models and\nexperiments at https://github.com/vzhong/e3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05381v2",
            "date": "2019-06-12 21:25:09+00:00",
            "title": "Compositional generalization through meta sequence-to-sequence learning",
            "authors": [
                "Brenden M. Lake"
            ],
            "abstract": "People can learn a new concept and use it compositionally, understanding how\nto \"blicket twice\" after learning how to \"blicket.\" In contrast, powerful\nsequence-to-sequence (seq2seq) neural networks fail such tests of\ncompositionality, especially when composing new concepts together with existing\nconcepts. In this paper, I show how memory-augmented neural networks can be\ntrained to generalize compositionally through meta seq2seq learning. In this\napproach, models train on a series of seq2seq problems to acquire the\ncompositional skills needed to solve new seq2seq problems. Meta se2seq learning\nsolves several of the SCAN tests for compositional learning and can learn to\napply implicit rules to variables.",
            "journal ref": "Advances in Neural Information Processing Systems 33 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05489v1",
            "date": "2019-06-13 05:39:42+00:00",
            "title": "Cognitive Knowledge Graph Reasoning for One-shot Relational Learning",
            "authors": [
                "Zhengxiao Du",
                "Chang Zhou",
                "Ming Ding",
                "Hongxia Yang",
                "Jie Tang"
            ],
            "abstract": "Inferring new facts from existing knowledge graphs (KG) with explainable\nreasoning processes is a significant problem and has received much attention\nrecently. However, few studies have focused on relation types unseen in the\noriginal KG, given only one or a few instances for training. To bridge this\ngap, we propose CogKR for one-shot KG reasoning. The one-shot relational\nlearning problem is tackled through two modules: the summary module summarizes\nthe underlying relationship of the given instances, based on which the\nreasoning module infers the correct answers. Motivated by the dual process\ntheory in cognitive science, in the reasoning module, a cognitive graph is\nbuilt by iteratively coordinating retrieval (System 1, collecting relevant\nevidence intuitively) and reasoning (System 2, conducting relational reasoning\nover collected information). The structural information offered by the\ncognitive graph enables our model to aggregate pieces of evidence from multiple\nreasoning paths and explain the reasoning process graphically. Experiments show\nthat CogKR substantially outperforms previous state-of-the-art models on\none-shot KG reasoning benchmarks, with relative improvements of 24.3%-29.7% on\nMRR. The source code is available at https://github.com/THUDM/CogKR.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.05651v1",
            "date": "2019-06-12 17:29:22+00:00",
            "title": "Representation Learning for Words and Entities",
            "authors": [
                "Pushpendre Rastogi"
            ],
            "abstract": "This thesis presents new methods for unsupervised learning of distributed\nrepresentations of words and entities from text and knowledge bases. The first\nalgorithm presented in the thesis is a multi-view algorithm for learning\nrepresentations of words called Multiview Latent Semantic Analysis (MVLSA). By\nincorporating up to 46 different types of co-occurrence statistics for the same\nvocabulary of english words, I show that MVLSA outperforms other\nstate-of-the-art word embedding models. Next, I focus on learning entity\nrepresentations for search and recommendation and present the second method of\nthis thesis, Neural Variational Set Expansion (NVSE). NVSE is also an\nunsupervised learning method, but it is based on the Variational Autoencoder\nframework. Evaluations with human annotators show that NVSE can facilitate\nbetter search and recommendation of information gathered from noisy, automatic\nannotation of unstructured natural language corpora. Finally, I move from\nunstructured data and focus on structured knowledge graphs. I present novel\napproaches for learning embeddings of vertices and edges in a knowledge graph\nthat obey logical constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05664v1",
            "date": "2019-06-11 17:00:49+00:00",
            "title": "Calibration, Entropy Rates, and Memory in Language Models",
            "authors": [
                "Mark Braverman",
                "Xinyi Chen",
                "Sham M. Kakade",
                "Karthik Narasimhan",
                "Cyril Zhang",
                "Yi Zhang"
            ],
            "abstract": "Building accurate language models that capture meaningful long-term\ndependencies is a core challenge in natural language processing. Towards this\nend, we present a calibration-based approach to measure long-term discrepancies\nbetween a generative sequence model and the true distribution, and use these\ndiscrepancies to improve the model. Empirically, we show that state-of-the-art\nlanguage models, including LSTMs and Transformers, are \\emph{miscalibrated}:\nthe entropy rates of their generations drift dramatically upward over time. We\nthen provide provable methods to mitigate this phenomenon. Furthermore, we show\nhow this calibration-based approach can also be used to measure the amount of\nmemory that language models use for prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05685v2",
            "date": "2019-06-11 15:38:34+00:00",
            "title": "A Focus on Neural Machine Translation for African Languages",
            "authors": [
                "Laura Martinus",
                "Jade Z. Abbott"
            ],
            "abstract": "African languages are numerous, complex and low-resourced. The datasets\nrequired for machine translation are difficult to discover, and existing\nresearch is hard to reproduce. Minimal attention has been given to machine\ntranslation for African languages so there is scant research regarding the\nproblems that arise when using machine translation techniques. To begin\naddressing these problems, we trained models to translate English to five of\nthe official South African languages (Afrikaans, isiZulu, Northern Sotho,\nSetswana, Xitsonga), making use of modern neural machine translation\ntechniques. The results obtained show the promise of using neural machine\ntranslation techniques for African languages. By providing reproducible\npublicly-available data, code and results, this research aims to provide a\nstarting point for other researchers in African machine translation to compare\nto and build upon.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05691v1",
            "date": "2019-06-13 13:53:10+00:00",
            "title": "Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking",
            "authors": [
                "Masaru Isonuma",
                "Junichiro Mori",
                "Ichiro Sakata"
            ],
            "abstract": "This paper focuses on the end-to-end abstractive summarization of a single\nproduct review without supervision. We assume that a review can be described as\na discourse tree, in which the summary is the root, and the child sentences\nexplain their parent in detail. By recursively estimating a parent from its\nchildren, our model learns the latent discourse tree without an external parser\nand generates a concise summary. We also introduce an architecture that ranks\nthe importance of each sentence on the tree to support summary generation\nfocusing on the main review point. The experimental results demonstrate that\nour model is competitive with or outperforms other unsupervised approaches. In\nparticular, for relatively long reviews, it achieves a competitive or better\nperformance than supervised models. The induced tree shows that the child\nsentences provide additional information about their parent, and the generated\nsummary abstracts the entire review.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.05714v1",
            "date": "2019-06-12 15:45:26+00:00",
            "title": "A Multiscale Visualization of Attention in the Transformer Model",
            "authors": [
                "Jesse Vig"
            ],
            "abstract": "The Transformer is a sequence model that forgoes traditional recurrent\narchitectures in favor of a fully attention-based approach. Besides improving\nperformance, an advantage of using attention is that it can also help to\ninterpret a model by showing how the model assigns weight to different input\nelements. However, the multi-layer, multi-head attention mechanism in the\nTransformer model can be difficult to decipher. To make the model more\naccessible, we introduce an open-source tool that visualizes attention at\nmultiple scales, each of which provides a unique perspective on the attention\nmechanism. We demonstrate the tool on BERT and OpenAI GPT-2 and present three\nexample use cases: detecting model bias, locating relevant attention heads, and\nlinking neurons to model behavior.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1906.05962v1",
            "date": "2019-06-13 23:52:16+00:00",
            "title": "Speaker-Targeted Audio-Visual Models for Speech Recognition in Cocktail-Party Environments",
            "authors": [
                "Guan-Lin Chao",
                "William Chan",
                "Ian Lane"
            ],
            "abstract": "Speech recognition in cocktail-party environments remains a significant\nchallenge for state-of-the-art speech recognition systems, as it is extremely\ndifficult to extract an acoustic signal of an individual speaker from a\nbackground of overlapping speech with similar frequency and temporal\ncharacteristics. We propose the use of speaker-targeted acoustic and\naudio-visual models for this task. We complement the acoustic features in a\nhybrid DNN-HMM model with information of the target speaker's identity as well\nas visual features from the mouth region of the target speaker. Experimentation\nwas performed using simulated cocktail-party data generated from the GRID\naudio-visual corpus by overlapping two speakers's speech on a single acoustic\nchannel. Our audio-only baseline achieved a WER of 26.3%. The audio-visual\nmodel improved the WER to 4.4%. Introducing speaker identity information had an\neven more pronounced effect, improving the WER to 3.6%. Combining both\napproaches, however, did not significantly improve performance further. Our\nwork demonstrates that speaker-targeted models can significantly improve the\nspeech recognition in cocktail party environments.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.06298v3",
            "date": "2019-06-14 17:10:42+00:00",
            "title": "Augmenting Neural Networks with First-order Logic",
            "authors": [
                "Tao Li",
                "Vivek Srikumar"
            ],
            "abstract": "Today, the dominant paradigm for training neural networks involves minimizing\ntask loss on a large dataset. Using world knowledge to inform a model, and yet\nretain the ability to perform end-to-end training remains an open question. In\nthis paper, we present a novel framework for introducing declarative knowledge\nto neural network architectures in order to guide training and prediction. Our\nframework systematically compiles logical statements into computation graphs\nthat augment a neural network without extra learnable parameters or manual\nredesign. We evaluate our modeling strategy on three tasks: machine\ncomprehension, natural language inference, and text chunking. Our experiments\nshow that knowledge-augmented networks can strongly improve over baselines,\nespecially in low-data regimes.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.06465v2",
            "date": "2019-06-13 01:42:23+00:00",
            "title": "Correlating Twitter Language with Community-Level Health Outcomes",
            "authors": [
                "Arno Schneuwly",
                "Ralf Grubenmann",
                "S\u00e9verine Rion Logean",
                "Mark Cieliebak",
                "Martin Jaggi"
            ],
            "abstract": "We study how language on social media is linked to diseases such as\natherosclerotic heart disease (AHD), diabetes and various types of cancer. Our\nproposed model leverages state-of-the-art sentence embeddings, followed by a\nregression model and clustering, without the need of additional labelled data.\nIt allows to predict community-level medical outcomes from language, and\nthereby potentially translate these to the individual level. The method is\napplicable to a wide range of target variables and allows us to discover known\nand potentially novel correlations of medical outcomes with life-style aspects\nand other socioeconomic risk factors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06582v2",
            "date": "2019-06-15 15:57:57+00:00",
            "title": "A Computational-Hermeneutic Approach for Conceptual Explicitation",
            "authors": [
                "David Fuenmayor",
                "Christoph Benzm\u00fcller"
            ],
            "abstract": "We present a computer-supported approach for the logical analysis and\nconceptual explicitation of argumentative discourse. Computational hermeneutics\nharnesses recent progresses in automated reasoning for higher-order logics and\naims at formalizing natural-language argumentative discourse using flexible\ncombinations of expressive non-classical logics. In doing so, it allows us to\nrender explicit the tacit conceptualizations implicit in argumentative\ndiscursive practices. Our approach operates on networks of structured arguments\nand is iterative and two-layered. At one layer we search for logically correct\nformalizations for each of the individual arguments. At the next layer we\nselect among those correct formalizations the ones which honor the argument's\ndialectic role, i.e. attacking or supporting other arguments as intended. We\noperate at these two layers in parallel and continuously rate sentences'\nformalizations by using, primarily, inferential adequacy criteria. An\ninterpretive, logical theory will thus gradually evolve. This theory is\ncomposed of meaning postulates serving as explications for concepts playing a\nrole in the analyzed arguments. Such a recursive, iterative approach to\ninterpretation does justice to the inherent circularity of understanding: the\nwhole is understood compositionally on the basis of its parts, while each part\nis understood only in the context of the whole (hermeneutic circle). We\nsummarily discuss previous work on exemplary applications of human-in-the-loop\ncomputational hermeneutics in metaphysical discourse. We also discuss some of\nthe main challenges involved in fully-automating our approach. By sketching\nsome design ideas and reviewing relevant technologies, we argue for the\ntechnological feasibility of a highly-automated computational hermeneutics.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.06719v4",
            "date": "2019-06-16 15:41:07+00:00",
            "title": "Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation",
            "authors": [
                "Wenxian Shi",
                "Hao Zhou",
                "Ning Miao",
                "Lei Li"
            ],
            "abstract": "Deep generative models are commonly used for generating images and text.\nInterpretability of these models is one important pursuit, other than the\ngeneration quality. Variational auto-encoder (VAE) with Gaussian distribution\nas prior has been successfully applied in text generation, but it is hard to\ninterpret the meaning of the latent variable. To enhance the controllability\nand interpretability, one can replace the Gaussian prior with a mixture of\nGaussian distributions (GM-VAE), whose mixture components could be related to\nhidden semantic aspects of data. In this paper, we generalize the practice and\nintroduce DEM-VAE, a class of models for text generation using VAEs with a\nmixture distribution of exponential family. Unfortunately, a standard\nvariational training algorithm fails due to the mode-collapse problem. We\ntheoretically identify the root cause of the problem and propose an effective\nalgorithm to train DEM-VAE. Our method penalizes the training with an extra\ndispersion term to induce a well-structured latent space. Experimental results\nshow that our approach does obtain a meaningful space, and it outperforms\nstrong baselines in text generation benchmarks. The code is available at\nhttps://github.com/wenxianxian/demvae.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.06725v2",
            "date": "2019-06-16 16:43:02+00:00",
            "title": "Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good",
            "authors": [
                "Xuewei Wang",
                "Weiyan Shi",
                "Richard Kim",
                "Yoojung Oh",
                "Sijia Yang",
                "Jingwen Zhang",
                "Zhou Yu"
            ],
            "abstract": "Developing intelligent persuasive conversational agents to change people's\nopinions and actions for social good is the frontier in advancing the ethical\ndevelopment of automated dialogue systems. To do so, the first step is to\nunderstand the intricate organization of strategic disclosures and appeals\nemployed in human persuasion conversations. We designed an online persuasion\ntask where one participant was asked to persuade the other to donate to a\nspecific charity. We collected a large dataset with 1,017 dialogues and\nannotated emerging persuasion strategies from a subset. Based on the\nannotation, we built a baseline classifier with context information and\nsentence-level features to predict the 10 persuasion strategies used in the\ncorpus. Furthermore, to develop an understanding of personalized persuasion\nprocesses, we analyzed the relationships between individuals' demographic and\npsychological backgrounds including personality, morality, value systems, and\ntheir willingness for donation. Then, we analyzed which types of persuasion\nstrategies led to a greater amount of donation depending on the individuals'\npersonal backgrounds. This work lays the ground for developing a personalized\npersuasive dialogue system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06755v2",
            "date": "2019-06-16 19:19:49+00:00",
            "title": "Theoretical Limitations of Self-Attention in Neural Sequence Models",
            "authors": [
                "Michael Hahn"
            ],
            "abstract": "Transformers are emerging as the new workhorse of NLP, showing great success\nacross tasks. Unlike LSTMs, transformers process input sequences entirely\nthrough self-attention. Previous work has suggested that the computational\ncapabilities of self-attention to process hierarchical structures are limited.\nIn this work, we mathematically investigate the computational power of\nself-attention to model formal languages. Across both soft and hard attention,\nwe show strong theoretical limitations of the computational abilities of\nself-attention, finding that it cannot model periodic finite-state languages,\nnor hierarchical structure, unless the number of layers or heads increases with\ninput length. These limitations seem surprising given the practical success of\nself-attention and the prominent role assigned to hierarchical structure in\nlinguistics, suggesting that natural language can be approximated well with\nmodels that are too weak for the formal languages typically assumed in\ntheoretical linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.06788v4",
            "date": "2019-06-16 22:36:33+00:00",
            "title": "SEntNet: Source-aware Recurrent Entity Network for Dialogue Response Selection",
            "authors": [
                "Jiahuan Pei",
                "Arent Stienstra",
                "Julia Kiseleva",
                "Maarten de Rijke"
            ],
            "abstract": "Dialogue response selection is an important part of Task-oriented Dialogue\nSystems (TDSs); it aims to predict an appropriate response given a dialogue\ncontext. Obtaining key information from a complex, long dialogue context is\nchallenging, especially when different sources of information are available,\ne.g., the user's utterances, the system's responses, and results retrieved from\na knowledge base (KB). Previous work ignores the type of information source and\nmerges sources for response selection. However, accounting for the source type\nmay lead to remarkable differences in the quality of response selection. We\npropose the Source-aware Recurrent Entity Network (SEntNet), which is aware of\ndifferent information sources for the response selection process. SEntNet\nachieves this by employing source-specific memories to exploit differences in\nthe usage of words and syntactic structure from different information sources\n(user, system, and KB). Experimental results show that SEntNet obtains 91.0%\naccuracy on the Dialog bAbI dataset, outperforming prior work by 4.7%. On the\nDSTC2 dataset, SEntNet obtains an accuracy of 41.2%, beating source unaware\nrecurrent entity networks by 2.4%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.06892v1",
            "date": "2019-06-17 08:26:43+00:00",
            "title": "ParNet: Position-aware Aggregated Relation Network for Image-Text matching",
            "authors": [
                "Yaxian Xia",
                "Lun Huang",
                "Wenmin Wang",
                "Xiaoyong Wei",
                "Wenmin Wang"
            ],
            "abstract": "Exploring fine-grained relationship between entities(e.g. objects in image or\nwords in sentence) has great contribution to understand multimedia content\nprecisely. Previous attention mechanism employed in image-text matching either\ntakes multiple self attention steps to gather correspondences or uses image\nobjects (or words) as context to infer image-text similarity. However, they\nonly take advantage of semantic information without considering that objects'\nrelative position also contributes to image understanding. To this end, we\nintroduce a novel position-aware relation module to model both the semantic and\nspatial relationship simultaneously for image-text matching in this paper.\nGiven an image, our method utilizes the location of different objects to\ncapture spatial relationship innovatively. With the combination of semantic and\nspatial relationship, it's easier to understand the content of different\nmodalities (images and sentences) and capture fine-grained latent\ncorrespondences of image-text pairs. Besides, we employ a two-step aggregated\nrelation module to capture interpretable alignment of image-text pairs. The\nfirst step, we call it intra-modal relation mechanism, in which we computes\nresponses between different objects in an image or different words in a\nsentence separately; The second step, we call it inter-modal relation\nmechanism, in which the query plays a role of textual context to refine the\nrelationship among object proposals in an image. In this way, our\nposition-aware aggregated relation network (ParNet) not only knows which\nentities are relevant by attending on different objects (words) adaptively, but\nalso adjust the inter-modal correspondence according to the latent alignments\naccording to query's content. Our approach achieves the state-of-the-art\nresults on MS-COCO dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.07093v1",
            "date": "2019-06-17 15:42:26+00:00",
            "title": "Adversarial Training for Multilingual Acoustic Modeling",
            "authors": [
                "Ke Hu",
                "Hasim Sak",
                "Hank Liao"
            ],
            "abstract": "Multilingual training has been shown to improve acoustic modeling performance\nby sharing and transferring knowledge in modeling different languages.\nKnowledge sharing is usually achieved by using common lower-level layers for\ndifferent languages in a deep neural network. Recently, the domain adversarial\nnetwork was proposed to reduce domain mismatch of training data and learn\ndomain-invariant features. It is thus worth exploring whether adversarial\ntraining can further promote knowledge sharing in multilingual models. In this\nwork, we apply the domain adversarial network to encourage the shared layers of\na multilingual model to learn language-invariant features. Bidirectional Long\nShort-Term Memory (LSTM) recurrent neural networks (RNN) are used as building\nblocks. We show that shared layers learned this way contain less language\nidentification information and lead to better performance. In an automatic\nspeech recognition task for seven languages, the resultant acoustic model\nimproves the word error rate (WER) of the multilingual model by 4% relative on\naverage, and the monolingual models by 10%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07234v3",
            "date": "2019-06-17 19:40:46+00:00",
            "title": "Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling",
            "authors": [
                "Siyuan Feng",
                "Tan Lee",
                "Zhiyuan Peng"
            ],
            "abstract": "This study addresses the problem of unsupervised subword unit discovery from\nuntranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech\n2019, building text-to-speech systems without text labels. In this work, unit\ndiscovery is formulated as a pipeline of phonetically discriminative feature\nlearning and unit inference. One major difficulty in robust unsupervised\nfeature learning is dealing with speaker variation. Here the robustness towards\nspeaker variation is achieved by applying adversarial training and FHVAE based\ndisentangled speech representation learning. A comparison of the two approaches\nas well as their combination is studied in a DNN-bottleneck feature (DNN-BNF)\narchitecture. Experiments are conducted on ZeroSpeech 2019 and 2017.\nExperimental results on ZeroSpeech 2017 show that both approaches are effective\nwhile the latter is more prominent, and that their combination brings further\nmarginal improvement in across-speaker condition. Results on ZeroSpeech 2019\nshow that in the ABX discriminability task, our approaches significantly\noutperform the official baseline, and are competitive to or even outperform the\nofficial topline. The proposed unit sequence smoothing algorithm improves\nsynthesis quality, at a cost of slight decrease in ABX discriminability.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.07307v1",
            "date": "2019-06-17 23:48:05+00:00",
            "title": "Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models",
            "authors": [
                "Wei Fang",
                "Yu-An Chung",
                "James Glass"
            ],
            "abstract": "Modern text-to-speech (TTS) systems are able to generate audio that sounds\nalmost as natural as human speech. However, the bar of developing high-quality\nTTS systems remains high since a sizable set of studio-quality <text, audio>\npairs is usually required. Compared to commercial data used to develop\nstate-of-the-art systems, publicly available data are usually worse in terms of\nboth quality and size. Audio generated by TTS systems trained on publicly\navailable data tends to not only sound less natural, but also exhibits more\nbackground noise. In this work, we aim to lower TTS systems' reliance on\nhigh-quality data by providing them the textual knowledge extracted by deep\npre-trained language models during training. In particular, we investigate the\nuse of BERT to assist the training of Tacotron-2, a state of the art TTS\nconsisting of an encoder and an attention-based decoder. BERT representations\nlearned from large amounts of unlabeled text data are shown to contain very\nrich semantic and syntactic information about the input text, and have\npotential to be leveraged by a TTS system to compensate the lack of\nhigh-quality data. We incorporate BERT as a parallel branch to the Tacotron-2\nencoder with its own attention head. For an input text, it is simultaneously\npassed into BERT and the Tacotron-2 encoder. The representations extracted by\nthe two branches are concatenated and then fed to the decoder. As a preliminary\nstudy, although we have not found incorporating BERT into Tacotron-2 generates\nmore natural or cleaner speech at a human-perceivable level, we observe\nimprovements in other aspects such as the model is being significantly better\nat knowing when to stop decoding such that there is much less babbling at the\nend of the synthesized audio and faster convergence during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07317v1",
            "date": "2019-06-18 00:31:04+00:00",
            "title": "Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition",
            "authors": [
                "Xu Xiang",
                "Shuai Wang",
                "Houjun Huang",
                "Yanmin Qian",
                "Kai Yu"
            ],
            "abstract": "Recently, speaker embeddings extracted from a speaker discriminative deep\nneural network (DNN) yield better performance than the conventional methods\nsuch as i-vector. In most cases, the DNN speaker classifier is trained using\ncross entropy loss with softmax. However, this kind of loss function does not\nexplicitly encourage inter-class separability and intra-class compactness. As a\nresult, the embeddings are not optimal for speaker recognition tasks. In this\npaper, to address this issue, three different margin based losses which not\nonly separate classes but also demand a fixed margin between classes are\nintroduced to deep speaker embedding learning. It could be demonstrated that\nthe margin is the key to obtain more discriminative speaker embeddings.\nExperiments are conducted on two public text independent tasks: VoxCeleb1 and\nSpeaker in The Wild (SITW). The proposed approach can achieve the\nstate-of-the-art performance, with 25% ~ 30% equal error rate (EER) reduction\non both tasks when compared to strong baselines using cross entropy loss with\nsoftmax, obtaining 2.238% EER on VoxCeleb1 test set and 2.761% EER on SITW\ncore-core test set, respectively.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.07343v2",
            "date": "2019-06-18 02:27:45+00:00",
            "title": "Language as an Abstraction for Hierarchical Deep Reinforcement Learning",
            "authors": [
                "Yiding Jiang",
                "Shixiang Gu",
                "Kevin Murphy",
                "Chelsea Finn"
            ],
            "abstract": "Solving complex, temporally-extended tasks is a long-standing problem in\nreinforcement learning (RL). We hypothesize that one critical element of\nsolving such problems is the notion of compositionality. With the ability to\nlearn concepts and sub-skills that can be composed to solve longer tasks, i.e.\nhierarchical RL, we can acquire temporally-extended behaviors. However,\nacquiring effective yet general abstractions for hierarchical RL is remarkably\nchallenging. In this paper, we propose to use language as the abstraction, as\nit provides unique compositional structure, enabling fast learning and\ncombinatorial generalization, while retaining tremendous flexibility, making it\nsuitable for a variety of problems. Our approach learns an\ninstruction-following low-level policy and a high-level policy that can reuse\nabstractions across tasks, in essence, permitting agents to reason using\nstructured language. To study compositional task learning, we introduce an\nopen-source object interaction environment built using the MuJoCo physics\nengine and the CLEVR engine. We find that, using our approach, agents can learn\nto solve to diverse, temporally-extended tasks such as object sorting and\nmulti-object rearrangement, including from raw pixel observations. Our analysis\nreveals that the compositional nature of language is critical for learning\ndiverse sub-skills and systematically generalizing to new sub-skills in\ncomparison to non-compositional abstractions that use the same supervision.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.07389v1",
            "date": "2019-06-18 05:51:13+00:00",
            "title": "Uncovering Probabilistic Implications in Typological Knowledge Bases",
            "authors": [
                "Johannes Bjerva",
                "Yova Kementchedjhieva",
                "Ryan Cotterell",
                "Isabelle Augenstein"
            ],
            "abstract": "The study of linguistic typology is rooted in the implications we find\nbetween linguistic features, such as the fact that languages with object-verb\nword ordering tend to have post-positions. Uncovering such implications\ntypically amounts to time-consuming manual processing by trained and\nexperienced linguists, which potentially leaves key linguistic universals\nunexplored. In this paper, we present a computational model which successfully\nidentifies known universals, including Greenberg universals, but also uncovers\nnew ones, worthy of further linguistic investigation. Our approach outperforms\nbaselines previously used for this problem, as well as a strong baseline from\nknowledge base population.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07414v2",
            "date": "2019-06-18 07:24:45+00:00",
            "title": "A Unified Speaker Adaptation Method for Speech Synthesis using Transcribed and Untranscribed Speech with Backpropagation",
            "authors": [
                "Hieu-Thi Luong",
                "Junichi Yamagishi"
            ],
            "abstract": "By representing speaker characteristic as a single fixed-length vector\nextracted solely from speech, we can train a neural multi-speaker speech\nsynthesis model by conditioning the model on those vectors. This model can also\nbe adapted to unseen speakers regardless of whether the transcript of\nadaptation data is available or not. However, this setup restricts the speaker\ncomponent to just a single bias vector, which in turn limits the performance of\nadaptation process. In this study, we propose a novel speech synthesis model,\nwhich can be adapted to unseen speakers by fine-tuning part of or all of the\nnetwork using either transcribed or untranscribed speech. Our methodology\nessentially consists of two steps: first, we split the conventional acoustic\nmodel into a speaker-independent (SI) linguistic encoder and a speaker-adaptive\n(SA) acoustic decoder; second, we train an auxiliary acoustic encoder that can\nbe used as a substitute for the linguistic encoder whenever linguistic features\nare unobtainable. The results of objective and subjective evaluations show that\nadaptation using either transcribed or untranscribed speech with our\nmethodology achieved a reasonable level of performance with an extremely\nlimited amount of data and greatly improved performance with more data.\nSurprisingly, adaptation with untranscribed speech surpassed the transcribed\ncounterpart in the subjective test, which reveals the limitations of the\nconventional acoustic model and hints at potential directions for improvements.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.07523v2",
            "date": "2019-06-18 12:24:32+00:00",
            "title": "Multi-Graph Decoding for Code-Switching ASR",
            "authors": [
                "Emre Y\u0131lmaz",
                "Samuel Cohen",
                "Xianghu Yue",
                "David van Leeuwen",
                "Haizhou Li"
            ],
            "abstract": "In the FAME! Project, a code-switching (CS) automatic speech recognition\n(ASR) system for Frisian-Dutch speech is developed that can accurately\ntranscribe the local broadcaster's bilingual archives with CS speech. This\narchive contains recordings with monolingual Frisian and Dutch speech segments\nas well as Frisian-Dutch CS speech, hence the recognition performance on\nmonolingual segments is also vital for accurate transcriptions. In this work,\nwe propose a multi-graph decoding and rescoring strategy using bilingual and\nmonolingual graphs together with a unified acoustic model for CS ASR. The\nproposed decoding scheme gives the freedom to design and employ alternative\nsearch spaces for each (monolingual or bilingual) recognition task and enables\nthe effective use of monolingual resources of the high-resourced mixed language\nin low-resourced CS scenarios. In our scenario, Dutch is the high-resourced and\nFrisian is the low-resourced language. We therefore use additional monolingual\nDutch text resources to improve the Dutch language model (LM) and compare the\nperformance of single- and multi-graph CS ASR systems on Dutch segments using\nlarger Dutch LMs. The ASR results show that the proposed approach outperforms\nbaseline single-graph CS ASR systems, providing better performance on the\nmonolingual Dutch segments without any accuracy loss on monolingual Frisian and\ncode-mixed segments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07601v1",
            "date": "2019-06-18 14:19:52+00:00",
            "title": "Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability",
            "authors": [
                "Antoine Caubri\u00e8re",
                "Natalia Tomashenko",
                "Antoine Laurent",
                "Emmanuel Morin",
                "Nathalie Camelin",
                "Yannick Est\u00e8ve"
            ],
            "abstract": "We present an end-to-end approach to extract semantic concepts directly from\nthe speech audio signal. To overcome the lack of data available for this spoken\nlanguage understanding approach, we investigate the use of a transfer learning\nstrategy based on the principles of curriculum learning. This approach allows\nus to exploit out-of-domain data that can help to prepare a fully neural\narchitecture. Experiments are carried out on the French MEDIA and PORTMEDIA\ncorpora and show that this end-to-end SLU approach reaches the best results\never published on this task. We compare our approach to a classical pipeline\napproach that uses ASR, POS tagging, lemmatizer, chunker... and other NLP tools\nthat aim to enrich ASR outputs that feed an SLU text to concepts system. Last,\nwe explore the promising capacity of our end-to-end SLU approach to address the\nproblem of domain portability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07668v1",
            "date": "2019-06-15 20:56:48+00:00",
            "title": "Yoga-Veganism: Correlation Mining of Twitter Health Data",
            "authors": [
                "Tunazzina Islam"
            ],
            "abstract": "Nowadays social media is a huge platform of data. People usually share their\ninterest, thoughts via discussions, tweets, status. It is not possible to go\nthrough all the data manually. We need to mine the data to explore hidden\npatterns or unknown correlations, find out the dominant topic in data and\nunderstand people's interest through the discussions. In this work, we explore\nTwitter data related to health. We extract the popular topics under different\ncategories (e.g. diet, exercise) discussed in Twitter via topic modeling,\nobserve model behavior on new tweets, discover interesting correlation (i.e.\nYoga-Veganism). We evaluate accuracy by comparing with ground truth using\nmanual annotation both for train and test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07689v2",
            "date": "2019-06-18 17:01:21+00:00",
            "title": "Expressing Visual Relationships via Language",
            "authors": [
                "Hao Tan",
                "Franck Dernoncourt",
                "Zhe Lin",
                "Trung Bui",
                "Mohit Bansal"
            ],
            "abstract": "Describing images with text is a fundamental problem in vision-language\nresearch. Current studies in this domain mostly focus on single image\ncaptioning. However, in various real applications (e.g., image editing,\ndifference interpretation, and retrieval), generating relational captions for\ntwo images, can also be very useful. This important problem has not been\nexplored mostly due to lack of datasets and effective models. To push forward\nthe research in this direction, we first introduce a new language-guided image\nediting dataset that contains a large number of real image pairs with\ncorresponding editing instructions. We then propose a new relational speaker\nmodel based on an encoder-decoder architecture with static relational attention\nand sequential multi-head attention. We also extend the model with dynamic\nrelational attention, which calculates visual alignment while decoding. Our\nmodels are evaluated on our newly collected and two public datasets consisting\nof image pairs annotated with relationship sentences. Experimental results,\nbased on both automatic and human evaluation, demonstrate that our model\noutperforms all baselines and existing methods on all the datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07886v2",
            "date": "2019-06-19 02:43:50+00:00",
            "title": "Learning with Partially Ordered Representations",
            "authors": [
                "Jane Chandlee",
                "Remi Eyraud",
                "Jeffrey Heinz",
                "Adam Jardine",
                "Jonathan Rawski"
            ],
            "abstract": "This paper examines the characterization and learning of grammars defined\nwith enriched representational models. Model-theoretic approaches to formal\nlanguage theory traditionally assume that each position in a string belongs to\nexactly one unary relation. We consider unconventional string models where\npositions can have multiple, shared properties, which are arguably useful in\nmany applications. We show the structures given by these models are partially\nordered, and present a learning algorithm that exploits this ordering relation\nto effectively prune the hypothesis space. We prove this learning algorithm,\nwhich takes positive examples as input, finds the most general grammar which\ncovers the data.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07901v1",
            "date": "2019-06-19 03:52:42+00:00",
            "title": "Multimodal Abstractive Summarization for How2 Videos",
            "authors": [
                "Shruti Palaskar",
                "Jindrich Libovick\u00fd",
                "Spandana Gella",
                "Florian Metze"
            ],
            "abstract": "In this paper, we study abstractive summarization for open-domain videos.\nUnlike the traditional text news summarization, the goal is less to \"compress\"\ntext information but rather to provide a fluent textual summary of information\nthat has been collected and fused from different source modalities, in our case\nvideo and audio transcripts (or text). We show how a multi-source\nsequence-to-sequence model with hierarchical attention can integrate\ninformation from different modalities into a coherent output, compare various\nmodels trained with different modalities and present pilot experiments on the\nHow2 corpus of instructional videos. We also propose a new evaluation metric\n(Content F1) for abstractive summarization task that measures semantic adequacy\nrather than fluency of the summaries, which is covered by metrics like ROUGE\nand BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.07955v2",
            "date": "2019-06-19 07:58:42+00:00",
            "title": "Large-Scale Speaker Diarization of Radio Broadcast Archives",
            "authors": [
                "Emre Y\u0131lmaz",
                "Adem Derinel",
                "Zhou Kun",
                "Henk van den Heuvel",
                "Niko Brummer",
                "Haizhou Li",
                "David A. van Leeuwen"
            ],
            "abstract": "This paper describes our initial efforts to build a large-scale speaker\ndiarization (SD) and identification system on a recently digitized radio\nbroadcast archive from the Netherlands which has more than 6500 audio tapes\nwith 3000 hours of Frisian-Dutch speech recorded between 1950-2016. The\nemployed large-scale diarization scheme involves two stages: (1) tape-level\nspeaker diarization providing pseudo-speaker identities and (2) speaker linking\nto relate pseudo-speakers appearing in multiple tapes. Having access to the\nspeaker models of several frequently appearing speakers from the previously\ncollected FAME! speech corpus, we further perform speaker identification by\nlinking these known speakers to the pseudo-speakers identified at the first\nstage. In this work, we present a recently created longitudinal and\nmultilingual SD corpus designed for large-scale SD research and evaluate the\nperformance of a new speaker linking system using x-vectors with PLDA to\nquantify cross-tape speaker similarity on this corpus. The performance of this\nspeaker linking system is evaluated on a small subset of the archive which is\nmanually annotated with speaker information. The speaker linking performance\nreported on this subset (53 hours) and the whole archive (3000 hours) is\ncompared to quantify the impact of scaling up in the amount of speech data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08003v1",
            "date": "2019-06-19 09:56:34+00:00",
            "title": "Code-Switching Detection Using ASR-Generated Language Posteriors",
            "authors": [
                "Qinyi Wang",
                "Emre Y\u0131lmaz",
                "Adem Derinel",
                "Haizhou Li"
            ],
            "abstract": "Code-switching (CS) detection refers to the automatic detection of language\nswitches in code-mixed utterances. This task can be achieved by using a CS\nautomatic speech recognition (ASR) system that can handle such language\nswitches. In our previous work, we have investigated the code-switching\ndetection performance of the Frisian-Dutch CS ASR system by using the time\nalignment of the most likely hypothesis and found that this technique suffers\nfrom over-switching due to numerous very short spurious language switches. In\nthis paper, we propose a novel method for CS detection aiming to remedy this\nshortcoming by using the language posteriors which are the sum of the\nframe-level posteriors of phones belonging to the same language. The CS\nASR-generated language posteriors contain more complete language-specific\ninformation on frame level compared to the time alignment of the ASR output.\nHence, it is expected to yield more accurate and robust CS detection. The CS\ndetection experiments demonstrate that the proposed language posterior-based\napproach provides higher detection accuracy than the baseline system in terms\nof equal error rate. Moreover, a detailed CS detection error analysis reveals\nthat using language posteriors reduces the false alarms and results in more\nrobust CS detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08041v2",
            "date": "2019-06-17 23:00:15+00:00",
            "title": "Multi-Stream End-to-End Speech Recognition",
            "authors": [
                "Ruizhi Li",
                "Xiaofei Wang",
                "Sri Harish Mallidi",
                "Shinji Watanabe",
                "Takaaki Hori",
                "Hynek Hermansky"
            ],
            "abstract": "Attention-based methods and Connectionist Temporal Classification (CTC)\nnetwork have been promising research directions for end-to-end (E2E) Automatic\nSpeech Recognition (ASR). The joint CTC/Attention model has achieved great\nsuccess by utilizing both architectures during multi-task training and joint\ndecoding. In this work, we present a multi-stream framework based on joint\nCTC/Attention E2E ASR with parallel streams represented by separate encoders\naiming to capture diverse information. On top of the regular attention\nnetworks, the Hierarchical Attention Network (HAN) is introduced to steer the\ndecoder toward the most informative encoders. A separate CTC network is\nassigned to each stream to force monotonic alignments. Two representative\nframework have been proposed and discussed, which are Multi-Encoder\nMulti-Resolution (MEM-Res) framework and Multi-Encoder Multi-Array (MEM-Array)\nframework, respectively. In MEM-Res framework, two heterogeneous encoders with\ndifferent architectures, temporal resolutions and separate CTC networks work in\nparallel to extract complimentary information from same acoustics. Experiments\nare conducted on Wall Street Journal (WSJ) and CHiME-4, resulting in relative\nWord Error Rate (WER) reduction of 18.0-32.1% and the best WER of 3.6% in the\nWSJ eval92 test set. The MEM-Array framework aims at improving the far-field\nASR robustness using multiple microphone arrays which are activated by separate\nencoders. Compared with the best single-array results, the proposed framework\nhas achieved relative WER reduction of 3.7% and 9.7% in AMI and DIRHA\nmulti-array corpora, respectively, which also outperforms conventional fusion\nstrategies.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.08042v1",
            "date": "2019-06-17 20:33:24+00:00",
            "title": "Low-resource Deep Entity Resolution with Transfer and Active Learning",
            "authors": [
                "Jungo Kasai",
                "Kun Qian",
                "Sairam Gurajada",
                "Yunyao Li",
                "Lucian Popa"
            ],
            "abstract": "Entity resolution (ER) is the task of identifying different representations\nof the same real-world entities across databases. It is a key step for\nknowledge base creation and text mining. Recent adaptation of deep learning\nmethods for ER mitigates the need for dataset-specific feature engineering by\nconstructing distributed representations of entity records. While these methods\nachieve state-of-the-art performance over benchmark data, they require large\namounts of labeled data, which are typically unavailable in realistic ER\napplications. In this paper, we develop a deep learning-based method that\ntargets low-resource settings for ER through a novel combination of transfer\nlearning and active learning. We design an architecture that allows us to learn\na transferable model from a high-resource setting to a low-resource one. To\nfurther adapt to the target dataset, we incorporate active learning that\ncarefully selects a few informative examples to fine-tune the transferred\nmodel. Empirical evaluation demonstrates that our method achieves comparable,\nif not better, performance compared to state-of-the-art learning-based methods\nwhile using an order of magnitude fewer labels.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1906.08043v1",
            "date": "2019-06-17 20:07:45+00:00",
            "title": "Real to H-space Encoder for Speech Recognition",
            "authors": [
                "Titouan Parcollet",
                "Mohamed Morchid",
                "Georges Linar\u00e8s",
                "Renato De Mori"
            ],
            "abstract": "Deep neural networks (DNNs) and more precisely recurrent neural networks\n(RNNs) are at the core of modern automatic speech recognition systems, due to\ntheir efficiency to process input sequences. Recently, it has been shown that\ndifferent input representations, based on multidimensional algebras, such as\ncomplex and quaternion numbers, are able to bring to neural networks a more\nnatural, compressive and powerful representation of the input signal by\noutperforming common real-valued NNs. Indeed, quaternion-valued neural networks\n(QNNs) better learn both internal dependencies, such as the relation between\nthe Mel-filter-bank value of a specific time frame and its time derivatives,\nand global dependencies, describing the relations that exist between time\nframes. Nonetheless, QNNs are limited to quaternion-valued input signals, and\nit is difficult to benefit from this powerful representation with real-valued\ninput data. This paper proposes to tackle this weakness by introducing a\nreal-to-quaternion encoder that allows QNNs to process any one dimensional\ninput features, such as traditional Mel-filter-banks for automatic speech\nrecognition.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.08430v1",
            "date": "2019-06-20 03:28:09+00:00",
            "title": "Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects",
            "authors": [
                "Gabriel Grand",
                "Yonatan Belinkov"
            ],
            "abstract": "Visual question answering (VQA) models have been shown to over-rely on\nlinguistic biases in VQA datasets, answering questions \"blindly\" without\nconsidering visual context. Adversarial regularization (AdvReg) aims to address\nthis issue via an adversary sub-network that encourages the main model to learn\na bias-free representation of the question. In this work, we investigate the\nstrengths and shortcomings of AdvReg with the goal of better understanding how\nit affects inference in VQA models. Despite achieving a new state-of-the-art on\nVQA-CP, we find that AdvReg yields several undesirable side-effects, including\nunstable gradients and sharply reduced performance on in-domain examples. We\ndemonstrate that gradual introduction of regularization during training helps\nto alleviate, but not completely solve, these issues. Through error analyses,\nwe observe that AdvReg improves generalization to binary questions, but impairs\nperformance on questions with heterogeneous answer distributions.\nQualitatively, we also find that regularized models tend to over-rely on visual\nfeatures, while ignoring important linguistic cues in the question. Our results\nsuggest that AdvReg requires further refinement before it can be considered a\nviable bias mitigation technique for VQA.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.08647v2",
            "date": "2019-06-20 14:11:55+00:00",
            "title": "Semi-supervised acoustic model training for five-lingual code-switched ASR",
            "authors": [
                "Astik Biswas",
                "Emre Y\u0131lmaz",
                "Febe de Wet",
                "Ewald van der Westhuizen",
                "Thomas Niesler"
            ],
            "abstract": "This paper presents recent progress in the acoustic modelling of\nunder-resourced code-switched (CS) speech in multiple South African languages.\nWe consider two approaches. The first constructs separate bilingual acoustic\nmodels corresponding to language pairs (English-isiZulu, English-isiXhosa,\nEnglish-Setswana and English-Sesotho). The second constructs a single unified\nfive-lingual acoustic model representing all the languages (English, isiZulu,\nisiXhosa, Setswana and Sesotho). For these two approaches we consider the\neffectiveness of semi-supervised training to increase the size of the very\nsparse acoustic training sets. Using approximately 11 hours of untranscribed\nspeech, we show that both approaches benefit from semi-supervised training. The\nbilingual TDNN-F acoustic models also benefit from the addition of CNN layers\n(CNN-TDNN-F), while the five-lingual system does not show any significant\nimprovement. Furthermore, because English is common to all language pairs in\nour data, it dominates when training a unified language model, leading to\nimproved English ASR performance at the expense of the other languages.\nNevertheless, the five-lingual model offers flexibility because it can process\nmore than two languages simultaneously, and is therefore an attractive option\nas an automatic transcription system in a semi-supervised training pipeline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08911v2",
            "date": "2019-06-21 01:47:32+00:00",
            "title": "Toward a Standard Interface for User-Defined Scheduling in OpenMP",
            "authors": [
                "Vivek Kale",
                "Christian Iwainsky",
                "Michael Klemm",
                "Jonas H. Muller Korndorfer",
                "Florina M. Ciorba"
            ],
            "abstract": "Parallel loops are an important part of OpenMP programs. Efficient scheduling\nof parallel loops can improve performance of the programs. The current OpenMP\nspecification only offers three options for loop scheduling, which are\ninsufficient in certain instances. Given the large number of other possible\nscheduling strategies, it is infeasible to standardize each one. A more viable\napproach is to extend the OpenMP standard to allow for users to define loop\nscheduling strategies. The approach will enable standard-compliant\napplication-specific scheduling. This work analyzes the principal components\nrequired by user-defined scheduling and proposes two competing interfaces as\ncandidates for the OpenMP standard. We conceptually compare the two proposed\ninterfaces with respect to the three host languages of OpenMP, i.e., C, C++,\nand Fortran. These interfaces serve the OpenMP community as a basis for\ndiscussion and prototype implementation for user-defined scheduling.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1906.08934v2",
            "date": "2019-06-21 03:39:46+00:00",
            "title": "Meta-learning of textual representations",
            "authors": [
                "Jorge Madrid",
                "Hugo Jair Escalante",
                "Eduardo Morales"
            ],
            "abstract": "Recent progress in AutoML has lead to state-of-the-art methods (e.g.,\nAutoSKLearn) that can be readily used by non-experts to approach any supervised\nlearning problem. Whereas these methods are quite effective, they are still\nlimited in the sense that they work for tabular (matrix formatted) data only.\nThis paper describes one step forward in trying to automate the design of\nsupervised learning methods in the context of text mining. We introduce a meta\nlearning methodology for automatically obtaining a representation for text\nmining tasks starting from raw text. We report experiments considering 60\ndifferent textual representations and more than 80 text mining datasets\nassociated to a wide variety of tasks. Experimental results show the proposed\nmethodology is a promising solution to obtain highly effective off the shell\ntext classification pipelines.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.08939v1",
            "date": "2019-06-21 04:14:07+00:00",
            "title": "Learning Bilingual Word Embeddings Using Lexical Definitions",
            "authors": [
                "Weijia Shi",
                "Muhao Chen",
                "Yingtao Tian",
                "Kai-Wei Chang"
            ],
            "abstract": "Bilingual word embeddings, which representlexicons of different languages in\na shared em-bedding space, are essential for supporting se-mantic and knowledge\ntransfers in a variety ofcross-lingual NLP tasks. Existing approachesto\ntraining bilingual word embeddings requireoften require pre-defined seed\nlexicons that areexpensive to obtain, or parallel sentences thatcomprise coarse\nand noisy alignment. In con-trast, we propose BilLex that leverages pub-licly\navailable lexical definitions for bilingualword embedding learning. Without the\nneedof predefined seed lexicons, BilLex comprisesa novel word pairing strategy\nto automati-cally identify and propagate the precise fine-grained word\nalignment from lexical defini-tions. We evaluate BilLex in word-level\nandsentence-level translation tasks, which seek tofind the cross-lingual\ncounterparts of wordsand sentences respectively.BilLex signifi-cantly\noutperforms previous embedding meth-ods on both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.08990v1",
            "date": "2019-06-21 07:49:36+00:00",
            "title": "A Comparative Survey of Recent Natural Language Interfaces for Databases",
            "authors": [
                "Katrin Affolter",
                "Kurt Stockinger",
                "Abraham Bernstein"
            ],
            "abstract": "Over the last few years natural language interfaces (NLI) for databases have\ngained significant traction both in academia and industry. These systems use\nvery different approaches as described in recent survey papers. However, these\nsystems have not been systematically compared against a set of benchmark\nquestions in order to rigorously evaluate their functionalities and expressive\npower.\n  In this paper, we give an overview over 24 recently developed NLIs for\ndatabases. Each of the systems is evaluated using a curated list of ten sample\nquestions to show their strengths and weaknesses. We categorize the NLIs into\nfour groups based on the methodology they are using: keyword-, pattern-,\nparsing-, and grammar-based NLI. Overall, we learned that keyword-based systems\nare enough to answer simple questions. To solve more complex questions\ninvolving subqueries, the system needs to apply some sort of parsing to\nidentify structural dependencies. Grammar-based systems are overall the most\npowerful ones, but are highly dependent on their manually designed rules. In\naddition to providing a systematic analysis of the major systems, we derive\nlessons learned that are vital for designing NLIs that can answer a wide range\nof user questions.",
            "journal ref": "VLDB Journal 2019",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1906.09292v3",
            "date": "2019-06-21 19:04:39+00:00",
            "title": "Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models",
            "authors": [
                "Ke Hu",
                "Antoine Bruguier",
                "Tara N. Sainath",
                "Rohit Prabhavalkar",
                "Golan Pundak"
            ],
            "abstract": "Contextual automatic speech recognition, i.e., biasing recognition towards a\ngiven context (e.g. user's playlists, or contacts), is challenging in\nend-to-end (E2E) models. Such models maintain a limited number of candidates\nduring beam-search decoding, and have been found to recognize rare named\nentities poorly. The problem is exacerbated when biasing towards proper nouns\nin foreign languages, e.g., geographic location names, which are virtually\nunseen in training and are thus out-of-vocabulary (OOV). While grapheme or\nwordpiece E2E models might have a difficult time spelling OOV words, phonemes\nare more acoustically salient and past work has shown that E2E phoneme models\ncan better predict such words. In this work, we propose an E2E model containing\nboth English wordpieces and phonemes in the modeling space, and perform\ncontextual biasing of foreign words at the phoneme level by mapping\npronunciations of foreign words into similar English phonemes. In experimental\nevaluations, we find that the proposed approach performs 16% better than a\ngrapheme-only biasing model, and 8% better than a wordpiece-only biasing model\non a foreign place name recognition task, with only slight degradation on\nregular English tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09308v2",
            "date": "2019-06-21 20:08:18+00:00",
            "title": "Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems",
            "authors": [
                "Asma Ghandeharioun",
                "Judy Hanwen Shen",
                "Natasha Jaques",
                "Craig Ferguson",
                "Noah Jones",
                "Agata Lapedriza",
                "Rosalind Picard"
            ],
            "abstract": "Building an open-domain conversational agent is a challenging problem.\nCurrent evaluation methods, mostly post-hoc judgments of static conversation,\ndo not capture conversation quality in a realistic interactive context. In this\npaper, we investigate interactive human evaluation and provide evidence for its\nnecessity; we then introduce a novel, model-agnostic, and dataset-agnostic\nmethod to approximate it. In particular, we propose a self-play scenario where\nthe dialog system talks to itself and we calculate a combination of proxies\nsuch as sentiment and semantic coherence on the conversation trajectory. We\nshow that this metric is capable of capturing the human-rated quality of a\ndialog model better than any automated metric known to-date, achieving a\nsignificant Pearson correlation (r>.7, p<.05). To investigate the strengths of\nthis novel metric and interactive evaluation in comparison to state-of-the-art\nmetrics and human evaluation of static conversations, we perform extended\nexperiments with a set of models, including several that make novel\nimprovements to recent hierarchical dialog generation architectures through\nsentiment and semantic knowledge distillation on the utterance level. Finally,\nwe open-source the interactive evaluation platform we built and the dataset we\ncollected to allow researchers to efficiently deploy and evaluate dialog\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09310v1",
            "date": "2019-06-21 20:16:52+00:00",
            "title": "A Study of State Aliasing in Structured Prediction with RNNs",
            "authors": [
                "Layla El Asri",
                "Adam Trischler"
            ],
            "abstract": "End-to-end reinforcement learning agents learn a state representation and a\npolicy at the same time. Recurrent neural networks (RNNs) have been trained\nsuccessfully as reinforcement learning agents in settings like dialogue that\nrequire structured prediction. In this paper, we investigate the\nrepresentations learned by RNN-based agents when trained with both policy\ngradient and value-based methods. We show through extensive experiments and\nanalysis that, when trained with policy gradient, recurrent neural networks\noften fail to learn a state representation that leads to an optimal policy in\nsettings where the same action should be taken at different states. To explain\nthis failure, we highlight the problem of state aliasing, which entails\nconflating two or more distinct states in the representation space. We\ndemonstrate that state aliasing occurs when several states share the same\noptimal action and the agent is trained via policy gradient. We characterize\nthis phenomenon through experiments on a simple maze setting and a more complex\ntext-based game, and make recommendations for training RNNs with reinforcement\nlearning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.09322v1",
            "date": "2019-06-15 09:09:12+00:00",
            "title": "A Syllable-Structured, Contextually-Based Conditionally Generation of Chinese Lyrics",
            "authors": [
                "Xu Lu",
                "Jie Wang",
                "Bojin Zhuang",
                "Shaojun Wang",
                "Jing Xiao"
            ],
            "abstract": "This paper presents a novel, syllable-structured Chinese lyrics generation\nmodel given a piece of original melody. Most previously reported lyrics\ngeneration models fail to include the relationship between lyrics and melody.\nIn this work, we propose to interpret lyrics-melody alignments as syllable\nstructural information and use a multi-channel sequence-to-sequence model with\nconsidering both phrasal structures and semantics. Two different RNN encoders\nare applied, one of which is for encoding syllable structures while the other\nfor semantic encoding with contextual sentences or input keywords. Moreover, a\nlarge Chinese lyrics corpus for model training is leveraged. With automatic and\nhuman evaluations, results demonstrate the effectiveness of our proposed lyrics\ngeneration model. To the best of our knowledge, there is few previous reports\non lyrics generation considering both music and linguistic perspectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09324v1",
            "date": "2019-06-15 09:20:41+00:00",
            "title": "Automatic Conditional Generation of Personalized Social Media Short Texts",
            "authors": [
                "Ziwen Wang",
                "Jie Wang",
                "Haiqian Gu",
                "Fei Su",
                "Bojin Zhuang"
            ],
            "abstract": "Automatic text generation has received much attention owing to rapid\ndevelopment of deep neural networks. In general, text generation systems based\non statistical language model will not consider anthropomorphic\ncharacteristics, which results in machine-like generated texts. To fill the\ngap, we propose a conditional language generation model with Big Five\nPersonality (BFP) feature vectors as input context, which writes human-like\nshort texts. The short text generator consists of a layer of long short memory\nnetwork (LSTM), where a BFP feature vector is concatenated as one part of input\nfor each cell. To enable supervised training generation model, a text\nclassification model based convolution neural network (CNN) has been used to\nprepare BFP-tagged Chinese micro-blog corpora. Validated by a BFP linguistic\ncomputational model, our generated Chinese short texts exhibit discriminative\npersonality styles, which are also syntactically correct and semantically\nsmooth with appropriate emoticons. With combination of natural language\ngeneration with psychological linguistics, our proposed BFP-dependent text\ngeneration model can be widely used for individualization in machine\ntranslation, image caption, dialogue generation and so on.",
            "journal ref": "In: Geng X., Kang BH. (eds) PRICAI 2018: Trends in Artificial\n  Intelligence",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09325v1",
            "date": "2019-06-17 13:11:26+00:00",
            "title": "Exploiting Unsupervised Pre-training and Automated Feature Engineering for Low-resource Hate Speech Detection in Polish",
            "authors": [
                "Renard Korzeniowski",
                "Rafa\u0142 Rolczy\u0144ski",
                "Przemys\u0142aw Sadownik",
                "Tomasz Korbak",
                "Marcin Mo\u017cejko"
            ],
            "abstract": "This paper presents our contribution to PolEval 2019 Task 6: Hate speech and\nbullying detection. We describe three parallel approaches that we followed:\nfine-tuning a pre-trained ULMFiT model to our classification task, fine-tuning\na pre-trained BERT model to our classification task, and using the TPOT library\nto find the optimal pipeline. We present results achieved by these three tools\nand review their advantages and disadvantages in terms of user experience. Our\nteam placed second in subtask 2 with a shallow model found by TPOT: a~logistic\nregression classifier with non-trivial feature engineering.",
            "journal ref": "Proceedings of the PolEval 2019 Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09384v1",
            "date": "2019-06-22 04:02:26+00:00",
            "title": "A Bandit Approach to Posterior Dialog Orchestration Under a Budget",
            "authors": [
                "Sohini Upadhyay",
                "Mayank Agarwal",
                "Djallel Bounneffouf",
                "Yasaman Khazaeni"
            ],
            "abstract": "Building multi-domain AI agents is a challenging task and an open problem in\nthe area of AI. Within the domain of dialog, the ability to orchestrate\nmultiple independently trained dialog agents, or skills, to create a unified\nsystem is of particular significance. In this work, we study the task of online\nposterior dialog orchestration, where we define posterior orchestration as the\ntask of selecting a subset of skills which most appropriately answer a user\ninput using features extracted from both the user input and the individual\nskills. To account for the various costs associated with extracting skill\nfeatures, we consider online posterior orchestration under a skill execution\nbudget. We formalize this setting as Context Attentive Bandit with Observations\n(CABO), a variant of context attentive bandits, and evaluate it on simulated\nnon-conversational and proprietary conversational datasets.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1906.09426v1",
            "date": "2019-06-22 10:23:42+00:00",
            "title": "End-to-End ASR for Code-switched Hindi-English Speech",
            "authors": [
                "Brij Mohan Lal Srivastava",
                "Basil Abraham",
                "Sunayana Sitaram",
                "Rupesh Mehta",
                "Preethi Jyothi"
            ],
            "abstract": "End-to-end (E2E) models have been explored for large speech corpora and have\nbeen found to match or outperform traditional pipeline-based systems in some\nlanguages. However, most prior work on end-to-end models use speech corpora\nexceeding hundreds or thousands of hours. In this study, we explore end-to-end\nmodels for code-switched Hindi-English language with less than 50 hours of\ndata. We utilize two specific measures to improve network performance in the\nlow-resource setting, namely multi-task learning (MTL) and balancing the corpus\nto deal with the inherent class imbalance problem i.e. the skewed frequency\ndistribution over graphemes. We compare the results of the proposed approaches\nwith traditional, cascaded ASR systems. While the lack of data adversely\naffects the performance of end-to-end models, we see promising improvements\nwith MTL and balancing the corpus.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.09444v1",
            "date": "2019-06-22 13:20:57+00:00",
            "title": "Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Chenze Shao",
                "Yang Feng",
                "Jinchao Zhang",
                "Fandong Meng",
                "Xilin Chen",
                "Jie Zhou"
            ],
            "abstract": "Non-Autoregressive Transformer (NAT) aims to accelerate the Transformer model\nthrough discarding the autoregressive mechanism and generating target words\nindependently, which fails to exploit the target sequential information.\nOver-translation and under-translation errors often occur for the above reason,\nespecially in the long sentence translation scenario. In this paper, we propose\ntwo approaches to retrieve the target sequential information for NAT to enhance\nits translation ability while preserving the fast-decoding property. Firstly,\nwe propose a sequence-level training method based on a novel reinforcement\nalgorithm for NAT (Reinforce-NAT) to reduce the variance and stabilize the\ntraining procedure. Secondly, we propose an innovative Transformer decoder\nnamed FS-decoder to fuse the target sequential information into the top layer\nof the decoder. Experimental results on three translation tasks show that the\nReinforce-NAT surpasses the baseline NAT system by a significant margin on BLEU\nwithout decelerating the decoding speed and the FS-decoder achieves comparable\ntranslation performance to the autoregressive Transformer with considerable\nspeedup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09569v1",
            "date": "2019-06-23 09:23:08+00:00",
            "title": "Systematic improvement of user engagement with academic titles using computational linguistics",
            "authors": [
                "Nim Dvir",
                "Ruti Gafni"
            ],
            "abstract": "This paper describes a novel approach to systematically improve information\ninteractions based solely on its wording. Following an interdisciplinary\nliterature review, we recognized three key attributes of words that drive user\nengagement: (1) Novelty (2) Familiarity (3) Emotionality. Based on these\nattributes, we developed a model to systematically improve a given content\nusing computational linguistics, natural language processing (NLP) and text\nanalysis (word frequency, sentiment analysis and lexical substitution). We\nconducted a pilot study (n=216) in which the model was used to formalize\nevaluation and optimization of academic titles. A between-group design (A/B\ntesting) was used to compare responses to the original and modified (treatment)\ntitles. Data was collected for selection and evaluation (User Engagement\nScale). The pilot results suggest that user engagement with digital information\nis fostered by, and perhaps dependent upon, the wording being used. They also\nprovide empirical support that engaging content can be systematically evaluated\nand produced. The preliminary results show that the modified (treatment) titles\nhad significantly higher scores for information use and user engagement\n(selection and evaluation). We propose that computational linguistics is a\nuseful approach for optimizing information interactions. The empirically based\ninsights can inform the development of digital content strategies, thereby\nimproving the success of information interactions.elop more sophisticated\ninteraction measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09601v1",
            "date": "2019-06-23 16:13:45+00:00",
            "title": "Sequence Generation: From Both Sides to the Middle",
            "authors": [
                "Long Zhou",
                "Jiajun Zhang",
                "Chengqing Zong",
                "Heng Yu"
            ],
            "abstract": "The encoder-decoder framework has achieved promising process for many\nsequence generation tasks, such as neural machine translation and text\nsummarization. Such a framework usually generates a sequence token by token\nfrom left to right, hence (1) this autoregressive decoding procedure is\ntime-consuming when the output sentence becomes longer, and (2) it lacks the\nguidance of future context which is crucial to avoid under translation. To\nalleviate these issues, we propose a synchronous bidirectional sequence\ngeneration (SBSG) model which predicts its outputs from both sides to the\nmiddle simultaneously. In the SBSG model, we enable the left-to-right (L2R) and\nright-to-left (R2L) generation to help and interact with each other by\nleveraging interactive bidirectional attention network. Experiments on neural\nmachine translation (En-De, Ch-En, and En-Ro) and text summarization tasks show\nthat the proposed model significantly speeds up decoding while improving the\ngeneration quality compared to the autoregressive Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09676v1",
            "date": "2019-06-24 00:30:32+00:00",
            "title": "CORAL8: Concurrent Object Regression for Area Localization in Medical Image Panels",
            "authors": [
                "Sam Maksoud",
                "Arnold Wiliem",
                "Kun Zhao",
                "Teng Zhang",
                "Lin Wu",
                "Brian C. Lovell"
            ],
            "abstract": "This work tackles the problem of generating a medical report for multi-image\npanels. We apply our solution to the Renal Direct Immunofluorescence (RDIF)\nassay which requires a pathologist to generate a report based on observations\nacross the eight different WSI in concert with existing clinical features. To\nthis end, we propose a novel attention-based multi-modal generative recurrent\nneural network (RNN) architecture capable of dynamically sampling image data\nconcurrently across the RDIF panel. The proposed methodology incorporates text\nfrom the clinical notes of the requesting physician to regulate the output of\nthe network to align with the overall clinical context. In addition, we found\nthe importance of regularizing the attention weights for word generation\nprocesses. This is because the system can ignore the attention mechanism by\nassigning equal weights for all members. Thus, we propose two regularizations\nwhich force the system to utilize the attention mechanism. Experiments on our\nnovel collection of RDIF WSIs provided by a large clinical laboratory\ndemonstrate that our framework offers significant improvements over existing\nmethods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.09825v1",
            "date": "2019-06-24 10:05:23+00:00",
            "title": "SylNet: An Adaptable End-to-End Syllable Count Estimator for Speech",
            "authors": [
                "Shreyas Seshadri",
                "Okko R\u00e4s\u00e4nen"
            ],
            "abstract": "Automatic syllable count estimation (SCE) is used in a variety of\napplications ranging from speaking rate estimation to detecting social activity\nfrom wearable microphones or developmental research concerned with quantifying\nspeech heard by language-learning children in different environments. The\nmajority of previously utilized SCE methods have relied on heuristic DSP\nmethods, and only a small number of bi-directional long short-term memory\n(BLSTM) approaches have made use of modern machine learning approaches in the\nSCE task. This paper presents a novel end-to-end method called SylNet for\nautomatic syllable counting from speech, built on the basis of a recent\ndevelopments in neural network architectures. We describe how the entire model\ncan be optimized directly to minimize SCE error on the training data without\nannotations aligned at the syllable level, and how it can be adapted to new\nlanguages using limited speech data with known syllable counts. Experiments on\nseveral different languages reveal that SylNet generalizes to languages beyond\nits training data and further improves with adaptation. It also outperforms\nseveral previously proposed methods for syllabification, including end-to-end\nBLSTMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.09832v1",
            "date": "2019-06-24 10:14:24+00:00",
            "title": "A computational model of early language acquisition from audiovisual experiences of young infants",
            "authors": [
                "Okko R\u00e4s\u00e4nen",
                "Khazar Khorrami"
            ],
            "abstract": "Earlier research has suggested that human infants might use statistical\ndependencies between speech and non-linguistic multimodal input to bootstrap\ntheir language learning before they know how to segment words from running\nspeech. However, feasibility of this hypothesis in terms of real-world infant\nexperiences has remained unclear. This paper presents a step towards a more\nrealistic test of the multimodal bootstrapping hypothesis by describing a\nneural network model that can learn word segments and their meanings from\nreferentially ambiguous acoustic input. The model is tested on recordings of\nreal infant-caregiver interactions using utterance-level labels for concrete\nvisual objects that were attended by the infant when caregiver spoke an\nutterance containing the name of the object, and using random visual labels for\nutterances during absence of attention. The results show that beginnings of\nlexical knowledge may indeed emerge from individually ambiguous learning\nscenarios. In addition, the hidden layers of the network show gradually\nincreasing selectivity to phonetic categories as a function of layer depth,\nresembling models trained for phone recognition in a supervised manner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10169v2",
            "date": "2019-06-24 18:55:24+00:00",
            "title": "RUBi: Reducing Unimodal Biases in Visual Question Answering",
            "authors": [
                "Remi Cadene",
                "Corentin Dancette",
                "Hedi Ben-younes",
                "Matthieu Cord",
                "Devi Parikh"
            ],
            "abstract": "Visual Question Answering (VQA) is the task of answering questions about an\nimage. Some VQA models often exploit unimodal biases to provide the correct\nanswer without using the image information. As a result, they suffer from a\nhuge drop in performance when evaluated on data outside their training set\ndistribution. This critical issue makes them unsuitable for real-world\nsettings.\n  We propose RUBi, a new learning strategy to reduce biases in any VQA model.\nIt reduces the importance of the most biased examples, i.e. examples that can\nbe correctly classified without looking at the image. It implicitly forces the\nVQA model to use the two input modalities instead of relying on statistical\nregularities between the question and the answer. We leverage a question-only\nmodel that captures the language biases by identifying when these unwanted\nregularities are used. It prevents the base VQA model from learning them by\ninfluencing its predictions. This leads to dynamically adjusting the loss in\norder to compensate for biases. We validate our contributions by surpassing the\ncurrent state-of-the-art results on VQA-CP v2. This dataset is specifically\ndesigned to assess the robustness of VQA models when exposed to different\nquestion biases at test time than what was seen during training.\n  Our code is available: github.com/cdancette/rubi.bootstrap.pytorch",
            "journal ref": "Advances in Neural Information Processing Systems 2019 (pp.\n  839-850)",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.10198v1",
            "date": "2019-06-24 19:47:23+00:00",
            "title": "Multimodal and Multi-view Models for Emotion Recognition",
            "authors": [
                "Gustavo Aguilar",
                "Viktor Rozgi\u0107",
                "Weiran Wang",
                "Chao Wang"
            ],
            "abstract": "Studies on emotion recognition (ER) show that combining lexical and acoustic\ninformation results in more robust and accurate models. The majority of the\nstudies focus on settings where both modalities are available in training and\nevaluation. However, in practice, this is not always the case; getting ASR\noutput may represent a bottleneck in a deployment pipeline due to computational\ncomplexity or privacy-related constraints. To address this challenge, we study\nthe problem of efficiently combining acoustic and lexical modalities during\ntraining while still providing a deployable acoustic model that does not\nrequire lexical inputs. We first experiment with multimodal models and two\nattention mechanisms to assess the extent of the benefits that lexical\ninformation can provide. Then, we frame the task as a multi-view learning\nproblem to induce semantic information from a multimodal model into our\nacoustic-only network using a contrastive loss function. Our multimodal model\noutperforms the previous state of the art on the USC-IEMOCAP dataset reported\non lexical and acoustic information. Additionally, our multi-view-trained\nacoustic network significantly surpasses models that have been exclusively\ntrained with acoustic features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10369v1",
            "date": "2019-06-25 08:11:20+00:00",
            "title": "Acoustic Modeling for Automatic Lyrics-to-Audio Alignment",
            "authors": [
                "Chitralekha Gupta",
                "Emre Y\u0131lmaz",
                "Haizhou Li"
            ],
            "abstract": "Automatic lyrics to polyphonic audio alignment is a challenging task not only\nbecause the vocals are corrupted by background music, but also there is a lack\nof annotated polyphonic corpus for effective acoustic modeling. In this work,\nwe propose (1) using additional speech and music-informed features and (2)\nadapting the acoustic models trained on a large amount of solo singing vocals\ntowards polyphonic music using a small amount of in-domain data. Incorporating\nadditional information such as voicing and auditory features together with\nconventional acoustic features aims to bring robustness against the increased\nspectro-temporal variations in singing vocals. By adapting the acoustic model\nusing a small amount of polyphonic audio data, we reduce the domain mismatch\nbetween training and testing data. We perform several alignment experiments and\npresent an in-depth alignment error analysis on acoustic features, and model\nadaptation techniques. The results demonstrate that the proposed strategy\nprovides a significant error reduction of word boundary alignment over\ncomparable existing systems, especially on more challenging polyphonic data\nwith long-duration musical interludes.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.10511v1",
            "date": "2019-06-17 18:47:28+00:00",
            "title": "Benchmarking Neural Machine Translation for Southern African Languages",
            "authors": [
                "Laura Martinus",
                "Jade Z. Abbott"
            ],
            "abstract": "Unlike major Western languages, most African languages are very\nlow-resourced. Furthermore, the resources that do exist are often scattered and\ndifficult to obtain and discover. As a result, the data and code for existing\nresearch has rarely been shared. This has lead a struggle to reproduce reported\nresults, and few publicly available benchmarks for African machine translation\nmodels exist. To start to address these problems, we trained neural machine\ntranslation models for 5 Southern African languages on publicly-available\ndatasets. Code is provided for training the models and evaluate the models on a\nnewly released evaluation set, with the aim of spur future research in the\nfield for Southern African languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10551v1",
            "date": "2019-06-24 09:44:46+00:00",
            "title": "Assessing the Applicability of Authorship Verification Methods",
            "authors": [
                "Oren Halvani",
                "Christian Winter",
                "Lukas Graner"
            ],
            "abstract": "Authorship verification (AV) is a research subject in the field of digital\ntext forensics that concerns itself with the question, whether two documents\nhave been written by the same person. During the past two decades, an\nincreasing number of proposed AV approaches can be observed. However, a closer\nlook at the respective studies reveals that the underlying characteristics of\nthese methods are rarely addressed, which raises doubts regarding their\napplicability in real forensic settings. The objective of this paper is to fill\nthis gap by proposing clear criteria and properties that aim to improve the\ncharacterization of existing and future AV approaches. Based on these\nproperties, we conduct three experiments using 12 existing AV approaches,\nincluding the current state of the art. The examined methods were trained,\noptimized and evaluated on three self-compiled corpora, where each corpus\nfocuses on a different aspect of applicability. Our results indicate that part\nof the methods are able to cope with very challenging verification cases such\nas 250 characters long informal chat conversations (72.7% accuracy) or cases in\nwhich two scientific documents were written at different times with an average\ndifference of 15.6 years (> 75% accuracy). However, we also identified that all\ninvolved methods are prone to cross-topic verification cases.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.10607v1",
            "date": "2019-06-25 15:31:14+00:00",
            "title": "Newswire versus Social Media for Disaster Response and Recovery",
            "authors": [
                "Rakesh Verma",
                "Samaneh Karimi",
                "Daniel Lee",
                "Omprakash Gnawali",
                "Azadeh Shakery"
            ],
            "abstract": "In a disaster situation, first responders need to quickly acquire situational\nawareness and prioritize response based on the need, resources available and\nimpact. Can they do this based on digital media such as Twitter alone, or\nnewswire alone, or some combination of the two? We examine this question in the\ncontext of the 2015 Nepal Earthquakes. Because newswire articles are longer,\neffective summaries can be helpful in saving time yet giving key content. We\nevaluate the effectiveness of several unsupervised summarization techniques in\ncapturing key content. We propose a method to link tweets written by the public\nand newswire articles, so that we can compare their key characteristics:\ntimeliness, whether tweets appear earlier than their corresponding news\narticles, and content. A novel idea is to view relevant tweets as a summary of\nthe matching news article and evaluate these summaries. Whenever possible, we\npresent both quantitative and qualitative evaluations. One of our main findings\nis that tweets and newswire articles provide complementary perspectives that\nform a holistic view of the disaster situation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1906.10827v2",
            "date": "2019-06-26 03:26:23+00:00",
            "title": "Hierarchical Optimal Transport for Document Representation",
            "authors": [
                "Mikhail Yurochkin",
                "Sebastian Claici",
                "Edward Chien",
                "Farzaneh Mirzazadeh",
                "Justin Solomon"
            ],
            "abstract": "The ability to measure similarity between documents enables intelligent\nsummarization and analysis of large corpora. Past distances between documents\nsuffer from either an inability to incorporate semantic similarities between\nwords or from scalability issues. As an alternative, we introduce hierarchical\noptimal transport as a meta-distance between documents, where documents are\nmodeled as distributions over topics, which themselves are modeled as\ndistributions over words. We then solve an optimal transport problem on the\nsmaller topic space to compute a similarity score. We give conditions on the\ntopics under which this construction defines a distance, and we relate it to\nthe word mover's distance. We evaluate our technique for k-NN classification\nand show better interpretability and scalability with comparable performance to\ncurrent methods at a fraction of the cost.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.10834v1",
            "date": "2019-06-26 03:58:29+00:00",
            "title": "Essence Knowledge Distillation for Speech Recognition",
            "authors": [
                "Zhenchuan Yang",
                "Chun Zhang",
                "Weibin Zhang",
                "Jianxiu Jin",
                "Dongpeng Chen"
            ],
            "abstract": "It is well known that a speech recognition system that combines multiple\nacoustic models trained on the same data significantly outperforms a\nsingle-model system. Unfortunately, real time speech recognition using a whole\nensemble of models is too computationally expensive. In this paper, we propose\nto distill the knowledge of essence in an ensemble of models (i.e. the teacher\nmodel) to a single model (i.e. the student model) that needs much less\ncomputation to deploy. Previously, all the soften outputs of the teacher model\nare used to optimize the student model. We argue that not all the outputs of\nthe ensemble are necessary to be distilled. Some of the outputs may even\ncontain noisy information that is useless or even harmful to the training of\nthe student model. In addition, we propose to train the student model with a\nmultitask learning approach by utilizing both the soften outputs of the teacher\nmodel and the correct hard labels. The proposed method achieves some surprising\nresults on the Switchboard data set. When the student model is trained together\nwith the correct labels and the essence knowledge from the teacher model, it\nnot only significantly outperforms another single model with the same\narchitecture that is trained only with the correct labels, but also\nconsistently outperforms the teacher model that is used to generate the soft\nlabels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10876v1",
            "date": "2019-06-26 07:09:57+00:00",
            "title": "Auxiliary Interference Speaker Loss for Target-Speaker Speech Recognition",
            "authors": [
                "Naoyuki Kanda",
                "Shota Horiguchi",
                "Ryoichi Takashima",
                "Yusuke Fujita",
                "Kenji Nagamatsu",
                "Shinji Watanabe"
            ],
            "abstract": "In this paper, we propose a novel auxiliary loss function for target-speaker\nautomatic speech recognition (ASR). Our method automatically extracts and\ntranscribes target speaker's utterances from a monaural mixture of multiple\nspeakers speech given a short sample of the target speaker. The proposed\nauxiliary loss function attempts to additionally maximize interference speaker\nASR accuracy during training. This will regularize the network to achieve a\nbetter representation for speaker separation, thus achieving better accuracy on\nthe target-speaker ASR. We evaluated our proposed method using\ntwo-speaker-mixed speech in various signal-to-interference-ratio conditions. We\nfirst built a strong target-speaker ASR baseline based on the state-of-the-art\nlattice-free maximum mutual information. This baseline achieved a word error\nrate (WER) of 18.06% on the test set while a normal ASR trained with clean data\nproduced a completely corrupted result (WER of 84.71%). Then, our proposed loss\nfurther reduced the WER by 6.6% relative to this strong baseline, achieving a\nWER of 16.87%. In addition to the accuracy improvement, we also showed that the\nauxiliary output branch for the proposed loss can even be used for a secondary\nASR for interference speakers' speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.10910v2",
            "date": "2019-06-26 08:37:44+00:00",
            "title": "Creating A Neural Pedagogical Agent by Jointly Learning to Review and Assess",
            "authors": [
                "Youngnam Lee",
                "Youngduck Choi",
                "Junghyun Cho",
                "Alexander R. Fabbri",
                "Hyunbin Loh",
                "Chanyou Hwang",
                "Yongku Lee",
                "Sang-Wook Kim",
                "Dragomir Radev"
            ],
            "abstract": "Machine learning plays an increasing role in intelligent tutoring systems as\nboth the amount of data available and specialization among students grow.\nNowadays, these systems are frequently deployed on mobile applications. Users\non such mobile education platforms are dynamic, frequently being added,\naccessing the application with varying levels of focus, and changing while\nusing the service. The education material itself, on the other hand, is often\nstatic and is an exhaustible resource whose use in tasks such as problem\nrecommendation must be optimized. The ability to update user models with\nrespect to educational material in real-time is thus essential; however,\nexisting approaches require time-consuming re-training of user features\nwhenever new data is added. In this paper, we introduce a neural pedagogical\nagent for real-time user modeling in the task of predicting user response\ncorrectness, a central task for mobile education applications. Our model,\ninspired by work in natural language processing on sequence modeling and\nmachine translation, updates user features in real-time via bidirectional\nrecurrent neural networks with an attention mechanism over embedded\nquestion-response pairs. We experiment on the mobile education application\nSantaTOEIC, which has 559k users, 66M response data points as well as a set of\n10k study problems each expert-annotated with topic tags and gathered since\n2016. Our model outperforms existing approaches over several metrics in\npredicting user response correctness, notably out-performing other methods on\nnew users without large question-response histories. Additionally, our\nattention mechanism and annotated tag set allow us to create an interpretable\neducation platform, with a smart review system that addresses the\naforementioned issue of varied user attention and problem exhaustion.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.10924v1",
            "date": "2019-06-26 09:10:33+00:00",
            "title": "Interpretable Question Answering on Knowledge Bases and Text",
            "authors": [
                "Alona Sydorova",
                "Nina Poerner",
                "Benjamin Roth"
            ],
            "abstract": "Interpretability of machine learning (ML) models becomes more relevant with\ntheir increasing adoption. In this work, we address the interpretability of ML\nbased question answering (QA) models on a combination of knowledge bases (KB)\nand text documents. We adapt post hoc explanation methods such as LIME and\ninput perturbation (IP) and compare them with the self-explanatory attention\nmechanism of the model. For this purpose, we propose an automatic evaluation\nparadigm for explanation methods in the context of QA. We also conduct a study\nwith human annotators to evaluate whether explanations help them identify\nbetter QA models. Our results suggest that IP provides better explanations than\nLIME or attention, according to both automatic and human evaluation. We obtain\nthe same ranking of methods in both experiments, which supports the validity of\nour automatic evaluation paradigm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11521v1",
            "date": "2019-06-27 09:47:47+00:00",
            "title": "Lattice-Based Unsupervised Test-Time Adaptation of Neural Network Acoustic Models",
            "authors": [
                "Ondrej Klejch",
                "Joachim Fainberg",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "Acoustic model adaptation to unseen test recordings aims to reduce the\nmismatch between training and testing conditions. Most adaptation schemes for\nneural network models require the use of an initial one-best transcription for\nthe test data, generated by an unadapted model, in order to estimate the\nadaptation transform. It has been found that adaptation methods using\ndiscriminative objective functions - such as cross-entropy loss - often require\ncareful regularisation to avoid over-fitting to errors in the one-best\ntranscriptions. In this paper we solve this problem by performing\ndiscriminative adaptation using lattices obtained from a first pass decoding,\nan approach that can be readily integrated into the lattice-free maximum mutual\ninformation (LF-MMI) framework. We investigate this approach on three\ntranscription tasks of varying difficulty: TED talks, multi-genre broadcast\n(MGB) and a low-resource language (Somali). We find that our proposed approach\nenables many more parameters to be adapted without over-fitting being observed,\nand is successful even when the initial transcription has a WER in excess of\n50%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11604v1",
            "date": "2019-06-27 13:10:37+00:00",
            "title": "Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion",
            "authors": [
                "Suyoun Kim",
                "Siddharth Dalmia",
                "Florian Metze"
            ],
            "abstract": "We present a novel conversational-context aware end-to-end speech recognizer\nbased on a gated neural network that incorporates\nconversational-context/word/speech embeddings. Unlike conventional speech\nrecognition models, our model learns longer conversational-context information\nthat spans across sentences and is consequently better at recognizing long\nconversations. Specifically, we propose to use the text-based external word\nand/or sentence embeddings (i.e., fastText, BERT) within an end-to-end\nframework, yielding a significant improvement in word error rate with better\nconversational-context representation. We evaluated the models on the\nSwitchboard conversational speech corpus and show that our model outperforms\nstandard end-to-end speech recognition models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.11790v1",
            "date": "2019-06-27 16:52:29+00:00",
            "title": "Encoding Database Schemas with Relation-Aware Self-Attention for Text-to-SQL Parsers",
            "authors": [
                "Richard Shin"
            ],
            "abstract": "When translating natural language questions into SQL queries to answer\nquestions from a database, we would like our methods to generalize to domains\nand database schemas outside of the training set. To handle complex questions\nand database schemas with a neural encoder-decoder paradigm, it is critical to\nproperly encode the schema as part of the input with the question. In this\npaper, we use relation-aware self-attention within the encoder so that it can\nreason about how the tables and columns in the provided schema relate to each\nother and use this information in interpreting the question. We achieve\nsignificant gains on the recently-released Spider dataset with 42.94% exact\nmatch accuracy, compared to the 18.96% reported in published work.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.11906v1",
            "date": "2019-06-06 21:54:50+00:00",
            "title": "Data Extraction from Charts via Single Deep Neural Network",
            "authors": [
                "Xiaoyi Liu",
                "Diego Klabjan",
                "Patrick NBless"
            ],
            "abstract": "Automatic data extraction from charts is challenging for two reasons: there\nexist many relations among objects in a chart, which is not a common\nconsideration in general computer vision problems; and different types of\ncharts may not be processed by the same model. To address these problems, we\npropose a framework of a single deep neural network, which consists of object\ndetection, text recognition and object matching modules. The framework handles\nboth bar and pie charts, and it may also be extended to other types of charts\nby slight revisions and by augmenting the training data. Our model performs\nsuccessfully on 79.4% of test simulated bar charts and 88.0% of test simulated\npie charts, while for charts outside of the training domain it degrades for\n57.5% and 62.3%, respectively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.12188v1",
            "date": "2019-06-26 13:51:59+00:00",
            "title": "A Deep Decoder Structure Based on WordEmbedding Regression for An Encoder-Decoder Based Model for Image Captioning",
            "authors": [
                "Ahmad Asadi",
                "Reza Safabakhsh"
            ],
            "abstract": "Generating textual descriptions for images has been an attractive problem for\nthe computer vision and natural language processing researchers in recent\nyears. Dozens of models based on deep learning have been proposed to solve this\nproblem. The existing approaches are based on neural encoder-decoder structures\nequipped with the attention mechanism. These methods strive to train decoders\nto minimize the log likelihood of the next word in a sentence given the\nprevious ones, which results in the sparsity of the output space. In this work,\nwe propose a new approach to train decoders to regress the word embedding of\nthe next word with respect to the previous ones instead of minimizing the log\nlikelihood. The proposed method is able to learn and extract long-term\ninformation and can generate longer fine-grained captions without introducing\nany external memory cell. Furthermore, decoders trained by the proposed\ntechnique can take the importance of the generated words into consideration\nwhile generating captions. In addition, a novel semantic attention mechanism is\nproposed that guides attention points through the image, taking the meaning of\nthe previously generated word into account. We evaluate the proposed approach\nwith the MS-COCO dataset. The proposed model outperformed the state of the art\nmodels especially in generating longer captions. It achieved a CIDEr score\nequal to 125.0 and a BLEU-4 score equal to 50.5, while the best scores of the\nstate of the art models are 117.1 and 48.0, respectively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.12230v1",
            "date": "2019-06-28 14:11:13+00:00",
            "title": "FIESTA: Fast IdEntification of State-of-The-Art models using adaptive bandit algorithms",
            "authors": [
                "Henry B. Moss",
                "Andrew Moore",
                "David S. Leslie",
                "Paul Rayson"
            ],
            "abstract": "We present FIESTA, a model selection approach that significantly reduces the\ncomputational resources required to reliably identify state-of-the-art\nperformance from large collections of candidate models. Despite being known to\nproduce unreliable comparisons, it is still common practice to compare model\nevaluations based on single choices of random seeds. We show that reliable\nmodel selection also requires evaluations based on multiple train-test splits\n(contrary to common practice in many shared tasks). Using bandit theory from\nthe statistics literature, we are able to adaptively determine appropriate\nnumbers of data splits and random seeds used to evaluate each model, focusing\ncomputational resources on the evaluation of promising models whilst avoiding\nwasting evaluations on models with lower performance. Furthermore, our\nuser-friendly Python implementation produces confidence guarantees of correctly\nselecting the optimal model. We evaluate our algorithms by selecting between 8\ntarget-dependent sentiment analysis methods using dramatically fewer model\nevaluations than current model selection approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.12330v1",
            "date": "2019-06-21 03:05:17+00:00",
            "title": "Graph Star Net for Generalized Multi-Task Learning",
            "authors": [
                "Lu Haonan",
                "Seth H. Huang",
                "Tian Ye",
                "Guo Xiuyan"
            ],
            "abstract": "In this work, we present graph star net (GraphStar), a novel and unified\ngraph neural net architecture which utilizes message-passing relay and\nattention mechanism for multiple prediction tasks - node classification, graph\nclassification and link prediction. GraphStar addresses many earlier challenges\nfacing graph neural nets and achieves non-local representation without\nincreasing the model depth or bearing heavy computational costs. We also\npropose a new method to tackle topic-specific sentiment analysis based on node\nclassification and text classification as graph classification. Our work shows\nthat 'star nodes' can learn effective graph-data representation and improve on\ncurrent methods for the three tasks. Specifically, for graph classification and\nlink prediction, GraphStar outperforms the current state-of-the-art models by\n2-5% on several key benchmarks.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.00112v1",
            "date": "2019-06-28 22:57:36+00:00",
            "title": "Leveraging Acoustic Cues and Paralinguistic Embeddings to Detect Expression from Voice",
            "authors": [
                "Vikramjit Mitra",
                "Sue Booker",
                "Erik Marchi",
                "David Scott Farrar",
                "Ute Dorothea Peitz",
                "Bridget Cheng",
                "Ermine Teves",
                "Anuj Mehta",
                "Devang Naik"
            ],
            "abstract": "Millions of people reach out to digital assistants such as Siri every day,\nasking for information, making phone calls, seeking assistance, and much more.\nThe expectation is that such assistants should understand the intent of the\nusers query. Detecting the intent of a query from a short, isolated utterance\nis a difficult task. Intent cannot always be obtained from speech-recognized\ntranscriptions. A transcription driven approach can interpret what has been\nsaid but fails to acknowledge how it has been said, and as a consequence, may\nignore the expression present in the voice. Our work investigates whether a\nsystem can reliably detect vocal expression in queries using acoustic and\nparalinguistic embedding. Results show that the proposed method offers a\nrelative equal error rate (EER) decrease of 60% compared to a bag-of-word based\nsystem, corroborating that expression is significantly represented by vocal\nattributes, rather than being purely lexical. Addition of emotion embedding\nhelped to reduce the EER by 30% relative to the acoustic embedding,\ndemonstrating the relevance of emotion in expressive voice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00157v1",
            "date": "2019-06-29 06:51:34+00:00",
            "title": "Progressive Fashion Attribute Extraction",
            "authors": [
                "Sandeep Singh Adhikari",
                "Sukhneer Singh",
                "Anoop Rajagopal",
                "Aruna Rajan"
            ],
            "abstract": "Extracting fashion attributes from images of people wearing clothing/fashion\naccessories is a very hard multi-class classification problem. Most often, even\ncatalogues of fashion do not have all the fine-grained attributes tagged due to\nprohibitive cost of annotation. Using images of fashion articles, running\nmulti-class attribute extraction with a single model for all kinds of\nattributes (neck design detailing, sleeves detailing, etc) requires classifiers\nthat are robust to missing and ambiguously labelled data. In this work, we\npropose a progressive training approach for such multi-class classification,\nwhere weights learnt from an attribute are fine tuned for another attribute of\nthe same fashion article (say, dresses). We branch networks for each attributes\nfrom a base network progressively during training. While it may have many\nlabels, an image doesn't need to have all possible labels for fashion articles\npresent in it. We also compare our approach to multi-label classification, and\ndemonstrate improvements over overall classification accuracies using our\napproach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.00181v1",
            "date": "2019-06-29 11:00:22+00:00",
            "title": "Fake News Detection using Stance Classification: A Survey",
            "authors": [
                "Anders Edelbo Lillie",
                "Emil Refsgaard Middelboe"
            ],
            "abstract": "This paper surveys and presents recent academic work carried out within the\nfield of stance classification and fake news detection. Echo chambers and the\nmodel organism problem are examples that pose challenges to acquire data with\nhigh quality, due to opinions being polarised in microblogs. Nevertheless it is\nshown that several machine learning approaches achieve promising results in\nclassifying stance. Some use crowd stance for fake news detection, such as the\napproach in [Dungs et al., 2018] using Hidden Markov Models. Furthermore\nfeature engineering have significant importance in several approaches, which is\nshown in [Aker et al., 2017]. This paper additionally includes a proposal of a\nsystem implementation based on the presented survey.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00390v1",
            "date": "2019-06-30 14:54:01+00:00",
            "title": "A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling",
            "authors": [
                "Haihong E",
                "Peiqing Niu",
                "Zhongfu Chen",
                "Meina Song"
            ],
            "abstract": "A spoken language understanding (SLU) system includes two main tasks, slot\nfilling (SF) and intent detection (ID). The joint model for the two tasks is\nbecoming a tendency in SLU. But the bi-directional interrelated connections\nbetween the intent and slots are not established in the existing joint models.\nIn this paper, we propose a novel bi-directional interrelated model for joint\nintent detection and slot filling. We introduce an SF-ID network to establish\ndirect connections for the two tasks to help them promote each other mutually.\nBesides, we design an entirely new iteration mechanism inside the SF-ID network\nto enhance the bi-directional interrelated connections. The experimental\nresults show that the relative improvement in the sentence-level semantic frame\naccuracy of our model is 3.79% and 5.42% on ATIS and Snips datasets,\nrespectively, compared to the state-of-the-art model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00455v1",
            "date": "2019-06-30 20:51:43+00:00",
            "title": "Multiplicative Models for Recurrent Language Modeling",
            "authors": [
                "Diego Maupom\u00e9",
                "Marie-Jean Meurs"
            ],
            "abstract": "Recently, there has been interest in multiplicative recurrent neural networks\nfor language modeling. Indeed, simple Recurrent Neural Networks (RNNs)\nencounter difficulties recovering from past mistakes when generating sequences\ndue to high correlation between hidden states. These challenges can be\nmitigated by integrating second-order terms in the hidden-state update. One\nsuch model, multiplicative Long Short-Term Memory (mLSTM) is particularly\ninteresting in its original formulation because of the sharing of its\nsecond-order term, referred to as the intermediate state. We explore these\narchitectural improvements by introducing new models and testing them on\ncharacter-level language modeling tasks. This allows us to establish the\nrelevance of shared parametrization in recurrent language modeling.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.00457v1",
            "date": "2019-06-30 20:54:21+00:00",
            "title": "Contextual Phonetic Pretraining for End-to-end Utterance-level Language and Speaker Recognition",
            "authors": [
                "Shaoshi Ling",
                "Julian Salazar",
                "Katrin Kirchhoff"
            ],
            "abstract": "Pretrained contextual word representations in NLP have greatly improved\nperformance on various downstream tasks. For speech, we propose contextual\nframe representations that capture phonetic information at the acoustic frame\nlevel and can be used for utterance-level language, speaker, and speech\nrecognition. These representations come from the frame-wise intermediate\nrepresentations of an end-to-end, self-attentive ASR model (SAN-CTC) on spoken\nutterances. We first train the model on the Fisher English corpus with\ncontext-independent phoneme labels, then use its representations at inference\ntime as features for task-specific models on the NIST LRE07 closed-set language\nrecognition task and a Fisher speaker recognition task, giving significant\nimprovements over the state-of-the-art on both (e.g., language EER of 4.68% on\n3sec utterances, 23% relative reduction in speaker EER). Results remain\ncompetitive when using a novel dilated convolutional model for language\nrecognition, or when ASR pretraining is done with character labels only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00477v2",
            "date": "2019-06-30 21:49:07+00:00",
            "title": "Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions",
            "authors": [
                "Tejas Srinivasan",
                "Ramon Sanabria",
                "Florian Metze"
            ],
            "abstract": "Multimodal learning allows us to leverage information from multiple sources\n(visual, acoustic and text), similar to our experience of the real world.\nHowever, it is currently unclear to what extent auxiliary modalities improve\nperformance over unimodal models, and under what circumstances the auxiliary\nmodalities are useful. We examine the utility of the auxiliary visual context\nin Multimodal Automatic Speech Recognition in adversarial settings, where we\ndeprive the models from partial audio signal during inference time. Our\nexperiments show that while MMASR models show significant gains over\ntraditional speech-to-text architectures (upto 4.2% WER improvements), they do\nnot incorporate visual information when the audio signal has been corrupted.\nThis shows that current methods of integrating the visual modality do not\nimprove model robustness to noise, and we need better visually grounded\nadaptation techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00488v1",
            "date": "2019-06-30 22:40:37+00:00",
            "title": "Topic Modeling the Reading and Writing Behavior of Information Foragers",
            "authors": [
                "Jaimie Murdock"
            ],
            "abstract": "The general problem of \"information foraging\" in an environment about which\nagents have incomplete information has been explored in many fields, including\ncognitive psychology, neuroscience, economics, finance, ecology, and computer\nscience. In all of these areas, the searcher aims to enhance future performance\nby surveying enough of existing knowledge to orient themselves in the\ninformation space. Individuals can be viewed as conducting a cognitive search\nin which they must balance exploration of ideas that are novel to them against\nexploitation of knowledge in domains in which they are already expert.\n  In this dissertation, I present several case studies that demonstrate how\nreading and writing behaviors interact to construct personal knowledge bases.\nThese studies use LDA topic modeling to represent the information environment\nof the texts each author read and wrote. Three studies revolve around Charles\nDarwin. Darwin left detailed records of every book he read for 23 years, from\ndisembarking from the H.M.S. Beagle to just after publication of The Origin of\nSpecies. Additionally, he left copies of his drafts before publication. I\ncharacterize his reading behavior, then show how that reading behavior\ninteracted with the drafts and subsequent revisions of The Origin of Species,\nand expand the dataset to include later readings and writings. Then, through a\nstudy of Thomas Jefferson's correspondence, I expand the study to non-book\ndata. Finally, through an examination of neuroscience citation data, I move\nfrom individual behavior to collective behavior in constructing an information\nenvironment. Together, these studies reveal \"the interplay between individual\nand collective phenomena where innovation takes place\" (Tria et al. 2014).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00510v1",
            "date": "2019-07-01 01:48:49+00:00",
            "title": "Hidden in Plain Sight For Too Long: Using Text Mining Techniques to Shine a Light on Workplace Sexism and Sexual Harassment",
            "authors": [
                "Amir Karami",
                "Suzanne C. Swan",
                "Cynthia Nicole White",
                "Kayla Ford"
            ],
            "abstract": "Objective: The goal of this study is to understand how people experience\nsexism and sexual harassment in the workplace by discovering themes in 2,362\nexperiences posted on the Everyday Sexism Project's website everydaysexism.com.\nMethod: This study used both quantitative and qualitative methods. The\nquantitative method was a computational framework to collect and analyze a\nlarge number of workplace sexual harassment experiences. The qualitative method\nwas the analysis of the topics generated by a text mining method. Results:\nTwenty-three topics were coded and then grouped into three overarching themes\nfrom the sex discrimination and sexual harassment literature. The Sex\nDiscrimination theme included experiences in which women were treated\nunfavorably due to their sex, such as being passed over for promotion, denied\nopportunities, paid less than men, and ignored or talked over in meetings. The\nSex Discrimination and Gender harassment theme included stories about sex\ndiscrimination and gender harassment, such as sexist hostility behaviors\nranging from insults and jokes invoking misogynistic stereotypes to bullying\nbehavior. The last theme, Unwanted Sexual Attention, contained stories\ndescribing sexual comments and behaviors used to degrade women. Unwanted\ntouching was the highest weighted topic, indicating how common it was for\nwebsite users to endure being touched, hugged or kissed, groped, and grabbed.\nConclusions: This study illustrates how researchers can use automatic processes\nto go beyond the limits of traditional research methods and investigate\nnaturally occurring large scale datasets on the internet to achieve a better\nunderstanding of everyday workplace sexism experiences.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1907.00570v2",
            "date": "2019-07-01 06:46:43+00:00",
            "title": "Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?",
            "authors": [
                "Joris Baan",
                "Maartje ter Hoeve",
                "Marlies van der Wees",
                "Anne Schuth",
                "Maarten de Rijke"
            ],
            "abstract": "Learning algorithms become more powerful, often at the cost of increased\ncomplexity. In response, the demand for algorithms to be transparent is\ngrowing. In NLP tasks, attention distributions learned by attention-based deep\nlearning models are used to gain insights in the models' behavior. To which\nextent is this perspective valid for all NLP tasks? We investigate whether\ndistributions calculated by different attention heads in a transformer\narchitecture can be used to improve transparency in the task of abstractive\nsummarization. To this end, we present both a qualitative and quantitative\nanalysis to investigate the behavior of the attention heads. We show that some\nattention heads indeed specialize towards syntactically and semantically\ndistinct input. We propose an approach to evaluate to which extent the\nTransformer model relies on specifically learned attention distributions. We\nalso discuss what this implies for using attention distributions as a means of\ntransparency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00708v2",
            "date": "2019-06-24 08:13:45+00:00",
            "title": "EQuANt (Enhanced Question Answer Network)",
            "authors": [
                "Fran\u00e7ois-Xavier Aubet",
                "Dominic Danks",
                "Yuchen Zhu"
            ],
            "abstract": "Machine Reading Comprehension (MRC) is an important topic in the domain of\nautomated question answering and in natural language processing more generally.\nSince the release of the SQuAD 1.1 and SQuAD 2 datasets, progress in the field\nhas been particularly significant, with current state-of-the-art models now\nexhibiting near-human performance at both answering well-posed questions and\ndetecting questions which are unanswerable given a corresponding context. In\nthis work, we present Enhanced Question Answer Network (EQuANt), an MRC model\nwhich extends the successful QANet architecture of Yu et al. to cope with\nunanswerable questions. By training and evaluating EQuANt on SQuAD 2, we show\nthat it is indeed possible to extend QANet to the unanswerable domain. We\nachieve results which are close to 2 times better than our chosen baseline\nobtained by evaluating a lightweight version of the original QANet architecture\non SQuAD 2. In addition, we report that the performance of EQuANt on SQuAD 1.1\nafter being trained on SQuAD2 exceeds that of our lightweight QANet\narchitecture trained and evaluated on SQuAD 1.1, demonstrating the utility of\nmulti-task learning in the MRC context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00710v1",
            "date": "2019-06-25 04:39:26+00:00",
            "title": "Deep Conversational Recommender in Travel",
            "authors": [
                "Lizi Liao",
                "Ryuichi Takanobu",
                "Yunshan Ma",
                "Xun Yang",
                "Minlie Huang",
                "Tat-Seng Chua"
            ],
            "abstract": "When traveling to a foreign country, we are often in dire need of an\nintelligent conversational agent to provide instant and informative responses\nto our various queries. However, to build such a travel agent is non-trivial.\nFirst of all, travel naturally involves several sub-tasks such as hotel\nreservation, restaurant recommendation and taxi booking etc, which invokes the\nneed for global topic control. Secondly, the agent should consider various\nconstraints like price or distance given by the user to recommend an\nappropriate venue. In this paper, we present a Deep Conversational Recommender\n(DCR) and apply to travel. It augments the sequence-to-sequence (seq2seq)\nmodels with a neural latent topic component to better guide response generation\nand make the training easier. To consider the various constraints for venue\nrecommendation, we leverage a graph convolutional network (GCN) based approach\nto capture the relationships between different venues and the match between\nvenue and dialog context. For response generation, we combine the topic-based\ncomponent with the idea of pointer networks, which allows us to effectively\nincorporate recommendation results. We perform extensive evaluation on a\nmulti-turn task-oriented dialog dataset in travel domain and the results show\nthat our method achieves superior performance as compared to a wide range of\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00818v2",
            "date": "2019-07-01 14:25:51+00:00",
            "title": "Ultrasound tongue imaging for diarization and alignment of child speech therapy sessions",
            "authors": [
                "Manuel Sam Ribeiro",
                "Aciel Eshky",
                "Korin Richmond",
                "Steve Renals"
            ],
            "abstract": "We investigate the automatic processing of child speech therapy sessions\nusing ultrasound visual biofeedback, with a specific focus on complementing\nacoustic features with ultrasound images of the tongue for the tasks of speaker\ndiarization and time-alignment of target words. For speaker diarization, we\npropose an ultrasound-based time-domain signal which we call estimated tongue\nactivity. For word-alignment, we augment an acoustic model with low-dimensional\nrepresentations of ultrasound images of the tongue, learned by a convolutional\nneural network. We conduct our experiments using the Ultrasuite repository of\nultrasound and speech recordings for child speech therapy sessions. For both\ntasks, we observe that systems augmented with ultrasound data outperform\ncorresponding systems using only the audio signal.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.00820v1",
            "date": "2019-07-01 14:27:03+00:00",
            "title": "Understanding Memory Modules on Learning Simple Algorithms",
            "authors": [
                "Kexin Wang",
                "Yu Zhou",
                "Shaonan Wang",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Recent work has shown that memory modules are crucial for the generalization\nability of neural networks on learning simple algorithms. However, we still\nhave little understanding of the working mechanism of memory modules. To\nalleviate this problem, we apply a two-step analysis pipeline consisting of\nfirst inferring hypothesis about what strategy the model has learned according\nto visualization and then verify it by a novel proposed qualitative analysis\nmethod based on dimension reduction. Using this method, we have analyzed two\npopular memory-augmented neural networks, neural Turing machine and\nstack-augmented neural network on two simple algorithm tasks including\nreversing a random sequence and evaluation of arithmetic expressions. Results\nhave shown that on the former task both models can learn to generalize and on\nthe latter task only the stack-augmented model can do so. We show that\ndifferent strategies are learned by the models, in which specific categories of\ninput are monitored and different policies are made based on that to change the\nmemory.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.00854v2",
            "date": "2019-07-01 15:20:10+00:00",
            "title": "Katecheo: A Portable and Modular System for Multi-Topic Question Answering",
            "authors": [
                "Shirish Hirekodi",
                "Seban Sunny",
                "Leonard Topno",
                "Alwin Daniel",
                "Daniel Whitenack",
                "Reuben Skewes",
                "Stuart Cranney"
            ],
            "abstract": "We introduce a modular system that can be deployed on any Kubernetes cluster\nfor question answering via REST API. This system, called Katecheo, includes\nthree configurable modules that collectively enable identification of\nquestions, classification of those questions into topics, document search, and\nreading comprehension. We demonstrate the system using publicly available\nknowledge base articles extracted from Stack Exchange sites. However, users can\nextend the system to any number of topics, or domains, without the need to\nmodify any of the model serving code or train their own models. All components\nof the system are open source and available under a permissive Apache 2\nLicense.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01011v1",
            "date": "2019-07-01 18:40:52+00:00",
            "title": "Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization",
            "authors": [
                "Paul Pu Liang",
                "Zhun Liu",
                "Yao-Hung Hubert Tsai",
                "Qibin Zhao",
                "Ruslan Salakhutdinov",
                "Louis-Philippe Morency"
            ],
            "abstract": "There has been an increased interest in multimodal language processing\nincluding multimodal dialog, question answering, sentiment analysis, and speech\nrecognition. However, naturally occurring multimodal data is often imperfect as\na result of imperfect modalities, missing entries or noise corruption. To\naddress these concerns, we present a regularization method based on tensor rank\nminimization. Our method is based on the observation that high-dimensional\nmultimodal time series data often exhibit correlations across time and\nmodalities which leads to low-rank tensor representations. However, the\npresence of noise or incomplete values breaks these correlations and results in\ntensor representations of higher rank. We design a model to learn such tensor\nrepresentations and effectively regularize their rank. Experiments on\nmultimodal language data show that our model achieves good results across\nvarious levels of imperfection.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.01098v1",
            "date": "2019-07-01 23:20:45+00:00",
            "title": "Representation, Exploration and Recommendation of Music Playlists",
            "authors": [
                "Piyush Papreja",
                "Hemanth Venkateswara",
                "Sethuraman Panchanathan"
            ],
            "abstract": "Playlists have become a significant part of our listening experience because\nof the digital cloud-based services such as Spotify, Pandora, Apple Music.\nOwing to the meteoric rise in the usage of playlists, recommending playlists is\ncrucial to music services today. Although there has been a lot of work done in\nplaylist prediction, the area of playlist representation hasn't received that\nlevel of attention. Over the last few years, sequence-to-sequence models,\nespecially in the field of natural language processing, have shown the\neffectiveness of learned embeddings in capturing the semantic characteristics\nof sequences. We can apply similar concepts to music to learn fixed length\nrepresentations for playlists and use those representations for downstream\ntasks such as playlist discovery, browsing, and recommendation. In this work,\nwe formulate the problem of learning a fixed-length playlist representation in\nan unsupervised manner, using Sequence-to-sequence (Seq2seq) models,\ninterpreting playlists as sentences and songs as words. We compare our model\nwith two other encoding architectures for baseline comparison. We evaluate our\nwork using the suite of tasks commonly used for assessing sentence embeddings,\nalong with a few additional tasks pertaining to music, and a recommendation\ntask to study the traits captured by the playlist embeddings and their\neffectiveness for the purpose of music recommendation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01356v2",
            "date": "2019-07-02 13:35:37+00:00",
            "title": "Predicting Retrosynthetic Reaction using Self-Corrected Transformer Neural Networks",
            "authors": [
                "Shuangjia Zheng",
                "Jiahua Rao",
                "Zhongyue Zhang",
                "Jun Xu",
                "Yuedong Yang"
            ],
            "abstract": "Synthesis planning is the process of recursively decomposing target molecules\ninto available precursors. Computer-aided retrosynthesis can potentially assist\nchemists in designing synthetic routes, but at present it is cumbersome and\nprovides results of dissatisfactory quality. In this study, we develop a\ntemplate-free self-corrected retrosynthesis predictor (SCROP) to perform a\nretrosynthesis prediction task trained by using the Transformer neural network\narchitecture. In the method, the retrosynthesis planning is converted as a\nmachine translation problem between molecular linear notations of reactants and\nthe products. Coupled with a neural network-based syntax corrector, our method\nachieves an accuracy of 59.0% on a standard benchmark dataset, which increases\n>21% over other deep learning methods, and >6% over template-based methods.\nMore importantly, our method shows an accuracy 1.7 times higher than other\nstate-of-the-art methods for compounds not appearing in the training set.",
            "journal ref": "None",
            "category": "physics.chem-ph"
        },
        {
            "url": "http://arxiv.org/abs/1907.01369v2",
            "date": "2019-06-26 17:07:03+00:00",
            "title": "Analyzing Verbal and Nonverbal Features for Predicting Group Performance",
            "authors": [
                "Uliyana Kubasova",
                "Gabriel Murray",
                "McKenzie Braley"
            ],
            "abstract": "This work analyzes the efficacy of verbal and nonverbal features of group\nconversation for the task of automatic prediction of group task performance. We\ndescribe a new publicly available survival task dataset that was collected and\nannotated to facilitate this prediction task. In these experiments, the new\ndataset is merged with an existing survival task dataset, allowing us to\ncompare feature sets on a much larger amount of data than has been used in\nrecent related work. This work is also distinct from related research on social\nsignal processing (SSP) in that we compare verbal and nonverbal features,\nwhereas SSP is almost exclusively concerned with nonverbal aspects of social\ninteraction. A key finding is that nonverbal features from the speech signal\nare extremely effective for this task, even on their own. However, the most\neffective individual features are verbal features, and we highlight the most\nimportant ones.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.01470v1",
            "date": "2019-07-02 15:56:20+00:00",
            "title": "Augmenting Self-attention with Persistent Memory",
            "authors": [
                "Sainbayar Sukhbaatar",
                "Edouard Grave",
                "Guillaume Lample",
                "Herve Jegou",
                "Armand Joulin"
            ],
            "abstract": "Transformer networks have lead to important progress in language modeling and\nmachine translation. These models include two consecutive modules, a\nfeed-forward layer and a self-attention layer. The latter allows the network to\ncapture long term dependencies and are often regarded as the key ingredient in\nthe success of Transformers. Building upon this intuition, we propose a new\nmodel that solely consists of attention layers. More precisely, we augment the\nself-attention layers with persistent memory vectors that play a similar role\nas the feed-forward layer. Thanks to these vectors, we can remove the\nfeed-forward layer without degrading the performance of a transformer. Our\nevaluation shows the benefits brought by our model on standard character and\nword level language modeling benchmarks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.01549v2",
            "date": "2019-07-01 18:30:38+00:00",
            "title": "Learning to Rank Broad and Narrow Queries in E-Commerce",
            "authors": [
                "Siddhartha Devapujula",
                "Sagar Arora",
                "Sumit Borar"
            ],
            "abstract": "Search is a prominent channel for discovering products on an e-commerce\nplatform. Ranking products retrieved from search becomes crucial to address\ncustomer's need and optimize for business metrics. While learning to Rank\n(LETOR) models have been extensively studied and have demonstrated efficacy in\nthe context of web search; it is a relatively new research area to be explored\nin the e-commerce. In this paper, we present a framework for building LETOR\nmodel for an e-commerce platform. We analyze user queries and propose a\nmechanism to segment queries between broad and narrow based on user's intent.\nWe discuss different types of features - query, product and query-product and\ndiscuss challenges in using them. We show that sparsity in product features can\nbe tackled through a denoising auto-encoder while skip-gram based word\nembeddings help solve the query-product sparsity issues. We also present\nvarious target metrics that can be employed for evaluating search results and\ncompare their robustness. Further, we build and compare performances of both\npointwise and pairwise LETOR models on fashion category data set. We also build\nand compare distinct models for broad and narrow queries, analyze feature\nimportance across these and show that these specialized models perform better\nthan a combined model in the fashion world.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01636v1",
            "date": "2019-06-17 06:59:25+00:00",
            "title": "Analyses of Multi-collection Corpora via Compound Topic Modeling",
            "authors": [
                "Clint P. George",
                "Wei Xia",
                "George Michailidis"
            ],
            "abstract": "As electronically stored data grow in daily life, obtaining novel and\nrelevant information becomes challenging in text mining. Thus people have\nsought statistical methods based on term frequency, matrix algebra, or topic\nmodeling for text mining. Popular topic models have centered on one single text\ncollection, which is deficient for comparative text analyses. We consider a\nsetting where one can partition the corpus into subcollections. Each\nsubcollection shares a common set of topics, but there exists relative\nvariation in topic proportions among collections. Including any prior knowledge\nabout the corpus (e.g. organization structure), we propose the compound latent\nDirichlet allocation (cLDA) model, improving on previous work, encouraging\ngeneralizability, and depending less on user-input parameters. To identify the\nparameters of interest in cLDA, we study Markov chain Monte Carlo (MCMC) and\nvariational inference approaches extensively, and suggest an efficient MCMC\nmethod. We evaluate cLDA qualitatively and quantitatively using both synthetic\nand real-world corpora. The usability study on some real-world corpora\nillustrates the superiority of cLDA to explore the underlying topics\nautomatically but also model their connections and variations across multiple\ncollections.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01642v1",
            "date": "2019-06-28 08:27:53+00:00",
            "title": "Introducing MathQA -- A Math-Aware Question Answering System",
            "authors": [
                "Moritz Schubotz",
                "Philipp Scharpf",
                "Kaushal Dudhat",
                "Yash Nagar",
                "Felix Hamborg",
                "Bela Gipp"
            ],
            "abstract": "We present an open source math-aware Question Answering System based on Ask\nPlatypus. Our system returns as a single mathematical formula for a natural\nlanguage question in English or Hindi. This formulae originate from the\nknowledge-base Wikidata. We translate these formulae to computable data by\nintegrating the calculation engine sympy into our system. This way, users can\nenter numeric values for the variables occurring in the formula. Moreover, the\nsystem loads numeric values for constants occurring in the formula from\nWikidata. In a user study, our system outperformed a commercial computational\nmathematical knowledge engine by 13%. However, the performance of our system\nheavily depends on the size and quality of the formula data available in\nWikidata. Since only a few items in Wikidata contained formulae when we started\nthe project, we facilitated the import process by suggesting formula edits to\nWikidata editors. With the simple heuristic that the first formula is\nsignificant for the article, 80% of the suggestions were correct.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01643v1",
            "date": "2019-07-01 17:48:40+00:00",
            "title": "Pentagon at MEDIQA 2019: Multi-task Learning for Filtering and Re-ranking Answers using Language Inference and Question Entailment",
            "authors": [
                "Hemant Pugaliya",
                "Karan Saxena",
                "Shefali Garg",
                "Sheetal Shalini",
                "Prashant Gupta",
                "Eric Nyberg",
                "Teruko Mitamura"
            ],
            "abstract": "Parallel deep learning architectures like fine-tuned BERT and MT-DNN, have\nquickly become the state of the art, bypassing previous deep and shallow\nlearning methods by a large margin. More recently, pre-trained models from\nlarge related datasets have been able to perform well on many downstream tasks\nby just fine-tuning on domain-specific datasets . However, using powerful\nmodels on non-trivial tasks, such as ranking and large document classification,\nstill remains a challenge due to input size limitations of parallel\narchitecture and extremely small datasets (insufficient for fine-tuning). In\nthis work, we introduce an end-to-end system, trained in a multi-task setting,\nto filter and re-rank answers in the medical domain. We use task-specific\npre-trained models as deep feature extractors. Our model achieves the highest\nSpearman's Rho and Mean Reciprocal Rank of 0.338 and 0.9622 respectively, on\nthe ACL-BioNLP workshop MediQA Question Answering shared-task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.01749v1",
            "date": "2019-07-03 05:50:34+00:00",
            "title": "Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-level Embedding Features",
            "authors": [
                "Zexin Cai",
                "Yaogen Yang",
                "Chuxiong Zhang",
                "Xiaoyi Qin",
                "Ming Li"
            ],
            "abstract": "This paper describes a conditional neural network architecture for Mandarin\nChinese polyphone disambiguation. The system is composed of a bidirectional\nrecurrent neural network component acting as a sentence encoder to accumulate\nthe context correlations, followed by a prediction network that maps the\npolyphonic character embeddings along with the conditions to corresponding\npronunciations. We obtain the word-level condition from a pre-trained\nword-to-vector lookup table. One goal of polyphone disambiguation is to address\nthe homograph problem existing in the front-end processing of Mandarin Chinese\ntext-to-speech system. Our system achieves an accuracy of 94.69\\% on a publicly\navailable polyphonic character dataset. To further validate our choices on the\nconditional feature, we investigate polyphone disambiguation systems with\nmulti-level conditions respectively. The experimental results show that both\nthe sentence-level and the word-level conditional embedding features are able\nto attain good performance for Mandarin Chinese polyphone disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01752v4",
            "date": "2019-07-03 06:15:14+00:00",
            "title": "On the Weaknesses of Reinforcement Learning for Neural Machine Translation",
            "authors": [
                "Leshem Choshen",
                "Lior Fox",
                "Zohar Aizenbud",
                "Omri Abend"
            ],
            "abstract": "Reinforcement learning (RL) is frequently used to increase performance in\ntext generation tasks, including machine translation (MT), notably through the\nuse of Minimum Risk Training (MRT) and Generative Adversarial Networks (GAN).\nHowever, little is known about what and how these methods learn in the context\nof MT. We prove that one of the most common RL methods for MT does not optimize\nthe expected reward, as well as show that other methods take an infeasibly long\ntime to converge. In fact, our results suggest that RL practices in MT are\nlikely to improve performance only where the pre-trained parameters are already\nclose to yielding the correct translation. Our findings further suggest that\nobserved gains may be due to effects unrelated to the training signal, but\nrather from changes in the shape of the distribution curve.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01791v1",
            "date": "2019-07-03 08:39:14+00:00",
            "title": "Multi-Task Networks With Universe, Group, and Task Feature Learning",
            "authors": [
                "Shiva Pentyala",
                "Mengwen Liu",
                "Markus Dreyer"
            ],
            "abstract": "We present methods for multi-task learning that take advantage of natural\ngroupings of related tasks. Task groups may be defined along known properties\nof the tasks, such as task domain or language. Such task groups represent\nsupervised information at the inter-task level and can be encoded into the\nmodel. We investigate two variants of neural network architectures that\naccomplish this, learning different feature spaces at the levels of individual\ntasks, task groups, as well as the universe of all tasks: (1) parallel\narchitectures encode each input simultaneously into feature spaces at different\nlevels; (2) serial architectures encode each input successively into feature\nspaces at different levels in the task hierarchy. We demonstrate the methods on\nnatural language understanding (NLU) tasks, where a grouping of tasks into\ndifferent task domains leads to improved performance on ATIS, Snips, and a\nlarge inhouse dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01957v2",
            "date": "2019-07-03 14:14:27+00:00",
            "title": "End-to-End Speech Recognition with High-Frame-Rate Features Extraction",
            "authors": [
                "Cong-Thanh Do"
            ],
            "abstract": "State-of-the-art end-to-end automatic speech recognition (ASR) extracts\nacoustic features from input speech signal every 10 ms which corresponds to a\nframe rate of 100 frames/second. In this report, we investigate the use of\nhigh-frame-rate features extraction in end-to-end ASR. High frame rates of 200\nand 400 frames/second are used in the features extraction and provide\nadditional information for end-to-end ASR. The effectiveness of high-frame-rate\nfeatures extraction is evaluated independently and in combination with speed\nperturbation based data augmentation. Experiments performed on two speech\ncorpora, Wall Street Journal (WSJ) and CHiME-5, show that using high-frame-rate\nfeatures extraction yields improved performance for end-to-end ASR, both\nindependently and in combination with speed perturbation. On WSJ corpus, the\nrelative reduction of word error rate (WER) yielded by high-frame-rate features\nextraction independently and in combination with speed perturbation are up to\n21.3% and 24.1%, respectively. On CHiME-5 corpus, the corresponding relative\nWER reductions are up to 2.8% and 7.9%, respectively, on the test data recorded\nby microphone arrays and up to 11.8% and 21.2%, respectively, on the test data\nrecorded by binaural microphones.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.02022v2",
            "date": "2019-07-03 16:39:05+00:00",
            "title": "Chasing Ghosts: Instruction Following as Bayesian State Tracking",
            "authors": [
                "Peter Anderson",
                "Ayush Shrivastava",
                "Devi Parikh",
                "Dhruv Batra",
                "Stefan Lee"
            ],
            "abstract": "A visually-grounded navigation instruction can be interpreted as a sequence\nof expected observations and actions an agent following the correct trajectory\nwould encounter and perform. Based on this intuition, we formulate the problem\nof finding the goal location in Vision-and-Language Navigation (VLN) within the\nframework of Bayesian state tracking - learning observation and motion models\nconditioned on these expectable events. Together with a mapper that constructs\na semantic spatial map on-the-fly during navigation, we formulate an end-to-end\ndifferentiable Bayes filter and train it to identify the goal by predicting the\nmost likely trajectory through the map according to the instructions. The\nresulting navigation policy constitutes a new approach to instruction following\nthat explicitly models a probability distribution over states, encoding strong\ngeometric and algorithmic priors while enabling greater explainability. Our\nexperiments show that our approach outperforms a strong LingUNet baseline when\npredicting the goal location on the map. On the full VLN task, i.e. navigating\nto the goal location, our approach achieves promising results with less\nreliance on navigation constraints.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.02031v1",
            "date": "2019-07-03 16:53:28+00:00",
            "title": "Combining Q&A Pair Quality and Question Relevance Features on Community-based Question Retrieval",
            "authors": [
                "Dong Li",
                "Lin Li"
            ],
            "abstract": "The Q&A community has become an important way for people to access knowledge\nand information from the Internet. However, the existing translation based on\nmodels does not consider the query specific semantics when assigning weights to\nquery terms in question retrieval. So we improve the term weighting model based\non the traditional topic translation model and further considering the quality\ncharacteristics of question and answer pairs, this paper proposes a\ncommunitybased question retrieval method that combines question and answer on\nquality and question relevance (T2LM+). We have also proposed a question\nretrieval method based on convolutional neural networks. The results show that\nCompared with the relatively advanced methods, the two methods proposed in this\npaper increase MAP by 4.91% and 6.31%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.02046v1",
            "date": "2019-07-03 17:24:14+00:00",
            "title": "Deep neural network-based classification model for Sentiment Analysis",
            "authors": [
                "Donghang Pan",
                "Jingling Yuan",
                "Lin Li",
                "Deming Sheng"
            ],
            "abstract": "The growing prosperity of social networks has brought great challenges to the\nsentimental tendency mining of users. As more and more researchers pay\nattention to the sentimental tendency of online users, rich research results\nhave been obtained based on the sentiment classification of explicit texts.\nHowever, research on the implicit sentiment of users is still in its infancy.\nAiming at the difficulty of implicit sentiment classification, a research on\nimplicit sentiment classification model based on deep neural network is carried\nout. Classification models based on DNN, LSTM, Bi-LSTM and CNN were established\nto judge the tendency of the user's implicit sentiment text. Based on the\nBi-LSTM model, the classification model of word-level attention mechanism is\nstudied. The experimental results on the public dataset show that the\nestablished LSTM series classification model and CNN classification model can\nachieve good sentiment classification effect, and the classification effect is\nsignificantly better than the DNN model. The Bi-LSTM based attention mechanism\nclassification model obtained the optimal R value in the positive category\nidentification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02052v1",
            "date": "2019-07-01 00:02:59+00:00",
            "title": "Patent Claim Generation by Fine-Tuning OpenAI GPT-2",
            "authors": [
                "Jieh-Sheng Lee",
                "Jieh Hsiang"
            ],
            "abstract": "In this work, we focus on fine-tuning an OpenAI GPT-2 pre-trained model for\ngenerating patent claims. GPT-2 has demonstrated impressive efficacy of\npre-trained language models on various tasks, particularly coherent text\ngeneration. Patent claim language itself has rarely been explored in the past\nand poses a unique challenge. We are motivated to generate coherent patent\nclaims automatically so that augmented inventing might be viable someday. In\nour implementation, we identified a unique language structure in patent claims\nand leveraged its implicit human annotations. We investigated the fine-tuning\nprocess by probing the first 100 steps and observing the generated text at each\nstep. Based on both conditional and unconditional random sampling, we analyze\nthe overall quality of generated patent claims. Our contributions include: (1)\nbeing the first to generate patent claims by machines and being the first to\napply GPT-2 to patent claim generation, (2) providing various experiment\nresults for qualitative analysis and future research, (3) proposing a new\nsampling approach for text generation, and (4) building an e-mail bot for\nfuture researchers to explore the fine-tuned GPT-2 model further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02065v1",
            "date": "2019-07-02 22:49:25+00:00",
            "title": "Neural Image Captioning",
            "authors": [
                "Elaina Tan",
                "Lakshay Sharma"
            ],
            "abstract": "In recent years, the biggest advances in major Computer Vision tasks, such as\nobject recognition, handwritten-digit identification, facial recognition, and\nmany others., have all come through the use of Convolutional Neural Networks\n(CNNs). Similarly, in the domain of Natural Language Processing, Recurrent\nNeural Networks (RNNs), and Long Short Term Memory networks (LSTMs) in\nparticular, have been crucial to some of the biggest breakthroughs in\nperformance for tasks such as machine translation, part-of-speech tagging,\nsentiment analysis, and many others. These individual advances have greatly\nbenefited tasks even at the intersection of NLP and Computer Vision, and\ninspired by this success, we studied some existing neural image captioning\nmodels that have proven to work well. In this work, we study some existing\ncaptioning models that provide near state-of-the-art performances, and try to\nenhance one such model. We also present a simple image captioning model that\nmakes use of a CNN, an LSTM, and the beam search1 algorithm, and study its\nperformance based on various qualitative and quantitative metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02106v1",
            "date": "2019-07-03 18:58:49+00:00",
            "title": "Use of OWL and Semantic Web Technologies at Pinterest",
            "authors": [
                "Rafael S. Gon\u00e7alves",
                "Matthew Horridge",
                "Rui Li",
                "Yu Liu",
                "Mark A. Musen",
                "Csongor I. Nyulas",
                "Evelyn Obamos",
                "Dhananjay Shrouty",
                "David Temple"
            ],
            "abstract": "Pinterest is a popular Web application that has over 250 million active\nusers. It is a visual discovery engine for finding ideas for recipes, fashion,\nweddings, home decoration, and much more. In the last year, the company adopted\nSemantic Web technologies to create a knowledge graph that aims to represent\nthe vast amount of content and users on Pinterest, to help both content\nrecommendation and ads targeting. In this paper, we present the engineering of\nan OWL ontology---the Pinterest Taxonomy---that forms the core of Pinterest's\nknowledge graph, the Pinterest Taste Graph. We describe modeling choices and\nenhancements to WebProt\\'eg\\'e that we used for the creation of the ontology.\nIn two months, eight Pinterest engineers, without prior experience of OWL and\nWebProt\\'eg\\'e, revamped an existing taxonomy of noisy terms into an OWL\nontology. We share our experience and present the key aspects of our work that\nwe believe will be useful for others working in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.02606v1",
            "date": "2019-07-04 21:42:29+00:00",
            "title": "A Road-map Towards Explainable Question Answering A Solution for Information Pollution",
            "authors": [
                "Saeedeh Shekarpour",
                "Faisal Alshargi"
            ],
            "abstract": "The increasing rate of information pollution on the Web requires novel\nsolutions to tackle that. Question Answering (QA) interfaces are simplified and\nuser-friendly interfaces to access information on the Web. However, similar to\nother AI applications, they are black boxes which do not manifest the details\nof the learning or reasoning steps for augmenting an answer. The Explainable\nQuestion Answering (XQA) system can alleviate the pain of information pollution\nwhere it provides transparency to the underlying computational model and\nexposes an interface enabling the end-user to access and validate provenance,\nvalidity, context, circulation, interpretation, and feedbacks of information.\nThis position paper sheds light on the core concepts, expectations, and\nchallenges in favor of the following questions (i) What is an XQA system?, (ii)\nWhy do we need XQA?, (iii) When do we need XQA? (iv) How to represent the\nexplanations? (iv) How to evaluate XQA systems?",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.02784v1",
            "date": "2019-07-05 12:00:53+00:00",
            "title": "A Methodology for Controlling the Emotional Expressiveness in Synthetic Speech -- a Deep Learning approach",
            "authors": [
                "No\u00e9 Tits"
            ],
            "abstract": "In this project, we aim to build a Text-to-Speech system able to produce\nspeech with a controllable emotional expressiveness. We propose a methodology\nfor solving this problem in three main steps. The first is the collection of\nemotional speech data. We discuss the various formats of existing datasets and\ntheir usability in speech generation. The second step is the development of a\nsystem to automatically annotate data with emotion/expressiveness features. We\ncompare several techniques using transfer learning to extract such a\nrepresentation through other tasks and propose a method to visualize and\ninterpret the correlation between vocal and emotional features. The third step\nis the development of a deep learning-based system taking text and\nemotion/expressiveness as input and producing speech as output. We study the\nimpact of fine tuning from a neutral TTS towards an emotional TTS in terms of\nintelligibility and perception of the emotion.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.02864v1",
            "date": "2019-07-05 14:43:55+00:00",
            "title": "Deep Neural Baselines for Computational Paralinguistics",
            "authors": [
                "Daniel Elsner",
                "Stefan Langer",
                "Fabian Ritz",
                "Robert M\u00fcller",
                "Steffen Illium"
            ],
            "abstract": "Detecting sleepiness from spoken language is an ambitious task, which is\naddressed by the Interspeech 2019 Computational Paralinguistics Challenge\n(ComParE). We propose an end-to-end deep learning approach to detect and\nclassify patterns reflecting sleepiness in the human voice. Our approach is\nbased solely on a moderately complex deep neural network architecture. It may\nbe applied directly on the audio data without requiring any specific feature\nengineering, thus remaining transferable to other audio classification tasks.\nNevertheless, our approach performs similar to state-of-the-art machine\nlearning models.",
            "journal ref": "Proc. Interspeech 2019, 2388-2392",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1907.03007v1",
            "date": "2019-07-05 19:47:10+00:00",
            "title": "NeuType: A Simple and Effective Neural Network Approach for Predicting Missing Entity Type Information in Knowledge Bases",
            "authors": [
                "Jon Arne B\u00f8 Hovda",
                "Dar\u00edo Garigliotti",
                "Krisztian Balog"
            ],
            "abstract": "Knowledge bases store information about the semantic types of entities, which\ncan be utilized in a range of information access tasks. This information,\nhowever, is often incomplete, due to new entities emerging on a daily basis. We\naddress the task of automatically assigning types to entities in a knowledge\nbase from a type taxonomy. Specifically, we present two neural network\narchitectures, which take short entity descriptions and, optionally,\ninformation about related entities as input. Using the DBpedia knowledge base\nfor experimental evaluation, we demonstrate that these simple architectures\nyield significant improvements over the current state of the art.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.03064v1",
            "date": "2019-07-06 02:53:10+00:00",
            "title": "Improved low-resource Somali speech recognition by semi-supervised acoustic and language model training",
            "authors": [
                "Astik Biswas",
                "Raghav Menon",
                "Ewald van der Westhuizen",
                "Thomas Niesler"
            ],
            "abstract": "We present improvements in automatic speech recognition (ASR) for Somali, a\ncurrently extremely under-resourced language. This forms part of a continuing\nUnited Nations (UN) effort to employ ASR-based keyword spotting systems to\nsupport humanitarian relief programmes in rural Africa. Using just 1.57 hours\nof annotated speech data as a seed corpus, we increase the pool of training\ndata by applying semi-supervised training to 17.55 hours of untranscribed\nspeech. We make use of factorised time-delay neural networks (TDNN-F) for\nacoustic modelling, since these have recently been shown to be effective in\nresource-scarce situations. Three semi-supervised training passes were\nperformed, where the decoded output from each pass was used for acoustic model\ntraining in the subsequent pass. The automatic transcriptions from the best\nperforming pass were used for language model augmentation. To ensure the\nquality of automatic transcriptions, decoder confidence is used as a threshold.\nThe acoustic and language models obtained from the semi-supervised approach\nshow significant improvement in terms of WER and perplexity compared to the\nbaseline. Incorporating the automatically generated transcriptions yields a\n6.55\\% improvement in language model perplexity. The use of 17.55 hour of\nSomali acoustic data in semi-supervised training shows an improvement of 7.74\\%\nrelative over the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03167v1",
            "date": "2019-07-06 18:58:54+00:00",
            "title": "Exploring difference in public perceptions on HPV vaccine between gender groups from Twitter using deep learning",
            "authors": [
                "Jingcheng Du",
                "Chongliang Luo",
                "Qiang Wei",
                "Yong Chen",
                "Cui Tao"
            ],
            "abstract": "In this study, we proposed a convolutional neural network model for gender\nprediction using English Twitter text as input. Ensemble of proposed model\nachieved an accuracy at 0.8237 on gender prediction and compared favorably with\nthe state-of-the-art performance in a recent author profiling task. We further\nleveraged the trained models to predict the gender labels from an HPV vaccine\nrelated corpus and identified gender difference in public perceptions regarding\nHPV vaccine. The findings are largely consistent with previous survey-based\nstudies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03233v1",
            "date": "2019-07-07 07:03:09+00:00",
            "title": "NIESR: Nuisance Invariant End-to-end Speech Recognition",
            "authors": [
                "I-Hung Hsu",
                "Ayush Jaiswal",
                "Premkumar Natarajan"
            ],
            "abstract": "Deep neural network models for speech recognition have achieved great success\nrecently, but they can learn incorrect associations between the target and\nnuisance factors of speech (e.g., speaker identities, background noise, etc.),\nwhich can lead to overfitting. While several methods have been proposed to\ntackle this problem, existing methods incorporate additional information about\nnuisance factors during training to develop invariant models. However,\nenumeration of all possible nuisance factors in speech data and the collection\nof their annotations is difficult and expensive. We present a robust training\nscheme for end-to-end speech recognition that adopts an unsupervised\nadversarial invariance induction framework to separate out essential factors\nfor speech-recognition from nuisances without using any supplementary labels\nbesides the transcriptions. Experiments show that the speech recognition model\ntrained with the proposed training scheme achieves relative improvements of\n5.48% on WSJ0, 6.16% on CHiME3, and 6.61% on TIMIT dataset over the base model.\nAdditionally, the proposed method achieves a relative improvement of 14.44% on\nthe combined WSJ0+CHiME3 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03533v2",
            "date": "2019-07-04 04:52:45+00:00",
            "title": "A Formal Axiomatization of Computation",
            "authors": [
                "Rasoul Ramezanian"
            ],
            "abstract": "We introduce an axiomatization for the notion of computation. Based on the\nidea of Brouwer choice sequences, we construct a model, denoted by $E$, which\nsatisfies our axioms and $E \\models \\mathrm{ P \\neq NP}$. In other words,\nregarding \"effective computability\" in Brouwer intuitionism viewpoint, we show\n$\\mathrm{ P \\neq NP}$.",
            "journal ref": "None",
            "category": "cs.CC"
        },
        {
            "url": "http://arxiv.org/abs/1907.03715v1",
            "date": "2019-07-08 16:39:23+00:00",
            "title": "Predicting Customer Call Intent by Analyzing Phone Call Transcripts based on CNN for Multi-Class Classification",
            "authors": [
                "Junmei Zhong",
                "William Li"
            ],
            "abstract": "Auto dealerships receive thousands of calls daily from customers who are\ninterested in sales, service, vendors and jobseekers. With so many calls, it is\nvery important for auto dealers to understand the intent of these calls to\nprovide positive customer experiences that ensure customer satisfaction, deep\ncustomer engagement to boost sales and revenue, and optimum allocation of\nagents or customer service representatives across the business. In this paper,\nwe define the problem of customer phone call intent as a multi-class\nclassification problem stemming from the large database of recorded phone call\ntranscripts. To solve this problem, we develop a convolutional neural network\n(CNN)-based supervised learning model to classify the customer calls into four\nintent categories: sales, service, vendor and jobseeker. Experimental results\nshow that with the thrust of our scalable data labeling method to provide\nsufficient training data, the CNN-based predictive model performs very well on\nlong text classification according to the quantitative metrics of F1-Score,\nprecision, recall, and accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.03748v1",
            "date": "2019-07-06 10:04:12+00:00",
            "title": "Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss",
            "authors": [
                "Laura Jehl",
                "Carolin Lawrence",
                "Stefan Riezler"
            ],
            "abstract": "In many machine learning scenarios, supervision by gold labels is not\navailable and consequently neural models cannot be trained directly by maximum\nlikelihood estimation (MLE). In a weak supervision scenario, metric-augmented\nobjectives can be employed to assign feedback to model outputs, which can be\nused to extract a supervision signal for training. We present several\nobjectives for two separate weakly supervised tasks, machine translation and\nsemantic parsing. We show that objectives should actively discourage negative\noutputs in addition to promoting a surrogate gold structure. This notion of\nbipolarity is naturally present in ramp loss objectives, which we adapt to\nneural models. We show that bipolar ramp loss objectives outperform other\nnon-bipolar ramp loss objectives and minimum risk training (MRT) on both weakly\nsupervised tasks, as well as on a supervised machine translation task.\nAdditionally, we introduce a novel token-level ramp loss objective, which is\nable to outperform even the best sequence-level ramp loss on both weakly\nsupervised tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03750v1",
            "date": "2019-07-07 12:59:04+00:00",
            "title": "Neural Aspect and Opinion Term Extraction with Mined Rules as Weak Supervision",
            "authors": [
                "Hongliang Dai",
                "Yangqiu Song"
            ],
            "abstract": "Lack of labeled training data is a major bottleneck for neural network based\naspect and opinion term extraction on product reviews. To alleviate this\nproblem, we first propose an algorithm to automatically mine extraction rules\nfrom existing training examples based on dependency parsing results. The mined\nrules are then applied to label a large amount of auxiliary data. Finally, we\nstudy training procedures to train a neural model which can learn from both the\ndata automatically labeled by the rules and a small amount of data accurately\nannotated by human. Experimental results show that although the mined rules\nthemselves do not perform well due to their limited flexibility, the\ncombination of human annotated data and rule labeled auxiliary data can improve\nthe neural model and allow it to achieve performance better than or comparable\nwith the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03752v2",
            "date": "2019-07-07 18:05:12+00:00",
            "title": "Improving short text classification through global augmentation methods",
            "authors": [
                "Vukosi Marivate",
                "Tshephisho Sefara"
            ],
            "abstract": "We study the effect of different approaches to text augmentation. To do this\nwe use 3 datasets that include social media and formal text in the form of news\narticles. Our goal is to provide insights for practitioners and researchers on\nmaking choices for augmentation for classification use cases. We observe that\nWord2vec-based augmentation is a viable option when one does not have access to\na formal synonym model (like WordNet-based augmentation). The use of\n\\emph{mixup} further improves performance of all text based augmentations and\nreduces the effects of overfitting on a tested deep learning model. Round-trip\ntranslation with a translation service proves to be harder to use due to cost\nand as such is less accessible for both normal and low resource use-cases.",
            "journal ref": "Machine Learning and Knowledge Extraction. CD-MAKE 2020. Lecture\n  Notes in Computer Science, vol 12279 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03885v1",
            "date": "2019-07-08 21:39:29+00:00",
            "title": "An Intrinsic Nearest Neighbor Analysis of Neural Machine Translation Architectures",
            "authors": [
                "Hamidreza Ghader",
                "Christof Monz"
            ],
            "abstract": "Earlier approaches indirectly studied the information captured by the hidden\nstates of recurrent and non-recurrent neural machine translation models by\nfeeding them into different classifiers. In this paper, we look at the encoder\nhidden states of both transformer and recurrent machine translation models from\nthe nearest neighbors perspective. We investigate to what extent the nearest\nneighbors share information with the underlying word embeddings as well as\nrelated WordNet entries. Additionally, we study the underlying syntactic\nstructure of the nearest neighbors to shed light on the role of syntactic\nsimilarities in bringing the neighbors together. We compare transformer and\nrecurrent models in a more intrinsic way in terms of capturing lexical\nsemantics and syntactic structures, in contrast to extrinsic approaches used by\nprevious works. In agreement with the extrinsic evaluations in the earlier\nworks, our experimental results show that transformers are superior in\ncapturing lexical semantics, but not necessarily better in capturing the\nunderlying syntax. Additionally, we show that the backward recurrent layer in a\nrecurrent model learns more about the semantics of words, whereas the forward\nrecurrent layer encodes more context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.03950v4",
            "date": "2019-07-09 03:08:41+00:00",
            "title": "Learning by Abstraction: The Neural State Machine",
            "authors": [
                "Drew A. Hudson",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce the Neural State Machine, seeking to bridge the gap between the\nneural and symbolic views of AI and integrate their complementary strengths for\nthe task of visual reasoning. Given an image, we first predict a probabilistic\ngraph that represents its underlying semantics and serves as a structured world\nmodel. Then, we perform sequential reasoning over the graph, iteratively\ntraversing its nodes to answer a given question or draw a new inference. In\ncontrast to most neural architectures that are designed to closely interact\nwith the raw sensory data, our model operates instead in an abstract latent\nspace, by transforming both the visual and linguistic modalities into semantic\nconcept-based representations, thereby achieving enhanced transparency and\nmodularity. We evaluate our model on VQA-CP and GQA, two recent VQA datasets\nthat involve compositionality, multi-step inference and diverse reasoning\nskills, achieving state-of-the-art results in both cases. We provide further\nexperiments that illustrate the model's strong generalization capacity across\nmultiple dimensions, including novel compositions of concepts, changes in the\nanswer distribution, and unseen linguistic structures, demonstrating the\nqualities and efficacy of our approach.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.04072v1",
            "date": "2019-07-09 10:42:33+00:00",
            "title": "Multitask Learning for Blackmarket Tweet Detection",
            "authors": [
                "Udit Arora",
                "William Scott Paka",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Online social media platforms have made the world more connected than ever\nbefore, thereby making it easier for everyone to spread their content across a\nwide variety of audiences. Twitter is one such popular platform where people\npublish tweets to spread their messages to everyone. Twitter allows users to\nRetweet other users' tweets in order to broadcast it to their network. The more\nretweets a particular tweet gets, the faster it spreads. This creates\nincentives for people to obtain artificial growth in the reach of their tweets\nby using certain blackmarket services to gain inorganic appraisals for their\ncontent.\n  In this paper, we attempt to detect such tweets that have been posted on\nthese blackmarket services in order to gain artificially boosted retweets. We\nuse a multitask learning framework to leverage soft parameter sharing between a\nclassification and a regression based task on separate inputs. This allows us\nto effectively detect tweets that have been posted to these blackmarket\nservices, achieving an F1-score of 0.89 when classifying tweets as blackmarket\nor genuine.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.04152v3",
            "date": "2019-07-03 15:22:04+00:00",
            "title": "Interpretable Segmentation of Medical Free-Text Records Based on Word Embeddings",
            "authors": [
                "Adam Gabriel Dobrakowski",
                "Agnieszka Mykowiecka",
                "Ma\u0142gorzata Marciniak",
                "Wojciech Jaworski",
                "Przemys\u0142aw Biecek"
            ],
            "abstract": "Is it true that patients with similar conditions get similar diagnoses? In\nthis paper we show NLP methods and a unique corpus of documents to validate\nthis claim. We (1) introduce a method for representation of medical visits\nbased on free-text descriptions recorded by doctors, (2) introduce a new method\nfor clustering of patients' visits and (3) present an~application of the\nproposed method on a corpus of 100,000 visits. With the proposed method we\nobtained stable and separated segments of visits which were positively\nvalidated against final medical diagnoses. We show how the presented algorithm\nmay be used to aid doctors during their practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04197v1",
            "date": "2019-07-08 03:50:43+00:00",
            "title": "Attending to Emotional Narratives",
            "authors": [
                "Zhengxuan Wu",
                "Xiyu Zhang",
                "Tan Zhi-Xuan",
                "Jamil Zaki",
                "Desmond C. Ong"
            ],
            "abstract": "Attention mechanisms in deep neural networks have achieved excellent\nperformance on sequence-prediction tasks. Here, we show that these\nrecently-proposed attention-based mechanisms---in particular, the Transformer\nwith its parallelizable self-attention layers, and the Memory Fusion Network\nwith attention across modalities and time---also generalize well to multimodal\ntime-series emotion recognition. Using a recently-introduced dataset of\nemotional autobiographical narratives, we adapt and apply these two attention\nmechanisms to predict emotional valence over time. Our models perform extremely\nwell, in some cases reaching a performance comparable with human raters. We end\nwith a discussion of the implications of attention mechanisms to affective\ncomputing.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.04198v1",
            "date": "2019-07-09 14:41:50+00:00",
            "title": "Sequence-to-Sequence Natural Language to Humanoid Robot Sign Language",
            "authors": [
                "Jennifer J. Gago",
                "Valentina Vasco",
                "Bartek \u0141ukawski",
                "Ugo Pattacini",
                "Vadim Tikhanoff",
                "Juan G. Victores",
                "Carlos Balaguer"
            ],
            "abstract": "This paper presents a study on natural language to sign language translation\nwith human-robot interaction application purposes. By means of the presented\nmethodology, the humanoid robot TEO is expected to represent Spanish sign\nlanguage automatically by converting text into movements, thanks to the\nperformance of neural networks. Natural language to sign language translation\npresents several challenges to developers, such as the discordance between the\nlength of input and output data and the use of non-manual markers. Therefore,\nneural networks and, consequently, sequence-to-sequence models, are selected as\na data-driven system to avoid traditional expert system approaches or temporal\ndependencies limitations that lead to limited or too complex translation\nsystems. To achieve these objectives, it is necessary to find a way to perform\nhuman skeleton acquisition in order to collect the signing input data. OpenPose\nand skeletonRetriever are proposed for this purpose and a 3D sensor\nspecification study is developed to select the best acquisition hardware.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1907.04211v1",
            "date": "2019-07-06 20:12:13+00:00",
            "title": "Universal One-Dimensional Cellular Automata Derived for Turing Machines and its Dynamical Behaviour",
            "authors": [
                "Sergio J. Martinez",
                "Ivan M. Mendoza",
                "Genaro J. Martinez",
                "Shigeru Ninagawa"
            ],
            "abstract": "Universality in cellular automata theory is a central problem studied and\ndeveloped from their origins by John von Neumann. In this paper, we present an\nalgorithm where any Turing machine can be converted to one-dimensional cellular\nautomaton with a 2-linear time and display its spatial dynamics. Three\nparticular Turing machines are converted in three universal one-dimensional\ncellular automata, they are: binary sum, rule 110 and a universal reversible\nTuring machine.",
            "journal ref": "International Journal of Unconventional Computing 14(2) pages\n  121-138, 2019",
            "category": "nlin.CG"
        },
        {
            "url": "http://arxiv.org/abs/1907.04224v2",
            "date": "2019-07-09 14:59:16+00:00",
            "title": "Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition",
            "authors": [
                "Yonatan Belinkov",
                "Ahmed Ali",
                "James Glass"
            ],
            "abstract": "End-to-end neural network systems for automatic speech recognition (ASR) are\ntrained from acoustic features to text transcriptions. In contrast to modular\nASR systems, which contain separately-trained components for acoustic modeling,\npronunciation lexicon, and language modeling, the end-to-end paradigm is both\nconceptually simpler and has the potential benefit of training the entire\nsystem on the end task. However, such neural network models are more opaque: it\nis not clear how to interpret the role of different parts of the network and\nwhat information it learns during training. In this paper, we analyze the\nlearned internal representations in an end-to-end ASR model. We evaluate the\nrepresentation quality in terms of several classification tasks, comparing\nphonemes and graphemes, as well as different articulatory features. We study\ntwo languages (English and Arabic) and three datasets, finding remarkable\nconsistency in how different properties are represented in different layers of\nthe deep neural network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04286v1",
            "date": "2019-07-09 16:47:50+00:00",
            "title": "UW-BHI at MEDIQA 2019: An Analysis of Representation Methods for Medical Natural Language Inference",
            "authors": [
                "William R. Kearns",
                "Wilson Lau",
                "Jason A. Thomas"
            ],
            "abstract": "Recent advances in distributed language modeling have led to large\nperformance increases on a variety of natural language processing (NLP) tasks.\nHowever, it is not well understood how these methods may be augmented by\nknowledge-based approaches. This paper compares the performance and internal\nrepresentation of an Enhanced Sequential Inference Model (ESIM) between three\nexperimental conditions based on the representation method: Bidirectional\nEncoder Representations from Transformers (BERT), Embeddings of Semantic\nPredications (ESP), or Cui2Vec. The methods were evaluated on the Medical\nNatural Language Inference (MedNLI) subtask of the MEDIQA 2019 shared task.\nThis task relied heavily on semantic understanding and thus served as a\nsuitable evaluation set for the comparison of these representation methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.04355v1",
            "date": "2019-07-09 18:23:32+00:00",
            "title": "Transfer Learning from Audio-Visual Grounding to Speech Recognition",
            "authors": [
                "Wei-Ning Hsu",
                "David Harwath",
                "James Glass"
            ],
            "abstract": "Transfer learning aims to reduce the amount of data required to excel at a\nnew task by re-using the knowledge acquired from learning other related tasks.\nThis paper proposes a novel transfer learning scenario, which distills robust\nphonetic features from grounding models that are trained to tell whether a pair\nof image and speech are semantically correlated, without using any textual\ntranscripts. As semantics of speech are largely determined by its lexical\ncontent, grounding models learn to preserve phonetic information while\ndisregarding uncorrelated factors, such as speaker and channel. To study the\nproperties of features distilled from different layers, we use them as input\nseparately to train multiple speech recognition models. Empirical results\ndemonstrate that layers closer to input retain more phonetic information, while\nfollowing layers exhibit greater invariance to domain shift. Moreover, while\nmost previous studies include training data for speech recognition for feature\nextractor training, our grounding models are not trained on any of those data,\nindicating more universal applicability to new domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04433v2",
            "date": "2019-07-09 21:59:44+00:00",
            "title": "GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing",
            "authors": [
                "Jian Guo",
                "He He",
                "Tong He",
                "Leonard Lausen",
                "Mu Li",
                "Haibin Lin",
                "Xingjian Shi",
                "Chenguang Wang",
                "Junyuan Xie",
                "Sheng Zha",
                "Aston Zhang",
                "Hang Zhang",
                "Zhi Zhang",
                "Zhongyue Zhang",
                "Shuai Zheng",
                "Yi Zhu"
            ],
            "abstract": "We present GluonCV and GluonNLP, the deep learning toolkits for computer\nvision and natural language processing based on Apache MXNet (incubating).\nThese toolkits provide state-of-the-art pre-trained models, training scripts,\nand training logs, to facilitate rapid prototyping and promote reproducible\nresearch. We also provide modular APIs with flexible building blocks to enable\nefficient customization. Leveraging the MXNet ecosystem, the deep learning\nmodels in GluonCV and GluonNLP can be deployed onto a variety of platforms with\ndifferent programming languages. The Apache 2.0 license has been adopted by\nGluonCV and GluonNLP to allow for software distribution, modification, and\nusage.",
            "journal ref": "Journal of Machine Learning Research 21 (2020) 1-7",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.04448v2",
            "date": "2019-07-09 22:43:55+00:00",
            "title": "Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning",
            "authors": [
                "Yu Zhang",
                "Ron J. Weiss",
                "Heiga Zen",
                "Yonghui Wu",
                "Zhifeng Chen",
                "RJ Skerry-Ryan",
                "Ye Jia",
                "Andrew Rosenberg",
                "Bhuvana Ramabhadran"
            ],
            "abstract": "We present a multispeaker, multilingual text-to-speech (TTS) synthesis model\nbased on Tacotron that is able to produce high quality speech in multiple\nlanguages. Moreover, the model is able to transfer voices across languages,\ne.g. synthesize fluent Spanish speech using an English speaker's voice, without\ntraining on any bilingual or parallel examples. Such transfer works across\ndistantly related languages, e.g. English and Mandarin.\n  Critical to achieving this result are: 1. using a phonemic input\nrepresentation to encourage sharing of model capacity across languages, and 2.\nincorporating an adversarial loss term to encourage the model to disentangle\nits representation of speaker identity (which is perfectly correlated with\nlanguage in the training data) from the speech content. Further scaling up the\nmodel by training on multiple speakers of each language, and incorporating an\nautoencoding input to help stabilize attention during training, results in a\nmodel which can be used to consistently synthesize intelligible speech for\ntraining speakers in all languages seen during training, and in native or\nforeign accents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04462v1",
            "date": "2019-07-09 23:53:39+00:00",
            "title": "Multi-Speaker End-to-End Speech Synthesis",
            "authors": [
                "Jihyun Park",
                "Kexin Zhao",
                "Kainan Peng",
                "Wei Ping"
            ],
            "abstract": "In this work, we extend ClariNet (Ping et al., 2019), a fully end-to-end\nspeech synthesis model (i.e., text-to-wave), to generate high-fidelity speech\nfrom multiple speakers. To model the unique characteristic of different voices,\nlow dimensional trainable speaker embeddings are shared across each component\nof ClariNet and trained together with the rest of the model. We demonstrate\nthat the multi-speaker ClariNet outperforms state-of-the-art systems in terms\nof naturalness, because the whole model is jointly optimized in an end-to-end\nmanner.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04670v4",
            "date": "2019-07-09 15:12:51+00:00",
            "title": "Improving the Performance of the LSTM and HMM Model via Hybridization",
            "authors": [
                "Larkin Liu",
                "Yu-Chung Lin",
                "Joshua Reid"
            ],
            "abstract": "Language models based on deep neural networks and traditional stochastic\nmodelling have become both highly functional and effective in recent times. In\nthis work, a general survey into the two types of language modelling is\nconducted. We investigate the effectiveness of the Hidden Markov Model (HMM),\nand the Long Short-Term Memory Model (LSTM). We analyze the hidden state\nstructures common to both models, and present an analysis on structural\nsimilarity of the hidden states, common to both HMM's and LSTM's. We compare\nthe LSTM's predictive accuracy and hidden state output with respect to the HMM\nfor a varying number of hidden states. In this work, we justify that the less\ncomplex HMM can serve as an appropriate approximation of the LSTM model.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.04743v1",
            "date": "2019-07-10 14:20:02+00:00",
            "title": "Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech",
            "authors": [
                "Daniel Korzekwa",
                "Roberto Barra-Chicote",
                "Bozena Kostek",
                "Thomas Drugman",
                "Mateusz Lajszczak"
            ],
            "abstract": "This paper proposed a novel approach for the detection and reconstruction of\ndysarthric speech. The encoder-decoder model factorizes speech into a\nlow-dimensional latent space and encoding of the input text. We showed that the\nlatent space conveys interpretable characteristics of dysarthria, such as\nintelligibility and fluency of speech. MUSHRA perceptual test demonstrated that\nthe adaptation of the latent space let the model generate speech of improved\nfluency. The multi-task supervised approach for predicting both the probability\nof dysarthric speech and the mel-spectrogram helps improve the detection of\ndysarthria with higher accuracy. This is thanks to a low-dimensional latent\nspace of the auto-encoder as opposed to directly predicting dysarthria from a\nhighly dimensional mel-spectrogram.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.04882v1",
            "date": "2019-07-10 18:38:44+00:00",
            "title": "Acoustic Model Optimization Based On Evolutionary Stochastic Gradient Descent with Anchors for Automatic Speech Recognition",
            "authors": [
                "Xiaodong Cui",
                "Michael Picheny"
            ],
            "abstract": "Evolutionary stochastic gradient descent (ESGD) was proposed as a\npopulation-based approach that combines the merits of gradient-aware and\ngradient-free optimization algorithms for superior overall optimization\nperformance. In this paper we investigate a variant of ESGD for optimization of\nacoustic models for automatic speech recognition (ASR). In this variant, we\nassume the existence of a well-trained acoustic model and use it as an anchor\nin the parent population whose good \"gene\" will propagate in the evolution to\nthe offsprings. We propose an ESGD algorithm leveraging the anchor models such\nthat it guarantees the best fitness of the population will never degrade from\nthe anchor model. Experiments on 50-hour Broadcast News (BN50) and 300-hour\nSwitchboard (SWB300) show that the ESGD with anchors can further improve the\nloss and ASR performance over the existing well-trained acoustic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.04887v1",
            "date": "2019-07-10 18:52:11+00:00",
            "title": "Large-Scale Mixed-Bandwidth Deep Neural Network Acoustic Modeling for Automatic Speech Recognition",
            "authors": [
                "Khoi-Nguyen C. Mac",
                "Xiaodong Cui",
                "Wei Zhang",
                "Michael Picheny"
            ],
            "abstract": "In automatic speech recognition (ASR), wideband (WB) and narrowband (NB)\nspeech signals with different sampling rates typically use separate acoustic\nmodels. Therefore mixed-bandwidth (MB) acoustic modeling has important\npractical values for ASR system deployment. In this paper, we extensively\ninvestigate large-scale MB deep neural network acoustic modeling for ASR using\n1,150 hours of WB data and 2,300 hours of NB data. We study various MB\nstrategies including downsampling, upsampling and bandwidth extension for MB\nacoustic modeling and evaluate their performance on 8 diverse WB and NB test\nsets from various application domains. To deal with the large amounts of\ntraining data, distributed training is carried out on multiple GPUs using\nsynchronous data parallelism.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.04905v1",
            "date": "2019-07-08 03:24:53+00:00",
            "title": "Development of email classifier in Brazilian Portuguese using feature selection for automatic response",
            "authors": [
                "Rogerio Bonatti",
                "Arthur Gola de Paula"
            ],
            "abstract": "Automatic email categorization is an important application of text\nclassification. We study the automatic reply of email business messages in\nBrazilian Portuguese. We present a novel corpus containing messages from a real\napplication, and baseline categorization experiments using Naive Bayes and\nsupport Vector Machines. We then discuss the effect of lemmatization and the\nrole of part-of-speech tagging filtering on precision and recall. Support\nVector Machines classification coupled with nonlemmatized selection of verbs,\nnouns and adjectives was the best approach, with 87.3% maximum accuracy.\nStraightforward lemmatization in Portuguese led to the lowest classification\nresults in the group, with 85.3% and 81.7% precision in SVM and Naive Bayes\nrespectively. Thus, while lemmatization reduced precision and recall,\npart-of-speech filtering improved overall results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.04907v1",
            "date": "2019-07-08 03:50:57+00:00",
            "title": "Topic Modeling in Embedding Spaces",
            "authors": [
                "Adji B. Dieng",
                "Francisco J. R. Ruiz",
                "David M. Blei"
            ],
            "abstract": "Topic modeling analyzes documents to learn meaningful patterns of words.\nHowever, existing topic models fail to learn interpretable topics when working\nwith large and heavy-tailed vocabularies. To this end, we develop the Embedded\nTopic Model (ETM), a generative model of documents that marries traditional\ntopic models with word embeddings. In particular, it models each word with a\ncategorical distribution whose natural parameter is the inner product between a\nword embedding and an embedding of its assigned topic. To fit the ETM, we\ndevelop an efficient amortized variational inference algorithm. The ETM\ndiscovers interpretable topics even with large vocabularies that include rare\nwords and stop words. It outperforms existing document models, such as latent\nDirichlet allocation (LDA), in terms of both topic quality and predictive\nperformance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.04957v3",
            "date": "2019-07-10 23:41:46+00:00",
            "title": "Vision-and-Dialog Navigation",
            "authors": [
                "Jesse Thomason",
                "Michael Murray",
                "Maya Cakmak",
                "Luke Zettlemoyer"
            ],
            "abstract": "Robots navigating in human environments should use language to ask for\nassistance and be able to understand human responses. To study this challenge,\nwe introduce Cooperative Vision-and-Dialog Navigation, a dataset of over 2k\nembodied, human-human dialogs situated in simulated, photorealistic home\nenvironments. The Navigator asks questions to their partner, the Oracle, who\nhas privileged access to the best next steps the Navigator should take\naccording to a shortest path planner. To train agents that search an\nenvironment for a goal location, we define the Navigation from Dialog History\ntask. An agent, given a target object and a dialog history between humans\ncooperating to find that object, must infer navigation actions towards the goal\nin unexplored environments. We establish an initial, multi-modal\nsequence-to-sequence model and demonstrate that looking farther back in the\ndialog history improves performance. Sourcecode and a live interface demo can\nbe found at https://cvdn.dev/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05092v1",
            "date": "2019-07-11 10:29:04+00:00",
            "title": "Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos",
            "authors": [
                "Shizhe Chen",
                "Yuqing Song",
                "Yida Zhao",
                "Qin Jin",
                "Zhaoyang Zeng",
                "Bei Liu",
                "Jianlong Fu",
                "Alexander Hauptmann"
            ],
            "abstract": "Contextual reasoning is essential to understand events in long untrimmed\nvideos. In this work, we systematically explore different captioning models\nwith various contexts for the dense-captioning events in video task, which aims\nto generate captions for different events in the untrimmed video. We propose\nfive types of contexts as well as two categories of event captioning models,\nand evaluate their contributions for event captioning from both accuracy and\ndiversity aspects. The proposed captioning models are plugged into our pipeline\nsystem for the dense video captioning challenge. The overall system achieves\nthe state-of-the-art performance on the dense-captioning events in video task\nwith 9.91 METEOR score on the challenge testing set.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.05336v1",
            "date": "2019-07-09 12:32:40+00:00",
            "title": "Adaptive Margin Ranking Loss for Knowledge Graph Embeddings via a Correntropy Objective Function",
            "authors": [
                "Mojtaba Nayyeri",
                "Xiaotian Zhou",
                "Sahar Vahdati",
                "Hamed Shariat Yazdi",
                "Jens Lehmann"
            ],
            "abstract": "Translation-based embedding models have gained significant attention in link\nprediction tasks for knowledge graphs. TransE is the primary model among\ntranslation-based embeddings and is well-known for its low complexity and high\nefficiency. Therefore, most of the earlier works have modified the score\nfunction of the TransE approach in order to improve the performance of link\nprediction tasks. Nevertheless, proven theoretically and experimentally, the\nperformance of TransE strongly depends on the loss function. Margin Ranking\nLoss (MRL) has been one of the earlier loss functions which is widely used for\ntraining TransE. However, the scores of positive triples are not necessarily\nenforced to be sufficiently small to fulfill the translation from head to tail\nby using relation vector (original assumption of TransE). To tackle this\nproblem, several loss functions have been proposed recently by adding upper\nbounds and lower bounds to the scores of positive and negative samples.\nAlthough highly effective, previously developed models suffer from an expansion\nin search space for a selection of the hyperparameters (in particular the upper\nand lower bounds of scores) on which the performance of the translation-based\nmodels is highly dependent. In this paper, we propose a new loss function\ndubbed Adaptive Margin Loss (AML) for training translation-based embedding\nmodels. The formulation of the proposed loss function enables an adaptive and\nautomated adjustment of the margin during the learning process. Therefore,\ninstead of obtaining two values (upper bound and lower bound), only the center\nof a margin needs to be determined. During learning, the margin is expanded\nautomatically until it converges. In our experiments on a set of standard\nbenchmark datasets including Freebase and WordNet, the effectiveness of AML is\nconfirmed for training TransE on link prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05337v1",
            "date": "2019-07-09 00:23:22+00:00",
            "title": "Joint Speech Recognition and Speaker Diarization via Sequence Transduction",
            "authors": [
                "Laurent El Shafey",
                "Hagen Soltau",
                "Izhak Shafran"
            ],
            "abstract": "Speech applications dealing with conversations require not only recognizing\nthe spoken words, but also determining who spoke when. The task of assigning\nwords to speakers is typically addressed by merging the outputs of two separate\nsystems, namely, an automatic speech recognition (ASR) system and a speaker\ndiarization (SD) system. The two systems are trained independently with\ndifferent objective functions. Often the SD systems operate directly on the\nacoustics and are not constrained to respect word boundaries and this\ndeficiency is overcome in an ad hoc manner. Motivated by recent advances in\nsequence to sequence learning, we propose a novel approach to tackle the two\ntasks by a joint ASR and SD system using a recurrent neural network transducer.\nOur approach utilizes both linguistic and acoustic cues to infer speaker roles,\nas opposed to typical SD systems, which only use acoustic cues. We evaluated\nthe performance of our approach on a large corpus of medical conversations\nbetween physicians and patients. Compared to a competitive conventional\nbaseline, our approach improves word-level diarization error rate from 15.8% to\n2.2%.",
            "journal ref": "Proc. Interspeech 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05346v1",
            "date": "2019-07-10 13:25:50+00:00",
            "title": "A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts",
            "authors": [
                "Jiahuan Pei",
                "Pengjie Ren",
                "Maarten de Rijke"
            ],
            "abstract": "End-to-end Task-oriented Dialogue Systems (TDSs) have attracted a lot of\nattention for their superiority (e.g., in terms of global optimization) over\npipeline modularized TDSs. Previous studies on end-to-end TDSs use a\nsingle-module model to generate responses for complex dialogue contexts.\nHowever, no model consistently outperforms the others in all cases. We propose\na neural Modular Task-oriented Dialogue System(MTDS) framework, in which a few\nexpert bots are combined to generate the response for a given dialogue context.\nMTDS consists of a chair bot and several expert bots. Each expert bot is\nspecialized for a particular situation, e.g., one domain, one type of action of\na system, etc. The chair bot coordinates multiple expert bots and adaptively\nselects an expert bot to generate the appropriate response. We further propose\na Token-level Mixture-of-Expert (TokenMoE) model to implement MTDS, where the\nexpert bots predict multiple tokens at each timestamp and the chair bot\ndetermines the final generated token by fully taking into consideration the\noutputs of all expert bots. Both the chair bot and the expert bots are jointly\ntrained in an end-to-end fashion. To verify the effectiveness of TokenMoE, we\ncarry out extensive experiments on a benchmark dataset. Compared with the\nbaseline using a single-module model, our TokenMoE improves the performance by\n8.1% of inform rate and 0.8% of success rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05446v2",
            "date": "2019-07-11 18:42:03+00:00",
            "title": "General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping",
            "authors": [
                "Gabriel Ilharco",
                "Vihan Jain",
                "Alexander Ku",
                "Eugene Ie",
                "Jason Baldridge"
            ],
            "abstract": "In instruction conditioned navigation, agents interpret natural language and\ntheir surroundings to navigate through an environment. Datasets for studying\nthis task typically contain pairs of these instructions and reference\ntrajectories. Yet, most evaluation metrics used thus far fail to properly\naccount for the latter, relying instead on insufficient similarity comparisons.\nWe address fundamental flaws in previously used metrics and show how Dynamic\nTime Warping (DTW), a long known method of measuring similarity between two\ntime series, can be used for evaluation of navigation agents. For such, we\ndefine the normalized Dynamic Time Warping (nDTW) metric, that softly penalizes\ndeviations from the reference path, is naturally sensitive to the order of the\nnodes composing each path, is suited for both continuous and graph-based\nevaluations, and can be efficiently calculated. Further, we define SDTW, which\nconstrains nDTW to only successful paths. We collect human similarity judgments\nfor simulated paths and find nDTW correlates better with human rankings than\nall other metrics. We also demonstrate that using nDTW as a reward signal for\nReinforcement Learning navigation agents improves their performance on both the\nRoom-to-Room (R2R) and Room-for-Room (R4R) datasets. The R4R results in\nparticular highlight the superiority of SDTW over previous success-constrained\nmetrics.",
            "journal ref": "Thirty-third Conference on Neural Information Processing Systems\n  (NeurIPS 2019)",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1907.05572v1",
            "date": "2019-07-12 04:01:57+00:00",
            "title": "R-Transformer: Recurrent Neural Network Enhanced Transformer",
            "authors": [
                "Zhiwei Wang",
                "Yao Ma",
                "Zitao Liu",
                "Jiliang Tang"
            ],
            "abstract": "Recurrent Neural Networks have long been the dominating choice for sequence\nmodeling. However, it severely suffers from two issues: impotent in capturing\nvery long-term dependencies and unable to parallelize the sequential\ncomputation procedure. Therefore, many non-recurrent sequence models that are\nbuilt on convolution and attention operations have been proposed recently.\nNotably, models with multi-head attention such as Transformer have demonstrated\nextreme effectiveness in capturing long-term dependencies in a variety of\nsequence modeling tasks. Despite their success, however, these models lack\nnecessary components to model local structures in sequences and heavily rely on\nposition embeddings that have limited effects and require a considerable amount\nof design efforts. In this paper, we propose the R-Transformer which enjoys the\nadvantages of both RNNs and the multi-head attention mechanism while avoids\ntheir respective drawbacks. The proposed model can effectively capture both\nlocal structures and global long-term dependencies in sequences without any use\nof position embeddings. We evaluate R-Transformer through extensive experiments\nwith data from a wide range of domains and the empirical results show that\nR-Transformer outperforms the state-of-the-art methods by a large margin in\nmost of the tasks. We have made the code publicly available at\n\\url{https://github.com/DSE-MSU/R-transformer}.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.05576v1",
            "date": "2019-07-12 04:50:33+00:00",
            "title": "Neural News Recommendation with Attentive Multi-View Learning",
            "authors": [
                "Chuhan Wu",
                "Fangzhao Wu",
                "Mingxiao An",
                "Jianqiang Huang",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "Personalized news recommendation is very important for online news platforms\nto help users find interested news and improve user experience. News and user\nrepresentation learning is critical for news recommendation. Existing news\nrecommendation methods usually learn these representations based on single news\ninformation, e.g., title, which may be insufficient. In this paper we propose a\nneural news recommendation approach which can learn informative representations\nof users and news by exploiting different kinds of news information. The core\nof our approach is a news encoder and a user encoder. In the news encoder we\npropose an attentive multi-view learning model to learn unified news\nrepresentations from titles, bodies and topic categories by regarding them as\ndifferent views of news. In addition, we apply both word-level and view-level\nattention mechanism to news encoder to select important words and views for\nlearning informative news representations. In the user encoder we learn the\nrepresentations of users based on their browsed news and apply attention\nmechanism to select informative news for user representation learning.\nExtensive experiments on a real-world dataset show our approach can effectively\nimprove the performance of news recommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05599v1",
            "date": "2019-07-12 07:37:00+00:00",
            "title": "Effective Incorporation of Speaker Information in Utterance Encoding in Dialog",
            "authors": [
                "Tianyu Zhao",
                "Tatsuya Kawahara"
            ],
            "abstract": "In dialog studies, we often encode a dialog using a hierarchical encoder\nwhere each utterance is converted into an utterance vector, and then a sequence\nof utterance vectors is converted into a dialog vector. Since knowing who\nproduced which utterance is essential to understanding a dialog, conventional\nmethods tried integrating speaker labels into utterance vectors. We found the\nmethod problematic in some cases where speaker annotations are inconsistent\namong different dialogs. A relative speaker modeling method is proposed to\naddress the problem. Experimental evaluations on dialog act recognition and\nresponse generation show that the proposed method yields superior and more\nconsistent performances.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.05698v1",
            "date": "2019-07-09 08:26:25+00:00",
            "title": "Teach an all-rounder with experts in different domains",
            "authors": [
                "Zhao You",
                "Dan Su",
                "Dong Yu"
            ],
            "abstract": "In many automatic speech recognition (ASR) tasks, an ideal model has to be\napplicable over multiple domains. In this paper, we propose to teach an\nall-rounder with experts in different domains. Concretely, we build a\nmulti-domain acoustic model by applying the teacher-student training framework.\nFirst, for each domain, a teacher model (domain-dependent model) is trained by\nfine-tuning a multi-condition model with domain-specific subset. Then all these\nteacher models are used to teach one single student model simultaneously. We\nperform experiments on two predefined domain setups. One is domains with\ndifferent speaking styles, the other is nearfield, far-field and far-field with\nnoise. Moreover, two types of models are examined: deep feedforward sequential\nmemory network (DFSMN) and long short term memory (LSTM). Experimental results\nshow that the model trained with this framework outperforms not only\nmulti-condition model but also domain-dependent model. Specially, our training\nmethod provides up to 10.4% relative character error rate improvement over\nbaseline model (multi-condition model).",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.05790v1",
            "date": "2019-07-06 08:29:21+00:00",
            "title": "Qwant Research @DEFT 2019: Document matching and information retrieval using clinical cases",
            "authors": [
                "Estelle Maudet",
                "Oralie Cattan",
                "Maureen de Seyssel",
                "Christophe Servan"
            ],
            "abstract": "This paper reports on Qwant Research contribution to tasks 2 and 3 of the\nDEFT 2019's challenge, focusing on French clinical cases analysis. Task 2 is a\ntask on semantic similarity between clinical cases and discussions. For this\ntask, we propose an approach based on language models and evaluate the impact\non the results of different preprocessings and matching techniques. For task 3,\nwe have developed an information extraction system yielding very encouraging\nresults accuracy-wise. We have experimented two different approaches, one based\non the exclusive use of neural networks, the other based on a linguistic\nanalysis.",
            "journal ref": "DEFT 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.05792v1",
            "date": "2019-07-11 15:55:24+00:00",
            "title": "Knowledge-incorporating ESIM models for Response Selection in Retrieval-based Dialog Systems",
            "authors": [
                "Jatin Ganhotra",
                "Siva Sankalp Patel",
                "Kshitij Fadnis"
            ],
            "abstract": "Goal-oriented dialog systems, which can be trained end-to-end without\nmanually encoding domain-specific features, show tremendous promise in the\ncustomer support use-case e.g. flight booking, hotel reservation, technical\nsupport, student advising etc. These dialog systems must learn to interact with\nexternal domain knowledge to achieve the desired goal e.g. recommending courses\nto a student, booking a table at a restaurant etc. This paper presents extended\nEnhanced Sequential Inference Model (ESIM) models: a) K-ESIM (Knowledge-ESIM),\nwhich incorporates the external domain knowledge and b) T-ESIM (Targeted-ESIM),\nwhich leverages information from similar conversations to improve the\nprediction accuracy. Our proposed models and the baseline ESIM model are\nevaluated on the Ubuntu and Advising datasets in the Sentence Selection track\nof the latest Dialog System Technology Challenge (DSTC7), where the goal is to\nfind the correct next utterance, given a partial conversation, from a set of\ncandidates. Our preliminary results suggest that incorporating external\nknowledge sources and leveraging information from similar dialogs leads to\nperformance improvements for predicting the next utterance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06111v1",
            "date": "2019-07-13 17:52:17+00:00",
            "title": "Speaker Recognition with Random Digit Strings Using Uncertainty Normalized HMM-based i-vectors",
            "authors": [
                "Nooshin Maghsoodi",
                "Hossein Sameti",
                "Hossein Zeinali",
                "Themos~Stafylakis"
            ],
            "abstract": "In this paper, we combine Hidden Markov Models (HMMs) with i-vector\nextractors to address the problem of text-dependent speaker recognition with\nrandom digit strings. We employ digit-specific HMMs to segment the utterances\ninto digits, to perform frame alignment to HMM states and to extract Baum-Welch\nstatistics. By making use of the natural partition of input features into\ndigits, we train digit-specific i-vector extractors on top of each HMM and we\nextract well-localized i-vectors, each modelling merely the phonetic content\ncorresponding to a single digit. We then examine ways to perform channel and\nuncertainty compensation, and we propose a novel method for using the\nuncertainty in the i-vector estimates. The experiments on RSR2015 part III show\nthat the proposed method attains 1.52\\% and 1.77\\% Equal Error Rate (EER) for\nmale and female respectively, outperforming state-of-the-art methods such as\nx-vectors, trained on vast amounts of data. Furthermore, these results are\nattained by a single system trained entirely on RSR2015, and by a simple\nscore-normalized cosine distance. Moreover, we show that the omission of\nchannel compensation yields only a minor degradation in performance, meaning\nthat the system attains state-of-the-art results even without recordings from\nmultiple handsets per speaker for training or enrolment. Similar conclusions\nare drawn from our experiments on the RedDots corpus, where the same method is\nevaluated on phrases. Finally, we report results with bottleneck features and\nshow that further improvement is attained when fusing them with spectral\nfeatures.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.06112v1",
            "date": "2019-07-13 17:57:55+00:00",
            "title": "BUT VOiCES 2019 System Description",
            "authors": [
                "Hossein Zeinali",
                "Pavel Mat\u011bjka",
                "Ladislav Mo\u0161ner",
                "Old\u0159ich Plchot",
                "Anna Silnova",
                "Ond\u0159ej Novotn\u00fd",
                "J\u00e1n Profant",
                "Ond\u0159ej Glembek",
                "Luk\u00e1\u0161 Burget"
            ],
            "abstract": "This is a description of our effort in VOiCES 2019 Speaker Recognition\nchallenge. All systems in the fixed condition are based on the x-vector\nparadigm with different features and DNN topologies. The single best system\nreaches 1.2% EER and a fusion of 3 systems yields 1.0% EER, which is 15%\nrelative improvement. The open condition allowed us to use external data which\nwe did for the PLDA adaptation and achieved less than ~10% relative\nimprovement. In the submission to open condition, we used 3 x-vector systems\nand also one i-vector based system.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.06226v5",
            "date": "2019-07-14 14:19:22+00:00",
            "title": "Lexical Simplification with Pretrained Encoders",
            "authors": [
                "Jipeng Qiang",
                "Yun Li",
                "Yi Zhu",
                "Yunhao Yuan",
                "Xindong Wu"
            ],
            "abstract": "Lexical simplification (LS) aims to replace complex words in a given sentence\nwith their simpler alternatives of equivalent meaning. Recently unsupervised\nlexical simplification approaches only rely on the complex word itself\nregardless of the given sentence to generate candidate substitutions, which\nwill inevitably produce a large number of spurious candidates. We present a\nsimple LS approach that makes use of the Bidirectional Encoder Representations\nfrom Transformers (BERT) which can consider both the given sentence and the\ncomplex word during generating candidate substitutions for the complex word.\nSpecifically, we mask the complex word of the original sentence for feeding\ninto the BERT to predict the masked token. The predicted results will be used\nas candidate substitutions. Despite being entirely unsupervised, experimental\nresults show that our approach obtains obvious improvement compared with these\nbaselines leveraging linguistic databases and parallel corpus, outperforming\nthe state-of-the-art by more than 12 Accuracy points on three well-known\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06330v1",
            "date": "2019-07-15 04:48:34+00:00",
            "title": "Ranking sentences from product description & bullets for better search",
            "authors": [
                "Prateek Verma",
                "Aliasgar Kutiyanawala",
                "Ke Shen"
            ],
            "abstract": "Products in an ecommerce catalog contain information-rich fields like\ndescription and bullets that can be useful to extract entities (attributes)\nusing NER based systems. However, these fields are often verbose and contain\nlot of information that is not relevant from a search perspective. Treating\neach sentence within these fields equally can lead to poor full text match and\nintroduce problems in extracting attributes to develop ontologies, semantic\nsearch etc. To address this issue, we describe two methods based on extractive\nsummarization with reinforcement learning by leveraging information in product\ntitles and search click through logs to rank sentences from bullets,\ndescription, etc. Finally, we compare the accuracy of these two models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.06342v1",
            "date": "2019-07-15 06:30:15+00:00",
            "title": "Joint Language Identification of Code-Switching Speech using Attention based E2E Network",
            "authors": [
                "Sreeram Ganji",
                "Kunal Dhawan",
                "Kumar Priyadarshi",
                "Rohit Sinha"
            ],
            "abstract": "Language identification (LID) has relevance in many speech processing\napplications. For the automatic recognition of code-switching speech, the\nconventional approaches often employ an LID system for detecting the languages\npresent within an utterance. In the existing works, the LID on code-switching\nspeech involves modelling of the underlying languages separately. In this work,\nwe propose a joint modelling based LID system for code-switching speech. To\nachieve the same, an attention-based end-to-end (E2E) network has been\nexplored. For the development and evaluation of the proposed approach, a\nrecently created Hindi-English code-switching corpus has been used. For the\ncontrast purpose, an LID system employing the connectionist temporal\nclassification-based E2E network is also developed. On comparing both the LID\nsystems, the attention based approach is noted to result in better LID\naccuracy. The effective location of code-switching boundaries within the\nutterance by the proposed approach has been demonstrated by plotting the\nattention weights of E2E network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06458v3",
            "date": "2019-07-15 12:10:24+00:00",
            "title": "RaKUn: Rank-based Keyword extraction via Unsupervised learning and Meta vertex aggregation",
            "authors": [
                "Bla\u017e \u0160krlj",
                "Andra\u017e Repar",
                "Senja Pollak"
            ],
            "abstract": "Keyword extraction is used for summarizing the content of a document and\nsupports efficient document retrieval, and is as such an indispensable part of\nmodern text-based systems. We explore how load centrality, a graph-theoretic\nmeasure applied to graphs derived from a given text can be used to efficiently\nidentify and rank keywords. Introducing meta vertices (aggregates of existing\nvertices) and systematic redundancy filters, the proposed method performs on\npar with state-of-the-art for the keyword extraction task on 14 diverse\ndatasets. The proposed method is unsupervised, interpretable and can also be\nused for document visualization.",
            "journal ref": "Statistical Language and Speech Processing 2019 Proceedings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06554v1",
            "date": "2019-07-15 15:45:37+00:00",
            "title": "Asking Clarifying Questions in Open-Domain Information-Seeking Conversations",
            "authors": [
                "Mohammad Aliannejadi",
                "Hamed Zamani",
                "Fabio Crestani",
                "W. Bruce Croft"
            ],
            "abstract": "Users often fail to formulate their complex information needs in a single\nquery. As a consequence, they may need to scan multiple result pages or\nreformulate their queries, which may be a frustrating experience.\nAlternatively, systems can improve user satisfaction by proactively asking\nquestions of the users to clarify their information needs. Asking clarifying\nquestions is especially important in conversational systems since they can only\nreturn a limited number of (often only one) result(s). In this paper, we\nformulate the task of asking clarifying questions in open-domain\ninformation-seeking conversational systems. To this end, we propose an offline\nevaluation methodology for the task and collect a dataset, called Qulac,\nthrough crowdsourcing. Our dataset is built on top of the TREC Web Track\n2009-2012 data and consists of over 10K question-answer pairs for 198 TREC\ntopics with 762 facets. Our experiments on an oracle model demonstrate that\nasking only one good question leads to over 170% retrieval performance\nimprovement in terms of P@1, which clearly demonstrates the potential impact of\nthe task. We further propose a retrieval framework consisting of three\ncomponents: question retrieval, question selection, and document retrieval. In\nparticular, our question selection model takes into account the original query\nand previous question-answer interactions while selecting the next question.\nOur model significantly outperforms competitive baselines. To foster research\nin this area, we have made Qulac publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06679v2",
            "date": "2019-07-15 18:17:13+00:00",
            "title": "Towards Near-imperceptible Steganographic Text",
            "authors": [
                "Falcon Z. Dai",
                "Zheng Cai"
            ],
            "abstract": "We show that the imperceptibility of several existing linguistic\nsteganographic systems (Fang et al., 2017; Yang et al., 2018) relies on\nimplicit assumptions on statistical behaviors of fluent text. We formally\nanalyze them and empirically evaluate these assumptions. Furthermore, based on\nthese observations, we propose an encoding algorithm called patient-Huffman\nwith improved near-imperceptible guarantees.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06745v1",
            "date": "2019-07-15 20:43:53+00:00",
            "title": "Low-supervision urgency detection and transfer in short crisis messages",
            "authors": [
                "Mayank Kejriwal",
                "Peilin Zhou"
            ],
            "abstract": "Humanitarian disasters have been on the rise in recent years due to the\neffects of climate change and socio-political situations such as the refugee\ncrisis. Technology can be used to best mobilize resources such as food and\nwater in the event of a natural disaster, by semi-automatically flagging tweets\nand short messages as indicating an urgent need. The problem is challenging not\njust because of the sparseness of data in the immediate aftermath of a\ndisaster, but because of the varying characteristics of disasters in developing\ncountries (making it difficult to train just one system) and the noise and\nquirks in social media. In this paper, we present a robust, low-supervision\nsocial media urgency system that adapts to arbitrary crises by leveraging both\nlabeled and unlabeled data in an ensemble setting. The system is also able to\nadapt to new crises where an unlabeled background corpus may not be available\nyet by utilizing a simple and effective transfer learning methodology.\nExperimentally, our transfer learning and low-supervision approaches are found\nto outperform viable baselines with high significance on myriad disaster\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.06773v2",
            "date": "2019-07-15 22:16:00+00:00",
            "title": "Logic Conditionals, Supervenience, and Selection Tasks",
            "authors": [
                "Giovanni Sileno"
            ],
            "abstract": "Principles of cognitive economy would require that concepts about objects,\nproperties and relations should be introduced only if they simplify the\nconceptualisation of a domain. Unexpectedly, classic logic conditionals,\nspecifying structures holding within elements of a formal conceptualisation, do\nnot always satisfy this crucial principle. The paper argues that this\nrequirement is captured by supervenience, hereby further identified as a\nproperty necessary for compression. The resulting theory suggests an\nalternative explanation of the empirical experiences observable in Wason's\nselection tasks, associating human performance with conditionals on the ability\nof dealing with compression, rather than with logic necessity.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.07322v1",
            "date": "2019-07-16 15:32:04+00:00",
            "title": "MedCATTrainer: A Biomedical Free Text Annotation Interface with Active Learning and Research Use Case Specific Customisation",
            "authors": [
                "Thomas Searle",
                "Zeljko Kraljevic",
                "Rebecca Bendayan",
                "Daniel Bean",
                "Richard Dobson"
            ],
            "abstract": "We present MedCATTrainer an interface for building, improving and customising\na given Named Entity Recognition and Linking (NER+L) model for biomedical\ndomain text. NER+L is often used as a first step in deriving value from\nclinical text. Collecting labelled data for training models is difficult due to\nthe need for specialist domain knowledge. MedCATTrainer offers an interactive\nweb-interface to inspect and improve recognised entities from an underlying\nNER+L model via active learning. Secondary use of data for clinical research\noften has task and context specific criteria. MedCATTrainer provides a further\ninterface to define and collect supervised learning training data for\nresearcher specific use cases. Initial results suggest our approach allows for\nefficient and accurate collection of research use case specific training data.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1907.07323v1",
            "date": "2019-07-16 16:14:09+00:00",
            "title": "STRASS: A Light and Effective Method for Extractive Summarization Based on Sentence Embeddings",
            "authors": [
                "L\u00e9o Bouscarrat",
                "Antoine Bonnefoy",
                "Thomas Peel",
                "C\u00e9cile Pereira"
            ],
            "abstract": "This paper introduces STRASS: Summarization by TRAnsformation Selection and\nScoring. It is an extractive text summarization method which leverages the\nsemantic information in existing sentence embedding spaces. Our method creates\nan extractive summary by selecting the sentences with the closest embeddings to\nthe document embedding. The model learns a transformation of the document\nembedding to minimize the similarity between the extractive summary and the\nground truth summary. As the transformation is only composed of a dense layer,\nthe training can be done on CPU, therefore, inexpensive. Moreover, inference\ntime is short and linear according to the number of sentences. As a second\ncontribution, we introduce the French CASS dataset, composed of judgments from\nthe French Court of cassation and their corresponding summaries. On this\ndataset, our results show that our method performs similarly to the state of\nthe art extractive methods with effective training and inferring time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07626v3",
            "date": "2019-07-16 03:55:21+00:00",
            "title": "AP19-OLR Challenge: Three Tasks and Their Baselines",
            "authors": [
                "Zhiyuan Tang",
                "Dong Wang",
                "Liming Song"
            ],
            "abstract": "This paper introduces the fourth oriental language recognition (OLR)\nchallenge AP19-OLR, including the data profile, the tasks and the evaluation\nprinciples. The OLR challenge has been held successfully for three consecutive\nyears, along with APSIPA Annual Summit and Conference (APSIPA ASC). The\nchallenge this year still focuses on practical and challenging tasks, precisely\n(1) short-utterance LID, (2) cross-channel LID and (3) zero-resource LID. The\nevent this year includes more languages and more real-life data provided by\nSpeechOcean and the NSFC M2ASR project. All the data is free for participants.\nRecipes for x-vector system and back-end evaluation are also conducted as\nbaselines for the three tasks. The participants can refer to these\nonline-published recipes to deploy LID systems for convenience. We report the\nbaseline results on the three tasks and demonstrate that the three tasks are\nworth some efforts to achieve better performance.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.07672v3",
            "date": "2019-07-17 23:40:46+00:00",
            "title": "Analysis of Word Embeddings Using Fuzzy Clustering",
            "authors": [
                "Shahin Atakishiyev",
                "Marek Z. Reformat"
            ],
            "abstract": "In data dominated systems and applications, a concept of representing words\nin a numerical format has gained a lot of attention. There are a few approaches\nused to generate such a representation. An interesting issue that should be\nconsidered is the ability of such representations - called embeddings - to\nimitate human-based semantic similarity between words. In this study, we\nperform a fuzzy-based analysis of vector representations of words, i.e., word\nembeddings. We use two popular fuzzy clustering algorithms on count-based word\nembeddings, known as GloVe, of different dimensionality. Words from\nWordSim-353, called the gold standard, are represented as vectors and\nclustered. The results indicate that fuzzy clustering algorithms are very\nsensitive to high-dimensional data, and parameter tuning can dramatically\nchange their performance. We show that by adjusting the value of the fuzzifier\nparameter, fuzzy clustering can be successfully applied to vectors of high - up\nto one hundred - dimensions. Additionally, we illustrate that fuzzy clustering\nallows to provide interesting results regarding membership of words to\ndifferent clusters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.07757v1",
            "date": "2019-07-08 18:29:58+00:00",
            "title": "XFake: Explainable Fake News Detector with Visualizations",
            "authors": [
                "Fan Yang",
                "Shiva K. Pentyala",
                "Sina Mohseni",
                "Mengnan Du",
                "Hao Yuan",
                "Rhema Linder",
                "Eric D. Ragan",
                "Shuiwang Ji",
                "Xia Hu"
            ],
            "abstract": "In this demo paper, we present the XFake system, an explainable fake news\ndetector that assists end-users to identify news credibility. To effectively\ndetect and interpret the fakeness of news items, we jointly consider both\nattributes (e.g., speaker) and statements. Specifically, MIMIC, ATTN and PERT\nframeworks are designed, where MIMIC is built for attribute analysis, ATTN is\nfor statement semantic analysis and PERT is for statement linguistic analysis.\nBeyond the explanations extracted from the designed frameworks, relevant\nsupporting examples as well as visualization are further provided to facilitate\nthe interpretation. Our implemented system is demonstrated on a real-world\ndataset crawled from PolitiFact, where thousands of verified political news\nhave been collected.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1907.07972v1",
            "date": "2019-07-18 10:36:03+00:00",
            "title": "Deep Neural Models for Medical Concept Normalization in User-Generated Texts",
            "authors": [
                "Zulfat Miftahutdinov",
                "Elena Tutubalina"
            ],
            "abstract": "In this work, we consider the medical concept normalization problem, i.e.,\nthe problem of mapping a health-related entity mention in a free-form text to a\nconcept in a controlled vocabulary, usually to the standard thesaurus in the\nUnified Medical Language System (UMLS). This is a challenging task since\nmedical terminology is very different when coming from health care\nprofessionals or from the general public in the form of social media texts. We\napproach it as a sequence learning problem with powerful neural networks such\nas recurrent neural networks and contextualized word representation models\ntrained to obtain semantic representations of social media expressions. Our\nexperimental evaluation over three different benchmarks shows that neural\narchitectures leverage the semantic meaning of the entity mention and\nsignificantly outperform an existing state of the art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08259v1",
            "date": "2019-07-18 19:29:29+00:00",
            "title": "WriterForcing: Generating more interesting story endings",
            "authors": [
                "Prakhar Gupta",
                "Vinayshekhar Bannihatti Kumar",
                "Mukul Bhutani",
                "Alan W Black"
            ],
            "abstract": "We study the problem of generating interesting endings for stories. Neural\ngenerative models have shown promising results for various text generation\nproblems. Sequence to Sequence (Seq2Seq) models are typically trained to\ngenerate a single output sequence for a given input sequence. However, in the\ncontext of a story, multiple endings are possible. Seq2Seq models tend to\nignore the context and generate generic and dull responses. Very few works have\nstudied generating diverse and interesting story endings for a given story\ncontext. In this paper, we propose models which generate more diverse and\ninteresting outputs by 1) training models to focus attention on important\nkeyphrases of the story, and 2) promoting generation of non-generic words. We\nshow that the combination of the two leads to more diverse and interesting\nendings.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.08293v1",
            "date": "2019-07-15 06:34:28+00:00",
            "title": "Investigating Target Set Reduction for End-to-End Speech Recognition of Hindi-English Code-Switching Data",
            "authors": [
                "Kunal Dhawan",
                "Ganji Sreeram",
                "Kumar Priyadarshi",
                "Rohit Sinha"
            ],
            "abstract": "End-to-end (E2E) systems are fast replacing the conventional systems in the\ndomain of automatic speech recognition. As the target labels are learned\ndirectly from speech data, the E2E systems need a bigger corpus for effective\ntraining. In the context of code-switching task, the E2E systems face two\nchallenges: (i) the expansion of the target set due to multiple languages\ninvolved, and (ii) the lack of availability of sufficiently large\ndomain-specific corpus. Towards addressing those challenges, we propose an\napproach for reducing the number of target labels for reliable training of the\nE2E systems on limited data. The efficacy of the proposed approach has been\ndemonstrated on two prominent architectures, namely CTC-based and\nattention-based E2E networks. The experimental validations are performed on a\nrecently created Hindi-English code-switching corpus. For contrast purpose, the\nresults for the full target set based E2E system and a hybrid DNN-HMM system\nare also reported.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.08321v3",
            "date": "2019-07-18 23:48:21+00:00",
            "title": "SentiMATE: Learning to play Chess through Natural Language Processing",
            "authors": [
                "Isaac Kamlish",
                "Isaac Bentata Chocron",
                "Nicholas McCarthy"
            ],
            "abstract": "We present SentiMATE, a novel end-to-end Deep Learning model for Chess,\nemploying Natural Language Processing that aims to learn an effective\nevaluation function assessing move quality. This function is pre-trained on the\nsentiment of commentary associated with the training moves and is used to guide\nand optimize the agent's game-playing decision making. The contributions of\nthis research are three-fold: we build and put forward both a classifier which\nextracts commentary describing the quality of Chess moves in vast commentary\ndatasets, and a Sentiment Analysis model trained on Chess commentary to\naccurately predict the quality of said moves, to then use those predictions to\nevaluate the optimal next move of a Chess agent. Both classifiers achieve over\n90 % classification accuracy. Lastly, we present a Chess engine, SentiMATE,\nwhich evaluates Chess moves based on a pre-trained sentiment evaluation\nfunction. Our results exhibit strong evidence to support our initial hypothesis\n- \"Can Natural Language Processing be used to train a novel and sample\nefficient evaluation function in Chess Engines?\" - as we integrate our\nevaluation function into modern Chess engines and play against agents with\ntraditional Chess move evaluation functions, beating both random agents and a\nDeepChess implementation at a level-one search depth - representing the number\nof moves a traditional Chess agent (employing the alpha-beta search algorithm)\nlooks ahead in order to evaluate a given chess state.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.08696v1",
            "date": "2019-07-15 21:48:28+00:00",
            "title": "Multi-modal Sentiment Analysis using Deep Canonical Correlation Analysis",
            "authors": [
                "Zhongkai Sun",
                "Prathusha K Sarma",
                "William Sethares",
                "Erik P. Bucy"
            ],
            "abstract": "This paper learns multi-modal embeddings from text, audio, and video\nviews/modes of data in order to improve upon down-stream sentiment\nclassification. The experimental framework also allows investigation of the\nrelative contributions of the individual views in the final multi-modal\nembedding. Individual features derived from the three views are combined into a\nmulti-modal embedding using Deep Canonical Correlation Analysis (DCCA) in two\nways i) One-Step DCCA and ii) Two-Step DCCA. This paper learns text embeddings\nusing BERT, the current state-of-the-art in text encoders. We posit that this\nhighly optimized algorithm dominates over the contribution of other views,\nthough each view does contribute to the final result. Classification tasks are\ncarried out on two benchmark datasets and on a new Debate Emotion data set, and\ntogether these demonstrate that the one-Step DCCA outperforms the current\nstate-of-the-art in learning multi-modal embeddings.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1907.08736v4",
            "date": "2019-07-20 02:07:02+00:00",
            "title": "ER-AE: Differentially Private Text Generation for Authorship Anonymization",
            "authors": [
                "Haohan Bo",
                "Steven H. H. Ding",
                "Benjamin C. M. Fung",
                "Farkhund Iqbal"
            ],
            "abstract": "Most of privacy protection studies for textual data focus on removing\nexplicit sensitive identifiers. However, personal writing style, as a strong\nindicator of the authorship, is often neglected. Recent studies, such as SynTF,\nhave shown promising results on privacy-preserving text mining. However, their\nanonymization algorithm can only output numeric term vectors which are\ndifficult for the recipients to interpret. We propose a novel text generation\nmodel with a two-set exponential mechanism for authorship anonymization. By\naugmenting the semantic information through a REINFORCE training reward\nfunction, the model can generate differentially private text that has a close\nsemantic and similar grammatical structure to the original text while removing\npersonal traits of the writing style. It does not assume any conditioned labels\nor paralleled text data for training. We evaluate the performance of the\nproposed model on the real-life peer reviews dataset and the Yelp review\ndataset. The result suggests that our model outperforms the state-of-the-art on\nsemantic preservation, authorship obfuscation, and stylometric transformation.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1907.08922v1",
            "date": "2019-07-21 06:58:22+00:00",
            "title": "Using Word Embeddings to Examine Gender Bias in Dutch Newspapers, 1950-1990",
            "authors": [
                "Melvin Wevers"
            ],
            "abstract": "Contemporary debates on filter bubbles and polarization in public and social\nmedia raise the question to what extent news media of the past exhibited\nbiases. This paper specifically examines bias related to gender in six Dutch\nnational newspapers between 1950 and 1990. We measure bias related to gender by\ncomparing local changes in word embedding models trained on newspapers with\ndivergent ideological backgrounds. We demonstrate clear differences in gender\nbias and changes within and between newspapers over time. In relation to themes\nsuch as sexuality and leisure, we see the bias moving toward women, whereas,\ngenerally, the bias shifts in the direction of men, despite growing female\nemployment number and feminist movements. Even though Dutch society became less\nstratified ideologically (depillarization), we found an increasing divergence\nin gender bias between religious and social-democratic on the one hand and\nliberal newspapers on the other. Methodologically, this paper illustrates how\nword embeddings can be used to examine historical language change. Future work\nwill investigate how fine-tuning deep contextualized embedding models, such as\nELMO, might be used for similar tasks with greater contextual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.08937v1",
            "date": "2019-07-21 09:22:50+00:00",
            "title": "Quantifying Similarity between Relations with Fact Distribution",
            "authors": [
                "Weize Chen",
                "Hao Zhu",
                "Xu Han",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "We introduce a conceptually simple and effective method to quantify the\nsimilarity between relations in knowledge bases. Specifically, our approach is\nbased on the divergence between the conditional probability distributions over\nentity pairs. In this paper, these distributions are parameterized by a very\nsimple neural network. Although computing the exact similarity is in-tractable,\nwe provide a sampling-based method to get a good approximation. We empirically\nshow the outputs of our approach significantly correlate with human judgments.\nBy applying our method to various tasks, we also find that (1) our approach\ncould effectively detect redundant relations extracted by open information\nextraction (Open IE) models, that (2) even the most competitive models for\nrelational classification still make mistakes among very similar relations, and\nthat (3) our approach could be incorporated into negative sampling and softmax\nclassification to alleviate these mistakes. The source code and experiment\ndetails of this paper can be obtained from\nhttps://github.com/thunlp/relation-similarity.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.08971v2",
            "date": "2019-07-21 13:05:45+00:00",
            "title": "Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network",
            "authors": [
                "Martin Gleize",
                "Eyal Shnarch",
                "Leshem Choshen",
                "Lena Dankin",
                "Guy Moshkowich",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "With the advancement in argument detection, we suggest to pay more attention\nto the challenging task of identifying the more convincing arguments. Machines\ncapable of responding and interacting with humans in helpful ways have become\nubiquitous. We now expect them to discuss with us the more delicate questions\nin our world, and they should do so armed with effective arguments. But what\nmakes an argument more persuasive? What will convince you? In this paper, we\npresent a new data set, IBM-EviConv, of pairs of evidence labeled for\nconvincingness, designed to be more challenging than existing alternatives. We\nalso propose a Siamese neural network architecture shown to outperform several\nbaselines on both a prior convincingness data set and our own. Finally, we\nprovide insights into our experimental results and the various kinds of\nargumentative value our method is capable of detecting.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.09006v1",
            "date": "2019-07-18 12:24:30+00:00",
            "title": "Forward-Backward Decoding for Regularizing End-to-End TTS",
            "authors": [
                "Yibin Zheng",
                "Xi Wang",
                "Lei He",
                "Shifeng Pan",
                "Frank K. Soong",
                "Zhengqi Wen",
                "Jianhua Tao"
            ],
            "abstract": "Neural end-to-end TTS can generate very high-quality synthesized speech, and\neven close to human recording within similar domain text. However, it performs\nunsatisfactory when scaling it to challenging test sets. One concern is that\nthe encoder-decoder with attention-based network adopts autoregressive\ngenerative sequence model with the limitation of \"exposure bias\" To address\nthis issue, we propose two novel methods, which learn to predict future by\nimproving agreement between forward and backward decoding sequence. The first\none is achieved by introducing divergence regularization terms into model\ntraining objective to reduce the mismatch between two directional models,\nnamely L2R and R2L (which generates targets from left-to-right and\nright-to-left, respectively). While the second one operates on decoder-level\nand exploits the future information during decoding. In addition, we employ a\njoint training strategy to allow forward and backward decoding to improve each\nother in an interactive process. Experimental results show our proposed methods\nespecially the second one (bidirectional decoder regularization), leads a\nsignificantly improvement on both robustness and overall naturalness, as\noutperforming baseline (the revised version of Tacotron2) with a MOS gap of\n0.14 in a challenging test, and achieving close to human quality (4.42 vs. 4.49\nin MOS) on general test.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.09007v2",
            "date": "2019-07-21 17:07:07+00:00",
            "title": "A Comprehensive Analysis of Twitter Trending Topics",
            "authors": [
                "Issa Annamoradnejad",
                "Jafar Habibi"
            ],
            "abstract": "In Twitter, a name, phrase, or topic that is mentioned at a greater rate than\nothers is called a \"trending topic\" or simply \"trend\". Twitter trends list has\na powerful ability to promote public events such as natural events, political\nscandals, market changes and other types of breaking news. Nevertheless, there\nhave been very few works focused on the dynamics of these trending topics. In\nthis article, we thoroughly examined the Twitter's trending topics of 2018. To\nthis end, we automatically accessed Twitter's trends API and stored the\nresulting 50 top trending topics in a novel dataset. We propose and analyze our\ndataset according to six criteria: lexical analysis, time to reach, trend\nreoccurrence, trending time, tweets count, and language analysis. Based on our\nresults, 77.6% of the topics that reached the Top-10 list were trending with\nless than 100k tweets. More than 50% of the topics could not hold the position\nfor more than an hour. English and Arabic languages comprised close to 40% and\n20% of the first rank topics, respectively.",
            "journal ref": "2019 5th International Conference on Web Research (ICWR), Tehran,\n  Iran, 2019, pp. 22-27",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.09177v2",
            "date": "2019-07-22 08:22:08+00:00",
            "title": "Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection",
            "authors": [
                "David Ifeoluwa Adelani",
                "Haotian Mai",
                "Fuming Fang",
                "Huy H. Nguyen",
                "Junichi Yamagishi",
                "Isao Echizen"
            ],
            "abstract": "Advanced neural language models (NLMs) are widely used in sequence generation\ntasks because they are able to produce fluent and meaningful sentences. They\ncan also be used to generate fake reviews, which can then be used to attack\nonline review systems and influence the buying decisions of online shoppers. To\nperform such attacks, it is necessary for experts to train a tailored LM for a\nspecific topic. In this work, we show that a low-skilled threat model can be\nbuilt just by combining publicly available LMs and show that the produced fake\nreviews can fool both humans and machines. In particular, we use the GPT-2 NLM\nto generate a large number of high-quality reviews based on a review with the\ndesired sentiment and then using a BERT based text classifier (with accuracy of\n96%) to filter out reviews with undesired sentiments. Because none of the words\nin the review are modified, fluent samples like the training data can be\ngenerated from the learned distribution. A subjective evaluation with 80\nparticipants demonstrated that this simple method can produce reviews that are\nas fluent as those written by people. It also showed that the participants\ntended to distinguish fake reviews randomly. Three countermeasures, Grover,\nGLTR, and OpenAI GPT-2 detector, were found to be difficult to accurately\ndetect fake review.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09340v1",
            "date": "2019-07-22 14:33:43+00:00",
            "title": "VIFIDEL: Evaluating the Visual Fidelity of Image Descriptions",
            "authors": [
                "Pranava Madhyastha",
                "Josiah Wang",
                "Lucia Specia"
            ],
            "abstract": "We address the task of evaluating image description generation systems. We\npropose a novel image-aware metric for this task: VIFIDEL. It estimates the\nfaithfulness of a generated caption with respect to the content of the actual\nimage, based on the semantic similarity between labels of objects depicted in\nimages and words in the description. The metric is also able to take into\naccount the relative importance of objects mentioned in human reference\ndescriptions during evaluation. Even if these human reference descriptions are\nnot available, VIFIDEL can still reliably evaluate system descriptions. The\nmetric achieves high correlation with human judgments on two well-known\ndatasets and is competitive with metrics that depend on human references",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09358v2",
            "date": "2019-07-22 14:53:48+00:00",
            "title": "Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods",
            "authors": [
                "Aditya Mogadala",
                "Marimuthu Kalimuthu",
                "Dietrich Klakow"
            ],
            "abstract": "The interest in Artificial Intelligence (AI) and its applications has seen\nunprecedented growth in the last few years. This success can be partly\nattributed to the advancements made in the sub-fields of AI such as Machine\nLearning (ML), Computer Vision (CV), and Natural Language Processing (NLP). The\nlargest of the growths in these fields has been made possible with deep\nlearning, a sub-area of machine learning, which uses the principles of\nartificial neural networks. This has created significant interest in the\nintegration of vision and language. The tasks are designed such that they\nperfectly embrace the ideas of deep learning. In this survey, we focus on ten\nprominent tasks that integrate language and vision by discussing their problem\nformulations, methods, existing datasets, evaluation measures, and compare the\nresults obtained with corresponding state-of-the-art methods. Our efforts go\nbeyond earlier surveys which are either task-specific or concentrate only on\none type of visual content, i.e., image or video. Furthermore, we also provide\nsome potential future directions in this field of research with an anticipation\nthat this survey brings in innovative thoughts and ideas to address the\nexisting challenges and build new applications.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.09361v1",
            "date": "2019-07-22 14:57:13+00:00",
            "title": "Introduction to Neural Network based Approaches for Question Answering over Knowledge Graphs",
            "authors": [
                "Nilesh Chakraborty",
                "Denis Lukovnikov",
                "Gaurav Maheshwari",
                "Priyansh Trivedi",
                "Jens Lehmann",
                "Asja Fischer"
            ],
            "abstract": "Question answering has emerged as an intuitive way of querying structured\ndata sources, and has attracted significant advancements over the years. In\nthis article, we provide an overview over these recent advancements, focusing\non neural network based question answering systems over knowledge graphs. We\nintroduce readers to the challenges in the tasks, current paradigms of\napproaches, discuss notable advancements, and outline the emerging trends in\nthe field. Through this article, we aim to provide newcomers to the field with\na suitable entry point, and ease their process of making informed decisions\nwhile creating their own QA system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09369v1",
            "date": "2019-07-22 15:33:53+00:00",
            "title": "Emotion Detection in Text: Focusing on Latent Representation",
            "authors": [
                "Armin Seyeditabari",
                "Narges Tabari",
                "Shafie Gholizadeh",
                "Wlodek Zadrozny"
            ],
            "abstract": "In recent years, emotion detection in text has become more popular due to its\nvast potential applications in marketing, political science, psychology,\nhuman-computer interaction, artificial intelligence, etc. In this work, we\nargue that current methods which are based on conventional machine learning\nmodels cannot grasp the intricacy of emotional language by ignoring the\nsequential nature of the text, and the context. These methods, therefore, are\nnot sufficient to create an applicable and generalizable emotion detection\nmethodology. Understanding these limitations, we present a new network based on\na bidirectional GRU model to show that capturing more meaningful information\nfrom text can significantly improve the performance of these models. The\nresults show significant improvement with an average of 26.8 point increase in\nF-measure on our test data and 38.6 increase on the totally new dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09527v1",
            "date": "2019-07-22 18:57:14+00:00",
            "title": "Maximizing Stylistic Control and Semantic Accuracy in NLG: Personality Variation and Discourse Contrast",
            "authors": [
                "Vrindavan Harrison",
                "Lena Reed",
                "Shereen Oraby",
                "Marilyn Walker"
            ],
            "abstract": "Neural generation methods for task-oriented dialogue typically generate from\na meaning representation that is populated using a database of domain\ninformation, such as a table of data describing a restaurant. While earlier\nwork focused solely on the semantic fidelity of outputs, recent work has\nstarted to explore methods for controlling the style of the generated text\nwhile simultaneously achieving semantic accuracy. Here we experiment with two\nstylistic benchmark tasks, generating language that exhibits variation in\npersonality, and generating discourse contrast. We report a huge performance\nimprovement in both stylistic control and semantic accuracy over the state of\nthe art on both of these benchmarks. We test several different models and show\nthat putting stylistic conditioning in the decoder and eliminating the semantic\nre-ranker used in earlier models results in more than 15 points higher BLEU for\nPersonality, with a reduction of semantic error to near zero. We also report an\nimprovement from .75 to .81 in controlling contrast and a reduction in semantic\nerror from 16% to 2%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09600v2",
            "date": "2019-07-22 21:58:40+00:00",
            "title": "Evaluation of Embeddings of Laboratory Test Codes for Patients at a Cancer Center",
            "authors": [
                "Lorenzo A. Rossi",
                "Chad Shawber",
                "Janet Munu",
                "Finly Zachariah"
            ],
            "abstract": "Laboratory test results are an important and generally high dimensional\ncomponent of a patient's Electronic Health Record (EHR). We train embedding\nrepresentations (via Word2Vec and GloVe) for LOINC codes of laboratory tests\nfrom the EHRs of about 80,000 patients at a cancer center. To include\ninformation about lab test outcomes, we also train embeddings on the\nconcatenation of a LOINC code with a symbol indicating normality or abnormality\nof the result. We observe several clinically meaningful similarities among\nLOINC embeddings trained over our data. For the embeddings of the concatenation\nof LOINCs with abnormality codes, we evaluate the performance for mortality\nprediction tasks and the ability to preserve ordinality properties: i.e. a lab\ntest with normal outcome should be more similar to an abnormal one than to the\na very abnormal one.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.09636v4",
            "date": "2019-07-22 23:53:41+00:00",
            "title": "On Modeling ASR Word Confidence",
            "authors": [
                "Woojay Jeon",
                "Maxwell Jordan",
                "Mahesh Krishnamoorthy"
            ],
            "abstract": "We present a new method for computing ASR word confidences that effectively\nmitigates the effect of ASR errors for diverse downstream applications,\nimproves the word error rate of the 1-best result, and allows better comparison\nof scores across different models. We propose 1) a new method for modeling word\nconfidence using a Heterogeneous Word Confusion Network (HWCN) that addresses\nsome key flaws in conventional Word Confusion Networks, and 2) a new score\ncalibration method for facilitating direct comparison of scores from different\nmodels. Using a bidirectional lattice recurrent neural network to compute the\nconfidence scores of each word in the HWCN, we show that the word sequence with\nthe best overall confidence is more accurate than the default 1-best result of\nthe recognizer, and that the calibration method can substantially improve the\nreliability of recognizer combination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09671v1",
            "date": "2019-07-23 03:11:07+00:00",
            "title": "Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following",
            "authors": [
                "David Gaddy",
                "Dan Klein"
            ],
            "abstract": "We consider the problem of learning to map from natural language instructions\nto state transitions (actions) in a data-efficient manner. Our method takes\ninspiration from the idea that it should be easier to ground language to\nconcepts that have already been formed through pre-linguistic observation. We\naugment a baseline instruction-following learner with an initial\nenvironment-learning phase that uses observations of language-free state\ntransitions to induce a suitable latent representation of actions before\nprocessing the instruction-following training data. We show that mapping to\npre-learned representations substantially improves performance over systems\nwhose representations are learned from limited instructional data alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09692v1",
            "date": "2019-07-23 04:27:57+00:00",
            "title": "Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference",
            "authors": [
                "Boyuan Pan",
                "Yazheng Yang",
                "Zhou Zhao",
                "Yueting Zhuang",
                "Deng Cai",
                "Xiaofei He"
            ],
            "abstract": "Natural Language Inference (NLI), also known as Recognizing Textual\nEntailment (RTE), is one of the most important problems in natural language\nprocessing. It requires to infer the logical relationship between two given\nsentences. While current approaches mostly focus on the interaction\narchitectures of the sentences, in this paper, we propose to transfer knowledge\nfrom some important discourse markers to augment the quality of the NLI model.\nWe observe that people usually use some discourse markers such as \"so\" or \"but\"\nto represent the logical relationship between two sentences. These words\npotentially have deep connections with the meanings of the sentences, thus can\nbe utilized to help improve the representations of them. Moreover, we use\nreinforcement learning to optimize a new objective function with a reward\ndefined by the property of the NLI datasets to make full use of the labels\ninformation. Experiments show that our method achieves the state-of-the-art\nperformance on several large-scale datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09705v1",
            "date": "2019-07-23 05:55:28+00:00",
            "title": "2D-CTC for Scene Text Recognition",
            "authors": [
                "Zhaoyi Wan",
                "Fengming Xie",
                "Yibo Liu",
                "Xiang Bai",
                "Cong Yao"
            ],
            "abstract": "Scene text recognition has been an important, active research topic in\ncomputer vision for years. Previous approaches mainly consider text as 1D\nsignals and cast scene text recognition as a sequence prediction problem, by\nfeat of CTC or attention based encoder-decoder framework, which is originally\ndesigned for speech recognition. However, different from speech voices, which\nare 1D signals, text instances are essentially distributed in 2D image spaces.\nTo adhere to and make use of the 2D nature of text for higher recognition\naccuracy, we extend the vanilla CTC model to a second dimension, thus creating\n2D-CTC. 2D-CTC can adaptively concentrate on most relevant features while\nexcluding the impact from clutters and noises in the background; It can also\nnaturally handle text instances with various forms (horizontal, oriented and\ncurved) while giving more interpretable intermediate predictions. The\nexperiments on standard benchmarks for scene text recognition, such as IIIT-5K,\nICDAR 2015, SVP-Perspective, and CUTE80, demonstrate that the proposed 2D-CTC\nmodel outperforms state-of-the-art methods on the text of both regular and\nirregular shapes. Moreover, 2D-CTC exhibits its superiority over prior art on\ntraining and testing speed. Our implementation and models of 2D-CTC will be\nmade publicly available soon later.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.09748v1",
            "date": "2019-07-23 08:23:42+00:00",
            "title": "Position Focused Attention Network for Image-Text Matching",
            "authors": [
                "Yaxiong Wang",
                "Hao Yang",
                "Xueming Qian",
                "Lin Ma",
                "Jing Lu",
                "Biao Li",
                "Xin Fan"
            ],
            "abstract": "Image-text matching tasks have recently attracted a lot of attention in the\ncomputer vision field. The key point of this cross-domain problem is how to\naccurately measure the similarity between the visual and the textual contents,\nwhich demands a fine understanding of both modalities. In this paper, we\npropose a novel position focused attention network (PFAN) to investigate the\nrelation between the visual and the textual views. In this work, we integrate\nthe object position clue to enhance the visual-text joint-embedding learning.\nWe first split the images into blocks, by which we infer the relative position\nof region in the image. Then, an attention mechanism is proposed to model the\nrelations between the image region and blocks and generate the valuable\nposition feature, which will be further utilized to enhance the region\nexpression and model a more reliable relationship between the visual image and\nthe textual sentence. Experiments on the popular datasets Flickr30K and MS-COCO\nshow the effectiveness of the proposed method. Besides the public datasets, we\nalso conduct experiments on our collected practical large-scale news dataset\n(Tencent-News) to validate the practical application value of proposed method.\nAs far as we know, this is the first attempt to test the performance on the\npractical application. Our method achieves the state-of-art performance on all\nof these three datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.09854v1",
            "date": "2019-07-23 13:06:01+00:00",
            "title": "Overview and Results: CL-SciSumm Shared Task 2019",
            "authors": [
                "Muthu Kumar Chandrasekaran",
                "Michihiro Yasunaga",
                "Dragomir Radev",
                "Dayne Freitag",
                "Min-Yen Kan"
            ],
            "abstract": "The CL-SciSumm Shared Task is the first medium-scale shared task on\nscientific document summarization in the computational linguistics~(CL) domain.\nIn 2019, it comprised three tasks: (1A) identifying relationships between\nciting documents and the referred document, (1B) classifying the discourse\nfacets, and (2) generating the abstractive summary. The dataset comprised 40\nannotated sets of citing and reference papers of the CL-SciSumm 2018 corpus and\n1000 more from the SciSummNet dataset. All papers are from the open access\nresearch papers in the CL domain. This overview describes the participation and\nthe official results of the CL-SciSumm 2019 Shared Task, organized as a part of\nthe 42nd Annual Conference of the Special Interest Group in Information\nRetrieval (SIGIR), held in Paris, France in July 2019. We compare the\nparticipating systems in terms of two evaluation metrics and discuss the use of\nROUGE as an evaluation metric. The annotated dataset used for this shared task\nand the scripts used for evaluation can be accessed and used by the community\nat: https://github.com/WING-NUS/scisumm-corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10016v1",
            "date": "2019-07-23 17:20:13+00:00",
            "title": "Structured Fusion Networks for Dialog",
            "authors": [
                "Shikib Mehri",
                "Tejas Srinivasan",
                "Maxine Eskenazi"
            ],
            "abstract": "Neural dialog models have exhibited strong performance, however their\nend-to-end nature lacks a representation of the explicit structure of dialog.\nThis results in a loss of generalizability, controllability and a data-hungry\nnature. Conversely, more traditional dialog systems do have strong models of\nexplicit structure. This paper introduces several approaches for explicitly\nincorporating structure into neural models of dialog. Structured Fusion\nNetworks first learn neural dialog modules corresponding to the structured\ncomponents of traditional dialog systems and then incorporate these modules in\na higher-level generative model. Structured Fusion Networks obtain strong\nresults on the MultiWOZ dataset, both with and without reinforcement learning.\nStructured Fusion Networks are shown to have several valuable properties,\nincluding better domain generalizability, improved performance in reduced data\nscenarios and robustness to divergence during reinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10165v1",
            "date": "2019-07-23 22:41:22+00:00",
            "title": "Optimal Transport-based Alignment of Learned Character Representations for String Similarity",
            "authors": [
                "Derek Tam",
                "Nicholas Monath",
                "Ari Kobren",
                "Aaron Traylor",
                "Rajarshi Das",
                "Andrew McCallum"
            ],
            "abstract": "String similarity models are vital for record linkage, entity resolution, and\nsearch. In this work, we present STANCE --a learned model for computing the\nsimilarity of two strings. Our approach encodes the characters of each string,\naligns the encodings using Sinkhorn Iteration (alignment is posed as an\ninstance of optimal transport) and scores the alignment with a convolutional\nneural network. We evaluate STANCE's ability to detect whether two strings can\nrefer to the same entity--a task we term alias detection. We construct five new\nalias detection datasets (and make them publicly available). We show that\nSTANCE or one of its variants outperforms both state-of-the-art and classic,\nparameter-free similarity models on four of the five datasets. We also\ndemonstrate STANCE's ability to improve downstream tasks by applying it to an\ninstance of cross-document coreference and show that it leads to a 2.8 point\nimprovement in B^3 F1 over the previous state-of-the-art approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.10185v1",
            "date": "2019-07-24 00:37:20+00:00",
            "title": "Non-Parallel Voice Conversion with Cyclic Variational Autoencoder",
            "authors": [
                "Patrick Lumban Tobing",
                "Yi-Chiao Wu",
                "Tomoki Hayashi",
                "Kazuhiro Kobayashi",
                "Tomoki Toda"
            ],
            "abstract": "In this paper, we present a novel technique for a non-parallel voice\nconversion (VC) with the use of cyclic variational autoencoder (CycleVAE)-based\nspectral modeling. In a variational autoencoder(VAE) framework, a latent space,\nusually with a Gaussian prior, is used to encode a set of input features. In a\nVAE-based VC, the encoded latent features are fed into a decoder, along with\nspeaker-coding features, to generate estimated spectra with either the original\nspeaker identity (reconstructed) or another speaker identity (converted). Due\nto the non-parallel modeling condition, the converted spectra can not be\ndirectly optimized, which heavily degrades the performance of a VAE-based VC.\nIn this work, to overcome this problem, we propose to use CycleVAE-based\nspectral model that indirectly optimizes the conversion flow by recycling the\nconverted features back into the system to obtain corresponding cyclic\nreconstructed spectra that can be directly optimized. The cyclic flow can be\ncontinued by using the cyclic reconstructed features as input for the next\ncycle. The experimental results demonstrate the effectiveness of the proposed\nCycleVAE-based VC, which yields higher accuracy of converted spectra, generates\nlatent features with higher correlation degree, and significantly improves the\nquality and conversion accuracy of the converted speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.10210v1",
            "date": "2019-07-24 02:34:58+00:00",
            "title": "A CNN-based tool for automatic tongue contour tracking in ultrasound images",
            "authors": [
                "Jian Zhu",
                "Will Styler",
                "Ian Calloway"
            ],
            "abstract": "For speech research, ultrasound tongue imaging provides a non-invasive means\nfor visualizing tongue position and movement during articulation. Extracting\ntongue contours from ultrasound images is a basic step in analyzing ultrasound\ndata but this task often requires non-trivial manual annotation. This study\npresents an open source tool for fully automatic tracking of tongue contours in\nultrasound frames using neural network based methods. We have implemented and\nsystematically compared two convolutional neural networks, U-Net and\nDenseU-Net, under different conditions. Though both models can perform\nautomatic contour tracking with comparable accuracy, Dense U-Net architecture\nseems more generalizable across test datasets while U-Net has faster extraction\nspeed. Our comparison also shows that the choice of loss function and data\naugmentation have a greater effect on tracking performance in this task. This\npublic available segmentation tool shows considerable promise for the automated\ntongue contour annotation of ultrasound images in speech research.",
            "journal ref": "None",
            "category": "eess.IV"
        },
        {
            "url": "http://arxiv.org/abs/1907.10717v5",
            "date": "2019-07-24 20:50:33+00:00",
            "title": "Dynamical Triangulation Induced by Quantum Walk",
            "authors": [
                "Quentin Aristote",
                "Nathana\u00ebl Eon",
                "Giuseppe Di Molfetta"
            ],
            "abstract": "We present the single-particle sector of a quantum cellular automaton, namely\na quantum walk, on a simple dynamical triangulated $2-$manifold. The\ntriangulation is changed through Pachner moves, induced by the walker density\nitself, allowing the surface to transform into any topologically equivalent\none. This model extends the quantum walk over triangular grid, introduced in a\nprevious work, by one of the authors, whose space-time limit recovers the Dirac\nequation in (2+1)-dimensions. Numerical simulations show that the number of\ntriangles and the local curvature grow as $t^\\alpha e^{-\\beta t^2}$, where\n$\\alpha$ and $\\beta$ parametrize the way geometry changes upon the local\ndensity of the walker, and that, in the long run, flatness emerges. Finally, we\nalso prove that the global behavior of the walker, remains the same under\nspacetime random fluctuations.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1907.10726v1",
            "date": "2019-07-24 21:18:39+00:00",
            "title": "Cross-Attention End-to-End ASR for Two-Party Conversations",
            "authors": [
                "Suyoun Kim",
                "Siddharth Dalmia",
                "Florian Metze"
            ],
            "abstract": "We present an end-to-end speech recognition model that learns interaction\nbetween two speakers based on the turn-changing information. Unlike\nconventional speech recognition models, our model exploits two speakers'\nhistory of conversational-context information that spans across multiple turns\nwithin an end-to-end framework. Specifically, we propose a speaker-specific\ncross-attention mechanism that can look at the output of the other speaker side\nas well as the one of the current speaker for better at recognizing long\nconversations. We evaluated the models on the Switchboard conversational speech\ncorpus and show that our model outperforms standard end-to-end speech\nrecognition models.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.10738v1",
            "date": "2019-07-24 21:37:16+00:00",
            "title": "Careful Selection of Knowledge to solve Open Book Question Answering",
            "authors": [
                "Pratyay Banerjee",
                "Kuntal Kumar Pal",
                "Arindam Mitra",
                "Chitta Baral"
            ],
            "abstract": "Open book question answering is a type of natural language based QA (NLQA)\nwhere questions are expected to be answered with respect to a given set of open\nbook facts, and common knowledge about a topic. Recently a challenge involving\nsuch QA, OpenBookQA, has been proposed. Unlike most other NLQA tasks that focus\non linguistic understanding, OpenBookQA requires deeper reasoning involving\nlinguistic understanding as well as reasoning with common knowledge. In this\npaper we address QA with respect to the OpenBookQA dataset and combine state of\nthe art language models with abductive information retrieval (IR), information\ngain based re-ranking, passage selection and weighted scoring to achieve 72.0%\naccuracy, an 11.6% improvement over the current state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.10739v1",
            "date": "2019-07-24 21:37:29+00:00",
            "title": "Visual Interaction with Deep Learning Models through Collaborative Semantic Inference",
            "authors": [
                "Sebastian Gehrmann",
                "Hendrik Strobelt",
                "Robert Kr\u00fcger",
                "Hanspeter Pfister",
                "Alexander M. Rush"
            ],
            "abstract": "Automation of tasks can have critical consequences when humans lose agency\nover decision processes. Deep learning models are particularly susceptible\nsince current black-box approaches lack explainable reasoning. We argue that\nboth the visual interface and model structure of deep learning systems need to\ntake into account interaction design. We propose a framework of collaborative\nsemantic inference (CSI) for the co-design of interactions and models to enable\nvisual collaboration between humans and algorithms. The approach exposes the\nintermediate reasoning process of models which allows semantic interactions\nwith the visual metaphors of a problem, which means that a user can both\nunderstand and control parts of the model reasoning process. We demonstrate the\nfeasibility of CSI with a co-designed case study of a document summarization\nsystem.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1907.10761v1",
            "date": "2019-07-24 22:30:04+00:00",
            "title": "Bilingual Lexicon Induction through Unsupervised Machine Translation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre"
            ],
            "abstract": "A recent research line has obtained strong results on bilingual lexicon\ninduction by aligning independently trained word embeddings in two languages\nand using the resulting cross-lingual embeddings to induce word translation\npairs through nearest neighbor or related retrieval methods. In this paper, we\npropose an alternative approach to this problem that builds on the recent work\non unsupervised machine translation. This way, instead of directly inducing a\nbilingual lexicon from cross-lingual embeddings, we use them to build a\nphrase-table, combine it with a language model, and use the resulting machine\ntranslation system to generate a synthetic parallel corpus, from which we\nextract the bilingual lexicon using statistical word alignment techniques. As\nsuch, our method can work with any word embedding and cross-lingual mapping\ntechnique, and it does not require any additional resource besides the\nmonolingual corpus used to train the embeddings. When evaluated on the exact\nsame cross-lingual embeddings, our proposed method obtains an average\nimprovement of 6 accuracy points over nearest neighbor and 4 points over CSLS\nretrieval, establishing a new state-of-the-art in the standard MUSE dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11184v1",
            "date": "2019-07-25 16:45:06+00:00",
            "title": "HEIDL: Learning Linguistic Expressions with Deep Learning and Human-in-the-Loop",
            "authors": [
                "Yiwei Yang",
                "Eser Kandogan",
                "Yunyao Li",
                "Walter S. Lasecki",
                "Prithviraj Sen"
            ],
            "abstract": "While the role of humans is increasingly recognized in machine learning\ncommunity, representation of and interaction with models in current\nhuman-in-the-loop machine learning (HITL-ML) approaches are too low-level and\nfar-removed from human's conceptual models. We demonstrate HEIDL, a prototype\nHITL-ML system that exposes the machine-learned model through high-level,\nexplainable linguistic expressions formed of predicates representing semantic\nstructure of text. In HEIDL, human's role is elevated from simply evaluating\nmodel predictions to interpreting and even updating the model logic directly by\nenabling interaction with rule predicates themselves. Raising the currency of\ninteraction to such semantic levels calls for new interaction paradigms between\nhumans and machines that result in improved productivity for text analytics\nmodel development process. Moreover, by involving humans in the process, the\nhuman-machine co-created models generalize better to unseen data as domain\nexperts are able to instill their expertise by extrapolating from what has been\nlearned by automated algorithms from few labelled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11499v1",
            "date": "2019-07-26 11:53:15+00:00",
            "title": "Weakly Supervised Domain Detection",
            "authors": [
                "Yumo Xu",
                "Mirella Lapata"
            ],
            "abstract": "In this paper we introduce domain detection as a new natural language\nprocessing task. We argue that the ability to detect textual segments which are\ndomain-heavy, i.e., sentences or phrases which are representative of and\nprovide evidence for a given domain could enhance the robustness and\nportability of various text classification applications. We propose an\nencoder-detector framework for domain detection and bootstrap classifiers with\nmultiple instance learning (MIL). The model is hierarchically organized and\nsuited to multilabel classification. We demonstrate that despite learning with\nminimal supervision, our model can be applied to text spans of different\ngranularities, languages, and genres. We also showcase the potential of domain\ndetection for text summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11640v1",
            "date": "2019-07-26 15:52:04+00:00",
            "title": "On the Use/Misuse of the Term 'Phoneme'",
            "authors": [
                "Roger K. Moore",
                "Lucy Skidmore"
            ],
            "abstract": "The term 'phoneme' lies at the heart of speech science and technology, and\nyet it is not clear that the research community fully appreciates its meaning\nand implications. In particular, it is suspected that many researchers use the\nterm in a casual sense to refer to the sounds of speech, rather than as a well\ndefined abstract concept. If true, this means that some sections of the\ncommunity may be missing an opportunity to understand and exploit the\nimplications of this important psychological phenomenon. Here we review the\ncorrect meaning of the term 'phoneme' and report the results of an\ninvestigation into its use/misuse in the accepted papers at INTERSPEECH-2018.\nIt is confirmed that a significant proportion of the community (i) may not be\naware of the critical difference between `phonetic' and 'phonemic' levels of\ndescription, (ii) may not fully understand the significance of 'phonemic\ncontrast', and as a consequence, (iii) consistently misuse the term 'phoneme'.\nThese findings are discussed, and recommendations are made as to how this\nsituation might be mitigated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11843v1",
            "date": "2019-07-27 04:08:55+00:00",
            "title": "Analyzing Linguistic Complexity and Scientific Impact",
            "authors": [
                "Chao Lu",
                "Yi Bu",
                "Xianlei Dong",
                "Jie Wang",
                "Ying Ding",
                "Vincent Larivi\u00e8re",
                "Cassidy R. Sugimoto",
                "Logan Paul",
                "Chengzhi Zhang"
            ],
            "abstract": "The number of publications and the number of citations received have become\nthe most common indicators of scholarly success. In this context, scientific\nwriting increasingly plays an important role in scholars' scientific careers.\nTo understand the relationship between scientific writing and scientific\nimpact, this paper selected 12 variables of linguistic complexity as a proxy\nfor depicting scientific writing. We then analyzed these features from 36,400\nfull-text Biology articles and 1,797 full-text Psychology articles. These\nfeatures were compared to the scientific impact of articles, grouped into high,\nmedium, and low categories. The results suggested no practical significant\nrelationship between linguistic complexity and citation strata in either\ndiscipline. This suggests that textual complexity plays little role in\nscientific impact in our data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11889v1",
            "date": "2019-07-27 10:19:19+00:00",
            "title": "Towards Effective Rebuttal: Listening Comprehension using Corpus-Wide Claim Mining",
            "authors": [
                "Tamar Lavee",
                "Matan Orbach",
                "Lili Kotlerman",
                "Yoav Kantor",
                "Shai Gretz",
                "Lena Dankin",
                "Shachar Mirkin",
                "Michal Jacovi",
                "Yonatan Bilu",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "Engaging in a live debate requires, among other things, the ability to\neffectively rebut arguments claimed by your opponent. In particular, this\nrequires identifying these arguments. Here, we suggest doing so by\nautomatically mining claims from a corpus of news articles containing billions\nof sentences, and searching for them in a given speech. This raises the\nquestion of whether such claims indeed correspond to those made in spoken\nspeeches. To this end, we collected a large dataset of $400$ speeches in\nEnglish discussing $200$ controversial topics, mined claims for each topic, and\nasked annotators to identify the mined claims mentioned in each speech. Results\nshow that in the vast majority of speeches debaters indeed make use of such\nclaims. In addition, we present several baselines for the automatic detection\nof mined claims in speeches, forming the basis for future work. All collected\ndata is freely available for research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.11932v6",
            "date": "2019-07-27 15:07:04+00:00",
            "title": "Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment",
            "authors": [
                "Di Jin",
                "Zhijing Jin",
                "Joey Tianyi Zhou",
                "Peter Szolovits"
            ],
            "abstract": "Machine learning algorithms are often vulnerable to adversarial examples that\nhave imperceptible alterations from the original counterparts but can fool the\nstate-of-the-art models. It is helpful to evaluate or even improve the\nrobustness of these models by exposing the maliciously crafted adversarial\nexamples. In this paper, we present TextFooler, a simple but strong baseline to\ngenerate natural adversarial text. By applying it to two fundamental natural\nlanguage tasks, text classification and textual entailment, we successfully\nattacked three target models, including the powerful pre-trained BERT, and the\nwidely used convolutional and recurrent neural networks. We demonstrate the\nadvantages of this framework in three ways: (1) effective---it outperforms\nstate-of-the-art attacks in terms of success rate and perturbation rate, (2)\nutility-preserving---it preserves semantic content and grammaticality, and\nremains correctly classified by humans, and (3) efficient---it generates\nadversarial text with computational complexity linear to the text length. *The\ncode, pre-trained target models, and test examples are available at\nhttps://github.com/jind11/TextFooler.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12008v1",
            "date": "2019-07-28 03:57:16+00:00",
            "title": "Fusing location and text features for sentiment classification",
            "authors": [
                "Wei Lun Lim",
                "Chiung Ching Ho",
                "Choo-Yee Ting"
            ],
            "abstract": "Geo-tagged Twitter data has been used recently to infer insights on the human\naspects of social media. Insights related to demographics, spatial distribution\nof cultural activities, space-time travel trajectories for humans as well as\nhappiness has been mined from geo-tagged twitter data in recent studies. To\ndate, not much study has been done on the impact of the geolocation features of\na Tweet on its sentiment. This observation has inspired us to propose the usage\nof geo-location features as a method to perform sentiment classification. In\nthis method, the sentiment classification of geo-tagged tweets is performed by\nconcatenating geo-location features and one-hot encoded word vectors as inputs\nfor convolutional neural networks (CNN) and long short-term memory (LSTM)\nnetworks. The addition of language-independent features in the form of\ngeo-location features has helped to enrich the tweet representation in order to\ncombat the sparse nature of short tweet message. The results achieved has\ndemonstrated that concatenating geo-location features to one-hot encoded word\nvectors can achieve higher accuracy as compared to the usage of word vectors\nalone for the purpose of sentiment classification.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.12021v1",
            "date": "2019-07-28 06:15:35+00:00",
            "title": "What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialog",
            "authors": [
                "Pushkar Shukla",
                "Carlos Elmadjian",
                "Richika Sharan",
                "Vivek Kulkarni",
                "Matthew Turk",
                "William Yang Wang"
            ],
            "abstract": "The ability to engage in goal-oriented conversations has allowed humans to\ngain knowledge, reduce uncertainty, and perform tasks more efficiently.\nArtificial agents, however, are still far behind humans in having goal-driven\nconversations. In this work, we focus on the task of goal-oriented visual\ndialogue, aiming to automatically generate a series of questions about an image\nwith a single objective. This task is challenging since these questions must\nnot only be consistent with a strategy to achieve a goal, but also consider the\ncontextual information in the image. We propose an end-to-end goal-oriented\nvisual dialogue system, that combines reinforcement learning with regularized\ninformation gain. Unlike previous approaches that have been proposed for the\ntask, our work is motivated by the Rational Speech Act framework, which models\nthe process of human inquiry to reach a goal. We test the two versions of our\nmodel on the GuessWhat?! dataset, obtaining significant results that outperform\nthe current state-of-the-art models in the task of generating questions to find\nan undisclosed object in an image.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12048v1",
            "date": "2019-07-28 08:56:06+00:00",
            "title": "Probabilistic Models of Relational Implication",
            "authors": [
                "Xavier Holt"
            ],
            "abstract": "Relational data in its most basic form is a static collection of known facts.\nHowever, by learning to infer and deduct additional information and structure,\nwe can massively increase the usefulness of the underlying data. One common\nform of inferential reasoning in knowledge bases is implication discovery.\nHere, by learning when one relation implies another, we can extend our\nknowledge representation. There are several existing models for relational\nimplication, however we argue they are motivated but not principled. To this\nend, we define a formal probabilistic model of relational implication. By using\nestimators based on the empirical distribution of our dataset, we demonstrate\nthat our model outperforms existing approaches. While previous work achieves a\nbest score of 0.7812 AUC on an evaluatory dataset, our ProbE model improves\nthis to 0.7915. Furthermore, we demonstrate that our model can be improved\nsubstantially through the use of link prediction models and dense latent\nrepresentations of the underlying argument and relations. This variant, denoted\nProbL, improves the state of the art on our evaluation dataset to 0.8143. In\naddition to developing a new framework and providing novel scores of relational\nimplication, we provide two pragmatic resources to assist future research.\nFirst, we motivate and develop an improved crowd framework for constructing\nlabelled datasets of relational implication. Using this, we reannotate and make\npublic a dataset comprised of 17,848 instances of labelled relational\nimplication. We demonstrate that precision (as evaluated by expert consensus\nwith the crowd labels) on the resulting dataset improves from 53% to 95%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.12293v7",
            "date": "2019-07-29 09:25:49+00:00",
            "title": "A mathematical model for universal semantics",
            "authors": [
                "Weinan E",
                "Yajun Zhou"
            ],
            "abstract": "We characterize the meaning of words with language-independent numerical\nfingerprints, through a mathematical analysis of recurring patterns in texts.\nApproximating texts by Markov processes on a long-range time scale, we are able\nto extract topics, discover synonyms, and sketch semantic fields from a\nparticular document of moderate length, without consulting external\nknowledge-base or thesaurus. Our Markov semantic model allows us to represent\neach topical concept by a low-dimensional vector, interpretable as algebraic\ninvariants in succinct statistical operations on the document, targeting local\nenvironments of individual words. These language-independent semantic\nrepresentations enable a robot reader to both understand short texts in a given\nlanguage (automated question-answering) and match medium-length texts across\ndifferent languages (automated word translation). Our semantic fingerprints\nquantify local meaning of words in 14 representative languages across 5 major\nlanguage families, suggesting a universal and cost-effective mechanism by which\nhuman languages are processed at the semantic level. Our protocols and source\ncodes are publicly available on\nhttps://github.com/yajun-zhou/linguae-naturalis-principia-mathematica",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.12477v2",
            "date": "2019-07-29 15:27:50+00:00",
            "title": "Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions",
            "authors": [
                "Robert Tjarko Lange",
                "Aldo Faisal"
            ],
            "abstract": "Hierarchical Reinforcement Learning algorithms have successfully been applied\nto temporal credit assignment problems with sparse reward signals. However,\nstate-of-the-art algorithms require manual specification of sub-task\nstructures, a sample inefficient exploration phase or lack semantic\ninterpretability. Humans, on the other hand, efficiently detect hierarchical\nsub-structures induced by their surroundings. It has been argued that this\ninference process universally applies to language, logical reasoning as well as\nmotor control. Therefore, we propose a cognitive-inspired Reinforcement\nLearning architecture which uses grammar induction to identify sub-goal\npolicies. By treating an on-policy trajectory as a sentence sampled from the\npolicy-conditioned language of the environment, we identify hierarchical\nconstituents with the help of unsupervised grammatical inference. The resulting\nset of temporal abstractions is called action grammar (Pastra & Aloimonos,\n2012) and unifies symbolic and connectionist approaches to Reinforcement\nLearning. It can be used to facilitate efficient imitation, transfer and online\nlearning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.12697v1",
            "date": "2019-07-30 01:37:34+00:00",
            "title": "Dual-FOFE-net Neural Models for Entity Linking with PageRank",
            "authors": [
                "Feng Wei",
                "Uyen Trang Nguyen",
                "Hui Jiang"
            ],
            "abstract": "This paper presents a simple and computationally efficient approach for\nentity linking (EL), compared with recurrent neural networks (RNNs) or\nconvolutional neural networks (CNNs), by making use of feedforward neural\nnetworks (FFNNs) and the recent dual fixed-size ordinally forgetting encoding\n(dual-FOFE) method to fully encode the sentence fragment and its left/right\ncontexts into a fixed-size representation. Furthermore, in this work, we\npropose to incorporate PageRank based distillation in our candidate generation\nmodule. Our neural linking models consist of three parts: a PageRank based\ncandidate generation module, a dual-FOFE-net neural ranking model and a simple\nNIL entity clustering system. Experimental results have shown that our proposed\nneural linking models achieved higher EL accuracy than state-of-the-art models\non the TAC2016 task dataset over the baseline system, without requiring any\nin-house data or complicated handcrafted features. Moreover, it achieves a\ncompetitive accuracy on the TAC2017 task dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13121v1",
            "date": "2019-07-29 19:56:46+00:00",
            "title": "Multi-Frame Cross-Entropy Training for Convolutional Neural Networks in Speech Recognition",
            "authors": [
                "Tom Sercu",
                "Neil Mallinar"
            ],
            "abstract": "We introduce Multi-Frame Cross-Entropy training (MFCE) for convolutional\nneural network acoustic models. Recognizing that similar to RNNs, CNNs are in\nnature sequence models that take variable length inputs, we propose to take as\ninput to the CNN a part of an utterance long enough that multiple labels are\npredicted at once, therefore getting cross-entropy loss signal from multiple\nadjacent frames. This increases the amount of label information drastically for\nsmall marginal computational cost. We show large WER improvements on hub5 and\nrt02 after training on the 2000-hour Switchboard benchmark.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.13295v2",
            "date": "2019-07-31 03:11:33+00:00",
            "title": "Lifelong and Interactive Learning of Factual Knowledge in Dialogues",
            "authors": [
                "Sahisnu Mazumder",
                "Bing Liu",
                "Shuai Wang",
                "Nianzu Ma"
            ],
            "abstract": "Dialogue systems are increasingly using knowledge bases (KBs) storing\nreal-world facts to help generate quality responses. However, as the KBs are\ninherently incomplete and remain fixed during conversation, it limits dialogue\nsystems' ability to answer questions and to handle questions involving entities\nor relations that are not in the KB. In this paper, we make an attempt to\npropose an engine for Continuous and Interactive Learning of Knowledge (CILK)\nfor dialogue systems to give them the ability to continuously and interactively\nlearn and infer new knowledge during conversations. With more knowledge\naccumulated over time, they will be able to learn better and answer more\nquestions. Our empirical evaluation shows that CILK is promising.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.13511v1",
            "date": "2019-07-31 14:07:27+00:00",
            "title": "Personalizing ASR for Dysarthric and Accented Speech with Limited Data",
            "authors": [
                "Joel Shor",
                "Dotan Emanuel",
                "Oran Lang",
                "Omry Tuval",
                "Michael Brenner",
                "Julie Cattiau",
                "Fernando Vieira",
                "Maeve McNally",
                "Taylor Charbonneau",
                "Melissa Nollstadt",
                "Avinatan Hassidim",
                "Yossi Matias"
            ],
            "abstract": "Automatic speech recognition (ASR) systems have dramatically improved over\nthe last few years. ASR systems are most often trained from 'typical' speech,\nwhich means that underrepresented groups don't experience the same level of\nimprovement. In this paper, we present and evaluate finetuning techniques to\nimprove ASR for users with non-standard speech. We focus on two types of\nnon-standard speech: speech from people with amyotrophic lateral sclerosis\n(ALS) and accented speech. We train personalized models that achieve 62% and\n35% relative WER improvement on these two groups, bringing the absolute WER for\nALS speakers, on a test set of message bank phrases, down to 10% for mild\ndysarthria and 20% for more serious dysarthria. We show that 71% of the\nimprovement comes from only 5 minutes of training data. Finetuning a particular\nsubset of layers (with many fewer parameters) often gives better results than\nfinetuning the entire model. This is the first step towards building state of\nthe art ASR models for dysarthric speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.00449v1",
            "date": "2019-08-01 15:05:41+00:00",
            "title": "Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data",
            "authors": [
                "Jacob Harer",
                "Chris Reale",
                "Peter Chin"
            ],
            "abstract": "Many common sequential data sources, such as source code and natural\nlanguage, have a natural tree-structured representation. These trees can be\ngenerated by fitting a sequence to a grammar, yielding a hierarchical ordering\nof the tokens in the sequence. This structure encodes a high degree of\nsyntactic information, making it ideal for problems such as grammar correction.\nHowever, little work has been done to develop neural networks that can operate\non and exploit tree-structured data. In this paper we present the\nTree-Transformer \\textemdash{} a novel neural network architecture designed to\ntranslate between arbitrary input and output trees. We applied this\narchitecture to correction tasks in both the source code and natural language\ndomains. On source code, our model achieved an improvement of $25\\%$\n$\\text{F}0.5$ over the best sequential method. On natural language, we achieved\ncomparable results to the most complex state of the art systems, obtaining a\n$10\\%$ improvement in recall on the CoNLL 2014 benchmark and the highest to\ndate $\\text{F}0.5$ score on the AESW benchmark of $50.43$.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.00475v1",
            "date": "2019-08-01 16:02:04+00:00",
            "title": "Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections",
            "authors": [
                "Mennatallah El-Assady",
                "Rebecca Kehlbeck",
                "Christopher Collins",
                "Daniel Keim",
                "Oliver Deussen"
            ],
            "abstract": "We present a framework that allows users to incorporate the semantics of\ntheir domain knowledge for topic model refinement while remaining\nmodel-agnostic. Our approach enables users to (1) understand the semantic space\nof the model, (2) identify regions of potential conflicts and problems, and (3)\nreadjust the semantic relation of concepts based on their understanding,\ndirectly influencing the topic modeling. These tasks are supported by an\ninteractive visual analytics workspace that uses word-embedding projections to\ndefine concept regions which can then be refined. The user-refined concepts are\nindependent of a particular document collection and can be transferred to\nrelated corpora. All user interactions within the concept space directly affect\nthe semantic relations of the underlying vector space model, which, in turn,\nchange the topic modeling. In addition to direct manipulation, our system\nguides the users' decision-making process through recommended interactions that\npoint out potential improvements. This targeted refinement aims at minimizing\nthe feedback required for an efficient human-in-the-loop process. We confirm\nthe improvements achieved through our approach in two user studies that show\ntopic model quality improvements through our visual knowledge externalization\nand learning process.",
            "journal ref": "IEEE Transactions on Visualization and Computer Graphics, 2019",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1908.00785v1",
            "date": "2019-08-02 10:09:51+00:00",
            "title": "Deep learning languages: a key fundamental shift from probabilities to weights?",
            "authors": [
                "Fran\u00e7ois Coste"
            ],
            "abstract": "Recent successes in language modeling, notably with deep learning methods,\ncoincide with a shift from probabilistic to weighted representations. We raise\nhere the question of the importance of this evolution, in the light of the\npractical limitations of a classical and simple probabilistic modeling approach\nfor the classification of protein sequences and in relation to the need for\nprincipled methods to learn non-probabilistic models.",
            "journal ref": "None",
            "category": "q-bio.OT"
        },
        {
            "url": "http://arxiv.org/abs/1908.01060v1",
            "date": "2019-08-02 21:08:13+00:00",
            "title": "Multilingual Speech Recognition with Corpus Relatedness Sampling",
            "authors": [
                "Xinjian Li",
                "Siddharth Dalmia",
                "Alan W. Black",
                "Florian Metze"
            ],
            "abstract": "Multilingual acoustic models have been successfully applied to low-resource\nspeech recognition. Most existing works have combined many small corpora\ntogether and pretrained a multilingual model by sampling from each corpus\nuniformly. The model is eventually fine-tuned on each target corpus. This\napproach, however, fails to exploit the relatedness and similarity among\ncorpora in the training set. For example, the target corpus might benefit more\nfrom a corpus in the same domain or a corpus from a close language. In this\nwork, we propose a simple but useful sampling strategy to take advantage of\nthis relatedness. We first compute the corpus-level embeddings and estimate the\nsimilarity between each corpus. Next, we start training the multilingual model\nwith uniform-sampling from each corpus at first, then we gradually increase the\nprobability to sample from related corpora based on its similarity with the\ntarget corpus. Finally, the model would be fine-tuned automatically on the\ntarget corpus. Our sampling strategy outperforms the baseline multilingual\nmodel on 16 low-resource tasks. Additionally, we demonstrate that our corpus\nembeddings capture the language and domain information of each corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01067v1",
            "date": "2019-08-02 21:13:27+00:00",
            "title": "SANTLR: Speech Annotation Toolkit for Low Resource Languages",
            "authors": [
                "Xinjian Li",
                "Zhong Zhou",
                "Siddharth Dalmia",
                "Alan W. Black",
                "Florian Metze"
            ],
            "abstract": "While low resource speech recognition has attracted a lot of attention from\nthe speech community, there are a few tools available to facilitate low\nresource speech collection. In this work, we present SANTLR: Speech Annotation\nToolkit for Low Resource Languages. It is a web-based toolkit which allows\nresearchers to easily collect and annotate a corpus of speech in a low resource\nlanguage. Annotators may use this toolkit for two purposes: transcription or\nrecording. In transcription, annotators would transcribe audio files provided\nby the researchers; in recording, annotators would record their voice by\nreading provided texts. We highlight two properties of this toolkit. First,\nSANTLR has a very user-friendly User Interface (UI). Both researchers and\nannotators may use this simple web interface to interact. There is no\nrequirement for the annotators to have any expertise in audio or text\nprocessing. The toolkit would handle all preprocessing and postprocessing\nsteps. Second, we employ a multi-step ranking mechanism facilitate the\nannotation process. In particular, the toolkit would give higher priority to\nutterances which are easier to annotate and are more beneficial to achieving\nthe goal of the annotation, e.g. quickly training an acoustic model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01165v3",
            "date": "2019-08-03 12:59:40+00:00",
            "title": "Exploring the Robustness of NMT Systems to Nonsensical Inputs",
            "authors": [
                "Akshay Chaturvedi",
                "Abijith KP",
                "Utpal Garain"
            ],
            "abstract": "Neural machine translation (NMT) systems have been shown to give undesirable\ntranslation when a small change is made in the source sentence. In this paper,\nwe study the behaviour of NMT systems when multiple changes are made to the\nsource sentence. In particular, we ask the following question \"Is it possible\nfor an NMT system to predict same translation even when multiple words in the\nsource sentence have been replaced?\". To this end, we propose a soft-attention\nbased technique to make the aforementioned word replacements. The experiments\nare conducted on two language pairs: English-German (en-de) and English-French\n(en-fr) and two state-of-the-art NMT systems: BLSTM-based encoder-decoder with\nattention and Transformer. The proposed soft-attention based technique achieves\nhigh success rate and outperforms existing methods like HotFlip by a\nsignificant margin for all the conducted experiments. The results demonstrate\nthat state-of-the-art NMT systems are unable to capture the semantics of the\nsource language. The proposed soft-attention based technique is an\ninvariance-based adversarial attack on NMT systems. To better evaluate such\nattacks, we propose an alternate metric and argue its benefits in comparison\nwith success rate.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.01211v1",
            "date": "2019-08-03 18:09:56+00:00",
            "title": "Word2vec to behavior: morphology facilitates the grounding of language in machines",
            "authors": [
                "David Matthews",
                "Sam Kriegman",
                "Collin Cappelle",
                "Josh Bongard"
            ],
            "abstract": "Enabling machines to respond appropriately to natural language commands could\ngreatly expand the number of people to whom they could be of service. Recently,\nadvances in neural network-trained word embeddings have empowered non-embodied\ntext-processing algorithms, and suggest they could be of similar utility for\nembodied machines. Here we introduce a method that does so by training robots\nto act similarly to semantically-similar word2vec encoded commands. We show\nthat this enables them to act appropriately, after training, to\npreviously-unheard commands. Finally, we show that inducing such an alignment\nbetween motoric and linguistic similarities can be facilitated or hindered by\nthe mechanical structure of the robot. This points to future, large scale\nmethods that find and exploit relationships between action, language, and robot\nstructure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01587v1",
            "date": "2019-08-05 16:48:22+00:00",
            "title": "Performance Evaluation of Supervised Machine Learning Techniques for Efficient Detection of Emotions from Online Content",
            "authors": [
                "Muhammad Zubair Asghar",
                "Fazli Subhan",
                "Muhammad Imran",
                "Fazal Masud Kundi",
                "Shahboddin Shamshirband",
                "Amir Mosavi",
                "Peter Csiba",
                "Annamaria R. Varkonyi-Koczy"
            ],
            "abstract": "Emotion detection from the text is an important and challenging problem in\ntext analytics. The opinion-mining experts are focusing on the development of\nemotion detection applications as they have received considerable attention of\nonline community including users and business organization for collecting and\ninterpreting public emotions. However, most of the existing works on emotion\ndetection used less efficient machine learning classifiers with limited\ndatasets, resulting in performance degradation. To overcome this issue, this\nwork aims at the evaluation of the performance of different machine learning\nclassifiers on a benchmark emotion dataset. The experimental results show the\nperformance of different machine learning classifiers in terms of different\nevaluation metrics like precision, recall ad f-measure. Finally, a classifier\nwith the best performance is recommended for the emotion classification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.01767v3",
            "date": "2019-08-04 16:48:24+00:00",
            "title": "Exploring Neural Net Augmentation to BERT for Question Answering on SQUAD 2.0",
            "authors": [
                "Suhas Gupta"
            ],
            "abstract": "Enhancing machine capabilities to answer questions has been a topic of\nconsiderable focus in recent years of NLP research. Language models like\nEmbeddings from Language Models (ELMo)[1] and Bidirectional Encoder\nRepresentations from Transformers (BERT) [2] have been very successful in\ndeveloping general purpose language models that can be optimized for a large\nnumber of downstream language tasks. In this work, we focused on augmenting the\npre-trained BERT language model with different output neural net architectures\nand compared their performance on question answering task posed by the Stanford\nQuestion Answering Dataset 2.0 (SQUAD 2.0) [3]. Additionally, we also\nfine-tuned the pre-trained BERT model parameters to demonstrate its\neffectiveness in adapting to specialized language tasks. Our best output\nnetwork, is the contextualized CNN that performs on both the unanswerable and\nanswerable question answering tasks with F1 scores of 75.32 and 64.85\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01798v1",
            "date": "2019-08-05 18:28:09+00:00",
            "title": "Unsupervised Context Retrieval for Long-tail Entities",
            "authors": [
                "Dar\u00edo Garigliotti",
                "Dyaa Albakour",
                "Miguel Martinez",
                "Krisztian Balog"
            ],
            "abstract": "Monitoring entities in media streams often relies on rich entity\nrepresentations, like structured information available in a knowledge base\n(KB). For long-tail entities, such monitoring is highly challenging, due to\ntheir limited, if not entirely missing, representation in the reference KB. In\nthis paper, we address the problem of retrieving textual contexts for\nmonitoring long-tail entities. We propose an unsupervised method to overcome\nthe limited representation of long-tail entities by leveraging established\nentities and their contexts as support information. Evaluation on a\npurpose-built test collection shows the suitability of our approach and its\nrobustness for out-of-KB entities.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.01801v2",
            "date": "2019-08-05 18:47:30+00:00",
            "title": "Answering Questions about Data Visualizations using Efficient Bimodal Fusion",
            "authors": [
                "Kushal Kafle",
                "Robik Shrestha",
                "Brian Price",
                "Scott Cohen",
                "Christopher Kanan"
            ],
            "abstract": "Chart question answering (CQA) is a newly proposed visual question answering\n(VQA) task where an algorithm must answer questions about data visualizations,\ne.g. bar charts, pie charts, and line graphs. CQA requires capabilities that\nnatural-image VQA algorithms lack: fine-grained measurements, optical character\nrecognition, and handling out-of-vocabulary words in both questions and\nanswers. Without modifications, state-of-the-art VQA algorithms perform poorly\non this task. Here, we propose a novel CQA algorithm called parallel recurrent\nfusion of image and language (PReFIL). PReFIL first learns bimodal embeddings\nby fusing question and image features and then intelligently aggregates these\nlearned embeddings to answer the given question. Despite its simplicity, PReFIL\ngreatly surpasses state-of-the art systems and human baselines on both the\nFigureQA and DVQA datasets. Additionally, we demonstrate that PReFIL can be\nused to reconstruct tables by asking a series of questions about a chart.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.01815v3",
            "date": "2019-07-26 16:19:07+00:00",
            "title": "Pars-ABSA: an Aspect-based Sentiment Analysis dataset for Persian",
            "authors": [
                "Taha Shangipour Ataei",
                "Kamyar Darvishi",
                "Soroush Javdan",
                "Behrouz Minaei-Bidgoli",
                "Sauleh Eetemadi"
            ],
            "abstract": "Due to the increased availability of online reviews, sentiment analysis had\nbeen witnessed a booming interest from the researchers. Sentiment analysis is a\ncomputational treatment of sentiment used to extract and understand the\nopinions of authors. While many systems were built to predict the sentiment of\na document or a sentence, many others provide the necessary detail on various\naspects of the entity (i.e. aspect-based sentiment analysis). Most of the\navailable data resources were tailored to English and the other popular\nEuropean languages. Although Persian is a language with more than 110 million\nspeakers, to the best of our knowledge, there is a lack of public dataset on\naspect-based sentiment analysis for Persian. This paper provides a manually\nannotated Persian dataset, Pars-ABSA, which is verified by 3 native Persian\nspeakers. The dataset consists of 5,114 positive, 3,061 negative and 1,827\nneutral data samples from 5,602 unique reviews. Moreover, as a baseline, this\npaper reports the performance of some state-of-the-art aspect-based sentiment\nanalysis methods with a focus on deep learning, on Pars-ABSA. The obtained\nresults are impressive compared to similar English state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01817v1",
            "date": "2019-07-22 14:06:15+00:00",
            "title": "Sparsity Emerges Naturally in Neural Language Models",
            "authors": [
                "Naomi Saphra",
                "Adam Lopez"
            ],
            "abstract": "Concerns about interpretability, computational resources, and principled\ninductive priors have motivated efforts to engineer sparse neural models for\nNLP tasks. If sparsity is important for NLP, might well-trained neural models\nnaturally become roughly sparse? Using the Taxi-Euclidean norm to measure\nsparsity, we find that frequent input words are associated with concentrated or\nsparse activations, while frequent target words are associated with dispersed\nactivations but concentrated gradients. We find that gradients associated with\nfunction words are more concentrated than the gradients of content words, even\ncontrolling for word frequency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01819v1",
            "date": "2019-07-19 09:34:11+00:00",
            "title": "An Unsupervised Character-Aware Neural Approach to Word and Context Representation Learning",
            "authors": [
                "Giuseppe Marra",
                "Andrea Zugarini",
                "Stefano Melacci",
                "Marco Maggini"
            ],
            "abstract": "In the last few years, neural networks have been intensively used to develop\nmeaningful distributed representations of words and contexts around them. When\nthese representations, also known as \"embeddings\", are learned from\nunsupervised large corpora, they can be transferred to different tasks with\npositive effects in terms of performances, especially when only a few\nsupervisions are available. In this work, we further extend this concept, and\nwe present an unsupervised neural architecture that jointly learns word and\ncontext embeddings, processing words as sequences of characters. This allows\nour model to spot the regularities that are due to the word morphology, and to\navoid the need of a fixed-sized input vocabulary of words. We show that we can\nlearn compact encoders that, despite the relatively small number of parameters,\nreach high-level performances in downstream tasks, comparing them with related\nstate-of-the-art approaches or with fully supervised methods.",
            "journal ref": "Lecture Notes in Computer Science, vol 11141. Springer, Cham 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01821v1",
            "date": "2019-08-02 17:12:38+00:00",
            "title": "Dialogue Act Classification in Group Chats with DAG-LSTMs",
            "authors": [
                "Ozan \u0130rsoy",
                "Rakesh Gosangi",
                "Haimin Zhang",
                "Mu-Hsin Wei",
                "Peter Lund",
                "Duccio Pappadopulo",
                "Brendan Fahy",
                "Neophytos Nephytou",
                "Camilo Ortiz"
            ],
            "abstract": "Dialogue act (DA) classification has been studied for the past two decades\nand has several key applications such as workflow automation and conversation\nanalytics. Researchers have used, to address this problem, various traditional\nmachine learning models, and more recently deep neural network models such as\nhierarchical convolutional neural networks (CNNs) and long short-term memory\n(LSTM) networks. In this paper, we introduce a new model architecture,\ndirected-acyclic-graph LSTM (DAG-LSTM) for DA classification. A DAG-LSTM\nexploits the turn-taking structure naturally present in a multi-party\nconversation, and encodes this relation in its model structure. Using the STAC\ncorpus, we show that the proposed method performs roughly 0.8% better in\naccuracy and 1.2% better in macro-F1 score when compared to existing methods.\nThe proposed method is generic and not limited to conversation applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01832v1",
            "date": "2019-07-21 07:16:55+00:00",
            "title": "Word Sense Disambiguation using Diffusion Kernel PCA",
            "authors": [
                "Bilge Sipal",
                "Ozcan Sari",
                "Asena Teke",
                "Nurullah Demirci"
            ],
            "abstract": "One of the major problems in natural language processing (NLP) is the word\nsense disambiguation (WSD) problem. It is the task of computationally\nidentifying the right sense of a polysemous word based on its context.\nResolving the WSD problem boosts the accuracy of many NLP focused algorithms\nsuch as text classification and machine translation. In this paper, we\nintroduce a new supervised algorithm for WSD, that is based on Kernel PCA and\nSemantic Diffusion Kernel, which is called Diffusion Kernel PCA (DKPCA). DKPCA\ngrasps the semantic similarities within terms, and it is based on PCA. These\nproperties enable us to perform feature extraction and dimension reduction\nguided by semantic similarities and within the algorithm. Our empirical results\non SensEval data demonstrate that DKPCA achieves higher or very close accuracy\nresults compared to SVM and KPCA with various well-known kernels when the\nlabeled data ratio is meager. Considering the scarcity of labeled data, whereas\nlarge quantities of unlabeled textual data are easily accessible, these are\nhighly encouraging first results to develop DKPCA further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01837v1",
            "date": "2019-07-23 20:45:41+00:00",
            "title": "Structured Knowledge Discovery from Massive Text Corpus",
            "authors": [
                "Chenwei Zhang"
            ],
            "abstract": "Nowadays, with the booming development of the Internet, people benefit from\nits convenience due to its open and sharing nature. A large volume of natural\nlanguage texts is being generated by users in various forms, such as search\nqueries, documents, and social media posts. As the unstructured text corpus is\nusually noisy and messy, it becomes imperative to correctly identify and\naccurately annotate structured information in order to obtain meaningful\ninsights or better understand unstructured texts. On the other hand, the\nexisting structured information, which embodies our knowledge such as entity or\nconcept relations, often suffers from incompleteness or quality-related issues.\nGiven a gigantic collection of texts which offers rich semantic information, it\nis also important to harness the massiveness of the unannotated text corpus to\nexpand and refine existing structured knowledge with fewer annotation efforts.\n  In this dissertation, I will introduce principles, models, and algorithms for\neffective structured knowledge discovery from the massive text corpus. We are\ngenerally interested in obtaining insights and better understanding\nunstructured texts with the help of structured annotations or by\nstructure-aware modeling. Also, given the existing structured knowledge, we are\ninterested in expanding its scale and improving its quality harnessing the\nmassiveness of the text corpus. In particular, four problems are studied in\nthis dissertation: Structured Intent Detection for Natural Language\nUnderstanding, Structure-aware Natural Language Modeling, Generative Structured\nKnowledge Expansion, and Synonym Refinement on Structured Knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01839v2",
            "date": "2019-07-28 21:04:05+00:00",
            "title": "Text-to-SQL Generation for Question Answering on Electronic Medical Records",
            "authors": [
                "Ping Wang",
                "Tian Shi",
                "Chandan K. Reddy"
            ],
            "abstract": "Electronic medical records (EMR) contain comprehensive patient information\nand are typically stored in a relational database with multiple tables.\nEffective and efficient patient information retrieval from EMR data is a\nchallenging task for medical experts. Question-to-SQL generation methods tackle\nthis problem by first predicting the SQL query for a given question about a\ndatabase, and then, executing the query on the database. However, most of the\nexisting approaches have not been adapted to the healthcare domain due to a\nlack of healthcare Question-to-SQL dataset for learning models specific to this\ndomain. In addition, wide use of the abbreviation of terminologies and possible\ntypos in questions introduce additional challenges for accurately generating\nthe corresponding SQL queries. In this paper, we tackle these challenges by\ndeveloping a deep learning based TRanslate-Edit Model for Question-to-SQL\n(TREQS) generation, which adapts the widely used sequence-to-sequence model to\ndirectly generate the SQL query for a given question, and further performs the\nrequired edits using an attentive-copying mechanism and task-specific look-up\ntables. Based on the widely used publicly available electronic medical\ndatabase, we create a new large-scale Question-SQL pair dataset, named\nMIMICSQL, in order to perform the Question-to-SQL generation task in healthcare\ndomain. An extensive set of experiments are conducted to evaluate the\nperformance of our proposed model on MIMICSQL. Both quantitative and\nqualitative experimental results indicate the flexibility and efficiency of our\nproposed method in predicting condition values and its robustness to random\nquestions with abbreviations and typos.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01841v2",
            "date": "2019-07-26 21:53:09+00:00",
            "title": "DLGNet: A Transformer-based Model for Dialogue Response Generation",
            "authors": [
                "Oluwatobi Olabiyi",
                "Erik T. Mueller"
            ],
            "abstract": "Neural dialogue models, despite their successes, still suffer from lack of\nrelevance, diversity, and in many cases coherence in their generated responses.\nThese issues can attributed to reasons including (1) short-range model\narchitectures that capture limited temporal dependencies, (2) limitations of\nthe maximum likelihood training objective, (3) the concave entropy profile of\ndialogue datasets resulting in short and generic responses, and (4) the\nout-of-vocabulary problem leading to generation of a large number of <UNK>\ntokens. On the other hand, transformer-based models such as GPT-2 have\ndemonstrated an excellent ability to capture long-range structures in language\nmodeling tasks. In this paper, we present DLGNet, a transformer-based model for\ndialogue modeling. We specifically examine the use of DLGNet for multi-turn\ndialogue response generation. In our experiments, we evaluate DLGNet on the\nopen-domain Movie Triples dataset and the closed-domain Ubuntu Dialogue\ndataset. DLGNet models, although trained with only the maximum likelihood\nobjective, achieve significant improvements over state-of-the-art multi-turn\ndialogue models. They also produce best performance to date on the two datasets\nbased on several metrics, including BLEU, ROUGE, and distinct n-gram. Our\nanalysis shows that the performance improvement is mostly due to the\ncombination of (1) the long-range transformer architecture with (2) the\ninjection of random informative paddings. Other contributing factors include\nthe joint modeling of dialogue context and response, and the 100% tokenization\ncoverage from the byte pair encoding (BPE).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01843v1",
            "date": "2019-07-22 08:25:16+00:00",
            "title": "GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification",
            "authors": [
                "Jie Zhou",
                "Xu Han",
                "Cheng Yang",
                "Zhiyuan Liu",
                "Lifeng Wang",
                "Changcheng Li",
                "Maosong Sun"
            ],
            "abstract": "Fact verification (FV) is a challenging task which requires to retrieve\nrelevant evidence from plain text and use the evidence to verify given claims.\nMany claims require to simultaneously integrate and reason over several pieces\nof evidence for verification. However, previous work employs simple models to\nextract information from evidence without letting evidence communicate with\neach other, e.g., merely concatenate the evidence for processing. Therefore,\nthese methods are unable to grasp sufficient relational and logical information\namong the evidence. To alleviate this issue, we propose a graph-based evidence\naggregating and reasoning (GEAR) framework which enables information to\ntransfer on a fully-connected evidence graph and then utilizes different\naggregators to collect multi-evidence information. We further employ BERT, an\neffective pre-trained language representation model, to improve the\nperformance. Experimental results on a large-scale benchmark dataset FEVER have\ndemonstrated that GEAR could leverage multi-evidence information for FV and\nthus achieves the promising result with a test FEVER score of 67.10%. Our code\nis available at https://github.com/thunlp/GEAR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.01851v1",
            "date": "2019-08-02 15:17:27+00:00",
            "title": "Self-Knowledge Distillation in Natural Language Processing",
            "authors": [
                "Sangchul Hahn",
                "Heeyoul Choi"
            ],
            "abstract": "Since deep learning became a key player in natural language processing (NLP),\nmany deep learning models have been showing remarkable performances in a\nvariety of NLP tasks, and in some cases, they are even outperforming humans.\nSuch high performance can be explained by efficient knowledge representation of\ndeep learning models. While many methods have been proposed to learn more\nefficient representation, knowledge distillation from pretrained deep networks\nsuggest that we can use more information from the soft target probability to\ntrain other neural networks. In this paper, we propose a new knowledge\ndistillation method self-knowledge distillation, based on the soft target\nprobabilities of the training model itself, where multimode information is\ndistilled from the word embedding space right below the softmax layer. Due to\nthe time complexity, our method approximates the soft target probabilities. In\nexperiments, we applied the proposed method to two different and fundamental\nNLP tasks: language model and neural machine translation. The experiment\nresults show that our proposed method improves performance on the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02127v1",
            "date": "2019-08-06 13:19:24+00:00",
            "title": "Aligning Linguistic Words and Visual Semantic Units for Image Captioning",
            "authors": [
                "Longteng Guo",
                "Jing Liu",
                "Jinhui Tang",
                "Jiangwei Li",
                "Wei Luo",
                "Hanqing Lu"
            ],
            "abstract": "Image captioning attempts to generate a sentence composed of several\nlinguistic words, which are used to describe objects, attributes, and\ninteractions in an image, denoted as visual semantic units in this paper. Based\non this view, we propose to explicitly model the object interactions in\nsemantics and geometry based on Graph Convolutional Networks (GCNs), and fully\nexploit the alignment between linguistic words and visual semantic units for\nimage captioning. Particularly, we construct a semantic graph and a geometry\ngraph, where each node corresponds to a visual semantic unit, i.e., an object,\nan attribute, or a semantic (geometrical) interaction between two objects.\nAccordingly, the semantic (geometrical) context-aware embeddings for each unit\nare obtained through the corresponding GCN learning processers. At each time\nstep, a context gated attention module takes as inputs the embeddings of the\nvisual semantic units and hierarchically align the current word with these\nunits by first deciding which type of visual semantic unit (object, attribute,\nor interaction) the current word is about, and then finding the most correlated\nvisual semantic units under this type. Extensive experiments are conducted on\nthe challenging MS-COCO image captioning dataset, and superior results are\nreported when comparing to state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.02282v1",
            "date": "2019-08-05 18:14:06+00:00",
            "title": "A Weakly-Supervised Attention-based Visualization Tool for Assessing Political Affiliation",
            "authors": [
                "Srijith Rajamohan",
                "Alana Romanella",
                "Amit Ramesh"
            ],
            "abstract": "In this work, we seek to finetune a weakly-supervised expert-guided Deep\nNeural Network (DNN) for the purpose of determining political affiliations. In\nthis context, stance detection is used for determining political affiliation or\nideology which is framed in the form of relative proximities between entities\nin a low-dimensional space. An attention-based mechanism is used to provide\nmodel interpretability. A Deep Neural Network for Natural Language\nUnderstanding (NLU) using static and contextual embeddings is trained and\nevaluated. Various techniques to visualize the projections generated from the\nnetwork are evaluated for visualization efficiency. An overview of the pipeline\nfrom data ingestion, processing and generation of visualization is given here.\nA web-based framework created to faciliate this interaction and exploration is\npresented here. Preliminary results of this study are summarized and future\nwork is outlined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02283v1",
            "date": "2019-08-06 04:23:27+00:00",
            "title": "Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification",
            "authors": [
                "Zongze Ren",
                "Zhiyong Chen",
                "Shugong Xu"
            ],
            "abstract": "Speaker embeddings become growing popular in the text-independent speaker\nverification task. In this paper, we propose two improvements during the\ntraining stage. The improvements are both based on triplet cause the training\nstage and the evaluation stage of the baseline x-vector system focus on\ndifferent aims. Firstly, we introduce triplet loss for optimizing the Euclidean\ndistances between embeddings while minimizing the multi-class cross entropy\nloss. Secondly, we design an embedding similarity measurement network for\ncontrolling the similarity between the two selected embeddings. We further\njointly train the two new methods with the original network and achieve\nstate-of-the-art. The multi-task training synergies are shown with a 9%\nreduction equal error rate (EER) and detected cost function (DCF) on the 2016\nNIST Speaker Recognition Evaluation (SRE) Test Set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.02284v2",
            "date": "2019-08-06 04:28:56+00:00",
            "title": "Two-stage Training for Chinese Dialect Recognition",
            "authors": [
                "Zongze Ren",
                "Guofu Yang",
                "Shugong Xu"
            ],
            "abstract": "In this paper, we present a two-stage language identification (LID) system\nbased on a shallow ResNet14 followed by a simple 2-layer recurrent neural\nnetwork (RNN) architecture, which was used for Xunfei (iFlyTek) Chinese Dialect\nRecognition Challenge and won the first place among 110 teams. The system\ntrains an acoustic model (AM) firstly with connectionist temporal\nclassification (CTC) to recognize the given phonetic sequence annotation and\nthen train another RNN to classify dialect category by utilizing the\nintermediate features as inputs from the AM. Compared with a three-stage system\nwe further explore, our results show that the two-stage system can achieve high\naccuracy for Chinese dialects recognition under both short utterance and long\nutterance conditions with less training time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02367v1",
            "date": "2019-08-05 09:40:18+00:00",
            "title": "Semantic Role Labeling with Associated Memory Network",
            "authors": [
                "Chaoyu Guan",
                "Yuhao Cheng",
                "Hai Zhao"
            ],
            "abstract": "Semantic role labeling (SRL) is a task to recognize all the\npredicate-argument pairs of a sentence, which has been in a performance\nimprovement bottleneck after a series of latest works were presented. This\npaper proposes a novel syntax-agnostic SRL model enhanced by the proposed\nassociated memory network (AMN), which makes use of inter-sentence attention of\nlabel-known associated sentences as a kind of memory to further enhance\ndependency-based SRL. In detail, we use sentences and their labels from train\ndataset as an associated memory cue to help label the target sentence.\nFurthermore, we compare several associated sentences selecting strategies and\nlabel merging methods in AMN to find and utilize the label of associated\nsentences while attending them. By leveraging the attentive memory from known\ntraining data, Our full model reaches state-of-the-art on CoNLL-2009 benchmark\ndatasets for syntax-agnostic setting, showing a new effective research line of\nSRL enhancement other than exploiting external resources such as well\npre-trained language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02402v1",
            "date": "2019-08-06 23:56:25+00:00",
            "title": "Flexibly-Structured Model for Task-Oriented Dialogues",
            "authors": [
                "Lei Shu",
                "Piero Molino",
                "Mahdi Namazifar",
                "Hu Xu",
                "Bing Liu",
                "Huaixiu Zheng",
                "Gokhan Tur"
            ],
            "abstract": "This paper proposes a novel end-to-end architecture for task-oriented\ndialogue systems. It is based on a simple and practical yet very effective\nsequence-to-sequence approach, where language understanding and state tracking\ntasks are modeled jointly with a structured copy-augmented sequential decoder\nand a multi-label decoder for each slot. The policy engine and language\ngeneration tasks are modeled jointly following that. The copy-augmented\nsequential decoder deals with new or unknown values in the conversation, while\nthe multi-label decoder combined with the sequential decoder ensures the\nexplicit assignment of values to slots. On the generation part, slot binary\nclassifiers are used to improve performance. This architecture is scalable to\nreal-world scenarios and is shown through an empirical evaluation to achieve\nstate-of-the-art performance on both the Cambridge Restaurant dataset and the\nStanford in-car assistant dataset\\footnote{The code is available at\n\\url{https://github.com/uber-research/FSDM}}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02551v1",
            "date": "2019-07-10 02:29:39+00:00",
            "title": "Tweets Can Tell: Activity Recognition using Hybrid Long Short-Term Memory Model",
            "authors": [
                "Renhao Cui",
                "Gagan Agrawal",
                "Rajiv Ramnath"
            ],
            "abstract": "This paper presents techniques to detect the \"offline\" activity a person is\nengaged in when she is tweeting (such as dining, shopping or entertainment), in\norder to create a dynamic profile of the user, for uses such as better\ntargeting of advertisements. To this end, we propose a hybrid LSTM model for\nrich contextual learning, along with studies on the effects of applying and\ncombining multiple LSTM based methods with different contextual features. The\nhybrid model is shown to outperform a set of baselines and state-of-the-art\nmethods. Finally, this paper presents an orthogonal validation with a real-case\napplication. Our model generates an offline activity analysis for the followers\nof several well-known accounts, which is quite representative of the expected\ncharacteristics of these accounts.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.02810v1",
            "date": "2019-08-07 19:46:11+00:00",
            "title": "Debiasing Embeddings for Reduced Gender Bias in Text Classification",
            "authors": [
                "Flavien Prost",
                "Nithum Thain",
                "Tolga Bolukbasi"
            ],
            "abstract": "(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can\ninherit gender bias from the data they were trained on. We investigate how this\nbias affects downstream classification tasks, using the case study of\noccupation classification (De-Arteaga et al.,2019). We show that traditional\ntechniques for debiasing embeddings can actually worsen the bias of the\ndownstream classifier by providing a less noisy channel for communicating\ngender information. With a relatively minor adjustment, however, we show how\nthese same techniques can be used to simultaneously reduce bias and maintain\nhigh classification accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.03142v2",
            "date": "2019-08-07 03:59:19+00:00",
            "title": "The Hitchhiker's Guide to LDA",
            "authors": [
                "Chen Ma"
            ],
            "abstract": "Latent Dirichlet Allocation (LDA) model is a famous model in the topic model\nfield, it has been studied for years due to its extensive application value in\nindustry and academia. However, the mathematical derivation of LDA model is\nchallenging and difficult, which makes it difficult for the beginners to learn.\nTo help the beginners in learning LDA, this book analyzes the mathematical\nderivation of LDA in detail, and it also introduces all the knowledge\nbackground to make it easy for beginners to understand. Thus, this book\ncontains the author's unique insights. It should be noted that this book is\nwritten in Chinese.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03265v3",
            "date": "2019-08-08 20:51:17+00:00",
            "title": "On the Variance of the Adaptive Learning Rate and Beyond",
            "authors": [
                "Liyuan Liu",
                "Haoming Jiang",
                "Pengcheng He",
                "Weizhu Chen",
                "Xiaodong Liu",
                "Jianfeng Gao",
                "Jiawei Han"
            ],
            "abstract": "The learning rate warmup heuristic achieves remarkable success in stabilizing\ntraining, accelerating convergence and improving generalization for adaptive\nstochastic optimization algorithms like RMSprop and Adam. Here, we study its\nmechanism in details. Pursuing the theory behind warmup, we identify a problem\nof the adaptive learning rate (i.e., it has problematically large variance in\nthe early stage), suggest warmup works as a variance reduction technique, and\nprovide both empirical and theoretical evidence to verify our hypothesis. We\nfurther propose RAdam, a new variant of Adam, by introducing a term to rectify\nthe variance of the adaptive learning rate. Extensive experimental results on\nimage classification, language modeling, and neural machine translation verify\nour intuition and demonstrate the effectiveness and robustness of our proposed\nmethod. All implementations are available at:\nhttps://github.com/LiyuanLucasLiu/RAdam.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.03409v2",
            "date": "2019-08-09 10:58:01+00:00",
            "title": "Transferable Representation Learning in Vision-and-Language Navigation",
            "authors": [
                "Haoshuo Huang",
                "Vihan Jain",
                "Harsh Mehta",
                "Alexander Ku",
                "Gabriel Magalhaes",
                "Jason Baldridge",
                "Eugene Ie"
            ],
            "abstract": "Vision-and-Language Navigation (VLN) tasks such as Room-to-Room (R2R) require\nmachine agents to interpret natural language instructions and learn to act in\nvisually realistic environments to achieve navigation goals. The overall task\nrequires competence in several perception problems: successful agents combine\nspatio-temporal, vision and language understanding to produce appropriate\naction sequences. Our approach adapts pre-trained vision and language\nrepresentations to relevant in-domain tasks making them more effective for VLN.\nSpecifically, the representations are adapted to solve both a cross-modal\nsequence alignment and sequence coherence task. In the sequence alignment task,\nthe model determines whether an instruction corresponds to a sequence of visual\nframes. In the sequence coherence task, the model determines whether the\nperceptual sequences are predictive sequentially in the instruction-conditioned\nlatent space. By transferring the domain-adapted representations, we improve\ncompetitive agents in R2R as measured by the success rate weighted by path\nlength (SPL) metric.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.03451v2",
            "date": "2019-08-09 13:24:21+00:00",
            "title": "Interactive Variance Attention based Online Spoiler Detection for Time-Sync Comments",
            "authors": [
                "Wenmian Yang",
                "Weijia Jia",
                "Wenyuan Gao",
                "Xiaojie Zhou",
                "Yutao Luo"
            ],
            "abstract": "Nowadays, time-sync comment (TSC), a new form of interactive comments, has\nbecome increasingly popular in Chinese video websites. By posting TSCs, people\ncan easily express their feelings and exchange their opinions with others when\nwatching online videos. However, some spoilers appear among the TSCs. These\nspoilers reveal crucial plots in videos that ruin people's surprise when they\nfirst watch the video. In this paper, we proposed a novel Similarity-Based\nNetwork with Interactive Variance Attention (SBN-IVA) to classify comments as\nspoilers or not. In this framework, we firstly extract textual features of TSCs\nthrough the word-level attentive encoder. We design Similarity-Based Network\n(SBN) to acquire neighbor and keyframe similarity according to semantic\nsimilarity and timestamps of TSCs. Then, we implement Interactive Variance\nAttention (IVA) to eliminate the impact of noise comments. Finally, we obtain\nthe likelihood of spoiler based on the difference between the neighbor and\nkeyframe similarity. Experiments show SBN-IVA is on average 11.2\\% higher than\nthe state-of-the-art method on F1-score in baselines.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03455v1",
            "date": "2019-08-09 13:35:45+00:00",
            "title": "Challenging the Boundaries of Speech Recognition: The MALACH Corpus",
            "authors": [
                "Michael Picheny",
                "Z\u00f3ltan T\u00fcske",
                "Brian Kingsbury",
                "Kartik Audhkhasi",
                "Xiaodong Cui",
                "George Saon"
            ],
            "abstract": "There has been huge progress in speech recognition over the last several\nyears. Tasks once thought extremely difficult, such as SWITCHBOARD, now\napproach levels of human performance. The MALACH corpus (LDC catalog\nLDC2012S05), a 375-Hour subset of a large archive of Holocaust testimonies\ncollected by the Survivors of the Shoah Visual History Foundation, presents\nsignificant challenges to the speech community. The collection consists of\nunconstrained, natural speech filled with disfluencies, heavy accents,\nage-related coarticulations, un-cued speaker and language switching, and\nemotional speech - all still open problems for speech recognition systems.\nTranscription is challenging even for skilled human annotators. This paper\nproposes that the community place focus on the MALACH corpus to develop speech\nrecognition systems that are more robust with respect to accents, disfluencies\nand emotional speech. To reduce the barrier for entry, a lexicon and training\nand testing setups have been created and baseline results using current deep\nlearning technologies are presented. The metadata has just been released by LDC\n(LDC2019S11). It is hoped that this resource will enable the community to build\non top of these baselines so that the extremely important information in these\nand related oral histories becomes accessible to a wider audience.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.03548v1",
            "date": "2019-08-09 17:19:43+00:00",
            "title": "BERT-based Ranking for Biomedical Entity Normalization",
            "authors": [
                "Zongcheng Ji",
                "Qiang Wei",
                "Hua Xu"
            ],
            "abstract": "Developing high-performance entity normalization algorithms that can\nalleviate the term variation problem is of great interest to the biomedical\ncommunity. Although deep learning-based methods have been successfully applied\nto biomedical entity normalization, they often depend on traditional\ncontext-independent word embeddings. Bidirectional Encoder Representations from\nTransformers (BERT), BERT for Biomedical Text Mining (BioBERT) and BERT for\nClinical Text Mining (ClinicalBERT) were recently introduced to pre-train\ncontextualized word representation models using bidirectional Transformers,\nadvancing the state-of-the-art for many natural language processing tasks. In\nthis study, we proposed an entity normalization architecture by fine-tuning the\npre-trained BERT / BioBERT / ClinicalBERT models and conducted extensive\nexperiments to evaluate the effectiveness of the pre-trained models for\nbiomedical entity normalization using three different types of datasets. Our\nexperimental results show that the best fine-tuned models consistently\noutperformed previous methods and advanced the state-of-the-art for biomedical\nentity normalization, with up to 1.17% increase in accuracy.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03557v1",
            "date": "2019-08-09 17:57:13+00:00",
            "title": "VisualBERT: A Simple and Performant Baseline for Vision and Language",
            "authors": [
                "Liunian Harold Li",
                "Mark Yatskar",
                "Da Yin",
                "Cho-Jui Hsieh",
                "Kai-Wei Chang"
            ],
            "abstract": "We propose VisualBERT, a simple and flexible framework for modeling a broad\nrange of vision-and-language tasks. VisualBERT consists of a stack of\nTransformer layers that implicitly align elements of an input text and regions\nin an associated input image with self-attention. We further propose two\nvisually-grounded language model objectives for pre-training VisualBERT on\nimage caption data. Experiments on four vision-and-language tasks including\nVQA, VCR, NLVR2, and Flickr30K show that VisualBERT outperforms or rivals with\nstate-of-the-art models while being significantly simpler. Further analysis\ndemonstrates that VisualBERT can ground elements of language to image regions\nwithout any explicit supervision and is even sensitive to syntactic\nrelationships, tracking, for example, associations between verbs and image\nregions corresponding to their arguments.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.03640v1",
            "date": "2019-08-09 21:37:42+00:00",
            "title": "Catching the Phish: Detecting Phishing Attacks using Recurrent Neural Networks (RNNs)",
            "authors": [
                "Lukas Halgas",
                "Ioannis Agrafiotis",
                "Jason R. C. Nurse"
            ],
            "abstract": "The emergence of online services in our daily lives has been accompanied by a\nrange of malicious attempts to trick individuals into performing undesired\nactions, often to the benefit of the adversary. The most popular medium of\nthese attempts is phishing attacks, particularly through emails and websites.\nIn order to defend against such attacks, there is an urgent need for automated\nmechanisms to identify this malevolent content before it reaches users. Machine\nlearning techniques have gradually become the standard for such classification\nproblems. However, identifying common measurable features of phishing content\n(e.g., in emails) is notoriously difficult. To address this problem, we engage\nin a novel study into a phishing content classifier based on a recurrent neural\nnetwork (RNN), which identifies such features without human input. At this\nstage, we scope our research to emails, but our approach can be extended to\napply to websites. Our results show that the proposed system outperforms\nstate-of-the-art tools. Furthermore, our classifier is efficient and takes into\naccount only the text and, in particular, the textual structure of the email.\nSince these features are rarely considered in email classification, we argue\nthat our classifier can complement existing classifiers with high information\ngain.",
            "journal ref": "20th World Conference on Information Security Applications (WISA\n  2019)",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1908.03971v4",
            "date": "2019-08-11 23:15:23+00:00",
            "title": "TAPER: Time-Aware Patient EHR Representation",
            "authors": [
                "Sajad Darabi",
                "Mohammad Kachuee",
                "Shayan Fazeli",
                "Majid Sarrafzadeh"
            ],
            "abstract": "Effective representation learning of electronic health records is a\nchallenging task and is becoming more important as the availability of such\ndata is becoming pervasive. The data contained in these records are irregular\nand contain multiple modalities such as notes, and medical codes. They are\npreempted by medical conditions the patient may have, and are typically jotted\ndown by medical staff. Accompanying codes are notes containing valuable\ninformation about patients beyond the structured information contained in\nelectronic health records. We use transformer networks and the recently\nproposed BERT language model to embed these data streams into a unified vector\nrepresentation. The presented approach effectively encodes a patient's visit\ndata into a single distributed representation, which can be used for downstream\ntasks. Our model demonstrates superior performance and generalization on\nmortality, readmission and length of stay tasks using the publicly available\nMIMIC-III ICU dataset. Code avaialble at\nhttps://github.com/sajaddarabi/TAPER-EHR",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.04200v1",
            "date": "2019-08-12 15:32:10+00:00",
            "title": "Assessing the Quality of Scientific Papers",
            "authors": [
                "Roman Vainshtein",
                "Gilad Katz",
                "Bracha Shapira",
                "Lior Rokach"
            ],
            "abstract": "A multitude of factors are responsible for the overall quality of scientific\npapers, including readability, linguistic quality, fluency,semantic complexity,\nand of course domain-specific technical factors. These factors vary from one\nfield of study to another. In this paper, we propose a measure and method for\nassessing the overall quality of the scientific papers in a particular field of\nstudy. We evaluate our method in the computer science domain, but it can be\napplied to other technical and scientific fields.Our method is based on the\ncorpus linguistics technique. This technique enables the extraction of required\ninformation and knowledge associated with a specific domain. For this purpose,\nwe have created a large corpus, consisting of papers from very high impact\nconferences. First, we analyze this corpus in order to extract rich\ndomain-specific terminology and knowledge. Then we use the acquired knowledge\nto estimate the quality of scientific papers by applying our proposed measure.\nWe examine our measure on high and low scientific impact test corpora. Our\nresults show a significant difference in the measure scores of the high and low\nimpact test corpora. Second, we develop a classifier based on our proposed\nmeasure and compare it to the baseline classifier. Our results show that the\nclassifier based on our measure over-performed the baseline classifier. Based\non the presented results the proposed measure and the technique can be used for\nautomated assessment of scientific papers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.04319v2",
            "date": "2019-08-12 18:09:04+00:00",
            "title": "Neural Text Generation with Unlikelihood Training",
            "authors": [
                "Sean Welleck",
                "Ilia Kulikov",
                "Stephen Roller",
                "Emily Dinan",
                "Kyunghyun Cho",
                "Jason Weston"
            ],
            "abstract": "Neural text generation is a key tool in natural language applications, but it\nis well known there are major problems at its core. In particular, standard\nlikelihood training and decoding leads to dull and repetitive outputs. While\nsome post-hoc fixes have been proposed, in particular top-$k$ and nucleus\nsampling, they do not address the fact that the token-level probabilities\npredicted by the model are poor. In this paper we show that the likelihood\nobjective itself is at fault, resulting in a model that assigns too much\nprobability to sequences containing repeats and frequent words, unlike those\nfrom the human training distribution. We propose a new objective, unlikelihood\ntraining, which forces unlikely generations to be assigned lower probability by\nthe model. We show that both token and sequence level unlikelihood training\ngive less repetitive, less dull text while maintaining perplexity, giving\nsuperior generations using standard greedy or beam search. According to human\nevaluations, our approach with standard beam search also outperforms the\ncurrently popular decoding methods of nucleus sampling or beam blocking, thus\nproviding a strong alternative to existing techniques.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.04369v4",
            "date": "2019-08-12 20:25:41+00:00",
            "title": "Wasserstein Index Generation Model: Automatic Generation of Time-series Index with Application to Economic Policy Uncertainty",
            "authors": [
                "Fangzhou Xie"
            ],
            "abstract": "I propose a novel method, the Wasserstein Index Generation model (WIG), to\ngenerate a public sentiment index automatically. To test the model`s\neffectiveness, an application to generate Economic Policy Uncertainty (EPU)\nindex is showcased.",
            "journal ref": "None",
            "category": "econ.GN"
        },
        {
            "url": "http://arxiv.org/abs/1908.04737v1",
            "date": "2019-08-13 16:56:41+00:00",
            "title": "End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning",
            "authors": [
                "Pavel Denisov",
                "Ngoc Thang Vu"
            ],
            "abstract": "This paper presents our latest investigation on end-to-end automatic speech\nrecognition (ASR) for overlapped speech. We propose to train an end-to-end\nsystem conditioned on speaker embeddings and further improved by transfer\nlearning from clean speech. This proposed framework does not require any\nparallel non-overlapped speech materials and is independent of the number of\nspeakers. Our experimental results on overlapped speech datasets show that\njoint conditioning on speaker embeddings and transfer learning significantly\nimproves the ASR performance.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.04743v1",
            "date": "2019-08-13 17:03:24+00:00",
            "title": "IMS-Speech: A Speech to Text Tool",
            "authors": [
                "Pavel Denisov",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present the IMS-Speech, a web based tool for German and English speech\ntranscription aiming to facilitate research in various disciplines which\nrequire accesses to lexical information in spoken language materials. This tool\nis based on modern open source software stack, advanced speech recognition\nmethods and public data resources and is freely available for academic\nresearchers. The utilized models are built to be generic in order to provide\ntranscriptions of competitive accuracy on a diverse set of tasks and\nconditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04911v2",
            "date": "2019-08-14 01:38:07+00:00",
            "title": "Architecture and evolution of semantic networks in mathematics texts",
            "authors": [
                "Nicolas H. Christianson",
                "Ann Sizemore Blevins",
                "Danielle S. Bassett"
            ],
            "abstract": "Knowledge is a network of interconnected concepts. Yet, precisely how the\ntopological structure of knowledge constrains its acquisition remains unknown,\nhampering the development of learning enhancement strategies. Here we study the\ntopological structure of semantic networks reflecting mathematical concepts and\ntheir relations in college-level linear algebra texts. We hypothesize that\nthese networks will exhibit structural order, reflecting the logical sequence\nof topics that ensures accessibility. We find that the networks exhibit strong\ncore-periphery architecture, where a dense core of concepts presented early is\ncomplemented with a sparse periphery presented evenly throughout the\nexposition; the latter is composed of many small modules each reflecting more\nnarrow domains. Using tools from applied topology, we find that the\nexpositional evolution of the semantic networks produces and subsequently fills\nknowledge gaps, and that the density of these gaps tracks negatively with\ncommunity ratings of each textbook. Broadly, our study lays the groundwork for\nfuture efforts developing optimal design principles for textbook exposition and\nteaching in a classroom setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.04950v1",
            "date": "2019-08-14 04:44:26+00:00",
            "title": "VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering",
            "authors": [
                "C\u0103t\u0103lina Cangea",
                "Eugene Belilovsky",
                "Pietro Li\u00f2",
                "Aaron Courville"
            ],
            "abstract": "Embodied Question Answering (EQA) is a recently proposed task, where an agent\nis placed in a rich 3D environment and must act based solely on its egocentric\ninput to answer a given question. The desired outcome is that the agent learns\nto combine capabilities such as scene understanding, navigation and language\nunderstanding in order to perform complex reasoning in the visual world.\nHowever, initial advancements combining standard vision and language methods\nwith imitation and reinforcement learning algorithms have shown EQA might be\ntoo complex and challenging for these techniques. In order to investigate the\nfeasibility of EQA-type tasks, we build the VideoNavQA dataset that contains\npairs of questions and videos generated in the House3D environment. The goal of\nthis dataset is to assess question-answering performance from nearly-ideal\nnavigation paths, while considering a much more complete variety of questions\nthan current instantiations of the EQA task. We investigate several models,\nadapted from popular VQA methods, on this new benchmark. This establishes an\ninitial understanding of how well VQA-style methods can perform within this\nnovel EQA paradigm.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.05054v2",
            "date": "2019-08-14 10:03:12+00:00",
            "title": "Fusion of Detected Objects in Text for Visual Question Answering",
            "authors": [
                "Chris Alberti",
                "Jeffrey Ling",
                "Michael Collins",
                "David Reitter"
            ],
            "abstract": "To advance models of multimodal context, we introduce a simple yet powerful\nneural architecture for data that combines vision and natural language. The\n\"Bounding Boxes in Text Transformer\" (B2T2) also leverages referential\ninformation binding words to portions of the image in a single unified\narchitecture. B2T2 is highly effective on the Visual Commonsense Reasoning\nbenchmark (https://visualcommonsense.com), achieving a new state-of-the-art\nwith a 25% relative reduction in error rate compared to published baselines and\nobtaining the best performance to date on the public leaderboard (as of May 22,\n2019). A detailed ablation analysis shows that the early integration of the\nvisual features into the text analysis is key to the effectiveness of the new\narchitecture. A reference implementation of our models is provided\n(https://github.com/google-research/language/tree/master/language/question_answering/b2t2).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05135v1",
            "date": "2019-08-14 14:10:21+00:00",
            "title": "Mastering emergent language: learning to guide in simulated navigation",
            "authors": [
                "Mathijs Mul",
                "Diane Bouchacourt",
                "Elia Bruni"
            ],
            "abstract": "To cooperate with humans effectively, virtual agents need to be able to\nunderstand and execute language instructions. A typical setup to achieve this\nis with a scripted teacher which guides a virtual agent using language\ninstructions. However, such setup has clear limitations in scalability and,\nmore importantly, it is not interactive. Here, we introduce an autonomous agent\nthat uses discrete communication to interactively guide other agents to\nnavigate and act on a simulated environment. The developed communication\nprotocol is trainable, emergent and requires no additional supervision. The\nemergent language speeds up learning of new agents, it generalizes across\nincrementally more difficult tasks and, contrary to most other emergent\nlanguages, it is highly interpretable. We demonstrate how the emitted messages\ncorrelate with particular actions and observations, and how new agents become\nless dependent on this guidance as training progresses. By exploiting the\ncorrelations identified in our analysis, we manage to successfully address the\nagents in their own language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05161v3",
            "date": "2019-08-14 15:06:48+00:00",
            "title": "Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding",
            "authors": [
                "Oren Barkan",
                "Noam Razin",
                "Itzik Malkiel",
                "Ori Katz",
                "Avi Caciularu",
                "Noam Koenigstein"
            ],
            "abstract": "Recent state-of-the-art natural language understanding models, such as BERT\nand XLNet, score a pair of sentences (A and B) using multiple cross-attention\noperations - a process in which each word in sentence A attends to all words in\nsentence B and vice versa. As a result, computing the similarity between a\nquery sentence and a set of candidate sentences, requires the propagation of\nall query-candidate sentence-pairs throughout a stack of cross-attention\nlayers. This exhaustive process becomes computationally prohibitive when the\nnumber of candidate sentences is large. In contrast, sentence embedding\ntechniques learn a sentence-to-vector mapping and compute the similarity\nbetween the sentence vectors via simple elementary operations. In this paper,\nwe introduce Distilled Sentence Embedding (DSE) - a model that is based on\nknowledge distillation from cross-attentive models, focusing on sentence-pair\ntasks. The outline of DSE is as follows: Given a cross-attentive teacher model\n(e.g. a fine-tuned BERT), we train a sentence embedding based student model to\nreconstruct the sentence-pair scores obtained by the teacher model. We\nempirically demonstrate the effectiveness of DSE on five GLUE sentence-pair\ntasks. DSE significantly outperforms several ELMO variants and other sentence\nembedding methods, while accelerating computation of the query-candidate\nsentence-pairs similarities by several orders of magnitude, with an average\nrelative degradation of 4.6% compared to BERT. Furthermore, we show that DSE\nproduces sentence embeddings that reach state-of-the-art performance on\nuniversal sentence representation benchmarks. Our code is made publicly\navailable at https://github.com/microsoft/Distilled-Sentence-Embedding.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.05391v2",
            "date": "2019-08-15 01:49:19+00:00",
            "title": "Towards Knowledge-Based Recommender Dialog System",
            "authors": [
                "Qibin Chen",
                "Junyang Lin",
                "Yichang Zhang",
                "Ming Ding",
                "Yukuo Cen",
                "Hongxia Yang",
                "Jie Tang"
            ],
            "abstract": "In this paper, we propose a novel end-to-end framework called KBRD, which\nstands for Knowledge-Based Recommender Dialog System. It integrates the\nrecommender system and the dialog generation system. The dialog system can\nenhance the performance of the recommendation system by introducing\nknowledge-grounded information about users' preferences, and the recommender\nsystem can improve that of the dialog generation system by providing\nrecommendation-aware vocabulary bias. Experimental results demonstrate that our\nproposed model has significant advantages over the baselines in both the\nevaluation of dialog generation and recommendation. A series of analyses show\nthat the two systems can bring mutual benefits to each other, and the\nintroduced knowledge contributes to both their performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05426v1",
            "date": "2019-08-15 05:38:14+00:00",
            "title": "Feature-Less End-to-End Nested Term Extraction",
            "authors": [
                "Yuze Gao",
                "Yu Yuan"
            ],
            "abstract": "In this paper, we proposed a deep learning-based end-to-end method on the\ndomain specified automatic term extraction (ATE), it considers possible term\nspans within a fixed length in the sentence and predicts them whether they can\nbe conceptual terms. In comparison with current ATE methods, the model supports\nnested term extraction and does not crucially need extra (extracted) features.\nResults show that it can achieve high recall and a comparable precision on term\nextraction task with inputting segmented raw text.",
            "journal ref": "NLPCC XAI 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05434v2",
            "date": "2019-08-15 06:25:46+00:00",
            "title": "Sex Trafficking Detection with Ordinal Regression Neural Networks",
            "authors": [
                "Longshaokan Wang",
                "Eric Laber",
                "Yeng Saanchi",
                "Sherrie Caltagirone"
            ],
            "abstract": "Sex trafficking is a global epidemic. Escort websites are a primary vehicle\nfor selling the services of such trafficking victims and thus a major driver of\ntrafficker revenue. Many law enforcement agencies do not have the resources to\nmanually identify leads from the millions of escort ads posted across dozens of\npublic websites. We propose an ordinal regression neural network to identify\nescort ads that are likely linked to sex trafficking. Our model uses a modified\ncost function to mitigate inconsistencies in predictions often associated with\nnonparametric ordinal regression and leverages recent advancements in deep\nlearning to improve prediction accuracy. The proposed method significantly\nimproves on the previous state-of-the-art on Trafficking-10K, an\nexpert-annotated dataset of escort ads. Additionally, because traffickers use\nacronyms, deliberate typographical errors, and emojis to replace explicit\nkeywords, we demonstrate how to expand the lexicon of trafficking flags through\nword embeddings and t-SNE.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.05541v1",
            "date": "2019-08-15 13:51:12+00:00",
            "title": "Hamming Sentence Embeddings for Information Retrieval",
            "authors": [
                "Felix Hamann",
                "Nadja Kurz",
                "Adrian Ulges"
            ],
            "abstract": "In retrieval applications, binary hashes are known to offer significant\nimprovements in terms of both memory and speed. We investigate the compression\nof sentence embeddings using a neural encoder-decoder architecture, which is\ntrained by minimizing reconstruction error. Instead of employing the original\nreal-valued embeddings, we use latent representations in Hamming space produced\nby the encoder for similarity calculations.\n  In quantitative experiments on several benchmarks for semantic similarity\ntasks, we show that our compressed hamming embeddings yield a comparable\nperformance to uncompressed embeddings (Sent2Vec, InferSent, Glove-BoW), at\ncompression ratios of up to 256:1. We further demonstrate that our model\nstrongly decorrelates input features, and that the compressor generalizes well\nwhen pre-trained on Wikipedia sentences. We publish the source code on Github\nand all experimental results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.05596v1",
            "date": "2019-08-14 14:06:11+00:00",
            "title": "Two-stage Federated Phenotyping and Patient Representation Learning",
            "authors": [
                "Dianbo Liu",
                "Dmitriy Dligach",
                "Timothy Miller"
            ],
            "abstract": "A large percentage of medical information is in unstructured text format in\nelectronic medical record systems. Manual extraction of information from\nclinical notes is extremely time consuming. Natural language processing has\nbeen widely used in recent years for automatic information extraction from\nmedical texts. However, algorithms trained on data from a single healthcare\nprovider are not generalizable and error-prone due to the heterogeneity and\nuniqueness of medical documents. We develop a two-stage federated natural\nlanguage processing method that enables utilization of clinical notes from\ndifferent hospitals or clinics without moving the data, and demonstrate its\nperformance using obesity and comorbities phenotyping as medical task. This\napproach not only improves the quality of a specific clinical task but also\nfacilitates knowledge progression in the whole healthcare system, which is an\nessential part of learning health system. To the best of our knowledge, this is\nthe first application of federated machine learning in clinical NLP.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.05762v2",
            "date": "2019-08-14 03:51:25+00:00",
            "title": "Entity-aware ELMo: Learning Contextual Entity Representation for Entity Disambiguation",
            "authors": [
                "Hamed Shahbazi",
                "Xiaoli Z. Fern",
                "Reza Ghaeini",
                "Rasha Obeidat",
                "Prasad Tadepalli"
            ],
            "abstract": "We present a new local entity disambiguation system. The key to our system is\na novel approach for learning entity representations. In our approach we learn\nan entity aware extension of Embedding for Language Model (ELMo) which we call\nEntity-ELMo (E-ELMo). Given a paragraph containing one or more named entity\nmentions, each mention is first defined as a function of the entire paragraph\n(including other mentions), then they predict the referent entities. Utilizing\nE-ELMo for local entity disambiguation, we outperform all of the\nstate-of-the-art local and global models on the popular benchmarks by improving\nabout 0.5\\% on micro average accuracy for AIDA test-b with Yago candidate set.\nThe evaluation setup of the training data and candidate set are the same as our\nbaselines for fair comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.05780v1",
            "date": "2019-08-15 21:54:06+00:00",
            "title": "Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review",
            "authors": [
                "Seyedmostafa Sheikhalishahi",
                "Riccardo Miotto",
                "Joel T Dudley",
                "Alberto Lavelli",
                "Fabio Rinaldi",
                "Venet Osmani"
            ],
            "abstract": "Of the 2652 articles considered, 106 met the inclusion criteria. Review of\nthe included papers resulted in identification of 43 chronic diseases, which\nwere then further classified into 10 disease categories using ICD-10. The\nmajority of studies focused on diseases of the circulatory system (n=38) while\nendocrine and metabolic diseases were fewest (n=14). This was due to the\nstructure of clinical records related to metabolic diseases, which typically\ncontain much more structured data, compared with medical records for diseases\nof the circulatory system, which focus more on unstructured data and\nconsequently have seen a stronger focus of NLP. The review has shown that there\nis a significant increase in the use of machine learning methods compared to\nrule-based approaches; however, deep learning methods remain emergent (n=3).\nConsequently, the majority of works focus on classification of disease\nphenotype with only a handful of papers addressing extraction of comorbidities\nfrom the free text or integration of clinical notes with structured data. There\nis a notable use of relatively simple methods, such as shallow classifiers (or\ncombination with rule-based methods), due to the interpretability of\npredictions, which still represents a significant issue for more complex\nmethods. Finally, scarcity of publicly available data may also have contributed\nto insufficient development of more advanced methods, such as extraction of\nword embeddings from clinical notes. Further efforts are still required to\nimprove (1) progression of clinical NLP methods from extraction toward\nunderstanding; (2) recognition of relations among entities rather than entities\nin isolation; (3) temporal extraction to understand past, current, and future\nclinical events; (4) exploitation of alternative sources of clinical knowledge;\nand (5) availability of large-scale, de-identified clinical corpora.",
            "journal ref": "JMIR Medical Informatics 2019;7(2):e12239, PMID:31066697",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1908.05787v3",
            "date": "2019-08-15 22:51:21+00:00",
            "title": "Integrating Multimodal Information in Large Pretrained Transformers",
            "authors": [
                "Wasifur Rahman",
                "Md. Kamrul Hasan",
                "Sangwu Lee",
                "Amir Zadeh",
                "Chengfeng Mao",
                "Louis-Philippe Morency",
                "Ehsan Hoque"
            ],
            "abstract": "Recent Transformer-based contextual word representations, including BERT and\nXLNet, have shown state-of-the-art performance in multiple disciplines within\nNLP. Fine-tuning the trained contextual models on task-specific datasets has\nbeen the key to achieving superior performance downstream. While fine-tuning\nthese pre-trained models is straightforward for lexical applications\n(applications with only language modality), it is not trivial for multimodal\nlanguage (a growing area in NLP focused on modeling face-to-face\ncommunication). Pre-trained models don't have the necessary components to\naccept two extra modalities of vision and acoustic. In this paper, we proposed\nan attachment to BERT and XLNet called Multimodal Adaptation Gate (MAG). MAG\nallows BERT and XLNet to accept multimodal nonverbal data during fine-tuning.\nIt does so by generating a shift to internal representation of BERT and XLNet;\na shift that is conditioned on the visual and acoustic modalities. In our\nexperiments, we study the commonly used CMU-MOSI and CMU-MOSEI datasets for\nmultimodal sentiment analysis. Fine-tuning MAG-BERT and MAG-XLNet significantly\nboosts the sentiment analysis performance over previous baselines as well as\nlanguage-only fine-tuning of BERT and XLNet. On the CMU-MOSI dataset, MAG-XLNet\nachieves human-level multimodal sentiment analysis performance for the first\ntime in the NLP community.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.05915v2",
            "date": "2019-08-16 10:00:45+00:00",
            "title": "Attending to Future Tokens For Bidirectional Sequence Generation",
            "authors": [
                "Carolin Lawrence",
                "Bhushan Kotnis",
                "Mathias Niepert"
            ],
            "abstract": "Neural sequence generation is typically performed token-by-token and\nleft-to-right. Whenever a token is generated only previously produced tokens\nare taken into consideration. In contrast, for problems such as sequence\nclassification, bidirectional attention, which takes both past and future\ntokens into consideration, has been shown to perform much better. We propose to\nmake the sequence generation process bidirectional by employing special\nplaceholder tokens. Treated as a node in a fully connected graph, a placeholder\ntoken can take past and future tokens into consideration when generating the\nactual output token. We verify the effectiveness of our approach experimentally\non two conversational tasks where the proposed bidirectional model outperforms\ncompetitive baselines by a large margin.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1908.06008v1",
            "date": "2019-08-13 13:39:19+00:00",
            "title": "Variational Fusion for Multimodal Sentiment Analysis",
            "authors": [
                "Navonil Majumder",
                "Soujanya Poria",
                "Gangeshwar Krishnamurthy",
                "Niyati Chhaya",
                "Rada Mihalcea",
                "Alexander Gelbukh"
            ],
            "abstract": "Multimodal fusion is considered a key step in multimodal tasks such as\nsentiment analysis, emotion detection, question answering, and others. Most of\nthe recent work on multimodal fusion does not guarantee the fidelity of the\nmultimodal representation with respect to the unimodal representations. In this\npaper, we propose a variational autoencoder-based approach for modality fusion\nthat minimizes information loss between unimodal and multimodal\nrepresentations. We empirically show that this method outperforms the\nstate-of-the-art methods by a significant margin on several popular datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.06082v1",
            "date": "2019-08-16 20:25:13+00:00",
            "title": "Shallow Domain Adaptive Embeddings for Sentiment Analysis",
            "authors": [
                "Prathusha K Sarma",
                "Yingyu Liang",
                "William A Sethares"
            ],
            "abstract": "This paper proposes a way to improve the performance of existing algorithms\nfor text classification in domains with strong language semantics. We propose a\ndomain adaptation layer learns weights to combine a generic and a domain\nspecific (DS) word embedding into a domain adapted (DA) embedding. The DA word\nembeddings are then used as inputs to a generic encoder + classifier framework\nto perform a downstream task such as classification. This adaptation layer is\nparticularly suited to datasets that are modest in size, and which are,\ntherefore, not ideal candidates for (re)training a deep neural network\narchitecture. Results on binary and multi-class classification tasks using\npopular encoder architectures, including current state-of-the-art methods (with\nand without the shallow adaptation layer) show the effectiveness of the\nproposed approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.06177v2",
            "date": "2019-08-16 21:12:15+00:00",
            "title": "CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text",
            "authors": [
                "Koustuv Sinha",
                "Shagun Sodhani",
                "Jin Dong",
                "Joelle Pineau",
                "William L. Hamilton"
            ],
            "abstract": "The recent success of natural language understanding (NLU) systems has been\ntroubled by results highlighting the failure of these models to generalize in a\nsystematic and robust way. In this work, we introduce a diagnostic benchmark\nsuite, named CLUTRR, to clarify some key issues related to the robustness and\nsystematicity of NLU systems. Motivated by classic work on inductive logic\nprogramming, CLUTRR requires that an NLU system infer kinship relations between\ncharacters in short stories. Successful performance on this task requires both\nextracting relationships between entities, as well as inferring the logical\nrules governing these relationships. CLUTRR allows us to precisely measure a\nmodel's ability for systematic generalization by evaluating on held-out\ncombinations of logical rules, and it allows us to evaluate a model's\nrobustness by adding curated noise facts. Our empirical results highlight a\nsubstantial performance gap between state-of-the-art NLU models (e.g., BERT and\nMAC) and a graph neural network model that works directly with symbolic\ninputs---with the graph-based model exhibiting both stronger generalization and\ngreater robustness.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.06306v4",
            "date": "2019-08-17 14:39:36+00:00",
            "title": "U-CAM: Visual Explanation using Uncertainty based Class Activation Maps",
            "authors": [
                "Badri N. Patro",
                "Mayank Lunayach",
                "Shivansh Patel",
                "Vinay P. Namboodiri"
            ],
            "abstract": "Understanding and explaining deep learning models is an imperative task.\nTowards this, we propose a method that obtains gradient-based certainty\nestimates that also provide visual attention maps. Particularly, we solve for\nvisual question answering task. We incorporate modern probabilistic deep\nlearning methods that we further improve by using the gradients for these\nestimates. These have two-fold benefits: a) improvement in obtaining the\ncertainty estimates that correlate better with misclassified samples and b)\nimproved attention maps that provide state-of-the-art results in terms of\ncorrelation with human attention regions. The improved attention maps result in\nconsistent improvement for various methods for visual question answering.\nTherefore, the proposed technique can be thought of as a recipe for obtaining\nimproved certainty estimates and explanation for deep learning models. We\nprovide detailed empirical analysis for the visual question answering task on\nall standard benchmarks and comparison with state of the art methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.06336v2",
            "date": "2019-08-17 20:12:39+00:00",
            "title": "What is needed for simple spatial language capabilities in VQA?",
            "authors": [
                "Alexander Kuhnle",
                "Ann Copestake"
            ],
            "abstract": "Visual question answering (VQA) comprises a variety of language capabilities.\nThe diagnostic benchmark dataset CLEVR has fueled progress by helping to better\nassess and distinguish models in basic abilities like counting, comparing and\nspatial reasoning in vitro. Following this approach, we focus on spatial\nlanguage capabilities and investigate the question: what are the key\ningredients to handle simple visual-spatial relations? We look at the SAN,\nRelNet, FiLM and MC models and evaluate their learning behavior on diagnostic\ndata which is solely focused on spatial relations. Via comparative analysis and\ntargeted model modification we identify what really is required to\nsubstantially improve upon the CNN-LSTM baseline.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.06449v2",
            "date": "2019-08-18 14:49:16+00:00",
            "title": "RefNet: A Reference-aware Network for Background Based Conversation",
            "authors": [
                "Chuan Meng",
                "Pengjie Ren",
                "Zhumin Chen",
                "Christof Monz",
                "Jun Ma",
                "Maarten de Rijke"
            ],
            "abstract": "Existing conversational systems tend to generate generic responses. Recently,\nBackground Based Conversations (BBCs) have been introduced to address this\nissue. Here, the generated responses are grounded in some background\ninformation. The proposed methods for BBCs are able to generate more\ninformative responses, they either cannot generate natural responses or have\ndifficulty in locating the right background information. In this paper, we\npropose a Reference-aware Network (RefNet) to address the two issues. Unlike\nexisting methods that generate responses token by token, RefNet incorporates a\nnovel reference decoder that provides an alternative way to learn to directly\ncite a semantic unit (e.g., a span containing complete semantic information)\nfrom the background. Experimental results show that RefNet significantly\noutperforms state-of-the-art methods in terms of both automatic and human\nevaluations, indicating that RefNet can generate more appropriate and\nhuman-like responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06556v1",
            "date": "2019-08-19 01:52:00+00:00",
            "title": "Transfer in Deep Reinforcement Learning using Knowledge Graphs",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Mark O. Riedl"
            ],
            "abstract": "Text adventure games, in which players must make sense of the world through\ntext descriptions and declare actions through text descriptions, provide a\nstepping stone toward grounding action in language. Prior work has demonstrated\nthat using a knowledge graph as a state representation and question-answering\nto pre-train a deep Q-network facilitates faster control policy transfer. In\nthis paper, we explore the use of knowledge graphs as a representation for\ndomain knowledge transfer for training text-adventure playing reinforcement\nlearning agents. Our methods are tested across multiple computer generated and\nhuman authored games, varying in domain and complexity, and demonstrate that\nour transfer learning methods let us learn a higher-quality control policy\nfaster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06601v2",
            "date": "2019-08-19 05:43:13+00:00",
            "title": "Implicit Recursive Characteristics of STOP",
            "authors": [
                "Mike H. Ji"
            ],
            "abstract": "The most important notations of Communicating Sequential Process(CSP) are the\nprocess and the prefix (event)$\\rightarrow$(process) operator. While we can\nformally apply the $\\rightarrow$ operator to define a live process's behavior,\nthe STOP process, which usually resulted from deadlock, starving or livelock,\nis lack of formal description, defined by most literatures as \"doing nothing\nbut halt\". In this paper, we argue that the STOP process should not be\nconsidered as a black box, it should follow the prefix $\\rightarrow$ schema and\nthe same inference rules so that a unified and consistent process algebra model\ncan be established. In order to achieve this goal, we introduce a special event\ncalled \"nil\" that any process can take. This nil event will do nothing\nmeaningful and leave nothing on a process's observable record. With the nil\nevent and its well-defined rules, we can successfully use the $\\rightarrow$\noperator to formally describe a process's complete behavior in its whole life\ncircle. More interestingly, we can use prefix $\\rightarrow$ and nil event to\nfully describe the STOP process's internal behavior and conclude that the\nSTOP's formal equation can be given as simple as STOP$_{\\alpha X} = \\mu$ X. nil\n$\\rightarrow$ X.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1908.06709v1",
            "date": "2019-08-19 11:45:11+00:00",
            "title": "Two-Staged Acoustic Modeling Adaption for Robust Speech Recognition by the Example of German Oral History Interviews",
            "authors": [
                "Michael Gref",
                "Christoph Schmidt",
                "Sven Behnke",
                "Joachim K\u00f6hler"
            ],
            "abstract": "In automatic speech recognition, often little training data is available for\nspecific challenging tasks, but training of state-of-the-art automatic speech\nrecognition systems requires large amounts of annotated speech. To address this\nissue, we propose a two-staged approach to acoustic modeling that combines\nnoise and reverberation data augmentation with transfer learning to robustly\naddress challenges such as difficult acoustic recording conditions, spontaneous\nspeech, and speech of elderly people. We evaluate our approach using the\nexample of German oral history interviews, where a relative average reduction\nof the word error rate by 19.3% is achieved.",
            "journal ref": "IEEE International Conference on Multimedia and Expo (ICME),\n  Shanghai, China, July 2019",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.06917v1",
            "date": "2019-08-19 16:31:29+00:00",
            "title": "Message Passing for Complex Question Answering over Knowledge Graphs",
            "authors": [
                "Svitlana Vakulenko",
                "Javier David Fernandez Garcia",
                "Axel Polleres",
                "Maarten de Rijke",
                "Michael Cochez"
            ],
            "abstract": "Question answering over knowledge graphs (KGQA) has evolved from simple\nsingle-fact questions to complex questions that require graph traversal and\naggregation. We propose a novel approach for complex KGQA that uses\nunsupervised message passing, which propagates confidence scores obtained by\nparsing an input question and matching terms in the knowledge graph to a set of\npossible answers. First, we identify entity, relationship, and class names\nmentioned in a natural language question, and map these to their counterparts\nin the graph. Then, the confidence scores of these mappings propagate through\nthe graph structure to locate the answer entities. Finally, these are\naggregated depending on the identified question type. This approach can be\nefficiently implemented as a series of sparse matrix multiplications mimicking\njoins over small local subgraphs. Our evaluation results show that the proposed\napproach outperforms the state-of-the-art on the LC-QuAD benchmark. Moreover,\nwe show that the performance of the approach depends only on the quality of the\nquestion interpretation results, i.e., given a correct relevance score\ndistribution, our approach always produces a correct answer ranking. Our error\nanalysis reveals correct answers missing from the benchmark dataset and\ninconsistencies in the DBpedia knowledge graph. Finally, we provide a\ncomprehensive evaluation of the proposed approach accompanied with an ablation\nstudy and an error analysis, which showcase the pitfalls for each of the\nquestion answering components in more detail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07018v3",
            "date": "2019-08-19 18:43:06+00:00",
            "title": "Tale of tails using rule augmented sequence labeling for event extraction",
            "authors": [
                "Ayush Maheshwari",
                "Hrishikesh Patel",
                "Nandan Rathod",
                "Ritesh Kumar",
                "Ganesh Ramakrishnan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "The problem of event extraction is a relatively difficult task for low\nresource languages due to the non-availability of sufficient annotated data.\nMoreover, the task becomes complex for tail (rarely occurring) labels wherein\nextremely less data is available. In this paper, we present a new dataset\n(InDEE-2019) in the disaster domain for multiple Indic languages, collected\nfrom news websites. Using this dataset, we evaluate several rule-based\nmechanisms to augment deep learning based models. We formulate our problem of\nevent extraction as a sequence labeling task and perform extensive experiments\nto study and understand the effectiveness of different approaches. We further\nshow that tail labels can be easily incorporated by creating new rules without\nthe requirement of large annotated data.",
            "journal ref": "StarAI Workshop at AAAI 2020",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.07064v1",
            "date": "2019-08-19 20:58:24+00:00",
            "title": "Domain-Independent turn-level Dialogue Quality Evaluation via User Satisfaction Estimation",
            "authors": [
                "Praveen Kumar Bodigutla",
                "Longshaokan Wang",
                "Kate Ridgeway",
                "Joshua Levy",
                "Swanand Joshi",
                "Alborz Geramifard",
                "Spyros Matsoukas"
            ],
            "abstract": "An automated metric to evaluate dialogue quality is vital for optimizing data\ndriven dialogue management. The common approach of relying on explicit user\nfeedback during a conversation is intrusive and sparse. Current models to\nestimate user satisfaction use limited feature sets and rely on annotation\nschemes with low inter-rater reliability, limiting generalizability to\nconversations spanning multiple domains. To address these gaps, we created a\nnew Response Quality annotation scheme, based on which we developed turn-level\nUser Satisfaction metric. We introduced five new domain-independent feature\nsets and experimented with six machine learning models to estimate the new\nsatisfaction metric.\n  Using Response Quality annotation scheme, across randomly sampled single and\nmulti-turn conversations from 26 domains, we achieved high inter-annotator\nagreement (Spearman's rho 0.94). The Response Quality labels were highly\ncorrelated (0.76) with explicit turn-level user ratings. Gradient boosting\nregression achieved best correlation of ~0.79 between predicted and annotated\nuser satisfaction labels. Multi Layer Perceptron and Gradient Boosting\nregression models generalized to an unseen domain better (linear correlation\n0.67) than other models. Finally, our ablation study verified that our novel\nfeatures significantly improved model performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.07069v1",
            "date": "2019-08-16 13:01:30+00:00",
            "title": "CommentsRadar: Dive into Unique Data on All Comments on the Web",
            "authors": [
                "Sergey Nikolenko",
                "Elena Tutubalina",
                "Zulfat Miftahutdinov",
                "Eugene Beloded"
            ],
            "abstract": "We introduce an entity-centric search engineCommentsRadarthatpairs entity\nqueries with articles and user opinions covering a widerange of topics from top\ncommented sites. The engine aggregatesarticles and comments for these articles,\nextracts named entities,links them together and with knowledge base entries,\nperformssentiment analysis, and aggregates the results, aiming to mine\nfortemporal trends and other insights. In this work, we present thegeneral\nengine, discuss the models used for all steps of this pipeline,and introduce\nseveral case studies that discover important insightsfrom online commenting\ndata.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.07226v1",
            "date": "2019-08-20 08:52:52+00:00",
            "title": "Prosodic Phrase Alignment for Machine Dubbing",
            "authors": [
                "Alp \u00d6ktem",
                "Mireia Farr\u00fas",
                "Antonio Bonafonte"
            ],
            "abstract": "Dubbing is a type of audiovisual translation where dialogues are translated\nand enacted so that they give the impression that the media is in the target\nlanguage. It requires a careful alignment of dubbed recordings with the lip\nmovements of performers in order to achieve visual coherence. In this paper, we\ndeal with the specific problem of prosodic phrase synchronization within the\nframework of machine dubbing. Our methodology exploits the attention mechanism\noutput in neural machine translation to find plausible phrasing for the\ntranslated dialogue lines and then uses them to condition their synthesis. Our\ninitial work in this field records comparable speech rate ratio to professional\ndubbing translation, and improvement in terms of lip-syncing of long dialogue\nlines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07281v1",
            "date": "2019-08-20 11:40:16+00:00",
            "title": "Unsupervised Hierarchical Grouping of Knowledge Graph Entities",
            "authors": [
                "Sameh K. Mohamed"
            ],
            "abstract": "Knowledge graphs have attracted lots of attention in academic and industrial\nenvironments. Despite their usefulness, popular knowledge graphs suffer from\nincompleteness of information, especially in their type assertions. This has\nencouraged research in the automatic discovery of entity types. In this\ncontext, multiple works were developed to utilize logical inference on\nontologies and statistical machine learning methods to learn type assertion in\nknowledge graphs. However, these approaches suffer from limited performance on\nnoisy data, limited scalability and the dependence on labeled training samples.\nIn this work, we propose a new unsupervised approach that learns to categorize\nentities into a hierarchy of named groups. We show that our approach is able to\neffectively learn entity groups using a scalable procedure in noisy and sparse\ndatasets. We experiment our approach on a set of popular knowledge graph\nbenchmarking datasets, and we publish a collection of the outcome group\nhierarchies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.07490v3",
            "date": "2019-08-20 17:05:18+00:00",
            "title": "LXMERT: Learning Cross-Modality Encoder Representations from Transformers",
            "authors": [
                "Hao Tan",
                "Mohit Bansal"
            ],
            "abstract": "Vision-and-language reasoning requires an understanding of visual concepts,\nlanguage semantics, and, most importantly, the alignment and relationships\nbetween these two modalities. We thus propose the LXMERT (Learning\nCross-Modality Encoder Representations from Transformers) framework to learn\nthese vision-and-language connections. In LXMERT, we build a large-scale\nTransformer model that consists of three encoders: an object relationship\nencoder, a language encoder, and a cross-modality encoder. Next, to endow our\nmodel with the capability of connecting vision and language semantics, we\npre-train the model with large amounts of image-and-sentence pairs, via five\ndiverse representative pre-training tasks: masked language modeling, masked\nobject prediction (feature regression and label classification), cross-modality\nmatching, and image question answering. These tasks help in learning both\nintra-modality and cross-modality relationships. After fine-tuning from our\npre-trained parameters, our model achieves the state-of-the-art results on two\nvisual question answering datasets (i.e., VQA and GQA). We also show the\ngeneralizability of our pre-trained cross-modality model by adapting it to a\nchallenging visual-reasoning task, NLVR2, and improve the previous best result\nby 22% absolute (54% to 76%). Lastly, we demonstrate detailed ablation studies\nto prove that both our novel model components and pre-training strategies\nsignificantly contribute to our strong results; and also present several\nattention visualizations for the different encoders. Code and pre-trained\nmodels publicly available at: https://github.com/airsplay/lxmert",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07553v1",
            "date": "2019-08-20 18:07:37+00:00",
            "title": "Phrase Localization Without Paired Training Examples",
            "authors": [
                "Josiah Wang",
                "Lucia Specia"
            ],
            "abstract": "Localizing phrases in images is an important part of image understanding and\ncan be useful in many applications that require mappings between textual and\nvisual information. Existing work attempts to learn these mappings from\nexamples of phrase-image region correspondences (strong supervision) or from\nphrase-image pairs (weak supervision). We postulate that such paired\nannotations are unnecessary, and propose the first method for the phrase\nlocalization problem where neither training procedure nor paired, task-specific\ndata is required. Our method is simple but effective: we use off-the-shelf\napproaches to detect objects, scenes and colours in images, and explore\ndifferent approaches to measure semantic similarity between the categories of\ndetected visual elements and words in phrases. Experiments on two well-known\nphrase localization datasets show that this approach surpasses all weakly\nsupervised methods by a large margin and performs very competitively to\nstrongly supervised methods, and can thus be considered a strong baseline to\nthe task. The non-paired nature of our method makes it applicable to any domain\nand where no paired phrase localization annotation is available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.07590v1",
            "date": "2019-08-20 20:10:05+00:00",
            "title": "From Text to Sound: A Preliminary Study on Retrieving Sound Effects to Radio Stories",
            "authors": [
                "Songwei Ge",
                "Curtis Xuan",
                "Ruihua Song",
                "Chao Zou",
                "Wei Liu",
                "Jin Zhou"
            ],
            "abstract": "Sound effects play an essential role in producing high-quality radio stories\nbut require enormous labor cost to add. In this paper, we address the problem\nof automatically adding sound effects to radio stories with a retrieval-based\nmodel. However, directly implementing a tag-based retrieval model leads to high\nfalse positives due to the ambiguity of story contents. To solve this problem,\nwe introduce a retrieval-based framework hybridized with a semantic inference\nmodel which helps to achieve robust retrieval results. Our model relies on\nfine-designed features extracted from the context of candidate triggers. We\ncollect two story dubbing datasets through crowdsourcing to analyze the setting\nof adding sound effects and to train and test our proposed methods. We further\ndiscuss the importance of each feature and introduce several heuristic rules\nfor the trade-off between precision and recall. Together with the\ntext-to-speech technology, our results reveal a promising automatic pipeline on\nproducing high-quality radio stories.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.07795v2",
            "date": "2019-08-21 11:07:14+00:00",
            "title": "Dialog State Tracking with Reinforced Data Augmentation",
            "authors": [
                "Yichun Yin",
                "Lifeng Shang",
                "Xin Jiang",
                "Xiao Chen",
                "Qun Liu"
            ],
            "abstract": "Neural dialog state trackers are generally limited due to the lack of\nquantity and diversity of annotated training data. In this paper, we address\nthis difficulty by proposing a reinforcement learning (RL) based framework for\ndata augmentation that can generate high-quality data to improve the neural\nstate tracker. Specifically, we introduce a novel contextual bandit generator\nto learn fine-grained augmentation policies that can generate new effective\ninstances by choosing suitable replacements for the specific context. Moreover,\nby alternately learning between the generator and the state tracker, we can\nkeep refining the generative policies to generate more high-quality training\ndata for neural state tracker. Experimental results on the WoZ and MultiWoZ\n(restaurant) datasets demonstrate that the proposed framework significantly\nimproves the performance over the state-of-the-art models, especially with\nlimited training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07816v3",
            "date": "2019-08-15 12:52:53+00:00",
            "title": "A Multi-Turn Emotionally Engaging Dialog Model",
            "authors": [
                "Yubo Xie",
                "Ekaterina Svikhnushina",
                "Pearl Pu"
            ],
            "abstract": "Open-domain dialog systems (also known as chatbots) have increasingly drawn\nattention in natural language processing. Some of the recent work aims at\nincorporating affect information into sequence-to-sequence neural dialog\nmodeling, making the response emotionally richer, while others use hand-crafted\nrules to determine the desired emotion response. However, they do not\nexplicitly learn the subtle emotional interactions captured in human dialogs.\nIn this paper, we propose a multi-turn dialog system aimed at learning and\ngenerating emotional responses that so far only humans know how to do. Compared\nwith two baseline models, offline experiments show that our method performs the\nbest in perplexity scores. Further human evaluations confirm that our chatbot\ncan keep track of the conversation context and generate emotionally more\nappropriate responses while performing equally well on grammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07817v1",
            "date": "2019-08-15 11:05:45+00:00",
            "title": "Disentangling Latent Emotions of Word Embeddings on Complex Emotional Narratives",
            "authors": [
                "Zhengxuan Wu",
                "Yueyi Jiang"
            ],
            "abstract": "Word embedding models such as GloVe are widely used in natural language\nprocessing (NLP) research to convert words into vectors. Here, we provide a\npreliminary guide to probe latent emotions in text through GloVe word vectors.\nFirst, we trained a neural network model to predict continuous emotion valence\nratings by taking linguistic inputs from Stanford Emotional Narratives Dataset\n(SEND). After interpreting the weights in the model, we found that only a few\ndimensions of the word vectors contributed to expressing emotions in text, and\nwords were clustered on the basis of their emotional polarities. Furthermore,\nwe performed a linear transformation that projected high dimensional embedded\nvectors into an emotion space. Based on NRC Emotion Lexicon (EmoLex), we\nvisualized the entanglement of emotions in the lexicon by using both projected\nand raw GloVe word vectors. We showed that, in the proposed emotion space, we\nwere able to better disentangle emotions than using raw GloVe vectors alone. In\naddition, we found that the sum vectors of different pairs of emotion words\nsuccessfully captured expressed human feelings in the EmoLex. For example, the\nsum of two embedded word vectors expressing Joy and Trust which express Love\nshared high similarity (similarity score .62) with the embedded vector\nexpressing Optimism. On the contrary, this sum vector was dissimilar\n(similarity score -.19) with the the embedded vector expressing Remorse. In\nthis paper, we argue that through the proposed emotion space, arithmetic of\nemotions is preserved in the word vectors. The affective representation\nuncovered in emotion vector space could shed some light on how to help machines\nto disentangle emotion expressed in word embeddings.",
            "journal ref": "NLPCC2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07818v1",
            "date": "2019-08-15 04:09:12+00:00",
            "title": "Replication of the Keyword Extraction part of the paper \"'Without the Clutter of Unimportant Words': Descriptive Keyphrases for Text Visualization\"",
            "authors": [
                "Shibamouli Lahiri"
            ],
            "abstract": "\"Keyword Extraction\" refers to the task of automatically identifying the most\nrelevant and informative phrases in natural language text. As we are deluged\nwith large amounts of text data in many different forms and content - emails,\nblogs, tweets, Facebook posts, academic papers, news articles - the task of\n\"making sense\" of all this text by somehow summarizing them into a coherent\nstructure assumes paramount importance. Keyword extraction - a well-established\nproblem in Natural Language Processing - can help us here. In this report, we\nconstruct and test three different hypotheses (all related to the task of\nkeyword extraction) that take us one step closer to understanding how to\nmeaningfully identify and extract \"descriptive\" keyphrases. The work reported\nhere was done as part of replicating the study by Chuang et al. [3].",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07822v3",
            "date": "2019-08-18 10:34:59+00:00",
            "title": "A Multi-level Neural Network for Implicit Causality Detection in Web Texts",
            "authors": [
                "Shining Liang",
                "Wanli Zuo",
                "Zhenkun Shi",
                "Sen Wang",
                "Junhu Wang",
                "Xianglin Zuo"
            ],
            "abstract": "Mining causality from text is a complex and crucial natural language\nunderstanding task corresponding to the human cognition. Existing studies at\nits solution can be grouped into two primary categories: feature engineering\nbased and neural model based methods. In this paper, we find that the former\nhas incomplete coverage and inherent errors but provide prior knowledge; while\nthe latter leverages context information but causal inference of which is\ninsufficiency. To handle the limitations, we propose a novel causality\ndetection model named MCDN to explicitly model causal reasoning process, and\nfurthermore, to exploit the advantages of both methods. Specifically, we adopt\nmulti-head self-attention to acquire semantic feature at word level and develop\nthe SCRN to infer causality at segment level. To the best of our knowledge,\nwith regards to the causality tasks, this is the first time that the Relation\nNetwork is applied. The experimental results show that: 1) the proposed\napproach performs prominent performance on causality detection; 2) further\nanalysis manifests the effectiveness and robustness of MCDN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07832v2",
            "date": "2019-08-18 00:45:16+00:00",
            "title": "Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings",
            "authors": [
                "Ahmed El-Kishky",
                "Frank Xu",
                "Aston Zhang",
                "Jiawei Han"
            ],
            "abstract": "Traditionally, many text-mining tasks treat individual word-tokens as the\nfinest meaningful semantic granularity. However, in many languages and\nspecialized corpora, words are composed by concatenating semantically\nmeaningful subword structures. Word-level analysis cannot leverage the semantic\ninformation present in such subword structures. With regard to word embedding\ntechniques, this leads to not only poor embeddings for infrequent words in\nlong-tailed text corpora but also weak capabilities for handling\nout-of-vocabulary words. In this paper we propose MorphMine for unsupervised\nmorpheme segmentation. MorphMine applies a parsimony criterion to\nhierarchically segment words into the fewest number of morphemes at each level\nof the hierarchy. This leads to longer shared morphemes at each level of\nsegmentation. Experiments show that MorphMine segments words in a variety of\nlanguages into human-verified morphemes. Additionally, we experimentally\ndemonstrate that utilizing MorphMine morphemes to enrich word embeddings\nconsistently improves embedding quality on a variety of of embedding\nevaluations and a downstream language modeling task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07844v1",
            "date": "2019-08-20 04:08:58+00:00",
            "title": "Similarity Learning for Authorship Verification in Social Media",
            "authors": [
                "Benedikt Boenninghoff",
                "Robert M. Nickel",
                "Steffen Zeiler",
                "Dorothea Kolossa"
            ],
            "abstract": "Authorship verification tries to answer the question if two documents with\nunknown authors were written by the same author or not. A range of successful\ntechnical approaches has been proposed for this task, many of which are based\non traditional linguistic features such as n-grams. These algorithms achieve\ngood results for certain types of written documents like books and novels.\nForensic authorship verification for social media, however, is a much more\nchallenging task since messages tend to be relatively short, with a large\nvariety of different genres and topics. At this point, traditional methods\nbased on features like n-grams have had limited success. In this work, we\npropose a new neural network topology for similarity learning that\nsignificantly improves the performance on the author verification task with\nsuch challenging data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07846v3",
            "date": "2019-08-19 17:28:29+00:00",
            "title": "Representing text as abstract images enables image classifiers to also simultaneously classify text",
            "authors": [
                "Stephen M. Petrie",
                "T'Mir D. Julius"
            ],
            "abstract": "We introduce a novel method for converting text data into abstract image\nrepresentations, which allows image-based processing techniques (e.g. image\nclassification networks) to be applied to text-based comparison problems. We\napply the technique to entity disambiguation of inventor names in US patents.\nThe method involves converting text from each pairwise comparison between two\ninventor name records into a 2D RGB (stacked) image representation. We then\ntrain an image classification neural network to discriminate between such\npairwise comparison images, and use the trained network to label each pair of\nrecords as either matched (same inventor) or non-matched (different inventors),\nobtaining highly accurate results. Our new text-to-image representation method\ncould also be used more broadly for other NLP comparison problems, such as\ndisambiguation of academic publications, or for problems that require\nsimultaneous classification of both text and image datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.07899v1",
            "date": "2019-08-21 14:50:13+00:00",
            "title": "Evaluating Defensive Distillation For Defending Text Processing Neural Networks Against Adversarial Examples",
            "authors": [
                "Marcus Soll",
                "Tobias Hinz",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "Adversarial examples are artificially modified input samples which lead to\nmisclassifications, while not being detectable by humans. These adversarial\nexamples are a challenge for many tasks such as image and text classification,\nespecially as research shows that many adversarial examples are transferable\nbetween different classifiers. In this work, we evaluate the performance of a\npopular defensive strategy for adversarial examples called defensive\ndistillation, which can be successful in hardening neural networks against\nadversarial examples in the image domain. However, instead of applying\ndefensive distillation to networks for image classification, we examine, for\nthe first time, its performance on text classification tasks and also evaluate\nits effect on the transferability of adversarial text examples. Our results\nindicate that defensive distillation only has a minimal impact on text\nclassifying neural networks and does neither help with increasing their\nrobustness against adversarial examples nor prevent the transferability of\nadversarial examples between neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08351v2",
            "date": "2019-08-22 13:08:26+00:00",
            "title": "Compositionality decomposed: how do neural networks generalise?",
            "authors": [
                "Dieuwke Hupkes",
                "Verna Dankers",
                "Mathijs Mul",
                "Elia Bruni"
            ],
            "abstract": "Despite a multitude of empirical studies, little consensus exists on whether\nneural networks are able to generalise compositionally, a controversy that, in\npart, stems from a lack of agreement about what it means for a neural model to\nbe compositional. As a response to this controversy, we present a set of tests\nthat provide a bridge between, on the one hand, the vast amount of linguistic\nand philosophical theory about compositionality of language and, on the other,\nthe successful neural models of language. We collect different interpretations\nof compositionality and translate them into five theoretically grounded tests\nfor models that are formulated on a task-independent level. In particular, we\nprovide tests to investigate (i) if models systematically recombine known parts\nand rules (ii) if models can extend their predictions beyond the length they\nhave seen in the training data (iii) if models' composition operations are\nlocal or global (iv) if models' predictions are robust to synonym substitutions\nand (v) if models favour rules or exceptions during training. To demonstrate\nthe usefulness of this evaluation paradigm, we instantiate these five tests on\na highly compositional data set which we dub PCFG SET and apply the resulting\ntests to three popular sequence-to-sequence models: a recurrent, a\nconvolution-based and a transformer model. We provide an in-depth analysis of\nthe results, which uncover the strengths and weaknesses of these three\narchitectures and point to potential areas of improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08507v1",
            "date": "2019-08-22 17:27:54+00:00",
            "title": "Transfer Learning for Relation Extraction via Relation-Gated Adversarial Learning",
            "authors": [
                "Ningyu Zhang",
                "Shumin Deng",
                "Zhanlin Sun",
                "Jiaoyan Chen",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "Relation extraction aims to extract relational facts from sentences. Previous\nmodels mainly rely on manually labeled datasets, seed instances or\nhuman-crafted patterns, and distant supervision. However, the human annotation\nis expensive, while human-crafted patterns suffer from semantic drift and\ndistant supervision samples are usually noisy. Domain adaptation methods enable\nleveraging labeled data from a different but related domain. However, different\ndomains usually have various textual relation descriptions and different label\nspace (the source label space is usually a superset of the target label space).\nTo solve these problems, we propose a novel model of relation-gated adversarial\nlearning for relation extraction, which extends the adversarial based domain\nadaptation. Experimental results have shown that the proposed approach\noutperforms previous domain adaptation methods regarding partial domain\nadaptation and can improve the accuracy of distance supervised relation\nextraction through fine-tuning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.08529v1",
            "date": "2019-08-22 17:59:08+00:00",
            "title": "Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning",
            "authors": [
                "Jyoti Aneja",
                "Harsh Agrawal",
                "Dhruv Batra",
                "Alexander Schwing"
            ],
            "abstract": "Diverse and accurate vision+language modeling is an important goal to retain\ncreative freedom and maintain user engagement. However, adequately capturing\nthe intricacies of diversity in language models is challenging. Recent works\ncommonly resort to latent variable models augmented with more or less\nsupervision from object detectors or part-of-speech tags. Common to all those\nmethods is the fact that the latent variable either only initializes the\nsentence generation process or is identical across the steps of generation.\nBoth methods offer no fine-grained control. To address this concern, we propose\nSeq-CVAE which learns a latent space for every word position. We encourage this\ntemporal latent space to capture the 'intention' about how to complete the\nsentence by mimicking a representation which summarizes the future. We\nillustrate the efficacy of the proposed approach to anticipate the sentence\ncontinuation on the challenging MSCOCO dataset, significantly improving\ndiversity metrics compared to baselines while performing on par w.r.t sentence\nquality.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08530v4",
            "date": "2019-08-22 17:59:30+00:00",
            "title": "VL-BERT: Pre-training of Generic Visual-Linguistic Representations",
            "authors": [
                "Weijie Su",
                "Xizhou Zhu",
                "Yue Cao",
                "Bin Li",
                "Lewei Lu",
                "Furu Wei",
                "Jifeng Dai"
            ],
            "abstract": "We introduce a new pre-trainable generic representation for visual-linguistic\ntasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the\nsimple yet powerful Transformer model as the backbone, and extends it to take\nboth visual and linguistic embedded features as input. In it, each element of\nthe input is either of a word from the input sentence, or a region-of-interest\n(RoI) from the input image. It is designed to fit for most of the\nvisual-linguistic downstream tasks. To better exploit the generic\nrepresentation, we pre-train VL-BERT on the massive-scale Conceptual Captions\ndataset, together with text-only corpus. Extensive empirical analysis\ndemonstrates that the pre-training procedure can better align the\nvisual-linguistic clues and benefit the downstream tasks, such as visual\ncommonsense reasoning, visual question answering and referring expression\ncomprehension. It is worth noting that VL-BERT achieved the first place of\nsingle model on the leaderboard of the VCR benchmark. Code is released at\n\\url{https://github.com/jackroos/VL-BERT}.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08584v1",
            "date": "2019-08-22 20:29:05+00:00",
            "title": "Feedbackward Decoding for Semantic Segmentation",
            "authors": [
                "Beinan Wang",
                "John Glossner",
                "Daniel Iancu",
                "Georgi N. Gaydadjiev"
            ],
            "abstract": "We propose a novel approach for semantic segmentation that uses an encoder in\nthe reverse direction to decode. Many semantic segmentation networks adopt a\nfeedforward encoder-decoder architecture. Typically, an input is first\ndownsampled by the encoder to extract high-level semantic features and\ncontinues to be fed forward through the decoder module to recover low-level\nspatial clues. Our method works in an alternative direction that lets\ninformation flow backward from the last layer of the encoder towards the first.\nThe encoder performs encoding in the forward pass and the same network performs\ndecoding in the backward pass. Therefore, the encoder itself is also the\ndecoder. Compared to conventional encoder-decoder architectures, ours doesn't\nrequire additional layers for decoding and further reuses the encoder weights\nthereby reducing the total number of parameters required for processing. We\nshow by using only the 13 convolutional layers from VGG-16 plus one tiny\nclassification layer, our model significantly outperforms other frequently\ncited models that are also adapted from VGG-16. On the Cityscapes semantic\nsegmentation benchmark, our model uses 50.0% less parameters than SegNet and\nachieves an 18.1% higher \"IoU class\" score; it uses 28.3% less parameters than\nDeepLab LargeFOV and the achieved \"IoU class\" score is 3.9% higher; it uses\n89.1% fewer parameters than FCN-8s and the achieved \"IoU class\" score is 3.1%\nhigher. Our code will be publicly available on Github later.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08593v2",
            "date": "2019-08-21 04:27:38+00:00",
            "title": "Revealing the Dark Secrets of BERT",
            "authors": [
                "Olga Kovaleva",
                "Alexey Romanov",
                "Anna Rogers",
                "Anna Rumshisky"
            ],
            "abstract": "BERT-based architectures currently give state-of-the-art performance on many\nNLP tasks, but little is known about the exact mechanisms that contribute to\nits success. In the current work, we focus on the interpretation of\nself-attention, which is one of the fundamental underlying components of BERT.\nUsing a subset of GLUE tasks and a set of handcrafted features-of-interest, we\npropose the methodology and carry out a qualitative and quantitative analysis\nof the information encoded by the individual BERT's heads. Our findings suggest\nthat there is a limited set of attention patterns that are repeated across\ndifferent heads, indicating the overall model overparametrization. While\ndifferent heads consistently use the same attention patterns, they have varying\nimpact on performance across different tasks. We show that manually disabling\nattention in certain heads leads to a performance improvement over the regular\nfine-tuned BERT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.08674v1",
            "date": "2019-08-23 05:45:27+00:00",
            "title": "A BLSTM Network for Printed Bengali OCR System with High Accuracy",
            "authors": [
                "Debabrata Paul",
                "Bidyut Baran Chaudhuri"
            ],
            "abstract": "This paper presents a printed Bengali and English text OCR system developed\nby us using a single hidden BLSTM-CTC architecture having 128 units. Here, we\ndid not use any peephole connection and dropout in the BLSTM, which helped us\nin getting better accuracy. This architecture was trained by 47,720 text lines\nthat include English words also. When tested over 20 different Bengali fonts,\nit has produced character level accuracy of 99.32% and word level accuracy of\n96.65%. A good Indic multi script OCR system is also developed by Google. It\nsometimes recognizes a character of Bengali into the same character of a\nnon-Bengali script, especially Assamese, which has no distinction from Bengali,\nexcept for a few characters. For example, Bengali character for 'RA' is\nsometimes recognized as that of Assamese, mainly in conjunct consonant forms.\nOur OCR is free from such errors. This OCR system is available online at\nhttps://banglaocr.nltr.org",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08717v1",
            "date": "2019-08-23 08:51:19+00:00",
            "title": "Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance",
            "authors": [
                "Mahault Garnerin",
                "Solange Rossato",
                "Laurent Besacier"
            ],
            "abstract": "This paper analyzes the gender representation in four major corpora of French\nbroadcast. These corpora being widely used within the speech processing\ncommunity, they are a primary material for training automatic speech\nrecognition (ASR) systems. As gender bias has been highlighted in numerous\nnatural language processing (NLP) applications, we study the impact of the\ngender imbalance in TV and radio broadcast on the performance of an ASR system.\nThis analysis shows that women are under-represented in our data in terms of\nspeakers and speech turns. We introduce the notion of speaker role to refine\nour analysis and find that women are even fewer within the Anchor category\ncorresponding to prominent speakers. The disparity of available data for both\ngender causes performance to decrease on women. However this global trend can\nbe counterbalanced for speaker who are used to speak in the media when\nsufficient amount of data is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09137v2",
            "date": "2019-08-24 13:37:35+00:00",
            "title": "Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks",
            "authors": [
                "Seunghyun Yoon",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Trung Bui",
                "Kyomin Jung"
            ],
            "abstract": "In this study, we propose a novel graph neural network called\npropagate-selector (PS), which propagates information over sentences to\nunderstand information that cannot be inferred when considering sentences in\nisolation. First, we design a graph structure in which each node represents an\nindividual sentence, and some pairs of nodes are selectively connected based on\nthe text structure. Then, we develop an iterative attentive aggregation and a\nskip-combine method in which a node interacts with its neighborhood nodes to\naccumulate the necessary information. To evaluate the performance of the\nproposed approaches, we conduct experiments with the standard HotpotQA dataset.\nThe empirical results demonstrate the superiority of our proposed approach,\nwhich obtains the best performances, compared to the widely used\nanswer-selection models that do not consider the intersentential relationship.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09174v2",
            "date": "2019-08-24 17:38:30+00:00",
            "title": "Representation Learning with Autoencoders for Electronic Health Records: A Comparative Study",
            "authors": [
                "Najibesadat Sadati",
                "Milad Zafar Nezhad",
                "Ratna Babu Chinnam",
                "Dongxiao Zhu"
            ],
            "abstract": "Increasing volume of Electronic Health Records (EHR) in recent years provides\ngreat opportunities for data scientists to collaborate on different aspects of\nhealthcare research by applying advanced analytics to these EHR clinical data.\nA key requirement however is obtaining meaningful insights from high\ndimensional, sparse and complex clinical data. Data science approaches\ntypically address this challenge by performing feature learning in order to\nbuild more reliable and informative feature representations from clinical data\nfollowed by supervised learning. In this paper, we propose a predictive\nmodeling approach based on deep learning based feature representations and word\nembedding techniques. Our method uses different deep architectures (stacked\nsparse autoencoders, deep belief network, adversarial autoencoders and\nvariational autoencoders) for feature representation in higher-level\nabstraction to obtain effective and robust features from EHRs, and then build\nprediction models on top of them. Our approach is particularly useful when the\nunlabeled data is abundant whereas labeled data is scarce. We investigate the\nperformance of representation learning through a supervised learning approach.\nOur focus is to present a comparative study to evaluate the performance of\ndifferent deep architectures through supervised learning and provide insights\nin the choice of deep feature representation techniques. Our experiments\ndemonstrate that for small data sets, stacked sparse autoencoder demonstrates a\nsuperior generality performance in prediction due to sparsity regularization\nwhereas variational autoencoders outperform the competing approaches for large\ndata sets due to its capability of learning the representation distribution",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.09203v2",
            "date": "2019-08-24 20:41:40+00:00",
            "title": "Release Strategies and the Social Impacts of Language Models",
            "authors": [
                "Irene Solaiman",
                "Miles Brundage",
                "Jack Clark",
                "Amanda Askell",
                "Ariel Herbert-Voss",
                "Jeff Wu",
                "Alec Radford",
                "Gretchen Krueger",
                "Jong Wook Kim",
                "Sarah Kreps",
                "Miles McCain",
                "Alex Newhouse",
                "Jason Blazakis",
                "Kris McGuffie",
                "Jasmine Wang"
            ],
            "abstract": "Large language models have a range of beneficial uses: they can assist in\nprose, poetry, and programming; analyze dataset biases; and more. However,\ntheir flexibility and generative capabilities also raise misuse concerns. This\nreport discusses OpenAI's work related to the release of its GPT-2 language\nmodel. It discusses staged release, which allows time between model releases to\nconduct risk and benefit analyses as model sizes increased. It also discusses\nongoing partnership-based research and provides recommendations for better\ncoordination and responsible publication in AI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09246v1",
            "date": "2019-08-25 03:17:38+00:00",
            "title": "Open Event Extraction from Online Text using a Generative Adversarial Network",
            "authors": [
                "Rui Wang",
                "Deyu Zhou",
                "Yulan He"
            ],
            "abstract": "To extract the structured representations of open-domain events, Bayesian\ngraphical models have made some progress. However, these approaches typically\nassume that all words in a document are generated from a single event. While\nthis may be true for short text such as tweets, such an assumption does not\ngenerally hold for long text such as news articles. Moreover, Bayesian\ngraphical models often rely on Gibbs sampling for parameter inference which may\ntake long time to converge. To address these limitations, we propose an event\nextraction model based on Generative Adversarial Nets, called\nAdversarial-neural Event Model (AEM). AEM models an event with a Dirichlet\nprior and uses a generator network to capture the patterns underlying latent\nevents. A discriminator is used to distinguish documents reconstructed from the\nlatent events and the original documents. A byproduct of the discriminator is\nthat the features generated by the learned discriminator network allow the\nvisualization of the extracted events. Our model has been evaluated on two\nTwitter datasets and a news article dataset. Experimental results show that our\nmodel outperforms the baseline approaches on all the datasets, with more\nsignificant improvements observed on the news article dataset where an increase\nof 15\\% is observed in F-measure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09341v2",
            "date": "2019-08-25 14:54:49+00:00",
            "title": "A Method for Estimating the Proximity of Vector Representation Groups in Multidimensional Space. On the Example of the Paraphrase Task",
            "authors": [
                "Artem Artemov",
                "Boris Alekseev"
            ],
            "abstract": "The following paper presents a method of comparing two sets of vectors. The\nmethod can be applied in all tasks, where it is necessary to measure the\ncloseness of two objects presented as sets of vectors. It may be applicable\nwhen we compare the meanings of two sentences as part of the problem of\nparaphrasing. This is the problem of measuring semantic similarity of two\nsentences (group of words). The existing methods are not sensible for the word\norder or syntactic connections in the considered sentences. The method appears\nto be advantageous because it neither presents a group of words as one scalar\nvalue, nor does it try to show the closeness through an aggregation vector,\nwhich is mean for the set of vectors. Instead of that we measure the cosine of\nthe angle as the mean for the first group vectors projections (the context) on\none side and each vector of the second group on the other side. The similarity\nof two sentences defined by these means does not lose any semantic\ncharacteristics and takes account of the words traits. The method was verified\non the comparison of sentence pairs in Russian.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.09451v2",
            "date": "2019-08-26 03:29:21+00:00",
            "title": "Improving Neural Story Generation by Targeted Common Sense Grounding",
            "authors": [
                "Huanru Henry Mao",
                "Bodhisattwa Prasad Majumder",
                "Julian McAuley",
                "Garrison W. Cottrell"
            ],
            "abstract": "Stories generated with neural language models have shown promise in\ngrammatical and stylistic consistency. However, the generated stories are still\nlacking in common sense reasoning, e.g., they often contain sentences deprived\nof world knowledge. We propose a simple multi-task learning scheme to achieve\nquantitatively better common sense reasoning in language models by leveraging\nauxiliary training signals from datasets designed to provide common sense\ngrounding. When combined with our two-stage fine-tuning pipeline, our method\nachieves improved common sense reasoning and state-of-the-art perplexity on the\nWriting Prompts (Fan et al., 2018) story generation dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.09641v1",
            "date": "2019-08-26 12:35:28+00:00",
            "title": "Semi-supervised Learning for Word Sense Disambiguation",
            "authors": [
                "Dar\u00edo Garigliotti"
            ],
            "abstract": "This work is a study of the impact of multiple aspects in a classic\nunsupervised word sense disambiguation algorithm. We identify relevant factors\nin a decision rule algorithm, including the initial labeling of examples, the\nformalization of the rule confidence, and the criteria for accepting a decision\nrule. Some of these factors are only implicitly considered in the original\nliterature. We then propose a lightly supervised version of the algorithm, and\nemploy a pseudo-word-based strategy to evaluate the impact of these factors.\nThe obtained performances are comparable with those of highly optimized\nformulations of the word sense disambiguation method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09720v2",
            "date": "2019-08-26 15:01:24+00:00",
            "title": "Ensemble approach for natural language question answering problem",
            "authors": [
                "Anna Aniol",
                "Marcin Pietron"
            ],
            "abstract": "Machine comprehension, answering a question depending on a given context\nparagraph is a typical task of Natural Language Understanding. It requires to\nmodel complex dependencies existing between the question and the context\nparagraph. There are many neural network models attempting to solve the problem\nof question answering. The best models have been selected, studied and compared\nwith each other. All the selected models are based on the neural attention\nmechanism concept. Additionally, studies on a SQUAD dataset were performed. The\nsubsets of queries were extracted and then each model was analyzed how it deals\nwith specific group of queries. Based on these three model ensemble model was\ncreated and tested on SQUAD dataset. It outperforms the best Mnemonic Reader\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09738v1",
            "date": "2019-08-26 15:32:44+00:00",
            "title": "Connecting and Comparing Language Model Interpolation Techniques",
            "authors": [
                "Ernest Pusateri",
                "Christophe Van Gysel",
                "Rami Botros",
                "Sameer Badaskar",
                "Mirko Hannemann",
                "Youssef Oualil",
                "Ilya Oparin"
            ],
            "abstract": "In this work, we uncover a theoretical connection between two language model\ninterpolation techniques, count merging and Bayesian interpolation. We compare\nthese techniques as well as linear interpolation in three scenarios with\nabundant training data per component model. Consistent with prior work, we show\nthat both count merging and Bayesian interpolation outperform linear\ninterpolation. We include the first (to our knowledge) published comparison of\ncount merging and Bayesian interpolation, showing that the two techniques\nperform similarly. Finally, we argue that other considerations will make\nBayesian interpolation the preferred approach in most circumstances.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.09756v3",
            "date": "2019-08-26 15:56:10+00:00",
            "title": "Differentiable Product Quantization for End-to-End Embedding Compression",
            "authors": [
                "Ting Chen",
                "Lala Li",
                "Yizhou Sun"
            ],
            "abstract": "Embedding layers are commonly used to map discrete symbols into continuous\nembedding vectors that reflect their semantic meanings. Despite their\neffectiveness, the number of parameters in an embedding layer increases\nlinearly with the number of symbols and poses a critical challenge on memory\nand storage constraints. In this work, we propose a generic and end-to-end\nlearnable compression framework termed differentiable product quantization\n(DPQ). We present two instantiations of DPQ that leverage different\napproximation techniques to enable differentiability in end-to-end learning.\nOur method can readily serve as a drop-in alternative for any existing\nembedding layer. Empirically, DPQ offers significant compression ratios\n(14-238$\\times$) at negligible or no performance cost on 10 datasets across\nthree different language tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.09890v1",
            "date": "2019-08-26 19:41:21+00:00",
            "title": "Multi-Granularity Representations of Dialog",
            "authors": [
                "Shikib Mehri",
                "Maxine Eskenazi"
            ],
            "abstract": "Neural models of dialog rely on generalized latent representations of\nlanguage. This paper introduces a novel training procedure which explicitly\nlearns multiple representations of language at several levels of granularity.\nThe multi-granularity training algorithm modifies the mechanism by which\nnegative candidate responses are sampled in order to control the granularity of\nlearned latent representations. Strong performance gains are observed on the\nnext utterance retrieval task using both the MultiWOZ dataset and the Ubuntu\ndialog corpus. Analysis significantly demonstrates that multiple granularities\nof representation are being learned, and that multi-granularity training\nfacilitates better transfer to downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09919v2",
            "date": "2019-08-22 07:36:48+00:00",
            "title": "Gender Prediction from Tweets: Improving Neural Representations with Hand-Crafted Features",
            "authors": [
                "Erhan Sezerer",
                "Ozan Polatbilek",
                "Selma Tekir"
            ],
            "abstract": "Author profiling is the characterization of an author through some key\nattributes such as gender, age, and language. In this paper, a RNN model with\nAttention (RNNwA) is proposed to predict the gender of a twitter user using\ntheir tweets. Both word level and tweet level attentions are utilized to learn\n'where to look'. This model\n(https://github.com/Darg-Iztech/gender-prediction-from-tweets) is improved by\nconcatenating LSA-reduced n-gram features with the learned neural\nrepresentation of a user. Both models are tested on three languages: English,\nSpanish, Arabic. The improved version of the proposed model (RNNwA + n-gram)\nachieves state-of-the-art performance on English and has competitive results on\nSpanish and Arabic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.09936v1",
            "date": "2019-08-26 22:08:55+00:00",
            "title": "Leveraging External Knowledge for Out-Of-Vocabulary Entity Labeling",
            "authors": [
                "Adrian de Wynter",
                "Lambert Mathias"
            ],
            "abstract": "Dealing with previously unseen slots is a challenging problem in a real-world\nmulti-domain dialogue state tracking task. Other approaches rely on predefined\nmappings to generate candidate slot keys, as well as their associated values.\nThis, however, may fail when the key, the value, or both, are not seen during\ntraining. To address this problem we introduce a neural network that leverages\nexternal knowledge bases (KBs) to better classify out-of-vocabulary slot keys\nand values. This network projects the slot into an attribute space derived from\nthe KB, and, by leveraging similarities in this space, we propose candidate\nslot keys and values to the dialogue state tracker. We provide extensive\nexperiments that demonstrate that our stratagem can improve upon a previous\napproach, which relies on predefined candidate mappings. In particular, we\nevaluate this approach by training a state-of-the-art model with candidates\ngenerated from our network, and obtained relative increases of 57.7% and 82.7%\nin F1 score and accuracy, respectively, for the aforementioned model, when\ncompared to the current candidate generation strategy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.09951v1",
            "date": "2019-08-26 22:49:35+00:00",
            "title": "An Emotional Analysis of False Information in Social Media and News Articles",
            "authors": [
                "Bilal Ghanem",
                "Paolo Rosso",
                "Francisco Rangel"
            ],
            "abstract": "Fake news is risky since it has been created to manipulate the readers'\nopinions and beliefs. In this work, we compared the language of false news to\nthe real one of real news from an emotional perspective, considering a set of\nfalse information types (propaganda, hoax, clickbait, and satire) from social\nmedia and online news articles sources. Our experiments showed that false\ninformation has different emotional patterns in each of its types, and emotions\nplay a key role in deceiving the reader. Based on that, we proposed a LSTM\nneural network model that is emotionally-infused to detect false news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10001v1",
            "date": "2019-08-27 03:13:53+00:00",
            "title": "Real-world Conversational AI for Hotel Bookings",
            "authors": [
                "Bai Li",
                "Nanyi Jiang",
                "Joey Sham",
                "Henry Shi",
                "Hussein Fazal"
            ],
            "abstract": "In this paper, we present a real-world conversational AI system to search for\nand book hotels through text messaging. Our architecture consists of a\nframe-based dialogue management system, which calls machine learning models for\nintent classification, named entity recognition, and information retrieval\nsubtasks. Our chatbot has been deployed on a commercial scale, handling tens of\nthousands of hotel searches every day. We describe the various opportunities\nand challenges of developing a chatbot in the travel industry.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.10149v1",
            "date": "2019-08-27 11:54:23+00:00",
            "title": "Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates using Machine Learning",
            "authors": [
                "Michael Barz",
                "Daniel Sonntag"
            ],
            "abstract": "We implement a method for re-ranking top-10 results of a state-of-the-art\nquestion answering (QA) system. The goal of our re-ranking approach is to\nimprove the answer selection given the user question and the top-10 candidates.\nWe focus on improving deployed QA systems that do not allow re-training or\nre-training comes at a high cost. Our re-ranking approach learns a similarity\nfunction using n-gram based features using the query, the answer and the\ninitial system confidence as input. Our contributions are: (1) we generate a QA\ntraining corpus starting from 877 answers from the customer care domain of\nT-Mobile Austria, (2) we implement a state-of-the-art QA pipeline using neural\nsentence embeddings that encode queries in the same space than the answer\nindex, and (3) we evaluate the QA pipeline and our re-ranking approach using a\nseparately provided test set. The test set can be considered to be available\nafter deployment of the system, e.g., based on feedback of users. Our results\nshow that the system performance, in terms of top-n accuracy and the mean\nreciprocal rank, benefits from re-ranking using gradient boosted regression\ntrees. On average, the mean reciprocal rank improves by 9.15%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.10275v1",
            "date": "2019-08-27 15:30:56+00:00",
            "title": "The Wiki Music dataset: A tool for computational analysis of popular music",
            "authors": [
                "Fabio Celli"
            ],
            "abstract": "Is it possible use algorithms to find trends in the history of popular music?\nAnd is it possible to predict the characteristics of future music genres? In\norder to answer these questions, we produced a hand-crafted dataset with the\nintent to put together features about style, psychology, sociology and\ntypology, annotated by music genre and indexed by time and decade. We collected\na list of popular genres by decade from Wikipedia and scored music genres based\non Wikipedia descriptions. Using statistical and machine learning techniques,\nwe find trends in the musical preferences and use time series forecasting to\nevaluate the prediction of future music genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10285v1",
            "date": "2019-08-27 15:44:17+00:00",
            "title": "Is the Red Square Big? MALeViC: Modeling Adjectives Leveraging Visual Contexts",
            "authors": [
                "Sandro Pezzelle",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "This work aims at modeling how the meaning of gradable adjectives of size\n(`big', `small') can be learned from visually-grounded contexts. Inspired by\ncognitive and linguistic evidence showing that the use of these expressions\nrelies on setting a threshold that is dependent on a specific context, we\ninvestigate the ability of multi-modal models in assessing whether an object is\n`big' or `small' in a given visual scene. In contrast with the standard\ncomputational approach that simplistically treats gradable adjectives as\n`fixed' attributes, we pose the problem as relational: to be successful, a\nmodel has to consider the full visual context. By means of four main tasks, we\nshow that state-of-the-art models (but not a relatively strong baseline) can\nlearn the function subtending the meaning of size adjectives, though their\nperformance is found to decrease while moving from simple to more complex\ntasks. Crucially, models fail in developing abstract representations of\ngradable adjectives that can be used compositionally.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10322v1",
            "date": "2019-08-27 16:53:59+00:00",
            "title": "Bridging the Gap for Tokenizer-Free Language Models",
            "authors": [
                "Dokook Choe",
                "Rami Al-Rfou",
                "Mandy Guo",
                "Heeyoung Lee",
                "Noah Constant"
            ],
            "abstract": "Purely character-based language models (LMs) have been lagging in quality on\nlarge scale datasets, and current state-of-the-art LMs rely on word\ntokenization. It has been assumed that injecting the prior knowledge of a\ntokenizer into the model is essential to achieving competitive results. In this\npaper, we show that contrary to this conventional wisdom, tokenizer-free LMs\nwith sufficient capacity can achieve competitive performance on a large scale\ndataset. We train a vanilla transformer network with 40 self-attention layers\non the One Billion Word (lm1b) benchmark and achieve a new state of the art for\ntokenizer-free LMs, pushing these models to be on par with their word-based\ncounterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10331v1",
            "date": "2019-08-27 17:06:15+00:00",
            "title": "Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards",
            "authors": [
                "Heriberto Cuay\u00e1huitl",
                "Donghyeon Lee",
                "Seonghan Ryu",
                "Sungja Choi",
                "Inchul Hwang",
                "Jihie Kim"
            ],
            "abstract": "Training chatbots using the reinforcement learning paradigm is challenging\ndue to high-dimensional states, infinite action spaces and the difficulty in\nspecifying the reward function. We address such problems using clustered\nactions instead of infinite actions, and a simple but promising reward function\nbased on human-likeness scores derived from human-human dialogue data. We train\nDeep Reinforcement Learning (DRL) agents using chitchat data in raw\ntext---without any manual annotations. Experimental results using different\nsplits of training data report the following. First, that our agents learn\nreasonable policies in the environments they get familiarised with, but their\nperformance drops substantially when they are exposed to a test set of unseen\ndialogues. Second, that the choice of sentence embedding size between 100 and\n300 dimensions is not significantly different on test data. Third, that our\nproposed human-likeness rewards are reasonable for training chatbots as long as\nthey use lengthy dialogue histories of >=10 sentences.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1908.10419v1",
            "date": "2019-08-27 19:15:26+00:00",
            "title": "Hierarchical Text Classification with Reinforced Label Assignment",
            "authors": [
                "Yuning Mao",
                "Jingjing Tian",
                "Jiawei Han",
                "Xiang Ren"
            ],
            "abstract": "While existing hierarchical text classification (HTC) methods attempt to\ncapture label hierarchies for model training, they either make local decisions\nregarding each label or completely ignore the hierarchy information during\ninference. To solve the mismatch between training and inference as well as\nmodeling label dependencies in a more principled way, we formulate HTC as a\nMarkov decision process and propose to learn a Label Assignment Policy via deep\nreinforcement learning to determine where to place an object and when to stop\nthe assignment process. The proposed method, HiLAP, explores the hierarchy\nduring both training and inference time in a consistent manner and makes\ninter-dependent decisions. As a general framework, HiLAP can incorporate\ndifferent neural encoders as base models for end-to-end training. Experiments\non five public datasets and four base models show that HiLAP yields an average\nimprovement of 33.4% in Macro-F1 over flat classifiers and outperforms\nstate-of-the-art HTC methods by a large margin. Data and code can be found at\nhttps://github.com/morningmoni/HiLAP.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.10422v1",
            "date": "2019-08-27 19:18:09+00:00",
            "title": "Ensemble-Based Deep Reinforcement Learning for Chatbots",
            "authors": [
                "Heriberto Cuay\u00e1huitl",
                "Donghyeon Lee",
                "Seonghan Ryu",
                "Yongjin Cho",
                "Sungja Choi",
                "Satish Indurthi",
                "Seunghak Yu",
                "Hyungtak Choi",
                "Inchul Hwang",
                "Jihie Kim"
            ],
            "abstract": "Trainable chatbots that exhibit fluent and human-like conversations remain a\nbig challenge in artificial intelligence. Deep Reinforcement Learning (DRL) is\npromising for addressing this challenge, but its successful application remains\nan open question. This article describes a novel ensemble-based approach\napplied to value-based DRL chatbots, which use finite action sets as a form of\nmeaning representation. In our approach, while dialogue actions are derived\nfrom sentence clustering, the training datasets in our ensemble are derived\nfrom dialogue clustering. The latter aim to induce specialised agents that\nlearn to interact in a particular style. In order to facilitate neural chatbot\ntraining using our proposed approach, we assume dialogue data in raw text only\n-- without any manually-labelled data. Experimental results using chitchat data\nreveal that (1) near human-like dialogue policies can be induced, (2)\ngeneralisation to unseen data is a difficult problem, and (3) training an\nensemble of chatbot agents is essential for improved performance over using a\nsingle agent. In addition to evaluations using held-out data, our results are\nfurther supported by a human evaluation that rated dialogues in terms of\nfluency, engagingness and consistency -- which revealed that our proposed\ndialogue rewards strongly correlate with human judgements.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1908.10449v3",
            "date": "2019-08-27 20:11:54+00:00",
            "title": "Interactive Machine Comprehension with Information Seeking Agents",
            "authors": [
                "Xingdi Yuan",
                "Jie Fu",
                "Marc-Alexandre Cote",
                "Yi Tay",
                "Christopher Pal",
                "Adam Trischler"
            ],
            "abstract": "Existing machine reading comprehension (MRC) models do not scale effectively\nto real-world applications like web-level information retrieval and question\nanswering (QA). We argue that this stems from the nature of MRC datasets: most\nof these are static environments wherein the supporting documents and all\nnecessary information are fully observed. In this paper, we propose a simple\nmethod that reframes existing MRC datasets as interactive, partially observable\nenvironments. Specifically, we \"occlude\" the majority of a document's text and\nadd context-sensitive commands that reveal \"glimpses\" of the hidden text to a\nmodel. We repurpose SQuAD and NewsQA as an initial case study, and then show\nhow the interactive corpora can be used to train a model that seeks relevant\ninformation through sequential decision making. We believe that this setting\ncan contribute in scaling models to web-level QA scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10621v1",
            "date": "2019-08-28 10:06:19+00:00",
            "title": "Onto Word Segmentation of the Complete Tang Poems",
            "authors": [
                "Chao-Lin Liu"
            ],
            "abstract": "We aim at segmenting words in the Complete Tang Poems (CTP). Although it is\npossible to do some research about CTP without doing full-scale word\nsegmentation, we must move forward to word-level analysis of CTP for conducting\nadvanced research topics. In November 2018 when we submitted the manuscript for\nDH 2019 (ADHO), we collected only 2433 poems that were segmented by trained\nexperts, and used the segmented poems to evaluate the segmenter that considered\ndomain knowledge of Chinese poetry. We trained pointwise mutual information\n(PMI) between Chinese characters based on the CTP poems (excluding the 2433\npoems, which were used exclusively only for testing) and the domain knowledge.\nThe segmenter relied on the PMI information to the recover 85.7% of words in\nthe test poems. We could segment a poem completely correct only 17.8% of the\ntime, however. When we presented our work at DH 2019, we have annotated more\nthan 20000 poems. With a much larger amount of data, we were able to apply\nbiLSTM models for this word segmentation task, and we segmented a poem\ncompletely correct above 20% of the time. In contrast, human annotators\ncompletely agreed on their annotations about 40% of the time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10731v1",
            "date": "2019-08-28 14:03:44+00:00",
            "title": "DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks",
            "authors": [
                "Semih Yavuz",
                "Abhinav Rastogi",
                "Guan-Lin Chao",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Recent advances in neural sequence-to-sequence models have led to promising\nresults for several language generation-based tasks, including dialogue\nresponse generation, summarization, and machine translation. However, these\nmodels are known to have several problems, especially in the context of\nchit-chat based dialogue systems: they tend to generate short and dull\nresponses that are often too generic. Furthermore, these models do not ground\nconversational responses on knowledge and facts, resulting in turns that are\nnot accurate, informative and engaging for the users. In this paper, we propose\nand experiment with a series of response generation models that aim to serve in\nthe general scenario where in addition to the dialogue context, relevant\nunstructured external knowledge in the form of text is also assumed to be\navailable for models to harness. Our proposed approach extends\npointer-generator networks (See et al., 2017) by allowing the decoder to\nhierarchically attend and copy from external knowledge in addition to the\ndialogue context. We empirically show the effectiveness of the proposed model\ncompared to several baselines including (Ghazvininejad et al., 2018; Zhang et\nal., 2018) through both automatic evaluation metrics and human evaluation on\nCONVAI2 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10784v2",
            "date": "2019-08-28 15:39:02+00:00",
            "title": "Semantic Hypergraphs",
            "authors": [
                "Telmo Menezes",
                "Camille Roth"
            ],
            "abstract": "Approaches to Natural language processing (NLP) may be classified along a\ndouble dichotomy open/opaque - strict/adaptive. The former axis relates to the\npossibility of inspecting the underlying processing rules, the latter to the\nuse of fixed or adaptive rules. We argue that many techniques fall into either\nthe open-strict or opaque-adaptive categories. Our contribution takes steps in\nthe open-adaptive direction, which we suggest is likely to provide key\ninstruments for interdisciplinary research. The central idea of our approach is\nthe Semantic Hypergraph (SH), a novel knowledge representation model that is\nintrinsically recursive and accommodates the natural hierarchical richness of\nnatural language. The SH model is hybrid in two senses. First, it attempts to\ncombine the strengths of ML and symbolic approaches. Second, it is a formal\nlanguage representation that reduces but tolerates ambiguity and structural\nvariability. We will see that SH enables simple yet powerful methods of pattern\ndetection, and features a good compromise for intelligibility both for humans\nand machines. It also provides a semantically deep starting point (in terms of\nexplicit meaning) for further algorithms to operate and collaborate on. We show\nhow modern NLP ML-based building blocks can be used in combination with a\nrandom forest classifier and a simple search tree to parse NL to SH, and that\nthis parser can achieve high precision in a diversity of text categories. We\ndefine a pattern language representable in SH itself, and a process to discover\nknowledge inference rules. We then illustrate the efficiency of the SH\nframework in a variety of tasks, including conjunction decomposition, open\ninformation extraction, concept taxonomy inference and co-reference resolution,\nand an applied example of claim and conflict analysis in a news corpus.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1908.10797v2",
            "date": "2019-08-28 15:53:13+00:00",
            "title": "Image Captioning with Sparse Recurrent Neural Network",
            "authors": [
                "Jia Huei Tan",
                "Chee Seng Chan",
                "Joon Huang Chuah"
            ],
            "abstract": "Recurrent Neural Network (RNN) has been widely used to tackle a wide variety\nof language generation problems and are capable of attaining state-of-the-art\n(SOTA) performance. However despite its impressive results, the large number of\nparameters in the RNN model makes deployment to mobile and embedded devices\ninfeasible. Driven by this problem, many works have proposed a number of\npruning methods to reduce the sizes of the RNN model. In this work, we propose\nan end-to-end pruning method for image captioning models equipped with visual\nattention. Our proposed method is able to achieve sparsity levels up to 97.5%\nwithout significant performance loss relative to the baseline (~ 2% loss at 40x\ncompression after fine-tuning). Our method is also simple to use and tune,\nfacilitating faster development times for neural network practitioners. We\nperform extensive experiments on the popular MS-COCO dataset in order to\nempirically validate the efficacy of our proposed method.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.10992v1",
            "date": "2019-08-29 00:18:05+00:00",
            "title": "Two-Pass End-to-End Speech Recognition",
            "authors": [
                "Tara N. Sainath",
                "Ruoming Pang",
                "David Rybach",
                "Yanzhang He",
                "Rohit Prabhavalkar",
                "Wei Li",
                "Mirk\u00f3 Visontai",
                "Qiao Liang",
                "Trevor Strohman",
                "Yonghui Wu",
                "Ian McGraw",
                "Chung-Cheng Chiu"
            ],
            "abstract": "The requirements for many applications of state-of-the-art speech recognition\nsystems include not only low word error rate (WER) but also low latency.\nSpecifically, for many use-cases, the system must be able to decode utterances\nin a streaming fashion and faster than real-time. Recently, a streaming\nrecurrent neural network transducer (RNN-T) end-to-end (E2E) model has shown to\nbe a good candidate for on-device speech recognition, with improved WER and\nlatency metrics compared to conventional on-device models [1]. However, this\nmodel still lags behind a large state-of-the-art conventional model in quality\n[2]. On the other hand, a non-streaming E2E Listen, Attend and Spell (LAS)\nmodel has shown comparable quality to large conventional models [3]. This work\naims to bring the quality of an E2E streaming model closer to that of a\nconventional system by incorporating a LAS network as a second-pass component,\nwhile still abiding by latency constraints. Our proposed two-pass model\nachieves a 17%-22% relative reduction in WER compared to RNN-T alone and\nincreases latency by a small fraction over RNN-T.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.10993v1",
            "date": "2019-08-29 00:25:38+00:00",
            "title": "Scientific Statement Classification over arXiv.org",
            "authors": [
                "Deyan Ginev",
                "Bruce R. Miller"
            ],
            "abstract": "We introduce a new classification task for scientific statements and release\na large-scale dataset for supervised learning. Our resource is derived from a\nmachine-readable representation of the arXiv.org collection of preprint\narticles. We explore fifty author-annotated categories and empirically motivate\na task design of grouping 10.5 million annotated paragraphs into thirteen\nclasses. We demonstrate that the task setup aligns with known success rates\nfrom the state of the art, peaking at a 0.91 F1-score via a BiLSTM\nencoder-decoder model. Additionally, we introduce a lexeme serialization for\nmathematical formulas, and observe that context-aware models could improve when\nalso trained on the symbolic modality. Finally, we discuss the limitations of\nboth data and task design, and outline potential directions towards\nincreasingly complex models of scientific discourse, beyond isolated\nstatements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11057v1",
            "date": "2019-08-29 05:19:53+00:00",
            "title": "A Deep Neural Information Fusion Architecture for Textual Network Embeddings",
            "authors": [
                "Zenan Xu",
                "Qinliang Su",
                "Xiaojun Quan",
                "Weijia Zhang"
            ],
            "abstract": "Textual network embeddings aim to learn a low-dimensional representation for\nevery node in the network so that both the structural and textual information\nfrom the networks can be well preserved in the representations. Traditionally,\nthe structural and textual embeddings were learned by models that rarely take\nthe mutual influences between them into account. In this paper, a deep neural\narchitecture is proposed to effectively fuse the two kinds of informations into\none representation. The novelties of the proposed architecture are manifested\nin the aspects of a newly defined objective function, the complementary\ninformation fusion method for structural and textual features, and the mutual\ngate mechanism for textual feature extraction. Experimental results show that\nthe proposed model outperforms the comparing methods on all three datasets.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.11216v3",
            "date": "2019-08-29 13:34:50+00:00",
            "title": "From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining",
            "authors": [
                "Alexandre Garcia",
                "Pierre Colombo",
                "Slim Essid",
                "Florence d'Alch\u00e9-Buc",
                "Chlo\u00e9 Clavel"
            ],
            "abstract": "The task of predicting fine grained user opinion based on spontaneous spoken\nlanguage is a key problem arising in the development of Computational Agents as\nwell as in the development of social network based opinion miners.\nUnfortunately, gathering reliable data on which a model can be trained is\nnotoriously difficult and existing works rely only on coarsely labeled\nopinions. In this work we aim at bridging the gap separating fine grained\nopinion models already developed for written language and coarse grained models\ndeveloped for spontaneous multimodal opinion mining. We take advantage of the\nimplicit hierarchical structure of opinions to build a joint fine and coarse\ngrained opinion model that exploits different views of the opinion expression.\nThe resulting model shares some properties with attention-based models and is\nshown to provide competitive results on a recently released multimodal fine\ngrained annotated corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11302v1",
            "date": "2019-08-29 15:36:40+00:00",
            "title": "HARE: a Flexible Highlighting Annotator for Ranking and Exploration",
            "authors": [
                "Denis Newman-Griffis",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Exploration and analysis of potential data sources is a significant challenge\nin the application of NLP techniques to novel information domains. We describe\nHARE, a system for highlighting relevant information in document collections to\nsupport ranking and triage, which provides tools for post-processing and\nqualitative analysis for model development and tuning. We apply HARE to the use\ncase of narrative descriptions of mobility information in clinical data, and\ndemonstrate its utility in comparing candidate embedding features. We provide a\nweb-based interface for annotation visualization and document ranking, with a\nmodular backend to support interoperability with existing annotation tools. Our\nsystem is available online at https://github.com/OSU-slatelab/HARE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11317v1",
            "date": "2019-08-29 16:00:25+00:00",
            "title": "Memorizing All for Implicit Discourse Relation Recognition",
            "authors": [
                "Hongxiao Bai",
                "Hai Zhao",
                "Junhan Zhao"
            ],
            "abstract": "Implicit discourse relation recognition is a challenging task due to the\nabsence of the necessary informative clue from explicit connectives. The\nprediction of relations requires a deep understanding of the semantic meanings\nof sentence pairs. As implicit discourse relation recognizer has to carefully\ntackle the semantic similarity of the given sentence pairs and the severe data\nsparsity issue exists in the meantime, it is supposed to be beneficial from\nmastering the entire training data. Thus in this paper, we propose a novel\nmemory mechanism to tackle the challenges for further performance improvement.\nThe memory mechanism is adequately memorizing information by pairing\nrepresentations and discourse relations of all training instances, which right\nfills the slot of the data-hungry issue in the current implicit discourse\nrelation recognizer. Our experiments show that our full model with memorizing\nthe entire training set reaches new state-of-the-art against strong baselines,\nwhich especially for the first time exceeds the milestone of 60% accuracy in\nthe 4-way task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11355v1",
            "date": "2019-08-29 17:12:04+00:00",
            "title": "Human-grounded Evaluations of Explanation Methods for Text Classification",
            "authors": [
                "Piyawat Lertvittayakumjorn",
                "Francesca Toni"
            ],
            "abstract": "Due to the black-box nature of deep learning models, methods for explaining\nthe models' results are crucial to gain trust from humans and support\ncollaboration between AIs and humans. In this paper, we consider several\nmodel-agnostic and model-specific explanation methods for CNNs for text\nclassification and conduct three human-grounded evaluations, focusing on\ndifferent purposes of explanations: (1) revealing model behavior, (2)\njustifying model predictions, and (3) helping humans investigate uncertain\npredictions. The results highlight dissimilar qualities of the various\nexplanation methods we consider and show the degree to which these methods\ncould serve for each purpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11521v1",
            "date": "2019-08-30 03:44:10+00:00",
            "title": "Charge-Based Prison Term Prediction with Deep Gating Network",
            "authors": [
                "Huajie Chen",
                "Deng Cai",
                "Wei Dai",
                "Zehui Dai",
                "Yadong Ding"
            ],
            "abstract": "Judgment prediction for legal cases has attracted much research efforts for\nits practice use, of which the ultimate goal is prison term prediction. While\nexisting work merely predicts the total prison term, in reality a defendant is\noften charged with multiple crimes. In this paper, we argue that charge-based\nprison term prediction (CPTP) not only better fits realistic needs, but also\nmakes the total prison term prediction more accurate and interpretable. We\ncollect the first large-scale structured data for CPTP and evaluate several\ncompetitive baselines. Based on the observation that fine-grained feature\nselection is the key to achieving good performance, we propose the Deep Gating\nNetwork (DGN) for charge-specific feature selection and aggregation.\nExperiments show that DGN achieves the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11527v3",
            "date": "2019-08-30 04:12:08+00:00",
            "title": "Implicit Deep Latent Variable Models for Text Generation",
            "authors": [
                "Le Fang",
                "Chunyuan Li",
                "Jianfeng Gao",
                "Wen Dong",
                "Changyou Chen"
            ],
            "abstract": "Deep latent variable models (LVM) such as variational auto-encoder (VAE) have\nrecently played an important role in text generation. One key factor is the\nexploitation of smooth latent structures to guide the generation. However, the\nrepresentation power of VAEs is limited due to two reasons: (1) the Gaussian\nassumption is often made on the variational posteriors; and meanwhile (2) a\nnotorious \"posterior collapse\" issue occurs. In this paper, we advocate\nsample-based representations of variational distributions for natural language,\nleading to implicit latent features, which can provide flexible representation\npower compared with Gaussian-based posteriors. We further develop an LVM to\ndirectly match the aggregated posterior to the prior. It can be viewed as a\nnatural extension of VAEs with a regularization of maximizing mutual\ninformation, mitigating the \"posterior collapse\" issue. We demonstrate the\neffectiveness and versatility of our models in various text generation\nscenarios, including language modeling, unaligned style transfer, and dialog\nresponse generation. The source code to reproduce our experimental results is\navailable on GitHub.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.11535v1",
            "date": "2019-08-30 05:00:06+00:00",
            "title": "Initial investigation of an encoder-decoder end-to-end TTS framework using marginalization of monotonic hard latent alignments",
            "authors": [
                "Yusuke Yasuda",
                "Xin Wang",
                "Junichi Yamagishi"
            ],
            "abstract": "End-to-end text-to-speech (TTS) synthesis is a method that directly converts\ninput text to output acoustic features using a single network. A recent advance\nof end-to-end TTS is due to a key technique called attention mechanisms, and\nall successful methods proposed so far have been based on soft attention\nmechanisms. However, although network structures are becoming increasingly\ncomplex, end-to-end TTS systems with soft attention mechanisms may still fail\nto learn and to predict accurate alignment between the input and output. This\nmay be because the soft attention mechanisms are too flexible. Therefore, we\npropose an approach that has more explicit but natural constraints suitable for\nspeech signals to make alignment learning and prediction of end-to-end TTS\nsystems more robust. The proposed system, with the constrained alignment scheme\nborrowed from segment-to-segment neural transduction (SSNT), directly\ncalculates the joint probability of acoustic features and alignment given an\ninput text. The alignment is designed to be hard and monotonically increase by\nconsidering the speech nature, and it is treated as a latent variable and\nmarginalized during training. During prediction, both the alignment and\nacoustic features can be generated from the probabilistic distributions. The\nadvantages of our approach are that we can simplify many modules for the soft\nattention and that we can train the end-to-end TTS model using a single\nlikelihood function. As far as we know, our approach is the first end-to-end\nTTS without a soft attention mechanism.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1908.11593v1",
            "date": "2019-08-30 08:32:10+00:00",
            "title": "On Laughter and Speech-Laugh, Based on Observations of Child-Robot Interaction",
            "authors": [
                "Anton Batliner",
                "Stefan Steidl",
                "Florian Eyben",
                "Bj\u00f6rn Schuller"
            ],
            "abstract": "In this article, we study laughter found in child-robot interaction where it\nhad not been prompted intentionally. Different types of laughter and\nspeech-laugh are annotated and processed. In a descriptive part, we report on\nthe position of laughter and speech-laugh in syntax and dialogue structure, and\non communicative functions. In a second part, we report on automatic\nclassification performance and on acoustic characteristics, based on extensive\nfeature selection procedures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.11658v1",
            "date": "2019-08-30 11:31:07+00:00",
            "title": "Autoregressive Text Generation Beyond Feedback Loops",
            "authors": [
                "Florian Schmidt",
                "Stephan Mandt",
                "Thomas Hofmann"
            ],
            "abstract": "Autoregressive state transitions, where predictions are conditioned on past\npredictions, are the predominant choice for both deterministic and stochastic\nsequential models. However, autoregressive feedback exposes the evolution of\nthe hidden state trajectory to potential biases from well-known train-test\ndiscrepancies. In this paper, we combine a latent state space model with a CRF\nobservation model. We argue that such autoregressive observation models form an\ninteresting middle ground that expresses local correlations on the word level\nbut keeps the state evolution non-autoregressive. On unconditional sentence\ngeneration we show performance improvements compared to RNN and GAN baselines\nwhile avoiding some prototypical failure modes of autoregressive models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.11685v1",
            "date": "2019-08-26 00:43:32+00:00",
            "title": "Using LSTMs to Model the Java Programming Language",
            "authors": [
                "Brendon Boldt"
            ],
            "abstract": "Recurrent neural networks (RNNs), specifically long-short term memory\nnetworks (LSTMs), can model natural language effectively. This research\ninvestigates the ability for these same LSTMs to perform next \"word\" prediction\non the Java programming language. Java source code from four different\nrepositories undergoes a transformation that preserves the logical structure of\nthe source code and removes the code's various specificities such as variable\nnames and literal values. Such datasets and an additional English language\ncorpus are used to train and test standard LSTMs' ability to predict the next\nelement in a sequence. Results suggest that LSTMs can effectively model Java\ncode achieving perplexities under 22 and accuracies above 0.47, which is an\nimprovement over LSTM's performance on the English language which demonstrated\na perplexity of 85 and an accuracy of 0.27. This research can have\napplicability in other areas such as syntactic template suggestion and\nautomated bug patching.",
            "journal ref": "Artificial Neural Networks and Machine Learning -- ICANN 2017.\n  ICANN 2017. Lecture Notes in Computer Science, vol 10614",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1908.11722v1",
            "date": "2019-08-30 13:12:21+00:00",
            "title": "Fact-Checking Meets Fauxtography: Verifying Claims About Images",
            "authors": [
                "Dimitrina Zlatkova",
                "Preslav Nakov",
                "Ivan Koychev"
            ],
            "abstract": "The recent explosion of false claims in social media and on the Web in\ngeneral has given rise to a lot of manual fact-checking initiatives.\nUnfortunately, the number of claims that need to be fact-checked is several\norders of magnitude larger than what humans can handle manually. Thus, there\nhas been a lot of research aiming at automating the process. Interestingly,\nprevious work has largely ignored the growing number of claims about images.\nThis is despite the fact that visual imagery is more influential than text and\nnaturally appears alongside fake news. Here we aim at bridging this gap. In\nparticular, we create a new dataset for this problem, and we explore a variety\nof features modeling the claim, the image, and the relationship between the\nclaim and the image. The evaluation results show sizable improvements over the\nbaseline. We release our dataset, hoping to enable further research on\nfact-checking claims about images.",
            "journal ref": "EMNLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00021v2",
            "date": "2019-08-30 18:18:04+00:00",
            "title": "Approximating Stacked and Bidirectional Recurrent Architectures with the Delayed Recurrent Neural Network",
            "authors": [
                "Javier S. Turek",
                "Shailee Jain",
                "Vy Vo",
                "Mihai Capota",
                "Alexander G. Huth",
                "Theodore L. Willke"
            ],
            "abstract": "Recent work has shown that topological enhancements to recurrent neural\nnetworks (RNNs) can increase their expressiveness and representational\ncapacity. Two popular enhancements are stacked RNNs, which increases the\ncapacity for learning non-linear functions, and bidirectional processing, which\nexploits acausal information in a sequence. In this work, we explore the\ndelayed-RNN, which is a single-layer RNN that has a delay between the input and\noutput. We prove that a weight-constrained version of the delayed-RNN is\nequivalent to a stacked-RNN. We also show that the delay gives rise to partial\nacausality, much like bidirectional networks. Synthetic experiments confirm\nthat the delayed-RNN can mimic bidirectional networks, solving some acausal\ntasks similarly, and outperforming them in others. Moreover, we show similar\nperformance to bidirectional networks in a real-world natural language\nprocessing task. These results suggest that delayed-RNNs can approximate\ntopologies including stacked RNNs, bidirectional RNNs, and stacked\nbidirectional RNNs - but with equivalent or faster runtimes for the\ndelayed-RNNs.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00088v2",
            "date": "2019-08-30 23:10:28+00:00",
            "title": "Keep Calm and Switch On! Preserving Sentiment and Fluency in Semantic Text Exchange",
            "authors": [
                "Steven Y. Feng",
                "Aaron W. Li",
                "Jesse Hoey"
            ],
            "abstract": "In this paper, we present a novel method for measurably adjusting the\nsemantics of text while preserving its sentiment and fluency, a task we call\nsemantic text exchange. This is useful for text data augmentation and the\nsemantic correction of text generated by chatbots and virtual assistants. We\nintroduce a pipeline called SMERTI that combines entity replacement, similarity\nmasking, and text infilling. We measure our pipeline's success by its Semantic\nText Exchange Score (STES): the ability to preserve the original text's\nsentiment and fluency while adjusting semantic content. We propose to use\nmasking (replacement) rate threshold as an adjustable parameter to control the\namount of semantic change in the text. Our experiments demonstrate that SMERTI\ncan outperform baseline models on Yelp reviews, Amazon reviews, and news\nheadlines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00102v1",
            "date": "2019-08-31 01:04:58+00:00",
            "title": "Knowledge Enhanced Attention for Robust Natural Language Inference",
            "authors": [
                "Alexander Hanbo Li",
                "Abhinav Sethy"
            ],
            "abstract": "Neural network models have been very successful at achieving high accuracy on\nnatural language inference (NLI) tasks. However, as demonstrated in recent\nliterature, when tested on some simple adversarial examples, most of the models\nsuffer a significant drop in performance. This raises the concern about the\nrobustness of NLI models. In this paper, we propose to make NLI models robust\nby incorporating external knowledge to the attention mechanism using a simple\ntransformation. We apply the new attention to two popular types of NLI models:\none is Transformer encoder, and the other is a decomposable model, and show\nthat our method can significantly improve their robustness. Moreover, when\ncombined with BERT pretraining, our method achieves the human-level performance\non the adversarial SNLI data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00105v1",
            "date": "2019-08-31 01:50:42+00:00",
            "title": "Generating Personalized Recipes from Historical User Preferences",
            "authors": [
                "Bodhisattwa Prasad Majumder",
                "Shuyang Li",
                "Jianmo Ni",
                "Julian McAuley"
            ],
            "abstract": "Existing approaches to recipe generation are unable to create recipes for\nusers with culinary preferences but incomplete knowledge of ingredients in\nspecific dishes. We propose a new task of personalized recipe generation to\nhelp these users: expanding a name and incomplete ingredient details into\ncomplete natural-text instructions aligned with the user's historical\npreferences. We attend on technique- and recipe-level representations of a\nuser's previously consumed recipes, fusing these 'user-aware' representations\nin an attention fusion layer to control recipe text generation. Experiments on\na new dataset of 180K recipes and 700K interactions show our model's ability to\ngenerate plausible and personalized recipes compared to non-personalized\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00126v4",
            "date": "2019-08-31 04:38:06+00:00",
            "title": "A Logic-Driven Framework for Consistency of Neural Models",
            "authors": [
                "Tao Li",
                "Vivek Gupta",
                "Maitrey Mehta",
                "Vivek Srikumar"
            ],
            "abstract": "While neural models show remarkable accuracy on individual predictions, their\ninternal beliefs can be inconsistent across examples. In this paper, we\nformalize such inconsistency as a generalization of prediction error. We\npropose a learning framework for constraining models using logic rules to\nregularize them away from inconsistency. Our framework can leverage both\nlabeled and unlabeled examples and is directly compatible with off-the-shelf\nlearning schemes without model redesign. We instantiate our framework on\nnatural language inference, where experiments show that enforcing invariants\nstated in logic can help make the predictions of neural models both accurate\nand consistent.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.00131v1",
            "date": "2019-08-31 05:28:51+00:00",
            "title": "Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite",
            "authors": [
                "Prathyusha Jwalapuram",
                "Shafiq Joty",
                "Irina Temnikova",
                "Preslav Nakov"
            ],
            "abstract": "The ongoing neural revolution in machine translation has made it easier to\nmodel larger contexts beyond the sentence-level, which can potentially help\nresolve some discourse-level ambiguities such as pronominal anaphora, thus\nenabling better translations. Unfortunately, even when the resulting\nimprovements are seen as substantial by humans, they remain virtually unnoticed\nby traditional automatic evaluation measures like BLEU, as only a few words end\nup being affected. Thus, specialized evaluation measures are needed. With this\naim in mind, we contribute an extensive, targeted dataset that can be used as a\ntest suite for pronoun translation, covering multiple source languages and\ndifferent pronoun errors drawn from real system translations, for English. We\nfurther propose an evaluation measure to differentiate good and bad pronoun\ntranslations. We also conduct a user study to report correlations with human\njudgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00141v2",
            "date": "2019-08-31 06:13:33+00:00",
            "title": "Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization",
            "authors": [
                "Siyao Li",
                "Deren Lei",
                "Pengda Qin",
                "William Yang Wang"
            ],
            "abstract": "Deep reinforcement learning (RL) has been a commonly-used strategy for the\nabstractive summarization task to address both the exposure bias and\nnon-differentiable task issues. However, the conventional reward Rouge-L simply\nlooks for exact n-grams matches between candidates and annotated references,\nwhich inevitably makes the generated sentences repetitive and incoherent. In\nthis paper, instead of Rouge-L, we explore the practicability of utilizing the\ndistributional semantics to measure the matching degrees. With distributional\nsemantics, sentence-level evaluation can be obtained, and semantically-correct\nphrases can also be generated without being limited to the surface form of the\nreference sentences. Human judgments on Gigaword and CNN/Daily Mail datasets\nshow that our proposed distributional semantics reward (DSR) has distinct\nsuperiority in capturing the lexical and compositional diversity of natural\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00154v1",
            "date": "2019-08-31 07:05:43+00:00",
            "title": "Rethinking travel behavior modeling representations through embeddings",
            "authors": [
                "Francisco C. Pereira"
            ],
            "abstract": "This paper introduces the concept of travel behavior embeddings, a method for\nre-representing discrete variables that are typically used in travel demand\nmodeling, such as mode, trip purpose, education level, family type or\noccupation. This re-representation process essentially maps those variables\ninto a latent space called the \\emph{embedding space}. The benefit of this is\nthat such spaces allow for richer nuances than the typical transformations used\nin categorical variables (e.g. dummy encoding, contrasted encoding, principal\ncomponents analysis). While the usage of latent variable representations is not\nnew per se in travel demand modeling, the idea presented here brings several\ninnovations: it is an entirely data driven algorithm; it is informative and\nconsistent, since the latent space can be visualized and interpreted based on\ndistances between different categories; it preserves interpretability of\ncoefficients, despite being based on Neural Network principles; and it is\ntransferrable, in that embeddings learned from one dataset can be reused for\nother ones, as long as travel behavior keeps consistent between the datasets.\n  The idea is strongly inspired on natural language processing techniques,\nnamely the word2vec algorithm. Such algorithm is behind recent developments\nsuch as in automatic translation or next word prediction. Our method is\ndemonstrated using a model choice model, and shows improvements of up to 60\\%\nwith respect to initial likelihood, and up to 20% with respect to likelihood of\nthe corresponding traditional model (i.e. using dummy variables) in\nout-of-sample evaluation. We provide a new Python package, called PyTre (PYthon\nTRavel Embeddings), that others can straightforwardly use to replicate our\nresults or improve their own models. Our experiments are themselves based on an\nopen dataset (swissmetro).",
            "journal ref": "None",
            "category": "econ.EM"
        },
        {
            "url": "http://arxiv.org/abs/1909.00160v1",
            "date": "2019-08-31 07:41:42+00:00",
            "title": "Incorporating Domain Knowledge into Medical NLI using Knowledge Graphs",
            "authors": [
                "Soumya Sharma",
                "Bishal Santra",
                "Abhik Jana",
                "T. Y. S. S. Santosh",
                "Niloy Ganguly",
                "Pawan Goyal"
            ],
            "abstract": "Recently, biomedical version of embeddings obtained from language models such\nas BioELMo have shown state-of-the-art results for the textual inference task\nin the medical domain. In this paper, we explore how to incorporate structured\ndomain knowledge, available in the form of a knowledge graph (UMLS), for the\nMedical NLI task. Specifically, we experiment with fusing embeddings obtained\nfrom knowledge graph with the state-of-the-art approaches for NLI task (ESIM\nmodel). We also experiment with fusing the domain-specific sentiment\ninformation for the task. Experiments conducted on MedNLI dataset clearly show\nthat this strategy improves the baseline BioELMo architecture for the Medical\nNLI task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00215v1",
            "date": "2019-08-31 13:50:28+00:00",
            "title": "QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization",
            "authors": [
                "Yi-Ting Yeh",
                "Yun-Nung Chen"
            ],
            "abstract": "Standard accuracy metrics indicate that modern reading comprehension systems\nhave achieved strong performance in many question answering datasets. However,\nthe extent these systems truly understand language remains unknown, and\nexisting systems are not good at distinguishing distractor sentences, which\nlook related but do not actually answer the question. To address this problem,\nwe propose QAInfomax as a regularizer in reading comprehension systems by\nmaximizing mutual information among passages, a question, and its answer.\nQAInfomax helps regularize the model to not simply learn the superficial\ncorrelation for answering questions. The experiments show that our proposed\nQAInfomax achieves the state-of-the-art performance on the benchmark\nAdversarial-SQuAD dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00230v1",
            "date": "2019-08-31 15:46:05+00:00",
            "title": "Collaborative Policy Learning for Open Knowledge Graph Reasoning",
            "authors": [
                "Cong Fu",
                "Tong Chen",
                "Meng Qu",
                "Woojeong Jin",
                "Xiang Ren"
            ],
            "abstract": "In recent years, there has been a surge of interests in interpretable graph\nreasoning methods. However, these models often suffer from limited performance\nwhen working on sparse and incomplete graphs, due to the lack of evidential\npaths that can reach target entities. Here we study open knowledge graph\nreasoning---a task that aims to reason for missing facts over a graph augmented\nby a background text corpus. A key challenge of the task is to filter out\n\"irrelevant\" facts extracted from corpus, in order to maintain an effective\nsearch space during path inference. We propose a novel reinforcement learning\nframework to train two collaborative agents jointly, i.e., a multi-hop graph\nreasoner and a fact extractor. The fact extraction agent generates fact triples\nfrom corpora to enrich the graph on the fly; while the reasoning agent provides\nfeedback to the fact extractor and guides it towards promoting facts that are\nhelpful for the interpretable reasoning. Experiments on two public datasets\ndemonstrate the effectiveness of the proposed approach. Source code and\ndatasets used in this paper can be downloaded at\nhttps://github.com/shanzhenren/CPL",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.00279v1",
            "date": "2019-08-31 20:07:25+00:00",
            "title": "Generating Classical Chinese Poems from Vernacular Chinese",
            "authors": [
                "Zhichao Yang",
                "Pengshan Cai",
                "Yansong Feng",
                "Fei Li",
                "Weijiang Feng",
                "Elena Suet-Ying Chiu",
                "Hong Yu"
            ],
            "abstract": "Classical Chinese poetry is a jewel in the treasure house of Chinese culture.\nPrevious poem generation models only allow users to employ keywords to\ninterfere the meaning of generated poems, leaving the dominion of generation to\nthe model. In this paper, we propose a novel task of generating classical\nChinese poems from vernacular, which allows users to have more control over the\nsemantic of generated poems. We adapt the approach of unsupervised machine\ntranslation (UMT) to our task. We use segmentation-based padding and\nreinforcement learning to address under-translation and over-translation\nrespectively. According to experiments, our approach significantly improve the\nperplexity and BLEU compared with typical UMT models. Furthermore, we explored\nguidelines on how to write the input vernacular to generate better poems. Human\nevaluation showed our approach can generate high-quality poems which are\ncomparable to amateur poems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00333v3",
            "date": "2019-09-01 06:30:57+00:00",
            "title": "QuASE: Question-Answer Driven Sentence Encoding",
            "authors": [
                "Hangfeng He",
                "Qiang Ning",
                "Dan Roth"
            ],
            "abstract": "Question-answering (QA) data often encodes essential information in many\nfacets. This paper studies a natural question: Can we get supervision from QA\ndata for other tasks (typically, non-QA ones)? For example, {\\em can we use\nQAMR (Michael et al., 2017) to improve named entity recognition?} We suggest\nthat simply further pre-training BERT is often not the best option, and propose\nthe {\\em question-answer driven sentence encoding (QuASE)} framework. QuASE\nlearns representations from QA data, using BERT or other state-of-the-art\ncontextual language models. In particular, we observe the need to distinguish\nbetween two types of sentence encodings, depending on whether the target task\nis a single- or multi-sentence input; in both cases, the resulting encoding is\nshown to be an easy-to-use plugin for many downstream tasks. This work may\npoint out an alternative way to supervise NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00349v1",
            "date": "2019-09-01 08:16:53+00:00",
            "title": "A Unified Neural Coherence Model",
            "authors": [
                "Han Cheol Moon",
                "Tasnim Mohiuddin",
                "Shafiq Joty",
                "Xu Chi"
            ],
            "abstract": "Recently, neural approaches to coherence modeling have achieved\nstate-of-the-art results in several evaluation tasks. However, we show that\nmost of these models often fail on harder tasks with more realistic application\nscenarios. In particular, the existing models underperform on tasks that\nrequire the model to be sensitive to local contexts such as candidate ranking\nin conversational dialogue and in machine translation. In this paper, we\npropose a unified coherence model that incorporates sentence grammar,\ninter-sentence coherence relations, and global coherence patterns into a common\nneural framework. With extensive experiments on local and global discrimination\ntasks, we demonstrate that our proposed model outperforms existing models by a\ngood margin, and establish a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00361v1",
            "date": "2019-09-01 09:14:52+00:00",
            "title": "Cross-Lingual Machine Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Wanxiang Che",
                "Ting Liu",
                "Bing Qin",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Though the community has made great progress on Machine Reading Comprehension\n(MRC) task, most of the previous works are solving English-based MRC problems,\nand there are few efforts on other languages mainly due to the lack of\nlarge-scale training data. In this paper, we propose Cross-Lingual Machine\nReading Comprehension (CLMRC) task for the languages other than English.\nFirstly, we present several back-translation approaches for CLMRC task, which\nis straightforward to adopt. However, to accurately align the answer into\nanother language is difficult and could introduce additional noise. In this\ncontext, we propose a novel model called Dual BERT, which takes advantage of\nthe large-scale training data provided by rich-resource language (such as\nEnglish) and learn the semantic relations between the passage and question in a\nbilingual context, and then utilize the learned knowledge to improve reading\ncomprehension performance of low-resource language. We conduct experiments on\ntwo Chinese machine reading comprehension datasets CMRC 2018 and DRCD. The\nresults show consistent and significant improvements over various\nstate-of-the-art systems by a large margin, which demonstrate the potentials in\nCLMRC task. Resources available: https://github.com/ymcui/Cross-Lingual-MRC",
            "journal ref": "EMNLP 2019 1586-1595",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00393v1",
            "date": "2019-09-01 13:24:35+00:00",
            "title": "A Dataset of General-Purpose Rebuttal",
            "authors": [
                "Matan Orbach",
                "Yonatan Bilu",
                "Ariel Gera",
                "Yoav Kantor",
                "Lena Dankin",
                "Tamar Lavee",
                "Lili Kotlerman",
                "Shachar Mirkin",
                "Michal Jacovi",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "In Natural Language Understanding, the task of response generation is usually\nfocused on responses to short texts, such as tweets or a turn in a dialog. Here\nwe present a novel task of producing a critical response to a long\nargumentative text, and suggest a method based on general rebuttal arguments to\naddress it. We do this in the context of the recently-suggested task of\nlistening comprehension over argumentative content: given a speech on some\nspecified topic, and a list of relevant arguments, the goal is to determine\nwhich of the arguments appear in the speech. The general rebuttals we describe\nhere (written in English) overcome the need for topic-specific arguments to be\nprovided, by proving to be applicable for a large set of topics. This allows\ncreating responses beyond the scope of topics for which specific arguments are\navailable. All data collected during this work is freely available for\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00415v1",
            "date": "2019-09-01 15:12:23+00:00",
            "title": "Leveraging Just a Few Keywords for Fine-Grained Aspect Detection Through Weakly Supervised Co-Training",
            "authors": [
                "Giannis Karamanolakis",
                "Daniel Hsu",
                "Luis Gravano"
            ],
            "abstract": "User-generated reviews can be decomposed into fine-grained segments (e.g.,\nsentences, clauses), each evaluating a different aspect of the principal entity\n(e.g., price, quality, appearance). Automatically detecting these aspects can\nbe useful for both users and downstream opinion mining applications. Current\nsupervised approaches for learning aspect classifiers require many fine-grained\naspect labels, which are labor-intensive to obtain. And, unfortunately,\nunsupervised topic models often fail to capture the aspects of interest. In\nthis work, we consider weakly supervised approaches for training aspect\nclassifiers that only require the user to provide a small set of seed words\n(i.e., weakly positive indicators) for the aspects of interest. First, we show\nthat current weakly supervised approaches do not effectively leverage the\npredictive power of seed words for aspect detection. Next, we propose a\nstudent-teacher approach that effectively leverages seed words in a\nbag-of-words classifier (teacher); in turn, we use the teacher to train a\nsecond model (student) that is potentially more powerful (e.g., a neural\nnetwork that uses pre-trained word embeddings). Finally, we show that iterative\nco-training can be used to cope with noisy seed words, leading to both improved\nteacher and student models. Our proposed approach consistently outperforms\nprevious weakly supervised approaches (by 14.1 absolute F1 points on average)\nin six different domains of product reviews and six multilingual datasets of\nrestaurant reviews.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00430v1",
            "date": "2019-09-01 17:11:35+00:00",
            "title": "Transfer Learning Between Related Tasks Using Expected Label Proportions",
            "authors": [
                "Matan Ben Noach",
                "Yoav Goldberg"
            ],
            "abstract": "Deep learning systems thrive on abundance of labeled training data but such\ndata is not always available, calling for alternative methods of supervision.\nOne such method is expectation regularization (XR) (Mann and McCallum, 2007),\nwhere models are trained based on expected label proportions. We propose a\nnovel application of the XR framework for transfer learning between related\ntasks, where knowing the labels of task A provides an estimation of the label\nproportion of task B. We then use a model trained for A to label a large\ncorpus, and use this corpus with an XR loss to train a model for task B. To\nmake the XR framework applicable to large-scale deep-learning setups, we\npropose a stochastic batched approximation procedure. We demonstrate the\napproach on the task of Aspect-based Sentiment classification, where we\neffectively use a sentence-level sentiment predictor to train accurate\naspect-based predictor. The method improves upon fully supervised neural system\ntrained on aspect-level data, and is also cumulative with LM-based pretraining,\nas we demonstrate by improving a BERT-based Aspect-based Sentiment model.",
            "journal ref": "2019 Conference on Empirical Methods in Natural Language\n  Processing and 9th International Joint Conference on Natural Language\n  Processing",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00453v1",
            "date": "2019-09-01 19:18:44+00:00",
            "title": "Topics to Avoid: Demoting Latent Confounds in Text Classification",
            "authors": [
                "Sachin Kumar",
                "Shuly Wintner",
                "Noah A. Smith",
                "Yulia Tsvetkov"
            ],
            "abstract": "Despite impressive performance on many text classification tasks, deep neural\nnetworks tend to learn frequent superficial patterns that are specific to the\ntraining data and do not always generalize well. In this work, we observe this\nlimitation with respect to the task of native language identification. We find\nthat standard text classifiers which perform well on the test set end up\nlearning topical features which are confounds of the prediction task (e.g., if\nthe input text mentions Sweden, the classifier predicts that the author's\nnative language is Swedish). We propose a method that represents the latent\ntopical confounds and a model which \"unlearns\" confounding features by\npredicting both the label of the input text and the confound; but we train the\ntwo predictors adversarially in an alternating fashion to learn a text\nrepresentation that predicts the correct label but is less prone to using\ninformation about the confound. We show that this model generalizes better and\nlearns features that are indicative of the writing style rather than the\ncontent.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00505v1",
            "date": "2019-09-02 01:41:00+00:00",
            "title": "Commonsense Knowledge Mining from Pretrained Models",
            "authors": [
                "Joshua Feldman",
                "Joe Davison",
                "Alexander M. Rush"
            ],
            "abstract": "Inferring commonsense knowledge is a key challenge in natural language\nprocessing, but due to the sparsity of training data, previous work has shown\nthat supervised methods for commonsense knowledge mining underperform when\nevaluated on novel data. In this work, we develop a method for generating\ncommonsense knowledge using a large, pre-trained bidirectional language model.\nBy transforming relational triples into masked sentences, we can use this model\nto rank a triple's validity by the estimated pointwise mutual information\nbetween the two entities. Since we do not update the weights of the\nbidirectional model, our approach is not biased by the coverage of any one\ncommonsense knowledge base. Though this method performs worse on a test set\nthan models explicitly trained on a corresponding training set, it outperforms\nthese methods when mining commonsense knowledge from new sources, suggesting\nthat unsupervised techniques may generalize better than current supervised\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.00562v2",
            "date": "2019-09-02 06:41:34+00:00",
            "title": "Hybrid Data-Model Parallel Training for Sequence-to-Sequence Recurrent Neural Network Machine Translation",
            "authors": [
                "Junya Ono",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "Reduction of training time is an important issue in many tasks like patent\ntranslation involving neural networks. Data parallelism and model parallelism\nare two common approaches for reducing training time using multiple graphics\nprocessing units (GPUs) on one machine. In this paper, we propose a hybrid\ndata-model parallel approach for sequence-to-sequence (Seq2Seq) recurrent\nneural network (RNN) machine translation. We apply a model parallel approach to\nthe RNN encoder-decoder part of the Seq2Seq model and a data parallel approach\nto the attention-softmax part of the model. We achieved a speed-up of 4.13 to\n4.20 times when using 4 GPUs compared with the training speed when using 1 GPU\nwithout affecting machine translation accuracy as measured in terms of BLEU\nscores.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1909.00868v1",
            "date": "2019-09-02 21:08:00+00:00",
            "title": "A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text",
            "authors": [
                "Bohan Li",
                "Junxian He",
                "Graham Neubig",
                "Taylor Berg-Kirkpatrick",
                "Yiming Yang"
            ],
            "abstract": "When trained effectively, the Variational Autoencoder (VAE) is both a\npowerful language model and an effective representation learning framework. In\npractice, however, VAEs are trained with the evidence lower bound (ELBO) as a\nsurrogate objective to the intractable marginal data likelihood. This approach\nto training yields unstable results, frequently leading to a disastrous local\noptimum known as posterior collapse. In this paper, we investigate a simple fix\nfor posterior collapse which yields surprisingly effective results. The\ncombination of two known heuristics, previously considered only in isolation,\nsubstantially improves held-out likelihood, reconstruction, and latent\nrepresentation learning when compared with previous state-of-the-art methods.\nMore interestingly, while our experiments demonstrate superiority on these\nprinciple evaluations, our method obtains a worse ELBO. We use these results to\nargue that the typical surrogate objective for VAEs may not be sufficient or\nnecessarily appropriate for balancing the goals of representation learning and\ndata distribution modeling.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00997v3",
            "date": "2019-09-03 08:23:51+00:00",
            "title": "PlotQA: Reasoning over Scientific Plots",
            "authors": [
                "Nitesh Methani",
                "Pritha Ganguly",
                "Mitesh M. Khapra",
                "Pratyush Kumar"
            ],
            "abstract": "Existing synthetic datasets (FigureQA, DVQA) for reasoning over plots do not\ncontain variability in data labels, real-valued data, or complex reasoning\nquestions. Consequently, proposed models for these datasets do not fully\naddress the challenge of reasoning over plots. In particular, they assume that\nthe answer comes either from a small fixed size vocabulary or from a bounding\nbox within the image. However, in practice, this is an unrealistic assumption\nbecause many questions require reasoning and thus have real-valued answers\nwhich appear neither in a small fixed size vocabulary nor in the image. In this\nwork, we aim to bridge this gap between existing datasets and real-world plots.\nSpecifically, we propose PlotQA with 28.9 million question-answer pairs over\n224,377 plots on data from real-world sources and questions based on\ncrowd-sourced question templates. Further, 80.76% of the out-of-vocabulary\n(OOV) questions in PlotQA have answers that are not in a fixed vocabulary.\nAnalysis of existing models on PlotQA reveals that they cannot deal with OOV\nquestions: their overall accuracy on our dataset is in single digits. This is\nnot surprising given that these models were not designed for such questions. As\na step towards a more holistic model which can address fixed vocabulary as well\nas OOV questions, we propose a hybrid approach: Specific questions are answered\nby choosing the answer from a fixed vocabulary or by extracting it from a\npredicted bounding box in the plot, while other questions are answered with a\ntable question-answering engine which is fed with a structured table generated\nby detecting visual elements from the image. On the existing DVQA dataset, our\nmodel has an accuracy of 58%, significantly improving on the highest reported\naccuracy of 46%. On PlotQA, our model has an accuracy of 22.52%, which is\nsignificantly better than state of the art models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.01120v2",
            "date": "2019-09-03 12:45:02+00:00",
            "title": "Local Embeddings for Relational Data Integration",
            "authors": [
                "Riccardo Cappuzzo",
                "Paolo Papotti",
                "Saravanan Thirumuruganathan"
            ],
            "abstract": "Deep learning based techniques have been recently used with promising results\nfor data integration problems. Some methods directly use pre-trained embeddings\nthat were trained on a large corpus such as Wikipedia. However, they may not\nalways be an appropriate choice for enterprise datasets with custom vocabulary.\nOther methods adapt techniques from natural language processing to obtain\nembeddings for the enterprise's relational data. However, this approach blindly\ntreats a tuple as a sentence, thus losing a large amount of contextual\ninformation present in the tuple.\n  We propose algorithms for obtaining local embeddings that are effective for\ndata integration tasks on relational databases. We make four major\ncontributions. First, we describe a compact graph-based representation that\nallows the specification of a rich set of relationships inherent in the\nrelational world. Second, we propose how to derive sentences from such a graph\nthat effectively \"describe\" the similarity across elements (tokens, attributes,\nrows) in the two datasets. The embeddings are learned based on such sentences.\nThird, we propose effective optimization to improve the quality of the learned\nembeddings and the performance of integration tasks. Finally, we propose a\ndiverse collection of criteria to evaluate relational embeddings and perform an\nextensive set of experiments validating them against multiple baseline methods.\nOur experiments show that our framework, EmbDI, produces meaningful results for\ndata integration tasks such as schema matching and entity resolution both in\nsupervised and unsupervised settings.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1909.01135v3",
            "date": "2019-08-28 23:58:50+00:00",
            "title": "HTMLPhish: Enabling Phishing Web Page Detection by Applying Deep Learning Techniques on HTML Analysis",
            "authors": [
                "Chidimma Opara",
                "Bo Wei",
                "Yingke Chen"
            ],
            "abstract": "Recently, the development and implementation of phishing attacks require\nlittle technical skills and costs. This uprising has led to an ever-growing\nnumber of phishing attacks on the World Wide Web. Consequently, proactive\ntechniques to fight phishing attacks have become extremely necessary. In this\npaper, we propose HTMLPhish, a deep learning based data-driven end-to-end\nautomatic phishing web page classification approach. Specifically, HTMLPhish\nreceives the content of the HTML document of a web page and employs\nConvolutional Neural Networks (CNNs) to learn the semantic dependencies in the\ntextual contents of the HTML. The CNNs learn appropriate feature\nrepresentations from the HTML document embeddings without extensive manual\nfeature engineering. Furthermore, our proposed approach of the concatenation of\nthe word and character embeddings allows our model to manage new features and\nensure easy extrapolation to test data. We conduct comprehensive experiments on\na dataset of more than 50,000 HTML documents that provides a distribution of\nphishing to benign web pages obtainable in the real-world that yields over 93\npercent Accuracy and True Positive Rate. Also, HTMLPhish is a completely\nlanguage-independent and client-side strategy which can, therefore, conduct web\npage phishing detection regardless of the textual language.",
            "journal ref": "International Joint Conference on Neural Networks (IJCNN) 2020",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1909.01136v5",
            "date": "2019-08-30 17:25:06+00:00",
            "title": "Pre-training A Neural Language Model Improves The Sample Efficiency of an Emergency Room Classification Model",
            "authors": [
                "Binbin Xu",
                "C\u00e9dric Gil-Jardin\u00e9",
                "Frantz Thiessard",
                "Eric Tellier",
                "Marta Avalos",
                "Emmanuel Lagarde"
            ],
            "abstract": "To build a French national electronic injury surveillance system based on\nemergency room visits, we aim to develop a coding system to classify their\ncauses from clinical notes in free-text. Supervised learning techniques have\nshown good results in this area but require a large amount of expert annotated\ndataset which is time consuming and costly to obtain. We hypothesize that the\nNatural Language Processing Transformer model incorporating a generative\nself-supervised pre-training step can significantly reduce the required number\nof annotated samples for supervised fine-tuning. In this preliminary study, we\ntest our hypothesis in the simplified problem of predicting whether a visit is\nthe consequence of a traumatic event or not from free-text clinical notes.\nUsing fully re-trained GPT-2 models (without OpenAI pre-trained weights), we\nassess the gain of applying a self-supervised pre-training phase with unlabeled\nnotes prior to the supervised learning task. Results show that the number of\ndata required to achieve a ginve level of performance (AUC>0.95) was reduced by\na factor of 10 when applying pre-training. Namely, for 16 times more data, the\nfully-supervised model achieved an improvement <1% in AUC. To conclude, it is\npossible to adapt a multi-purpose neural language model such as the GPT-2 to\ncreate a powerful tool for classification of free-text notes with only a small\nnumber of labeled samples.",
            "journal ref": "The 33rd Florida Artificial Intelligence Research Society\n  Conference, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01145v2",
            "date": "2019-08-30 04:03:14+00:00",
            "title": "Maximizing Mutual Information for Tacotron",
            "authors": [
                "Peng Liu",
                "Xixin Wu",
                "Shiyin Kang",
                "Guangzhi Li",
                "Dan Su",
                "Dong Yu"
            ],
            "abstract": "End-to-end speech synthesis methods already achieve close-to-human quality\nperformance. However compared to HMM-based and NN-based frame-to-frame\nregression methods, they are prone to some synthesis errors, such as missing or\nrepeating words and incomplete synthesis. We attribute the comparatively high\nutterance error rate to the local information preference of conditional\nautoregressive models, and the ill-posed training objective of the model, which\ndescribes mostly the training status of the autoregressive module, but rarely\nthat of the condition module. Inspired by InfoGAN, we propose to maximize the\nmutual information between the text condition and the predicted acoustic\nfeatures to strengthen the dependency between them for CAR speech synthesis\nmodel, which would alleviate the local information preference issue and reduce\nthe utterance error rate. The training objective of maximizing mutual\ninformation can be considered as a metric of the dependency between the\nautoregressive module and the condition module. Experiment results show that\nour method can reduce the utterance error rate.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.01401v1",
            "date": "2019-09-03 18:54:43+00:00",
            "title": "Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings",
            "authors": [
                "Pengfei Sun",
                "Gopala K. Anumanchipalli",
                "Edward F. Chang"
            ],
            "abstract": "Decoding language representations directly from the brain can enable new\nBrain-Computer Interfaces (BCI) for high bandwidth human-human and\nhuman-machine communication. Clinically, such technologies can restore\ncommunication in people with neurological conditions affecting their ability to\nspeak. In this study, we propose a novel deep network architecture Brain2Char,\nfor directly decoding text (specifically character sequences) from direct brain\nrecordings (called Electrocorticography, ECoG). Brain2Char framework combines\nstate-of-the-art deep learning modules --- 3D Inception layers for multiband\nspatiotemporal feature extraction from neural data and bidirectional recurrent\nlayers, dilated convolution layers followed by language model weighted beam\nsearch to decode character sequences, optimizing a connectionist temporal\nclassification (CTC) loss. Additionally, given the highly non-linear\ntransformations that underlie the conversion of cortical function to character\nsequences, we perform regularizations on the network's latent representations\nmotivated by insights into cortical encoding of speech production and\nartifactual aspects specific to ECoG data acquisition. To do this, we impose\nauxiliary losses on latent representations for articulatory movements, speech\nacoustics and session specific non-linearities. In 3 participants tested here,\nBrain2Char achieves 10.6\\%, 8.5\\% and 7.0\\% Word Error Rates (WER) respectively\non vocabulary sizes ranging from 1200 to 1900 words. Brain2Char also performs\nwell when 2 participants silently mimed sentences. These results set a new\nstate-of-the-art on decoding text from brain and demonstrate the potential of\nBrain2Char as a high-performance communication BCI.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.01459v1",
            "date": "2019-09-03 21:20:28+00:00",
            "title": "Interpretable Word Embeddings via Informative Priors",
            "authors": [
                "Miriam Hurtado Bodell",
                "Martin Arvidsson",
                "M\u00e5ns Magnusson"
            ],
            "abstract": "Word embeddings have demonstrated strong performance on NLP tasks. However,\nlack of interpretability and the unsupervised nature of word embeddings have\nlimited their use within computational social science and digital humanities.\nWe propose the use of informative priors to create interpretable and\ndomain-informed dimensions for probabilistic word embeddings. Experimental\nresults show that sensible priors can capture latent semantic concepts better\nthan or on-par with the current state of the art, while retaining the\nsimplicity and generalizability of using priors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01492v2",
            "date": "2019-09-03 23:03:10+00:00",
            "title": "Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation",
            "authors": [
                "Po-Sen Huang",
                "Robert Stanforth",
                "Johannes Welbl",
                "Chris Dyer",
                "Dani Yogatama",
                "Sven Gowal",
                "Krishnamurthy Dvijotham",
                "Pushmeet Kohli"
            ],
            "abstract": "Neural networks are part of many contemporary NLP systems, yet their\nempirical successes come at the price of vulnerability to adversarial attacks.\nPrevious work has used adversarial training and data augmentation to partially\nmitigate such brittleness, but these are unlikely to find worst-case\nadversaries due to the complexity of the search space arising from discrete\ntext perturbations. In this work, we approach the problem from the opposite\ndirection: to formally verify a system's robustness against a predefined class\nof adversarial attacks. We study text classification under synonym replacements\nor character flip perturbations. We propose modeling these input perturbations\nas a simplex and then using Interval Bound Propagation -- a formal model\nverification method. We modify the conventional log-likelihood training\nobjective to train models that can be efficiently verified, which would\notherwise come with exponential search complexity. The resulting models show\nonly little difference in terms of nominal accuracy, but have much improved\nverified accuracy under perturbations and come with an efficiently computable\nformal guarantee on worst case adversaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01496v1",
            "date": "2019-09-03 23:15:19+00:00",
            "title": "Neural Linguistic Steganography",
            "authors": [
                "Zachary M. Ziegler",
                "Yuntian Deng",
                "Alexander M. Rush"
            ],
            "abstract": "Whereas traditional cryptography encrypts a secret message into an\nunintelligible form, steganography conceals that communication is taking place\nby encoding a secret message into a cover signal. Language is a particularly\npragmatic cover signal due to its benign occurrence and independence from any\none medium. Traditionally, linguistic steganography systems encode secret\nmessages in existing text via synonym substitution or word order\nrearrangements. Advances in neural language models enable previously\nimpractical generation-based techniques. We propose a steganography technique\nbased on arithmetic coding with large-scale neural language models. We find\nthat our approach can generate realistic looking cover sentences as evaluated\nby humans, while at the same time preserving security by matching the cover\nmessage distribution with the language model distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01610v1",
            "date": "2019-09-04 08:20:31+00:00",
            "title": "Answers Unite! Unsupervised Metrics for Reinforced Summarization Models",
            "authors": [
                "Thomas Scialom",
                "Sylvain Lamprier",
                "Benjamin Piwowarski",
                "Jacopo Staiano"
            ],
            "abstract": "Abstractive summarization approaches based on Reinforcement Learning (RL)\nhave recently been proposed to overcome classical likelihood maximization. RL\nenables to consider complex, possibly non-differentiable, metrics that globally\nassess the quality and relevance of the generated outputs. ROUGE, the most used\nsummarization metric, is known to suffer from bias towards lexical similarity\nas well as from suboptimal accounting for fluency and readability of the\ngenerated abstracts. We thus explore and propose alternative evaluation\nmeasures: the reported human-evaluation analysis shows that the proposed\nmetrics, based on Question Answering, favorably compares to ROUGE -- with the\nadditional property of not requiring reference summaries. Training a RL-based\nmodel on these metrics leads to improvements (both in terms of human or\nautomated metrics) over current approaches that use ROUGE as a reward.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01627v2",
            "date": "2019-09-04 08:58:53+00:00",
            "title": "On the k-synchronizability of systems",
            "authors": [
                "Cinzia Di Giusto",
                "Cinzia Giusto",
                "Laetitia Laversa",
                "Etienne Lozes"
            ],
            "abstract": "In this paper, we work on the notion of k-synchronizability: a system is\nk-synchronizable if any of its executions, up to reordering causally\nindependent actions, can be divided into a succession of k-bounded interaction\nphases. We show two results (both for mailbox and peer-to-peer automata):\nfirst, the reachability problem is decidable for k-synchronizable systems;\nsecond, the membership problem (whether a given system is k-synchronizable) is\ndecidable as well. Our proofs fix several important issues in previous attempts\nto prove these two results for mailbox automata.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01700v2",
            "date": "2019-09-04 11:35:48+00:00",
            "title": "DurIAN: Duration Informed Attention Network For Multimodal Synthesis",
            "authors": [
                "Chengzhu Yu",
                "Heng Lu",
                "Na Hu",
                "Meng Yu",
                "Chao Weng",
                "Kun Xu",
                "Peng Liu",
                "Deyi Tuo",
                "Shiyin Kang",
                "Guangzhi Lei",
                "Dan Su",
                "Dong Yu"
            ],
            "abstract": "In this paper, we present a generic and robust multimodal synthesis system\nthat produces highly natural speech and facial expression simultaneously. The\nkey component of this system is the Duration Informed Attention Network\n(DurIAN), an autoregressive model in which the alignments between the input\ntext and the output acoustic features are inferred from a duration model. This\nis different from the end-to-end attention mechanism used, and accounts for\nvarious unavoidable artifacts, in existing end-to-end speech synthesis systems\nsuch as Tacotron. Furthermore, DurIAN can be used to generate high quality\nfacial expression which can be synchronized with generated speech with/without\nparallel speech and face data. To improve the efficiency of speech generation,\nwe also propose a multi-band parallel generation strategy on top of the WaveRNN\nmodel. The proposed Multi-band WaveRNN effectively reduces the total\ncomputational complexity from 9.8 to 5.5 GFLOPS, and is able to generate audio\nthat is 6 times faster than real time on a single CPU core. We show that DurIAN\ncould generate highly natural speech that is on par with current state of the\nart end-to-end systems, while at the same time avoid word skipping/repeating\nerrors in those systems. Finally, a simple yet effective approach for\nfine-grained control of expressiveness of speech and facial expression is\nintroduced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01716v3",
            "date": "2019-09-04 12:04:48+00:00",
            "title": "ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks",
            "authors": [
                "Michihiro Yasunaga",
                "Jungo Kasai",
                "Rui Zhang",
                "Alexander R. Fabbri",
                "Irene Li",
                "Dan Friedman",
                "Dragomir R. Radev"
            ],
            "abstract": "Scientific article summarization is challenging: large, annotated corpora are\nnot available, and the summary should ideally include the article's impacts on\nresearch community. This paper provides novel solutions to these two\nchallenges. We 1) develop and release the first large-scale manually-annotated\ncorpus for scientific papers (on computational linguistics) by enabling faster\nannotation, and 2) propose summarization methods that integrate the authors'\noriginal highlights (abstract) and the article's actual impacts on the\ncommunity (citations), to create comprehensive, hybrid summaries. We conduct\nexperiments to demonstrate the efficacy of our corpus in training data-driven\nmodels for scientific paper summarization and the advantage of our hybrid\nsummaries over abstracts and traditional citation-based summaries. Our large\nannotated corpus and hybrid methods provide a new framework for scientific\npaper summarization research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01860v2",
            "date": "2019-08-27 07:03:03+00:00",
            "title": "Visual Question Answering using Deep Learning: A Survey and Performance Analysis",
            "authors": [
                "Yash Srivastava",
                "Vaishnav Murali",
                "Shiv Ram Dubey",
                "Snehasis Mukherjee"
            ],
            "abstract": "The Visual Question Answering (VQA) task combines challenges for processing\ndata with both Visual and Linguistic processing, to answer basic `common sense'\nquestions about given images. Given an image and a question in natural\nlanguage, the VQA system tries to find the correct answer to it using visual\nelements of the image and inference gathered from textual questions. In this\nsurvey, we cover and discuss the recent datasets released in the VQA domain\ndealing with various types of question-formats and robustness of the\nmachine-learning models. Next, we discuss about new deep learning models that\nhave shown promising results over the VQA datasets. At the end, we present and\ndiscuss some of the results computed by us over the vanilla VQA model, Stacked\nAttention Network and the VQA Challenge 2017 winner model. We also provide the\ndetailed analysis along with the challenges and future research directions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.02027v1",
            "date": "2019-09-04 18:04:56+00:00",
            "title": "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction",
            "authors": [
                "Stefan Larson",
                "Anish Mahendran",
                "Joseph J. Peper",
                "Christopher Clarke",
                "Andrew Lee",
                "Parker Hill",
                "Jonathan K. Kummerfeld",
                "Kevin Leach",
                "Michael A. Laurenzano",
                "Lingjia Tang",
                "Jason Mars"
            ],
            "abstract": "Task-oriented dialog systems need to know when a query falls outside their\nrange of supported intents, but current text classification corpora only define\nlabel sets that cover every example. We introduce a new dataset that includes\nqueries that are out-of-scope---i.e., queries that do not fall into any of the\nsystem's supported intents. This poses a new challenge because models cannot\nassume that every query at inference time belongs to a system-supported intent\nclass. Our dataset also covers 150 intent classes over 10 domains, capturing\nthe breadth that a production task-oriented agent must handle. We evaluate a\nrange of benchmark classifiers on our dataset along with several different\nout-of-scope identification schemes. We find that while the classifiers perform\nwell on in-scope intent classification, they struggle to identify out-of-scope\nqueries. Our dataset and evaluation fill an important gap in the field,\noffering a way of more rigorously and realistically benchmarking text\nclassification in task-driven dialog systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02059v1",
            "date": "2019-09-04 19:07:29+00:00",
            "title": "An Entity-Driven Framework for Abstractive Summarization",
            "authors": [
                "Eva Sharma",
                "Luyang Huang",
                "Zhe Hu",
                "Lu Wang"
            ],
            "abstract": "Abstractive summarization systems aim to produce more coherent and concise\nsummaries than their extractive counterparts. Popular neural models have\nachieved impressive results for single-document summarization, yet their\noutputs are often incoherent and unfaithful to the input. In this paper, we\nintroduce SENECA, a novel System for ENtity-drivEn Coherent Abstractive\nsummarization framework that leverages entity information to generate\ninformative and coherent abstracts. Our framework takes a two-step approach:\n(1) an entity-aware content selection module first identifies salient sentences\nfrom the input, then (2) an abstract generation module conducts cross-sentence\ninformation compression and abstraction to generate the final summary, which is\ntrained with rewards to promote coherence, conciseness, and clarity. The two\ncomponents are further connected using reinforcement learning. Automatic\nevaluation shows that our model significantly outperforms previous\nstate-of-the-art on ROUGE and our proposed coherence measures on New York Times\nand CNN/Daily Mail datasets. Human judges further rate our system summaries as\nmore informative and coherent than those by popular summarization models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02060v1",
            "date": "2019-09-04 19:07:33+00:00",
            "title": "Distributionally Robust Language Modeling",
            "authors": [
                "Yonatan Oren",
                "Shiori Sagawa",
                "Tatsunori B. Hashimoto",
                "Percy Liang"
            ],
            "abstract": "Language models are generally trained on data spanning a wide range of topics\n(e.g., news, reviews, fiction), but they might be applied to an a priori\nunknown target distribution (e.g., restaurant reviews). In this paper, we first\nshow that training on text outside the test distribution can degrade test\nperformance when using standard maximum likelihood (MLE) training. To remedy\nthis without the knowledge of the test distribution, we propose an approach\nwhich trains a model that performs well over a wide range of potential test\ndistributions. In particular, we derive a new distributionally robust\noptimization (DRO) procedure which minimizes the loss of the model over the\nworst-case mixture of topics with sufficient overlap with the training\ndistribution. Our approach, called topic conditional value at risk (topic\nCVaR), obtains a 5.5 point perplexity reduction over MLE when the language\nmodels are trained on a mixture of Yelp reviews and news and tested only on\nreviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02063v2",
            "date": "2019-09-04 19:10:20+00:00",
            "title": "The Diversity-Innovation Paradox in Science",
            "authors": [
                "Bas Hofstra",
                "Vivek V. Kulkarni",
                "Sebastian Munoz-Najar Galvez",
                "Bryan He",
                "Dan Jurafsky",
                "Daniel A. McFarland"
            ],
            "abstract": "Prior work finds a diversity paradox: diversity breeds innovation, and yet,\nunderrepresented groups that diversify organizations have less successful\ncareers within them. Does the diversity paradox hold for scientists as well? We\nstudy this by utilizing a near-population of ~1.2 million US doctoral\nrecipients from 1977-2015 and following their careers into publishing and\nfaculty positions. We use text analysis and machine learning to answer a series\nof questions: How do we detect scientific innovations? Are underrepresented\ngroups more likely to generate scientific innovations? And are the innovations\nof underrepresented groups adopted and rewarded? Our analyses show that\nunderrepresented groups produce higher rates of scientific novelty. However,\ntheir novel contributions are devalued and discounted: e.g., novel\ncontributions by gender and racial minorities are taken up by other scholars at\nlower rates than novel contributions by gender and racial majorities, and\nequally impactful contributions of gender and racial minorities are less likely\nto result in successful scientific careers than for majority groups. These\nresults suggest there may be unwarranted reproduction of stratification in\nacademic careers that discounts diversity's role in innovation and partly\nexplains the underrepresentation of some groups in academia.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1909.02218v1",
            "date": "2019-09-05 05:48:51+00:00",
            "title": "A Better Way to Attend: Attention with Trees for Video Question Answering",
            "authors": [
                "Hongyang Xue",
                "Wenqing Chu",
                "Zhou Zhao",
                "Deng Cai"
            ],
            "abstract": "We propose a new attention model for video question answering. The main idea\nof the attention models is to locate on the most informative parts of the\nvisual data. The attention mechanisms are quite popular these days. However,\nmost existing visual attention mechanisms regard the question as a whole. They\nignore the word-level semantics where each word can have different attentions\nand some words need no attention. Neither do they consider the semantic\nstructure of the sentences. Although the Extended Soft Attention (E-SA) model\nfor video question answering leverages the word-level attention, it performs\npoorly on long question sentences. In this paper, we propose the heterogeneous\ntree-structured memory network (HTreeMN) for video question answering. Our\nproposed approach is based upon the syntax parse trees of the question\nsentences. The HTreeMN treats the words differently where the \\textit{visual}\nwords are processed with an attention module and the \\textit{verbal} ones not.\nIt also utilizes the semantic structure of the sentences by combining the\nneighbors based on the recursive structure of the parse trees. The\nunderstandings of the words and the videos are propagated and merged from\nleaves to the root. Furthermore, we build a hierarchical attention mechanism to\ndistill the attended features. We evaluate our approach on two datasets. The\nexperimental results show the superiority of our HTreeMN model over the other\nattention models especially on complex questions. Our code is available on\ngithub.\n  Our code is available at https://github.com/ZJULearning/TreeAttention",
            "journal ref": "IEEE Transactions on Image Processing ( Volume: 27 , Issue: 11 ,\n  Nov. 2018 )",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.02244v1",
            "date": "2019-09-05 07:31:58+00:00",
            "title": "Robust Navigation with Language Pretraining and Stochastic Sampling",
            "authors": [
                "Xiujun Li",
                "Chunyuan Li",
                "Qiaolin Xia",
                "Yonatan Bisk",
                "Asli Celikyilmaz",
                "Jianfeng Gao",
                "Noah Smith",
                "Yejin Choi"
            ],
            "abstract": "Core to the vision-and-language navigation (VLN) challenge is building robust\ninstruction representations and action decoding schemes, which can generalize\nwell to previously unseen instructions and environments. In this paper, we\nreport two simple but highly effective methods to address these challenges and\nlead to a new state-of-the-art performance. First, we adapt large-scale\npretrained language models to learn text representations that generalize better\nto previously unseen instructions. Second, we propose a stochastic sampling\nscheme to reduce the considerable gap between the expert actions in training\nand sampled actions in test, so that the agent can learn to correct its own\nmistakes during long sequential action decoding. Combining the two techniques,\nwe achieve a new state of the art on the Room-to-Room benchmark with 6%\nabsolute gain over the previous best result (47% -> 53%) on the Success Rate\nweighted by Path Length metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02307v1",
            "date": "2019-09-05 10:34:07+00:00",
            "title": "Fusing Vector Space Models for Domain-Specific Applications",
            "authors": [
                "Laura Rettig",
                "Julien Audiffren",
                "Philippe Cudr\u00e9-Mauroux"
            ],
            "abstract": "We address the problem of tuning word embeddings for specific use cases and\ndomains. We propose a new method that automatically combines multiple\ndomain-specific embeddings, selected from a wide range of pre-trained\ndomain-specific embeddings, to improve their combined expressive power. Our\napproach relies on two key components: 1) a ranking function, based on a new\nembedding similarity measure, that selects the most relevant embeddings to use\ngiven a domain and 2) a dimensionality reduction method that combines the\nselected embeddings to produce a more compact and efficient encoding that\npreserves the expressiveness. We empirically show that our method produces\neffective domain-specific embeddings that consistently improve the performance\nof state-of-the-art machine learning algorithms on multiple tasks, compared to\ngeneric embeddings trained on large text corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02667v1",
            "date": "2019-09-05 23:07:26+00:00",
            "title": "Bandwidth Embeddings for Mixed-bandwidth Speech Recognition",
            "authors": [
                "Gautam Mantena",
                "Ozlem Kalinli",
                "Ossama Abdel-Hamid",
                "Don McAllaster"
            ],
            "abstract": "In this paper, we tackle the problem of handling narrowband and wideband\nspeech by building a single acoustic model (AM), also called mixed bandwidth\nAM. In the proposed approach, an auxiliary input feature is used to provide the\nbandwidth information to the model, and bandwidth embeddings are jointly\nlearned as part of acoustic model training. Experimental evaluations show that\nusing bandwidth embeddings helps the model to handle the variability of the\nnarrow and wideband speech, and makes it possible to train a mixed-bandwidth\nAM. Furthermore, we propose to use parallel convolutional layers to handle the\nmismatch between the narrow and wideband speech better, where separate\nconvolution layers are used for each type of input speech signal. Our best\nsystem achieves 13% relative improvement on narrowband speech, while not\ndegrading on wideband speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.02764v2",
            "date": "2019-09-06 08:33:00+00:00",
            "title": "Towards Multimodal Emotion Recognition in German Speech Events in Cars using Transfer Learning",
            "authors": [
                "Deniz Cevher",
                "Sebastian Zepf",
                "Roman Klinger"
            ],
            "abstract": "The recognition of emotions by humans is a complex process which considers\nmultiple interacting signals such as facial expressions and both prosody and\nsemantic content of utterances. Commonly, research on automatic recognition of\nemotions is, with few exceptions, limited to one modality. We describe an\nin-car experiment for emotion recognition from speech interactions for three\nmodalities: the audio signal of a spoken interaction, the visual signal of the\ndriver's face, and the manually transcribed content of utterances of the\ndriver. We use off-the-shelf tools for emotion detection in audio and face and\ncompare that to a neural transfer learning approach for emotion recognition\nfrom text which utilizes existing resources from other domains. We see that\ntransfer learning enables models based on out-of-domain corpora to perform\nwell. This method contributes up to 10 percentage points in F1, with up to 76\nmicro-average F1 across the emotions joy, annoyance and insecurity. Our\nfindings also indicate that off-the-shelf-tools analyzing face and audio are\nnot ready yet for emotion detection in in-car speech interactions without\nfurther adjustments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.02930v1",
            "date": "2019-09-06 14:29:00+00:00",
            "title": "Structured Query Construction via Knowledge Graph Embedding",
            "authors": [
                "Ruijie Wang",
                "Meng Wang",
                "Jun Liu",
                "Michael Cochez",
                "Stefan Decker"
            ],
            "abstract": "In order to facilitate the accesses of general users to knowledge graphs, an\nincreasing effort is being exerted to construct graph-structured queries of\ngiven natural language questions. At the core of the construction is to deduce\nthe structure of the target query and determine the vertices/edges which\nconstitute the query. Existing query construction methods rely on question\nunderstanding and conventional graph-based algorithms which lead to inefficient\nand degraded performances facing complex natural language questions over\nknowledge graphs with large scales. In this paper, we focus on this problem and\npropose a novel framework standing on recent knowledge graph embedding\ntechniques. Our framework first encodes the underlying knowledge graph into a\nlow-dimensional embedding space by leveraging generalized local knowledge\ngraphs. Given a natural language question, the learned embedding\nrepresentations of the knowledge graph are utilized to compute the query\nstructure and assemble vertices/edges into the target query. Extensive\nexperiments were conducted on the benchmark dataset, and the results\ndemonstrate that our framework outperforms state-of-the-art baseline models\nregarding effectiveness and efficiency.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.02950v2",
            "date": "2019-09-06 14:59:18+00:00",
            "title": "Supervised Multimodal Bitransformers for Classifying Images and Text",
            "authors": [
                "Douwe Kiela",
                "Suvrat Bhooshan",
                "Hamed Firooz",
                "Ethan Perez",
                "Davide Testuggine"
            ],
            "abstract": "Self-supervised bidirectional transformer models such as BERT have led to\ndramatic improvements in a wide variety of textual classification tasks. The\nmodern digital world is increasingly multimodal, however, and textual\ninformation is often accompanied by other modalities such as images. We\nintroduce a supervised multimodal bitransformer model that fuses information\nfrom text and image encoders, and obtain state-of-the-art performance on\nvarious multimodal classification benchmark tasks, outperforming strong\nbaselines, including on hard test sets specifically designed to measure\nmultimodal performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03004v1",
            "date": "2019-09-06 16:40:42+00:00",
            "title": "Show Your Work: Improved Reporting of Experimental Results",
            "authors": [
                "Jesse Dodge",
                "Suchin Gururangan",
                "Dallas Card",
                "Roy Schwartz",
                "Noah A. Smith"
            ],
            "abstract": "Research in natural language processing proceeds, in part, by demonstrating\nthat new models achieve superior performance (e.g., accuracy) on held-out test\ndata, compared to previous results. In this paper, we demonstrate that test-set\nperformance scores alone are insufficient for drawing accurate conclusions\nabout which model performs best. We argue for reporting additional details,\nespecially performance on validation data obtained during model development. We\npresent a novel technique for doing so: expected validation performance of the\nbest-found model as a function of computation budget (i.e., the number of\nhyperparameter search trials or the overall training time). Using our approach,\nwe find multiple recent model comparisons where authors would have reached a\ndifferent conclusion if they had used more (or less) computation. Our approach\nalso allows us to estimate the amount of computation required to obtain a given\naccuracy; applying it to several recently published results yields massive\nvariation across papers, from hours to weeks. We conclude with a set of best\npractices for reporting experimental results which allow for robust future\ncomparisons, and provide code to allow researchers to use our technique.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.03011v1",
            "date": "2019-09-06 16:51:21+00:00",
            "title": "RNN Architecture Learning with Sparse Regularization",
            "authors": [
                "Jesse Dodge",
                "Roy Schwartz",
                "Hao Peng",
                "Noah A. Smith"
            ],
            "abstract": "Neural models for NLP typically use large numbers of parameters to reach\nstate-of-the-art performance, which can lead to excessive memory usage and\nincreased runtime. We present a structure learning method for learning sparse,\nparameter-efficient NLP models. Our method applies group lasso to rational RNNs\n(Peng et al., 2018), a family of models that is closely connected to weighted\nfinite-state automata (WFSAs). We take advantage of rational RNNs' natural\ngrouping of the weights, so the group lasso penalty directly removes WFSA\nstates, substantially reducing the number of parameters in the model. Our\nexperiments on a number of sentiment analysis datasets, using both GloVe and\nBERT embeddings, show that our approach learns neural structures which have\nfewer parameters without sacrificing performance relative to parameter-rich\nbaselines. Our method also highlights the interpretable properties of rational\nRNNs. We show that sparsifying such models makes them easier to visualize, and\nwe present models that rely exclusively on as few as three WFSAs after pruning\nmore than 90% of the weights. We publicly release our code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03039v3",
            "date": "2019-09-06 17:49:56+00:00",
            "title": "Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes",
            "authors": [
                "Jonas Kemp",
                "Alvin Rajkomar",
                "Andrew M. Dai"
            ],
            "abstract": "Clinical notes in electronic health records contain highly heterogeneous\nwriting styles, including non-standard terminology or abbreviations. Using\nthese notes in predictive modeling has traditionally required preprocessing\n(e.g. taking frequent terms or topic modeling) that removes much of the\nrichness of the source data. We propose a pretrained hierarchical recurrent\nneural network model that parses minimally processed clinical notes in an\nintuitive fashion, and show that it improves performance for discharge\ndiagnosis classification tasks on the Medical Information Mart for Intensive\nCare III (MIMIC-III) dataset, compared to models that treat the notes as an\nunordered collection of terms or that conduct no pretraining. We also apply an\nattribution technique to examples to identify the words that the model uses to\nmake its prediction, and show the importance of the words' nearby context.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.03044v1",
            "date": "2019-09-06 17:56:01+00:00",
            "title": "Deep learning with sentence embeddings pre-trained on biomedical corpora improves the performance of finding similar sentences in electronic medical records",
            "authors": [
                "Qingyu Chen",
                "Jingcheng Du",
                "Sun Kim",
                "W. John Wilbur",
                "Zhiyong Lu"
            ],
            "abstract": "Capturing sentence semantics plays a vital role in a range of text mining\napplications. Despite continuous efforts on the development of related datasets\nand models in the general domain, both datasets and models are limited in\nbiomedical and clinical domains. The BioCreative/OHNLP organizers have made the\nfirst attempt to annotate 1,068 sentence pairs from clinical notes and have\ncalled for a community effort to tackle the Semantic Textual Similarity\n(BioCreative/OHNLP STS) challenge. We developed models using traditional\nmachine learning and deep learning approaches. For the post challenge, we focus\non two models: the Random Forest and the Encoder Network. We applied sentence\nembeddings pre-trained on PubMed abstracts and MIMIC-III clinical notes and\nupdated the Random Forest and the Encoder Network accordingly. The official\nresults demonstrated our best submission was the ensemble of eight models. It\nachieved a Person correlation coefficient of 0.8328, the highest performance\namong 13 submissions from 4 teams. For the post challenge, the performance of\nboth Random Forest and the Encoder Network was improved; in particular, the\ncorrelation of the Encoder Network was improved by ~13%. During the challenge\ntask, no end-to-end deep learning models had better performance than machine\nlearning models that take manually-crafted features. In contrast, with the\nsentence embeddings pre-trained on biomedical corpora, the Encoder Network now\nachieves a correlation of ~0.84, which is higher than the original best model.\nThe ensembled model taking the improved versions of the Random Forest and\nEncoder Network as inputs further increased performance to 0.8528. Deep\nlearning models with sentence embeddings pre-trained on biomedical corpora\nachieve the highest performance on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03242v2",
            "date": "2019-09-07 10:57:29+00:00",
            "title": "MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims",
            "authors": [
                "Isabelle Augenstein",
                "Christina Lioma",
                "Dongsheng Wang",
                "Lucas Chaves Lima",
                "Casper Hansen",
                "Christian Hansen",
                "Jakob Grue Simonsen"
            ],
            "abstract": "We contribute the largest publicly available dataset of naturally occurring\nfactual claims for the purpose of automatic claim verification. It is collected\nfrom 26 fact checking websites in English, paired with textual sources and rich\nmetadata, and labelled for veracity by human expert journalists. We present an\nin-depth analysis of the dataset, highlighting characteristics and challenges.\nFurther, we present results for automatic veracity prediction, both with\nestablished baselines and with a novel method for joint ranking of evidence\npages and predicting veracity that outperforms all baselines. Significant\nperformance increases are achieved by encoding evidence, and by modelling\nmetadata. Our best-performing model achieves a Macro F1 of 49.2%, showing that\nthis is a challenging testbed for claim veracity prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03315v1",
            "date": "2019-09-07 18:24:57+00:00",
            "title": "Relationships from Entity Stream",
            "authors": [
                "Martin Andrews",
                "Sam Witteveen"
            ],
            "abstract": "Relational reasoning is a central component of intelligent behavior, but has\nproven difficult for neural networks to learn. The Relation Network (RN) module\nwas recently proposed by DeepMind to solve such problems, and demonstrated\nstate-of-the-art results on a number of datasets. However, the RN module scales\nquadratically in the size of the input, since it calculates relationship\nfactors between every patch in the visual field, including those that do not\ncorrespond to entities. In this paper, we describe an architecture that enables\nrelationships to be determined from a stream of entities obtained by an\nattention mechanism over the input field. The model is trained end-to-end, and\ndemonstrates equivalent performance with greater interpretability while\nrequiring only a fraction of the model parameters of the original RN module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03317v1",
            "date": "2019-09-07 18:32:28+00:00",
            "title": "Dependency Parsing for Spoken Dialog Systems",
            "authors": [
                "Sam Davidson",
                "Dian Yu",
                "Zhou Yu"
            ],
            "abstract": "Dependency parsing of conversational input can play an important role in\nlanguage understanding for dialog systems by identifying the relationships\nbetween entities extracted from user utterances. Additionally, effective\ndependency parsing can elucidate differences in language structure and usage\nfor discourse analysis of human-human versus human-machine dialogs. However,\nmodels trained on datasets based on news articles and web data do not perform\nwell on spoken human-machine dialog, and currently available annotation schemes\ndo not adapt well to dialog data. Therefore, we propose the Spoken Conversation\nUniversal Dependencies (SCUD) annotation scheme that extends the Universal\nDependencies (UD) (Nivre et al., 2016) guidelines to spoken human-machine\ndialogs. We also provide ConvBank, a conversation dataset between humans and an\nopen-domain conversational dialog system with SCUD annotation. Finally, to\ndemonstrate the utility of the dataset, we train a dependency parser on the\nConvBank dataset. We demonstrate that by pre-training a dependency parser on a\nset of larger public datasets and fine-tuning on ConvBank data, we achieved the\nbest result, 85.05% unlabeled and 77.82% labeled attachment accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03359v2",
            "date": "2019-09-08 01:06:03+00:00",
            "title": "Distributed Training of Embeddings using Graph Analytics",
            "authors": [
                "Gurbinder Gill",
                "Roshan Dathathri",
                "Saeed Maleki",
                "Madan Musuvathi",
                "Todd Mytkowicz",
                "Olli Saarikivi"
            ],
            "abstract": "Many applications today, such as NLP, network analysis, and code analysis,\nrely on semantically embedding objects into low-dimensional fixed-length\nvectors. Such embeddings naturally provide a way to perform useful downstream\ntasks, such as identifying relations among objects or predicting objects for a\ngiven context, etc. Unfortunately, the training necessary for accurate\nembeddings is usually computationally intensive and requires processing large\namounts of data. Furthermore, distributing this training is challenging. Most\nembedding training uses stochastic gradient descent (SGD), an \"inherently\"\nsequential algorithm. Prior approaches to parallelizing SGD do not honor these\ndependencies and thus potentially suffer poor convergence.\n  This paper presents a distributed training framework for a class of\napplications that use Skip-gram-like models to generate embeddings. We call\nthis class Any2Vec and it includes Word2Vec, DeepWalk, and Node2Vec among\nothers. We first formulate Any2Vec training algorithm as a graph application\nand leverage the state-of-the-art distributed graph analytics framework,\nD-Galois. We adapt D-Galois to support dynamic graph generation and\nrepartitioning, and incorporate novel communication optimizations. Finally, we\nintroduce a novel way to combine gradients during distributed training to\nprevent accuracy loss. We show that our framework, called GraphAny2Vec, matches\non a cluster of 32 hosts the accuracy of the state-of-the-art shared-memory\nimplementations of Word2Vec and Vertex2Vec on 1 host, and gives a geo-mean\nspeedup of 12x and 5x respectively. Furthermore, GraphAny2Vec is on average 2x\nfaster than the state-of-the-art distributed Word2Vec implementation, DMTK, on\n32 hosts. We also show the superiority of our Gradient Combiner independent of\nGraphAny2Vec by incorporating it in DMTK, which raises its accuracy by > 30%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.03409v2",
            "date": "2019-09-08 09:31:20+00:00",
            "title": "Conditional Text Generation for Harmonious Human-Machine Interaction",
            "authors": [
                "Bin Guo",
                "Hao Wang",
                "Yasan Ding",
                "Wei Wu",
                "Shaoyang Hao",
                "Yueqi Sun",
                "Zhiwen Yu"
            ],
            "abstract": "In recent years, with the development of deep learning, text generation\ntechnology has undergone great changes and provided many kinds of services for\nhuman beings, such as restaurant reservation and daily communication. The\nautomatically generated text is becoming more and more fluent so researchers\nbegin to consider more anthropomorphic text generation technology, that is the\nconditional text generation, including emotional text generation, personalized\ntext generation, and so on. Conditional Text Generation (CTG) has thus become a\nresearch hotspot. As a promising research field, we find that many efforts have\nbeen paid to exploring it. Therefore, we aim to give a comprehensive review of\nthe new research trends of CTG. We first summary several key techniques and\nillustrate the technical evolution route in the field of neural text\ngeneration, based on the concept model of CTG. We further make an investigation\nof existing CTG fields and propose several general learning models for CTG.\nFinally, we discuss the open issues and promising research directions of CTG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03480v2",
            "date": "2019-09-08 15:09:32+00:00",
            "title": "Story Realization: Expanding Plot Events into Sentences",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Ethan Tien",
                "Wesley Cheung",
                "Zhaochen Luo",
                "William Ma",
                "Lara J. Martin",
                "Mark O. Riedl"
            ],
            "abstract": "Neural network based approaches to automated story plot generation attempt to\nlearn how to generate novel plots from a corpus of natural language plot\nsummaries. Prior work has shown that a semantic abstraction of sentences called\nevents improves neural plot generation and and allows one to decompose the\nproblem into: (1) the generation of a sequence of events (event-to-event) and\n(2) the transformation of these events into natural language sentences\n(event-to-sentence). However, typical neural language generation approaches to\nevent-to-sentence can ignore the event details and produce\ngrammatically-correct but semantically-unrelated sentences. We present an\nensemble-based model that generates natural language guided by events.We\nprovide results---including a human subjects study---for a full end-to-end\nautomated story generation system showing that our method generates more\ncoherent and plausible stories than baseline approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03508v1",
            "date": "2019-09-08 16:57:26+00:00",
            "title": "Transformer to CNN: Label-scarce distillation for efficient text classification",
            "authors": [
                "Yew Ken Chia",
                "Sam Witteveen",
                "Martin Andrews"
            ],
            "abstract": "Significant advances have been made in Natural Language Processing (NLP)\nmodelling since the beginning of 2018. The new approaches allow for accurate\nresults, even when there is little labelled data, because these NLP models can\nbenefit from training on both task-agnostic and task-specific unlabelled data.\nHowever, these advantages come with significant size and computational costs.\nThis workshop paper outlines how our proposed convolutional student\narchitecture, having been trained by a distillation process from a large-scale\nmodel, can achieve 300x inference speedup and 39x reduction in parameter count.\nIn some cases, the student model performance surpasses its teacher on the\nstudied tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.03527v2",
            "date": "2019-09-08 18:35:03+00:00",
            "title": "Large Scale Question Answering using Tourism Data",
            "authors": [
                "Danish Contractor",
                "Krunal Shah",
                "Aditi Partap",
                "Mausam",
                "Parag Singla"
            ],
            "abstract": "We introduce the novel task of answering entity-seeking recommendation\nquestions using a collection of reviews that describe candidate answer\nentities. We harvest a QA dataset that contains 47,124 paragraph-sized real\nuser questions from travelers seeking recommendations for hotels, attractions\nand restaurants. Each question can have thousands of candidate answers to\nchoose from and each candidate is associated with a collection of unstructured\nreviews. This dataset is especially challenging because commonly used neural\narchitectures for reasoning and QA are prohibitively expensive for a task of\nthis scale. As a solution, we design a scalable cluster-select-rerank approach.\nIt first clusters text for each entity to identify exemplar sentences\ndescribing an entity. It then uses a scalable neural information retrieval (IR)\nmodule to select a set of potential entities from the large candidate set. A\nreranker uses a deeper attention-based architecture to pick the best answers\nfrom the selected entities. This strategy performs better than a pure IR or a\npure attention-based reasoning approach yielding nearly 25% relative\nimprovement in Accuracy@3 over both approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03564v2",
            "date": "2019-09-08 23:35:00+00:00",
            "title": "Transfer Learning Robustness in Multi-Class Categorization by Fine-Tuning Pre-Trained Contextualized Language Models",
            "authors": [
                "Xinyi Liu",
                "Artit Wangperawong"
            ],
            "abstract": "This study compares the effectiveness and robustness of multi-class\ncategorization of Amazon product data using transfer learning on pre-trained\ncontextualized language models. Specifically, we fine-tuned BERT and XLNet, two\nbidirectional models that have achieved state-of-the-art performance on many\nnatural language tasks and benchmarks, including text classification. While\nexisting classification studies and benchmarks focus on binary targets, with\nthe exception of ordinal ranking tasks, here we examine the robustness of such\nmodels as the number of classes grows from 1 to 20. Our experiments demonstrate\nan approximately linear decrease in performance metrics (i.e., precision,\nrecall, $F_1$ score, and accuracy) with the number of class labels. BERT\nconsistently outperforms XLNet using identical hyperparameters on the entire\nrange of class label quantities for categorizing products based on their\ntextual descriptions. BERT is also more affordable than XLNet in terms of the\ncomputational cost (i.e., time and memory) required for training. In all cases\nstudied, the performance degradation rates were estimated to be 1% per\nadditional class label.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1909.03569v1",
            "date": "2019-09-09 00:10:58+00:00",
            "title": "Neural Gaussian Copula for Variational Autoencoder",
            "authors": [
                "Prince Zizhuang Wang",
                "William Yang Wang"
            ],
            "abstract": "Variational language models seek to estimate the posterior of latent\nvariables with an approximated variational posterior. The model often assumes\nthe variational posterior to be factorized even when the true posterior is not.\nThe learned variational posterior under this assumption does not capture the\ndependency relationships over latent variables. We argue that this would cause\na typical training problem called posterior collapse observed in all other\nvariational language models. We propose Gaussian Copula Variational Autoencoder\n(VAE) to avert this problem. Copula is widely used to model correlation and\ndependencies of high-dimensional random variables, and therefore it is helpful\nto maintain the dependency relationships that are lost in VAE. The empirical\nresults show that by modeling the correlation of latent variables explicitly\nusing a neural parametric copula, we can avert this training difficulty while\ngetting competitive results among all other VAE approaches.",
            "journal ref": "EMNLP 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03622v1",
            "date": "2019-09-09 03:36:42+00:00",
            "title": "Transfer Reward Learning for Policy Gradient-Based Text Generation",
            "authors": [
                "James O' Neill",
                "Danushka Bollegala"
            ],
            "abstract": "Task-specific scores are often used to optimize for and evaluate the\nperformance of conditional text generation systems. However, such scores are\nnon-differentiable and cannot be used in the standard supervised learning\nparadigm. Hence, policy gradient methods are used since the gradient can be\ncomputed without requiring a differentiable objective.\n  However, we argue that current n-gram overlap based measures that are used as\nrewards can be improved by using model-based rewards transferred from tasks\nthat directly compare the similarity of sentence pairs. These reward models\neither output a score of sentence-level syntactic and semantic similarity\nbetween entire predicted and target sentences as the expected return, or for\nintermediate phrases as segmented accumulative rewards.\n  We demonstrate that using a \\textit{Transferable Reward Learner} leads to\nimproved results on semantical evaluation measures in policy-gradient models\nfor image captioning tasks. Our InferSent actor-critic model improves over a\nBLEU trained actor-critic model on MSCOCO when evaluated on a Word Mover's\nDistance similarity measure by 6.97 points, also improving on a Sliding Window\nCosine Similarity measure by 10.48 points. Similar performance improvements are\nalso obtained on the smaller Flickr-30k dataset, demonstrating the general\napplicability of the proposed transfer learning method.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.03683v1",
            "date": "2019-09-09 07:44:24+00:00",
            "title": "Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases",
            "authors": [
                "Christopher Clark",
                "Mark Yatskar",
                "Luke Zettlemoyer"
            ],
            "abstract": "State-of-the-art models often make use of superficial patterns in the data\nthat do not generalize well to out-of-domain or adversarial settings. For\nexample, textual entailment models often learn that particular key words imply\nentailment, irrespective of context, and visual question answering models learn\nto predict prototypical answers, without considering evidence in the image. In\nthis paper, we show that if we have prior knowledge of such biases, we can\ntrain a model to be more robust to domain shift. Our method has two stages: we\n(1) train a naive model that makes predictions exclusively based on dataset\nbiases, and (2) train a robust model as part of an ensemble with the naive one\nin order to encourage it to focus on other patterns in the data that are more\nlikely to generalize. Experiments on five datasets with out-of-domain test sets\nshow significantly improved robustness in all settings, including a 12 point\ngain on a changing priors visual question answering dataset and a 9 point gain\non an adversarial question answering test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03792v2",
            "date": "2019-08-30 13:36:21+00:00",
            "title": "Tehran Stock Exchange Prediction Using Sentiment Analysis of Online Textual Opinions",
            "authors": [
                "Arezoo Hatefi Ghahfarrokhi",
                "Mehrnoush Shamsfard"
            ],
            "abstract": "In this paper, we investigate the impact of the social media data in\npredicting the Tehran Stock Exchange (TSE) variables for the first time. We\nconsider the closing price and daily return of three different stocks for this\ninvestigation. We collected our social media data from Sahamyab.com/stocktwits\nfor about three months. To extract information from online comments, we propose\na hybrid sentiment analysis approach that combines lexicon-based and\nlearning-based methods. Since lexicons that are available for the Persian\nlanguage are not practical for sentiment analysis in the stock market domain,\nwe built a particular sentiment lexicon for this domain. After designing and\ncalculating daily sentiment indices using the sentiment of the comments, we\nexamine their impact on the baseline models that only use historical market\ndata and propose new predictor models using multi regression analysis. In\naddition to the sentiments, we also examine the comments volume and the users'\nreliabilities. We conclude that the predictability of various stocks in TSE is\ndifferent depending on their attributes. Moreover, we indicate that for\npredicting the closing price only comments volume and for predicting the daily\nreturn both the volume and the sentiment of the comments could be useful. We\ndemonstrate that Users' Trust coefficients have different behaviors toward the\nthree stocks.",
            "journal ref": "None",
            "category": "q-fin.ST"
        },
        {
            "url": "http://arxiv.org/abs/1909.03794v1",
            "date": "2019-09-09 12:22:28+00:00",
            "title": "Composing Knowledge Graph Embeddings via Word Embeddings",
            "authors": [
                "Lianbo Ma",
                "Peng Sun",
                "Zhiwei Lin",
                "Hui Wang"
            ],
            "abstract": "Learning knowledge graph embedding from an existing knowledge graph is very\nimportant to knowledge graph completion. For a fact $(h,r,t)$ with the head\nentity $h$ having a relation $r$ with the tail entity $t$, the current\napproaches aim to learn low dimensional representations\n$(\\mathbf{h},\\mathbf{r},\\mathbf{t})$, each of which corresponds to the elements\nin $(h, r, t)$, respectively. As $(\\mathbf{h},\\mathbf{r},\\mathbf{t})$ is\nlearned from the existing facts within a knowledge graph, these representations\ncan not be used to detect unknown facts (if the entities or relations never\noccur in the knowledge graph).\n  This paper proposes a new approach called TransW, aiming to go beyond the\ncurrent work by composing knowledge graph embeddings using word embeddings.\nGiven the fact that an entity or a relation contains one or more words (quite\noften), it is sensible to learn a mapping function from word embedding spaces\nto knowledge embedding spaces, which shows how entities are constructed using\nhuman words. More importantly, composing knowledge embeddings using word\nembeddings makes it possible to deal with the emerging new facts (either new\nentities or relations). Experimental results using three public datasets show\nthe consistency and outperformance of the proposed TransW.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.03862v3",
            "date": "2019-09-09 13:49:10+00:00",
            "title": "Out-of-domain Detection for Natural Language Understanding in Dialog Systems",
            "authors": [
                "Yinhe Zheng",
                "Guanyi Chen",
                "Minlie Huang"
            ],
            "abstract": "Natural Language Understanding (NLU) is a vital component of dialogue\nsystems, and its ability to detect Out-of-Domain (OOD) inputs is critical in\npractical applications, since the acceptance of the OOD input that is\nunsupported by the current system may lead to catastrophic failure. However,\nmost existing OOD detection methods rely heavily on manually labeled OOD\nsamples and cannot take full advantage of unlabeled data. This limits the\nfeasibility of these models in practical applications.\n  In this paper, we propose a novel model to generate high-quality pseudo OOD\nsamples that are akin to IN-Domain (IND) input utterances, and thereby improves\nthe performance of OOD detection. To this end, an autoencoder is trained to map\nan input utterance into a latent code. and the codes of IND and OOD samples are\ntrained to be indistinguishable by utilizing a generative adversarial network.\nTo provide more supervision signals, an auxiliary classifier is introduced to\nregularize the generated OOD samples to have indistinguishable intent labels.\nExperiments show that these pseudo OOD samples generated by our model can be\nused to effectively improve OOD detection in NLU. Besides, we also demonstrate\nthat the effectiveness of these pseudo OOD data can be further improved by\nefficiently utilizing unlabeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.03965v1",
            "date": "2019-09-09 16:13:20+00:00",
            "title": "Evaluating Long-form Text-to-Speech: Comparing the Ratings of Sentences and Paragraphs",
            "authors": [
                "Rob Clark",
                "Hanna Silen",
                "Tom Kenter",
                "Ralph Leith"
            ],
            "abstract": "Text-to-speech systems are typically evaluated on single sentences. When\nlong-form content, such as data consisting of full paragraphs or dialogues is\nconsidered, evaluating sentences in isolation is not always appropriate as the\ncontext in which the sentences are synthesized is missing. In this paper, we\ninvestigate three different ways of evaluating the naturalness of long-form\ntext-to-speech synthesis. We compare the results obtained from evaluating\nsentences in isolation, evaluating whole paragraphs of speech, and presenting a\nselection of speech or text as context and evaluating the subsequent speech. We\nfind that, even though these three evaluations are based upon the same\nmaterial, the outcomes differ per setting, and moreover that these outcomes do\nnot necessarily correlate with each other. We show that our findings are\nconsistent between a single speaker setting of read paragraphs and a\ntwo-speaker dialogue scenario. We conclude that to evaluate the quality of\nlong-form speech, the traditional way of evaluating sentences in isolation does\nnot suffice, and that multiple evaluations are required.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.04120v2",
            "date": "2019-09-09 19:32:31+00:00",
            "title": "Span Selection Pre-training for Question Answering",
            "authors": [
                "Michael Glass",
                "Alfio Gliozzo",
                "Rishav Chakravarti",
                "Anthony Ferritto",
                "Lin Pan",
                "G P Shrivatsa Bhargav",
                "Dinesh Garg",
                "Avirup Sil"
            ],
            "abstract": "BERT (Bidirectional Encoder Representations from Transformers) and related\npre-trained Transformers have provided large gains across many language\nunderstanding tasks, achieving a new state-of-the-art (SOTA). BERT is\npre-trained on two auxiliary tasks: Masked Language Model and Next Sentence\nPrediction. In this paper we introduce a new pre-training task inspired by\nreading comprehension to better align the pre-training from memorization to\nunderstanding. Span Selection Pre-Training (SSPT) poses cloze-like training\ninstances, but rather than draw the answer from the model's parameters, it is\nselected from a relevant passage. We find significant and consistent\nimprovements over both BERT-BASE and BERT-LARGE on multiple reading\ncomprehension (MRC) datasets. Specifically, our proposed model has strong\nempirical evidence as it obtains SOTA results on Natural Questions, a new\nbenchmark MRC dataset, outperforming BERT-LARGE by 3 F1 points on short answer\nprediction. We also show significant impact in HotpotQA, improving answer\nprediction F1 by 4 points and supporting fact prediction F1 by 1 point and\noutperforming the previous best system. Moreover, we show that our pre-training\napproach is particularly effective when training data is limited, improving the\nlearning curve by a large amount.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04189v2",
            "date": "2019-09-09 22:43:02+00:00",
            "title": "Follow the Leader: Documents on the Leading Edge of Semantic Change Get More Citations",
            "authors": [
                "Sandeep Soni",
                "Kristina Lerman",
                "Jacob Eisenstein"
            ],
            "abstract": "Diachronic word embeddings -- vector representations of words over time --\noffer remarkable insights into the evolution of language and provide a tool for\nquantifying sociocultural change from text documents. Prior work has used such\nembeddings to identify shifts in the meaning of individual words. However,\nsimply knowing that a word has changed in meaning is insufficient to identify\nthe instances of word usage that convey the historical or the newer meaning. In\nthis paper, we link diachronic word embeddings to documents, by situating those\ndocuments as leaders or laggards with respect to ongoing semantic changes.\nSpecifically, we propose a novel method to quantify the degree of semantic\nprogressiveness in each word usage, and then show how these usages can be\naggregated to obtain scores for each document. We analyze two large collections\nof documents, representing legal opinions and scientific articles. Documents\nthat are scored as semantically progressive receive a larger number of\ncitations, indicating that they are especially influential. Our work thus\nprovides a new technique for identifying lexical semantic leaders and\ndemonstrates a new link between progressive use of language and influence in a\ncitation network.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04251v1",
            "date": "2019-09-10 03:00:58+00:00",
            "title": "A Benchmark Dataset for Learning to Intervene in Online Hate Speech",
            "authors": [
                "Jing Qian",
                "Anna Bethke",
                "Yinyin Liu",
                "Elizabeth Belding",
                "William Yang Wang"
            ],
            "abstract": "Countering online hate speech is a critical yet challenging task, but one\nwhich can be aided by the use of Natural Language Processing (NLP) techniques.\nPrevious research has primarily focused on the development of NLP methods to\nautomatically and effectively detect online hate speech while disregarding\nfurther action needed to calm and discourage individuals from using hate speech\nin the future. In addition, most existing hate speech datasets treat each post\nas an isolated instance, ignoring the conversational context. In this paper, we\npropose a novel task of generative hate speech intervention, where the goal is\nto automatically generate responses to intervene during online conversations\nthat contain hate speech. As a part of this work, we introduce two\nfully-labeled large-scale hate speech intervention datasets collected from Gab\nand Reddit. These datasets provide conversation segments, hate speech labels,\nas well as intervention responses written by Mechanical Turk Workers. In this\npaper, we also analyze the datasets to understand the common intervention\nstrategies and explore the performance of common automatic response generation\nmethods on these new datasets to provide a benchmark for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04402v2",
            "date": "2019-09-10 10:55:56+00:00",
            "title": "Compositional Generalization in Image Captioning",
            "authors": [
                "Mitja Nikolaus",
                "Mostafa Abdou",
                "Matthew Lamm",
                "Rahul Aralikatte",
                "Desmond Elliott"
            ],
            "abstract": "Image captioning models are usually evaluated on their ability to describe a\nheld-out set of images, not on their ability to generalize to unseen concepts.\nWe study the problem of compositional generalization, which measures how well a\nmodel composes unseen combinations of concepts when describing images.\nState-of-the-art image captioning models show poor generalization performance\non this task. We propose a multi-task model to address the poor performance,\nthat combines caption generation and image--sentence ranking, and uses a\ndecoding mechanism that re-ranks the captions according their similarity to the\nimage. This model is substantially better at generalizing to unseen\ncombinations of concepts compared to state-of-the-art captioning models.",
            "journal ref": "Proceedings of the 23rd Conference on Computational Natural\n  Language Learning (CoNLL), pp. 87--98, ACL, 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.04465v2",
            "date": "2019-09-10 13:18:59+00:00",
            "title": "Jointly embedding the local and global relations of heterogeneous graph for rumor detection",
            "authors": [
                "Chunyuan Yuan",
                "Qianwen Ma",
                "Wei Zhou",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "The development of social media has revolutionized the way people\ncommunicate, share information and make decisions, but it also provides an\nideal platform for publishing and spreading rumors. Existing rumor detection\nmethods focus on finding clues from text content, user profiles, and\npropagation patterns. However, the local semantic relation and global\nstructural information in the message propagation graph have not been well\nutilized by previous works.\n  In this paper, we present a novel global-local attention network (GLAN) for\nrumor detection, which jointly encodes the local semantic and global structural\ninformation. We first generate a better integrated representation for each\nsource tweet by fusing the semantic information of related retweets with the\nattention mechanism. Then, we model the global relationships among all source\ntweets, retweets, and users as a heterogeneous graph to capture the rich\nstructural information for rumor detection. We conduct experiments on three\nreal-world datasets, and the results demonstrate that GLAN significantly\noutperforms the state-of-the-art models in both rumor detection and early\ndetection scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04493v1",
            "date": "2019-09-06 07:47:20+00:00",
            "title": "Context-aware Deep Model for Entity Recommendation in Search Engine at Alibaba",
            "authors": [
                "Qianghuai Jia",
                "Ningyu Zhang",
                "Nengwei Hua"
            ],
            "abstract": "Entity recommendation, providing search users with an improved experience via\nassisting them in finding related entities for a given query, has become an\nindispensable feature of today's search engines. Existing studies typically\nonly consider the queries with explicit entities. They usually fail to handle\ncomplex queries that without entities, such as \"what food is good for cold\nweather\", because their models could not infer the underlying meaning of the\ninput text. In this work, we believe that contexts convey valuable evidence\nthat could facilitate the semantic modeling of queries, and take them into\nconsideration for entity recommendation. In order to better model the semantics\nof queries and entities, we learn the representation of queries and entities\njointly with attentive deep neural networks. We evaluate our approach using\nlarge-scale, real-world search logs from a widely used commercial Chinese\nsearch engine. Our system has been deployed in ShenMa Search Engine and you can\nfetch it in UC Browser of Alibaba. Results from online A/B test suggest that\nthe impression efficiency of click-through rate increased by 5.1% and page view\nincreased by 5.5%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.04495v1",
            "date": "2019-09-06 07:22:01+00:00",
            "title": "Natural Adversarial Sentence Generation with Gradient-based Perturbation",
            "authors": [
                "Yu-Lun Hsieh",
                "Minhao Cheng",
                "Da-Cheng Juan",
                "Wei Wei",
                "Wen-Lian Hsu",
                "Cho-Jui Hsieh"
            ],
            "abstract": "This work proposes a novel algorithm to generate natural language adversarial\ninput for text classification models, in order to investigate the robustness of\nthese models. It involves applying gradient-based perturbation on the sentence\nembeddings that are used as the features for the classifier, and learning a\ndecoder for generation. We employ this method to a sentiment analysis model and\nverify its effectiveness in inducing incorrect predictions by the model. We\nalso conduct quantitative and qualitative analysis on these examples and\ndemonstrate that our approach can generate more natural adversaries. In\naddition, it can be used to successfully perform black-box attacks, which\ninvolves attacking other existing models whose parameters are not known. On a\npublic sentiment analysis API, the proposed method introduces a 20% relative\ndecrease in average accuracy and 74% relative increase in absolute error.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.04625v1",
            "date": "2019-09-10 17:02:15+00:00",
            "title": "Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study",
            "authors": [
                "Aixiu An",
                "Peng Qian",
                "Ethan Wilcox",
                "Roger Levy"
            ],
            "abstract": "Neural language models have achieved state-of-the-art performances on many\nNLP tasks, and recently have been shown to learn a number of\nhierarchically-sensitive syntactic dependencies between individual words.\nHowever, equally important for language processing is the ability to combine\nwords into phrasal constituents, and use constituent-level features to drive\ndownstream expectations. Here we investigate neural models' ability to\nrepresent constituent-level features, using coordinated noun phrases as a case\nstudy. We assess whether different neural language models trained on English\nand French represent phrase-level number and gender features, and use those\nfeatures to drive downstream expectations. Our results suggest that models use\na linear combination of NP constituent number to drive CoordNP/verb number\nagreement. This behavior is highly regular and even sensitive to local\nsyntactic context, however it differs crucially from observed human behavior.\nModels have less success with gender agreement. Models trained on large corpora\nperform best, and there is no obvious advantage for models trained using\nexplicit syntactic supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04702v1",
            "date": "2019-09-10 18:39:26+00:00",
            "title": "Neural Embedding Allocation: Distributed Representations of Topic Models",
            "authors": [
                "Kamrun Naher Keya",
                "Yannis Papanikolaou",
                "James R. Foulds"
            ],
            "abstract": "Word embedding models such as the skip-gram learn vector representations of\nwords' semantic relationships, and document embedding models learn similar\nrepresentations for documents. On the other hand, topic models provide latent\nrepresentations of the documents' topical themes. To get the benefits of these\nrepresentations simultaneously, we propose a unifying algorithm, called neural\nembedding allocation (NEA), which deconstructs topic models into interpretable\nvector-space embeddings of words, topics, documents, authors, and so on, by\nlearning neural embeddings to mimic the topic models. We showcase NEA's\neffectiveness and generality on LDA, author-topic models and the recently\nproposed mixed membership skip gram topic model and achieve better performance\nwith the embeddings compared to several state-of-the-art models. Furthermore,\nwe demonstrate that using NEA to smooth out the topics improves coherence\nscores over the original topic models when the number of topics is large.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04800v2",
            "date": "2019-09-11 00:25:12+00:00",
            "title": "Probabilistic framework for solving Visual Dialog",
            "authors": [
                "Badri N. Patro",
                "Anupriy",
                "Vinay P. Namboodiri"
            ],
            "abstract": "In this paper, we propose a probabilistic framework for solving the task of\n`Visual Dialog'. Solving this task requires reasoning and understanding of\nvisual modality, language modality, and common sense knowledge to answer.\nVarious architectures have been proposed to solve this task by variants of\nmulti-modal deep learning techniques that combine visual and language\nrepresentations. However, we believe that it is crucial to understand and\nanalyze the sources of uncertainty for solving this task. Our approach allows\nfor estimating uncertainty and also aids a diverse generation of answers. The\nproposed approach is obtained through a probabilistic representation module\nthat provides us with representations for image, question and conversation\nhistory, a module that ensures that diverse latent representations for\ncandidate answers are obtained given the probabilistic representations and an\nuncertainty representation module that chooses the appropriate answer that\nminimizes uncertainty. We thoroughly evaluate the model with a detailed\nablation analysis, comparison with state of the art and visualization of the\nuncertainty that aids in the understanding of the method. Using the proposed\nprobabilistic framework, we thus obtain an improved visual dialog system that\nis also more explainable.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.04822v2",
            "date": "2019-09-11 02:20:57+00:00",
            "title": "Global Locality in Biomedical Relation and Event Extraction",
            "authors": [
                "Elaheh ShafieiBavani",
                "Antonio Jimeno Yepes",
                "Xu Zhong",
                "David Martinez Iraola"
            ],
            "abstract": "Due to the exponential growth of biomedical literature, event and relation\nextraction are important tasks in biomedical text mining. Most work only focus\non relation extraction, and detect a single entity pair mention on a short span\nof text, which is not ideal due to long sentences that appear in biomedical\ncontexts. We propose an approach to both relation and event extraction, for\nsimultaneously predicting relationships between all mention pairs in a text. We\nalso perform an empirical study to discuss different network setups for this\npurpose. The best performing model includes a set of multi-head attentions and\nconvolutions, an adaptation of the transformer architecture, which offers\nself-attention the ability to strengthen dependencies among related elements,\nand models the interaction between features extracted by multiple attention\nheads. Experiment results demonstrate that our approach outperforms the state\nof the art on a set of benchmark biomedical corpora including BioNLP 2009,\n2011, 2013 and BioCreative 2017 shared tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04948v2",
            "date": "2019-09-11 09:51:02+00:00",
            "title": "BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding",
            "authors": [
                "Timo I. Denk",
                "Christian Reisswig"
            ],
            "abstract": "For understanding generic documents, information like font sizes, column\nlayout, and generally the positioning of words may carry semantic information\nthat is crucial for solving a downstream document intelligence task. Our novel\nBERTgrid, which is based on Chargrid by Katti et al. (2018), represents a\ndocument as a grid of contextualized word piece embedding vectors, thereby\nmaking its spatial structure and semantics accessible to the processing neural\nnetwork. The contextualized embedding vectors are retrieved from a BERT\nlanguage model. We use BERTgrid in combination with a fully convolutional\nnetwork on a semantic instance segmentation task for extracting fields from\ninvoices. We demonstrate its performance on tabulated line item and document\nheader field extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.04985v1",
            "date": "2019-09-11 11:51:43+00:00",
            "title": "Learning Dynamic Author Representations with Temporal Language Models",
            "authors": [
                "Edouard Delasalles",
                "Sylvain Lamprier",
                "Ludovic Denoyer"
            ],
            "abstract": "Language models are at the heart of numerous works, notably in the text\nmining and information retrieval communities. These statistical models aim at\nextracting word distributions, from simple unigram models to recurrent\napproaches with latent variables that capture subtle dependencies in texts.\nHowever, those models are learned from word sequences only, and authors'\nidentities, as well as publication dates, are seldom considered. We propose a\nneural model, based on recurrent language modeling, which aims at capturing\nlanguage diffusion tendencies in author communities through time. By\nconditioning language models with author and temporal vector states, we are\nable to leverage the latent dependencies between the text contexts. This allows\nus to beat several temporal and non-temporal language baselines on two\nreal-world corpora, and to learn meaningful author representations that vary\nthrough time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05017v2",
            "date": "2019-09-09 19:48:53+00:00",
            "title": "Question Generation by Transformers",
            "authors": [
                "Kettip Kriangchaivech",
                "Artit Wangperawong"
            ],
            "abstract": "A machine learning model was developed to automatically generate questions\nfrom Wikipedia passages using transformers, an attention-based model eschewing\nthe paradigm of existing recurrent neural networks (RNNs). The model was\ntrained on the inverted Stanford Question Answering Dataset (SQuAD), which is a\nreading comprehension dataset consisting of 100,000+ questions posed by\ncrowdworkers on a set of Wikipedia articles. After training, the question\ngeneration model is able to generate simple questions relevant to unseen\npassages and answers containing an average of 8 words per question. The word\nerror rate (WER) was used as a metric to compare the similarity between SQuAD\nquestions and the model-generated questions. Although the high average WER\nsuggests that the questions generated differ from the original SQuAD questions,\nthe questions generated are mostly grammatically correct and plausible in their\nown right.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05023v2",
            "date": "2019-09-09 17:59:23+00:00",
            "title": "A Quantum Search Decoder for Natural Language Processing",
            "authors": [
                "Johannes Bausch",
                "Sathyawageeswar Subramanian",
                "Stephen Piddock"
            ],
            "abstract": "Probabilistic language models, e.g. those based on an LSTM, often face the\nproblem of finding a high probability prediction from a sequence of random\nvariables over a set of tokens. This is commonly addressed using a form of\ngreedy decoding such as beam search, where a limited number of\nhighest-likelihood paths (the beam width) of the decoder are kept, and at the\nend the maximum-likelihood path is chosen. In this work, we construct a quantum\nalgorithm to find the globally optimal parse (i.e. for infinite beam width)\nwith high constant success probability. When the input to the decoder is\ndistributed as a power-law with exponent $k>0$, our algorithm has runtime $R^{n\nf(R,k)}$, where $R$ is the alphabet size, $n$ the input length; here $f<1/2$,\nand $f\\rightarrow 0$ exponentially fast with increasing $k$, hence making our\nalgorithm always more than quadratically faster than its classical counterpart.\nWe further modify our procedure to recover a finite beam width variant, which\nenables an even stronger empirical speedup while still retaining higher\naccuracy than possible classically. Finally, we apply this quantum beam search\ndecoder to Mozilla's implementation of Baidu's DeepSpeech neural net, which we\nshow to exhibit such a power law word rank frequency.",
            "journal ref": "Quantum Mach. Intell. 3, 16 (2021)",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1909.05190v2",
            "date": "2019-09-09 03:00:39+00:00",
            "title": "Event Representation Learning Enhanced with External Commonsense Knowledge",
            "authors": [
                "Xiao Ding",
                "Kuo Liao",
                "Ting Liu",
                "Zhongyang Li",
                "Junwen Duan"
            ],
            "abstract": "Prior work has proposed effective methods to learn event representations that\ncan capture syntactic and semantic information over text corpus, demonstrating\ntheir effectiveness for downstream tasks such as script event prediction. On\nthe other hand, events extracted from raw texts lacks of commonsense knowledge,\nsuch as the intents and emotions of the event participants, which are useful\nfor distinguishing event pairs when there are only subtle differences in their\nsurface realizations. To address this issue, this paper proposes to leverage\nexternal commonsense knowledge about the intent and sentiment of the event.\nExperiments on three event-related tasks, i.e., event similarity, script event\nprediction and stock market prediction, show that our model obtains much better\nevent embeddings for the tasks, achieving 78% improvements on hard similarity\ntask, yielding more precise inferences on subsequent events under given\ncontexts, and better accuracies in predicting the volatilities of the stock\nmarket.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.05233v2",
            "date": "2019-09-07 00:32:11+00:00",
            "title": "The Neural State Pushdown Automata",
            "authors": [
                "Ankur Mali",
                "Alexander Ororbia",
                "C. Lee Giles"
            ],
            "abstract": "In order to learn complex grammars, recurrent neural networks (RNNs) require\nsufficient computational resources to ensure correct grammar recognition. A\nwidely-used approach to expand model capacity would be to couple an RNN to an\nexternal memory stack. Here, we introduce a \"neural state\" pushdown automaton\n(NSPDA), which consists of a digital stack, instead of an analog one, that is\ncoupled to a neural network state machine. We empirically show its\neffectiveness in recognizing various context-free grammars (CFGs). First, we\ndevelop the underlying mechanics of the proposed higher order recurrent network\nand its manipulation of a stack as well as how to stably program its underlying\npushdown automaton (PDA) to achieve desired finite-state network dynamics.\nNext, we introduce a noise regularization scheme for higher-order (tensor)\nnetworks, to our knowledge the first of its kind, and design an algorithm for\nimproved incremental learning. Finally, we design a method for inserting\ngrammar rules into a NSPDA and empirically show that this prior knowledge\nimproves its training convergence time by an order of magnitude and, in some\ncases, leads to better generalization. The NSPDA is also compared to a\nclassical analog stack neural network pushdown automaton (NNPDA) as well as a\nwide array of first and second-order RNNs with and without external memory,\ntrained using different learning algorithms. Our results show that, for Dyck(2)\nlanguages, prior rule-based knowledge is critical for optimization convergence\nand for ensuring generalization to longer sequences at test time. We observe\nthat many RNNs with and without memory, but no prior knowledge, fail to\nconverge and generalize poorly on CFGs.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1909.05246v1",
            "date": "2019-09-11 03:40:26+00:00",
            "title": "Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems",
            "authors": [
                "Mansour Saffar Mehrjardi",
                "Amine Trabelsi",
                "Osmar R. Zaiane"
            ],
            "abstract": "Self-attentional models are a new paradigm for sequence modelling tasks which\ndiffer from common sequence modelling methods, such as recurrence-based and\nconvolution-based sequence learning, in the way that their architecture is only\nbased on the attention mechanism. Self-attentional models have been used in the\ncreation of the state-of-the-art models in many NLP tasks such as neural\nmachine translation, but their usage has not been explored for the task of\ntraining end-to-end task-oriented dialogue generation systems yet. In this\nstudy, we apply these models on the three different datasets for training\ntask-oriented chatbots. Our finding shows that self-attentional models can be\nexploited to create end-to-end task-oriented chatbots which not only achieve\nhigher evaluation scores compared to recurrence-based models, but also do so\nmore efficiently.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05356v2",
            "date": "2019-08-31 17:40:21+00:00",
            "title": "Entity Projection via Machine Translation for Cross-Lingual NER",
            "authors": [
                "Alankar Jain",
                "Bhargavi Paranjape",
                "Zachary C. Lipton"
            ],
            "abstract": "Although over 100 languages are supported by strong off-the-shelf machine\ntranslation systems, only a subset of them possess large annotated corpora for\nnamed entity recognition. Motivated by this fact, we leverage machine\ntranslation to improve annotation-projection approaches to cross-lingual named\nentity recognition. We propose a system that improves over prior\nentity-projection methods by: (a) leveraging machine translation systems twice:\nfirst for translating sentences and subsequently for translating entities; (b)\nmatching entities based on orthographic and phonetic similarity; and (c)\nidentifying matches based on distributional statistics derived from the\ndataset. Our approach improves upon current state-of-the-art methods for\ncross-lingual named entity recognition on 5 diverse languages by an average of\n4.1 points. Further, our method achieves state-of-the-art F_1 scores for\nArmenian, outperforming even a monolingual model trained on Armenian source\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05357v1",
            "date": "2019-08-31 20:23:26+00:00",
            "title": "Out-of-Domain Detection for Low-Resource Text Classification Tasks",
            "authors": [
                "Ming Tan",
                "Yang Yu",
                "Haoyu Wang",
                "Dakuo Wang",
                "Saloni Potdar",
                "Shiyu Chang",
                "Mo Yu"
            ],
            "abstract": "Out-of-domain (OOD) detection for low-resource text classification is a\nrealistic but understudied task. The goal is to detect the OOD cases with\nlimited in-domain (ID) training data, since we observe that training data is\noften insufficient in machine learning applications. In this work, we propose\nan OOD-resistant Prototypical Network to tackle this zero-shot OOD detection\nand few-shot ID classification task. Evaluation on real-world datasets show\nthat the proposed solution outperforms state-of-the-art methods in zero-shot\nOOD detection task, while maintaining a competitive performance on ID\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05358v1",
            "date": "2019-09-01 22:18:39+00:00",
            "title": "Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset",
            "authors": [
                "Bill Byrne",
                "Karthik Krishnamoorthi",
                "Chinnadhurai Sankar",
                "Arvind Neelakantan",
                "Daniel Duckworth",
                "Semih Yavuz",
                "Ben Goodrich",
                "Amit Dubey",
                "Andy Cedilnik",
                "Kyu-Young Kim"
            ],
            "abstract": "A significant barrier to progress in data-driven approaches to building\ndialog systems is the lack of high quality, goal-oriented conversational data.\nTo help satisfy this elementary requirement, we introduce the initial release\nof the Taskmaster-1 dataset which includes 13,215 task-based dialogs comprising\nsix domains. Two procedures were used to create this collection, each with\nunique advantages. The first involves a two-person, spoken \"Wizard of Oz\" (WOz)\napproach in which trained agents and crowdsourced workers interact to complete\nthe task while the second is \"self-dialog\" in which crowdsourced workers write\nthe entire dialog themselves. We do not restrict the workers to detailed\nscripts or to a small knowledge base and hence we observe that our dataset\ncontains more realistic and diverse conversations in comparison to existing\ndatasets. We offer several baseline models including state of the art neural\nseq2seq architectures with benchmark performance as well as qualitative human\nevaluations. Dialogs are labeled with API calls and arguments, a simple and\ncost effective approach which avoids the requirement of complex annotation\nschema. The layer of abstraction between the dialog model and the service\nprovider API allows for a given model to interact with multiple services that\nprovide similar functionally. Finally, the dataset will evoke interest in\nwritten vs. spoken language, discourse patterns, error handling and other\nlinguistic phenomena related to dialog system research, development and design.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05362v1",
            "date": "2019-09-04 06:45:51+00:00",
            "title": "Problems with automating translation of movie/TV show subtitles",
            "authors": [
                "Prabhakar Gupta",
                "Mayank Sharma",
                "Kartik Pitale",
                "Keshav Kumar"
            ],
            "abstract": "We present 27 problems encountered in automating the translation of movie/TV\nshow subtitles. We categorize each problem in one of the three categories viz.\nproblems directly related to textual translation, problems related to subtitle\ncreation guidelines, and problems due to adaptability of machine translation\n(MT) engines. We also present the findings of a translation quality evaluation\nexperiment where we share the frequency of 16 key problems. We show that the\nsystems working at the frontiers of Natural Language Processing do not perform\nwell for subtitles and require some post-processing solutions for redressal of\nthese problems",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05365v2",
            "date": "2019-09-06 01:28:34+00:00",
            "title": "Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation",
            "authors": [
                "Mingyang Zhou",
                "Josh Arnold",
                "Zhou Yu"
            ],
            "abstract": "Reinforcement learning (RL) is an effective approach to learn an optimal\ndialog policy for task-oriented visual dialog systems. A common practice is to\napply RL on a neural sequence-to-sequence (seq2seq) framework with the action\nspace being the output vocabulary in the decoder. However, it is difficult to\ndesign a reward function that can achieve a balance between learning an\neffective policy and generating a natural dialog response. This paper proposes\na novel framework that alternatively trains a RL policy for image guessing and\na supervised seq2seq model to improve dialog generation quality. We evaluate\nour framework on the GuessWhich task and the framework achieves the\nstate-of-the-art performance in both task completion and dialog quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05367v3",
            "date": "2019-09-06 15:01:07+00:00",
            "title": "Learning in Text Streams: Discovery and Disambiguation of Entity and Relation Instances",
            "authors": [
                "Marco Maggini",
                "Giuseppe Marra",
                "Stefano Melacci",
                "Andrea Zugarini"
            ],
            "abstract": "We consider a scenario where an artificial agent is reading a stream of text\ncomposed of a set of narrations, and it is informed about the identity of some\nof the individuals that are mentioned in the text portion that is currently\nbeing read. The agent is expected to learn to follow the narrations, thus\ndisambiguating mentions and discovering new individuals. We focus on the case\nin which individuals are entities and relations, and we propose an end-to-end\ntrainable memory network that learns to discover and disambiguate them in an\nonline manner, performing one-shot learning, and dealing with a small number of\nsparse supervisions. Our system builds a not-given-in-advance knowledge base,\nand it improves its skills while reading unsupervised text. The model deals\nwith abrupt changes in the narration, taking into account their effects when\nresolving co-references. We showcase the strong disambiguation and discovery\nskills of our model on a corpus of Wikipedia documents and on a newly\nintroduced dataset, that we make publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05370v1",
            "date": "2019-09-06 19:24:51+00:00",
            "title": "An Auxiliary Classifier Generative Adversarial Framework for Relation Extraction",
            "authors": [
                "Yun Zhao"
            ],
            "abstract": "Relation extraction models suffer from limited qualified training data. Using\nhuman annotators to label sentences is too expensive and does not scale well\nespecially when dealing with large datasets. In this paper, we use Auxiliary\nClassifier Generative Adversarial Networks (AC-GANs) to generate high-quality\nrelational sentences and to improve the performance of relation classifier in\nend-to-end models. In AC-GAN, the discriminator gives not only a probability\ndistribution over the real source, but also a probability distribution over the\nrelation labels. This helps to generate meaningful relational sentences.\nExperimental results show that our proposed data augmentation method\nsignificantly improves the performance of relation extraction compared to\nstate-of-the-art methods",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05372v1",
            "date": "2019-09-07 03:51:13+00:00",
            "title": "Overton: A Data System for Monitoring and Improving Machine-Learned Products",
            "authors": [
                "Christopher R\u00e9",
                "Feng Niu",
                "Pallavi Gudipati",
                "Charles Srisuwananukorn"
            ],
            "abstract": "We describe a system called Overton, whose main design goal is to support\nengineers in building, monitoring, and improving production machine learning\nsystems. Key challenges engineers face are monitoring fine-grained quality,\ndiagnosing errors in sophisticated applications, and handling contradictory or\nincomplete supervision data. Overton automates the life cycle of model\nconstruction, deployment, and monitoring by providing a set of novel\nhigh-level, declarative abstractions. Overton's vision is to shift developers\nto these higher-level tasks instead of lower-level machine learning tasks. In\nfact, using Overton, engineers can build deep-learning-based applications\nwithout writing any code in frameworks like TensorFlow. For over a year,\nOverton has been used in production to support multiple applications in both\nnear-real-time applications and back-of-house processing. In that time,\nOverton-based applications have answered billions of queries in multiple\nlanguages and processed trillions of records reducing errors 1.7-2.9 times\nversus production systems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.05478v1",
            "date": "2019-09-12 06:39:07+00:00",
            "title": "A Robust Hybrid Approach for Textual Document Classification",
            "authors": [
                "Muhammad Nabeel Asim",
                "Muhammad Usman Ghani Khan",
                "Muhammad Imran Malik",
                "Andreas Dengel",
                "Sheraz Ahmed"
            ],
            "abstract": "Text document classification is an important task for diverse natural\nlanguage processing based applications. Traditional machine learning approaches\nmainly focused on reducing dimensionality of textual data to perform\nclassification. This although improved the overall classification accuracy, the\nclassifiers still faced sparsity problem due to lack of better data\nrepresentation techniques. Deep learning based text document classification, on\nthe other hand, benefitted greatly from the invention of word embeddings that\nhave solved the sparsity problem and researchers focus mainly remained on the\ndevelopment of deep architectures. Deeper architectures, however, learn some\nredundant features that limit the performance of deep learning based solutions.\nIn this paper, we propose a two stage text document classification methodology\nwhich combines traditional feature engineering with automatic feature\nengineering (using deep learning). The proposed methodology comprises a filter\nbased feature selection (FSE) algorithm followed by a deep convolutional neural\nnetwork. This methodology is evaluated on the two most commonly used public\ndatasets, i.e., 20 Newsgroups data and BBC news data. Evaluation results reveal\nthat the proposed methodology outperforms the state-of-the-art of both the\n(traditional) machine learning and deep learning based text document\nclassification methodologies with a significant margin of 7.7% on 20 Newsgroups\nand 6.6% on BBC news datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05504v1",
            "date": "2019-09-12 08:35:54+00:00",
            "title": "Classifying Multilingual User Feedback using Traditional Machine Learning and Deep Learning",
            "authors": [
                "Christoph Stanik",
                "Marlo Haering",
                "Walid Maalej"
            ],
            "abstract": "With the rise of social media like Twitter and of software distribution\nplatforms like app stores, users got various ways to express their opinion\nabout software products. Popular software vendors get user feedback\nthousandfold per day. Research has shown that such feedback contains valuable\ninformation for software development teams such as problem reports or feature\nand support inquires. Since the manual analysis of user feedback is cumbersome\nand hard to manage many researchers and tool vendors suggested to use automated\nanalyses based on traditional supervised machine learning approaches. In this\nwork, we compare the results of traditional machine learning and deep learning\nin classifying user feedback in English and Italian into problem reports,\ninquiries, and irrelevant. Our results show that using traditional machine\nlearning, we can still achieve comparable results to deep learning, although we\ncollected thousands of labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05639v1",
            "date": "2019-09-03 16:18:34+00:00",
            "title": "Quantifying and Correlating Rhythm Formants in Speech",
            "authors": [
                "Dafydd Gibbon",
                "Peng Li"
            ],
            "abstract": "The objective of the present study is exploratory: to introduce and apply a\nnew theory of speech rhythm zones or rhythm formants (R-formants). R-formants\nare zones of high magnitude frequencies in the low frequency (LF) long-term\nspectrum (LTS), rather like formants in the short-term spectra of vowels and\nconsonants. After an illustration of the method, an R-formant analysis is made\nof non-elicited extracts from public speeches. The LF-LTS of three domains, the\namplitude modulated (AM) absolute (rectified) signal, the amplitude envelope\nmodulation (AEM) and frequency modulation (FM, F0, 'pitch') of the signal are\ncompared. The first two correlate well, but the third does not correlate\nconsistently with the other two, presumably due to variability of tone, pitch\naccent and intonation. Consequently, only the LF LTS of the absolute speech\nsignal is used in the empirical analysis. An informal discussion of the\nrelation between R-formant patterns and utterance structure and a selection of\npragmatic variables over the same utterances showed some trends for R-formant\nfunctionality and thus useful directions for future research.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.05645v2",
            "date": "2019-09-06 03:06:38+00:00",
            "title": "Learning Alignment for Multimodal Emotion Recognition from Speech",
            "authors": [
                "Haiyang Xu",
                "Hui Zhang",
                "Kun Han",
                "Yun Wang",
                "Yiping Peng",
                "Xiangang Li"
            ],
            "abstract": "Speech emotion recognition is a challenging problem because human convey\nemotions in subtle and complex ways. For emotion recognition on human speech,\none can either extract emotion related features from audio signals or employ\nspeech recognition techniques to generate text from speech and then apply\nnatural language processing to analyze the sentiment. Further, emotion\nrecognition will be beneficial from using audio-textual multimodal information,\nit is not trivial to build a system to learn from multimodality. One can build\nmodels for two input sources separately and combine them in a decision level,\nbut this method ignores the interaction between speech and text in the temporal\ndomain. In this paper, we propose to use an attention mechanism to learn the\nalignment between speech frames and text words, aiming to produce more accurate\nmultimodal feature representations. The aligned multimodal features are fed\ninto a sequential model for emotion recognition. We evaluate the approach on\nthe IEMOCAP dataset and the experimental results show the proposed approach\nachieves the state-of-the-art performance on the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05780v2",
            "date": "2019-09-12 16:29:24+00:00",
            "title": "Fine-Grained Entity Typing for Domain Independent Entity Linking",
            "authors": [
                "Yasumasa Onoe",
                "Greg Durrett"
            ],
            "abstract": "Neural entity linking models are very powerful, but run the risk of\noverfitting to the domain they are trained in. For this problem, a domain is\ncharacterized not just by genre of text but even by factors as specific as the\nparticular distribution of entities, as neural models tend to overfit by\nmemorizing properties of frequent entities in a dataset. We tackle the problem\nof building robust entity linking models that generalize effectively and do not\nrely on labeled entity linking data with a specific entity distribution. Rather\nthan predicting entities directly, our approach models fine-grained entity\nproperties, which can help disambiguate between even closely related entities.\nWe derive a large inventory of types (tens of thousands) from Wikipedia\ncategories, and use hyperlinked mentions in Wikipedia to distantly label data\nand train an entity typing model. At test time, we classify a mention with this\ntyping model and use soft type predictions to link the mention to the most\nsimilar candidate entity. We evaluate our entity linking system on the\nCoNLL-YAGO dataset (Hoffart et al., 2011) and show that our approach\noutperforms prior domain-independent entity linking systems. We also test our\napproach in a harder setting derived from the WikilinksNED dataset (Eshel et\nal., 2017) where all the mention-entity pairs are unseen during test time.\nResults indicate that our approach generalizes better than a state-of-the-art\nneural model on the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05803v2",
            "date": "2019-09-12 17:00:45+00:00",
            "title": "Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning",
            "authors": [
                "Yichen Jiang",
                "Mohit Bansal"
            ],
            "abstract": "Multi-hop QA requires a model to connect multiple pieces of evidence\nscattered in a long context to answer the question. The recently proposed\nHotpotQA (Yang et al., 2018) dataset is comprised of questions embodying four\ndifferent multi-hop reasoning paradigms (two bridge entity setups, checking\nmultiple properties, and comparing two entities), making it challenging for a\nsingle neural network to handle all four. In this work, we present an\ninterpretable, controller-based Self-Assembling Neural Modular Network (Hu et\nal., 2017, 2018) for multi-hop reasoning, where we design four novel modules\n(Find, Relocate, Compare, NoOp) to perform unique types of language reasoning.\nBased on a question, our layout controller RNN dynamically infers a series of\nreasoning modules to construct the entire network. Empirically, we show that\nour dynamic, multi-hop modular network achieves significant improvements over\nthe static, single-hop baseline (on both regular and adversarial evaluation).\nWe further demonstrate the interpretability of our model via three analyses.\nFirst, the controller can softly decompose the multi-hop question into multiple\nsingle-hop sub-questions to promote compositional reasoning behavior of the\nmain network. Second, the controller can predict layouts that conform to the\nlayouts designed by human experts. Finally, the intermediate module can infer\nthe entity that connects two distantly-located supporting facts by addressing\nthe sub-question from the controller.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05819v1",
            "date": "2019-09-12 17:27:46+00:00",
            "title": "Anonymising Queries by Semantic Decomposition",
            "authors": [
                "Danushka Bollegala",
                "Tomoya Machide",
                "Ken-ichi Kawarabayashi"
            ],
            "abstract": "Protecting the privacy of search engine users is an important requirement in\nmany information retrieval scenarios. A user might not want a search engine to\nguess his or her information need despite requesting relevant results. We\npropose a method to protect the privacy of search engine users by decomposing\nthe queries using semantically \\emph{related} and unrelated \\emph{distractor}\nterms. Instead of a single query, the search engine receives multiple\ndecomposed query terms. Next, we reconstruct the search results relevant to the\noriginal query term by aggregating the search results retrieved for the\ndecomposed query terms. We show that the word embeddings learnt using a\ndistributed representation learning method can be used to find semantically\nrelated and distractor query terms. We derive the relationship between the\n\\emph{anonymity} achieved through the proposed query anonymisation method and\nthe \\emph{reconstructability} of the original search results using the\ndecomposed queries. We analytically study the risk of discovering the search\nengine users' information intents under the proposed query anonymisation\nmethod, and empirically evaluate its robustness against clustering-based\nattacks. Our experimental results show that the proposed method can accurately\nreconstruct the search results for user queries, without compromising the\nprivacy of the search engine users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05863v1",
            "date": "2019-09-12 18:00:00+00:00",
            "title": "Finding Generalizable Evidence by Learning to Convince Q&A Models",
            "authors": [
                "Ethan Perez",
                "Siddharth Karamcheti",
                "Rob Fergus",
                "Jason Weston",
                "Douwe Kiela",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose a system that finds the strongest supporting evidence for a given\nanswer to a question, using passage-based question-answering (QA) as a testbed.\nWe train evidence agents to select the passage sentences that most convince a\npretrained QA model of a given answer, if the QA model received those sentences\ninstead of the full passage. Rather than finding evidence that convinces one\nmodel alone, we find that agents select evidence that generalizes; agent-chosen\nevidence increases the plausibility of the supported answer, as judged by other\nQA models and humans. Given its general nature, this approach improves QA in a\nrobust manner: using agent-selected evidence (i) humans can correctly answer\nquestions with only ~20% of the full passage and (ii) QA models can generalize\nto longer passages and harder questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05882v2",
            "date": "2019-09-12 18:02:03+00:00",
            "title": "The emotions that we perceive in music: the influence of language and lyrics comprehension on agreement",
            "authors": [
                "Juan Sebasti\u00e1n G\u00f3mez Ca\u00f1\u00f3n",
                "Perfecto Herrera",
                "Emilia G\u00f3mez",
                "Estefan\u00eda Cano"
            ],
            "abstract": "In the present study, we address the relationship between the emotions\nperceived in pop and rock music (mainly in Euro-American styles with English\nlyrics) and the language spoken by the listener. Our goal is to understand the\ninfluence of lyrics comprehension on the perception of emotions and use this\ninformation to improve Music Emotion Recognition (MER) models. Two main\nresearch questions are addressed: 1. Are there differences and similarities\nbetween the emotions perceived in pop/rock music by listeners raised with\ndifferent mother tongues? 2. Do personal characteristics have an influence on\nthe perceived emotions for listeners of a given language? Personal\ncharacteristics include the listeners' general demographics, familiarity and\npreference for the fragments, and music sophistication. Our hypothesis is that\ninter-rater agreement (as defined by Krippendorff's alpha coefficient) from\nsubjects is directly influenced by the comprehension of lyrics.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1909.05885v1",
            "date": "2019-09-12 18:03:17+00:00",
            "title": "Analyzing machine-learned representations: A natural language case study",
            "authors": [
                "Ishita Dasgupta",
                "Demi Guo",
                "Samuel J. Gershman",
                "Noah D. Goodman"
            ],
            "abstract": "As modern deep networks become more complex, and get closer to human-like\ncapabilities in certain domains, the question arises of how the representations\nand decision rules they learn compare to the ones in humans. In this work, we\nstudy representations of sentences in one such artificial system for natural\nlanguage processing. We first present a diagnostic test dataset to examine the\ndegree of abstract composable structure represented. Analyzing performance on\nthese diagnostic tests indicates a lack of systematicity in the representations\nand decision rules, and reveals a set of heuristic strategies. We then\ninvestigate the effect of the training distribution on learning these heuristic\nstrategies, and study changes in these representations with various\naugmentations to the training set. Our results reveal parallels to the\nanalogous representations in people. We find that these systems can learn\nabstract rules and generalize them to new contexts under certain circumstances\n-- similar to human zero-shot reasoning. However, we also note some\nshortcomings in this generalization behavior -- similar to human judgment\nerrors like belief bias. Studying these parallels suggests new ways to\nunderstand psychological phenomena in humans as well as informs best strategies\nfor building artificial intelligence with human-like language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.05890v1",
            "date": "2019-09-12 18:11:56+00:00",
            "title": "Determining the Scale of Impact from Denial-of-Service Attacks in Real Time Using Twitter",
            "authors": [
                "Chi Zhang",
                "Bryan Wilkinson",
                "Ashwinkumar Ganesan",
                "Tim Oates"
            ],
            "abstract": "Denial of Service (DoS) attacks are common in on-line and mobile services\nsuch as Twitter, Facebook and banking. As the scale and frequency of\nDistributed Denial of Service (DDoS) attacks increase, there is an urgent need\nfor determining the impact of the attack. Two central challenges of the task\nare to get feedback from a large number of users and to get it in a timely\nmanner. In this paper, we present a weakly-supervised model that does not need\nannotated data to measure the impact of DoS issues by applying Latent Dirichlet\nAllocation and symmetric Kullback-Leibler divergence on tweets. There is a\nlimitation to the weakly-supervised module. It assumes that the event detected\nin a time window is a DoS attack event. This will become less of a problem,\nwhen more non-attack events twitter got collected and become less likely to be\nidentified as a new event. Another way to remove that limitation, an optional\nclassification layer, trained on manually annotated DoS attack tweets, to\nfilter out non-attack tweets can be used to increase precision at the expense\nof recall. Experimental results show that we can learn weakly-supervised models\nthat can achieve comparable precision to supervised ones and can be generalized\nacross entities in the same industry.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1909.05952v1",
            "date": "2019-09-12 21:12:10+00:00",
            "title": "End-to-End Neural Speaker Diarization with Permutation-Free Objectives",
            "authors": [
                "Yusuke Fujita",
                "Naoyuki Kanda",
                "Shota Horiguchi",
                "Kenji Nagamatsu",
                "Shinji Watanabe"
            ],
            "abstract": "In this paper, we propose a novel end-to-end neural-network-based speaker\ndiarization method. Unlike most existing methods, our proposed method does not\nhave separate modules for extraction and clustering of speaker representations.\nInstead, our model has a single neural network that directly outputs speaker\ndiarization results. To realize such a model, we formulate the speaker\ndiarization problem as a multi-label classification problem, and introduces a\npermutation-free objective function to directly minimize diarization errors\nwithout being suffered from the speaker-label permutation problem. Besides its\nend-to-end simplicity, the proposed method also benefits from being able to\nexplicitly handle overlapping speech during training and inference. Because of\nthe benefit, our model can be easily trained/adapted with real-recorded\nmulti-speaker conversations just by feeding the corresponding multi-speaker\nsegment labels. We evaluated the proposed method on simulated speech mixtures.\nThe proposed method achieved diarization error rate of 12.28%, while a\nconventional clustering-based system produced diarization error rate of 28.77%.\nFurthermore, the domain adaptation with real-recorded speech provided 25.6%\nrelative improvement on the CALLHOME dataset. Our source code is available\nonline at https://github.com/hitachi-speech/EEND.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06044v3",
            "date": "2019-09-13 05:50:50+00:00",
            "title": "Say What I Want: Towards the Dark Side of Neural Dialogue Models",
            "authors": [
                "Haochen Liu",
                "Tyler Derr",
                "Zitao Liu",
                "Jiliang Tang"
            ],
            "abstract": "Neural dialogue models have been widely adopted in various chatbot\napplications because of their good performance in simulating and generalizing\nhuman conversations. However, there exists a dark side of these models -- due\nto the vulnerability of neural networks, a neural dialogue model can be\nmanipulated by users to say what they want, which brings in concerns about the\nsecurity of practical chatbot services. In this work, we investigate whether we\ncan craft inputs that lead a well-trained black-box neural dialogue model to\ngenerate targeted outputs. We formulate this as a reinforcement learning (RL)\nproblem and train a Reverse Dialogue Generator which efficiently finds such\ninputs for targeted outputs. Experiments conducted on a representative neural\ndialogue model show that our proposed model is able to discover such desired\ninputs in a considerable portion of cases. Overall, our work reveals this\nweakness of neural dialogue models and may prompt further researches of\ndeveloping corresponding solutions to avoid it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06058v2",
            "date": "2019-09-13 06:44:30+00:00",
            "title": "Neural Correction Model for Open-Domain Named Entity Recognition",
            "authors": [
                "Mengdi Zhu",
                "Zheye Deng",
                "Wenhan Xiong",
                "Mo Yu",
                "Ming Zhang",
                "William Yang Wang"
            ],
            "abstract": "Named Entity Recognition (NER) plays an important role in a wide range of\nnatural language processing tasks, such as relation extraction, question\nanswering, etc. However, previous studies on NER are limited to particular\ngenres, using small manually-annotated or large but low-quality datasets.\nMeanwhile, previous datasets for open-domain NER, built using distant\nsupervision, suffer from low precision, recall and ratio of annotated tokens\n(RAT). In this work, to address the low precision and recall problems, we first\nutilize DBpedia as the source of distant supervision to annotate abstracts from\nWikipedia and design a neural correction model trained with a human-annotated\nNER dataset, DocRED, to correct the false entity labels. In this way, we build\na large and high-quality dataset called AnchorNER and then train various models\nwith it. To address the low RAT problem of previous datasets, we introduce a\nmulti-task learning method to exploit the context information. We evaluate our\nmethods on five NER datasets and our experimental results show that models\ntrained with AnchorNER and our multi-task learning method obtain\nstate-of-the-art performances in the open-domain setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06146v1",
            "date": "2019-09-13 11:18:20+00:00",
            "title": "PubMedQA: A Dataset for Biomedical Research Question Answering",
            "authors": [
                "Qiao Jin",
                "Bhuwan Dhingra",
                "Zhengping Liu",
                "William W. Cohen",
                "Xinghua Lu"
            ],
            "abstract": "We introduce PubMedQA, a novel biomedical question answering (QA) dataset\ncollected from PubMed abstracts. The task of PubMedQA is to answer research\nquestions with yes/no/maybe (e.g.: Do preoperative statins reduce atrial\nfibrillation after coronary artery bypass grafting?) using the corresponding\nabstracts. PubMedQA has 1k expert-annotated, 61.2k unlabeled and 211.3k\nartificially generated QA instances. Each PubMedQA instance is composed of (1)\na question which is either an existing research article title or derived from\none, (2) a context which is the corresponding abstract without its conclusion,\n(3) a long answer, which is the conclusion of the abstract and, presumably,\nanswers the research question, and (4) a yes/no/maybe answer which summarizes\nthe conclusion. PubMedQA is the first QA dataset where reasoning over\nbiomedical research texts, especially their quantitative contents, is required\nto answer the questions. Our best performing model, multi-phase fine-tuning of\nBioBERT with long answer bag-of-word statistics as additional supervision,\nachieves 68.1% accuracy, compared to single human performance of 78.0% accuracy\nand majority-baseline of 55.2% accuracy, leaving much room for improvement.\nPubMedQA is publicly available at https://pubmedqa.github.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06162v1",
            "date": "2019-09-13 12:11:47+00:00",
            "title": "Neural Architectures for Fine-Grained Propaganda Detection in News",
            "authors": [
                "Pankaj Gupta",
                "Khushbu Saxena",
                "Usama Yaseen",
                "Thomas Runkler",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper describes our system (MIC-CIS) details and results of\nparticipation in the fine-grained propaganda detection shared task 2019. To\naddress the tasks of sentence (SLC) and fragment level (FLC) propaganda\ndetection, we explore different neural architectures (e.g., CNN, LSTM-CRF and\nBERT) and extract linguistic (e.g., part-of-speech, named entity, readability,\nsentiment, emotion, etc.), layout and topical features. Specifically, we have\ndesigned multi-granularity and multi-tasking neural architectures to jointly\nperform both the sentence and fragment level propaganda detection.\nAdditionally, we investigate different ensemble schemes such as\nmajority-voting, relax-voting, etc. to boost overall system performance.\nCompared to the other participating systems, our submissions are ranked 3rd and\n4th in FLC and SLC tasks, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06200v2",
            "date": "2019-09-13 13:05:27+00:00",
            "title": "A Neural Approach to Irony Generation",
            "authors": [
                "Mengdi Zhu",
                "Zhiwei Yu",
                "Xiaojun Wan"
            ],
            "abstract": "Ironies can not only express stronger emotions but also show a sense of\nhumor. With the development of social media, ironies are widely used in public.\nAlthough many prior research studies have been conducted in irony detection,\nfew studies focus on irony generation. The main challenges for irony generation\nare the lack of large-scale irony dataset and difficulties in modeling the\nironic pattern. In this work, we first systematically define irony generation\nbased on style transfer task. To address the lack of data, we make use of\ntwitter and build a large-scale dataset. We also design a combination of\nrewards for reinforcement learning to control the generation of ironic\nsentences. Experimental results demonstrate the effectiveness of our model in\nterms of irony accuracy, sentiment preservation, and content preservation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06247v1",
            "date": "2019-09-13 14:18:18+00:00",
            "title": "End-to-End Neural Speaker Diarization with Self-attention",
            "authors": [
                "Yusuke Fujita",
                "Naoyuki Kanda",
                "Shota Horiguchi",
                "Yawen Xue",
                "Kenji Nagamatsu",
                "Shinji Watanabe"
            ],
            "abstract": "Speaker diarization has been mainly developed based on the clustering of\nspeaker embeddings. However, the clustering-based approach has two major\nproblems; i.e., (i) it is not optimized to minimize diarization errors\ndirectly, and (ii) it cannot handle speaker overlaps correctly. To solve these\nproblems, the End-to-End Neural Diarization (EEND), in which a bidirectional\nlong short-term memory (BLSTM) network directly outputs speaker diarization\nresults given a multi-talker recording, was recently proposed. In this study,\nwe enhance EEND by introducing self-attention blocks instead of BLSTM blocks.\nIn contrast to BLSTM, which is conditioned only on its previous and next hidden\nstates, self-attention is directly conditioned on all the other frames, making\nit much suitable for dealing with the speaker diarization problem. We evaluated\nour proposed method on simulated mixtures, real telephone calls, and real\ndialogue recordings. The experimental results revealed that the self-attention\nwas the key to achieving good performance and that our proposed method\nperformed significantly better than the conventional BLSTM-based method. Our\nmethod was even better than that of the state-of-the-art x-vector\nclustering-based method. Finally, by visualizing the latent representation, we\nshow that the self-attention can capture global speaker characteristics in\naddition to local speech activity dynamics. Our source code is available online\nat https://github.com/hitachi-speech/EEND.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06273v1",
            "date": "2019-09-13 14:54:37+00:00",
            "title": "Scene Graph Parsing by Attention Graph",
            "authors": [
                "Martin Andrews",
                "Yew Ken Chia",
                "Sam Witteveen"
            ],
            "abstract": "Scene graph representations, which form a graph of visual object nodes\ntogether with their attributes and relations, have proved useful across a\nvariety of vision and language applications. Recent work in the area has used\nNatural Language Processing dependency tree methods to automatically build\nscene graphs.\n  In this work, we present an 'Attention Graph' mechanism that can be trained\nend-to-end, and produces a scene graph structure that can be lifted directly\nfrom the top layer of a standard Transformer model.\n  The scene graphs generated by our model achieve an F-score similarity of\n52.21% to ground-truth graphs on the evaluation set using the SPICE metric,\nsurpassing the best previous approaches by 2.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06317v2",
            "date": "2019-09-13 16:27:08+00:00",
            "title": "A Comparative Study on Transformer vs RNN in Speech Applications",
            "authors": [
                "Shigeki Karita",
                "Nanxin Chen",
                "Tomoki Hayashi",
                "Takaaki Hori",
                "Hirofumi Inaguma",
                "Ziyan Jiang",
                "Masao Someki",
                "Nelson Enrique Yalta Soplin",
                "Ryuichi Yamamoto",
                "Xiaofei Wang",
                "Shinji Watanabe",
                "Takenori Yoshimura",
                "Wangyou Zhang"
            ],
            "abstract": "Sequence-to-sequence models have been widely used in end-to-end speech\nprocessing, for example, automatic speech recognition (ASR), speech translation\n(ST), and text-to-speech (TTS). This paper focuses on an emergent\nsequence-to-sequence model called Transformer, which achieves state-of-the-art\nperformance in neural machine translation and other natural language processing\napplications. We undertook intensive studies in which we experimentally\ncompared and analyzed Transformer and conventional recurrent neural networks\n(RNN) in a total of 15 ASR, one multilingual ASR, one ST, and two TTS\nbenchmarks. Our experiments revealed various training tips and significant\nperformance benefits obtained with Transformer for each task including the\nsurprising superiority of Transformer in 13/15 ASR benchmarks in comparison\nwith RNN. We are preparing to release Kaldi-style reproducible recipes using\nopen source and publicly available datasets for all the ASR, ST, and TTS tasks\nfor the community to succeed our exciting outcomes.",
            "journal ref": "IEEE Automatic Speech Recognition and Understanding Workshop 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06351v2",
            "date": "2019-09-13 17:56:13+00:00",
            "title": "Probing the Information Encoded in X-vectors",
            "authors": [
                "Desh Raj",
                "David Snyder",
                "Daniel Povey",
                "Sanjeev Khudanpur"
            ],
            "abstract": "Deep neural network based speaker embeddings, such as x-vectors, have been\nshown to perform well in text-independent speaker recognition/verification\ntasks. In this paper, we use simple classifiers to investigate the contents\nencoded by x-vector embeddings. We probe these embeddings for information\nrelated to the speaker, channel, transcription (sentence, words, phones), and\nmeta information about the utterance (duration and augmentation type), and\ncompare these with the information encoded by i-vectors across a varying number\nof dimensions. We also study the effect of data augmentation during extractor\ntraining on the information captured by x-vectors. Experiments on the RedDots\ndata set show that x-vectors capture spoken content and channel-related\ninformation, while performing well on speaker verification tasks.",
            "journal ref": "IEEE Automatic Speech Recognition and Understanding Workshop\n  (ASRU) (2019): 726-733",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06356v1",
            "date": "2019-09-13 17:59:03+00:00",
            "title": "Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering",
            "authors": [
                "Shiyue Zhang",
                "Mohit Bansal"
            ],
            "abstract": "Text-based Question Generation (QG) aims at generating natural and relevant\nquestions that can be answered by a given answer in some context. Existing QG\nmodels suffer from a \"semantic drift\" problem, i.e., the semantics of the\nmodel-generated question drifts away from the given context and answer. In this\npaper, we first propose two semantics-enhanced rewards obtained from downstream\nquestion paraphrasing and question answering tasks to regularize the QG model\nto generate semantically valid questions. Second, since the traditional\nevaluation metrics (e.g., BLEU) often fall short in evaluating the quality of\ngenerated questions, we propose a QA-based evaluation method which measures the\nQG model's ability to mimic human annotators in generating QA training data.\nExperiments show that our method achieves the new state-of-the-art performance\nw.r.t. traditional metrics, and also performs best on our QA-based evaluation\nmetrics. Further, we investigate how to use our QG model to augment QA datasets\nand enable semi-supervised QA. We propose two ways to generate synthetic QA\npairs: generate new questions from existing articles or collect QA pairs from\nnew articles. We also propose two empirically effective strategies, a data\nfilter and mixing mini-batch training, to properly use the QG-generated data\nfor QA. Experiments show that our method improves over both BiDAF and BERT QA\nbaselines, even without introducing new articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06414v1",
            "date": "2019-09-13 19:16:53+00:00",
            "title": "Learning Household Task Knowledge from WikiHow Descriptions",
            "authors": [
                "Yilun Zhou",
                "Julie A. Shah",
                "Steven Schockaert"
            ],
            "abstract": "Commonsense procedural knowledge is important for AI agents and robots that\noperate in a human environment. While previous attempts at constructing\nprocedural knowledge are mostly rule- and template-based, recent advances in\ndeep learning provide the possibility of acquiring such knowledge directly from\nnatural language sources. As a first step in this direction, we propose a model\nto learn embeddings for tasks, as well as the individual steps that need to be\ntaken to solve them, based on WikiHow articles. We learn these embeddings such\nthat they are predictive of both step relevance and step ordering. We also\nexperiment with the use of integer programming for inferring consistent global\nstep orderings from noisy pairwise predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06434v1",
            "date": "2019-09-13 20:23:40+00:00",
            "title": "Adaptive Scheduling for Multi-Task Learning",
            "authors": [
                "S\u00e9bastien Jean",
                "Orhan Firat",
                "Melvin Johnson"
            ],
            "abstract": "To train neural machine translation models simultaneously on multiple tasks\n(languages), it is common to sample each task uniformly or in proportion to\ndataset sizes. As these methods offer little control over performance\ntrade-offs, we explore different task scheduling approaches. We first consider\nexisting non-adaptive techniques, then move on to adaptive schedules that\nover-sample tasks with poorer results compared to their respective baseline. As\nexplicit schedules can be inefficient, especially if one task is highly\nover-sampled, we also consider implicit schedules, learning to scale learning\nrates or gradients of individual tasks instead. These techniques allow training\nmultilingual models that perform better for low-resource language pairs (tasks\nwith small amount of data), while minimizing negative effects on high-resource\ntasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.06515v2",
            "date": "2019-09-14 03:05:30+00:00",
            "title": "Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade",
            "authors": [
                "Juan Pino",
                "Liezl Puzon",
                "Jiatao Gu",
                "Xutai Ma",
                "Arya D. McCarthy",
                "Deepak Gopinath"
            ],
            "abstract": "For automatic speech translation (AST), end-to-end approaches are\noutperformed by cascaded models that transcribe with automatic speech\nrecognition (ASR), then translate with machine translation (MT). A major cause\nof the performance gap is that, while existing AST corpora are small, massive\ndatasets exist for both the ASR and MT subsystems. In this work, we evaluate\nseveral data augmentation and pretraining approaches for AST, by comparing all\non the same datasets. Simple data augmentation by translating ASR transcripts\nproves most effective on the English--French augmented LibriSpeech dataset,\nclosing the performance gap from 8.2 to 1.4 BLEU, compared to a very strong\ncascade that could directly utilize copious ASR and MT data. The same\nend-to-end approach plus fine-tuning closes the gap on the English--Romanian\nMuST-C dataset from 6.7 to 3.7 BLEU. In addition to these results, we present\npractical recommendations for augmentation and pretraining approaches. Finally,\nwe decrease the performance gap to 0.01 BLEU using a Transformer-based\narchitecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06522v3",
            "date": "2019-09-14 03:46:49+00:00",
            "title": "Multilingual Graphemic Hybrid ASR with Massive Data Augmentation",
            "authors": [
                "Chunxi Liu",
                "Qiaochu Zhang",
                "Xiaohui Zhang",
                "Kritika Singh",
                "Yatharth Saraf",
                "Geoffrey Zweig"
            ],
            "abstract": "Towards developing high-performing ASR for low-resource languages, approaches\nto address the lack of resources are to make use of data from multiple\nlanguages, and to augment the training data by creating acoustic variations. In\nthis work we present a single grapheme-based ASR model learned on 7\ngeographically proximal languages, using standard hybrid BLSTM-HMM acoustic\nmodels with lattice-free MMI objective. We build the single ASR grapheme set\nvia taking the union over each language-specific grapheme set, and we find such\nmultilingual graphemic hybrid ASR model can perform language-independent\nrecognition on all 7 languages, and substantially outperform each monolingual\nASR model. Secondly, we evaluate the efficacy of multiple data augmentation\nalternatives within language, as well as their complementarity with\nmultilingual modeling. Overall, we show that the proposed multilingual\ngraphemic hybrid ASR with various data augmentation can not only recognize any\nwithin training set languages, but also provide large ASR performance\nimprovements.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06532v1",
            "date": "2019-09-14 04:43:32+00:00",
            "title": "Bootstrapping non-parallel voice conversion from speaker-adaptive text-to-speech",
            "authors": [
                "Hieu-Thi Luong",
                "Junichi Yamagishi"
            ],
            "abstract": "Voice conversion (VC) and text-to-speech (TTS) are two tasks that share a\nsimilar objective, generating speech with a target voice. However, they are\nusually developed independently under vastly different frameworks. In this\npaper, we propose a methodology to bootstrap a VC system from a pretrained\nspeaker-adaptive TTS model and unify the techniques as well as the\ninterpretations of these two tasks. Moreover by offloading the heavy data\ndemand to the training stage of the TTS model, our VC system can be built using\na small amount of target speaker speech data. It also opens up the possibility\nof using speech in a foreign unseen language to build the system. Our\nsubjective evaluations show that the proposed framework is able to not only\nachieve competitive performance in the standard intra-language scenario but\nalso adapt and convert using speech utterances in an unseen language.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06563v2",
            "date": "2019-09-14 09:16:05+00:00",
            "title": "Multi-view and Multi-source Transfers in Neural Topic Modeling with Pretrained Topic and Word Embeddings",
            "authors": [
                "Pankaj Gupta",
                "Yatin Chaudhary",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Though word embeddings and topics are complementary representations, several\npast works have only used pre-trained word embeddings in (neural) topic\nmodeling to address data sparsity problem in short text or small collection of\ndocuments. However, no prior work has employed (pre-trained latent) topics in\ntransfer learning paradigm. In this paper, we propose an approach to (1)\nperform knowledge transfer using latent topics obtained from a large source\ncorpus, and (2) jointly transfer knowledge via the two representations (or\nviews) in neural topic modeling to improve topic quality, better deal with\npolysemy and data sparsity issues in a target corpus. In doing so, we first\naccumulate topics and word representations from one or many source corpora to\nbuild a pool of topics and word vectors. Then, we identify one or multiple\nrelevant source domain(s) and take advantage of corresponding topics and word\nfeatures via the respective pools to guide meaningful learning in the sparse\ntarget domain. We quantify the quality of topic and document representations\nvia generalization (perplexity), interpretability (topic coherence) and\ninformation retrieval (IR) using short-text, long-text, small and large\ndocument collections from news and medical domains. We have demonstrated the\nstate-of-the-art results on topic modeling with the proposed framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06614v2",
            "date": "2019-09-14 15:40:27+00:00",
            "title": "Integrating Source-channel and Attention-based Sequence-to-sequence Models for Speech Recognition",
            "authors": [
                "Qiujia Li",
                "Chao Zhang",
                "Philip C. Woodland"
            ],
            "abstract": "This paper proposes a novel automatic speech recognition (ASR) framework\ncalled Integrated Source-Channel and Attention (ISCA) that combines the\nadvantages of traditional systems based on the noisy source-channel model (SC)\nand end-to-end style systems using attention-based sequence-to-sequence models.\nThe traditional SC system framework includes hidden Markov models and\nconnectionist temporal classification (CTC) based acoustic models, language\nmodels (LMs), and a decoding procedure based on a lexicon, whereas the\nend-to-end style attention-based system jointly models the whole process with a\nsingle model. By rescoring the hypotheses produced by traditional systems using\nend-to-end style systems based on an extended noisy source-channel model, ISCA\nallows structured knowledge to be easily incorporated via the SC-based model\nwhile exploiting the complementarity of the attention-based model. Experiments\non the AMI meeting corpus show that ISCA is able to give a relative word error\nrate reduction up to 21% over an individual system, and by 13% over an\nalternative method which also involves combining CTC and attention-based\nmodels.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.06618v1",
            "date": "2019-09-14 16:03:49+00:00",
            "title": "Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Using data-driven models for solving text summarization or similar tasks has\nbecome very common in the last years. Yet most of the studies report basic\naccuracy scores only, and nothing is known about the ability of the proposed\nmodels to improve when trained on more data. In this paper, we define and\npropose three data efficiency metrics: data score efficiency, data time\ndeficiency and overall data efficiency. We also propose a simple scheme that\nuses those metrics and apply it for a more comprehensive evaluation of popular\nmethods on text summarization and title generation tasks. For the latter task,\nwe process and release a huge collection of 35 million abstract-title pairs\nfrom scientific articles. Our results reveal that among the tested models, the\nTransformer is the most efficient on both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06635v1",
            "date": "2019-09-14 17:25:03+00:00",
            "title": "Joint Wasserstein Autoencoders for Aligning Multimodal Embeddings",
            "authors": [
                "Shweta Mahajan",
                "Teresa Botschen",
                "Iryna Gurevych",
                "Stefan Roth"
            ],
            "abstract": "One of the key challenges in learning joint embeddings of multiple\nmodalities, e.g. of images and text, is to ensure coherent cross-modal\nsemantics that generalize across datasets. We propose to address this through\njoint Gaussian regularization of the latent representations. Building on\nWasserstein autoencoders (WAEs) to encode the input in each domain, we enforce\nthe latent embeddings to be similar to a Gaussian prior that is shared across\nthe two domains, ensuring compatible continuity of the encoded semantic\nrepresentations of images and texts. Semantic alignment is achieved through\nsupervision from matching image-text pairs. To show the benefits of our\nsemi-supervised representation, we apply it to cross-modal retrieval and phrase\nlocalization. We not only achieve state-of-the-art accuracy, but significantly\nbetter generalization across datasets, owing to the semantic continuity of the\nlatent space.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.06654v1",
            "date": "2019-09-14 18:52:47+00:00",
            "title": "musicnn: Pre-trained convolutional neural networks for music audio tagging",
            "authors": [
                "Jordi Pons",
                "Xavier Serra"
            ],
            "abstract": "Pronounced as \"musician\", the musicnn library contains a set of pre-trained\nmusically motivated convolutional neural networks for music audio tagging:\nhttps://github.com/jordipons/musicnn. This repository also includes some\npre-trained vgg-like baselines. These models can be used as out-of-the-box\nmusic audio taggers, as music feature extractors, or as pre-trained models for\ntransfer learning.\n  We also provide the code to train the aforementioned models:\nhttps://github.com/jordipons/musicnn-training. This framework also allows\nimplementing novel models. For example, a musically motivated convolutional\nneural network with an attention-based output layer (instead of the temporal\npooling layer) can achieve state-of-the-art results for music audio tagging:\n90.77 ROC-AUC / 38.61 PR-AUC on the MagnaTagATune dataset --- and 88.81 ROC-AUC\n/ 31.51 PR-AUC on the Million Song Dataset.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1909.06695v1",
            "date": "2019-09-14 23:21:56+00:00",
            "title": "Ouroboros: On Accelerating Training of Transformer-Based Language Models",
            "authors": [
                "Qian Yang",
                "Zhouyuan Huo",
                "Wenlin Wang",
                "Heng Huang",
                "Lawrence Carin"
            ],
            "abstract": "Language models are essential for natural language processing (NLP) tasks,\nsuch as machine translation and text summarization. Remarkable performance has\nbeen demonstrated recently across many NLP domains via a Transformer-based\nlanguage model with over a billion parameters, verifying the benefits of model\nsize. Model parallelism is required if a model is too large to fit in a single\ncomputing device. Current methods for model parallelism either suffer from\nbackward locking in backpropagation or are not applicable to language models.\nWe propose the first model-parallel algorithm that speeds the training of\nTransformer-based language models. We also prove that our proposed algorithm is\nguaranteed to converge to critical points for non-convex problems. Extensive\nexperiments on Transformer and Transformer-XL language models demonstrate that\nthe proposed algorithm obtains a much faster speedup beyond data parallelism,\nwith comparable or better accuracy. Code to reproduce experiments is to be\nfound at \\url{https://github.com/LaraQianYang/Ouroboros}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06708v1",
            "date": "2019-09-15 01:39:59+00:00",
            "title": "Hint-Based Training for Non-Autoregressive Machine Translation",
            "authors": [
                "Zhuohan Li",
                "Zi Lin",
                "Di He",
                "Fei Tian",
                "Tao Qin",
                "Liwei Wang",
                "Tie-Yan Liu"
            ],
            "abstract": "Due to the unparallelizable nature of the autoregressive factorization,\nAutoRegressive Translation (ART) models have to generate tokens sequentially\nduring decoding and thus suffer from high inference latency. Non-AutoRegressive\nTranslation (NART) models were proposed to reduce the inference time, but could\nonly achieve inferior translation accuracy. In this paper, we proposed a novel\napproach to leveraging the hints from hidden states and word alignments to help\nthe training of NART models. The results achieve significant improvement over\nprevious NART models for the WMT14 En-De and De-En datasets and are even\ncomparable to a strong LSTM-based ART baseline but one order of magnitude\nfaster in inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.06845v2",
            "date": "2019-09-15 17:40:57+00:00",
            "title": "Quantum Structure in Cognition: Human Language as a Boson Gas of Entangled Words",
            "authors": [
                "Diederik Aerts",
                "Lester Beltran"
            ],
            "abstract": "We model a piece of text of human language telling a story by means of the\nquantum structure describing a Bose gas in a state close to a Bose-Einstein\ncondensate near absolute zero temperature. For this we introduce energy levels\nfor the words (concepts) used in the story and we also introduce the new notion\nof 'cogniton' as the quantum of human thought. Words (concepts) are then\ncognitons in different energy states as it is the case for photons in different\nenergy states, or states of different radiative frequency, when the considered\nboson gas is that of the quanta of the electromagnetic field. We show that\nBose-Einstein statistics delivers a very good model for these pieces of texts\ntelling stories, both for short stories and for long stories of the size of\nnovels. We analyze an unexpected connection with Zipf's law in human language,\nthe Zipf ranking relating to the energy levels of the words, and the\nBose-Einstein graph coinciding with the Zipf graph. We investigate the issue of\n'identity and indistinguishability' from this new perspective and conjecture\nthat the way one can easily understand how two of 'the same concepts' are\n'absolutely identical and indistinguishable' in human language is also the way\nin which quantum particles are absolutely identical and indistinguishable in\nphysical reality, providing in this way new evidence for our conceptuality\ninterpretation of quantum theory.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1909.07063v2",
            "date": "2019-09-16 08:46:30+00:00",
            "title": "Global Autoregressive Models for Data-Efficient Sequence Learning",
            "authors": [
                "Tetiana Parshakova",
                "Jean-Marc Andreoli",
                "Marc Dymetman"
            ],
            "abstract": "Standard autoregressive seq2seq models are easily trained by max-likelihood,\nbut tend to show poor results under small-data conditions. We introduce a class\nof seq2seq models, GAMs (Global Autoregressive Models), which combine an\nautoregressive component with a log-linear component, allowing the use of\nglobal \\textit{a priori} features to compensate for lack of data. We train\nthese models in two steps. In the first step, we obtain an \\emph{unnormalized}\nGAM that maximizes the likelihood of the data, but is improper for fast\ninference or evaluation. In the second step, we use this GAM to train (by\ndistillation) a second autoregressive model that approximates the\n\\emph{normalized} distribution associated with the GAM, and can be used for\nfast inference and evaluation. Our experiments focus on language modelling\nunder synthetic conditions and show a strong perplexity reduction of using the\nsecond autoregressive model over the standard one.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.07083v2",
            "date": "2019-09-16 09:29:52+00:00",
            "title": "Controllable Text-to-Image Generation",
            "authors": [
                "Bowen Li",
                "Xiaojuan Qi",
                "Thomas Lukasiewicz",
                "Philip H. S. Torr"
            ],
            "abstract": "In this paper, we propose a novel controllable text-to-image generative\nadversarial network (ControlGAN), which can effectively synthesise high-quality\nimages and also control parts of the image generation according to natural\nlanguage descriptions. To achieve this, we introduce a word-level spatial and\nchannel-wise attention-driven generator that can disentangle different visual\nattributes, and allow the model to focus on generating and manipulating\nsubregions corresponding to the most relevant words. Also, a word-level\ndiscriminator is proposed to provide fine-grained supervisory feedback by\ncorrelating words with image regions, facilitating training an effective\ngenerator which is able to manipulate specific visual attributes without\naffecting the generation of other content. Furthermore, perceptual loss is\nadopted to reduce the randomness involved in the image generation, and to\nencourage the generator to manipulate specific attributes required in the\nmodified text. Extensive experiments on benchmark datasets demonstrate that our\nmethod outperforms existing state of the art, and is able to effectively\nmanipulate synthetic images using natural language descriptions. Code is\navailable at https://github.com/mrlibw/ControlGAN.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.07158v3",
            "date": "2019-09-16 12:43:17+00:00",
            "title": "Prediction Uncertainty Estimation for Hate Speech Classification",
            "authors": [
                "Kristian Miok",
                "Dong Nguyen-Doan",
                "Bla\u017e \u0160krlj",
                "Daniela Zaharie",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "As a result of social network popularity, in recent years, hate speech\nphenomenon has significantly increased. Due to its harmful effect on minority\ngroups as well as on large communities, there is a pressing need for hate\nspeech detection and filtering. However, automatic approaches shall not\njeopardize free speech, so they shall accompany their decisions with\nexplanations and assessment of uncertainty. Thus, there is a need for\npredictive machine learning models that not only detect hate speech but also\nhelp users understand when texts cross the line and become unacceptable. The\nreliability of predictions is usually not addressed in text classification. We\nfill this gap by proposing the adaptation of deep neural networks that can\nefficiently estimate prediction uncertainty. To reliably detect hate speech, we\nuse Monte Carlo dropout regularization, which mimics Bayesian inference within\nneural networks. We evaluate our approach using different text embedding\nmethods. We visualize the reliability of results with a novel technique that\naids in understanding the classification reliability and errors.",
            "journal ref": "Statistical Language and Speech Processing 2019 Proceedings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.07502v2",
            "date": "2019-09-16 22:21:27+00:00",
            "title": "Automatic Detection and Classification of Cognitive Distortions in Mental Health Text",
            "authors": [
                "Benjamin Shickel",
                "Scott Siegel",
                "Martin Heesacker",
                "Sherry Benton",
                "Parisa Rashidi"
            ],
            "abstract": "In cognitive psychology, automatic and self-reinforcing irrational thought\npatterns are known as cognitive distortions. Left unchecked, patients\nexhibiting these types of thoughts can become stuck in negative feedback loops\nof unhealthy thinking, leading to inaccurate perceptions of reality commonly\nassociated with anxiety and depression. In this paper, we present a machine\nlearning framework for the automatic detection and classification of 15 common\ncognitive distortions in two novel mental health free text datasets collected\nfrom both crowdsourcing and a real-world online therapy program. When\ndifferentiating between distorted and non-distorted passages, our model\nachieved a weighted F1 score of 0.88. For classifying distorted passages into\none of 15 distortion categories, our model yielded weighted F1 scores of 0.68\nin the larger crowdsourced dataset and 0.45 in the smaller online counseling\ndataset, both of which outperformed random baseline metrics by a large margin.\nFor both tasks, we also identified the most discriminative words and phrases\nbetween classes to highlight common thematic elements for improving targeted\nand therapist-guided mental health treatment. Furthermore, we performed an\nexploratory analysis using unsupervised content-based clustering and topic\nmodeling algorithms as first efforts towards a data-driven perspective on the\nthematic relationship between similar cognitive distortions traditionally\ndeemed unique. Finally, we highlight the difficulties in applying mental\nhealth-based machine learning in a real-world setting and comment on the\nimplications and benefits of our framework for improving automated delivery of\ntherapeutic treatment in conjunction with traditional cognitive-behavioral\ntherapy.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1909.07523v1",
            "date": "2019-09-16 23:53:59+00:00",
            "title": "Discovering Differential Features: Adversarial Learning for Information Credibility Evaluation",
            "authors": [
                "Lianwei Wu",
                "Yuan Rao",
                "Ambreen Nazir",
                "Haolin Jin"
            ],
            "abstract": "A series of deep learning approaches extract a large number of credibility\nfeatures to detect fake news on the Internet. However, these extracted features\nstill suffer from many irrelevant and noisy features that restrict severely the\nperformance of the approaches. In this paper, we propose a novel model based on\nAdversarial Networks and inspirited by the Shared-Private model (ANSP), which\naims at reducing common, irrelevant features from the extracted features for\ninformation credibility evaluation. Specifically, ANSP involves two tasks: one\nis to prevent the binary classification of true and false information for\ncapturing common features relying on adversarial networks guided by\nreinforcement learning. Another extracts credibility features (henceforth,\nprivate features) from multiple types of credibility information and compares\nwith the common features through two strategies, i.e., orthogonality\nconstraints and KL-divergence for making the private features more\ndifferential. Experiments first on two six-label LIAR and Weibo datasets\ndemonstrate that ANSP achieves the state-of-the-art performance, boosting the\naccuracy by 2.1%, 3.1%, respectively and then on four-label Twitter16 validate\nthe robustness of the model with 1.8% performance improvements.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1909.07583v2",
            "date": "2019-09-17 04:41:12+00:00",
            "title": "Inverse Visual Question Answering with Multi-Level Attentions",
            "authors": [
                "Yaser Alwattar",
                "Yuhong Guo"
            ],
            "abstract": "In this paper, we propose a novel deep multi-level attention model to address\ninverse visual question answering. The proposed model generates regional visual\nand semantic features at the object level and then enhances them with the\nanswer cue by using attention mechanisms. Two levels of multiple attentions are\nemployed in the model, including the dual attention at the partial question\nencoding step and the dynamic attention at the next question word generation\nstep. We evaluate the proposed model on the VQA V1 dataset. It demonstrates\nstate-of-the-art performance in terms of multiple commonly used metrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.07746v2",
            "date": "2019-09-17 12:23:33+00:00",
            "title": "Multi Sense Embeddings from Topic Models",
            "authors": [
                "Shobhit Jain",
                "Sravan Babu Bodapati",
                "Ramesh Nallapati",
                "Anima Anandkumar"
            ],
            "abstract": "Distributed word embeddings have yielded state-of-the-art performance in many\nNLP tasks, mainly due to their success in capturing useful semantic\ninformation. These representations assign only a single vector to each word\nwhereas a large number of words are polysemous (i.e., have multiple meanings).\nIn this work, we approach this critical problem in lexical semantics, namely\nthat of representing various senses of polysemous words in vector spaces. We\npropose a topic modeling based skip-gram approach for learning multi-prototype\nword embeddings. We also introduce a method to prune the embeddings determined\nby the probabilistic representation of the word in each topic. We use our\nembeddings to show that they can capture the context and word similarity\nstrongly and outperform various state-of-the-art implementations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.07873v1",
            "date": "2019-09-17 15:05:31+00:00",
            "title": "Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model",
            "authors": [
                "Prashanth Vijayaraghavan",
                "Deb Roy"
            ],
            "abstract": "Recently, generating adversarial examples has become an important means of\nmeasuring robustness of a deep learning model. Adversarial examples help us\nidentify the susceptibilities of the model and further counter those\nvulnerabilities by applying adversarial training techniques. In natural\nlanguage domain, small perturbations in the form of misspellings or paraphrases\ncan drastically change the semantics of the text. We propose a reinforcement\nlearning based approach towards generating adversarial examples in black-box\nsettings. We demonstrate that our method is able to fool well-trained models\nfor (a) IMDB sentiment classification task and (b) AG's news corpus news\ncategorization task with significantly high success rates. We find that the\nadversarial examples generated are semantics-preserving perturbations to the\noriginal text.",
            "journal ref": "Joint European Conference on Machine Learning and Knowledge\n  Discovery in Databases. Springer, Cham, 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.07881v1",
            "date": "2019-09-17 15:14:51+00:00",
            "title": "Estimating Glycemic Impact of Cooking Recipes via Online Crowdsourcing and Machine Learning",
            "authors": [
                "Helena Lee",
                "Palakorn Achananuparp",
                "Yue Liu",
                "Ee-Peng Lim",
                "Lav R. Varshney"
            ],
            "abstract": "Consumption of diets with low glycemic impact is highly recommended for\ndiabetics and pre-diabetics as it helps maintain their blood glucose levels.\nHowever, laboratory analysis of dietary glycemic potency is time-consuming and\nexpensive. In this paper, we explore a data-driven approach utilizing online\ncrowdsourcing and machine learning to estimate the glycemic impact of cooking\nrecipes. We show that a commonly used healthiness metric may not always be\neffective in determining recipes suitable for diabetics, thus emphasizing the\nimportance of the glycemic-impact estimation task. Our best classification\nmodel, trained on nutritional and crowdsourced data obtained from Amazon\nMechanical Turk (AMT), can accurately identify recipes which are unhealthful\nfor diabetics.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1909.08041v1",
            "date": "2019-09-17 19:21:11+00:00",
            "title": "Revealing the Importance of Semantic Retrieval for Machine Reading at Scale",
            "authors": [
                "Yixin Nie",
                "Songhe Wang",
                "Mohit Bansal"
            ],
            "abstract": "Machine Reading at Scale (MRS) is a challenging task in which a system is\ngiven an input query and is asked to produce a precise output by \"reading\"\ninformation from a large knowledge base. The task has gained popularity with\nits natural combination of information retrieval (IR) and machine comprehension\n(MC). Advancements in representation learning have led to separated progress in\nboth IR and MC; however, very few studies have examined the relationship and\ncombined design of retrieval and comprehension at different levels of\ngranularity, for development of MRS systems. In this work, we give general\nguidelines on system design for MRS by proposing a simple yet effective\npipeline system with special consideration on hierarchical semantic retrieval\nat both paragraph and sentence level, and their potential effects on the\ndownstream task. The system is evaluated on both fact verification and\nopen-domain multihop QA, achieving state-of-the-art results on the leaderboard\ntest sets of both FEVER and HOTPOTQA. To further demonstrate the importance of\nsemantic retrieval, we present ablation and analysis studies to quantify the\ncontribution of neural retrieval modules at both paragraph-level and\nsentence-level, and illustrate that intermediate semantic retrieval modules are\nvital for not only effectively filtering upstream information and thus saving\ndownstream computation, but also for shaping upstream data distribution and\nproviding better data for downstream modeling. Code/data made publicly\navailable at: https://github.com/easonnie/semanticRetrievalMRS",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08079v1",
            "date": "2019-09-17 20:29:57+00:00",
            "title": "Relaxed Softmax for learning from Positive and Unlabeled data",
            "authors": [
                "Ugo Tanielian",
                "Flavian Vasile"
            ],
            "abstract": "In recent years, the softmax model and its fast approximations have become\nthe de-facto loss functions for deep neural networks when dealing with\nmulti-class prediction. This loss has been extended to language modeling and\nrecommendation, two fields that fall into the framework of learning from\nPositive and Unlabeled data. In this paper, we stress the different drawbacks\nof the current family of softmax losses and sampling schemes when applied in a\nPositive and Unlabeled learning setup. We propose both a Relaxed Softmax loss\n(RS) and a new negative sampling scheme based on Boltzmann formulation. We show\nthat the new training objective is better suited for the tasks of density\nestimation, item similarity and next-event prediction by driving uplifts in\nperformance on textual and recommendation datasets against classical softmax.",
            "journal ref": "RecSys 2019 Proceedings of the 13th ACM Conference on Recommender\n  Systems",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1909.08103v1",
            "date": "2019-09-17 21:12:11+00:00",
            "title": "Simultaneous Speech Recognition and Speaker Diarization for Monaural Dialogue Recordings with Target-Speaker Acoustic Models",
            "authors": [
                "Naoyuki Kanda",
                "Shota Horiguchi",
                "Yusuke Fujita",
                "Yawen Xue",
                "Kenji Nagamatsu",
                "Shinji Watanabe"
            ],
            "abstract": "This paper investigates the use of target-speaker automatic speech\nrecognition (TS-ASR) for simultaneous speech recognition and speaker\ndiarization of single-channel dialogue recordings. TS-ASR is a technique to\nautomatically extract and recognize only the speech of a target speaker given a\nshort sample utterance of that speaker. One obvious drawback of TS-ASR is that\nit cannot be used when the speakers in the recordings are unknown because it\nrequires a sample of the target speakers in advance of decoding. To remove this\nlimitation, we propose an iterative method, in which (i) the estimation of\nspeaker embeddings and (ii) TS-ASR based on the estimated speaker embeddings\nare alternately executed. We evaluated the proposed method by using very\nchallenging dialogue recordings in which the speaker overlap ratio was over\n20%. We confirmed that the proposed method significantly reduced both the word\nerror rate (WER) and diarization error rate (DER). Our proposed method combined\nwith i-vector speaker embeddings ultimately achieved a WER that differed by\nonly 2.1 % from that of TS-ASR given oracle speaker embeddings. Furthermore,\nour method can solve speaker diarization simultaneously as a by-product and\nachieved better DER than that of the conventional clustering-based speaker\ndiarization method based on i-vector.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08167v1",
            "date": "2019-09-18 02:03:03+00:00",
            "title": "Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis",
            "authors": [
                "Minlong Peng",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "Cross-domain sentiment analysis is currently a hot topic in the research and\nengineering areas. One of the most popular frameworks in this field is the\ndomain-invariant representation learning (DIRL) paradigm, which aims to learn a\ndistribution-invariant feature representation across domains. However, in this\nwork, we find out that applying DIRL may harm domain adaptation when the label\ndistribution $\\rm{P}(\\rm{Y})$ changes across domains. To address this problem,\nwe propose a modification to DIRL, obtaining a novel weighted domain-invariant\nrepresentation learning (WDIRL) framework. We show that it is easy to transfer\nexisting SOTA DIRL models to WDIRL. Empirical studies on extensive cross-domain\nsentiment analysis tasks verified our statements and showed the effectiveness\nof our proposed solution.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.08187v1",
            "date": "2019-09-17 05:27:45+00:00",
            "title": "Learning to Generate Questions with Adaptive Copying Neural Networks",
            "authors": [
                "Xinyuan Lu",
                "Yuhong Guo"
            ],
            "abstract": "Automatic question generation is an important problem in natural language\nprocessing. In this paper we propose a novel adaptive copying recurrent neural\nnetwork model to tackle the problem of question generation from sentences and\nparagraphs. The proposed model adds a copying mechanism component onto a\nbidirectional LSTM architecture to generate more suitable questions adaptively\nfrom the input data. Our experimental results show the proposed model can\noutperform the state-of-the-art question generation methods in terms of BLEU\nand ROUGE evaluation scores.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.08191v1",
            "date": "2019-09-17 04:32:00+00:00",
            "title": "Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space",
            "authors": [
                "Hung Nghiep Tran",
                "Atsuhiro Takasu"
            ],
            "abstract": "The trends of open science have enabled several open scholarly datasets which\ninclude millions of papers and authors. Managing, exploring, and utilizing such\nlarge and complicated datasets effectively are challenging. In recent years,\nthe knowledge graph has emerged as a universal data format for representing\nknowledge about heterogeneous entities and their relationships. The knowledge\ngraph can be modeled by knowledge graph embedding methods, which represent\nentities and relations as embedding vectors in semantic space, then model the\ninteractions between these embedding vectors. However, the semantic structures\nin the knowledge graph embedding space are not well-studied, thus knowledge\ngraph embedding methods are usually only used for knowledge graph completion\nbut not data representation and analysis. In this paper, we propose to analyze\nthese semantic structures based on the well-studied word embedding space and\nuse them to support data exploration. We also define the semantic queries,\nwhich are algebraic operations between the embedding vectors in the knowledge\ngraph embedding space, to solve queries such as similarity and analogy between\nthe entities on the original datasets. We then design a general framework for\ndata exploration by semantic queries and discuss the solution to some\ntraditional scholarly data exploration tasks. We also propose some new\ninteresting tasks that can be solved based on the uncanny semantic structures\nof the embedding space.",
            "journal ref": "International Conference on Theory and Practice of Digital\n  Libraries (TPDL 2019)",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.08314v1",
            "date": "2019-09-18 09:39:14+00:00",
            "title": "Memory-Augmented Neural Networks for Machine Translation",
            "authors": [
                "Mark Collier",
                "Joeran Beel"
            ],
            "abstract": "Memory-augmented neural networks (MANNs) have been shown to outperform other\nrecurrent neural network architectures on a series of artificial sequence\nlearning tasks, yet they have had limited application to real-world tasks. We\nevaluate direct application of Neural Turing Machines (NTM) and Differentiable\nNeural Computers (DNC) to machine translation. We further propose and evaluate\ntwo models which extend the attentional encoder-decoder with capabilities\ninspired by memory augmented neural networks. We evaluate our proposed models\non IWSLT Vietnamese to English and ACL Romanian to English datasets. Our\nproposed models and the memory augmented neural networks perform similarly to\nthe attentional encoder-decoder on the Vietnamese to English translation task\nwhile have a 0.3-1.9 lower BLEU score for the Romanian to English task.\nInterestingly, our analysis shows that despite being equipped with additional\nflexibility and being randomly initialized memory augmented neural networks\nlearn an algorithm for machine translation almost identical to the attentional\nencoder-decoder.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.08402v1",
            "date": "2019-09-18 12:40:39+00:00",
            "title": "Enriching BERT with Knowledge Graph Embeddings for Document Classification",
            "authors": [
                "Malte Ostendorff",
                "Peter Bourgonje",
                "Maria Berger",
                "Julian Moreno-Schneider",
                "Georg Rehm",
                "Bela Gipp"
            ],
            "abstract": "In this paper, we focus on the classification of books using short\ndescriptive texts (cover blurbs) and additional metadata. Building upon BERT, a\ndeep neural language model, we demonstrate how to combine text representations\nwith metadata and knowledge graph embeddings, which encode author information.\nCompared to the standard BERT approach we achieve considerably better results\nfor the classification task. For a more coarse-grained classification using\neight labels we achieve an F1- score of 87.20, while a detailed classification\nusing 343 labels yields an F1-score of 64.70. We make the source code and\ntrained models of our experiments publicly available",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08593v2",
            "date": "2019-09-18 17:33:39+00:00",
            "title": "Fine-Tuning Language Models from Human Preferences",
            "authors": [
                "Daniel M. Ziegler",
                "Nisan Stiennon",
                "Jeffrey Wu",
                "Tom B. Brown",
                "Alec Radford",
                "Dario Amodei",
                "Paul Christiano",
                "Geoffrey Irving"
            ],
            "abstract": "Reward learning enables the application of reinforcement learning (RL) to\ntasks where reward is defined by human judgment, building a model of reward by\nasking humans questions. Most work on reward learning has used simulated\nenvironments, but complex information about values is often expressed in\nnatural language, and we believe reward learning for language is a key to\nmaking RL practical and safe for real-world tasks. In this paper, we build on\nadvances in generative pretraining of language models to apply reward learning\nto four natural language tasks: continuing text with positive sentiment or\nphysically descriptive language, and summarization tasks on the TL;DR and\nCNN/Daily Mail datasets. For stylistic continuation we achieve good results\nwith only 5,000 comparisons evaluated by humans. For summarization, models\ntrained with 60,000 comparisons copy whole sentences from the input but skip\nirrelevant preamble; this leads to reasonable ROUGE scores and very good\nperformance according to our human labelers, but may be exploiting the fact\nthat labelers rely on simple heuristics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08663v2",
            "date": "2019-09-18 19:02:53+00:00",
            "title": "Do We Need Neural Models to Explain Human Judgments of Acceptability?",
            "authors": [
                "Wang Jing",
                "M. A. Kelly",
                "David Reitter"
            ],
            "abstract": "Native speakers can judge whether a sentence is an acceptable instance of\ntheir language. Acceptability provides a means of evaluating whether\ncomputational language models are processing language in a human-like manner.\nWe test the ability of computational language models, simple language features,\nand word embeddings to predict native English speakers judgments of\nacceptability on English-language essays written by non-native speakers. We\nfind that much of the sentence acceptability variance can be captured by a\ncombination of features including misspellings, word order, and word similarity\n(Pearson's r = 0.494). While predictive neural models fit acceptability\njudgments well (r = 0.527), we find that a 4-gram model with statistical\nsmoothing is just as good (r = 0.528). Thanks to incorporating a count of\nmisspellings, our 4-gram model surpasses both the previous unsupervised\nstate-of-the art (Lau et al., 2015; r = 0.472), and the average non-expert\nnative speaker (r = 0.46). Our results demonstrate that acceptability is well\ncaptured by n-gram statistics and simple language features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08723v3",
            "date": "2019-09-18 22:05:05+00:00",
            "title": "Espresso: A Fast End-to-end Neural Speech Recognition Toolkit",
            "authors": [
                "Yiming Wang",
                "Tongfei Chen",
                "Hainan Xu",
                "Shuoyang Ding",
                "Hang Lv",
                "Yiwen Shao",
                "Nanyun Peng",
                "Lei Xie",
                "Shinji Watanabe",
                "Sanjeev Khudanpur"
            ],
            "abstract": "We present Espresso, an open-source, modular, extensible end-to-end neural\nautomatic speech recognition (ASR) toolkit based on the deep learning library\nPyTorch and the popular neural machine translation toolkit fairseq. Espresso\nsupports distributed training across GPUs and computing nodes, and features\nvarious decoding approaches commonly employed in ASR, including look-ahead\nword-based language model fusion, for which a fast, parallelized decoder is\nimplemented. Espresso achieves state-of-the-art ASR performance on the WSJ,\nLibriSpeech, and Switchboard data sets among other end-to-end systems without\ndata augmentation, and is 4--11x faster for decoding than similar systems (e.g.\nESPnet).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08752v3",
            "date": "2019-09-19 00:47:13+00:00",
            "title": "Summary Level Training of Sentence Rewriting for Abstractive Summarization",
            "authors": [
                "Sanghwan Bae",
                "Taeuk Kim",
                "Jihoon Kim",
                "Sang-goo Lee"
            ],
            "abstract": "As an attempt to combine extractive and abstractive summarization, Sentence\nRewriting models adopt the strategy of extracting salient sentences from a\ndocument first and then paraphrasing the selected ones to generate a summary.\nHowever, the existing models in this framework mostly rely on sentence-level\nrewards or suboptimal labels, causing a mismatch between a training objective\nand evaluation metric. In this paper, we present a novel training signal that\ndirectly maximizes summary-level ROUGE scores through reinforcement learning.\nIn addition, we incorporate BERT into our model, making good use of its ability\non natural language understanding. In extensive experiments, we show that a\ncombination of our proposed model and training procedure obtains new\nstate-of-the-art performance on both CNN/Daily Mail and New York Times\ndatasets. We also demonstrate that it generalizes better on DUC-2002 test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08782v1",
            "date": "2019-09-19 02:50:23+00:00",
            "title": "Large-scale representation learning from visually grounded untranscribed speech",
            "authors": [
                "Gabriel Ilharco",
                "Yuan Zhang",
                "Jason Baldridge"
            ],
            "abstract": "Systems that can associate images with their spoken audio captions are an\nimportant step towards visually grounded language learning. We describe a\nscalable method to automatically generate diverse audio for image captioning\ndatasets. This supports pretraining deep networks for encoding both audio and\nimages, which we do via a dual encoder that learns to align latent\nrepresentations from both modalities. We show that a masked margin softmax loss\nfor such models is superior to the standard triplet loss. We fine-tune these\nmodels on the Flickr8k Audio Captions Corpus and obtain state-of-the-art\nresults---improving recall in the top 10 from 29.6% to 49.5%. We also obtain\nhuman ratings on retrieval outputs to better assess the impact of incidentally\nmatching image-caption pairs that were not associated in the data, finding that\nautomatic evaluation substantially underestimates the quality of the retrieved\nresults.",
            "journal ref": "The SIGNLL Conference on Computational Natural Language Learning\n  (CoNLL), 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.08824v3",
            "date": "2019-09-19 06:46:02+00:00",
            "title": "Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder",
            "authors": [
                "Li Du",
                "Xiao Ding",
                "Ting Liu",
                "Zhongyang Li"
            ],
            "abstract": "Understanding event and event-centered commonsense reasoning are crucial for\nnatural language processing (NLP). Given an observed event, it is trivial for\nhuman to infer its intents and effects, while this type of If-Then reasoning\nstill remains challenging for NLP systems. To facilitate this, a If-Then\ncommonsense reasoning dataset Atomic is proposed, together with an RNN-based\nSeq2Seq model to conduct such reasoning. However, two fundamental problems\nstill need to be addressed: first, the intents of an event may be multiple,\nwhile the generations of RNN-based Seq2Seq models are always semantically\nclose; second, external knowledge of the event background may be necessary for\nunderstanding events and conducting the If-Then reasoning. To address these\nissues, we propose a novel context-aware variational autoencoder effectively\nlearning event background information to guide the If-Then reasoning.\nExperimental results show that our approach improves the accuracy and diversity\nof inferences compared with state-of-the-art baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08855v3",
            "date": "2019-09-19 08:25:47+00:00",
            "title": "How Additional Knowledge can Improve Natural Language Commonsense Question Answering?",
            "authors": [
                "Arindam Mitra",
                "Pratyay Banerjee",
                "Kuntal Kumar Pal",
                "Swaroop Mishra",
                "Chitta Baral"
            ],
            "abstract": "Recently several datasets have been proposed to encourage research in\nQuestion Answering domains where commonsense knowledge is expected to play an\nimportant role. Recent language models such as ROBERTA, BERT and GPT that have\nbeen pre-trained on Wikipedia articles and books have shown reasonable\nperformance with little fine-tuning on several such Multiple Choice\nQuestion-Answering (MCQ) datasets. Our goal in this work is to develop methods\nto incorporate additional (commonsense) knowledge into language model-based\napproaches for better question-answering in such domains. In this work, we\nfirst categorize external knowledge sources, and show performance does improve\non using such sources. We then explore three different strategies for knowledge\nincorporation and four different models for question-answering using external\ncommonsense knowledge. We analyze our predictions to explore the scope of\nfurther improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08863v1",
            "date": "2019-09-19 08:47:16+00:00",
            "title": "ASU at TextGraphs 2019 Shared Task: Explanation ReGeneration using Language Models and Iterative Re-Ranking",
            "authors": [
                "Pratyay Banerjee"
            ],
            "abstract": "In this work we describe the system from Natural Language Processing group at\nArizona State University for the TextGraphs 2019 Shared Task. The task focuses\non Explanation Regeneration, an intermediate step towards general multi-hop\ninference on large graphs. Our approach consists of modeling the explanation\nregeneration task as a \\textit{learning to rank} problem, for which we use\nstate-of-the-art language models and explore dataset preparation techniques. We\nutilize an iterative re-ranking based approach to further improve the rankings.\nOur system secured 2nd rank in the task with a mean average precision (MAP) of\n41.3\\% on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.08975v1",
            "date": "2019-09-19 13:24:29+00:00",
            "title": "Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment",
            "authors": [
                "Jaap Jumelet",
                "Willem Zuidema",
                "Dieuwke Hupkes"
            ],
            "abstract": "Extensive research has recently shown that recurrent neural language models\nare able to process a wide range of grammatical phenomena. How these models are\nable to perform these remarkable feats so well, however, is still an open\nquestion. To gain more insight into what information LSTMs base their decisions\non, we propose a generalisation of Contextual Decomposition (GCD). In\nparticular, this setup enables us to accurately distil which part of a\nprediction stems from semantic heuristics, which part truly emanates from\nsyntactic cues and which part arise from the model biases themselves instead.\nWe investigate this technique on tasks pertaining to syntactic agreement and\nco-reference resolution and discover that the model strongly relies on a\ndefault reasoning effect to perform these tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09018v1",
            "date": "2019-09-18 10:07:01+00:00",
            "title": "Corporate IT-support Help-Desk Process Hybrid-Automation Solution with Machine Learning Approach",
            "authors": [
                "Kuruparan Shanmugalingam",
                "Nisal Chandrasekara",
                "Calvin Hindle",
                "Gihan Fernando",
                "Chanaka Gunawardhana"
            ],
            "abstract": "Comprehensive IT support teams in large scale organizations require more man\npower for handling engagement and requests of employees from different channels\non a 24*7 basis. Automated email technical queries help desk is proposed to\nhave instant real-time quick solutions and email categorisation. Email topic\nmodelling with various machine learning, deep-learning approaches are compared\nwith different features for a scalable, generalised solution along with\nsure-shot static rules. Email's title, body, attachment, OCR text, and some\nfeature engineered custom features are given as input elements. XGBoost\ncascaded hierarchical models, Bi-LSTM model with word embeddings perform well\nshowing 77.3 overall accuracy For the real world corporate email data set. By\nintroducing the thresholding techniques, the overall automation system\narchitecture provides 85.6 percentage of accuracy for real world corporate\nemails. Combination of quick fixes, static rules, ML categorization as a low\ncost inference solution reduces 81 percentage of the human effort in the\nprocess of automation and real time implementation.",
            "journal ref": "The International Conference on Digital Image Computing:\n  Techniques and Applications (DICTA) 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.09070v1",
            "date": "2019-09-19 16:10:15+00:00",
            "title": "Look, Read and Enrich. Learning from Scientific Figures and their Captions",
            "authors": [
                "Jose Manuel Gomez-Perez",
                "Raul Ortega"
            ],
            "abstract": "Compared to natural images, understanding scientific figures is particularly\nhard for machines. However, there is a valuable source of information in\nscientific literature that until now has remained untapped: the correspondence\nbetween a figure and its caption. In this paper we investigate what can be\nlearnt by looking at a large number of figures and reading their captions, and\nintroduce a figure-caption correspondence learning task that makes use of our\nobservations. Training visual and language networks without supervision other\nthan pairs of unconstrained figures and captions is shown to successfully solve\nthis task. We also show that transferring lexical and semantic knowledge from a\nknowledge graph significantly enriches the resulting features. Finally, we\ndemonstrate the positive impact of such features in other tasks involving\nscientific text and figures, like multi-modal classification and machine\ncomprehension for question answering, outperforming supervised baselines and\nad-hoc approaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.09116v2",
            "date": "2019-09-19 17:42:56+00:00",
            "title": "Self-Training for End-to-End Speech Recognition",
            "authors": [
                "Jacob Kahn",
                "Ann Lee",
                "Awni Hannun"
            ],
            "abstract": "We revisit self-training in the context of end-to-end speech recognition. We\ndemonstrate that training with pseudo-labels can substantially improve the\naccuracy of a baseline model. Key to our approach are a strong baseline\nacoustic and language model used to generate the pseudo-labels, filtering\nmechanisms tailored to common errors from sequence-to-sequence models, and a\nnovel ensemble approach to increase pseudo-label diversity. Experiments on the\nLibriSpeech corpus show that with an ensemble of four models and label\nfiltering, self-training yields a 33.9% relative improvement in WER compared\nwith a baseline trained on 100 hours of labelled data in the noisy speech\nsetting. In the clean speech setting, self-training recovers 59.3% of the gap\nbetween the baseline and an oracle model, which is at least 93.8% relatively\nhigher than what previous approaches can achieve.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09192v1",
            "date": "2019-09-19 18:55:54+00:00",
            "title": "Learning Sparse Mixture of Experts for Visual Question Answering",
            "authors": [
                "Vardaan Pahuja",
                "Jie Fu",
                "Christopher J. Pal"
            ],
            "abstract": "There has been a rapid progress in the task of Visual Question Answering with\nimproved model architectures. Unfortunately, these models are usually\ncomputationally intensive due to their sheer size which poses a serious\nchallenge for deployment. We aim to tackle this issue for the specific task of\nVisual Question Answering (VQA). A Convolutional Neural Network (CNN) is an\nintegral part of the visual processing pipeline of a VQA model (assuming the\nCNN is trained along with entire VQA model). In this project, we propose an\nefficient and modular neural architecture for the VQA task with focus on the\nCNN module. Our experiments demonstrate that a sparsely activated CNN based VQA\nmodel achieves comparable performance to a standard CNN based VQA model\narchitecture.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.09268v2",
            "date": "2019-09-20 00:24:59+00:00",
            "title": "Towards Neural Language Evaluators",
            "authors": [
                "Hassan Kan\u00e9",
                "Yusuf Kocyigit",
                "Pelkins Ajanoh",
                "Ali Abdalla",
                "Mohamed Coulibali"
            ],
            "abstract": "We review three limitations of BLEU and ROUGE -- the most popular metrics\nused to assess reference summaries against hypothesis summaries, come up with\ncriteria for what a good metric should behave like and propose concrete ways to\nuse recent Transformers-based Language Models to assess reference summaries\nagainst hypothesis summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09292v1",
            "date": "2019-09-20 01:53:19+00:00",
            "title": "BERT Meets Chinese Word Segmentation",
            "authors": [
                "Haiqin Yang"
            ],
            "abstract": "Chinese word segmentation (CWS) is a fundamental task for Chinese language\nunderstanding. Recently, neural network-based models have attained superior\nperformance in solving the in-domain CWS task. Last year, Bidirectional Encoder\nRepresentation from Transformers (BERT), a new language representation model,\nhas been proposed as a backbone model for many natural language tasks and\nredefined the corresponding performance. The excellent performance of BERT\nmotivates us to apply it to solve the CWS task. By conducting intensive\nexperiments in the benchmark datasets from the second International Chinese\nWord Segmentation Bake-off, we obtain several keen observations. BERT can\nslightly improve the performance even when the datasets contain the issue of\nlabeling inconsistency. When applying sufficiently learned features, Softmax, a\nsimpler classifier, can attain the same performance as that of a more\ncomplicated classifier, e.g., Conditional Random Field (CRF). The performance\nof BERT usually increases as the model size increases. The features extracted\nby BERT can be also applied as good candidates for other neural network models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09482v1",
            "date": "2019-09-18 18:50:18+00:00",
            "title": "Language models and Automated Essay Scoring",
            "authors": [
                "Pedro Uria Rodriguez",
                "Amir Jafari",
                "Christopher M. Ormerod"
            ],
            "abstract": "In this paper, we present a new comparative study on automatic essay scoring\n(AES). The current state-of-the-art natural language processing (NLP) neural\nnetwork architectures are used in this work to achieve above human-level\naccuracy on the publicly available Kaggle AES dataset. We compare two powerful\nlanguage models, BERT and XLNet, and describe all the layers and network\narchitectures in these models. We elucidate the network architectures of BERT\nand XLNet using clear notation and diagrams and explain the advantages of\ntransformer architectures over traditional recurrent neural network\narchitectures. Linear algebra notation is used to clarify the functions of\ntransformers and attention mechanisms. We compare the results with more\ntraditional methods, such as bag of words (BOW) and long short term memory\n(LSTM) networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09485v1",
            "date": "2019-09-17 18:44:54+00:00",
            "title": "BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation",
            "authors": [
                "Iftitahu Ni'mah",
                "Vlado Menkovski",
                "Mykola Pechenizkiy"
            ],
            "abstract": "This study mainly investigates two decoding problems in neural keyphrase\ngeneration: sequence length bias and beam diversity. We introduce an extension\nof beam search inference based on word-level and n-gram level attention score\nto adjust and constrain Seq2Seq prediction at test time. Results show that our\nproposed solution can overcome the algorithm bias to shorter and nearly\nidentical sequences, resulting in a significant improvement of the decoding\nperformance on generating keyphrases that are present and absent in source\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09490v1",
            "date": "2019-09-19 11:58:18+00:00",
            "title": "Deep Contextualized Pairwise Semantic Similarity for Arabic Language Questions",
            "authors": [
                "Hesham Al-Bataineh",
                "Wael Farhan",
                "Ahmad Mustafa",
                "Haitham Seelawi",
                "Hussein T. Al-Natsheh"
            ],
            "abstract": "Question semantic similarity is a challenging and active research problem\nthat is very useful in many NLP applications, such as detecting duplicate\nquestions in community question answering platforms such as Quora. Arabic is\nconsidered to be an under-resourced language, has many dialects, and rich in\nmorphology. Combined together, these challenges make identifying semantically\nsimilar questions in Arabic even more difficult. In this paper, we introduce a\nnovel approach to tackle this problem, and test it on two benchmarks; one for\nModern Standard Arabic (MSA), and another for the 24 major Arabic dialects. We\nare able to show that our new system outperforms state-of-the-art approaches by\nachieving 93% F1-score on the MSA benchmark and 82% on the dialectical one.\nThis is achieved by utilizing contextualized word representations (ELMo\nembeddings) trained on a text corpus containing MSA and dialectic sentences.\nThis in combination with a pairwise fine-grained similarity layer, helps our\nquestion-to-question similarity model to generalize predictions on different\ndialects while being trained only on question-to-question MSA data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09577v1",
            "date": "2019-09-14 03:51:46+00:00",
            "title": "NeMo: a toolkit for building AI applications using Neural Modules",
            "authors": [
                "Oleksii Kuchaiev",
                "Jason Li",
                "Huyen Nguyen",
                "Oleksii Hrinchuk",
                "Ryan Leary",
                "Boris Ginsburg",
                "Samuel Kriman",
                "Stanislav Beliaev",
                "Vitaly Lavrukhin",
                "Jack Cook",
                "Patrice Castonguay",
                "Mariya Popova",
                "Jocelyn Huang",
                "Jonathan M. Cohen"
            ],
            "abstract": "NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI\napplications through re-usability, abstraction, and composition. NeMo is built\naround neural modules, conceptual blocks of neural networks that take typed\ninputs and produce typed outputs. Such modules typically represent data layers,\nencoders, decoders, language models, loss functions, or methods of combining\nactivations. NeMo makes it easy to combine and re-use these building blocks\nwhile providing a level of semantic correctness checking via its neural type\nsystem. The toolkit comes with extendable collections of pre-built modules for\nautomatic speech recognition and natural language processing. Furthermore, NeMo\nprovides built-in support for distributed training and mixed precision on\nlatest NVIDIA GPUs. NeMo is open-source https://github.com/NVIDIA/NeMo",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.09586v1",
            "date": "2019-09-12 15:44:51+00:00",
            "title": "Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks",
            "authors": [
                "Ralf C. Staudemeyer",
                "Eric Rothstein Morris"
            ],
            "abstract": "Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) are one of the\nmost powerful dynamic classifiers publicly known. The network itself and the\nrelated learning algorithms are reasonably well documented to get an idea how\nit works. This paper will shed more light into understanding how LSTM-RNNs\nevolved and why they work impressively well, focusing on the early,\nground-breaking publications. We significantly improved documentation and fixed\na number of errors and inconsistencies that accumulated in previous\npublications. To support understanding we as well revised and unified the\nnotation used.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1909.09587v1",
            "date": "2019-09-15 10:33:05+00:00",
            "title": "Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model",
            "authors": [
                "Tsung-yuan Hsu",
                "Chi-liang Liu",
                "Hung-yi Lee"
            ],
            "abstract": "Because it is not feasible to collect training data for every language, there\nis a growing interest in cross-lingual transfer learning. In this paper, we\nsystematically explore zero-shot cross-lingual transfer learning on reading\ncomprehension tasks with a language representation model pre-trained on\nmulti-lingual corpus. The experimental results show that with pre-trained\nlanguage representation zero-shot learning is feasible, and translating the\nsource data into the target language is not necessary and even degrades the\nperformance. We further explore what does the model learn in zero-shot setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09595v1",
            "date": "2019-09-13 05:59:40+00:00",
            "title": "SANVis: Visual Analytics for Understanding Self-Attention Networks",
            "authors": [
                "Cheonbok Park",
                "Inyoup Na",
                "Yongjang Jo",
                "Sungbok Shin",
                "Jaehyo Yoo",
                "Bum Chul Kwon",
                "Jian Zhao",
                "Hyungjong Noh",
                "Yeonsoo Lee",
                "Jaegul Choo"
            ],
            "abstract": "Attention networks, a deep neural network architecture inspired by humans'\nattention mechanism, have seen significant success in image captioning, machine\ntranslation, and many other applications. Recently, they have been further\nevolved into an advanced approach called multi-head self-attention networks,\nwhich can encode a set of input vectors, e.g., word vectors in a sentence, into\nanother set of vectors. Such encoding aims at simultaneously capturing diverse\nsyntactic and semantic features within a set, each of which corresponds to a\nparticular attention head, forming altogether multi-head attention. Meanwhile,\nthe increased model complexity prevents users from easily understanding and\nmanipulating the inner workings of models. To tackle the challenges, we present\na visual analytics system called SANVis, which helps users understand the\nbehaviors and the characteristics of multi-head self-attention networks. Using\na state-of-the-art self-attention model called Transformer, we demonstrate\nusage scenarios of SANVis in machine translation tasks. Our system is available\nat http://short.sanvis.org",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09690v2",
            "date": "2019-09-12 16:29:14+00:00",
            "title": "A Deep Learning-Based Approach for Measuring the Domain Similarity of Persian Texts",
            "authors": [
                "Hossein Keshavarz",
                "Shohreh Tabatabayi Seifi",
                "Mohammad Izadi"
            ],
            "abstract": "In this paper, we propose a novel approach for measuring the degree of\nsimilarity between categories of two pieces of Persian text, which were\npublished as descriptions of two separate advertisements. We built an\nappropriate dataset for this work using a dataset which consists of\nadvertisements posted on an e-commerce website. We generated a significant\nnumber of paired texts from this dataset and assigned each pair a score from 0\nto 3, which demonstrates the degree of similarity between the domains of the\npair. In this work, we represent words with word embedding vectors derived from\nword2vec. Then deep neural network models are used to represent texts.\nEventually, we employ concatenation of absolute difference and bit-wise\nmultiplication and a fully-connected neural network to produce a probability\ndistribution vector for the score of the pairs. Through a supervised learning\napproach, we trained our model on a GPU, and our best model achieved an F1\nscore of 0.9865.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09691v1",
            "date": "2019-09-12 14:45:43+00:00",
            "title": "NSURL-2019 Shared Task 8: Semantic Question Similarity in Arabic",
            "authors": [
                "Haitham Seelawi",
                "Ahmad Mustafa",
                "Hesham Al-Bataineh",
                "Wael Farhan",
                "Hussein T. Al-Natsheh"
            ],
            "abstract": "Question semantic similarity (Q2Q) is a challenging task that is very useful\nin many NLP applications, such as detecting duplicate questions and question\nanswering systems. In this paper, we present the results and findings of the\nshared task (Semantic Question Similarity in Arabic). The task was organized as\npart of the first workshop on NLP Solutions for Under Resourced Languages\n(NSURL 2019) The goal of the task is to predict whether two questions are\nsemantically similar or not, even if they are phrased differently. A total of 9\nteams participated in the task. The datasets created for this task are made\npublicly available to support further research on Arabic Q2Q.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09699v1",
            "date": "2019-09-15 01:09:01+00:00",
            "title": "Induction and Reference of Entities in a Visual Story",
            "authors": [
                "Ruo-Ping Dong",
                "Khyathi Raghavi Chandu",
                "Alan W Black"
            ],
            "abstract": "We are enveloped by stories of visual interpretations in our everyday lives.\nThe way we narrate a story often comprises of two stages, which are, forming a\ncentral mind map of entities and then weaving a story around them. A\ncontributing factor to coherence is not just basing the story on these entities\nbut also, referring to them using appropriate terms to avoid repetition. In\nthis paper, we address these two stages of introducing the right entities at\nseemingly reasonable junctures and also referring them coherently in the\ncontext of visual storytelling. The building blocks of the central mind map,\nalso known as entity skeleton are entity chains including nominal and\ncoreference expressions. This entity skeleton is also represented in different\nlevels of abstractions to compose a generalized frame to weave the story. We\nbuild upon an encoder-decoder framework to penalize the model when the decoded\nstory does not adhere to this entity skeleton. We establish a strong baseline\nfor skeleton informed generation and then extend this to have the capability of\nmultitasking by predicting the skeleton in addition to generating the story.\nFinally, we build upon this model and propose a glocal hierarchical attention\nmodel that attends to the skeleton both at the sentence (local) and the story\n(global) levels. We observe that our proposed models outperform the baseline in\nterms of automatic evaluation metric, METEOR. We perform various analysis\ntargeted to evaluate the performance of our task of enforcing the entity\nskeleton such as the number and diversity of the entities generated. We also\nconduct human evaluation from which it is concluded that the visual stories\ngenerated by our model are preferred 82% of the times. In addition, we show\nthat our glocal hierarchical attention model improves coherence by introducing\nmore pronouns as required by the presence of nouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09702v2",
            "date": "2019-09-12 04:27:32+00:00",
            "title": "Using Clinical Notes with Time Series Data for ICU Management",
            "authors": [
                "Swaraj Khadanga",
                "Karan Aggarwal",
                "Shafiq Joty",
                "Jaideep Srivastava"
            ],
            "abstract": "Monitoring patients in ICU is a challenging and high-cost task. Hence,\npredicting the condition of patients during their ICU stay can help provide\nbetter acute care and plan the hospital's resources. There has been continuous\nprogress in machine learning research for ICU management, and most of this work\nhas focused on using time series signals recorded by ICU instruments. In our\nwork, we show that adding clinical notes as another modality improves the\nperformance of the model for three benchmark tasks: in-hospital mortality\nprediction, modeling decompensation, and length of stay forecasting that play\nan important role in ICU management. While the time-series data is measured at\nregular intervals, doctor notes are charted at irregular times, making it\nchallenging to model them together. We propose a method to model them jointly,\nachieving considerable improvement across benchmark tasks over baseline\ntime-series model. Our implementation can be found at\n\\url{https://github.com/kaggarwal/ClinicalNotesICU}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09704v2",
            "date": "2019-09-12 17:25:24+00:00",
            "title": "Measuring Domain Portability and ErrorPropagation in Biomedical QA",
            "authors": [
                "Stefan Hosein",
                "Daniel Andor",
                "Ryan McDonald"
            ],
            "abstract": "In this work we present Google's submission to the BioASQ 7 biomedical\nquestion answering (QA) task (specifically Task 7b, Phase B). The core of our\nsystems are based on BERT QA models, specifically the model of\n\\cite{alberti2019bert}. In this report, and via our submissions, we aimed to\ninvestigate two research questions. We start by studying how domain portable\nare QA systems that have been pre-trained and fine-tuned on general texts,\ne.g., Wikipedia. We measure this via two submissions. The first is a\nnon-adapted model that uses a public pre-trained BERT model and is fine-tuned\non the Natural Questions data set \\cite{kwiatkowski2019natural}. The second\nsystem takes this non-adapted model and fine-tunes it with the BioASQ training\ndata. Next, we study the impact of error propagation in end-to-end retrieval\nand QA systems. Again we test this via two submissions. The first uses human\nannotated relevant documents and snippets as input to the model and the second\npredicted documents and snippets. Our main findings are that domain specific\nfine-tuning can benefit Biomedical QA. However, the biggest quality bottleneck\nis at the retrieval stage, where we see large drops in metrics -- over 10pts\nabsolute -- when using non gold inputs to the QA model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09708v1",
            "date": "2019-09-20 20:22:15+00:00",
            "title": "Measuring Conceptual Entanglement in Collections of Documents",
            "authors": [
                "Tomas Veloz",
                "Xiazhao Zhao",
                "Diederik Aerts"
            ],
            "abstract": "Conceptual entanglement is a crucial phenomenon in quantum cognition because\nit implies that classical probabilities cannot model non--compositional\nconceptual phenomena. While several psychological experiments have been\ndeveloped to test conceptual entanglement, this has not been explored in the\ncontext of Natural Language Processing. In this paper, we apply the hypothesis\nthat words of a document are traces of the concepts that a person has in mind\nwhen writing the document. Therefore, if these concepts are entangled, we\nshould be able to observe traces of their entanglement in the documents. In\nparticular, we test conceptual entanglement by contrasting language simulations\nwith results obtained from a text corpus. Our analysis indicates that\nconceptual entanglement is strongly linked to the way in which language is\nstructured. We discuss the implications of this finding in the context of\nconceptual modeling and of Natural Language Processing.",
            "journal ref": "In International Symposium on Quantum Interaction (pp. 134-146).\n  Springer, Berlin, Heidelberg (2013, July)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09788v1",
            "date": "2019-09-21 07:56:09+00:00",
            "title": "Visuallly Grounded Generation of Entailments from Premises",
            "authors": [
                "Somaye Jafaritazehjani",
                "Albert Gatt",
                "Marc Tanti"
            ],
            "abstract": "Natural Language Inference (NLI) is the task of determining the semantic\nrelationship between a premise and a hypothesis. In this paper, we focus on the\n{\\em generation} of hypotheses from premises in a multimodal setting, to\ngenerate a sentence (hypothesis) given an image and/or its description\n(premise) as the input. The main goals of this paper are (a) to investigate\nwhether it is reasonable to frame NLI as a generation task; and (b) to consider\nthe degree to which grounding textual premises in visual information is\nbeneficial to generation. We compare different neural architectures, showing\nthrough automatic and human evaluation that entailments can indeed be generated\nsuccessfully. We also show that multimodal models outperform unimodal models in\nthis task, albeit marginally.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.09922v2",
            "date": "2019-09-22 01:12:18+00:00",
            "title": "Using Chinese Glyphs for Named Entity Recognition",
            "authors": [
                "Arijit Sehanobish",
                "Chan Hee Song"
            ],
            "abstract": "Most Named Entity Recognition (NER) systems use additional features like\npart-of-speech (POS) tags, shallow parsing, gazetteers, etc. Such kind of\ninformation requires external knowledge like unlabeled texts and trained\ntaggers. Adding these features to NER systems have been shown to have a\npositive impact. However, sometimes creating gazetteers or taggers can take a\nlot of time and may require extensive data cleaning. In this paper for Chinese\nNER systems, we do not use these traditional features but we use lexicographic\nfeatures of Chinese characters. Chinese characters are composed of graphical\ncomponents called radicals and these components often have some semantic\nindicators. We propose CNN based models that incorporate this semantic\ninformation and use them for NER. Our models show an improvement over the\nbaseline BERT-BiLSTM-CRF model. We set a new baseline score for Chinese\nOntoNotes v5.0 and show an improvement of +.64 F1 score. We present a\nstate-of-the-art F1 score on Weibo dataset of 71.81 and show a competitive\nimprovement of +0.72 over baseline on ResumeNER dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10261v1",
            "date": "2019-09-23 10:12:13+00:00",
            "title": "Sliding window property testing for regular languages",
            "authors": [
                "Moses Ganardi",
                "Danny Hucke",
                "Markus Lohrey",
                "Tatiana Starikovskaya"
            ],
            "abstract": "We study the problem of recognizing regular languages in a variant of the\nstreaming model of computation, called the sliding window model. In this model,\nwe are given a size of the sliding window $n$ and a stream of symbols. At each\ntime instant, we must decide whether the suffix of length $n$ of the current\nstream (\"the active window\") belongs to a given regular language.\n  Recent works showed that the space complexity of an optimal deterministic\nsliding window algorithm for this problem is either constant, logarithmic or\nlinear in the window size $n$ and provided natural language theoretic\ncharacterizations of the space complexity classes. Subsequently, those results\nwere extended to randomized algorithms to show that any such algorithm admits\neither constant, double logarithmic, logarithmic or linear space complexity.\n  In this work, we make an important step forward and combine the sliding\nwindow model with the property testing setting, which results in\nultra-efficient algorithms for all regular languages. Informally, a sliding\nwindow property tester must accept the active window if it belongs to the\nlanguage and reject it if it is far from the language. We consider\ndeterministic and randomized sliding window property testers with one-sided and\ntwo-sided errors. In particular, we show that for any regular language, there\nis a deterministic sliding window property tester that uses logarithmic space\nand a randomized sliding window property tester with two-sided error that uses\nconstant space.",
            "journal ref": "None",
            "category": "cs.DS"
        },
        {
            "url": "http://arxiv.org/abs/1909.10351v5",
            "date": "2019-09-23 13:05:35+00:00",
            "title": "TinyBERT: Distilling BERT for Natural Language Understanding",
            "authors": [
                "Xiaoqi Jiao",
                "Yichun Yin",
                "Lifeng Shang",
                "Xin Jiang",
                "Xiao Chen",
                "Linlin Li",
                "Fang Wang",
                "Qun Liu"
            ],
            "abstract": "Language model pre-training, such as BERT, has significantly improved the\nperformances of many natural language processing tasks. However, pre-trained\nlanguage models are usually computationally expensive, so it is difficult to\nefficiently execute them on resource-restricted devices. To accelerate\ninference and reduce model size while maintaining accuracy, we first propose a\nnovel Transformer distillation method that is specially designed for knowledge\ndistillation (KD) of the Transformer-based models. By leveraging this new KD\nmethod, the plenty of knowledge encoded in a large teacher BERT can be\neffectively transferred to a small student Tiny-BERT. Then, we introduce a new\ntwo-stage learning framework for TinyBERT, which performs Transformer\ndistillation at both the pretraining and task-specific learning stages. This\nframework ensures that TinyBERT can capture he general-domain as well as the\ntask-specific knowledge in BERT.\n  TinyBERT with 4 layers is empirically effective and achieves more than 96.8%\nthe performance of its teacher BERTBASE on GLUE benchmark, while being 7.5x\nsmaller and 9.4x faster on inference. TinyBERT with 4 layers is also\nsignificantly better than 4-layer state-of-the-art baselines on BERT\ndistillation, with only about 28% parameters and about 31% inference time of\nthem. Moreover, TinyBERT with 6 layers performs on-par with its teacher\nBERTBASE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10447v1",
            "date": "2019-09-23 16:00:06+00:00",
            "title": "On Model Stability as a Function of Random Seed",
            "authors": [
                "Pranava Madhyastha",
                "Rishabh Jain"
            ],
            "abstract": "In this paper, we focus on quantifying model stability as a function of\nrandom seed by investigating the effects of the induced randomness on model\nperformance and the robustness of the model in general. We specifically perform\na controlled study on the effect of random seeds on the behaviour of attention,\ngradient-based and surrogate model based (LIME) interpretations. Our analysis\nsuggests that random seeds can adversely affect the consistency of models\nresulting in counterfactual interpretations. We propose a technique called\nAggressive Stochastic Weight Averaging (ASWA)and an extension called\nNorm-filtered Aggressive Stochastic Weight Averaging (NASWA) which improves the\nstability of models over random seeds. With our ASWA and NASWA based\noptimization, we are able to improve the robustness of the original model, on\naverage reducing the standard deviation of the model's performance by 72%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.10506v1",
            "date": "2019-09-23 17:52:34+00:00",
            "title": "Learning Dense Representations for Entity Retrieval",
            "authors": [
                "Daniel Gillick",
                "Sayali Kulkarni",
                "Larry Lansing",
                "Alessandro Presta",
                "Jason Baldridge",
                "Eugene Ie",
                "Diego Garcia-Olano"
            ],
            "abstract": "We show that it is feasible to perform entity linking by training a dual\nencoder (two-tower) model that encodes mentions and entities in the same dense\nvector space, where candidate entities are retrieved by approximate nearest\nneighbor search. Unlike prior work, this setup does not rely on an alias table\nfollowed by a re-ranker, and is thus the first fully learned entity retrieval\nmodel. We show that our dual encoder, trained using only anchor-text links in\nWikipedia, outperforms discrete alias table and BM25 baselines, and is\ncompetitive with the best comparable results on the standard TACKBP-2010\ndataset. In addition, it can retrieve candidates extremely fast, and\ngeneralizes well to a new dataset derived from Wikinews. On the modeling side,\nwe demonstrate the dramatic value of an unsupervised negative mining algorithm\nfor this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10572v2",
            "date": "2019-09-23 18:54:52+00:00",
            "title": "Hypernym Detection Using Strict Partial Order Networks",
            "authors": [
                "Sarthak Dash",
                "Md Faisal Mahbub Chowdhury",
                "Alfio Gliozzo",
                "Nandana Mihindukulasooriya",
                "Nicolas Rodolfo Fauceglia"
            ],
            "abstract": "This paper introduces Strict Partial Order Networks (SPON), a novel neural\nnetwork architecture designed to enforce asymmetry and transitive properties as\nsoft constraints. We apply it to induce hypernymy relations by training with\nis-a pairs. We also present an augmented variant of SPON that can generalize\ntype information learned for in-vocabulary terms to previously unseen ones. An\nextensive evaluation over eleven benchmarks across different tasks shows that\nSPON consistently either outperforms or attains the state of the art on all but\none of these benchmarks.",
            "journal ref": "AAAI 2020",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.10649v2",
            "date": "2019-09-23 23:21:42+00:00",
            "title": "Portuguese Named Entity Recognition using BERT-CRF",
            "authors": [
                "F\u00e1bio Souza",
                "Rodrigo Nogueira",
                "Roberto Lotufo"
            ],
            "abstract": "Recent advances in language representation using neural networks have made it\nviable to transfer the learned internal states of a trained model to downstream\nnatural language processing tasks, such as named entity recognition (NER) and\nquestion answering. It has been shown that the leverage of pre-trained language\nmodels improves the overall performance on many tasks and is highly beneficial\nwhen labeled data is scarce. In this work, we train Portuguese BERT models and\nemploy a BERT-CRF architecture to the NER task on the Portuguese language,\ncombining the transfer capabilities of BERT with the structured predictions of\nCRF. We explore feature-based and fine-tuning training strategies for the BERT\nmodel. Our fine-tuning approach obtains new state-of-the-art results on the\nHAREM I dataset, improving the F1-score by 1 point on the selective scenario (5\nNE classes) and by 4 points on the total scenario (10 NE classes).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10681v2",
            "date": "2019-09-24 02:08:29+00:00",
            "title": "Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations",
            "authors": [
                "Peixiang Zhong",
                "Di Wang",
                "Chunyan Miao"
            ],
            "abstract": "Messages in human conversations inherently convey emotions. The task of\ndetecting emotions in textual conversations leads to a wide range of\napplications such as opinion mining in social networks. However, enabling\nmachines to analyze emotions in conversations is challenging, partly because\nhumans often rely on the context and commonsense knowledge to express emotions.\nIn this paper, we address these challenges by proposing a Knowledge-Enriched\nTransformer (KET), where contextual utterances are interpreted using\nhierarchical self-attention and external commonsense knowledge is dynamically\nleveraged using a context-aware affective graph attention mechanism.\nExperiments on multiple textual conversation datasets demonstrate that both\ncontext and commonsense knowledge are consistently beneficial to the emotion\ndetection performance. In addition, the experimental results show that our KET\nmodel outperforms the state-of-the-art models on most of the tested datasets in\nF1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10699v1",
            "date": "2019-09-24 03:56:48+00:00",
            "title": "LitGen: Genetic Literature Recommendation Guided by Human Explanations",
            "authors": [
                "Allen Nie",
                "Arturo L. Pineda",
                "Matt W. Wright Hannah Wand",
                "Bryan Wulf",
                "Helio A. Costa",
                "Ronak Y. Patel",
                "Carlos D. Bustamante",
                "James Zou"
            ],
            "abstract": "As genetic sequencing costs decrease, the lack of clinical interpretation of\nvariants has become the bottleneck in using genetics data. A major rate\nlimiting step in clinical interpretation is the manual curation of evidence in\nthe genetic literature by highly trained biocurators. What makes curation\nparticularly time-consuming is that the curator needs to identify papers that\nstudy variant pathogenicity using different types of approaches and\nevidences---e.g. biochemical assays or case control analysis. In collaboration\nwith the Clinical Genomic Resource (ClinGen)---the flagship NIH program for\nclinical curation---we propose the first machine learning system, LitGen, that\ncan retrieve papers for a particular variant and filter them by specific\nevidence types used by curators to assess for pathogenicity. LitGen uses\nsemi-supervised deep learning to predict the type of evidence provided by each\npaper. It is trained on papers annotated by ClinGen curators and systematically\nevaluated on new test data collected by ClinGen. LitGen further leverages rich\nhuman explanations and unlabeled data to gain 7.9%-12.6% relative performance\nimprovement over models learned only on the annotated papers. It is a useful\nframework to improve clinical variant curation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10705v1",
            "date": "2019-09-24 04:26:27+00:00",
            "title": "Do Massively Pretrained Language Models Make Better Storytellers?",
            "authors": [
                "Abigail See",
                "Aneesh Pappu",
                "Rohun Saxena",
                "Akhila Yerukola",
                "Christopher D. Manning"
            ],
            "abstract": "Large neural language models trained on massive amounts of text have emerged\nas a formidable strategy for Natural Language Understanding tasks. However, the\nstrength of these models as Natural Language Generators is less clear. Though\nanecdotal evidence suggests that these models generate better quality text,\nthere has been no detailed study characterizing their generation abilities. In\nthis work, we compare the performance of an extensively pretrained model,\nOpenAI GPT2-117 (Radford et al., 2019), to a state-of-the-art neural story\ngeneration model (Fan et al., 2018). By evaluating the generated text across a\nwide variety of automatic metrics, we characterize the ways in which pretrained\nmodels do, and do not, make better storytellers. We find that although GPT2-117\nconditions more strongly on context, is more sensitive to ordering of events,\nand uses more unusual words, it is just as likely to produce repetitive and\nunder-diverse text when using likelihood-maximizing decoding algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10812v1",
            "date": "2019-09-24 11:04:02+00:00",
            "title": "Deep Text Mining of Instagram Data Without Strong Supervision",
            "authors": [
                "Kim Hammar",
                "Shatha Jaradat",
                "Nima Dokoohaki",
                "Mihhail Matskin"
            ],
            "abstract": "With the advent of social media, our online feeds increasingly consist of\nshort, informal, and unstructured text. This textual data can be analyzed for\nthe purpose of improving user recommendations and detecting trends. Instagram\nis one of the largest social media platforms, containing both text and images.\nHowever, most of the prior research on text processing in social media is\nfocused on analyzing Twitter data, and little attention has been paid to text\nmining of Instagram data. Moreover, many text mining methods rely on annotated\ntraining data, which in practice is both difficult and expensive to obtain. In\nthis paper, we present methods for unsupervised mining of fashion attributes\nfrom Instagram text, which can enable a new kind of user recommendation in the\nfashion domain. In this context, we analyze a corpora of Instagram posts from\nthe fashion domain, introduce a system for extracting fashion attributes from\nInstagram, and train a deep clothing classifier with weak supervision to\nclassify Instagram posts based on the associated text.\n  With our experiments, we confirm that word embeddings are a useful asset for\ninformation extraction. Experimental results show that information extraction\nusing word embeddings outperforms a baseline that uses Levenshtein distance.\nThe results also show the benefit of combining weak supervision signals using\ngenerative models instead of majority voting. Using weak supervision and\ngenerative modeling, an F1 score of 0.61 is achieved on the task of classifying\nthe image contents of Instagram posts based solely on the associated text,\nwhich is on level with human performance. Finally, our empirical study provides\none of the few available studies on Instagram text and shows that the text is\nnoisy, that the text distribution exhibits the long-tail phenomenon, and that\ncomment sections on Instagram are multi-lingual.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10838v2",
            "date": "2019-09-24 12:29:27+00:00",
            "title": "Talk2Car: Taking Control of Your Self-Driving Car",
            "authors": [
                "Thierry Deruyttere",
                "Simon Vandenhende",
                "Dusan Grujicic",
                "Luc Van Gool",
                "Marie-Francine Moens"
            ],
            "abstract": "A long-term goal of artificial intelligence is to have an agent execute\ncommands communicated through natural language. In many cases the commands are\ngrounded in a visual environment shared by the human who gives the command and\nthe agent. Execution of the command then requires mapping the command into the\nphysical visual space, after which the appropriate action can be taken. In this\npaper we consider the former. Or more specifically, we consider the problem in\nan autonomous driving setting, where a passenger requests an action that can be\nassociated with an object found in a street scene. Our work presents the\nTalk2Car dataset, which is the first object referral dataset that contains\ncommands written in natural language for self-driving cars. We provide a\ndetailed comparison with related datasets such as ReferIt, RefCOCO, RefCOCO+,\nRefCOCOg, Cityscape-Ref and CLEVR-Ref. Additionally, we include a performance\nanalysis using strong state-of-the-art models. The results show that the\nproposed object referral task is a challenging one for which the models show\npromising results but still require additional research in natural language\nprocessing, computer vision and the intersection of these fields. The dataset\ncan be found on our website: http://macchina-ai.eu/",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.10861v2",
            "date": "2019-09-24 13:06:43+00:00",
            "title": "Learning ASR-Robust Contextualized Embeddings for Spoken Language Understanding",
            "authors": [
                "Chao-Wei Huang",
                "Yun-Nung Chen"
            ],
            "abstract": "Employing pre-trained language models (LM) to extract contextualized word\nrepresentations has achieved state-of-the-art performance on various NLP tasks.\nHowever, applying this technique to noisy transcripts generated by automatic\nspeech recognizer (ASR) is concerned. Therefore, this paper focuses on making\ncontextualized representations more ASR-robust. We propose a novel\nconfusion-aware fine-tuning method to mitigate the impact of ASR errors to\npre-trained LMs. Specifically, we fine-tune LMs to produce similar\nrepresentations for acoustically confusable words that are obtained from word\nconfusion networks (WCNs) produced by ASR. Experiments on the benchmark ATIS\ndataset show that the proposed method significantly improves the performance of\nspoken language understanding when performing on ASR transcripts. Our source\ncode is available at https://github.com/MiuLab/SpokenVec",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.10924v1",
            "date": "2019-09-24 13:49:14+00:00",
            "title": "Understanding Semantics from Speech Through Pre-training",
            "authors": [
                "Pengwei Wang",
                "Liangchen Wei",
                "Yong Cao",
                "Jinghui Xie",
                "Yuji Cao",
                "Zaiqing Nie"
            ],
            "abstract": "End-to-end Spoken Language Understanding (SLU) is proposed to infer the\nsemantic meaning directly from audio features without intermediate text\nrepresentation. Although the acoustic model component of an end-to-end SLU\nsystem can be pre-trained with Automatic Speech Recognition (ASR) targets, the\nSLU component can only learn semantic features from limited task-specific\ntraining data. In this paper, for the first time we propose to do large-scale\nunsupervised pre-training for the SLU component of an end-to-end SLU system, so\nthat the SLU component may preserve semantic features from massive unlabeled\naudio data. As the output of the acoustic model component, i.e. phoneme\nposterior sequences, has much different characteristic from text sequences, we\npropose a novel pre-training model called BERT-PLM, which stands for\nBidirectional Encoder Representations from Transformers through Permutation\nLanguage Modeling. BERT-PLM trains the SLU component on unlabeled data through\na regression objective equivalent to the partial permutation language modeling\nobjective, while leverages full bi-directional context information with BERT\nnetworks. The experiment results show that our approach out-perform the\nstate-of-the-art end-to-end systems with over 12.5% error reduction.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.11060v1",
            "date": "2019-09-24 17:18:48+00:00",
            "title": "Paying Attention to Function Words",
            "authors": [
                "Shane Steinert-Threlkeld"
            ],
            "abstract": "All natural languages exhibit a distinction between content words (like nouns\nand adjectives) and function words (like determiners, auxiliaries,\nprepositions). Yet surprisingly little has been said about the emergence of\nthis universal architectural feature of natural languages. Why have human\nlanguages evolved to exhibit this division of labor between content and\nfunction words? How could such a distinction have emerged in the first place?\nThis paper takes steps towards answering these questions by showing how the\ndistinction can emerge through reinforcement learning in agents playing a\nsignaling game across contexts which contain multiple objects that possess\nmultiple perceptually salient gradable properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11200v2",
            "date": "2019-09-24 21:46:42+00:00",
            "title": "Improving Noise Robustness In Speaker Identification Using A Two-Stage Attention Model",
            "authors": [
                "Yanpei Shi",
                "Qiang Huang",
                "Thomas Hain"
            ],
            "abstract": "While the use of deep neural networks has significantly boosted speaker\nrecognition performance, it is still challenging to separate speakers in poor\nacoustic environments. To improve robustness of speaker recognition system\nperformance in noise, a novel two-stage attention mechanism which can be used\nin existing architectures such as Time Delay Neural Networks (TDNNs) and\nConvolutional Neural Networks (CNNs) is proposed. Noise is known to often mask\nimportant information in both time and frequency domain. The proposed mechanism\nallows the models to concentrate on reliable time/frequency components of the\nsignal. The proposed approach is evaluated using the Voxceleb1 dataset, which\naims at assessment of speaker recognition in real world situations. In addition\nthree types of noise at different signal-noise-ratios (SNRs) were added for\nthis work. The proposed mechanism is compared with three strong baselines:\nX-vectors, Attentive X-vector, and Resnet-34. Results on both identification\nand verification tasks show that the two-stage attention mechanism consistently\nimproves upon these for all noise conditions.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.11232v1",
            "date": "2019-09-24 23:44:49+00:00",
            "title": "Sign Language Recognition Analysis using Multimodal Data",
            "authors": [
                "Al Amin Hosain",
                "Panneer Selvam Santhalingam",
                "Parth Pathak",
                "Jana Kosecka",
                "Huzefa Rangwala"
            ],
            "abstract": "Voice-controlled personal and home assistants (such as the Amazon Echo and\nApple Siri) are becoming increasingly popular for a variety of applications.\nHowever, the benefits of these technologies are not readily accessible to Deaf\nor Hard-ofHearing (DHH) users. The objective of this study is to develop and\nevaluate a sign recognition system using multiple modalities that can be used\nby DHH signers to interact with voice-controlled devices. With the advancement\nof depth sensors, skeletal data is used for applications like video analysis\nand activity recognition. Despite having similarity with the well-studied human\nactivity recognition, the use of 3D skeleton data in sign language recognition\nis rare. This is because unlike activity recognition, sign language is mostly\ndependent on hand shape pattern. In this work, we investigate the feasibility\nof using skeletal and RGB video data for sign language recognition using a\ncombination of different deep learning architectures. We validate our results\non a large-scale American Sign Language (ASL) dataset of 12 users and 13107\nsamples across 51 signs. It is named as GMUASL51. We collected the dataset over\n6 months and it will be publicly released in the hope of spurring further\nmachine learning research towards providing improved accessibility for digital\nassistants.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.11258v1",
            "date": "2019-09-25 02:25:23+00:00",
            "title": "PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space",
            "authors": [
                "Omer Anjum",
                "Hongyu Gong",
                "Suma Bhat",
                "Wen-Mei Hwu",
                "Jinjun Xiong"
            ],
            "abstract": "Finding the right reviewers to assess the quality of conference submissions\nis a time consuming process for conference organizers. Given the importance of\nthis step, various automated reviewer-paper matching solutions have been\nproposed to alleviate the burden. Prior approaches, including bag-of-words\nmodels and probabilistic topic models have been inadequate to deal with the\nvocabulary mismatch and partial topic overlap between a paper submission and\nthe reviewer's expertise. Our approach, the common topic model, jointly models\nthe topics common to the submission and the reviewer's profile while relying on\nabstract topic vectors. Experiments and insightful evaluations on two datasets\ndemonstrate that the proposed method achieves consistent improvements compared\nto available state-of-the-art implementations of paper-reviewer matching.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11288v1",
            "date": "2019-09-25 04:47:49+00:00",
            "title": "Annotated Guidelines and Building Reference Corpus for Myanmar-English Word Alignment",
            "authors": [
                "Nway Nway Han",
                "Aye Thida"
            ],
            "abstract": "Reference corpus for word alignment is an important resource for developing\nand evaluating word alignment methods. For Myanmar-English language pairs,\nthere is no reference corpus to evaluate the word alignment tasks. Therefore,\nwe created the guidelines for Myanmar-English word alignment annotation between\ntwo languages over contrastive learning and built the Myanmar-English reference\ncorpus consisting of verified alignments from Myanmar ALT of the Asian Language\nTreebank (ALT). This reference corpus contains confident labels sure (S) and\npossible (P) for word alignments which are used to test for the purpose of\nevaluation of the word alignments tasks. We discuss the most linking\nambiguities to define consistent and systematic instructions to align manual\nwords. We evaluated the results of annotators agreement using our reference\ncorpus in terms of alignment error rate (AER) in word alignment tasks and\ndiscuss the words relationships in terms of BLEU scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11556v1",
            "date": "2019-09-25 15:35:03+00:00",
            "title": "Reducing Transformer Depth on Demand with Structured Dropout",
            "authors": [
                "Angela Fan",
                "Edouard Grave",
                "Armand Joulin"
            ],
            "abstract": "Overparameterized transformer networks have obtained state of the art results\nin various natural language processing tasks, such as machine translation,\nlanguage modeling, and question answering. These models contain hundreds of\nmillions of parameters, necessitating a large amount of computation and making\nthem prone to overfitting. In this work, we explore LayerDrop, a form of\nstructured dropout, which has a regularization effect during training and\nallows for efficient pruning at inference time. In particular, we show that it\nis possible to select sub-networks of any depth from one large network without\nhaving to finetune them and with limited impact on performance. We demonstrate\nthe effectiveness of our approach by improving the state of the art on machine\ntranslation, language modeling, summarization, question answering, and language\nunderstanding benchmarks. Moreover, we show that our approach leads to small\nBERT-like models of higher quality compared to training from scratch or using\ndistillation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.11699v1",
            "date": "2019-09-25 18:32:50+00:00",
            "title": "Speech Recognition with Augmented Synthesized Speech",
            "authors": [
                "Andrew Rosenberg",
                "Yu Zhang",
                "Bhuvana Ramabhadran",
                "Ye Jia",
                "Pedro Moreno",
                "Yonghui Wu",
                "Zelin Wu"
            ],
            "abstract": "Recent success of the Tacotron speech synthesis architecture and its variants\nin producing natural sounding multi-speaker synthesized speech has raised the\nexciting possibility of replacing expensive, manually transcribed,\ndomain-specific, human speech that is used to train speech recognizers. The\nmulti-speaker speech synthesis architecture can learn latent embedding spaces\nof prosody, speaker and style variations derived from input acoustic\nrepresentations thereby allowing for manipulation of the synthesized speech. In\nthis paper, we evaluate the feasibility of enhancing speech recognition\nperformance using speech synthesis using two corpora from different domains. We\nexplore algorithms to provide the necessary acoustic and lexical diversity\nneeded for robust speech recognition. Finally, we demonstrate the feasibility\nof this approach as a data augmentation strategy for domain-transfer.\n  We find that improvements to speech recognition performance is achievable by\naugmenting training data with synthesized material. However, there remains a\nsubstantial gap in performance between recognizers trained on human speech\nthose trained on synthesized speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11706v3",
            "date": "2019-09-25 18:43:22+00:00",
            "title": "The Power of Communities: A Text Classification Model with Automated Labeling Process Using Network Community Detection",
            "authors": [
                "Minjun Kim",
                "Hiroki Sayama"
            ],
            "abstract": "Text classification is one of the most critical areas in machine learning and\nartificial intelligence research. It has been actively adopted in many business\napplications such as conversational intelligence systems, news articles\ncategorizations, sentiment analysis, emotion detection systems, and many other\nrecommendation systems in our daily life. One of the problems in supervised\ntext classification models is that the models' performance depends heavily on\nthe quality of data labeling that is typically done by humans. In this study,\nwe propose a new network community detection-based approach to automatically\nlabel and classify text data into multiclass value spaces. Specifically, we\nbuild networks with sentences as the network nodes and pairwise cosine\nsimilarities between the Term Frequency-Inversed Document Frequency (TFIDF)\nvector representations of the sentences as the network link weights. We use the\nLouvain method to detect the communities in the sentence networks. We train and\ntest the Support Vector Machine and the Random Forest models on both the\nhuman-labeled data and network community detection labeled data. Results showed\nthat models with the data labeled by the network community detection\noutperformed the models with the human-labeled data by 2.68-3.75% of\nclassification accuracy. Our method may help developments of more accurate\nconversational intelligence and other text classification systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11740v3",
            "date": "2019-09-25 20:02:54+00:00",
            "title": "UNITER: UNiversal Image-TExt Representation Learning",
            "authors": [
                "Yen-Chun Chen",
                "Linjie Li",
                "Licheng Yu",
                "Ahmed El Kholy",
                "Faisal Ahmed",
                "Zhe Gan",
                "Yu Cheng",
                "Jingjing Liu"
            ],
            "abstract": "Joint image-text embedding is the bedrock for most Vision-and-Language (V+L)\ntasks, where multimodality inputs are simultaneously processed for joint visual\nand textual understanding. In this paper, we introduce UNITER, a UNiversal\nImage-TExt Representation, learned through large-scale pre-training over four\nimage-text datasets (COCO, Visual Genome, Conceptual Captions, and SBU\nCaptions), which can power heterogeneous downstream V+L tasks with joint\nmultimodal embeddings. We design four pre-training tasks: Masked Language\nModeling (MLM), Masked Region Modeling (MRM, with three variants), Image-Text\nMatching (ITM), and Word-Region Alignment (WRA). Different from previous work\nthat applies joint random masking to both modalities, we use conditional\nmasking on pre-training tasks (i.e., masked language/region modeling is\nconditioned on full observation of image/text). In addition to ITM for global\nimage-text alignment, we also propose WRA via the use of Optimal Transport (OT)\nto explicitly encourage fine-grained alignment between words and image regions\nduring pre-training. Comprehensive analysis shows that both conditional masking\nand OT-based WRA contribute to better pre-training. We also conduct a thorough\nablation study to find an optimal combination of pre-training tasks. Extensive\nexperiments show that UNITER achieves new state of the art across six V+L tasks\n(over nine datasets), including Visual Question Answering, Image-Text\nRetrieval, Referring Expression Comprehension, Visual Commonsense Reasoning,\nVisual Entailment, and NLVR$^2$. Code is available at\nhttps://github.com/ChenRocks/UNITER.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.11974v3",
            "date": "2019-09-26 08:34:05+00:00",
            "title": "Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation",
            "authors": [
                "Ze Yang",
                "Can Xu",
                "Wei Wu",
                "Zhoujun Li"
            ],
            "abstract": "Automatic news comment generation is a new testbed for techniques of natural\nlanguage generation. In this paper, we propose a \"read-attend-comment\"\nprocedure for news comment generation and formalize the procedure with a\nreading network and a generation network. The reading network comprehends a\nnews article and distills some important points from it, then the generation\nnetwork creates a comment by attending to the extracted discrete points and the\nnews title. We optimize the model in an end-to-end manner by maximizing a\nvariational lower bound of the true objective using the back-propagation\nalgorithm. Experimental results on two datasets indicate that our model can\nsignificantly outperform existing methods in terms of both automatic evaluation\nand human judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11980v1",
            "date": "2019-09-26 08:46:02+00:00",
            "title": "Spoken Conversational Search for General Knowledge",
            "authors": [
                "Lina M. Rojas-Barahona",
                "Pascal Bellec",
                "Benoit Besset",
                "Martinho Dos-Santos",
                "Johannes Heinecke",
                "Munshi Asadullah",
                "Olivier Le-Blouch",
                "Jean Y. Lancien",
                "G\u00e9raldine Damnati",
                "Emmanuel Mory",
                "Fr\u00e9d\u00e9ric Herledan"
            ],
            "abstract": "We present a spoken conversational question answering proof of concept that\nis able to answer questions about general knowledge from Wikidata. The dialogue\ncomponent does not only orchestrate various components but also solve\ncoreferences and ellipsis.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.12066v2",
            "date": "2019-09-26 12:55:14+00:00",
            "title": "Towards a Metric for Automated Conversational Dialogue System Evaluation and Improvement",
            "authors": [
                "Jan Deriu",
                "Mark Cieliebak"
            ],
            "abstract": "We present \"AutoJudge\", an automated evaluation method for conversational\ndialogue systems. The method works by first generating dialogues based on\nself-talk, i.e. dialogue systems talking to itself. Then, it uses human ratings\non these dialogues to train an automated judgement model. Our experiments show\nthat AutoJudge correlates well with the human ratings and can be used to\nautomatically evaluate dialogue systems, even in deployed systems. In a second\npart, we attempt to apply AutoJudge to improve existing systems. This works\nwell for re-ranking a set of candidate utterances. However, our experiments\nshow that AutoJudge cannot be applied as reward for reinforcement learning,\nalthough the metric can distinguish good from bad dialogues. We discuss\npotential reasons, but state here already that this is still an open question\nfor further research.",
            "journal ref": "Proceedings of the 12th International Conference on Natural\n  Language Generation. 2019",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1909.12086v1",
            "date": "2019-09-26 13:34:26+00:00",
            "title": "GECOR: An End-to-End Generative Ellipsis and Co-reference Resolution Model for Task-Oriented Dialogue",
            "authors": [
                "Jun Quan",
                "Deyi Xiong",
                "Bonnie Webber",
                "Changjian Hu"
            ],
            "abstract": "Ellipsis and co-reference are common and ubiquitous especially in multi-turn\ndialogues. In this paper, we treat the resolution of ellipsis and co-reference\nin dialogue as a problem of generating omitted or referred expressions from the\ndialogue context. We therefore propose a unified end-to-end Generative Ellipsis\nand CO-reference Resolution model (GECOR) in the context of dialogue. The model\ncan generate a new pragmatically complete user utterance by alternating the\ngeneration and copy mode for each user utterance. A multi-task learning\nframework is further proposed to integrate the GECOR into an end-to-end\ntask-oriented dialogue. In order to train both the GECOR and the multi-task\nlearning framework, we manually construct a new dataset on the basis of the\npublic dataset CamRest676 with both ellipsis and co-reference annotation. On\nthis dataset, intrinsic evaluations on the resolution of ellipsis and\nco-reference show that the GECOR model significantly outperforms the\nsequence-to-sequence (seq2seq) baseline model in terms of EM, BLEU and F1 while\nextrinsic evaluations on the downstream dialogue task demonstrate that our\nmulti-task learning framework with GECOR achieves a higher success rate of task\ncompletion than TSCP, a state-of-the-art end-to-end task-oriented dialogue\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12229v1",
            "date": "2019-09-24 02:46:58+00:00",
            "title": "Keyphrase Generation for Scientific Articles using GANs",
            "authors": [
                "Avinash Swaminathan",
                "Raj Kuwar Gupta",
                "Haimin Zhang",
                "Debanjan Mahata",
                "Rakesh Gosangi",
                "Rajiv Ratn Shah"
            ],
            "abstract": "In this paper, we present a keyphrase generation approach using conditional\nGenerative Adversarial Networks (GAN). In our GAN model, the generator outputs\na sequence of keyphrases based on the title and abstract of a scientific\narticle. The discriminator learns to distinguish between machine-generated and\nhuman-curated keyphrases. We evaluate this approach on standard benchmark\ndatasets. Our model achieves state-of-the-art performance in generation of\nabstractive keyphrases and is also comparable to the best performing extractive\ntechniques. We also demonstrate that our method generates more diverse\nkeyphrases and make our implementation publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12231v1",
            "date": "2019-09-20 10:21:55+00:00",
            "title": "Learning to Create Sentence Semantic Relation Graphs for Multi-Document Summarization",
            "authors": [
                "Diego Antognini",
                "Boi Faltings"
            ],
            "abstract": "Linking facts across documents is a challenging task, as the language used to\nexpress the same information in a sentence can vary significantly, which\ncomplicates the task of multi-document summarization. Consequently, existing\napproaches heavily rely on hand-crafted features, which are domain-dependent\nand hard to craft, or additional annotated data, which is costly to gather. To\novercome these limitations, we present a novel method, which makes use of two\ntypes of sentence embeddings: universal embeddings, which are trained on a\nlarge unrelated corpus, and domain-specific embeddings, which are learned\nduring training.\n  To this end, we develop SemSentSum, a fully data-driven model able to\nleverage both types of sentence embeddings by building a sentence semantic\nrelation graph. SemSentSum achieves competitive results on two types of\nsummary, consisting of 665 bytes and 100 words. Unlike other state-of-the-art\nmodels, neither hand-crafted features nor additional annotated data are\nnecessary, and the method is easily adaptable for other tasks. To our\nknowledge, we are the first to use multiple sentence embeddings for the task of\nmulti-document summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12289v2",
            "date": "2019-09-26 17:52:15+00:00",
            "title": "Attention Forcing for Sequence-to-sequence Model Training",
            "authors": [
                "Qingyun Dou",
                "Yiting Lu",
                "Joshua Efiong",
                "Mark J. F. Gales"
            ],
            "abstract": "Auto-regressive sequence-to-sequence models with attention mechanism have\nachieved state-of-the-art performance in many tasks such as machine translation\nand speech synthesis. These models can be difficult to train. The standard\napproach, teacher forcing, guides a model with reference output history during\ntraining. The problem is that the model is unlikely to recover from its\nmistakes during inference, where the reference output is replaced by generated\noutput. Several approaches deal with this problem, largely by guiding the model\nwith generated output history. To make training stable, these approaches often\nrequire a heuristic schedule or an auxiliary classifier. This paper introduces\nattention forcing, which guides the model with generated output history and\nreference attention. This approach can train the model to recover from its\nmistakes, in a stable fashion, without the need for a schedule or a classifier.\nIn addition, it allows the model to generate output sequences aligned with the\nreferences, which can be important for cascaded systems like many speech\nsynthesis systems. Experiments on speech synthesis show that attention forcing\nyields significant performance gain. Experiments on machine translation show\nthat for tasks where various re-orderings of the output are valid, guiding the\nmodel with generated output history is challenging, while guiding the model\nwith reference attention is beneficial.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.12299v2",
            "date": "2019-09-26 17:59:33+00:00",
            "title": "Expert2Coder: Capturing Divergent Brain Regions Using Mixture of Regression Experts",
            "authors": [
                "Subba Reddy Oota",
                "Naresh Manwani",
                "Raju S. Bapi"
            ],
            "abstract": "fMRI semantic category understanding using linguistic encoding models\nattempts to learn a forward mapping that relates stimuli to the corresponding\nbrain activation. State-of-the-art encoding models use a single global model\n(linear or non-linear) to predict brain activation given the stimulus. However,\nthe critical assumption in these methods is that a priori different brain\nregions respond the same way to all the stimuli, that is, there is no\nmodularity or specialization assumed for any region. This goes against the\nmodularity theory, supported by many cognitive neuroscience investigations\nsuggesting that there are functionally specialized regions in the brain. In\nthis paper, we achieve this by clustering similar regions together and for\nevery cluster we learn a different linear regression model using a mixture of\nlinear experts model. The key idea here is that each linear expert captures the\nbehaviour of similar brain regions. Given a new stimulus, the utility of the\nproposed model is twofold (i) predicts the brain activation as a weighted\nlinear combination of the activations of multiple linear experts and (ii) to\nlearn multiple experts corresponding to different brain regions. We argue that\neach expert captures activity patterns related to a particular region of\ninterest (ROI) in the human brain. This study helps in understanding the brain\nregions that are activated together given different kinds of stimuli.\nImportantly, we suggest that the mixture of regression experts (MoRE) framework\nsuccessfully combines the two principles of organization of function in the\nbrain, namely that of specialization and integration. Experiments on fMRI data\nfrom paradigm 1 [1]where participants view linguistic stimuli show that the\nproposed MoRE model has better prediction accuracy compared to that of\nconventional models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.12401v1",
            "date": "2019-09-26 21:25:41+00:00",
            "title": "A Hierarchical Approach for Visual Storytelling Using Image Description",
            "authors": [
                "Md Sultan Al Nahian",
                "Tasmia Tasrin",
                "Sagar Gandhi",
                "Ryan Gaines",
                "Brent Harrison"
            ],
            "abstract": "One of the primary challenges of visual storytelling is developing techniques\nthat can maintain the context of the story over long event sequences to\ngenerate human-like stories. In this paper, we propose a hierarchical deep\nlearning architecture based on encoder-decoder networks to address this\nproblem. To better help our network maintain this context while also generating\nlong and diverse sentences, we incorporate natural language image descriptions\nalong with the images themselves to generate each story sentence. We evaluate\nour system on the Visual Storytelling (VIST) dataset and show that our method\noutperforms state-of-the-art techniques on a suite of different automatic\nevaluation metrics. The empirical results from this evaluation demonstrate the\nnecessities of different components of our proposed architecture and shows the\neffectiveness of the architecture for visual storytelling.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1909.12408v3",
            "date": "2019-09-26 21:43:53+00:00",
            "title": "Optimizing Speech Recognition For The Edge",
            "authors": [
                "Yuan Shangguan",
                "Jian Li",
                "Qiao Liang",
                "Raziel Alvarez",
                "Ian McGraw"
            ],
            "abstract": "While most deployed speech recognition systems today still run on servers, we\nare in the midst of a transition towards deployments on edge devices. This leap\nto the edge is powered by the progression from traditional speech recognition\npipelines to end-to-end (E2E) neural architectures, and the parallel\ndevelopment of more efficient neural network topologies and optimization\ntechniques. Thus, we are now able to create highly accurate speech recognizers\nthat are both small and fast enough to execute on typical mobile devices. In\nthis paper, we begin with a baseline RNN-Transducer architecture comprised of\nLong Short-Term Memory (LSTM) layers. We then experiment with a variety of more\ncomputationally efficient layer types, as well as apply optimization techniques\nlike neural connection pruning and parameter quantization to construct a small,\nhigh quality, on-device speech recognizer that is an order of magnitude smaller\nthan the baseline system without any optimizations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12434v2",
            "date": "2019-09-26 23:25:25+00:00",
            "title": "Learning the Difference that Makes a Difference with Counterfactually-Augmented Data",
            "authors": [
                "Divyansh Kaushik",
                "Eduard Hovy",
                "Zachary C. Lipton"
            ],
            "abstract": "Despite alarm over the reliance of machine learning systems on so-called\nspurious patterns, the term lacks coherent meaning in standard statistical\nframeworks. However, the language of causality offers clarity: spurious\nassociations are due to confounding (e.g., a common cause), but not direct or\nindirect causal effects. In this paper, we focus on natural language\nprocessing, introducing methods and resources for training models less\nsensitive to spurious patterns. Given documents and their initial labels, we\ntask humans with revising each document so that it (i) accords with a\ncounterfactual target label; (ii) retains internal coherence; and (iii) avoids\nunnecessary changes. Interestingly, on sentiment analysis and natural language\ninference tasks, classifiers trained on original data fail on their\ncounterfactually-revised counterparts and vice versa. Classifiers trained on\ncombined datasets perform remarkably well, just shy of those specialized to\neither domain. While classifiers trained on either original or manipulated data\nalone are sensitive to spurious features (e.g., mentions of genre), models\ntrained on the combined data are less sensitive to this signal. Both datasets\nare publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12486v2",
            "date": "2019-09-27 04:10:10+00:00",
            "title": "Reweighted Proximal Pruning for Large-Scale Language Representation",
            "authors": [
                "Fu-Ming Guo",
                "Sijia Liu",
                "Finlay S. Mungall",
                "Xue Lin",
                "Yanzhi Wang"
            ],
            "abstract": "Recently, pre-trained language representation flourishes as the mainstay of\nthe natural language understanding community, e.g., BERT. These pre-trained\nlanguage representations can create state-of-the-art results on a wide range of\ndownstream tasks. Along with continuous significant performance improvement,\nthe size and complexity of these pre-trained neural models continue to increase\nrapidly. Is it possible to compress these large-scale language representation\nmodels? How will the pruned language representation affect the downstream\nmulti-task transfer learning objectives? In this paper, we propose Reweighted\nProximal Pruning (RPP), a new pruning method specifically designed for a\nlarge-scale language representation model. Through experiments on SQuAD and the\nGLUE benchmark suite, we show that proximal pruned BERT keeps high accuracy for\nboth the pre-training task and the downstream multiple fine-tuning tasks at\nhigh prune ratio. RPP provides a new perspective to help us analyze what\nlarge-scale language representation might learn. Additionally, RPP makes it\npossible to deploy a large state-of-the-art language representation model such\nas BERT on a series of distinct devices (e.g., online servers, mobile phones,\nand edge devices).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.12622v1",
            "date": "2019-09-27 11:19:21+00:00",
            "title": "Multi-Modal Citizen Science: From Disambiguation to Transcription of Classical Literature",
            "authors": [
                "Maryam Foradi",
                "Jan Ka\u00dfel",
                "Johannes Pein",
                "Gregory R. Crane"
            ],
            "abstract": "The engagement of citizens in the research projects, including Digital\nHumanities projects, has risen in prominence in recent years. This type of\nengagement not only leads to incidental learning of participants but also\nindicates the added value of corpus enrichment via different types of\nannotations undertaken by users generating so-called smart texts. Our work\nfocuses on the continuous task of adding new layers of annotation to Classical\nLiterature. We aim to provide more extensive tools for readers of smart texts,\nenhancing their reading comprehension and at the same time empowering the\nlanguage learning by introducing intellectual tasks, i.e., linking, tagging,\nand disambiguation. The current study adds a new mode of annotation-audio\nannotations-to the extensively annotated corpus of poetry by the Persian poet\nHafiz. By proposing tasks with three different difficulty levels, we estimate\nthe users' ability of providing correct annotations in order to rate their\nanswers in further stages of the project, where no ground truth data is\navailable. While proficiency in Persian is beneficial, annotators with no\nknowledge of Persian are also able to add annotations to the corpus.",
            "journal ref": "Proceedings of the 30th ACM Conference on Hypertext and Social\n  Media - HT 2019, 49-53. Hof, Germany: ACM Press",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12673v2",
            "date": "2019-09-27 13:27:53+00:00",
            "title": "A Constructive Prediction of the Generalization Error Across Scales",
            "authors": [
                "Jonathan S. Rosenfeld",
                "Amir Rosenfeld",
                "Yonatan Belinkov",
                "Nir Shavit"
            ],
            "abstract": "The dependency of the generalization error of neural networks on model and\ndataset size is of critical importance both in practice and for understanding\nthe theory of neural networks. Nevertheless, the functional form of this\ndependency remains elusive. In this work, we present a functional form which\napproximates well the generalization error in practice. Capitalizing on the\nsuccessful concept of model scaling (e.g., width, depth), we are able to\nsimultaneously construct such a form and specify the exact models which can\nattain it across model/data scales. Our construction follows insights obtained\nfrom observations conducted over a range of model/data scales, in various model\ntypes and datasets, in vision and language tasks. We show that the form both\nfits the observations well across scales, and provides accurate predictions\nfrom small- to large-scale models and data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.12868v1",
            "date": "2019-09-27 18:40:23+00:00",
            "title": "Automatically Learning Data Augmentation Policies for Dialogue Tasks",
            "authors": [
                "Tong Niu",
                "Mohit Bansal"
            ],
            "abstract": "Automatic data augmentation (AutoAugment) (Cubuk et al., 2019) searches for\noptimal perturbation policies via a controller trained using performance\nrewards of a sampled policy on the target task, hence reducing data-level model\nbias. While being a powerful algorithm, their work has focused on computer\nvision tasks, where it is comparatively easy to apply imperceptible\nperturbations without changing an image's semantic meaning. In our work, we\nadapt AutoAugment to automatically discover effective perturbation policies for\nnatural language processing (NLP) tasks such as dialogue generation. We start\nwith a pool of atomic operations that apply subtle semantic-preserving\nperturbations to the source inputs of a dialogue task (e.g., different POS-tag\ntypes of stopword dropout, grammatical errors, and paraphrasing). Next, we\nallow the controller to learn more complex augmentation policies by searching\nover the space of the various combinations of these atomic operations.\nMoreover, we also explore conditioning the controller on the source inputs of\nthe target task, since certain strategies may not apply to inputs that do not\ncontain that strategy's required linguistic features. Empirically, we\ndemonstrate that both our input-agnostic and input-aware controllers discover\nuseful data augmentation policies, and achieve significant improvements over\nthe previous state-of-the-art, including trained on manually-designed policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.12940v4",
            "date": "2019-09-11 04:22:20+00:00",
            "title": "Hope Speech Detection: A Computational Analysis of the Voice of Peace",
            "authors": [
                "Shriphani Palakodety",
                "Ashiqur R. KhudaBukhsh",
                "Jaime G. Carbonell"
            ],
            "abstract": "The recent Pulwama terror attack (February 14, 2019, Pulwama, Kashmir)\ntriggered a chain of escalating events between India and Pakistan adding\nanother episode to their 70-year-old dispute over Kashmir. The present era of\nubiquitious social media has never seen nuclear powers closer to war. In this\npaper, we analyze this evolving international crisis via a substantial corpus\nconstructed using comments on YouTube videos (921,235 English comments posted\nby 392,460 users out of 2.04 million overall comments by 791,289 users on 2,890\nvideos). Our main contributions in the paper are three-fold. First, we present\nan observation that polyglot word-embeddings reveal precise and accurate\nlanguage clusters, and subsequently construct a document\nlanguage-identification technique with negligible annotation requirements. We\ndemonstrate the viability and utility across a variety of data sets involving\nseveral low-resource languages. Second, we present an analysis on temporal\ntrends of pro-peace and pro-war intent observing that when tensions between the\ntwo nations were at their peak, pro-peace intent in the corpus was at its\nhighest point. Finally, in the context of heated discussions in a politically\ntense situation where two nations are at the brink of a full-fledged war, we\nargue the importance of automatic identification of user-generated web content\nthat can diffuse hostility and address this prediction task, dubbed\n\\emph{hope-speech detection}.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1909.13037v1",
            "date": "2019-09-28 06:48:47+00:00",
            "title": "Self-Attention Transducers for End-to-End Speech Recognition",
            "authors": [
                "Zhengkun Tian",
                "Jiangyan Yi",
                "Jianhua Tao",
                "Ye Bai",
                "Zhengqi Wen"
            ],
            "abstract": "Recurrent neural network transducers (RNN-T) have been successfully applied\nin end-to-end speech recognition. However, the recurrent structure makes it\ndifficult for parallelization . In this paper, we propose a self-attention\ntransducer (SA-T) for speech recognition. RNNs are replaced with self-attention\nblocks, which are powerful to model long-term dependencies inside sequences and\nable to be efficiently parallelized. Furthermore, a path-aware regularization\nis proposed to assist SA-T to learn alignments and improve the performance.\nAdditionally, a chunk-flow mechanism is utilized to achieve online decoding.\nAll experiments are conducted on a Mandarin Chinese dataset AISHELL-1. The\nresults demonstrate that our proposed approach achieves a 21.3% relative\nreduction in character error rate compared with the baseline RNN-T. In\naddition, the SA-T with chunk-flow mechanism can perform online decoding with\nonly a little degradation of the performance.",
            "journal ref": "Proc. Interspeech 2019, 4395-4399",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.13070v2",
            "date": "2019-09-28 11:16:50+00:00",
            "title": "Emirati-Accented Speaker Identification in Stressful Talking Conditions",
            "authors": [
                "Ismail Shahin",
                "Ali Bou Nassif"
            ],
            "abstract": "This research is dedicated to improving text-independent Emirati-accented\nspeaker identification performance in stressful talking conditions using three\ndistinct classifiers: First-Order Hidden Markov Models (HMM1s), Second-Order\nHidden Markov Models (HMM2s), and Third-Order Hidden Markov Models (HMM3s). The\ndatabase that has been used in this work was collected from 25 per gender\nEmirati native speakers uttering eight widespread Emirati sentences in each of\nneutral, shouted, slow, loud, soft, and fast talking conditions. The extracted\nfeatures of the captured database are called Mel-Frequency Cepstral\nCoefficients (MFCCs). Based on HMM1s, HMM2s, and HMM3s, average\nEmirati-accented speaker identification accuracy in stressful conditions is\n58.6%, 61.1%, and 65.0%, respectively. The achieved average speaker\nidentification accuracy in stressful conditions based on HMM3s is so similar to\nthat attained in subjective assessment by human listeners.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1909.13104v2",
            "date": "2019-09-28 14:32:46+00:00",
            "title": "Attention-based method for categorizing different types of online harassment language",
            "authors": [
                "Christos Karatsalos",
                "Yannis Panagiotakis"
            ],
            "abstract": "In the era of social media and networking platforms, Twitter has been doomed\nfor abuse and harassment toward users specifically women. Monitoring the\ncontents including sexism and sexual harassment in traditional media is easier\nthan monitoring on the online social media platforms like Twitter, because of\nthe large amount of user generated content in these media. So, the research\nabout the automated detection of content containing sexual or racist harassment\nis an important issue and could be the basis for removing that content or\nflagging it for human evaluation. Previous studies have been focused on\ncollecting data about sexism and racism in very broad terms. However, there is\nno much study focusing on different types of online harassment attracting\nnatural language processing techniques. In this work, we present an\nmulti-attention based approach for the detection of different types of\nharassment in tweets. Our approach is based on the Recurrent Neural Networks\nand particularly we are using a deep, classification specific multi-attention\nmechanism. Moreover, we tackle the problem of imbalanced data, using a\nback-translation method. Finally, we present a comparison between different\napproaches based on the Recurrent Neural Networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13193v1",
            "date": "2019-09-29 02:56:24+00:00",
            "title": "Gated Task Interaction Framework for Multi-task Sequence Tagging",
            "authors": [
                "Isaac K. E. Ampomah",
                "Sally McClean",
                "Zhiwei Lin",
                "Glenn Hawe"
            ],
            "abstract": "Recent studies have shown that neural models can achieve high performance on\nseveral sequence labelling/tagging problems without the explicit use of\nlinguistic features such as part-of-speech (POS) tags. These models are trained\nonly using the character-level and the word embedding vectors as inputs. Others\nhave shown that linguistic features can improve the performance of neural\nmodels on tasks such as chunking and named entity recognition (NER). However,\nthe change in performance depends on the degree of semantic relatedness between\nthe linguistic features and the target task; in some instances, linguistic\nfeatures can have a negative impact on performance. This paper presents an\napproach to jointly learn these linguistic features along with the target\nsequence labelling tasks with a new multi-task learning (MTL) framework called\nGated Tasks Interaction (GTI) network for solving multiple sequence tagging\ntasks. The GTI network exploits the relations between the multiple tasks via\nneural gate modules. These gate modules control the flow of information between\nthe different tasks. Experiments on benchmark datasets for chunking and NER\nshow that our framework outperforms other competitive baselines trained with\nand without external training resources.",
            "journal ref": "2019 International Joint Conference on Neural Networks (IJCNN)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.13315v1",
            "date": "2019-09-29 16:43:30+00:00",
            "title": "Lifelong Neural Topic Learning in Contextualized Autoregressive Topic Models of Language via Informative Transfers",
            "authors": [
                "Yatin Chaudhary",
                "Pankaj Gupta",
                "Thomas Runkler"
            ],
            "abstract": "Topic models such as LDA, DocNADE, iDocNADEe have been popular in document\nanalysis. However, the traditional topic models have several limitations\nincluding: (1) Bag-of-words (BoW) assumption, where they ignore word ordering,\n(2) Data sparsity, where the application of topic models is challenging due to\nlimited word co-occurrences, leading to incoherent topics and (3) No Continuous\nLearning framework for topic learning in lifelong fashion, exploiting\nhistorical knowledge (or latent topics) and minimizing catastrophic forgetting.\nThis thesis focuses on addressing the above challenges within neural topic\nmodeling framework. We propose: (1) Contextualized topic model that combines a\ntopic and a language model and introduces linguistic structures (such as word\nordering, syntactic and semantic features, etc.) in topic modeling, (2) A novel\nlifelong learning mechanism into neural topic modeling framework to demonstrate\ncontinuous learning in sequential document collections and minimizing\ncatastrophic forgetting. Additionally, we perform a selective data augmentation\nto alleviate the need for complete historical corpora during data hallucination\nor replay.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1909.13447v1",
            "date": "2019-09-30 04:15:59+00:00",
            "title": "DiPCo -- Dinner Party Corpus",
            "authors": [
                "Maarten Van Segbroeck",
                "Ahmed Zaid",
                "Ksenia Kutsenko",
                "Cirenia Huerta",
                "Tinh Nguyen",
                "Xuewen Luo",
                "Bj\u00f6rn Hoffmeister",
                "Jan Trmal",
                "Maurizio Omologo",
                "Roland Maas"
            ],
            "abstract": "We present a speech data corpus that simulates a \"dinner party\" scenario\ntaking place in an everyday home environment. The corpus was created by\nrecording multiple groups of four Amazon employee volunteers having a natural\nconversation in English around a dining table. The participants were recorded\nby a single-channel close-talk microphone and by five far-field 7-microphone\narray devices positioned at different locations in the recording room. The\ndataset contains the audio recordings and human labeled transcripts of a total\nof 10 sessions with a duration between 15 and 45 minutes. The corpus was\ncreated to advance in the field of noise robust and distant speech processing\nand is intended to serve as a public research and benchmarking data set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.13456v1",
            "date": "2019-09-30 05:03:25+00:00",
            "title": "Improving Textual Network Learning with Variational Homophilic Embeddings",
            "authors": [
                "Wenlin Wang",
                "Chenyang Tao",
                "Zhe Gan",
                "Guoyin Wang",
                "Liqun Chen",
                "Xinyuan Zhang",
                "Ruiyi Zhang",
                "Qian Yang",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "The performance of many network learning applications crucially hinges on the\nsuccess of network embedding algorithms, which aim to encode rich network\ninformation into low-dimensional vertex-based vector representations. This\npaper considers a novel variational formulation of network embeddings, with\nspecial focus on textual networks. Different from most existing methods that\noptimize a discriminative objective, we introduce Variational Homophilic\nEmbedding (VHE), a fully generative model that learns network embeddings by\nmodeling the semantic (textual) information with a variational autoencoder,\nwhile accounting for the structural (topology) information through a novel\nhomophilic prior design. Homophilic vertex embeddings encourage similar\nembedding vectors for related (connected) vertices. The proposed VHE promises\nbetter generalization for downstream tasks, robustness to incomplete\nobservations, and the ability to generalize to unseen vertices. Extensive\nexperiments on real-world networks, for multiple tasks, demonstrate that the\nproposed method consistently achieves superior performance relative to\ncompeting state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.13537v1",
            "date": "2019-09-30 09:04:16+00:00",
            "title": "Embeddings for DNN speaker adaptive training",
            "authors": [
                "Joanna Rownicka",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "In this work, we investigate the use of embeddings for speaker-adaptive\ntraining of DNNs (DNN-SAT) focusing on a small amount of adaptation data per\nspeaker. DNN-SAT can be viewed as learning a mapping from each embedding to\ntransformation parameters that are applied to the shared parameters of the DNN.\nWe investigate different approaches to applying these transformations, and find\nthat with a good training strategy, a multi-layer adaptation network applied to\nall hidden layers is no more effective than a single linear layer acting on the\nembeddings to transform the input features. In the second part of our work, we\nevaluate different embeddings (i-vectors, x-vectors and deep CNN embeddings) in\nan additional speaker recognition task in order to gain insight into what\nshould characterize an embedding for DNN-SAT. We find the performance for\nspeaker recognition of a given representation is not correlated with its ASR\nperformance; in fact, ability to capture more speech attributes than just\nspeaker identity was the most important characteristic of the embeddings for\nefficient DNN-SAT ASR. Our best models achieved relative WER gains of 4% and 9%\nover DNN baselines using speaker-level cepstral mean normalisation (CMN), and a\nfully speaker-independent model, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.13695v1",
            "date": "2019-09-30 13:42:06+00:00",
            "title": "Non-native Speaker Verification for Spoken Language Assessment",
            "authors": [
                "Linlin Wang",
                "Yu Wang",
                "Mark J. F. Gales"
            ],
            "abstract": "Automatic spoken language assessment systems are becoming more popular in\norder to handle increasing interests in second language learning. One challenge\nfor these systems is to detect malpractice. Malpractice can take a range of\nforms, this paper focuses on detecting when a candidate attempts to impersonate\nanother in a speaking test. This form of malpractice is closely related to\nspeaker verification, but applied in the specific domain of spoken language\nassessment. Advanced speaker verification systems, which leverage deep-learning\napproaches to extract speaker representations, have been successfully applied\nto a range of native speaker verification tasks. These systems are explored for\nnon-native spoken English data in this paper. The data used for speaker\nenrolment and verification is mainly taken from the BULATS test, which assesses\nEnglish language skills for business. Performance of systems trained on\nrelatively limited amounts of BULATS data, and standard large speaker\nverification corpora, is compared. Experimental results on large-scale test\nsets with millions of trials show that the best performance is achieved by\nadapting the imported model to non-native data. Breakdown of impostor trials\nacross different first languages (L1s) and grades is analysed, which shows that\ninter-L1 impostors are more challenging for speaker verification systems.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.13714v1",
            "date": "2019-09-20 00:00:41+00:00",
            "title": "Towards Multimodal Understanding of Passenger-Vehicle Interactions in Autonomous Vehicles: Intent/Slot Recognition Utilizing Audio-Visual Data",
            "authors": [
                "Eda Okur",
                "Shachi H Kumar",
                "Saurav Sahay",
                "Lama Nachman"
            ],
            "abstract": "Understanding passenger intents from spoken interactions and car's vision\n(both inside and outside the vehicle) are important building blocks towards\ndeveloping contextual dialog systems for natural interactions in autonomous\nvehicles (AV). In this study, we continued exploring AMIE (Automated-vehicle\nMultimodal In-cabin Experience), the in-cabin agent responsible for handling\ncertain multimodal passenger-vehicle interactions. When the passengers give\ninstructions to AMIE, the agent should parse such commands properly considering\navailable three modalities (language/text, audio, video) and trigger the\nappropriate functionality of the AV system. We had collected a multimodal\nin-cabin dataset with multi-turn dialogues between the passengers and AMIE\nusing a Wizard-of-Oz scheme via realistic scavenger hunt game. In our previous\nexplorations, we experimented with various RNN-based models to detect\nutterance-level intents (set destination, change route, go faster, go slower,\nstop, park, pull over, drop off, open door, and others) along with intent\nkeywords and relevant slots (location, position/direction, object,\ngesture/gaze, time-guidance, person) associated with the action to be performed\nin our AV scenarios. In this recent work, we propose to discuss the benefits of\nmultimodal understanding of in-cabin utterances by incorporating\nverbal/language input (text and speech embeddings) together with the\nnon-verbal/acoustic and visual input from inside and outside the vehicle (i.e.,\npassenger gestures and gaze from in-cabin video stream, referred objects\noutside of the vehicle from the road view camera stream). Our experimental\nresults outperformed text-only baselines and with multimodality, we achieved\nimproved performances for utterance-level intent detection and slot filling.",
            "journal ref": "Proceedings of the 23rd Workshop on the Semantics and Pragmatics\n  of Dialogue (SEMDIAL), pp. 213-215, London, United Kingdom, September 2019",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1909.13759v1",
            "date": "2019-09-30 14:49:24+00:00",
            "title": "Acoustic Model Adaptation from Raw Waveforms with SincNet",
            "authors": [
                "Joachim Fainberg",
                "Ond\u0159ej Klejch",
                "Erfan Loweimi",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "Raw waveform acoustic modelling has recently gained interest due to neural\nnetworks' ability to learn feature extraction, and the potential for finding\nbetter representations for a given scenario than hand-crafted features. SincNet\nhas been proposed to reduce the number of parameters required in raw-waveform\nmodelling, by restricting the filter functions, rather than having to learn\nevery tap of each filter. We study the adaptation of the SincNet filter\nparameters from adults' to children's speech, and show that the\nparameterisation of the SincNet layer is well suited for adaptation in\npractice: we can efficiently adapt with a very small number of parameters,\nproducing error rates comparable to techniques using orders of magnitude more\nparameters.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.13788v3",
            "date": "2019-09-30 15:30:00+00:00",
            "title": "Revisiting Self-Training for Neural Sequence Generation",
            "authors": [
                "Junxian He",
                "Jiatao Gu",
                "Jiajun Shen",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Self-training is one of the earliest and simplest semi-supervised methods.\nThe key idea is to augment the original labeled dataset with unlabeled data\npaired with the model's prediction (i.e. the pseudo-parallel data). While\nself-training has been extensively studied on classification problems, in\ncomplex sequence generation tasks (e.g. machine translation) it is still\nunclear how self-training works due to the compositionality of the target\nspace. In this work, we first empirically show that self-training is able to\ndecently improve the supervised baseline on neural sequence generation tasks.\nThrough careful examination of the performance gains, we find that the\nperturbation on the hidden states (i.e. dropout) is critical for self-training\nto benefit from the pseudo-parallel data, which acts as a regularizer and\nforces the model to yield close predictions for similar unlabeled inputs. Such\neffect helps the model correct some incorrect predictions on unlabeled data. To\nfurther encourage this mechanism, we propose to inject noise to the input\nspace, resulting in a \"noisy\" version of self-training. Empirical study on\nstandard machine translation and text summarization benchmarks shows that noisy\nself-training is able to effectively utilize unlabeled data and improve the\nperformance of the supervised baseline by a large margin.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.13790v1",
            "date": "2019-09-30 15:35:07+00:00",
            "title": "Incremental processing of noisy user utterances in the spoken language understanding task",
            "authors": [
                "Stefan Constantin",
                "Jan Niehues",
                "Alex Waibel"
            ],
            "abstract": "The state-of-the-art neural network architectures make it possible to create\nspoken language understanding systems with high quality and fast processing\ntime. One major challenge for real-world applications is the high latency of\nthese systems caused by triggered actions with high executions times. If an\naction can be separated into subactions, the reaction time of the systems can\nbe improved through incremental processing of the user utterance and starting\nsubactions while the utterance is still being uttered. In this work, we present\na model-agnostic method to achieve high quality in processing incrementally\nproduced partial utterances. Based on clean and noisy versions of the ATIS\ndataset, we show how to create datasets with our method to create low-latency\nnatural language understanding components. We get improvements of up to 47.91\nabsolute percentage points in the metric F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00054v1",
            "date": "2019-09-30 18:40:59+00:00",
            "title": "Weakly Supervised Attention Networks for Fine-Grained Opinion Mining and Public Health",
            "authors": [
                "Giannis Karamanolakis",
                "Daniel Hsu",
                "Luis Gravano"
            ],
            "abstract": "In many review classification applications, a fine-grained analysis of the\nreviews is desirable, because different segments (e.g., sentences) of a review\nmay focus on different aspects of the entity in question. However, training\nsupervised models for segment-level classification requires segment labels,\nwhich may be more difficult or expensive to obtain than review labels. In this\npaper, we employ Multiple Instance Learning (MIL) and use only weak supervision\nin the form of a single label per review. First, we show that when\ninappropriate MIL aggregation functions are used, then MIL-based networks are\noutperformed by simpler baselines. Second, we propose a new aggregation\nfunction based on the sigmoid attention mechanism and show that our proposed\nmodel outperforms the state-of-the-art models for segment-level sentiment\nclassification (by up to 9.8% in F1). Finally, we highlight the importance of\nfine-grained predictions in an important public-health application: finding\nactionable reports of foodborne illness. We show that our model achieves 48.6%\nhigher recall compared to previous models, thus increasing the chance of\nidentifying previously unknown foodborne outbreaks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00084v1",
            "date": "2019-09-30 20:20:48+00:00",
            "title": "Contextual Graph Attention for Answering Logical Queries over Incomplete Knowledge Graphs",
            "authors": [
                "Gengchen Mai",
                "Krzysztof Janowicz",
                "Bo Yan",
                "Rui Zhu",
                "Ling Cai",
                "Ni Lao"
            ],
            "abstract": "Recently, several studies have explored methods for using KG embedding to\nanswer logical queries. These approaches either treat embedding learning and\nquery answering as two separated learning tasks, or fail to deal with the\nvariability of contributions from different query paths. We proposed to\nleverage a graph attention mechanism to handle the unequal contribution of\ndifferent query paths. However, commonly used graph attention assumes that the\ncenter node embedding is provided, which is unavailable in this task since the\ncenter node is to be predicted. To solve this problem we propose a multi-head\nattention-based end-to-end logical query answering model, called Contextual\nGraph Attention model(CGA), which uses an initial neighborhood aggregation\nlayer to generate the center embedding, and the whole model is trained jointly\non the original KG structure as well as the sampled query-answer pairs. We also\nintroduce two new datasets, DB18 and WikiGeo19, which are rather large in size\ncompared to the existing datasets and contain many more relation types, and use\nthem to evaluate the performance of the proposed model. Our result shows that\nthe proposed CGA with fewer learnable parameters consistently outperforms the\nbaseline models on both datasets as well as Bio dataset.",
            "journal ref": "K-CAP 2019, Nov. 19 - 21, 2019, Marina del Rey, CA, USA",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00290v1",
            "date": "2019-10-01 10:26:08+00:00",
            "title": "Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks",
            "authors": [
                "Mokanarangan Thayaparan",
                "Marco Valentino",
                "Viktor Schlegel",
                "Andre Freitas"
            ],
            "abstract": "Recent advances in reading comprehension have resulted in models that surpass\nhuman performance when the answer is contained in a single, continuous passage\nof text. However, complex Question Answering (QA) typically requires multi-hop\nreasoning - i.e. the integration of supporting facts from different sources, to\ninfer the correct answer. This paper proposes Document Graph Network (DGN), a\nmessage passing architecture for the identification of supporting facts over a\ngraph-structured representation of text. The evaluation on HotpotQA shows that\nDGN obtains competitive results when compared to a reading comprehension\nbaseline operating on raw text, confirming the relevance of structured\nrepresentations for supporting multi-hop reasoning.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.00292v2",
            "date": "2019-10-01 10:28:32+00:00",
            "title": "Generalization in Generation: A closer look at Exposure Bias",
            "authors": [
                "Florian Schmidt"
            ],
            "abstract": "Exposure bias refers to the train-test discrepancy that seemingly arises when\nan autoregressive generative model uses only ground-truth contexts at training\ntime but generated ones at test time. We separate the contributions of the\nmodel and the learning framework to clarify the debate on consequences and\nreview proposed counter-measures. In this light, we argue that generalization\nis the underlying property to address and propose unconditional generation as\nits fundamental benchmark. Finally, we combine latent variable modeling with a\nrecent formulation of exploration in reinforcement learning to obtain a\nrigorous handling of true and generated contexts. Results on language modeling\nand variational sentence auto-encoding confirm the model's generalization\ncapability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00314v2",
            "date": "2019-10-01 11:47:36+00:00",
            "title": "BioNLP-OST 2019 RDoC Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions",
            "authors": [
                "Yatin Chaudhary",
                "Pankaj Gupta",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper presents our system details and results of participation in the\nRDoC Tasks of BioNLP-OST 2019. Research Domain Criteria (RDoC) construct is a\nmulti-dimensional and broad framework to describe mental health disorders by\ncombining knowledge from genomics to behaviour. Non-availability of RDoC\nlabelled dataset and tedious labelling process hinders the use of RDoC\nframework to reach its full potential in Biomedical research community and\nHealthcare industry. Therefore, Task-1 aims at retrieval and ranking of PubMed\nabstracts relevant to a given RDoC construct and Task-2 aims at extraction of\nthe most relevant sentence from a given PubMed abstract. We investigate (1)\nattention based supervised neural topic model and SVM for retrieval and ranking\nof PubMed abstracts and, further utilize BM25 and other relevance measures for\nre-ranking, (2) supervised and unsupervised sentence ranking models utilizing\nmulti-view representations comprising of query-aware attention-based sentence\nrepresentation (QAR), bag-of-words (BoW) and TF-IDF. Our best systems achieved\n1st rank and scored 0.86 mean average precision (mAP) and 0.58 macro average\naccuracy (MAA) in Task-1 and Task-2 respectively.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00330v1",
            "date": "2019-10-01 12:03:24+00:00",
            "title": "A Multi-Modal Feature Embedding Approach to Diagnose Alzheimer Disease from Spoken Language",
            "authors": [
                "S. Soroush Haj Zargarbashi",
                "Bagher Babaali"
            ],
            "abstract": "Introduction: Alzheimer's disease is a type of dementia in which early\ndiagnosis plays a major rule in the quality of treatment. Among new works in\nthe diagnosis of Alzheimer's disease, there are many of them analyzing the\nvoice stream acoustically, syntactically or both. The mostly used tools to\nperform these analysis usually include machine learning techniques. Objective:\nDesigning an automatic machine learning based diagnosis system will help in the\nprocedure of early detection. Also, systems, using noninvasive data are\npreferable. Methods: We used are classification system based on spoken\nlanguage. We use three (statistical and neural) approaches to classify audio\nsignals from spoken language into two classes of dementia and control. Result:\nThis work designs a multi-modal feature embedding on the spoken language audio\nsignal using three approaches; N-gram, i-vector, and x-vector. The evaluation\nof the system is done on the cookie picture description task from Pitt Corpus\ndementia bank with the accuracy of 83:6",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00334v1",
            "date": "2019-10-01 12:18:42+00:00",
            "title": "Towards French Smart Building Code: Compliance Checking Based on Semantic Rules",
            "authors": [
                "Nicolas Bus",
                "Ana Roxin",
                "Guillaume Picinbono",
                "Muhammad Fahad"
            ],
            "abstract": "Manually checking models for compliance against building regulation is a\ntime-consuming task for architects and construction engineers. There is thus a\nneed for algorithms that process information from construction projects and\nreport non-compliant elements. Still automated code-compliance checking raises\nseveral obstacles. Building regulations are usually published as human readable\ntexts and their content is often ambiguous or incomplete. Also, the vocabulary\nused for expressing such regulations is very different from the vocabularies\nused to express Building Information Models (BIM). Furthermore, the high level\nof details associated to BIM-contained geometries induces complex calculations.\nFinally, the level of complexity of the IFC standard also hinders the\nautomation of IFC processing tasks. Model chart, formal rules and\npre-processors approach allows translating construction regulations into\nsemantic queries. We further demonstrate the usefulness of this approach\nthrough several use cases. We argue our approach is a step forward in bridging\nthe gap between regulation texts and automated checking algorithms. Finally\nwith the recent building ontology BOT recommended by the W3C Linked Building\nData Community Group, we identify perspectives for standardizing and extending\nour approach.",
            "journal ref": "Linked Data for Architecture and Construction (LDAC'2018), Jun\n  2018, Londres, United Kingdom",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.00421v4",
            "date": "2019-10-01 14:19:40+00:00",
            "title": "Global Voices: Crossing Borders in Automatic News Summarization",
            "authors": [
                "Khanh Nguyen",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We construct Global Voices, a multilingual dataset for evaluating\ncross-lingual summarization methods. We extract social-network descriptions of\nGlobal Voices news articles to cheaply collect evaluation data for into-English\nand from-English summarization in 15 languages. Especially, for the\ninto-English summarization task, we crowd-source a high-quality evaluation\ndataset based on guidelines that emphasize accuracy, coverage, and\nunderstandability. To ensure the quality of this dataset, we collect human\nratings to filter out bad summaries, and conduct a survey on humans, which\nshows that the remaining summaries are preferred over the social-network\nsummaries. We study the effect of translation quality in cross-lingual\nsummarization, comparing a translate-then-summarize approach with several\nbaselines. Our results highlight the limitations of the ROUGE metric that are\noverlooked in monolingual summarization. Our dataset is available for download\nat https://forms.gle/gpkJDT6RJWHM1Ztz9 .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00565v1",
            "date": "2019-10-01 17:40:49+00:00",
            "title": "Domain Expansion in DNN-based Acoustic Models for Robust Speech Recognition",
            "authors": [
                "Shahram Ghorbani",
                "Soheil Khorram",
                "John H. L. Hansen"
            ],
            "abstract": "Training acoustic models with sequentially incoming data -- while both\nleveraging new data and avoiding the forgetting effect-- is an essential\nobstacle to achieving human intelligence level in speech recognition. An\nobvious approach to leverage data from a new domain (e.g., new accented speech)\nis to first generate a comprehensive dataset of all domains, by combining all\navailable data, and then use this dataset to retrain the acoustic models.\nHowever, as the amount of training data grows, storing and retraining on such a\nlarge-scale dataset becomes practically impossible. To deal with this problem,\nin this study, we study several domain expansion techniques which exploit only\nthe data of the new domain to build a stronger model for all domains. These\ntechniques are aimed at learning the new domain with a minimal forgetting\neffect (i.e., they maintain original model performance). These techniques\nmodify the adaptation procedure by imposing new constraints including (1)\nweight constraint adaptation (WCA): keeping the model parameters close to the\noriginal model parameters; (2) elastic weight consolidation (EWC): slowing down\ntraining for parameters that are important for previously established domains;\n(3) soft KL-divergence (SKLD): restricting the KL-divergence between the\noriginal and the adapted model output distributions; and (4) hybrid SKLD-EWC:\nincorporating both SKLD and EWC constraints. We evaluate these techniques in an\naccent adaptation task in which we adapt a deep neural network (DNN) acoustic\nmodel trained with native English to three different English accents:\nAustralian, Hispanic, and Indian. The experimental results show that SKLD\nsignificantly outperforms EWC, and EWC works better than WCA. The hybrid\nSKLD-EWC technique results in the best overall performance.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.00702v1",
            "date": "2019-10-01 22:34:40+00:00",
            "title": "TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction",
            "authors": [
                "Ling Cai",
                "Bo Yan",
                "Gengchen Mai",
                "Krzysztof Janowicz",
                "Rui Zhu"
            ],
            "abstract": "Link prediction is an important and frequently studied task that contributes\nto an understanding of the structure of knowledge graphs (KGs) in statistical\nrelational learning. Inspired by the success of graph convolutional networks\n(GCN) in modeling graph data, we propose a unified GCN framework, named\nTransGCN, to address this task, in which relation and entity embeddings are\nlearned simultaneously. To handle heterogeneous relations in KGs, we introduce\na novel way of representing heterogeneous neighborhood by introducing\ntransformation assumptions on the relationship between the subject, the\nrelation, and the object of a triple. Specifically, a relation is treated as a\ntransformation operator transforming a head entity to a tail entity. Both\ntranslation assumption in TransE and rotation assumption in RotatE are explored\nin our framework. Additionally, instead of only learning entity embeddings in\nthe convolution-based encoder while learning relation embeddings in the decoder\nas done by the state-of-art models, e.g., R-GCN, the TransGCN framework trains\nrelation embeddings and entity embeddings simultaneously during the graph\nconvolution operation, thus having fewer parameters compared with R-GCN.\nExperiments show that our models outperform the-state-of-arts methods on both\nFB15K-237 and WN18RR.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00716v1",
            "date": "2019-10-01 23:41:28+00:00",
            "title": "State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention With Dilated 1D Convolutions",
            "authors": [
                "Kyu J. Han",
                "Ramon Prieto",
                "Kaixing Wu",
                "Tao Ma"
            ],
            "abstract": "Self-attention has been a huge success for many downstream tasks in NLP,\nwhich led to exploration of applying self-attention to speech problems as well.\nThe efficacy of self-attention in speech applications, however, seems not fully\nblown yet since it is challenging to handle highly correlated speech frames in\nthe context of self-attention. In this paper we propose a new neural network\nmodel architecture, namely multi-stream self-attention, to address the issue\nthus make the self-attention mechanism more effective for speech recognition.\nThe proposed model architecture consists of parallel streams of self-attention\nencoders, and each stream has layers of 1D convolutions with dilated kernels\nwhose dilation rates are unique given stream, followed by a self-attention\nlayer. The self-attention mechanism in each stream pays attention to only one\nresolution of input speech frames and the attentive computation can be more\nefficient. In a later stage, outputs from all the streams are concatenated then\nlinearly projected to the final embedding. By stacking the proposed\nmulti-stream self-attention encoder blocks and rescoring the resultant lattices\nwith neural network language models, we achieve the word error rate of 2.2% on\nthe test-clean dataset of the LibriSpeech corpus, the best number reported thus\nfar on the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00748v2",
            "date": "2019-10-02 02:24:12+00:00",
            "title": "A Deep Factorization of Style and Structure in Fonts",
            "authors": [
                "Nikita Srivatsan",
                "Jonathan T. Barron",
                "Dan Klein",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We propose a deep factorization model for typographic analysis that\ndisentangles content from style. Specifically, a variational inference\nprocedure factors each training glyph into the combination of a\ncharacter-specific content embedding and a latent font-specific style variable.\nThe underlying generative model combines these factors through an asymmetric\ntranspose convolutional process to generate the image of the glyph itself. When\ntrained on corpora of fonts, our model learns a manifold over font styles that\ncan be used to analyze or reconstruct new, unseen fonts. On the task of\nreconstructing missing glyphs from an unknown font given only a small number of\nobservations, our model outperforms both a strong nearest neighbors baseline\nand a state-of-the-art discriminative model from prior work.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00795v2",
            "date": "2019-10-02 06:42:57+00:00",
            "title": "Speech-to-speech Translation between Untranscribed Unknown Languages",
            "authors": [
                "Andros Tjandra",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "In this paper, we explore a method for training speech-to-speech translation\ntasks without any transcription or linguistic supervision. Our proposed method\nconsists of two steps: First, we train and generate discrete representation\nwith unsupervised term discovery with a discrete quantized autoencoder. Second,\nwe train a sequence-to-sequence model that directly maps the source language\nspeech to the target language's discrete representation. Our proposed method\ncan directly generate target speech without any auxiliary or pre-training steps\nwith a source or target transcription. To the best of our knowledge, this is\nthe first work that performed pure speech-to-speech translation between\nuntranscribed unknown languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.00998v1",
            "date": "2019-10-02 14:57:55+00:00",
            "title": "SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders",
            "authors": [
                "Peter J. Liu",
                "Yu-An Chung",
                "Jie Ren"
            ],
            "abstract": "We propose an end-to-end neural model for zero-shot abstractive text\nsummarization of paragraphs, and introduce a benchmark task, ROCSumm, based on\nROCStories, a subset for which we collected human summaries. In this task,\nfive-sentence stories (paragraphs) are summarized with one sentence, using\nhuman summaries only for evaluation. We show results for extractive and human\nbaselines to demonstrate a large abstractive gap in performance. Our model,\nSummAE, consists of a denoising auto-encoder that embeds sentences and\nparagraphs in a common space, from which either can be decoded. Summaries for\nparagraphs are generated by decoding a sentence from the paragraph\nrepresentations. We find that traditional sequence-to-sequence auto-encoders\nfail to produce good summaries and describe how specific architectural choices\nand pre-training techniques can significantly improve performance,\noutperforming extractive baselines. The data, training, evaluation code, and\nbest model weights are open-sourced.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01289v2",
            "date": "2019-10-03 03:19:55+00:00",
            "title": "Neural Zero-Inflated Quality Estimation Model For Automatic Speech Recognition System",
            "authors": [
                "Kai Fan",
                "Jiayi Wang",
                "Bo Li",
                "Shiliang Zhang",
                "Boxing Chen",
                "Niyu Ge",
                "Zhijie Yan"
            ],
            "abstract": "The performances of automatic speech recognition (ASR) systems are usually\nevaluated by the metric word error rate (WER) when the manually transcribed\ndata are provided, which are, however, expensively available in the real\nscenario. In addition, the empirical distribution of WER for most ASR systems\nusually tends to put a significant mass near zero, making it difficult to\nsimulate with a single continuous distribution. In order to address the two\nissues of ASR quality estimation (QE), we propose a novel neural zero-inflated\nmodel to predict the WER of the ASR result without transcripts. We design a\nneural zero-inflated beta regression on top of a bidirectional transformer\nlanguage model conditional on speech features (speech-BERT). We adopt the\npre-training strategy of token level mask language modeling for speech-BERT as\nwell, and further fine-tune with our zero-inflated layer for the mixture of\ndiscrete and continuous outputs. The experimental results show that our\napproach achieves better performance on WER prediction in the metrics of\nPearson and MAE, compared with most existed quality estimation algorithms for\nASR or machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01340v1",
            "date": "2019-10-03 07:56:52+00:00",
            "title": "TexTrolls: Identifying Russian Trolls on Twitter from a Textual Perspective",
            "authors": [
                "Bilal Ghanem",
                "Davide Buscaldi",
                "Paolo Rosso"
            ],
            "abstract": "The online new emerging suspicious users, that usually are called trolls, are\none of the main sources of hate, fake, and deceptive online messages. Some\nagendas are utilizing these harmful users to spread incitement tweets, and as a\nconsequence, the audience get deceived. The challenge in detecting such\naccounts is that they conceal their identities which make them disguised in\nsocial media, adding more difficulty to identify them using just their social\nnetwork information. Therefore, in this paper, we propose a text-based approach\nto detect the online trolls such as those that were discovered during the US\n2016 presidential elections. Our approach is mainly based on textual features\nwhich utilize thematic information, and profiling features to identify the\naccounts from their way of writing tweets. We deduced the thematic information\nin a unsupervised way and we show that coupling them with the textual features\nenhanced the performance of the proposed model. In addition, we find that the\nproposed profiling features perform the best comparing to the textual features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01442v2",
            "date": "2019-10-03 13:16:36+00:00",
            "title": "CLEVRER: CoLlision Events for Video REpresentation and Reasoning",
            "authors": [
                "Kexin Yi",
                "Chuang Gan",
                "Yunzhu Li",
                "Pushmeet Kohli",
                "Jiajun Wu",
                "Antonio Torralba",
                "Joshua B. Tenenbaum"
            ],
            "abstract": "The ability to reason about temporal and causal events from videos lies at\nthe core of human intelligence. Most video reasoning benchmarks, however, focus\non pattern recognition from complex visual and language input, instead of on\ncausal structure. We study the complementary problem, exploring the temporal\nand causal structures behind videos of objects with simple visual appearance.\nTo this end, we introduce the CoLlision Events for Video REpresentation and\nReasoning (CLEVRER), a diagnostic video dataset for systematic evaluation of\ncomputational models on a wide range of reasoning tasks. Motivated by the\ntheory of human casual judgment, CLEVRER includes four types of questions:\ndescriptive (e.g., \"what color\"), explanatory (\"what is responsible for\"),\npredictive (\"what will happen next\"), and counterfactual (\"what if\"). We\nevaluate various state-of-the-art models for visual reasoning on our benchmark.\nWhile these models thrive on the perception-based task (descriptive), they\nperform poorly on the causal tasks (explanatory, predictive and\ncounterfactual), suggesting that a principled approach for causal reasoning\nshould incorporate the capability of both perceiving complex visual and\nlanguage inputs, and understanding the underlying dynamics and causal\nrelations. We also study an oracle model that explicitly combines these\ncomponents via symbolic representations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.01709v1",
            "date": "2019-10-03 20:18:45+00:00",
            "title": "Semi-Supervised Generative Modeling for Controllable Speech Synthesis",
            "authors": [
                "Raza Habib",
                "Soroosh Mariooryad",
                "Matt Shannon",
                "Eric Battenberg",
                "RJ Skerry-Ryan",
                "Daisy Stanton",
                "David Kao",
                "Tom Bagby"
            ],
            "abstract": "We present a novel generative model that combines state-of-the-art neural\ntext-to-speech (TTS) with semi-supervised probabilistic latent variable models.\nBy providing partial supervision to some of the latent variables, we are able\nto force them to take on consistent and interpretable purposes, which\npreviously hasn't been possible with purely unsupervised TTS models. We\ndemonstrate that our model is able to reliably discover and control important\nbut rarely labelled attributes of speech, such as affect and speaking rate,\nwith as little as 1% (30 minutes) supervision. Even at such low supervision\nlevels we do not observe a degradation of synthesis quality compared to a\nstate-of-the-art baseline. Audio samples are available on the web.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01769v2",
            "date": "2019-10-04 01:01:26+00:00",
            "title": "Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data",
            "authors": [
                "Subhabrata Mukherjee",
                "Ahmed Hassan Awadallah"
            ],
            "abstract": "Recent advances in pre-training huge models on large amounts of text through\nself supervision have obtained state-of-the-art results in various natural\nlanguage processing tasks. However, these huge and expensive models are\ndifficult to use in practise for downstream tasks. Some recent efforts use\nknowledge distillation to compress these models. However, we see a gap between\nthe performance of the smaller student models as compared to that of the large\nteacher. In this work, we leverage large amounts of in-domain unlabeled\ntransfer data in addition to a limited amount of labeled training instances to\nbridge this gap for distilling BERT. We show that simple RNN based student\nmodels even with hard distillation can perform at par with the huge teachers\ngiven the transfer set. The student performance can be further improved with\nsoft distillation and leveraging teacher intermediate representations. We show\nthat our student models can compress the huge teacher by up to 26x while still\nmatching or even marginally exceeding the teacher performance in low-resource\nsettings with small amount of labeled data. Additionally, for the multilingual\nextension of this work with XtremeDistil (Mukherjee and Hassan Awadallah,\n2020), we demonstrate massive distillation of multilingual BERT-like teacher\nmodels by upto 35x in terms of parameter compression and 51x in terms of\nlatency speedup for batch inference while retaining 95% of its F1-score for NER\nover 41 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01990v1",
            "date": "2019-10-04 15:28:01+00:00",
            "title": "Detecting Deception in Political Debates Using Acoustic and Textual Features",
            "authors": [
                "Daniel Kopev",
                "Ahmed Ali",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "We present work on deception detection, where, given a spoken claim, we aim\nto predict its factuality. While previous work in the speech community has\nrelied on recordings from staged setups where people were asked to tell the\ntruth or to lie and their statements were recorded, here we use real-world\npolitical debates. Thanks to the efforts of fact-checking organizations, it is\npossible to obtain annotations for statements in the context of a political\ndiscourse as true, half-true, or false. Starting with such data from the\nCLEF-2018 CheckThat! Lab, which was limited to text, we performed alignment to\nthe corresponding videos, thus producing a multimodal dataset. We further\ndeveloped a multimodal deep-learning architecture for the task of deception\ndetection, which yielded sizable improvements over the state of the art for the\nCLEF-2018 Lab task 2. Our experiments show that the use of the acoustic signal\nconsistently helped to improve the performance compared to using textual and\nmetadata features only, based on several different evaluation measures. We\nrelease the new dataset to the research community, hoping to help advance the\noverall field of multimodal deception detection.",
            "journal ref": "ASRU-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02001v1",
            "date": "2019-10-04 15:50:30+00:00",
            "title": "Predicting the Role of Political Trolls in Social Media",
            "authors": [
                "Atanas Atanasov",
                "Gianmarco De Francisci Morales",
                "Preslav Nakov"
            ],
            "abstract": "We investigate the political roles of \"Internet trolls\" in social media.\nPolitical trolls, such as the ones linked to the Russian Internet Research\nAgency (IRA), have recently gained enormous attention for their ability to sway\npublic opinion and even influence elections. Analysis of the online traces of\ntrolls has shown different behavioral patterns, which target different slices\nof the population. However, this analysis is manual and labor-intensive, thus\nmaking it impractical as a first-response tool for newly-discovered troll\nfarms. In this paper, we show how to automate this analysis by using machine\nlearning in a realistic setting. In particular, we show how to classify trolls\naccording to their political role ---left, news feed, right--- by using\nfeatures extracted from social media, i.e., Twitter, in two scenarios: (i) in a\ntraditional supervised learning scenario, where labels for trolls are\navailable, and (ii) in a distant supervision scenario, where labels for trolls\nare not available, and we rely on more-commonly-available labels for news\noutlets mentioned by the trolls. Technically, we leverage the community\nstructure and the text of the messages in the online social network of trolls\nrepresented as a graph, from which we extract several types of learned\nrepresentations, i.e.,~embeddings, for the trolls. Experiments on the \"IRA\nRussian Troll\" dataset show that our methodology improves over the\nstate-of-the-art in the first scenario, while providing a compelling case for\nthe second scenario, which has not been explored in the literature thus far.",
            "journal ref": "CoNLL-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02029v3",
            "date": "2019-10-04 16:44:59+00:00",
            "title": "Talk2Nav: Long-Range Vision-and-Language Navigation with Dual Attention and Spatial Memory",
            "authors": [
                "Arun Balajee Vasudevan",
                "Dengxin Dai",
                "Luc Van Gool"
            ],
            "abstract": "The role of robots in society keeps expanding, bringing with it the necessity\nof interacting and communicating with humans. In order to keep such interaction\nintuitive, we provide automatic wayfinding based on verbal navigational\ninstructions. Our first contribution is the creation of a large-scale dataset\nwith verbal navigation instructions. To this end, we have developed an\ninteractive visual navigation environment based on Google Street View; we\nfurther design an annotation method to highlight mined anchor landmarks and\nlocal directions between them in order to help annotators formulate typical,\nhuman references to those. The annotation task was crowdsourced on the AMT\nplatform, to construct a new Talk2Nav dataset with $10,714$ routes. Our second\ncontribution is a new learning method. Inspired by spatial cognition research\non the mental conceptualization of navigational instructions, we introduce a\nsoft dual attention mechanism defined over the segmented language instructions\nto jointly extract two partial instructions -- one for matching the next\nupcoming visual landmark and the other for matching the local directions to the\nnext landmark. On the similar lines, we also introduce spatial memory scheme to\nencode the local directional transitions. Our work takes advantage of the\nadvance in two lines of research: mental formalization of verbal navigational\ninstructions and training neural network agents for automatic way finding.\nExtensive experiments show that our method significantly outperforms previous\nnavigation methods. For demo video, dataset and code, please refer to our\nproject page: https://www.trace.ethz.ch/publications/2019/talk2nav/index.html",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.02145v1",
            "date": "2019-10-03 10:43:43+00:00",
            "title": "Types for Parallel Complexity in the Pi-calculus",
            "authors": [
                "Patrick Baillot",
                "Alexis Ghyselen"
            ],
            "abstract": "Type systems as a way to control or analyze programs have been largely\nstudied in the context of functional programming languages. Some of those work\nallow to extract from a typing derivation for a program a complexity bound on\nthis program. We present how to adapt this result for parallel complexity in\nthe pi-calculus, as a model of concurrency and parallel communication. We study\ntwo notions of time complexity: the total computation time without parallelism\n(the work) and the computation time under maximal parallelism (the span). We\ndefine reduction relations in the pi-calculus to capture those two notions, and\nwe present two type systems from which one can extract a complexity bound on a\nprocess. The type systems are inspired by input/output types and size types,\nwith temporal information about communications.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1910.02187v3",
            "date": "2019-10-05 01:16:33+00:00",
            "title": "Dynamic Embedding on Textual Networks via a Gaussian Process",
            "authors": [
                "Pengyu Cheng",
                "Yitong Li",
                "Xinyuan Zhang",
                "Liqun Cheng",
                "David Carlson",
                "Lawrence Carin"
            ],
            "abstract": "Textual network embedding aims to learn low-dimensional representations of\ntext-annotated nodes in a graph. Prior work in this area has typically focused\non fixed graph structures; however, real-world networks are often dynamic. We\naddress this challenge with a novel end-to-end node-embedding model, called\nDynamic Embedding for Textual Networks with a Gaussian Process (DetGP). After\ntraining, DetGP can be applied efficiently to dynamic graphs without\nre-training or backpropagation. The learned representation of each node is a\ncombination of textual and structural embeddings. Because the structure is\nallowed to be dynamic, our method uses the Gaussian process to take advantage\nof its non-parametric properties. To use both local and global graph\nstructures, diffusion is used to model multiple hops between neighbors. The\nrelative importance of global versus local structure for the embeddings is\nlearned automatically. With the non-parametric nature of the Gaussian process,\nupdating the embeddings for a changed graph structure requires only a forward\npass through the learned model. Considering link prediction and node\nclassification, experiments demonstrate the empirical effectiveness of our\nmethod compared to baseline approaches. We further show that DetGP can be\nstraightforwardly and efficiently applied to dynamic textual networks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.02202v1",
            "date": "2019-10-05 03:23:45+00:00",
            "title": "Learning from Fact-checkers: Analysis and Generation of Fact-checking Language",
            "authors": [
                "Nguyen Vo",
                "Kyumin Lee"
            ],
            "abstract": "In fighting against fake news, many fact-checking systems comprised of\nhuman-based fact-checking sites (e.g., snopes.com and politifact.com) and\nautomatic detection systems have been developed in recent years. However,\nonline users still keep sharing fake news even when it has been debunked. It\nmeans that early fake news detection may be insufficient and we need another\ncomplementary approach to mitigate the spread of misinformation. In this paper,\nwe introduce a novel application of text generation for combating fake news. In\nparticular, we (1) leverage online users named \\emph{fact-checkers}, who cite\nfact-checking sites as credible evidences to fact-check information in public\ndiscourse; (2) analyze linguistic characteristics of fact-checking tweets; and\n(3) propose and build a deep learning framework to generate responses with\nfact-checking intention to increase the fact-checkers' engagement in\nfact-checking activities. Our analysis reveals that the fact-checkers tend to\nrefute misinformation and use formal language (e.g. few swear words and\nInternet slangs). Our framework successfully generates relevant responses, and\noutperforms competing models by achieving up to 30\\% improvements. Our\nqualitative study also confirms that the superiority of our generated responses\ncompared with responses generated from the existing models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02216v2",
            "date": "2019-10-05 05:51:45+00:00",
            "title": "JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation",
            "authors": [
                "Rajas Agashe",
                "Srinivasan Iyer",
                "Luke Zettlemoyer"
            ],
            "abstract": "Interactive programming with interleaved code snippet cells and natural\nlanguage markdown is recently gaining popularity in the form of Jupyter\nnotebooks, which accelerate prototyping and collaboration. To study code\ngeneration conditioned on a long context history, we present JuICe, a corpus of\n1.5 million examples with a curated test set of 3.7K instances based on online\nprogramming assignments. Compared with existing contextual code generation\ndatasets, JuICe provides refined human-curated data, open-domain code, and an\norder of magnitude more training data. Using JuICe, we train models for two\ntasks: (1) generation of the API call sequence in a code cell, and (2) full\ncode cell generation, both conditioned on the NL-Code history up to a\nparticular code cell. Experiments using current baseline code generation models\nshow that both context and distant supervision aid in generation, and that the\ndataset is challenging for current systems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.02334v1",
            "date": "2019-10-05 22:05:43+00:00",
            "title": "Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation",
            "authors": [
                "Benet Oriol Sabat",
                "Cristian Canton Ferrer",
                "Xavier Giro-i-Nieto"
            ],
            "abstract": "This work addresses the challenge of hate speech detection in Internet memes,\nand attempts using visual information to automatically detect hate speech,\nunlike any previous work of our knowledge. Memes are pixel-based multimedia\ndocuments that contain photos or illustrations together with phrases which,\nwhen combined, usually adopt a funny meaning. However, hate memes are also used\nto spread hate through social networks, so their automatic detection would help\nreduce their harmful societal impact. Our results indicate that the model can\nlearn to detect some of the memes, but that the task is far from being solved\nwith this simple architecture. While previous work focuses on linguistic hate\nspeech, our experiments indicate how the visual modality can be much more\ninformative for hate speech detection than the linguistic one in memes. In our\nexperiments, we built a dataset of 5,020 memes to train and evaluate a\nmulti-layer perceptron over the visual and language representations, whether\nindependently or fused. The source code and mode and models are available\nhttps://github.com/imatge-upc/hate-speech-detection .",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1910.02517v1",
            "date": "2019-10-06 20:26:12+00:00",
            "title": "Fine-Grained Analysis of Propaganda in News Articles",
            "authors": [
                "Giovanni Da San Martino",
                "Seunghak Yu",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Rostislav Petrov",
                "Preslav Nakov"
            ],
            "abstract": "Propaganda aims at influencing people's mindset with the purpose of advancing\na specific agenda. Previous work has addressed propaganda detection at the\ndocument level, typically labelling all articles from a propagandistic news\noutlet as propaganda. Such noisy gold labels inevitably affect the quality of\nany learning system trained on them. A further issue with most existing systems\nis the lack of explainability. To overcome these limitations, we propose a\nnovel task: performing fine-grained analysis of texts by detecting all\nfragments that contain propaganda techniques as well as their type. In\nparticular, we create a corpus of news articles manually annotated at the\nfragment level with eighteen propaganda techniques and we propose a suitable\nevaluation measure. We further design a novel multi-granularity neural network,\nand we show that it outperforms several strong BERT-based baselines.",
            "journal ref": "EMNLP-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02545v1",
            "date": "2019-10-06 22:54:00+00:00",
            "title": "Early Prediction of 30-day ICU Re-admissions Using Natural Language Processing and Machine Learning",
            "authors": [
                "Zhiheng Li",
                "Xinyue Xing",
                "Bingzhang Lu",
                "Zhixiang Li"
            ],
            "abstract": "ICU readmission is associated with longer hospitalization, mortality and\nadverse outcomes. An early recognition of ICU re-admission can help prevent\npatients from worse situation and lower treatment cost. As the abundance of\nElectronics Health Records (EHR), it is popular to design clinical decision\ntools with machine learning technique manipulating on healthcare large scale\ndata. We designed data-driven predictive models to estimate the risk of ICU\nreadmission. The discharge summary of each hospital admission was carefully\nrepresented by natural language processing techniques. Unified Medical Language\nSystem (UMLS) was further used to standardize inconsistency of discharge\nsummaries. 5 machine learning classifiers were adopted to construct predictive\nmodels. The best configuration yielded a competitive AUC of 0.748. Our work\nsuggests that natural language processing of discharge summaries is capable to\nsend clinicians warning of unplanned 30-day readmission upon discharge.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.02789v2",
            "date": "2019-10-02 11:06:17+00:00",
            "title": "Language is Power: Representing States Using Natural Language in Reinforcement Learning",
            "authors": [
                "Erez Schwartz",
                "Guy Tennenholtz",
                "Chen Tessler",
                "Shie Mannor"
            ],
            "abstract": "Recent advances in reinforcement learning have shown its potential to tackle\ncomplex real-life tasks. However, as the dimensionality of the task increases,\nreinforcement learning methods tend to struggle. To overcome this, we explore\nmethods for representing the semantic information embedded in the state. While\nprevious methods focused on information in its raw form (e.g., raw visual\ninput), we propose to represent the state using natural language. Language can\nrepresent complex scenarios and concepts, making it a favorable candidate for\nrepresentation. Empirical evidence, within the domain of ViZDoom, suggests that\nnatural language based agents are more robust, converge faster and perform\nbetter than vision based agents, showing the benefit of using natural language\nrepresentations for reinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.02974v3",
            "date": "2019-10-07 18:03:14+00:00",
            "title": "SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability",
            "authors": [
                "Marcella Cornia",
                "Lorenzo Baraldi",
                "Rita Cucchiara"
            ],
            "abstract": "The ability to generate natural language explanations conditioned on the\nvisual perception is a crucial step towards autonomous agents which can explain\nthemselves and communicate with humans. While the research efforts in image and\nvideo captioning are giving promising results, this is often done at the\nexpense of the computational requirements of the approaches, limiting their\napplicability to real contexts. In this paper, we propose a fully-attentive\ncaptioning algorithm which can provide state-of-the-art performances on\nlanguage generation while restricting its computational demands. Our model is\ninspired by the Transformer model and employs only two Transformer layers in\nthe encoding and decoding stages. Further, it incorporates a novel memory-aware\nencoding of image regions. Experiments demonstrate that our approach achieves\ncompetitive results in terms of caption quality while featuring reduced\ncomputational demands. Further, to evaluate its applicability on autonomous\nagents, we conduct experiments on simulated scenes taken from the perspective\nof domestic robots.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.02993v1",
            "date": "2019-10-07 18:18:37+00:00",
            "title": "Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels",
            "authors": [
                "Daniel Y. Fu",
                "Will Crichton",
                "James Hong",
                "Xinwei Yao",
                "Haotian Zhang",
                "Anh Truong",
                "Avanika Narayan",
                "Maneesh Agrawala",
                "Christopher R\u00e9",
                "Kayvon Fatahalian"
            ],
            "abstract": "Many real-world video analysis applications require the ability to identify\ndomain-specific events in video, such as interviews and commercials in TV news\nbroadcasts, or action sequences in film. Unfortunately, pre-trained models to\ndetect all the events of interest in video may not exist, and training new\nmodels from scratch can be costly and labor-intensive. In this paper, we\nexplore the utility of specifying new events in video in a more traditional\nmanner: by writing queries that compose outputs of existing, pre-trained\nmodels. To write these queries, we have developed Rekall, a library that\nexposes a data model and programming model for compositional video event\nspecification. Rekall represents video annotations from different sources\n(object detectors, transcripts, etc.) as spatiotemporal labels associated with\ncontinuous volumes of spacetime in a video, and provides operators for\ncomposing labels into queries that model new video events. We demonstrate the\nuse of Rekall in analyzing video from cable TV news broadcasts, films,\nstatic-camera vehicular video streams, and commercial autonomous vehicle logs.\nIn these efforts, domain experts were able to quickly (in a few hours to a day)\nauthor queries that enabled the accurate detection of new events (on par with,\nand in some cases much more accurate than, learned approaches) and to rapidly\nretrieve video clips for human-in-the-loop tasks such as video content curation\nand training data curation. Finally, in a user study, novice users of Rekall\nwere able to author queries to retrieve new events in video given just one hour\nof query development time.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1910.03042v1",
            "date": "2019-10-07 19:24:36+00:00",
            "title": "Gunrock: A Social Bot for Complex and Engaging Long Conversations",
            "authors": [
                "Dian Yu",
                "Michelle Cohn",
                "Yi Mang Yang",
                "Chun-Yen Chen",
                "Weiming Wen",
                "Jiaping Zhang",
                "Mingyang Zhou",
                "Kevin Jesse",
                "Austin Chau",
                "Antara Bhowmick",
                "Shreenath Iyer",
                "Giritheja Sreenivasulu",
                "Sam Davidson",
                "Ashwin Bhandare",
                "Zhou Yu"
            ],
            "abstract": "Gunrock is the winner of the 2018 Amazon Alexa Prize, as evaluated by\ncoherence and engagement from both real users and Amazon-selected expert\nconversationalists. We focus on understanding complex sentences and having\nin-depth conversations in open domains. In this paper, we introduce some\ninnovative system designs and related validation analysis. Overall, we found\nthat users produce longer sentences to Gunrock, which are directly related to\nusers' engagement (e.g., ratings, number of turns). Additionally, users'\nbackstory queries about Gunrock are positively correlated to user satisfaction.\nFinally, we found dialog flows that interleave facts and personal opinions and\nstories lead to better user satisfaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03118v1",
            "date": "2019-10-07 22:37:33+00:00",
            "title": "The Query Translation Landscape: a Survey",
            "authors": [
                "Mohamed Nadjib Mami",
                "Damien Graux",
                "Harsh Thakkar",
                "Simon Scerri",
                "S\u00f6ren Auer",
                "Jens Lehmann"
            ],
            "abstract": "Whereas the availability of data has seen a manyfold increase in past years,\nits value can be only shown if the data variety is effectively tackled ---one\nof the prominent Big Data challenges. The lack of data interoperability limits\nthe potential of its collective use for novel applications. Achieving\ninteroperability through the full transformation and integration of diverse\ndata structures remains an ideal that is hard, if not impossible, to achieve.\nInstead, methods that can simultaneously interpret different types of data\navailable in different data structures and formats have been explored. On the\nother hand, many query languages have been designed to enable users to interact\nwith the data, from relational, to object-oriented, to hierarchical, to the\nmultitude emerging NoSQL languages. Therefore, the interoperability issue could\nbe solved not by enforcing physical data transformation, but by looking at\ntechniques that are able to query heterogeneous sources using one uniform\nlanguage. Both industry and research communities have been keen to develop such\ntechniques, which require the translation of a chosen 'universal' query\nlanguage to the various data model specific query languages that make the\nunderlying data accessible. In this article, we survey more than forty query\ntranslation methods and tools for popular query languages, and classify them\naccording to eight criteria. In particular, we study which query language is a\nmost suitable candidate for that 'universal' query language. Further, the\nresults enable us to discover the weakly addressed and unexplored translation\npaths, to discover gaps and to learn lessons that can benefit future research\nin the area.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1910.03177v2",
            "date": "2019-10-08 02:36:02+00:00",
            "title": "Read, Highlight and Summarize: A Hierarchical Neural Semantic Encoder-based Approach",
            "authors": [
                "Rajeev Bhatt Ambati",
                "Saptarashmi Bandyopadhyay",
                "Prasenjit Mitra"
            ],
            "abstract": "Traditional sequence-to-sequence (seq2seq) models and other variations of the\nattention-mechanism such as hierarchical attention have been applied to the\ntext summarization problem. Though there is a hierarchy in the way humans use\nlanguage by forming paragraphs from sentences and sentences from words,\nhierarchical models have usually not worked that much better than their\ntraditional seq2seq counterparts. This effect is mainly because either the\nhierarchical attention mechanisms are too sparse using hard attention or noisy\nusing soft attention. In this paper, we propose a method based on extracting\nthe highlights of a document; a key concept that is conveyed in a few\nsentences. In a typical text summarization dataset consisting of documents that\nare 800 tokens in length (average), capturing long-term dependencies is very\nimportant, e.g., the last sentence can be grouped with the first sentence of a\ndocument to form a summary. LSTMs (Long Short-Term Memory) proved useful for\nmachine translation. However, they often fail to capture long-term dependencies\nwhile modeling long sequences. To address these issues, we have adapted Neural\nSemantic Encoders (NSE) to text summarization, a class of memory-augmented\nneural networks by improving its functionalities and proposed a novel\nhierarchical NSE that outperforms similar previous models significantly. The\nquality of summarization was improved by augmenting linguistic factors, namely\nlemma, and Part-of-Speech (PoS) tags, to each word in the dataset for improved\nvocabulary coverage and generalization. The hierarchical NSE model on factored\ndataset outperformed the state-of-the-art by nearly 4 ROUGE points. We further\ndesigned and used the first GPU-based self-critical Reinforcement Learning\nmodel.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.03206v2",
            "date": "2019-10-08 04:17:33+00:00",
            "title": "Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas",
            "authors": [
                "Shriphani Palakodety",
                "Ashiqur R. KhudaBukhsh",
                "Jaime G. Carbonell"
            ],
            "abstract": "The Rohingya refugee crisis is one of the biggest humanitarian crises of\nmodern times with more than 600,000 Rohingyas rendered homeless according to\nthe United Nations High Commissioner for Refugees. While it has received\nsustained press attention globally, no comprehensive research has been\nperformed on social media pertaining to this large evolving crisis. In this\nwork, we construct a substantial corpus of YouTube video comments (263,482\ncomments from 113,250 users in 5,153 relevant videos) with an aim to analyze\nthe possible role of AI in helping a marginalized community. Using a novel\ncombination of multiple Active Learning strategies and a novel active sampling\nstrategy based on nearest-neighbors in the comment-embedding space, we\nconstruct a classifier that can detect comments defending the Rohingyas among\nlarger numbers of disparaging and neutral ones. We advocate that beyond the\nburgeoning field of hate-speech detection, automatic detection of\n\\emph{help-speech} can lend voice to the voiceless people and make the internet\nsafer for marginalized communities.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1910.03291v1",
            "date": "2019-10-08 09:13:39+00:00",
            "title": "Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task",
            "authors": [
                "Alireza Mohammadshahi",
                "Remi Lebret",
                "Karl Aberer"
            ],
            "abstract": "In this paper, we propose a new approach to learn multimodal multilingual\nembeddings for matching images and their relevant captions in two languages. We\ncombine two existing objective functions to make images and captions close in a\njoint embedding space while adapting the alignment of word embeddings between\nexisting languages in our model. We show that our approach enables better\ngeneralization, achieving state-of-the-art performance in text-to-image and\nimage-to-text retrieval task, and caption-caption similarity task. Two\nmultimodal multilingual datasets are used for evaluation: Multi30k with German\nand English captions and Microsoft-COCO with English and Japanese captions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03343v2",
            "date": "2019-10-08 11:28:38+00:00",
            "title": "Modulated Self-attention Convolutional Network for VQA",
            "authors": [
                "Jean-Benoit Delbrouck",
                "Antoine Maiorca",
                "Nathan Hubens",
                "St\u00e9phane Dupont"
            ],
            "abstract": "As new data-sets for real-world visual reasoning and compositional question\nanswering are emerging, it might be needed to use the visual feature extraction\nas a end-to-end process during training. This small contribution aims to\nsuggest new ideas to improve the visual processing of traditional convolutional\nnetwork for visual question answering (VQA). In this paper, we propose to\nmodulate by a linguistic input a CNN augmented with self-attention. We show\nencouraging relative improvements for future research in this direction.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.03467v2",
            "date": "2019-10-07 03:11:13+00:00",
            "title": "Overcoming the Rare Word Problem for Low-Resource Language Pairs in Neural Machine Translation",
            "authors": [
                "Thi-Vinh Ngo",
                "Thanh-Le Ha",
                "Phuong-Thai Nguyen",
                "Le-Minh Nguyen"
            ],
            "abstract": "Among the six challenges of neural machine translation (NMT) coined by (Koehn\nand Knowles, 2017), rare-word problem is considered the most severe one,\nespecially in translation of low-resource languages. In this paper, we propose\nthree solutions to address the rare words in neural machine translation\nsystems. First, we enhance source context to predict the target words by\nconnecting directly the source embeddings to the output of the attention\ncomponent in NMT. Second, we propose an algorithm to learn morphology of\nunknown words for English in supervised way in order to minimize the adverse\neffect of rare-word problem. Finally, we exploit synonymous relation from the\nWordNet to overcome out-of-vocabulary (OOV) problem of NMT. We evaluate our\napproaches on two low-resource language pairs: English-Vietnamese and\nJapanese-Vietnamese. In our experiments, we have achieved significant\nimprovements of up to roughly +1.0 BLEU points in both language pairs.",
            "journal ref": "Proceedings of the 6th Workshop on Asian Translation, WAT 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03474v1",
            "date": "2019-10-04 09:20:48+00:00",
            "title": "Fine-grained Sentiment Classification using BERT",
            "authors": [
                "Manish Munikar",
                "Sushil Shakya",
                "Aakash Shrestha"
            ],
            "abstract": "Sentiment classification is an important process in understanding people's\nperception towards a product, service, or topic. Many natural language\nprocessing models have been proposed to solve the sentiment classification\nproblem. However, most of them have focused on binary sentiment classification.\nIn this paper, we use a promising deep learning model called BERT to solve the\nfine-grained sentiment classification task. Experiments show that our model\noutperforms other popular models for this task without sophisticated\narchitecture. We also demonstrate the effectiveness of transfer learning in\nnatural language processing in the process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03487v1",
            "date": "2019-10-04 20:44:21+00:00",
            "title": "Controlled Text Generation for Data Augmentation in Intelligent Artificial Agents",
            "authors": [
                "Nikolaos Malandrakis",
                "Minmin Shen",
                "Anuj Goyal",
                "Shuyang Gao",
                "Abhishek Sethi",
                "Angeliki Metallinou"
            ],
            "abstract": "Data availability is a bottleneck during early stages of development of new\ncapabilities for intelligent artificial agents. We investigate the use of text\ngeneration techniques to augment the training data of a popular commercial\nartificial agent across categories of functionality, with the goal of faster\ndevelopment of new functionality. We explore a variety of encoder-decoder\ngenerative models for synthetic training data generation and propose using\nconditional variational auto-encoders. Our approach requires only direct\noptimization, works well with limited data and significantly outperforms the\nprevious controlled text generation techniques. Further, the generated data are\nused as additional training samples in an extrinsic intent classification task,\nleading to improved performance by up to 5\\% absolute f-score in low-resource\ncases, validating the usefulness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03492v1",
            "date": "2019-10-04 16:44:33+00:00",
            "title": "Neural Language Priors",
            "authors": [
                "Joseph Enguehard",
                "Dan Busbridge",
                "Vitalii Zhelezniak",
                "Nils Hammerla"
            ],
            "abstract": "The choice of sentence encoder architecture reflects assumptions about how a\nsentence's meaning is composed from its constituent words. We examine the\ncontribution of these architectures by holding them randomly initialised and\nfixed, effectively treating them as as hand-crafted language priors, and\nevaluating the resulting sentence encoders on downstream language tasks. We\nfind that even when encoders are presented with additional information that can\nbe used to solve tasks, the corresponding priors do not leverage this\ninformation, except in an isolated case. We also find that apparently\nuninformative priors are just as good as seemingly informative priors on almost\nall tasks, indicating that learning is a necessary component to leverage\ninformation provided by architecture choice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03505v2",
            "date": "2019-10-04 11:00:36+00:00",
            "title": "Investigating the Effectiveness of Representations Based on Word-Embeddings in Active Learning for Labelling Text Datasets",
            "authors": [
                "Jinghui Lu",
                "Maeve Henchion",
                "Brian Mac Namee"
            ],
            "abstract": "Manually labelling large collections of text data is a time-consuming,\nexpensive, and laborious task, but one that is necessary to support machine\nlearning based on text datasets. Active learning has been shown to be an\neffective way to alleviate some of the effort required in utilising large\ncollections of unlabelled data for machine learning tasks without needing to\nfully label them. The representation mechanism used to represent text documents\nwhen performing active learning, however, has a significant influence on how\neffective the process will be. While simple vector representations such as bag\nof words have been shown to be an effective way to represent documents during\nactive learning, the emergence of representation mechanisms based on the word\nembeddings prevalent in neural network research (e.g. word2vec and\ntransformer-based models like BERT) offer a promising, and as yet not fully\nexplored, alternative. This paper describes a large-scale evaluation of the\neffectiveness of different text representation mechanisms for active learning\nacross 8 datasets from varied domains. This evaluation shows that using\nrepresentations based on modern word embeddings---especially BERT---, which\nhave not yet been widely used in active learning, achieves a significant\nimprovement over more commonly used vector-based methods like bag of words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03506v2",
            "date": "2019-09-30 19:12:10+00:00",
            "title": "Towards Controllable and Personalized Review Generation",
            "authors": [
                "Pan Li",
                "Alexander Tuzhilin"
            ],
            "abstract": "In this paper, we propose a novel model RevGAN that automatically generates\ncontrollable and personalized user reviews based on the arbitrarily given\nsentimental and stylistic information. RevGAN utilizes the combination of three\nnovel components, including self-attentive recursive autoencoders, conditional\ndiscriminators, and personalized decoders. We test its performance on the\nseveral real-world datasets, where our model significantly outperforms\nstate-of-the-art generation models in terms of sentence quality, coherence,\npersonalization and human evaluations. We also empirically show that the\ngenerated reviews could not be easily distinguished from the organically\nproduced reviews and that they follow the same statistical linguistics laws.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03634v1",
            "date": "2019-10-08 18:39:49+00:00",
            "title": "Prose for a Painting",
            "authors": [
                "Prerna Kashyap",
                "Samrat Phatale",
                "Iddo Drori"
            ],
            "abstract": "Painting captions are often dry and simplistic which motivates us to describe\na painting creatively in the style of Shakespearean prose. This is a difficult\nproblem, since there does not exist a large supervised dataset from paintings\nto Shakespearean prose. Our solution is to use an intermediate English poem\ndescription of the painting and then apply language style transfer which\nresults in Shakespearean prose describing the painting. We rate our results by\nhuman evaluation on a Likert scale, and evaluate the quality of language style\ntransfer using BLEU score as a function of prose length. We demonstrate the\napplicability and limitations of our approach by generating Shakespearean prose\nfor famous paintings. We make our models and code publicly available.",
            "journal ref": "ICCV Workshop on Closing the Loop Between Vision and Language,\n  2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.03641v1",
            "date": "2019-10-08 18:55:08+00:00",
            "title": "Linking emotions to behaviors through deep transfer learning",
            "authors": [
                "Haoqi Li",
                "Brian Baucom",
                "Panayiotis Georgiou"
            ],
            "abstract": "Human behavior refers to the way humans act and interact. Understanding human\nbehavior is a cornerstone of observational practice, especially in\npsychotherapy. An important cue of behavior analysis is the dynamical changes\nof emotions during the conversation. Domain experts integrate emotional\ninformation in a highly nonlinear manner, thus, it is challenging to explicitly\nquantify the relationship between emotions and behaviors. In this work, we\nemploy deep transfer learning to analyze their inferential capacity and\ncontextual importance. We first train a network to quantify emotions from\nacoustic signals and then use information from the emotion recognition network\nas features for behavior recognition. We treat this emotion-related information\nas behavioral primitives and further train higher level layers towards behavior\nquantification. Through our analysis, we find that emotion-related information\nis an important cue for behavior recognition. Further, we investigate the\nimportance of emotional-context in the expression of behavior by constraining\n(or not) the neural networks' contextual view of the data. This demonstrates\nthat the sequence of emotions is critical in behavior expression. To achieve\nthese frameworks we employ hybrid architectures of convolutional networks and\nrecurrent networks to extract emotion-related behavior primitives and\nfacilitate automatic behavior recognition from speech.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.03655v3",
            "date": "2019-10-08 19:22:58+00:00",
            "title": "Executing Instructions in Situated Collaborative Interactions",
            "authors": [
                "Alane Suhr",
                "Claudia Yan",
                "Jacob Schluger",
                "Stanley Yu",
                "Hadi Khader",
                "Marwa Mouallem",
                "Iris Zhang",
                "Yoav Artzi"
            ],
            "abstract": "We study a collaborative scenario where a user not only instructs a system to\ncomplete tasks, but also acts alongside it. This allows the user to adapt to\nthe system abilities by changing their language or deciding to simply\naccomplish some tasks themselves, and requires the system to effectively\nrecover from errors as the user strategically assigns it new goals. We build a\ngame environment to study this scenario, and learn to map user instructions to\nsystem actions. We introduce a learning approach focused on recovery from\ncascading errors between instructions, and modeling methods to explicitly\nreason about instructions with multiple goals. We evaluate with a new\nevaluation protocol using recorded interactions and online games with human\nusers, and observe how users adapt to the system abilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03678v1",
            "date": "2019-09-29 21:33:46+00:00",
            "title": "Unfolding the Structure of a Document using Deep Learning",
            "authors": [
                "Muhammad Mahbubur Rahman",
                "Tim Finin"
            ],
            "abstract": "Understanding and extracting of information from large documents, such as\nbusiness opportunities, academic articles, medical documents and technical\nreports, poses challenges not present in short documents. Such large documents\nmay be multi-themed, complex, noisy and cover diverse topics. We describe a\nframework that can analyze large documents and help people and computer systems\nlocate desired information in them. We aim to automatically identify and\nclassify different sections of documents and understand their purpose within\nthe document. A key contribution of our research is modeling and extracting the\nlogical and semantic structure of electronic documents using deep learning\ntechniques. We evaluate the effectiveness and robustness of our framework\nthrough extensive experiments on two collections: more than one million\nscholarly articles from arXiv and a collection of requests for proposal\ndocuments from government sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03698v1",
            "date": "2019-10-08 21:28:47+00:00",
            "title": "AutoML using Metadata Language Embeddings",
            "authors": [
                "Iddo Drori",
                "Lu Liu",
                "Yi Nian",
                "Sharath C. Koorathota",
                "Jie S. Li",
                "Antonio Khalil Moretti",
                "Juliana Freire",
                "Madeleine Udell"
            ],
            "abstract": "As a human choosing a supervised learning algorithm, it is natural to begin\nby reading a text description of the dataset and documentation for the\nalgorithms you might use. We demonstrate that the same idea improves the\nperformance of automated machine learning methods. We use language embeddings\nfrom modern NLP to improve state-of-the-art AutoML systems by augmenting their\nrecommendations with vector embeddings of datasets and of algorithms. We use\nthese embeddings in a neural architecture to learn the distance between\nbest-performing pipelines. The resulting (meta-)AutoML framework improves on\nthe performance of existing AutoML frameworks. Our zero-shot AutoML system\nusing dataset metadata embeddings provides good solutions instantaneously,\nrunning in under one second of computation. Performance is competitive with\nAutoML systems OBOE, AutoSklearn, AlphaD3M, and TPOT when each framework is\nallocated a minute of computation. We make our data, models, and code publicly\navailable.",
            "journal ref": "NeurIPS Workshop on Meta-Learning, 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.03704v1",
            "date": "2019-10-08 22:11:55+00:00",
            "title": "Do People Prefer \"Natural\" code?",
            "authors": [
                "Casey Casalnuovo",
                "Kevin Lee",
                "Hulin Wang",
                "Prem Devanbu",
                "Emily Morgan"
            ],
            "abstract": "Natural code is known to be very repetitive (much more so than natural\nlanguage corpora); furthermore, this repetitiveness persists, even after\naccounting for the simpler syntax of code. However, programming languages are\nvery expressive, allowing a great many different ways (all clear and\nunambiguous) to express even very simple computations. So why is natural code\nrepetitive? We hypothesize that the reasons for this lie in fact that code is\nbimodal: it is executed by machines, but also read by humans. This bimodality,\nwe argue, leads developers to write code in certain preferred ways that would\nbe familiar to code readers. To test this theory, we 1) model familiarity using\na language model estimated over a large training corpus and 2) run an\nexperiment applying several meaning preserving transformations to Java and\nPython expressions in a distinct test corpus to see if forms more familiar to\nreaders (as predicted by the language models) are in fact the ones actually\nwritten. We find that these transformations generally produce program\nstructures that are less common in practice, supporting the theory that the\nhigh repetitiveness in code is a matter of deliberate preference. Finally, 3)\nwe use a human subject study to show alignment between language model score and\nhuman preference for the first time in code, providing support for using this\nmeasure to improve code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.03943v1",
            "date": "2019-09-30 23:47:55+00:00",
            "title": "Hotel2vec: Learning Attribute-Aware Hotel Embeddings with Self-Supervision",
            "authors": [
                "Ali Sadeghian",
                "Shervin Minaee",
                "Ioannis Partalas",
                "Xinxin Li",
                "Daisy Zhe Wang",
                "Brooke Cowan"
            ],
            "abstract": "We propose a neural network architecture for learning vector representations\nof hotels. Unlike previous works, which typically only use user click\ninformation for learning item embeddings, we propose a framework that combines\nseveral sources of data, including user clicks, hotel attributes (e.g.,\nproperty type, star rating, average user rating), amenity information (e.g.,\nthe hotel has free Wi-Fi or free breakfast), and geographic information. During\nmodel training, a joint embedding is learned from all of the above information.\nWe show that including structured attributes about hotels enables us to make\nbetter predictions in a downstream task than when we rely exclusively on click\ndata. We train our embedding model on more than 40 million user click sessions\nfrom a leading online travel platform and learn embeddings for more than one\nmillion hotels. Our final learned embeddings integrate distinct sub-embeddings\nfor user clicks, hotel attributes, and geographic information, providing an\ninterpretable representation that can be used flexibly depending on the\napplication. We show empirically that our model generates high-quality\nrepresentations that boost the performance of a hotel recommendation system in\naddition to other applications. An important advantage of the proposed neural\nmodel is that it addresses the cold-start problem for hotels with insufficient\nhistorical click information by incorporating additional hotel attributes which\nare available for all hotels.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.04056v1",
            "date": "2019-10-09 15:14:09+00:00",
            "title": "Text-to-Image Synthesis Based on Machine Generated Captions",
            "authors": [
                "Marco Menardi",
                "Alex Falcon",
                "Saida S. Mohamed",
                "Lorenzo Seidenari",
                "Giuseppe Serra",
                "Alberto Del Bimbo",
                "Carlo Tasso"
            ],
            "abstract": "Text to Image Synthesis refers to the process of automatic generation of a\nphoto-realistic image starting from a given text and is revolutionizing many\nreal-world applications. In order to perform such process it is necessary to\nexploit datasets containing captioned images, meaning that each image is\nassociated with one (or more) captions describing it. Despite the abundance of\nuncaptioned images datasets, the number of captioned datasets is limited. To\naddress this issue, in this paper we propose an approach capable of generating\nimages starting from a given text using conditional GANs trained on uncaptioned\nimages dataset. In particular, uncaptioned images are fed to an Image\nCaptioning Module to generate the descriptions. Then, the GAN Module is trained\non both the input image and the machine-generated caption. To evaluate the\nresults, the performance of our solution is compared with the results obtained\nby the unconditional GAN. For the experiments, we chose to use the uncaptioned\ndataset LSUN bedroom. The results obtained in our study are preliminary but\nstill promising.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.04176v1",
            "date": "2019-10-09 18:00:04+00:00",
            "title": "A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classification",
            "authors": [
                "Varun Kumar",
                "Hadrien Glaude",
                "Cyprien de Lichy",
                "William Campbell"
            ],
            "abstract": "New conversation topics and functionalities are constantly being added to\nconversational AI agents like Amazon Alexa and Apple Siri. As data collection\nand annotation is not scalable and is often costly, only a handful of examples\nfor the new functionalities are available, which results in poor generalization\nperformance. We formulate it as a Few-Shot Integration (FSI) problem where a\nfew examples are used to introduce a new intent. In this paper, we study six\nfeature space data augmentation methods to improve classification performance\nin FSI setting in combination with both supervised and unsupervised\nrepresentation learning methods such as BERT. Through realistic experiments on\ntwo public conversational datasets, SNIPS, and the Facebook Dialog corpus, we\nshow that data augmentation in feature space provides an effective way to\nimprove intent classification performance in few-shot setting beyond\ntraditional transfer learning approaches. In particular, we show that (a)\nupsampling in latent space is a competitive baseline for feature space\naugmentation (b) adding the difference between two examples to a new example is\na simple yet effective data augmentation method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04192v2",
            "date": "2019-10-09 18:19:48+00:00",
            "title": "Domain-Relevant Embeddings for Medical Question Similarity",
            "authors": [
                "Clara McCreery",
                "Namit Katariya",
                "Anitha Kannan",
                "Manish Chablani",
                "Xavier Amatriain"
            ],
            "abstract": "The rate at which medical questions are asked online far exceeds the capacity\nof qualified people to answer them, and many of these questions are not unique.\nIdentifying same-question pairs could enable questions to be answered more\neffectively. While many research efforts have focused on the problem of general\nquestion similarity for non-medical applications, these approaches do not\ngeneralize well to the medical domain, where medical expertise is often\nrequired to determine semantic similarity. In this paper, we show how a\nsemi-supervised approach of pre-training a neural network on medical\nquestion-answer pairs is a particularly useful intermediate task for the\nultimate goal of determining medical question similarity. While other\npre-training tasks yield an accuracy below 78.7% on this task, our model\nachieves an accuracy of 82.6% with the same number of training examples, and an\naccuracy of 80.0% with a much smaller training set.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.04385v2",
            "date": "2019-10-10 06:29:22+00:00",
            "title": "Learning Only from Relevant Keywords and Unlabeled Documents",
            "authors": [
                "Nontawat Charoenphakdee",
                "Jongyeong Lee",
                "Yiping Jin",
                "Dittaya Wanvarie",
                "Masashi Sugiyama"
            ],
            "abstract": "We consider a document classification problem where document labels are\nabsent but only relevant keywords of a target class and unlabeled documents are\ngiven. Although heuristic methods based on pseudo-labeling have been\nconsidered, theoretical understanding of this problem has still been limited.\nMoreover, previous methods cannot easily incorporate well-developed techniques\nin supervised text classification. In this paper, we propose a theoretically\nguaranteed learning framework that is simple to implement and has flexible\nchoices of models, e.g., linear models or neural networks. We demonstrate how\nto optimize the area under the receiver operating characteristic curve (AUC)\neffectively and also discuss how to adjust it to optimize other well-known\nevaluation metrics such as the accuracy and F1-measure. Finally, we show the\neffectiveness of our framework using benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04424v1",
            "date": "2019-10-10 08:25:42+00:00",
            "title": "Contract Statements Knowledge Service for Chatbots",
            "authors": [
                "Boris Ruf",
                "Matteo Sammarco",
                "Marcin Detyniecki"
            ],
            "abstract": "Towards conversational agents that are capable of handling more complex\nquestions on contractual conditions, formalizing contract statements in a\nmachine readable way is crucial. However, constructing a formal model which\ncaptures the full scope of a contract proves difficult due to the overall\ncomplexity its set of rules represent. Instead, this paper presents a top-down\napproach to the problem. After identifying the most relevant contract\nstatements, we model their underlying rules in a novel knowledge engineering\nmethod. A user-friendly tool we developed for this purpose allows to do so\neasily and at scale. Then, we expose the statements as service so they can get\nsmoothly integrated in any chatbot framework.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.04519v1",
            "date": "2019-10-10 12:42:19+00:00",
            "title": "Language Transfer for Early Warning of Epidemics from Social Media",
            "authors": [
                "Mattias Appelgren",
                "Patrick Schrempf",
                "Mat\u00fa\u0161 Falis",
                "Satoshi Ikeda",
                "Alison Q O'Neil"
            ],
            "abstract": "Statements on social media can be analysed to identify individuals who are\nexperiencing red flag medical symptoms, allowing early detection of the spread\nof disease such as influenza. Since disease does not respect cultural borders\nand may spread between populations speaking different languages, we would like\nto build multilingual models. However, the data required to train models for\nevery language may be difficult, expensive and time-consuming to obtain,\nparticularly for low-resource languages. Taking Japanese as our target\nlanguage, we explore methods by which data in one language might be used to\nbuild models for a different language. We evaluate strategies of training on\nmachine translated data and of zero-shot transfer through the use of\nmultilingual models. We find that the choice of source language impacts the\nperformance, with Chinese-Japanese being a better language pair than\nEnglish-Japanese. Training on machine translated data shows promise, especially\nwhen used in conjunction with a small amount of target language data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04618v1",
            "date": "2019-10-10 14:48:22+00:00",
            "title": "Universal Adversarial Perturbation for Text Classification",
            "authors": [
                "Hang Gao",
                "Tim Oates"
            ],
            "abstract": "Given a state-of-the-art deep neural network text classifier, we show the\nexistence of a universal and very small perturbation vector (in the embedding\nspace) that causes natural text to be misclassified with high probability.\nUnlike images on which a single fixed-size adversarial perturbation can be\nfound, text is of variable length, so we define the \"universality\" as\n\"token-agnostic\", where a single perturbation is applied to each token,\nresulting in different perturbations of flexible sizes at the sequence level.\nWe propose an algorithm to compute universal adversarial perturbations, and\nshow that the state-of-the-art deep neural networks are highly vulnerable to\nthem, even though they keep the neighborhood of tokens mostly preserved. We\nalso show how to use these adversarial perturbations to generate adversarial\ntext samples. The surprising existence of universal \"token-agnostic\"\nadversarial perturbations may reveal important properties of a text classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04732v2",
            "date": "2019-10-10 17:44:18+00:00",
            "title": "Structured Pruning of Large Language Models",
            "authors": [
                "Ziheng Wang",
                "Jeremy Wohlwend",
                "Tao Lei"
            ],
            "abstract": "Large language models have recently achieved state of the art performance\nacross a wide variety of natural language tasks. Meanwhile, the size of these\nmodels and their latency have significantly increased, which makes their usage\ncostly, and raises an interesting question: do language models need to be\nlarge? We study this question through the lens of model compression. We present\na generic, structured pruning approach by parameterizing each weight matrix\nusing its low-rank factorization, and adaptively removing rank-1 components\nduring training. On language modeling tasks, our structured approach\noutperforms other unstructured and block-structured pruning baselines at\nvarious compression levels, while achieving significant speedups during both\ntraining and inference. We also demonstrate that our method can be applied to\npruning adaptive word embeddings in large language models, and to pruning the\nBERT model on several downstream fine-tuning classification benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.04887v1",
            "date": "2019-10-10 21:46:26+00:00",
            "title": "Visual Natural Language Query Auto-Completion for Estimating Instance Probabilities",
            "authors": [
                "Samuel Sharpe",
                "Jin Yan",
                "Fan Wu",
                "Iddo Drori"
            ],
            "abstract": "We present a new task of query auto-completion for estimating instance\nprobabilities. We complete a user query prefix conditioned upon an image. Given\nthe complete query, we fine tune a BERT embedding for estimating probabilities\nof a broad set of instances. The resulting instance probabilities are used for\nselection while being agnostic to the segmentation or attention mechanism. Our\nresults demonstrate that auto-completion using both language and vision\nperforms better than using only language, and that fine tuning a BERT embedding\nallows to efficiently rank instances in the image. In the spirit of\nreproducible research we make our data, models, and code available.",
            "journal ref": "CVPR Language and Vision Workshop, 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.04979v1",
            "date": "2019-10-11 05:37:11+00:00",
            "title": "Learning Invariant Representations of Social Media Users",
            "authors": [
                "Nicholas Andrews",
                "Marcus Bishop"
            ],
            "abstract": "The evolution of social media users' behavior over time complicates\nuser-level comparison tasks such as verification, classification, clustering,\nand ranking. As a result, na\\\"ive approaches may fail to generalize to new\nusers or even to future observations of previously known users. In this paper,\nwe propose a novel procedure to learn a mapping from short episodes of user\nactivity on social media to a vector space in which the distance between points\ncaptures the similarity of the corresponding users' invariant features. We fit\nthe model by optimizing a surrogate metric learning objective over a large\ncorpus of unlabeled social media content. Once learned, the mapping may be\napplied to users not seen at training time and enables efficient comparisons of\nusers in the resulting vector space. We present a comprehensive evaluation to\nvalidate the benefits of the proposed approach using data from Reddit, Twitter,\nand Wikipedia.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1910.05171v3",
            "date": "2019-10-11 13:28:03+00:00",
            "title": "Query-by-example on-device keyword spotting",
            "authors": [
                "Byeonggeun Kim",
                "Mingu Lee",
                "Jinkyu Lee",
                "Yeonseok Kim",
                "Kyuwoong Hwang"
            ],
            "abstract": "A keyword spotting (KWS) system determines the existence of, usually\npredefined, keyword in a continuous speech stream. This paper presents a\nquery-by-example on-device KWS system which is user-specific. The proposed\nsystem consists of two main steps: query enrollment and testing. In query\nenrollment step, phonetic posteriors are output by a small-footprint automatic\nspeech recognition model based on connectionist temporal classification. Using\nthe phonetic-level posteriorgram, hypothesis graph of finite-state transducer\n(FST) is built, thus can enroll any keywords thus avoiding an out-of-vocabulary\nproblem. In testing, a log-likelihood is scored for input audio using the FST.\nWe propose a threshold prediction method while using the user-specific keyword\nhypothesis only. The system generates query-specific negatives by rearranging\neach query utterance in waveform. The threshold is decided based on the\nenrollment queries and generated negatives. We tested two keywords in English,\nand the proposed work shows promising performance while preserving simplicity.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.05291v1",
            "date": "2019-10-11 16:34:01+00:00",
            "title": "The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents",
            "authors": [
                "Shangmin Guo",
                "Yi Ren",
                "Serhii Havrylov",
                "Stella Frank",
                "Ivan Titov",
                "Kenny Smith"
            ],
            "abstract": "Since first introduced, computer simulation has been an increasingly\nimportant tool in evolutionary linguistics. Recently, with the development of\ndeep learning techniques, research in grounded language learning has also\nstarted to focus on facilitating the emergence of compositional languages\nwithout pre-defined elementary linguistic knowledge. In this work, we explore\nthe emergence of compositional languages for numeric concepts in multi-agent\ncommunication systems. We demonstrate that compositional language for encoding\nnumeric concepts can emerge through iterated learning in populations of deep\nneural network agents. However, language properties greatly depend on the input\nrepresentations given to agents. We found that compositional languages only\nemerge if they require less iterations to be fully learnt than other\nnon-degenerate languages for agents on a given input representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05495v2",
            "date": "2019-10-12 05:08:43+00:00",
            "title": "Prediction Focused Topic Models via Feature Selection",
            "authors": [
                "Jason Ren",
                "Russell Kunes",
                "Finale Doshi-Velez"
            ],
            "abstract": "Supervised topic models are often sought to balance prediction quality and\ninterpretability. However, when models are (inevitably) misspecified, standard\napproaches rarely deliver on both. We introduce a novel approach, the\nprediction-focused topic model, that uses the supervisory signal to retain only\nvocabulary terms that improve, or at least do not hinder, prediction\nperformance. By removing terms with irrelevant signal, the topic model is able\nto learn task-relevant, coherent topics. We demonstrate on several data sets\nthat compared to existing approaches, prediction-focused topic models learn\nmuch more coherent topics while maintaining competitive predictions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.05603v1",
            "date": "2019-10-12 17:37:52+00:00",
            "title": "VAIS ASR: Building a conversational speech recognition system using language model combination",
            "authors": [
                "Quang Minh Nguyen",
                "Thai Binh Nguyen",
                "Ngoc Phuong Pham",
                "The Loc Nguyen"
            ],
            "abstract": "Automatic Speech Recognition (ASR) systems have been evolving quickly and\nreaching human parity in certain cases. The systems usually perform pretty well\non reading style and clean speech, however, most of the available systems\nsuffer from situation where the speaking style is conversation and in noisy\nenvironments. It is not straight-forward to tackle such problems due to\ndifficulties in data collection for both speech and text. In this paper, we\nattempt to mitigate the problems using language models combination techniques\nthat allows us to utilize both large amount of writing style text and small\nnumber of conversation text data. Evaluation on the VLSP 2019 ASR challenges\nshowed that our system achieved 4.85% WER on the VLSP 2018 and 15.09% WER on\nthe VLSP 2019 data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.05624v1",
            "date": "2019-10-12 18:59:50+00:00",
            "title": "A Research Platform for Multi-Robot Dialogue with Humans",
            "authors": [
                "Matthew Marge",
                "Stephen Nogar",
                "Cory J. Hayes",
                "Stephanie M. Lukin",
                "Jesse Bloecker",
                "Eric Holder",
                "Clare Voss"
            ],
            "abstract": "This paper presents a research platform that supports spoken dialogue\ninteraction with multiple robots. The demonstration showcases our crafted\nMultiBot testing scenario in which users can verbally issue search, navigate,\nand follow instructions to two robotic teammates: a simulated ground robot and\nan aerial robot. This flexible language and robotic platform takes advantage of\nexisting tools for speech recognition and dialogue management that are\ncompatible with new domains, and implements an inter-agent communication\nprotocol (tactical behavior specification), where verbal instructions are\nencoded for tasks assigned to the appropriate robot.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1910.05728v1",
            "date": "2019-10-13 10:49:41+00:00",
            "title": "Granular Multimodal Attention Networks for Visual Dialog",
            "authors": [
                "Badri N. Patro",
                "Shivansh Patel",
                "Vinay P. Namboodiri"
            ],
            "abstract": "Vision and language tasks have benefited from attention. There have been a\nnumber of different attention models proposed. However, the scale at which\nattention needs to be applied has not been well examined. Particularly, in this\nwork, we propose a new method Granular Multi-modal Attention, where we aim to\nparticularly address the question of the right granularity at which one needs\nto attend while solving the Visual Dialog task. The proposed method shows\nimprovement in both image and text attention networks. We then propose a\ngranular Multi-modal Attention network that jointly attends on the image and\ntext granules and shows the best performance. With this work, we observe that\nobtaining granular attention and doing exhaustive Multi-modal Attention appears\nto be the best way to attend while solving visual dialog.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.05895v2",
            "date": "2019-10-14 02:23:43+00:00",
            "title": "Transformers without Tears: Improving the Normalization of Self-Attention",
            "authors": [
                "Toan Q. Nguyen",
                "Julian Salazar"
            ],
            "abstract": "We evaluate three simple, normalization-centric changes to improve\nTransformer training. First, we show that pre-norm residual connections\n(PreNorm) and smaller initializations enable warmup-free, validation-based\ntraining with large learning rates. Second, we propose $\\ell_2$ normalization\nwith a single scale parameter (ScaleNorm) for faster training and better\nperformance. Finally, we reaffirm the effectiveness of normalizing word\nembeddings to a fixed length (FixNorm). On five low-resource translation pairs\nfrom TED Talks-based corpora, these changes always converge, giving an average\n+1.1 BLEU over state-of-the-art bilingual baselines and a new 32.8 BLEU on\nIWSLT'15 English-Vietnamese. We observe sharper performance curves, more\nconsistent gradient norms, and a linear relationship between activation scaling\nand decoder depth. Surprisingly, in the high-resource setting (WMT'14\nEnglish-German), ScaleNorm and FixNorm remain competitive but PreNorm degrades\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06048v1",
            "date": "2019-10-14 11:39:45+00:00",
            "title": "STANCY: Stance Classification Based on Consistency Cues",
            "authors": [
                "Kashyap Popat",
                "Subhabrata Mukherjee",
                "Andrew Yates",
                "Gerhard Weikum"
            ],
            "abstract": "Controversial claims are abundant in online media and discussion forums. A\nbetter understanding of such claims requires analyzing them from different\nperspectives. Stance classification is a necessary step for inferring these\nperspectives in terms of supporting or opposing the claim. In this work, we\npresent a neural network model for stance classification leveraging BERT\nrepresentations and augmenting them with a novel consistency constraint.\nExperiments on the Perspectrum dataset, consisting of claims and users'\nperspectives from various debate websites, demonstrate the effectiveness of our\napproach over state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06315v1",
            "date": "2019-10-14 17:57:21+00:00",
            "title": "Dynamic Attention Networks for Task Oriented Grounding",
            "authors": [
                "Soumik Dasgupta",
                "Badri N. Patro",
                "Vinay P. Namboodiri"
            ],
            "abstract": "In order to successfully perform tasks specified by natural language\ninstructions, an artificial agent operating in a visual world needs to map\nwords, concepts, and actions from the instruction to visual elements in its\nenvironment. This association is termed as Task-Oriented Grounding. In this\nwork, we propose a novel Dynamic Attention Network architecture for the\nefficient multi-modal fusion of text and visual representations which can\ngenerate a robust definition of state for the policy learner. Our model assumes\nno prior knowledge from visual and textual domains and is an end to end\ntrainable. For a 3D visual world where the observation changes continuously,\nthe attention on the visual elements tends to be highly co-related from a\none-time step to the next. We term this as \"Dynamic Attention\". In this work,\nwe show that Dynamic Attention helps in achieving grounding and also aids in\nthe policy learning objective. Since most practical robotic applications take\nplace in the real world where the observation space is continuous, our\nframework can be used as a generalized multi-modal fusion unit for robotic\ncontrol through natural language. We show the effectiveness of using 1D\nconvolution over Gated Attention Hadamard product on the rate of convergence of\nthe network. We demonstrate that the cell-state of a Long Short Term Memory\n(LSTM) is a natural choice for modeling Dynamic Attention and shows through\nvisualization that the generated attention is very close to how humans tend to\nfocus on the environment.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.06522v2",
            "date": "2019-10-15 04:45:34+00:00",
            "title": "MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition",
            "authors": [
                "Xuankai Chang",
                "Wangyou Zhang",
                "Yanmin Qian",
                "Jonathan Le Roux",
                "Shinji Watanabe"
            ],
            "abstract": "Recently, the end-to-end approach has proven its efficacy in monaural\nmulti-speaker speech recognition. However, high word error rates (WERs) still\nprevent these systems from being used in practical applications. On the other\nhand, the spatial information in multi-channel signals has proven helpful in\nfar-field speech recognition tasks. In this work, we propose a novel neural\nsequence-to-sequence (seq2seq) architecture, MIMO-Speech, which extends the\noriginal seq2seq to deal with multi-channel input and multi-channel output so\nthat it can fully model multi-channel multi-speaker speech separation and\nrecognition. MIMO-Speech is a fully neural end-to-end framework, which is\noptimized only via an ASR criterion. It is comprised of: 1) a monaural masking\nnetwork, 2) a multi-source neural beamformer, and 3) a multi-output speech\nrecognition model. With this processing, the input overlapped speech is\ndirectly mapped to text sequences. We further adopted a curriculum learning\nstrategy, making the best use of the training set to improve the performance.\nThe experiments on the spatialized wsj1-2mix corpus show that our model can\nachieve more than 60% WER reduction compared to the single-channel system with\nhigh quality enhanced signals (SI-SDR = 23.1 dB) obtained by the above\nseparation function.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.06707v1",
            "date": "2019-10-09 15:34:28+00:00",
            "title": "A Deep Learning Based Chatbot for Campus Psychological Therapy",
            "authors": [
                "Junjie Yin",
                "Zixun Chen",
                "Kelai Zhou",
                "Chongyuan Yu"
            ],
            "abstract": "In this paper, we propose Evebot, an innovative, sequence to sequence\n(Seq2seq) based, fully generative conversational system for the diagnosis of\nnegative emotions and prevention of depression through positively suggestive\nresponses. The system consists of an assembly of deep-learning based models,\nincluding Bi-LSTM based model for detecting negative emotions of users and\nobtaining psychological counselling related corpus for training the chatbot,\nanti-language sequence to sequence neural network, and maximum mutual\ninformation (MMI) model. As adolescents are reluctant to show their negative\nemotions in physical interaction, traditional methods of emotion analysis and\ncomforting methods may not work. Therefore, this system puts emphasis on using\nvirtual platform to detect signs of depression or anxiety, channel adolescents'\nstress and mood, and thus prevent the emergence of mental illness. We launched\nthe integrated chatbot system onto an online platform for real-world campus\napplications. Through a one-month user study, we observe better results in the\nincrease in positivity than other public chatbots in the control group.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.06711v3",
            "date": "2019-10-08 15:03:08+00:00",
            "title": "MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis",
            "authors": [
                "Kundan Kumar",
                "Rithesh Kumar",
                "Thibault de Boissiere",
                "Lucas Gestin",
                "Wei Zhen Teoh",
                "Jose Sotelo",
                "Alexandre de Brebisson",
                "Yoshua Bengio",
                "Aaron Courville"
            ],
            "abstract": "Previous works (Donahue et al., 2018a; Engel et al., 2019a) have found that\ngenerating coherent raw audio waveforms with GANs is challenging. In this\npaper, we show that it is possible to train GANs reliably to generate high\nquality coherent waveforms by introducing a set of architectural changes and\nsimple training techniques. Subjective evaluation metric (Mean Opinion Score,\nor MOS) shows the effectiveness of the proposed approach for high quality\nmel-spectrogram inversion. To establish the generality of the proposed\ntechniques, we show qualitative results of our model in speech synthesis, music\ndomain translation and unconditional music synthesis. We evaluate the various\ncomponents of the model through ablation studies and suggest a set of\nguidelines to design general purpose discriminators and generators for\nconditional sequence synthesis tasks. Our model is non-autoregressive, fully\nconvolutional, with significantly fewer parameters than competing models and\ngeneralizes to unseen speakers for mel-spectrogram inversion. Our pytorch\nimplementation runs at more than 100x faster than realtime on GTX 1080Ti GPU\nand more than 2x faster than real-time on CPU, without any hardware specific\noptimization tricks.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.06717v1",
            "date": "2019-10-01 19:21:26+00:00",
            "title": "Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation",
            "authors": [
                "Kenton Murray",
                "Jeffery Kinnison",
                "Toan Q. Nguyen",
                "Walter Scheirer",
                "David Chiang"
            ],
            "abstract": "Neural sequence-to-sequence models, particularly the Transformer, are the\nstate of the art in machine translation. Yet these neural networks are very\nsensitive to architecture and hyperparameter settings. Optimizing these\nsettings by grid or random search is computationally expensive because it\nrequires many training runs. In this paper, we incorporate architecture search\ninto a single training run through auto-sizing, which uses regularization to\ndelete neurons in a network over the course of training. On very low-resource\nlanguage pairs, we show that auto-sizing can improve BLEU scores by up to 3.9\npoints while removing one-third of the parameters from the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.06826v1",
            "date": "2019-10-12 18:23:42+00:00",
            "title": "Statically Detecting Vulnerabilities by Processing Programming Languages as Natural Languages",
            "authors": [
                "Ib\u00e9ria Medeiros",
                "Nuno Neves",
                "Miguel Correia"
            ],
            "abstract": "Web applications continue to be a favorite target for hackers due to a\ncombination of wide adoption and rapid deployment cycles, which often lead to\nthe introduction of high impact vulnerabilities. Static analysis tools are\nimportant to search for bugs automatically in the program source code,\nsupporting developers on their removal. However, building these tools requires\nprogramming the knowledge on how to discover the vulnerabilities. This paper\npresents an alternative approach in which tools learn to detect flaws\nautomatically by resorting to artificial intelligence concepts, more concretely\nto natural language processing. The approach employs a sequence model to learn\nto characterize vulnerabilities based on an annotated corpus. Afterwards, the\nmodel is utilized to discover and identify vulnerabilities in the source code.\nIt was implemented in the DEKANT tool and evaluated experimentally with a large\nset of PHP applications and WordPress plugins. Overall, we found several\nhundred vulnerabilities belonging to 12 classes of input validation\nvulnerabilities, where 62 of them were zero-day.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1910.06954v3",
            "date": "2019-10-15 17:51:01+00:00",
            "title": "Context Matters: Recovering Human Semantic Structure from Machine Learning Analysis of Large-Scale Text Corpora",
            "authors": [
                "Marius C\u0103t\u0103lin Iordan",
                "Tyler Giallanza",
                "Cameron T. Ellis",
                "Nicole M. Beckage",
                "Jonathan D. Cohen"
            ],
            "abstract": "Applying machine learning algorithms to large-scale, text-based corpora\n(embeddings) presents a unique opportunity to investigate at scale how human\nsemantic knowledge is organized and how people use it to judge fundamental\nrelationships, such as similarity between concepts. However, efforts to date\nhave shown a substantial discrepancy between algorithm predictions and\nempirical judgments. Here, we introduce a novel approach of generating\nembeddings motivated by the psychological theory that semantic context plays a\ncritical role in human judgments. Specifically, we train state-of-the-art\nmachine learning algorithms using contextually-constrained text corpora and\nshow that this greatly improves predictions of similarity judgments and feature\nratings. By improving the correspondence between representations derived using\nembeddings generated by machine learning methods and empirical measurements of\nhuman judgments, the approach we describe helps advance the use of large-scale\ntext corpora to understand the structure of human semantic representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07117v5",
            "date": "2019-10-16 01:10:10+00:00",
            "title": "Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models",
            "authors": [
                "Tianxing He",
                "Jun Liu",
                "Kyunghyun Cho",
                "Myle Ott",
                "Bing Liu",
                "James Glass",
                "Fuchun Peng"
            ],
            "abstract": "In this work, we study how the finetuning stage in the pretrain-finetune\nframework changes the behavior of a pretrained neural language generator. We\nfocus on the transformer encoder-decoder model for the open-domain dialogue\nresponse generation task. Our major finding is that after standard finetuning,\nthe model forgets some of the important language generation skills acquired\nduring large-scale pretraining. We demonstrate the forgetting phenomenon\nthrough a set of detailed behavior analysis from the perspectives of knowledge\ntransfer, context sensitivity, and function space projection. As a preliminary\nattempt to alleviate the forgetting problem, we propose an intuitive finetuning\nstrategy named \"mix-review\". We find that mix-review effectively regularizes\nthe finetuning process, and the forgetting problem is alleviated to some\nextent. Finally, we discuss interesting behavior of the resulting dialogue\nmodel and its implications.",
            "journal ref": "EACL 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07150v1",
            "date": "2019-10-16 03:28:14+00:00",
            "title": "Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding",
            "authors": [
                "Jiewen Wu",
                "Luis Fernando D'Haro",
                "Nancy F. Chen",
                "Pavitra Krishnaswamy",
                "Rafael E. Banchs"
            ],
            "abstract": "We propose an architecture to jointly learn word and label embeddings for\nslot filling in spoken language understanding. The proposed approach encodes\nlabels using a combination of word embeddings and straightforward word-label\nassociation from the training data. Compared to the state-of-the-art methods,\nour approach does not require label embeddings as part of the input and\ntherefore lends itself nicely to a wide range of model architectures. In\naddition, our architecture computes contextual distances between words and\nlabels to avoid adding contextual windows, thus reducing memory footprint. We\nvalidate the approach on established spoken dialogue datasets and show that it\ncan achieve state-of-the-art performance with much fewer trainable parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07323v1",
            "date": "2019-10-16 12:55:34+00:00",
            "title": "Lead2Gold: Towards exploiting the full potential of noisy transcriptions for speech recognition",
            "authors": [
                "Adrien Dufraux",
                "Emmanuel Vincent",
                "Awni Hannun",
                "Armelle Brun",
                "Matthijs Douze"
            ],
            "abstract": "The transcriptions used to train an Automatic Speech Recognition (ASR) system\nmay contain errors. Usually, either a quality control stage discards\ntranscriptions with too many errors, or the noisy transcriptions are used as\nis. We introduce Lead2Gold, a method to train an ASR system that exploits the\nfull potential of noisy transcriptions. Based on a noise model of transcription\nerrors, Lead2Gold searches for better transcriptions of the training data with\na beam search that takes this noise model into account. The beam search is\ndifferentiable and does not require a forced alignment step, thus the whole\nsystem is trained end-to-end. Lead2Gold can be viewed as a new loss function\nthat can be used on top of any sequence-to-sequence deep neural network. We\nconduct proof-of-concept experiments on noisy transcriptions generated from\nletter corruptions with different noise levels. We show that Lead2Gold obtains\na better ASR accuracy than a competitive baseline which does not account for\nthe (artificially-introduced) transcription noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07467v1",
            "date": "2019-10-16 16:44:22+00:00",
            "title": "Root Mean Square Layer Normalization",
            "authors": [
                "Biao Zhang",
                "Rico Sennrich"
            ],
            "abstract": "Layer normalization (LayerNorm) has been successfully applied to various deep\nneural networks to help stabilize training and boost model convergence because\nof its capability in handling re-centering and re-scaling of both inputs and\nweight matrix. However, the computational overhead introduced by LayerNorm\nmakes these improvements expensive and significantly slows the underlying\nnetwork, e.g. RNN in particular. In this paper, we hypothesize that\nre-centering invariance in LayerNorm is dispensable and propose root mean\nsquare layer normalization, or RMSNorm. RMSNorm regularizes the summed inputs\nto a neuron in one layer according to root mean square (RMS), giving the model\nre-scaling invariance property and implicit learning rate adaptation ability.\nRMSNorm is computationally simpler and thus more efficient than LayerNorm. We\nalso present partial RMSNorm, or pRMSNorm where the RMS is estimated from p% of\nthe summed inputs without breaking the above properties. Extensive experiments\non several tasks using diverse network architectures show that RMSNorm achieves\ncomparable performance against LayerNorm but reduces the running time by 7%~64%\non different models. Source code is available at\nhttps://github.com/bzhangGo/rmsnorm.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.07475v3",
            "date": "2019-10-16 17:05:21+00:00",
            "title": "MLQA: Evaluating Cross-lingual Extractive Question Answering",
            "authors": [
                "Patrick Lewis",
                "Barlas O\u011fuz",
                "Ruty Rinott",
                "Sebastian Riedel",
                "Holger Schwenk"
            ],
            "abstract": "Question answering (QA) models have shown rapid progress enabled by the\navailability of large, high-quality benchmark datasets. Such annotated datasets\nare difficult and costly to collect, and rarely exist in languages other than\nEnglish, making training QA systems in other languages challenging. An\nalternative to building large monolingual training datasets is to develop\ncross-lingual systems which can transfer to a target language without requiring\ntraining data in that language. In order to develop such systems, it is crucial\nto invest in high quality multilingual evaluation benchmarks to measure\nprogress. We present MLQA, a multi-way aligned extractive QA evaluation\nbenchmark intended to spur research in this area. MLQA contains QA instances in\n7 languages, namely English, Arabic, German, Spanish, Hindi, Vietnamese and\nSimplified Chinese. It consists of over 12K QA instances in English and 5K in\neach other language, with each QA instance being parallel between 4 languages\non average. MLQA is built using a novel alignment context strategy on Wikipedia\narticles, and serves as a cross-lingual extension to existing extractive QA\ndatasets. We evaluate current state-of-the-art cross-lingual representations on\nMLQA, and also provide machine-translation-based baselines. In all cases,\ntransfer results are shown to be significantly behind training-language\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07514v1",
            "date": "2019-10-16 17:58:36+00:00",
            "title": "Designing Style Matching Conversational Agents",
            "authors": [
                "Deepali Aneja",
                "Rens Hoegen",
                "Daniel McDuff",
                "Mary Czerwinski"
            ],
            "abstract": "Advances in machine intelligence have enabled conversational interfaces that\nhave the potential to radically change the way humans interact with machines.\nHowever, even with the progress in the abilities of these agents, there remain\ncritical gaps in their capacity for natural interactions. One limitation is\nthat the agents are often monotonic in behavior and do not adapt to their\npartner. We built two end-to-end conversational agents: a voice-based agent\nthat can engage in naturalistic, multi-turn dialogue and align with the\ninterlocutor's conversational style, and a 2nd, expressive, embodied\nconversational agent (ECA) that can recognize human behavior during open-ended\nconversations and automatically align its responses to the visual and\nconversational style of the other party. The embodied conversational agent\nleverages multimodal inputs to produce rich and perceptually valid vocal and\nfacial responses (e.g., lip syncing and expressions) during the conversation.\nBased on empirical results from a set of user studies, we highlight several\nsignificant challenges in building such systems and provide design guidelines\nfor multi-turn dialogue interactions using style adaptation for future\nresearch.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1910.07897v1",
            "date": "2019-10-17 13:31:45+00:00",
            "title": "Keyphrase Extraction from Disaster-related Tweets",
            "authors": [
                "Jishnu Ray Chowdhury",
                "Cornelia Caragea",
                "Doina Caragea"
            ],
            "abstract": "While keyphrase extraction has received considerable attention in recent\nyears, relatively few studies exist on extracting keyphrases from social media\nplatforms such as Twitter, and even fewer for extracting disaster-related\nkeyphrases from such sources. During a disaster, keyphrases can be extremely\nuseful for filtering relevant tweets that can enhance situational awareness.\nPreviously, joint training of two different layers of a stacked Recurrent\nNeural Network for keyword discovery and keyphrase extraction had been shown to\nbe effective in extracting keyphrases from general Twitter data. We improve the\nmodel's performance on both general Twitter data and disaster-related Twitter\ndata by incorporating contextual word embeddings, POS-tags, phonetics, and\nphonological features. Moreover, we discuss the shortcomings of the often used\nF1-measure for evaluating the quality of predicted keyphrases with respect to\nthe ground truth annotations. Instead of the F1-measure, we propose the use of\nembedding-based metrics to better capture the correctness of the predicted\nkeyphrases. In addition, we also present a novel extension of an\nembedding-based metric. The extension allows one to better control the penalty\nfor the difference in the number of ground-truth and predicted keyphrases",
            "journal ref": "In The World Wide Web Conference (WWW '19), Ling Liu and Ryen\n  White (Eds.). ACM, New York, NY, USA, 1555-1566 (2019)",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.07900v2",
            "date": "2019-10-17 13:33:41+00:00",
            "title": "H-VECTORS: Utterance-level Speaker Embedding Using A Hierarchical Attention Model",
            "authors": [
                "Yanpei Shi",
                "Qiang Huang",
                "Thomas Hain"
            ],
            "abstract": "In this paper, a hierarchical attention network to generate utterance-level\nembeddings (H-vectors) for speaker identification is proposed. Since different\nparts of an utterance may have different contributions to speaker identities,\nthe use of hierarchical structure aims to learn speaker related information\nlocally and globally. In the proposed approach, frame-level encoder and\nattention are applied on segments of an input utterance and generate individual\nsegment vectors. Then, segment level attention is applied on the segment\nvectors to construct an utterance representation. To evaluate the effectiveness\nof the proposed approach, NIST SRE 2008 Part1 dataset is used for training, and\ntwo datasets, Switchboard Cellular part1 and CallHome American English Speech,\nare used to evaluate the quality of extracted utterance embeddings on speaker\nidentification and verification tasks. In comparison with two baselines,\nX-vector, X-vector+Attention, the obtained results show that H-vectors can\nachieve a significantly better performance. Furthermore, the extracted\nutterance-level embeddings are more discriminative than the two baselines when\nmapped into a 2D space using t-SNE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08210v6",
            "date": "2019-10-18 00:49:15+00:00",
            "title": "RTFM: Generalising to Novel Environment Dynamics via Reading",
            "authors": [
                "Victor Zhong",
                "Tim Rockt\u00e4schel",
                "Edward Grefenstette"
            ],
            "abstract": "Obtaining policies that can generalise to new environments in reinforcement\nlearning is challenging. In this work, we demonstrate that language\nunderstanding via a reading policy learner is a promising vehicle for\ngeneralisation to new environments. We propose a grounded policy learning\nproblem, Read to Fight Monsters (RTFM), in which the agent must jointly reason\nover a language goal, relevant dynamics described in a document, and\nenvironment observations. We procedurally generate environment dynamics and\ncorresponding language descriptions of the dynamics, such that agents must read\nto understand new environment dynamics instead of memorising any particular\ninformation. In addition, we propose txt2$\\pi$, a model that captures three-way\ninteractions between the goal, document, and observations. On RTFM, txt2$\\pi$\ngeneralises to new environments with dynamics not seen during training via\nreading. Furthermore, our model outperforms baselines such as FiLM and\nlanguage-conditioned CNNs on RTFM. Through curriculum learning, txt2$\\pi$\nproduces policies that excel on complex RTFM tasks requiring several reasoning\nand coreference steps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08270v2",
            "date": "2019-10-18 05:28:46+00:00",
            "title": "Learning to Answer Subjective, Specific Product-Related Queries using Customer Reviews by Adversarial Domain Adaptation",
            "authors": [
                "Manirupa Das",
                "Zhen Wang",
                "Evan Jaffe",
                "Madhuja Chattopadhyay",
                "Eric Fosler-Lussier",
                "Rajiv Ramnath"
            ],
            "abstract": "Online customer reviews on large-scale e-commerce websites, represent a rich\nand varied source of opinion data, often providing subjective qualitative\nassessments of product usage that can help potential customers to discover\nfeatures that meet their personal needs and preferences. Thus they have the\npotential to automatically answer specific queries about products, and to\naddress the problems of answer starvation and answer augmentation on associated\nconsumer Q & A forums, by providing good answer alternatives. In this work, we\nexplore several recently successful neural approaches to modeling sentence\npairs, that could better learn the relationship between questions and ground\ntruth answers, and thus help infer reviews that can best answer a question or\naugment a given answer. In particular, we hypothesize that our adversarial\ndomain adaptation-based approach, due to its ability to additionally learn\ndomain-invariant features from a large number of unlabeled, unpaired\nquestion-review samples, would perform better than our proposed baselines, at\nanswering specific, subjective product-related queries using reviews. We\nvalidate this hypothesis using a small gold standard dataset of question-review\npairs evaluated by human experts, significantly surpassing our chosen\nbaselines. Moreover, our approach, using no labeled question-review sentence\npair data for training, gives performance at par with another method utilizing\nlabeled question-review samples for the same task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08282v2",
            "date": "2019-10-18 06:49:55+00:00",
            "title": "Unsupervised Context Rewriting for Open Domain Conversation",
            "authors": [
                "Kun Zhou",
                "Kai Zhang",
                "Yu Wu",
                "Shujie Liu",
                "Jingsong Yu"
            ],
            "abstract": "Context modeling has a pivotal role in open domain conversation. Existing\nworks either use heuristic methods or jointly learn context modeling and\nresponse generation with an encoder-decoder framework. This paper proposes an\nexplicit context rewriting method, which rewrites the last utterance by\nconsidering context history. We leverage pseudo-parallel data and elaborate a\ncontext rewriting network, which is built upon the CopyNet with the\nreinforcement learning method. The rewritten utterance is beneficial to\ncandidate retrieval, explainable context modeling, as well as enabling to\nemploy a single-turn framework to the multi-turn scenario. The empirical\nresults show that our model outperforms baselines in terms of the rewriting\nquality, the multi-turn response generation, and the end-to-end retrieval-based\nchatbots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08293v3",
            "date": "2019-10-18 07:52:01+00:00",
            "title": "ALOHA: Artificial Learning of Human Attributes for Dialogue Agents",
            "authors": [
                "Aaron W. Li",
                "Veronica Jiang",
                "Steven Y. Feng",
                "Julia Sprague",
                "Wei Zhou",
                "Jesse Hoey"
            ],
            "abstract": "For conversational AI and virtual assistants to communicate with humans in a\nrealistic way, they must exhibit human characteristics such as expression of\nemotion and personality. Current attempts toward constructing human-like\ndialogue agents have presented significant difficulties. We propose Human Level\nAttributes (HLAs) based on tropes as the basis of a method for learning\ndialogue agents that can imitate the personalities of fictional characters.\nTropes are characteristics of fictional personalities that are observed\nrecurrently and determined by viewers' impressions. By combining detailed HLA\ndata with dialogue data for specific characters, we present a dataset,\nHLA-Chat, that models character profiles and gives dialogue agents the ability\nto learn characters' language styles through their HLAs. We then introduce a\nthree-component system, ALOHA (which stands for Artificial Learning of Human\nAttributes), that combines character space mapping, character community\ndetection, and language style retrieval to build a character (or personality)\nspecific language model. Our preliminary experiments demonstrate that two\nvariations of ALOHA, combined with our proposed dataset, can outperform\nbaseline models at identifying the correct dialogue responses of chosen target\ncharacters, and are stable regardless of the character's identity, the genre of\nthe show, and the context of the dialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08534v1",
            "date": "2019-10-18 17:59:59+00:00",
            "title": "Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification",
            "authors": [
                "Vivian Lai",
                "Jon Z. Cai",
                "Chenhao Tan"
            ],
            "abstract": "Feature importance is commonly used to explain machine predictions. While\nfeature importance can be derived from a machine learning model with a variety\nof methods, the consistency of feature importance via different methods remains\nunderstudied. In this work, we systematically compare feature importance from\nbuilt-in mechanisms in a model such as attention values and post-hoc methods\nthat approximate model behavior such as LIME. Using text classification as a\ntestbed, we find that 1) no matter which method we use, important features from\ntraditional models such as SVM and XGBoost are more similar with each other,\nthan with deep learning models; 2) post-hoc methods tend to generate more\nsimilar important features for two models than built-in methods. We further\ndemonstrate how such similarity varies across instances. Notably, important\nfeatures do not always resemble each other better when two models agree on the\npredicted label than when they disagree.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08902v1",
            "date": "2019-10-20 05:12:23+00:00",
            "title": "Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations",
            "authors": [
                "Oluwaseyi Feyisetan",
                "Borja Balle",
                "Thomas Drake",
                "Tom Diethe"
            ],
            "abstract": "Accurately learning from user data while providing quantifiable privacy\nguarantees provides an opportunity to build better ML models while maintaining\nuser trust. This paper presents a formal approach to carrying out privacy\npreserving text perturbation using the notion of dx-privacy designed to achieve\ngeo-indistinguishability in location data. Our approach applies carefully\ncalibrated noise to vector representation of words in a high dimension space as\ndefined by word embedding models. We present a privacy proof that satisfies\ndx-privacy where the privacy parameter epsilon provides guarantees with respect\nto a distance metric defined by the word embedding space. We demonstrate how\nepsilon can be selected by analyzing plausible deniability statistics backed up\nby large scale analysis on GloVe and fastText embeddings. We conduct privacy\naudit experiments against 2 baseline models and utility experiments on 3\ndatasets to demonstrate the tradeoff between privacy and utility for varying\nvalues of epsilon on different task types. Our results demonstrate practical\nutility (< 2% utility loss for training binary classifiers) while providing\nbetter privacy guarantees than baseline models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.08910v2",
            "date": "2019-10-20 06:43:21+00:00",
            "title": "Improving Sequence Modeling Ability of Recurrent Neural Networks via Sememes",
            "authors": [
                "Yujia Qin",
                "Fanchao Qi",
                "Sicong Ouyang",
                "Zhiyuan Liu",
                "Cheng Yang",
                "Yasheng Wang",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Sememes, the minimum semantic units of human languages, have been\nsuccessfully utilized in various natural language processing applications.\nHowever, most existing studies exploit sememes in specific tasks and few\nefforts are made to utilize sememes more fundamentally. In this paper, we\npropose to incorporate sememes into recurrent neural networks (RNNs) to improve\ntheir sequence modeling ability, which is beneficial to all kinds of downstream\ntasks. We design three different sememe incorporation methods and employ them\nin typical RNNs including LSTM, GRU and their bidirectional variants. In\nevaluation, we use several benchmark datasets involving PTB and WikiText-2 for\nlanguage modeling, SNLI for natural language inference and another two datasets\nfor sentiment analysis and paraphrase detection. Experimental results show\nevident and consistent improvement of our sememe-incorporated models compared\nwith vanilla RNNs, which proves the effectiveness of our sememe incorporation\nmethods. Moreover, we find the sememe-incorporated models have higher\nrobustness and outperform adversarial training in defending adversarial attack.\nAll the code and data of this work can be obtained at\nhttps://github.com/thunlp/SememeRNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08917v1",
            "date": "2019-10-20 07:16:29+00:00",
            "title": "Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text",
            "authors": [
                "Oluwaseyi Feyisetan",
                "Tom Diethe",
                "Thomas Drake"
            ],
            "abstract": "Guaranteeing a certain level of user privacy in an arbitrary piece of text is\na challenging issue. However, with this challenge comes the potential of\nunlocking access to vast data stores for training machine learning models and\nsupporting data driven decisions. We address this problem through the lens of\ndx-privacy, a generalization of Differential Privacy to non Hamming distance\nmetrics. In this work, we explore word representations in Hyperbolic space as a\nmeans of preserving privacy in text. We provide a proof satisfying dx-privacy,\nthen we define a probability distribution in Hyperbolic space and describe a\nway to sample from it in high dimensions. Privacy is provided by perturbing\nvector representations of words in high dimensional Hyperbolic space to obtain\na semantic generalization. We conduct a series of experiments to demonstrate\nthe tradeoff between privacy and utility. Our privacy experiments illustrate\nprotections against an authorship attribution algorithm while our utility\nexperiments highlight the minimal impact of our perturbations on several\ndownstream machine learning models. Compared to the Euclidean baseline, we\nobserve > 20x greater guarantees on expected privacy against comparable worst\ncase statistics.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.08948v1",
            "date": "2019-10-20 11:05:05+00:00",
            "title": "Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information",
            "authors": [
                "Yoan Dinkov",
                "Ahmed Ali",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "We address the problem of predicting the leading political ideology, i.e.,\nleft-center-right bias, for YouTube channels of news media. Previous work on\nthe problem has focused exclusively on text and on analysis of the language\nused, topics discussed, sentiment, and the like. In contrast, here we study\nvideos, which yields an interesting multimodal setup. Starting with gold\nannotations about the leading political ideology of major world news media from\nMedia Bias/Fact Check, we searched on YouTube to find their corresponding\nchannels, and we downloaded a recent sample of videos from each channel. We\ncrawled more than 1,000 YouTube hours along with the corresponding subtitles\nand metadata, thus producing a new multimodal dataset. We further developed a\nmultimodal deep-learning architecture for the task. Our analysis shows that the\nuse of acoustic signal helped to improve bias detection by more than 6%\nabsolute over using text and metadata only. We release the dataset to the\nresearch community, hoping to help advance the field of multi-modal political\nbias detection.",
            "journal ref": "INTERSPEECH-2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08987v2",
            "date": "2019-10-20 14:18:51+00:00",
            "title": "Representation Learning for Discovering Phonemic Tone Contours",
            "authors": [
                "Bai Li",
                "Jing Yi Xie",
                "Frank Rudzicz"
            ],
            "abstract": "Tone is a prosodic feature used to distinguish words in many languages, some\nof which are endangered and scarcely documented. In this work, we use\nunsupervised representation learning to identify probable clusters of syllables\nthat share the same phonemic tone. Our method extracts the pitch for each\nsyllable, then trains a convolutional autoencoder to learn a low dimensional\nrepresentation for each contour. We then apply the mean shift algorithm to\ncluster tones in high-density regions of the latent space. Furthermore, by\nfeeding the centers of each cluster into the decoder, we produce a prototypical\ncontour that represents each cluster. We apply this method to spoken\nmulti-syllable words in Mandarin Chinese and Cantonese and evaluate how closely\nour clusters match the ground truth tone categories. Finally, we discuss some\ndifficulties with our approach, including contextual tone variation and\nallophony effects.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.09058v5",
            "date": "2019-10-20 20:24:39+00:00",
            "title": "Deep speech inpainting of time-frequency masks",
            "authors": [
                "Mikolaj Kegler",
                "Pierre Beckmann",
                "Milos Cernak"
            ],
            "abstract": "Transient loud intrusions, often occurring in noisy environments, can\ncompletely overpower speech signal and lead to an inevitable loss of\ninformation. While existing algorithms for noise suppression can yield\nimpressive results, their efficacy remains limited for very low signal-to-noise\nratios or when parts of the signal are missing. To address these limitations,\nhere we propose an end-to-end framework for speech inpainting, the\ncontext-based retrieval of missing or severely distorted parts of\ntime-frequency representation of speech. The framework is based on a\nconvolutional U-Net trained via deep feature losses, obtained using speechVGG,\na deep speech feature extractor pre-trained on an auxiliary word classification\ntask. Our evaluation results demonstrate that the proposed framework can\nrecover large portions of missing or distorted time-frequency representation of\nspeech, up to 400 ms and 3.2 kHz in bandwidth. In particular, our approach\nprovided a substantial increase in STOI & PESQ objective metrics of the\ninitially corrupted speech samples. Notably, using deep feature losses to train\nthe framework led to the best results, as compared to conventional approaches.",
            "journal ref": "Proc. Interspeech 2020, 3276-3280",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.09113v3",
            "date": "2019-10-21 02:12:51+00:00",
            "title": "Discovering the Compositional Structure of Vector Representations with Role Learning Networks",
            "authors": [
                "Paul Soulos",
                "Tom McCoy",
                "Tal Linzen",
                "Paul Smolensky"
            ],
            "abstract": "How can neural networks perform so well on compositional tasks even though\nthey lack explicit compositional representations? We use a novel analysis\ntechnique called ROLE to show that recurrent neural networks perform well on\nsuch tasks by converging to solutions which implicitly represent symbolic\nstructure. This method uncovers a symbolic structure which, when properly\nembedded in vector space, closely approximates the encodings of a standard\nseq2seq network trained to perform the compositional SCAN task. We verify the\ncausal importance of the discovered symbolic structure by showing that, when we\nsystematically manipulate hidden embeddings based on this symbolic structure,\nthe model's output is changed in the way predicted by our analysis.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.09129v2",
            "date": "2019-10-21 03:09:02+00:00",
            "title": "A Comparison of Semantic Similarity Methods for Maximum Human Interpretability",
            "authors": [
                "Pinky Sitikhu",
                "Kritish Pahi",
                "Pujan Thapa",
                "Subarna Shakya"
            ],
            "abstract": "The inclusion of semantic information in any similarity measures improves the\nefficiency of the similarity measure and provides human interpretable results\nfor further analysis. The similarity calculation method that focuses on\nfeatures related to the text's words only, will give less accurate results.\nThis paper presents three different methods that not only focus on the text's\nwords but also incorporates semantic information of texts in their feature\nvector and computes semantic similarities. These methods are based on\ncorpus-based and knowledge-based methods, which are: cosine similarity using\ntf-idf vectors, cosine similarity using word embedding and soft cosine\nsimilarity using word embedding. Among these three, cosine similarity using\ntf-idf vectors performed best in finding similarities between short news texts.\nThe similar texts given by the method are easy to interpret and can be used\ndirectly in other information retrieval applications.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.09255v3",
            "date": "2019-10-21 10:19:53+00:00",
            "title": "Constructing Artificial Data for Fine-tuning for Low-Resource Biomedical Text Tagging with Applications in PICO Annotation",
            "authors": [
                "Gaurav Singh",
                "Zahra Sabet",
                "John Shawe-Taylor",
                "James Thomas"
            ],
            "abstract": "Biomedical text tagging systems are plagued by the dearth of labeled training\ndata. There have been recent attempts at using pre-trained encoders to deal\nwith this issue. Pre-trained encoder provides representation of the input text\nwhich is then fed to task-specific layers for classification. The entire\nnetwork is fine-tuned on the labeled data from the target task. Unfortunately,\na low-resource biomedical task often has too few labeled instances for\nsatisfactory fine-tuning. Also, if the label space is large, it contains few or\nno labeled instances for majority of the labels. Most biomedical tagging\nsystems treat labels as indexes, ignoring the fact that these labels are often\nconcepts expressed in natural language e.g. `Appearance of lesion on brain\nimaging'. To address these issues, we propose constructing extra labeled\ninstances using label-text (i.e. label's name) as input for the corresponding\nlabel-index (i.e. label's index). In fact, we propose a number of strategies\nfor manufacturing multiple artificial labeled instances from a single label.\nThe network is then fine-tuned on a combination of real and these newly\nconstructed artificial labeled instances. We evaluate the proposed approach on\nan important low-resource biomedical task called \\textit{PICO annotation},\nwhich requires tagging raw text describing clinical trials with labels\ncorresponding to different aspects of the trial i.e. PICO (Population,\nIntervention/Control, Outcome) characteristics of the trial. Our empirical\nresults show that the proposed method achieves a new state-of-the-art\nperformance for PICO annotation with very significant improvements over\ncompetitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09387v1",
            "date": "2019-10-21 14:06:01+00:00",
            "title": "Clotho: An Audio Captioning Dataset",
            "authors": [
                "Konstantinos Drossos",
                "Samuel Lipping",
                "Tuomas Virtanen"
            ],
            "abstract": "Audio captioning is the novel task of general audio content description using\nfree text. It is an intermodal translation task (not speech-to-text), where a\nsystem accepts as an input an audio signal and outputs the textual description\n(i.e. the caption) of that signal. In this paper we present Clotho, a dataset\nfor audio captioning consisting of 4981 audio samples of 15 to 30 seconds\nduration and 24 905 captions of eight to 20 words length, and a baseline method\nto provide initial results. Clotho is built with focus on audio content and\ncaption diversity, and the splits of the data are not hampering the training or\nevaluation of methods. All sounds are from the Freesound platform, and captions\nare crowdsourced using Amazon Mechanical Turk and annotators from English\nspeaking countries. Unique words, named entities, and speech transcription are\nremoved with post-processing. Clotho is freely available online\n(https://zenodo.org/record/3490684).",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.09399v1",
            "date": "2019-10-21 14:23:14+00:00",
            "title": "A Survey and Taxonomy of Adversarial Neural Networks for Text-to-Image Synthesis",
            "authors": [
                "Jorge Agnese",
                "Jonathan Herrera",
                "Haicheng Tao",
                "Xingquan Zhu"
            ],
            "abstract": "Text-to-image synthesis refers to computational methods which translate human\nwritten textual descriptions, in the form of keywords or sentences, into images\nwith similar semantic meaning to the text. In earlier research, image synthesis\nrelied mainly on word to image correlation analysis combined with supervised\nmethods to find best alignment of the visual content matching to the text.\nRecent progress in deep learning (DL) has brought a new set of unsupervised\ndeep learning methods, particularly deep generative models which are able to\ngenerate realistic visual images using suitably trained neural network models.\nIn this paper, we review the most recent development in the text-to-image\nsynthesis research domain. Our survey first introduces image synthesis and its\nchallenges, and then reviews key concepts such as generative adversarial\nnetworks (GANs) and deep convolutional encoder-decoder neural networks (DCNN).\nAfter that, we propose a taxonomy to summarize GAN based text-to-image\nsynthesis into four major categories: Semantic Enhancement GANs, Resolution\nEnhancement GANs, Diversity Enhancement GANS, and Motion Enhancement GANs. We\nelaborate the main objective of each group, and further review typical GAN\narchitectures in each group. The taxonomy and the review outline the techniques\nand the evolution of different approaches, and eventually provide a clear\nroadmap to summarize the list of contemporaneous solutions that utilize GANs\nand DCNNs to generate enthralling results in categories such as human faces,\nbirds, flowers, room interiors, object reconstruction from edge maps (games)\netc. The survey will conclude with a comparison of the proposed solutions,\nchallenges that remain unresolved, and future developments in the text-to-image\nsynthesis domain.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.09451v3",
            "date": "2019-10-21 15:31:29+00:00",
            "title": "HIGhER : Improving instruction following with Hindsight Generation for Experience Replay",
            "authors": [
                "Geoffrey Cideron",
                "Mathieu Seurin",
                "Florian Strub",
                "Olivier Pietquin"
            ],
            "abstract": "Language creates a compact representation of the world and allows the\ndescription of unlimited situations and objectives through compositionality.\nWhile these characterizations may foster instructing, conditioning or\nstructuring interactive agent behavior, it remains an open-problem to correctly\nrelate language understanding and reinforcement learning in even simple\ninstruction following scenarios. This joint learning problem is alleviated\nthrough expert demonstrations, auxiliary losses, or neural inductive biases. In\nthis paper, we propose an orthogonal approach called Hindsight Generation for\nExperience Replay (HIGhER) that extends the Hindsight Experience Replay (HER)\napproach to the language-conditioned policy setting. Whenever the agent does\nnot fulfill its instruction, HIGhER learns to output a new directive that\nmatches the agent trajectory, and it relabels the episode with a positive\nreward. To do so, HIGhER learns to map a state into an instruction by using\npast successful trajectories, which removes the need to have external expert\ninterventions to relabel episodes as in vanilla HER. We show the efficiency of\nour approach in the BabyAI environment, and demonstrate how it complements\nother instruction following methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.09563v1",
            "date": "2019-10-21 18:00:04+00:00",
            "title": "Content Removal as a Moderation Strategy: Compliance and Other Outcomes in the ChangeMyView Community",
            "authors": [
                "Kumar Bhargav Srinivasan",
                "Cristian Danescu-Niculescu-Mizil",
                "Lillian Lee",
                "Chenhao Tan"
            ],
            "abstract": "Moderators of online communities often employ comment deletion as a tool. We\nask here whether, beyond the positive effects of shielding a community from\nundesirable content, does comment removal actually cause the behavior of the\ncomment's author to improve? We examine this question in a particularly\nwell-moderated community, the ChangeMyView subreddit.\n  The standard analytic approach of interrupted time-series analysis\nunfortunately cannot answer this question of causality because it fails to\ndistinguish the effect of having made a non-compliant comment from the effect\nof being subjected to moderator removal of that comment. We therefore leverage\na \"delayed feedback\" approach based on the observation that some users may\nremain active between the time when they posted the non-compliant comment and\nthe time when that comment is deleted. Applying this approach to such users, we\nreveal the causal role of comment deletion in reducing immediate noncompliance\nrates, although we do not find evidence of it having a causal role in inducing\nother behavior improvements. Our work thus empirically demonstrates both the\npromise and some potential limits of content removal as a positive moderation\nstrategy, and points to future directions for identifying causal effects from\nobservational data.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1910.09909v4",
            "date": "2019-10-22 11:58:59+00:00",
            "title": "Speech-VGG: A deep feature extractor for speech processing",
            "authors": [
                "Pierre Beckmann",
                "Mikolaj Kegler",
                "Hugues Saltini",
                "Milos Cernak"
            ],
            "abstract": "Recent breakthroughs in deep learning often rely on representation learning\nand knowledge transfer. In particular, readily available models pre-trained on\nlarge datasets are key for the efficient transfer of knowledge. They can be\napplied as feature extractors for data preprocessing, fine-tuned to perform a\nvariety of tasks, or used for computing feature losses in the training of deep\nlearning systems. While applications of transfer learning are common in the\nfields of computer vision and natural language processing, audio- and speech\nprocessing are surprisingly lacking readily available and transferable models.\nHere, we introduce speechVGG, a flexible, transferable feature extractor\ntailored for integration with deep learning frameworks for speech processing.\nOur transferable model adopts the classic VGG-16 architecture and is trained on\na spoken word classification task. We demonstrate the application of the\npre-trained model in four speech processing tasks, including speech\nenhancement, language identification, speech, noise and music classification,\nand speaker identification. Each time, we compare the performance of our\napproach to existing baselines. Our results confirm that the representation of\nnatural speech captured using speechVGG is transferable and generalizable\nacross various speech processing problems and datasets. Notably, relatively\nsimple applications of our pre-trained model are capable of achieving\ncompetitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09932v3",
            "date": "2019-10-22 12:47:29+00:00",
            "title": "Improving Transformer-based Speech Recognition Using Unsupervised Pre-training",
            "authors": [
                "Dongwei Jiang",
                "Xiaoning Lei",
                "Wubo Li",
                "Ne Luo",
                "Yuxuan Hu",
                "Wei Zou",
                "Xiangang Li"
            ],
            "abstract": "Speech recognition technologies are gaining enormous popularity in various\nindustrial applications. However, building a good speech recognition system\nusually requires large amounts of transcribed data, which is expensive to\ncollect. To tackle this problem, an unsupervised pre-training method called\nMasked Predictive Coding is proposed, which can be applied for unsupervised\npre-training with Transformer based model. Experiments on HKUST show that using\nthe same training data, we can achieve CER 23.3%, exceeding the best end-to-end\nmodel by over 0.2% absolute CER. With more pre-training data, we can further\nreduce the CER to 21.0%, or a 11.8% relative CER reduction over baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.09991v1",
            "date": "2019-10-22 14:07:17+00:00",
            "title": "Toward Automated Website Classification by Deep Learning",
            "authors": [
                "Fabrizio De Fausti",
                "Francesco Pugliese",
                "Diego Zardetto"
            ],
            "abstract": "In recent years, the interest in Big Data sources has been steadily growing\nwithin the Official Statistic community. The Italian National Institute of\nStatistics (Istat) is currently carrying out several Big Data pilot studies.\nOne of these studies, the ICT Big Data pilot, aims at exploiting massive\namounts of textual data automatically scraped from the websites of Italian\nenterprises in order to predict a set of target variables (e.g. e-commerce)\nthat are routinely observed by the traditional ICT Survey. In this paper, we\nshow that Deep Learning techniques can successfully address this problem.\nEssentially, we tackle a text classification task: an algorithm must learn to\ninfer whether an Italian enterprise performs e-commerce from the textual\ncontent of its website. To reach this goal, we developed a sophisticated\nprocessing pipeline and evaluated its performance through extensive\nexperiments. Our pipeline uses Convolutional Neural Networks and relies on Word\nEmbeddings to encode raw texts into grayscale images (i.e. normalized numeric\nmatrices). Web-scraped texts are huge and have very low signal to noise ratio:\nto overcome these issues, we adopted a framework known as False Positive\nReduction, which has seldom (if ever) been applied before to text\nclassification tasks. Several original contributions enable our processing\npipeline to reach good classification results. Empirical evidence shows that\nour proposal outperforms all the alternative Machine Learning solutions already\ntested in Istat for the same task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.10034v1",
            "date": "2019-10-22 15:09:02+00:00",
            "title": "Language-guided Semantic Mapping and Mobile Manipulation in Partially Observable Environments",
            "authors": [
                "Siddharth Patki",
                "Ethan Fahnestock",
                "Thomas M. Howard",
                "Matthew R. Walter"
            ],
            "abstract": "Recent advances in data-driven models for grounded language understanding\nhave enabled robots to interpret increasingly complex instructions. Two\nfundamental limitations of these methods are that most require a full model of\nthe environment to be known a priori, and they attempt to reason over a world\nrepresentation that is flat and unnecessarily detailed, which limits\nscalability. Recent semantic mapping methods address partial observability by\nexploiting language as a sensor to infer a distribution over topological,\nmetric and semantic properties of the environment. However, maintaining a\ndistribution over highly detailed maps that can support grounding of diverse\ninstructions is computationally expensive and hinders real-time human-robot\ncollaboration. We propose a novel framework that learns to adapt perception\naccording to the task in order to maintain compact distributions over semantic\nmaps. Experiments with a mobile manipulator demonstrate more efficient\ninstruction following in a priori unknown environments.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1910.10287v2",
            "date": "2019-10-23 00:17:26+00:00",
            "title": "RNN based Incremental Online Spoken Language Understanding",
            "authors": [
                "Prashanth Gurunath Shivakumar",
                "Naveen Kumar",
                "Panayiotis Georgiou",
                "Shrikanth Narayanan"
            ],
            "abstract": "Spoken Language Understanding (SLU) typically comprises of an automatic\nspeech recognition (ASR) followed by a natural language understanding (NLU)\nmodule. The two modules process signals in a blocking sequential fashion, i.e.,\nthe NLU often has to wait for the ASR to finish processing on an utterance\nbasis, potentially leading to high latencies that render the spoken interaction\nless natural. In this paper, we propose recurrent neural network (RNN) based\nincremental processing towards the SLU task of intent detection. The proposed\nmethodology offers lower latencies than a typical SLU system, without any\nsignificant reduction in system accuracy. We introduce and analyze different\nrecurrent neural network architectures for incremental and online processing of\nthe ASR transcripts and compare it to the existing offline systems. A lexical\nEnd-of-Sentence (EOS) detector is proposed for segmenting the stream of\ntranscript into sentences for intent classification. Intent detection\nexperiments are conducted on benchmark ATIS, Snips and Facebook's multilingual\ntask oriented dialog datasets modified to emulate a continuous incremental\nstream of words with no utterance demarcation. We also analyze the prospects of\nearly intent detection, before EOS, with our proposed system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10288v2",
            "date": "2019-10-23 00:21:33+00:00",
            "title": "Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis",
            "authors": [
                "Eric Battenberg",
                "RJ Skerry-Ryan",
                "Soroosh Mariooryad",
                "Daisy Stanton",
                "David Kao",
                "Matt Shannon",
                "Tom Bagby"
            ],
            "abstract": "Despite the ability to produce human-level speech for in-domain text,\nattention-based end-to-end text-to-speech (TTS) systems suffer from text\nalignment failures that increase in frequency for out-of-domain text. We show\nthat these failures can be addressed using simple location-relative attention\nmechanisms that do away with content-based query/key comparisons. We compare\ntwo families of attention mechanisms: location-relative GMM-based mechanisms\nand additive energy-based mechanisms. We suggest simple modifications to\nGMM-based attention that allow it to align quickly and consistently during\ntraining, and introduce a new location-relative attention mechanism to the\nadditive energy-based family, called Dynamic Convolution Attention (DCA). We\ncompare the various mechanisms in terms of alignment speed and consistency\nduring training, naturalness, and ability to generalize to long utterances, and\nconclude that GMM attention and DCA can generalize to very long utterances,\nwhile preserving naturalness for shorter, in-domain utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10294v1",
            "date": "2019-10-23 00:50:29+00:00",
            "title": "A Unifying Framework of Bilinear LSTMs",
            "authors": [
                "Mohit Rajpal",
                "Bryan Kian Hsiang Low"
            ],
            "abstract": "This paper presents a novel unifying framework of bilinear LSTMs that can\nrepresent and utilize the nonlinear interaction of the input features present\nin sequence datasets for achieving superior performance over a linear LSTM and\nyet not incur more parameters to be learned. To realize this, our unifying\nframework allows the expressivity of the linear vs. bilinear terms to be\nbalanced by correspondingly trading off between the hidden state vector size\nvs. approximation quality of the weight matrix in the bilinear term so as to\noptimize the performance of our bilinear LSTM, while not incurring more\nparameters to be learned. We empirically evaluate the performance of our\nbilinear LSTM in several language-based sequence learning tasks to demonstrate\nits general applicability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.10324v2",
            "date": "2019-10-23 02:48:30+00:00",
            "title": "Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks",
            "authors": [
                "Andros Tjandra",
                "Chunxi Liu",
                "Frank Zhang",
                "Xiaohui Zhang",
                "Yongqiang Wang",
                "Gabriel Synnaeve",
                "Satoshi Nakamura",
                "Geoffrey Zweig"
            ],
            "abstract": "Deep acoustic models typically receive features in the first layer of the\nnetwork, and process increasingly abstract representations in the subsequent\nlayers. Here, we propose to feed the input features at multiple depths in the\nacoustic model. As our motivation is to allow acoustic models to re-examine\ntheir input features in light of partial hypotheses we introduce intermediate\nmodel heads and loss function. We study this architecture in the context of\ndeep Transformer networks, and we use an attention mechanism over both the\nprevious layer activations and the input features. To train this model's\nintermediate output hypothesis, we apply the objective function at each layer\nright before feature re-use. We find that the use of such iterated loss\nsignificantly improves performance by itself, as well as enabling input feature\nre-use. We present results on both Librispeech, and a large scale video\ndataset, with relative improvements of 10 - 20% for Librispeech and 3.2 - 13%\nfor videos.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10352v1",
            "date": "2019-10-23 04:57:51+00:00",
            "title": "A Transformer with Interleaved Self-attention and Convolution for Hybrid Acoustic Models",
            "authors": [
                "Liang Lu"
            ],
            "abstract": "Transformer with self-attention has achieved great success in the area of\nnature language processing. Recently, there have been a few studies on\ntransformer for end-to-end speech recognition, while its application for hybrid\nacoustic model is still very limited. In this paper, we revisit the\ntransformer-based hybrid acoustic model, and propose a model structure with\ninterleaved self-attention and 1D convolution, which is proven to have faster\nconvergence and higher recognition accuracy. We also study several aspects of\nthe transformer model, including the impact of the positional encoding feature,\ndropout regularization, as well as training with and without time restriction.\nWe show competitive recognition results on the public Librispeech dataset when\ncompared to the Kaldi baseline at both cross entropy training and sequence\ntraining stages. For reproducible research, we release our source code and\nrecipe within the PyKaldi2 toolbox.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.10363v2",
            "date": "2019-10-23 05:41:39+00:00",
            "title": "A Hybrid Semantic Parsing Approach for Tabular Data Analysis",
            "authors": [
                "Yan Gao",
                "Jian-Guang Lou",
                "Dongmei Zhang"
            ],
            "abstract": "This paper presents a novel approach to translating natural language\nquestions to SQL queries for given tables, which meets three requirements as a\nreal-world data analysis application: cross-domain, multilingualism and\nenabling quick-start. Our proposed approach consists of: (1) a novel data\nabstraction step before the parser to make parsing table-agnosticism; (2) a set\nof semantic rules for parsing abstracted data-analysis questions to\nintermediate logic forms as tree derivations to reduce the search space; (3) a\nneural-based model as a local scoring function on a span-based semantic parser\nfor structured optimization and efficient inference. Experiments show that our\napproach outperforms state-of-the-art algorithms on a large open benchmark\ndataset WikiSQL. We also achieve promising results on a small dataset for more\ncomplex queries in both English and Chinese, which demonstrates our language\nexpansion and quick-start ability.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1910.10479v1",
            "date": "2019-10-19 21:39:03+00:00",
            "title": "XL-Editor: Post-editing Sentences with XLNet",
            "authors": [
                "Yong-Siang Shih",
                "Wei-Cheng Chang",
                "Yiming Yang"
            ],
            "abstract": "While neural sequence generation models achieve initial success for many NLP\napplications, the canonical decoding procedure with left-to-right generation\norder (i.e., autoregressive) in one-pass can not reflect the true nature of\nhuman revising a sentence to obtain a refined result. In this work, we propose\nXL-Editor, a novel training framework that enables state-of-the-art generalized\nautoregressive pretraining methods, XLNet specifically, to revise a given\nsentence by the variable-length insertion probability. Concretely, XL-Editor\ncan (1) estimate the probability of inserting a variable-length sequence into a\nspecific position of a given sentence; (2) execute post-editing operations such\nas insertion, deletion, and replacement based on the estimated variable-length\ninsertion probability; (3) complement existing sequence-to-sequence models to\nrefine the generated sequences. Empirically, we first demonstrate better\npost-editing capabilities of XL-Editor over XLNet on the text insertion and\ndeletion tasks, which validates the effectiveness of our proposed framework.\nFurthermore, we extend XL-Editor to the unpaired text style transfer task,\nwhere transferring the target style onto a given sentence can be naturally\nviewed as post-editing the sentence into the target style. XL-Editor achieves\nsignificant improvement in style transfer accuracy and also maintains coherent\nsemantic of the original sentence, showing the broad applicability of our\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10485v3",
            "date": "2019-10-17 01:29:12+00:00",
            "title": "Fully Quantized Transformer for Machine Translation",
            "authors": [
                "Gabriele Prato",
                "Ella Charlaix",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "State-of-the-art neural machine translation methods employ massive amounts of\nparameters. Drastically reducing computational costs of such methods without\naffecting performance has been up to this point unsuccessful. To this end, we\npropose FullyQT: an all-inclusive quantization strategy for the Transformer. To\nthe best of our knowledge, we are the first to show that it is possible to\navoid any loss in translation quality with a fully quantized Transformer.\nIndeed, compared to full-precision, our 8-bit models score greater or equal\nBLEU on most tasks. Comparing ourselves to all previously proposed methods, we\nachieve state-of-the-art quantization results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10487v1",
            "date": "2019-10-16 15:59:33+00:00",
            "title": "Memory-Augmented Recurrent Networks for Dialogue Coherence",
            "authors": [
                "David Donahue",
                "Yuanliang Meng",
                "Anna Rumshisky"
            ],
            "abstract": "Recent dialogue approaches operate by reading each word in a conversation\nhistory, and aggregating accrued dialogue information into a single state. This\nfixed-size vector is not expandable and must maintain a consistent format over\ntime. Other recent approaches exploit an attention mechanism to extract useful\ninformation from past conversational utterances, but this introduces an\nincreased computational complexity. In this work, we explore the use of the\nNeural Turing Machine (NTM) to provide a more permanent and flexible storage\nmechanism for maintaining dialogue coherence. Specifically, we introduce two\nseparate dialogue architectures based on this NTM design. The first design\nfeatures a sequence-to-sequence architecture with two separate NTM modules, one\nfor each participant in the conversation. The second memory architecture\nincorporates a single NTM module, which stores parallel context information for\nboth speakers. This second design also replaces the sequence-to-sequence\narchitecture with a neural language model, to allow for longer context of the\nNTM and greater understanding of the dialogue history. We report perplexity\nperformance for both models, and compare them to existing baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10488v2",
            "date": "2019-10-16 15:48:46+00:00",
            "title": "Injecting Hierarchy with U-Net Transformers",
            "authors": [
                "David Donahue",
                "Vladislav Lialin",
                "Anna Rumshisky"
            ],
            "abstract": "The Transformer architecture has become increasingly popular over the past\ntwo years, owing to its impressive performance on a number of natural language\nprocessing (NLP) tasks. However, all Transformer computations occur at the\nlevel of word representations and therefore, it may be argued that Transformer\nmodels do not explicitly attempt to learn hierarchical structure which is\nwidely assumed to be integral to language. In the present work, we introduce\nhierarchical processing into the Transformer model, taking inspiration from the\nU-Net architecture, popular in computer vision for its hierarchical view of\nnatural images. We empirically demonstrate that the proposed architecture\noutperforms both the vanilla Transformer and some strong baselines in the\ndomain of chit-chat dialogue.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.10491v1",
            "date": "2019-10-18 06:17:07+00:00",
            "title": "Estimator Vectors: OOV Word Embeddings based on Subword and Context Clue Estimates",
            "authors": [
                "Raj Patel",
                "Carlotta Domeniconi"
            ],
            "abstract": "Semantic representations of words have been successfully extracted from\nunlabeled corpuses using neural network models like word2vec. These\nrepresentations are generally high quality and are computationally inexpensive\nto train, making them popular. However, these approaches generally fail to\napproximate out of vocabulary (OOV) words, a task humans can do quite easily,\nusing word roots and context clues. This paper proposes a neural network model\nthat learns high quality word representations, subword representations, and\ncontext clue representations jointly. Learning all three types of\nrepresentations together enhances the learning of each, leading to enriched\nword vectors, along with strong estimates for OOV words, via the combination of\nthe corresponding context clue and subword embeddings. Our model, called\nEstimator Vectors (EV), learns strong word embeddings and is competitive with\nstate of the art methods for OOV estimation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10495v2",
            "date": "2019-10-22 08:30:26+00:00",
            "title": "IPOD: An Industrial and Professional Occupations Dataset and its Applications to Occupational Data Mining and Analysis",
            "authors": [
                "Junhua Liu",
                "Yung Chuen Ng",
                "Kristin L. Wood",
                "Kwan Hui Lim"
            ],
            "abstract": "Occupational data mining and analysis is an important task in understanding\ntoday's industry and job market. Various machine learning techniques are\nproposed and gradually deployed to improve companies' operations for upstream\ntasks, such as employee churn prediction, career trajectory modelling and\nautomated interview. Job titles analysis and embedding, as the fundamental\nbuilding blocks, are crucial upstream tasks to address these occupational data\nmining and analysis problems. In this work, we present the Industrial and\nProfessional Occupations Dataset (IPOD), which consists of over 190,000 job\ntitles crawled from over 56,000 profiles from Linkedin. We also illustrate the\nusefulness of IPOD by addressing two challenging upstream tasks, including: (i)\nproposing Title2vec, a contextual job title vector representation using a\nbidirectional Language Model (biLM) approach; and (ii) addressing the important\noccupational Named Entity Recognition problem using Conditional Random Fields\n(CRF) and bidirectional Long Short-Term Memory with CRF (LSTM-CRF). Both CRF\nand LSTM-CRF outperform human and baselines in both exact-match accuracy and F1\nscores. The dataset and pre-trained embeddings are available at\nhttps://www.github.com/junhua/ipod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10605v1",
            "date": "2019-10-23 15:22:22+00:00",
            "title": "Speaker Adaptive Training using Model Agnostic Meta-Learning",
            "authors": [
                "Ond\u0159ej Klejch",
                "Joachim Fainberg",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "Speaker adaptive training (SAT) of neural network acoustic models learns\nmodels in a way that makes them more suitable for adaptation to test\nconditions. Conventionally, model-based speaker adaptive training is performed\nby having a set of speaker dependent parameters that are jointly optimised with\nspeaker independent parameters in order to remove speaker variation. However,\nthis does not scale well if all neural network weights are to be adapted to the\nspeaker. In this paper we formulate speaker adaptive training as a\nmeta-learning task, in which an adaptation process using gradient descent is\nencoded directly into the training of the model. We compare our approach with\ntest-only adaptation of a standard baseline model and a SAT-LHUC model with a\nlearned speaker adaptation schedule and demonstrate that the meta-learning\napproach achieves comparable results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10671v1",
            "date": "2019-10-23 17:12:48+00:00",
            "title": "A practical two-stage training strategy for multi-stream end-to-end speech recognition",
            "authors": [
                "Ruizhi Li",
                "Gregory Sell",
                "Xiaofei Wang",
                "Shinji Watanabe",
                "Hynek Hermansky"
            ],
            "abstract": "The multi-stream paradigm of audio processing, in which several sources are\nsimultaneously considered, has been an active research area for information\nfusion. Our previous study offered a promising direction within end-to-end\nautomatic speech recognition, where parallel encoders aim to capture diverse\ninformation followed by a stream-level fusion based on attention mechanisms to\ncombine the different views. However, with an increasing number of streams\nresulting in an increasing number of encoders, the previous approach could\nrequire substantial memory and massive amounts of parallel data for joint\ntraining. In this work, we propose a practical two-stage training scheme.\nStage-1 is to train a Universal Feature Extractor (UFE), where encoder outputs\nare produced from a single-stream model trained with all data. Stage-2\nformulates a multi-stream scheme intending to solely train the attention fusion\nmodule using the UFE features and pretrained components from Stage-1.\nExperiments have been conducted on two datasets, DIRHA and AMI, as a\nmulti-stream scenario. Compared with our previous method, this strategy\nachieves relative word error rate reductions of 8.2--32.4%, while consistently\noutperforming several conventional combination methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10683v3",
            "date": "2019-10-23 17:37:36+00:00",
            "title": "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer",
            "authors": [
                "Colin Raffel",
                "Noam Shazeer",
                "Adam Roberts",
                "Katherine Lee",
                "Sharan Narang",
                "Michael Matena",
                "Yanqi Zhou",
                "Wei Li",
                "Peter J. Liu"
            ],
            "abstract": "Transfer learning, where a model is first pre-trained on a data-rich task\nbefore being fine-tuned on a downstream task, has emerged as a powerful\ntechnique in natural language processing (NLP). The effectiveness of transfer\nlearning has given rise to a diversity of approaches, methodology, and\npractice. In this paper, we explore the landscape of transfer learning\ntechniques for NLP by introducing a unified framework that converts all\ntext-based language problems into a text-to-text format. Our systematic study\ncompares pre-training objectives, architectures, unlabeled data sets, transfer\napproaches, and other factors on dozens of language understanding tasks. By\ncombining the insights from our exploration with scale and our new ``Colossal\nClean Crawled Corpus'', we achieve state-of-the-art results on many benchmarks\ncovering summarization, question answering, text classification, and more. To\nfacilitate future work on transfer learning for NLP, we release our data set,\npre-trained models, and code.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.10697v1",
            "date": "2019-10-23 17:57:11+00:00",
            "title": "Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model",
            "authors": [
                "Oleksii Hrinchuk",
                "Mariya Popova",
                "Boris Ginsburg"
            ],
            "abstract": "In this work, we introduce a simple yet efficient post-processing model for\nautomatic speech recognition (ASR). Our model has Transformer-based\nencoder-decoder architecture which \"translates\" ASR model output into\ngrammatically and semantically correct text. We investigate different\nstrategies for regularizing and optimizing the model and show that extensive\ndata augmentation and the initialization with pre-trained weights are required\nto achieve good performance. On the LibriSpeech benchmark, our method\ndemonstrates significant improvement in word error rate over the baseline\nacoustic model with greedy decoding, especially on much noisier dev-other and\ntest-other portions of the evaluation dataset. Our model also outperforms\nbaseline with 6-gram language model re-scoring and approaches the performance\nof re-scoring with Transformer-XL neural language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10781v1",
            "date": "2019-10-23 19:51:50+00:00",
            "title": "Hierarchical Transformers for Long Document Classification",
            "authors": [
                "Raghavendra Pappagari",
                "Piotr \u017belasko",
                "Jes\u00fas Villalba",
                "Yishay Carmiel",
                "Najim Dehak"
            ],
            "abstract": "BERT, which stands for Bidirectional Encoder Representations from\nTransformers, is a recently introduced language representation model based upon\nthe transfer learning paradigm. We extend its fine-tuning procedure to address\none of its major limitations - applicability to inputs longer than a few\nhundred words, such as transcripts of human call conversations. Our method is\nconceptually simple. We segment the input into smaller chunks and feed each of\nthem into the base model. Then, we propagate each output through a single\nrecurrent layer, or another transformer, followed by a softmax activation. We\nobtain the final classification decision after the last segment has been\nconsumed. We show that both BERT extensions are quick to fine-tune and converge\nafter as little as 1 epoch of training on a small, domain-specific data set. We\nsuccessfully apply them in three different tasks involving customer call\nsatisfaction prediction and topic classification, and obtain a significant\nimprovement over the baseline models in two of them.",
            "journal ref": "Automatic Speech Recognition and Understanding Workshop, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.10843v1",
            "date": "2019-10-23 23:55:23+00:00",
            "title": "Relation Module for Non-answerable Prediction on Question Answering",
            "authors": [
                "Kevin Huang",
                "Yun Tang",
                "Jing Huang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Machine reading comprehension(MRC) has attracted significant amounts of\nresearch attention recently, due to an increase of challenging reading\ncomprehension datasets. In this paper, we aim to improve a MRC model's ability\nto determine whether a question has an answer in a given context (e.g. the\nrecently proposed SQuAD 2.0 task). Our solution is a relation module that is\nadaptable to any MRC model. The relation module consists of both semantic\nextraction and relational information. We first extract high level semantics as\nobjects from both question and context with multi-head self-attentive pooling.\nThese semantic objects are then passed to a relation network, which generates\nrelationship scores for each object pair in a sentence. These scores are used\nto determine whether a question is non-answerable. We test the relation module\non the SQuAD 2.0 dataset using both BiDAF and BERT models as baseline readers.\nWe obtain 1.8% gain of F1 on top of the BiDAF reader, and 1.0% on top of the\nBERT base model. These results show the effectiveness of our relation module on\nMRC",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11119v1",
            "date": "2019-10-21 18:06:26+00:00",
            "title": "Designovel's system description for Fashion-IQ challenge 2019",
            "authors": [
                "Jianri Li",
                "Jae-whan Lee",
                "Woo-sang Song",
                "Ki-young Shin",
                "Byung-hyun Go"
            ],
            "abstract": "This paper describes Designovel's systems which are submitted to the Fashion\nIQ Challenge 2019. Goal of the challenge is building an image retrieval system\nwhere input query is a candidate image plus two text phrases describe user's\nfeedback about visual differences between the candidate image and the search\ntarget. We built the systems by combining methods from recent work on deep\nmetric learning, multi-modal retrieval and natual language processing. First,\nwe encode both candidate and target images with CNNs into high-level\nrepresentations, and encode text descriptions to a single text vector using\nTransformer-based encoder. Then we compose candidate image vector and text\nrepresentation into a single vector which is exptected to be biased toward\ntarget image vector. Finally, we compute cosine similarities between composed\nvector and encoded vectors of whole dataset, and rank them in desceding order\nto get ranked list. We experimented with Fashion IQ 2019 dataset in various\nsettings of hyperparameters, achieved 39.12% average recall by a single model\nand 43.67% average recall by an ensemble of 16 models on test dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.11124v2",
            "date": "2019-10-21 02:33:18+00:00",
            "title": "Enforcing Reasoning in Visual Commonsense Reasoning",
            "authors": [
                "Hammad A. Ayyubi",
                "Md. Mehrab Tanjim",
                "David J. Kriegman"
            ],
            "abstract": "The task of Visual Commonsense Reasoning is extremely challenging in the\nsense that the model has to not only be able to answer a question given an\nimage, but also be able to learn to reason. The baselines introduced in this\ntask are quite limiting because two networks are trained for predicting answers\nand rationales separately. Question and image is used as input to train answer\nprediction network while question, image and correct answer are used as input\nin the rationale prediction network. As rationale is conditioned on the correct\nanswer, it is based on the assumption that we can solve Visual Question\nAnswering task without any error - which is over ambitious. Moreover, such an\napproach makes both answer and rationale prediction two completely independent\nVQA tasks rendering cognition task meaningless. In this paper, we seek to\naddress these issues by proposing an end-to-end trainable model which considers\nboth answers and their reasons jointly. Specifically, we first predict the\nanswer for the question and then use the chosen answer to predict the\nrationale. However, a trivial design of such a model becomes non-differentiable\nwhich makes it difficult to train. We solve this issue by proposing four\napproaches - softmax, gumbel-softmax, reinforcement learning based sampling and\ndirect cross entropy against all pairs of answers and rationales. We\ndemonstrate through experiments that our model performs competitively against\ncurrent state-of-the-art. We conclude with an analysis of presented approaches\nand discuss avenues for further work.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1910.11241v2",
            "date": "2019-10-23 07:37:14+00:00",
            "title": "Healthcare NER Models Using Language Model Pretraining",
            "authors": [
                "Amogh Kamat Tarcar",
                "Aashis Tiwari",
                "Vineet Naique Dhaimodker",
                "Penjo Rebelo",
                "Rahul Desai",
                "Dattaraj Rao"
            ],
            "abstract": "In this paper, we present our approach to extracting structured information\nfrom unstructured Electronic Health Records (EHR) [2] which can be used to, for\nexample, study adverse drug reactions in patients due to chemicals in their\nproducts. Our solution uses a combination of Natural Language Processing (NLP)\ntechniques and a web-based annotation tool to optimize the performance of a\ncustom Named Entity Recognition (NER) [1] model trained on a limited amount of\nEHR training data. This work was presented at the first Health Search and Data\nMining Workshop (HSDM 2020) [26]. We showcase a combination of tools and\ntechniques leveraging the recent advancements in NLP aimed at targeting domain\nshifts by applying transfer learning and language model pre-training techniques\n[3]. We present a comparison of our technique to the current popular approaches\nand show the effective increase in performance of the NER model and the\nreduction in time to annotate data.A key observation of the results presented\nis that the F1 score of model (0.734) trained with our approach with just 50%\nof available training data outperforms the F1 score of the blank spaCy model\nwithout language model component (0.704) trained with 100% of the available\ntraining data. We also demonstrate an annotation tool to minimize domain expert\ntime and the manual effort required to generate such a training dataset.\nFurther, we plan to release the annotated dataset as well as the pre-trained\nmodel to the community to further research in medical health records.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11242v1",
            "date": "2019-10-23 15:00:39+00:00",
            "title": "A context sensitive real-time Spell Checker with language adaptability",
            "authors": [
                "Prabhakar Gupta"
            ],
            "abstract": "We present a novel language adaptable spell checking system which detects\nspelling errors and suggests context sensitive corrections in real-time. We\nshow that our system can be extended to new languages with minimal\nlanguage-specific processing. Available literature majorly discusses spell\ncheckers for English but there are no publicly available systems which can be\nextended to work for other languages out of the box. Most of the systems do not\nwork in real-time. We explain the process of generating a language's word\ndictionary and n-gram probability dictionaries using Wikipedia-articles data\nand manually curated video subtitles. We present the results of generating a\nlist of suggestions for a misspelled word. We also propose three approaches to\ncreate noisy channel datasets of real-world typographic errors. We compare our\nsystem with industry-accepted spell checker tools for 11 languages. Finally, we\nshow the performance of our system on synthetic datasets for 24 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11263v1",
            "date": "2019-10-24 16:16:45+00:00",
            "title": "Conversational Emotion Analysis via Attention Mechanisms",
            "authors": [
                "Zheng Lian",
                "Jianhua Tao",
                "Bin Liu",
                "Jian Huang"
            ],
            "abstract": "Different from the emotion recognition in individual utterances, we propose a\nmultimodal learning framework using relation and dependencies among the\nutterances for conversational emotion analysis. The attention mechanism is\napplied to the fusion of the acoustic and lexical features. Then these fusion\nrepresentations are fed into the self-attention based bi-directional gated\nrecurrent unit (GRU) layer to capture long-term contextual information. To\nimitate real interaction patterns of different speakers, speaker embeddings are\nalso utilized as additional inputs to distinguish the speaker identities during\nconversational dialogs. To verify the effectiveness of the proposed method, we\nconduct experiments on the IEMOCAP database. Experimental results demonstrate\nthat our method shows absolute 2.42% performance improvement over the\nstate-of-the-art strategies.",
            "journal ref": "Proc. Interspeech 2019, 1936-1940",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11292v3",
            "date": "2019-10-24 17:10:34+00:00",
            "title": "Predicting In-game Actions from Interviews of NBA Players",
            "authors": [
                "Nadav Oved",
                "Amir Feder",
                "Roi Reichart"
            ],
            "abstract": "Sports competitions are widely researched in computer and social science,\nwith the goal of understanding how players act under uncertainty. While there\nis an abundance of computational work on player metrics prediction based on\npast performance, very few attempts to incorporate out-of-game signals have\nbeen made. Specifically, it was previously unclear whether linguistic signals\ngathered from players' interviews can add information which does not appear in\nperformance metrics. To bridge that gap, we define text classification tasks of\npredicting deviations from mean in NBA players' in-game actions, which are\nassociated with strategic choices, player behavior and risk, using their choice\nof language prior to the game. We collected a dataset of transcripts from key\nNBA players' pre-game interviews and their in-game performance metrics,\ntotalling in 5,226 interview-metric pairs. We design neural models for players'\naction prediction based on increasingly more complex aspects of the language\nsignals in their open-ended interviews. Our models can make their predictions\nbased on the textual signal alone, or on a combination with signals from\npast-performance metrics. Our text-based models outperform strong baselines\ntrained on performance metrics only, demonstrating the importance of language\nusage for action prediction. Moreover, the models that employ both textual\ninput and past-performance metrics produced the best results. Finally, as\nneural networks are notoriously difficult to interpret, we propose a method for\ngaining further insight into what our models have learned. Particularly, we\npresent an LDA-based analysis, where we interpret model predictions in terms of\ncorrelated topics. We find that our best performing textual model is most\nassociated with topics that are intuitively related to each prediction task and\nthat better models yield higher correlation with more informative topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11301v3",
            "date": "2019-10-24 17:32:38+00:00",
            "title": "Cross-Lingual Vision-Language Navigation",
            "authors": [
                "An Yan",
                "Xin Eric Wang",
                "Jiangtao Feng",
                "Lei Li",
                "William Yang Wang"
            ],
            "abstract": "Commanding a robot to navigate with natural language instructions is a\nlong-term goal for grounded language understanding and robotics. But the\ndominant language is English, according to previous studies on vision-language\nnavigation (VLN). To go beyond English and serve people speaking different\nlanguages, we collect a bilingual Room-to-Room (BL-R2R) dataset, extending the\noriginal benchmark with new Chinese instructions. Based on this newly\nintroduced dataset, we study how an agent can be trained on existing English\ninstructions but navigate effectively with another language under a zero-shot\nlearning scenario. Without any training data of the target language, our model\nshows competitive results even compared to a model with full access to the\ntarget language training data. Moreover, we investigate the transferring\nability of our model when given a certain amount of target language training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11368v1",
            "date": "2019-10-24 18:20:48+00:00",
            "title": "Extending Event Detection to New Types with Learning from Keywords",
            "authors": [
                "Viet Dac Lai",
                "Thien Huu Nguyen"
            ],
            "abstract": "Traditional event detection classifies a word or a phrase in a given sentence\nfor a set of predefined event types. The limitation of such predefined set is\nthat it prevents the adaptation of the event detection models to new event\ntypes. We study a novel formulation of event detection that describes types via\nseveral keywords to match the contexts in documents. This facilitates the\noperation of the models to new types. We introduce a novel feature-based\nattention mechanism for convolutional neural networks for event detection in\nthe new formulation. Our extensive experiments demonstrate the benefits of the\nnew formulation for new type extension for event detection as well as the\nproposed attention mechanism for this problem.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.11386v4",
            "date": "2019-10-24 19:13:21+00:00",
            "title": "Detecting gender differences in perception of emotion in crowdsourced data",
            "authors": [
                "Shahan Ali Memon",
                "Hira Dhamyal",
                "Oren Wright",
                "Daniel Justice",
                "Vijaykumar Palat",
                "William Boler",
                "Bhiksha Raj",
                "Rita Singh"
            ],
            "abstract": "Do men and women perceive emotions differently? Popular convictions place\nwomen as more emotionally perceptive than men. Empirical findings, however,\nremain inconclusive. Most prior studies focus on visual modalities. In\naddition, almost all of the studies are limited to experiments within\ncontrolled environments. Generalizability and scalability of these studies has\nnot been sufficiently established. In this paper, we study the differences in\nperception of emotion between genders from speech data in the wild, annotated\nthrough crowdsourcing. While we limit ourselves to a single modality (i.e.\nspeech), our framework is applicable to studies of emotion perception from all\nsuch loosely annotated data in general. Our paper addresses multiple serious\nchallenges related to making statistically viable conclusions from crowdsourced\ndata. Overall, the contributions of this paper are two fold: a reliable novel\nframework for perceptual studies from crowdsourced data; and the demonstration\nof statistically significant differences in speech-based emotion perception\nbetween genders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11455v1",
            "date": "2019-10-24 23:18:07+00:00",
            "title": "Recognizing long-form speech using streaming end-to-end models",
            "authors": [
                "Arun Narayanan",
                "Rohit Prabhavalkar",
                "Chung-Cheng Chiu",
                "David Rybach",
                "Tara N. Sainath",
                "Trevor Strohman"
            ],
            "abstract": "All-neural end-to-end (E2E) automatic speech recognition (ASR) systems that\nuse a single neural network to transduce audio to word sequences have been\nshown to achieve state-of-the-art results on several tasks. In this work, we\nexamine the ability of E2E models to generalize to unseen domains, where we\nfind that models trained on short utterances fail to generalize to long-form\nspeech. We propose two complementary solutions to address this: training on\ndiverse acoustic data, and LSTM state manipulation to simulate long-form audio\nwhen training using short utterances. On a synthesized long-form test set,\nadding data diversity improves word error rate (WER) by 90% relative, while\nsimulating long-form training improves it by 67% relative, though the\ncombination doesn't improve over data diversity alone. On a real long-form\ncall-center test set, adding data diversity improves WER by 40% relative.\nSimulating long-form training on top of data diversity improves performance by\nan additional 27% relative.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.11471v1",
            "date": "2019-10-25 00:46:07+00:00",
            "title": "Machine Translation from Natural Language to Code using Long-Short Term Memory",
            "authors": [
                "K. M. Tahsin Hassan Rahit",
                "Rashidul Hasan Nabil",
                "Md Hasibul Huq"
            ],
            "abstract": "Making computer programming language more understandable and easy for the\nhuman is a longstanding problem. From assembly language to present day's\nobject-oriented programming, concepts came to make programming easier so that a\nprogrammer can focus on the logic and the architecture rather than the code and\nlanguage itself. To go a step further in this journey of removing\nhuman-computer language barrier, this paper proposes machine learning approach\nusing Recurrent Neural Network (RNN) and Long-Short Term Memory (LSTM) to\nconvert human language into programming language code. The programmer will\nwrite expressions for codes in layman's language, and the machine learning\nmodel will translate it to the targeted programming language. The proposed\napproach yields result with 74.40% accuracy. This can be further improved by\nincorporating additional techniques, which are also discussed in this paper.",
            "journal ref": "Proceedings of the Future Technologies Conference (FTC) 2019.\n  Advances in Intelligent Systems and Computing, vol 1069. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11494v3",
            "date": "2019-10-25 02:21:33+00:00",
            "title": "KRED: Knowledge-Aware Document Representation for News Recommendations",
            "authors": [
                "Danyang Liu",
                "Jianxun Lian",
                "Shiyin Wang",
                "Ying Qiao",
                "Jiun-Hung Chen",
                "Guangzhong Sun",
                "Xing Xie"
            ],
            "abstract": "News articles usually contain knowledge entities such as celebrities or\norganizations. Important entities in articles carry key messages and help to\nunderstand the content in a more direct way. An industrial news recommender\nsystem contains various key applications, such as personalized recommendation,\nitem-to-item recommendation, news category classification, news popularity\nprediction and local news detection. We find that incorporating knowledge\nentities for better document understanding benefits these applications\nconsistently. However, existing document understanding models either represent\nnews articles without considering knowledge entities (e.g., BERT) or rely on a\nspecific type of text encoding model (e.g., DKN) so that the generalization\nability and efficiency is compromised. In this paper, we propose KRED, which is\na fast and effective model to enhance arbitrary document representation with a\nknowledge graph. KRED first enriches entities' embeddings by attentively\naggregating information from their neighborhood in the knowledge graph. Then a\ncontext embedding layer is applied to annotate the dynamic context of different\nentities such as frequency, category and position. Finally, an information\ndistillation layer aggregates the entity embeddings under the guidance of the\noriginal document representation and transforms the document vector into a new\none. We advocate to optimize the model with a multi-task framework, so that\ndifferent news recommendation applications can be united and useful information\ncan be shared across different tasks. Experiments on a real-world Microsoft\nNews dataset demonstrate that KRED greatly benefits a variety of news\nrecommendation applications.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.11496v1",
            "date": "2019-10-25 02:25:34+00:00",
            "title": "L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition",
            "authors": [
                "Yuanfeng Song",
                "Di Jiang",
                "Xuefang Zhao",
                "Qian Xu",
                "Raymond Chi-Wing Wong",
                "Lixin Fan",
                "Qiang Yang"
            ],
            "abstract": "Modern Automatic Speech Recognition (ASR) systems primarily rely on scores\nfrom an Acoustic Model (AM) and a Language Model (LM) to rescore the N-best\nlists. With the abundance of recent natural language processing advances, the\ninformation utilized by current ASR for evaluating the linguistic and semantic\nlegitimacy of the N-best hypotheses is rather limited. In this paper, we\npropose a novel Learning-to-Rescore (L2RS) mechanism, which is specialized for\nutilizing a wide range of textual information from the state-of-the-art NLP\nmodels and automatically deciding their weights to rescore the N-best lists for\nASR systems. Specifically, we incorporate features including BERT sentence\nembedding, topic vector, and perplexity scores produced by n-gram LM, topic\nmodeling LM, BERT LM and RNNLM to train a rescoring model. We conduct extensive\nexperiments based on a public dataset, and experimental results show that L2RS\noutperforms not only traditional rescoring methods but also its deep neural\nnetwork counterparts by a substantial improvement of 20.67% in terms of\nNDCG@10. L2RS paves the way for developing more effective rescoring models for\nASR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11555v2",
            "date": "2019-10-25 07:32:52+00:00",
            "title": "Fast Structured Decoding for Sequence Models",
            "authors": [
                "Zhiqing Sun",
                "Zhuohan Li",
                "Haoqing Wang",
                "Zi Lin",
                "Di He",
                "Zhi-Hong Deng"
            ],
            "abstract": "Autoregressive sequence models achieve state-of-the-art performance in\ndomains like machine translation. However, due to the autoregressive\nfactorization nature, these models suffer from heavy latency during inference.\nRecently, non-autoregressive sequence models were proposed to reduce the\ninference time. However, these models assume that the decoding process of each\ntoken is conditionally independent of others. Such a generation process\nsometimes makes the output sentence inconsistent, and thus the learned\nnon-autoregressive models could only achieve inferior accuracy compared to\ntheir autoregressive counterparts. To improve then decoding consistency and\nreduce the inference cost at the same time, we propose to incorporate a\nstructured inference module into the non-autoregressive models. Specifically,\nwe design an efficient approximation for Conditional Random Fields (CRF) for\nnon-autoregressive sequence models, and further propose a dynamic transition\ntechnique to model positional contexts in the CRF. Experiments in machine\ntranslation show that while increasing little latency (8~14ms), our model could\nachieve significantly better translation performance than previous\nnon-autoregressive models on different translation datasets. In particular, for\nthe WMT14 En-De dataset, our model obtains a BLEU score of 26.80, which largely\noutperforms the previous non-autoregressive baselines and is only 0.61 lower in\nBLEU than purely autoregressive models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.11559v4",
            "date": "2019-10-25 07:46:39+00:00",
            "title": "SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering",
            "authors": [
                "Yung-Sung Chuang",
                "Chi-Liang Liu",
                "Hung-Yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "While various end-to-end models for spoken language understanding tasks have\nbeen explored recently, this paper is probably the first known attempt to\nchallenge the very difficult task of end-to-end spoken question answering\n(SQA). Learning from the very successful BERT model for various text processing\ntasks, here we proposed an audio-and-text jointly learned SpeechBERT model.\nThis model outperformed the conventional approach of cascading ASR with the\nfollowing text question answering (TQA) model on datasets including ASR errors\nin answer spans, because the end-to-end model was shown to be able to extract\ninformation out of audio data before ASR produced errors. When ensembling the\nproposed end-to-end model with the cascade architecture, even better\nperformance was achieved. In addition to the potential of end-to-end SQA, the\nSpeechBERT can also be considered for many other spoken language understanding\ntasks just as BERT for many text processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11691v2",
            "date": "2019-10-24 01:57:34+00:00",
            "title": "Improving Diarization Robustness using Diversification, Randomization and the DOVER Algorithm",
            "authors": [
                "Andreas Stolcke"
            ],
            "abstract": "Speaker diarization based on bottom-up clustering of speech segments by\nacoustic similarity is often highly sensitive to the choice of hyperparameters,\nsuch as the initial number of clusters and feature weighting. Optimizing these\nhyperparameters is difficult and often not robust across different data sets.\nWe recently proposed the DOVER algorithm for combining multiple diarization\nhypotheses by voting. Here we propose to mitigate the robustness problem in\ndiarization by using DOVER to average across different parameter choices. We\nalso investigate the combination of diverse outputs obtained by following\ndifferent merge choices pseudo-randomly in the course of clustering, thereby\nmitigating the greediness of best-first clustering. We show on two conference\nmeeting data sets drawn from NIST evaluations that the proposed methods indeed\nyield more robust, and in several cases overall improved, results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11768v1",
            "date": "2019-10-25 14:38:18+00:00",
            "title": "Exploring Multilingual Syntactic Sentence Representations",
            "authors": [
                "Chen Liu",
                "Anderson de Andrade",
                "Muhammad Osama"
            ],
            "abstract": "We study methods for learning sentence embeddings with syntactic structure.\nWe focus on methods of learning syntactic sentence-embeddings by using a\nmultilingual parallel-corpus augmented by Universal Parts-of-Speech tags. We\nevaluate the quality of the learned embeddings by examining sentence-level\nnearest neighbours and functional dissimilarity in the embedding space. We also\nevaluate the ability of the method to learn syntactic sentence-embeddings for\nlow-resource languages and demonstrate strong evidence for transfer learning.\nOur results show that syntactic sentence-embeddings can be learned while using\nless training data, fewer model parameters, and resulting in better evaluation\nmetrics than state-of-the-art language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11856v3",
            "date": "2019-10-25 17:30:20+00:00",
            "title": "On the Cross-lingual Transferability of Monolingual Representations",
            "authors": [
                "Mikel Artetxe",
                "Sebastian Ruder",
                "Dani Yogatama"
            ],
            "abstract": "State-of-the-art unsupervised multilingual models (e.g., multilingual BERT)\nhave been shown to generalize in a zero-shot cross-lingual setting. This\ngeneralization ability has been attributed to the use of a shared subword\nvocabulary and joint training across multiple languages giving rise to deep\nmultilingual abstractions. We evaluate this hypothesis by designing an\nalternative approach that transfers a monolingual model to new languages at the\nlexical level. More concretely, we first train a transformer-based masked\nlanguage model on one language, and transfer it to a new language by learning a\nnew embedding matrix with the same masked language modeling objective, freezing\nparameters of all other layers. This approach does not rely on a shared\nvocabulary or joint training. However, we show that it is competitive with\nmultilingual BERT on standard cross-lingual classification benchmarks and on a\nnew Cross-lingual Question Answering Dataset (XQuAD). Our results contradict\ncommon beliefs of the basis of the generalization ability of multilingual\nmodels and suggest that deep monolingual models learn some abstractions that\ngeneralize across languages. We also release XQuAD as a more comprehensive\ncross-lingual benchmark, which comprises 240 paragraphs and 1190\nquestion-answer pairs from SQuAD v1.1 translated into ten languages by\nprofessional translators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11871v1",
            "date": "2019-10-25 05:28:17+00:00",
            "title": "Towards Online End-to-end Transformer Automatic Speech Recognition",
            "authors": [
                "Emiru Tsunoo",
                "Yosuke Kashiwagi",
                "Toshiyuki Kumakura",
                "Shinji Watanabe"
            ],
            "abstract": "The Transformer self-attention network has recently shown promising\nperformance as an alternative to recurrent neural networks in end-to-end (E2E)\nautomatic speech recognition (ASR) systems. However, Transformer has a drawback\nin that the entire input sequence is required to compute self-attention. We\nhave proposed a block processing method for the Transformer encoder by\nintroducing a context-aware inheritance mechanism. An additional context\nembedding vector handed over from the previously processed block helps to\nencode not only local acoustic information but also global linguistic, channel,\nand speaker attributes. In this paper, we extend it towards an entire online\nE2E ASR system by introducing an online decoding process inspired by monotonic\nchunkwise attention (MoChA) into the Transformer decoder. Our novel MoChA\ntraining and inference algorithms exploit the unique properties of Transformer,\nwhose attentions are not always monotonic or peaky, and have multiple heads and\nresidual connections of the decoder layers. Evaluations of the Wall Street\nJournal (WSJ) and AISHELL-1 show that our proposed online Transformer decoder\noutperforms conventional chunkwise approaches.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.11922v1",
            "date": "2019-10-25 20:15:38+00:00",
            "title": "Current Limitations in Cyberbullying Detection: on Evaluation Criteria, Reproducibility, and Data Scarcity",
            "authors": [
                "Chris Emmery",
                "Ben Verhoeven",
                "Guy De Pauw",
                "Gilles Jacobs",
                "Cynthia Van Hee",
                "Els Lefever",
                "Bart Desmet",
                "V\u00e9ronique Hoste",
                "Walter Daelemans"
            ],
            "abstract": "The detection of online cyberbullying has seen an increase in societal\nimportance, popularity in research, and available open data. Nevertheless,\nwhile computational power and affordability of resources continue to increase,\nthe access restrictions on high-quality data limit the applicability of\nstate-of-the-art techniques. Consequently, much of the recent research uses\nsmall, heterogeneous datasets, without a thorough evaluation of applicability.\nIn this paper, we further illustrate these issues, as we (i) evaluate many\npublicly available resources for this task and demonstrate difficulties with\ndata collection. These predominantly yield small datasets that fail to capture\nthe required complex social dynamics and impede direct comparison of progress.\nWe (ii) conduct an extensive set of experiments that indicate a general lack of\ncross-domain generalization of classifiers trained on these sources, and openly\nprovide this framework to replicate and extend our evaluation criteria.\nFinally, we (iii) present an effective crowdsourcing method: simulating\nreal-life bullying scenarios in a lab setting generates plausible data that can\nbe effectively used to enrich real data. This largely circumvents the\nrestrictions on data that can be collected, and increases classifier\nperformance. We believe these contributions can aid in improving the empirical\npractices of future research in the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.11949v2",
            "date": "2019-10-25 21:47:52+00:00",
            "title": "Automatic Reminiscence Therapy for Dementia",
            "authors": [
                "Mariona Caros",
                "Maite Garolera",
                "Petia Radeva",
                "Xavier Giro-i-Nieto"
            ],
            "abstract": "With people living longer than ever, the number of cases with dementia such\nas Alzheimer's disease increases steadily. It affects more than 46 million\npeople worldwide, and it is estimated that in 2050 more than 100 million will\nbe affected. While there are not effective treatments for these terminal\ndiseases, therapies such as reminiscence, that stimulate memories from the past\nare recommended. Currently, reminiscence therapy takes place in care homes and\nis guided by a therapist or a carer. In this work, we present an AI-based\nsolution to automatize the reminiscence therapy, which consists in a dialogue\nsystem that uses photos as input to generate questions. We run a usability case\nstudy with patients diagnosed of mild cognitive impairment that shows they\nfound the system very entertaining and challenging. Overall, this paper\npresents how reminiscence therapy can be automatized by using machine learning,\nand deployed to smartphones and laptops, making the therapy more accessible to\nevery person affected by dementia.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1910.12073v1",
            "date": "2019-10-26 14:29:37+00:00",
            "title": "Disinformation Detection: A review of linguistic feature selection and classification models in news veracity assessments",
            "authors": [
                "Jillian Tompkins"
            ],
            "abstract": "Over the past couple of years, the topic of \"fake news\" and its influence\nover people's opinions has become a growing cause for concern. Although the\nspread of disinformation on the Internet is not a new phenomenon, the\nwidespread use of social media has exacerbated its effects, providing more\nchannels for dissemination and the potential to \"go viral.\" Nowhere was this\nmore evident than during the 2016 United States Presidential Election. Although\nthe current of disinformation spread via trolls, bots, and hyperpartisan media\noutlets likely reinforced existing biases rather than sway undecided voters,\nthe effects of this deluge of disinformation are by no means trivial. The\nconsequences range in severity from an overall distrust in news media, to an\nill-informed citizenry, and in extreme cases, provocation of violent action. It\nis clear that human ability to discern lies from truth is flawed at best. As\nsuch, greater attention has been given towards applying machine learning\napproaches to detect deliberately deceptive news articles. This paper looks at\nthe work that has already been done in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12094v1",
            "date": "2019-10-26 16:00:44+00:00",
            "title": "Meta Learning for End-to-End Low-Resource Speech Recognition",
            "authors": [
                "Jui-Yang Hsu",
                "Yuan-Jui Chen",
                "Hung-yi Lee"
            ],
            "abstract": "In this paper, we proposed to apply meta learning approach for low-resource\nautomatic speech recognition (ASR). We formulated ASR for different languages\nas different tasks, and meta-learned the initialization parameters from many\npretraining languages to achieve fast adaptation on unseen target language, via\nrecently proposed model-agnostic meta learning algorithm (MAML). We evaluated\nthe proposed approach using six languages as pretraining tasks and four\nlanguages as target tasks. Preliminary results showed that the proposed method,\nMetaASR, significantly outperforms the state-of-the-art multitask pretraining\napproach on all target languages with different combinations of pretraining\nlanguages. In addition, since MAML's model-agnostic property, this paper also\nopens new research direction of applying meta learning to more speech-related\napplications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.12180v1",
            "date": "2019-10-27 04:53:35+00:00",
            "title": "SoulMate: Short-text author linking through Multi-aspect temporal-textual embedding",
            "authors": [
                "Saeed Najafipour",
                "Saeid Hosseini",
                "Wen Hua",
                "Mohammad Reza Kangavari",
                "Xiaofang Zhou"
            ],
            "abstract": "Linking authors of short-text contents has important usages in many\napplications, including Named Entity Recognition (NER) and human community\ndetection. However, certain challenges lie ahead. Firstly, the input short-text\ncontents are noisy, ambiguous, and do not follow the grammatical rules.\nSecondly, traditional text mining methods fail to effectively extract concepts\nthrough words and phrases. Thirdly, the textual contents are temporally skewed,\nwhich can affect the semantic understanding by multiple time facets. Finally,\nusing the complementary knowledge-bases makes the results biased to the content\nof the external database and deviates the understanding and interpretation away\nfrom the real nature of the given short text corpus. To overcome these\nchallenges, we devise a neural network-based temporal-textual framework that\ngenerates the tightly connected author subgraphs from microblog short-text\ncontents. Our approach, on the one hand, computes the relevance score (edge\nweight) between the authors through considering a portmanteau of contents and\nconcepts, and on the other hand, employs a stack-wise graph cutting algorithm\nto extract the communities of the related authors. Experimental results show\nthat compared to other knowledge-centered competitors, our multi-aspect vector\nspace model can achieve a higher performance in linking short-text authors.\nAdditionally, given the author linking task, the more comprehensive the dataset\nis, the higher the significance of the extracted concepts will be.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12196v4",
            "date": "2019-10-27 06:54:27+00:00",
            "title": "Word-level Textual Adversarial Attacking as Combinatorial Optimization",
            "authors": [
                "Yuan Zang",
                "Fanchao Qi",
                "Chenghao Yang",
                "Zhiyuan Liu",
                "Meng Zhang",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Adversarial attacks are carried out to reveal the vulnerability of deep\nneural networks. Textual adversarial attacking is challenging because text is\ndiscrete and a small perturbation can bring significant change to the original\ninput. Word-level attacking, which can be regarded as a combinatorial\noptimization problem, is a well-studied class of textual attack methods.\nHowever, existing word-level attack models are far from perfect, largely\nbecause unsuitable search space reduction methods and inefficient optimization\nalgorithms are employed. In this paper, we propose a novel attack model, which\nincorporates the sememe-based word substitution method and particle swarm\noptimization-based search algorithm to solve the two problems separately. We\nconduct exhaustive experiments to evaluate our attack model by attacking BiLSTM\nand BERT on three benchmark datasets. Experimental results demonstrate that our\nmodel consistently achieves much higher attack success rates and crafts more\nhigh-quality adversarial examples as compared to baseline methods. Also,\nfurther experiments show our model has higher transferability and can bring\nmore robustness enhancement to victim models by adversarial training. All the\ncode and data of this paper can be obtained on\nhttps://github.com/thunlp/SememePSO-Attack.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12203v1",
            "date": "2019-10-27 07:44:33+00:00",
            "title": "Do Sentence Interactions Matter? Leveraging Sentence Level Representations for Fake News Classification",
            "authors": [
                "Vaibhav Vaibhav",
                "Raghuram Mandyam Annasamy",
                "Eduard Hovy"
            ],
            "abstract": "The rising growth of fake news and misleading information through online\nmedia outlets demands an automatic method for detecting such news articles. Of\nthe few limited works which differentiate between trusted vs other types of\nnews article (satire, propaganda, hoax), none of them model sentence\ninteractions within a document. We observe an interesting pattern in the way\nsentences interact with each other across different kind of news articles. To\ncapture this kind of information for long news articles, we propose a graph\nneural network-based model which does away with the need of feature engineering\nfor fine grained fake news classification. Through experiments, we show that\nour proposed method beats strong neural baselines and achieves state-of-the-art\naccuracy on existing datasets. Moreover, we establish the generalizability of\nour model by evaluating its performance in out-of-domain scenarios. Code is\navailable at https://github.com/MysteryVaibhav/fake_news_semantics",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12274v3",
            "date": "2019-10-27 14:51:53+00:00",
            "title": "Algorithmic Copywriting: Automated Generation of Health-Related Advertisements to Improve their Performance",
            "authors": [
                "Brit Youngmann",
                "Ran Gilad-Bachrach",
                "Danny Karmon",
                "Elad Yom-Tov"
            ],
            "abstract": "Search advertising, a popular method for online marketing, has been employed\nto improve health by eliciting positive behavioral change. However, writing\neffective advertisements requires expertise and experimentation, which may not\nbe available to health authorities wishing to elicit such changes, especially\nwhen dealing with public health crises such as epidemic outbreaks.\n  Here we develop a framework, comprised of two neural networks models, that\nautomatically generate ads. First, it employs a generator model, which create\nads from web pages. It then employs a translation model, which transcribes ads\nto improve performance.\n  We trained the networks using 114K health-related ads shown on Microsoft\nAdvertising. We measure ads performance using the click-through rates (CTR).\n  Our experiments show that the generated advertisements received approximately\nthe same CTR as human-authored ads. The marginal contribution of the generator\nmodel was, on average, 28\\% lower than that of human-authored ads, while the\ntranslator model received, on average, 32\\% more clicks than human-authored\nads. Our analysis shows that the translator model produces ads reflecting\nhigher values of psychological attributes associated with a user action,\nincluding higher valance and arousal, and more calls-to-actions. In contrast,\nlevels of these attributes in ads produced by the generator model are similar\nto those of human-authored ads.\n  Our results demonstrate the ability to automatically generate useful\nadvertisements for the health domain. We believe that our work offers health\nauthorities an improved ability to nudge people towards healthier behaviors\nwhile saving the time and cost needed to build effective advertising campaigns.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.12299v2",
            "date": "2019-10-27 16:54:26+00:00",
            "title": "Induced Inflection-Set Keyword Search in Speech",
            "authors": [
                "Oliver Adams",
                "Matthew Wiesner",
                "Jan Trmal",
                "Garrett Nicolai",
                "David Yarowsky"
            ],
            "abstract": "We investigate the problem of searching for a lexeme-set in speech by\nsearching for its inflectional variants. Experimental results indicate how\nlexeme-set search performance changes with the number of hypothesized\ninflections, while ablation experiments highlight the relative importance of\ndifferent components in the lexeme-set search pipeline and the value of using\ncurated inflectional paradigms. We provide a recipe and evaluation set for the\ncommunity to use as an extrinsic measure of the performance of inflection\ngeneration approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12354v1",
            "date": "2019-10-27 21:11:42+00:00",
            "title": "Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order",
            "authors": [
                "Vladislav Kurenkov",
                "Bulat Maksudov",
                "Adil Khan"
            ],
            "abstract": "In this work, we analyze the performance of general deep reinforcement\nlearning algorithms for a task-oriented language grounding problem, where\nlanguage input contains multiple sub-goals and their order of execution is\nnon-linear.\n  We generate a simple instructional language for the GridWorld environment,\nthat is built around three language elements (order connectors) defining the\norder of execution: one linear - \"comma\" and two non-linear - \"but first\", \"but\nbefore\". We apply one of the deep reinforcement learning baselines - Double DQN\nwith frame stacking and ablate several extensions such as Prioritized\nExperience Replay and Gated-Attention architecture.\n  Our results show that the introduction of non-linear order connectors\nimproves the success rate on instructions with a higher number of sub-goals in\n2-3 times, but it still does not exceed 20%. Also, we observe that the usage of\nGated-Attention provides no competitive advantage against concatenation in this\nsetting. Source code and experiments' results are available at\nhttps://github.com/vkurenkov/language-grounding-multigoal",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12366v3",
            "date": "2019-10-27 22:09:13+00:00",
            "title": "Thieves on Sesame Street! Model Extraction of BERT-based APIs",
            "authors": [
                "Kalpesh Krishna",
                "Gaurav Singh Tomar",
                "Ankur P. Parikh",
                "Nicolas Papernot",
                "Mohit Iyyer"
            ],
            "abstract": "We study the problem of model extraction in natural language processing, in\nwhich an adversary with only query access to a victim model attempts to\nreconstruct a local copy of that model. Assuming that both the adversary and\nvictim model fine-tune a large pretrained language model such as BERT (Devlin\net al. 2019), we show that the adversary does not need any real training data\nto successfully mount the attack. In fact, the attacker need not even use\ngrammatical or semantically meaningful queries: we show that random sequences\nof words coupled with task-specific heuristics form effective queries for model\nextraction on a diverse set of NLP tasks, including natural language inference\nand question answering. Our work thus highlights an exploit only made feasible\nby the shift towards transfer learning methods within the NLP community: for a\nquery budget of a few hundred dollars, an attacker can extract a model that\nperforms only slightly worse than the victim model. Finally, we study two\ndefense strategies against model extraction---membership classification and API\nwatermarking---which while successful against naive adversaries, are\nineffective against more sophisticated ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12367v2",
            "date": "2019-10-27 22:13:09+00:00",
            "title": "Training ASR models by Generation of Contextual Information",
            "authors": [
                "Kritika Singh",
                "Dmytro Okhonko",
                "Jun Liu",
                "Yongqiang Wang",
                "Frank Zhang",
                "Ross Girshick",
                "Sergey Edunov",
                "Fuchun Peng",
                "Yatharth Saraf",
                "Geoffrey Zweig",
                "Abdelrahman Mohamed"
            ],
            "abstract": "Supervised ASR models have reached unprecedented levels of accuracy, thanks\nin part to ever-increasing amounts of labelled training data. However, in many\napplications and locales, only moderate amounts of data are available, which\nhas led to a surge in semi- and weakly-supervised learning research. In this\npaper, we conduct a large-scale study evaluating the effectiveness of\nweakly-supervised learning for speech recognition by using loosely related\ncontextual information as a surrogate for ground-truth labels. For weakly\nsupervised training, we use 50k hours of public English social media videos\nalong with their respective titles and post text to train an encoder-decoder\ntransformer model. Our best encoder-decoder models achieve an average of 20.8%\nWER reduction over a 1000 hours supervised baseline, and an average of 13.4%\nWER reduction when using only the weakly supervised encoder for CTC\nfine-tuning. Our results show that our setup for weak supervision improved both\nthe encoder acoustic representations as well as the decoder language generation\nabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12383v1",
            "date": "2019-10-28 00:01:12+00:00",
            "title": "Effect of choice of probability distribution, randomness, and search methods for alignment modeling in sequence-to-sequence text-to-speech synthesis using hard alignment",
            "authors": [
                "Yusuke Yasuda",
                "Xin Wang",
                "Junichi Yamagishi"
            ],
            "abstract": "Sequence-to-sequence text-to-speech (TTS) is dominated by\nsoft-attention-based methods. Recently, hard-attention-based methods have been\nproposed to prevent fatal alignment errors, but their sampling method of\ndiscrete alignment is poorly investigated. This research investigates various\ncombinations of sampling methods and probability distributions for alignment\ntransition modeling in a hard-alignment-based sequence-to-sequence TTS method\ncalled SSNT-TTS. We clarify the common sampling methods of discrete variables\nincluding greedy search, beam search, and random sampling from a Bernoulli\ndistribution in a more general way. Furthermore, we introduce the binary\nConcrete distribution to model discrete variables more properly. The results of\na listening test shows that deterministic search is more preferable than\nstochastic search, and the binary Concrete distribution is robust with\nstochastic search for natural alignment transition.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.12418v2",
            "date": "2019-10-28 03:17:18+00:00",
            "title": "Unsupervised pre-training for sequence to sequence speech recognition",
            "authors": [
                "Zhiyun Fan",
                "Shiyu Zhou",
                "Bo Xu"
            ],
            "abstract": "This paper proposes a novel approach to pre-train encoder-decoder\nsequence-to-sequence (seq2seq) model with unpaired speech and transcripts\nrespectively. Our pre-training method is divided into two stages, named\nacoustic pre-trianing and linguistic pre-training. In the acoustic pre-training\nstage, we use a large amount of speech to pre-train the encoder by predicting\nmasked speech feature chunks with its context. In the linguistic pre-training\nstage, we generate synthesized speech from a large number of transcripts using\na single-speaker text to speech (TTS) system, and use the synthesized paired\ndata to pre-train decoder. This two-stage pre-training method integrates rich\nacoustic and linguistic knowledge into seq2seq model, which will benefit\ndownstream automatic speech recognition (ASR) tasks. The unsupervised\npre-training is finished on AISHELL-2 dataset and we apply the pre-trained\nmodel to multiple paired data ratios of AISHELL-1 and HKUST. We obtain relative\ncharacter error rate reduction (CERR) from 38.24% to 7.88% on AISHELL-1 and\nfrom 12.00% to 1.20% on HKUST. Besides, we apply our pretrained model to a\ncross-lingual case with CALLHOME dataset. For all six languages in CALLHOME\ndataset, our pre-training method makes model outperform baseline consistently.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.12441v1",
            "date": "2019-10-28 04:50:22+00:00",
            "title": "Online News Media Website Ranking Using User Generated Content",
            "authors": [
                "Samaneh Karimi",
                "Azadeh Shakery",
                "Rakesh Verma"
            ],
            "abstract": "News media websites are important online resources that have drawn great\nattention of text mining researchers. The main aim of this study is to propose\na framework for ranking online news websites from different viewpoints. The\nranking of news websites is useful information, which can benefit many\nnews-related tasks such as news retrieval and news recommendation. In the\nproposed framework, the ranking of news websites is obtained by calculating\nthree measures introduced in the paper and based on user-generated content.\nEach proposed measure is concerned with the performance of news websites from a\nparticular viewpoint including the completeness of news reports, the diversity\nof events being covered by the website and its speed. The use of user-generated\ncontent in this framework, as a partly-unbiased, real-time and low cost content\non the web distinguishes the proposed news website ranking framework from the\nliterature. The results obtained for three prominent news websites, BBC, CNN,\nNYTimes, show that BBC has the best performance in terms of news completeness\nand speed, and NYTimes has the best diversity in comparison with the other two\nwebsites.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.12446v1",
            "date": "2019-10-28 05:14:41+00:00",
            "title": "Towards Successful Social Media Advertising: Predicting the Influence of Commercial Tweets",
            "authors": [
                "Renhao Cui",
                "Gagan Agrawal",
                "Rajiv Ramnath"
            ],
            "abstract": "Businesses communicate using Twitter for a variety of reasons -- to raise\nawareness of their brands, to market new products, to respond to community\ncomments, and to connect with their customers and potential customers in a\ntargeted manner. For businesses to do this effectively, they need to understand\nwhich content and structural elements about a tweet make it influential, that\nis, widely liked, followed, and retweeted. This paper presents a systematic\nmethodology for analyzing commercial tweets, and predicting the influence on\ntheir readers. Our model, which use a combination of decoration and meta\nfeatures, outperforms the prediction ability of the baseline model as well as\nthe tweet embedding model. Further, in order to demonstrate a practical use of\nthis work, we show how an unsuccessful tweet may be engineered (for example,\nreworded) to increase its potential for success.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1910.12531v1",
            "date": "2019-10-28 10:23:03+00:00",
            "title": "Modeling Inter-Speaker Relationship in XLNet for Contextual Spoken Language Understanding",
            "authors": [
                "Jonggu Kim",
                "Jong-Hyeok Lee"
            ],
            "abstract": "We propose two methods to capture relevant history information in a\nmulti-turn dialogue by modeling inter-speaker relationship for spoken language\nunderstanding (SLU). Our methods are tailored for and therefore compatible with\nXLNet, which is a state-of-the-art pretrained model, so we verified our models\nbuilt on the top of XLNet. In our experiments, all models achieved higher\naccuracy than state-of-the-art contextual SLU models on two benchmark datasets.\nAnalysis on the results demonstrated that the proposed methods are effective to\nimprove SLU accuracy of XLNet. These methods to identify important dialogue\nhistory will be useful to alleviate ambiguity in SLU of the current utterance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12554v1",
            "date": "2019-10-28 11:06:21+00:00",
            "title": "Exploring Kernel Functions in the Softmax Layer for Contextual Word Classification",
            "authors": [
                "Yingbo Gao",
                "Christian Herold",
                "Weiyue Wang",
                "Hermann Ney"
            ],
            "abstract": "Prominently used in support vector machines and logistic regressions, kernel\nfunctions (kernels) can implicitly map data points into high dimensional spaces\nand make it easier to learn complex decision boundaries. In this work, by\nreplacing the inner product function in the softmax layer, we explore the use\nof kernels for contextual word classification. In order to compare the\nindividual kernels, experiments are conducted on standard language modeling and\nmachine translation tasks. We observe a wide range of performances across\ndifferent kernel settings. Extending the results, we look at the gradient\nproperties, investigate various mixture strategies and examine the\ndisambiguation abilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12574v1",
            "date": "2019-10-28 12:13:38+00:00",
            "title": "A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media",
            "authors": [
                "Marzieh Mozafari",
                "Reza Farahbakhsh",
                "Noel Crespi"
            ],
            "abstract": "Generated hateful and toxic content by a portion of users in social media is\na rising phenomenon that motivated researchers to dedicate substantial efforts\nto the challenging direction of hateful content identification. We not only\nneed an efficient automatic hate speech detection model based on advanced\nmachine learning and natural language processing, but also a sufficiently large\namount of annotated data to train a model. The lack of a sufficient amount of\nlabelled hate speech data, along with the existing biases, has been the main\nissue in this domain of research. To address these needs, in this study we\nintroduce a novel transfer learning approach based on an existing pre-trained\nlanguage model called BERT (Bidirectional Encoder Representations from\nTransformers). More specifically, we investigate the ability of BERT at\ncapturing hateful context within social media content by using new fine-tuning\nmethods based on transfer learning. To evaluate our proposed approach, we use\ntwo publicly available datasets that have been annotated for racism, sexism,\nhate, or offensive content on Twitter. The results show that our solution\nobtains considerable performance on these datasets in terms of precision and\nrecall in comparison to existing approaches. Consequently, our model can\ncapture some biases in data annotation and collection process and can\npotentially lead us to a more accurate model.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1910.12592v1",
            "date": "2019-10-16 11:27:27+00:00",
            "title": "BUT System Description to VoxCeleb Speaker Recognition Challenge 2019",
            "authors": [
                "Hossein Zeinali",
                "Shuai Wang",
                "Anna Silnova",
                "Pavel Mat\u011bjka",
                "Old\u0159ich Plchot"
            ],
            "abstract": "In this report, we describe the submission of Brno University of Technology\n(BUT) team to the VoxCeleb Speaker Recognition Challenge (VoxSRC) 2019. We also\nprovide a brief analysis of different systems on VoxCeleb-1 test sets.\nSubmitted systems for both Fixed and Open conditions are a fusion of 4\nConvolutional Neural Network (CNN) topologies. The first and second networks\nhave ResNet34 topology and use two-dimensional CNNs. The last two networks are\none-dimensional CNN and are based on the x-vector extraction topology. Some of\nthe networks are fine-tuned using additive margin angular softmax. Kaldi FBanks\nand Kaldi PLPs were used as features. The difference between Fixed and Open\nsystems lies in the used training data and fusion strategy. The best systems\nfor Fixed and Open conditions achieved 1.42% and 1.26% ERR on the challenge\nevaluation set respectively.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.12607v2",
            "date": "2019-10-23 15:28:51+00:00",
            "title": "Generative Pre-Training for Speech with Autoregressive Predictive Coding",
            "authors": [
                "Yu-An Chung",
                "James Glass"
            ],
            "abstract": "Learning meaningful and general representations from unannotated speech that\nare applicable to a wide range of tasks remains challenging. In this paper we\npropose to use autoregressive predictive coding (APC), a recently proposed\nself-supervised objective, as a generative pre-training approach for learning\nmeaningful, non-specific, and transferable speech representations. We pre-train\nAPC on large-scale unlabeled data and conduct transfer learning experiments on\nthree speech applications that require different information about speech\ncharacteristics to perform well: speech recognition, speech translation, and\nspeaker identification. Extensive experiments show that APC not only\noutperforms surface features (e.g., log Mel spectrograms) and other popular\nrepresentation learning methods on all three tasks, but is also effective at\nreducing downstream labeled data size and model parameters. We also investigate\nthe use of Transformers for modeling APC and find it superior to RNNs.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.12618v2",
            "date": "2019-10-25 07:47:56+00:00",
            "title": "Textual Data for Time Series Forecasting",
            "authors": [
                "David Obst",
                "Badih Ghattas",
                "Sandra Claudel",
                "Jairo Cugliari",
                "Yannig Goude",
                "Georges Oppenheim"
            ],
            "abstract": "While ubiquitous, textual sources of information such as company reports,\nsocial media posts, etc. are hardly included in prediction algorithms for time\nseries, despite the relevant information they may contain. In this work, openly\naccessible daily weather reports from France and the United-Kingdom are\nleveraged to predict time series of national electricity consumption, average\ntemperature and wind-speed with a single pipeline. Two methods of numerical\nrepresentation of text are considered, namely traditional Term Frequency -\nInverse Document Frequency (TF-IDF) as well as our own neural word embedding.\nUsing exclusively text, we are able to predict the aforementioned time series\nwith sufficient accuracy to be used to replace missing data. Furthermore the\nproposed word embeddings display geometric properties relating to the behavior\nof the time series and context similarity between words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12638v2",
            "date": "2019-10-25 01:55:12+00:00",
            "title": "Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders",
            "authors": [
                "Andy T. Liu",
                "Shu-wen Yang",
                "Po-Han Chi",
                "Po-chun Hsu",
                "Hung-yi Lee"
            ],
            "abstract": "We present Mockingjay as a new speech representation learning approach, where\nbidirectional Transformer encoders are pre-trained on a large amount of\nunlabeled speech. Previous speech representation methods learn through\nconditioning on past frames and predicting information about future frames.\nWhereas Mockingjay is designed to predict the current frame through jointly\nconditioning on both past and future contexts. The Mockingjay representation\nimproves performance for a wide range of downstream tasks, including phoneme\nclassification, speaker recognition, and sentiment classification on spoken\ncontent, while outperforming other approaches. Mockingjay is empirically\npowerful and can be fine-tuned with downstream models, with only 2 epochs we\nfurther improve performance dramatically. In a low resource setting with only\n0.1% of labeled data, we outperform the result of Mel-features that uses all\n100% labeled data.",
            "journal ref": "ICASSP 2020 - 2020 IEEE International Conference on Acoustics,\n  Speech and Signal Processing (ICASSP)",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.12647v2",
            "date": "2019-10-25 06:25:25+00:00",
            "title": "HUBERT Untangles BERT to Improve Transfer across NLP Tasks",
            "authors": [
                "Mehrad Moradshahi",
                "Hamid Palangi",
                "Monica S. Lam",
                "Paul Smolensky",
                "Jianfeng Gao"
            ],
            "abstract": "We introduce HUBERT which combines the structured-representational power of\nTensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional\nTransformer language model. We show that there is shared structure between\ndifferent NLP datasets that HUBERT, but not BERT, is able to learn and\nleverage. We validate the effectiveness of our model on the GLUE benchmark and\nHANS dataset. Our experiment results show that untangling data-specific\nsemantics from general language structure is key for better transfer among NLP\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12729v2",
            "date": "2019-10-28 14:50:03+00:00",
            "title": "Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learning",
            "authors": [
                "Alexander H. Liu",
                "Tao Tu",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "In this paper we propose a Sequential Representation Quantization AutoEncoder\n(SeqRQ-AE) to learn from primarily unpaired audio data and produce sequences of\nrepresentations very close to phoneme sequences of speech utterances. This is\nachieved by proper temporal segmentation to make the representations\nphoneme-synchronized, and proper phonetic clustering to have total number of\ndistinct representations close to the number of phonemes. Mapping between the\ndistinct representations and phonemes is learned from a small amount of\nannotated paired data. Preliminary experiments on LJSpeech demonstrated the\nlearned representations for vowels have relative locations in latent space in\ngood parallel to that shown in the IPA vowel chart defined by linguistics\nexperts. With less than 20 minutes of annotated speech, our method outperformed\nexisting methods on phoneme recognition and is able to synthesize intelligible\nspeech that beats our baseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12740v2",
            "date": "2019-10-28 15:09:01+00:00",
            "title": "Sequence-to-sequence Automatic Speech Recognition with Word Embedding Regularization and Fused Decoding",
            "authors": [
                "Alexander H. Liu",
                "Tzu-Wei Sung",
                "Shun-Po Chuang",
                "Hung-yi Lee",
                "Lin-shan Lee"
            ],
            "abstract": "In this paper, we investigate the benefit that off-the-shelf word embedding\ncan bring to the sequence-to-sequence (seq-to-seq) automatic speech recognition\n(ASR). We first introduced the word embedding regularization by maximizing the\ncosine similarity between a transformed decoder feature and the target word\nembedding. Based on the regularized decoder, we further proposed the fused\ndecoding mechanism. This allows the decoder to consider the semantic\nconsistency during decoding by absorbing the information carried by the\ntransformed decoder feature, which is learned to be close to the target word\nembedding. Initial results on LibriSpeech demonstrated that pre-trained word\nembedding can significantly lower ASR recognition error with a negligible cost,\nand the choice of word embedding algorithms among Skip-gram, CBOW and BERT is\nimportant.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12795v1",
            "date": "2019-10-28 16:46:24+00:00",
            "title": "Learning Data Manipulation for Augmentation and Weighting",
            "authors": [
                "Zhiting Hu",
                "Bowen Tan",
                "Ruslan Salakhutdinov",
                "Tom Mitchell",
                "Eric P. Xing"
            ],
            "abstract": "Manipulating data, such as weighting data examples or augmenting with new\ninstances, has been increasingly used to improve model training. Previous work\nhas studied various rule- or learning-based approaches designed for specific\ntypes of data manipulation. In this work, we propose a new method that supports\nlearning different manipulation schemes with the same gradient-based algorithm.\nOur approach builds upon a recent connection of supervised learning and\nreinforcement learning (RL), and adapts an off-the-shelf reward learning\nalgorithm from RL for joint data manipulation learning and model training.\nDifferent parameterization of the \"data reward\" function instantiates different\nmanipulation schemes. We showcase data augmentation that learns a text\ntransformation network, and data weighting that dynamically adapts the data\nsample importance. Experiments show the resulting algorithms significantly\nimprove the image and text classification performance in low data regime and\nclass-imbalance problems.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.12853v1",
            "date": "2019-10-28 17:59:02+00:00",
            "title": "A Game Theoretic Approach to Class-wise Selective Rationalization",
            "authors": [
                "Shiyu Chang",
                "Yang Zhang",
                "Mo Yu",
                "Tommi S. Jaakkola"
            ],
            "abstract": "Selection of input features such as relevant pieces of text has become a\ncommon technique of highlighting how complex neural predictors operate. The\nselection can be optimized post-hoc for trained models or incorporated directly\ninto the method itself (self-explaining). However, an overall selection does\nnot properly capture the multi-faceted nature of useful rationales such as pros\nand cons for decisions. To this end, we propose a new game theoretic approach\nto class-dependent rationalization, where the method is specifically trained to\nhighlight evidence supporting alternative conclusions. Each class involves\nthree players set up competitively to find evidence for factual and\ncounterfactual scenarios. We show theoretically in a simplified scenario how\nthe game drives the solution towards meaningful class-dependent rationales. We\nevaluate the method in single- and multi-aspect sentiment classification tasks\nand demonstrate that the proposed method is able to identify both factual\n(justifying the ground truth label) and counterfactual (countering the ground\ntruth label) rationales consistent with human rationalization. The code for our\nmethod is publicly available.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.12944v1",
            "date": "2019-10-28 20:01:54+00:00",
            "title": "Moving Towards Open Set Incremental Learning: Readily Discovering New Authors",
            "authors": [
                "Justin Leo",
                "Jugal Kalita"
            ],
            "abstract": "The classification of textual data often yields important information. Most\nclassifiers work in a closed world setting where the classifier is trained on a\nknown corpus, and then it is tested on unseen examples that belong to one of\nthe classes seen during training. Despite the usefulness of this design, often\nthere is a need to classify unseen examples that do not belong to any of the\nclasses on which the classifier was trained. This paper describes the open set\nscenario where unseen examples from previously unseen classes are handled while\ntesting. This further examines a process of enhanced open set classification\nwith a deep neural network that discovers new classes by clustering the\nexamples identified as belonging to unknown classes, followed by a process of\nretraining the classifier with newly recognized classes. Through this process\nthe model moves to an incremental learning model where it continuously finds\nand learns from novel classes of data that have been identified automatically.\nThis paper also develops a new metric that measures multiple attributes of\nclustering open set data. Multiple experiments across two author attribution\ndata sets demonstrate the creation an incremental model that produces excellent\nresults.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.12956v3",
            "date": "2019-10-28 20:32:56+00:00",
            "title": "Cross-Domain Ambiguity Detection using Linear Transformation of Word Embedding Spaces",
            "authors": [
                "Vaibhav Jain",
                "Ruchika Malhotra",
                "Sanskar Jain",
                "Nishant Tanwar"
            ],
            "abstract": "The requirements engineering process is a crucial stage of the software\ndevelopment life cycle. It involves various stakeholders from different\nprofessional backgrounds, particularly in the requirements elicitation phase.\nEach stakeholder carries distinct domain knowledge, causing them to differently\ninterpret certain words, leading to cross-domain ambiguity. This can result in\nmisunderstanding amongst them and jeopardize the entire project. This paper\nproposes a natural language processing approach to find potentially ambiguous\nwords for a given set of domains. The idea is to apply linear transformations\non word embedding models trained on different domain corpora, to bring them\ninto a unified embedding space. The approach then finds words with divergent\nembeddings as they signify a variation in the meaning across the domains. It\ncan help a requirements analyst in preventing misunderstandings during\nelicitation interviews and meetings by defining a set of potentially ambiguous\nterms in advance. The paper also discusses certain problems with the existing\napproaches and discusses how the proposed approach resolves them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12977v1",
            "date": "2019-10-28 21:29:21+00:00",
            "title": "Transformer-Transducer: End-to-End Speech Recognition with Self-Attention",
            "authors": [
                "Ching-Feng Yeh",
                "Jay Mahadeokar",
                "Kaustubh Kalgaonkar",
                "Yongqiang Wang",
                "Duc Le",
                "Mahaveer Jain",
                "Kjell Schubert",
                "Christian Fuegen",
                "Michael L. Seltzer"
            ],
            "abstract": "We explore options to use Transformer networks in neural transducer for\nend-to-end speech recognition. Transformer networks use self-attention for\nsequence modeling and comes with advantages in parallel computation and\ncapturing contexts. We propose 1) using VGGNet with causal convolution to\nincorporate positional information and reduce frame rate for efficient\ninference 2) using truncated self-attention to enable streaming for Transformer\nand reduce computational complexity. All experiments are conducted on the\npublic LibriSpeech corpus. The proposed Transformer-Transducer outperforms\nneural transducer with LSTM/BLSTM networks and achieved word error rates of\n6.37 % on the test-clean set and 15.30 % on the test-other set, while remaining\nstreamable, compact with 45.7M parameters for the entire system, and\ncomputationally efficient with complexity of O(T), where T is input sequence\nlength.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.13162v1",
            "date": "2019-10-29 10:06:56+00:00",
            "title": "An Efficient Model for Sentiment Analysis of Electronic Product Reviews in Vietnamese",
            "authors": [
                "Suong N. Hoang",
                "Linh V. Nguyen",
                "Tai Huynh",
                "Vuong T. Pham"
            ],
            "abstract": "In the past few years, the growth of e-commerce and digital marketing in\nVietnam has generated a huge volume of opinionated data. Analyzing those data\nwould provide enterprises with insight for better business decisions. In this\nwork, as part of the Advosights project, we study sentiment analysis of product\nreviews in Vietnamese. The final solution is based on Self-attention neural\nnetworks, a flexible architecture for text classification task with about\n90.16% of accuracy in 0.0124 second, a very fast inference time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13276v2",
            "date": "2019-10-29 14:06:49+00:00",
            "title": "a novel cross-lingual voice cloning approach with a few text-free samples",
            "authors": [
                "Xinyong Zhou",
                "Hao Che",
                "Xiaorui Wang",
                "Lei Xie"
            ],
            "abstract": "In this paper, we present a cross-lingual voice cloning approach. BN features\nobtained by SI-ASR model are used as a bridge across speakers and language\nboundaries. The relationships between text and BN features are modeled by the\nlatent prosody model. The acoustic model learns the translation from BN\nfeatures to acoustic features. The acoustic model is fine-tuned with a few\nsamples of the target speaker to realize voice cloning. This system can\ngenerate speech of arbitrary utterance of target language in cross-lingual\nspeakers' voice. We verify that with small amount of audio data, our proposed\napproach can well handle cross-lingual tasks. And in intra-lingual tasks, our\nproposed approach also performs better than baseline approach in naturalness\nand similarity.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.13425v2",
            "date": "2019-10-29 17:43:37+00:00",
            "title": "Weakly-Supervised Deep Learning for Domain Invariant Sentiment Classification",
            "authors": [
                "Pratik Kayal",
                "Mayank Singh",
                "Pawan Goyal"
            ],
            "abstract": "The task of learning a sentiment classification model that adapts well to any\ntarget domain, different from the source domain, is a challenging problem.\nMajority of the existing approaches focus on learning a common representation\nby leveraging both source and target data during training. In this paper, we\nintroduce a two-stage training procedure that leverages weakly supervised\ndatasets for developing simple lift-and-shift-based predictive models without\nbeing exposed to the target domain during the training phase. Experimental\nresults show that transfer with weak supervision from a source domain to\nvarious target domains provides performance very close to that obtained via\nsupervised training on the target domain itself.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.13461v1",
            "date": "2019-10-29 18:01:00+00:00",
            "title": "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension",
            "authors": [
                "Mike Lewis",
                "Yinhan Liu",
                "Naman Goyal",
                "Marjan Ghazvininejad",
                "Abdelrahman Mohamed",
                "Omer Levy",
                "Ves Stoyanov",
                "Luke Zettlemoyer"
            ],
            "abstract": "We present BART, a denoising autoencoder for pretraining sequence-to-sequence\nmodels. BART is trained by (1) corrupting text with an arbitrary noising\nfunction, and (2) learning a model to reconstruct the original text. It uses a\nstandard Tranformer-based neural machine translation architecture which,\ndespite its simplicity, can be seen as generalizing BERT (due to the\nbidirectional encoder), GPT (with the left-to-right decoder), and many other\nmore recent pretraining schemes. We evaluate a number of noising approaches,\nfinding the best performance by both randomly shuffling the order of the\noriginal sentences and using a novel in-filling scheme, where spans of text are\nreplaced with a single mask token. BART is particularly effective when fine\ntuned for text generation but also works well for comprehension tasks. It\nmatches the performance of RoBERTa with comparable training resources on GLUE\nand SQuAD, achieves new state-of-the-art results on a range of abstractive\ndialogue, question answering, and summarization tasks, with gains of up to 6\nROUGE. BART also provides a 1.1 BLEU increase over a back-translation system\nfor machine translation, with only target language pretraining. We also report\nablation experiments that replicate other pretraining schemes within the BART\nframework, to better measure which factors most influence end-task performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13488v2",
            "date": "2019-10-29 19:23:16+00:00",
            "title": "Does Speech enhancement of publicly available data help build robust Speech Recognition Systems?",
            "authors": [
                "Bhavya Ghai",
                "Buvana Ramanan",
                "Klaus Mueller"
            ],
            "abstract": "Automatic speech recognition (ASR) systems play a key role in many commercial\nproducts including voice assistants. Typically, they require large amounts of\nclean speech data for training which gives an undue advantage to large\norganizations which have tons of private data. In this paper, we have first\ncurated a fairly big dataset using publicly available data sources. Thereafter,\nwe tried to investigate if we can use publicly available noisy data to train\nrobust ASR systems. We have used speech enhancement to clean the noisy data\nfirst and then used it together with its cleaned version to train ASR systems.\nWe have found that using speech enhancement gives 9.5\\% better word error rate\nthan training on just noisy data and 9\\% better than training on just clean\ndata. It's performance is also comparable to the ideal case scenario when\ntrained on noisy and its clean version.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.13573v2",
            "date": "2019-10-29 23:25:59+00:00",
            "title": "Semi-Supervised Natural Language Approach for Fine-Grained Classification of Medical Reports",
            "authors": [
                "Neil Deshmukh",
                "Selin Gumustop",
                "Romane Gauriau",
                "Varun Buch",
                "Bradley Wright",
                "Christopher Bridge",
                "Ram Naidu",
                "Katherine Andriole",
                "Bernardo Bizzo"
            ],
            "abstract": "Although machine learning has become a powerful tool to augment doctors in\nclinical analysis, the immense amount of labeled data that is necessary to\ntrain supervised learning approaches burdens each development task as time and\nresource intensive. The vast majority of dense clinical information is stored\nin written reports, detailing pertinent patient information. The challenge with\nutilizing natural language data for standard model development is due to the\ncomplex nature of the modality. In this research, a model pipeline was\ndeveloped to utilize an unsupervised approach to train an encoder-language\nmodel, a recurrent network, to generate document encodings; which then can be\nused as features passed into a decoder-classifier model that requires\nmagnitudes less labeled data than previous approaches to differentiate between\nfine-grained disease classes accurately. The language model was trained on\nunlabeled radiology reports from the Massachusetts General Hospital Radiology\nDepartment (n=218,159) and terminated with a loss of 1.62. The classification\nmodels were trained on three labeled datasets of head CT studies of reported\npatients, presenting large vessel occlusion (n=1403), acute ischemic strokes\n(n=331), and intracranial hemorrhage (n=4350), to identify a variety of\ndifferent findings directly from the radiology report data; resulting in AUCs\nof 0.98, 0.95, and 0.99, respectively, for the large vessel occlusion, acute\nischemic stroke, and intracranial hemorrhage datasets. The output encodings are\nable to be used in conjunction with imaging data, to create models that can\nprocess a multitude of different modalities. The ability to automatically\nextract relevant features from textual data allows for faster model development\nand integration of textual modality, overall, allowing clinical reports to\nbecome a more viable input for more encompassing and accurate deep learning\nmodels.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.13689v1",
            "date": "2019-10-30 06:11:15+00:00",
            "title": "ON-TRAC Consortium End-to-End Speech Translation Systems for the IWSLT 2019 Shared Task",
            "authors": [
                "Ha Nguyen",
                "Natalia Tomashenko",
                "Marcely Zanon Boito",
                "Antoine Caubriere",
                "Fethi Bougares",
                "Mickael Rouvier",
                "Laurent Besacier",
                "Yannick Esteve"
            ],
            "abstract": "This paper describes the ON-TRAC Consortium translation systems developed for\nthe end-to-end model task of IWSLT Evaluation 2019 for the\nEnglish-to-Portuguese language pair. ON-TRAC Consortium is composed of\nresearchers from three French academic laboratories: LIA (Avignon\nUniversit\\'e), LIG (Universit\\'e Grenoble Alpes), and LIUM (Le Mans\nUniversit\\'e). A single end-to-end model built as a neural encoder-decoder\narchitecture with attention mechanism was used for two primary submissions\ncorresponding to the two EN-PT evaluations sets: (1) TED (MuST-C) and (2) How2.\nIn this paper, we notably investigate impact of pooling heterogeneous corpora\nfor training, impact of target tokenization (characters or BPEs), impact of\nspeech input segmentation and we also compare our best end-to-end model (BLEU\nof 26.91 on MuST-C and 43.82 on How2 validation sets) to a pipeline (ASR+MT)\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13707v1",
            "date": "2019-10-30 08:11:05+00:00",
            "title": "Jointly optimal dereverberation and beamforming",
            "authors": [
                "Christoph Boeddeker",
                "Tomohiro Nakatani",
                "Keisuke Kinoshita",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "We previously proposed an optimal (in the maximum likelihood sense)\nconvolutional beamformer that can perform simultaneous denoising and\ndereverberation, and showed its superiority over the widely used cascade of a\nWPE dereverberation filter and a conventional MPDR beamformer. However, it has\nnot been fully investigated which components in the convolutional beamformer\nyield such superiority. To this end, this paper presents a new derivation of\nthe convolutional beamformer that allows us to factorize it into a WPE\ndereverberation filter, and a special type of a (non-convolutional) beamformer,\nreferred to as a wMPDR beamformer, without loss of optimality. With\nexperiments, we show that the superiority of the convolutional beamformer in\nfact comes from its wMPDR part.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.13923v3",
            "date": "2019-10-30 15:20:07+00:00",
            "title": "Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer",
            "authors": [
                "Genta Indra Winata",
                "Samuel Cahyawijaya",
                "Zhaojiang Lin",
                "Zihan Liu",
                "Pascale Fung"
            ],
            "abstract": "Highly performing deep neural networks come at the cost of computational\ncomplexity that limits their practicality for deployment on portable devices.\nWe propose the low-rank transformer (LRT), a memory-efficient and fast neural\narchitecture that significantly reduces the parameters and boosts the speed of\ntraining and inference for end-to-end speech recognition. Our approach reduces\nthe number of parameters of the network by more than 50% and speeds up the\ninference time by around 1.35x compared to the baseline transformer model. The\nexperiments show that our LRT model generalizes better and yields lower error\nrates on both validation and test sets compared to an uncompressed transformer\nmodel. The LRT model outperforms those from existing works on several datasets\nin an end-to-end setting without using an external language model or acoustic\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.13934v1",
            "date": "2019-10-30 15:39:31+00:00",
            "title": "SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition",
            "authors": [
                "Lukas Drude",
                "Jens Heitkaemper",
                "Christoph Boeddeker",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "We present a multi-channel database of overlapping speech for training,\nevaluation, and detailed analysis of source separation and extraction\nalgorithms: SMS-WSJ -- Spatialized Multi-Speaker Wall Street Journal. It\nconsists of artificially mixed speech taken from the WSJ database, but unlike\nearlier databases we consider all WSJ0+1 utterances and take care of strictly\nseparating the speaker sets present in the training, validation and test sets.\nWhen spatializing the data we ensure a high degree of randomness w.r.t. room\nsize, array center and rotation, as well as speaker position. Furthermore, this\npaper offers a critical assessment of recently proposed measures of source\nseparation performance. Alongside the code to generate the database we provide\na source separation baseline and a Kaldi recipe with competitive word error\nrates to provide common ground for evaluation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1910.14025v2",
            "date": "2019-10-30 08:04:43+00:00",
            "title": "Graph Neural News Recommendation with Long-term and Short-term Interest Modeling",
            "authors": [
                "Linmei Hu",
                "Chen Li",
                "Chuan Shi",
                "Cheng Yang",
                "Chao Shao"
            ],
            "abstract": "With the information explosion of news articles, personalized news\nrecommendation has become important for users to quickly find news that they\nare interested in. Existing methods on news recommendation mainly include\ncollaborative filtering methods which rely on direct user-item interactions and\ncontent based methods which characterize the content of user reading history.\nAlthough these methods have achieved good performances, they still suffer from\ndata sparse problem, since most of them fail to extensively exploit high-order\nstructure information (similar users tend to read similar news articles) in\nnews recommendation systems. In this paper, we propose to build a heterogeneous\ngraph to explicitly model the interactions among users, news and latent topics.\nThe incorporated topic information would help indicate a user's interest and\nalleviate the sparsity of user-item interactions. Then we take advantage of\ngraph neural networks to learn user and news representations that encode\nhigh-order structure information by propagating embeddings over the graph. The\nlearned user embeddings with complete historic user clicks capture the users'\nlong-term interests. We also consider a user's short-term interest using the\nrecent reading history with an attention based LSTM model. Experimental results\non real-world datasets show that our proposed model significantly outperforms\nstate-of-the-art methods on news recommendation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1910.14084v1",
            "date": "2019-10-30 18:57:28+00:00",
            "title": "Building an Application Independent Natural Language Interface",
            "authors": [
                "Sahisnu Mazumder",
                "Bing Liu",
                "Shuai Wang",
                "Sepideh Esmaeilpour"
            ],
            "abstract": "Traditional approaches to building natural language (NL) interfaces typically\nuse a semantic parser to parse the user command and convert it to a logical\nform, which is then translated to an executable action in an application.\nHowever, it is still challenging for a semantic parser to correctly parse\nnatural language. For a different domain, the parser may need to be retrained\nor tuned, and a new translator also needs to be written to convert the logical\nforms to executable actions. In this work, we propose a novel and application\nindependent approach to building NL interfaces that does not need a semantic\nparser or a translator. It is based on natural language to natural language\nmatching and learning, where the representation of each action and each user\ncommand are both in natural language. To perform a user intended action, the\nsystem only needs to match the user command with the correct action\nrepresentation, and then execute the corresponding action. The system also\ninteractively learns new (paraphrased) commands for actions to expand the\naction representations over time. Our experimental results show the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14353v1",
            "date": "2019-10-31 10:32:43+00:00",
            "title": "Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task",
            "authors": [
                "Valeriya Slovikovskaya"
            ],
            "abstract": "In this paper, we report improved results of the Fake News Challenge Stage 1\n(FNC-1) stance detection task. This gain in performance is due to the\ngeneralization power of large language models based on Transformer\narchitecture, invented, trained and publicly released over the last two years.\nSpecifically (1) we improved the FNC-1 best performing model adding BERT\nsentence embedding of input sequences as a model feature, (2) we fine-tuned\nBERT, XLNet, and RoBERTa transformers on FNC-1 extended dataset and obtained\nstate-of-the-art results on FNC-1 task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14443v1",
            "date": "2019-10-31 13:15:24+00:00",
            "title": "Multi-scale Octave Convolutions for Robust Speech Recognition",
            "authors": [
                "Joanna Rownicka",
                "Peter Bell",
                "Steve Renals"
            ],
            "abstract": "We propose a multi-scale octave convolution layer to learn robust speech\nrepresentations efficiently. Octave convolutions were introduced by Chen et al\n[1] in the computer vision field to reduce the spatial redundancy of the\nfeature maps by decomposing the output of a convolutional layer into feature\nmaps at two different spatial resolutions, one octave apart. This approach\nimproved the efficiency as well as the accuracy of the CNN models. The accuracy\ngain was attributed to the enlargement of the receptive field in the original\ninput space. We argue that octave convolutions likewise improve the robustness\nof learned representations due to the use of average pooling in the lower\nresolution group, acting as a low-pass filter. We test this hypothesis by\nevaluating on two noisy speech corpora - Aurora-4 and AMI. We extend the octave\nconvolution concept to multiple resolution groups and multiple octaves. To\nevaluate the robustness of the inferred representations, we report the\nsimilarity between clean and noisy encodings using an affine projection loss as\na proxy robustness measure. The results show that proposed method reduces the\nWER by up to 6.6% relative for Aurora-4 and 3.6% for AMI, while improving the\ncomputational efficiency of the CNN acoustic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14464v3",
            "date": "2019-10-31 13:38:01+00:00",
            "title": "What Question Answering can Learn from Trivia Nerds",
            "authors": [
                "Jordan Boyd-Graber",
                "Benjamin B\u00f6rschinger"
            ],
            "abstract": "In addition to the traditional task of getting machines to answer questions,\na major research question in question answering is to create interesting,\nchallenging questions that can help systems learn how to answer questions and\nalso reveal which systems are the best at answering questions. We argue that\ncreating a question answering dataset -- and the ubiquitous leaderboard that\ngoes with it -- closely resembles running a trivia tournament: you write\nquestions, have agents (either humans or machines) answer the questions, and\ndeclare a winner. However, the research community has ignored the decades of\nhard-learned lessons from decades of the trivia community creating vibrant,\nfair, and effective question answering competitions. After detailing problems\nwith existing QA datasets, we outline the key lessons -- removing ambiguity,\ndiscriminating skill, and adjudicating disputes -- that can transfer to QA\nresearch and how they might be implemented for the QA community.",
            "journal ref": "ACL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14609v1",
            "date": "2019-10-31 16:59:14+00:00",
            "title": "Can adversarial training learn image captioning ?",
            "authors": [
                "Jean-Benoit Delbrouck",
                "Bastien Vanderplaetse",
                "St\u00e9phane Dupont"
            ],
            "abstract": "Recently, generative adversarial networks (GAN) have gathered a lot of\ninterest. Their efficiency in generating unseen samples of high quality,\nespecially images, has improved over the years. In the field of Natural\nLanguage Generation (NLG), the use of the adversarial setting to generate\nmeaningful sentences has shown to be difficult for two reasons: the lack of\nexisting architectures to produce realistic sentences and the lack of\nevaluation tools. In this paper, we propose an adversarial architecture related\nto the conditional GAN (cGAN) that generates sentences according to a given\nimage (also called image captioning). This attempt is the first that uses no\npre-training or reinforcement methods. We also explain why our experiment\nsettings can be safely evaluated and interpreted for further works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14613v1",
            "date": "2019-10-31 17:01:24+00:00",
            "title": "Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning",
            "authors": [
                "Arvind Neelakantan",
                "Semih Yavuz",
                "Sharan Narang",
                "Vishaal Prasad",
                "Ben Goodrich",
                "Daniel Duckworth",
                "Chinnadhurai Sankar",
                "Xifeng Yan"
            ],
            "abstract": "Task-oriented dialog presents a difficult challenge encompassing multiple\nproblems including multi-turn language understanding and generation, knowledge\nretrieval and reasoning, and action prediction. Modern dialog systems typically\nbegin by converting conversation history to a symbolic object referred to as\nbelief state by using supervised learning. The belief state is then used to\nreason on an external knowledge source whose result along with the conversation\nhistory is used in action prediction and response generation tasks\nindependently. Such a pipeline of individually optimized components not only\nmakes the development process cumbersome but also makes it non-trivial to\nleverage session-level user reinforcement signals. In this paper, we develop\nNeural Assistant: a single neural network model that takes conversation history\nand an external knowledge source as input and jointly produces both text\nresponse and action to be taken by the system as output. The model learns to\nreason on the provided knowledge source with weak supervision signal coming\nfrom the text generation and the action prediction tasks, hence removing the\nneed for belief state annotations. In the MultiWOZ dataset, we study the effect\nof distant supervision, and the size of knowledge base on model performance. We\nfind that the Neural Assistant without belief states is able to incorporate\nexternal knowledge information achieving higher factual accuracy scores\ncompared to Transformer. In settings comparable to reported baseline systems,\nNeural Assistant when provided with oracle belief state significantly improves\nlanguage generation performance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.14659v3",
            "date": "2019-10-31 17:51:21+00:00",
            "title": "Masked Language Model Scoring",
            "authors": [
                "Julian Salazar",
                "Davis Liang",
                "Toan Q. Nguyen",
                "Katrin Kirchhoff"
            ],
            "abstract": "Pretrained masked language models (MLMs) require finetuning for most NLP\ntasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood\nscores (PLLs), which are computed by masking tokens one by one. We show that\nPLLs outperform scores from autoregressive language models like GPT-2 in a\nvariety of tasks. By rescoring ASR and NMT hypotheses, RoBERTa reduces an\nend-to-end LibriSpeech model's WER by 30% relative and adds up to +1.7 BLEU on\nstate-of-the-art baselines for low-resource translation pairs, with further\ngains from domain adaptation. We attribute this success to PLL's unsupervised\nexpression of linguistic acceptability without a left-to-right bias, greatly\nimproving on scores from GPT-2 (+10 points on island effects, NPI licensing in\nBLiMP). One can finetune MLMs to give scores without masking, enabling\ncomputation in a single inference pass. In all, PLLs and their associated\npseudo-perplexities (PPPLs) enable plug-and-play use of the growing number of\npretrained MLMs; e.g., we use a single cross-lingual model to rescore\ntranslations in multiple languages. We release our library for language model\nscoring at https://github.com/awslabs/mlm-scoring.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics (2020), 2699-2712",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.14671v2",
            "date": "2019-10-31 17:59:57+00:00",
            "title": "TAB-VCR: Tags and Attributes based Visual Commonsense Reasoning Baselines",
            "authors": [
                "Jingxiang Lin",
                "Unnat Jain",
                "Alexander G. Schwing"
            ],
            "abstract": "Reasoning is an important ability that we learn from a very early age. Yet,\nreasoning is extremely hard for algorithms. Despite impressive recent progress\nthat has been reported on tasks that necessitate reasoning, such as visual\nquestion answering and visual dialog, models often exploit biases in datasets.\nTo develop models with better reasoning abilities, recently, the new visual\ncommonsense reasoning (VCR) task has been introduced. Not only do models have\nto answer questions, but also do they have to provide a reason for the given\nanswer. The proposed baseline achieved compelling results, leveraging a\nmeticulously designed model composed of LSTM modules and attention nets. Here\nwe show that a much simpler model obtained by ablating and pruning the existing\nintricate baseline can perform better with half the number of trainable\nparameters. By associating visual features with attribute information and\nbetter text to image grounding, we obtain further improvements for our simpler\n& effective baseline, TAB-VCR. We show that this approach results in a 5.3%,\n4.4% and 6.5% absolute improvement over the previous state-of-the-art on\nquestion answering, answer justification and holistic VCR.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.00069v1",
            "date": "2019-10-31 19:30:54+00:00",
            "title": "Neural Cross-Lingual Relation Extraction Based on Bilingual Word Embedding Mapping",
            "authors": [
                "Jian Ni",
                "Radu Florian"
            ],
            "abstract": "Relation extraction (RE) seeks to detect and classify semantic relationships\nbetween entities, which provides useful information for many NLP applications.\nSince the state-of-the-art RE models require large amounts of manually\nannotated data and language-specific resources to achieve high accuracy, it is\nvery challenging to transfer an RE model of a resource-rich language to a\nresource-poor language. In this paper, we propose a new approach for\ncross-lingual RE model transfer based on bilingual word embedding mapping. It\nprojects word embeddings from a target language to a source language, so that a\nwell-trained source-language neural network RE model can be directly applied to\nthe target language. Experiment results show that the proposed approach\nachieves very good performance for a number of target languages on both\nin-house and open datasets, using a small bilingual dictionary with only 1K\nword pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00212v1",
            "date": "2019-11-01 05:53:50+00:00",
            "title": "Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning",
            "authors": [
                "Tao Jin",
                "Siyu Huang",
                "Yingming Li",
                "Zhongfei Zhang"
            ],
            "abstract": "This paper addresses the challenging task of video captioning which aims to\ngenerate descriptions for video data. Recently, the attention-based\nencoder-decoder structures have been widely used in video captioning. In\nexisting literature, the attention weights are often built from the information\nof an individual modality, while, the association relationships between\nmultiple modalities are neglected. Motivated by this observation, we propose a\nvideo captioning model with High-Order Cross-Modal Attention (HOCA) where the\nattention weights are calculated based on the high-order correlation tensor to\ncapture the frame-level cross-modal interaction of different modalities\nsufficiently. Furthermore, we novelly introduce Low-Rank HOCA which adopts\ntensor decomposition to reduce the extremely large space requirement of HOCA,\nleading to a practical and efficient implementation in real-world applications.\nExperimental results on two benchmark datasets, MSVD and MSR-VTT, show that\nLow-rank HOCA establishes a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.00262v1",
            "date": "2019-11-01 08:46:35+00:00",
            "title": "Finding the most similar textual documents using Case-Based Reasoning",
            "authors": [
                "Marko Mihajlovic",
                "Ning Xiong"
            ],
            "abstract": "In recent years, huge amounts of unstructured textual data on the Internet\nare a big difficulty for AI algorithms to provide the best recommendations for\nusers and their search queries. Since the Internet became widespread, a lot of\nresearch has been done in the field of Natural Language Processing (NLP) and\nmachine learning. Almost every solution transforms documents into Vector Space\nModels (VSM) in order to apply AI algorithms over them. One such approach is\nbased on Case-Based Reasoning (CBR). Therefore, the most important part of\nthose systems is to compute the similarity between numerical data points. In\n2016, the new similarity TS-SS metric is proposed, which showed\nstate-of-the-art results in the field of textual mining for unsupervised\nlearning. However, no one before has investigated its performances for\nsupervised learning (classification task). In this work, we devised a CBR\nsystem capable of finding the most similar documents for a given query aiming\nto investigate performances of the new state-of-the-art metric, TS-SS, in\naddition to the two other geometrical similarity measures --- Euclidean\ndistance and Cosine similarity --- that showed the best predictive results over\nseveral benchmark corpora. The results show surprising inappropriateness of\nTS-SS measure for high dimensional features.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.00359v2",
            "date": "2019-11-01 13:09:28+00:00",
            "title": "CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data",
            "authors": [
                "Guillaume Wenzek",
                "Marie-Anne Lachaux",
                "Alexis Conneau",
                "Vishrav Chaudhary",
                "Francisco Guzm\u00e1n",
                "Armand Joulin",
                "Edouard Grave"
            ],
            "abstract": "Pre-training text representations have led to significant improvements in\nmany areas of natural language processing. The quality of these models benefits\ngreatly from the size of the pretraining corpora as long as its quality is\npreserved. In this paper, we describe an automatic pipeline to extract massive\nhigh-quality monolingual datasets from Common Crawl for a variety of languages.\nOur pipeline follows the data processing introduced in fastText (Mikolov et\nal., 2017; Grave et al., 2018), that deduplicates documents and identifies\ntheir language. We augment this pipeline with a filtering step to select\ndocuments that are close to high quality corpora like Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00473v1",
            "date": "2019-11-01 17:30:21+00:00",
            "title": "BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding",
            "authors": [
                "Emad Elwany",
                "Dave Moore",
                "Gaurav Oberoi"
            ],
            "abstract": "Fine-tuning language models, such as BERT, on domain specific corpora has\nproven to be valuable in domains like scientific papers and biomedical text. In\nthis paper, we show that fine-tuning BERT on legal documents similarly provides\nvaluable improvements on NLP tasks in the legal domain. Demonstrating this\noutcome is significant for analyzing commercial agreements, because obtaining\nlarge legal corpora is challenging due to their confidential nature. As such,\nwe show that having access to large legal corpora is a competitive advantage\nfor commercial applications, and academic research on analyzing contracts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00497v1",
            "date": "2019-10-31 22:37:54+00:00",
            "title": "A Narration-based Reward Shaping Approach using Grounded Natural Language Commands",
            "authors": [
                "Nicholas Waytowich",
                "Sean L. Barton",
                "Vernon Lawhern",
                "Garrett Warnell"
            ],
            "abstract": "While deep reinforcement learning techniques have led to agents that are\nsuccessfully able to learn to perform a number of tasks that had been\npreviously unlearnable, these techniques are still susceptible to the\nlongstanding problem of reward sparsity. This is especially true for tasks such\nas training an agent to play StarCraft II, a real-time strategy game where\nreward is only given at the end of a game which is usually very long. While\nthis problem can be addressed through reward shaping, such approaches typically\nrequire a human expert with specialized knowledge. Inspired by the vision of\nenabling reward shaping through the more-accessible paradigm of\nnatural-language narration, we develop a technique that can provide the\nbenefits of reward shaping using natural language commands. Our\nnarration-guided RL agent projects sequences of natural-language commands into\nthe same high-dimensional representation space as corresponding goal states. We\nshow that we can get improved performance with our method compared to\ntraditional reward-shaping approaches. Additionally, we demonstrate the ability\nof our method to generalize to unseen natural-language commands.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.00760v2",
            "date": "2019-11-02 18:05:20+00:00",
            "title": "GRAPHENE: A Precise Biomedical Literature Retrieval Engine with Graph Augmented Deep Learning and External Knowledge Empowerment",
            "authors": [
                "Sendong Zhao",
                "Chang Su",
                "Andrea Sboner",
                "Fei Wang"
            ],
            "abstract": "Effective biomedical literature retrieval (BLR) plays a central role in\nprecision medicine informatics. In this paper, we propose GRAPHENE, which is a\ndeep learning based framework for precise BLR. GRAPHENE consists of three main\ndifferent modules 1) graph-augmented document representation learning; 2) query\nexpansion and representation learning and 3) learning to rank biomedical\narticles. The graph-augmented document representation learning module\nconstructs a document-concept graph containing biomedical concept nodes and\ndocument nodes so that global biomedical related concept from external\nknowledge source can be captured, which is further connected to a BiLSTM so\nboth local and global topics can be explored. Query expansion and\nrepresentation learning module expands the query with abbreviations and\ndifferent names, and then builds a CNN-based model to convolve the expanded\nquery and obtain a vector representation for each query. Learning to rank\nminimizes a ranking loss between biomedical articles with the query to learn\nthe retrieval function. Experimental results on applying our system to TREC\nPrecision Medicine track data are provided to demonstrate its effectiveness.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.00773v1",
            "date": "2019-11-02 19:19:28+00:00",
            "title": "Design and Challenges of Cloze-Style Reading Comprehension Tasks on Multiparty Dialogue",
            "authors": [
                "Changmao Li",
                "Tianhao Liu",
                "Jinho Choi"
            ],
            "abstract": "This paper analyzes challenges in cloze-style reading comprehension on\nmultiparty dialogue and suggests two new tasks for more comprehensive\npredictions of personal entities in daily conversations. We first demonstrate\nthat there are substantial limitations to the evaluation methods of previous\nwork, namely that randomized assignment of samples to training and test data\nsubstantially decreases the complexity of cloze-style reading comprehension.\nAccording to our analysis, replacing the random data split with a chronological\ndata split reduces test accuracy on previous single-variable passage completion\ntask from 72\\% to 34\\%, that leaves much more room to improve. Our proposed\ntasks extend the previous single-variable passage completion task by replacing\nmore character mentions with variables. Several deep learning models are\ndeveloped to validate these three tasks. A thorough error analysis is provided\nto understand the challenges and guide the future direction of this research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00850v1",
            "date": "2019-11-03 08:00:38+00:00",
            "title": "Scene Graph based Image Retrieval -- A case study on the CLEVR Dataset",
            "authors": [
                "Sahana Ramnath",
                "Amrita Saha",
                "Soumen Chakrabarti",
                "Mitesh M. Khapra"
            ],
            "abstract": "With the prolification of multimodal interaction in various domains, recently\nthere has been much interest in text based image retrieval in the computer\nvision community. However most of the state of the art techniques model this\nproblem in a purely neural way, which makes it difficult to incorporate\npragmatic strategies in searching a large scale catalog especially when the\nsearch requirements are insufficient and the model needs to resort to an\ninteractive retrieval process through multiple iterations of\nquestion-answering. Motivated by this, we propose a neural-symbolic approach\nfor a one-shot retrieval of images from a large scale catalog, given the\ncaption description. To facilitate this, we represent the catalog and caption\nas scene-graphs and model the retrieval task as a learnable graph matching\nproblem, trained end-to-end with a REINFORCE algorithm. Further, we briefly\ndescribe an extension of this pipeline to an iterative retrieval framework,\nbased on interactive questioning and answering.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.00891v3",
            "date": "2019-11-03 14:05:55+00:00",
            "title": "Interpreting Verbal Irony: Linguistic Strategies and the Connection to the Type of Semantic Incongruity",
            "authors": [
                "Debanjan Ghosh",
                "Elena Musi",
                "Kartikeya Upasani",
                "Smaranda Muresan"
            ],
            "abstract": "Human communication often involves the use of verbal irony or sarcasm, where\nthe speakers usually mean the opposite of what they say. To better understand\nhow verbal irony is expressed by the speaker and interpreted by the hearer we\nconduct a crowdsourcing task: given an utterance expressing verbal irony, users\nare asked to verbalize their interpretation of the speaker's ironic message. We\npropose a typology of linguistic strategies for verbal irony interpretation and\nlink it to various theoretical linguistic frameworks. We design computational\nmodels to capture these strategies and present empirical studies aimed to\nanswer three questions: (1) what is the distribution of linguistic strategies\nused by hearers to interpret ironic messages?; (2) do hearers adopt similar\nstrategies for interpreting the speaker's ironic intent?; and (3) does the type\nof semantic incongruity in the ironic message (explicit vs. implicit) influence\nthe choice of interpretation strategies by the hearers?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.00949v1",
            "date": "2019-11-03 19:16:51+00:00",
            "title": "Attributed Sequence Embedding",
            "authors": [
                "Zhongfang Zhuang",
                "Xiangnan Kong",
                "Elke Rundensteiner",
                "Jihane Zouaoui",
                "Aditya Arora"
            ],
            "abstract": "Mining tasks over sequential data, such as clickstreams and gene sequences,\nrequire a careful design of embeddings usable by learning algorithms. Recent\nresearch in feature learning has been extended to sequential data, where each\ninstance consists of a sequence of heterogeneous items with a variable length.\nHowever, many real-world applications often involve attributed sequences, where\neach instance is composed of both a sequence of categorical items and a set of\nattributes. In this paper, we study this new problem of attributed sequence\nembedding, where the goal is to learn the representations of attributed\nsequences in an unsupervised fashion. This problem is core to many important\ndata mining tasks ranging from user behavior analysis to the clustering of gene\nsequences. This problem is challenging due to the dependencies between\nsequences and their associated attributes. We propose a deep multimodal\nlearning framework, called NAS, to produce embeddings of attributed sequences.\nThe embeddings are task independent and can be used on various mining tasks of\nattributed sequences. We demonstrate the effectiveness of our embeddings of\nattributed sequences in various unsupervised learning tasks on real-world\ndatasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.00985v1",
            "date": "2019-11-03 22:06:03+00:00",
            "title": "Sentiment analysis model for Twitter data in Polish language",
            "authors": [
                "Karol Chlasta"
            ],
            "abstract": "Text mining analysis of tweets gathered during Polish presidential election\non May 10th, 2015. The project included implementation of engine to retrieve\ninformation from Twitter, building document corpora, corpora cleaning, and\ncreating Term-Document Matrix. Each tweet from the text corpora was assigned a\ncategory based on its sentiment score. The score was calculated using the\nnumber of positive and/or negative emoticons and Polish words in each document.\nThe result data set was used to train and test four machine learning\nclassifiers, to select these providing most accurate automatic tweet\nclassification results. The Naive Bayes and Maximum Entropy algorithms achieved\nthe best accuracy of respectively 71.76% and 77.32%. All implementation tasks\nwere completed using R programming language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01026v1",
            "date": "2019-11-04 04:27:29+00:00",
            "title": "Metric Learning for Dynamic Text Classification",
            "authors": [
                "Jeremy Wohlwend",
                "Ethan R. Elenberg",
                "Samuel Altschul",
                "Shawn Henry",
                "Tao Lei"
            ],
            "abstract": "Traditional text classifiers are limited to predicting over a fixed set of\nlabels. However, in many real-world applications the label set is frequently\nchanging. For example, in intent classification, new intents may be added over\ntime while others are removed. We propose to address the problem of dynamic\ntext classification by replacing the traditional, fixed-size output layer with\na learned, semantically meaningful metric space. Here the distances between\ntextual inputs are optimized to perform nearest-neighbor classification across\noverlapping label sets. Changing the label set does not involve removing\nparameters, but rather simply adding or removing support points in the metric\nspace. Then the learned metric can be fine-tuned with only a few additional\ntraining examples. We demonstrate that this simple strategy is robust to\nchanges in the label space. Furthermore, our results show that learning a\nnon-Euclidean metric can improve performance in the low data regime, suggesting\nthat further work on metric spaces may benefit low-resource research.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.01102v1",
            "date": "2019-11-04 10:07:13+00:00",
            "title": "What does a network layer hear? Analyzing hidden representations of end-to-end ASR through speech synthesis",
            "authors": [
                "Chung-Yi Li",
                "Pei-Chieh Yuan",
                "Hung-Yi Lee"
            ],
            "abstract": "End-to-end speech recognition systems have achieved competitive results\ncompared to traditional systems. However, the complex transformations involved\nbetween layers given highly variable acoustic signals are hard to analyze. In\nthis paper, we present our ASR probing model, which synthesizes speech from\nhidden representations of end-to-end ASR to examine the information maintain\nafter each layer calculation. Listening to the synthesized speech, we observe\ngradual removal of speaker variability and noise as the layer goes deeper,\nwhich aligns with the previous studies on how deep network functions in speech\nrecognition. This paper is the first study analyzing the end-to-end speech\nrecognition model by demonstrating what each layer hears. Speaker verification\nand speech enhancement measurements on synthesized speech are also conducted to\nconfirm our observation further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01196v1",
            "date": "2019-11-04 18:36:12+00:00",
            "title": "Spherical Text Embedding",
            "authors": [
                "Yu Meng",
                "Jiaxin Huang",
                "Guangyuan Wang",
                "Chao Zhang",
                "Honglei Zhuang",
                "Lance Kaplan",
                "Jiawei Han"
            ],
            "abstract": "Unsupervised text embedding has shown great power in a wide range of NLP\ntasks. While text embeddings are typically learned in the Euclidean space,\ndirectional similarity is often more effective in tasks such as word similarity\nand document clustering, which creates a gap between the training stage and\nusage stage of text embedding. To close this gap, we propose a spherical\ngenerative model based on which unsupervised word and paragraph embeddings are\njointly learned. To learn text embeddings in the spherical space, we develop an\nefficient optimization algorithm with convergence guarantee based on Riemannian\noptimization. Our model enjoys high efficiency and achieves state-of-the-art\nperformances on various text embedding tasks including word similarity and\ndocument clustering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01198v1",
            "date": "2019-10-29 12:39:37+00:00",
            "title": "Understand customer reviews with less data and in short time: pretrained language representation and active learning",
            "authors": [
                "Yanwei Cui",
                "Xavier Illy"
            ],
            "abstract": "In this paper, we address customer review understanding problems by using\nsupervised machine learning approaches, in order to achieve a fully automatic\nreview aspects categorisation and sentiment analysis. In general, such\nsupervised learning algorithms require domain-specific expert knowledge for\ngenerating high quality labeled training data, and the cost of labeling can be\nvery high. To achieve an in-production customer review machine learning enabled\nanalysis tool with only a limited amount of data and within a reasonable\ntraining data collection time, we propose to use pre-trained language\nrepresentation to boost model performance and active learning framework for\naccelerating the iterative training process. The results show that with\nintegration of both components, the fully automatic review analysis can be\nachieved at a much faster pace.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01208v3",
            "date": "2019-10-30 23:47:25+00:00",
            "title": "Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship",
            "authors": [
                "Alon Kipnis"
            ],
            "abstract": "We adapt the Higher Criticism (HC) goodness-of-fit test to measure closeness\nbetween word-frequency tables. We apply this measure to authorship attribution\nchallenges, where the goal is to identify the author of a document using other\ndocuments whose authorship is known. The method is simple yet performs well\nwithout handcrafting and tuning; reporting accuracy at the state of the art\nlevel in various current challenges. As an inherent side effect, the HC\ncalculation identifies a subset of discriminating words. In practice, the\nidentified words have low variance across documents belonging to a corpus of\nhomogeneous authorship. We conclude that in comparing the similarity of a new\ndocument and a corpus of a single author, HC is mostly affected by words\ncharacteristic of the author and is relatively unaffected by topic structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01212v1",
            "date": "2019-10-30 12:22:37+00:00",
            "title": "Ordering Matters: Word Ordering Aware Unsupervised NMT",
            "authors": [
                "Tamali Banerjee",
                "Rudra Murthy V",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Denoising-based Unsupervised Neural Machine Translation (U-NMT) models\ntypically employ denoising strategy at the encoder module to prevent the model\nfrom memorizing the input source sentence. Specifically, given an input\nsentence of length n, the model applies n/2 random swaps between consecutive\nwords and trains the denoising-based U-NMT model. Though effective, applying\ndenoising strategy on every sentence in the training data leads to uncertainty\nin the model thereby, limiting the benefits from the denoising-based U-NMT\nmodel. In this paper, we propose a simple fine-tuning strategy where we\nfine-tune the trained denoising-based U-NMT system without the denoising\nstrategy. The input sentences are presented as is i.e., without any shuffling\nnoise added. We observe significant improvements in translation performance on\nmany language pairs from our fine-tuning strategy. Our analysis reveals that\nour proposed models lead to increase in higher n-gram BLEU score compared to\nthe denoising U-NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01217v1",
            "date": "2019-10-29 01:42:22+00:00",
            "title": "Detect Toxic Content to Improve Online Conversations",
            "authors": [
                "Deepshi Mediratta",
                "Nikhil Oswal"
            ],
            "abstract": "Social media is filled with toxic content. The aim of this paper is to build\na model that can detect insincere questions. We use the 'Quora Insincere\nQuestions Classification' dataset for our analysis. The dataset is composed of\nsincere and insincere questions, with the majority of sincere questions. The\ndataset is processed and analyzed using Python and its libraries such as\nsklearn, numpy, pandas, keras etc. The dataset is converted to vector form\nusing word embeddings such as GloVe, Wiki-news and TF-IDF. The imbalance in the\ndataset is handled by resampling techniques. We train and compare various\nmachine learning and deep learning models to come up with the best results.\nModels discussed include SVM, Naive Bayes, GRU and LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01226v2",
            "date": "2019-10-31 22:09:19+00:00",
            "title": "Human-centric Metric for Accelerating Pathology Reports Annotation",
            "authors": [
                "Ruibin Ma",
                "Po-Hsuan Cameron Chen",
                "Gang Li",
                "Wei-Hung Weng",
                "Angela Lin",
                "Krishna Gadepalli",
                "Yuannan Cai"
            ],
            "abstract": "Pathology reports contain useful information such as the main involved organ,\ndiagnosis, etc. These information can be identified from the free text reports\nand used for large-scale statistical analysis or serve as annotation for other\nmodalities such as pathology slides images. However, manual classification for\na huge number of reports on multiple tasks is labor-intensive. In this paper,\nwe have developed an automatic text classifier based on BERT and we propose a\nhuman-centric metric to evaluate the model. According to the model confidence,\nwe identify low-confidence cases that require further expert annotation and\nhigh-confidence cases that are automatically classified. We report the\npercentage of low-confidence cases and the performance of automatically\nclassified cases. On the high-confidence cases, the model achieves\nclassification accuracy comparable to pathologists. This leads a potential of\nreducing 80% to 98% of the manual annotation workload.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01371v1",
            "date": "2019-11-04 17:59:18+00:00",
            "title": "Affective Behaviour Analysis of On-line User Interactions: Are On-line Support Groups more Therapeutic than Twitter?",
            "authors": [
                "Giuliano Tortoreto",
                "Evgeny A. Stepanov",
                "Alessandra Cervone",
                "Mateusz Dubiel",
                "Giuseppe Riccardi"
            ],
            "abstract": "The increase in the prevalence of mental health problems has coincided with a\ngrowing popularity of health related social networking sites. Regardless of\ntheir therapeutic potential, On-line Support Groups (OSGs) can also have\nnegative effects on patients. In this work we propose a novel methodology to\nautomatically verify the presence of therapeutic factors in social networking\nwebsites by using Natural Language Processing (NLP) techniques. The methodology\nis evaluated on On-line asynchronous multi-party conversations collected from\nan OSG and Twitter. The results of the analysis indicate that therapeutic\nfactors occur more frequently in OSG conversations than in Twitter\nconversations. Moreover, the analysis of OSG conversations reveals that the\nusers of that platform are supportive, and interactions are likely to lead to\nthe improvement of their emotional state. We believe that our method provides a\nstepping stone towards automatic analysis of emotional states of users of\nonline platforms. Possible applications of the method include provision of\nguidelines that highlight potential implications of using such platforms on\nusers' mental health, and/or support in the analysis of their impact on\nspecific individuals.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.01474v1",
            "date": "2019-11-04 20:21:32+00:00",
            "title": "VASTA: A Vision and Language-assisted Smartphone Task Automation System",
            "authors": [
                "Alborz Rezazadeh Sereshkeh",
                "Gary Leung",
                "Krish Perumal",
                "Caleb Phillips",
                "Minfan Zhang",
                "Afsaneh Fazly",
                "Iqbal Mohomed"
            ],
            "abstract": "We present VASTA, a novel vision and language-assisted Programming By\nDemonstration (PBD) system for smartphone task automation. Development of a\nrobust PBD automation system requires overcoming three key challenges: first,\nhow to make a particular demonstration robust to positional and visual changes\nin the user interface (UI) elements; secondly, how to recognize changes in the\nautomation parameters to make the demonstration as generalizable as possible;\nand thirdly, how to recognize from the user utterance what automation the user\nwishes to carry out. To address the first challenge, VASTA leverages\nstate-of-the-art computer vision techniques, including object detection and\noptical character recognition, to accurately label interactions demonstrated by\na user, without relying on the underlying UI structures. To address the second\nand third challenges, VASTA takes advantage of advanced natural language\nunderstanding algorithms for analyzing the user utterance to trigger the VASTA\nautomation scripts, and to determine the automation parameters for\ngeneralization. We run an initial user study that demonstrates the\neffectiveness of VASTA at clustering user utterances, understanding changes in\nthe automation parameters, detecting desired UI elements, and, most\nimportantly, automating various tasks. A demo video of the system is available\nhere: http://y2u.be/kr2xE-FixjI",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.01599v1",
            "date": "2019-11-05 03:49:20+00:00",
            "title": "LIDA: Lightweight Interactive Dialogue Annotator",
            "authors": [
                "Edward Collins",
                "Nikolai Rozanov",
                "Bingbing Zhang"
            ],
            "abstract": "Dialogue systems have the potential to change how people interact with\nmachines but are highly dependent on the quality of the data used to train\nthem. It is therefore important to develop good dialogue annotation tools which\ncan improve the speed and quality of dialogue data annotation. With this in\nmind, we introduce LIDA, an annotation tool designed specifically for\nconversation data. As far as we know, LIDA is the first dialogue annotation\nsystem that handles the entire dialogue annotation pipeline from raw text, as\nmay be the output of transcription services, to structured conversation data.\nFurthermore it supports the integration of arbitrary machine learning models as\nannotation recommenders and also has a dedicated interface to resolve\ninter-annotator disagreements such as after crowdsourcing annotations for a\ndataset. LIDA is fully open source, documented and publicly available [\nhttps://github.com/Wluper/lida ]",
            "journal ref": "ACL, EMNLP(D19-3021), 121--126, (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01600v1",
            "date": "2019-11-05 03:50:16+00:00",
            "title": "Integrating Dictionary Feature into A Deep Learning Model for Disease Named Entity Recognition",
            "authors": [
                "Hamada A. Nayel",
                "Shashrekha H. L"
            ],
            "abstract": "In recent years, Deep Learning (DL) models are becoming important due to\ntheir demonstrated success at overcoming complex learning problems. DL models\nhave been applied effectively for different Natural Language Processing (NLP)\ntasks such as part-of-Speech (PoS) tagging and Machine Translation (MT).\nDisease Named Entity Recognition (Disease-NER) is a crucial task which aims at\nextracting disease Named Entities (NEs) from text. In this paper, a DL model\nfor Disease-NER using dictionary information is proposed and evaluated on\nNational Center for Biotechnology Information (NCBI) disease corpus and BC5CDR\ndataset. Word embeddings trained over general domain texts as well as\nbiomedical texts have been used to represent input to the proposed model. This\nstudy also compares two different Segment Representation (SR) schemes, namely\nIOB2 and IOBES for Disease-NER. The results illustrate that using dictionary\ninformation, pre-trained word embeddings, character embeddings and CRF with\nglobal score improves the performance of Disease-NER system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01629v2",
            "date": "2019-11-05 05:46:52+00:00",
            "title": "RNN-T For Latency Controlled ASR With Improved Beam Search",
            "authors": [
                "Mahaveer Jain",
                "Kjell Schubert",
                "Jay Mahadeokar",
                "Ching-Feng Yeh",
                "Kaustubh Kalgaonkar",
                "Anuroop Sriram",
                "Christian Fuegen",
                "Michael L. Seltzer"
            ],
            "abstract": "Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic\nspeech recognition (ASR) blend the individual components of a traditional\nhybrid ASR systems (acoustic model, language model, punctuation model, inverse\ntext normalization) into one single model. This greatly simplifies training and\ninference and hence makes RNN-T a desirable choice for ASR systems. In this\nwork, we investigate use of RNN-T in applications that require a tune-able\nlatency budget during inference time. We also improved the decoding speed of\nthe originally proposed RNN-T beam search algorithm. We evaluated our proposed\nsystem on English videos ASR dataset and show that neural RNN-T models can\nachieve comparable WER and better computational efficiency compared to a well\ntuned hybrid ASR baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01702v2",
            "date": "2019-11-05 10:42:08+00:00",
            "title": "DocParser: Hierarchical Structure Parsing of Document Renderings",
            "authors": [
                "Johannes Rausch",
                "Octavio Martinez",
                "Fabian Bissig",
                "Ce Zhang",
                "Stefan Feuerriegel"
            ],
            "abstract": "Translating renderings (e. g. PDFs, scans) into hierarchical document\nstructures is extensively demanded in the daily routines of many real-world\napplications. However, a holistic, principled approach to inferring the\ncomplete hierarchical structure of documents is missing. As a remedy, we\ndeveloped \"DocParser\": an end-to-end system for parsing the complete document\nstructure - including all text elements, nested figures, tables, and table cell\nstructures. Our second contribution is to provide a dataset for evaluating\nhierarchical document structure parsing. Our third contribution is to propose a\nscalable learning framework for settings where domain-specific data are scarce,\nwhich we address by a novel approach to weak supervision that significantly\nimproves the document structure parsing performance. Our experiments confirm\nthe effectiveness of our proposed weak supervision: Compared to the baseline\nwithout weak supervision, it improves the mean average precision for detecting\ndocument entities by 39.1 % and improves the F1 score of classifying\nhierarchical relations by 35.8 %.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.01799v1",
            "date": "2019-10-31 08:25:45+00:00",
            "title": "CN-CELEB: a challenging Chinese speaker recognition dataset",
            "authors": [
                "Yue Fan",
                "Jiawen Kang",
                "Lantian Li",
                "Kaicheng Li",
                "Haolin Chen",
                "Sitong Cheng",
                "Pengyuan Zhang",
                "Ziya Zhou",
                "Yunqi Cai",
                "Dong Wang"
            ],
            "abstract": "Recently, researchers set an ambitious goal of conducting speaker recognition\nin unconstrained conditions where the variations on ambient, channel and\nemotion could be arbitrary. However, most publicly available datasets are\ncollected under constrained environments, i.e., with little noise and limited\nchannel variation. These datasets tend to deliver over optimistic performance\nand do not meet the request of research on speaker recognition in unconstrained\nconditions. In this paper, we present CN-Celeb, a large-scale speaker\nrecognition dataset collected `in the wild'. This dataset contains more than\n130,000 utterances from 1,000 Chinese celebrities, and covers 11 different\ngenres in real world. Experiments conducted with two state-of-the-art speaker\nrecognition approaches (i-vector and x-vector) show that the performance on\nCN-Celeb is far inferior to the one obtained on VoxCeleb, a widely used speaker\nrecognition dataset. This result demonstrates that in real-life conditions, the\nperformance of existing techniques might be much worse than it was thought. Our\ndatabase is free for researchers and can be downloaded from\nhttp://project.cslt.org.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.01933v1",
            "date": "2019-11-05 16:48:30+00:00",
            "title": "Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)",
            "authors": [
                "Amelia Drew",
                "Alexander Heinecke"
            ],
            "abstract": "We implement a Tensor Train layer in the TensorFlow Neural Machine\nTranslation (NMT) model using the t3f library. We perform training runs on the\nIWSLT English-Vietnamese '15 and WMT German-English '16 datasets with learning\nrates $\\in \\{0.0004,0.0008,0.0012\\}$, maximum ranks $\\in \\{2,4,8,16\\}$ and a\nrange of core dimensions. We compare against a target BLEU test score of 24.0,\nobtained by our benchmark run. For the IWSLT English-Vietnamese training, we\nobtain BLEU test/dev scores of 24.0/21.9 and 24.2/21.9 using core dimensions\n$(2, 2, 256) \\times (2, 2, 512)$ with learning rate 0.0012 and rank\ndistributions $(1,4,4,1)$ and $(1,4,16,1)$ respectively. These runs use 113\\%\nand 397\\% of the flops of the benchmark run respectively. We find that, of the\nparameters surveyed, a higher learning rate and more `rectangular' core\ndimensions generally produce higher BLEU scores. For the WMT German-English\ndataset, we obtain BLEU scores of 24.0/23.8 using core dimensions $(4, 4, 128)\n\\times (4, 4, 256)$ with learning rate 0.0012 and rank distribution\n$(1,2,2,1)$. We discuss the potential for future optimization and application\nof Tensor Train decomposition to other NMT models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.02086v2",
            "date": "2019-11-05 21:13:19+00:00",
            "title": "Small-Footprint Keyword Spotting on Raw Audio Data with Sinc-Convolutions",
            "authors": [
                "Simon Mittermaier",
                "Ludwig K\u00fcrzinger",
                "Bernd Waschneck",
                "Gerhard Rigoll"
            ],
            "abstract": "Keyword Spotting (KWS) enables speech-based user interaction on smart\ndevices. Always-on and battery-powered application scenarios for smart devices\nput constraints on hardware resources and power consumption, while also\ndemanding high accuracy as well as real-time capability. Previous architectures\nfirst extracted acoustic features and then applied a neural network to classify\nkeyword probabilities, optimizing towards memory footprint and execution time.\nCompared to previous publications, we took additional steps to reduce power and\nmemory consumption without reducing classification accuracy. Power-consuming\naudio preprocessing and data transfer steps are eliminated by directly\nclassifying from raw audio. For this, our end-to-end architecture extracts\nspectral features using parametrized Sinc-convolutions. Its memory footprint is\nfurther reduced by grouping depthwise separable convolutions. Our network\nachieves the competitive accuracy of 96.4% on Google's Speech Commands test set\nwith only 62k parameters.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.02103v1",
            "date": "2019-11-05 21:49:55+00:00",
            "title": "Recurrent Instance Segmentation using Sequences of Referring Expressions",
            "authors": [
                "Alba Herrera-Palacio",
                "Carles Ventura",
                "Carina Silberer",
                "Ionut-Teodor Sorodoc",
                "Gemma Boleda",
                "Xavier Giro-i-Nieto"
            ],
            "abstract": "The goal of this work is to segment the objects in an image that are referred\nto by a sequence of linguistic descriptions (referring expressions). We propose\na deep neural network with recurrent layers that output a sequence of binary\nmasks, one for each referring expression provided by the user. The recurrent\nlayers in the architecture allow the model to condition each predicted mask on\nthe previous ones, from a spatial perspective within the same image. Our\nmultimodal approach uses off-the-shelf architectures to encode both the image\nand the referring expressions. The visual branch provides a tensor of pixel\nembeddings that are concatenated with the phrase embeddings produced by a\nlanguage encoder. Our experiments on the RefCOCO dataset for still images\nindicate how the proposed architecture successfully exploits the sequences of\nreferring expressions to solve a pixel-wise task of instance segmentation.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.02133v1",
            "date": "2019-11-05 23:23:58+00:00",
            "title": "Contextual Grounding of Natural Language Entities in Images",
            "authors": [
                "Farley Lai",
                "Ning Xie",
                "Derek Doran",
                "Asim Kadav"
            ],
            "abstract": "In this paper, we introduce a contextual grounding approach that captures the\ncontext in corresponding text entities and image regions to improve the\ngrounding accuracy. Specifically, the proposed architecture accepts pre-trained\ntext token embeddings and image object features from an off-the-shelf object\ndetector as input. Additional encoding to capture the positional and spatial\ninformation can be added to enhance the feature quality. There are separate\ntext and image branches facilitating respective architectural refinements for\ndifferent modalities. The text branch is pre-trained on a large-scale masked\nlanguage modeling task while the image branch is trained from scratch. Next,\nthe model learns the contextual representations of the text tokens and image\nobjects through layers of high-order interaction respectively. The final\ngrounding head ranks the correspondence between the textual and visual\nrepresentations through cross-modal interaction. In the evaluation, we show\nthat our model achieves the state-of-the-art grounding accuracy of 71.36% over\nthe Flickr30K Entities dataset. No additional pre-training is necessary to\ndeliver competitive results compared with related work that often requires\ntask-agnostic and task-specific pre-training on cross-modal dadasets. The\nimplementation is publicly available at https://gitlab.com/necla-ml/grounding.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.02150v1",
            "date": "2019-11-06 00:19:05+00:00",
            "title": "Fast Transformer Decoding: One Write-Head is All You Need",
            "authors": [
                "Noam Shazeer"
            ],
            "abstract": "Multi-head attention layers, as used in the Transformer neural sequence\nmodel, are a powerful alternative to RNNs for moving information across and\nbetween sequences. While training these layers is generally fast and simple,\ndue to parallelizability across the length of the sequence, incremental\ninference (where such paralleization is impossible) is often slow, due to the\nmemory-bandwidth cost of repeatedly loading the large \"keys\" and \"values\"\ntensors. We propose a variant called multi-query attention, where the keys and\nvalues are shared across all of the different attention \"heads\", greatly\nreducing the size of these tensors and hence the memory bandwidth requirements\nof incremental decoding. We verify experimentally that the resulting models can\nindeed be much faster to decode, and incur only minor quality degradation from\nthe baseline.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/1911.02242v1",
            "date": "2019-11-06 08:01:16+00:00",
            "title": "A comparison of end-to-end models for long-form speech recognition",
            "authors": [
                "Chung-Cheng Chiu",
                "Wei Han",
                "Yu Zhang",
                "Ruoming Pang",
                "Sergey Kishchenko",
                "Patrick Nguyen",
                "Arun Narayanan",
                "Hank Liao",
                "Shuyuan Zhang",
                "Anjuli Kannan",
                "Rohit Prabhavalkar",
                "Zhifeng Chen",
                "Tara Sainath",
                "Yonghui Wu"
            ],
            "abstract": "End-to-end automatic speech recognition (ASR) models, including both\nattention-based models and the recurrent neural network transducer (RNN-T),\nhave shown superior performance compared to conventional systems. However,\nprevious studies have focused primarily on short utterances that typically last\nfor just a few seconds or, at most, a few tens of seconds. Whether such\narchitectures are practical on long utterances that last from minutes to hours\nremains an open question. In this paper, we both investigate and improve the\nperformance of end-to-end models on long-form transcription. We first present\nan empirical comparison of different end-to-end models on a real world\nlong-form task and demonstrate that the RNN-T model is much more robust than\nattention-based systems in this regime. We next explore two improvements to\nattention-based systems that significantly improve its performance: restricting\nthe attention to be monotonic, and applying a novel decoding algorithm that\nbreaks long utterances into shorter overlapping segments. Combining these two\nimprovements, we show that attention-based end-to-end models can be very\ncompetitive to RNN-T on long-form speech recognition.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.02247v2",
            "date": "2019-11-06 08:20:13+00:00",
            "title": "Unsupervised Opinion Summarization as Copycat-Review Generation",
            "authors": [
                "Arthur Bra\u017einskas",
                "Mirella Lapata",
                "Ivan Titov"
            ],
            "abstract": "Opinion summarization is the task of automatically creating summaries that\nreflect subjective information expressed in multiple documents, such as product\nreviews. While the majority of previous work has focused on the extractive\nsetting, i.e., selecting fragments from input reviews to produce a summary, we\nlet the model generate novel sentences and hence produce abstractive summaries.\nRecent progress in summarization has seen the development of supervised models\nwhich rely on large quantities of document-summary pairs. Since such training\ndata is expensive to acquire, we instead consider the unsupervised setting, in\nother words, we do not use any summaries in training. We define a generative\nmodel for a review collection which capitalizes on the intuition that when\ngenerating a new review given a set of other reviews of a product, we should be\nable to control the \"amount of novelty\" going into the new review or,\nequivalently, vary the extent to which it deviates from the input. At test\ntime, when generating summaries, we force the novelty to be minimal, and\nproduce a text reflecting consensus opinions. We capture this intuition by\ndefining a hierarchical variational autoencoder model. Both individual reviews\nand the products they correspond to are associated with stochastic latent\ncodes, and the review generator (\"decoder\") has direct access to the text of\ninput reviews through the pointer-generator mechanism. Experiments on Amazon\nand Yelp datasets, show that setting at test time the review's latent code to\nits mean, allows the model to produce fluent and coherent summaries reflecting\ncommon opinions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02365v1",
            "date": "2019-11-06 13:23:41+00:00",
            "title": "Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds",
            "authors": [
                "Tassilo Klein",
                "Moin Nabi"
            ],
            "abstract": "Automatic question generation aims at the generation of questions from a\ncontext, with the corresponding answers being sub-spans of the given passage.\nWhereas, most of the methods mostly rely on heuristic rules to generate\nquestions, more recently also neural network approaches have been proposed. In\nthis work, we propose a variant of the self-attention Transformer network\narchitectures model to generate meaningful and diverse questions. To this end,\nwe propose an easy to use model consisting of the conjunction of the\nTransformer decoder GPT-2 model with Transformer encoder BERT for the\ndownstream task for question answering. The model is trained in an end-to-end\nfashion, where the language model is trained to produce a question-answer-aware\ninput representation that facilitates to generate an answer focused question.\nOur result of neural question generation from text on the SQuAD 1.1 dataset\nsuggests that our method can produce semantically correct and diverse\nquestions. Additionally, we assessed the performance of our proposed method for\nthe downstream task of question answering. The analysis shows that our proposed\ngeneration & answering collaboration framework relatively improves both tasks\nand is particularly powerful in the semi-supervised setup. The results further\nsuggest a robust and comparably lean pipeline facilitating question generation\nin the small-data regime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02471v1",
            "date": "2019-11-06 16:38:19+00:00",
            "title": "Designing Evaluations of Machine Learning Models for Subjective Inference: The Case of Sentence Toxicity",
            "authors": [
                "Agathe Balayn",
                "Alessandro Bozzon"
            ],
            "abstract": "Machine Learning (ML) is increasingly applied in real-life scenarios, raising\nconcerns about bias in automatic decision making. We focus on bias as a notion\nof opinion exclusion, that stems from the direct application of traditional ML\npipelines to infer subjective properties. We argue that such ML systems should\nbe evaluated with subjectivity and bias in mind. Considering the lack of\nevaluation standards yet to create evaluation benchmarks, we propose an initial\nlist of specifications to define prior to creating evaluation datasets, in\norder to later accurately evaluate the biases. With the example of a sentence\ntoxicity inference system, we illustrate how the specifications support the\nanalysis of biases related to subjectivity. We highlight difficulties in\ninstantiating these specifications and list future work for the crowdsourcing\ncommunity to help the creation of appropriate evaluation datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.02499v1",
            "date": "2019-11-06 17:16:26+00:00",
            "title": "Toward Dimensional Emotion Detection from Categorical Emotion Annotations",
            "authors": [
                "Sungjoon Park",
                "Jiseon Kim",
                "Jaeyeol Jeon",
                "Heeyoung Park",
                "Alice Oh"
            ],
            "abstract": "We propose a framework which makes a model predict fine-grained dimensional\nemotions (valence-arousal-dominance, VAD) trained on corpus annotated with\ncoarse-grained categorical emotions. We train a model by minimizing EMD\ndistances between predicted VAD score distribution and \\textit{sorted}\ncategorical emotion distributions in terms of VAD, as a proxy of target VAD\nscore distributions. With our model, we can simultaneously classify a given\nsentence to categorical emotions as well as predict VAD scores. We use\npre-trained BERT-Large and fine-tune on SemEval dataset (11 categorical\nemotions) and evaluate on EmoBank (VAD dimensional emotions), in order to show\nour approach reaches comparable performance to that of the state-of-the-art\nclassifiers in categorical emotion classification task and significant positive\ncorrelations with ground truth VAD scores. Also, if one continues training our\nmodel with supervision of VAD labels, it outperforms state-of-the-art VAD\nregression models. We further present examples showing our model can annotate\nemotional words suitable for a given text even those words are not seen as\ncategorical labels during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02524v1",
            "date": "2019-11-06 18:05:13+00:00",
            "title": "A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World",
            "authors": [
                "Georgiy Platonov",
                "Benjamin Kane",
                "Aaron Gindi",
                "Lenhart K. Schubert"
            ],
            "abstract": "The blocks world is a classic toy domain that has long been used to build and\ntest spatial reasoning systems. Despite its relative simplicity, tackling this\ndomain in its full complexity requires the agent to exhibit a rich set of\nfunctional capabilities, ranging from vision to natural language understanding.\nThere is currently a resurgence of interest in solving problems in such limited\ndomains using modern techniques. In this work we tackle spatial question\nanswering in a holistic way, using a vision system, speech input and output\nmediated by an animated avatar, a dialogue system that robustly interprets\nspatial queries, and a constraint solver that derives answers based on 3-D\nspatial modeling. The contributions of this work include a semantic parser that\nmaps spatial questions into logical forms consistent with a general approach to\nmeaning representation, a dialog manager based on a schema representation, and\na constraint solver for spatial questions that provides answers in agreement\nwith human perception. These and other components are integrated into a\nmulti-modal human-computer interaction pipeline.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.02648v2",
            "date": "2019-10-21 16:36:51+00:00",
            "title": "Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome",
            "authors": [
                "Philippe Vincent-Lamarre",
                "Vincent Larivi\u00e8re"
            ],
            "abstract": "We analysed a dataset of scientific manuscripts that were submitted to\nvarious conferences in artificial intelligence. We performed a combination of\nsemantic, lexical and psycholinguistic analyses of the full text of the\nmanuscripts and compared them with the outcome of the peer review process. We\nfound that accepted manuscripts scored lower than rejected manuscripts on two\nindicators of readability, and that they also used more scientific and\nartificial intelligence jargon. We also found that accepted manuscripts were\nwritten with words that are less frequent, that are acquired at an older age,\nand that are more abstract than rejected manuscripts. The analysis of\nreferences included in the manuscripts revealed that the subset of accepted\nsubmissions were more likely to cite the same publications. This finding was\nechoed by pairwise comparisons of the word content of the manuscripts (i.e. an\nindicator or semantic similarity), which were more similar in the subset of\naccepted manuscripts. Finally, we predicted the peer review outcome of\nmanuscripts with their word content, with words related to machine learning and\nneural networks positively related with acceptance, whereas words related to\nlogic, symbolic processing and knowledge-based systems negatively related with\nacceptance.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02656v1",
            "date": "2019-11-06 22:41:04+00:00",
            "title": "Invariance and identifiability issues for word embeddings",
            "authors": [
                "Rachel Carrington",
                "Karthik Bharath",
                "Simon Preston"
            ],
            "abstract": "Word embeddings are commonly obtained as optimizers of a criterion function\n$f$ of a text corpus, but assessed on word-task performance using a different\nevaluation function $g$ of the test data. We contend that a possible source of\ndisparity in performance on tasks is the incompatibility between classes of\ntransformations that leave $f$ and $g$ invariant. In particular, word\nembeddings defined by $f$ are not unique; they are defined only up to a class\nof transformations to which $f$ is invariant, and this class is larger than the\nclass to which $g$ is invariant. One implication of this is that the apparent\nsuperiority of one word embedding over another, as measured by word task\nperformance, may largely be a consequence of the arbitrary elements selected\nfrom the respective solution sets. We provide a formal treatment of the above\nidentifiability issue, present some numerical examples, and discuss possible\nresolutions.",
            "journal ref": "None",
            "category": "stat.ML"
        },
        {
            "url": "http://arxiv.org/abs/1911.02739v3",
            "date": "2019-11-07 03:28:38+00:00",
            "title": "DCA: Diversified Co-Attention towards Informative Live Video Commenting",
            "authors": [
                "Zhihan Zhang",
                "Zhiyi Yin",
                "Shuhuai Ren",
                "Xinhang Li",
                "Shicheng Li"
            ],
            "abstract": "We focus on the task of Automatic Live Video Commenting (ALVC), which aims to\ngenerate real-time video comments with both video frames and other viewers'\ncomments as inputs. A major challenge in this task is how to properly leverage\nthe rich and diverse information carried by video and text. In this paper, we\naim to collect diversified information from video and text for informative\ncomment generation. To achieve this, we propose a Diversified Co-Attention\n(DCA) model for this task. Our model builds bidirectional interactions between\nvideo frames and surrounding comments from multiple perspectives via metric\nlearning, to collect a diversified and informative context for comment\ngeneration. We also propose an effective parameter orthogonalization technique\nto avoid excessive overlap of information learned from different perspectives.\nResults show that our approach outperforms existing methods in the ALVC task,\nachieving new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.02750v3",
            "date": "2019-11-07 04:22:54+00:00",
            "title": "Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework",
            "authors": [
                "Mingbo Ma",
                "Baigong Zheng",
                "Kaibo Liu",
                "Renjie Zheng",
                "Hairong Liu",
                "Kainan Peng",
                "Kenneth Church",
                "Liang Huang"
            ],
            "abstract": "Text-to-speech synthesis (TTS) has witnessed rapid progress in recent years,\nwhere neural methods became capable of producing audios with high naturalness.\nHowever, these efforts still suffer from two types of latencies: (a) the {\\em\ncomputational latency} (synthesizing time), which grows linearly with the\nsentence length even with parallel approaches, and (b) the {\\em input latency}\nin scenarios where the input text is incrementally generated (such as in\nsimultaneous translation, dialog generation, and assistive technologies). To\nreduce these latencies, we devise the first neural incremental TTS approach\nbased on the recently proposed prefix-to-prefix framework. We synthesize speech\nin an online fashion, playing a segment of audio while generating the next,\nresulting in an $O(1)$ rather than $O(n)$ latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02839v2",
            "date": "2019-11-07 10:47:30+00:00",
            "title": "Teacher-Student Training for Robust Tacotron-based TTS",
            "authors": [
                "Rui Liu",
                "Berrak Sisman",
                "Jingdong Li",
                "Feilong Bao",
                "Guanglai Gao",
                "Haizhou Li"
            ],
            "abstract": "While neural end-to-end text-to-speech (TTS) is superior to conventional\nstatistical methods in many ways, the exposure bias problem in the\nautoregressive models remains an issue to be resolved. The exposure bias\nproblem arises from the mismatch between the training and inference process,\nthat results in unpredictable performance for out-of-domain test data at\nrun-time. To overcome this, we propose a teacher-student training scheme for\nTacotron-based TTS by introducing a distillation loss function in addition to\nthe feature loss function. We first train a Tacotron2-based TTS model by always\nproviding natural speech frames to the decoder, that serves as a teacher model.\nWe then train another Tacotron2-based model as a student model, of which the\ndecoder takes the predicted speech frames as input, similar to how the decoder\nworks during run-time inference. With the distillation loss, the student model\nlearns the output probabilities from the teacher model, that is called\nknowledge distillation. Experiments show that our proposed training scheme\nconsistently improves the voice quality for out-of-domain test data both in\nChinese and English systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.02970v1",
            "date": "2019-11-07 16:21:40+00:00",
            "title": "SENSE: Semantically Enhanced Node Sequence Embedding",
            "authors": [
                "Swati Rallapalli",
                "Liang Ma",
                "Mudhakar Srivatsa",
                "Ananthram Swami",
                "Heesung Kwon",
                "Graham Bent",
                "Christopher Simpkin"
            ],
            "abstract": "Effectively capturing graph node sequences in the form of vector embeddings\nis critical to many applications. We achieve this by (i) first learning vector\nembeddings of single graph nodes and (ii) then composing them to compactly\nrepresent node sequences. Specifically, we propose SENSE-S (Semantically\nEnhanced Node Sequence Embedding - for Single nodes), a skip-gram based novel\nembedding mechanism, for single graph nodes that co-learns graph structure as\nwell as their textual descriptions. We demonstrate that SENSE-S vectors\nincrease the accuracy of multi-label classification tasks by up to 50% and\nlink-prediction tasks by up to 78% under a variety of scenarios using real\ndatasets. Based on SENSE-S, we next propose generic SENSE to compute composite\nvectors that represent a sequence of nodes, where preserving the node order is\nimportant. We prove that this approach is efficient in embedding node\nsequences, and our experiments on real data confirm its high accuracy in node\norder decoding.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.02971v1",
            "date": "2019-11-07 16:34:31+00:00",
            "title": "Probing Contextualized Sentence Representations with Visual Awareness",
            "authors": [
                "Zhuosheng Zhang",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Hai Zhao"
            ],
            "abstract": "We present a universal framework to model contextualized sentence\nrepresentations with visual awareness that is motivated to overcome the\nshortcomings of the multimodal parallel data with manual annotations. For each\nsentence, we first retrieve a diversity of images from a shared cross-modal\nembedding space, which is pre-trained on a large-scale of text-image pairs.\nThen, the texts and images are respectively encoded by transformer encoder and\nconvolutional neural network. The two sequences of representations are further\nfused by a simple and effective attention layer. The architecture can be easily\napplied to text-only natural language processing tasks without manually\nannotating multimodal parallel corpora. We apply the proposed method on three\ntasks, including neural machine translation, natural language inference and\nsequence labeling and experimental results verify the effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03064v3",
            "date": "2019-11-08 05:56:01+00:00",
            "title": "Reducing Sentiment Bias in Language Models via Counterfactual Evaluation",
            "authors": [
                "Po-Sen Huang",
                "Huan Zhang",
                "Ray Jiang",
                "Robert Stanforth",
                "Johannes Welbl",
                "Jack Rae",
                "Vishal Maini",
                "Dani Yogatama",
                "Pushmeet Kohli"
            ],
            "abstract": "Advances in language modeling architectures and the availability of large\ntext corpora have driven progress in automatic text generation. While this\nresults in models capable of generating coherent texts, it also prompts models\nto internalize social biases present in the training corpus. This paper aims to\nquantify and reduce a particular type of bias exhibited by language models:\nbias in the sentiment of generated text. Given a conditioning context (e.g., a\nwriting prompt) and a language model, we analyze if (and how) the sentiment of\nthe generated text is affected by changes in values of sensitive attributes\n(e.g., country names, occupations, genders) in the conditioning context using a\nform of counterfactual evaluation. We quantify sentiment bias by adopting\nindividual and group fairness metrics from the fair machine learning\nliterature, and demonstrate that large-scale models trained on two different\ncorpora (news articles, and Wikipedia) exhibit considerable levels of bias. We\nthen propose embedding and sentiment prediction-derived regularization on the\nlanguage model's latent representations. The regularizations improve fairness\nmetrics while retaining comparable levels of perplexity and semantic\nsimilarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03167v3",
            "date": "2019-11-08 10:22:10+00:00",
            "title": "Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates",
            "authors": [
                "Javier Iranzo-S\u00e1nchez",
                "Joan Albert Silvestre-Cerd\u00e0",
                "Javier Jorge",
                "Nahuel Rosell\u00f3",
                "Adri\u00e0 Gim\u00e9nez",
                "Albert Sanchis",
                "Jorge Civera",
                "Alfons Juan"
            ],
            "abstract": "Current research into spoken language translation (SLT),or speech-to-text\ntranslation, is often hampered by the lack of specific data resources for this\ntask, as currently available SLT datasets are restricted to a limited set of\nlanguage pairs. In this paper we present Europarl-ST, a novel multilingual SLT\ncorpus containing paired audio-text samples for SLT from and into 6 European\nlanguages, for a total of 30 different translation directions. This corpus has\nbeen compiled using the debates held in the European Parliament in the period\nbetween 2008 and 2012. This paper describes the corpus creation process and\npresents a series of automatic speech recognition, machine translation and\nspoken language translation experiments that highlight the potential of this\nnew resource. The corpus is released under a Creative Commons license and is\nfreely accessible and downloadable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03219v1",
            "date": "2019-11-08 12:42:22+00:00",
            "title": "Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning",
            "authors": [
                "Nicolas Lair",
                "C\u00e9dric Colas",
                "R\u00e9my Portelas",
                "Jean-Michel Dussoux",
                "Peter Ford Dominey",
                "Pierre-Yves Oudeyer"
            ],
            "abstract": "Autonomous reinforcement learning agents, like children, do not have access\nto predefined goals and reward functions. They must discover potential goals,\nlearn their own reward functions and engage in their own learning trajectory.\nChildren, however, benefit from exposure to language, helping to organize and\nmediate their thought. We propose LE2 (Language Enhanced Exploration), a\nlearning algorithm leveraging intrinsic motivations and natural language (NL)\ninteractions with a descriptive social partner (SP). Using NL descriptions from\nthe SP, it can learn an NL-conditioned reward function to formulate goals for\nintrinsically motivated goal exploration and learn a goal-conditioned policy.\nBy exploring, collecting descriptions from the SP and jointly learning the\nreward function and the policy, the agent grounds NL descriptions into real\nbehavioral goals. From simple goals discovered early to more complex goals\ndiscovered by experimenting on simpler ones, our agent autonomously builds its\nown behavioral repertoire. This naturally occurring curriculum is supplemented\nby an active learning curriculum resulting from the agent's intrinsic\nmotivations. Experiments are presented with a simulated robotic arm that\ninteracts with several objects including tools.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03249v1",
            "date": "2019-11-08 13:30:21+00:00",
            "title": "Advances in Machine Learning for the Behavioral Sciences",
            "authors": [
                "Tom\u00e1\u0161 Kliegr",
                "\u0160t\u011bp\u00e1n Bahn\u00edk",
                "Johannes F\u00fcrnkranz"
            ],
            "abstract": "The areas of machine learning and knowledge discovery in databases have\nconsiderably matured in recent years. In this article, we briefly review recent\ndevelopments as well as classical algorithms that stood the test of time. Our\ngoal is to provide a general introduction into different tasks such as learning\nfrom tabular data, behavioral data, or textual data, with a particular focus on\nactual and potential applications in behavioral sciences. The supplemental\nappendix to the article also provides practical guidance for using the methods\nby pointing the reader to proven software implementations. The focus is on R,\nbut we also cover some libraries in other programming languages as well as\nsystems with easy-to-use graphical interfaces.",
            "journal ref": "American Behavioral Scientist 64.2 (2020): 145-175",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03268v1",
            "date": "2019-10-29 23:28:16+00:00",
            "title": "Inducing brain-relevant bias in natural language processing models",
            "authors": [
                "Dan Schwartz",
                "Mariya Toneva",
                "Leila Wehbe"
            ],
            "abstract": "Progress in natural language processing (NLP) models that estimate\nrepresentations of word sequences has recently been leveraged to improve the\nunderstanding of language processing in the brain. However, these models have\nnot been specifically designed to capture the way the brain represents language\nmeaning. We hypothesize that fine-tuning these models to predict recordings of\nbrain activity of people reading text will lead to representations that encode\nmore brain-activity-relevant language information. We demonstrate that a\nversion of BERT, a recently introduced and powerful language model, can improve\nthe prediction of brain activity after fine-tuning. We show that the\nrelationship between language and brain activity learned by BERT during this\nfine-tuning transfers across multiple participants. We also show that, for some\nparticipants, the fine-tuned representations learned from both\nmagnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI)\nare better for predicting fMRI than the representations learned from fMRI\nalone, indicating that the learned representations capture\nbrain-activity-relevant information that is not simply an artifact of the\nmodality. While changes to language representations help the model predict\nbrain activity, they also do not harm the model's ability to perform downstream\nNLP tasks. Our findings are notable for research on language understanding in\nthe brain.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1911.03329v1",
            "date": "2019-11-08 15:33:51+00:00",
            "title": "Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages",
            "authors": [
                "Mirac Suzgun",
                "Sebastian Gehrmann",
                "Yonatan Belinkov",
                "Stuart M. Shieber"
            ],
            "abstract": "We introduce three memory-augmented Recurrent Neural Networks (MARNNs) and\nexplore their capabilities on a series of simple language modeling tasks whose\nsolutions require stack-based mechanisms. We provide the first demonstration of\nneural networks recognizing the generalized Dyck languages, which express the\ncore of what it means to be a language with hierarchical structure. Our\nmemory-augmented architectures are easy to train in an end-to-end fashion and\ncan learn the Dyck languages over as many as six parenthesis-pairs, in addition\nto two deterministic palindrome languages and the string-reversal transduction\ntask, by emulating pushdown automata. Our experiments highlight the increased\nmodeling capacity of memory-augmented models over simple RNNs, while inflecting\nour understanding of the limitations of these models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03362v2",
            "date": "2019-11-06 17:30:57+00:00",
            "title": "Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation",
            "authors": [
                "Nikolay Bogoychev",
                "Rico Sennrich"
            ],
            "abstract": "The quality of neural machine translation can be improved by leveraging\nadditional monolingual resources to create synthetic training data. Source-side\nmonolingual data can be (forward-)translated into the target language for\nself-training; target-side monolingual data can be back-translated. It has been\nwidely reported that back-translation delivers superior results, but could this\nbe due to artefacts in the test sets? We perform a case study using\nFrench-English news translation task and separate test sets based on their\noriginal languages. We show that forward translation delivers superior gains in\nterms of BLEU on sentences that were originally in the source language,\ncomplementing previous studies which show large improvements with\nback-translation on sentences that were originally in the target language. To\nbetter understand when and why forward and back-translation are effective, we\nstudy the role of domains, translationese, and noise. While translationese\neffects are well known to influence MT evaluation, we also find evidence that\nnews data from different languages shows subtle domain differences, which is\nanother explanation for varying performance on different portions of the test\nset. We perform additional low-resource experiments which demonstrate that\nforward translation is more sensitive to the quality of the initial translation\nsystem than back-translation, and tends to perform worse in low-resource\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03378v1",
            "date": "2019-11-08 16:59:17+00:00",
            "title": "Investigation of Error Simulation Techniques for Learning Dialog Policies for Conversational Error Recovery",
            "authors": [
                "Maryam Fazel-Zarandi",
                "Longshaokan Wang",
                "Aditya Tiwari",
                "Spyros Matsoukas"
            ],
            "abstract": "Training dialog policies for speech-based virtual assistants requires a\nplethora of conversational data. The data collection phase is often expensive\nand time consuming due to human involvement. To address this issue, a common\nsolution is to build user simulators for data generation. For the successful\ndeployment of the trained policies into real world domains, it is vital that\nthe user simulator mimics realistic conditions. In particular, speech-based\nassistants are heavily affected by automatic speech recognition and language\nunderstanding errors, hence the user simulator should be able to simulate\nsimilar errors. In this paper, we review the existing error simulation methods\nthat induce errors at audio, phoneme, text, or semantic level; and conduct\ndetailed comparisons between the audio-level and text-level methods. In the\nprocess, we improve the existing text-level method by introducing confidence\nscore prediction and out-of-vocabulary word mapping. We also explore the impact\nof audio-level and text-level methods on learning a simple clarification dialog\npolicy to recover from errors to provide insight on future improvement for both\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03429v2",
            "date": "2019-11-08 18:29:03+00:00",
            "title": "ERASER: A Benchmark to Evaluate Rationalized NLP Models",
            "authors": [
                "Jay DeYoung",
                "Sarthak Jain",
                "Nazneen Fatema Rajani",
                "Eric Lehman",
                "Caiming Xiong",
                "Richard Socher",
                "Byron C. Wallace"
            ],
            "abstract": "State-of-the-art models in NLP are now predominantly based on deep neural\nnetworks that are opaque in terms of how they come to make predictions. This\nlimitation has increased interest in designing more interpretable deep models\nfor NLP that reveal the `reasoning' behind model outputs. But work in this\ndirection has been conducted on different datasets and tasks with\ncorrespondingly unique aims and metrics; this makes it difficult to track\nprogress. We propose the Evaluating Rationales And Simple English Reasoning\n(ERASER) benchmark to advance research on interpretable models in NLP. This\nbenchmark comprises multiple datasets and tasks for which human annotations of\n\"rationales\" (supporting evidence) have been collected. We propose several\nmetrics that aim to capture how well the rationales provided by models align\nwith human rationales, and also how faithful these rationales are (i.e., the\ndegree to which provided rationales influenced the corresponding predictions).\nOur hope is that releasing this benchmark facilitates progress on designing\nmore interpretable NLP systems. The benchmark, code, and documentation are\navailable at https://www.eraserbenchmark.com/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03437v4",
            "date": "2019-11-08 18:41:31+00:00",
            "title": "SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization",
            "authors": [
                "Haoming Jiang",
                "Pengcheng He",
                "Weizhu Chen",
                "Xiaodong Liu",
                "Jianfeng Gao",
                "Tuo Zhao"
            ],
            "abstract": "Transfer learning has fundamentally changed the landscape of natural language\nprocessing (NLP) research. Many existing state-of-the-art models are first\npre-trained on a large text corpus and then fine-tuned on downstream tasks.\nHowever, due to limited data resources from downstream tasks and the extremely\nlarge capacity of pre-trained models, aggressive fine-tuning often causes the\nadapted model to overfit the data of downstream tasks and forget the knowledge\nof the pre-trained model. To address the above issue in a more principled\nmanner, we propose a new computational framework for robust and efficient\nfine-tuning for pre-trained language models. Specifically, our proposed\nframework contains two important ingredients: 1. Smoothness-inducing\nregularization, which effectively manages the capacity of the model; 2. Bregman\nproximal point optimization, which is a class of trust-region methods and can\nprevent knowledge forgetting. Our experiments demonstrate that our proposed\nmethod achieves the state-of-the-art performance on multiple NLP benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03584v2",
            "date": "2019-11-08 23:48:38+00:00",
            "title": "On the Relationship between Self-Attention and Convolutional Layers",
            "authors": [
                "Jean-Baptiste Cordonnier",
                "Andreas Loukas",
                "Martin Jaggi"
            ],
            "abstract": "Recent trends of incorporating attention mechanisms in vision have led\nresearchers to reconsider the supremacy of convolutional layers as a primary\nbuilding block. Beyond helping CNNs to handle long-range dependencies,\nRamachandran et al. (2019) showed that attention can completely replace\nconvolution and achieve state-of-the-art performance on vision tasks. This\nraises the question: do learned attention layers operate similarly to\nconvolutional layers? This work provides evidence that attention layers can\nperform convolution and, indeed, they often learn to do so in practice.\nSpecifically, we prove that a multi-head self-attention layer with sufficient\nnumber of heads is at least as expressive as any convolutional layer. Our\nnumerical experiments then show that self-attention layers attend to pixel-grid\npatterns similarly to CNN layers, corroborating our analysis. Our code is\npublicly available.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03598v2",
            "date": "2019-11-09 03:05:50+00:00",
            "title": "Interactive Classification by Asking Informative Questions",
            "authors": [
                "Lili Yu",
                "Howard Chen",
                "Sida Wang",
                "Tao Lei",
                "Yoav Artzi"
            ],
            "abstract": "We study the potential for interaction in natural language classification. We\nadd a limited form of interaction for intent classification, where users\nprovide an initial query using natural language, and the system asks for\nadditional information using binary or multi-choice questions. At each turn,\nour system decides between asking the most informative question or making the\nfinal classification prediction.The simplicity of the model allows for\nbootstrapping of the system without interaction data, instead relying on simple\ncrowdsourcing tasks. We evaluate our approach on two domains, showing the\nbenefit of interaction and the advantage of learning to balance between asking\nadditional questions and making the final prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03604v4",
            "date": "2019-11-09 03:29:06+00:00",
            "title": "A Simplified Fully Quantized Transformer for End-to-end Speech Recognition",
            "authors": [
                "Alex Bie",
                "Bharat Venkitesh",
                "Joao Monteiro",
                "Md. Akmal Haidar",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "While significant improvements have been made in recent years in terms of\nend-to-end automatic speech recognition (ASR) performance, such improvements\nwere obtained through the use of very large neural networks, unfit for embedded\nuse on edge devices. That being said, in this paper, we work on simplifying and\ncompressing Transformer-based encoder-decoder architectures for the end-to-end\nASR task. We empirically introduce a more compact Speech-Transformer by\ninvestigating the impact of discarding particular modules on the performance of\nthe model. Moreover, we evaluate reducing the numerical precision of our\nnetwork's weights and activations while maintaining the performance of the\nfull-precision model. Our experiments show that we can reduce the number of\nparameters of the full-precision model and then further compress the model 4x\nby fully quantizing to 8-bit fixed point precision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03614v1",
            "date": "2019-11-09 05:31:40+00:00",
            "title": "Improving Machine Reading Comprehension via Adversarial Training",
            "authors": [
                "Ziqing Yang",
                "Yiming Cui",
                "Wanxiang Che",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Adversarial training (AT) as a regularization method has proved its\neffectiveness in various tasks, such as image classification and text\nclassification. Though there are successful applications of AT in many tasks of\nnatural language processing (NLP), the mechanism behind it is still unclear. In\nthis paper, we aim to apply AT on machine reading comprehension (MRC) and study\nits effects from multiple perspectives. We experiment with three different\nkinds of RC tasks: span-based RC, span-based RC with unanswerable questions and\nmulti-choice RC. The experimental results show that the proposed method can\nimprove the performance significantly and universally on SQuAD1.1, SQuAD2.0 and\nRACE. With virtual adversarial training (VAT), we explore the possibility of\nimproving the RC models with semi-supervised learning and prove that examples\nfrom a different task are also beneficial. We also find that AT helps little in\ndefending against artificial adversarial examples, but AT helps the model to\nlearn better on examples that contain more low-frequency words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03626v2",
            "date": "2019-11-09 06:55:39+00:00",
            "title": "Beyond Statistical Relations: Integrating Knowledge Relations into Style Correlations for Multi-Label Music Style Classification",
            "authors": [
                "Qianwen Ma",
                "Chunyuan Yuan",
                "Wei Zhou",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Automatically labeling multiple styles for every song is a comprehensive\napplication in all kinds of music websites. Recently, some researches explore\nreview-driven multi-label music style classification and exploit style\ncorrelations for this task. However, their methods focus on mining the\nstatistical relations between different music styles and only consider shallow\nstyle relations. Moreover, these statistical relations suffer from the\nunderfitting problem because some music styles have little training data.\n  To tackle these problems, we propose a novel knowledge relations integrated\nframework (KRF) to capture the complete style correlations, which jointly\nexploits the inherent relations between music styles according to external\nknowledge and their statistical relations. Based on the two types of relations,\nwe use a graph convolutional network to learn the deep correlations between\nstyles automatically. Experimental results show that our framework\nsignificantly outperforms state-of-the-art methods. Further studies demonstrate\nthat our framework can effectively alleviate the underfitting problem and learn\nmeaningful style correlations. The source code can be available at\nhttps://github.com/Makwen1995/MusicGenre.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03642v3",
            "date": "2019-11-09 08:43:02+00:00",
            "title": "Towards Understanding Gender Bias in Relation Extraction",
            "authors": [
                "Andrew Gaut",
                "Tony Sun",
                "Shirlyn Tang",
                "Yuxin Huang",
                "Jing Qian",
                "Mai ElSherief",
                "Jieyu Zhao",
                "Diba Mirza",
                "Elizabeth Belding",
                "Kai-Wei Chang",
                "William Yang Wang"
            ],
            "abstract": "Recent developments in Neural Relation Extraction (NRE) have made significant\nstrides towards Automated Knowledge Base Construction (AKBC). While much\nattention has been dedicated towards improvements in accuracy, there have been\nno attempts in the literature to our knowledge to evaluate social biases in NRE\nsystems. We create WikiGenderBias, a distantly supervised dataset with a human\nannotated test set. WikiGenderBias has sentences specifically curated to\nanalyze gender bias in relation extraction systems. We use WikiGenderBias to\nevaluate systems for bias and find that NRE systems exhibit gender biased\npredictions and lay groundwork for future evaluation of bias in NRE. We also\nanalyze how name anonymization, hard debiasing for word embeddings, and\ncounterfactual data augmentation affect gender bias in predictions and\nperformance.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03663v2",
            "date": "2019-11-09 10:55:34+00:00",
            "title": "Style is NOT a single variable: Case Studies for Cross-Style Language Understanding",
            "authors": [
                "Dongyeop Kang",
                "Eduard Hovy"
            ],
            "abstract": "Every natural text is written in some style. Style is formed by a complex\ncombination of different stylistic factors, including formality markers,\nemotions, metaphors, etc. One cannot form a complete understanding of a text\nwithout considering these factors. The factors combine and co-vary in complex\nways to form styles. Studying the nature of the co-varying combinations sheds\nlight on stylistic language in general, sometimes called cross-style language\nunderstanding. This paper provides the benchmark corpus (xSLUE) that combines\nexisting datasets and collects a new one for sentence-level cross-style\nlanguage understanding and evaluation. The benchmark contains text in 15\ndifferent styles under the proposed four theoretical groupings: figurative,\npersonal, affective, and interpersonal groups. For valid evaluation, we collect\nan additional diagnostic set by annotating all 15 styles on the same text.\nUsing xSLUE, we propose three interesting cross-style applications in\nclassification, correlation, and generation. First, our proposed cross-style\nclassifier trained with multiple styles together helps improve overall\nclassification performance against individually-trained style classifiers.\nSecond, our study shows that some styles are highly dependent on each other in\nhuman-written text. Finally, we find that combinations of some contradictive\nstyles likely generate stylistically less appropriate text. We believe our\nbenchmark and case studies help explore interesting future directions for\ncross-style research. The preprocessed datasets and code are publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03698v1",
            "date": "2019-11-09 14:22:57+00:00",
            "title": "Conditioned Query Generation for Task-Oriented Dialogue Systems",
            "authors": [
                "St\u00e9phane d'Ascoli",
                "Alice Coucke",
                "Francesco Caltagirone",
                "Alexandre Caulier",
                "Marc Lelarge"
            ],
            "abstract": "Scarcity of training data for task-oriented dialogue systems is a well known\nproblem that is usually tackled with costly and time-consuming manual data\nannotation. An alternative solution is to rely on automatic text generation\nwhich, although less accurate than human supervision, has the advantage of\nbeing cheap and fast. In this paper we propose a novel controlled data\ngeneration method that could be used as a training augmentation framework for\nclosed-domain dialogue. Our contribution is twofold. First we show how to\noptimally train and control the generation of intent-specific sentences using a\nconditional variational autoencoder. Then we introduce a novel protocol called\nquery transfer that allows to leverage a broad, unlabelled dataset to extract\nrelevant information. Comparison with two different baselines shows that our\nmethod, in the appropriate regime, consistently improves the diversity of the\ngenerated queries without compromising their quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03705v4",
            "date": "2019-11-09 14:53:59+00:00",
            "title": "CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning",
            "authors": [
                "Bill Yuchen Lin",
                "Wangchunshu Zhou",
                "Ming Shen",
                "Pei Zhou",
                "Chandra Bhagavatula",
                "Yejin Choi",
                "Xiang Ren"
            ],
            "abstract": "Recently, large-scale pre-trained language models have demonstrated\nimpressive performance on several commonsense-reasoning benchmark datasets.\nHowever, building machines with commonsense to compose realistically plausible\nsentences remains challenging. In this paper, we present a constrained text\ngeneration task, CommonGen associated with a benchmark dataset, to explicitly\ntest machines for the ability of generative commonsense reasoning. Given a set\nof common concepts (e.g., {dog, frisbee, catch, throw}); the task is to\ngenerate a coherent sentence describing an everyday scenario using these\nconcepts (e.g., \"a man throws a frisbee and his dog catches it\").\n  The CommonGen task is challenging because it inherently requires 1)\nrelational reasoning with background commonsense knowledge, and 2)\ncompositional generalization ability to work on unseen concept combinations.\nOur dataset, constructed through a combination of crowdsourced and existing\ncaption corpora, consists of 79k commonsense descriptions over 35k unique\nconcept-sets. Experiments show that there is a large gap between\nstate-of-the-art text generation models (e.g., T5) and human performance.\nFurthermore, we demonstrate that the learned generative commonsense reasoning\ncapability can be transferred to improve downstream tasks such as CommonsenseQA\nby generating additional context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03743v1",
            "date": "2019-11-09 17:56:47+00:00",
            "title": "A perspective on multi-agent communication for information fusion",
            "authors": [
                "Homagni Saha",
                "Vijay Venkataraman",
                "Alberto Speranzon",
                "Soumik Sarkar"
            ],
            "abstract": "Collaborative decision making in multi-agent systems typically requires a\npredefined communication protocol among agents. Usually, agent-level\nobservations are locally processed and information is exchanged using the\npredefined protocol, enabling the team to perform more efficiently than each\nagent operating in isolation. In this work, we consider the situation where\nagents, with complementary sensing modalities must co-operate to achieve a\ncommon goal/task by learning an efficient communication protocol. We frame the\nproblem within an actor-critic scheme, where the agents learn optimal policies\nin a centralized fashion, while taking action in a distributed manner. We\nprovide an interpretation of the emergent communication between the agents. We\nobserve that the information exchanged is not just an encoding of the raw\nsensor data but is, rather, a specific set of directive actions that depend on\nthe overall task. Simulation results demonstrate the interpretability of the\nlearnt communication in a variety of tasks.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/1911.03762v1",
            "date": "2019-11-09 19:41:50+00:00",
            "title": "Speaker Adaptation for Attention-Based End-to-End Speech Recognition",
            "authors": [
                "Zhong Meng",
                "Yashesh Gaur",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "We propose three regularization-based speaker adaptation approaches to adapt\nthe attention-based encoder-decoder (AED) model with very limited adaptation\ndata from target speakers for end-to-end automatic speech recognition. The\nfirst method is Kullback-Leibler divergence (KLD) regularization, in which the\noutput distribution of a speaker-dependent (SD) AED is forced to be close to\nthat of the speaker-independent (SI) model by adding a KLD regularization to\nthe adaptation criterion. To compensate for the asymmetric deficiency in KLD\nregularization, an adversarial speaker adaptation (ASA) method is proposed to\nregularize the deep-feature distribution of the SD AED through the adversarial\nlearning of an auxiliary discriminator and the SD AED. The third approach is\nthe multi-task learning, in which an SD AED is trained to jointly perform the\nprimary task of predicting a large number of output units and an auxiliary task\nof predicting a small number of output units to alleviate the target sparsity\nissue. Evaluated on a Microsoft short message dictation task, all three methods\nare highly effective in adapting the AED model, achieving up to 12.2% and 3.0%\nword error rate improvement over an SI AED trained from 3400 hours data for\nsupervised and unsupervised adaptation, respectively.",
            "journal ref": "Interspeech 2019, Graz, Austria",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03821v2",
            "date": "2019-11-10 01:39:46+00:00",
            "title": "Adaptive Fusion Techniques for Multimodal Data",
            "authors": [
                "Gaurav Sahu",
                "Olga Vechtomova"
            ],
            "abstract": "Effective fusion of data from multiple modalities, such as video, speech, and\ntext, is challenging due to the heterogeneous nature of multimodal data. In\nthis paper, we propose adaptive fusion techniques that aim to model context\nfrom different modalities effectively. Instead of defining a deterministic\nfusion operation, such as concatenation, for the network, we let the network\ndecide \"how\" to combine a given set of multimodal features more effectively. We\npropose two networks: 1) Auto-Fusion, which learns to compress information from\ndifferent modalities while preserving the context, and 2) GAN-Fusion, which\nregularizes the learned latent space given context from complementing\nmodalities. A quantitative evaluation on the tasks of multimodal machine\ntranslation and emotion recognition suggests that our lightweight, adaptive\nnetworks can better model context from other modalities than existing methods,\nmany of which employ massive transformer-based networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03853v2",
            "date": "2019-11-10 04:55:46+00:00",
            "title": "Modelling Bahdanau Attention using Election methods aided by Q-Learning",
            "authors": [
                "Rakesh Bal",
                "Sayan Sinha"
            ],
            "abstract": "Neural Machine Translation has lately gained a lot of \"attention\" with the\nadvent of more and more sophisticated but drastically improved models.\nAttention mechanism has proved to be a boon in this direction by providing\nweights to the input words, making it easy for the decoder to identify words\nrepresenting the present context. But by and by, as newer attention models with\nmore complexity came into development, they involved large computation, making\ninference slow. In this paper, we have modelled the attention network using\ntechniques resonating with social choice theory. Along with that, the attention\nmechanism, being a Markov Decision Process, has been represented by\nreinforcement learning techniques. Thus, we propose to use an election method\n($k$-Borda), fine-tuned using Q-learning, as a replacement for attention\nnetworks. The inference time for this network is less than a standard Bahdanau\ntranslator, and the results of the translation are comparable. This not only\nexperimentally verifies the claims stated above but also helped provide a\nfaster inference.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03854v2",
            "date": "2019-11-10 05:06:38+00:00",
            "title": "r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection",
            "authors": [
                "Kai Nakamura",
                "Sharon Levy",
                "William Yang Wang"
            ],
            "abstract": "Fake news has altered society in negative ways in politics and culture. It\nhas adversely affected both online social network systems as well as offline\ncommunities and conversations. Using automatic machine learning classification\nmodels is an efficient way to combat the widespread dissemination of fake news.\nHowever, a lack of effective, comprehensive datasets has been a problem for\nfake news research and detection model development. Prior fake news datasets do\nnot provide multimodal text and image data, metadata, comment data, and\nfine-grained fake news categorization at the scale and breadth of our dataset.\nWe present Fakeddit, a novel multimodal dataset consisting of over 1 million\nsamples from multiple categories of fake news. After being processed through\nseveral stages of review, the samples are labeled according to 2-way, 3-way,\nand 6-way classification categories through distant supervision. We construct\nhybrid text+image models and perform extensive experiments for multiple\nvariations of classification, demonstrating the importance of the novel aspect\nof multimodality and fine-grained classification unique to Fakeddit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03855v2",
            "date": "2019-11-10 05:13:29+00:00",
            "title": "Correcting Sociodemographic Selection Biases for Accurate Population Prediction from Social Media",
            "authors": [
                "Salvatore Giorgi",
                "Veronica Lynn",
                "Sandra Matz",
                "Lyle Ungar",
                "H. Andrew Schwartz"
            ],
            "abstract": "Social media is increasingly used for large-scale population predictions,\nsuch as estimating community health statistics. However, social media users are\nnot typically a representative sample of the intended population - a \"selection\nbias\". Across six tasks for predicting U.S. county population health statistics\nfrom Twitter, we explore standard restratification techniques - bias mitigation\napproaches that reweight people-specific variables according to how\nunder-sampled their socio-demographic groups are. We found standard\nrestratification provided no improvement and often degraded population\nprediction accuracy. The core reason for this seemed to be both shrunken and\nsparse estimates of each population's socio-demographics for which we thus\ndevelop and evaluate three methods to address: predictive redistribution to\naccount for shrinking, as well as adaptive binning and informed smoothing to\nhandle sparse socio-demographic estimates. We show each of our methods can\nsignificantly improve over the standard restratification approaches. Combining\napproaches, we find substantial improvements over non-restratified models,\nyielding a 44.9% increase in variance explained for predicting surveyed life\nsatisfaction, and an 10.5% average increase across all tasks.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.03869v4",
            "date": "2019-11-10 07:05:25+00:00",
            "title": "Knowledge Guided Named Entity Recognition for BioMedical Text",
            "authors": [
                "Pratyay Banerjee",
                "Kuntal Kumar Pal",
                "Murthy Devarakonda",
                "Chitta Baral"
            ],
            "abstract": "In this work, we formulate the NER task as a multi-answer knowledge guided QA\ntask (KGQA) which helps to predict entities only by assigning B, I and O tags\nwithout associating entity types with the tags. We provide different knowledge\ncontexts, such as, entity types, questions, definitions and examples along with\nthe text and train on a combined dataset of 18 biomedical corpora. This\nformulation (a) enables systems to jointly learn NER specific features from\nvaried NER datasets, (b) can use knowledge-text attention to identify words\nhaving higher similarity to provided knowledge, improving performance, (c)\nreduces system confusion by reducing the prediction classes to B, I, O only,\nand (d) makes detection of nested entities easier. We perform extensive\nexperiments of this KGQA formulation on 18 biomedical NER datasets, and through\nexperiments we note that knowledge helps in achieving better performance. Our\nproblem formulation is able to achieve state-of-the-art results in 12 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03882v4",
            "date": "2019-11-10 09:23:42+00:00",
            "title": "Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders",
            "authors": [
                "Yu Duan",
                "Canwen Xu",
                "Jiaxin Pei",
                "Jialong Han",
                "Chenliang Li"
            ],
            "abstract": "Conditional Text Generation has drawn much attention as a topic of Natural\nLanguage Generation (NLG) which provides the possibility for humans to control\nthe properties of generated contents. Current conditional generation models\ncannot handle emerging conditions due to their joint end-to-end learning\nfashion. When a new condition added, these techniques require full retraining.\nIn this paper, we present a new framework named Pre-train and Plug-in\nVariational Auto-Encoder (PPVAE) towards flexible conditional text generation.\nPPVAE decouples the text generation module from the condition representation\nmodule to allow \"one-to-many\" conditional generation. When a fresh condition\nemerges, only a lightweight network needs to be trained and works as a plug-in\nfor PPVAE, which is efficient and desirable for real-world applications.\nExtensive experiments demonstrate the superiority of PPVAE against the existing\nalternatives with better conditionality and diversity but less training effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03904v2",
            "date": "2019-11-10 11:23:37+00:00",
            "title": "HighwayGraph: Modelling Long-distance Node Relations for Improving General Graph Neural Network",
            "authors": [
                "Deli Chen",
                "Xiaoqian Liu",
                "Yankai Lin",
                "Peng Li",
                "Jie Zhou",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Graph Neural Networks (GNNs) are efficient approaches to process\ngraph-structured data. Modelling long-distance node relations is essential for\nGNN training and applications. However, conventional GNNs suffer from bad\nperformance in modelling long-distance node relations due to limited-layer\ninformation propagation. Existing studies focus on building deep GNN\narchitectures, which face the over-smoothing issue and cannot model node\nrelations in particularly long distance. To address this issue, we propose to\nmodel long-distance node relations by simply relying on shallow GNN\narchitectures with two solutions: (1) Implicitly modelling by learning to\npredict node pair relations (2) Explicitly modelling by adding edges between\nnodes that potentially have the same label. To combine our two solutions, we\npropose a model-agnostic training framework named HighwayGraph, which overcomes\nthe challenge of insufficient labeled nodes by sampling node pairs from the\ntraining set and adopting the self-training method. Extensive experimental\nresults show that our HighwayGraph achieves consistent and significant\nimprovements over four representative GNNs on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03934v2",
            "date": "2019-11-10 13:45:06+00:00",
            "title": "Evaluating Voice Conversion-based Privacy Protection against Informed Attackers",
            "authors": [
                "Brij Mohan Lal Srivastava",
                "Nathalie Vauquier",
                "Md Sahidullah",
                "Aur\u00e9lien Bellet",
                "Marc Tommasi",
                "Emmanuel Vincent"
            ],
            "abstract": "Speech data conveys sensitive speaker attributes like identity or accent.\nWith a small amount of found data, such attributes can be inferred and\nexploited for malicious purposes: voice cloning, spoofing, etc. Anonymization\naims to make the data unlinkable, i.e., ensure that no utterance can be linked\nto its original speaker. In this paper, we investigate anonymization methods\nbased on voice conversion. In contrast to prior work, we argue that various\nlinkage attacks can be designed depending on the attackers' knowledge about the\nanonymization scheme. We compare two frequency warping-based conversion methods\nand a deep learning based method in three attack scenarios. The utility of\nconverted speech is measured via the word error rate achieved by automatic\nspeech recognition, while privacy protection is assessed by the increase in\nequal error rate achieved by state-of-the-art i-vector or x-vector based\nspeaker verification. Our results show that voice conversion schemes are unable\nto effectively protect against an attacker that has extensive knowledge of the\ntype of conversion and how it has been applied, but may provide some protection\nagainst less knowledgeable attackers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03936v1",
            "date": "2019-11-10 14:00:27+00:00",
            "title": "Can Neural Image Captioning be Controlled via Forced Attention?",
            "authors": [
                "Philipp Sadler",
                "Tatjana Scheffler",
                "David Schlangen"
            ],
            "abstract": "Learned dynamic weighting of the conditioning signal (attention) has been\nshown to improve neural language generation in a variety of settings. The\nweights applied when generating a particular output sequence have also been\nviewed as providing a potentially explanatory insight into the internal\nworkings of the generator. In this paper, we reverse the direction of this\nconnection and ask whether through the control of the attention of the model we\ncan control its output. Specifically, we take a standard neural image\ncaptioning model that uses attention, and fix the attention to pre-determined\nareas in the image. We evaluate whether the resulting output is more likely to\nmention the class of the object in that area than the normally generated\ncaption. We introduce three effective methods to control the attention and find\nthat these are producing expected results in up to 28.56% of the cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.03962v3",
            "date": "2019-11-10 16:56:25+00:00",
            "title": "Classical linear logic, cobordisms and categorial grammars",
            "authors": [
                "Sergey Slavnov"
            ],
            "abstract": "We propose a categorial grammar based on classical multiplicative linear\nlogic.\n  This can be seen as an extension of abstract categorial grammars (ACG) and is\nat least as expressive. However, constituents of {\\it linear logic grammars\n(LLG)} are not abstract ${\\lambda}$-terms, but simply tuples of words with\nlabeled endpoints and supplied with specific {\\it plugging instructions}: the\nsets of endpoints are subdivided into the {\\it incoming} and the {\\it outgoing}\nparts. We call such objects {\\it word cobordisms}.\n  A key observation is that word cobordisms can be organized in a category,\nvery similar to the familiar category of topological cobordisms. This category\nis symmetric monoidal closed and compact closed and thus is a model of linear\n$\\lambda$-calculus and classical, as well as intuitionistic linear logic. This\nallows us using linear logic as a typing system for word cobordisms.\n  At least, this gives a concrete and intuitive representation of ACG.\n  We think, however, that the category of word cobordisms, which has a rich\nstructure and is independent of any grammar, might be interesting on its own\nright.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/1911.03970v3",
            "date": "2019-11-10 17:41:11+00:00",
            "title": "Improved Large-margin Softmax Loss for Speaker Diarisation",
            "authors": [
                "Yassir Fathullah",
                "Chao Zhang",
                "Philip C. Woodland"
            ],
            "abstract": "Speaker diarisation systems nowadays use embeddings generated from speech\nsegments in a bottleneck layer, which are needed to be discriminative for\nunseen speakers. It is well-known that large-margin training can improve the\ngeneralisation ability to unseen data, and its use in such open-set problems\nhas been widespread. Therefore, this paper introduces a general approach to the\nlarge-margin softmax loss without any approximations to improve the quality of\nspeaker embeddings for diarisation. Furthermore, a novel and simple way to\nstabilise training, when large-margin softmax is used, is proposed. Finally, to\ncombat the effect of overlapping speech, different training margins are used to\nreduce the negative effect overlapping speech has on creating discriminative\nembeddings. Experiments on the AMI meeting corpus show that the use of\nlarge-margin softmax significantly improves the speaker error rate (SER). By\nusing all hyper parameters of the loss in a unified way, further improvements\nwere achieved which reached a relative SER reduction of 24.6% over the\nbaseline. However, by training overlapping and single speaker speech samples\nwith different margins, the best result was achieved, giving overall a 29.5%\nSER reduction relative to the baseline.",
            "journal ref": "ICASSP 2020, Barcelona, Spain, 2020, pp. 7104-7108",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.03976v2",
            "date": "2019-11-10 18:50:46+00:00",
            "title": "On Posterior Collapse and Encoder Feature Dispersion in Sequence VAEs",
            "authors": [
                "Teng Long",
                "Yanshuai Cao",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Variational autoencoders (VAEs) hold great potential for modelling text, as\nthey could in theory separate high-level semantic and syntactic properties from\nlocal regularities of natural language. Practically, however, VAEs with\nautoregressive decoders often suffer from posterior collapse, a phenomenon\nwhere the model learns to ignore the latent variables, causing the sequence VAE\nto degenerate into a language model. In this paper, we argue that posterior\ncollapse is in part caused by the lack of dispersion in encoder features. We\nprovide empirical evidence to verify this hypothesis, and propose a\nstraightforward fix using pooling. This simple technique effectively prevents\nposterior collapse, allowing model to achieve significantly better data\nlog-likelihood than standard sequence VAEs. Comparing to existing work, our\nproposed method is able to achieve comparable or superior performances while\nbeing more computationally efficient.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.03977v3",
            "date": "2019-11-10 18:58:20+00:00",
            "title": "Multimodal Intelligence: Representation Learning, Information Fusion, and Applications",
            "authors": [
                "Chao Zhang",
                "Zichao Yang",
                "Xiaodong He",
                "Li Deng"
            ],
            "abstract": "Deep learning methods have revolutionized speech recognition, image\nrecognition, and natural language processing since 2010. Each of these tasks\ninvolves a single modality in their input signals. However, many applications\nin the artificial intelligence field involve multiple modalities. Therefore, it\nis of broad interest to study the more difficult and complex problem of\nmodeling and learning across multiple modalities. In this paper, we provide a\ntechnical review of available models and learning methods for multimodal\nintelligence. The main focus of this review is the combination of vision and\nnatural language modalities, which has become an important topic in both the\ncomputer vision and natural language processing research communities. This\nreview provides a comprehensive analysis of recent works on multimodal deep\nlearning from three perspectives: learning multimodal representations, fusing\nmultimodal signals at various levels, and multimodal applications. Regarding\nmultimodal representation learning, we review the key concepts of embedding,\nwhich unify multimodal signals into a single vector space and thereby enable\ncross-modality signal processing. We also review the properties of many types\nof embeddings that are constructed and learned for general downstream tasks.\nRegarding multimodal fusion, this review focuses on special architectures for\nthe integration of representations of unimodal signals for a particular task.\nRegarding applications, selected areas of a broad interest in the current\nliterature are covered, including image-to-text caption generation,\ntext-to-image generation, and visual question answering. We believe that this\nreview will facilitate future studies in the emerging field of multimodal\nintelligence for related communities.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.04111v1",
            "date": "2019-11-11 06:54:46+00:00",
            "title": "A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis",
            "authors": [
                "Junjie Pan",
                "Xiang Yin",
                "Zhiling Zhang",
                "Shichao Liu",
                "Yang Zhang",
                "Zejun Ma",
                "Yuxuan Wang"
            ],
            "abstract": "In Mandarin text-to-speech (TTS) system, the front-end text processing module\nsignificantly influences the intelligibility and naturalness of synthesized\nspeech. Building a typical pipeline-based front-end which consists of multiple\nindividual components requires extensive efforts. In this paper, we proposed a\nunified sequence-to-sequence front-end model for Mandarin TTS that converts raw\ntexts to linguistic features directly. Compared to the pipeline-based\nfront-end, our unified front-end can achieve comparable performance in\npolyphone disambiguation and prosody word prediction, and improve intonation\nphrase prediction by 0.0738 in F1 score. We also implemented the unified\nfront-end with Tacotron and WaveRNN to build a Mandarin TTS system. The\nsynthesized speech by that got a comparable MOS (4.38) with the pipeline-based\nfront-end (4.37) and close to human recordings (4.49).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04192v2",
            "date": "2019-11-11 11:35:21+00:00",
            "title": "Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication",
            "authors": [
                "Ruize Wang",
                "Zhongyu Wei",
                "Ying Cheng",
                "Piji Li",
                "Haijun Shan",
                "Ji Zhang",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "Visual storytelling aims to generate a narrative paragraph from a sequence of\nimages automatically. Existing approaches construct text description\nindependently for each image and roughly concatenate them as a story, which\nleads to the problem of generating semantically incoherent content. In this\npaper, we propose a new way for visual storytelling by introducing a topic\ndescription task to detect the global semantic context of an image stream. A\nstory is then constructed with the guidance of the topic description. In order\nto combine the two generation tasks, we propose a multi-agent communication\nframework that regards the topic description generator and the story generator\nas two agents and learn them simultaneously via iterative updating mechanism.\nWe validate our approach on VIST dataset, where quantitative results,\nablations, and human evaluation demonstrate our method's good ability in\ngenerating stories with higher quality compared to state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04283v2",
            "date": "2019-11-11 14:03:52+00:00",
            "title": "Data Efficient Direct Speech-to-Text Translation with Modality Agnostic Meta-Learning",
            "authors": [
                "Sathish Indurthi",
                "Houjeung Han",
                "Nikhil Kumar Lakumarapu",
                "Beomseok Lee",
                "Insoo Chung",
                "Sangha Kim",
                "Chanwoo Kim"
            ],
            "abstract": "End-to-end Speech Translation (ST) models have several advantages such as\nlower latency, smaller model size, and less error compounding over conventional\npipelines that combine Automatic Speech Recognition (ASR) and text Machine\nTranslation (MT) models. However, collecting large amounts of parallel data for\nST task is more difficult compared to the ASR and MT tasks. Previous studies\nhave proposed the use of transfer learning approaches to overcome the above\ndifficulty. These approaches benefit from weakly supervised training data, such\nas ASR speech-to-transcript or MT text-to-text translation pairs. However, the\nparameters in these models are updated independently of each task, which may\nlead to sub-optimal solutions. In this work, we adopt a meta-learning algorithm\nto train a modality agnostic multi-task model that transfers knowledge from\nsource tasks=ASR+MT to target task=ST where ST task severely lacks data. In the\nmeta-learning phase, the parameters of the model are exposed to vast amounts of\nspeech transcripts (e.g., English ASR) and text translations (e.g.,\nEnglish-German MT). During this phase, parameters are updated in such a way to\nunderstand speech, text representations, the relation between them, as well as\nact as a good initialization point for the target ST task. We evaluate the\nproposed meta-learning approach for ST tasks on English-German (En-De) and\nEnglish-French (En-Fr) language pairs from the Multilingual Speech Translation\nCorpus (MuST-C). Our method outperforms the previous transfer learning\napproaches and sets new state-of-the-art results for En-De and En-Fr ST tasks\nby obtaining 9.18, and 11.76 BLEU point improvements, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04292v1",
            "date": "2019-11-11 14:11:21+00:00",
            "title": "Diversity by Phonetics and its Application in Neural Machine Translation",
            "authors": [
                "Abdul Rafae Khan",
                "Jia Xu"
            ],
            "abstract": "We introduce a powerful approach for Neural Machine Translation (NMT),\nwhereby, during training and testing, together with the input we provide its\nphonetic encoding and the variants of such an encoding. This way we obtain very\nsignificant improvements up to 4 BLEU points over the state-of-the-art\nlarge-scale system. The phonetic encoding is the first part of our\ncontribution, with a second being a theory that aims to understand the reason\nfor this improvement. Our hypothesis states that the phonetic encoding helps\nNMT because it encodes a procedure to emphasize the difference between\nsemantically diverse sentences. We conduct an empirical geometric validation of\nour hypothesis in support of which we obtain overwhelming evidence.\nSubsequently, as our third contribution and based on our theory, we develop\nartificial mechanisms that leverage during learning the hypothesized (and\nverified) effect phonetics. We achieve significant and consistent improvements\noverall language pairs and datasets: French-English, German-English, and\nChinese-English in medium task IWSLT'17 and French-English in large task WMT'18\nBio, with up to 4 BLEU points over the state-of-the-art. Moreover, our\napproaches are more robust than baselines when evaluated on unknown\nout-of-domain test sets with up to a 5 BLEU point increase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04362v1",
            "date": "2019-11-06 23:51:45+00:00",
            "title": "To Populate is To Regulate",
            "authors": [
                "Nicole Fitzgerald"
            ],
            "abstract": "We examine the effects of instantiating Lewis signaling games within a\npopulation of speaker and listener agents with the aim of producing a set of\ngeneral and robust representations of unstructured pixel data. Preliminary\nexperiments suggest that the set of representations associated with languages\ngenerated within a population outperform those generated between a single\nspeaker-listener pair on this objective, making a case for the adoption of\npopulation-based approaches in emergent communication studies. Furthermore,\npost-hoc analysis reveals that population-based learning induces a number of\nnovel factors to the conventional emergent communication setup, inviting a wide\nrange of future research questions regarding communication dynamics and the\nflow of information within them.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.04427v1",
            "date": "2019-11-11 18:13:05+00:00",
            "title": "Sequence-to-Set Semantic Tagging: End-to-End Multi-label Prediction using Neural Attention for Complex Query Reformulation and Automated Text Categorization",
            "authors": [
                "Manirupa Das",
                "Juanxi Li",
                "Eric Fosler-Lussier",
                "Simon Lin",
                "Soheil Moosavinasab",
                "Steve Rust",
                "Yungui Huang",
                "Rajiv Ramnath"
            ],
            "abstract": "Novel contexts may often arise in complex querying scenarios such as in\nevidence-based medicine (EBM) involving biomedical literature, that may not\nexplicitly refer to entities or canonical concept forms occurring in any fact-\nor rule-based knowledge source such as an ontology like the UMLS. Moreover,\nhidden associations between candidate concepts meaningful in the current\ncontext, may not exist within a single document, but within the collection, via\nalternate lexical forms. Therefore, inspired by the recent success of\nsequence-to-sequence neural models in delivering the state-of-the-art in a wide\nrange of NLP tasks, we develop a novel sequence-to-set framework with neural\nattention for learning document representations that can effect term transfer\nwithin the corpus, for semantically tagging a large collection of documents. We\ndemonstrate that our proposed method can be effective in both a supervised\nmulti-label classification setup for text categorization, as well as in a\nunique unsupervised setting with no human-annotated document labels that uses\nno external knowledge resources and only corpus-derived term statistics to\ndrive the training. Further, we show that semi-supervised training using our\narchitecture on large amounts of unlabeled data can augment performance on the\ntext categorization task when limited labeled data is available. Our approach\nto generate document encodings employing our sequence-to-set models for\ninference of semantic tags, gives to the best of our knowledge, the\nstate-of-the-art for both, the unsupervised query expansion task for the TREC\nCDS 2016 challenge dataset when evaluated on an Okapi BM25--based document\nretrieval system; and also over the MLTM baseline (Soleimani et al, 2016), for\nboth supervised and semi-supervised multi-label prediction tasks on the\ndel.icio.us and Ohsumed datasets. We will make our code and data publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04571v1",
            "date": "2019-11-11 21:18:53+00:00",
            "title": "Long-span language modeling for speech recognition",
            "authors": [
                "Sarangarajan Parthasarathy",
                "William Gale",
                "Xie Chen",
                "George Polovets",
                "Shuangyu Chang"
            ],
            "abstract": "We explore neural language modeling for speech recognition where the context\nspans multiple sentences. Rather than encode history beyond the current\nsentence using a cache of words or document-level features, we focus our study\non the ability of LSTM and Transformer language models to implicitly learn to\ncarry over context across sentence boundaries. We introduce a new architecture\nthat incorporates an attention mechanism into LSTM to combine the benefits of\nrecurrent and attention architectures. We conduct language modeling and speech\nrecognition experiments on the publicly available LibriSpeech corpus. We show\nthat conventional training on a paragraph-level corpus results in significant\nreductions in perplexity compared to training on a sentence-level corpus. We\nalso describe speech recognition experiments using long-span language models in\nsecond-pass re-ranking, and provide insights into the ability of such models to\ntake advantage of context beyond the current sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04669v2",
            "date": "2019-11-12 04:38:19+00:00",
            "title": "How to Evaluate Word Representations of Informal Domain?",
            "authors": [
                "Yekun Chai",
                "Naomi Saphra",
                "Adam Lopez"
            ],
            "abstract": "Diverse word representations have surged in most state-of-the-art natural\nlanguage processing (NLP) applications. Nevertheless, how to efficiently\nevaluate such word embeddings in the informal domain such as Twitter or forums,\nremains an ongoing challenge due to the lack of sufficient evaluation dataset.\nWe derived a large list of variant spelling pairs from UrbanDictionary with the\nautomatic approaches of weakly-supervised pattern-based bootstrapping and\nself-training linear-chain conditional random field (CRF). With these extracted\nrelation pairs we promote the odds of eliding the text normalization procedure\nof traditional NLP pipelines and directly adopting representations of\nnon-standard words in the informal domain. Our code is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04700v1",
            "date": "2019-11-12 07:13:42+00:00",
            "title": "A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data",
            "authors": [
                "Yinhe Zheng",
                "Rongsheng Zhang",
                "Xiaoxi Mao",
                "Minlie Huang"
            ],
            "abstract": "Endowing dialogue systems with personas is essential to deliver more\nhuman-like conversations. However, this problem is still far from well explored\ndue to the difficulties of both embodying personalities in natural languages\nand the persona sparsity issue observed in most dialogue corpora. This paper\nproposes a pre-training based personalized dialogue model that can generate\ncoherent responses using persona-sparse dialogue data. In this method, a\npre-trained language model is used to initialize an encoder and decoder, and\npersonal attribute embeddings are devised to model richer dialogue contexts by\nencoding speakers' personas together with dialogue histories. Further, to\nincorporate the target persona in the decoding process and to balance its\ncontribution, an attention routing structure is devised in the decoder to merge\nfeatures extracted from the target persona and dialogue contexts using\ndynamically predicted weights. Our model can utilize persona-sparse dialogues\nin a unified manner during the training process, and can also control the\namount of persona-related features to exhibit during the inference process.\nBoth automatic and manual evaluation demonstrates that the proposed model\noutperforms state-of-the-art methods for generating more coherent and persona\nconsistent responses with persona-sparse data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04759v1",
            "date": "2019-11-12 09:41:44+00:00",
            "title": "Prediction of Missing Semantic Relations in Lexical-Semantic Network using Random Forest Classifier",
            "authors": [
                "K\u00e9vin Cousot",
                "Mehdi Mirzapour",
                "Waleed Ragheb"
            ],
            "abstract": "This study focuses on the prediction of missing six semantic relations (such\nas is_a and has_part) between two given nodes in RezoJDM a French\nlexical-semantic network. The output of this prediction is a set of pairs in\nwhich the first entries are semantic relations and the second entries are the\nprobabilities of existence of such relations. Due to the statement of the\nproblem we choose the random forest (RF) predictor classifier approach to\ntackle this problem. We take for granted the existing semantic relations, for\ntraining/test dataset, gathered and validated by crowdsourcing. We describe how\nall of the mentioned ideas can be followed after using the node2vec approach in\nthe feature extraction phase. We show how this approach can lead to acceptable\nresults.",
            "journal ref": "CJC PRAXILING 2019, Nov 2019, Montpellier, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04822v2",
            "date": "2019-11-12 12:44:26+00:00",
            "title": "A Capsule Network-based Model for Learning Node Embeddings",
            "authors": [
                "Dai Quoc Nguyen",
                "Tu Dinh Nguyen",
                "Dat Quoc Nguyen",
                "Dinh Phung"
            ],
            "abstract": "In this paper, we focus on learning low-dimensional embeddings for nodes in\ngraph-structured data. To achieve this, we propose Caps2NE -- a new\nunsupervised embedding model leveraging a network of two capsule layers.\nCaps2NE induces a routing process to aggregate feature vectors of context\nneighbors of a given target node at the first capsule layer, then feed these\nfeatures into the second capsule layer to infer a plausible embedding for the\ntarget node. Experimental results show that our proposed Caps2NE obtains\nstate-of-the-art performances on benchmark datasets for the node classification\ntask. Our code is available at: \\url{https://github.com/daiquocnguyen/Caps2NE}.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.04873v2",
            "date": "2019-11-07 11:22:44+00:00",
            "title": "Can Neural Networks Learn Symbolic Rewriting?",
            "authors": [
                "Bartosz Piotrowski",
                "Josef Urban",
                "Chad E. Brown",
                "Cezary Kaliszyk"
            ],
            "abstract": "This work investigates if the current neural architectures are adequate for\nlearning symbolic rewriting. Two kinds of data sets are proposed for this\nresearch -- one based on automated proofs and the other being a synthetic set\nof polynomial terms. The experiments with use of the current neural machine\ntranslation models are performed and its results are discussed. Ideas for\nextending this line of research are proposed, and its relevance is motivated.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.04910v3",
            "date": "2019-11-09 07:02:33+00:00",
            "title": "Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding",
            "authors": [
                "Yun Tang",
                "Jing Huang",
                "Guangtao Wang",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Translational distance-based knowledge graph embedding has shown progressive\nimprovements on the link prediction task, from TransE to the latest\nstate-of-the-art RotatE. However, N-1, 1-N and N-N predictions still remain\nchallenging. In this work, we propose a novel translational distance-based\napproach for knowledge graph link prediction. The proposed method includes\ntwo-folds, first we extend the RotatE from 2D complex domain to high dimension\nspace with orthogonal transforms to model relations for better modeling\ncapacity. Second, the graph context is explicitly modeled via two directed\ncontext representations. These context representations are used as part of the\ndistance scoring function to measure the plausibility of the triples during\ntraining and inference. The proposed approach effectively improves prediction\naccuracy on the difficult N-1, 1-N and N-N cases for knowledge graph link\nprediction task. The experimental results show that it achieves better\nperformance on two benchmark data sets compared to the baseline RotatE,\nespecially on data set (FB15k-237) with many high in-degree connection nodes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.04952v2",
            "date": "2019-11-12 15:53:40+00:00",
            "title": "'Warriors of the Word' -- Deciphering Lyrical Topics in Music and Their Connection to Audio Feature Dimensions Based on a Corpus of Over 100,000 Metal Songs",
            "authors": [
                "Isabella Czedik-Eysenberg",
                "Oliver Wieczorek",
                "Christoph Reuter"
            ],
            "abstract": "We look into the connection between the musical and lyrical content of metal\nmusic by combining automated extraction of high-level audio features and\nquantitative text analysis on a corpus of 124.288 song lyrics from this genre.\nBased on this text corpus, a topic model was first constructed using Latent\nDirichlet Allocation (LDA). For a subsample of 503 songs, scores for predicting\nperceived musical hardness/heaviness and darkness/gloominess were extracted\nusing audio feature models. By combining both audio feature and text analysis,\nwe (1) offer a comprehensive overview of the lyrical topics present within the\nmetal genre and (2) are able to establish whether or not levels of hardness and\nother music dimensions are associated with the occurrence of particularly harsh\n(and other) textual topics. Twenty typical topics were identified and projected\ninto a topic space using multidimensional scaling (MDS). After Bonferroni\ncorrection, positive correlations were found between musical hardness and\ndarkness and textual topics dealing with 'brutal death', 'dystopia', 'archaisms\nand occultism', 'religion and satanism', 'battle' and '(psychological)\nmadness', while there is a negative associations with topics like 'personal\nlife' and 'love and romance'.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.05013v1",
            "date": "2019-11-12 17:11:55+00:00",
            "title": "EDUQA: Educational Domain Question Answering System using Conceptual Network Mapping",
            "authors": [
                "Abhishek Agarwal",
                "Nikhil Sachdeva",
                "Raj Kamal Yadav",
                "Vishaal Udandarao",
                "Vrinda Mittal",
                "Anubha Gupta",
                "Abhinav Mathur"
            ],
            "abstract": "Most of the existing question answering models can be largely compiled into\ntwo categories: i) open domain question answering models that answer generic\nquestions and use large-scale knowledge base along with the targeted web-corpus\nretrieval and ii) closed domain question answering models that address focused\nquestioning area and use complex deep learning models. Both the above models\nderive answers through textual comprehension methods. Due to their inability to\ncapture the pedagogical meaning of textual content, these models are not\nappropriately suited to the educational field for pedagogy. In this paper, we\npropose an on-the-fly conceptual network model that incorporates educational\nsemantics. The proposed model preserves correlations between conceptual\nentities by applying intelligent indexing algorithms on the concept network so\nas to improve answer generation. This model can be utilized for building\ninteractive conversational agents for aiding classroom learning.",
            "journal ref": "IEEE ICASSP (2019) 8137-8141",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.05585v1",
            "date": "2019-11-13 16:26:22+00:00",
            "title": "Structured Sparsification of Gated Recurrent Neural Networks",
            "authors": [
                "Ekaterina Lobacheva",
                "Nadezhda Chirkova",
                "Alexander Markovich",
                "Dmitry Vetrov"
            ],
            "abstract": "Recently, a lot of techniques were developed to sparsify the weights of\nneural networks and to remove networks' structure units, e.g. neurons. We\nadjust the existing sparsification approaches to the gated recurrent\narchitectures. Specifically, in addition to the sparsification of weights and\nneurons, we propose sparsifying the preactivations of gates. This makes some\ngates constant and simplifies LSTM structure. We test our approach on the text\nclassification and language modeling tasks. We observe that the resulting\nstructure of gate sparsity depends on the task and connect the learned\nstructure to the specifics of the particular tasks. Our method also improves\nneuron-wise compression of the model in most of the tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.05652v1",
            "date": "2019-10-30 22:40:05+00:00",
            "title": "Relative contributions of Shakespeare and Fletcher in Henry VIII: An Analysis Based on Most Frequent Words and Most Frequent Rhythmic Patterns",
            "authors": [
                "Petr Plech\u00e1\u010d"
            ],
            "abstract": "The versified play Henry VIII is nowadays widely recognized to be a\ncollaborative work not written solely by William Shakespeare. We employ\ncombined analysis of vocabulary and versification together with machine\nlearning techniques to determine which authors also took part in the writing of\nthe play and what were their relative contributions. Unlike most previous\nstudies, we go beyond the attribution of particular scenes and use the rolling\nattribution approach to determine the probabilities of authorship of pieces of\ntexts, without respecting the scene boundaries. Our results highly support the\ncanonical division of the play between William Shakespeare and John Fletcher\nproposed by James Spedding, but also bring new evidence supporting the\nmodifications proposed later by Thomas Merriam.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.05659v2",
            "date": "2019-11-09 01:58:03+00:00",
            "title": "M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues",
            "authors": [
                "Trisha Mittal",
                "Uttaran Bhattacharya",
                "Rohan Chandra",
                "Aniket Bera",
                "Dinesh Manocha"
            ],
            "abstract": "We present M3ER, a learning-based method for emotion recognition from\nmultiple input modalities. Our approach combines cues from multiple\nco-occurring modalities (such as face, text, and speech) and also is more\nrobust than other methods to sensor noise in any of the individual modalities.\nM3ER models a novel, data-driven multiplicative fusion method to combine the\nmodalities, which learn to emphasize the more reliable cues and suppress others\non a per-sample basis. By introducing a check step which uses Canonical\nCorrelational Analysis to differentiate between ineffective and effective\nmodalities, M3ER is robust to sensor noise. M3ER also generates proxy features\nin place of the ineffectual modalities. We demonstrate the efficiency of our\nnetwork through experimentation on two benchmark datasets, IEMOCAP and\nCMU-MOSEI. We report a mean accuracy of 82.7% on IEMOCAP and 89.0% on\nCMU-MOSEI, which, collectively, is an improvement of about 5% over prior work.",
            "journal ref": "None",
            "category": "eess.SP"
        },
        {
            "url": "http://arxiv.org/abs/1911.05733v3",
            "date": "2019-11-13 03:44:08+00:00",
            "title": "The phonetic bases of vocal expressed emotion: natural versus acted",
            "authors": [
                "Hira Dhamyal",
                "Shahan Ali Memon",
                "Bhiksha Raj",
                "Rita Singh"
            ],
            "abstract": "Can vocal emotions be emulated? This question has been a recurrent concern of\nthe speech community, and has also been vigorously investigated. It has been\nfueled further by its link to the issue of validity of acted emotion databases.\nMuch of the speech and vocal emotion research has relied on acted emotion\ndatabases as valid proxies for studying natural emotions. To create models that\ngeneralize to natural settings, it is crucial to work with valid prototypes --\nones that can be assumed to reliably represent natural emotions. More\nconcretely, it is important to study emulated emotions against natural emotions\nin terms of their physiological, and psychological concomitants. In this paper,\nwe present an on-scale systematic study of the differences between natural and\nacted vocal emotions. We use a self-attention based emotion classification\nmodel to understand the phonetic bases of emotions by discovering the most\n'attended' phonemes for each class of emotions. We then compare these\nattended-phonemes in their importance and distribution across acted and natural\nclasses. Our tests show significant differences in the manner and choice of\nphonemes in acted and natural speech, concluding moderate to low validity and\nvalue in using acted speech databases for emotion classification tasks.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.05978v1",
            "date": "2019-11-14 07:45:32+00:00",
            "title": "HUSE: Hierarchical Universal Semantic Embeddings",
            "authors": [
                "Pradyumna Narayana",
                "Aniket Pednekar",
                "Abishek Krishnamoorthy",
                "Kazoo Sone",
                "Sugato Basu"
            ],
            "abstract": "There is a recent surge of interest in cross-modal representation learning\ncorresponding to images and text. The main challenge lies in mapping images and\ntext to a shared latent space where the embeddings corresponding to a similar\nsemantic concept lie closer to each other than the embeddings corresponding to\ndifferent semantic concepts, irrespective of the modality. Ranking losses are\ncommonly used to create such shared latent space -- however, they do not impose\nany constraints on inter-class relationships resulting in neighboring clusters\nto be completely unrelated. The works in the domain of visual semantic\nembeddings address this problem by first constructing a semantic embedding\nspace based on some external knowledge and projecting image embeddings onto\nthis fixed semantic embedding space. These works are confined only to image\ndomain and constraining the embeddings to a fixed space adds additional burden\non learning. This paper proposes a novel method, HUSE, to learn cross-modal\nrepresentation with semantic information. HUSE learns a shared latent space\nwhere the distance between any two universal embeddings is similar to the\ndistance between their corresponding class embeddings in the semantic embedding\nspace. HUSE also uses a classification objective with a shared classification\nlayer to make sure that the image and text embeddings are in the same shared\nlatent space. Experiments on UPMC Food-101 show our method outperforms previous\nstate-of-the-art on retrieval, hierarchical precision and classification\nresults.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.06111v3",
            "date": "2019-11-08 22:23:30+00:00",
            "title": "Instance-based Transfer Learning for Multilingual Deep Retrieval",
            "authors": [
                "Andrew O. Arnold",
                "William W. Cohen"
            ],
            "abstract": "We focus on the problem of search in the multilingual setting. Examining the\nproblems of next-sentence prediction and inverse cloze, we show that at large\nscale, instance-based transfer learning is surprisingly effective in the\nmultilingual setting, leading to positive transfer on all of the 35 target\nlanguages and two tasks tested. We analyze this improvement and argue that the\nmost natural explanation, namely direct vocabulary overlap between languages,\nonly partially explains the performance gains: in fact, we demonstrate\ntarget-language improvement can occur after adding data from an auxiliary\nlanguage even with no vocabulary in common with the target. This surprising\nresult is due to the effect of transitive vocabulary overlaps between pairs of\nauxiliary and target languages.",
            "journal ref": "The Web Conference Workshop on Multilingual Search, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06118v1",
            "date": "2019-11-12 06:59:38+00:00",
            "title": "Learning Multi-Sense Word Distributions using Approximate Kullback-Leibler Divergence",
            "authors": [
                "P. Jayashree",
                "Ballijepalli Shreya",
                "P. K. Srijith"
            ],
            "abstract": "Learning word representations has garnered greater attention in the recent\npast due to its diverse text applications. Word embeddings encapsulate the\nsyntactic and semantic regularities of sentences. Modelling word embedding as\nmulti-sense gaussian mixture distributions, will additionally capture\nuncertainty and polysemy of words. We propose to learn the Gaussian mixture\nrepresentation of words using a Kullback-Leibler (KL) divergence based\nobjective function. The KL divergence based energy function provides a better\ndistance metric which can effectively capture entailment and distribution\nsimilarity among the words. Due to the intractability of KL divergence for\nGaussian mixture, we go for a KL approximation between Gaussian mixtures. We\nperform qualitative and quantitative experiments on benchmark word similarity\nand entailment datasets which demonstrate the effectiveness of the proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06147v2",
            "date": "2019-11-11 22:06:40+00:00",
            "title": "t-SS3: a text classifier with dynamic n-grams for early risk detection over text streams",
            "authors": [
                "Sergio G. Burdisso",
                "Marcelo Errecalde",
                "Manuel Montes-y-G\u00f3mez"
            ],
            "abstract": "A recently introduced classifier, called SS3, has shown to be well suited to\ndeal with early risk detection (ERD) problems on text streams. It obtained\nstate-of-the-art performance on early depression and anorexia detection on\nReddit in the CLEF's eRisk open tasks. SS3 was created to deal with ERD\nproblems naturally since: it supports incremental training and classification\nover text streams, and it can visually explain its rationale. However, SS3\nprocesses the input using a bag-of-word model lacking the ability to recognize\nimportant word sequences. This aspect could negatively affect the\nclassification performance and also reduces the descriptiveness of visual\nexplanations. In the standard document classification field, it is very common\nto use word n-grams to try to overcome some of these limitations.\nUnfortunately, when working with text streams, using n-grams is not trivial\nsince the system must learn and recognize which n-grams are important \"on the\nfly\". This paper introduces t-SS3, an extension of SS3 that allows it to\nrecognize useful patterns over text streams dynamically. We evaluated our model\nin the eRisk 2017 and 2018 tasks on early depression and anorexia detection.\nExperimental results suggest that t-SS3 is able to improve both current results\nand the richness of visual explanations.",
            "journal ref": "Pattern Recognition Letters, Elsevier, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06149v2",
            "date": "2019-11-11 19:53:58+00:00",
            "title": "Emotional Voice Conversion using Multitask Learning with Text-to-speech",
            "authors": [
                "Tae-Ho Kim",
                "Sungjae Cho",
                "Shinkook Choi",
                "Sejik Park",
                "Soo-Young Lee"
            ],
            "abstract": "Voice conversion (VC) is a task to transform a person's voice to different\nstyle while conserving linguistic contents. Previous state-of-the-art on VC is\nbased on sequence-to-sequence (seq2seq) model, which could mislead linguistic\ninformation. There was an attempt to overcome it by using textual supervision,\nit requires explicit alignment which loses the benefit of using seq2seq model.\nIn this paper, a voice converter using multitask learning with text-to-speech\n(TTS) is presented. The embedding space of seq2seq-based TTS has abundant\ninformation on the text. The role of the decoder of TTS is to convert embedding\nspace to speech, which is same to VC. In the proposed model, the whole network\nis trained to minimize loss of VC and TTS. VC is expected to capture more\nlinguistic information and to preserve training stability by multitask\nlearning. Experiments of VC were performed on a male Korean emotional\ntext-speech dataset, and it is shown that multitask learning is helpful to keep\nlinguistic contents in VC.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.06154v2",
            "date": "2019-11-10 02:09:11+00:00",
            "title": "CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs",
            "authors": [
                "Ahmed El-Kishky",
                "Vishrav Chaudhary",
                "Francisco Guzman",
                "Philipp Koehn"
            ],
            "abstract": "Cross-lingual document alignment aims to identify pairs of documents in two\ndistinct languages that are of comparable content or translations of each\nother. In this paper, we exploit the signals embedded in URLs to label web\ndocuments at scale with an average precision of 94.5% across different language\npairs. We mine sixty-eight snapshots of the Common Crawl corpus and identify\nweb document pairs that are translations of each other. We release a new web\ndataset consisting of over 392 million URL pairs from Common Crawl covering\ndocuments in 8144 language pairs of which 137 pairs include English. In\naddition to curating this massive dataset, we introduce baseline methods that\nleverage cross-lingual representations to identify aligned documents based on\ntheir textual content. Finally, we demonstrate the value of this parallel\ndocuments dataset through a downstream task of mining parallel sentences and\nmeasuring the quality of machine translations from models trained on this mined\ndata. Our objective in releasing this dataset is to foster new research in\ncross-lingual NLP across a variety of low, medium, and high-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06156v1",
            "date": "2019-11-10 04:42:13+00:00",
            "title": "Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding",
            "authors": [
                "Dhanasekar Sundararaman",
                "Vivek Subramanian",
                "Guoyin Wang",
                "Shijing Si",
                "Dinghan Shen",
                "Dong Wang",
                "Lawrence Carin"
            ],
            "abstract": "Attention-based models have shown significant improvement over traditional\nalgorithms in several NLP tasks. The Transformer, for instance, is an\nillustrative example that generates abstract representations of tokens inputted\nto an encoder based on their relationships to all tokens in a sequence. Recent\nstudies have shown that although such models are capable of learning syntactic\nfeatures purely by seeing examples, explicitly feeding this information to deep\nlearning models can significantly enhance their performance. Leveraging\nsyntactic information like part of speech (POS) may be particularly beneficial\nin limited training data settings for complex models such as the Transformer.\nWe show that the syntax-infused Transformer with multiple features achieves an\nimprovement of 0.7 BLEU when trained on the full WMT 14 English to German\ntranslation dataset and a maximum improvement of 1.99 BLEU points when trained\non a fraction of the dataset. In addition, we find that the incorporation of\nsyntax into BERT fine-tuning outperforms baseline on a number of downstream\ntasks from the GLUE benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06182v1",
            "date": "2019-11-05 21:21:40+00:00",
            "title": "MML: Maximal Multiverse Learning for Robust Fine-Tuning of Language Models",
            "authors": [
                "Itzik Malkiel",
                "Lior Wolf"
            ],
            "abstract": "Recent state-of-the-art language models utilize a two-phase training\nprocedure comprised of (i) unsupervised pre-training on unlabeled text, and\n(ii) fine-tuning for a specific supervised task. More recently, many studies\nhave been focused on trying to improve these models by enhancing the\npre-training phase, either via better choice of hyperparameters or by\nleveraging an improved formulation. However, the pre-training phase is\ncomputationally expensive and often done on private datasets. In this work, we\npresent a method that leverages BERT's fine-tuning phase to its fullest, by\napplying an extensive number of parallel classifier heads, which are enforced\nto be orthogonal, while adaptively eliminating the weaker heads during\ntraining. Our method allows the model to converge to an optimal number of\nparallel classifiers, depending on the given dataset at hand.\n  We conduct an extensive inter- and intra-dataset evaluations, showing that\nour method improves the robustness of BERT, sometimes leading to a +9\\% gain in\naccuracy. These results highlight the importance of a proper fine-tuning\nprocedure, especially for relatively smaller-sized datasets. Our code is\nattached as supplementary and our models will be made completely public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06191v1",
            "date": "2019-11-07 03:55:53+00:00",
            "title": "Microsoft Research Asia's Systems for WMT19",
            "authors": [
                "Yingce Xia",
                "Xu Tan",
                "Fei Tian",
                "Fei Gao",
                "Weicong Chen",
                "Yang Fan",
                "Linyuan Gong",
                "Yichong Leng",
                "Renqian Luo",
                "Yiren Wang",
                "Lijun Wu",
                "Jinhua Zhu",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "We Microsoft Research Asia made submissions to 11 language directions in the\nWMT19 news translation tasks. We won the first place for 8 of the 11 directions\nand the second place for the other three. Our basic systems are built on\nTransformer, back translation and knowledge distillation. We integrate several\nof our rececent techniques to enhance the baseline systems: multi-agent dual\nlearning (MADL), masked sequence-to-sequence pre-training (MASS), neural\narchitecture optimization (NAO), and soft contextual data augmentation (SCA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06192v2",
            "date": "2019-11-07 10:00:16+00:00",
            "title": "Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering",
            "authors": [
                "Li Zhou",
                "Kevin Small"
            ],
            "abstract": "Multi-domain dialogue state tracking (DST) is a critical component for\nconversational AI systems. The domain ontology (i.e., specification of domains,\nslots, and values) of a conversational AI system is generally incomplete,\nmaking the capability for DST models to generalize to new slots, values, and\ndomains during inference imperative. In this paper, we propose to model\nmulti-domain DST as a question answering problem, referred to as Dialogue State\nTracking via Question Answering (DSTQA). Within DSTQA, each turn generates a\nquestion asking for the value of a (domain, slot) pair, thus making it\nnaturally extensible to unseen domains, slots, and values. Additionally, we use\na dynamically-evolving knowledge graph to explicitly learn relationships\nbetween (domain, slot) pairs. Our model has a 5.80% and 12.21% relative\nimprovement over the current state-of-the-art model on MultiWOZ 2.0 and\nMultiWOZ 2.1 datasets, respectively. Additionally, our model consistently\noutperforms the state-of-the-art model in domain adaptation settings. (Code is\nreleased at https://github.com/alexa/dstqa )",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06194v2",
            "date": "2019-11-08 03:25:04+00:00",
            "title": "Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models",
            "authors": [
                "Xisen Jin",
                "Zhongyu Wei",
                "Junyi Du",
                "Xiangyang Xue",
                "Xiang Ren"
            ],
            "abstract": "The impressive performance of neural networks on natural language processing\ntasks attributes to their ability to model complicated word and phrase\ncompositions. To explain how the model handles semantic compositions, we study\nhierarchical explanation of neural network predictions. We identify\nnon-additivity and context independent importance attributions within\nhierarchies as two desirable properties for highlighting word and phrase\ncompositions. We show some prior efforts on hierarchical explanations, e.g.\ncontextual decomposition, do not satisfy the desired properties mathematically,\nleading to inconsistent explanation quality in different models. In this paper,\nwe start by proposing a formal and general way to quantify the importance of\neach word and phrase. Following the formulation, we propose Sampling and\nContextual Decomposition (SCD) algorithm and Sampling and Occlusion (SOC)\nalgorithm. Human and metrics evaluation on both LSTM models and BERT\nTransformer models on multiple datasets show that our algorithms outperform\nprior hierarchical explanation algorithms. Our algorithms help to visualize\nsemantic composition captured by models, extract classification rules and\nimprove human trust of models. Project page: https://inklab.usc.edu/hiexpl/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06197v1",
            "date": "2019-11-08 14:49:25+00:00",
            "title": "Towards automatic extractive text summarization of A-133 Single Audit reports with machine learning",
            "authors": [
                "Vivian T. Chou",
                "LeAnna Kent",
                "Joel A. G\u00f3ngora",
                "Sam Ballerini",
                "Carl D. Hoover"
            ],
            "abstract": "The rapid growth of text data has motivated the development of\nmachine-learning based automatic text summarization strategies that concisely\ncapture the essential ideas in a larger text. This study aimed to devise an\nextractive summarization method for A-133 Single Audits, which assess if\nrecipients of federal grants are compliant with program requirements for use of\nfederal funding. Currently, these voluminous audits must be manually analyzed\nby officials for oversight, risk management, and prioritization purposes.\nAutomated summarization has the potential to streamline these processes.\nAnalysis focused on the \"Findings\" section of ~20,000 Single Audits spanning\n2016-2018. Following text preprocessing and GloVe embedding, sentence-level\nk-means clustering was performed to partition sentences by topic and to\nestablish the importance of each sentence. For each audit, key summary\nsentences were extracted by proximity to cluster centroids. Summaries were\njudged by non-expert human evaluation and compared to human-generated summaries\nusing the ROUGE metric. Though the goal was to fully automate summarization of\nA-133 audits, human input was required at various stages due to large\nvariability in audit writing style, content, and context. Examples of human\ninputs include the number of clusters, the choice to keep or discard certain\nclusters based on their content relevance, and the definition of a top\nsentence. Overall, this approach made progress towards automated extractive\nsummaries of A-133 audits, with future work to focus on full automation and\nimproving summary consistency. This work highlights the inherent difficulty and\nsubjective nature of automated summarization in a real-world application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06311v3",
            "date": "2019-11-14 18:51:59+00:00",
            "title": "Sato: Contextual Semantic Type Detection in Tables",
            "authors": [
                "Dan Zhang",
                "Yoshihiko Suhara",
                "Jinfeng Li",
                "Madelon Hulsebos",
                "\u00c7a\u011fatay Demiralp",
                "Wang-Chiew Tan"
            ],
            "abstract": "Detecting the semantic types of data columns in relational tables is\nimportant for various data preparation and information retrieval tasks such as\ndata cleaning, schema matching, data discovery, and semantic search. However,\nexisting detection approaches either perform poorly with dirty data, support\nonly a limited number of semantic types, fail to incorporate the table context\nof columns or rely on large sample sizes for training data. We introduce Sato,\na hybrid machine learning model to automatically detect the semantic types of\ncolumns in tables, exploiting the signals from the context as well as the\ncolumn values. Sato combines a deep learning model trained on a large-scale\ntable corpus with topic modeling and structured prediction to achieve\nsupport-weighted and macro average F1 scores of 0.925 and 0.735, respectively,\nexceeding the state-of-the-art performance by a significant margin. We\nextensively analyze the overall and per-type performance of Sato, discussing\nhow individual modeling components, as well as feature categories, contribute\nto its performance.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1911.06573v1",
            "date": "2019-11-15 11:33:50+00:00",
            "title": "Independent and automatic evaluation of acoustic-to-articulatory inversion models",
            "authors": [
                "Maud Parrot",
                "Juliette Millet",
                "Ewan Dunbar"
            ],
            "abstract": "Reconstruction of articulatory trajectories from the acoustic speech signal\nhas been proposed for improving speech recognition and text-to-speech\nsynthesis. However, to be useful in these settings, articulatory reconstruction\nmust be speaker independent. Furthermore, as most research focuses on single,\nsmall datasets with few speakers, robust articulatory reconstrucion could\nprofit from combining datasets. Standard evaluation measures such as root mean\nsquare error and Pearson correlation are inappropriate for evaluating the\nspeaker-independence of models or the usefulness of combining datasets. We\npresent a new evaluation for articulatory reconstruction which is independent\nof the articulatory data set used for training: the phone discrimination ABX\ntask. We use the ABX measure to evaluate a Bi-LSTM based model trained on 3\ndatasets (14 speakers), and show that it gives information complementary to the\nstandard measures, and enables us to evaluate the effects of dataset merging,\nas well as the speaker independence of the model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.06602v1",
            "date": "2019-11-14 18:20:36+00:00",
            "title": "Radically Compositional Cognitive Concepts",
            "authors": [
                "Toby B. St Clere Smithe"
            ],
            "abstract": "Despite ample evidence that our concepts, our cognitive architecture, and\nmathematics itself are all deeply compositional, few models take advantage of\nthis structure. We therefore propose a radically compositional approach to\ncomputational neuroscience, drawing on the methods of applied category theory.\nWe describe how these tools grant us a means to overcome complexity and improve\ninterpretability, and supply a rigorous common language for scientific\nmodelling, analogous to the type theories of computer science. As a case study,\nwe sketch how to translate from compositional narrative concepts to neural\ncircuits and back again.",
            "journal ref": "None",
            "category": "q-bio.NC"
        },
        {
            "url": "http://arxiv.org/abs/1911.06603v3",
            "date": "2019-10-21 12:49:55+00:00",
            "title": "Transformer-CNN: Fast and Reliable tool for QSAR",
            "authors": [
                "Pavel Karpov",
                "Guillaume Godin",
                "Igor V. Tetko"
            ],
            "abstract": "We present SMILES-embeddings derived from the internal encoder state of a\nTransformer [1] model trained to canonize SMILES as a Seq2Seq problem. Using a\nCharNN [2] architecture upon the embeddings results in higher quality\ninterpretable QSAR/QSPR models on diverse benchmark datasets including\nregression and classification tasks. The proposed Transformer-CNN method uses\nSMILES augmentation for training and inference, and thus the prognosis is based\non an internal consensus. That both the augmentation and transfer learning are\nbased on embeddings allows the method to provide good results for small\ndatasets. We discuss the reasons for such effectiveness and draft future\ndirections for the development of the method. The source code and the\nembeddings needed to train a QSAR model are available on\nhttps://github.com/bigchem/transformer-cnn. The repository also has a\nstandalone program for QSAR prognosis which calculates individual atoms\ncontributions, thus interpreting the model's result. OCHEM [3] environment\n(https://ochem.eu) hosts the on-line implementation of the method proposed.",
            "journal ref": "None",
            "category": "q-bio.QM"
        },
        {
            "url": "http://arxiv.org/abs/1911.06641v2",
            "date": "2019-11-15 14:03:30+00:00",
            "title": "CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation",
            "authors": [
                "Zhiyue Liu",
                "Jiahai Wang",
                "Zhiwei Liang"
            ],
            "abstract": "Generating multiple categories of texts is a challenging task and draws more\nand more attention. Since generative adversarial nets (GANs) have shown\ncompetitive results on general text generation, they are extended for category\ntext generation in some previous works. However, the complicated model\nstructures and learning strategies limit their performance and exacerbate the\ntraining instability. This paper proposes a category-aware GAN (CatGAN) which\nconsists of an efficient category-aware model for category text generation and\na hierarchical evolutionary learning algorithm for training our model. The\ncategory-aware model directly measures the gap between real samples and\ngenerated samples on each category, then reducing this gap will guide the model\nto generate high-quality category samples. The Gumbel-Softmax relaxation\nfurther frees our model from complicated learning strategies for updating\nCatGAN on discrete data. Moreover, only focusing on the sample quality normally\nleads the mode collapse problem, thus a hierarchical evolutionary learning\nalgorithm is introduced to stabilize the training procedure and obtain the\ntrade-off between quality and diversity while training CatGAN. Experimental\nresults demonstrate that CatGAN outperforms most of the existing\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06747v1",
            "date": "2019-11-15 16:56:17+00:00",
            "title": "Towards Personalized Dialog Policies for Conversational Skill Discovery",
            "authors": [
                "Maryam Fazel-Zarandi",
                "Sampat Biswas",
                "Ryan Summers",
                "Ahmed Elmalt",
                "Andy McCraw",
                "Michael McPhilips",
                "John Peach"
            ],
            "abstract": "Many businesses and consumers are extending the capabilities of voice-based\nservices such as Amazon Alexa, Google Home, Microsoft Cortana, and Apple Siri\nto create custom voice experiences (also known as skills). As the number of\nthese experiences increases, a key problem is the discovery of skills that can\nbe used to address a user's request. In this paper, we focus on conversational\nskill discovery and present a conversational agent which engages in a dialog\nwith users to help them find the skills that fulfill their needs. To this end,\nwe start with a rule-based agent and improve it by using reinforcement\nlearning. In this way, we enable the agent to adapt to different user\nattributes and conversational styles as it interacts with users. We evaluate\nour approach in a real production setting by deploying the agent to interact\nwith real users, and show the effectiveness of the conversational agent in\nhelping users find the skills that serve their request.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.06910v2",
            "date": "2019-11-15 23:24:17+00:00",
            "title": "CNN-based Dual-Chain Models for Knowledge Graph Learning",
            "authors": [
                "Bo Peng",
                "Renqiang Min",
                "Xia Ning"
            ],
            "abstract": "Knowledge graph learning plays a critical role in integrating domain specific\nknowledge bases when deploying machine learning and data mining models in\npractice. Existing methods on knowledge graph learning primarily focus on\nmodeling the relations among entities as translations among the relations and\nentities, and many of these methods are not able to handle zero-shot problems,\nwhen new entities emerge. In this paper, we present a new convolutional neural\nnetwork (CNN)-based dual-chain model. Different from translation based methods,\nin our model, interactions among relations and entities are directly captured\nvia CNN over their embeddings. Moreover, a secondary chain of learning is\nconducted simultaneously to incorporate additional information and to enable\nbetter performance. We also present an extension of this model, which\nincorporates descriptions of entities and learns a second set of entity\nembeddings from the descriptions. As a result, the extended model is able to\neffectively handle zero-shot problems. We conducted comprehensive experiments,\ncomparing our methods with 15 methods on 8 benchmark datasets. Extensive\nexperimental results demonstrate that our proposed methods achieve or\noutperform the state-of-the-art results on knowledge graph learning, and\noutperform other methods on zero-shot problems. In addition, our methods\napplied to real-world biomedical data are able to produce results that conform\nto expert domain knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07013v1",
            "date": "2019-11-16 11:00:16+00:00",
            "title": "Understanding and Improving Layer Normalization",
            "authors": [
                "Jingjing Xu",
                "Xu Sun",
                "Zhiyuan Zhang",
                "Guangxiang Zhao",
                "Junyang Lin"
            ],
            "abstract": "Layer normalization (LayerNorm) is a technique to normalize the distributions\nof intermediate layers. It enables smoother gradients, faster training, and\nbetter generalization accuracy. However, it is still unclear where the\neffectiveness stems from. In this paper, our main contribution is to take a\nstep further in understanding LayerNorm. Many of previous studies believe that\nthe success of LayerNorm comes from forward normalization. Unlike them, we find\nthat the derivatives of the mean and variance are more important than forward\nnormalization by re-centering and re-scaling backward gradients. Furthermore,\nwe find that the parameters of LayerNorm, including the bias and gain, increase\nthe risk of over-fitting and do not work in most cases. Experiments show that a\nsimple version of LayerNorm (LayerNorm-simple) without the bias and gain\noutperforms LayerNorm on four datasets. It obtains the state-of-the-art\nperformance on En-Vi machine translation. To address the over-fitting problem,\nwe propose a new normalization method, Adaptive Normalization (AdaNorm), by\nreplacing the bias and gain with a new transformation function. Experiments\nshow that AdaNorm demonstrates better results than LayerNorm on seven out of\neight datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.07141v4",
            "date": "2019-11-17 03:14:02+00:00",
            "title": "Working Memory Graphs",
            "authors": [
                "Ricky Loynd",
                "Roland Fernandez",
                "Asli Celikyilmaz",
                "Adith Swaminathan",
                "Matthew Hausknecht"
            ],
            "abstract": "Transformers have increasingly outperformed gated RNNs in obtaining new\nstate-of-the-art results on supervised tasks involving text sequences. Inspired\nby this trend, we study the question of how Transformer-based models can\nimprove the performance of sequential decision-making agents. We present the\nWorking Memory Graph (WMG), an agent that employs multi-head self-attention to\nreason over a dynamic set of vectors representing observed and recurrent state.\nWe evaluate WMG in three environments featuring factored observation spaces: a\nPathfinding environment that requires complex reasoning over past observations,\nBabyAI gridworld levels that involve variable goals, and Sokoban which\nemphasizes future planning. We find that the combination of WMG's\nTransformer-based architecture with factored observation spaces leads to\nsignificant gains in learning efficiency compared to baseline architectures\nacross all tasks. WMG demonstrates how Transformer-based models can\ndramatically boost sample efficiency in RL environments for which observations\ncan be factored.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.07199v1",
            "date": "2019-11-17 09:40:24+00:00",
            "title": "Rumor Detection on Social Media: Datasets, Methods and Opportunities",
            "authors": [
                "Quanzhi Li",
                "Qiong Zhang",
                "Luo Si",
                "Yingchi Liu"
            ],
            "abstract": "Social media platforms have been used for information and news gathering, and\nthey are very valuable in many applications. However, they also lead to the\nspreading of rumors and fake news. Many efforts have been taken to detect and\ndebunk rumors on social media by analyzing their content and social context\nusing machine learning techniques. This paper gives an overview of the recent\nstudies in the rumor detection field. It provides a comprehensive list of\ndatasets used for rumor detection, and reviews the important studies based on\nwhat types of information they exploit and the approaches they take. And more\nimportantly, we also present several new directions for future research.",
            "journal ref": "EMNLP 2019",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.07223v1",
            "date": "2019-11-17 12:32:50+00:00",
            "title": "Deep Learning versus Traditional Classifiers on Vietnamese Students' Feedback Corpus",
            "authors": [
                "Phu X. V. Nguyen",
                "Tham T. T. Hong",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Student's feedback is an important source of collecting students' opinions to\nimprove the quality of training activities. Implementing sentiment analysis\ninto student feedback data, we can determine sentiments polarities which\nexpress all problems in the institution since changes necessary will be applied\nto improve the quality of teaching and learning. This study focused on machine\nlearning and natural language processing techniques (NaiveBayes, Maximum\nEntropy, Long Short-Term Memory, Bi-Directional Long Short-Term Memory) on the\nVietnameseStudents' Feedback Corpus collected from a university. The final\nresults were compared and evaluated to find the most effective model based on\ndifferent evaluation criteria. The experimental results show that the\nBi-Directional LongShort-Term Memory algorithm outperformed than three other\nalgorithms in terms of the F1-score measurement with 92.0% on the sentiment\nclassification task and 89.6% on the topic classification task. In addition, we\ndeveloped a sentiment analysis application analyzing student feedback. The\napplication will help the institution to recognize students' opinions about a\nproblem and identify shortcomings that still exist. With the use of this\napplication, the institution can propose an appropriate method to improve the\nquality of training activities in the future.",
            "journal ref": "5th NAFOSTED Conference on Information and Computer Science (NICS\n  2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07228v2",
            "date": "2019-11-17 13:03:07+00:00",
            "title": "Error Analysis for Vietnamese Named Entity Recognition on Deep Neural Network Models",
            "authors": [
                "Binh An Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In recent years, Vietnamese Named Entity Recognition (NER) systems have had a\ngreat breakthrough when using Deep Neural Network methods. This paper describes\nthe primary errors of the state-of-the-art NER systems on Vietnamese language.\nAfter conducting experiments on BLSTM-CNN-CRF and BLSTM-CRF models with\ndifferent word embeddings on the Vietnamese NER dataset. This dataset is\nprovided by VLSP in 2016 and used to evaluate most of the current Vietnamese\nNER systems. We noticed that BLSTM-CNN-CRF gives better results, therefore, we\nanalyze the errors on this model in detail. Our error-analysis results provide\nus thorough insights in order to increase the performance of NER for the\nVietnamese language and improve the quality of the corpus in the future works.",
            "journal ref": "19th International Conference on Computational Linguistics and\n  Intelligent Text Processing (CICLING 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07335v2",
            "date": "2019-11-17 20:41:32+00:00",
            "title": "Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition",
            "authors": [
                "Haw-Shiuan Chang",
                "Shankar Vembu",
                "Sunil Mohan",
                "Rheeya Uppaal",
                "Andrew McCallum"
            ],
            "abstract": "Existing deep active learning algorithms achieve impressive sampling\nefficiency on natural language processing tasks. However, they exhibit several\nweaknesses in practice, including (a) inability to use uncertainty sampling\nwith black-box models, (b) lack of robustness to labeling noise, and (c) lack\nof transparency. In response, we propose a transparent batch active sampling\nframework by estimating the error decay curves of multiple feature-defined\nsubsets of the data. Experiments on four named entity recognition (NER) tasks\ndemonstrate that the proposed methods significantly outperform\ndiversification-based methods for black-box NER taggers, and can make the\nsampling process more robust to labeling noise when combined with\nuncertainty-based methods. Furthermore, the analysis of experimental results\nsheds light on the weaknesses of different active sampling strategies, and when\ntraditional uncertainty-based or diversification-based methods can be expected\nto work well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07405v1",
            "date": "2019-11-18 03:11:36+00:00",
            "title": "Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems",
            "authors": [
                "Qiang Huang",
                "Jianhui Bu",
                "Weijian Xie",
                "Shengwen Yang",
                "Weijia Wu",
                "Liping Liu"
            ],
            "abstract": "Question Answering (QA) systems are used to provide proper responses to\nusers' questions automatically. Sentence matching is an essential task in the\nQA systems and is usually reformulated as a Paraphrase Identification (PI)\nproblem. Given a question, the aim of the task is to find the most similar\nquestion from a QA knowledge base. In this paper, we propose a Multi-task\nSentence Encoding Model (MSEM) for the PI problem, wherein a connected graph is\nemployed to depict the relation between sentences, and a multi-task learning\nmodel is applied to address both the sentence matching and sentence intent\nclassification problem. In addition, we implement a general semantic retrieval\nframework that combines our proposed model and the Approximate Nearest Neighbor\n(ANN) technology, which enables us to find the most similar question from all\navailable candidates very quickly during online serving. The experiments show\nthe superiority of our proposed method as compared with the existing sentence\nmatching models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07629v1",
            "date": "2019-11-15 09:20:32+00:00",
            "title": "Selection-based Question Answering of an MOOC",
            "authors": [
                "Atul Sahay",
                "Smita Gholkar",
                "Kavi Arya"
            ],
            "abstract": "e-Yantra Robotics Competition (eYRC) is a unique Robotics Competition hosted\nby IIT Bombay that is actually an Embedded Systems and Robotics MOOC.\nRegistrations have been growing exponentially in each year from 4500 in 2012 to\nover 34000 in 2019. In this 5-month long competition students learn complex\nskills under severe time pressure and have access to a discussion forum to post\ndoubts about the learning material. Responding to questions in real-time is a\nchallenge for project staff. Here, we illustrate the advantage of Deep Learning\nfor real-time question answering in the eYRC discussion forum. We illustrate\nthe advantage of Transformer based contextual embedding mechanisms such as\nBidirectional Encoder Representation From Transformer (BERT) over word\nembedding mechanisms such as Word2Vec. We propose a weighted similarity metric\nas a measure of matching and find it more reliable than Content-Content or\nTitle-Title similarities alone. The automation of replying to questions has\nbrought the turn around response time(TART) down from a minimum of 21 mins to a\nminimum of 0.3 secs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07819v2",
            "date": "2019-11-18 18:32:25+00:00",
            "title": "Drug Repurposing for Cancer: An NLP Approach to Identify Low-Cost Therapies",
            "authors": [
                "Shivashankar Subramanian",
                "Ioana Baldini",
                "Sushma Ravichandran",
                "Dmitriy A. Katz-Rogozhnikov",
                "Karthikeyan Natesan Ramamurthy",
                "Prasanna Sattigeri",
                "Kush R. Varshney",
                "Annmarie Wang",
                "Pradeep Mangalath",
                "Laura B. Kleiman"
            ],
            "abstract": "More than 200 generic drugs approved by the U.S. Food and Drug Administration\nfor non-cancer indications have shown promise for treating cancer. Due to their\nlong history of safe patient use, low cost, and widespread availability,\nrepurposing of generic drugs represents a major opportunity to rapidly improve\noutcomes for cancer patients and reduce healthcare costs worldwide. Evidence on\nthe efficacy of non-cancer generic drugs being tested for cancer exists in\nscientific publications, but trying to manually identify and extract such\nevidence is intractable. In this paper, we introduce a system to automate this\nevidence extraction from PubMed abstracts. Our primary contribution is to\ndefine the natural language processing pipeline required to obtain such\nevidence, comprising the following modules: querying, filtering, cancer type\nentity extraction, therapeutic association classification, and study type\nclassification. Using the subject matter expertise on our team, we create our\nown datasets for these specialized domain-specific tasks. We obtain promising\nperformance in each of the modules by utilizing modern language modeling\ntechniques and plan to treat them as baseline approaches for future improvement\nof individual components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.07918v1",
            "date": "2019-11-18 20:30:06+00:00",
            "title": "Improving Document Classification with Multi-Sense Embeddings",
            "authors": [
                "Vivek Gupta",
                "Ankit Saw",
                "Pegah Nokhiz",
                "Harshit Gupta",
                "Partha Talukdar"
            ],
            "abstract": "Efficient representation of text documents is an important building block in\nmany NLP tasks. Research on long text categorization has shown that simple\nweighted averaging of word vectors for sentence representation often\noutperforms more sophisticated neural models. Recently proposed Sparse\nComposite Document Vector (SCDV) (Mekala et. al, 2017) extends this approach\nfrom sentences to documents using soft clustering over word vectors. However,\nSCDV disregards the multi-sense nature of words, and it also suffers from the\ncurse of higher dimensionality. In this work, we address these shortcomings and\npropose SCDV-MS. SCDV-MS utilizes multi-sense word embeddings and learns a\nlower dimensional manifold. Through extensive experiments on multiple\nreal-world datasets, we show that SCDV-MS embeddings outperform previous\nstate-of-the-art embeddings on multi-class and multi-label text categorization\ntasks. Furthermore, SCDV-MS embeddings are more efficient than SCDV in terms of\ntime and space complexity on textual classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08112v1",
            "date": "2019-11-19 06:38:14+00:00",
            "title": "Extended Answer and Uncertainty Aware Neural Question Generation",
            "authors": [
                "Hongwei Zeng",
                "Zhuo Zhi",
                "Jun Liu",
                "Bifan Wei"
            ],
            "abstract": "In this paper, we study automatic question generation, the task of creating\nquestions from corresponding text passages where some certain spans of the text\ncan serve as the answers. We propose an Extended Answer-aware Network (EAN)\nwhich is trained with Word-based Coverage Mechanism (WCM) and decodes with\nUncertainty-aware Beam Search (UBS). The EAN represents the target answer by\nits surrounding sentence with an encoder, and incorporates the information of\nthe extended answer into paragraph representation with gated\nparagraph-to-answer attention to tackle the problem of the inadequate\nrepresentation of the target answer. To reduce undesirable repetition, the WCM\npenalizes repeatedly attending to the same words at different time-steps in the\ntraining stage. The UBS aims to seek a better balance between the model\nconfidence in copying words from an input text paragraph and the confidence in\ngenerating words from a vocabulary. We conduct experiments on the SQuAD\ndataset, and the results show our approach achieves significant performance\nimprovement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08113v1",
            "date": "2019-11-19 06:42:23+00:00",
            "title": "Hunting for Troll Comments in News Community Forums",
            "authors": [
                "Todor Mihaylov",
                "Preslav Nakov"
            ],
            "abstract": "There are different definitions of what a troll is. Certainly, a troll can be\nsomebody who teases people to make them angry, or somebody who offends people,\nor somebody who wants to dominate any single discussion, or somebody who tries\nto manipulate people's opinion (sometimes for money), etc. The last definition\nis the one that dominates the public discourse in Bulgaria and Eastern Europe,\nand this is our focus in this paper. In our work, we examine two types of\nopinion manipulation trolls: paid trolls that have been revealed from leaked\nreputation management contracts and mentioned trolls that have been called such\nby several different people. We show that these definitions are sensible: we\nbuild two classifiers that can distinguish a post by such a paid troll from one\nby a non-troll with 81-82% accuracy; the same classifier achieves 81-82%\naccuracy on so called mentioned troll vs. non-troll posts.",
            "journal ref": "ACL-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08125v1",
            "date": "2019-11-19 07:06:22+00:00",
            "title": "In Search of Credible News",
            "authors": [
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "We study the problem of finding fake online news. This is an important\nproblem as news of questionable credibility have recently been proliferating in\nsocial media at an alarming scale. As this is an understudied problem,\nespecially for languages other than English, we first collect and release to\nthe research community three new balanced credible vs. fake news datasets\nderived from four online sources. We then propose a language-independent\napproach for automatically distinguishing credible from fake news, based on a\nrich feature set. In particular, we use linguistic (n-gram),\ncredibility-related (capitalization, punctuation, pronoun use, sentiment\npolarity), and semantic (embeddings and DBPedia data) features. Our experiments\non three different testsets show that our model can distinguish credible from\nfake news with very high accuracy.",
            "journal ref": "AIMSA-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08151v2",
            "date": "2019-11-19 08:20:45+00:00",
            "title": "Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response Generation",
            "authors": [
                "Jiahuan Pei",
                "Pengjie Ren",
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "Dialogue response generation (DRG) is a critical component of task-oriented\ndialogue systems (TDSs). Its purpose is to generate proper natural language\nresponses given some context, e.g., historical utterances, system states, etc.\nState-of-the-art work focuses on how to better tackle DRG in an end-to-end way.\nTypically, such studies assume that each token is drawn from a single\ndistribution over the output vocabulary, which may not always be optimal.\nResponses vary greatly with different intents, e.g., domains, system actions.\n  We propose a novel mixture-of-generators network (MoGNet) for DRG, where we\nassume that each token of a response is drawn from a mixture of distributions.\nMoGNet consists of a chair generator and several expert generators. Each expert\nis specialized for DRG w.r.t. a particular intent. The chair coordinates\nmultiple experts and combines the output they have generated to produce more\nappropriate responses. We propose two strategies to help the chair make better\ndecisions, namely, a retrospective mixture-of-generators (RMoG) and prospective\nmixture-of-generators (PMoG). The former only considers the historical\nexpert-generated responses until the current time step while the latter also\nconsiders possible expert-generated responses in the future by encouraging\nexploration. In order to differentiate experts, we also devise a\nglobal-and-local (GL) learning scheme that forces each expert to be specialized\ntowards a particular intent using a local loss and trains the chair and all\nexperts to coordinate using a global loss.\n  We carry out extensive experiments on the MultiWOZ benchmark dataset. MoGNet\nsignificantly outperforms state-of-the-art methods in terms of both automatic\nand human evaluations, demonstrating its effectiveness for DRG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08340v1",
            "date": "2019-11-19 15:18:39+00:00",
            "title": "Unsupervised Natural Question Answering with a Small Model",
            "authors": [
                "Martin Andrews",
                "Sam Witteveen"
            ],
            "abstract": "The recent (2019-02) demonstration of the power of huge language models such\nas GPT-2 to memorise the answers to factoid questions raises questions about\nthe extent to which knowledge is being embedded directly within these large\nmodels. This short paper describes an architecture through which much smaller\nmodels can also answer such questions - by making use of 'raw' external\nknowledge. The contribution of this work is that the methods presented here\nrely on unsupervised learning techniques, complementing the unsupervised\ntraining of the Language Model. The goal of this line of research is to be able\nto add knowledge explicitly, without extensive training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08437v1",
            "date": "2019-11-19 18:02:06+00:00",
            "title": "Towards unstructured mortality prediction with free-text clinical notes",
            "authors": [
                "Mohammad Hashir",
                "Rapinder Sawhney"
            ],
            "abstract": "Healthcare data continues to flourish yet a relatively small portion, mostly\nstructured, is being utilized effectively for predicting clinical outcomes. The\nrich subjective information available in unstructured clinical notes can\npossibly facilitate higher discrimination but tends to be under-utilized in\nmortality prediction. This work attempts to assess the gain in performance when\nmultiple notes that have been minimally preprocessed are used as an input for\nprediction. A hierarchical architecture consisting of both convolutional and\nrecurrent layers is used to concurrently model the different notes compiled in\nan individual hospital stay. This approach is evaluated on predicting\nin-hospital mortality on the MIMIC-III dataset. On comparison to approaches\nutilizing structured data, it achieved higher metrics despite requiring less\ncleaning and preprocessing. This demonstrates the potential of unstructured\ndata in enhancing mortality prediction and signifies the need to incorporate\nmore raw unstructured data into current clinical prediction methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.08460v3",
            "date": "2019-11-19 18:40:02+00:00",
            "title": "End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures",
            "authors": [
                "Gabriel Synnaeve",
                "Qiantong Xu",
                "Jacob Kahn",
                "Tatiana Likhomanenko",
                "Edouard Grave",
                "Vineel Pratap",
                "Anuroop Sriram",
                "Vitaliy Liptchinsky",
                "Ronan Collobert"
            ],
            "abstract": "We study pseudo-labeling for the semi-supervised training of ResNet,\nTime-Depth Separable ConvNets, and Transformers for speech recognition, with\neither CTC or Seq2Seq loss functions. We perform experiments on the standard\nLibriSpeech dataset, and leverage additional unlabeled data from LibriVox\nthrough pseudo-labeling. We show that while Transformer-based acoustic models\nhave superior performance with the supervised dataset alone, semi-supervision\nimproves all models across architectures and loss functions and bridges much of\nthe performance gaps between them. In doing so, we reach a new state-of-the-art\nfor end-to-end acoustic models decoded with an external language model in the\nstandard supervised learning setting, and a new absolute state-of-the-art with\nsemi-supervised training. Finally, we study the effect of leveraging different\namounts of unlabeled audio, propose several ways of evaluating the\ncharacteristics of unlabeled audio which improve acoustic modeling, and show\nthat acoustic models trained with more audio rely less on external language\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08522v2",
            "date": "2019-11-19 19:34:15+00:00",
            "title": "Aging Memories Generate More Fluent Dialogue Responses with Memory Augmented Neural Networks",
            "authors": [
                "Omar U. Florez",
                "Erik Mueller"
            ],
            "abstract": "Memory Networks have emerged as effective models to incorporate Knowledge\nBases (KB) into neural networks. By storing KB embeddings into a memory\ncomponent, these models can learn meaningful representations that are grounded\nto external knowledge. However, as the memory unit becomes full, the oldest\nmemories are replaced by newer representations.\n  In this paper, we question this approach and provide experimental evidence\nthat conventional Memory Networks store highly correlated vectors during\ntraining. While increasing the memory size mitigates this problem, this also\nleads to overfitting as the memory stores a large number of training latent\nrepresentations. To address these issues, we propose a novel regularization\nmechanism named memory dropout which 1) Samples a single latent vector from the\ndistribution of redundant memories. 2) Ages redundant memories thus increasing\ntheir probability of overwriting them during training. This fully\ndifferentiable technique allows us to achieve state-of-the-art response\ngeneration in the Stanford Multi-Turn Dialogue and Cambridge Restaurant\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08554v1",
            "date": "2019-11-16 01:58:27+00:00",
            "title": "Classification as Decoder: Trading Flexibility for Control in Medical Dialogue",
            "authors": [
                "Sam Shleifer",
                "Manish Chablani",
                "Anitha Kannan",
                "Namit Katariya",
                "Xavier Amatriain"
            ],
            "abstract": "Generative seq2seq dialogue systems are trained to predict the next word in\ndialogues that have already occurred. They can learn from large unlabeled\nconversation datasets, build a deeper understanding of conversational context,\nand generate a wide variety of responses. This flexibility comes at the cost of\ncontrol, a concerning tradeoff in doctor/patient interactions. Inaccuracies,\ntypos, or undesirable content in the training data will be reproduced by the\nmodel at inference time. We trade a small amount of labeling effort and some\nloss of response variety in exchange for quality control. More specifically, a\npretrained language model encodes the conversational context, and we finetune a\nclassification head to map an encoded conversational context to a response\nclass, where each class is a noisily labeled group of interchangeable\nresponses. Experts can update these exemplar responses over time as best\npractices change without retraining the classifier or invalidating old training\ndata. Expert evaluation of 775 unseen doctor/patient conversations shows that\nonly 12% of the discriminative model's responses are worse than the what the\ndoctor ended up writing, compared to 18% for the generative model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08743v1",
            "date": "2019-11-20 07:16:16+00:00",
            "title": "SemanticZ at SemEval-2016 Task 3: Ranking Relevant Answers in Community Question Answering Using Semantic Similarity Based on Fine-tuned Word Embeddings",
            "authors": [
                "Todor Mihaylov",
                "Preslav Nakov"
            ],
            "abstract": "We describe our system for finding good answers in a community forum, as\ndefined in SemEval-2016, Task 3 on Community Question Answering. Our approach\nrelies on several semantic similarity features based on fine-tuned word\nembeddings and topics similarities. In the main Subtask C, our primary\nsubmission was ranked third, with a MAP of 51.68 and accuracy of 69.94. In\nSubtask A, our primary submission was also third, with MAP of 77.58 and\naccuracy of 73.39.",
            "journal ref": "SemEval-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08755v1",
            "date": "2019-11-20 08:09:36+00:00",
            "title": "Global Thread-Level Inference for Comment Classification in Community Question Answering",
            "authors": [
                "Shafiq Joty",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Giovanni Da San Martino",
                "Simone Filice",
                "Llu\u00eds M\u00e0rquez",
                "Alessandro Moschitti",
                "Preslav Nakov"
            ],
            "abstract": "Community question answering, a recent evolution of question answering in the\nWeb context, allows a user to quickly consult the opinion of a number of people\non a particular topic, thus taking advantage of the wisdom of the crowd. Here\nwe try to help the user by deciding automatically which answers are good and\nwhich are bad for a given question. In particular, we focus on exploiting the\noutput structure at the thread level in order to make more consistent global\ndecisions. More specifically, we exploit the relations between pairs of\ncomments at any distance in the thread, which we incorporate in a graph-cut and\nin an ILP frameworks. We evaluated our approach on the benchmark dataset of\nSemEval-2015 Task 3. Results improved over the state of the art, confirming the\nimportance of using thread level information.",
            "journal ref": "EMNLP-2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08762v1",
            "date": "2019-11-20 08:29:10+00:00",
            "title": "Paraphrasing Verbs for Noun Compound Interpretation",
            "authors": [
                "Preslav Nakov"
            ],
            "abstract": "An important challenge for the automatic analysis of English written text is\nthe abundance of noun compounds: sequences of nouns acting as a single noun. In\nour view, their semantics is best characterized by the set of all possible\nparaphrasing verbs, with associated weights, e.g., malaria mosquito is carry\n(23), spread (16), cause (12), transmit (9), etc. Using Amazon's Mechanical\nTurk, we collect paraphrasing verbs for 250 noun-noun compounds previously\nproposed in the linguistic literature, thus creating a valuable resource for\nnoun compound interpretation. Using these verbs, we further construct a dataset\nof pairs of sentences representing a special kind of textual entailment task,\nwhere a binary decision is to be made about whether an expression involving a\nverb and two nouns can be transformed into a noun compound, while preserving\nthe sentence meaning.",
            "journal ref": "MWE-2008",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08776v2",
            "date": "2019-11-20 09:05:11+00:00",
            "title": "Joint Embedding Learning of Educational Knowledge Graphs",
            "authors": [
                "Siyu Yao",
                "Ruijie Wang",
                "Shen Sun",
                "Derui Bu",
                "Jun Liu"
            ],
            "abstract": "As an efficient model for knowledge organization, the knowledge graph has\nbeen widely adopted in several fields, e.g., biomedicine, sociology, and\neducation. And there is a steady trend of learning embedding representations of\nknowledge graphs to facilitate knowledge graph construction and downstream\ntasks. In general, knowledge graph embedding techniques aim to learn vectorized\nrepresentations which preserve the structural information of the graph. And\nconventional embedding learning models rely on structural relationships among\nentities and relations. However, in educational knowledge graphs, structural\nrelationships are not the focus. Instead, rich literals of the graphs are more\nvaluable. In this paper, we focus on this problem and propose a novel model for\nembedding learning of educational knowledge graphs. Our model considers both\nstructural and literal information and jointly learns embedding\nrepresentations. Three experimental graphs were constructed based on an\neducational knowledge graph which has been applied in real-world teaching. We\nconducted two experiments on the three graphs and other common benchmark\ngraphs. The experimental results proved the effectiveness of our model and its\nsuperiority over other baselines when processing educational knowledge graphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08779v1",
            "date": "2019-11-20 09:12:58+00:00",
            "title": "Characterizing Scalability of Sparse Matrix-Vector Multiplications on Phytium FT-2000+ Many-cores",
            "authors": [
                "Donglin Chen",
                "Jianbin Fang",
                "Chuanfu Xu",
                "Shizhao Chen",
                "Zheng Wang"
            ],
            "abstract": "Understanding the scalability of parallel programs is crucial for software\noptimization and hardware architecture design. As HPC hardware is moving\ntowards many-core design, it becomes increasingly difficult for a parallel\nprogram to make effective use of all available processor cores. This makes\nscalability analysis increasingly important. This paper presents a quantitative\nstudy for characterizing the scalability of sparse matrix-vector\nmultiplications (SpMV) on Phytium FT-2000+, an ARM-based many-core architecture\nfor HPC computing. We choose to study SpMV as it is a common operation in\nscientific and HPC applications. Due to the newness of ARM-based many-core\narchitectures, there is little work on understanding the SpMV scalability on\nsuch hardware design. To close the gap, we carry out a large-scale empirical\nevaluation involved over 1,000 representative SpMV datasets. We show that,\nwhile many computation-intensive SpMV applications contain extensive\nparallelism, achieving a linear speedup is non-trivial on Phytium FT-2000+. To\nbetter understand what software and hardware parameters are most important for\ndetermining the scalability of a given SpMV kernel, we develop a performance\nanalytical model based on the regression tree. We show that our model is highly\neffective in characterizing SpMV scalability, offering useful insights to help\napplication developers for better optimizing SpMV on an emerging HPC\narchitecture.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/1911.08870v1",
            "date": "2019-11-20 13:01:56+00:00",
            "title": "A Comparative Study on End-to-end Speech to Text Translation",
            "authors": [
                "Parnia Bahar",
                "Tobias Bieschke",
                "Hermann Ney"
            ],
            "abstract": "Recent advances in deep learning show that end-to-end speech to text\ntranslation model is a promising approach to direct the speech translation\nfield. In this work, we provide an overview of different end-to-end\narchitectures, as well as the usage of an auxiliary connectionist temporal\nclassification (CTC) loss for better convergence. We also investigate on\npre-training variants such as initializing different components of a model\nusing pre-trained models, and their impact on the final performance, which\ngives boosts up to 4% in BLEU and 5% in TER. Our experiments are performed on\n270h IWSLT TED-talks En->De, and 100h LibriSpeech Audiobooks En->Fr. We also\nshow improvements over the current end-to-end state-of-the-art systems on both\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08876v1",
            "date": "2019-11-20 13:11:41+00:00",
            "title": "On Using SpecAugment for End-to-End Speech Translation",
            "authors": [
                "Parnia Bahar",
                "Albert Zeyer",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "This work investigates a simple data augmentation technique, SpecAugment, for\nend-to-end speech translation. SpecAugment is a low-cost implementation method\napplied directly to the audio input features and it consists of masking blocks\nof frequency channels, and/or time steps. We apply SpecAugment on end-to-end\nspeech translation tasks and achieve up to +2.2\\% \\BLEU on LibriSpeech\nAudiobooks En->Fr and +1.2% on IWSLT TED-talks En->De by alleviating\noverfitting to some extent. We also examine the effectiveness of the method in\na variety of data scenarios and show that the method also leads to significant\nimprovements in various data conditions irrespective of the amount of training\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08888v1",
            "date": "2019-11-20 13:25:06+00:00",
            "title": "On using 2D sequence-to-sequence models for speech recognition",
            "authors": [
                "Parnia Bahar",
                "Albert Zeyer",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "Attention-based sequence-to-sequence models have shown promising results in\nautomatic speech recognition. Using these architectures, one-dimensional input\nand output sequences are related by an attention approach, thereby replacing\nmore explicit alignment processes, like in classical HMM-based modeling. In\ncontrast, here we apply a novel two-dimensional long short-term memory (2DLSTM)\narchitecture to directly model the input/output relation between audio/feature\nvector sequences and word sequences. The proposed model is an alternative model\nsuch that instead of using any type of attention components, we apply a 2DLSTM\nlayer to assimilate the context from both input observations and output\ntranscriptions. The experimental evaluation on the Switchboard 300h automatic\nspeech recognition task shows word error rates for the 2DLSTM model that are\ncompetitive to end-to-end attention-based model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08895v2",
            "date": "2019-11-20 13:28:34+00:00",
            "title": "Demystifying TasNet: A Dissecting Approach",
            "authors": [
                "Jens Heitkaemper",
                "Darius Jakobeit",
                "Christoph Boeddeker",
                "Lukas Drude",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "In recent years time domain speech separation has excelled over frequency\ndomain separation in single channel scenarios and noise-free environments. In\nthis paper we dissect the gains of the time-domain audio separation network\n(TasNet) approach by gradually replacing components of an utterance-level\npermutation invariant training (u-PIT) based separation system in the frequency\ndomain until the TasNet system is reached, thus blending components of\nfrequency domain approaches with those of time domain approaches. Some of the\nintermediate variants achieve comparable signal-to-distortion ratio (SDR) gains\nto TasNet, but retain the advantage of frequency domain processing:\ncompatibility with classic signal processing tools such as frequency-domain\nbeamforming and the human interpretability of the masks. Furthermore, we show\nthat the scale invariant signal-to-distortion ratio (si-SDR) criterion used as\nloss function in TasNet is related to a logarithmic mean square error criterion\nand that it is this criterion which contributes most reliable to the\nperformance advantage of TasNet. Finally, we critically assess which gains in a\nnoise-free single channel environment generalize to more realistic reverberant\nconditions.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1911.08935v2",
            "date": "2019-11-20 14:38:58+00:00",
            "title": "Rule-Guided Compositional Representation Learning on Knowledge Graphs",
            "authors": [
                "Guanglin Niu",
                "Yongfei Zhang",
                "Bo Li",
                "Peng Cui",
                "Si Liu",
                "Jingyang Li",
                "Xiaowei Zhang"
            ],
            "abstract": "Representation learning on a knowledge graph (KG) is to embed entities and\nrelations of a KG into low-dimensional continuous vector spaces. Early KG\nembedding methods only pay attention to structured information encoded in\ntriples, which would cause limited performance due to the structure sparseness\nof KGs. Some recent attempts consider paths information to expand the structure\nof KGs but lack explainability in the process of obtaining the path\nrepresentations. In this paper, we propose a novel Rule and Path-based Joint\nEmbedding (RPJE) scheme, which takes full advantage of the explainability and\naccuracy of logic rules, the generalization of KG embedding as well as the\nsupplementary semantic structure of paths. Specifically, logic rules of\ndifferent lengths (the number of relations in rule body) in the form of Horn\nclauses are first mined from the KG and elaborately encoded for representation\nlearning. Then, the rules of length 2 are applied to compose paths accurately\nwhile the rules of length 1 are explicitly employed to create semantic\nassociations among relations and constrain relation embeddings. Besides, the\nconfidence level of each rule is also considered in optimization to guarantee\nthe availability of applying the rule to representation learning. Extensive\nexperimental results illustrate that RPJE outperforms other state-of-the-art\nbaselines on KG completion task, which also demonstrate the superiority of\nutilizing logic rules as well as paths for improving the accuracy and\nexplainability of representation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08936v1",
            "date": "2019-11-20 14:40:23+00:00",
            "title": "Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation",
            "authors": [
                "Zequn Sun",
                "Chengming Wang",
                "Wei Hu",
                "Muhao Chen",
                "Jian Dai",
                "Wei Zhang",
                "Yuzhong Qu"
            ],
            "abstract": "Graph neural networks (GNNs) have emerged as a powerful paradigm for\nembedding-based entity alignment due to their capability of identifying\nisomorphic subgraphs. However, in real knowledge graphs (KGs), the counterpart\nentities usually have non-isomorphic neighborhood structures, which easily\ncauses GNNs to yield different representations for them. To tackle this\nproblem, we propose a new KG alignment network, namely AliNet, aiming at\nmitigating the non-isomorphism of neighborhood structures in an end-to-end\nmanner. As the direct neighbors of counterpart entities are usually dissimilar\ndue to the schema heterogeneity, AliNet introduces distant neighbors to expand\nthe overlap between their neighborhood structures. It employs an attention\nmechanism to highlight helpful distant neighbors and reduce noises. Then, it\ncontrols the aggregation of both direct and distant neighborhood information\nusing a gating mechanism. We further propose a relation loss to refine entity\nrepresentations. We perform thorough experiments with detailed ablation studies\nand analyses on five entity alignment datasets, demonstrating the effectiveness\nof AliNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08976v1",
            "date": "2019-11-20 15:41:47+00:00",
            "title": "Red Dragon AI at TextGraphs 2019 Shared Task: Language Model Assisted Explanation Generation",
            "authors": [
                "Yew Ken Chia",
                "Sam Witteveen",
                "Martin Andrews"
            ],
            "abstract": "The TextGraphs-13 Shared Task on Explanation Regeneration asked participants\nto develop methods to reconstruct gold explanations for elementary science\nquestions. Red Dragon AI's entries used the language of the questions and\nexplanation text directly, rather than a constructing a separate graph-like\nrepresentation. Our leaderboard submission placed us 3rd in the competition,\nbut we present here three methods of increasing sophistication, each of which\nscored successively higher on the test set after the competition close.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09194v2",
            "date": "2019-11-20 22:20:52+00:00",
            "title": "Generating Interactive Worlds with Text",
            "authors": [
                "Angela Fan",
                "Jack Urbanek",
                "Pratik Ringshia",
                "Emily Dinan",
                "Emma Qian",
                "Siddharth Karamcheti",
                "Shrimai Prabhumoye",
                "Douwe Kiela",
                "Tim Rocktaschel",
                "Arthur Szlam",
                "Jason Weston"
            ],
            "abstract": "Procedurally generating cohesive and interesting game environments is\nchallenging and time-consuming. In order for the relationships between the game\nelements to be natural, common-sense has to be encoded into arrangement of the\nelements. In this work, we investigate a machine learning approach for world\ncreation using content from the multi-player text adventure game environment\nLIGHT. We introduce neural network based models to compositionally arrange\nlocations, characters, and objects into a coherent whole. In addition to\ncreating worlds based on existing elements, our models can generate new game\ncontent. Humans can also leverage our models to interactively aid in\nworldbuilding. We show that the game environments created with our approach are\ncohesive, diverse, and preferred by human evaluators compared to other machine\nlearning based world construction algorithms.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1911.09242v2",
            "date": "2019-11-21 02:08:14+00:00",
            "title": "How Do You #relax When You're #stressed? A Content Analysis and Infodemiology Study of Stress-Related Tweets",
            "authors": [
                "Son Doan",
                "Amanda Ritchart",
                "Nicholas Perry",
                "Juan D Chaparro",
                "Mike Conway"
            ],
            "abstract": "Background: Stress is a contributing factor to many major health problems in\nthe United States, such as heart disease, depression, and autoimmune diseases.\nRelaxation is often recommended in mental health treatment as a frontline\nstrategy to reduce stress, thereby improving health conditions.\n  Objective: The objective of our study was to understand how people express\ntheir feelings of stress and relaxation through Twitter messages.\n  Methods: We first performed a qualitative content analysis of 1326 and 781\ntweets containing the keywords \"stress\" and \"relax\", respectively. We then\ninvestigated the use of machine learning algorithms to automatically classify\ntweets as stress versus non stress and relaxation versus non relaxation.\nFinally, we applied these classifiers to sample datasets drawn from 4 cities\nwith the goal of evaluating the extent of any correlation between our automatic\nclassification of tweets and results from public stress surveys.\n  Results: Content analysis showed that the most frequent topic of stress\ntweets was education, followed by work and social relationships. The most\nfrequent topic of relaxation tweets was rest and vacation, followed by nature\nand water. When we applied the classifiers to the cities dataset, the\nproportion of stress tweets in New York and San Diego was substantially higher\nthan that in Los Angeles and San Francisco.\n  Conclusions: This content analysis and infodemiology study revealed that\nTwitter, when used in conjunction with natural language processing techniques,\nis a useful data source for understanding stress and stress management\nstrategies, and can potentially supplement infrequently collected survey-based\nstress data.",
            "journal ref": "JMIR Public Health Surveill 2017;3(2):e35",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09345v1",
            "date": "2019-11-21 08:47:36+00:00",
            "title": "Empirical Autopsy of Deep Video Captioning Frameworks",
            "authors": [
                "Nayyer Aafaq",
                "Naveed Akhtar",
                "Wei Liu",
                "Ajmal Mian"
            ],
            "abstract": "Contemporary deep learning based video captioning follows encoder-decoder\nframework. In encoder, visual features are extracted with 2D/3D Convolutional\nNeural Networks (CNNs) and a transformed version of those features is passed to\nthe decoder. The decoder uses word embeddings and a language model to map\nvisual features to natural language captions. Due to its composite nature, the\nencoder-decoder pipeline provides the freedom of multiple choices for each of\nits components, e.g the choices of CNNs models, feature transformations, word\nembeddings, and language models etc. Component selection can have drastic\neffects on the overall video captioning performance. However, current\nliterature is void of any systematic investigation in this regard. This article\nfills this gap by providing the first thorough empirical analysis of the role\nthat each major component plays in a contemporary video captioning pipeline. We\nperform extensive experiments by varying the constituent components of the\nvideo captioning framework, and quantify the performance gains that are\npossible by mere component selection. We use the popular MSVD dataset as the\ntest-bed, and demonstrate that substantial performance gains are possible by\ncareful selection of the constituent components without major changes to the\npipeline itself. These results are expected to provide guiding principles for\nfuture research in the fast growing direction of video captioning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.09373v1",
            "date": "2019-11-21 10:01:16+00:00",
            "title": "Entity Extraction with Knowledge from Web Scale Corpora",
            "authors": [
                "Zeyi Wen",
                "Zeyu Huang",
                "Rui Zhang"
            ],
            "abstract": "Entity extraction is an important task in text mining and natural language\nprocessing. A popular method for entity extraction is by comparing substrings\nfrom free text against a dictionary of entities. In this paper, we present\nseveral techniques as a post-processing step for improving the effectiveness of\nthe existing entity extraction technique. These techniques utilise models\ntrained with the web-scale corpora which makes our techniques robust and\nversatile. Experiments show that our techniques bring a notable improvement on\nefficiency and effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09375v1",
            "date": "2019-11-21 10:03:25+00:00",
            "title": "ChartNet: Visual Reasoning over Statistical Charts using MAC-Networks",
            "authors": [
                "Monika Sharma",
                "Shikha Gupta",
                "Arindam Chowdhury",
                "Lovekesh Vig"
            ],
            "abstract": "Despite the improvements in perception accuracies brought about via deep\nlearning, developing systems combining accurate visual perception with the\nability to reason over the visual percepts remains extremely challenging. A\nparticular application area of interest from an accessibility perspective is\nthat of reasoning over statistical charts such as bar and pie charts. To this\nend, we formulate the problem of reasoning over statistical charts as a\nclassification task using MAC-Networks to give answers from a predefined\nvocabulary of generic answers. Additionally, we enhance the capabilities of\nMAC-Networks to give chart-specific answers to open-ended questions by\nreplacing the classification layer by a regression layer to localize the\ntextual answers present over the images. We call our network ChartNet, and\ndemonstrate its efficacy on predicting both in vocabulary and out of vocabulary\nanswers. To test our methods, we generated our own dataset of statistical chart\nimages and corresponding question answer pairs. Results show that ChartNet\nconsistently outperform other state-of-the-art methods on reasoning over these\nquestions and may be a viable candidate for applications containing images of\nstatistical charts.",
            "journal ref": "International Joint Conference on Neural Networks (IJCNN) 2019",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.09419v2",
            "date": "2019-11-21 11:37:18+00:00",
            "title": "Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction",
            "authors": [
                "Zhanqiu Zhang",
                "Jianyu Cai",
                "Yongdong Zhang",
                "Jie Wang"
            ],
            "abstract": "Knowledge graph embedding, which aims to represent entities and relations as\nlow dimensional vectors (or matrices, tensors, etc.), has been shown to be a\npowerful technique for predicting missing links in knowledge graphs. Existing\nknowledge graph embedding models mainly focus on modeling relation patterns\nsuch as symmetry/antisymmetry, inversion, and composition. However, many\nexisting approaches fail to model semantic hierarchies, which are common in\nreal-world applications. To address this challenge, we propose a novel\nknowledge graph embedding model---namely, Hierarchy-Aware Knowledge Graph\nEmbedding (HAKE)---which maps entities into the polar coordinate system. HAKE\nis inspired by the fact that concentric circles in the polar coordinate system\ncan naturally reflect the hierarchy. Specifically, the radial coordinate aims\nto model entities at different levels of the hierarchy, and entities with\nsmaller radii are expected to be at higher levels; the angular coordinate aims\nto distinguish entities at the same level of the hierarchy, and these entities\nare expected to have roughly the same radii but different angles. Experiments\ndemonstrate that HAKE can effectively model the semantic hierarchies in\nknowledge graphs, and significantly outperforms existing state-of-the-art\nmethods on benchmark datasets for the link prediction task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.09572v1",
            "date": "2019-11-21 16:05:02+00:00",
            "title": "Automatically Generating Macro Research Reports from a Piece of News",
            "authors": [
                "Wenxin Hu",
                "Xiaofeng Zhang",
                "Gang Yang"
            ],
            "abstract": "Automatically generating macro research reports from economic news is an\nimportant yet challenging task. As we all know, it requires the macro analysts\nto write such reports within a short period of time after the important\neconomic news are released. This motivates our work, i.e., using AI techniques\nto save manual cost. The goal of the proposed system is to generate macro\nresearch reports as the draft for macro analysts. Essentially, the core\nchallenge is the long text generation issue. To address this issue, we propose\na novel deep learning technique based approach which includes two components,\ni.e., outline generation and macro research report generation.For the model\nperformance evaluation, we first crawl a large news-to-report dataset and then\nevaluate our approach on this dataset, and the generated reports are given for\nthe subjective evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09602v2",
            "date": "2019-11-21 16:54:14+00:00",
            "title": "Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech",
            "authors": [
                "David Harwath",
                "Wei-Ning Hsu",
                "James Glass"
            ],
            "abstract": "In this paper, we present a method for learning discrete linguistic units by\nincorporating vector quantization layers into neural models of visually\ngrounded speech. We show that our method is capable of capturing both\nword-level and sub-word units, depending on how it is configured. What\ndifferentiates this paper from prior work on speech unit learning is the choice\nof training objective. Rather than using a reconstruction-based loss, we use a\ndiscriminative, multimodal grounding objective which forces the learned units\nto be useful for semantic image retrieval. We evaluate the sub-word units on\nthe ZeroSpeech 2019 challenge, achieving a 27.3\\% reduction in ABX error rate\nover the top-performing submission, while keeping the bitrate approximately the\nsame. We also present experiments demonstrating the noise robustness of these\nunits. Finally, we show that a model with multiple quantizers can\nsimultaneously learn phone-like detectors at a lower layer and word-like\ndetectors at a higher layer. We show that these detectors are highly accurate,\ndiscovering 279 words with an F1 score of greater than 0.5.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09645v2",
            "date": "2019-11-21 18:04:47+00:00",
            "title": "Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features",
            "authors": [
                "Siddharth Gururani",
                "Kilol Gupta",
                "Dhaval Shah",
                "Zahra Shakeri",
                "Jervis Pinto"
            ],
            "abstract": "This paper presents a simple yet effective method to achieve prosody transfer\nfrom a reference speech signal to synthesized speech. The main idea is to\nincorporate well-known acoustic correlates of prosody such as pitch and\nloudness contours of the reference speech into a modern neural text-to-speech\n(TTS) synthesizer such as Tacotron2 (TC2). More specifically, a small set of\nacoustic features are extracted from reference audio and then used to condition\na TC2 synthesizer. The trained model is evaluated using subjective listening\ntests and a novel objective evaluation of prosody transfer is proposed.\nListening tests show that the synthesized speech is rated as highly natural and\nthat prosody is successfully transferred from the reference speech signal to\nthe synthesized signal.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1911.09655v1",
            "date": "2019-11-21 18:26:30+00:00",
            "title": "Temporal Reasoning via Audio Question Answering",
            "authors": [
                "Haytham M. Fayek",
                "Justin Johnson"
            ],
            "abstract": "Multimodal question answering tasks can be used as proxy tasks to study\nsystems that can perceive and reason about the world. Answering questions about\ndifferent types of input modalities stresses different aspects of reasoning\nsuch as visual reasoning, reading comprehension, story understanding, or\nnavigation. In this paper, we use the task of Audio Question Answering (AQA) to\nstudy the temporal reasoning abilities of machine learning models. To this end,\nwe introduce the Diagnostic Audio Question Answering (DAQA) dataset comprising\naudio sequences of natural sound events and programmatically generated\nquestions and answers that probe various aspects of temporal reasoning. We\nadapt several recent state-of-the-art methods for visual question answering to\nthe AQA task, and use DAQA to demonstrate that they perform poorly on questions\nthat require in-depth temporal reasoning. Finally, we propose a new model,\nMultiple Auxiliary Controllers for Linear Modulation (MALiMo) that extends the\nrecent Feature-wise Linear Modulation (FiLM) model and significantly improves\nits temporal reasoning capabilities. We envisage DAQA to foster research on AQA\nand temporal reasoning and MALiMo a step towards models for AQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09732v1",
            "date": "2019-11-21 20:19:28+00:00",
            "title": "Separate and Attend in Personal Email Search",
            "authors": [
                "Yu Meng",
                "Maryam Karimzadehgan",
                "Honglei Zhuang",
                "Donald Metzler"
            ],
            "abstract": "In personal email search, user queries often impose different requirements on\ndifferent aspects of the retrieved emails. For example, the query \"my recent\nflight to the US\" requires emails to be ranked based on both textual contents\nand recency of the email documents, while other queries such as \"medical\nhistory\" do not impose any constraints on the recency of the email. Recent deep\nlearning-to-rank models for personal email search often directly concatenate\ndense numerical features (e.g., document age) with embedded sparse features\n(e.g., n-gram embeddings). In this paper, we first show with a set of\nexperiments on synthetic datasets that direct concatenation of dense and sparse\nfeatures does not lead to the optimal search performance of deep neural ranking\nmodels. To effectively incorporate both sparse and dense email features into\npersonal email search ranking, we propose a novel neural model, SepAttn.\nSepAttn first builds two separate neural models to learn from sparse and dense\nfeatures respectively, and then applies an attention mechanism at the\nprediction level to derive the final prediction from these two models. We\nconduct a comprehensive set of experiments on a large-scale email search\ndataset, and demonstrate that our SepAttn model consistently improves the\nsearch quality over the baseline models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.09735v1",
            "date": "2019-11-21 20:26:29+00:00",
            "title": "Global Health Monitor: A Web-based System for Detecting and Mapping Infectious Diseases",
            "authors": [
                "Son Doan",
                "Quoc-Hung Ngo",
                "Ai Kawazoe",
                "Nigel Collier"
            ],
            "abstract": "We present the Global Health Monitor, an online Web-based system for\ndetecting and mapping infectious disease outbreaks that appear in news stories.\nThe system analyzes English news stories from news feed providers, classifies\nthem for topical relevance and plots them onto a Google map using geo-coding\ninformation, helping public health workers to monitor the spread of diseases in\na geo-temporal context. The background knowledge for the system is contained in\nthe BioCaster ontology (BCO) (Collier et al., 2007a) which includes both\ninformation on infectious diseases as well as geographical locations with their\nlatitudes/longitudes. The system consists of four main stages: topic\nclassification, named entity recognition (NER), disease/location detection and\nvisualization. Evaluation of the system shows that it achieved high accuracy on\na gold standard corpus. The system is now in practical use. Running on a\nclustercomputer, it monitors more than 1500 news feeds 24/7, updating the map\nevery hour.",
            "journal ref": "Proc. of the International Joint Conference on Natural Language\n  Processing (IJCNLP) 2008, pages 951-956",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.09762v2",
            "date": "2019-11-21 21:38:36+00:00",
            "title": "Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Models",
            "authors": [
                "Zhiyun Lu",
                "Liangliang Cao",
                "Yu Zhang",
                "Chung-Cheng Chiu",
                "James Fan"
            ],
            "abstract": "In this paper, we propose to use pre-trained features from end-to-end ASR\nmodels to solve speech sentiment analysis as a down-stream task. We show that\nend-to-end ASR features, which integrate both acoustic and text information\nfrom speech, achieve promising results. We use RNN with self-attention as the\nsentiment classifier, which also provides an easy visualization through\nattention weights to help interpret model predictions. We use well benchmarked\nIEMOCAP dataset and a new large-scale speech sentiment dataset SWBD-sentiment\nfor evaluation. Our approach improves the-state-of-the-art accuracy on IEMOCAP\nfrom 66.6% to 71.7%, and achieves an accuracy of 70.10% on SWBD-sentiment with\nmore than 49,500 utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09787v2",
            "date": "2019-11-21 23:55:15+00:00",
            "title": "LATTE: Latent Type Modeling for Biomedical Entity Linking",
            "authors": [
                "Ming Zhu",
                "Busra Celikkaya",
                "Parminder Bhatia",
                "Chandan K. Reddy"
            ],
            "abstract": "Entity linking is the task of linking mentions of named entities in natural\nlanguage text, to entities in a curated knowledge-base. This is of significant\nimportance in the biomedical domain, where it could be used to semantically\nannotate a large volume of clinical records and biomedical literature, to\nstandardized concepts described in an ontology such as Unified Medical Language\nSystem (UMLS). We observe that with precise type information, entity\ndisambiguation becomes a straightforward task. However, fine-grained type\ninformation is usually not available in biomedical domain. Thus, we propose\nLATTE, a LATent Type Entity Linking model, that improves entity linking by\nmodeling the latent fine-grained type information about mentions and entities.\nUnlike previous methods that perform entity linking directly between the\nmentions and the entities, LATTE jointly does entity disambiguation, and latent\nfine-grained type learning, without direct supervision. We evaluate our model\non two biomedical datasets: MedMentions, a large scale public dataset annotated\nwith UMLS concepts, and a de-identified corpus of dictated doctor's notes that\nhas been annotated with ICD concepts. Extensive experimental evaluation shows\nour model achieves significant performance improvements over several\nstate-of-the-art techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.09818v1",
            "date": "2019-11-22 02:39:41+00:00",
            "title": "Order Matters at Fanatics Recommending Sequentially Ordered Products by LSTM Embedded with Word2Vec",
            "authors": [
                "Jing Pan",
                "Weian Sheng",
                "Santanu Dey"
            ],
            "abstract": "A unique challenge for e-commerce recommendation is that customers are often\ninterested in products that are more advanced than their already purchased\nproducts, but not reversed. The few existing recommender systems modeling\nunidirectional sequence output a limited number of categories or continuous\nvariables. To model the ordered sequence, we design the first recommendation\nsystem that both embed purchased items with Word2Vec, and model the sequence\nwith stateless LSTM RNN. The click-through rate of this recommender system in\nproduction outperforms its solely Word2Vec based predecessor. Developed in\n2017, it was perhaps the first published real-world application that makes\ndistributed predictions of a single machine trained Keras model on Spark slave\nnodes at a scale of more than 0.4 million columns per row.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.09826v1",
            "date": "2019-11-22 03:14:32+00:00",
            "title": "Factorized Multimodal Transformer for Multimodal Sequential Learning",
            "authors": [
                "Amir Zadeh",
                "Chengfeng Mao",
                "Kelly Shi",
                "Yiwei Zhang",
                "Paul Pu Liang",
                "Soujanya Poria",
                "Louis-Philippe Morency"
            ],
            "abstract": "The complex world around us is inherently multimodal and sequential\n(continuous). Information is scattered across different modalities and requires\nmultiple continuous sensors to be captured. As machine learning leaps towards\nbetter generalization to real world, multimodal sequential learning becomes a\nfundamental research area. Arguably, modeling arbitrarily distributed\nspatio-temporal dynamics within and across modalities is the biggest challenge\nin this research area. In this paper, we present a new transformer model,\ncalled the Factorized Multimodal Transformer (FMT) for multimodal sequential\nlearning. FMT inherently models the intramodal and intermodal (involving two or\nmore modalities) dynamics within its multimodal input in a factorized manner.\nThe proposed factorization allows for increasing the number of self-attentions\nto better model the multimodal phenomena at hand; without encountering\ndifficulties during training (e.g. overfitting) even on relatively low-resource\nsetups. All the attention mechanisms within FMT have a full time-domain\nreceptive field which allows them to asynchronously capture long-range\nmultimodal dynamics. In our experiments we focus on datasets that contain the\nthree commonly studied modalities of language, vision and acoustic. We perform\na wide range of experiments, spanning across 3 well-studied datasets and 21\ndistinct labels. FMT shows superior performance over previously proposed\nmodels, setting new state of the art in the studied datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.09860v1",
            "date": "2019-11-22 05:05:42+00:00",
            "title": "Data Programming using Continuous and Quality-Guided Labeling Functions",
            "authors": [
                "Oishik Chatterjee",
                "Ganesh Ramakrishnan",
                "Sunita Sarawagi"
            ],
            "abstract": "Scarcity of labeled data is a bottleneck for supervised learning models. A\nparadigm that has evolved for dealing with this problem is data programming. An\nexisting data programming paradigm allows human supervision to be provided as a\nset of discrete labeling functions (LF) that output possibly noisy labels to\ninput instances and a generative modelfor consolidating the weak labels. We\nenhance and generalize this paradigm by supporting functions that output a\ncontinuous score (instead of a hard label) that noisily correlates with labels.\nWe show across five applications that continuous LFs are more natural to\nprogram and lead to improved recall. We also show that accuracy of existing\ngenerative models is unstable with respect to initialization, training epochs,\nand learning rates. We give control to the data programmer to guide the\ntraining process by providing intuitive quality guides with each LF. We propose\nan elegant method of incorporating these guides into the generative model. Our\noverall method, called CAGE, makes the data programming paradigm more reliable\nthan other tricks based on initialization, sign-penalties, or soft-accuracy\nconstraints.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.09923v1",
            "date": "2019-11-22 08:44:23+00:00",
            "title": "SWift -- A SignWriting editor to bridge between deaf world and e-learning",
            "authors": [
                "Claudia S. Bianchini",
                "Fabrizio Borgia",
                "Maria de Marsico"
            ],
            "abstract": "SWift (SignWriting improved fast transcriber) is an advanced editor for\nSignWriting (SW). At present, SW is a promising alternative to provide\ndocuments in an easy-to-grasp written form of (any) Sign Language, the gestural\nway of communication which is widely adopted by the deaf community. SWift was\ndeveloped SW users, either deaf or not, to support collaboration and exchange\nof ideas. The application allows composing and saving desired signs using\nelementary components, called glyphs. The procedure that was devised guides and\nsimplifies the editing process. SWift aims at breaking the \"electronic\"\nbarriers that keep the deaf community away from ICT in general, and from\ne-learning in particular. The editor can be contained in a pluggable module;\ntherefore, it can be integrated everywhere the use of SW is an advisable\nalternative to written \"verbal\" language, which often hinders information\ngrasping by deaf users.",
            "journal ref": "Proceedings of the International Conference on Advanced Learning\n  Technologies (ICALT), IEEE - Institute of Electrical and Electronics\n  Engineers, pp.526-530, 2012, 978-0-7695-4702-2",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1911.09969v4",
            "date": "2019-11-22 10:55:50+00:00",
            "title": "The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service",
            "authors": [
                "Meng Chen",
                "Ruixue Liu",
                "Lei Shen",
                "Shaozu Yuan",
                "Jingyan Zhou",
                "Youzheng Wu",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Human conversations are complicated and building a human-like dialogue agent\nis an extremely challenging task. With the rapid development of deep learning\ntechniques, data-driven models become more and more prevalent which need a huge\namount of real conversation data. In this paper, we construct a large-scale\nreal scenario Chinese E-commerce conversation corpus, JDDC, with more than 1\nmillion multi-turn dialogues, 20 million utterances, and 150 million words. The\ndataset reflects several characteristics of human-human conversations, e.g.,\ngoal-driven, and long-term dependency among the context. It also covers various\ndialogue types including task-oriented, chitchat and question-answering. Extra\nintent information and three well-annotated challenge sets are also provided.\nThen, we evaluate several retrieval-based and generative models to provide\nbasic benchmark performance on the JDDC corpus. And we hope JDDC can serve as\nan effective testbed and benefit the development of fundamental research in\ndialogue task",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10088v2",
            "date": "2019-11-22 15:38:01+00:00",
            "title": "Optimizing Data Usage via Differentiable Rewards",
            "authors": [
                "Xinyi Wang",
                "Hieu Pham",
                "Paul Michel",
                "Antonios Anastasopoulos",
                "Jaime Carbonell",
                "Graham Neubig"
            ],
            "abstract": "To acquire a new skill, humans learn better and faster if a tutor, based on\ntheir current knowledge level, informs them of how much attention they should\npay to particular content or practice problems. Similarly, a machine learning\nmodel could potentially be trained better with a scorer that \"adapts\" to its\ncurrent learning state and estimates the importance of each training data\ninstance. Training such an adaptive scorer efficiently is a challenging\nproblem; in order to precisely quantify the effect of a data instance at a\ngiven time during the training, it is typically necessary to first complete the\nentire training process. To efficiently optimize data usage, we propose a\nreinforcement learning approach called Differentiable Data Selection (DDS). In\nDDS, we formulate a scorer network as a learnable function of the training\ndata, which can be efficiently updated along with the main model being trained.\nSpecifically, DDS updates the scorer with an intuitive reward signal: it should\nup-weigh the data that has a similar gradient with a dev set upon which we\nwould finally like to perform well. Without significant computing overhead, DDS\ndelivers strong and consistent improvements over several strong baselines on\ntwo very different tasks of machine translation and image classification.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.10097v1",
            "date": "2019-11-22 15:51:08+00:00",
            "title": "HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs",
            "authors": [
                "Fangyu Liu",
                "Rongtian Ye",
                "Xun Wang",
                "Shuaipeng Li"
            ],
            "abstract": "The hubness problem widely exists in high-dimensional embedding space and is\na fundamental source of error for cross-modal matching tasks. In this work, we\nstudy the emergence of hubs in Visual Semantic Embeddings (VSE) with\napplication to text-image matching. We analyze the pros and cons of two widely\nadopted optimization objectives for training VSE and propose a novel\nhubness-aware loss function (HAL) that addresses previous methods' defects.\nUnlike (Faghri et al.2018) which simply takes the hardest sample within a\nmini-batch, HAL takes all samples into account, using both local and global\nstatistics to scale up the weights of \"hubs\". We experiment our method with\nvarious configurations of model architectures and datasets. The method exhibits\nexceptionally good robustness and brings consistent improvement on the task of\ntext-image matching across all settings. Specifically, under the same model\narchitectures as (Faghri et al. 2018) and (Lee at al. 2018), by switching only\nthe learning objective, we report a maximum R@1improvement of 7.4% on MS-COCO\nand 8.3% on Flickr30k.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.10115v1",
            "date": "2019-11-22 16:17:21+00:00",
            "title": "TPsgtR: Neural-Symbolic Tensor Product Scene-Graph-Triplet Representation for Image Captioning",
            "authors": [
                "Chiranjib Sur"
            ],
            "abstract": "Image captioning can be improved if the structure of the graphical\nrepresentations can be formulated with conceptual positional binding. In this\nwork, we have introduced a novel technique for caption generation using the\nneural-symbolic encoding of the scene-graphs, derived from regional visual\ninformation of the images and we call it Tensor Product Scene-Graph-Triplet\nRepresentation (TP$_{sgt}$R). While, most of the previous works concentrated on\nidentification of the object features in images, we introduce a neuro-symbolic\nembedding that can embed identified relationships among different regions of\nthe image into concrete forms, instead of relying on the model to compose for\nany/all combinations. These neural symbolic representation helps in better\ndefinition of the neural symbolic space for neuro-symbolic attention and can be\ntransformed to better captions. With this approach, we introduced two novel\narchitectures (TP$_{sgt}$R-TDBU and TP$_{sgt}$R-sTDBU) for comparison and\nexperiment result demonstrates that our approaches outperformed the other\nmodels, and generated captions are more comprehensive and natural.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10354v3",
            "date": "2019-11-23 12:18:03+00:00",
            "title": "Unsupervised Keyword Extraction for Full-sentence VQA",
            "authors": [
                "Kohei Uehara",
                "Tatsuya Harada"
            ],
            "abstract": "In the majority of the existing Visual Question Answering (VQA) research, the\nanswers consist of short, often single words, as per instructions given to the\nannotators during dataset construction. This study envisions a VQA task for\nnatural situations, where the answers are more likely to be sentences rather\nthan single words. To bridge the gap between this natural VQA and existing VQA\napproaches, a novel unsupervised keyword extraction method is proposed. The\nmethod is based on the principle that the full-sentence answers can be\ndecomposed into two parts: one that contains new information answering the\nquestion (i.e., keywords), and one that contains information already included\nin the question. Discriminative decoders were designed to achieve such\ndecomposition, and the method was experimentally implemented on VQA datasets\ncontaining full-sentence answers. The results show that the proposed model can\naccurately extract the keywords without being given explicit annotations\ndescribing them.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.10421v1",
            "date": "2019-11-23 21:42:23+00:00",
            "title": "SemEval-2013 Task 4: Free Paraphrases of Noun Compounds",
            "authors": [
                "Iris Hendrickx",
                "Preslav Nakov",
                "Stan Szpakowicz",
                "Zornitsa Kozareva",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Tony Veale"
            ],
            "abstract": "In this paper, we describe SemEval-2013 Task 4: the definition, the data, the\nevaluation and the results. The task is to capture some of the meaning of\nEnglish noun compounds via paraphrasing. Given a two-word noun compound, the\nparticipating system is asked to produce an explicitly ranked list of its\nfree-form paraphrases. The list is automatically compared and evaluated against\na similarly ranked list of paraphrases proposed by human annotators, recruited\nand managed through Amazon's Mechanical Turk. The comparison of raw paraphrases\nis sensitive to syntactic and morphological variation. The \"gold\" ranking is\nbased on the relative popularity of paraphrases among annotators. To make the\nranking more reliable, highly similar paraphrases are grouped, so as to\ndownplay superficial differences in syntax and morphology. Three systems\nparticipated in the task. They all beat a simple baseline on one of the two\nevaluation measures, but not on both measures. This shows that the task is\ndifficult.",
            "journal ref": "SemEval-2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10422v1",
            "date": "2019-11-23 21:49:10+00:00",
            "title": "SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals",
            "authors": [
                "Iris Hendrickx",
                "Su Nam Kim",
                "Zornitsa Kozareva",
                "Preslav Nakov",
                "Diarmuid \u00d3 S\u00e9aghdha",
                "Sebastian Pad\u00f3",
                "Marco Pennacchiotti",
                "Lorenza Romano",
                "Stan Szpakowicz"
            ],
            "abstract": "In response to the continuing research interest in computational semantic\nanalysis, we have proposed a new task for SemEval-2010: multi-way\nclassification of mutually exclusive semantic relations between pairs of\nnominals. The task is designed to compare different approaches to the problem\nand to provide a standard testbed for future research. In this paper, we define\nthe task, describe the creation of the datasets, and discuss the results of the\nparticipating 28 systems submitted by 10 teams.",
            "journal ref": "SemEval-2010",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10460v1",
            "date": "2019-11-24 05:06:41+00:00",
            "title": "Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences",
            "authors": [
                "Shizhe Chen",
                "Bei Liu",
                "Jianlong Fu",
                "Ruihua Song",
                "Qin Jin",
                "Pingping Lin",
                "Xiaoyu Qi",
                "Chunting Wang",
                "Jin Zhou"
            ],
            "abstract": "A storyboard is a sequence of images to illustrate a story containing\nmultiple sentences, which has been a key process to create different story\nproducts. In this paper, we tackle a new multimedia task of automatic\nstoryboard creation to facilitate this process and inspire human artists.\nInspired by the fact that our understanding of languages is based on our past\nexperience, we propose a novel inspire-and-create framework with a\nstory-to-image retriever that selects relevant cinematic images for inspiration\nand a storyboard creator that further refines and renders images to improve the\nrelevancy and visual consistency. The proposed retriever dynamically employs\ncontextual information in the story with hierarchical attentions and applies\ndense visual-semantic matching to accurately retrieve and ground images. The\ncreator then employs three rendering steps to increase the flexibility of\nretrieved images, which include erasing irrelevant regions, unifying styles of\nimages and substituting consistent characters. We carry out extensive\nexperiments on both in-domain and out-of-domain visual story datasets. The\nproposed model achieves better quantitative performance than the\nstate-of-the-art baselines for storyboard creation. Qualitative visualizations\nand user studies further verify that our approach can create high-quality\nstoryboards even for stories in the wild.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.10708v1",
            "date": "2019-11-25 05:46:56+00:00",
            "title": "hauWE: Hausa Words Embedding for Natural Language Processing",
            "authors": [
                "Idris Abdulmumin",
                "Bashir Shehu Galadanci"
            ],
            "abstract": "Words embedding (distributed word vector representations) have become an\nessential component of many natural language processing (NLP) tasks such as\nmachine translation, sentiment analysis, word analogy, named entity recognition\nand word similarity. Despite this, the only work that provides word vectors for\nHausa language is that of Bojanowski et al. [1] trained using fastText,\nconsisting of only a few words vectors. This work presents words embedding\nmodels using Word2Vec's Continuous Bag of Words (CBoW) and Skip Gram (SG)\nmodels. The models, hauWE (Hausa Words Embedding), are bigger and better than\nthe only previous model, making them more useful in NLP tasks. To compare the\nmodels, they were used to predict the 10 most similar words to 30 randomly\nselected Hausa words. hauWE CBoW's 88.7% and hauWE SG's 79.3% prediction\naccuracy greatly outperformed Bojanowski et al. [1]'s 22.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10742v1",
            "date": "2019-11-25 07:34:37+00:00",
            "title": "End-to-End Trainable Non-Collaborative Dialog System",
            "authors": [
                "Yu Li",
                "Kun Qian",
                "Weiyan Shi",
                "Zhou Yu"
            ],
            "abstract": "End-to-end task-oriented dialog models have achieved promising performance on\ncollaborative tasks where users willingly coordinate with the system to\ncomplete a given task. While in non-collaborative settings, for example,\nnegotiation and persuasion, users and systems do not share a common goal. As a\nresult, compared to collaborate tasks, people use social content to build\nrapport and trust in these non-collaborative settings in order to advance their\ngoals. To handle social content, we introduce a hierarchical intent annotation\nscheme, which can be generalized to different non-collaborative dialog tasks.\nBuilding upon TransferTransfo (Wolf et al. 2019), we propose an end-to-end\nneural network model to generate diverse coherent responses. Our model utilizes\nintent and semantic slots as the intermediate sentence representation to guide\nthe generation process. In addition, we design a filter to select appropriate\nresponses based on whether these intermediate representations fit the designed\ntask and conversation constraints. Our non-collaborative dialog model guides\nusers to complete the task while simultaneously keeps them engaged. We test our\napproach on our newly proposed ANTISCAM dataset and an existing\nPERSUASIONFORGOOD dataset. Both automatic and human evaluations suggest that\nour model outperforms multiple baselines in these two non-collaborative tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10763v1",
            "date": "2019-11-25 08:29:37+00:00",
            "title": "Corpus Wide Argument Mining -- a Working Solution",
            "authors": [
                "Liat Ein-Dor",
                "Eyal Shnarch",
                "Lena Dankin",
                "Alon Halfon",
                "Benjamin Sznajder",
                "Ariel Gera",
                "Carlos Alzate",
                "Martin Gleize",
                "Leshem Choshen",
                "Yufang Hou",
                "Yonatan Bilu",
                "Ranit Aharonov",
                "Noam Slonim"
            ],
            "abstract": "One of the main tasks in argument mining is the retrieval of argumentative\ncontent pertaining to a given topic. Most previous work addressed this task by\nretrieving a relatively small number of relevant documents as the initial\nsource for such content. This line of research yielded moderate success, which\nis of limited use in a real-world system. Furthermore, for such a system to\nyield a comprehensive set of relevant arguments, over a wide range of topics,\nit requires leveraging a large and diverse corpus in an appropriate manner.\nHere we present a first end-to-end high-precision, corpus-wide argument mining\nsystem. This is made possible by combining sentence-level queries over an\nappropriate indexing of a very large corpus of newspaper articles, with an\niterative annotation scheme. This scheme addresses the inherent label bias in\nthe data and pinpoints the regions of the sample space whose manual labeling is\nrequired to obtain high-precision among top-ranked candidates.",
            "journal ref": "AAAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.10924v1",
            "date": "2019-11-25 13:59:05+00:00",
            "title": "Discovering topics with neural topic models built from PLSA assumptions",
            "authors": [
                "Sileye 0. Ba"
            ],
            "abstract": "In this paper we present a model for unsupervised topic discovery in texts\ncorpora. The proposed model uses documents, words, and topics lookup table\nembedding as neural network model parameters to build probabilities of words\ngiven topics, and probabilities of topics given documents. These probabilities\nare used to recover by marginalization probabilities of words given documents.\nFor very large corpora where the number of documents can be in the order of\nbillions, using a neural auto-encoder based document embedding is more scalable\nthen using a lookup table embedding as classically done. We thus extended the\nlookup based document embedding model to continuous auto-encoder based model.\nOur models are trained using probabilistic latent semantic analysis (PLSA)\nassumptions. We evaluated our models on six datasets with a rich variety of\ncontents. Conducted experiments demonstrate that the proposed neural topic\nmodels are very effective in capturing relevant topics. Furthermore,\nconsidering perplexity metric, conducted evaluation benchmarks show that our\ntopic models outperform latent Dirichlet allocation (LDA) model which is\nclassically used to address topic discovery tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11025v1",
            "date": "2019-11-25 16:15:42+00:00",
            "title": "Women, politics and Twitter: Using machine learning to change the discourse",
            "authors": [
                "Lana Cuthbertson",
                "Alex Kearney",
                "Riley Dawson",
                "Ashia Zawaduk",
                "Eve Cuthbertson",
                "Ann Gordon-Tighe",
                "Kory W Mathewson"
            ],
            "abstract": "Including diverse voices in political decision-making strengthens our\ndemocratic institutions. Within the Canadian political system, there is gender\ninequality across all levels of elected government. Online abuse, such as\nhateful tweets, leveled at women engaged in politics contributes to this\ninequity, particularly tweets focusing on their gender. In this paper, we\npresent ParityBOT: a Twitter bot which counters abusive tweets aimed at women\nin politics by sending supportive tweets about influential female leaders and\nfacts about women in public life. ParityBOT is the first artificial\nintelligence-based intervention aimed at affecting online discourse for women\nin politics for the better. The goal of this project is to: $1$) raise\nawareness of issues relating to gender inequity in politics, and $2$)\npositively influence public discourse in politics. The main contribution of\nthis paper is a scalable model to classify and respond to hateful tweets with\nquantitative and qualitative assessments. The ParityBOT abusive classification\nsystem was validated on public online harassment datasets. We conclude with\nanalysis of the impact of ParityBOT, drawing from data gathered during\ninterventions in both the $2019$ Alberta provincial and $2019$ Canadian federal\nelections.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1911.11062v1",
            "date": "2019-11-19 20:37:03+00:00",
            "title": "Automatic Detection of Satire in Bangla Documents: A CNN Approach Based on Hybrid Feature Extraction Model",
            "authors": [
                "Arnab Sen Sharma",
                "Maruf Ahmed Mridul",
                "Md Saiful Islam"
            ],
            "abstract": "Widespread of satirical news in online communities is an ongoing trend. The\nnature of satires is so inherently ambiguous that sometimes it's too hard even\nfor humans to understand whether it's actually satire or not. So, research\ninterest has grown in this field. The purpose of this research is to detect\nBangla satirical news spread in online news portals as well as social media. In\nthis paper, we propose a hybrid technique for extracting features from text\ndocuments combining Word2Vec and TF-IDF. Using our proposed feature extraction\ntechnique, with standard CNN architecture we could detect whether a Bangla text\ndocument is satire or not with an accuracy of more than 96%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.11065v1",
            "date": "2019-11-11 21:02:54+00:00",
            "title": "Knowledge Distillation in Document Retrieval",
            "authors": [
                "Siamak Shakeri",
                "Abhinav Sethy",
                "Cheng Cheng"
            ],
            "abstract": "Complex deep learning models now achieve state of the art performance for\nmany document retrieval tasks. The best models process the query or claim\njointly with the document. However for fast scalable search it is desirable to\nhave document embeddings which are independent of the claim. In this paper we\nshow that knowledge distillation can be used to encourage a model that\ngenerates claim independent document encodings to mimic the behavior of a more\ncomplex model which generates claim dependent encodings. We explore this\napproach in document retrieval for a fact extraction and verification task. We\nshow that by using the soft labels from a complex cross attention teacher\nmodel, the performance of claim independent student LSTM or CNN models is\nimproved across all the ranking metrics. The student models we use are 12x\nfaster in runtime and 20x smaller in number of parameters than the teacher",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.11069v1",
            "date": "2019-11-14 22:34:02+00:00",
            "title": "Query Expansion for Patent Searching using Word Embedding and Professional Crowdsourcing",
            "authors": [
                "Arthi Krishna",
                "Ye Jin",
                "Christine Foster",
                "Greg Gabel",
                "Britt Hanley",
                "Abdou Youssef"
            ],
            "abstract": "The patent examination process includes a search of previous work to verify\nthat a patent application describes a novel invention. Patent examiners\nprimarily use keyword-based searches to uncover prior art. A critical part of\nkeyword searching is query expansion, which is the process of including\nalternate terms such as synonyms and other related words, since the same\nconcepts are often described differently in the literature. Patent terminology\nis often domain specific. By curating technology-specific corpora and training\nword embedding models based on these corpora, we are able to automatically\nidentify the most relevant expansions of a given word or phrase. We compare the\nperformance of several automated query expansion techniques against expert\nspecified expansions. Furthermore, we explore a novel mechanism to extract\nrelated terms not just based on one input term but several terms in conjunction\nby computing their centroid and identifying the nearest neighbors to this\ncentroid. Highly skilled patent examiners are often the best and most reliable\nsource of identifying related terms. By designing a user interface that allows\nexaminers to interact with the word embedding suggestions, we are able to use\nthese interactions to power crowdsourced modes of related terms. Learning from\nusers allows us to overcome several challenges such as identifying words that\nare bleeding edge and have not been published in the corpus yet. This paper\nstudies the effectiveness of word embedding and crowdsourced models across 11\ndisparate technical areas.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.11139v2",
            "date": "2019-11-26 00:09:30+00:00",
            "title": "Learning to Determine the Quality of News Headlines",
            "authors": [
                "Amin Omidvar",
                "Hossein Poormodheji",
                "Aijun An",
                "Gordon Edall"
            ],
            "abstract": "Today, most newsreaders read the online version of news articles rather than\ntraditional paper-based newspapers. Also, news media publishers rely heavily on\nthe income generated from subscriptions and website visits made by newsreaders.\nThus, online user engagement is a very important issue for online newspapers.\nMuch effort has been spent on writing interesting headlines to catch the\nattention of online users. On the other hand, headlines should not be\nmisleading (e.g., clickbaits); otherwise, readers would be disappointed when\nreading the content. In this paper, we propose four indicators to determine the\nquality of published news headlines based on their click count and dwell time,\nwhich are obtained by website log analysis. Then, we use soft target\ndistribution of the calculated quality indicators to train our proposed deep\nlearning model which can predict the quality of unpublished news headlines. The\nproposed model not only processes the latent features of both headline and body\nof the article to predict its headline quality but also considers the semantic\nrelation between headline and body as well. To evaluate our model, we use a\nreal dataset from a major Canadian newspaper. Results show our proposed model\noutperforms other state-of-the-art NLP models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1911.11214v1",
            "date": "2019-11-25 20:29:01+00:00",
            "title": "Examining the Role of Clickbait Headlines to Engage Readers with Reliable Health-related Information",
            "authors": [
                "Sima Bhowmik",
                "Md Main Uddin Rony",
                "Md Mahfuzul Haque",
                "Kristen Alley Swain",
                "Naeemul Hassan"
            ],
            "abstract": "Clickbait headlines are frequently used to attract readers to read articles.\nAlthough this headline type has turned out to be a technique to engage readers\nwith misleading items, it is still unknown whether the technique can be used to\nattract readers to reliable pieces. This study takes the opportunity to test\nits efficacy to engage readers with reliable health articles. A set of online\nsurveys would be conducted to test readers' engagement with and perception\nabout clickbait headlines with reliable articles. After that, we would design\nan automation system to generate clickabit headlines to maximize user\nengagement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11237v3",
            "date": "2019-11-25 21:19:31+00:00",
            "title": "Learning to Learn Words from Visual Scenes",
            "authors": [
                "D\u00eddac Sur\u00eds",
                "Dave Epstein",
                "Heng Ji",
                "Shih-Fu Chang",
                "Carl Vondrick"
            ],
            "abstract": "Language acquisition is the process of learning words from the surrounding\nscene. We introduce a meta-learning framework that learns how to learn word\nrepresentations from unconstrained scenes. We leverage the natural\ncompositional structure of language to create training episodes that cause a\nmeta-learner to learn strong policies for language acquisition. Experiments on\ntwo datasets show that our approach is able to more rapidly acquire novel words\nas well as more robustly generalize to unseen compositions, significantly\noutperforming established baselines. A key advantage of our approach is that it\nis data efficient, allowing representations to be learned from scratch without\nlanguage pre-training. Visualizations and analysis suggest visual information\nhelps our approach learn a rich cross-modal representation from minimal\nexamples. Project webpage is available at https://expert.cs.columbia.edu/",
            "journal ref": "European Conference on Computer Vision (ECCV), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11298v1",
            "date": "2019-11-26 01:01:37+00:00",
            "title": "Few-Shot Knowledge Graph Completion",
            "authors": [
                "Chuxu Zhang",
                "Huaxiu Yao",
                "Chao Huang",
                "Meng Jiang",
                "Zhenhui Li",
                "Nitesh V. Chawla"
            ],
            "abstract": "Knowledge graphs (KGs) serve as useful resources for various natural language\nprocessing applications. Previous KG completion approaches require a large\nnumber of training instances (i.e., head-tail entity pairs) for every relation.\nThe real case is that for most of the relations, very few entity pairs are\navailable. Existing work of one-shot learning limits method generalizability\nfor few-shot scenarios and does not fully use the supervisory information;\nhowever, few-shot KG completion has not been well studied yet. In this work, we\npropose a novel few-shot relation learning model (FSRL) that aims at\ndiscovering facts of new relations with few-shot references. FSRL can\neffectively capture knowledge from heterogeneous graph structure, aggregate\nrepresentations of few-shot references, and match similar entity pairs of\nreference set for every relation. Extensive experiments on two public datasets\ndemonstrate that FSRL outperforms the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11358v1",
            "date": "2019-11-26 06:02:33+00:00",
            "title": "CAWA: An Attention-Network for Credit Attribution",
            "authors": [
                "Saurav Manchanda",
                "George Karypis"
            ],
            "abstract": "Credit attribution is the task of associating individual parts in a document\nwith their most appropriate class labels. It is an important task with\napplications to information retrieval and text summarization. When labeled\ntraining data is available, traditional approaches for sequence tagging can be\nused for credit attribution. However, generating such labeled datasets is\nexpensive and time-consuming. In this paper, we present \"Credit Attribution\nWith Attention (CAWA)\", a neural-network-based approach, that instead of using\nsentence-level labeled data, uses the set of class labels that are associated\nwith an entire document as a source of distant-supervision. CAWA combines an\nattention mechanism with a multilabel classifier into an end-to-end learning\nframework to perform credit attribution. CAWA labels the individual sentences\nfrom the input document using the resultant attention-weights. CAWA improves\nupon the state-of-the-art credit attribution approach by not constraining a\nsentence to belong to just one class, but modeling each sentence as a\ndistribution over all classes, leading to better modeling of\nsemantically-similar classes. Experiments on the credit attribution task on a\nvariety of datasets show that the sentence class labels generated by CAWA\noutperform the competing approaches. Additionally, on the multilabel text\nclassification task, CAWA performs better than the competing credit attribution\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11365v1",
            "date": "2019-11-26 06:35:08+00:00",
            "title": "ATCSpeech: a multilingual pilot-controller speech corpus from real Air Traffic Control environment",
            "authors": [
                "Bo Yang",
                "Xianlong Tan",
                "Zhengmao Chen",
                "Bing Wang",
                "Dan Li",
                "Zhongping Yang",
                "Xiping Wu",
                "Yi Lin"
            ],
            "abstract": "Automatic Speech Recognition (ASR) is greatly developed in recent years,\nwhich expedites many applications on other fields. For the ASR research, speech\ncorpus is always an essential foundation, especially for the vertical industry,\nsuch as Air Traffic Control (ATC). There are some speech corpora for common\napplications, public or paid. However, for the ATC, it is difficult to collect\nraw speeches from real systems due to safety issues. More importantly, for a\nsupervised learning task like ASR, annotating the transcription is a more\nlaborious work, which hugely restricts the prospect of ASR application. In this\npaper, a multilingual speech corpus (ATCSpeech) from real ATC systems,\nincluding accented Mandarin Chinese and English, is built and released to\nencourage the non-commercial ASR research in ATC domain. The corpus is detailly\nintroduced from the perspective of data amount, speaker gender and role, speech\nquality and other attributions. In addition, the performance of our baseline\nASR models is also reported. A community edition for our speech database can be\napplied and used under a special contrast. To our best knowledge, this is the\nfirst work that aims at building a real and multilingual ASR corpus for the air\ntraffic related research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11403v1",
            "date": "2019-11-26 08:40:49+00:00",
            "title": "SemEval-2015 Task 3: Answer Selection in Community Question Answering",
            "authors": [
                "Preslav Nakov",
                "Llu\u00eds M\u00e0rquez",
                "Walid Magdy",
                "Alessandro Moschitti",
                "James Glass",
                "Bilal Randeree"
            ],
            "abstract": "Community Question Answering (cQA) provides new interesting research\ndirections to the traditional Question Answering (QA) field, e.g., the\nexploitation of the interaction between users and the structure of related\nposts. In this context, we organized SemEval-2015 Task 3 on \"Answer Selection\nin cQA\", which included two subtasks: (a) classifying answers as \"good\", \"bad\",\nor \"potentially relevant\" with respect to the question, and (b) answering a\nYES/NO question with \"yes\", \"no\", or \"unsure\", based on the list of all\nanswers. We set subtask A for Arabic and English on two relatively different\ncQA domains, i.e., the Qatar Living website for English, and a Quran-related\nwebsite for Arabic. We used crowdsourcing on Amazon Mechanical Turk to label a\nlarge English training dataset, which we released to the research community.\nThirteen teams participated in the challenge with a total of 61 submissions: 24\nprimary and 37 contrastive. The best systems achieved an official score\n(macro-averaged F1) of 57.19 and 63.7 for the English subtasks A and B, and\n78.55 for the Arabic subtask A.",
            "journal ref": "SemEval-2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11423v2",
            "date": "2019-11-26 09:45:33+00:00",
            "title": "Single Headed Attention RNN: Stop Thinking With Your Head",
            "authors": [
                "Stephen Merity"
            ],
            "abstract": "The leading approaches in language modeling are all obsessed with TV shows of\nmy youth - namely Transformers and Sesame Street. Transformers this,\nTransformers that, and over here a bonfire worth of GPU-TPU-neuromorphic wafer\nscale silicon. We opt for the lazy path of old and proven techniques with a\nfancy crypto inspired acronym: the Single Headed Attention RNN (SHA-RNN). The\nauthor's lone goal is to show that the entire field might have evolved a\ndifferent direction if we had instead been obsessed with a slightly different\nacronym and slightly different result. We take a previously strong language\nmodel based only on boring LSTMs and get it to within a stone's throw of a\nstone's throw of state-of-the-art byte level language model results on enwik8.\nThis work has undergone no intensive hyperparameter optimization and lived\nentirely on a commodity desktop machine that made the author's small studio\napartment far too warm in the midst of a San Franciscan summer. The final\nresults are achievable in plus or minus 24 hours on a single GPU as the author\nis impatient. The attention mechanism is also readily extended to large\ncontexts with minimal computation. Take that Sesame Street.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11506v1",
            "date": "2019-11-26 13:11:00+00:00",
            "title": "Word-Class Embeddings for Multiclass Text Classification",
            "authors": [
                "Alejandro Moreo",
                "Andrea Esuli",
                "Fabrizio Sebastiani"
            ],
            "abstract": "Pre-trained word embeddings encode general word semantics and lexical\nregularities of natural language, and have proven useful across many NLP tasks,\nincluding word sense disambiguation, machine translation, and sentiment\nanalysis, to name a few. In supervised tasks such as multiclass text\nclassification (the focus of this article) it seems appealing to enhance word\nrepresentations with ad-hoc embeddings that encode task-specific information.\nWe propose (supervised) word-class embeddings (WCEs), and show that, when\nconcatenated to (unsupervised) pre-trained word embeddings, they substantially\nfacilitate the training of deep-learning models in multiclass classification by\ntopic. We show empirical evidence that WCEs yield a consistent improvement in\nmulticlass classification accuracy, using four popular neural architectures and\nsix widely used and publicly available datasets for multiclass text\nclassification. Our code that implements WCEs is publicly available at\nhttps://github.com/AlexMoreo/word-class-embeddings",
            "journal ref": "Data Mining and Knowledge Discovery, 2021",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.11558v1",
            "date": "2019-11-25 09:55:52+00:00",
            "title": "FairyTED: A Fair Rating Predictor for TED Talk Data",
            "authors": [
                "Rupam Acharyya",
                "Shouman Das",
                "Ankani Chattoraj",
                "Md. Iftekhar Tanveer"
            ],
            "abstract": "With the recent trend of applying machine learning in every aspect of human\nlife, it is important to incorporate fairness into the core of the predictive\nalgorithms. We address the problem of predicting the quality of public speeches\nwhile being fair with respect to sensitive attributes of the speakers, e.g.\ngender and race. We use the TED talks as an input repository of public speeches\nbecause it consists of speakers from a diverse community and has a wide\noutreach. Utilizing the theories of Causal Models, Counterfactual Fairness and\nstate-of-the-art neural language models, we propose a mathematical framework\nfor fair prediction of the public speaking quality. We employ grounded\nassumptions to construct a causal model capturing how different attributes\naffect public speaking quality. This causal model contributes in generating\ncounterfactual data to train a fair predictive model. Our framework is general\nenough to utilize any assumption within the causal model. Experimental results\nshow that while prediction accuracy is comparable to recent work on this\ndataset, our predictions are counterfactually fair with respect to a novel\nmetric when compared to true data labels. The FairyTED setup not only allows\norganizers to make informed and diverse selection of speakers from the\nunobserved counterfactual possibilities but it also ensures that viewers and\nnew users are not influenced by unfair and unbalanced ratings from arbitrary\nvisitors to the www.ted.com website when deciding to view a talk.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.11641v1",
            "date": "2019-11-26 15:31:46+00:00",
            "title": "PIQA: Reasoning about Physical Commonsense in Natural Language",
            "authors": [
                "Yonatan Bisk",
                "Rowan Zellers",
                "Ronan Le Bras",
                "Jianfeng Gao",
                "Yejin Choi"
            ],
            "abstract": "To apply eyeshadow without a brush, should I use a cotton swab or a\ntoothpick? Questions requiring this kind of physical commonsense pose a\nchallenge to today's natural language understanding systems. While recent\npretrained models (such as BERT) have made progress on question answering over\nmore abstract domains - such as news articles and encyclopedia entries, where\ntext is plentiful - in more physical domains, text is inherently limited due to\nreporting bias. Can AI systems learn to reliably answer physical common-sense\nquestions without experiencing the physical world? In this paper, we introduce\nthe task of physical commonsense reasoning and a corresponding benchmark\ndataset Physical Interaction: Question Answering or PIQA. Though humans find\nthe dataset easy (95% accuracy), large pretrained models struggle (77%). We\nprovide analysis about the dimensions of knowledge that existing models lack,\nwhich offers significant opportunities for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11657v1",
            "date": "2019-11-26 15:55:39+00:00",
            "title": "Hybrid Text Feature Modeling for Disease Group Prediction using Unstructured Physician Notes",
            "authors": [
                "Gokul S Krishnan",
                "Sowmya Kamath S"
            ],
            "abstract": "Existing Clinical Decision Support Systems (CDSSs) largely depend on the\navailability of structured patient data and Electronic Health Records (EHRs) to\naid caregivers. However, in case of hospitals in developing countries,\nstructured patient data formats are not widely adopted, where medical\nprofessionals still rely on clinical notes in the form of unstructured text.\nSuch unstructured clinical notes recorded by medical personnel can also be a\npotential source of rich patient-specific information which can be leveraged to\nbuild CDSSs, even for hospitals in developing countries. If such unstructured\nclinical text can be used, the manual and time-consuming process of EHR\ngeneration will no longer be required, with huge person-hours and cost savings.\nIn this paper, we propose a generic ICD9 disease group prediction CDSS built on\nunstructured physician notes modeled using hybrid word embeddings. These word\nembeddings are used to train a deep neural network for effectively predicting\nICD9 disease groups. Experimental evaluation showed that the proposed approach\noutperformed the state-of-the-art disease group prediction model built on\nstructured EHRs by 15% in terms of AUROC and 40% in terms of AUPRC, thus\nproving our hypothesis and eliminating dependency on availability of structured\npatient data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11668v2",
            "date": "2019-11-26 16:07:44+00:00",
            "title": "Biology and Compositionality: Empirical Considerations for Emergent-Communication Protocols",
            "authors": [
                "Travis LaCroix"
            ],
            "abstract": "Significant advances have been made in artificial systems by using biological\nsystems as a guide. However, there is often little interaction between\ncomputational models for emergent communication and biological models of the\nemergence of language. Many researchers in language origins and emergent\ncommunication take compositionality as their primary target for explaining how\nsimple communication systems can become more like natural language. However,\nthere is reason to think that compositionality is the wrong target on the\nbiological side, and so too the wrong target on the machine-learning side. As\nsuch, the purpose of this paper is to explore this claim. This has theoretical\nimplications for language origins research more generally, but the focus here\nwill be the implications for research on emergent communication in computer\nscience and machine learning---specifically regarding the types of programmes\nthat might be expected to work and those which will not. I further suggest an\nalternative approach for future research which focuses on reflexivity, rather\nthan compositionality, as a target for explaining how simple communication\nsystems may become more like natural language. I end by providing some\nreference to the language origins literature that may be of some use to\nresearchers in machine learning.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.11744v1",
            "date": "2019-11-26 18:27:51+00:00",
            "title": "Imitation Learning of Robot Policies by Combining Language, Vision and Demonstration",
            "authors": [
                "Simon Stepputtis",
                "Joseph Campbell",
                "Mariano Phielipp",
                "Chitta Baral",
                "Heni Ben Amor"
            ],
            "abstract": "In this work we propose a novel end-to-end imitation learning approach which\ncombines natural language, vision, and motion information to produce an\nabstract representation of a task, which in turn is used to synthesize specific\nmotion controllers at run-time. This multimodal approach enables generalization\nto a wide variety of environmental conditions and allows an end-user to direct\na robot policy through verbal communication. We empirically validate our\napproach with an extensive set of simulations and show that it achieves a high\ntask success rate over a variety of conditions while remaining amenable to\nprobabilistic interpretability.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1911.11750v1",
            "date": "2019-11-26 18:38:59+00:00",
            "title": "A Measure of Similarity in Textual Data Using Spearman's Rank Correlation Coefficient",
            "authors": [
                "Nino Arsov",
                "Milan Dukovski",
                "Blagoja Evkoski",
                "Stefan Cvetkovski"
            ],
            "abstract": "In the last decade, many diverse advances have occurred in the field of\ninformation extraction from data. Information extraction in its simplest form\ntakes place in computing environments, where structured data can be extracted\nthrough a series of queries. The continuous expansion of quantities of data\nhave therefore provided an opportunity for knowledge extraction (KE) from a\ntextual document (TD). A typical problem of this kind is the extraction of\ncommon characteristics and knowledge from a group of TDs, with the possibility\nto group such similar TDs in a process known as clustering. In this paper we\npresent a technique for such KE among a group of TDs related to the common\ncharacteristics and meaning of their content. Our technique is based on the\nSpearman's Rank Correlation Coefficient (SRCC), for which the conducted\nexperiments have proven to be comprehensive measure to achieve a high-quality\nKE.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.11756v1",
            "date": "2019-11-26 18:47:48+00:00",
            "title": "Semi-Supervised Learning for Text Classification by Layer Partitioning",
            "authors": [
                "Alexander Hanbo Li",
                "Abhinav Sethy"
            ],
            "abstract": "Most recent neural semi-supervised learning algorithms rely on adding small\nperturbation to either the input vectors or their representations. These\nmethods have been successful on computer vision tasks as the images form a\ncontinuous manifold, but are not appropriate for discrete input such as\nsentence. To adapt these methods to text input, we propose to decompose a\nneural network $M$ into two components $F$ and $U$ so that $M = U\\circ F$. The\nlayers in $F$ are then frozen and only the layers in $U$ will be updated during\nmost time of the training. In this way, $F$ serves as a feature extractor that\nmaps the input to high-level representation and adds systematical noise using\ndropout. We can then train $U$ using any state-of-the-art SSL algorithms such\nas $\\Pi$-model, temporal ensembling, mean teacher, etc. Furthermore, this\ngradually unfreezing schedule also prevents a pretrained model from\ncatastrophic forgetting. The experimental results demonstrate that our approach\nprovides improvements when compared to state of the art methods especially on\nshort texts.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.11926v1",
            "date": "2019-11-27 02:52:02+00:00",
            "title": "Recency predicts bursts in the evolution of author citations",
            "authors": [
                "Filipi Nascimento Silva",
                "Aditya Tandon",
                "Diego Raphael Amancio",
                "Alessandro Flammini",
                "Filippo Menczer",
                "Sta\u0161a Milojevi\u0107",
                "Santo Fortunato"
            ],
            "abstract": "The citations process for scientific papers has been studied extensively. But\nwhile the citations accrued by authors are the sum of the citations of their\npapers, translating the dynamics of citation accumulation from the paper to the\nauthor level is not trivial. Here we conduct a systematic study of the\nevolution of author citations, and in particular their bursty dynamics. We find\nempirical evidence of a correlation between the number of citations most\nrecently accrued by an author and the number of citations they receive in the\nfuture. Using a simple model where the probability for an author to receive new\ncitations depends only on the number of citations collected in the previous\n12-24 months, we are able to reproduce both the citation and burst size\ndistributions of authors across multiple decades.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11935v1",
            "date": "2019-11-27 03:42:21+00:00",
            "title": "AIPNet: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition",
            "authors": [
                "Yi-Chen Chen",
                "Zhaojun Yang",
                "Ching-Feng Yeh",
                "Mahaveer Jain",
                "Michael L. Seltzer"
            ],
            "abstract": "As one of the major sources in speech variability, accents have posed a grand\nchallenge to the robustness of speech recognition systems. In this paper, our\ngoal is to build a unified end-to-end speech recognition system that\ngeneralizes well across accents. For this purpose, we propose a novel\npre-training framework AIPNet based on generative adversarial nets (GAN) for\naccent-invariant representation learning: Accent Invariant Pre-training\nNetworks. We pre-train AIPNet to disentangle accent-invariant and\naccent-specific characteristics from acoustic features through adversarial\ntraining on accented data for which transcriptions are not necessarily\navailable. We further fine-tune AIPNet by connecting the accent-invariant\nmodule with an attention-based encoder-decoder model for multi-accent speech\nrecognition. In the experiments, our approach is compared against four\nbaselines including both accent-dependent and accent-independent models.\nExperimental results on 9 English accents show that the proposed approach\noutperforms all the baselines by 2.3 \\sim 4.5% relative reduction on average\nWER when transcriptions are available in all accents and by 1.6 \\sim 6.1%\nrelative reduction when transcriptions are only available in US accent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.11952v1",
            "date": "2019-11-27 04:54:14+00:00",
            "title": "Label Dependent Deep Variational Paraphrase Generation",
            "authors": [
                "Siamak Shakeri",
                "Abhinav Sethy"
            ],
            "abstract": "Generating paraphrases that are lexically similar but semantically different\nis a challenging task. Paraphrases of this form can be used to augment data\nsets for various NLP tasks such as machine reading comprehension and question\nanswering with non-trivial negative examples. In this article, we propose a\ndeep variational model to generate paraphrases conditioned on a label that\nspecifies whether the paraphrases are semantically related or not. We also\npresent new training recipes and KL regularization techniques that improve the\nperformance of variational paraphrasing models. Our proposed model demonstrates\npromising results in enhancing the generative power of the model by employing\nlabel-dependent generation on paraphrasing datasets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.12085v1",
            "date": "2019-11-27 11:25:43+00:00",
            "title": "Large-Scale Noun Compound Interpretation Using Bootstrapping and the Web as a Corpus",
            "authors": [
                "Su Nam Kim",
                "Preslav Nakov"
            ],
            "abstract": "Responding to the need for semantic lexical resources in natural language\nprocessing applications, we examine methods to acquire noun compounds (NCs),\ne.g., \"orange juice\", together with suitable fine-grained semantic\ninterpretations, e.g., \"squeezed from\", which are directly usable as\nparaphrases. We employ bootstrapping and web statistics, and utilize the\nrelationship between NCs and paraphrasing patterns to jointly extract NCs and\nsuch patterns in multiple alternating iterations. In evaluation, we found that\nhaving one compound noun fixed yields both a higher number of semantically\ninterpreted NCs and improved accuracy due to stronger semantic restrictions.",
            "journal ref": "EMNLP-2011",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12091v1",
            "date": "2019-11-27 11:45:15+00:00",
            "title": "Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Prediction",
            "authors": [
                "Liane Guillou",
                "Christian Hardmeier",
                "Preslav Nakov",
                "Sara Stymne",
                "J\u00f6rg Tiedemann",
                "Yannick Versley",
                "Mauro Cettolo",
                "Bonnie Webber",
                "Andrei Popescu-Belis"
            ],
            "abstract": "We describe the design, the evaluation setup, and the results of the 2016 WMT\nshared task on cross-lingual pronoun prediction. This is a classification task\nin which participants are asked to provide predictions on what pronoun class\nlabel should replace a placeholder value in the target-language text, provided\nin lemmatised and PoS-tagged form. We provided four subtasks, for the\nEnglish-French and English-German language pairs, in both directions. Eleven\nteams participated in the shared task; nine for the English-French subtask,\nfive for French-English, nine for English-German, and six for German-English.\nMost of the submissions outperformed two strong language-model based baseline\nsystems, with systems using deep recurrent neural networks outperforming those\nusing other architectures for most language pairs.",
            "journal ref": "WMT-2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12377v2",
            "date": "2019-11-27 19:00:24+00:00",
            "title": "Perceive, Transform, and Act: Multi-Modal Attention Networks for Vision-and-Language Navigation",
            "authors": [
                "Federico Landi",
                "Lorenzo Baraldi",
                "Marcella Cornia",
                "Massimiliano Corsini",
                "Rita Cucchiara"
            ],
            "abstract": "Vision-and-Language Navigation (VLN) is a challenging task in which an agent\nneeds to follow a language-specified path to reach a target destination. In\nthis paper, we strive for the creation of an agent able to tackle three key\nissues: multi-modality, long-term dependencies, and adaptability towards\ndifferent locomotive settings. To that end, we devise \"Perceive, Transform, and\nAct\" (PTA): a fully-attentive VLN architecture that leaves the recurrent\napproach behind and the first Transformer-like architecture incorporating three\ndifferent modalities - natural language, images, and discrete actions for the\nagent control. In particular, we adopt an early fusion strategy to merge\nlingual and visual information efficiently in our encoder. We then propose to\nrefine the decoding phase with a late fusion extension between the agent's\nhistory of actions and the perception modalities. We experimentally validate\nour model on two datasets and two different action settings. PTA surpasses\nprevious state-of-the-art architectures for low-level VLN on R2R and achieves\nthe first place for both setups in the recently proposed R4R benchmark. Our\ncode is publicly available at\nhttps://github.com/aimagelab/perceive-transform-and-act.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1911.12478v2",
            "date": "2019-11-28 01:35:51+00:00",
            "title": "Metre as a stylometric feature in Latin hexameter poetry",
            "authors": [
                "Benjamin Nagy"
            ],
            "abstract": "This paper demonstrates that metre is a privileged indicator of authorial\nstyle in classical Latin hexameter poetry. Using only metrical features,\npairwise classification experiments are performed between 5 first-century\nauthors (10 comparisons) using four different machine-learning models. The\nresults showed a two-label classification accuracy of at least 95% with samples\nas small as ten lines and no greater than eighty lines (up to around 500\nwords). These sample sizes are an order of magnitude smaller than those\ntypically recommended for BOW ('bag of words') or n-gram approaches, and the\nreported accuracy is outstanding. Additionally, this paper explores the\npotential for novelty (forgery) detection, or 'one-class classification'. An\nanalysis of the disputed Aldine Additamentum (Sil. Ital. Puni. 8:144-225)\nconcludes (p=0.0013) that the metrical style differs significantly from that of\nthe rest of the poem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12487v1",
            "date": "2019-11-28 02:17:56+00:00",
            "title": "Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition",
            "authors": [
                "Chao Weng",
                "Chengzhu Yu",
                "Jia Cui",
                "Chunlei Zhang",
                "Dong Yu"
            ],
            "abstract": "In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer\n(RNN-T) for end-to-end speech recognition. Specifically, initialized with a\nRNN-T trained model, MBR training is conducted via minimizing the expected edit\ndistance between the reference label sequence and on-the-fly generated N-best\nhypothesis. We also introduce a heuristic to incorporate an external neural\nnetwork language model (NNLM) in RNN-T beam search decoding and explore MBR\ntraining with the external NNLM. Experimental results demonstrate an MBR\ntrained model outperforms a RNN-T trained model substantially and further\nimprovements can be achieved if trained with an external NNLM. Our best MBR\ntrained system achieves absolute character error rate (CER) reductions of 1.2%\nand 0.5% on read and spontaneous Mandarin speech respectively over a strong\nconvolution and transformer based RNN-T baseline trained on ~21,000 hours of\nspeech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12544v1",
            "date": "2019-11-28 05:55:44+00:00",
            "title": "Language-Independent Sentiment Analysis Using Subjectivity and Positional Information",
            "authors": [
                "Veselin Raychev",
                "Preslav Nakov"
            ],
            "abstract": "We describe a novel language-independent approach to the task of determining\nthe polarity, positive or negative, of the author's opinion on a specific topic\nin natural language text. In particular, weights are assigned to attributes,\nindividual words or word bi-grams, based on their position and on their\nlikelihood of being subjective. The subjectivity of each attribute is estimated\nin a two-step process, where first the probability of being subjective is\ncalculated for each sentence containing the attribute, and then these\nprobabilities are used to alter the attribute's weights for polarity\nclassification. The evaluation results on a standard dataset of movie reviews\nshows 89.85% classification accuracy, which rivals the best previously\npublished results for this dataset for systems that use no additional\nlinguistic information nor external resources.",
            "journal ref": "RANLP-2009",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12674v2",
            "date": "2019-11-28 12:37:26+00:00",
            "title": "RETRO: Relation Retrofitting For In-Database Machine Learning on Textual Data",
            "authors": [
                "Michael G\u00fcnther",
                "Maik Thiele",
                "Wolfgang Lehner"
            ],
            "abstract": "There are massive amounts of textual data residing in databases, valuable for\nmany machine learning (ML) tasks. Since ML techniques depend on numerical input\nrepresentations, word embeddings are increasingly utilized to convert symbolic\nrepresentations such as text into meaningful numbers. However, a naive\none-to-one mapping of each word in a database to a word embedding vector is not\nsufficient and would lead to poor accuracies in ML tasks. Thus, we argue to\nadditionally incorporate the information given by the database schema into the\nembedding, e.g. which words appear in the same column or are related to each\nother. In this paper, we propose RETRO (RElational reTROfitting), a novel\napproach to learn numerical representations of text values in databases,\ncapturing the best of both worlds, the rich information encoded by word\nembeddings and the relational information encoded by database tables. We\nformulate relation retrofitting as a learning problem and present an efficient\nalgorithm solving it. We investigate the impact of various hyperparameters on\nthe learning problem and derive good settings for all of them. Our evaluation\nshows that the proposed embeddings are ready-to-use for many ML tasks such as\nclassification and regression and even outperform state-of-the-art techniques\nin integration tasks such as null value imputation and link prediction.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/1911.12848v1",
            "date": "2019-11-28 20:00:40+00:00",
            "title": "Sentiment Analysis On Indian Indigenous Languages: A Review On Multilingual Opinion Mining",
            "authors": [
                "Sonali Rajesh Shah",
                "Abhishek Kaushik"
            ],
            "abstract": "An increase in the use of smartphones has laid to the use of the internet and\nsocial media platforms. The most commonly used social media platforms are\nTwitter, Facebook, WhatsApp and Instagram. People are sharing their personal\nexperiences, reviews, feedbacks on the web. The information which is available\non the web is unstructured and enormous. Hence, there is a huge scope of\nresearch on understanding the sentiment of the data available on the web.\nSentiment Analysis (SA) can be carried out on the reviews, feedbacks,\ndiscussions available on the web. There has been extensive research carried out\non SA in the English language, but data on the web also contains different\nother languages which should be analyzed. This paper aims to analyze, review\nand discuss the approaches, algorithms, challenges faced by the researchers\nwhile carrying out the SA on Indigenous languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.12855v2",
            "date": "2019-11-28 20:24:11+00:00",
            "title": "Proq: Projection-based Runtime Assertions for Debugging on a Quantum Computer",
            "authors": [
                "Gushu Li",
                "Li Zhou",
                "Nengkun Yu",
                "Yufei Ding",
                "Mingsheng Ying",
                "Yuan Xie"
            ],
            "abstract": "In this paper, we propose Proq, a runtime assertion scheme for testing and\ndebugging quantum programs on a quantum computer. The predicates in Proq are\nrepresented by projections (or equivalently, closed subspaces of the state\nspace), following Birkhoff-von Neumann quantum logic. The satisfaction of a\nprojection by a quantum state can be directly checked upon a small number of\nprojective measurements rather than a large number of repeated executions. On\nthe theory side, we rigorously prove that checking projection-based assertions\ncan help locate bugs or statistically assure that the semantic function of the\ntested program is close to what we expect, for both exact and approximate\nquantum programs. On the practice side, we consider hardware constraints and\nintroduce several techniques to transform the assertions, making them directly\nexecutable on the measurement-restricted quantum computers. We also propose to\nachieve simplified assertion implementation using local projection technique\nwith soundness guaranteed. We compare Proq with existing quantum program\nassertions and demonstrate the effectiveness and efficiency of Proq by its\napplications to assert two ingenious quantum algorithms, the\nHarrow-Hassidim-Lloyd algorithm and Shor's algorithm.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13087v2",
            "date": "2019-11-29 12:55:41+00:00",
            "title": "Kurdish (Sorani) Speech to Text: Presenting an Experimental Dataset",
            "authors": [
                "Akam Qader",
                "Hossein Hassani"
            ],
            "abstract": "We present an experimental dataset, Basic Dataset for Sorani Kurdish\nAutomatic Speech Recognition (BD-4SK-ASR), which we used in the first attempt\nin developing an automatic speech recognition for Sorani Kurdish. The objective\nof the project was to develop a system that automatically could recognize\nsimple sentences based on the vocabulary which is used in grades one to three\nof the primary schools in the Kurdistan Region of Iraq. We used CMUSphinx as\nour experimental environment. We developed a dataset to train the system. The\ndataset is publicly available for non-commercial use under the CC BY-NC-SA 4.0\nlicense.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.13096v1",
            "date": "2019-11-29 13:19:45+00:00",
            "title": "Method and Dataset Mining in Scientific Papers",
            "authors": [
                "Rujing Yao",
                "Linlin Hou",
                "Yingchun Ye",
                "Ou Wu",
                "Ji Zhang",
                "Jian Wu"
            ],
            "abstract": "Literature analysis facilitates researchers better understanding the\ndevelopment of science and technology. The conventional literature analysis\nfocuses on the topics, authors, abstracts, keywords, references, etc., and\nrarely pays attention to the content of papers. In the field of machine\nlearning, the involved methods (M) and datasets (D) are key information in\npapers. The extraction and mining of M and D are useful for discipline analysis\nand algorithm recommendation. In this paper, we propose a novel entity\nrecognition model, called MDER, and constructe datasets from the papers of the\nPAKDD conferences (2009-2019). Some preliminary experiments are conducted to\nassess the extraction performance and the mining results are visualized.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.00124v2",
            "date": "2019-11-30 03:45:17+00:00",
            "title": "A Free Lunch in Generating Datasets: Building a VQG and VQA System with Attention and Humans in the Loop",
            "authors": [
                "Jihyeon Lee",
                "Sho Arora"
            ],
            "abstract": "Despite their importance in training artificial intelligence systems, large\ndatasets remain challenging to acquire. For example, the ImageNet dataset\nrequired fourteen million labels of basic human knowledge, such as whether an\nimage contains a chair. Unfortunately, this knowledge is so simple that it is\ntedious for human annotators but also tacit enough such that they are\nnecessary. However, human collaborative efforts for tasks like labeling massive\namounts of data are costly, inconsistent, and prone to failure, and this method\ndoes not resolve the issue of the resulting dataset being static in nature.\nWhat if we asked people questions they want to answer and collected their\nresponses as data? This would mean we could gather data at a much lower cost,\nand expanding a dataset would simply become a matter of asking more questions.\nWe focus on the task of Visual Question Answering (VQA) and propose a system\nthat uses Visual Question Generation (VQG) to produce questions, asks them to\nsocial media users, and collects their responses. We present two models that\ncan then parse clean answers from the noisy human responses significantly\nbetter than our baselines, with the goal of eventually incorporating the\nanswers into a Visual Question Answering (VQA) dataset. By demonstrating how\nour system can collect large amounts of data at little to no cost, we envision\nsimilar systems being used to improve performance on other tasks in the future.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.00127v3",
            "date": "2019-11-30 04:00:31+00:00",
            "title": "A Hybrid Approach Towards Two Stage Bengali Question Classification Utilizing Smart Data Balancing Technique",
            "authors": [
                "Md. Hasibur Rahman",
                "Chowdhury Rafeed Rahman",
                "Ruhul Amin",
                "Md. Habibur Rahman Sifat",
                "Afra Anika"
            ],
            "abstract": "Question classification (QC) is the primary step of the Question Answering\n(QA) system. Question Classification (QC) system classifies the questions in\nparticular classes so that Question Answering (QA) System can provide correct\nanswers for the questions. Our system categorizes the factoid type questions\nasked in natural language after extracting features of the questions. We\npresent a two stage QC system for Bengali. It utilizes one dimensional\nconvolutional neural network for classifying questions into coarse classes in\nthe first stage. Word2vec representation of existing words of the question\ncorpus have been constructed and used for assisting 1D CNN. A smart data\nbalancing technique has been employed for giving data hungry convolutional\nneural network the advantage of a greater number of effective samples to learn\nfrom. For each coarse class, a separate Stochastic Gradient Descent (SGD) based\nclassifier has been used in order to differentiate among the finer classes\nwithin that coarse class. TF-IDF representation of each word has been used as\nfeature for the SGD classifiers implemented as part of second stage\nclassification. Experiments show the effectiveness of our proposed method for\nBengali question classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00311v1",
            "date": "2019-12-01 03:30:00+00:00",
            "title": "Learning to Relate from Captions and Bounding Boxes",
            "authors": [
                "Sarthak Garg",
                "Joel Ruben Antony Moniz",
                "Anshu Aviral",
                "Priyatham Bollimpalli"
            ],
            "abstract": "In this work, we propose a novel approach that predicts the relationships\nbetween various entities in an image in a weakly supervised manner by relying\non image captions and object bounding box annotations as the sole source of\nsupervision. Our proposed approach uses a top-down attention mechanism to align\nentities in captions to objects in the image, and then leverage the syntactic\nstructure of the captions to align the relations. We use these alignments to\ntrain a relation classification network, thereby obtaining both grounded\ncaptions and dense relationships. We demonstrate the effectiveness of our model\non the Visual Genome dataset by achieving a recall@50 of 15% and recall@100 of\n25% on the relationships present in the image. We also show that the model\nsuccessfully predicts relations that are not present in the corresponding\ncaptions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.00315v2",
            "date": "2019-12-01 04:22:51+00:00",
            "title": "Topic-aware chatbot using Recurrent Neural Networks and Nonnegative Matrix Factorization",
            "authors": [
                "Yuchen Guo",
                "Nicholas Hanoian",
                "Zhexiao Lin",
                "Nicholas Liskij",
                "Hanbaek Lyu",
                "Deanna Needell",
                "Jiahao Qu",
                "Henry Sojico",
                "Yuliang Wang",
                "Zhe Xiong",
                "Zhenhong Zou"
            ],
            "abstract": "We propose a novel model for a topic-aware chatbot by combining the\ntraditional Recurrent Neural Network (RNN) encoder-decoder model with a topic\nattention layer based on Nonnegative Matrix Factorization (NMF). After learning\ntopic vectors from an auxiliary text corpus via NMF, the decoder is trained so\nthat it is more likely to sample response words from the most correlated topic\nvectors. One of the main advantages in our architecture is that the user can\neasily switch the NMF-learned topic vectors so that the chatbot obtains desired\ntopic-awareness. We demonstrate our model by training on a single\nconversational data set which is then augmented with topic matrices learned\nfrom different auxiliary data sets. We show that our topic-aware chatbot not\nonly outperforms the non-topic counterpart, but also that each topic-aware\nmodel qualitatively and contextually gives the most relevant answer depending\non the topic of question.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00349v1",
            "date": "2019-12-01 07:57:41+00:00",
            "title": "Not All Attention Is Needed: Gated Attention Network for Sequence Data",
            "authors": [
                "Lanqing Xue",
                "Xiaopeng Li",
                "Nevin L. Zhang"
            ],
            "abstract": "Although deep neural networks generally have fixed network structures, the\nconcept of dynamic mechanism has drawn more and more attention in recent years.\nAttention mechanisms compute input-dependent dynamic attention weights for\naggregating a sequence of hidden states. Dynamic network configuration in\nconvolutional neural networks (CNNs) selectively activates only part of the\nnetwork at a time for different inputs. In this paper, we combine the two\ndynamic mechanisms for text classification tasks. Traditional attention\nmechanisms attend to the whole sequence of hidden states for an input sentence,\nwhile in most cases not all attention is needed especially for long sequences.\nWe propose a novel method called Gated Attention Network (GA-Net) to\ndynamically select a subset of elements to attend to using an auxiliary\nnetwork, and compute attention weights to aggregate the selected elements. It\navoids a significant amount of unnecessary computation on unattended elements,\nand allows the model to pay attention to important parts of the sequence.\nExperiments in various datasets show that the proposed method achieves better\nperformance compared with all baseline models with global or local attention\nwhile requiring less computation and achieving better interpretability. It is\nalso promising to extend the idea to more complex attention-based models, such\nas transformers and seq-to-seq models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.00398v2",
            "date": "2019-12-01 13:03:03+00:00",
            "title": "Deep Human Answer Understanding for Natural Reverse QA",
            "authors": [
                "Rujing Yao",
                "Linlin Hou",
                "Lei Yang",
                "Jie Gui",
                "Qing Yin",
                "Ou Wu"
            ],
            "abstract": "This study focuses on a reverse question answering (QA) procedure, in which\nmachines proactively raise questions and humans supply the answers. This\nprocedure exists in many real human-machine interaction applications. However,\na crucial problem in human-machine interaction is answer understanding. The\nexisting solutions have relied on mandatory option term selection to avoid\nautomatic answer understanding. However, these solutions have led to unnatural\nhuman-computer interaction and negatively affected user experience. To this\nend, the current study proposes a novel deep answer understanding network,\ncalled AntNet, for reverse QA. The network consists of three new modules,\nnamely, skeleton attention for questions, relevance-aware representation of\nanswers, and multi-hop based fusion. As answer understanding for reverse QA has\nnot been explored, a new data corpus is compiled in this study. Experimental\nresults indicate that our proposed network is significantly better than\nexisting methods and those modified from classical natural language processing\ndeep models. The effectiveness of the three new modules is also verified.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00501v1",
            "date": "2019-12-01 21:32:11+00:00",
            "title": "Interpreting Context of Images using Scene Graphs",
            "authors": [
                "Himangi Mittal",
                "Ajith Abraham",
                "Anuja Arora"
            ],
            "abstract": "Understanding a visual scene incorporates objects, relationships, and\ncontext. Traditional methods working on an image mostly focus on object\ndetection and fail to capture the relationship between the objects.\nRelationships can give rich semantic information about the objects in a scene.\nThe context can be conducive to comprehending an image since it will help us to\nperceive the relation between the objects and thus, give us a deeper insight\ninto the image. Through this idea, our project delivers a model that focuses on\nfinding the context present in an image by representing the image as a graph,\nwhere the nodes will the objects and edges will be the relation between them.\nThe context is found using the visual and semantic cues which are further\nconcatenated and given to the Support Vector Machines (SVM) to detect the\nrelation between two objects. This presents us with the context of the image\nwhich can be further used in applications such as similar image retrieval,\nimage captioning, or story generation.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.00512v2",
            "date": "2019-12-01 22:36:14+00:00",
            "title": "Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning",
            "authors": [
                "Ugur Kursuncu",
                "Manas Gaur",
                "Amit Sheth"
            ],
            "abstract": "Learning the underlying patterns in data goes beyond instance-based\ngeneralization to external knowledge represented in structured graphs or\nnetworks. Deep learning that primarily constitutes neural computing stream in\nAI has shown significant advances in probabilistically learning latent patterns\nusing a multi-layered network of computational nodes (i.e., neurons/hidden\nunits). Structured knowledge that underlies symbolic computing approaches and\noften supports reasoning, has also seen significant growth in recent years, in\nthe form of broad-based (e.g., DBPedia, Yago) and domain, industry or\napplication specific knowledge graphs. A common substrate with careful\nintegration of the two will raise opportunities to develop neuro-symbolic\nlearning approaches for AI, where conceptual and probabilistic representations\nare combined. As the incorporation of external knowledge will aid in\nsupervising the learning of features for the model, deep infusion of\nrepresentational knowledge from knowledge graphs within hidden layers will\nfurther enhance the learning process. Although much work remains, we believe\nthat knowledge graphs will play an increasing role in developing hybrid\nneuro-symbolic intelligent systems (bottom-up deep learning with top-down\nsymbolic computing) as well as in building explainable AI systems for which\nknowledge graphs will provide scaffolding for punctuating neural computing. In\nthis position paper, we describe our motivation for such a neuro-symbolic\napproach and framework that combines knowledge graph and neural networks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.00547v1",
            "date": "2019-12-02 02:12:15+00:00",
            "title": "Large-scale text processing pipeline with Apache Spark",
            "authors": [
                "Alexey Svyatkovskiy",
                "Kosuke Imai",
                "Mary Kroeger",
                "Yuki Shiraito"
            ],
            "abstract": "In this paper, we evaluate Apache Spark for a data-intensive machine learning\nproblem. Our use case focuses on policy diffusion detection across the state\nlegislatures in the United States over time. Previous work on policy diffusion\nhas been unable to make an all-pairs comparison between bills due to\ncomputational intensity. As a substitute, scholars have studied single topic\nareas.\n  We provide an implementation of this analysis workflow as a distributed text\nprocessing pipeline with Spark dataframes and Scala application programming\ninterface. We discuss the challenges and strategies of unstructured data\nprocessing, data formats for storage and efficient access, and graph processing\nat scale.",
            "journal ref": "Published in Proceedings of Big NLP workshop at the IEEE Big Data\n  Conference 2016",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00667v1",
            "date": "2019-12-02 10:18:27+00:00",
            "title": "A Human-AI Loop Approach for Joint Keyword Discovery and Expectation Estimation in Micropost Event Detection",
            "authors": [
                "Akansha Bhardwaj",
                "Jie Yang",
                "Philippe Cudr\u00e9-Mauroux"
            ],
            "abstract": "Microblogging platforms such as Twitter are increasingly being used in event\ndetection. Existing approaches mainly use machine learning models and rely on\nevent-related keywords to collect the data for model training. These approaches\nmake strong assumptions on the distribution of the relevant micro-posts\ncontaining the keyword -- referred to as the expectation of the distribution --\nand use it as a posterior regularization parameter during model training. Such\napproaches are, however, limited as they fail to reliably estimate the\ninformativeness of a keyword and its expectation for model training. This paper\nintroduces a Human-AI loop approach to jointly discover informative keywords\nfor model training while estimating their expectation. Our approach iteratively\nleverages the crowd to estimate both keyword specific expectation and the\ndisagreement between the crowd and the model in order to discover new keywords\nthat are most beneficial for model training. These keywords and their\nexpectation not only improve the resulting performance but also make the model\ntraining process more transparent. We empirically demonstrate the merits of our\napproach, both in terms of accuracy and interpretability, on multiple\nreal-world datasets and show that our approach improves the state of the art by\n24.3%.",
            "journal ref": "AAAI, 2020",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.00690v1",
            "date": "2019-12-02 11:32:53+00:00",
            "title": "EduBERT: Pretrained Deep Language Models for Learning Analytics",
            "authors": [
                "Benjamin Clavi\u00e9",
                "Kobi Gal"
            ],
            "abstract": "The use of large pretrained neural networks to create contextualized word\nembeddings has drastically improved performance on several natural language\nprocessing (NLP) tasks. These computationally expensive models have begun to be\napplied to domain-specific NLP tasks such as re-hospitalization prediction from\nclinical notes. This paper demonstrates that using large pretrained models\nproduces excellent results on common learning analytics tasks. Pre-training\ndeep language models using student forum data from a wide array of online\ncourses improves performance beyond the state of the art on three text\nclassification tasks. We also show that a smaller, distilled version of our\nmodel produces the best results on two of the three tasks while limiting\ncomputational cost. We make both models available to the research community at\nlarge.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1912.00730v1",
            "date": "2019-12-02 12:57:52+00:00",
            "title": "SemEval-2017 Task 3: Community Question Answering",
            "authors": [
                "Preslav Nakov",
                "Doris Hoogeveen",
                "Llu\u00eds M\u00e0rquez",
                "Alessandro Moschitti",
                "Hamdy Mubarak",
                "Timothy Baldwin",
                "Karin Verspoor"
            ],
            "abstract": "We describe SemEval-2017 Task 3 on Community Question Answering. This year,\nwe reran the four subtasks from SemEval-2016:(A) Question-Comment\nSimilarity,(B) Question-Question Similarity,(C) Question-External Comment\nSimilarity, and (D) Rerank the correct answers for a new question in Arabic,\nproviding all the data from 2015 and 2016 for training, and fresh data for\ntesting. Additionally, we added a new subtask E in order to enable\nexperimentation with Multi-domain Question Duplicate Detection in a\nlarger-scale scenario, using StackExchange subforums. A total of 23 teams\nparticipated in the task, and submitted a total of 85 runs (36 primary and 49\ncontrastive) for subtasks A-D. Unfortunately, no teams participated in subtask\nE. A variety of approaches and features were used by the participating systems\nto address the different subtasks. The best systems achieved an official score\n(MAP) of 88.43, 47.22, 15.46, and 61.16 in subtasks A, B, C, and D,\nrespectively. These scores are better than the baselines, especially for\nsubtasks A-C.",
            "journal ref": "SemEval-2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00741v1",
            "date": "2019-12-02 13:04:35+00:00",
            "title": "SemEval-2017 Task 4: Sentiment Analysis in Twitter",
            "authors": [
                "Sara Rosenthal",
                "Noura Farra",
                "Preslav Nakov"
            ],
            "abstract": "This paper describes the fifth year of the Sentiment Analysis in Twitter\ntask. SemEval-2017 Task 4 continues with a rerun of the subtasks of\nSemEval-2016 Task 4, which include identifying the overall sentiment of the\ntweet, sentiment towards a topic with classification on a two-point and on a\nfive-point ordinal scale, and quantification of the distribution of sentiment\ntowards a topic across a number of tweets: again on a two-point and on a\nfive-point ordinal scale. Compared to 2016, we made two changes: (i) we\nintroduced a new language, Arabic, for all subtasks, and (ii)~we made available\ninformation from the profiles of the Twitter users who posted the target\ntweets. The task continues to be very popular, with a total of 48 teams\nparticipating this year.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00778v1",
            "date": "2019-11-27 15:48:26+00:00",
            "title": "Learning a faceted customer segmentation for discovering new business opportunities at Intel",
            "authors": [
                "Itay Lieder",
                "Meirav Segal",
                "Eran Avidan",
                "Asaf Cohen",
                "Tom Hope"
            ],
            "abstract": "For sales and marketing organizations within large enterprises, identifying\nand understanding new markets, customers and partners is a key challenge.\nIntel's Sales and Marketing Group (SMG) faces similar challenges while growing\nin new markets and domains and evolving its existing business. In today's\ncomplex technological and commercial landscape, there is need for intelligent\nautomation supporting a fine-grained understanding of businesses in order to\nhelp SMG sift through millions of companies across many geographies and\nlanguages and identify relevant directions. We present a system developed in\nour company that mines millions of public business web pages, and extracts a\nfaceted customer representation. We focus on two key customer aspects that are\nessential for finding relevant opportunities: industry segments (ranging from\nbroad verticals such as healthcare, to more specific fields such as 'video\nanalytics') and functional roles (e.g., 'manufacturer' or 'retail'). To address\nthe challenge of labeled data collection, we enrich our data with external\ninformation gleaned from Wikipedia, and develop a semi-supervised multi-label,\nmulti-lingual deep learning model that parses customer website texts and\nclassifies them into their respective facets. Our system scans and indexes\ncompanies as part of a large-scale knowledge graph that currently holds tens of\nmillions of connected entities with thousands being fetched, enriched and\nconnected to the graph by the hour in real time, and also supports knowledge\nand insight discovery. In experiments conducted in our company, we are able to\nsignificantly boost the performance of sales personnel in the task of\ndiscovering new customers and commercial partnership opportunities.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.00819v3",
            "date": "2019-12-02 14:29:22+00:00",
            "title": "EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators",
            "authors": [
                "Chandrakant Bothe",
                "Cornelius Weber",
                "Sven Magg",
                "Stefan Wermter"
            ],
            "abstract": "The recognition of emotion and dialogue acts enriches conversational analysis\nand help to build natural dialogue systems. Emotion interpretation makes us\nunderstand feelings and dialogue acts reflect the intentions and performative\nfunctions in the utterances. However, most of the textual and multi-modal\nconversational emotion corpora contain only emotion labels but not dialogue\nacts. To address this problem, we propose to use a pool of various recurrent\nneural models trained on a dialogue act corpus, with and without context. These\nneural models annotate the emotion corpora with dialogue act labels, and an\nensemble annotator extracts the final dialogue act label. We annotated two\naccessible multi-modal emotion corpora: IEMOCAP and MELD. We analyzed the\nco-occurrence of emotion and dialogue act labels and discovered specific\nrelations. For example, Accept/Agree dialogue acts often occur with the Joy\nemotion, Apology with Sadness, and Thanking with Joy. We make the Emotional\nDialogue Acts (EDA) corpus publicly available to the research community for\nfurther study and analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00846v2",
            "date": "2019-11-29 13:23:23+00:00",
            "title": "Attentive Modality Hopping Mechanism for Speech Emotion Recognition",
            "authors": [
                "Seunghyun Yoon",
                "Subhadeep Dey",
                "Hwanhee Lee",
                "Kyomin Jung"
            ],
            "abstract": "In this work, we explore the impact of visual modality in addition to speech\nand text for improving the accuracy of the emotion detection system. The\ntraditional approaches tackle this task by fusing the knowledge from the\nvarious modalities independently for performing emotion classification. In\ncontrast to these approaches, we tackle the problem by introducing an attention\nmechanism to combine the information. In this regard, we first apply a neural\nnetwork to obtain hidden representations of the modalities. Then, the attention\nmechanism is defined to select and aggregate important parts of the video data\nby conditioning on the audio and text data. Furthermore, the attention\nmechanism is again applied to attend important parts of the speech and textual\ndata, by considering other modality. Experiments are performed on the standard\nIEMOCAP dataset using all three modalities (audio, text, and video). The\nachieved results show a significant improvement of 3.65% in terms of weighted\naccuracy compared to the baseline system.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.00864v1",
            "date": "2019-11-25 07:06:25+00:00",
            "title": "Conclusion-Supplement Answer Generation for Non-Factoid Questions",
            "authors": [
                "Makoto Nakatsuji",
                "Sohei Okui"
            ],
            "abstract": "This paper tackles the goal of conclusion-supplement answer generation for\nnon-factoid questions, which is a critical issue in the field of Natural\nLanguage Processing (NLP) and Artificial Intelligence (AI), as users often\nrequire supplementary information before accepting a conclusion. The current\nencoder-decoder framework, however, has difficulty generating such answers,\nsince it may become confused when it tries to learn several different long\nanswers to the same non-factoid question. Our solution, called an ensemble\nnetwork, goes beyond single short sentences and fuses logically connected\nconclusion statements and supplementary statements. It extracts the context\nfrom the conclusion decoder's output sequence and uses it to create\nsupplementary decoder states on the basis of an attention mechanism. It also\nassesses the closeness of the question encoder's output sequence and the\nseparate outputs of the conclusion and supplement decoders as well as their\ncombination. As a result, it generates answers that match the questions and\nhave natural-sounding supplementary sequences in line with the context\nexpressed by the conclusion sequence. Evaluations conducted on datasets\nincluding \"Love Advice\" and \"Arts & Humanities\" categories indicate that our\nmodel outputs much more accurate results than the tested baseline models do.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00871v1",
            "date": "2019-12-02 15:42:06+00:00",
            "title": "Solving Arithmetic Word Problems Automatically Using Transformer and Unambiguous Representations",
            "authors": [
                "Kaden Griffith",
                "Jugal Kalita"
            ],
            "abstract": "Constructing accurate and automatic solvers of math word problems has proven\nto be quite challenging. Prior attempts using machine learning have been\ntrained on corpora specific to math word problems to produce arithmetic\nexpressions in infix notation before answer computation. We find that\ncustom-built neural networks have struggled to generalize well. This paper\noutlines the use of Transformer networks trained to translate math word\nproblems to equivalent arithmetic expressions in infix, prefix, and postfix\nnotations. In addition to training directly on domain-specific corpora, we use\nan approach that pre-trains on a general text corpus to provide foundational\nlanguage abilities to explore if it improves performance. We compare results\nproduced by a large number of neural configurations and find that most\nconfigurations outperform previously reported approaches on three of four\ndatasets with significant increases in accuracy of over 20 percentage points.\nThe best neural approaches boost accuracy by almost 10% on average when\ncompared to the previous state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00879v3",
            "date": "2019-12-02 15:57:40+00:00",
            "title": "Improving Question Generation with Sentence-level Semantic Matching and Answer Position Inferring",
            "authors": [
                "Xiyao Ma",
                "Qile Zhu",
                "Yanlin Zhou",
                "Xiaolin Li",
                "Dapeng Wu"
            ],
            "abstract": "Taking an answer and its context as input, sequence-to-sequence models have\nmade considerable progress on question generation. However, we observe that\nthese approaches often generate wrong question words or keywords and copy\nanswer-irrelevant words from the input. We believe that lacking global question\nsemantics and exploiting answer position-awareness not well are the key root\ncauses. In this paper, we propose a neural question generation model with two\nconcrete modules: sentence-level semantic matching and answer position\ninferring. Further, we enhance the initial state of the decoder by leveraging\nthe answer-aware gated fusion mechanism. Experimental results demonstrate that\nour model outperforms the state-of-the-art (SOTA) models on SQuAD and MARCO\ndatasets. Owing to its generality, our work also improves the existing models\nsignificantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00955v3",
            "date": "2019-12-02 17:32:59+00:00",
            "title": "Dynamic Prosody Generation for Speech Synthesis using Linguistics-Driven Acoustic Embedding Selection",
            "authors": [
                "Shubhi Tyagi",
                "Marco Nicolis",
                "Jonas Rohnke",
                "Thomas Drugman",
                "Jaime Lorenzo-Trueba"
            ],
            "abstract": "Recent advances in Text-to-Speech (TTS) have improved quality and naturalness\nto near-human capabilities when considering isolated sentences. But something\nwhich is still lacking in order to achieve human-like communication is the\ndynamic variations and adaptability of human speech. This work attempts to\nsolve the problem of achieving a more dynamic and natural intonation in TTS\nsystems, particularly for stylistic speech such as the newscaster speaking\nstyle. We propose a novel embedding selection approach which exploits\nlinguistic information, leveraging the speech variability present in the\ntraining dataset. We analyze the contribution of both semantic and syntactic\nfeatures. Our results show that the approach improves the prosody and\nnaturalness for complex utterances as well as in Long Form Reading (LFR).",
            "journal ref": "INTERSPEECH 2020: 4407-4411",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.00982v3",
            "date": "2019-12-02 18:21:31+00:00",
            "title": "TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP",
            "authors": [
                "Nils Rethmeier",
                "Vageesh Kumar Saxena",
                "Isabelle Augenstein"
            ],
            "abstract": "While state-of-the-art NLP explainability (XAI) methods focus on explaining\nper-sample decisions in supervised end or probing tasks, this is insufficient\nto explain and quantify model knowledge transfer during (un-)supervised\ntraining. Thus, for TX-Ray, we modify the established computer vision\nexplainability principle of 'visualizing preferred inputs of neurons' to make\nit usable transfer analysis and NLP. This allows one to analyze, track and\nquantify how self- or supervised NLP models first build knowledge abstractions\nin pretraining (1), and then transfer these abstractions to a new domain (2),\nor adapt them during supervised fine-tuning (3). TX-Ray expresses neurons as\nfeature preference distributions to quantify fine-grained knowledge transfer or\nadaptation and guide human analysis. We find that, similar to Lottery Ticket\nbased pruning, TX-Ray based pruning can improve test set generalization and\nthat it can reveal how early stages of self-supervision automatically learn\nlinguistic abstractions like parts-of-speech.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01070v1",
            "date": "2019-12-02 20:37:18+00:00",
            "title": "Simultaneously Linking Entities and Extracting Relations from Biomedical Text Without Mention-level Supervision",
            "authors": [
                "Trapit Bansal",
                "Pat Verga",
                "Neha Choudhary",
                "Andrew McCallum"
            ],
            "abstract": "Understanding the meaning of text often involves reasoning about entities and\ntheir relationships. This requires identifying textual mentions of entities,\nlinking them to a canonical concept, and discerning their relationships. These\ntasks are nearly always viewed as separate components within a pipeline, each\nrequiring a distinct model and training data. While relation extraction can\noften be trained with readily available weak or distant supervision, entity\nlinkers typically require expensive mention-level supervision -- which is not\navailable in many domains. Instead, we propose a model which is trained to\nsimultaneously produce entity linking and relation decisions while requiring no\nmention-level annotations. This approach avoids cascading errors that arise\nfrom pipelined methods and more accurately predicts entity relationships from\ntext. We show that our model outperforms a state-of-the art entity linking and\nrelation extraction pipeline on two biomedical datasets and can drastically\nimprove the overall recall of the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01109v1",
            "date": "2019-11-18 13:28:37+00:00",
            "title": "On the Vietnamese Name Entity Recognition: A Deep Learning Method Approach",
            "authors": [
                "Ngoc C. L\u00ea",
                "Ngoc-Yen Nguyen",
                "Anh-Duong Trinh"
            ],
            "abstract": "Named entity recognition (NER) plays an important role in text-based\ninformation retrieval. In this paper, we combine Bidirectional Long Short-Term\nMemory (Bi-LSTM) \\cite{hochreiter1997,schuster1997} with Conditional Random\nField (CRF) \\cite{lafferty2001} to create a novel deep learning model for the\nNER problem. Each word as input of the deep learning model is represented by a\nWord2vec-trained vector. A word embedding set trained from about one million\narticles in 2018 collected through a Vietnamese news portal (baomoi.com). In\naddition, we concatenate a Word2Vec\\cite{mikolov2013}-trained vector with\nsemantic feature vector (Part-Of-Speech (POS) tagging, chunk-tag) and hidden\nsyntactic feature vector (extracted by Bi-LSTM nerwork) to achieve the (so far\nbest) result in Vietnamese NER system. The result was conducted on the data set\nVLSP2016 (Vietnamese Language and Speech Processing 2016 \\cite{vlsp2016})\ncompetition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01110v1",
            "date": "2019-11-19 04:50:45+00:00",
            "title": "A Multi-language Platform for Generating Algebraic Mathematical Word Problems",
            "authors": [
                "Vijini Liyanage",
                "Surangika Ranathunga"
            ],
            "abstract": "Existing approaches for automatically generating mathematical word problems\nare deprived of customizability and creativity due to the inherent nature of\ntemplate-based mechanisms they employ. We present a solution to this problem\nwith the use of deep neural language generation mechanisms. Our approach uses a\nCharacter Level Long Short Term Memory Network (LSTM) to generate word\nproblems, and uses POS (Part of Speech) tags to resolve the constraints found\nin the generated problems. Our approach is capable of generating Mathematics\nWord Problems in both English and Sinhala languages with an accuracy over 90%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01116v1",
            "date": "2019-12-02 23:00:13+00:00",
            "title": "Long Distance Relationships without Time Travel: Boosting the Performance of a Sparse Predictive Autoencoder in Sequence Modeling",
            "authors": [
                "Jeremy Gordon",
                "David Rawlinson",
                "Subutai Ahmad"
            ],
            "abstract": "In sequence learning tasks such as language modelling, Recurrent Neural\nNetworks must learn relationships between input features separated by time.\nState of the art models such as LSTM and Transformer are trained by\nbackpropagation of losses into prior hidden states and inputs held in memory.\nThis allows gradients to flow from present to past and effectively learn with\nperfect hindsight, but at a significant memory cost. In this paper we show that\nit is possible to train high performance recurrent networks using information\nthat is local in time, and thereby achieve a significantly reduced memory\nfootprint. We describe a predictive autoencoder called bRSM featuring recurrent\nconnections, sparse activations, and a boosting rule for improved cell\nutilization. The architecture demonstrates near optimal performance on a\nnon-deterministic (stochastic) partially-observable sequence learning task\nconsisting of high-Markov-order sequences of MNIST digits. We find that this\nmodel learns these sequences faster and more completely than an LSTM, and offer\nseveral possible explanations why the LSTM architecture might struggle with the\npartially observable sequence structure in this task. We also apply our model\nto a next word prediction task on the Penn Treebank (PTB) dataset. We show that\na 'flattened' RSM network, when paired with a modern semantic word embedding\nand the addition of boosting, achieves 103.5 PPL (a 20-point improvement over\nthe best N-gram models), beating ordinary RNNs trained with BPTT and\napproaching the scores of early LSTM implementations. This work provides\nencouraging evidence that strong results on challenging tasks such as language\nmodelling may be possible using less memory intensive, biologically-plausible\ntraining regimes.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01219v4",
            "date": "2019-12-03 07:00:13+00:00",
            "title": "WaveFlow: A Compact Flow-based Model for Raw Audio",
            "authors": [
                "Wei Ping",
                "Kainan Peng",
                "Kexin Zhao",
                "Zhao Song"
            ],
            "abstract": "In this work, we propose WaveFlow, a small-footprint generative flow for raw\naudio, which is directly trained with maximum likelihood. It handles the\nlong-range structure of 1-D waveform with a dilated 2-D convolutional\narchitecture, while modeling the local variations using expressive\nautoregressive functions. WaveFlow provides a unified view of likelihood-based\nmodels for 1-D data, including WaveNet and WaveGlow as special cases. It\ngenerates high-fidelity speech as WaveNet, while synthesizing several orders of\nmagnitude faster as it only requires a few sequential steps to generate very\nlong waveforms with hundreds of thousands of time-steps. Furthermore, it can\nsignificantly reduce the likelihood gap that has existed between autoregressive\nmodels and flow-based models for efficient synthesis. Finally, our\nsmall-footprint WaveFlow has only 5.91M parameters, which is 15$\\times$ smaller\nthan WaveGlow. It can generate 22.05 kHz high-fidelity audio 42.6$\\times$\nfaster than real-time (at a rate of 939.3 kHz) on a V100 GPU without engineered\ninference kernels.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.01252v1",
            "date": "2019-12-03 09:20:41+00:00",
            "title": "Facilitating on-line opinion dynamics by mining expressions of causation. The case of climate change debates on The Guardian",
            "authors": [
                "Tom Willaert",
                "Sven Banisch",
                "Paul Van Eecke",
                "Katrien Beuls"
            ],
            "abstract": "News website comment sections are spaces where potentially conflicting\nopinions and beliefs are voiced. Addressing questions of how to study such\ncultural and societal conflicts through technological means, the present\narticle critically examines possibilities and limitations of machine-guided\nexploration and potential facilitation of on-line opinion dynamics. These\ninvestigations are guided by a discussion of an experimental observatory for\nmining and analyzing opinions from climate change-related user comments on news\narticles from the TheGuardian.com. This observatory combines causal mapping\nmethods with computational text analysis in order to mine beliefs and visualize\nopinion landscapes based on expressions of causation. By (1) introducing\ndigital methods and open infrastructures for data exploration and analysis and\n(2) engaging in debates about the implications of such methods and\ninfrastructures, notably in terms of the leap from opinion observation to\ndebate facilitation, the article aims to make a practical and theoretical\ncontribution to the study of opinion dynamics and conflict in new media\nenvironments.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1912.01389v2",
            "date": "2019-11-19 19:48:02+00:00",
            "title": "Towards Lingua Franca Named Entity Recognition with BERT",
            "authors": [
                "Taesun Moon",
                "Parul Awasthy",
                "Jian Ni",
                "Radu Florian"
            ],
            "abstract": "Information extraction is an important task in NLP, enabling the automatic\nextraction of data for relational database filling. Historically, research and\ndata was produced for English text, followed in subsequent years by datasets in\nArabic, Chinese (ACE/OntoNotes), Dutch, Spanish, German (CoNLL evaluations),\nand many others. The natural tendency has been to treat each language as a\ndifferent dataset and build optimized models for each. In this paper we\ninvestigate a single Named Entity Recognition model, based on a multilingual\nBERT, that is trained jointly on many languages simultaneously, and is able to\ndecode these languages with better accuracy than models trained only on one\nlanguage. To improve the initial model, we study the use of regularization\nstrategies such as multitask learning and partial gradient updates. In addition\nto being a single model that can tackle multiple languages (including code\nswitch), the model could be used to make zero-shot predictions on a new\nlanguage, even ones for which training data is not available, out of the box.\nThe results show that this model not only performs competitively with\nmonolingual models, but it also achieves state-of-the-art results on the\nCoNLL02 Dutch and Spanish datasets, OntoNotes Arabic and Chinese datasets.\nMoreover, it performs reasonably well on unseen languages, achieving\nstate-of-the-art for zero-shot on three CoNLL languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01521v1",
            "date": "2019-12-03 16:51:09+00:00",
            "title": "Multiscale Self Attentive Convolutions for Vision and Language Modeling",
            "authors": [
                "Oren Barkan"
            ],
            "abstract": "Self attention mechanisms have become a key building block in many\nstate-of-the-art language understanding models. In this paper, we show that the\nself attention operator can be formulated in terms of 1x1 convolution\noperations. Following this observation, we propose several novel operators:\nFirst, we introduce a 2D version of self attention that is applicable for 2D\nsignals such as images. Second, we present the 1D and 2D Self Attentive\nConvolutions (SAC) operator that generalizes self attention beyond 1x1\nconvolutions to 1xm and nxm convolutions, respectively. While 1D and 2D self\nattention operate on individual words and pixels, SAC operates on m-grams and\nimage patches, respectively. Third, we present a multiscale version of SAC\n(MSAC) which analyzes the input by employing multiple SAC operators that vary\nby filter size, in parallel. Finally, we explain how MSAC can be utilized for\nvision and language modeling, and further harness MSAC to form a cross\nattentive image similarity machinery.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01580v2",
            "date": "2019-12-03 18:26:13+00:00",
            "title": "A Comparative Study of Pretrained Language Models on Thai Social Text Categorization",
            "authors": [
                "Thanapapas Horsuwan",
                "Kasidis Kanwatchara",
                "Peerapon Vateekul",
                "Boonserm Kijsirikul"
            ],
            "abstract": "The ever-growing volume of data of user-generated content on social media\nprovides a nearly unlimited corpus of unlabeled data even in languages where\nresources are scarce. In this paper, we demonstrate that state-of-the-art\nresults on two Thai social text categorization tasks can be realized by\npretraining a language model on a large noisy Thai social media corpus of over\n1.26 billion tokens and later fine-tuned on the downstream classification\ntasks. Due to the linguistically noisy and domain-specific nature of the\ncontent, our unique data preprocessing steps designed for Thai social media\nwere utilized to ease the training comprehension of the model. We compared four\nmodern language models: ULMFiT, ELMo with biLSTM, OpenAI GPT, and BERT. We\nsystematically compared the models across different dimensions including speed\nof pretraining and fine-tuning, perplexity, downstream classification\nbenchmarks, and performance in limited pretraining data.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01679v2",
            "date": "2019-12-03 20:32:50+00:00",
            "title": "Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition",
            "authors": [
                "Shaoshi Ling",
                "Yuzong Liu",
                "Julian Salazar",
                "Katrin Kirchhoff"
            ],
            "abstract": "We propose a novel approach to semi-supervised automatic speech recognition\n(ASR). We first exploit a large amount of unlabeled audio data via\nrepresentation learning, where we reconstruct a temporal slice of filterbank\nfeatures from past and future context frames. The resulting deep contextualized\nacoustic representations (DeCoAR) are then used to train a CTC-based end-to-end\nASR system using a smaller amount of labeled audio data. In our experiments, we\nshow that systems trained on DeCoAR consistently outperform ones trained on\nconventional filterbank features, giving 42% and 19% relative improvement over\nthe baseline on WSJ eval92 and LibriSpeech test-clean, respectively. Our\napproach can drastically reduce the amount of labeled data required;\nunsupervised training on LibriSpeech then supervision with 100 hours of labeled\ndata achieves performance on par with training on all 960 hours directly.\nPre-trained models and code will be released online.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.01706v2",
            "date": "2019-12-03 22:07:47+00:00",
            "title": "A Robust Self-Learning Method for Fully Unsupervised Cross-Lingual Mappings of Word Embeddings: Making the Method Robustly Reproducible as Well",
            "authors": [
                "Nicolas Garneau",
                "Mathieu Godbout",
                "David Beauchemin",
                "Audrey Durand",
                "Luc Lamontagne"
            ],
            "abstract": "In this paper, we reproduce the experiments of Artetxe et al. (2018b)\nregarding the robust self-learning method for fully unsupervised cross-lingual\nmappings of word embeddings. We show that the reproduction of their method is\nindeed feasible with some minor assumptions. We further investigate the\nrobustness of their model by introducing four new languages that are less\nsimilar to English than the ones proposed by the original paper. In order to\nassess the stability of their model, we also conduct a grid search over\nsensible hyperparameters. We then propose key recommendations applicable to any\nresearch project in order to deliver fully reproducible research.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01728v2",
            "date": "2019-12-03 22:46:59+00:00",
            "title": "Fast Intent Classification for Spoken Language Understanding",
            "authors": [
                "Akshit Tyagi",
                "Varun Sharma",
                "Rahul Gupta",
                "Lynn Samson",
                "Nan Zhuang",
                "Zihang Wang",
                "Bill Campbell"
            ],
            "abstract": "Spoken Language Understanding (SLU) systems consist of several machine\nlearning components operating together (e.g. intent classification, named\nentity recognition and resolution). Deep learning models have obtained state of\nthe art results on several of these tasks, largely attributed to their better\nmodeling capacity. However, an increase in modeling capacity comes with added\ncosts of higher latency and energy usage, particularly when operating on low\ncomplexity devices. To address the latency and computational complexity issues,\nwe explore a BranchyNet scheme on an intent classification scheme within SLU\nsystems. The BranchyNet scheme when applied to a high complexity model, adds\nexit points at various stages in the model allowing early decision making for a\nset of queries to the SLU model. We conduct experiments on the Facebook\nSemantic Parsing dataset with two candidate model architectures for intent\nclassification. Our experiments show that the BranchyNet scheme provides gains\nin terms of computational complexity without compromising model accuracy. We\nalso conduct analytical studies regarding the improvements in the computational\ncost, distribution of utterances that egress from various exit points and the\nimpact of adding more complexity to models with the BranchyNet scheme.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01777v2",
            "date": "2019-12-04 03:12:27+00:00",
            "title": "Integrating Knowledge into End-to-End Speech Recognition from External Text-Only Data",
            "authors": [
                "Ye Bai",
                "Jiangyan Yi",
                "Jianhua Tao",
                "Zhengqi Wen",
                "Zhengkun Tian",
                "Shuai Zhang"
            ],
            "abstract": "Attention-based encoder-decoder (AED) models have achieved promising\nperformance in speech recognition. However, because of the end-to-end training,\nan AED model is usually trained with speech-text paired data. It is challenging\nto incorporate external text-only data into AED models. Another issue of the\nAED model is that it does not use the right context of a text token while\npredicting the token. To alleviate the above two issues, we propose a unified\nmethod called LST (Learn Spelling from Teachers) to integrate knowledge into an\nAED model from the external text-only data and leverage the whole context in a\nsentence. The method is divided into two stages. First, in the representation\nstage, a language model is trained on the text. It can be seen as that the\nknowledge in the text is compressed into the LM. Then, at the transferring\nstage, the knowledge is transferred to the AED model via teacher-student\nlearning. To further use the whole context of the text sentence, we propose an\nLM called causal cloze completer (COR), which estimates the probability of a\ntoken, given both the left context and the right context of it. Therefore, with\nLST training, the AED model can leverage the whole context in the sentence.\nDifferent from fusion based methods, which use LM during decoding, the proposed\nmethod does not increase any extra complexity at the inference stage. We\nconduct experiments on two scales of public Chinese datasets AISHELL-1 and\nAISHELL-2. The experimental results demonstrate the effectiveness of leveraging\nexternal text-only data and the whole context in a sentence with our proposed\nmethod, compared with baseline hybrid systems and AED model based systems.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.01831v1",
            "date": "2019-12-04 07:22:32+00:00",
            "title": "Towards Constructing a Corpus for Studying the Effects of Treatments and Substances Reported in PubMed Abstracts",
            "authors": [
                "Evgeni Stefchov",
                "Galia Angelova",
                "Preslav Nakov"
            ],
            "abstract": "We present the construction of an annotated corpus of PubMed abstracts\nreporting about positive, negative or neutral effects of treatments or\nsubstances. Our ultimate goal is to annotate one sentence (rationale) for each\nabstract and to use this resource as a training set for text classification of\neffects discussed in PubMed abstracts. Currently, the corpus consists of 750\nabstracts. We describe the automatic processing that supports the corpus\nconstruction, the manual annotation activities and some features of the medical\nlanguage in the abstracts selected for the annotated corpus. It turns out that\nrecognizing the terminology and the abbreviations is key for determining the\nrationale sentence. The corpus will be applied to improve our classifier, which\ncurrently has accuracy of 78.80% achieved with normalization of the abstract\nterms based on UMLS concepts from specific semantic groups and an SVM with a\nlinear kernel. Finally, we discuss some other possible applications of this\ncorpus.",
            "journal ref": "AIMSA-2016: The 17th International Conference on Artificial\n  Intelligence: Methodology, Systems, Applications",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01852v2",
            "date": "2019-12-04 08:56:13+00:00",
            "title": "PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network",
            "authors": [
                "Chengqi Deng",
                "Chengzhu Yu",
                "Heng Lu",
                "Chao Weng",
                "Dong Yu"
            ],
            "abstract": "Singing voice conversion is to convert a singer's voice to another one's\nvoice without changing singing content. Recent work shows that unsupervised\nsinging voice conversion can be achieved with an autoencoder-based approach\n[1]. However, the converted singing voice can be easily out of key, showing\nthat the existing approach cannot model the pitch information precisely. In\nthis paper, we propose to advance the existing unsupervised singing voice\nconversion method proposed in [1] to achieve more accurate pitch translation\nand flexible pitch manipulation. Specifically, the proposed PitchNet added an\nadversarially trained pitch regression network to enforce the encoder network\nto learn pitch invariant phoneme representation, and a separate module to feed\npitch extracted from the source audio to the decoder network. Our evaluation\nshows that the proposed method can greatly improve the quality of the converted\nsinging voice (2.92 vs 3.75 in MOS). We also demonstrate that the pitch of\nconverted singing can be easily controlled during generation by changing the\nlevels of the extracted pitch before passing it to the decoder network.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.01987v2",
            "date": "2019-12-04 13:56:38+00:00",
            "title": "Scalable Bayesian Preference Learning for Crowds",
            "authors": [
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "We propose a scalable Bayesian preference learning method for jointly\npredicting the preferences of individuals as well as the consensus of a crowd\nfrom pairwise labels. Peoples' opinions often differ greatly, making it\ndifficult to predict their preferences from small amounts of personal data.\nIndividual biases also make it harder to infer the consensus of a crowd when\nthere are few labels per item. We address these challenges by combining matrix\nfactorisation with Gaussian processes, using a Bayesian approach to account for\nuncertainty arising from noisy and sparse data. Our method exploits input\nfeatures, such as text embeddings and user metadata, to predict preferences for\nnew items and users that are not in the training set. As previous solutions\nbased on Gaussian processes do not scale to large numbers of users, items or\npairwise labels, we propose a stochastic variational inference approach that\nlimits computational and memory costs. Our experiments on a recommendation task\nshow that our method is competitive with previous approaches despite our\nscalable inference approximation. We demonstrate the method's scalability on a\nnatural language processing task with thousands of users and items, and show\nimprovements over the state of the art on this task. We make our software\npublicly available for future work.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.02164v4",
            "date": "2019-12-04 18:32:15+00:00",
            "title": "Plug and Play Language Models: A Simple Approach to Controlled Text Generation",
            "authors": [
                "Sumanth Dathathri",
                "Andrea Madotto",
                "Janice Lan",
                "Jane Hung",
                "Eric Frank",
                "Piero Molino",
                "Jason Yosinski",
                "Rosanne Liu"
            ],
            "abstract": "Large transformer-based language models (LMs) trained on huge text corpora\nhave shown unparalleled generation capabilities. However, controlling\nattributes of the generated language (e.g. switching topic or sentiment) is\ndifficult without modifying the model architecture or fine-tuning on\nattribute-specific data and entailing the significant cost of retraining. We\npropose a simple alternative: the Plug and Play Language Model (PPLM) for\ncontrollable language generation, which combines a pretrained LM with one or\nmore simple attribute classifiers that guide text generation without any\nfurther training of the LM. In the canonical scenario we present, the attribute\nmodels are simple classifiers consisting of a user-specified bag of words or a\nsingle learned layer with 100,000 times fewer parameters than the LM. Sampling\nentails a forward and backward pass in which gradients from the attribute model\npush the LM's hidden activations and thus guide the generation. Model samples\ndemonstrate control over a range of topics and sentiment styles, and extensive\nautomated and human annotated evaluations show attribute alignment and fluency.\nPPLMs are flexible in that any combination of differentiable attribute models\nmay be used to steer text generation, which will allow for diverse and creative\napplications beyond the examples given in this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02315v2",
            "date": "2019-12-05 00:07:35+00:00",
            "title": "12-in-1: Multi-Task Vision and Language Representation Learning",
            "authors": [
                "Jiasen Lu",
                "Vedanuj Goswami",
                "Marcus Rohrbach",
                "Devi Parikh",
                "Stefan Lee"
            ],
            "abstract": "Much of vision-and-language research focuses on a small but diverse set of\nindependent tasks and supporting datasets often studied in isolation; however,\nthe visually-grounded language understanding skills required for success at\nthese tasks overlap significantly. In this work, we investigate these\nrelationships between vision-and-language tasks by developing a large-scale,\nmulti-task training regime. Our approach culminates in a single model on 12\ndatasets from four broad categories of task including visual question\nanswering, caption-based image retrieval, grounding referring expressions, and\nmulti-modal verification. Compared to independently trained single-task models,\nthis represents a reduction from approximately 3 billion parameters to 270\nmillion while simultaneously improving performance by 2.05 points on average\nacross tasks. We use our multi-task framework to perform in-depth analysis of\nthe effect of joint training diverse tasks. Further, we show that finetuning\ntask-specific models from our single multi-task model can lead to further\nimprovements, achieving performance at or above the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.02379v2",
            "date": "2019-12-05 04:51:11+00:00",
            "title": "Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline",
            "authors": [
                "Vishvak Murahari",
                "Dhruv Batra",
                "Devi Parikh",
                "Abhishek Das"
            ],
            "abstract": "Prior work in visual dialog has focused on training deep neural models on\nVisDial in isolation. Instead, we present an approach to leverage pretraining\non related vision-language datasets before transferring to visual dialog. We\nadapt the recently proposed ViLBERT (Lu et al., 2019) model for multi-turn\nvisually-grounded conversations. Our model is pretrained on the Conceptual\nCaptions and Visual Question Answering datasets, and finetuned on VisDial. Our\nbest single model outperforms prior published work (including model ensembles)\nby more than 1% absolute on NDCG and MRR. Next, we find that additional\nfinetuning using \"dense\" annotations in VisDial leads to even higher NDCG --\nmore than 10% over our base model -- but hurts MRR -- more than 17% below our\nbase model! This highlights a trade-off between the two primary metrics -- NDCG\nand MRR -- which we find is due to dense annotations not correlating well with\nthe original ground-truth answers to questions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.02387v1",
            "date": "2019-12-05 05:08:36+00:00",
            "title": "SemEval-2015 Task 10: Sentiment Analysis in Twitter",
            "authors": [
                "Sara Rosenthal",
                "Saif M Mohammad",
                "Preslav Nakov",
                "Alan Ritter",
                "Svetlana Kiritchenko",
                "Veselin Stoyanov"
            ],
            "abstract": "In this paper, we describe the 2015 iteration of the SemEval shared task on\nSentiment Analysis in Twitter. This was the most popular sentiment analysis\nshared task to date with more than 40 teams participating in each of the last\nthree years. This year's shared task competition consisted of five sentiment\nprediction subtasks. Two were reruns from previous years: (A) sentiment\nexpressed by a phrase in the context of a tweet, and (B) overall sentiment of a\ntweet. We further included three new subtasks asking to predict (C) the\nsentiment towards a topic in a single tweet, (D) the overall sentiment towards\na topic in a set of tweets, and (E) the degree of prior polarity of a phrase.",
            "journal ref": "SemEval-2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02636v2",
            "date": "2019-12-05 15:13:15+00:00",
            "title": "Exploration of Neural Machine Translation in Autoformalization of Mathematics in Mizar",
            "authors": [
                "Qingxiang Wang",
                "Chad Brown",
                "Cezary Kaliszyk",
                "Josef Urban"
            ],
            "abstract": "In this paper we share several experiments trying to automatically translate\ninformal mathematics into formal mathematics. In our context informal\nmathematics refers to human-written mathematical sentences in the LaTeX format;\nand formal mathematics refers to statements in the Mizar language. We conducted\nour experiments against three established neural network-based machine\ntranslation models that are known to deliver competitive results on translating\nbetween natural languages. To train these models we also prepared four\ninformal-to-formal datasets. We compare and analyze our results according to\nwhether the model is supervised or unsupervised. In order to augment the data\navailable for auto-formalization and improve the results, we develop a custom\ntype-elaboration mechanism and integrate it in the supervised translation.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1912.02703v1",
            "date": "2019-12-05 16:33:23+00:00",
            "title": "Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency",
            "authors": [
                "Xing Meng",
                "Craig H. Ganoe",
                "Ryan T. Sieberg",
                "Yvonne Y. Cheung",
                "Saeed Hassanpour"
            ],
            "abstract": "Machine learning methods have recently achieved high-performance in\nbiomedical text analysis. However, a major bottleneck in the widespread\napplication of these methods is obtaining the required large amounts of\nannotated training data, which is resource intensive and time consuming. Recent\nprogress in self-supervised learning has shown promise in leveraging large text\ncorpora without explicit annotations. In this work, we built a self-supervised\ncontextual language representation model using BERT, a deep bidirectional\ntransformer architecture, to identify radiology reports requiring prompt\ncommunication to the referring physicians. We pre-trained the BERT model on a\nlarge unlabeled corpus of radiology reports and used the resulting contextual\nrepresentations in a final text classifier for communication urgency. Our model\nachieved a precision of 97.0%, recall of 93.3%, and F-measure of 95.1% on an\nindependent test set in identifying radiology reports for prompt communication,\nand significantly outperformed the previous state-of-the-art model based on\nword2vec representations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.02866v1",
            "date": "2019-12-05 20:34:53+00:00",
            "title": "Classifying Diagrams and Their Parts using Graph Neural Networks: A Comparison of Crowd-Sourced and Expert Annotations",
            "authors": [
                "Tuomo Hiippala"
            ],
            "abstract": "This article compares two multimodal resources that consist of diagrams which\ndescribe topics in elementary school natural sciences. Both resources contain\nthe same diagrams and represent their structure using graphs, but differ in\nterms of their annotation schema and how the annotations have been created -\ndepending on the resource in question - either by crowd-sourced workers or\ntrained experts. This article reports on two experiments that evaluate how\neffectively crowd-sourced and expert-annotated graphs can represent the\nmultimodal structure of diagrams for representation learning using various\ngraph neural networks. The results show that the identity of diagram elements\ncan be learned from their layout features, while the expert annotations provide\nbetter representations of diagram types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02958v2",
            "date": "2019-12-06 03:05:12+00:00",
            "title": "Synchronous Transformers for End-to-End Speech Recognition",
            "authors": [
                "Zhengkun Tian",
                "Jiangyan Yi",
                "Ye Bai",
                "Jianhua Tao",
                "Shuai Zhang",
                "Zhengqi Wen"
            ],
            "abstract": "For most of the attention-based sequence-to-sequence models, the decoder\npredicts the output sequence conditioned on the entire input sequence processed\nby the encoder. The asynchronous problem between the encoding and decoding\nmakes these models difficult to be applied for online speech recognition. In\nthis paper, we propose a model named synchronous transformer to address this\nproblem, which can predict the output sequence chunk by chunk. Once a\nfixed-length chunk of the input sequence is processed by the encoder, the\ndecoder begins to predict symbols immediately. During training, a\nforward-backward algorithm is introduced to optimize all the possible alignment\npaths. Our model is evaluated on a Mandarin dataset AISHELL-1. The experiments\nshow that the synchronous transformer is able to perform encoding and decoding\nsynchronously, and achieves a character error rate of 8.91% on the test set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.02990v1",
            "date": "2019-12-06 06:23:19+00:00",
            "title": "SemEval-2014 Task 9: Sentiment Analysis in Twitter",
            "authors": [
                "Sara Rosenthal",
                "Preslav Nakov",
                "Alan Ritter",
                "Veselin Stoyanov"
            ],
            "abstract": "We describe the Sentiment Analysis in Twitter task, ran as part of\nSemEval-2014. It is a continuation of the last year's task that ran\nsuccessfully as part of SemEval-2013. As in 2013, this was the most popular\nSemEval task; a total of 46 teams contributed 27 submissions for subtask A (21\nteams) and 50 submissions for subtask B (44 teams). This year, we introduced\nthree new test sets: (i) regular tweets, (ii) sarcastic tweets, and (iii)\nLiveJournal sentences. We further tested on (iv) 2013 tweets, and (v) 2013 SMS\nmessages. The highest F1-score on (i) was achieved by NRC-Canada at 86.63 for\nsubtask A and by TeamX at 70.96 for subtask B.",
            "journal ref": "SemEval-2014",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.02998v1",
            "date": "2019-12-06 06:35:21+00:00",
            "title": "Machine Translation Evaluation Meets Community Question Answering",
            "authors": [
                "Francisco Guzm\u00e1n",
                "Llu\u00eds M\u00e0rquez",
                "Preslav Nakov"
            ],
            "abstract": "We explore the applicability of machine translation evaluation (MTE) methods\nto a very different problem: answer ranking in community Question Answering. In\nparticular, we adopt a pairwise neural network (NN) architecture, which\nincorporates MTE features, as well as rich syntactic and semantic embeddings,\nand which efficiently models complex non-linear interactions. The evaluation\nresults show state-of-the-art performance, with sizeable contribution from both\nthe MTE features and from the pairwise NN architecture.",
            "journal ref": "Annual meeting of the Association for Computational Linguistics\n  (ACL-2016)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03010v2",
            "date": "2019-12-06 07:55:04+00:00",
            "title": "Semantic Mask for Transformer based End-to-End Speech Recognition",
            "authors": [
                "Chengyi Wang",
                "Yu Wu",
                "Yujiao Du",
                "Jinyu Li",
                "Shujie Liu",
                "Liang Lu",
                "Shuo Ren",
                "Guoli Ye",
                "Sheng Zhao",
                "Ming Zhou"
            ],
            "abstract": "Attention-based encoder-decoder model has achieved impressive results for\nboth automatic speech recognition (ASR) and text-to-speech (TTS) tasks. This\napproach takes advantage of the memorization capacity of neural networks to\nlearn the mapping from the input sequence to the output sequence from scratch,\nwithout the assumption of prior knowledge such as the alignments. However, this\nmodel is prone to overfitting, especially when the amount of training data is\nlimited. Inspired by SpecAugment and BERT, in this paper, we propose a semantic\nmask based regularization for training such kind of end-to-end (E2E) model. The\nidea is to mask the input features corresponding to a particular output token,\ne.g., a word or a word-piece, in order to encourage the model to fill the token\nbased on the contextual information. While this approach is applicable to the\nencoder-decoder framework with any type of neural network architecture, we\nstudy the transformer-based model for ASR in this work. We perform experiments\non Librispeech 960h and TedLium2 data sets, and achieve the state-of-the-art\nperformance on the test set in the scope of E2E models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03041v1",
            "date": "2019-12-06 09:38:23+00:00",
            "title": "Integrating Deep Learning with Logic Fusion for Information Extraction",
            "authors": [
                "Wenya Wang",
                "Sinno Jialin Pan"
            ],
            "abstract": "Information extraction (IE) aims to produce structured information from an\ninput text, e.g., Named Entity Recognition and Relation Extraction. Various\nattempts have been proposed for IE via feature engineering or deep learning.\nHowever, most of them fail to associate the complex relationships inherent in\nthe task itself, which has proven to be especially crucial. For example, the\nrelation between 2 entities is highly dependent on their entity types. These\ndependencies can be regarded as complex constraints that can be efficiently\nexpressed as logical rules. To combine such logic reasoning capabilities with\nlearning capabilities of deep neural networks, we propose to integrate logical\nknowledge in the form of first-order logic into a deep learning system, which\ncan be trained jointly in an end-to-end manner. The integrated framework is\nable to enhance neural outputs with knowledge regularization via logic rules,\nand at the same time update the weights of logic rules to comply with the\ncharacteristics of the training data. We demonstrate the effectiveness and\ngeneralization of the proposed model on multiple IE tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03048v1",
            "date": "2019-12-06 10:09:20+00:00",
            "title": "Document Network Embedding: Coping for Missing Content and Missing Links",
            "authors": [
                "Jean Dupuy",
                "Adrien Guille",
                "Julien Jacques"
            ],
            "abstract": "Searching through networks of documents is an important task. A promising\npath to improve the performance of information retrieval systems in this\ncontext is to leverage dense node and content representations learned with\nembedding techniques. However, these techniques cannot learn representations\nfor documents that are either isolated or whose content is missing. To tackle\nthis issue, assuming that the topology of the network and the content of the\ndocuments correlate, we propose to estimate the missing node representations\nfrom the available content representations, and conversely. Inspired by recent\nadvances in machine translation, we detail in this paper how to learn a linear\ntransformation from a set of aligned content and node representations. The\nprojection matrix is efficiently calculated in terms of the singular value\ndecomposition. The usefulness of the proposed method is highlighted by the\nimproved ability to predict the neighborhood of nodes whose links are\nunobserved based on the projected content representations, and to retrieve\nsimilar documents when content is missing, based on the projected node\nrepresentations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.03063v1",
            "date": "2019-12-06 11:04:08+00:00",
            "title": "Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks",
            "authors": [
                "Corentin Kervadec",
                "Grigory Antipov",
                "Moez Baccouche",
                "Christian Wolf"
            ],
            "abstract": "The large adoption of the self-attention (i.e. transformer model) and\nBERT-like training principles has recently resulted in a number of high\nperforming models on a large panoply of vision-and-language problems (such as\nVisual Question Answering (VQA), image retrieval, etc.). In this paper we claim\nthat these State-Of-The-Art (SOTA) approaches perform reasonably well in\nstructuring information inside a single modality but, despite their impressive\nperformances , they tend to struggle to identify fine-grained inter-modality\nrelationships. Indeed, such relations are frequently assumed to be implicitly\nlearned during training from application-specific losses, mostly cross-entropy\nfor classification. While most recent works provide inductive bias for\ninter-modality relationships via cross attention modules, in this work, we\ndemonstrate (1) that the latter assumption does not hold, i.e. modality\nalignment does not necessarily emerge automatically, and (2) that adding weak\nsupervision for alignment between visual objects and words improves the quality\nof the learned models on tasks requiring reasoning. In particular , we\nintegrate an object-word alignment loss into SOTA vision-language reasoning\nmodels and evaluate it on two tasks VQA and Language-driven Comparison of\nImages. We show that the proposed fine-grained inter-modality supervision\nsignificantly improves performance on both tasks. In particular, this new\nlearning signal allows obtaining SOTA-level performances on GQA dataset (VQA\ntask) with pre-trained models without finetuning on the task, and a new SOTA on\nNLVR2 dataset (Language-driven Comparison of Images). Finally, we also\nillustrate the impact of the contribution on the models reasoning by\nvisualizing attention distributions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.03135v1",
            "date": "2019-12-05 05:17:05+00:00",
            "title": "Pairwise Neural Machine Translation Evaluation",
            "authors": [
                "Francisco Guzman",
                "Shafiq Joty",
                "Lluis Marquez",
                "Preslav Nakov"
            ],
            "abstract": "We present a novel framework for machine translation evaluation using neural\nnetworks in a pairwise setting, where the goal is to select the better\ntranslation from a pair of hypotheses, given the reference translation. In this\nframework, lexical, syntactic and semantic information from the reference and\nthe two hypotheses is compacted into relatively small distributed vector\nrepresentations, and fed into a multi-layer neural network that models the\ninteraction between each of the hypotheses and the reference, as well as\nbetween the two hypotheses. These compact representations are in turn based on\nword and sentence embeddings, which are learned using neural networks. The\nframework is flexible, allows for efficient learning and classification, and\nyields correlation with humans that rivals the state of the art.",
            "journal ref": "Conference of the Association for Computational Linguistics\n  (ACL'2015)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03184v3",
            "date": "2019-12-06 15:30:58+00:00",
            "title": "GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception",
            "authors": [
                "Laura Bostan",
                "Evgeny Kim",
                "Roman Klinger"
            ],
            "abstract": "Most research on emotion analysis from text focuses on the task of emotion\nclassification or emotion intensity regression. Fewer works address emotions as\na phenomenon to be tackled with structured learning, which can be explained by\nthe lack of relevant datasets. We fill this gap by releasing a dataset of 5000\nEnglish news headlines annotated via crowdsourcing with their associated\nemotions, the corresponding emotion experiencers and textual cues, related\nemotion causes and targets, as well as the reader's perception of the emotion\nof the headline. This annotation task is comparably challenging, given the\nlarge number of classes and roles to be identified. We therefore propose a\nmultiphase annotation procedure in which we first find relevant instances with\nemotional content and then annotate the more fine-grained aspects. Finally, we\ndevelop a baseline for the task of automatic prediction of semantic role\nstructures and discuss the results. The corpus we release enables further\nresearch on emotion classification, emotion intensity prediction, emotion cause\ndetection, and supports further qualitative studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03234v1",
            "date": "2019-12-06 17:17:39+00:00",
            "title": "What Do You Mean I'm Funny? Personalizing the Joke Skill of a Voice-Controlled Virtual Assistant",
            "authors": [
                "Alejandro Mottini",
                "Amber Roy Chowdhury"
            ],
            "abstract": "A considerable part of the success experienced by Voice-controlled virtual\nassistants (VVA) is due to the emotional and personalized experience they\ndeliver, with humor being a key component in providing an engaging interaction.\nIn this paper we describe methods used to improve the joke skill of a VVA\nthrough personalization. The first method, based on traditional NLP techniques,\nis robust and scalable. The others combine self-attentional network and\nmulti-task learning to obtain better results, at the cost of added complexity.\nA significant challenge facing these systems is the lack of explicit user\nfeedback needed to provide labels for the models. Instead, we explore the use\nof two implicit feedback-based labelling strategies. All models were evaluated\non real production data. Online results show that models trained on any of the\nconsidered labels outperform a heuristic method, presenting a positive\nreal-world impact on user satisfaction. Offline results suggest that the\ndeep-learning approaches can improve the joke experience with respect to the\nother considered methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.03363v2",
            "date": "2019-12-06 22:09:07+00:00",
            "title": "Audio-attention discriminative language model for ASR rescoring",
            "authors": [
                "Ankur Gandhe",
                "Ariya Rastrow"
            ],
            "abstract": "End-to-end approaches for automatic speech recognition (ASR) benefit from\ndirectly modeling the probability of the word sequence given the input audio\nstream in a single neural network. However, compared to conventional ASR\nsystems, these models typically require more data to achieve comparable\nresults. Well-known model adaptation techniques, to account for domain and\nstyle adaptation, are not easily applicable to end-to-end systems. Conventional\nHMM-based systems, on the other hand, have been optimized for various\nproduction environments and use cases. In this work, we propose to combine the\nbenefits of end-to-end approaches with a conventional system using an\nattention-based discriminative language model that learns to rescore the output\nof a first-pass ASR system. We show that learning to rescore a list of\npotential ASR outputs is much simpler than learning to generate the hypothesis.\nThe proposed model results in 8% improvement in word error rate even when the\namount of training data is a fraction of data used for training the first-pass\nsystem.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.03393v2",
            "date": "2019-12-06 23:46:37+00:00",
            "title": "Re-Translation Strategies For Long Form, Simultaneous, Spoken Language Translation",
            "authors": [
                "Naveen Arivazhagan",
                "Colin Cherry",
                "Te I",
                "Wolfgang Macherey",
                "Pallavi Baljekar",
                "George Foster"
            ],
            "abstract": "We investigate the problem of simultaneous machine translation of long-form\nspeech content. We target a continuous speech-to-text scenario, generating\ntranslated captions for a live audio feed, such as a lecture or play-by-play\ncommentary. As this scenario allows for revisions to our incremental\ntranslations, we adopt a re-translation approach to simultaneous translation,\nwhere the source is repeatedly translated from scratch as it grows. This\napproach naturally exhibits very low latency and high final quality, but at the\ncost of incremental instability as the output is continuously refined. We\nexperiment with a pipeline of industry-grade speech recognition and translation\ntools, augmented with simple inference heuristics to improve stability. We use\nTED Talks as a source of multilingual test data, developing our techniques on\nEnglish-to-German spoken language translation. Our minimalist approach to\nsimultaneous translation allows us to easily scale our final evaluation to six\nmore target languages, dramatically improving incremental stability for all of\nthem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.03553v1",
            "date": "2019-12-07 20:12:43+00:00",
            "title": "Learning Norms from Stories: A Prior for Value Aligned Agents",
            "authors": [
                "Spencer Frazier",
                "Md Sultan Al Nahian",
                "Mark Riedl",
                "Brent Harrison"
            ],
            "abstract": "Value alignment is a property of an intelligent agent indicating that it can\nonly pursue goals and activities that are beneficial to humans. Traditional\napproaches to value alignment use imitation learning or preference learning to\ninfer the values of humans by observing their behavior. We introduce a\ncomplementary technique in which a value aligned prior is learned from\nnaturally occurring stories which encode societal norms. Training data is\nsourced from the childrens educational comic strip, Goofus and Gallant. In this\nwork, we train multiple machine learning models to classify natural language\ndescriptions of situations found in the comic strip as normative or non\nnormative by identifying if they align with the main characters behavior. We\nalso report the models performance when transferring to two unrelated tasks\nwith little to no additional training on the new task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.03627v1",
            "date": "2019-12-08 07:05:36+00:00",
            "title": "A Multi Purpose and Large Scale Speech Corpus in Persian and English for Speaker and Speech Recognition: the DeepMine Database",
            "authors": [
                "Hossein Zeinali",
                "Luk\u00e1\u0161 Burget",
                "Jan \"Honza'' \u010cernock\u00fd"
            ],
            "abstract": "DeepMine is a speech database in Persian and English designed to build and\nevaluate text-dependent, text-prompted, and text-independent speaker\nverification, as well as Persian speech recognition systems. It contains more\nthan 1850 speakers and 540 thousand recordings overall, more than 480 hours of\nspeech are transcribed. It is the first public large-scale speaker verification\ndatabase in Persian, the largest public text-dependent and text-prompted\nspeaker verification database in English, and the largest public evaluation\ndataset for text-independent speaker verification. It has a good coverage of\nage, gender, and accents. We provide several evaluation protocols for each part\nof the database to allow for research on different aspects of speaker\nverification. We also provide the results of several experiments that can be\nconsidered as baselines: HMM-based i-vectors for text-dependent speaker\nverification, and HMM-based as well as state-of-the-art deep neural network\nbased ASR. We demonstrate that the database can serve for training robust ASR\nmodels.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.03656v2",
            "date": "2019-12-08 11:20:35+00:00",
            "title": "Bidirectional Scene Text Recognition with a Single Decoder",
            "authors": [
                "Maurits Bleeker",
                "Maarten de Rijke"
            ],
            "abstract": "Scene Text Recognition (STR) is the problem of recognizing the correct word\nor character sequence in a cropped word image. To obtain more robust output\nsequences, the notion of bidirectional STR has been introduced. So far,\nbidirectional STRs have been implemented by using two separate decoders; one\nfor left-to-right decoding and one for right-to-left. Having two separate\ndecoders for almost the same task with the same output space is undesirable\nfrom a computational and optimization point of view. We introduce the\nbidirectional Scene Text Transformer (Bi-STET), a novel bidirectional STR\nmethod with a single decoder for bidirectional text decoding. With its single\ndecoder, Bi-STET outperforms methods that apply bidirectional decoding by using\ntwo separate decoders while also being more efficient than those methods,\nFurthermore, we achieve or beat state-of-the-art (SOTA) methods on all STR\nbenchmarks with Bi-STET. Finally, we provide analyses and insights into the\nperformance of Bi-STET.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.03720v2",
            "date": "2019-12-08 17:22:07+00:00",
            "title": "Attentive Representation Learning with Adversarial Training for Short Text Clustering",
            "authors": [
                "Wei Zhang",
                "Chao Dong",
                "Jianhua Yin",
                "Jianyong Wang"
            ],
            "abstract": "Short text clustering has far-reaching effects on semantic analysis, showing\nits importance for multiple applications such as corpus summarization and\ninformation retrieval. However, it inevitably encounters the severe sparsity of\nshort text representations, making the previous clustering approaches still far\nfrom satisfactory. In this paper, we present a novel attentive representation\nlearning model for shot text clustering, wherein cluster-level attention is\nproposed to capture the correlations between text representations and cluster\nrepresentations. Relying on this, the representation learning and clustering\nfor short texts are seamlessly integrated into a unified model. To further\nensure robust model training for short texts, we apply adversarial training to\nthe unsupervised clustering setting, by injecting perturbations into the\ncluster representations. The model parameters and perturbations are optimized\nalternately through a minimax game. Extensive experiments on four real-world\nshort text datasets demonstrate the superiority of the proposed model over\nseveral strong competitors, verifying that robust adversarial training yields\nsubstantial performance gains.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.03884v1",
            "date": "2019-12-09 07:44:32+00:00",
            "title": "MITAS: A Compressed Time-Domain Audio Separation Network with Parameter Sharing",
            "authors": [
                "Chao-I Tuan",
                "Yuan-Kuei Wu",
                "Hung-yi Lee",
                "Yu Tsao"
            ],
            "abstract": "Deep learning methods have brought substantial advancements in speech\nseparation (SS). Nevertheless, it remains challenging to deploy\ndeep-learning-based models on edge devices. Thus, identifying an effective way\nto compress these large models without hurting SS performance has become an\nimportant research topic. Recently, TasNet and Conv-TasNet have been proposed.\nThey achieved state-of-the-art results on several standardized SS tasks.\nMoreover, their low latency natures make them definitely suitable for real-time\non-device applications. In this study, we propose two parameter-sharing schemes\nto lower the memory consumption on TasNet and Conv-TasNet. Accordingly, we\nderive a novel so-called MiTAS (Mini TasNet). Our experimental results first\nconfirmed the robustness of our MiTAS on two types of perturbations in mixed\naudio. We also designed a series of ablation experiments to analyze the\nrelation between SS performance and the amount of parameters in the model. The\nresults show that MiTAS is able to reduce the model size by a factor of four\nwhile maintaining comparable SS performance with improved stability as compared\nto TasNet and Conv-TasNet. This suggests that MiTAS is more suitable for\nreal-time low latency applications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.04471v1",
            "date": "2019-12-10 03:23:05+00:00",
            "title": "Duet at TREC 2019 Deep Learning Track",
            "authors": [
                "Bhaskar Mitra",
                "Nick Craswell"
            ],
            "abstract": "This report discusses three submissions based on the Duet architecture to the\nDeep Learning track at TREC 2019. For the document retrieval task, we adapt the\nDuet model to ingest a \"multiple field\" view of documents---we refer to the new\narchitecture as Duet with Multiple Fields (DuetMF). A second submission\ncombines the DuetMF model with other neural and traditional relevance\nestimators in a learning-to-rank framework and achieves improved performance\nover the DuetMF baseline. For the passage retrieval task, we submit a single\nrun based on an ensemble of eight Duet models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.04639v1",
            "date": "2019-12-10 10:59:47+00:00",
            "title": "Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset",
            "authors": [
                "Gustavo Penha",
                "Alexandru Balan",
                "Claudia Hauff"
            ],
            "abstract": "Conversational search is an approach to information retrieval (IR), where\nusers engage in a dialogue with an agent in order to satisfy their information\nneeds. Previous conceptual work described properties and actions a good agent\nshould exhibit. Unlike them, we present a novel conceptual model defined in\nterms of conversational goals, which enables us to reason about current\nresearch practices in conversational search. Based on the literature, we elicit\nhow existing tasks and test collections from the fields of IR, natural language\nprocessing (NLP) and dialogue systems (DS) fit into this model. We describe a\nset of characteristics that an ideal conversational search dataset should have.\nLastly, we introduce MANtIS (the code and dataset are available at\nhttps://guzpenha.github.io/MANtIS/), a large-scale dataset containing\nmulti-domain and grounded information seeking dialogues that fulfill all of our\ndataset desiderata. We provide baseline results for the conversation response\nranking and user intent prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04784v1",
            "date": "2019-12-10 15:53:59+00:00",
            "title": "A Novel Topology for End-to-end Temporal Classification and Segmentation with Recurrent Neural Network",
            "authors": [
                "Taiyang Zhao"
            ],
            "abstract": "Connectionist temporal classification (CTC) has matured as an alignment free\nto sequence transduction and shows competitive for end-to-end speech\nrecognition. In the CTC topology, the blank symbol occupies more than half of\nthe state trellis, which results the spike phenomenon of the non-blank symbols.\nFor classification task, the spikes work quite well, but as to the segmentation\ntask it does not provide boundaries information. In this paper, a novel\ntopology is introduced to combine the temporal classification and segmentation\nability in one framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.04853v2",
            "date": "2019-12-10 17:46:43+00:00",
            "title": "Embedding Comparator: Visualizing Differences in Global Structure and Local Neighborhoods via Small Multiples",
            "authors": [
                "Angie Boggust",
                "Brandon Carter",
                "Arvind Satyanarayan"
            ],
            "abstract": "Embeddings mapping high-dimensional discrete input to lower-dimensional\ncontinuous vector spaces have been widely adopted in machine learning\napplications as a way to capture domain semantics. Interviewing 13 embedding\nusers across disciplines, we find comparing embeddings is a key task for\ndeployment or downstream analysis but unfolds in a tedious fashion that poorly\nsupports systematic exploration. In response, we present the Embedding\nComparator, an interactive system that presents a global comparison of\nembedding spaces alongside fine-grained inspection of local neighborhoods. It\nsystematically surfaces points of comparison by computing the similarity of the\n$k$-nearest neighbors of every embedded object between a pair of spaces.\nThrough case studies, we demonstrate our system rapidly reveals insights, such\nas semantic changes following fine-tuning, language changes over time, and\ndifferences between seemingly similar models. In evaluations with 15\nparticipants, we find our system accelerates comparisons by shifting from\nlaborious manual specification to browsing and manipulating visualizations.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1912.04961v3",
            "date": "2019-12-10 20:18:39+00:00",
            "title": "Medication Regimen Extraction From Medical Conversations",
            "authors": [
                "Sai P. Selvaraj",
                "Sandeep Konam"
            ],
            "abstract": "Extracting relevant information from medical conversations and providing it\nto doctors and patients might help in addressing doctor burnout and patient\nforgetfulness. In this paper, we focus on extracting the Medication Regimen\n(dosage and frequency for medications) discussed in a medical conversation. We\nframe the problem as a Question Answering (QA) task and perform comparative\nanalysis over: a QA approach, a new combined QA and Information Extraction\napproach, and other baselines. We use a small corpus of 6,692 annotated\ndoctor-patient conversations for the task. Clinical conversation corpora are\ncostly to create, difficult to handle (because of data privacy concerns), and\nthus scarce. We address this data scarcity challenge through data augmentation\nmethods, using publicly available embeddings and pretrain part of the network\non a related task (summarization) to improve the model's performance. Compared\nto the baseline, our best-performing models improve the dosage and frequency\nextractions' ROUGE-1 F1 scores from 54.28 and 37.13 to 89.57 and 45.94,\nrespectively. Using our best-performing model, we present the first fully\nautomated system that can extract Medication Regimen tags from spontaneous\ndoctor-patient conversations with about $\\approx$71% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05066v1",
            "date": "2019-12-11 00:30:24+00:00",
            "title": "Event Outcome Prediction using Sentiment Analysis and Crowd Wisdom in Microblog Feeds",
            "authors": [
                "Rahul Radhakrishnan Iyer",
                "Ronghuo Zheng",
                "Yuezhang Li",
                "Katia Sycara"
            ],
            "abstract": "Sentiment Analysis of microblog feeds has attracted considerable interest in\nrecent times. Most of the current work focuses on tweet sentiment\nclassification. But not much work has been done to explore how reliable the\nopinions of the mass (crowd wisdom) in social network microblogs such as\ntwitter are in predicting outcomes of certain events such as election debates.\nIn this work, we investigate whether crowd wisdom is useful in predicting such\noutcomes and whether their opinions are influenced by the experts in the field.\nWe work in the domain of multi-label classification to perform sentiment\nclassification of tweets and obtain the opinion of the crowd. This learnt\nsentiment is then used to predict outcomes of events such as: US Presidential\nDebate winners, Grammy Award winners, Super Bowl Winners. We find that in most\nof the cases, the wisdom of the crowd does indeed match with that of the\nexperts, and in cases where they don't (particularly in the case of debates),\nwe see that the crowd's opinion is actually influenced by that of the experts.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.05124v1",
            "date": "2019-12-11 05:44:04+00:00",
            "title": "Small-footprint Keyword Spotting with Graph Convolutional Network",
            "authors": [
                "Xi Chen",
                "Shouyi Yin",
                "Dandan Song",
                "Peng Ouyang",
                "Leibo Liu",
                "Shaojun Wei"
            ],
            "abstract": "Despite the recent successes of deep neural networks, it remains challenging\nto achieve high precision keyword spotting task (KWS) on resource-constrained\ndevices. In this study, we propose a novel context-aware and compact\narchitecture for keyword spotting task. Based on residual connection and\nbottleneck structure, we design a compact and efficient network for KWS task.\nTo leverage the long range dependencies and global context of the convolutional\nfeature maps, the graph convolutional network is introduced to encode the\nnon-local relations. By evaluated on the Google Speech Command Dataset, the\nproposed method achieves state-of-the-art performance and outperforms the prior\nworks by a large margin with lower computational cost.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.05156v1",
            "date": "2019-12-11 07:56:31+00:00",
            "title": "Lifelong learning for text retrieval and recognition in historical handwritten document collections",
            "authors": [
                "Lambert Schomaker"
            ],
            "abstract": "This chapter provides an overview of the problems that need to be dealt with\nwhen constructing a lifelong-learning retrieval, recognition and indexing\nengine for large historical document collections in multiple scripts and\nlanguages, the Monk system. This application is highly variable over time,\nsince the continuous labeling by end users changes the concept of what a\n'ground truth' constitutes. Although current advances in deep learning provide\na huge potential in this application domain, the scale of the problem, i.e.,\nmore than 520 hugely diverse books, documents and manuscripts precludes the\ncurrent meticulous and painstaking human effort which is required in designing\nand developing successful deep-learning systems. The ball-park principle is\nintroduced, which describes the evolution from the sparsely-labeled stage that\ncan only be addressed by traditional methods or nearest-neighbor methods on\nembedded vectors of pre-trained neural networks, up to the other end of the\nspectrum where massive labeling allows reliable training of deep-learning\nmethods. Contents: Introduction, Expectation management, Deep learning, The\nball-park principle, Technical realization, Work flow, Quality and quantity of\nmaterial, Industrialization and scalability, Human effort, Algorithms, Object\nof recognition, Processing pipeline, Performance,Compositionality, Conclusion.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.05238v1",
            "date": "2019-12-11 11:27:06+00:00",
            "title": "BERT has a Moral Compass: Improvements of ethical and moral values of machines",
            "authors": [
                "Patrick Schramowski",
                "Cigdem Turan",
                "Sophie Jentzsch",
                "Constantin Rothkopf",
                "Kristian Kersting"
            ],
            "abstract": "Allowing machines to choose whether to kill humans would be devastating for\nworld peace and security. But how do we equip machines with the ability to\nlearn ethical or even moral choices? Jentzsch et al.(2019) showed that applying\nmachine learning to human texts can extract deontological ethical reasoning\nabout \"right\" and \"wrong\" conduct by calculating a moral bias score on a\nsentence level using sentence embeddings. The machine learned that it is\nobjectionable to kill living beings, but it is fine to kill time; It is\nessential to eat, yet one might not eat dirt; it is important to spread\ninformation, yet one should not spread misinformation. However, the evaluated\nmoral bias was restricted to simple actions -- one verb -- and a ranking of\nactions with surrounding context. Recently BERT ---and variants such as RoBERTa\nand SBERT--- has set a new state-of-the-art performance for a wide range of NLP\ntasks. But has BERT also a better moral compass? In this paper, we discuss and\nshow that this is indeed the case. Thus, recent improvements of language\nrepresentations also improve the representation of the underlying ethical and\nmoral values of the machine. We argue that through an advanced semantic\nrepresentation of text, BERT allows one to get better insights of moral and\nethical values implicitly represented in text. This enables the Moral Choice\nMachine (MCM) to extract more accurate imprints of moral choices and ethical\nvalues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.05289v2",
            "date": "2019-12-11 13:34:43+00:00",
            "title": "Voice Conversion for Whispered Speech Synthesis",
            "authors": [
                "Marius Cotescu",
                "Thomas Drugman",
                "Goeric Huybrechts",
                "Jaime Lorenzo-Trueba",
                "Alexis Moinet"
            ],
            "abstract": "We present an approach to synthesize whisper by applying a handcrafted signal\nprocessing recipe and Voice Conversion (VC) techniques to convert normally\nphonated speech to whispered speech. We investigate using Gaussian Mixture\nModels (GMM) and Deep Neural Networks (DNN) to model the mapping between\nacoustic features of normal speech and those of whispered speech. We evaluate\nnaturalness and speaker similarity of the converted whisper on an internal\ncorpus and on the publicly available wTIMIT corpus. We show that applying VC\ntechniques is significantly better than using rule-based signal processing\nmethods and it achieves results that are indistinguishable from copy-synthesis\nof natural whisper recordings. We investigate the ability of the DNN model to\ngeneralize on unseen speakers, when trained with data from multiple speakers.\nWe show that excluding the target speaker from the training set has little or\nno impact on the perceived naturalness and speaker similarity of the converted\nwhisper. The proposed DNN method is used in the newly released Whisper Mode of\nAmazon Alexa.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.05308v1",
            "date": "2019-12-10 16:08:26+00:00",
            "title": "Unsupervised Transfer Learning via BERT Neuron Selection",
            "authors": [
                "Mehrdad Valipour",
                "En-Shiun Annie Lee",
                "Jaime R. Jamacaro",
                "Carolina Bessega"
            ],
            "abstract": "Recent advancements in language representation models such as BERT have led\nto a rapid improvement in numerous natural language processing tasks. However,\nlanguage models usually consist of a few hundred million trainable parameters\nwith embedding space distributed across multiple layers, thus making them\nchallenging to be fine-tuned for a specific task or to be transferred to a new\ndomain. To determine whether there are task-specific neurons that can be\nexploited for unsupervised transfer learning, we introduce a method for\nselecting the most important neurons to solve a specific classification task.\nThis algorithm is further extended to multi-source transfer learning by\ncomputing the importance of neurons for several single-source transfer learning\nscenarios between different subsets of data sources. Besides, a task-specific\nfingerprint for each data source is obtained based on the percentage of the\nselected neurons in each layer. We perform extensive experiments in\nunsupervised transfer learning for sentiment analysis, natural language\ninference and sentence similarity, and compare our results with the existing\nliterature and baselines. Significantly, we found that the source and target\ndata sources with higher degrees of similarity between their task-specific\nfingerprints demonstrate a better transferability property. We conclude that\nour method can lead to better performance using just a few hundred\ntask-specific and interpretable neurons.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.05525v1",
            "date": "2019-12-11 18:48:05+00:00",
            "title": "Learning to Request Guidance in Emergent Communication",
            "authors": [
                "Benjamin Kolb",
                "Leon Lang",
                "Henning Bartsch",
                "Arwin Gansekoele",
                "Raymond Koopmanschap",
                "Leonardo Romor",
                "David Speck",
                "Mathijs Mul",
                "Elia Bruni"
            ],
            "abstract": "Previous research into agent communication has shown that a pre-trained guide\ncan speed up the learning process of an imitation learning agent. The guide\nachieves this by providing the agent with discrete messages in an emerged\nlanguage about how to solve the task. We extend this one-directional\ncommunication by a one-bit communication channel from the learner back to the\nguide: It is able to ask the guide for help, and we limit the guidance by\npenalizing the learner for these requests. During training, the agent learns to\ncontrol this gate based on its current observation. We find that the amount of\nrequested guidance decreases over time and guidance is requested in situations\nof high uncertainty. We investigate the agent's performance in cases of open\nand closed gates and discuss potential motives for the observed gating\nbehavior.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.05533v1",
            "date": "2019-12-11 18:58:58+00:00",
            "title": "SpecAugment on Large Scale Datasets",
            "authors": [
                "Daniel S. Park",
                "Yu Zhang",
                "Chung-Cheng Chiu",
                "Youzheng Chen",
                "Bo Li",
                "William Chan",
                "Quoc V. Le",
                "Yonghui Wu"
            ],
            "abstract": "Recently, SpecAugment, an augmentation scheme for automatic speech\nrecognition that acts directly on the spectrogram of input utterances, has\nshown to be highly effective in enhancing the performance of end-to-end\nnetworks on public datasets. In this paper, we demonstrate its effectiveness on\ntasks with large scale datasets by investigating its application to the Google\nMultidomain Dataset (Narayanan et al., 2018). We achieve improvement across all\ntest domains by mixing raw training data augmented with SpecAugment and\nnoise-perturbed training data when training the acoustic model. We also\nintroduce a modification of SpecAugment that adapts the time mask size and/or\nmultiplicity depending on the length of the utterance, which can potentially\nbenefit large scale tasks. By using adaptive masking, we are able to further\nimprove the performance of the Listen, Attend and Spell model on LibriSpeech to\n2.2% WER on test-clean and 5.2% WER on test-other.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.05676v1",
            "date": "2019-12-11 22:39:51+00:00",
            "title": "Biases for Emergent Communication in Multi-agent Reinforcement Learning",
            "authors": [
                "Tom Eccles",
                "Yoram Bachrach",
                "Guy Lever",
                "Angeliki Lazaridou",
                "Thore Graepel"
            ],
            "abstract": "We study the problem of emergent communication, in which language arises\nbecause speakers and listeners must communicate information in order to solve\ntasks. In temporally extended reinforcement learning domains, it has proved\nhard to learn such communication without centralized training of agents, due in\npart to a difficult joint exploration problem. We introduce inductive biases\nfor positive signalling and positive listening, which ease this problem. In a\nsimple one-step environment, we demonstrate how these biases ease the learning\nproblem. We also apply our methods to a more extended environment, showing that\nagents with these inductive biases achieve better performance, and analyse the\nresulting communication protocols.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/1912.05846v1",
            "date": "2019-12-12 09:30:02+00:00",
            "title": "The Benefits of Close-Domain Fine-Tuning for Table Detection in Document Images",
            "authors": [
                "\u00c1ngela Casado-Garc\u00eda",
                "C\u00e9sar Dom\u00ednguez",
                "J\u00f3nathan Heras",
                "Eloy Mata",
                "Vico Pascual"
            ],
            "abstract": "A correct localisation of tables in a document is instrumental for\ndetermining their structure and extracting their contents; therefore, table\ndetection is a key step in table understanding. Nowadays, the most successful\nmethods for table detection in document images employ deep learning algorithms;\nand, particularly, a technique known as fine-tuning. In this context, such a\ntechnique exports the knowledge acquired to detect objects in natural images to\ndetect tables in document images. However, there is only a vague relation\nbetween natural and document images, and fine-tuning works better when there is\na close relation between the source and target task. In this paper, we show\nthat it is more beneficial to employ fine-tuning from a closer domain. To this\naim, we train different object detection algorithms (namely, Mask R-CNN,\nRetinaNet, SSD and YOLO) using the TableBank dataset (a dataset of images of\nacademic documents designed for table detection and recognition), and fine-tune\nthem for several heterogeneous table detection datasets. Using this approach,\nwe considerably improve the accuracy of the detection models fine-tuned from\nnatural images (in mean a 17%, and, in the best case, up to a 60%).",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.05881v2",
            "date": "2019-12-12 11:17:30+00:00",
            "title": "Singing Synthesis: with a little help from my attention",
            "authors": [
                "Orazio Angelini",
                "Alexis Moinet",
                "Kayoko Yanagisawa",
                "Thomas Drugman"
            ],
            "abstract": "We present UTACO, a singing synthesis model based on an attention-based\nsequence-to-sequence mechanism and a vocoder based on dilated causal\nconvolutions. These two classes of models have significantly affected the field\nof text-to-speech, but have never been thoroughly applied to the task of\nsinging synthesis. UTACO demonstrates that attention can be successfully\napplied to the singing synthesis field and improves naturalness over the state\nof the art. The system requires considerably less explicit modelling of voice\nfeatures such as F0 patterns, vibratos, and note and phoneme durations, than\nprevious models in the literature. Despite this, it shows a strong improvement\nin naturalness with respect to previous neural singing synthesis models. The\nmodel does not require any durations or pitch patterns as inputs, and learns to\ninsert vibrato autonomously according to the musical context. However, we\nobserve that, by completely dispensing with any explicit duration modelling it\nbecomes harder to obtain the fine control of timing needed to exactly match the\ntempo of a song.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.06174v1",
            "date": "2019-12-12 19:32:41+00:00",
            "title": "Training without training data: Improving the generalizability of automated medical abbreviation disambiguation",
            "authors": [
                "Marta Skreta",
                "Aryan Arbabi",
                "Jixuan Wang",
                "Michael Brudno"
            ],
            "abstract": "Abbreviation disambiguation is important for automated clinical note\nprocessing due to the frequent use of abbreviations in clinical settings.\nCurrent models for automated abbreviation disambiguation are restricted by the\nscarcity and imbalance of labeled training data, decreasing their\ngeneralizability to orthogonal sources. In this work we propose a novel data\naugmentation technique that utilizes information from related medical concepts,\nwhich improves our model's ability to generalize. Furthermore, we show that\nincorporating the global context information within the whole medical note (in\naddition to the traditional local context window), can significantly improve\nthe model's representation for abbreviations. We train our model on a public\ndataset (MIMIC III) and test its performance on datasets from different sources\n(CASI, i2b2). Together, these two techniques boost the accuracy of abbreviation\ndisambiguation by almost 14% on the CASI dataset and 4% on i2b2.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.06203v2",
            "date": "2019-12-12 20:48:52+00:00",
            "title": "ManiGAN: Text-Guided Image Manipulation",
            "authors": [
                "Bowen Li",
                "Xiaojuan Qi",
                "Thomas Lukasiewicz",
                "Philip H. S. Torr"
            ],
            "abstract": "The goal of our paper is to semantically edit parts of an image matching a\ngiven text that describes desired attributes (e.g., texture, colour, and\nbackground), while preserving other contents that are irrelevant to the text.\nTo achieve this, we propose a novel generative adversarial network (ManiGAN),\nwhich contains two key components: text-image affine combination module (ACM)\nand detail correction module (DCM). The ACM selects image regions relevant to\nthe given text and then correlates the regions with corresponding semantic\nwords for effective manipulation. Meanwhile, it encodes original image features\nto help reconstruct text-irrelevant contents. The DCM rectifies mismatched\nattributes and completes missing contents of the synthetic image. Finally, we\nsuggest a new metric for evaluating image manipulation results, in terms of\nboth the generation of new attributes and the reconstruction of text-irrelevant\ncontents. Extensive experiments on the CUB and COCO datasets demonstrate the\nsuperior performance of the proposed method. Code is available at\nhttps://github.com/mrlibw/ManiGAN.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.06262v2",
            "date": "2019-12-12 23:18:16+00:00",
            "title": "Extracting clinical concepts from user queries",
            "authors": [
                "Yue Zhao",
                "John Handley"
            ],
            "abstract": "Clinical concept extraction often begins with clinical Named Entity\nRecognition (NER). Often trained on annotated clinical notes, clinical NER\nmodels tend to struggle with tagging clinical entities in user queries because\nof the structural differences between clinical notes and user queries. User\nqueries, unlike clinical notes, are often ungrammatical and incoherent. In many\ncases, user queries are compounded of multiple clinical entities, without comma\nor conjunction words separating them. By using as dataset a mixture of\nannotated clinical notes and synthesized user queries, we adapt a clinical NER\nmodel based on the BiLSTM-CRF architecture for tagging clinical entities in\nuser queries. Our contribution are the following: 1) We found that when trained\non a mixture of synthesized user queries and clinical notes, the NER model\nperforms better on both user queries and clinical notes. 2) We provide an\nend-to-end and easy-to-implement framework for clinical concept extraction from\nuser queries.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.06311v3",
            "date": "2019-12-13 03:38:14+00:00",
            "title": "Short-duration Speaker Verification (SdSV) Challenge 2021: the Challenge Evaluation Plan",
            "authors": [
                "Hossein Zeinali",
                "Kong Aik Lee",
                "Jahangir Alam",
                "Lukas Burget"
            ],
            "abstract": "This document describes the Short-duration Speaker Verification (SdSV)\nChallenge 2021. The main goal of the challenge is to evaluate new technologies\nfor text-dependent (TD) and text-independent (TI) speaker verification (SV) in\na short duration scenario. The proposed challenge evaluates SdSV with varying\ndegree of phonetic overlap between the enrollment and test utterances\n(cross-lingual). It is the first challenge with a broad focus on systematic\nbenchmark and analysis on varying degrees of phonetic variability on\nshort-duration speaker recognition. We expect that modern methods (deep neural\nnetworks in particular) will play a key role.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.06728v1",
            "date": "2019-12-13 22:06:59+00:00",
            "title": "Associating Natural Language Comment and Source Code Entities",
            "authors": [
                "Sheena Panthaplackel",
                "Milos Gligoric",
                "Raymond J. Mooney",
                "Junyi Jessy Li"
            ],
            "abstract": "Comments are an integral part of software development; they are natural\nlanguage descriptions associated with source code elements. Understanding\nexplicit associations can be useful in improving code comprehensibility and\nmaintaining the consistency between code and comments. As an initial step\ntowards this larger goal, we address the task of associating entities in\nJavadoc comments with elements in Java source code. We propose an approach for\nautomatically extracting supervised data using revision histories of open\nsource projects and present a manually annotated evaluation dataset for this\ntask. We develop a binary classifier and a sequence labeling model by crafting\na rich feature set which encompasses various aspects of code, comments, and the\nrelationships between them. Experiments show that our systems outperform\nseveral baselines learning from the proposed supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06745v1",
            "date": "2019-12-13 23:32:38+00:00",
            "title": "An Unsupervised Domain-Independent Framework for Automated Detection of Persuasion Tactics in Text",
            "authors": [
                "Rahul Radhakrishnan Iyer",
                "Katia Sycara"
            ],
            "abstract": "With the increasing growth of social media, people have started relying\nheavily on the information shared therein to form opinions and make decisions.\nWhile such a reliance is motivation for a variety of parties to promote\ninformation, it also makes people vulnerable to exploitation by slander,\nmisinformation, terroristic and predatorial advances. In this work, we aim to\nunderstand and detect such attempts at persuasion. Existing works on detecting\npersuasion in text make use of lexical features for detecting persuasive\ntactics, without taking advantage of the possible structures inherent in the\ntactics used. We formulate the task as a multi-class classification problem and\npropose an unsupervised, domain-independent machine learning framework for\ndetecting the type of persuasion used in text, which exploits the inherent\nsentence structure present in the different persuasion tactics. Our work shows\npromising results as compared to existing work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06806v1",
            "date": "2019-12-14 08:44:18+00:00",
            "title": "SemEval-2013 Task 2: Sentiment Analysis in Twitter",
            "authors": [
                "Preslav Nakov",
                "Zornitsa Kozareva",
                "Alan Ritter",
                "Sara Rosenthal",
                "Veselin Stoyanov",
                "Theresa Wilson"
            ],
            "abstract": "In recent years, sentiment analysis in social media has attracted a lot of\nresearch interest and has been used for a number of applications.\nUnfortunately, research has been hindered by the lack of suitable datasets,\ncomplicating the comparison between approaches. To address this issue, we have\nproposed SemEval-2013 Task 2: Sentiment Analysis in Twitter, which included two\nsubtasks: A, an expression-level subtask, and B, a message-level subtask. We\nused crowdsourcing on Amazon Mechanical Turk to label a large Twitter training\ndataset along with additional test sets of Twitter and SMS messages for both\nsubtasks. All datasets used in the evaluation are released to the research\ncommunity. The task attracted significant interest and a total of 149\nsubmissions from 44 teams. The best-performing team achieved an F1 of 88.9% and\n69% for subtasks A and B, respectively.",
            "journal ref": "SemEval-2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06810v1",
            "date": "2019-12-14 08:58:01+00:00",
            "title": "Proppy: A System to Unmask Propaganda in Online News",
            "authors": [
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Giovanni Da San Martino",
                "Israa Jaradat",
                "Preslav Nakov"
            ],
            "abstract": "We present proppy, the first publicly available real-world, real-time\npropaganda detection system for online news, which aims at raising awareness,\nthus potentially limiting the impact of propaganda and helping fight\ndisinformation. The system constantly monitors a number of news sources,\ndeduplicates and clusters the news into events, and organizes the articles\nabout an event on the basis of the likelihood that they contain propagandistic\ncontent. The system is trained on known propaganda sources using a variety of\nstylistic features. The evaluation results on a standard dataset show\nstate-of-the-art results for propaganda detection.",
            "journal ref": "Thirty-Third AAAI Conference on Artificial Intelligence\n  (AAAI-2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.06813v1",
            "date": "2019-12-14 09:30:52+00:00",
            "title": "Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining",
            "authors": [
                "Wen-Chin Huang",
                "Tomoki Hayashi",
                "Yi-Chiao Wu",
                "Hirokazu Kameoka",
                "Tomoki Toda"
            ],
            "abstract": "We introduce a novel sequence-to-sequence (seq2seq) voice conversion (VC)\nmodel based on the Transformer architecture with text-to-speech (TTS)\npretraining. Seq2seq VC models are attractive owing to their ability to convert\nprosody. While seq2seq models based on recurrent neural networks (RNNs) and\nconvolutional neural networks (CNNs) have been successfully applied to VC, the\nuse of the Transformer network, which has shown promising results in various\nspeech processing tasks, has not yet been investigated. Nonetheless, their\ndata-hungry property and the mispronunciation of converted speech make seq2seq\nmodels far from practical. To this end, we propose a simple yet effective\npretraining technique to transfer knowledge from learned TTS models, which\nbenefit from large-scale, easily accessible TTS corpora. VC models initialized\nwith such pretrained model parameters are able to generate effective hidden\nrepresentations for high-fidelity, highly intelligible converted speech.\nExperimental results show that such a pretraining scheme can facilitate\ndata-efficient training and outperform an RNN-based seq2seq VC model in terms\nof intelligibility, naturalness, and similarity.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.06979v1",
            "date": "2019-12-15 05:34:45+00:00",
            "title": "Breaking Speech Recognizers to Imagine Lyrics",
            "authors": [
                "Jon Gillick",
                "David Bamman"
            ],
            "abstract": "We introduce a new method for generating text, and in particular song lyrics,\nbased on the speech-like acoustic qualities of a given audio file. We repurpose\na vocal source separation algorithm and an acoustic model trained to recognize\nisolated speech, instead inputting instrumental music or environmental sounds.\nFeeding the \"mistakes\" of the vocal separator into the recognizer, we obtain a\ntranscription of words \\emph{imagined} to be spoken in the input audio. We\ndescribe the key components of our approach, present initial analysis, and\ndiscuss the potential of the method for machine-in-the-loop collaboration in\ncreative applications.",
            "journal ref": "NeurIPS 2019 Workshop on Machine Learning for Creativity and\n  Design",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1912.07050v1",
            "date": "2019-12-15 14:38:58+00:00",
            "title": "Computational Induction of Prosodic Structure",
            "authors": [
                "Dafydd Gibbon"
            ],
            "abstract": "The present study has two goals relating to the grammar of prosody,\nunderstood as the rhythms and melodies of speech. First, an overview is\nprovided of the computable grammatical and phonetic approaches to prosody\nanalysis which use hypothetico-deductive methods and are based on learned\nhermeneutic intuitions about language. Second, a proposal is presented for an\ninductive grounding in the physical signal, in which prosodic structure is\ninferred using a language-independent method from the low-frequency spectrum of\nthe speech signal. The overview includes a discussion of computational aspects\nof standard generative and post-generative models, and suggestions for\nreformulating these to form inductive approaches. Also included is a discussion\nof linguistic phonetic approaches to analysis of annotations (pairs of speech\nunit labels with time-stamps) of recorded spoken utterances. The proposal\nintroduces the inductive approach of Rhythm Formant Theory (RFT) and the\nassociated Rhythm Formant Analysis (RFA) method are introduced, with the aim of\ncompleting a gap in the linguistic hypothetico-deductive cycle by grounding in\na language-independent inductive procedure of speech signal analysis. The\nvalidity of the method is demonstrated and applied to rhythm patterns in\nread-aloud Mandarin Chinese, finding differences from English which are related\nto lexical and grammatical differences between the languages, as well as\nindividual variation. The overall conclusions are (1) that normative\nlanguage-to-language phonological or phonetic comparisons of rhythm, for\nexample of Mandarin and English, are too simplistic, in view of diverse\nlanguage-internal factors due to genre and style differences as well as\nutterance dynamics, and (2) that language-independent empirical grounding of\nrhythm in the physical signal is called for.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07421v1",
            "date": "2019-12-16 14:45:56+00:00",
            "title": "Semantic Similarity To Improve Question Understanding in a Virtual Patient",
            "authors": [
                "Fr\u00e9jus A. A. Laleye",
                "Antonia Blani\u00e9",
                "Antoine Brouquet",
                "Dan Behnamou",
                "Ga\u00ebl de Chalendar"
            ],
            "abstract": "In medicine, a communicating virtual patient or doctor allows students to\ntrain in medical diagnosis and develop skills to conduct a medical\nconsultation. In this paper, we describe a conversational virtual standardized\npatient system to allow medical students to simulate a diagnosis strategy of an\nabdominal surgical emergency. We exploited the semantic properties captured by\ndistributed word representations to search for similar questions in the virtual\npatient dialogue system. We created two dialogue systems that were evaluated on\ndatasets collected during tests with students. The first system based on\nhand-crafted rules obtains $92.29\\%$ as $F1$-score on the studied clinical case\nwhile the second system that combines rules and semantic similarity achieves\n$94.88\\%$. It represents an error reduction of $9.70\\%$ as compared to the\nrules-only-based system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07478v1",
            "date": "2019-12-16 16:21:13+00:00",
            "title": "Image Manipulation with Natural Language using Two-sidedAttentive Conditional Generative Adversarial Network",
            "authors": [
                "Dawei Zhu",
                "Aditya Mogadala",
                "Dietrich Klakow"
            ],
            "abstract": "Altering the content of an image with photo editing tools is a tedious task\nfor an inexperienced user. Especially, when modifying the visual attributes of\na specific object in an image without affecting other constituents such as\nbackground etc. To simplify the process of image manipulation and to provide\nmore control to users, it is better to utilize a simpler interface like natural\nlanguage. Therefore, in this paper, we address the challenge of manipulating\nimages using natural language description. We propose the Two-sidEd Attentive\nconditional Generative Adversarial Network (TEA-cGAN) to generate semantically\nmanipulated images while preserving other contents such as background intact.\nTEA-cGAN uses fine-grained attention both in the generator and discriminator of\nGenerative Adversarial Network (GAN) based framework at different scales.\nExperimental results show that TEA-cGAN which generates 128x128 and 256x256\nresolution images outperforms existing methods on CUB and Oxford-102 datasets\nboth quantitatively and qualitatively.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.07538v3",
            "date": "2019-12-16 17:45:01+00:00",
            "title": "Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing",
            "authors": [
                "Vedika Agarwal",
                "Rakshith Shetty",
                "Mario Fritz"
            ],
            "abstract": "Despite significant success in Visual Question Answering (VQA), VQA models\nhave been shown to be notoriously brittle to linguistic variations in the\nquestions. Due to deficiencies in models and datasets, today's models often\nrely on correlations rather than predictions that are causal w.r.t. data. In\nthis paper, we propose a novel way to analyze and measure the robustness of the\nstate of the art models w.r.t semantic visual variations as well as propose\nways to make models more robust against spurious correlations. Our method\nperforms automated semantic image manipulations and tests for consistency in\nmodel predictions to quantify the model robustness as well as generate\nsynthetic data to counter these problems. We perform our analysis on three\ndiverse, state of the art VQA models and diverse question types with a\nparticular focus on challenging counting questions. In addition, we show that\nmodels can be made significantly more robust against inconsistent predictions\nusing our edited data. Finally, we show that results also translate to\nreal-world error cases of state of the art models, which results in improved\noverall performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.07747v1",
            "date": "2019-12-16 23:04:03+00:00",
            "title": "Pipelines for Procedural Information Extraction from Scientific Literature: Towards Recipes using Machine Learning and Data Science",
            "authors": [
                "Huichen Yang",
                "Carlos A. Aguirre",
                "Maria F. De La Torre",
                "Derek Christensen",
                "Luis Bobadilla",
                "Emily Davich",
                "Jordan Roth",
                "Lei Luo",
                "Yihong Theis",
                "Alice Lam",
                "T. Yong-Jin Han",
                "David Buttler",
                "William H. Hsu"
            ],
            "abstract": "This paper describes a machine learning and data science pipeline for\nstructured information extraction from documents, implemented as a suite of\nopen-source tools and extensions to existing tools. It centers around a\nmethodology for extracting procedural information in the form of recipes,\nstepwise procedures for creating an artifact (in this case synthesizing a\nnanomaterial), from published scientific literature. From our overall goal of\nproducing recipes from free text, we derive the technical objectives of a\nsystem consisting of pipeline stages: document acquisition and filtering,\npayload extraction, recipe step extraction as a relationship extraction task,\nrecipe assembly, and presentation through an information retrieval interface\nwith question answering (QA) functionality. This system meets computational\ninformation and knowledge management (CIKM) requirements of metadata-driven\npayload extraction, named entity extraction, and relationship extraction from\ntext. Functional contributions described in this paper include semi-supervised\nmachine learning methods for PDF filtering and payload extraction tasks,\nfollowed by structured extraction and data transformation tasks beginning with\nsection extraction, recipe steps as information tuples, and finally assembled\nrecipes. Measurable objective criteria for extraction quality include precision\nand recall of recipe steps, ordering constraints, and QA accuracy, precision,\nand recall. Results, key novel contributions, and significant open problems\nderived from this work center around the attribution of these holistic quality\nmeasures to specific machine learning and inference stages of the pipeline,\neach with their performance measures. The desired recipes contain identified\npreconditions, material inputs, and operations, and constitute the overall\noutput generated by our computational information and knowledge management\n(CIKM) system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.07840v2",
            "date": "2019-12-17 06:53:05+00:00",
            "title": "Cross-Lingual Ability of Multilingual BERT: An Empirical Study",
            "authors": [
                "Karthikeyan K",
                "Zihan Wang",
                "Stephen Mayhew",
                "Dan Roth"
            ],
            "abstract": "Recent work has exhibited the surprising cross-lingual abilities of\nmultilingual BERT (M-BERT) -- surprising since it is trained without any\ncross-lingual objective and with no aligned data. In this work, we provide a\ncomprehensive study of the contribution of different components in M-BERT to\nits cross-lingual ability. We study the impact of linguistic properties of the\nlanguages, the architecture of the model, and the learning objectives. The\nexperimental study is done in the context of three typologically different\nlanguages -- Spanish, Hindi, and Russian -- and using two conceptually\ndifferent NLP tasks, textual entailment and named entity recognition. Among our\nkey conclusions is the fact that the lexical overlap between languages plays a\nnegligible role in the cross-lingual success, while the depth of the network is\nan integral part of it. All our models and implementations can be found on our\nproject page: http://cogcomp.org/page/publication_view/900 .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07875v1",
            "date": "2019-12-17 08:47:30+00:00",
            "title": "Libri-Light: A Benchmark for ASR with Limited or No Supervision",
            "authors": [
                "Jacob Kahn",
                "Morgane Rivi\u00e8re",
                "Weiyi Zheng",
                "Evgeny Kharitonov",
                "Qiantong Xu",
                "Pierre-Emmanuel Mazar\u00e9",
                "Julien Karadayi",
                "Vitaliy Liptchinsky",
                "Ronan Collobert",
                "Christian Fuegen",
                "Tatiana Likhomanenko",
                "Gabriel Synnaeve",
                "Armand Joulin",
                "Abdelrahman Mohamed",
                "Emmanuel Dupoux"
            ],
            "abstract": "We introduce a new collection of spoken English audio suitable for training\nspeech recognition systems under limited or no supervision. It is derived from\nopen-source audio books from the LibriVox project. It contains over 60K hours\nof audio, which is, to our knowledge, the largest freely-available corpus of\nspeech. The audio has been segmented using voice activity detection and is\ntagged with SNR, speaker ID and genre descriptions. Additionally, we provide\nbaseline systems and evaluation metrics working under three settings: (1) the\nzero resource/unsupervised setting (ABX), (2) the semi-supervised setting (PER,\nCER) and (3) the distant supervision setting (WER). Settings (2) and (3) use\nlimited textual resources (10 minutes to 10 hours) aligned with the speech.\nSetting (3) uses large amounts of unaligned text. They are evaluated on the\nstandard LibriSpeech dev and test sets for comparison with the supervised\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.07915v1",
            "date": "2019-12-17 10:33:17+00:00",
            "title": "Knowledge-Enhanced Attentive Learning for Answer Selection in Community Question Answering Systems",
            "authors": [
                "Fengshi Jing",
                "Qingpeng Zhang"
            ],
            "abstract": "In the community question answering (CQA) system, the answer selection task\naims to identify the best answer for a specific question, and thus is playing a\nkey role in enhancing the service quality through recommending appropriate\nanswers for new questions. Recent advances in CQA answer selection focus on\nenhancing the performance by incorporating the community information,\nparticularly the expertise (previous answers) and authority (position in the\nsocial network) of an answerer. However, existing approaches for incorporating\nsuch information are limited in (a) only considering either the expertise or\nthe authority, but not both; (b) ignoring the domain knowledge to differentiate\ntopics of previous answers; and (c) simply using the authority information to\nadjust the similarity score, instead of fully utilizing it in the process of\nmeasuring the similarity between segments of the question and the answer. We\npropose the Knowledge-enhanced Attentive Answer Selection (KAAS) model, which\nenhances the performance through (a) considering both the expertise and the\nauthority of the answerer; (b) utilizing the human-labeled tags, the taxonomy\nof the tags, and the votes as the domain knowledge to infer the expertise of\nthe answer; (c) using matrix decomposition of the social network (formed by\nfollowing-relationship) to infer the authority of the answerer and\nincorporating such information in the process of evaluating the similarity\nbetween segments. Besides, for vertical community, we incorporate an external\nknowledge graph to capture more professional information for vertical CQA\nsystems. Then we adopt the attention mechanism to integrate the analysis of the\ntext of questions and answers and the aforementioned community information.\nExperiments with both vertical and general CQA sites demonstrate the superior\nperformance of the proposed KAAS model.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.07942v3",
            "date": "2019-12-17 11:46:08+00:00",
            "title": "Analyzing Information Leakage of Updates to Natural Language Models",
            "authors": [
                "Marc Brockschmidt",
                "Boris K\u00f6pf",
                "Olga Ohrimenko",
                "Andrew Paverd",
                "Victor R\u00fchle",
                "Shruti Tople",
                "Lukas Wutschitz",
                "Santiago Zanella-B\u00e9guelin"
            ],
            "abstract": "To continuously improve quality and reflect changes in data, machine learning\napplications have to regularly retrain and update their core models. We show\nthat a differential analysis of language model snapshots before and after an\nupdate can reveal a surprising amount of detailed information about changes in\nthe training data. We propose two new metrics---differential score and\ndifferential rank---for analyzing the leakage due to updates of natural\nlanguage models. We perform leakage analysis using these metrics across models\ntrained on several different datasets using different methods and\nconfigurations. We discuss the privacy implications of our findings, propose\nmitigation strategies and evaluate their effect.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.07943v1",
            "date": "2019-12-17 11:49:13+00:00",
            "title": "Pioneer dataset and automatic recognition of Urdu handwritten characters using a deep autoencoder and convolutional neural network",
            "authors": [
                "Hazrat Ali",
                "Ahsan Ullah",
                "Talha Iqbal",
                "Shahid Khattak"
            ],
            "abstract": "Automatic recognition of Urdu handwritten digits and characters, is a\nchallenging task. It has applications in postal address reading, bank's cheque\nprocessing, and digitization and preservation of handwritten manuscripts from\nold ages. While there exists a significant work for automatic recognition of\nhandwritten English characters and other major languages of the world, the work\ndone for Urdu lan-guage is extremely insufficient. This paper has two goals.\nFirstly, we introduce a pioneer dataset for handwritten digits and characters\nof Urdu, containing samples from more than 900 individuals. Secondly, we report\nresults for automatic recog-nition of handwritten digits and characters as\nachieved by using deep auto-encoder network and convolutional neural network.\nMore specifically, we use a two-layer and a three-layer deep autoencoder\nnetwork and convolutional neural network and evaluate the two frameworks in\nterms of recognition accuracy. The proposed framework of deep autoencoder can\nsuccessfully recognize digits and characters with an accuracy of 97% for digits\nonly, 81% for characters only and 82% for both digits and characters\nsimultaneously. In comparison, the framework of convolutional neural network\nhas accuracy of 96.7% for digits only, 86.5% for characters only and 82.7% for\nboth digits and characters simultaneously. These frameworks can serve as\nbaselines for future research on Urdu handwritten text.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.07946v3",
            "date": "2019-12-17 11:59:41+00:00",
            "title": "In Nomine Function: Naming Functions in Stripped Binaries with Neural Networks",
            "authors": [
                "Fiorella Artuso",
                "Giuseppe Antonio Di Luna",
                "Luca Massarelli",
                "Leonardo Querzoni"
            ],
            "abstract": "In this paper we investigate the problem of automatically naming pieces of\nassembly code. Where by naming we mean assigning to an assembly function a\nstring of words that would likely be assigned by a human reverse engineer. We\nformally and precisely define the framework in which our investigation takes\nplace. That is we define the problem, we provide reasonable justifications for\nthe choices that we made for the design of training and the tests. We performed\nan analysis on a large real-world corpora constituted by nearly 9 millions of\nfunctions taken from more than 22k softwares. In such framework we test\nbaselines coming from the field of Natural Language Processing (e.g., Seq2Seq\nnetworks and Transformer). Interestingly, our evaluation shows promising\nresults beating the state-of-the-art and reaching good performance. We\ninvestigate the applicability of tine-tuning (i.e., taking a model already\ntrained on a large generic corpora and retraining it for a specific task). Such\ntechnique is popular and well-known in the NLP field. Our results confirm that\nfine-tuning is effective even when neural networks are applied to binaries. We\nshow that a model, pre-trained on the aforementioned corpora, when fine-tuned\nhas higher performances on specific domains (such as predicting names in system\nutilites, malware, etc).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.08011v2",
            "date": "2019-12-17 13:45:13+00:00",
            "title": "Application of Word2vec in Phoneme Recognition",
            "authors": [
                "Xin Feng",
                "Lei Wang"
            ],
            "abstract": "In this paper, we present how to hybridize a Word2vec model and an\nattention-based end-to-end speech recognition model. We build a phoneme\nrecognition system based on Listen, Attend and Spell model. And the phoneme\nrecognition model uses a word2vec model to initialize the embedding matrix for\nthe improvement of the performance, which can increase the distance among the\nphoneme vectors. At the same time, in order to solve the problem of overfitting\nin the 61 phoneme recognition model on TIMIT dataset, we propose a new training\nmethod. A 61-39 phoneme mapping comparison table is used to inverse map the\nphonemes of the dataset to generate more 61 phoneme training data. At the end\nof training, replace the dataset with a standard dataset for corrective\ntraining. Our model can achieve the best result under the TIMIT dataset which\nis 16.5% PER (Phoneme Error Rate).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08084v1",
            "date": "2019-12-14 10:29:13+00:00",
            "title": "A Context-Aware Approach for Detecting Check-Worthy Claims in Political Debates",
            "authors": [
                "Pepa Gencheva",
                "Ivan Koychev",
                "Llu\u00eds M\u00e0rquez",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Preslav Nakov"
            ],
            "abstract": "In the context of investigative journalism, we address the problem of\nautomatically identifying which claims in a given document are most worthy and\nshould be prioritized for fact-checking. Despite its importance, this is a\nrelatively understudied problem. Thus, we create a new dataset of political\ndebates, containing statements that have been fact-checked by nine reputable\nsources, and we train machine learning models to predict which claims should be\nprioritized for fact-checking, i.e., we model the problem as a ranking task.\nUnlike previous work, which has looked primarily at sentences in isolation, in\nthis paper we focus on a rich input representation modeling the context:\nrelationship between the target statement and the larger context of the debate,\ninteraction between the opponents, and reaction by the moderator and by the\npublic. Our experiments show state-of-the-art results, outperforming a strong\nrivaling system by a margin, while also confirming the importance of the\ncontextual information.",
            "journal ref": "RANLP-2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08320v1",
            "date": "2019-12-17 23:49:19+00:00",
            "title": "Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?",
            "authors": [
                "R. Stuart Geiger",
                "Kevin Yu",
                "Yanlai Yang",
                "Mindy Dai",
                "Jie Qiu",
                "Rebekah Tang",
                "Jenny Huang"
            ],
            "abstract": "Many machine learning projects for new application areas involve teams of\nhumans who label data for a particular purpose, from hiring crowdworkers to the\npaper's authors labeling the data themselves. Such a task is quite similar to\n(or a form of) structured content analysis, which is a longstanding methodology\nin the social sciences and humanities, with many established best practices. In\nthis paper, we investigate to what extent a sample of machine learning\napplication papers in social computing --- specifically papers from ArXiv and\ntraditional publications performing an ML classification task on Twitter data\n--- give specific details about whether such best practices were followed. Our\nteam conducted multiple rounds of structured content analysis of each paper,\nmaking determinations such as: Does the paper report who the labelers were,\nwhat their qualifications were, whether they independently labeled the same\nitems, whether inter-rater reliability metrics were disclosed, what level of\ntraining and/or instructions were given to labelers, whether compensation for\ncrowdworkers is disclosed, and if the training data is publicly available. We\nfind a wide divergence in whether such practices were followed and documented.\nMuch of machine learning research and education focuses on what is done once a\n\"gold standard\" of training data is available, but we discuss issues around the\nequally-important aspect of whether such data is reliable in the first place.",
            "journal ref": "Proc ACM FAT* 2020",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1912.08404v3",
            "date": "2019-12-18 06:25:12+00:00",
            "title": "Collective Entity Alignment via Adaptive Features",
            "authors": [
                "Weixin Zeng",
                "Xiang Zhao",
                "Jiuyang Tang",
                "Xuemin Lin"
            ],
            "abstract": "Entity alignment (EA) identifies entities that refer to the same real-world\nobject but locate in different knowledge graphs (KGs), and has been harnessed\nfor KG construction and integration. When generating EA results, current\nsolutions treat entities independently and fail to take into account the\ninterdependence between entities. To fill this gap, we propose a collective EA\nframework. We first employ three representative features, i.e., structural,\nsemantic and string signals, which are adapted to capture different aspects of\nthe similarity between entities in heterogeneous KGs. In order to make\ncollective EA decisions, we formulate EA as the classical stable matching\nproblem, which is further effectively solved by deferred acceptance algorithm.\nOur proposal is evaluated on both cross-lingual and mono-lingual EA benchmarks\nagainst state-of-the-art solutions, and the empirical results verify its\neffectiveness and superiority.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.08441v2",
            "date": "2019-12-18 08:13:43+00:00",
            "title": "Multi-channel Reverse Dictionary Model",
            "authors": [
                "Lei Zhang",
                "Fanchao Qi",
                "Zhiyuan Liu",
                "Yasheng Wang",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "A reverse dictionary takes the description of a target word as input and\noutputs the target word together with other words that match the description.\nExisting reverse dictionary methods cannot deal with highly variable input\nqueries and low-frequency target words successfully. Inspired by the\ndescription-to-word inference process of humans, we propose the multi-channel\nreverse dictionary model, which can mitigate the two problems simultaneously.\nOur model comprises a sentence encoder and multiple predictors. The predictors\nare expected to identify different characteristics of the target word from the\ninput query. We evaluate our model on English and Chinese datasets including\nboth dictionary definitions and human-written descriptions. Experimental\nresults show that our model achieves the state-of-the-art performance, and even\noutperforms the most popular commercial reverse dictionary system on the\nhuman-written description dataset. We also conduct quantitative analyses and a\ncase study to demonstrate the effectiveness and robustness of our model. All\nthe code and data of this work can be obtained on\nhttps://github.com/thunlp/MultiRD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08442v2",
            "date": "2019-12-18 08:14:10+00:00",
            "title": "MALA: Cross-Domain Dialogue Generation with Action Learning",
            "authors": [
                "Xinting Huang",
                "Jianzhong Qi",
                "Yu Sun",
                "Rui Zhang"
            ],
            "abstract": "Response generation for task-oriented dialogues involves two basic\ncomponents: dialogue planning and surface realization. These two components,\nhowever, have a discrepancy in their objectives, i.e., task completion and\nlanguage quality. To deal with such discrepancy, conditioned response\ngeneration has been introduced where the generation process is factorized into\naction decision and language generation via explicit action representations. To\nobtain action representations, recent studies learn latent actions in an\nunsupervised manner based on the utterance lexical similarity. Such an action\nlearning approach is prone to diversities of language surfaces, which may\nimpinge task completion and language quality. To address this issue, we propose\nmulti-stage adaptive latent action learning (MALA) that learns semantic latent\nactions by distinguishing the effects of utterances on dialogue progress. We\nmodel the utterance effect using the transition of dialogue states caused by\nthe utterance and develop a semantic similarity measurement that estimates\nwhether utterances have similar effects. For learning semantic actions on\ndomains without dialogue states, MsALA extends the semantic similarity\nmeasurement across domains progressively, i.e., from aligning shared actions to\nlearning domain-specific actions. Experiments using multi-domain datasets, SMD\nand MultiWOZ, show that our proposed model achieves consistent improvements\nover the baselines models in terms of both task completion and language\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.08462v3",
            "date": "2019-12-18 09:08:05+00:00",
            "title": "End-to-end training of time domain audio separation and recognition",
            "authors": [
                "Thilo von Neumann",
                "Keisuke Kinoshita",
                "Lukas Drude",
                "Christoph Boeddeker",
                "Marc Delcroix",
                "Tomohiro Nakatani",
                "Reinhold Haeb-Umbach"
            ],
            "abstract": "The rising interest in single-channel multi-speaker speech separation sparked\ndevelopment of End-to-End (E2E) approaches to multi-speaker speech recognition.\nHowever, up until now, state-of-the-art neural network-based time domain source\nseparation has not yet been combined with E2E speech recognition. We here\ndemonstrate how to combine a separation module based on a Convolutional Time\ndomain Audio Separation Network (Conv-TasNet) with an E2E speech recognizer and\nhow to train such a model jointly by distributing it over multiple GPUs or by\napproximating truncated back-propagation for the convolutional front-end. To\nput this work into perspective and illustrate the complexity of the design\nspace, we provide a compact overview of single-channel multi-speaker\nrecognition systems. Our experiments show a word error rate of 11.0% on\nWSJ0-2mix and indicate that our joint time domain model can yield substantial\nimprovements over cascade DNN-HMM and monolithic E2E frequency domain systems\nproposed so far.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.08555v1",
            "date": "2019-12-18 12:13:30+00:00",
            "title": "Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking",
            "authors": [
                "Gustavo Penha",
                "Claudia Hauff"
            ],
            "abstract": "Neural ranking models are traditionally trained on a series of random\nbatches, sampled uniformly from the entire training set. Curriculum learning\nhas recently been shown to improve neural models' effectiveness by sampling\nbatches non-uniformly, going from easy to difficult instances during training.\nIn the context of neural Information Retrieval (IR) curriculum learning has not\nbeen explored yet, and so it remains unclear (1) how to measure the difficulty\nof training instances and (2) how to transition from easy to difficult\ninstances during training. To address both challenges and determine whether\ncurriculum learning is beneficial for neural ranking models, we need\nlarge-scale datasets and a retrieval task that allows us to conduct a wide\nrange of experiments. For this purpose, we resort to the task of conversation\nresponse ranking: ranking responses given the conversation history. In order to\ndeal with challenge (1), we explore scoring functions to measure the difficulty\nof conversations based on different input spaces. To address challenge (2) we\nevaluate different pacing functions, which determine the velocity in which we\ngo from easy to difficult instances. We find that, overall, by just\nintelligently sorting the training data (i.e., by performing curriculum\nlearning) we can improve the retrieval effectiveness by up to 2%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.08633v2",
            "date": "2019-12-18 14:33:05+00:00",
            "title": "iASiS Open Data Graph: Automated Semantic Integration of Disease-Specific Knowledge",
            "authors": [
                "Anastasios Nentidis",
                "Konstantinos Bougiatiotis",
                "Anastasia Krithara",
                "Georgios Paliouras"
            ],
            "abstract": "In biomedical research, unified access to up-to-date domain-specific\nknowledge is crucial, as such knowledge is continuously accumulated in\nscientific literature and structured resources. Identifying and extracting\nspecific information is a challenging task and computational analysis of\nknowledge bases can be valuable in this direction. However, for\ndisease-specific analyses researchers often need to compile their own datasets,\nintegrating knowledge from different resources, or reuse existing datasets,\nthat can be out-of-date. In this study, we propose a framework to automatically\nretrieve and integrate disease-specific knowledge into an up-to-date semantic\ngraph, the iASiS Open Data Graph. This disease-specific semantic graph provides\naccess to knowledge relevant to specific concepts and their individual aspects,\nin the form of concept relations and attributes. The proposed approach is\nimplemented as an open-source framework and applied to three diseases (Lung\nCancer, Dementia, and Duchenne Muscular Dystrophy). Exemplary queries are\npresented, investigating the potential of this automatically generated semantic\ngraph as a basis for retrieval and analysis of disease-specific knowledge.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.08830v3",
            "date": "2019-12-18 19:00:49+00:00",
            "title": "ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language",
            "authors": [
                "Dave Zhenyu Chen",
                "Angel X. Chang",
                "Matthias Nie\u00dfner"
            ],
            "abstract": "We introduce the task of 3D object localization in RGB-D scans using natural\nlanguage descriptions. As input, we assume a point cloud of a scanned 3D scene\nalong with a free-form description of a specified target object. To address\nthis task, we propose ScanRefer, learning a fused descriptor from 3D object\nproposals and encoded sentence embeddings. This fused descriptor correlates\nlanguage expressions with geometric features, enabling regression of the 3D\nbounding box of a target object. We also introduce the ScanRefer dataset,\ncontaining 51,583 descriptions of 11,046 objects from 800 ScanNet scenes.\nScanRefer is the first large-scale effort to perform object localization via\nnatural language expression directly in 3D.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.08904v1",
            "date": "2019-12-18 21:51:22+00:00",
            "title": "Macaw: An Extensible Conversational Information Seeking Platform",
            "authors": [
                "Hamed Zamani",
                "Nick Craswell"
            ],
            "abstract": "Conversational information seeking (CIS) has been recognized as a major\nemerging research area in information retrieval. Such research will require\ndata and tools, to allow the implementation and study of conversational\nsystems. This paper introduces Macaw, an open-source framework with a modular\narchitecture for CIS research. Macaw supports multi-turn, multi-modal, and\nmixed-initiative interactions, and enables research for tasks such as document\nretrieval, question answering, recommendation, and structured data exploration.\nIt has a modular design to encourage the study of new CIS algorithms, which can\nbe evaluated in batch mode. It can also integrate with a user interface, which\nallows user studies and data collection in an interactive mode, where the back\nend can be fully algorithmic or a wizard of oz setup. Macaw is distributed\nunder the MIT License.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.09003v1",
            "date": "2019-12-19 03:20:33+00:00",
            "title": "LSTM-TDNN with convolutional front-end for Dialect Identification in the 2019 Multi-Genre Broadcast Challenge",
            "authors": [
                "Xiaoxiao Miao",
                "Ian McLoughlin"
            ],
            "abstract": "This paper presents a novel Dialect Identification (DID) system developed for\nthe Fifth Edition of the Multi-Genre Broadcast challenge, the task of\nFine-grained Arabic Dialect Identification (MGB-5 ADI Challenge). The system\nimproves upon traditional DNN x-vector performance by employing a Convolutional\nand Long Short Term Memory-Recurrent (CLSTM) architecture to combine the\nbenefits of a convolutional neural network front-end for feature extraction and\na back-end recurrent neural to capture longer temporal dependencies.\nFurthermore we investigate intensive augmentation of one low resource dialect\nin the highly unbalanced training set using time-scale modification (TSM). This\nconverts an utterance to several time-stretched or time-compressed versions,\nsubsequently used to train the CLSTM system without using any other corpus. In\nthis paper, we also investigate speech augmentation using MUSAN and the RIR\ndatasets to increase the quantity and diversity of the existing training data\nin the normal way. Results show firstly that the CLSTM architecture outperforms\na traditional DNN x-vector implementation. Secondly, adopting TSM-based speed\nperturbation yields a small performance improvement for the unbalanced data,\nfinally that traditional data augmentation techniques yield further benefit, in\nline with evidence from related speaker and language recognition tasks. Our\nsystem achieved 2nd place ranking out of 15 entries in the MGB-5 ADI challenge,\npresented at ASRU 2019.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.09257v2",
            "date": "2019-12-19 15:09:07+00:00",
            "title": "Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems",
            "authors": [
                "Nick Rossenbach",
                "Albert Zeyer",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "Recent advances in text-to-speech (TTS) led to the development of flexible\nmulti-speaker end-to-end TTS systems. We extend state-of-the-art\nattention-based automatic speech recognition (ASR) systems with synthetic audio\ngenerated by a TTS system trained only on the ASR corpora itself. ASR and TTS\nsystems are built separately to show that text-only data can be used to enhance\nexisting end-to-end ASR systems without the necessity of parameter or\narchitecture changes. We compare our method with language model integration of\nthe same text data and with simple data augmentation methods like SpecAugment\nand show that performance improvements are mostly independent. We achieve\nimprovements of up to 33% relative in word-error-rate (WER) over a strong\nbaseline with data-augmentation in a low-resource environment\n(LibriSpeech-100h), closing the gap to a comparable oracle experiment by more\nthan 50\\%. We also show improvements of up to 5% relative WER over our most\nrecent ASR baseline on LibriSpeech-960h.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.09551v1",
            "date": "2019-12-19 21:29:22+00:00",
            "title": "Deep Exemplar Networks for VQA and VQG",
            "authors": [
                "Badri N. Patro",
                "Vinay P. Namboodiri"
            ],
            "abstract": "In this paper, we consider the problem of solving semantic tasks such as\n`Visual Question Answering' (VQA), where one aims to answers related to an\nimage and `Visual Question Generation' (VQG), where one aims to generate a\nnatural question pertaining to an image. Solutions for VQA and VQG tasks have\nbeen proposed using variants of encoder-decoder deep learning based frameworks\nthat have shown impressive performance. Humans however often show\ngeneralization by relying on exemplar based approaches. For instance, the work\nby Tversky and Kahneman suggests that humans use exemplars when making\ncategorizations and decisions. In this work, we propose the incorporation of\nexemplar based approaches towards solving these problems. Specifically, we\nincorporate exemplar based approaches and show that an exemplar based module\ncan be incorporated in almost any of the deep learning architectures proposed\nin the literature and the addition of such a block results in improved\nperformance for solving these tasks. Thus, just as the incorporation of\nattention is now considered de facto useful for solving these tasks, similarly,\nincorporating exemplars also can be considered to improve any proposed\narchitecture for solving this task. We provide extensive empirical analysis for\nthe same through various architectures, ablations, and state of the art\ncomparisons.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.09713v2",
            "date": "2019-12-20 09:32:41+00:00",
            "title": "Measuring Compositional Generalization: A Comprehensive Method on Realistic Data",
            "authors": [
                "Daniel Keysers",
                "Nathanael Sch\u00e4rli",
                "Nathan Scales",
                "Hylke Buisman",
                "Daniel Furrer",
                "Sergii Kashubin",
                "Nikola Momchev",
                "Danila Sinopalnikov",
                "Lukasz Stafiniak",
                "Tibor Tihon",
                "Dmitry Tsarkov",
                "Xiao Wang",
                "Marc van Zee",
                "Olivier Bousquet"
            ],
            "abstract": "State-of-the-art machine learning methods exhibit limited compositional\ngeneralization. At the same time, there is a lack of realistic benchmarks that\ncomprehensively measure this ability, which makes it challenging to find and\nevaluate improvements. We introduce a novel method to systematically construct\nsuch benchmarks by maximizing compound divergence while guaranteeing a small\natom divergence between train and test sets, and we quantitatively compare this\nmethod to other approaches for creating compositional generalization\nbenchmarks. We present a large and realistic natural language question\nanswering dataset that is constructed according to this method, and we use it\nto analyze the compositional generalization ability of three machine learning\narchitectures. We find that they fail to generalize compositionally and that\nthere is a surprisingly strong negative correlation between compound divergence\nand accuracy. We also demonstrate how our method can be used to create new\ncompositionality benchmarks on top of the existing SCAN dataset, which confirms\nthese findings.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.10000v2",
            "date": "2019-12-20 18:31:33+00:00",
            "title": "Probability Calibration for Knowledge Graph Embedding Models",
            "authors": [
                "Pedro Tabacof",
                "Luca Costabello"
            ],
            "abstract": "Knowledge graph embedding research has overlooked the problem of probability\ncalibration. We show popular embedding models are indeed uncalibrated. That\nmeans probability estimates associated to predicted triples are unreliable. We\npresent a novel method to calibrate a model when ground truth negatives are not\navailable, which is the usual case in knowledge graphs. We propose to use Platt\nscaling and isotonic regression alongside our method. Experiments on three\ndatasets with ground truth negatives show our contribution leads to\nwell-calibrated models when compared to the gold standard of using negatives.\nWe get significantly better results than the uncalibrated models from all\ncalibration methods. We show isotonic regression offers the best the\nperformance overall, not without trade-offs. We also show that calibrated\nmodels reach state-of-the-art accuracy without the need to define\nrelation-specific decision thresholds.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1912.10011v1",
            "date": "2019-12-20 18:41:32+00:00",
            "title": "A Hierarchical Model for Data-to-Text Generation",
            "authors": [
                "Cl\u00e9ment Rebuffel",
                "Laure Soulier",
                "Geoffrey Scoutheeten",
                "Patrick Gallinari"
            ],
            "abstract": "Transcribing structured data into natural language descriptions has emerged\nas a challenging task, referred to as \"data-to-text\". These structures\ngenerally regroup multiple elements, as well as their attributes. Most attempts\nrely on translation encoder-decoder methods which linearize elements into a\nsequence. This however loses most of the structure contained in the data. In\nthis work, we propose to overpass this limitation with a hierarchical model\nthat encodes the data-structure at the element-level and the structure level.\nEvaluations on RotoWire show the effectiveness of our model w.r.t. qualitative\nand quantitative metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10128v1",
            "date": "2019-12-20 22:45:23+00:00",
            "title": "Learning Singing From Speech",
            "authors": [
                "Liqiang Zhang",
                "Chengzhu Yu",
                "Heng Lu",
                "Chao Weng",
                "Yusong Wu",
                "Xiang Xie",
                "Zijin Li",
                "Dong Yu"
            ],
            "abstract": "We propose an algorithm that is capable of synthesizing high quality target\nspeaker's singing voice given only their normal speech samples. The proposed\nalgorithm first integrate speech and singing synthesis into a unified\nframework, and learns universal speaker embeddings that are shareable between\nspeech and singing synthesis tasks. Specifically, the speaker embeddings\nlearned from normal speech via the speech synthesis objective are shared with\nthose learned from singing samples via the singing synthesis objective in the\nunified training framework. This makes the learned speaker embedding a\ntransferable representation for both speaking and singing. We evaluate the\nproposed algorithm on singing voice conversion task where the content of\noriginal singing is covered with the timbre of another speaker's voice learned\npurely from their normal speech samples. Our experiments indicate that the\nproposed algorithm generates high-quality singing voices that sound highly\nsimilar to target speaker's voice given only his or her normal speech samples.\nWe believe that proposed algorithm will open up new opportunities for singing\nsynthesis and conversion for broader users and applications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.10131v1",
            "date": "2019-12-20 22:55:40+00:00",
            "title": "Leveraging Topics and Audio Features with Multimodal Attention for Audio Visual Scene-Aware Dialog",
            "authors": [
                "Shachi H Kumar",
                "Eda Okur",
                "Saurav Sahay",
                "Jonathan Huang",
                "Lama Nachman"
            ],
            "abstract": "With the recent advancements in Artificial Intelligence (AI), Intelligent\nVirtual Assistants (IVA) such as Alexa, Google Home, etc., have become a\nubiquitous part of many homes. Currently, such IVAs are mostly audio-based, but\ngoing forward, we are witnessing a confluence of vision, speech and dialog\nsystem technologies that are enabling the IVAs to learn audio-visual groundings\nof utterances. This will enable agents to have conversations with users about\nthe objects, activities and events surrounding them. In this work, we present\nthree main architectural explorations for the Audio Visual Scene-Aware Dialog\n(AVSD): 1) investigating `topics' of the dialog as an important contextual\nfeature for the conversation, 2) exploring several multimodal attention\nmechanisms during response generation, 3) incorporating an end-to-end audio\nclassification ConvNet, AclNet, into our architecture. We discuss detailed\nanalysis of the experimental results and show that our model variations\noutperform the baseline system presented for the AVSD task.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1912.10160v1",
            "date": "2019-12-04 17:35:03+00:00",
            "title": "AMUSED: A Multi-Stream Vector Representation Method for Use in Natural Dialogue",
            "authors": [
                "Gaurav Kumar",
                "Rishabh Joshi",
                "Jaspreet Singh",
                "Promod Yenigalla"
            ],
            "abstract": "The problem of building a coherent and non-monotonous conversational agent\nwith proper discourse and coverage is still an area of open research. Current\narchitectures only take care of semantic and contextual information for a given\nquery and fail to completely account for syntactic and external knowledge which\nare crucial for generating responses in a chit-chat system. To overcome this\nproblem, we propose an end to end multi-stream deep learning architecture which\nlearns unified embeddings for query-response pairs by leveraging contextual\ninformation from memory networks and syntactic information by incorporating\nGraph Convolution Networks (GCN) over their dependency parse. A stream of this\nnetwork also utilizes transfer learning by pre-training a bidirectional\ntransformer to extract semantic representation for each input sentence and\nincorporates external knowledge through the the neighborhood of the entities\nfrom a Knowledge Base (KB). We benchmark these embeddings on next sentence\nprediction task and significantly improve upon the existing techniques.\nFurthermore, we use AMUSED to represent query and responses along with its\ncontext to develop a retrieval based conversational agent which has been\nvalidated by expert linguists to have comprehensive engagement with humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10162v1",
            "date": "2019-12-05 13:29:27+00:00",
            "title": "Design and implementation of an open source Greek POS Tagger and Entity Recognizer using spaCy",
            "authors": [
                "Eleni Partalidou",
                "Eleftherios Spyromitros-Xioufis",
                "Stavros Doropoulos",
                "Stavros Vologiannidis",
                "Konstantinos I. Diamantaras"
            ],
            "abstract": "This paper proposes a machine learning approach to part-of-speech tagging and\nnamed entity recognition for Greek, focusing on the extraction of morphological\nfeatures and classification of tokens into a small set of classes for named\nentities. The architecture model that was used is introduced. The greek version\nof the spaCy platform was added into the source code, a feature that did not\nexist before our contribution, and was used for building the models.\nAdditionally, a part of speech tagger was trained that can detect the\nmorphology of the tokens and performs higher than the state-of-the-art results\nwhen classifying only the part of speech. For named entity recognition using\nspaCy, a model that extends the standard ENAMEX type (organization, location,\nperson) was built. Certain experiments that were conducted indicate the need\nfor flexibility in out-of-vocabulary words and there is an effort for resolving\nthis issue. Finally, the evaluation results are discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10166v1",
            "date": "2019-12-18 17:42:31+00:00",
            "title": "MedCAT -- Medical Concept Annotation Tool",
            "authors": [
                "Zeljko Kraljevic",
                "Daniel Bean",
                "Aurelie Mascio",
                "Lukasz Roguski",
                "Amos Folarin",
                "Angus Roberts",
                "Rebecca Bendayan",
                "Richard Dobson"
            ],
            "abstract": "Biomedical documents such as Electronic Health Records (EHRs) contain a large\namount of information in an unstructured format. The data in EHRs is a hugely\nvaluable resource documenting clinical narratives and decisions, but whilst the\ntext can be easily understood by human doctors it is challenging to use in\nresearch and clinical applications. To uncover the potential of biomedical\ndocuments we need to extract and structure the information they contain. The\ntask at hand is Named Entity Recognition and Linking (NER+L). The number of\nentities, ambiguity of words, overlapping and nesting make the biomedical area\nsignificantly more difficult than many others. To overcome these difficulties,\nwe have developed the Medical Concept Annotation Tool (MedCAT), an open-source\nunsupervised approach to NER+L. MedCAT uses unsupervised machine learning to\ndisambiguate entities. It was validated on MIMIC-III (a freely accessible\ncritical care database) and MedMentions (Biomedical papers annotated with\nmentions from the Unified Medical Language System). In case of NER+L, the\ncomparison with existing tools shows that MedCAT improves the previous best\nwith only unsupervised learning (F1=0.848 vs 0.691 for disease detection;\nF1=0.710 vs. 0.222 for general concept detection). A qualitative analysis of\nthe vector embeddings learnt by MedCAT shows that it captures latent medical\nknowledge available in EHRs (MIMIC-III). Unsupervised learning can improve the\nperformance of large scale entity extraction, but it has some limitations when\nworking with only a couple of entities and a small dataset. In that case\noptions are supervised learning or active learning, both of which are supported\nin MedCAT via the MedCATtrainer extension. Our approach can detect and link\nmillions of different biomedical concepts with state-of-the-art performance,\nwhilst being lightweight, fast and easy to use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10168v1",
            "date": "2019-12-12 21:21:45+00:00",
            "title": "Two Way Adversarial Unsupervised Word Translation",
            "authors": [
                "Blaine Cole"
            ],
            "abstract": "Word translation is a problem in machine translation that seeks to build\nmodels that recover word level correspondence between languages. Recent\napproaches to this problem have shown that word translation models can learned\nwith very small seeding dictionaries, and even without any starting\nsupervision. In this paper we propose a method to jointly find translations\nbetween a pair of languages. Not only does our method learn translations in\nboth directions but it improves accuracy of those translations over past\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10204v1",
            "date": "2019-12-21 05:47:58+00:00",
            "title": "A Machine Learning Framework for Authorship Identification From Texts",
            "authors": [
                "Rahul Radhakrishnan Iyer",
                "Carolyn Penstein Rose"
            ],
            "abstract": "Authorship identification is a process in which the author of a text is\nidentified. Most known literary texts can easily be attributed to a certain\nauthor because they are, for example, signed. Yet sometimes we find unfinished\npieces of work or a whole bunch of manuscripts with a wide variety of possible\nauthors. In order to assess the importance of such a manuscript, it is vital to\nknow who wrote it. In this work, we aim to develop a machine learning framework\nto effectively determine authorship. We formulate the task as a single-label\nmulti-class text categorization problem and propose a supervised machine\nlearning framework incorporating stylometric features. This task is highly\ninterdisciplinary in that it takes advantage of machine learning, information\nretrieval, and natural language processing. We present an approach and a model\nwhich learns the differences in writing style between $50$ different authors\nand is able to predict the author of a new text with high accuracy. The\naccuracy is seen to increase significantly after introducing certain linguistic\nstylometric features along with text features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10306v1",
            "date": "2019-12-21 17:49:13+00:00",
            "title": "Predicting Heart Failure Readmission from Clinical Notes Using Deep Learning",
            "authors": [
                "Xiong Liu",
                "Yu Chen",
                "Jay Bae",
                "Hu Li",
                "Joseph Johnston",
                "Todd Sanger"
            ],
            "abstract": "Heart failure hospitalization is a severe burden on healthcare. How to\npredict and therefore prevent readmission has been a significant challenge in\noutcomes research. To address this, we propose a deep learning approach to\npredict readmission from clinical notes. Unlike conventional methods that use\nstructured data for prediction, we leverage the unstructured clinical notes to\ntrain deep learning models based on convolutional neural networks (CNN). We\nthen use the trained models to classify and predict potentially high-risk\nadmissions/patients. For evaluation, we trained CNNs using the discharge\nsummary notes in the MIMIC III database. We also trained regular machine\nlearning models based on random forest using the same datasets. The result\nshows that deep learning models outperform the regular models in prediction\ntasks. CNN method achieves a F1 score of 0.756 in general readmission\nprediction and 0.733 in 30-day readmission prediction, while random forest only\nachieves a F1 score of 0.674 and 0.656 respectively. We also propose a\nchi-square test based method to interpret key features associated with deep\nlearning predicted readmissions. It reveals clinical insights about readmission\nembedded in the clinical notes. Collectively, our method can make the human\nevaluation process more efficient and potentially facilitate the reduction of\nreadmission rates.",
            "journal ref": "2019 IEEE International Conference on Bioinformatics and\n  Biomedicine (BIBM)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10337v2",
            "date": "2019-12-21 21:11:35+00:00",
            "title": "Recurrent Hierarchical Topic-Guided RNN for Language Generation",
            "authors": [
                "Dandan Guo",
                "Bo Chen",
                "Ruiying Lu",
                "Mingyuan Zhou"
            ],
            "abstract": "To simultaneously capture syntax and global semantics from a text corpus, we\npropose a new larger-context recurrent neural network (RNN) based language\nmodel, which extracts recurrent hierarchical semantic structure via a dynamic\ndeep topic model to guide natural language generation. Moving beyond a\nconventional RNN-based language model that ignores long-range word dependencies\nand sentence order, the proposed model captures not only intra-sentence word\ndependencies, but also temporal transitions between sentences and\ninter-sentence topic dependencies. For inference, we develop a hybrid of\nstochastic-gradient Markov chain Monte Carlo and recurrent autoencoding\nvariational Bayes. Experimental results on a variety of real-world text corpora\ndemonstrate that the proposed model not only outperforms larger-context\nRNN-based language models, but also learns interpretable recurrent multilayer\ntopics and generates diverse sentences and paragraphs that are syntactically\ncorrect and semantically coherent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10458v1",
            "date": "2019-12-22 14:43:14+00:00",
            "title": "Emotion Recognition from Speech",
            "authors": [
                "Kannan Venkataramanan",
                "Haresh Rengaraj Rajamohan"
            ],
            "abstract": "In this work, we conduct an extensive comparison of various approaches to\nspeech based emotion recognition systems. The analyses were carried out on\naudio recordings from Ryerson Audio-Visual Database of Emotional Speech and\nSong (RAVDESS). After pre-processing the raw audio files, features such as\nLog-Mel Spectrogram, Mel-Frequency Cepstral Coefficients (MFCCs), pitch and\nenergy were considered. The significance of these features for emotion\nclassification was compared by applying methods such as Long Short Term Memory\n(LSTM), Convolutional Neural Networks (CNNs), Hidden Markov Models (HMMs) and\nDeep Neural Networks (DNNs). On the 14-class (2 genders x 7 emotions)\nclassification task, an accuracy of 68% was achieved with a 4-layer 2\ndimensional CNN using the Log-Mel Spectrogram features. We also observe that,\nin emotion recognition, the choice of audio features impacts the results much\nmore than the model complexity.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.10514v3",
            "date": "2019-12-22 19:20:10+00:00",
            "title": "Tag-less Back-Translation",
            "authors": [
                "Idris Abdulmumin",
                "Bashir Shehu Galadanci",
                "Aliyu Garba"
            ],
            "abstract": "An effective method to generate a large number of parallel sentences for\ntraining improved neural machine translation (NMT) systems is the use of the\nback-translations of the target-side monolingual data. The standard\nback-translation method has been shown to be unable to efficiently utilize the\navailable huge amount of existing monolingual data because of the inability of\ntranslation models to differentiate between the authentic and synthetic\nparallel data during training. Tagging, or using gates, has been used to enable\ntranslation models to distinguish between synthetic and authentic data,\nimproving standard back-translation and also enabling the use of iterative\nback-translation on language pairs that underperformed using standard\nback-translation. In this work, we approach back-translation as a domain\nadaptation problem, eliminating the need for explicit tagging. In the approach\n-- \\emph{tag-less back-translation} -- the synthetic and authentic parallel\ndata are treated as out-of-domain and in-domain data respectively and, through\npre-training and fine-tuning, the translation model is shown to be able to\nlearn more efficiently from them during training. Experimental results have\nshown that the approach outperforms the standard and tagged back-translation\napproaches on low resource English-Vietnamese and English-German neural machine\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10554v2",
            "date": "2019-12-22 22:39:04+00:00",
            "title": "Harnessing Evolution of Multi-Turn Conversations for Effective Answer Retrieval",
            "authors": [
                "Mohammad Aliannejadi",
                "Manajit Chakraborty",
                "Esteban Andr\u00e9s R\u00edssola",
                "Fabio Crestani"
            ],
            "abstract": "With the improvements in speech recognition and voice generation technologies\nover the last years, a lot of companies have sought to develop conversation\nunderstanding systems that run on mobile phones or smart home devices through\nnatural language interfaces. Conversational assistants, such as Google\nAssistant and Microsoft Cortana, can help users to complete various types of\ntasks. This requires an accurate understanding of the user's information need\nas the conversation evolves into multiple turns. Finding relevant context in a\nconversation's history is challenging because of the complexity of natural\nlanguage and the evolution of a user's information need. In this work, we\npresent an extensive analysis of language, relevance, dependency of user\nutterances in a multi-turn information-seeking conversation. To this aim, we\nhave annotated relevant utterances in the conversations released by the TREC\nCaST 2019 track. The annotation labels determine which of the previous\nutterances in a conversation can be used to improve the current one.\nFurthermore, we propose a neural utterance relevance model based on BERT\nfine-tuning, outperforming competitive baselines. We study and compare the\nperformance of multiple retrieval models, utilizing different strategies to\nincorporate the user's context. The experimental results on both classification\nand retrieval tasks show that our proposed approach can effectively identify\nand incorporate the conversation context. We show that processing the current\nutterance using the predicted relevant utterance leads to a 38% relative\nimprovement in terms of nDCG@20. Finally, to foster research in this area, we\nhave released the dataset of the annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10729v1",
            "date": "2019-12-23 10:51:58+00:00",
            "title": "TextNAS: A Neural Architecture Search Space tailored for Text Representation",
            "authors": [
                "Yujing Wang",
                "Yaming Yang",
                "Yiren Chen",
                "Jing Bai",
                "Ce Zhang",
                "Guinan Su",
                "Xiaoyu Kou",
                "Yunhai Tong",
                "Mao Yang",
                "Lidong Zhou"
            ],
            "abstract": "Learning text representation is crucial for text classification and other\nlanguage related tasks. There are a diverse set of text representation networks\nin the literature, and how to find the optimal one is a non-trivial problem.\nRecently, the emerging Neural Architecture Search (NAS) techniques have\ndemonstrated good potential to solve the problem. Nevertheless, most of the\nexisting works of NAS focus on the search algorithms and pay little attention\nto the search space. In this paper, we argue that the search space is also an\nimportant human prior to the success of NAS in different applications. Thus, we\npropose a novel search space tailored for text representation. Through\nautomatic search, the discovered network architecture outperforms\nstate-of-the-art models on various public datasets on text classification and\nnatural language inference tasks. Furthermore, some of the design principles\nfound in the automatic network agree well with human intuition.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.10809v1",
            "date": "2019-11-25 12:02:15+00:00",
            "title": "Visual Summarization of Scholarly Videos using Word Embeddings and Keyphrase Extraction",
            "authors": [
                "Hang Zhou",
                "Christian Otto",
                "Ralph Ewerth"
            ],
            "abstract": "Effective learning with audiovisual content depends on many factors. Besides\nthe quality of the learning resource's content, it is essential to discover the\nmost relevant and suitable video in order to support the learning process most\neffectively. Video summarization techniques facilitate this goal by providing a\nquick overview over the content. It is especially useful for longer recordings\nsuch as conference presentations or lectures. In this paper, we present an\napproach that generates a visual summary of video content based on semantic\nword embeddings and keyphrase extraction. For this purpose, we exploit video\nannotations that are automatically generated by speech recognition and video\nOCR (optical character recognition).",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1912.10818v1",
            "date": "2019-12-15 19:48:48+00:00",
            "title": "Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models",
            "authors": [
                "Lizhen Liang",
                "Daniel E. Acuna"
            ],
            "abstract": "Detecting biases in artificial intelligence has become difficult because of\nthe impenetrable nature of deep learning. The central difficulty is in relating\nunobservable phenomena deep inside models with observable, outside quantities\nthat we can measure from inputs and outputs. For example, can we detect\ngendered perceptions of occupations (e.g., female librarian, male electrician)\nusing questions to and answers from a word embedding-based system? Current\ntechniques for detecting biases are often customized for a task, dataset, or\nmethod, affecting their generalization. In this work, we draw from\nPsychophysics in Experimental Psychology---meant to relate quantities from the\nreal world (i.e., \"Physics\") into subjective measures in the mind (i.e.,\n\"Psyche\")---to propose an intellectually coherent and generalizable framework\nto detect biases in AI. Specifically, we adapt the two-alternative forced\nchoice task (2AFC) to estimate potential biases and the strength of those\nbiases in black-box models. We successfully reproduce previously-known biased\nperceptions in word embeddings and sentiment analysis predictions. We discuss\nhow concepts in experimental psychology can be naturally applied to\nunderstanding artificial mental phenomena, and how psychophysics can form a\nuseful methodological foundation to study fairness in AI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10819v1",
            "date": "2019-12-16 15:42:30+00:00",
            "title": "Predicting the Outcome of Judicial Decisions made by the European Court of Human Rights",
            "authors": [
                "Conor O'Sullivan",
                "Joeran Beel"
            ],
            "abstract": "In this study, machine learning models were constructed to predict whether\njudgments made by the European Court of Human Rights (ECHR) would lead to a\nviolation of an Article in the Convention on Human Rights. The problem is\nframed as a binary classification task where a judgment can lead to a\n\"violation\" or \"non-violation\" of a particular Article. Using auto-sklearn, an\nautomated algorithm selection package, models were constructed for 12 Articles\nin the Convention. To train these models, textual features were obtained from\nthe ECHR Judgment documents using N-grams, word embeddings and paragraph\nembeddings. Additional documents, from the ECHR, were incorporated into the\nmodels through the creation of a word embedding (echr2vec) and a doc2vec model.\nThe features obtained using the echr2vec embedding provided the highest\ncross-validation accuracy for 5 of the Articles. The overall test accuracy,\nacross the 12 Articles, was 68.83%. As far as we could tell, this is the first\nestimate of the accuracy of such machine learning models using a realistic test\nset. This provides an important benchmark for future work. As a baseline, a\nsimple heuristic of always predicting the most common outcome in the past was\nused. The heuristic achieved an overall test accuracy of 86.68% which is 29.7%\nhigher than the models. Again, this was seemingly the first study that included\nsuch a heuristic with which to compare model results. The higher accuracy\nachieved by the heuristic highlights the importance of including such a\nbaseline.",
            "journal ref": "27th AIAI Irish Conference on Artificial Intelligence and\n  Cognitive Science. 2019",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.10824v1",
            "date": "2019-12-17 23:01:54+00:00",
            "title": "Differentiable Reasoning on Large Knowledge Bases and Natural Language",
            "authors": [
                "Pasquale Minervini",
                "Matko Bo\u0161njak",
                "Tim Rockt\u00e4schel",
                "Sebastian Riedel",
                "Edward Grefenstette"
            ],
            "abstract": "Reasoning with knowledge expressed in natural language and Knowledge Bases\n(KBs) is a major challenge for Artificial Intelligence, with applications in\nmachine reading, dialogue, and question answering. General neural architectures\nthat jointly learn representations and transformations of text are very\ndata-inefficient, and it is hard to analyse their reasoning process. These\nissues are addressed by end-to-end differentiable reasoning systems such as\nNeural Theorem Provers (NTPs), although they can only be used with small-scale\nsymbolic KBs. In this paper we first propose Greedy NTPs (GNTPs), an extension\nto NTPs addressing their complexity and scalability limitations, thus making\nthem applicable to real-world datasets. This result is achieved by dynamically\nconstructing the computation graph of NTPs and including only the most\npromising proof paths during inference, thus obtaining orders of magnitude more\nefficient models. Then, we propose a novel approach for jointly reasoning over\nKBs and textual mentions, by embedding logic facts and natural language\nsentences in a shared embedding space. We show that GNTPs perform on par with\nNTPs at a fraction of their cost while achieving competitive link prediction\nresults on large datasets, providing explanations for predictions, and inducing\ninterpretable models. Source code, datasets, and supplementary material are\navailable online at https://github.com/uclnlp/gntp.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.10839v1",
            "date": "2019-12-19 15:55:42+00:00",
            "title": "Gaussianity and typicality in matrix distributional semantics",
            "authors": [
                "Sanjaye Ramgoolam",
                "Mehrnoosh Sadrzadeh",
                "Lewis Sword"
            ],
            "abstract": "Constructions in type-driven compositional distributional semantics associate\nlarge collections of matrices of size $D$ to linguistic corpora. We develop the\nproposal of analysing the statistical characteristics of this data in the\nframework of permutation invariant matrix models. The observables in this\nframework are permutation invariant polynomial functions of the matrix entries,\nwhich correspond to directed graphs. Using the general 13-parameter permutation\ninvariant Gaussian matrix models recently solved, we find, using a dataset of\nmatrices constructed via standard techniques in distributional semantics, that\nthe expectation values of a large class of cubic and quartic observables show\nhigh gaussianity at levels between 90 to 99 percent. Beyond expectation values,\nwhich are averages over words, the dataset allows the computation of standard\ndeviations for each observable, which can be viewed as a measure of typicality\nfor each observable. There is a wide range of magnitudes in the measures of\ntypicality. The permutation invariant matrix models, considered as functions of\nrandom couplings, give a very good prediction of the magnitude of the\ntypicality for different observables. We find evidence that observables with\nsimilar matrix model characteristics of Gaussianity and typicality also have\nhigh degrees of correlation between the ranked lists of words associated to\nthese observables.",
            "journal ref": "None",
            "category": "hep-th"
        },
        {
            "url": "http://arxiv.org/abs/1912.10846v1",
            "date": "2019-12-23 14:46:46+00:00",
            "title": "BioConceptVec: creating and evaluating literature-based biomedical concept embeddings on a large scale",
            "authors": [
                "Qingyu Chen",
                "Kyubum Lee",
                "Shankai Yan",
                "Sun Kim",
                "Chih-Hsuan Wei",
                "Zhiyong Lu"
            ],
            "abstract": "Capturing the semantics of related biological concepts, such as genes and\nmutations, is of significant importance to many research tasks in computational\nbiology such as protein-protein interaction detection, gene-drug association\nprediction, and biomedical literature-based discovery. Here, we propose to\nleverage state-of-the-art text mining tools and machine learning models to\nlearn the semantics via vector representations (aka. embeddings) of over\n400,000 biological concepts mentioned in the entire PubMed abstracts. Our\nlearned embeddings, namely BioConceptVec, can capture related concepts based on\ntheir surrounding contextual information in the literature, which is beyond\nexact term match or co-occurrence-based methods. BioConceptVec has been\nthoroughly evaluated in multiple bioinformatics tasks consisting of over 25\nmillion instances from nine different biological datasets. The evaluation\nresults demonstrate that BioConceptVec has better performance than existing\nmethods in all tasks. Finally, BioConceptVec is made freely available to the\nresearch community and general public via\nhttps://github.com/ncbi-nlp/BioConceptVec.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10847v1",
            "date": "2019-12-20 18:28:29+00:00",
            "title": "What do Asian Religions Have in Common? An Unsupervised Text Analytics Exploration",
            "authors": [
                "Preeti Sah",
                "Ernest Fokou\u00e9"
            ],
            "abstract": "The main source of various religious teachings is their sacred texts which\nvary from religion to religion based on different factors like the geographical\nlocation or time of the birth of a particular religion. Despite these\ndifferences, there could be similarities between the sacred texts based on what\nlessons it teaches to its followers. This paper attempts to find the similarity\nusing text mining techniques. The corpus consisting of Asian (Tao Te Ching,\nBuddhism, Yogasutra, Upanishad) and non-Asian (four Bible texts) is used to\nexplore findings of similarity measures like Euclidean, Manhattan, Jaccard and\nCosine on raw Document Term Frequency [DTM], normalized DTM which reveals\nsimilarity based on word usage. The performance of Supervised learning\nalgorithms like K-Nearest Neighbor [KNN], Support Vector Machine [SVM] and\nRandom Forest is measured based on its accuracy to predict correct scared text\nfor any given chapter in the corpus. The K-means clustering visualizations on\nEuclidean distances of raw DTM reveals that there exists a pattern of\nsimilarity among these sacred texts with Upanishads and Tao Te Ching is the\nmost similar text in the corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10858v1",
            "date": "2019-12-20 14:37:27+00:00",
            "title": "\"The Squawk Bot\": Joint Learning of Time Series and Text Data Modalities for Automated Financial Information Filtering",
            "authors": [
                "Xuan-Hong Dang",
                "Syed Yousaf Shah",
                "Petros Zerfos"
            ],
            "abstract": "Multimodal analysis that uses numerical time series and textual corpora as\ninput data sources is becoming a promising approach, especially in the\nfinancial industry. However, the main focus of such analysis has been on\nachieving high prediction accuracy while little effort has been spent on the\nimportant task of understanding the association between the two data\nmodalities. Performance on the time series hence receives little explanation\nthough human-understandable textual information is available. In this work, we\naddress the problem of given a numerical time series, and a general corpus of\ntextual stories collected in the same period of the time series, the task is to\ntimely discover a succinct set of textual stories associated with that time\nseries. Towards this goal, we propose a novel multi-modal neural model called\nMSIN that jointly learns both numerical time series and categorical text\narticles in order to unearth the association between them. Through multiple\nsteps of data interrelation between the two data modalities, MSIN learns to\nfocus on a small subset of text articles that best align with the performance\nin the time series. This succinct set is timely discovered and presented as\nrecommended documents, acting as automated information filtering, for the given\ntime series. We empirically evaluate the performance of our model on\ndiscovering relevant news articles for two stock time series from Apple and\nGoogle companies, along with the daily news articles collected from the Thomson\nReuters over a period of seven consecutive years. The experimental results\ndemonstrate that MSIN achieves up to 84.9% and 87.2% in recalling the ground\ntruth articles respectively to the two examined time series, far more superior\nto state-of-the-art algorithms that rely on conventional attention mechanism in\ndeep learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.10915v1",
            "date": "2019-12-23 15:25:45+00:00",
            "title": "Probing the phonetic and phonological knowledge of tones in Mandarin TTS models",
            "authors": [
                "Jian Zhu"
            ],
            "abstract": "This study probes the phonetic and phonological knowledge of lexical tones in\nTTS models through two experiments. Controlled stimuli for testing tonal\ncoarticulation and tone sandhi in Mandarin were fed into Tacotron 2 and\nWaveGlow to generate speech samples, which were subject to acoustic analysis\nand human evaluation. Results show that both baseline Tacotron 2 and Tacotron 2\nwith BERT embeddings capture the surface tonal coarticulation patterns well but\nfail to consistently apply the Tone-3 sandhi rule to novel sentences.\nIncorporating pre-trained BERT embeddings into Tacotron 2 improves the\nnaturalness and prosody performance, and yields better generalization of Tone-3\nsandhi rules to novel complex sentences, although the overall accuracy for\nTone-3 sandhi was still low. Given that TTS models do capture some linguistic\nphenomena, it is argued that they can be used to generate and validate certain\nlinguistic hypotheses. On the other hand, it is also suggested that\nlinguistically informed stimuli should be included in the training and the\nevaluation of TTS models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11078v2",
            "date": "2019-11-09 23:53:19+00:00",
            "title": "Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview",
            "authors": [
                "Deven Shah",
                "H. Andrew Schwartz",
                "Dirk Hovy"
            ],
            "abstract": "An increasing number of works in natural language processing have addressed\nthe effect of bias on the predicted outcomes, introducing mitigation techniques\nthat act on different parts of the standard NLP pipeline (data and models).\nHowever, these works have been conducted in isolation, without a unifying\nframework to organize efforts within the field. This leads to repetitive\napproaches, and puts an undue focus on the effects of bias, rather than on\ntheir origins. Research focused on bias symptoms rather than the underlying\norigins could limit the development of effective countermeasures. In this\npaper, we propose a unifying conceptualization: the predictive bias framework\nfor NLP. We summarize the NLP literature and propose a general mathematical\ndefinition of predictive bias in NLP along with a conceptual framework,\ndifferentiating four main origins of biases: label bias, selection bias, model\noveramplification, and semantic bias. We discuss how past work has countered\neach bias origin. Our framework serves to guide an introductory overview of\npredictive bias in NLP, integrating existing work into a single structure and\nopening avenues for future research.",
            "journal ref": "Association for Computational Linguistics. (2020) 5248--5264",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11151v1",
            "date": "2019-12-18 12:26:38+00:00",
            "title": "A Cycle-GAN Approach to Model Natural Perturbations in Speech for ASR Applications",
            "authors": [
                "Sri Harsha Dumpala",
                "Imran Sheikh",
                "Rupayan Chakraborty",
                "Sunil Kumar Kopparapu"
            ],
            "abstract": "Naturally introduced perturbations in audio signal, caused by emotional and\nphysical states of the speaker, can significantly degrade the performance of\nAutomatic Speech Recognition (ASR) systems. In this paper, we propose a\nfront-end based on Cycle-Consistent Generative Adversarial Network (CycleGAN)\nwhich transforms naturally perturbed speech into normal speech, and hence\nimproves the robustness of an ASR system. The CycleGAN model is trained on\nnon-parallel examples of perturbed and normal speech. Experiments on\nspontaneous laughter-speech and creaky-speech datasets show that the\nperformance of four different ASR systems improve by using speech obtained from\nCycleGAN based front-end, as compared to directly using the original perturbed\nspeech. Visualization of the features of the laughter perturbed speech and\nthose generated by the proposed front-end further demonstrates the\neffectiveness of our approach.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.11585v1",
            "date": "2019-12-25 03:44:31+00:00",
            "title": "THUEE system description for NIST 2019 SRE CTS Challenge",
            "authors": [
                "Yi Liu",
                "Tianyu Liang",
                "Can Xu",
                "Xianwei Zhang",
                "Xianhong Chen",
                "Wei-Qiang Zhang",
                "Liang He",
                "Dandan song",
                "Ruyun Li",
                "Yangcheng Wu",
                "Peng Ouyang",
                "Shouyi Yin"
            ],
            "abstract": "This paper describes the systems submitted by the department of electronic\nengineering, institute of microelectronics of Tsinghua university and\nTsingMicro Co. Ltd. (THUEE) to the NIST 2019 speaker recognition evaluation CTS\nchallenge. Six subsystems, including etdnn/ams, ftdnn/as, eftdnn/ams, resnet,\nmultitask and c-vector are developed in this evaluation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.11600v1",
            "date": "2019-12-25 05:59:29+00:00",
            "title": "A statistical test for correspondence of texts to the Zipf-Mandelbrot law",
            "authors": [
                "Anik Chakrabarty",
                "Mikhail Chebunin",
                "Artyom Kovalevskii",
                "Ilya Pupyshev",
                "Natalia Zakrevskaya",
                "Qianqian Zhou"
            ],
            "abstract": "We analyse correspondence of a text to a simple probabilistic model. The\nmodel assumes that the words are selected independently from an infinite\ndictionary. The probability distribution correspond to the Zipf---Mandelbrot\nlaw. We count sequentially the numbers of different words in the text and get\nthe process of the numbers of different words. Then we estimate\nZipf---Mandelbrot law parameters using the same sequence and construct an\nestimate of the expectation of the number of different words in the text. Then\nwe subtract the corresponding values of the estimate from the sequence and\nnormalize along the coordinate axes, obtaining a random process on a segment\nfrom 0 to 1. We prove that this process (the empirical text bridge) converges\nweakly in the uniform metric on $C (0,1)$ to a centered Gaussian process with\ncontinuous a.s. paths. We develop and implement an algorithm for approximate\ncalculation of eigenvalues of the covariance function of the limit Gaussian\nprocess, and then an algorithm for calculating the probability distribution of\nthe integral of the square of this process. We use the algorithm to analyze\nuniformity of texts in English, French, Russian and Chinese.",
            "journal ref": "None",
            "category": "math.ST"
        },
        {
            "url": "http://arxiv.org/abs/1912.11688v1",
            "date": "2019-12-25 16:25:29+00:00",
            "title": "Unity in Diversity: Learning Distributed Heterogeneous Sentence Representation for Extractive Summarization",
            "authors": [
                "Abhishek Kumar Singh",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Automated multi-document extractive text summarization is a widely studied\nresearch problem in the field of natural language understanding. Such\nextractive mechanisms compute in some form the worthiness of a sentence to be\nincluded into the summary. While the conventional approaches rely on human\ncrafted document-independent features to generate a summary, we develop a\ndata-driven novel summary system called HNet, which exploits the various\nsemantic and compositional aspects latent in a sentence to capture document\nindependent features. The network learns sentence representation in a way that,\nsalient sentences are closer in the vector space than non-salient sentences.\nThis semantic and compositional feature vector is then concatenated with the\ndocument-dependent features for sentence ranking. Experiments on the DUC\nbenchmark datasets (DUC-2001, DUC-2002 and DUC-2004) indicate that our model\nshows significant performance gain of around 1.5-2 points in terms of ROUGE\nscore compared with the state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11701v1",
            "date": "2019-12-25 17:48:09+00:00",
            "title": "Hybrid MemNet for Extractive Summarization",
            "authors": [
                "Abhishek Kumar Singh",
                "Manish Gupta",
                "Vasudeva Varma"
            ],
            "abstract": "Extractive text summarization has been an extensive research problem in the\nfield of natural language understanding. While the conventional approaches rely\nmostly on manually compiled features to generate the summary, few attempts have\nbeen made in developing data-driven systems for extractive summarization. To\nthis end, we present a fully data-driven end-to-end deep network which we call\nas Hybrid MemNet for single document summarization task. The network learns the\ncontinuous unified representation of a document before generating its summary.\nIt jointly captures local and global sentential information along with the\nnotion of summary worthy sentences. Experimental results on two different\ncorpora confirm that our model shows significant performance gains compared\nwith the state-of-the-art baselines.",
            "journal ref": "In Proceedings of the 2017 ACM on Conference on Information and\n  Knowledge Management (CIKM '17). ACM, New York, NY, USA, pages 2303-2306",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11739v2",
            "date": "2019-12-26 01:12:31+00:00",
            "title": "Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation",
            "authors": [
                "Haiyue Song",
                "Raj Dabre",
                "Atsushi Fujita",
                "Sadao Kurohashi"
            ],
            "abstract": "Lectures translation is a case of spoken language translation and there is a\nlack of publicly available parallel corpora for this purpose. To address this,\nwe examine a language independent framework for parallel corpus mining which is\na quick and effective way to mine a parallel corpus from publicly available\nlectures at Coursera. Our approach determines sentence alignments, relying on\nmachine translation and cosine similarity over continuous-space sentence\nrepresentations. We also show how to use the resulting corpora in a multistage\nfine-tuning based domain adaptation for high-quality lectures translation. For\nJapanese--English lectures translation, we extracted parallel data of\napproximately 40,000 lines and created development and test sets through manual\nfiltering for benchmarking translation performance. We demonstrate that the\nmined corpus greatly enhances the quality of translation when used in\nconjunction with out-of-domain parallel corpora via multistage training. This\npaper also suggests some guidelines to gather and clean corpora, mine parallel\nsentences, address noise in the mined data, and create high-quality evaluation\nsplits. For the sake of reproducibility, we will release our code for parallel\ndata creation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.11959v2",
            "date": "2019-12-27 02:01:13+00:00",
            "title": "Is Attention All What You Need? -- An Empirical Investigation on Convolution-Based Active Memory and Self-Attention",
            "authors": [
                "Thomas Dowdell",
                "Hongyu Zhang"
            ],
            "abstract": "The key to a Transformer model is the self-attention mechanism, which allows\nthe model to analyze an entire sequence in a computationally efficient manner.\nRecent work has suggested the possibility that general attention mechanisms\nused by RNNs could be replaced by active-memory mechanisms. In this work, we\nevaluate whether various active-memory mechanisms could replace self-attention\nin a Transformer. Our experiments suggest that active-memory alone achieves\ncomparable results to the self-attention mechanism for language modelling, but\noptimal results are mostly achieved by using both active-memory and\nself-attention mechanisms together. We also note that, for some specific\nalgorithmic tasks, active-memory mechanisms alone outperform both\nself-attention and a combination of the two.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.12362v1",
            "date": "2019-12-27 23:15:49+00:00",
            "title": "Structural characterization of musical harmonies",
            "authors": [
                "Maria Rojo Gonz\u00e1lez",
                "Simone Santini"
            ],
            "abstract": "Understanding the structural characteristics of harmony is essential for an\neffective use of music as a communication medium. Of the three expressive axes\nof music (melody, rhythm, harmony), harmony is the foundation on which the\nemotional content is built, and its understanding is important in areas such as\nmultimedia and affective computing. The common tool for studying this kind of\nstructure in computing science is the formal grammar but, in the case of music,\ngrammars run into problems due to the ambiguous nature of some of the concepts\ndefined in music theory. In this paper, we consider one of such constructs:\nmodulation, that is, the change of key in the middle of a musical piece, an\nimportant tool used by many authors to enhance the capacity of music to express\nemotions. We develop a hybrid method in which an evidence-gathering numerical\nmethod detects modulation and then, based on the detected tonalities, a\nnon-ambiguous grammar can be used for analyzing the structure of each tonal\ncomponent. Experiments with music from the XVII and XVIII centuries show that\nwe can detect the precise point of modulation with an error of at most two\nchords in almost 97% of the cases. Finally, we show examples of complete\nmodulation and structural analysis of musical harmonies.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1912.12394v2",
            "date": "2019-12-28 03:51:52+00:00",
            "title": "All-in-One Image-Grounded Conversational Agents",
            "authors": [
                "Da Ju",
                "Kurt Shuster",
                "Y-Lan Boureau",
                "Jason Weston"
            ],
            "abstract": "As single-task accuracy on individual language and image tasks has improved\nsubstantially in the last few years, the long-term goal of a generally skilled\nagent that can both see and talk becomes more feasible to explore. In this\nwork, we focus on leveraging individual language and image tasks, along with\nresources that incorporate both vision and language towards that objective. We\ndesign an architecture that combines state-of-the-art Transformer and ResNeXt\nmodules fed into a novel attentive multimodal module to produce a combined\nmodel trained on many tasks. We provide a thorough analysis of the components\nof the model, and transfer performance when training on one, some, or all of\nthe tasks. Our final models provide a single system that obtains good results\non all vision and language tasks considered, and improves the state-of-the-art\nin image-grounded conversational applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12422v2",
            "date": "2019-12-28 07:52:00+00:00",
            "title": "TextScanner: Reading Characters in Order for Robust Scene Text Recognition",
            "authors": [
                "Zhaoyi Wan",
                "Minghang He",
                "Haoran Chen",
                "Xiang Bai",
                "Cong Yao"
            ],
            "abstract": "Driven by deep learning and the large volume of data, scene text recognition\nhas evolved rapidly in recent years. Formerly, RNN-attention based methods have\ndominated this field, but suffer from the problem of \\textit{attention drift}\nin certain situations. Lately, semantic segmentation based algorithms have\nproven effective at recognizing text of different forms (horizontal, oriented\nand curved). However, these methods may produce spurious characters or miss\ngenuine characters, as they rely heavily on a thresholding procedure operated\non segmentation maps. To tackle these challenges, we propose in this paper an\nalternative approach, called TextScanner, for scene text recognition.\nTextScanner bears three characteristics: (1) Basically, it belongs to the\nsemantic segmentation family, as it generates pixel-wise, multi-channel\nsegmentation maps for character class, position and order; (2) Meanwhile, akin\nto RNN-attention based methods, it also adopts RNN for context modeling; (3)\nMoreover, it performs paralleled prediction for character position and class,\nand ensures that characters are transcripted in correct order. The experiments\non standard benchmark datasets demonstrate that TextScanner outperforms the\nstate-of-the-art methods. Moreover, TextScanner shows its superiority in\nrecognizing more difficult text such Chinese transcripts and aligning with\ntarget characters.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.12481v2",
            "date": "2019-12-28 16:18:33+00:00",
            "title": "Robust Cross-lingual Embeddings from Parallel Sentences",
            "authors": [
                "Ali Sabet",
                "Prakhar Gupta",
                "Jean-Baptiste Cordonnier",
                "Robert West",
                "Martin Jaggi"
            ],
            "abstract": "Recent advances in cross-lingual word embeddings have primarily relied on\nmapping-based methods, which project pretrained word embeddings from different\nlanguages into a shared space through a linear transformation. However, these\napproaches assume word embedding spaces are isomorphic between different\nlanguages, which has been shown not to hold in practice (S{\\o}gaard et al.,\n2018), and fundamentally limits their performance. This motivates investigating\njoint learning methods which can overcome this impediment, by simultaneously\nlearning embeddings across languages via a cross-lingual term in the training\nobjective. We propose a bilingual extension of the CBOW method which leverages\nsentence-aligned corpora to obtain robust cross-lingual word and sentence\nrepresentations. Our approach significantly improves cross-lingual sentence\nretrieval performance over all other approaches while maintaining parity with\nthe current state-of-the-art methods on word-translation. It also achieves\nparity with a deep RNN method on a zero-shot cross-lingual document\nclassification task, requiring far fewer computational resources for training\nand inference. As an additional advantage, our bilingual method leads to a much\nmore pronounced improvement in the the quality of monolingual word vectors\ncompared to other competing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.12520v2",
            "date": "2019-12-28 21:20:25+00:00",
            "title": "Weak Supervision for Fake News Detection via Reinforcement Learning",
            "authors": [
                "Yaqing Wang",
                "Weifeng Yang",
                "Fenglong Ma",
                "Jin Xu",
                "Bin Zhong",
                "Qiang Deng",
                "Jing Gao"
            ],
            "abstract": "Today social media has become the primary source for news. Via social media\nplatforms, fake news travel at unprecedented speeds, reach global audiences and\nput users and communities at great risk. Therefore, it is extremely important\nto detect fake news as early as possible. Recently, deep learning based\napproaches have shown improved performance in fake news detection. However, the\ntraining of such models requires a large amount of labeled data, but manual\nannotation is time-consuming and expensive. Moreover, due to the dynamic nature\nof news, annotated samples may become outdated quickly and cannot represent the\nnews articles on newly emerged events. Therefore, how to obtain fresh and\nhigh-quality labeled samples is the major challenge in employing deep learning\nmodels for fake news detection. In order to tackle this challenge, we propose a\nreinforced weakly-supervised fake news detection framework, i.e., WeFEND, which\ncan leverage users' reports as weak supervision to enlarge the amount of\ntraining data for fake news detection. The proposed framework consists of three\nmain components: the annotator, the reinforced selector and the fake news\ndetector. The annotator can automatically assign weak labels for unlabeled news\nbased on users' reports. The reinforced selector using reinforcement learning\ntechniques chooses high-quality samples from the weakly labeled data and\nfilters out those low-quality ones that may degrade the detector's prediction\nperformance. The fake news detector aims to identify fake news based on the\nnews content. We tested the proposed framework on a large collection of news\narticles published via WeChat official accounts and associated user reports.\nExtensive experiments on this dataset show that the proposed WeFEND model\nachieves the best performance compared with the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1912.12602v1",
            "date": "2019-12-29 07:58:18+00:00",
            "title": "Complex Cepstrum-based Decomposition of Speech for Glottal Source Estimation",
            "authors": [
                "Thomas Drugman",
                "Baris Bozkurt",
                "Thierry Dutoit"
            ],
            "abstract": "Homomorphic analysis is a well-known method for the separation of\nnon-linearly combined signals. More particularly, the use of complex cepstrum\nfor source-tract deconvolution has been discussed in various articles. However\nthere exists no study which proposes a glottal flow estimation methodology\nbased on cepstrum and reports effective results. In this paper, we show that\ncomplex cepstrum can be effectively used for glottal flow estimation by\nseparating the causal and anticausal components of a windowed speech signal as\ndone by the Zeros of the Z-Transform (ZZT) decomposition. Based on exactly the\nsame principles presented for ZZT decomposition, windowing should be applied\nsuch that the windowed speech signals exhibit mixed-phase characteristics which\nconform the speech production model that the anticausal component is mainly due\nto the glottal flow open phase. The advantage of the complex cepstrum-based\napproach compared to the ZZT decomposition is its much higher speed.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.12604v1",
            "date": "2019-12-29 08:13:58+00:00",
            "title": "Glottal Source Processing: from Analysis to Applications",
            "authors": [
                "Thomas Drugman",
                "Paavo Alku",
                "Abeer Alwan",
                "Bayya Yegnanarayana"
            ],
            "abstract": "The great majority of current voice technology applications relies on\nacoustic features characterizing the vocal tract response, such as the widely\nused MFCC of LPC parameters. Nonetheless, the airflow passing through the vocal\nfolds, and called glottal flow, is expected to exhibit a relevant\ncomplementarity. Unfortunately, glottal analysis from speech recordings\nrequires specific and more complex processing operations, which explains why it\nhas been generally avoided. This review gives a general overview of techniques\nwhich have been designed for glottal source processing. Starting from\nfundamental analysis tools of pitch tracking, glottal closure instant\ndetection, glottal flow estimation and modelling, this paper then highlights\nhow these solutions can be properly integrated within various voice technology\napplications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.12628v1",
            "date": "2019-12-29 11:05:47+00:00",
            "title": "Dirichlet uncertainty wrappers for actionable algorithm accuracy accountability and auditability",
            "authors": [
                "Jos\u00e9 Mena",
                "Oriol Pujol",
                "Jordi Vitri\u00e0"
            ],
            "abstract": "Nowadays, the use of machine learning models is becoming a utility in many\napplications. Companies deliver pre-trained models encapsulated as application\nprogramming interfaces (APIs) that developers combine with third party\ncomponents and their own models and data to create complex data products to\nsolve specific problems. The complexity of such products and the lack of\ncontrol and knowledge of the internals of each component used cause unavoidable\neffects, such as lack of transparency, difficulty in auditability, and\nemergence of potential uncontrolled risks. They are effectively black-boxes.\nAccountability of such solutions is a challenge for the auditors and the\nmachine learning community. In this work, we propose a wrapper that given a\nblack-box model enriches its output prediction with a measure of uncertainty.\nBy using this wrapper, we make the black-box auditable for the accuracy risk\n(risk derived from low quality or uncertain decisions) and at the same time we\nprovide an actionable mechanism to mitigate that risk in the form of decision\nrejection; we can choose not to issue a prediction when the risk or uncertainty\nin that decision is significant. Based on the resulting uncertainty measure, we\nadvocate for a rejection system that selects the more confident predictions,\ndiscarding those more uncertain, leading to an improvement in the trustability\nof the resulting system. We showcase the proposed technique and methodology in\na practical scenario where a simulated sentiment analysis API based on natural\nlanguage processing is applied to different domains. Results demonstrate the\neffectiveness of the uncertainty computed by the wrapper and its high\ncorrelation to bad quality predictions and misclassifications.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.12843v1",
            "date": "2019-12-30 08:12:03+00:00",
            "title": "Causal-Anticausal Decomposition of Speech using Complex Cepstrum for Glottal Source Estimation",
            "authors": [
                "Thomas Drugman",
                "Baris Bozkurt",
                "Thierry Dutoit"
            ],
            "abstract": "Complex cepstrum is known in the literature for linearly separating causal\nand anticausal components. Relying on advances achieved by the Zeros of the\nZ-Transform (ZZT) technique, we here investigate the possibility of using\ncomplex cepstrum for glottal flow estimation on a large-scale database. Via a\nsystematic study of the windowing effects on the deconvolution quality, we show\nthat the complex cepstrum causal-anticausal decomposition can be effectively\nused for glottal flow estimation when specific windowing criteria are met. It\nis also shown that this complex cepstral decomposition gives similar glottal\nestimates as obtained with the ZZT method. However, as complex cepstrum uses\nFFT operations instead of requiring the factoring of high-degree polynomials,\nthe method benefits from a much higher speed. Finally in our tests on a large\ncorpus of real expressive speech, we show that the proposed method has the\npotential to be used for voice quality analysis.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.12887v1",
            "date": "2019-12-30 11:34:39+00:00",
            "title": "Using a Pitch-Synchronous Residual Codebook for Hybrid HMM/Frame Selection Speech Synthesis",
            "authors": [
                "Thomas Drugman",
                "Alexis Moinet",
                "Thierry Dutoit",
                "Geoffrey Wilfart"
            ],
            "abstract": "This paper proposes a method to improve the quality delivered by statistical\nparametric speech synthesizers. For this, we use a codebook of\npitch-synchronous residual frames, so as to construct a more realistic source\nsignal. First a limited codebook of typical excitations is built from some\ntraining database. During the synthesis part, HMMs are used to generate filter\nand source coefficients. The latter coefficients contain both the pitch and a\ncompact representation of target residual frames. The source signal is obtained\nby concatenating excitation frames picked up from the codebook, based on a\nselection criterion and taking target residual coefficients as input.\nSubjective results show a relevant improvement compared to the basic technique.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1912.12999v3",
            "date": "2019-12-30 16:44:41+00:00",
            "title": "AutoDiscern: Rating the Quality of Online Health Information with Hierarchical Encoder Attention-based Neural Networks",
            "authors": [
                "Laura Kinkead",
                "Ahmed Allam",
                "Michael Krauthammer"
            ],
            "abstract": "Patients increasingly turn to search engines and online content before, or in\nplace of, talking with a health professional. Low quality health information,\nwhich is common on the internet, presents risks to the patient in the form of\nmisinformation and a possibly poorer relationship with their physician. To\naddress this, the DISCERN criteria (developed at University of Oxford) are used\nto evaluate the quality of online health information. However, patients are\nunlikely to take the time to apply these criteria to the health websites they\nvisit. We built an automated implementation of the DISCERN instrument (Brief\nversion) using machine learning models. We compared the performance of a\ntraditional model (Random Forest) with that of a hierarchical encoder\nattention-based neural network (HEA) model using two language embeddings, BERT\nand BioBERT. The HEA BERT and BioBERT models achieved average F1-macro scores\nacross all criteria of 0.75 and 0.74, respectively, outperforming the Random\nForest model (average F1-macro = 0.69). Overall, the neural network based\nmodels achieved 81% and 86% average accuracy at 100% and 80% coverage,\nrespectively, compared to 94% manual rating accuracy. The attention mechanism\nimplemented in the HEA architectures not only provided 'model explainability'\nby identifying reasonable supporting sentences for the documents fulfilling the\nBrief DISCERN criteria, but also boosted F1 performance by 0.05 compared to the\nsame architecture without an attention mechanism. Our research suggests that it\nis feasible to automate online health information quality assessment, which is\nan important step towards empowering patients to become informed partners in\nthe healthcare process.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.13072v2",
            "date": "2019-12-30 20:05:37+00:00",
            "title": "AraNet: A Deep Learning Toolkit for Arabic Social Media",
            "authors": [
                "Muhammad Abdul-Mageed",
                "Chiyu Zhang",
                "Azadeh Hashemi",
                "El Moatez Billah Nagoudi"
            ],
            "abstract": "We describe AraNet, a collection of deep learning Arabic social media\nprocessing tools. Namely, we exploit an extensive host of publicly available\nand novel social media datasets to train bidirectional encoders from\ntransformer models (BERT) to predict age, dialect, gender, emotion, irony, and\nsentiment. AraNet delivers state-of-the-art performance on a number of the\ncited tasks and competitively on others. In addition, AraNet has the advantage\nof being exclusively based on a deep learning framework and hence feature\nengineering free. To the best of our knowledge, AraNet is the first to performs\npredictions across such a wide range of tasks for Arabic NLP and thus meets a\ncritical needs. We publicly release AraNet to accelerate research and\nfacilitate comparisons across the different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13080v1",
            "date": "2019-12-30 20:46:38+00:00",
            "title": "Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning",
            "authors": [
                "Sean MacAvaney",
                "Luca Soldaini",
                "Nazli Goharian"
            ],
            "abstract": "While billions of non-English speaking users rely on search engines every\nday, the problem of ad-hoc information retrieval is rarely studied for\nnon-English languages. This is primarily due to a lack of data set that are\nsuitable to train ranking algorithms. In this paper, we tackle the lack of data\nby leveraging pre-trained multilingual language models to transfer a retrieval\nsystem trained on English collections to non-English queries and documents. Our\nmodel is evaluated in a zero-shot setting, meaning that we use them to predict\nrelevance scores for query-document pairs in languages never seen during\ntraining. Our results show that the proposed approach can significantly\noutperform unsupervised retrieval techniques for Arabic, Chinese Mandarin, and\nSpanish. We also show that augmenting the English training collection with some\nexamples from the target language can sometimes improve performance.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1912.13109v1",
            "date": "2019-12-30 23:01:28+00:00",
            "title": "\"Hinglish\" Language -- Modeling a Messy Code-Mixed Language",
            "authors": [
                "Vivek Kumar Gupta"
            ],
            "abstract": "With a sharp rise in fluency and users of \"Hinglish\" in linguistically\ndiverse country, India, it has increasingly become important to analyze social\ncontent written in this language in platforms such as Twitter, Reddit,\nFacebook. This project focuses on using deep learning techniques to tackle a\nclassification problem in categorizing social content written in Hindi-English\ninto Abusive, Hate-Inducing and Not offensive categories. We utilize\nbi-directional sequence models with easy text augmentation techniques such as\nsynonym replacement, random insertion, random swap, and random deletion to\nproduce a state of the art classifier that outperforms the previous work done\non analyzing this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13283v2",
            "date": "2019-12-31 12:11:35+00:00",
            "title": "oLMpics -- On what Language Model Pre-training Captures",
            "authors": [
                "Alon Talmor",
                "Yanai Elazar",
                "Yoav Goldberg",
                "Jonathan Berant"
            ],
            "abstract": "Recent success of pre-trained language models (LMs) has spurred widespread\ninterest in the language capabilities that they possess. However, efforts to\nunderstand whether LM representations are useful for symbolic reasoning tasks\nhave been limited and scattered. In this work, we propose eight reasoning\ntasks, which conceptually require operations such as comparison, conjunction,\nand composition. A fundamental challenge is to understand whether the\nperformance of a LM on a task should be attributed to the pre-trained\nrepresentations or to the process of fine-tuning on the task data. To address\nthis, we propose an evaluation protocol that includes both zero-shot evaluation\n(no fine-tuning), as well as comparing the learning curve of a fine-tuned LM to\nthe learning curve of multiple controls, which paints a rich picture of the LM\ncapabilities. Our main findings are that: (a) different LMs exhibit\nqualitatively different reasoning abilities, e.g., RoBERTa succeeds in\nreasoning tasks where BERT fails completely; (b) LMs do not reason in an\nabstract manner and are context-dependent, e.g., while RoBERTa can compare\nages, it can do so only when the ages are in the typical range of human ages;\n(c) On half of our reasoning tasks all models fail completely. Our findings and\ninfrastructure can help future work on designing new datasets, models and\nobjective functions for pre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13362v1",
            "date": "2019-12-26 08:38:49+00:00",
            "title": "Text Classification for Azerbaijani Language Using Machine Learning and Embedding",
            "authors": [
                "Umid Suleymanov",
                "Behnam Kiani Kalejahi",
                "Elkhan Amrahov",
                "Rashid Badirkhanli"
            ],
            "abstract": "Text classification systems will help to solve the text clustering problem in\nthe Azerbaijani language. There are some text-classification applications for\nforeign languages, but we tried to build a newly developed system to solve this\nproblem for the Azerbaijani language. Firstly, we tried to find out potential\npractice areas. The system will be useful in a lot of areas. It will be mostly\nused in news feed categorization. News websites can automatically categorize\nnews into classes such as sports, business, education, science, etc. The system\nis also used in sentiment analysis for product reviews. For example, the\ncompany shares a photo of a new product on Facebook and the company receives a\nthousand comments for new products. The systems classify the comments into\ncategories like positive or negative. The system can also be applied in\nrecommended systems, spam filtering, etc. Various machine learning techniques\nsuch as Naive Bayes, SVM, Decision Trees have been devised to solve the text\nclassification problem in Azerbaijani language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.13455v1",
            "date": "2019-12-31 17:52:05+00:00",
            "title": "Essential Sentences for Navigating Stack Overflow Answers",
            "authors": [
                "Sarah Nadi",
                "Christoph Treude"
            ],
            "abstract": "Stack Overflow (SO) has become an essential resource for software\ndevelopment. Despite its success and prevalence, navigating SO remains a\nchallenge. Ideally, SO users could benefit from highlighted navigational cues\nthat help them decide if an answer is relevant to their task and context. Such\nnavigational cues could be in the form of essential sentences that help the\nsearcher decide whether they want to read the answer or skip over it. In this\npaper, we compare four potential approaches for identifying essential\nsentences. We adopt two existing approaches and develop two new approaches\nbased on the idea that contextual information in a sentence (e.g., \"if using\nwindows\") could help identify essential sentences. We compare the four\ntechniques using a survey of 43 participants. Our participants indicate that it\nis not always easy to figure out what the best solution for their specific\nproblem is, given the options, and that they would indeed like to easily spot\ncontextual information that may narrow down the search. Our quantitative\ncomparison of the techniques shows that there is no single technique sufficient\nfor identifying essential sentences that can serve as navigational cues, while\nour qualitative analysis shows that participants valued explanations and\nspecific conditions, and did not value filler sentences or speculations. Our\nwork sheds light on the importance of navigational cues, and our findings can\nbe used to guide future research to find the best combination of techniques to\nidentify such cues.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2001.00056v1",
            "date": "2019-12-31 19:54:27+00:00",
            "title": "Deep Attentive Ranking Networks for Learning to Order Sentences",
            "authors": [
                "Pawan Kumar",
                "Dhanajit Brahma",
                "Harish Karnick",
                "Piyush Rai"
            ],
            "abstract": "We present an attention-based ranking framework for learning to order\nsentences given a paragraph. Our framework is built on a bidirectional sentence\nencoder and a self-attention based transformer network to obtain an input order\ninvariant representation of paragraphs. Moreover, it allows seamless training\nusing a variety of ranking based loss functions, such as pointwise, pairwise,\nand listwise ranking. We apply our framework on two tasks: Sentence Ordering\nand Order Discrimination. Our framework outperforms various state-of-the-art\nmethods on these tasks on a variety of evaluation metrics. We also show that it\nachieves better results when using pairwise and listwise ranking losses, rather\nthan the pointwise ranking loss, which suggests that incorporating relative\npositions of two or more sentences in the loss function contributes to better\nlearning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00059v3",
            "date": "2019-12-21 05:05:22+00:00",
            "title": "Learning and Evaluating Contextual Embedding of Source Code",
            "authors": [
                "Aditya Kanade",
                "Petros Maniatis",
                "Gogul Balakrishnan",
                "Kensen Shi"
            ],
            "abstract": "Recent research has achieved impressive results on understanding and\nimproving source code by building up on machine-learning techniques developed\nfor natural languages. A significant advancement in natural-language\nunderstanding has come with the development of pre-trained contextual\nembeddings, such as BERT, which can be fine-tuned for downstream tasks with\nless labeled data and training budget, while achieving better accuracies.\nHowever, there is no attempt yet to obtain a high-quality contextual embedding\nof source code, and to evaluate it on multiple program-understanding tasks\nsimultaneously; that is the gap that this paper aims to mitigate. Specifically,\nfirst, we curate a massive, deduplicated corpus of 7.4M Python files from\nGitHub, which we use to pre-train CuBERT, an open-sourced code-understanding\nBERT model; and, second, we create an open-sourced benchmark that comprises\nfive classification tasks and one program-repair task, akin to\ncode-understanding tasks proposed in the literature before. We fine-tune CuBERT\non our benchmark tasks, and compare the resulting models to different variants\nof Word2Vec token embeddings, BiLSTM and Transformer models, as well as\npublished state-of-the-art models, showing that CuBERT outperforms them all,\neven with shorter training, and with fewer labeled examples. Future work on\nsource-code embedding can benefit from reusing our benchmark, and from\ncomparing against CuBERT models as a strong baseline.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2001.00459v1",
            "date": "2019-12-28 13:45:29+00:00",
            "title": "Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics",
            "authors": [
                "Thomas Drugman",
                "Abeer Alwan"
            ],
            "abstract": "This paper focuses on the problem of pitch tracking in noisy conditions. A\nmethod using harmonic information in the residual signal is presented. The\nproposed criterion is used both for pitch estimation, as well as for\ndetermining the voicing segments of speech. In the experiments, the method is\ncompared to six state-of-the-art pitch trackers on the Keele and CSTR\ndatabases. The proposed technique is shown to be particularly robust to\nadditive noise, leading to a significant improvement in adverse conditions.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00471v1",
            "date": "2019-12-20 18:58:25+00:00",
            "title": "A Voice Interactive Multilingual Student Support System using IBM Watson",
            "authors": [
                "Kennedy Ralston",
                "Yuhao Chen",
                "Haruna Isah",
                "Farhana Zulkernine"
            ],
            "abstract": "Systems powered by artificial intelligence are being developed to be more\nuser-friendly by communicating with users in a progressively human-like\nconversational way. Chatbots, also known as dialogue systems, interactive\nconversational agents, or virtual agents are an example of such systems used in\na wide variety of applications ranging from customer support in the business\ndomain to companionship in the healthcare sector. It is becoming increasingly\nimportant to develop chatbots that can best respond to the personalized needs\nof their users so that they can be as helpful to the user as possible in a real\nhuman way. This paper investigates and compares three popular existing chatbots\nAPI offerings and then propose and develop a voice interactive and multilingual\nchatbot that can effectively respond to users mood, tone, and language using\nIBM Watson Assistant, Tone Analyzer, and Language Translator. The chatbot was\nevaluated using a use case that was targeted at responding to users needs\nregarding exam stress based on university students survey data generated using\nGoogle Forms. The results of measuring the chatbot effectiveness at analyzing\nresponses regarding exam stress indicate that the chatbot responding\nappropriately to the user queries regarding how they are feeling about exams\n76.5%. The chatbot could also be adapted for use in other application areas\nsuch as student info-centers, government kiosks, and mental health support\nsystems.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2001.00473v1",
            "date": "2019-12-28 14:12:16+00:00",
            "title": "Detection of Glottal Closure Instants from Speech Signals: a Quantitative Review",
            "authors": [
                "Thomas Drugman",
                "Mark Thomas",
                "Jon Gudnason",
                "Patrick Naylor",
                "Thierry Dutoit"
            ],
            "abstract": "The pseudo-periodicity of voiced speech can be exploited in several speech\nprocessing applications. This requires however that the precise locations of\nthe Glottal Closure Instants (GCIs) are available. The focus of this paper is\nthe evaluation of automatic methods for the detection of GCIs directly from the\nspeech waveform. Five state-of-the-art GCI detection algorithms are compared\nusing six different databases with contemporaneous electroglottographic\nrecordings as ground truth, and containing many hours of speech by multiple\nspeakers. The five techniques compared are the Hilbert Envelope-based detection\n(HE), the Zero Frequency Resonator-based method (ZFR), the Dynamic Programming\nPhase Slope Algorithm (DYPSA), the Speech Event Detection using the Residual\nExcitation And a Mean-based Signal (SEDREAMS) and the Yet Another GCI Algorithm\n(YAGA). The efficacy of these methods is first evaluated on clean speech, both\nin terms of reliabililty and accuracy. Their robustness to additive noise and\nto reverberation is also assessed. A further contribution of the paper is the\nevaluation of their performance on a concrete application of speech processing:\nthe causal-anticausal decomposition of speech. It is shown that for clean\nspeech, SEDREAMS and YAGA are the best performing techniques, both in terms of\nidentification rate and accuracy. ZFR and SEDREAMS also show a superior\nrobustness to additive noise and reverberation.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00569v1",
            "date": "2019-12-31 18:33:03+00:00",
            "title": "Emergent Behaviors from Folksonomy Driven Interactions",
            "authors": [
                "Massimiliano Dal Mas"
            ],
            "abstract": "To reflect the evolving knowledge on the Web this paper considers ontologies\nbased on folksonomies according to a new concept structure called\n\"Folksodriven\" to represent folksonomies. This paper describes a research\nprogram for studying Folksodriven tags interactions leading to Folksodriven\ncluster behavior. The goal of the research is to understand the type of simple\nlocal interactions which produce complex and purposive group behaviors on\nFolksodriven tags. We describe a synthetic, bottom-up approach to studying\ngroup behavior, consisting of designing and testing a variety of social\ninteractions and cultural scenarios with Folksodriven tags. We propose a set of\nbasic interactions which can be used to structure and simplify the process of\nboth designing and analyzing emergent group behaviors. The presented behavior\nrepertories was developed and tested on a folksonomy environment.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2001.00573v1",
            "date": "2019-12-31 23:31:42+00:00",
            "title": "A Hybrid Framework for Topic Structure using Laughter Occurrences",
            "authors": [
                "Sucheta Ghosh"
            ],
            "abstract": "Conversational discourse coherence depends on both linguistic and\nparalinguistic phenomena. In this work we combine both paralinguistic and\nlinguistic knowledge into a hybrid framework through a multi-level hierarchy.\nThus it outputs the discourse-level topic structures. The laughter occurrences\nare used as paralinguistic information from the multiparty meeting transcripts\nof ICSI database. A clustering-based algorithm is proposed that chose the best\ntopic-segment cluster from two independent, optimized clusters, namely,\nhierarchical agglomerative clustering and $K$-medoids. Then it is iteratively\nhybridized with an existing lexical cohesion based Bayesian topic segmentation\nframework. The hybrid approach improves the performance of both of the\nstand-alone approaches. This leads to the brief study of interactions between\ntopic structures with discourse relational structure. This training-free topic\nstructuring approach can be applicable to online understanding of spoken\ndialogs.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2001.00840v1",
            "date": "2019-12-28 20:40:08+00:00",
            "title": "A Comparative Study of Glottal Source Estimation Techniques",
            "authors": [
                "Thomas Drugman",
                "Baris Bozkurt",
                "Thierry Dutoit"
            ],
            "abstract": "Source-tract decomposition (or glottal flow estimation) is one of the basic\nproblems of speech processing. For this, several techniques have been proposed\nin the literature. However studies comparing different approaches are almost\nnonexistent. Besides, experiments have been systematically performed either on\nsynthetic speech or on sustained vowels. In this study we compare three of the\nmain representative state-of-the-art methods of glottal flow estimation:\nclosed-phase inverse filtering, iterative and adaptive inverse filtering, and\nmixed-phase decomposition. These techniques are first submitted to an objective\nassessment test on synthetic speech signals. Their sensitivity to various\nfactors affecting the estimation quality, as well as their robustness to noise\nare studied. In a second experiment, their ability to label voice quality\n(tensed, modal, soft) is studied on a large corpus of real connected speech. It\nis shown that changes of voice quality are reflected by significant\nmodifications in glottal feature distributions. Techniques based on the\nmixed-phase decomposition and on a closed-phase inverse filtering process turn\nout to give the best results on both clean synthetic and real speech signals.\nOn the other hand, iterative and adaptive inverse filtering is recommended in\nnoisy environments for its high robustness.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00841v1",
            "date": "2019-12-28 19:27:45+00:00",
            "title": "Glottal Closure and Opening Instant Detection from Speech Signals",
            "authors": [
                "Thomas Drugman",
                "Thierry Dutoit"
            ],
            "abstract": "This paper proposes a new procedure to detect Glottal Closure and Opening\nInstants (GCIs and GOIs) directly from speech waveforms. The procedure is\ndivided into two successive steps. First a mean-based signal is computed, and\nintervals where speech events are expected to occur are extracted from it.\nSecondly, at each interval a precise position of the speech event is assigned\nby locating a discontinuity in the Linear Prediction residual. The proposed\nmethod is compared to the DYPSA algorithm on the CMU ARCTIC database. A\nsignificant improvement as well as a better noise robustness are reported.\nBesides, results of GOI identification accuracy are promising for the glottal\nsource characterization.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00842v1",
            "date": "2019-12-29 07:26:47+00:00",
            "title": "A Deterministic plus Stochastic Model of the Residual Signal for Improved Parametric Speech Synthesis",
            "authors": [
                "Thomas Drugman",
                "Geoffrey Wilfart",
                "Thierry Dutoit"
            ],
            "abstract": "Speech generated by parametric synthesizers generally suffers from a typical\nbuzziness, similar to what was encountered in old LPC-like vocoders. In order\nto alleviate this problem, a more suited modeling of the excitation should be\nadopted. For this, we hereby propose an adaptation of the Deterministic plus\nStochastic Model (DSM) for the residual. In this model, the excitation is\ndivided into two distinct spectral bands delimited by the maximum voiced\nfrequency. The deterministic part concerns the low-frequency contents and\nconsists of a decomposition of pitch-synchronous residual frames on an\northonormal basis obtained by Principal Component Analysis. The stochastic\ncomponent is a high-pass filtered noise whose time structure is modulated by an\nenergy-envelope, similarly to what is done in the Harmonic plus Noise Model\n(HNM). The proposed residual model is integrated within a HMM-based speech\nsynthesizer and is compared to the traditional excitation through a subjective\ntest. Results show a significative improvement for both male and female voices.\nIn addition the proposed model requires few computational load and memory,\nwhich is essential for its integration in commercial applications.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.01000v1",
            "date": "2019-12-29 07:52:37+00:00",
            "title": "The Deterministic plus Stochastic Model of the Residual Signal and its Applications",
            "authors": [
                "Thomas Drugman",
                "Thierry Dutoit"
            ],
            "abstract": "The modeling of speech production often relies on a source-filter approach.\nAlthough methods parameterizing the filter have nowadays reached a certain\nmaturity, there is still a lot to be gained for several speech processing\napplications in finding an appropriate excitation model. This manuscript\npresents a Deterministic plus Stochastic Model (DSM) of the residual signal.\nThe DSM consists of two contributions acting in two distinct spectral bands\ndelimited by a maximum voiced frequency. Both components are extracted from an\nanalysis performed on a speaker-dependent dataset of pitch-synchronous residual\nframes. The deterministic part models the low-frequency contents and arises\nfrom an orthonormal decomposition of these frames. As for the stochastic\ncomponent, it is a high-frequency noise modulated both in time and frequency.\nSome interesting phonetic and computational properties of the DSM are also\nhighlighted. The applicability of the DSM in two fields of speech processing is\nthen studied. First, it is shown that incorporating the DSM vocoder in\nHMM-based speech synthesis enhances the delivered quality. The proposed\napproach turns out to significantly outperform the traditional pulse excitation\nand provides a quality equivalent to STRAIGHT. In a second application, the\npotential of glottal signatures derived from the proposed DSM is investigated\nfor speaker identification purpose. Interestingly, these signatures are shown\nto lead to better recognition rates than other glottal-based methods.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.04345v1",
            "date": "2019-12-20 22:12:47+00:00",
            "title": "Shareable Representations for Search Query Understanding",
            "authors": [
                "Mukul Kumar",
                "Youna Hu",
                "Will Headden",
                "Rahul Goutam",
                "Heran Lin",
                "Bing Yin"
            ],
            "abstract": "Understanding search queries is critical for shopping search engines to\ndeliver a satisfying customer experience. Popular shopping search engines\nreceive billions of unique queries yearly, each of which can depict any of\nhundreds of user preferences or intents. In order to get the right results to\ncustomers it must be known queries like \"inexpensive prom dresses\" are intended\nto not only surface results of a certain product type but also products with a\nlow price. Referred to as query intents, examples also include preferences for\nauthor, brand, age group, or simply a need for customer service. Recent works\nsuch as BERT have demonstrated the success of a large transformer encoder\narchitecture with language model pre-training on a variety of NLP tasks. We\nadapt such an architecture to learn intents for search queries and describe\nmethods to account for the noisiness and sparseness of search query data. We\nalso describe cost effective ways of hosting transformer encoder models in\ncontext with low latency requirements. With the right domain-specific training\nwe can build a shareable deep learning model whose internal representation can\nbe reused for a variety of query understanding tasks including query intent\nidentification. Model sharing allows for fewer large models needed to be served\nat inference time and provides a platform to quickly build and roll out new\nsearch query classifiers.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1902.01385v1",
            "date": "2019-02-04 18:53:14+00:00",
            "title": "Embodied Multimodal Multitask Learning",
            "authors": [
                "Devendra Singh Chaplot",
                "Lisa Lee",
                "Ruslan Salakhutdinov",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "Recent efforts on training visual navigation agents conditioned on language\nusing deep reinforcement learning have been successful in learning policies for\ndifferent multimodal tasks, such as semantic goal navigation and embodied\nquestion answering. In this paper, we propose a multitask model capable of\njointly learning these multimodal tasks, and transferring knowledge of words\nand their grounding in visual objects across the tasks. The proposed model uses\na novel Dual-Attention unit to disentangle the knowledge of words in the\ntextual representations and visual concepts in the visual representations, and\nalign them with each other. This disentangled task-invariant alignment of\nrepresentations facilitates grounding and knowledge transfer across both tasks.\nWe show that the proposed model outperforms a range of baselines on both tasks\nin simulated 3D environments. We also show that this disentanglement of\nrepresentations makes our model modular, interpretable, and allows for transfer\nto instructions containing new words by leveraging object detectors.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.02263v1",
            "date": "2019-02-06 16:28:26+00:00",
            "title": "Unsupervised Polyglot Text To Speech",
            "authors": [
                "Eliya Nachmani",
                "Lior Wolf"
            ],
            "abstract": "We present a TTS neural network that is able to produce speech in multiple\nlanguages. The proposed network is able to transfer a voice, which was\npresented as a sample in a source language, into one of several target\nlanguages. Training is done without using matching or parallel data, i.e.,\nwithout samples of the same speaker in multiple languages, making the method\nmuch more applicable. The conversion is based on learning a polyglot network\nthat has multiple per-language sub-networks and adding loss terms that preserve\nthe speaker's identity in multiple languages. We evaluate the proposed polyglot\nneural network for three languages with a total of more than 400 speakers and\ndemonstrate convincing conversion capabilities.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1902.07178v1",
            "date": "2019-02-19 18:18:59+00:00",
            "title": "A spelling correction model for end-to-end speech recognition",
            "authors": [
                "Jinxi Guo",
                "Tara N. Sainath",
                "Ron J. Weiss"
            ],
            "abstract": "Attention-based sequence-to-sequence models for speech recognition jointly\ntrain an acoustic model, language model (LM), and alignment mechanism using a\nsingle neural network and require only parallel audio-text pairs. Thus, the\nlanguage model component of the end-to-end model is only trained on transcribed\naudio-text pairs, which leads to performance degradation especially on rare\nwords. While there have been a variety of work that look at incorporating an\nexternal LM trained on text-only data into the end-to-end framework, none of\nthem have taken into account the characteristic error distribution made by the\nmodel. In this paper, we propose a novel approach to utilizing text-only data,\nby training a spelling correction (SC) model to explicitly correct those\nerrors. On the LibriSpeech dataset, we demonstrate that the proposed model\nresults in an 18.6% relative improvement in WER over the baseline model when\ndirectly correcting top ASR hypothesis, and a 29.0% relative improvement when\nfurther rescoring an expanded n-best list using an external LM.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1902.09705v1",
            "date": "2019-02-26 02:14:10+00:00",
            "title": "Beyond the Self: Using Grounded Affordances to Interpret and Describe Others' Actions",
            "authors": [
                "Giovanni Saponaro",
                "Lorenzo Jamone",
                "Alexandre Bernardino",
                "Giampiero Salvi"
            ],
            "abstract": "We propose a developmental approach that allows a robot to interpret and\ndescribe the actions of human agents by reusing previous experience. The robot\nfirst learns the association between words and object affordances by\nmanipulating the objects in its environment. It then uses this information to\nlearn a mapping between its own actions and those performed by a human in a\nshared environment. It finally fuses the information from these two models to\ninterpret and describe human actions in light of its own experience. In our\nexperiments, we show that the model can be used flexibly to do inference on\ndifferent aspects of the scene. We can predict the effects of an action on the\nbasis of object properties. We can revise the belief that a certain action\noccurred, given the observed effects of the human action. In an early action\nrecognition fashion, we can anticipate the effects when the action has only\nbeen partially observed. By estimating the probability of words given the\nevidence and feeding them into a pre-defined grammar, we can generate relevant\ndescriptions of the scene. We believe that this is a step towards providing\nrobots with the fundamental skills to engage in social collaboration with\nhumans.",
            "journal ref": "IEEE Transactions on Cognitive and Developmental Systems, vol. 12,\n  no. 2, pp. 209-221, June 2020",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1901.02348v3",
            "date": "2019-01-05 06:22:40+00:00",
            "title": "Improving noise robustness of automatic speech recognition via parallel data and teacher-student learning",
            "authors": [
                "Ladislav Mo\u0161ner",
                "Minhua Wu",
                "Anirudh Raju",
                "Sree Hari Krishnan Parthasarathi",
                "Kenichi Kumatani",
                "Shiva Sundaram",
                "Roland Maas",
                "Bj\u00f6rn Hoffmeister"
            ],
            "abstract": "For real-world speech recognition applications, noise robustness is still a\nchallenge. In this work, we adopt the teacher-student (T/S) learning technique\nusing a parallel clean and noisy corpus for improving automatic speech\nrecognition (ASR) performance under multimedia noise. On top of that, we apply\na logits selection method which only preserves the k highest values to prevent\nwrong emphasis of knowledge from the teacher and to reduce bandwidth needed for\ntransferring data. We incorporate up to 8000 hours of untranscribed data for\ntraining and present our results on sequence trained models apart from cross\nentropy trained ones. The best sequence trained student model yields relative\nword error rate (WER) reductions of approximately 10.1%, 28.7% and 19.6% on our\nclean, simulated noisy and real test sets respectively comparing to a sequence\ntrained teacher.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1901.03860v1",
            "date": "2019-01-12 13:20:54+00:00",
            "title": "Prototypical Metric Transfer Learning for Continuous Speech Keyword Spotting With Limited Training Data",
            "authors": [
                "Harshita Seth",
                "Pulkit Kumar",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "Continuous Speech Keyword Spotting (CSKS) is the problem of spotting keywords\nin recorded conversations, when a small number of instances of keywords are\navailable in training data. Unlike the more common Keyword Spotting, where an\nalgorithm needs to detect lone keywords or short phrases like \"Alexa\",\n\"Cortana\", \"Hi Alexa!\", \"Whatsup Octavia?\" etc. in speech, CSKS needs to filter\nout embedded words from a continuous flow of speech, ie. spot \"Anna\" and\n\"github\" in \"I know a developer named Anna who can look into this github\nissue.\" Apart from the issue of limited training data availability, CSKS is an\nextremely imbalanced classification problem. We address the limitations of\nsimple keyword spotting baselines for both aforementioned challenges by using a\nnovel combination of loss functions (Prototypical networks' loss and metric\nloss) and transfer learning. Our method improves F1 score by over 10%.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1901.04911v1",
            "date": "2019-01-15 16:24:07+00:00",
            "title": "Unconstrained Church-Turing thesis cannot possibly be true",
            "authors": [
                "Yuri Gurevich"
            ],
            "abstract": "The Church-Turing thesis asserts that if a partial strings-to-strings\nfunction is effectively computable then it is computable by a Turing machine.\n  In the 1930s, when Church and Turing worked on their versions of the thesis,\nthere was a robust notion of algorithm. These traditional algorithms are known\nalso as classical or sequential. In the original thesis, effectively computable\nmeant computable by an effective classical algorithm. Based on an earlier\naxiomatization of classical algorithms, the original thesis was proven in 2008.\n  Since the 1930s, the notion of algorithm has changed dramatically. New\nspecies of algorithms have been and are being introduced. We argue that the\ngeneralization of the original thesis, where effectively computable means\ncomputable by an effective algorithm of any species, cannot possibly be true.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1901.05389v1",
            "date": "2019-01-16 16:56:14+00:00",
            "title": "Location, Occupation, and Semantics based Socioeconomic Status Inference on Twitter",
            "authors": [
                "Jacobo Levy Abitbol",
                "M\u00e1rton Karsai",
                "Eric Fleury"
            ],
            "abstract": "The socioeconomic status of people depends on a combination of individual\ncharacteristics and environmental variables, thus its inference from online\nbehavioral data is a difficult task. Attributes like user semantics in\ncommunication, habitat, occupation, or social network are all known to be\ndeterminant predictors of this feature. In this paper we propose three\ndifferent data collection and combination methods to first estimate and, in\nturn, infer the socioeconomic status of French Twitter users from their online\nsemantics. Our methods are based on open census data, crawled professional\nprofiles, and remotely sensed, expert annotated information on living\nenvironment. Our inference models reach similar performance of earlier results\nwith the advantage of relying on broadly available datasets and of providing a\ngeneralizable framework to estimate socioeconomic status of large numbers of\nTwitter users. These results may contribute to the scientific discussion on\nsocial stratification and inequalities, and may fuel several applications.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1901.05415v4",
            "date": "2019-01-16 18:02:44+00:00",
            "title": "Learning from Dialogue after Deployment: Feed Yourself, Chatbot!",
            "authors": [
                "Braden Hancock",
                "Antoine Bordes",
                "Pierre-Emmanuel Mazar\u00e9",
                "Jason Weston"
            ],
            "abstract": "The majority of conversations a dialogue agent sees over its lifetime occur\nafter it has already been trained and deployed, leaving a vast store of\npotential training signal untapped. In this work, we propose the self-feeding\nchatbot, a dialogue agent with the ability to extract new training examples\nfrom the conversations it participates in. As our agent engages in\nconversation, it also estimates user satisfaction in its responses. When the\nconversation appears to be going well, the user's responses become new training\nexamples to imitate. When the agent believes it has made a mistake, it asks for\nfeedback; learning to predict the feedback that will be given improves the\nchatbot's dialogue abilities further. On the PersonaChat chit-chat dataset with\nover 131k training examples, we find that learning from dialogue with a\nself-feeding chatbot significantly improves performance, regardless of the\namount of traditional supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.07878v1",
            "date": "2019-01-23 13:42:02+00:00",
            "title": "\"Is this an example image?\" -- Predicting the Relative Abstractness Level of Image and Text",
            "authors": [
                "Christian Otto",
                "Sebastian Holzki",
                "Ralph Ewerth"
            ],
            "abstract": "Successful multimodal search and retrieval requires the automatic\nunderstanding of semantic cross-modal relations, which, however, is still an\nopen research problem. Previous work has suggested the metrics cross-modal\nmutual information and semantic correlation to model and predict cross-modal\nsemantic relations of image and text. In this paper, we present an approach to\npredict the (cross-modal) relative abstractness level of a given image-text\npair, that is whether the image is an abstraction of the text or vice versa.\nFor this purpose, we introduce a new metric that captures this specific\nrelationship between image and text at the Abstractness Level (ABS). We present\na deep learning approach to predict this metric, which relies on an autoencoder\narchitecture that allows us to significantly reduce the required amount of\nlabeled training data. A comprehensive set of publicly available scientific\ndocuments has been gathered. Experimental results on a challenging test set\ndemonstrate the feasibility of the approach.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.02172v1",
            "date": "2019-03-06 04:49:07+00:00",
            "title": "AAAI-2019 Workshop on Games and Simulations for Artificial Intelligence",
            "authors": [
                "Marwan Mattar",
                "Roozbeh Mottaghi",
                "Julian Togelius",
                "Danny Lange"
            ],
            "abstract": "This volume represents the accepted submissions from the AAAI-2019 Workshop\non Games and Simulations for Artificial Intelligence held on January 29, 2019\nin Honolulu, Hawaii, USA. https://www.gamesim.ai",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1903.02547v2",
            "date": "2019-03-06 18:54:55+00:00",
            "title": "Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation",
            "authors": [
                "Liyiming Ke",
                "Xiujun Li",
                "Yonatan Bisk",
                "Ari Holtzman",
                "Zhe Gan",
                "Jingjing Liu",
                "Jianfeng Gao",
                "Yejin Choi",
                "Siddhartha Srinivasa"
            ],
            "abstract": "We present the Frontier Aware Search with backTracking (FAST) Navigator, a\ngeneral framework for action decoding, that achieves state-of-the-art results\non the Room-to-Room (R2R) Vision-and-Language navigation challenge of Anderson\net. al. (2018). Given a natural language instruction and photo-realistic image\nviews of a previously unseen environment, the agent was tasked with navigating\nfrom source to target location as quickly as possible. While all current\napproaches make local action decisions or score entire trajectories using beam\nsearch, ours balances local and global signals when exploring an unobserved\nenvironment. Importantly, this lets us act greedily but use global signals to\nbacktrack when necessary. Applying FAST framework to existing state-of-the-art\nmodels achieved a 17% relative gain, an absolute 6% gain on Success rate\nweighted by Path Length (SPL).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.07395v1",
            "date": "2019-03-13 18:43:31+00:00",
            "title": "Voice command generation using Progressive Wavegans",
            "authors": [
                "Thomas Wiest",
                "Nicholas Cummins",
                "Alice Baird",
                "Simone Hantke",
                "Judith Dineley",
                "Bj\u00f6rn Schuller"
            ],
            "abstract": "Generative Adversarial Networks (GANs) have become exceedingly popular in a\nwide range of data-driven research fields, due in part to their success in\nimage generation. Their ability to generate new samples, often from only a\nsmall amount of input data, makes them an exciting research tool in areas with\nlimited data resources. One less-explored application of GANs is the synthesis\nof speech and audio samples. Herein, we propose a set of extensions to the\nWaveGAN paradigm, a recently proposed approach for sound generation using GANs.\nThe aim of these extensions - preprocessing, Audio-to-Audio generation, skip\nconnections and progressive structures - is to improve the human likeness of\nsynthetic speech samples. Scores from listening tests with 30 volunteers\ndemonstrated a moderate improvement (Cohen's d coefficient of 0.65) in human\nlikeness using the proposed extensions compared to the original WaveGAN\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1903.08756v1",
            "date": "2019-03-20 21:52:13+00:00",
            "title": "Distributed Vector Representations of Folksong Motifs",
            "authors": [
                "Aitor Arronte-Alvarez",
                "Francisco G\u00f3mez-Martin"
            ],
            "abstract": "This article presents a distributed vector representation model for learning\nfolksong motifs. A skip-gram version of word2vec with negative sampling is used\nto represent high quality embeddings. Motifs from the Essen Folksong collection\nare compared based on their cosine similarity. A new evaluation method for\ntesting the quality of the embeddings based on a melodic similarity task is\npresented to show how the vector space can represent complex contextual\nfeatures, and how it can be utilized for the study of folksong variation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/1903.10145v3",
            "date": "2019-03-25 06:28:24+00:00",
            "title": "Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing",
            "authors": [
                "Hao Fu",
                "Chunyuan Li",
                "Xiaodong Liu",
                "Jianfeng Gao",
                "Asli Celikyilmaz",
                "Lawrence Carin"
            ],
            "abstract": "Variational autoencoders (VAEs) with an auto-regressive decoder have been\napplied for many natural language processing (NLP) tasks. The VAE objective\nconsists of two terms, (i) reconstruction and (ii) KL regularization, balanced\nby a weighting hyper-parameter \\beta. One notorious training difficulty is that\nthe KL term tends to vanish. In this paper we study scheduling schemes for\n\\beta, and show that KL vanishing is caused by the lack of good latent codes in\ntraining the decoder at the beginning of optimization. To remedy this, we\npropose a cyclical annealing schedule, which repeats the process of increasing\n\\beta multiple times. This new procedure allows the progressive learning of\nmore meaningful latent codes, by leveraging the informative representations of\nprevious cycles as warm re-starts. The effectiveness of cyclical annealing is\nvalidated on a broad range of NLP tasks, including language modeling, dialog\nresponse generation and unsupervised language pre-training.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1903.11570v1",
            "date": "2019-03-27 17:33:33+00:00",
            "title": "Visualization and Interpretation of Latent Spaces for Controlling Expressive Speech Synthesis through Audio Analysis",
            "authors": [
                "No\u00e9 Tits",
                "Fengna Wang",
                "Kevin El Haddad",
                "Vincent Pagel",
                "Thierry Dutoit"
            ],
            "abstract": "The field of Text-to-Speech has experienced huge improvements last years\nbenefiting from deep learning techniques. Producing realistic speech becomes\npossible now. As a consequence, the research on the control of the\nexpressiveness, allowing to generate speech in different styles or manners, has\nattracted increasing attention lately. Systems able to control style have been\ndeveloped and show impressive results. However the control parameters often\nconsist of latent variables and remain complex to interpret. In this paper, we\nanalyze and compare different latent spaces and obtain an interpretation of\ntheir influence on expressive speech. This will enable the possibility to build\ncontrollable speech synthesis systems with an understandable behaviour.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.01201v2",
            "date": "2019-04-02 03:52:27+00:00",
            "title": "Habitat: A Platform for Embodied AI Research",
            "authors": [
                "Manolis Savva",
                "Abhishek Kadian",
                "Oleksandr Maksymets",
                "Yili Zhao",
                "Erik Wijmans",
                "Bhavana Jain",
                "Julian Straub",
                "Jia Liu",
                "Vladlen Koltun",
                "Jitendra Malik",
                "Devi Parikh",
                "Dhruv Batra"
            ],
            "abstract": "We present Habitat, a platform for research in embodied artificial\nintelligence (AI). Habitat enables training embodied agents (virtual robots) in\nhighly efficient photorealistic 3D simulation. Specifically, Habitat consists\nof: (i) Habitat-Sim: a flexible, high-performance 3D simulator with\nconfigurable agents, sensors, and generic 3D dataset handling. Habitat-Sim is\nfast -- when rendering a scene from Matterport3D, it achieves several thousand\nframes per second (fps) running single-threaded, and can reach over 10,000 fps\nmulti-process on a single GPU. (ii) Habitat-API: a modular high-level library\nfor end-to-end development of embodied AI algorithms -- defining tasks (e.g.,\nnavigation, instruction following, question answering), configuring, training,\nand benchmarking embodied agents.\n  These large-scale engineering contributions enable us to answer scientific\nquestions requiring experiments that were till now impracticable or 'merely'\nimpractical. Specifically, in the context of point-goal navigation: (1) we\nrevisit the comparison between learning and SLAM approaches from two recent\nworks and find evidence for the opposite conclusion -- that learning\noutperforms SLAM if scaled to an order of magnitude more experience than\nprevious investigations, and (2) we conduct the first cross-dataset\ngeneralization experiments {train, test} x {Matterport3D, Gibson} for multiple\nsensors {blind, RGB, RGBD, D} and find that only agents with depth (D) sensors\ngeneralize across datasets. We hope that our open-source platform and these\nfindings will advance research in embodied AI.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.04358v1",
            "date": "2019-04-08 21:11:40+00:00",
            "title": "Deep Learning the EEG Manifold for Phonological Categorization from Active Thoughts",
            "authors": [
                "Pramit Saha",
                "Muhammad Abdul-Mageed",
                "Sidney Fels"
            ],
            "abstract": "Speech-related Brain Computer Interfaces (BCI) aim primarily at finding an\nalternative vocal communication pathway for people with speaking disabilities.\nAs a step towards full decoding of imagined speech from active thoughts, we\npresent a BCI system for subject-independent classification of phonological\ncategories exploiting a novel deep learning based hierarchical feature\nextraction scheme. To better capture the complex representation of\nhigh-dimensional electroencephalography (EEG) data, we compute the joint\nvariability of EEG electrodes into a channel cross-covariance matrix. We then\nextract the spatio-temporal information encoded within the matrix using a mixed\ndeep neural network strategy. Our model framework is composed of a\nconvolutional neural network (CNN), a long-short term network (LSTM), and a\ndeep autoencoder. We train the individual networks hierarchically, feeding\ntheir combined outputs in a final gradient boosting classification step. Our\nbest models achieve an average accuracy of 77.9% across five different binary\nclassification tasks, providing a significant 22.5% improvement over previous\nmethods. As we also show visually, our work demonstrates that the speech\nimagery EEG possesses significant discriminative information about the intended\narticulatory movements responsible for natural speech synthesis.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.04956v1",
            "date": "2019-04-10 01:00:26+00:00",
            "title": "Distributed Deep Learning Strategies For Automatic Speech Recognition",
            "authors": [
                "Wei Zhang",
                "Xiaodong Cui",
                "Ulrich Finkler",
                "Brian Kingsbury",
                "George Saon",
                "David Kung",
                "Michael Picheny"
            ],
            "abstract": "In this paper, we propose and investigate a variety of distributed deep\nlearning strategies for automatic speech recognition (ASR) and evaluate them\nwith a state-of-the-art Long short-term memory (LSTM) acoustic model on the\n2000-hour Switchboard (SWB2000), which is one of the most widely used datasets\nfor ASR performance benchmark. We first investigate what are the proper\nhyper-parameters (e.g., learning rate) to enable the training with sufficiently\nlarge batch size without impairing the model accuracy. We then implement\nvarious distributed strategies, including Synchronous (SYNC), Asynchronous\nDecentralized Parallel SGD (ADPSGD) and the hybrid of the two HYBRID, to study\ntheir runtime/accuracy trade-off. We show that we can train the LSTM model\nusing ADPSGD in 14 hours with 16 NVIDIA P100 GPUs to reach a 7.6% WER on the\nHub5- 2000 Switchboard (SWB) test set and a 13.1% WER on the CallHome (CH) test\nset. Furthermore, we can train the model using HYBRID in 11.5 hours with 32\nNVIDIA V100 GPUs without loss in accuracy.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.05440v1",
            "date": "2019-04-10 21:04:54+00:00",
            "title": "Generating Animations from Screenplays",
            "authors": [
                "Yeyao Zhang",
                "Eleftheria Tsipidi",
                "Sasha Schriber",
                "Mubbasir Kapadia",
                "Markus Gross",
                "Ashutosh Modi"
            ],
            "abstract": "Automatically generating animation from natural language text finds\napplication in a number of areas e.g. movie script writing, instructional\nvideos, and public safety. However, translating natural language text into\nanimation is a challenging task. Existing text-to-animation systems can handle\nonly very simple sentences, which limits their applications. In this paper, we\ndevelop a text-to-animation system which is capable of handling complex\nsentences. We achieve this by introducing a text simplification step into the\nprocess. Building on an existing animation generation system for screenwriting,\nwe create a robust NLP pipeline to extract information from screenplays and map\nthem to the system's knowledge base. We develop a set of linguistic\ntransformation rules that simplify complex sentences. Information extracted\nfrom the simplified sentences is used to generate a rough storyboard and video\ndepicting the text. Our sentence simplification module outperforms existing\nsystems in terms of BLEU and SARI metrics.We further evaluated our system via a\nuser study: 68 % participants believe that our system generates reasonable\nanimation from input screenplays.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.05746v1",
            "date": "2019-04-08 21:41:54+00:00",
            "title": "SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning",
            "authors": [
                "Pramit Saha",
                "Muhammad Abdul-Mageed",
                "Sidney Fels"
            ],
            "abstract": "Speech-related Brain Computer Interface (BCI) technologies provide effective\nvocal communication strategies for controlling devices through speech commands\ninterpreted from brain signals. In order to infer imagined speech from active\nthoughts, we propose a novel hierarchical deep learning BCI system for\nsubject-independent classification of 11 speech tokens including phonemes and\nwords. Our novel approach exploits predicted articulatory information of six\nphonological categories (e.g., nasal, bilabial) as an intermediate step for\nclassifying the phonemes and words, thereby finding discriminative signal\nresponsible for natural speech synthesis. The proposed network is composed of\nhierarchical combination of spatial and temporal CNN cascaded with a deep\nautoencoder. Our best models on the KARA database achieve an average accuracy\nof 83.42% across the six different binary phonological classification tasks,\nand 53.36% for the individual token identification task, significantly\noutperforming our baselines. Ultimately, our work suggests the possible\nexistence of a brain imagery footprint for the underlying articulatory movement\nrelated to different sounds that can be used to aid imagined speech decoding.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.05880v3",
            "date": "2019-04-11 17:59:58+00:00",
            "title": "Factor Graph Attention",
            "authors": [
                "Idan Schwartz",
                "Seunghak Yu",
                "Tamir Hazan",
                "Alexander Schwing"
            ],
            "abstract": "Dialog is an effective way to exchange information, but subtle details and\nnuances are extremely important. While significant progress has paved a path to\naddress visual dialog with algorithms, details and nuances remain a challenge.\nAttention mechanisms have demonstrated compelling results to extract details in\nvisual question answering and also provide a convincing framework for visual\ndialog due to their interpretability and effectiveness. However, the many data\nutilities that accompany visual dialog challenge existing attention techniques.\nWe address this issue and develop a general attention mechanism for visual\ndialog which operates on any number of data utilities. To this end, we design a\nfactor graph based attention mechanism which combines any number of utility\nrepresentations. We illustrate the applicability of the proposed approach on\nthe challenging and recently introduced VisDial datasets, outperforming recent\nstate-of-the-art methods by 1.1% for VisDial0.9 and by 2% for VisDial1.0 on\nMRR. Our ensemble model improved the MRR score on VisDial1.0 by more than 6%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.08067v5",
            "date": "2019-04-17 03:29:05+00:00",
            "title": "Text Classification Algorithms: A Survey",
            "authors": [
                "Kamran Kowsari",
                "Kiana Jafari Meimandi",
                "Mojtaba Heidarysafa",
                "Sanjana Mendu",
                "Laura E. Barnes",
                "Donald E. Brown"
            ],
            "abstract": "In recent years, there has been an exponential growth in the number of\ncomplex documents and texts that require a deeper understanding of machine\nlearning methods to be able to accurately classify texts in many applications.\nMany machine learning approaches have achieved surpassing results in natural\nlanguage processing. The success of these learning algorithms relies on their\ncapacity to understand complex models and non-linear relationships within data.\nHowever, finding suitable structures, architectures, and techniques for text\nclassification is a challenge for researchers. In this paper, a brief overview\nof text classification algorithms is discussed. This overview covers different\ntext feature extractions, dimensionality reduction methods, existing algorithms\nand techniques, and evaluations methods. Finally, the limitations of each\ntechnique and their application in the real-world problem are discussed.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.08504v1",
            "date": "2019-04-09 12:19:09+00:00",
            "title": "Exploring Uncertainty Measures for Image-Caption Embedding-and-Retrieval Task",
            "authors": [
                "Kenta Hama",
                "Takashi Matsubara",
                "Kuniaki Uehara",
                "Jianfei Cai"
            ],
            "abstract": "With the wide development of black-box machine learning algorithms,\nparticularly deep neural network (DNN), the practical demand for the\nreliability assessment is rapidly rising. On the basis of the concept that\n`Bayesian deep learning knows what it does not know,' the uncertainty of DNN\noutputs has been investigated as a reliability measure for the classification\nand regression tasks. However, in the image-caption retrieval task, well-known\nsamples are not always easy-to-retrieve samples. This study investigates two\naspects of image-caption embedding-and-retrieval systems. On one hand, we\nquantify feature uncertainty by considering image-caption embedding as a\nregression task, and use it for model averaging, which can improve the\nretrieval performance. On the other hand, we further quantify posterior\nuncertainty by considering the retrieval as a classification task, and use it\nas a reliability measure, which can greatly improve the retrieval performance\nby rejecting uncertain queries. The consistent performance of two uncertainty\nmeasures is observed with different datasets (MS COCO and Flickr30k), different\ndeep learning architectures (dropout and batch normalization), and different\nsimilarity functions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1904.08779v3",
            "date": "2019-04-18 17:53:38+00:00",
            "title": "SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition",
            "authors": [
                "Daniel S. Park",
                "William Chan",
                "Yu Zhang",
                "Chung-Cheng Chiu",
                "Barret Zoph",
                "Ekin D. Cubuk",
                "Quoc V. Le"
            ],
            "abstract": "We present SpecAugment, a simple data augmentation method for speech\nrecognition. SpecAugment is applied directly to the feature inputs of a neural\nnetwork (i.e., filter bank coefficients). The augmentation policy consists of\nwarping the features, masking blocks of frequency channels, and masking blocks\nof time steps. We apply SpecAugment on Listen, Attend and Spell networks for\nend-to-end speech recognition tasks. We achieve state-of-the-art performance on\nthe LibriSpeech 960h and Swichboard 300h tasks, outperforming all prior work.\nOn LibriSpeech, we achieve 6.8% WER on test-other without the use of a language\nmodel, and 5.8% WER with shallow fusion with a language model. This compares to\nthe previous state-of-the-art hybrid system of 7.5% WER. For Switchboard, we\nachieve 7.2%/14.6% on the Switchboard/CallHome portion of the Hub5'00 test set\nwithout the use of a language model, and 6.8%/14.1% with shallow fusion, which\ncompares to the previous state-of-the-art hybrid system at 8.3%/17.3% WER.",
            "journal ref": "Proc. Interspeech 2019, 2613-2617",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.09317v2",
            "date": "2019-04-19 19:04:12+00:00",
            "title": "Challenges and Prospects in Vision and Language Research",
            "authors": [
                "Kushal Kafle",
                "Robik Shrestha",
                "Christopher Kanan"
            ],
            "abstract": "Language grounded image understanding tasks have often been proposed as a\nmethod for evaluating progress in artificial intelligence. Ideally, these tasks\nshould test a plethora of capabilities that integrate computer vision,\nreasoning, and natural language understanding. However, rather than behaving as\nvisual Turing tests, recent studies have demonstrated state-of-the-art systems\nare achieving good performance through flaws in datasets and evaluation\nprocedures. We review the current state of affairs and outline a path forward.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.10500v1",
            "date": "2019-04-23 19:13:51+00:00",
            "title": "Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances",
            "authors": [
                "Eda Okur",
                "Shachi H Kumar",
                "Saurav Sahay",
                "Asli Arslan Esme",
                "Lama Nachman"
            ],
            "abstract": "Understanding passenger intents and extracting relevant slots are important\nbuilding blocks towards developing contextual dialogue systems for natural\ninteractions in autonomous vehicles (AV). In this work, we explored AMIE\n(Automated-vehicle Multi-modal In-cabin Experience), the in-cabin agent\nresponsible for handling certain passenger-vehicle interactions. When the\npassengers give instructions to AMIE, the agent should parse such commands\nproperly and trigger the appropriate functionality of the AV system. In our\ncurrent explorations, we focused on AMIE scenarios describing usages around\nsetting or changing the destination and route, updating driving behavior or\nspeed, finishing the trip and other use-cases to support various natural\ncommands. We collected a multi-modal in-cabin dataset with multi-turn dialogues\nbetween the passengers and AMIE using a Wizard-of-Oz scheme via a realistic\nscavenger hunt game activity. After exploring various recent Recurrent Neural\nNetworks (RNN) based techniques, we introduced our own hierarchical joint\nmodels to recognize passenger intents along with relevant slots associated with\nthe action to be performed in AV scenarios. Our experimental results\noutperformed certain competitive baselines and achieved overall F1 scores of\n0.91 for utterance-level intent detection and 0.96 for slot filling tasks. In\naddition, we conducted initial speech-to-text explorations by comparing\nintent/slot models trained and tested on human transcriptions versus noisy\nAutomatic Speech Recognition (ASR) outputs. Finally, we compared the results\nwith single passenger rides versus the rides with multiple passengers.",
            "journal ref": "Springer LNCS Proceedings for CICLing 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.10788v2",
            "date": "2019-04-23 13:09:21+00:00",
            "title": "Speech Emotion Recognition Using Multi-hop Attention Mechanism",
            "authors": [
                "Seunghyun Yoon",
                "Seokhyun Byun",
                "Subhadeep Dey",
                "Kyomin Jung"
            ],
            "abstract": "In this paper, we are interested in exploiting textual and acoustic data of\nan utterance for the speech emotion classification task. The baseline approach\nmodels the information from audio and text independently using two deep neural\nnetworks (DNNs). The outputs from both the DNNs are then fused for\nclassification. As opposed to using knowledge from both the modalities\nseparately, we propose a framework to exploit acoustic information in tandem\nwith lexical data. The proposed framework uses two bi-directional long\nshort-term memory (BLSTM) for obtaining hidden representations of the\nutterance. Furthermore, we propose an attention mechanism, referred to as the\nmulti-hop, which is trained to automatically infer the correlation between the\nmodalities. The multi-hop attention first computes the relevant segments of the\ntextual data corresponding to the audio signal. The relevant textual data is\nthen applied to attend parts of the audio signal. To evaluate the performance\nof the proposed system, experiments are performed in the IEMOCAP dataset.\nExperimental results show that the proposed technique outperforms the\nstate-of-the-art system by 6.5% relative improvement in terms of weighted\naccuracy.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.12399v1",
            "date": "2019-04-28 23:43:20+00:00",
            "title": "Conditional Teacher-Student Learning",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yong Zhao",
                "Yifan Gong"
            ],
            "abstract": "The teacher-student (T/S) learning has been shown to be effective for a\nvariety of problems such as domain adaptation and model compression. One\nshortcoming of the T/S learning is that a teacher model, not always perfect,\nsporadically produces wrong guidance in form of posterior probabilities that\nmisleads the student model towards a suboptimal performance. To overcome this\nproblem, we propose a conditional T/S learning scheme, in which a \"smart\"\nstudent model selectively chooses to learn from either the teacher model or the\nground truth labels conditioned on whether the teacher can correctly predict\nthe ground truth. Unlike a naive linear combination of the two knowledge\nsources, the conditional learning is exclusively engaged with the teacher model\nwhen the teacher model's prediction is correct, and otherwise backs off to the\nground truth. Thus, the student model is able to learn effectively from the\nteacher and even potentially surpass the teacher. We examine the proposed\nlearning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker\nadaptation on Microsoft short message dictation dataset. The proposed method\nachieves 9.8% and 12.8% relative word error rate reductions, respectively, over\nT/S learning for environment adaptation and speaker-independent model for\nspeaker adaptation.",
            "journal ref": "2019 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Brighton, UK",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.12400v1",
            "date": "2019-04-28 23:44:29+00:00",
            "title": "Attentive Adversarial Learning for Domain-Invariant Training",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "Adversarial domain-invariant training (ADIT) proves to be effective in\nsuppressing the effects of domain variability in acoustic modeling and has led\nto improved performance in automatic speech recognition (ASR). In ADIT, an\nauxiliary domain classifier takes in equally-weighted deep features from a deep\nneural network (DNN) acoustic model and is trained to improve their\ndomain-invariance by optimizing an adversarial loss function. In this work, we\npropose an attentive ADIT (AADIT) in which we advance the domain classifier\nwith an attention mechanism to automatically weight the input deep features\naccording to their importance in domain classification. With this attentive\nre-weighting, AADIT can focus on the domain normalization of phonetic\ncomponents that are more susceptible to domain variability and generates deep\nfeatures with improved domain-invariance and senone-discriminativity over ADIT.\nMost importantly, the attention block serves only as an external component to\nthe DNN acoustic model and is not involved in ASR, so AADIT can be used to\nimprove the acoustic modeling with any DNN architectures. More generally, the\nsame methodology can improve any adversarial learning system with an auxiliary\ndiscriminator. Evaluated on CHiME-3 dataset, the AADIT achieves 13.6% and 9.3%\nrelative WER improvements, respectively, over a multi-conditional model and a\nstrong ADIT baseline.",
            "journal ref": "2019 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Brighton, United Kingdom",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.12406v1",
            "date": "2019-04-29 00:37:27+00:00",
            "title": "Adversarial Speaker Verification",
            "authors": [
                "Zhong Meng",
                "Yong Zhao",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "The use of deep networks to extract embeddings for speaker recognition has\nproven successfully. However, such embeddings are susceptible to performance\ndegradation due to the mismatches among the training, enrollment, and test\nconditions. In this work, we propose an adversarial speaker verification (ASV)\nscheme to learn the condition-invariant deep embedding via adversarial\nmulti-task training. In ASV, a speaker classification network and a condition\nidentification network are jointly optimized to minimize the speaker\nclassification loss and simultaneously mini-maximize the condition loss. The\ntarget labels of the condition network can be categorical (environment types)\nand continuous (SNR values). We further propose multi-factorial ASV to\nsimultaneously suppress multiple factors that constitute the condition\nvariability. Evaluated on a Microsoft Cortana text-dependent speaker\nverification task, the ASV achieves 8.8% and 14.5% relative improvements in\nequal error rates (EER) for known and unknown conditions, respectively.",
            "journal ref": "2019 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Brighton, United Kingdom",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.12407v1",
            "date": "2019-04-29 00:38:16+00:00",
            "title": "Adversarial Speaker Adaptation",
            "authors": [
                "Zhong Meng",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "We propose a novel adversarial speaker adaptation (ASA) scheme, in which\nadversarial learning is applied to regularize the distribution of deep hidden\nfeatures in a speaker-dependent (SD) deep neural network (DNN) acoustic model\nto be close to that of a fixed speaker-independent (SI) DNN acoustic model\nduring adaptation. An additional discriminator network is introduced to\ndistinguish the deep features generated by the SD model from those produced by\nthe SI model. In ASA, with a fixed SI model as the reference, an SD model is\njointly optimized with the discriminator network to minimize the senone\nclassification loss, and simultaneously to mini-maximize the SI/SD\ndiscrimination loss on the adaptation data. With ASA, a senone-discriminative\ndeep feature is learned in the SD model with a similar distribution to that of\nthe SI model. With such a regularized and adapted deep feature, the SD model\ncan perform improved automatic speech recognition on the target speaker's\nspeech. Evaluated on the Microsoft short message dictation dataset, ASA\nachieves 14.4% and 7.9% relative word error rate improvements for supervised\nand unsupervised adaptation, respectively, over an SI model trained from 2600\nhours data, with 200 adaptation utterances per speaker.",
            "journal ref": "2019 IEEE International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), Brighton, United Kingdom",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1904.12535v1",
            "date": "2019-04-29 09:37:31+00:00",
            "title": "Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction",
            "authors": [
                "Mingming Sun",
                "Xu Li",
                "Xin Wang",
                "Miao Fan",
                "Yue Feng",
                "Ping Li"
            ],
            "abstract": "In this paper, we consider the problem of open information extraction (OIE)\nfor extracting entity and relation level intermediate structures from sentences\nin open-domain. We focus on four types of valuable intermediate structures\n(Relation, Attribute, Description, and Concept), and propose a unified\nknowledge expression form, SAOKE, to express them. We publicly release a data\nset which contains more than forty thousand sentences and the corresponding\nfacts in the SAOKE format labeled by crowd-sourcing. To our knowledge, this is\nthe largest publicly available human labeled data set for open information\nextraction tasks. Using this labeled SAOKE data set, we train an end-to-end\nneural model using the sequenceto-sequence paradigm, called Logician, to\ntransform sentences into facts. For each sentence, different to existing\nalgorithms which generally focus on extracting each single fact without\nconcerning other possible facts, Logician performs a global optimization over\nall possible involved facts, in which facts not only compete with each other to\nattract the attention of words, but also cooperate to share words. An\nexperimental study on various types of open domain relation extraction tasks\nreveals the consistent superiority of Logician to other states-of-the-art\nalgorithms. The experiments verify the reasonableness of SAOKE format, the\nvaluableness of SAOKE data set, the effectiveness of the proposed Logician\nmodel, and the feasibility of the methodology to apply end-to-end learning\nparadigm on supervised data sets for the challenging tasks of open information\nextraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1904.12848v6",
            "date": "2019-04-29 17:56:59+00:00",
            "title": "Unsupervised Data Augmentation for Consistency Training",
            "authors": [
                "Qizhe Xie",
                "Zihang Dai",
                "Eduard Hovy",
                "Minh-Thang Luong",
                "Quoc V. Le"
            ],
            "abstract": "Semi-supervised learning lately has shown much promise in improving deep\nlearning models when labeled data is scarce. Common among recent approaches is\nthe use of consistency training on a large amount of unlabeled data to\nconstrain model predictions to be invariant to input noise. In this work, we\npresent a new perspective on how to effectively noise unlabeled examples and\nargue that the quality of noising, specifically those produced by advanced data\naugmentation methods, plays a crucial role in semi-supervised learning. By\nsubstituting simple noising operations with advanced data augmentation methods\nsuch as RandAugment and back-translation, our method brings substantial\nimprovements across six language and three vision tasks under the same\nconsistency training framework. On the IMDb text classification dataset, with\nonly 20 labeled examples, our method achieves an error rate of 4.20,\noutperforming the state-of-the-art model trained on 25,000 labeled examples. On\na standard semi-supervised learning benchmark, CIFAR-10, our method outperforms\nall previous approaches and achieves an error rate of 5.43 with only 250\nexamples. Our method also combines well with transfer learning, e.g., when\nfinetuning from BERT, and yields improvements in high-data regime, such as\nImageNet, whether when there is only 10% labeled data or when a full labeled\nset with 1.3M extra unlabeled examples is used. Code is available at\nhttps://github.com/google-research/uda.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1905.01152v2",
            "date": "2019-04-30 16:13:41+00:00",
            "title": "Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text",
            "authors": [
                "Murali Karthick Baskar",
                "Shinji Watanabe",
                "Ramon Astudillo",
                "Takaaki Hori",
                "Luk\u00e1\u0161 Burget",
                "Jan \u010cernock\u00fd"
            ],
            "abstract": "Sequence-to-sequence automatic speech recognition (ASR) models require large\nquantities of data to attain high performance. For this reason, there has been\na recent surge in interest for unsupervised and semi-supervised training in\nsuch models. This work builds upon recent results showing notable improvements\nin semi-supervised training using cycle-consistency and related techniques.\nSuch techniques derive training procedures and losses able to leverage unpaired\nspeech and/or text data by combining ASR with Text-to-Speech (TTS) models. In\nparticular, this work proposes a new semi-supervised loss combining an\nend-to-end differentiable ASR$\\rightarrow$TTS loss with TTS$\\rightarrow$ASR\nloss. The method is able to leverage both unpaired speech and text data to\noutperform recently proposed related techniques in terms of \\%WER. We provide\nextensive results analyzing the impact of data quantity and speech and text\nmodalities and show consistent gains across WSJ and Librispeech corpora. Our\ncode is provided in ESPnet to reproduce the experiments.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1905.05605v1",
            "date": "2019-05-11 00:14:09+00:00",
            "title": "Encrypted Speech Recognition using Deep Polynomial Networks",
            "authors": [
                "Shi-Xiong Zhang",
                "Yifan Gong",
                "Dong Yu"
            ],
            "abstract": "The cloud-based speech recognition/API provides developers or enterprises an\neasy way to create speech-enabled features in their applications. However,\nsending audios about personal or company internal information to the cloud,\nraises concerns about the privacy and security issues. The recognition results\ngenerated in cloud may also reveal some sensitive information. This paper\nproposes a deep polynomial network (DPN) that can be applied to the encrypted\nspeech as an acoustic model. It allows clients to send their data in an\nencrypted form to the cloud to ensure that their data remains confidential, at\nmean while the DPN can still make frame-level predictions over the encrypted\nspeech and return them in encrypted form. One good property of the DPN is that\nit can be trained on unencrypted speech features in the traditional way. To\nkeep the cloud away from the raw audio and recognition results, a cloud-local\njoint decoding framework is also proposed. We demonstrate the effectiveness of\nmodel and framework on the Switchboard and Cortana voice assistant tasks with\nsmall performance degradation and latency increased comparing with the\ntraditional cloud-based DNNs.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/1905.08067v1",
            "date": "2019-05-15 17:14:34+00:00",
            "title": "Understanding the Radical Mind: Identifying Signals to Detect Extremist Content on Twitter",
            "authors": [
                "Mariam Nouh",
                "Jason R. C. Nurse",
                "Michael Goldsmith"
            ],
            "abstract": "The Internet and, in particular, Online Social Networks have changed the way\nthat terrorist and extremist groups can influence and radicalise individuals.\nRecent reports show that the mode of operation of these groups starts by\nexposing a wide audience to extremist material online, before migrating them to\nless open online platforms for further radicalization. Thus, identifying\nradical content online is crucial to limit the reach and spread of the\nextremist narrative. In this paper, our aim is to identify measures to\nautomatically detect radical content in social media. We identify several\nsignals, including textual, psychological and behavioural, that together allow\nfor the classification of radical messages. Our contribution is three-fold: (1)\nwe analyze propaganda material published by extremist groups and create a\ncontextual text-based model of radical content, (2) we build a model of\npsychological properties inferred from these material, and (3) we evaluate\nthese models on Twitter to determine the extent to which it is possible to\nautomatically identify online radical tweets. Our results show that radical\nusers do exhibit distinguishable textual, psychological, and behavioural\nproperties. We find that the psychological properties are among the most\ndistinguishing features. Additionally, our results show that textual models\nusing vector embedding features significantly improves the detection over\nTF-IDF features. We validate our approach on two experiments achieving high\naccuracy. Our findings can be utilized as signals for detecting online\nradicalization activities.",
            "journal ref": "17th IEEE International Conference on Intelligence and Security\n  Informatics (ISI), 2019",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1905.08772v1",
            "date": "2019-05-18 15:46:38+00:00",
            "title": "A Text Classification Framework for Simple and Effective Early Depression Detection Over Social Media Streams",
            "authors": [
                "Sergio G. Burdisso",
                "Marcelo Errecalde",
                "Manuel Montes-y-G\u00f3mez"
            ],
            "abstract": "With the rise of the Internet, there is a growing need to build intelligent\nsystems that are capable of efficiently dealing with early risk detection (ERD)\nproblems on social media, such as early depression detection, early rumor\ndetection or identification of sexual predators. These systems, nowadays mostly\nbased on machine learning techniques, must be able to deal with data streams\nsince users provide their data over time. In addition, these systems must be\nable to decide when the processed data is sufficient to actually classify\nusers. Moreover, since ERD tasks involve risky decisions by which people's\nlives could be affected, such systems must also be able to justify their\ndecisions. However, most standard and state-of-the-art supervised machine\nlearning models (such as SVM, MNB, Neural Networks, etc.) are not well suited\nto deal with this scenario. This is due to the fact that they either act as\nblack boxes or do not support incremental classification/learning. In this\npaper we introduce SS3, a novel supervised learning model for text\nclassification that naturally supports these aspects. SS3 was designed to be\nused as a general framework to deal with ERD problems. We evaluated our model\non the CLEF's eRisk2017 pilot task on early depression detection. Most of the\n30 contributions submitted to this competition used state-of-the-art methods.\nExperimental results show that our classifier was able to outperform these\nmodels and standard classifiers, despite being less computationally expensive\nand having the ability to explain its rationale.",
            "journal ref": "18 May 2019, Volume 133, Expert Systems With Applications,\n  Elsevier",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1905.11235v4",
            "date": "2019-05-27 14:00:45+00:00",
            "title": "CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition",
            "authors": [
                "Linhao Dong",
                "Bo Xu"
            ],
            "abstract": "In this paper, we propose a novel soft and monotonic alignment mechanism used\nfor sequence transduction. It is inspired by the integrate-and-fire model in\nspiking neural networks and employed in the encoder-decoder framework consists\nof continuous functions, thus being named as: Continuous Integrate-and-Fire\n(CIF). Applied to the ASR task, CIF not only shows a concise calculation, but\nalso supports online recognition and acoustic boundary positioning, thus\nsuitable for various ASR scenarios. Several support strategies are also\nproposed to alleviate the unique problems of CIF-based model. With the joint\naction of these methods, the CIF-based model shows competitive performance.\nNotably, it achieves a word error rate (WER) of 2.86% on the test-clean of\nLibrispeech and creates new state-of-the-art result on Mandarin telephone ASR\nbenchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1905.13561v1",
            "date": "2019-05-30 01:33:31+00:00",
            "title": "Speaker Anonymization Using X-vector and Neural Waveform Models",
            "authors": [
                "Fuming Fang",
                "Xin Wang",
                "Junichi Yamagishi",
                "Isao Echizen",
                "Massimiliano Todisco",
                "Nicholas Evans",
                "Jean-Francois Bonastre"
            ],
            "abstract": "The social media revolution has produced a plethora of web services to which\nusers can easily upload and share multimedia documents. Despite the popularity\nand convenience of such services, the sharing of such inherently personal data,\nincluding speech data, raises obvious security and privacy concerns. In\nparticular, a user's speech data may be acquired and used with speech synthesis\nsystems to produce high-quality speech utterances which reflect the same user's\nspeaker identity. These utterances may then be used to attack speaker\nverification systems. One solution to mitigate these concerns involves the\nconcealing of speaker identities before the sharing of speech data. For this\npurpose, we present a new approach to speaker anonymization. The idea is to\nextract linguistic and speaker identity features from an utterance and then to\nuse these with neural acoustic and waveform models to synthesize anonymized\nspeech. The original speaker identity, in the form of timbre, is suppressed and\nreplaced with that of an anonymous pseudo identity. The approach exploits\nstate-of-the-art x-vector speaker representations. These are used to derive\nanonymized pseudo speaker identities through the combination of multiple,\nrandom speaker x-vectors. Experimental results show that the proposed approach\nis effective in concealing speaker identities. It increases the equal error\nrate of a speaker verification system while maintaining high quality,\nanonymized speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.01040v3",
            "date": "2019-06-03 19:33:43+00:00",
            "title": "A Surprising Density of Illusionable Natural Speech",
            "authors": [
                "Melody Y. Guan",
                "Gregory Valiant"
            ],
            "abstract": "Recent work on adversarial examples has demonstrated that most natural inputs\ncan be perturbed to fool even state-of-the-art machine learning systems. But\ndoes this happen for humans as well? In this work, we investigate: what\nfraction of natural instances of speech can be turned into \"illusions\" which\neither alter humans' perception or result in different people having\nsignificantly different perceptions? We first consider the McGurk effect, the\nphenomenon by which adding a carefully chosen video clip to the audio channel\naffects the viewer's perception of what is said (McGurk and MacDonald, 1976).\nWe obtain empirical estimates that a significant fraction of both words and\nsentences occurring in natural speech have some susceptibility to this effect.\nWe also learn models for predicting McGurk illusionability. Finally we\ndemonstrate that the Yanny or Laurel auditory illusion (Pressnitzer et al.,\n2018) is not an isolated occurrence by generating several very different new\ninstances. We believe that the surprising density of illusionable natural\nspeech warrants further investigation, from the perspectives of both security\nand cognitive science. Supplementary videos are available at:\nhttps://www.youtube.com/playlist?list=PLaX7t1K-e_fF2iaenoKznCatm0RC37B_k.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1906.01983v1",
            "date": "2019-06-03 17:26:36+00:00",
            "title": "The Computational Structure of Unintentional Meaning",
            "authors": [
                "Mark K. Ho",
                "Joanna Korman",
                "Thomas L. Griffiths"
            ],
            "abstract": "Speech-acts can have literal meaning as well as pragmatic meaning, but these\nboth involve consequences typically intended by a speaker. Speech-acts can also\nhave unintentional meaning, in which what is conveyed goes above and beyond\nwhat was intended. Here, we present a Bayesian analysis of how, to a listener,\nthe meaning of an utterance can significantly differ from a speaker's intended\nmeaning. Our model emphasizes how comprehending the intentional and\nunintentional meaning of speech-acts requires listeners to engage in\nsophisticated model-based perspective-taking and reasoning about the history of\nthe state of the world, each other's actions, and each other's observations. To\ntest our model, we have human participants make judgments about vignettes where\nspeakers make utterances that could be interpreted as intentional insults or\nunintentional faux pas. In elucidating the mechanics of speech-acts with\nunintentional meanings, our account provides insight into how communication\nboth functions and malfunctions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.02246v1",
            "date": "2019-06-05 18:51:26+00:00",
            "title": "Complex Evolution Recurrent Neural Networks (ceRNNs)",
            "authors": [
                "Izhak Shafran",
                "Tom Bagby",
                "R. J. Skerry-Ryan"
            ],
            "abstract": "Unitary Evolution Recurrent Neural Networks (uRNNs) have three attractive\nproperties: (a) the unitary property, (b) the complex-valued nature, and (c)\ntheir efficient linear operators. The literature so far does not address -- how\ncritical is the unitary property of the model? Furthermore, uRNNs have not been\nevaluated on large tasks. To study these shortcomings, we propose the complex\nevolution Recurrent Neural Networks (ceRNNs), which is similar to uRNNs but\ndrops the unitary property selectively. On a simple multivariate linear\nregression task, we illustrate that dropping the constraints improves the\nlearning trajectory. In copy memory task, ceRNNs and uRNNs perform identically,\ndemonstrating that their superior performance over LSTMs is due to\ncomplex-valued nature and their linear operators. In a large scale real-world\nspeech recognition, we find that pre-pending a uRNN degrades the performance of\nour baseline LSTM acoustic models, while pre-pending a ceRNN improves the\nperformance over the baseline by 0.8% absolute WER.",
            "journal ref": "Proc. International Conference on Acoustics, Speech and Signal\n  Processing (ICASSP), pages 5854-5858, 2018",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.02671v1",
            "date": "2019-04-24 17:43:40+00:00",
            "title": "Grounding Natural Language Commands to StarCraft II Game States for Narration-Guided Reinforcement Learning",
            "authors": [
                "Nicholas Waytowich",
                "Sean L. Barton",
                "Vernon Lawhern",
                "Ethan Stump",
                "Garrett Warnell"
            ],
            "abstract": "While deep reinforcement learning techniques have led to agents that are\nsuccessfully able to learn to perform a number of tasks that had been\npreviously unlearnable, these techniques are still susceptible to the\nlongstanding problem of {\\em reward sparsity}. This is especially true for\ntasks such as training an agent to play StarCraft II, a real-time strategy game\nwhere reward is only given at the end of a game which is usually very long.\nWhile this problem can be addressed through reward shaping, such approaches\ntypically require a human expert with specialized knowledge. Inspired by the\nvision of enabling reward shaping through the more-accessible paradigm of\nnatural-language narration, we investigate to what extent we can contextualize\nthese narrations by grounding them to the goal-specific states. We present a\nmutual-embedding model using a multi-input deep-neural network that projects a\nsequence of natural language commands into the same high-dimensional\nrepresentation space as corresponding goal states. We show that using this\nmodel we can learn an embedding space with separable and distinct clusters that\naccurately maps natural-language commands to corresponding game states . We\nalso discuss how this model can allow for the use of narrations as a robust\nform of reward shaping to improve RL performance and efficiency.",
            "journal ref": "None",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/1906.04165v1",
            "date": "2019-06-07 19:50:30+00:00",
            "title": "Leveraging BERT for Extractive Text Summarization on Lectures",
            "authors": [
                "Derek Miller"
            ],
            "abstract": "In the last two decades, automatic extractive text summarization on lectures\nhas demonstrated to be a useful tool for collecting key phrases and sentences\nthat best represent the content. However, many current approaches utilize dated\napproaches, producing sub-par outputs or requiring several hours of manual\ntuning to produce meaningful results. Recently, new machine learning\narchitectures have provided mechanisms for extractive summarization through the\nclustering of output embeddings from deep learning models. This paper reports\non the project called Lecture Summarization Service, a python based RESTful\nservice that utilizes the BERT model for text embeddings and KMeans clustering\nto identify sentences closes to the centroid for summary selection. The purpose\nof the service was to provide students a utility that could summarize lecture\ncontent, based on their desired number of sentences. On top of the summary\nwork, the service also includes lecture and summary management, storing content\non the cloud which can be used for collaboration. While the results of\nutilizing BERT for extractive summarization were promising, there were still\nareas where the model struggled, providing feature research opportunities for\nfurther improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04233v2",
            "date": "2019-06-10 19:05:03+00:00",
            "title": "Using generative modelling to produce varied intonation for speech synthesis",
            "authors": [
                "Zack Hodari",
                "Oliver Watts",
                "Simon King"
            ],
            "abstract": "Unlike human speakers, typical text-to-speech (TTS) systems are unable to\nproduce multiple distinct renditions of a given sentence. This has previously\nbeen addressed by adding explicit external control. In contrast, generative\nmodels are able to capture a distribution over multiple renditions and thus\nproduce varied renditions using sampling. Typical neural TTS models learn the\naverage of the data because they minimise mean squared error. In the context of\nprosody, taking the average produces flatter, more boring speech: an \"average\nprosody\". A generative model that can synthesise multiple prosodies will, by\ndesign, not model average prosody. We use variational autoencoders (VAEs) which\nexplicitly place the most \"average\" data close to the mean of the Gaussian\nprior. We propose that by moving towards the tails of the prior distribution,\nthe model will transition towards generating more idiosyncratic, varied\nrenditions. Focusing here on intonation, we investigate the trade-off between\nnaturalness and intonation variation and find that typical acoustic models can\neither be natural, or varied, but not both. However, sampling from the tails of\nthe VAE prior produces much more varied intonation than the traditional\napproaches, whilst maintaining the same level of naturalness.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.05678v1",
            "date": "2019-06-13 17:04:46+00:00",
            "title": "Telephonetic: Making Neural Language Models Robust to ASR and Semantic Noise",
            "authors": [
                "Chris Larson",
                "Tarek Lahlou",
                "Diana Mingels",
                "Zachary Kulis",
                "Erik Mueller"
            ],
            "abstract": "Speech processing systems rely on robust feature extraction to handle\nphonetic and semantic variations found in natural language. While techniques\nexist for desensitizing features to common noise patterns produced by\nSpeech-to-Text (STT) and Text-to-Speech (TTS) systems, the question remains how\nto best leverage state-of-the-art language models (which capture rich semantic\nfeatures, but are trained on only written text) on inputs with ASR errors. In\nthis paper, we present Telephonetic, a data augmentation framework that helps\nrobustify language model features to ASR corrupted inputs. To capture phonetic\nalterations, we employ a character-level language model trained using\nprobabilistic masking. Phonetic augmentations are generated in two stages: a\nTTS encoder (Tacotron 2, WaveGlow) and a STT decoder (DeepSpeech). Similarly,\nsemantic perturbations are produced by sampling from nearby words in an\nembedding space, which is computed using the BERT language model. Words are\nselected for augmentation according to a hierarchical grammar sampling\nstrategy. Telephonetic is evaluated on the Penn Treebank (PTB) corpus, and\ndemonstrates its effectiveness as a bootstrapping technique for transferring\nneural language models to the speech domain. Notably, our language model\nachieves a test perplexity of 37.49 on PTB, which to our knowledge is\nstate-of-the-art among models trained only on PTB.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.05681v1",
            "date": "2019-06-11 17:35:02+00:00",
            "title": "Deep Learning based Emotion Recognition System Using Speech Features and Transcriptions",
            "authors": [
                "Suraj Tripathi",
                "Abhay Kumar",
                "Abhiram Ramesh",
                "Chirag Singh",
                "Promod Yenigalla"
            ],
            "abstract": "This paper proposes a speech emotion recognition method based on speech\nfeatures and speech transcriptions (text). Speech features such as Spectrogram\nand Mel-frequency Cepstral Coefficients (MFCC) help retain emotion-related\nlow-level characteristics in speech whereas text helps capture semantic\nmeaning, both of which help in different aspects of emotion detection. We\nexperimented with several Deep Neural Network (DNN) architectures, which take\nin different combinations of speech features and text as inputs. The proposed\nnetwork architectures achieve higher accuracies when compared to\nstate-of-the-art methods on a benchmark dataset. The combined MFCC-Text\nConvolutional Neural Network (CNN) model proved to be the most accurate in\nrecognizing emotions in IEMOCAP data.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.08333v1",
            "date": "2019-06-19 20:13:27+00:00",
            "title": "Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification",
            "authors": [
                "Youngmoon Jung",
                "Younggwan Kim",
                "Hyungjun Lim",
                "Yeunju Choi",
                "Hoirin Kim"
            ],
            "abstract": "In this paper, we propose a new pooling method called spatial pyramid\nencoding (SPE) to generate speaker embeddings for text-independent speaker\nverification. We first partition the output feature maps from a deep residual\nnetwork (ResNet) into increasingly fine sub-regions and extract speaker\nembeddings from each sub-region through a learnable dictionary encoding layer.\nThese embeddings are concatenated to obtain the final speaker representation.\nThe SPE layer not only generates a fixed-dimensional speaker embedding for a\nvariable-length speech segment, but also aggregates the information of feature\ndistribution from multi-level temporal bins. Furthermore, we apply deep length\nnormalization by augmenting the loss function with ring loss. By applying ring\nloss, the network gradually learns to normalize the speaker embeddings using\nmodel weights themselves while preserving convexity, leading to more robust\nspeaker embeddings. Experiments on the VoxCeleb1 dataset show that the proposed\nsystem using the SPE layer and ring loss-based deep length normalization\noutperforms both i-vector and d-vector baselines.",
            "journal ref": "Proc. of Interspeech 2019, 2019, pp. 4030-4034",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.08871v9",
            "date": "2019-06-17 23:06:51+00:00",
            "title": "Advancing Speech Recognition With No Speech Or With Noisy Speech",
            "authors": [
                "Gautam Krishna",
                "Co Tran",
                "Mason Carnahan",
                "Ahmed H Tewfik"
            ],
            "abstract": "In this paper we demonstrate end-to-end continuous speech recognition (CSR)\nusing electroencephalography (EEG) signals with no speech signal as input. An\nattention model based automatic speech recognition (ASR) and connectionist\ntemporal classification (CTC) based ASR systems were implemented for performing\nrecognition. We further demonstrate CSR for noisy speech by fusing with EEG\nfeatures.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1906.10816v4",
            "date": "2019-06-26 02:28:10+00:00",
            "title": "Program Synthesis and Semantic Parsing with Learned Code Idioms",
            "authors": [
                "Richard Shin",
                "Miltiadis Allamanis",
                "Marc Brockschmidt",
                "Oleksandr Polozov"
            ],
            "abstract": "Program synthesis of general-purpose source code from natural language\nspecifications is challenging due to the need to reason about high-level\npatterns in the target program and low-level implementation details at the same\ntime. In this work, we present PATOIS, a system that allows a neural program\nsynthesizer to explicitly interleave high-level and low-level reasoning at\nevery generation step. It accomplishes this by automatically mining common code\nidioms from a given corpus, incorporating them into the underlying language for\nneural synthesis, and training a tree-based neural synthesizer to use these\nidioms during code generation. We evaluate PATOIS on two complex semantic\nparsing datasets and show that using learned code idioms improves the\nsynthesizer's accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1906.11889v5",
            "date": "2019-06-20 10:36:40+00:00",
            "title": "Deep Eyedentification: Biometric Identification using Micro-Movements of the Eye",
            "authors": [
                "Lena A. J\u00e4ger",
                "Silvia Makowski",
                "Paul Prasse",
                "Sascha Liehr",
                "Maximilian Seidler",
                "Tobias Scheffer"
            ],
            "abstract": "We study involuntary micro-movements of the eye for biometric identification.\nWhile prior studies extract lower-frequency macro-movements from the output of\nvideo-based eye-tracking systems and engineer explicit features of these\nmacro-movements, we develop a deep convolutional architecture that processes\nthe raw eye-tracking signal. Compared to prior work, the network attains a\nlower error rate by one order of magnitude and is faster by two orders of\nmagnitude: it identifies users accurately within seconds.",
            "journal ref": "In: U. Brefeld et al. (Eds.): Machine Learning and Knowledge\n  Discovery in Databases, ECML PKDD 2019, LNCS 11907, pp. 299-314, Springer\n  Nature, Switzerland, 2020",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.00443v1",
            "date": "2019-06-30 20:14:19+00:00",
            "title": "Multilingual Bottleneck Features for Query by Example Spoken Term Detection",
            "authors": [
                "Dhananjay Ram",
                "Lesly Miculicich",
                "Herv\u00e9 Bourlard"
            ],
            "abstract": "State of the art solutions to query by example spoken term detection\n(QbE-STD) usually rely on bottleneck feature representation of the query and\naudio document to perform dynamic time warping (DTW) based template matching.\nHere, we present a study on QbE-STD performance using several monolingual as\nwell as multilingual bottleneck features extracted from feed forward networks.\nThen, we propose to employ residual networks (ResNet) to estimate the\nbottleneck features and show significant improvements over the corresponding\nfeed forward network based features. The neural networks are trained on\nGlobalPhone corpus and QbE-STD experiments are performed on a very challenging\nQUESST 2014 database.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.00835v1",
            "date": "2019-07-01 14:54:31+00:00",
            "title": "UltraSuite: A Repository of Ultrasound and Acoustic Data from Child Speech Therapy Sessions",
            "authors": [
                "Aciel Eshky",
                "Manuel Sam Ribeiro",
                "Joanne Cleland",
                "Korin Richmond",
                "Zoe Roxburgh",
                "James Scobbie",
                "Alan Wrench"
            ],
            "abstract": "We introduce UltraSuite, a curated repository of ultrasound and acoustic\ndata, collected from recordings of child speech therapy sessions. This release\nincludes three data collections, one from typically developing children and two\nfrom children with speech sound disorders. In addition, it includes a set of\nannotations, some manual and some automatically produced, and software tools to\nprocess, transform and visualise the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01160v1",
            "date": "2019-07-02 04:27:55+00:00",
            "title": "WHAM!: Extending Speech Separation to Noisy Environments",
            "authors": [
                "Gordon Wichern",
                "Joe Antognini",
                "Michael Flynn",
                "Licheng Richard Zhu",
                "Emmett McQuinn",
                "Dwight Crow",
                "Ethan Manilow",
                "Jonathan Le Roux"
            ],
            "abstract": "Recent progress in separating the speech signals from multiple overlapping\nspeakers using a single audio channel has brought us closer to solving the\ncocktail party problem. However, most studies in this area use a constrained\nproblem setup, comparing performance when speakers overlap almost completely,\nat artificially low sampling rates, and with no external background noise. In\nthis paper, we strive to move the field towards more realistic and challenging\nscenarios. To that end, we created the WSJ0 Hipster Ambient Mixtures (WHAM!)\ndataset, consisting of two speaker mixtures from the wsj0-2mix dataset combined\nwith real ambient noise samples. The samples were collected in coffee shops,\nrestaurants, and bars in the San Francisco Bay Area, and are made publicly\navailable. We benchmark various speech separation architectures and objective\nfunctions to evaluate their robustness to noise. While separation performance\ndecreases as a result of noise, we still observe substantial gains relative to\nthe noisy signals for most approaches.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1907.01409v1",
            "date": "2019-07-01 15:16:04+00:00",
            "title": "Comparison of Lattice-Free and Lattice-Based Sequence Discriminative Training Criteria for LVCSR",
            "authors": [
                "Wilfried Michel",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "Sequence discriminative training criteria have long been a standard tool in\nautomatic speech recognition for improving the performance of acoustic models\nover their maximum likelihood / cross entropy trained counterparts. While\npreviously a lattice approximation of the search space has been necessary to\nreduce computational complexity, recently proposed methods use other\napproximations to dispense of the need for the computationally expensive step\nof separate lattice creation.\n  In this work we present a memory efficient implementation of the\nforward-backward computation that allows us to use uni-gram word-level language\nmodels in the denominator calculation while still doing a full summation on\nGPU. This allows for a direct comparison of lattice-based and lattice-free\nsequence discriminative training criteria such as MMI and sMBR, both using the\nsame language model during training.\n  We compared performance, speed of convergence, and stability on large\nvocabulary continuous speech recognition tasks like Switchboard and Quaero. We\nfound that silence modeling seriously impacts the performance in the\nlattice-free case and needs special treatment. In our experiments lattice-free\nMMI comes on par with its lattice-based counterpart. Lattice-based sMBR still\noutperforms all lattice-free training criteria.",
            "journal ref": "Interspeech 2019, 20th Annual Conference of the International\n  Speech Communication Association, Graz, Austria, 15-19 September 2019, pp.\n  1601--1605",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.01914v1",
            "date": "2019-07-02 10:30:27+00:00",
            "title": "Attention model for articulatory features detection",
            "authors": [
                "Ievgen Karaulov",
                "Dmytro Tkanov"
            ],
            "abstract": "Articulatory distinctive features, as well as phonetic transcription, play\nimportant role in speech-related tasks: computer-assisted pronunciation\ntraining, text-to-speech conversion (TTS), studying speech production\nmechanisms, speech recognition for low-resourced languages. End-to-end\napproaches to speech-related tasks got a lot of traction in recent years. We\napply Listen, Attend and Spell~(LAS)~\\cite{Chan-LAS2016} architecture to phones\nrecognition on a small small training set, like TIMIT~\\cite{TIMIT-1992}. Also,\nwe introduce a novel decoding technique that allows to train manners and places\nof articulation detectors end-to-end using attention models. We also explore\njoint phones recognition and articulatory features detection in multitask\nlearning setting.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.02704v3",
            "date": "2019-07-05 07:27:31+00:00",
            "title": "Extraction and Analysis of Fictional Character Networks: A Survey",
            "authors": [
                "Vincent Labatut",
                "Xavier Bost"
            ],
            "abstract": "A character network is a graph extracted from a narrative, in which vertices\nrepresent characters and edges correspond to interactions between them. A\nnumber of narrative-related problems can be addressed automatically through the\nanalysis of character networks, such as summarization, classification, or role\ndetection. Character networks are particularly relevant when considering works\nof fictions (e.g. novels, plays, movies, TV series), as their exploitation\nallows developing information retrieval and recommendation systems. However,\nworks of fiction possess specific properties making these tasks harder. This\nsurvey aims at presenting and organizing the scientific literature related to\nthe extraction of character networks from works of fiction, as well as their\nanalysis. We first describe the extraction process in a generic way, and\nexplain how its constituting steps are implemented in practice, depending on\nthe medium of the narrative, the goal of the network analysis, and other\nfactors. We then review the descriptive tools used to characterize character\nnetworks, with a focus on the way they are interpreted in this context. We\nillustrate the relevance of character networks by also providing a review of\napplications derived from their analysis. Finally, we identify the limitations\nof the existing approaches, and the most promising perspectives.",
            "journal ref": "ACM Computing Surveys, Association for Computing Machinery, 2019,\n  52 (5), pp.89",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1907.04378v1",
            "date": "2019-07-09 19:33:01+00:00",
            "title": "M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention",
            "authors": [
                "Shuang Ma",
                "Daniel McDuff",
                "Yale Song"
            ],
            "abstract": "Generative adversarial networks have led to significant advances in\ncross-modal/domain translation. However, typically these networks are designed\nfor a specific task (e.g., dialogue generation or image synthesis, but not\nboth). We present a unified model, M3D-GAN, that can translate across a wide\nrange of modalities (e.g., text, image, and speech) and domains (e.g.,\nattributes in images or emotions in speech). Our model consists of modality\nsubnets that convert data from different modalities into unified\nrepresentations, and a unified computing body where data from different\nmodalities share the same network architecture. We introduce a universal\nattention module that is jointly trained with the whole network and learns to\nencode a large range of domain information into a highly structured latent\nspace. We use this to control synthesis in novel ways, such as producing\ndiverse realistic pictures from a sketch or varying the emotion of synthesized\nspeech. We evaluate our approach on extensive benchmark tasks, including\nimage-to-image, text-to-image, image captioning, text-to-speech, speech\nrecognition, and machine translation. Our results show state-of-the-art\nperformance on some of the tasks.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1907.04916v1",
            "date": "2019-07-08 15:09:40+00:00",
            "title": "Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR",
            "authors": [
                "Felix Weninger",
                "Jes\u00fas Andr\u00e9s-Ferrer",
                "Xinwei Li",
                "Puming Zhan"
            ],
            "abstract": "Sequence-to-sequence (seq2seq) based ASR systems have shown state-of-the-art\nperformances while having clear advantages in terms of simplicity. However,\ncomparisons are mostly done on speaker independent (SI) ASR systems, though\nspeaker adapted conventional systems are commonly used in practice for\nimproving robustness to speaker and environment variations. In this paper, we\napply speaker adaptation to seq2seq models with the goal of matching the\nperformance of conventional ASR adaptation. Specifically, we investigate\nKullback-Leibler divergence (KLD) as well as Linear Hidden Network (LHN) based\nadaptation for seq2seq ASR, using different amounts (up to 20 hours) of\nadaptation data per speaker. Our SI models are trained on large amounts of\ndictation data and achieve state-of-the-art results. We obtained 25% relative\nword error rate (WER) improvement with KLD adaptation of the seq2seq model vs.\n18.7% gain from acoustic model adaptation in the conventional system. We also\nshow that the WER of the seq2seq model decreases log-linearly with the amount\nof adaptation data. Finally, we analyze adaptation based on the minimum WER\ncriterion and adapting the language model (LM) for score fusion with the\nspeaker adapted seq2seq model, which result in further improvements of the\nseq2seq system performance.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.05671v1",
            "date": "2019-07-12 10:51:48+00:00",
            "title": "Justifying Diagnosis Decisions by Deep Neural Networks",
            "authors": [
                "Graham Spinks",
                "Marie-Francine Moens"
            ],
            "abstract": "An integrated approach is proposed across visual and textual data to both\ndetermine and justify a medical diagnosis by a neural network. As deep learning\ntechniques improve, interest grows to apply them in medical applications. To\nenable a transition to workflows in a medical context that are aided by machine\nlearning, the need exists for such algorithms to help justify the obtained\noutcome so human clinicians can judge their validity. In this work, deep\nlearning methods are used to map a frontal X-Ray image to a continuous textual\nrepresentation. This textual representation is decoded into a diagnosis and the\nassociated textual justification that will help a clinician evaluate the\noutcome. Additionally, more explanatory data is provided for the diagnosis by\ngenerating a realistic X-Ray that belongs to the nearest alternative diagnosis.\nWith a clinical expert opinion study on a subset of the X-Ray data set from the\nIndiana University hospital network, we demonstrate that our justification\nmechanism significantly outperforms existing methods that use saliency maps.\nWhile performing multi-task training with multiple loss functions, our method\nachieves excellent diagnosis accuracy and captioning quality when compared to\ncurrent state-of-the-art single-task methods.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.07804v2",
            "date": "2019-07-17 22:59:56+00:00",
            "title": "OmniNet: A unified architecture for multi-modal multi-task learning",
            "authors": [
                "Subhojeet Pramanik",
                "Priyanka Agrawal",
                "Aman Hussain"
            ],
            "abstract": "Transformer is a popularly used neural network architecture, especially for\nlanguage understanding. We introduce an extended and unified architecture that\ncan be used for tasks involving a variety of modalities like image, text,\nvideos, etc. We propose a spatio-temporal cache mechanism that enables learning\nspatial dimension of the input in addition to the hidden states corresponding\nto the temporal input sequence. The proposed architecture further enables a\nsingle model to support tasks with multiple input modalities as well as\nasynchronous multi-task learning, thus we refer to it as OmniNet. For example,\na single instance of OmniNet can concurrently learn to perform the tasks of\npart-of-speech tagging, image captioning, visual question answering and video\nactivity recognition. We demonstrate that training these four tasks together\nresults in about three times compressed model while retaining the performance\nin comparison to training them individually. We also show that using this\nneural network pre-trained on some modalities assists in learning unseen tasks\nsuch as video captioning and video question answering. This illustrates the\ngeneralization capacity of the self-attention mechanism on the spatio-temporal\ncache present in OmniNet.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.09293v1",
            "date": "2019-07-19 01:49:37+00:00",
            "title": "DREAMT -- Embodied Motivational Conversational Storytelling",
            "authors": [
                "David M W Powers"
            ],
            "abstract": "Storytelling is fundamental to language, including culture, conversation and\ncommunication in their broadest senses. It thus emerges as an essential\ncomponent of intelligent systems, including systems where natural language is\nnot a primary focus or where we do not usually think of a story being involved.\nIn this paper we explore the emergence of storytelling as a requirement in\nembodied conversational agents, including its role in educational and health\ninterventions, as well as in a general-purpose computer interface for people\nwith disabilities or other constraints that prevent the use of traditional\nkeyboard and speech interfaces. We further present a characterization of\nstorytelling as an inventive fleshing out of detail according to a particular\npersonal perspective, and propose the DREAMT model to focus attention on the\ndifferent layers that need to be present in a character-driven storytelling\nsystem. Most if not all aspects of the DREAMT model have arisen from or been\nexplored in some aspect of our implemented research systems, but currently only\nat a primitive and relatively unintegrated level. However, this experience\nleads us to formalize and elaborate the DREAMT model mnemonically as follows: -\nDescription/Dialogue/Definition/Denotation - Realization/Representation/Role -\nExplanation/Education/Entertainment - Actualization/Activation -\nMotivation/Modelling - Topicalization/Transformation",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/1907.10597v3",
            "date": "2019-07-22 19:36:18+00:00",
            "title": "Green AI",
            "authors": [
                "Roy Schwartz",
                "Jesse Dodge",
                "Noah A. Smith",
                "Oren Etzioni"
            ],
            "abstract": "The computations required for deep learning research have been doubling every\nfew months, resulting in an estimated 300,000x increase from 2012 to 2018 [2].\nThese computations have a surprisingly large carbon footprint [38]. Ironically,\ndeep learning was inspired by the human brain, which is remarkably energy\nefficient. Moreover, the financial cost of the computations can make it\ndifficult for academics, students, and researchers, in particular those from\nemerging economies, to engage in deep learning research.\n  This position paper advocates a practical solution by making efficiency an\nevaluation criterion for research alongside accuracy and related measures. In\naddition, we propose reporting the financial cost or \"price tag\" of developing,\ntraining, and running models to provide baselines for the investigation of\nincreasingly efficient methods. Our goal is to make AI both greener and more\ninclusive---enabling any inspired undergraduate with a laptop to write\nhigh-quality research papers. Green AI is an emerging focus at the Allen\nInstitute for AI.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1908.00286v1",
            "date": "2019-08-01 09:19:27+00:00",
            "title": "Reinforcement Learning for Personalized Dialogue Management",
            "authors": [
                "Floris den Hengst",
                "Mark Hoogendoorn",
                "Frank van Harmelen",
                "Joost Bosman"
            ],
            "abstract": "Language systems have been of great interest to the research community and\nhave recently reached the mass market through various assistant platforms on\nthe web. Reinforcement Learning methods that optimize dialogue policies have\nseen successes in past years and have recently been extended into methods that\npersonalize the dialogue, e.g. take the personal context of users into account.\nThese works, however, are limited to personalization to a single user with whom\nthey require multiple interactions and do not generalize the usage of context\nacross users. This work introduces a problem where a generalized usage of\ncontext is relevant and proposes two Reinforcement Learning (RL)-based\napproaches to this problem. The first approach uses a single learner and\nextends the traditional POMDP formulation of dialogue state with features that\ndescribe the user context. The second approach segments users by context and\nthen employs a learner per context. We compare these approaches in a benchmark\nof existing non-RL and RL-based methods in three established and one novel\napplication domain of financial product recommendation. We compare the\ninfluence of context and training experiences on performance and find that\nlearning approaches generally outperform a handcrafted gold standard.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.00493v1",
            "date": "2019-08-01 16:42:47+00:00",
            "title": "Learning Joint Acoustic-Phonetic Word Embeddings",
            "authors": [
                "Mohamed El-Geish"
            ],
            "abstract": "Most speech recognition tasks pertain to mapping words across two modalities:\nacoustic and orthographic. In this work, we suggest learning encoders that map\nvariable-length, acoustic or phonetic, sequences that represent words into\nfixed-dimensional vectors in a shared latent space; such that the distance\nbetween two word vectors represents how closely the two words sound. Instead of\ndirectly learning the distances between word vectors, we employ weak\nsupervision and model a binary classification task to predict whether two\ninputs, one of each modality, represent the same word given a distance\nthreshold. We explore various deep-learning models, bimodal contrastive losses,\nand techniques for mining hard negative examples such as the semi-supervised\ntechnique of self-labeling. Our best model achieves an $F_1$ score of 0.95 for\nthe binary classification task.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1908.00648v1",
            "date": "2019-08-01 22:42:36+00:00",
            "title": "Contrastive Reasons Detection and Clustering from Online Polarized Debate",
            "authors": [
                "Amine Trabelsi",
                "Osmar R. Zaiane"
            ],
            "abstract": "This work tackles the problem of unsupervised modeling and extraction of the\nmain contrastive sentential reasons conveyed by divergent viewpoints on\npolarized issues. It proposes a pipeline approach centered around the detection\nand clustering of phrases, assimilated to argument facets using a novel Phrase\nAuthor Interaction Topic-Viewpoint model. The evaluation is based on the\ninformativeness, the relevance and the clustering accuracy of extracted\nreasons. The pipeline approach shows a significant improvement over\nstate-of-the-art methods in contrastive summarization on online debate\ndatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1908.02588v2",
            "date": "2019-08-01 09:01:19+00:00",
            "title": "Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness",
            "authors": [
                "Luke S. Snyder",
                "Yi-Shan Lin",
                "Morteza Karimzadeh",
                "Dan Goldwasser",
                "David S. Ebert"
            ],
            "abstract": "Various domain users are increasingly leveraging real-time social media data\nto gain rapid situational awareness. However, due to the high noise in the\ndeluge of data, effectively determining semantically relevant information can\nbe difficult, further complicated by the changing definition of relevancy by\neach end user for different events. The majority of existing methods for short\ntext relevance classification fail to incorporate users' knowledge into the\nclassification process. Existing methods that incorporate interactive user\nfeedback focus on historical datasets. Therefore, classifiers cannot be\ninteractively retrained for specific events or user-dependent needs in\nreal-time. This limits real-time situational awareness, as streaming data that\nis incorrectly classified cannot be corrected immediately, permitting the\npossibility for important incoming data to be incorrectly classified as well.\nWe present a novel interactive learning framework to improve the classification\nprocess in which the user iteratively corrects the relevancy of tweets in\nreal-time to train the classification model on-the-fly for immediate predictive\nimprovements. We computationally evaluate our classification model adapted to\nlearn at interactive rates. Our results show that our approach outperforms\nstate-of-the-art machine learning models. In addition, we integrate our\nframework with the extended Social Media Analytics and Reporting Toolkit\n(SMART) 2.0 system, allowing the use of our interactive learning framework\nwithin a visual analytics system tailored for real-time situational awareness.\nTo demonstrate our framework's effectiveness, we provide domain expert feedback\nfrom first responders who used the extended SMART 2.0 system.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1908.08597v1",
            "date": "2019-08-22 21:05:17+00:00",
            "title": "Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective",
            "authors": [
                "Danielle Bragg",
                "Oscar Koller",
                "Mary Bellard",
                "Larwan Berke",
                "Patrick Boudrealt",
                "Annelies Braffort",
                "Naomi Caselli",
                "Matt Huenerfauth",
                "Hernisa Kacorri",
                "Tessa Verhoef",
                "Christian Vogler",
                "Meredith Ringel Morris"
            ],
            "abstract": "Developing successful sign language recognition, generation, and translation\nsystems requires expertise in a wide range of fields, including computer\nvision, computer graphics, natural language processing, human-computer\ninteraction, linguistics, and Deaf culture. Despite the need for deep\ninterdisciplinary knowledge, existing research occurs in separate disciplinary\nsilos, and tackles separate portions of the sign language processing pipeline.\nThis leads to three key questions: 1) What does an interdisciplinary view of\nthe current landscape reveal? 2) What are the biggest challenges facing the\nfield? and 3) What are the calls to action for people working in the field? To\nhelp answer these questions, we brought together a diverse group of experts for\na two-day workshop. This paper presents the results of that interdisciplinary\nworkshop, providing key background that is often overlooked by computer\nscientists, a review of the state-of-the-art, a set of pressing challenges, and\na call to action for the research community.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1908.08979v1",
            "date": "2019-08-23 19:00:18+00:00",
            "title": "Controlling for Confounders in Multimodal Emotion Classification via Adversarial Learning",
            "authors": [
                "Mimansa Jaiswal",
                "Zakaria Aldeneh",
                "Emily Mower Provost"
            ],
            "abstract": "Various psychological factors affect how individuals express emotions. Yet,\nwhen we collect data intended for use in building emotion recognition systems,\nwe often try to do so by creating paradigms that are designed just with a focus\non eliciting emotional behavior. Algorithms trained with these types of data\nare unlikely to function outside of controlled environments because our\nemotions naturally change as a function of these other factors. In this work,\nwe study how the multimodal expressions of emotion change when an individual is\nunder varying levels of stress. We hypothesize that stress produces modulations\nthat can hide the true underlying emotions of individuals and that we can make\nemotion recognition algorithms more generalizable by controlling for variations\nin stress. To this end, we use adversarial networks to decorrelate stress\nmodulations from emotion representations. We study how stress alters acoustic\nand lexical emotional predictions, paying special attention to how modulations\ndue to stress affect the transferability of learned emotion recognition models\nacross domains. Our results show that stress is indeed encoded in trained\nemotion classifiers and that this encoding varies across levels of emotions and\nacross the lexical and acoustic modalities. Our results also show that emotion\nrecognition models that control for stress during training have better\ngeneralizability when applied to new domains, compared to models that do not\ncontrol for stress during training. We conclude that is is necessary to\nconsider the effect of extraneous psychological factors when building and\ntesting emotion recognition models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00183v1",
            "date": "2019-08-31 10:03:11+00:00",
            "title": "Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records",
            "authors": [
                "M. Tarik Altuncu",
                "Eloise Sorin",
                "Joshua D. Symons",
                "Erik Mayer",
                "Sophia N. Yaliraki",
                "Francesca Toni",
                "Mauricio Barahona"
            ],
            "abstract": "The large volume of text in electronic healthcare records often remains\nunderused due to a lack of methodologies to extract interpretable content. Here\nwe present an unsupervised framework for the analysis of free text that\ncombines text-embedding with paragraph vectors and graph-theoretical multiscale\ncommunity detection. We analyse text from a corpus of patient incident reports\nfrom the National Health Service in England to find content-based clusters of\nreports in an unsupervised manner and at different levels of resolution. Our\nunsupervised method extracts groups with high intrinsic textual consistency and\ncompares well against categories hand-coded by healthcare personnel. We also\nshow how to use our content-driven clusters to improve the supervised\nprediction of the degree of harm of the incident based on the text of the\nreport. Finally, we discuss future directions to monitor reports over time, and\nto detect emerging trends outside pre-existing categories.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.00521v2",
            "date": "2019-09-02 03:12:19+00:00",
            "title": "Modeling Long-Range Context for Concurrent Dialogue Acts Recognition",
            "authors": [
                "Yue Yu",
                "Siyao Peng",
                "Grace Hui Yang"
            ],
            "abstract": "In dialogues, an utterance is a chain of consecutive sentences produced by\none speaker which ranges from a short sentence to a thousand-word post. When\nstudying dialogues at the utterance level, it is not uncommon that an utterance\nwould serve multiple functions. For instance, \"Thank you. It works great.\"\nexpresses both gratitude and positive feedback in the same utterance. Multiple\ndialogue acts (DA) for one utterance breeds complex dependencies across\ndialogue turns. Therefore, DA recognition challenges a model's predictive power\nover long utterances and complex DA context. We term this problem Concurrent\nDialogue Acts (CDA) recognition. Previous work on DA recognition either assumes\none DA per utterance or fails to realize the sequential nature of dialogues. In\nthis paper, we present an adapted Convolutional Recurrent Neural Network (CRNN)\nwhich models the interactions between utterances of long-range context. Our\nmodel significantly outperforms existing work on CDA recognition on a tech\nforum dataset.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.01093v1",
            "date": "2019-08-25 18:36:55+00:00",
            "title": "Empirical Study on Detecting Controversy in Social Media",
            "authors": [
                "Azadeh Nematzadeh",
                "Grace Bang",
                "Xiaomo Liu",
                "Zhiqiang Ma"
            ],
            "abstract": "Companies and financial investors are paying increasing attention to social\nconsciousness in developing their corporate strategies and making investment\ndecisions to support a sustainable economy for the future. Public discussion on\nincidents and events -- controversies -- of companies can provide valuable\ninsights on how well the company operates with regards to social consciousness\nand indicate the company's overall operational capability. However, there are\nchallenges in evaluating the degree of a company's social consciousness and\nenvironmental sustainability due to the lack of systematic data. We introduce a\nsystem that utilizes Twitter data to detect and monitor controversial events\nand show their impact on market volatility. In our study, controversial events\nare identified from clustered tweets that share the same 5W terms and sentiment\npolarities of these clusters. Credible news links inside the event tweets are\nused to validate the truth of the event. A case study on the Starbucks\nPhiladelphia arrests shows that this method can provide the desired\nfunctionality.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/1909.01362v1",
            "date": "2019-09-03 18:00:05+00:00",
            "title": "Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop",
            "authors": [
                "Jonathan P. Chang",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Online discussions often derail into toxic exchanges between participants.\nRecent efforts mostly focused on detecting antisocial behavior after the fact,\nby analyzing single comments in isolation. To provide more timely notice to\nhuman moderators, a system needs to preemptively detect that a conversation is\nheading towards derailment before it actually turns toxic. This means modeling\nderailment as an emerging property of a conversation rather than as an isolated\nutterance-level event.\n  Forecasting emerging conversational properties, however, poses several\ninherent modeling challenges. First, since conversations are dynamic, a\nforecasting model needs to capture the flow of the discussion, rather than\nproperties of individual comments. Second, real conversations have an unknown\nhorizon: they can end or derail at any time; thus a practical forecasting model\nneeds to assess the risk in an online fashion, as the conversation develops. In\nthis work we introduce a conversational forecasting model that learns an\nunsupervised representation of conversational dynamics and exploits it to\npredict future derailment as the conversation develops. By applying this model\nto two new diverse datasets of online conversations with labels for antisocial\nevents, we show that it outperforms state-of-the-art systems at forecasting\nderailment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.01871v6",
            "date": "2019-09-04 15:20:01+00:00",
            "title": "Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning",
            "authors": [
                "Khanh Nguyen",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "Mobile agents that can leverage help from humans can potentially accomplish\nmore complex tasks than they could entirely on their own. We develop \"Help,\nAnna!\" (HANNA), an interactive photo-realistic simulator in which an agent\nfulfills object-finding tasks by requesting and interpreting natural\nlanguage-and-vision assistance. An agent solving tasks in a HANNA environment\ncan leverage simulated human assistants, called ANNA (Automatic Natural\nNavigation Assistants), which, upon request, provide natural language and\nvisual instructions to direct the agent towards the goals. To address the HANNA\nproblem, we develop a memory-augmented neural agent that hierarchically models\nmultiple levels of decision-making, and an imitation learning algorithm that\nteaches the agent to avoid repeating past mistakes while simultaneously\npredicting its own chances of making future progress. Empirically, our approach\nis able to ask for help more effectively than competitive baselines and, thus,\nattains higher task success rate on both previously seen and previously unseen\nenvironments. We publicly release code and data at\nhttps://github.com/khanhptnk/hanna . A video demo is available at\nhttps://youtu.be/18P94aaaLKg .",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1909.02851v1",
            "date": "2019-09-02 22:57:10+00:00",
            "title": "Avaya Conversational Intelligence: A Real-Time System for Spoken Language Understanding in Human-Human Call Center Conversations",
            "authors": [
                "Jan Mizgajski",
                "Adrian Szymczak",
                "Robert G\u0142owski",
                "Piotr Szyma\u0144ski",
                "Piotr \u017belasko",
                "\u0141ukasz Augustyniak",
                "Miko\u0142aj Morzy",
                "Yishay Carmiel",
                "Jeff Hodson",
                "\u0141ukasz W\u00f3jciak",
                "Daniel Smoczyk",
                "Adam Wr\u00f3bel",
                "Bartosz Borowik",
                "Adam Artajew",
                "Marcin Baran",
                "Cezary Kwiatkowski",
                "Marzena \u017by\u0142a-Hoppe"
            ],
            "abstract": "Avaya Conversational Intelligence(ACI) is an end-to-end, cloud-based solution\nfor real-time Spoken Language Understanding for call centers. It combines large\nvocabulary, real-time speech recognition, transcript refinement, and entity and\nintent recognition in order to convert live audio into a rich, actionable\nstream of structured events. These events can be further leveraged with a\nbusiness rules engine, thus serving as a foundation for real-time supervision\nand assistance applications. After the ingestion, calls are enriched with\nunsupervised keyword extraction, abstractive summarization, and\nbusiness-defined attributes, enabling offline use cases, such as business\nintelligence, topic mining, full-text search, quality assurance, and agent\ntraining. ACI comes with a pretrained, configurable library of hundreds of\nintents and a robust intent training environment that allows for efficient,\ncost-effective creation and customization of customer-specific intents.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.03434v1",
            "date": "2019-09-08 11:53:24+00:00",
            "title": "Order-free Learning Alleviating Exposure Bias in Multi-label Classification",
            "authors": [
                "Che-Ping Tsai",
                "Hung-Yi Lee"
            ],
            "abstract": "Multi-label classification (MLC) assigns multiple labels to each sample.\nPrior studies show that MLC can be transformed to a sequence prediction problem\nwith a recurrent neural network (RNN) decoder to model the label dependency.\nHowever, training a RNN decoder requires a predefined order of labels, which is\nnot directly available in the MLC specification. Besides, RNN thus trained\ntends to overfit the label combinations in the training set and have difficulty\ngenerating unseen label sequences. In this paper, we propose a new framework\nfor MLC which does not rely on a predefined label order and thus alleviates\nexposure bias. The experimental results on three multi-label classification\nbenchmark datasets show that our method outperforms competitive baselines by a\nlarge margin. We also find the proposed approach has a higher probability of\ngenerating label combinations not seen during training than the baseline\nmodels. The result shows that the proposed approach has better generalization\ncapability.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.04157v1",
            "date": "2019-09-09 21:11:35+00:00",
            "title": "Self-Teaching Networks",
            "authors": [
                "Liang Lu",
                "Eric Sun",
                "Yifan Gong"
            ],
            "abstract": "We propose self-teaching networks to improve the generalization capacity of\ndeep neural networks. The idea is to generate soft supervision labels using the\noutput layer for training the lower layers of the network. During the network\ntraining, we seek an auxiliary loss that drives the lower layer to mimic the\nbehavior of the output layer. The connection between the two network layers\nthrough the auxiliary loss can help the gradient flow, which works similar to\nthe residual networks. Furthermore, the auxiliary loss also works as a\nregularizer, which improves the generalization capacity of the network. We\nevaluated the self-teaching network with deep recurrent neural networks on\nspeech recognition tasks, where we trained the acoustic model using 30 thousand\nhours of data. We tested the acoustic model using data collected from 4\nscenarios. We show that the self-teaching network can achieve consistent\nimprovements and outperform existing methods such as label smoothing and\nconfidence penalization.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.10470v2",
            "date": "2019-09-23 16:47:15+00:00",
            "title": "Improving Generative Visual Dialog by Answering Diverse Questions",
            "authors": [
                "Vishvak Murahari",
                "Prithvijit Chattopadhyay",
                "Dhruv Batra",
                "Devi Parikh",
                "Abhishek Das"
            ],
            "abstract": "Prior work on training generative Visual Dialog models with reinforcement\nlearning(Das et al.) has explored a Qbot-Abot image-guessing game and shown\nthat this 'self-talk' approach can lead to improved performance at the\ndownstream dialog-conditioned image-guessing task. However, this improvement\nsaturates and starts degrading after a few rounds of interaction, and does not\nlead to a better Visual Dialog model. We find that this is due in part to\nrepeated interactions between Qbot and Abot during self-talk, which are not\ninformative with respect to the image. To improve this, we devise a simple\nauxiliary objective that incentivizes Qbot to ask diverse questions, thus\nreducing repetitions and in turn enabling Abot to explore a larger state space\nduring RL ie. be exposed to more visual concepts to talk about, and varied\nquestions to answer. We evaluate our approach via a host of automatic metrics\nand human studies, and demonstrate that it leads to better dialog, ie. dialog\nthat is more diverse (ie. less repetitive), consistent (ie. has fewer\nconflicting exchanges), fluent (ie. more human-like),and detailed, while still\nbeing comparably image-relevant as prior work and ablations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.10493v1",
            "date": "2019-09-23 17:25:50+00:00",
            "title": "Formalism for Supporting the Development of Verifiably Safe Medical Guidelines with Statecharts",
            "authors": [
                "Chunhui Guo",
                "Zhicheng Fu",
                "Zhenyu Zhang",
                "Shangping Ren",
                "Lui Sha"
            ],
            "abstract": "Improving the effectiveness and safety of patient care is the ultimate\nobjective for medical cyber-physical systems. Many medical best practice\nguidelines exist, but most of the existing guidelines in handbooks are\ndifficult for medical staff to remember and apply clinically. Furthermore,\nalthough the guidelines have gone through clinical validations, validations by\nmedical professionals alone do not provide guarantees for the safety of medical\ncyber-physical systems. Hence, formal verification is also needed. The paper\npresents the formal semantics for a framework that we developed to support the\ndevelopment of verifiably safe medical guidelines.\n  The framework allows computer scientists to work together with medical\nprofessionals to transform medical best practice guidelines into executable\nstatechart models, Yakindu in particular, so that medical functionalities and\nproperties can be quickly prototyped and validated. Existing formal\nverification technologies, UPPAAL timed automata in particular, is integrated\ninto the framework to provide formal verification capabilities to verify safety\nproperties. However, some components used/built into the framework, such as the\nopen-source Yakindu statecharts as well as the transformation rules from\nstatecharts to timed automata, do not have built-in semantics. The ambiguity\nbecomes unavoidable unless formal semantics is defined for the framework, which\nis what the paper is to present.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1909.10881v1",
            "date": "2019-09-21 03:15:04+00:00",
            "title": "Application of Fuzzy Clustering for Text Data Dimensionality Reduction",
            "authors": [
                "Amir Karami"
            ],
            "abstract": "Large textual corpora are often represented by the document-term frequency\nmatrix whose elements are the frequency of terms; however, this matrix has two\nproblems: sparsity and high dimensionality. Four dimension reduction strategies\nare used to address these problems. Of the four strategies, unsupervised\nfeature transformation (UFT) is a popular and efficient strategy to map the\nterms to a new basis in the document-term frequency matrix. Although several\nUFT-based methods have been developed, fuzzy clustering has not been considered\nfor dimensionality reduction. This research explores fuzzy clustering as a new\nUFT-based approach to create a lower-dimensional representation of documents.\nPerformance of fuzzy clustering with and without using global term weighting\nmethods is shown to exceed principal component analysis and singular value\ndecomposition. This study also explores the effect of applying different\nfuzzifier values on fuzzy clustering for dimensionality reduction purpose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1909.11886v1",
            "date": "2019-09-26 04:38:01+00:00",
            "title": "Self-Adaptive Soft Voice Activity Detection using Deep Neural Networks for Robust Speaker Verification",
            "authors": [
                "Youngmoon Jung",
                "Yeunju Choi",
                "Hoirin Kim"
            ],
            "abstract": "Voice activity detection (VAD), which classifies frames as speech or\nnon-speech, is an important module in many speech applications including\nspeaker verification. In this paper, we propose a novel method, called\nself-adaptive soft VAD, to incorporate a deep neural network (DNN)-based VAD\ninto a deep speaker embedding system. The proposed method is a combination of\nthe following two approaches. The first approach is soft VAD, which performs a\nsoft selection of frame-level features extracted from a speaker feature\nextractor. The frame-level features are weighted by their corresponding speech\nposteriors estimated from the DNN-based VAD, and then aggregated to generate a\nspeaker embedding. The second approach is self-adaptive VAD, which fine-tunes\nthe pre-trained VAD on the speaker verification data to reduce the domain\nmismatch. Here, we introduce two unsupervised domain adaptation (DA) schemes,\nnamely speech posterior-based DA (SP-DA) and joint learning-based DA (JL-DA).\nExperiments on a Korean speech database demonstrate that the verification\nperformance is improved significantly in real-world environments by using\nself-adaptive soft VAD.",
            "journal ref": "Proc. of ASRU 2019, pp. 365-372",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1909.12232v1",
            "date": "2019-09-19 09:51:35+00:00",
            "title": "A Comparison of Hybrid and End-to-End Models for Syllable Recognition",
            "authors": [
                "Sebastian P. Bayerl",
                "Korbinian Riedhammer"
            ],
            "abstract": "This paper presents a comparison of a traditional hybrid speech recognition\nsystem (kaldi using WFST and TDNN with lattice-free MMI) and a lexicon-free\nend-to-end (TensorFlow implementation of multi-layer LSTM with CTC training)\nmodels for German syllable recognition on the Verbmobil corpus. The results\nshow that explicitly modeling prior knowledge is still valuable in building\nrecognition systems. With a strong language model (LM) based on syllables, the\nstructured approach significantly outperforms the end-to-end model. The best\nword error rate (WER) regarding syllables was achieved using kaldi with a\n4-gram LM, modeling all syllables observed in the training set. It achieved\n10.0% WER w.r.t. the syllables, compared to the end-to-end approach where the\nbest WER was 27.53%. The work presented here has implications for building\nfuture recognition systems that operate independent of a large vocabulary, as\ntypically used in a tasks such as recognition of syllabic or agglutinative\nlanguages, out-of-vocabulary techniques, keyword search indexing and medical\nspeech processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.01992v3",
            "date": "2019-10-04 15:31:48+00:00",
            "title": "SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognition",
            "authors": [
                "Zhen Huang",
                "Tim Ng",
                "Leo Liu",
                "Henry Mason",
                "Xiaodan Zhuang",
                "Daben Liu"
            ],
            "abstract": "Very deep CNNs achieve state-of-the-art results in both computer vision and\nspeech recognition, but are difficult to train. The most popular way to train\nvery deep CNNs is to use shortcut connections (SC) together with batch\nnormalization (BN). Inspired by Self- Normalizing Neural Networks, we propose\nthe self-normalizing deep CNN (SNDCNN) based acoustic model topology, by\nremoving the SC/BN and replacing the typical RELU activations with scaled\nexponential linear unit (SELU) in ResNet-50. SELU activations make the network\nself-normalizing and remove the need for both shortcut connections and batch\nnormalization. Compared to ResNet- 50, we can achieve the same or lower (up to\n4.5% relative) word error rate (WER) while boosting both training and inference\nspeed by 60%-80%. We also explore other model inference optimization schemes to\nfurther reduce latency for production use.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.02290v1",
            "date": "2019-10-05 16:25:56+00:00",
            "title": "Few-shot tweet detection in emerging disaster events",
            "authors": [
                "Anna Kruspe"
            ],
            "abstract": "Social media sources can provide crucial information in crisis situations,\nbut discovering relevant messages is not trivial. Methods have so far focused\non universal detection models for all kinds of crises or for certain crisis\ntypes (e.g. floods). Event-specific models could implement a more focused\nsearch area, but collecting data and training new models for a crisis that is\nalready in progress is costly and may take too much time for a prompt response.\nAs a compromise, manually collecting a small amount of example messages is\nfeasible. Few-shot models can generalize to unseen classes with such a small\nhandful of examples, and do not need be trained anew for each event. We compare\nhow few-shot approaches (matching networks and prototypical networks) perform\nfor this task. Since this is essentially a one-class problem, we also\ndemonstrate how a modified one-class version of prototypical models can be used\nfor this application.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.03498v1",
            "date": "2019-10-07 06:49:52+00:00",
            "title": "SentiCite: An Approach for Publication Sentiment Analysis",
            "authors": [
                "Dominique Mercier",
                "Akansha Bhardwaj",
                "Andreas Dengel",
                "Sheraz Ahmed"
            ],
            "abstract": "With the rapid growth in the number of scientific publications, year after\nyear, it is becoming increasingly difficult to identify quality authoritative\nwork on a single topic. Though there is an availability of scientometric\nmeasures which promise to offer a solution to this problem, these measures are\nmostly quantitative and rely, for instance, only on the number of times an\narticle is cited. With this approach, it becomes irrelevant if an article is\ncited 10 times in a positive, negative or neutral way. In this context, it is\nquite important to study the qualitative aspect of a citation to understand its\nsignificance. This paper presents a novel system for sentiment analysis of\ncitations in scientific documents (SentiCite) and is also capable of detecting\nnature of citations by targeting the motivation behind a citation, e.g.,\nreference to a dataset, reading reference. Furthermore, the paper also presents\ntwo datasets (SentiCiteDB and IntentCiteDB) containing about 2,600 citations\nwith their ground truth for sentiment and nature of citation. SentiCite along\nwith other state-of-the-art methods for sentiment analysis are evaluated on the\npresented datasets. Evaluation results reveal that SentiCite outperforms\nstate-of-the-art methods for sentiment analysis in scientific publications by\nachieving a F1-measure of 0.71.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.07601v2",
            "date": "2019-10-16 20:36:41+00:00",
            "title": "Contextual Joint Factor Acoustic Embeddings",
            "authors": [
                "Yanpei Shi",
                "Thomas Hain"
            ],
            "abstract": "Embedding acoustic information into fixed length representations is of\ninterest for a whole range of applications in speech and audio technology. Two\nnovel unsupervised approaches to generate acoustic embeddings by modelling of\nacoustic context are proposed. The first approach is a contextual joint factor\nsynthesis encoder, where the encoder in an encoder/decoder framework is trained\nto extract joint factors from surrounding audio frames to best generate the\ntarget output. The second approach is a contextual joint factor analysis\nencoder, where the encoder is trained to analyse joint factors from the source\nsignal that correlates best with the neighbouring audio. To evaluate the\neffectiveness of our approaches compared to prior work, two tasks are conducted\n-- phone classification and speaker recognition -- and test on different TIMIT\ndata sets. Experimental results show that one of the proposed approaches\noutperforms phone classification baselines, yielding a classification accuracy\nof 74.1%. When using additional out-of-domain data for training, an additional\n3% improvements can be obtained, for both for phone classification and speaker\nrecognition tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08549v1",
            "date": "2019-10-18 15:20:38+00:00",
            "title": "Towards Learning Cross-Modal Perception-Trace Models",
            "authors": [
                "Achim Rettinger",
                "Viktoria Bogdanova",
                "Philipp Niemann"
            ],
            "abstract": "Representation learning is a key element of state-of-the-art deep learning\napproaches. It enables to transform raw data into structured vector space\nembeddings. Such embeddings are able to capture the distributional semantics of\ntheir context, e.g. by word windows on natural language sentences, graph walks\non knowledge graphs or convolutions on images. So far, this context is manually\ndefined, resulting in heuristics which are solely optimized for computational\nperformance on certain tasks like link-prediction. However, such heuristic\nmodels of context are fundamentally different to how humans capture\ninformation. For instance, when reading a multi-modal webpage (i) humans do not\nperceive all parts of a document equally: Some words and parts of images are\nskipped, others are revisited several times which makes the perception trace\nhighly non-sequential; (ii) humans construct meaning from a document's content\nby shifting their attention between text and image, among other things, guided\nby layout and design elements. In this paper we empirically investigate the\ndifference between human perception and context heuristics of basic embedding\nmodels. We conduct eye tracking experiments to capture the underlying\ncharacteristics of human perception of media documents containing a mixture of\ntext and images. Based on that, we devise a prototypical computational\nperception-trace model, called CMPM. We evaluate empirically how CMPM can\nimprove a basic skip-gram embedding approach. Our results suggest, that even\nwith a basic human-inspired computational perception model, there is a huge\npotential for improving embeddings since such a model does inherently capture\nmultiple modalities, as well as layout and design elements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.08955v2",
            "date": "2019-10-20 11:54:05+00:00",
            "title": "Computer-supported Analysis of Positive Properties, Ultrafilters and Modal Collapse in Variants of G\u00f6del's Ontological Argument",
            "authors": [
                "Christoph Benzm\u00fcller",
                "David Fuenmayor"
            ],
            "abstract": "Three variants of Kurt G\\\"odel's ontological argument, proposed by Dana\nScott, C. Anthony Anderson and Melvin Fitting, are encoded and rigorously\nassessed on the computer. In contrast to Scott's version of G\\\"odel's argument\nthe two variants contributed by Anderson and Fitting avoid modal collapse.\nAlthough they appear quite different on a cursory reading they are in fact\nclosely related. This has been revealed in the computer-supported formal\nanalysis presented in this article. Key to our formal analysis is the\nutilization of suitably adapted notions of (modal) ultrafilters, and a careful\ndistinction between extensions and intensions of positive properties.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/1910.09664v1",
            "date": "2019-10-21 21:19:33+00:00",
            "title": "Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight",
            "authors": [
                "Valts Blukis",
                "Yannick Terme",
                "Eyvind Niklasson",
                "Ross A. Knepper",
                "Yoav Artzi"
            ],
            "abstract": "We propose a joint simulation and real-world learning framework for mapping\nnavigation instructions and raw first-person observations to continuous\ncontrol. Our model estimates the need for environment exploration, predicts the\nlikelihood of visiting environment positions during execution, and controls the\nagent to both explore and visit high-likelihood positions. We introduce\nSupervised Reinforcement Asynchronous Learning (SuReAL). Learning uses both\nsimulation and real environments without requiring autonomous flight in the\nphysical environment during training, and combines supervised learning for\npredicting positions to visit and reinforcement learning for continuous\ncontrol. We evaluate our approach on a natural language instruction-following\ntask with a physical quadcopter, and demonstrate effective execution and\nexploration behavior.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/1910.09703v2",
            "date": "2019-10-22 00:09:22+00:00",
            "title": "Discriminative Neural Clustering for Speaker Diarisation",
            "authors": [
                "Qiujia Li",
                "Florian L. Kreyssig",
                "Chao Zhang",
                "Philip C. Woodland"
            ],
            "abstract": "In this paper, we propose Discriminative Neural Clustering (DNC) that\nformulates data clustering with a maximum number of clusters as a supervised\nsequence-to-sequence learning problem. Compared to traditional unsupervised\nclustering algorithms, DNC learns clustering patterns from training data\nwithout requiring an explicit definition of a similarity measure. An\nimplementation of DNC based on the Transformer architecture is shown to be\neffective on a speaker diarisation task using the challenging AMI dataset.\nSince AMI contains only 147 complete meetings as individual input sequences,\ndata scarcity is a significant issue for training a Transformer model for DNC.\nAccordingly, this paper proposes three data augmentation schemes: sub-sequence\nrandomisation, input vector randomisation, and Diaconis augmentation, which\ngenerates new data samples by rotating the entire input sequence of\nL2-normalised speaker embeddings. Experimental results on AMI show that DNC\nachieves a reduction in speaker error rate (SER) of 29.4% relative to spectral\nclustering.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1910.09804v2",
            "date": "2019-10-22 07:49:21+00:00",
            "title": "Two-Step Sound Source Separation: Training on Learned Latent Targets",
            "authors": [
                "Efthymios Tzinis",
                "Shrikant Venkataramani",
                "Zhepei Wang",
                "Cem Subakan",
                "Paris Smaragdis"
            ],
            "abstract": "In this paper, we propose a two-step training procedure for source separation\nvia a deep neural network. In the first step we learn a transform (and it's\ninverse) to a latent space where masking-based separation performance using\noracles is optimal. For the second step, we train a separation module that\noperates on the previously learned space. In order to do so, we also make use\nof a scale-invariant signal to distortion ratio (SI-SDR) loss function that\nworks in the latent space, and we prove that it lower-bounds the SI-SDR in the\ntime domain. We run various sound separation experiments that show how this\napproach can obtain better performance as compared to systems that learn the\ntransform and the separation module jointly. The proposed methodology is\ngeneral enough to be applicable to a large class of neural network end-to-end\nseparation systems.",
            "journal ref": "ICASSP 2020 - 2020 IEEE International Conference on Acoustics,\n  Speech and Signal Processing (ICASSP)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.11424v3",
            "date": "2019-10-24 21:06:38+00:00",
            "title": "Capacity, Bandwidth, and Compositionality in Emergent Language Learning",
            "authors": [
                "Cinjon Resnick",
                "Abhinav Gupta",
                "Jakob Foerster",
                "Andrew M. Dai",
                "Kyunghyun Cho"
            ],
            "abstract": "Many recent works have discussed the propensity, or lack thereof, for\nemergent languages to exhibit properties of natural languages. A favorite in\nthe literature is learning compositionality. We note that most of those works\nhave focused on communicative bandwidth as being of primary importance. While\nimportant, it is not the only contributing factor. In this paper, we\ninvestigate the learning biases that affect the efficacy and compositionality\nof emergent languages. Our foremost contribution is to explore how capacity of\na neural network impacts its ability to learn a compositional language. We\nadditionally introduce a set of evaluation metrics with which we analyze the\nlearned languages. Our hypothesis is that there should be a specific range of\nmodel capacity and channel bandwidth that induces compositional structure in\nthe resulting language and consequently encourages systematic generalization.\nWhile we empirically see evidence for the bottom of this range, we curiously do\nnot find evidence for the top part of the range and believe that this is an\nopen question for the community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1910.12611v4",
            "date": "2019-10-23 02:10:42+00:00",
            "title": "Suicidal Ideation Detection: A Review of Machine Learning Methods and Applications",
            "authors": [
                "Shaoxiong Ji",
                "Shirui Pan",
                "Xue Li",
                "Erik Cambria",
                "Guodong Long",
                "Zi Huang"
            ],
            "abstract": "Suicide is a critical issue in modern society. Early detection and prevention\nof suicide attempts should be addressed to save people's life. Current suicidal\nideation detection methods include clinical methods based on the interaction\nbetween social workers or experts and the targeted individuals and machine\nlearning techniques with feature engineering or deep learning for automatic\ndetection based on online social contents. This paper is the first survey that\ncomprehensively introduces and discusses the methods from these categories.\nDomain-specific applications of suicidal ideation detection are reviewed\naccording to their data sources, i.e., questionnaires, electronic health\nrecords, suicide notes, and online user content. Several specific tasks and\ndatasets are introduced and summarized to facilitate further research. Finally,\nwe summarize the limitations of current work and provide an outlook of further\nresearch directions.",
            "journal ref": "IEEE Transactions on Computational Social Systems, 8(1), 2021, pp.\n  214-226",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/1910.13212v1",
            "date": "2019-10-29 11:49:30+00:00",
            "title": "Privacy Enhanced Multimodal Neural Representations for Emotion Recognition",
            "authors": [
                "Mimansa Jaiswal",
                "Emily Mower Provost"
            ],
            "abstract": "Many mobile applications and virtual conversational agents now aim to\nrecognize and adapt to emotions. To enable this, data are transmitted from\nusers' devices and stored on central servers. Yet, these data contain sensitive\ninformation that could be used by mobile applications without user's consent\nor, maliciously, by an eavesdropping adversary. In this work, we show how\nmultimodal representations trained for a primary task, here emotion\nrecognition, can unintentionally leak demographic information, which could\noverride a selected opt-out option by the user. We analyze how this leakage\ndiffers in representations obtained from textual, acoustic, and multimodal\ndata. We use an adversarial learning paradigm to unlearn the private\ninformation present in a representation and investigate the effect of varying\nthe strength of the adversarial component on the primary task and on the\nprivacy metric, defined here as the inability of an attacker to predict\nspecific demographic information. We evaluate this paradigm on multiple\ndatasets and show that we can improve the privacy metric while not\nsignificantly impacting the performance on the primary task. To the best of our\nknowledge, this is the first work to analyze how the privacy metric differs\nacross modalities and how multiple privacy concerns can be tackled while still\nmaintaining performance on emotion recognition.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.00523v1",
            "date": "2019-11-01 18:00:05+00:00",
            "title": "What Gets Echoed? Understanding the \"Pointers\" in Explanations of Persuasive Arguments",
            "authors": [
                "David Atkinson",
                "Kumar Bhargav Srinivasan",
                "Chenhao Tan"
            ],
            "abstract": "Explanations are central to everyday life, and are a topic of growing\ninterest in the AI community. To investigate the process of providing natural\nlanguage explanations, we leverage the dynamics of the /r/ChangeMyView\nsubreddit to build a dataset with 36K naturally occurring explanations of why\nan argument is persuasive. We propose a novel word-level prediction task to\ninvestigate how explanations selectively reuse, or echo, information from what\nis being explained (henceforth, explanandum). We develop features to capture\nthe properties of a word in the explanandum, and show that our proposed\nfeatures not only have relatively strong predictive power on the echoing of a\nword in an explanation, but also enhance neural methods of generating\nexplanations. In particular, while the non-contextual properties of a word\nitself are more valuable for stopwords, the interaction between the constituent\nparts of an explanandum is crucial in predicting the echoing of content words.\nWe also find intriguing patterns of a word being echoed. For example, although\nnouns are generally less likely to be echoed, subjects and objects can,\ndepending on their source, be more likely to be echoed in the explanations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01485v1",
            "date": "2019-11-04 20:57:54+00:00",
            "title": "Assessing Social and Intersectional Biases in Contextualized Word Representations",
            "authors": [
                "Yi Chern Tan",
                "L. Elisa Celis"
            ],
            "abstract": "Social bias in machine learning has drawn significant attention, with work\nranging from demonstrations of bias in a multitude of applications, curating\ndefinitions of fairness for different contexts, to developing algorithms to\nmitigate bias. In natural language processing, gender bias has been shown to\nexist in context-free word embeddings. Recently, contextual word\nrepresentations have outperformed word embeddings in several downstream NLP\ntasks. These word representations are conditioned on their context within a\nsentence, and can also be used to encode the entire sentence. In this paper, we\nanalyze the extent to which state-of-the-art models for contextual word\nrepresentations, such as BERT and GPT-2, encode biases with respect to gender,\nrace, and intersectional identities. Towards this, we propose assessing bias at\nthe contextual word level. This novel approach captures the contextual effects\nof bias missing in context-free word embeddings, yet avoids confounding effects\nthat underestimate bias at the sentence encoding level. We demonstrate evidence\nof bias at the corpus level, find varying evidence of bias in embedding\nassociation tests, show in particular that racial bias is strongly encoded in\ncontextual word models, and observe that bias effects for intersectional\nminorities are exacerbated beyond their constituent minority identities.\nFurther, evaluating bias effects at the contextual word level captures biases\nthat are not captured at the sentence level, confirming the need for our novel\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.01803v1",
            "date": "2019-10-30 04:11:29+00:00",
            "title": "Temporal Feedback Convolutional Recurrent Neural Networks for Keyword Spotting",
            "authors": [
                "Taejun Kim",
                "Juhan Nam"
            ],
            "abstract": "While end-to-end learning has become a trend in deep learning, the model\narchitecture is often designed to incorporate domain knowledge. We propose a\nnovel convolutional recurrent neural network (CRNN) architecture with temporal\nfeedback connections, inspired by the feedback pathways from the brain to ears\nin the human auditory system. The proposed architecture uses a hidden state of\nthe RNN module at the previous time to control the sensitivity of channel-wise\nfeature activations in the CNN blocks at the current time, which is analogous\nto the mechanism of the outer hair-cell. We apply the proposed model to keyword\nspotting where the speech commands have sequential nature. We show the proposed\nmodel consistently outperforms the compared model without temporal feedback for\ndifferent input/output settings in the CRNN framework. We also investigate the\ndetails of the performance improvement by conducting a failure analysis of the\nkeyword spotting task and a visualization of the channel-wise feature scaling\nin each CNN block.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.03078v2",
            "date": "2019-11-08 06:38:14+00:00",
            "title": "Adversarial Attacks on GMM i-vector based Speaker Verification Systems",
            "authors": [
                "Xu Li",
                "Jinghua Zhong",
                "Xixin Wu",
                "Jianwei Yu",
                "Xunying Liu",
                "Helen Meng"
            ],
            "abstract": "This work investigates the vulnerability of Gaussian Mixture Model (GMM)\ni-vector based speaker verification systems to adversarial attacks, and the\ntransferability of adversarial samples crafted from GMM i-vector based systems\nto x-vector based systems. In detail, we formulate the GMM i-vector system as a\nscoring function of enrollment and testing utterance pairs. Then we leverage\nthe fast gradient sign method (FGSM) to optimize testing utterances for\nadversarial samples generation. These adversarial samples are used to attack\nboth GMM i-vector and x-vector systems. We measure the system vulnerability by\nthe degradation of equal error rate and false acceptance rate. Experiment\nresults show that GMM i-vector systems are seriously vulnerable to adversarial\nattacks, and the crafted adversarial samples prove to be transferable and pose\nthreats to neuralnetwork speaker embedding based systems (e.g. x-vector\nsystems).",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.04890v1",
            "date": "2019-11-08 22:01:42+00:00",
            "title": "Recurrent Neural Network Transducer for Audio-Visual Speech Recognition",
            "authors": [
                "Takaki Makino",
                "Hank Liao",
                "Yannis Assael",
                "Brendan Shillingford",
                "Basilio Garcia",
                "Otavio Braga",
                "Olivier Siohan"
            ],
            "abstract": "This work presents a large-scale audio-visual speech recognition system based\non a recurrent neural network transducer (RNN-T) architecture. To support the\ndevelopment of such a system, we built a large audio-visual (A/V) dataset of\nsegmented utterances extracted from YouTube public videos, leading to 31k hours\nof audio-visual training content. The performance of an audio-only,\nvisual-only, and audio-visual system are compared on two large-vocabulary test\nsets: a set of utterance segments from public YouTube videos called YTDEV18 and\nthe publicly available LRS3-TED set. To highlight the contribution of the\nvisual modality, we also evaluated the performance of our system on the YTDEV18\nset artificially corrupted with background noise and overlapping speech. To the\nbest of our knowledge, our system significantly improves the state-of-the-art\non the LRS3-TED set.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.07523v1",
            "date": "2019-11-18 10:16:38+00:00",
            "title": "Fine-Grained Static Detection of Obfuscation Transforms Using Ensemble-Learning and Semantic Reasoning",
            "authors": [
                "Ramtine Tofighi-Shirazi",
                "Irina Mariuca Asavoae",
                "Philippe Elbaz-Vincent"
            ],
            "abstract": "The ability to efficiently detect the software protections used is at a prime\nto facilitate the selection and application of adequate deob-fuscation\ntechniques. We present a novel approach that combines semantic reasoning\ntechniques with ensemble learning classification for the purpose of providing a\nstatic detection framework for obfuscation transformations. By contrast to\nexisting work, we provide a methodology that can detect multiple layers of\nobfuscation, without depending on knowledge of the underlying functionality of\nthe training-set used. We also extend our work to detect constructions of\nobfuscation transformations, thus providing a fine-grained methodology. To that\nend, we provide several studies for the best practices of the use of machine\nlearning techniques for a scalable and efficient model. According to our\nexperimental results and evaluations on obfuscators such as Tigress and OLLVM,\nour models have up to 91% accuracy on state-of-the-art obfuscation\ntransformations. Our overall accuracies for their constructions are up to 100%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1911.08332v1",
            "date": "2019-11-19 15:07:07+00:00",
            "title": "Neural Network based End-to-End Query by Example Spoken Term Detection",
            "authors": [
                "Dhananjay Ram",
                "Lesly Miculicich",
                "Herv\u00e9 Bourlard"
            ],
            "abstract": "This paper focuses on the problem of query by example spoken term detection\n(QbE-STD) in zero-resource scenario. State-of-the-art approaches primarily rely\non dynamic time warping (DTW) based template matching techniques using phone\nposterior or bottleneck features extracted from a deep neural network (DNN). We\nuse both monolingual and multilingual bottleneck features, and show that\nmultilingual features perform increasingly better with more training languages.\nPreviously, it has been shown that the DTW based matching can be replaced with\na CNN based matching while using posterior features. Here, we show that the CNN\nbased matching outperforms DTW based matching using bottleneck features as\nwell. In this case, the feature extraction and pattern matching stages of our\nQbE-STD system are optimized independently of each other. We propose to\nintegrate these two stages in a fully neural network based end-to-end learning\nframework to enable joint optimization of those two stages simultaneously. The\nproposed approaches are evaluated on two challenging multilingual datasets:\nSpoken Web Search 2013 and Query by Example Search on Speech Task 2014,\ndemonstrating in each case significant improvements.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1911.12760v2",
            "date": "2019-11-28 15:57:14+00:00",
            "title": "Using VAEs and Normalizing Flows for One-shot Text-To-Speech Synthesis of Expressive Speech",
            "authors": [
                "Vatsal Aggarwal",
                "Marius Cotescu",
                "Nishant Prateek",
                "Jaime Lorenzo-Trueba",
                "Roberto Barra-Chicote"
            ],
            "abstract": "We propose a Text-to-Speech method to create an unseen expressive style using\none utterance of expressive speech of around one second. Specifically, we\nenhance the disentanglement capabilities of a state-of-the-art\nsequence-to-sequence based system with a Variational AutoEncoder (VAE) and a\nHouseholder Flow. The proposed system provides a 22% KL-divergence reduction\nwhile jointly improving perceptual metrics over state-of-the-art. At synthesis\ntime we use one example of expressive style as a reference input to the encoder\nfor generating any text in the desired style. Perceptual MUSHRA evaluations\nshow that we can create a voice with a 9% relative naturalness improvement over\nstandard Neural Text-to-Speech, while also improving the perceived emotional\nintensity (59 compared to the 55 of neutral speech).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.01111v1",
            "date": "2019-11-22 16:07:02+00:00",
            "title": "Use of Artificial Intelligence to Analyse Risk in Legal Documents for a Better Decision Support",
            "authors": [
                "Dipankar Chakrabarti",
                "Neelam Patodia",
                "Udayan Bhattacharya",
                "Indranil Mitra",
                "Satyaki Roy",
                "Jayanta Mandi",
                "Nandini Roy",
                "Prasun Nandy"
            ],
            "abstract": "Assessing risk for voluminous legal documents such as request for proposal;\ncontracts is tedious and error prone. We have developed \"risk-o-meter\", a\nframework, based on machine learning and natural language processing to review\nand assess risks of any legal document. Our framework uses Paragraph Vector, an\nunsupervised model to generate vector representation of text. This enables the\nframework to learn contextual relations of legal terms and generate sensible\ncontext aware embedding. The framework then feeds the vector space into a\nsupervised classification algorithm to predict whether a paragraph belongs to a\nper-defined risk category or not. The framework thus extracts risk prone\nparagraphs. This technique efficiently overcomes the limitations of\nkeyword-based search. We have achieved an accuracy of 91% for the risk category\nhaving the largest training dataset. This framework will help organizations\noptimize effort to identify risk from large document base with minimal human\nintervention and thus will help to have risk mitigated sustainable growth. Its\nmachine learning capability makes it scalable to uncover relevant information\nfrom any type of document apart from legal documents, provided the library is\nper-populated and rich.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1912.01734v2",
            "date": "2019-12-03 23:18:59+00:00",
            "title": "ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks",
            "authors": [
                "Mohit Shridhar",
                "Jesse Thomason",
                "Daniel Gordon",
                "Yonatan Bisk",
                "Winson Han",
                "Roozbeh Mottaghi",
                "Luke Zettlemoyer",
                "Dieter Fox"
            ],
            "abstract": "We present ALFRED (Action Learning From Realistic Environments and\nDirectives), a benchmark for learning a mapping from natural language\ninstructions and egocentric vision to sequences of actions for household tasks.\nALFRED includes long, compositional tasks with non-reversible state changes to\nshrink the gap between research benchmarks and real-world applications. ALFRED\nconsists of expert demonstrations in interactive visual environments for 25k\nnatural language directives. These directives contain both high-level goals\nlike \"Rinse off a mug and place it in the coffee maker.\" and low-level language\ninstructions like \"Walk to the coffee maker on the right.\" ALFRED tasks are\nmore complex in terms of sequence length, action space, and language than\nexisting vision-and-language task datasets. We show that a baseline model based\non recent embodied vision-and-language tasks performs poorly on ALFRED,\nsuggesting that there is significant room for developing innovative grounded\nvisual language understanding models with this benchmark.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1912.02610v1",
            "date": "2019-11-29 23:25:20+00:00",
            "title": "Bimodal Speech Emotion Recognition Using Pre-Trained Language Models",
            "authors": [
                "Verena Heusser",
                "Niklas Freymuth",
                "Stefan Constantin",
                "Alex Waibel"
            ],
            "abstract": "Speech emotion recognition is a challenging task and an important step\ntowards more natural human-machine interaction. We show that pre-trained\nlanguage models can be fine-tuned for text emotion recognition, achieving an\naccuracy of 69.5% on Task 4A of SemEval 2017, improving upon the previous state\nof the art by over 3% absolute. We combine these language models with speech\nemotion recognition, achieving results of 73.5% accuracy when using provided\ntranscriptions and speech data on a subset of four classes of the IEMOCAP\ndataset. The use of noise-induced transcriptions and speech data results in an\naccuracy of 71.4%. For our experiments, we created IEmoNet, a modular and\nadaptable bimodal framework for speech emotion recognition based on pre-trained\nlanguage models. Lastly, we discuss the idea of using an emotional classifier\nas a reward for reinforcement learning as a step towards more successful and\nconvenient human-machine interaction.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.04370v1",
            "date": "2019-12-04 19:48:54+00:00",
            "title": "Cross-Language Aphasia Detection using Optimal Transport Domain Adaptation",
            "authors": [
                "Aparna Balagopalan",
                "Jekaterina Novikova",
                "Matthew B. A. McDermott",
                "Bret Nestor",
                "Tristan Naumann",
                "Marzyeh Ghassemi"
            ],
            "abstract": "Multi-language speech datasets are scarce and often have small sample sizes\nin the medical domain. Robust transfer of linguistic features across languages\ncould improve rates of early diagnosis and therapy for speakers of low-resource\nlanguages when detecting health conditions from speech. We utilize\nout-of-domain, unpaired, single-speaker, healthy speech data for training\nmultiple Optimal Transport (OT) domain adaptation systems. We learn mappings\nfrom other languages to English and detect aphasia from linguistic\ncharacteristics of speech, and show that OT domain adaptation improves aphasia\ndetection over unilingual baselines for French (6% increased F1) and Mandarin\n(5% increased F1). Further, we show that adding aphasic data to the domain\nadaptation system significantly increases performance for both French and\nMandarin, increasing the F1 scores further (10% and 8% increase in F1 scores\nfor French and Mandarin, respectively, over unilingual baselines).",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.04979v1",
            "date": "2019-12-10 20:59:24+00:00",
            "title": "Advances in Online Audio-Visual Meeting Transcription",
            "authors": [
                "Takuya Yoshioka",
                "Igor Abramovski",
                "Cem Aksoylar",
                "Zhuo Chen",
                "Moshe David",
                "Dimitrios Dimitriadis",
                "Yifan Gong",
                "Ilya Gurvich",
                "Xuedong Huang",
                "Yan Huang",
                "Aviv Hurvitz",
                "Li Jiang",
                "Sharon Koubi",
                "Eyal Krupka",
                "Ido Leichter",
                "Changliang Liu",
                "Partha Parthasarathy",
                "Alon Vinnikov",
                "Lingfeng Wu",
                "Xiong Xiao",
                "Wayne Xiong",
                "Huaming Wang",
                "Zhenghao Wang",
                "Jun Zhang",
                "Yong Zhao",
                "Tianyan Zhou"
            ],
            "abstract": "This paper describes a system that generates speaker-annotated transcripts of\nmeetings by using a microphone array and a 360-degree camera. The hallmark of\nthe system is its ability to handle overlapped speech, which has been an\nunsolved problem in realistic settings for over a decade. We show that this\nproblem can be addressed by using a continuous speech separation approach. In\naddition, we describe an online audio-visual speaker diarization method that\nleverages face tracking and identification, sound source localization, speaker\nidentification, and, if available, prior speaker information for robustness to\nvarious real world challenges. All components are integrated in a meeting\ntranscription framework called SRD, which stands for \"separate, recognize, and\ndiarize\". Experimental results using recordings of natural meetings involving\nup to 11 attendees are reported. The continuous speech separation improves a\nword error rate (WER) by 16.1% compared with a highly tuned beamformer. When a\ncomplete list of meeting attendees is available, the discrepancy between WER\nand speaker-attributed WER is only 1.0%, indicating accurate word-to-speaker\nassociation. This increases marginally to 1.6% when 50% of the attendees are\nunknown to the system.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1912.07911v1",
            "date": "2019-12-17 10:29:26+00:00",
            "title": "A Heterogeneous Graphical Model to Understand User-Level Sentiments in Social Media",
            "authors": [
                "Rahul Radhakrishnan Iyer",
                "Jing Chen",
                "Haonan Sun",
                "Keyang Xu"
            ],
            "abstract": "Social Media has seen a tremendous growth in the last decade and is\ncontinuing to grow at a rapid pace. With such adoption, it is increasingly\nbecoming a rich source of data for opinion mining and sentiment analysis. The\ndetection and analysis of sentiment in social media is thus a valuable topic\nand attracts a lot of research efforts. Most of the earlier efforts focus on\nsupervised learning approaches to solve this problem, which require expensive\nhuman annotations and therefore limits their practical use. In our work, we\npropose a semi-supervised approach to predict user-level sentiments for\nspecific topics. We define and utilize a heterogeneous graph built from the\nsocial networks of the users with the knowledge that connected users in social\nnetworks typically share similar sentiments. Compared with the previous works,\nwe have several novelties: (1) we incorporate the influences/authoritativeness\nof the users into the model, 2) we include comment-based and like-based\nuser-user links to the graph, 3) we superimpose multiple heterogeneous graphs\ninto one thereby allowing multiple types of links to exist between two users.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.08868v1",
            "date": "2019-12-18 20:11:03+00:00",
            "title": "Topic subject creation using unsupervised learning for topic modeling",
            "authors": [
                "Rashid Mehdiyev",
                "Jean Nava",
                "Karan Sodhi",
                "Saurav Acharya",
                "Annie Ibrahim Rana"
            ],
            "abstract": "We describe the use of Non-Negative Matrix Factorization (NMF) and Latent\nDirichlet Allocation (LDA) algorithms to perform topic mining and labelling\napplied to retail customer communications in attempt to characterize the\nsubject of customers inquiries. In this paper we compare both algorithms in the\ntopic mining performance and propose methods to assign topic subject labels in\nan automated way.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1912.10170v1",
            "date": "2019-12-15 14:37:06+00:00",
            "title": "Na\u00efveRole: Author-Contribution Extraction and Parsing from Biomedical Manuscripts",
            "authors": [
                "Dominika Tkaczyk",
                "Andrew Collins",
                "Joeran Beel"
            ],
            "abstract": "Information about the contributions of individual authors to scientific\npublications is important for assessing authors' achievements. Some biomedical\npublications have a short section that describes authors' roles and\ncontributions. It is usually written in natural language and hence author\ncontributions cannot be trivially extracted in a machine-readable format. In\nthis paper, we present 1) A statistical analysis of roles in author\ncontributions sections, and 2) Na\\\"iveRole, a novel approach to extract\nstructured authors' roles from author contribution sections. For the first\npart, we used co-clustering techniques, as well as Open Information Extraction,\nto semi-automatically discover the popular roles within a corpus of 2,000\ncontributions sections from PubMed Central. The discovered roles were used to\nautomatically build a training set for Na\\\"iveRole, our role extractor\napproach, based on Na\\\"ive Bayes. Na\\\"iveRole extracts roles with a\nmicro-averaged precision of 0.68, recall of 0.48 and F1 of 0.57. It is, to the\nbest of our knowledge, the first attempt to automatically extract author roles\nfrom research papers. This paper is an extended version of a previous poster\npublished at JCDL 2018.",
            "journal ref": "27th AIAI Irish Conference on Artificial Intelligence and\n  Cognitive Science, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00051v1",
            "date": "2019-12-24 17:09:54+00:00",
            "title": "Simultaneous Identification of Tweet Purpose and Position",
            "authors": [
                "Rahul Radhakrishnan Iyer",
                "Yulong Pei",
                "Katia Sycara"
            ],
            "abstract": "Tweet classification has attracted considerable attention recently. Most of\nthe existing work on tweet classification focuses on topic classification,\nwhich classifies tweets into several predefined categories, and sentiment\nclassification, which classifies tweets into positive, negative and neutral.\nSince tweets are different from conventional text in that they generally are of\nlimited length and contain informal, irregular or new words, so it is difficult\nto determine user intention to publish a tweet and user attitude towards\ncertain topic. In this paper, we aim to simultaneously classify tweet purpose,\ni.e., the intention for user to publish a tweet, and position, i.e.,\nsupporting, opposing or being neutral to a given topic. By transforming this\nproblem to a multi-label classification problem, a multi-label classification\nmethod with post-processing is proposed. Experiments on real-world data sets\ndemonstrate the effectiveness of this method and the results outperform the\nindividual classification methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1901.10826v1",
            "date": "2019-01-28 16:16:34+00:00",
            "title": "Additive Margin SincNet for Speaker Recognition",
            "authors": [
                "Jo\u00e3o Ant\u00f4nio Chagas Nunes",
                "David Mac\u00eado",
                "Cleber Zanchettin"
            ],
            "abstract": "Speaker Recognition is a challenging task with essential applications such as\nauthentication, automation, and security. The SincNet is a new deep learning\nbased model which has produced promising results to tackle the mentioned task.\nTo train deep learning systems, the loss function is essential to the network\nperformance. The Softmax loss function is a widely used function in deep\nlearning methods, but it is not the best choice for all kind of problems. For\ndistance-based problems, one new Softmax based loss function called Additive\nMargin Softmax (AM-Softmax) is proving to be a better choice than the\ntraditional Softmax. The AM-Softmax introduces a margin of separation between\nthe classes that forces the samples from the same class to be closer to each\nother and also maximizes the distance between classes. In this paper, we\npropose a new approach for speaker recognition systems called AM-SincNet, which\nis based on the SincNet but uses an improved AM-Softmax layer. The proposed\nmethod is evaluated in the TIMIT dataset and obtained an improvement of\napproximately 40% in the Frame Error Rate compared to SincNet.",
            "journal ref": "2019 International Joint Conference on Neural Networks (IJCNN)",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1904.05576v1",
            "date": "2019-04-11 08:37:43+00:00",
            "title": "STC Antispoofing Systems for the ASVspoof2019 Challenge",
            "authors": [
                "Galina Lavrentyeva",
                "Sergey Novoselov",
                "Andzhukaev Tseren",
                "Marina Volkova",
                "Artem Gorlanov",
                "Alexandr Kozlov"
            ],
            "abstract": "This paper describes the Speech Technology Center (STC) antispoofing systems\nsubmitted to the ASVspoof 2019 challenge. The ASVspoof2019 is the extended\nversion of the previous challenges and includes 2 evaluation conditions:\nlogical access use-case scenario with speech synthesis and voice conversion\nattack types and physical access use-case scenario with replay attacks. During\nthe challenge we developed anti-spoofing solutions for both scenarios. The\nproposed systems are implemented using deep learning approach and are based on\ndifferent types of acoustic features. We enhanced Light CNN architecture\npreviously considered by the authors for replay attacks detection and which\nperformed high spoofing detection quality during the ASVspoof2017 challenge. In\nparticular here we investigate the efficiency of angular margin based softmax\nactivation for training robust deep Light CNN classifier to solve the\nmentioned-above tasks. Submitted systems achieved EER of 1.86% in logical\naccess scenario and 0.54% in physical access scenario on the evaluation part of\nthe Challenge corpora. High performance obtained for the unknown types of\nspoofing attacks demonstrates the stability of the offered approach in both\nevaluation conditions.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/1904.05876v1",
            "date": "2019-04-11 17:59:51+00:00",
            "title": "A Simple Baseline for Audio-Visual Scene-Aware Dialog",
            "authors": [
                "Idan Schwartz",
                "Alexander Schwing",
                "Tamir Hazan"
            ],
            "abstract": "The recently proposed audio-visual scene-aware dialog task paves the way to a\nmore data-driven way of learning virtual assistants, smart speakers and car\nnavigation systems. However, very little is known to date about how to\neffectively extract meaningful information from a plethora of sensors that\npound the computational engine of those devices. Therefore, in this paper, we\nprovide and carefully analyze a simple baseline for audio-visual scene-aware\ndialog which is trained end-to-end. Our method differentiates in a data-driven\nmanner useful signals from distracting ones using an attention mechanism. We\nevaluate the proposed approach on the recently introduced and challenging\naudio-visual scene-aware dataset, and demonstrate the key features that permit\nto outperform the current state-of-the-art by more than 20\\% on CIDEr.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/1906.02125v2",
            "date": "2019-05-14 13:44:37+00:00",
            "title": "Strong and Simple Baselines for Multimodal Utterance Embeddings",
            "authors": [
                "Paul Pu Liang",
                "Yao Chong Lim",
                "Yao-Hung Hubert Tsai",
                "Ruslan Salakhutdinov",
                "Louis-Philippe Morency"
            ],
            "abstract": "Human language is a rich multimodal signal consisting of spoken words, facial\nexpressions, body gestures, and vocal intonations. Learning representations for\nthese spoken utterances is a complex research problem due to the presence of\nmultiple heterogeneous sources of information. Recent advances in multimodal\nlearning have followed the general trend of building more complex models that\nutilize various attention, memory and recurrent components. In this paper, we\npropose two simple but strong baselines to learn embeddings of multimodal\nutterances. The first baseline assumes a conditional factorization of the\nutterance into unimodal factors. Each unimodal factor is modeled using the\nsimple form of a likelihood function obtained via a linear transformation of\nthe embedding. We show that the optimal embedding can be derived in closed form\nby taking a weighted average of the unimodal features. In order to capture\nricher representations, our second baseline extends the first by factorizing\ninto unimodal, bimodal, and trimodal factors, while retaining simplicity and\nefficiency during learning and inference. From a set of experiments across two\ntasks, we show strong performance on both supervised and semi-supervised\nmultimodal prediction, as well as significant (10 times) speedups over neural\nmodels during inference. Overall, we believe that our strong baseline models\noffer new benchmarking options for future research in multimodal learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1906.04301v1",
            "date": "2019-06-10 22:17:08+00:00",
            "title": "Transfer Learning for Ultrasound Tongue Contour Extraction with Different Domains",
            "authors": [
                "M. Hamed Mozaffari",
                "Won-Sook Lee"
            ],
            "abstract": "Medical ultrasound technology is widely used in routine clinical applications\nsuch as disease diagnosis and treatment as well as other applications like\nreal-time monitoring of human tongue shapes and motions as visual feedback in\nsecond language training. Due to the low-contrast characteristic and noisy\nnature of ultrasound images, it might require expertise for non-expert users to\nrecognize tongue gestures. Manual tongue segmentation is a cumbersome,\nsubjective, and error-prone task. Furthermore, it is not a feasible solution\nfor real-time applications. In the last few years, deep learning methods have\nbeen used for delineating and tracking tongue dorsum. Deep convolutional neural\nnetworks (DCNNs), which have shown to be successful in medical image analysis\ntasks, are typically weak for the same task on different domains. In many\ncases, DCNNs trained on data acquired with one ultrasound device, do not\nperform well on data of varying ultrasound device or acquisition protocol.\nDomain adaptation is an alternative solution for this difficulty by\ntransferring the weights from the model trained on a large annotated legacy\ndataset to a new model for adapting on another different dataset using\nfine-tuning. In this study, after conducting extensive experiments, we\naddressed the problem of domain adaptation on small ultrasound datasets for\ntongue contour extraction. We trained a U-net network comprises of an\nencoder-decoder path from scratch, and then with several surrogate scenarios,\nsome parts of the trained network were fine-tuned on another dataset as the\ndomain-adapted networks. We repeat scenarios from target to source domains to\nfind a balance point for knowledge transfer from source to target and vice\nversa. The performance of new fine-tuned networks was evaluated on the same\ntask with images from different domains.",
            "journal ref": "The Journal of the Acoustical Society of America 146, 2940 (2019)",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1907.00758v2",
            "date": "2019-07-01 13:22:48+00:00",
            "title": "Synchronising audio and ultrasound by learning cross-modal embeddings",
            "authors": [
                "Aciel Eshky",
                "Manuel Sam Ribeiro",
                "Korin Richmond",
                "Steve Renals"
            ],
            "abstract": "Audiovisual synchronisation is the task of determining the time offset\nbetween speech audio and a video recording of the articulators. In child speech\ntherapy, audio and ultrasound videos of the tongue are captured using\ninstruments which rely on hardware to synchronise the two modalities at\nrecording time. Hardware synchronisation can fail in practice, and no mechanism\nexists to synchronise the signals post hoc. To address this problem, we employ\na two-stream neural network which exploits the correlation between the two\nmodalities to find the offset. We train our model on recordings from 69\nspeakers, and show that it correctly synchronises 82.9% of test utterances from\nunseen therapy sessions and unseen speakers, thus considerably reducing the\nnumber of utterances to be manually synchronised. An analysis of model\nperformance on the test utterances shows that directed phone articulations are\nmore difficult to automatically synchronise compared to utterances containing\nnatural variation in speech such as words, sentences, or conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/1907.01413v1",
            "date": "2019-07-01 12:04:13+00:00",
            "title": "Speaker-independent classification of phonetic segments from raw ultrasound in child speech",
            "authors": [
                "Manuel Sam Ribeiro",
                "Aciel Eshky",
                "Korin Richmond",
                "Steve Renals"
            ],
            "abstract": "Ultrasound tongue imaging (UTI) provides a convenient way to visualize the\nvocal tract during speech production. UTI is increasingly being used for speech\ntherapy, making it important to develop automatic methods to assist various\ntime-consuming manual tasks currently performed by speech therapists. A key\nchallenge is to generalize the automatic processing of ultrasound tongue images\nto previously unseen speakers. In this work, we investigate the classification\nof phonetic segments (tongue shapes) from raw ultrasound recordings under\nseveral training scenarios: speaker-dependent, multi-speaker,\nspeaker-independent, and speaker-adapted. We observe that models underperform\nwhen applied to data from speakers not seen at training time. However, when\nprovided with minimal additional speaker information, such as the mean\nultrasound frame, the models generalize better to unseen speakers.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/1907.06205v1",
            "date": "2019-07-14 11:14:14+00:00",
            "title": "Automatic Repair and Type Binding of Undeclared Variables using Neural Networks",
            "authors": [
                "Venkatesh Theru Mohan",
                "Ali Jannesari"
            ],
            "abstract": "Deep learning had been used in program analysis for the prediction of hidden\nsoftware defects using software defect datasets, security vulnerabilities using\ngenerative adversarial networks as well as identifying syntax errors by\nlearning a trained neural machine translation on program codes. However, all\nthese approaches either require defect datasets or bug-free source codes that\nare executable for training the deep learning model. Our neural network model\nis neither trained with any defect datasets nor bug-free programming source\ncodes, instead it is trained using structural semantic details of Abstract\nSyntax Tree (AST) where each node represents a construct appearing in the\nsource code. This model is implemented to fix one of the most common semantic\nerrors, such as undeclared variable errors as well as infer their type\ninformation before program compilation. By this approach, the model has\nachieved in correctly locating and identifying 81% of the programs on prutor\ndataset of 1059 programs with only undeclared variable errors and also\ninferring their types correctly in 80% of the programs.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/1907.07564v1",
            "date": "2019-07-16 09:25:13+00:00",
            "title": "Conversational Help for Task Completion and Feature Discovery in Personal Assistants",
            "authors": [
                "Madan Gopal Jhawar",
                "Vipindeep Vangala",
                "Nishchay Sharma",
                "Ankur Hayatnagarkar",
                "Mansi Saxena",
                "Swati Valecha"
            ],
            "abstract": "Intelligent Personal Assistants (IPAs) have become widely popular in recent\ntimes. Most of the commercial IPAs today support a wide range of skills\nincluding Alarms, Reminders, Weather Updates, Music, News, Factual\nQuestioning-Answering, etc. The list grows every day, making it difficult to\nremember the command structures needed to execute various tasks. An IPA must\nhave the ability to communicate information about supported skills and direct\nusers towards the right commands needed to execute them. Users interact with\npersonal assistants in natural language. A query is defined to be a Help Query\nif it seeks information about a personal assistant's capabilities, or asks for\ninstructions to execute a task. In this paper, we propose an interactive system\nwhich identifies help queries and retrieves appropriate responses. Our system\ncomprises of a C-BiLSTM based classifier, which is a fusion of Convolutional\nNeural Networks (CNN) and Bidirectional LSTM (BiLSTM) architectures, to detect\nhelp queries and a semantic Approximate Nearest Neighbours (ANN) module to map\nthe query to an appropriate predefined response. Evaluation of our system on\nreal-world queries from a commercial IPA and a detailed comparison with popular\ntraditional machine learning and deep learning based models reveal that our\nsystem outperforms other approaches and returns relevant responses for help\nqueries.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/1908.08909v2",
            "date": "2019-08-23 17:32:39+00:00",
            "title": "Predicting Features of Quantum Systems from Very Few Measurements",
            "authors": [
                "Hsin-Yuan Huang",
                "Richard Kueng"
            ],
            "abstract": "Predicting features of complex, large-scale quantum systems is essential to\nthe characterization and engineering of quantum architectures. We present an\nefficient approach for constructing an approximate classical description,\ncalled the classical shadow, of a quantum system from very few quantum\nmeasurements that can later be used to predict a large collection of features.\nThis approach is guaranteed to accurately predict M linear functions with\nbounded Hilbert-Schmidt norm from only order of log(M) measurements. This is\ncompletely independent of the system size and saturates fundamental lower\nbounds from information theory. We support our theoretical findings with\nnumerical experiments over a wide range of problem sizes (2 to 162 qubits).\nThese highlight advantages compared to existing machine learning approaches.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/1909.03881v1",
            "date": "2019-09-09 14:20:05+00:00",
            "title": "Nearly-Unsupervised Hashcode Representations for Relation Extraction",
            "authors": [
                "Sahil Garg",
                "Aram Galstyan",
                "Greg Ver Steeg",
                "Guillermo Cecchi"
            ],
            "abstract": "Recently, kernelized locality sensitive hashcodes have been successfully\nemployed as representations of natural language text, especially showing high\nrelevance to biomedical relation extraction tasks. In this paper, we propose to\noptimize the hashcode representations in a nearly unsupervised manner, in which\nwe only use data points, but not their class labels, for learning. The\noptimized hashcode representations are then fed to a supervised classifier\nfollowing the prior work. This nearly unsupervised approach allows fine-grained\noptimization of each hash function, which is particularly suitable for building\nhashcode representations generalizing from a training set to a test set. We\nempirically evaluate the proposed approach for biomedical relation extraction\ntasks, obtaining significant accuracy improvements w.r.t. state-of-the-art\nsupervised and semi-supervised approaches.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1909.07930v1",
            "date": "2019-09-17 16:54:29+00:00",
            "title": "Ludwig: a type-based declarative deep learning toolbox",
            "authors": [
                "Piero Molino",
                "Yaroslav Dudin",
                "Sai Sumanth Miryala"
            ],
            "abstract": "In this work we present Ludwig, a flexible, extensible and easy to use\ntoolbox which allows users to train deep learning models and use them for\nobtaining predictions without writing code. Ludwig implements a novel approach\nto deep learning model building based on two main abstractions: data types and\ndeclarative configuration files. The data type abstraction allows for easier\ncode and sub-model reuse, and the standardized interfaces imposed by this\nabstraction allow for encapsulation and make the code easy to extend.\nDeclarative model definition configuration files enable inexperienced users to\nobtain effective models and increase the productivity of expert users.\nAlongside these two innovations, Ludwig introduces a general modularized deep\nlearning architecture called Encoder-Combiner-Decoder that can be instantiated\nto perform a vast amount of machine learning tasks. These innovations make it\npossible for engineers, scientists from other fields and, in general, a much\nbroader audience to adopt deep learning models for their tasks, concretely\nhelping in its democratization.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1910.00741v1",
            "date": "2019-10-02 01:35:14+00:00",
            "title": "Emergence of Writing Systems Through Multi-Agent Cooperation",
            "authors": [
                "Shresth Verma",
                "Joydip Dhar"
            ],
            "abstract": "Learning to communicate is considered an essential task to develop a general\nAI. While recent literature in language evolution has studied emergent language\nthrough discrete or continuous message symbols, there has been little work in\nthe emergence of writing systems in artificial agents. In this paper, we\npresent a referential game setup with two agents, where the mode of\ncommunication is a written language system that emerges during the play. We\nshow that the agents can learn to coordinate successfully using this mode of\ncommunication. Further, we study how the game rules affect the writing system\ntaxonomy by proposing a consistency metric.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/1910.04023v4",
            "date": "2019-10-09 14:33:37+00:00",
            "title": "On the Possibility of Rewarding Structure Learning Agents: Mutual Information on Linguistic Random Sets",
            "authors": [
                "Ignacio Arroyo-Fern\u00e1ndez",
                "Mauricio Carrasco-Ru\u00edz",
                "J. Anibal Arias-Aguilar"
            ],
            "abstract": "We present a first attempt to elucidate a theoretical and empirical approach\nto design the reward provided by a natural language environment to some\nstructure learning agent. To this end, we revisit the Information Theory of\nunsupervised induction of phrase-structure grammars to characterize the\nbehavior of simulated actions modeled as set-valued random variables (random\nsets of linguistic samples) constituting semantic structures. Our results\nshowed empirical evidence of that simulated semantic structures (Open\nInformation Extraction triplets) can be distinguished from randomly constructed\nones by observing the Mutual Information among their constituents. This\nsuggests the possibility of rewarding structure learning agents without using\npretrained structural analyzers (oracle actors/experts).",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/1911.05186v1",
            "date": "2019-10-23 05:02:15+00:00",
            "title": "TCT: A Cross-supervised Learning Method for Multimodal Sequence Representation",
            "authors": [
                "Wubo Li",
                "Wei Zou",
                "Xiangang Li"
            ],
            "abstract": "Multimodalities provide promising performance than unimodality in most tasks.\nHowever, learning the semantic of the representations from multimodalities\nefficiently is extremely challenging. To tackle this, we propose the\nTransformer based Cross-modal Translator (TCT) to learn unimodal sequence\nrepresentations by translating from other related multimodal sequences on a\nsupervised learning method. Combined TCT with Multimodal Transformer Network\n(MTN), we evaluate MTN-TCT on the video-grounded dialogue which uses\nmultimodality. The proposed method reports new state-of-the-art performance on\nvideo-grounded dialogue which indicates representations learned by TCT are more\nsemantics compared to directly use unimodality.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2001.11552v1",
            "date": "2019-12-11 07:37:45+00:00",
            "title": "Unwanted Advances in Higher Education: Uncovering Sexual Harassment Experiences in Academia with Text Mining",
            "authors": [
                "Amir Karami",
                "Cynthia Nicole White",
                "Kayla Ford",
                "Suzanne Swan",
                "Melek Yildiz Spinel"
            ],
            "abstract": "Sexual harassment in academia is often a hidden problem because victims are\nusually reluctant to report their experiences. Recently, a web survey was\ndeveloped to provide an opportunity to share thousands of sexual harassment\nexperiences in academia. Using an efficient approach, this study collected and\ninvestigated more than 2,000 sexual harassment experiences to better understand\nthese unwanted advances in higher education. This paper utilized text mining to\ndisclose hidden topics and explore their weight across three variables:\nharasser gender, institution type, and victim's field of study. We mapped the\ntopics on five themes drawn from the sexual harassment literature and found\nthat more than 50% of the topics were assigned to the unwanted sexual attention\ntheme. Fourteen percent of the topics were in the gender harassment theme, in\nwhich insulting, sexist, or degrading comments or behavior was directed towards\nwomen. Five percent of the topics involved sexual coercion (a benefit is\noffered in exchange for sexual favors), 5% involved sex discrimination, and 7%\nof the topics discussed retaliation against the victim for reporting the\nharassment, or for simply not complying with the harasser. Findings highlight\nthe power differential between faculty and students, and the toll on students\nwhen professors abuse their power. While some topics did differ based on type\nof institution, there were no differences between the topics based on gender of\nharasser or field of study. This research can be beneficial to researchers in\nfurther investigation of this paper's dataset, and to policymakers in improving\nexisting policies to create a safe and supportive environment in academia.",
            "journal ref": "None",
            "category": "physics.soc-ph"
        }
    ],
    "2020": [
        {
            "url": "http://arxiv.org/abs/2001.00295v1",
            "date": "2020-01-02 02:24:53+00:00",
            "title": "Chemical-induced Disease Relation Extraction with Dependency Information and Prior Knowledge",
            "authors": [
                "Huiwei Zhou",
                "Shixian Ning",
                "Yunlong Yang",
                "Zhuang Liu",
                "Chengkun Lang",
                "Yingyu Lin"
            ],
            "abstract": "Chemical-disease relation (CDR) extraction is significantly important to\nvarious areas of biomedical research and health care. Nowadays, many\nlarge-scale biomedical knowledge bases (KBs) containing triples about entity\npairs and their relations have been built. KBs are important resources for\nbiomedical relation extraction. However, previous research pays little\nattention to prior knowledge. In addition, the dependency tree contains\nimportant syntactic and semantic information, which helps to improve relation\nextraction. So how to effectively use it is also worth studying. In this paper,\nwe propose a novel convolutional attention network (CAN) for CDR extraction.\nFirstly, we extract the shortest dependency path (SDP) between chemical and\ndisease pairs in a sentence, which includes a sequence of words, dependency\ndirections, and dependency relation tags. Then the convolution operations are\nperformed on the SDP to produce deep semantic dependency features. After that,\nan attention mechanism is employed to learn the importance/weight of each\nsemantic dependency vector related to knowledge representations learned from\nKBs. Finally, in order to combine dependency information and prior knowledge,\nthe concatenation of weighted semantic dependency representations and knowledge\nrepresentations is fed to the softmax layer for classification. Experiments on\nthe BioCreative V CDR dataset show that our method achieves comparable\nperformance with the state-of-the-art systems, and both dependency information\nand prior knowledge play important roles in CDR extraction task.",
            "journal ref": "Journal of Biomedical Informatics, 2018, 84:171-178",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00725v3",
            "date": "2020-01-03 05:15:41+00:00",
            "title": "TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising",
            "authors": [
                "Ziyi Yang",
                "Chenguang Zhu",
                "Robert Gmyr",
                "Michael Zeng",
                "Xuedong Huang",
                "Eric Darve"
            ],
            "abstract": "Text summarization aims to extract essential information from a piece of text\nand transform the text into a concise version. Existing unsupervised\nabstractive summarization models leverage recurrent neural networks framework\nwhile the recently proposed transformer exhibits much more capability.\nMoreover, most of previous summarization models ignore abundant unlabeled\ncorpora resources available for pretraining. In order to address these issues,\nwe propose TED, a transformer-based unsupervised abstractive summarization\nsystem with pretraining on large-scale data. We first leverage the lead bias in\nnews articles to pretrain the model on millions of unlabeled corpora. Next, we\nfinetune TED on target domains through theme modeling and a denoising\nautoencoder to enhance the quality of generated summaries. Notably, TED\noutperforms all unsupervised abstractive baselines on NYT, CNN/DM and English\nGigaword datasets with various document styles. Further analysis shows that the\nsummaries generated by TED are highly abstractive, and each component in the\nobjective function of TED is highly effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00891v1",
            "date": "2020-01-03 17:06:41+00:00",
            "title": "Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation",
            "authors": [
                "Goran Glava\u0161",
                "Swapna Somasundaran"
            ],
            "abstract": "Breaking down the structure of long texts into semantically coherent segments\nmakes the texts more readable and supports downstream applications like\nsummarization and retrieval. Starting from an apparent link between text\ncoherence and segmentation, we introduce a novel supervised model for text\nsegmentation with simple but explicit coherence modeling. Our model -- a neural\narchitecture consisting of two hierarchically connected Transformer networks --\nis a multi-task learning model that couples the sentence-level segmentation\nobjective with the coherence objective that differentiates correct sequences of\nsentences from corrupt ones. The proposed model, dubbed Coherence-Aware Text\nSegmentation (CATS), yields state-of-the-art segmentation performance on a\ncollection of benchmark datasets. Furthermore, by coupling CATS with\ncross-lingual word embeddings, we demonstrate its effectiveness in zero-shot\nlanguage transfer: it can successfully segment texts in languages unseen in\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01047v1",
            "date": "2020-01-04 06:31:15+00:00",
            "title": "Adapting Deep Learning for Sentiment Classification of Code-Switched Informal Short Text",
            "authors": [
                "Muhammad Haroon Shakeel",
                "Asim Karim"
            ],
            "abstract": "Nowadays, an abundance of short text is being generated that uses nonstandard\nwriting styles influenced by regional languages. Such informal and\ncode-switched content are under-resourced in terms of labeled datasets and\nlanguage models even for popular tasks like sentiment classification. In this\nwork, we (1) present a labeled dataset called MultiSenti for sentiment\nclassification of code-switched informal short text, (2) explore the\nfeasibility of adapting resources from a resource-rich language for an informal\none, and (3) propose a deep learning-based model for sentiment classification\nof code-switched informal short text. We aim to achieve this without any\nlexical normalization, language translation, or code-switching indication. The\nperformance of the proposed models is compared with three existing multilingual\nsentiment classification models. The results show that the proposed model\nperforms better in general and adapting character-based embeddings yield\nequivalent performance while being computationally more efficient than training\nword-based domain-specific embeddings.",
            "journal ref": "The 35th ACM/SIGAPP Symposium on Applied Computing (SAC) 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01167v2",
            "date": "2020-01-05 04:50:38+00:00",
            "title": "Computationally Efficient NER Taggers with Combined Embeddings and Constrained Decoding",
            "authors": [
                "Brian Lester",
                "Daniel Pressel",
                "Amy Hemmeter",
                "Sagnik Ray Choudhury"
            ],
            "abstract": "Current State-of-the-Art models in Named Entity Recognition (NER) are neural\nmodels with a Conditional Random Field (CRF) as the final network layer, and\npre-trained \"contextual embeddings\". The CRF layer is used to facilitate global\ncoherence between labels, and the contextual embeddings provide a better\nrepresentation of words in context. However, both of these improvements come at\na high computational cost. In this work, we explore two simple techniques that\nsubstantially improve NER performance over a strong baseline with negligible\ncost. First, we use multiple pre-trained embeddings as word representations via\nconcatenation. Second, we constrain the tagger, trained using a cross-entropy\nloss, during decoding to eliminate illegal transitions. While training a tagger\non CoNLL 2003 we find a $786$\\% speed-up over a contextual embeddings-based\ntagger without sacrificing strong performance. We also show that the\nconcatenation technique works across multiple tasks and datasets. We analyze\naspects of similarity and coverage between pre-trained embeddings and the\ndynamics of tag co-occurrence to explain why these techniques work. We provide\nan open source implementation of our tagger using these techniques in three\npopular deep learning frameworks --- TensorFlow, Pytorch, and DyNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01269v2",
            "date": "2020-01-05 16:34:32+00:00",
            "title": "Generating Word and Document Embeddings for Sentiment Analysis",
            "authors": [
                "Cem R\u0131fk\u0131 Ayd\u0131n",
                "Tunga G\u00fcng\u00f6r",
                "Ali Erkan"
            ],
            "abstract": "Sentiments of words differ from one corpus to another. Inducing general\nsentiment lexicons for languages and using them cannot, in general, produce\nmeaningful results for different domains. In this paper, we combine contextual\nand supervised information with the general semantic representations of words\noccurring in the dictionary. Contexts of words help us capture the\ndomain-specific information and supervised scores of words are indicative of\nthe polarities of those words. When we combine supervised features of words\nwith the features extracted from their dictionary definitions, we observe an\nincrease in the success rates. We try out the combinations of contextual,\nsupervised, and dictionary-based approaches, and generate original vectors. We\nalso combine the word2vec approach with hand-crafted features. We induce\ndomain-specific sentimental vectors for two corpora, which are the movie domain\nand the Twitter datasets in Turkish. When we thereafter generate document\nvectors and employ the support vector machines method utilising those vectors,\nour approaches perform better than the baseline studies for Turkish with a\nsignificant margin. We evaluated our models on two English corpora as well and\nthese also outperformed the word2vec approach. It shows that our approaches are\ncross-domain and portable to other languages.",
            "journal ref": "Springer LNCS Proceedings for CICLing 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01447v1",
            "date": "2020-01-06 09:18:29+00:00",
            "title": "Improving Entity Linking by Modeling Latent Entity Type Information",
            "authors": [
                "Shuang Chen",
                "Jinpeng Wang",
                "Feng Jiang",
                "Chin-Yew Lin"
            ],
            "abstract": "Existing state of the art neural entity linking models employ attention-based\nbag-of-words context model and pre-trained entity embeddings bootstrapped from\nword embeddings to assess topic level context compatibility. However, the\nlatent entity type information in the immediate context of the mention is\nneglected, which causes the models often link mentions to incorrect entities\nwith incorrect type. To tackle this problem, we propose to inject latent entity\ntype information into the entity embeddings based on pre-trained BERT. In\naddition, we integrate a BERT-based entity similarity score into the local\ncontext model of a state-of-the-art model to better capture latent entity type\ninformation. Our model significantly outperforms the state-of-the-art entity\nlinking models on standard benchmark (AIDA-CoNLL). Detailed experiment analysis\ndemonstrates that our model corrects most of the type errors produced by the\ndirect baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01565v1",
            "date": "2020-01-06 13:37:51+00:00",
            "title": "Stance Detection Benchmark: How Robust Is Your Stance Detection?",
            "authors": [
                "Benjamin Schiller",
                "Johannes Daxenberger",
                "Iryna Gurevych"
            ],
            "abstract": "Stance Detection (StD) aims to detect an author's stance towards a certain\ntopic or claim and has become a key component in applications like fake news\ndetection, claim validation, and argument search. However, while stance is\neasily detected by humans, machine learning models are clearly falling short of\nthis task. Given the major differences in dataset sizes and framing of StD\n(e.g. number of classes and inputs), we introduce a StD benchmark that learns\nfrom ten StD datasets of various domains in a multi-dataset learning (MDL)\nsetting, as well as from related tasks via transfer learning. Within this\nbenchmark setup, we are able to present new state-of-the-art results on five of\nthe datasets. Yet, the models still perform well below human capabilities and\neven simple adversarial attacks severely hurt the performance of MDL models.\nDeeper investigation into this phenomenon suggests the existence of biases\ninherited from multiple datasets by design. Our analysis emphasizes the need of\nfocus on robustness and de-biasing strategies in multi-task learning\napproaches. The benchmark dataset and code is made available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01582v2",
            "date": "2020-01-06 13:54:06+00:00",
            "title": "A Survey on Machine Reading Comprehension Systems",
            "authors": [
                "Razieh Baradaran",
                "Razieh Ghiasi",
                "Hossein Amirkhani"
            ],
            "abstract": "Machine reading comprehension is a challenging task and hot topic in natural\nlanguage processing. Its goal is to develop systems to answer the questions\nregarding a given context. In this paper, we present a comprehensive survey on\ndifferent aspects of machine reading comprehension systems, including their\napproaches, structures, input/outputs, and research novelties. We illustrate\nthe recent trends in this field based on 241 reviewed papers from 2016 to 2020.\nOur investigations demonstrate that the focus of research has changed in recent\nyears from answer extraction to answer generation, from single to\nmulti-document reading comprehension, and from learning from scratch to using\npre-trained embeddings. We also discuss the popular datasets and the evaluation\nmetrics in this field. The paper ends with investigating the most cited papers\nand their contributions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01589v1",
            "date": "2020-01-02 10:05:02+00:00",
            "title": "Morphological Word Segmentation on Agglutinative Languages for Neural Machine Translation",
            "authors": [
                "Yirong Pan",
                "Xiao Li",
                "Yating Yang",
                "Rui Dong"
            ],
            "abstract": "Neural machine translation (NMT) has achieved impressive performance on\nmachine translation task in recent years. However, in consideration of\nefficiency, a limited-size vocabulary that only contains the top-N highest\nfrequency words are employed for model training, which leads to many rare and\nunknown words. It is rather difficult when translating from the low-resource\nand morphologically-rich agglutinative languages, which have complex morphology\nand large vocabulary. In this paper, we propose a morphological word\nsegmentation method on the source-side for NMT that incorporates morphology\nknowledge to preserve the linguistic and semantic information in the word\nstructure while reducing the vocabulary size at training time. It can be\nutilized as a preprocessing tool to segment the words in agglutinative\nlanguages for other natural language processing (NLP) tasks. Experimental\nresults show that our morphologically motivated word segmentation method is\nbetter suitable for the NMT model, which achieves significant improvements on\nTurkish-English and Uyghur-Chinese machine translation tasks on account of\nreducing data sparseness and language complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01863v7",
            "date": "2020-01-07 02:42:57+00:00",
            "title": "Text Complexity Classification Based on Linguistic Information: Application to Intelligent Tutoring of ESL",
            "authors": [
                "M. Zakaria Kurdi"
            ],
            "abstract": "The goal of this work is to build a classifier that can identify text\ncomplexity within the context of teaching reading to English as a Second\nLanguage (ESL) learners. To present language learners with texts that are\nsuitable to their level of English, a set of features that can describe the\nphonological, morphological, lexical, syntactic, discursive, and psychological\ncomplexity of a given text were identified. Using a corpus of 6171 texts, which\nhad already been classified into three different levels of difficulty by ESL\nexperts, different experiments were conducted with five machine learning\nalgorithms. The results showed that the adopted linguistic features provide a\ngood overall classification performance (F-Score = 0.97). A scalability\nevaluation was conducted to test if such a classifier could be used within real\napplications, where it can be, for example, plugged into a search engine or a\nweb-scraping module. In this evaluation, the texts in the test set are not only\ndifferent from those from the training set but also of different types (ESL\ntexts vs. children reading texts). Although the overall performance of the\nclassifier decreased significantly (F-Score = 0.65), the confusion matrix shows\nthat most of the classification errors are between the classes two and three\n(the middle-level classes) and that the system has a robust performance in\ncategorizing texts of class one and four. This behavior can be explained by the\ndifference in classification criteria between the two corpora. Hence, the\nobserved results confirm the usability of such a classifier within a real-world\napplication.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2020 (September 19,\n  2020) jdmdh:6674",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01989v1",
            "date": "2020-01-07 11:50:54+00:00",
            "title": "Latent Opinions Transfer Network for Target-Oriented Opinion Words Extraction",
            "authors": [
                "Zhen Wu",
                "Fei Zhao",
                "Xin-Yu Dai",
                "Shujian Huang",
                "Jiajun Chen"
            ],
            "abstract": "Target-oriented opinion words extraction (TOWE) is a new subtask of ABSA,\nwhich aims to extract the corresponding opinion words for a given opinion\ntarget in a sentence. Recently, neural network methods have been applied to\nthis task and achieve promising results. However, the difficulty of annotation\ncauses the datasets of TOWE to be insufficient, which heavily limits the\nperformance of neural models. By contrast, abundant review sentiment\nclassification data are easily available at online review sites. These reviews\ncontain substantial latent opinions information and semantic patterns. In this\npaper, we propose a novel model to transfer these opinions knowledge from\nresource-rich review sentiment classification datasets to low-resource task\nTOWE. To address the challenges in the transfer process, we design an effective\ntransformation method to obtain latent opinions, then integrate them into TOWE.\nExtensive experimental results show that our model achieves better performance\ncompared to other state-of-the-art methods and significantly outperforms the\nbase model without transferring opinions knowledge. Further analysis validates\nthe effectiveness of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02091v1",
            "date": "2020-01-07 15:02:28+00:00",
            "title": "Knowledge-aware Attention Network for Protein-Protein Interaction Extraction",
            "authors": [
                "Huiwei Zhou",
                "Zhuang Liu1",
                "Shixian Ning",
                "Chengkun Lang",
                "Yingyu Lin",
                "Lei Du"
            ],
            "abstract": "Protein-protein interaction (PPI) extraction from published scientific\nliterature provides additional support for precision medicine efforts. However,\nmany of the current PPI extraction methods need extensive feature engineering\nand cannot make full use of the prior knowledge in knowledge bases (KB). KBs\ncontain huge amounts of structured information about entities and\nrelationships, therefore plays a pivotal role in PPI extraction. This paper\nproposes a knowledge-aware attention network (KAN) to fuse prior knowledge\nabout protein-protein pairs and context information for PPI extraction. The\nproposed model first adopts a diagonal-disabled multi-head attention mechanism\nto encode context sequence along with knowledge representations learned from\nKB. Then a novel multi-dimensional attention mechanism is used to select the\nfeatures that can best describe the encoded context. Experiment results on the\nBioCreative VI PPI dataset show that the proposed approach could acquire\nknowledge-aware dependencies between different words in a sequence and lead to\na new state-of-the-art performance.",
            "journal ref": "Journal of Biomedical Informatics, 2019, 96: 103234",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02107v1",
            "date": "2020-01-07 15:11:27+00:00",
            "title": "Leveraging Prior Knowledge for Protein-Protein Interaction Extraction with Memory Network",
            "authors": [
                "Huiwei Zhou",
                "Zhuang Liu",
                "Shixian Ning",
                "Yunlong Yang",
                "Chengkun Lang",
                "Yingyu Lin",
                "Kun Ma"
            ],
            "abstract": "Automatically extracting Protein-Protein Interactions (PPI) from biomedical\nliterature provides additional support for precision medicine efforts. This\npaper proposes a novel memory network-based model (MNM) for PPI extraction,\nwhich leverages prior knowledge about protein-protein pairs with memory\nnetworks. The proposed MNM captures important context clues related to\nknowledge representations learned from knowledge bases. Both entity embeddings\nand relation embeddings of prior knowledge are effective in improving the PPI\nextraction model, leading to a new state-of-the-art performance on the\nBioCreative VI PPI dataset. The paper also shows that multiple computational\nlayers over an external memory are superior to long short-term memory networks\nwith the local memories.",
            "journal ref": "Database-The Journal of Biological Databases and Curation, 2018,\n  2018: bay071",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02332v1",
            "date": "2020-01-08 01:19:08+00:00",
            "title": "Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs",
            "authors": [
                "Pengda Qin",
                "Xin Wang",
                "Wenhu Chen",
                "Chunyun Zhang",
                "Weiran Xu",
                "William Yang Wang"
            ],
            "abstract": "Large-scale knowledge graphs (KGs) are shown to become more important in\ncurrent information systems. To expand the coverage of KGs, previous studies on\nknowledge graph completion need to collect adequate training instances for\nnewly-added relations. In this paper, we consider a novel formulation,\nzero-shot learning, to free this cumbersome curation. For newly-added\nrelations, we attempt to learn their semantic features from their text\ndescriptions and hence recognize the facts of unseen relations with no examples\nbeing seen. For this purpose, we leverage Generative Adversarial Networks\n(GANs) to establish the connection between text and knowledge graph domain: The\ngenerator learns to generate the reasonable relation embeddings merely with\nnoisy text descriptions. Under this setting, zero-shot learning is naturally\nconverted to a traditional supervised classification task. Empirically, our\nmethod is model-agnostic that could be potentially applied to any version of KG\nembeddings, and consistently yields performance improvements on NELL and Wiki\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02380v2",
            "date": "2020-01-08 05:14:49+00:00",
            "title": "A Neural Approach to Discourse Relation Signal Detection",
            "authors": [
                "Amir Zeldes",
                "Yang Liu"
            ],
            "abstract": "Previous data-driven work investigating the types and distributions of\ndiscourse relation signals, including discourse markers such as 'however' or\nphrases such as 'as a result' has focused on the relative frequencies of signal\nwords within and outside text from each discourse relation. Such approaches do\nnot allow us to quantify the signaling strength of individual instances of a\nsignal on a scale (e.g. more or less discourse-relevant instances of 'and'), to\nassess the distribution of ambiguity for signals, or to identify words that\nhinder discourse relation identification in context ('anti-signals' or\n'distractors'). In this paper we present a data-driven approach to signal\ndetection using a distantly supervised neural network and develop a metric,\nDelta s (or 'delta-softmax'), to quantify signaling strength. Ranging between\n-1 and 1 and relying on recent advances in contextualized words embeddings, the\nmetric represents each word's positive or negative contribution to the\nidentifiability of a relation in specific instances in context. Based on an\nEnglish corpus annotated for discourse relations using Rhetorical Structure\nTheory and signal type annotations anchored to specific tokens, our analysis\nexamines the reliability of the metric, the places where it overlaps with and\ndiffers from human judgments, and the implications for identifying features\nthat neural models may need in order to perform better on automatic discourse\nrelation classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02524v2",
            "date": "2020-01-08 13:44:10+00:00",
            "title": "LTP: A New Active Learning Strategy for CRF-Based Named Entity Recognition",
            "authors": [
                "Mingyi Liu",
                "Zhiying Tu",
                "Tong Zhang",
                "Tonghua Su",
                "Zhongjie Wang"
            ],
            "abstract": "In recent years, deep learning has achieved great success in many natural\nlanguage processing tasks including named entity recognition. The shortcoming\nis that a large amount of manually-annotated data is usually required. Previous\nstudies have demonstrated that active learning could elaborately reduce the\ncost of data annotation, but there is still plenty of room for improvement. In\nreal applications we found existing uncertainty-based active learning\nstrategies have two shortcomings. Firstly, these strategies prefer to choose\nlong sequence explicitly or implicitly, which increase the annotation burden of\nannotators. Secondly, some strategies need to invade the model and modify to\ngenerate some additional information for sample selection, which will increase\nthe workload of the developer and increase the training/prediction time of the\nmodel. In this paper, we first examine traditional active learning strategies\nin a specific case of BiLstm-CRF that has widely used in named entity\nrecognition on several typical datasets. Then we propose an uncertainty-based\nactive learning strategy called Lowest Token Probability (LTP) which combines\nthe input and output of CRF to select informative instance. LTP is simple and\npowerful strategy that does not favor long sequences and does not need to\ninvade the model. We test LTP on multiple datasets, and the experiments show\nthat LTP performs slightly better than traditional strategies with obviously\nless annotation tokens on both sentence-level accuracy and entity-level\nF1-score. Related code have been release on https://github.com/HIT-ICES/AL-NER",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02836v1",
            "date": "2020-01-09 04:47:14+00:00",
            "title": "Multiplex Word Embeddings for Selectional Preference Acquisition",
            "authors": [
                "Hongming Zhang",
                "Jiaxin Bai",
                "Yan Song",
                "Kun Xu",
                "Changlong Yu",
                "Yangqiu Song",
                "Wilfred Ng",
                "Dong Yu"
            ],
            "abstract": "Conventional word embeddings represent words with fixed vectors, which are\nusually trained based on co-occurrence patterns among words. In doing so,\nhowever, the power of such representations is limited, where the same word\nmight be functionalized separately under different syntactic relations. To\naddress this limitation, one solution is to incorporate relational dependencies\nof different words into their embeddings. Therefore, in this paper, we propose\na multiplex word embedding model, which can be easily extended according to\nvarious relations among words. As a result, each word has a center embedding to\nrepresent its overall semantics, and several relational embeddings to represent\nits relational dependencies. Compared to existing models, our model can\neffectively distinguish words with respect to different relations without\nintroducing unnecessary sparseness. Moreover, to accommodate various relations,\nwe use a small dimension for relational embeddings and our model is able to\nkeep their effectiveness. Experiments on selectional preference acquisition and\nword similarity demonstrate the effectiveness of the proposed model, and a\nfurther study of scalability also proves that our embeddings only need 1/20 of\nthe original embedding size to achieve better performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02885v1",
            "date": "2020-01-09 08:43:30+00:00",
            "title": "Resolving the Scope of Speculation and Negation using Transformer-Based Architectures",
            "authors": [
                "Benita Kathleen Britto",
                "Aditya Khandelwal"
            ],
            "abstract": "Speculation is a naturally occurring phenomena in textual data, forming an\nintegral component of many systems, especially in the biomedical information\nretrieval domain. Previous work addressing cue detection and scope resolution\n(the two subtasks of speculation detection) have ranged from rule-based systems\nto deep learning-based approaches. In this paper, we apply three popular\ntransformer-based architectures, BERT, XLNet and RoBERTa to this task, on two\npublicly available datasets, BioScope Corpus and SFU Review Corpus, reporting\nsubstantial improvements over previously reported results (by at least 0.29 F1\npoints on cue detection and 4.27 F1 points on scope resolution). We also\nexperiment with joint training of the model on multiple datasets, which\noutperforms the single dataset training approach by a good margin. We observe\nthat XLNet consistently outperforms BERT and RoBERTa, contrary to results on\nother benchmark datasets. To confirm this observation, we apply XLNet and\nRoBERTa to negation detection and scope resolution, reporting state-of-the-art\nresults on negation scope resolution for the BioScope Corpus (increase of 3.16\nF1 points on the BioScope Full Papers, 0.06 F1 points on the BioScope\nAbstracts) and the SFU Review Corpus (increase of 0.3 F1 points).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02943v2",
            "date": "2020-01-09 12:34:01+00:00",
            "title": "Binary and Multitask Classification Model for Dutch Anaphora Resolution: Die/Dat Prediction",
            "authors": [
                "Liesbeth Allein",
                "Artuur Leeuwenberg",
                "Marie-Francine Moens"
            ],
            "abstract": "The correct use of Dutch pronouns 'die' and 'dat' is a stumbling block for\nboth native and non-native speakers of Dutch due to the multiplicity of\nsyntactic functions and the dependency on the antecedent's gender and number.\nDrawing on previous research conducted on neural context-dependent dt-mistake\ncorrection models (Heyman et al. 2018), this study constructs the first neural\nnetwork model for Dutch demonstrative and relative pronoun resolution that\nspecifically focuses on the correction and part-of-speech prediction of these\ntwo pronouns. Two separate datasets are built with sentences obtained from,\nrespectively, the Dutch Europarl corpus (Koehn 2015) - which contains the\nproceedings of the European Parliament from 1996 to the present - and the SoNaR\ncorpus (Oostdijk et al. 2013) - which contains Dutch texts from a variety of\ndomains such as newspapers, blogs and legal texts. Firstly, a binary\nclassification model solely predicts the correct 'die' or 'dat'. The classifier\nwith a bidirectional long short-term memory architecture achieves 84.56%\naccuracy. Secondly, a multitask classification model simultaneously predicts\nthe correct 'die' or 'dat' and its part-of-speech tag. The model containing a\ncombination of a sentence and context encoder with both a bidirectional long\nshort-term memory architecture results in 88.63% accuracy for die/dat\nprediction and 87.73% accuracy for part-of-speech prediction. More\nevenly-balanced data, larger word embeddings, an extra bidirectional long\nshort-term memory layer and integrated part-of-speech knowledge positively\naffects die/dat prediction performance, while a context encoder architecture\nraises part-of-speech prediction performance. This study shows promising\nresults and can serve as a starting point for future research on machine\nlearning models for Dutch anaphora resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03131v1",
            "date": "2020-01-09 17:48:44+00:00",
            "title": "Offensive Language Detection: A Comparative Analysis",
            "authors": [
                "Vyshnav M T",
                "Sachin Kumar S",
                "Soman K P"
            ],
            "abstract": "Offensive behaviour has become pervasive in the Internet community.\nIndividuals take the advantage of anonymity in the cyber world and indulge in\noffensive communications which they may not consider in the real life.\nGovernments, online communities, companies etc are investing into prevention of\noffensive behaviour content in social media. One of the most effective solution\nfor tacking this enigmatic problem is the use of computational techniques to\nidentify offensive content and take action. The current work focuses on\ndetecting offensive language in English tweets. The dataset used for the\nexperiment is obtained from SemEval-2019 Task 6 on Identifying and Categorizing\nOffensive Language in Social Media (OffensEval). The dataset contains 14,460\nannotated English tweets. The present paper provides a comparative analysis and\nRandom kitchen sink (RKS) based approach for offensive language detection. We\nexplore the effectiveness of Google sentence encoder, Fasttext, Dynamic mode\ndecomposition (DMD) based features and Random kitchen sink (RKS) method for\noffensive language detection. From the experiments and evaluation we observed\nthat RKS with fastetxt achieved competing results. The evaluation measures used\nare accuracy, precision, recall, f1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03216v1",
            "date": "2020-01-09 20:37:49+00:00",
            "title": "Simulating Lexical Semantic Change from Sense-Annotated Data",
            "authors": [
                "Dominik Schlechtweg",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We present a novel procedure to simulate lexical semantic change from\nsynchronic sense-annotated data, and demonstrate its usefulness for assessing\nlexical semantic change detection models. The induced dataset represents a\nstronger correspondence to empirically observed lexical semantic change than\nprevious synthetic datasets, because it exploits the intimate relationship\nbetween synchronic polysemy and diachronic change. We publish the data and\nprovide the first large-scale evaluation gold standard for LSC detection\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03278v1",
            "date": "2020-01-10 01:45:45+00:00",
            "title": "A Scalable Chatbot Platform Leveraging Online Community Posts: A Proof-of-Concept Study",
            "authors": [
                "Sihyeon Jo",
                "Seungryong Yoo",
                "Sangwon Im",
                "Seung Hee Yang",
                "Tong Zuo",
                "Hee-Eun Kim",
                "SangWook Han",
                "Seong-Woo Kim"
            ],
            "abstract": "The development of natural language processing algorithms and the explosive\ngrowth of conversational data are encouraging researches on the human-computer\nconversation. Still, getting qualified conversational data on a large scale is\ndifficult and expensive. In this paper, we verify the feasibility of\nconstructing a data-driven chatbot with processed online community posts by\nusing them as pseudo-conversational data. We argue that chatbots for various\npurposes can be built extensively through the pipeline exploiting the common\nstructure of community posts. Our experiment demonstrates that chatbots created\nalong the pipeline can yield the proper responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03361v3",
            "date": "2020-01-10 09:29:20+00:00",
            "title": "Co-evolution of language and agents in referential games",
            "authors": [
                "Gautier Dagan",
                "Dieuwke Hupkes",
                "Elia Bruni"
            ],
            "abstract": "Referential games offer a grounded learning environment for neural agents\nwhich accounts for the fact that language is functionally used to communicate.\nHowever, they do not take into account a second constraint considered to be\nfundamental for the shape of human language: that it must be learnable by new\nlanguage learners.\n  Cogswell et al. (2019) introduced cultural transmission within referential\ngames through a changing population of agents to constrain the emerging\nlanguage to be learnable. However, the resulting languages remain inherently\nbiased by the agents' underlying capabilities.\n  In this work, we introduce Language Transmission Engine to model both\ncultural and architectural evolution in a population of agents. As our core\ncontribution, we empirically show that the optimal situation is to take into\naccount also the learning biases of the language learners and thus let language\nand agents co-evolve. When we allow the agent population to evolve through\narchitectural evolution, we achieve across the board improvements on all\nconsidered metrics and surpass the gains made with cultural transmission. These\nresults stress the importance of studying the underlying agent architecture and\npave the way to investigate the co-evolution of language and agent in language\nemergence studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03632v1",
            "date": "2020-01-10 19:02:52+00:00",
            "title": "Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks",
            "authors": [
                "R. Thomas McCoy",
                "Robert Frank",
                "Tal Linzen"
            ],
            "abstract": "Learners that are exposed to the same training data might generalize\ndifferently due to differing inductive biases. In neural network models,\ninductive biases could in theory arise from any aspect of the model\narchitecture. We investigate which architectural factors affect the\ngeneralization behavior of neural sequence-to-sequence models trained on two\nsyntactic tasks, English question formation and English tense reinflection. For\nboth tasks, the training set is consistent with a generalization based on\nhierarchical structure and a generalization based on linear order. All\narchitectural factors that we investigated qualitatively affected how models\ngeneralized, including factors with no clear connection to hierarchical\nstructure. For example, LSTMs and GRUs displayed qualitatively different\ninductive biases. However, the only factor that consistently contributed a\nhierarchical bias across tasks was the use of a tree-structured model rather\nthan a model with sequential recurrence, suggesting that human-like syntactic\ngeneralization requires architectural syntactic structure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03708v1",
            "date": "2020-01-11 03:54:31+00:00",
            "title": "PatentTransformer-2: Controlling Patent Text Generation by Structural Metadata",
            "authors": [
                "Jieh-Sheng Lee",
                "Jieh Hsiang"
            ],
            "abstract": "PatentTransformer is our codename for patent text generation based on\nTransformer-based models. Our goal is \"Augmented Inventing.\" In this second\nversion, we leverage more of the structural metadata in patents. The structural\nmetadata includes patent title, abstract, and dependent claim, in addition to\nindependent claim previously. Metadata controls what kind of patent text for\nthe model to generate. Also, we leverage the relation between metadata to build\na text-to-text generation flow, for example, from a few words to a title, the\ntitle to an abstract, the abstract to an independent claim, and the independent\nclaim to multiple dependent claims. The text flow can go backward because the\nrelation is trained bidirectionally. We release our GPT-2 models trained from\nscratch and our code for inference so that readers can verify and generate\npatent text on their own. As for generation quality, we measure it by both\nROUGE and Google Universal Sentence Encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03765v1",
            "date": "2020-01-11 15:30:56+00:00",
            "title": "Learning Cross-Context Entity Representations from Text",
            "authors": [
                "Jeffrey Ling",
                "Nicholas FitzGerald",
                "Zifei Shan",
                "Livio Baldini Soares",
                "Thibault F\u00e9vry",
                "David Weiss",
                "Tom Kwiatkowski"
            ],
            "abstract": "Language modeling tasks, in which words, or word-pieces, are predicted on the\nbasis of a local context, have been very effective for learning word embeddings\nand context dependent representations of phrases. Motivated by the observation\nthat efforts to code world knowledge into machine readable knowledge bases or\nhuman readable encyclopedias tend to be entity-centric, we investigate the use\nof a fill-in-the-blank task to learn context independent representations of\nentities from the text contexts in which those entities were mentioned. We show\nthat large scale training of neural models allows us to learn high quality\nentity representations, and we demonstrate successful results on four domains:\n(1) existing entity-level typing benchmarks, including a 64% error reduction\nover previous work on TypeNet (Murty et al., 2018); (2) a novel few-shot\ncategory reconstruction task; (3) existing entity linking benchmarks, where we\nmatch the state-of-the-art on CoNLL-Aida without linking-specific features and\nobtain a score of 89.8% on TAC-KBP 2010 without using any alias table, external\nknowledge base or in domain training data and (4) answering trivia questions,\nwhich uniquely identify entities. Our global entity representations encode\nfine-grained type categories, such as Scottish footballers, and can answer\ntrivia questions such as: Who was the last inmate of Spandau jail in Berlin?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03830v1",
            "date": "2020-01-12 02:31:07+00:00",
            "title": "Revisiting Challenges in Data-to-Text Generation with Fact Grounding",
            "authors": [
                "Hongmin Wang"
            ],
            "abstract": "Data-to-text generation models face challenges in ensuring data fidelity by\nreferring to the correct input source. To inspire studies in this area, Wiseman\net al. (2017) introduced the RotoWire corpus on generating NBA game summaries\nfrom the box- and line-score tables. However, limited attempts have been made\nin this direction and the challenges remain. We observe a prominent bottleneck\nin the corpus where only about 60% of the summary contents can be grounded to\nthe boxscore records. Such information deficiency tends to misguide a\nconditioned language model to produce unconditioned random facts and thus leads\nto factual hallucinations. In this work, we restore the information balance and\nrevamp this task to focus on fact-grounded data-to-text generation. We\nintroduce a purified and larger-scale dataset, RotoWire-FG (Fact-Grounding),\nwith 50% more data from the year 2017-19 and enriched input tables, hoping to\nattract more research focuses in this direction. Moreover, we achieve improved\ndata fidelity over the state-of-the-art models by integrating a new form of\ntable reconstruction as an auxiliary task to boost the generation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03844v1",
            "date": "2020-01-12 04:33:53+00:00",
            "title": "Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study",
            "authors": [
                "Jinlan Fu",
                "Pengfei Liu",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "While neural network-based models have achieved impressive performance on a\nlarge body of NLP tasks, the generalization behavior of different models\nremains poorly understood: Does this excellent performance imply a perfect\ngeneralization model, or are there still some limitations? In this paper, we\ntake the NER task as a testbed to analyze the generalization behavior of\nexisting models from different perspectives and characterize the differences of\ntheir generalization abilities through the lens of our proposed measures, which\nguides us to better design models and training methods. Experiments with\nin-depth analyses diagnose the bottleneck of existing neural NER models in\nterms of breakdown performance analysis, annotation errors, dataset bias, and\ncategory relationships, which suggest directions for improvement. We have\nreleased the datasets: (ReCoNLL, PLONER) for the future research at our project\npage: http://pfliu.com/InterpretNER/. As a by-product of this paper, we have\nopen-sourced a project that involves a comprehensive summary of recent NER\npapers and classifies them into different research topics:\nhttps://github.com/pfliu-nlp/Named-Entity-Recognition-NER-Papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04063v3",
            "date": "2020-01-13 05:12:38+00:00",
            "title": "ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training",
            "authors": [
                "Weizhen Qi",
                "Yu Yan",
                "Yeyun Gong",
                "Dayiheng Liu",
                "Nan Duan",
                "Jiusheng Chen",
                "Ruofei Zhang",
                "Ming Zhou"
            ],
            "abstract": "This paper presents a new sequence-to-sequence pre-training model called\nProphetNet, which introduces a novel self-supervised objective named future\nn-gram prediction and the proposed n-stream self-attention mechanism. Instead\nof optimizing one-step-ahead prediction in the traditional sequence-to-sequence\nmodel, the ProphetNet is optimized by n-step ahead prediction that predicts the\nnext n tokens simultaneously based on previous context tokens at each time\nstep. The future n-gram prediction explicitly encourages the model to plan for\nthe future tokens and prevent overfitting on strong local correlations. We\npre-train ProphetNet using a base scale dataset (16GB) and a large-scale\ndataset (160GB), respectively. Then we conduct experiments on CNN/DailyMail,\nGigaword, and SQuAD 1.1 benchmarks for abstractive summarization and question\ngeneration tasks. Experimental results show that ProphetNet achieves new\nstate-of-the-art results on all these datasets compared to the models using the\nsame scale pre-training corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05136v2",
            "date": "2020-01-15 05:32:18+00:00",
            "title": "Non-Autoregressive Machine Translation with Disentangled Context Transformer",
            "authors": [
                "Jungo Kasai",
                "James Cross",
                "Marjan Ghazvininejad",
                "Jiatao Gu"
            ],
            "abstract": "State-of-the-art neural machine translation models generate a translation\nfrom left to right and every step is conditioned on the previously generated\ntokens. The sequential nature of this generation process causes fundamental\nlatency in inference since we cannot generate multiple tokens in each sentence\nin parallel. We propose an attention-masking based model, called Disentangled\nContext (DisCo) transformer, that simultaneously generates all tokens given\ndifferent contexts. The DisCo transformer is trained to predict every output\ntoken given an arbitrary subset of the other reference tokens. We also develop\nthe parallel easy-first inference algorithm, which iteratively refines every\ntoken in parallel and reduces the number of required iterations. Our extensive\nexperiments on 7 translation directions with varying data sizes demonstrate\nthat our model achieves competitive, if not better, performance compared to the\nstate of the art in non-autoregressive machine translation while significantly\nreducing decoding time on average. Our code is available at\nhttps://github.com/facebookresearch/DisCo.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05139v1",
            "date": "2020-01-15 05:42:27+00:00",
            "title": "A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation",
            "authors": [
                "Jian Guan",
                "Fei Huang",
                "Zhihao Zhao",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "Story generation, namely generating a reasonable story from a leading\ncontext, is an important but challenging task. In spite of the success in\nmodeling fluency and local coherence, existing neural language generation\nmodels (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of\nlong-range coherence in generated stories. We conjecture that this is because\nof the difficulty of associating relevant commonsense knowledge, understanding\nthe causal relationships, and planning entities and events with proper temporal\norder. In this paper, we devise a knowledge-enhanced pretraining model for\ncommonsense story generation. We propose to utilize commonsense knowledge from\nexternal knowledge bases to generate reasonable stories. To further capture the\ncausal and temporal dependencies between the sentences in a reasonable story,\nwe employ multi-task learning which combines a discriminative objective to\ndistinguish true and fake stories during fine-tuning. Automatic and manual\nevaluation shows that our model can generate more reasonable stories than\nstate-of-the-art baselines, particularly in terms of logic and global\ncoherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05285v1",
            "date": "2020-01-12 21:54:52+00:00",
            "title": "Detecting New Word Meanings: A Comparison of Word Embedding Models in Spanish",
            "authors": [
                "Andr\u00e9s Torres-Rivera",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Semantic neologisms (SN) are defined as words that acquire a new word meaning\nwhile maintaining their form. Given the nature of this kind of neologisms, the\ntask of identifying these new word meanings is currently performed manually by\nspecialists at observatories of neology. To detect SN in a semi-automatic way,\nwe developed a system that implements a combination of the following\nstrategies: topic modeling, keyword extraction, and word sense disambiguation.\nThe role of topic modeling is to detect the themes that are treated in the\ninput text. Themes within a text give clues about the particular meaning of the\nwords that are used, for example: viral has one meaning in the context of\ncomputer science (CS) and another when talking about health. To extract\nkeywords, we used TextRank with POS tag filtering. With this method, we can\nobtain relevant words that are already part of the Spanish lexicon. We use a\ndeep learning model to determine if a given keyword could have a new meaning.\nEmbeddings that are different from all the known meanings (or topics) indicate\nthat a word might be a valid SN candidate. In this study, we examine the\nfollowing word embedding models: Word2Vec, Sense2Vec, and FastText. The models\nwere trained with equivalent parameters using Wikipedia in Spanish as corpora.\nThen we used a list of words and their concordances (obtained from our database\nof neologisms) to show the different embeddings that each model yields.\nFinally, we present a comparison of these outcomes with the concordances of\neach word to show how we can determine if a word could be a valid candidate for\nSN.",
            "journal ref": "COnference en Recherche d'Informations et Applications {CORIA}\n  2019 France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05286v1",
            "date": "2020-01-11 18:05:15+00:00",
            "title": "Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses",
            "authors": [
                "Kashyap Coimbatore Murali"
            ],
            "abstract": "In this paper, we explore the robustness of the Multi-Task Deep Neural\nNetworks (MT-DNN) against non-targeted adversarial attacks across Natural\nLanguage Understanding (NLU) tasks as well as some possible ways to defend\nagainst them. Liu et al., have shown that the Multi-Task Deep Neural Network,\ndue to the regularization effect produced when training as a result of its\ncross task data, is more robust than a vanilla BERT model trained only on one\ntask (1.1%-1.5% absolute difference). We further show that although the MT-DNN\nhas generalized better, making it easily transferable across domains and tasks,\nit can still be compromised as after only 2 attacks (1-character and\n2-character) the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail\ntasks. Finally, we propose a domain agnostic defense which restores the model's\naccuracy (36.75% and 25.94% respectively) as opposed to a general-purpose\ndefense or an off-the-shelf spell checker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05292v1",
            "date": "2020-01-09 20:52:38+00:00",
            "title": "The empirical structure of word frequency distributions",
            "authors": [
                "Michael Ramscar"
            ],
            "abstract": "The frequencies at which individual words occur across languages follow power\nlaw distributions, a pattern of findings known as Zipf's law. A vast literature\nargues over whether this serves to optimize the efficiency of human\ncommunication, however this claim is necessarily post hoc, and it has been\nsuggested that Zipf's law may in fact describe mixtures of other distributions.\nFrom this perspective, recent findings that Sinosphere first (family) names are\ngeometrically distributed are notable, because this is actually consistent with\ninformation theoretic predictions regarding optimal coding. First names form\nnatural communicative distributions in most languages, and I show that when\nanalyzed in relation to the communities in which they are used, first name\ndistributions across a diverse set of languages are both geometric and,\nhistorically, remarkably similar, with power law distributions only emerging\nwhen empirical distributions are aggregated. I then show this pattern of\nfindings replicates in communicative distributions of English nouns and verbs.\nThese results indicate that if lexical distributions support efficient\ncommunication, they do so because their functional structures directly satisfy\nthe constraints described by information theory, and not because of Zipf's law.\nUnderstanding the function of these information structures is likely to be key\nto explaining humankind's remarkable communicative capacities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05296v1",
            "date": "2020-01-12 17:30:42+00:00",
            "title": "Urdu-English Machine Transliteration using Neural Networks",
            "authors": [
                "Usman Mohy ud Din"
            ],
            "abstract": "Machine translation has gained much attention in recent years. It is a\nsub-field of computational linguistic which focus on translating text from one\nlanguage to other language. Among different translation techniques, neural\nnetwork currently leading the domain with its capabilities of providing a\nsingle large neural network with attention mechanism, sequence-to-sequence and\nlong-short term modelling. Despite significant progress in domain of machine\ntranslation, translation of out-of-vocabulary words(OOV) which include\ntechnical terms, named-entities, foreign words are still a challenge for\ncurrent state-of-art translation systems, and this situation becomes even worse\nwhile translating between low resource languages or languages having different\nstructures. Due to morphological richness of a language, a word may have\ndifferent meninges in different context. In such scenarios, translation of word\nis not only enough in order provide the correct/quality translation.\nTransliteration is a way to consider the context of word/sentence during\ntranslation. For low resource language like Urdu, it is very difficult to\nhave/find parallel corpus for transliteration which is large enough to train\nthe system. In this work, we presented transliteration technique based on\nExpectation Maximization (EM) which is un-supervised and language independent.\nSystems learns the pattern and out-of-vocabulary (OOV) words from parallel\ncorpus and there is no need to train it on transliteration corpus explicitly.\nThis approach is tested on three models of statistical machine translation\n(SMT) which include phrasebased, hierarchical phrase-based and factor based\nmodels and two models of neural machine translation which include LSTM and\ntransformer model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05297v1",
            "date": "2020-01-13 11:23:28+00:00",
            "title": "Dialectal Layers in West Iranian: a Hierarchical Dirichlet Process Approach to Linguistic Relationships",
            "authors": [
                "Chundra Aroor Cathcart"
            ],
            "abstract": "This paper addresses a series of complex and unresolved issues in the\nhistorical phonology of West Iranian languages. The West Iranian languages\n(Persian, Kurdish, Balochi, and other languages) display a high degree of\nnon-Lautgesetzlich behavior. Most of this irregularity is undoubtedly due to\nlanguage contact; we argue, however, that an oversimplified view of the\nprocesses at work has prevailed in the literature on West Iranian dialectology,\nwith specialists assuming that deviations from an expected outcome in a given\nnon-Persian language are due to lexical borrowing from some chronological stage\nof Persian. It is demonstrated that this qualitative approach yields at times\nproblematic conclusions stemming from the lack of explicit probabilistic\ninferences regarding the distribution of the data: Persian may not be the sole\ndonor language; additionally, borrowing at the lexical level is not always the\nmechanism that introduces irregularity. In many cases, the possibility that\nWest Iranian languages show different reflexes in different conditioning\nenvironments remains under-explored. We employ a novel Bayesian approach\ndesigned to overcome these problems and tease apart the different determinants\nof irregularity in patterns of West Iranian sound change. Our methodology\nallows us to provisionally resolve a number of outstanding questions in the\nliterature on West Iranian dialectology concerning the dialectal affiliation of\ncertain sound changes. We outline future directions for work of this sort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05326v1",
            "date": "2020-01-14 13:50:08+00:00",
            "title": "A BERT based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts",
            "authors": [
                "Lingyun Zhao",
                "Lin Li",
                "Xinhao Zheng"
            ],
            "abstract": "The emergence and rapid progress of the Internet have brought ever-increasing\nimpact on financial domain. How to rapidly and accurately mine the key\ninformation from the massive negative financial texts has become one of the key\nissues for investors and decision makers. Aiming at the issue, we propose a\nsentiment analysis and key entity detection approach based on BERT, which is\napplied in online financial text mining and public opinion analysis in social\nmedia. By using pre-train model, we first study sentiment analysis, and then we\nconsider key entity detection as a sentence matching or Machine Reading\nComprehension (MRC) task in different granularity. Among them, we mainly focus\non negative sentimental information. We detect the specific entity by using our\napproach, which is different from traditional Named Entity Recognition (NER).\nIn addition, we also use ensemble learning to improve the performance of\nproposed approach. Experimental results show that the performance of our\napproach is generally higher than SVM, LR, NBM, and BERT for two financial\nsentiment analysis and key entity detection datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05609v5",
            "date": "2020-01-16 01:49:16+00:00",
            "title": "Schema2QA: High-Quality and Low-Cost Q&A Agents for the Structured Web",
            "authors": [
                "Silei Xu",
                "Giovanni Campagna",
                "Jian Li",
                "Monica S. Lam"
            ],
            "abstract": "Building a question-answering agent currently requires large annotated\ndatasets, which are prohibitively expensive. This paper proposes Schema2QA, an\nopen-source toolkit that can generate a Q&A system from a database schema\naugmented with a few annotations for each field. The key concept is to cover\nthe space of possible compound queries on the database with a large number of\nin-domain questions synthesized with the help of a corpus of generic query\ntemplates. The synthesized data and a small paraphrase set are used to train a\nnovel neural network based on the BERT pretrained model. We use Schema2QA to\ngenerate Q&A systems for five Schema.org domains, restaurants, people, movies,\nbooks and music, and obtain an overall accuracy between 64% and 75% on\ncrowdsourced questions for these domains. Once annotations and paraphrases are\nobtained for a Schema.org schema, no additional manual effort is needed to\ncreate a Q&A agent for any website that uses the same schema. Furthermore, we\ndemonstrate that learning can be transferred from the restaurant to the hotel\ndomain, obtaining a 64% accuracy on crowdsourced questions with no manual\neffort. Schema2QA achieves an accuracy of 60% on popular restaurant questions\nthat can be answered using Schema.org. Its performance is comparable to Google\nAssistant, 7% lower than Siri, and 15% higher than Alexa. It outperforms all\nthese assistants by at least 18% on more complex, long-tail questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05672v1",
            "date": "2020-01-16 06:31:53+00:00",
            "title": "AandP: Utilizing Prolog for converting between active sentence and passive sentence with three-steps conversion",
            "authors": [
                "Trung Q. Tran"
            ],
            "abstract": "I introduce a simple but efficient method to solve one of the critical\naspects of English grammar which is the relationship between active sentence\nand passive sentence. In fact, an active sentence and its corresponding passive\nsentence express the same meaning, but their structure is different. I utilized\nProlog [4] along with Definite Clause Grammars (DCG) [5] for doing the\nconversion between active sentence and passive sentence. Some advanced\ntechniques were also used such as Extra Arguments, Extra Goals, Lexicon, etc. I\ntried to solve a variety of cases of active and passive sentences such as 12\nEnglish tenses, modal verbs, negative form, etc. More details and my\ncontributions will be presented in the following sections. The source code is\navailable at https://github.com/tqtrunghnvn/ActiveAndPassive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05687v5",
            "date": "2020-01-16 08:09:51+00:00",
            "title": "Enhancing lexical-based approach with external knowledge for Vietnamese multiple-choice machine reading comprehension",
            "authors": [
                "Kiet Van Nguyen",
                "Khiem Vinh Tran",
                "Son T. Luu",
                "Anh Gia-Tuan Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Although Vietnamese is the 17th most popular native-speaker language in the\nworld, there are not many research studies on Vietnamese machine reading\ncomprehension (MRC), the task of understanding a text and answering questions\nabout it. One of the reasons is because of the lack of high-quality benchmark\ndatasets for this task. In this work, we construct a dataset which consists of\n2,783 pairs of multiple-choice questions and answers based on 417 Vietnamese\ntexts which are commonly used for teaching reading comprehension for elementary\nschool pupils. In addition, we propose a lexical-based MRC method that utilizes\nsemantic similarity measures and external knowledge sources to analyze\nquestions and extract answers from the given text. We compare the performance\nof the proposed model with several baseline lexical-based and neural\nnetwork-based models. Our proposed method achieves 61.81% by accuracy, which is\n5.51% higher than the best baseline model. We also measure human performance on\nour dataset and find that there is a big gap between machine-model and human\nperformances. This indicates that significant progress can be made on this\ntask. The dataset is freely available on our website for research purposes.",
            "journal ref": "IEEE Access, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05714v1",
            "date": "2020-01-16 09:42:29+00:00",
            "title": "Comparing Rule-based, Feature-based and Deep Neural Methods for De-identification of Dutch Medical Records",
            "authors": [
                "Jan Trienes",
                "Dolf Trieschnigg",
                "Christin Seifert",
                "Djoerd Hiemstra"
            ],
            "abstract": "Unstructured information in electronic health records provide an invaluable\nresource for medical research. To protect the confidentiality of patients and\nto conform to privacy regulations, de-identification methods automatically\nremove personally identifying information from these medical records. However,\ndue to the unavailability of labeled data, most existing research is\nconstrained to English medical text and little is known about the\ngeneralizability of de-identification methods across languages and domains. In\nthis study, we construct a varied dataset consisting of the medical records of\n1260 patients by sampling data from 9 institutes and three domains of Dutch\nhealthcare. We test the generalizability of three de-identification methods\nacross languages and domains. Our experiments show that an existing rule-based\nmethod specifically developed for the Dutch language fails to generalize to\nthis new data. Furthermore, a state-of-the-art neural architecture performs\nstrongly across languages and domains, even with limited training data.\nCompared to feature-based and rule-based methods the neural method requires\nsignificantly less configuration effort and domain-knowledge. We make all code\nand pre-trained de-identification models available to the research community,\nallowing practitioners to apply them to their datasets and to enable future\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05954v1",
            "date": "2020-01-16 17:30:36+00:00",
            "title": "Lexical Sememe Prediction using Dictionary Definitions by Capturing Local Semantic Correspondence",
            "authors": [
                "Jiaju Du",
                "Fanchao Qi",
                "Maosong Sun",
                "Zhiyuan Liu"
            ],
            "abstract": "Sememes, defined as the minimum semantic units of human languages in\nlinguistics, have been proven useful in many NLP tasks. Since manual\nconstruction and update of sememe knowledge bases (KBs) are costly, the task of\nautomatic sememe prediction has been proposed to assist sememe annotation. In\nthis paper, we explore the approach of applying dictionary definitions to\npredicting sememes for unannotated words. We find that sememes of each word are\nusually semantically matched to different words in its dictionary definition,\nand we name this matching relationship local semantic correspondence.\nAccordingly, we propose a Sememe Correspondence Pooling (SCorP) model, which is\nable to capture this kind of matching to predict sememes. We evaluate our model\nand baseline methods on a famous sememe KB HowNet and find that our model\nachieves state-of-the-art performance. Moreover, further quantitative analysis\nshows that our model can properly learn the local semantic correspondence\nbetween sememes and words in dictionary definitions, which explains the\neffectiveness of our model. The source codes of this paper can be obtained from\nhttps://github.com/thunlp/scorp.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.06206v1",
            "date": "2020-01-17 09:18:00+00:00",
            "title": "Multi-step Joint-Modality Attention Network for Scene-Aware Dialogue System",
            "authors": [
                "Yun-Wei Chu",
                "Kuan-Yen Lin",
                "Chao-Chun Hsu",
                "Lun-Wei Ku"
            ],
            "abstract": "Understanding dynamic scenes and dialogue contexts in order to converse with\nusers has been challenging for multimodal dialogue systems. The 8-th Dialog\nSystem Technology Challenge (DSTC8) proposed an Audio Visual Scene-Aware Dialog\n(AVSD) task, which contains multiple modalities including audio, vision, and\nlanguage, to evaluate how dialogue systems understand different modalities and\nresponse to users. In this paper, we proposed a multi-step joint-modality\nattention network (JMAN) based on recurrent neural network (RNN) to reason on\nvideos. Our model performs a multi-step attention mechanism and jointly\nconsiders both visual and textual representations in each reasoning process to\nbetter integrate information from the two different modalities. Compared to the\nbaseline released by AVSD organizers, our model achieves a relative 12.1% and\n22.4% improvement over the baseline on ROUGE-L score and CIDEr score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.06381v1",
            "date": "2020-01-17 15:42:29+00:00",
            "title": "A Common Semantic Space for Monolingual and Cross-Lingual Meta-Embeddings",
            "authors": [
                "Iker Garc\u00eda",
                "Rodrigo Agerri",
                "German Rigau"
            ],
            "abstract": "This paper presents a new technique for creating monolingual and\ncross-lingual meta-embeddings. Our method integrates multiple word embeddings\ncreated from complementary techniques, textual sources, knowledge bases and\nlanguages. Existing word vectors are projected to a common semantic space using\nlinear transformations and averaging. With our method the resulting\nmeta-embeddings maintain the dimensionality of the original embeddings without\nlosing information while dealing with the out-of-vocabulary problem. An\nextensive empirical evaluation demonstrates the effectiveness of our technique\nwith respect to previous work on various intrinsic and extrinsic multilingual\nevaluations, obtaining competitive results for Semantic Textual Similarity and\nstate-of-the-art performance for word similarity and POS tagging (English and\nSpanish). The resulting cross-lingual meta-embeddings also exhibit excellent\ncross-lingual transfer learning capabilities. In other words, we can leverage\npre-trained source embeddings from a resource-rich language in order to improve\nthe word representations for under-resourced languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.06629v2",
            "date": "2020-01-18 09:04:42+00:00",
            "title": "Capturing Evolution in Word Usage: Just Add More Clusters?",
            "authors": [
                "Matej Martinc",
                "Syrielle Montariol",
                "Elaine Zosa",
                "Lidia Pivovarova"
            ],
            "abstract": "The way the words are used evolves through time, mirroring cultural or\ntechnological evolution of society. Semantic change detection is the task of\ndetecting and analysing word evolution in textual data, even in short periods\nof time. In this paper we focus on a new set of methods relying on\ncontextualised embeddings, a type of semantic modelling that revolutionised the\nNLP field recently. We leverage the ability of the transformer-based BERT model\nto generate contextualised embeddings capable of detecting semantic change of\nwords across time. Several approaches are compared in a common setting in order\nto establish strengths and weaknesses for each of them. We also propose several\nideas for improvements, managing to drastically improve the performance of\nexisting approaches.",
            "journal ref": "WWW 20 Companion Proceedings of the Web Conference 2020 (April\n  2020) p. 343-349",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07209v1",
            "date": "2020-01-20 18:52:45+00:00",
            "title": "Text-based inference of moral sentiment change",
            "authors": [
                "Jing Yi Xie",
                "Renato Ferreira Pinto Jr.",
                "Graeme Hirst",
                "Yang Xu"
            ],
            "abstract": "We present a text-based framework for investigating moral sentiment change of\nthe public via longitudinal corpora. Our framework is based on the premise that\nlanguage use can inform people's moral perception toward right or wrong, and we\nbuild our methodology by exploring moral biases learned from diachronic word\nembeddings. We demonstrate how a parameter-free model supports inference of\nhistorical shifts in moral sentiment toward concepts such as slavery and\ndemocracy over centuries at three incremental levels: moral relevance, moral\npolarity, and fine-grained moral dimensions. We apply this methodology to\nvisualizing moral time courses of individual concepts and analyzing the\nrelations between psycholinguistic variables and rates of moral sentiment\nchange at scale. Our work offers opportunities for applying natural language\nprocessing toward characterizing moral sentiment change in society.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07234v1",
            "date": "2020-01-20 20:02:02+00:00",
            "title": "Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching",
            "authors": [
                "Shuohang Wang",
                "Yunshi Lan",
                "Yi Tay",
                "Jing Jiang",
                "Jingjing Liu"
            ],
            "abstract": "Transformer has been successfully applied to many natural language processing\ntasks. However, for textual sequence matching, simple matching between the\nrepresentation of a pair of sequences might bring in unnecessary noise. In this\npaper, we propose a new approach to sequence pair matching with Transformer, by\nlearning head-wise matching representations on multiple levels. Experiments\nshow that our proposed approach can achieve new state-of-the-art performance on\nmultiple tasks that rely only on pre-computed sequence-vector-representation,\nsuch as SNLI, MNLI-match, MNLI-mismatch, QQP, and SQuAD-binary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07320v1",
            "date": "2020-01-21 03:10:02+00:00",
            "title": "A Hierarchical Location Normalization System for Text",
            "authors": [
                "Dongyun Liang",
                "Guohua Wang",
                "Jing Nie",
                "Binxu Zhai",
                "Xiusen Gu"
            ],
            "abstract": "It's natural these days for people to know the local events from massive\ndocuments. Many texts contain location information, such as city name or road\nname, which is always incomplete or latent. It's significant to extract the\nadministrative area of the text and organize the hierarchy of area, called\nlocation normalization. Existing detecting location systems either exclude\nhierarchical normalization or present only a few specific regions. We propose a\nsystem named ROIBase that normalizes the text by the Chinese hierarchical\nadministrative divisions. ROIBase adopts a co-occurrence constraint as the\nbasic framework to score the hit of the administrative area, achieves the\ninference by special embeddings, and expands the recall by the ROI (region of\ninterest). It has high efficiency and interpretability because it mainly\nestablishes on the definite knowledge and has less complex logic than the\nsupervised models. We demonstrate that ROIBase achieves better performance\nagainst feasible solutions and is useful as a strong support system for\nlocation normalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07331v1",
            "date": "2020-01-21 04:01:58+00:00",
            "title": "Length-controllable Abstractive Summarization by Guiding with Summary Prototype",
            "authors": [
                "Itsumi Saito",
                "Kyosuke Nishida",
                "Kosuke Nishida",
                "Atsushi Otsuka",
                "Hisako Asano",
                "Junji Tomita",
                "Hiroyuki Shindo",
                "Yuji Matsumoto"
            ],
            "abstract": "We propose a new length-controllable abstractive summarization model. Recent\nstate-of-the-art abstractive summarization models based on encoder-decoder\nmodels generate only one summary per source text. However, controllable\nsummarization, especially of the length, is an important aspect for practical\napplications. Previous studies on length-controllable abstractive summarization\nincorporate length embeddings in the decoder module for controlling the summary\nlength. Although the length embeddings can control where to stop decoding, they\ndo not decide which information should be included in the summary within the\nlength constraint. Unlike the previous models, our length-controllable\nabstractive summarization model incorporates a word-level extractive module in\nthe encoder-decoder model instead of length embeddings. Our model generates a\nsummary in two steps. First, our word-level extractor extracts a sequence of\nimportant words (we call it the \"prototype text\") from the source text\naccording to the word-level importance scores and the length constraint.\nSecond, the prototype text is used as additional input to the encoder-decoder\nmodel, which generates a summary by jointly encoding and copying words from\nboth the prototype text and source text. Since the prototype text is a guide to\nboth the content and length of the summary, our model can generate an\ninformative and length-controlled summary. Experiments with the CNN/Daily Mail\ndataset and the NEWSROOM dataset show that our model outperformed previous\nmodels in length-controlled settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07418v1",
            "date": "2020-01-21 10:02:18+00:00",
            "title": "A Physical Embedding Model for Knowledge Graphs",
            "authors": [
                "Caglar Demir",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "Knowledge graph embedding methods learn continuous vector representations for\nentities in knowledge graphs and have been used successfully in a large number\nof applications. We present a novel and scalable paradigm for the computation\nof knowledge graph embeddings, which we dub PYKE . Our approach combines a\nphysical model based on Hooke's law and its inverse with ideas from simulated\nannealing to compute embeddings for knowledge graphs efficiently. We prove that\nPYKE achieves a linear space complexity. While the time complexity for the\ninitialization of our approach is quadratic, the time complexity of each of its\niterations is linear in the size of the input knowledge graph. Hence, PYKE's\noverall runtime is close to linear. Consequently, our approach easily scales up\nto knowledge graphs containing millions of triples. We evaluate our approach\nagainst six state-of-the-art embedding approaches on the DrugBank and DBpedia\ndatasets in two series of experiments. The first series shows that the cluster\npurity achieved by PYKE is up to 26% (absolute) better than that of the state\nof art. In addition, PYKE is more than 22 times faster than existing embedding\nsolutions in the best case. The results of our second series of experiments\nshow that PYKE is up to 23% (absolute) better than the state of art on the task\nof type prediction while maintaining its superior scalability. Our\nimplementation and results are open-source and are available at\nhttp://github.com/dice-group/PYKE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07574v1",
            "date": "2020-01-21 14:39:20+00:00",
            "title": "Generating Sense Embeddings for Syntactic and Semantic Analogy for Portuguese",
            "authors": [
                "Jessica Rodrigues da Silva",
                "Helena de Medeiros Caseli"
            ],
            "abstract": "Word embeddings are numerical vectors which can represent words or concepts\nin a low-dimensional continuous space. These vectors are able to capture useful\nsyntactic and semantic information. The traditional approaches like Word2Vec,\nGloVe and FastText have a strict drawback: they produce a single vector\nrepresentation per word ignoring the fact that ambiguous words can assume\ndifferent meanings. In this paper we use techniques to generate sense\nembeddings and present the first experiments carried out for Portuguese. Our\nexperiments show that sense vectors outperform traditional word vectors in\nsyntactic and semantic analogy tasks, proving that the language resource\ngenerated here can improve the performance of NLP tasks in Portuguese.",
            "journal ref": "STIL XII (2019) 104-113",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07676v3",
            "date": "2020-01-21 17:57:33+00:00",
            "title": "Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Some NLP tasks can be solved in a fully unsupervised fashion by providing a\npretrained language model with \"task descriptions\" in natural language (e.g.,\nRadford et al., 2019). While this approach underperforms its supervised\ncounterpart, we show in this work that the two ideas can be combined: We\nintroduce Pattern-Exploiting Training (PET), a semi-supervised training\nprocedure that reformulates input examples as cloze-style phrases to help\nlanguage models understand a given task. These phrases are then used to assign\nsoft labels to a large set of unlabeled examples. Finally, standard supervised\ntraining is performed on the resulting training set. For several tasks and\nlanguages, PET outperforms supervised training and strong semi-supervised\napproaches in low-resource settings by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07740v1",
            "date": "2020-01-21 19:09:49+00:00",
            "title": "Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods",
            "authors": [
                "Maria Ryskina",
                "Ella Rabinovich",
                "Taylor Berg-Kirkpatrick",
                "David R. Mortensen",
                "Yulia Tsvetkov"
            ],
            "abstract": "We perform statistical analysis of the phenomenon of neology, the process by\nwhich new words emerge in a language, using large diachronic corpora of\nEnglish. We investigate the importance of two factors, semantic sparsity and\nfrequency growth rates of semantic neighbors, formalized in the distributional\nsemantics paradigm. We show that both factors are predictive of word emergence\nalthough we find more support for the latter hypothesis. Besides presenting a\nnew linguistic application of distributional semantics, this study tackles the\nlinguistic question of the role of language-internal factors (in our case,\nsparsity) in language change motivated by language-external factors (reflected\nin frequency growth).",
            "journal ref": "Proceedings of the Society for Computation in Linguistics 3.1\n  (2020): 43-52",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07786v2",
            "date": "2020-01-21 21:47:27+00:00",
            "title": "Shared task: Lexical semantic change detection in German (Student Project Report)",
            "authors": [
                "Adnan Ahmad",
                "Kiflom Desta",
                "Fabian Lang",
                "Dominik Schlechtweg"
            ],
            "abstract": "Recent NLP architectures have illustrated in various ways how semantic change\ncan be captured across time and domains. However, in terms of evaluation there\nis a lack of benchmarks to compare the performance of these systems against\neach other. We present the results of the first shared task on unsupervised\nlexical semantic change detection (LSCD) in German based on the evaluation\nframework proposed by Schlechtweg et al. (2019).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08051v1",
            "date": "2020-01-22 15:14:09+00:00",
            "title": "TLT-school: a Corpus of Non Native Children Speech",
            "authors": [
                "Roberto Gretter",
                "Marco Matassoni",
                "Stefano Bann\u00f2",
                "Daniele Falavigna"
            ],
            "abstract": "This paper describes \"TLT-school\" a corpus of speech utterances collected in\nschools of northern Italy for assessing the performance of students learning\nboth English and German. The corpus was recorded in the years 2017 and 2018\nfrom students aged between nine and sixteen years, attending primary, middle\nand high school. All utterances have been scored, in terms of some predefined\nproficiency indicators, by human experts. In addition, most of utterances\nrecorded in 2017 have been manually transcribed carefully. Guidelines and\nprocedures used for manual transcriptions of utterances will be described in\ndetail, as well as results achieved by means of an automatic speech recognition\nsystem developed by us. Part of the corpus is going to be freely distributed to\nscientific community particularly interested both in non-native speech\nrecognition and automatic assessment of second language proficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08210v2",
            "date": "2020-01-22 18:59:17+00:00",
            "title": "Multilingual Denoising Pre-training for Neural Machine Translation",
            "authors": [
                "Yinhan Liu",
                "Jiatao Gu",
                "Naman Goyal",
                "Xian Li",
                "Sergey Edunov",
                "Marjan Ghazvininejad",
                "Mike Lewis",
                "Luke Zettlemoyer"
            ],
            "abstract": "This paper demonstrates that multilingual denoising pre-training produces\nsignificant performance gains across a wide variety of machine translation (MT)\ntasks. We present mBART -- a sequence-to-sequence denoising auto-encoder\npre-trained on large-scale monolingual corpora in many languages using the BART\nobjective. mBART is one of the first methods for pre-training a complete\nsequence-to-sequence model by denoising full texts in multiple languages, while\nprevious approaches have focused only on the encoder, decoder, or\nreconstructing parts of the text. Pre-training a complete model allows it to be\ndirectly fine tuned for supervised (both sentence-level and document-level) and\nunsupervised machine translation, with no task-specific modifications. We\ndemonstrate that adding mBART initialization produces performance gains in all\nbut the highest-resource settings, including up to 12 BLEU points for low\nresource MT and over 5 BLEU points for many document-level and unsupervised\nmodels. We also show it also enables new types of transfer to language pairs\nwith no bi-text or that were not in the pre-training corpus, and present\nextensive analysis of which factors contribute the most to effective\npre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08728v1",
            "date": "2020-01-23 18:41:21+00:00",
            "title": "Coordinated Reasoning for Cross-Lingual Knowledge Graph Alignment",
            "authors": [
                "Kun Xu",
                "Linfeng Song",
                "Yansong Feng",
                "Yan Song",
                "Dong Yu"
            ],
            "abstract": "Existing entity alignment methods mainly vary on the choices of encoding the\nknowledge graph, but they typically use the same decoding method, which\nindependently chooses the local optimal match for each source entity. This\ndecoding method may not only cause the \"many-to-one\" problem but also neglect\nthe coordinated nature of this task, that is, each alignment decision may\nhighly correlate to the other decisions. In this paper, we introduce two\ncoordinated reasoning methods, i.e., the Easy-to-Hard decoding strategy and\njoint entity alignment algorithm. Specifically, the Easy-to-Hard strategy first\nretrieves the model-confident alignments from the predicted results and then\nincorporates them as additional knowledge to resolve the remaining\nmodel-uncertain alignments. To achieve this, we further propose an enhanced\nalignment model that is built on the current state-of-the-art baseline. In\naddition, to address the many-to-one problem, we propose to jointly predict\nentity alignments so that the one-to-one constraint can be naturally\nincorporated into the alignment prediction. Experimental results show that our\nmodel achieves the state-of-the-art performance and our reasoning methods can\nalso significantly improve existing baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08764v2",
            "date": "2020-01-23 19:06:18+00:00",
            "title": "Reducing Non-Normative Text Generation from Language Models",
            "authors": [
                "Xiangyu Peng",
                "Siyan Li",
                "Spencer Frazier",
                "Mark Riedl"
            ],
            "abstract": "Large-scale, transformer-based language models such as GPT-2 are pretrained\non diverse corpora scraped from the internet. Consequently, they are prone to\ngenerating non-normative text (i.e. in violation of social norms). We introduce\na technique for fine-tuning GPT-2, using a policy gradient reinforcement\nlearning technique and a normative text classifier to produce reward and\npunishment values. We evaluate our technique on five data sets using automated\nand human participant experiments. The normative text classifier is 81-90%\naccurate when compared to gold-standard human judgments of normative and\nnon-normative generated text. Our normative fine-tuning technique is able to\nreduce non-normative text by 27-61%, depending on the data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09415v5",
            "date": "2020-01-26 07:35:02+00:00",
            "title": "DUMA: Reading Comprehension with Transposition Thinking",
            "authors": [
                "Pengfei Zhu",
                "Hai Zhao",
                "Xiaoguang Li"
            ],
            "abstract": "Multi-choice Machine Reading Comprehension (MRC) requires model to decide the\ncorrect answer from a set of answer options when given a passage and a\nquestion. Thus in addition to a powerful Pre-trained Language Model (PrLM) as\nencoder, multi-choice MRC especially relies on a matching network design which\nis supposed to effectively capture the relationships among the triplet of\npassage, question and answers. While the newer and more powerful PrLMs have\nshown their mightiness even without the support from a matching network, we\npropose a new DUal Multi-head Co-Attention (DUMA) model, which is inspired by\nhuman's transposition thinking process solving the multi-choice MRC problem:\nrespectively considering each other's focus from the standpoint of passage and\nquestion. The proposed DUMA has been shown effective and is capable of\ngenerally promoting PrLMs. Our proposed method is evaluated on two benchmark\nmulti-choice MRC tasks, DREAM and RACE, showing that in terms of powerful\nPrLMs, DUMA can still boost the model to reach new state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09466v3",
            "date": "2020-01-26 15:16:37+00:00",
            "title": "From Stock Prediction to Financial Relevance: Repurposing Attention Weights to Assess News Relevance Without Manual Annotations",
            "authors": [
                "Luciano Del Corro",
                "Johannes Hoffart"
            ],
            "abstract": "We present a method to automatically identify financially relevant news using\nstock price movements and news headlines as input. The method repurposes the\nattention weights of a neural network initially trained to predict stock prices\nto assign a relevance score to each headline, eliminating the need for manually\nlabeled training data. Our experiments on the four most relevant US stock\nindices and 1.5M news headlines show that the method ranks relevant news\nhighly, positively correlated with the accuracy of the initial stock price\nprediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09879v1",
            "date": "2020-01-27 16:01:10+00:00",
            "title": "Towards Quantifying the Distance between Opinions",
            "authors": [
                "Saket Gurukar",
                "Deepak Ajwani",
                "Sourav Dutta",
                "Juho Lauri",
                "Srinivasan Parthasarathy",
                "Alessandra Sala"
            ],
            "abstract": "Increasingly, critical decisions in public policy, governance, and business\nstrategy rely on a deeper understanding of the needs and opinions of\nconstituent members (e.g. citizens, shareholders). While it has become easier\nto collect a large number of opinions on a topic, there is a necessity for\nautomated tools to help navigate the space of opinions. In such contexts\nunderstanding and quantifying the similarity between opinions is key. We find\nthat measures based solely on text similarity or on overall sentiment often\nfail to effectively capture the distance between opinions. Thus, we propose a\nnew distance measure for capturing the similarity between opinions that\nleverages the nuanced observation -- similar opinions express similar sentiment\npolarity on specific relevant entities-of-interest. Specifically, in an\nunsupervised setting, our distance measure achieves significantly better\nAdjusted Rand Index scores (up to 56x) and Silhouette coefficients (up to 21x)\ncompared to existing approaches. Similarly, in a supervised setting, our\nopinion distance measure achieves considerably better accuracy (up to 20%\nincrease) compared to extant approaches that rely on text similarity, stance\nsimilarity, and sentiment similarity",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09907v1",
            "date": "2020-01-27 16:51:39+00:00",
            "title": "PMIndia -- A Collection of Parallel Corpora of Languages of India",
            "authors": [
                "Barry Haddow",
                "Faheem Kirefu"
            ],
            "abstract": "Parallel text is required for building high-quality machine translation (MT)\nsystems, as well as for other multilingual NLP applications. For many South\nAsian languages, such data is in short supply. In this paper, we described a\nnew publicly available corpus (PMIndia) consisting of parallel sentences which\npair 13 major languages of India with English. The corpus includes up to 56000\nsentences for each language pair. We explain how the corpus was constructed,\nincluding an assessment of two different automatic sentence alignment methods,\nand present some initial NMT results on the corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10106v1",
            "date": "2020-01-27 22:34:07+00:00",
            "title": "Guiding Corpus-based Set Expansion by Auxiliary Sets Generation and Co-Expansion",
            "authors": [
                "Jiaxin Huang",
                "Yiqing Xie",
                "Yu Meng",
                "Jiaming Shen",
                "Yunyi Zhang",
                "Jiawei Han"
            ],
            "abstract": "Given a small set of seed entities (e.g., ``USA'', ``Russia''), corpus-based\nset expansion is to induce an extensive set of entities which share the same\nsemantic class (Country in this example) from a given corpus. Set expansion\nbenefits a wide range of downstream applications in knowledge discovery, such\nas web search, taxonomy construction, and query suggestion. Existing\ncorpus-based set expansion algorithms typically bootstrap the given seeds by\nincorporating lexical patterns and distributional similarity. However, due to\nno negative sets provided explicitly, these methods suffer from semantic drift\ncaused by expanding the seed set freely without guidance. We propose a new\nframework, Set-CoExpan, that automatically generates auxiliary sets as negative\nsets that are closely related to the target set of user's interest, and then\nperforms multiple sets co-expansion that extracts discriminative features by\ncomparing target set with auxiliary sets, to form multiple cohesive sets that\nare distinctive from one another, thus resolving the semantic drift issue. In\nthis paper we demonstrate that by generating auxiliary sets, we can guide the\nexpansion process of target set to avoid touching those ambiguous areas around\nthe border with auxiliary sets, and we show that Set-CoExpan outperforms strong\nbaseline methods significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10169v1",
            "date": "2020-01-28 05:03:15+00:00",
            "title": "A Deep Neural Framework for Contextual Affect Detection",
            "authors": [
                "Kumar Shikhar Deep",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "A short and simple text carrying no emotion can represent some strong\nemotions when reading along with its context, i.e., the same sentence can\nexpress extreme anger as well as happiness depending on its context. In this\npaper, we propose a Contextual Affect Detection (CAD) framework which learns\nthe inter-dependence of words in a sentence, and at the same time the\ninter-dependence of sentences in a dialogue. Our proposed CAD framework is\nbased on a Gated Recurrent Unit (GRU), which is further assisted by contextual\nword embeddings and other diverse hand-crafted feature sets. Evaluation and\nanalysis suggest that our model outperforms the state-of-the-art methods by\n5.49% and 9.14% on Friends and EmotionPush dataset, respectively.",
            "journal ref": "LNCS 11955 (2019) 398-409",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10175v1",
            "date": "2020-01-28 05:30:53+00:00",
            "title": "Extraction of Templates from Phrases Using Sequence Binary Decision Diagrams",
            "authors": [
                "Daiki Hirano",
                "Kumiko Tanaka-Ishii",
                "Andrew Finch"
            ],
            "abstract": "The extraction of templates such as ``regard X as Y'' from a set of related\nphrases requires the identification of their internal structures. This paper\npresents an unsupervised approach for extracting templates on-the-fly from only\ntagged text by using a novel relaxed variant of the Sequence Binary Decision\nDiagram (SeqBDD). A SeqBDD can compress a set of sequences into a graphical\nstructure equivalent to a minimal DFA, but more compact and better suited to\nthe task of template extraction. The main contribution of this paper is a\nrelaxed form of the SeqBDD construction algorithm that enables it to form\ngeneral representations from a small amount of data. The process of compression\nof shared structures in the text during Relaxed SeqBDD construction, naturally\ninduces the templates we wish to extract. Experiments show that the method is\ncapable of high-quality extraction on tasks based on verb+preposition templates\nfrom corpora and phrasal templates from short messages from social media.",
            "journal ref": "Natural Language Engineering, 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10179v1",
            "date": "2020-01-28 05:45:03+00:00",
            "title": "Multi-modal Sentiment Analysis using Super Characters Method on Low-power CNN Accelerator Device",
            "authors": [
                "Baohua Sun",
                "Lin Yang",
                "Hao Sha",
                "Michael Lin"
            ],
            "abstract": "Recent years NLP research has witnessed the record-breaking accuracy\nimprovement by DNN models. However, power consumption is one of the practical\nconcerns for deploying NLP systems. Most of the current state-of-the-art\nalgorithms are implemented on GPUs, which is not power-efficient and the\ndeployment cost is also very high. On the other hand, CNN Domain Specific\nAccelerator (CNN-DSA) has been in mass production providing low-power and low\ncost computation power. In this paper, we will implement the Super Characters\nmethod on the CNN-DSA. In addition, we modify the Super Characters method to\nutilize the multi-modal data, i.e. text plus tabular data in the CL-Aff\nsharedtask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11003v2",
            "date": "2020-01-29 18:24:14+00:00",
            "title": "Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs",
            "authors": [
                "Leonardo F. R. Ribeiro",
                "Yue Zhang",
                "Claire Gardent",
                "Iryna Gurevych"
            ],
            "abstract": "Recent graph-to-text models generate text from graph-based data using either\nglobal or local aggregation to learn node representations. Global node encoding\nallows explicit communication between two distant nodes, thereby neglecting\ngraph topology as all nodes are directly connected. In contrast, local node\nencoding considers the relations between neighbor nodes capturing the graph\nstructure, but it can fail to capture long-range relations. In this work, we\ngather both encoding strategies, proposing novel neural models which encode an\ninput graph combining both global and local node contexts, in order to learn\nbetter contextualized node embeddings. In our experiments, we demonstrate that\nour approaches lead to significant improvements on two graph-to-text datasets\nachieving BLEU scores of 18.01 on AGENDA dataset, and 63.69 on the WebNLG\ndataset for seen categories, outperforming state-of-the-art models by 3.7 and\n3.1 points, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11136v2",
            "date": "2020-01-30 00:09:53+00:00",
            "title": "The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures",
            "authors": [
                "Haim Dubossarsky",
                "Ivan Vuli\u0107",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "Performance in cross-lingual NLP tasks is impacted by the (dis)similarity of\nlanguages at hand: e.g., previous work has suggested there is a connection\nbetween the expected success of bilingual lexicon induction (BLI) and the\nassumption of (approximate) isomorphism between monolingual embedding spaces.\nIn this work we present a large-scale study focused on the correlations between\nmonolingual embedding space similarity and task performance, covering thousands\nof language pairs and four different tasks: BLI, parsing, POS tagging and MT.\nWe hypothesize that statistics of the spectrum of each monolingual embedding\nspace indicate how well they can be aligned. We then introduce several\nisomorphism measures between two embedding spaces, based on the relevant\nstatistics of their individual spectra. We empirically show that 1) language\nsimilarity scores derived from such spectral isomorphism measures are strongly\nassociated with performance observed in different cross-lingual tasks, and 2)\nour spectral-based measures consistently outperform previous standard\nisomorphism measures, while being computationally more tractable and easier to\ninterpret. Finally, our measures capture complementary information to\ntypologically driven language distance measures, and the combination of\nmeasures from the two families yields even higher task performance\ncorrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11164v4",
            "date": "2020-01-30 03:35:44+00:00",
            "title": "On the Importance of Word Order Information in Cross-lingual Sequence Labeling",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Samuel Cahyawijaya",
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Pascale Fung"
            ],
            "abstract": "Word order variances generally exist in different languages. In this paper,\nwe hypothesize that cross-lingual models that fit into the word order of the\nsource language might fail to handle target languages. To verify this\nhypothesis, we investigate whether making models insensitive to the word order\nof the source language can improve the adaptation performance in target\nlanguages. To do so, we reduce the source language word order information\nfitted to sequence encoders and observe the performance changes. In addition,\nbased on this hypothesis, we propose a new method for fine-tuning multilingual\nBERT in downstream cross-lingual sequence labeling tasks. Experimental results\non dialogue natural language understanding, part-of-speech tagging, and named\nentity recognition tasks show that reducing word order information fitted to\nthe model can achieve better zero-shot cross-lingual performance. Furthermore,\nour proposed methods can also be applied to strong cross-lingual baselines, and\nimprove their performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11285v1",
            "date": "2020-01-30 12:47:50+00:00",
            "title": "LowResourceEval-2019: a shared task on morphological analysis for low-resource languages",
            "authors": [
                "Elena Klyachko",
                "Alexey Sorokin",
                "Natalia Krizhanovskaya",
                "Andrew Krizhanovsky",
                "Galina Ryazanskaya"
            ],
            "abstract": "The paper describes the results of the first shared task on morphological\nanalysis for the languages of Russia, namely, Evenki, Karelian, Selkup, and\nVeps. For the languages in question, only small-sized corpora are available.\nThe tasks include morphological analysis, word form generation and morpheme\nsegmentation. Four teams participated in the shared task. Most of them use\nmachine-learning approaches, outperforming the existing rule-based ones. The\narticle describes the datasets prepared for the shared tasks and contains\nanalysis of the participants' solutions. Language corpora having different\nformats were transformed into CONLL-U format. The universal format makes the\ndatasets comparable to other language corpura and facilitates using them in\nother NLP tasks.",
            "journal ref": "Dialog 2019, Issue 18, Supplementary volume, Pp. 45-62",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11381v1",
            "date": "2020-01-17 15:42:14+00:00",
            "title": "Generaci\u00f3n autom\u00e1tica de frases literarias en espa\u00f1ol",
            "authors": [
                "Luis-Gil Moreno-Jim\u00e9nez",
                "Juan-Manuel Torres-Moreno",
                "Roseli S. Wedemann"
            ],
            "abstract": "In this work we present a state of the art in the area of Computational\nCreativity (CC). In particular, we address the automatic generation of literary\nsentences in Spanish. We propose three models of text generation based mainly\non statistical algorithms and shallow parsing analysis. We also present some\nrather encouraging preliminary results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11383v2",
            "date": "2020-01-16 07:30:19+00:00",
            "title": "Fact-aware Sentence Split and Rephrase with Permutation Invariant Training",
            "authors": [
                "Yinuo Guo",
                "Tao Ge",
                "Furu Wei"
            ],
            "abstract": "Sentence Split and Rephrase aims to break down a complex sentence into\nseveral simple sentences with its meaning preserved. Previous studies tend to\naddress the issue by seq2seq learning from parallel sentence pairs, which takes\na complex sentence as input and sequentially generates a series of simple\nsentences. However, the conventional seq2seq learning has two limitations for\nthis task: (1) it does not take into account the facts stated in the long\nsentence; As a result, the generated simple sentences may miss or inaccurately\nstate the facts in the original sentence. (2) The order variance of the simple\nsentences to be generated may confuse the seq2seq model during training because\nthe simple sentences derived from the long source sentence could be in any\norder.\n  To overcome the challenges, we first propose the Fact-aware Sentence\nEncoding, which enables the model to learn facts from the long sentence and\nthus improves the precision of sentence split; then we introduce Permutation\nInvariant Training to alleviate the effects of order variance in seq2seq\nlearning for this task. Experiments on the WebSplit-v1.0 benchmark dataset show\nthat our approaches can largely improve the performance over the previous\nseq2seq learning approaches. Moreover, an extrinsic evaluation on oie-benchmark\nverifies the effectiveness of our approaches by an observation that splitting\nlong sentences with our state-of-the-art model as preprocessing is helpful for\nimproving OpenIE performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11453v3",
            "date": "2020-01-30 16:58:56+00:00",
            "title": "Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages",
            "authors": [
                "Edoardo M. Ponti",
                "Ivan Vuli\u0107",
                "Ryan Cotterell",
                "Marinela Parovic",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "Most combinations of NLP tasks and language varieties lack in-domain examples\nfor supervised training because of the paucity of annotated data. How can\nneural models make sample-efficient generalizations from task-language\ncombinations with available data to low-resource ones? In this work, we propose\na Bayesian generative model for the space of neural parameters. We assume that\nthis space can be factorized into latent variables for each language and each\ntask. We infer the posteriors over such latent variables based on data from\nseen task-language combinations through variational inference. This enables\nzero-shot classification on unseen combinations at prediction time. For\ninstance, given training data for named entity recognition (NER) in Vietnamese\nand for part-of-speech (POS) tagging in Wolof, our model can perform accurate\npredictions for NER in Wolof. In particular, we experiment with a typologically\ndiverse sample of 33 languages from 4 continents and 11 families, and show that\nour model yields comparable or better results than state-of-the-art, zero-shot\ncross-lingual transfer methods. Moreover, we demonstrate that approximate\nBayesian model averaging results in smoother predictive distributions, whose\nentropy inversely correlates with accuracy. Hence, the proposed framework also\noffers robust estimates of prediction uncertainty. Our code is located at\ngithub.com/cambridgeltl/parameter-factorization",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11458v1",
            "date": "2020-01-30 17:11:00+00:00",
            "title": "Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing",
            "authors": [
                "Subendhu Rongali",
                "Luca Soldaini",
                "Emilio Monti",
                "Wael Hamza"
            ],
            "abstract": "Virtual assistants such as Amazon Alexa, Apple Siri, and Google Assistant\noften rely on a semantic parsing component to understand which action(s) to\nexecute for an utterance spoken by its users. Traditionally, rule-based or\nstatistical slot-filling systems have been used to parse \"simple\" queries; that\nis, queries that contain a single action and can be decomposed into a set of\nnon-overlapping entities. More recently, shift-reduce parsers have been\nproposed to process more complex utterances. These methods, while powerful,\nimpose specific limitations on the type of queries that can be parsed; namely,\nthey require a query to be representable as a parse tree.\n  In this work, we propose a unified architecture based on Sequence to Sequence\nmodels and Pointer Generator Network to handle both simple and complex queries.\nUnlike other works, our approach does not impose any restriction on the\nsemantic parse schema. Furthermore, experiments show that it achieves state of\nthe art performance on three publicly available datasets (ATIS, SNIPS, Facebook\nTOP), relatively improving between 3.3% and 7.7% in exact match accuracy over\nprevious systems. Finally, we show the effectiveness of our approach on two\ninternal datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11701v1",
            "date": "2020-01-31 08:28:07+00:00",
            "title": "Teaching Machines to Converse",
            "authors": [
                "Jiwei Li"
            ],
            "abstract": "The ability of a machine to communicate with humans has long been associated\nwith the general success of AI. This dates back to Alan Turing's epoch-making\nwork in the early 1950s, which proposes that a machine's intelligence can be\ntested by how well it, the machine, can fool a human into believing that the\nmachine is a human through dialogue conversations. Many systems learn\ngeneration rules from a minimal set of authored rules or labels on top of\nhand-coded rules or templates, and thus are both expensive and difficult to\nextend to open-domain scenarios. Recently, the emergence of neural network\nmodels the potential to solve many of the problems in dialogue learning that\nearlier systems cannot tackle: the end-to-end neural frameworks offer the\npromise of scalability and language-independence, together with the ability to\ntrack the dialogue state and then mapping between states and dialogue actions\nin a way not possible with conventional systems. On the other hand, neural\nsystems bring about new challenges: they tend to output dull and generic\nresponses; they lack a consistent or a coherent persona; they are usually\noptimized through single-turn conversations and are incapable of handling the\nlong-term success of a conversation; and they are not able to take the\nadvantage of the interactions with humans. This dissertation attempts to tackle\nthese challenges: Contributions are two-fold: (1) we address new challenges\npresented by neural network models in open-domain dialogue generation systems;\n(2) we develop interactive question-answering dialogue systems by (a) giving\nthe agent the ability to ask questions and (b) training a conversation agent\nthrough interactions with humans in an online fashion, where a bot improves\nthrough communicating with humans and learning from the mistakes that it makes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11770v1",
            "date": "2020-01-31 11:04:52+00:00",
            "title": "Break It Down: A Question Understanding Benchmark",
            "authors": [
                "Tomer Wolfson",
                "Mor Geva",
                "Ankit Gupta",
                "Matt Gardner",
                "Yoav Goldberg",
                "Daniel Deutch",
                "Jonathan Berant"
            ],
            "abstract": "Understanding natural language questions entails the ability to break down a\nquestion into the requisite steps for computing its answer. In this work, we\nintroduce a Question Decomposition Meaning Representation (QDMR) for questions.\nQDMR constitutes the ordered list of steps, expressed through natural language,\nthat are necessary for answering a question. We develop a crowdsourcing\npipeline, showing that quality QDMRs can be annotated at scale, and release the\nBreak dataset, containing over 83K pairs of questions and their QDMRs. We\ndemonstrate the utility of QDMR by showing that (a) it can be used to improve\nopen-domain question answering on the HotpotQA dataset, (b) it can be\ndeterministically converted to a pseudo-SQL formal language, which can\nalleviate annotation in semantic parsing applications. Last, we use Break to\ntrain a sequence-to-sequence model with copying that parses questions into QDMR\nstructures, and show that it substantially outperforms several natural\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11857v1",
            "date": "2020-01-31 14:08:15+00:00",
            "title": "Hybrid Tiled Convolutional Neural Networks for Text Sentiment Classification",
            "authors": [
                "Maria Mihaela Trusca",
                "Gerasimos Spanakis"
            ],
            "abstract": "The tiled convolutional neural network (tiled CNN) has been applied only to\ncomputer vision for learning invariances. We adjust its architecture to NLP to\nimprove the extraction of the most salient features for sentiment analysis.\nKnowing that the major drawback of the tiled CNN in the NLP field is its\ninflexible filter structure, we propose a novel architecture called hybrid\ntiled CNN that applies a filter only on the words that appear in the similar\ncontexts and on their neighbor words (a necessary step for preventing the loss\nof some n-grams). The experiments on the datasets of IMDB movie reviews and\nSemEval 2017 demonstrate the efficiency of the hybrid tiled CNN that performs\nbetter than both CNN and tiled CNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11899v1",
            "date": "2020-01-31 15:25:56+00:00",
            "title": "An efficient automated data analytics approach to large scale computational comparative linguistics",
            "authors": [
                "Gabija Mikulyte",
                "David Gilbert"
            ],
            "abstract": "This research project aimed to overcome the challenge of analysing human\nlanguage relationships, facilitate the grouping of languages and formation of\ngenealogical relationship between them by developing automated comparison\ntechniques. Techniques were based on the phonetic representation of certain key\nwords and concept. Example word sets included numbers 1-10 (curated), large\ndatabase of numbers 1-10 and sheep counting numbers 1-10 (other sources),\ncolours (curated), basic words (curated).\n  To enable comparison within the sets the measure of Edit distance was\ncalculated based on Levenshtein distance metric. This metric between two\nstrings is the minimum number of single-character edits, operations including:\ninsertions, deletions or substitutions. To explore which words exhibit more or\nless variation, which words are more preserved and examine how languages could\nbe grouped based on linguistic distances within sets, several data analytics\ntechniques were involved. Those included density evaluation, hierarchical\nclustering, silhouette, mean, standard deviation and Bhattacharya coefficient\ncalculations. These techniques lead to the development of a workflow which was\nlater implemented by combining Unix shell scripts, a developed R package and\nSWI Prolog. This proved to be computationally efficient and permitted the fast\nexploration of large language sets and their analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00037v1",
            "date": "2020-01-31 19:48:58+00:00",
            "title": "Unsupervised Bilingual Lexicon Induction Across Writing Systems",
            "authors": [
                "Parker Riley",
                "Daniel Gildea"
            ],
            "abstract": "Recent embedding-based methods in unsupervised bilingual lexicon induction\nhave shown good results, but generally have not leveraged orthographic\n(spelling) information, which can be helpful for pairs of related languages.\nThis work augments a state-of-the-art method with orthographic features, and\nextends prior work in this space by proposing methods that can learn and\nutilize orthographic correspondences even between languages with different\nscripts. We demonstrate this by experimenting on three language pairs with\ndifferent scripts and varying degrees of lexical similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00163v1",
            "date": "2020-02-01 07:50:43+00:00",
            "title": "Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog",
            "authors": [
                "Zekang Li",
                "Zongjia Li",
                "Jinchao Zhang",
                "Yang Feng",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Audio-Visual Scene-Aware Dialog (AVSD) is a task to generate responses when\nchatting about a given video, which is organized as a track of the 8th Dialog\nSystem Technology Challenge (DSTC8). To solve the task, we propose a universal\nmultimodal transformer and introduce the multi-task learning method to learn\njoint representations among different modalities as well as generate\ninformative and fluent responses. Our method extends the natural language\ngeneration pre-trained model to multimodal dialogue generation task. Our system\nachieves the best performance in both objective and subjective evaluations in\nthe challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00171v1",
            "date": "2020-02-01 08:49:17+00:00",
            "title": "Novel Language Resources for Hindi: An Aesthetics Text Corpus and a Comprehensive Stop Lemma List",
            "authors": [
                "Gayatri Venugopal-Wairagade",
                "Jatinderkumar R. Saini",
                "Dhanya Pramod"
            ],
            "abstract": "This paper is an effort to complement the contributions made by researchers\nworking toward the inclusion of non-English languages in natural language\nprocessing studies. Two novel Hindi language resources have been created and\nreleased for public consumption. The first resource is a corpus consisting of\nnearly thousand pre-processed fictional and nonfictional texts spanning over\nhundred years. The second resource is an exhaustive list of stop lemmas created\nfrom 12 corpora across multiple domains, consisting of over 13 million words,\nfrom which more than 200,000 lemmas were generated, and 11 publicly available\nstop word lists comprising over 1000 words, from which nearly 400 unique lemmas\nwere generated. This research lays emphasis on the use of stop lemmas instead\nof stop words owing to the presence of various, but not all morphological forms\nof a word in stop word lists, as opposed to the presence of only the root form\nof the word, from which variations could be derived if required. It was also\nobserved that stop lemmas were more consistent across multiple sources as\ncompared to stop words. In order to generate a stop lemma list, the parts of\nspeech of the lemmas were investigated but rejected as it was found that there\nwas no significant correlation between the rank of a word in the frequency list\nand its part of speech. The stop lemma list was assessed using a comparative\nmethod. A formal evaluation method is suggested as future work arising from\nthis study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00175v1",
            "date": "2020-02-01 09:26:07+00:00",
            "title": "UIT-ViIC: A Dataset for the First Evaluation on Vietnamese Image Captioning",
            "authors": [
                "Quan Hoang Lam",
                "Quang Duy Le",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Image Captioning, the task of automatic generation of image captions, has\nattracted attentions from researchers in many fields of computer science, being\ncomputer vision, natural language processing and machine learning in recent\nyears. This paper contributes to research on Image Captioning task in terms of\nextending dataset to a different language - Vietnamese. So far, there is no\nexisted Image Captioning dataset for Vietnamese language, so this is the\nforemost fundamental step for developing Vietnamese Image Captioning. In this\nscope, we first build a dataset which contains manually written captions for\nimages from Microsoft COCO dataset relating to sports played with balls, we\ncalled this dataset UIT-ViIC. UIT-ViIC consists of 19,250 Vietnamese captions\nfor 3,850 images. Following that, we evaluate our dataset on deep neural\nnetwork models and do comparisons with English dataset and two Vietnamese\ndatasets built by different methods. UIT-ViIC is published on our lab website\nfor research purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00181v1",
            "date": "2020-02-01 10:00:06+00:00",
            "title": "Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking",
            "authors": [
                "Yu-Ping Ruan",
                "Zhen-Hua Ling",
                "Jia-Chen Gu",
                "Quan Liu"
            ],
            "abstract": "We present our work on Track 4 in the Dialogue System Technology Challenges 8\n(DSTC8). The DSTC8-Track 4 aims to perform dialogue state tracking (DST) under\nthe zero-shot settings, in which the model needs to generalize on unseen\nservice APIs given a schema definition of these target APIs. Serving as the\ncore for many virtual assistants such as Siri, Alexa, and Google Assistant, the\nDST keeps track of the user's goal and what happened in the dialogue history,\nmainly including intent prediction, slot filling, and user state tracking,\nwhich tests models' ability of natural language understanding. Recently, the\npretrained language models have achieved state-of-the-art results and shown\nimpressive generalization ability on various NLP tasks, which provide a\npromising way to perform zero-shot learning for language understanding. Based\non this, we propose a schema-guided paradigm for zero-shot dialogue state\ntracking (SGP-DST) by fine-tuning BERT, one of the most popular pretrained\nlanguage models. The SGP-DST system contains four modules for intent\nprediction, slot prediction, slot transfer prediction, and user state\nsummarizing respectively. According to the official evaluation results, our\nSGP-DST (team12) ranked 3rd on the joint goal accuracy (primary evaluation\nmetric for ranking submissions) and 1st on the requsted slots F1 among 25\nparticipant teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00293v2",
            "date": "2020-02-02 00:22:55+00:00",
            "title": "Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension",
            "authors": [
                "Max Bartolo",
                "Alastair Roberts",
                "Johannes Welbl",
                "Sebastian Riedel",
                "Pontus Stenetorp"
            ],
            "abstract": "Innovations in annotation methodology have been a catalyst for Reading\nComprehension (RC) datasets and models. One recent trend to challenge current\nRC models is to involve a model in the annotation process: humans create\nquestions adversarially, such that the model fails to answer them correctly. In\nthis work we investigate this annotation methodology and apply it in three\ndifferent settings, collecting a total of 36,000 samples with progressively\nstronger models in the annotation loop. This allows us to explore questions\nsuch as the reproducibility of the adversarial effect, transfer from data\ncollected with varying model-in-the-loop strengths, and generalisation to data\ncollected without a model. We find that training on adversarially collected\nsamples leads to strong generalisation to non-adversarially collected datasets,\nyet with progressive performance deterioration with increasingly stronger\nmodels-in-the-loop. Furthermore, we find that stronger models can still learn\nfrom datasets collected with substantially weaker models-in-the-loop. When\ntrained on data collected with a BiDAF model in the loop, RoBERTa achieves\n39.9F1 on questions that it cannot answer when trained on SQuAD - only\nmarginally lower than when trained on data collected using RoBERTa itself\n(41.0F1).",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  Volume 8, 2020 p.662-678",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00317v2",
            "date": "2020-02-02 03:54:47+00:00",
            "title": "Citation Text Generation",
            "authors": [
                "Kelvin Luu",
                "Rik Koncel-Kedziorski",
                "Kyle Lo",
                "Isabel Cachola",
                "Noah A. Smith"
            ],
            "abstract": "We introduce the task of citation text generation: given a pair of scientific\ndocuments, explain their relationship in natural language text in the manner of\na citation from one text to the other. This task encourages systems to learn\nrich relationships between scientific texts and to express them concretely in\nnatural language. Models for citation text generation will require robust\ndocument understanding including the capacity to quickly adapt to new\nvocabulary and to reason about document content. We believe this challenging\ndirection of research will benefit high-impact applications such as automatic\nliterature review or scientific writing assistance systems. In this paper we\nestablish the task of citation text generation with a standard evaluation\ncorpus and develop several strong baseline models. We provide extensive\nautomatic and human evaluations to illustrate the successes and shortcomings of\ncurrent text generation techniques for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00481v1",
            "date": "2020-02-02 20:08:34+00:00",
            "title": "Assessment of Amazon Comprehend Medical: Medication Information Extraction",
            "authors": [
                "Benedict Guzman",
                "MS",
                "Isabel Metzger",
                "MS",
                "Yindalon Aphinyanaphongs",
                "M. D.",
                "Ph. D.",
                "Himanshu Grover",
                "Ph. D"
            ],
            "abstract": "In November 27, 2018, Amazon Web Services (AWS) released Amazon Comprehend\nMedical (ACM), a deep learning based system that automatically extracts\nclinical concepts (which include anatomy, medical conditions, protected health\ninformation (PH)I, test names, treatment names, and medical procedures, and\nmedications) from clinical text notes. Uptake and trust in any new data product\nrelies on independent validation across benchmark datasets and tools to\nestablish and confirm expected quality of results. This work focuses on the\nmedication extraction task, and particularly, ACM was evaluated using the\nofficial test sets from the 2009 i2b2 Medication Extraction Challenge and 2018\nn2c2 Track 2: Adverse Drug Events and Medication Extraction in EHRs. Overall,\nACM achieved F-scores of 0.768 and 0.828. These scores ranked the lowest when\ncompared to the three best systems in the respective challenges. To further\nestablish the generalizability of its medication extraction performance, a set\nof random internal clinical text notes from NYU Langone Medical Center were\nalso included in this work. And in this corpus, ACM garnered an F-score of\n0.753.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00730v1",
            "date": "2020-01-25 16:40:43+00:00",
            "title": "Reducing Noise from Competing Neighbours: Word Retrieval with Lateral Inhibition in Multilink",
            "authors": [
                "Aaron van Geffen"
            ],
            "abstract": "Multilink is a computational model for word retrieval in monolingual and\nmultilingual individuals under different task circumstances (Dijkstra et al.,\n2018). In the present study, we added lateral inhibition to Multilink's lexical\nnetwork. Parameters were fit on the basis of reaction times from the English,\nBritish, and Dutch Lexicon Projects. We found a maximum correlation of 0.643\n(N=1,205) on these data sets as a whole. Furthermore, the simulations\nthemselves became faster as a result of adding lateral inhibition. We tested\nthe fitted model to stimuli from a neighbourhood study (Mulder et al., 2018).\nLateral inhibition was found to improve Multilink's correlations for this\nstudy, yielding an overall correlation of 0.67. Next, we explored the role of\nlateral inhibition as part of the model's task/decision system by running\nsimulations on data from two studies concerning interlingual homographs\n(Vanlangendonck et al., in press; Goertz, 2018). We found that, while lateral\ninhibition plays a substantial part in the word selection process, this alone\nis not enough to result in a correct response selection. To solve this problem,\nwe added a new task component to Multilink, especially designed to account for\nthe translation process of interlingual homographs, cognates, and\nlanguage-specific control words. The subsequent simulation results showed\npatterns remarkably similar to those in the Goertz study. The isomorphicity of\nthe simulated data to the empirical data was further attested by an overall\ncorrelation of 0.538 (N=254) between reaction times and simulated model cycle\ntimes, as well as a condition pattern correlation of 0.853 (N=8). We conclude\nthat Multilink yields an excellent fit to empirical data, particularly when a\ntask-specific setting of the inhibition parameters is allowed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00735v1",
            "date": "2020-01-30 21:51:58+00:00",
            "title": "Self-attention-based BiGRU and capsule network for named entity recognition",
            "authors": [
                "Jianfeng Deng",
                "Lianglun Cheng",
                "Zhuowei Wang"
            ],
            "abstract": "Named entity recognition(NER) is one of the tasks of natural language\nprocessing(NLP). In view of the problem that the traditional character\nrepresentation ability is weak and the neural network method is unable to\ncapture the important sequence information. An self-attention-based\nbidirectional gated recurrent unit(BiGRU) and capsule network(CapsNet) for NER\nis proposed. This model generates character vectors through bidirectional\nencoder representation of transformers(BERT) pre-trained model. BiGRU is used\nto capture sequence context features, and self-attention mechanism is proposed\nto give different focus on the information captured by hidden layer of BiGRU.\nFinally, we propose to use CapsNet for entity recognition. We evaluated the\nrecognition performance of the model on two datasets. Experimental results show\nthat the model has better performance without relying on external dictionary\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00738v1",
            "date": "2020-01-30 06:54:39+00:00",
            "title": "An Efficient Architecture for Predicting the Case of Characters using Sequence Models",
            "authors": [
                "Gopi Ramena",
                "Divija Nagaraju",
                "Sukumar Moharana",
                "Debi Prasanna Mohanty",
                "Naresh Purre"
            ],
            "abstract": "The dearth of clean textual data often acts as a bottleneck in several\nnatural language processing applications. The data available often lacks proper\ncase (uppercase or lowercase) information. This often comes up when text is\nobtained from social media, messaging applications and other online platforms.\nThis paper attempts to solve this problem by restoring the correct case of\ncharacters, commonly known as Truecasing. Doing so improves the accuracy of\nseveral processing tasks further down in the NLP pipeline. Our proposed\narchitecture uses a combination of convolutional neural networks (CNN),\nbi-directional long short-term memory networks (LSTM) and conditional random\nfields (CRF), which work at a character level without any explicit feature\nengineering. In this study we compare our approach to previous statistical and\ndeep learning based approaches. Our method shows an increment of 0.83 in F1\nscore over the current state of the art. Since truecasing acts as a\npreprocessing step in several applications, every increment in the F1 score\nleads to a significant improvement in the language processing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00754v1",
            "date": "2020-01-31 11:54:46+00:00",
            "title": "Benchmarking Popular Classification Models' Robustness to Random and Targeted Corruptions",
            "authors": [
                "Utkarsh Desai",
                "Srikanth Tamilselvam",
                "Jassimran Kaur",
                "Senthil Mani",
                "Shreya Khare"
            ],
            "abstract": "Text classification models, especially neural networks based models, have\nreached very high accuracy on many popular benchmark datasets. Yet, such models\nwhen deployed in real world applications, tend to perform badly. The primary\nreason is that these models are not tested against sufficient real world\nnatural data. Based on the application users, the vocabulary and the style of\nthe model's input may greatly vary. This emphasizes the need for a model\nagnostic test dataset, which consists of various corruptions that are natural\nto appear in the wild. Models trained and tested on such benchmark datasets,\nwill be more robust against real world data. However, such data sets are not\neasily available. In this work, we address this problem, by extending the\nbenchmark datasets along naturally occurring corruptions such as Spelling\nErrors, Text Noise and Synonyms and making them publicly available. Through\nextensive experiments, we compare random and targeted corruption strategies\nusing Local Interpretable Model-Agnostic Explanations(LIME). We report the\nvulnerabilities in two popular text classification models along these\ncorruptions and also find that targeted corruptions can expose vulnerabilities\nof a model better than random choices in most cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00757v1",
            "date": "2020-01-31 09:37:00+00:00",
            "title": "Similarit\u00e0 per la ricerca del dominio di una frase",
            "authors": [
                "Massimiliano Morrelli",
                "Giacomo Pansini",
                "Massimiliano Polito",
                "Arturo Vitale"
            ],
            "abstract": "English. This document aims to study the best algorithms to verify the\nbelonging of a specific document to a related domain by comparing different\nmethods for calculating the distance between two vectors. This study has been\nmade possible with the help of the structures made available by the Apache\nSpark framework. Starting from the study illustrated in the publication \"New\nfrontier of textual classification: Big data and distributed calculus\" by\nMassimiliano Morrelli et al., We wanted to carry out a study on the possible\nimplementation of a solution capable of calculating the Similarity of a\nsentence using the distributed environment.\n  Italiano. Il presente documento persegue l'obiettivo di studiare gli\nalgoritmi migliori per verificare l'appartenenza di un determinato documento a\nun relativo dominio tramite un confronto di diversi metodi per il calcolo della\ndistanza fra due vettori. Tale studio \\`e stato condotto con l'ausilio delle\nstrutture messe a disposizione dal framework Apache Spark. Partendo dallo\nstudio illustrato nella pubblicazione \"Nuova frontiera della classificazione\ntestuale: Big data e calcolo distribuito\" di Massimiliano Morrelli et al., si\n\\`e voluto realizzare uno studio sulla possibile implementazione di una\nsoluzione in grado di calcolare la Similarit\\`a di una frase sfruttando\nl'ambiente distribuito.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00759v2",
            "date": "2020-01-31 09:28:57+00:00",
            "title": "Comparison Between Traditional Machine Learning Models And Neural Network Models For Vietnamese Hate Speech Detection",
            "authors": [
                "Son T. Luu",
                "Hung P. Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Hate-speech detection on social network language has become one of the main\nresearching fields recently due to the spreading of social networks like\nFacebook and Twitter. In Vietnam, the threat of offensive and harassment cause\nbad impacts for online user. The VLSP - Shared task about Hate Speech Detection\non social networks showed many proposed approaches for detecting whatever\ncomment is clean or not. However, this problem still needs further researching.\nConsequently, we compare traditional machine learning and deep learning on a\nlarge dataset about the user's comments on social network in Vietnamese and\nfind out what is the advantage and disadvantage of each model by comparing\ntheir accuracy on F1-score, then we pick two models in which has highest\naccuracy in traditional machine learning models and deep neural models\nrespectively. Next, we compare these two models capable of predicting the right\nlabel by referencing their confusion matrices and considering the advantages\nand disadvantages of each model. Finally, from the comparison result, we\npropose our ensemble method that concentrates the abilities of traditional\nmethods and deep learning methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00835v1",
            "date": "2020-02-03 15:47:19+00:00",
            "title": "Learning Contextualized Document Representations for Healthcare Answer Retrieval",
            "authors": [
                "Sebastian Arnold",
                "Betty van Aken",
                "Paul Grundmann",
                "Felix A. Gers",
                "Alexander L\u00f6ser"
            ],
            "abstract": "We present Contextual Discourse Vectors (CDV), a distributed document\nrepresentation for efficient answer retrieval from long healthcare documents.\nOur approach is based on structured query tuples of entities and aspects from\nfree text and medical taxonomies. Our model leverages a dual encoder\narchitecture with hierarchical LSTM layers and multi-task training to encode\nthe position of clinical entities and aspects alongside the document discourse.\nWe use our continuous representations to resolve queries with short latency\nusing approximate nearest neighbor search on sentence level. We apply the CDV\nmodel for retrieving coherent answer passages from nine English public health\nresources from the Web, addressing both patients and medical professionals.\nBecause there is no end-to-end training data available for all application\nscenarios, we train our model with self-supervised data from Wikipedia. We show\nthat our generalized model significantly outperforms several state-of-the-art\nbaselines for healthcare passage ranking and is able to adapt to heterogeneous\ndomains without additional fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01127v2",
            "date": "2020-02-04 04:53:45+00:00",
            "title": "Variational Template Machine for Data-to-Text Generation",
            "authors": [
                "Rong Ye",
                "Wenxian Shi",
                "Hao Zhou",
                "Zhongyu Wei",
                "Lei Li"
            ],
            "abstract": "How to generate descriptions from structured data organized in tables?\nExisting approaches using neural encoder-decoder models often suffer from\nlacking diversity. We claim that an open set of templates is crucial for\nenriching the phrase constructions and realizing varied generations. Learning\nsuch templates is prohibitive since it often requires a large paired <table,\ndescription> corpus, which is seldom available. This paper explores the problem\nof automatically learning reusable \"templates\" from paired and non-paired data.\nWe propose the variational template machine (VTM), a novel method to generate\ntext descriptions from data tables. Our contributions include: a) we carefully\ndevise a specific model architecture and losses to explicitly disentangle text\ntemplate and semantic content information, in the latent spaces, and b)we\nutilize both small parallel data and large raw text without aligned tables to\nenrich the template learning. Experiments on datasets from a variety of\ndifferent domains show that VTM is able to generate more diversely while\nkeeping a good fluency and quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01145v2",
            "date": "2020-02-04 06:26:37+00:00",
            "title": "Syntactically Look-Ahead Attention Network for Sentence Compression",
            "authors": [
                "Hidetaka Kamigaito",
                "Manabu Okumura"
            ],
            "abstract": "Sentence compression is the task of compressing a long sentence into a short\none by deleting redundant words. In sequence-to-sequence (Seq2Seq) based\nmodels, the decoder unidirectionally decides to retain or delete words. Thus,\nit cannot usually explicitly capture the relationships between decoded words\nand unseen words that will be decoded in the future time steps. Therefore, to\navoid generating ungrammatical sentences, the decoder sometimes drops important\nwords in compressing sentences. To solve this problem, we propose a novel\nSeq2Seq model, syntactically look-ahead attention network (SLAHAN), that can\ngenerate informative summaries by explicitly tracking both dependency parent\nand child words during decoding and capturing important words that will be\ndecoded in the future. The results of the automatic evaluation on the Google\nsentence compression dataset showed that SLAHAN achieved the best\nkept-token-based-F1, ROUGE-1, ROUGE-2 and ROUGE-L scores of 85.5, 79.3, 71.3\nand 79.1, respectively. SLAHAN also improved the summarization performance on\nlonger sentences. Furthermore, in the human evaluation, SLAHAN improved\ninformativeness without losing readability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01196v2",
            "date": "2020-02-04 09:49:36+00:00",
            "title": "Dynamic Knowledge Routing Network For Target-Guided Open-Domain Conversation",
            "authors": [
                "Jinghui Qin",
                "Zheng Ye",
                "Jianheng Tang",
                "Xiaodan Liang"
            ],
            "abstract": "Target-guided open-domain conversation aims to proactively and naturally\nguide a dialogue agent or human to achieve specific goals, topics or keywords\nduring open-ended conversations. Existing methods mainly rely on single-turn\ndatadriven learning and simple target-guided strategy without considering\nsemantic or factual knowledge relations among candidate topics/keywords. This\nresults in poor transition smoothness and low success rate. In this work, we\nadopt a structured approach that controls the intended content of system\nresponses by introducing coarse-grained keywords, attains smooth conversation\ntransition through turn-level supervised learning and knowledge relations\nbetween candidate keywords, and drives an conversation towards an specified\ntarget with discourse-level guiding strategy. Specially, we propose a novel\ndynamic knowledge routing network (DKRN) which considers semantic knowledge\nrelations among candidate keywords for accurate next topic prediction of next\ndiscourse. With the help of more accurate keyword prediction, our\nkeyword-augmented response retrieval module can achieve better retrieval\nperformance and more meaningful conversations. Besides, we also propose a novel\ndual discourse-level target-guided strategy to guide conversations to reach\ntheir goals smoothly with higher success rate. Furthermore, to push the\nresearch boundary of target-guided open-domain conversation to match real-world\nscenarios better, we introduce a new large-scale Chinese target-guided\nopen-domain conversation dataset (more than 900K conversations) crawled from\nSina Weibo. Quantitative and human evaluations show our method can produce\nmeaningful and effective target-guided conversations, significantly improving\nover other state-of-the-art methods by more than 20% in success rate and more\nthan 0.6 in average smoothness score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01320v2",
            "date": "2020-02-04 14:35:28+00:00",
            "title": "CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus",
            "authors": [
                "Changhan Wang",
                "Juan Pino",
                "Anne Wu",
                "Jiatao Gu"
            ],
            "abstract": "Spoken language translation has recently witnessed a resurgence in\npopularity, thanks to the development of end-to-end models and the creation of\nnew corpora, such as Augmented LibriSpeech and MuST-C. Existing datasets\ninvolve language pairs with English as a source language, involve very specific\ndomains or are low resource. We introduce CoVoST, a multilingual speech-to-text\ntranslation corpus from 11 languages into English, diversified with over 11,000\nspeakers and over 60 accents. We describe the dataset creation methodology and\nprovide empirical evidence of the quality of the data. We also provide initial\nbenchmarks, including, to our knowledge, the first end-to-end many-to-one\nmultilingual models for spoken language translation. CoVoST is released under\nCC0 license and free to use. We also provide additional evaluation data derived\nfrom Tatoeba under CC licenses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01359v1",
            "date": "2020-02-02 05:59:27+00:00",
            "title": "Schema-Guided Dialogue State Tracking Task at DSTC8",
            "authors": [
                "Abhinav Rastogi",
                "Xiaoxue Zang",
                "Srinivas Sunkara",
                "Raghav Gupta",
                "Pranav Khaitan"
            ],
            "abstract": "This paper gives an overview of the Schema-Guided Dialogue State Tracking\ntask of the 8th Dialogue System Technology Challenge. The goal of this task is\nto develop dialogue state tracking models suitable for large-scale virtual\nassistants, with a focus on data-efficient joint modeling across domains and\nzero-shot generalization to new APIs. This task provided a new dataset\nconsisting of over 16000 dialogues in the training set spanning 16 domains to\nhighlight these challenges, and a baseline model capable of zero-shot\ngeneralization to new APIs. Twenty-five teams participated, developing a range\nof neural network models, exceeding the performance of the baseline model by a\nvery high margin. The submissions incorporated a variety of pre-trained\nencoders and data augmentation techniques. This paper describes the task\ndefinition, dataset and evaluation methodology. We also summarize the approach\nand results of the submitted systems to highlight the overall trends in the\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01415v3",
            "date": "2020-02-04 17:16:36+00:00",
            "title": "Plague Dot Text: Text mining and annotation of outbreak reports of the Third Plague Pandemic (1894-1952)",
            "authors": [
                "Arlene Casey",
                "Mike Bennett",
                "Richard Tobin",
                "Claire Grover",
                "Iona Walker",
                "Lukas Engelmann",
                "Beatrice Alex"
            ],
            "abstract": "The design of models that govern diseases in population is commonly built on\ninformation and data gathered from past outbreaks. However, epidemic outbreaks\nare never captured in statistical data alone but are communicated by\nnarratives, supported by empirical observations. Outbreak reports discuss\ncorrelations between populations, locations and the disease to infer insights\ninto causes, vectors and potential interventions. The problem with these\nnarratives is usually the lack of consistent structure or strong conventions,\nwhich prohibit their formal analysis in larger corpora. Our interdisciplinary\nresearch investigates more than 100 reports from the third plague pandemic\n(1894-1952) evaluating ways of building a corpus to extract and structure this\nnarrative information through text mining and manual annotation. In this paper\nwe discuss the progress of our ongoing exploratory project, how we enhance\noptical character recognition (OCR) methods to improve text capture, our\napproach to structure the narratives and identify relevant entities in the\nreports. The structured corpus is made available via Solr enabling search and\nanalysis across the whole collection for future research dedicated, for\nexample, to the identification of concepts. We show preliminary visualisations\nof the characteristics of causation and differences with respect to gender as a\nresult of syntactic-category-dependent corpus statistics. Our goal is to\ndevelop structured accounts of some of the most significant concepts that were\nused to understand the epidemiology of the third plague pandemic around the\nglobe. The corpus enables researchers to analyse the reports collectively\nallowing for deep insights into the global epidemiological consideration of\nplague in the early twentieth century.",
            "journal ref": "Journal of Data Mining & Digital Humanities, HistoInformatics,\n  HistoInformatics (January 20, 2021) jdmdh:7105",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01454v1",
            "date": "2020-02-04 18:31:25+00:00",
            "title": "From Topic Networks to Distributed Cognitive Maps: Zipfian Topic Universes in the Area of Volunteered Geographic Information",
            "authors": [
                "Alexander Mehler",
                "R\u00fcdiger Gleim",
                "Regina Gaitsch",
                "Wahed Hemati",
                "Tolga Uslu"
            ],
            "abstract": "Are nearby places (e.g. cities) described by related words? In this article\nwe transfer this research question in the field of lexical encoding of\ngeographic information onto the level of intertextuality. To this end, we\nexplore Volunteered Geographic Information (VGI) to model texts addressing\nplaces at the level of cities or regions with the help of so-called topic\nnetworks. This is done to examine how language encodes and networks geographic\ninformation on the aboutness level of texts. Our hypothesis is that the\nnetworked thematizations of places are similar - regardless of their distances\nand the underlying communities of authors. To investigate this we introduce\nMultiplex Topic Networks (MTN), which we automatically derive from Linguistic\nMultilayer Networks (LMN) as a novel model, especially of thematic networking\nin text corpora. Our study shows a Zipfian organization of the thematic\nuniverse in which geographical places (especially cities) are located in online\ncommunication. We interpret this finding in the context of cognitive maps, a\nnotion which we extend by so-called thematic maps. According to our\ninterpretation of this finding, the organization of thematic maps as part of\ncognitive maps results from a tendency of authors to generate shareable content\nthat ensures the continued existence of the underlying media. We test our\nhypothesis by example of special wikis and extracts of Wikipedia. In this way\nwe come to the conclusion: Places, whether close to each other or not, are\nlocated in neighboring places that span similar subnetworks in the topic\nuniverse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01824v1",
            "date": "2020-02-05 15:12:03+00:00",
            "title": "Discontinuous Constituent Parsing with Pointer Networks",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "One of the most complex syntactic representations used in computational\nlinguistics and NLP are discontinuous constituent trees, crucial for\nrepresenting all grammatical phenomena of languages such as German. Recent\nadvances in dependency parsing have shown that Pointer Networks excel in\nefficiently parsing syntactic relations between words in a sentence. This kind\nof sequence-to-sequence models achieve outstanding accuracies in building\nnon-projective dependency trees, but its potential has not been proved yet on a\nmore difficult task. We propose a novel neural network architecture that, by\nmeans of Pointer Networks, is able to generate the most accurate discontinuous\nconstituent representations to date, even without the need of Part-of-Speech\ntagging information. To do so, we internally model discontinuous constituent\nstructures as augmented non-projective dependency structures. The proposed\napproach achieves state-of-the-art results on the two widely-used NEGRA and\nTIGER benchmarks, outperforming previous work by a wide margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02154v1",
            "date": "2020-02-06 08:36:38+00:00",
            "title": "Related Tasks can Share! A Multi-task Framework for Affective language",
            "authors": [
                "Kumar Shikhar Deep",
                "Md Shad Akhtar",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Expressing the polarity of sentiment as 'positive' and 'negative' usually\nhave limited scope compared with the intensity/degree of polarity. These two\ntasks (i.e. sentiment classification and sentiment intensity prediction) are\nclosely related and may offer assistance to each other during the learning\nprocess. In this paper, we propose to leverage the relatedness of multiple\ntasks in a multi-task learning framework. Our multi-task model is based on\nconvolutional-Gated Recurrent Unit (GRU) framework, which is further assisted\nby a diverse hand-crafted feature set. Evaluation and analysis suggest that\njoint-learning of the related tasks in a multi-task framework can outperform\neach of the individual tasks in the single-task frameworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02224v1",
            "date": "2020-02-06 12:35:14+00:00",
            "title": "Citation Data of Czech Apex Courts",
            "authors": [
                "Jakub Hara\u0161ta",
                "Tereza Novotn\u00e1",
                "Jarom\u00edr \u0160avelka"
            ],
            "abstract": "In this paper, we introduce the citation data of the Czech apex courts\n(Supreme Court, Supreme Administrative Court and Constitutional Court). This\ndataset was automatically extracted from the corpus of texts of Czech court\ndecisions - CzCDC 1.0. We obtained the citation data by building the natural\nlanguage processing pipeline for extraction of the court decision identifiers.\nThe pipeline included the (i) document segmentation model and the (ii)\nreference recognition model. Furthermore, the dataset was manually processed to\nachieve high-quality citation data as a base for subsequent qualitative and\nquantitative analyses. The dataset will be made available to the general\npublic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02353v1",
            "date": "2020-02-06 16:57:27+00:00",
            "title": "Conversational Structure Aware and Context Sensitive Topic Model for Online Discussions",
            "authors": [
                "Yingcheng Sun",
                "Kenneth Loparo",
                "Richard Kolacinski"
            ],
            "abstract": "Millions of online discussions are generated everyday on social media\nplatforms. Topic modelling is an efficient way of better understanding large\ntext datasets at scale. Conventional topic models have had limited success in\nonline discussions, and to overcome their limitations, we use the discussion\nthread tree structure and propose a \"popularity\" metric to quantify the number\nof replies to a comment to extend the frequency of word occurrences, and the\n\"transitivity\" concept to characterize topic dependency among nodes in a nested\ndiscussion thread. We build a Conversational Structure Aware Topic Model\n(CSATM) based on popularity and transitivity to infer topics and their\nassignments to comments. Experiments on real forum datasets are used to\ndemonstrate improved performance for topic extraction with six different\nmeasurements of coherence and impressive accuracy for topic assignments.",
            "journal ref": "2020 IEEE 14th International Conference on Semantic Computing\n  (ICSC)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02649v1",
            "date": "2020-02-07 07:19:15+00:00",
            "title": "Multimodal Matching Transformer for Live Commenting",
            "authors": [
                "Chaoqun Duan",
                "Lei Cui",
                "Shuming Ma",
                "Furu Wei",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Automatic live commenting aims to provide real-time comments on videos for\nviewers. It encourages users engagement on online video sites, and is also a\ngood benchmark for video-to-text generation. Recent work on this task adopts\nencoder-decoder models to generate comments. However, these methods do not\nmodel the interaction between videos and comments explicitly, so they tend to\ngenerate popular comments that are often irrelevant to the videos. In this\nwork, we aim to improve the relevance between live comments and videos by\nmodeling the cross-modal interactions among different modalities. To this end,\nwe propose a multimodal matching transformer to capture the relationships among\ncomments, vision, and audio. The proposed model is based on the transformer\nframework and can iteratively learn the attention-aware representations for\neach modality. We evaluate the model on a publicly available live commenting\ndataset. Experiments show that the multimodal matching transformer model\noutperforms the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02734v1",
            "date": "2020-02-07 12:26:41+00:00",
            "title": "Incorporating Visual Semantics into Sentence Representations within a Grounded Space",
            "authors": [
                "Patrick Bordes",
                "Eloi Zablocki",
                "Laure Soulier",
                "Benjamin Piwowarski",
                "Patrick Gallinari"
            ],
            "abstract": "Language grounding is an active field aiming at enriching textual\nrepresentations with visual information. Generally, textual and visual elements\nare embedded in the same representation space, which implicitly assumes a\none-to-one correspondence between modalities. This hypothesis does not hold\nwhen representing words, and becomes problematic when used to learn sentence\nrepresentations --- the focus of this paper --- as a visual scene can be\ndescribed by a wide variety of sentences. To overcome this limitation, we\npropose to transfer visual information to textual representations by learning\nan intermediate representation space: the grounded space. We further propose\ntwo new complementary objectives ensuring that (1) sentences associated with\nthe same visual content are close in the grounded space and (2) similarities\nbetween related elements are preserved across modalities. We show that this\nmodel outperforms the previous state-of-the-art on classification and semantic\nrelatedness tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02955v4",
            "date": "2020-02-07 18:50:21+00:00",
            "title": "A Multilingual View of Unsupervised Machine Translation",
            "authors": [
                "Xavier Garcia",
                "Pierre Foret",
                "Thibault Sellam",
                "Ankur P. Parikh"
            ],
            "abstract": "We present a probabilistic framework for multilingual neural machine\ntranslation that encompasses supervised and unsupervised setups, focusing on\nunsupervised translation. In addition to studying the vanilla case where there\nis only monolingual data available, we propose a novel setup where one language\nin the (source, target) pair is not associated with any parallel data, but\nthere may exist auxiliary parallel data that contains the other. This auxiliary\ndata can naturally be utilized in our probabilistic framework via a novel\ncross-translation loss term. Empirically, we show that our approach results in\nhigher BLEU scores over state-of-the-art unsupervised models on the WMT'14\nEnglish-French, WMT'16 English-German, and WMT'16 English-Romanian datasets in\nmost directions. In particular, we obtain a +1.65 BLEU advantage over the\nbest-performing unsupervised model in the Romanian-English direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03067v3",
            "date": "2020-02-08 02:14:28+00:00",
            "title": "Description Based Text Classification with Reinforcement Learning",
            "authors": [
                "Duo Chai",
                "Wei Wu",
                "Qinghong Han",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "The task of text classification is usually divided into two stages: {\\it text\nfeature extraction} and {\\it classification}. In this standard formalization\ncategories are merely represented as indexes in the label vocabulary, and the\nmodel lacks for explicit instructions on what to classify. Inspired by the\ncurrent trend of formalizing NLP problems as question answering tasks, we\npropose a new framework for text classification, in which each category label\nis associated with a category description. Descriptions are generated by\nhand-crafted templates or using abstractive/extractive models from\nreinforcement learning. The concatenation of the description and the text is\nfed to the classifier to decide whether or not the current label should be\nassigned to the text. The proposed strategy forces the model to attend to the\nmost salient texts with respect to the label, which can be regarded as a hard\nversion of attention, leading to better performances. We observe significant\nperformance boosts over strong baselines on a wide range of text classification\ntasks including single-label classification, multi-label classification and\nmulti-aspect sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03084v1",
            "date": "2020-02-08 04:11:03+00:00",
            "title": "LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention",
            "authors": [
                "Xiaoya Li",
                "Yuxian Meng",
                "Arianna Yuan",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Non-autoregressive translation (NAT) models generate multiple tokens in one\nforward pass and is highly efficient at inference stage compared with\nautoregressive translation (AT) methods. However, NAT models often suffer from\nthe multimodality problem, i.e., generating duplicated tokens or missing\ntokens. In this paper, we propose two novel methods to address this issue, the\nLook-Around (LA) strategy and the Vocabulary Attention (VA) mechanism. The\nLook-Around strategy predicts the neighbor tokens in order to predict the\ncurrent token, and the Vocabulary Attention models long-term token dependencies\ninside the decoder by attending the whole vocabulary for each position to\nacquire knowledge of which token is about to generate. %We also propose a\ndynamic bidirectional decoding approach to accelerate the inference process of\nthe LAVA model while preserving the high-quality of the generated output. Our\nproposed model uses significantly less time during inference compared with\nautoregressive models and most other NAT models. Our experiments on four\nbenchmarks (WMT14 En$\\rightarrow$De, WMT14 De$\\rightarrow$En, WMT16\nRo$\\rightarrow$En and IWSLT14 De$\\rightarrow$En) show that the proposed model\nachieves competitive performance compared with the state-of-the-art\nnon-autoregressive and autoregressive models while significantly reducing the\ntime cost in inference phase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03140v1",
            "date": "2020-02-08 11:06:27+00:00",
            "title": "HHH: An Online Medical Chatbot System based on Knowledge Graph and Hierarchical Bi-Directional Attention",
            "authors": [
                "Qiming Bao",
                "Lin Ni",
                "Jiamou Liu"
            ],
            "abstract": "This paper proposes a chatbot framework that adopts a hybrid model which\nconsists of a knowledge graph and a text similarity model. Based on this\nchatbot framework, we build HHH, an online question-and-answer (QA) Healthcare\nHelper system for answering complex medical questions. HHH maintains a\nknowledge graph constructed from medical data collected from the Internet. HHH\nalso implements a novel text representation and similarity deep learning model,\nHierarchical BiLSTM Attention Model (HBAM), to find the most similar question\nfrom a large QA dataset. We compare HBAM with other state-of-the-art language\nmodels such as bidirectional encoder representation from transformers (BERT)\nand Manhattan LSTM Model (MaLSTM). We train and test the models with a subset\nof the Quora duplicate questions dataset in the medical area. The experimental\nresults show that our model is able to achieve a superior performance than\nthese existing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03149v3",
            "date": "2020-02-08 12:02:45+00:00",
            "title": "Mining Commonsense Facts from the Physical World",
            "authors": [
                "Yanyan Zou",
                "Wei Lu",
                "Xu Sun"
            ],
            "abstract": "Textual descriptions of the physical world implicitly mention commonsense\nfacts, while the commonsense knowledge bases explicitly represent such facts as\ntriples. Compared to dramatically increased text data, the coverage of existing\nknowledge bases is far away from completion. Most of the prior studies on\npopulating knowledge bases mainly focus on Freebase. To automatically complete\ncommonsense knowledge bases to improve their coverage is under-explored. In\nthis paper, we propose a new task of mining commonsense facts from the raw text\nthat describes the physical world. We build an effective new model that fuses\ninformation from both sequence text and existing knowledge base resource. Then\nwe create two large annotated datasets each with approximate 200k instances for\ncommonsense knowledge base completion. Empirical results demonstrate that our\nmodel significantly outperforms baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03350v2",
            "date": "2020-02-09 12:08:43+00:00",
            "title": "Short Text Classification via Knowledge powered Attention with Similarity Matrix based CNN",
            "authors": [
                "Mingchen Li",
                "Gabtone. Clinton",
                "Yijia Miao",
                "Feng Gao"
            ],
            "abstract": "Short text is becoming more and more popular on the web, such as Chat\nMessage, SMS and Product Reviews. Accurately classifying short text is an\nimportant and challenging task. A number of studies have difficulties in\naddressing this problem because of the word ambiguity and data sparsity. To\naddress this issue, we propose a knowledge powered attention with similarity\nmatrix based convolutional neural network (KASM) model, which can compute\ncomprehensive information by utilizing the knowledge and deep neural network.\nWe use knowledge graph (KG) to enrich the semantic representation of short\ntext, specially, the information of parent-entity is introduced in our model.\nMeanwhile, we consider the word interaction in the literal-level between short\ntext and the representation of label, and utilize similarity matrix based\nconvolutional neural network (CNN) to extract it. For the purpose of measuring\nthe importance of knowledge, we introduce the attention mechanisms to choose\nthe important information. Experimental results on five standard datasets show\nthat our model significantly outperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03536v1",
            "date": "2020-02-10 04:27:48+00:00",
            "title": "What Changed Your Mind: The Roles of Dynamic Topics and Discourse in Argumentation Process",
            "authors": [
                "Jichuan Zeng",
                "Jing Li",
                "Yulan He",
                "Cuiyun Gao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "In our world with full of uncertainty, debates and argumentation contribute\nto the progress of science and society. Despite of the increasing attention to\ncharacterize human arguments, most progress made so far focus on the debate\noutcome, largely ignoring the dynamic patterns in argumentation processes. This\npaper presents a study that automatically analyzes the key factors in argument\npersuasiveness, beyond simply predicting who will persuade whom. Specifically,\nwe propose a novel neural model that is able to dynamically track the changes\nof latent topics and discourse in argumentative conversations, allowing the\ninvestigation of their roles in influencing the outcomes of persuasion.\nExtensive experiments have been conducted on argumentative conversations on\nboth social media and supreme court. The results show that our model\noutperforms state-of-the-art models in identifying persuasive arguments via\nexplicitly exploring dynamic factors of topic and discourse. We further analyze\nthe effects of topics and discourse on persuasiveness, and find that they are\nboth useful - topics provide concrete evidence while superior discourse styles\nmay bias participants, especially in social media arguments. In addition, we\ndraw some findings from our empirical results, which will help people better\nengage in future persuasive conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03604v1",
            "date": "2020-02-10 08:53:27+00:00",
            "title": "A Study of Human Summaries of Scientific Articles",
            "authors": [
                "Odellia Boni",
                "Guy Feigenblat",
                "Doron Cohen",
                "Haggai Roitman",
                "David Konopnicki"
            ],
            "abstract": "Researchers and students face an explosion of newly published papers which\nmay be relevant to their work. This led to a trend of sharing human summaries\nof scientific papers. We analyze the summaries shared in one of these platforms\nShortscience.org. The goal is to characterize human summaries of scientific\npapers, and use some of the insights obtained to improve and adapt existing\nautomatic summarization systems to the domain of scientific papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04095v2",
            "date": "2020-02-10 21:35:39+00:00",
            "title": "Automatic Discourse Segmentation: an evaluation in French",
            "authors": [
                "R\u00e9my Saksik",
                "Alejandro Molina-Villegas",
                "Andr\u00e9a Carneiro Linhares",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "In this article, we describe some discursive segmentation methods as well as\na preliminary evaluation of the segmentation quality. Although our experiment\nwere carried for documents in French, we have developed three discursive\nsegmentation models solely based on resources simultaneously available in\nseveral languages: marker lists and a statistic POS labeling. We have also\ncarried out automatic evaluations of these systems against the Annodis corpus,\nwhich is a manually annotated reference. The results obtained are very\nencouraging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04165v2",
            "date": "2020-02-11 01:52:05+00:00",
            "title": "Training with Streaming Annotation",
            "authors": [
                "Tongtao Zhang",
                "Heng Ji",
                "Shih-Fu Chang",
                "Marjorie Freedman"
            ],
            "abstract": "In this paper, we address a practical scenario where training data is\nreleased in a sequence of small-scale batches and annotation in earlier phases\nhas lower quality than the later counterparts. To tackle the situation, we\nutilize a pre-trained transformer network to preserve and integrate the most\nsalient document information from the earlier batches while focusing on the\nannotation (presumably with higher quality) from the current batch. Using event\nextraction as a case study, we demonstrate in the experiments that our proposed\nframework can perform better than conventional approaches (the improvement\nranges from 3.6 to 14.9% absolute F-score gain), especially when there is more\nnoise in the early annotation; and our approach spares 19.1% time with regard\nto the best conventional method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04250v2",
            "date": "2020-02-11 08:19:28+00:00",
            "title": "Non-Autoregressive Neural Dialogue Generation",
            "authors": [
                "Qinghong Han",
                "Yuxian Meng",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Maximum Mutual information (MMI), which models the bidirectional dependency\nbetween responses ($y$) and contexts ($x$), i.e., the forward probability $\\log\np(y|x)$ and the backward probability $\\log p(x|y)$, has been widely used as the\nobjective in the \\sts model to address the dull-response issue in open-domain\ndialog generation. Unfortunately, under the framework of the \\sts model, direct\ndecoding from $\\log p(y|x) + \\log p(x|y)$ is infeasible since the second part\n(i.e., $p(x|y)$) requires the completion of target generation before it can be\ncomputed, and the search space for $y$ is enormous. Empirically, an N-best list\nis first generated given $p(y|x)$, and $p(x|y)$ is then used to rerank the\nN-best list, which inevitably results in non-globally-optimal solutions. In\nthis paper, we propose to use non-autoregressive (non-AR) generation model to\naddress this non-global optimality issue. Since target tokens are generated\nindependently in non-AR generation, $p(x|y)$ for each target word can be\ncomputed as soon as it's generated, and does not have to wait for the\ncompletion of the whole sequence. This naturally resolves the non-global\noptimal issue in decoding. Experimental results demonstrate that the proposed\nnon-AR strategy produces more diverse, coherent, and appropriate responses,\nyielding substantive gains in BLEU scores and in human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04678v1",
            "date": "2020-02-11 20:59:34+00:00",
            "title": "Adjusting Image Attributes of Localized Regions with Low-level Dialogue",
            "authors": [
                "Tzu-Hsiang Lin",
                "Alexander Rudnicky",
                "Trung Bui",
                "Doo Soon Kim",
                "Jean Oh"
            ],
            "abstract": "Natural Language Image Editing (NLIE) aims to use natural language\ninstructions to edit images. Since novices are inexperienced with image editing\ntechniques, their instructions are often ambiguous and contain high-level\nabstractions that tend to correspond to complex editing steps to accomplish.\nMotivated by this inexperience aspect, we aim to smooth the learning curve by\nteaching the novices to edit images using low-level commanding terminologies.\nTowards this end, we develop a task-oriented dialogue system to investigate\nlow-level instructions for NLIE. Our system grounds language on the level of\nedit operations, and suggests options for a user to choose from. Though\ncompelled to express in low-level terms, a user evaluation shows that 25% of\nusers found our system easy-to-use, resonating with our motivation. An analysis\nshows that users generally adapt to utilizing the proposed low-level language\ninterface. In this study, we identify that object segmentation as the key\nfactor to the user satisfaction. Our work demonstrates the advantages of the\nlow-level, direct language-action mapping approach that can be applied to other\nproblem domains beyond image editing such as audio editing or industrial\ndesign.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05407v1",
            "date": "2020-02-13 09:48:31+00:00",
            "title": "Keyphrase Extraction with Span-based Feature Representations",
            "authors": [
                "Funan Mu",
                "Zhenting Yu",
                "LiFeng Wang",
                "Yequan Wang",
                "Qingyu Yin",
                "Yibo Sun",
                "Liqun Liu",
                "Teng Ma",
                "Jing Tang",
                "Xing Zhou"
            ],
            "abstract": "Keyphrases are capable of providing semantic metadata characterizing\ndocuments and producing an overview of the content of a document. Since\nkeyphrase extraction is able to facilitate the management, categorization, and\nretrieval of information, it has received much attention in recent years. There\nare three approaches to address keyphrase extraction: (i) traditional two-step\nranking method, (ii) sequence labeling and (iii) generation using neural\nnetworks. Two-step ranking approach is based on feature engineering, which is\nlabor intensive and domain dependent. Sequence labeling is not able to tackle\noverlapping phrases. Generation methods (i.e., Sequence-to-sequence neural\nnetwork models) overcome those shortcomings, so they have been widely studied\nand gain state-of-the-art performance. However, generation methods can not\nutilize context information effectively. In this paper, we propose a novelty\nSpan Keyphrase Extraction model that extracts span-based feature representation\nof keyphrase directly from all the content tokens. In this way, our model\nobtains representation for each keyphrase and further learns to capture the\ninteraction between keyphrases in one document to get better ranking results.\nIn addition, with the help of tokens, our model is able to extract overlapped\nkeyphrases. Experimental results on the benchmark datasets show that our\nproposed model outperforms the existing methods by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05417v1",
            "date": "2020-02-13 10:09:31+00:00",
            "title": "Comparison of Turkish Word Representations Trained on Different Morphological Forms",
            "authors": [
                "G\u00f6khan G\u00fcler",
                "A. C\u00fcneyd Tantu\u011f"
            ],
            "abstract": "Increased popularity of different text representations has also brought many\nimprovements in Natural Language Processing (NLP) tasks. Without need of\nsupervised data, embeddings trained on large corpora provide us meaningful\nrelations to be used on different NLP tasks. Even though training these vectors\nis relatively easy with recent methods, information gained from the data\nheavily depends on the structure of the corpus language. Since the popularly\nresearched languages have a similar morphological structure, problems occurring\nfor morphologically rich languages are mainly disregarded in studies. For\nmorphologically rich languages, context-free word vectors ignore morphological\nstructure of languages. In this study, we prepared texts in morphologically\ndifferent forms in a morphologically rich language, Turkish, and compared the\nresults on different intrinsic and extrinsic tasks. To see the effect of\nmorphological structure, we trained word2vec model on texts which lemma and\nsuffixes are treated differently. We also trained subword model fastText and\ncompared the embeddings on word analogy, text classification, sentimental\nanalysis, and language model tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05829v1",
            "date": "2020-02-14 01:04:19+00:00",
            "title": "HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing",
            "authors": [
                "Xiyou Zhou",
                "Zhiyu Chen",
                "Xiaoyong Jin",
                "William Yang Wang"
            ],
            "abstract": "Computation-intensive pretrained models have been taking the lead of many\nnatural language processing benchmarks such as GLUE. However, energy efficiency\nin the process of model training and inference becomes a critical bottleneck.\nWe introduce HULK, a multi-task energy efficiency benchmarking platform for\nresponsible natural language processing. With HULK, we compare pretrained\nmodels' energy efficiency from the perspectives of time and cost. Baseline\nbenchmarking results are provided for further analysis. The fine-tuning\nefficiency of different pretrained models can differ a lot among different\ntasks and fewer parameter number does not necessarily imply better efficiency.\nWe analyzed such phenomenon and demonstrate the method of comparing the\nmulti-task efficiency of pretrained models. Our platform is available at\nhttps://sites.engineering.ucsb.edu/~xiyou/hulk/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05902v1",
            "date": "2020-02-14 07:45:33+00:00",
            "title": "Understanding patient complaint characteristics using contextual clinical BERT embeddings",
            "authors": [
                "Budhaditya Saha",
                "Sanal Lisboa",
                "Shameek Ghosh"
            ],
            "abstract": "In clinical conversational applications, extracted entities tend to capture\nthe main subject of a patient's complaint, namely symptoms or diseases.\nHowever, they mostly fail to recognize the characterizations of a complaint\nsuch as the time, the onset, and the severity. For example, if the input is \"I\nhave a headache and it is extreme\", state-of-the-art models only recognize the\nmain symptom entity - headache, but ignore the severity factor of \"extreme\",\nthat characterizes headache. In this paper, we design a two-stage approach to\ndetect the characterizations of entities like symptoms presented by general\nusers in contexts where they would describe their symptoms to a clinician. We\nuse Word2Vec and BERT to encode clinical text given by the patients. We\ntransform the output and re-frame the task as multi-label classification\nproblem. Finally, we combine the processed encodings with the Linear\nDiscriminant Analysis (LDA) algorithm to classify the characterizations of the\nmain entity. Experimental results demonstrate that our method achieves 40-50%\nimprovement on the accuracy over the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06424v1",
            "date": "2020-02-15 18:34:52+00:00",
            "title": "Deeper Task-Specificity Improves Joint Entity and Relation Extraction",
            "authors": [
                "Phil Crone"
            ],
            "abstract": "Multi-task learning (MTL) is an effective method for learning related tasks,\nbut designing MTL models necessitates deciding which and how many parameters\nshould be task-specific, as opposed to shared between tasks. We investigate\nthis issue for the problem of jointly learning named entity recognition (NER)\nand relation extraction (RE) and propose a novel neural architecture that\nallows for deeper task-specificity than does prior work. In particular, we\nintroduce additional task-specific bidirectional RNN layers for both the NER\nand RE tasks and tune the number of shared and task-specific layers separately\nfor different datasets. We achieve state-of-the-art (SOTA) results for both\ntasks on the ADE dataset; on the CoNLL04 dataset, we achieve SOTA results on\nthe NER task and competitive results on the RE task while using an order of\nmagnitude fewer trainable parameters than the current SOTA architecture. An\nablation study confirms the importance of the additional task-specific layers\nfor achieving these results. Our work suggests that previous solutions to joint\nNER and RE undervalue task-specificity and demonstrates the importance of\ncorrectly balancing the number of shared and task-specific parameters for MTL\napproaches in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06484v1",
            "date": "2020-02-16 01:09:41+00:00",
            "title": "A Multimodal Dialogue System for Conversational Image Editing",
            "authors": [
                "Tzu-Hsiang Lin",
                "Trung Bui",
                "Doo Soon Kim",
                "Jean Oh"
            ],
            "abstract": "In this paper, we present a multimodal dialogue system for Conversational\nImage Editing. We formulate our multimodal dialogue system as a Partially\nObserved Markov Decision Process (POMDP) and trained it with Deep Q-Network\n(DQN) and a user simulator. Our evaluation shows that the DQN policy\noutperforms a rule-based baseline policy, achieving 90\\% success rate under\nhigh error rates. We also conducted a real user study and analyzed real user\nbehavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06546v2",
            "date": "2020-02-16 09:42:01+00:00",
            "title": "Neural Machine Translation with Joint Representation",
            "authors": [
                "Yanyang Li",
                "Qiang Wang",
                "Tong Xiao",
                "Tongran Liu",
                "Jingbo Zhu"
            ],
            "abstract": "Though early successes of Statistical Machine Translation (SMT) systems are\nattributed in part to the explicit modelling of the interaction between any two\nsource and target units, e.g., alignment, the recent Neural Machine Translation\n(NMT) systems resort to the attention which partially encodes the interaction\nfor efficiency. In this paper, we employ Joint Representation that fully\naccounts for each possible interaction. We sidestep the inefficiency issue by\nrefining representations with the proposed efficient attention operation. The\nresulting Reformer models offer a new Sequence-to- Sequence modelling paradigm\nbesides the Encoder-Decoder framework and outperform the Transformer baseline\nin either the small scale IWSLT14 German-English, English-German and IWSLT15\nVietnamese-English or the large scale NIST12 Chinese-English translation tasks\nby about 1 BLEU point.We also propose a systematic model scaling approach,\nallowing the Reformer model to beat the state-of-the-art Transformer in IWSLT14\nGerman-English and NIST12 Chinese-English with about 50% fewer parameters. The\ncode is publicly available at https://github.com/lyy1994/reformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06644v1",
            "date": "2020-02-16 18:39:16+00:00",
            "title": "Towards Detection of Subjective Bias using Contextualized Word Embeddings",
            "authors": [
                "Tanvi Dadu",
                "Kartikey Pant",
                "Radhika Mamidi"
            ],
            "abstract": "Subjective bias detection is critical for applications like propaganda\ndetection, content recommendation, sentiment analysis, and bias neutralization.\nThis bias is introduced in natural language via inflammatory words and phrases,\ncasting doubt over facts, and presupposing the truth. In this work, we perform\ncomprehensive experiments for detecting subjective bias using BERT-based models\non the Wiki Neutrality Corpus(WNC). The dataset consists of $360k$ labeled\ninstances, from Wikipedia edits that remove various instances of the bias. We\nfurther propose BERT-based ensembles that outperform state-of-the-art methods\nlike $BERT_{large}$ by a margin of $5.6$ F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06714v1",
            "date": "2020-02-16 23:53:07+00:00",
            "title": "Multi-layer Representation Fusion for Neural Machine Translation",
            "authors": [
                "Qiang Wang",
                "Fuxue Li",
                "Tong Xiao",
                "Yanyang Li",
                "Yinqiao Li",
                "Jingbo Zhu"
            ],
            "abstract": "Neural machine translation systems require a number of stacked layers for\ndeep models. But the prediction depends on the sentence representation of the\ntop-most layer with no access to low-level representations. This makes it more\ndifficult to train the model and poses a risk of information loss to\nprediction. In this paper, we propose a multi-layer representation fusion\n(MLRF) approach to fusing stacked layers. In particular, we design three fusion\nfunctions to learn a better representation from the stack. Experimental results\nshow that our approach yields improvements of 0.92 and 0.56 BLEU points over\nthe strong Transformer baseline on IWSLT German-English and NIST\nChinese-English MT tasks respectively. The result is new state-of-the-art in\nGerman-English translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06823v1",
            "date": "2020-02-17 08:13:36+00:00",
            "title": "Incorporating BERT into Neural Machine Translation",
            "authors": [
                "Jinhua Zhu",
                "Yingce Xia",
                "Lijun Wu",
                "Di He",
                "Tao Qin",
                "Wengang Zhou",
                "Houqiang Li",
                "Tie-Yan Liu"
            ],
            "abstract": "The recently proposed BERT has shown great power on a variety of natural\nlanguage understanding tasks, such as text classification, reading\ncomprehension, etc. However, how to effectively apply BERT to neural machine\ntranslation (NMT) lacks enough exploration. While BERT is more commonly used as\nfine-tuning instead of contextual embedding for downstream language\nunderstanding tasks, in NMT, our preliminary exploration of using BERT as\ncontextual embedding is better than using for fine-tuning. This motivates us to\nthink how to better leverage BERT for NMT along this direction. We propose a\nnew algorithm named BERT-fused model, in which we first use BERT to extract\nrepresentations for an input sequence, and then the representations are fused\nwith each layer of the encoder and decoder of the NMT model through attention\nmechanisms. We conduct experiments on supervised (including sentence-level and\ndocument-level translations), semi-supervised and unsupervised machine\ntranslation, and achieve state-of-the-art results on seven benchmark datasets.\nOur code is available at \\url{https://github.com/bert-nmt/bert-nmt}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06851v1",
            "date": "2020-02-17 09:25:19+00:00",
            "title": "GameWikiSum: a Novel Large Multi-Document Summarization Dataset",
            "authors": [
                "Diego Antognini",
                "Boi Faltings"
            ],
            "abstract": "Today's research progress in the field of multi-document summarization is\nobstructed by the small number of available datasets. Since the acquisition of\nreference summaries is costly, existing datasets contain only hundreds of\nsamples at most, resulting in heavy reliance on hand-crafted features or\nnecessitating additional, manually annotated data. The lack of large corpora\ntherefore hinders the development of sophisticated models. Additionally, most\npublicly available multi-document summarization corpora are in the news domain,\nand no analogous dataset exists in the video game domain. In this paper, we\npropose GameWikiSum, a new domain-specific dataset for multi-document\nsummarization, which is one hundred times larger than commonly used datasets,\nand in another domain than news. Input documents consist of long professional\nvideo game reviews as well as references of their gameplay sections in\nWikipedia pages. We analyze the proposed dataset and show that both abstractive\nand extractive models can be trained on it. We release GameWikiSum for further\nresearch: https://github.com/Diego999/GameWikiSum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07306v2",
            "date": "2020-02-18 00:22:54+00:00",
            "title": "From English To Foreign Languages: Transferring Pre-trained Language Models",
            "authors": [
                "Ke Tran"
            ],
            "abstract": "Pre-trained models have demonstrated their effectiveness in many downstream\nnatural language processing (NLP) tasks. The availability of multilingual\npre-trained models enables zero-shot transfer of NLP tasks from high resource\nlanguages to low resource ones. However, recent research in improving\npre-trained models focuses heavily on English. While it is possible to train\nthe latest neural architectures for other languages from scratch, it is\nundesirable due to the required amount of compute. In this work, we tackle the\nproblem of transferring an existing pre-trained model from English to other\nlanguages under a limited computational budget. With a single GPU, our approach\ncan obtain a foreign BERT base model within a day and a foreign BERT large\nwithin two days. Furthermore, evaluating our models on six languages, we\ndemonstrate that our models are better than multilingual BERT on two zero-shot\ntasks: natural language inference and dependency parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07339v1",
            "date": "2020-02-18 02:30:03+00:00",
            "title": "Annotating and Extracting Synthesis Process of All-Solid-State Batteries from Scientific Literature",
            "authors": [
                "Fusataka Kuniyoshi",
                "Kohei Makino",
                "Jun Ozawa",
                "Makoto Miwa"
            ],
            "abstract": "The synthesis process is essential for achieving computational experiment\ndesign in the field of inorganic materials chemistry. In this work, we present\na novel corpus of the synthesis process for all-solid-state batteries and an\nautomated machine reading system for extracting the synthesis processes buried\nin the scientific literature. We define the representation of the synthesis\nprocesses using flow graphs, and create a corpus from the experimental sections\nof 243 papers. The automated machine-reading system is developed by a deep\nlearning-based sequence tagger and simple heuristic rule-based relation\nextractor. Our experimental results demonstrate that the sequence tagger with\nthe optimal setting can detect the entities with a macro-averaged F1 score of\n0.826, while the rule-based relation extractor can achieve high performance\nwith a macro-averaged F1 score of 0.887.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07458v1",
            "date": "2020-02-18 09:58:59+00:00",
            "title": "A New Clustering neural network for Chinese word segmentation",
            "authors": [
                "Yuze Zhao"
            ],
            "abstract": "In this article I proposed a new model to achieve Chinese word\nsegmentation(CWS),which may have the potentiality to apply in other domains in\nthe future.It is a new thinking in CWS compared to previous works,to consider\nit as a clustering problem instead of a labeling problem.In this model,LSTM and\nself attention structures are used to collect context also sentence level\nfeatures in every layer,and after several layers,a clustering model is applied\nto split characters into groups,which are the final segmentation results.I call\nthis model CLNN.This algorithm can reach 98 percent of F score (without OOV\nwords) and 85 percent to 95 percent F score (with OOV words) in training data\nsets.Error analyses shows that OOV words will greatly reduce performances,which\nneeds a deeper research in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07510v2",
            "date": "2020-02-18 11:59:59+00:00",
            "title": "Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue",
            "authors": [
                "Byeongchang Kim",
                "Jaewoo Ahn",
                "Gunhee Kim"
            ],
            "abstract": "Knowledge-grounded dialogue is a task of generating an informative response\nbased on both discourse context and external knowledge. As we focus on better\nmodeling the knowledge selection in the multi-turn knowledge-grounded dialogue,\nwe propose a sequential latent variable model as the first approach to this\nmatter. The model named sequential knowledge transformer (SKT) can keep track\nof the prior and posterior distribution over knowledge; as a result, it can not\nonly reduce the ambiguity caused from the diversity in knowledge selection of\nconversation but also better leverage the response information for proper\nchoice of knowledge. Our experimental results show that the proposed model\nimproves the knowledge selection accuracy and subsequently the performance of\nutterance generation. We achieve the new state-of-the-art performance on Wizard\nof Wikipedia (Dinan et al., 2019) as one of the most large-scale and\nchallenging benchmarks. We further validate the effectiveness of our model over\nexisting conversation methods in another knowledge-based dialogue Holl-E\ndataset (Moghe et al., 2018).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07526v1",
            "date": "2020-02-18 12:49:14+00:00",
            "title": "A Survey of Deep Learning Techniques for Neural Machine Translation",
            "authors": [
                "Shuoheng Yang",
                "Yuxin Wang",
                "Xiaowen Chu"
            ],
            "abstract": "In recent years, natural language processing (NLP) has got great development\nwith deep learning techniques. In the sub-field of machine translation, a new\napproach named Neural Machine Translation (NMT) has emerged and got massive\nattention from both academia and industry. However, with a significant number\nof researches proposed in the past several years, there is little work in\ninvestigating the development process of this new technology trend. This\nliterature survey traces back the origin and principal development timeline of\nNMT, investigates the important branches, categorizes different research\norientations, and discusses some future research trends in this field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07767v2",
            "date": "2020-02-18 17:59:02+00:00",
            "title": "Learning by Semantic Similarity Makes Abstractive Summarization Better",
            "authors": [
                "Wonjin Yoon",
                "Yoon Sun Yeo",
                "Minbyul Jeong",
                "Bong-Jun Yi",
                "Jaewoo Kang"
            ],
            "abstract": "By harnessing pre-trained language models, summarization models had rapid\nprogress recently. However, the models are mainly assessed by automatic\nevaluation metrics such as ROUGE. Although ROUGE is known for having a positive\ncorrelation with human evaluation scores, it has been criticized for its\nvulnerability and the gap between actual qualities. In this paper, we compare\nthe generated summaries from recent LM, BART, and the reference summaries from\na benchmark dataset, CNN/DM, using a crowd-sourced human evaluation metric.\nInterestingly, model-generated summaries receive higher scores relative to\nreference summaries. Stemming from our experimental results, we first argue the\nintrinsic characteristics of the CNN/DM dataset, the progress of pre-trained\nlanguage models, and their ability to generalize on the training data. Finally,\nwe share our insights into the model-generated summaries and presents our\nthought on learning methods for abstractive summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07845v2",
            "date": "2020-02-18 19:40:20+00:00",
            "title": "Interpretable Multi-Headed Attention for Abstractive Summarization at Controllable Lengths",
            "authors": [
                "Ritesh Sarkhel",
                "Moniba Keymanesh",
                "Arnab Nandi",
                "Srinivasan Parthasarathy"
            ],
            "abstract": "Abstractive summarization at controllable lengths is a challenging task in\nnatural language processing. It is even more challenging for domains where\nlimited training data is available or scenarios in which the length of the\nsummary is not known beforehand. At the same time, when it comes to trusting\nmachine-generated summaries, explaining how a summary was constructed in\nhuman-understandable terms may be critical. We propose Multi-level Summarizer\n(MLS), a supervised method to construct abstractive summaries of a text\ndocument at controllable lengths. The key enabler of our method is an\ninterpretable multi-headed attention mechanism that computes attention\ndistribution over an input document using an array of timestep independent\nsemantic kernels. Each kernel optimizes a human-interpretable syntactic or\nsemantic property. Exhaustive experiments on two low-resource datasets in the\nEnglish language show that MLS outperforms strong baselines by up to 14.70% in\nthe METEOR score. Human evaluation of the summaries also suggests that they\ncapture the key concepts of the document at various length-budgets.",
            "journal ref": "International Conference on Computational Linguistics (COLING)\n  2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07972v2",
            "date": "2020-02-19 03:05:28+00:00",
            "title": "The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding",
            "authors": [
                "Xiaodong Liu",
                "Yu Wang",
                "Jianshu Ji",
                "Hao Cheng",
                "Xueyun Zhu",
                "Emmanuel Awa",
                "Pengcheng He",
                "Weizhu Chen",
                "Hoifung Poon",
                "Guihong Cao",
                "Jianfeng Gao"
            ],
            "abstract": "We present MT-DNN, an open-source natural language understanding (NLU)\ntoolkit that makes it easy for researchers and developers to train customized\ndeep learning models. Built upon PyTorch and Transformers, MT-DNN is designed\nto facilitate rapid customization for a broad spectrum of NLU tasks, using a\nvariety of objectives (classification, regression, structured prediction) and\ntext encoders (e.g., RNNs, BERT, RoBERTa, UniLM). A unique feature of MT-DNN is\nits built-in support for robust and transferable learning using the adversarial\nmulti-task learning paradigm. To enable efficient production deployment, MT-DNN\nsupports multi-task knowledge distillation, which can substantially compress a\ndeep neural model without significant performance drop. We demonstrate the\neffectiveness of MT-DNN on a wide range of NLU applications across general and\nbiomedical domains. The software and pre-trained models will be publicly\navailable at https://github.com/namisan/mt-dnn.",
            "journal ref": "ACL 2020 demo",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07982v2",
            "date": "2020-02-19 03:30:00+00:00",
            "title": "Towards Making the Most of Context in Neural Machine Translation",
            "authors": [
                "Zaixiang Zheng",
                "Xiang Yue",
                "Shujian Huang",
                "Jiajun Chen",
                "Alexandra Birch"
            ],
            "abstract": "Document-level machine translation manages to outperform sentence level\nmodels by a small margin, but have failed to be widely adopted. We argue that\nprevious research did not make a clear use of the global context, and propose a\nnew document-level NMT framework that deliberately models the local context of\neach sentence with the awareness of the global context of the document in both\nsource and target languages. We specifically design the model to be able to\ndeal with documents containing any number of sentences, including single\nsentences. This unified approach allows our model to be trained elegantly on\nstandard datasets without needing to train on sentence and document level data\nseparately. Experimental results demonstrate that our model outperforms\nTransformer baselines and previous document-level NMT models with substantial\nmargins of up to 2.1 BLEU on state-of-the-art baselines. We also provide\nanalyses which show the benefit of context far beyond the neighboring two or\nthree sentences, which previous studies have typically incorporated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08087v5",
            "date": "2020-02-19 09:48:39+00:00",
            "title": "LAMBERT: Layout-Aware (Language) Modeling for information extraction",
            "authors": [
                "\u0141ukasz Garncarek",
                "Rafa\u0142 Powalski",
                "Tomasz Stanis\u0142awek",
                "Bartosz Topolski",
                "Piotr Halama",
                "Micha\u0142 Turski",
                "Filip Grali\u0144ski"
            ],
            "abstract": "We introduce a simple new approach to the problem of understanding documents\nwhere non-trivial layout influences the local semantics. To this end, we modify\nthe Transformer encoder architecture in a way that allows it to use layout\nfeatures obtained from an OCR system, without the need to re-learn language\nsemantics from scratch. We only augment the input of the model with the\ncoordinates of token bounding boxes, avoiding, in this way, the use of raw\nimages. This leads to a layout-aware language model which can then be\nfine-tuned on downstream tasks.\n  The model is evaluated on an end-to-end information extraction task using\nfour publicly available datasets: Kleister NDA, Kleister Charity, SROIE and\nCORD. We show that our model achieves superior performance on datasets\nconsisting of visually rich documents, while also outperforming the baseline\nRoBERTa on documents with flat layout (NDA \\(F_{1}\\) increase from 78.50 to\n80.42). Our solution ranked first on the public leaderboard for the Key\nInformation Extraction from the SROIE dataset, improving the SOTA\n\\(F_{1}\\)-score from 97.81 to 98.17.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08307v2",
            "date": "2020-02-19 17:40:57+00:00",
            "title": "Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning",
            "authors": [
                "Mitchell A. Gordon",
                "Kevin Duh",
                "Nicholas Andrews"
            ],
            "abstract": "Pre-trained universal feature extractors, such as BERT for natural language\nprocessing and VGG for computer vision, have become effective methods for\nimproving deep learning models without requiring more labeled data. While\neffective, feature extractors like BERT may be prohibitively large for some\ndeployment scenarios. We explore weight pruning for BERT and ask: how does\ncompression during pre-training affect transfer learning? We find that pruning\naffects transfer learning in three broad regimes. Low levels of pruning\n(30-40%) do not affect pre-training loss or transfer to downstream tasks at\nall. Medium levels of pruning increase the pre-training loss and prevent useful\npre-training information from being transferred to downstream tasks. High\nlevels of pruning additionally prevent models from fitting downstream datasets,\nleading to further degradation. Finally, we observe that fine-tuning BERT on a\nspecific task does not improve its prunability. We conclude that BERT can be\npruned once during pre-training rather than separately for each task without\naffecting performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08880v1",
            "date": "2020-02-20 17:31:04+00:00",
            "title": "The Fluidity of Concept Representations in Human Brain Signals",
            "authors": [
                "Eva Hendrikx",
                "Lisa Beinborn"
            ],
            "abstract": "Cognitive theories of human language processing often distinguish between\nconcrete and abstract concepts. In this work, we analyze the discriminability\nof concrete and abstract concepts in fMRI data using a range of analysis\nmethods. We find that the distinction can be decoded from the signal with an\naccuracy significantly above chance, but it is not found to be a relevant\nstructuring factor in clustering and relational analyses. From our detailed\ncomparison, we obtain the impression that human concept representations are\nmore fluid than dichotomous categories can capture. We argue that fluid concept\nrepresentations lead to more realistic models of human language processing\nbecause they better capture the ambiguity and underspecification present in\nnatural language use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08899v1",
            "date": "2020-02-06 18:51:16+00:00",
            "title": "Compositional Neural Machine Translation by Removing the Lexicon from Syntax",
            "authors": [
                "Tristan Thrush"
            ],
            "abstract": "The meaning of a natural language utterance is largely determined from its\nsyntax and words. Additionally, there is evidence that humans process an\nutterance by separating knowledge about the lexicon from syntax knowledge.\nTheories from semantics and neuroscience claim that complete word meanings are\nnot encoded in the representation of syntax. In this paper, we propose neural\nunits that can enforce this constraint over an LSTM encoder and decoder. We\ndemonstrate that our model achieves competitive performance across a variety of\ndomains including semantic parsing, syntactic parsing, and English to Mandarin\nChinese translation. In these cases, our model outperforms the standard LSTM\nencoder and decoder architecture on many or all of our metrics. To demonstrate\nthat our model achieves the desired separation between the lexicon and syntax,\nwe analyze its weights and explore its behavior when different neural modules\nare damaged. When damaged, we find that the model displays the knowledge\ndistortions that aphasics are evidenced to have.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09084v1",
            "date": "2020-02-21 01:47:09+00:00",
            "title": "On the impressive performance of randomly weighted encoders in summarization tasks",
            "authors": [
                "Jonathan Pilault",
                "Jaehong Park",
                "Christopher Pal"
            ],
            "abstract": "In this work, we investigate the performance of untrained randomly\ninitialized encoders in a general class of sequence to sequence models and\ncompare their performance with that of fully-trained encoders on the task of\nabstractive summarization. We hypothesize that random projections of an input\ntext have enough representational power to encode the hierarchical structure of\nsentences and semantics of documents. Using a trained decoder to produce\nabstractive text summaries, we empirically demonstrate that architectures with\nuntrained randomly initialized encoders perform competitively with respect to\nthe equivalent architectures with fully-trained encoders. We further find that\nthe capacity of the encoder not only improves overall model generalization but\nalso closes the performance gap between untrained randomly initialized and\nfull-trained encoders. To our knowledge, it is the first time that general\nsequence to sequence models with attention are assessed for trained and\nrandomly projected representations on abstractive summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09419v2",
            "date": "2020-02-21 17:09:19+00:00",
            "title": "Guider l'attention dans les modeles de sequence a sequence pour la prediction des actes de dialogue",
            "authors": [
                "Pierre Colombo",
                "Emile Chapuis",
                "Matteo Manica",
                "Emmanuel Vignon",
                "Giovanna Varni",
                "Chloe Clavel"
            ],
            "abstract": "The task of predicting dialog acts (DA) based on conversational dialog is a\nkey component in the development of conversational agents. Accurately\npredicting DAs requires a precise modeling of both the conversation and the\nglobal tag dependencies. We leverage seq2seq approaches widely adopted in\nNeural Machine Translation (NMT) to improve the modelling of tag sequentiality.\nSeq2seq models are known to learn complex global dependencies while currently\nproposed approaches using linear conditional random fields (CRF) only model\nlocal tag dependencies. In this work, we introduce a seq2seq model tailored for\nDA classification using: a hierarchical encoder, a novel guided attention\nmechanism and beam search applied to both training and inference. Compared to\nthe state of the art our model does not require handcrafted features and is\ntrained end-to-end. Furthermore, the proposed approach achieves an unmatched\naccuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on\nMRDA.",
            "journal ref": "WACAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09543v1",
            "date": "2020-02-21 20:39:09+00:00",
            "title": "Modelling Latent Skills for Multitask Language Generation",
            "authors": [
                "Kris Cao",
                "Dani Yogatama"
            ],
            "abstract": "We present a generative model for multitask conditional language generation.\nOur guiding hypothesis is that a shared set of latent skills underlies many\ndisparate language generation tasks, and that explicitly modelling these skills\nin a task embedding space can help with both positive transfer across tasks and\nwith efficient adaptation to new tasks. We instantiate this task embedding\nspace as a latent variable in a latent variable sequence-to-sequence model. We\nevaluate this hypothesis by curating a series of monolingual text-to-text\nlanguage generation datasets - covering a broad range of tasks and domains -\nand comparing the performance of models both in the multitask and few-shot\nregimes. We show that our latent task variable model outperforms other\nsequence-to-sequence baselines on average across tasks in the multitask\nsetting. In the few-shot learning setting on an unseen test dataset (i.e., a\nnew task), we demonstrate that model adaptation based on inference in the\nlatent task space is more robust than standard fine-tuning based parameter\nadaptation and performs comparably in terms of overall performance. Finally, we\nexamine the latent task representations learnt by our model and show that they\ncluster tasks in a natural way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09581v1",
            "date": "2020-02-22 00:35:26+00:00",
            "title": "Extracting and Validating Explanatory Word Archipelagoes using Dual Entropy",
            "authors": [
                "Yukio Ohsawa",
                "Teruaki Hayashi"
            ],
            "abstract": "The logical connectivity of text is represented by the connectivity of words\nthat form archipelagoes. Here, each archipelago is a sequence of islands of the\noccurrences of a certain word. An island here means the local sequence of\nsentences where the word is emphasized, and an archipelago of a length\ncomparable to the target text is extracted using the co-variation of entropy A\n(the window-based entropy) on the distribution of the word's occurrences with\nthe width of each time window. Then, the logical connectivity of text is\nevaluated on entropy B (the graph-based entropy) computed on the distribution\nof sentences to connected word-clusters obtained on the co-occurrence of words.\nThe results show the parts of the target text with words forming archipelagoes\nextracted on entropy A, without learned or prepared knowledge, form an\nexplanatory part of the text that is of smaller entropy B than the parts\nextracted by the baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09616v3",
            "date": "2020-02-22 04:05:41+00:00",
            "title": "\"Wait, I'm Still Talking!\" Predicting the Dialogue Interaction Behavior Using Imagine-Then-Arbitrate Model",
            "authors": [
                "Zehao Lin",
                "Xiaoming Kang",
                "Guodun Li",
                "Feng Ji",
                "Haiqing Chen",
                "Yin Zhang"
            ],
            "abstract": "Producing natural and accurate responses like human beings is the ultimate\ngoal of intelligent dialogue agents. So far, most of the past works concentrate\non selecting or generating one pertinent and fluent response according to\ncurrent query and its context. These models work on a one-to-one environment,\nmaking one response to one utterance each round. However, in real human-human\nconversations, human often sequentially sends several short messages for\nreadability instead of a long message in one turn. Thus messages will not end\nwith an explicit ending signal, which is crucial for agents to decide when to\nreply. So the first step for an intelligent dialogue agent is not replying but\ndeciding if it should reply at the moment. To address this issue, in this\npaper, we propose a novel Imagine-then-Arbitrate (ITA) neural dialogue model to\nhelp the agent decide whether to wait or to make a response directly. Our\nmethod has two imaginator modules and an arbitrator module. The two imaginators\nwill learn the agent's and user's speaking style respectively, generate\npossible utterances as the input of the arbitrator, combining with dialogue\nhistory. And the arbitrator decides whether to wait or to make a response to\nthe user directly. To verify the performance and effectiveness of our method,\nwe prepared two dialogue datasets and compared our approach with several\npopular models. Experimental results show that our model performs well on\naddressing ending prediction issue and outperforms baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09620v2",
            "date": "2020-02-22 04:12:37+00:00",
            "title": "Efficient Sentence Embedding via Semantic Subspace Analysis",
            "authors": [
                "Bin Wang",
                "Fenxiao Chen",
                "Yuncheng Wang",
                "C. -C. Jay Kuo"
            ],
            "abstract": "A novel sentence embedding method built upon semantic subspace analysis,\ncalled semantic subspace sentence embedding (S3E), is proposed in this work.\nGiven the fact that word embeddings can capture semantic relationship while\nsemantically similar words tend to form semantic groups in a high-dimensional\nembedding space, we develop a sentence representation scheme by analyzing\nsemantic subspaces of its constituent words. Specifically, we construct a\nsentence model from two aspects. First, we represent words that lie in the same\nsemantic group using the intra-group descriptor. Second, we characterize the\ninteraction between multiple semantic groups with the inter-group descriptor.\nThe proposed S3E method is evaluated on both textual similarity tasks and\nsupervised tasks. Experimental results show that it offers comparable or better\nperformance than the state-of-the-art. The complexity of our S3E method is also\nmuch lower than other parameterized models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09634v1",
            "date": "2020-02-22 05:40:32+00:00",
            "title": "Data Augmentation for Copy-Mechanism in Dialogue State Tracking",
            "authors": [
                "Xiaohui Song",
                "Liangjun Zang",
                "Yipeng Su",
                "Xing Wu",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "While several state-of-the-art approaches to dialogue state tracking (DST)\nhave shown promising performances on several benchmarks, there is still a\nsignificant performance gap between seen slot values (i.e., values that occur\nin both training set and test set) and unseen ones (values that occur in\ntraining set but not in test set). Recently, the copy-mechanism has been widely\nused in DST models to handle unseen slot values, which copies slot values from\nuser utterance directly. In this paper, we aim to find out the factors that\ninfluence the generalization ability of a common copy-mechanism model for DST.\nOur key observations include: 1) the copy-mechanism tends to memorize values\nrather than infer them from contexts, which is the primary reason for\nunsatisfactory generalization performance; 2) greater diversity of slot values\nin the training set increase the performance on unseen values but slightly\ndecrease the performance on seen values. Moreover, we propose a simple but\neffective algorithm of data augmentation to train copy-mechanism models, which\naugments the input dataset by copying user utterances and replacing the real\nslot values with randomly generated strings. Users could use two\nhyper-parameters to realize a trade-off between the performances on seen values\nand unseen ones, as well as a trade-off between overall performance and\ncomputational cost. Experimental results on three widely used datasets (WoZ\n2.0, DSTC2, and Multi-WoZ 2.0) show the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09637v2",
            "date": "2020-02-22 06:01:52+00:00",
            "title": "Markov Chain Monte-Carlo Phylogenetic Inference Construction in Computational Historical Linguistics",
            "authors": [
                "Tianyi Ni"
            ],
            "abstract": "More and more languages in the world are under study nowadays, as a result,\nthe traditional way of historical linguistics study is facing some challenges.\nFor example, the linguistic comparative research among languages needs manual\nannotation, which becomes more and more impossible with the increasing amount\nof language data coming out all around the world. Although it could hardly\nreplace linguists work, the automatic computational methods have been taken\ninto consideration and it can help people reduce their workload. One of the\nmost important work in historical linguistics is word comparison from different\nlanguages and find the cognate words for them, which means people try to figure\nout if the two languages are related to each other or not. In this paper, I am\ngoing to use computational method to cluster the languages and use Markov Chain\nMonte Carlo (MCMC) method to build the language typology relationship tree\nbased on the clusters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09646v2",
            "date": "2020-02-22 06:54:04+00:00",
            "title": "Machine Translation System Selection from Bandit Feedback",
            "authors": [
                "Jason Naradowsky",
                "Xuan Zhang",
                "Kevin Duh"
            ],
            "abstract": "Adapting machine translation systems in the real world is a difficult\nproblem. In contrast to offline training, users cannot provide the type of\nfine-grained feedback (such as correct translations) typically used for\nimproving the system. Moreover, different users have different translation\nneeds, and even a single user's needs may change over time.\n  In this work we take a different approach, treating the problem of adaptation\nas one of selection. Instead of adapting a single system, we train many\ntranslation systems using different architectures, datasets, and optimization\nmethods. Using bandit learning techniques on simulated user feedback, we learn\na policy to choose which system to use for a particular translation task. We\nshow that our approach can (1) quickly adapt to address domain changes in\ntranslation tasks, (2) outperform the single best system in mixed-domain\ntranslation tasks, and (3) make effective instance-specific decisions when\nusing contextual bandit strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09673v1",
            "date": "2020-02-22 10:06:37+00:00",
            "title": "Incorporating Effective Global Information via Adaptive Gate Attention for Text Classification",
            "authors": [
                "Xianming Li",
                "Zongxi Li",
                "Yingbin Zhao",
                "Haoran Xie",
                "Qing Li"
            ],
            "abstract": "The dominant text classification studies focus on training classifiers using\ntextual instances only or introducing external knowledge (e.g., hand-craft\nfeatures and domain expert knowledge). In contrast, some corpus-level\nstatistical features, like word frequency and distribution, are not well\nexploited. Our work shows that such simple statistical information can enhance\nclassification performance both efficiently and significantly compared with\nseveral baseline models. In this paper, we propose a classifier with gate\nmechanism named Adaptive Gate Attention model with Global Information (AGA+GI),\nin which the adaptive gate mechanism incorporates global statistical features\ninto latent semantic features and the attention layer captures dependency\nrelationship within the sentence. To alleviate the overfitting issue, we\npropose a novel Leaky Dropout mechanism to improve generalization ability and\nperformance stability. Our experiments show that the proposed method can\nachieve better accuracy than CNN-based and RNN-based approaches without global\ninformation on several benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09685v3",
            "date": "2020-02-22 11:17:16+00:00",
            "title": "Investigating Typed Syntactic Dependencies for Targeted Sentiment Classification Using Graph Attention Neural Network",
            "authors": [
                "Xuefeng Bai",
                "Pengbo Liu",
                "Yue Zhang"
            ],
            "abstract": "Targeted sentiment classification predicts the sentiment polarity on given\ntarget mentions in input texts. Dominant methods employ neural networks for\nencoding the input sentence and extracting relations between target mentions\nand their contexts. Recently, graph neural network has been investigated for\nintegrating dependency syntax for the task, achieving the state-of-the-art\nresults. However, existing methods do not consider dependency label\ninformation, which can be intuitively useful. To solve the problem, we\ninvestigate a novel relational graph attention network that integrates typed\nsyntactic dependency information. Results on standard benchmarks show that our\nmethod can effectively leverage label information for improving targeted\nsentiment classification performances. Our final model significantly\noutperforms state-of-the-art syntax-based approaches.",
            "journal ref": "IEEE/ACM Transactions on Audio, Speech, and Language Processing\n  (TASLP),2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09836v2",
            "date": "2020-02-23 06:21:43+00:00",
            "title": "Fill in the BLANC: Human-free quality estimation of document summaries",
            "authors": [
                "Oleg Vasilyev",
                "Vedant Dharnidharka",
                "John Bohannon"
            ],
            "abstract": "We present BLANC, a new approach to the automatic estimation of document\nsummary quality. Our goal is to measure the functional performance of a summary\nwith an objective, reproducible, and fully automated method. Our approach\nachieves this by measuring the performance boost gained by a pre-trained\nlanguage model with access to a document summary while carrying out its\nlanguage understanding task on the document's text. We present evidence that\nBLANC scores have as good correlation with human evaluations as do the ROUGE\nfamily of summary quality measurements. And unlike ROUGE, the BLANC method does\nnot require human-written reference summaries, allowing for fully human-free\nsummary quality estimation.",
            "journal ref": "Proceedings of the First Workshop on Evaluation and Comparison of\n  NLP Systems (Nov.2020) 11-20",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10101v1",
            "date": "2020-02-24 07:37:17+00:00",
            "title": "GRET: Global Representation Enhanced Transformer",
            "authors": [
                "Rongxiang Weng",
                "Haoran Wei",
                "Shujian Huang",
                "Heng Yu",
                "Lidong Bing",
                "Weihua Luo",
                "Jiajun Chen"
            ],
            "abstract": "Transformer, based on the encoder-decoder framework, has achieved\nstate-of-the-art performance on several natural language generation tasks. The\nencoder maps the words in the input sentence into a sequence of hidden states,\nwhich are then fed into the decoder to generate the output sentence. These\nhidden states usually correspond to the input words and focus on capturing\nlocal information. However, the global (sentence level) information is seldom\nexplored, leaving room for the improvement of generation quality. In this\npaper, we propose a novel global representation enhanced Transformer (GRET) to\nexplicitly model global representation in the Transformer network.\nSpecifically, in the proposed model, an external state is generated for the\nglobal representation from the encoder. The global representation is then fused\ninto the decoder during the decoding process to improve generation quality. We\nconduct experiments in two text generation tasks: machine translation and text\nsummarization. Experimental results on four WMT machine translation tasks and\nLCSTS text summarization task demonstrate the effectiveness of the proposed\napproach on natural language generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10210v1",
            "date": "2020-02-24 12:52:10+00:00",
            "title": "Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation",
            "authors": [
                "Xiaocheng Feng",
                "Yawei Sun",
                "Bing Qin",
                "Heng Gong",
                "Yibo Sun",
                "Wei Bi",
                "Xiaojiang Liu",
                "Ting Liu"
            ],
            "abstract": "In this paper, we focus on a new practical task, document-scale text content\nmanipulation, which is the opposite of text style transfer and aims to preserve\ntext styles while altering the content. In detail, the input is a set of\nstructured records and a reference text for describing another recordset. The\noutput is a summary that accurately describes the partial content in the source\nrecordset with the same writing style of the reference. The task is\nunsupervised due to lack of parallel data, and is challenging to select\nsuitable records and style words from bi-aspect inputs respectively and\ngenerate a high-fidelity long document. To tackle those problems, we first\nbuild a dataset based on a basketball game report corpus as our testbed, and\npresent an unsupervised neural model with interactive attention mechanism,\nwhich is used for learning the semantic relationship between records and\nreference texts to achieve better content transfer and better style\npreservation. In addition, we also explore the effectiveness of the\nback-translation in our task for constructing some pseudo-training pairs.\nEmpirical results show superiority of our approaches over competitive methods,\nand the models also yield a new state-of-the-art result on a sentence-level\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10260v3",
            "date": "2020-02-24 13:53:06+00:00",
            "title": "Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation",
            "authors": [
                "Alessandro Raganato",
                "Yves Scherrer",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "Transformer-based models have brought a radical change to neural machine\ntranslation. A key feature of the Transformer architecture is the so-called\nmulti-head attention mechanism, which allows the model to focus simultaneously\non different parts of the input. However, recent works have shown that most\nattention heads learn simple, and often redundant, positional patterns. In this\npaper, we propose to replace all but one attention head of each encoder layer\nwith simple fixed -- non-learnable -- attentive patterns that are solely based\non position and do not require any external knowledge. Our experiments with\ndifferent data sizes and multiple language pairs show that fixing the attention\nheads on the encoder side of the Transformer at training time does not impact\nthe translation quality and even increases BLEU scores by up to 3 points in\nlow-resource scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10348v1",
            "date": "2020-02-24 16:20:32+00:00",
            "title": "Low-Resource Knowledge-Grounded Dialogue Generation",
            "authors": [
                "Xueliang Zhao",
                "Wei Wu",
                "Chongyang Tao",
                "Can Xu",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Responding with knowledge has been recognized as an important capability for\nan intelligent conversational agent. Yet knowledge-grounded dialogues, as\ntraining data for learning such a response generation model, are difficult to\nobtain. Motivated by the challenge in practice, we consider knowledge-grounded\ndialogue generation under a natural assumption that only limited training\nexamples are available. In such a low-resource setting, we devise a\ndisentangled response decoder in order to isolate parameters that depend on\nknowledge-grounded dialogues from the entire generation model. By this means,\nthe major part of the model can be learned from a large number of ungrounded\ndialogues and unstructured documents, while the remaining small parameters can\nbe well fitted using the limited training examples. Evaluation results on two\nbenchmarks indicate that with only 1/8 training data, our model can achieve the\nstate-of-the-art performance and generalize well on out-of-domain knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10361v2",
            "date": "2020-02-24 16:45:59+00:00",
            "title": "Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition",
            "authors": [
                "Xiaolei Huang",
                "Linzi Xing",
                "Franck Dernoncourt",
                "Michael J. Paul"
            ],
            "abstract": "Existing research on fairness evaluation of document classification models\nmainly uses synthetic monolingual data without ground truth for author\ndemographic attributes. In this work, we assemble and publish a multilingual\nTwitter corpus for the task of hate speech detection with inferred four author\ndemographic factors: age, country, gender and race/ethnicity. The corpus covers\nfive languages: English, Italian, Polish, Portuguese and Spanish. We evaluate\nthe inferred demographic labels with a crowdsourcing platform, Figure Eight. To\nexamine factors that can cause biases, we take an empirical analysis of\ndemographic predictability on the English corpus. We measure the performance of\nfour popular document classifiers and evaluate the fairness and bias of the\nbaseline classifiers on the author-level demographic attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10416v1",
            "date": "2020-02-24 17:59:11+00:00",
            "title": "Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool",
            "authors": [
                "Utku T\u00fcrk",
                "Furkan Atmaca",
                "\u015eaziye Bet\u00fcl \u00d6zate\u015f",
                "G\u00f6zde Berk",
                "Seyyit Talha Bedir",
                "Abdullatif K\u00f6ksal",
                "Balk\u0131z \u00d6zt\u00fcrk Ba\u015faran",
                "Tunga G\u00fcng\u00f6r",
                "Arzucan \u00d6zg\u00fcr"
            ],
            "abstract": "In this paper, we describe our contributions and efforts to develop Turkish\nresources, which include a new treebank (BOUN Treebank) with novel sentences,\nalong with the guidelines we adopted and a new annotation tool we developed\n(BoAT). The manual annotation process we employed was shaped and implemented by\na team of four linguists and five NLP specialists. Decisions regarding the\nannotation of the BOUN Treebank were made in line with the Universal\nDependencies framework, which originated from the works of De Marneffe et al.\n(2014) and Nivre et al. (2016). We took into account the recent unifying\nefforts based on the re-annotation of other Turkish treebanks in the UD\nframework (T\\\"urk et al., 2019). Through the BOUN Treebank, we introduced a\ntotal of 9,757 sentences from various topics including biographical texts,\nnational newspapers, instructional texts, popular culture articles, and essays.\nIn addition, we report the parsing results of a graph-based dependency parser\nobtained over each text type, the total of the BOUN Treebank, and all Turkish\ntreebanks that we either re-annotated or introduced. We show that a\nstate-of-the-art dependency parser has improved scores for identifying the\nproper head and the syntactic relationships between the heads and the\ndependents. In light of these results, we have observed that the unification of\nthe Turkish annotation scheme and introducing a more comprehensive treebank\nimproves performance with regards to dependency parsing",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10546v1",
            "date": "2020-02-24 21:04:51+00:00",
            "title": "Parsing Early Modern English for Linguistic Search",
            "authors": [
                "Seth Kulick",
                "Neville Ryant"
            ],
            "abstract": "We investigate the question of whether advances in NLP over the last few\nyears make it possible to vastly increase the size of data usable for research\nin historical syntax. This brings together many of the usual tools in NLP -\nword embeddings, tagging, and parsing - in the service of linguistic queries\nover automatically annotated corpora. We train a part-of-speech (POS) tagger\nand parser on a corpus of historical English, using ELMo embeddings trained\nover a billion words of similar text. The evaluation is based on the standard\nmetrics, as well as on the accuracy of the query searches using the parsed\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10710v3",
            "date": "2020-02-25 07:49:12+00:00",
            "title": "End-to-end Emotion-Cause Pair Extraction via Learning to Link",
            "authors": [
                "Haolin Song",
                "Chen Zhang",
                "Qiuchi Li",
                "Dawei Song"
            ],
            "abstract": "Emotion-cause pair extraction (ECPE), as an emergent natural language\nprocessing task, aims at jointly investigating emotions and their underlying\ncauses in documents. It extends the previous emotion cause extraction (ECE)\ntask, yet without requiring a set of pre-given emotion clauses as in ECE.\nExisting approaches to ECPE generally adopt a two-stage method, i.e., (1)\nemotion and cause detection, and then (2) pairing the detected emotions and\ncauses. Such pipeline method, while intuitive, suffers from two critical\nissues, including error propagation across stages that may hinder the\neffectiveness, and high computational cost that would limit the practical\napplication of the method. To tackle these issues, we propose a multi-task\nlearning model that can extract emotions, causes and emotion-cause pairs\nsimultaneously in an end-to-end manner. Specifically, our model regards pair\nextraction as a link prediction task, and learns to link from emotion clauses\nto cause clauses, i.e., the links are directional. Emotion extraction and cause\nextraction are incorporated into the model as auxiliary tasks, which further\nboost the pair extraction. Experiments are conducted on an ECPE benchmarking\ndataset. The results show that our proposed model outperforms a range of\nstate-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10757v2",
            "date": "2020-02-25 09:18:26+00:00",
            "title": "Edge-Enhanced Graph Convolution Networks for Event Detection with Syntactic Relation",
            "authors": [
                "Shiyao Cui",
                "Bowen Yu",
                "Tingwen Liu",
                "Zhenyu Zhang",
                "Xuebin Wang",
                "Jinqiao Shi"
            ],
            "abstract": "Event detection (ED), a key subtask of information extraction, aims to\nrecognize instances of specific event types in text. Previous studies on the\ntask have verified the effectiveness of integrating syntactic dependency into\ngraph convolutional networks. However, these methods usually ignore dependency\nlabel information, which conveys rich and useful linguistic knowledge for ED.\nIn this paper, we propose a novel architecture named Edge-Enhanced Graph\nConvolution Networks (EE-GCN), which simultaneously exploits syntactic\nstructure and typed dependency label information to perform ED. Specifically,\nan edge-aware node update module is designed to generate expressive word\nrepresentations by aggregating syntactically-connected words through specific\ndependency types. Furthermore, to fully explore clues hidden in dependency\nedges, a node-aware edge update module is introduced, which refines the\nrelation representations with contextual information. These two modules are\ncomplementary to each other and work in a mutual promotion way. We conduct\nexperiments on the widely used ACE2005 dataset and the results show significant\nimprovement over competitive baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10829v1",
            "date": "2020-02-25 12:40:06+00:00",
            "title": "MuST-Cinema: a Speech-to-Subtitles corpus",
            "authors": [
                "Alina Karakanta",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Growing needs in localising audiovisual content in multiple languages through\nsubtitles call for the development of automatic solutions for human subtitling.\nNeural Machine Translation (NMT) can contribute to the automatisation of\nsubtitling, facilitating the work of human subtitlers and reducing turn-around\ntimes and related costs. NMT requires high-quality, large, task-specific\ntraining data. The existing subtitling corpora, however, are missing both\nalignments to the source language audio and important information about\nsubtitle breaks. This poses a significant limitation for developing efficient\nautomatic approaches for subtitling, since the length and form of a subtitle\ndirectly depends on the duration of the utterance. In this work, we present\nMuST-Cinema, a multilingual speech translation corpus built from TED subtitles.\nThe corpus is comprised of (audio, transcription, translation) triplets.\nSubtitle breaks are preserved by inserting special symbols. We show that the\ncorpus can be used to build models that efficiently segment sentences into\nsubtitles and propose a method for annotating existing subtitling corpora with\nsubtitle breaks, conforming to the constraint of length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10851v1",
            "date": "2020-02-25 13:27:31+00:00",
            "title": "Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks",
            "authors": [
                "Th\u00e9odore Bluche",
                "Ma\u00ebl Primet",
                "Thibault Gisselbrecht"
            ],
            "abstract": "We explore a keyword-based spoken language understanding system, in which the\nintent of the user can directly be derived from the detection of a sequence of\nkeywords in the query. In this paper, we focus on an open-vocabulary keyword\nspotting method, allowing the user to define their own keywords without having\nto retrain the whole model. We describe the different design choices leading to\na fast and small-footprint system, able to run on tiny devices, for any\narbitrary set of user-defined keywords, without training data specific to those\nkeywords. The model, based on a quantized long short-term memory (LSTM) neural\nnetwork, trained with connectionist temporal classification (CTC), weighs less\nthan 500KB. Our approach takes advantage of some properties of the predictions\nof CTC-trained networks to calibrate the confidence scores and implement a fast\ndetection algorithm. The proposed system outperforms a standard keyword-filler\nmodel approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10903v2",
            "date": "2020-02-25 14:43:56+00:00",
            "title": "KEML: A Knowledge-Enriched Meta-Learning Framework for Lexical Relation Classification",
            "authors": [
                "Chengyu Wang",
                "Minghui Qiu",
                "Jun Huang",
                "Xiaofeng He"
            ],
            "abstract": "Lexical relations describe how concepts are semantically related, in the form\nof relation triples. The accurate prediction of lexical relations between\nconcepts is challenging, due to the sparsity of patterns indicating the\nexistence of such relations. We propose the Knowledge-Enriched Meta-Learning\n(KEML) framework to address the task of lexical relation classification. In\nKEML, the LKB-BERT (Lexical Knowledge Base-BERT) model is presented to learn\nconcept representations from massive text corpora, with rich lexical knowledge\ninjected by distant supervision. A probabilistic distribution of auxiliary\ntasks is defined to increase the model's ability to recognize different types\nof lexical relations. We further combine a meta-learning process over the\nauxiliary task distribution and supervised learning to train the neural lexical\nrelation classifier. Experiments over multiple datasets show that KEML\noutperforms state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10931v1",
            "date": "2020-02-25 15:05:06+00:00",
            "title": "Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge",
            "authors": [
                "Bonnie J. Dorr",
                "Archna Bhatia",
                "Adam Dalton",
                "Brodie Mather",
                "Bryanna Hebenstreit",
                "Sashank Santhanam",
                "Zhuo Cheng",
                "Samira Shaikh",
                "Alan Zemel",
                "Tomek Strzalkowski"
            ],
            "abstract": "Social engineers attempt to manipulate users into undertaking actions such as\ndownloading malware by clicking links or providing access to money or sensitive\ninformation. Natural language processing, computational sociolinguistics, and\nmedia-specific structural clues provide a means for detecting both the ask\n(e.g., buy gift card) and the risk/reward implied by the ask, which we call\nframing (e.g., lose your job, get a raise). We apply linguistic resources such\nas Lexical Conceptual Structure to tackle ask detection and also leverage\nstructural clues such as links and their proximity to identified asks to\nimprove confidence in our results. Our experiments indicate that the\nperformance of ask detection, framing detection, and identification of the top\nask is improved by linguistically motivated classes coupled with structural\nclues such as links. Our approach is implemented in a system that informs users\nabout social engineering risk situations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10957v2",
            "date": "2020-02-25 15:21:10+00:00",
            "title": "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers",
            "authors": [
                "Wenhui Wang",
                "Furu Wei",
                "Li Dong",
                "Hangbo Bao",
                "Nan Yang",
                "Ming Zhou"
            ],
            "abstract": "Pre-trained language models (e.g., BERT (Devlin et al., 2018) and its\nvariants) have achieved remarkable success in varieties of NLP tasks. However,\nthese models usually consist of hundreds of millions of parameters which brings\nchallenges for fine-tuning and online serving in real-life applications due to\nlatency and capacity constraints. In this work, we present a simple and\neffective approach to compress large Transformer (Vaswani et al., 2017) based\npre-trained models, termed as deep self-attention distillation. The small model\n(student) is trained by deeply mimicking the self-attention module, which plays\na vital role in Transformer networks, of the large model (teacher).\nSpecifically, we propose distilling the self-attention module of the last\nTransformer layer of the teacher, which is effective and flexible for the\nstudent. Furthermore, we introduce the scaled dot-product between values in the\nself-attention module as the new deep self-attention knowledge, in addition to\nthe attention distributions (i.e., the scaled dot-product of queries and keys)\nthat have been used in existing works. Moreover, we show that introducing a\nteacher assistant (Mirzadeh et al., 2019) also helps the distillation of large\npre-trained Transformer models. Experimental results demonstrate that our\nmonolingual model outperforms state-of-the-art baselines in different parameter\nsize of student models. In particular, it retains more than 99% accuracy on\nSQuAD 2.0 and several GLUE benchmark tasks using 50% of the Transformer\nparameters and computations of the teacher model. We also obtain competitive\nresults in applying deep self-attention distillation to multilingual\npre-trained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10959v1",
            "date": "2020-02-25 15:22:23+00:00",
            "title": "A more abstractive summarization model",
            "authors": [
                "Satyaki Chakraborty",
                "Xinya Li",
                "Sayak Chakraborty"
            ],
            "abstract": "Pointer-generator network is an extremely popular method of text\nsummarization. More recent works in this domain still build on top of the\nbaseline pointer generator by augmenting a content selection phase, or by\ndecomposing the decoder into a contextual network and a language model.\nHowever, all such models that are based on the pointer-generator base\narchitecture cannot generate novel words in the summary and mostly copy words\nfrom the source text. In our work, we first thoroughly investigate why the\npointer-generator network is unable to generate novel words, and then address\nthat by adding an Out-of-vocabulary (OOV) penalty. This enables us to improve\nthe amount of novelty/abstraction significantly. We use normalized n-gram\nnovelty scores as a metric for determining the level of abstraction. Moreover,\nwe also report rouge scores of our model since most summarization models are\nevaluated with R-1, R-2, R-L scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11023v1",
            "date": "2020-02-25 16:44:50+00:00",
            "title": "Semantic Relatedness for Keyword Disambiguation: Exploiting Different Embeddings",
            "authors": [
                "Mar\u00eda G. Buey",
                "Carlos Bobed",
                "Jorge Gracia",
                "Eduardo Mena"
            ],
            "abstract": "Understanding the meaning of words is crucial for many tasks that involve\nhuman-machine interaction. This has been tackled by research in Word Sense\nDisambiguation (WSD) in the Natural Language Processing (NLP) field. Recently,\nWSD and many other NLP tasks have taken advantage of embeddings-based\nrepresentation of words, sentences, and documents. However, when it comes to\nWSD, most embeddings models suffer from ambiguity as they do not capture the\ndifferent possible meanings of the words. Even when they do, the list of\npossible meanings for a word (sense inventory) has to be known in advance at\ntraining time to be included in the embeddings space. Unfortunately, there are\nsituations in which such a sense inventory is not known in advance (e.g., an\nontology selected at run-time), or it evolves with time and its status diverges\nfrom the one at training time. This hampers the use of embeddings models for\nWSD. Furthermore, traditional WSD techniques do not perform well in situations\nin which the available linguistic information is very scarce, such as the case\nof keyword-based queries. In this paper, we propose an approach to keyword\ndisambiguation which grounds on a semantic relatedness between words and senses\nprovided by an external inventory (ontology) that is not known at training\ntime. Building on previous works, we present a semantic relatedness measure\nthat uses word embeddings, and explore different disambiguation algorithms to\nalso exploit both word and sentence representations. Experimental results show\nthat this approach achieves results comparable with the state of the art when\napplied for WSD, without training for a particular domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11143v1",
            "date": "2020-02-25 19:07:54+00:00",
            "title": "End-to-End Entity Linking and Disambiguation leveraging Word and Knowledge Graph Embeddings",
            "authors": [
                "Rostislav Nedelchev",
                "Debanjan Chaudhuri",
                "Jens Lehmann",
                "Asja Fischer"
            ],
            "abstract": "Entity linking - connecting entity mentions in a natural language utterance\nto knowledge graph (KG) entities is a crucial step for question answering over\nKGs. It is often based on measuring the string similarity between the entity\nlabel and its mention in the question. The relation referred to in the question\ncan help to disambiguate between entities with the same label. This can be\nmisleading if an incorrect relation has been identified in the relation linking\nstep. However, an incorrect relation may still be semantically similar to the\nrelation in which the correct entity forms a triple within the KG; which could\nbe captured by the similarity of their KG embeddings. Based on this idea, we\npropose the first end-to-end neural network approach that employs KG as well as\nword embeddings to perform joint relation and entity classification of simple\nquestions while implicitly performing entity disambiguation with the help of a\nnovel gating mechanism. An empirical evaluation shows that the proposed\napproach achieves a performance comparable to state-of-the-art entity linking\nwhile requiring less post-processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11402v2",
            "date": "2020-02-26 10:48:53+00:00",
            "title": "Detecting Potential Topics In News Using BERT, CRF and Wikipedia",
            "authors": [
                "Swapnil Ashok Jadhav"
            ],
            "abstract": "For a news content distribution platform like Dailyhunt, Named Entity\nRecognition is a pivotal task for building better user recommendation and\nnotification algorithms. Apart from identifying names, locations, organisations\nfrom the news for 13+ Indian languages and use them in algorithms, we also need\nto identify n-grams which do not necessarily fit in the definition of\nNamed-Entity, yet they are important. For example, \"me too movement\", \"beef\nban\", \"alwar mob lynching\". In this exercise, given an English language text,\nwe are trying to detect case-less n-grams which convey important information\nand can be used as topics and/or hashtags for a news. Model is built using\nWikipedia titles data, private English news corpus and BERT-Multilingual\npre-trained model, Bi-GRU and CRF architecture. It shows promising results when\ncompared with industry best Flair, Spacy and Stanford-caseless-NER in terms of\nF1 and especially Recall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11506v1",
            "date": "2020-02-24 20:11:35+00:00",
            "title": "Using Distributional Thesaurus Embedding for Co-hyponymy Detection",
            "authors": [
                "Abhik Jana",
                "Nikhil Reddy Varimalla",
                "Pawan Goyal"
            ],
            "abstract": "Discriminating lexical relations among distributionally similar words has\nalways been a challenge for natural language processing (NLP) community. In\nthis paper, we investigate whether the network embedding of distributional\nthesaurus can be effectively utilized to detect co-hyponymy relations. By\nextensive experiments over three benchmark datasets, we show that the vector\nrepresentation obtained by applying node2vec on distributional thesaurus\noutperforms the state-of-the-art models for binary classification of\nco-hyponymy vs. hypernymy, as well as co-hyponymy vs. meronymy, by huge\nmargins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11643v1",
            "date": "2020-02-26 17:18:49+00:00",
            "title": "Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers",
            "authors": [
                "Swapnil Ashok Jadhav"
            ],
            "abstract": "There have been very few attempts to benchmark performances of\nstate-of-the-art algorithms for Neural Machine Translation task on Indian\nLanguages. Google, Bing, Facebook and Yandex are some of the very few companies\nwhich have built translation systems for few of the Indian Languages. Among\nthem, translation results from Google are supposed to be better, based on\ngeneral inspection. Bing-Translator do not even support Marathi language which\nhas around 95 million speakers and ranks 15th in the world in terms of combined\nprimary and secondary speakers. In this exercise, we trained and compared\nvariety of Neural Machine Marathi to English Translators trained with\nBERT-tokenizer by huggingface and various Transformer based architectures using\nFacebook's Fairseq platform with limited but almost correct parallel corpus to\nachieve better BLEU scores than Google on Tatoeba and Wikimedia open datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11893v2",
            "date": "2020-02-27 03:06:35+00:00",
            "title": "CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset",
            "authors": [
                "Qi Zhu",
                "Kaili Huang",
                "Zheng Zhang",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "To advance multi-domain (cross-domain) dialogue modeling as well as alleviate\nthe shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first\nlarge-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dataset. It\ncontains 6K dialogue sessions and 102K utterances for 5 domains, including\nhotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains\nrich annotation of dialogue states and dialogue acts at both user and system\nsides. About 60% of the dialogues have cross-domain user goals that favor\ninter-domain dependency and encourage natural transition across domains in\nconversation. We also provide a user simulator and several benchmark models for\npipelined task-oriented dialogue systems, which will facilitate researchers to\ncompare and evaluate their models on this corpus. The large size and rich\nannotation of CrossWOZ make it suitable to investigate a variety of tasks in\ncross-domain dialogue modeling, such as dialogue state tracking, policy\nlearning, user simulation, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11910v1",
            "date": "2020-02-27 04:29:13+00:00",
            "title": "Integrating Boundary Assembling into a DNN Framework for Named Entity Recognition in Chinese Social Media Text",
            "authors": [
                "Zhaoheng Gong",
                "Ping Chen",
                "Jiang Zhou"
            ],
            "abstract": "Named entity recognition is a challenging task in Natural Language\nProcessing, especially for informal and noisy social media text. Chinese word\nboundaries are also entity boundaries, therefore, named entity recognition for\nChinese text can benefit from word boundary detection, outputted by Chinese\nword segmentation. Yet Chinese word segmentation poses its own difficulty\nbecause it is influenced by several factors, e.g., segmentation criteria,\nemployed algorithm, etc. Dealt improperly, it may generate a cascading failure\nto the quality of named entity recognition followed. In this paper we integrate\na boundary assembling method with the state-of-the-art deep neural network\nmodel, and incorporate the updated word boundary information into a conditional\nrandom field model for named entity recognition. Our method shows a 2% absolute\nimprovement over previous state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12097v1",
            "date": "2020-02-27 14:02:31+00:00",
            "title": "Improving cross-lingual model transfer by chunking",
            "authors": [
                "Ayan Das",
                "Sudeshna Sarkar"
            ],
            "abstract": "We present a shallow parser guided cross-lingual model transfer approach in\norder to address the syntactic differences between source and target languages\nmore effectively. In this work, we assume the chunks or phrases in a sentence\nas transfer units in order to address the syntactic differences between the\nsource and target languages arising due to the differences in ordering of words\nin the phrases and the ordering of phrases in a sentence separately.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12327v3",
            "date": "2020-02-27 18:46:42+00:00",
            "title": "A Primer in BERTology: What we know about how BERT works",
            "authors": [
                "Anna Rogers",
                "Olga Kovaleva",
                "Anna Rumshisky"
            ],
            "abstract": "Transformer-based models have pushed state of the art in many areas of NLP,\nbut our understanding of what is behind their success is still limited. This\npaper is the first survey of over 150 studies of the popular BERT model. We\nreview the current state of knowledge about how BERT works, what kind of\ninformation it learns and how it is represented, common modifications to its\ntraining objectives and architecture, the overparameterization issue and\napproaches to compression. We then outline directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12328v1",
            "date": "2020-02-27 18:48:33+00:00",
            "title": "Few-shot Natural Language Generation for Task-Oriented Dialog",
            "authors": [
                "Baolin Peng",
                "Chenguang Zhu",
                "Chunyuan Li",
                "Xiujun Li",
                "Jinchao Li",
                "Michael Zeng",
                "Jianfeng Gao"
            ],
            "abstract": "As a crucial component in task-oriented dialog systems, the Natural Language\nGeneration (NLG) module converts a dialog act represented in a semantic form\ninto a response in natural language. The success of traditional template-based\nor statistical models typically relies on heavily annotated data, which is\ninfeasible for new domains. Therefore, it is pivotal for an NLG system to\ngeneralize well with limited labelled data in real applications. To this end,\nwe present FewShotWoz, the first NLG benchmark to simulate the few-shot\nlearning setting in task-oriented dialog systems. Further, we develop the\nSC-GPT model. It is pre-trained on a large set of annotated NLG corpus to\nacquire the controllable generation ability, and fine-tuned with only a few\ndomain-specific labels to adapt to new domains. Experiments on FewShotWoz and\nthe large Multi-Domain-WOZ datasets show that the proposed SC-GPT significantly\noutperforms existing methods, measured by various automatic metrics and human\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12344v1",
            "date": "2020-02-27 18:58:15+00:00",
            "title": "Generating Followup Questions for Interpretable Multi-hop Question Answering",
            "authors": [
                "Christopher Malon",
                "Bing Bai"
            ],
            "abstract": "We propose a framework for answering open domain multi-hop questions in which\npartial information is read and used to generate followup questions, to finally\nbe answered by a pretrained single-hop answer extractor. This framework makes\neach hop interpretable, and makes the retrieval associated with later hops as\nflexible and specific as for the first hop. As a first instantiation of this\nframework, we train a pointer-generator network to predict followup questions\nbased on the question and partial information. This provides a novel\napplication of a neural question generation network, which is applied to give\nweak ground truth single-hop followup questions based on the final answers and\ntheir supporting facts. Learning to generate followup questions that select the\nrelevant answer spans against downstream supporting facts, while avoiding\ndistracting premises, poses an exciting semantic challenge for text generation.\nWe present an evaluation using the two-hop bridge questions of HotpotQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12540v1",
            "date": "2020-02-28 04:32:16+00:00",
            "title": "UKARA 1.0 Challenge Track 1: Automatic Short-Answer Scoring in Bahasa Indonesia",
            "authors": [
                "Ali Akbar Septiandri",
                "Yosef Ardhito Winatmoko"
            ],
            "abstract": "We describe our third-place solution to the UKARA 1.0 challenge on automated\nessay scoring. The task consists of a binary classification problem on two\ndatasets | answers from two different questions. We ended up using two\ndifferent models for the two datasets. For task A, we applied a random forest\nalgorithm on features extracted using unigram with latent semantic analysis\n(LSA). On the other hand, for task B, we only used logistic regression on\nTF-IDF features. Our model results in F1 score of 0.812.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12549v2",
            "date": "2020-02-28 05:17:55+00:00",
            "title": "Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training",
            "authors": [
                "Haipeng Sun",
                "Rui Wang",
                "Kehai Chen",
                "Xugang Lu",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Tiejun Zhao"
            ],
            "abstract": "Unsupervised neural machine translation (UNMT) has recently attracted great\ninterest in the machine translation community. The main advantage of the UNMT\nlies in its easy collection of required large training text sentences while\nwith only a slightly worse performance than supervised neural machine\ntranslation which requires expensive annotated translation pairs on some\ntranslation tasks. In most studies, the UMNT is trained with clean data without\nconsidering its robustness to the noisy data. However, in real-world scenarios,\nthere usually exists noise in the collected input sentences which degrades the\nperformance of the translation system since the UNMT is sensitive to the small\nperturbations of the input sentences. In this paper, we first time explicitly\ntake the noisy data into consideration to improve the robustness of the UNMT\nbased systems. First of all, we clearly defined two types of noises in training\nsentences, i.e., word noise and word order noise, and empirically investigate\nits effect in the UNMT, then we propose adversarial training methods with\ndenoising process in the UNMT. Experimental results on several language pairs\nshow that our proposed methods substantially improved the robustness of the\nconventional UNMT systems in noisy scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12558v1",
            "date": "2020-02-28 05:37:06+00:00",
            "title": "Modeling Future Cost for Neural Machine Translation",
            "authors": [
                "Chaoqun Duan",
                "Kehai Chen",
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Existing neural machine translation (NMT) systems utilize\nsequence-to-sequence neural networks to generate target translation word by\nword, and then make the generated word at each time-step and the counterpart in\nthe references as consistent as possible. However, the trained translation\nmodel tends to focus on ensuring the accuracy of the generated target word at\nthe current time-step and does not consider its future cost which means the\nexpected cost of generating the subsequent target translation (i.e., the next\ntarget word). To respond to this issue, we propose a simple and effective\nmethod to model the future cost of each target word for NMT systems. In detail,\na time-dependent future cost is estimated based on the current generated target\nword and its contextual information to boost the training of the NMT model.\nFurthermore, the learned future context representation at the current time-step\nis used to help the generation of the next target word in the decoding.\nExperimental results on three widely-used translation datasets, including the\nWMT14 German-to-English, WMT14 English-to-French, and WMT17 Chinese-to-English,\nshow that the proposed approach achieves significant improvements over strong\nTransformer-based NMT baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12591v1",
            "date": "2020-02-28 08:18:37+00:00",
            "title": "DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding",
            "authors": [
                "Yuyu Zhang",
                "Ping Nie",
                "Xiubo Geng",
                "Arun Ramamurthy",
                "Le Song",
                "Daxin Jiang"
            ],
            "abstract": "Recent studies on open-domain question answering have achieved prominent\nperformance improvement using pre-trained language models such as BERT.\nState-of-the-art approaches typically follow the \"retrieve and read\" pipeline\nand employ BERT-based reranker to filter retrieved documents before feeding\nthem into the reader module. The BERT retriever takes as input the\nconcatenation of question and each retrieved document. Despite the success of\nthese approaches in terms of QA accuracy, due to the concatenation, they can\nbarely handle high-throughput of incoming questions each with a large\ncollection of retrieved documents. To address the efficiency problem, we\npropose DC-BERT, a decoupled contextual encoding framework that has dual BERT\nmodels: an online BERT which encodes the question only once, and an offline\nBERT which pre-encodes all the documents and caches their encodings. On SQuAD\nOpen and Natural Questions Open datasets, DC-BERT achieves 10x speedup on\ndocument retrieval, while retaining most (about 98%) of the QA performance\ncompared to state-of-the-art approaches for open-domain question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12699v1",
            "date": "2020-02-28 13:20:09+00:00",
            "title": "Automatic Section Recognition in Obituaries",
            "authors": [
                "Valentino Sabbatino",
                "Laura Bostan",
                "Roman Klinger"
            ],
            "abstract": "Obituaries contain information about people's values across times and\ncultures, which makes them a useful resource for exploring cultural history.\nThey are typically structured similarly, with sections corresponding to\nPersonal Information, Biographical Sketch, Characteristics, Family, Gratitude,\nTribute, Funeral Information and Other aspects of the person. To make this\ninformation available for further studies, we propose a statistical model which\nrecognizes these sections. To achieve that, we collect a corpus of 20058\nEnglish obituaries from TheDaily Item, Remembering.CA and The London Free\nPress. The evaluation of our annotation guidelines with three annotators on\n1008 obituaries shows a substantial agreement of Fleiss k = 0.87. Formulated as\nan automatic segmentation task, a convolutional neural network outperforms\nbag-of-words and embedding-based BiLSTMs and BiLSTM-CRFs with a micro F1 =\n0.81.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12804v1",
            "date": "2020-02-28 15:28:49+00:00",
            "title": "UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training",
            "authors": [
                "Hangbo Bao",
                "Li Dong",
                "Furu Wei",
                "Wenhui Wang",
                "Nan Yang",
                "Xiaodong Liu",
                "Yu Wang",
                "Songhao Piao",
                "Jianfeng Gao",
                "Ming Zhou",
                "Hsiao-Wuen Hon"
            ],
            "abstract": "We propose to pre-train a unified language model for both autoencoding and\npartially autoregressive language modeling tasks using a novel training\nprocedure, referred to as a pseudo-masked language model (PMLM). Given an input\ntext with masked tokens, we rely on conventional masks to learn inter-relations\nbetween corrupted tokens and context via autoencoding, and pseudo masks to\nlearn intra-relations between masked spans via partially autoregressive\nmodeling. With well-designed position embeddings and self-attention masks, the\ncontext encodings are reused to avoid redundant computation. Moreover,\nconventional masks used for autoencoding provide global masking information, so\nthat all the position embeddings are accessible in partially autoregressive\nlanguage modeling. In addition, the two tasks pre-train a unified language\nmodel as a bidirectional encoder and a sequence-to-sequence decoder,\nrespectively. Our experiments show that the unified language models pre-trained\nusing PMLM achieve new state-of-the-art results on a wide range of natural\nlanguage understanding and generation tasks across several widely used\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12854v1",
            "date": "2020-02-28 16:30:33+00:00",
            "title": "Metaphoric Paraphrase Generation",
            "authors": [
                "Kevin Stowe",
                "Leonardo Ribeiro",
                "Iryna Gurevych"
            ],
            "abstract": "This work describes the task of metaphoric paraphrase generation, in which we\nare given a literal sentence and are charged with generating a metaphoric\nparaphrase. We propose two different models for this task: a lexical\nreplacement baseline and a novel sequence to sequence model, 'metaphor\nmasking', that generates free metaphoric paraphrases. We use crowdsourcing to\nevaluate our results, as well as developing an automatic metric for evaluating\nmetaphoric paraphrases. We show that while the lexical replacement baseline is\ncapable of producing accurate paraphrases, they often lack metaphoricity, while\nour metaphor masking model excels in generating metaphoric sentences while\nperforming nearly as well with regard to fluency and paraphrase quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00104v4",
            "date": "2020-02-28 22:59:24+00:00",
            "title": "AraBERT: Transformer-based Model for Arabic Language Understanding",
            "authors": [
                "Wissam Antoun",
                "Fady Baly",
                "Hazem Hajj"
            ],
            "abstract": "The Arabic language is a morphologically rich language with relatively few\nresources and a less explored syntax compared to English. Given these\nlimitations, Arabic Natural Language Processing (NLP) tasks like Sentiment\nAnalysis (SA), Named Entity Recognition (NER), and Question Answering (QA),\nhave proven to be very challenging to tackle. Recently, with the surge of\ntransformers based models, language-specific BERT based models have proven to\nbe very efficient at language understanding, provided they are pre-trained on a\nvery large corpus. Such models were able to set new standards and achieve\nstate-of-the-art results for most NLP tasks. In this paper, we pre-trained BERT\nspecifically for the Arabic language in the pursuit of achieving the same\nsuccess that BERT did for the English language. The performance of AraBERT is\ncompared to multilingual BERT from Google and other state-of-the-art\napproaches. The results showed that the newly developed AraBERT achieved\nstate-of-the-art performance on most tested Arabic NLP tasks. The pretrained\naraBERT models are publicly available on https://github.com/aub-mind/arabert\nhoping to encourage research and applications for Arabic NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00166v1",
            "date": "2020-02-29 03:09:55+00:00",
            "title": "Depth-Adaptive Graph Recurrent Network for Text Classification",
            "authors": [
                "Yijin Liu",
                "Fandong Meng",
                "Yufeng Chen",
                "Jinan Xu",
                "Jie Zhou"
            ],
            "abstract": "The Sentence-State LSTM (S-LSTM) is a powerful and high efficient graph\nrecurrent network, which views words as nodes and performs layer-wise recurrent\nsteps between them simultaneously. Despite its successes on text\nrepresentations, the S-LSTM still suffers from two drawbacks. Firstly, given a\nsentence, certain words are usually more ambiguous than others, and thus more\ncomputation steps need to be taken for these difficult words and vice versa.\nHowever, the S-LSTM takes fixed computation steps for all words, irrespective\nof their hardness. The secondary one comes from the lack of sequential\ninformation (e.g., word order) that is inherently important for natural\nlanguage. In this paper, we try to address these issues and propose a\ndepth-adaptive mechanism for the S-LSTM, which allows the model to learn how\nmany computational steps to conduct for different words as required. In\naddition, we integrate an extra RNN layer to inject sequential information,\nwhich also serves as an input feature for the decision of adaptive depths.\nResults on the classic text classification task (24 datasets in various sizes\nand domains) show that our model brings significant improvements against the\nconventional S-LSTM and other high-performance models (e.g., the Transformer),\nmeanwhile achieving a good accuracy-speed trade off.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00353v1",
            "date": "2020-02-29 22:15:15+00:00",
            "title": "Clinical Text Summarization with Syntax-Based Negation and Semantic Concept Identification",
            "authors": [
                "Wei-Hung Weng",
                "Yu-An Chung",
                "Schrasing Tong"
            ],
            "abstract": "In the era of clinical information explosion, a good strategy for clinical\ntext summarization is helpful to improve the clinical workflow. The ideal\nsummarization strategy can preserve important information in the informative\nbut less organized, ill-structured clinical narrative texts. Instead of using\npure statistical learning approaches, which are difficult to interpret and\nexplain, we utilized knowledge of computational linguistics with human\nexperts-curated biomedical knowledge base to achieve the interpretable and\nmeaningful clinical text summarization. Our research objective is to use the\nbiomedical ontology with semantic information, and take the advantage from the\nlanguage hierarchical structure, the constituency tree, in order to identify\nthe correct clinical concepts and the corresponding negation information, which\nis critical for summarizing clinical concepts from narrative text. We achieved\nthe clinically acceptable performance for both negation detection and concept\nidentification, and the clinical concepts with common negated patterns can be\nidentified and negated by the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00576v2",
            "date": "2020-03-01 20:32:51+00:00",
            "title": "StructSum: Summarization via Structured Representations",
            "authors": [
                "Vidhisha Balachandran",
                "Artidoro Pagnoni",
                "Jay Yoon Lee",
                "Dheeraj Rajagopal",
                "Jaime Carbonell",
                "Yulia Tsvetkov"
            ],
            "abstract": "Abstractive text summarization aims at compressing the information of a long\nsource document into a rephrased, condensed summary. Despite advances in\nmodeling techniques, abstractive summarization models still suffer from several\nkey challenges: (i) layout bias: they overfit to the style of training corpora;\n(ii) limited abstractiveness: they are optimized to copying n-grams from the\nsource rather than generating novel abstractive summaries; (iii) lack of\ntransparency: they are not interpretable. In this work, we propose a framework\nbased on document-level structure induction for summarization to address these\nchallenges. To this end, we propose incorporating latent and explicit\ndependencies across sentences in the source document into end-to-end\nsingle-document summarization models. Our framework complements standard\nencoder-decoder summarization models by augmenting them with rich\nstructure-aware document representations based on implicitly learned (latent)\nstructures and externally-derived linguistic (explicit) structures. We show\nthat our summarization framework, trained on the CNN/DM dataset, improves the\ncoverage of content in the source documents, generates more abstractive\nsummaries by generating more novel n-grams, and incorporates interpretable\nsentence-level structures, while performing on par with standard baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00639v2",
            "date": "2020-03-02 03:09:28+00:00",
            "title": "Learning from Easy to Complex: Adaptive Multi-curricula Learning for Neural Dialogue Generation",
            "authors": [
                "Hengyi Cai",
                "Hongshen Chen",
                "Cheng Zhang",
                "Yonghao Song",
                "Xiaofang Zhao",
                "Yangxi Li",
                "Dongsheng Duan",
                "Dawei Yin"
            ],
            "abstract": "Current state-of-the-art neural dialogue systems are mainly data-driven and\nare trained on human-generated responses. However, due to the subjectivity and\nopen-ended nature of human conversations, the complexity of training dialogues\nvaries greatly. The noise and uneven complexity of query-response pairs impede\nthe learning efficiency and effects of the neural dialogue generation models.\nWhat is more, so far, there are no unified dialogue complexity measurements,\nand the dialogue complexity embodies multiple aspects of\nattributes---specificity, repetitiveness, relevance, etc. Inspired by human\nbehaviors of learning to converse, where children learn from easy dialogues to\ncomplex ones and dynamically adjust their learning progress, in this paper, we\nfirst analyze five dialogue attributes to measure the dialogue complexity in\nmultiple perspectives on three publicly available corpora. Then, we propose an\nadaptive multi-curricula learning framework to schedule a committee of the\norganized curricula. The framework is established upon the reinforcement\nlearning paradigm, which automatically chooses different curricula at the\nevolving learning process according to the learning status of the neural\ndialogue generation model. Extensive experiments conducted on five\nstate-of-the-art models demonstrate its learning efficiency and effectiveness\nwith respect to 13 automatic evaluation metrics and human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00674v1",
            "date": "2020-03-02 05:40:57+00:00",
            "title": "Style Example-Guided Text Generation using Generative Adversarial Transformers",
            "authors": [
                "Kuo-Hao Zeng",
                "Mohammad Shoeybi",
                "Ming-Yu Liu"
            ],
            "abstract": "We introduce a language generative model framework for generating a styled\nparagraph based on a context sentence and a style reference example. The\nframework consists of a style encoder and a texts decoder. The style encoder\nextracts a style code from the reference example, and the text decoder\ngenerates texts based on the style code and the context. We propose a novel\nobjective function to train our framework. We also investigate different\nnetwork design choices. We conduct extensive experimental validation with\ncomparison to strong baselines to validate the effectiveness of the proposed\nframework using a newly collected dataset with diverse text styles. Both code\nand dataset will be released upon publication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01018v1",
            "date": "2020-03-02 16:50:33+00:00",
            "title": "Identification of primary and collateral tracks in stuttered speech",
            "authors": [
                "Rachid Riad",
                "Anne-Catherine Bachoud-L\u00e9vi",
                "Frank Rudzicz",
                "Emmanuel Dupoux"
            ],
            "abstract": "Disfluent speech has been previously addressed from two main perspectives:\nthe clinical perspective focusing on diagnostic, and the Natural Language\nProcessing (NLP) perspective aiming at modeling these events and detect them\nfor downstream tasks. In addition, previous works often used different metrics\ndepending on whether the input features are text or speech, making it difficult\nto compare the different contributions. Here, we introduce a new evaluation\nframework for disfluency detection inspired by the clinical and NLP perspective\ntogether with the theory of performance from \\cite{clark1996using} which\ndistinguishes between primary and collateral tracks. We introduce a novel\nforced-aligned disfluency dataset from a corpus of semi-directed interviews,\nand present baseline results directly comparing the performance of text-based\nfeatures (word and span information) and speech-based (acoustic-prosodic\ninformation). Finally, we introduce new audio features inspired by the\nword-based span features. We show experimentally that using these features\noutperformed the baselines for speech-based predictions on the present dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01305v1",
            "date": "2020-03-03 02:56:36+00:00",
            "title": "Transfer Learning for Context-Aware Spoken Language Understanding",
            "authors": [
                "Qian Chen",
                "Zhu Zhuo",
                "Wen Wang",
                "Qiuyun Xu"
            ],
            "abstract": "Spoken language understanding (SLU) is a key component of task-oriented\ndialogue systems. SLU parses natural language user utterances into semantic\nframes. Previous work has shown that incorporating context information\nsignificantly improves SLU performance for multi-turn dialogues. However,\ncollecting a large-scale human-labeled multi-turn dialogue corpus for the\ntarget domains is complex and costly. To reduce dependency on the collection\nand annotation effort, we propose a Context Encoding Language Transformer\n(CELT) model facilitating exploiting various context information for SLU. We\nexplore different transfer learning approaches to reduce dependency on data\ncollection and annotation. In addition to unsupervised pre-training using\nlarge-scale general purpose unlabeled corpora, such as Wikipedia, we explore\nunsupervised and supervised adaptive training approaches for transfer learning\nto benefit from other in-domain and out-of-domain dialogue corpora.\nExperimental results demonstrate that the proposed model with the proposed\ntransfer learning approaches achieves significant improvement on the SLU\nperformance over state-of-the-art models on two large-scale single-turn\ndialogue benchmarks and one large-scale multi-turn dialogue benchmark.",
            "journal ref": "2019 IEEE Automatic Speech Recognition and Understanding Workshop\n  (ASRU), SG, Singapore, 2019, pp. 779-786",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01343v1",
            "date": "2020-03-03 05:32:09+00:00",
            "title": "Improving Candidate Generation for Low-resource Cross-lingual Entity Linking",
            "authors": [
                "Shuyan Zhou",
                "Shruti Rijhwani",
                "John Wieting",
                "Jaime Carbonell",
                "Graham Neubig"
            ],
            "abstract": "Cross-lingual entity linking (XEL) is the task of finding referents in a\ntarget-language knowledge base (KB) for mentions extracted from source-language\ntexts. The first step of (X)EL is candidate generation, which retrieves a list\nof plausible candidate entities from the target-language KB for each mention.\nApproaches based on resources from Wikipedia have proven successful in the\nrealm of relatively high-resource languages (HRL), but these do not extend well\nto low-resource languages (LRL) with few, if any, Wikipedia pages. Recently,\ntransfer learning methods have been shown to reduce the demand for resources in\nthe LRL by utilizing resources in closely-related languages, but the\nperformance still lags far behind their high-resource counterparts. In this\npaper, we first assess the problems faced by current entity candidate\ngeneration methods for low-resource XEL, then propose three improvements that\n(1) reduce the disconnect between entity mentions and KB entries, and (2)\nimprove the robustness of the model to low-resource scenarios. The methods are\nsimple, but effective: we experiment with our approach on seven XEL datasets\nand find that they yield an average gain of 16.9% in Top-30 gold candidate\nrecall, compared to state-of-the-art baselines. Our improved model also yields\nan average gain of 7.9% in in-KB accuracy of end-to-end XEL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01355v2",
            "date": "2020-03-03 06:39:27+00:00",
            "title": "CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model",
            "authors": [
                "Liang Xu",
                "Xuanwei Zhang",
                "Qianqian Dong"
            ],
            "abstract": "In this paper, we introduce the Chinese corpus from CLUE organization,\nCLUECorpus2020, a large-scale corpus that can be used directly for\nself-supervised learning such as pre-training of a language model, or language\ngeneration. It has 100G raw corpus with 35 billion Chinese characters, which is\nretrieved from Common Crawl. To better understand this corpus, we conduct\nlanguage understanding experiments on both small and large scale, and results\nshow that the models trained on this corpus can achieve excellent performance\non Chinese. We release a new Chinese vocabulary with a size of 8K, which is\nonly one-third of the vocabulary size used in Chinese Bert released by Google.\nIt saves computational cost and memory while works as good as original\nvocabulary. We also release both large and tiny versions of the pre-trained\nmodel on this corpus. The former achieves the state-of-the-art result, and the\nlatter retains most precision while accelerating training and prediction speed\nfor eight times compared to Bert-base. To facilitate future work on\nself-supervised learning on Chinese, we release our dataset, new vocabulary,\ncodes, and pre-trained models on Github.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01371v3",
            "date": "2020-03-03 07:34:24+00:00",
            "title": "Meta-Embeddings Based On Self-Attention",
            "authors": [
                "Qichen Li",
                "Yuanqing Lin",
                "Luofeng Zhou",
                "Jian Li"
            ],
            "abstract": "Creating meta-embeddings for better performance in language modelling has\nreceived attention lately, and methods based on concatenation or merely\ncalculating the arithmetic mean of more than one separately trained embeddings\nto perform meta-embeddings have shown to be beneficial. In this paper, we\ndevise a new meta-embedding model based on the self-attention mechanism, namely\nthe Duo. With less than 0.4M parameters, the Duo mechanism achieves\nstate-of-the-art accuracy in text classification tasks such as 20NG.\nAdditionally, we propose a new meta-embedding sequece-to-sequence model for\nmachine translation, which to the best of our knowledge, is the first machine\ntranslation model based on more than one word-embedding. Furthermore, it has\nturned out that our model outperform the Transformer not only in terms of\nachieving a better result, but also a faster convergence on recognized\nbenchmarks, such as the WMT 2014 English-to-French translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01472v2",
            "date": "2020-03-03 12:11:12+00:00",
            "title": "Seshat: A tool for managing and verifying annotation campaigns of audio data",
            "authors": [
                "Hadrien Titeux",
                "Rachid Riad",
                "Xuan-Nga Cao",
                "Nicolas Hamilakis",
                "Kris Madden",
                "Alejandrina Cristia",
                "Anne-Catherine Bachoud-L\u00e9vi",
                "Emmanuel Dupoux"
            ],
            "abstract": "We introduce Seshat, a new, simple and open-source software to efficiently\nmanage annotations of speech corpora. The Seshat software allows users to\neasily customise and manage annotations of large audio corpora while ensuring\ncompliance with the formatting and naming conventions of the annotated output\nfiles. In addition, it includes procedures for checking the content of\nannotations following specific rules that can be implemented in personalised\nparsers. Finally, we propose a double-annotation mode, for which Seshat\ncomputes automatically an associated inter-annotator agreement with the\n$\\gamma$ measure taking into account the categorisation and segmentation\ndiscrepancies.",
            "journal ref": "LREC 2020 - 12th Language Resources and Evaluation Conference, May\n  2020, Marseille, France. pp.6976-6982",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01680v2",
            "date": "2020-03-03 18:07:42+00:00",
            "title": "Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation",
            "authors": [
                "Igor Shalyminov",
                "Alessandro Sordoni",
                "Adam Atkinson",
                "Hannes Schulz"
            ],
            "abstract": "Domain adaptation has recently become a key problem in dialogue systems\nresearch. Deep learning, while being the preferred technique for modeling such\nsystems, works best given massive training data. However, in the real-world\nscenario, such resources aren't available for every new domain, so the ability\nto train with a few dialogue examples can be considered essential. Pre-training\non large data sources and adapting to the target data has become the standard\nmethod for few-shot problems within the deep learning framework. In this paper,\nwe present the winning entry at the fast domain adaptation task of DSTC8, a\nhybrid generative-retrieval model based on GPT-2 fine-tuned to the multi-domain\nMetaLWOz dataset. Robust and diverse in response generation, our model uses\nretrieval logic as a fallback, being SoTA on MetaLWOz in human evaluation (>4%\nimprovement over the 2nd place system) and attaining competitive generalization\nperformance in adaptation to the unseen MultiWOZ dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01821v2",
            "date": "2020-03-03 22:44:10+00:00",
            "title": "HyperEmbed: Tradeoffs Between Resources and Performance in NLP Tasks with Hyperdimensional Computing enabled Embedding of n-gram Statistics",
            "authors": [
                "Pedro Alonso",
                "Kumar Shridhar",
                "Denis Kleyko",
                "Evgeny Osipov",
                "Marcus Liwicki"
            ],
            "abstract": "Recent advances in Deep Learning have led to a significant performance\nincrease on several NLP tasks, however, the models become more and more\ncomputationally demanding. Therefore, this paper tackles the domain of\ncomputationally efficient algorithms for NLP tasks. In particular, it\ninvestigates distributed representations of n-gram statistics of texts. The\nrepresentations are formed using hyperdimensional computing enabled embedding.\nThese representations then serve as features, which are used as input to\nstandard classifiers. We investigate the applicability of the embedding on one\nlarge and three small standard datasets for classification tasks using nine\nclassifiers. The embedding achieved on par F1 scores while decreasing the time\nand memory requirements by several times compared to the conventional n-gram\nstatistics, e.g., for one of the classifiers on a small dataset, the memory\nreduction was 6.18 times; while train and test speed-ups were 4.62 and 3.84\ntimes, respectively. For many classifiers on the large dataset, memory\nreduction was ca. 100 times and train and test speed-ups were over 100 times.\nImportantly, the usage of distributed representations formed via\nhyperdimensional computing allows dissecting strict dependency between the\ndimensionality of the representation and n-gram size, thus, opening a room for\ntradeoffs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02020v1",
            "date": "2020-03-04 11:57:53+00:00",
            "title": "Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network",
            "authors": [
                "Shaoxiong Feng",
                "Hongshen Chen",
                "Kan Li",
                "Dawei Yin"
            ],
            "abstract": "Neural conversational models learn to generate responses by taking into\naccount the dialog history. These models are typically optimized over the\nquery-response pairs with a maximum likelihood estimation objective. However,\nthe query-response tuples are naturally loosely coupled, and there exist\nmultiple responses that can respond to a given query, which leads the\nconversational model learning burdensome. Besides, the general dull response\nproblem is even worsened when the model is confronted with meaningless response\ntraining instances. Intuitively, a high-quality response not only responds to\nthe given query but also links up to the future conversations, in this paper,\nwe leverage the query-response-future turn triples to induce the generated\nresponses that consider both the given context and the future conversations. To\nfacilitate the modeling of these triples, we further propose a novel\nencoder-decoder based generative adversarial learning framework, Posterior\nGenerative Adversarial Network (Posterior-GAN), which consists of a forward and\na backward generative discriminator to cooperatively encourage the generated\nresponse to be informative and coherent by two complementary assessment\nperspectives. Experimental results demonstrate that our method effectively\nboosts the informativeness and coherence of the generated response on both\nautomatic and human evaluation, which verifies the advantages of considering\ntwo assessment perspectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02126v1",
            "date": "2020-03-03 04:36:33+00:00",
            "title": "Sequential Neural Networks for Noetic End-to-End Response Selection",
            "authors": [
                "Qian Chen",
                "Wen Wang"
            ],
            "abstract": "The noetic end-to-end response selection challenge as one track in the 7th\nDialog System Technology Challenges (DSTC7) aims to push the state of the art\nof utterance classification for real world goal-oriented dialog systems, for\nwhich participants need to select the correct next utterances from a set of\ncandidates for the multi-turn context. This paper presents our systems that are\nranked top 1 on both datasets under this challenge, one focused and small\n(Advising) and the other more diverse and large (Ubuntu). Previous\nstate-of-the-art models use hierarchy-based (utterance-level and token-level)\nneural networks to explicitly model the interactions among different turns'\nutterances for context modeling. In this paper, we investigate a sequential\nmatching model based only on chain sequence for multi-turn response selection.\nOur results demonstrate that the potentials of sequential matching approaches\nhave not yet been fully exploited in the past for multi-turn response\nselection. In addition to ranking top 1 in the challenge, the proposed model\noutperforms all previous models, including state-of-the-art hierarchy-based\nmodels, on two large-scale public multi-turn response selection benchmark\ndatasets.",
            "journal ref": "Computer Speech & Language, Volume 62, July 2020, 101072",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02197v2",
            "date": "2020-03-04 17:10:10+00:00",
            "title": "Evaluating Low-Resource Machine Translation between Chinese and Vietnamese with Back-Translation",
            "authors": [
                "Hongzheng Li",
                "Heyan Huang"
            ],
            "abstract": "Back translation (BT) has been widely used and become one of standard\ntechniques for data augmentation in Neural Machine Translation (NMT), BT has\nproven to be helpful for improving the performance of translation effectively,\nespecially for low-resource scenarios. While most works related to BT mainly\nfocus on European languages, few of them study languages in other areas around\nthe world. In this paper, we investigate the impacts of BT on Asia language\ntranslations between the extremely low-resource Chinese and Vietnamese language\npair. We evaluate and compare the effects of different sizes of synthetic data\non both NMT and Statistical Machine Translation (SMT) models for Chinese to\nVietnamese and Vietnamese to Chinese, with character-based and word-based\nsettings. Some conclusions from previous works are partially confirmed and we\nalso draw some other interesting findings and conclusions, which are beneficial\nto understand BT further.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02244v2",
            "date": "2020-03-04 18:31:32+00:00",
            "title": "Unsupervised Adversarial Domain Adaptation for Implicit Discourse Relation Classification",
            "authors": [
                "Hsin-Ping Huang",
                "Junyi Jessy Li"
            ],
            "abstract": "Implicit discourse relations are not only more challenging to classify, but\nalso to annotate, than their explicit counterparts. We tackle situations where\ntraining data for implicit relations are lacking, and exploit domain adaptation\nfrom explicit relations (Ji et al., 2015). We present an unsupervised\nadversarial domain adaptive network equipped with a reconstruction component.\nOur system outperforms prior works and other adversarial benchmarks for\nunsupervised domain adaptation. Additionally, we extend our system to take\nadvantage of labeled data if some are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02249v2",
            "date": "2020-03-04 18:41:10+00:00",
            "title": "jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models",
            "authors": [
                "Yada Pruksachatkun",
                "Phil Yeres",
                "Haokun Liu",
                "Jason Phang",
                "Phu Mon Htut",
                "Alex Wang",
                "Ian Tenney",
                "Samuel R. Bowman"
            ],
            "abstract": "We introduce jiant, an open source toolkit for conducting multitask and\ntransfer learning experiments on English NLU tasks. jiant enables modular and\nconfiguration-driven experimentation with state-of-the-art models and\nimplements a broad set of tasks for probing, transfer learning, and multitask\ntraining experiments. jiant implements over 50 NLU tasks, including all GLUE\nand SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published\nperformance on a variety of tasks and models, including BERT and RoBERTa. jiant\nis available at https://jiant.info.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02349v1",
            "date": "2020-03-04 22:12:18+00:00",
            "title": "A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection",
            "authors": [
                "Daniele Bonadiman",
                "Alessandro Moschitti"
            ],
            "abstract": "An essential task of most Question Answering (QA) systems is to re-rank the\nset of answer candidates, i.e., Answer Sentence Selection (A2S). These\ncandidates are typically sentences either extracted from one or more documents\npreserving their natural order or retrieved by a search engine. Most\nstate-of-the-art approaches to the task use huge neural models, such as BERT,\nor complex attentive architectures. In this paper, we argue that by exploiting\nthe intrinsic structure of the original rank together with an effective\nword-relatedness encoder, we can achieve competitive results with respect to\nthe state of the art while retaining high efficiency. Our model takes 9.5\nseconds to train on the WikiQA dataset, i.e., very fast in comparison with the\n$\\sim 18$ minutes required by a standard BERT-base fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02356v2",
            "date": "2020-03-04 22:45:22+00:00",
            "title": "Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout",
            "authors": [
                "Filip Grali\u0144ski",
                "Tomasz Stanis\u0142awek",
                "Anna Wr\u00f3blewska",
                "Dawid Lipi\u0144ski",
                "Agnieszka Kaliska",
                "Paulina Rosalska",
                "Bartosz Topolski",
                "Przemys\u0142aw Biecek"
            ],
            "abstract": "State-of-the-art solutions for Natural Language Processing (NLP) are able to\ncapture a broad range of contexts, like the sentence-level context or\ndocument-level context for short documents. But these solutions are still\nstruggling when it comes to longer, real-world documents with the information\nencoded in the spatial structure of the document, such as page elements like\ntables, forms, headers, openings or footers; complex page layout or presence of\nmultiple pages.\n  To encourage progress on deeper and more complex Information Extraction (IE)\nwe introduce a new task (named Kleister) with two new datasets. Utilizing both\ntextual and structural layout features, an NLP system must find the most\nimportant information, about various types of entities, in long formal\ndocuments. We propose Pipeline method as a text-only baseline with different\nNamed Entity Recognition architectures (Flair, BERT, RoBERTa). Moreover, we\nchecked the most popular PDF processing tools for text extraction (pdf2djvu,\nTesseract and Textract) in order to analyze behavior of IE system in presence\nof errors introduced by these tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02739v4",
            "date": "2020-03-05 16:07:32+00:00",
            "title": "Zero-Shot Cross-Lingual Transfer with Meta Learning",
            "authors": [
                "Farhad Nooralahzadeh",
                "Giannis Bekoulis",
                "Johannes Bjerva",
                "Isabelle Augenstein"
            ],
            "abstract": "Learning what to share between tasks has been a topic of great importance\nrecently, as strategic sharing of knowledge has been shown to improve\ndownstream task performance. This is particularly important for multilingual\napplications, as most languages in the world are under-resourced. Here, we\nconsider the setting of training models on multiple different languages at the\nsame time, when little or no data is available for languages other than\nEnglish. We show that this challenging setup can be approached using\nmeta-learning, where, in addition to training a source language model, another\nmodel learns to select which training instances are the most beneficial to the\nfirst. We experiment using standard supervised, zero-shot cross-lingual, as\nwell as few-shot cross-lingual settings for different natural language\nunderstanding tasks (natural language inference, question answering). Our\nextensive experimental setup demonstrates the consistent effectiveness of\nmeta-learning for a total of 15 languages. We improve upon the state-of-the-art\nfor zero-shot and few-shot NLI (on MultiNLI and XNLI) and QA (on the MLQA\ndataset). A comprehensive error analysis indicates that the correlation of\ntypological features between languages can partly explain when parameter\nsharing learned via meta-learning is beneficial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02877v3",
            "date": "2020-03-05 19:14:33+00:00",
            "title": "Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation",
            "authors": [
                "Mitchell A. Gordon",
                "Kevin Duh"
            ],
            "abstract": "We explore best practices for training small, memory efficient machine\ntranslation models with sequence-level knowledge distillation in the domain\nadaptation setting. While both domain adaptation and knowledge distillation are\nwidely-used, their interaction remains little understood. Our large-scale\nempirical results in machine translation (on three language pairs with three\ndomains each) suggest distilling twice for best performance: once using\ngeneral-domain data and again using in-domain data with an adapted teacher.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02912v1",
            "date": "2020-03-05 20:42:51+00:00",
            "title": "What the [MASK]? Making Sense of Language-Specific BERT Models",
            "authors": [
                "Debora Nozza",
                "Federico Bianchi",
                "Dirk Hovy"
            ],
            "abstract": "Recently, Natural Language Processing (NLP) has witnessed an impressive\nprogress in many areas, due to the advent of novel, pretrained contextual\nrepresentation models. In particular, Devlin et al. (2019) proposed a model,\ncalled BERT (Bidirectional Encoder Representations from Transformers), which\nenables researchers to obtain state-of-the art performance on numerous NLP\ntasks by fine-tuning the representations on their data set and task, without\nthe need for developing and training highly-specific architectures. The authors\nalso released multilingual BERT (mBERT), a model trained on a corpus of 104\nlanguages, which can serve as a universal language model. This model obtained\nimpressive results on a zero-shot cross-lingual natural inference task. Driven\nby the potential of BERT models, the NLP community has started to investigate\nand generate an abundant number of BERT models that are trained on a particular\nlanguage, and tested on a specific data domain and task. This allows us to\nevaluate the true potential of mBERT as a universal language model, by\ncomparing it to the performance of these more specific models. This paper\npresents the current state of the art in language-specific BERT models,\nproviding an overall picture with respect to different dimensions (i.e.\narchitectures, data domains, and tasks). Our aim is to provide an immediate and\nstraightforward overview of the commonalities and differences between\nLanguage-Specific (language-specific) BERT models and mBERT. We also provide an\ninteractive and constantly updated website that can be used to explore the\ninformation we have collected, at https://bertlang.unibocconi.it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02931v1",
            "date": "2020-03-05 21:25:00+00:00",
            "title": "Neural Cross-Lingual Transfer and Limited Annotated Data for Named Entity Recognition in Danish",
            "authors": [
                "Barbara Plank"
            ],
            "abstract": "Named Entity Recognition (NER) has greatly advanced by the introduction of\ndeep neural architectures. However, the success of these methods depends on\nlarge amounts of training data. The scarcity of publicly-available\nhuman-labeled datasets has resulted in limited evaluation of existing NER\nsystems, as is the case for Danish. This paper studies the effectiveness of\ncross-lingual transfer for Danish, evaluates its complementarity to limited\ngold data, and sheds light on performance of Danish NER.",
            "journal ref": "NoDaLiDa, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02955v1",
            "date": "2020-03-05 22:55:45+00:00",
            "title": "Automatic Compilation of Resources for Academic Writing and Evaluating with Informal Word Identification and Paraphrasing System",
            "authors": [
                "Seid Muhie Yimam",
                "Gopalakrishnan Venkatesh",
                "John Sie Yuen Lee",
                "Chris Biemann"
            ],
            "abstract": "We present the first approach to automatically building resources for\nacademic writing. The aim is to build a writing aid system that automatically\nedits a text so that it better adheres to the academic style of writing. On top\nof existing academic resources, such as the Corpus of Contemporary American\nEnglish (COCA) academic Word List, the New Academic Word List, and the Academic\nCollocation List, we also explore how to dynamically build such resources that\nwould be used to automatically identify informal or non-academic words or\nphrases. The resources are compiled using different generic approaches that can\nbe extended for different domains and languages. We describe the evaluation of\nresources with a system implementation. The system consists of an informal word\nidentification (IWI), academic candidate paraphrase generation, and paraphrase\nranking components. To generate candidates and rank them in context, we have\nused the PPDB and WordNet paraphrase resources. We use the Concepts in Context\n(CoInCO) \"All-Words\" lexical substitution dataset both for the informal word\nidentification and paraphrase generation experiments. Our informal word\nidentification component achieves an F-1 score of 82%, significantly\noutperforming a stratified classifier baseline. The main contribution of this\nwork is a domain-independent methodology to build targeted resources for\nwriting aids.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03014v2",
            "date": "2020-03-06 03:02:12+00:00",
            "title": "A Framework for the Computational Linguistic Analysis of Dehumanization",
            "authors": [
                "Julia Mendelsohn",
                "Yulia Tsvetkov",
                "Dan Jurafsky"
            ],
            "abstract": "Dehumanization is a pernicious psychological process that often leads to\nextreme intergroup bias, hate speech, and violence aimed at targeted social\ngroups. Despite these serious consequences and the wealth of available data,\ndehumanization has not yet been computationally studied on a large scale.\nDrawing upon social psychology research, we create a computational linguistic\nframework for analyzing dehumanizing language by identifying linguistic\ncorrelates of salient components of dehumanization. We then apply this\nframework to analyze discussions of LGBTQ people in the New York Times from\n1986 to 2015. Overall, we find increasingly humanizing descriptions of LGBTQ\npeople over time. However, we find that the label homosexual has emerged to be\nmuch more strongly associated with dehumanizing attitudes than other labels,\nsuch as gay. Our proposed techniques highlight processes of linguistic\nvariation and change in discourses surrounding marginalized groups.\nFurthermore, the ability to analyze dehumanizing language at a large scale has\nimplications for automatically detecting and understanding media bias as well\nas abusive language online.",
            "journal ref": "Frontiers in Artificial Intelligence 3 (2020):55",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03106v2",
            "date": "2020-03-06 09:46:51+00:00",
            "title": "Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT",
            "authors": [
                "Aitor Garc\u00eda-Pablos",
                "Naiara Perez",
                "Montse Cuadros"
            ],
            "abstract": "Massive digital data processing provides a wide range of opportunities and\nbenefits, but at the cost of endangering personal data privacy. Anonymisation\nconsists in removing or replacing sensitive information from data, enabling its\nexploitation for different purposes while preserving the privacy of\nindividuals. Over the years, a lot of automatic anonymisation systems have been\nproposed; however, depending on the type of data, the target language or the\navailability of training documents, the task remains challenging still. The\nemergence of novel deep-learning models during the last two years has brought\nlarge improvements to the state of the art in the field of Natural Language\nProcessing. These advancements have been most noticeably led by BERT, a model\nproposed by Google in 2018, and the shared language models pre-trained on\nmillions of documents. In this paper, we use a BERT-based sequence labelling\nmodel to conduct a series of anonymisation experiments on several clinical\ndatasets in Spanish. We also compare BERT to other algorithms. The experiments\nshow that a simple BERT-based model with general-domain pre-training obtains\nhighly competitive results without any domain specific feature engineering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03131v1",
            "date": "2020-03-06 10:58:59+00:00",
            "title": "Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning",
            "authors": [
                "Stig-Arne Gr\u00f6nroos",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "Data-driven segmentation of words into subword units has been used in various\nnatural language processing applications such as automatic speech recognition\nand statistical machine translation for almost 20 years. Recently it has became\nmore widely adopted, as models based on deep neural networks often benefit from\nsubword units even for morphologically simpler languages. In this paper, we\ndiscuss and compare training algorithms for a unigram subword model, based on\nthe Expectation Maximization algorithm and lexicon pruning. Using English,\nFinnish, North Sami, and Turkish data sets, we show that this approach is able\nto find better solutions to the optimization problem defined by the Morfessor\nBaseline model than its original recursive training algorithm. The improved\noptimization also leads to higher morphological segmentation accuracy when\ncompared to a linguistic gold standard. We publish implementations of the new\nalgorithms in the widely-used Morfessor software package.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03235v1",
            "date": "2020-03-06 14:25:50+00:00",
            "title": "Practical Annotation Strategies for Question Answering Datasets",
            "authors": [
                "Bernhard Kratzwald",
                "Xiang Yue",
                "Huan Sun",
                "Stefan Feuerriegel"
            ],
            "abstract": "Annotating datasets for question answering (QA) tasks is very costly, as it\nrequires intensive manual labor and often domain-specific knowledge. Yet\nstrategies for annotating QA datasets in a cost-effective manner are scarce. To\nprovide a remedy for practitioners, our objective is to develop heuristic rules\nfor annotating a subset of questions, so that the annotation cost is reduced\nwhile maintaining both in- and out-of-domain performance. For this, we conduct\na large-scale analysis in order to derive practical recommendations. First, we\ndemonstrate experimentally that more training samples contribute often only to\na higher in-domain test-set performance, but do not help the model in\ngeneralizing to unseen datasets. Second, we develop a model-guided annotation\nstrategy: it makes a recommendation with regard to which subset of samples\nshould be annotated. Its effectiveness is demonstrated in a case study based on\ndomain customization of QA to a clinical setting. Here, remarkably, annotating\na stratified subset with only 1.2% of the original training set achieves 97.7%\nof the performance as if the complete dataset was annotated. Hence, the\nlabeling effort can be reduced immensely. Altogether, our work fulfills a\ndemand in practice when labeling budgets are limited and where thus\nrecommendations are needed for annotating QA datasets more cost-effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03444v3",
            "date": "2020-03-06 21:19:44+00:00",
            "title": "NYTWIT: A Dataset of Novel Words in the New York Times",
            "authors": [
                "Yuval Pinter",
                "Cassandra L. Jacobs",
                "Max Bittker"
            ],
            "abstract": "We present the New York Times Word Innovation Types dataset, or NYTWIT, a\ncollection of over 2,500 novel English words published in the New York Times\nbetween November 2017 and March 2019, manually annotated for their class of\nnovelty (such as lexical derivation, dialectal variation, blending, or\ncompounding). We present baseline results for both uncontextual and contextual\nprediction of novelty class, showing that there is room for improvement even\nfor state-of-the-art NLP systems. We hope this resource will prove useful for\nlinguists and NLP practitioners by providing a real-world environment of novel\nword appearance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03484v2",
            "date": "2020-03-07 01:52:19+00:00",
            "title": "Synthetic Error Dataset Generation Mimicking Bengali Writing Pattern",
            "authors": [
                "Md. Habibur Rahman Sifat",
                "Chowdhury Rafeed Rahman",
                "Mohammad Rafsan",
                "Md. Hasibur Rahman"
            ],
            "abstract": "While writing Bengali using English keyboard, users often make spelling\nmistakes. The accuracy of any Bengali spell checker or paragraph correction\nmodule largely depends on the kind of error dataset it is based on. Manual\ngeneration of such error dataset is a cumbersome process. In this research, We\npresent an algorithm for automatic misspelled Bengali word generation from\ncorrect word through analyzing Bengali writing pattern using QWERTY layout\nEnglish keyboard. As part of our analysis, we have formed a list of most\ncommonly used Bengali words, phonetically similar replaceable clusters,\nfrequently mispressed replaceable clusters, frequently mispressed insertion\nprone clusters and some rules for Juktakkhar (constant letter clusters)\nhandling while generating errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03507v1",
            "date": "2020-03-07 03:36:47+00:00",
            "title": "ECSP: A New Task for Emotion-Cause Span-Pair Extraction and Classification",
            "authors": [
                "Hongliang Bi",
                "Pengyuan Liu"
            ],
            "abstract": "Emotion cause analysis such as emotion cause extraction (ECE) and\nemotion-cause pair extraction (ECPE) have gradually attracted the attention of\nmany researchers. However, there are still two shortcomings in the existing\nresearch: 1) In most cases, emotion expression and cause are not the whole\nclause, but the span in the clause, so extracting the clause-pair rather than\nthe span-pair greatly limits its applications in real-world scenarios; 2) It is\nnot enough to extract the emotion expression clause without identifying the\nemotion categories, the presence of emotion clause does not necessarily convey\nemotional information explicitly due to different possible causes. In this\npaper, we propose a new task: Emotion-Cause Span-Pair extraction and\nclassification (ECSP), which aims to extract the potential span-pair of emotion\nand corresponding causes in a document, and make emotion classification for\neach pair. In the new ECSP task, ECE and ECPE can be regarded as two special\ncases at the clause-level. We propose a span-based extract-then-classify (ETC)\nmodel, where emotion and cause are directly extracted and paired from the\ndocument under the supervision of target span boundaries, and corresponding\ncategories are then classified using their pair representations and localized\ncontext. Experiments show that our proposed ETC model outperforms the SOTA\nmodel of ECE and ECPE task respectively and gets a fair-enough results on ECSP\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03556v2",
            "date": "2020-03-07 11:21:03+00:00",
            "title": "Automatic Recognition of the General-Purpose Communicative Functions defined by the ISO 24617-2 Standard for Dialog Act Annotation",
            "authors": [
                "Eug\u00e9nio Ribeiro",
                "Ricardo Ribeiro",
                "David Martins de Matos"
            ],
            "abstract": "ISO 24617-2, the standard for dialog act annotation, defines a hierarchically\norganized set of general-purpose communicative functions. The automatic\nrecognition of these functions, although practically unexplored, is relevant\nfor a dialog system, since they provide cues regarding the intention behind the\nsegments and how they should be interpreted. We explore the recognition of\ngeneral-purpose communicative functions in the DialogBank, which is a reference\nset of dialogs annotated according to this standard. To do so, we propose\nadaptations of existing approaches to flat dialog act recognition that allow\nthem to deal with the hierarchical classification problem. More specifically,\nwe propose the use of a hierarchical network with cascading outputs and maximum\na posteriori path estimation to predict the communicative function at each\nlevel of the hierarchy, preserve the dependencies between the functions in the\npath, and decide at which level to stop. Furthermore, since the amount of\ndialogs in the DialogBank is reduced, we rely on transfer learning processes to\nreduce overfitting and improve performance. The results of our experiments show\nthat the hierarchical approach outperforms a flat one and that each of its\ncomponents plays an important role towards the recognition of general-purpose\ncommunicative functions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03654v1",
            "date": "2020-03-07 20:21:50+00:00",
            "title": "Discovering linguistic (ir)regularities in word embeddings through max-margin separating hyperplanes",
            "authors": [
                "Noel Kennedy",
                "Imogen Schofield",
                "Dave C. Brodbelt",
                "David B. Church",
                "Dan G. O'Neill"
            ],
            "abstract": "We experiment with new methods for learning how related words are positioned\nrelative to each other in word embedding spaces. Previous approaches learned\nconstant vector offsets: vectors that point from source tokens to target tokens\nwith an assumption that these offsets were parallel to each other. We show that\nthe offsets between related tokens are closer to orthogonal than parallel, and\nthat they have low cosine similarities. We proceed by making a different\nassumption; target tokens are linearly separable from source and un-labeled\ntokens. We show that a max-margin hyperplane can separate target tokens and\nthat vectors orthogonal to this hyperplane represent the relationship between\nsource and targets. We find that this representation of the relationship\nobtains the best results in dis-covering linguistic regularities. We experiment\nwith vector space models trained by a variety of algorithms (Word2vec:\nCBOW/skip-gram, fastText, or GloVe), and various word context choices such as\nlinear word-order, syntax dependency grammars, and with and without knowledge\nof word position. These experiments show that our model, SVMCos, is robust to a\nrange of experimental choices when training word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03666v2",
            "date": "2020-03-07 21:21:29+00:00",
            "title": "Multi-task Learning Based Neural Bridging Reference Resolution",
            "authors": [
                "Juntao Yu",
                "Massimo Poesio"
            ],
            "abstract": "We propose a multi task learning-based neural model for resolving bridging\nreferences tackling two key challenges. The first challenge is the lack of\nlarge corpora annotated with bridging references. To address this, we use\nmulti-task learning to help bridging reference resolution with coreference\nresolution. We show that substantial improvements of up to 8 p.p. can be\nachieved on full bridging resolution with this architecture. The second\nchallenge is the different definitions of bridging used in different corpora,\nmeaning that hand-coded systems or systems using special features designed for\none corpus do not work well with other corpora. Our neural model only uses a\nsmall number of corpus independent features, thus can be applied to different\ncorpora. Evaluations with very different bridging corpora (ARRAU, ISNOTES,\nBASHI and SCICORP) suggest that our architecture works equally well on all\ncorpora, and achieves the SoTA results on full bridging resolution for all\ncorpora, outperforming the best reported results by up to 36.3 p.p..",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03716v1",
            "date": "2020-03-08 04:36:04+00:00",
            "title": "Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead Approach",
            "authors": [
                "Yu-Siang Wang",
                "Yen-Ling Kuo",
                "Boris Katz"
            ],
            "abstract": "We demonstrate how we can practically incorporate multi-step future\ninformation into a decoder of maximum likelihood sequence models. We propose a\n\"k-step look-ahead\" module to consider the likelihood information of a rollout\nup to k steps. Unlike other approaches that need to train another value network\nto evaluate the rollouts, we can directly apply this look-ahead module to\nimprove the decoding of any sequence model trained in a maximum likelihood\nframework. We evaluate our look-ahead module on three datasets of varying\ndifficulties: IM2LATEX-100k OCR image to LaTeX, WMT16 multimodal machine\ntranslation, and WMT14 machine translation. Our look-ahead module improves the\nperformance of the simpler datasets such as IM2LATEX-100k and WMT16 multimodal\nmachine translation. However, the improvement of the more difficult dataset\n(e.g., containing longer sequences), WMT14 machine translation, becomes\nmarginal. Our further investigation using the k-step look-ahead suggests that\nthe more difficult tasks suffer from the overestimated EOS (end-of-sentence)\nprobability. We argue that the overestimated EOS probability also causes the\ndecreased performance of beam search when increasing its beam width. We tackle\nthe EOS problem by integrating an auxiliary EOS loss into the training to\nestimate if the model should emit EOS or other words. Our experiments show that\nimproving EOS estimation not only increases the performance of our proposed\nlook-ahead module but also the robustness of the beam search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03728v1",
            "date": "2020-03-08 06:00:15+00:00",
            "title": "Pseudo Labeling and Negative Feedback Learning for Large-scale Multi-label Domain Classification",
            "authors": [
                "Joo-Kyung Kim",
                "Young-Bum Kim"
            ],
            "abstract": "In large-scale domain classification, an utterance can be handled by multiple\ndomains with overlapped capabilities. However, only a limited number of\nground-truth domains are provided for each training utterance in practice while\nknowing as many as correct target labels is helpful for improving the model\nperformance. In this paper, given one ground-truth domain for each training\nutterance, we regard domains consistently predicted with the highest\nconfidences as additional pseudo labels for the training. In order to reduce\nprediction errors due to incorrect pseudo labels, we leverage utterances with\nnegative system responses to decrease the confidences of the incorrectly\npredicted domains. Evaluating on user utterances from an intelligent\nconversational system, we show that the proposed approach significantly\nimproves the performance of domain classification with hypothesis reranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04032v2",
            "date": "2020-03-09 10:50:16+00:00",
            "title": "Shallow Discourse Annotation for Chinese TED Talks",
            "authors": [
                "Wanqiu Long",
                "Xinyi Cai",
                "James E. M. Reid",
                "Bonnie Webber",
                "Deyi Xiong"
            ],
            "abstract": "Text corpora annotated with language-related properties are an important\nresource for the development of Language Technology. The current work\ncontributes a new resource for Chinese Language Technology and for\nChinese-English translation, in the form of a set of TED talks (some originally\ngiven in English, some in Chinese) that have been annotated with discourse\nrelations in the style of the Penn Discourse TreeBank, adapted to properties of\nChinese text that are not present in English. The resource is currently unique\nin annotating discourse-level properties of planned spoken monologues rather\nthan of written text. An inter-annotator agreement study demonstrates that the\nannotation scheme is able to achieve highly reliable results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04036v1",
            "date": "2020-03-09 10:58:38+00:00",
            "title": "Sentence Analogies: Exploring Linguistic Relationships and Regularities in Sentence Embeddings",
            "authors": [
                "Xunjie Zhu",
                "Gerard de Melo"
            ],
            "abstract": "While important properties of word vector representations have been studied\nextensively, far less is known about the properties of sentence vector\nrepresentations. Word vectors are often evaluated by assessing to what degree\nthey exhibit regularities with regard to relationships of the sort considered\nin word analogies. In this paper, we investigate to what extent commonly used\nsentence vector representation spaces as well reflect certain kinds of\nregularities. We propose a number of schemes to induce evaluation data, based\non lexical analogy data as well as semantic relationships between sentences.\nOur experiments consider a wide range of sentence embedding methods, including\nones based on BERT-style contextual embeddings. We find that different models\ndiffer substantially in their ability to reflect such regularities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04073v1",
            "date": "2020-03-09 12:34:48+00:00",
            "title": "A Multi-Source Entity-Level Sentiment Corpus for the Financial Domain: The FinLin Corpus",
            "authors": [
                "Tobias Daudert"
            ],
            "abstract": "We introduce FinLin, a novel corpus containing investor reports, company\nreports, news articles, and microblogs from StockTwits, targeting multiple\nentities stemming from the automobile industry and covering a 3-month period.\nFinLin was annotated with a sentiment score and a relevance score in the range\n[-1.0, 1.0] and [0.0, 1.0], respectively. The annotations also include the text\nspans selected for the sentiment, thus, providing additional insight into the\nannotators' reasoning. Overall, FinLin aims to complement the current knowledge\nby providing a novel and publicly available financial sentiment corpus and to\nfoster research on the topic of financial sentiment analysis and potential\napplications in behavioural science.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04642v1",
            "date": "2020-03-10 11:30:22+00:00",
            "title": "A Framework for Evaluation of Machine Reading Comprehension Gold Standards",
            "authors": [
                "Viktor Schlegel",
                "Marco Valentino",
                "Andr\u00e9 Freitas",
                "Goran Nenadic",
                "Riza Batista-Navarro"
            ],
            "abstract": "Machine Reading Comprehension (MRC) is the task of answering a question over\na paragraph of text. While neural MRC systems gain popularity and achieve\nnoticeable performance, issues are being raised with the methodology used to\nestablish their performance, particularly concerning the data design of gold\nstandards that are used to evaluate them. There is but a limited understanding\nof the challenges present in this data, which makes it hard to draw comparisons\nand formulate reliable hypotheses. As a first step towards alleviating the\nproblem, this paper proposes a unifying framework to systematically investigate\nthe present linguistic features, required reasoning and background knowledge\nand factual correctness on one hand, and the presence of lexical cues as a\nlower bound for the requirement of understanding on the other hand. We propose\na qualitative annotation schema for the first and a set of approximative\nmetrics for the latter. In a first application of the framework, we analyse\nmodern MRC gold standards and present our findings: the absence of features\nthat contribute towards lexical ambiguity, the varying factual correctness of\nthe expected answers and the presence of lexical cues, all of which potentially\nlower the reading comprehension complexity and quality of the evaluation data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04807v1",
            "date": "2020-03-10 15:33:54+00:00",
            "title": "Efficient Intent Detection with Dual Sentence Encoders",
            "authors": [
                "I\u00f1igo Casanueva",
                "Tadas Tem\u010dinas",
                "Daniela Gerz",
                "Matthew Henderson",
                "Ivan Vuli\u0107"
            ],
            "abstract": "Building conversational systems in new domains and with added functionality\nrequires resource-efficient models that work under low-data regimes (i.e., in\nfew-shot setups). Motivated by these requirements, we introduce intent\ndetection methods backed by pretrained dual sentence encoders such as USE and\nConveRT. We demonstrate the usefulness and wide applicability of the proposed\nintent detectors, showing that: 1) they outperform intent detectors based on\nfine-tuning the full BERT-Large model or using BERT as a fixed black-box\nencoder on three diverse intent detection data sets; 2) the gains are\nespecially pronounced in few-shot setups (i.e., with only 10 or 30 annotated\nexamples per intent); 3) our intent detectors can be trained in a matter of\nminutes on a single CPU; and 4) they are stable across different hyperparameter\nsettings. In hope of facilitating and democratizing research focused on\nintention detection, we release our code, as well as a new challenging\nsingle-domain intent detection dataset comprising 13,083 annotated examples\nover 77 intents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04866v1",
            "date": "2020-03-10 17:17:01+00:00",
            "title": "Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity",
            "authors": [
                "Ivan Vuli\u0107",
                "Simon Baker",
                "Edoardo Maria Ponti",
                "Ulla Petti",
                "Ira Leviant",
                "Kelly Wing",
                "Olga Majewska",
                "Eden Bar",
                "Matt Malone",
                "Thierry Poibeau",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "We introduce Multi-SimLex, a large-scale lexical resource and evaluation\nbenchmark covering datasets for 12 typologically diverse languages, including\nmajor languages (e.g., Mandarin Chinese, Spanish, Russian) as well as\nless-resourced ones (e.g., Welsh, Kiswahili). Each language dataset is\nannotated for the lexical relation of semantic similarity and contains 1,888\nsemantically aligned concept pairs, providing a representative coverage of word\nclasses (nouns, verbs, adjectives, adverbs), frequency ranks, similarity\nintervals, lexical fields, and concreteness levels. Additionally, owing to the\nalignment of concepts across languages, we provide a suite of 66 cross-lingual\nsemantic similarity datasets. Due to its extensive size and language coverage,\nMulti-SimLex provides entirely novel opportunities for experimental evaluation\nand analysis. On its monolingual and cross-lingual benchmarks, we evaluate and\nanalyze a wide array of recent state-of-the-art monolingual and cross-lingual\nrepresentation models, including static and contextualized word embeddings\n(such as fastText, M-BERT and XLM), externally informed lexical\nrepresentations, as well as fully unsupervised and (weakly) supervised\ncross-lingual word embeddings. We also present a step-by-step dataset creation\nprotocol for creating consistent, Multi-Simlex-style resources for additional\nlanguages. We make these contributions -- the public release of Multi-SimLex\ndatasets, their creation protocol, strong baseline results, and in-depth\nanalyses which can be be helpful in guiding future developments in multilingual\nlexical semantics and representation learning -- available via a website which\nwill encourage community effort in further expansion of Multi-Simlex to many\nmore languages. Such a large-scale semantic resource could inspire significant\nfurther advances in NLP across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04985v1",
            "date": "2020-02-27 22:07:11+00:00",
            "title": "Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT",
            "authors": [
                "Lichao Sun",
                "Kazuma Hashimoto",
                "Wenpeng Yin",
                "Akari Asai",
                "Jia Li",
                "Philip Yu",
                "Caiming Xiong"
            ],
            "abstract": "There is an increasing amount of literature that claims the brittleness of\ndeep neural networks in dealing with adversarial examples that are created\nmaliciously. It is unclear, however, how the models will perform in realistic\nscenarios where \\textit{natural rather than malicious} adversarial instances\noften exist. This work systematically explores the robustness of BERT, the\nstate-of-the-art Transformer-style model in NLP, in dealing with noisy data,\nparticularly mistakes in typing the keyboard, that occur inadvertently.\nIntensive experiments on sentiment analysis and question answering benchmarks\nindicate that: (i) Typos in various words of a sentence do not influence\nequally. The typos in informative words make severer damages; (ii) Mistype is\nthe most damaging factor, compared with inserting, deleting, etc.; (iii) Humans\nand machines have different focuses on recognizing adversarial attacks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04993v1",
            "date": "2020-03-05 02:54:32+00:00",
            "title": "Learning to mirror speaking styles incrementally",
            "authors": [
                "Siyi Liu",
                "Ziang Leng",
                "Derry Wijaya"
            ],
            "abstract": "Mirroring is the behavior in which one person subconsciously imitates the\ngesture, speech pattern, or attitude of another. In conversations, mirroring\noften signals the speakers enjoyment and engagement in their communication. In\nchatbots, methods have been proposed to add personas to the chatbots and to\ntrain them to speak or to shift their dialogue style to that of the personas.\nHowever, they often require a large dataset consisting of dialogues of the\ntarget personalities to train. In this work, we explore a method that can learn\nto mirror the speaking styles of a person incrementally. Our method extracts\nngrams that capture a persons speaking styles and uses the ngrams to create\npatterns for transforming sentences to the persons speaking styles. Our\nexperiments show that our method is able to capture patterns of speaking style\nthat can be used to transform regular sentences into sentences with the target\nstyle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04996v1",
            "date": "2020-02-28 14:58:41+00:00",
            "title": "GASP! Generating Abstracts of Scientific Papers from Abstracts of Cited Papers",
            "authors": [
                "Fabio Massimo Zanzotto",
                "Viviana Bono",
                "Paola Vocca",
                "Andrea Santilli",
                "Danilo Croce",
                "Giorgio Gambosi",
                "Roberto Basili"
            ],
            "abstract": "Creativity is one of the driving forces of human kind as it allows to break\ncurrent understanding to envision new ideas, which may revolutionize entire\nfields of knowledge. Scientific research offers a challenging environment where\nto learn a model for the creative process. In fact, scientific research is a\ncreative act in the formal settings of the scientific method and this creative\nact is described in articles.\n  In this paper, we dare to introduce the novel, scientifically and\nphilosophically challenging task of Generating Abstracts of Scientific Papers\nfrom abstracts of cited papers (GASP) as a text-to-text task to investigate\nscientific creativity, To foster research in this novel, challenging task, we\nprepared a dataset by using services where that solve the problem of copyright\nand, hence, the dataset is public available with its standard split. Finally,\nwe experimented with two vanilla summarization systems to start the analysis of\nthe complexity of the GASP task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05259v1",
            "date": "2020-03-11 12:36:17+00:00",
            "title": "Capturing document context inside sentence-level neural machine translation models with self-training",
            "authors": [
                "Elman Mansimov",
                "G\u00e1bor Melis",
                "Lei Yu"
            ],
            "abstract": "Neural machine translation (NMT) has arguably achieved human level parity\nwhen trained and evaluated at the sentence-level. Document-level neural machine\ntranslation has received less attention and lags behind its sentence-level\ncounterpart. The majority of the proposed document-level approaches investigate\nways of conditioning the model on several source or target sentences to capture\ndocument context. These approaches require training a specialized NMT model\nfrom scratch on parallel document-level corpora. We propose an approach that\ndoesn't require training a specialized model on parallel document-level corpora\nand is applied to a trained sentence-level NMT model at decoding time. We\nprocess the document from left to right multiple times and self-train the\nsentence-level model on pairs of source sentences and generated translations.\nOur approach reinforces the choices made by the model, thus making it more\nlikely that the same choices will be made in other sentences in the document.\nWe evaluate our approach on three document-level datasets: NIST\nChinese-English, WMT'19 Chinese-English and OpenSubtitles English-Russian. We\ndemonstrate that our approach has higher BLEU score and higher human preference\nthan the baseline. Qualitative analysis of our approach shows that choices made\nby model are consistent across the document.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05443v1",
            "date": "2020-03-11 04:08:27+00:00",
            "title": "A Precisely Xtreme-Multi Channel Hybrid Approach For Roman Urdu Sentiment Analysis",
            "authors": [
                "Faiza Memood",
                "Muhammad Usman Ghani",
                "Muhammad Ali Ibrahim",
                "Rehab Shehzadi",
                "Muhammad Nabeel Asim"
            ],
            "abstract": "In order to accelerate the performance of various Natural Language Processing\ntasks for Roman Urdu, this paper for the very first time provides 3 neural word\nembeddings prepared using most widely used approaches namely Word2vec,\nFastText, and Glove. The integrity of generated neural word embeddings is\nevaluated using intrinsic and extrinsic evaluation approaches. Considering the\nlack of publicly available benchmark datasets, it provides a first-ever Roman\nUrdu dataset which consists of 3241 sentiments annotated against positive,\nnegative and neutral classes. To provide benchmark baseline performance over\nthe presented dataset, we adapt diverse machine learning (Support Vector\nMachine Logistic Regression, Naive Bayes), deep learning (convolutional neural\nnetwork, recurrent neural network), and hybrid approaches. Effectiveness of\ngenerated neural word embeddings is evaluated by comparing the performance of\nmachine and deep learning based methodologies using 7, and 5 distinct feature\nrepresentation approaches respectively. Finally, it proposes a novel precisely\nextreme multi-channel hybrid methodology which outperforms state-of-the-art\nadapted machine and deep learning approaches by the figure of 9%, and 4% in\nterms of F1-score. Roman Urdu Sentiment Analysis, Pretrain word embeddings for\nRoman Urdu, Word2Vec, Glove, Fast-Text",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05522v1",
            "date": "2020-03-11 21:04:49+00:00",
            "title": "Semantic Holism and Word Representations in Artificial Neural Networks",
            "authors": [
                "Tom\u00e1\u0161 Musil"
            ],
            "abstract": "Artificial neural networks are a state-of-the-art solution for many problems\nin natural language processing. What can we learn about language and meaning\nfrom the way artificial neural networks represent it? Word representations\nobtained from the Skip-gram variant of the word2vec model exhibit interesting\nsemantic properties. This is usually explained by referring to the general\ndistributional hypothesis, which states that the meaning of the word is given\nby the contexts where it occurs. We propose a more specific approach based on\nFrege's holistic and functional approach to meaning. Taking Tugendhat's formal\nreinterpretation of Frege's work as a starting point, we demonstrate that it is\nanalogical to the process of training the Skip-gram model and offers a possible\nexplanation of its semantic properties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06044v1",
            "date": "2020-03-12 22:26:11+00:00",
            "title": "Local Contextual Attention with Hierarchical Structure for Dialogue Act Recognition",
            "authors": [
                "Zhigang Dai",
                "Jinhua Fu",
                "Qile Zhu",
                "Hengbin Cui",
                "Xiaolong li",
                "Yuan Qi"
            ],
            "abstract": "Dialogue act recognition is a fundamental task for an intelligent dialogue\nsystem. Previous work models the whole dialog to predict dialog acts, which may\nbring the noise from unrelated sentences. In this work, we design a\nhierarchical model based on self-attention to capture intra-sentence and\ninter-sentence information. We revise the attention distribution to focus on\nthe local and contextual semantic information by incorporating the relative\nposition information between utterances. Based on the found that the length of\ndialog affects the performance, we introduce a new dialog segmentation\nmechanism to analyze the effect of dialog length and context padding length\nunder online and offline settings. The experiment shows that our method\nachieves promising performance on two datasets: Switchboard Dialogue Act and\nDailyDialog with the accuracy of 80.34\\% and 85.81\\% respectively.\nVisualization of the attention weights shows that our method can learn the\ncontext dependency between utterances explicitly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06094v1",
            "date": "2020-03-13 03:31:29+00:00",
            "title": "MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space",
            "authors": [
                "Xiaoyuan Yi",
                "Ruoyu Li",
                "Cheng Yang",
                "Wenhao Li",
                "Maosong Sun"
            ],
            "abstract": "As an essential step towards computer creativity, automatic poetry generation\nhas gained increasing attention these years. Though recent neural models make\nprominent progress in some criteria of poetry quality, generated poems still\nsuffer from the problem of poor diversity. Related literature researches show\nthat different factors, such as life experience, historical background, etc.,\nwould influence composition styles of poets, which considerably contributes to\nthe high diversity of human-authored poetry. Inspired by this, we propose\nMixPoet, a novel model that absorbs multiple factors to create various styles\nand promote diversity. Based on a semi-supervised variational autoencoder, our\nmodel disentangles the latent space into some subspaces, with each conditioned\non one influence factor by adversarial training. In this way, the model learns\na controllable latent variable to capture and mix generalized factor-related\nproperties. Different factor mixtures lead to diverse styles and hence further\ndifferentiate generated poems from each other. Experiment results on Chinese\npoetry demonstrate that MixPoet improves both diversity and quality against\nthree state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06190v1",
            "date": "2020-03-13 10:26:45+00:00",
            "title": "WAC: A Corpus of Wikipedia Conversations for Online Abuse Detection",
            "authors": [
                "No\u00e9 Cecillon",
                "Vincent Labatut",
                "Richard Dufour",
                "Georges Linares"
            ],
            "abstract": "With the spread of online social networks, it is more and more difficult to\nmonitor all the user-generated content. Automating the moderation process of\nthe inappropriate exchange content on Internet has thus become a priority task.\nMethods have been proposed for this purpose, but it can be challenging to find\na suitable dataset to train and develop them. This issue is especially true for\napproaches based on information derived from the structure and the dynamic of\nthe conversation. In this work, we propose an original framework, based on the\nWikipedia Comment corpus, with comment-level abuse annotations of different\ntypes. The major contribution concerns the reconstruction of conversations, by\ncomparison to existing corpora, which focus only on isolated messages (i.e.\ntaken out of their conversational context). This large corpus of more than 380k\nannotated messages opens perspectives for online abuse detection and especially\nfor context-based approaches. We also propose, in addition to this corpus, a\ncomplete benchmarking platform to stimulate and fairly compare scientific works\naround the problem of content abuse detection, trying to avoid the recurring\nproblem of result replication. Finally, we apply two classification methods to\nour dataset to demonstrate its potential.",
            "journal ref": "12th Language Resources and Evaluation Conference (LREC 2020),\n  p.1375-1383 , May 2020, Marseille, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06209v1",
            "date": "2020-03-13 11:34:29+00:00",
            "title": "Review-guided Helpful Answer Identification in E-commerce",
            "authors": [
                "Wenxuan Zhang",
                "Wai Lam",
                "Yang Deng",
                "Jing Ma"
            ],
            "abstract": "Product-specific community question answering platforms can greatly help\naddress the concerns of potential customers. However, the user-provided answers\non such platforms often vary a lot in their qualities. Helpfulness votes from\nthe community can indicate the overall quality of the answer, but they are\noften missing. Accurately predicting the helpfulness of an answer to a given\nquestion and thus identifying helpful answers is becoming a demanding need.\nSince the helpfulness of an answer depends on multiple perspectives instead of\nonly topical relevance investigated in typical QA tasks, common answer\nselection algorithms are insufficient for tackling this task. In this paper, we\npropose the Review-guided Answer Helpfulness Prediction (RAHP) model that not\nonly considers the interactions between QA pairs but also investigates the\nopinion coherence between the answer and crowds' opinions reflected in the\nreviews, which is another important factor to identify helpful answers.\nMoreover, we tackle the task of determining opinion coherence as a language\ninference problem and explore the utilization of pre-training strategy to\ntransfer the textual inference knowledge obtained from a specifically designed\ntrained network. Extensive experiments conducted on real-world data across\nseven product categories show that our proposed model achieves superior\nperformance on the prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06381v1",
            "date": "2020-03-13 16:57:55+00:00",
            "title": "Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks",
            "authors": [
                "Yu Yuan",
                "Serge Sharoff"
            ],
            "abstract": "This paper explores the use of Deep Learning methods for automatic estimation\nof quality of human translations. Automatic estimation can provide useful\nfeedback for translation teaching, examination and quality control.\nConventional methods for solving this task rely on manually engineered features\nand external knowledge. This paper presents an end-to-end neural model without\nfeature engineering, incorporating a cross attention mechanism to detect which\nparts in sentence pairs are most relevant for assessing quality. Another\ncontribution concerns of prediction of fine-grained scores for measuring\ndifferent aspects of translation quality. Empirical results on a large human\nannotated dataset show that the neural model outperforms feature-based methods\nsignificantly. The dataset and the tools are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06389v1",
            "date": "2020-03-13 17:21:57+00:00",
            "title": "Know thy corpus! Robust methods for digital curation of Web corpora",
            "authors": [
                "Serge Sharoff"
            ],
            "abstract": "This paper proposes a novel framework for digital curation of Web corpora in\norder to provide robust estimation of their parameters, such as their\ncomposition and the lexicon. In recent years language models pre-trained on\nlarge corpora emerged as clear winners in numerous NLP tasks, but no proper\nanalysis of the corpora which led to their success has been conducted. The\npaper presents a procedure for robust frequency estimation, which helps in\nestablishing the core lexicon for a given corpus, as well as a procedure for\nestimating the corpus composition via unsupervised topic models and via\nsupervised genre classification of Web pages. The results of the digital\ncuration study applied to several Web-derived corpora demonstrate their\nconsiderable differences. First, this concerns different frequency bursts which\nimpact the core lexicon obtained from each corpus. Second, this concerns the\nkinds of texts they contain. For example, OpenWebText contains considerably\nmore topical news and political argumentation in comparison to ukWac or\nWikipedia. The tools and the results of analysis have been released.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06634v1",
            "date": "2020-03-14 14:02:27+00:00",
            "title": "Text Similarity Using Word Embeddings to Classify Misinformation",
            "authors": [
                "Caio Almeida",
                "D\u00e9bora Santos"
            ],
            "abstract": "Fake news is a growing problem in the last years, especially during\nelections. It's hard work to identify what is true and what is false among all\nthe user generated content that circulates every day. Technology can help with\nthat work and optimize the fact-checking process. In this work, we address the\nchallenge of finding similar content in order to be able to suggest to a\nfact-checker articles that could have been verified before and thus avoid that\nthe same information is verified more than once. This is especially important\nin collaborative approaches to fact-checking where members of large teams will\nnot know what content others have already fact-checked.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06651v1",
            "date": "2020-03-14 14:50:04+00:00",
            "title": "Word Sense Disambiguation for 158 Languages using Word Embeddings Only",
            "authors": [
                "Varvara Logacheva",
                "Denis Teslenko",
                "Artem Shelmanov",
                "Steffen Remus",
                "Dmitry Ustalov",
                "Andrey Kutuzov",
                "Ekaterina Artemova",
                "Chris Biemann",
                "Simone Paolo Ponzetto",
                "Alexander Panchenko"
            ],
            "abstract": "Disambiguation of word senses in context is easy for humans, but is a major\nchallenge for automatic approaches. Sophisticated supervised and\nknowledge-based models were developed to solve this task. However, (i) the\ninherent Zipfian distribution of supervised training instances for a given word\nand/or (ii) the quality of linguistic knowledge representations motivate the\ndevelopment of completely unsupervised and knowledge-free approaches to word\nsense disambiguation (WSD). They are particularly useful for under-resourced\nlanguages which do not have any resources for building either supervised and/or\nknowledge-based models. In this paper, we present a method that takes as input\na standard pre-trained word embedding model and induces a fully-fledged word\nsense inventory, which can be used for disambiguation in context. We use this\nmethod to induce a collection of sense inventories for 158 languages on the\nbasis of the original pre-trained fastText word embeddings by Grave et al.\n(2018), enabling WSD in these languages. Models and system are available\nonline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07008v3",
            "date": "2020-03-16 03:54:22+00:00",
            "title": "CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data",
            "authors": [
                "Matthew Shardlow",
                "Michael Cooper",
                "Marcos Zampieri"
            ],
            "abstract": "Predicting which words are considered hard to understand for a given target\npopulation is a vital step in many NLP applications such as text\nsimplification. This task is commonly referred to as Complex Word\nIdentification (CWI). With a few exceptions, previous studies have approached\nthe task as a binary classification task in which systems predict a complexity\nvalue (complex vs. non-complex) for a set of target words in a text. This\nchoice is motivated by the fact that all CWI datasets compiled so far have been\nannotated using a binary annotation scheme. Our paper addresses this limitation\nby presenting the first English dataset for continuous lexical complexity\nprediction. We use a 5-point Likert scale scheme to annotate complex words in\ntexts from three sources/domains: the Bible, Europarl, and biomedical texts.\nThis resulted in a corpus of 9,476 sentences each annotated by around 7\nannotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07082v2",
            "date": "2020-03-16 09:05:53+00:00",
            "title": "Stanza: A Python Natural Language Processing Toolkit for Many Human Languages",
            "authors": [
                "Peng Qi",
                "Yuhao Zhang",
                "Yuhui Zhang",
                "Jason Bolton",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce Stanza, an open-source Python natural language processing\ntoolkit supporting 66 human languages. Compared to existing widely used\ntoolkits, Stanza features a language-agnostic fully neural pipeline for text\nanalysis, including tokenization, multi-word token expansion, lemmatization,\npart-of-speech and morphological feature tagging, dependency parsing, and named\nentity recognition. We have trained Stanza on a total of 112 datasets,\nincluding the Universal Dependencies treebanks and other multilingual corpora,\nand show that the same neural architecture generalizes well and achieves\ncompetitive performance on all languages tested. Additionally, Stanza includes\na native Python interface to the widely used Java Stanford CoreNLP software,\nwhich further extends its functionality to cover other tasks such as\ncoreference resolution and relation extraction. Source code, documentation, and\npretrained models for 66 languages are available at\nhttps://stanfordnlp.github.io/stanza.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07428v1",
            "date": "2020-03-16 20:19:21+00:00",
            "title": "Developing a Multilingual Annotated Corpus of Misogyny and Aggression",
            "authors": [
                "Shiladitya Bhattacharya",
                "Siddharth Singh",
                "Ritesh Kumar",
                "Akanksha Bansal",
                "Akash Bhagat",
                "Yogesh Dawer",
                "Bornini Lahiri",
                "Atul Kr. Ojha"
            ],
            "abstract": "In this paper, we discuss the development of a multilingual annotated corpus\nof misogyny and aggression in Indian English, Hindi, and Indian Bangla as part\nof a project on studying and automatically identifying misogyny and communalism\non social media (the ComMA Project). The dataset is collected from comments on\nYouTube videos and currently contains a total of over 20,000 comments. The\ncomments are annotated at two levels - aggression (overtly aggressive, covertly\naggressive, and non-aggressive) and misogyny (gendered and non-gendered). We\ndescribe the process of data collection, the tagset used for annotation, and\nissues and challenges faced during the process of annotation. Finally, we\ndiscuss the results of the baseline experiments conducted to develop a\nclassifier for misogyny in the three languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07456v1",
            "date": "2020-03-16 22:10:35+00:00",
            "title": "HELFI: a Hebrew-Greek-Finnish Parallel Bible Corpus with Cross-Lingual Morpheme Alignment",
            "authors": [
                "Anssi Yli-Jyr\u00e4",
                "Josi Purhonen",
                "Matti Liljeqvist",
                "Arto Antturi",
                "Pekka Nieminen",
                "Kari M. R\u00e4ntil\u00e4",
                "Valtter Luoto"
            ],
            "abstract": "Twenty-five years ago, morphologically aligned Hebrew-Finnish and\nGreek-Finnish bitexts (texts accompanied by a translation) were constructed\nmanually in order to create an analytical concordance (Luoto et al., 1997) for\na Finnish Bible translation. The creators of the bitexts recently secured the\npublisher's permission to release its fine-grained alignment, but the alignment\nwas still dependent on proprietary, third-party resources such as a copyrighted\ntext edition and proprietary morphological analyses of the source texts. In\nthis paper, we describe a nontrivial editorial process starting from the\ncreation of the original one-purpose database and ending with its\nreconstruction using only freely available text editions and annotations. This\nprocess produced an openly available dataset that contains (i) the source texts\nand their translations, (ii) the morphological analyses, (iii) the\ncross-lingual morpheme alignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07459v2",
            "date": "2020-03-16 22:47:27+00:00",
            "title": "Offensive Language Identification in Greek",
            "authors": [
                "Zeses Pitenis",
                "Marcos Zampieri",
                "Tharindu Ranasinghe"
            ],
            "abstract": "As offensive language has become a rising issue for online communities and\nsocial media platforms, researchers have been investigating ways of coping with\nabusive content and developing systems to detect its different types:\ncyberbullying, hate speech, aggression, etc. With a few notable exceptions,\nmost research on this topic so far has dealt with English. This is mostly due\nto the availability of language resources for English. To address this\nshortcoming, this paper presents the first Greek annotated dataset for\noffensive language identification: the Offensive Greek Tweet Dataset (OGTD).\nOGTD is a manually annotated dataset containing 4,779 posts from Twitter\nannotated as offensive and not offensive. Along with a detailed description of\nthe dataset, we evaluate several computational models trained and tested on\nthis data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07490v3",
            "date": "2020-03-17 01:34:56+00:00",
            "title": "Recent Advances and Challenges in Task-oriented Dialog System",
            "authors": [
                "Zheng Zhang",
                "Ryuichi Takanobu",
                "Qi Zhu",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Due to the significance and value in human-computer interaction and natural\nlanguage processing, task-oriented dialog systems are attracting more and more\nattention in both academic and industrial communities. In this paper, we survey\nrecent advances and challenges in task-oriented dialog systems. We also discuss\nthree critical topics for task-oriented dialog systems: (1) improving data\nefficiency to facilitate dialog modeling in low-resource settings, (2) modeling\nmulti-turn dynamics for dialog policy learning to achieve better\ntask-completion performance, and (3) integrating domain ontology knowledge into\nthe dialog model. Besides, we review the recent progresses in dialog evaluation\nand some widely-used corpora. We believe that this survey, though incomplete,\ncan shed a light on future research in task-oriented dialog systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07507v1",
            "date": "2020-03-17 02:56:27+00:00",
            "title": "Multi-label natural language processing to identify diagnosis and procedure codes from MIMIC-III inpatient notes",
            "authors": [
                "A. K. Bhavani Singh",
                "Mounika Guntu",
                "Ananth Reddy Bhimireddy",
                "Judy W. Gichoya",
                "Saptarshi Purkayastha"
            ],
            "abstract": "In the United States, 25% or greater than 200 billion dollars of hospital\nspending accounts for administrative costs that involve services for medical\ncoding and billing. With the increasing number of patient records, manual\nassignment of the codes performed is overwhelming, time-consuming and\nerror-prone, causing billing errors. Natural language processing can automate\nthe extraction of codes/labels from unstructured clinical notes, which can aid\nhuman coders to save time, increase productivity, and verify medical coding\nerrors. Our objective is to identify appropriate diagnosis and procedure codes\nfrom clinical notes by performing multi-label classification. We used\nde-identified data of critical care patients from the MIMIC-III database and\nsubset the data to select the ten (top-10) and fifty (top-50) most common\ndiagnoses and procedures, which covers 47.45% and 74.12% of all admissions\nrespectively. We implemented state-of-the-art Bidirectional Encoder\nRepresentations from Transformers (BERT) to fine-tune the language model on 80%\nof the data and validated on the remaining 20%. The model achieved an overall\naccuracy of 87.08%, an F1 score of 85.82%, and an AUC of 91.76% for top-10\ncodes. For the top-50 codes, our model achieved an overall accuracy of 93.76%,\nan F1 score of 92.24%, and AUC of 91%. When compared to previously published\nresearch, our model outperforms in predicting codes from the clinical text. We\ndiscuss approaches to generalize the knowledge discovery process of our\nMIMIC-BERT to other clinical notes. This can help human coders to save time,\nprevent backlogs, and additional costs due to coding errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07568v2",
            "date": "2020-03-17 07:52:08+00:00",
            "title": "XPersona: Evaluating Multilingual Personalized Chatbot",
            "authors": [
                "Zhaojiang Lin",
                "Zihan Liu",
                "Genta Indra Winata",
                "Samuel Cahyawijaya",
                "Andrea Madotto",
                "Yejin Bang",
                "Etsuko Ishii",
                "Pascale Fung"
            ],
            "abstract": "Personalized dialogue systems are an essential step toward better\nhuman-machine interaction. Existing personalized dialogue agents rely on\nproperly designed conversational datasets, which are mostly monolingual (e.g.,\nEnglish), which greatly limits the usage of conversational agents in other\nlanguages. In this paper, we propose a multi-lingual extension of Persona-Chat,\nnamely XPersona. Our dataset includes persona conversations in six different\nlanguages other than English for building and evaluating multilingual\npersonalized agents. We experiment with both multilingual and cross-lingual\ntrained baselines, and evaluate them against monolingual and\ntranslation-pipeline models using both automatic and human evaluation.\nExperimental results show that the multilingual trained models outperform the\ntranslation-pipeline and that they are on par with the monolingual models, with\nthe advantage of having a single model across multiple languages. On the other\nhand, the state-of-the-art cross-lingual trained models achieve inferior\nperformance to the other models, showing that cross-lingual conversation\nmodeling is a challenging task. We hope that our dataset and baselines will\naccelerate research in multilingual dialogue systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07634v1",
            "date": "2020-03-17 10:49:03+00:00",
            "title": "Adapting Deep Learning Methods for Mental Health Prediction on Social Media",
            "authors": [
                "Ivan Sekuli\u0107",
                "Michael Strube"
            ],
            "abstract": "Mental health poses a significant challenge for an individual's well-being.\nText analysis of rich resources, like social media, can contribute to deeper\nunderstanding of illnesses and provide means for their early detection. We\ntackle a challenge of detecting social media users' mental status through deep\nlearning-based models, moving away from traditional approaches to the task. In\na binary classification task on predicting if a user suffers from one of nine\ndifferent disorders, a hierarchical attention network outperforms previously\nset benchmarks for four of the disorders. Furthermore, we explore the\nlimitations of our model and analyze phrases relevant for classification by\ninspecting the model's word-level attention weights.",
            "journal ref": "Proceedings of the 5th Workshop on Noisy User-generated Text,\n  2019, 322-327",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07723v2",
            "date": "2020-03-17 13:54:48+00:00",
            "title": "PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry",
            "authors": [
                "Thomas Haider",
                "Steffen Eger",
                "Evgeny Kim",
                "Roman Klinger",
                "Winfried Menninghaus"
            ],
            "abstract": "Most approaches to emotion analysis of social media, literature, news, and\nother domains focus exclusively on basic emotion categories as defined by Ekman\nor Plutchik. However, art (such as literature) enables engagement in a broader\nrange of more complex and subtle emotions. These have been shown to also\ninclude mixed emotional responses. We consider emotions in poetry as they are\nelicited in the reader, rather than what is expressed in the text or intended\nby the author. Thus, we conceptualize a set of aesthetic emotions that are\npredictive of aesthetic appreciation in the reader, and allow the annotation of\nmultiple labels per line to capture mixed emotions within their context. We\nevaluate this novel setting in an annotation experiment both with carefully\ntrained experts and via crowdsourcing. Our annotation with experts leads to an\nacceptable agreement of kappa = .70, resulting in a consistent dataset for\nfuture large scale analysis. Finally, we conduct first emotion classification\nexperiments based on BERT, showing that identifying aesthetic emotions is\nchallenging in our data, with up to .52 F1-micro on the German subset. Data and\nresources are available at https://github.com/tnhaider/poetry-emotion",
            "journal ref": "International Conference on Language Resources and Evaluation\n  (LREC) 2020, Marseille",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08132v1",
            "date": "2020-03-18 10:23:36+00:00",
            "title": "Gender Representation in Open Source Speech Resources",
            "authors": [
                "Mahault Garnerin",
                "Solange Rossato",
                "Laurent Besacier"
            ],
            "abstract": "With the rise of artificial intelligence (AI) and the growing use of\ndeep-learning architectures, the question of ethics, transparency and fairness\nof AI systems has become a central concern within the research community. We\naddress transparency and fairness in spoken language systems by proposing a\nstudy about gender representation in speech resources available through the\nOpen Speech and Language Resource platform. We show that finding gender\ninformation in open source corpora is not straightforward and that gender\nbalance depends on other corpus characteristics (elicited/non elicited speech,\nlow/high resource language, speech task targeted). The paper ends with\nrecommendations about metadata and gender information for researchers in order\nto assure better transparency of the speech systems built using such corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08272v2",
            "date": "2020-03-18 15:27:35+00:00",
            "title": "Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training",
            "authors": [
                "Ernie Chang",
                "David Ifeoluwa Adelani",
                "Xiaoyu Shen",
                "Vera Demberg"
            ],
            "abstract": "West African Pidgin English is a language that is significantly spoken in\nWest Africa, consisting of at least 75 million speakers. Nevertheless, proper\nmachine translation systems and relevant NLP datasets for pidgin English are\nvirtually absent. In this work, we develop techniques targeted at bridging the\ngap between Pidgin English and English in the context of natural language\ngeneration. %As a proof of concept, we explore the proposed techniques in the\narea of data-to-text generation. By building upon the previously released\nmonolingual Pidgin English text and parallel English data-to-text corpus, we\nhope to build a system that can automatically generate Pidgin English\ndescriptions from structured data. We first train a data-to-English text\ngeneration system, before employing techniques in unsupervised neural machine\ntranslation and self-training to establish the Pidgin-to-English cross-lingual\nalignment. The human evaluation performed on the generated Pidgin texts shows\nthat, though still far from being practically usable, the pivoting +\nself-training technique improves both Pidgin text fluency and relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08385v2",
            "date": "2020-03-18 17:58:10+00:00",
            "title": "X-Stance: A Multilingual Multi-Target Dataset for Stance Detection",
            "authors": [
                "Jannis Vamvas",
                "Rico Sennrich"
            ],
            "abstract": "We extract a large-scale stance detection dataset from comments written by\ncandidates of elections in Switzerland. The dataset consists of German, French\nand Italian text, allowing for a cross-lingual evaluation of stance detection.\nIt contains 67 000 comments on more than 150 political issues (targets). Unlike\nstance detection models that have specific target issues, we use the dataset to\ntrain a single model on all the issues. To make learning across targets\npossible, we prepend to each instance a natural question that represents the\ntarget (e.g. \"Do you support X?\"). Baseline results from multilingual BERT show\nthat zero-shot cross-lingual and cross-target transfer of stance detection is\nmoderately successful with this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08489v1",
            "date": "2020-03-18 22:17:48+00:00",
            "title": "An Analysis on the Learning Rules of the Skip-Gram Model",
            "authors": [
                "Canlin Zhang",
                "Xiuwen Liu",
                "Daniel Bis"
            ],
            "abstract": "To improve the generalization of the representations for natural language\nprocessing tasks, words are commonly represented using vectors, where distances\namong the vectors are related to the similarity of the words. While word2vec,\nthe state-of-the-art implementation of the skip-gram model, is widely used and\nimproves the performance of many natural language processing tasks, its\nmechanism is not yet well understood.\n  In this work, we derive the learning rules for the skip-gram model and\nestablish their close relationship to competitive learning. In addition, we\nprovide the global optimal solution constraints for the skip-gram model and\nvalidate them by experimental results.",
            "journal ref": "The 2019 International Joint Conference on Neural Networks (IJCNN)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08529v1",
            "date": "2020-03-19 00:48:32+00:00",
            "title": "Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections",
            "authors": [
                "Yi-An Lai",
                "Xuan Zhu",
                "Yi Zhang",
                "Mona Diab"
            ],
            "abstract": "Summarizing data samples by quantitative measures has a long history, with\ndescriptive statistics being a case in point. However, as natural language\nprocessing methods flourish, there are still insufficient characteristic\nmetrics to describe a collection of texts in terms of the words, sentences, or\nparagraphs they comprise. In this work, we propose metrics of diversity,\ndensity, and homogeneity that quantitatively measure the dispersion, sparsity,\nand uniformity of a text collection. We conduct a series of simulations to\nverify that each metric holds desired properties and resonates with human\nintuitions. Experiments on real-world datasets demonstrate that the proposed\ncharacteristic metrics are highly correlated with text classification\nperformance of a renowned model, BERT, which could inspire future applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08612v8",
            "date": "2020-03-19 07:36:10+00:00",
            "title": "Enhancing Factual Consistency of Abstractive Summarization",
            "authors": [
                "Chenguang Zhu",
                "William Hinthorn",
                "Ruochen Xu",
                "Qingkai Zeng",
                "Michael Zeng",
                "Xuedong Huang",
                "Meng Jiang"
            ],
            "abstract": "Automatic abstractive summaries are found to often distort or fabricate facts\nin the article. This inconsistency between summary and original text has\nseriously impacted its applicability. We propose a fact-aware summarization\nmodel FASum to extract and integrate factual relations into the summary\ngeneration process via graph attention. We then design a factual corrector\nmodel FC to automatically correct factual errors from summaries generated by\nexisting systems. Empirical results show that the fact-aware summarization can\nproduce abstractive summaries with higher factual consistency compared with\nexisting systems, and the correction model improves the factual consistency of\ngiven summaries via modifying only a few keywords.",
            "journal ref": "North American Chapter of the Association for Computational\n  Linguistics (NAACL), Mexico City, Mexico, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08925v1",
            "date": "2020-03-19 17:43:28+00:00",
            "title": "Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent",
            "authors": [
                "Anoop Kunchukuttan",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this work, we present an extensive study of statistical machine\ntranslation involving languages of the Indian subcontinent. These languages are\nrelated by genetic and contact relationships. We describe the similarities\nbetween Indic languages arising from these relationships. We explore how\nlexical and orthographic similarity among these languages can be utilized to\nimprove translation quality between Indic languages when limited parallel\ncorpora is available. We also explore how the structural correspondence between\nIndic languages can be utilized to re-use linguistic resources for English to\nIndic language translation. Our observations span 90 language pairs from 9\nIndic languages and English. To the best of our knowledge, this is the first\nlarge-scale study specifically devoted to utilizing language relatedness to\nimprove translation between related languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09029v1",
            "date": "2020-03-19 22:09:51+00:00",
            "title": "NSURL-2019 Task 7: Named Entity Recognition (NER) in Farsi",
            "authors": [
                "Nasrin Taghizadeh",
                "Zeinab Borhanifard",
                "Melika GolestaniPour",
                "Heshaam Faili"
            ],
            "abstract": "NSURL-2019 Task 7 focuses on Named Entity Recognition (NER) in Farsi. This\ntask was chosen to compare different approaches to find phrases that specify\nNamed Entities in Farsi texts, and to establish a standard testbed for future\nresearches on this task in Farsi. This paper describes the process of making\ntraining and test data, a list of participating teams (6 teams), and evaluation\nresults of their systems. The best system obtained 85.4% of F1 score based on\nphrase-level evaluation on seven classes of NEs including person, organization,\nlocation, date, time, money and percent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09166v2",
            "date": "2020-03-20 09:55:10+00:00",
            "title": "TNT-KID: Transformer-based Neural Tagger for Keyword Identification",
            "authors": [
                "Matej Martinc",
                "Bla\u017e \u0160krlj",
                "Senja Pollak"
            ],
            "abstract": "With growing amounts of available textual data, development of algorithms\ncapable of automatic analysis, categorization and summarization of these data\nhas become a necessity. In this research we present a novel algorithm for\nkeyword identification, i.e., an extraction of one or multi-word phrases\nrepresenting key aspects of a given document, called Transformer-based Neural\nTagger for Keyword IDentification (TNT-KID). By adapting the transformer\narchitecture for a specific task at hand and leveraging language model\npretraining on a domain specific corpus, the model is capable of overcoming\ndeficiencies of both supervised and unsupervised state-of-the-art approaches to\nkeyword extraction by offering competitive and robust performance on a variety\nof different datasets while requiring only a fraction of manually labeled data\nrequired by the best performing systems. This study also offers thorough error\nanalysis with valuable insights into the inner workings of the model and an\nablation study measuring the influence of specific components of the keyword\nidentification workflow on the overall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09244v1",
            "date": "2020-03-20 12:54:43+00:00",
            "title": "Language Technology Programme for Icelandic 2019-2023",
            "authors": [
                "Anna Bj\u00f6rk Nikul\u00e1sd\u00f3ttir",
                "J\u00f3n Gu\u00f0nason",
                "Anton Karl Ingason",
                "Hrafn Loftsson",
                "Eir\u00edkur R\u00f6gnvaldsson",
                "Einar Freyr Sigur\u00f0sson",
                "Stein\u00fe\u00f3r Steingr\u00edmsson"
            ],
            "abstract": "In this paper, we describe a new national language technology programme for\nIcelandic. The programme, which spans a period of five years, aims at making\nIcelandic usable in communication and interactions in the digital world, by\ndeveloping accessible, open-source language resources and software. The\nresearch and development work within the programme is carried out by a\nconsortium of universities, institutions, and private companies, with a strong\nemphasis on cooperation between academia and industries. Five core projects\nwill be the main content of the programme: language resources, speech\nrecognition, speech synthesis, machine translation, and spell and grammar\nchecking. We also describe other national language technology programmes and\ngive an overview over the history of language technology in Iceland.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09288v2",
            "date": "2020-03-20 14:17:16+00:00",
            "title": "FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning",
            "authors": [
                "Suyu Ge",
                "Fangzhao Wu",
                "Chuhan Wu",
                "Tao Qi",
                "Yongfeng Huang",
                "Xing Xie"
            ],
            "abstract": "Medical named entity recognition (NER) has wide applications in intelligent\nhealthcare. Sufficient labeled data is critical for training accurate medical\nNER model. However, the labeled data in a single medical platform is usually\nlimited. Although labeled datasets may exist in many different medical\nplatforms, they cannot be directly shared since medical data is highly\nprivacy-sensitive. In this paper, we propose a privacy-preserving medical NER\nmethod based on federated learning, which can leverage the labeled data in\ndifferent platforms to boost the training of medical NER model and remove the\nneed of exchanging raw data among different platforms. Since the labeled data\nin different platforms usually has some differences in entity type and\nannotation criteria, instead of constraining different platforms to share the\nsame model, we decompose the medical NER model in each platform into a shared\nmodule and a private module. The private module is used to capture the\ncharacteristics of the local data in each platform, and is updated using local\nlabeled data. The shared module is learned across different medical platform to\ncapture the shared NER knowledge. Its local gradients from different platforms\nare aggregated to update the global shared module, which is further delivered\nto each platform to update their local shared modules. Experiments on three\npublicly available datasets validate the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09586v2",
            "date": "2020-03-21 06:12:14+00:00",
            "title": "Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers",
            "authors": [
                "Hongfei Xu",
                "Josef van Genabith",
                "Qiuhui Liu",
                "Deyi Xiong"
            ],
            "abstract": "Due to its effectiveness and performance, the Transformer translation model\nhas attracted wide attention, most recently in terms of probing-based\napproaches. Previous work focuses on using or probing source linguistic\nfeatures in the encoder. To date, the way word translation evolves in\nTransformer layers has not yet been investigated. Naively, one might assume\nthat encoder layers capture source information while decoder layers translate.\nIn this work, we show that this is not quite the case: translation already\nhappens progressively in encoder layers and even in the input embeddings. More\nsurprisingly, we find that some of the lower decoder layers do not actually do\nthat much decoding. We show all of this in terms of a probing approach where we\nproject representations of the layer analyzed to the final trained and frozen\nclassifier level of the Transformer decoder to measure word translation\naccuracy. Our findings motivate and explain a Transformer configuration change:\nif translation already happens in the encoder layers, perhaps we can increase\nthe number of encoder layers, while decreasing the number of decoder layers,\nboosting decoding speed, without loss in translation quality? Our experiments\nshow that this is indeed the case: we can increase speed by up to a factor 2.3\nwith small gains in translation quality, while an 18-4 deep encoder\nconfiguration boosts translation quality by +1.42 BLEU (En-De) at a speed-up of\n1.4.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09606v1",
            "date": "2020-03-21 09:00:52+00:00",
            "title": "A Joint Approach to Compound Splitting and Idiomatic Compound Detection",
            "authors": [
                "Irina Krotova",
                "Sergey Aksenov",
                "Ekaterina Artemova"
            ],
            "abstract": "Applications such as machine translation, speech recognition, and information\nretrieval require efficient handling of noun compounds as they are one of the\npossible sources for out-of-vocabulary (OOV) words. In-depth processing of noun\ncompounds requires not only splitting them into smaller components (or even\nroots) but also the identification of instances that should remain unsplitted\nas they are of idiomatic nature. We develop a two-fold deep learning-based\napproach of noun compound splitting and idiomatic compound detection for the\nGerman language that we train using a newly collected corpus of annotated\nGerman compounds. Our neural noun compound splitter operates on a sub-word\nlevel and outperforms the current state of the art by about 5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10097v1",
            "date": "2020-03-23 06:46:28+00:00",
            "title": "E2EET: From Pipeline to End-to-end Entity Typing via Transformer-Based Embeddings",
            "authors": [
                "Michael Stewart",
                "Wei Liu"
            ],
            "abstract": "Entity Typing (ET) is the process of identifying the semantic types of every\nentity within a corpus. In contrast to Named Entity Recognition, where each\ntoken in a sentence is labelled with zero or one class label, ET involves\nlabelling each entity mention with one or more class labels. Existing entity\ntyping models, which operate at the mention level, are limited by two key\nfactors: they do not make use of recently-proposed context-dependent\nembeddings, and are trained on fixed context windows. They are therefore\nsensitive to window size selection and are unable to incorporate the context of\nthe entire document. In light of these drawbacks we propose to incorporate\ncontext using transformer-based embeddings for a mention-level model, and an\nend-to-end model using a Bi-GRU to remove the dependency on window size. An\nextensive ablative study demonstrates the effectiveness of contextualised\nembeddings for mention-level models and the competitiveness of our end-to-end\nmodel for entity typing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10224v2",
            "date": "2020-03-23 12:38:40+00:00",
            "title": "Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings",
            "authors": [
                "Christos Xypolopoulos",
                "Antoine J. -P. Tixier",
                "Michalis Vazirgiannis"
            ],
            "abstract": "The number of senses of a given word, or polysemy, is a very subjective\nnotion, which varies widely across annotators and resources. We propose a novel\nmethod to estimate polysemy, based on simple geometry in the contextual\nembedding space. Our approach is fully unsupervised and purely data-driven. We\nshow through rigorous experiments that our rankings are well correlated (with\nstrong statistical significance) with 6 different rankings derived from famous\nhuman-constructed resources such as WordNet, OntoNotes, Oxford, Wikipedia etc.,\nfor 6 different standard metrics. We also visualize and analyze the correlation\nbetween the human rankings. A valuable by-product of our method is the ability\nto sample, at no extra cost, sentences containing different senses of a given\nword. Finally, the fully unsupervised nature of our method makes it applicable\nto any language.\n  Code and data are publicly available at\nhttps://github.com/ksipos/polysemy-assessment .\n  The paper was accepted as a long paper at EACL 2021.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10254v1",
            "date": "2020-03-23 13:05:27+00:00",
            "title": "Fast Cross-domain Data Augmentation through Neural Sentence Editing",
            "authors": [
                "Guillaume Raille",
                "Sandra Djambazovska",
                "Claudiu Musat"
            ],
            "abstract": "Data augmentation promises to alleviate data scarcity. This is most important\nin cases where the initial data is in short supply. This is, for existing\nmethods, also where augmenting is the most difficult, as learning the full data\ndistribution is impossible. For natural language, sentence editing offers a\nsolution - relying on small but meaningful changes to the original ones.\nLearning which changes are meaningful also requires large amounts of training\ndata. We thus aim to learn this in a source domain where data is abundant and\napply it in a different, target domain, where data is scarce - cross-domain\naugmentation.\n  We create the Edit-transformer, a Transformer-based sentence editor that is\nsignificantly faster than the state of the art and also works cross-domain. We\nargue that, due to its structure, the Edit-transformer is better suited for\ncross-domain environments than its edit-based predecessors. We show this\nperformance gap on the Yelp-Wikipedia domain pairs. Finally, we show that due\nto this cross-domain performance advantage, the Edit-transformer leads to\nmeaningful performance gains in several downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10555v1",
            "date": "2020-03-23 21:17:42+00:00",
            "title": "ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators",
            "authors": [
                "Kevin Clark",
                "Minh-Thang Luong",
                "Quoc V. Le",
                "Christopher D. Manning"
            ],
            "abstract": "Masked language modeling (MLM) pre-training methods such as BERT corrupt the\ninput by replacing some tokens with [MASK] and then train a model to\nreconstruct the original tokens. While they produce good results when\ntransferred to downstream NLP tasks, they generally require large amounts of\ncompute to be effective. As an alternative, we propose a more sample-efficient\npre-training task called replaced token detection. Instead of masking the\ninput, our approach corrupts it by replacing some tokens with plausible\nalternatives sampled from a small generator network. Then, instead of training\na model that predicts the original identities of the corrupted tokens, we train\na discriminative model that predicts whether each token in the corrupted input\nwas replaced by a generator sample or not. Thorough experiments demonstrate\nthis new pre-training task is more efficient than MLM because the task is\ndefined over all input tokens rather than just the small subset that was masked\nout. As a result, the contextual representations learned by our approach\nsubstantially outperform the ones learned by BERT given the same model size,\ndata, and compute. The gains are particularly strong for small models; for\nexample, we train a model on one GPU for 4 days that outperforms GPT (trained\nusing 30x more compute) on the GLUE natural language understanding benchmark.\nOur approach also works well at scale, where it performs comparably to RoBERTa\nand XLNet while using less than 1/4 of their compute and outperforms them when\nusing the same amount of compute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10564v1",
            "date": "2020-03-23 22:07:15+00:00",
            "title": "Improving Yor\u00f9b\u00e1 Diacritic Restoration",
            "authors": [
                "Iroro Orife",
                "David I. Adelani",
                "Timi Fasubaa",
                "Victor Williamson",
                "Wuraola Fisayo Oyewusi",
                "Olamilekan Wahab",
                "Kola Tubosun"
            ],
            "abstract": "Yor\\`ub\\'a is a widely spoken West African language with a writing system\nrich in orthographic and tonal diacritics. They provide morphological\ninformation, are crucial for lexical disambiguation, pronunciation and are\nvital for any computational Speech or Natural Language Processing tasks.\nHowever diacritic marks are commonly excluded from electronic texts due to\nlimited device and application support as well as general education on proper\nusage. We report on recent efforts at dataset cultivation. By aggregating and\nimproving disparate texts from the web and various personal libraries, we were\nable to significantly grow our clean Yor\\`ub\\'a dataset from a majority\nBibilical text corpora with three sources to millions of tokens from over a\ndozen sources. We evaluate updated diacritic restoration models on a new,\ngeneral purpose, public-domain Yor\\`ub\\'a evaluation dataset of modern\njournalistic news text, selected to be multi-purpose and reflecting\ncontemporary usage. All pre-trained models, datasets and source-code have been\nreleased as an open-source project to advance efforts on Yor\\`ub\\'a language\ntechnology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10687v1",
            "date": "2020-03-24 07:01:09+00:00",
            "title": "Felix: Flexible Text Editing Through Tagging and Insertion",
            "authors": [
                "Jonathan Mallinson",
                "Aliaksei Severyn",
                "Eric Malmi",
                "Guillermo Garrido"
            ],
            "abstract": "We present Felix --- a flexible text-editing approach for generation,\ndesigned to derive the maximum benefit from the ideas of decoding with\nbi-directional contexts and self-supervised pre-training. In contrast to\nconventional sequence-to-sequence (seq2seq) models, Felix is efficient in\nlow-resource settings and fast at inference time, while being capable of\nmodeling flexible input-output transformations. We achieve this by decomposing\nthe text-editing task into two sub-tasks: tagging to decide on the subset of\ninput tokens and their order in the output text and insertion to in-fill the\nmissing tokens in the output not present in the input. The tagging model\nemploys a novel Pointer mechanism, while the insertion model is based on a\nMasked Language Model. Both of these models are chosen to be non-autoregressive\nto guarantee faster inference. Felix performs favourably when compared to\nrecent text-editing methods and strong seq2seq baselines when evaluated on four\nNLG tasks: Sentence Fusion, Machine Translation Automatic Post-Editing,\nSummarization, and Text Simplification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10704v1",
            "date": "2020-03-24 07:53:41+00:00",
            "title": "Towards Neural Machine Translation for Edoid Languages",
            "authors": [
                "Iroro Orife"
            ],
            "abstract": "Many Nigerian languages have relinquished their previous prestige and purpose\nin modern society to English and Nigerian Pidgin. For the millions of L1\nspeakers of indigenous languages, there are inequalities that manifest\nthemselves as unequal access to information, communications, health care,\nsecurity as well as attenuated participation in political and civic life. To\nminimize exclusion and promote socio-linguistic and economic empowerment, this\nwork explores the feasibility of Neural Machine Translation (NMT) for the Edoid\nlanguage family of Southern Nigeria. Using the new JW300 public dataset, we\ntrained and evaluated baseline translation models for four widely spoken\nlanguages in this group: \\`Ed\\'o, \\'Es\\'an, Urhobo and Isoko. Trained models,\ncode and datasets have been open-sourced to advance future research efforts on\nEdoid language technology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10816v2",
            "date": "2020-03-24 13:04:06+00:00",
            "title": "Cross-Lingual Adaptation Using Universal Dependencies",
            "authors": [
                "Nasrin Taghizadeh",
                "Heshaam Faili"
            ],
            "abstract": "We describe a cross-lingual adaptation method based on syntactic parse trees\nobtained from the Universal Dependencies (UD), which are consistent across\nlanguages, to develop classifiers in low-resource languages. The idea of UD\nparsing is to capture similarities as well as idiosyncrasies among\ntypologically different languages. In this paper, we show that models trained\nusing UD parse trees for complex NLP tasks can characterize very different\nlanguages. We study two tasks of paraphrase identification and semantic\nrelation extraction as case studies. Based on UD parse trees, we develop\nseveral models using tree kernels and show that these models trained on the\nEnglish dataset can correctly classify data of other languages e.g. French,\nFarsi, and Arabic. The proposed approach opens up avenues for exploiting UD\nparsing in solving similar cross-lingual tasks, which is very useful for\nlanguages that no labeled data is available for them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11516v1",
            "date": "2020-03-11 10:18:32+00:00",
            "title": "Keyword-Attentive Deep Semantic Matching",
            "authors": [
                "Changyu Miao",
                "Zhen Cao",
                "Yik-Cheung Tam"
            ],
            "abstract": "Deep Semantic Matching is a crucial component in various natural language\nprocessing applications such as question and answering (QA), where an input\nquery is compared to each candidate question in a QA corpus in terms of\nrelevance. Measuring similarities between a query-question pair in an open\ndomain scenario can be challenging due to diverse word tokens in the\nqueryquestion pair. We propose a keyword-attentive approach to improve deep\nsemantic matching. We first leverage domain tags from a large corpus to\ngenerate a domain-enhanced keyword dictionary. Built upon BERT, we stack a\nkeyword-attentive transformer layer to highlight the importance of keywords in\nthe query-question pair. During model training, we propose a new negative\nsampling approach based on keyword coverage between the input pair. We evaluate\nour approach on a Chinese QA corpus using various metrics, including precision\nof retrieval candidates and accuracy of semantic matching. Experiments show\nthat our approach outperforms existing strong baselines. Our approach is\ngeneral and can be applied to other text matching tasks with little adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11521v1",
            "date": "2020-03-09 15:25:37+00:00",
            "title": "Matching Text with Deep Mutual Information Estimation",
            "authors": [
                "Xixi Zhou",
                "Chengxi Li",
                "Jiajun Bu",
                "Chengwei Yao",
                "Keyue Shi",
                "Zhi Yu",
                "Zhou Yu"
            ],
            "abstract": "Text matching is a core natural language processing research problem. How to\nretain sufficient information on both content and structure information is one\nimportant challenge. In this paper, we present a neural approach for\ngeneral-purpose text matching with deep mutual information estimation\nincorporated. Our approach, Text matching with Deep Info Max (TIM), is\nintegrated with a procedure of unsupervised learning of representations by\nmaximizing the mutual information between text matching neural network's input\nand output. We use both global and local mutual information to learn text\nrepresentations. We evaluate our text matching approach on several tasks\nincluding natural language inference, paraphrase identification, and answer\nselection. Compared to the state-of-the-art approaches, the experiments show\nthat our method integrated with mutual information estimation learns better\ntext representation and achieves better experimental results of text matching\ntasks without exploiting pretraining on external data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11523v1",
            "date": "2020-03-09 10:34:25+00:00",
            "title": "Tigrinya Neural Machine Translation with Transfer Learning for Humanitarian Response",
            "authors": [
                "Alp \u00d6ktem",
                "Mirko Plitt",
                "Grace Tang"
            ],
            "abstract": "We report our experiments in building a domain-specific Tigrinya-to-English\nneural machine translation system. We use transfer learning from other Ge'ez\nscript languages and report an improvement of 1.3 BLEU points over a classic\nneural baseline. We publish our development pipeline as an open-source library\nand also provide a demonstration application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11528v1",
            "date": "2020-03-13 14:16:25+00:00",
            "title": "Generating Major Types of Chinese Classical Poetry in a Uniformed Framework",
            "authors": [
                "Jinyi Hu",
                "Maosong Sun"
            ],
            "abstract": "Poetry generation is an interesting research topic in the field of text\ngeneration. As one of the most valuable literary and cultural heritages of\nChina, Chinese classical poetry is very familiar and loved by Chinese people\nfrom generation to generation. It has many particular characteristics in its\nlanguage structure, ranging from form, sound to meaning, thus is regarded as an\nideal testing task for text generation. In this paper, we propose a GPT-2 based\nuniformed framework for generating major types of Chinese classical poems. We\ndefine a unified format for formulating all types of training samples by\nintegrating detailed form information, then present a simple form-stressed\nweighting method in GPT-2 to strengthen the control to the form of the\ngenerated poems, with special emphasis on those forms with longer body length.\nPreliminary experimental results show this enhanced model can generate Chinese\nclassical poems of major types with high quality in both form and content,\nvalidating the effectiveness of the proposed strategy. The model has been\nincorporated into Jiuge, the most influential Chinese classical poetry\ngeneration system developed by Tsinghua University (Guo et al., 2019).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11529v1",
            "date": "2020-03-13 09:01:02+00:00",
            "title": "Masakhane -- Machine Translation For Africa",
            "authors": [
                "Iroro Orife",
                "Julia Kreutzer",
                "Blessing Sibanda",
                "Daniel Whitenack",
                "Kathleen Siminyu",
                "Laura Martinus",
                "Jamiil Toure Ali",
                "Jade Abbott",
                "Vukosi Marivate",
                "Salomon Kabongo",
                "Musie Meressa",
                "Espoir Murhabazi",
                "Orevaoghene Ahia",
                "Elan van Biljon",
                "Arshath Ramkilowan",
                "Adewale Akinfaderin",
                "Alp \u00d6ktem",
                "Wole Akin",
                "Ghollah Kioko",
                "Kevin Degila",
                "Herman Kamper",
                "Bonaventure Dossou",
                "Chris Emezue",
                "Kelechi Ogueji",
                "Abdallah Bashir"
            ],
            "abstract": "Africa has over 2000 languages. Despite this, African languages account for a\nsmall portion of available resources and publications in Natural Language\nProcessing (NLP). This is due to multiple factors, including: a lack of focus\nfrom government and funding, discoverability, a lack of community, sheer\nlanguage complexity, difficulty in reproducing papers and no benchmarks to\ncompare techniques. To begin to address the identified problems, MASAKHANE, an\nopen-source, continent-wide, distributed, online research effort for machine\ntranslation for African languages, was founded. In this paper, we discuss our\nmethodology for building the community and spurring research from the African\ncontinent, as well as outline the success of the community in terms of\naddressing the identified problems affecting African NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11531v1",
            "date": "2020-03-12 03:10:25+00:00",
            "title": "The Medical Scribe: Corpus Development and Model Performance Analyses",
            "authors": [
                "Izhak Shafran",
                "Nan Du",
                "Linh Tran",
                "Amanda Perry",
                "Lauren Keyes",
                "Mark Knichel",
                "Ashley Domin",
                "Lei Huang",
                "Yuhui Chen",
                "Gang Li",
                "Mingqiu Wang",
                "Laurent El Shafey",
                "Hagen Soltau",
                "Justin S. Paul"
            ],
            "abstract": "There is a growing interest in creating tools to assist in clinical note\ngeneration using the audio of provider-patient encounters. Motivated by this\ngoal and with the help of providers and medical scribes, we developed an\nannotation scheme to extract relevant clinical concepts. We used this\nannotation scheme to label a corpus of about 6k clinical encounters. This was\nused to train a state-of-the-art tagging model. We report ontologies, labeling\nresults, model performances, and detailed analyses of the results. Our results\nshow that the entities related to medications can be extracted with a\nrelatively high accuracy of 0.90 F-score, followed by symptoms at 0.72 F-score,\nand conditions at 0.57 F-score. In our task, we not only identify where the\nsymptoms are mentioned but also map them to canonical forms as they appear in\nthe clinical notes. Of the different types of errors, in about 19-38% of the\ncases, we find that the model output was correct, and about 17-32% of the\nerrors do not impact the clinical note. Taken together, the models developed in\nthis work are more useful than the F-scores reflect, making it a promising\napproach for practical applications.",
            "journal ref": "Proceedings of Language Resources and Evaluation, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11545v1",
            "date": "2020-03-24 19:32:11+00:00",
            "title": "Forensic Authorship Analysis of Microblogging Texts Using N-Grams and Stylometric Features",
            "authors": [
                "Nicole Mariah Sharon Belvisi",
                "Naveed Muhammad",
                "Fernando Alonso-Fernandez"
            ],
            "abstract": "In recent years, messages and text posted on the Internet are used in\ncriminal investigations. Unfortunately, the authorship of many of them remains\nunknown. In some channels, the problem of establishing authorship may be even\nharder, since the length of digital texts is limited to a certain number of\ncharacters. In this work, we aim at identifying authors of tweet messages,\nwhich are limited to 280 characters. We evaluate popular features employed\ntraditionally in authorship attribution which capture properties of the writing\nstyle at different levels. We use for our experiments a self-captured database\nof 40 users, with 120 to 200 tweets per user. Results using this small set are\npromising, with the different features providing a classification accuracy\nbetween 92% and 98.5%. These results are competitive in comparison to existing\nstudies which employ short texts such as tweets or SMS.",
            "journal ref": "Proc. 8th International Workshop on Biometrics and Forensics,\n  IWBF, Porto, Portugal, April 29-30, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11963v2",
            "date": "2020-03-26 15:01:37+00:00",
            "title": "TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation",
            "authors": [
                "Shaojie Jiang",
                "Thomas Wolf",
                "Christof Monz",
                "Maarten de Rijke"
            ],
            "abstract": "Natural Language Generation (NLG) models are prone to generating repetitive\nutterances. In this work, we study the repetition problem for encoder-decoder\nmodels, using both recurrent neural network (RNN) and transformer\narchitectures. To this end, we consider the chit-chat task, where the problem\nis more prominent than in other tasks that need encoder-decoder architectures.\nWe first study the influence of model architectures. By using pre-attention and\nhighway connections for RNNs, we manage to achieve lower repetition rates.\nHowever, this method does not generalize to other models such as transformers.\nWe hypothesize that the deeper reason is that in the training corpora, there\nare hard tokens that are more difficult for a generative model to learn than\nothers and, once learning has finished, hard tokens are still under-learned, so\nthat repetitive generations are more likely to happen. Based on this\nhypothesis, we propose token loss dynamic reweighting (TLDR) that applies\ndifferentiable weights to individual token losses. By using higher weights for\nhard tokens and lower weights for easy tokens, NLG models are able to learn\nindividual tokens at different paces. Experiments on chit-chat benchmark\ndatasets show that TLDR is more effective in repetition reduction for both RNN\nand transformer architectures than baselines using different weighting\nfunctions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12111v1",
            "date": "2020-03-26 19:01:31+00:00",
            "title": "FFR V1.0: Fon-French Neural Machine Translation",
            "authors": [
                "Bonaventure F. P. Dossou",
                "Chris C. Emezue"
            ],
            "abstract": "Africa has the highest linguistic diversity in the world. On account of the\nimportance of language to communication, and the importance of reliable,\npowerful and accurate machine translation models in modern inter-cultural\ncommunication, there have been (and still are) efforts to create\nstate-of-the-art translation models for the many African languages. However,\nthe low-resources, diacritical and tonal complexities of African languages are\nmajor issues facing African NLP today. The FFR is a major step towards creating\na robust translation model from Fon, a very low-resource and tonal language, to\nFrench, for research and public use. In this paper, we describe our pilot\nproject: the creation of a large growing corpora for Fon-to-French translations\nand our FFR v1.0 model, trained on this dataset. The dataset and model are made\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12298v1",
            "date": "2020-03-27 09:35:38+00:00",
            "title": "Information-Theoretic Probing with Minimum Description Length",
            "authors": [
                "Elena Voita",
                "Ivan Titov"
            ],
            "abstract": "To measure how well pretrained representations encode some linguistic\nproperty, it is common to use accuracy of a probe, i.e. a classifier trained to\npredict the property from the representations. Despite widespread adoption of\nprobes, differences in their accuracy fail to adequately reflect differences in\nrepresentations. For example, they do not substantially favour pretrained\nrepresentations over randomly initialized ones. Analogously, their accuracy can\nbe similar when probing for genuine linguistic labels and probing for random\nsynthetic tasks. To see reasonable differences in accuracy with respect to\nthese random baselines, previous work had to constrain either the amount of\nprobe training data or its model size. Instead, we propose an alternative to\nthe standard probes, information-theoretic probing with minimum description\nlength (MDL). With MDL probing, training a probe to predict labels is recast as\nteaching it to effectively transmit the data. Therefore, the measure of\ninterest changes from probe accuracy to the description length of labels given\nrepresentations. In addition to probe quality, the description length evaluates\n\"the amount of effort\" needed to achieve the quality. This amount of effort\ncharacterizes either (i) size of a probing model, or (ii) the amount of data\nneeded to achieve the high quality. We consider two methods for estimating MDL\nwhich can be easily implemented on top of the standard probing pipelines:\nvariational coding and online coding. We show that these methods agree in\nresults and are more informative and stable than the standard probes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12450v1",
            "date": "2020-03-27 14:52:55+00:00",
            "title": "Semantic Enrichment of Nigerian Pidgin English for Contextual Sentiment Classification",
            "authors": [
                "Wuraola Fisayo Oyewusi",
                "Olubayo Adekanmbi",
                "Olalekan Akinsande"
            ],
            "abstract": "Nigerian English adaptation, Pidgin, has evolved over the years through\nmulti-language code switching, code mixing and linguistic adaptation. While\nPidgin preserves many of the words in the normal English language corpus, both\nin spelling and pronunciation, the fundamental meaning of these words have\nchanged significantly. For example,'ginger' is not a plant but an expression of\nmotivation and 'tank' is not a container but an expression of gratitude. The\nimplication is that the current approach of using direct English sentiment\nanalysis of social media text from Nigeria is sub-optimal, as it will not be\nable to capture the semantic variation and contextual evolution in the\ncontemporary meaning of these words. In practice, while many words in Nigerian\nPidgin adaptation are the same as the standard English, the full English\nlanguage based sentiment analysis models are not designed to capture the full\nintent of the Nigerian pidgin when used alone or code-mixed. By augmenting\nscarce human labelled code-changed text with ample synthetic code-reformatted\ntext and meaning, we achieve significant improvements in sentiment scoring. Our\nresearch explores how to understand sentiment in an intrasentential code mixing\nand switching context where there has been significant word localization.This\nwork presents a 300 VADER lexicon compatible Nigerian Pidgin sentiment tokens\nand their scores and a 14,000 gold standard Nigerian Pidgin tweets and their\nsentiments labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12738v1",
            "date": "2020-03-28 07:48:02+00:00",
            "title": "Variational Transformers for Diverse Response Generation",
            "authors": [
                "Zhaojiang Lin",
                "Genta Indra Winata",
                "Peng Xu",
                "Zihan Liu",
                "Pascale Fung"
            ],
            "abstract": "Despite the great promise of Transformers in many sequence modeling tasks\n(e.g., machine translation), their deterministic nature hinders them from\ngeneralizing to high entropy tasks such as dialogue response generation.\nPrevious work proposes to capture the variability of dialogue responses with a\nrecurrent neural network (RNN)-based conditional variational autoencoder\n(CVAE). However, the autoregressive computation of the RNN limits the training\nefficiency. Therefore, we propose the Variational Transformer (VT), a\nvariational self-attentive feed-forward sequence model. The VT combines the\nparallelizability and global receptive field of the Transformer with the\nvariational nature of the CVAE by incorporating stochastic latent variables\ninto Transformers. We explore two types of the VT: 1) modeling the\ndiscourse-level diversity with a global latent variable; and 2) augmenting the\nTransformer decoder with a sequence of fine-grained latent variables. Then, the\nproposed models are evaluated on three conversational datasets with both\nautomatic metric and human evaluation. The experimental results show that our\nmodels improve standard Transformers and other baselines in terms of diversity,\nsemantic relevance, and human judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12754v1",
            "date": "2020-03-28 09:32:31+00:00",
            "title": "HIN: Hierarchical Inference Network for Document-Level Relation Extraction",
            "authors": [
                "Hengzhu Tang",
                "Yanan Cao",
                "Zhenyu Zhang",
                "Jiangxia Cao",
                "Fang Fang",
                "Shi Wang",
                "Pengfei Yin"
            ],
            "abstract": "Document-level RE requires reading, inferring and aggregating over multiple\nsentences. From our point of view, it is necessary for document-level RE to\ntake advantage of multi-granularity inference information: entity level,\nsentence level and document level. Thus, how to obtain and aggregate the\ninference information with different granularity is challenging for\ndocument-level RE, which has not been considered by previous work. In this\npaper, we propose a Hierarchical Inference Network (HIN) to make full use of\nthe abundant information from entity level, sentence level and document level.\nTranslation constraint and bilinear transformation are applied to target entity\npair in multiple subspaces to get entity-level inference information. Next, we\nmodel the inference between entity-level information and sentence\nrepresentation to achieve sentence-level inference information. Finally, a\nhierarchical aggregation approach is adopted to obtain the document-level\ninference information. In this way, our model can effectively aggregate\ninference information from these three different granularities. Experimental\nresults show that our method achieves state-of-the-art performance on the\nlarge-scale DocRED dataset. We also demonstrate that using BERT representations\ncan further substantially boost the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12932v3",
            "date": "2020-03-29 02:49:11+00:00",
            "title": "Noisy Text Data: Achilles' Heel of BERT",
            "authors": [
                "Ankit Kumar",
                "Piyush Makhija",
                "Anuj Gupta"
            ],
            "abstract": "Owing to the phenomenal success of BERT on various NLP tasks and benchmark\ndatasets, industry practitioners are actively experimenting with fine-tuning\nBERT to build NLP applications for solving industry use cases. For most\ndatasets that are used by practitioners to build industrial NLP applications,\nit is hard to guarantee absence of any noise in the data. While BERT has\nperformed exceedingly well for transferring the learnings from one use case to\nanother, it remains unclear how BERT performs when fine-tuned on noisy text. In\nthis work, we explore the sensitivity of BERT to noise in the data. We work\nwith most commonly occurring noise (spelling mistakes, typos) and show that\nthis results in significant degradation in the performance of BERT. We present\nexperimental results to show that BERT's performance on fundamental NLP tasks\nlike sentiment analysis and textual similarity drops significantly in the\npresence of (simulated) noise on benchmark datasets viz. IMDB Movie Review,\nSTS-B, SST-2. Further, we identify shortcomings in the existing BERT pipeline\nthat are responsible for this drop in performance. Our findings suggest that\npractitioners need to be vary of presence of noise in their datasets while\nfine-tuning BERT to solve industry use cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13028v1",
            "date": "2020-03-29 14:00:25+00:00",
            "title": "Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models",
            "authors": [
                "Itsumi Saito",
                "Kyosuke Nishida",
                "Kosuke Nishida",
                "Junji Tomita"
            ],
            "abstract": "Pre-trained sequence-to-sequence (seq-to-seq) models have significantly\nimproved the accuracy of several language generation tasks, including\nabstractive summarization. Although the fluency of abstractive summarization\nhas been greatly improved by fine-tuning these models, it is not clear whether\nthey can also identify the important parts of the source text to be included in\nthe summary. In this study, we investigated the effectiveness of combining\nsaliency models that identify the important parts of the source text with the\npre-trained seq-to-seq models through extensive experiments. We also proposed a\nnew combination model consisting of a saliency model that extracts a token\nsequence from a source text and a seq-to-seq model that takes the sequence as\nan additional input text. Experimental results showed that most of the\ncombination models outperformed a simple fine-tuned seq-to-seq model on both\nthe CNN/DM and XSum datasets even if the seq-to-seq model is pre-trained on\nlarge-scale corpora. Moreover, for the CNN/DM dataset, the proposed combination\nmodel exceeded the previous best-performed model by 1.33 points on ROUGE-L.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13032v1",
            "date": "2020-03-29 14:08:43+00:00",
            "title": "Named Entities in Medical Case Reports: Corpus and Experiments",
            "authors": [
                "Sarah Schulz",
                "Jurica \u0160eva",
                "Samuel Rodriguez",
                "Malte Ostendorff",
                "Georg Rehm"
            ],
            "abstract": "We present a new corpus comprising annotations of medical entities in case\nreports, originating from PubMed Central's open access library. In the case\nreports, we annotate cases, conditions, findings, factors and negation\nmodifiers. Moreover, where applicable, we annotate relations between these\nentities. As such, this is the first corpus of this kind made available to the\nscientific community in English. It enables the initial investigation of\nautomatic information extraction from case reports through tasks like Named\nEntity Recognition, Relation Extraction and (sentence/paragraph) relevance\ndetection. Additionally, we present four strong baseline systems for the\ndetection of medical entities made available through the annotated dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13205v1",
            "date": "2020-03-30 03:38:01+00:00",
            "title": "Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation",
            "authors": [
                "Pei Zhang",
                "Xu Zhang",
                "Wei Chen",
                "Jian Yu",
                "Yanfeng Wang",
                "Deyi Xiong"
            ],
            "abstract": "Document-level machine translation incorporates inter-sentential dependencies\ninto the translation of a source sentence. In this paper, we propose a new\nframework to model cross-sentence dependencies by training neural machine\ntranslation (NMT) to predict both the target translation and surrounding\nsentences of a source sentence. By enforcing the NMT model to predict source\ncontext, we want the model to learn \"contextualized\" source sentence\nrepresentations that capture document-level dependencies on the source side. We\nfurther propose two different methods to learn and integrate such\ncontextualized sentence embeddings into NMT: a joint training method that\njointly trains an NMT model with the source context prediction model and a\npre-training & fine-tuning method that pretrains the source context prediction\nmodel on a large-scale monolingual document corpus and then fine-tunes it with\nthe NMT model. Experiments on Chinese-English and English-German translation\nshow that both methods can substantially improve the translation quality over a\nstrong document-level Transformer baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13325v1",
            "date": "2020-03-30 10:30:34+00:00",
            "title": "Investigating Language Impact in Bilingual Approaches for Computational Language Documentation",
            "authors": [
                "Marcely Zanon Boito",
                "Aline Villavicencio",
                "Laurent Besacier"
            ],
            "abstract": "For endangered languages, data collection campaigns have to accommodate the\nchallenge that many of them are from oral tradition, and producing\ntranscriptions is costly. Therefore, it is fundamental to translate them into a\nwidely spoken language to ensure interpretability of the recordings. In this\npaper we investigate how the choice of translation language affects the\nposterior documentation work and potential automatic approaches which will work\non top of the produced bilingual corpus. For answering this question, we use\nthe MaSS multilingual speech corpus (Boito et al., 2020) for creating 56\nbilingual pairs that we apply to the task of low-resource unsupervised word\nsegmentation and alignment. Our results highlight that the choice of language\nfor translation influences the word segmentation performance, and that\ndifferent lexicons are learned by using different aligned translations. Lastly,\nthis paper proposes a hybrid approach for bilingual word segmentation,\ncombining boundary clues extracted from a non-parametric Bayesian model\n(Goldwater et al., 2009a) with the attentional word segmentation neural model\nfrom Godard et al. (2018). Our results suggest that incorporating these clues\ninto the neural models' input representation increases their translation and\nalignment quality, specially for challenging language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13342v1",
            "date": "2020-03-30 11:17:31+00:00",
            "title": "A Corpus of Controlled Opinionated and Knowledgeable Movie Discussions for Training Neural Conversation Models",
            "authors": [
                "Fabian Galetzka",
                "Chukwuemeka U. Eneh",
                "David Schlangen"
            ],
            "abstract": "Fully data driven Chatbots for non-goal oriented dialogues are known to\nsuffer from inconsistent behaviour across their turns, stemming from a general\ndifficulty in controlling parameters like their assumed background personality\nand knowledge of facts. One reason for this is the relative lack of labeled\ndata from which personality consistency and fact usage could be learned\ntogether with dialogue behaviour. To address this, we introduce a new labeled\ndialogue dataset in the domain of movie discussions, where every dialogue is\nbased on pre-specified facts and opinions. We thoroughly validate the collected\ndialogue for adherence of the participants to their given fact and opinion\nprofile, and find that the general quality in this respect is high. This\nprocess also gives us an additional layer of annotation that is potentially\nuseful for training models. We introduce as a baseline an end-to-end trained\nself-attention decoder model trained on this data and show that it is able to\ngenerate opinionated responses that are judged to be natural and knowledgeable\nand show attentiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13352v3",
            "date": "2020-03-30 11:34:53+00:00",
            "title": "Empirical Analysis of Zipf's Law, Power Law, and Lognormal Distributions in Medical Discharge Reports",
            "authors": [
                "Juan C Quiroz",
                "Liliana Laranjo",
                "Catalin Tufanaru",
                "Ahmet Baki Kocaballi",
                "Dana Rezazadegan",
                "Shlomo Berkovsky",
                "Enrico Coiera"
            ],
            "abstract": "Bayesian modelling and statistical text analysis rely on informed probability\npriors to encourage good solutions. This paper empirically analyses whether\ntext in medical discharge reports follow Zipf's law, a commonly assumed\nstatistical property of language where word frequency follows a discrete power\nlaw distribution. We examined 20,000 medical discharge reports from the\nMIMIC-III dataset. Methods included splitting the discharge reports into\ntokens, counting token frequency, fitting power law distributions to the data,\nand testing whether alternative distributions--lognormal, exponential,\nstretched exponential, and truncated power law--provided superior fits to the\ndata. Results show that discharge reports are best fit by the truncated power\nlaw and lognormal distributions. Our findings suggest that Bayesian modelling\nand statistical text analysis of discharge report text would benefit from using\ntruncated power law and lognormal probability priors.",
            "journal ref": "International Journal of Medical Informatics (2020), 104324",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13551v1",
            "date": "2020-03-30 15:25:34+00:00",
            "title": "European Language Grid: An Overview",
            "authors": [
                "Georg Rehm",
                "Maria Berger",
                "Ela Elsholz",
                "Stefanie Hegele",
                "Florian Kintzel",
                "Katrin Marheinecke",
                "Stelios Piperidis",
                "Miltos Deligiannis",
                "Dimitris Galanis",
                "Katerina Gkirtzou",
                "Penny Labropoulou",
                "Kalina Bontcheva",
                "David Jones",
                "Ian Roberts",
                "Jan Hajic",
                "Jana Hamrlov\u00e1",
                "Luk\u00e1\u0161 Ka\u010dena",
                "Khalid Choukri",
                "Victoria Arranz",
                "Andrejs Vasi\u013cjevs",
                "Orians Anvari",
                "Andis Lagzdi\u0146\u0161",
                "J\u016blija Me\u013c\u0146ika",
                "Gerhard Backfried",
                "Erin\u00e7 Dikici",
                "Miroslav Janosik",
                "Katja Prinz",
                "Christoph Prinz",
                "Severin Stampler",
                "Dorothea Thomas-Aniola",
                "Jos\u00e9 Manuel G\u00f3mez P\u00e9rez",
                "Andres Garcia Silva",
                "Christian Berr\u00edo",
                "Ulrich Germann",
                "Steve Renals",
                "Ondrej Klejch"
            ],
            "abstract": "With 24 official EU and many additional languages, multilingualism in Europe\nand an inclusive Digital Single Market can only be enabled through Language\nTechnologies (LTs). European LT business is dominated by hundreds of SMEs and a\nfew large players. Many are world-class, with technologies that outperform the\nglobal players. However, European LT business is also fragmented, by nation\nstates, languages, verticals and sectors, significantly holding back its\nimpact. The European Language Grid (ELG) project addresses this fragmentation\nby establishing the ELG as the primary platform for LT in Europe. The ELG is a\nscalable cloud platform, providing, in an easy-to-integrate way, access to\nhundreds of commercial and non-commercial LTs for all European languages,\nincluding running tools and services as well as data sets and resources. Once\nfully operational, it will enable the commercial and non-commercial European LT\ncommunity to deposit and upload their technologies and data sets into the ELG,\nto deploy them through the grid, and to connect with other resources. The ELG\nwill boost the Multilingual Digital Single Market towards a thriving European\nLT community, creating new jobs and opportunities. Furthermore, the ELG project\norganises two open calls for up to 20 pilot projects. It also sets up 32\nNational Competence Centres (NCCs) and the European LT Council (LTC) for\noutreach and coordination purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13785v1",
            "date": "2020-03-30 19:54:20+00:00",
            "title": "Span-based discontinuous constituency parsing: a family of exact chart-based algorithms with time complexities from O(n^6) down to O(n^3)",
            "authors": [
                "Caio Corro"
            ],
            "abstract": "We introduce a novel chart-based algorithm for span-based parsing of\ndiscontinuous constituency trees of block degree two, including ill-nested\nstructures. In particular, we show that we can build variants of our parser\nwith smaller search spaces and time complexities ranging from $\\mathcal O(n^6)$\ndown to $\\mathcal O(n^3)$. The cubic time variant covers 98\\% of constituents\nobserved in linguistic treebanks while having the same complexity as continuous\nconstituency parsers. We evaluate our approach on German and English treebanks\n(Negra, Tiger and Discontinuous PTB) and report state-of-the-art results in the\nfully supervised setting. We also experiment with pre-trained word embeddings\nand \\bert{}-based neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13878v1",
            "date": "2020-03-31 00:06:29+00:00",
            "title": "Procedural Reading Comprehension with Attribute-Aware Context Flow",
            "authors": [
                "Aida Amini",
                "Antoine Bosselut",
                "Bhavana Dalvi Mishra",
                "Yejin Choi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Procedural texts often describe processes (e.g., photosynthesis and cooking)\nthat happen over entities (e.g., light, food). In this paper, we introduce an\nalgorithm for procedural reading comprehension by translating the text into a\ngeneral formalism that represents processes as a sequence of transitions over\nentity attributes (e.g., location, temperature). Leveraging pre-trained\nlanguage models, our model obtains entity-aware and attribute-aware\nrepresentations of the text by joint prediction of entity attributes and their\ntransitions. Our model dynamically obtains contextual encodings of the\nprocedural text exploiting information that is encoded about previous and\ncurrent states to predict the transition of a certain attribute which can be\nidentified as a span of text or from a pre-defined set of classes. Moreover,\nour model achieves state of the art results on two procedural reading\ncomprehension datasets, namely ProPara and npn-cooking",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14026v1",
            "date": "2020-03-31 08:45:52+00:00",
            "title": "MULTEXT-East",
            "authors": [
                "Toma\u017e Erjavec"
            ],
            "abstract": "MULTEXT-East language resources, a multilingual dataset for language\nengineering research, focused on the morphosyntactic level of linguistic\ndescription. The MULTEXT-East dataset includes the EAGLES-based morphosyntactic\nspecifications, morphosyntactic lexicons, and an annotated multilingual\ncorpora. The parallel corpus, the novel \"1984\" by George Orwell, is sentence\naligned and contains hand-validated morphosyntactic descriptions and lemmas.\nThe resources are uniformly encoded in XML, using the Text Encoding Initiative\nGuidelines, TEI P5, and cover 16 languages: Bulgarian, Croatian, Czech,\nEnglish, Estonian, Hungarian, Macedonian, Persian, Polish, Resian, Romanian,\nRussian, Serbian, Slovak, Slovene, and Ukrainian. This dataset is extensively\ndocumented, and freely available for research purposes. This case study gives a\nhistory of the development of the MULTEXT-East resources, presents their\nencoding and components, discusses related work and gives some conclusions.",
            "journal ref": "Published in: Nancy Ide, James Pustejovsky, eds. 2007. Handbook of\n  linguistic annotation. pp. 441-462. Springer",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14155v6",
            "date": "2020-03-31 12:43:54+00:00",
            "title": "Appraisal Theories for Emotion Classification in Text",
            "authors": [
                "Jan Hofmann",
                "Enrica Troiano",
                "Kai Sassenberg",
                "Roman Klinger"
            ],
            "abstract": "Automatic emotion categorization has been predominantly formulated as text\nclassification in which textual units are assigned to an emotion from a\npredefined inventory, for instance following the fundamental emotion classes\nproposed by Paul Ekman (fear, joy, anger, disgust, sadness, surprise) or Robert\nPlutchik (adding trust, anticipation). This approach ignores existing\npsychological theories to some degree, which provide explanations regarding the\nperception of events. For instance, the description that somebody discovers a\nsnake is associated with fear, based on the appraisal as being an unpleasant\nand non-controllable situation. This emotion reconstruction is even possible\nwithout having access to explicit reports of a subjective feeling (for instance\nexpressing this with the words \"I am afraid.\"). Automatic classification\napproaches therefore need to learn properties of events as latent variables\n(for instance that the uncertainty and the mental or physical effort associated\nwith the encounter of a snake leads to fear). With this paper, we propose to\nmake such interpretations of events explicit, following theories of cognitive\nappraisal of events, and show their potential for emotion classification when\nbeing encoded in classification models. Our results show that high quality\nappraisal dimension assignments in event descriptions lead to an improvement in\nthe classification of discrete emotion categories. We make our corpus of\nappraisal-annotated emotion-associated event descriptions publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14282v1",
            "date": "2020-03-31 15:11:12+00:00",
            "title": "Inherent Dependency Displacement Bias of Transition-Based Algorithms",
            "authors": [
                "Mark Anderson",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "A wide variety of transition-based algorithms are currently used for\ndependency parsers. Empirical studies have shown that performance varies across\ndifferent treebanks in such a way that one algorithm outperforms another on one\ntreebank and the reverse is true for a different treebank. There is often no\ndiscernible reason for what causes one algorithm to be more suitable for a\ncertain treebank and less so for another. In this paper we shed some light on\nthis by introducing the concept of an algorithm's inherent dependency\ndisplacement distribution. This characterises the bias of the algorithm in\nterms of dependency displacement, which quantify both distance and direction of\nsyntactic relations. We show that the similarity of an algorithm's inherent\ndistribution to a treebank's displacement distribution is clearly correlated to\nthe algorithm's parsing performance on that treebank, specifically with highly\nsignificant and substantial correlations for the predominant sentence lengths\nin Universal Dependency treebanks. We also obtain results which show a more\ndiscrete analysis of dependency displacement does not result in any meaningful\ncorrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14386v1",
            "date": "2020-03-31 17:30:08+00:00",
            "title": "Evaluating Amharic Machine Translation",
            "authors": [
                "Asmelash Teka Hadgu",
                "Adam Beaudoin",
                "Abel Aregawi"
            ],
            "abstract": "Machine translation (MT) systems are now able to provide very accurate\nresults for high resource language pairs. However, for many low resource\nlanguages, MT is still under active research. In this paper, we develop and\nshare a dataset to automatically evaluate the quality of MT systems for\nAmharic. We compare two commercially available MT systems that support\ntranslation of Amharic to and from English to assess the current state of MT\nfor Amharic. The BLEU score results show that the results for Amharic\ntranslation are promising but still low. We hope that this dataset will be\nuseful to the research community both in academia and industry as a benchmark\nto evaluate Amharic MT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14402v1",
            "date": "2020-03-31 17:50:07+00:00",
            "title": "Low Resource Neural Machine Translation: A Benchmark for Five African Languages",
            "authors": [
                "Surafel M. Lakew",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Recent advents in Neural Machine Translation (NMT) have shown improvements in\nlow-resource language (LRL) translation tasks. In this work, we benchmark NMT\nbetween English and five African LRL pairs (Swahili, Amharic, Tigrigna, Oromo,\nSomali [SATOS]). We collected the available resources on the SATOS languages to\nevaluate the current state of NMT for LRLs. Our evaluation, comparing a\nbaseline single language pair NMT model against semi-supervised learning,\ntransfer learning, and multilingual modeling, shows significant performance\nimprovements both in the En-LRL and LRL-En directions. In terms of averaged\nBLEU score, the multilingual approach shows the largest gains, up to +5 points,\nin six out of ten translation directions. To demonstrate the generalization\ncapability of each model, we also report results on multi-domain test sets. We\nrelease the standardized experimental data and the test sets for future works\naddressing the challenges of NMT in under-resourced settings, in particular for\nthe SATOS languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00033v2",
            "date": "2020-03-31 18:01:56+00:00",
            "title": "Give your Text Representation Models some Love: the Case for Basque",
            "authors": [
                "Rodrigo Agerri",
                "I\u00f1aki San Vicente",
                "Jon Ander Campos",
                "Ander Barrena",
                "Xabier Saralegi",
                "Aitor Soroa",
                "Eneko Agirre"
            ],
            "abstract": "Word embeddings and pre-trained language models allow to build rich\nrepresentations of text and have enabled improvements across most NLP tasks.\nUnfortunately they are very expensive to train, and many small companies and\nresearch groups tend to use models that have been pre-trained and made\navailable by third parties, rather than building their own. This is suboptimal\nas, for many languages, the models have been trained on smaller (or lower\nquality) corpora. In addition, monolingual pre-trained models for non-English\nlanguages are not always available. At best, models for those languages are\nincluded in multilingual versions, where each language shares the quota of\nsubstrings and parameters with the rest of the languages. This is particularly\ntrue for smaller languages such as Basque. In this paper we show that a number\nof monolingual models (FastText word embeddings, FLAIR and BERT language\nmodels) trained with larger Basque corpora produce much better results than\npublicly available versions in downstream NLP tasks, including topic\nclassification, sentiment classification, PoS tagging and NER. This work sets a\nnew state-of-the-art in those tasks for Basque. All benchmarks and models used\nin this work are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00050v1",
            "date": "2020-03-31 18:28:36+00:00",
            "title": "Multilingual Stance Detection: The Catalonia Independence Corpus",
            "authors": [
                "Elena Zotova",
                "Rodrigo Agerri",
                "Manuel Nu\u00f1ez",
                "German Rigau"
            ],
            "abstract": "Stance detection aims to determine the attitude of a given text with respect\nto a specific topic or claim. While stance detection has been fairly well\nresearched in the last years, most the work has been focused on English. This\nis mainly due to the relative lack of annotated data in other languages. The\nTW-10 Referendum Dataset released at IberEval 2018 is a previous effort to\nprovide multilingual stance-annotated data in Catalan and Spanish.\nUnfortunately, the TW-10 Catalan subset is extremely imbalanced. This paper\naddresses these issues by presenting a new multilingual dataset for stance\ndetection in Twitter for the Catalan and Spanish languages, with the aim of\nfacilitating research on stance detection in multilingual and cross-lingual\nsettings. The dataset is annotated with stance towards one topic, namely, the\nindependence of Catalonia. We also provide a semi-automatic method to annotate\nthe dataset based on a categorization of Twitter users. We experiment on the\nnew corpus with a number of supervised approaches, including linear classifiers\nand deep learning methods. Comparison of our new corpus with the with the TW-1O\ndataset shows both the benefits and potential of a well balanced corpus for\nmultilingual and cross-lingual research on stance detection. Finally, we\nestablish new state-of-the-art results on the TW-10 dataset, both for Catalan\nand Spanish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00068v1",
            "date": "2020-03-31 19:28:35+00:00",
            "title": "Assessing Human Translations from French to Bambara for Machine Learning: a Pilot Study",
            "authors": [
                "Michael Leventhal",
                "Allahsera Tapo",
                "Sarah Luger",
                "Marcos Zampieri",
                "Christopher M. Homan"
            ],
            "abstract": "We present novel methods for assessing the quality of human-translated\naligned texts for learning machine translation models of under-resourced\nlanguages. Malian university students translated French texts, producing either\nwritten or oral translations to Bambara. Our results suggest that similar\nquality can be obtained from either written or spoken translations for certain\nkinds of texts. They also suggest specific instructions that human translators\nshould be given in order to improve the quality of their work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00089v1",
            "date": "2020-03-31 20:22:10+00:00",
            "title": "Automatic Extraction of Bengali Root Verbs using Paninian Grammar",
            "authors": [
                "Arijit Das",
                "Tapas Halder",
                "Diganta Saha"
            ],
            "abstract": "In this research work, we have proposed an algorithm based on supervised\nlearning methodology to extract the root forms of the Bengali verbs using the\ngrammatical rules proposed by Panini [1] in Ashtadhyayi. This methodology can\nbe applied for the languages which are derived from Sanskrit. The proposed\nsystem has been developed based on tense, person and morphological inflections\nof the verbs to find their root forms. The work has been executed in two\nphases: first, the surface level forms or inflected forms of the verbs have\nbeen classified into a certain number of groups of similar tense and person.\nFor this task, a standard pattern, available in Bengali language has been used.\nNext, a set of rules have been applied to extract the root form from the\nsurface level forms of a verb. The system has been tested on 10000 verbs\ncollected from the Bengali text corpus developed in the TDIL project of the\nGovt. of India. The accuracy of the output has been achieved 98% which is\nverified by a linguistic expert. Root verb identification is a key step in\nsemantic searching, multi-sentence search query processing, understanding the\nmeaning of a language, disambiguation of word sense, classification of the\nsentences etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00139v1",
            "date": "2020-03-31 22:10:43+00:00",
            "title": "A Swiss German Dictionary: Variation in Speech and Writing",
            "authors": [
                "Larissa Schmidt",
                "Lucy Linder",
                "Sandra Djambazovska",
                "Alexandros Lazaridis",
                "Tanja Samard\u017ei\u0107",
                "Claudiu Musat"
            ],
            "abstract": "We introduce a dictionary containing forms of common words in various Swiss\nGerman dialects normalized into High German. As Swiss German is, for now, a\npredominantly spoken language, there is a significant variation in the written\nforms, even between speakers of the same dialect. To alleviate the uncertainty\nassociated with this diversity, we complement the pairs of Swiss German - High\nGerman words with the Swiss German phonetic transcriptions (SAMPA). This\ndictionary becomes thus the first resource to combine large-scale spontaneous\ntranslation with phonetic transcriptions. Moreover, we control for the regional\ndistribution and insure the equal representation of the major Swiss dialects.\nThe coupling of the phonetic and written Swiss German forms is powerful. We\nshow that they are sufficient to train a Transformer-based phoneme to grapheme\nmodel that generates credible novel Swiss German writings. In addition, we show\nthat the inverse mapping - from graphemes to phonemes - can be modeled with a\ntransformer trained with the novel dictionary. This generation of\npronunciations for previously unknown words is key in training extensible\nautomated speech recognition (ASR) systems, which are key beneficiaries of this\ndictionary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00375v2",
            "date": "2020-04-01 12:24:47+00:00",
            "title": "Comparative Analysis of N-gram Text Representation on Igbo Text Document Similarity",
            "authors": [
                "Nkechi Ifeanyi-Reuben",
                "Chidiebere Ugwu",
                "Nwachukwu E. O"
            ],
            "abstract": "The improvement in Information Technology has encouraged the use of Igbo in\nthe creation of text such as resources and news articles online. Text\nsimilarity is of great importance in any text-based applications. This paper\npresents a comparative analysis of n-gram text representation on Igbo text\ndocument similarity. It adopted Euclidean similarity measure to determine the\nsimilarities between Igbo text documents represented with two word-based n-gram\ntext representation (unigram and bigram) models. The evaluation of the\nsimilarity measure is based on the adopted text representation models. The\nmodel is designed with Object-Oriented Methodology and implemented with Python\nprogramming language with tools from Natural Language Toolkits (NLTK). The\nresult shows that unigram represented text has highest distance values whereas\nbigram has the lowest corresponding distance values. The lower the distance\nvalue, the more similar the two documents and better the quality of the model\nwhen used for a task that requires similarity measure. The similarity of two\ndocuments increases as the distance value moves down to zero (0). Ideally, the\nresult analyzed revealed that Igbo text document similarity measured on bigram\nrepresented text gives accurate similarity result. This will give better,\neffective and accurate result when used for tasks such as text classification,\nclustering and ranking on Igbo text.",
            "journal ref": "International Journal of Applied Information Systems (IJAIS).\n  Volume 12 number 9, pages 1-7, December 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00798v1",
            "date": "2020-04-02 03:42:14+00:00",
            "title": "Mapping Languages: The Corpus of Global Language Use",
            "authors": [
                "Jonathan Dunn"
            ],
            "abstract": "This paper describes a web-based corpus of global language use with a focus\non how this corpus can be used for data-driven language mapping. First, the\ncorpus provides a representation of where national varieties of major languages\nare used (e.g., English, Arabic, Russian) together with consistently collected\ndata for each variety. Second, the paper evaluates a language identification\nmodel that supports more local languages with smaller sample sizes than\nalternative off-the-shelf models. Improved language identification is essential\nfor moving beyond majority languages. Given the focus on language mapping, the\npaper analyzes how well this digital language data represents actual\npopulations by (i) systematically comparing the corpus with demographic\nground-truth data and (ii) triangulating the corpus with an alternate\nTwitter-based dataset. In total, the corpus contains 423 billion words\nrepresenting 148 languages (with over 1 million words from each language) and\n158 countries (again with over 1 million words from each country), all\ndistilled from Common Crawl web data. The main contribution of this paper, in\naddition to describing this publicly-available corpus, is to provide a\ncomprehensive analysis of the relationship between two sources of digital data\n(the web and Twitter) as well as their connection to underlying populations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00809v1",
            "date": "2020-04-02 04:34:11+00:00",
            "title": "Mapping Languages and Demographics with Georeferenced Corpora",
            "authors": [
                "Jonathan Dunn",
                "Ben Adams"
            ],
            "abstract": "This paper evaluates large georeferenced corpora, taken from both web-crawled\nand social media sources, against ground-truth population and language-census\ndatasets. The goal is to determine (i) which dataset best represents population\ndemographics; (ii) in what parts of the world the datasets are most\nrepresentative of actual populations; and (iii) how to weight the datasets to\nprovide more accurate representations of underlying populations. The paper\nfinds that the two datasets represent very different populations and that they\ncorrelate with actual populations with values of r=0.60 (social media) and\nr=0.49 (web-crawled). Further, Twitter data makes better predictions about the\ninventory of languages used in each country.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00881v1",
            "date": "2020-04-02 08:58:44+00:00",
            "title": "How Furiously Can Colourless Green Ideas Sleep? Sentence Acceptability in Context",
            "authors": [
                "Jey Han Lau",
                "Carlos S. Armendariz",
                "Shalom Lappin",
                "Matthew Purver",
                "Chang Shu"
            ],
            "abstract": "We study the influence of context on sentence acceptability. First we compare\nthe acceptability ratings of sentences judged in isolation, with a relevant\ncontext, and with an irrelevant context. Our results show that context induces\na cognitive load for humans, which compresses the distribution of ratings.\nMoreover, in relevant contexts we observe a discourse coherence effect which\nuniformly raises acceptability. Next, we test unidirectional and bidirectional\nlanguage models in their ability to predict acceptability ratings. The\nbidirectional models show very promising results, with the best model achieving\na new state-of-the-art for unsupervised acceptability prediction. The two sets\nof experiments provide insights into the cognitive aspects of sentence\nprocessing and central issues in the computational modelling of text and\ndiscourse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01079v1",
            "date": "2020-04-02 15:40:59+00:00",
            "title": "Revisiting the linearity in cross-lingual embedding mappings: from a perspective of word analogies",
            "authors": [
                "Xutan Peng",
                "Chenghua Lin",
                "Mark Stevenson",
                "Chen li"
            ],
            "abstract": "Most cross-lingual embedding mapping algorithms assume the optimised\ntransformation functions to be linear. Recent studies showed that on some\noccasions, learning a linear mapping does not work, indicating that the\ncommonly-used assumption may fail. However, it still remains unclear under\nwhich conditions the linearity of cross-lingual embedding mappings holds. In\nthis paper, we rigorously explain that the linearity assumption relies on the\nconsistency of analogical relations encoded by multilingual embeddings. We did\nextensive experiments to validate this claim. Empirical results based on the\nanalogy completion benchmark and the BLI task demonstrate a strong correlation\nbetween whether mappings capture analogical information and are linear.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01092v1",
            "date": "2020-04-02 15:51:31+00:00",
            "title": "NUBES: A Corpus of Negation and Uncertainty in Spanish Clinical Texts",
            "authors": [
                "Salvador Lima",
                "Naiara Perez",
                "Montse Cuadros",
                "German Rigau"
            ],
            "abstract": "This paper introduces the first version of the NUBes corpus (Negation and\nUncertainty annotations in Biomedical texts in Spanish). The corpus is part of\nan on-going research and currently consists of 29,682 sentences obtained from\nanonymised health records annotated with negation and uncertainty. The article\nincludes an exhaustive comparison with similar corpora in Spanish, and presents\nthe main annotation and design decisions. Additionally, we perform preliminary\nexperiments using deep learning algorithms to validate the annotated dataset.\nAs far as we know, NUBes is the largest publicly available corpus for negation\nin Spanish and the first that also incorporates the annotation of speculation\ncues, scopes, and events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01174v1",
            "date": "2020-04-02 17:54:24+00:00",
            "title": "Causal Inference of Script Knowledge",
            "authors": [
                "Noah Weber",
                "Rachel Rudinger",
                "Benjamin Van Durme"
            ],
            "abstract": "When does a sequence of events define an everyday scenario and how can this\nknowledge be induced from text? Prior works in inducing such scripts have\nrelied on, in one form or another, measures of correlation between instances of\nevents in a corpus. We argue from both a conceptual and practical sense that a\npurely correlation-based approach is insufficient, and instead propose an\napproach to script induction based on the causal effect between events,\nformally defined via interventions. Through both human and automatic\nevaluations, we show that the output of our method based on causal effects\nbetter matches the intuition of what a script represents",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01267v2",
            "date": "2020-04-02 21:17:33+00:00",
            "title": "MZET: Memory Augmented Zero-Shot Fine-grained Named Entity Typing",
            "authors": [
                "Tao Zhang",
                "Congying Xia",
                "Chun-Ta Lu",
                "Philip Yu"
            ],
            "abstract": "Named entity typing (NET) is a classification task of assigning an entity\nmention in the context with given semantic types. However, with the growing\nsize and granularity of the entity types, rare researches in previous concern\nwith newly emerged entity types. In this paper, we propose MZET, a novel memory\naugmented FNET (Fine-grained NET) model, to tackle the unseen types in a\nzero-shot manner. MZET incorporates character-level, word-level, and\ncontextural-level information to learn the entity mention representation.\nBesides, MZET considers the semantic meaning and the hierarchical structure\ninto the entity type representation. Finally, through the memory component\nwhich models the relationship between the entity mention and the entity type,\nMZET transfer the knowledge from seen entity types to the zero-shot ones.\nExtensive experiments on three public datasets show prominent performance\nobtained by MZET, which surpasses the state-of-the-art FNET neural network\nmodels with up to 7% gain in Micro-F1 and Macro-F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01401v3",
            "date": "2020-04-03 07:03:12+00:00",
            "title": "XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation",
            "authors": [
                "Yaobo Liang",
                "Nan Duan",
                "Yeyun Gong",
                "Ning Wu",
                "Fenfei Guo",
                "Weizhen Qi",
                "Ming Gong",
                "Linjun Shou",
                "Daxin Jiang",
                "Guihong Cao",
                "Xiaodong Fan",
                "Ruofei Zhang",
                "Rahul Agrawal",
                "Edward Cui",
                "Sining Wei",
                "Taroon Bharti",
                "Ying Qiao",
                "Jiun-Hung Chen",
                "Winnie Wu",
                "Shuguang Liu",
                "Fan Yang",
                "Daniel Campos",
                "Rangan Majumder",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we introduce XGLUE, a new benchmark dataset that can be used\nto train large-scale cross-lingual pre-trained models using multilingual and\nbilingual corpora and evaluate their performance across a diverse set of\ncross-lingual tasks. Comparing to GLUE(Wang et al., 2019), which is labeled in\nEnglish for natural language understanding tasks only, XGLUE has two main\nadvantages: (1) it provides 11 diversified tasks that cover both natural\nlanguage understanding and generation scenarios; (2) for each task, it provides\nlabeled data in multiple languages. We extend a recent cross-lingual\npre-trained model Unicoder(Huang et al., 2019) to cover both understanding and\ngeneration tasks, which is evaluated on XGLUE as a strong baseline. We also\nevaluate the base versions (12-layer) of Multilingual BERT, XLM and XLM-R for\ncomparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01647v1",
            "date": "2020-04-03 16:11:57+00:00",
            "title": "Analyzing autoencoder-based acoustic word embeddings",
            "authors": [
                "Yevgen Matusevych",
                "Herman Kamper",
                "Sharon Goldwater"
            ],
            "abstract": "Recent studies have introduced methods for learning acoustic word embeddings\n(AWEs)---fixed-size vector representations of words which encode their acoustic\nfeatures. Despite the widespread use of AWEs in speech processing research,\nthey have only been evaluated quantitatively in their ability to discriminate\nbetween whole word tokens. To better understand the applications of AWEs in\nvarious downstream tasks and in cognitive modeling, we need to analyze the\nrepresentation spaces of AWEs. Here we analyze basic properties of AWE spaces\nlearned by a sequence-to-sequence encoder-decoder model in six typologically\ndiverse languages. We first show that these AWEs preserve some information\nabout words' absolute duration and speaker. At the same time, the\nrepresentation space of these AWEs is organized such that the distance between\nwords' embeddings increases with those words' phonetic dissimilarity. Finally,\nthe AWEs exhibit a word onset bias, similar to patterns reported in various\nstudies on human speech processing and lexical access. We argue this is a\npromising result and encourage further evaluation of AWEs as a potentially\nuseful tool in cognitive science, which could provide a link between speech\nprocessing and lexical memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01670v2",
            "date": "2020-04-03 16:51:33+00:00",
            "title": "Directions in Abusive Language Training Data: Garbage In, Garbage Out",
            "authors": [
                "Bertie Vidgen",
                "Leon Derczynski"
            ],
            "abstract": "Data-driven analysis and detection of abusive online content covers many\ndifferent tasks, phenomena, contexts, and methodologies. This paper\nsystematically reviews abusive language dataset creation and content in\nconjunction with an open website for cataloguing abusive language data. This\ncollection of knowledge leads to a synthesis providing evidence-based\nrecommendations for practitioners working with this complex and highly diverse\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01853v4",
            "date": "2020-04-04 05:06:26+00:00",
            "title": "Pre-training for Abstractive Document Summarization by Reinstating Source Text",
            "authors": [
                "Yanyan Zou",
                "Xingxing Zhang",
                "Wei Lu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Abstractive document summarization is usually modeled as a\nsequence-to-sequence (Seq2Seq) learning problem. Unfortunately, training large\nSeq2Seq based summarization models on limited supervised summarization data is\nchallenging. This paper presents three pre-training objectives which allow us\nto pre-train a Seq2Seq based abstractive summarization model on unlabeled text.\nThe main idea is that, given an input text artificially constructed from a\ndocument, a model is pre-trained to reinstate the original document. These\nobjectives include sentence reordering, next sentence generation, and masked\ndocument generation, which have close relations with the abstractive document\nsummarization task. Experiments on two benchmark summarization datasets (i.e.,\nCNN/DailyMail and New York Times) show that all three objectives can improve\nperformance upon baselines. Compared to models pre-trained on large-scale data\n(more than 160GB), our method, with only 19GB text for pre-training, achieves\ncomparable results, which demonstrates its effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01878v1",
            "date": "2020-04-04 07:17:16+00:00",
            "title": "News-Driven Stock Prediction With Attention-Based Noisy Recurrent State Transition",
            "authors": [
                "Xiao Liu",
                "Heyan Huang",
                "Yue Zhang",
                "Changsen Yuan"
            ],
            "abstract": "We consider direct modeling of underlying stock value movement sequences over\ntime in the news-driven stock movement prediction. A recurrent state transition\nmodel is constructed, which better captures a gradual process of stock movement\ncontinuously by modeling the correlation between past and future price\nmovements. By separating the effects of news and noise, a noisy random factor\nis also explicitly fitted based on the recurrent states. Results show that the\nproposed model outperforms strong baselines. Thanks to the use of attention\nover news events, our model is also more explainable. To our knowledge, we are\nthe first to explicitly model both events and noise over a fundamental stock\nvalue state for news-driven stock movement prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01894v1",
            "date": "2020-04-04 09:03:04+00:00",
            "title": "Evaluating Multimodal Representations on Visual Semantic Textual Similarity",
            "authors": [
                "Oier Lopez de Lacalle",
                "Ander Salaberria",
                "Aitor Soroa",
                "Gorka Azkune",
                "Eneko Agirre"
            ],
            "abstract": "The combination of visual and textual representations has produced excellent\nresults in tasks such as image captioning and visual question answering, but\nthe inference capabilities of multimodal representations are largely untested.\nIn the case of textual representations, inference tasks such as Textual\nEntailment and Semantic Textual Similarity have been often used to benchmark\nthe quality of textual representations. The long term goal of our research is\nto devise multimodal representation techniques that improve current inference\ncapabilities. We thus present a novel task, Visual Semantic Textual Similarity\n(vSTS), where such inference ability can be tested directly. Given two items\ncomprised each by an image and its accompanying caption, vSTS systems need to\nassess the degree to which the captions in context are semantically equivalent\nto each other. Our experiments using simple multimodal representations show\nthat the addition of image representations produces better inference, compared\nto text-only representations. The improvement is observed both when directly\ncomputing the similarity between the representations of the two items, and when\nlearning a siamese network based on vSTS training data. Our work shows, for the\nfirst time, the successful contribution of visual information to textual\ninference, with ample room for benchmarking more complex multimodal\nrepresentation options.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01912v2",
            "date": "2020-04-04 11:45:27+00:00",
            "title": "Benchmarking Machine Reading Comprehension: A Psychological Perspective",
            "authors": [
                "Saku Sugawara",
                "Pontus Stenetorp",
                "Akiko Aizawa"
            ],
            "abstract": "Machine reading comprehension (MRC) has received considerable attention as a\nbenchmark for natural language understanding. However, the conventional task\ndesign of MRC lacks explainability beyond the model interpretation, i.e.,\nreading comprehension by a model cannot be explained in human terms. To this\nend, this position paper provides a theoretical basis for the design of MRC\ndatasets based on psychology as well as psychometrics, and summarizes it in\nterms of the prerequisites for benchmarking MRC. We conclude that future\ndatasets should (i) evaluate the capability of the model for constructing a\ncoherent and grounded representation to understand context-dependent situations\nand (ii) ensure substantive validity by shortcut-proof questions and\nexplanation as a part of the task design.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01926v2",
            "date": "2020-04-04 13:06:03+00:00",
            "title": "\"None of the Above\":Measure Uncertainty in Dialog Response Retrieval",
            "authors": [
                "Yulan Feng",
                "Shikib Mehri",
                "Maxine Eskenazi",
                "Tiancheng Zhao"
            ],
            "abstract": "This paper discusses the importance of uncovering uncertainty in end-to-end\ndialog tasks, and presents our experimental results on uncertainty\nclassification on the Ubuntu Dialog Corpus. We show that, instead of retraining\nmodels for this specific purpose, the original retrieval model's underlying\nconfidence concerning the best prediction can be captured with trivial\nadditional computation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01935v1",
            "date": "2020-04-04 13:49:54+00:00",
            "title": "An Iterative Knowledge Transfer Network with Routing for Aspect-based Sentiment Analysis",
            "authors": [
                "Yunlong Liang",
                "Fandong Meng",
                "Jinchao Zhang",
                "Jinan Xu",
                "Yufeng Chen",
                "Jie Zhou"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) mainly involves three subtasks: aspect\nterm extraction, opinion term extraction and aspect-level sentiment\nclassification, which are typically handled separately or (partially) jointly.\nHowever, the semantic interrelationships among all the three subtasks are not\nwell exploited in previous approaches, which restricts their performance.\nAdditionally, the linguistic knowledge from document-level labeled sentiment\ncorpora is usually used in a coarse way for the ABSA. To address these issues,\nwe propose a novel Iterative Knowledge Transfer Network (IKTN) for the\nend-to-end ABSA. For one thing, to fully exploit the semantic correlations\namong the three aspect-level subtasks for mutual promotion, the IKTN transfers\nthe task-specific knowledge from any two of the three subtasks to another one\nby leveraging a specially-designed routing algorithm, that is, any two of the\nthree subtasks will help the third one. Besides, the IKTN discriminately\ntransfers the document-level linguistic knowledge, i.e., domain-specific and\nsentiment-related knowledge, to the aspect-level subtasks to benefit the\ncorresponding ones. Experimental results on three benchmark datasets\ndemonstrate the effectiveness of our approach, which significantly outperforms\nexisting state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01940v1",
            "date": "2020-04-04 14:14:43+00:00",
            "title": "Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems",
            "authors": [
                "Jia-Chen Gu",
                "Tianda Li",
                "Quan Liu",
                "Xiaodan Zhu",
                "Zhen-Hua Ling",
                "Yu-Ping Ruan"
            ],
            "abstract": "The NOESIS II challenge, as the Track 2 of the 8th Dialogue System Technology\nChallenges (DSTC 8), is the extension of DSTC 7. This track incorporates new\nelements that are vital for the creation of a deployed task-oriented dialogue\nsystem. This paper describes our systems that are evaluated on all subtasks\nunder this challenge. We study the problem of employing pre-trained\nattention-based network for multi-turn dialogue systems. Meanwhile, several\nadaptation methods are proposed to adapt the pre-trained language models for\nmulti-turn dialogue systems, in order to keep the intrinsic property of\ndialogue systems. In the released evaluation results of Track 2 of DSTC 8, our\nproposed models ranked fourth in subtask 1, third in subtask 2, and first in\nsubtask 3 and subtask 4 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01951v1",
            "date": "2020-04-04 14:59:32+00:00",
            "title": "A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis",
            "authors": [
                "Yunlong Liang",
                "Fandong Meng",
                "Jinchao Zhang",
                "Jinan Xu",
                "Yufeng Chen",
                "Jie Zhou"
            ],
            "abstract": "The aspect-based sentiment analysis (ABSA) task remains to be a long-standing\nchallenge, which aims to extract the aspect term and then identify its\nsentiment orientation.In previous approaches, the explicit syntactic structure\nof a sentence, which reflects the syntax properties of natural language and\nhence is intuitively crucial for aspect term extraction and sentiment\nrecognition, is typically neglected or insufficiently modeled. In this paper,\nwe thus propose a novel dependency syntactic knowledge augmented interactive\narchitecture with multi-task learning for end-to-end ABSA. This model is\ncapable of fully exploiting the syntactic knowledge (dependency relations and\ntypes) by leveraging a well-designed Dependency Relation Embedded Graph\nConvolutional Network (DreGcn). Additionally, we design a simple yet effective\nmessage-passing mechanism to ensure that our model learns from multiple related\ntasks in a multi-task learning framework. Extensive experimental results on\nthree benchmark datasets demonstrate the effectiveness of our approach, which\nsignificantly outperforms existing state-of-the-art methods. Besides, we\nachieve further improvements by using BERT as an additional feature extractor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01970v3",
            "date": "2020-04-04 16:25:48+00:00",
            "title": "BAE: BERT-based Adversarial Examples for Text Classification",
            "authors": [
                "Siddhant Garg",
                "Goutham Ramakrishnan"
            ],
            "abstract": "Modern text classification models are susceptible to adversarial examples,\nperturbed versions of the original text indiscernible by humans which get\nmisclassified by the model. Recent works in NLP use rule-based synonym\nreplacement strategies to generate adversarial examples. These strategies can\nlead to out-of-context and unnaturally complex token replacements, which are\neasily identifiable by humans. We present BAE, a black box attack for\ngenerating adversarial examples using contextual perturbations from a BERT\nmasked language model. BAE replaces and inserts tokens in the original text by\nmasking a portion of the text and leveraging the BERT-MLM to generate\nalternatives for the masked tokens. Through automatic and human evaluations, we\nshow that BAE performs a stronger attack, in addition to generating adversarial\nexamples with improved grammaticality and semantic coherence as compared to\nprior work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01972v2",
            "date": "2020-04-04 16:37:00+00:00",
            "title": "Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks",
            "authors": [
                "Yufan Zhao",
                "Can Xu",
                "Wei Wu",
                "Lei Yu"
            ],
            "abstract": "We study multi-turn response generation for open-domain dialogues. The\nexisting state-of-the-art addresses the problem with deep neural architectures.\nWhile these models improved response quality, their complexity also hinders the\napplication of the models in real systems. In this work, we pursue a model that\nhas a simple structure yet can effectively leverage conversation contexts for\nresponse generation. To this end, we propose four auxiliary tasks including\nword order recovery, utterance order recovery, masked word recovery, and masked\nutterance recovery, and optimize the objectives of these tasks together with\nmaximizing the likelihood of generation. By this means, the auxiliary tasks\nthat relate to context understanding can guide the learning of the generation\nmodel to achieve a better local optimum. Empirical studies with three\nbenchmarks indicate that our model can significantly outperform\nstate-of-the-art generation models in terms of response quality on both\nautomatic evaluation and human judgment, and at the same time enjoys a much\nfaster decoding process.",
            "journal ref": "EMNLP2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02015v3",
            "date": "2020-04-04 20:56:37+00:00",
            "title": "Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection",
            "authors": [
                "Hanjie Chen",
                "Guangtao Zheng",
                "Yangfeng Ji"
            ],
            "abstract": "Generating explanations for neural networks has become crucial for their\napplications in real-world with respect to reliability and trustworthiness. In\nnatural language processing, existing methods usually provide important\nfeatures which are words or phrases selected from an input text as an\nexplanation, but ignore the interactions between them. It poses challenges for\nhumans to interpret an explanation and connect it to model prediction. In this\nwork, we build hierarchical explanations by detecting feature interactions.\nSuch explanations visualize how words and phrases are combined at different\nlevels of the hierarchy, which can help users understand the decision-making of\nblack-box models. The proposed method is evaluated with three neural text\nclassifiers (LSTM, CNN, and BERT) on two benchmark datasets, via both automatic\nand human evaluations. Experiments show the effectiveness of the proposed\nmethod in providing explanations that are both faithful to models and\ninterpretable to humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02016v4",
            "date": "2020-04-04 21:00:41+00:00",
            "title": "A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining",
            "authors": [
                "Chenguang Zhu",
                "Ruochen Xu",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "With the abundance of automatic meeting transcripts, meeting summarization is\nof great interest to both participants and other parties. Traditional methods\nof summarizing meetings depend on complex multi-step pipelines that make joint\noptimization intractable. Meanwhile, there are a handful of deep neural models\nfor text summarization and dialogue systems. However, the semantic structure\nand styles of meeting transcripts are quite different from articles and\nconversations. In this paper, we propose a novel abstractive summary network\nthat adapts to the meeting scenario. We design a hierarchical structure to\naccommodate long meeting transcripts and a role vector to depict the difference\namong speakers. Furthermore, due to the inadequacy of meeting summary data, we\npretrain the model on large-scale news summary data. Empirical results show\nthat our model outperforms previous approaches in both automatic metrics and\nhuman evaluation. For example, on ICSI dataset, the ROUGE-1 score increases\nfrom 34.66% to 46.28%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02071v1",
            "date": "2020-04-05 02:14:14+00:00",
            "title": "Incorporating Bilingual Dictionaries for Low Resource Semi-Supervised Neural Machine Translation",
            "authors": [
                "Sreyashi Nag",
                "Mihir Kale",
                "Varun Lakshminarasimhan",
                "Swapnil Singhavi"
            ],
            "abstract": "We explore ways of incorporating bilingual dictionaries to enable\nsemi-supervised neural machine translation. Conventional back-translation\nmethods have shown success in leveraging target side monolingual data. However,\nsince the quality of back-translation models is tied to the size of the\navailable parallel corpora, this could adversely impact the synthetically\ngenerated sentences in a low resource setting. We propose a simple data\naugmentation technique to address both this shortcoming. We incorporate widely\navailable bilingual dictionaries that yield word-by-word translations to\ngenerate synthetic sentences. This automatically expands the vocabulary of the\nmodel while maintaining high quality content. Our method shows an appreciable\nimprovement in performance over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02077v1",
            "date": "2020-04-05 02:47:16+00:00",
            "title": "Machine Translation Pre-training for Data-to-Text Generation -- A Case Study in Czech",
            "authors": [
                "Mihir Kale",
                "Scott Roy"
            ],
            "abstract": "While there is a large body of research studying deep learning methods for\ntext generation from structured data, almost all of it focuses purely on\nEnglish. In this paper, we study the effectiveness of machine translation based\npre-training for data-to-text generation in non-English languages. Since the\nstructured data is generally expressed in English, text generation into other\nlanguages involves elements of translation, transliteration and copying -\nelements already encoded in neural machine translation systems. Moreover, since\ndata-to-text corpora are typically small, this task can benefit greatly from\npre-training. Based on our experiments on Czech, a morphologically complex\nlanguage, we find that pre-training lets us train end-to-end models with\nsignificantly improved performance, as judged by automatic metrics and human\nevaluation. We also show that this approach enjoys several desirable\nproperties, including improved performance in low data scenarios and robustness\nto unseen slot values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02083v1",
            "date": "2020-04-05 03:30:01+00:00",
            "title": "A Resource for Studying Chatino Verbal Morphology",
            "authors": [
                "Hilaria Cruz",
                "Gregory Stump",
                "Antonios Anastasopoulos"
            ],
            "abstract": "We present the first resource focusing on the verbal inflectional morphology\nof San Juan Quiahije Chatino, a tonal mesoamerican language spoken in Mexico.\nWe provide a collection of complete inflection tables of 198 lemmata, with\nmorphological tags based on the UniMorph schema. We also provide baseline\nresults on three core NLP tasks: morphological analysis, lemmatization, and\nmorphological inflection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02105v2",
            "date": "2020-04-05 06:22:16+00:00",
            "title": "Unsupervised Domain Clusters in Pretrained Language Models",
            "authors": [
                "Roee Aharoni",
                "Yoav Goldberg"
            ],
            "abstract": "The notion of \"in-domain data\" in NLP is often over-simplistic and vague, as\ntextual data varies in many nuanced linguistic aspects such as topic, style or\nlevel of formality. In addition, domain labels are many times unavailable,\nmaking it challenging to build domain-specific systems. We show that massive\npre-trained language models implicitly learn sentence representations that\ncluster by domains without supervision -- suggesting a simple data-driven\ndefinition of domains in textual data. We harness this property and propose\ndomain data selection methods based on such models, which require only a small\nset of in-domain monolingual data. We evaluate our data selection methods for\nneural machine translation across five diverse domains, where they outperform\nan established approach as measured by both BLEU and by precision and recall of\nsentence selection with respect to an oracle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02118v1",
            "date": "2020-04-05 07:51:23+00:00",
            "title": "GIANT: Scalable Creation of a Web-scale Ontology",
            "authors": [
                "Bang Liu",
                "Weidong Guo",
                "Di Niu",
                "Jinwen Luo",
                "Chaoyue Wang",
                "Zhen Wen",
                "Yu Xu"
            ],
            "abstract": "Understanding what online users may pay attention to is key to content\nrecommendation and search services. These services will benefit from a highly\nstructured and web-scale ontology of entities, concepts, events, topics and\ncategories. While existing knowledge bases and taxonomies embody a large volume\nof entities and categories, we argue that they fail to discover properly\ngrained concepts, events and topics in the language style of online population.\nNeither is a logically structured ontology maintained among these notions. In\nthis paper, we present GIANT, a mechanism to construct a user-centered,\nweb-scale, structured ontology, containing a large number of natural language\nphrases conforming to user attentions at various granularities, mined from a\nvast volume of web documents and search click graphs. Various types of edges\nare also constructed to maintain a hierarchy in the ontology. We present our\ngraph-neural-network-based techniques used in GIANT, and evaluate the proposed\nmethods as compared to a variety of baselines. GIANT has produced the Attention\nOntology, which has been deployed in various Tencent applications involving\nover a billion users. Online A/B testing performed on Tencent QQ Browser shows\nthat Attention Ontology can significantly improve click-through rates in news\nrecommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02127v2",
            "date": "2020-04-05 08:28:08+00:00",
            "title": "Reference Language based Unsupervised Neural Machine Translation",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "Exploiting a common language as an auxiliary for better translation has a\nlong tradition in machine translation and lets supervised learning-based\nmachine translation enjoy the enhancement delivered by the well-used pivot\nlanguage in the absence of a source language to target language parallel\ncorpus. The rise of unsupervised neural machine translation (UNMT) almost\ncompletely relieves the parallel corpus curse, though UNMT is still subject to\nunsatisfactory performance due to the vagueness of the clues available for its\ncore back-translation training. Further enriching the idea of pivot translation\nby extending the use of parallel corpora beyond the source-target paradigm, we\npropose a new reference language-based framework for UNMT, RUNMT, in which the\nreference language only shares a parallel corpus with the source, but this\ncorpus still indicates a signal clear enough to help the reconstruction\ntraining of UNMT through a proposed reference agreement mechanism. Experimental\nresults show that our methods improve the quality of UNMT over that of a strong\nbaseline that uses only one auxiliary language, demonstrating the usefulness of\nthe proposed reference language-based UNMT and establishing a good start for\nthe community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02143v4",
            "date": "2020-04-05 10:16:59+00:00",
            "title": "Reinforced Multi-task Approach for Multi-hop Question Generation",
            "authors": [
                "Deepak Gupta",
                "Hardik Chauhan",
                "Akella Ravi Tej",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Question generation (QG) attempts to solve the inverse of question answering\n(QA) problem by generating a natural language question given a document and an\nanswer. While sequence to sequence neural models surpass rule-based systems for\nQG, they are limited in their capacity to focus on more than one supporting\nfact. For QG, we often require multiple supporting facts to generate\nhigh-quality questions. Inspired by recent works on multi-hop reasoning in QA,\nwe take up Multi-hop question generation, which aims at generating relevant\nquestions based on supporting facts in the context. We employ multitask\nlearning with the auxiliary task of answer-aware supporting fact prediction to\nguide the question generator. In addition, we also proposed a question-aware\nreward function in a Reinforcement Learning (RL) framework to maximize the\nutilization of the supporting facts. We demonstrate the effectiveness of our\napproach through experiments on the multi-hop question answering dataset,\nHotPotQA. Empirical evaluation shows our model to outperform the single-hop\nneural question generation models on both automatic evaluation metrics such as\nBLEU, METEOR, and ROUGE, and human evaluation metrics for quality and coverage\nof the generated questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02178v2",
            "date": "2020-04-05 12:29:20+00:00",
            "title": "FastBERT: a Self-distilling BERT with Adaptive Inference Time",
            "authors": [
                "Weijie Liu",
                "Peng Zhou",
                "Zhe Zhao",
                "Zhiruo Wang",
                "Haotang Deng",
                "Qi Ju"
            ],
            "abstract": "Pre-trained language models like BERT have proven to be highly performant.\nHowever, they are often computationally expensive in many practical scenarios,\nfor such heavy models can hardly be readily implemented with limited resources.\nTo improve their efficiency with an assured model performance, we propose a\nnovel speed-tunable FastBERT with adaptive inference time. The speed at\ninference can be flexibly adjusted under varying demands, while redundant\ncalculation of samples is avoided. Moreover, this model adopts a unique\nself-distillation mechanism at fine-tuning, further enabling a greater\ncomputational efficacy with minimal loss in performance. Our model achieves\npromising results in twelve English and Chinese datasets. It is able to speed\nup by a wide range from 1 to 12 times than BERT if given different speedup\nthresholds to make a speed-performance tradeoff.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02181v1",
            "date": "2020-04-05 12:33:51+00:00",
            "title": "Detecting and Understanding Generalization Barriers for Neural Machine Translation",
            "authors": [
                "Guanlin Li",
                "Lemao Liu",
                "Conghui Zhu",
                "Tiejun Zhao",
                "Shuming Shi"
            ],
            "abstract": "Generalization to unseen instances is our eternal pursuit for all data-driven\nmodels. However, for realistic task like machine translation, the traditional\napproach measuring generalization in an average sense provides poor\nunderstanding for the fine-grained generalization ability. As a remedy, this\npaper attempts to identify and understand generalization barrier words within\nan unseen input sentence that \\textit{cause} the degradation of fine-grained\ngeneralization. We propose a principled definition of generalization barrier\nwords and a modified version which is tractable in computation. Based on the\nmodified one, we propose three simple methods for barrier detection by the\nsearch-aware risk estimation through counterfactual generation. We then conduct\nextensive analyses on those detected generalization barrier words on both\nZh$\\Leftrightarrow$En NIST benchmarks from various perspectives. Potential\nusage of the detected barrier words is also discussed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02192v3",
            "date": "2020-04-05 13:05:11+00:00",
            "title": "Arabic Offensive Language on Twitter: Analysis and Experiments",
            "authors": [
                "Hamdy Mubarak",
                "Ammar Rashed",
                "Kareem Darwish",
                "Younes Samih",
                "Ahmed Abdelali"
            ],
            "abstract": "Detecting offensive language on Twitter has many applications ranging from\ndetecting/predicting bullying to measuring polarization. In this paper, we\nfocus on building a large Arabic offensive tweet dataset. We introduce a method\nfor building a dataset that is not biased by topic, dialect, or target. We\nproduce the largest Arabic dataset to date with special tags for vulgarity and\nhate speech. We thoroughly analyze the dataset to determine which topics,\ndialects, and gender are most associated with offensive tweets and how Arabic\nspeakers use offensive language. Lastly, we conduct many experiments to produce\nstrong results (F1 = 83.2) on the dataset using SOTA techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02196v1",
            "date": "2020-04-05 13:18:18+00:00",
            "title": "AR: Auto-Repair the Synthetic Data for Neural Machine Translation",
            "authors": [
                "Shanbo Cheng",
                "Shaohui Kuang",
                "Rongxiang Weng",
                "Heng Yu",
                "Changfeng Zhu",
                "Weihua Luo"
            ],
            "abstract": "Compared with only using limited authentic parallel data as training corpus,\nmany studies have proved that incorporating synthetic parallel data, which\ngenerated by back translation (BT) or forward translation (FT, or\nselftraining), into the NMT training process can significantly improve\ntranslation quality. However, as a well-known shortcoming, synthetic parallel\ndata is noisy because they are generated by an imperfect NMT system. As a\nresult, the improvements in translation quality bring by the synthetic parallel\ndata are greatly diminished. In this paper, we propose a novel Auto- Repair\n(AR) framework to improve the quality of synthetic data. Our proposed AR model\ncan learn the transformation from low quality (noisy) input sentence to high\nquality sentence based on large scale monolingual data with BT and FT\ntechniques. The noise in synthetic parallel data will be sufficiently\neliminated by the proposed AR model and then the repaired synthetic parallel\ndata can help the NMT models to achieve larger improvements. Experimental\nresults show that our approach can effective improve the quality of synthetic\nparallel data and the NMT model with the repaired synthetic data achieves\nconsistent improvements on both WMT14 EN!DE and IWSLT14 DE!EN translation\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02202v1",
            "date": "2020-04-05 13:58:14+00:00",
            "title": "Stylistic Dialogue Generation via Information-Guided Reinforcement Learning Strategy",
            "authors": [
                "Yixuan Su",
                "Deng Cai",
                "Yan Wang",
                "Simon Baker",
                "Anna Korhonen",
                "Nigel Collier",
                "Xiaojiang Liu"
            ],
            "abstract": "Stylistic response generation is crucial for building an engaging dialogue\nsystem for industrial use. While it has attracted much research interest,\nexisting methods often generate stylistic responses at the cost of the content\nquality (relevance and fluency). To enable better balance between the content\nquality and the style, we introduce a new training strategy, know as\nInformation-Guided Reinforcement Learning (IG-RL). In IG-RL, a training model\nis encouraged to explore stylistic expressions while being constrained to\nmaintain its content quality. This is achieved by adopting reinforcement\nlearning strategy with statistical style information guidance for\nquality-preserving explorations. Experiments on two datasets show that the\nproposed approach outperforms several strong baselines in terms of the overall\nresponse performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02211v2",
            "date": "2020-04-05 14:29:40+00:00",
            "title": "Syntax-driven Iterative Expansion Language Models for Controllable Text Generation",
            "authors": [
                "Noe Casas",
                "Jos\u00e9 A. R. Fonollosa",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "The dominant language modeling paradigm handles text as a sequence of\ndiscrete tokens. While that approach can capture the latent structure of the\ntext, it is inherently constrained to sequential dynamics for text generation.\nWe propose a new paradigm for introducing a syntactic inductive bias into\nneural text generation, where the dependency parse tree is used to drive the\nTransformer model to generate sentences iteratively.\n  Our experiments show that this paradigm is effective at text generation, with\nquality between LSTMs and Transformers, and comparable diversity, requiring\nless than half their decoding steps, and its generation process allows direct\ncontrol over the syntactic constructions of the generated text, enabling the\ninduction of stylistic variations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02214v1",
            "date": "2020-04-05 14:36:15+00:00",
            "title": "Prototype-to-Style: Dialogue Generation with Style-Aware Editing on Retrieval Memory",
            "authors": [
                "Yixuan Su",
                "Yan Wang",
                "Simon Baker",
                "Deng Cai",
                "Xiaojiang Liu",
                "Anna Korhonen",
                "Nigel Collier"
            ],
            "abstract": "The ability of a dialog system to express prespecified language style during\nconversations has a direct, positive impact on its usability and on user\nsatisfaction. We introduce a new prototype-to-style (PS) framework to tackle\nthe challenge of stylistic dialogue generation. The framework uses an\nInformation Retrieval (IR) system and extracts a response prototype from the\nretrieved response. A stylistic response generator then takes the prototype and\nthe desired language style as model input to obtain a high-quality and\nstylistic response. To effectively train the proposed model, we propose a new\nstyle-aware learning objective as well as a de-noising learning strategy.\nResults on three benchmark datasets from two languages demonstrate that the\nproposed approach significantly outperforms existing baselines in both\nin-domain and cross-domain evaluations",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02251v1",
            "date": "2020-04-05 16:55:09+00:00",
            "title": "Semantics of the Unwritten",
            "authors": [
                "He Bai",
                "Peng Shi",
                "Jimmy Lin",
                "Luchen Tan",
                "Kun Xiong",
                "Wen Gao",
                "Jie Liu",
                "Ming Li"
            ],
            "abstract": "The semantics of a text is manifested not only by what is read, but also by\nwhat is not read. In this article, we will study how those implicit \"not read\"\ninformation such as end-of-paragraph (EOP) and end-of-sequence (EOS) affect the\nquality of text generation. Transformer-based pretrained language models (LMs)\nhave demonstrated the ability to generate long continuations with good quality.\nThis model gives us a platform for the first time to demonstrate that paragraph\nlayouts and text endings are also important components of human writing.\nSpecifically, we find that pretrained LMs can generate better continuations by\nlearning to generate the end of the paragraph (EOP) in the fine-tuning stage.\nExperimental results on English story generation show that EOP can lead to\nhigher BLEU score and lower EOS perplexity. To further investigate the\nrelationship between text ending and EOP, we conduct experiments with a\nself-collected Chinese essay dataset on Chinese-GPT2, a character level LM\nwithout paragraph breaker or EOS during pre-training. Experimental results show\nthat the Chinese GPT2 can generate better essay endings with paragraph\ninformation. Experiments on both English stories and Chinese essays demonstrate\nthat learning to end paragraphs can benefit the continuation generation with\npretrained LMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02286v1",
            "date": "2020-04-05 19:27:18+00:00",
            "title": "Hierarchical Entity Typing via Multi-level Learning to Rank",
            "authors": [
                "Tongfei Chen",
                "Yunmo Chen",
                "Benjamin Van Durme"
            ],
            "abstract": "We propose a novel method for hierarchical entity classification that\nembraces ontological structure at both training and during prediction. At\ntraining, our novel multi-level learning-to-rank loss compares positive types\nagainst negative siblings according to the type tree. During prediction, we\ndefine a coarse-to-fine decoder that restricts viable candidates at each level\nof the ontology based on already predicted parent type(s). We achieve\nstate-of-the-art across multiple datasets, particularly with respect to strict\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02346v2",
            "date": "2020-04-05 23:00:09+00:00",
            "title": "Domain-based Latent Personal Analysis and its use for impersonation detection in social media",
            "authors": [
                "Osnat Mokryn",
                "Hagit Ben-Shoshan"
            ],
            "abstract": "Zipf's law defines an inverse proportion between a word's ranking in a given\ncorpus and its frequency in it, roughly dividing the vocabulary into frequent\nwords and infrequent ones. Here, we stipulate that within a domain an author's\nsignature can be derived from, in loose terms, the author's missing popular\nwords and frequently used infrequent-words. We devise a method, termed Latent\nPersonal Analysis (LPA), for finding domain-based attributes for entities in a\ndomain: their distance from the domain and their signature, which determines\nhow they most differ from a domain. We identify the most suitable distance\nmetric for the method among several and construct the distances and personal\nsignatures for authors, the domain's entities. The signature consists of both\nover-used terms (compared to the average), and missing popular terms. We\nvalidate the correctness and power of the signatures in identifying users and\nset existence conditions. We then show uses for the method in explainable\nauthorship attribution: we define algorithms that utilize LPA to identify two\ntypes of impersonation in social media: (1) authors with sockpuppets (multiple)\naccounts; (2) front users accounts, operated by several authors. We validate\nthe algorithms and employ them over a large scale dataset obtained from a\nsocial media site with over 4000 users. We corroborate these results using\ntemporal rate analysis. LPA can further be used to devise personal attributes\nin a wide range of scientific domains in which the constituents have a\nlong-tail distribution of elements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02393v1",
            "date": "2020-04-06 03:54:38+00:00",
            "title": "Learning to Recover Reasoning Chains for Multi-Hop Question Answering via Cooperative Games",
            "authors": [
                "Yufei Feng",
                "Mo Yu",
                "Wenhan Xiong",
                "Xiaoxiao Guo",
                "Junjie Huang",
                "Shiyu Chang",
                "Murray Campbell",
                "Michael Greenspan",
                "Xiaodan Zhu"
            ],
            "abstract": "We propose the new problem of learning to recover reasoning chains from\nweakly supervised signals, i.e., the question-answer pairs. We propose a\ncooperative game approach to deal with this problem, in which how the evidence\npassages are selected and how the selected passages are connected are handled\nby two models that cooperate to select the most confident chains from a large\nset of candidates (from distant supervision). For evaluation, we created\nbenchmarks based on two multi-hop QA datasets, HotpotQA and MedHop; and\nhand-labeled reasoning chains for the latter. The experimental results\ndemonstrate the effectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02399v1",
            "date": "2020-04-06 04:36:33+00:00",
            "title": "PONE: A Novel Automatic Evaluation Metric for Open-Domain Generative Dialogue Systems",
            "authors": [
                "Tian Lan",
                "Xian-Ling Mao",
                "Wei Wei",
                "Xiaoyan Gao",
                "Heyan Huang"
            ],
            "abstract": "Open-domain generative dialogue systems have attracted considerable attention\nover the past few years. Currently, how to automatically evaluate them, is\nstill a big challenge problem. As far as we know, there are three kinds of\nautomatic methods to evaluate the open-domain generative dialogue systems: (1)\nWord-overlap-based metrics; (2) Embedding-based metrics; (3) Learning-based\nmetrics. Due to the lack of systematic comparison, it is not clear which kind\nof metrics are more effective. In this paper, we will first measure\nsystematically all kinds of automatic evaluation metrics over the same\nexperimental setting to check which kind is best. Through extensive\nexperiments, the learning-based metrics are demonstrated that they are the most\neffective evaluation metrics for open-domain generative dialogue systems.\nMoreover, we observe that nearly all learning-based metrics depend on the\nnegative sampling mechanism, which obtains an extremely imbalanced and\nlow-quality dataset to train a score model. In order to address this issue, we\npropose a novel and feasible learning-based metric that can significantly\nimprove the correlation with human judgments by using augmented POsitive\nsamples and valuable NEgative samples, called PONE. Extensive experiments\ndemonstrate that our proposed evaluation method significantly outperforms the\nstate-of-the-art learning-based evaluation methods, with an average correlation\nimprovement of 13.18%. In addition, we have publicly released the codes of our\nproposed method and state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02421v4",
            "date": "2020-04-06 06:34:54+00:00",
            "title": "The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection",
            "authors": [
                "Zibo Lin",
                "Deng Cai",
                "Yan Wang",
                "Xiaojiang Liu",
                "Hai-Tao Zheng",
                "Shuming Shi"
            ],
            "abstract": "Response selection plays a vital role in building retrieval-based\nconversation systems. Despite that response selection is naturally a\nlearning-to-rank problem, most prior works take a point-wise view and train\nbinary classifiers for this task: each response candidate is labeled either\nrelevant (one) or irrelevant (zero). On the one hand, this formalization can be\nsub-optimal due to its ignorance of the diversity of response quality. On the\nother hand, annotating grayscale data for learning-to-rank can be prohibitively\nexpensive and challenging. In this work, we show that grayscale data can be\nautomatically constructed without human effort. Our method employs\noff-the-shelf response retrieval models and response generation models as\nautomatic grayscale data generators. With the constructed grayscale data, we\npropose multi-level ranking objectives for training, which can (1) teach a\nmatching model to capture more fine-grained context-response relevance\ndifference and (2) reduce the train-test discrepancy in terms of distractor\nstrength. Our method is simple, effective, and universal. Experiments on three\nbenchmark datasets and four state-of-the-art matching models show that the\nproposed approach brings significant and consistent performance improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02451v3",
            "date": "2020-04-06 07:47:34+00:00",
            "title": "An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models",
            "authors": [
                "Hiroshi Noji",
                "Hiroya Takamura"
            ],
            "abstract": "We explore the utilities of explicit negative examples in training neural\nlanguage models. Negative examples here are incorrect words in a sentence, such\nas \"barks\" in \"*The dogs barks\". Neural language models are commonly trained\nonly on positive examples, a set of sentences in the training data, but recent\nstudies suggest that the models trained in this way are not capable of robustly\nhandling complex syntactic constructions, such as long-distance agreement. In\nthis paper, using English data, we first demonstrate that appropriately using\nnegative examples about particular constructions (e.g., subject-verb agreement)\nwill boost the model's robustness on them, with a negligible loss of\nperplexity. The key to our success is an additional margin loss between the\nlog-likelihoods of a correct word and an incorrect word. We then provide a\ndetailed analysis of the trained models. One of our findings is the difficulty\nof object-relative clauses for RNNs. We find that even with our direct learning\nsignals the models still suffer from resolving agreement across an\nobject-relative clause. Augmentation of training sentences involving the\nconstructions somewhat helps, but the accuracy still does not reach the level\nof subject-relative clauses. Although not directly cognitively appealing, our\nmethod can be a tool to analyze the true architectural limitation of neural\nmodels on challenging linguistic constructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02509v1",
            "date": "2020-04-06 09:24:11+00:00",
            "title": "Building a Norwegian Lexical Resource for Medical Entity Recognition",
            "authors": [
                "Ildik\u00f3 Pil\u00e1n",
                "P\u00e5l H. Brekke",
                "Lilja \u00d8vrelid"
            ],
            "abstract": "We present a large Norwegian lexical resource of categorized medical terms.\nThe resource merges information from large medical databases, and contains over\n77,000 unique entries, including automatically mapped terms from a Norwegian\nmedical dictionary. We describe the methodology behind this automatic\ndictionary entry mapping based on keywords and suffixes and further present the\nresults of a manual evaluation performed on a subset by a domain expert. The\nevaluation indicated that ca. 80% of the mappings were correct.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02577v1",
            "date": "2020-04-06 11:50:49+00:00",
            "title": "Dictionary-based Data Augmentation for Cross-Domain Neural Machine Translation",
            "authors": [
                "Wei Peng",
                "Chongxuan Huang",
                "Tianhao Li",
                "Yun Chen",
                "Qun Liu"
            ],
            "abstract": "Existing data augmentation approaches for neural machine translation (NMT)\nhave predominantly relied on back-translating in-domain (IND) monolingual\ncorpora. These methods suffer from issues associated with a domain information\ngap, which leads to translation errors for low frequency and out-of-vocabulary\nterminology. This paper proposes a dictionary-based data augmentation (DDA)\nmethod for cross-domain NMT. DDA synthesizes a domain-specific dictionary with\ngeneral domain corpora to automatically generate a large-scale pseudo-IND\nparallel corpus. The generated pseudo-IND data can be used to enhance a general\ndomain trained baseline. The experiments show that the DDA-enhanced NMT models\ndemonstrate consistent significant improvements, outperforming the baseline\nmodels by 3.75-11.53 BLEU. The proposed method is also able to further improve\nthe performance of the back-translation based and IND-finetuned NMT models. The\nimprovement is associated with the enhanced domain coverage produced by DDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02585v4",
            "date": "2020-04-06 12:05:02+00:00",
            "title": "Bootstrapping a Crosslingual Semantic Parser",
            "authors": [
                "Tom Sherborne",
                "Yumo Xu",
                "Mirella Lapata"
            ],
            "abstract": "Recent progress in semantic parsing scarcely considers languages other than\nEnglish but professional translation can be prohibitively expensive. We adapt a\nsemantic parser trained on a single language, such as English, to new languages\nand multiple domains with minimal annotation. We query if machine translation\nis an adequate substitute for training data, and extend this to investigate\nbootstrapping using joint training with English, paraphrasing, and multilingual\npre-trained models. We develop a Transformer-based parser combining paraphrases\nby ensembling attention over multiple encoders and present new versions of ATIS\nand Overnight in German and Chinese for evaluation. Experimental results\nindicate that MT can approximate training data in a new language for accurate\nparsing when augmented with paraphrasing through multiple MT engines.\nConsidering when MT is inadequate, we also find that using our approach\nachieves parsing accuracy within 2% of complete translation using only 50% of\ntraining data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02592v1",
            "date": "2020-04-06 12:11:50+00:00",
            "title": "Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories",
            "authors": [
                "Qingyu Zhou",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we propose a method for automatically constructing a\npassage-to-summary dataset by mining the Wikipedia page revision histories. In\nparticular, the method mines the main body passages and the introduction\nsentences which are added to the pages simultaneously. The constructed dataset\ncontains more than one hundred thousand passage-summary pairs. The quality\nanalysis shows that it is promising that the dataset can be used as a training\nand validation set for passage summarization. We validate and analyze the\nperformance of various summarization systems on the proposed dataset. The\ndataset will be available online at https://res.qyzhou.me.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02644v3",
            "date": "2020-04-06 13:09:10+00:00",
            "title": "Sparse Text Generation",
            "authors": [
                "Pedro Henrique Martins",
                "Zita Marinho",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Current state-of-the-art text generators build on powerful language models\nsuch as GPT-2, achieving impressive performance. However, to avoid degenerate\ntext, they require sampling from a modified softmax, via temperature parameters\nor ad-hoc truncation techniques, as in top-$k$ or nucleus sampling. This\ncreates a mismatch between training and testing conditions. In this paper, we\nuse the recently introduced entmax transformation to train and sample from a\nnatively sparse language model, avoiding this mismatch. The result is a text\ngenerator with favorable performance in terms of fluency and consistency, fewer\nrepetitions, and n-gram diversity closer to human text. In order to evaluate\nour model, we propose three new metrics for comparing sparse or truncated\ndistributions: $\\epsilon$-perplexity, sparsemax score, and Jensen-Shannon\ndivergence. Human-evaluated experiments in story completion and dialogue\ngeneration show that entmax sampling leads to more engaging and coherent\nstories and conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02664v2",
            "date": "2020-04-06 13:35:10+00:00",
            "title": "At Which Level Should We Extract? An Empirical Analysis on Extractive Document Summarization",
            "authors": [
                "Qingyu Zhou",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Extractive methods have been proven effective in automatic document\nsummarization. Previous works perform this task by identifying informative\ncontents at sentence level. However, it is unclear whether performing\nextraction at sentence level is the best solution. In this work, we show that\nunnecessity and redundancy issues exist when extracting full sentences, and\nextracting sub-sentential units is a promising alternative. Specifically, we\npropose extracting sub-sentential units based on the constituency parsing tree.\nA neural extractive model which leverages the sub-sentential information and\nextracts them is presented. Extensive experiments and analyses show that\nextracting sub-sentential units performs competitively comparing to full\nsentence extraction under the evaluation of both automatic and human\nevaluations. Hopefully, our work could provide some inspiration of the basic\nextraction units in extractive summarization for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02709v2",
            "date": "2020-04-06 14:47:18+00:00",
            "title": "Evaluating Models' Local Decision Boundaries via Contrast Sets",
            "authors": [
                "Matt Gardner",
                "Yoav Artzi",
                "Victoria Basmova",
                "Jonathan Berant",
                "Ben Bogin",
                "Sihao Chen",
                "Pradeep Dasigi",
                "Dheeru Dua",
                "Yanai Elazar",
                "Ananth Gottumukkala",
                "Nitish Gupta",
                "Hanna Hajishirzi",
                "Gabriel Ilharco",
                "Daniel Khashabi",
                "Kevin Lin",
                "Jiangming Liu",
                "Nelson F. Liu",
                "Phoebe Mulcaire",
                "Qiang Ning",
                "Sameer Singh",
                "Noah A. Smith",
                "Sanjay Subramanian",
                "Reut Tsarfaty",
                "Eric Wallace",
                "Ally Zhang",
                "Ben Zhou"
            ],
            "abstract": "Standard test sets for supervised learning evaluate in-distribution\ngeneralization. Unfortunately, when a dataset has systematic gaps (e.g.,\nannotation artifacts), these evaluations are misleading: a model can learn\nsimple decision rules that perform well on the test set but do not capture a\ndataset's intended capabilities. We propose a new annotation paradigm for NLP\nthat helps to close systematic gaps in the test data. In particular, after a\ndataset is constructed, we recommend that the dataset authors manually perturb\nthe test instances in small but meaningful ways that (typically) change the\ngold label, creating contrast sets. Contrast sets provide a local view of a\nmodel's decision boundary, which can be used to more accurately evaluate a\nmodel's true linguistic capabilities. We demonstrate the efficacy of contrast\nsets by creating them for 10 diverse NLP datasets (e.g., DROP reading\ncomprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets\nare not explicitly adversarial, model performance is significantly lower on\nthem than on the original test sets---up to 25\\% in some cases. We release our\ncontrast sets as new evaluation benchmarks and encourage future dataset\nconstruction efforts to follow similar annotation processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02745v1",
            "date": "2020-04-06 15:38:55+00:00",
            "title": "Meta-Learning for Few-Shot NMT Adaptation",
            "authors": [
                "Amr Sharaf",
                "Hany Hassan",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "We present META-MT, a meta-learning approach to adapt Neural Machine\nTranslation (NMT) systems in a few-shot setting. META-MT provides a new\napproach to make NMT models easily adaptable to many target domains with the\nminimal amount of in-domain data. We frame the adaptation of NMT systems as a\nmeta-learning problem, where we learn to adapt to new unseen domains based on\nsimulated offline meta-training domain adaptation tasks. We evaluate the\nproposed meta-learning strategy on ten domains with general large scale NMT\nsystems. We show that META-MT significantly outperforms classical domain\nadaptation when very few in-domain examples are available. Our experiments\nshows that META-MT can outperform classical fine-tuning by up to 2.5 BLEU\npoints after seeing only 4, 000 translated words (300 parallel sentences).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02929v1",
            "date": "2020-04-06 18:41:43+00:00",
            "title": "An Annotated Corpus of Emerging Anglicisms in Spanish Newspaper Headlines",
            "authors": [
                "Elena \u00c1lvarez-Mellado"
            ],
            "abstract": "The extraction of anglicisms (lexical borrowings from English) is relevant\nboth for lexicographic purposes and for NLP downstream tasks. We introduce a\ncorpus of European Spanish newspaper headlines annotated with anglicisms and a\nbaseline model for anglicism extraction. In this paper we present: (1) a corpus\nof 21,570 newspaper headlines written in European Spanish annotated with\nemergent anglicisms and (2) a conditional random field baseline model with\nhandcrafted features for anglicism extraction. We present the newspaper\nheadlines corpus, describe the annotation tagset and guidelines and introduce a\nCRF model that can serve as baseline for the task of detecting anglicisms. The\npresented work is a first step towards the creation of an anglicism extractor\nfor Spanish newswire.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02986v1",
            "date": "2020-04-06 20:24:33+00:00",
            "title": "Zero-Shot Learning of Text Adventure Games with Sentence-Level Semantics",
            "authors": [
                "Xusen Yin",
                "Jonathan May"
            ],
            "abstract": "Reinforcement learning algorithms such as Q-learning have shown great promise\nin training models to learn the optimal action to take for a given system\nstate; a goal in applications with an exploratory or adversarial nature such as\ntask-oriented dialogues or games. However, models that do not have direct\naccess to their state are harder to train; when the only state access is via\nthe medium of language, this can be particularly pronounced. We introduce a new\nmodel amenable to deep Q-learning that incorporates a Siamese neural network\narchitecture and a novel refactoring of the Q-value function in order to better\nrepresent system state given its approximation over a language channel. We\nevaluate the model in the context of zero-shot text-based adventure game\nlearning. Extrinsically, our model reaches the baseline's convergence\nperformance point needing only 15% of its iterations, reaches a convergence\nperformance point 15% higher than the baseline's, and is able to play unseen,\nunrelated games with no fine-tuning. We probe our new model's representation\nspace to determine that intrinsically, this is due to the appropriate\nclustering of different linguistic mediation into the same state.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02990v3",
            "date": "2020-04-06 20:44:10+00:00",
            "title": "Evaluating the Evaluation of Diversity in Natural Language Generation",
            "authors": [
                "Guy Tevet",
                "Jonathan Berant"
            ],
            "abstract": "Despite growing interest in natural language generation (NLG) models that\nproduce diverse outputs, there is currently no principled method for evaluating\nthe diversity of an NLG system. In this work, we propose a framework for\nevaluating diversity metrics. The framework measures the correlation between a\nproposed diversity metric and a diversity parameter, a single parameter that\ncontrols some aspect of diversity in generated text. For example, a diversity\nparameter might be a binary variable used to instruct crowdsourcing workers to\ngenerate text with either low or high content diversity. We demonstrate the\nutility of our framework by: (a) establishing best practices for eliciting\ndiversity judgments from humans, (b) showing that humans substantially\noutperform automatic metrics in estimating content diversity, and (c)\ndemonstrating that existing methods for controlling diversity by tuning a\n\"decoding parameter\" mostly affect form but not meaning. Our framework can\nadvance the understanding of different diversity metrics, an essential step on\nthe road towards better NLG systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02995v1",
            "date": "2020-04-06 21:12:53+00:00",
            "title": "Multi-Step Inference for Reasoning Over Paragraphs",
            "authors": [
                "Jiangming Liu",
                "Matt Gardner"
            ],
            "abstract": "Complex reasoning over text requires understanding and chaining together\nfree-form predicates and logical connectives. Prior work has largely tried to\ndo this either symbolically or with black-box transformers. We present a middle\nground between these two extremes: a compositional model reminiscent of neural\nmodule networks that can perform chained logical reasoning. This model first\nfinds relevant sentences in the context and then chains them together using\nneural modules. Our model gives significant performance improvements (up to\n29\\% relative error reduction when combined with a reranker) on ROPES, a\nrecently-introduced complex reasoning dataset",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03012v2",
            "date": "2020-04-06 21:48:39+00:00",
            "title": "\"You are grounded!\": Latent Name Artifacts in Pre-trained Language Models",
            "authors": [
                "Vered Shwartz",
                "Rachel Rudinger",
                "Oyvind Tafjord"
            ],
            "abstract": "Pre-trained language models (LMs) may perpetuate biases originating in their\ntraining corpus to downstream models. We focus on artifacts associated with the\nrepresentation of given names (e.g., Donald), which, depending on the corpus,\nmay be associated with specific entities, as indicated by next token prediction\n(e.g., Trump). While helpful in some contexts, grounding happens also in\nunder-specified or inappropriate contexts. For example, endings generated for\n`Donald is a' substantially differ from those of other names, and often have\nmore-than-average negative sentiment. We demonstrate the potential effect on\ndownstream tasks with reading comprehension probes where name perturbation\nchanges the model answers. As a silver lining, our experiments suggest that\nadditional pre-training on different corpora may mitigate this bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03020v1",
            "date": "2020-04-06 22:11:30+00:00",
            "title": "Enhancing Review Comprehension with Domain-Specific Commonsense",
            "authors": [
                "Aaron Traylor",
                "Chen Chen",
                "Behzad Golshan",
                "Xiaolan Wang",
                "Yuliang Li",
                "Yoshihiko Suhara",
                "Jinfeng Li",
                "Cagatay Demiralp",
                "Wang-Chiew Tan"
            ],
            "abstract": "Review comprehension has played an increasingly important role in improving\nthe quality of online services and products and commonsense knowledge can\nfurther enhance review comprehension. However, existing general-purpose\ncommonsense knowledge bases lack sufficient coverage and precision to\nmeaningfully improve the comprehension of domain-specific reviews. In this\npaper, we introduce xSense, an effective system for review comprehension using\ndomain-specific commonsense knowledge bases (xSense KBs). We show that xSense\nKBs can be constructed inexpensively and present a knowledge distillation\nmethod that enables us to use xSense KBs along with BERT to boost the\nperformance of various review comprehension tasks. We evaluate xSense over\nthree review comprehension tasks: aspect extraction, aspect sentiment\nclassification, and question answering. We find that xSense outperforms the\nstate-of-the-art models for the first two tasks and improves the baseline BERT\nQA model significantly, demonstrating the usefulness of incorporating\ncommonsense into review comprehension pipelines. To facilitate future research\nand applications, we publicly release three domain-specific knowledge bases and\na domain-specific question answering benchmark along with this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03032v1",
            "date": "2020-04-06 22:50:27+00:00",
            "title": "A Systematic Analysis of Morphological Content in BERT Models for Multiple Languages",
            "authors": [
                "Daniel Edmiston"
            ],
            "abstract": "This work describes experiments which probe the hidden representations of\nseveral BERT-style models for morphological content. The goal is to examine the\nextent to which discrete linguistic structure, in the form of morphological\nfeatures and feature values, presents itself in the vector representations and\nattention distributions of pre-trained language models for five European\nlanguages. The experiments contained herein show that (i) Transformer\narchitectures largely partition their embedding space into convex sub-regions\nhighly correlated with morphological feature value, (ii) the contextualized\nnature of transformer embeddings allows models to distinguish ambiguous\nmorphological forms in many, but not all cases, and (iii) very specific\nattention head/layer combinations appear to hone in on subject-verb agreement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03034v1",
            "date": "2020-04-06 23:00:37+00:00",
            "title": "The Role of Pragmatic and Discourse Context in Determining Argument Impact",
            "authors": [
                "Esin Durmus",
                "Faisal Ladhak",
                "Claire Cardie"
            ],
            "abstract": "Research in the social sciences and psychology has shown that the\npersuasiveness of an argument depends not only the language employed, but also\non attributes of the source/communicator, the audience, and the appropriateness\nand strength of the argument's claims given the pragmatic and discourse context\nof the argument. Among these characteristics of persuasive arguments, prior\nwork in NLP does not explicitly investigate the effect of the pragmatic and\ndiscourse context when determining argument quality. This paper presents a new\ndataset to initiate the study of this aspect of argumentation: it consists of a\ndiverse collection of arguments covering 741 controversial topics and\ncomprising over 47,000 claims. We further propose predictive models that\nincorporate the pragmatic and discourse context of argumentative claims and\nshow that they outperform models that rely only on claim-specific linguistic\nfeatures for predicting the perceived impact of individual claims within a\nparticular line of argument.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03066v2",
            "date": "2020-04-07 01:20:55+00:00",
            "title": "Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition",
            "authors": [
                "Paloma Jeretic",
                "Alex Warstadt",
                "Suvrat Bhooshan",
                "Adina Williams"
            ],
            "abstract": "Natural language inference (NLI) is an increasingly important task for\nnatural language understanding, which requires one to infer whether a sentence\nentails another. However, the ability of NLI models to make pragmatic\ninferences remains understudied. We create an IMPlicature and PRESupposition\ndiagnostic dataset (IMPPRES), consisting of >25k semiautomatically generated\nsentence pairs illustrating well-studied pragmatic inference types. We use\nIMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on\nMultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although\nMultiNLI appears to contain very few pairs illustrating these inference types,\nwe find that BERT learns to draw pragmatic inferences. It reliably treats\nscalar implicatures triggered by \"some\" as entailments. For some presupposition\ntriggers like \"only\", BERT reliably recognizes the presupposition as an\nentailment, even when the trigger is embedded under an entailment canceling\noperator like negation. BOW and InferSent show weaker evidence of pragmatic\nreasoning. We conclude that NLI training encourages models to learn some, but\nnot all, pragmatic inferences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03090v1",
            "date": "2020-04-07 02:44:50+00:00",
            "title": "Interview: A Large-Scale Open-Source Corpus of Media Dialog",
            "authors": [
                "Bodhisattwa Prasad Majumder",
                "Shuyang Li",
                "Jianmo Ni",
                "Julian McAuley"
            ],
            "abstract": "Existing conversational datasets consist either of written proxies for dialog\nor small-scale transcriptions of natural speech. We introduce 'Interview': a\nlarge-scale (105K conversations) media dialog dataset collected from news\ninterview transcripts. Compared to existing large-scale proxies for\nconversational data, language models trained on our dataset exhibit better\nzero-shot out-of-domain performance on existing spoken dialog datasets,\ndemonstrating its usefulness in modeling real-world conversations. 'Interview'\ncontains speaker role annotations for each turn, facilitating the development\nof engaging, responsive dialog systems. In fact, experiments on two dialog\ntasks show that leveraging such labels improves performance over strong\nspeaker-agnostic baselines, and enabling models to generate more specific and\ninquisitive responses in interview-style conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03093v1",
            "date": "2020-04-07 02:54:20+00:00",
            "title": "Exemplar Auditing for Multi-Label Biomedical Text Classification",
            "authors": [
                "Allen Schmaltz",
                "Andrew Beam"
            ],
            "abstract": "Many practical applications of AI in medicine consist of semi-supervised\ndiscovery: The investigator aims to identify features of interest at a\nresolution more fine-grained than that of the available human labels. This is\noften the scenario faced in healthcare applications as coarse, high-level\nlabels (e.g., billing codes) are often the only sources that are readily\navailable. These challenges are compounded for modalities such as text, where\nthe feature space is very high-dimensional, and often contains considerable\namounts of noise.\n  In this work, we generalize a recently proposed zero-shot sequence labeling\nmethod, \"binary labeling via a convolutional decomposition\", to the case where\nthe available document-level human labels are themselves relatively\nhigh-dimensional. The approach yields classification with \"introspection\",\nrelating the fine-grained features of an inference-time prediction to their\nnearest neighbors from the training set, under the model. The approach is\neffective, yet parsimonious, as demonstrated on a well-studied MIMIC-III\nmulti-label classification task of electronic health record data, and is useful\nas a tool for organizing the analysis of neural model predictions and\nhigh-dimensional datasets. Our proposed approach yields both a competitively\neffective classification model and an interrogation mechanism to aid healthcare\nworkers in understanding the salient features that drive the model's\npredictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03096v2",
            "date": "2020-04-07 02:59:42+00:00",
            "title": "Is Graph Structure Necessary for Multi-hop Question Answering?",
            "authors": [
                "Nan Shao",
                "Yiming Cui",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Recently, attempting to model texts as graph structure and introducing graph\nneural networks to deal with it has become a trend in many NLP research areas.\nIn this paper, we investigate whether the graph structure is necessary for\nmulti-hop question answering. Our analysis is centered on HotpotQA. We\nconstruct a strong baseline model to establish that, with the proper use of\npre-trained models, graph structure may not be necessary for multi-hop question\nanswering. We point out that both graph structure and adjacency matrix are\ntask-related prior knowledge, and graph-attention can be considered as a\nspecial case of self-attention. Experiments and visualized analysis demonstrate\nthat graph-attention or the entire graph structure can be replaced by\nself-attention or Transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03097v1",
            "date": "2020-04-07 03:03:00+00:00",
            "title": "Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation",
            "authors": [
                "Bowen Wu",
                "Huan Zhang",
                "Mengyuan Li",
                "Zongsheng Wang",
                "Qihang Feng",
                "Junhong Huang",
                "Baoxun Wang"
            ],
            "abstract": "Recently, BERT has become an essential ingredient of various NLP deep models\ndue to its effectiveness and universal-usability. However, the online\ndeployment of BERT is often blocked by its large-scale parameters and high\ncomputational cost. There are plenty of studies showing that the knowledge\ndistillation is efficient in transferring the knowledge from BERT into the\nmodel with a smaller size of parameters. Nevertheless, current BERT\ndistillation approaches mainly focus on task-specified distillation, such\nmethodologies lead to the loss of the general semantic knowledge of BERT for\nuniversal-usability. In this paper, we propose a sentence representation\napproximating oriented distillation framework that can distill the pre-trained\nBERT into a simple LSTM based model without specifying tasks. Consistent with\nBERT, our distilled model is able to perform transfer learning via fine-tuning\nto adapt to any sentence-level downstream task. Besides, our model can further\ncooperate with task-specific distillation procedures. The experimental results\non multiple NLP tasks from the GLUE benchmark show that our approach\noutperforms other task-specific distillation methods or even much larger\nmodels, i.e., ELMO, with efficiency well-improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03116v2",
            "date": "2020-04-07 04:09:00+00:00",
            "title": "A Sentence Cloze Dataset for Chinese Machine Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Ting Liu",
                "Ziqing Yang",
                "Zhipeng Chen",
                "Wentao Ma",
                "Wanxiang Che",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Owing to the continuous efforts by the Chinese NLP community, more and more\nChinese machine reading comprehension datasets become available. To add\ndiversity in this area, in this paper, we propose a new task called Sentence\nCloze-style Machine Reading Comprehension (SC-MRC). The proposed task aims to\nfill the right candidate sentence into the passage that has several blanks. We\nbuilt a Chinese dataset called CMRC 2019 to evaluate the difficulty of the\nSC-MRC task. Moreover, to add more difficulties, we also made fake candidates\nthat are similar to the correct ones, which requires the machine to judge their\ncorrectness in the context. The proposed dataset contains over 100K blanks\n(questions) within over 10K passages, which was originated from Chinese\nnarrative stories. To evaluate the dataset, we implement several baseline\nsystems based on the pre-trained models, and the results show that the\nstate-of-the-art model still underperforms human performance by a large margin.\nWe release the dataset and baseline system to further facilitate our community.\nResources available through https://github.com/ymcui/cmrc2019",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03125v1",
            "date": "2020-04-07 04:51:04+00:00",
            "title": "RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases",
            "authors": [
                "DongHyun Choi",
                "Myeong Cheol Shin",
                "EungGyun Kim",
                "Dong Ryeol Shin"
            ],
            "abstract": "Text-to-SQL is the problem of converting a user question into an SQL query,\nwhen the question and database are given. In this paper, we present a neural\nnetwork approach called RYANSQL (Recursively Yielding Annotation Network for\nSQL) to solve complex Text-to-SQL tasks for cross-domain databases. State-ment\nPosition Code (SPC) is defined to trans-form a nested SQL query into a set of\nnon-nested SELECT statements; a sketch-based slot filling approach is proposed\nto synthesize each SELECT statement for its corresponding SPC. Additionally,\ntwo input manipulation methods are presented to improve generation performance\nfurther. RYANSQL achieved 58.2% accuracy on the challenging Spider benchmark,\nwhich is a 3.2%p improvement over previous state-of-the-art approaches. At the\ntime of writing, RYANSQL achieves the first position on the Spider leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03136v5",
            "date": "2020-04-07 05:44:58+00:00",
            "title": "g2pM: A Neural Grapheme-to-Phoneme Conversion Package for Mandarin Chinese Based on a New Open Benchmark Dataset",
            "authors": [
                "Kyubyong Park",
                "Seanie Lee"
            ],
            "abstract": "Conversion of Chinese graphemes to phonemes (G2P) is an essential component\nin Mandarin Chinese Text-To-Speech (TTS) systems. One of the biggest challenges\nin Chinese G2P conversion is how to disambiguate the pronunciation of\npolyphones - characters having multiple pronunciations. Although many academic\nefforts have been made to address it, there has been no open dataset that can\nserve as a standard benchmark for fair comparison to date. In addition, most of\nthe reported systems are hard to employ for researchers or practitioners who\nwant to convert Chinese text into pinyin at their convenience. Motivated by\nthese, in this work, we introduce a new benchmark dataset that consists of\n99,000+ sentences for Chinese polyphone disambiguation. We train a simple\nneural network model on it, and find that it outperforms other preexisting G2P\nsystems. Finally, we package our project and share it on PyPi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03137v3",
            "date": "2020-04-07 05:46:49+00:00",
            "title": "Cross-lingual Supervision Improves Unsupervised Neural Machine Translation",
            "authors": [
                "Mingxuan Wang",
                "Hongxiao Bai",
                "Hai Zhao",
                "Lei Li"
            ],
            "abstract": "Neural machine translation~(NMT) is ineffective for zero-resource languages.\nRecent works exploring the possibility of unsupervised neural machine\ntranslation (UNMT) with only monolingual data can achieve promising results.\nHowever, there are still big gaps between UNMT and NMT with parallel\nsupervision. In this work, we introduce a multilingual unsupervised NMT\n(\\method) framework to leverage weakly supervised signals from high-resource\nlanguage pairs to zero-resource translation directions. More specifically, for\nunsupervised language pairs \\texttt{En-De}, we can make full use of the\ninformation from parallel dataset \\texttt{En-Fr} to jointly train the\nunsupervised translation directions all in one model. \\method is based on\nmultilingual models which require no changes to the standard unsupervised NMT.\nEmpirical results demonstrate that \\method significantly improves the\ntranslation quality by more than 3 BLEU score on six benchmark unsupervised\ntranslation directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03151v2",
            "date": "2020-04-07 06:45:45+00:00",
            "title": "Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation",
            "authors": [
                "Dana Ruiter",
                "Josef van Genabith",
                "Cristina Espa\u00f1a-Bonet"
            ],
            "abstract": "Self-supervised neural machine translation (SSNMT) jointly learns to identify\nand select suitable training data from comparable (rather than parallel)\ncorpora and to translate, in a way that the two tasks support each other in a\nvirtuous circle. In this study, we provide an in-depth analysis of the sampling\nchoices the SSNMT model makes during training. We show how, without it having\nbeen told to do so, the model self-selects samples of increasing (i) complexity\nand (ii) task-relevance in combination with (iii) performing a denoising\ncurriculum. We observe that the dynamics of the mutual-supervision signals of\nboth system internal representation types are vital for the extraction and\ntranslation performance. We show that in terms of the Gunning-Fog Readability\nindex, SSNMT starts extracting and learning from Wikipedia data suitable for\nhigh school students and quickly moves towards content suitable for first year\nundergraduate students.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03176v1",
            "date": "2020-04-07 07:55:41+00:00",
            "title": "Machine Translation with Unsupervised Length-Constraints",
            "authors": [
                "Jan Niehues"
            ],
            "abstract": "We have seen significant improvements in machine translation due to the usage\nof deep learning. While the improvements in translation quality are impressive,\nthe encoder-decoder architecture enables many more possibilities. In this\npaper, we explore one of these, the generation of constraint translation. We\nfocus on length constraints, which are essential if the translation should be\ndisplayed in a given format. In this work, we propose an end-to-end approach\nfor this task. Compared to a traditional method that first translates and then\nperforms sentence compression, the text compression is learned completely\nunsupervised. By combining the idea with zero-shot multilingual machine\ntranslation, we are also able to perform unsupervised monolingual sentence\ncompression. In order to fulfill the length constraints, we investigated\nseveral methods to integrate the constraints into the model. Using the\npresented technique, we are able to significantly improve the translation\nquality under constraints. Furthermore, we are able to perform unsupervised\nmonolingual sentence compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03180v2",
            "date": "2020-04-07 08:02:21+00:00",
            "title": "Towards Multimodal Simultaneous Neural Machine Translation",
            "authors": [
                "Aizhan Imankulova",
                "Masahiro Kaneko",
                "Tosho Hirasawa",
                "Mamoru Komachi"
            ],
            "abstract": "Simultaneous translation involves translating a sentence before the speaker's\nutterance is completed in order to realize real-time understanding in multiple\nlanguages. This task is significantly more challenging than the general full\nsentence translation because of the shortage of input information during\ndecoding. To alleviate this shortage, we propose multimodal simultaneous neural\nmachine translation (MSNMT), which leverages visual information as an\nadditional modality. Our experiments with the Multi30k dataset showed that\nMSNMT significantly outperforms its text-only counterpart in more timely\ntranslation situations with low latency. Furthermore, we verified the\nimportance of visual information during decoding by performing an adversarial\nevaluation of MSNMT, where we studied how models behaved with incongruent input\nmodality and analyzed the effect of different word order between source and\ntarget languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03186v3",
            "date": "2020-04-07 08:15:21+00:00",
            "title": "More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction",
            "authors": [
                "Xu Han",
                "Tianyu Gao",
                "Yankai Lin",
                "Hao Peng",
                "Yaoliang Yang",
                "Chaojun Xiao",
                "Zhiyuan Liu",
                "Peng Li",
                "Maosong Sun",
                "Jie Zhou"
            ],
            "abstract": "Relational facts are an important component of human knowledge, which are\nhidden in vast amounts of text. In order to extract these facts from text,\npeople have been working on relation extraction (RE) for years. From early\npattern matching to current neural networks, existing RE methods have achieved\nsignificant progress. Yet with explosion of Web text and emergence of new\nrelations, human knowledge is increasing drastically, and we thus require\n\"more\" from RE: a more powerful RE system that can robustly utilize more data,\nefficiently learn more relations, easily handle more complicated context, and\nflexibly generalize to more open domains. In this paper, we look back at\nexisting RE methods, analyze key challenges we are facing nowadays, and show\npromising directions towards more powerful RE. We hope our view can advance\nthis field and inspire more efforts in the community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03227v1",
            "date": "2020-04-07 09:40:54+00:00",
            "title": "Improving Fluency of Non-Autoregressive Machine Translation",
            "authors": [
                "Zden\u011bk Kasner",
                "Jind\u0159ich Libovick\u00fd",
                "Jind\u0159ich Helcl"
            ],
            "abstract": "Non-autoregressive (nAR) models for machine translation (MT) manifest\nsuperior decoding speed when compared to autoregressive (AR) models, at the\nexpense of impaired fluency of their outputs. We improve the fluency of a nAR\nmodel with connectionist temporal classification (CTC) by employing additional\nfeatures in the scoring model used during beam search decoding. Since the beam\nsearch decoding in our model only requires to run the network in a single\nforward pass, the decoding speed is still notably higher than in standard AR\nmodels. We train models for three language pairs: German, Czech, and Romanian\nfrom and into English. The results show that our proposed models can be more\nefficient in terms of decoding speed and still achieve a competitive BLEU score\nrelative to AR models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03283v1",
            "date": "2020-04-07 11:39:50+00:00",
            "title": "A German Corpus for Fine-Grained Named Entity Recognition and Relation Extraction of Traffic and Industry Events",
            "authors": [
                "Martin Schiersch",
                "Veselina Mironova",
                "Maximilian Schmitt",
                "Philippe Thomas",
                "Aleksandra Gabryszak",
                "Leonhard Hennig"
            ],
            "abstract": "Monitoring mobility- and industry-relevant events is important in areas such\nas personal travel planning and supply chain management, but extracting events\npertaining to specific companies, transit routes and locations from\nheterogeneous, high-volume text streams remains a significant challenge. This\nwork describes a corpus of German-language documents which has been annotated\nwith fine-grained geo-entities, such as streets, stops and routes, as well as\nstandard named entity types. It has also been annotated with a set of 15\ntraffic- and industry-related n-ary relations and events, such as accidents,\ntraffic jams, acquisitions, and strikes. The corpus consists of newswire texts,\nTwitter messages, and traffic reports from radio stations, police and railway\ncompanies. It allows for training and evaluating both named entity recognition\nalgorithms that aim for fine-grained typing of geo-entities, as well as n-ary\nrelation extraction systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03287v1",
            "date": "2020-04-07 11:45:22+00:00",
            "title": "A Corpus Study and Annotation Schema for Named Entity Recognition and Relation Extraction of Business Products",
            "authors": [
                "Saskia Sch\u00f6n",
                "Veselina Mironova",
                "Aleksandra Gabryszak",
                "Leonhard Hennig"
            ],
            "abstract": "Recognizing non-standard entity types and relations, such as B2B products,\nproduct classes and their producers, in news and forum texts is important in\napplication areas such as supply chain monitoring and market research. However,\nthere is a decided lack of annotated corpora and annotation guidelines in this\ndomain. In this work, we present a corpus study, an annotation schema and\nassociated guidelines, for the annotation of product entity and company-product\nrelation mentions. We find that although product mentions are often realized as\nnoun phrases, defining their exact extent is difficult due to high boundary\nambiguity and the broad syntactic and semantic variety of their surface\nrealizations. We also describe our ongoing annotation effort, and present a\npreliminary corpus of English web and social media documents annotated\naccording to the proposed guidelines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03289v3",
            "date": "2020-04-07 11:49:15+00:00",
            "title": "KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding",
            "authors": [
                "Jiyeon Ham",
                "Yo Joong Choe",
                "Kyubyong Park",
                "Ilji Choi",
                "Hyungjoon Soh"
            ],
            "abstract": "Natural language inference (NLI) and semantic textual similarity (STS) are\nkey tasks in natural language understanding (NLU). Although several benchmark\ndatasets for those tasks have been released in English and a few other\nlanguages, there are no publicly available NLI or STS datasets in the Korean\nlanguage. Motivated by this, we construct and release new datasets for Korean\nNLI and STS, dubbed KorNLI and KorSTS, respectively. Following previous\napproaches, we machine-translate existing English training sets and manually\ntranslate development and test sets into Korean. To accelerate research on\nKorean NLU, we also establish baselines on KorNLI and KorSTS. Our datasets are\npublicly available at https://github.com/kakaobrain/KorNLUDatasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03324v1",
            "date": "2020-04-07 13:01:26+00:00",
            "title": "Windowing Models for Abstractive Summarization of Long Texts",
            "authors": [
                "Leon Sch\u00fcller",
                "Florian Wilhelm",
                "Nico Kreiling",
                "Goran Glava\u0161"
            ],
            "abstract": "Neural summarization models suffer from the fixed-size input limitation: if\ntext length surpasses the model's maximal number of input tokens, some document\ncontent (possibly summary-relevant) gets truncated Independently summarizing\nwindows of maximal input size disallows for information flow between windows\nand leads to incoherent summaries. We propose windowing models for neural\nabstractive summarization of (arbitrarily) long texts. We extend the\nsequence-to-sequence model augmented with pointer generator network by (1)\nallowing the encoder to slide over different windows of the input document and\n(2) sharing the decoder and retaining its state across different input windows.\nWe explore two windowing variants: Static Windowing precomputes the number of\ntokens the decoder should generate from each window (based on training corpus\nstatistics); in Dynamic Windowing the decoder learns to emit a token that\nsignals encoder's shift to the next input window. Empirical results render our\nmodels effective in their intended use-case: summarizing long texts with\nrelevant content not bound to the very document beginning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03354v4",
            "date": "2020-04-07 13:31:06+00:00",
            "title": "Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA",
            "authors": [
                "Nina Poerner",
                "Ulli Waltinger",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Domain adaptation of Pretrained Language Models (PTLMs) is typically achieved\nby unsupervised pretraining on target-domain text. While successful, this\napproach is expensive in terms of hardware, runtime and CO_2 emissions. Here,\nwe propose a cheaper alternative: We train Word2Vec on target-domain text and\nalign the resulting word vectors with the wordpiece vectors of a general-domain\nPTLM. We evaluate on eight biomedical Named Entity Recognition (NER) tasks and\ncompare against the recently proposed BioBERT model. We cover over 60% of the\nBioBERT-BERT F1 delta, at 5% of BioBERT's CO_2 footprint and 2% of its cloud\ncompute cost. We also show how to quickly adapt an existing general-domain\nQuestion Answering (QA) model to an emerging domain: the Covid-19 pandemic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03422v2",
            "date": "2020-04-07 14:13:38+00:00",
            "title": "Operationalizing the legal concept of 'Incitement to Hatred' as an NLP task",
            "authors": [
                "Frederike Zufall",
                "Huangpan Zhang",
                "Katharina Kloppenborg",
                "Torsten Zesch"
            ],
            "abstract": "Hate speech detection or offensive language detection are well-established\nbut controversial NLP tasks. With 'hate speech' not being a legal term, these\ntasks often elaborate on the question of which statements are perceived as\nbeing 'appropriate' or 'offensive'. Looking beyond this cursory understanding,\nthis article proposes a way to operationalize the legal concept of incitement\nto hatred as an NLP task. We pursue this task based on the criminal offense of\nincitement to hatred in {\\S} 130 of the German Criminal Code along with the\nunderlying EU Framework. Under the German Network Enforcement Act, social media\nproviders are subject to a direct obligation to delete postings violating this\noffense. We take this as a use case to study the transition from the\nill-defined concepts of hate speech or offensive language which are usually\nused in NLP to an operationalization of an actual legally binding obligation.\nWe first translate the legal assessment into a series of binary decisions and\nthen collect, annotate, and analyze a dataset according to our annotation\nscheme. Finally, we translate each of the legal decisions into an NLP task\nbased on the annotated data. The two subtasks of target group detection and\ntargeting act detection can be considered crucial from a legal viewpoint. We\nshow that both can be annotated by non-legally trained persons with sufficient\nreliability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03490v2",
            "date": "2020-04-07 15:41:55+00:00",
            "title": "What do Models Learn from Question Answering Datasets?",
            "authors": [
                "Priyanka Sen",
                "Amir Saffari"
            ],
            "abstract": "While models have reached superhuman performance on popular question\nanswering (QA) datasets such as SQuAD, they have yet to outperform humans on\nthe task of question answering itself. In this paper, we investigate if models\nare learning reading comprehension from QA datasets by evaluating BERT-based\nmodels across five datasets. We evaluate models on their generalizability to\nout-of-domain examples, responses to missing or incorrect data, and ability to\nhandle question variations. We find that no single dataset is robust to all of\nour experiments and identify shortcomings in both datasets and evaluation\nmethods. Following our analysis, we make recommendations for building future QA\ndatasets that better evaluate the task of question answering through reading\ncomprehension. We also release code to convert QA datasets to a shared format\nfor easier experimentation at\nhttps://github.com/amazon-research/qa-dataset-converter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03554v1",
            "date": "2020-04-07 17:26:36+00:00",
            "title": "Fine-Grained Named Entity Typing over Distantly Supervised Data Based on Refined Representations",
            "authors": [
                "Muhammad Asif Ali",
                "Yifang Sun",
                "Bing Li",
                "Wei Wang"
            ],
            "abstract": "Fine-Grained Named Entity Typing (FG-NET) is a key component in Natural\nLanguage Processing (NLP). It aims at classifying an entity mention into a wide\nrange of entity types. Due to a large number of entity types, distant\nsupervision is used to collect training data for this task, which noisily\nassigns type labels to entity mentions irrespective of the context. In order to\nalleviate the noisy labels, existing approaches on FGNET analyze the entity\nmentions entirely independent of each other and assign type labels solely based\non mention sentence-specific context. This is inadequate for highly overlapping\nand noisy type labels as it hinders information passing across sentence\nboundaries. For this, we propose an edge-weighted attentive graph convolution\nnetwork that refines the noisy mention representations by attending over\ncorpus-level contextual clues prior to the end classification. Experimental\nevaluation shows that the proposed model outperforms the existing research by a\nrelative score of upto 10.2% and 8.3% for macro f1 and micro f1 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03555v1",
            "date": "2020-04-07 17:28:28+00:00",
            "title": "Entity Linking via Dual and Cross-Attention Encoders",
            "authors": [
                "Oshin Agarwal",
                "Daniel M. Bikel"
            ],
            "abstract": "Entity Linking has two main open areas of research: 1) generate candidate\nentities without using alias tables and 2) generate more contextual\nrepresentations for both mentions and entities. Recently, a solution has been\nproposed for the former as a dual-encoder entity retrieval system (Gillick et\nal., 2019) that learns mention and entity representations in the same space,\nand performs linking by selecting the nearest entity to the mention in this\nspace. In this work, we use this retrieval system solely for generating\ncandidate entities. We then rerank the entities by using a cross-attention\nencoder over the target mention and each of the candidate entities. Whereas a\ndual encoder approach forces all information to be contained in the small,\nfixed set of vector dimensions used to represent mentions and entities, a\ncrossattention model allows for the use of detailed information (read:\nfeatures) from the entirety of each <mention, context, candidate entity> tuple.\nWe experiment with features used in the reranker including different ways of\nincorporating document-level context. We achieve state-of-the-art results on\nTACKBP-2010 dataset, with 92.05% accuracy. Furthermore, we show how the\nrescoring model generalizes well when trained on the larger CoNLL-2003 dataset\nand evaluated on TACKBP-2010.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03588v2",
            "date": "2020-04-07 02:08:04+00:00",
            "title": "Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots",
            "authors": [
                "Jia-Chen Gu",
                "Tianda Li",
                "Quan Liu",
                "Zhen-Hua Ling",
                "Zhiming Su",
                "Si Wei",
                "Xiaodan Zhu"
            ],
            "abstract": "In this paper, we study the problem of employing pre-trained language models\nfor multi-turn response selection in retrieval-based chatbots. A new model,\nnamed Speaker-Aware BERT (SA-BERT), is proposed in order to make the model\naware of the speaker change information, which is an important and intrinsic\nproperty of multi-turn dialogues. Furthermore, a speaker-aware disentanglement\nstrategy is proposed to tackle the entangled dialogues. This strategy selects a\nsmall number of most important utterances as the filtered context according to\nthe speakers' information in them. Finally, domain adaptation is performed to\nincorporate the in-domain knowledge into pre-trained language models.\nExperiments on five public datasets show that our proposed model outperforms\nthe present models on all metrics by large margins and achieves new\nstate-of-the-art performances for multi-turn response selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03607v2",
            "date": "2020-04-07 18:00:03+00:00",
            "title": "TuringAdvice: A Generative and Dynamic Evaluation of Language Use",
            "authors": [
                "Rowan Zellers",
                "Ari Holtzman",
                "Elizabeth Clark",
                "Lianhui Qin",
                "Ali Farhadi",
                "Yejin Choi"
            ],
            "abstract": "We propose TuringAdvice, a new challenge task and dataset for language\nunderstanding models. Given a written situation that a real person is currently\nfacing, a model must generate helpful advice in natural language. Our\nevaluation framework tests a fundamental aspect of human language\nunderstanding: our ability to use language to resolve open-ended situations by\ncommunicating with each other.\n  Empirical results show that today's models struggle at TuringAdvice, even\nmultibillion parameter models finetuned on 600k in-domain training examples.\nThe best model, a finetuned T5, writes advice that is at least as helpful as\nhuman-written advice in only 14% of cases; a much larger non-finetunable GPT3\nmodel does even worse at 4%. This low performance reveals language\nunderstanding errors that are hard to spot outside of a generative setting,\nshowing much room for progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03643v3",
            "date": "2020-04-07 18:27:32+00:00",
            "title": "Re-translation versus Streaming for Simultaneous Translation",
            "authors": [
                "Naveen Arivazhagan",
                "Colin Cherry",
                "Wolfgang Macherey",
                "George Foster"
            ],
            "abstract": "There has been great progress in improving streaming machine translation, a\nsimultaneous paradigm where the system appends to a growing hypothesis as more\nsource content becomes available. We study a related problem in which revisions\nto the hypothesis beyond strictly appending words are permitted. This is\nsuitable for applications such as live captioning an audio feed. In this\nsetting, we compare custom streaming approaches to re-translation, a\nstraightforward strategy where each new source token triggers a distinct\ntranslation from scratch. We find re-translation to be as good or better than\nstate-of-the-art streaming systems, even when operating under constraints that\nallow very few revisions. We attribute much of this success to a previously\nproposed data-augmentation technique that adds prefix-pairs to the training\ndata, which alongside wait-k inference forms a strong baseline for streaming\ntranslation. We also highlight re-translation's ability to wrap arbitrarily\npowerful MT systems with an experiment showing large improvements from an\nupgrade to its base model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03659v1",
            "date": "2020-04-07 19:26:13+00:00",
            "title": "The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews",
            "authors": [
                "Elena Tutubalina",
                "Ilseyar Alimova",
                "Zulfat Miftahutdinov",
                "Andrey Sakhovskiy",
                "Valentin Malykh",
                "Sergey Nikolenko"
            ],
            "abstract": "The Russian Drug Reaction Corpus (RuDReC) is a new partially annotated corpus\nof consumer reviews in Russian about pharmaceutical products for the detection\nof health-related named entities and the effectiveness of pharmaceutical\nproducts. The corpus itself consists of two parts, the raw one and the labelled\none. The raw part includes 1.4 million health-related user-generated texts\ncollected from various Internet sources, including social media. The labelled\npart contains 500 consumer reviews about drug therapy with drug- and\ndisease-related information. Labels for sentences include health-related issues\nor their absence. The sentences with one are additionally labelled at the\nexpression level for identification of fine-grained subtypes such as drug\nclasses and drug forms, drug indications, and drug reactions. Further, we\npresent a baseline model for named entity recognition (NER) and multi-label\nsentence classification tasks on this corpus. The macro F1 score of 74.85% in\nthe NER task was achieved by our RuDR-BERT model. For the sentence\nclassification task, our model achieves the macro F1 score of 68.82% gaining\n7.47% over the score of BERT model trained on Russian data. We make the RuDReC\ncorpus and pretrained weights of domain-specific BERT models freely available\nat https://github.com/cimm-kzn/RuDReC",
            "journal ref": "Bioinformatics, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03672v2",
            "date": "2020-04-07 19:49:58+00:00",
            "title": "Dynamic Data Selection and Weighting for Iterative Back-Translation",
            "authors": [
                "Zi-Yi Dou",
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "Back-translation has proven to be an effective method to utilize monolingual\ndata in neural machine translation (NMT), and iteratively conducting\nback-translation can further improve the model performance. Selecting which\nmonolingual data to back-translate is crucial, as we require that the resulting\nsynthetic data are of high quality and reflect the target domain. To achieve\nthese two goals, data selection and weighting strategies have been proposed,\nwith a common practice being to select samples close to the target domain but\nalso dissimilar to the average general-domain text. In this paper, we provide\ninsights into this commonly used approach and generalize it to a dynamic\ncurriculum learning strategy, which is applied to iterative back-translation\nmodels. In addition, we propose weighting strategies based on both the current\nquality of the sentence and its improvement over the previous iteration. We\nevaluate our models on domain adaptation, low-resource, and high-resource MT\nsettings and on two language pairs. Experimental results demonstrate that our\nmethods achieve improvements of up to 1.8 BLEU points over competitive\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03720v2",
            "date": "2020-04-07 21:21:06+00:00",
            "title": "Byte Pair Encoding is Suboptimal for Language Model Pretraining",
            "authors": [
                "Kaj Bostrom",
                "Greg Durrett"
            ],
            "abstract": "The success of pretrained transformer language models (LMs) in natural\nlanguage processing has led to a wide range of pretraining setups. In\nparticular, these models employ a variety of subword tokenization methods, most\nnotably byte-pair encoding (BPE) (Sennrich et al., 2016; Gage, 1994), the\nWordPiece method (Schuster and Nakajima, 2012), and unigram language modeling\n(Kudo, 2018), to segment text. However, to the best of our knowledge, the\nliterature does not contain a direct evaluation of the impact of tokenization\non language model pretraining. We analyze differences between BPE and unigram\nLM tokenization, finding that the latter method recovers subword units that\nalign more closely with morphology and avoids problems stemming from BPE's\ngreedy construction procedure. We then compare the fine-tuned task performance\nof identical transformer masked language models pretrained with these\ntokenizations. Across downstream tasks and two languages (English and\nJapanese), we find that the unigram LM tokenization method matches or\noutperforms BPE. We hope that developers of future pretrained LMs will consider\nadopting the unigram LM method over the more prevalent BPE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03760v1",
            "date": "2020-04-08 00:54:01+00:00",
            "title": "DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement",
            "authors": [
                "Tianda Li",
                "Jia-Chen Gu",
                "Xiaodan Zhu",
                "Quan Liu",
                "Zhen-Hua Ling",
                "Zhiming Su",
                "Si Wei"
            ],
            "abstract": "Disentanglement is a problem in which multiple conversations occur in the\nsame channel simultaneously, and the listener should decide which utterance is\npart of the conversation he will respond to. We propose a new model, named\nDialogue BERT (DialBERT), which integrates local and global semantics in a\nsingle stream of messages to disentangle the conversations that mixed together.\nWe employ BERT to capture the matching information in each utterance pair at\nthe utterance-level, and use a BiLSTM to aggregate and incorporate the\ncontext-level information. With only a 3% increase in parameters, a 12%\nimprovement has been attained in comparison to BERT, based on the F1-Score. The\nmodel achieves a state-of-the-art result on the a new dataset proposed by IBM\nand surpasses previous work by a substantial margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03794v1",
            "date": "2020-04-08 03:51:17+00:00",
            "title": "CALM: Continuous Adaptive Learning for Language Modeling",
            "authors": [
                "Kristjan Arumae",
                "Parminder Bhatia"
            ],
            "abstract": "Training large language representation models has become a standard in the\nnatural language processing community. This allows for fine tuning on any\nnumber of specific tasks, however, these large high capacity models can\ncontinue to train on domain specific unlabeled data to make initialization even\nmore robust for supervised tasks. We demonstrate that in practice these\npre-trained models present performance deterioration in the form of\ncatastrophic forgetting when evaluated on tasks from a general domain such as\nGLUE. In this work we propose CALM, Continuous Adaptive Learning for Language\nModeling: techniques to render models which retain knowledge across multiple\ndomains. With these methods, we are able to reduce the performance gap across\nsupervised tasks introduced by task specific models which we demonstrate using\na continual learning setting in biomedical and clinical domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03829v2",
            "date": "2020-04-08 06:18:44+00:00",
            "title": "Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning",
            "authors": [
                "Zhaojiang Lin",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Fine-tuning pre-trained generative language models to down-stream language\ngeneration tasks has shown promising results. However, this comes with the cost\nof having a single, large model for each task, which is not ideal in\nlow-memory/power scenarios (e.g., mobile). In this paper, we propose an\neffective way to fine-tune multiple down-stream generation tasks simultaneously\nusing a single, large pre-trained model. The experiments on five diverse\nlanguage generation tasks show that by just using an additional 2-3% parameters\nfor each task, our model can maintain or even improve the performance of\nfine-tuning the whole model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03849v1",
            "date": "2020-04-08 07:19:18+00:00",
            "title": "ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing",
            "authors": [
                "Xinyu Wang",
                "Yixian Liu",
                "Zixia Jia",
                "Chengyue Jiang",
                "Kewei Tu"
            ],
            "abstract": "This paper presents the system used in our submission to the \\textit{CoNLL\n2019 shared task: Cross-Framework Meaning Representation Parsing}. Our system\nis a graph-based parser which combines an extended pointer-generator network\nthat generates nodes and a second-order mean field variational inference module\nthat predicts edges. Our system achieved \\nth{1} and \\nth{2} place for the DM\nand PSD frameworks respectively on the in-framework ranks and achieved \\nth{3}\nplace for the DM framework on the cross-framework ranks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03875v2",
            "date": "2020-04-08 08:30:05+00:00",
            "title": "Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation",
            "authors": [
                "Dayiheng Liu",
                "Yeyun Gong",
                "Jie Fu",
                "Wei Liu",
                "Yu Yan",
                "Bo Shao",
                "Daxin Jiang",
                "Jiancheng Lv",
                "Nan Duan"
            ],
            "abstract": "News headline generation aims to produce a short sentence to attract readers\nto read the news. One news article often contains multiple keyphrases that are\nof interest to different users, which can naturally have multiple reasonable\nheadlines. However, most existing methods focus on the single headline\ngeneration. In this paper, we propose generating multiple headlines with\nkeyphrases of user interests, whose main idea is to generate multiple\nkeyphrases of interest to users for the news first, and then generate multiple\nkeyphrase-relevant headlines. We propose a multi-source Transformer decoder,\nwhich takes three sources as inputs: (a) keyphrase, (b) keyphrase-filtered\narticle, and (c) original article to generate keyphrase-relevant, high-quality,\nand diverse headlines. Furthermore, we propose a simple and effective method to\nmine the keyphrases of interest in the news article and build a first\nlarge-scale keyphrase-aware news headline corpus, which contains over 180K\naligned triples of $<$news article, headline, keyphrase$>$. Extensive\nexperimental comparisons on the real-world dataset show that the proposed\nmethod achieves state-of-the-art results in terms of quality and diversity",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03902v2",
            "date": "2020-04-08 09:34:24+00:00",
            "title": "Deep daxes: Mutual exclusivity arises through both learning biases and pragmatic strategies in neural networks",
            "authors": [
                "Kristina Gulordava",
                "Thomas Brochhagen",
                "Gemma Boleda"
            ],
            "abstract": "Children's tendency to associate novel words with novel referents has been\ntaken to reflect a bias toward mutual exclusivity. This tendency may be\nadvantageous both as (1) an ad-hoc referent selection heuristic to single out\nreferents lacking a label and as (2) an organizing principle of lexical\nacquisition. This paper investigates under which circumstances\ncross-situational neural models can come to exhibit analogous behavior to\nchildren, focusing on these two possibilities and their interaction. To this\nend, we evaluate neural networks' on both symbolic data and, as a first, on\nlarge-scale image data. We find that constraints in both learning and selection\ncan foster mutual exclusivity, as long as they put words in competition for\nlexical meaning. For computational models, these findings clarify the role of\navailable options for better performance in tasks where mutual exclusivity is\nadvantageous. For cognitive research, they highlight latent interactions\nbetween word learning, referent selection mechanisms, and the structure of\nstimuli of varying complexity: symbolic and visual.",
            "journal ref": "Proceedings of the Annual Conference of the Cognitive Science\n  Society, 42, 2020,2089-2095",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03965v2",
            "date": "2020-04-08 12:24:10+00:00",
            "title": "Rapformer: Conditional Rap Lyrics Generation with Denoising Autoencoders",
            "authors": [
                "Nikola I. Nikolov",
                "Eric Malmi",
                "Curtis G. Northcutt",
                "Loreto Parisi"
            ],
            "abstract": "The ability to combine symbols to generate language is a defining\ncharacteristic of human intelligence, particularly in the context of artistic\nstory-telling through lyrics. We develop a method for synthesizing a rap verse\nbased on the content of any text (e.g., a news article), or for augmenting\npre-existing rap lyrics. Our method, called Rapformer, is based on training a\nTransformer-based denoising autoencoder to reconstruct rap lyrics from content\nwords extracted from the lyrics, trying to preserve the essential meaning,\nwhile matching the target style. Rapformer features a novel BERT-based\nparaphrasing scheme for rhyme enhancement which increases the average rhyme\ndensity of output lyrics by 10%. Experimental results on three diverse input\ndomains show that Rapformer is capable of generating technically fluent verses\nthat offer a good trade-off between content preservation and style transfer.\nFurthermore, a Turing-test-like experiment reveals that Rapformer fools human\nlyrics experts 25% of the time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03974v1",
            "date": "2020-04-08 12:37:51+00:00",
            "title": "Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence",
            "authors": [
                "Federico Bianchi",
                "Silvia Terragni",
                "Dirk Hovy"
            ],
            "abstract": "Topic models extract meaningful groups of words from documents, allowing for\na better understanding of data. However, the solutions are often not coherent\nenough, and thus harder to interpret. Coherence can be improved by adding more\ncontextual knowledge to the model. Recently, neural topic models have become\navailable, while BERT-based representations have further pushed the state of\nthe art of neural models in general. We combine pre-trained representations and\nneural topic models. Pre-trained BERT sentence embeddings indeed support the\ngeneration of more meaningful and coherent topics than either standard LDA or\nexisting neural topic models. Results on four datasets show that our approach\neffectively increases topic coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04002v2",
            "date": "2020-04-08 14:19:05+00:00",
            "title": "Transfer learning and subword sampling for asymmetric-resource one-to-many neural translation",
            "authors": [
                "Stig-Arne Gr\u00f6nroos",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "There are several approaches for improving neural machine translation for\nlow-resource languages: Monolingual data can be exploited via pretraining or\ndata augmentation; Parallel corpora on related language pairs can be used via\nparameter sharing or transfer learning in multilingual models; Subword\nsegmentation and regularization techniques can be applied to ensure high\ncoverage of the vocabulary. We review these approaches in the context of an\nasymmetric-resource one-to-many translation task, in which the pair of target\nlanguages are related, with one being a very low-resource and the other a\nhigher-resource language. We test various methods on three artificially\nrestricted translation tasks -- English to Estonian (low-resource) and Finnish\n(high-resource), English to Slovak and Czech, English to Danish and Swedish --\nand one real-world task, Norwegian to North S\\'ami and Finnish. The experiments\nshow positive effects especially for scheduled multi-task learning, denoising\nautoencoder, and subword sampling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04060v2",
            "date": "2020-04-08 15:31:26+00:00",
            "title": "Self-Attention Gazetteer Embeddings for Named-Entity Recognition",
            "authors": [
                "Stanislav Peshterliev",
                "Christophe Dupuy",
                "Imre Kiss"
            ],
            "abstract": "Recent attempts to ingest external knowledge into neural models for\nnamed-entity recognition (NER) have exhibited mixed results. In this work, we\npresent GazSelfAttn, a novel gazetteer embedding approach that uses\nself-attention and match span encoding to build enhanced gazetteer embeddings.\nIn addition, we demonstrate how to build gazetteer resources from the open\nsource Wikidata knowledge base. Evaluations on CoNLL-03 and Ontonotes 5\ndatasets, show F1 improvements over baseline model from 92.34 to 92.86 and\n89.11 to 89.32 respectively, achieving performance comparable to large\nstate-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04070v2",
            "date": "2020-04-08 15:49:19+00:00",
            "title": "Are All Good Word Vector Spaces Isomorphic?",
            "authors": [
                "Ivan Vuli\u0107",
                "Sebastian Ruder",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Existing algorithms for aligning cross-lingual word vector spaces assume that\nvector spaces are approximately isomorphic. As a result, they perform poorly or\nfail completely on non-isomorphic spaces. Such non-isomorphism has been\nhypothesised to result from typological differences between languages. In this\nwork, we ask whether non-isomorphism is also crucially a sign of degenerate\nword vector spaces. We present a series of experiments across diverse languages\nwhich show that variance in performance across language pairs is not only due\nto typological differences, but can mostly be attributed to the size of the\nmonolingual resources available, and to the properties and duration of\nmonolingual training (e.g. \"under-training\").",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04103v2",
            "date": "2020-04-08 16:28:16+00:00",
            "title": "Cross-lingual Emotion Intensity Prediction",
            "authors": [
                "Irean Navas Alejo",
                "Toni Badia",
                "Jeremy Barnes"
            ],
            "abstract": "Emotion intensity prediction determines the degree or intensity of an emotion\nthat the author expresses in a text, extending previous categorical approaches\nto emotion detection. While most previous work on this topic has concentrated\non English texts, other languages would also benefit from fine-grained emotion\nclassification, preferably without having to recreate the amount of annotated\ndata available in English in each new language. Consequently, we explore\ncross-lingual transfer approaches for fine-grained emotion detection in Spanish\nand Catalan tweets. To this end we annotate a test set of Spanish and Catalan\ntweets using Best-Worst scaling. We compare six cross-lingual approaches, e.g.,\nmachine translation and cross-lingual embeddings, which have varying\nrequirements for parallel data -- from millions of parallel sentences to\ncompletely unsupervised. The results show that on this data, methods with low\nparallel-data requirements perform surprisingly better than methods that use\nmore parallel data, which we explain through an in-depth error analysis. We\nmake the dataset and the code available at\n\\url{https://github.com/jerbarnes/fine-grained_cross-lingual_emotion}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04106v1",
            "date": "2020-04-08 16:34:19+00:00",
            "title": "Frequency, Acceptability, and Selection: A case study of clause-embedding",
            "authors": [
                "Aaron Steven White",
                "Kyle Rawlins"
            ],
            "abstract": "We investigate the relationship between the frequency with which verbs are\nfound in particular subcategorization frames and the acceptability of those\nverbs in those frames, focusing in particular on subordinate clause-taking\nverbs, such as \"think\", \"want\", and \"tell\". We show that verbs'\nsubcategorization frame frequency distributions are poor predictors of their\nacceptability in those frames---explaining, at best, less than 1/3 of the total\ninformation about acceptability across the lexicon---and, further, that common\nmatrix factorization techniques used to model the acquisition of verbs'\nacceptability in subcategorization frames fare only marginally better. All data\nand code are available at http://megaattitude.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04123v2",
            "date": "2020-04-08 17:11:31+00:00",
            "title": "Entity-Switched Datasets: An Approach to Auditing the In-Domain Robustness of Named Entity Recognition Models",
            "authors": [
                "Oshin Agarwal",
                "Yinfei Yang",
                "Byron C. Wallace",
                "Ani Nenkova"
            ],
            "abstract": "Named entity recognition systems perform well on standard datasets comprising\nEnglish news. But given the paucity of data, it is difficult to draw\nconclusions about the robustness of systems with respect to recognizing a\ndiverse set of entities. We propose a method for auditing the in-domain\nrobustness of systems, focusing specifically on differences in performance due\nto the national origin of entities. We create entity-switched datasets, in\nwhich named entities in the original texts are replaced by plausible named\nentities of the same type but of different national origin. We find that\nstate-of-the-art systems' performance vary widely even in-domain: In the same\ncontext, entities from certain origins are more reliably recognized than\nentities from elsewhere. Systems perform best on American and Indian entities,\nand worst on Vietnamese and Indonesian entities. This auditing approach can\nfacilitate the development of more robust named entity recognition systems, and\nwill allow research in this area to consider fairness criteria that have\nreceived heightened attention in other predictive technology work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04228v1",
            "date": "2020-04-08 20:01:09+00:00",
            "title": "Asking and Answering Questions to Evaluate the Factual Consistency of Summaries",
            "authors": [
                "Alex Wang",
                "Kyunghyun Cho",
                "Mike Lewis"
            ],
            "abstract": "Practical applications of abstractive summarization models are limited by\nfrequent factual inconsistencies with respect to their input. Existing\nautomatic evaluation metrics for summarization are largely insensitive to such\nerrors. We propose an automatic evaluation protocol called QAGS (pronounced\n\"kags\") that is designed to identify factual inconsistencies in a generated\nsummary. QAGS is based on the intuition that if we ask questions about a\nsummary and its source, we will receive similar answers if the summary is\nfactually consistent with the source. To evaluate QAGS, we collect human\njudgments of factual consistency on model-generated summaries for the\nCNN/DailyMail (Hermann et al., 2015) and XSUM (Narayan et al., 2018)\nsummarization datasets. QAGS has substantially higher correlations with these\njudgments than other automatic evaluation metrics. Also, QAGS offers a natural\nform of interpretability: The answers and questions generated while computing\nQAGS indicate which tokens of a summary are inconsistent and why. We believe\nQAGS is a promising tool in automatically generating usable and factually\nconsistent text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04243v1",
            "date": "2020-04-08 20:49:10+00:00",
            "title": "Error-correction and extraction in request dialogs",
            "authors": [
                "Stefan Constantin",
                "Alex Waibel"
            ],
            "abstract": "We propose a component that gets a request and a correction and outputs a\ncorrected request. To get this corrected request, the entities in the\ncorrection phrase replace their corresponding entities in the request. In\naddition, the proposed component outputs these pairs of corresponding\nreparandum and repair entity. These entity pairs can be used, for example, for\nlearning in a life-long learning component of a dialog system to reduce the\nneed for correction in future dialogs. For the approach described in this work,\nwe fine-tune BERT for sequence labeling. We created a dataset to evaluate our\ncomponent; for which we got an accuracy of 93.28 %. An accuracy of 88.58 % has\nbeen achieved for out-of-domain data. This accuracy shows that the proposed\ncomponent is learning the concept of corrections and can be developed to be\nused as an upstream component to avoid the need for collecting data for request\ncorrections for every new domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04270v3",
            "date": "2020-04-08 21:25:00+00:00",
            "title": "The Spotify Podcast Dataset",
            "authors": [
                "Ann Clifton",
                "Aasish Pappu",
                "Sravana Reddy",
                "Yongze Yu",
                "Jussi Karlgren",
                "Ben Carterette",
                "Rosie Jones"
            ],
            "abstract": "Podcasts are a relatively new form of audio media. Episodes appear on a\nregular cadence, and come in many different formats and levels of formality.\nThey can be formal news journalism or conversational chat; fiction or\nnon-fiction. They are rapidly growing in popularity and yet have been\nrelatively little studied. As an audio format, podcasts are more varied in\nstyle and production types than, say, broadcast news, and contain many more\ngenres than typically studied in video research. The medium is therefore a rich\ndomain with many research avenues for the IR and NLP communities. We present\nthe Spotify Podcast Dataset, a set of approximately 100K podcast episodes\ncomprised of raw audio files along with accompanying ASR transcripts. This\nrepresents over 47,000 hours of transcribed audio, and is an order of magnitude\nlarger than previous speech-to-text corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04295v1",
            "date": "2020-04-08 23:17:10+00:00",
            "title": "Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events",
            "authors": [
                "Miguel Ballesteros",
                "Rishita Anubhai",
                "Shuai Wang",
                "Nima Pourdamghani",
                "Yogarshi Vyas",
                "Jie Ma",
                "Parminder Bhatia",
                "Kathleen McKeown",
                "Yaser Al-Onaizan"
            ],
            "abstract": "In this paper, we propose a neural architecture and a set of training methods\nfor ordering events by predicting temporal relations. Our proposed models\nreceive a pair of events within a span of text as input and they identify\ntemporal relations (Before, After, Equal, Vague) between them. Given that a key\nchallenge with this task is the scarcity of annotated data, our models rely on\neither pretrained representations (i.e. RoBERTa, BERT or ELMo), transfer and\nmulti-task learning (by leveraging complementary datasets), and self-training\ntechniques. Experiments on the MATRES dataset of English documents establish a\nnew state-of-the-art on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04438v1",
            "date": "2020-04-09 09:26:42+00:00",
            "title": "Improving Readability for Automatic Speech Recognition Transcription",
            "authors": [
                "Junwei Liao",
                "Sefik Emre Eskimez",
                "Liyang Lu",
                "Yu Shi",
                "Ming Gong",
                "Linjun Shou",
                "Hong Qu",
                "Michael Zeng"
            ],
            "abstract": "Modern Automatic Speech Recognition (ASR) systems can achieve high\nperformance in terms of recognition accuracy. However, a perfectly accurate\ntranscript still can be challenging to read due to grammatical errors,\ndisfluency, and other errata common in spoken communication. Many downstream\ntasks and human readers rely on the output of the ASR system; therefore, errors\nintroduced by the speaker and ASR system alike will be propagated to the next\ntask in the pipeline. In this work, we propose a novel NLP task called ASR\npost-processing for readability (APR) that aims to transform the noisy ASR\noutput into a readable text for humans and downstream tasks while maintaining\nthe semantic meaning of the speaker. In addition, we describe a method to\naddress the lack of task-specific data by synthesizing examples for the APR\ntask using the datasets collected for Grammatical Error Correction (GEC)\nfollowed by text-to-speech (TTS) and ASR. Furthermore, we propose metrics\nborrowed from similar tasks to evaluate performance on the APR task. We compare\nfine-tuned models based on several open-sourced and adapted pre-trained models\nwith the traditional pipeline method. Our results suggest that finetuned models\nimprove the performance on the APR task significantly, hinting at the potential\nbenefits of using APR systems. We hope that the read, understand, and rewrite\napproach of our work can serve as a basis that many NLP tasks and human readers\ncan benefit from.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04468v1",
            "date": "2020-04-09 10:35:16+00:00",
            "title": "A Multilingual Study of Multi-Sentence Compression using Word Vertex-Labeled Graphs and Integer Linear Programming",
            "authors": [
                "Elvys Linhares Pontes",
                "St\u00e9phane Huet",
                "Juan-Manuel Torres-Moreno",
                "Thiago G. da Silva",
                "Andr\u00e9a Carneiro Linhares"
            ],
            "abstract": "Multi-Sentence Compression (MSC) aims to generate a short sentence with the\nkey information from a cluster of similar sentences. MSC enables summarization\nand question-answering systems to generate outputs combining fully formed\nsentences from one or several documents. This paper describes an Integer Linear\nProgramming method for MSC using a vertex-labeled graph to select different\nkeywords, with the goal of generating more informative sentences while\nmaintaining their grammaticality. Our system is of good quality and outperforms\nthe state of the art for evaluations led on news datasets in three languages:\nFrench, Portuguese and Spanish. We led both automatic and manual evaluations to\ndetermine the informativeness and the grammaticality of compressions for each\ndataset. In additional tests, which take advantage of the fact that the length\nof compressions can be modulated, we still improve ROUGE scores with shorter\noutput sentences.",
            "journal ref": "Computaci\\'on y Sistemas Vo. 24, No. 2, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04478v1",
            "date": "2020-04-09 10:55:01+00:00",
            "title": "Recommendation Chart of Domains for Cross-Domain Sentiment Analysis:Findings of A 20 Domain Study",
            "authors": [
                "Akash Sheoran",
                "Diptesh Kanojia",
                "Aditya Joshi",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Cross-domain sentiment analysis (CDSA) helps to address the problem of data\nscarcity in scenarios where labelled data for a domain (known as the target\ndomain) is unavailable or insufficient. However, the decision to choose a\ndomain (known as the source domain) to leverage from is, at best, intuitive. In\nthis paper, we investigate text similarity metrics to facilitate source domain\nselection for CDSA. We report results on 20 domains (all possible pairs) using\n11 similarity metrics. Specifically, we compare CDSA performance with these\nmetrics for different domain-pairs to enable the selection of a suitable source\ndomain, given a target domain. These metrics include two novel metrics for\nevaluating domain adaptability to help source domain selection of labelled data\nand utilize word and sentence-based embeddings as metrics for unlabelled data.\nThe goal of our experiments is a recommendation chart that gives the K best\nsource domains for CDSA for a given target domain. We show that the best K\nsource domains returned by our similarity metrics have a precision of over 50%,\nfor varying values of K.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04487v1",
            "date": "2020-04-09 11:14:56+00:00",
            "title": "Injecting Numerical Reasoning Skills into Language Models",
            "authors": [
                "Mor Geva",
                "Ankit Gupta",
                "Jonathan Berant"
            ],
            "abstract": "Large pre-trained language models (LMs) are known to encode substantial\namounts of linguistic information. However, high-level reasoning skills, such\nas numerical reasoning, are difficult to learn from a language-modeling\nobjective only. Consequently, existing models for numerical reasoning have used\nspecialized architectures with limited flexibility. In this work, we show that\nnumerical reasoning is amenable to automatic data generation, and thus one can\ninject this skill into pre-trained LMs, by generating large amounts of data,\nand training in a multi-task setup. We show that pre-training our model,\nGenBERT, on this data, dramatically improves performance on DROP (49.3\n$\\rightarrow$ 72.3 F1), reaching performance that matches state-of-the-art\nmodels of comparable size, while using a simple and general-purpose\nencoder-decoder architecture. Moreover, GenBERT generalizes well to math word\nproblem datasets, while maintaining high performance on standard RC tasks. Our\napproach provides a general recipe for injecting skills into large pre-trained\nLMs, whenever the skill is amenable to automatic data augmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04494v1",
            "date": "2020-04-09 11:42:33+00:00",
            "title": "MuTual: A Dataset for Multi-Turn Dialogue Reasoning",
            "authors": [
                "Leyang Cui",
                "Yu Wu",
                "Shujie Liu",
                "Yue Zhang",
                "Ming Zhou"
            ],
            "abstract": "Non-task oriented dialogue systems have achieved great success in recent\nyears due to largely accessible conversation data and the development of deep\nlearning techniques. Given a context, current systems are able to yield a\nrelevant and fluent response, but sometimes make logical mistakes because of\nweak reasoning capabilities. To facilitate the conversation reasoning research,\nwe introduce MuTual, a novel dataset for Multi-Turn dialogue Reasoning,\nconsisting of 8,860 manually annotated dialogues based on Chinese student\nEnglish listening comprehension exams. Compared to previous benchmarks for\nnon-task oriented dialogue systems, MuTual is much more challenging since it\nrequires a model that can handle various reasoning problems. Empirical results\nshow that state-of-the-art methods only reach 71%, which is far behind the\nhuman performance of 94%, indicating that there is ample room for improving\nreasoning ability. MuTual is available at https://github.com/Nealcly/MuTual.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04498v3",
            "date": "2020-04-09 11:55:13+00:00",
            "title": "Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem",
            "authors": [
                "Danielle Saunders",
                "Bill Byrne"
            ],
            "abstract": "Training data for NLP tasks often exhibits gender bias in that fewer\nsentences refer to women than to men. In Neural Machine Translation (NMT)\ngender bias has been shown to reduce translation quality, particularly when the\ntarget language has grammatical gender. The recent WinoMT challenge set allows\nus to measure this effect directly (Stanovsky et al, 2019).\n  Ideally we would reduce system bias by simply debiasing all data prior to\ntraining, but achieving this effectively is itself a challenge. Rather than\nattempt to create a `balanced' dataset, we use transfer learning on a small set\nof trusted, gender-balanced examples. This approach gives strong and consistent\nimprovements in gender debiasing with much less computational cost than\ntraining from scratch.\n  A known pitfall of transfer learning on new domains is `catastrophic\nforgetting', which we address both in adaptation and in inference. During\nadaptation we show that Elastic Weight Consolidation allows a performance\ntrade-off between general translation quality and bias reduction. During\ninference we propose a lattice-rescoring scheme which outperforms all systems\nevaluated in Stanovsky et al (2019) on WinoMT with no degradation of general\ntest set BLEU, and we show this scheme can be applied to remove gender bias in\nthe output of `black box` online commercial MT systems. We demonstrate our\napproach translating from English into three languages with varied linguistic\nproperties and data availability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04507v2",
            "date": "2020-04-09 12:07:17+00:00",
            "title": "Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios",
            "authors": [
                "Haipeng Sun",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Tiejun Zhao"
            ],
            "abstract": "Unsupervised neural machine translation (UNMT) that relies solely on massive\nmonolingual corpora has achieved remarkable results in several translation\ntasks. However, in real-world scenarios, massive monolingual corpora do not\nexist for some extremely low-resource languages such as Estonian, and UNMT\nsystems usually perform poorly when there is not adequate training corpus for\none language. In this paper, we first define and analyze the unbalanced\ntraining data scenario for UNMT. Based on this scenario, we propose UNMT\nself-training mechanisms to train a robust UNMT system and improve its\nperformance in this case. Experimental results on several language pairs show\nthat the proposed methods substantially outperform conventional UNMT systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04564v2",
            "date": "2020-04-09 14:37:12+00:00",
            "title": "Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve",
            "authors": [
                "Oshin Agarwal",
                "Yinfei Yang",
                "Byron C. Wallace",
                "Ani Nenkova"
            ],
            "abstract": "Named Entity Recognition systems achieve remarkable performance on domains\nsuch as English news. It is natural to ask: What are these models actually\nlearning to achieve this? Are they merely memorizing the names themselves? Or\nare they capable of interpreting the text and inferring the correct entity type\nfrom the linguistic context? We examine these questions by contrasting the\nperformance of several variants of LSTM-CRF architectures for named entity\nrecognition, with some provided only representations of the context as\nfeatures. We also perform similar experiments for BERT. We find that context\nrepresentations do contribute to system performance, but that the main factor\ndriving high performance is learning the name tokens themselves. We enlist\nhuman annotators to evaluate the feasibility of inferring entity types from the\ncontext alone and find that, while people are not able to infer the entity type\neither for the majority of the errors made by the context-only system, there is\nsome room for improvement. A system should be able to recognize any name in a\npredictive context correctly and our experiments indicate that current systems\nmay be further improved by such capability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04696v5",
            "date": "2020-04-09 17:26:52+00:00",
            "title": "BLEURT: Learning Robust Metrics for Text Generation",
            "authors": [
                "Thibault Sellam",
                "Dipanjan Das",
                "Ankur P. Parikh"
            ],
            "abstract": "Text generation has made significant advances in the last few years. Yet,\nevaluation metrics have lagged behind, as the most popular choices (e.g., BLEU\nand ROUGE) may correlate poorly with human judgments. We propose BLEURT, a\nlearned evaluation metric based on BERT that can model human judgments with a\nfew thousand possibly biased training examples. A key aspect of our approach is\na novel pre-training scheme that uses millions of synthetic examples to help\nthe model generalize. BLEURT provides state-of-the-art results on the last\nthree years of the WMT Metrics shared task and the WebNLG Competition dataset.\nIn contrast to a vanilla BERT-based approach, it yields superior results even\nwhen the training data is scarce and out-of-distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04877v2",
            "date": "2020-04-10 01:48:50+00:00",
            "title": "Probing Neural Language Models for Human Tacit Assumptions",
            "authors": [
                "Nathaniel Weir",
                "Adam Poliak",
                "Benjamin Van Durme"
            ],
            "abstract": "Humans carry stereotypic tacit assumptions (STAs) (Prince, 1978), or\npropositional beliefs about generic concepts. Such associations are crucial for\nunderstanding natural language. We construct a diagnostic set of word\nprediction prompts to evaluate whether recent neural contextualized language\nmodels trained on large text corpora capture STAs. Our prompts are based on\nhuman responses in a psychological study of conceptual associations. We find\nmodels to be profoundly effective at retrieving concepts given associated\nproperties. Our results demonstrate empirical evidence that stereotypic\nconceptual representations are captured in neural models derived from\nsemi-supervised linguistic exposure.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04906v3",
            "date": "2020-04-10 04:53:17+00:00",
            "title": "Dense Passage Retrieval for Open-Domain Question Answering",
            "authors": [
                "Vladimir Karpukhin",
                "Barlas O\u011fuz",
                "Sewon Min",
                "Patrick Lewis",
                "Ledell Wu",
                "Sergey Edunov",
                "Danqi Chen",
                "Wen-tau Yih"
            ],
            "abstract": "Open-domain question answering relies on efficient passage retrieval to\nselect candidate contexts, where traditional sparse vector space models, such\nas TF-IDF or BM25, are the de facto method. In this work, we show that\nretrieval can be practically implemented using dense representations alone,\nwhere embeddings are learned from a small number of questions and passages by a\nsimple dual-encoder framework. When evaluated on a wide range of open-domain QA\ndatasets, our dense retriever outperforms a strong Lucene-BM25 system largely\nby 9%-19% absolute in terms of top-20 passage retrieval accuracy, and helps our\nend-to-end QA system establish new state-of-the-art on multiple open-domain QA\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04908v2",
            "date": "2020-04-10 04:59:37+00:00",
            "title": "Designing Precise and Robust Dialogue Response Evaluators",
            "authors": [
                "Tianyu Zhao",
                "Divesh Lala",
                "Tatsuya Kawahara"
            ],
            "abstract": "Automatic dialogue response evaluator has been proposed as an alternative to\nautomated metrics and human evaluation. However, existing automatic evaluators\nachieve only moderate correlation with human judgement and they are not robust.\nIn this work, we propose to build a reference-free evaluator and exploit the\npower of semi-supervised training and pretrained (masked) language models.\nExperimental results demonstrate that the proposed evaluator achieves a strong\ncorrelation (> 0.6) with human judgement and generalizes robustly to diverse\nresponses and corpora. We open-source the code and data in\nhttps://github.com/ZHAOTING/dialog-processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04934v1",
            "date": "2020-04-10 08:00:40+00:00",
            "title": "Scalable Multilingual Frontend for TTS",
            "authors": [
                "Alistair Conkie",
                "Andrew Finch"
            ],
            "abstract": "This paper describes progress towards making a Neural Text-to-Speech (TTS)\nFrontend that works for many languages and can be easily extended to new\nlanguages. We take a Machine Translation (MT) inspired approach to constructing\nthe frontend, and model both text normalization and pronunciation on a sentence\nlevel by building and using sequence-to-sequence (S2S) models. We experimented\nwith training normalization and pronunciation as separate S2S models and with\ntraining a single S2S model combining both functions.\n  For our language-independent approach to pronunciation we do not use a\nlexicon. Instead all pronunciations, including context-based pronunciations,\nare captured in the S2S model. We also present a language-independent chunking\nand splicing technique that allows us to process arbitrary-length sentences.\nModels for 18 languages were trained and evaluated. Many of the accuracy\nmeasurements are above 99%. We also evaluated the models in the context of\nend-to-end synthesis against our current production system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04987v1",
            "date": "2020-04-10 10:59:44+00:00",
            "title": "Automated Spelling Correction for Clinical Text Mining in Russian",
            "authors": [
                "Ksenia Balabaeva",
                "Anastasia Funkner",
                "Sergey Kovalchuk"
            ],
            "abstract": "The main goal of this paper is to develop a spell checker module for clinical\ntext in Russian. The described approach combines string distance measure\nalgorithms with technics of machine learning embedding methods. Our overall\nprecision is 0.86, lexical precision - 0.975 and error precision is 0.74. We\ndevelop spell checker as a part of medical text mining tool regarding the\nproblems of misspelling, negation, experiencer and temporality detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05009v2",
            "date": "2020-04-10 12:24:49+00:00",
            "title": "Minimum Latency Training Strategies for Streaming Sequence-to-Sequence ASR",
            "authors": [
                "Hirofumi Inaguma",
                "Yashesh Gaur",
                "Liang Lu",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "Recently, a few novel streaming attention-based sequence-to-sequence (S2S)\nmodels have been proposed to perform online speech recognition with linear-time\ndecoding complexity. However, in these models, the decisions to generate tokens\nare delayed compared to the actual acoustic boundaries since their\nunidirectional encoders lack future information. This leads to an inevitable\nlatency during inference. To alleviate this issue and reduce latency, we\npropose several strategies during training by leveraging external hard\nalignments extracted from the hybrid model. We investigate to utilize the\nalignments in both the encoder and the decoder. On the encoder side, (1)\nmulti-task learning and (2) pre-training with the framewise classification task\nare studied. On the decoder side, we (3) remove inappropriate alignment paths\nbeyond an acceptable latency during the alignment marginalization, and (4)\ndirectly minimize the differentiable expected latency loss. Experiments on the\nCortana voice search task demonstrate that our proposed methods can\nsignificantly reduce the latency, and even improve the recognition accuracy in\ncertain cases on the decoder side. We also present some analysis to understand\nthe behaviors of streaming S2S models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05051v2",
            "date": "2020-04-10 14:32:01+00:00",
            "title": "A New Dataset for Natural Language Inference from Code-mixed Conversations",
            "authors": [
                "Simran Khanuja",
                "Sandipan Dandapat",
                "Sunayana Sitaram",
                "Monojit Choudhury"
            ],
            "abstract": "Natural Language Inference (NLI) is the task of inferring the logical\nrelationship, typically entailment or contradiction, between a premise and\nhypothesis. Code-mixing is the use of more than one language in the same\nconversation or utterance, and is prevalent in multilingual communities all\nover the world. In this paper, we present the first dataset for code-mixed NLI,\nin which both the premises and hypotheses are in code-mixed Hindi-English. We\nuse data from Hindi movies (Bollywood) as premises, and crowd-source hypotheses\nfrom Hindi-English bilinguals. We conduct a pilot annotation study and describe\nthe final annotation protocol based on observations from the pilot. Currently,\nthe data collected consists of 400 premises in the form of code-mixed\nconversation snippets and 2240 code-mixed hypotheses. We conduct an extensive\nanalysis to infer the linguistic phenomena commonly observed in the dataset\nobtained. We evaluate the dataset using a standard mBERT-based pipeline for NLI\nand report results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05067v1",
            "date": "2020-04-10 15:13:03+00:00",
            "title": "Overestimation of Syntactic Representationin Neural Language Models",
            "authors": [
                "Jordan Kodner",
                "Nitish Gupta"
            ],
            "abstract": "With the advent of powerful neural language models over the last few years,\nresearch attention has increasingly focused on what aspects of language they\nrepresent that make them so successful. Several testing methodologies have been\ndeveloped to probe models' syntactic representations. One popular method for\ndetermining a model's ability to induce syntactic structure trains a model on\nstrings generated according to a template then tests the model's ability to\ndistinguish such strings from superficially similar ones with different syntax.\nWe illustrate a fundamental problem with this approach by reproducing positive\nresults from a recent paper with two non-syntactic baseline language models: an\nn-gram model and an LSTM model trained on scrambled inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05080v3",
            "date": "2020-04-10 15:52:08+00:00",
            "title": "Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure",
            "authors": [
                "Jiaqi Li",
                "Ming Liu",
                "Min-Yen Kan",
                "Zihao Zheng",
                "Zekun Wang",
                "Wenqiang Lei",
                "Ting Liu",
                "Bing Qin"
            ],
            "abstract": "Research into the area of multiparty dialog has grown considerably over\nrecent years. We present the Molweni dataset, a machine reading comprehension\n(MRC) dataset with discourse structure built over multiparty dialog. Molweni's\nsource samples from the Ubuntu Chat Corpus, including 10,000 dialogs comprising\n88,303 utterances. We annotate 30,066 questions on this corpus, including both\nanswerable and unanswerable questions. Molweni also uniquely contributes\ndiscourse dependency annotations in a modified Segmented Discourse\nRepresentation Theory (SDRT; Asher et al., 2016) style for all of its\nmultiparty dialogs, contributing large-scale (78,245 annotated discourse\nrelations) data to bear on the task of multiparty dialog discourse parsing. Our\nexperiments show that Molweni is a challenging dataset for current MRC models:\nBERT-wwm, a current, strong SQuAD 2.0 performer, achieves only 67.7% F1 on\nMolweni's questions, a 20+% significant drop as compared against its SQuAD 2.0\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05109v3",
            "date": "2020-04-10 16:45:08+00:00",
            "title": "Towards Automatic Generation of Questions from Long Answers",
            "authors": [
                "Shlok Kumar Mishra",
                "Pranav Goel",
                "Abhishek Sharma",
                "Abhyuday Jagannatha",
                "David Jacobs",
                "Hal Daum\u00e9 III"
            ],
            "abstract": "Automatic question generation (AQG) has broad applicability in domains such\nas tutoring systems, conversational agents, healthcare literacy, and\ninformation retrieval. Existing efforts at AQG have been limited to short\nanswer lengths of up to two or three sentences. However, several real-world\napplications require question generation from answers that span several\nsentences. Therefore, we propose a novel evaluation benchmark to assess the\nperformance of existing AQG systems for long-text answers. We leverage the\nlarge-scale open-source Google Natural Questions dataset to create the\naforementioned long-answer AQG benchmark. We empirically demonstrate that the\nperformance of existing AQG methods significantly degrades as the length of the\nanswer increases. Transformer-based methods outperform other existing AQG\nmethods on long answers in terms of automatic as well as human evaluation.\nHowever, we still observe degradation in the performance of our best performing\nmodels with increasing sentence length, suggesting that long answer QA is a\nchallenging benchmark task for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05119v2",
            "date": "2020-04-10 16:57:06+00:00",
            "title": "Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer",
            "authors": [
                "Siddhant Garg",
                "Rohit Kumar Sharma",
                "Yingyu Liang"
            ],
            "abstract": "Fine-tuning (FT) pre-trained sentence embedding models on small datasets has\nbeen shown to have limitations. In this paper we show that concatenating the\nembeddings from the pre-trained model with those from a simple sentence\nembedding model trained only on the target data, can improve over the\nperformance of FT for few-sample tasks. To this end, a linear classifier is\ntrained on the combined embeddings, either by freezing the embedding model\nweights or training the classifier and embedding models end-to-end. We perform\nevaluation on seven small datasets from NLP tasks and show that our approach\nwith end-to-end training outperforms FT with negligible computational overhead.\nFurther, we also show that sophisticated combination techniques like CCA and\nKCCA do not work as well in practice as concatenation. We provide theoretical\nanalysis to explain this empirical observation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05140v2",
            "date": "2020-04-10 17:36:27+00:00",
            "title": "One Model to Recognize Them All: Marginal Distillation from NER Models with Different Tag Sets",
            "authors": [
                "Keunwoo Peter Yu",
                "Yi Yang"
            ],
            "abstract": "Named entity recognition (NER) is a fundamental component in the modern\nlanguage understanding pipeline. Public NER resources such as annotated data\nand model services are available in many domains. However, given a particular\ndownstream application, there is often no single NER resource that supports all\nthe desired entity types, so users must leverage multiple resources with\ndifferent tag sets. This paper presents a marginal distillation (MARDI)\napproach for training a unified NER model from resources with disjoint or\nheterogeneous tag sets. In contrast to recent works, MARDI merely requires\naccess to pre-trained models rather than the original training datasets. This\nflexibility makes it easier to work with sensitive domains like healthcare and\nfinance. Furthermore, our approach is general enough to integrate with\ndifferent NER architectures, including local models (e.g., BiLSTM) and global\nmodels (e.g., CRF). Experiments on two benchmark datasets show that MARDI\nperforms on par with a strong marginal CRF baseline, while being more flexible\nin the form of required NER resources. MARDI also sets a new state of the art\non the progressive NER task. MARDI significantly outperforms the\nstart-of-the-art model on the task of progressive NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05150v2",
            "date": "2020-04-10 17:54:09+00:00",
            "title": "Longformer: The Long-Document Transformer",
            "authors": [
                "Iz Beltagy",
                "Matthew E. Peters",
                "Arman Cohan"
            ],
            "abstract": "Transformer-based models are unable to process long sequences due to their\nself-attention operation, which scales quadratically with the sequence length.\nTo address this limitation, we introduce the Longformer with an attention\nmechanism that scales linearly with sequence length, making it easy to process\ndocuments of thousands of tokens or longer. Longformer's attention mechanism is\na drop-in replacement for the standard self-attention and combines a local\nwindowed attention with a task motivated global attention. Following prior work\non long-sequence transformers, we evaluate Longformer on character-level\nlanguage modeling and achieve state-of-the-art results on text8 and enwik8. In\ncontrast to most prior work, we also pretrain Longformer and finetune it on a\nvariety of downstream tasks. Our pretrained Longformer consistently outperforms\nRoBERTa on long document tasks and sets new state-of-the-art results on WikiHop\nand TriviaQA. We finally introduce the Longformer-Encoder-Decoder (LED), a\nLongformer variant for supporting long document generative sequence-to-sequence\ntasks, and demonstrate its effectiveness on the arXiv summarization dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05160v4",
            "date": "2020-04-09 19:50:32+00:00",
            "title": "On the Language Neutrality of Pre-trained Multilingual Representations",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Rudolf Rosa",
                "Alexander Fraser"
            ],
            "abstract": "Multilingual contextual embeddings, such as multilingual BERT and\nXLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work\nprobed the cross-linguality of the representations indirectly using zero-shot\ntransfer learning on morphological and syntactic tasks. We instead investigate\nthe language-neutrality of multilingual contextual embeddings directly and with\nrespect to lexical semantics. Our results show that contextual embeddings are\nmore language-neutral and, in general, more informative than aligned static\nword-type embeddings, which are explicitly trained for language neutrality.\nContextual embeddings are still only moderately language-neutral by default, so\nwe propose two simple methods for achieving stronger language neutrality:\nfirst, by unsupervised centering of the representation for each language and\nsecond, by fitting an explicit projection on small parallel data. Besides, we\nshow how to reach state-of-the-art accuracy on language identification and\nmatch the performance of statistical methods for word alignment of parallel\nsentences without using parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05323v2",
            "date": "2020-04-11 06:53:08+00:00",
            "title": "Improving Disfluency Detection by Self-Training a Self-Attentive Model",
            "authors": [
                "Paria Jamshid Lou",
                "Mark Johnson"
            ],
            "abstract": "Self-attentive neural syntactic parsers using contextualized word embeddings\n(e.g. ELMo or BERT) currently produce state-of-the-art results in joint parsing\nand disfluency detection in speech transcripts. Since the contextualized word\nembeddings are pre-trained on a large amount of unlabeled data, using\nadditional unlabeled data to train a neural model might seem redundant.\nHowever, we show that self-training - a semi-supervised technique for\nincorporating unlabeled data - sets a new state-of-the-art for the\nself-attentive parser on disfluency detection, demonstrating that self-training\nprovides benefits orthogonal to the pre-trained contextualized word\nrepresentations. We also show that ensembling self-trained parsers provides\nfurther gains for disfluency detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05456v1",
            "date": "2020-04-11 18:17:18+00:00",
            "title": "End to End Chinese Lexical Fusion Recognition with Sememe Knowledge",
            "authors": [
                "Yijiang Liu",
                "Meishan Zhang",
                "Donghong Ji"
            ],
            "abstract": "In this paper, we present Chinese lexical fusion recognition, a new task\nwhich could be regarded as one kind of coreference recognition. First, we\nintroduce the task in detail, showing the relationship with coreference\nrecognition and differences from the existing tasks. Second, we propose an\nend-to-end joint model for the task, which exploits the state-of-the-art BERT\nrepresentations as encoder, and is further enhanced with the sememe knowledge\nfrom HowNet by graph attention networks. We manually annotate a benchmark\ndataset for the task and then conduct experiments on it. Results demonstrate\nthat our joint model is effective and competitive for the task. Detailed\nanalysis is offered for comprehensively understanding the new task and our\nproposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05483v2",
            "date": "2020-04-11 20:43:37+00:00",
            "title": "Unsupervised Commonsense Question Answering with Self-Talk",
            "authors": [
                "Vered Shwartz",
                "Peter West",
                "Ronan Le Bras",
                "Chandra Bhagavatula",
                "Yejin Choi"
            ],
            "abstract": "Natural language understanding involves reading between the lines with\nimplicit background knowledge. Current systems either rely on pre-trained\nlanguage models as the sole implicit source of world knowledge, or resort to\nexternal knowledge bases (KBs) to incorporate additional relevant knowledge. We\npropose an unsupervised framework based on self-talk as a novel alternative to\nmultiple-choice commonsense tasks. Inspired by inquiry-based discovery learning\n(Bruner, 1961), our approach inquires language models with a number of\ninformation seeking questions such as \"$\\textit{what is the definition of\n...}$\" to discover additional background knowledge. Empirical results\ndemonstrate that the self-talk procedure substantially improves the performance\nof zero-shot language model baselines on four out of six commonsense\nbenchmarks, and competes with models that obtain knowledge from external KBs.\nWhile our approach improves performance on several benchmarks, the self-talk\ninduced knowledge even when leading to correct answers is not always seen as\nuseful by human judges, raising interesting questions about the inner-workings\nof pre-trained language models for commonsense reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05516v3",
            "date": "2020-04-12 00:57:47+00:00",
            "title": "When Does Unsupervised Machine Translation Work?",
            "authors": [
                "Kelly Marchisio",
                "Kevin Duh",
                "Philipp Koehn"
            ],
            "abstract": "Despite the reported success of unsupervised machine translation (MT), the\nfield has yet to examine the conditions under which these methods succeed, and\nwhere they fail. We conduct an extensive empirical evaluation of unsupervised\nMT using dissimilar language pairs, dissimilar domains, diverse datasets, and\nauthentic low-resource languages. We find that performance rapidly deteriorates\nwhen source and target corpora are from different domains, and that random word\nembedding initialization can dramatically affect downstream translation\nperformance. We additionally find that unsupervised MT performance declines\nwhen source and target languages use different scripts, and observe very poor\nperformance on authentic low-resource language pairs. We advocate for extensive\nempirical evaluation of unsupervised MT systems to highlight failure points and\nencourage continued research on the most promising paradigms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05568v1",
            "date": "2020-04-12 09:05:47+00:00",
            "title": "Pre-training Text Representations as Meta Learning",
            "authors": [
                "Shangwen Lv",
                "Yuechen Wang",
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Fuqing Zhu",
                "Ming Gong",
                "Linjun Shou",
                "Ryan Ma",
                "Daxin Jiang",
                "Guihong Cao",
                "Ming Zhou",
                "Songlin Hu"
            ],
            "abstract": "Pre-training text representations has recently been shown to significantly\nimprove the state-of-the-art in many natural language processing tasks. The\ncentral goal of pre-training is to learn text representations that are useful\nfor subsequent tasks. However, existing approaches are optimized by minimizing\na proxy objective, such as the negative log likelihood of language modeling. In\nthis work, we introduce a learning algorithm which directly optimizes model's\nability to learn text representations for effective learning of downstream\ntasks. We show that there is an intrinsic connection between multi-task\npre-training and model-agnostic meta-learning with a sequence of meta-train\nsteps. The standard multi-task learning objective adopted in BERT is a special\ncase of our learning algorithm where the depth of meta-train is zero. We study\nthe problem in two settings: unsupervised pre-training and supervised\npre-training with different pre-training objects to verify the generality of\nour approach.Experimental results show that our algorithm brings improvements\nand learns better initializations for a variety of downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05744v1",
            "date": "2020-04-13 02:11:19+00:00",
            "title": "Integrated Eojeol Embedding for Erroneous Sentence Classification in Korean Chatbots",
            "authors": [
                "DongHyun Choi",
                "IlNam Park",
                "Myeong Cheol Shin",
                "EungGyun Kim",
                "Dong Ryeol Shin"
            ],
            "abstract": "This paper attempts to analyze the Korean sentence classification system for\na chatbot. Sentence classification is the task of classifying an input sentence\nbased on predefined categories. However, spelling or space error contained in\nthe input sentence causes problems in morphological analysis and tokenization.\nThis paper proposes a novel approach of Integrated Eojeol (Korean syntactic\nword separated by space) Embedding to reduce the effect that poorly analyzed\nmorphemes may make on sentence classification. It also proposes two noise\ninsertion methods that further improve classification performance. Our\nevaluation results indicate that the proposed system classifies erroneous\nsentences more accurately than the baseline system by 17%p.0",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05808v1",
            "date": "2020-04-13 07:50:04+00:00",
            "title": "Unified Multi-Criteria Chinese Word Segmentation with BERT",
            "authors": [
                "Zhen Ke",
                "Liang Shi",
                "Erli Meng",
                "Bin Wang",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Multi-Criteria Chinese Word Segmentation (MCCWS) aims at finding word\nboundaries in a Chinese sentence composed of continuous characters while\nmultiple segmentation criteria exist. The unified framework has been widely\nused in MCCWS and shows its effectiveness. Besides, the pre-trained BERT\nlanguage model has been also introduced into the MCCWS task in a multi-task\nlearning framework. In this paper, we combine the superiority of the unified\nframework and pretrained language model, and propose a unified MCCWS model\nbased on BERT. Moreover, we augment the unified BERT-based MCCWS model with the\nbigram features and an auxiliary criterion classification task. Experiments on\neight datasets with diverse criteria demonstrate that our methods could achieve\nnew state-of-the-art results for MCCWS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05809v1",
            "date": "2020-04-13 07:53:57+00:00",
            "title": "Neural Machine Translation: Challenges, Progress and Future",
            "authors": [
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Machine translation (MT) is a technique that leverages computers to translate\nhuman languages automatically. Nowadays, neural machine translation (NMT) which\nmodels direct mapping between source and target languages with deep neural\nnetworks has achieved a big breakthrough in translation performance and become\nthe de facto paradigm of MT. This article makes a review of NMT framework,\ndiscusses the challenges in NMT, introduces some exciting recent progresses and\nfinally looks forward to some potential future research trends. In addition, we\nmaintain the state-of-the-art methods for various NMT tasks at the website\nhttps://github.com/ZNLP/SOTA-MT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05816v2",
            "date": "2020-04-13 08:16:16+00:00",
            "title": "Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness",
            "authors": [
                "Hyunwoo Kim",
                "Byeongchang Kim",
                "Gunhee Kim"
            ],
            "abstract": "We explore the task of improving persona consistency of dialogue agents.\nRecent models tackling consistency often train with additional Natural Language\nInference (NLI) labels or attach trained extra modules to the generative agent\nfor maintaining consistency. However, such additional labels and training can\nbe demanding. Also, we find even the best-performing persona-based agents are\ninsensitive to contradictory words. Inspired by social cognition and\npragmatics, we endow existing dialogue agents with public self-consciousness on\nthe fly through an imaginary listener. Our approach, based on the Rational\nSpeech Acts framework (Frank and Goodman, 2012), can enforce dialogue agents to\nrefrain from uttering contradiction. We further extend the framework by\nlearning the distractor selection, which has been usually done manually or\nrandomly. Results on Dialogue NLI (Welleck et al., 2019) and PersonaChat (Zhang\net al., 2018) dataset show that our approach reduces contradiction and improves\nconsistency of existing dialogue models. Moreover, we show that it can be\ngeneralized to improve context-consistency beyond persona in dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05887v2",
            "date": "2020-04-13 12:11:36+00:00",
            "title": "Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples",
            "authors": [
                "Maximilian Mozes",
                "Pontus Stenetorp",
                "Bennett Kleinberg",
                "Lewis D. Griffin"
            ],
            "abstract": "Recent efforts have shown that neural text processing models are vulnerable\nto adversarial examples, but the nature of these examples is poorly understood.\nIn this work, we show that adversarial attacks against CNN, LSTM and\nTransformer-based classification models perform word substitutions that are\nidentifiable through frequency differences between replaced words and their\ncorresponding substitutions. Based on these findings, we propose\nfrequency-guided word substitutions (FGWS), a simple algorithm exploiting the\nfrequency properties of adversarial word substitutions for the detection of\nadversarial examples. FGWS achieves strong performance by accurately detecting\nadversarial examples on the SST-2 and IMDb sentiment datasets, with F1\ndetection scores of up to 91.4% against RoBERTa-based classification models. We\ncompare our approach against a recently proposed perturbation discrimination\nframework and show that we outperform it by up to 13.0% F1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06015v2",
            "date": "2020-04-13 15:43:22+00:00",
            "title": "Toward Subgraph Guided Knowledge Graph Question Generation with Graph Neural Networks",
            "authors": [
                "Yu Chen",
                "Lingfei Wu",
                "Mohammed J. Zaki"
            ],
            "abstract": "Knowledge graph (KG) question generation (QG) aims to generate natural\nlanguage questions from KGs and target answers. Previous works mostly focus on\na simple setting which is to generate questions from a single KG triple. In\nthis work, we focus on a more realistic setting where we aim to generate\nquestions from a KG subgraph and target answers. In addition, most of previous\nworks built on either RNN-based or Transformer-based models to encode a\nlinearized KG sugraph, which totally discards the explicit structure\ninformation of a KG subgraph. To address this issue, we propose to apply a\nbidirectional Graph2Seq model to encode the KG subgraph. Furthermore, we\nenhance our RNN decoder with node-level copying mechanism to allow directly\ncopying node attributes from the KG subgraph to the output question. Both\nautomatic and human evaluation results demonstrate that our model achieves new\nstate-of-the-art scores, outperforming existing methods by a significant margin\non two QG benchmarks. Experimental results also show that our QG model can\nconsistently benefit the Question Answering (QA) task as a mean of data\naugmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06176v2",
            "date": "2020-04-13 20:02:03+00:00",
            "title": "AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization",
            "authors": [
                "Keping Bi",
                "Rahul Jha",
                "W. Bruce Croft",
                "Asli Celikyilmaz"
            ],
            "abstract": "Redundancy-aware extractive summarization systems score the redundancy of the\nsentences to be included in a summary either jointly with their salience\ninformation or separately as an additional sentence scoring step. Previous work\nshows the efficacy of jointly scoring and selecting sentences with neural\nsequence generation models. It is, however, not well-understood if the gain is\ndue to better encoding techniques or better redundancy reduction approaches.\nSimilarly, the contribution of salience versus diversity components on the\ncreated summary is not studied well. Building on the state-of-the-art encoding\nmethods for summarization, we present two adaptive learning models: AREDSUM-SEQ\nthat jointly considers salience and novelty during sentence selection; and a\ntwo-step AREDSUM-CTX that scores salience first, then learns to balance\nsalience and redundancy, enabling the measurement of the impact of each aspect.\nEmpirical results on CNN/DailyMail and NYT50 datasets show that by modeling\ndiversity explicitly in a separate step, AREDSUM-CTX achieves significantly\nbetter performance than AREDSUM-SEQ as well as state-of-the-art extractive\nsummarization baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06188v4",
            "date": "2020-04-13 20:36:57+00:00",
            "title": "PoKi: A Large Dataset of Poems by Children",
            "authors": [
                "Will E. Hipson",
                "Saif M. Mohammad"
            ],
            "abstract": "Child language studies are crucial in improving our understanding of child\nwell-being; especially in determining the factors that impact happiness, the\nsources of anxiety, techniques of emotion regulation, and the mechanisms to\ncope with stress. However, much of this research is stymied by the lack of\navailability of large child-written texts. We present a new corpus of\nchild-written text, PoKi, which includes about 62 thousand poems written by\nchildren from grades 1 to 12. PoKi is especially useful in studying child\nlanguage because it comes with information about the age of the child authors\n(their grade). We analyze the words in PoKi along several emotion dimensions\n(valence, arousal, dominance) and discrete emotions (anger, fear, sadness,\njoy). We use non-parametric regressions to model developmental differences from\nearly childhood to late-adolescence. Results show decreases in valence that are\nespecially pronounced during mid-adolescence, while arousal and dominance\npeaked during adolescence. Gender differences in the developmental trajectory\nof emotions are also observed. Our results support and extend the current state\nof emotion development research.",
            "journal ref": "Proceedings of the 12th Language Resources and Evaluation\n  Conference (LREC-2020), May 2020, Marseille, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06190v3",
            "date": "2020-04-13 20:38:49+00:00",
            "title": "A Divide-and-Conquer Approach to the Summarization of Long Documents",
            "authors": [
                "Alexios Gidiotis",
                "Grigorios Tsoumakas"
            ],
            "abstract": "We present a novel divide-and-conquer method for the neural summarization of\nlong documents. Our method exploits the discourse structure of the document and\nuses sentence similarity to split the problem into an ensemble of smaller\nsummarization problems. In particular, we break a long document and its summary\ninto multiple source-target pairs, which are used for training a model that\nlearns to summarize each part of the document separately. These partial\nsummaries are then combined in order to produce a final complete summary. With\nthis approach we can decompose the problem of long document summarization into\nsmaller and simpler problems, reducing computational complexity and creating\nmore training examples, which at the same time contain less noise in the target\nsummaries compared to the standard approach. We demonstrate that this approach\npaired with different summarization models, including sequence-to-sequence RNNs\nand Transformers, can lead to improved summarization performance. Our best\nmodels achieve results that are on par with the state-of-the-art in two two\npublicly available datasets of academic articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06295v2",
            "date": "2020-04-14 04:16:43+00:00",
            "title": "Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus",
            "authors": [
                "Hao Fei",
                "Meishan Zhang",
                "Donghong Ji"
            ],
            "abstract": "Many efforts of research are devoted to semantic role labeling (SRL) which is\ncrucial for natural language understanding. Supervised approaches have achieved\nimpressing performances when large-scale corpora are available for\nresource-rich languages such as English. While for the low-resource languages\nwith no annotated SRL dataset, it is still challenging to obtain competitive\nperformances. Cross-lingual SRL is one promising way to address the problem,\nwhich has achieved great advances with the help of model transferring and\nannotation projection. In this paper, we propose a novel alternative based on\ncorpus translation, constructing high-quality training datasets for the target\nlanguages from the source gold-standard SRL annotations. Experimental results\non Universal Proposition Bank show that the translation-based method is highly\neffective, and the automatic pseudo datasets can improve the target-language\nSRL performances significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06358v1",
            "date": "2020-04-14 08:43:51+00:00",
            "title": "Speech Translation and the End-to-End Promise: Taking Stock of Where We Are",
            "authors": [
                "Matthias Sperber",
                "Matthias Paulik"
            ],
            "abstract": "Over its three decade history, speech translation has experienced several\nshifts in its primary research themes; moving from loosely coupled cascades of\nspeech recognition and machine translation, to exploring questions of tight\ncoupling, and finally to end-to-end models that have recently attracted much\nattention. This paper provides a brief survey of these developments, along with\na discussion of the main challenges of traditional approaches which stem from\ncommitting to intermediate representations from the speech recognizer, and from\ntraining cascaded models separately towards different objectives.\n  Recent end-to-end modeling techniques promise a principled way of overcoming\nthese issues by allowing joint training of all model components and removing\nthe need for explicit intermediate representations. However, a closer look\nreveals that many end-to-end models fall short of solving these issues, due to\ncompromises made to address data scarcity. This paper provides a unifying\ncategorization and nomenclature that covers both traditional and recent\napproaches and that may help researchers by highlighting both trade-offs and\nopen research questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06438v1",
            "date": "2020-04-14 12:04:28+00:00",
            "title": "Query-Variant Advertisement Text Generation with Association Knowledge",
            "authors": [
                "Siyu Duan",
                "Wei Li",
                "Cai Jing",
                "Yancheng He",
                "Yunfang Wu",
                "Xu Sun"
            ],
            "abstract": "Advertising is an important revenue source for many companies. However, it is\nexpensive to manually create advertisements that meet the needs of various\nqueries for massive items. In this paper, we propose the query-variant\nadvertisement text generation task that aims to generate candidate\nadvertisements for different queries with various needs given the item\nkeywords. In this task, for many different queries there is only one general\npurposed advertisement with no predefined query-advertisement pair, which would\ndiscourage traditional End-to-End models from generating query-variant\nadvertisements for different queries with different needs. To deal with the\nproblem, we propose a query-variant advertisement text generation model that\ntakes keywords and associated external knowledge as input during training and\nadds different queries during inference. Adding external knowledge helps the\nmodel adapted to the information besides the item keywords during training,\nwhich makes the transition between training and inference more smoothing when\nthe query is added during inference. Both automatic and human evaluation show\nthat our model can generate more attractive and query-focused advertisements\nthan the strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06499v2",
            "date": "2020-04-14 13:41:48+00:00",
            "title": "What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models",
            "authors": [
                "Wietse de Vries",
                "Andreas van Cranenburgh",
                "Malvina Nissim"
            ],
            "abstract": "Peeking into the inner workings of BERT has shown that its layers resemble\nthe classical NLP pipeline, with progressively more complex tasks being\nconcentrated in later layers. To investigate to what extent these results also\nhold for a language other than English, we probe a Dutch BERT-based model and\nthe multilingual BERT model for Dutch NLP tasks. In addition, through a deeper\nanalysis of part-of-speech tagging, we show that also within a given task,\ninformation is spread over different parts of the network and the pipeline\nmight not be as neat as it seems. Each layer has different specialisations, so\nthat it may be more useful to combine information from different layers,\ninstead of selecting a single one based on the best overall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06555v1",
            "date": "2020-04-14 14:38:41+00:00",
            "title": "Multi-Ontology Refined Embeddings (MORE): A Hybrid Multi-Ontology and Corpus-based Semantic Representation for Biomedical Concepts",
            "authors": [
                "Steven Jiang",
                "Weiyi Wu",
                "Naofumi Tomita",
                "Craig Ganoe",
                "Saeed Hassanpour"
            ],
            "abstract": "Objective: Currently, a major limitation for natural language processing\n(NLP) analyses in clinical applications is that a concept can be referenced in\nvarious forms across different texts. This paper introduces Multi-Ontology\nRefined Embeddings (MORE), a novel hybrid framework for incorporating domain\nknowledge from multiple ontologies into a distributional semantic model,\nlearned from a corpus of clinical text.\n  Materials and Methods: We use the RadCore and MIMIC-III free-text datasets\nfor the corpus-based component of MORE. For the ontology-based part, we use the\nMedical Subject Headings (MeSH) ontology and three state-of-the-art\nontology-based similarity measures. In our approach, we propose a new learning\nobjective, modified from the Sigmoid cross-entropy objective function.\n  Results and Discussion: We evaluate the quality of the generated word\nembeddings using two established datasets of semantic similarities among\nbiomedical concept pairs. On the first dataset with 29 concept pairs, with the\nsimilarity scores established by physicians and medical coders, MORE's\nsimilarity scores have the highest combined correlation (0.633), which is 5.0%\nhigher than that of the baseline model and 12.4% higher than that of the best\nontology-based similarity measure.On the second dataset with 449 concept pairs,\nMORE's similarity scores have a correlation of 0.481, with the average of four\nmedical residents' similarity ratings, and that outperforms the skip-gram model\nby 8.1% and the best ontology measure by 6.9%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06575v1",
            "date": "2020-04-14 15:02:24+00:00",
            "title": "Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa",
                "Mikel Artetxe"
            ],
            "abstract": "State-of-the-art multilingual machine translation relies on a universal\nencoder-decoder, which requires retraining the entire system to add new\nlanguages. In this paper, we propose an alternative approach that is based on\nlanguage-specific encoder-decoders, and can thus be more easily extended to new\nlanguages by learning their corresponding modules. So as to encourage a common\ninterlingua representation, we simultaneously train the N initial languages.\nOur experiments show that the proposed approach outperforms the universal\nencoder-decoder by 3.28 BLEU points on average, and when adding new languages,\nwithout the need to retrain the rest of the modules. All in all, our work\ncloses the gap between shared and language-specific encoder-decoders, advancing\ntoward modular multilingual machine translation systems that can be flexibly\nextended in lifelong learning settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06577v2",
            "date": "2020-04-08 11:16:53+00:00",
            "title": "Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity",
            "authors": [
                "Hamza Harkous",
                "Isabel Groves",
                "Amir Saffari"
            ],
            "abstract": "End-to-end neural data-to-text (D2T) generation has recently emerged as an\nalternative to pipeline-based architectures. However, it has faced challenges\nin generalizing to new domains and generating semantically consistent text. In\nthis work, we present DataTuner, a neural, end-to-end data-to-text generation\nsystem that makes minimal assumptions about the data representation and the\ntarget domain. We take a two-stage generation-reranking approach, combining a\nfine-tuned language model with a semantic fidelity classifier. Each of our\ncomponents is learnt end-to-end without the need for dataset-specific\nheuristics, entity delexicalization, or post-processing. We show that DataTuner\nachieves state of the art results on the automated metrics across four major\nD2T datasets (LDC2017T10, WebNLG, ViGGO, and Cleaned E2E), with a fluency\nassessed by human annotators nearing or exceeding the human-written reference\ntexts. We further demonstrate that the model-based semantic fidelity scorer in\nDataTuner is a better assessment tool compared to traditional, heuristic-based\nmeasures. Our generated text has a significantly better semantic fidelity than\nthe state of the art across all four datasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06748v4",
            "date": "2020-04-14 18:23:28+00:00",
            "title": "Balancing Training for Multilingual Neural Machine Translation",
            "authors": [
                "Xinyi Wang",
                "Yulia Tsvetkov",
                "Graham Neubig"
            ],
            "abstract": "When training multilingual machine translation (MT) models that can translate\nto/from multiple languages, we are faced with imbalanced training sets: some\nlanguages have much more training data than others. Standard practice is to\nup-sample less resourced languages to increase representation, and the degree\nof up-sampling has a large effect on the overall performance. In this paper, we\npropose a method that instead automatically learns how to weight training data\nthrough a data scorer that is optimized to maximize performance on all test\nlanguages. Experiments on two sets of languages under both one-to-many and\nmany-to-one MT settings show our method not only consistently outperforms\nheuristic baselines in terms of average performance, but also offers flexible\ncontrol over the performance of which languages are optimized.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06753v1",
            "date": "2020-04-14 18:48:57+00:00",
            "title": "A Simple Yet Strong Pipeline for HotpotQA",
            "authors": [
                "Dirk Groeneveld",
                "Tushar Khot",
                "Mausam",
                "Ashish Sabharwal"
            ],
            "abstract": "State-of-the-art models for multi-hop question answering typically augment\nlarge-scale language models like BERT with additional, intuitively useful\ncapabilities such as named entity recognition, graph-based reasoning, and\nquestion decomposition. However, does their strong performance on popular\nmulti-hop datasets really justify this added design complexity? Our results\nsuggest that the answer may be no, because even our simple pipeline based on\nBERT, named Quark, performs surprisingly well. Specifically, on HotpotQA, Quark\noutperforms these models on both question answering and support identification\n(and achieves performance very close to a RoBERTa model). Our pipeline has\nthree steps: 1) use BERT to identify potentially relevant sentences\nindependently of each other; 2) feed the set of selected sentences as context\ninto a standard BERT span prediction model to choose an answer; and 3) use the\nsentence selection model, now with the chosen answer, to produce supporting\nsentences. The strong performance of Quark resurfaces the importance of\ncarefully exploring simple model designs before using popular benchmarks to\njustify the value of complex techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06814v2",
            "date": "2020-04-14 21:41:30+00:00",
            "title": "A Human Evaluation of AMR-to-English Generation Systems",
            "authors": [
                "Emma Manning",
                "Shira Wein",
                "Nathan Schneider"
            ],
            "abstract": "Most current state-of-the art systems for generating English text from\nAbstract Meaning Representation (AMR) have been evaluated only using automated\nmetrics, such as BLEU, which are known to be problematic for natural language\ngeneration. In this work, we present the results of a new human evaluation\nwhich collects fluency and adequacy scores, as well as categorization of error\ntypes, for several recent AMR generation systems. We discuss the relative\nquality of these systems and how our results compare to those of automatic\nmetrics, finding that while the metrics are mostly successful in ranking\nsystems overall, collecting human judgments allows for more nuanced\ncomparisons. We also analyze common errors made by these systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06870v2",
            "date": "2020-04-15 03:57:45+00:00",
            "title": "Coreferential Reasoning Learning for Language Representation",
            "authors": [
                "Deming Ye",
                "Yankai Lin",
                "Jiaju Du",
                "Zhenghao Liu",
                "Peng Li",
                "Maosong Sun",
                "Zhiyuan Liu"
            ],
            "abstract": "Language representation models such as BERT could effectively capture\ncontextual semantic information from plain text, and have been proved to\nachieve promising results in lots of downstream NLP tasks with appropriate\nfine-tuning. However, most existing language representation models cannot\nexplicitly handle coreference, which is essential to the coherent understanding\nof the whole discourse. To address this issue, we present CorefBERT, a novel\nlanguage representation model that can capture the coreferential relations in\ncontext. The experimental results show that, compared with existing baseline\nmodels, CorefBERT can achieve significant improvements consistently on various\ndownstream NLP tasks that require coreferential reasoning, while maintaining\ncomparable performance to previous models on other common NLP tasks. The source\ncode and experiment details of this paper can be obtained from\nhttps://github.com/thunlp/CorefBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06871v3",
            "date": "2020-04-15 04:09:05+00:00",
            "title": "TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue",
            "authors": [
                "Chien-Sheng Wu",
                "Steven Hoi",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "The underlying difference of linguistic patterns between general text and\ntask-oriented dialogue makes existing pre-trained language models less useful\nin practice. In this work, we unify nine human-human and multi-turn\ntask-oriented dialogue datasets for language modeling. To better model dialogue\nbehavior during pre-training, we incorporate user and system tokens into the\nmasked language modeling. We propose a contrastive objective function to\nsimulate the response selection task. Our pre-trained task-oriented dialogue\nBERT (TOD-BERT) outperforms strong baselines like BERT on four downstream\ntask-oriented dialogue applications, including intention recognition, dialogue\nstate tracking, dialogue act prediction, and response selection. We also show\nthat TOD-BERT has a stronger few-shot ability that can mitigate the data\nscarcity problem for task-oriented dialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07000v1",
            "date": "2020-04-15 11:00:07+00:00",
            "title": "Exploring Probabilistic Soft Logic as a framework for integrating top-down and bottom-up processing of language in a task context",
            "authors": [
                "Johannes Dellert"
            ],
            "abstract": "This technical report describes a new prototype architecture designed to\nintegrate top-down and bottom-up analysis of non-standard linguistic input,\nwhere a semantic model of the context of an utterance is used to guide the\nanalysis of the non-standard surface forms, including their automated\nnormalization in context. While the architecture is generally applicable, as a\nconcrete use case of the architecture we target the generation of\nsemantically-informed target hypotheses for answers written by German learners\nin response to reading comprehension questions, where the reading context and\npossible target answers are given.\n  The architecture integrates existing NLP components to produce candidate\nanalyses on eight levels of linguistic modeling, all of which are broken down\ninto atomic statements and connected into a large graphical model using\nProbabilistic Soft Logic (PSL) as a framework. Maximum a posteriori inference\non the resulting graphical model then assigns a belief distribution to\ncandidate target hypotheses. The current version of the architecture builds on\nUniversal Dependencies (UD) as its representation formalism on the form level\nand on Abstract Meaning Representations (AMRs) to represent semantic analyses\nof learner answers and the context information provided by the target answers.\nThese general choices will make it comparatively straightforward to apply the\narchitecture to other tasks and other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07159v2",
            "date": "2020-04-14 06:25:36+00:00",
            "title": "PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation",
            "authors": [
                "Bin Bi",
                "Chenliang Li",
                "Chen Wu",
                "Ming Yan",
                "Wei Wang",
                "Songfang Huang",
                "Fei Huang",
                "Luo Si"
            ],
            "abstract": "Self-supervised pre-training, such as BERT, MASS and BART, has emerged as a\npowerful technique for natural language understanding and generation. Existing\npre-training techniques employ autoencoding and/or autoregressive objectives to\ntrain Transformer-based models by recovering original word tokens from\ncorrupted text with some masked tokens. The training goals of existing\ntechniques are often inconsistent with the goals of many language generation\ntasks, such as generative question answering and conversational response\ngeneration, for producing new text given context.\n  This work presents PALM with a novel scheme that jointly pre-trains an\nautoencoding and autoregressive language model on a large unlabeled corpus,\nspecifically designed for generating new text conditioned on context. The new\nscheme alleviates the mismatch introduced by the existing denoising scheme\nbetween pre-training and fine-tuning where generation is more than\nreconstructing original text. An extensive set of experiments show that PALM\nachieves new state-of-the-art results on a variety of language generation\nbenchmarks covering generative question answering (Rank 1 on the official MARCO\nleaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword,\nquestion generation on SQuAD, and conversational response generation on Cornell\nMovie Dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07180v4",
            "date": "2020-04-15 16:05:51+00:00",
            "title": "SPECTER: Document-level Representation Learning using Citation-informed Transformers",
            "authors": [
                "Arman Cohan",
                "Sergey Feldman",
                "Iz Beltagy",
                "Doug Downey",
                "Daniel S. Weld"
            ],
            "abstract": "Representation learning is a critical ingredient for natural language\nprocessing systems. Recent Transformer language models like BERT learn powerful\ntextual representations, but these models are targeted towards token- and\nsentence-level training objectives and do not leverage information on\ninter-document relatedness, which limits their document-level representation\npower. For applications on scientific documents, such as classification and\nrecommendation, the embeddings power strong performance on end tasks. We\npropose SPECTER, a new method to generate document-level embedding of\nscientific documents based on pretraining a Transformer language model on a\npowerful signal of document-level relatedness: the citation graph. Unlike\nexisting pretrained language models, SPECTER can be easily applied to\ndownstream applications without task-specific fine-tuning. Additionally, to\nencourage further research on document-level models, we introduce SciDocs, a\nnew evaluation benchmark consisting of seven document-level tasks ranging from\ncitation prediction, to document classification and recommendation. We show\nthat SPECTER outperforms a variety of competitive baselines on the benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07265v1",
            "date": "2020-04-15 18:01:36+00:00",
            "title": "Learning Structured Embeddings of Knowledge Graphs with Adversarial Learning Framework",
            "authors": [
                "Jiehang Zeng",
                "Lu Liu",
                "Xiaoqing Zheng"
            ],
            "abstract": "Many large-scale knowledge graphs are now available and ready to provide\nsemantically structured information that is regarded as an important resource\nfor question answering and decision support tasks. However, they are built on\nrigid symbolic frameworks which makes them hard to be used in other intelligent\nsystems. We present a learning method using generative adversarial architecture\ndesigned to embed the entities and relations of the knowledge graphs into a\ncontinuous vector space. A generative network (GN) takes two elements of a\n(subject, predicate, object) triple as input and generates the vector\nrepresentation of the missing element. A discriminative network (DN) scores a\ntriple to distinguish a positive triple from those generated by GN. The\ntraining goal for GN is to deceive DN to make wrong classification. When\narriving at a convergence, GN recovers the training data and can be used for\nknowledge graph completion, while DN is trained to be a good triple classifier.\nUnlike few previous studies based on generative adversarial architectures, our\nGN is able to generate unseen instances while they just use GN to better choose\nnegative samples (already existed) for DN. Experiments demonstrate our method\ncan improve classical relational learning models (e.g.TransE) with a\nsignificant margin on both the link prediction and triple classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07324v1",
            "date": "2020-04-15 20:21:19+00:00",
            "title": "Building a Multi-domain Neural Machine Translation Model using Knowledge Distillation",
            "authors": [
                "Idriss Mghabbar",
                "Pirashanth Ratnamogan"
            ],
            "abstract": "Lack of specialized data makes building a multi-domain neural machine\ntranslation tool challenging. Although emerging literature dealing with low\nresource languages starts to show promising results, most state-of-the-art\nmodels used millions of sentences. Today, the majority of multi-domain\nadaptation techniques are based on complex and sophisticated architectures that\nare not adapted for real-world applications. So far, no scalable method is\nperforming better than the simple yet effective mixed-finetuning, i.e\nfinetuning a generic model with a mix of all specialized data and generic data.\nIn this paper, we propose a new training pipeline where knowledge distillation\nand multiple specialized teachers allow us to efficiently finetune a model\nwithout adding new costs at inference time. Our experiments demonstrated that\nour training pipeline allows improving the performance of multi-domain\ntranslation over finetuning in configurations with 2, 3, and 4 domains by up to\n2 points in BLEU.",
            "journal ref": "24th European Conference on Artificial Intelligence (ECAI), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07426v2",
            "date": "2020-04-16 02:50:51+00:00",
            "title": "Neural Data-to-Text Generation with Dynamic Content Planning",
            "authors": [
                "Kai Chen",
                "Fayuan Li",
                "Baotian Hu",
                "Weihua Peng",
                "Qingcai Chen",
                "Hong Yu"
            ],
            "abstract": "Neural data-to-text generation models have achieved significant advancement\nin recent years. However, these models have two shortcomings: the generated\ntexts tend to miss some vital information, and they often generate descriptions\nthat are not consistent with the structured input data. To alleviate these\nproblems, we propose a Neural data-to-text generation model with Dynamic\ncontent Planning, named NDP for abbreviation. The NDP can utilize the\npreviously generated text to dynamically select the appropriate entry from the\ngiven structured data. We further design a reconstruction mechanism with a\nnovel objective function that can reconstruct the whole entry of the used data\nsequentially from the hidden states of the decoder, which aids the accuracy of\nthe generated text. Empirical results show that the NDP achieves superior\nperformance over the state-of-the-art on ROTOWIRE dataset, in terms of relation\ngeneration (RG), content selection (CS), content ordering (CO) and BLEU\nmetrics. The human evaluation result shows that the texts generated by the\nproposed NDP are better than the corresponding ones generated by NCP in most of\ntime. And using the proposed reconstruction mechanism, the fidelity of the\ngenerated text can be further improved significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07642v1",
            "date": "2020-04-16 13:18:55+00:00",
            "title": "Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers",
            "authors": [
                "Robert Litschko",
                "Ivan Vuli\u0107",
                "\u017deljko Agi\u0107",
                "Goran Glava\u0161"
            ],
            "abstract": "Current methods of cross-lingual parser transfer focus on predicting the best\nparser for a low-resource target language globally, that is, \"at treebank\nlevel\". In this work, we propose and argue for a novel cross-lingual transfer\nparadigm: instance-level parser selection (ILPS), and present a\nproof-of-concept study focused on instance-level selection in the framework of\ndelexicalized parser transfer. We start from an empirical observation that\ndifferent source parsers are the best choice for different Universal POS\nsequences in the target language. We then propose to predict the best parser at\nthe instance level. To this end, we train a supervised regression model, based\non the Transformer architecture, to predict parser accuracies for individual\nPOS-sequences. We compare ILPS against two strong single-best parser selection\nbaselines (SBPS): (1) a model that compares POS n-gram distributions between\nthe source and target languages (KL) and (2) a model that selects the source\nbased on the similarity between manually created language vectors encoding\nsyntactic properties of languages (L2V). The results from our extensive\nevaluation, coupling 42 source parsers and 20 diverse low-resource test\nlanguages, show that ILPS outperforms KL and L2V on 13/20 and 14/20 test\nlanguages, respectively. Further, we show that by predicting the best parser\n\"at the treebank level\" (SBPS), using the aggregation of predictions from our\ninstance-level model, we outperform the same baselines on 17/20 and 16/20 test\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07672v4",
            "date": "2020-04-16 14:10:24+00:00",
            "title": "Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation",
            "authors": [
                "Haoyu Song",
                "Yan Wang",
                "Wei-Nan Zhang",
                "Xiaojiang Liu",
                "Ting Liu"
            ],
            "abstract": "Maintaining a consistent personality in conversations is quite natural for\nhuman beings, but is still a non-trivial task for machines. The persona-based\ndialogue generation task is thus introduced to tackle the\npersonality-inconsistent problem by incorporating explicit persona text into\ndialogue generation models. Despite the success of existing persona-based\nmodels on generating human-like responses, their one-stage decoding framework\ncan hardly avoid the generation of inconsistent persona words. In this work, we\nintroduce a three-stage framework that employs a generate-delete-rewrite\nmechanism to delete inconsistent words from a generated response prototype and\nfurther rewrite it to a personality-consistent one. We carry out evaluations by\nboth human and automatic metrics. Experiments on the Persona-Chat dataset show\nthat our approach achieves good performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07737v2",
            "date": "2020-04-16 16:21:17+00:00",
            "title": "Cross-lingual Contextualized Topic Models with Zero-shot Learning",
            "authors": [
                "Federico Bianchi",
                "Silvia Terragni",
                "Dirk Hovy",
                "Debora Nozza",
                "Elisabetta Fersini"
            ],
            "abstract": "Many data sets (e.g., reviews, forums, news, etc.) exist parallelly in\nmultiple languages. They all cover the same content, but the linguistic\ndifferences make it impossible to use traditional, bag-of-word-based topic\nmodels. Models have to be either single-language or suffer from a huge, but\nextremely sparse vocabulary. Both issues can be addressed by transfer learning.\nIn this paper, we introduce a zero-shot cross-lingual topic model. Our model\nlearns topics on one language (here, English), and predicts them for unseen\ndocuments in different languages (here, Italian, French, German, and\nPortuguese). We evaluate the quality of the topic predictions for the same\ndocument in different languages. Our results show that the transferred topics\nare coherent and stable across languages, which suggests exciting future\nresearch directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07776v1",
            "date": "2020-04-16 17:11:02+00:00",
            "title": "Kvistur 2.0: a BiLSTM Compound Splitter for Icelandic",
            "authors": [
                "J\u00f3n Fri\u00f0rik Da\u00f0ason",
                "David Erik Mollberg",
                "Hrafn Loftsson",
                "Krist\u00edn Bjarnad\u00f3ttir"
            ],
            "abstract": "In this paper, we present a character-based BiLSTM model for splitting\nIcelandic compound words, and show how varying amounts of training data affects\nthe performance of the model. Compounding is highly productive in Icelandic,\nand new compounds are constantly being created. This results in a large number\nof out-of-vocabulary (OOV) words, negatively impacting the performance of many\nNLP tools. Our model is trained on a dataset of 2.9 million unique word forms\nand their constituent structures from the Database of Icelandic Morphology. The\nmodel learns how to split compound words into two parts and can be used to\nderive the constituent structure of any word form. Knowing the constituent\nstructure of a word form makes it possible to generate the optimal split for a\ngiven task, e.g., a full split for subword tokenization, or, in the case of\npart-of-speech tagging, splitting an OOV word until the largest known\nmorphological head is found. The model outperforms other previously published\nmethods when evaluated on a corpus of manually split word forms. This method\nhas been integrated into Kvistur, an Icelandic compound word analyzer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07898v3",
            "date": "2020-04-16 19:42:43+00:00",
            "title": "Bridging Anaphora Resolution as Question Answering",
            "authors": [
                "Yufang Hou"
            ],
            "abstract": "Most previous studies on bridging anaphora resolution (Poesio et al., 2004;\nHou et al., 2013b; Hou, 2018a) use the pairwise model to tackle the problem and\nassume that the gold mention information is given. In this paper, we cast\nbridging anaphora resolution as question answering based on context. This\nallows us to find the antecedent for a given anaphor without knowing any gold\nmention information (except the anaphor itself). We present a question\nanswering framework (BARQA) for this task, which leverages the power of\ntransfer learning. Furthermore, we propose a novel method to generate a large\namount of \"quasi-bridging\" training data. We show that our model pre-trained on\nthis dataset and fine-tuned on a small amount of in-domain dataset achieves new\nstate-of-the-art results for bridging anaphora resolution on two bridging\ncorpora (ISNotes (Markert et al., 2012) and BASHI (Roesiger, 2018)).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08031v1",
            "date": "2020-04-17 02:02:18+00:00",
            "title": "AlloVera: A Multilingual Allophone Database",
            "authors": [
                "David R. Mortensen",
                "Xinjian Li",
                "Patrick Littell",
                "Alexis Michaud",
                "Shruti Rijhwani",
                "Antonios Anastasopoulos",
                "Alan W. Black",
                "Florian Metze",
                "Graham Neubig"
            ],
            "abstract": "We introduce a new resource, AlloVera, which provides mappings from 218\nallophones to phonemes for 14 languages. Phonemes are contrastive phonological\nunits, and allophones are their various concrete realizations, which are\npredictable from phonological context. While phonemic representations are\nlanguage specific, phonetic representations (stated in terms of (allo)phones)\nare much closer to a universal (language-independent) transcription. AlloVera\nallows the training of speech recognition models that output phonetic\ntranscriptions in the International Phonetic Alphabet (IPA), regardless of the\ninput language. We show that a \"universal\" allophone model, Allosaurus, built\nwith AlloVera, outperforms \"universal\" phonemic models and language-specific\nmodels on a speech-transcription task. We explore the implications of this\ntechnology (and related technologies) for the documentation of endangered and\nminority languages. We further explore other applications for which AlloVera\nwill be suitable as it grows, including phonological typology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08053v2",
            "date": "2020-04-17 03:40:13+00:00",
            "title": "Enriching the Transformer with Linguistic Factors for Low-Resource Machine Translation",
            "authors": [
                "Jordi Armengol-Estap\u00e9",
                "Marta R. Costa-juss\u00e0",
                "Carlos Escolano"
            ],
            "abstract": "Introducing factors, that is to say, word features such as linguistic\ninformation referring to the source tokens, is known to improve the results of\nneural machine translation systems in certain settings, typically in recurrent\narchitectures. This study proposes enhancing the current state-of-the-art\nneural machine translation architecture, the Transformer, so that it allows to\nintroduce external knowledge. In particular, our proposed modification, the\nFactored Transformer, uses linguistic factors that insert additional knowledge\ninto the machine translation system. Apart from using different kinds of\nfeatures, we study the effect of different architectural configurations.\nSpecifically, we analyze the performance of combining words and features at the\nembedding level or at the encoder level, and we experiment with two different\ncombination strategies. With the best-found configuration, we show improvements\nof 0.8 BLEU over the baseline Transformer in the IWSLT German-to-English task.\nMoreover, we experiment with the more challenging FLoRes English-to-Nepali\nbenchmark, which includes both extremely low-resourced and very distant\nlanguages, and obtain an improvement of 1.2 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08056v1",
            "date": "2020-04-17 03:51:57+00:00",
            "title": "Dialogue-Based Relation Extraction",
            "authors": [
                "Dian Yu",
                "Kai Sun",
                "Claire Cardie",
                "Dong Yu"
            ],
            "abstract": "We present the first human-annotated dialogue-based relation extraction (RE)\ndataset DialogRE, aiming to support the prediction of relation(s) between two\narguments that appear in a dialogue. We further offer DialogRE as a platform\nfor studying cross-sentence RE as most facts span multiple sentences. We argue\nthat speaker-related information plays a critical role in the proposed task,\nbased on an analysis of similarities and differences between dialogue-based and\ntraditional RE tasks. Considering the timeliness of communication in a\ndialogue, we design a new metric to evaluate the performance of RE methods in a\nconversational setting and investigate the performance of several\nrepresentative RE methods on DialogRE. Experimental results demonstrate that a\nspeaker-aware extension on the best-performing model leads to gains in both the\nstandard and conversational evaluation settings. DialogRE is available at\nhttps://dataset.org/dialogre/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08076v1",
            "date": "2020-04-17 06:47:15+00:00",
            "title": "Neural Approaches for Data Driven Dependency Parsing in Sanskrit",
            "authors": [
                "Amrith Krishna",
                "Ashim Gupta",
                "Deepak Garasangi",
                "Jivnesh Sandhan",
                "Pavankumar Satuluri",
                "Pawan Goyal"
            ],
            "abstract": "Data-driven approaches for dependency parsing have been of great interest in\nNatural Language Processing for the past couple of decades. However, Sanskrit\nstill lacks a robust purely data-driven dependency parser, probably with an\nexception to Krishna (2019). This can primarily be attributed to the lack of\navailability of task-specific labelled data and the morphologically rich nature\nof the language. In this work, we evaluate four different data-driven machine\nlearning models, originally proposed for different languages, and compare their\nperformances on Sanskrit data. We experiment with 2 graph based and 2\ntransition based parsers. We compare the performance of each of the models in a\nlow-resource setting, with 1,500 sentences for training. Further, since our\nfocus is on the learning power of each of the models, we do not incorporate any\nSanskrit specific features explicitly into the models, and rather use the\ndefault settings in each of the paper for obtaining the feature functions. In\nthis work, we analyse the performance of the parsers using both an in-domain\nand an out-of-domain test dataset. We also investigate the impact of word\nordering in which the sentences are provided as input to these systems, by\nparsing verses and their corresponding prose order (anvaya) sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08097v1",
            "date": "2020-04-17 07:43:38+00:00",
            "title": "Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning",
            "authors": [
                "Joongbo Shin",
                "Yoonhyung Lee",
                "Seunghyun Yoon",
                "Kyomin Jung"
            ],
            "abstract": "Even though BERT achieves successful performance improvements in various\nsupervised learning tasks, applying BERT for unsupervised tasks still holds a\nlimitation that it requires repetitive inference for computing contextual\nlanguage representations. To resolve the limitation, we propose a novel deep\nbidirectional language model called Transformer-based Text Autoencoder (T-TA).\nThe T-TA computes contextual language representations without repetition and\nhas benefits of the deep bidirectional architecture like BERT. In run-time\nexperiments on CPU environments, the proposed T-TA performs over six times\nfaster than the BERT-based model in the reranking task and twelve times faster\nin the semantic similarity task. Furthermore, the T-TA shows competitive or\neven better accuracies than those of BERT on the above tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08134v1",
            "date": "2020-04-17 09:17:40+00:00",
            "title": "Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction",
            "authors": [
                "Christoph Alt",
                "Aleksandra Gabryszak",
                "Leonhard Hennig"
            ],
            "abstract": "Despite the recent progress, little is known about the features captured by\nstate-of-the-art neural relation extraction (RE) models. Common methods encode\nthe source sentence, conditioned on the entity mentions, before classifying the\nrelation. However, the complexity of the task makes it difficult to understand\nhow encoder architecture and supporting linguistic knowledge affect the\nfeatures learned by the encoder. We introduce 14 probing tasks targeting\nlinguistic properties relevant to RE, and we use them to study representations\nlearned by more than 40 different encoder architecture and linguistic feature\ncombinations trained on two datasets, TACRED and SemEval 2010 Task 8. We find\nthat the bias induced by the architecture and the inclusion of linguistic\nfeatures are clearly expressed in the probing task performance. For example,\nadding contextualized word representations greatly increases performance on\nprobing tasks with a focus on named entity and part-of-speech information, and\nyields better results in RE. In contrast, entity masking improves RE, but\nconsiderably lowers performance on entity type related probing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08166v2",
            "date": "2020-04-17 10:55:07+00:00",
            "title": "Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness",
            "authors": [
                "Yavuz Selim Kartal",
                "Busra Guvenen",
                "Mucahid Kutlu"
            ],
            "abstract": "The massive amount of misinformation spreading on the Internet on a daily\nbasis has enormous negative impacts on societies. Therefore, we need automated\nsystems helping fact-checkers in the combat against misinformation. In this\npaper, we propose a model prioritizing the claims based on their\ncheck-worthiness. We use BERT model with additional features including\ndomain-specific controversial topics, word embeddings, and others. In our\nexperiments, we show that our proposed model outperforms all state-of-the-art\nmodels in both test collections of CLEF Check That! Lab in 2018 and 2019. We\nalso conduct a qualitative analysis to shed light-detecting check-worthy\nclaims. We suggest requesting rationales behind judgments are needed to\nunderstand subjective nature of the task and problematic labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08361v2",
            "date": "2020-04-17 17:36:20+00:00",
            "title": "Unsupervised Discovery of Implicit Gender Bias",
            "authors": [
                "Anjalie Field",
                "Yulia Tsvetkov"
            ],
            "abstract": "Despite their prevalence in society, social biases are difficult to identify,\nprimarily because human judgements in this domain can be unreliable. We take an\nunsupervised approach to identifying gender bias against women at a comment\nlevel and present a model that can surface text likely to contain bias. Our\nmain challenge is forcing the model to focus on signs of implicit bias, rather\nthan other artifacts in the data. Thus, our methodology involves reducing the\ninfluence of confounds through propensity matching and adversarial learning.\nOur analysis shows how biased comments directed towards female politicians\ncontain mixed criticisms, while comments directed towards other female public\nfigures focus on appearance and sexualization. Ultimately, our work offers a\nway to capture subtle biases in various domains without relying on subjective\nhuman judgements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08449v1",
            "date": "2020-04-17 20:51:40+00:00",
            "title": "Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills",
            "authors": [
                "Eric Michael Smith",
                "Mary Williamson",
                "Kurt Shuster",
                "Jason Weston",
                "Y-Lan Boureau"
            ],
            "abstract": "Being engaging, knowledgeable, and empathetic are all desirable general\nqualities in a conversational agent. Previous work has introduced tasks and\ndatasets that aim to help agents to learn those qualities in isolation and\ngauge how well they can express them. But rather than being specialized in one\nsingle quality, a good open-domain conversational agent should be able to\nseamlessly blend them all into one cohesive conversational flow. In this work,\nwe investigate several ways to combine models trained towards isolated\ncapabilities, ranging from simple model aggregation schemes that require\nminimal additional training, to various forms of multi-task training that\nencompass several skills at all training stages. We further propose a new\ndataset, BlendedSkillTalk, to analyze how these capabilities would mesh\ntogether in a natural conversation, and compare the performance of different\narchitectures and training schemes. Our experiments show that multi-tasking\nover several tasks that focus on particular capabilities results in better\nblended conversation performance compared to models trained on a single skill,\nand that both unified or two-stage approaches perform well if they are\nconstructed to avoid unwanted bias in skill selection or are fine-tuned on our\nnew task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08511v1",
            "date": "2020-04-18 02:58:00+00:00",
            "title": "Exclusive Hierarchical Decoding for Deep Keyphrase Generation",
            "authors": [
                "Wang Chen",
                "Hou Pong Chan",
                "Piji Li",
                "Irwin King"
            ],
            "abstract": "Keyphrase generation (KG) aims to summarize the main ideas of a document into\na set of keyphrases. A new setting is recently introduced into this problem, in\nwhich, given a document, the model needs to predict a set of keyphrases and\nsimultaneously determine the appropriate number of keyphrases to produce.\nPrevious work in this setting employs a sequential decoding process to generate\nkeyphrases. However, such a decoding method ignores the intrinsic hierarchical\ncompositionality existing in the keyphrase set of a document. Moreover,\nprevious work tends to generate duplicated keyphrases, which wastes time and\ncomputing resources. To overcome these limitations, we propose an exclusive\nhierarchical decoding framework that includes a hierarchical decoding process\nand either a soft or a hard exclusion mechanism. The hierarchical decoding\nprocess is to explicitly model the hierarchical compositionality of a keyphrase\nset. Both the soft and the hard exclusion mechanisms keep track of\npreviously-predicted keyphrases within a window size to enhance the diversity\nof the generated keyphrases. Extensive experiments on multiple KG benchmark\ndatasets demonstrate the effectiveness of our method to generate less\nduplicated and more accurate keyphrases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08673v1",
            "date": "2020-04-18 17:54:55+00:00",
            "title": "A Hybrid Approach for Aspect-Based Sentiment Analysis Using Deep Contextual Word Embeddings and Hierarchical Attention",
            "authors": [
                "Maria Mihaela Trusca",
                "Daan Wassenberg",
                "Flavius Frasincar",
                "Rommert Dekker"
            ],
            "abstract": "The Web has become the main platform where people express their opinions\nabout entities of interest and their associated aspects. Aspect-Based Sentiment\nAnalysis (ABSA) aims to automatically compute the sentiment towards these\naspects from opinionated text. In this paper we extend the state-of-the-art\nHybrid Approach for Aspect-Based Sentiment Analysis (HAABSA) method in two\ndirections. First we replace the non-contextual word embeddings with deep\ncontextual word embeddings in order to better cope with the word semantics in a\ngiven text. Second, we use hierarchical attention by adding an extra attention\nlayer to the HAABSA high-level representations in order to increase the method\nflexibility in modeling the input data. Using two standard datasets (SemEval\n2015 and SemEval 2016) we show that the proposed extensions improve the\naccuracy of the built model for ABSA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08728v4",
            "date": "2020-04-18 23:10:36+00:00",
            "title": "SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings",
            "authors": [
                "Masoud Jalili Sabet",
                "Philipp Dufter",
                "Fran\u00e7ois Yvon",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Word alignments are useful for tasks like statistical and neural machine\ntranslation (NMT) and cross-lingual annotation projection. Statistical word\naligners perform well, as do methods that extract alignments jointly with\ntranslations in NMT. However, most approaches require parallel training data,\nand quality decreases as less training data is available. We propose word\nalignment methods that require no parallel data. The key idea is to leverage\nmultilingual word embeddings, both static and contextualized, for word\nalignment. Our multilingual embeddings are created from monolingual data only\nwithout relying on any parallel data or dictionaries. We find that alignments\ncreated from embeddings are superior for four and comparable for two language\npairs compared to those produced by traditional statistical aligners, even with\nabundant parallel data; e.g., contextualized embeddings achieve a word\nalignment F1 for English-German that is 5 percentage points higher than\neflomal, a high-quality statistical aligner, trained on 100k parallel\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08789v1",
            "date": "2020-04-19 07:42:22+00:00",
            "title": "BanFakeNews: A Dataset for Detecting Fake News in Bangla",
            "authors": [
                "Md Zobaer Hossain",
                "Md Ashraful Rahman",
                "Md Saiful Islam",
                "Sudipta Kar"
            ],
            "abstract": "Observing the damages that can be done by the rapid propagation of fake news\nin various sectors like politics and finance, automatic identification of fake\nnews using linguistic analysis has drawn the attention of the research\ncommunity. However, such methods are largely being developed for English where\nlow resource languages remain out of the focus. But the risks spawned by fake\nand manipulative news are not confined by languages. In this work, we propose\nan annotated dataset of ~50K news that can be used for building automated fake\nnews detection systems for a low resource language like Bangla. Additionally,\nwe provide an analysis of the dataset and develop a benchmark system with state\nof the art NLP techniques to identify Bangla fake news. To create this system,\nwe explore traditional linguistic features and neural network based methods. We\nexpect this dataset will be a valuable resource for building technologies to\nprevent the spreading of fake news and contribute in research with low resource\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08793v1",
            "date": "2020-04-19 08:13:13+00:00",
            "title": "Pattern Learning for Detecting Defect Reports and Improvement Requests in App Reviews",
            "authors": [
                "Gino V. H. Mangnoesing",
                "Maria Mihaela Trusca",
                "Flavius Frasincar"
            ],
            "abstract": "Online reviews are an important source of feedback for understanding\ncustomers. In this study, we follow novel approaches that target this absence\nof actionable insights by classifying reviews as defect reports and requests\nfor improvement. Unlike traditional classification methods based on expert\nrules, we reduce the manual labour by employing a supervised system that is\ncapable of learning lexico-semantic patterns through genetic programming.\nAdditionally, we experiment with a distantly-supervised SVM that makes use of\nnoisy labels generated by patterns. Using a real-world dataset of app reviews,\nwe show that the automatically learned patterns outperform the manually created\nones, to be generated. Also the distantly-supervised SVM models are not far\nbehind the pattern-based solutions, showing the usefulness of this approach\nwhen the amount of annotated data is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08795v1",
            "date": "2020-04-19 08:27:57+00:00",
            "title": "Extractive Summarization as Text Matching",
            "authors": [
                "Ming Zhong",
                "Pengfei Liu",
                "Yiran Chen",
                "Danqing Wang",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "This paper creates a paradigm shift with regard to the way we build neural\nextractive summarization systems. Instead of following the commonly used\nframework of extracting sentences individually and modeling the relationship\nbetween sentences, we formulate the extractive summarization task as a semantic\ntext matching problem, in which a source document and candidate summaries will\nbe (extracted from the original text) matched in a semantic space. Notably,\nthis paradigm shift to semantic matching framework is well-grounded in our\ncomprehensive analysis of the inherent gap between sentence-level and\nsummary-level extractors based on the property of the dataset.\n  Besides, even instantiating the framework with a simple form of a matching\nmodel, we have driven the state-of-the-art extractive result on CNN/DailyMail\nto a new level (44.41 in ROUGE-1). Experiments on the other five datasets also\nshow the effectiveness of the matching framework. We believe the power of this\nmatching-based summarization framework has not been fully exploited. To\nencourage more instantiations in the future, we have released our codes,\nprocessed dataset, as well as generated summaries in\nhttps://github.com/maszhongming/MatchSum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08798v1",
            "date": "2020-04-19 08:41:12+00:00",
            "title": "Knowledge-graph based Proactive Dialogue Generation with Improved Meta-Learning",
            "authors": [
                "Hongcai Xu",
                "Junpeng Bao",
                "Junqing Wang"
            ],
            "abstract": "Knowledge graph-based dialogue systems can narrow down knowledge candidates\nfor generating informative and diverse responses with the use of prior\ninformation, e.g., triple attributes or graph paths. However, most current\nknowledge graph (KG) cover incomplete domain-specific knowledge. To overcome\nthis drawback, we propose a knowledge graph based proactive dialogue generation\nmodel (KgDg) with three components, improved model-agnostic meta-learning\nalgorithm (MAML), knowledge selection in knowledge triplets embedding, and\nknowledge aware proactive response generator. For knowledge triplets embedding\nand selection, we formulate it as a problem of sentence embedding to better\ncapture semantic information. Our improved MAML algorithm is capable of\nlearning general features from a limited number of knowledge graphs, which can\nalso quickly adapt to dialogue generation with unseen knowledge triplets.\nExtensive experiments are conducted on a knowledge aware dialogue dataset\n(DuConv). The results show that KgDg adapts both fast and well to knowledge\ngraph-based dialogue generation and outperforms state-of-the-art baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08825v1",
            "date": "2020-04-19 11:53:32+00:00",
            "title": "A Chinese Corpus for Fine-grained Entity Typing",
            "authors": [
                "Chin Lee",
                "Hongliang Dai",
                "Yangqiu Song",
                "Xin Li"
            ],
            "abstract": "Fine-grained entity typing is a challenging task with wide applications.\nHowever, most existing datasets for this task are in English. In this paper, we\nintroduce a corpus for Chinese fine-grained entity typing that contains 4,800\nmentions manually labeled through crowdsourcing. Each mention is annotated with\nfree-form entity types. To make our dataset useful in more possible scenarios,\nwe also categorize all the fine-grained types into 10 general types. Finally,\nwe conduct experiments with some neural models whose structures are typical in\nfine-grained entity typing and show how well they perform on our dataset. We\nalso show the possibility of improving Chinese fine-grained entity typing\nthrough cross-lingual transfer learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08833v1",
            "date": "2020-04-19 12:27:49+00:00",
            "title": "Dynamic Knowledge Graph-based Dialogue Generation with Improved Adversarial Meta-Learning",
            "authors": [
                "Hongcai Xu",
                "Junpeng Bao",
                "Gaojie Zhang"
            ],
            "abstract": "Knowledge graph-based dialogue systems are capable of generating more\ninformative responses and can implement sophisticated reasoning mechanisms.\nHowever, these models do not take into account the sparseness and\nincompleteness of knowledge graph (KG)and current dialogue models cannot be\napplied to dynamic KG. This paper proposes a dynamic Knowledge graph-based\ndialogue generation method with improved adversarial Meta-Learning (KDAD). KDAD\nformulates dynamic knowledge triples as a problem of adversarial attack and\nincorporates the objective of quickly adapting to dynamic knowledge-aware\ndialogue generation. We train a knowledge graph-based dialog model with\nimproved ADML using minimal training samples. The model can initialize the\nparameters and adapt to previous unseen knowledge so that training can be\nquickly completed based on only a few knowledge triples. We show that our model\nsignificantly outperforms other baselines. We evaluate and demonstrate that our\nmethod adapts extremely fast and well to dynamic knowledge graph-based dialogue\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08994v2",
            "date": "2020-04-20 00:07:18+00:00",
            "title": "Adversarial Training for Large Neural Language Models",
            "authors": [
                "Xiaodong Liu",
                "Hao Cheng",
                "Pengcheng He",
                "Weizhu Chen",
                "Yu Wang",
                "Hoifung Poon",
                "Jianfeng Gao"
            ],
            "abstract": "Generalization and robustness are both key desiderata for designing machine\nlearning methods. Adversarial training can enhance robustness, but past work\noften finds it hurts generalization. In natural language processing (NLP),\npre-training large neural language models such as BERT have demonstrated\nimpressive gain in generalization for a variety of tasks, with further\nimprovement from adversarial fine-tuning. However, these models are still\nvulnerable to adversarial attacks. In this paper, we show that adversarial\npre-training can improve both generalization and robustness. We propose a\ngeneral algorithm ALUM (Adversarial training for large neural LangUage Models),\nwhich regularizes the training objective by applying perturbations in the\nembedding space that maximizes the adversarial loss. We present the first\ncomprehensive study of adversarial training in all stages, including\npre-training from scratch, continual pre-training on a well-trained model, and\ntask-specific fine-tuning. ALUM obtains substantial gains over BERT on a wide\nrange of NLP tasks, in both regular and adversarial scenarios. Even for models\nthat have been well trained on extremely large text corpora, such as RoBERTa,\nALUM can still produce significant gains from continual pre-training, whereas\nconventional non-adversarial methods can not. ALUM can be further combined with\ntask-specific fine-tuning to attain additional gains. The ALUM code is publicly\navailable at https://github.com/namisan/mt-dnn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09015v1",
            "date": "2020-04-20 01:45:27+00:00",
            "title": "Incorporating External Knowledge through Pre-training for Natural Language to Code Generation",
            "authors": [
                "Frank F. Xu",
                "Zhengbao Jiang",
                "Pengcheng Yin",
                "Bogdan Vasilescu",
                "Graham Neubig"
            ],
            "abstract": "Open-domain code generation aims to generate code in a general-purpose\nprogramming language (such as Python) from natural language (NL) intents.\nMotivated by the intuition that developers usually retrieve resources on the\nweb when writing code, we explore the effectiveness of incorporating two\nvarieties of external knowledge into NL-to-code generation: automatically mined\nNL-code pairs from the online programming QA forum StackOverflow and\nprogramming language API documentation. Our evaluations show that combining the\ntwo sources with data augmentation and retrieval-based data re-sampling\nimproves the current state-of-the-art by up to 2.2% absolute BLEU score on the\ncode generation testbed CoNaLa. The code and resources are available at\nhttps://github.com/neulab/external-knowledge-codegen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09050v1",
            "date": "2020-04-20 04:37:55+00:00",
            "title": "Adaptation of a Lexical Organization for Social Engineering Detection and Response Generation",
            "authors": [
                "Archna Bhatia",
                "Adam Dalton",
                "Brodie Mather",
                "Sashank Santhanam",
                "Samira Shaikh",
                "Alan Zemel",
                "Tomek Strzalkowski",
                "Bonnie J. Dorr"
            ],
            "abstract": "We present a paradigm for extensible lexicon development based on Lexical\nConceptual Structure to support social engineering detection and response\ngeneration. We leverage the central notions of ask (elicitation of behaviors\nsuch as providing access to money) and framing (risk/reward implied by the\nask). We demonstrate improvements in ask/framing detection through refinements\nto our lexical organization and show that response generation qualitatively\nimproves as ask/framing detection performance improves. The paradigm presents a\nsystematic and efficient approach to resource adaptation for improved\ntask-specific performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09095v3",
            "date": "2020-04-20 07:19:22+00:00",
            "title": "The State and Fate of Linguistic Diversity and Inclusion in the NLP World",
            "authors": [
                "Pratik Joshi",
                "Sebastin Santy",
                "Amar Budhiraja",
                "Kalika Bali",
                "Monojit Choudhury"
            ],
            "abstract": "Language technologies contribute to promoting multilingualism and linguistic\ndiversity around the world. However, only a very small number of the over 7000\nlanguages of the world are represented in the rapidly evolving language\ntechnologies and applications. In this paper we look at the relation between\nthe types of languages, resources, and their representation in NLP conferences\nto understand the trajectory that different languages have followed over time.\nOur quantitative investigation underlines the disparity between languages,\nespecially in terms of their resources, and calls into question the \"language\nagnostic\" status of current models and systems. Through this paper, we attempt\nto convince the ACL community to prioritise the resolution of the predicaments\nhighlighted here, so that no language is left behind.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09143v4",
            "date": "2020-04-20 09:08:59+00:00",
            "title": "Variational Inference for Learning Representations of Natural Language Edits",
            "authors": [
                "Edison Marrese-Taylor",
                "Machel Reid",
                "Yutaka Matsuo"
            ],
            "abstract": "Document editing has become a pervasive component of the production of\ninformation, with version control systems enabling edits to be efficiently\nstored and applied. In light of this, the task of learning distributed\nrepresentations of edits has been recently proposed. With this in mind, we\npropose a novel approach that employs variational inference to learn a\ncontinuous latent space of vector representations to capture the underlying\nsemantic information with regard to the document editing process. We achieve\nthis by introducing a latent variable to explicitly model the aforementioned\nfeatures. This latent variable is then combined with a document representation\nto guide the generation of an edited version of this document. Additionally, to\nfacilitate standardized automatic evaluation of edit representations, which has\nheavily relied on direct human input thus far, we also propose a suite of\ndownstream tasks, PEER, specifically designed to measure the quality of edit\nrepresentations in the context of natural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09205v1",
            "date": "2020-04-20 11:13:16+00:00",
            "title": "A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT",
            "authors": [
                "Chi-Liang Liu",
                "Tsung-Yuan Hsu",
                "Yung-Sung Chuang",
                "Hung-Yi Lee"
            ],
            "abstract": "Recently, multilingual BERT works remarkably well on cross-lingual transfer\ntasks, superior to static non-contextualized word embeddings. In this work, we\nprovide an in-depth experimental study to supplement the existing literature of\ncross-lingual ability. We compare the cross-lingual ability of\nnon-contextualized and contextualized representation model with the same data.\nWe found that datasize and context window size are crucial factors to the\ntransferability. We also observe the language-specific information in\nmultilingual BERT. By manipulating the latent representations, we can control\nthe output languages of multilingual BERT, and achieve unsupervised token\ntranslation. We further show that based on the observation, there is a\ncomputationally cheap but effective approach to improve the cross-lingual\nability of multilingual BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09447v1",
            "date": "2020-04-20 17:04:22+00:00",
            "title": "PHINC: A Parallel Hinglish Social Media Code-Mixed Corpus for Machine Translation",
            "authors": [
                "Vivek Srivastava",
                "Mayank Singh"
            ],
            "abstract": "Code-mixing is the phenomenon of using more than one language in a sentence.\nIt is a very frequently observed pattern of communication on social media\nplatforms. Flexibility to use multiple languages in one text message might help\nto communicate efficiently with the target audience. But, it adds to the\nchallenge of processing and understanding natural language to a much larger\nextent. This paper presents a parallel corpus of the 13,738 code-mixed\nEnglish-Hindi sentences and their corresponding translation in English. The\ntranslations of sentences are done manually by the annotators. We are releasing\nthe parallel corpus to facilitate future research opportunities in code-mixed\nmachine translation. The annotated corpus is available at\nhttps://doi.org/10.5281/zenodo.3605597.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09544v2",
            "date": "2020-04-20 18:05:53+00:00",
            "title": "Grounding Conversations with Improvised Dialogues",
            "authors": [
                "Hyundong Cho",
                "Jonathan May"
            ],
            "abstract": "Effective dialogue involves grounding, the process of establishing mutual\nknowledge that is essential for communication between people. Modern dialogue\nsystems are not explicitly trained to build common ground, and therefore\noverlook this important aspect of communication. Improvisational theater\n(improv) intrinsically contains a high proportion of dialogue focused on\nbuilding common ground, and makes use of the yes-and principle, a strong\ngrounding speech act, to establish coherence and an actionable objective\nreality. We collect a corpus of more than 26,000 yes-and turns, transcribing\nthem from improv dialogues and extracting them from larger, but more sparsely\npopulated movie script dialogue corpora, via a bootstrapped classifier. We\nfine-tune chit-chat dialogue systems with our corpus to encourage more\ngrounded, relevant conversation and confirm these findings with human\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09601v1",
            "date": "2020-04-20 19:57:37+00:00",
            "title": "An Automated Pipeline for Character and Relationship Extraction from Readers' Literary Book Reviews on Goodreads.com",
            "authors": [
                "Shadi Shahsavari",
                "Ehsan Ebrahimzadeh",
                "Behnam Shahbazi",
                "Misagh Falahi",
                "Pavan Holur",
                "Roja Bandari",
                "Timothy R. Tangherlini",
                "Vwani Roychowdhury"
            ],
            "abstract": "Reader reviews of literary fiction on social media, especially those in\npersistent, dedicated forums, create and are in turn driven by underlying\nnarrative frameworks. In their comments about a novel, readers generally\ninclude only a subset of characters and their relationships, thus offering a\nlimited perspective on that work. Yet in aggregate, these reviews capture an\nunderlying narrative framework comprised of different actants (people, places,\nthings), their roles, and interactions that we label the \"consensus narrative\nframework\". We represent this framework in the form of an actant-relationship\nstory graph. Extracting this graph is a challenging computational problem,\nwhich we pose as a latent graphical model estimation problem. Posts and reviews\nare viewed as samples of sub graphs/networks of the hidden narrative framework.\nInspired by the qualitative narrative theory of Greimas, we formulate a\ngraphical generative Machine Learning (ML) model where nodes represent actants,\nand multi-edges and self-loops among nodes capture context-specific\nrelationships. We develop a pipeline of interlocking automated methods to\nextract key actants and their relationships, and apply it to thousands of\nreviews and comments posted on Goodreads.com. We manually derive the ground\ntruth narrative framework from SparkNotes, and then use word embedding tools to\ncompare relationships in ground truth networks with our extracted networks. We\nfind that our automated methodology generates highly accurate consensus\nnarrative frameworks: for our four target novels, with approximately 2900\nreviews per novel, we report average coverage/recall of important relationships\nof > 80% and an average edge detection rate of >89\\%. These extracted narrative\nframeworks can generate insight into how people (or classes of people) read and\nhow they recount what they have read to others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09731v1",
            "date": "2020-04-21 03:13:44+00:00",
            "title": "Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness",
            "authors": [
                "Zheng Zhang",
                "Lizi Liao",
                "Xiaoyan Zhu",
                "Tat-Seng Chua",
                "Zitao Liu",
                "Yan Huang",
                "Minlie Huang"
            ],
            "abstract": "Most existing approaches for goal-oriented dialogue policy learning used\nreinforcement learning, which focuses on the target agent policy and simply\ntreat the opposite agent policy as part of the environment. While in real-world\nscenarios, the behavior of an opposite agent often exhibits certain patterns or\nunderlies hidden policies, which can be inferred and utilized by the target\nagent to facilitate its own decision making. This strategy is common in human\nmental simulation by first imaging a specific action and the probable results\nbefore really acting it. We therefore propose an opposite behavior aware\nframework for policy learning in goal-oriented dialogues. We estimate the\nopposite agent's policy from its behavior and use this estimation to improve\nthe target agent by regarding it as part of the target policy. We evaluate our\nmodel on both cooperative and competitive dialogue tasks, showing superior\nperformance over state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09733v2",
            "date": "2020-04-21 03:14:22+00:00",
            "title": "Train No Evil: Selective Masking for Task-Guided Pre-Training",
            "authors": [
                "Yuxian Gu",
                "Zhengyan Zhang",
                "Xiaozhi Wang",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Recently, pre-trained language models mostly follow the\npre-train-then-fine-tuning paradigm and have achieved great performance on\nvarious downstream tasks. However, since the pre-training stage is typically\ntask-agnostic and the fine-tuning stage usually suffers from insufficient\nsupervised data, the models cannot always well capture the domain-specific and\ntask-specific patterns. In this paper, we propose a three-stage framework by\nadding a task-guided pre-training stage with selective masking between general\npre-training and fine-tuning. In this stage, the model is trained by masked\nlanguage modeling on in-domain unsupervised data to learn domain-specific\npatterns and we propose a novel selective masking strategy to learn\ntask-specific patterns. Specifically, we design a method to measure the\nimportance of each token in sequences and selectively mask the important\ntokens. Experimental results on two sentiment analysis tasks show that our\nmethod can achieve comparable or even better performance with less than 50% of\ncomputation cost, which indicates our method is both effective and efficient.\nThe source code of this paper can be obtained from\nhttps://github.com/thunlp/SelectiveMasking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09800v1",
            "date": "2020-04-21 07:58:27+00:00",
            "title": "Keyphrase Generation with Cross-Document Attention",
            "authors": [
                "Shizhe Diao",
                "Yan Song",
                "Tong Zhang"
            ],
            "abstract": "Keyphrase generation aims to produce a set of phrases summarizing the\nessentials of a given document. Conventional methods normally apply an\nencoder-decoder architecture to generate the output keyphrases for an input\ndocument, where they are designed to focus on each current document so they\ninevitably omit crucial corpus-level information carried by other similar\ndocuments, i.e., the cross-document dependency and latent topics. In this\npaper, we propose CDKGen, a Transformer-based keyphrase generator, which\nexpands the Transformer to global attention with cross-document attention\nnetworks to incorporate available documents as references so as to generate\nbetter keyphrases with the guidance of topic information. On top of the\nproposed Transformer + cross-document attention architecture, we also adopt a\ncopy mechanism to enhance our model via selecting appropriate words from\ndocuments to deal with out-of-vocabulary words in keyphrases. Experiment\nresults on five benchmark datasets illustrate the validity and effectiveness of\nour model, which achieves the state-of-the-art performance on all datasets.\nFurther analyses confirm that the proposed model is able to generate keyphrases\nconsistent with references while keeping sufficient diversity. The code of\nCDKGen is available at https://github.com/SVAIGBA/CDKGen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09813v2",
            "date": "2020-04-21 08:20:25+00:00",
            "title": "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "We present an easy and efficient method to extend existing sentence embedding\nmodels to new languages. This allows to create multilingual versions from\npreviously monolingual models. The training is based on the idea that a\ntranslated sentence should be mapped to the same location in the vector space\nas the original sentence. We use the original (monolingual) model to generate\nsentence embeddings for the source language and then train a new system on\ntranslated sentences to mimic the original model. Compared to other methods for\ntraining multilingual sentence embeddings, this approach has several\nadvantages: It is easy to extend existing models with relatively few samples to\nnew languages, it is easier to ensure desired properties for the vector space,\nand the hardware requirements for training is lower. We demonstrate the\neffectiveness of our approach for 50+ languages from various language families.\nCode to extend sentence embeddings models to more than 400 languages is\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09894v2",
            "date": "2020-04-21 10:45:48+00:00",
            "title": "Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns",
            "authors": [
                "KayYen Wong",
                "Sameen Maruf",
                "Gholamreza Haffari"
            ],
            "abstract": "The advent of context-aware NMT has resulted in promising improvements in the\noverall translation quality and specifically in the translation of discourse\nphenomena such as pronouns. Previous works have mainly focused on the use of\npast sentences as context with a focus on anaphora translation. In this work,\nwe investigate the effect of future sentences as context by comparing the\nperformance of a contextual NMT model trained with the future context to the\none trained with the past context. Our experiments and evaluation, using\ngeneric and pronoun-focused automatic metrics, show that the use of future\ncontext not only achieves significant improvements over the context-agnostic\nTransformer, but also demonstrates comparable and in some cases improved\nperformance over its counterpart trained on past context. We also perform an\nevaluation on a targeted cataphora test suite and report significant gains over\nthe context-agnostic Transformer in terms of BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09930v1",
            "date": "2020-04-21 12:03:04+00:00",
            "title": "Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents",
            "authors": [
                "Daoyuan Chen",
                "Yaliang Li",
                "Kai Lei",
                "Ying Shen"
            ],
            "abstract": "Distant supervision based methods for entity and relation extraction have\nreceived increasing popularity due to the fact that these methods require light\nhuman annotation efforts. In this paper, we consider the problem of\n\\textit{shifted label distribution}, which is caused by the inconsistency\nbetween the noisy-labeled training set subject to external knowledge graph and\nthe human-annotated test set, and exacerbated by the pipelined\nentity-then-relation extraction manner with noise propagation. We propose a\njoint extraction approach to address this problem by re-labeling noisy\ninstances with a group of cooperative multiagents. To handle noisy instances in\na fine-grained manner, each agent in the cooperative group evaluates the\ninstance by calculating a continuous confidence score from its own perspective;\nTo leverage the correlations between these two extraction tasks, a confidence\nconsensus module is designed to gather the wisdom of all agents and\nre-distribute the noisy training set with confidence-scored labels. Further,\nthe confidences are used to adjust the training losses of extractors.\nExperimental results on two real-world datasets verify the benefits of\nre-labeling noisy instance, and show that the proposed model significantly\noutperforms the state-of-the-art entity and relation extraction methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09936v3",
            "date": "2020-04-21 12:10:48+00:00",
            "title": "DIET: Lightweight Language Understanding for Dialogue Systems",
            "authors": [
                "Tanja Bunk",
                "Daksh Varshneya",
                "Vladimir Vlasov",
                "Alan Nichol"
            ],
            "abstract": "Large-scale pre-trained language models have shown impressive results on\nlanguage understanding benchmarks like GLUE and SuperGLUE, improving\nconsiderably over other pre-training methods like distributed representations\n(GloVe) and purely supervised approaches. We introduce the Dual Intent and\nEntity Transformer (DIET) architecture, and study the effectiveness of\ndifferent pre-trained representations on intent and entity prediction, two\ncommon dialogue language understanding tasks. DIET advances the state of the\nart on a complex multi-domain NLU dataset and achieves similarly high\nperformance on other simpler datasets. Surprisingly, we show that there is no\nclear benefit to using large pre-trained models for this task, and in fact DIET\nimproves upon the current state of the art even in a purely supervised setup\nwithout any pre-trained embeddings. Our best performing model outperforms\nfine-tuning BERT and is about six times faster to train.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09984v3",
            "date": "2020-04-21 13:30:02+00:00",
            "title": "BERT-ATTACK: Adversarial Attack Against BERT Using BERT",
            "authors": [
                "Linyang Li",
                "Ruotian Ma",
                "Qipeng Guo",
                "Xiangyang Xue",
                "Xipeng Qiu"
            ],
            "abstract": "Adversarial attacks for discrete data (such as texts) have been proved\nsignificantly more challenging than continuous data (such as images) since it\nis difficult to generate adversarial samples with gradient-based methods.\nCurrent successful attack methods for texts usually adopt heuristic replacement\nstrategies on the character or word level, which remains challenging to find\nthe optimal solution in the massive space of possible combinations of\nreplacements while preserving semantic consistency and language fluency. In\nthis paper, we propose \\textbf{BERT-Attack}, a high-quality and effective\nmethod to generate adversarial samples using pre-trained masked language models\nexemplified by BERT. We turn BERT against its fine-tuned models and other deep\nneural models in downstream tasks so that we can successfully mislead the\ntarget models to predict incorrectly. Our method outperforms state-of-the-art\nattack strategies in both success rate and perturb percentage, while the\ngenerated adversarial samples are fluent and semantically preserved. Also, the\ncost of calculation is low, thus possible for large-scale generations. The code\nis available at https://github.com/LinyangLee/BERT-Attack.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10051v1",
            "date": "2020-04-21 14:41:38+00:00",
            "title": "Learning Relation Ties with a Force-Directed Graph in Distant Supervised Relation Extraction",
            "authors": [
                "Yuming Shang",
                "Heyan Huang",
                "Xin Sun",
                "Xianling Mao"
            ],
            "abstract": "Relation ties, defined as the correlation and mutual exclusion between\ndifferent relations, are critical for distant supervised relation extraction.\nExisting approaches model this property by greedily learning local\ndependencies. However, they are essentially limited by failing to capture the\nglobal topology structure of relation ties. As a result, they may easily fall\ninto a locally optimal solution. To solve this problem, in this paper, we\npropose a novel force-directed graph based relation extraction model to\ncomprehensively learn relation ties. Specifically, we first build a graph\naccording to the global co-occurrence of relations. Then, we borrow the idea of\nCoulomb's Law from physics and introduce the concept of attractive force and\nrepulsive force to this graph to learn correlation and mutual exclusion between\nrelations. Finally, the obtained relation representations are applied as an\ninter-dependent relation classifier. Experimental results on a large scale\nbenchmark dataset demonstrate that our model is capable of modeling global\nrelation ties and significantly outperforms other baselines. Furthermore, the\nproposed force-directed graph can be used as a module to augment existing\nrelation extraction systems and improve their performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10102v2",
            "date": "2020-04-21 15:22:27+00:00",
            "title": "Attention is Not Only a Weight: Analyzing Transformers with Vector Norms",
            "authors": [
                "Goro Kobayashi",
                "Tatsuki Kuribayashi",
                "Sho Yokoi",
                "Kentaro Inui"
            ],
            "abstract": "Attention is a key component of Transformers, which have recently achieved\nconsiderable success in natural language processing. Hence, attention is being\nextensively studied to investigate various linguistic capabilities of\nTransformers, focusing on analyzing the parallels between attention weights and\nspecific linguistic phenomena. This paper shows that attention weights alone\nare only one of the two factors that determine the output of attention and\nproposes a norm-based analysis that incorporates the second factor, the norm of\nthe transformed input vectors. The findings of our norm-based analyses of BERT\nand a Transformer-based neural machine translation system include the\nfollowing: (i) contrary to previous studies, BERT pays poor attention to\nspecial tokens, and (ii) reasonable word alignment can be extracted from\nattention mechanisms of Transformer. These findings provide insights into the\ninner workings of Transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10150v1",
            "date": "2020-04-21 16:54:57+00:00",
            "title": "Unsupervised Opinion Summarization with Noising and Denoising",
            "authors": [
                "Reinald Kim Amplayo",
                "Mirella Lapata"
            ],
            "abstract": "The supervised training of high-capacity models on large datasets containing\nhundreds of thousands of document-summary pairs is critical to the recent\nsuccess of deep learning techniques for abstractive summarization.\nUnfortunately, in most domains (other than news) such training data is not\navailable and cannot be easily sourced. In this paper we enable the use of\nsupervised learning for the setting where there are only documents available\n(e.g.,~product or business reviews) without ground truth summaries. We create a\nsynthetic dataset from a corpus of user reviews by sampling a review,\npretending it is a summary, and generating noisy versions thereof which we\ntreat as pseudo-review input. We introduce several linguistically motivated\nnoise generation functions and a summarization model which learns to denoise\nthe input and generate the original review. At test time, the model accepts\ngenuine reviews and generates a summary containing salient opinions, treating\nthose that do not reach consensus as noise. Extensive automatic and human\nevaluation shows that our model brings substantial improvements over both\nabstractive and extractive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10157v2",
            "date": "2020-04-21 17:03:08+00:00",
            "title": "Logic-Guided Data Augmentation and Regularization for Consistent Question Answering",
            "authors": [
                "Akari Asai",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Many natural language questions require qualitative, quantitative or logical\ncomparisons between two entities or events. This paper addresses the problem of\nimproving the accuracy and consistency of responses to comparison questions by\nintegrating logic rules and neural models. Our method leverages logical and\nlinguistic knowledge to augment labeled training data and then uses a\nconsistency-based regularizer to train the model. Improving the global\nconsistency of predictions, our approach achieves large improvements over\nprevious methods in a variety of question answering (QA) tasks including\nmultiple-choice qualitative reasoning, cause-effect reasoning, and extractive\nmachine reading comprehension. In particular, our method significantly improves\nthe performance of RoBERTa-based models by 1-5% across datasets. We advance the\nstate of the art by around 5-8% on WIQA and QuaRel and reduce consistency\nviolations by 58% on HotpotQA. We further demonstrate that our approach can\nlearn effectively from limited data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10171v1",
            "date": "2020-04-21 17:26:16+00:00",
            "title": "Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation",
            "authors": [
                "Haipeng Sun",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita",
                "Tiejun Zhao"
            ],
            "abstract": "Unsupervised neural machine translation (UNMT) has recently achieved\nremarkable results for several language pairs. However, it can only translate\nbetween a single language pair and cannot produce translation results for\nmultiple language pairs at the same time. That is, research on multilingual\nUNMT has been limited. In this paper, we empirically introduce a simple method\nto translate between thirteen languages using a single encoder and a single\ndecoder, making use of multilingual data to improve UNMT for all language\npairs. On the basis of the empirical findings, we propose two knowledge\ndistillation methods to further enhance multilingual UNMT performance. Our\nexperiments on a dataset with English translated to and from twelve other\nlanguages (including three language families and six language branches) show\nremarkable results, surpassing strong unsupervised individual baselines while\nachieving promising performance between non-English language pairs in zero-shot\ntranslation scenarios and alleviating poor performance in low-resource language\npairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10283v1",
            "date": "2020-04-21 20:29:50+00:00",
            "title": "Observations on Annotations",
            "authors": [
                "Georg Rehm"
            ],
            "abstract": "The annotation of textual information is a fundamental activity in\nLinguistics and Computational Linguistics. This article presents various\nobservations on annotations. It approaches the topic from several angles\nincluding Hypertext, Computational Linguistics and Language Technology,\nArtificial Intelligence and Open Science. Annotations can be examined along\ndifferent dimensions. In terms of complexity, they can range from trivial to\nhighly sophisticated, in terms of maturity from experimental to standardised.\nAnnotations can be annotated themselves using more abstract annotations.\nPrimary research data such as, e.g., text documents can be annotated on\ndifferent layers concurrently, which are independent but can be exploited using\nmulti-layer querying. Standards guarantee interoperability and reusability of\ndata sets. The chapter concludes with four final observations, formulated as\nresearch questions or rather provocative remarks on the current state of\nannotation research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10320v1",
            "date": "2020-04-21 22:02:43+00:00",
            "title": "A Deep Learning System for Sentiment Analysis of Service Calls",
            "authors": [
                "Yanan Jia",
                "Sony SungChu"
            ],
            "abstract": "Sentiment analysis is crucial for the advancement of artificial intelligence\n(AI). Sentiment understanding can help AI to replicate human language and\ndiscourse. Studying the formation and response of sentiment state from\nwell-trained Customer Service Representatives (CSRs) can help make the\ninteraction between humans and AI more intelligent. In this paper, a sentiment\nanalysis pipeline is first carried out with respect to real-world multi-party\nconversations - that is, service calls. Based on the acoustic and linguistic\nfeatures extracted from the source information, a novel aggregated method for\nvoice sentiment recognition framework is built. Each party's sentiment pattern\nduring the communication is investigated along with the interaction sentiment\npattern between all parties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10353v2",
            "date": "2020-04-22 00:53:40+00:00",
            "title": "Supervised Grapheme-to-Phoneme Conversion of Orthographic Schwas in Hindi and Punjabi",
            "authors": [
                "Aryaman Arora",
                "Luke Gessler",
                "Nathan Schneider"
            ],
            "abstract": "Hindi grapheme-to-phoneme (G2P) conversion is mostly trivial, with one\nexception: whether a schwa represented in the orthography is pronounced or\nunpronounced (deleted). Previous work has attempted to predict schwa deletion\nin a rule-based fashion using prosodic or phonetic analysis. We present the\nfirst statistical schwa deletion classifier for Hindi, which relies solely on\nthe orthography as the input and outperforms previous approaches. We trained\nour model on a newly-compiled pronunciation lexicon extracted from various\nonline dictionaries. Our best Hindi model achieves state of the art\nperformance, and also achieves good performance on a closely related language,\nPunjabi, without modification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10450v1",
            "date": "2020-04-22 09:12:10+00:00",
            "title": "Trading Off Diversity and Quality in Natural Language Generation",
            "authors": [
                "Hugh Zhang",
                "Daniel Duckworth",
                "Daphne Ippolito",
                "Arvind Neelakantan"
            ],
            "abstract": "For open-ended language generation tasks such as storytelling and dialogue,\nchoosing the right decoding algorithm is critical to controlling the tradeoff\nbetween generation quality and diversity. However, there presently exists no\nconsensus on which decoding procedure is best or even the criteria by which to\ncompare them. We address these issues by casting decoding as a multi-objective\noptimization problem aiming to simultaneously maximize both response quality\nand diversity. Our framework enables us to perform the first large-scale\nevaluation of decoding methods along the entire quality-diversity spectrum. We\nfind that when diversity is a priority, all methods perform similarly, but when\nquality is viewed as more important, the recently proposed nucleus sampling\n(Holtzman et al. 2019) outperforms all other evaluated decoding algorithms. Our\nexperiments also confirm the existence of the `likelihood trap', the\ncounter-intuitive observation that high likelihood sequences are often\nsurprisingly low quality. We leverage our findings to create and evaluate an\nalgorithm called \\emph{selective sampling} which tractably approximates\nglobally-normalized temperature sampling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10462v1",
            "date": "2020-04-22 09:35:02+00:00",
            "title": "Keyphrase Prediction With Pre-trained Language Model",
            "authors": [
                "Rui Liu",
                "Zheng Lin",
                "Weiping Wang"
            ],
            "abstract": "Recently, generative methods have been widely used in keyphrase prediction,\nthanks to their capability to produce both present keyphrases that appear in\nthe source text and absent keyphrases that do not match any source text.\nHowever, the absent keyphrases are generated at the cost of the performance on\npresent keyphrase prediction, since previous works mainly use generative models\nthat rely on the copying mechanism and select words step by step. Besides, the\nextractive model that directly extracts a text span is more suitable for\npredicting the present keyphrase. Considering the different characteristics of\nextractive and generative methods, we propose to divide the keyphrase\nprediction into two subtasks, i.e., present keyphrase extraction (PKE) and\nabsent keyphrase generation (AKG), to fully exploit their respective\nadvantages. On this basis, a joint inference framework is proposed to make the\nmost of BERT in two subtasks. For PKE, we tackle this task as a sequence\nlabeling problem with the pre-trained language model BERT. For AKG, we\nintroduce a Transformer-based architecture, which fully integrates the present\nkeyphrase knowledge learned from PKE by the fine-tuned BERT. The experimental\nresults show that our approach can achieve state-of-the-art results on both\ntasks on benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10473v1",
            "date": "2020-04-22 10:05:52+00:00",
            "title": "Where is the context? -- A critique of recent dialogue datasets",
            "authors": [
                "Johannes E. M. Mosig",
                "Vladimir Vlasov",
                "Alan Nichol"
            ],
            "abstract": "Recent dialogue datasets like MultiWOZ 2.1 and Taskmaster-1 constitute some\nof the most challenging tasks for present-day dialogue models and, therefore,\nare widely used for system evaluation. We identify several issues with the\nabove-mentioned datasets, such as history independence, strong knowledge base\ndependence, and ambiguous system responses. Finally, we outline key desiderata\nfor future datasets that we believe would be more suitable for the construction\nof conversational artificial intelligence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10581v1",
            "date": "2020-04-22 14:00:55+00:00",
            "title": "When and Why is Unsupervised Neural Machine Translation Useless?",
            "authors": [
                "Yunsu Kim",
                "Miguel Gra\u00e7a",
                "Hermann Ney"
            ],
            "abstract": "This paper studies the practicality of the current state-of-the-art\nunsupervised methods in neural machine translation (NMT). In ten translation\ntasks with various data settings, we analyze the conditions under which the\nunsupervised methods fail to produce reasonable translations. We show that\ntheir performance is severely affected by linguistic dissimilarity and domain\nmismatch between source and target monolingual data. Such conditions are common\nfor low-resource language pairs, where unsupervised learning works poorly. In\nall of our experiments, supervised and semi-supervised baselines with\n50k-sentence bilingual data outperform the best unsupervised results. Our\nanalyses pinpoint the limits of the current unsupervised NMT and also suggest\nimmediate research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10610v1",
            "date": "2020-04-22 14:48:03+00:00",
            "title": "R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning",
            "authors": [
                "Irene Li",
                "Alexander Fabbri",
                "Swapnil Hingmire",
                "Dragomir Radev"
            ],
            "abstract": "The task of concept prerequisite chain learning is to automatically determine\nthe existence of prerequisite relationships among concept pairs. In this paper,\nwe frame learning prerequisite relationships among concepts as an unsupervised\ntask with no access to labeled concept pairs during training. We propose a\nmodel called the Relational-Variational Graph AutoEncoder (R-VGAE) to predict\nconcept relations within a graph consisting of concept and resource nodes.\nResults show that our unsupervised approach outperforms graph-based\nsemi-supervised methods and other baseline methods by up to 9.77% and 10.47% in\nterms of prerequisite relation prediction accuracy and F1 score. Our method is\nnotably the first graph-based model that attempts to make use of deep learning\nrepresentations for the task of unsupervised prerequisite learning. We also\nexpand an existing corpus which totals 1,717 English Natural Language\nProcessing (NLP)-related lecture slide files and manual concept pair\nannotations over 322 topics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10640v1",
            "date": "2020-04-22 15:34:11+00:00",
            "title": "Semantic Entity Enrichment by Leveraging Multilingual Descriptions for Link Prediction",
            "authors": [
                "Genet Asefa Gesese",
                "Mehwish Alam",
                "Harald Sack"
            ],
            "abstract": "Most Knowledge Graphs (KGs) contain textual descriptions of entities in\nvarious natural languages. These descriptions of entities provide valuable\ninformation that may not be explicitly represented in the structured part of\nthe KG. Based on this fact, some link prediction methods which make use of the\ninformation presented in the textual descriptions of entities have been\nproposed to learn representations of (monolingual) KGs. However, these methods\nuse entity descriptions in only one language and ignore the fact that\ndescriptions given in different languages may provide complementary information\nand thereby also additional semantics. In this position paper, the problem of\neffectively leveraging multilingual entity descriptions for the purpose of link\nprediction in KGs will be discussed along with potential solutions to the\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10643v1",
            "date": "2020-04-22 15:38:18+00:00",
            "title": "Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection",
            "authors": [
                "Joakim Nivre",
                "Marie-Catherine de Marneffe",
                "Filip Ginter",
                "Jan Haji\u010d",
                "Christopher D. Manning",
                "Sampo Pyysalo",
                "Sebastian Schuster",
                "Francis Tyers",
                "Daniel Zeman"
            ],
            "abstract": "Universal Dependencies is an open community effort to create\ncross-linguistically consistent treebank annotation for many languages within a\ndependency-based lexicalist framework. The annotation consists in a\nlinguistically motivated word segmentation; a morphological layer comprising\nlemmas, universal part-of-speech tags, and standardized morphological features;\nand a syntactic layer focusing on syntactic relations between predicates,\narguments and modifiers. In this paper, we describe version 2 of the guidelines\n(UD v2), discuss the major changes from UD v1 to UD v2, and give an overview of\nthe currently available treebanks for 90 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10663v2",
            "date": "2020-04-22 16:00:09+00:00",
            "title": "Fast and Scalable Dialogue State Tracking with Explicit Modular Decomposition",
            "authors": [
                "Dingmin Wang",
                "Chenghua Lin",
                "Li Zhong",
                "Kam-Fai Wong"
            ],
            "abstract": "We present a fast and scalable architecture called Explicit Modular\nDecomposition (EMD), in which we incorporate both classification-based and\nextraction-based methods and design four modules (for classification and\nsequence labelling) to jointly extract dialogue states. Experimental results\nbased on the MultiWoz 2.0 dataset validates the superiority of our proposed\nmodel in terms of both complexity and scalability when compared to the\nstate-of-the-art methods, especially in the scenario of multi-domain dialogues\nentangled with many turns of utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10793v1",
            "date": "2020-04-22 18:54:38+00:00",
            "title": "Learning to Classify Intents and Slot Labels Given a Handful of Examples",
            "authors": [
                "Jason Krone",
                "Yi Zhang",
                "Mona Diab"
            ],
            "abstract": "Intent classification (IC) and slot filling (SF) are core components in most\ngoal-oriented dialogue systems. Current IC/SF models perform poorly when the\nnumber of training examples per class is small. We propose a new few-shot\nlearning task, few-shot IC/SF, to study and improve the performance of IC and\nSF models on classes not seen at training time in ultra low resource scenarios.\nWe establish a few-shot IC/SF benchmark by defining few-shot splits for three\npublic IC/SF datasets, ATIS, TOP, and Snips. We show that two popular few-shot\nlearning algorithms, model agnostic meta learning (MAML) and prototypical\nnetworks, outperform a fine-tuning baseline on this benchmark. Prototypical\nnetworks achieves significant gains in IC performance on the ATIS and TOP\ndatasets, while both prototypical networks and MAML outperform the baseline\nwith respect to SF on all three datasets. In addition, we demonstrate that\njoint training as well as the use of pre-trained language models, ELMo and BERT\nin our case, are complementary to these few-shot learning methods and yield\nfurther gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10809v2",
            "date": "2020-04-22 19:26:09+00:00",
            "title": "Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation",
            "authors": [
                "Vikash Balasubramanian",
                "Ivan Kobyzev",
                "Hareesh Bahuleyan",
                "Ilya Shapiro",
                "Olga Vechtomova"
            ],
            "abstract": "Learning disentangled representations of real-world data is a challenging\nopen problem. Most previous methods have focused on either supervised\napproaches which use attribute labels or unsupervised approaches that\nmanipulate the factorization in the latent space of models such as the\nvariational autoencoder (VAE) by training with task-specific losses. In this\nwork, we propose polarized-VAE, an approach that disentangles select attributes\nin the latent space based on proximity measures reflecting the similarity\nbetween data points with respect to these attributes. We apply our method to\ndisentangle the semantics and syntax of sentences and carry out transfer\nexperiments. Polarized-VAE outperforms the VAE baseline and is competitive with\nstate-of-the-art approaches, while being more a general framework that is\napplicable to other attribute disentanglement tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10813v1",
            "date": "2020-04-22 19:29:43+00:00",
            "title": "Revisiting the Context Window for Cross-lingual Word Embeddings",
            "authors": [
                "Ryokan Ri",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "Existing approaches to mapping-based cross-lingual word embeddings are based\non the assumption that the source and target embedding spaces are structurally\nsimilar. The structures of embedding spaces largely depend on the co-occurrence\nstatistics of each word, which the choice of context window determines. Despite\nthis obvious connection between the context window and mapping-based\ncross-lingual embeddings, their relationship has been underexplored in prior\nwork. In this work, we provide a thorough evaluation, in various languages,\ndomains, and tasks, of bilingual embeddings trained with different context\nwindows. The highlight of our findings is that increasing the size of both the\nsource and target window sizes improves the performance of bilingual lexicon\ninduction, especially the performance on frequent nouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10827v1",
            "date": "2020-04-22 20:02:49+00:00",
            "title": "Syntactic Structure from Deep Learning",
            "authors": [
                "Tal Linzen",
                "Marco Baroni"
            ],
            "abstract": "Modern deep neural networks achieve impressive performance in engineering\napplications that require extensive linguistic skills, such as machine\ntranslation. This success has sparked interest in probing whether these models\nare inducing human-like grammatical knowledge from the raw data they are\nexposed to, and, consequently, whether they can shed new light on long-standing\ndebates concerning the innate structure necessary for language acquisition. In\nthis article, we survey representative studies of the syntactic abilities of\ndeep networks, and discuss the broader implications that this work has for\ntheoretical linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10863v1",
            "date": "2020-04-22 21:09:47+00:00",
            "title": "Preserving the Hypernym Tree of WordNet in Dense Embeddings",
            "authors": [
                "Canlin Zhang",
                "Xiuwen Liu"
            ],
            "abstract": "In this paper, we provide a novel way to generate low-dimension (dense)\nvector embeddings for the noun and verb synsets in WordNet, so that the\nhypernym-hyponym tree structure is preserved in the embeddings. We call this\nembedding the sense spectrum (and sense spectra for embeddings). In order to\ncreate suitable labels for the training of sense spectra, we designed a new\nsimilarity measurement for noun and verb synsets in WordNet. We call this\nsimilarity measurement the hypernym intersection similarity (HIS), since it\ncompares the common and unique hypernyms between two synsets.\n  Our experiments show that on the noun and verb pairs of the SimLex-999\ndataset, HIS outperforms the three similarity measurements in WordNet.\nMoreover, to the best of our knowledge, the sense spectra is the first dense\nembedding system that can explicitly and completely measure the\nhypernym-hyponym relationship in WordNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10972v1",
            "date": "2020-04-23 05:02:31+00:00",
            "title": "Semi-Supervised Models via Data Augmentationfor Classifying Interactive Affective Responses",
            "authors": [
                "Jiaao Chen",
                "Yuwei Wu",
                "Diyi Yang"
            ],
            "abstract": "We present semi-supervised models with data augmentation (SMDA), a\nsemi-supervised text classification system to classify interactive affective\nresponses. SMDA utilizes recent transformer-based models to encode each\nsentence and employs back translation techniques to paraphrase given sentences\nas augmented data. For labeled sentences, we performed data augmentations to\nuniform the label distributions and computed supervised loss during training\nprocess. For unlabeled sentences, we explored self-training by regarding\nlow-entropy predictions over unlabeled sentences as pseudo labels, assuming\nhigh-confidence predictions as labeled data for training. We further introduced\nconsistency regularization as unsupervised loss after data augmentations on\nunlabeled data, based on the assumption that the model should predict similar\nclass distributions with original unlabeled sentences as input and augmented\nsentences as input. Via a set of experiments, we demonstrated that our system\noutperformed baseline models in terms of F1-score and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11019v3",
            "date": "2020-04-23 08:17:22+00:00",
            "title": "Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog",
            "authors": [
                "Libo Qin",
                "Xiao Xu",
                "Wanxiang Che",
                "Yue Zhang",
                "Ting Liu"
            ],
            "abstract": "Recent studies have shown remarkable success in end-to-end task-oriented\ndialog system. However, most neural models rely on large training data, which\nare only available for a certain number of task domains, such as navigation and\nscheduling.\n  This makes it difficult to scalable for a new domain with limited labeled\ndata. However, there has been relatively little research on how to effectively\nuse data from all domains to improve the performance of each domain and also\nunseen domains. To this end, we investigate methods that can make explicit use\nof domain knowledge and introduce a shared-private network to learn shared and\nspecific knowledge. In addition, we propose a novel Dynamic Fusion Network\n(DF-Net) which automatically exploit the relevance between the target domain\nand each domain. Results show that our model outperforms existing methods on\nmulti-domain dialogue, giving the state-of-the-art in the literature. Besides,\nwith little training data, we show its transferability by outperforming prior\nbest model by 13.9\\% on average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11026v1",
            "date": "2020-04-23 08:41:52+00:00",
            "title": "QURIOUS: Question Generation Pretraining for Text Generation",
            "authors": [
                "Shashi Narayan",
                "Gon\u00e7alo Simoes",
                "Ji Ma",
                "Hannah Craighead",
                "Ryan Mcdonald"
            ],
            "abstract": "Recent trends in natural language processing using pretraining have shifted\nfocus towards pretraining and fine-tuning approaches for text generation. Often\nthe focus has been on task-agnostic approaches that generalize the language\nmodeling objective. We propose question generation as a pretraining method,\nwhich better aligns with the text generation objectives. Our text generation\nmodels pretrained with this method are better at understanding the essence of\nthe input and are better language models for the target task. When evaluated on\ntwo text generation tasks, abstractive summarization and answer-focused\nquestion generation, our models result in state-of-the-art performances in\nterms of automatic metrics. Human evaluators also found our summaries and\ngenerated questions to be more natural, concise and informative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11142v1",
            "date": "2020-04-23 13:38:18+00:00",
            "title": "DuReaderrobust: A Chinese Dataset Towards Evaluating the Robustness of Machine Reading Comprehension Models",
            "authors": [
                "Hongxuan Tang",
                "Jing Liu",
                "Hongyu Li",
                "Yu Hong",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Machine Reading Comprehension (MRC) is a crucial and challenging task in\nnatural language processing. Although several MRC models obtains human parity\nperformance on several datasets, we find that these models are still far from\nrobust. To comprehensively evaluate the robustness of MRC models, we create a\nChinese dataset, namely DuReader_{robust}. It is designed to challenge MRC\nmodels from the following aspects: (1) over-sensitivity, (2) over-stability and\n(3) generalization. Most of previous work studies these problems by altering\nthe inputs to unnatural texts. By contrast, the advantage of DuReader_{robust}\nis that its questions and documents are natural texts. It presents the\nrobustness challenges when applying MRC models to real-world applications. The\nexperimental results show that MRC models based on the pre-trained language\nmodels perform much worse than human does on the robustness test set, although\nthey perform as well as human on in-domain test set. Additionally, we analyze\nthe behavior of existing models on the robustness test set, which might give\nsuggestions for future model development. The dataset and codes are available\nat\n\\url{https://github.com/PaddlePaddle/Research/tree/master/NLP/DuReader-Robust-BASELINE}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11207v2",
            "date": "2020-04-23 14:58:22+00:00",
            "title": "Self-Attention Attribution: Interpreting Information Interactions Inside Transformer",
            "authors": [
                "Yaru Hao",
                "Li Dong",
                "Furu Wei",
                "Ke Xu"
            ],
            "abstract": "The great success of Transformer-based models benefits from the powerful\nmulti-head self-attention mechanism, which learns token dependencies and\nencodes contextual information from the input. Prior work strives to attribute\nmodel decisions to individual input features with different saliency measures,\nbut they fail to explain how these input features interact with each other to\nreach predictions. In this paper, we propose a self-attention attribution\nmethod to interpret the information interactions inside Transformer. We take\nBERT as an example to conduct extensive studies. Firstly, we apply\nself-attention attribution to identify the important attention heads, while\nothers can be pruned with marginal performance degradation. Furthermore, we\nextract the most salient dependencies in each layer to construct an attribution\ntree, which reveals the hierarchical interactions inside Transformer. Finally,\nwe show that the attribution results can be used as adversarial patterns to\nimplement non-targeted attacks towards BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11222v1",
            "date": "2020-04-23 15:17:37+00:00",
            "title": "Correct Me If You Can: Learning from Error Corrections and Markings",
            "authors": [
                "Julia Kreutzer",
                "Nathaniel Berger",
                "Stefan Riezler"
            ],
            "abstract": "Sequence-to-sequence learning involves a trade-off between signal strength\nand annotation cost of training data. For example, machine translation data\nrange from costly expert-generated translations that enable supervised\nlearning, to weak quality-judgment feedback that facilitate reinforcement\nlearning. We present the first user study on annotation cost and machine\nlearnability for the less popular annotation mode of error markings. We show\nthat error markings for translations of TED talks from English to German allow\nprecise credit assignment while requiring significantly less human effort than\ncorrecting/post-editing, and that error-marked data can be used successfully to\nfine-tune neural machine translation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11405v2",
            "date": "2020-04-23 18:03:41+00:00",
            "title": "Transliteration of Judeo-Arabic Texts into Arabic Script Using Recurrent Neural Networks",
            "authors": [
                "Ori Terner",
                "Kfir Bar",
                "Nachum Dershowitz"
            ],
            "abstract": "We trained a model to automatically transliterate Judeo-Arabic texts into\nArabic script, enabling Arabic readers to access those writings. We employ a\nrecurrent neural network (RNN), combined with the connectionist temporal\nclassification (CTC) loss to deal with unequal input/output lengths. This\nobligates adjustments in the training data to avoid input sequences that are\nshorter than their corresponding outputs. We also utilize a pretraining stage\nwith a different loss function to improve network converge. Since only a single\nsource of parallel text was available for training, we take advantage of the\npossibility of generating data synthetically. We train a model that has the\ncapability to memorize words in the output language, and that also utilizes\ncontext for distinguishing ambiguities in the transliteration. We obtain an\nimprovement over the baseline 9.5% character error, achieving 2% error with our\nbest configuration. To measure the contribution of context to learning, we also\ntested word-shuffled data, for which the error rises to 2.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11471v1",
            "date": "2020-04-23 21:40:30+00:00",
            "title": "A Tool for Facilitating OCR Postediting in Historical Documents",
            "authors": [
                "Alberto Poncelas",
                "Mohammad Aboomar",
                "Jan Buts",
                "James Hadley",
                "Andy Way"
            ],
            "abstract": "Optical character recognition (OCR) for historical documents is a complex\nprocedure subject to a unique set of material issues, including inconsistencies\nin typefaces and low quality scanning. Consequently, even the most\nsophisticated OCR engines produce errors. This paper reports on a tool built\nfor postediting the output of Tesseract, more specifically for correcting\ncommon errors in digitized historical documents. The proposed tool suggests\nalternatives for word forms not found in a specified vocabulary. The assumed\nerror is replaced by a presumably correct alternative in the post-edition based\non the scores of a Language Model (LM). The tool is tested on a chapter of the\nbook An Essay Towards Regulating the Trade and Employing the Poor of this\nKingdom (Cary ,1719). As demonstrated below, the tool is successful in\ncorrecting a number of common errors. If sometimes unreliable, it is also\ntransparent and subject to human intervention.",
            "journal ref": "Workshop on Language Technologies for Historical and Ancient\n  Languages, LT4HALA (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11472v1",
            "date": "2020-04-23 21:48:58+00:00",
            "title": "Multiple Segmentations of Thai Sentences for Neural Machine Translation",
            "authors": [
                "Alberto Poncelas",
                "Wichaya Pidchamook",
                "Chao-Hong Liu",
                "James Hadley",
                "Andy Way"
            ],
            "abstract": "Thai is a low-resource language, so it is often the case that data is not\navailable in sufficient quantities to train an Neural Machine Translation (NMT)\nmodel which perform to a high level of quality. In addition, the Thai script\ndoes not use white spaces to delimit the boundaries between words, which adds\nmore complexity when building sequence to sequence models. In this work, we\nexplore how to augment a set of English--Thai parallel data by replicating\nsentence-pairs with different word segmentation methods on Thai, as training\ndata for NMT model training. Using different merge operations of Byte Pair\nEncoding, different segmentations of Thai sentences can be obtained. The\nexperiments show that combining these datasets, performance is improved for NMT\nmodels trained with a dataset that has been split using a supervised splitting\ntool.",
            "journal ref": "Spoken Language Technologies for Under-resourced languages and\n  CCURL Collaboration and Computing for Under-Resourced Languages Workshop,\n  SLTU-CCURL (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11493v2",
            "date": "2020-04-23 23:59:58+00:00",
            "title": "UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection",
            "authors": [
                "Gregor Wiedemann",
                "Seid Muhie Yimam",
                "Chris Biemann"
            ],
            "abstract": "Fine-tuning of pre-trained transformer networks such as BERT yield\nstate-of-the-art results for text classification tasks. Typically, fine-tuning\nis performed on task-specific training datasets in a supervised manner. One can\nalso fine-tune in unsupervised manner beforehand by further pre-training the\nmasked language modeling (MLM) task. Hereby, in-domain data for unsupervised\nMLM resembling the actual classification target dataset allows for domain\nadaptation of the model. In this paper, we compare current pre-trained\ntransformer networks with and without MLM fine-tuning on their performance for\noffensive language detection. Our MLM fine-tuned RoBERTa-based classifier\nofficially ranks 1st in the SemEval 2020 Shared Task~12 for the English\nlanguage. Further experiments with the ALBERT model even surpass this result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11546v3",
            "date": "2020-04-24 06:12:10+00:00",
            "title": "Generative Data Augmentation for Commonsense Reasoning",
            "authors": [
                "Yiben Yang",
                "Chaitanya Malaviya",
                "Jared Fernandez",
                "Swabha Swayamdipta",
                "Ronan Le Bras",
                "Ji-Ping Wang",
                "Chandra Bhagavatula",
                "Yejin Choi",
                "Doug Downey"
            ],
            "abstract": "Recent advances in commonsense reasoning depend on large-scale\nhuman-annotated training data to achieve peak performance. However, manual\ncuration of training examples is expensive and has been shown to introduce\nannotation artifacts that neural models can readily exploit and overfit on. We\ninvestigate G-DAUG^C, a novel generative data augmentation method that aims to\nachieve more accurate and robust learning in the low-resource setting. Our\napproach generates synthetic examples using pretrained language models, and\nselects the most informative and diverse set of examples for data augmentation.\nIn experiments with multiple commonsense reasoning benchmarks, G-DAUG^C\nconsistently outperforms existing data augmentation methods based on\nback-translation, and establishes a new state-of-the-art on WinoGrande, CODAH,\nand CommonsenseQA. Further, in addition to improvements in in-distribution\naccuracy, G-DAUG^C-augmented training also enhances out-of-distribution\ngeneralization, showing greater robustness against adversarial or perturbed\nexamples. Our analysis demonstrates that G-DAUG^C produces a diverse set of\nfluent training examples, and that its selection and training approaches are\nimportant for performance. Our findings encourage future research toward\ngenerative data augmentation to enhance both in-distribution learning and\nout-of-distribution generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11579v1",
            "date": "2020-04-24 07:38:19+00:00",
            "title": "Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order",
            "authors": [
                "Yi Liao",
                "Xin Jiang",
                "Qun Liu"
            ],
            "abstract": "Masked language model and autoregressive language model are two types of\nlanguage models. While pretrained masked language models such as BERT overwhelm\nthe line of natural language understanding (NLU) tasks, autoregressive language\nmodels such as GPT are especially capable in natural language generation (NLG).\nIn this paper, we propose a probabilistic masking scheme for the masked\nlanguage model, which we call probabilistically masked language model (PMLM).\nWe implement a specific PMLM with a uniform prior distribution on the masking\nratio named u-PMLM. We prove that u-PMLM is equivalent to an autoregressive\npermutated language model. One main advantage of the model is that it supports\ntext generation in arbitrary order with surprisingly good quality, which could\npotentially enable new applications over traditional unidirectional generation.\nBesides, the pretrained u-PMLM also outperforms BERT on a set of downstream NLU\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11583v1",
            "date": "2020-04-24 07:49:45+00:00",
            "title": "Customization and modifications of SignWriting by LIS users",
            "authors": [
                "Claudia S. Bianchini",
                "Fabrizio Borgia",
                "Margherita Castelli"
            ],
            "abstract": "Historically, the various sign languages (SL) have not developed an own\nwriting system; nevertheless, some systems exist, among which the SignWriting\n(SW) is a powerful and flexible one. In this paper, we present the mechanisms\nadopted by signers of the Italian Sign Language (LIS), expert users of SW, to\nmodify the standard SW glyphs and increase their writing skills and/or\nrepresent peculiar linguistic phenomena. We identify these glyphs and show\nwhich characteristics make them \"acceptable\" by the expert community.\nEventually, we analyze the potentialities of these glyphs in hand writing and\nin computer-assisted writing, focusing on SWift, a software designed to allow\nthe electronic writing-down of user-modified glyphs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11622v1",
            "date": "2020-04-24 09:43:14+00:00",
            "title": "Learning the grammar of prescription: recurrent neural network grammars for medication information extraction in clinical texts",
            "authors": [
                "Ivan Lerner",
                "Jordan Jouffroy",
                "Anita Burgun",
                "Antoine Neuraz"
            ],
            "abstract": "In this study, we evaluated the RNNG, a neural top-down transition based\nparser, for medication information extraction in clinical texts. We evaluated\nthis model on a French clinical corpus. The task was to extract the name of a\ndrug (or class of drug), as well as fields informing its administration:\nfrequency, dosage, duration, condition and route of administration. We compared\nthe RNNG model that jointly identify entities and their relations with separate\nBiLSTMs models for entities and relations as baselines. We call seq-BiLSTMs the\nbaseline models for relations extraction that takes as extra-input the output\nof the BiLSTMs for entities. RNNG outperforms seq-BiLSTM for identifying\nrelations, with on average 88.5% [87.2-89.8] versus 84.6 [83.1-86.1] F-measure.\nHowever, RNNG is weaker than the baseline BiLSTM on detecting entities, with on\naverage 82.4 [80.8-83.8] versus 84.1 [82.7-85.6] % F- measure. RNNG trained\nonly for detecting relations is weaker than RNNG with the joint modelling\nobjective, 87.4 [85.8-88.8] versus 88.5% [87.2-89.8]. The performance of RNNG\non relations can be explained both by the model architecture, which provides\nshortcut between distant parts of the sentence, and the joint modelling\nobjective which allow the RNNG to learn richer representations. RNNG is\nefficient for modeling relations between entities in medical texts and its\nperformances are close to those of a BiLSTM for entity detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11727v1",
            "date": "2020-04-24 13:07:12+00:00",
            "title": "Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Peng Xu",
                "Pascale Fung"
            ],
            "abstract": "As an essential task in task-oriented dialog systems, slot filling requires\nextensive training data in a certain domain. However, such data are not always\navailable. Hence, cross-domain slot filling has naturally arisen to cope with\nthis data scarcity problem. In this paper, we propose a Coarse-to-fine approach\n(Coach) for cross-domain slot filling. Our model first learns the general\npattern of slot entities by detecting whether the tokens are slot entities or\nnot. It then predicts the specific types for the slot entities. In addition, we\npropose a template regularization approach to improve the adaptation robustness\nby regularizing the representation of utterances based on utterance templates.\nExperimental results show that our model significantly outperforms\nstate-of-the-art approaches in slot filling. Furthermore, our model can also be\napplied to the cross-domain named entity recognition task, and it achieves\nbetter adaptation performance than other existing baselines. The code is\navailable at https://github.com/zliucr/coach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11742v1",
            "date": "2020-04-24 13:36:38+00:00",
            "title": "ST$^2$: Small-data Text Style Transfer via Multi-task Meta-Learning",
            "authors": [
                "Xiwen Chen",
                "Kenny Q. Zhu"
            ],
            "abstract": "Text style transfer aims to paraphrase a sentence in one style into another\nstyle while preserving content. Due to lack of parallel training data,\nstate-of-art methods are unsupervised and rely on large datasets that share\ncontent. Furthermore, existing methods have been applied on very limited\ncategories of styles such as positive/negative and formal/informal. In this\nwork, we develop a meta-learning framework to transfer between any kind of text\nstyles, including personal writing styles that are more fine-grained, share\nless content and have much smaller training data. While state-of-art models\nfail in the few-shot style transfer task, our framework effectively utilizes\ninformation from other styles to improve both language fluency and style\ntransfer accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11779v2",
            "date": "2020-04-24 14:39:34+00:00",
            "title": "Exploring Explainable Selection to Control Abstractive Summarization",
            "authors": [
                "Wang Haonan",
                "Gao Yang",
                "Bai Yu",
                "Mirella Lapata",
                "Huang Heyan"
            ],
            "abstract": "Like humans, document summarization models can interpret a document's\ncontents in a number of ways. Unfortunately, the neural models of today are\nlargely black boxes that provide little explanation of how or why they\ngenerated a summary in the way they did. Therefore, to begin prying open the\nblack box and to inject a level of control into the substance of the final\nsummary, we developed a novel select-and-generate framework that focuses on\nexplainability. By revealing the latent centrality and interactions between\nsentences, along with scores for sentence novelty and relevance, users are\ngiven a window into the choices a model is making and an opportunity to guide\nthose choices in a more desirable direction. A novel pair-wise matrix captures\nthe sentence interactions, centrality, and attribute scores, and a mask with\ntunable attribute thresholds allows the user to control which sentences are\nlikely to be included in the extraction. A sentence-deployed attention\nmechanism in the abstractor ensures the final summary emphasizes the desired\ncontent. Additionally, the encoder is adaptable, supporting both Transformer-\nand BERT-based configurations. In a series of experiments assessed with ROUGE\nmetrics and two human evaluations, ESCA outperformed eight state-of-the-art\nmodels on the CNN/DailyMail and NYT50 benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11795v2",
            "date": "2020-04-24 15:27:49+00:00",
            "title": "FLAT: Chinese NER Using Flat-Lattice Transformer",
            "authors": [
                "Xiaonan Li",
                "Hang Yan",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, the character-word lattice structure has been proved to be\neffective for Chinese named entity recognition (NER) by incorporating the word\ninformation. However, since the lattice structure is complex and dynamic, most\nexisting lattice-based models are hard to fully utilize the parallel\ncomputation of GPUs and usually have a low inference-speed. In this paper, we\npropose FLAT: Flat-LAttice Transformer for Chinese NER, which converts the\nlattice structure into a flat structure consisting of spans. Each span\ncorresponds to a character or latent word and its position in the original\nlattice. With the power of Transformer and well-designed position encoding,\nFLAT can fully leverage the lattice information and has an excellent\nparallelization ability. Experiments on four datasets show FLAT outperforms\nother lexicon-based models in performance and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11854v1",
            "date": "2020-04-24 16:57:52+00:00",
            "title": "On Sparsifying Encoder Outputs in Sequence-to-Sequence Models",
            "authors": [
                "Biao Zhang",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "Sequence-to-sequence models usually transfer all encoder outputs to the\ndecoder for generation. In this work, by contrast, we hypothesize that these\nencoder outputs can be compressed to shorten the sequence delivered for\ndecoding. We take Transformer as the testbed and introduce a layer of\nstochastic gates in-between the encoder and the decoder. The gates are\nregularized using the expected value of the sparsity-inducing L0penalty,\nresulting in completely masking-out a subset of encoder outputs. In other\nwords, via joint training, the L0DROP layer forces Transformer to route\ninformation through a subset of its encoder states. We investigate the effects\nof this sparsification on two machine translation and two summarization tasks.\nExperiments show that, depending on the task, around 40-70% of source encodings\ncan be pruned without significantly compromising quality. The decrease of the\noutput length endows L0DROP with the potential of improving decoding\nefficiency, where it yields a speedup of up to 1.65x on document summarization\ntasks against the standard Transformer. We analyze the L0DROP behaviour and\nobserve that it exhibits systematic preferences for pruning certain word types,\ne.g., function words and punctuation get pruned most. Inspired by these\nobservations, we explore the feasibility of specifying rule-based patterns that\nmask out encoder outputs based on information such as part-of-speech tags, word\nfrequency and word position.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11867v1",
            "date": "2020-04-24 17:21:32+00:00",
            "title": "Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation",
            "authors": [
                "Biao Zhang",
                "Philip Williams",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "Massively multilingual models for neural machine translation (NMT) are\ntheoretically attractive, but often underperform bilingual models and deliver\npoor zero-shot translations. In this paper, we explore ways to improve them. We\nargue that multilingual NMT requires stronger modeling capacity to support\nlanguage pairs with varying typological characteristics, and overcome this\nbottleneck via language-specific components and deepening NMT architectures. We\nidentify the off-target translation issue (i.e. translating into a wrong target\nlanguage) as the major source of the inferior zero-shot performance, and\npropose random online backtranslation to enforce the translation of unseen\ntraining language pairs. Experiments on OPUS-100 (a novel multilingual dataset\nwith 100 languages) show that our approach substantially narrows the\nperformance gap with bilingual models in both one-to-many and many-to-many\nsettings, and improves zero-shot performance by ~10 BLEU, approaching\nconventional pivot-based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11886v1",
            "date": "2020-04-24 17:52:25+00:00",
            "title": "Lite Transformer with Long-Short Range Attention",
            "authors": [
                "Zhanghao Wu",
                "Zhijian Liu",
                "Ji Lin",
                "Yujun Lin",
                "Song Han"
            ],
            "abstract": "Transformer has become ubiquitous in natural language processing (e.g.,\nmachine translation, question answering); however, it requires enormous amount\nof computations to achieve high performance, which makes it not suitable for\nmobile applications that are tightly constrained by the hardware resources and\nbattery. In this paper, we present an efficient mobile NLP architecture, Lite\nTransformer to facilitate deploying mobile NLP applications on edge devices.\nThe key primitive is the Long-Short Range Attention (LSRA), where one group of\nheads specializes in the local context modeling (by convolution) while another\ngroup specializes in the long-distance relationship modeling (by attention).\nSuch specialization brings consistent improvement over the vanilla transformer\non three well-established language tasks: machine translation, abstractive\nsummarization, and language modeling. Under constrained resources (500M/100M\nMACs), Lite Transformer outperforms transformer on WMT'14 English-French by\n1.2/1.7 BLEU, respectively. Lite Transformer reduces the computation of\ntransformer base model by 2.5x with 0.3 BLEU score degradation. Combining with\npruning and quantization, we further compressed the model size of Lite\nTransformer by 18.2x. For language modeling, Lite Transformer achieves 1.8\nlower perplexity than the transformer at around 500M MACs. Notably, Lite\nTransformer outperforms the AutoML-based Evolved Transformer by 0.5 higher BLEU\nfor the mobile NLP setting without the costly architecture search that requires\nmore than 250 GPU years. Code has been made available at\nhttps://github.com/mit-han-lab/lite-transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11892v1",
            "date": "2020-04-24 17:57:45+00:00",
            "title": "Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering",
            "authors": [
                "Alexander R. Fabbri",
                "Patrick Ng",
                "Zhiguo Wang",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "Question Answering (QA) is in increasing demand as the amount of information\navailable online and the desire for quick access to this content grows. A\ncommon approach to QA has been to fine-tune a pretrained language model on a\ntask-specific labeled dataset. This paradigm, however, relies on scarce, and\ncostly to obtain, large-scale human-labeled data. We propose an unsupervised\napproach to training QA models with generated pseudo-training data. We show\nthat generating questions for QA training by applying a simple template on a\nrelated, retrieved sentence rather than the original context sentence improves\ndownstream QA performance by allowing the model to learn more complex\ncontext-question relationships. Training a QA model on this data gives a\nrelative improvement over a previous unsupervised model in F1 score on the\nSQuAD dataset by about 14%, and 20% when the answer is a named entity,\nachieving state-of-the-art performance on SQuAD for unsupervised QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11954v2",
            "date": "2020-04-24 19:30:38+00:00",
            "title": "Practical Comparable Data Collection for Low-Resource Languages via Images",
            "authors": [
                "Aman Madaan",
                "Shruti Rijhwani",
                "Antonios Anastasopoulos",
                "Yiming Yang",
                "Graham Neubig"
            ],
            "abstract": "We propose a method of curating high-quality comparable training data for\nlow-resource languages with monolingual annotators. Our method involves using a\ncarefully selected set of images as a pivot between the source and target\nlanguages by getting captions for such images in both languages independently.\nHuman evaluations on the English-Hindi comparable corpora created with our\nmethod show that 81.1% of the pairs are acceptable translations, and only 2.47%\nof the pairs are not translations at all. We further establish the potential of\nthe dataset collected through our approach by experimenting on two downstream\ntasks - machine translation and dictionary extraction. All code and data are\navailable at https://github.com/madaan/PML4DC-Comparable-Data-Collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11964v1",
            "date": "2020-04-24 19:52:40+00:00",
            "title": "The Inception Team at NSURL-2019 Task 8: Semantic Question Similarity in Arabic",
            "authors": [
                "Hana Al-Theiabat",
                "Aisha Al-Sadi"
            ],
            "abstract": "This paper describes our method for the task of Semantic Question Similarity\nin Arabic in the workshop on NLP Solutions for Under-Resourced Languages\n(NSURL). The aim is to build a model that is able to detect similar semantic\nquestions in the Arabic language for the provided dataset. Different methods of\ndetermining questions similarity are explored in this work. The proposed models\nachieved high F1-scores, which range from (88% to 96%). Our official best\nresult is produced from the ensemble model of using a pre-trained multilingual\nBERT model with different random seeds with 95.924% F1-Score, which ranks the\nfirst among nine participants teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11997v2",
            "date": "2020-04-24 21:31:57+00:00",
            "title": "New Protocols and Negative Results for Textual Entailment Data Collection",
            "authors": [
                "Samuel R. Bowman",
                "Jennimaria Palomaki",
                "Livio Baldini Soares",
                "Emily Pitler"
            ],
            "abstract": "Natural language inference (NLI) data has proven useful in benchmarking and,\nespecially, as pretraining data for tasks requiring language understanding.\nHowever, the crowdsourcing protocol that was used to collect this data has\nknown issues and was not explicitly optimized for either of these purposes, so\nit is likely far from ideal. We propose four alternative protocols, each aimed\nat improving either the ease with which annotators can produce sound training\nexamples or the quality and diversity of those examples. Using these\nalternatives and a fifth baseline protocol, we collect and compare five new\n8.5k-example training sets. In evaluations focused on transfer learning\napplications, our results are solidly negative, with models trained on our\nbaseline dataset yielding good transfer performance to downstream tasks, but\nnone of our four new methods (nor the recent ANLI) showing any improvements\nover that baseline. In a small silver lining, we observe that all four new\nprotocols, especially those where annotators edit pre-filled text boxes, reduce\npreviously observed issues with annotation artifacts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11999v1",
            "date": "2020-04-24 21:35:26+00:00",
            "title": "Syntactic Data Augmentation Increases Robustness to Inference Heuristics",
            "authors": [
                "Junghyun Min",
                "R. Thomas McCoy",
                "Dipanjan Das",
                "Emily Pitler",
                "Tal Linzen"
            ],
            "abstract": "Pretrained neural models such as BERT, when fine-tuned to perform natural\nlanguage inference (NLI), often show high accuracy on standard datasets, but\ndisplay a surprising lack of sensitivity to word order on controlled challenge\nsets. We hypothesize that this issue is not primarily caused by the pretrained\nmodel's limitations, but rather by the paucity of crowdsourced NLI examples\nthat might convey the importance of syntactic structure at the fine-tuning\nstage. We explore several methods to augment standard training sets with\nsyntactically informative examples, generated by applying syntactic\ntransformations to sentences from the MNLI corpus. The best-performing\naugmentation method, subject/object inversion, improved BERT's accuracy on\ncontrolled examples that diagnose sensitivity to word order from 0.28 to 0.73,\nwithout affecting performance on the MNLI test set. This improvement\ngeneralized beyond the particular construction used for data augmentation,\nsuggesting that augmentation causes BERT to recruit abstract syntactic\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12006v1",
            "date": "2020-04-24 22:08:09+00:00",
            "title": "Contextualized Representations Using Textual Encyclopedic Knowledge",
            "authors": [
                "Mandar Joshi",
                "Kenton Lee",
                "Yi Luan",
                "Kristina Toutanova"
            ],
            "abstract": "We present a method to represent input texts by contextualizing them jointly\nwith dynamically retrieved textual encyclopedic background knowledge from\nmultiple documents. We apply our method to reading comprehension tasks by\nencoding questions and passages together with background sentences about the\nentities they mention. We show that integrating background knowledge from text\nis effective for tasks focusing on factual reasoning and allows direct reuse of\npowerful pretrained BERT-style encoders. Moreover, knowledge integration can be\nfurther improved with suitable pretraining via a self-supervised masked\nlanguage model objective over words in background-augmented input text. On\nTriviaQA, our approach obtains improvements of 1.6 to 3.1 F1 over comparable\nRoBERTa models which do not integrate background knowledge dynamically. On\nMRQA, a large collection of diverse QA datasets, we see consistent gains\nin-domain along with large improvements out-of-domain on BioASQ (2.1 to 4.2\nF1), TextbookQA (1.6 to 2.0 F1), and DuoRC (1.1 to 2.0 F1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12043v1",
            "date": "2020-04-25 02:42:12+00:00",
            "title": "When do Word Embeddings Accurately Reflect Surveys on our Beliefs About People?",
            "authors": [
                "Kenneth Joseph",
                "Jonathan H. Morgan"
            ],
            "abstract": "Social biases are encoded in word embeddings. This presents a unique\nopportunity to study society historically and at scale, and a unique danger\nwhen embeddings are used in downstream applications. Here, we investigate the\nextent to which publicly-available word embeddings accurately reflect beliefs\nabout certain kinds of people as measured via traditional survey methods. We\nfind that biases found in word embeddings do, on average, closely mirror survey\ndata across seventeen dimensions of social meaning. However, we also find that\nbiases in embeddings are much more reflective of survey data for some\ndimensions of meaning (e.g. gender) than others (e.g. race), and that we can be\nhighly confident that embedding-based measures reflect survey data only for the\nmost salient biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12057v1",
            "date": "2020-04-25 04:53:54+00:00",
            "title": "A Heterogeneous Graph with Factual, Temporal and Logical Knowledge for Question Answering Over Dynamic Contexts",
            "authors": [
                "Wanjun Zhong",
                "Duyu Tang",
                "Nan Duan",
                "Ming Zhou",
                "Jiahai Wang",
                "Jian Yin"
            ],
            "abstract": "We study question answering over a dynamic textual environment. Although\nneural network models achieve impressive accuracy via learning from\ninput-output examples, they rarely leverage various types of knowledge and are\ngenerally not interpretable. In this work, we propose a graph-based approach,\nwhere a heterogeneous graph is automatically built with factual knowledge of\nthe context, temporal knowledge of the past states, and logical knowledge that\ncombines human-curated knowledge bases and rule bases. We develop a graph\nneural network over the constructed graph, and train the model in an end-to-end\nmanner. Experimental results on a benchmark dataset show that the injection of\nvarious types of knowledge improves a strong neural network baseline. An\nadditional benefit of our approach is that the graph itself naturally serves as\na rational behind the decision making.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12126v2",
            "date": "2020-04-25 12:30:16+00:00",
            "title": "A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land?",
            "authors": [
                "Hongyu Lin",
                "Yaojie Lu",
                "Jialong Tang",
                "Xianpei Han",
                "Le Sun",
                "Zhicheng Wei",
                "Nicholas Jing Yuan"
            ],
            "abstract": "Fine-tuning pretrained model has achieved promising performance on standard\nNER benchmarks. Generally, these benchmarks are blessed with strong name\nregularity, high mention coverage and sufficient context diversity.\nUnfortunately, when scaling NER to open situations, these advantages may no\nlonger exist. And therefore it raises a critical question of whether previous\ncreditable approaches can still work well when facing these challenges. As\nthere is no currently available dataset to investigate this problem, this paper\nproposes to conduct randomization test on standard benchmarks. Specifically, we\nerase name regularity, mention coverage and context diversity respectively from\nthe benchmarks, in order to explore their impact on the generalization ability\nof models. To further verify our conclusions, we also construct a new open NER\ndataset that focuses on entity types with weaker name regularity and lower\nmention coverage to verify our conclusion. From both randomization test and\nempirical experiments, we draw the conclusions that 1) name regularity is\ncritical for the models to generalize to unseen mentions; 2) high mention\ncoverage may undermine the model generalization ability and 3) context patterns\nmay not require enormous data to capture when using pretrained encoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12158v5",
            "date": "2020-04-25 14:45:15+00:00",
            "title": "How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence",
            "authors": [
                "Haoxi Zhong",
                "Chaojun Xiao",
                "Cunchao Tu",
                "Tianyang Zhang",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Legal Artificial Intelligence (LegalAI) focuses on applying the technology of\nartificial intelligence, especially natural language processing, to benefit\ntasks in the legal domain. In recent years, LegalAI has drawn increasing\nattention rapidly from both AI researchers and legal professionals, as LegalAI\nis beneficial to the legal system for liberating legal professionals from a\nmaze of paperwork. Legal professionals often think about how to solve tasks\nfrom rule-based and symbol-based methods, while NLP researchers concentrate\nmore on data-driven and embedding methods. In this paper, we introduce the\nhistory, the current state, and the future directions of research in LegalAI.\nWe illustrate the tasks from the perspectives of legal professionals and NLP\nresearchers and show several representative applications in LegalAI. We conduct\nexperiments and provide an in-depth analysis of the advantages and\ndisadvantages of existing works to explore possible future directions. You can\nfind the implementation of our work from https://github.com/thunlp/CLAIM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12190v1",
            "date": "2020-04-25 17:09:56+00:00",
            "title": "Towards Discourse Parsing-inspired Semantic Storytelling",
            "authors": [
                "Georg Rehm",
                "Karolina Zaczynska",
                "Juli\u00e1n Moreno-Schneider",
                "Malte Ostendorff",
                "Peter Bourgonje",
                "Maria Berger",
                "Jens Rauenbusch",
                "Andr\u00e9 Schmidt",
                "Mikka Wild"
            ],
            "abstract": "Previous work of ours on Semantic Storytelling uses text analytics procedures\nincluding Named Entity Recognition and Event Detection. In this paper, we\noutline our longer-term vision on Semantic Storytelling and describe the\ncurrent conceptual and technical approach. In the project that drives our\nresearch we develop AI-based technologies that are verified by partners from\nindustry. One long-term goal is the development of an approach for Semantic\nStorytelling that has broad coverage and that is, furthermore, robust. We\nprovide first results on experiments that involve discourse parsing, applied to\na concrete use case, \"Explore the Neighbourhood!\", which is based on a\nsemi-automatically collected data set with documents about noteworthy people in\none of Berlin's districts. Though automatically obtaining annotations for\ncoherence relations from plain text is a non-trivial challenge, our preliminary\nresults are promising. We envision our approach to be combined with additional\nfeatures (NER, coreference resolution, knowledge graphs",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12198v2",
            "date": "2020-04-25 17:49:37+00:00",
            "title": "Quantifying the Contextualization of Word Representations with Semantic Class Probing",
            "authors": [
                "Mengjie Zhao",
                "Philipp Dufter",
                "Yadollah Yaghoobzadeh",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Pretrained language models have achieved a new state of the art on many NLP\ntasks, but there are still many open questions about how and why they work so\nwell. We investigate the contextualization of words in BERT. We quantify the\namount of contextualization, i.e., how well words are interpreted in context,\nby studying the extent to which semantic classes of a word can be inferred from\nits contextualized embeddings. Quantifying contextualization helps in\nunderstanding and utilizing pretrained language models. We show that top layer\nrepresentations achieve high accuracy inferring semantic classes; that the\nstrongest contextualization effects occur in the lower layers; that local\ncontext is mostly sufficient for semantic class inference; and that top layer\nrepresentations are more task-specific after finetuning while lower layer\nrepresentations are more transferable. Finetuning uncovers task related\nfeatures, but pretrained knowledge is still largely preserved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12238v1",
            "date": "2020-04-25 21:37:12+00:00",
            "title": "MCQA: Multimodal Co-attention Based Network for Question Answering",
            "authors": [
                "Abhishek Kumar",
                "Trisha Mittal",
                "Dinesh Manocha"
            ],
            "abstract": "We present MCQA, a learning-based algorithm for multimodal question\nanswering. MCQA explicitly fuses and aligns the multimodal input (i.e. text,\naudio, and video), which forms the context for the query (question and answer).\nOur approach fuses and aligns the question and the answer within this context.\nMoreover, we use the notion of co-attention to perform cross-modal alignment\nand multimodal context-query alignment. Our context-query alignment module\nmatches the relevant parts of the multimodal context and the query with each\nother and aligns them to improve the overall performance. We evaluate the\nperformance of MCQA on Social-IQ, a benchmark dataset for multimodal question\nanswering. We compare the performance of our algorithm with prior methods and\nobserve an accuracy improvement of 4-7%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12265v2",
            "date": "2020-04-26 01:53:03+00:00",
            "title": "Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias",
            "authors": [
                "Jesse Vig",
                "Sebastian Gehrmann",
                "Yonatan Belinkov",
                "Sharon Qian",
                "Daniel Nevo",
                "Simas Sakenis",
                "Jason Huang",
                "Yaron Singer",
                "Stuart Shieber"
            ],
            "abstract": "Common methods for interpreting neural models in natural language processing\ntypically examine either their structure or their behavior, but not both. We\npropose a methodology grounded in the theory of causal mediation analysis for\ninterpreting which parts of a model are causally implicated in its behavior. It\nenables us to analyze the mechanisms by which information flows from input to\noutput through various model components, known as mediators. We apply this\nmethodology to analyze gender bias in pre-trained Transformer language models.\nWe study the role of individual neurons and attention heads in mediating gender\nbias across three datasets designed to gauge a model's sensitivity to gender\nbias. Our mediation analysis reveals that gender bias effects are (i) sparse,\nconcentrated in a small part of the network; (ii) synergistic, amplified or\nrepressed by different components; and (iii) decomposable into effects flowing\ndirectly from the input and indirectly through the mediators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12376v2",
            "date": "2020-04-26 13:28:34+00:00",
            "title": "GLUECoS : An Evaluation Benchmark for Code-Switched NLP",
            "authors": [
                "Simran Khanuja",
                "Sandipan Dandapat",
                "Anirudh Srinivasan",
                "Sunayana Sitaram",
                "Monojit Choudhury"
            ],
            "abstract": "Code-switching is the use of more than one language in the same conversation\nor utterance. Recently, multilingual contextual embedding models, trained on\nmultiple monolingual corpora, have shown promising results on cross-lingual and\nmultilingual tasks. We present an evaluation benchmark, GLUECoS, for\ncode-switched languages, that spans several NLP tasks in English-Hindi and\nEnglish-Spanish. Specifically, our evaluation benchmark includes Language\nIdentification from text, POS tagging, Named Entity Recognition, Sentiment\nAnalysis, Question Answering and a new task for code-switching, Natural\nLanguage Inference. We present results on all these tasks using cross-lingual\nword embedding models and multilingual models. In addition, we fine-tune\nmultilingual models on artificially generated code-switched data. Although\nmultilingual models perform significantly better than cross-lingual models, our\nresults show that in most tasks, across both language pairs, multilingual\nmodels fine-tuned on code-switched data perform best, showing that multilingual\nmodels can be further optimized for code-switching tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12393v1",
            "date": "2020-04-26 14:38:11+00:00",
            "title": "Heterogeneous Graph Neural Networks for Extractive Document Summarization",
            "authors": [
                "Danqing Wang",
                "Pengfei Liu",
                "Yining Zheng",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "As a crucial step in extractive document summarization, learning\ncross-sentence relations has been explored by a plethora of approaches. An\nintuitive way is to put them in the graph-based neural network, which has a\nmore complex structure for capturing inter-sentence relationships. In this\npaper, we present a heterogeneous graph-based neural network for extractive\nsummarization (HeterSumGraph), which contains semantic nodes of different\ngranularity levels apart from sentences. These additional nodes act as the\nintermediary between sentences and enrich the cross-sentence relations.\nBesides, our graph structure is flexible in natural extension from a\nsingle-document setting to multi-document via introducing document nodes. To\nour knowledge, we are the first one to introduce different types of nodes into\ngraph-based neural networks for extractive document summarization and perform a\ncomprehensive qualitative analysis to investigate their benefits. The code will\nbe released on Github",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12406v2",
            "date": "2020-04-26 15:03:47+00:00",
            "title": "Masking as an Efficient Alternative to Finetuning for Pretrained Language Models",
            "authors": [
                "Mengjie Zhao",
                "Tao Lin",
                "Fei Mi",
                "Martin Jaggi",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present an efficient method of utilizing pretrained language models, where\nwe learn selective binary masks for pretrained weights in lieu of modifying\nthem through finetuning. Extensive evaluations of masking BERT and RoBERTa on a\nseries of NLP tasks show that our masking scheme yields performance comparable\nto finetuning, yet has a much smaller memory footprint when several tasks need\nto be inferred simultaneously. Through intrinsic evaluations, we show that\nrepresentations computed by masked language models encode information necessary\nfor solving downstream tasks. Analyzing the loss landscape, we show that\nmasking and finetuning produce models that reside in minima that can be\nconnected by a line segment with nearly constant test accuracy. This confirms\nthat masking can be utilized as an efficient alternative to finetuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12429v1",
            "date": "2020-04-26 16:29:06+00:00",
            "title": "Towards Multimodal Response Generation with Exemplar Augmentation and Curriculum Optimization",
            "authors": [
                "Zeyang Lei",
                "Zekang Li",
                "Jinchao Zhang",
                "Fandong Meng",
                "Yang Feng",
                "Yujiu Yang",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Recently, variational auto-encoder (VAE) based approaches have made\nimpressive progress on improving the diversity of generated responses. However,\nthese methods usually suffer the cost of decreased relevance accompanied by\ndiversity improvements. In this paper, we propose a novel multimodal response\ngeneration framework with exemplar augmentation and curriculum optimization to\nenhance relevance and diversity of generated responses. First, unlike existing\nVAE-based models that usually approximate a simple Gaussian posterior\ndistribution, we present a Gaussian mixture posterior distribution (i.e,\nmultimodal) to further boost response diversity, which helps capture complex\nsemantics of responses. Then, to ensure that relevance does not decrease while\ndiversity increases, we fully exploit similar examples (exemplars) retrieved\nfrom the training data into posterior distribution modeling to augment response\nrelevance. Furthermore, to facilitate the convergence of Gaussian mixture prior\nand posterior distributions, we devise a curriculum optimization strategy to\nprogressively train the model under multiple training criteria from easy to\nhard. Experimental results on widely used SwitchBoard and DailyDialog datasets\ndemonstrate that our model achieves significant improvements compared to strong\nbaselines in terms of diversity and relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12440v2",
            "date": "2020-04-26 17:22:09+00:00",
            "title": "Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language",
            "authors": [
                "Qianhui Wu",
                "Zijia Lin",
                "B\u00f6rje F. Karlsson",
                "Jian-Guang Lou",
                "Biqing Huang"
            ],
            "abstract": "To better tackle the named entity recognition (NER) problem on languages with\nlittle/no labeled data, cross-lingual NER must effectively leverage knowledge\nlearned from source languages with rich labeled data. Previous works on\ncross-lingual NER are mostly based on label projection with pairwise texts or\ndirect model transfer. However, such methods either are not applicable if the\nlabeled data in the source languages is unavailable, or do not leverage\ninformation contained in unlabeled data in the target language. In this paper,\nwe propose a teacher-student learning method to address such limitations, where\nNER models in the source languages are used as teachers to train a student\nmodel on unlabeled data in the target language. The proposed method works for\nboth single-source and multi-source cross-lingual NER. For the latter, we\nfurther propose a similarity measuring method to better weight the supervision\nfrom different teacher models. Extensive experiments for 3 target languages on\nbenchmark datasets well demonstrate that our method outperforms existing\nstate-of-the-art methods for both single-source and multi-source cross-lingual\nNER.",
            "journal ref": "In ACL, pages 6505-6514, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12450v1",
            "date": "2020-04-26 18:29:31+00:00",
            "title": "Semi-Supervised Neural System for Tagging, Parsing and Lematization",
            "authors": [
                "Piotr Rybak",
                "Alina Wr\u00f3blewska"
            ],
            "abstract": "This paper describes the ICS PAS system which took part in CoNLL 2018 shared\ntask on Multilingual Parsing from Raw Text to Universal Dependencies. The\nsystem consists of jointly trained tagger, lemmatizer, and dependency parser\nwhich are based on features extracted by a biLSTM network. The system uses both\nfully connected and dilated convolutional neural architectures. The novelty of\nour approach is the use of an additional loss function, which reduces the\nnumber of cycles in the predicted dependency graphs, and the use of\nself-training to increase the system performance. The proposed system, i.e. ICS\nPAS (Warszawa), ranked 3th/4th in the official evaluation obtaining the\nfollowing overall results: 73.02 (LAS), 60.25 (MLAS) and 64.44 (BLEX).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12495v1",
            "date": "2020-04-26 22:47:29+00:00",
            "title": "Experiments with LVT and FRE for Transformer model",
            "authors": [
                "Ilshat Gibadullin",
                "Aidar Valeev"
            ],
            "abstract": "In this paper, we experiment with Large Vocabulary Trick and Feature-rich\nencoding applied to the Transformer model for Text Summarization. We could not\nachieve better results, than the analogous RNN-based sequence-to-sequence\nmodel, so we tried more models to find out, what improves the results and what\ndeteriorates them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12502v1",
            "date": "2020-04-26 23:22:41+00:00",
            "title": "PTPARL-D: Annotated Corpus of 44 years of Portuguese Parliament debates",
            "authors": [
                "Paulo Almeida",
                "Manuel Marques-Pita",
                "Joana Gon\u00e7alves-S\u00e1"
            ],
            "abstract": "In a representative democracy, some decide in the name of the rest, and these\nelected officials are commonly gathered in public assemblies, such as\nparliaments, where they discuss policies, legislate, and vote on fundamental\ninitiatives. A core aspect of such democratic processes are the plenary\ndebates, where important public discussions take place. Many parliaments around\nthe world are increasingly keeping the transcripts of such debates, and other\nparliamentary data, in digital formats accessible to the public, increasing\ntransparency and accountability. Furthermore, some parliaments are bringing old\npaper transcripts to semi-structured digital formats. However, these records\nare often only provided as raw text or even as images, with little to no\nannotation, and inconsistent formats, making them difficult to analyze and\nstudy, reducing both transparency and public reach. Here, we present PTPARL-D,\nan annotated corpus of debates in the Portuguese Parliament, from 1976 to 2019,\ncovering the entire period of Portuguese democracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12506v3",
            "date": "2020-04-26 23:29:27+00:00",
            "title": "Assessing Discourse Relations in Language Generation from GPT-2",
            "authors": [
                "Wei-Jen Ko",
                "Junyi Jessy Li"
            ],
            "abstract": "Recent advances in NLP have been attributed to the emergence of large-scale\npre-trained language models. GPT-2, in particular, is suited for generation\ntasks given its left-to-right language modeling objective, yet the linguistic\nquality of its generated text has largely remain unexplored. Our work takes a\nstep in understanding GPT-2's outputs in terms of discourse coherence. We\nperform a comprehensive study on the validity of explicit discourse relations\nin GPT-2's outputs under both organic generation and fine-tuned scenarios.\nResults show GPT-2 does not always generate text containing valid discourse\nrelations; nevertheless, its text is more aligned with human expectation in the\nfine-tuned scenario. We propose a decoupled strategy to mitigate these problems\nand highlight the importance of explicitly modeling discourse information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12617v2",
            "date": "2020-04-27 07:41:02+00:00",
            "title": "On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification",
            "authors": [
                "Xin Liu",
                "Jiefu Ou",
                "Yangqiu Song",
                "Xin Jiang"
            ],
            "abstract": "Implicit discourse relation classification is one of the most difficult parts\nin shallow discourse parsing as the relation prediction without explicit\nconnectives requires the language understanding at both the text span level and\nthe sentence level. Previous studies mainly focus on the interactions between\ntwo arguments. We argue that a powerful contextualized representation module, a\nbilateral multi-perspective matching module, and a global information fusion\nmodule are all important to implicit discourse analysis. We propose a novel\nmodel to combine these modules together. Extensive experiments show that our\nproposed model outperforms BERT and other state-of-the-art systems on the PDTB\ndataset by around 8% and CoNLL 2016 datasets around 16%. We also analyze the\neffectiveness of different modules in the implicit discourse relation\nclassification task and demonstrate how different levels of representation\nlearning can affect the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12651v1",
            "date": "2020-04-27 08:59:57+00:00",
            "title": "Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting",
            "authors": [
                "Sanyuan Chen",
                "Yutai Hou",
                "Yiming Cui",
                "Wanxiang Che",
                "Ting Liu",
                "Xiangzhan Yu"
            ],
            "abstract": "Deep pretrained language models have achieved great success in the way of\npretraining first and then fine-tuning. But such a sequential transfer learning\nparadigm often confronts the catastrophic forgetting problem and leads to\nsub-optimal performance. To fine-tune with less forgetting, we propose a recall\nand learn mechanism, which adopts the idea of multi-task learning and jointly\nlearns pretraining tasks and downstream tasks. Specifically, we propose a\nPretraining Simulation mechanism to recall the knowledge from pretraining tasks\nwithout data, and an Objective Shifting mechanism to focus the learning on\ndownstream tasks gradually. Experiments show that our method achieves\nstate-of-the-art performance on the GLUE benchmark. Our method also enables\nBERT-base to achieve better performance than directly fine-tuning of\nBERT-large. Further, we provide the open-source RecAdam optimizer, which\nintegrates the proposed mechanisms into Adam optimizer, to facility the NLP\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12681v1",
            "date": "2020-04-27 09:59:27+00:00",
            "title": "Lexically Constrained Neural Machine Translation with Levenshtein Transformer",
            "authors": [
                "Raymond Hendy Susanto",
                "Shamil Chollampatt",
                "Liling Tan"
            ],
            "abstract": "This paper proposes a simple and effective algorithm for incorporating\nlexical constraints in neural machine translation. Previous work either\nrequired re-training existing models with the lexical constraints or\nincorporating them during beam search decoding with significantly higher\ncomputational overheads. Leveraging the flexibility and speed of a recently\nproposed Levenshtein Transformer model (Gu et al., 2019), our method injects\nterminology constraints at inference time without any impact on decoding speed.\nOur method does not require any modification to the training procedure and can\nbe easily applied at runtime with custom dictionaries. Experiments on\nEnglish-German WMT datasets show that our approach improves an unconstrained\nbaseline and previous approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12704v1",
            "date": "2020-04-27 10:52:52+00:00",
            "title": "Semantic Graphs for Generating Deep Questions",
            "authors": [
                "Liangming Pan",
                "Yuxi Xie",
                "Yansong Feng",
                "Tat-Seng Chua",
                "Min-Yen Kan"
            ],
            "abstract": "This paper proposes the problem of Deep Question Generation (DQG), which aims\nto generate complex questions that require reasoning over multiple pieces of\ninformation of the input passage. In order to capture the global structure of\nthe document and facilitate reasoning, we propose a novel framework which first\nconstructs a semantic-level graph for the input document and then encodes the\nsemantic graph by introducing an attention-based GGNN (Att-GGNN). Afterwards,\nwe fuse the document-level and graph-level representations to perform joint\ntraining of content selection and question decoding. On the HotpotQA\ndeep-question centric dataset, our model greatly improves performance over\nquestions requiring reasoning over multiple facts, leading to state-of-the-art\nperformance. The code is publicly available at\nhttps://github.com/WING-NUS/SG-Deep-Question-Generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12727v1",
            "date": "2020-04-27 11:54:19+00:00",
            "title": "Screenplay Summarization Using Latent Narrative Structure",
            "authors": [
                "Pinelopi Papalampidi",
                "Frank Keller",
                "Lea Frermann",
                "Mirella Lapata"
            ],
            "abstract": "Most general-purpose extractive summarization models are trained on news\narticles, which are short and present all important information upfront. As a\nresult, such models are biased on position and often perform a smart selection\nof sentences from the beginning of the document. When summarizing long\nnarratives, which have complex structure and present information piecemeal,\nsimple position heuristics are not sufficient. In this paper, we propose to\nexplicitly incorporate the underlying structure of narratives into general\nunsupervised and supervised extractive summarization models. We formalize\nnarrative structure in terms of key narrative events (turning points) and treat\nit as latent in order to summarize screenplays (i.e., extract an optimal\nsequence of scenes). Experimental results on the CSI corpus of TV screenplays,\nwhich we augment with scene-level summarization labels, show that latent\nturning points correlate with important aspects of a CSI episode and improve\nsummarization performance over general extractive algorithms leading to more\ncomplete and diverse summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12744v2",
            "date": "2020-04-27 12:37:26+00:00",
            "title": "Augmenting Transformers with KNN-Based Composite Memory for Dialogue",
            "authors": [
                "Angela Fan",
                "Claire Gardent",
                "Chloe Braud",
                "Antoine Bordes"
            ],
            "abstract": "Various machine learning tasks can benefit from access to external\ninformation of different modalities, such as text and images. Recent work has\nfocused on learning architectures with large memories capable of storing this\nknowledge. We propose augmenting generative Transformer neural networks with\nKNN-based Information Fetching (KIF) modules. Each KIF module learns a read\noperation to access fixed external knowledge. We apply these modules to\ngenerative dialog modeling, a challenging task where information must be\nflexibly retrieved and incorporated to maintain the topic and flow of\nconversation. We demonstrate the effectiveness of our approach by identifying\nrelevant knowledge required for knowledgeable but engaging dialog from\nWikipedia, images, and human-written dialog utterances, and show that\nleveraging this retrieved information improves model performance, measured by\nautomatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12752v2",
            "date": "2020-04-27 12:52:20+00:00",
            "title": "The Gutenberg Dialogue Dataset",
            "authors": [
                "Richard Csaky",
                "Gabor Recski"
            ],
            "abstract": "Large datasets are essential for neural modeling of many NLP tasks. Current\npublicly available open-domain dialogue datasets offer a trade-off between\nquality (e.g., DailyDialog) and size (e.g., Opensubtitles). We narrow this gap\nby building a high-quality dataset of 14.8M utterances in English, and smaller\ndatasets in German, Dutch, Spanish, Portuguese, Italian, and Hungarian. We\nextract and process dialogues from public-domain books made available by\nProject Gutenberg. We describe our dialogue extraction pipeline, analyze the\neffects of the various heuristics used, and present an error analysis of\nextracted dialogues. Finally, we conduct experiments showing that better\nresponse quality can be achieved in zero-shot and finetuning settings by\ntraining on our data than on the larger but much noisier Opensubtitles dataset.\nOur open-source pipeline (https://github.com/ricsinaruto/gutenberg-dialog) can\nbe extended to further languages with little additional effort. Researchers can\nalso build their versions of existing datasets by adjusting various trade-off\nparameters. We also built a web demo for interacting with our models:\nhttps://ricsinaruto.github.io/chatbot.html.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12864v1",
            "date": "2020-04-27 15:18:49+00:00",
            "title": "DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking",
            "authors": [
                "Christopher Hidey",
                "Tuhin Chakrabarty",
                "Tariq Alhindi",
                "Siddharth Varia",
                "Kriste Krstovski",
                "Mona Diab",
                "Smaranda Muresan"
            ],
            "abstract": "The increased focus on misinformation has spurred development of data and\nsystems for detecting the veracity of a claim as well as retrieving\nauthoritative evidence. The Fact Extraction and VERification (FEVER) dataset\nprovides such a resource for evaluating end-to-end fact-checking, requiring\nretrieval of evidence from Wikipedia to validate a veracity prediction. We show\nthat current systems for FEVER are vulnerable to three categories of realistic\nchallenges for fact-checking -- multiple propositions, temporal reasoning, and\nambiguity and lexical variation -- and introduce a resource with these types of\nclaims. Then we present a system designed to be resilient to these \"attacks\"\nusing multiple pointer networks for document selection and jointly modeling a\nsequence of evidence sentences and veracity relation predictions. We find that\nin handling these attacks we obtain state-of-the-art results on FEVER, largely\ndue to improved evidence retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12894v1",
            "date": "2020-04-27 15:54:29+00:00",
            "title": "Intelligent Translation Memory Matching and Retrieval with Sentence Encoders",
            "authors": [
                "Tharindu Ranasinghe",
                "Constantin Orasan",
                "Ruslan Mitkov"
            ],
            "abstract": "Matching and retrieving previously translated segments from a Translation\nMemory is the key functionality in Translation Memories systems. However this\nmatching and retrieving process is still limited to algorithms based on edit\ndistance which we have identified as a major drawback in Translation Memories\nsystems. In this paper we introduce sentence encoders to improve the matching\nand retrieving process in Translation Memories systems - an effective and\nefficient solution to replace edit distance based algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12934v1",
            "date": "2020-04-27 16:48:54+00:00",
            "title": "SCDE: Sentence Cloze Dataset with High Quality Distractors From Examinations",
            "authors": [
                "Xiang Kong",
                "Varun Gangal",
                "Eduard Hovy"
            ],
            "abstract": "We introduce SCDE, a dataset to evaluate the performance of computational\nmodels through sentence prediction. SCDE is a human-created sentence cloze\ndataset, collected from public school English examinations. Our task requires a\nmodel to fill up multiple blanks in a passage from a shared candidate set with\ndistractors designed by English teachers. Experimental results demonstrate that\nthis task requires the use of non-local, discourse-level context beyond the\nimmediate sentence neighborhood. The blanks require joint solving and\nsignificantly impair each other's context. Furthermore, through ablations, we\nshow that the distractors are of high quality and make the task more\nchallenging. Our experiments show that there is a significant performance gap\nbetween advanced models (72%) and humans (87%), encouraging future models to\nbridge this gap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12935v2",
            "date": "2020-04-27 16:51:21+00:00",
            "title": "Natural language processing for achieving sustainable development: the case of neural labelling to enhance community profiling",
            "authors": [
                "Costanza Conforti",
                "Stephanie Hirmer",
                "David Morgan",
                "Marco Basaldella",
                "Yau Ben Or"
            ],
            "abstract": "In recent years, there has been an increasing interest in the application of\nArtificial Intelligence - and especially Machine Learning - to the field of\nSustainable Development (SD). However, until now, NLP has not been applied in\nthis context. In this research paper, we show the high potential of NLP\napplications to enhance the sustainability of projects. In particular, we focus\non the case of community profiling in developing countries, where, in contrast\nto the developed world, a notable data gap exists. In this context, NLP could\nhelp to address the cost and time barrier of structuring qualitative data that\nprohibits its widespread use and associated benefits. We propose the new task\nof Automatic UPV classification, which is an extreme multi-class multi-label\nclassification problem. We release Stories2Insights, an expert-annotated\ndataset, provide a detailed corpus analysis, and implement a number of strong\nneural baselines to address the task. Experimental results show that the\nproblem is challenging, and leave plenty of room for future research at the\nintersection of NLP and SD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13150v2",
            "date": "2020-04-27 20:11:50+00:00",
            "title": "Octa: Omissions and Conflicts in Target-Aspect Sentiment Analysis",
            "authors": [
                "Zhe Zhang",
                "Chung-Wei Hang",
                "Munindar P. Singh"
            ],
            "abstract": "Sentiments in opinionated text are often determined by both aspects and\ntarget words (or targets). We observe that targets and aspects interrelate in\nsubtle ways, often yielding conflicting sentiments. Thus, a naive aggregation\nof sentiments from aspects and targets treated separately, as in existing\nsentiment analysis models, impairs performance.\n  We propose Octa, an approach that jointly considers aspects and targets when\ninferring sentiments. To capture and quantify relationships between targets and\ncontext words, Octa uses a selective self-attention mechanism that handles\nimplicit or missing targets. Specifically, Octa involves two layers of\nattention mechanisms for, respectively, selective attention between targets and\ncontext words and attention over words based on aspects. On benchmark datasets,\nOcta outperforms leading models by a large margin, yielding (absolute) gains in\naccuracy of 1.6% to 4.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13169v2",
            "date": "2020-04-27 20:56:20+00:00",
            "title": "Simultaneous Translation Policies: From Fixed to Adaptive",
            "authors": [
                "Baigong Zheng",
                "Kaibo Liu",
                "Renjie Zheng",
                "Mingbo Ma",
                "Hairong Liu",
                "Liang Huang"
            ],
            "abstract": "Adaptive policies are better than fixed policies for simultaneous\ntranslation, since they can flexibly balance the tradeoff between translation\nquality and latency based on the current context information. But previous\nmethods on obtaining adaptive policies either rely on complicated training\nprocess, or underperform simple fixed policies. We design an algorithm to\nachieve adaptive policies via a simple heuristic composition of a set of fixed\npolicies. Experiments on Chinese -> English and German -> English show that our\nadaptive policies can outperform fixed ones by up to 4 BLEU points for the same\nlatency, and more surprisingly, it even surpasses the BLEU score of\nfull-sentence translation in the greedy mode (and very close to beam mode), but\nwith much lower latency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13203v1",
            "date": "2020-04-27 22:55:55+00:00",
            "title": "A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization",
            "authors": [
                "Graham Neubig",
                "Shruti Rijhwani",
                "Alexis Palmer",
                "Jordan MacKenzie",
                "Hilaria Cruz",
                "Xinjian Li",
                "Matthew Lee",
                "Aditi Chaudhary",
                "Luke Gessler",
                "Steven Abney",
                "Shirley Anugrah Hayati",
                "Antonios Anastasopoulos",
                "Olga Zamaraeva",
                "Emily Prud'hommeaux",
                "Jennette Child",
                "Sara Child",
                "Rebecca Knowles",
                "Sarah Moeller",
                "Jeffrey Micher",
                "Yiyuan Li",
                "Sydney Zink",
                "Mengzhou Xia",
                "Roshan S Sharma",
                "Patrick Littell"
            ],
            "abstract": "Despite recent advances in natural language processing and other language\ntechnology, the application of such technology to language documentation and\nconservation has been limited. In August 2019, a workshop was held at Carnegie\nMellon University in Pittsburgh to attempt to bring together language community\nmembers, documentary linguists, and technologists to discuss how to bridge this\ngap and create prototypes of novel and practical language revitalization\ntechnologies. This paper reports the results of this workshop, including issues\ndiscussed, and various conceived and implemented technologies for nine\nlanguages: Arapaho, Cayuga, Inuktitut, Irish Gaelic, Kidaw'ida, Kwak'wala,\nOjibwe, San Juan Quiahije Chatino, and Seneca.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13221v1",
            "date": "2020-04-28 00:28:09+00:00",
            "title": "KoParadigm: A Korean Conjugation Paradigm Generator",
            "authors": [
                "Kyubyong Park"
            ],
            "abstract": "Korean is a morphologically rich language. Korean verbs change their forms in\na fickle manner depending on tense, mood, speech level, meaning, etc.\nTherefore, it is challenging to construct comprehensive conjugation paradigms\nof Korean verbs. In this paper we introduce a Korean (verb) conjugation\nparadigm generator, dubbed KoParadigm. To the best of our knowledge, it is the\nfirst Korean conjugation module that covers all contemporary Korean verbs and\nendings. KoParadigm is not only linguistically well established, but also\ncomputationally simple and efficient. We share it via PyPi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13249v1",
            "date": "2020-04-28 02:43:36+00:00",
            "title": "Conversational Word Embedding for Retrieval-Based Dialog System",
            "authors": [
                "Wentao Ma",
                "Yiming Cui",
                "Ting Liu",
                "Dong Wang",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Human conversations contain many types of information, e.g., knowledge,\ncommon sense, and language habits. In this paper, we propose a conversational\nword embedding method named PR-Embedding, which utilizes the conversation pairs\n$ \\left\\langle{post, reply} \\right\\rangle$ to learn word embedding. Different\nfrom previous works, PR-Embedding uses the vectors from two different semantic\nspaces to represent the words in post and reply. To catch the information among\nthe pair, we first introduce the word alignment model from statistical machine\ntranslation to generate the cross-sentence window, then train the embedding on\nword-level and sentence-level. We evaluate the method on single-turn and\nmulti-turn response selection tasks for retrieval-based dialog systems. The\nexperiment results show that PR-Embedding can improve the quality of the\nselected response. PR-Embedding source code is available at\nhttps://github.com/wtma/PR-Embedding",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13304v1",
            "date": "2020-04-28 05:13:17+00:00",
            "title": "Learning to Learn Morphological Inflection for Resource-Poor Languages",
            "authors": [
                "Katharina Kann",
                "Samuel R. Bowman",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose to cast the task of morphological inflection - mapping a lemma to\nan indicated inflected form - for resource-poor languages as a meta-learning\nproblem. Treating each language as a separate task, we use data from\nhigh-resource source languages to learn a set of model parameters that can\nserve as a strong initialization point for fine-tuning on a resource-poor\ntarget language. Experiments with two model architectures on 29 target\nlanguages from 3 families show that our suggested approach outperforms all\nbaselines. In particular, it obtains a 31.7% higher absolute accuracy than a\npreviously proposed cross-lingual transfer model and outperforms the previous\nstate of the art by 1.7% absolute accuracy on average over languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13305v1",
            "date": "2020-04-28 05:14:08+00:00",
            "title": "Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languages",
            "authors": [
                "Katharina Kann",
                "Oph\u00e9lie Lacroix",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Part-of-speech (POS) taggers for low-resource languages which are exclusively\nbased on various forms of weak supervision - e.g., cross-lingual transfer,\ntype-level supervision, or a combination thereof - have been reported to\nperform almost as well as supervised ones. However, weakly supervised POS\ntaggers are commonly only evaluated on languages that are very different from\ntruly low-resource languages, and the taggers use sources of information, like\nhigh-coverage and almost error-free dictionaries, which are likely not\navailable for resource-poor languages. We train and evaluate state-of-the-art\nweakly supervised POS taggers for a typologically diverse set of 15 truly\nlow-resource languages. On these languages, given a realistic amount of\nresources, even our best model gets only less than half of the words right. Our\nresults highlight the need for new and different approaches to POS tagging for\ntruly low-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13310v4",
            "date": "2020-04-28 05:23:43+00:00",
            "title": "Self-Attention with Cross-Lingual Position Representation",
            "authors": [
                "Liang Ding",
                "Longyue Wang",
                "Dacheng Tao"
            ],
            "abstract": "Position encoding (PE), an essential part of self-attention networks (SANs),\nis used to preserve the word order information for natural language processing\ntasks, generating fixed position indices for input sequences. However, in\ncross-lingual scenarios, e.g. machine translation, the PEs of source and target\nsentences are modeled independently. Due to word order divergences in different\nlanguages, modeling the cross-lingual positional relationships might help SANs\ntackle this problem. In this paper, we augment SANs with \\emph{cross-lingual\nposition representations} to model the bilingually aware latent structure for\nthe input sentence. Specifically, we utilize bracketing transduction grammar\n(BTG)-based reordering information to encourage SANs to learn bilingual\ndiagonal alignments. Experimental results on WMT'14 English$\\Rightarrow$German,\nWAT'17 Japanese$\\Rightarrow$English, and WMT'17 Chinese$\\Leftrightarrow$English\ntranslation tasks demonstrate that our approach significantly and consistently\nimproves translation quality over strong baselines. Extensive analyses confirm\nthat the performance gains come from the cross-lingual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13317v2",
            "date": "2020-04-28 06:06:18+00:00",
            "title": "Let's be Humorous: Knowledge Enhanced Humor Generation",
            "authors": [
                "Hang Zhang",
                "Dayiheng Liu",
                "Jiancheng Lv",
                "Cheng Luo"
            ],
            "abstract": "The generation of humor is an under-explored and challenging problem.\nPrevious works mainly utilize templates or replace phrases to generate humor.\nHowever, few works focus on freer forms and the background knowledge of humor.\nThe linguistic theory of humor defines the structure of a humor sentence as\nset-up and punchline. In this paper, we explore how to generate a punchline\ngiven the set-up with the relevant knowledge. We propose a framework that can\nfuse the knowledge to end-to-end models. To our knowledge, this is the first\nattempt to generate punchlines with knowledge enhanced model. Furthermore, we\ncreate the first humor-knowledge dataset. The experimental results demonstrate\nthat our method can make use of knowledge to generate fluent, funny punchlines,\nwhich outperforms several baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13338v1",
            "date": "2020-04-28 07:24:43+00:00",
            "title": "Semantics-Aware Inferential Network for Natural Language Understanding",
            "authors": [
                "Shuailiang Zhang",
                "Hai Zhao",
                "Junru Zhou"
            ],
            "abstract": "For natural language understanding tasks, either machine reading\ncomprehension or natural language inference, both semantics-aware and inference\nare favorable features of the concerned modeling for better understanding\nperformance. Thus we propose a Semantics-Aware Inferential Network (SAIN) to\nmeet such a motivation. Taking explicit contextualized semantics as a\ncomplementary input, the inferential module of SAIN enables a series of\nreasoning steps over semantic clues through an attention mechanism. By\nstringing these steps, the inferential network effectively learns to perform\niterative reasoning which incorporates both explicit semantics and\ncontextualized representations. In terms of well pre-trained language models as\nfront-end encoder, our model achieves significant improvement on 11 tasks\nincluding machine reading comprehension and natural language inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13454v1",
            "date": "2020-04-28 12:19:12+00:00",
            "title": "An Effective Transition-based Model for Discontinuous NER",
            "authors": [
                "Xiang Dai",
                "Sarvnaz Karimi",
                "Ben Hachey",
                "Cecile Paris"
            ],
            "abstract": "Unlike widely used Named Entity Recognition (NER) data sets in generic\ndomains, biomedical NER data sets often contain mentions consisting of\ndiscontinuous spans. Conventional sequence tagging techniques encode Markov\nassumptions that are efficient but preclude recovery of these mentions. We\npropose a simple, effective transition-based model with generic neural encoding\nfor discontinuous NER. Through extensive experiments on three biomedical data\nsets, we show that our model can effectively recognize discontinuous mentions\nwithout sacrificing the accuracy on continuous mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13542v4",
            "date": "2020-04-27 15:08:10+00:00",
            "title": "Faster Depth-Adaptive Transformers",
            "authors": [
                "Yijin Liu",
                "Fandong Meng",
                "Jie Zhou",
                "Yufeng Chen",
                "Jinan Xu"
            ],
            "abstract": "Depth-adaptive neural networks can dynamically adjust depths according to the\nhardness of input words, and thus improve efficiency. The main challenge is how\nto measure such hardness and decide the required depths (i.e., layers) to\nconduct. Previous works generally build a halting unit to decide whether the\ncomputation should continue or stop at each layer. As there is no specific\nsupervision of depth selection, the halting unit may be under-optimized and\ninaccurate, which results in suboptimal and unstable performance when modeling\nsentences. In this paper, we get rid of the halting unit and estimate the\nrequired depths in advance, which yields a faster depth-adaptive model.\nSpecifically, two approaches are proposed to explicitly measure the hardness of\ninput words and estimate corresponding adaptive depth, namely 1) mutual\ninformation (MI) based estimation and 2) reconstruction loss based estimation.\nWe conduct experiments on the text classification task with 24 datasets in\nvarious sizes and domains. Results confirm that our approaches can speed up the\nvanilla Transformer (up to 7x) while preserving high accuracy. Moreover,\nefficiency and robustness are significantly improved when compared with other\ndepth-adaptive approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13580v1",
            "date": "2020-04-28 15:09:51+00:00",
            "title": "Embarrassingly Simple Unsupervised Aspect Extraction",
            "authors": [
                "St\u00e9phan Tulkens",
                "Andreas van Cranenburgh"
            ],
            "abstract": "We present a simple but effective method for aspect identification in\nsentiment analysis. Our unsupervised method only requires word embeddings and a\nPOS tagger, and is therefore straightforward to apply to new domains and\nlanguages. We introduce Contrastive Attention (CAt), a novel single-head\nattention mechanism based on an RBF kernel, which gives a considerable boost in\nperformance and makes the model interpretable. Previous work relied on\nsyntactic features and complex neural models. We show that given the simplicity\nof current benchmark datasets for aspect extraction, such complex models are\nnot needed. The code to reproduce the experiments reported in this paper is\navailable at https://github.com/clips/cat",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13590v2",
            "date": "2020-04-28 15:25:19+00:00",
            "title": "MAVEN: A Massive General Domain Event Detection Dataset",
            "authors": [
                "Xiaozhi Wang",
                "Ziqi Wang",
                "Xu Han",
                "Wangyi Jiang",
                "Rong Han",
                "Zhiyuan Liu",
                "Juanzi Li",
                "Peng Li",
                "Yankai Lin",
                "Jie Zhou"
            ],
            "abstract": "Event detection (ED), which means identifying event trigger words and\nclassifying event types, is the first and most fundamental step for extracting\nevent knowledge from plain text. Most existing datasets exhibit the following\nissues that limit further development of ED: (1) Data scarcity. Existing\nsmall-scale datasets are not sufficient for training and stably benchmarking\nincreasingly sophisticated modern neural methods. (2) Low coverage. Limited\nevent types of existing datasets cannot well cover general-domain events, which\nrestricts the applications of ED models. To alleviate these problems, we\npresent a MAssive eVENt detection dataset (MAVEN), which contains 4,480\nWikipedia documents, 118,732 event mention instances, and 168 event types.\nMAVEN alleviates the data scarcity problem and covers much more general event\ntypes. We reproduce the recent state-of-the-art ED models and conduct a\nthorough evaluation on MAVEN. The experimental results show that existing ED\nmethods cannot achieve promising results on MAVEN as on the small datasets,\nwhich suggests that ED in the real world remains a challenging task and\nrequires further research efforts. We also discuss further directions for\ngeneral domain ED with empirical analyses. The source code and dataset can be\nobtained from https://github.com/THU-KEG/MAVEN-dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13625v2",
            "date": "2020-04-28 16:15:46+00:00",
            "title": "Event Extraction by Answering (Almost) Natural Questions",
            "authors": [
                "Xinya Du",
                "Claire Cardie"
            ],
            "abstract": "The problem of event extraction requires detecting the event trigger and\nextracting its corresponding arguments. Existing work in event argument\nextraction typically relies heavily on entity recognition as a\npreprocessing/concurrent step, causing the well-known problem of error\npropagation. To avoid this issue, we introduce a new paradigm for event\nextraction by formulating it as a question answering (QA) task that extracts\nthe event arguments in an end-to-end manner. Empirical results demonstrate that\nour framework outperforms prior methods substantially; in addition, it is\ncapable of extracting event arguments for roles not seen at training time\n(zero-shot learning setting).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13639v1",
            "date": "2020-04-28 16:34:35+00:00",
            "title": "Joint Keyphrase Chunking and Salience Ranking with BERT",
            "authors": [
                "Si Sun",
                "Chenyan Xiong",
                "Zhenghao Liu",
                "Zhiyuan Liu",
                "Jie Bao"
            ],
            "abstract": "An effective keyphrase extraction system requires to produce self-contained\nhigh quality phrases that are also key to the document topic. This paper\npresents BERT-JointKPE, a multi-task BERT-based model for keyphrase extraction.\nJointKPE employs a chunking network to identify high-quality phrases and a\nranking network to learn their salience in the document. The model is trained\njointly on the chunking task and the ranking task, balancing the estimation of\nkeyphrase quality and salience. Experiments on two benchmarks demonstrate\nJointKPE's robust effectiveness with different BERT variants. Our analyses show\nthat JointKPE has advantages in predicting long keyphrases and extracting\nphrases that are not entities but also meaningful. The source code of this\npaper can be obtained from https://github.com/thunlp/BERT-KPE",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13640v1",
            "date": "2020-04-28 16:36:41+00:00",
            "title": "Extending Multilingual BERT to Low-Resource Languages",
            "authors": [
                "Zihan Wang",
                "Karthikeyan K",
                "Stephen Mayhew",
                "Dan Roth"
            ],
            "abstract": "Multilingual BERT (M-BERT) has been a huge success in both supervised and\nzero-shot cross-lingual transfer learning. However, this success has focused\nonly on the top 104 languages in Wikipedia that it was trained on. In this\npaper, we propose a simple but effective approach to extend M-BERT (E-BERT) so\nthat it can benefit any new language, and show that our approach benefits\nlanguages that are already in M-BERT as well. We perform an extensive set of\nexperiments with Named Entity Recognition (NER) on 27 languages, only 16 of\nwhich are in M-BERT, and show an average increase of about 6% F1 on languages\nthat are already in M-BERT and 23% F1 increase on new languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13645v1",
            "date": "2020-04-28 16:41:00+00:00",
            "title": "Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Data",
            "authors": [
                "Alana Marzoev",
                "Samuel Madden",
                "M. Frans Kaashoek",
                "Michael Cafarella",
                "Jacob Andreas"
            ],
            "abstract": "Large, human-annotated datasets are central to the development of natural\nlanguage processing models. Collecting these datasets can be the most\nchallenging part of the development process. We address this problem by\nintroducing a general purpose technique for ``simulation-to-real'' transfer in\nlanguage understanding problems with a delimited set of target behaviors,\nmaking it possible to develop models that can interpret natural utterances\nwithout natural training data. We begin with a synthetic data generation\nprocedure, and train a model that can accurately interpret utterances produced\nby the data generator. To generalize to natural utterances, we automatically\nfind projections of natural language utterances onto the support of the\nsynthetic language, using learned sentence embeddings to define a distance\nmetric. With only synthetic training data, our approach matches or outperforms\nstate-of-the-art models trained on natural language data in several domains.\nThese results suggest that simulation-to-real transfer is a practical framework\nfor developing NLP applications, and that improved models for transfer might\nprovide wide-ranging improvements in downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13659v1",
            "date": "2020-04-28 17:04:19+00:00",
            "title": "LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network",
            "authors": [
                "Wanjun Zhong",
                "Duyu Tang",
                "Zhangyin Feng",
                "Nan Duan",
                "Ming Zhou",
                "Ming Gong",
                "Linjun Shou",
                "Daxin Jiang",
                "Jiahai Wang",
                "Jian Yin"
            ],
            "abstract": "Verifying the correctness of a textual statement requires not only semantic\nreasoning about the meaning of words, but also symbolic reasoning about logical\noperations like count, superlative, aggregation, etc. In this work, we propose\nLogicalFactChecker, a neural network approach capable of leveraging logical\noperations for fact checking. It achieves the state-of-the-art performance on\nTABFACT, a large-scale, benchmark dataset built for verifying a textual\nstatement with semi-structured tables. This is achieved by a graph module\nnetwork built upon the Transformer-based architecture. With a textual statement\nand a table as the input, LogicalFactChecker automatically derives a program\n(a.k.a. logical form) of the statement in a semantic parsing manner. A\nheterogeneous graph is then constructed to capture not only the structures of\nthe table and the program, but also the connections between inputs with\ndifferent modalities. Such a graph reveals the related contexts of each word in\nthe statement, the table and the program. The graph is used to obtain\ngraph-enhanced contextual representations of words in Transformer-based\narchitecture. After that, a program-driven module network is further introduced\nto exploit the hierarchical structure of the program, where semantic\ncompositionality is dynamically modeled along the program structure with a set\nof function-specific modules. Ablation experiments suggest that both the\nheterogeneous graph and the module network are important to obtain strong\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13671v3",
            "date": "2020-04-28 17:17:11+00:00",
            "title": "Active Learning for Coreference Resolution using Discrete Annotation",
            "authors": [
                "Belinda Z. Li",
                "Gabriel Stanovsky",
                "Luke Zettlemoyer"
            ],
            "abstract": "We improve upon pairwise annotation for active learning in coreference\nresolution, by asking annotators to identify mention antecedents if a presented\nmention pair is deemed not coreferent. This simple modification, when combined\nwith a novel mention clustering algorithm for selecting which examples to\nlabel, is much more efficient in terms of the performance obtained per\nannotation budget. In experiments with existing benchmark coreference datasets,\nwe show that the signal from this additional question leads to significant\nperformance gains per human-annotation hour. Future work can use our annotation\nprotocol to effectively develop coreference models for new domains. Our code is\npublicly available at\nhttps://github.com/belindal/discrete-active-learning-coref .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13703v1",
            "date": "2020-04-28 17:58:14+00:00",
            "title": "Autoencoding Word Representations through Time for Semantic Change Detection",
            "authors": [
                "Adam Tsakalidis",
                "Maria Liakata"
            ],
            "abstract": "Semantic change detection concerns the task of identifying words whose\nmeaning has changed over time. The current state-of-the-art detects the level\nof semantic change in a word by comparing its vector representation in two\ndistinct time periods, without considering its evolution through time. In this\nwork, we propose three variants of sequential models for detecting semantically\nshifted words, effectively accounting for the changes in the word\nrepresentations over time, in a temporally sensitive manner. Through extensive\nexperimentation under various settings with both synthetic and real data we\nshowcase the importance of sequential modelling of word vectors through time\nfor detecting the words whose semantics have changed the most. Finally, we take\na step towards comparing different approaches in a quantitative manner,\ndemonstrating that the temporal modelling of word representations yields a\nclear-cut advantage in performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13816v1",
            "date": "2020-04-28 21:07:32+00:00",
            "title": "DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis",
            "authors": [
                "Hu Xu",
                "Bing Liu",
                "Lei Shu",
                "Philip S. Yu"
            ],
            "abstract": "This paper focuses on learning domain-oriented language models driven by end\ntasks, which aims to combine the worlds of both general-purpose language models\n(such as ELMo and BERT) and domain-specific language understanding. We propose\nDomBERT, an extension of BERT to learn from both in-domain corpus and relevant\ndomain corpora. This helps in learning domain language models with\nlow-resources. Experiments are conducted on an assortment of tasks in\naspect-based sentiment analysis, demonstrating promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13819v1",
            "date": "2020-04-19 17:29:34+00:00",
            "title": "Neural Machine Translation for Low-Resourced Indian Languages",
            "authors": [
                "Himanshu Choudhary",
                "Shivansh Rao",
                "Rajesh Rohilla"
            ],
            "abstract": "A large number of significant assets are available online in English, which\nis frequently translated into native languages to ease the information sharing\namong local people who are not much familiar with English. However, manual\ntranslation is a very tedious, costly, and time-taking process. To this end,\nmachine translation is an effective approach to convert text to a different\nlanguage without any human involvement. Neural machine translation (NMT) is one\nof the most proficient translation techniques amongst all existing machine\ntranslation systems. In this paper, we have applied NMT on two of the most\nmorphological rich Indian languages, i.e. English-Tamil and English-Malayalam.\nWe proposed a novel NMT model using Multihead self-attention along with\npre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an\nefficient translation system that overcomes the OOV (Out Of Vocabulary) problem\nfor low resourced morphological rich Indian languages which do not have much\ntranslation available online. We also collected corpus from different sources,\naddressed the issues with these publicly available data and refined them for\nfurther uses. We used the BLEU score for evaluating our system performance.\nExperimental results and survey confirmed that our proposed translator (24.34\nand 9.78 BLEU score) outperforms Google translator (9.40 and 5.94 BLEU score)\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13826v2",
            "date": "2020-04-22 07:23:47+00:00",
            "title": "Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks",
            "authors": [
                "Yufeng Zhang",
                "Xueli Yu",
                "Zeyu Cui",
                "Shu Wu",
                "Zhongzhen Wen",
                "Liang Wang"
            ],
            "abstract": "Text classification is fundamental in natural language processing (NLP), and\nGraph Neural Networks (GNN) are recently applied in this task. However, the\nexisting graph-based works can neither capture the contextual word\nrelationships within each document nor fulfil the inductive learning of new\nwords. In this work, to overcome such problems, we propose TextING for\ninductive text classification via GNN. We first build individual graphs for\neach document and then use GNN to learn the fine-grained word representations\nbased on their local structures, which can also effectively produce embeddings\nfor unseen words in the new document. Finally, the word nodes are aggregated as\nthe document embedding. Extensive experiments on four benchmark datasets show\nthat our method outperforms state-of-the-art text classification methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13833v1",
            "date": "2020-04-24 09:27:09+00:00",
            "title": "Data Annealing for Informal Language Understanding Tasks",
            "authors": [
                "Jing Gu",
                "Zhou Yu"
            ],
            "abstract": "There is a huge performance gap between formal and informal language\nunderstanding tasks. The recent pre-trained models that improved the\nperformance of formal language understanding tasks did not achieve a comparable\nresult on informal language. We pro-pose a data annealing transfer learning\nprocedure to bridge the performance gap on informal natural language\nunderstanding tasks. It successfully utilizes a pre-trained model such as BERT\nin informal language. In our data annealing procedure, the training set\ncontains mainly formal text data at first; then, the proportion of the informal\ntext data is gradually increased during the training process. Our data\nannealing procedure is model-independent and can be applied to various tasks.\nWe validate its effectiveness in exhaustive experiments. When BERT is\nimplemented with our learning procedure, it outperforms all the\nstate-of-the-art models on the three common informal language tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13838v1",
            "date": "2020-04-28 21:14:38+00:00",
            "title": "How Chaotic Are Recurrent Neural Networks?",
            "authors": [
                "Pourya Vakilipourtakalou",
                "Lili Mou"
            ],
            "abstract": "Recurrent neural networks (RNNs) are non-linear dynamic systems. Previous\nwork believes that RNN may suffer from the phenomenon of chaos, where the\nsystem is sensitive to initial states and unpredictable in the long run. In\nthis paper, however, we perform a systematic empirical analysis, showing that a\nvanilla or long short term memory (LSTM) RNN does not exhibit chaotic behavior\nalong the training process in real applications such as text generation. Our\nfindings suggest that future work in this direction should address the other\nside of non-linear dynamics for RNN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13842v1",
            "date": "2020-03-30 18:03:15+00:00",
            "title": "Low resource language dataset creation, curation and classification: Setswana and Sepedi -- Extended Abstract",
            "authors": [
                "Vukosi Marivate",
                "Tshephisho Sefara",
                "Vongani Chabalala",
                "Keamogetswe Makhaya",
                "Tumisho Mokgonyane",
                "Rethabile Mokoena",
                "Abiodun Modupe"
            ],
            "abstract": "The recent advances in Natural Language Processing have only been a boon for\nwell represented languages, negating research in lesser known global languages.\nThis is in part due to the availability of curated data and research resources.\nOne of the current challenges concerning low-resourced languages are clear\nguidelines on the collection, curation and preparation of datasets for\ndifferent use-cases. In this work, we take on the task of creating two datasets\nthat are focused on news headlines (i.e short text) for Setswana and Sepedi and\nthe creation of a news topic classification task from these datasets. In this\nstudy, we document our work, propose baselines for classification, and\ninvestigate an approach on data augmentation better suited to low-resourced\nlanguages in order to improve the performance of the classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13844v2",
            "date": "2020-04-07 06:37:27+00:00",
            "title": "Decomposing Word Embedding with the Capsule Network",
            "authors": [
                "Xin Liu",
                "Qingcai Chen",
                "Yan Liu",
                "Joanna Siebert",
                "Baotian Hu",
                "Xiangping Wu",
                "Buzhou Tang"
            ],
            "abstract": "Word sense disambiguation tries to learn the appropriate sense of an\nambiguous word in a given context. The existing pre-trained language methods\nand the methods based on multi-embeddings of word did not explore the power of\nthe unsupervised word embedding sufficiently.\n  In this paper, we discuss a capsule network-based approach, taking advantage\nof capsule's potential for recognizing highly overlapping features and dealing\nwith segmentation. We propose a Capsule network-based method to Decompose the\nunsupervised word Embedding of an ambiguous word into context specific Sense\nembedding, called CapsDecE2S. In this approach, the unsupervised ambiguous\nembedding is fed into capsule network to produce its multiple morpheme-like\nvectors, which are defined as the basic semantic language units of meaning.\nWith attention operations, CapsDecE2S integrates the word context to\nreconstruct the multiple morpheme-like vectors into the context-specific sense\nembedding. To train CapsDecE2S, we propose a sense matching training method. In\nthis method, we convert the sense learning into a binary classification that\nexplicitly learns the relation between senses by the label of matching and\nnon-matching. The CapsDecE2S was experimentally evaluated on two sense learning\ntasks, i.e., word in context and word sense disambiguation. Results on two\npublic corpora Word-in-Context and English all-words Word Sense Disambiguation\nshow that, the CapsDecE2S model achieves the new state-of-the-art for the word\nin context and word sense disambiguation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13848v2",
            "date": "2020-04-10 09:32:07+00:00",
            "title": "A Natural Language Processing Pipeline of Chinese Free-text Radiology Reports for Liver Cancer Diagnosis",
            "authors": [
                "Honglei Liu",
                "Yan Xu",
                "Zhiqiang Zhang",
                "Ni Wang",
                "Yanqun Huang",
                "Yanjun Hu",
                "Zhenghan Yang",
                "Rui Jiang",
                "Hui Chen"
            ],
            "abstract": "Despite the rapid development of natural language processing (NLP)\nimplementation in electronic medical records (EMRs), Chinese EMRs processing\nremains challenging due to the limited corpus and specific grammatical\ncharacteristics, especially for radiology reports. In this study, we designed\nan NLP pipeline for the direct extraction of clinically relevant features from\nChinese radiology reports, which is the first key step in computer-aided\nradiologic diagnosis. The pipeline was comprised of named entity recognition,\nsynonyms normalization, and relationship extraction to finally derive the\nradiological features composed of one or more terms. In named entity\nrecognition, we incorporated lexicon into deep learning model bidirectional\nlong short-term memory-conditional random field (BiLSTM-CRF), and the model\nfinally achieved an F1 score of 93.00%. With the extracted radiological\nfeatures, least absolute shrinkage and selection operator and machine learning\nmethods (support vector machine, random forest, decision tree, and logistic\nregression) were used to build the classifiers for liver cancer prediction. For\nliver cancer diagnosis, random forest had the highest predictive performance in\nliver cancer diagnosis (F1 score 86.97%, precision 87.71%, and recall 86.25%).\nThis work was a comprehensive NLP study focusing on Chinese radiology reports\nand the application of NLP in cancer risk prediction. The proposed NLP pipeline\nfor the radiological feature extraction could be easily implemented in other\nkinds of Chinese clinical texts and other disease predictive tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13876v2",
            "date": "2020-04-28 22:27:19+00:00",
            "title": "The Explanation Game: Towards Prediction Explainability through Sparse Communication",
            "authors": [
                "Marcos V. Treviso",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Explainability is a topic of growing importance in NLP. In this work, we\nprovide a unified perspective of explainability as a communication problem\nbetween an explainer and a layperson about a classifier's decision. We use this\nframework to compare several prior approaches for extracting explanations,\nincluding gradient methods, representation erasure, and attention mechanisms,\nin terms of their communication success. In addition, we reinterpret these\nmethods at the light of classical feature selection, and we use this as\ninspiration to propose new embedded methods for explainability, through the use\nof selective, sparse attention. Experiments in text classification, natural\nlanguage entailment, and machine translation, using different configurations of\nexplainers and laypeople (including both machines and humans), reveal an\nadvantage of attention-based explainers over gradient and erasure methods.\nFurthermore, human evaluation experiments show promising results with post-hoc\nexplainers trained to optimize communication success and faithfulness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13878v2",
            "date": "2020-04-28 22:31:18+00:00",
            "title": "Analyzing Political Parody in Social Media",
            "authors": [
                "Antonis Maronikolakis",
                "Danae Sanchez Villegas",
                "Daniel Preotiuc-Pietro",
                "Nikolaos Aletras"
            ],
            "abstract": "Parody is a figurative device used to imitate an entity for comedic or\ncritical purposes and represents a widespread phenomenon in social media\nthrough many popular parody accounts. In this paper, we present the first\ncomputational study of parody. We introduce a new publicly available data set\nof tweets from real politicians and their corresponding parody accounts. We run\na battery of supervised machine learning models for automatically detecting\nparody tweets with an emphasis on robustness by testing on tweets from accounts\nunseen in training, across different genders and across countries. Our results\nshow that political parody tweets can be predicted with an accuracy up to 90%.\nFinally, we identify the markers of parody through a linguistic analysis.\nBeyond research in linguistics and political communication, accurately and\nautomatically detecting parody is important to improving fact checking for\njournalists and analytics such as sentiment analysis through filtering out\nparodical utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13886v2",
            "date": "2020-04-28 23:15:02+00:00",
            "title": "Synonymy = Translational Equivalence",
            "authors": [
                "Bradley Hauer",
                "Grzegorz Kondrak"
            ],
            "abstract": "Synonymy and translational equivalence are the relations of sameness of\nmeaning within and across languages. As the principal relations in wordnets and\nmulti-wordnets, they are vital to computational lexical semantics, yet the\nfield suffers from the absence of a common formal framework to define their\nproperties and relationship. This paper proposes a unifying treatment of these\ntwo relations, which is validated by experiments on existing resources. In our\nview, synonymy and translational equivalence are simply different types of\nsemantic identity. The theory establishes a solid foundation for critically\nre-evaluating prior work in cross-lingual semantics, and facilitating the\ncreation, verification, and amelioration of lexical resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13897v2",
            "date": "2020-04-29 00:09:43+00:00",
            "title": "Empower Entity Set Expansion via Language Model Probing",
            "authors": [
                "Yunyi Zhang",
                "Jiaming Shen",
                "Jingbo Shang",
                "Jiawei Han"
            ],
            "abstract": "Entity set expansion, aiming at expanding a small seed entity set with new\nentities belonging to the same semantic class, is a critical task that benefits\nmany downstream NLP and IR applications, such as question answering, query\nunderstanding, and taxonomy construction. Existing set expansion methods\nbootstrap the seed entity set by adaptively selecting context features and\nextracting new entities. A key challenge for entity set expansion is to avoid\nselecting ambiguous context features which will shift the class semantics and\nlead to accumulative errors in later iterations. In this study, we propose a\nnovel iterative set expansion framework that leverages automatically generated\nclass names to address the semantic drift issue. In each iteration, we select\none positive and several negative class names by probing a pre-trained language\nmodel, and further score each candidate entity based on selected class names.\nExperiments on two datasets show that our framework generates high-quality\nclass names and outperforms previous state-of-the-art methods significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13922v2",
            "date": "2020-04-29 02:08:30+00:00",
            "title": "Revisiting Pre-Trained Models for Chinese Natural Language Processing",
            "authors": [
                "Yiming Cui",
                "Wanxiang Che",
                "Ting Liu",
                "Bing Qin",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Bidirectional Encoder Representations from Transformers (BERT) has shown\nmarvelous improvements across various NLP tasks, and consecutive variants have\nbeen proposed to further improve the performance of the pre-trained language\nmodels. In this paper, we target on revisiting Chinese pre-trained language\nmodels to examine their effectiveness in a non-English language and release the\nChinese pre-trained language model series to the community. We also propose a\nsimple but effective model called MacBERT, which improves upon RoBERTa in\nseveral ways, especially the masking strategy that adopts MLM as correction\n(Mac). We carried out extensive experiments on eight Chinese NLP tasks to\nrevisit the existing pre-trained language models as well as the proposed\nMacBERT. Experimental results show that MacBERT could achieve state-of-the-art\nperformances on many NLP tasks, and we also ablate details with several\nfindings that may help future research. Resources available:\nhttps://github.com/ymcui/MacBERT",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13939v2",
            "date": "2020-04-29 03:34:53+00:00",
            "title": "Evaluating Transformer-Based Multilingual Text Classification",
            "authors": [
                "Sophie Groenwold",
                "Samhita Honnavalli",
                "Lily Ou",
                "Aesha Parekh",
                "Sharon Levy",
                "Diba Mirza",
                "William Yang Wang"
            ],
            "abstract": "As NLP tools become ubiquitous in today's technological landscape, they are\nincreasingly applied to languages with a variety of typological structures.\nHowever, NLP research does not focus primarily on typological differences in\nits analysis of state-of-the-art language models. As a result, NLP tools\nperform unequally across languages with different syntactic and morphological\nstructures. Through a detailed discussion of word order typology, morphological\ntypology, and comparative linguistics, we identify which variables most affect\nlanguage modeling efficacy; in addition, we calculate word order and\nmorphological similarity indices to aid our empirical study. We then use this\nbackground to support our analysis of an experiment we conduct using\nmulti-class text classification on eight languages and eight models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13945v1",
            "date": "2020-04-29 03:58:55+00:00",
            "title": "Basic Linguistic Resources and Baselines for Bhojpuri, Magahi and Maithili for Natural Language Processing",
            "authors": [
                "Rajesh Kumar Mundotiya",
                "Manish Kumar Singh",
                "Rahul Kapur",
                "Swasti Mishra",
                "Anil Kumar Singh"
            ],
            "abstract": "Corpus preparation for low-resource languages and for development of human\nlanguage technology to analyze or computationally process them is a laborious\ntask, primarily due to the unavailability of expert linguists who are native\nspeakers of these languages and also due to the time and resources required.\nBhojpuri, Magahi, and Maithili, languages of the Purvanchal region of India (in\nthe north-eastern parts), are low-resource languages belonging to the\nIndo-Aryan (or Indic) family. They are closely related to Hindi, which is a\nrelatively high-resource language, which is why we make our comparisons with\nHindi. We collected corpora for these three languages from various sources and\ncleaned them to the extent possible, without changing the data in them. The\ntext belongs to different domains and genres. We calculated some basic\nstatistical measures for these corpora at character, word, syllable, and\nmorpheme levels. These corpora were also annotated with parts-of-speech (POS)\nand chunk tags. The basic statistical measures were both absolute and relative\nand were meant to give an indication of linguistic properties such as\nmorphological, lexical, phonological, and syntactic complexities (or richness).\nThe results were compared with a standard Hindi corpus. For most of the\nmeasures, we tried to keep the size of the corpus the same across the languages\nso as to avoid the effect of corpus size, but in some cases it turned out that\nusing the full corpus was better, even if sizes were very different. Although\nthe results are not very clear, we try to draw some conclusions about the\nlanguages and the corpora. For POS tagging and chunking, the BIS tagset was\nused to manually annotate the data. The sizes of the POS tagged data are 16067,\n14669 and 12310 sentences, respectively for Bhojpuri, Magahi and Maithili. The\nsizes for chunking are 9695 and 1954 sentences for Bhojpuri and Maithili,\nrespect",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13947v2",
            "date": "2020-04-29 04:01:52+00:00",
            "title": "BURT: BERT-inspired Universal Representation from Twin Structure",
            "authors": [
                "Yian Li",
                "Hai Zhao"
            ],
            "abstract": "Pre-trained contextualized language models such as BERT have shown great\neffectiveness in a wide range of downstream Natural Language Processing (NLP)\ntasks. However, the effective representations offered by the models target at\neach token inside a sequence rather than each sequence and the fine-tuning step\ninvolves the input of both sequences at one time, leading to unsatisfying\nrepresentations of various sequences with different granularities. Especially,\nas sentence-level representations taken as the full training context in these\nmodels, there comes inferior performance on lower-level linguistic units\n(phrases and words). In this work, we present BURT (BERT inspired Universal\nRepresentation from Twin Structure) that is capable of generating universal,\nfixed-size representations for input sequences of any granularity, i.e., words,\nphrases, and sentences, using a large scale of natural language inference and\nparaphrase data with multiple training objectives. Our proposed BURT adopts the\nSiamese network, learning sentence-level representations from natural language\ninference dataset and word/phrase-level representations from paraphrasing\ndataset, respectively. We evaluate BURT across different granularities of text\nsimilarity tasks, including STS tasks, SemEval2013 Task 5(a) and some commonly\nused word similarity tasks, where BURT substantially outperforms other\nrepresentation models on sentence-level datasets and achieves significant\nimprovements in word/phrase-level representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13952v2",
            "date": "2020-04-29 04:07:12+00:00",
            "title": "Data Augmentation for Spoken Language Understanding via Pretrained Language Models",
            "authors": [
                "Baolin Peng",
                "Chenguang Zhu",
                "Michael Zeng",
                "Jianfeng Gao"
            ],
            "abstract": "The training of spoken language understanding (SLU) models often faces the\nproblem of data scarcity. In this paper, we put forward a data augmentation\nmethod using pretrained language models to boost the variability and accuracy\nof generated utterances. Furthermore, we investigate and propose solutions to\ntwo previously overlooked semi-supervised learning scenarios of data scarcity\nin SLU: i) Rich-in-Ontology: ontology information with numerous valid dialogue\nacts is given; ii) Rich-in-Utterance: a large number of unlabelled utterances\nare available. Empirical results show that our method can produce synthetic\ntraining data that boosts the performance of language understanding models in\nvarious scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13956v1",
            "date": "2020-04-29 04:39:28+00:00",
            "title": "Zero-shot topic generation",
            "authors": [
                "Oleg Vasilyev",
                "Kathryn Evans",
                "Anna Venancio-Marques",
                "John Bohannon"
            ],
            "abstract": "We present an approach to generating topics using a model trained only for\ndocument title generation, with zero examples of topics given during training.\nWe leverage features that capture the relevance of a candidate span in a\ndocument for the generation of a title for that document. The output is a\nweighted collection of the phrases that are most relevant for describing the\ndocument and distinguishing it within a corpus, without requiring access to the\nrest of the corpus. We conducted a double-blind trial in which human annotators\nscored the quality of our machine-generated topics along with original\nhuman-written topics associated with news articles from The Guardian and The\nHuffington Post. The results show that our zero-shot model generates topic\nlabels for news documents that are on average equal to or higher quality than\nthose written by humans, as judged by humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13983v3",
            "date": "2020-04-29 06:52:15+00:00",
            "title": "Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization",
            "authors": [
                "Zhengyuan Liu",
                "Ke Shi",
                "Nancy F. Chen"
            ],
            "abstract": "Much progress has been made in text summarization, fueled by neural\narchitectures using large-scale training corpora. However, in the news domain,\nneural models easily overfit by leveraging position-related features due to the\nprevalence of the inverted pyramid writing style. In addition, there is an\nunmet need to generate a variety of summaries for different users. In this\npaper, we propose a neural framework that can flexibly control summary\ngeneration by introducing a set of sub-aspect functions (i.e. importance,\ndiversity, position). These sub-aspect functions are regulated by a set of\ncontrol codes to decide which sub-aspect to focus on during summary generation.\nWe demonstrate that extracted summaries with minimal position bias is\ncomparable with those generated by standard models that take advantage of\nposition preference. We also show that news summaries generated with a focus on\ndiversity can be more preferred by human raters. These results suggest that a\nmore flexible neural summarization framework providing more control options\ncould be desirable in tailoring to different user preferences, which is useful\nsince it is often impractical to articulate such preferences for different\napplications a priori.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13988v2",
            "date": "2020-04-29 07:04:43+00:00",
            "title": "Knowledgeable Dialogue Reading Comprehension on Key Turns",
            "authors": [
                "Junlong Li",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "Multi-choice machine reading comprehension (MRC) requires models to choose\nthe correct answer from candidate options given a passage and a question. Our\nresearch focuses dialogue-based MRC, where the passages are multi-turn\ndialogues. It suffers from two challenges, the answer selection decision is\nmade without support of latently helpful commonsense, and the multi-turn\ncontext may hide considerable irrelevant information. This work thus makes the\nfirst attempt to tackle those two challenges by extracting substantially\nimportant turns and utilizing external knowledge to enhance the representation\nof context. In this paper, the relevance of each turn to the question are\ncalculated to choose key turns. Besides, terms related to the context and the\nquestion in a knowledge graph are extracted as external knowledge. The original\ncontext, question and external knowledge are encoded with the pre-trained\nlanguage model, then the language representation and key turns are combined\ntogether with a will-designed mechanism to predict the answer. Experimental\nresults on a DREAM dataset show that our proposed model achieves great\nimprovements on baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14004v2",
            "date": "2020-04-29 08:05:32+00:00",
            "title": "Benchmarking Robustness of Machine Reading Comprehension Models",
            "authors": [
                "Chenglei Si",
                "Ziqing Yang",
                "Yiming Cui",
                "Wentao Ma",
                "Ting Liu",
                "Shijin Wang"
            ],
            "abstract": "Machine Reading Comprehension (MRC) is an important testbed for evaluating\nmodels' natural language understanding (NLU) ability. There has been rapid\nprogress in this area, with new models achieving impressive performance on\nvarious benchmarks. However, existing benchmarks only evaluate models on\nin-domain test sets without considering their robustness under test-time\nperturbations or adversarial attacks. To fill this important gap, we construct\nAdvRACE (Adversarial RACE), a new model-agnostic benchmark for evaluating the\nrobustness of MRC models under four different types of adversarial attacks,\nincluding our novel distractor extraction and generation attacks. We show that\nstate-of-the-art (SOTA) models are vulnerable to all of these attacks. We\nconclude that there is substantial room for building more robust MRC models and\nour benchmark can help motivate and measure progress in this area. We release\nour data and code at https://github.com/NoviScl/AdvRACE .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14008v2",
            "date": "2020-04-29 08:08:32+00:00",
            "title": "Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness",
            "authors": [
                "Reina Akama",
                "Sho Yokoi",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Large-scale dialogue datasets have recently become available for training\nneural dialogue agents. However, these datasets have been reported to contain a\nnon-negligible number of unacceptable utterance pairs. In this paper, we\npropose a method for scoring the quality of utterance pairs in terms of their\nconnectivity and relatedness. The proposed scoring method is designed based on\nfindings widely shared in the dialogue and linguistics research communities. We\ndemonstrate that it has a relatively good correlation with the human judgment\nof dialogue quality. Furthermore, the method is applied to filter out\npotentially unacceptable utterance pairs from a large-scale noisy dialogue\ncorpus to ensure its quality. We experimentally confirm that training data\nfiltered by the proposed method improves the quality of neural dialogue agents\nin response generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14021v3",
            "date": "2020-04-29 08:36:08+00:00",
            "title": "Multiscale Collaborative Deep Models for Neural Machine Translation",
            "authors": [
                "Xiangpeng Wei",
                "Heng Yu",
                "Yue Hu",
                "Yue Zhang",
                "Rongxiang Weng",
                "Weihua Luo"
            ],
            "abstract": "Recent evidence reveals that Neural Machine Translation (NMT) models with\ndeeper neural networks can be more effective but are difficult to train. In\nthis paper, we present a MultiScale Collaborative (MSC) framework to ease the\ntraining of NMT models that are substantially deeper than those used\npreviously. We explicitly boost the gradient back-propagation from top to\nbottom levels by introducing a block-scale collaboration mechanism into deep\nNMT models. Then, instead of forcing the whole encoder stack directly learns a\ndesired representation, we let each encoder block learns a fine-grained\nrepresentation and enhance it by encoding spatial dependencies using a\ncontext-scale collaboration. We provide empirical evidence showing that the MSC\nnets are easy to optimize and can obtain improvements of translation quality\nfrom considerably increased depth. On IWSLT translation tasks with three\ntranslation directions, our extremely deep models (with 72-layer encoders)\nsurpass strong baselines by +2.2~+3.1 BLEU points. In addition, our deep MSC\nachieves a BLEU score of 30.56 on WMT14 English-German task that significantly\noutperforms state-of-the-art deep NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14062v1",
            "date": "2020-04-29 10:30:25+00:00",
            "title": "Morphological Disambiguation of South S\u00e1mi with FSTs and Neural Networks",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Linda Wiechetek"
            ],
            "abstract": "We present a method for conducting morphological disambiguation for South\nS\\'ami, which is an endangered language. Our method uses an FST-based\nmorphological analyzer to produce an ambiguous set of morphological readings\nfor each word in a sentence. These readings are disambiguated with a Bi-RNN\nmodel trained on the related North S\\'ami UD Treebank and some synthetically\ngenerated South S\\'ami data. The disambiguation is done on the level of\nmorphological tags ignoring word forms and lemmas; this makes it possible to\nuse North S\\'ami training data for South S\\'ami without the need for a\nbilingual dictionary or aligned word embeddings. Our approach requires only\nminimal resources for South S\\'ami, which makes it usable and applicable in the\ncontexts of any other endangered language as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14065v2",
            "date": "2020-04-29 10:38:09+00:00",
            "title": "Automatically Identifying Gender Issues in Machine Translation using Perturbations",
            "authors": [
                "Hila Gonen",
                "Kellie Webster"
            ],
            "abstract": "The successful application of neural methods to machine translation has\nrealized huge quality advances for the community. With these improvements, many\nhave noted outstanding challenges, including the modeling and treatment of\ngendered language. While previous studies have identified issues using\nsynthetic examples, we develop a novel technique to mine examples from real\nworld data to explore challenges for deployed systems. We use our method to\ncompile an evaluation benchmark spanning examples for four languages from three\nlanguage families, which we publicly release to facilitate research. The\nexamples in our benchmark expose where model representations are gendered, and\nthe unintended consequences these gendered representations can have in\ndownstream application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14080v1",
            "date": "2020-04-29 11:02:25+00:00",
            "title": "Modeling Long Context for Task-Oriented Dialogue State Generation",
            "authors": [
                "Jun Quan",
                "Deyi Xiong"
            ],
            "abstract": "Based on the recently proposed transferable dialogue state generator (TRADE)\nthat predicts dialogue states from utterance-concatenated dialogue context, we\npropose a multi-task learning model with a simple yet effective utterance\ntagging technique and a bidirectional language model as an auxiliary task for\ntask-oriented dialogue state generation. By enabling the model to learn a\nbetter representation of the long dialogue context, our approaches attempt to\nsolve the problem that the performance of the baseline significantly drops when\nthe input dialogue context sequence is long. In our experiments, our proposed\nmodel achieves a 7.03% relative improvement over the baseline, establishing a\nnew state-of-the-art joint goal accuracy of 52.04% on the MultiWOZ 2.0 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14109v2",
            "date": "2020-04-29 12:06:42+00:00",
            "title": "Adversarial Subword Regularization for Robust Neural Machine Translation",
            "authors": [
                "Jungsoo Park",
                "Mujeen Sung",
                "Jinhyuk Lee",
                "Jaewoo Kang"
            ],
            "abstract": "Exposing diverse subword segmentations to neural machine translation (NMT)\nmodels often improves the robustness of machine translation as NMT models can\nexperience various subword candidates. However, the diversification of subword\nsegmentations mostly relies on the pre-trained subword language models from\nwhich erroneous segmentations of unseen words are less likely to be sampled. In\nthis paper, we present adversarial subword regularization (ADVSR) to study\nwhether gradient signals during training can be a substitute criterion for\nexposing diverse subword segmentations. We experimentally show that our\nmodel-based adversarial samples effectively encourage NMT models to be less\nsensitive to segmentation errors and improve the performance of NMT models in\nlow-resource and out-domain datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14134v2",
            "date": "2020-04-09 06:44:08+00:00",
            "title": "Using Punkt for Sentence Segmentation in non-Latin Scripts: Experiments on Kurdish (Sorani) Texts",
            "authors": [
                "Roshna Omer Abdulrahman",
                "Hossein Hassani"
            ],
            "abstract": "Segmentation is a fundamental step for most Natural Language Processing\ntasks. The Kurdish language is a multi-dialect, under-resourced language which\nis written in different scripts. The lack of various segmented corpora is one\nof the major bottlenecks in Kurdish language processing. We used Punkt, an\nunsupervised machine learning method, to segment a Kurdish corpus of Sorani\ndialect, written in Persian-Arabic script. According to the literature, studies\non using Punkt on non-Latin data are scanty. In our experiment, we achieved an\nF1 score of 91.10% and had an Error Rate of 16.32%. The high Error Rate is\nmainly due to the situation of abbreviations in Kurdish and partly because of\nordinal numerals. The data is publicly available at\nhttps://github.com/KurdishBLARK/ KTC-Segmented for non-commercial use under the\nCC BY-NC-SA 4.0 licence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14145v1",
            "date": "2020-04-29 12:47:02+00:00",
            "title": "Entity Candidate Network for Whole-Aware Named Entity Recognition",
            "authors": [
                "Wendong He",
                "Yizhen Shao",
                "Pingjian Zhang"
            ],
            "abstract": "Named Entity Recognition (NER) is a crucial upstream task in Natural Language\nProcessing (NLP). Traditional tag scheme approaches offer a single recognition\nthat does not meet the needs of many downstream tasks such as coreference\nresolution. Meanwhile, Tag scheme approaches ignore the continuity of entities.\nInspired by one-stage object detection models in computer vision (CV), this\npaper proposes a new no-tag scheme, the Whole-Aware Detection, which makes NER\nan object detection task. Meanwhile, this paper presents a novel model, Entity\nCandidate Network (ECNet), and a specific convolution network, Adaptive Context\nConvolution Network (ACCN), to fuse multi-scale contexts and encode entity\ninformation at each position. ECNet identifies the full span of a named entity\nand its type at each position based on Entity Loss. Furthermore, ECNet is\nregulable between the highest precision and the highest recall, while the tag\nscheme approaches are not. Experimental results on the CoNLL 2003 English\ndataset and the WNUT 2017 dataset show that ECNet outperforms other previous\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14165v1",
            "date": "2020-04-26 05:40:36+00:00",
            "title": "Classification of Cuisines from Sequentially Structured Recipes",
            "authors": [
                "Tript Sharma",
                "Utkarsh Upadhyay",
                "Ganesh Bagler"
            ],
            "abstract": "Cultures across the world are distinguished by the idiosyncratic patterns in\ntheir cuisines. These cuisines are characterized in terms of their\nsubstructures such as ingredients, cooking processes and utensils. A complex\nfusion of these substructures intrinsic to a region defines the identity of a\ncuisine. Accurate classification of cuisines based on their culinary features\nis an outstanding problem and has hitherto been attempted to solve by\naccounting for ingredients of a recipe as features. Previous studies have\nattempted cuisine classification by using unstructured recipes without\naccounting for details of cooking techniques. In reality, the cooking\nprocesses/techniques and their order are highly significant for the recipe's\nstructure and hence for its classification. In this article, we have\nimplemented a range of classification techniques by accounting for this\ninformation on the RecipeDB dataset containing sequential data on recipes. The\nstate-of-the-art RoBERTa model presented the highest accuracy of 73.30% among a\nrange of classification models from Logistic Regression and Naive Bayes to\nLSTMs and Transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14166v2",
            "date": "2020-04-26 03:34:06+00:00",
            "title": "SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check",
            "authors": [
                "Xingyi Cheng",
                "Weidi Xu",
                "Kunlong Chen",
                "Shaohua Jiang",
                "Feng Wang",
                "Taifeng Wang",
                "Wei Chu",
                "Yuan Qi"
            ],
            "abstract": "Chinese Spelling Check (CSC) is a task to detect and correct spelling errors\nin Chinese natural language. Existing methods have made attempts to incorporate\nthe similarity knowledge between Chinese characters. However, they take the\nsimilarity knowledge as either an external input resource or just heuristic\nrules. This paper proposes to incorporate phonological and visual similarity\nknowledge into language models for CSC via a specialized graph convolutional\nnetwork (SpellGCN). The model builds a graph over the characters, and SpellGCN\nis learned to map this graph into a set of inter-dependent character\nclassifiers. These classifiers are applied to the representations extracted by\nanother network, such as BERT, enabling the whole network to be end-to-end\ntrainable. Experiments (The dataset and all code for this paper are available\nat https://github.com/ACL2020SpellGCN/SpellGCN) are conducted on three\nhuman-annotated datasets. Our method achieves superior performance against\nprevious models by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14198v2",
            "date": "2020-04-29 13:42:22+00:00",
            "title": "Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis",
            "authors": [
                "Yao-Hung Hubert Tsai",
                "Martin Q. Ma",
                "Muqiao Yang",
                "Ruslan Salakhutdinov",
                "Louis-Philippe Morency"
            ],
            "abstract": "The human language can be expressed through multiple sources of information\nknown as modalities, including tones of voice, facial gestures, and spoken\nlanguage. Recent multimodal learning with strong performances on human-centric\ntasks such as sentiment analysis and emotion recognition are often black-box,\nwith very limited interpretability. In this paper we propose Multimodal\nRouting, which dynamically adjusts weights between input modalities and output\nrepresentations differently for each input sample. Multimodal routing can\nidentify relative importance of both individual modalities and cross-modality\nfeatures. Moreover, the weight assignment by routing allows us to interpret\nmodality-prediction relationships not only globally (i.e. general trends over\nthe whole dataset), but also locally for each single input sample, meanwhile\nkeeping competitive performance compared to state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14200v1",
            "date": "2020-04-29 13:45:30+00:00",
            "title": "Syntax-aware Data Augmentation for Neural Machine Translation",
            "authors": [
                "Sufeng Duan",
                "Hai Zhao",
                "Dongdong Zhang",
                "Rui Wang"
            ],
            "abstract": "Data augmentation is an effective performance enhancement in neural machine\ntranslation (NMT) by generating additional bilingual data. In this paper, we\npropose a novel data augmentation enhancement strategy for neural machine\ntranslation. Different from existing data augmentation methods which simply\nchoose words with the same probability across different sentences for\nmodification, we set sentence-specific probability for word selection by\nconsidering their roles in sentence. We use dependency parse tree of input\nsentence as an effective clue to determine selecting probability for every\nwords in each sentence. Our proposed method is evaluated on WMT14\nEnglish-to-German dataset and IWSLT14 German-to-English dataset. The result of\nextensive experiments show our proposed syntax-aware data augmentation method\nmay effectively boost existing sentence-independent methods for significant\ntranslation performance improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14236v2",
            "date": "2020-04-29 14:35:50+00:00",
            "title": "Normalizing Compositional Structures Across Graphbanks",
            "authors": [
                "Lucia Donatelli",
                "Jonas Groschwitz",
                "Alexander Koller",
                "Matthias Lindemann",
                "Pia Wei\u00dfenhorn"
            ],
            "abstract": "The emergence of a variety of graph-based meaning representations (MRs) has\nsparked an important conversation about how to adequately represent semantic\nstructure. These MRs exhibit structural differences that reflect different\ntheoretical and design considerations, presenting challenges to uniform\nlinguistic analysis and cross-framework semantic parsing. Here, we ask the\nquestion of which design differences between MRs are meaningful and\nsemantically-rooted, and which are superficial. We present a methodology for\nnormalizing discrepancies between MRs at the compositional level (Lindemann et\nal., 2019), finding that we can normalize the majority of divergent phenomena\nusing linguistically-grounded rules. Our work significantly increases the match\nin compositional structure between MRs and improves multi-task learning (MTL)\nin a low-resource setting, demonstrating the usefulness of careful MR design\nanalysis and comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14243v1",
            "date": "2020-04-29 14:47:50+00:00",
            "title": "Towards Transparent and Explainable Attention Models",
            "authors": [
                "Akash Kumar Mohankumar",
                "Preksha Nema",
                "Sharan Narasimhan",
                "Mitesh M. Khapra",
                "Balaji Vasan Srinivasan",
                "Balaraman Ravindran"
            ],
            "abstract": "Recent studies on interpretability of attention distributions have led to\nnotions of faithful and plausible explanations for a model's predictions.\nAttention distributions can be considered a faithful explanation if a higher\nattention weight implies a greater impact on the model's prediction. They can\nbe considered a plausible explanation if they provide a human-understandable\njustification for the model's predictions. In this work, we first explain why\ncurrent attention mechanisms in LSTM based encoders can neither provide a\nfaithful nor a plausible explanation of the model's predictions. We observe\nthat in LSTM based encoders the hidden representations at different time-steps\nare very similar to each other (high conicity) and attention weights in these\nsituations do not carry much meaning because even a random permutation of the\nattention weights does not affect the model's predictions. Based on experiments\non a wide variety of tasks and datasets, we observe attention distributions\noften attribute the model's predictions to unimportant words such as\npunctuation and fail to offer a plausible explanation for the predictions. To\nmake attention mechanisms more faithful and plausible, we propose a modified\nLSTM cell with a diversity-driven training objective that ensures that the\nhidden representations learned at different time steps are diverse. We show\nthat the resulting attention distributions offer more transparency as they (i)\nprovide a more precise importance ranking of the hidden states (ii) are better\nindicative of words important for the model's predictions (iii) correlate\nbetter with gradient-based attribution methods. Human evaluations indicate that\nthe attention distributions learned by our model offer a plausible explanation\nof the model's predictions. Our code has been made publicly available at\nhttps://github.com/akashkm99/Interpretable-Attention",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14253v1",
            "date": "2020-04-29 15:02:01+00:00",
            "title": "GePpeTto Carves Italian into a Language Model",
            "authors": [
                "Lorenzo De Mattei",
                "Michele Cafagna",
                "Felice Dell'Orletta",
                "Malvina Nissim",
                "Marco Guerini"
            ],
            "abstract": "In the last few years, pre-trained neural architectures have provided\nimpressive improvements across several NLP tasks. Still, generative language\nmodels are available mainly for English. We develop GePpeTto, the first\ngenerative language model for Italian, built using the GPT-2 architecture. We\nprovide a thorough analysis of GePpeTto's quality by means of both an automatic\nand a human-based evaluation. The automatic assessment consists in (i)\ncalculating perplexity across different genres and (ii) a profiling analysis\nover GePpeTto's writing characteristics. We find that GePpeTto's production is\na sort of bonsai version of human production, with shorter but yet complex\nsentences. Human evaluation is performed over a sentence completion task, where\nGePpeTto's output is judged as natural more often than not, and much closer to\nthe original human texts than to a simpler language model which we take as\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14257v2",
            "date": "2020-04-29 15:08:53+00:00",
            "title": "Politeness Transfer: A Tag and Generate Approach",
            "authors": [
                "Aman Madaan",
                "Amrith Setlur",
                "Tanmay Parekh",
                "Barnabas Poczos",
                "Graham Neubig",
                "Yiming Yang",
                "Ruslan Salakhutdinov",
                "Alan W Black",
                "Shrimai Prabhumoye"
            ],
            "abstract": "This paper introduces a new task of politeness transfer which involves\nconverting non-polite sentences to polite sentences while preserving the\nmeaning. We also provide a dataset of more than 1.39 instances automatically\nlabeled for politeness to encourage benchmark evaluations on this new task. We\ndesign a tag and generate pipeline that identifies stylistic attributes and\nsubsequently generates a sentence in the target style while preserving most of\nthe source content. For politeness as well as five other transfer tasks, our\nmodel outperforms the state-of-the-art methods on automatic metrics for content\npreservation, with a comparable or better performance on style transfer\naccuracy. Additionally, our model surpasses existing methods on human\nevaluations for grammaticality, meaning preservation and transfer accuracy\nacross all the six style transfer tasks. The data and code is located at\nhttps://github.com/tag-and-generate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14280v2",
            "date": "2020-04-29 15:56:02+00:00",
            "title": "Towards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems",
            "authors": [
                "Jind\u0159ich Libovick\u00fd",
                "Alexander Fraser"
            ],
            "abstract": "Applying the Transformer architecture on the character level usually requires\nvery deep architectures that are difficult and slow to train. These problems\ncan be partially overcome by incorporating a segmentation into tokens in the\nmodel. We show that by initially training a subword model and then finetuning\nit on characters, we can obtain a neural machine translation model that works\nat the character level without requiring token segmentation. We use only the\nvanilla 6-layer Transformer Base architecture. Our character-level models\nbetter capture morphological phenomena and show more robustness to noise at the\nexpense of somewhat worse overall translation quality. Our study is a\nsignificant step towards high-performance and easy to train character-based\nmodels that are not extremely large.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14287v1",
            "date": "2020-04-29 16:11:26+00:00",
            "title": "General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference",
            "authors": [
                "Jingfei Du",
                "Myle Ott",
                "Haoran Li",
                "Xing Zhou",
                "Veselin Stoyanov"
            ],
            "abstract": "The state of the art on many NLP tasks is currently achieved by large\npre-trained language models, which require a considerable amount of\ncomputation. We explore a setting where many different predictions are made on\na single piece of text. In that case, some of the computational cost during\ninference can be amortized over the different tasks using a shared text\nencoder. We compare approaches for training such an encoder and show that\nencoders pre-trained over multiple tasks generalize well to unseen tasks. We\nalso compare ways of extracting fixed- and limited-size representations from\nthis encoder, including different ways of pooling features extracted from\nmultiple layers or positions. Our best approach compares favorably to knowledge\ndistillation, achieving higher accuracy and lower computational cost once the\nsystem is handling around 7 tasks. Further, we show that through binary\nquantization, we can reduce the size of the extracted representations by a\nfactor of 16 making it feasible to store them for later use. The resulting\nmethod offers a compelling solution for using large-scale pre-trained models at\na fraction of the computational cost when multiple tasks are performed on the\nsame text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14302v1",
            "date": "2020-04-29 16:21:50+00:00",
            "title": "Evaluating Dialogue Generation Systems via Response Selection",
            "authors": [
                "Shiki Sato",
                "Reina Akama",
                "Hiroki Ouchi",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Existing automatic evaluation metrics for open-domain dialogue response\ngeneration systems correlate poorly with human evaluation. We focus on\nevaluating response generation systems via response selection. To evaluate\nsystems properly via response selection, we propose the method to construct\nresponse selection test sets with well-chosen false candidates. Specifically,\nwe propose to construct test sets filtering out some types of false candidates:\n(i) those unrelated to the ground-truth response and (ii) those acceptable as\nappropriate responses. Through experiments, we demonstrate that evaluating\nsystems via response selection with the test sets developed by our method\ncorrelates more strongly with human evaluation, compared with widely used\nautomatic evaluation metrics such as BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14312v1",
            "date": "2020-04-29 16:36:38+00:00",
            "title": "A Cross-Genre Ensemble Approach to Robust Reddit Part of Speech Tagging",
            "authors": [
                "Shabnam Behzad",
                "Amir Zeldes"
            ],
            "abstract": "Part of speech tagging is a fundamental NLP task often regarded as solved for\nhigh-resource languages such as English. Current state-of-the-art models have\nachieved high accuracy, especially on the news domain. However, when these\nmodels are applied to other corpora with different genres, and especially\nuser-generated data from the Web, we see substantial drops in performance. In\nthis work, we study how a state-of-the-art tagging model trained on different\ngenres performs on Web content from unfiltered Reddit forum discussions. More\nspecifically, we use data from multiple sources: OntoNotes, a large benchmark\ncorpus with 'well-edited' text, the English Web Treebank with 5 Web genres, and\nGUM, with 7 further genres other than Reddit. We report the results when\ntraining on different splits of the data, tested on Reddit. Our results show\nthat even small amounts of in-domain data can outperform the contribution of\ndata an order of magnitude larger coming from other Web domains. To make\nprogress on out-of-domain tagging, we also evaluate an ensemble approach using\nmultiple single-genre taggers as input features to a meta-classifier. We\npresent state of the art performance on tagging Reddit data, as well as error\nanalysis of the results of these models, and offer a typology of the most\ncommon error types among them, broken down by training corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14325v3",
            "date": "2020-04-29 16:51:21+00:00",
            "title": "Don't Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation",
            "authors": [
                "Daniel Loureiro",
                "Jose Camacho-Collados"
            ],
            "abstract": "State-of-the-art methods for Word Sense Disambiguation (WSD) combine two\ndifferent features: the power of pre-trained language models and a propagation\nmethod to extend the coverage of such models. This propagation is needed as\ncurrent sense-annotated corpora lack coverage of many instances in the\nunderlying sense inventory (usually WordNet). At the same time, unambiguous\nwords make for a large portion of all words in WordNet, while being poorly\ncovered in existing sense-annotated corpora. In this paper, we propose a simple\nmethod to provide annotations for most unambiguous words in a large corpus. We\nintroduce the UWA (Unambiguous Word Annotations) dataset and show how a\nstate-of-the-art propagation-based model can use it to extend the coverage and\nquality of its word sense embeddings by a significant margin, improving on its\noriginal results on WSD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14327v2",
            "date": "2020-04-29 16:52:50+00:00",
            "title": "UDapter: Language Adaptation for Truly Universal Dependency Parsing",
            "authors": [
                "Ahmet \u00dcst\u00fcn",
                "Arianna Bisazza",
                "Gosse Bouma",
                "Gertjan van Noord"
            ],
            "abstract": "Recent advances in multilingual dependency parsing have brought the idea of a\ntruly universal parser closer to reality. However, cross-language interference\nand restrained model capacity remain major obstacles. To address this, we\npropose a novel multilingual task adaptation approach based on contextual\nparameter generation and adapter modules. This approach enables to learn\nadapters via language embeddings while sharing model parameters across\nlanguages. It also allows for an easy but effective integration of existing\nlinguistic typology features into the parsing network. The resulting parser,\nUDapter, outperforms strong monolingual and multilingual baselines on the\nmajority of both high-resource and low-resource (zero-shot) languages, showing\nthe success of the proposed adaptation approach. Our in-depth analyses show\nthat soft parameter sharing via typological features is key to this success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14357v1",
            "date": "2020-04-29 17:35:05+00:00",
            "title": "Detecting Domain Polarity-Changes of Words in a Sentiment Lexicon",
            "authors": [
                "Shuai Wang",
                "Guangyi Lv",
                "Sahisnu Mazumder",
                "Bing Liu"
            ],
            "abstract": "Sentiment lexicons are instrumental for sentiment analysis. One can use a set\nof sentiment words provided in a sentiment lexicon and a lexicon-based\nclassifier to perform sentiment classification. One major issue with this\napproach is that many sentiment words are domain dependent. That is, they may\nbe positive in some domains but negative in some others. We refer to this\nproblem as domain polarity-changes of words. Detecting such words and\ncorrecting their sentiment for an application domain is very important. In this\npaper, we propose a graph-based technique to tackle this problem. Experimental\nresults show its effectiveness on multiple real-world datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14425v1",
            "date": "2020-04-29 18:39:15+00:00",
            "title": "A Benchmark Dataset of Check-worthy Factual Claims",
            "authors": [
                "Fatma Arslan",
                "Naeemul Hassan",
                "Chengkai Li",
                "Mark Tremayne"
            ],
            "abstract": "In this paper we present the ClaimBuster dataset of 23,533 statements\nextracted from all U.S. general election presidential debates and annotated by\nhuman coders. The ClaimBuster dataset can be leveraged in building\ncomputational methods to identify claims that are worth fact-checking from the\nmyriad of sources of digital or traditional media. The ClaimBuster dataset is\npublicly available to the research community, and it can be found at\nhttp://doi.org/10.5281/zenodo.3609356.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14448v1",
            "date": "2020-04-29 19:46:26+00:00",
            "title": "What Happens To BERT Embeddings During Fine-tuning?",
            "authors": [
                "Amil Merchant",
                "Elahe Rahimtoroghi",
                "Ellie Pavlick",
                "Ian Tenney"
            ],
            "abstract": "While there has been much recent work studying how linguistic information is\nencoded in pre-trained sentence representations, comparatively little is\nunderstood about how these models change when adapted to solve downstream\ntasks. Using a suite of analysis techniques (probing classifiers,\nRepresentational Similarity Analysis, and model ablations), we investigate how\nfine-tuning affects the representations of the BERT model. We find that while\nfine-tuning necessarily makes significant changes, it does not lead to\ncatastrophic forgetting of linguistic phenomena. We instead find that\nfine-tuning primarily affects the top layers of BERT, but with noteworthy\nvariation across tasks. In particular, dependency parsing reconfigures most of\nthe model, whereas SQuAD and MNLI appear to involve much shallower processing.\nFinally, we also find that fine-tuning has a weaker effect on representations\nof out-of-domain sentences, suggesting room for improvement in model\ngeneralization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14454v1",
            "date": "2020-04-29 20:02:58+00:00",
            "title": "A Large-Scale Semi-Supervised Dataset for Offensive Language Identification",
            "authors": [
                "Sara Rosenthal",
                "Pepa Atanasova",
                "Georgi Karadzhov",
                "Marcos Zampieri",
                "Preslav Nakov"
            ],
            "abstract": "The use of offensive language is a major problem in social media which has\nled to an abundance of research in detecting content such as hate speech,\ncyberbulling, and cyber-aggression. There have been several attempts to\nconsolidate and categorize these efforts. Recently, the OLID dataset used at\nSemEval-2019 proposed a hierarchical three-level annotation taxonomy which\naddresses different types of offensive language as well as important\ninformation such as the target of such content. The categorization provides\nmeaningful and important information for understanding offensive language.\nHowever, the OLID dataset is limited in size, especially for some of the\nlow-level categories, which included only a few hundred instances, thus making\nit challenging to train robust deep learning models. Here, we address this\nlimitation by creating the largest available dataset for this task, SOLID.\nSOLID contains over nine million English tweets labeled in a semi-supervised\nmanner. We further demonstrate experimentally that using SOLID along with OLID\nyields improved performance on the OLID test set for two different models,\nespecially for the lower levels of the taxonomy. Finally, we perform analysis\nof the models' performance on easy and hard examples of offensive language\nusing data annotated in a semi-supervised way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14457v1",
            "date": "2020-04-29 20:12:20+00:00",
            "title": "\"The Boating Store Had Its Best Sail Ever\": Pronunciation-attentive Contextualized Pun Recognition",
            "authors": [
                "Yichao Zhou",
                "Jyun-Yu Jiang",
                "Jieyu Zhao",
                "Kai-Wei Chang",
                "Wei Wang"
            ],
            "abstract": "Humor plays an important role in human languages and it is essential to model\nhumor when building intelligence systems. Among different forms of humor, puns\nperform wordplay for humorous effects by employing words with double entendre\nand high phonetic similarity. However, identifying and modeling puns are\nchallenging as puns usually involved implicit semantic or phonological tricks.\nIn this paper, we propose Pronunciation-attentive Contextualized Pun\nRecognition (PCPR) to perceive human humor, detect if a sentence contains puns\nand locate them in the sentence. PCPR derives contextualized representation for\neach word in a sentence by capturing the association between the surrounding\ncontext and its corresponding phonetic symbols. Extensive experiments are\nconducted on two benchmark datasets. Results demonstrate that the proposed\napproach significantly outperforms the state-of-the-art methods in pun\ndetection and location tasks. In-depth analyses verify the effectiveness and\nrobustness of PCPR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14513v2",
            "date": "2020-04-29 23:20:40+00:00",
            "title": "Asking without Telling: Exploring Latent Ontologies in Contextual Representations",
            "authors": [
                "Julian Michael",
                "Jan A. Botha",
                "Ian Tenney"
            ],
            "abstract": "The success of pretrained contextual encoders, such as ELMo and BERT, has\nbrought a great deal of interest in what these models learn: do they, without\nexplicit supervision, learn to encode meaningful notions of linguistic\nstructure? If so, how is this structure encoded? To investigate this, we\nintroduce latent subclass learning (LSL): a modification to existing\nclassifier-based probing methods that induces a latent categorization (or\nontology) of the probe's inputs. Without access to fine-grained gold labels,\nLSL extracts emergent structure from input representations in an interpretable\nand quantifiable form. In experiments, we find strong evidence of familiar\ncategories, such as a notion of personhood in ELMo, as well as novel\nontological distinctions, such as a preference for fine-grained semantic roles\non core arguments. Our results provide unique new evidence of emergent\nstructure in pretrained encoders, including departures from existing\nannotations which are inaccessible to earlier methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14516v1",
            "date": "2020-04-29 23:40:08+00:00",
            "title": "A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT",
            "authors": [
                "Masaaki Nagata",
                "Chousa Katsuki",
                "Masaaki Nishino"
            ],
            "abstract": "We present a novel supervised word alignment method based on cross-language\nspan prediction. We first formalize a word alignment problem as a collection of\nindependent predictions from a token in the source sentence to a span in the\ntarget sentence. As this is equivalent to a SQuAD v2.0 style question answering\ntask, we then solve this problem by using multilingual BERT, which is\nfine-tuned on a manually created gold word alignment data. We greatly improved\nthe word alignment accuracy by adding the context of the token to the question.\nIn the experiments using five word alignment datasets among Chinese, Japanese,\nGerman, Romanian, French, and English, we show that the proposed method\nsignificantly outperformed previous supervised and unsupervised word alignment\nmethods without using any bitexts for pretraining. For example, we achieved an\nF1 score of 86.7 for the Chinese-English data, which is 13.3 points higher than\nthe previous state-of-the-art supervised methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14517v1",
            "date": "2020-04-29 23:41:32+00:00",
            "title": "Bilingual Text Extraction as Reading Comprehension",
            "authors": [
                "Katsuki Chousa",
                "Masaaki Nagata",
                "Masaaki Nishino"
            ],
            "abstract": "In this paper, we propose a method to extract bilingual texts automatically\nfrom noisy parallel corpora by framing the problem as a token-level span\nprediction, such as SQuAD-style Reading Comprehension. To extract a span of the\ntarget document that is a translation of a given source sentence (span), we use\neither QANet or multilingual BERT. QANet can be trained for a specific parallel\ncorpus from scratch, while multilingual BERT can utilize pre-trained\nmultilingual representations. For the span prediction method using QANet, we\nintroduce a total optimization method using integer linear programming to\nachieve consistency in the predicted parallel spans. We conduct a parallel\nsentence extraction experiment using simulated noisy parallel corpora with two\nlanguage pairs (En-Fr and En-Ja) and find that the proposed method using QANet\nachieves significantly better accuracy than a baseline method using two\nbi-directional RNN encoders, particularly for distant language pairs (En-Ja).\nWe also conduct a sentence alignment experiment using En-Ja newspaper articles\nand find that the proposed method using multilingual BERT achieves\nsignificantly better accuracy than a baseline method using a bilingual\ndictionary and dynamic programming.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14519v5",
            "date": "2020-04-30 00:01:08+00:00",
            "title": "An Empirical Study of Pre-trained Transformers for Arabic Information Extraction",
            "authors": [
                "Wuwei Lan",
                "Yang Chen",
                "Wei Xu",
                "Alan Ritter"
            ],
            "abstract": "Multilingual pre-trained Transformers, such as mBERT (Devlin et al., 2019)\nand XLM-RoBERTa (Conneau et al., 2020a), have been shown to enable the\neffective cross-lingual zero-shot transfer. However, their performance on\nArabic information extraction (IE) tasks is not very well studied. In this\npaper, we pre-train a customized bilingual BERT, dubbed GigaBERT, that is\ndesigned specifically for Arabic NLP and English-to-Arabic zero-shot transfer\nlearning. We study GigaBERT's effectiveness on zero-short transfer across four\nIE tasks: named entity recognition, part-of-speech tagging, argument role\nlabeling, and relation extraction. Our best model significantly outperforms\nmBERT, XLM-RoBERTa, and AraBERT (Antoun et al., 2020) in both the supervised\nand zero-shot transfer settings. We have made our pre-trained models publicly\navailable at https://github.com/lanwuwei/GigaBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14523v2",
            "date": "2020-04-30 00:11:34+00:00",
            "title": "Exploiting Sentence Order in Document Alignment",
            "authors": [
                "Brian Thompson",
                "Philipp Koehn"
            ],
            "abstract": "We present a simple document alignment method that incorporates sentence\norder information in both candidate generation and candidate re-scoring. Our\nmethod results in 61% relative reduction in error compared to the best\npreviously published result on the WMT16 document alignment shared task. Our\nmethod improves downstream MT performance on web-scraped Sinhala--English\ndocuments from ParaCrawl, outperforming the document alignment method used in\nthe most recent ParaCrawl release. It also outperforms a comparable corpora\nmethod which uses the same multilingual embeddings, demonstrating that\nexploiting sentence order is beneficial even if the end goal is sentence-level\nbitext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14524v2",
            "date": "2020-04-30 00:11:53+00:00",
            "title": "Simulated Multiple Reference Training Improves Low-Resource Machine Translation",
            "authors": [
                "Huda Khayrallah",
                "Brian Thompson",
                "Matt Post",
                "Philipp Koehn"
            ],
            "abstract": "Many valid translations exist for a given sentence, yet machine translation\n(MT) is trained with a single reference translation, exacerbating data sparsity\nin low-resource settings. We introduce Simulated Multiple Reference Training\n(SMRT), a novel MT training method that approximates the full space of possible\ntranslations by sampling a paraphrase of the reference sentence from a\nparaphraser and training the MT model to predict the paraphraser's distribution\nover possible tokens. We demonstrate the effectiveness of SMRT in low-resource\nsettings when translating to English, with improvements of 1.2 to 7.0 BLEU. We\nalso find SMRT is complementary to back-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14530v2",
            "date": "2020-04-30 00:49:14+00:00",
            "title": "Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations",
            "authors": [
                "Peng Qi",
                "Yuhao Zhang",
                "Christopher D. Manning"
            ],
            "abstract": "We investigate the problem of generating informative questions in\ninformation-asymmetric conversations. Unlike previous work on question\ngeneration which largely assumes knowledge of what the answer might be, we are\ninterested in the scenario where the questioner is not given the context from\nwhich answers are drawn, but must reason pragmatically about how to acquire new\ninformation, given the shared conversation history. We identify two core\nchallenges: (1) formally defining the informativeness of potential questions,\nand (2) exploring the prohibitively large space of potential questions to find\nthe good candidates. To generate pragmatic questions, we use reinforcement\nlearning to optimize an informativeness metric we propose, combined with a\nreward function designed to promote more specific questions. We demonstrate\nthat the resulting pragmatic questioner substantially improves the\ninformativeness and specificity of questions generated over a baseline model,\nas evaluated by our metrics as well as humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14532v1",
            "date": "2020-04-30 01:15:40+00:00",
            "title": "Hierarchical Encoders for Modeling and Interpreting Screenplays",
            "authors": [
                "Gayatri Bhat",
                "Avneesh Saluja",
                "Melody Dye",
                "Jan Florjanczyk"
            ],
            "abstract": "While natural language understanding of long-form documents is still an open\nchallenge, such documents often contain structural information that can inform\nthe design of models for encoding them. Movie scripts are an example of such\nrichly structured text - scripts are segmented into scenes, which are further\ndecomposed into dialogue and descriptive components. In this work, we propose a\nneural architecture for encoding this structure, which performs robustly on a\npair of multi-label tag classification datasets, without the need for\nhandcrafted features. We add a layer of insight by augmenting an unsupervised\n\"interpretability\" module to the encoder, allowing for the extraction and\nvisualization of narrative trajectories. Though this work specifically tackles\nscreenplays, we discuss how the underlying approach can be generalized to a\nrange of structured documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14535v2",
            "date": "2020-04-30 01:36:52+00:00",
            "title": "Text Segmentation by Cross Segment Attention",
            "authors": [
                "Michal Lukasik",
                "Boris Dadachev",
                "Gon\u00e7alo Sim\u00f5es",
                "Kishore Papineni"
            ],
            "abstract": "Document and discourse segmentation are two fundamental NLP tasks pertaining\nto breaking up text into constituents, which are commonly used to help\ndownstream tasks such as information retrieval or text summarization. In this\nwork, we propose three transformer-based architectures and provide\ncomprehensive comparisons with previously proposed approaches on three standard\ndatasets. We establish a new state-of-the-art, reducing in particular the error\nrates by a large margin in all cases. We further analyze model sizes and find\nthat we can build models with many fewer parameters while keeping good\nperformance, thus facilitating real-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14543v3",
            "date": "2020-04-30 02:03:24+00:00",
            "title": "TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding",
            "authors": [
                "Linyang Li",
                "Xipeng Qiu"
            ],
            "abstract": "Gradient-based adversarial training is widely used in improving the\nrobustness of neural networks, while it cannot be easily adapted to natural\nlanguage processing tasks since the embedding space is discrete. In natural\nlanguage processing fields, virtual adversarial training is introduced since\ntexts are discrete and cannot be perturbed by gradients directly.\nAlternatively, virtual adversarial training, which generates perturbations on\nthe embedding space, is introduced in NLP tasks. Despite its success, existing\nvirtual adversarial training methods generate perturbations roughly constrained\nby Frobenius normalization balls. To craft fine-grained perturbations, we\npropose a Token-Aware Virtual Adversarial Training method. We introduce a\ntoken-level accumulated perturbation vocabulary to initialize the perturbations\nbetter and use a token-level normalization ball to constrain these\nperturbations pertinently. Experiments show that our method improves the\nperformance of pre-trained models such as BERT and ALBERT in various tasks by a\nconsiderable margin. The proposed method improves the score of the GLUE\nbenchmark from 78.3 to 80.9 using BERT model and it also enhances the\nperformance of sequence labeling and text classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14550v2",
            "date": "2020-04-30 02:27:12+00:00",
            "title": "Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots",
            "authors": [
                "Jia-Chen Gu",
                "Zhen-Hua Ling",
                "Quan Liu",
                "Zhigang Chen",
                "Xiaodan Zhu"
            ],
            "abstract": "The challenges of building knowledge-grounded retrieval-based chatbots lie in\nhow to ground a conversation on its background knowledge and how to match\nresponse candidates with both context and knowledge simultaneously. This paper\nproposes a method named Filtering before Iteratively REferring (FIRE) for this\ntask. In this method, a context filter and a knowledge filter are first built,\nwhich derive knowledge-aware context representations and context-aware\nknowledge representations respectively by global and bidirectional attention.\nBesides, the entries irrelevant to the conversation are discarded by the\nknowledge filter. After that, iteratively referring is performed between\ncontext and response representations as well as between knowledge and response\nrepresentations, in order to collect deep matching features for scoring\nresponse candidates. Experimental results show that FIRE outperforms previous\nmethods by margins larger than 2.8% and 4.1% on the PERSONA-CHAT dataset with\noriginal and revised personas respectively, and margins larger than 3.1% on the\nCMU_DoG dataset in terms of top-1 accuracy. We also show that FIRE is more\ninterpretable by visualizing the knowledge grounding process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14555v1",
            "date": "2020-04-30 03:14:16+00:00",
            "title": "User-Guided Aspect Classification for Domain-Specific Texts",
            "authors": [
                "Peiran Li",
                "Fang Guo",
                "Jingbo Shang"
            ],
            "abstract": "Aspect classification, identifying aspects of text segments, facilitates\nnumerous applications, such as sentiment analysis and review summarization. To\nalleviate the human effort on annotating massive texts, in this paper, we study\nthe problem of classifying aspects based on only a few user-provided seed words\nfor pre-defined aspects. The major challenge lies in how to handle the noisy\nmisc aspect, which is designed for texts without any pre-defined aspects. Even\ndomain experts have difficulties to nominate seed words for the misc aspect,\nmaking existing seed-driven text classification methods not applicable. We\npropose a novel framework, ARYA, which enables mutual enhancements between\npre-defined aspects and the misc aspect via iterative classifier training and\nseed updating. Specifically, it trains a classifier for pre-defined aspects and\nthen leverages it to induce the supervision for the misc aspect. The prediction\nresults of the misc aspect are later utilized to filter out noisy seed words\nfor pre-defined aspects. Experiments in two domains demonstrate the superior\nperformance of our proposed framework, as well as the necessity and importance\nof properly modeling the misc aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14560v1",
            "date": "2020-04-30 03:29:21+00:00",
            "title": "RikiNet: Reading Wikipedia Pages for Natural Question Answering",
            "authors": [
                "Dayiheng Liu",
                "Yeyun Gong",
                "Jie Fu",
                "Yu Yan",
                "Jiusheng Chen",
                "Daxin Jiang",
                "Jiancheng Lv",
                "Nan Duan"
            ],
            "abstract": "Reading long documents to answer open-domain questions remains challenging in\nnatural language understanding. In this paper, we introduce a new model, called\nRikiNet, which reads Wikipedia pages for natural question answering. RikiNet\ncontains a dynamic paragraph dual-attention reader and a multi-level cascaded\nanswer predictor. The reader dynamically represents the document and question\nby utilizing a set of complementary attention mechanisms. The representations\nare then fed into the predictor to obtain the span of the short answer, the\nparagraph of the long answer, and the answer type in a cascaded manner. On the\nNatural Questions (NQ) dataset, a single RikiNet achieves 74.3 F1 and 57.9 F1\non long-answer and short-answer tasks. To our best knowledge, it is the first\nsingle model that outperforms the single human performance. Furthermore, an\nensemble RikiNet obtains 76.1 F1 and 61.3 F1 on long-answer and short-answer\ntasks, achieving the best performance on the official NQ leaderboard",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14564v2",
            "date": "2020-04-30 03:32:34+00:00",
            "title": "Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing",
            "authors": [
                "Brian Thompson",
                "Matt Post"
            ],
            "abstract": "We frame the task of machine translation evaluation as one of scoring machine\ntranslation output with a sequence-to-sequence paraphraser, conditioned on a\nhuman reference. We propose training the paraphraser as a multilingual NMT\nsystem, treating paraphrasing as a zero-shot translation task (e.g., Czech to\nCzech). This results in the paraphraser's output mode being centered around a\ncopy of the input sequence, which represents the best case scenario where the\nMT system output matches a human reference. Our method is simple and intuitive,\nand does not require human judgements for training. Our single model (trained\nin 39 languages) outperforms or statistically ties with all prior metrics on\nthe WMT 2019 segment-level shared metrics task in all languages (excluding\nGujarati where the model had no training data). We also explore using our model\nfor the task of quality estimation as a metric--conditioning on the source\ninstead of the reference--and find that it significantly outperforms every\nsubmission to the WMT 2019 shared task on quality estimation in every language\npair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14565v2",
            "date": "2020-04-30 03:35:20+00:00",
            "title": "Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training",
            "authors": [
                "Chenguang Zhu"
            ],
            "abstract": "The natural language generation (NLG) module in a task-oriented dialogue\nsystem produces user-facing utterances conveying required information. Thus, it\nis critical for the generated response to be natural and fluent. We propose to\nintegrate adversarial training to produce more human-like responses. The model\nuses Straight-Through Gumbel-Softmax estimator for gradient computation. We\nalso propose a two-stage training scheme to boost performance. Empirical\nresults show that the adversarial training can effectively improve the quality\nof language generation in both automatic and human evaluations. For example, in\nthe RNN-LG Restaurant dataset, our model AdvNLG outperforms the previous\nstate-of-the-art result by 3.6% in BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14571v1",
            "date": "2020-04-30 03:48:14+00:00",
            "title": "memeBot: Towards Automatic Image Meme Generation",
            "authors": [
                "Aadhavan Sadasivam",
                "Kausic Gunasekar",
                "Hasan Davulcu",
                "Yezhou Yang"
            ],
            "abstract": "Image memes have become a widespread tool used by people for interacting and\nexchanging ideas over social media, blogs, and open messengers. This work\nproposes to treat automatic image meme generation as a translation process, and\nfurther present an end to end neural and probabilistic approach to generate an\nimage-based meme for any given sentence using an encoder-decoder architecture.\nFor a given input sentence, an image meme is generated by combining a meme\ntemplate image and a text caption where the meme template image is selected\nfrom a set of popular candidates using a selection module, and the meme caption\nis generated by an encoder-decoder model. An encoder is used to map the\nselected meme template and the input sentence into a meme embedding and a\ndecoder is used to decode the meme caption from the meme embedding. The\ngenerated natural language meme caption is conditioned on the input sentence\nand the selected meme template. The model learns the dependencies between the\nmeme captions and the meme template images and generates new memes using the\nlearned dependencies. The quality of the generated captions and the generated\nmemes is evaluated through both automated and human evaluation. An experiment\nis designed to score how well the generated memes can represent the tweets from\nTwitter conversations. Experiments on Twitter data show the efficacy of the\nmodel in generating memes for sentences in online social interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14577v2",
            "date": "2020-04-30 03:59:13+00:00",
            "title": "Exploring Contextualized Neural Language Models for Temporal Dependency Parsing",
            "authors": [
                "Hayley Ross",
                "Jonathon Cai",
                "Bonan Min"
            ],
            "abstract": "Extracting temporal relations between events and time expressions has many\napplications such as constructing event timelines and time-related question\nanswering. It is a challenging problem which requires syntactic and semantic\ninformation at sentence or discourse levels, which may be captured by deep\ncontextualized language models (LMs) such as BERT (Devlin et al., 2019). In\nthis paper, we develop several variants of BERT-based temporal dependency\nparser, and show that BERT significantly improves temporal dependency parsing\n(Zhang and Xue, 2018a). We also present a detailed analysis on why deep\ncontextualized neural LMs help and where they may fall short. Source code and\nresources are made available at https://github.com/bnmin/tdp_ranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14579v2",
            "date": "2020-04-30 04:06:06+00:00",
            "title": "Logic2Text: High-Fidelity Natural Language Generation from Logical Forms",
            "authors": [
                "Zhiyu Chen",
                "Wenhu Chen",
                "Hanwen Zha",
                "Xiyou Zhou",
                "Yunkai Zhang",
                "Sairam Sundaresan",
                "William Yang Wang"
            ],
            "abstract": "Previous works on Natural Language Generation (NLG) from structured data have\nprimarily focused on surface-level descriptions of record sequences. However,\nfor complex structured data, e.g., multi-row tables, it is often desirable for\nan NLG system to describe interesting facts from logical inferences across\nrecords. If only provided with the table, it is hard for existing models to\nproduce controllable and high-fidelity logical generations. In this work, we\nformulate logical level NLG as generation from logical forms in order to obtain\ncontrollable, high-fidelity, and faithful generations. We present a new\nlarge-scale dataset, \\textsc{Logic2Text}, with 10,753 descriptions involving\ncommon logic types paired with the underlying logical forms. The logical forms\nshow diversified graph structure of free schema, which poses great challenges\non the model's ability to understand the semantics. We experiment on (1)\nFully-supervised training with the full datasets, and (2) Few-shot setting,\nprovided with hundreds of paired examples; We compare several popular\ngeneration models and analyze their performances. We hope our dataset can\nencourage research towards building an advanced NLG system capable of natural,\nfaithful, and human-like generation. The dataset and code are available at\nhttps://github.com/czyssrs/Logic2Text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14601v3",
            "date": "2020-04-30 06:24:03+00:00",
            "title": "Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models",
            "authors": [
                "Isabel Papadimitriou",
                "Dan Jurafsky"
            ],
            "abstract": "We propose transfer learning as a method for analyzing the encoding of\ngrammatical structure in neural language models. We train LSTMs on\nnon-linguistic data and evaluate their performance on natural language to\nassess which kinds of data induce generalizable structural features that LSTMs\ncan use for natural language. We find that training on non-linguistic data with\nlatent structure (MIDI music or Java code) improves test performance on natural\nlanguage, despite no overlap in surface form or vocabulary. To pinpoint the\nkinds of abstract structure that models may be encoding to lead to this\nimprovement, we run similar experiments with two artificial parentheses\nlanguages: one which has a hierarchical recursive structure, and a control\nwhich has paired tokens but no recursion. Surprisingly, training a model on\neither of these artificial languages leads to the same substantial gains when\ntesting on natural language. Further experiments on transfer between natural\nlanguages controlling for vocabulary overlap show that zero-shot performance on\na test language is highly correlated with typological syntactic similarity to\nthe training language, suggesting that representations induced by pre-training\ncorrespond to the cross-linguistic syntactic properties. Our results provide\ninsights into the ways that neural models represent abstract syntactic\nstructure, and also about the kind of structural inductive biases which allow\nfor natural language acquisition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14602v4",
            "date": "2020-04-30 06:25:16+00:00",
            "title": "Look at the First Sentence: Position Bias in Question Answering",
            "authors": [
                "Miyoung Ko",
                "Jinhyuk Lee",
                "Hyunjae Kim",
                "Gangwoo Kim",
                "Jaewoo Kang"
            ],
            "abstract": "Many extractive question answering models are trained to predict start and\nend positions of answers. The choice of predicting answers as positions is\nmainly due to its simplicity and effectiveness. In this study, we hypothesize\nthat when the distribution of the answer positions is highly skewed in the\ntraining set (e.g., answers lie only in the k-th sentence of each passage), QA\nmodels predicting answers as positions can learn spurious positional cues and\nfail to give answers in different positions. We first illustrate this position\nbias in popular extractive QA models such as BiDAF and BERT and thoroughly\nexamine how position bias propagates through each layer of BERT. To safely\ndeliver position information without position bias, we train models with\nvarious de-biasing methods including entropy regularization and bias\nensembling. Among them, we found that using the prior distribution of answer\npositions as a bias model is very effective at reducing position bias,\nrecovering the performance of BERT from 37.48% to 81.64% when trained on a\nbiased SQuAD dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14607v1",
            "date": "2020-04-30 07:15:36+00:00",
            "title": "Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation",
            "authors": [
                "Prathyusha Jwalapuram",
                "Barbara Rychalska",
                "Shafiq Joty",
                "Dominika Basaj"
            ],
            "abstract": "Despite increasing instances of machine translation (MT) systems including\ncontextual information, the evidence for translation quality improvement is\nsparse, especially for discourse phenomena. Popular metrics like BLEU are not\nexpressive or sensitive enough to capture quality improvements or drops that\nare minor in size but significant in perception. We introduce the first of\ntheir kind MT benchmark datasets that aim to track and hail improvements across\nfour main discourse phenomena: anaphora, lexical consistency, coherence and\nreadability, and discourse connective translation. We also introduce evaluation\nmethods for these tasks, and evaluate several baseline MT systems on the\ncurated datasets. Surprisingly, we find that existing context-aware models do\nnot improve discourse-related translations consistently across languages and\nphenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14614v2",
            "date": "2020-04-30 07:31:24+00:00",
            "title": "Knowledge Injection into Dialogue Generation via Language Models",
            "authors": [
                "Yi-Lin Tuan",
                "Wei Wei",
                "William Yang Wang"
            ],
            "abstract": "Dialogue generation has been successfully learned from scratch by neural\nnetworks, but tends to produce the same general response, e.g., \"what are you\ntalking about?\", in many conversations. To reduce this homogeneity, external\nknowledge such as the speaker's profile and domain knowledge is applied as an\nadditional condition to diversify a model's output. The required knowledge to\ndevelop an effective conversation, however, is not always available, which is\ndifferent from prior work's assumption that a model always has acquired\nsufficient knowledge before chatting. This problem can be detrimental when\napplying a dialogue model like this chatting online with unconstrained people\nand topics, because the model does not have the needed knowledge. To address\nthis problem, we propose InjK, which is a two-stage approach to inject\nknowledge into a dialogue generation model. First, we train a large-scale\nlanguage model and query it as textual knowledge. Second, we frame a dialogue\ngeneration model to sequentially generate textual knowledge and a corresponding\nresponse. Empirically, when a dialogue generation model can only access limited\nknowledge, our method outperforms prior work by producing more coherent and\ninformative responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14620v3",
            "date": "2020-04-30 07:48:07+00:00",
            "title": "Universal Dependencies according to BERT: both more specific and more general",
            "authors": [
                "Tomasz Limisiewicz",
                "Rudolf Rosa",
                "David Mare\u010dek"
            ],
            "abstract": "This work focuses on analyzing the form and extent of syntactic abstraction\ncaptured by BERT by extracting labeled dependency trees from self-attentions.\n  Previous work showed that individual BERT heads tend to encode particular\ndependency relation types. We extend these findings by explicitly comparing\nBERT relations to Universal Dependencies (UD) annotations, showing that they\noften do not match one-to-one.\n  We suggest a method for relation identification and syntactic tree\nconstruction. Our approach produces significantly more consistent dependency\ntrees than previous work, showing that it better explains the syntactic\nabstractions in BERT. At the same time, it can be successfully applied with\nonly a minimal amount of supervision and generalizes well across languages.",
            "journal ref": "Findings of the Association for Computational Linguistics: EMNLP\n  2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14623v4",
            "date": "2020-04-30 07:53:20+00:00",
            "title": "Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation",
            "authors": [
                "Atticus Geiger",
                "Kyle Richardson",
                "Christopher Potts"
            ],
            "abstract": "We address whether neural models for Natural Language Inference (NLI) can\nlearn the compositional interactions between lexical entailment and negation,\nusing four methods: the behavioral evaluation methods of (1) challenge test\nsets and (2) systematic generalization tasks, and the structural evaluation\nmethods of (3) probes and (4) interventions. To facilitate this holistic\nevaluation, we present Monotonicity NLI (MoNLI), a new naturalistic dataset\nfocused on lexical entailment and negation. In our behavioral evaluations, we\nfind that models trained on general-purpose NLI datasets fail systematically on\nMoNLI examples containing negation, but that MoNLI fine-tuning addresses this\nfailure. In our structural evaluations, we look for evidence that our\ntop-performing BERT-based model has learned to implement the monotonicity\nalgorithm behind MoNLI. Probes yield evidence consistent with this conclusion,\nand our intervention experiments bolster this, showing that the causal dynamics\nof the model mirror the causal dynamics of this algorithm on subsets of MoNLI.\nThis suggests that the BERT model at least partially embeds a theory of lexical\nentailment and negation at an algorithmic level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14626v2",
            "date": "2020-04-30 08:00:42+00:00",
            "title": "Rethinking Coherence Modeling: Synthetic vs. Downstream Tasks",
            "authors": [
                "Tasnim Mohiuddin",
                "Prathyusha Jwalapuram",
                "Xiang Lin",
                "Shafiq Joty"
            ],
            "abstract": "Although coherence modeling has come a long way in developing novel models,\ntheir evaluation on downstream applications for which they are purportedly\ndeveloped has largely been neglected. With the advancements made by neural\napproaches in applications such as machine translation (MT), summarization and\ndialog systems, the need for coherence evaluation of these tasks is now more\ncrucial than ever. However, coherence models are typically evaluated only on\nsynthetic tasks, which may not be representative of their performance in\ndownstream applications. To investigate how representative the synthetic tasks\nare of downstream use cases, we conduct experiments on benchmarking well-known\ntraditional and neural coherence models on synthetic sentence ordering tasks,\nand contrast this with their performance on three downstream applications:\ncoherence evaluation for MT and summarization, and next utterance prediction in\nretrieval-based dialog. Our results demonstrate a weak correlation between the\nmodel performances in the synthetic tasks and the downstream applications,\n{motivating alternate training and evaluation methods for coherence models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14649v1",
            "date": "2020-04-30 09:11:38+00:00",
            "title": "Capsule-Transformer for Neural Machine Translation",
            "authors": [
                "Sufeng Duan",
                "Juncheng Cao",
                "Hai Zhao"
            ],
            "abstract": "Transformer hugely benefits from its key design of the multi-head\nself-attention network (SAN), which extracts information from various\nperspectives through transforming the given input into different subspaces.\nHowever, its simple linear transformation aggregation strategy may still\npotentially fail to fully capture deeper contextualized information. In this\npaper, we thus propose the capsule-Transformer, which extends the linear\ntransformation into a more general capsule routing algorithm by taking SAN as a\nspecial case of capsule network. So that the resulted capsule-Transformer is\ncapable of obtaining a better attention distribution representation of the\ninput sequence via information aggregation among different heads and words.\nSpecifically, we see groups of attention weights in SAN as low layer capsules.\nBy applying the iterative capsule routing algorithm they can be further\naggregated into high layer capsules which contain deeper contextualized\ninformation. Experimental results on the widely-used machine translation\ndatasets show our proposed capsule-Transformer outperforms strong Transformer\nbaseline significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14675v1",
            "date": "2020-04-30 10:29:37+00:00",
            "title": "End-to-End Neural Word Alignment Outperforms GIZA++",
            "authors": [
                "Thomas Zenkel",
                "Joern Wuebker",
                "John DeNero"
            ],
            "abstract": "Word alignment was once a core unsupervised learning task in natural language\nprocessing because of its essential role in training statistical machine\ntranslation (MT) models. Although unnecessary for training neural MT models,\nword alignment still plays an important role in interactive applications of\nneural machine translation, such as annotation transfer and lexicon injection.\nWhile statistical MT methods have been replaced by neural approaches with\nsuperior performance, the twenty-year-old GIZA++ toolkit remains a key\ncomponent of state-of-the-art word alignment systems. Prior work on neural word\nalignment has only been able to outperform GIZA++ by using its output during\ntraining. We present the first end-to-end neural word alignment method that\nconsistently outperforms GIZA++ on three data sets. Our approach repurposes a\nTransformer model trained for supervised translation to also serve as an\nunsupervised word alignment model in a manner that is tightly integrated and\ndoes not affect translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14693v1",
            "date": "2020-04-30 11:19:04+00:00",
            "title": "Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders",
            "authors": [
                "Yanbin Zhao",
                "Lu Chen",
                "Zhi Chen",
                "Kai Yu"
            ],
            "abstract": "Text simplification (TS) rephrases long sentences into simplified variants\nwhile preserving inherent semantics. Traditional sequence-to-sequence models\nheavily rely on the quantity and quality of parallel sentences, which limits\ntheir applicability in different languages and domains. This work investigates\nhow to leverage large amounts of unpaired corpora in TS task. We adopt the\nback-translation architecture in unsupervised machine translation (NMT),\nincluding denoising autoencoders for language modeling and automatic generation\nof parallel data by iterative back-translation. However, it is non-trivial to\ngenerate appropriate complex-simple pair if we directly treat the set of simple\nand complex corpora as two different languages, since the two types of\nsentences are quite similar and it is hard for the model to capture the\ncharacteristics in different types of sentences. To tackle this problem, we\npropose asymmetric denoising methods for sentences with separate complexity.\nWhen modeling simple and complex sentences with autoencoders, we introduce\ndifferent types of noise into the training process. Such a method can\nsignificantly improve the simplification performance. Our model can be trained\nin both unsupervised and semi-supervised manner. Automatic and human\nevaluations show that our unsupervised model outperforms the previous systems,\nand with limited supervision, our model can perform competitively with multiple\nstate-of-the-art simplification systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14710v1",
            "date": "2020-04-30 12:02:33+00:00",
            "title": "Towards Unsupervised Language Understanding and Generation by Joint Dual Learning",
            "authors": [
                "Shang-Yu Su",
                "Chao-Wei Huang",
                "Yun-Nung Chen"
            ],
            "abstract": "In modular dialogue systems, natural language understanding (NLU) and natural\nlanguage generation (NLG) are two critical components, where NLU extracts the\nsemantics from the given texts and NLG is to construct corresponding natural\nlanguage sentences based on the input semantic representations. However, the\ndual property between understanding and generation has been rarely explored.\nThe prior work is the first attempt that utilized the duality between NLU and\nNLG to improve the performance via a dual supervised learning framework.\nHowever, the prior work still learned both components in a supervised manner,\ninstead, this paper introduces a general learning framework to effectively\nexploit such duality, providing flexibility of incorporating both supervised\nand unsupervised learning algorithms to train language understanding and\ngeneration models in a joint fashion. The benchmark experiments demonstrate\nthat the proposed approach is capable of boosting the performance of both NLU\nand NLG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14769v2",
            "date": "2020-04-30 13:34:04+00:00",
            "title": "Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation",
            "authors": [
                "Kun Li",
                "Chengbo Chen",
                "Xiaojun Quan",
                "Qing Ling",
                "Yan Song"
            ],
            "abstract": "Aspect term extraction aims to extract aspect terms from review texts as\nopinion targets for sentiment analysis. One of the big challenges with this\ntask is the lack of sufficient annotated data. While data augmentation is\npotentially an effective technique to address the above issue, it is\nuncontrollable as it may change aspect words and aspect labels unexpectedly. In\nthis paper, we formulate the data augmentation as a conditional generation\ntask: generating a new sentence while preserving the original opinion targets\nand labels. We propose a masked sequence-to-sequence method for conditional\naugmentation of aspect term extraction. Unlike existing augmentation\napproaches, ours is controllable and allows us to generate more diversified\nsentences. Experimental results confirm that our method alleviates the data\nscarcity problem significantly. It also effectively boosts the performances of\nseveral current models for aspect term extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14781v2",
            "date": "2020-04-30 13:50:34+00:00",
            "title": "Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion",
            "authors": [
                "Bo Wang",
                "Tao Shen",
                "Guodong Long",
                "Tianyi Zhou",
                "Yi Chang"
            ],
            "abstract": "Human-curated knowledge graphs provide critical supportive information to\nvarious natural language processing tasks, but these graphs are usually\nincomplete, urging auto-completion of them. Prevalent graph embedding\napproaches, e.g., TransE, learn structured knowledge via representing graph\nelements into dense embeddings and capturing their triple-level relationship\nwith spatial distance. However, they are hardly generalizable to the elements\nnever visited in training and are intrinsically vulnerable to graph\nincompleteness. In contrast, textual encoding approaches, e.g., KG-BERT, resort\nto graph triple's text and triple-level contextualized representations. They\nare generalizable enough and robust to the incompleteness, especially when\ncoupled with pre-trained encoders. But two major drawbacks limit the\nperformance: (1) high overheads due to the costly scoring of all possible\ntriples in inference, and (2) a lack of structured knowledge in the textual\nencoder. In this paper, we follow the textual encoding paradigm and aim to\nalleviate its drawbacks by augmenting it with graph embedding techniques -- a\ncomplementary hybrid of both paradigms. Specifically, we partition each triple\ninto two asymmetric parts as in translation-based graph embedding approach, and\nencode both parts into contextualized representations by a Siamese-style\ntextual encoder. Built upon the representations, our model employs both\ndeterministic classifier and spatial measurement for representation and\nstructure learning respectively. Moreover, we develop a self-adaptive ensemble\nscheme to further improve the performance by incorporating triple scores from\nan existing graph embedding model. In experiments, we achieve state-of-the-art\nperformance on three benchmarks and a zero-shot dataset for link prediction,\nwith highlights of inference costs reduced by 1-2 orders of magnitude compared\nto a textual encoding method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14786v3",
            "date": "2020-04-30 14:02:29+00:00",
            "title": "Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT",
            "authors": [
                "Zhiyong Wu",
                "Yun Chen",
                "Ben Kao",
                "Qun Liu"
            ],
            "abstract": "By introducing a small set of additional parameters, a probe learns to solve\nspecific linguistic tasks (e.g., dependency parsing) in a supervised manner\nusing feature representations (e.g., contextualized embeddings). The\neffectiveness of such probing tasks is taken as evidence that the pre-trained\nmodel encodes linguistic knowledge. However, this approach of evaluating a\nlanguage model is undermined by the uncertainty of the amount of knowledge that\nis learned by the probe itself. Complementary to those works, we propose a\nparameter-free probing technique for analyzing pre-trained language models\n(e.g., BERT). Our method does not require direct supervision from the probing\ntasks, nor do we introduce additional parameters to the probing process. Our\nexperiments on BERT show that syntactic trees recovered from BERT using our\nmethod are significantly better than linguistically-uninformed baselines. We\nfurther feed the empirically induced dependency structures into a downstream\nsentiment classification task and find its improvement compatible with or even\nsuperior to a human-designed dependency schema.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14788v1",
            "date": "2020-04-30 14:05:26+00:00",
            "title": "Character-Level Translation with Self-attention",
            "authors": [
                "Yingqiang Gao",
                "Nikola I. Nikolov",
                "Yuhuang Hu",
                "Richard H. R. Hahnloser"
            ],
            "abstract": "We explore the suitability of self-attention models for character-level\nneural machine translation. We test the standard transformer model, as well as\na novel variant in which the encoder block combines information from nearby\ncharacters using convolutions. We perform extensive experiments on WMT and UN\ndatasets, testing both bilingual and multilingual translation to English using\nup to three input languages (French, Spanish, and Chinese). Our transformer\nvariant consistently outperforms the standard transformer at the\ncharacter-level and converges faster while learning more robust character-level\nalignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14797v1",
            "date": "2020-04-30 14:08:50+00:00",
            "title": "STARC: Structured Annotations for Reading Comprehension",
            "authors": [
                "Yevgeni Berzak",
                "Jonathan Malmaud",
                "Roger Levy"
            ],
            "abstract": "We present STARC (Structured Annotations for Reading Comprehension), a new\nannotation framework for assessing reading comprehension with multiple choice\nquestions. Our framework introduces a principled structure for the answer\nchoices and ties them to textual span annotations. The framework is implemented\nin OneStopQA, a new high-quality dataset for evaluation and analysis of reading\ncomprehension in English. We use this dataset to demonstrate that STARC can be\nleveraged for a key new application for the development of SAT-like reading\ncomprehension materials: automatic annotation quality probing via span ablation\nexperiments. We further show that it enables in-depth analyses and comparisons\nbetween machine and human reading comprehension behavior, including error\ndistributions and guessing ability. Our experiments also reveal that the\nstandard multiple choice dataset in NLP, RACE, is limited in its ability to\nmeasure reading comprehension. 47% of its questions can be guessed by machines\nwithout accessing the passage, and 18% are unanimously judged by humans as not\nhaving a unique correct answer. OneStopQA provides an alternative test set for\nreading comprehension which alleviates these shortcomings and has a\nsubstantially higher human ceiling performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14813v2",
            "date": "2020-04-30 14:16:19+00:00",
            "title": "ENT-DESC: Entity Description Generation by Exploring Knowledge Graph",
            "authors": [
                "Liying Cheng",
                "Dekun Wu",
                "Lidong Bing",
                "Yan Zhang",
                "Zhanming Jie",
                "Wei Lu",
                "Luo Si"
            ],
            "abstract": "Previous works on knowledge-to-text generation take as input a few RDF\ntriples or key-value pairs conveying the knowledge of some entities to generate\na natural language description. Existing datasets, such as WIKIBIO, WebNLG, and\nE2E, basically have a good alignment between an input triple/pair set and its\noutput text. However, in practice, the input knowledge could be more than\nenough, since the output description may only cover the most significant\nknowledge. In this paper, we introduce a large-scale and challenging dataset to\nfacilitate the study of such a practical scenario in KG-to-text. Our dataset\ninvolves retrieving abundant knowledge of various types of main entities from a\nlarge knowledge graph (KG), which makes the current graph-to-sequence models\nseverely suffer from the problems of information loss and parameter explosion\nwhile generating the descriptions. We address these challenges by proposing a\nmulti-graph structure that is able to represent the original graph information\nmore comprehensively. Furthermore, we also incorporate aggregation methods that\nlearn to extract the rich graph information. Extensive experiments demonstrate\nthe effectiveness of our model architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14821v2",
            "date": "2020-04-30 14:27:59+00:00",
            "title": "Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation",
            "authors": [
                "Shoetsu Sato",
                "Jin Sakuma",
                "Naoki Yoshinaga",
                "Masashi Toyoda",
                "Masaru Kitsuregawa"
            ],
            "abstract": "Neural network methods exhibit strong performance only in a few resource-rich\ndomains. Practitioners, therefore, employ domain adaptation from resource-rich\ndomains that are, in most cases, distant from the target domain. Domain\nadaptation between distant domains (e.g., movie subtitles and research papers),\nhowever, cannot be performed effectively due to mismatches in vocabulary; it\nwill encounter many domain-specific words (e.g., \"angstrom\") and words whose\nmeanings shift across domains(e.g., \"conductor\"). In this study, aiming to\nsolve these vocabulary mismatches in domain adaptation for neural machine\ntranslation (NMT), we propose vocabulary adaptation, a simple method for\neffective fine-tuning that adapts embedding layers in a given pre-trained NMT\nmodel to the target domain. Prior to fine-tuning, our method replaces the\nembedding layers of the NMT model by projecting general word embeddings induced\nfrom monolingual data in a target domain onto a source-domain embedding space.\nExperimental results indicate that our method improves the performance of\nconventional fine-tuning by 3.86 and 3.28 BLEU points in En-Ja and De-En\ntranslation, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14848v1",
            "date": "2020-04-30 15:00:21+00:00",
            "title": "Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection",
            "authors": [
                "Momchil Hardalov",
                "Ivan Koychev",
                "Preslav Nakov"
            ],
            "abstract": "Detecting the user's intent and finding the corresponding slots among the\nutterance's words are important tasks in natural language understanding. Their\ninterconnected nature makes their joint modeling a standard part of training\nsuch models. Moreover, data scarceness and specialized vocabularies pose\nadditional challenges. Recently, the advances in pre-trained language models,\nnamely contextualized models such as ELMo and BERT have revolutionized the\nfield by tapping the potential of training very large models with just a few\nsteps of fine-tuning on a task-specific dataset. Here, we leverage such model,\nnamely BERT, and we design a novel architecture on top it. Moreover, we propose\nan intent pooling attention mechanism, and we reinforce the slot filling task\nby fusing intent distributions, word features, and token representations. The\nexperimental results on standard datasets show that our model outperforms both\nthe current non-BERT state of the art as well as some stronger BERT-based\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14855v1",
            "date": "2020-04-30 15:07:37+00:00",
            "title": "TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task",
            "authors": [
                "Christoph Alt",
                "Aleksandra Gabryszak",
                "Leonhard Hennig"
            ],
            "abstract": "TACRED (Zhang et al., 2017) is one of the largest, most widely used\ncrowdsourced datasets in Relation Extraction (RE). But, even with recent\nadvances in unsupervised pre-training and knowledge enhanced neural RE, models\nstill show a high error rate. In this paper, we investigate the questions: Have\nwe reached a performance ceiling or is there still room for improvement? And\nhow do crowd annotations, dataset, and models contribute to this error rate? To\nanswer these questions, we first validate the most challenging 5K examples in\nthe development and test sets using trained annotators. We find that label\nerrors account for 8% absolute F1 test error, and that more than 50% of the\nexamples need to be relabeled. On the relabeled test set the average F1 score\nof a large baseline model set improves from 62.1 to 70.1. After validation, we\nanalyze misclassifications on the challenging instances, categorize them into\nlinguistically motivated error groups, and verify the resulting error\nhypotheses on three state-of-the-art RE models. We show that two groups of\nambiguous relations are responsible for most of the remaining errors and that\nmodels may adopt shallow heuristics on the dataset when entities are not\nmasked.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14871v1",
            "date": "2020-04-30 15:15:40+00:00",
            "title": "Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization",
            "authors": [
                "Libo Qin",
                "Minheng Ni",
                "Yue Zhang",
                "Wanxiang Che",
                "Yangming Li",
                "Ting Liu"
            ],
            "abstract": "Spoken language understanding has been addressed as a supervised learning\nproblem, where a set of training data is available for each domain. However,\nannotating data for each domain is both financially costly and non-scalable so\nwe should fully utilize information across all domains. One existing approach\nsolves the problem by conducting multi-domain learning, using shared parameters\nfor joint training across domains. We propose to improve the parameterization\nof this method by using domain-specific and task-specific model parameters to\nimprove knowledge learning and transfer. Experiments on 5 domains show that our\nmodel is more effective for multi-domain SLU and obtain the best results. In\naddition, we show its transferability by outperforming the prior best model by\n12.4\\% when adapting to a new domain with little data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14876v1",
            "date": "2020-04-30 15:24:43+00:00",
            "title": "Analyzing the Surprising Variability in Word Embedding Stability Across Languages",
            "authors": [
                "Laura Burdick",
                "Jonathan K. Kummerfeld",
                "Rada Mihalcea"
            ],
            "abstract": "Word embeddings are powerful representations that form the foundation of many\nnatural language processing architectures and tasks, both in English and in\nother languages. To gain further insight into word embeddings in multiple\nlanguages, we explore their stability, defined as the overlap between the\nnearest neighbors of a word in different embedding spaces. We discuss\nlinguistic properties that are related to stability, drawing out insights about\nhow morphological and other features relate to stability. This has implications\nfor the usage of embeddings, particularly in research that uses embeddings to\nstudy language trends.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14900v1",
            "date": "2020-04-30 15:58:34+00:00",
            "title": "MLSUM: The Multilingual Summarization Corpus",
            "authors": [
                "Thomas Scialom",
                "Paul-Alexis Dray",
                "Sylvain Lamprier",
                "Benjamin Piwowarski",
                "Jacopo Staiano"
            ],
            "abstract": "We present MLSUM, the first large-scale MultiLingual SUMmarization dataset.\nObtained from online newspapers, it contains 1.5M+ article/summary pairs in\nfive different languages -- namely, French, German, Spanish, Russian, Turkish.\nTogether with English newspapers from the popular CNN/Daily mail dataset, the\ncollected data form a large scale multilingual dataset which can enable new\nresearch directions for the text summarization community. We report\ncross-lingual comparative analyses based on state-of-the-art systems. These\nhighlight existing biases which motivate the use of a multi-lingual dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14907v1",
            "date": "2020-04-30 16:06:02+00:00",
            "title": "You are right. I am ALARMED -- But by Climate Change Counter Movement",
            "authors": [
                "Shraey Bhatia",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "The world is facing the challenge of climate crisis. Despite the consensus in\nscientific community about anthropogenic global warming, the web is flooded\nwith articles spreading climate misinformation. These articles are carefully\nconstructed by climate change counter movement (cccm) organizations to\ninfluence the narrative around climate change. We revisit the literature on\nclimate misinformation in social sciences and repackage it to introduce in the\ncommunity of NLP. Despite considerable work in detection of fake news, there is\nno misinformation dataset available that is specific to the domain.of climate\nchange. We try to bridge this gap by scraping and releasing articles with known\nclimate change misinformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14911v1",
            "date": "2020-04-30 16:09:22+00:00",
            "title": "Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation",
            "authors": [
                "Asa Cooper Stickland",
                "Xian Li",
                "Marjan Ghazvininejad"
            ],
            "abstract": "There has been recent success in pre-training on monolingual data and\nfine-tuning on Machine Translation (MT), but it remains unclear how to best\nleverage a pre-trained model for a given MT task. This paper investigates the\nbenefits and drawbacks of freezing parameters, and adding new ones, when\nfine-tuning a pre-trained model on MT. We focus on 1) Fine-tuning a model\ntrained only on English monolingual data, BART. 2) Fine-tuning a model trained\non monolingual data from 25 languages, mBART. For BART we get the best\nperformance by freezing most of the model parameters, and adding extra\npositional embeddings. For mBART we match the performance of naive fine-tuning\nfor most language pairs, and outperform it for Nepali to English (0.5 BLEU) and\nCzech to English (0.6 BLEU), all with a lower memory cost at training time.\nWhen constraining ourselves to an out-of-domain training set for Vietnamese to\nEnglish we outperform the fine-tuning baseline by 0.9 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14914v2",
            "date": "2020-04-30 16:18:18+00:00",
            "title": "Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!",
            "authors": [
                "Suzanna Sia",
                "Ayush Dalmia",
                "Sabrina J. Mielke"
            ],
            "abstract": "Topic models are a useful analysis tool to uncover the underlying themes\nwithin document collections. The dominant approach is to use probabilistic\ntopic models that posit a generative story, but in this paper we propose an\nalternative way to obtain topics: clustering pre-trained word embeddings while\nincorporating document information for weighted clustering and reranking top\nwords. We provide benchmarks for the combination of different word embeddings\nand clustering algorithms, and analyse their performance under dimensionality\nreduction with PCA. The best performing combination for our approach performs\nas well as classical topic models, but with lower runtime and computational\ncomplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14923v2",
            "date": "2020-04-30 16:25:39+00:00",
            "title": "Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations",
            "authors": [
                "Arturo Oncevay",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "Sparse language vectors from linguistic typology databases and learned\nembeddings from tasks like multilingual machine translation have been\ninvestigated in isolation, without analysing how they could benefit from each\nother's language characterisation. We propose to fuse both views using singular\nvector canonical correlation analysis and study what kind of information is\ninduced from each source. By inferring typological features and language\nphylogenies, we observe that our representations embed typology and strengthen\ncorrelations with language relationships. We then take advantage of our\nmulti-view language vector space for multilingual machine translation, where we\nachieve competitive overall translation accuracy in tasks that require\ninformation about language similarities, such as language clustering and\nranking candidates for multilingual transfer. With our method, which is also\nreleased as a tool, we can easily project and assess new languages without\nexpensive retraining of massive multilingual or ranking models, which are major\ndisadvantages of related approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14927v2",
            "date": "2020-04-30 16:28:19+00:00",
            "title": "Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation",
            "authors": [
                "Dario Stojanovski",
                "Alexander Fraser"
            ],
            "abstract": "Achieving satisfying performance in machine translation on domains for which\nthere is no training data is challenging. Traditional supervised domain\nadaptation is not suitable for addressing such zero-resource domains because it\nrelies on in-domain parallel data. We show that when in-domain parallel data is\nnot available, access to document-level context enables better capturing of\ndomain generalities compared to only having access to a single sentence. Having\naccess to more information provides a more reliable domain estimation. We\npresent two document-level Transformer models which are capable of using large\ncontext sizes and we compare these models against strong Transformer baselines.\nWe obtain improvements for the two zero resource domains we study. We\nadditionally provide an analysis where we vary the amount of context and look\nat the case where in-domain data is available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14928v3",
            "date": "2020-04-30 16:29:56+00:00",
            "title": "Language Model Prior for Low-Resource Neural Machine Translation",
            "authors": [
                "Christos Baziotis",
                "Barry Haddow",
                "Alexandra Birch"
            ],
            "abstract": "The scarcity of large parallel corpora is an important obstacle for neural\nmachine translation. A common solution is to exploit the knowledge of language\nmodels (LM) trained on abundant monolingual data. In this work, we propose a\nnovel approach to incorporate a LM as prior in a neural translation model (TM).\nSpecifically, we add a regularization term, which pushes the output\ndistributions of the TM to be probable under the LM prior, while avoiding wrong\npredictions when the TM \"disagrees\" with the LM. This objective relates to\nknowledge distillation, where the LM can be viewed as teaching the TM about the\ntarget language. The proposed approach does not compromise decoding speed,\nbecause the LM is used only at training time, unlike previous work that\nrequires it during inference. We present an analysis of the effects that\ndifferent methods have on the distributions of the TM. Results on two\nlow-resource machine translation datasets show clear improvements even with\nlimited monolingual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14959v1",
            "date": "2020-04-30 17:08:03+00:00",
            "title": "Natural Language Premise Selection: Finding Supporting Statements for Mathematical Text",
            "authors": [
                "Deborah Ferreira",
                "Andre Freitas"
            ],
            "abstract": "Mathematical text is written using a combination of words and mathematical\nexpressions. This combination, along with a specific way of structuring\nsentences makes it challenging for state-of-art NLP tools to understand and\nreason on top of mathematical discourse. In this work, we propose a new NLP\ntask, the natural premise selection, which is used to retrieve supporting\ndefinitions and supporting propositions that are useful for generating an\ninformal mathematical proof for a particular statement. We also make available\na dataset, NL-PS, which can be used to evaluate different approaches for the\nnatural premise selection task. Using different baselines, we demonstrate the\nunderlying interpretation challenges associated with the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14961v1",
            "date": "2020-04-30 17:09:51+00:00",
            "title": "Mutlitask Learning for Cross-Lingual Transfer of Semantic Dependencies",
            "authors": [
                "Maryam Aminian",
                "Mohammad Sadegh Rasooli",
                "Mona Diab"
            ],
            "abstract": "We describe a method for developing broad-coverage semantic dependency\nparsers for languages for which no semantically annotated resource is\navailable. We leverage a multitask learning framework coupled with an\nannotation projection method. We transfer supervised semantic dependency parse\nannotations from a rich-resource language to a low-resource language through\nparallel data, and train a semantic parser on projected data. We make use of\nsupervised syntactic parsing as an auxiliary task in a multitask learning\nframework, and show that with different multitask learning settings, we\nconsistently improve over the single-task baseline. In the setting in which\nEnglish is the source, and Czech is the target language, our best multitask\nmodel improves the labeled F1 score over the single-task baseline by 1.8 in the\nin-domain SemEval data (Oepen et al., 2015), as well as 2.5 in the\nout-of-domain test set. Moreover, we observe that syntactic and semantic\ndependency direction match is an important factor in improving the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14963v3",
            "date": "2020-04-30 17:12:52+00:00",
            "title": "Data and Representation for Turkish Natural Language Inference",
            "authors": [
                "Emrah Budur",
                "R\u0131za \u00d6z\u00e7elik",
                "Tunga G\u00fcng\u00f6r",
                "Christopher Potts"
            ],
            "abstract": "Large annotated datasets in NLP are overwhelmingly in English. This is an\nobstacle to progress in other languages. Unfortunately, obtaining new annotated\nresources for each task in each language would be prohibitively expensive. At\nthe same time, commercial machine translation systems are now robust. Can we\nleverage these systems to translate English-language datasets automatically? In\nthis paper, we offer a positive response for natural language inference (NLI)\nin Turkish. We translated two large English NLI datasets into Turkish and had a\nteam of experts validate their translation quality and fidelity to the original\nlabels. Using these datasets, we address core issues of representation for\nTurkish NLI. We find that in-language embeddings are essential and that\nmorphological parsing can be avoided where the training set is large. Finally,\nwe show that models trained on our machine-translated datasets are successful\non human-translated evaluation sets. We share all code, models, and data\npublicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14967v2",
            "date": "2020-04-30 17:16:31+00:00",
            "title": "PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking",
            "authors": [
                "Hannah Rashkin",
                "Asli Celikyilmaz",
                "Yejin Choi",
                "Jianfeng Gao"
            ],
            "abstract": "We propose the task of outline-conditioned story generation: given an outline\nas a set of phrases that describe key characters and events to appear in a\nstory, the task is to generate a coherent narrative that is consistent with the\nprovided outline. This task is challenging as the input only provides a rough\nsketch of the plot, and thus, models need to generate a story by interweaving\nthe key points provided in the outline. This requires the model to keep track\nof the dynamic states of the latent plot, conditioning on the input outline\nwhile generating the full story. We present PlotMachines, a neural narrative\nmodel that learns to transform an outline into a coherent story by tracking the\ndynamic plot states. In addition, we enrich PlotMachines with high-level\ndiscourse structure so that the model can learn different writing styles\ncorresponding to different parts of the narrative. Comprehensive experiments\nover three fiction and non-fiction datasets demonstrate that large-scale\nlanguage models, such as GPT-2 and Grover, despite their impressive generation\nperformance, are not sufficient in generating coherent narratives for the given\noutline, and dynamic plot state tracking is important for composing narratives\nwith tighter, more consistent plots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14974v6",
            "date": "2020-04-30 17:22:57+00:00",
            "title": "Fact or Fiction: Verifying Scientific Claims",
            "authors": [
                "David Wadden",
                "Shanchuan Lin",
                "Kyle Lo",
                "Lucy Lu Wang",
                "Madeleine van Zuylen",
                "Arman Cohan",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce scientific claim verification, a new task to select abstracts\nfrom the research literature containing evidence that SUPPORTS or REFUTES a\ngiven scientific claim, and to identify rationales justifying each decision. To\nstudy this task, we construct SciFact, a dataset of 1.4K expert-written\nscientific claims paired with evidence-containing abstracts annotated with\nlabels and rationales. We develop baseline models for SciFact, and demonstrate\nthat simple domain adaptation techniques substantially improve performance\ncompared to models trained on Wikipedia or political news. We show that our\nsystem is able to verify claims related to COVID-19 by identifying evidence\nfrom the CORD-19 corpus. Our experiments indicate that SciFact will provide a\nchallenging testbed for the development of new systems designed to retrieve and\nreason over corpora containing specialized domain knowledge. Data and code for\nthis new task are publicly available at https://github.com/allenai/scifact. A\nleaderboard and COVID-19 fact-checking demo are available at\nhttps://scifact.apps.allenai.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14979v2",
            "date": "2020-04-30 17:29:17+00:00",
            "title": "Paraphrasing vs Coreferring: Two Sides of the Same Coin",
            "authors": [
                "Yehudit Meged",
                "Avi Caciularu",
                "Vered Shwartz",
                "Ido Dagan"
            ],
            "abstract": "We study the potential synergy between two different NLP tasks, both\nconfronting predicate lexical variability: identifying predicate paraphrases,\nand event coreference resolution. First, we used annotations from an event\ncoreference dataset as distant supervision to re-score heuristically-extracted\npredicate paraphrases. The new scoring gained more than 18 points in average\nprecision upon their ranking by the original scoring method. Then, we used the\nsame re-ranking features as additional inputs to a state-of-the-art event\ncoreference resolution model, which yielded modest but consistent improvements\nto the model's performance. The results suggest a promising direction to\nleverage data and models for each of the tasks to the benefit of the other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14983v2",
            "date": "2020-04-30 17:31:16+00:00",
            "title": "Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation",
            "authors": [
                "Giuseppe Russo",
                "Nora Hollenstein",
                "Claudiu Musat",
                "Ce Zhang"
            ],
            "abstract": "We introduce CGA, a conditional VAE architecture, to control, generate, and\naugment text. CGA is able to generate natural English sentences controlling\nmultiple semantic and syntactic attributes by combining adversarial learning\nwith a context-aware loss and a cyclical word dropout routine. We demonstrate\nthe value of the individual model components in an ablation study. The\nscalability of our approach is ensured through a single discriminator,\nindependently of the number of attributes. We show high quality, diversity and\nattribute control in the generated sentences through a series of automatic and\nhuman assessments. As the main application of our work, we test the potential\nof this new NLG model in a data augmentation scenario. In a downstream NLP\ntask, the sentences generated by our CGA model show significant improvements\nover a strong baseline, and a classification performance often comparable to\nadding same amount of additional real data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14989v3",
            "date": "2020-04-30 17:34:52+00:00",
            "title": "A Study in Improving BLEU Reference Coverage with Diverse Automatic Paraphrasing",
            "authors": [
                "Rachel Bawden",
                "Biao Zhang",
                "Lisa Yankovskaya",
                "Andre T\u00e4ttar",
                "Matt Post"
            ],
            "abstract": "We investigate a long-perceived shortcoming in the typical use of BLEU: its\nreliance on a single reference. Using modern neural paraphrasing techniques, we\nstudy whether automatically generating additional diverse references can\nprovide better coverage of the space of valid translations and thereby improve\nits correlation with human judgments. Our experiments on the into-English\nlanguage directions of the WMT19 metrics task (at both the system and sentence\nlevel) show that using paraphrased references does generally improve BLEU, and\nwhen it does, the more diverse the better. However, we also show that better\nresults could be achieved if those paraphrases were to specifically target the\nparts of the space most relevant to the MT outputs being evaluated. Moreover,\nthe gains remain slight even when human paraphrases are used, suggesting\ninherent limitations to BLEU's capacity to correctly exploit multiple\nreferences. Surprisingly, we also find that adequacy appears to be less\nimportant, as shown by the high results of a strong sampling approach, which\neven beats human paraphrases when used with sentence-level BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14999v1",
            "date": "2020-04-30 17:45:16+00:00",
            "title": "A Matter of Framing: The Impact of Linguistic Formalism on Probing Results",
            "authors": [
                "Ilia Kuznetsov",
                "Iryna Gurevych"
            ],
            "abstract": "Deep pre-trained contextualized encoders like BERT (Delvin et al., 2019)\ndemonstrate remarkable performance on a range of downstream tasks. A recent\nline of research in probing investigates the linguistic knowledge implicitly\nlearned by these models during pre-training. While most work in probing\noperates on the task level, linguistic tasks are rarely uniform and can be\nrepresented in a variety of formalisms. Any linguistics-based probing study\nthereby inevitably commits to the formalism used to annotate the underlying\ndata. Can the choice of formalism affect probing results? To investigate, we\nconduct an in-depth cross-formalism layer probing study in role semantics. We\nfind linguistically meaningful differences in the encoding of semantic role-\nand proto-role information by BERT depending on the formalism and demonstrate\nthat layer probing can detect subtle differences between the implementations of\nthe same linguistic formalism. Our results suggest that linguistic formalism is\nan important dimension in probing studies, along with the commonly used\ncross-task and cross-lingual experimental settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15003v3",
            "date": "2020-04-30 17:48:42+00:00",
            "title": "Word Rotator's Distance",
            "authors": [
                "Sho Yokoi",
                "Ryo Takahashi",
                "Reina Akama",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "A key principle in assessing textual similarity is measuring the degree of\nsemantic overlap between two texts by considering the word alignment. Such\nalignment-based approaches are intuitive and interpretable; however, they are\nempirically inferior to the simple cosine similarity between general-purpose\nsentence vectors. To address this issue, we focus on and demonstrate the fact\nthat the norm of word vectors is a good proxy for word importance, and their\nangle is a good proxy for word similarity. Alignment-based approaches do not\ndistinguish them, whereas sentence-vector approaches automatically use the norm\nas the word importance. Accordingly, we propose a method that first decouples\nword vectors into their norm and direction, and then computes alignment-based\nsimilarity using earth mover's distance (i.e., optimal transport cost), which\nwe refer to as word rotator's distance. Besides, we find how to grow the norm\nand direction of word vectors (vector converter), which is a new systematic\napproach derived from sentence-vector estimation methods. On several textual\nsimilarity datasets, the combination of these simple proposed methods\noutperformed not only alignment-based approaches but also strong baselines. The\nsource code is available at https://github.com/eumesy/wrd",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15006v2",
            "date": "2020-04-30 17:51:08+00:00",
            "title": "Template Guided Text Generation for Task-Oriented Dialogue",
            "authors": [
                "Mihir Kale",
                "Abhinav Rastogi"
            ],
            "abstract": "Virtual assistants such as Google Assistant, Amazon Alexa, and Apple Siri\nenable users to interact with a large number of services and APIs on the web\nusing natural language. In this work, we investigate two methods for Natural\nLanguage Generation (NLG) using a single domain-independent model across a\nlarge number of APIs. First, we propose a schema-guided approach which\nconditions the generation on a schema describing the API in natural language.\nOur second method investigates the use of a small number of templates, growing\nlinearly in number of slots, to convey the semantics of the API. To generate\nutterances for an arbitrary slot combination, a few simple templates are first\nconcatenated to give a semantically correct, but possibly incoherent and\nungrammatical utterance. A pre-trained language model is subsequently employed\nto rewrite it into coherent, natural sounding text. Through automatic metrics\nand human evaluation, we show that our method improves over strong baselines,\nis robust to out-of-domain inputs and shows improved sample efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15008v1",
            "date": "2020-04-30 17:52:11+00:00",
            "title": "Lexical Semantic Recognition",
            "authors": [
                "Nelson F. Liu",
                "Daniel Hershcovich",
                "Michael Kranzlein",
                "Nathan Schneider"
            ],
            "abstract": "Segmentation and (segment) labeling are generally treated separately in\nlexical semantics, raising issues due to their close inter-dependence and\nnecessitating joint annotation. We therefore investigate the lexical semantic\nrecognition task of multiword expression segmentation and supersense\ndisambiguation, unifying several previously-disparate styles of lexical\nsemantic annotation. We evaluate a neural CRF model along all annotation axes\navailable in version 4.3 of the STREUSLE corpus: lexical unit segmentation\n(multiword expressions), word-level syntactic tags, and supersense classes for\nnoun, verb, and preposition/possessive units. As the label set generalizes that\nof previous tasks (DiMSUM, PARSEME), we additionally evaluate how well the\nmodel generalizes to those test sets, with encouraging results. By establishing\nbaseline models and evaluation metrics, we pave the way for comprehensive and\naccurate modeling of lexical semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15011v3",
            "date": "2020-04-30 17:56:18+00:00",
            "title": "TLDR: Extreme Summarization of Scientific Documents",
            "authors": [
                "Isabel Cachola",
                "Kyle Lo",
                "Arman Cohan",
                "Daniel S. Weld"
            ],
            "abstract": "We introduce TLDR generation, a new form of extreme summarization, for\nscientific papers. TLDR generation involves high source compression and\nrequires expert background knowledge and understanding of complex\ndomain-specific language. To facilitate study on this task, we introduce\nSciTLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers. SciTLDR\ncontains both author-written and expert-derived TLDRs, where the latter are\ncollected using a novel annotation protocol that produces high-quality\nsummaries while minimizing annotation burden. We propose CATTS, a simple yet\neffective learning strategy for generating TLDRs that exploits titles as an\nauxiliary training signal. CATTS improves upon strong baselines under both\nautomated metrics and human evaluations. Data and code are publicly available\nat https://github.com/allenai/scitldr.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15012v2",
            "date": "2020-04-30 17:56:30+00:00",
            "title": "Does Data Augmentation Improve Generalization in NLP?",
            "authors": [
                "Rohan Jha",
                "Charles Lovering",
                "Ellie Pavlick"
            ],
            "abstract": "Neural models often exploit superficial features to achieve good performance,\nrather than deriving more general features. Overcoming this tendency is a\ncentral challenge in areas such as representation learning and ML fairness.\nRecent work has proposed using data augmentation, i.e., generating training\nexamples where the superficial features fail, as a means of encouraging models\nto prefer the stronger features. We design a series of toy learning problems to\ntest the hypothesis that data augmentation leads models to unlearn weaker\nheuristics, but not to learn stronger features in their place. We find partial\nsupport for this hypothesis: Data augmentation often hurts before it helps, and\nit is less effective when the preferred strong feature is much more difficult\nto extract than the competing weak feature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15016v3",
            "date": "2020-04-30 17:57:27+00:00",
            "title": "WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context",
            "authors": [
                "Anna Breit",
                "Artem Revenko",
                "Kiamehr Rezaee",
                "Mohammad Taher Pilehvar",
                "Jose Camacho-Collados"
            ],
            "abstract": "We present WiC-TSV, a new multi-domain evaluation benchmark for Word Sense\nDisambiguation. More specifically, we introduce a framework for Target Sense\nVerification of Words in Context which grounds its uniqueness in the\nformulation as a binary classification task thus being independent of external\nsense inventories, and the coverage of various domains. This makes the dataset\nhighly flexible for the evaluation of a diverse set of models and systems in\nand across domains. WiC-TSV provides three different evaluation settings,\ndepending on the input signals provided to the model. We set baseline\nperformance on the dataset using state-of-the-art language models. Experimental\nresults show that even though these models can perform decently on the task,\nthere remains a gap between machine and human performance, especially in\nout-of-domain settings. WiC-TSV data is available at\nhttps://competitions.codalab.org/competitions/23683",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15020v3",
            "date": "2020-04-30 17:59:17+00:00",
            "title": "Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO",
            "authors": [
                "Zarana Parekh",
                "Jason Baldridge",
                "Daniel Cer",
                "Austin Waters",
                "Yinfei Yang"
            ],
            "abstract": "By supporting multi-modal retrieval training and evaluation, image captioning\ndatasets have spurred remarkable progress on representation learning.\nUnfortunately, datasets have limited cross-modal associations: images are not\npaired with other images, captions are only paired with other captions of the\nsame image, there are no negative associations and there are missing positive\ncross-modal associations. This undermines research into how inter-modality\nlearning impacts intra-modality tasks. We address this gap with Crisscrossed\nCaptions (CxC), an extension of the MS-COCO dataset with human semantic\nsimilarity judgments for 267,095 intra- and inter-modality pairs. We report\nbaseline results on CxC for strong existing unimodal and multimodal models. We\nalso evaluate a multitask dual encoder trained on both image-caption and\ncaption-caption pairs that crucially demonstrates CxC's value for measuring the\ninfluence of intra- and inter-modality learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00019v1",
            "date": "2020-04-30 18:00:06+00:00",
            "title": "Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs",
            "authors": [
                "Michael A. Lepori",
                "Tal Linzen",
                "R. Thomas McCoy"
            ],
            "abstract": "Sequence-based neural networks show significant sensitivity to syntactic\nstructure, but they still perform less well on syntactic tasks than tree-based\nnetworks. Such tree-based networks can be provided with a constituency parse, a\ndependency parse, or both. We evaluate which of these two representational\nschemes more effectively introduces biases for syntactic structure that\nincrease performance on the subject-verb agreement prediction task. We find\nthat a constituency-based network generalizes more robustly than a\ndependency-based one, and that combining the two types of structure does not\nyield further improvement. Finally, we show that the syntactic robustness of\nsequential models can be substantially improved by fine-tuning on a small\namount of constructed data, suggesting that data augmentation is a viable\nalternative to explicit constituency structure for imparting the syntactic\nbiases that sequential models are lacking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00036v2",
            "date": "2020-04-30 18:08:22+00:00",
            "title": "Improving Factual Consistency Between a Response and Persona Facts",
            "authors": [
                "Mohsen Mesgar",
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "Neural models for response generation produce responses that are semantically\nplausible but not necessarily factually consistent with facts describing the\nspeaker's persona. These models are trained with fully supervised learning\nwhere the objective function barely captures factual consistency. We propose to\nfine-tune these models by reinforcement learning and an efficient reward\nfunction that explicitly captures the consistency between a response and\npersona facts as well as semantic plausibility. Our automatic and human\nevaluations on the PersonaChat corpus confirm that our approach increases the\nrate of responses that are factually consistent with persona facts over its\nsupervised counterpart while retaining the language quality of responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00038v2",
            "date": "2020-04-30 18:09:50+00:00",
            "title": "Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering",
            "authors": [
                "Wenhan Xiong",
                "Hong Wang",
                "William Yang Wang"
            ],
            "abstract": "To extract answers from a large corpus, open-domain question answering (QA)\nsystems usually rely on information retrieval (IR) techniques to narrow the\nsearch space. Standard inverted index methods such as TF-IDF are commonly used\nas thanks to their efficiency. However, their retrieval performance is limited\nas they simply use shallow and sparse lexical features. To break the IR\nbottleneck, recent studies show that stronger retrieval performance can be\nachieved by pretraining a effective paragraph encoder that index paragraphs\ninto dense vectors. Once trained, the corpus can be pre-encoded into\nlow-dimensional vectors and stored within an index structure where the\nretrieval can be efficiently implemented as maximum inner product search.\n  Despite the promising results, pretraining such a dense index is expensive\nand often requires a very large batch size. In this work, we propose a simple\nand resource-efficient method to pretrain the paragraph encoder. First, instead\nof using heuristically created pseudo question-paragraph pairs for pretraining,\nwe utilize an existing pretrained sequence-to-sequence model to build a strong\nquestion generator that creates high-quality pretraining data. Second, we\npropose a progressive pretraining algorithm to ensure the existence of\neffective negative samples in each batch. Across three datasets, our method\noutperforms an existing dense retrieval method that uses 7 times more\ncomputational resources for pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00050v3",
            "date": "2020-04-30 18:43:57+00:00",
            "title": "UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection",
            "authors": [
                "Andrey Kutuzov",
                "Mario Giulianelli"
            ],
            "abstract": "We apply contextualised word embeddings to lexical semantic change detection\nin the SemEval-2020 Shared Task 1. This paper focuses on Subtask 2, ranking\nwords by the degree of their semantic drift over time. We analyse the\nperformance of two contextualising architectures (BERT and ELMo) and three\nchange detection algorithms. We find that the most effective algorithms rely on\nthe cosine similarity between averaged token embeddings and the pairwise\ndistances between token embeddings. They outperform strong baselines by a large\nmargin (in the post-evaluation phase, we have the best Subtask 2 submission for\nSemEval-2020 Task 1), but interestingly, the choice of a particular algorithm\ndepends on the distribution of gold scores in the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00052v3",
            "date": "2020-04-30 18:54:43+00:00",
            "title": "MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer",
            "authors": [
                "Jonas Pfeiffer",
                "Ivan Vuli\u0107",
                "Iryna Gurevych",
                "Sebastian Ruder"
            ],
            "abstract": "The main goal behind state-of-the-art pre-trained multilingual models such as\nmultilingual BERT and XLM-R is enabling and bootstrapping NLP applications in\nlow-resource languages through zero-shot or few-shot cross-lingual transfer.\nHowever, due to limited model capacity, their transfer performance is the\nweakest exactly on such low-resource languages and languages unseen during\npre-training. We propose MAD-X, an adapter-based framework that enables high\nportability and parameter-efficient transfer to arbitrary tasks and languages\nby learning modular language and task representations. In addition, we\nintroduce a novel invertible adapter architecture and a strong baseline method\nfor adapting a pre-trained multilingual model to a new language. MAD-X\noutperforms the state of the art in cross-lingual transfer across a\nrepresentative set of typologically diverse languages on named entity\nrecognition and causal commonsense reasoning, and achieves competitive results\non question answering. Our code and adapters are available at AdapterHub.ml",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00084v1",
            "date": "2020-04-30 20:17:22+00:00",
            "title": "Aspect-Controlled Neural Argument Generation",
            "authors": [
                "Benjamin Schiller",
                "Johannes Daxenberger",
                "Iryna Gurevych"
            ],
            "abstract": "We rely on arguments in our daily lives to deliver our opinions and base them\non evidence, making them more convincing in turn. However, finding and\nformulating arguments can be challenging. In this work, we train a language\nmodel for argument generation that can be controlled on a fine-grained level to\ngenerate sentence-level arguments for a given topic, stance, and aspect. We\ndefine argument aspect detection as a necessary method to allow this\nfine-granular control and crowdsource a dataset with 5,032 arguments annotated\nwith aspects. Our evaluation shows that our generation model is able to\ngenerate high-quality, aspect-specific arguments. Moreover, these arguments can\nbe used to improve the performance of stance detection models via data\naugmentation and to generate counter-arguments. We publish all datasets and\ncode to fine-tune the language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00085v1",
            "date": "2020-04-30 20:21:02+00:00",
            "title": "AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages",
            "authors": [
                "Anoop Kunchukuttan",
                "Divyanshu Kakwani",
                "Satish Golla",
                "Gokul N. C.",
                "Avik Bhattacharyya",
                "Mitesh M. Khapra",
                "Pratyush Kumar"
            ],
            "abstract": "We present the IndicNLP corpus, a large-scale, general-domain corpus\ncontaining 2.7 billion words for 10 Indian languages from two language\nfamilies. We share pre-trained word embeddings trained on these corpora. We\ncreate news article category classification datasets for 9 languages to\nevaluate the embeddings. We show that the IndicNLP embeddings significantly\noutperform publicly available pre-trained embedding on multiple evaluation\ntasks. We hope that the availability of the corpus will accelerate Indic NLP\nresearch. The resources are available at\nhttps://github.com/ai4bharat-indicnlp/indicnlp_corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00087v1",
            "date": "2020-04-30 20:22:45+00:00",
            "title": "Revisiting Unsupervised Relation Extraction",
            "authors": [
                "Thy Thy Tran",
                "Phong Le",
                "Sophia Ananiadou"
            ],
            "abstract": "Unsupervised relation extraction (URE) extracts relations between named\nentities from raw text without manually-labelled data and existing knowledge\nbases (KBs). URE methods can be categorised into generative and discriminative\napproaches, which rely either on hand-crafted features or surface form.\nHowever, we demonstrate that by using only named entities to induce relation\ntypes, we can outperform existing methods on two popular datasets. We conduct a\ncomparison and evaluation of our findings with other URE techniques, to\nascertain the important features in URE. We conclude that entity types provide\na strong inductive bias for URE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00128v2",
            "date": "2020-04-30 22:31:20+00:00",
            "title": "Incremental Neural Coreference Resolution in Constant Memory",
            "authors": [
                "Patrick Xia",
                "Jo\u00e3o Sedoc",
                "Benjamin Van Durme"
            ],
            "abstract": "We investigate modeling coreference resolution under a fixed memory\nconstraint by extending an incremental clustering algorithm to utilize\ncontextualized encoders and neural components. Given a new sentence, our\nend-to-end algorithm proposes and scores each mention span against explicit\nentity representations created from the earlier document context (if any).\nThese spans are then used to update the entity's representations before being\nforgotten; we only retain a fixed set of salient entities throughout the\ndocument. In this work, we successfully convert a high-performing model (Joshi\net al., 2020), asymptotically reducing its memory usage to constant space with\nonly a 0.3% relative loss in F1 on OntoNotes 5.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00163v1",
            "date": "2020-05-01 01:12:49+00:00",
            "title": "Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization",
            "authors": [
                "Sajad Sotudeh",
                "Nazli Goharian",
                "Ross W. Filice"
            ],
            "abstract": "Sequence-to-sequence (seq2seq) network is a well-established model for text\nsummarization task. It can learn to produce readable content; however, it falls\nshort in effectively identifying key regions of the source. In this paper, we\napproach the content selection problem for clinical abstractive summarization\nby augmenting salient ontological terms into the summarizer. Our experiments on\ntwo publicly available clinical data sets (107,372 reports of MIMIC-CXR, and\n3,366 reports of OpenI) show that our model statistically significantly boosts\nstate-of-the-art results in terms of Rouge metrics (with improvements: 2.9%\nRG-1, 2.5% RG-2, 1.9% RG-L), in the healthcare domain where any range of\nimprovement impacts patients' welfare.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00165v3",
            "date": "2020-05-01 01:21:47+00:00",
            "title": "Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment",
            "authors": [
                "Forrest Davis",
                "Marten van Schijndel"
            ],
            "abstract": "A standard approach to evaluating language models analyzes how models assign\nprobabilities to valid versus invalid syntactic constructions (i.e. is a\ngrammatical sentence more probable than an ungrammatical sentence). Our work\nuses ambiguous relative clause attachment to extend such evaluations to cases\nof multiple simultaneous valid interpretations, where stark grammaticality\ndifferences are absent. We compare model performance in English and Spanish to\nshow that non-linguistic biases in RNN LMs advantageously overlap with\nsyntactic structure in English but not Spanish. Thus, English models may appear\nto acquire human-like syntactic preferences, while models trained on Spanish\nfail to acquire comparable human-like preferences. We conclude by relating\nthese results to broader concerns about the relationship between comprehension\n(i.e. typical language model use cases) and production (which generates the\ntraining data for language models), suggesting that necessary linguistic biases\nare not present in the training signal at all.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00172v2",
            "date": "2020-05-01 01:55:09+00:00",
            "title": "Information Seeking in the Spirit of Learning: a Dataset for Conversational Curiosity",
            "authors": [
                "Pedro Rodriguez",
                "Paul Crook",
                "Seungwhan Moon",
                "Zhiguang Wang"
            ],
            "abstract": "Open-ended human learning and information-seeking are increasingly mediated\nby digital assistants. However, such systems often ignore the user's\npre-existing knowledge. Assuming a correlation between engagement and user\nresponses such as \"liking\" messages or asking followup questions, we design a\nWizard-of-Oz dialog task that tests the hypothesis that engagement increases\nwhen users are presented with facts related to what they know. Through\ncrowd-sourcing of this experiment, we collect and release 14K dialogs (181K\nutterances) where users and assistants converse about geographic topics like\ngeopolitical entities and locations. This dataset is annotated with\npre-existing user knowledge, message-level dialog acts, grounding to Wikipedia,\nand user reactions to messages. Responses using a user's prior knowledge\nincrease engagement. We incorporate this knowledge into a multi-task model that\nreproduces human assistant policies and improves over a BERT content model by\n13 mean reciprocal rank points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00175v2",
            "date": "2020-05-01 02:01:18+00:00",
            "title": "Selecting Informative Contexts Improves Language Model Finetuning",
            "authors": [
                "Richard Antonello",
                "Nicole Beckage",
                "Javier Turek",
                "Alexander Huth"
            ],
            "abstract": "Language model fine-tuning is essential for modern natural language\nprocessing, but is computationally expensive and time-consuming. Further, the\neffectiveness of fine-tuning is limited by the inclusion of training examples\nthat negatively affect performance. Here we present a general fine-tuning\nmethod that we call information gain filtration for improving the overall\ntraining efficiency and final performance of language model fine-tuning. We\ndefine the information gain of an example as the improvement on a test metric\nafter training on that example. A secondary learner is then trained to\napproximate this quantity. During fine-tuning, this learner selects informative\nexamples and skips uninformative ones. We show that our method has consistent\nimprovement across datasets, fine-tuning tasks, and language model\narchitectures. For example, we achieve a median perplexity of 54.0 on a books\ndataset compared to 57.3 for standard fine-tuning. We present statistical\nevidence that offers insight into the improvements of our method over standard\nfine-tuning. The generality of our method leads us to propose a new paradigm\nfor language model fine-tuning -- we encourage researchers to release\npretrained secondary learners on common corpora to promote efficient and\neffective fine-tuning, thereby improving the performance and reducing the\noverall energy footprint of language model fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00181v3",
            "date": "2020-05-01 02:21:17+00:00",
            "title": "Sparse, Dense, and Attentional Representations for Text Retrieval",
            "authors": [
                "Yi Luan",
                "Jacob Eisenstein",
                "Kristina Toutanova",
                "Michael Collins"
            ],
            "abstract": "Dual encoders perform retrieval by encoding documents and queries into dense\nlowdimensional vectors, scoring each document by its inner product with the\nquery. We investigate the capacity of this architecture relative to sparse\nbag-of-words models and attentional neural networks. Using both theoretical and\nempirical analysis, we establish connections between the encoding dimension,\nthe margin between gold and lower-ranked documents, and the document length,\nsuggesting limitations in the capacity of fixed-length encodings to support\nprecise retrieval of long documents. Building on these insights, we propose a\nsimple neural model that combines the efficiency of dual encoders with some of\nthe expressiveness of more costly attentional architectures, and explore\nsparse-dense hybrids to capitalize on the precision of sparse retrieval. These\nmodels outperform strong alternatives in large-scale retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00187v2",
            "date": "2020-05-01 02:51:20+00:00",
            "title": "Cross-Linguistic Syntactic Evaluation of Word Prediction Models",
            "authors": [
                "Aaron Mueller",
                "Garrett Nicolai",
                "Panayiota Petrou-Zeniou",
                "Natalia Talmina",
                "Tal Linzen"
            ],
            "abstract": "A range of studies have concluded that neural word prediction models can\ndistinguish grammatical from ungrammatical sentences with high accuracy.\nHowever, these studies are based primarily on monolingual evidence from\nEnglish. To investigate how these models' ability to learn syntax varies by\nlanguage, we introduce CLAMS (Cross-Linguistic Assessment of Models on Syntax),\na syntactic evaluation suite for monolingual and multilingual models. CLAMS\nincludes subject-verb agreement challenge sets for English, French, German,\nHebrew and Russian, generated from grammars we develop. We use CLAMS to\nevaluate LSTM language models as well as monolingual and multilingual BERT.\nAcross languages, monolingual LSTMs achieved high accuracy on dependencies\nwithout attractors, and generally poor accuracy on agreement across object\nrelative clauses. On other constructions, agreement accuracy was generally\nhigher in languages with richer morphology. Multilingual models generally\nunderperformed monolingual models. Multilingual BERT showed high syntactic\naccuracy on English, but noticeable deficiencies in other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00190v1",
            "date": "2020-05-01 03:05:43+00:00",
            "title": "Evaluating Neural Machine Comprehension Model Robustness to Noisy Inputs and Adversarial Attacks",
            "authors": [
                "Winston Wu",
                "Dustin Arendt",
                "Svitlana Volkova"
            ],
            "abstract": "We evaluate machine comprehension models' robustness to noise and adversarial\nattacks by performing novel perturbations at the character, word, and sentence\nlevel. We experiment with different amounts of perturbations to examine model\nconfidence and misclassification rate, and contrast model performance in\nadversarial training with different embedding types on two benchmark datasets.\nWe demonstrate improving model performance with ensembling. Finally, we analyze\nfactors that effect model behavior under adversarial training and develop a\nmodel to predict model errors during adversarial attacks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00192v3",
            "date": "2020-05-01 03:24:36+00:00",
            "title": "KPQA: A Metric for Generative Question Answering Using Keyphrase Weights",
            "authors": [
                "Hwanhee Lee",
                "Seunghyun Yoon",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Trung Bui",
                "Joongbo Shin",
                "Kyomin Jung"
            ],
            "abstract": "In the automatic evaluation of generative question answering (GenQA) systems,\nit is difficult to assess the correctness of generated answers due to the\nfree-form of the answer. Especially, widely used n-gram similarity metrics\noften fail to discriminate the incorrect answers since they equally consider\nall of the tokens. To alleviate this problem, we propose KPQA-metric, a new\nmetric for evaluating the correctness of GenQA. Specifically, our new metric\nassigns different weights to each token via keyphrase prediction, thereby\njudging whether a generated answer sentence captures the key meaning of the\nreference answer. To evaluate our metric, we create high-quality human\njudgments of correctness on two GenQA datasets. Using our human-evaluation\ndatasets, we show that our proposed metric has a significantly higher\ncorrelation with human judgments than existing metrics. The code is available\nat https://github.com/hwanheelee1993/KPQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00242v2",
            "date": "2020-05-01 06:29:56+00:00",
            "title": "TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions",
            "authors": [
                "Qiang Ning",
                "Hao Wu",
                "Rujun Han",
                "Nanyun Peng",
                "Matt Gardner",
                "Dan Roth"
            ],
            "abstract": "A critical part of reading is being able to understand the temporal\nrelationships between events described in a passage of text, even when those\nrelationships are not explicitly stated. However, current machine reading\ncomprehension benchmarks have practically no questions that test temporal\nphenomena, so systems trained on these benchmarks have no capacity to answer\nquestions such as \"what happened before/after [some event]?\" We introduce\nTORQUE, a new English reading comprehension benchmark built on 3.2k news\nsnippets with 21k human-generated questions querying temporal relationships.\nResults show that RoBERTa-large achieves an exact-match score of 51% on the\ntest set of TORQUE, about 30% behind human performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00247v3",
            "date": "2020-05-01 07:03:42+00:00",
            "title": "AdapterFusion: Non-Destructive Task Composition for Transfer Learning",
            "authors": [
                "Jonas Pfeiffer",
                "Aishwarya Kamath",
                "Andreas R\u00fcckl\u00e9",
                "Kyunghyun Cho",
                "Iryna Gurevych"
            ],
            "abstract": "Sequential fine-tuning and multi-task learning are methods aiming to\nincorporate knowledge from multiple tasks; however, they suffer from\ncatastrophic forgetting and difficulties in dataset balancing. To address these\nshortcomings, we propose AdapterFusion, a new two stage learning algorithm that\nleverages knowledge from multiple tasks. First, in the knowledge extraction\nstage we learn task specific parameters called adapters, that encapsulate the\ntask-specific information. We then combine the adapters in a separate knowledge\ncomposition step. We show that by separating the two stages, i.e., knowledge\nextraction and knowledge composition, the classifier can effectively exploit\nthe representations learned from multiple tasks in a non-destructive manner. We\nempirically evaluate AdapterFusion on 16 diverse NLU tasks, and find that it\neffectively combines various types of knowledge at different layers of the\nmodel. We show that our approach outperforms traditional strategies such as\nfull fine-tuning as well as multi-task learning. Our code and adapters are\navailable at AdapterHub.ml.",
            "journal ref": "Proceedings of EACL 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00250v1",
            "date": "2020-05-01 07:11:34+00:00",
            "title": "Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields",
            "authors": [
                "Jonas Pfeiffer",
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "We compare different models for low resource multi-task sequence tagging that\nleverage dependencies between label sequences for different tasks. Our analysis\nis aimed at datasets where each example has labels for multiple tasks. Current\napproaches use either a separate model for each task or standard multi-task\nlearning to learn shared feature representations. However, these approaches\nignore correlations between label sequences, which can provide important\ninformation in settings with small training datasets. To analyze which\nscenarios can profit from modeling dependencies between labels in different\ntasks, we revisit dynamic conditional random fields (CRFs) and combine them\nwith deep neural networks. We compare single-task, multi-task and dynamic CRF\nsetups for three diverse datasets at both sentence and document levels in\nEnglish and German low resource scenarios. We show that including silver labels\nfrom pretrained part-of-speech taggers as auxiliary tasks can improve\nperformance on downstream tasks. We find that especially in low-resource\nscenarios, the explicit modeling of inter-dependencies between task predictions\noutperforms single-task as well as standard multi-task models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00268v2",
            "date": "2020-05-01 08:25:11+00:00",
            "title": "Towards Controllable Biases in Language Generation",
            "authors": [
                "Emily Sheng",
                "Kai-Wei Chang",
                "Premkumar Natarajan",
                "Nanyun Peng"
            ],
            "abstract": "We present a general approach towards controllable societal biases in natural\nlanguage generation (NLG). Building upon the idea of adversarial triggers, we\ndevelop a method to induce societal biases in generated text when input prompts\ncontain mentions of specific demographic groups. We then analyze two scenarios:\n1) inducing negative biases for one demographic and positive biases for another\ndemographic, and 2) equalizing biases between demographics. The former scenario\nenables us to detect the types of biases present in the model. Specifically, we\nshow the effectiveness of our approach at facilitating bias analysis by finding\ntopics that correspond to demographic inequalities in generated text and\ncomparing the relative effectiveness of inducing biases for different\ndemographics. The second scenario is useful for mitigating biases in downstream\napplications such as dialogue generation. In our experiments, the mitigation\ntechnique proves to be effective at equalizing the amount of biases across\ndemographics while simultaneously generating less negatively biased text\noverall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00283v1",
            "date": "2020-05-01 09:31:38+00:00",
            "title": "Facilitating Access to Multilingual COVID-19 Information via Neural Machine Translation",
            "authors": [
                "Andy Way",
                "Rejwanul Haque",
                "Guodong Xie",
                "Federico Gaspari",
                "Maja Popovic",
                "Alberto Poncelas"
            ],
            "abstract": "Every day, more people are becoming infected and dying from exposure to\nCOVID-19. Some countries in Europe like Spain, France, the UK and Italy have\nsuffered particularly badly from the virus. Others such as Germany appear to\nhave coped extremely well. Both health professionals and the general public are\nkeen to receive up-to-date information on the effects of the virus, as well as\ntreatments that have proven to be effective. In cases where language is a\nbarrier to access of pertinent information, machine translation (MT) may help\npeople assimilate information published in different languages. Our MT systems\ntrained on COVID-19 data are freely available for anyone to use to help\ntranslate information published in German, French, Italian, Spanish into\nEnglish, as well as the reverse direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00308v1",
            "date": "2020-05-01 10:50:53+00:00",
            "title": "Selecting Backtranslated Data from Multiple Sources for Improved Neural Machine Translation",
            "authors": [
                "Xabier Soto",
                "Dimitar Shterionov",
                "Alberto Poncelas",
                "Andy Way"
            ],
            "abstract": "Machine translation (MT) has benefited from using synthetic training data\noriginating from translating monolingual corpora, a technique known as\nbacktranslation. Combining backtranslated data from different sources has led\nto better results than when using such data in isolation. In this work we\nanalyse the impact that data translated with rule-based, phrase-based\nstatistical and neural MT systems has on new MT systems. We use a real-world\nlow-resource use-case (Basque-to-Spanish in the clinical domain) as well as a\nhigh-resource language pair (German-to-English) to test different scenarios\nwith backtranslation and employ data selection to optimise the synthetic\ncorpora. We exploit different data selection strategies in order to reduce the\namount of data used, while at the same time maintaining high-quality MT\nsystems. We further tune the data selection method by taking into account the\nquality of the MT systems used for backtranslation and lexical diversity of the\nresulting corpora. Our experiments show that incorporating backtranslated data\nfrom different sources can be beneficial, and that availing of data selection\ncan yield improved performance.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics, ACL (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00315v1",
            "date": "2020-05-01 11:22:55+00:00",
            "title": "Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance",
            "authors": [
                "Prasetya Ajie Utama",
                "Nafise Sadat Moosavi",
                "Iryna Gurevych"
            ],
            "abstract": "Models for natural language understanding (NLU) tasks often rely on the\nidiosyncratic biases of the dataset, which make them brittle against test cases\noutside the training distribution. Recently, several proposed debiasing methods\nare shown to be very effective in improving out-of-distribution performance.\nHowever, their improvements come at the expense of performance drop when models\nare evaluated on the in-distribution data, which contain examples with higher\ndiversity. This seemingly inevitable trade-off may not tell us much about the\nchanges in the reasoning and understanding capabilities of the resulting models\non broader types of examples beyond the small subset represented in the\nout-of-distribution data. In this paper, we address this trade-off by\nintroducing a novel debiasing method, called confidence regularization, which\ndiscourage models from exploiting biases while enabling them to receive enough\nincentive to learn from all the training examples. We evaluate our method on\nthree NLU tasks and show that, in contrast to its predecessors, it improves the\nperformance on out-of-distribution datasets (e.g., 7pp gain on HANS dataset)\nwhile maintaining the original in-distribution accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00333v2",
            "date": "2020-05-01 12:22:33+00:00",
            "title": "XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning",
            "authors": [
                "Edoardo Maria Ponti",
                "Goran Glava\u0161",
                "Olga Majewska",
                "Qianchu Liu",
                "Ivan Vuli\u0107",
                "Anna Korhonen"
            ],
            "abstract": "In order to simulate human language capacity, natural language processing\nsystems must be able to reason about the dynamics of everyday situations,\nincluding their possible causes and effects. Moreover, they should be able to\ngeneralise the acquired world knowledge to new languages, modulo cultural\ndifferences. Advances in machine reasoning and cross-lingual transfer depend on\nthe availability of challenging evaluation benchmarks. Motivated by both\ndemands, we introduce Cross-lingual Choice of Plausible Alternatives (XCOPA), a\ntypologically diverse multilingual dataset for causal commonsense reasoning in\n11 languages, which includes resource-poor languages like Eastern Apur\\'imac\nQuechua and Haitian Creole. We evaluate a range of state-of-the-art models on\nthis novel dataset, revealing that the performance of current methods based on\nmultilingual pretraining and zero-shot fine-tuning falls short compared to\ntranslation-based transfer. Finally, we propose strategies to adapt\nmultilingual models to out-of-sample resource-lean languages where only a small\ncorpus or a bilingual dictionary is available, and report substantial\nimprovements over the random baseline. The XCOPA dataset is freely available at\ngithub.com/cambridgeltl/xcopa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00388v1",
            "date": "2020-05-01 14:10:37+00:00",
            "title": "Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter",
            "authors": [
                "Costanza Conforti",
                "Jakob Berndt",
                "Mohammad Taher Pilehvar",
                "Chryssi Giannitsarou",
                "Flavio Toxvaerd",
                "Nigel Collier"
            ],
            "abstract": "We present a new challenging stance detection dataset, called\nWill-They-Won't-They (WT-WT), which contains 51,284 tweets in English, making\nit by far the largest available dataset of the type. All the annotations are\ncarried out by experts; therefore, the dataset constitutes a high-quality and\nreliable benchmark for future research in stance detection. Our experiments\nwith a wide range of recent state-of-the-art stance detection systems show that\nthe dataset poses a strong challenge to existing models in this domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00396v3",
            "date": "2020-05-01 14:27:14+00:00",
            "title": "Identifying Necessary Elements for BERT's Multilinguality",
            "authors": [
                "Philipp Dufter",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "It has been shown that multilingual BERT (mBERT) yields high quality\nmultilingual representations and enables effective zero-shot transfer. This is\nsurprising given that mBERT does not use any crosslingual signal during\ntraining. While recent literature has studied this phenomenon, the reasons for\nthe multilinguality are still somewhat obscure. We aim to identify\narchitectural properties of BERT and linguistic properties of languages that\nare necessary for BERT to become multilingual. To allow for fast\nexperimentation we propose an efficient setup with small BERT models trained on\na mix of synthetic and natural data. Overall, we identify four architectural\nand two linguistic elements that influence multilinguality. Based on our\ninsights, we experiment with a multilingual pretraining setup that modifies the\nmasking strategy using VecMap, i.e., unsupervised embedding alignment.\nExperiments on XNLI with three languages indicate that our findings transfer\nfrom our small setup to larger scale settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00432v1",
            "date": "2020-05-01 15:07:59+00:00",
            "title": "Topological Sort for Sentence Ordering",
            "authors": [
                "Shrimai Prabhumoye",
                "Ruslan Salakhutdinov",
                "Alan W Black"
            ],
            "abstract": "Sentence ordering is the task of arranging the sentences of a given text in\nthe correct order. Recent work using deep neural networks for this task has\nframed it as a sequence prediction problem. In this paper, we propose a new\nframing of this task as a constraint solving problem and introduce a new\ntechnique to solve it. Additionally, we propose a human evaluation for this\ntask. The results on both automatic and human metrics across four different\ndatasets show that this new technique is better at capturing coherence in\ndocuments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00458v1",
            "date": "2020-05-01 15:53:16+00:00",
            "title": "Style Variation as a Vantage Point for Code-Switching",
            "authors": [
                "Khyathi Raghavi Chandu",
                "Alan W Black"
            ],
            "abstract": "Code-Switching (CS) is a common phenomenon observed in several bilingual and\nmultilingual communities, thereby attaining prevalence in digital and social\nmedia platforms. This increasing prominence demands the need to model CS\nlanguages for critical downstream tasks. A major problem in this domain is the\ndearth of annotated data and a substantial corpora to train large scale neural\nmodels. Generating vast amounts of quality text assists several down stream\ntasks that heavily rely on language modeling such as speech recognition,\ntext-to-speech synthesis etc,. We present a novel vantage point of CS to be\nstyle variations between both the participating languages. Our approach does\nnot need any external annotations such as lexical language ids. It mainly\nrelies on easily obtainable monolingual corpora without any parallel alignment\nand a limited set of naturally CS sentences. We propose a two-stage generative\nadversarial training approach where the first stage generates competitive\nnegative examples for CS and the second stage generates more realistic CS\nsentences. We present our experiments on the following pairs of languages:\nSpanish-English, Mandarin-English, Hindi-English and Arabic-French. We show\nthat the trends in metrics for generated CS move closer to real CS data in each\nof the above language pairs through the dual stage training process. We believe\nthis viewpoint of CS as style variations opens new perspectives for modeling\nvarious tasks in CS text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00460v3",
            "date": "2020-05-01 15:55:50+00:00",
            "title": "Improving Broad-Coverage Medical Entity Linking with Semantic Type Prediction and Large-Scale Datasets",
            "authors": [
                "Shikhar Vashishth",
                "Denis Newman-Griffis",
                "Rishabh Joshi",
                "Ritam Dutt",
                "Carolyn Rose"
            ],
            "abstract": "Medical entity linking is the task of identifying and standardizing medical\nconcepts referred to in an unstructured text. Most of the existing methods\nadopt a three-step approach of (1) detecting mentions, (2) generating a list of\ncandidate concepts, and finally (3) picking the best concept among them. In\nthis paper, we probe into alleviating the problem of overgeneration of\ncandidate concepts in the candidate generation module, the most under-studied\ncomponent of medical entity linking. For this, we present MedType, a fully\nmodular system that prunes out irrelevant candidate concepts based on the\npredicted semantic type of an entity mention. We incorporate MedType into five\noff-the-shelf toolkits for medical entity linking and demonstrate that it\nconsistently improves entity linking performance across several benchmark\ndatasets. To address the dearth of annotated training data for medical entity\nlinking, we present WikiMed and PubMedDS, two large-scale medical entity\nlinking datasets, and demonstrate that pre-training MedType on these datasets\nfurther improves entity linking performance. We make our source code and\ndatasets publicly available for medical entity linking research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00481v1",
            "date": "2020-05-01 16:44:54+00:00",
            "title": "ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations",
            "authors": [
                "Fernando Alva-Manchego",
                "Louis Martin",
                "Antoine Bordes",
                "Carolina Scarton",
                "Beno\u00eet Sagot",
                "Lucia Specia"
            ],
            "abstract": "In order to simplify a sentence, human editors perform multiple rewriting\ntransformations: they split it into several shorter sentences, paraphrase words\n(i.e. replacing complex words or phrases by simpler synonyms), reorder\ncomponents, and/or delete information deemed unnecessary. Despite these varied\nrange of possible text alterations, current models for automatic sentence\nsimplification are evaluated using datasets that are focused on a single\ntransformation, such as lexical paraphrasing or splitting. This makes it\nimpossible to understand the ability of simplification models in more realistic\nsettings. To alleviate this limitation, this paper introduces ASSET, a new\ndataset for assessing sentence simplification in English. ASSET is a\ncrowdsourced multi-reference corpus where each simplification was produced by\nexecuting several rewriting transformations. Through quantitative and\nqualitative experiments, we show that simplifications in ASSET are better at\ncapturing characteristics of simplicity when compared to other standard\nevaluation datasets for the task. Furthermore, we motivate the need for\ndeveloping better methods for automatic evaluation using ASSET, since we show\nthat current popular metrics may not be suitable when multiple simplification\ntransformations are performed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00513v2",
            "date": "2020-05-01 17:31:11+00:00",
            "title": "Discourse-Aware Unsupervised Summarization of Long Scientific Documents",
            "authors": [
                "Yue Dong",
                "Andrei Mircea",
                "Jackie C. K. Cheung"
            ],
            "abstract": "We propose an unsupervised graph-based ranking model for extractive\nsummarization of long scientific documents. Our method assumes a two-level\nhierarchical graph representation of the source document, and exploits\nasymmetrical positional cues to determine sentence importance. Results on the\nPubMed and arXiv datasets show that our approach outperforms strong\nunsupervised baselines by wide margins in automatic metrics and human\nevaluation. In addition, it achieves performance comparable to many\nstate-of-the-art supervised approaches which are trained on hundreds of\nthousands of examples. These results suggest that patterns in the discourse\nstructure are a strong signal for determining importance in scientific\narticles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00547v2",
            "date": "2020-05-01 18:00:02+00:00",
            "title": "GoEmotions: A Dataset of Fine-Grained Emotions",
            "authors": [
                "Dorottya Demszky",
                "Dana Movshovitz-Attias",
                "Jeongwoo Ko",
                "Alan Cowen",
                "Gaurav Nemade",
                "Sujith Ravi"
            ],
            "abstract": "Understanding emotion expressed in language has a wide range of applications,\nfrom building empathetic chatbots to detecting harmful online behavior.\nAdvancement in this area can be improved using large-scale datasets with a\nfine-grained typology, adaptable to multiple downstream tasks. We introduce\nGoEmotions, the largest manually annotated dataset of 58k English Reddit\ncomments, labeled for 27 emotion categories or Neutral. We demonstrate the high\nquality of the annotations via Principal Preserved Component Analysis. We\nconduct transfer learning experiments with existing emotion benchmarks to show\nthat our dataset generalizes well to other domains and different emotion\ntaxonomies. Our BERT-based model achieves an average F1-score of .46 across our\nproposed taxonomy, leaving much room for improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00574v1",
            "date": "2020-05-01 19:07:33+00:00",
            "title": "Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset",
            "authors": [
                "Xiang Yue",
                "Bernal Jimenez Gutierrez",
                "Huan Sun"
            ],
            "abstract": "Machine reading comprehension has made great progress in recent years owing\nto large-scale annotated datasets. In the clinical domain, however, creating\nsuch datasets is quite difficult due to the domain expertise required for\nannotation. Recently, Pampari et al. (EMNLP'18) tackled this issue by using\nexpert-annotated question templates and existing i2b2 annotations to create\nemrQA, the first large-scale dataset for question answering (QA) based on\nclinical notes. In this paper, we provide an in-depth analysis of this dataset\nand the clinical reading comprehension (CliniRC) task. From our qualitative\nanalysis, we find that (i) emrQA answers are often incomplete, and (ii) emrQA\nquestions are often answerable without using domain knowledge. From our\nquantitative experiments, surprising results include that (iii) using a small\nsampled subset (5%-20%), we can obtain roughly equal performance compared to\nthe model trained on the entire dataset, (iv) this performance is close to\nhuman expert's performance, and (v) BERT models do not beat the best performing\nbase model. Following our analysis of the emrQA, we further explore two desired\naspects of CliniRC systems: the ability to utilize clinical domain knowledge\nand to generalize to unseen questions and contexts. We argue that both should\nbe considered when creating future datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00580v1",
            "date": "2020-05-01 19:54:46+00:00",
            "title": "Evaluating Robustness to Input Perturbations for Neural Machine Translation",
            "authors": [
                "Xing Niu",
                "Prashant Mathur",
                "Georgiana Dinu",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Neural Machine Translation (NMT) models are sensitive to small perturbations\nin the input. Robustness to such perturbations is typically measured using\ntranslation quality metrics such as BLEU on the noisy input. This paper\nproposes additional metrics which measure the relative degradation and changes\nin translation when small perturbations are added to the input. We focus on a\nclass of models employing subword regularization to address robustness and\nperform extensive evaluations of these models using the robustness measures\nproposed. Results show that our proposed metrics reveal a clear trend of\nimproved robustness to perturbations when subword regularization methods are\nused.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00613v1",
            "date": "2020-05-01 21:22:08+00:00",
            "title": "A Controllable Model of Grounded Response Generation",
            "authors": [
                "Zeqiu Wu",
                "Michel Galley",
                "Chris Brockett",
                "Yizhe Zhang",
                "Xiang Gao",
                "Chris Quirk",
                "Rik Koncel-Kedziorski",
                "Jianfeng Gao",
                "Hannaneh Hajishirzi",
                "Mari Ostendorf",
                "Bill Dolan"
            ],
            "abstract": "Current end-to-end neural conversation models inherently lack the flexibility\nto impose semantic control in the response generation process. This control is\nessential to ensure that users' semantic intents are satisfied and to impose a\ndegree of specificity on generated outputs. Attempts to boost informativeness\nalone come at the expense of factual accuracy, as attested by GPT-2's\npropensity to \"hallucinate\" facts. While this may be mitigated by access to\nbackground knowledge, there is scant guarantee of relevance and informativeness\nin generated responses. We propose a framework that we call controllable\ngrounded response generation (CGRG), in which lexical control phrases are\neither provided by an user or automatically extracted by a content planner from\ndialogue context and grounding knowledge. Quantitative and qualitative results\nshow that, using this framework, a GPT-2 based model trained on a\nconversation-like Reddit dataset outperforms strong generation baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00614v1",
            "date": "2020-05-01 21:23:20+00:00",
            "title": "Multi-Dimensional Gender Bias Classification",
            "authors": [
                "Emily Dinan",
                "Angela Fan",
                "Ledell Wu",
                "Jason Weston",
                "Douwe Kiela",
                "Adina Williams"
            ],
            "abstract": "Machine learning models are trained to find patterns in data. NLP models can\ninadvertently learn socially undesirable patterns when training on gender\nbiased text. In this work, we propose a general framework that decomposes\ngender bias in text along several pragmatic and semantic dimensions: bias from\nthe gender of the person being spoken about, bias from the gender of the person\nbeing spoken to, and bias from the gender of the speaker. Using this\nfine-grained framework, we automatically annotate eight large scale datasets\nwith gender information. In addition, we collect a novel, crowdsourced\nevaluation benchmark of utterance-level gender rewrites. Distinguishing between\ngender bias along multiple dimensions is important, as it enables us to train\nfiner-grained gender bias classifiers. We show our classifiers prove valuable\nfor a variety of important applications, such as controlling for gender bias in\ngenerative models, detecting gender bias in arbitrary text, and shed light on\noffensive language in terms of genderedness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00626v2",
            "date": "2020-05-01 21:48:48+00:00",
            "title": "Predicting Declension Class from Form and Meaning",
            "authors": [
                "Adina Williams",
                "Tiago Pimentel",
                "Arya D. McCarthy",
                "Hagen Blix",
                "Eleanor Chodroff",
                "Ryan Cotterell"
            ],
            "abstract": "The noun lexica of many natural languages are divided into several declension\nclasses with characteristic morphological properties. Class membership is far\nfrom deterministic, but the phonological form of a noun and/or its meaning can\noften provide imperfect clues. Here, we investigate the strength of those\nclues. More specifically, we operationalize this by measuring how much\ninformation, in bits, we can glean about declension class from knowing the form\nand/or meaning of nouns. We know that form and meaning are often also\nindicative of grammatical gender---which, as we quantitatively verify, can\nitself share information with declension class---so we also control for gender.\nWe find for two Indo-European languages (Czech and German) that form and\nmeaning respectively share significant amounts of information with class (and\ncontribute additional information above and beyond gender). The three-way\ninteraction between class, form, and meaning (given gender) is also\nsignificant. Our study is important for two reasons: First, we introduce a new\nmethod that provides additional quantitative support for a classic linguistic\nfinding that form and meaning are relevant for the classification of nouns into\ndeclensions. Secondly, we show not only that individual declensions classes\nvary in the strength of their clues within a language, but also that these\nvariations themselves vary across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00628v2",
            "date": "2020-05-01 21:49:34+00:00",
            "title": "Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?",
            "authors": [
                "Yada Pruksachatkun",
                "Jason Phang",
                "Haokun Liu",
                "Phu Mon Htut",
                "Xiaoyi Zhang",
                "Richard Yuanzhe Pang",
                "Clara Vania",
                "Katharina Kann",
                "Samuel R. Bowman"
            ],
            "abstract": "While pretrained models such as BERT have shown large gains across natural\nlanguage understanding tasks, their performance can be improved by further\ntraining the model on a data-rich intermediate task, before fine-tuning it on a\ntarget task. However, it is still poorly understood when and why\nintermediate-task training is beneficial for a given target task. To\ninvestigate this, we perform a large-scale study on the pretrained RoBERTa\nmodel with 110 intermediate-target task combinations. We further evaluate all\ntrained models with 25 probing tasks meant to reveal the specific skills that\ndrive transfer. We observe that intermediate tasks requiring high-level\ninference and reasoning abilities tend to work best. We also observe that\ntarget task performance is strongly correlated with higher-level abilities such\nas coreference resolution. However, we fail to observe more granular\ncorrelations between probing and target task performance, highlighting the need\nfor further work on broad-coverage probing benchmarks. We also observe evidence\nthat the forgetting of knowledge learned during pretraining may limit our\nanalysis, highlighting the need for further work on transfer learning methods\nin these settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00630v1",
            "date": "2020-05-01 21:55:40+00:00",
            "title": "KLEJ: Comprehensive Benchmark for Polish Language Understanding",
            "authors": [
                "Piotr Rybak",
                "Robert Mroczkowski",
                "Janusz Tracz",
                "Ireneusz Gawlik"
            ],
            "abstract": "In recent years, a series of Transformer-based models unlocked major\nimprovements in general natural language understanding (NLU) tasks. Such a fast\npace of research would not be possible without general NLU benchmarks, which\nallow for a fair comparison of the proposed methods. However, such benchmarks\nare available only for a handful of languages. To alleviate this issue, we\nintroduce a comprehensive multi-task benchmark for the Polish language\nunderstanding, accompanied by an online leaderboard. It consists of a diverse\nset of tasks, adopted from existing datasets for named entity recognition,\nquestion-answering, textual entailment, and others. We also introduce a new\nsentiment analysis task for the e-commerce domain, named Allegro Reviews (AR).\nTo ensure a common evaluation scheme and promote models that generalize to\ndifferent NLU tasks, the benchmark includes datasets from varying domains and\napplications. Additionally, we release HerBERT, a Transformer-based model\ntrained specifically for the Polish language, which has the best average\nperformance and obtains the best results for three out of nine tasks. Finally,\nwe provide an extensive evaluation, including several standard baselines and\nrecently proposed, multilingual Transformer-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00633v1",
            "date": "2020-05-01 22:04:58+00:00",
            "title": "From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers",
            "authors": [
                "Anne Lauscher",
                "Vinit Ravishankar",
                "Ivan Vuli\u0107",
                "Goran Glava\u0161"
            ],
            "abstract": "Massively multilingual transformers pretrained with language modeling\nobjectives (e.g., mBERT, XLM-R) have become a de facto default transfer\nparadigm for zero-shot cross-lingual transfer in NLP, offering unmatched\ntransfer performance. Current downstream evaluations, however, verify their\nefficacy predominantly in transfer settings involving languages with sufficient\namounts of pretraining data, and with lexically and typologically close\nlanguages. In this work, we analyze their limitations and show that\ncross-lingual transfer via massively multilingual transformers, much like\ntransfer via cross-lingual word embeddings, is substantially less effective in\nresource-lean scenarios and for distant languages. Our experiments,\nencompassing three lower-level tasks (POS tagging, dependency parsing, NER), as\nwell as two high-level semantic tasks (NLI, QA), empirically correlate transfer\nperformance with linguistic similarity between the source and target languages,\nbut also with the size of pretraining corpora of target languages. We also\ndemonstrate a surprising effectiveness of inexpensive few-shot transfer (i.e.,\nfine-tuning on a few target-language instances after fine-tuning in the source)\nacross the board. This suggests that additional research efforts should be\ninvested to reach beyond the limiting zero-shot conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00635v1",
            "date": "2020-05-01 22:10:35+00:00",
            "title": "Using Noisy Self-Reports to Predict Twitter User Demographics",
            "authors": [
                "Zach Wood-Doughty",
                "Paiheng Xu",
                "Xiao Liu",
                "Mark Dredze"
            ],
            "abstract": "Computational social science studies often contextualize content analysis\nwithin standard demographics. Since demographics are unavailable on many social\nmedia platforms (e.g. Twitter) numerous studies have inferred demographics\nautomatically. Despite many studies presenting proof of concept inference of\nrace and ethnicity, training of practical systems remains elusive since there\nare few annotated datasets. Existing datasets are small, inaccurate, or fail to\ncover the four most common racial and ethnic groups in the United States. We\npresent a method to identify self-reports of race and ethnicity from Twitter\nprofile descriptions. Despite errors inherent in automated supervision, we\nproduce models with good performance when measured on gold standard self-report\nsurvey data. The result is a reproducible method for creating large-scale\ntraining resources for race and ethnicity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00637v2",
            "date": "2020-05-01 22:17:37+00:00",
            "title": "Explainable Link Prediction for Emerging Entities in Knowledge Graphs",
            "authors": [
                "Rajarshi Bhowmik",
                "Gerard de Melo"
            ],
            "abstract": "Despite their large-scale coverage, cross-domain knowledge graphs invariably\nsuffer from inherent incompleteness and sparsity. Link prediction can alleviate\nthis by inferring a target entity, given a source entity and a query relation.\nRecent embedding-based approaches operate in an uninterpretable latent semantic\nvector space of entities and relations, while path-based approaches operate in\nthe symbolic space, making the inference process explainable. However, these\napproaches typically consider static snapshots of the knowledge graphs,\nseverely restricting their applicability for evolving knowledge graphs with\nnewly emerging entities. To overcome this issue, we propose an inductive\nrepresentation learning framework that is able to learn representations of\npreviously unseen entities. Our method finds reasoning paths between source and\ntarget entities, thereby making the link prediction for unseen entities\ninterpretable and providing support evidence for the inferred link.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00649v1",
            "date": "2020-05-01 23:20:49+00:00",
            "title": "Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates",
            "authors": [
                "Katherine A. Keith",
                "David Jensen",
                "Brendan O'Connor"
            ],
            "abstract": "Many applications of computational social science aim to infer causal\nconclusions from non-experimental data. Such observational data often contains\nconfounders, variables that influence both potential causes and potential\neffects. Unmeasured or latent confounders can bias causal estimates, and this\nhas motivated interest in measuring potential confounders from observed text.\nFor example, an individual's entire history of social media posts or the\ncontent of a news article could provide a rich measurement of multiple\nconfounders. Yet, methods and applications for this problem are scattered\nacross different communities and evaluation practices are inconsistent. This\nreview is the first to gather and categorize these examples and provide a guide\nto data-processing and evaluation decisions. Despite increased attention on\nadjusting for confounding using text, there are still many open problems, which\nwe highlight in this paper.",
            "journal ref": "ACL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00661v1",
            "date": "2020-05-02 00:09:16+00:00",
            "title": "On Faithfulness and Factuality in Abstractive Summarization",
            "authors": [
                "Joshua Maynez",
                "Shashi Narayan",
                "Bernd Bohnet",
                "Ryan McDonald"
            ],
            "abstract": "It is well known that the standard likelihood training and approximate\ndecoding objectives in neural text generation models lead to less human-like\nresponses for open-ended tasks such as language modeling and story generation.\nIn this paper we have analyzed limitations of these models for abstractive\ndocument summarization and found that these models are highly prone to\nhallucinate content that is unfaithful to the input document. We conducted a\nlarge scale human evaluation of several neural abstractive summarization\nsystems to better understand the types of hallucinations they produce. Our\nhuman annotators found substantial amounts of hallucinated content in all model\ngenerated summaries. However, our analysis does show that pretrained models are\nbetter summarizers not only in terms of raw metrics, i.e., ROUGE, but also in\ngenerating faithful and factual summaries as evaluated by humans. Furthermore,\nwe show that textual entailment measures better correlate with faithfulness\nthan standard metrics, potentially leading the way to automatic evaluation\nmetrics as well as training and decoding criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00663v1",
            "date": "2020-05-02 00:16:09+00:00",
            "title": "Benchmarking Multimodal Regex Synthesis with Complex Structures",
            "authors": [
                "Xi Ye",
                "Qiaochu Chen",
                "Isil Dillig",
                "Greg Durrett"
            ],
            "abstract": "Existing datasets for regular expression (regex) generation from natural\nlanguage are limited in complexity; compared to regex tasks that users post on\nStackOverflow, the regexes in these datasets are simple, and the language used\nto describe them is not diverse. We introduce StructuredRegex, a new regex\nsynthesis dataset differing from prior ones in three aspects. First, to obtain\nstructurally complex and realistic regexes, we generate the regexes using a\nprobabilistic grammar with pre-defined macros observed from real-world\nStackOverflow posts. Second, to obtain linguistically diverse natural language\ndescriptions, we show crowdworkers abstract depictions of the underlying regex\nand ask them to describe the pattern they see, rather than having them\nparaphrase synthetic language. Third, we augment each regex example with a\ncollection of strings that are and are not matched by the ground truth regex,\nsimilar to how real users give examples. Our quantitative and qualitative\nanalysis demonstrates the advantages of StructuredRegex over prior datasets.\nFurther experimental results using various multimodal synthesis techniques\nhighlight the challenge presented by our dataset, including non-local\nconstraints and multi-modal inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00672v2",
            "date": "2020-05-02 01:26:46+00:00",
            "title": "DagoBERT: Generating Derivational Morphology with a Pretrained Language Model",
            "authors": [
                "Valentin Hofmann",
                "Janet B. Pierrehumbert",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Can pretrained language models (PLMs) generate derivationally complex words?\nWe present the first study investigating this question, taking BERT as the\nexample PLM. We examine BERT's derivational capabilities in different settings,\nranging from using the unmodified pretrained model to full finetuning. Our best\nmodel, DagoBERT (Derivationally and generatively optimized BERT), clearly\noutperforms the previous state of the art in derivation generation (DG).\nFurthermore, our experiments show that the input segmentation crucially impacts\nBERT's derivational knowledge, suggesting that the performance of PLMs could be\nfurther improved if a morphologically informed vocabulary of units were used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00675v1",
            "date": "2020-05-02 01:41:02+00:00",
            "title": "Opportunistic Decoding with Timely Correction for Simultaneous Translation",
            "authors": [
                "Renjie Zheng",
                "Mingbo Ma",
                "Baigong Zheng",
                "Kaibo Liu",
                "Liang Huang"
            ],
            "abstract": "Simultaneous translation has many important application scenarios and\nattracts much attention from both academia and industry recently. Most existing\nframeworks, however, have difficulties in balancing between the translation\nquality and latency, i.e., the decoding policy is usually either too aggressive\nor too conservative. We propose an opportunistic decoding technique with timely\ncorrection ability, which always (over-)generates a certain mount of extra\nwords at each step to keep the audience on track with the latest information.\nAt the same time, it also corrects, in a timely fashion, the mistakes in the\nformer overgenerated words when observing more source context to ensure high\ntranslation quality. Experiments show our technique achieves substantial\nreduction in latency and up to +3.1 increase in BLEU, with revision rate under\n8% in Chinese-to-English and English-to-Chinese translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00692v2",
            "date": "2020-05-02 04:00:26+00:00",
            "title": "Design Challenges in Low-resource Cross-lingual Entity Linking",
            "authors": [
                "Xingyu Fu",
                "Weijia Shi",
                "Xiaodong Yu",
                "Zian Zhao",
                "Dan Roth"
            ],
            "abstract": "Cross-lingual Entity Linking (XEL), the problem of grounding mentions of\nentities in a foreign language text into an English knowledge base such as\nWikipedia, has seen a lot of research in recent years, with a range of\npromising techniques. However, current techniques do not rise to the challenges\nintroduced by text in low-resource languages (LRL) and, surprisingly, fail to\ngeneralize to text not taken from Wikipedia, on which they are usually trained.\n  This paper provides a thorough analysis of low-resource XEL techniques,\nfocusing on the key step of identifying candidate English Wikipedia titles that\ncorrespond to a given foreign language mention. Our analysis indicates that\ncurrent methods are limited by their reliance on Wikipedia's interlanguage\nlinks and thus suffer when the foreign language's Wikipedia is small. We\nconclude that the LRL setting requires the use of outside-Wikipedia\ncross-lingual resources and present a simple yet effective zero-shot XEL\nsystem, QuEL, that utilizes search engines query logs. With experiments on 25\nlanguages, QuEL~shows an average increase of 25\\% in gold candidate recall and\nof 13\\% in end-to-end linking accuracy over state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00696v2",
            "date": "2020-05-02 04:11:33+00:00",
            "title": "Robust and Interpretable Grounding of Spatial References with Relation Networks",
            "authors": [
                "Tsung-Yen Yang",
                "Andrew S. Lan",
                "Karthik Narasimhan"
            ],
            "abstract": "Learning representations of spatial references in natural language is a key\nchallenge in tasks like autonomous navigation and robotic manipulation. Recent\nwork has investigated various neural architectures for learning multi-modal\nrepresentations for spatial concepts. However, the lack of explicit reasoning\nover entities makes such approaches vulnerable to noise in input text or state\nobservations. In this paper, we develop effective models for understanding\nspatial references in text that are robust and interpretable, without\nsacrificing performance. We design a text-conditioned \\textit{relation network}\nwhose parameters are dynamically computed with a cross-modal attention module\nto capture fine-grained spatial relations between entities. This design choice\nprovides interpretability of learned intermediate outputs. Experiments across\nthree tasks demonstrate that our model achieves superior performance, with a\n17\\% improvement in predicting goal locations and a 15\\% improvement in\nrobustness compared to state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00701v1",
            "date": "2020-05-02 04:50:20+00:00",
            "title": "Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen",
            "authors": [
                "Yixin Cao",
                "Ruihao Shui",
                "Liangming Pan",
                "Min-Yen Kan",
                "Zhiyuan Liu",
                "Tat-Seng Chua"
            ],
            "abstract": "The curse of knowledge can impede communication between experts and laymen.\nWe propose a new task of expertise style transfer and contribute a manually\nannotated dataset with the goal of alleviating such cognitive biases. Solving\nthis task not only simplifies the professional language, but also improves the\naccuracy and expertise level of laymen descriptions using simple words. This is\na challenging task, unaddressed in previous work, as it requires the models to\nhave expert intelligence in order to modify text with a deep understanding of\ndomain knowledge and structures. We establish the benchmark performance of five\nstate-of-the-art models for style transfer and text simplification. The results\ndemonstrate a significant gap between machine and human performance. We also\ndiscuss the challenges of automatic evaluation, to provide insights into future\nresearch directions. The dataset is publicly available at\nhttps://srhthu.github.io/expertise-style-transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00719v3",
            "date": "2020-05-02 06:19:20+00:00",
            "title": "Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?",
            "authors": [
                "Abhilasha Ravichander",
                "Yonatan Belinkov",
                "Eduard Hovy"
            ],
            "abstract": "Although neural models have achieved impressive results on several NLP\nbenchmarks, little is understood about the mechanisms they use to perform\nlanguage tasks. Thus, much recent attention has been devoted to analyzing the\nsentence representations learned by neural encoders, through the lens of\n`probing' tasks. However, to what extent was the information encoded in\nsentence representations, as discovered through a probe, actually used by the\nmodel to perform its task? In this work, we examine this probing paradigm\nthrough a case study in Natural Language Inference, showing that models can\nlearn to encode linguistic properties even if they are not needed for the task\non which the model was trained. We further identify that pretrained word\nembeddings play a considerable role in encoding these properties rather than\nthe training task itself, highlighting the importance of careful controls when\ndesigning probing experiments. Finally, through a set of controlled synthetic\ntasks, we demonstrate models can encode these properties considerably above\nchance-level even when distributed in the data as random noise, calling into\nquestion the interpretation of absolute claims on probing tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00742v1",
            "date": "2020-05-02 08:16:13+00:00",
            "title": "Hard-Coded Gaussian Attention for Neural Machine Translation",
            "authors": [
                "Weiqiu You",
                "Simeng Sun",
                "Mohit Iyyer"
            ],
            "abstract": "Recent work has questioned the importance of the Transformer's multi-headed\nattention for achieving high translation quality. We push further in this\ndirection by developing a \"hard-coded\" attention variant without any learned\nparameters. Surprisingly, replacing all learned self-attention heads in the\nencoder and decoder with fixed, input-agnostic Gaussian distributions minimally\nimpacts BLEU scores across four different language pairs. However, additionally\nhard-coding cross attention (which connects the decoder to the encoder)\nsignificantly lowers BLEU, suggesting that it is more important than\nself-attention. Much of this BLEU drop can be recovered by adding just a single\nlearned cross attention head to an otherwise hard-coded Transformer. Taken as a\nwhole, our results offer insight into which components of the Transformer are\nactually important, which we hope will guide future work into the development\nof simpler and more efficient attention-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00766v2",
            "date": "2020-05-02 09:34:42+00:00",
            "title": "BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA",
            "authors": [
                "Nora Kassner",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Khandelwal et al. (2020) use a k-nearest-neighbor (kNN) component to improve\nlanguage model performance. We show that this idea is beneficial for\nopen-domain question answering (QA). To improve the recall of facts encountered\nduring training, we combine BERT (Devlin et al., 2019) with a traditional\ninformation retrieval step (IR) and a kNN search over a large datastore of an\nembedded text collection. Our contributions are as follows: i) BERT-kNN\noutperforms BERT on cloze-style QA by large margins without any further\ntraining. ii) We show that BERT often identifies the correct response category\n(e.g., US city), but only kNN recovers the factually correct answer (e.g.,\n\"Miami\"). iii) Compared to BERT, BERT-kNN excels for rare facts. iv) BERT-kNN\ncan easily handle facts not covered by BERT's training set, e.g., recent\nevents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00770v2",
            "date": "2020-05-02 09:39:36+00:00",
            "title": "Exploring and Predicting Transferability across NLP Tasks",
            "authors": [
                "Tu Vu",
                "Tong Wang",
                "Tsendsuren Munkhdalai",
                "Alessandro Sordoni",
                "Adam Trischler",
                "Andrew Mattarella-Micke",
                "Subhransu Maji",
                "Mohit Iyyer"
            ],
            "abstract": "Recent advances in NLP demonstrate the effectiveness of training large-scale\nlanguage models and transferring them to downstream tasks. Can fine-tuning\nthese models on tasks other than language modeling further improve performance?\nIn this paper, we conduct an extensive study of the transferability between 33\nNLP tasks across three broad classes of problems (text classification, question\nanswering, and sequence labeling). Our results show that transfer learning is\nmore beneficial than previously thought, especially when target task data is\nscarce, and can improve performance even when the source task is small or\ndiffers substantially from the target task (e.g., part-of-speech tagging\ntransfers well to the DROP QA dataset). We also develop task embeddings that\ncan be used to predict the most transferable source tasks for a given target\ntask, and we validate their effectiveness in experiments controlled for source\nand target data size. Overall, our experiments reveal that factors such as\nsource data size, task and domain similarity, and task complexity all play a\nrole in determining transferability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00771v3",
            "date": "2020-05-02 09:40:05+00:00",
            "title": "ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning",
            "authors": [
                "Michael Boratko",
                "Xiang Lorraine Li",
                "Rajarshi Das",
                "Tim O'Gorman",
                "Dan Le",
                "Andrew McCallum"
            ],
            "abstract": "Given questions regarding some prototypical situation such as Name something\nthat people usually do before they leave the house for work? a human can easily\nanswer them via acquired experiences. There can be multiple right answers for\nsuch questions, with some more common for a situation than others. This paper\nintroduces a new question answering dataset for training and evaluating common\nsense reasoning capabilities of artificial intelligence systems in such\nprototypical situations. The training set is gathered from an existing set of\nquestions played in a long-running international game show FAMILY- FEUD. The\nhidden evaluation set is created by gathering answers for each question from\n100 crowd-workers. We also propose a generative evaluation task where a model\nhas to output a ranked list of answers, ideally covering all prototypical\nanswers for a question. After presenting multiple competitive baseline models,\nwe find that human performance still exceeds model scores on all evaluation\nmetrics with a meaningful gap, supporting the challenging nature of the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00785v5",
            "date": "2020-05-02 10:45:30+00:00",
            "title": "Visually Grounded Continual Learning of Compositional Phrases",
            "authors": [
                "Xisen Jin",
                "Junyi Du",
                "Arka Sadhu",
                "Ram Nevatia",
                "Xiang Ren"
            ],
            "abstract": "Humans acquire language continually with much more limited access to data\nsamples at a time, as compared to contemporary NLP systems. To study this\nhuman-like language acquisition ability, we present VisCOLL, a visually\ngrounded language learning task, which simulates the continual acquisition of\ncompositional phrases from streaming visual scenes. In the task, models are\ntrained on a paired image-caption stream which has shifting object\ndistribution; while being constantly evaluated by a visually-grounded masked\nlanguage prediction task on held-out test sets. VisCOLL compounds the\nchallenges of continual learning (i.e., learning from continuously shifting\ndata distribution) and compositional generalization (i.e., generalizing to\nnovel compositions). To facilitate research on VisCOLL, we construct two\ndatasets, COCO-shift and Flickr-shift, and benchmark them using different\ncontinual learning methods. Results reveal that SoTA continual learning\napproaches provide little to no improvements on VisCOLL, since storing examples\nof all possible compositions is infeasible. We conduct further ablations and\nanalysis to guide future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00791v2",
            "date": "2020-05-02 11:03:25+00:00",
            "title": "KinGDOM: Knowledge-Guided DOMain adaptation for sentiment analysis",
            "authors": [
                "Deepanway Ghosal",
                "Devamanyu Hazarika",
                "Abhinaba Roy",
                "Navonil Majumder",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "Cross-domain sentiment analysis has received significant attention in recent\nyears, prompted by the need to combat the domain gap between different\napplications that make use of sentiment analysis. In this paper, we take a\nnovel perspective on this task by exploring the role of external commonsense\nknowledge. We introduce a new framework, KinGDOM, which utilizes the ConceptNet\nknowledge graph to enrich the semantics of a document by providing both\ndomain-specific and domain-general background concepts. These concepts are\nlearned by training a graph convolutional autoencoder that leverages\ninter-domain concepts in a domain-invariant manner. Conditioning a popular\ndomain-adversarial baseline method with these learned concepts helps improve\nits performance over state-of-the-art approaches, demonstrating the efficacy of\nour proposed framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00796v3",
            "date": "2020-05-02 11:09:27+00:00",
            "title": "A Simple Language Model for Task-Oriented Dialogue",
            "authors": [
                "Ehsan Hosseini-Asl",
                "Bryan McCann",
                "Chien-Sheng Wu",
                "Semih Yavuz",
                "Richard Socher"
            ],
            "abstract": "Task-oriented dialogue is often decomposed into three tasks: understanding\nuser input, deciding actions, and generating a response. While such\ndecomposition might suggest a dedicated model for each sub-task, we find a\nsimple, unified approach leads to state-of-the-art performance on the MultiWOZ\ndataset. SimpleTOD is a simple approach to task-oriented dialogue that uses a\nsingle causal language model trained on all sub-tasks recast as a single\nsequence prediction problem. This allows SimpleTOD to fully leverage transfer\nlearning from pre-trained, open domain, causal language models such as GPT-2.\nSimpleTOD improves over the prior state-of-the-art by 0.49 points in joint goal\naccuracy for dialogue state tracking. More impressively, SimpleTOD also\nimproves the main metrics used to evaluate action decisions and response\ngeneration in an end-to-end setting for task-oriented dialog systems: inform\nrate by 8.1 points, success rate by 9.7 points, and combined score by 7.2\npoints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00800v1",
            "date": "2020-05-02 11:33:41+00:00",
            "title": "Treebank Embedding Vectors for Out-of-domain Dependency Parsing",
            "authors": [
                "Joachim Wagner",
                "James Barry",
                "Jennifer Foster"
            ],
            "abstract": "A recent advance in monolingual dependency parsing is the idea of a treebank\nembedding vector, which allows all treebanks for a particular language to be\nused as training data while at the same time allowing the model to prefer\ntraining data from one treebank over others and to select the preferred\ntreebank at test time. We build on this idea by 1) introducing a method to\npredict a treebank vector for sentences that do not come from a treebank used\nin training, and 2) exploring what happens when we move away from predefined\ntreebank embedding vectors during test time and instead devise tailored\ninterpolations. We show that 1) there are interpolated vectors that are\nsuperior to the predefined ones, and 2) treebank vectors can be predicted with\nsufficient accuracy, for nine out of ten test languages, to match the\nperformance of an oracle approach that knows the most suitable predefined\ntreebank embedding for the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00816v1",
            "date": "2020-05-02 12:34:17+00:00",
            "title": "DQI: Measuring Data Quality in NLP",
            "authors": [
                "Swaroop Mishra",
                "Anjana Arunkumar",
                "Bhavdeep Sachdeva",
                "Chris Bryan",
                "Chitta Baral"
            ],
            "abstract": "Neural language models have achieved human level performance across several\nNLP datasets. However, recent studies have shown that these models are not\ntruly learning the desired task; rather, their high performance is attributed\nto overfitting using spurious biases, which suggests that the capabilities of\nAI systems have been over-estimated. We introduce a generic formula for Data\nQuality Index (DQI) to help dataset creators create datasets free of such\nunwanted biases. We evaluate this formula using a recently proposed approach\nfor adversarial filtering, AFLite. We propose a new data creation paradigm\nusing DQI to create higher quality data. The data creation paradigm consists of\nseveral data visualizations to help data creators (i) understand the quality of\ndata and (ii) visualize the impact of the created data instance on the overall\nquality. It also has a couple of automation methods to (i) assist data creators\nand (ii) make the model more robust to adversarial attacks. We use DQI along\nwith these automation methods to renovate biased examples in SNLI. We show that\nmodels trained on the renovated SNLI dataset generalize better to out of\ndistribution tasks. Renovation results in reduced model performance, exposing a\nlarge gap with respect to human performance. DQI systematically helps in\ncreating harder benchmarks using active learning. Our work takes the process of\ndynamic dataset creation forward, wherein datasets evolve together with the\nevolving state of the art, therefore serving as a means of benchmarking the\ntrue progress of AI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00842v1",
            "date": "2020-05-02 14:32:40+00:00",
            "title": "Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese",
            "authors": [
                "Tatsuki Kuribayashi",
                "Takumi Ito",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "We examine a methodology using neural language models (LMs) for analyzing the\nword order of language. This LM-based method has the potential to overcome the\ndifficulties existing methods face, such as the propagation of preprocessor\nerrors in count-based methods. In this study, we explore whether the LM-based\nmethod is valid for analyzing the word order. As a case study, this study\nfocuses on Japanese due to its complex and flexible word order. To validate the\nLM-based method, we test (i) parallels between LMs and human word order\npreference, and (ii) consistency of the results obtained using the LM-based\nmethod with previous linguistic studies. Through our experiments, we\ntentatively conclude that LMs display sufficient word order knowledge for usage\nas an analysis tool. Finally, using the LM-based method, we demonstrate the\nrelationship between the canonical word order and topicalization, which had yet\nto be analyzed by large-scale experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00847v1",
            "date": "2020-05-02 15:00:02+00:00",
            "title": "Sources of Transfer in Multilingual Named Entity Recognition",
            "authors": [
                "David Mueller",
                "Nicholas Andrews",
                "Mark Dredze"
            ],
            "abstract": "Named-entities are inherently multilingual, and annotations in any given\nlanguage may be limited. This motivates us to consider polyglot named-entity\nrecognition (NER), where one model is trained using annotated data drawn from\nmore than one language. However, a straightforward implementation of this\nsimple idea does not always work in practice: naive training of NER models\nusing annotated data drawn from multiple languages consistently underperforms\nmodels trained on monolingual data alone, despite having access to more\ntraining data. The starting point of this paper is a simple solution to this\nproblem, in which polyglot models are fine-tuned on monolingual data to\nconsistently and significantly outperform their monolingual counterparts. To\nexplain this phenomena, we explore the sources of multilingual transfer in\npolyglot NER models and examine the weight structure of polyglot models\ncompared to their monolingual counterparts. We find that polyglot models\nefficiently share many parameters across languages and that fine-tuning may\nutilize a large number of those parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00851v1",
            "date": "2020-05-02 15:07:14+00:00",
            "title": "A language score based output selection method for multilingual speech recognition",
            "authors": [
                "Van Huy Nguyen",
                "Thi Quynh Khanh Dinh",
                "Truong Thinh Nguyen",
                "Dang Khoa Mac"
            ],
            "abstract": "The quality of a multilingual speech recognition system can be improved by\nadaptation methods if the input language is specified. For systems that can\naccept multilingual inputs, the popular approach is to apply a language\nidentifier to the input then switch or configure decoders in the next step, or\nuse one more subsequence model to select the output from a set of candidates.\nMotivated by the goal of reducing the latency for real-time applications, in\nthis paper, a language model rescoring method is firstly applied to produce all\npossible candidates for target languages, then a simple score is proposed to\nautomatically select the output without any identifier model or language\nspecification of the input language. The main point is that this score can be\nsimply and automatically estimated on-the-fly so that the whole decoding\npipeline is more simple and compact. Experimental results showed that this\nmethod can achieve the same quality as when the input language is specified. In\naddition, we present to design an English and Vietnamese End-to-End model to\ndeal with not only the problem of cross-lingual speakers but also as a solution\nto improve the accuracy of borrowed words of English in Vietnamese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00870v1",
            "date": "2020-05-02 16:02:18+00:00",
            "title": "Predicting Performance for Natural Language Processing Tasks",
            "authors": [
                "Mengzhou Xia",
                "Antonios Anastasopoulos",
                "Ruochen Xu",
                "Yiming Yang",
                "Graham Neubig"
            ],
            "abstract": "Given the complexity of combinations of tasks, languages, and domains in\nnatural language processing (NLP) research, it is computationally prohibitive\nto exhaustively test newly proposed models on each possible experimental\nsetting. In this work, we attempt to explore the possibility of gaining\nplausible judgments of how well an NLP model can perform under an experimental\nsetting, without actually training or testing the model. To do so, we build\nregression models to predict the evaluation score of an NLP experiment given\nthe experimental settings as input. Experimenting on 9 different NLP tasks, we\nfind that our predictors can produce meaningful predictions over unseen\nlanguages and different modeling architectures, outperforming reasonable\nbaselines as well as human experts. Going further, we outline how our predictor\ncan be used to find a small subset of representative experiments that should be\nrun in order to obtain plausible predictions for all other experimental\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00882v2",
            "date": "2020-05-02 16:33:37+00:00",
            "title": "Improving Truthfulness of Headline Generation",
            "authors": [
                "Kazuki Matsumaru",
                "Sho Takase",
                "Naoaki Okazaki"
            ],
            "abstract": "Most studies on abstractive summarization report ROUGE scores between system\nand reference summaries. However, we have a concern about the truthfulness of\ngenerated summaries: whether all facts of a generated summary are mentioned in\nthe source text. This paper explores improving the truthfulness in headline\ngeneration on two popular datasets. Analyzing headlines generated by the\nstate-of-the-art encoder-decoder model, we show that the model sometimes\ngenerates untruthful headlines. We conjecture that one of the reasons lies in\nuntruthful supervision data used for training the model. In order to quantify\nthe truthfulness of article-headline pairs, we consider the textual entailment\nof whether an article entails its headline. After confirming quite a few\nuntruthful instances in the datasets, this study hypothesizes that removing\nuntruthful instances from the supervision data may remedy the problem of the\nuntruthful behaviors of the model. Building a binary classifier that predicts\nan entailment relation between an article and its headline, we filter out\nuntruthful instances from the supervision data. Experimental results\ndemonstrate that the headline generation model trained on filtered supervision\ndata shows no clear difference in ROUGE scores but remarkable improvements in\nautomatic and manual evaluations of the generated headlines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00889v1",
            "date": "2020-05-02 17:39:40+00:00",
            "title": "Rationalizing Medical Relation Prediction from Corpus-level Statistics",
            "authors": [
                "Zhen Wang",
                "Jennifer Lee",
                "Simon Lin",
                "Huan Sun"
            ],
            "abstract": "Nowadays, the interpretability of machine learning models is becoming\nincreasingly important, especially in the medical domain. Aiming to shed some\nlight on how to rationalize medical relation prediction, we present a new\ninterpretable framework inspired by existing theories on how human memory\nworks, e.g., theories of recall and recognition. Given the corpus-level\nstatistics, i.e., a global co-occurrence graph of a clinical text corpus, to\npredict the relations between two entities, we first recall rich contexts\nassociated with the target entities, and then recognize relational interactions\nbetween these contexts to form model rationales, which will contribute to the\nfinal prediction. We conduct experiments on a real-world public clinical\ndataset and show that our framework can not only achieve competitive predictive\nperformance against a comprehensive list of neural baseline models, but also\npresent rationales to justify its prediction. We further collaborate with\nmedical experts deeply to verify the usefulness of our model rationales for\nclinical decision making.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00891v1",
            "date": "2020-05-02 18:00:48+00:00",
            "title": "Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking",
            "authors": [
                "Giovanni Campagna",
                "Agata Foryciarz",
                "Mehrad Moradshahi",
                "Monica S. Lam"
            ],
            "abstract": "Zero-shot transfer learning for multi-domain dialogue state tracking can\nallow us to handle new domains without incurring the high cost of data\nacquisition. This paper proposes new zero-short transfer learning technique for\ndialogue state tracking where the in-domain training data are all synthesized\nfrom an abstract dialogue model and the ontology of the domain. We show that\ndata augmentation through synthesized data can improve the accuracy of\nzero-shot learning for both the TRADE model and the BERT-based SUMBT model on\nthe MultiWOZ 2.1 dataset. We show training with only synthesized in-domain data\non the SUMBT model can reach about 2/3 of the accuracy obtained with the full\ntraining dataset. We improve the zero-shot learning state of the art on average\nacross domains by 21%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00955v1",
            "date": "2020-05-03 00:31:15+00:00",
            "title": "How Can We Accelerate Progress Towards Human-like Linguistic Generalization?",
            "authors": [
                "Tal Linzen"
            ],
            "abstract": "This position paper describes and critiques the Pretraining-Agnostic\nIdentically Distributed (PAID) evaluation paradigm, which has become a central\ntool for measuring progress in natural language understanding. This paradigm\nconsists of three stages: (1) pre-training of a word prediction model on a\ncorpus of arbitrary size; (2) fine-tuning (transfer learning) on a training set\nrepresenting a classification task; (3) evaluation on a test set drawn from the\nsame distribution as that training set. This paradigm favors simple, low-bias\narchitectures, which, first, can be scaled to process vast amounts of data, and\nsecond, can capture the fine-grained statistical properties of a particular\ndata set, regardless of whether those properties are likely to generalize to\nexamples of the task outside the data set. This contrasts with humans, who\nlearn language from several orders of magnitude less data than the systems\nfavored by this evaluation paradigm, and generalize to new tasks in a\nconsistent way. We advocate for supplementing or replacing PAID with paradigms\nthat reward architectures that generalize as quickly and robustly as humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00956v1",
            "date": "2020-05-03 00:35:19+00:00",
            "title": "Bootstrapping Techniques for Polysynthetic Morphological Analysis",
            "authors": [
                "William Lane",
                "Steven Bird"
            ],
            "abstract": "Polysynthetic languages have exceptionally large and sparse vocabularies,\nthanks to the number of morpheme slots and combinations in a word. This\ncomplexity, together with a general scarcity of written data, poses a challenge\nto the development of natural language technologies. To address this challenge,\nwe offer linguistically-informed approaches for bootstrapping a neural\nmorphological analyzer, and demonstrate its application to Kunwinjku, a\npolysynthetic Australian language. We generate data from a finite state\ntransducer to train an encoder-decoder model. We improve the model by\n\"hallucinating\" missing linguistic structure into the training data, and by\nresampling from a Zipf distribution to simulate a more natural distribution of\nmorphemes. The best model accounts for all instances of reduplication in the\ntest set and achieves an accuracy of 94.7% overall, a 10 percentage point\nimprovement over the FST baseline. This process demonstrates the feasibility of\nbootstrapping a neural morph analyzer from minimal resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00963v1",
            "date": "2020-05-03 02:03:56+00:00",
            "title": "On the Inference Calibration of Neural Machine Translation",
            "authors": [
                "Shuo Wang",
                "Zhaopeng Tu",
                "Shuming Shi",
                "Yang Liu"
            ],
            "abstract": "Confidence calibration, which aims to make model predictions equal to the\ntrue correctness measures, is important for neural machine translation (NMT)\nbecause it is able to offer useful indicators of translation errors in the\ngenerated output. While prior studies have shown that NMT models trained with\nlabel smoothing are well-calibrated on the ground-truth training data, we find\nthat miscalibration still remains a severe challenge for NMT during inference\ndue to the discrepancy between training and inference. By carefully designing\nexperiments on three language pairs, our work provides in-depth analyses of the\ncorrelation between calibration and translation performance as well as\nlinguistic properties of miscalibration and reports a number of interesting\nfindings that might help humans better analyze, understand and improve NMT\nmodels. Based on these observations, we further propose a new graduated label\nsmoothing method that can improve both inference calibration and translation\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00969v1",
            "date": "2020-05-03 02:54:26+00:00",
            "title": "Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints",
            "authors": [
                "Zhenyi Wang",
                "Xiaoyang Wang",
                "Bang An",
                "Dong Yu",
                "Changyou Chen"
            ],
            "abstract": "Text generation from a knowledge base aims to translate knowledge triples to\nnatural language descriptions. Most existing methods ignore the faithfulness\nbetween a generated text description and the original table, leading to\ngenerated information that goes beyond the content of the table. In this paper,\nfor the first time, we propose a novel Transformer-based generation framework\nto achieve the goal. The core techniques in our method to enforce faithfulness\ninclude a new table-text optimal-transport matching loss and a table-text\nembedding similarity loss based on the Transformer model. Furthermore, to\nevaluate faithfulness, we propose a new automatic metric specialized to the\ntable-to-text generation problem. We also provide detailed analysis on each\ncomponent of our model in our experiments. Automatic and human evaluations show\nthat our framework can significantly outperform state-of-the-art by a large\nmargin.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics, pp. 1072-1086, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00970v2",
            "date": "2020-05-03 02:56:05+00:00",
            "title": "Unsupervised Morphological Paradigm Completion",
            "authors": [
                "Huiming Jin",
                "Liwei Cai",
                "Yihui Peng",
                "Chen Xia",
                "Arya D. McCarthy",
                "Katharina Kann"
            ],
            "abstract": "We propose the task of unsupervised morphological paradigm completion. Given\nonly raw text and a lemma list, the task consists of generating the\nmorphological paradigms, i.e., all inflected forms, of the lemmas. From a\nnatural language processing (NLP) perspective, this is a challenging\nunsupervised task, and high-performing systems have the potential to improve\ntools for low-resource languages or to assist linguistic annotators. From a\ncognitive science perspective, this can shed light on how children acquire\nmorphological knowledge. We further introduce a system for the task, which\ngenerates morphological paradigms via the following steps: (i) EDIT TREE\nretrieval, (ii) additional lemma retrieval, (iii) paradigm size discovery, and\n(iv) inflection generation. We perform an evaluation on 14 typologically\ndiverse languages. Our system outperforms trivial baselines with ease and, for\nsome languages, even obtains a higher accuracy than minimally supervised\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00975v2",
            "date": "2020-05-03 03:18:59+00:00",
            "title": "Efficient Second-Order TreeCRF for Neural Dependency Parsing",
            "authors": [
                "Yu Zhang",
                "Zhenghua Li",
                "Min Zhang"
            ],
            "abstract": "In the deep learning (DL) era, parsing models are extremely simplified with\nlittle hurt on performance, thanks to the remarkable capability of multi-layer\nBiLSTMs in context representation. As the most popular graph-based dependency\nparser due to its high efficiency and performance, the biaffine parser directly\nscores single dependencies under the arc-factorization assumption, and adopts a\nvery simple local token-wise cross-entropy training loss. This paper for the\nfirst time presents a second-order TreeCRF extension to the biaffine parser.\nFor a long time, the complexity and inefficiency of the inside-outside\nalgorithm hinder the popularity of TreeCRF. To address this issue, we propose\nan effective way to batchify the inside and Viterbi algorithms for direct large\nmatrix operation on GPUs, and to avoid the complex outside algorithm via\nefficient back-propagation. Experiments and analysis on 27 datasets from 13\nlanguages clearly show that techniques developed before the DL era, such as\nstructural learning (global TreeCRF loss) and high-order modeling are still\nuseful, and can further boost parsing performance over the state-of-the-art\nbiaffine parser, especially for partially annotated training data. We release\nour code at https://github.com/yzhangcs/crfpar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00987v2",
            "date": "2020-05-03 04:49:31+00:00",
            "title": "Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction",
            "authors": [
                "Masahiro Kaneko",
                "Masato Mita",
                "Shun Kiyono",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "This paper investigates how to effectively incorporate a pre-trained masked\nlanguage model (MLM), such as BERT, into an encoder-decoder (EncDec) model for\ngrammatical error correction (GEC). The answer to this question is not as\nstraightforward as one might expect because the previous common methods for\nincorporating a MLM into an EncDec model have potential drawbacks when applied\nto GEC. For example, the distribution of the inputs to a GEC model can be\nconsiderably different (erroneous, clumsy, etc.) from that of the corpora used\nfor pre-training MLMs; however, this issue is not addressed in the previous\nmethods. Our experiments show that our proposed method, where we first\nfine-tune a MLM with a given GEC corpus and then use the output of the\nfine-tuned MLM as additional features in the GEC model, maximizes the benefit\nof the MLM. The best-performing model achieves state-of-the-art performances on\nthe BEA-2019 and CoNLL-2014 benchmarks. Our code is publicly available at:\nhttps://github.com/kanekomasahiro/bert-gec.",
            "journal ref": "Association for Computational Linguistics (ACL-2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01027v1",
            "date": "2020-05-03 09:22:03+00:00",
            "title": "A Position Aware Decay Weighted Network for Aspect based Sentiment Analysis",
            "authors": [
                "Avinash Madasu",
                "Vijjini Anvesh Rao"
            ],
            "abstract": "Aspect Based Sentiment Analysis (ABSA) is the task of identifying sentiment\npolarity of a text given another text segment or aspect. In ABSA, a text can\nhave multiple sentiments depending upon each aspect. Aspect Term Sentiment\nAnalysis (ATSA) is a subtask of ABSA, in which aspect terms are contained\nwithin the given sentence. Most of the existing approaches proposed for ATSA,\nincorporate aspect information through a different subnetwork thereby\noverlooking the advantage of aspect terms' presence within the sentence. In\nthis paper, we propose a model that leverages the positional information of the\naspect. The proposed model introduces a decay mechanism based on position. This\ndecay function mandates the contribution of input words for ABSA. The\ncontribution of a word declines as farther it is positioned from the aspect\nterms in the sentence. The performance is measured on two standard datasets\nfrom SemEval 2014 Task 4. In comparison with recent architectures, the\neffectiveness of the proposed model is demonstrated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01063v1",
            "date": "2020-05-03 12:06:06+00:00",
            "title": "A Two-Stage Masked LM Method for Term Set Expansion",
            "authors": [
                "Guy Kushilevitz",
                "Shaul Markovitch",
                "Yoav Goldberg"
            ],
            "abstract": "We tackle the task of Term Set Expansion (TSE): given a small seed set of\nexample terms from a semantic class, finding more members of that class. The\ntask is of great practical utility, and also of theoretical utility as it\nrequires generalization from few examples. Previous approaches to the TSE task\ncan be characterized as either distributional or pattern-based. We harness the\npower of neural masked language models (MLM) and propose a novel TSE algorithm,\nwhich combines the pattern-based and distributional approaches. Due to the\nsmall size of the seed set, fine-tuning methods are not effective, calling for\nmore creative use of the MLM. The gist of the idea is to use the MLM to first\nmine for informative patterns with respect to the seed set, and then to obtain\nmore members of the seed class by generalizing these patterns. Our method\noutperforms state-of-the-art TSE algorithms. Implementation is available at:\nhttps://github.com/ guykush/TermSetExpansion-MPB/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01107v3",
            "date": "2020-05-03 14:57:24+00:00",
            "title": "Simplifying Paragraph-level Question Generation via Transformer Language Models",
            "authors": [
                "Luis Enrico Lopez",
                "Diane Kathryn Cruz",
                "Jan Christian Blaise Cruz",
                "Charibeth Cheng"
            ],
            "abstract": "Question generation (QG) is a natural language generation task where a model\nis trained to ask questions corresponding to some input text. Most recent\napproaches frame QG as a sequence-to-sequence problem and rely on additional\nfeatures and mechanisms to increase performance; however, these often increase\nmodel complexity, and can rely on auxiliary data unavailable in practical use.\nA single Transformer-based unidirectional language model leveraging transfer\nlearning can be used to produce high quality questions while disposing of\nadditional task-specific complexity. Our QG model, finetuned from GPT-2 Small,\noutperforms several paragraph-level QG baselines on the SQuAD dataset by 0.95\nMETEOR points. Human evaluators rated questions as easy to answer, relevant to\ntheir context paragraph, and corresponding well to natural human speech. Also\nintroduced is a new set of baseline scores on the RACE dataset, which has not\npreviously been used for QG tasks. Further experimentation with varying model\ncapacities and datasets with non-identification type questions is recommended\nin order to further verify the robustness of pretrained Transformer-based LMs\nas question generators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01119v1",
            "date": "2020-05-03 15:38:33+00:00",
            "title": "Emergence of Syntax Needs Minimal Supervision",
            "authors": [
                "Rapha\u00ebl Bailly",
                "Kata G\u00e1bor"
            ],
            "abstract": "This paper is a theoretical contribution to the debate on the learnability of\nsyntax from a corpus without explicit syntax-specific guidance. Our approach\noriginates in the observable structure of a corpus, which we use to define and\nisolate grammaticality (syntactic information) and meaning/pragmatics\ninformation. We describe the formal characteristics of an autonomous syntax and\nshow that it becomes possible to search for syntax-based lexical categories\nwith a simple optimization process, without any prior hypothesis on the form of\nthe model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01159v1",
            "date": "2020-05-03 18:23:06+00:00",
            "title": "Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward",
            "authors": [
                "Luyang Huang",
                "Lingfei Wu",
                "Lu Wang"
            ],
            "abstract": "Sequence-to-sequence models for abstractive summarization have been studied\nextensively, yet the generated summaries commonly suffer from fabricated\ncontent, and are often found to be near-extractive. We argue that, to address\nthese issues, the summarizer should acquire semantic interpretation over input,\ne.g., via structured representation, to allow the generation of more\ninformative summaries. In this paper, we present ASGARD, a novel framework for\nAbstractive Summarization with Graph-Augmentation and semantic-driven RewarD.\nWe propose the use of dual encoders---a sequential document encoder and a\ngraph-structured encoder---to maintain the global context and local\ncharacteristics of entities, complementing each other. We further design a\nreward based on a multiple choice cloze test to drive the model to better\ncapture entity interactions. Results show that our models produce significantly\nhigher ROUGE scores than a variant without knowledge graph as input on both New\nYork Times and CNN/Daily Mail datasets. We also obtain better or comparable\nperformance compared to systems that are fine-tuned from large pretrained\nlanguage models. Human judges further rate our model outputs as more\ninformative and containing fewer unfaithful errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01172v1",
            "date": "2020-05-03 19:48:15+00:00",
            "title": "Similarity Analysis of Contextual Word Representation Models",
            "authors": [
                "John M. Wu",
                "Yonatan Belinkov",
                "Hassan Sajjad",
                "Nadir Durrani",
                "Fahim Dalvi",
                "James Glass"
            ],
            "abstract": "This paper investigates contextual word representation models from the lens\nof similarity analysis. Given a collection of trained models, we measure the\nsimilarity of their internal representations and attention. Critically, these\nmodels come from vastly different architectures. We use existing and novel\nsimilarity measures that aim to gauge the level of localization of information\nin the deep models, and facilitate the investigation of which design factors\naffect model similarity, without requiring any external linguistic annotation.\nThe analysis reveals that models within the same family are more similar to one\nanother, as may be expected. Surprisingly, different architectures have rather\nsimilar representations, but different individual neurons. We also observed\ndifferences in information localization in lower and higher layers and found\nthat higher layers are more affected by fine-tuning on downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01190v1",
            "date": "2020-05-03 21:10:31+00:00",
            "title": "Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models",
            "authors": [
                "Kaiji Lu",
                "Piotr Mardziel",
                "Klas Leino",
                "Matt Fedrikson",
                "Anupam Datta"
            ],
            "abstract": "LSTM-based recurrent neural networks are the state-of-the-art for many\nnatural language processing (NLP) tasks. Despite their performance, it is\nunclear whether, or how, LSTMs learn structural features of natural languages\nsuch as subject-verb number agreement in English. Lacking this understanding,\nthe generality of LSTM performance on this task and their suitability for\nrelated tasks remains uncertain. Further, errors cannot be properly attributed\nto a lack of structural capability, training data omissions, or other\nexceptional faults. We introduce *influence paths*, a causal account of\nstructural properties as carried by paths across gates and neurons of a\nrecurrent neural network. The approach refines the notion of influence (the\nsubject's grammatical number has influence on the grammatical number of the\nsubsequent verb) into a set of gate or neuron-level paths. The set localizes\nand segments the concept (e.g., subject-verb agreement), its constituent\nelements (e.g., the subject), and related or interfering elements (e.g.,\nattractors). We exemplify the methodology on a widely-studied multi-layer LSTM\nlanguage model, demonstrating its accounting for subject-verb number agreement.\nThe results offer both a finer and a more complete view of an LSTM's handling\nof this structural aspect of the English language than prior results based on\ndiagnostic classifiers and ablation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01196v3",
            "date": "2020-05-03 22:10:23+00:00",
            "title": "On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation",
            "authors": [
                "Wei Zhao",
                "Goran Glava\u0161",
                "Maxime Peyrard",
                "Yang Gao",
                "Robert West",
                "Steffen Eger"
            ],
            "abstract": "Evaluation of cross-lingual encoders is usually performed either via\nzero-shot cross-lingual transfer in supervised downstream tasks or via\nunsupervised cross-lingual textual similarity. In this paper, we concern\nourselves with reference-free machine translation (MT) evaluation where we\ndirectly compare source texts to (sometimes low-quality) system translations,\nwhich represents a natural adversarial setup for multilingual encoders.\nReference-free evaluation holds the promise of web-scale comparison of MT\nsystems. We systematically investigate a range of metrics based on\nstate-of-the-art cross-lingual semantic representations obtained with\npretrained M-BERT and LASER. We find that they perform poorly as semantic\nencoders for reference-free MT evaluation and identify their two key\nlimitations, namely, (a) a semantic mismatch between representations of mutual\ntranslations and, more prominently, (b) the inability to punish\n\"translationese\", i.e., low-quality literal translations. We propose two\npartial remedies: (1) post-hoc re-alignment of the vector spaces and (2)\ncoupling of semantic-similarity based metrics with target-side language\nmodeling. In segment-level MT evaluation, our best metric surpasses\nreference-based BLEU by 5.7 correlation points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01204v1",
            "date": "2020-05-03 22:49:44+00:00",
            "title": "On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs",
            "authors": [
                "Adina Williams",
                "Ryan Cotterell",
                "Lawrence Wolf-Sonkin",
                "Dami\u00e1n Blasi",
                "Hanna Wallach"
            ],
            "abstract": "We use large-scale corpora in six different gendered languages, along with\ntools from NLP and information theory, to test whether there is a relationship\nbetween the grammatical genders of inanimate nouns and the adjectives used to\ndescribe those nouns. For all six languages, we find that there is a\nstatistically significant relationship. We also find that there are\nstatistically significant relationships between the grammatical genders of\ninanimate nouns and the verbs that take those nouns as direct objects, as\nindirect objects, and as subjects. We defer a deeper investigation of these\nrelationships for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01281v3",
            "date": "2020-05-04 05:52:10+00:00",
            "title": "WikiUMLS: Aligning UMLS to Wikipedia via Cross-lingual Neural Ranking",
            "authors": [
                "Afshin Rahimi",
                "Timothy Baldwin",
                "Karin Verspoor"
            ],
            "abstract": "We present our work on aligning the Unified Medical Language System (UMLS) to\nWikipedia, to facilitate manual alignment of the two resources. We propose a\ncross-lingual neural reranking model to match a UMLS concept with a Wikipedia\npage, which achieves a recall@1 of 72%, a substantial improvement of 20% over\nword- and char-level BM25, enabling manual alignment with minimal effort. We\nrelease our resources, including ranked Wikipedia pages for 700k UMLS concepts,\nand WikiUMLS, a dataset for training and evaluation of alignment models between\nUMLS and Wikipedia. This will provide easier access to Wikipedia for health\nprofessionals, patients, and NLP systems, including in multilingual settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01282v2",
            "date": "2020-05-04 05:53:34+00:00",
            "title": "Distributional Discrepancy: A Metric for Unconditional Text Generation",
            "authors": [
                "Ping Cai",
                "Xingyuan Chen",
                "Peng Jin",
                "Hongjun Wang",
                "Tianrui Li"
            ],
            "abstract": "The purpose of unconditional text generation is to train a model with real\nsentences, then generate novel sentences of the same quality and diversity as\nthe training data. However, when different metrics are used for comparing the\nmethods of unconditional text generation, contradictory conclusions are drawn.\nThe difficulty is that both the diversity and quality of the sample should be\nconsidered simultaneously when the models are evaluated. To solve this problem,\na novel metric of distributional discrepancy (DD) is designed to evaluate\ngenerators based on the discrepancy between the generated and real training\nsentences. However, it cannot compute the DD directly because the distribution\nof real sentences is unavailable. Thus, we propose a method for estimating the\nDD by training a neural-network-based text classifier. For comparison, three\nexisting metrics, bi-lingual evaluation understudy (BLEU) versus self-BLEU,\nlanguage model score versus reverse language model score, and Fr\\'{e}chet\nembedding distance, along with the proposed DD, are used to evaluate two\npopular generative models of long short-term memory and generative pretrained\ntransformer 2 on both syntactic and real data. Experimental results show that\nDD is significantly better than the three existing metrics for ranking these\ngenerative models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01306v2",
            "date": "2020-05-04 07:38:34+00:00",
            "title": "pyBART: Evidence-based Syntactic Transformations for IE",
            "authors": [
                "Aryeh Tiktinsky",
                "Yoav Goldberg",
                "Reut Tsarfaty"
            ],
            "abstract": "Syntactic dependencies can be predicted with high accuracy, and are useful\nfor both machine-learned and pattern-based information extraction tasks.\nHowever, their utility can be improved. These syntactic dependencies are\ndesigned to accurately reflect syntactic relations, and they do not make\nsemantic relations explicit. Therefore, these representations lack many\nexplicit connections between content words, that would be useful for downstream\napplications. Proposals like English Enhanced UD improve the situation by\nextending universal dependency trees with additional explicit arcs. However,\nthey are not available to Python users, and are also limited in coverage. We\nintroduce a broad-coverage, data-driven and linguistically sound set of\ntransformations, that makes event-structure and many lexical relations\nexplicit. We present pyBART, an easy-to-use open-source Python library for\nconverting English UD trees either to Enhanced UD graphs or to our\nrepresentation. The library can work as a standalone package or be integrated\nwithin a spaCy NLP pipeline. When evaluated in a pattern-based relation\nextraction scenario, our representation results in higher extraction scores\nthan Enhanced UD, while requiring fewer patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01320v1",
            "date": "2020-05-04 08:37:27+00:00",
            "title": "NLP in FinTech Applications: Past, Present and Future",
            "authors": [
                "Chung-Chi Chen",
                "Hen-Hsen Huang",
                "Hsin-Hsi Chen"
            ],
            "abstract": "Financial Technology (FinTech) is one of the worldwide rapidly-rising topics\nin the past five years according to the statistics of FinTech from Google\nTrends. In this position paper, we focus on the researches applying natural\nlanguage processing (NLP) technologies in the finance domain. Our goal is to\nindicate the position we are now and provide the blueprint for future\nresearches. We go through the application scenarios from three aspects\nincluding Know Your Customer (KYC), Know Your Product (KYP), and Satisfy Your\nCustomer (SYC). Both formal documents and informal textual data are analyzed to\nunderstand corporate customers and personal customers. Furthermore, we talk\nover how to dynamically update the features of products from the prospect and\nthe risk points of view. Finally, we discuss satisfying the customers in both\nB2C and C2C business models. After summarizing the past and the recent\nchallenges, we highlight several promising future research directions in the\ntrend of FinTech and the open finance tendency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01328v2",
            "date": "2020-05-04 08:58:54+00:00",
            "title": "DoQA -- Accessing Domain-Specific FAQs via Conversational QA",
            "authors": [
                "Jon Ander Campos",
                "Arantxa Otegi",
                "Aitor Soroa",
                "Jan Deriu",
                "Mark Cieliebak",
                "Eneko Agirre"
            ],
            "abstract": "The goal of this work is to build conversational Question Answering (QA)\ninterfaces for the large body of domain-specific information available in FAQ\nsites. We present DoQA, a dataset with 2,437 dialogues and 10,917 QA pairs. The\ndialogues are collected from three Stack Exchange sites using the Wizard of Oz\nmethod with crowdsourcing. Compared to previous work, DoQA comprises\nwell-defined information needs, leading to more coherent and natural\nconversations with less factoid questions and is multi-domain. In addition, we\nintroduce a more realistic information retrieval(IR) scenario where the system\nneeds to find the answer in any of the FAQ documents. The results of an\nexisting, strong, system show that, thanks to transfer learning from a\nWikipedia QA dataset and fine tuning on a single FAQ domain, it is possible to\nbuild high quality conversational QA systems for FAQs without in-domain\ntraining data. The good results carry over into the more challenging IR\nscenario. In both cases, there is still ample room for improvement, as\nindicated by the higher human upperbound.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics. 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01330v1",
            "date": "2020-05-04 09:05:55+00:00",
            "title": "From SPMRL to NMRL: What Did We Learn (and Unlearn) in a Decade of Parsing Morphologically-Rich Languages (MRLs)?",
            "authors": [
                "Reut Tsarfaty",
                "Dan Bareket",
                "Stav Klein",
                "Amit Seker"
            ],
            "abstract": "It has been exactly a decade since the first establishment of SPMRL, a\nresearch initiative unifying multiple research efforts to address the peculiar\nchallenges of Statistical Parsing for Morphologically-Rich Languages\n(MRLs).Here we reflect on parsing MRLs in that decade, highlight the solutions\nand lessons learned for the architectural, modeling and lexical challenges in\nthe pre-neural era, and argue that similar challenges re-emerge in neural\narchitectures for MRLs. We then aim to offer a climax, suggesting that\nincorporating symbolic ideas proposed in SPMRL terms into nowadays neural\narchitectures has the potential to push NLP for MRLs to a new level. We sketch\nstrategies for designing Neural Models for MRLs (NMRL), and showcase\npreliminary support for these strategies via investigating the task of\nmulti-tagging in Hebrew, a morphologically-rich, high-fusion, language",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01387v3",
            "date": "2020-05-04 11:07:52+00:00",
            "title": "Introducing the VoicePrivacy Initiative",
            "authors": [
                "Natalia Tomashenko",
                "Brij Mohan Lal Srivastava",
                "Xin Wang",
                "Emmanuel Vincent",
                "Andreas Nautsch",
                "Junichi Yamagishi",
                "Nicholas Evans",
                "Jose Patino",
                "Jean-Fran\u00e7ois Bonastre",
                "Paul-Gauthier No\u00e9",
                "Massimiliano Todisco"
            ],
            "abstract": "The VoicePrivacy initiative aims to promote the development of privacy\npreservation tools for speech technology by gathering a new community to define\nthe tasks of interest and the evaluation methodology, and benchmarking\nsolutions through a series of challenges. In this paper, we formulate the voice\nanonymization task selected for the VoicePrivacy 2020 Challenge and describe\nthe datasets used for system development and evaluation. We also present the\nattack models and the associated objective and subjective evaluation metrics.\nWe introduce two anonymization baselines and report objective evaluation\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01483v1",
            "date": "2020-05-04 13:42:30+00:00",
            "title": "Using Context in Neural Machine Translation Training Objectives",
            "authors": [
                "Danielle Saunders",
                "Felix Stahlberg",
                "Bill Byrne"
            ],
            "abstract": "We present Neural Machine Translation (NMT) training using document-level\nmetrics with batch-level documents. Previous sequence-objective approaches to\nNMT training focus exclusively on sentence-level metrics like sentence BLEU\nwhich do not correspond to the desired evaluation metric, typically document\nBLEU. Meanwhile research into document-level NMT training focuses on data or\nmodel architecture rather than training procedure. We find that each of these\nlines of research has a clear space in it for the other, and propose merging\nthem with a scheme that allows a document-level evaluation metric to be used in\nthe NMT training objective.\n  We first sample pseudo-documents from sentence samples. We then approximate\nthe expected document BLEU gradient with Monte Carlo sampling for use as a cost\nfunction in Minimum Risk Training (MRT). This two-level sampling procedure\ngives NMT performance gains over sequence MRT and maximum-likelihood training.\nWe demonstrate that training is more robust for document-level metrics than\nwith sequence metrics. We further demonstrate improvements on NMT with TER and\nGrammatical Error Correction (GEC) using GLEU, both metrics used at the\ndocument level for evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01497v1",
            "date": "2020-05-04 13:56:43+00:00",
            "title": "Towards A Sign Language Gloss Representation Of Modern Standard Arabic",
            "authors": [
                "Salma El Anigri",
                "Mohammed Majid Himmi",
                "Abdelhak Mahmoudi"
            ],
            "abstract": "Over 5% of the world's population (466 million people) has disabling hearing\nloss. 4 million are children. They can be hard of hearing or deaf. Deaf people\nmostly have profound hearing loss. Which implies very little or no hearing.\nOver the world, deaf people often communicate using a sign language with\ngestures of both hands and facial expressions. The sign language is a\nfull-fledged natural language with its own grammar and lexicon. Therefore,\nthere is a need for translation models from and to sign languages. In this\nwork, we are interested in the translation of Modern Standard Arabic(MSAr) into\nsign language. We generated a gloss representation from MSAr that extracts the\nfeatures mandatory for the generation of animation signs. Our approach locates\nthe most pertinent features that maintain the meaning of the input Arabic\nsentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01526v2",
            "date": "2020-05-04 14:36:23+00:00",
            "title": "What-if I ask you to explain: Explaining the effects of perturbations in procedural text",
            "authors": [
                "Dheeraj Rajagopal",
                "Niket Tandon",
                "Bhavana Dalvi",
                "Peter Clark",
                "Eduard Hovy"
            ],
            "abstract": "We address the task of explaining the effects of perturbations in procedural\ntext, an important test of process comprehension. Consider a passage describing\na rabbit's life-cycle: humans can easily explain the effect on the rabbit\npopulation if a female rabbit becomes ill -- i.e., the female rabbit would not\nbecome pregnant, and as a result not have babies leading to a decrease in\nrabbit population. We present QUARTET, a system that constructs such\nexplanations from paragraphs, by modeling the explanation task as a multitask\nlearning problem. QUARTET provides better explanations (based on the sentences\nin the procedural text) compared to several strong baselines on a recent\nprocess comprehension benchmark. We also present a surprising secondary effect:\nour model also achieves a new SOTA with a 7% absolute F1 improvement on a\ndownstream QA task. This illustrates that good explanations do not have to come\nat the expense of end task performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01619v2",
            "date": "2020-05-04 16:24:21+00:00",
            "title": "From Arguments to Key Points: Towards Automatic Argument Summarization",
            "authors": [
                "Roy Bar-Haim",
                "Lilach Eden",
                "Roni Friedman",
                "Yoav Kantor",
                "Dan Lahav",
                "Noam Slonim"
            ],
            "abstract": "Generating a concise summary from a large collection of arguments on a given\ntopic is an intriguing yet understudied problem. We propose to represent such\nsummaries as a small set of talking points, termed \"key points\", each scored\naccording to its salience. We show, by analyzing a large dataset of\ncrowd-contributed arguments, that a small number of key points per topic is\ntypically sufficient for covering the vast majority of the arguments.\nFurthermore, we found that a domain expert can often predict these key points\nin advance. We study the task of argument-to-key point mapping, and introduce a\nnovel large-scale dataset for this task. We report empirical results for an\nextensive set of experiments with this dataset, showing promising performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01630v1",
            "date": "2020-05-04 16:38:54+00:00",
            "title": "The Paradigm Discovery Problem",
            "authors": [
                "Alexander Erdmann",
                "Micha Elsner",
                "Shijie Wu",
                "Ryan Cotterell",
                "Nizar Habash"
            ],
            "abstract": "This work treats the paradigm discovery problem (PDP), the task of learning\nan inflectional morphological system from unannotated sentences. We formalize\nthe PDP and develop evaluation metrics for judging systems. Using currently\navailable resources, we construct datasets for the task. We also devise a\nheuristic benchmark for the PDP and report empirical results on five diverse\nlanguages. Our benchmark system first makes use of word embeddings and string\nsimilarity to cluster forms by cell and by paradigm. Then, we bootstrap a\nneural transducer on top of the clustered data to predict words to realize the\nempty paradigm slots. An error analysis of our system suggests clustering by\ncell across different inflection classes is the most pressing challenge for\nfuture work. Our code and data are available for public use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01634v3",
            "date": "2020-05-04 16:48:29+00:00",
            "title": "Code and Named Entity Recognition in StackOverflow",
            "authors": [
                "Jeniya Tabassum",
                "Mounica Maddela",
                "Wei Xu",
                "Alan Ritter"
            ],
            "abstract": "There is an increasing interest in studying natural language and computer\ncode together, as large corpora of programming texts become readily available\non the Internet. For example, StackOverflow currently has over 15 million\nprogramming related questions written by 8.5 million users. Meanwhile, there is\nstill a lack of fundamental NLP techniques for identifying code tokens or\nsoftware-related named entities that appear within natural language sentences.\nIn this paper, we introduce a new named entity recognition (NER) corpus for the\ncomputer programming domain, consisting of 15,372 sentences annotated with 20\nfine-grained entity types. We trained in-domain BERT representations\n(BERTOverflow) on 152 million sentences from StackOverflow, which lead to an\nabsolute increase of +10 F-1 score over off-the-shelf BERT. We also present the\nSoftNER model which achieves an overall 79.10 F$_1$ score for code and named\nentity recognition on StackOverflow data. Our SoftNER model incorporates a\ncontext-independent code token classifier with corpus-level features to improve\nthe BERT-based tagging model. Our code and data are available at:\nhttps://github.com/jeniyat/StackOverflowNER/",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01641v2",
            "date": "2020-05-04 16:57:31+00:00",
            "title": "A Tale of a Probe and a Parser",
            "authors": [
                "Rowan Hall Maudslay",
                "Josef Valvoda",
                "Tiago Pimentel",
                "Adina Williams",
                "Ryan Cotterell"
            ],
            "abstract": "Measuring what linguistic information is encoded in neural models of language\nhas become popular in NLP. Researchers approach this enterprise by training\n\"probes\" - supervised models designed to extract linguistic structure from\nanother model's output. One such probe is the structural probe (Hewitt and\nManning, 2019), designed to quantify the extent to which syntactic information\nis encoded in contextualised word representations. The structural probe has a\nnovel design, unattested in the parsing literature, the precise benefit of\nwhich is not immediately obvious. To explore whether syntactic probes would do\nbetter to make use of existing techniques, we compare the structural probe to a\nmore traditional parser with an identical lightweight parameterisation. The\nparser outperforms structural probe on UUAS in seven of nine analysed\nlanguages, often by a substantial amount (e.g. by 11.1 points in English).\nUnder a second less common metric, however, there is the opposite trend - the\nstructural probe outperforms the parser. This begs the question: which metric\nshould we prefer?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01672v1",
            "date": "2020-05-04 17:26:25+00:00",
            "title": "Evaluating Explanation Methods for Neural Machine Translation",
            "authors": [
                "Jierui Li",
                "Lemao Liu",
                "Huayang Li",
                "Guanlin Li",
                "Guoping Huang",
                "Shuming Shi"
            ],
            "abstract": "Recently many efforts have been devoted to interpreting the black-box NMT\nmodels, but little progress has been made on metrics to evaluate explanation\nmethods. Word Alignment Error Rate can be used as such a metric that matches\nhuman understanding, however, it can not measure explanation methods on those\ntarget words that are not aligned to any source word. This paper thereby makes\nan initial attempt to evaluate explanation methods from an alternative\nviewpoint. To this end, it proposes a principled metric based on fidelity in\nregard to the predictive behavior of the NMT model. As the exact computation\nfor this metric is intractable, we employ an efficient approach as its\napproximation. On six standard translation tasks, we quantitatively evaluate\nseveral explanation methods in terms of the proposed metric and we reveal some\nvaluable findings for these explanation methods in our experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01677v1",
            "date": "2020-05-04 17:29:59+00:00",
            "title": "Fast and Robust Unsupervised Contextual Biasing for Speech Recognition",
            "authors": [
                "Young Mo Kang",
                "Yingbo Zhou"
            ],
            "abstract": "Automatic speech recognition (ASR) system is becoming a ubiquitous\ntechnology. Although its accuracy is closing the gap with that of human level\nunder certain settings, one area that can further improve is to incorporate\nuser-specific information or context to bias its prediction. A common framework\nis to dynamically construct a small language model from the provided contextual\nmini corpus and interpolate its score with the main language model during the\ndecoding process.\n  Here we propose an alternative approach that does not entail explicit\ncontextual language model. Instead, we derive the bias score for every word in\nthe system vocabulary from the training corpus. The method is unique in that 1)\nit does not require meta-data or class-label annotation for the context or the\ntraining corpus. 2) The bias score is proportional to the word's\nlog-probability, thus not only would it bias the provided context, but also\nrobust against irrelevant context (e.g. user mis-specified or in case where it\nis hard to quantify a tight scope). 3) The bias score for the entire vocabulary\nis pre-determined during the training stage, thereby eliminating\ncomputationally expensive language model construction during inference.\n  We show significant improvement in recognition accuracy when the relevant\ncontext is available. Additionally, we also demonstrate that the proposed\nmethod exhibits high tolerance to false-triggering errors in the presence of\nirrelevant context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01678v2",
            "date": "2020-05-04 17:32:20+00:00",
            "title": "What is Learned in Visually Grounded Neural Syntax Acquisition",
            "authors": [
                "Noriyuki Kojima",
                "Hadar Averbuch-Elor",
                "Alexander M. Rush",
                "Yoav Artzi"
            ],
            "abstract": "Visual features are a promising signal for learning bootstrap textual models.\nHowever, blackbox learning models make it difficult to isolate the specific\ncontribution of visual components. In this analysis, we consider the case study\nof the Visually Grounded Neural Syntax Learner (Shi et al., 2019), a recent\napproach for learning syntax from a visual training signal. By constructing\nsimplified versions of the model, we isolate the core factors that yield the\nmodel's strong performance. Contrary to what the model might be capable of\nlearning, we find significantly less expressive versions produce similar\npredictions and perform just as well, or even better. We also find that a\nsimple lexical signal of noun concreteness plays the main role in the model's\npredictions as opposed to more complex syntactic reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01791v1",
            "date": "2020-05-04 19:01:55+00:00",
            "title": "Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction",
            "authors": [
                "Raphael Schumann",
                "Lili Mou",
                "Yao Lu",
                "Olga Vechtomova",
                "Katja Markert"
            ],
            "abstract": "Automatic sentence summarization produces a shorter version of a sentence,\nwhile preserving its most important information. A good summary is\ncharacterized by language fluency and high information overlap with the source\nsentence. We model these two aspects in an unsupervised objective function,\nconsisting of language modeling and semantic similarity metrics. We search for\na high-scoring summary by discrete optimization. Our proposed method achieves a\nnew state-of-the art for unsupervised sentence summarization according to ROUGE\nscores. Additionally, we demonstrate that the commonly reported ROUGE F1 metric\nis sensitive to summary length. Since this is unwillingly exploited in recent\nwork, we emphasize that future evaluation should explicitly group summarization\nsystems by output length brackets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01822v2",
            "date": "2020-05-04 20:04:47+00:00",
            "title": "Exploring Controllable Text Generation Techniques",
            "authors": [
                "Shrimai Prabhumoye",
                "Alan W Black",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "Neural controllable text generation is an important area gaining attention\ndue to its plethora of applications. Although there is a large body of prior\nwork in controllable text generation, there is no unifying theme. In this work,\nwe provide a new schema of the pipeline of the generation process by\nclassifying it into five modules. The control of attributes in the generation\nprocess requires modification of these modules. We present an overview of\ndifferent techniques used to perform the modulation of these modules. We also\nprovide an analysis on the advantages and disadvantages of these techniques. We\nfurther pave ways to develop new architectures based on the combination of the\nmodules described in this paper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01828v1",
            "date": "2020-05-04 20:20:33+00:00",
            "title": "Understanding Scanned Receipts",
            "authors": [
                "Eric Melz"
            ],
            "abstract": "Tasking machines with understanding receipts can have important applications\nsuch as enabling detailed analytics on purchases, enforcing expense policies,\nand inferring patterns of purchase behavior on large collections of receipts.\nIn this paper, we focus on the task of Named Entity Linking (NEL) of scanned\nreceipt line items; specifically, the task entails associating shorthand text\nfrom OCR'd receipts with a knowledge base (KB) of grocery products. For\nexample, the scanned item \"STO BABY SPINACH\" should be linked to the catalog\nitem labeled \"Simple Truth Organic Baby Spinach\". Experiments that employ a\nvariety of Information Retrieval techniques in combination with statistical\nphrase detection shows promise for effective understanding of scanned receipt\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01840v3",
            "date": "2020-05-04 20:45:39+00:00",
            "title": "Exploring Content Selection in Summarization of Novel Chapters",
            "authors": [
                "Faisal Ladhak",
                "Bryan Li",
                "Yaser Al-Onaizan",
                "Kathleen McKeown"
            ],
            "abstract": "We present a new summarization task, generating summaries of novel chapters\nusing summary/chapter pairs from online study guides. This is a harder task\nthan the news summarization task, given the chapter length as well as the\nextreme paraphrasing and generalization found in the summaries. We focus on\nextractive summarization, which requires the creation of a gold-standard set of\nextractive summaries. We present a new metric for aligning reference summary\nsentences with chapter sentences to create gold extracts and also experiment\nwith different alignment methods. Our experiments demonstrate significant\nimprovement over prior alignment approaches for our task as shown through\nautomatic metrics and a crowd-sourced pyramid analysis. We make our data\ncollection scripts available at\nhttps://github.com/manestay/novel-chapter-dataset .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01866v1",
            "date": "2020-05-04 21:58:02+00:00",
            "title": "Soft Gazetteers for Low-Resource Named Entity Recognition",
            "authors": [
                "Shruti Rijhwani",
                "Shuyan Zhou",
                "Graham Neubig",
                "Jaime Carbonell"
            ],
            "abstract": "Traditional named entity recognition models use gazetteers (lists of\nentities) as features to improve performance. Although modern neural network\nmodels do not require such hand-crafted features for strong performance, recent\nwork has demonstrated their utility for named entity recognition on English\ndata. However, designing such features for low-resource languages is\nchallenging, because exhaustive entity gazetteers do not exist in these\nlanguages. To address this problem, we propose a method of \"soft gazetteers\"\nthat incorporates ubiquitously available information from English knowledge\nbases, such as Wikipedia, into neural named entity recognition models through\ncross-lingual entity linking. Our experiments on four low-resource languages\nshow an average improvement of 4 points in F1 score. Code and data are\navailable at https://github.com/neulab/soft-gazetteers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01879v1",
            "date": "2020-05-04 22:51:54+00:00",
            "title": "FarsBase-KBP: A Knowledge Base Population System for the Persian Knowledge Graph",
            "authors": [
                "Majid Asgari-Bidhendi",
                "Behrooz Janfada",
                "Behrouz Minaei-Bidgoli"
            ],
            "abstract": "While most of the knowledge bases already support the English language, there\nis only one knowledge base for the Persian language, known as FarsBase, which\nis automatically created via semi-structured web information. Unlike English\nknowledge bases such as Wikidata, which have tremendous community support, the\npopulation of a knowledge base like FarsBase must rely on automatically\nextracted knowledge. Knowledge base population can let FarsBase keep growing in\nsize, as the system continues working. In this paper, we present a knowledge\nbase population system for the Persian language, which extracts knowledge from\nunlabeled raw text, crawled from the Web. The proposed system consists of a set\nof state-of-the-art modules such as an entity linking module as well as\ninformation and relation extraction modules designed for FarsBase. Moreover, a\ncanonicalization system is introduced to link extracted relations to FarsBase\nproperties. Then, the system uses knowledge fusion techniques with minimal\nintervention of human experts to integrate and filter the proper knowledge\ninstances, extracted by each module. To evaluate the performance of the\npresented knowledge base population system, we present the first gold dataset\nfor benchmarking knowledge base population in the Persian language, which\nconsisting of 22015 FarsBase triples and verified by human experts. The\nevaluation results demonstrate the efficiency of the proposed system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01901v1",
            "date": "2020-05-05 01:22:29+00:00",
            "title": "OpinionDigest: A Simple Framework for Opinion Summarization",
            "authors": [
                "Yoshihiko Suhara",
                "Xiaolan Wang",
                "Stefanos Angelidis",
                "Wang-Chiew Tan"
            ],
            "abstract": "We present OpinionDigest, an abstractive opinion summarization framework,\nwhich does not rely on gold-standard summaries for training. The framework uses\nan Aspect-based Sentiment Analysis model to extract opinion phrases from\nreviews, and trains a Transformer model to reconstruct the original reviews\nfrom these extractions. At summarization time, we merge extractions from\nmultiple reviews and select the most popular ones. The selected opinions are\nused as input to the trained Transformer model, which verbalizes them into an\nopinion summary. OpinionDigest can also generate customized summaries, tailored\nto specific user needs, by filtering the selected opinions according to their\naspect and/or sentiment. Automatic evaluation on Yelp data shows that our\nframework outperforms competitive baselines. Human studies on two corpora\nverify that OpinionDigest produces informative summaries and shows promising\ncustomization capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02013v1",
            "date": "2020-05-05 09:02:25+00:00",
            "title": "Neural Syntactic Preordering for Controlled Paraphrase Generation",
            "authors": [
                "Tanya Goyal",
                "Greg Durrett"
            ],
            "abstract": "Paraphrasing natural language sentences is a multifaceted process: it might\ninvolve replacing individual words or short phrases, local rearrangement of\ncontent, or high-level restructuring like topicalization or passivization. Past\napproaches struggle to cover this space of paraphrase possibilities in an\ninterpretable manner. Our work, inspired by pre-ordering literature in machine\ntranslation, uses syntactic transformations to softly \"reorder'' the source\nsentence and guide our neural paraphrasing model. First, given an input\nsentence, we derive a set of feasible syntactic rearrangements using an\nencoder-decoder model. This model operates over a partially lexical, partially\nsyntactic view of the sentence and can reorder big chunks. Next, we use each\nproposed rearrangement to produce a sequence of position embeddings, which\nencourages our final encoder-decoder paraphrase model to attend to the source\nwords in a particular order. Our evaluation, both automatic and human, shows\nthat the proposed system retains the quality of the baseline approaches while\ngiving a substantial increase in the diversity of the generated paraphrases",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02068v1",
            "date": "2020-05-05 11:17:07+00:00",
            "title": "Establishing Baselines for Text Classification in Low-Resource Languages",
            "authors": [
                "Jan Christian Blaise Cruz",
                "Charibeth Cheng"
            ],
            "abstract": "While transformer-based finetuning techniques have proven effective in tasks\nthat involve low-resource, low-data environments, a lack of properly\nestablished baselines and benchmark datasets make it hard to compare different\napproaches that are aimed at tackling the low-resource setting. In this work,\nwe provide three contributions. First, we introduce two previously unreleased\ndatasets as benchmark datasets for text classification and low-resource\nmultilabel text classification for the low-resource language Filipino. Second,\nwe pretrain better BERT and DistilBERT models for use within the Filipino\nsetting. Third, we introduce a simple degradation test that benchmarks a\nmodel's resistance to performance degradation as the number of training samples\nare reduced. We analyze our pretrained model's degradation speeds and look\ntowards the use of this method for comparing models aimed at operating within\nthe low-resource setting. We release all our models and datasets for the\nresearch community to use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02146v2",
            "date": "2020-05-05 13:38:02+00:00",
            "title": "Artemis: A Novel Annotation Methodology for Indicative Single Document Summarization",
            "authors": [
                "Rahul Jha",
                "Keping Bi",
                "Yang Li",
                "Mahdi Pakdaman",
                "Asli Celikyilmaz",
                "Ivan Zhiboedov",
                "Kieran McDonald"
            ],
            "abstract": "We describe Artemis (Annotation methodology for Rich, Tractable, Extractive,\nMulti-domain, Indicative Summarization), a novel hierarchical annotation\nprocess that produces indicative summaries for documents from multiple domains.\nCurrent summarization evaluation datasets are single-domain and focused on a\nfew domains for which naturally occurring summaries can be easily found, such\nas news and scientific articles. These are not sufficient for training and\nevaluation of summarization models for use in document management and\ninformation retrieval systems, which need to deal with documents from multiple\ndomains. Compared to other annotation methods such as Relative Utility and\nPyramid, Artemis is more tractable because judges don't need to look at all the\nsentences in a document when making an importance judgment for one of the\nsentences, while providing similarly rich sentence importance annotations. We\ndescribe the annotation process in detail and compare it with other similar\nevaluation systems. We also present analysis and experimental results over a\nsample set of 532 annotated documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02233v2",
            "date": "2020-05-05 14:31:24+00:00",
            "title": "A Survey on Dialog Management: Recent Advances and Challenges",
            "authors": [
                "Yinpei Dai",
                "Huihua Yu",
                "Yixuan Jiang",
                "Chengguang Tang",
                "Yongbin Li",
                "Jian Sun"
            ],
            "abstract": "Dialog management (DM) is a crucial component in a task-oriented dialog\nsystem. Given the dialog history, DM predicts the dialog state and decides the\nnext action that the dialog agent should take. Recently, dialog policy learning\nhas been widely formulated as a Reinforcement Learning (RL) problem, and more\nworks focus on the applicability of DM. In this paper, we survey recent\nadvances and challenges within three critical topics for DM: (1) improving\nmodel scalability to facilitate dialog system modeling in new scenarios, (2)\ndealing with the data scarcity problem for dialog policy learning, and (3)\nenhancing the training efficiency to achieve better task-completion performance\n. We believe that this survey can shed a light on future research in dialog\nmanagement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02235v1",
            "date": "2020-05-05 14:31:47+00:00",
            "title": "Creating a Multimodal Dataset of Images and Text to Study Abusive Language",
            "authors": [
                "Alessio Palmero Aprosio",
                "Stefano Menini",
                "Sara Tonelli"
            ],
            "abstract": "In order to study online hate speech, the availability of datasets containing\nthe linguistic phenomena of interest are of crucial importance. However, when\nit comes to specific target groups, for example teenagers, collecting such data\nmay be problematic due to issues with consent and privacy restrictions.\nFurthermore, while text-only datasets of this kind have been widely used,\nlimitations set by image-based social media platforms like Instagram make it\ndifficult for researchers to experiment with multimodal hate speech data. We\ntherefore developed CREENDER, an annotation tool that has been used in school\nclasses to create a multimodal dataset of images and abusive comments, which we\nmake freely available under Apache 2.0 license. The corpus, with Italian\ncomments, has been analysed from different perspectives, to investigate whether\nthe subject of the images plays a role in triggering a comment. We find that\nusers judge the same images in different ways, although the presence of a\nperson in the picture increases the probability to get an offensive comment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02295v1",
            "date": "2020-05-05 15:48:34+00:00",
            "title": "Code-switching patterns can be an effective route to improve performance of downstream NLP applications: A case study of humour, sarcasm and hate speech detection",
            "authors": [
                "Srijan Bansal",
                "Vishal Garimella",
                "Ayush Suhane",
                "Jasabanta Patro",
                "Animesh Mukherjee"
            ],
            "abstract": "In this paper we demonstrate how code-switching patterns can be utilised to\nimprove various downstream NLP applications. In particular, we encode different\nswitching features to improve humour, sarcasm and hate speech detection tasks.\nWe believe that this simple linguistic observation can also be potentially\nhelpful in improving other similar NLP applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02324v3",
            "date": "2020-05-05 16:47:51+00:00",
            "title": "Neural CRF Model for Sentence Alignment in Text Simplification",
            "authors": [
                "Chao Jiang",
                "Mounica Maddela",
                "Wuwei Lan",
                "Yang Zhong",
                "Wei Xu"
            ],
            "abstract": "The success of a text simplification system heavily depends on the quality\nand quantity of complex-simple sentence pairs in the training corpus, which are\nextracted by aligning sentences between parallel articles. To evaluate and\nimprove sentence alignment quality, we create two manually annotated\nsentence-aligned datasets from two commonly used text simplification corpora,\nNewsela and Wikipedia. We propose a novel neural CRF alignment model which not\nonly leverages the sequential nature of sentences in parallel documents but\nalso utilizes a neural sentence pair model to capture semantic similarity.\nExperiments demonstrate that our proposed approach outperforms all the previous\nwork on monolingual sentence alignment task by more than 5 points in F1. We\napply our CRF aligner to construct two new text simplification datasets,\nNewsela-Auto and Wiki-Auto, which are much larger and of better quality\ncompared to the existing datasets. A Transformer-based seq2seq model trained on\nour datasets establishes a new state-of-the-art for text simplification in both\nautomatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02325v1",
            "date": "2020-05-05 16:48:37+00:00",
            "title": "Digraph of Senegal s local languages: issues, challenges and prospects of their transliteration",
            "authors": [
                "Elhadji Mamadou Nguer",
                "Diop Sokhna Bao",
                "Yacoub Ahmed Fall",
                "Mouhamadou Khoule"
            ],
            "abstract": "The local languages in Senegal, like those of West African countries in\ngeneral, are written based on two alphabets: supplemented Arabic alphabet\n(called Ajami) and Latin alphabet. Each writing has its own applications. Ajami\nwriting is generally used by people educated in Koranic schools for\ncommunication, business, literature (religious texts, poetry, etc.),\ntraditional religious medicine, etc. Writing with Latin characters is used for\nlocalization of ICT (Web, dictionaries, Windows and Google tools translated in\nWolof, etc.), the translation of legal texts (commercial code and constitution\ntranslated in Wolof) and religious ones (Quran and Bible in Wolof), book\nedition, etc. To facilitate both populations general access to knowledge, it is\nuseful to set up transliteration tools between these two scriptures. This work\nfalls within the framework of the implementation of project for a collaborative\nonline dictionary Wolof (Nguer E. M., Khoule M, Thiam M. N., Mbaye B. T.,\nThiare O., Cisse M. T., Mangeot M. 2014), which will involve people using Ajami\nwriting. Our goal will consist, on the one hand in raising the issues related\nto the transliteration and the challenges that this will raise, and on the\nother one, presenting the perspectives.",
            "journal ref": "LTC 2015",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02354v2",
            "date": "2020-05-05 17:38:48+00:00",
            "title": "It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information",
            "authors": [
                "Emanuele Bugliarello",
                "Sabrina J. Mielke",
                "Antonios Anastasopoulos",
                "Ryan Cotterell",
                "Naoaki Okazaki"
            ],
            "abstract": "The performance of neural machine translation systems is commonly evaluated\nin terms of BLEU. However, due to its reliance on target language properties\nand generation, the BLEU metric does not allow an assessment of which\ntranslation directions are more difficult to model. In this paper, we propose\ncross-mutual information (XMI): an asymmetric information-theoretic metric of\nmachine translation difficulty that exploits the probabilistic nature of most\nneural machine translation models. XMI allows us to better evaluate the\ndifficulty of translating text into the target language while controlling for\nthe difficulty of the target-side generation component independent of the\ntranslation task. We then present the first systematic and controlled study of\ncross-lingual translation difficulties using modern neural translation systems.\nCode for replicating our experiments is available online at\nhttps://github.com/e-bug/nmt-difficulty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02473v2",
            "date": "2020-05-05 20:22:18+00:00",
            "title": "Efficient strategies for hierarchical text classification: External knowledge and auxiliary tasks",
            "authors": [
                "Kervy Rivas Rojas",
                "Gina Bustamante",
                "Arturo Oncevay",
                "Marco A. Sobrevilla Cabezudo"
            ],
            "abstract": "In hierarchical text classification, we perform a sequence of inference steps\nto predict the category of a document from top to bottom of a given class\ntaxonomy. Most of the studies have focused on developing novels neural network\narchitectures to deal with the hierarchical structure, but we prefer to look\nfor efficient ways to strengthen a baseline model. We first define the task as\na sequence-to-sequence problem. Afterwards, we propose an auxiliary synthetic\ntask of bottom-up-classification. Then, from external dictionaries, we retrieve\ntextual definitions for the classes of all the hierarchy's layers, and map them\ninto the word vector space. We use the class-definition embeddings as an\nadditional input to condition the prediction of the next layer and in an\nadapted beam search. Whereas the modified search did not provide large gains,\nthe combination of the auxiliary task and the additional input of\nclass-definitions significantly enhance the classification accuracy. With our\nefficient approaches, we outperform previous studies, using a drastically\nreduced number of parameters, in two well-known English datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02517v1",
            "date": "2020-05-05 21:57:27+00:00",
            "title": "Phonetic and Visual Priors for Decipherment of Informal Romanization",
            "authors": [
                "Maria Ryskina",
                "Matthew R. Gormley",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Informal romanization is an idiosyncratic process used by humans in informal\ndigital communication to encode non-Latin script languages into Latin character\nsets found on common keyboards. Character substitution choices differ between\nusers but have been shown to be governed by the same main principles observed\nacross a variety of languages---namely, character pairs are often associated\nthrough phonetic or visual similarity. We propose a noisy-channel WFST cascade\nmodel for deciphering the original non-Latin script from observed romanized\ntext in an unsupervised fashion. We train our model directly on romanized data\nfrom two languages: Egyptian Arabic and Russian. We demonstrate that adding\ninductive bias through phonetic and visual priors on character mappings\nsubstantially improves the model's performance on both languages, yielding\nresults much closer to the supervised skyline. Finally, we introduce a new\ndataset of romanized Russian, collected from a Russian social network website\nand partially annotated for our experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02534v2",
            "date": "2020-05-05 23:32:01+00:00",
            "title": "The Cascade Transformer: an Application for Efficient Answer Sentence Selection",
            "authors": [
                "Luca Soldaini",
                "Alessandro Moschitti"
            ],
            "abstract": "Large transformer-based language models have been shown to be very effective\nin many classification tasks. However, their computational complexity prevents\ntheir use in applications requiring the classification of a large set of\ncandidates. While previous works have investigated approaches to reduce model\nsize, relatively little attention has been paid to techniques to improve batch\nthroughput during inference. In this paper, we introduce the Cascade\nTransformer, a simple yet effective technique to adapt transformer-based models\ninto a cascade of rankers. Each ranker is used to prune a subset of candidates\nin a batch, thus dramatically increasing throughput at inference time. Partial\nencodings from the transformer model are shared among rerankers, providing\nfurther speed-up. When compared to a state-of-the-art transformer model, our\napproach reduces computation by 37% with almost no impact on accuracy, as\nmeasured on two English Question Answering datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02539v2",
            "date": "2020-05-05 23:58:09+00:00",
            "title": "Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback",
            "authors": [
                "Ahmed Elgohary",
                "Saghar Hosseini",
                "Ahmed Hassan Awadallah"
            ],
            "abstract": "We study the task of semantic parse correction with natural language\nfeedback. Given a natural language utterance, most semantic parsing systems\npose the problem as one-shot translation where the utterance is mapped to a\ncorresponding logical form. In this paper, we investigate a more interactive\nscenario where humans can further interact with the system by providing\nfree-form natural language feedback to correct the system when it generates an\ninaccurate interpretation of an initial utterance. We focus on natural language\nto SQL systems and construct, SPLASH, a dataset of utterances, incorrect SQL\ninterpretations and the corresponding natural language feedback. We compare\nvarious reference models for the correction task and show that incorporating\nsuch a rich form of feedback can significantly improve the overall semantic\nparsing accuracy while retaining the flexibility of natural language\ninteraction. While we estimated human correction accuracy is 81.5%, our best\nmodel achieves only 25.1%, which leaves a large gap for improvement in future\nresearch. SPLASH is publicly available at https://aka.ms/Splash_dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02590v2",
            "date": "2020-05-06 04:21:45+00:00",
            "title": "Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders",
            "authors": [
                "Terra Blevins",
                "Luke Zettlemoyer"
            ],
            "abstract": "A major obstacle in Word Sense Disambiguation (WSD) is that word senses are\nnot uniformly distributed, causing existing models to generally perform poorly\non senses that are either rare or unseen during training. We propose a\nbi-encoder model that independently embeds (1) the target word with its\nsurrounding context and (2) the dictionary definition, or gloss, of each sense.\nThe encoders are jointly optimized in the same representation space, so that\nsense disambiguation can be performed by finding the nearest sense embedding\nfor each target word embedding. Our system outperforms previous\nstate-of-the-art models on English all-words WSD; these gains predominantly\ncome from improved performance on rare senses, leading to a 31.1% error\nreduction on less frequent senses over prior work. This demonstrates that rare\nsenses can be more effectively disambiguated by modeling their definitions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02680v4",
            "date": "2020-05-06 09:27:20+00:00",
            "title": "A Top-Down Neural Architecture towards Text-Level Parsing of Discourse Rhetorical Structure",
            "authors": [
                "Longyin Zhang",
                "Yuqing Xing",
                "Fang Kong",
                "Peifeng Li",
                "Guodong Zhou"
            ],
            "abstract": "Due to its great importance in deep natural language understanding and\nvarious down-stream applications, text-level parsing of discourse rhetorical\nstructure (DRS) has been drawing more and more attention in recent years.\nHowever, all the previous studies on text-level discourse parsing adopt\nbottom-up approaches, which much limit the DRS determination on local\ninformation and fail to well benefit from global information of the overall\ndiscourse. In this paper, we justify from both computational and perceptive\npoints-of-view that the top-down architecture is more suitable for text-level\nDRS parsing. On the basis, we propose a top-down neural architecture toward\ntext-level DRS parsing. In particular, we cast discourse parsing as a recursive\nsplit point ranking task, where a split point is classified to different levels\naccording to its rank and the elementary discourse units (EDUs) associated with\nit are arranged accordingly. In this way, we can determine the complete DRS as\na hierarchical tree structure via an encoder-decoder with an internal stack.\nExperimentation on both the English RST-DT corpus and the Chinese CDTB corpus\nshows the great effectiveness of our proposed top-down approach towards\ntext-level DRS parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02693v1",
            "date": "2020-05-06 10:00:55+00:00",
            "title": "Shape of synth to come: Why we should use synthetic data for English surface realization",
            "authors": [
                "Henry Elder",
                "Robert Burke",
                "Alexander O'Connor",
                "Jennifer Foster"
            ],
            "abstract": "The Surface Realization Shared Tasks of 2018 and 2019 were Natural Language\nGeneration shared tasks with the goal of exploring approaches to surface\nrealization from Universal-Dependency-like trees to surface strings for several\nlanguages. In the 2018 shared task there was very little difference in the\nabsolute performance of systems trained with and without additional,\nsynthetically created data, and a new rule prohibiting the use of synthetic\ndata was introduced for the 2019 shared task. Contrary to the findings of the\n2018 shared task, we show, in experiments on the English 2018 dataset, that the\nuse of synthetic data can have a substantial positive effect - an improvement\nof almost 8 BLEU points for a previously state-of-the-art system. We analyse\nthe effects of synthetic data, and we argue that its use should be encouraged\nrather than prohibited so that future research efforts continue to explore\nsystems that can take advantage of such data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02721v3",
            "date": "2020-05-06 10:47:02+00:00",
            "title": "Learning to Understand Child-directed and Adult-directed Speech",
            "authors": [
                "Lieke Gelderloos",
                "Grzegorz Chrupa\u0142a",
                "Afra Alishahi"
            ],
            "abstract": "Speech directed to children differs from adult-directed speech in linguistic\naspects such as repetition, word choice, and sentence length, as well as in\naspects of the speech signal itself, such as prosodic and phonemic variation.\nHuman language acquisition research indicates that child-directed speech helps\nlanguage learners. This study explores the effect of child-directed speech when\nlearning to extract semantic information from speech directly. We compare the\ntask performance of models trained on adult-directed speech (ADS) and\nchild-directed speech (CDS). We find indications that CDS helps in the initial\nstages of learning, but eventually, models trained on ADS reach comparable task\nperformance, and generalize better. The results suggest that this is at least\npartially due to linguistic rather than acoustic properties of the two\nregisters, as we see the same pattern when looking at models trained on\nacoustically comparable synthetic speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02799v1",
            "date": "2020-05-06 13:25:21+00:00",
            "title": "An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining",
            "authors": [
                "Yifan Peng",
                "Qingyu Chen",
                "Zhiyong Lu"
            ],
            "abstract": "Multi-task learning (MTL) has achieved remarkable success in natural language\nprocessing applications. In this work, we study a multi-task learning model\nwith multiple decoders on varieties of biomedical and clinical natural language\nprocessing tasks such as text similarity, relation extraction, named entity\nrecognition, and text inference. Our empirical results demonstrate that the MTL\nfine-tuned models outperform state-of-the-art transformer models (e.g., BERT\nand its variants) by 2.0% and 1.3% in biomedical and clinical domains,\nrespectively. Pairwise MTL further demonstrates more details about which tasks\ncan improve or decrease others. This is particularly helpful in the context\nthat researchers are in the hassle of choosing a suitable model for new\nproblems. The code and models are publicly available at\nhttps://github.com/ncbi-nlp/bluebert",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02827v1",
            "date": "2020-05-05 16:52:46+00:00",
            "title": "Digraphie des langues ouest africaines : Latin2Ajami : un algorithme de translitteration automatique",
            "authors": [
                "El hadji M. Fall",
                "El hadji M. Nguer",
                "Bao Diop Sokhna",
                "Mouhamadou Khoule",
                "Mathieu Mangeot",
                "Mame T. Cisse"
            ],
            "abstract": "The national languages of Senegal, like those of West Africa country in\ngeneral, are written with two alphabets : the Latin alphabet that draws its\nstrength from official decreesm and the completed Arabic script (Ajami),\nwidespread and well integrated, that has little institutional support. This\ndigraph created two worlds ignoring each other. Indeed, Ajami writing is\ngenerally used daily by populations from Koranic schools, while writing with\nthe Latin alphabet is used by people from the public school. To solve this\nproblem, it is useful to establish transliteration tools between these two\nscriptures. Preliminary work (Nguer, Bao-Diop, Fall, khoule, 2015) was\nperformed to locate the problems, challenges and prospects. This present work,\nmaking it subsequently fell into this. Its objective is the study and creation\nof a transliteration algorithm from latin towards Ajami.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02835v1",
            "date": "2020-05-06 14:04:13+00:00",
            "title": "TAG : Type Auxiliary Guiding for Code Comment Generation",
            "authors": [
                "Ruichu Cai",
                "Zhihao Liang",
                "Boyan Xu",
                "Zijian Li",
                "Yuexing Hao",
                "Yao Chen"
            ],
            "abstract": "Existing leading code comment generation approaches with the\nstructure-to-sequence framework ignores the type information of the\ninterpretation of the code, e.g., operator, string, etc. However, introducing\nthe type information into the existing framework is non-trivial due to the\nhierarchical dependence among the type information. In order to address the\nissues above, we propose a Type Auxiliary Guiding encoder-decoder framework for\nthe code comment generation task which considers the source code as an N-ary\ntree with type information associated with each node. Specifically, our\nframework is featured with a Type-associated Encoder and a Type-restricted\nDecoder which enables adaptive summarization of the source code. We further\npropose a hierarchical reinforcement learning method to resolve the training\ndifficulties of our proposed framework. Extensive evaluations demonstrate the\nstate-of-the-art performance of our framework with both the auto-evaluated\nmetrics and case studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02877v4",
            "date": "2020-05-06 14:52:48+00:00",
            "title": "TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking",
            "authors": [
                "Michael Heck",
                "Carel van Niekerk",
                "Nurul Lubis",
                "Christian Geishauser",
                "Hsien-Chin Lin",
                "Marco Moresi",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Task-oriented dialog systems rely on dialog state tracking (DST) to monitor\nthe user's goal during the course of an interaction. Multi-domain and\nopen-vocabulary settings complicate the task considerably and demand scalable\nsolutions. In this paper we present a new approach to DST which makes use of\nvarious copy mechanisms to fill slots with values. Our model has no need to\nmaintain a list of candidate values. Instead, all values are extracted from the\ndialog context on-the-fly. A slot is filled by one of three copy mechanisms:\n(1) Span prediction may extract values directly from the user input; (2) a\nvalue may be copied from a system inform memory that keeps track of the\nsystem's inform operations; (3) a value may be copied over from a different\nslot that is already contained in the dialog state to resolve coreferences\nwithin and across domains. Our approach combines the advantages of span-based\nslot filling methods with memory methods to avoid the use of value picklists\naltogether. We argue that our strategy simplifies the DST task while at the\nsame time achieving state of the art performance on various popular evaluation\nsets including Multiwoz 2.1, where we achieve a joint goal accuracy beyond 55%.",
            "journal ref": "Proceedings of the 21th Annual Meeting of the Special Interest\n  Group on Discourse and Dialogue (July 2020), Pages 35-44; Association for\n  Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02925v1",
            "date": "2020-05-06 15:56:06+00:00",
            "title": "Harvesting and Refining Question-Answer Pairs for Unsupervised QA",
            "authors": [
                "Zhongli Li",
                "Wenhui Wang",
                "Li Dong",
                "Furu Wei",
                "Ke Xu"
            ],
            "abstract": "Question Answering (QA) has shown great success thanks to the availability of\nlarge-scale datasets and the effectiveness of neural models. Recent research\nworks have attempted to extend these successes to the settings with few or no\nlabeled data available. In this work, we introduce two approaches to improve\nunsupervised QA. First, we harvest lexically and syntactically divergent\nquestions from Wikipedia to automatically construct a corpus of question-answer\npairs (named as RefQA). Second, we take advantage of the QA model to extract\nmore appropriate answers, which iteratively refines data over RefQA. We conduct\nexperiments on SQuAD 1.1, and NewsQA by fine-tuning BERT without access to\nmanually annotated data. Our approach outperforms previous unsupervised\napproaches by a large margin and is competitive with early supervised models.\nWe also show the effectiveness of our approach in the few-shot learning\nsetting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02982v1",
            "date": "2020-05-06 17:36:16+00:00",
            "title": "What are the Goals of Distributional Semantics?",
            "authors": [
                "Guy Emerson"
            ],
            "abstract": "Distributional semantic models have become a mainstay in NLP, providing\nuseful features for downstream tasks. However, assessing long-term progress\nrequires explicit long-term goals. In this paper, I take a broad linguistic\nperspective, looking at how well current models can deal with various semantic\nchallenges. Given stark differences between models proposed in different\nsubfields, a broad perspective is needed to see how we could integrate them. I\nconclude that, while linguistic insights can guide the design of model\narchitectures, future progress will require balancing the often conflicting\ndemands of linguistic expressiveness and computational tractability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02991v2",
            "date": "2020-05-06 17:46:40+00:00",
            "title": "Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics",
            "authors": [
                "Guy Emerson"
            ],
            "abstract": "Functional Distributional Semantics provides a linguistically interpretable\nframework for distributional semantics, by representing the meaning of a word\nas a function (a binary classifier), instead of a vector. However, the large\nnumber of latent variables means that inference is computationally expensive,\nand training a model is therefore slow to converge. In this paper, I introduce\nthe Pixie Autoencoder, which augments the generative model of Functional\nDistributional Semantics with a graph-convolutional neural network to perform\namortised variational inference. This allows the model to be trained more\neffectively, achieving better results on two tasks (semantic similarity in\ncontext and semantic composition), and outperforming BERT, a large pre-trained\nlanguage model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03035v1",
            "date": "2020-05-06 18:00:04+00:00",
            "title": "Extracting Headless MWEs from Dependency Parse Trees: Parsing, Tagging, and Joint Modeling Approaches",
            "authors": [
                "Tianze Shi",
                "Lillian Lee"
            ],
            "abstract": "An interesting and frequent type of multi-word expression (MWE) is the\nheadless MWE, for which there are no true internal syntactic dominance\nrelations; examples include many named entities (\"Wells Fargo\") and dates\n(\"July 5, 2020\") as well as certain productive constructions (\"blow for blow\",\n\"day after day\"). Despite their special status and prevalence, current\ndependency-annotation schemes require treating such flat structures as if they\nhad internal syntactic heads, and most current parsers handle them in the same\nfashion as headed constructions. Meanwhile, outside the context of parsing,\ntaggers are typically used for identifying MWEs, but taggers might benefit from\nstructural information. We empirically compare these two common\nstrategies--parsing and tagging--for predicting flat MWEs. Additionally, we\npropose an efficient joint decoding algorithm that combines scores from both\nstrategies. Experimental results on the MWE-Aware English Dependency Corpus and\non six non-English dependency treebanks with frequent flat structures show\nthat: (1) tagging is more accurate than parsing for identifying flat-structure\nMWEs, (2) our joint decoder reconciles the two different views and, for\nnon-BERT features, leads to higher accuracies, and (3) most of the gains result\nfrom feature sharing between the parsers and taggers.",
            "journal ref": "Proceedings of ACL, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03312v1",
            "date": "2020-05-07 08:15:55+00:00",
            "title": "Nakdan: Professional Hebrew Diacritizer",
            "authors": [
                "Avi Shmidman",
                "Shaltiel Shmidman",
                "Moshe Koppel",
                "Yoav Goldberg"
            ],
            "abstract": "We present a system for automatic diacritization of Hebrew text. The system\ncombines modern neural models with carefully curated declarative linguistic\nknowledge and comprehensive manually constructed tables and dictionaries.\nBesides providing state of the art diacritization accuracy, the system also\nsupports an interface for manual editing and correction of the automatic\noutput, and has several features which make it particularly useful for\npreparation of scientific editions of Hebrew texts. The system supports Modern\nHebrew, Rabbinic Hebrew and Poetic Hebrew. The system is freely accessible for\nall use at http://nakdanpro.dicta.org.il.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03361v1",
            "date": "2020-05-07 09:53:25+00:00",
            "title": "JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation",
            "authors": [
                "Zhuoyuan Mao",
                "Fabien Cromieres",
                "Raj Dabre",
                "Haiyue Song",
                "Sadao Kurohashi"
            ],
            "abstract": "Neural machine translation (NMT) needs large parallel corpora for\nstate-of-the-art translation quality. Low-resource NMT is typically addressed\nby transfer learning which leverages large monolingual or parallel corpora for\npre-training. Monolingual pre-training approaches such as MASS (MAsked Sequence\nto Sequence) are extremely effective in boosting NMT quality for languages with\nsmall parallel corpora. However, they do not account for linguistic information\nobtained using syntactic analyzers which is known to be invaluable for several\nNatural Language Processing (NLP) tasks. To this end, we propose JASS,\nJapanese-specific Sequence to Sequence, as a novel pre-training alternative to\nMASS for NMT involving Japanese as the source or target language. JASS is joint\nBMASS (Bunsetsu MASS) and BRSS (Bunsetsu Reordering Sequence to Sequence)\npre-training which focuses on Japanese linguistic units called bunsetsus. In\nour experiments on ASPEC Japanese--English and News Commentary\nJapanese--Russian translation we show that JASS can give results that are\ncompetitive with if not better than those given by MASS. Furthermore, we show\nfor the first time that joint MASS and JASS pre-training gives results that\nsignificantly surpass the individual methods indicating their complementary\nnature. We will release our code, pre-trained models and bunsetsu annotated\ndata as resources for researchers to use in their own NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03375v1",
            "date": "2020-05-07 10:53:05+00:00",
            "title": "2kenize: Tying Subword Sequences for Chinese Script Conversion",
            "authors": [
                "Pranav A",
                "Isabelle Augenstein"
            ],
            "abstract": "Simplified Chinese to Traditional Chinese character conversion is a common\npreprocessing step in Chinese NLP. Despite this, current approaches have poor\nperformance because they do not take into account that a simplified Chinese\ncharacter can correspond to multiple traditional characters. Here, we propose a\nmodel that can disambiguate between mappings and convert between the two\nscripts. The model is based on subword segmentation, two language models, as\nwell as a method for mapping between subword sequences. We further construct\nbenchmark datasets for topic classification and script conversion. Our proposed\nmethod outperforms previous Chinese Character conversion approaches by 6 points\nin accuracy. These results are further confirmed in a downstream application,\nwhere 2kenize is used to convert pretraining dataset for topic classification.\nAn error analysis reveals that our method's particular strengths are in dealing\nwith code-mixing and named entities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03393v2",
            "date": "2020-05-07 11:39:11+00:00",
            "title": "Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation",
            "authors": [
                "Bei Li",
                "Hui Liu",
                "Ziyang Wang",
                "Yufan Jiang",
                "Tong Xiao",
                "Jingbo Zhu",
                "Tongran Liu",
                "Changliang Li"
            ],
            "abstract": "In encoder-decoder neural models, multiple encoders are in general used to\nrepresent the contextual information in addition to the individual sentence. In\nthis paper, we investigate multi-encoder approaches in documentlevel neural\nmachine translation (NMT). Surprisingly, we find that the context encoder does\nnot only encode the surrounding sentences but also behaves as a noise\ngenerator. This makes us rethink the real benefits of multi-encoder in\ncontext-aware translation - some of the improvements come from robust training.\nWe compare several methods that introduce noise and/or well-tuned dropout setup\ninto the training of these encoders. Experimental results show that noisy\ntraining plays an important role in multi-encoder-based NMT, especially when\nthe training data is small. Also, we establish a new state-of-the-art on IWSLT\nFr-En task by careful use of noise generation and dropout methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03436v2",
            "date": "2020-05-07 13:05:03+00:00",
            "title": "Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences",
            "authors": [
                "Dmitry Nikolaev",
                "Ofir Arviv",
                "Taelin Karidi",
                "Neta Kenneth",
                "Veronika Mitnik",
                "Lilja Maria Saeboe",
                "Omri Abend"
            ],
            "abstract": "The patterns in which the syntax of different languages converges and\ndiverges are often used to inform work on cross-lingual transfer. Nevertheless,\nlittle empirical work has been done on quantifying the prevalence of different\nsyntactic divergences across language pairs. We propose a framework for\nextracting divergence patterns for any language pair from a parallel corpus,\nbuilding on Universal Dependencies. We show that our framework provides a\ndetailed picture of cross-language divergences, generalizes previous\napproaches, and lends itself to full automation. We further present a novel\ndataset, a manually word-aligned subset of the Parallel UD corpus in five\nlanguages, and use it to perform a detailed corpus study. We demonstrate the\nusefulness of the resulting analysis by showing that it can help account for\nperformance patterns of a cross-lingual parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03519v1",
            "date": "2020-05-02 01:50:10+00:00",
            "title": "Practical Perspectives on Quality Estimation for Machine Translation",
            "authors": [
                "Junpei Zhou",
                "Ciprian Chelba",
                "Yuezhang",
                "Li"
            ],
            "abstract": "Sentence level quality estimation (QE) for machine translation (MT) attempts\nto predict the translation edit rate (TER) cost of post-editing work required\nto correct MT output. We describe our view on sentence-level QE as dictated by\nseveral practical setups encountered in the industry. We find consumers of MT\noutput---whether human or algorithmic ones---to be primarily interested in a\nbinary quality metric: is the translated sentence adequate as-is or does it\nneed post-editing? Motivated by this we propose a quality classification (QC)\nview on sentence-level QE whereby we focus on maximizing recall at precision\nabove a given threshold. We demonstrate that, while classical QE regression\nmodels fare poorly on this task, they can be re-purposed by replacing the\noutput regression layer with a binary classification one, achieving 50-60\\%\nrecall at 90\\% precision. For a high-quality MT system producing 75-80\\%\ncorrect translations, this promises a significant reduction in post-editing\nwork indeed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03521v3",
            "date": "2020-05-07 14:40:56+00:00",
            "title": "The Danish Gigaword Project",
            "authors": [
                "Leon Str\u00f8mberg-Derczynski",
                "Manuel R. Ciosici",
                "Rebekah Baglini",
                "Morten H. Christiansen",
                "Jacob Aarup Dalsgaard",
                "Riccardo Fusaroli",
                "Peter Juel Henrichsen",
                "Rasmus Hvingelby",
                "Andreas Kirkedal",
                "Alex Speed Kjeldsen",
                "Claus Ladefoged",
                "Finn \u00c5rup Nielsen",
                "Malte Lau Petersen",
                "Jonathan Hvithamar Rystr\u00f8m",
                "Daniel Varab"
            ],
            "abstract": "Danish language technology has been hindered by a lack of broad-coverage\ncorpora at the scale modern NLP prefers. This paper describes the Danish\nGigaword Corpus, the result of a focused effort to provide a diverse and\nfreely-available one billion word corpus of Danish text. The Danish Gigaword\ncorpus covers a wide array of time periods, domains, speakers' socio-economic\nstatus, and Danish dialects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03593v2",
            "date": "2020-05-07 16:22:48+00:00",
            "title": "A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer's Type",
            "authors": [
                "Trevor Cohen",
                "Serguei Pakhomov"
            ],
            "abstract": "In recent years there has been a burgeoning interest in the use of\ncomputational methods to distinguish between elicited speech samples produced\nby patients with dementia, and those from healthy controls. The difference\nbetween perplexity estimates from two neural language models (LMs) - one\ntrained on transcripts of speech produced by healthy participants and the other\ntrained on transcripts from patients with dementia - as a single feature for\ndiagnostic classification of unseen transcripts has been shown to produce\nstate-of-the-art performance. However, little is known about why this approach\nis effective, and on account of the lack of case/control matching in the most\nwidely-used evaluation set of transcripts (DementiaBank), it is unclear if\nthese approaches are truly diagnostic, or are sensitive to other variables. In\nthis paper, we interrogate neural LMs trained on participants with and without\ndementia using synthetic narratives previously developed to simulate\nprogressive semantic dementia by manipulating lexical frequency. We find that\nperplexity of neural LMs is strongly and differentially associated with lexical\nfrequency, and that a mixture model resulting from interpolating control and\ndementia LMs improves upon the current state-of-the-art for models trained on\ntranscript text exclusively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03642v1",
            "date": "2020-05-07 17:46:02+00:00",
            "title": "On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation",
            "authors": [
                "Chaojun Wang",
                "Rico Sennrich"
            ],
            "abstract": "The standard training algorithm in neural machine translation (NMT) suffers\nfrom exposure bias, and alternative algorithms have been proposed to mitigate\nthis. However, the practical impact of exposure bias is under debate. In this\npaper, we link exposure bias to another well-known problem in NMT, namely the\ntendency to generate hallucinations under domain shift. In experiments on three\ndatasets with multiple test domains, we show that exposure bias is partially to\nblame for hallucinations, and that training with Minimum Risk Training, which\navoids exposure bias, can mitigate this. Our analysis explains why exposure\nbias is more problematic under domain shift, and also links exposure bias to\nthe beam search problem, i.e. performance deterioration with increasing beam\nsize. Our results provide a new justification for methods that reduce exposure\nbias: even if they do not increase performance on in-domain test sets, they can\nincrease model robustness to domain shift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03692v2",
            "date": "2020-05-07 18:35:25+00:00",
            "title": "A Systematic Assessment of Syntactic Generalization in Neural Language Models",
            "authors": [
                "Jennifer Hu",
                "Jon Gauthier",
                "Peng Qian",
                "Ethan Wilcox",
                "Roger P. Levy"
            ],
            "abstract": "While state-of-the-art neural network models continue to achieve lower\nperplexity scores on language modeling benchmarks, it remains unknown whether\noptimizing for broad-coverage predictive performance leads to human-like\nsyntactic knowledge. Furthermore, existing work has not provided a clear\npicture about the model properties required to produce proper syntactic\ngeneralizations. We present a systematic evaluation of the syntactic knowledge\nof neural language models, testing 20 combinations of model types and data\nsizes on a set of 34 English-language syntactic test suites. We find\nsubstantial differences in syntactic generalization performance by model\narchitecture, with sequential models underperforming other architectures.\nFactorially manipulating model architecture and training dataset size (1M--40M\nwords), we find that variability in syntactic generalization performance is\nsubstantially greater by architecture than by dataset size for the corpora\ntested in our experiments. Our results also reveal a dissociation between\nperplexity and syntactic generalization performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03695v2",
            "date": "2020-05-07 18:45:48+00:00",
            "title": "LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification",
            "authors": [
                "Erfan Ghadery",
                "Marie-Francine Moens"
            ],
            "abstract": "This paper presents our system entitled `LIIR' for SemEval-2020 Task 12 on\nMultilingual Offensive Language Identification in Social Media (OffensEval 2).\nWe have participated in sub-task A for English, Danish, Greek, Arabic, and\nTurkish languages. We adapt and fine-tune the BERT and Multilingual Bert models\nmade available by Google AI for English and non-English languages respectively.\nFor the English language, we use a combination of two fine-tuned BERT models.\nFor other languages we propose a cross-lingual augmentation approach in order\nto enrich training data and we use Multilingual BERT to obtain sentence\nrepresentations. LIIR achieved rank 14/38, 18/47, 24/86, 24/54, and 25/40 in\nGreek, Turkish, English, Arabic, and Danish languages, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03754v1",
            "date": "2020-05-07 21:00:08+00:00",
            "title": "FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization",
            "authors": [
                "Esin Durmus",
                "He He",
                "Mona Diab"
            ],
            "abstract": "Neural abstractive summarization models are prone to generate content\ninconsistent with the source document, i.e. unfaithful. Existing automatic\nmetrics do not capture such mistakes effectively. We tackle the problem of\nevaluating faithfulness of a generated summary given its source document. We\nfirst collected human annotations of faithfulness for outputs from numerous\nmodels on two datasets. We find that current models exhibit a trade-off between\nabstractiveness and faithfulness: outputs with less word overlap with the\nsource document are more likely to be unfaithful. Next, we propose an automatic\nquestion answering (QA) based metric for faithfulness, FEQA, which leverages\nrecent advances in reading comprehension. Given question-answer pairs generated\nfrom the summary, a QA model extracts answers from the document; non-matched\nanswers indicate unfaithful information in the summary. Among metrics based on\nword overlap, embedding similarity, and learned language understanding models,\nour QA-based metric has significantly higher correlation with human\nfaithfulness scores, especially on highly abstractive summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03774v1",
            "date": "2020-05-07 21:36:59+00:00",
            "title": "Phonotactic Complexity and its Trade-offs",
            "authors": [
                "Tiago Pimentel",
                "Brian Roark",
                "Ryan Cotterell"
            ],
            "abstract": "We present methods for calculating a measure of phonotactic complexity---bits\nper phoneme---that permits a straightforward cross-linguistic comparison. When\ngiven a word, represented as a sequence of phonemic segments such as symbols in\nthe international phonetic alphabet, and a statistical model trained on a\nsample of word types from the language, we can approximately measure bits per\nphoneme using the negative log-probability of that word under the model. This\nsimple measure allows us to compare the entropy across languages, giving\ninsight into how complex a language's phonotactics are. Using a collection of\n1016 basic concept words across 106 languages, we demonstrate a very strong\nnegative correlation of -0.74 between bits per phoneme and the average length\nof words.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  Vol. 8, 1-18",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03923v3",
            "date": "2020-05-08 09:22:33+00:00",
            "title": "Context-Sensitive Generation Network for Handing Unknown Slot Values in Dialogue State Tracking",
            "authors": [
                "Puhai Yang",
                "Heyan Huang",
                "Xian-Ling Mao"
            ],
            "abstract": "As a key component in a dialogue system, dialogue state tracking plays an\nimportant role. It is very important for dialogue state tracking to deal with\nthe problem of unknown slot values. As far as we known, almost all existing\napproaches depend on pointer network to solve the unknown slot value problem.\nThese pointer network-based methods usually have a hidden assumption that there\nis at most one out-of-vocabulary word in an unknown slot value because of the\ncharacter of a pointer network. However, often, there are multiple\nout-of-vocabulary words in an unknown slot value, and it makes the existing\nmethods perform bad. To tackle the problem, in this paper, we propose a novel\nContext-Sensitive Generation network (CSG) which can facilitate the\nrepresentation of out-of-vocabulary words when generating the unknown slot\nvalue. Extensive experiments show that our proposed method performs better than\nthe state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03925v1",
            "date": "2020-05-08 09:37:19+00:00",
            "title": "Learning to Detect Unacceptable Machine Translations for Downstream Tasks",
            "authors": [
                "Meng Zhang",
                "Xin Jiang",
                "Yang Liu",
                "Qun Liu"
            ],
            "abstract": "The field of machine translation has progressed tremendously in recent years.\nEven though the translation quality has improved significantly, current systems\nare still unable to produce uniformly acceptable machine translations for the\nvariety of possible use cases. In this work, we put machine translation in a\ncross-lingual pipeline and introduce downstream tasks to define task-specific\nacceptability of machine translations. This allows us to leverage parallel data\nto automatically generate acceptability annotations on a large scale, which in\nturn help to learn acceptability detectors for the downstream tasks. We conduct\nexperiments to demonstrate the effectiveness of our framework for a range of\ndownstream tasks and translation models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04044v1",
            "date": "2020-05-08 13:47:58+00:00",
            "title": "Literature Triage on Genomic Variation Publications by Knowledge-enhanced Multi-channel CNN",
            "authors": [
                "Chenhui Lv",
                "Qian Lu",
                "Xiang Zhang"
            ],
            "abstract": "Background: To investigate the correlation between genomic variation and\ncertain diseases or phenotypes, the fundamental task is to screen out the\nconcerning publications from massive literature, which is called literature\ntriage. Some knowledge bases, including UniProtKB/Swiss-Prot and NHGRI-EBI GWAS\nCatalog are created for collecting concerning publications. These publications\nare manually curated by experts, which is time-consuming. Moreover, the manual\ncuration of information from literature is not scalable due to the rapidly\nincreasing amount of publications. In order to cut down the cost of literature\ntriage, machine-learning models were adopted to automatically identify\nbiomedical publications. Methods: Comparing to previous studies utilizing\nmachine-learning models for literature triage, we adopt a multi-channel\nconvolutional network to utilize rich textual information and meanwhile bridge\nthe semantic gaps from different corpora. In addition, knowledge embeddings\nlearned from UMLS is also used to provide extra medical knowledge beyond\ntextual features in the process of triage. Results: We demonstrate that our\nmodel outperforms the state-of-the-art models over 5 datasets with the help of\nknowledge embedding and multiple channels. Our model improves the accuracy of\nbiomedical literature triage results. Conclusions: Multiple channels and\nknowledge embeddings enhance the performance of the CNN model in the task of\nbiomedical literature triage. Keywords: Literature Triage; Knowledge Embedding;\nMulti-channel Convolutional Network",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04114v4",
            "date": "2020-05-08 15:40:17+00:00",
            "title": "SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics",
            "authors": [
                "Da Yin",
                "Tao Meng",
                "Kai-Wei Chang"
            ],
            "abstract": "We propose SentiBERT, a variant of BERT that effectively captures\ncompositional sentiment semantics. The model incorporates contextualized\nrepresentation with binary constituency parse tree to capture semantic\ncomposition. Comprehensive experiments demonstrate that SentiBERT achieves\ncompetitive performance on phrase-level sentiment classification. We further\ndemonstrate that the sentiment composition learned from the phrase-level\nannotations on SST can be transferred to other sentiment analysis tasks as well\nas related tasks, such as emotion classification tasks. Moreover, we conduct\nablation studies and design visualization methods to understand SentiBERT. We\nshow that SentiBERT is better than baseline approaches in capturing negation\nand the contrastive relation and model the compositional sentiment semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04177v2",
            "date": "2020-05-08 17:16:35+00:00",
            "title": "Evidence Inference 2.0: More Data, Better Models",
            "authors": [
                "Jay DeYoung",
                "Eric Lehman",
                "Ben Nye",
                "Iain J. Marshall",
                "Byron C. Wallace"
            ],
            "abstract": "How do we most effectively treat a disease or condition? Ideally, we could\nconsult a database of evidence gleaned from clinical trials to answer such\nquestions. Unfortunately, no such database exists; clinical trial results are\ninstead disseminated primarily via lengthy natural language articles. Perusing\nall such articles would be prohibitively time-consuming for healthcare\npractitioners; they instead tend to depend on manually compiled systematic\nreviews of medical literature to inform care.\n  NLP may speed this process up, and eventually facilitate immediate consult of\npublished evidence. The Evidence Inference dataset was recently released to\nfacilitate research toward this end. This task entails inferring the\ncomparative performance of two treatments, with respect to a given outcome,\nfrom a particular article (describing a clinical trial) and identifying\nsupporting evidence. For instance: Does this article report that chemotherapy\nperformed better than surgery for five-year survival rates of operable cancers?\nIn this paper, we collect additional annotations to expand the Evidence\nInference dataset by 25\\%, provide stronger baseline models, systematically\ninspect the errors that these make, and probe dataset quality. We also release\nan abstract only (as opposed to full-texts) version of the task for rapid model\nprototyping. The updated corpus, documentation, and code for new baselines and\nevaluations are available at http://evidence-inference.ebm-nlp.com/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04245v1",
            "date": "2020-05-08 18:00:27+00:00",
            "title": "Balancing Objectives in Counseling Conversations: Advancing Forwards or Looking Backwards",
            "authors": [
                "Justine Zhang",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Throughout a conversation, participants make choices that can orient the flow\nof the interaction. Such choices are particularly salient in the consequential\ndomain of crisis counseling, where a difficulty for counselors is balancing\nbetween two key objectives: advancing the conversation towards a resolution,\nand empathetically addressing the crisis situation.\n  In this work, we develop an unsupervised methodology to quantify how\ncounselors manage this balance. Our main intuition is that if an utterance can\nonly receive a narrow range of appropriate replies, then its likely aim is to\nadvance the conversation forwards, towards a target within that range.\nLikewise, an utterance that can only appropriately follow a narrow range of\npossible utterances is likely aimed backwards at addressing a specific\nsituation within that range. By applying this intuition, we can map each\nutterance to a continuous orientation axis that captures the degree to which it\nis intended to direct the flow of the conversation forwards or backwards.\n  This unsupervised method allows us to characterize counselor behaviors in a\nlarge dataset of crisis counseling conversations, where we show that known\ncounseling strategies intuitively align with this axis. We also illustrate how\nour measure can be indicative of a conversation's progress, as well as its\neffectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04304v1",
            "date": "2020-05-08 22:20:16+00:00",
            "title": "Temporal Common Sense Acquisition with Minimal Supervision",
            "authors": [
                "Ben Zhou",
                "Qiang Ning",
                "Daniel Khashabi",
                "Dan Roth"
            ],
            "abstract": "Temporal common sense (e.g., duration and frequency of events) is crucial for\nunderstanding natural language. However, its acquisition is challenging, partly\nbecause such information is often not expressed explicitly in text, and human\nannotation on such concepts is costly. This work proposes a novel sequence\nmodeling approach that exploits explicit and implicit mentions of temporal\ncommon sense, extracted from a large corpus, to build TACOLM, a temporal common\nsense language model. Our method is shown to give quality predictions of\nvarious dimensions of temporal common sense (on UDST and a newly collected\ndataset from RealNews). It also produces representations of events for relevant\ntasks such as duration comparison, parent-child relations, event coreference\nand temporal QA (on TimeBank, HiEVE and MCTACO) that are better than using the\nstandard BERT. Thus, it will be an important component of temporal NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04315v2",
            "date": "2020-05-08 23:31:31+00:00",
            "title": "Probing Linguistic Systematicity",
            "authors": [
                "Emily Goodwin",
                "Koustuv Sinha",
                "Timothy J. O'Donnell"
            ],
            "abstract": "Recently, there has been much interest in the question of whether deep\nnatural language understanding models exhibit systematicity; generalizing such\nthat units like words make consistent contributions to the meaning of the\nsentences in which they appear. There is accumulating evidence that neural\nmodels often generalize non-systematically. We examined the notion of\nsystematicity from a linguistic perspective, defining a set of probes and a set\nof metrics to measure systematic behaviour. We also identified ways in which\nnetwork architectures can generalize non-systematically, and discuss why such\nforms of generalization may be unsatisfying. As a case study, we performed a\nseries of experiments in the setting of natural language inference (NLI),\ndemonstrating that some NLU systems achieve high overall performance despite\nbeing non-systematic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04322v1",
            "date": "2020-05-09 00:00:08+00:00",
            "title": "LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation",
            "authors": [
                "Gustavo Aguilar",
                "Sudipta Kar",
                "Thamar Solorio"
            ],
            "abstract": "Recent trends in NLP research have raised an interest in linguistic\ncode-switching (CS); modern approaches have been proposed to solve a wide range\nof NLP tasks on multiple language pairs. Unfortunately, these proposed methods\nare hardly generalizable to different code-switched languages. In addition, it\nis unclear whether a model architecture is applicable for a different task\nwhile still being compatible with the code-switching setting. This is mainly\nbecause of the lack of a centralized benchmark and the sparse corpora that\nresearchers employ based on their specific needs and interests. To facilitate\nresearch in this direction, we propose a centralized benchmark for Linguistic\nCode-switching Evaluation (LinCE) that combines ten corpora covering four\ndifferent code-switched language pairs (i.e., Spanish-English, Nepali-English,\nHindi-English, and Modern Standard Arabic-Egyptian Arabic) and four tasks\n(i.e., language identification, named entity recognition, part-of-speech\ntagging, and sentiment analysis). As part of the benchmark centralization\neffort, we provide an online platform at ritual.uh.edu/lince, where researchers\ncan submit their results while comparing with others in real-time. In addition,\nwe provide the scores of different popular models, including LSTM, ELMo, and\nmultilingual BERT so that the NLP community can compare against\nstate-of-the-art systems. LinCE is a continuous effort, and we will expand it\nwith more low-resource languages and tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04346v2",
            "date": "2020-05-09 02:16:05+00:00",
            "title": "Diversifying Dialogue Generation with Non-Conversational Text",
            "authors": [
                "Hui Su",
                "Xiaoyu Shen",
                "Sanqiang Zhao",
                "Xiao Zhou",
                "Pengwei Hu",
                "Randy Zhong",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Neural network-based sequence-to-sequence (seq2seq) models strongly suffer\nfrom the low-diversity problem when it comes to open-domain dialogue\ngeneration. As bland and generic utterances usually dominate the frequency\ndistribution in our daily chitchat, avoiding them to generate more interesting\nresponses requires complex data filtering, sampling techniques or modifying the\ntraining objective. In this paper, we propose a new perspective to diversify\ndialogue generation by leveraging non-conversational text. Compared with\nbilateral conversations, non-conversational text are easier to obtain, more\ndiverse and cover a much broader range of topics. We collect a large-scale\nnon-conversational corpus from multi sources including forum comments, idioms\nand book snippets. We further present a training paradigm to effectively\nincorporate these text via iterative back translation. The resulting model is\ntested on two conversational datasets and is shown to produce significantly\nmore diverse responses without sacrificing the relevance with context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04418v1",
            "date": "2020-05-09 11:02:46+00:00",
            "title": "The Structured Weighted Violations MIRA",
            "authors": [
                "Dor Ringel",
                "Rotem Dror",
                "Roi Reichart"
            ],
            "abstract": "We present the Structured Weighted Violation MIRA (SWVM), a new structured\nprediction algorithm that is based on an hybridization between MIRA (Crammer\nand Singer, 2003) and the structured weighted violations perceptron (SWVP)\n(Dror and Reichart, 2016). We demonstrate that the concepts developed in (Dror\nand Reichart, 2016) combined with a powerful structured prediction algorithm\ncan improve performance on sequence labeling tasks. In experiments with\nsyntactic chunking and named entity recognition (NER), the new algorithm\nsubstantially outperforms the original MIRA as well as the original structured\nperceptron and SWVP. Our code is available at\nhttps://github.com/dorringel/SWVM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04470v2",
            "date": "2020-05-09 16:02:52+00:00",
            "title": "Empowering Active Learning to Jointly Optimize System and User Demands",
            "authors": [
                "Ji-Ung Lee",
                "Christian M. Meyer",
                "Iryna Gurevych"
            ],
            "abstract": "Existing approaches to active learning maximize the system performance by\nsampling unlabeled instances for annotation that yield the most efficient\ntraining. However, when active learning is integrated with an end-user\napplication, this can lead to frustration for participating users, as they\nspend time labeling instances that they would not otherwise be interested in\nreading. In this paper, we propose a new active learning approach that jointly\noptimizes the seemingly counteracting objectives of the active learning system\n(training efficiently) and the user (receiving useful instances). We study our\napproach in an educational application, which particularly benefits from this\ntechnique as the system needs to rapidly learn to predict the appropriateness\nof an exercise to a particular user, while the users should receive only\nexercises that match their skills. We evaluate multiple learning strategies and\nuser types with data from real users and find that our joint approach better\nsatisfies both objectives when alternative methods lead to many unsuitable\nexercises for end users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04611v1",
            "date": "2020-05-10 09:28:12+00:00",
            "title": "How Context Affects Language Models' Factual Predictions",
            "authors": [
                "Fabio Petroni",
                "Patrick Lewis",
                "Aleksandra Piktus",
                "Tim Rockt\u00e4schel",
                "Yuxiang Wu",
                "Alexander H. Miller",
                "Sebastian Riedel"
            ],
            "abstract": "When pre-trained on large unsupervised textual corpora, language models are\nable to store and retrieve factual knowledge to some extent, making it possible\nto use them directly for zero-shot cloze-style question answering. However,\nstoring factual knowledge in a fixed number of weights of a language model\nclearly has limitations. Previous approaches have successfully provided access\nto information outside the model weights using supervised architectures that\ncombine an information retrieval system with a machine reading component. In\nthis paper, we go a step further and integrate information from a retrieval\nsystem with a pre-trained language model in a purely unsupervised way. We\nreport that augmenting pre-trained language models in this way dramatically\nimproves performance and that the resulting system, despite being unsupervised,\nis competitive with a supervised machine reading baseline. Furthermore,\nprocessing query and context with different segment tokens allows BERT to\nutilize its Next Sentence Prediction pre-trained classifier to determine\nwhether the context is relevant or not, substantially improving BERT's\nzero-shot cloze-style question-answering performance and making its predictions\nrobust to noisy contexts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04749v2",
            "date": "2020-05-10 18:50:40+00:00",
            "title": "A SentiWordNet Strategy for Curriculum Learning in Sentiment Analysis",
            "authors": [
                "Vijjini Anvesh Rao",
                "Kaveri Anuranjana",
                "Radhika Mamidi"
            ],
            "abstract": "Curriculum Learning (CL) is the idea that learning on a training set\nsequenced or ordered in a manner where samples range from easy to difficult,\nresults in an increment in performance over otherwise random ordering. The idea\nparallels cognitive science's theory of how human brains learn, and that\nlearning a difficult task can be made easier by phrasing it as a sequence of\neasy to difficult tasks. This idea has gained a lot of traction in machine\nlearning and image processing for a while and recently in Natural Language\nProcessing (NLP). In this paper, we apply the ideas of curriculum learning,\ndriven by SentiWordNet in a sentiment analysis setting. In this setting, given\na text segment, our aim is to extract its sentiment or polarity. SentiWordNet\nis a lexical resource with sentiment polarity annotations. By comparing\nperformance with other curriculum strategies and with no curriculum, the\neffectiveness of the proposed strategy is presented. Convolutional, Recurrence,\nand Attention-based architectures are employed to assess this improvement. The\nmodels are evaluated on a standard sentiment dataset, Stanford Sentiment\nTreebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05189v2",
            "date": "2020-05-11 15:26:07+00:00",
            "title": "A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction",
            "authors": [
                "Yilin Niu",
                "Fangkai Jiao",
                "Mantong Zhou",
                "Ting Yao",
                "Jingfang Xu",
                "Minlie Huang"
            ],
            "abstract": "Neural models have achieved great success on machine reading comprehension\n(MRC), many of which typically consist of two components: an evidence extractor\nand an answer predictor. The former seeks the most relevant information from a\nreference text, while the latter is to locate or generate answers from the\nextracted evidence. Despite the importance of evidence labels for training the\nevidence extractor, they are not cheaply accessible, particularly in many\nnon-extractive MRC tasks such as YES/NO question answering and multi-choice\nMRC.\n  To address this problem, we present a Self-Training method (STM), which\nsupervises the evidence extractor with auto-generated evidence labels in an\niterative process. At each iteration, a base MRC model is trained with golden\nanswers and noisy evidence labels. The trained model will predict pseudo\nevidence labels as extra supervision in the next iteration. We evaluate STM on\nseven datasets over three MRC tasks. Experimental results demonstrate the\nimprovement on existing MRC models, and we also analyze how and why such a\nself-training method works in MRC. The source code can be obtained from\nhttps://github.com/SparkJiao/Self-Training-MRC",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05255v1",
            "date": "2020-05-11 16:54:19+00:00",
            "title": "Toward Better Storylines with Sentence-Level Language Models",
            "authors": [
                "Daphne Ippolito",
                "David Grangier",
                "Douglas Eck",
                "Chris Callison-Burch"
            ],
            "abstract": "We propose a sentence-level language model which selects the next sentence in\na story from a finite set of fluent alternatives. Since it does not need to\nmodel fluency, the sentence-level language model can focus on longer range\ndependencies, which are crucial for multi-sentence coherence. Rather than\ndealing with individual words, our method treats the story so far as a list of\npre-trained sentence embeddings and predicts an embedding for the next\nsentence, which is more efficient than predicting word embeddings. Notably this\nallows us to consider a large number of candidates for the next sentence during\ntraining. We demonstrate the effectiveness of our approach with\nstate-of-the-art accuracy on the unsupervised Story Cloze task and with\npromising results on larger-scale next sentence prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05257v3",
            "date": "2020-05-11 16:54:42+00:00",
            "title": "A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering",
            "authors": [
                "Nils Holzenberger",
                "Andrew Blair-Stanek",
                "Benjamin Van Durme"
            ],
            "abstract": "Legislation can be viewed as a body of prescriptive rules expressed in\nnatural language. The application of legislation to facts of a case we refer to\nas statutory reasoning, where those facts are also expressed in natural\nlanguage. Computational statutory reasoning is distinct from most existing work\nin machine reading, in that much of the information needed for deciding a case\nis declared exactly once (a law), while the information needed in much of\nmachine reading tends to be learned through distributional language statistics.\nTo investigate the performance of natural language understanding approaches on\nstatutory reasoning, we introduce a dataset, together with a legal-domain text\ncorpus. Straightforward application of machine reading models exhibits low\nout-of-the-box performance on our questions, whether or not they have been\nfine-tuned to the legal domain. We contrast this with a hand-constructed\nProlog-based system, designed to fully solve the task. These experiments\nsupport a discussion of the challenges facing statutory reasoning moving\nforward, which we argue is an interesting real-world task that can motivate the\ndevelopment of models able to utilize prescriptive rules specified in natural\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05264v1",
            "date": "2020-05-11 17:07:20+00:00",
            "title": "Multidirectional Associative Optimization of Function-Specific Word Representations",
            "authors": [
                "Daniela Gerz",
                "Ivan Vuli\u0107",
                "Marek Rei",
                "Roi Reichart",
                "Anna Korhonen"
            ],
            "abstract": "We present a neural framework for learning associations between interrelated\ngroups of words such as the ones found in Subject-Verb-Object (SVO) structures.\nOur model induces a joint function-specific word vector space, where vectors of\ne.g. plausible SVO compositions lie close together. The model retains\ninformation about word group membership even in the joint space, and can\nthereby effectively be applied to a number of tasks reasoning over the SVO\nstructure. We show the robustness and versatility of the proposed framework by\nreporting state-of-the-art results on the tasks of estimating selectional\npreference and event similarity. The results indicate that the combinations of\nrepresentations learned with our task-independent model outperform\ntask-specific architectures from prior work, while reducing the number of\nparameters by up to 95%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05414v2",
            "date": "2020-05-11 20:21:25+00:00",
            "title": "Segmenting Scientific Abstracts into Discourse Categories: A Deep Learning-Based Approach for Sparse Labeled Data",
            "authors": [
                "Soumya Banerjee",
                "Debarshi Kumar Sanyal",
                "Samiran Chattopadhyay",
                "Plaban Kumar Bhowmick",
                "Parthapratim Das"
            ],
            "abstract": "The abstract of a scientific paper distills the contents of the paper into a\nshort paragraph. In the biomedical literature, it is customary to structure an\nabstract into discourse categories like BACKGROUND, OBJECTIVE, METHOD, RESULT,\nand CONCLUSION, but this segmentation is uncommon in other fields like computer\nscience. Explicit categories could be helpful for more granular, that is,\ndiscourse-level search and recommendation. The sparsity of labeled data makes\nit challenging to construct supervised machine learning solutions for automatic\ndiscourse-level segmentation of abstracts in non-bio domains. In this paper, we\naddress this problem using transfer learning. In particular, we define three\ndiscourse categories BACKGROUND, TECHNIQUE, OBSERVATION-for an abstract because\nthese three categories are the most common. We train a deep neural network on\nstructured abstracts from PubMed, then fine-tune it on a small hand-labeled\ncorpus of computer science papers. We observe an accuracy of 75% on the test\ncorpus. We perform an ablation study to highlight the roles of the different\nparts of the model. Our method appears to be a promising solution to the\nautomatic segmentation of abstracts, where the labeled data is sparse.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05477v2",
            "date": "2020-05-11 22:57:04+00:00",
            "title": "Neural Polysynthetic Language Modelling",
            "authors": [
                "Lane Schwartz",
                "Francis Tyers",
                "Lori Levin",
                "Christo Kirov",
                "Patrick Littell",
                "Chi-kiu Lo",
                "Emily Prud'hommeaux",
                "Hyunji Hayley Park",
                "Kenneth Steimel",
                "Rebecca Knowles",
                "Jeffrey Micher",
                "Lonny Strunk",
                "Han Liu",
                "Coleman Haley",
                "Katherine J. Zhang",
                "Robbie Jimmerson",
                "Vasilisa Andriyanets",
                "Aldrian Obaja Muis",
                "Naoki Otani",
                "Jong Hyuk Park",
                "Zhisong Zhang"
            ],
            "abstract": "Research in natural language processing commonly assumes that approaches that\nwork well for English and and other widely-used languages are \"language\nagnostic\". In high-resource languages, especially those that are analytic, a\ncommon approach is to treat morphologically-distinct variants of a common root\nas completely independent word types. This assumes, that there are limited\nmorphological inflections per root, and that the majority will appear in a\nlarge enough corpus, so that the model can adequately learn statistics about\neach form. Approaches like stemming, lemmatization, or subword segmentation are\noften used when either of those assumptions do not hold, particularly in the\ncase of synthetic languages like Spanish or Russian that have more inflection\nthan English.\n  In the literature, languages like Finnish or Turkish are held up as extreme\nexamples of complexity that challenge common modelling assumptions. Yet, when\nconsidering all of the world's languages, Finnish and Turkish are closer to the\naverage case. When we consider polysynthetic languages (those at the extreme of\nmorphological complexity), approaches like stemming, lemmatization, or subword\nmodelling may not suffice. These languages have very high numbers of hapax\nlegomena, showing the need for appropriate morphological handling of words,\nwithout which it is not possible for a model to capture enough word statistics.\n  We examine the current state-of-the-art in language modelling, machine\ntranslation, and text prediction for four polysynthetic languages: Guaran\\'i,\nSt. Lawrence Island Yupik, Central Alaskan Yupik, and Inuktitut. We then\npropose a novel framework for language modelling that combines knowledge\nrepresentations from finite-state morphological analyzers with Tensor Product\nRepresentations in order to enable neural language models capable of handling\nthe full range of typologically variant languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05480v2",
            "date": "2020-05-11 23:01:22+00:00",
            "title": "Schema-Guided Natural Language Generation",
            "authors": [
                "Yuheng Du",
                "Shereen Oraby",
                "Vittorio Perera",
                "Minmin Shen",
                "Anjali Narayan-Chen",
                "Tagyoung Chung",
                "Anu Venkatesh",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Neural network based approaches to data-to-text natural language generation\n(NLG) have gained popularity in recent years, with the goal of generating a\nnatural language prompt that accurately realizes an input meaning\nrepresentation. To facilitate the training of neural network models,\nresearchers created large datasets of paired utterances and their meaning\nrepresentations. However, the creation of such datasets is an arduous task and\nthey mostly consist of simple meaning representations composed of slot and\nvalue tokens to be realized. These representations do not include any\ncontextual information that an NLG system can use when trying to generalize,\nsuch as domain information and descriptions of slots and values. In this paper,\nwe present the novel task of Schema-Guided Natural Language Generation\n(SG-NLG). Here, the goal is still to generate a natural language prompt, but in\nSG-NLG, the input MRs are paired with rich schemata providing contextual\ninformation. To generate a dataset for SG-NLG we re-purpose an existing dataset\nfor another task: dialog state tracking, which includes a large and rich schema\nspanning multiple different attributes, including information about the domain,\nuser intent, and slot descriptions. We train different state-of-the-art models\nfor neural natural language generation on this dataset and show that in many\ncases, including rich schema information allows our models to produce higher\nquality outputs both in terms of semantics and diversity. We also conduct\nexperiments comparing model performance on seen versus unseen domains, and\npresent a human evaluation demonstrating high ratings for overall output\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05507v1",
            "date": "2020-05-12 01:24:43+00:00",
            "title": "A Framework for Hierarchical Multilingual Machine Translation",
            "authors": [
                "Ion Madrazo Azpiazu",
                "Maria Soledad Pera"
            ],
            "abstract": "Multilingual machine translation has recently been in vogue given its\npotential for improving machine translation performance for low-resource\nlanguages via transfer learning. Empirical examinations demonstrating the\nsuccess of existing multilingual machine translation strategies, however, are\nlimited to experiments in specific language groups. In this paper, we present a\nhierarchical framework for building multilingual machine translation strategies\nthat takes advantage of a typological language family tree for enabling\ntransfer among similar languages while avoiding the negative effects that\nresult from incorporating languages that are too different to each other.\nExhaustive experimentation on a dataset with 41 languages demonstrates the\nvalidity of the proposed framework, especially when it comes to improving the\nperformance of low-resource languages via the use of typologically related\nfamilies for which richer sets of resources are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05607v1",
            "date": "2020-05-12 08:26:15+00:00",
            "title": "Neighborhood Matching Network for Entity Alignment",
            "authors": [
                "Yuting Wu",
                "Xiao Liu",
                "Yansong Feng",
                "Zheng Wang",
                "Dongyan Zhao"
            ],
            "abstract": "Structural heterogeneity between knowledge graphs is an outstanding challenge\nfor entity alignment. This paper presents Neighborhood Matching Network (NMN),\na novel entity alignment framework for tackling the structural heterogeneity\nchallenge. NMN estimates the similarities between entities to capture both the\ntopological structure and the neighborhood difference. It provides two\ninnovative components for better learning representations for entity alignment.\nIt first uses a novel graph sampling method to distill a discriminative\nneighborhood for each entity. It then adopts a cross-graph neighborhood\nmatching module to jointly encode the neighborhood difference for a given\nentity pair. Such strategies allow NMN to effectively construct\nmatching-oriented entity representations while ignoring noisy neighbors that\nhave a negative impact on the alignment task. Extensive experiments performed\non three entity alignment datasets show that NMN can well estimate the\nneighborhood similarity in more tough cases and significantly outperforms 12\nprevious state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05635v2",
            "date": "2020-05-12 09:23:32+00:00",
            "title": "SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis",
            "authors": [
                "Hao Tian",
                "Can Gao",
                "Xinyan Xiao",
                "Hao Liu",
                "Bolei He",
                "Hua Wu",
                "Haifeng Wang",
                "Feng Wu"
            ],
            "abstract": "Recently, sentiment analysis has seen remarkable advance with the help of\npre-training approaches. However, sentiment knowledge, such as sentiment words\nand aspect-sentiment pairs, is ignored in the process of pre-training, despite\nthe fact that they are widely used in traditional sentiment analysis\napproaches. In this paper, we introduce Sentiment Knowledge Enhanced\nPre-training (SKEP) in order to learn a unified sentiment representation for\nmultiple sentiment analysis tasks. With the help of automatically-mined\nknowledge, SKEP conducts sentiment masking and constructs three sentiment\nknowledge prediction objectives, so as to embed sentiment information at the\nword, polarity and aspect level into pre-trained sentiment representation. In\nparticular, the prediction of aspect-sentiment pairs is converted into\nmulti-label classification, aiming to capture the dependency between words in a\npair. Experiments on three kinds of sentiment tasks show that SKEP\nsignificantly outperforms strong pre-training baseline, and achieves new\nstate-of-the-art results on most of the test datasets. We release our code at\nhttps://github.com/baidu/Senta.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05639v2",
            "date": "2020-05-12 09:36:15+00:00",
            "title": "A Frobenius Algebraic Analysis for Parasitic Gaps",
            "authors": [
                "Michael Moortgat",
                "Mehrnoosh Sadrzadeh",
                "Gijs Wijnholds"
            ],
            "abstract": "The interpretation of parasitic gaps is an ostensible case of non-linearity\nin natural language composition. Existing categorial analyses, both in the\ntypelogical and in the combinatory traditions, rely on explicit forms of\nsyntactic copying. We identify two types of parasitic gapping where the\nduplication of semantic content can be confined to the lexicon. Parasitic gaps\nin adjuncts are analysed as forms of generalized coordination with a\npolymorphic type schema for the head of the adjunct phrase. For parasitic gaps\naffecting arguments of the same predicate, the polymorphism is associated with\nthe lexical item that introduces the primary gap. Our analysis is formulated in\nterms of Lambek calculus extended with structural control modalities. A\ncompositional translation relates syntactic types and derivations to the\ninterpreting compact closed category of finite dimensional vector spaces and\nlinear maps with Frobenius algebras over it. When interpreted over the\nnecessary semantic spaces, the Frobenius algebras provide the tools to model\nthe proposed instances of lexical polymorphism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05683v1",
            "date": "2020-05-12 11:01:44+00:00",
            "title": "On the Robustness of Language Encoders against Grammatical Errors",
            "authors": [
                "Fan Yin",
                "Quanyu Long",
                "Tao Meng",
                "Kai-Wei Chang"
            ],
            "abstract": "We conduct a thorough study to diagnose the behaviors of pre-trained language\nencoders (ELMo, BERT, and RoBERTa) when confronted with natural grammatical\nerrors. Specifically, we collect real grammatical errors from non-native\nspeakers and conduct adversarial attacks to simulate these errors on clean text\ndata. We use this approach to facilitate debugging models on downstream\napplications. Results confirm that the performance of all tested models is\naffected but the degree of impact varies. To interpret model behaviors, we\nfurther design a linguistic acceptability task to reveal their abilities in\nidentifying ungrammatical sentences and the position of errors. We find that\nfixed contextual encoders with a simple classifier trained on the prediction of\nsentence correctness are able to locate error positions. We also design a cloze\ntest for BERT and discover that BERT captures the interaction between errors\nand specific tokens in context. Our results shed light on understanding the\nrobustness and behaviors of language encoders against grammatical errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05692v1",
            "date": "2020-05-12 11:25:07+00:00",
            "title": "Detecting Multiword Expression Type Helps Lexical Complexity Assessment",
            "authors": [
                "Ekaterina Kochmar",
                "Sian Gooding",
                "Matthew Shardlow"
            ],
            "abstract": "Multiword expressions (MWEs) represent lexemes that should be treated as\nsingle lexical units due to their idiosyncratic nature. Multiple NLP\napplications have been shown to benefit from MWE identification, however the\nresearch on lexical complexity of MWEs is still an under-explored area. In this\nwork, we re-annotate the Complex Word Identification Shared Task 2018 dataset\nof Yimam et al. (2017), which provides complexity scores for a range of\nlexemes, with the types of MWEs. We release the MWE-annotated dataset with this\npaper, and we believe this dataset represents a valuable resource for the text\nsimplification community. In addition, we investigate which types of\nexpressions are most problematic for native and non-native readers. Finally, we\nshow that a lexical complexity assessment system benefits from the information\nabout MWE types.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05738v1",
            "date": "2020-05-12 13:09:29+00:00",
            "title": "Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019",
            "authors": [
                "Antonio Toral"
            ],
            "abstract": "We reassess the claims of human parity and super-human performance made at\nthe news shared task of WMT 2019 for three translation directions:\nEnglish-to-German, English-to-Russian and German-to-English. First we identify\nthree potential issues in the human evaluation of that shared task: (i) the\nlimited amount of intersentential context available, (ii) the limited\ntranslation proficiency of the evaluators and (iii) the use of a reference\ntranslation. We then conduct a modified evaluation taking these issues into\naccount. Our results indicate that all the claims of human parity and\nsuper-human performance made at WMT 2019 should be refuted, except the claim of\nhuman parity for English-to-German. Based on our findings, we put forward a set\nof recommendations and open questions for future assessments of human parity in\nmachine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05806v2",
            "date": "2020-05-12 14:20:09+00:00",
            "title": "Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension",
            "authors": [
                "Bo Zheng",
                "Haoyang Wen",
                "Yaobo Liang",
                "Nan Duan",
                "Wanxiang Che",
                "Daxin Jiang",
                "Ming Zhou",
                "Ting Liu"
            ],
            "abstract": "Natural Questions is a new challenging machine reading comprehension\nbenchmark with two-grained answers, which are a long answer (typically a\nparagraph) and a short answer (one or more entities inside the long answer).\nDespite the effectiveness of existing methods on this benchmark, they treat\nthese two sub-tasks individually during training while ignoring their\ndependencies. To address this issue, we present a novel multi-grained machine\nreading comprehension framework that focuses on modeling documents at their\nhierarchical nature, which are different levels of granularity: documents,\nparagraphs, sentences, and tokens. We utilize graph attention networks to\nobtain different levels of representations so that they can be learned\nsimultaneously. The long and short answers can be extracted from\nparagraph-level representation and token-level representation, respectively. In\nthis way, we can model the dependencies between the two-grained answers to\nprovide evidence for each other. We jointly train the two sub-tasks, and our\nexperiments show that our approach significantly outperforms previous systems\nat both long and short answer criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05864v1",
            "date": "2020-05-12 15:35:00+00:00",
            "title": "Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach",
            "authors": [
                "Wenyu Du",
                "Zhouhan Lin",
                "Yikang Shen",
                "Timothy J. O'Donnell",
                "Yoshua Bengio",
                "Yue Zhang"
            ],
            "abstract": "It is commonly believed that knowledge of syntactic structure should improve\nlanguage modeling. However, effectively and computationally efficiently\nincorporating syntactic structure into neural language models has been a\nchallenging topic. In this paper, we make use of a multi-task objective, i.e.,\nthe models simultaneously predict words as well as ground truth parse trees in\na form called \"syntactic distances\", where information between these two\nseparate objectives shares the same intermediate representation. Experimental\nresults on the Penn Treebank and Chinese Treebank datasets show that when\nground truth parse trees are provided as additional training signals, the model\nis able to achieve lower perplexity and induce trees with better quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06035v1",
            "date": "2020-05-12 20:17:25+00:00",
            "title": "Cross-Modality Relevance for Reasoning on Language and Vision",
            "authors": [
                "Chen Zheng",
                "Quan Guo",
                "Parisa Kordjamshidi"
            ],
            "abstract": "This work deals with the challenge of learning and reasoning over language\nand vision data for the related downstream tasks such as visual question\nanswering (VQA) and natural language for visual reasoning (NLVR). We design a\nnovel cross-modality relevance module that is used in an end-to-end framework\nto learn the relevance representation between components of various input\nmodalities under the supervision of a target task, which is more generalizable\nto unobserved data compared to merely reshaping the original representation\nspace. In addition to modeling the relevance between the textual entities and\nvisual entities, we model the higher-order relevance between entity relations\nin the text and object relations in the image. Our proposed approach shows\ncompetitive performance on two different language and vision tasks using public\nbenchmarks and improves the state-of-the-art published results. The learned\nalignments of input spaces and their relevance representations by NLVR task\nboost the training efficiency of VQA task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06114v1",
            "date": "2020-05-13 01:56:00+00:00",
            "title": "Large Scale Multi-Actor Generative Dialog Modeling",
            "authors": [
                "Alex Boyd",
                "Raul Puri",
                "Mohammad Shoeybi",
                "Mostofa Patwary",
                "Bryan Catanzaro"
            ],
            "abstract": "Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying and\nengaging conversations with a user; however, they typically exhibit either\ninconsistent personality across conversations or the average personality of all\nusers. This paper addresses these issues by controlling an agent's persona upon\ngeneration via conditioning on prior conversations of a target actor. In doing\nso, we are able to utilize more abstract patterns within a person's speech and\nbetter emulate them in generated responses. This work introduces the Generative\nConversation Control model, an augmented and fine-tuned GPT-2 language model\nthat conditions on past reference conversations to probabilistically model\nmulti-turn conversations in the actor's persona. We introduce an accompanying\ndata collection procedure to obtain 10.3M conversations from 6 months worth of\nReddit comments. We demonstrate that scaling model sizes from 117M to 8.3B\nparameters yields an improvement from 23.14 to 13.14 perplexity on 1.7M held\nout Reddit conversations. Increasing model scale yielded similar improvements\nin human evaluations that measure preference of model samples to the held out\ntarget distribution in terms of realism (31% increased to 37% preference),\nstyle matching (37% to 42%), grammar and content quality (29% to 42%), and\nconversation coherency (32% to 40%). We find that conditionally modeling past\nconversations improves perplexity by 0.47 in automatic evaluations. Through\nhuman trials we identify positive trends between conditional modeling and style\nmatching and outline steps to further improve persona control.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06123v1",
            "date": "2020-05-13 02:39:56+00:00",
            "title": "Screenplay Quality Assessment: Can We Predict Who Gets Nominated?",
            "authors": [
                "Ming-Chang Chiu",
                "Tiantian Feng",
                "Xiang Ren",
                "Shrikanth Narayanan"
            ],
            "abstract": "Deciding which scripts to turn into movies is a costly and time-consuming\nprocess for filmmakers. Thus, building a tool to aid script selection, an\ninitial phase in movie production, can be very beneficial. Toward that goal, in\nthis work, we present a method to evaluate the quality of a screenplay based on\nlinguistic cues. We address this in a two-fold approach: (1) we define the task\nas predicting nominations of scripts at major film awards with the hypothesis\nthat the peer-recognized scripts should have a greater chance to succeed. (2)\nbased on industry opinions and narratology, we extract and integrate\ndomain-specific features into common classification techniques. We face two\nchallenges (1) scripts are much longer than other document datasets (2)\nnominated scripts are limited and thus difficult to collect. However, with\nnarratology-inspired modeling and domain features, our approach offers clear\nimprovements over strong baselines. Our work provides a new approach for future\nwork in screenplay analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06128v1",
            "date": "2020-05-13 03:09:58+00:00",
            "title": "Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation",
            "authors": [
                "Zhiliang Tian",
                "Wei Bi",
                "Dongkyu Lee",
                "Lanqing Xue",
                "Yiping Song",
                "Xiaojiang Liu",
                "Nevin L. Zhang"
            ],
            "abstract": "Neural conversation models are known to generate appropriate but\nnon-informative responses in general. A scenario where informativeness can be\nsignificantly enhanced is Conversing by Reading (CbR), where conversations take\nplace with respect to a given external document. In previous work, the external\ndocument is utilized by (1) creating a context-aware document memory that\nintegrates information from the document and the conversational context, and\nthen (2) generating responses referring to the memory. In this paper, we\npropose to create the document memory with some anticipated responses in mind.\nThis is achieved using a teacher-student framework. The teacher is given the\nexternal document, the context, and the ground-truth response, and learns how\nto build a response-aware document memory from three sources of information.\nThe student learns to construct a response-anticipated document memory from the\nfirst two sources, and the teacher's insight on memory creation. Empirical\nresults show that our model outperforms the previous state-of-the-art for the\nCbR task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06312v3",
            "date": "2020-05-13 13:36:09+00:00",
            "title": "Reasoning with Latent Structure Refinement for Document-Level Relation Extraction",
            "authors": [
                "Guoshun Nan",
                "Zhijiang Guo",
                "Ivan Sekuli\u0107",
                "Wei Lu"
            ],
            "abstract": "Document-level relation extraction requires integrating information within\nand across multiple sentences of a document and capturing complex interactions\nbetween inter-sentence entities. However, effective aggregation of relevant\ninformation in the document remains a challenging research question. Existing\napproaches construct static document-level graphs based on syntactic trees,\nco-references or heuristics from the unstructured text to model the\ndependencies. Unlike previous methods that may not be able to capture rich\nnon-local interactions for inference, we propose a novel model that empowers\nthe relational reasoning across sentences by automatically inducing the latent\ndocument-level graph. We further develop a refinement strategy, which enables\nthe model to incrementally aggregate relevant information for multi-hop\nreasoning. Specifically, our model achieves an F1 score of 59.05 on a\nlarge-scale document-level dataset (DocRED), significantly improving over the\nprevious results, and also yields new state-of-the-art results on the CDR and\nGDA dataset. Furthermore, extensive analyses show that the model is able to\ndiscover more accurate inter-sentence relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06383v1",
            "date": "2020-05-13 15:50:03+00:00",
            "title": "Sanskrit Segmentation Revisited",
            "authors": [
                "Sriram Krishnan",
                "Amba Kulkarni"
            ],
            "abstract": "Computationally analyzing Sanskrit texts requires proper segmentation in the\ninitial stages. There have been various tools developed for Sanskrit text\nsegmentation. Of these, G\\'erard Huet's Reader in the Sanskrit Heritage Engine\nanalyzes the input text and segments it based on the word parameters - phases\nlike iic, ifc, Pr, Subst, etc., and sandhi (or transition) that takes place at\nthe end of a word with the initial part of the next word. And it enlists all\nthe possible solutions differentiating them with the help of the phases. The\nphases and their analyses have their use in the domain of sentential parsers.\nIn segmentation, though, they are not used beyond deciding whether the words\nformed with the phases are morphologically valid. This paper tries to modify\nthe above segmenter by ignoring the phase details (except for a few cases), and\nalso proposes a probability function to prioritize the list of solutions to\nbring up the most valid solutions at the top.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06537v1",
            "date": "2020-05-13 19:05:58+00:00",
            "title": "A Mixture of $h-1$ Heads is Better than $h$ Heads",
            "authors": [
                "Hao Peng",
                "Roy Schwartz",
                "Dianqi Li",
                "Noah A. Smith"
            ],
            "abstract": "Multi-head attentive neural architectures have achieved state-of-the-art\nresults on a variety of natural language processing tasks. Evidence has shown\nthat they are overparameterized; attention heads can be pruned without\nsignificant performance loss. In this work, we instead \"reallocate\" them -- the\nmodel learns to activate different heads on different inputs. Drawing\nconnections between multi-head attention and mixture of experts, we propose the\nmixture of attentive experts model (MAE). MAE is trained using a block\ncoordinate descent algorithm that alternates between updating (1) the\nresponsibilities of the experts and (2) their parameters. Experiments on\nmachine translation and language modeling show that MAE outperforms strong\nbaselines on both tasks. Particularly, on the WMT14 English to German\ntranslation dataset, MAE improves over \"transformer-base\" by 0.8 BLEU, with a\ncomparable number of parameters. Our analysis shows that our model learns to\nspecialize different experts to different inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06545v1",
            "date": "2020-05-13 19:23:43+00:00",
            "title": "Validation and Normalization of DCS corpus using Sanskrit Heritage tools to build a tagged Gold Corpus",
            "authors": [
                "Sriram Krishnan",
                "Amba Kulkarni",
                "G\u00e9rard Huet"
            ],
            "abstract": "The Digital Corpus of Sanskrit records around 650,000 sentences along with\ntheir morphological and lexical tagging. But inconsistencies in morphological\nanalysis, and in providing crucial information like the segmented word, urges\nthe need for standardization and validation of this corpus. Automating the\nvalidation process requires efficient analyzers which also provide the missing\ninformation. The Sanskrit Heritage Engine's Reader produces all possible\nsegmentations with morphological and lexical analyses. Aligning these systems\nwould help us in recording the linguistic differences, which can be used to\nupdate these systems to produce standardized results and will also provide a\nGold corpus tagged with complete morphological and lexical information along\nwith the segmented words. Krishna et al. (2017) aligned 115,000 sentences,\nconsidering some of the linguistic differences. As both these systems have\nevolved significantly, the alignment is done again considering all the\nremaining linguistic differences between these systems. This paper describes\nthe modified alignment process in detail and records the additional linguistic\ndifferences observed.\n  Reference: Amrith Krishna, Pavankumar Satuluri, and Pawan Goyal. 2017. A\ndataset for Sanskrit word segmentation. In Proceedings of the Joint SIGHUM\nWorkshop on Computational Linguistics for Cultural Heritage, Social Sciences,\nHumanities and Literature, page 105-114. Association for Computational\nLinguistics, August.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06557v2",
            "date": "2020-05-13 19:46:41+00:00",
            "title": "Arabic Dialect Identification in the Wild",
            "authors": [
                "Ahmed Abdelali",
                "Hamdy Mubarak",
                "Younes Samih",
                "Sabit Hassan",
                "Kareem Darwish"
            ],
            "abstract": "We present QADI, an automatically collected dataset of tweets belonging to a\nwide range of country-level Arabic dialects -covering 18 different countries in\nthe Middle East and North Africa region. Our method for building this dataset\nrelies on applying multiple filters to identify users who belong to different\ncountries based on their account descriptions and to eliminate tweets that are\neither written in Modern Standard Arabic or contain inappropriate language. The\nresultant dataset contains 540k tweets from 2,525 users who are evenly\ndistributed across 18 Arab countries. Using intrinsic evaluation, we show that\nthe labels of a set of randomly selected tweets are 91.5% accurate. For\nextrinsic evaluation, we are able to build effective country-level dialect\nidentification on tweets with a macro-averaged F1-score of 60.6% across 18\nclasses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06579v1",
            "date": "2020-05-13 20:42:17+00:00",
            "title": "Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding",
            "authors": [
                "Xinya Du",
                "Claire Cardie"
            ],
            "abstract": "Few works in the literature of event extraction have gone beyond individual\nsentences to make extraction decisions. This is problematic when the\ninformation needed to recognize an event argument is spread across multiple\nsentences. We argue that document-level event extraction is a difficult task\nsince it requires a view of a larger context to determine which spans of text\ncorrespond to event role fillers. We first investigate how end-to-end neural\nsequence models (with pre-trained language model representations) perform on\ndocument-level role filler extraction, as well as how the length of context\ncaptured affects the models' performance. To dynamically aggregate information\ncaptured by neural representations learned at different levels of granularity\n(e.g., the sentence- and paragraph-level), we propose a novel multi-granularity\nreader. We evaluate our models on the MUC-4 event extraction dataset, and show\nthat our best system performs substantially better than prior work. We also\nreport findings on the relationship between context length and neural model\nperformance on the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06588v1",
            "date": "2020-05-13 21:06:59+00:00",
            "title": "PERLEX: A Bilingual Persian-English Gold Dataset for Relation Extraction",
            "authors": [
                "Majid Asgari-Bidhendi",
                "Mehrdad Nasser",
                "Behrooz Janfada",
                "Behrouz Minaei-Bidgoli"
            ],
            "abstract": "Relation extraction is the task of extracting semantic relations between\nentities in a sentence. It is an essential part of some natural language\nprocessing tasks such as information extraction, knowledge extraction, and\nknowledge base population. The main motivations of this research stem from a\nlack of a dataset for relation extraction in the Persian language as well as\nthe necessity of extracting knowledge from the growing big-data in the Persian\nlanguage for different applications. In this paper, we present \"PERLEX\" as the\nfirst Persian dataset for relation extraction, which is an expert-translated\nversion of the \"Semeval-2010-Task-8\" dataset. Moreover, this paper addresses\nPersian relation extraction utilizing state-of-the-art language-agnostic\nalgorithms. We employ six different models for relation extraction on the\nproposed bilingual dataset, including a non-neural model (as the baseline),\nthree neural models, and two deep learning models fed by multilingual-BERT\ncontextual word representations. The experiments result in the maximum f-score\n77.66% (provided by BERTEM-MTB method) as the state-of-the-art of relation\nextraction in the Persian language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06607v1",
            "date": "2020-05-03 06:25:16+00:00",
            "title": "Improving Aspect-Level Sentiment Analysis with Aspect Extraction",
            "authors": [
                "Navonil Majumder",
                "Rishabh Bhardwaj",
                "Soujanya Poria",
                "Amir Zadeh",
                "Alexander Gelbukh",
                "Amir Hussain",
                "Louis-Philippe Morency"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA), a popular research area in NLP has\ntwo distinct parts -- aspect extraction (AE) and labeling the aspects with\nsentiment polarity (ALSA). Although distinct, these two tasks are highly\ncorrelated. The work primarily hypothesize that transferring knowledge from a\npre-trained AE model can benefit the performance of ALSA models. Based on this\nhypothesis, word embeddings are obtained during AE and subsequently, feed that\nto the ALSA model. Empirically, this work show that the added information\nsignificantly improves the performance of three different baseline ALSA models\non two distinct domains. This improvement also translates well across domains\nbetween AE and ALSA tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06872v2",
            "date": "2020-05-14 11:14:10+00:00",
            "title": "A Category Theory Approach to Interoperability",
            "authors": [
                "Riccardo Del Gratta"
            ],
            "abstract": "In this article, we propose a Category Theory approach to (syntactic)\ninteroperability between linguistic tools. The resulting category consists of\ntextual documents, including any linguistic annotations, NLP tools that analyze\ntexts and add additional linguistic information, and format converters. Format\nconverters are necessary to make the tools both able to read and to produce\ndifferent output formats, which is the key to interoperability. The idea behind\nthis document is the parallelism between the concepts of composition and\nassociativity in Category Theory with the NLP pipelines. We show how pipelines\nof linguistic tools can be modeled into the conceptual framework of Category\nTheory and we successfully apply this method to two real-life examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06901v1",
            "date": "2020-05-14 12:09:23+00:00",
            "title": "DRTS Parsing with Structure-Aware Encoding and Decoding",
            "authors": [
                "Qiankun Fu",
                "Yue Zhang",
                "Jiangming Liu",
                "Meishan Zhang"
            ],
            "abstract": "Discourse representation tree structure (DRTS) parsing is a novel semantic\nparsing task which has been concerned most recently. State-of-the-art\nperformance can be achieved by a neural sequence-to-sequence model, treating\nthe tree construction as an incremental sequence generation problem. Structural\ninformation such as input syntax and the intermediate skeleton of the partial\noutput has been ignored in the model, which could be potentially useful for the\nDRTS parsing. In this work, we propose a structural-aware model at both the\nencoder and decoder phase to integrate the structural information, where graph\nattention network (GAT) is exploited for effectively modeling. Experimental\nresults on a benchmark dataset show that our proposed model is effective and\ncan obtain the best performance in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06943v2",
            "date": "2020-05-14 13:23:59+00:00",
            "title": "NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor",
            "authors": [
                "Steve Durairaj Swamy",
                "Shubham Laddha",
                "Basil Abdussalam",
                "Debayan Datta",
                "Anupam Jamatia"
            ],
            "abstract": "The paper describes the systems submitted to SemEval-2020 Task 8: Memotion by\nthe `NIT-Agartala-NLP-Team'. A dataset of 8879 memes was made available by the\ntask organizers to train and test our models. Our systems include a Logistic\nRegression baseline, a BiLSTM + Attention-based learner and a transfer learning\napproach with BERT. For the three sub-tasks A, B and C, we attained ranks\n24/33, 11/29 and 15/26, respectively. We highlight our difficulties in\nharnessing image information as well as some techniques and handcrafted\nfeatures we employ to overcome these issues. We also discuss various modelling\nissues and theorize possible solutions and reasons as to why these problems\npersist.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06946v1",
            "date": "2020-04-02 10:17:55+00:00",
            "title": "4chan & 8chan embeddings",
            "authors": [
                "Pierre Vou\u00e9",
                "Tom De Smedt",
                "Guy De Pauw"
            ],
            "abstract": "We have collected over 30M messages from the publicly available /pol/ message\nboards on 4chan and 8chan, and compiled them into a model of toxic language\nuse. The trained word embeddings (0.4GB) are released for free and may be\nuseful for further study on toxic discourse or to boost hate speech detection\nsystems: https://textgain.com/8chan.",
            "journal ref": "Textgain Technical Reports 1 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07150v3",
            "date": "2020-05-14 17:11:41+00:00",
            "title": "Named Entity Recognition as Dependency Parsing",
            "authors": [
                "Juntao Yu",
                "Bernd Bohnet",
                "Massimo Poesio"
            ],
            "abstract": "Named Entity Recognition (NER) is a fundamental task in Natural Language\nProcessing, concerned with identifying spans of text expressing references to\nentities. NER research is often focused on flat entities only (flat NER),\nignoring the fact that entity references can be nested, as in [Bank of [China]]\n(Finkel and Manning, 2009). In this paper, we use ideas from graph-based\ndependency parsing to provide our model a global view on the input via a\nbiaffine model (Dozat and Manning, 2017). The biaffine model scores pairs of\nstart and end tokens in a sentence which we use to explore all spans, so that\nthe model is able to predict named entities accurately. We show that the model\nworks well for both nested and flat NER through evaluation on 8 corpora and\nachieving SoTA performance on all of them, with accuracy gains of up to 2.2\npercentage points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07162v1",
            "date": "2020-05-14 17:30:06+00:00",
            "title": "NAT: Noise-Aware Training for Robust Neural Sequence Labeling",
            "authors": [
                "Marcin Namysl",
                "Sven Behnke",
                "Joachim K\u00f6hler"
            ],
            "abstract": "Sequence labeling systems should perform reliably not only under ideal\nconditions but also with corrupted inputs - as these systems often process\nuser-generated text or follow an error-prone upstream component. To this end,\nwe formulate the noisy sequence labeling problem, where the input may undergo\nan unknown noising process and propose two Noise-Aware Training (NAT)\nobjectives that improve robustness of sequence labeling performed on perturbed\ninput: Our data augmentation method trains a neural model using a mixture of\nclean and noisy samples, whereas our stability training algorithm encourages\nthe model to create a noise-invariant latent representation. We employ a\nvanilla noise model at training time. For evaluation, we use both the original\ndata and its variants perturbed with real OCR errors and misspellings.\nExtensive experiments on English and German named entity recognition benchmarks\nconfirmed that NAT consistently improved robustness of popular sequence\nlabeling models, preserving accuracy on the original input. We make our code\nand data publicly available for the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07202v3",
            "date": "2020-05-14 18:00:01+00:00",
            "title": "Pre-training technique to localize medical BERT and enhance biomedical BERT",
            "authors": [
                "Shoya Wada",
                "Toshihiro Takeda",
                "Shiro Manabe",
                "Shozo Konishi",
                "Jun Kamohara",
                "Yasushi Matsumura"
            ],
            "abstract": "Pre-training large-scale neural language models on raw texts has made a\nsignificant contribution to improving transfer learning in natural language\nprocessing (NLP). With the introduction of transformer-based language models,\nsuch as bidirectional encoder representations from transformers (BERT), the\nperformance of information extraction from a free text by NLP has significantly\nimproved for both the general domain and medical domain; however, it is\ndifficult to train specific BERT models that perform well for domains in which\nthere are few publicly available databases of high quality and large size. We\nhypothesized that this problem can be addressed by up-sampling a\ndomain-specific corpus and using it for pre-training with a larger corpus in a\nbalanced manner. Our proposed method consists of a single intervention with one\noption: simultaneous pre-training after up-sampling and amplified vocabulary.\nWe conducted three experiments and evaluated the resulting products. We\nconfirmed that our Japanese medical BERT outperformed conventional baselines\nand the other BERT models in terms of the medical document classification task\nand that our English BERT pre-trained using both the general and medical-domain\ncorpora performed sufficiently well for practical use in terms of the\nbiomedical language understanding evaluation (BLUE) benchmark. Moreover, our\nenhanced biomedical BERT model, in which clinical notes were not used during\npre-training, showed that both the clinical and biomedical scores of the BLUE\nbenchmark were 0.3 points above that of the ablation model trained without our\nproposed method. Well-balanced pre-training by up-sampling instances derived\nfrom a corpus appropriate for the target task allows us to construct a\nhigh-performance BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07287v2",
            "date": "2020-05-14 22:47:37+00:00",
            "title": "VirAAL: Virtual Adversarial Active Learning For NLU",
            "authors": [
                "Gregory Senay",
                "Badr Youbi Idrissi",
                "Marine Haziza"
            ],
            "abstract": "This paper presents VirAAL, an Active Learning framework based on Adversarial\nTraining. VirAAL aims to reduce the effort of annotation in Natural Language\nUnderstanding (NLU). VirAAL is based on Virtual Adversarial Training (VAT), a\nsemi-supervised approach that regularizes the model through Local\nDistributional Smoothness. With that, adversarial perturbations are added to\nthe inputs making the posterior distribution more consistent. Therefore,\nentropy-based Active Learning becomes robust by querying more informative\nsamples without requiring additional components. The first set of experiments\nstudies the impact of an adapted VAT for joint-NLU tasks within low labeled\ndata regimes. The second set shows the effect of VirAAL in an Active Learning\n(AL) process. Results demonstrate that VAT is robust even on multi-task\ntraining, where the adversarial noise is computed from multiple loss functions.\nSubstantial improvements are observed with entropy-based AL with VirAAL for\nquerying data to annotate. VirAAL is an inexpensive method in terms of AL\ncomputation with a positive impact on data sampling. Furthermore, VirAAL\ndecreases annotations in AL up to 80% and shows improvements over existing data\naugmentation methods. The code is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07297v1",
            "date": "2020-05-14 23:46:10+00:00",
            "title": "OSACT4 Shared Task on Offensive Language Detection: Intensive Preprocessing-Based Approach",
            "authors": [
                "Fatemah Husain"
            ],
            "abstract": "The preprocessing phase is one of the key phases within the text\nclassification pipeline. This study aims at investigating the impact of the\npreprocessing phase on text classification, specifically on offensive language\nand hate speech classification for Arabic text. The Arabic language used in\nsocial media is informal and written using Arabic dialects, which makes the\ntext classification task very complex. Preprocessing helps in dimensionality\nreduction and removing useless content. We apply intensive preprocessing\ntechniques to the dataset before processing it further and feeding it into the\nclassification model. An intensive preprocessing-based approach demonstrates\nits significant impact on offensive language detection and hate speech\ndetection shared tasks of the fourth workshop on Open-Source Arabic Corpora and\nCorpora Processing Tools (OSACT). Our team wins the third place (3rd) in the\nSub-Task A Offensive Language Detection division and wins the first place (1st)\nin the Sub-Task B Hate Speech Detection division, with an F1 score of 89% and\n95%, respectively, by providing the state-of-the-art performance in terms of\nF1, accuracy, recall, and precision for Arabic hate speech detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07355v1",
            "date": "2020-05-15 04:36:06+00:00",
            "title": "A chatbot architecture for promoting youth resilience",
            "authors": [
                "Chester Holt-Quick",
                "Jim Warren",
                "Karolina Stasiak",
                "Ruth Williams",
                "Grant Christie",
                "Sarah Hetrick",
                "Sarah Hopkins",
                "Tania Cargo",
                "Sally Merry"
            ],
            "abstract": "E-health technologies have the potential to provide scalable and accessible\ninterventions for youth mental health. As part of a developing an ecosystem of\ne-screening and e-therapy tools for New Zealand young people, a dialog agent,\nHeadstrong, has been designed to promote resilience with methods grounded in\ncognitive behavioral therapy and positive psychology. This paper describes the\narchitecture underlying the chatbot. The architecture supports a range of over\n20 activities delivered in a 4-week program by relatable personas. The\narchitecture provides a visual authoring interface to its content management\nsystem. In addition to supporting the original adolescent resilience chatbot,\nthe architecture has been reused to create a 3-week 'stress-detox' intervention\nfor undergraduates, and subsequently for a chatbot to support young people with\nthe impacts of the COVID-19 pandemic, with all three systems having been used\nin field trials. The Headstrong architecture illustrates the feasibility of\ncreating a domain-focused authoring environment in the context of e-therapy\nthat supports non-technical expert input and rapid deployment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07505v2",
            "date": "2020-05-15 12:47:54+00:00",
            "title": "Corpus and Models for Lemmatisation and POS-tagging of Classical French Theatre",
            "authors": [
                "Jean-Baptiste Camps",
                "Simon Gabay",
                "Paul Fi\u00e8vre",
                "Thibault Cl\u00e9rice",
                "Florian Cafiero"
            ],
            "abstract": "This paper describes the process of building an annotated corpus and training\nmodels for classical French literature, with a focus on theatre, and\nparticularly comedies in verse. It was originally developed as a preliminary\nstep to the stylometric analyses presented in Cafiero and Camps [2019]. The use\nof a recent lemmatiser based on neural networks and a CRF tagger allows to\nachieve accuracies beyond the current state-of-the art on the in-domain test,\nand proves to be robust during out-of-domain tests, i.e.up to 20th c.novels.",
            "journal ref": "Journal of Data Mining & Digital Humanities, 2021, Digital\n  humanities in languages (February 14, 2021) jdmdh:7161",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07522v1",
            "date": "2020-05-14 04:05:29+00:00",
            "title": "Parallel Data Augmentation for Formality Style Transfer",
            "authors": [
                "Yi Zhang",
                "Tao Ge",
                "Xu Sun"
            ],
            "abstract": "The main barrier to progress in the task of Formality Style Transfer is the\ninadequacy of training data. In this paper, we study how to augment parallel\ndata and propose novel and simple data augmentation methods for this task to\nobtain useful sentence pairs with easily accessible models and systems.\nExperiments demonstrate that our augmented parallel data largely helps improve\nformality style transfer when it is used to pre-train the model, leading to the\nstate-of-the-art results in the GYAFC benchmark dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07617v1",
            "date": "2020-05-15 16:11:33+00:00",
            "title": "Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline",
            "authors": [
                "David Helbig",
                "Enrica Troiano",
                "Roman Klinger"
            ],
            "abstract": "We propose the task of emotion style transfer, which is particularly\nchallenging, as emotions (here: anger, disgust, fear, joy, sadness, surprise)\nare on the fence between content and style. To understand the particular\ndifficulties of this task, we design a transparent emotion style transfer\npipeline based on three steps: (1) select the words that are promising to be\nsubstituted to change the emotion (with a brute-force approach and selection\nbased on the attention mechanism of an emotion classifier), (2) find sets of\nwords as candidates for substituting the words (based on lexical and\ndistributional semantics), and (3) select the most promising combination of\nsubstitutions with an objective function which consists of components for\ncontent (based on BERT sentence embeddings), emotion (based on an emotion\nclassifier), and fluency (based on a neural language model). This comparably\nstraight-forward setup enables us to explore the task and understand in what\ncases lexical substitution can vary the emotional load of texts, how changes in\ncontent and style interact and if they are at odds. We further evaluate our\npipeline quantitatively in an automated and an annotation study based on Tweets\nand find, indeed, that simultaneous adjustments of content and emotion are\nconflicting objectives: as we show in a qualitative analysis motivated by\nScherer's emotion component model, this is particularly the case for implicit\nemotion expressions based on cognitive appraisal or descriptions of bodily\nreactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07734v1",
            "date": "2020-05-15 18:37:56+00:00",
            "title": "Uncovering Gender Bias in Media Coverage of Politicians with Machine Learning",
            "authors": [
                "Susan Leavy"
            ],
            "abstract": "This paper presents research uncovering systematic gender bias in the\nrepresentation of political leaders in the media, using artificial\nintelligence. Newspaper coverage of Irish ministers over a fifteen year period\nwas gathered and analysed with natural language processing techniques and\nmachine learning. Findings demonstrate evidence of gender bias in the portrayal\nof female politicians, the kind of policies they were associated with and how\nthey were evaluated in terms of their performance as political leaders. This\npaper also sets out a methodology whereby media content may be analysed on a\nlarge scale utilising techniques from artificial intelligence within a\ntheoretical framework founded in gender theory and feminist linguistics.",
            "journal ref": "Digital Scholarship in the Humanities 34.1 (2019): 48-63",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07751v2",
            "date": "2020-05-15 19:18:26+00:00",
            "title": "In Layman's Terms: Semi-Open Relation Extraction from Scientific Texts",
            "authors": [
                "Ruben Kruiper",
                "Julian F. V. Vincent",
                "Jessica Chen-Burger",
                "Marc P. Y. Desmulliez",
                "Ioannis Konstas"
            ],
            "abstract": "Information Extraction (IE) from scientific texts can be used to guide\nreaders to the central information in scientific documents. But narrow IE\nsystems extract only a fraction of the information captured, and Open IE\nsystems do not perform well on the long and complex sentences encountered in\nscientific texts. In this work we combine the output of both types of systems\nto achieve Semi-Open Relation Extraction, a new task that we explore in the\nBiology domain. First, we present the Focused Open Biological Information\nExtraction (FOBIE) dataset and use FOBIE to train a state-of-the-art narrow\nscientific IE system to extract trade-off relations and arguments that are\ncentral to biology texts. We then run both the narrow IE system and a\nstate-of-the-art Open IE system on a corpus of 10k open-access scientific\nbiological texts. We show that a significant amount (65%) of erroneous and\nuninformative Open IE extractions can be filtered using narrow IE extractions.\nFurthermore, we show that the retained extractions are significantly more often\ninformative to a reader.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07753v2",
            "date": "2020-05-15 19:25:12+00:00",
            "title": "A Scientific Information Extraction Dataset for Nature Inspired Engineering",
            "authors": [
                "Ruben Kruiper",
                "Julian F. V. Vincent",
                "Jessica Chen-Burger",
                "Marc P. Y. Desmulliez",
                "Ioannis Konstas"
            ],
            "abstract": "Nature has inspired various ground-breaking technological developments in\napplications ranging from robotics to aerospace engineering and the\nmanufacturing of medical devices. However, accessing the information captured\nin scientific biology texts is a time-consuming and hard task that requires\ndomain-specific knowledge. Improving access for outsiders can help\ninterdisciplinary research like Nature Inspired Engineering. This paper\ndescribes a dataset of 1,500 manually-annotated sentences that express\ndomain-independent relations between central concepts in a scientific biology\ntext, such as trade-offs and correlations. The arguments of these relations can\nbe Multi Word Expressions and have been annotated with modifying phrases to\nform non-projective graphs. The dataset allows for training and evaluating\nRelation Extraction algorithms that aim for coarse-grained typing of scientific\nbiological documents, enabling a high-level filter for engineers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07886v1",
            "date": "2020-05-16 06:29:14+00:00",
            "title": "Integrating Semantic and Structural Information with Graph Convolutional Network for Controversy Detection",
            "authors": [
                "Lei Zhong",
                "Juan Cao",
                "Qiang Sheng",
                "Junbo Guo",
                "Ziang Wang"
            ],
            "abstract": "Identifying controversial posts on social media is a fundamental task for\nmining public sentiment, assessing the influence of events, and alleviating the\npolarized views. However, existing methods fail to 1) effectively incorporate\nthe semantic information from content-related posts; 2) preserve the structural\ninformation for reply relationship modeling; 3) properly handle posts from\ntopics dissimilar to those in the training set. To overcome the first two\nlimitations, we propose Topic-Post-Comment Graph Convolutional Network\n(TPC-GCN), which integrates the information from the graph structure and\ncontent of topics, posts, and comments for post-level controversy detection. As\nto the third limitation, we extend our model to Disentangled TPC-GCN\n(DTPC-GCN), to disentangle topic-related and topic-unrelated features and then\nfuse dynamically. Extensive experiments on two real-world datasets demonstrate\nthat our models outperform existing methods. Analysis of the results and cases\nproves that our models can integrate both semantic and structural information\nwith significant generalizability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07934v2",
            "date": "2020-05-16 10:15:49+00:00",
            "title": "ApplicaAI at SemEval-2020 Task 11: On RoBERTa-CRF, Span CLS and Whether Self-Training Helps Them",
            "authors": [
                "Dawid Jurkiewicz",
                "\u0141ukasz Borchmann",
                "Izabela Kosmala",
                "Filip Grali\u0144ski"
            ],
            "abstract": "This paper presents the winning system for the propaganda Technique\nClassification (TC) task and the second-placed system for the propaganda Span\nIdentification (SI) task. The purpose of TC task was to identify an applied\npropaganda technique given propaganda text fragment. The goal of SI task was to\nfind specific text fragments which contain at least one propaganda technique.\nBoth of the developed solutions used semi-supervised learning technique of\nself-training. Interestingly, although CRF is barely used with\ntransformer-based language models, the SI task was approached with RoBERTa-CRF\narchitecture. An ensemble of RoBERTa-based models was proposed for the TC task,\nwith one of them making use of Span CLS layers we introduce in the present\npaper. In addition to describing the submitted systems, an impact of\narchitectural decisions and training schemes is investigated along with remarks\nregarding training models of the same or better quality with lower\ncomputational budget. Finally, the results of error analysis are presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07954v1",
            "date": "2020-05-16 11:11:48+00:00",
            "title": "Logical Inferences with Comparatives and Generalized Quantifiers",
            "authors": [
                "Izumi Haruta",
                "Koji Mineshima",
                "Daisuke Bekki"
            ],
            "abstract": "Comparative constructions pose a challenge in Natural Language Inference\n(NLI), which is the task of determining whether a text entails a hypothesis.\nComparatives are structurally complex in that they interact with other\nlinguistic phenomena such as quantifiers, numerals, and lexical antonyms. In\nformal semantics, there is a rich body of work on comparatives and gradable\nexpressions using the notion of degree. However, a logical inference system for\ncomparatives has not been sufficiently developed for use in the NLI task. In\nthis paper, we present a compositional semantics that maps various comparative\nconstructions in English to semantic representations via Combinatory Categorial\nGrammar (CCG) parsers and combine it with an inference system based on\nautomated theorem proving. We evaluate our system on three NLI datasets that\ncontain complex logical inferences with comparatives, generalized quantifiers,\nand numerals. We show that the system outperforms previous logic-based systems\nas well as recent deep learning-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07988v3",
            "date": "2020-05-16 13:28:17+00:00",
            "title": "A Text Reassembling Approach to Natural Language Generation",
            "authors": [
                "Xiao Li",
                "Kees van Deemter",
                "Chenghua Lin"
            ],
            "abstract": "Recent years have seen a number of proposals for performing Natural Language\nGeneration (NLG) based in large part on statistical techniques. Despite having\nmany attractive features, we argue that these existing approaches nonetheless\nhave some important drawbacks, sometimes because the approach in question is\nnot fully statistical (i.e., relies on a certain amount of handcrafting),\nsometimes because the approach in question lacks transparency. Focussing on\nsome of the key NLG tasks (namely Content Selection, Lexical Choice, and\nLinguistic Realisation), we propose a novel approach, called the Text\nReassembling approach to NLG (TRG), which approaches the ideal of a purely\nstatistical approach very closely, and which is at the same time highly\ntransparent. We evaluate the TRG approach and discuss how TRG may be extended\nto deal with other NLG tasks, such as Document Structuring, and Aggregation. We\ndiscuss the strengths and limitations of TRG, concluding that the method may\nhold particular promise for domain experts who want to build an NLG system\ndespite having little expertise in linguistics and NLG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08105v1",
            "date": "2020-05-16 21:10:28+00:00",
            "title": "Learning Probabilistic Sentence Representations from Paraphrases",
            "authors": [
                "Mingda Chen",
                "Kevin Gimpel"
            ],
            "abstract": "Probabilistic word embeddings have shown effectiveness in capturing notions\nof generality and entailment, but there is very little work on doing the\nanalogous type of investigation for sentences. In this paper we define\nprobabilistic models that produce distributions for sentences. Our\nbest-performing model treats each word as a linear transformation operator\napplied to a multivariate Gaussian distribution. We train our models on\nparaphrases and demonstrate that they naturally capture sentence specificity.\nWhile our proposed model achieves the best performance overall, we also show\nthat specificity is represented by simpler architectures via the norm of the\nsentence vectors. Qualitative analysis shows that our probabilistic model\ncaptures sentential entailment and provides ways to analyze the specificity and\npreciseness of individual words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08113v1",
            "date": "2020-05-16 21:53:31+00:00",
            "title": "RPD: A Distance Function Between Word Embeddings",
            "authors": [
                "Xuhui Zhou",
                "Zaixiang Zheng",
                "Shujian Huang"
            ],
            "abstract": "It is well-understood that different algorithms, training processes, and\ncorpora produce different word embeddings. However, less is known about the\nrelation between different embedding spaces, i.e. how far different sets of\nembeddings deviate from each other. In this paper, we propose a novel metric\ncalled Relative pairwise inner Product Distance (RPD) to quantify the distance\nbetween different sets of word embeddings. This metric has a unified scale for\ncomparing different sets of word embeddings. Based on the properties of RPD, we\nstudy the relations of word embeddings of different algorithms systematically\nand investigate the influence of different training processes and corpora. The\nresults shed light on the poorly understood word embeddings and justify RPD as\na measure of the distance of embedding spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08156v1",
            "date": "2020-05-17 03:28:12+00:00",
            "title": "Adversarial Training for Commonsense Inference",
            "authors": [
                "Lis Pereira",
                "Xiaodong Liu",
                "Fei Cheng",
                "Masayuki Asahara",
                "Ichiro Kobayashi"
            ],
            "abstract": "We propose an AdversariaL training algorithm for commonsense InferenCE\n(ALICE). We apply small perturbations to word embeddings and minimize the\nresultant adversarial risk to regularize the model. We exploit a novel\ncombination of two different approaches to estimate these perturbations: 1)\nusing the true label and 2) using the model prediction. Without relying on any\nhuman-crafted features, knowledge bases, or additional datasets other than the\ntarget datasets, our model boosts the fine-tuning performance of RoBERTa,\nachieving competitive results on multiple reading comprehension datasets that\nrequire commonsense inference.",
            "journal ref": "ACL2020 RepL4NLP workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08178v1",
            "date": "2020-05-17 07:04:08+00:00",
            "title": "IMoJIE: Iterative Memory-Based Joint Open Information Extraction",
            "authors": [
                "Keshav Kolluru",
                "Samarth Aggarwal",
                "Vipul Rathore",
                "Mausam",
                "Soumen Chakrabarti"
            ],
            "abstract": "While traditional systems for Open Information Extraction were statistical\nand rule-based, recently neural models have been introduced for the task. Our\nwork builds upon CopyAttention, a sequence generation OpenIE model (Cui et.\nal., 2018). Our analysis reveals that CopyAttention produces a constant number\nof extractions per sentence, and its extracted tuples often express redundant\ninformation.\n  We present IMoJIE, an extension to CopyAttention, which produces the next\nextraction conditioned on all previously extracted tuples. This approach\novercomes both shortcomings of CopyAttention, resulting in a variable number of\ndiverse extractions per sentence. We train IMoJIE on training data bootstrapped\nfrom extractions of several non-neural systems, which have been automatically\nfiltered to reduce redundancy and noise. IMoJIE outperforms CopyAttention by\nabout 18 F1 pts, and a BERT-based strong baseline by 2 F1 pts, establishing a\nnew state of the art for the task.",
            "journal ref": "ACL 2020, Long paper",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08188v1",
            "date": "2020-05-17 08:10:51+00:00",
            "title": "Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce",
            "authors": [
                "Juntao Li",
                "Chang Liu",
                "Jian Wang",
                "Lidong Bing",
                "Hongsong Li",
                "Xiaozhong Liu",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "With the prosperous of cross-border e-commerce, there is an urgent demand for\ndesigning intelligent approaches for assisting e-commerce sellers to offer\nlocal products for consumers from all over the world. In this paper, we explore\na new task of cross-lingual information retrieval, i.e., cross-lingual\nset-to-description retrieval in cross-border e-commerce, which involves\nmatching product attribute sets in the source language with persuasive product\ndescriptions in the target language. We manually collect a new and high-quality\npaired dataset, where each pair contains an unordered product attribute set in\nthe source language and an informative product description in the target\nlanguage. As the dataset construction process is both time-consuming and\ncostly, the new dataset only comprises of 13.5k pairs, which is a low-resource\nsetting and can be viewed as a challenging testbed for model development and\nevaluation in cross-border e-commerce. To tackle this cross-lingual\nset-to-description retrieval task, we propose a novel cross-lingual matching\nnetwork (CLMN) with the enhancement of context-dependent cross-lingual mapping\nupon the pre-trained monolingual BERT representations. Experimental results\nindicate that our proposed CLMN yields impressive results on the challenging\ntask and the context-dependent cross-lingual mapping on BERT yields noticeable\nimprovement over the pre-trained multi-lingual BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08206v1",
            "date": "2020-05-17 10:03:42+00:00",
            "title": "Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles",
            "authors": [
                "Ben Eyal",
                "Michael Elhadad"
            ],
            "abstract": "We present a semantic role labeling resource for Hebrew built\nsemi-automatically through annotation projection from English. This corpus is\nderived from the multilingual OpenSubtitles dataset and includes short informal\nsentences, for which reliable linguistic annotations have been computed. We\nprovide a fully annotated version of the data including morphological analysis,\ndependency syntax and semantic role labeling in both FrameNet and PropBank\nstyles. Sentences are aligned between English and Hebrew, both sides include\nfull annotations and the explicit mapping from the English arguments to the\nHebrew ones. We train a neural SRL model on this Hebrew resource exploiting the\npre-trained multilingual BERT transformer model, and provide the first\navailable baseline model for Hebrew SRL as a reference point. The code we\nprovide is generic and can be adapted to other languages to bootstrap SRL\nresources.",
            "journal ref": "Proceedings of The 12th Language Resources and Evaluation\n  Conference (2020) 5936-5944",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08294v1",
            "date": "2020-05-17 16:50:28+00:00",
            "title": "Support-BERT: Predicting Quality of Question-Answer Pairs in MSDN using Deep Bidirectional Transformer",
            "authors": [
                "Bhaskar Sen",
                "Nikhil Gopal",
                "Xinwei Xue"
            ],
            "abstract": "Quality of questions and answers from community support websites (e.g.\nMicrosoft Developers Network, Stackoverflow, Github, etc.) is difficult to\ndefine and a prediction model of quality questions and answers is even more\nchallenging to implement. Previous works have addressed the question quality\nmodels and answer quality models separately using meta-features like number of\nup-votes, trustworthiness of the person posting the questions or answers,\ntitles of the post, and context naive natural language processing features.\nHowever, there is a lack of an integrated question-answer quality model for\ncommunity question answering websites in the literature. In this brief paper,\nwe tackle the quality Q&A modeling problems from the community support websites\nusing a recently developed deep learning model using bidirectional\ntransformers. We investigate the applicability of transfer learning on Q&A\nquality modeling using Bidirectional Encoder Representations from Transformers\n(BERT) trained on a separate tasks originally using Wikipedia. It is found that\na further pre-training of BERT model along with finetuning on the Q&As\nextracted from Microsoft Developer Network (MSDN) can boost the performance of\nautomated quality prediction to more than 80%. Furthermore, the implementations\nare carried out for deploying the finetuned model in real-time scenario using\nAzureML in Azure knowledge base system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08319v2",
            "date": "2020-05-17 17:49:53+00:00",
            "title": "Context-Based Quotation Recommendation",
            "authors": [
                "Ansel MacLaughlin",
                "Tao Chen",
                "Burcu Karagol Ayan",
                "Dan Roth"
            ],
            "abstract": "While composing a new document, anything from a news article to an email or\nessay, authors often utilize direct quotes from a variety of sources. Although\nan author may know what point they would like to make, selecting an appropriate\nquote for the specific context may be time-consuming and difficult. We\ntherefore propose a novel context-aware quote recommendation system which\nutilizes the content an author has already written to generate a ranked list of\nquotable paragraphs and spans of tokens from a given source document.\n  We approach quote recommendation as a variant of open-domain question\nanswering and adapt the state-of-the-art BERT-based methods from open-QA to our\ntask. We conduct experiments on a collection of speech transcripts and\nassociated news articles, evaluating models' paragraph ranking and span\nprediction performances. Our experiments confirm the strong performance of\nBERT-based methods on this task, which outperform bag-of-words and neural\nranking baselines by more than 30% relative across all ranking metrics.\nQualitative analyses show the difficulty of the paragraph and span\nrecommendation tasks and confirm the quotability of the best BERT model's\npredictions, even if they are not the true selected quotes from the original\nnews articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08340v1",
            "date": "2020-05-17 18:57:23+00:00",
            "title": "Cross-Lingual Word Embeddings for Turkic Languages",
            "authors": [
                "Elmurod Kuriyozov",
                "Yerai Doval",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "There has been an increasing interest in learning cross-lingual word\nembeddings to transfer knowledge obtained from a resource-rich language, such\nas English, to lower-resource languages for which annotated data is scarce,\nsuch as Turkish, Russian, and many others. In this paper, we present the first\nviability study of established techniques to align monolingual embedding spaces\nfor Turkish, Uzbek, Azeri, Kazakh and Kyrgyz, members of the Turkic family\nwhich is heavily affected by the low-resource constraint. Those techniques are\nknown to require little explicit supervision, mainly in the form of bilingual\ndictionaries, hence being easily adaptable to different domains, including\nlow-resource ones. We obtain new bilingual dictionaries and new word embeddings\nfor these languages and show the steps for obtaining cross-lingual word\nembeddings using state-of-the-art techniques. Then, we evaluate the results\nusing the bilingual dictionary induction task. Our experiments confirm that the\nobtained bilingual dictionaries outperform previously-available ones, and that\nword embeddings from a low-resource language can benefit from resource-rich\nclosely-related languages when they are aligned together. Furthermore,\nevaluation on an extrinsic task (Sentiment analysis on Uzbek) proves that\nmonolingual word embeddings can, although slightly, benefit from cross-lingual\nalignments.",
            "journal ref": "Proceedings of The 12th Language Resources and Evaluation\n  Conference, Marseille, France, 2020, pp. 4047-4055",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08365v2",
            "date": "2020-05-17 20:29:27+00:00",
            "title": "MixingBoard: a Knowledgeable Stylized Integrated Text Generation Platform",
            "authors": [
                "Xiang Gao",
                "Michel Galley",
                "Bill Dolan"
            ],
            "abstract": "We present MixingBoard, a platform for quickly building demos with a focus on\nknowledge grounded stylized text generation. We unify existing text generation\nalgorithms in a shared codebase and further adapt earlier algorithms for\nconstrained generation. To borrow advantages from different models, we\nimplement strategies for cross-model integration, from the token probability\nlevel to the latent space level. An interface to external knowledge is provided\nvia a module that retrieves on-the-fly relevant knowledge from passages on the\nweb or any document collection. A user interface for local development, remote\nwebpage access, and a RESTful API are provided to make it simple for users to\nbuild their own demos.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08417v1",
            "date": "2020-05-18 01:31:28+00:00",
            "title": "Syntax-guided Controlled Generation of Paraphrases",
            "authors": [
                "Ashutosh Kumar",
                "Kabir Ahuja",
                "Raghuram Vadapalli",
                "Partha Talukdar"
            ],
            "abstract": "Given a sentence (e.g., \"I like mangoes\") and a constraint (e.g., sentiment\nflip), the goal of controlled text generation is to produce a sentence that\nadapts the input sentence to meet the requirements of the constraint (e.g., \"I\nhate mangoes\"). Going beyond such simple constraints, recent works have started\nexploring the incorporation of complex syntactic-guidance as constraints in the\ntask of controlled paraphrase generation. In these methods, syntactic-guidance\nis sourced from a separate exemplar sentence. However, these prior works have\nonly utilized limited syntactic information available in the parse tree of the\nexemplar sentence. We address this limitation in the paper and propose Syntax\nGuided Controlled Paraphraser (SGCP), an end-to-end framework for syntactic\nparaphrase generation. We find that SGCP can generate syntax conforming\nsentences while not compromising on relevance. We perform extensive automated\nand human evaluations over multiple real-world English language datasets to\ndemonstrate the efficacy of SGCP over state-of-the-art baselines. To drive\nfuture research, we have made SGCP's source code available",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08469v1",
            "date": "2020-05-18 06:04:58+00:00",
            "title": "Text Classification with Few Examples using Controlled Generalization",
            "authors": [
                "Abhijit Mahabal",
                "Jason Baldridge",
                "Burcu Karagol Ayan",
                "Vincent Perot",
                "Dan Roth"
            ],
            "abstract": "Training data for text classification is often limited in practice,\nespecially for applications with many output classes or involving many related\nclassification problems. This means classifiers must generalize from limited\nevidence, but the manner and extent of generalization is task dependent.\nCurrent practice primarily relies on pre-trained word embeddings to map words\nunseen in training to similar seen ones. Unfortunately, this squishes many\ncomponents of meaning into highly restricted capacity. Our alternative begins\nwith sparse pre-trained representations derived from unlabeled parsed corpora;\nbased on the available training data, we select features that offers the\nrelevant generalizations. This produces task-specific semantic vectors; here,\nwe show that a feed-forward network over these vectors is especially effective\nin low-data scenarios, compared to existing state-of-the-art methods. By\nfurther pairing this network with a convolutional neural network, we keep this\nedge in low data scenarios and remain competitive when using full training\nsets.",
            "journal ref": "Proceedings of NAACL-HLT 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08612v2",
            "date": "2020-05-18 11:49:35+00:00",
            "title": "The presence of occupational structure in online texts based on word embedding NLP models",
            "authors": [
                "Zolt\u00e1n Kmetty",
                "Julia Koltai",
                "Tam\u00e1s Rudas"
            ],
            "abstract": "Research on social stratification is closely linked to analysing the prestige\nassociated with different occupations. This research focuses on the positions\nof occupations in the semantic space represented by large amounts of textual\ndata. The results are compared to standard results in social stratification to\nsee whether the classical results are reproduced and if additional insights can\nbe gained into the social positions of occupations. The paper gives an\naffirmative answer to both questions. The results show fundamental similarity\nof the occupational structure obtained from text analysis to the structure\ndescribed by prestige and social distance scales. While our research reinforces\nmany theories and empirical findings of the traditional body of literature on\nsocial stratification and, in particular, occupational hierarchy, it pointed to\nthe importance of a factor not discussed in the main line of stratification\nliterature so far: the power and organizational aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08793v1",
            "date": "2020-05-18 15:14:33+00:00",
            "title": "Corpus of Chinese Dynastic Histories: Gender Analysis over Two Millennia",
            "authors": [
                "Sergey Zinin",
                "Yang Xu"
            ],
            "abstract": "Chinese dynastic histories form a large continuous linguistic space of\napproximately 2000 years, from the 3rd century BCE to the 18th century CE. The\nhistories are documented in Classical (Literary) Chinese in a corpus of over 20\nmillion characters, suitable for the computational analysis of historical\nlexicon and semantic change. However, there is no freely available open-source\ncorpus of these histories, making Classical Chinese low-resource. This project\nintroduces a new open-source corpus of twenty-four dynastic histories covered\nby Creative Commons license. An original list of Classical Chinese\ngender-specific terms was developed as a case study for analyzing the\nhistorical linguistic use of male and female terms. The study demonstrates\nconsiderable stability in the usage of these terms, with dominance of male\nterms. Exploration of word meanings uses keyword analysis of focus corpora\ncreated for genderspecific terms. This method yields meaningful semantic\nrepresentations that can be used for future studies of diachronic semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08805v1",
            "date": "2020-05-18 15:27:55+00:00",
            "title": "Interaction Matching for Long-Tail Multi-Label Classification",
            "authors": [
                "Sean MacAvaney",
                "Franck Dernoncourt",
                "Walter Chang",
                "Nazli Goharian",
                "Ophir Frieder"
            ],
            "abstract": "We present an elegant and effective approach for addressing limitations in\nexisting multi-label classification models by incorporating interaction\nmatching, a concept shown to be useful for ad-hoc search result ranking. By\nperforming soft n-gram interaction matching, we match labels with natural\nlanguage descriptions (which are common to have in most multi-labeling tasks).\nOur approach can be used to enhance existing multi-label classification\napproaches, which are biased toward frequently-occurring labels. We evaluate\nour approach on two challenging tasks: automatic medical coding of clinical\nnotes and automatic labeling of entities from software tutorial text. Our\nresults show that our method can yield up to an 11% relative improvement in\nmacro performance, with most of the gains stemming labels that appear\ninfrequently in the training set (i.e., the long tail of labels).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08826v1",
            "date": "2020-05-18 15:58:28+00:00",
            "title": "Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals",
            "authors": [
                "Kate McCurdy",
                "Sharon Goldwater",
                "Adam Lopez"
            ],
            "abstract": "Can artificial neural networks learn to represent inflectional morphology and\ngeneralize to new words as human speakers do? Kirov and Cotterell (2018) argue\nthat the answer is yes: modern Encoder-Decoder (ED) architectures learn\nhuman-like behavior when inflecting English verbs, such as extending the\nregular past tense form -(e)d to novel words. However, their work does not\naddress the criticism raised by Marcus et al. (1995): that neural models may\nlearn to extend not the regular, but the most frequent class -- and thus fail\non tasks like German number inflection, where infrequent suffixes like -s can\nstill be productively generalized.\n  To investigate this question, we first collect a new dataset from German\nspeakers (production and ratings of plural forms for novel nouns) that is\ndesigned to avoid sources of information unavailable to the ED model. The\nspeaker data show high variability, and two suffixes evince 'regular' behavior,\nappearing more often with phonologically atypical inputs. Encoder-decoder\nmodels do generalize the most frequently produced plural class, but do not show\nhuman-like variability or 'regular' extension of these other plural markers. We\nconclude that modern neural models may still struggle with minority-class\ngeneralization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08864v1",
            "date": "2020-05-18 16:39:16+00:00",
            "title": "Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings",
            "authors": [
                "Katherine McCurdy",
                "Oguz Serbetci"
            ],
            "abstract": "Recent research has demonstrated that vector space models of semantics can\nreflect undesirable biases in human culture. Our investigation of\ncrosslinguistic word embeddings reveals that topical gender bias interacts\nwith, and is surpassed in magnitude by, the effect of grammatical gender\nassociations, and both may be attenuated by corpus lemmatization. This finding\nhas implications for downstream applications such as machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08866v2",
            "date": "2020-05-18 16:40:30+00:00",
            "title": "Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations",
            "authors": [
                "Sam Coope",
                "Tyler Farghly",
                "Daniela Gerz",
                "Ivan Vuli\u0107",
                "Matthew Henderson"
            ],
            "abstract": "We introduce Span-ConveRT, a light-weight model for dialog slot-filling which\nframes the task as a turn-based span extraction task. This formulation allows\nfor a simple integration of conversational knowledge coded in large pretrained\nconversational models such as ConveRT (Henderson et al., 2019). We show that\nleveraging such knowledge in Span-ConveRT is especially useful for few-shot\nlearning scenarios: we report consistent gains over 1) a span extractor that\ntrains representations from scratch in the target domain, and 2) a BERT-based\nspan extractor. In order to inspire more work on span extraction for the\nslot-filling task, we also release RESTAURANTS-8K, a new challenging data set\nof 8,198 utterances, compiled from actual conversations in the restaurant\nbooking domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08932v1",
            "date": "2020-05-18 17:57:24+00:00",
            "title": "Reconstructing Maps from Text",
            "authors": [
                "Johnathan E. Avery",
                "Robert L. Goldstone",
                "Michael N. Jones"
            ],
            "abstract": "Previous research has demonstrated that Distributional Semantic Models (DSMs)\nare capable of reconstructing maps from news corpora (Louwerse & Zwaan, 2009)\nand novels (Louwerse & Benesh, 2012). The capacity for reproducing maps is\nsurprising since DSMs notoriously lack perceptual grounding (De Vega et al.,\n2012). In this paper we investigate the statistical sources required in\nlanguage to infer maps, and resulting constraints placed on mechanisms of\nsemantic representation. Study 1 brings word co-occurrence under experimental\ncontrol to demonstrate that direct co-occurrence in language is necessary for\ntraditional DSMs to successfully reproduce maps. Study 2 presents an\ninstance-based DSM that is capable of reconstructing maps independent of the\nfrequency of co-occurrence of city names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08946v1",
            "date": "2020-05-16 06:40:36+00:00",
            "title": "Arabic Offensive Language Detection Using Machine Learning and Ensemble Machine Learning Approaches",
            "authors": [
                "Fatemah Husain"
            ],
            "abstract": "This study aims at investigating the effect of applying single learner\nmachine learning approach and ensemble machine learning approach for offensive\nlanguage detection on Arabic language. Classifying Arabic social media text is\na very challenging task due to the ambiguity and informality of the written\nformat of the text. Arabic language has multiple dialects with diverse\nvocabularies and structures, which increase the complexity of obtaining high\nclassification performance. Our study shows significant impact for applying\nensemble machine learning approach over the single learner machine learning\napproach. Among the trained ensemble machine learning classifiers, bagging\nperforms the best in offensive language detection with F1 score of 88%, which\nexceeds the score obtained by the best single learner classifier by 6%. Our\nfindings highlight the great opportunities of investing more efforts in\npromoting the ensemble machine learning approach solutions for offensive\nlanguage detection models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09067v2",
            "date": "2020-05-18 20:36:11+00:00",
            "title": "Question-Driven Summarization of Answers to Consumer Health Questions",
            "authors": [
                "Max Savery",
                "Asma Ben Abacha",
                "Soumya Gayen",
                "Dina Demner-Fushman"
            ],
            "abstract": "Automatic summarization of natural language is a widely studied area in\ncomputer science, one that is broadly applicable to anyone who routinely needs\nto understand large quantities of information. For example, in the medical\ndomain, recent developments in deep learning approaches to automatic\nsummarization have the potential to make health information more easily\naccessible to patients and consumers. However, to evaluate the quality of\nautomatically generated summaries of health information, gold-standard, human\ngenerated summaries are required. Using answers provided by the National\nLibrary of Medicine's consumer health question answering system, we present the\nMEDIQA Answer Summarization dataset, the first summarization collection\ncontaining question-driven summaries of answers to consumer health questions.\nThis dataset can be used to evaluate single or multi-document summaries\ngenerated by algorithms using extractive or abstractive approaches. In order to\nbenchmark the dataset, we include results of baseline and state-of-the-art deep\nlearning summarization models, demonstrating that this dataset can be used to\neffectively evaluate question-driven machine-generated summaries and promote\nfurther machine learning research in medical question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09093v2",
            "date": "2020-05-18 21:15:39+00:00",
            "title": "Are All Languages Created Equal in Multilingual BERT?",
            "authors": [
                "Shijie Wu",
                "Mark Dredze"
            ],
            "abstract": "Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly\ngood cross-lingual performance on several NLP tasks, even without explicit\ncross-lingual signals. However, these evaluations have focused on cross-lingual\ntransfer with high-resource languages, covering only a third of the languages\ncovered by mBERT. We explore how mBERT performs on a much wider set of\nlanguages, focusing on the quality of representation for low-resource\nlanguages, measured by within-language performance. We consider three tasks:\nNamed Entity Recognition (99 languages), Part-of-speech Tagging, and Dependency\nParsing (54 languages each). mBERT does better than or comparable to baselines\non high resource languages but does much worse for low resource languages.\nFurthermore, monolingual BERT models for these languages do even worse. Paired\nwith similar languages, the performance gap between monolingual BERT and mBERT\ncan be narrowed. We find that better models for low resource languages require\nmore efficient pretraining techniques or more data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09099v1",
            "date": "2020-05-18 21:21:34+00:00",
            "title": "(Re)construing Meaning in NLP",
            "authors": [
                "Sean Trott",
                "Tiago Timponi Torrent",
                "Nancy Chang",
                "Nathan Schneider"
            ],
            "abstract": "Human speakers have an extensive toolkit of ways to express themselves. In\nthis paper, we engage with an idea largely absent from discussions of meaning\nin natural language understanding--namely, that the way something is expressed\nreflects different ways of conceptualizing or construing the information being\nconveyed. We first define this phenomenon more precisely, drawing on\nconsiderable prior work in theoretical cognitive semantics and\npsycholinguistics. We then survey some dimensions of construed meaning and show\nhow insights from construal could inform theoretical and practical work in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09128v1",
            "date": "2020-05-18 23:00:57+00:00",
            "title": "Neural Generation of Dialogue Response Timings",
            "authors": [
                "Matthew Roddy",
                "Naomi Harte"
            ],
            "abstract": "The timings of spoken response offsets in human dialogue have been shown to\nvary based on contextual elements of the dialogue. We propose neural models\nthat simulate the distributions of these response offsets, taking into account\nthe response turn as well as the preceding turn. The models are designed to be\nintegrated into the pipeline of an incremental spoken dialogue system (SDS). We\nevaluate our models using offline experiments as well as human listening tests.\nWe show that human listeners consider certain response timings to be more\nnatural based on the dialogue context. The introduction of these models into\nSDS pipelines could increase the perceived naturalness of interactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09276v2",
            "date": "2020-05-19 08:18:52+00:00",
            "title": "Matching Questions and Answers in Dialogues from Online Forums",
            "authors": [
                "Qi Jia",
                "Mengxue Zhang",
                "Shengyao Zhang",
                "Kenny Q. Zhu"
            ],
            "abstract": "Matching question-answer relations between two turns in conversations is not\nonly the first step in analyzing dialogue structures, but also valuable for\ntraining dialogue systems. This paper presents a QA matching model considering\nboth distance information and dialogue history by two simultaneous attention\nmechanisms called mutual attention. Given scores computed by the trained model\nbetween each non-question turn with its candidate questions, a greedy matching\nstrategy is used for final predictions. Because existing dialogue datasets such\nas the Ubuntu dataset are not suitable for the QA matching task, we further\ncreate a dataset with 1,000 labeled dialogues and demonstrate that our proposed\nmodel outperforms the state-of-the-art and other strong baselines, particularly\nfor matching long-distance QA pairs.",
            "journal ref": "ECAI 2020: 24th European Conference on Artificial Intelligence",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09379v1",
            "date": "2020-05-19 11:55:11+00:00",
            "title": "Staying True to Your Word: (How) Can Attention Become Explanation?",
            "authors": [
                "Martin Tutek",
                "Jan \u0160najder"
            ],
            "abstract": "The attention mechanism has quickly become ubiquitous in NLP. In addition to\nimproving performance of models, attention has been widely used as a glimpse\ninto the inner workings of NLP models. The latter aspect has in the recent\nyears become a common topic of discussion, most notably in work of Jain and\nWallace, 2019; Wiegreffe and Pinter, 2019. With the shortcomings of using\nattention weights as a tool of transparency revealed, the attention mechanism\nhas been stuck in a limbo without concrete proof when and whether it can be\nused as an explanation. In this paper, we provide an explanation as to why\nattention has seen rightful critique when used with recurrent networks in\nsequence classification tasks. We propose a remedy to these issues in the form\nof a word level objective and our findings give credibility for attention to\nprovide faithful interpretations of recurrent models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09382v1",
            "date": "2020-05-19 12:16:58+00:00",
            "title": "Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text",
            "authors": [
                "Felix Hill",
                "Sona Mokra",
                "Nathaniel Wong",
                "Tim Harley"
            ],
            "abstract": "Recent work has described neural-network-based agents that are trained with\nreinforcement learning (RL) to execute language-like commands in simulated\nworlds, as a step towards an intelligent agent or robot that can be instructed\nby human users. However, the optimisation of multi-goal motor policies via deep\nRL from scratch requires many episodes of experience. Consequently,\ninstruction-following with deep RL typically involves language generated from\ntemplates (by an environment simulator), which does not reflect the varied or\nambiguous expressions of real users. Here, we propose a conceptually simple\nmethod for training instruction-following agents with deep RL that are robust\nto natural human instructions. By applying our method with a state-of-the-art\npre-trained text-based language model (BERT), on tasks requiring agents to\nidentify and position everyday objects relative to other objects in a\nnaturalistic 3D simulated room, we demonstrate substantially-above-chance\nzero-shot transfer from synthetic template commands to natural instructions\ngiven by humans. Our approach is a general recipe for training any deep\nRL-based system to interface with human users, and bridges the gap between two\nresearch directions of notable recent success: agent-centric motor behavior and\ntext-based representation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09471v2",
            "date": "2020-05-19 14:17:49+00:00",
            "title": "Human Sentence Processing: Recurrence or Attention?",
            "authors": [
                "Danny Merkx",
                "Stefan L. Frank"
            ],
            "abstract": "Recurrent neural networks (RNNs) have long been an architecture of interest\nfor computational models of human sentence processing. The recently introduced\nTransformer architecture outperforms RNNs on many natural language processing\ntasks but little is known about its ability to model human language processing.\nWe compare Transformer- and RNN-based language models' ability to account for\nmeasures of human reading effort. Our analysis shows Transformers to outperform\nRNNs in explaining self-paced reading times and neural activity during reading\nEnglish sentences, challenging the widely held idea that human sentence\nprocessing involves recurrent and immediate processing and provides evidence\nfor cue-based retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09606v1",
            "date": "2020-05-19 17:27:00+00:00",
            "title": "A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks",
            "authors": [
                "Angela S. Lin",
                "Sudha Rao",
                "Asli Celikyilmaz",
                "Elnaz Nouri",
                "Chris Brockett",
                "Debadeepta Dey",
                "Bill Dolan"
            ],
            "abstract": "Many high-level procedural tasks can be decomposed into sequences of\ninstructions that vary in their order and choice of tools. In the cooking\ndomain, the web offers many partially-overlapping text and video recipes (i.e.\nprocedures) that describe how to make the same dish (i.e. high-level task).\nAligning instructions for the same dish across different sources can yield\ndescriptive visual explanations that are far richer semantically than\nconventional textual instructions, providing commonsense insight into how\nreal-world procedures are structured. Learning to align these different\ninstruction sets is challenging because: a) different recipes vary in their\norder of instructions and use of ingredients; and b) video instructions can be\nnoisy and tend to contain far more information than text instructions. To\naddress these challenges, we first use an unsupervised alignment algorithm that\nlearns pairwise alignments between instructions of different recipes for the\nsame dish. We then use a graph algorithm to derive a joint alignment between\nmultiple text and multiple video recipes for the same dish. We release the\nMicrosoft Research Multimodal Aligned Recipe Corpus containing 150K pairwise\nalignments between recipes across 4,262 dishes with rich commonsense\ninformation.",
            "journal ref": "Association of Computational Linguistics 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10043v1",
            "date": "2020-05-20 13:39:47+00:00",
            "title": "Leveraging Graph to Improve Abstractive Multi-Document Summarization",
            "authors": [
                "Wei Li",
                "Xinyan Xiao",
                "Jiachen Liu",
                "Hua Wu",
                "Haifeng Wang",
                "Junping Du"
            ],
            "abstract": "Graphs that capture relations between textual units have great benefits for\ndetecting salient information from multiple documents and generating overall\ncoherent summaries. In this paper, we develop a neural abstractive\nmulti-document summarization (MDS) model which can leverage well-known graph\nrepresentations of documents such as similarity graph and discourse graph, to\nmore effectively process multiple input documents and produce abstractive\nsummaries. Our model utilizes graphs to encode documents in order to capture\ncross-document relations, which is crucial to summarizing long documents. Our\nmodel can also take advantage of graphs to guide the summary generation\nprocess, which is beneficial for generating coherent and concise summaries.\nFurthermore, pre-trained language models can be easily combined with our model,\nwhich further improve the summarization performance significantly. Empirical\nresults on the WikiSum and MultiNews dataset show that the proposed\narchitecture brings substantial improvements over several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10070v1",
            "date": "2020-05-20 14:33:33+00:00",
            "title": "A Large-Scale Multi-Document Summarization Dataset from the Wikipedia Current Events Portal",
            "authors": [
                "Demian Gholipour Ghalandari",
                "Chris Hokamp",
                "Nghia The Pham",
                "John Glover",
                "Georgiana Ifrim"
            ],
            "abstract": "Multi-document summarization (MDS) aims to compress the content in large\ndocument collections into short summaries and has important applications in\nstory clustering for newsfeeds, presentation of search results, and timeline\ngeneration. However, there is a lack of datasets that realistically address\nsuch use cases at a scale large enough for training supervised models for this\ntask. This work presents a new dataset for MDS that is large both in the total\nnumber of document clusters and in the size of individual clusters. We build\nthis dataset by leveraging the Wikipedia Current Events Portal (WCEP), which\nprovides concise and neutral human-written summaries of news events, with links\nto external source articles. We also automatically extend these source articles\nby looking for related articles in the Common Crawl archive. We provide a\nquantitative analysis of the dataset and empirical results for several\nstate-of-the-art MDS techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10107v1",
            "date": "2020-05-20 15:06:39+00:00",
            "title": "Examining the State-of-the-Art in News Timeline Summarization",
            "authors": [
                "Demian Gholipour Ghalandari",
                "Georgiana Ifrim"
            ],
            "abstract": "Previous work on automatic news timeline summarization (TLS) leaves an\nunclear picture about how this task can generally be approached and how well it\nis currently solved. This is mostly due to the focus on individual subtasks,\nsuch as date selection and date summarization, and to the previous lack of\nappropriate evaluation metrics for the full TLS task. In this paper, we compare\ndifferent TLS strategies using appropriate evaluation frameworks, and propose a\nsimple and effective combination of methods that improves over the\nstate-of-the-art on all tested benchmarks. For a more robust evaluation, we\nalso present a new TLS dataset, which is larger and spans longer time periods\nthan previous datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10213v2",
            "date": "2020-05-20 17:25:43+00:00",
            "title": "Applying the Transformer to Character-level Transduction",
            "authors": [
                "Shijie Wu",
                "Ryan Cotterell",
                "Mans Hulden"
            ],
            "abstract": "The transformer has been shown to outperform recurrent neural network-based\nsequence-to-sequence models in various word-level NLP tasks. Yet for\ncharacter-level transduction tasks, e.g. morphological inflection generation\nand historical text normalization, there are few works that outperform\nrecurrent models using the transformer. In an empirical study, we uncover that,\nin contrast to recurrent sequence-to-sequence models, the batch size plays a\ncrucial role in the performance of the transformer on character-level tasks,\nand we show that with a large enough batch size, the transformer does indeed\noutperform recurrent models. We also introduce a simple technique to handle\nfeature-guided character-level transduction that further improves performance.\nWith these insights, we achieve state-of-the-art performance on morphological\ninflection and historical text normalization. We also show that the transformer\noutperforms a strong baseline on two other character-level transduction tasks:\ngrapheme-to-phoneme conversion and transliteration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10232v1",
            "date": "2020-05-20 17:47:56+00:00",
            "title": "Sentence level estimation of psycholinguistic norms using joint multidimensional annotations",
            "authors": [
                "Anil Ramakrishna",
                "Shrikanth Narayanan"
            ],
            "abstract": "Psycholinguistic normatives represent various affective and mental constructs\nusing numeric scores and are used in a variety of applications in natural\nlanguage processing. They are commonly used at the sentence level, the scores\nof which are estimated by extrapolating word level scores using simple\naggregation strategies, which may not always be optimal. In this work, we\npresent a novel approach to estimate the psycholinguistic norms at sentence\nlevel. We apply a multidimensional annotation fusion model on annotations at\nthe word level to estimate a parameter which captures relationships between\ndifferent norms. We then use this parameter at sentence level to estimate the\nnorms. We evaluate our approach by predicting sentence level scores for various\nnormative dimensions and compare with standard word aggregation schemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10283v2",
            "date": "2020-05-20 18:05:51+00:00",
            "title": "Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation",
            "authors": [
                "Bryan Eikema",
                "Wilker Aziz"
            ],
            "abstract": "Recent studies have revealed a number of pathologies of neural machine\ntranslation (NMT) systems. Hypotheses explaining these mostly suggest there is\nsomething fundamentally wrong with NMT as a model or its training algorithm,\nmaximum likelihood estimation (MLE). Most of this evidence was gathered using\nmaximum a posteriori (MAP) decoding, a decision rule aimed at identifying the\nhighest-scoring translation, i.e. the mode. We argue that the evidence\ncorroborates the inadequacy of MAP decoding more than casts doubt on the model\nand its training algorithm. In this work, we show that translation\ndistributions do reproduce various statistics of the data well, but that beam\nsearch strays from such statistics. We show that some of the known pathologies\nand biases of NMT are due to MAP decoding and not to NMT's statistical\nassumptions nor MLE. In particular, we show that the most likely translations\nunder the model accumulate so little probability mass that the mode can be\nconsidered essentially arbitrary. We therefore advocate for the use of decision\nrules that take into account the translation distribution holistically. We show\nthat an approximation to minimum Bayes risk decoding gives competitive results\nconfirming that NMT models do capture important aspects of translation well in\nexpectation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10331v2",
            "date": "2020-05-20 19:48:50+00:00",
            "title": "ScriptWriter: Narrative-Guided Script Generation",
            "authors": [
                "Yutao Zhu",
                "Ruihua Song",
                "Zhicheng Dou",
                "Jian-Yun Nie",
                "Jin Zhou"
            ],
            "abstract": "It is appealing to have a system that generates a story or scripts\nautomatically from a story-line, even though this is still out of our reach. In\ndialogue systems, it would also be useful to drive dialogues by a dialogue\nplan. In this paper, we address a key problem involved in these applications --\nguiding a dialogue by a narrative. The proposed model ScriptWriter selects the\nbest response among the candidates that fit the context as well as the given\nnarrative. It keeps track of what in the narrative has been said and what is to\nbe said. A narrative plays a different role than the context (i.e., previous\nutterances), which is generally used in current dialogue systems. Due to the\nunavailability of data for this new application, we construct a new large-scale\ndata collection GraphMovie from a movie website where end-users can upload\ntheir narratives freely when watching a movie. Experimental results on the\ndataset show that our proposed approach based on narratives significantly\noutperforms the baselines that simply use the narrative as a kind of context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10389v1",
            "date": "2020-05-20 23:21:43+00:00",
            "title": "Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models",
            "authors": [
                "Dan Iter",
                "Kelvin Guu",
                "Larry Lansing",
                "Dan Jurafsky"
            ],
            "abstract": "Recent models for unsupervised representation learning of text have employed\na number of techniques to improve contextual word representations but have put\nlittle focus on discourse-level representations. We propose CONPONO, an\ninter-sentence objective for pretraining language models that models discourse\ncoherence and the distance between sentences. Given an anchor sentence, our\nmodel is trained to predict the text k sentences away using a sampled-softmax\nobjective where the candidates consist of neighboring sentences and sentences\nrandomly sampled from the corpus. On the discourse representation benchmark\nDiscoEval, our model improves over the previous state-of-the-art by up to 13%\nand on average 4% absolute across 7 tasks. Our model is the same size as\nBERT-Base, but outperforms the much larger BERT- Large model and other more\nrecent approaches that incorporate discourse. We also show that CONPONO yields\ngains of 2%-6% absolute even for tasks that do not explicitly evaluate\ndiscourse: textual entailment (RTE), common sense reasoning (COPA) and reading\ncomprehension (ReCoRD).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10410v1",
            "date": "2020-05-21 01:17:46+00:00",
            "title": "Stance Prediction and Claim Verification: An Arabic Perspective",
            "authors": [
                "Jude Khouja"
            ],
            "abstract": "This work explores the application of textual entailment in news claim\nverification and stance prediction using a new corpus in Arabic. The publicly\navailable corpus comes in two perspectives: a version consisting of 4,547 true\nand false claims and a version consisting of 3,786 pairs (claim, evidence). We\ndescribe the methodology for creating the corpus and the annotation process.\nUsing the introduced corpus, we also develop two machine learning baselines for\ntwo proposed tasks: claim verification and stance prediction. Our best model\nutilizes pretraining (BERT) and achieves 76.7 F1 on the stance prediction task\nand 64.3 F1 on the claim verification task. Our preliminary experiments shed\nsome light on the limits of automatic claim verification that relies on claims\ntext only. Results hint that while the linguistic features and world knowledge\nlearned during pretraining are useful for stance prediction, such learned\nrepresentations from pretraining are insufficient for verifying claims without\naccess to context or evidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10433v2",
            "date": "2020-05-21 02:46:15+00:00",
            "title": "Text-to-Text Pre-Training for Data-to-Text Tasks",
            "authors": [
                "Mihir Kale"
            ],
            "abstract": "We study the pre-train + fine-tune strategy for data-to-text tasks.\nFine-tuning T5 achieves state-of-the-art results on the WebNLG, MultiWoz and\nToTTo benchmarks. Such transfer learning enables training of fully end-to-end\nmodels that do not rely on any intermediate planning steps, delexicalization or\ncopy mechanisms. T5 pre-training also enables stronger generalization, as\nevidenced by large improvements on out-of-domain test sets. We hope our work\nserves as a useful baseline for future research, as pre-training becomes ever\nmore prevalent for data-to-text tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10464v2",
            "date": "2020-05-21 04:57:01+00:00",
            "title": "Fluent Response Generation for Conversational Question Answering",
            "authors": [
                "Ashutosh Baheti",
                "Alan Ritter",
                "Kevin Small"
            ],
            "abstract": "Question answering (QA) is an important aspect of open-domain conversational\nagents, garnering specific research focus in the conversational QA (ConvQA)\nsubtask. One notable limitation of recent ConvQA efforts is the response being\nanswer span extraction from the target corpus, thus ignoring the natural\nlanguage generation (NLG) aspect of high-quality conversational agents. In this\nwork, we propose a method for situating QA responses within a SEQ2SEQ NLG\napproach to generate fluent grammatical answer responses while maintaining\ncorrectness. From a technical perspective, we use data augmentation to generate\ntraining data for an end-to-end system. Specifically, we develop Syntactic\nTransformations (STs) to produce question-specific candidate answer responses\nand rank them using a BERT-based classifier (Devlin et al., 2019). Human\nevaluation on SQuAD 2.0 data (Rajpurkar et al., 2018) demonstrate that the\nproposed model outperforms baseline CoQA and QuAC models in generating\nconversational responses. We further show our model's scalability by conducting\ntests on the CoQA dataset. The code and data are available at\nhttps://github.com/abaheti95/QADialogSystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10527v1",
            "date": "2020-05-21 09:03:03+00:00",
            "title": "LaCulturaNonSiFerma -- Report su uso e la diffusione degli hashtag delle istituzioni culturali italiane durante il periodo di lockdown",
            "authors": [
                "Carola Carlino",
                "Gennaro Nolano",
                "Maria Pia di Buono",
                "Johanna Monti"
            ],
            "abstract": "This report presents an analysis of #hashtags used by Italian Cultural\nHeritage institutions to promote and communicate cultural content during the\nCOVID-19 lock-down period in Italy. Several activities to support and engage\nusers' have been proposed using social media. Most of these activities present\none or more #hashtags which help to aggregate content and create a community on\nspecific topics. Results show that on one side Italian institutions have been\nvery proactive in adapting to the pandemic scenario and on the other side\nusers' reacted very positively increasing their participation in the proposed\nactivities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10583v1",
            "date": "2020-05-21 11:46:44+00:00",
            "title": "MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora",
            "authors": [
                "Lifeng Han",
                "Gareth J. F. Jones",
                "Alan F. Smeaton"
            ],
            "abstract": "Multi-word expressions (MWEs) are a hot topic in research in natural language\nprocessing (NLP), including topics such as MWE detection, MWE decomposition,\nand research investigating the exploitation of MWEs in other NLP fields such as\nMachine Translation. However, the availability of bilingual or multi-lingual\nMWE corpora is very limited. The only bilingual MWE corpora that we are aware\nof is from the PARSEME (PARSing and Multi-word Expressions) EU Project. This is\na small collection of only 871 pairs of English-German MWEs. In this paper, we\npresent multi-lingual and bilingual MWE corpora that we have extracted from\nroot parallel corpora. Our collections are 3,159,226 and 143,042 bilingual MWE\npairs for German-English and Chinese-English respectively after filtering. We\nexamine the quality of these extracted bilingual MWEs in MT experiments. Our\ninitial experiments applying MWEs in MT show improved translation performances\non MWE terms in qualitative analysis and better general evaluation scores in\nquantitative analysis, on both German-English and Chinese-English language\npairs. We follow a standard experimental pipeline to create our MultiMWE\ncorpora which are available online. Researchers can use this free corpus for\ntheir own models or use them in a knowledge base as model features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10608v2",
            "date": "2020-05-21 12:38:06+00:00",
            "title": "Unsupervised Quality Estimation for Neural Machine Translation",
            "authors": [
                "Marina Fomicheva",
                "Shuo Sun",
                "Lisa Yankovskaya",
                "Fr\u00e9d\u00e9ric Blain",
                "Francisco Guzm\u00e1n",
                "Mark Fishel",
                "Nikolaos Aletras",
                "Vishrav Chaudhary",
                "Lucia Specia"
            ],
            "abstract": "Quality Estimation (QE) is an important component in making Machine\nTranslation (MT) useful in real-world applications, as it is aimed to inform\nthe user on the quality of the MT output at test time. Existing approaches\nrequire large amounts of expert annotated data, computation and time for\ntraining. As an alternative, we devise an unsupervised approach to QE where no\ntraining or access to additional resources besides the MT system itself is\nrequired. Different from most of the current work that treats the MT system as\na black box, we explore useful information that can be extracted from the MT\nsystem as a by-product of translation. By employing methods for uncertainty\nquantification, we achieve very good correlation with human judgments of\nquality, rivalling state-of-the-art supervised QE models. To evaluate our\napproach we collect the first dataset that enables work on both black-box and\nglass-box approaches to QE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10652v1",
            "date": "2020-05-21 13:55:07+00:00",
            "title": "Towards Finite-State Morphology of Kurdish",
            "authors": [
                "Sina Ahmadi",
                "Hossein Hassani"
            ],
            "abstract": "Morphological analysis is the study of the formation and structure of words.\nIt plays a crucial role in various tasks in Natural Language Processing (NLP)\nand Computational Linguistics (CL) such as machine translation and text and\nspeech generation. Kurdish is a less-resourced multi-dialect Indo-European\nlanguage with highly inflectional morphology. In this paper, as the first\nattempt of its kind, the morphology of the Kurdish language (Sorani dialect) is\ndescribed from a computational point of view. We extract morphological rules\nwhich are transformed into finite-state transducers for generating and\nanalyzing words. The result of this research assists in conducting studies on\nlanguage generation for Kurdish and enhances the Information Retrieval (IR)\ncapacity for the language while leveraging the Kurdish NLP and CL into a more\nadvanced computational level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10659v1",
            "date": "2020-05-21 14:06:15+00:00",
            "title": "RuBQ: A Russian Dataset for Question Answering over Wikidata",
            "authors": [
                "Vladislav Korablinov",
                "Pavel Braslavski"
            ],
            "abstract": "The paper presents RuBQ, the first Russian knowledge base question answering\n(KBQA) dataset. The high-quality dataset consists of 1,500 Russian questions of\nvarying complexity, their English machine translations, SPARQL queries to\nWikidata, reference answers, as well as a Wikidata sample of triples containing\nentities with Russian labels. The dataset creation started with a large\ncollection of question-answer pairs from online quizzes. The data underwent\nautomatic filtering, crowd-assisted entity linking, automatic generation of\nSPARQL queries, and their subsequent in-house verification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10678v2",
            "date": "2020-05-21 14:22:35+00:00",
            "title": "Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation",
            "authors": [
                "Shun-Po Chuang",
                "Tzu-Wei Sung",
                "Alexander H. Liu",
                "Hung-yi Lee"
            ],
            "abstract": "Speech translation (ST) aims to learn transformations from speech in the\nsource language to the text in the target language. Previous works show that\nmultitask learning improves the ST performance, in which the recognition\ndecoder generates the text of the source language, and the translation decoder\nobtains the final translations based on the output of the recognition decoder.\nBecause whether the output of the recognition decoder has the correct semantics\nis more critical than its accuracy, we propose to improve the multitask ST\nmodel by utilizing word embedding as the intermediate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10790v1",
            "date": "2020-05-21 17:16:53+00:00",
            "title": "The Frankfurt Latin Lexicon: From Morphological Expansion and Word Embeddings to SemioGraphs",
            "authors": [
                "Alexander Mehler",
                "Bernhard Jussen",
                "Tim Geelhaar",
                "Alexander Henlein",
                "Giuseppe Abrami",
                "Daniel Baumartz",
                "Tolga Uslu",
                "Wahed Hemati"
            ],
            "abstract": "In this article we present the Frankfurt Latin Lexicon (FLL), a lexical\nresource for Medieval Latin that is used both for the lemmatization of Latin\ntexts and for the post-editing of lemmatizations. We describe recent advances\nin the development of lemmatizers and test them against the Capitularies corpus\n(comprising Frankish royal edicts, mid-6th to mid-9th century), a corpus\ncreated as a reference for processing Medieval Latin. We also consider the\npost-correction of lemmatizations using a limited crowdsourcing process aimed\nat continuous review and updating of the FLL. Starting from the texts resulting\nfrom this lemmatization process, we describe the extension of the FLL by means\nof word embeddings, whose interactive traversing by means of SemioGraphs\ncompletes the digital enhanced hermeneutic circle. In this way, the article\nargues for a more comprehensive understanding of lemmatization, encompassing\nclassical machine learning as well as intellectual post-corrections and, in\nparticular, human computation in the form of interpretation processes based on\ngraph representations of the underlying lexical resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10893v1",
            "date": "2020-05-21 20:36:32+00:00",
            "title": "Evaluating Neural Morphological Taggers for Sanskrit",
            "authors": [
                "Ashim Gupta",
                "Amrith Krishna",
                "Pawan Goyal",
                "Oliver Hellwig"
            ],
            "abstract": "Neural sequence labelling approaches have achieved state of the art results\nin morphological tagging. We evaluate the efficacy of four standard sequence\nlabelling models on Sanskrit, a morphologically rich, fusional Indian language.\nAs its label space can theoretically contain more than 40,000 labels, systems\nthat explicitly model the internal structure of a label are more suited for the\ntask, because of their ability to generalise to labels not seen during\ntraining. We find that although some neural models perform better than others,\none of the common causes for error for all of these models is mispredictions\ndue to syncretism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11009v1",
            "date": "2020-05-22 05:17:53+00:00",
            "title": "Investigating Label Bias in Beam Search for Open-ended Text Generation",
            "authors": [
                "Liang Wang",
                "Jinlong Liu",
                "Jingming Liu"
            ],
            "abstract": "Beam search is an effective and widely used decoding algorithm in many\nsequence-to-sequence (seq2seq) text generation tasks. However, in open-ended\ntext generation, beam search is often found to produce repetitive and generic\ntexts, sampling-based decoding algorithms like top-k sampling and nucleus\nsampling are more preferred. Standard seq2seq models suffer from label bias due\nto its locally normalized probability formulation. This paper provides a series\nof empirical evidence that label bias is a major reason for such degenerate\nbehaviors of beam search. By combining locally normalized maximum likelihood\nestimation and globally normalized sequence-level training, label bias can be\nreduced with almost no sacrifice in perplexity. To quantitatively measure label\nbias, we test the model's ability to discriminate the groundtruth text and a\nset of context-agnostic distractors. We conduct experiments on large-scale\nresponse generation datasets. Results show that beam search can produce more\ndiverse and meaningful texts with our approach, in terms of both automatic and\nhuman evaluation metrics. Our analysis also suggests several future working\ndirections towards the grand challenge of open-ended text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11055v1",
            "date": "2020-05-22 08:29:06+00:00",
            "title": "Improving Segmentation for Technical Support Problems",
            "authors": [
                "Kushal Chauhan",
                "Abhirut Gupta"
            ],
            "abstract": "Technical support problems are often long and complex. They typically contain\nuser descriptions of the problem, the setup, and steps for attempted\nresolution. Often they also contain various non-natural language text elements\nlike outputs of commands, snippets of code, error messages or stack traces.\nThese elements contain potentially crucial information for problem resolution.\nHowever, they cannot be correctly parsed by tools designed for natural\nlanguage. In this paper, we address the problem of segmentation for technical\nsupport questions. We formulate the problem as a sequence labelling task, and\nstudy the performance of state of the art approaches. We compare this against\nan intuitive contextual sentence-level classification baseline, and a state of\nthe art supervised text-segmentation approach. We also introduce a novel\ncomponent of combining contextual embeddings from multiple language models\npre-trained on different data sources, which achieves a marked improvement over\nusing embeddings from a single pre-trained language model. Finally, we also\ndemonstrate the usefulness of such segmentation with improvements on the\ndownstream task of answer retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11075v1",
            "date": "2020-05-22 09:35:30+00:00",
            "title": "Bootstrapping Named Entity Recognition in E-Commerce with Positive Unlabeled Learning",
            "authors": [
                "Hanchu Zhang",
                "Leonhard Hennig",
                "Christoph Alt",
                "Changjian Hu",
                "Yao Meng",
                "Chao Wang"
            ],
            "abstract": "Named Entity Recognition (NER) in domains like e-commerce is an understudied\nproblem due to the lack of annotated datasets. Recognizing novel entity types\nin this domain, such as products, components, and attributes, is challenging\nbecause of their linguistic complexity and the low coverage of existing\nknowledge resources. To address this problem, we present a bootstrapped\npositive-unlabeled learning algorithm that integrates domain-specific\nlinguistic features to quickly and efficiently expand the seed dictionary. The\nmodel achieves an average F1 score of 72.02% on a novel dataset of product\ndescriptions, an improvement of 3.63% over a baseline BiLSTM classifier, and in\nparticular exhibits better recall (4.96% on average).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11140v2",
            "date": "2020-05-22 12:35:18+00:00",
            "title": "Living Machines: A study of atypical animacy",
            "authors": [
                "Mariona Coll Ardanuy",
                "Federico Nanni",
                "Kaspar Beelen",
                "Kasra Hosseini",
                "Ruth Ahnert",
                "Jon Lawrence",
                "Katherine McDonough",
                "Giorgia Tolfo",
                "Daniel CS Wilson",
                "Barbara McGillivray"
            ],
            "abstract": "This paper proposes a new approach to animacy detection, the task of\ndetermining whether an entity is represented as animate in a text. In\nparticular, this work is focused on atypical animacy and examines the scenario\nin which typically inanimate objects, specifically machines, are given animate\nattributes. To address it, we have created the first dataset for atypical\nanimacy detection, based on nineteenth-century sentences in English, with\nmachines represented as either animate or inanimate. Our method builds on\nrecent innovations in language modeling, specifically BERT contextualized word\nembeddings, to better capture fine-grained contextual properties of words. We\npresent a fully unsupervised pipeline, which can be easily adapted to different\ncontexts, and report its performance on an established animacy dataset and our\nnewly introduced resource. We show that our method provides a substantially\nmore accurate characterization of atypical animacy, especially when applied to\nhighly complex forms of language use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11197v2",
            "date": "2020-05-22 14:15:53+00:00",
            "title": "Simplify-then-Translate: Automatic Preprocessing for Black-Box Machine Translation",
            "authors": [
                "Sneha Mehta",
                "Bahareh Azarnoush",
                "Boris Chen",
                "Avneesh Saluja",
                "Vinith Misra",
                "Ballav Bihani",
                "Ritwik Kumar"
            ],
            "abstract": "Black-box machine translation systems have proven incredibly useful for a\nvariety of applications yet by design are hard to adapt, tune to a specific\ndomain, or build on top of. In this work, we introduce a method to improve such\nsystems via automatic pre-processing (APP) using sentence simplification. We\nfirst propose a method to automatically generate a large in-domain paraphrase\ncorpus through back-translation with a black-box MT system, which is used to\ntrain a paraphrase model that \"simplifies\" the original sentence to be more\nconducive for translation. The model is used to preprocess source sentences of\nmultiple low-resource language pairs. We show that this preprocessing leads to\nbetter translation performance as compared to non-preprocessed source\nsentences. We further perform side-by-side human evaluation to verify that\ntranslations of the simplified sentences are better than the original ones.\nFinally, we provide some guidance on recommended language pairs for generating\nthe simplification model corpora by investigating the relationship between ease\nof translation of a language pair (as measured by BLEU) and quality of the\nresulting simplification model from back-translations of this language pair (as\nmeasured by SARI), and tie this into the downstream task of low-resource\ntranslation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11216v1",
            "date": "2020-05-22 14:49:07+00:00",
            "title": "A Generative Approach to Titling and Clustering Wikipedia Sections",
            "authors": [
                "Anjalie Field",
                "Sascha Rothe",
                "Simon Baumgartner",
                "Cong Yu",
                "Abe Ittycheriah"
            ],
            "abstract": "We evaluate the performance of transformer encoders with various decoders for\ninformation organization through a new task: generation of section headings for\nWikipedia articles. Our analysis shows that decoders containing attention\nmechanisms over the encoder output achieve high-scoring results by generating\nextractive text. In contrast, a decoder without attention better facilitates\nsemantic encoding and can be used to generate section embeddings. We\nadditionally introduce a new loss function, which further encourages the\ndecoder to generate high-quality embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11239v1",
            "date": "2020-05-22 15:40:43+00:00",
            "title": "Character-level Transformer-based Neural Machine Translation",
            "authors": [
                "Nikolay Banar",
                "Walter Daelemans",
                "Mike Kestemont"
            ],
            "abstract": "Neural machine translation (NMT) is nowadays commonly applied at the subword\nlevel, using byte-pair encoding. A promising alternative approach focuses on\ncharacter-level translation, which simplifies processing pipelines in NMT\nconsiderably. This approach, however, must consider relatively longer\nsequences, rendering the training process prohibitively expensive. In this\npaper, we discuss a novel, Transformer-based approach, that we compare, both in\nspeed and in quality to the Transformer at subword and character levels, as\nwell as previously developed character-level models. We evaluate our models on\n4 language pairs from WMT'15: DE-EN, CS-EN, FI-EN and RU-EN. The proposed novel\narchitecture can be trained on a single GPU and is 34% percent faster than the\ncharacter-level Transformer; still, the obtained results are at least on par\nwith it. In addition, our proposed model outperforms the subword-level model in\nFI-EN and shows close results in CS-EN. To stimulate further research in this\narea and close the gap with subword-level NMT, we make all our code and models\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11344v1",
            "date": "2020-05-22 18:27:28+00:00",
            "title": "The Discussion Tracker Corpus of Collaborative Argumentation",
            "authors": [
                "Christopher Olshefski",
                "Luca Lugini",
                "Ravneet Singh",
                "Diane Litman",
                "Amanda Godley"
            ],
            "abstract": "Although Natural Language Processing (NLP) research on argument mining has\nadvanced considerably in recent years, most studies draw on corpora of\nasynchronous and written texts, often produced by individuals. Few published\ncorpora of synchronous, multi-party argumentation are available. The Discussion\nTracker corpus, collected in American high school English classes, is an\nannotated dataset of transcripts of spoken, multi-party argumentation. The\ncorpus consists of 29 multi-party discussions of English literature transcribed\nfrom 985 minutes of audio. The transcripts were annotated for three dimensions\nof collaborative argumentation: argument moves (claims, evidence, and\nexplanations), specificity (low, medium, high) and collaboration (e.g.,\nextensions of and disagreements about others' ideas). In addition to providing\ndescriptive statistics on the corpus, we provide performance benchmarks and\nassociated code for predicting each dimension separately, illustrate the use of\nthe multiple annotations in the corpus to improve performance via multi-task\nlearning, and finally discuss other ways the corpus might be used to further\nNLP research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11355v1",
            "date": "2020-05-22 19:19:50+00:00",
            "title": "Towards Open Domain Event Trigger Identification using Adversarial Domain Adaptation",
            "authors": [
                "Aakanksha Naik",
                "Carolyn Ros\u00e9"
            ],
            "abstract": "We tackle the task of building supervised event trigger identification models\nwhich can generalize better across domains. Our work leverages the adversarial\ndomain adaptation (ADA) framework to introduce domain-invariance. ADA uses\nadversarial training to construct representations that are predictive for\ntrigger identification, but not predictive of the example's domain. It requires\nno labeled data from the target domain, making it completely unsupervised.\nExperiments with two domains (English literature and news) show that ADA leads\nto an average F1 score improvement of 3.9 on out-of-domain data. Our best\nperforming model (BERT-A) reaches 44-49 F1 across both domains, using no\nlabeled target data. Preliminary experiments reveal that finetuning on 1%\nlabeled data, followed by self-training leads to substantial improvement,\nreaching 51.5 and 67.2 F1 on literature and news respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11424v1",
            "date": "2020-05-22 23:41:35+00:00",
            "title": "Transformer-based Context-aware Sarcasm Detection in Conversation Threads from Social Media",
            "authors": [
                "Xiangjue Dong",
                "Changmao Li",
                "Jinho D. Choi"
            ],
            "abstract": "We present a transformer-based sarcasm detection model that accounts for the\ncontext from the entire conversation thread for more robust predictions. Our\nmodel uses deep transformer layers to perform multi-head attentions among the\ntarget utterance and the relevant context in the thread. The context-aware\nmodels are evaluated on two datasets from social media, Twitter and Reddit, and\nshow 3.1% and 7.0% improvements over their baselines. Our best models give the\nF1-scores of 79.0% and 75.0% for the Twitter and Reddit datasets respectively,\nbecoming one of the highest performing systems among 36 participants in this\nshared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11494v1",
            "date": "2020-05-23 08:56:18+00:00",
            "title": "From Witch's Shot to Music Making Bones -- Resources for Medical Laymen to Technical Language and Vice Versa",
            "authors": [
                "Laura Seiffe",
                "Oliver Marten",
                "Michael Mikhailov",
                "Sven Schmeier",
                "Sebastian M\u00f6ller",
                "Roland Roller"
            ],
            "abstract": "Many people share information in social media or forums, like food they eat,\nsports activities they do or events which have been visited. This also applies\nto information about a person's health status. Information we share online\nunveils directly or indirectly information about our lifestyle and health\nsituation and thus provides a valuable data resource. If we can make advantage\nof that data, applications can be created that enable e.g. the detection of\npossible risk factors of diseases or adverse drug reactions of medications.\nHowever, as most people are not medical experts, language used might be more\ndescriptive rather than the precise medical expression as medics do. To detect\nand use those relevant information, laymen language has to be translated and/or\nlinked to the corresponding medical concept. This work presents baseline data\nsources in order to address this challenge for German. We introduce a new data\nset which annotates medical laymen and technical expressions in a patient\nforum, along with a set of medical synonyms and definitions, and present first\nbaseline results on the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11694v1",
            "date": "2020-05-24 09:20:34+00:00",
            "title": "Integrated Node Encoder for Labelled Textual Networks",
            "authors": [
                "Ye Ma",
                "Lu Zong"
            ],
            "abstract": "Voluminous works have been implemented to exploit content-enhanced network\nembedding models, with little focus on the labelled information of nodes.\nAlthough TriDNR leverages node labels by treating them as node attributes, it\nfails to enrich unlabelled node vectors with the labelled information, which\nleads to the weaker classification result on the test set in comparison to\nexisting unsupervised textual network embedding models. In this study, we\ndesign an integrated node encoder (INE) for textual networks which is jointly\ntrained on the structure-based and label-based objectives. As a result, the\nnode encoder preserves the integrated knowledge of not only the network text\nand structure, but also the labelled information. Furthermore, INE allows the\ncreation of label-enhanced vectors for unlabelled nodes by entering their node\ncontents. Our node embedding achieves state-of-the-art performances in the\nclassification task on two public citation networks, namely Cora and DBLP,\npushing benchmarks up by 10.0\\% and 12.1\\%, respectively, with the 70\\%\ntraining ratio. Additionally, a feasible solution that generalizes our model\nfrom textual networks to a broader range of networks is proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11700v1",
            "date": "2020-05-24 09:29:36+00:00",
            "title": "When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications",
            "authors": [
                "Zequn Liu",
                "Ruiyi Zhang",
                "Yiping Song",
                "Ming Zhang"
            ],
            "abstract": "Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method,\nis successfully employed in NLP applications including few-shot text\nclassification and multi-domain low-resource language generation. Many\nimpacting factors, including data quantity, similarity among tasks, and the\nbalance between general language model and task-specific adaptation, can affect\nthe performance of MAML in NLP, but few works have thoroughly studied them. In\nthis paper, we conduct an empirical study to investigate these impacting\nfactors and conclude when MAML works the best based on the experimental\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11706v1",
            "date": "2020-05-24 10:01:27+00:00",
            "title": "A Novel Distributed Representation of News (DRNews) for Stock Market Predictions",
            "authors": [
                "Ye Ma",
                "Lu Zong",
                "Peiwan Wang"
            ],
            "abstract": "In this study, a novel Distributed Representation of News (DRNews) model is\ndeveloped and applied in deep learning-based stock market predictions. With the\nmerit of integrating contextual information and cross-documental knowledge, the\nDRNews model creates news vectors that describe both the semantic information\nand potential linkages among news events through an attributed news network.\nTwo stock market prediction tasks, namely the short-term stock movement\nprediction and stock crises early warning, are implemented in the framework of\nthe attention-based Long Short Term-Memory (LSTM) network. It is suggested that\nDRNews substantially enhances the results of both tasks comparing with five\nbaselines of news embedding models. Further, the attention mechanism suggests\nthat short-term stock trend and stock market crises both receive influences\nfrom daily news with the former demonstrates more critical responses on the\ninformation related to the stock market {\\em per se}, whilst the latter draws\nmore concerns on the banking sector and economic policies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11768v2",
            "date": "2020-05-24 15:09:21+00:00",
            "title": "KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for Comprehension And Generation",
            "authors": [
                "Jiajing Wan",
                "Xinting Huang"
            ],
            "abstract": "This paper presents our strategies in SemEval 2020 Task 4: Commonsense\nValidation and Explanation. We propose a novel way to search for evidence and\nchoose the different large-scale pre-trained models as the backbone for three\nsubtasks. The results show that our evidence-searching approach improves model\nperformance on commonsense explanation task. Our team ranks 2nd in subtask C\naccording to human evaluation score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11787v2",
            "date": "2020-05-24 15:49:57+00:00",
            "title": "Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers",
            "authors": [
                "Anne Lauscher",
                "Olga Majewska",
                "Leonardo F. R. Ribeiro",
                "Iryna Gurevych",
                "Nikolai Rozanov",
                "Goran Glava\u0161"
            ],
            "abstract": "Following the major success of neural language models (LMs) such as BERT or\nGPT-2 on a variety of language understanding tasks, recent work focused on\ninjecting (structured) knowledge from external resources into these models.\nWhile on the one hand, joint pretraining (i.e., training from scratch, adding\nobjectives based on external knowledge to the primary LM objective) may be\nprohibitively computationally expensive, post-hoc fine-tuning on external\nknowledge, on the other hand, may lead to the catastrophic forgetting of\ndistributional knowledge. In this work, we investigate models for complementing\nthe distributional knowledge of BERT with conceptual knowledge from ConceptNet\nand its corresponding Open Mind Common Sense (OMCS) corpus, respectively, using\nadapter training. While overall results on the GLUE benchmark paint an\ninconclusive picture, a deeper analysis reveals that our adapter-based models\nsubstantially outperform BERT (up to 15-20 performance points) on inference\ntasks that require the type of conceptual knowledge explicitly present in\nConceptNet and OMCS. All code and experiments are open sourced under:\nhttps://github.com/wluper/retrograph .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11838v2",
            "date": "2020-05-24 20:39:00+00:00",
            "title": "How Does That Sound? Multi-Language SpokenName2Vec Algorithm Using Speech Generation and Deep Learning",
            "authors": [
                "Aviad Elyashar",
                "Rami Puzis",
                "Michael Fire"
            ],
            "abstract": "Searching for information about a specific person is an online activity\nfrequently performed by many users. In most cases, users are aided by queries\ncontaining a name and sending back to the web search engines for finding their\nwill. Typically, Web search engines provide just a few accurate results\nassociated with a name-containing query. Currently, most solutions for\nsuggesting synonyms in online search are based on pattern matching and phonetic\nencoding, however very often, the performance of such solutions is less than\noptimal. In this paper, we propose SpokenName2Vec, a novel and generic approach\nwhich addresses the similar name suggestion problem by utilizing automated\nspeech generation, and deep learning to produce spoken name embeddings. This\nsophisticated and innovative embeddings captures the way people pronounce names\nin any language and accent. Utilizing the name pronunciation can be helpful for\nboth differentiating and detecting names that sound alike, but are written\ndifferently. The proposed approach was demonstrated on a large-scale dataset\nconsisting of 250,000 forenames and evaluated using a machine learning\nclassifier and 7,399 names with their verified synonyms. The performance of the\nproposed approach was found to be superior to 10 other algorithms evaluated in\nthis study, including well used phonetic and string similarity algorithms, and\ntwo recently proposed algorithms. The results obtained suggest that the\nproposed approach could serve as a useful and valuable tool for solving the\nsimilar name suggestion problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11849v2",
            "date": "2020-05-24 22:13:24+00:00",
            "title": "Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model",
            "authors": [
                "Satoru Katsumata",
                "Mamoru Komachi"
            ],
            "abstract": "Studies on grammatical error correction (GEC) have reported the effectiveness\nof pretraining a Seq2Seq model with a large amount of pseudodata. However, this\napproach requires time-consuming pretraining for GEC because of the size of the\npseudodata. In this study, we explore the utility of bidirectional and\nauto-regressive transformers (BART) as a generic pretrained encoder-decoder\nmodel for GEC. With the use of this generic pretrained model for GEC, the\ntime-consuming pretraining can be eliminated. We find that monolingual and\nmultilingual BART models achieve high performance in GEC, with one of the\nresults being comparable to the current strong results in English GEC. Our\nimplementations are publicly available at GitHub\n(https://github.com/Katsumata420/generic-pretrained-GEC).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11882v2",
            "date": "2020-05-25 01:37:31+00:00",
            "title": "Sentiment Analysis: Automatically Detecting Valence, Emotions, and Other Affectual States from Text",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "Recent advances in machine learning have led to computer systems that are\nhuman-like in behaviour. Sentiment analysis, the automatic determination of\nemotions in text, is allowing us to capitalize on substantial previously\nunattainable opportunities in commerce, public health, government policy,\nsocial sciences, and art. Further, analysis of emotions in text, from news to\nsocial media posts, is improving our understanding of not just how people\nconvey emotions through language but also how emotions shape our behaviour.\nThis article presents a sweeping overview of sentiment analysis research that\nincludes: the origins of the field, the rich landscape of tasks, challenges, a\nsurvey of the methods and resources used, and applications. We also discuss\ndiscuss how, without careful fore-thought, sentiment analysis has the potential\nfor harmful outcomes. We outline the latest lines of research in pursuit of\nfairness in sentiment analysis.",
            "journal ref": "Second Edition of Emotion Measurement, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11996v2",
            "date": "2020-05-25 09:27:31+00:00",
            "title": "Pointwise Paraphrase Appraisal is Potentially Problematic",
            "authors": [
                "Hannah Chen",
                "Yangfeng Ji",
                "David Evans"
            ],
            "abstract": "The prevailing approach for training and evaluating paraphrase identification\nmodels is constructed as a binary classification problem: the model is given a\npair of sentences, and is judged by how accurately it classifies pairs as\neither paraphrases or non-paraphrases. This pointwise-based evaluation method\ndoes not match well the objective of most real world applications, so the goal\nof our work is to understand how models which perform well under pointwise\nevaluation may fail in practice and find better methods for evaluating\nparaphrase identification models. As a first step towards that goal, we show\nthat although the standard way of fine-tuning BERT for paraphrase\nidentification by pairing two sentences as one sequence results in a model with\nstate-of-the-art performance, that model may perform poorly on simple tasks\nlike identifying pairs with two identical sentences. Moreover, we show that\nthese models may even predict a pair of randomly-selected sentences with higher\nparaphrase score than a pair of identical ones.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics: Student Research Workshop (2020) 150-155",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12040v1",
            "date": "2020-05-25 11:34:54+00:00",
            "title": "Knowledge Graph Simple Question Answering for Unseen Domains",
            "authors": [
                "Georgios Sidiropoulos",
                "Nikos Voskarides",
                "Evangelos Kanoulas"
            ],
            "abstract": "Knowledge graph simple question answering (KGSQA), in its standard form, does\nnot take into account that human-curated question answering training data only\ncover a small subset of the relations that exist in a Knowledge Graph (KG), or\neven worse, that new domains covering unseen and rather different to existing\ndomains relations are added to the KG. In this work, we study KGSQA in a\npreviously unstudied setting where new, unseen domains are added during test\ntime. In this setting, question-answer pairs of the new domain do not appear\nduring training, thus making the task more challenging. We propose a\ndata-centric domain adaptation framework that consists of a KGSQA system that\nis applicable to new domains, and a sequence to sequence question generation\nmethod that automatically generates question-answer pairs for the new domain.\nSince the effectiveness of question generation for KGSQA can be restricted by\nthe limited lexical variety of the generated questions, we use distant\nsupervision to extract a set of keywords that express each relation of the\nunseen domain and incorporate those in the question generation method.\nExperimental results demonstrate that our framework significantly improves over\nzero-shot baselines and is robust across domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12078v2",
            "date": "2020-05-25 12:38:47+00:00",
            "title": "Happy Are Those Who Grade without Seeing: A Multi-Task Learning Approach to Grade Essays Using Gaze Behaviour",
            "authors": [
                "Sandeep Mathias",
                "Rudra Murthy",
                "Diptesh Kanojia",
                "Abhijit Mishra",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "The gaze behaviour of a reader is helpful in solving several NLP tasks such\nas automatic essay grading. However, collecting gaze behaviour from readers is\ncostly in terms of time and money. In this paper, we propose a way to improve\nautomatic essay grading using gaze behaviour, which is learnt at run time using\na multi-task learning framework. To demonstrate the efficacy of this multi-task\nlearning based approach to automatic essay grading, we collect gaze behaviour\nfor 48 essays across 4 essay sets, and learn gaze behaviour for the rest of the\nessays, numbering over 7000 essays. Using the learnt gaze behaviour, we can\nachieve a statistically significant improvement in performance over the\nstate-of-the-art system for the essay sets where we have gaze data. We also\nachieve a statistically significant improvement for 4 other essay sets,\nnumbering about 6000 essays, where we have no gaze behaviour data available.\nOur approach establishes that learning gaze behaviour improves automatic essay\ngrading.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12086v3",
            "date": "2020-05-25 13:04:54+00:00",
            "title": "Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfer",
            "authors": [
                "Joosung Lee"
            ],
            "abstract": "Text style transfer is the task that generates a sentence by preserving the\ncontent of the input sentence and transferring the style. Most existing studies\nare progressing on non-parallel datasets because parallel datasets are limited\nand hard to construct. In this work, we introduce a method that follows two\nstages in non-parallel datasets. The first stage is to delete attribute markers\nof a sentence directly through a classifier. The second stage is to generate a\ntransferred sentence by combining the content tokens and the target style. We\nexperiment on two benchmark datasets and evaluate context, style, fluency, and\nsemantic. It is difficult to select the best system using only these automatic\nmetrics, but it is possible to select stable systems. We consider only robust\nsystems in all automatic evaluation metrics to be the minimum conditions that\ncan be used in real applications. Many previous systems are difficult to use in\ncertain situations because performance is significantly lower in several\nevaluation metrics. However, our system is stable in all automatic evaluation\nmetrics and has results comparable to other models. Also, we compare the\nperformance results of our system and the unstable system through human\nevaluation. Our code and data are available at the link\n(https://github.com/rungjoo/Stable-Style-Transformer).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12094v2",
            "date": "2020-05-25 13:17:09+00:00",
            "title": "K\u00f8psala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding",
            "authors": [
                "Daniel Hershcovich",
                "Miryam de Lhoneux",
                "Artur Kulmizev",
                "Elham Pejhan",
                "Joakim Nivre"
            ],
            "abstract": "We present K{\\o}psala, the Copenhagen-Uppsala system for the Enhanced\nUniversal Dependencies Shared Task at IWPT 2020. Our system is a pipeline\nconsisting of off-the-shelf models for everything but enhanced graph parsing,\nand for the latter, a transition-based graph parser adapted from Che et al.\n(2019). We train a single enhanced parser model per language, using gold\nsentence splitting and tokenization for training, and rely only on tokenized\nsurface forms and multilingual BERT for encoding. While a bug introduced just\nbefore submission resulted in a severe drop in precision, its post-submission\nfix would bring us to 4th place in the official ranking, according to average\nELAS. Our parser demonstrates that a unified pipeline is effective for both\nMeaning Representation Parsing and Enhanced Universal Dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12116v1",
            "date": "2020-05-25 13:56:03+00:00",
            "title": "NILE : Natural Language Inference with Faithful Natural Language Explanations",
            "authors": [
                "Sawan Kumar",
                "Partha Talukdar"
            ],
            "abstract": "The recent growth in the popularity and success of deep learning models on\nNLP classification tasks has accompanied the need for generating some form of\nnatural language explanation of the predicted labels. Such generated natural\nlanguage (NL) explanations are expected to be faithful, i.e., they should\ncorrelate well with the model's internal decision making. In this work, we\nfocus on the task of natural language inference (NLI) and address the following\nquestion: can we build NLI systems which produce labels with high accuracy,\nwhile also generating faithful explanations of its decisions? We propose\nNatural-language Inference over Label-specific Explanations (NILE), a novel NLI\nmethod which utilizes auto-generated label-specific NL explanations to produce\nlabels along with its faithful explanation. We demonstrate NILE's effectiveness\nover previously reported methods through automated and human evaluation of the\nproduced labels and explanations. Our evaluation of NILE also supports the\nclaim that accurate systems capable of providing testable explanations of their\ndecisions can be designed. We discuss the faithfulness of NILE's explanations\nin terms of sensitivity of the decisions to the corresponding explanations. We\nargue that explicit evaluation of faithfulness, in addition to label and\nexplanation accuracy, is an important step in evaluating model's explanations.\nFurther, we demonstrate that task-specific probes are necessary to establish\nsuch sensitivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12142v1",
            "date": "2020-05-25 14:41:28+00:00",
            "title": "An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering",
            "authors": [
                "Chia-Chih Kuo",
                "Shang-Bao Luo",
                "Kuan-Yu Chen"
            ],
            "abstract": "In a spoken multiple-choice question answering (SMCQA) task, given a passage,\na question, and multiple choices all in the form of speech, the machine needs\nto pick the correct choice to answer the question. While the audio could\ncontain useful cues for SMCQA, usually only the auto-transcribed text is\nutilized in system development. Thanks to the large-scaled pre-trained language\nrepresentation models, such as the bidirectional encoder representations from\ntransformers (BERT), systems with only auto-transcribed text can still achieve\na certain level of performance. However, previous studies have evidenced that\nacoustic-level statistics can offset text inaccuracies caused by the automatic\nspeech recognition systems or representation inadequacy lurking in word\nembedding generators, thereby making the SMCQA system robust. Along the line of\nresearch, this study concentrates on designing a BERT-based SMCQA framework,\nwhich not only inherits the advantages of contextualized language\nrepresentations learned by BERT, but integrates the complementary\nacoustic-level information distilled from audio with the text-level\ninformation. Consequently, an audio-enriched BERT-based SMCQA framework is\nproposed. A series of experiments demonstrates remarkable improvements in\naccuracy over selected baselines and SOTA systems on a published Chinese SMCQA\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12143v1",
            "date": "2020-05-25 14:42:26+00:00",
            "title": "Adapting End-to-End Speech Recognition for Readable Subtitles",
            "authors": [
                "Danni Liu",
                "Jan Niehues",
                "Gerasimos Spanakis"
            ],
            "abstract": "Automatic speech recognition (ASR) systems are primarily evaluated on\ntranscription accuracy. However, in some use cases such as subtitling, verbatim\ntranscription would reduce output readability given limited screen size and\nreading time. Therefore, this work focuses on ASR with output compression, a\ntask challenging for supervised approaches due to the scarcity of training\ndata. We first investigate a cascaded system, where an unsupervised compression\nmodel is used to post-edit the transcribed speech. We then compare several\nmethods of end-to-end speech recognition under output length constraints. The\nexperiments show that with limited data far less than needed for training a\nmodel from scratch, we can adapt a Transformer-based ASR model to incorporate\nboth transcription and compression capabilities. Furthermore, the best\nperformance in terms of WER and ROUGE scores is achieved by explicitly modeling\nthe length constraints within the end-to-end ASR system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12240v1",
            "date": "2020-05-25 17:26:02+00:00",
            "title": "A review of sentiment analysis research in Arabic language",
            "authors": [
                "Oumaima Oueslati",
                "Erik Cambria",
                "Moez Ben HajHmida",
                "Habib Ounelli"
            ],
            "abstract": "Sentiment analysis is a task of natural language processing which has\nrecently attracted increasing attention. However, sentiment analysis research\nhas mainly been carried out for the English language. Although Arabic is\nramping up as one of the most used languages on the Internet, only a few\nstudies have focused on Arabic sentiment analysis so far. In this paper, we\ncarry out an in-depth qualitative study of the most important research works in\nthis context by presenting limits and strengths of existing approaches. In\nparticular, we survey both approaches that leverage machine translation or\ntransfer learning to adapt English resources to Arabic and approaches that stem\ndirectly from the Arabic language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12246v1",
            "date": "2020-05-25 17:43:22+00:00",
            "title": "Demoting Racial Bias in Hate Speech Detection",
            "authors": [
                "Mengzhou Xia",
                "Anjalie Field",
                "Yulia Tsvetkov"
            ],
            "abstract": "In current hate speech datasets, there exists a high correlation between\nannotators' perceptions of toxicity and signals of African American English\n(AAE). This bias in annotated training data and the tendency of machine\nlearning models to amplify it cause AAE text to often be mislabeled as\nabusive/offensive/hate speech with a high false positive rate by current hate\nspeech classifiers. In this paper, we use adversarial training to mitigate this\nbias, introducing a hate speech classifier that learns to detect toxic\nsentences while demoting confounds corresponding to AAE texts. Experimental\nresults on a hate speech dataset and an AAE dataset suggest that our method is\nable to substantially reduce the false positive rate for AAE text while only\nminimally affecting the performance of hate speech classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12398v1",
            "date": "2020-05-25 20:54:23+00:00",
            "title": "The Unreasonable Volatility of Neural Machine Translation Models",
            "authors": [
                "Marzieh Fadaee",
                "Christof Monz"
            ],
            "abstract": "Recent works have shown that Neural Machine Translation (NMT) models achieve\nimpressive performance, however, questions about understanding the behavior of\nthese models remain unanswered. We investigate the unexpected volatility of NMT\nmodels where the input is semantically and syntactically correct. We discover\nthat with trivial modifications of source sentences, we can identify cases\nwhere \\textit{unexpected changes} happen in the translation and in the worst\ncase lead to mistranslations. This volatile behavior of translating extremely\nsimilar sentences in surprisingly different ways highlights the underlying\ngeneralization problem of current NMT models. We find that both RNN and\nTransformer models display volatile behavior in 26% and 19% of sentence\nvariations, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12443v1",
            "date": "2020-05-25 23:44:19+00:00",
            "title": "MaintNet: A Collaborative Open-Source Library for Predictive Maintenance Language Resources",
            "authors": [
                "Farhad Akhbardeh",
                "Travis Desell",
                "Marcos Zampieri"
            ],
            "abstract": "Maintenance record logbooks are an emerging text type in NLP. They typically\nconsist of free text documents with many domain specific technical terms,\nabbreviations, as well as non-standard spelling and grammar, which poses\ndifficulties to NLP pipelines trained on standard corpora. Analyzing and\nannotating such documents is of particular importance in the development of\npredictive maintenance systems, which aim to provide operational efficiencies,\nprevent accidents and save lives. In order to facilitate and encourage research\nin this area, we have developed MaintNet, a collaborative open-source library\nof technical and domain-specific language datasets. MaintNet provides novel\nlogbook data from the aviation, automotive, and facilities domains along with\ntools to aid in their (pre-)processing and clustering. Furthermore, it provides\na way to encourage discussion on and sharing of new datasets and tools for\nlogbook data analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12484v2",
            "date": "2020-05-26 02:21:31+00:00",
            "title": "Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading",
            "authors": [
                "Yifan Gao",
                "Chien-Sheng Wu",
                "Shafiq Joty",
                "Caiming Xiong",
                "Richard Socher",
                "Irwin King",
                "Michael R. Lyu",
                "Steven C. H. Hoi"
            ],
            "abstract": "The goal of conversational machine reading is to answer user questions given\na knowledge base text which may require asking clarification questions.\nExisting approaches are limited in their decision making due to struggles in\nextracting question-related rules and reasoning about them. In this paper, we\npresent a new framework of conversational machine reading that comprises a\nnovel Explicit Memory Tracker (EMT) to track whether conditions listed in the\nrule text have already been satisfied to make a decision. Moreover, our\nframework generates clarification questions by adopting a coarse-to-fine\nreasoning strategy, utilizing sentence-level entailment scores to weight\ntoken-level distributions. On the ShARC benchmark (blind, held-out) testset,\nEMT achieves new state-of-the-art results of 74.6% micro-averaged decision\naccuracy and 49.5 BLEU4. We also show that EMT is more interpretable by\nvisualizing the entailment-oriented reasoning process as the conversation\nflows. Code and models are released at\nhttps://github.com/Yifan-Gao/explicit_memory_tracker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12515v2",
            "date": "2020-05-26 05:05:32+00:00",
            "title": "ParsBERT: Transformer-based Model for Persian Language Understanding",
            "authors": [
                "Mehrdad Farahani",
                "Mohammad Gharachorloo",
                "Marzieh Farahani",
                "Mohammad Manthouri"
            ],
            "abstract": "The surge of pre-trained language models has begun a new era in the field of\nNatural Language Processing (NLP) by allowing us to build powerful language\nmodels. Among these models, Transformer-based models such as BERT have become\nincreasingly popular due to their state-of-the-art performance. However, these\nmodels are usually focused on English, leaving other languages to multilingual\nmodels with limited resources. This paper proposes a monolingual BERT for the\nPersian language (ParsBERT), which shows its state-of-the-art performance\ncompared to other architectures and multilingual models. Also, since the amount\nof data available for NLP tasks in Persian is very restricted, a massive\ndataset for different NLP tasks as well as pre-training the model is composed.\nParsBERT obtains higher scores in all datasets, including existing ones as well\nas composed ones and improves the state-of-the-art performance by outperforming\nboth multilingual BERT and other prior works in Sentiment Analysis, Text\nClassification and Named Entity Recognition tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12522v2",
            "date": "2020-05-26 05:41:58+00:00",
            "title": "What Are People Asking About COVID-19? A Question Classification Dataset",
            "authors": [
                "Jerry Wei",
                "Chengyu Huang",
                "Soroush Vosoughi",
                "Jason Wei"
            ],
            "abstract": "We present COVID-Q, a set of 1,690 questions about COVID-19 from 13 sources,\nwhich we annotate into 15 question categories and 207 question clusters. The\nmost common questions in our dataset asked about transmission, prevention, and\nsocietal effects of COVID, and we found that many questions that appeared in\nmultiple sources were not answered by any FAQ websites of reputable\norganizations such as the CDC and FDA. We post our dataset publicly at\nhttps://github.com/JerryWei03/COVID-Q. For classifying questions into 15\ncategories, a BERT baseline scored 58.1% accuracy when trained on 20 examples\nper category, and for a question clustering task, a BERT + triplet loss\nbaseline achieved 49.5% accuracy. We hope COVID-Q can help either for direct\nuse in developing applied systems or as a domain-specific resource for model\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12696v3",
            "date": "2020-05-26 13:17:36+00:00",
            "title": "Generating Semantically Valid Adversarial Questions for TableQA",
            "authors": [
                "Yi Zhu",
                "Yiwei Zhou",
                "Menglin Xia"
            ],
            "abstract": "Adversarial attack on question answering systems over tabular data (TableQA)\ncan help evaluate to what extent they can understand natural language questions\nand reason with tables. However, generating natural language adversarial\nquestions is difficult, because even a single character swap could lead to huge\nsemantic difference in human perception. In this paper, we propose SAGE\n(Semantically valid Adversarial GEnerator), a Wasserstein sequence-to-sequence\nmodel for TableQA white-box attack. To preserve meaning of original questions,\nwe apply minimum risk training with SIMILE and entity delexicalization. We use\nGumbel-Softmax to incorporate adversarial loss for end-to-end training. Our\nexperiments show that SAGE outperforms existing local attack models on semantic\nvalidity and fluency while achieving a good attack success rate. Finally, we\ndemonstrate that adversarial training with SAGE augmented data can improve\nperformance and robustness of TableQA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12889v4",
            "date": "2020-05-26 17:24:15+00:00",
            "title": "Refining Implicit Argument Annotation for UCCA",
            "authors": [
                "Ruixiang Cui",
                "Daniel Hershcovich"
            ],
            "abstract": "Predicate-argument structure analysis is a central component in meaning\nrepresentations of text. The fact that some arguments are not explicitly\nmentioned in a sentence gives rise to ambiguity in language understanding, and\nrenders it difficult for machines to interpret text correctly. However, only\nfew resources represent implicit roles for NLU, and existing studies in NLP\nonly make coarse distinctions between categories of arguments omitted from\nlinguistic form. This paper proposes a typology for fine-grained implicit\nargument annotation on top of Universal Conceptual Cognitive Annotation's\nfoundational layer. The proposed implicit argument categorisation is driven by\ntheories of implicit role interpretation and consists of six types: Deictic,\nGeneric, Genre-based, Type-identifiable, Non-specific, and Iterated-set. We\nexemplify our design by revisiting part of the UCCA EWT corpus, providing a new\ndataset annotated with the refinement layer, and making a comparative analysis\nwith other schemes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12898v1",
            "date": "2020-05-26 17:46:46+00:00",
            "title": "Analysis of the Penn Korean Universal Dependency Treebank (PKT-UD): Manual Revision to Build Robust Parsing Model in Korean",
            "authors": [
                "Tae Hwan Oh",
                "Ji Yoon Han",
                "Hyonsu Choe",
                "Seokwon Park",
                "Han He",
                "Jinho D. Choi",
                "Na-Rae Han",
                "Jena D. Hwang",
                "Hansaem Kim"
            ],
            "abstract": "In this paper, we first open on important issues regarding the Penn Korean\nUniversal Treebank (PKT-UD) and address these issues by revising the entire\ncorpus manually with the aim of producing cleaner UD annotations that are more\nfaithful to Korean grammar. For compatibility to the rest of UD corpora, we\nfollow the UDv2 guidelines, and extensively revise the part-of-speech tags and\nthe dependency relations to reflect morphological features and flexible\nword-order aspects in Korean. The original and the revised versions of PKT-UD\nare experimented with transformer-based parsing models using biaffine\nattention. The parsing model trained on the revised corpus shows a significant\nimprovement of 3.0% in labeled attachment score over the model trained on the\nprevious corpus. Our error analysis demonstrates that this revision allows the\nparsing model to learn relations more robustly, reducing several critical\nerrors that used to be made by the previous model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13013v2",
            "date": "2020-05-26 20:17:29+00:00",
            "title": "English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too",
            "authors": [
                "Jason Phang",
                "Iacer Calixto",
                "Phu Mon Htut",
                "Yada Pruksachatkun",
                "Haokun Liu",
                "Clara Vania",
                "Katharina Kann",
                "Samuel R. Bowman"
            ],
            "abstract": "Intermediate-task training---fine-tuning a pretrained model on an\nintermediate task before fine-tuning again on the target task---often improves\nmodel performance substantially on language understanding tasks in monolingual\nEnglish settings. We investigate whether English intermediate-task training is\nstill helpful on non-English target tasks. Using nine intermediate\nlanguage-understanding tasks, we evaluate intermediate-task transfer in a\nzero-shot cross-lingual setting on the XTREME benchmark. We see large\nimprovements from intermediate training on the BUCC and Tatoeba sentence\nretrieval tasks and moderate improvements on question-answering target tasks.\nMNLI, SQuAD and HellaSwag achieve the best overall results as intermediate\ntasks, while multi-task intermediate offers small additional improvements.\nUsing our best intermediate-task models for each target task, we obtain a 5.4\npoint improvement over XLM-R Large on the XTREME benchmark, setting the state\nof the art as of June 2020. We also investigate continuing multilingual MLM\nduring intermediate-task training and using machine-translated\nintermediate-task data, but neither consistently outperforms simply performing\nEnglish intermediate-task training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13084v1",
            "date": "2020-05-26 23:41:05+00:00",
            "title": "Learning with Weak Supervision for Email Intent Detection",
            "authors": [
                "Kai Shu",
                "Subhabrata Mukherjee",
                "Guoqing Zheng",
                "Ahmed Hassan Awadallah",
                "Milad Shokouhi",
                "Susan Dumais"
            ],
            "abstract": "Email remains one of the most frequently used means of online communication.\nPeople spend a significant amount of time every day on emails to exchange\ninformation, manage tasks and schedule events. Previous work has studied\ndifferent ways for improving email productivity by prioritizing emails,\nsuggesting automatic replies or identifying intents to recommend appropriate\nactions. The problem has been mostly posed as a supervised learning problem\nwhere models of different complexities were proposed to classify an email\nmessage into a predefined taxonomy of intents or classes. The need for labeled\ndata has always been one of the largest bottlenecks in training supervised\nmodels. This is especially the case for many real-world tasks, such as email\nintent classification, where large scale annotated examples are either hard to\nacquire or unavailable due to privacy or data access constraints. Email users\noften take actions in response to intents expressed in an email (e.g., setting\nup a meeting in response to an email with a scheduling request). Such actions\ncan be inferred from user interaction logs. In this paper, we propose to\nleverage user actions as a source of weak supervision, in addition to a limited\nset of annotated examples, to detect intents in emails. We develop an\nend-to-end robust deep neural network model for email intent identification\nthat leverages both clean annotated data and noisy weak supervision along with\na self-paced learning mechanism. Extensive experiments on three different\nintent detection tasks show that our approach can effectively leverage the\nweakly supervised data to improve intent detection in emails.",
            "journal ref": "ACM SIGIR 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13119v1",
            "date": "2020-05-27 01:48:54+00:00",
            "title": "Should Answer Immediately or Wait for Further Information? A Novel Wait-or-Answer Task and Its Predictive Approach",
            "authors": [
                "Zehao Lin",
                "Shaobo Cui",
                "Xiaoming Kang",
                "Guodun Li",
                "Feng Ji",
                "Haiqing Chen",
                "Yin Zhang"
            ],
            "abstract": "Different people have different habits of describing their intents in\nconversations. Some people may tend to deliberate their full intents in several\nsuccessive utterances, i.e., they use several consistent messages for\nreadability instead of a long sentence to express their question. This creates\na predicament faced by dialogue systems' application, especially in real-world\nindustrial scenarios, in which the dialogue system is unsure that whether it\nshould answer the user's query immediately or wait for users' further\nsupplementary input. Motivated by such interesting quandary, we define a novel\ntask: Wait-or-Answer to better tackle this dilemma faced by dialogue systems.\nWe shed light on a new research topic about how the dialogue system can be more\ncompetent to behave in this Wait-or-Answer quandary. Further, we propose a\npredictive approach dubbed Imagine-then-Arbitrate (ITA) to resolve this\nWait-or-Answer task. More specifically, we take advantage of an arbitrator\nmodel to help the dialogue system decide to wait or answer. The arbitrator's\ndecision is made with the assistance of two ancillary imaginator models: a wait\nimaginator and an answer imaginator. The wait imaginator tries to predict what\nthe user would supplement and use its prediction to persuade the arbitrator\nthat the user has some information to add, so the dialogue system should wait.\nThe answer imaginator, nevertheless, struggles to predict the answer of the\ndialogue system and convince the arbitrator that it's a superior choice to\nanswer the users' query immediately. To our best knowledge, our paper is the\nfirst work to explicitly define the Wait-or-Answer task in the dialogue system.\nAdditionally, our proposed ITA approach significantly outperforms the existing\nmodels in solving this Wait-or-Answer problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13156v1",
            "date": "2020-05-27 04:56:08+00:00",
            "title": "MT-Adapted Datasheets for Datasets: Template and Repository",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Roger Creus",
                "Oriol Domingo",
                "Albert Dom\u00ednguez",
                "Miquel Escobar",
                "Cayetana L\u00f3pez",
                "Marina Garcia",
                "Margarita Geleta"
            ],
            "abstract": "In this report we are taking the standardized model proposed by Gebru et al.\n(2018) for documenting the popular machine translation datasets of the EuroParl\n(Koehn, 2005) and News-Commentary (Barrault et al., 2019). Within this\ndocumentation process, we have adapted the original datasheet to the particular\ncase of data consumers within the Machine Translation area. We are also\nproposing a repository for collecting the adapted datasheets in this research\narea",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13213v1",
            "date": "2020-05-27 07:31:57+00:00",
            "title": "Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?",
            "authors": [
                "Kobi Leins",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "As part of growing NLP capabilities, coupled with an awareness of the ethical\ndimensions of research, questions have been raised about whether particular\ndatasets and tasks should be deemed off-limits for NLP research. We examine\nthis question with respect to a paper on automatic legal sentencing from EMNLP\n2019 which was a source of some debate, in asking whether the paper should have\nbeen allowed to be published, who should have been charged with making such a\ndecision, and on what basis. We focus in particular on the role of data\nstatements in ethically assessing research, but also discuss the topic of dual\nuse, and examine the outcomes of similar debates in other scientific\ndisciplines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13236v1",
            "date": "2020-05-27 08:44:09+00:00",
            "title": "Establishing a New State-of-the-Art for French Named Entity Recognition",
            "authors": [
                "Pedro Javier Ortiz Su\u00e1rez",
                "Yoann Dupont",
                "Benjamin Muller",
                "Laurent Romary",
                "Beno\u00eet Sagot"
            ],
            "abstract": "The French TreeBank developed at the University Paris 7 is the main source of\nmorphosyntactic and syntactic annotations for French. However, it does not\ninclude explicit information related to named entities, which are among the\nmost useful information for several natural language processing tasks and\napplications. Moreover, no large-scale French corpus with named entity\nannotations contain referential information, which complement the type and the\nspan of each mention with an indication of the entity it refers to. We have\nmanually annotated the French TreeBank with such information, after an\nautomatic pre-annotation step. We sketch the underlying annotation guidelines\nand we provide a few figures about the resulting annotations.",
            "journal ref": "LREC 2020 - 12th Language Resources and Evaluation Conference, May\n  2020, Marseille, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13316v1",
            "date": "2020-05-27 12:21:36+00:00",
            "title": "Tracking, exploring and analyzing recent developments in German-language online press in the face of the coronavirus crisis: cOWIDplus Analysis and cOWIDplus Viewer",
            "authors": [
                "Sascha Wolfer",
                "Alexander Koplenig",
                "Frank Michaelis",
                "Carolin M\u00fcller-Spitzer"
            ],
            "abstract": "The coronavirus pandemic may be the largest crisis the world has had to face\nsince World War II. It does not come as a surprise that it is also having an\nimpact on language as our primary communication tool. We present three\ninter-connected resources that are designed to capture and illustrate these\neffects on a subset of the German language: An RSS corpus of German-language\nnewsfeeds (with freely available untruncated unigram frequency lists), a static\nbut continuously updated HTML page tracking the diversity of the used\nvocabulary and a web application that enables other researchers and the broader\npublic to explore these effects without any or with little knowledge of corpus\nrepresentation/exploration or statistical analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13334v1",
            "date": "2020-05-27 13:01:02+00:00",
            "title": "Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Sequence-to-sequence constituent parsing requires a linearization to\nrepresent trees as sequences. Top-down tree linearizations, which can be based\non brackets or shift-reduce actions, have achieved the best accuracy to date.\nIn this paper, we show that these results can be improved by using an in-order\nlinearization instead. Based on this observation, we implement an enriched\nin-order shift-reduce linearization inspired by Vinyals et al. (2015)'s\napproach, achieving the best accuracy to date on the English PTB dataset among\nfully-supervised single-model sequence-to-sequence constituent parsers.\nFinally, we apply deterministic attention mechanisms to match the speed of\nstate-of-the-art transition-based parsers, thus showing that\nsequence-to-sequence models can match them, not only in accuracy, but also in\nspeed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13344v2",
            "date": "2020-05-27 13:18:27+00:00",
            "title": "Transition-based Semantic Dependency Parsing with Pointer Networks",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Transition-based parsers implemented with Pointer Networks have become the\nnew state of the art in dependency parsing, excelling in producing labelled\nsyntactic trees and outperforming graph-based models in this task. In order to\nfurther test the capabilities of these powerful neural networks on a harder NLP\nproblem, we propose a transition system that, thanks to Pointer Networks, can\nstraightforwardly produce labelled directed acyclic graphs and perform semantic\ndependency parsing. In addition, we enhance our approach with deep\ncontextualized word embeddings extracted from BERT. The resulting system not\nonly outperforms all existing transition-based models, but also matches the\nbest fully-supervised accuracy to date on the SemEval 2015 Task 18 English\ndatasets among previous state-of-the-art graph-based parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13421v1",
            "date": "2020-05-27 15:28:12+00:00",
            "title": "Thirty Musts for Meaning Banking",
            "authors": [
                "Johan Bos",
                "Lasha Abzianidze"
            ],
            "abstract": "Meaning banking--creating a semantically annotated corpus for the purpose of\nsemantic parsing or generation--is a challenging task. It is quite simple to\ncome up with a complex meaning representation, but it is hard to design a\nsimple meaning representation that captures many nuances of meaning. This paper\nlists some lessons learned in nearly ten years of meaning annotation during the\ndevelopment of the Groningen Meaning Bank (Bos et al., 2017) and the Parallel\nMeaning Bank (Abzianidze et al., 2017). The paper's format is rather\nunconventional: there is no explicit related work, no methodology section, no\nresults, and no discussion (and the current snippet is not an abstract but\nactually an introductory preface). Instead, its structure is inspired by work\nof Traum (2000) and Bender (2013). The list starts with a brief overview of the\nexisting meaning banks (Section 1) and the rest of the items are roughly\ndivided into three groups: corpus collection (Section 2 and 3, annotation\nmethods (Section 4-11), and design of meaning representations (Section 12-30).\nWe hope this overview will give inspiration and guidance in creating improved\nmeaning banks in the future.",
            "journal ref": "Proceedings of the First International Workshop on Designing\n  Meaning Representations, 2019, Association for Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13455v1",
            "date": "2020-05-27 16:11:09+00:00",
            "title": "Self-Training for Unsupervised Parsing with PRPN",
            "authors": [
                "Anhad Mohananey",
                "Katharina Kann",
                "Samuel R. Bowman"
            ],
            "abstract": "Neural unsupervised parsing (UP) models learn to parse without access to\nsyntactic annotations, while being optimized for another task like language\nmodeling. In this work, we propose self-training for neural UP models: we\nleverage aggregated annotations predicted by copies of our model as supervision\nfor future copies. To be able to use our model's predictions during training,\nwe extend a recent neural UP architecture, the PRPN (Shen et al., 2018a) such\nthat it can be trained in a semi-supervised fashion. We then add examples with\nparses predicted by our model to our unlabeled UP training data. Our\nself-trained model outperforms the PRPN by 8.1% F1 and the previous state of\nthe art by 1.6% F1. In addition, we show that our architecture can also be\nhelpful for semi-supervised parsing in ultra-low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13482v1",
            "date": "2020-05-27 16:44:01+00:00",
            "title": "Syntactic Structure Distillation Pretraining For Bidirectional Encoders",
            "authors": [
                "Adhiguna Kuncoro",
                "Lingpeng Kong",
                "Daniel Fried",
                "Dani Yogatama",
                "Laura Rimell",
                "Chris Dyer",
                "Phil Blunsom"
            ],
            "abstract": "Textual representation learners trained on large amounts of data have\nachieved notable success on downstream tasks; intriguingly, they have also\nperformed well on challenging tests of syntactic competence. Given this\nsuccess, it remains an open question whether scalable learners like BERT can\nbecome fully proficient in the syntax of natural language by virtue of data\nscale alone, or whether they still benefit from more explicit syntactic biases.\nTo answer this question, we introduce a knowledge distillation strategy for\ninjecting syntactic biases into BERT pretraining, by distilling the\nsyntactically informative predictions of a hierarchical---albeit harder to\nscale---syntactic language model. Since BERT models masked words in\nbidirectional context, we propose to distill the approximate marginal\ndistribution over words in context from the syntactic LM. Our approach reduces\nrelative error by 2-21% on a diverse set of structured prediction tasks,\nalthough we obtain mixed results on the GLUE benchmark. Our findings\ndemonstrate the benefits of syntactic biases, even in representation learners\nthat exploit large amounts of data, and contribute to a better understanding of\nwhere syntactic biases are most helpful in benchmarks of natural language\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13619v1",
            "date": "2020-05-27 20:01:56+00:00",
            "title": "Language Representation Models for Fine-Grained Sentiment Classification",
            "authors": [
                "Brian Cheang",
                "Bailey Wei",
                "David Kogan",
                "Howey Qiu",
                "Masud Ahmed"
            ],
            "abstract": "Sentiment classification is a quickly advancing field of study with\napplications in almost any field. While various models and datasets have shown\nhigh accuracy inthe task of binary classification, the task of fine-grained\nsentiment classification is still an area with room for significant\nimprovement. Analyzing the SST-5 dataset,previous work by Munikar et al. (2019)\nshowed that the embedding tool BERT allowed a simple model to achieve\nstate-of-the-art accuracy. Since that paper, several BERT alternatives have\nbeen published, with three primary ones being AlBERT (Lan et al., 2019),\nDistilBERT (Sanh et al. 2019), and RoBERTa (Liu etal. 2019). While these models\nreport some improvement over BERT on the popular benchmarks GLUE, SQuAD, and\nRACE, they have not been applied to the fine-grained classification task. In\nthis paper, we examine whether the improvements hold true when applied to a\nnovel task, by replicating the BERT model from Munikar et al., and swapping the\nembedding layer to the alternative models. Over the experiments, we found that\nAlBERT suffers significantly more accuracy loss than reported on other tasks,\nDistilBERT has accuracy loss similar to their reported loss on other tasks\nwhile being the fastest model to train, and RoBERTa reaches anew\nstate-of-the-art accuracy for prediction on the SST-5 root level (60.2%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13756v1",
            "date": "2020-05-28 03:09:58+00:00",
            "title": "The SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion",
            "authors": [
                "Katharina Kann",
                "Arya McCarthy",
                "Garrett Nicolai",
                "Mans Hulden"
            ],
            "abstract": "In this paper, we describe the findings of the SIGMORPHON 2020 shared task on\nunsupervised morphological paradigm completion (SIGMORPHON 2020 Task 2), a\nnovel task in the field of inflectional morphology. Participants were asked to\nsubmit systems which take raw text and a list of lemmas as input, and output\nall inflected forms, i.e., the entire morphological paradigm, of each lemma. In\norder to simulate a realistic use case, we first released data for 5\ndevelopment languages. However, systems were officially evaluated on 9 surprise\nlanguages, which were only revealed a few days before the submission deadline.\nWe provided a modular baseline system, which is a pipeline of 4 components. 3\nteams submitted a total of 7 systems, but, surprisingly, none of the submitted\nsystems was able to improve over the baseline on average over all 9 test\nlanguages. Only on 3 languages did a submitted system obtain the best results.\nThis shows that unsupervised morphological paradigm completion is still largely\nunsolved. We present an analysis here, so that this shared task will ground\nfurther research on the topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13798v1",
            "date": "2020-05-28 06:29:08+00:00",
            "title": "ConCET: Entity-Aware Topic Classification for Open-Domain Conversational Agents",
            "authors": [
                "Ali Ahmadvand",
                "Harshita Sahijwani",
                "Jason Ingyu Choi",
                "Eugene Agichtein"
            ],
            "abstract": "Identifying the topic (domain) of each user's utterance in open-domain\nconversational systems is a crucial step for all subsequent language\nunderstanding and response tasks. In particular, for complex domains, an\nutterance is often routed to a single component responsible for that domain.\nThus, correctly mapping a user utterance to the right domain is critical. To\naddress this problem, we introduce ConCET: a Concurrent Entity-aware\nconversational Topic classifier, which incorporates entity-type information\ntogether with the utterance content features. Specifically, ConCET utilizes\nentity information to enrich the utterance representation, combining character,\nword, and entity-type embeddings into a single representation. However, for\nrich domains with millions of available entities, unrealistic amounts of\nlabeled training data would be required. To complement our model, we propose a\nsimple and effective method for generating synthetic training data, to augment\nthe typically limited amounts of labeled training data, using commonly\navailable knowledge bases to generate additional labeled utterances. We\nextensively evaluate ConCET and our proposed training method first on an openly\navailable human-human conversational dataset called Self-Dialogue, to calibrate\nour approach against previous state-of-the-art methods; second, we evaluate\nConCET on a large dataset of human-machine conversations with real users,\ncollected as part of the Amazon Alexa Prize. Our results show that ConCET\nsignificantly improves topic classification performance on both datasets,\nincluding 8-10% improvements over state-of-the-art deep learning methods. We\ncomplement our quantitative results with detailed analysis of system\nperformance, which could be used for further improvements of conversational\nagents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13803v1",
            "date": "2020-05-28 06:41:18+00:00",
            "title": "Would you Like to Talk about Sports Now? Towards Contextual Topic Suggestion for Open-Domain Conversational Agents",
            "authors": [
                "Ali Ahmadvand",
                "Harshita Sahijwani",
                "Eugene Agichtein"
            ],
            "abstract": "To hold a true conversation, an intelligent agent should be able to\noccasionally take initiative and recommend the next natural conversation topic.\nThis is a challenging task. A topic suggested by the agent should be relevant\nto the person, appropriate for the conversation context, and the agent should\nhave something interesting to say about it. Thus, a scripted, or\none-size-fits-all, popularity-based topic suggestion is doomed to fail.\nInstead, we explore different methods for a personalized, contextual topic\nsuggestion for open-domain conversations. We formalize the Conversational Topic\nSuggestion problem (CTS) to more clearly identify the assumptions and\nrequirements. We also explore three possible approaches to solve this problem:\n(1) model-based sequential topic suggestion to capture the conversation context\n(CTS-Seq), (2) Collaborative Filtering-based suggestion to capture previous\nsuccessful conversations from similar users (CTS-CF), and (3) a hybrid approach\ncombining both conversation context and collaborative filtering. To evaluate\nthe effectiveness of these methods, we use real conversations collected as part\nof the Amazon Alexa Prize 2018 Conversational AI challenge. The results are\npromising: the CTS-Seq model suggests topics with 23% higher accuracy than the\nbaseline, and incorporating collaborative filtering signals into a hybrid\nCTS-Seq-CF model further improves recommendation accuracy by 12%. Together, our\nproposed models, experiments, and analysis significantly advance the study of\nopen-domain conversational agents, and suggest promising directions for future\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13804v1",
            "date": "2020-05-28 06:48:10+00:00",
            "title": "Contextual Dialogue Act Classification for Open-Domain Conversational Agents",
            "authors": [
                "Ali Ahmadvand",
                "Jason Ingyu Choi",
                "Eugene Agichtein"
            ],
            "abstract": "Classifying the general intent of the user utterance in a conversation, also\nknown as Dialogue Act (DA), e.g., open-ended question, statement of opinion, or\nrequest for an opinion, is a key step in Natural Language Understanding (NLU)\nfor conversational agents. While DA classification has been extensively studied\nin human-human conversations, it has not been sufficiently explored for the\nemerging open-domain automated conversational agents. Moreover, despite\nsignificant advances in utterance-level DA classification, full understanding\nof dialogue utterances requires conversational context. Another challenge is\nthe lack of available labeled data for open-domain human-machine conversations.\nTo address these problems, we propose a novel method, CDAC (Contextual Dialogue\nAct Classifier), a simple yet effective deep learning approach for contextual\ndialogue act classification. Specifically, we use transfer learning to adapt\nmodels trained on human-human conversations to predict dialogue acts in\nhuman-machine dialogues. To investigate the effectiveness of our method, we\ntrain our model on the well-known Switchboard human-human dialogue dataset, and\nfine-tune it for predicting dialogue acts in human-machine conversation data,\ncollected as part of the Amazon Alexa Prize 2018 competition. The results show\nthat the CDAC model outperforms an utterance-level state of the art baseline by\n8.0% on the Switchboard dataset, and is comparable to the latest reported\nstate-of-the-art contextual DA classification results. Furthermore, our results\nshow that fine-tuning the CDAC model on a small sample of manually labeled\nhuman-machine conversations allows CDAC to more accurately predict dialogue\nacts in real users' conversations, suggesting a promising direction for future\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13962v1",
            "date": "2020-05-28 13:03:51+00:00",
            "title": "A Corpus for Large-Scale Phonetic Typology",
            "authors": [
                "Elizabeth Salesky",
                "Eleanor Chodroff",
                "Tiago Pimentel",
                "Matthew Wiesner",
                "Ryan Cotterell",
                "Alan W Black",
                "Jason Eisner"
            ],
            "abstract": "A major hurdle in data-driven research on typology is having sufficient data\nin many languages to draw meaningful conclusions. We present VoxClamantis v1.0,\nthe first large-scale corpus for phonetic typology, with aligned segments and\nestimated phoneme-level labels in 690 readings spanning 635 languages, along\nwith acoustic-phonetic measures of vowels and sibilants. Access to such data\ncan greatly facilitate investigation of phonetic typology at a large scale and\nacross many languages. However, it is non-trivial and computationally intensive\nto obtain such alignments for hundreds of languages, many of which have few to\nno resources presently available. We describe the methodology to create our\ncorpus, discuss caveats with current methods and their impact on the utility of\nthis data, and illustrate possible research directions through a series of case\nstudies on the 48 highest-quality readings. Our corpus and scripts are publicly\navailable for non-commercial use at https://voxclamantisproject.github.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13978v1",
            "date": "2020-05-28 13:30:53+00:00",
            "title": "Variational Neural Machine Translation with Normalizing Flows",
            "authors": [
                "Hendra Setiawan",
                "Matthias Sperber",
                "Udhay Nallasamy",
                "Matthias Paulik"
            ],
            "abstract": "Variational Neural Machine Translation (VNMT) is an attractive framework for\nmodeling the generation of target translations, conditioned not only on the\nsource sentence but also on some latent random variables. The latent variable\nmodeling may introduce useful statistical dependencies that can improve\ntranslation accuracy. Unfortunately, learning informative latent variables is\nnon-trivial, as the latent space can be prohibitively large, and the latent\ncodes are prone to be ignored by many translation models at training time.\nPrevious works impose strong assumptions on the distribution of the latent code\nand limit the choice of the NMT architecture. In this paper, we propose to\napply the VNMT framework to the state-of-the-art Transformer and introduce a\nmore flexible approximate posterior based on normalizing flows. We demonstrate\nthe efficacy of our proposal under both in-domain and out-of-domain conditions,\nsignificantly outperforming strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14108v3",
            "date": "2020-05-28 15:58:45+00:00",
            "title": "Adversarial Attacks and Defense on Texts: A Survey",
            "authors": [
                "Aminul Huq",
                "Mst. Tasnim Pervin"
            ],
            "abstract": "Deep learning models have been used widely for various purposes in recent\nyears in object recognition, self-driving cars, face recognition, speech\nrecognition, sentiment analysis, and many others. However, in recent years it\nhas been shown that these models possess weakness to noises which force the\nmodel to misclassify. This issue has been studied profoundly in the image and\naudio domain. Very little has been studied on this issue concerning textual\ndata. Even less survey on this topic has been performed to understand different\ntypes of attacks and defense techniques. In this manuscript, we accumulated and\nanalyzed different attacking techniques and various defense models to provide a\nmore comprehensive idea. Later we point out some of the interesting findings of\nall papers and challenges that need to be overcome to move forward in this\nfield.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14165v4",
            "date": "2020-05-28 17:29:03+00:00",
            "title": "Language Models are Few-Shot Learners",
            "authors": [
                "Tom B. Brown",
                "Benjamin Mann",
                "Nick Ryder",
                "Melanie Subbiah",
                "Jared Kaplan",
                "Prafulla Dhariwal",
                "Arvind Neelakantan",
                "Pranav Shyam",
                "Girish Sastry",
                "Amanda Askell",
                "Sandhini Agarwal",
                "Ariel Herbert-Voss",
                "Gretchen Krueger",
                "Tom Henighan",
                "Rewon Child",
                "Aditya Ramesh",
                "Daniel M. Ziegler",
                "Jeffrey Wu",
                "Clemens Winter",
                "Christopher Hesse",
                "Mark Chen",
                "Eric Sigler",
                "Mateusz Litwin",
                "Scott Gray",
                "Benjamin Chess",
                "Jack Clark",
                "Christopher Berner",
                "Sam McCandlish",
                "Alec Radford",
                "Ilya Sutskever",
                "Dario Amodei"
            ],
            "abstract": "Recent work has demonstrated substantial gains on many NLP tasks and\nbenchmarks by pre-training on a large corpus of text followed by fine-tuning on\na specific task. While typically task-agnostic in architecture, this method\nstill requires task-specific fine-tuning datasets of thousands or tens of\nthousands of examples. By contrast, humans can generally perform a new language\ntask from only a few examples or from simple instructions - something which\ncurrent NLP systems still largely struggle to do. Here we show that scaling up\nlanguage models greatly improves task-agnostic, few-shot performance, sometimes\neven reaching competitiveness with prior state-of-the-art fine-tuning\napproaches. Specifically, we train GPT-3, an autoregressive language model with\n175 billion parameters, 10x more than any previous non-sparse language model,\nand test its performance in the few-shot setting. For all tasks, GPT-3 is\napplied without any gradient updates or fine-tuning, with tasks and few-shot\ndemonstrations specified purely via text interaction with the model. GPT-3\nachieves strong performance on many NLP datasets, including translation,\nquestion-answering, and cloze tasks, as well as several tasks that require\non-the-fly reasoning or domain adaptation, such as unscrambling words, using a\nnovel word in a sentence, or performing 3-digit arithmetic. At the same time,\nwe also identify some datasets where GPT-3's few-shot learning still struggles,\nas well as some datasets where GPT-3 faces methodological issues related to\ntraining on large web corpora. Finally, we find that GPT-3 can generate samples\nof news articles which human evaluators have difficulty distinguishing from\narticles written by humans. We discuss broader societal impacts of this finding\nand of GPT-3 in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14315v1",
            "date": "2020-05-28 22:06:03+00:00",
            "title": "On Incorporating Structural Information to improve Dialogue Response Generation",
            "authors": [
                "Nikita Moghe",
                "Priyesh Vijayan",
                "Balaraman Ravindran",
                "Mitesh M. Khapra"
            ],
            "abstract": "We consider the task of generating dialogue responses from background\nknowledge comprising of domain specific resources. Specifically, given a\nconversation around a movie, the task is to generate the next response based on\nbackground knowledge about the movie such as the plot, review, Reddit comments\netc. This requires capturing structural, sequential and semantic information\nfrom the conversation context and the background resources. This is a new task\nand has not received much attention from the community. We propose a new\narchitecture that uses the ability of BERT to capture deep contextualized\nrepresentations in conjunction with explicit structure and sequence\ninformation. More specifically, we use (i) Graph Convolutional Networks (GCNs)\nto capture structural information, (ii) LSTMs to capture sequential information\nand (iii) BERT for the deep contextualized representations that capture\nsemantic information. We analyze the proposed architecture extensively. To this\nend, we propose a plug-and-play Semantics-Sequences-Structures (SSS) framework\nwhich allows us to effectively combine such linguistic information. Through a\nseries of experiments we make some interesting observations. First, we observe\nthat the popular adaptation of the GCN model for NLP tasks where structural\ninformation (GCNs) was added on top of sequential information (LSTMs) performs\npoorly on our task. This leads us to explore interesting ways of combining\nsemantic and structural information to improve the performance. Second, we\nobserve that while BERT already outperforms other deep contextualized\nrepresentations such as ELMo, it still benefits from the additional structural\ninformation explicitly added using GCNs. This is a bit surprising given the\nrecent claims that BERT already captures structural information. Lastly, the\nproposed SSS framework gives an improvement of 7.95% over the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14489v1",
            "date": "2020-05-29 10:20:48+00:00",
            "title": "Neural Simultaneous Speech Translation Using Alignment-Based Chunking",
            "authors": [
                "Patrick Wilken",
                "Tamer Alkhouli",
                "Evgeny Matusov",
                "Pavel Golik"
            ],
            "abstract": "In simultaneous machine translation, the objective is to determine when to\nproduce a partial translation given a continuous stream of source words, with a\ntrade-off between latency and quality. We propose a neural machine translation\n(NMT) model that makes dynamic decisions when to continue feeding on input or\ngenerate output words. The model is composed of two main components: one to\ndynamically decide on ending a source chunk, and another that translates the\nconsumed chunk. We train the components jointly and in a manner consistent with\nthe inference conditions. To generate chunked training data, we propose a\nmethod that utilizes word alignment while also preserving enough context. We\ncompare models with bidirectional and unidirectional encoders of different\ndepths, both on real speech and text input. Our results on the IWSLT 2020\nEnglish-to-German task outperform a wait-k baseline by 2.6 to 3.7% BLEU\nabsolute.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14672v4",
            "date": "2020-05-29 16:54:50+00:00",
            "title": "Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP",
            "authors": [
                "Rob van der Goot",
                "Ahmet \u00dcst\u00fcn",
                "Alan Ramponi",
                "Ibrahim Sharaf",
                "Barbara Plank"
            ],
            "abstract": "Transfer learning, particularly approaches that combine multi-task learning\nwith pre-trained contextualized embeddings and fine-tuning, have advanced the\nfield of Natural Language Processing tremendously in recent years. In this\npaper we present MaChAmp, a toolkit for easy fine-tuning of contextualized\nembeddings in multi-task settings. The benefits of MaChAmp are its flexible\nconfiguration options, and the support of a variety of natural language\nprocessing tasks in a uniform toolkit, from text classification and sequence\nlabeling to dependency parsing, masked language modeling, and text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14690v1",
            "date": "2020-05-29 17:28:46+00:00",
            "title": "Investigating Deep Learning Approaches for Hate Speech Detection in Social Media",
            "authors": [
                "Prashant Kapil",
                "Asif Ekbal",
                "Dipankar Das"
            ],
            "abstract": "The phenomenal growth on the internet has helped in empowering individual's\nexpressions, but the misuse of freedom of expression has also led to the\nincrease of various cyber crimes and anti-social activities. Hate speech is one\nsuch issue that needs to be addressed very seriously as otherwise, this could\npose threats to the integrity of the social fabrics.\n  In this paper, we proposed deep learning approaches utilizing various\nembeddings for detecting various types of hate speeches in social media.\nDetecting hate speech from a large volume of text, especially tweets which\ncontains limited contextual information also poses several practical\nchallenges.\n  Moreover, the varieties in user-generated data and the presence of various\nforms of hate speech makes it very challenging to identify the degree and\nintention of the message. Our experiments on three publicly available datasets\nof different domains shows a significant improvement in accuracy and F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14709v1",
            "date": "2020-05-29 17:55:57+00:00",
            "title": "Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models",
            "authors": [
                "Viktor Schlegel",
                "Goran Nenadic",
                "Riza Batista-Navarro"
            ],
            "abstract": "Recent years have seen a growing number of publications that analyse Natural\nLanguage Inference (NLI) datasets for superficial cues, whether they undermine\nthe complexity of the tasks underlying those datasets and how they impact those\nmodels that are optimised and evaluated on this data. This structured survey\nprovides an overview of the evolving research area by categorising reported\nweaknesses in models and datasets and the methods proposed to reveal and\nalleviate those weaknesses for the English language. We summarise and discuss\nthe findings and conclude with a set of recommendations for possible future\nresearch directions. We hope it will be a useful resource for researchers who\npropose new datasets, to have a set of tools to assess the suitability and\nquality of their data to evaluate various phenomena of interest, as well as\nthose who develop novel architectures, to further understand the implications\nof their improvements with respect to their model's acquired capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14714v1",
            "date": "2020-05-29 17:58:19+00:00",
            "title": "The Importance of Suppressing Domain Style in Authorship Analysis",
            "authors": [
                "Sebastian Bischoff",
                "Niklas Deckers",
                "Marcel Schliebs",
                "Ben Thies",
                "Matthias Hagen",
                "Efstathios Stamatatos",
                "Benno Stein",
                "Martin Potthast"
            ],
            "abstract": "The prerequisite of many approaches to authorship analysis is a\nrepresentation of writing style. But despite decades of research, it still\nremains unclear to what extent commonly used and widely accepted\nrepresentations like character trigram frequencies actually represent an\nauthor's writing style, in contrast to more domain-specific style components or\neven topic. We address this shortcoming for the first time in a novel\nexperimental setup of fixed authors but swapped domains between training and\ntesting. With this setup, we reveal that approaches using character trigram\nfeatures are highly susceptible to favor domain information when applied\nwithout attention to domains, suffering drops of up to 55.4 percentage points\nin classification accuracy under domain swapping. We further propose a new\nremedy based on domain-adversarial learning and compare it to ones from the\nliterature based on heuristic rules. Both can work well, reducing accuracy\nlosses under domain swapping to 3.6% and 3.9%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00031v1",
            "date": "2020-05-29 18:43:22+00:00",
            "title": "A Comparative Study of Lexical Substitution Approaches based on Neural Language Models",
            "authors": [
                "Nikolay Arefyev",
                "Boris Sheludko",
                "Alexander Podolskiy",
                "Alexander Panchenko"
            ],
            "abstract": "Lexical substitution in context is an extremely powerful technology that can\nbe used as a backbone of various NLP applications, such as word sense\ninduction, lexical relation extraction, data augmentation, etc. In this paper,\nwe present a large-scale comparative study of popular neural language and\nmasked language models (LMs and MLMs), such as context2vec, ELMo, BERT, XLNet,\napplied to the task of lexical substitution. We show that already competitive\nresults achieved by SOTA LMs/MLMs can be further improved if information about\nthe target word is injected properly, and compare several target injection\nmethods. In addition, we provide analysis of the types of semantic relations\nbetween the target and substitutes generated by different models providing\ninsights into what kind of words are really generated or given by annotators as\nsubstitutes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00119v2",
            "date": "2020-05-29 23:11:48+00:00",
            "title": "Constructing Explainable Opinion Graphs from Review",
            "authors": [
                "Nofar Carmeli",
                "Xiaolan Wang",
                "Yoshihiko Suhara",
                "Stefanos Angelidis",
                "Yuliang Li",
                "Jinfeng Li",
                "Wang-Chiew Tan"
            ],
            "abstract": "The Web is a major resource of both factual and subjective information. While\nthere are significant efforts to organize factual information into knowledge\nbases, there is much less work on organizing opinions, which are abundant in\nsubjective data, into a structured format.\n  We present ExplainIt, a system that extracts and organizes opinions into an\nopinion graph, which are useful for downstream applications such as generating\nexplainable review summaries and facilitating search over opinion phrases. In\nsuch graphs, a node represents a set of semantically similar opinions extracted\nfrom reviews and an edge between two nodes signifies that one node explains the\nother. ExplainIt mines explanations in a supervised method and groups similar\nopinions together in a weakly supervised way before combining the clusters of\nopinions together with their explanation relationships into an opinion graph.\nWe experimentally demonstrate that the explanation relationships generated in\nthe opinion graph are of good quality and our labeled datasets for explanation\nmining and grouping opinions are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00148v1",
            "date": "2020-05-30 02:19:08+00:00",
            "title": "Topic Detection and Summarization of User Reviews",
            "authors": [
                "Pengyuan Li",
                "Lei Huang",
                "Guang-jie Ren"
            ],
            "abstract": "A massive amount of reviews are generated daily from various platforms. It is\nimpossible for people to read through tons of reviews and to obtain useful\ninformation. Automatic summarizing customer reviews thus is important for\nidentifying and extracting the essential information to help users to obtain\nthe gist of the data. However, as customer reviews are typically short,\ninformal, and multifaceted, it is extremely challenging to generate topic-wise\nsummarization.While there are several studies aims to solve this issue, they\nare heuristic methods that are developed only utilizing customer reviews.\nUnlike existing method, we propose an effective new summarization method by\nanalyzing both reviews and summaries.To do that, we first segment reviews and\nsummaries into individual sentiments. As the sentiments are typically short, we\ncombine sentiments talking about the same aspect into a single document and\napply topic modeling method to identify hidden topics among customer reviews\nand summaries. Sentiment analysis is employed to distinguish positive and\nnegative opinions among each detected topic. A classifier is also introduced to\ndistinguish the writing pattern of summaries and that of customer reviews.\nFinally, sentiments are selected to generate the summarization based on their\ntopic relevance, sentiment analysis score and the writing pattern. To test our\nmethod, a new dataset comprising product reviews and summaries about 1028\nproducts are collected from Amazon and CNET. Experimental results show the\neffectiveness of our method compared with other methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00184v1",
            "date": "2020-05-30 05:29:23+00:00",
            "title": "User Memory Reasoning for Conversational Recommendation",
            "authors": [
                "Hu Xu",
                "Seungwhan Moon",
                "Honglei Liu",
                "Bing Liu",
                "Pararth Shah",
                "Bing Liu",
                "Philip S. Yu"
            ],
            "abstract": "We study a conversational recommendation model which dynamically manages\nusers' past (offline) preferences and current (online) requests through a\nstructured and cumulative user memory knowledge graph, to allow for natural\ninteractions and accurate recommendations. For this study, we create a new\nMemory Graph (MG) <--> Conversational Recommendation parallel corpus called\nMGConvRex with 7K+ human-to-human role-playing dialogs, grounded on a\nlarge-scale user memory bootstrapped from real-world user scenarios. MGConvRex\ncaptures human-level reasoning over user memory and has disjoint\ntraining/testing sets of users for zero-shot (cold-start) reasoning for\nrecommendation. We propose a simple yet expandable formulation for constructing\nand updating the MG, and a reasoning model that predicts optimal dialog\npolicies and recommendation items in unconstrained graph space. The prediction\nof our proposed model inherits the graph structure, providing a natural way to\nexplain the model's recommendation. Experiments are conducted for both offline\nmetrics and online simulation, showing competitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00206v1",
            "date": "2020-05-30 07:17:27+00:00",
            "title": "Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text",
            "authors": [
                "Bharathi Raja Chakravarthi",
                "Vigneshwaran Muralidaran",
                "Ruba Priyadharshini",
                "John P. McCrae"
            ],
            "abstract": "Understanding the sentiment of a comment from a video or an image is an\nessential task in many applications. Sentiment analysis of a text can be useful\nfor various decision-making processes. One such application is to analyse the\npopular sentiments of videos on social media based on viewer comments. However,\ncomments from social media do not follow strict rules of grammar, and they\ncontain mixing of more than one language, often written in non-native scripts.\nNon-availability of annotated code-mixed data for a low-resourced language like\nTamil also adds difficulty to this problem. To overcome this, we created a gold\nstandard Tamil-English code-switched, sentiment-annotated corpus containing\n15,744 comment posts from YouTube. In this paper, we describe the process of\ncreating the corpus and assigning polarities. We present inter-annotator\nagreement and show the results of sentiment analysis trained on this corpus as\na benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00210v1",
            "date": "2020-05-30 07:32:37+00:00",
            "title": "A Sentiment Analysis Dataset for Code-Mixed Malayalam-English",
            "authors": [
                "Bharathi Raja Chakravarthi",
                "Navya Jose",
                "Shardul Suryawanshi",
                "Elizabeth Sherly",
                "John P. McCrae"
            ],
            "abstract": "There is an increasing demand for sentiment analysis of text from social\nmedia which are mostly code-mixed. Systems trained on monolingual data fail for\ncode-mixed data due to the complexity of mixing at different levels of the\ntext. However, very few resources are available for code-mixed data to create\nmodels specific for this data. Although much research in multilingual and\ncross-lingual sentiment analysis has used semi-supervised or unsupervised\nmethods, supervised methods still performs better. Only a few datasets for\npopular languages such as English-Spanish, English-Hindi, and English-Chinese\nare available. There are no resources available for Malayalam-English\ncode-mixed data. This paper presents a new gold standard corpus for sentiment\nanalysis of code-mixed text in Malayalam-English annotated by voluntary\nannotators. This gold standard corpus obtained a Krippendorff's alpha above 0.8\nfor the dataset. We use this new corpus to provide the benchmark for sentiment\nanalysis in Malayalam-English code-mixed texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00249v2",
            "date": "2020-05-30 12:23:10+00:00",
            "title": "Dynamic Masking for Improved Stability in Spoken Language Translation",
            "authors": [
                "Yuekun Yao",
                "Barry Haddow"
            ],
            "abstract": "For spoken language translation (SLT) in live scenarios such as conferences,\nlectures and meetings, it is desirable to show the translation to the user as\nquickly as possible, avoiding an annoying lag between speaker and translated\ncaptions. In other words, we would like low-latency, online SLT. If we assume a\npipeline of automatic speech recognition (ASR) and machine translation (MT)\nthen a viable approach to online SLT is to pair an online ASR system, with a a\nretranslation strategy, where the MT system re-translates every update received\nfrom ASR. However this can result in annoying \"flicker\" as the MT system\nupdates its translation. A possible solution is to add a fixed delay, or \"mask\"\nto the the output of the MT system, but a fixed global mask introduces\nundesirable latency to the output. We show how this mask can be set\ndynamically, improving the latency-flicker trade-off without sacrificing\ntranslation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00262v3",
            "date": "2020-05-30 13:28:03+00:00",
            "title": "Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings",
            "authors": [
                "Sosuke Nishikawa",
                "Ryokan Ri",
                "Yoshimasa Tsuruoka"
            ],
            "abstract": "Unsupervised cross-lingual word embedding (CLWE) methods learn a linear\ntransformation matrix that maps two monolingual embedding spaces that are\nseparately trained with monolingual corpora. This method relies on the\nassumption that the two embedding spaces are structurally similar, which does\nnot necessarily hold true in general. In this paper, we argue that using a\npseudo-parallel corpus generated by an unsupervised machine translation model\nfacilitates the structural similarity of the two embedding spaces and improves\nthe quality of CLWEs in the unsupervised mapping method. We show that our\napproach outperforms other alternative approaches given the same amount of\ndata, and, through detailed analysis, we show that data augmentation with the\npseudo data from unsupervised machine translation is especially effective for\nmapping-based CLWEs because (1) the pseudo data makes the source and target\ncorpora (partially) parallel; (2) the pseudo data contains information on the\noriginal language that helps to learn similar embedding spaces between the\nsource and target languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00418v1",
            "date": "2020-05-31 02:52:22+00:00",
            "title": "Learning to refer informatively by amortizing pragmatic reasoning",
            "authors": [
                "Julia White",
                "Jesse Mu",
                "Noah D. Goodman"
            ],
            "abstract": "A hallmark of human language is the ability to effectively and efficiently\nconvey contextually relevant information. One theory for how humans reason\nabout language is presented in the Rational Speech Acts (RSA) framework, which\ncaptures pragmatic phenomena via a process of recursive social reasoning\n(Goodman & Frank, 2016). However, RSA represents ideal reasoning in an\nunconstrained setting. We explore the idea that speakers might learn to\namortize the cost of RSA computation over time by directly optimizing for\nsuccessful communication with an internal listener model. In simulations with\ngrounded neural speakers and listeners across two communication game datasets\nrepresenting synthetic and human-generated data, we find that our amortized\nmodel is able to quickly generate language that is effective and concise across\na range of contexts, without the need for explicit pragmatic reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00490v1",
            "date": "2020-05-31 10:58:35+00:00",
            "title": "Detecting Group Beliefs Related to 2018's Brazilian Elections in Tweets A Combined Study on Modeling Topics and Sentiment Analysis",
            "authors": [
                "Brenda Salenave Santana",
                "Aline Aver Vanin"
            ],
            "abstract": "2018's Brazilian presidential elections highlighted the influence of\nalternative media and social networks, such as Twitter. In this work, we\nperform an analysis covering politically motivated discourses related to the\nsecond round in Brazilian elections. In order to verify whether similar\ndiscourses reinforce group engagement to personal beliefs, we collected a set\nof tweets related to political hashtags at that moment. To this end, we have\nused a combination of topic modeling approach with opinion mining techniques to\nanalyze the motivated political discourses. Using SentiLex-PT, a Portuguese\nsentiment lexicon, we extracted from the dataset the top 5 most frequent group\nof words related to opinions. Applying a bag-of-words model, the cosine\nsimilarity calculation was performed between each opinion and the observed\ngroups. This study allowed us to observe an exacerbated use of passionate\ndiscourses in the digital political scenario as a form of appreciation and\nengagement to the groups which convey similar beliefs.",
            "journal ref": "Proceedings of the Workshop on Digital Humanities and Natural\n  Language Processing (DHandNLP 2020) co-located with International Conference\n  on the Computational Processing of Portuguese (PROPOR 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00512v1",
            "date": "2020-05-31 12:48:37+00:00",
            "title": "Learning to Recognise Words using Visually Grounded Speech",
            "authors": [
                "Sebastiaan Scholten",
                "Danny Merkx",
                "Odette Scharenborg"
            ],
            "abstract": "We investigated word recognition in a Visually Grounded Speech model. The\nmodel has been trained on pairs of images and spoken captions to create\nvisually grounded embeddings which can be used for speech to image retrieval\nand vice versa. We investigate whether such a model can be used to recognise\nwords by embedding isolated words and using them to retrieve images of their\nvisual referents. We investigate the time-course of word recognition using a\ngating paradigm and perform a statistical analysis to see whether well known\nword competition effects in human speech processing influence word recognition.\nOur experiments show that the model is able to recognise words, and the gating\nparadigm reveals that words can be recognised from partial input as well and\nthat recognition is negatively influenced by word competition from the word\ninitial cohort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00575v2",
            "date": "2020-05-31 18:02:26+00:00",
            "title": "Neural Entity Linking: A Survey of Models Based on Deep Learning",
            "authors": [
                "Ozge Sevgili",
                "Artem Shelmanov",
                "Mikhail Arkhipov",
                "Alexander Panchenko",
                "Chris Biemann"
            ],
            "abstract": "In this survey, we provide a comprehensive description of recent neural\nentity linking (EL) systems developed since 2015 as a result of the \"deep\nlearning revolution\" in NLP. Our goal is to systemize design features of neural\nentity linking systems and compare their performances to the best classic\nmethods on the common benchmarks. We distill generic architectural components\nof a neural EL system, like candidate generation and entity ranking summarizing\nthe prominent methods for each of them, such as approaches to mention encoding\nbased on the self-attention architecture. The vast variety of modifications of\nthis general neural entity linking architecture are grouped by several common\nthemes: joint entity recognition and linking, models for global linking,\ndomain-independent techniques including zero-shot and distant supervision\nmethods, and cross-lingual approaches. Since many neural models take advantage\nof pre-trained entity embeddings to improve their generalization capabilities,\nwe provide an overview of popular entity embedding techniques. Finally, we\nbriefly discuss applications of entity linking, focusing on the recently\nemerged use-case of enhancing deep pre-trained masked language models such as\nBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00578v1",
            "date": "2020-05-31 18:11:52+00:00",
            "title": "\"Judge me by my size (noun), do you?'' YodaLib: A Demographic-Aware Humor Generation Framework",
            "authors": [
                "Aparna Garimella",
                "Carmen Banea",
                "Nabil Hossain",
                "Rada Mihalcea"
            ],
            "abstract": "The subjective nature of humor makes computerized humor generation a\nchallenging task. We propose an automatic humor generation framework for\nfilling the blanks in Mad Libs stories, while accounting for the demographic\nbackgrounds of the desired audience. We collect a dataset consisting of such\nstories, which are filled in and judged by carefully selected workers on Amazon\nMechanical Turk. We build upon the BERT platform to predict location-biased\nword fillings in incomplete sentences, and we fine tune BERT to classify\nlocation-specific humor in a sentence. We leverage these components to produce\nYodaLib, a fully-automated Mad Libs style humor generation framework, which\nselects and ranks appropriate candidate words and sentences in order to\ngenerate a coherent and funny story tailored to certain demographics. Our\nexperimental results indicate that YodaLib outperforms a previous\nsemi-automated approach proposed for this task, while also surpassing human\nannotators in both qualitative and quantitative analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00591v2",
            "date": "2020-05-31 19:16:27+00:00",
            "title": "Efficient Deployment of Conversational Natural Language Interfaces over Databases",
            "authors": [
                "Anthony Colas",
                "Trung Bui",
                "Franck Dernoncourt",
                "Moumita Sinha",
                "Doo Soon Kim"
            ],
            "abstract": "Many users communicate with chatbots and AI assistants in order to help them\nwith various tasks. A key component of the assistant is the ability to\nunderstand and answer a user's natural language questions for\nquestion-answering (QA). Because data can be usually stored in a structured\nmanner, an essential step involves turning a natural language question into its\ncorresponding query language. However, in order to train most natural\nlanguage-to-query-language state-of-the-art models, a large amount of training\ndata is needed first. In most domains, this data is not available and\ncollecting such datasets for various domains can be tedious and time-consuming.\nIn this work, we propose a novel method for accelerating the training dataset\ncollection for developing the natural language-to-query-language machine\nlearning models. Our system allows one to generate conversational multi-term\ndata, where multiple turns define a dialogue session, enabling one to better\nutilize chatbot interfaces. We train two current state-of-the-art NL-to-QL\nmodels, on both an SQL and SPARQL-based datasets in order to showcase the\nadaptability and efficacy of our created data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00593v2",
            "date": "2020-05-31 19:35:53+00:00",
            "title": "BPGC at SemEval-2020 Task 11: Propaganda Detection in News Articles with Multi-Granularity Knowledge Sharing and Linguistic Features based Ensemble Learning",
            "authors": [
                "Rajaswa Patil",
                "Somesh Singh",
                "Swati Agarwal"
            ],
            "abstract": "Propaganda spreads the ideology and beliefs of like-minded people,\nbrainwashing their audiences, and sometimes leading to violence. SemEval 2020\nTask-11 aims to design automated systems for news propaganda detection. Task-11\nconsists of two sub-tasks, namely, Span Identification - given any news\narticle, the system tags those specific fragments which contain at least one\npropaganda technique; and Technique Classification - correctly classify a given\npropagandist statement amongst 14 propaganda techniques. For sub-task 1, we use\ncontextual embeddings extracted from pre-trained transformer models to\nrepresent the text data at various granularities and propose a\nmulti-granularity knowledge sharing approach. For sub-task 2, we use an\nensemble of BERT and logistic regression classifiers with linguistic features.\nOur results reveal that the linguistic features are the strong indicators for\ncovering minority classes in a highly imbalanced dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00607v1",
            "date": "2020-05-31 20:55:08+00:00",
            "title": "LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading",
            "authors": [
                "Siddhant Mahurkar",
                "Rajaswa Patil"
            ],
            "abstract": "In this paper, we assess the ability of BERT and its derivative models\n(RoBERTa, DistilBERT, and ALBERT) for short-edits based humor grading. We test\nthese models for humor grading and classification tasks on the Humicroedit and\nthe FunLines dataset. We perform extensive experiments with these models to\ntest their language modeling and generalization abilities via zero-shot\ninference and cross-dataset inference based approaches. Further, we also\ninspect the role of self-attention layers in humor-grading by performing a\nqualitative analysis over the self-attention weights from the final layer of\nthe trained BERT model. Our experiments show that all the pre-trained BERT\nderivative models show significant generalization capabilities for\nhumor-grading related tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00609v1",
            "date": "2020-05-31 21:02:25+00:00",
            "title": "CNRL at SemEval-2020 Task 5: Modelling Causal Reasoning in Language with Multi-Head Self-Attention Weights based Counterfactual Detection",
            "authors": [
                "Rajaswa Patil",
                "Veeky Baths"
            ],
            "abstract": "In this paper, we describe an approach for modelling causal reasoning in\nnatural language by detecting counterfactuals in text using multi-head\nself-attention weights. We use pre-trained transformer models to extract\ncontextual embeddings and self-attention weights from the text. We show the use\nof convolutional layers to extract task-specific features from these\nself-attention weights. Further, we describe a fine-tuning approach with a\ncommon base model for knowledge sharing between the two closely related\nsub-tasks for counterfactual detection. We analyze and compare the performance\nof various transformer models in our experiments. Finally, we perform a\nqualitative analysis with the multi-head self-attention weights to interpret\nour models' dynamics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00632v2",
            "date": "2020-05-31 22:34:14+00:00",
            "title": "Neural Unsupervised Domain Adaptation in NLP---A Survey",
            "authors": [
                "Alan Ramponi",
                "Barbara Plank"
            ],
            "abstract": "Deep neural networks excel at learning from labeled data and achieve\nstate-of-the-art resultson a wide array of Natural Language Processing tasks.\nIn contrast, learning from unlabeled data, especially under domain shift,\nremains a challenge. Motivated by the latest advances, in this survey we review\nneural unsupervised domain adaptation techniques which do not require labeled\ntarget domain data. This is a more challenging yet a more widely applicable\nsetup. We outline methods, from early traditional non-neural methods to\npre-trained model transfer. We also revisit the notion of domain, and we\nuncover a bias in the type of Natural Language Processing tasks which received\nmost attention. Lastly, we outline future directions, particularly the broader\nneed for out-of-distribution generalization of future NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00635v2",
            "date": "2020-05-31 23:14:02+00:00",
            "title": "A Unified Feature Representation for Lexical Connotations",
            "authors": [
                "Emily Allaway",
                "Kathleen McKeown"
            ],
            "abstract": "Ideological attitudes and stance are often expressed through subtle meanings\nof words and phrases. Understanding these connotations is critical to\nrecognizing the cultural and emotional perspectives of the speaker. In this\npaper, we use distant labeling to create a new lexical resource representing\nconnotation aspects for nouns and adjectives. Our analysis shows that it aligns\nwell with human judgments. Additionally, we present a method for creating\nlexical representations that captures connotations within the embedding space\nand show that using the embeddings provides a statistically significant\nimprovement on the task of stance detection when data is limited.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00671v2",
            "date": "2020-06-01 02:20:08+00:00",
            "title": "Conversational Machine Comprehension: a Literature Review",
            "authors": [
                "Somil Gupta",
                "Bhanu Pratap Singh Rawat",
                "Hong Yu"
            ],
            "abstract": "Conversational Machine Comprehension (CMC), a research track in\nconversational AI, expects the machine to understand an open-domain natural\nlanguage text and thereafter engage in a multi-turn conversation to answer\nquestions related to the text. While most of the research in Machine Reading\nComprehension (MRC) revolves around single-turn question answering (QA),\nmulti-turn CMC has recently gained prominence, thanks to the advancement in\nnatural language understanding via neural language models such as BERT and the\nintroduction of large-scale conversational datasets such as CoQA and QuAC. The\nrise in interest has, however, led to a flurry of concurrent publications, each\nwith a different yet structurally similar modeling approach and an inconsistent\nview of the surrounding literature. With the volume of model submissions to\nconversational datasets increasing every year, there exists a need to\nconsolidate the scattered knowledge in this domain to streamline future\nresearch. This literature review attempts at providing a holistic overview of\nCMC with an emphasis on the common trends across recently published models,\nspecifically in their approach to tackling conversational history. The review\nsynthesizes a generic framework for CMC models while highlighting the\ndifferences in recent approaches and intends to serve as a compendium of CMC\nfor future researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00814v3",
            "date": "2020-06-01 09:43:54+00:00",
            "title": "Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English",
            "authors": [
                "Maha Elbayad",
                "Michael Ustaszewski",
                "Emmanuelle Esperan\u00e7a-Rodier",
                "Francis Brunet Manquat",
                "Jakob Verbeek",
                "Laurent Besacier"
            ],
            "abstract": "We conduct in this work an evaluation study comparing offline and online\nneural machine translation architectures. Two sequence-to-sequence models:\nconvolutional Pervasive Attention (Elbayad et al. 2018) and attention-based\nTransformer (Vaswani et al. 2017) are considered. We investigate, for both\narchitectures, the impact of online decoding constraints on the translation\nquality through a carefully designed human evaluation on English-German and\nGerman-English language pairs, the latter being particularly sensitive to\nlatency constraints. The evaluation results allow us to identify the strengths\nand shortcomings of each model when we shift to the online setup.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00838v1",
            "date": "2020-06-01 10:31:56+00:00",
            "title": "Efficient EUD Parsing",
            "authors": [
                "Mathieu Dehouck",
                "Mark Anderson",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present the system submission from the FASTPARSE team for the EUD Shared\nTask at IWPT 2020. We engaged with the task by focusing on efficiency. For this\nwe considered training costs and inference efficiency. Our models are a\ncombination of distilled neural dependency parsers and a rule-based system that\nprojects UD trees into EUD graphs. We obtained an average ELAS of 74.04 for our\nofficial submission, ranking 4th overall.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00843v2",
            "date": "2020-06-01 10:39:50+00:00",
            "title": "Rhetoric, Logic, and Dialectic: Advancing Theory-based Argument Quality Assessment in Natural Language Processing",
            "authors": [
                "Anne Lauscher",
                "Lily Ng",
                "Courtney Napoles",
                "Joel Tetreault"
            ],
            "abstract": "Though preceding work in computational argument quality (AQ) mostly focuses\non assessing overall AQ, researchers agree that writers would benefit from\nfeedback targeting individual dimensions of argumentation theory. However, a\nlarge-scale theory-based corpus and corresponding computational models are\nmissing. We fill this gap by conducting an extensive analysis covering three\ndiverse domains of online argumentative writing and presenting GAQCorpus: the\nfirst large-scale English multi-domain (community Q&A forums, debate forums,\nreview forums) corpus annotated with theory-based AQ scores. We then propose\nthe first computational approaches to theory-based assessment, which can serve\nas strong baselines for future work. We demonstrate the feasibility of\nlarge-scale AQ annotation, show that exploiting relations between dimensions\nyields performance improvements, and explore the synergies between theory-based\nprediction and practical AQ assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00844v1",
            "date": "2020-06-01 10:43:53+00:00",
            "title": "Distilling Neural Networks for Greener and Faster Dependency Parsing",
            "authors": [
                "Mark Anderson",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "The carbon footprint of natural language processing research has been\nincreasing in recent years due to its reliance on large and inefficient neural\nnetwork implementations. Distillation is a network compression technique which\nattempts to impart knowledge from a large model to a smaller one. We use\nteacher-student distillation to improve the efficiency of the Biaffine\ndependency parser which obtains state-of-the-art performance with respect to\naccuracy and parsing speed (Dozat and Manning, 2017). When distilling to 20\\%\nof the original model's trainable parameters, we only observe an average\ndecrease of $\\sim$1 point for both UAS and LAS across a number of diverse\nUniversal Dependency treebanks while being 2.30x (1.19x) faster than the\nbaseline model on CPU (GPU) at inference time. We also observe a small increase\nin performance when compressing to 80\\% for some treebanks. Finally, through\ndistillation we attain a parser which is not only faster but also more accurate\nthan the fastest modern parser on the Penn Treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00995v3",
            "date": "2020-06-01 15:00:11+00:00",
            "title": "Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals",
            "authors": [
                "Yanai Elazar",
                "Shauli Ravfogel",
                "Alon Jacovi",
                "Yoav Goldberg"
            ],
            "abstract": "A growing body of work makes use of probing to investigate the working of\nneural models, often considered black boxes. Recently, an ongoing debate\nemerged surrounding the limitations of the probing paradigm. In this work, we\npoint out the inability to infer behavioral conclusions from probing results\nand offer an alternative method that focuses on how the information is being\nused, rather than on what information is encoded. Our method, Amnesic Probing,\nfollows the intuition that the utility of a property for a given task can be\nassessed by measuring the influence of a causal intervention that removes it\nfrom the representation. Equipped with this new analysis tool, we can ask\nquestions that were not possible before, e.g. is part-of-speech information\nimportant for word prediction? We perform a series of analyses on BERT to\nanswer these types of questions. Our findings demonstrate that conventional\nprobing performance is not correlated to task importance, and we call for\nincreased scrutiny of claims that draw behavioral or causal conclusions from\nprobing results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00998v1",
            "date": "2020-06-01 15:03:48+00:00",
            "title": "Toxicity Detection: Does Context Really Matter?",
            "authors": [
                "John Pavlopoulos",
                "Jeffrey Sorensen",
                "Lucas Dixon",
                "Nithum Thain",
                "Ion Androutsopoulos"
            ],
            "abstract": "Moderation is crucial to promoting healthy on-line discussions. Although\nseveral `toxicity' detection datasets and models have been published, most of\nthem ignore the context of the posts, implicitly assuming that comments maybe\njudged independently. We investigate this assumption by focusing on two\nquestions: (a) does context affect the human judgement, and (b) does\nconditioning on context improve performance of toxicity detection systems? We\nexperiment with Wikipedia conversations, limiting the notion of context to the\nprevious post in the thread and the discussion title. We find that context can\nboth amplify or mitigate the perceived toxicity of posts. Moreover, a small but\nsignificant subset of manually labeled posts (5% in one of our experiments) end\nup having the opposite toxicity labels if the annotators are not provided with\ncontext. Surprisingly, we also find no evidence that context actually improves\nthe performance of toxicity classifiers, having tried a range of classifiers\nand mechanisms to make them context aware. This points to the need for larger\ndatasets of comments annotated in context. We make our code and data publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01038v3",
            "date": "2020-06-01 16:04:30+00:00",
            "title": "DocBank: A Benchmark Dataset for Document Layout Analysis",
            "authors": [
                "Minghao Li",
                "Yiheng Xu",
                "Lei Cui",
                "Shaohan Huang",
                "Furu Wei",
                "Zhoujun Li",
                "Ming Zhou"
            ],
            "abstract": "Document layout analysis usually relies on computer vision models to\nunderstand documents while ignoring textual information that is vital to\ncapture. Meanwhile, high quality labeled datasets with both visual and textual\ninformation are still insufficient. In this paper, we present \\textbf{DocBank},\na benchmark dataset that contains 500K document pages with fine-grained\ntoken-level annotations for document layout analysis. DocBank is constructed\nusing a simple yet effective way with weak supervision from the \\LaTeX{}\ndocuments available on the arXiv.com. With DocBank, models from different\nmodalities can be compared fairly and multi-modal approaches will be further\ninvestigated and boost the performance of document layout analysis. We build\nseveral strong baselines and manually split train/dev/test sets for evaluation.\nExperiment results show that models trained on DocBank accurately recognize the\nlayout information for a variety of documents. The DocBank dataset is publicly\navailable at \\url{https://github.com/doc-analysis/DocBank}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01080v1",
            "date": "2020-06-01 17:02:28+00:00",
            "title": "Is 42 the Answer to Everything in Subtitling-oriented Speech Translation?",
            "authors": [
                "Alina Karakanta",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Subtitling is becoming increasingly important for disseminating information,\ngiven the enormous amounts of audiovisual content becoming available daily.\nAlthough Neural Machine Translation (NMT) can speed up the process of\ntranslating audiovisual content, large manual effort is still required for\ntranscribing the source language, and for spotting and segmenting the text into\nproper subtitles. Creating proper subtitles in terms of timing and segmentation\nhighly depends on information present in the audio (utterance duration, natural\npauses). In this work, we explore two methods for applying Speech Translation\n(ST) to subtitling: a) a direct end-to-end and b) a classical cascade approach.\nWe discuss the benefit of having access to the source language speech for\nimproving the conformity of the generated subtitles to the spatial and temporal\nsubtitling constraints and show that length is not the answer to everything in\nthe case of subtitling-oriented ST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01117v1",
            "date": "2020-06-01 17:58:57+00:00",
            "title": "NSTM: Real-Time Query-Driven News Overview Composition at Bloomberg",
            "authors": [
                "Joshua Bambrick",
                "Minjie Xu",
                "Andy Almonte",
                "Igor Malioutov",
                "Guim Perarnau",
                "Vittorio Selo",
                "Iat Chong Chan"
            ],
            "abstract": "Millions of news articles from hundreds of thousands of sources around the\nglobe appear in news aggregators every day. Consuming such a volume of news\npresents an almost insurmountable challenge. For example, a reader searching on\nBloomberg's system for news about the U.K. would find 10,000 articles on a\ntypical day. Apple Inc., the world's most journalistically covered company,\ngarners around 1,800 news articles a day.\n  We realized that a new kind of summarization engine was needed, one that\nwould condense large volumes of news into short, easy to absorb points. The\nsystem would filter out noise and duplicates to identify and summarize key news\nabout companies, countries or markets.\n  When given a user query, Bloomberg's solution, Key News Themes (or NSTM),\nleverages state-of-the-art semantic clustering techniques and novel\nsummarization methods to produce comprehensive, yet concise, digests to\ndramatically simplify the news consumption process.\n  NSTM is available to hundreds of thousands of readers around the world and\nserves thousands of requests daily with sub-second latency. At ACL 2020, we\nwill present a demo of NSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01175v2",
            "date": "2020-06-01 18:07:15+00:00",
            "title": "Lexical Normalization for Code-switched Data and its Effect on POS-tagging",
            "authors": [
                "Rob van der Goot",
                "\u00d6zlem \u00c7etino\u011flu"
            ],
            "abstract": "Lexical normalization, the translation of non-canonical data to standard\nlanguage, has shown to improve the performance of manynatural language\nprocessing tasks on social media. Yet, using multiple languages in one\nutterance, also called code-switching (CS), is frequently overlooked by these\nnormalization systems, despite its common use in social media. In this paper,\nwe propose three normalization models specifically designed to handle\ncode-switched data which we evaluate for two language pairs: Indonesian-English\n(Id-En) and Turkish-German (Tr-De). For the latter, we introduce novel\nnormalization layers and their corresponding language ID and POS tags for the\ndataset, and evaluate the downstream effect of normalization on POS tagging.\nResults show that our CS-tailored normalization models outperform Id-En state\nof the art and Tr-De monolingual models, and lead to 5.4% relative performance\nincrease for POS tagging as compared to unnormalized input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01189v1",
            "date": "2020-06-01 18:27:48+00:00",
            "title": "An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features",
            "authors": [
                "Shi-Yan Weng",
                "Tien-Hong Lo",
                "Berlin Chen"
            ],
            "abstract": "Tremendous amounts of multimedia associated with speech information are\ndriving an urgent need to develop efficient and effective automatic\nsummarization methods. To this end, we have seen rapid progress in applying\nsupervised deep neural network-based methods to extractive speech\nsummarization. More recently, the Bidirectional Encoder Representations from\nTransformers (BERT) model was proposed and has achieved record-breaking success\non many natural language processing (NLP) tasks such as question answering and\nlanguage understanding. In view of this, we in this paper contextualize and\nenhance the state-of-the-art BERT-based model for speech summarization, while\nits contributions are at least three-fold. First, we explore the incorporation\nof confidence scores into sentence representations to see if such an attempt\ncould help alleviate the negative effects caused by imperfect automatic speech\nrecognition (ASR). Secondly, we also augment the sentence embeddings obtained\nfrom BERT with extra structural and linguistic features, such as sentence\nposition and inverse document frequency (IDF) statistics. Finally, we validate\nthe effectiveness of our proposed method on a benchmark dataset, in comparison\nto several classic and celebrated speech summarization methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01205v2",
            "date": "2020-05-17 13:20:10+00:00",
            "title": "CS-NLP team at SemEval-2020 Task 4: Evaluation of State-of-the-art NLP Deep Learning Architectures on Commonsense Reasoning Task",
            "authors": [
                "Sirwe Saeedi",
                "Aliakbar Panahi",
                "Seyran Saeedi",
                "Alvis C Fong"
            ],
            "abstract": "In this paper, we investigate a commonsense inference task that unifies\nnatural language understanding and commonsense reasoning. We describe our\nattempt at SemEval-2020 Task 4 competition: Commonsense Validation and\nExplanation (ComVE) challenge. We discuss several state-of-the-art deep\nlearning architectures for this challenge. Our system uses prepared labeled\ntextual datasets that were manually curated for three different natural\nlanguage inference subtasks. The goal of the first subtask is to test whether a\nmodel can distinguish between natural language statements that make sense and\nthose that do not make sense. We compare the performance of several language\nmodels and fine-tuned classifiers. Then, we propose a method inspired by\nquestion/answering tasks to treat a classification problem as a multiple choice\nquestion task to boost the performance of our experimental results (96.06%),\nwhich is significantly better than the baseline. For the second subtask, which\nis to select the reason why a statement does not make sense, we stand within\nthe first six teams (93.7%) among 27 participants with very competitive\nresults. Our result for last subtask of generating reason against the nonsense\nstatement shows many potentials for future researches as we applied the most\npowerful generative model of language (GPT-2) with 6.1732 BLEU score among\nfirst four teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01210v1",
            "date": "2020-05-26 05:18:56+00:00",
            "title": "The 'Letter' Distribution in the Chinese Language",
            "authors": [
                "Qinghua Chen",
                "Yan Wang",
                "Mengmeng Wang",
                "Xiaomeng Li"
            ],
            "abstract": "Corpus-based statistical analysis plays a significant role in linguistic\nresearch, and ample evidence has shown that different languages exhibit some\ncommon laws. Studies have found that letters in some alphabetic writing\nlanguages have strikingly similar statistical usage frequency distributions.\nDoes this hold for Chinese, which employs ideogram writing? We obtained letter\nfrequency data of some alphabetic writing languages and found the common law of\nthe letter distributions. In addition, we collected Chinese literature corpora\nfor different historical periods from the Tang Dynasty to the present, and we\ndismantled the Chinese written language into three kinds of basic particles:\ncharacters, strokes and constructive parts. The results of the statistical\nanalysis showed that, in different historical periods, the intensity of the use\nof basic particles in Chinese writing varied, but the form of the distribution\nwas consistent. In particular, the distributions of the Chinese constructive\nparts are certainly consistent with those alphabetic writing languages. This\nstudy provides new evidence of the consistency of human languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01266v1",
            "date": "2020-05-16 04:55:35+00:00",
            "title": "Leveraging Affective Bidirectional Transformers for Offensive Language Detection",
            "authors": [
                "AbdelRahim Elmadany",
                "Chiyu Zhang",
                "Muhammad Abdul-Mageed",
                "Azadeh Hashemi"
            ],
            "abstract": "Social media are pervasive in our life, making it necessary to ensure safe\nonline experiences by detecting and removing offensive and hate speech. In this\nwork, we report our submission to the Offensive Language and hate-speech\nDetection shared task organized with the 4th Workshop on Open-Source Arabic\nCorpora and Processing Tools Arabic (OSACT4). We focus on developing purely\ndeep learning systems, without a need for feature engineering. For that\npurpose, we develop an effective method for automatic data augmentation and\nshow the utility of training both offensive and hate speech models off (i.e.,\nby fine-tuning) previously trained affective models (i.e., sentiment and\nemotion). Our best models are significantly better than a vanilla BERT model,\nwith 89.60% acc (82.31% macro F1) for hate speech and 95.20% acc (70.51% macro\nF1) on official TEST data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01338v1",
            "date": "2020-06-02 01:46:04+00:00",
            "title": "A Survey of Neural Networks and Formal Languages",
            "authors": [
                "Joshua Ackerman",
                "George Cybenko"
            ],
            "abstract": "This report is a survey of the relationships between various state-of-the-art\nneural network architectures and formal languages as, for example, structured\nby the Chomsky Language Hierarchy. Of particular interest are the abilities of\na neural architecture to represent, recognize and generate words from a\nspecific language by learning from samples of the language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01372v2",
            "date": "2020-06-02 03:47:19+00:00",
            "title": "Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition",
            "authors": [
                "Takuma Kato",
                "Kaori Abe",
                "Hiroki Ouchi",
                "Shumpei Miyawaki",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "In general, the labels used in sequence labeling consist of different types\nof elements. For example, IOB-format entity labels, such as B-Person and\nI-Person, can be decomposed into span (B and I) and type information (Person).\nHowever, while most sequence labeling models do not consider such label\ncomponents, the shared components across labels, such as Person, can be\nbeneficial for label prediction. In this work, we propose to integrate label\ncomponent information as embeddings into models. Through experiments on English\nand Japanese fine-grained named entity recognition, we demonstrate that the\nproposed method improves performance, especially for instances with\nlow-frequency labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01432v1",
            "date": "2020-06-02 07:36:49+00:00",
            "title": "BERT Based Multilingual Machine Comprehension in English and Hindi",
            "authors": [
                "Somil Gupta",
                "Nilesh Khade"
            ],
            "abstract": "Multilingual Machine Comprehension (MMC) is a Question-Answering (QA)\nsub-task that involves quoting the answer for a question from a given snippet,\nwhere the question and the snippet can be in different languages. Recently\nreleased multilingual variant of BERT (m-BERT), pre-trained with 104 languages,\nhas performed well in both zero-shot and fine-tuned settings for multilingual\ntasks; however, it has not been used for English-Hindi MMC yet. We, therefore,\npresent in this article, our experiments with m-BERT for MMC in zero-shot,\nmono-lingual (e.g. Hindi Question-Hindi Snippet) and cross-lingual (e.g.\nEnglish QuestionHindi Snippet) fine-tune setups. These model variants are\nevaluated on all possible multilingual settings and results are compared\nagainst the current state-of-the-art sequential QA system for these languages.\nExperiments show that m-BERT, with fine-tuning, improves performance on all\nevaluation settings across both the datasets used by the prior model, therefore\nestablishing m-BERT based MMC as the new state-of-the-art for English and\nHindi. We also publish our results on an extended version of the recently\nreleased XQuAD dataset, which we propose to use as the evaluation benchmark for\nfuture research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01554v2",
            "date": "2020-06-02 12:25:44+00:00",
            "title": "A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking",
            "authors": [
                "Yong Shan",
                "Zekang Li",
                "Jinchao Zhang",
                "Fandong Meng",
                "Yang Feng",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Recent studies in dialogue state tracking (DST) leverage historical\ninformation to determine states which are generally represented as slot-value\npairs. However, most of them have limitations to efficiently exploit relevant\ncontext due to the lack of a powerful mechanism for modeling interactions\nbetween the slot and the dialogue history. Besides, existing methods usually\nignore the slot imbalance problem and treat all slots indiscriminately, which\nlimits the learning of hard slots and eventually hurts overall performance. In\nthis paper, we propose to enhance the DST through employing a contextual\nhierarchical attention network to not only discern relevant information at both\nword level and turn level but also learn contextual representations. We further\npropose an adaptive objective to alleviate the slot imbalance problem by\ndynamically adjust weights of different slots during training. Experimental\nresults show that our approach reaches 52.68% and 58.55% joint accuracy on\nMultiWOZ 2.0 and MultiWOZ 2.1 datasets respectively and achieves new\nstate-of-the-art performance with considerable improvements (+1.24% and\n+5.98%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01563v2",
            "date": "2020-06-02 12:34:52+00:00",
            "title": "Exploring Cross-sentence Contexts for Named Entity Recognition with BERT",
            "authors": [
                "Jouni Luoma",
                "Sampo Pyysalo"
            ],
            "abstract": "Named entity recognition (NER) is frequently addressed as a sequence\nclassification task where each input consists of one sentence of text. It is\nnevertheless clear that useful information for the task can often be found\noutside of the scope of a single-sentence context. Recently proposed\nself-attention models such as BERT can both efficiently capture long-distance\nrelationships in input as well as represent inputs consisting of several\nsentences, creating new opportunitites for approaches that incorporate\ncross-sentence information in natural language processing tasks. In this paper,\nwe present a systematic study exploring the use of cross-sentence information\nfor NER using BERT models in five languages. We find that adding context in the\nform of additional sentences to BERT input systematically increases NER\nperformance on all of the tested languages and models. Including multiple\nsentences in each input also allows us to study the predictions of the same\nsentences in different contexts. We propose a straightforward method,\nContextual Majority Voting (CMV), to combine different predictions for\nsentences and demonstrate this to further increase NER performance with BERT.\nOur approach does not require any changes to the underlying BERT architecture,\nrather relying on restructuring examples for training and prediction.\nEvaluation on established datasets, including the CoNLL'02 and CoNLL'03 NER\nbenchmarks, demonstrates that our proposed approach can improve on the\nstate-of-the-art NER results on English, Dutch, and Finnish, achieves the best\nreported BERT-based results on German, and is on par with performance reported\nwith other BERT-based approaches in Spanish. We release all methods implemented\nin this work under open licenses.",
            "journal ref": "Proceedings of the 28th International Conference on Computational\n  Linguistics, dec,2020, Barcelona, Spain (Online),International Committee on\n  Computational Linguistics, pages 904-914",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01592v2",
            "date": "2020-06-02 13:34:11+00:00",
            "title": "A Unified Dual-view Model for Review Summarization and Sentiment Classification with Inconsistency Loss",
            "authors": [
                "Hou Pong Chan",
                "Wang Chen",
                "Irwin King"
            ],
            "abstract": "Acquiring accurate summarization and sentiment from user reviews is an\nessential component of modern e-commerce platforms. Review summarization aims\nat generating a concise summary that describes the key opinions and sentiment\nof a review, while sentiment classification aims to predict a sentiment label\nindicating the sentiment attitude of a review. To effectively leverage the\nshared sentiment information in both review summarization and sentiment\nclassification tasks, we propose a novel dual-view model that jointly improves\nthe performance of these two tasks. In our model, an encoder first learns a\ncontext representation for the review, then a summary decoder generates a\nreview summary word by word. After that, a source-view sentiment classifier\nuses the encoded context representation to predict a sentiment label for the\nreview, while a summary-view sentiment classifier uses the decoder hidden\nstates to predict a sentiment label for the generated summary. During training,\nwe introduce an inconsistency loss to penalize the disagreement between these\ntwo classifiers. It helps the decoder to generate a summary to have a\nconsistent sentiment tendency with the review and also helps the two sentiment\nclassifiers learn from each other. Experiment results on four real-world\ndatasets from different domains demonstrate the effectiveness of our model.",
            "journal ref": "SIGIR 2020: 1191-1200",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01594v1",
            "date": "2020-05-29 19:00:59+00:00",
            "title": "Training Multilingual Machine Translation by Alternately Freezing Language-Specific Encoders-Decoders",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa",
                "Mikel Artetxe"
            ],
            "abstract": "We propose a modular architecture of language-specific encoder-decoders that\nconstitutes a multilingual machine translation system that can be incrementally\nextended to new languages without the need for retraining the existing system\nwhen adding new languages. Differently from previous works, we simultaneously\ntrain $N$ languages in all translation directions by alternately freezing\nencoder or decoder modules, which indirectly forces the system to train in a\ncommon intermediate representation for all languages. Experimental results from\nmultilingual machine translation show that we can successfully train this\nmodular architecture improving on the initial languages while falling slightly\nbehind when adding new languages or doing zero-shot translation. Additional\ncomparison of the quality of sentence representation in the task of natural\nlanguage inference shows that the alternately freezing training is also\nbeneficial in this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01603v1",
            "date": "2020-06-02 13:39:53+00:00",
            "title": "DiscSense: Automated Semantic Analysis of Discourse Markers",
            "authors": [
                "Damien Sileo",
                "Tim Van de Cruys",
                "Camille Pradel",
                "Philippe Muller"
            ],
            "abstract": "Discourse markers ({\\it by contrast}, {\\it happily}, etc.) are words or\nphrases that are used to signal semantic and/or pragmatic relationships between\nclauses or sentences. Recent work has fruitfully explored the prediction of\ndiscourse markers between sentence pairs in order to learn accurate sentence\nrepresentations, that are useful in various classification tasks. In this work,\nwe take another perspective: using a model trained to predict discourse markers\nbetween sentence pairs, we predict plausible markers between sentence pairs\nwith a known semantic relation (provided by existing classification datasets).\nThese predictions allow us to study the link between discourse markers and the\nsemantic relations annotated in classification datasets. Handcrafted mappings\nhave been proposed between markers and discourse relations on a limited set of\nmarkers and a limited set of categories, but there exist hundreds of discourse\nmarkers expressing a wide variety of relations, and there is no consensus on\nthe taxonomy of relations between competing discourse theories (which are\nlargely built in a top-down fashion). By using an automatic rediction method\nover existing semantically annotated datasets, we provide a bottom-up\ncharacterization of discourse markers in English. The resulting dataset, named\nDiscSense, is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01715v1",
            "date": "2020-06-02 15:35:05+00:00",
            "title": "Web Document Categorization Using Naive Bayes Classifier and Latent Semantic Analysis",
            "authors": [
                "Alireza Saleh Sedghpour",
                "Mohammad Reza Saleh Sedghpour"
            ],
            "abstract": "A rapid growth of web documents due to heavy use of World Wide Web\nnecessitates efficient techniques to efficiently classify the document on the\nweb. It is thus produced High volumes of data per second with high diversity.\nAutomatically classification of these growing amounts of web document is One of\nthe biggest challenges facing us today. Probabilistic classification algorithms\nsuch as Naive Bayes have become commonly used for web document classification.\nThis problem is mainly because of the irrelatively high classification accuracy\non plenty application areas as well as their lack of support to handle high\ndimensional and sparse data which is the exclusive characteristics of textual\ndata representation. also it is common to Lack of attention and support the\nsemantic relation between words using traditional feature selection method When\ndealing with the big data and large-scale web documents. In order to solve the\nproblem, we proposed a method for web document classification that uses LSA to\nincrease similarity of documents under the same class and improve the\nclassification precision. Using this approach, we designed a faster and much\naccurate classifier for Web Documents. Experimental results have shown that\nusing the mentioned preprocessing can improve accuracy and speed of Naive Bayes\navailably, the precision and recall metrics have indicated the improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01912v1",
            "date": "2020-06-02 19:47:01+00:00",
            "title": "On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior",
            "authors": [
                "Ethan Gotlieb Wilcox",
                "Jon Gauthier",
                "Jennifer Hu",
                "Peng Qian",
                "Roger Levy"
            ],
            "abstract": "Human reading behavior is tuned to the statistics of natural language: the\ntime it takes human subjects to read a word can be predicted from estimates of\nthe word's probability in context. However, it remains an open question what\ncomputational architecture best characterizes the expectations deployed in real\ntime by humans that determine the behavioral signatures of reading. Here we\ntest over two dozen models, independently manipulating computational\narchitecture and training dataset size, on how well their next-word\nexpectations predict human reading time behavior on naturalistic text corpora.\nWe find that across model architectures and training dataset sizes the\nrelationship between word log-probability and reading time is (near-)linear. We\nnext evaluate how features of these models determine their psychometric\npredictive power, or ability to predict human reading behavior. In general, the\nbetter a model's next-word expectations, the better its psychometric predictive\npower. However, we find nontrivial differences across model architectures. For\nany given perplexity, deep Transformer models and n-gram models generally show\nsuperior psychometric predictive power over LSTM or structurally supervised\nneural models, especially for eye movement data. Finally, we compare models'\npsychometric predictive power to the depth of their syntactic knowledge, as\nmeasured by a battery of syntactic generalization tests developed using methods\nfrom controlled psycholinguistic experiments. Once perplexity is controlled\nfor, we find no significant relationship between syntactic knowledge and\npredictive power. These results suggest that different approaches may be\nrequired to best model human real-time language comprehension behavior in\nnaturalistic reading versus behavior for controlled linguistic materials\ndesigned for targeted probing of syntactic knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01966v1",
            "date": "2020-06-02 22:31:40+00:00",
            "title": "The Typology of Polysemy: A Multilingual Distributional Framework",
            "authors": [
                "Ella Rabinovich",
                "Yang Xu",
                "Suzanne Stevenson"
            ],
            "abstract": "Lexical semantic typology has identified important cross-linguistic\ngeneralizations about the variation and commonalities in polysemy\npatterns---how languages package up meanings into words. Recent computational\nresearch has enabled investigation of lexical semantics at a much larger scale,\nbut little work has explored lexical typology across semantic domains, nor the\nfactors that influence cross-linguistic similarities. We present a novel\ncomputational framework that quantifies semantic affinity, the cross-linguistic\nsimilarity of lexical semantics for a concept. Our approach defines a common\nmultilingual semantic space that enables a direct comparison of the lexical\nexpression of concepts across languages. We validate our framework against\nempirical findings on lexical semantic typology at both the concept and domain\nlevels. Our results reveal an intricate interaction between semantic domains\nand extra-linguistic factors, beyond language phylogeny, that co-shape the\ntypology of polysemy across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02120v1",
            "date": "2020-06-03 09:28:17+00:00",
            "title": "Towards Large-Scale Data Mining for Data-Driven Analysis of Sign Languages",
            "authors": [
                "Boris Mocialov",
                "Graham Turner",
                "Helen Hastie"
            ],
            "abstract": "Access to sign language data is far from adequate. We show that it is\npossible to collect the data from social networking services such as TikTok,\nInstagram, and YouTube by applying data filtering to enforce quality standards\nand by discovering patterns in the filtered data, making it easier to analyse\nand model. Using our data collection pipeline, we collect and examine the\ninterpretation of songs in both the American Sign Language (ASL) and the\nBrazilian Sign Language (Libras). We explore their differences and similarities\nby looking at the co-dependence of the orientation and location phonological\nparameters",
            "journal ref": "Proceedings of the 9th Workshop on the Representation and\n  Processing of Sign Languages (LREC 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02144v1",
            "date": "2020-06-03 10:13:29+00:00",
            "title": "Transfer Learning for British Sign Language Modelling",
            "authors": [
                "Boris Mocialov",
                "Graham Turner",
                "Helen Hastie"
            ],
            "abstract": "Automatic speech recognition and spoken dialogue systems have made great\nadvances through the use of deep machine learning methods. This is partly due\nto greater computing power but also through the large amount of data available\nin common languages, such as English. Conversely, research in minority\nlanguages, including sign languages, is hampered by the severe lack of data.\nThis has led to work on transfer learning methods, whereby a model developed\nfor one language is reused as the starting point for a model on a second\nlanguage, which is less resourced. In this paper, we examine two transfer\nlearning techniques of fine-tuning and layer substitution for language\nmodelling of British Sign Language. Our results show improvement in perplexity\nwhen using transfer learning with standard stacked LSTM models, trained\ninitially using a large corpus for standard English from the Penn Treebank\ncorpus",
            "journal ref": "Proceedings of the Sixth Workshop on NLP for Similar Languages,\n  Varieties and Dialects, VarDial (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02964v1",
            "date": "2020-06-04 15:47:29+00:00",
            "title": "Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1",
            "authors": [
                "Maria Nadejde",
                "Joel Tetreault"
            ],
            "abstract": "Grammar error correction (GEC) systems have become ubiquitous in a variety of\nsoftware applications, and have started to approach human-level performance for\nsome datasets. However, very little is known about how to efficiently\npersonalize these systems to the user's characteristics, such as their\nproficiency level and first language, or to emerging domains of text. We\npresent the first results on adapting a general-purpose neural GEC system to\nboth the proficiency level and the first language of a writer, using only a few\nthousand annotated sentences. Our study is the broadest of its kind, covering\nfive proficiency levels and twelve different languages, and comparing three\ndifferent adaptation scenarios: adapting to the proficiency level only, to the\nfirst language only, or to both aspects simultaneously. We show that tailoring\nto both scenarios achieves the largest performance improvement (3.6 F0.5)\nrelative to a strong baseline.",
            "journal ref": "Proceedings of the 2019 EMNLP Workshop W-NUT: The 5th Workshop on\n  Noisy User-generated Text, pages 27-33, Hong Kong, Nov 4, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03002v1",
            "date": "2020-06-04 16:48:45+00:00",
            "title": "Linguists Who Use Probabilistic Models Love Them: Quantification in Functional Distributional Semantics",
            "authors": [
                "Guy Emerson"
            ],
            "abstract": "Functional Distributional Semantics provides a computationally tractable\nframework for learning truth-conditional semantics from a corpus. Previous work\nin this framework has provided a probabilistic version of first-order logic,\nrecasting quantification as Bayesian inference. In this paper, I show how the\nprevious formulation gives trivial truth values when a precise quantifier is\nused with vague predicates. I propose an improved account, avoiding this\nproblem by treating a vague predicate as a distribution over precise\npredicates. I connect this account to recent work in the Rational Speech Acts\nframework on modelling generic quantification, and I extend this to modelling\ndonkey sentences. Finally, I explain how the generic quantifier can be both\npragmatically complex and yet computationally simpler than precise quantifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03010v1",
            "date": "2020-06-04 16:59:01+00:00",
            "title": "Syntactic Search by Example",
            "authors": [
                "Micah Shlain",
                "Hillel Taub-Tabib",
                "Shoval Sadde",
                "Yoav Goldberg"
            ],
            "abstract": "We present a system that allows a user to search a large linguistically\nannotated corpus using syntactic patterns over dependency graphs. In contrast\nto previous attempts to this effect, we introduce a light-weight query language\nthat does not require the user to know the details of the underlying syntactic\nrepresentations, and instead to query the corpus by providing an example\nsentence coupled with simple markup. Search is performed at an interactive\nspeed due to an efficient linguistic graph-indexing and retrieval engine. This\nallows for rapid exploration, development and refinement of syntax-based\nqueries. We demonstrate the system using queries over two corpora: the English\nwikipedia, and a collection of English pubmed abstracts. A demo of the\nwikipedia system is available at: https://allenai.github.io/spike",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03051v1",
            "date": "2020-06-04 18:21:50+00:00",
            "title": "NewB: 200,000+ Sentences for Political Bias Detection",
            "authors": [
                "Jerry Wei"
            ],
            "abstract": "We present the Newspaper Bias Dataset (NewB), a text corpus of more than\n200,000 sentences from eleven news sources regarding Donald Trump. While\nprevious datasets have labeled sentences as either liberal or conservative,\nNewB covers the political views of eleven popular media sources, capturing more\nnuanced political viewpoints than a traditional binary classification system\ndoes. We train two state-of-the-art deep learning models to predict the news\nsource of a given sentence from eleven newspapers and find that a recurrent\nneural network achieved top-1, top-3, and top-5 accuracies of 33.3%, 61.4%, and\n77.6%, respectively, significantly outperforming a baseline logistic regression\nmodel's accuracies of 18.3%, 42.6%, and 60.8%. Using the news source label of\nsentences, we analyze the top n-grams with our model to gain meaningful insight\ninto the portrayal of Trump by media sources.We hope that the public release of\nour dataset will encourage further research in using natural language\nprocessing to analyze more complex political biases.\n  Our dataset is posted at https://github.com/JerryWei03/NewB .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03265v2",
            "date": "2020-06-05 07:23:03+00:00",
            "title": "Understanding Self-Attention of Self-Supervised Audio Transformers",
            "authors": [
                "Shu-wen Yang",
                "Andy T. Liu",
                "Hung-yi Lee"
            ],
            "abstract": "Self-supervised Audio Transformers (SAT) enable great success in many\ndownstream speech applications like ASR, but how they work has not been widely\nexplored yet. In this work, we present multiple strategies for the analysis of\nattention mechanisms in SAT. We categorize attentions into explainable\ncategories, where we discover each category possesses its own unique\nfunctionality. We provide a visualization tool for understanding multi-head\nself-attention, importance ranking strategies for identifying critical\nattention, and attention refinement techniques to improve model performance.",
            "journal ref": "INTERSPEECH 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03331v1",
            "date": "2020-06-05 09:29:57+00:00",
            "title": "ELITR Non-Native Speech Translation at IWSLT 2020",
            "authors": [
                "Dominik Mach\u00e1\u010dek",
                "Jon\u00e1\u0161 Kratochv\u00edl",
                "Sangeet Sagar",
                "Mat\u00fa\u0161 \u017dilinec",
                "Ond\u0159ej Bojar",
                "Thai-Son Nguyen",
                "Felix Schneider",
                "Philip Williams",
                "Yuekun Yao"
            ],
            "abstract": "This paper is an ELITR system submission for the non-native speech\ntranslation task at IWSLT 2020. We describe systems for offline ASR, real-time\nASR, and our cascaded approach to offline SLT and real-time SLT. We select our\nprimary candidates from a pool of pre-existing systems, develop a new\nend-to-end general ASR system, and a hybrid ASR trained on non-native speech.\nThe provided small validation set prevents us from carrying out a complex\nvalidation, but we submit all the unselected candidates for contrastive\nevaluation on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03679v1",
            "date": "2020-06-05 20:52:55+00:00",
            "title": "Prague Dependency Treebank -- Consolidated 1.0",
            "authors": [
                "Jan Haji\u010d",
                "Eduard Bej\u010dek",
                "Jaroslava Hlav\u00e1\u010dov\u00e1",
                "Marie Mikulov\u00e1",
                "Milan Straka",
                "Jan \u0160t\u011bp\u00e1nek",
                "Barbora \u0160t\u011bp\u00e1nkov\u00e1"
            ],
            "abstract": "We present a richly annotated and genre-diversified language resource, the\nPrague Dependency Treebank-Consolidated 1.0 (PDT-C 1.0), the purpose of which\nis - as it always been the case for the family of the Prague Dependency\nTreebanks - to serve both as a training data for various types of NLP tasks as\nwell as for linguistically-oriented research. PDT-C 1.0 contains four different\ndatasets of Czech, uniformly annotated using the standard PDT scheme (albeit\nnot everything is annotated manually, as we describe in detail here). The texts\ncome from different sources: daily newspaper articles, Czech translation of the\nWall Street Journal, transcribed dialogs and a small amount of user-generated,\nshort, often non-standard language segments typed into a web translator.\nAltogether, the treebank contains around 180,000 sentences with their\nmorphological, surface and deep syntactic annotation. The diversity of the\ntexts and annotations should serve well the NLP applications as well as it is\nan invaluable resource for linguistic research, including comparative studies\nregarding texts of different genres. The corpus is publicly and freely\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03687v1",
            "date": "2020-06-05 21:03:35+00:00",
            "title": "UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings",
            "authors": [
                "Milan Straka",
                "Jana Strakov\u00e1"
            ],
            "abstract": "We present our contribution to the EvaLatin shared task, which is the first\nevaluation campaign devoted to the evaluation of NLP tools for Latin. We\nsubmitted a system based on UDPipe 2.0, one of the winners of the CoNLL 2018\nShared Task, The 2018 Shared Task on Extrinsic Parser Evaluation and SIGMORPHON\n2019 Shared Task. Our system places first by a wide margin both in\nlemmatization and POS tagging in the open modality, where additional supervised\ndata is allowed, in which case we utilize all Universal Dependency Latin\ntreebanks. In the closed modality, where only the EvaLatin training data is\nallowed, our system achieves the best performance in lemmatization and in\nclassical subtask of POS tagging, while reaching second place in cross-genre\nand cross-time settings. In the ablation experiments, we also evaluate the\ninfluence of BERT and XLM-RoBERTa contextualized embeddings, and the treebank\nencodings of the different flavors of Latin treebanks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03866v1",
            "date": "2020-06-06 13:37:51+00:00",
            "title": "A Cross-Task Analysis of Text Span Representations",
            "authors": [
                "Shubham Toshniwal",
                "Haoyue Shi",
                "Bowen Shi",
                "Lingyu Gao",
                "Karen Livescu",
                "Kevin Gimpel"
            ],
            "abstract": "Many natural language processing (NLP) tasks involve reasoning with textual\nspans, including question answering, entity recognition, and coreference\nresolution. While extensive research has focused on functional architectures\nfor representing words and sentences, there is less work on representing\narbitrary spans of text within sentences. In this paper, we conduct a\ncomprehensive empirical evaluation of six span representation methods using\neight pretrained language representation models across six tasks, including two\ntasks that we introduce. We find that, although some simple span\nrepresentations are fairly reliable across tasks, in general the optimal span\nrepresentation varies by task, and can also vary within different facets of\nindividual tasks. We also find that the choice of span representation has a\nbigger impact with a fixed pretrained encoder than with a fine-tuned encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04014v1",
            "date": "2020-06-07 01:17:18+00:00",
            "title": "Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings",
            "authors": [
                "Katikapalli Subramanyam Kalyan",
                "S. Sangeetha"
            ],
            "abstract": "Medical concept normalization helps in discovering standard concepts in\nfree-form text i.e., maps health-related mentions to standard concepts in a\nvocabulary. It is much beyond simple string matching and requires a deep\nsemantic understanding of concept mentions. Recent research approach concept\nnormalization as either text classification or text matching. The main drawback\nin existing a) text classification approaches is ignoring valuable target\nconcepts information in learning input concept mention representation b) text\nmatching approach is the need to separately generate target concept embeddings\nwhich is time and resource consuming. Our proposed model overcomes these\ndrawbacks by jointly learning the representations of input concept mention and\ntarget concepts. First, it learns the input concept mention representation\nusing RoBERTa. Second, it finds cosine similarity between embeddings of input\nconcept mention and all the target concepts. Here, embeddings of target\nconcepts are randomly initialized and then updated during training. Finally,\nthe target concept with maximum cosine similarity is assigned to the input\nconcept mention. Our model surpasses all the existing methods across three\nstandard datasets by improving accuracy up to 2.31%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04229v2",
            "date": "2020-06-07 18:48:58+00:00",
            "title": "Pre-training Polish Transformer-based Language Models at Scale",
            "authors": [
                "S\u0142awomir Dadas",
                "Micha\u0142 Pere\u0142kiewicz",
                "Rafa\u0142 Po\u015bwiata"
            ],
            "abstract": "Transformer-based language models are now widely used in Natural Language\nProcessing (NLP). This statement is especially true for English language, in\nwhich many pre-trained models utilizing transformer-based architecture have\nbeen published in recent years. This has driven forward the state of the art\nfor a variety of standard NLP tasks such as classification, regression, and\nsequence labeling, as well as text-to-text tasks, such as machine translation,\nquestion answering, or summarization. The situation have been different for\nlow-resource languages, such as Polish, however. Although some\ntransformer-based language models for Polish are available, none of them have\ncome close to the scale, in terms of corpus size and the number of parameters,\nof the largest English-language models. In this study, we present two language\nmodels for Polish based on the popular BERT architecture. The larger model was\ntrained on a dataset consisting of over 1 billion polish sentences, or 135GB of\nraw text. We describe our methodology for collecting the data, preparing the\ncorpus, and pre-training the model. We then evaluate our models on thirteen\nPolish linguistic tasks, and demonstrate improvements over previous approaches\nin eleven of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04721v1",
            "date": "2020-06-08 16:24:03+00:00",
            "title": "Modeling Discourse Structure for Document-level Neural Machine Translation",
            "authors": [
                "Junxuan Chen",
                "Xiang Li",
                "Jiarui Zhang",
                "Chulun Zhou",
                "Jianwei Cui",
                "Bin Wang",
                "Jinsong Su"
            ],
            "abstract": "Recently, document-level neural machine translation (NMT) has become a hot\ntopic in the community of machine translation. Despite its success, most of\nexisting studies ignored the discourse structure information of the input\ndocument to be translated, which has shown effective in other tasks. In this\npaper, we propose to improve document-level NMT with the aid of discourse\nstructure information. Our encoder is based on a hierarchical attention network\n(HAN). Specifically, we first parse the input document to obtain its discourse\nstructure. Then, we introduce a Transformer-based path encoder to embed the\ndiscourse structure information of each word. Finally, we combine the discourse\nstructure information with the word embedding before it is fed into the\nencoder. Experimental results on the English-to-German dataset show that our\nmodel can significantly outperform both Transformer and Transformer+HAN.",
            "journal ref": "AutoSimTrans2020 camera-ready",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04781v1",
            "date": "2020-06-08 17:55:14+00:00",
            "title": "What's the Difference Between Professional Human and Machine Translation? A Blind Multi-language Study on Domain-specific MT",
            "authors": [
                "Lukas Fischer",
                "Samuel L\u00e4ubli"
            ],
            "abstract": "Machine translation (MT) has been shown to produce a number of errors that\nrequire human post-editing, but the extent to which professional human\ntranslation (HT) contains such errors has not yet been compared to MT. We\ncompile pre-translated documents in which MT and HT are interleaved, and ask\nprofessional translators to flag errors and post-edit these documents in a\nblind evaluation. We find that the post-editing effort for MT segments is only\nhigher in two out of three language pairs, and that the number of segments with\nwrong terminology, omissions, and typographical problems is similar in HT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05003v1",
            "date": "2020-06-09 01:13:57+00:00",
            "title": "Universal Vector Neural Machine Translation With Effective Attention",
            "authors": [
                "Satish Mylapore",
                "Ryan Quincy Paul",
                "Joshua Yi",
                "Robert D. Slater"
            ],
            "abstract": "Neural Machine Translation (NMT) leverages one or more trained neural\nnetworks for the translation of phrases. Sutskever introduced a sequence to\nsequence based encoder-decoder model which became the standard for NMT based\nsystems. Attention mechanisms were later introduced to address the issues with\nthe translation of long sentences and improving overall accuracy. In this\npaper, we propose a singular model for Neural Machine Translation based on\nencoder-decoder models. Most translation models are trained as one model for\none translation. We introduce a neutral/universal model representation that can\nbe used to predict more than one language depending on the source and a\nprovided target. Secondly, we introduce an attention model by adding an overall\nlearning vector to the multiplicative model. With these two changes, by using\nthe novel universal model the number of models needed for multiple language\ntranslation applications are reduced.",
            "journal ref": "SMU Data Science Review: Vol. 3 : No. 1 , Article 10. Available\n  at: https://scholar.smu.edu/datasciencereview/vol3/iss1/10 Year March 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05165v1",
            "date": "2020-06-09 10:12:16+00:00",
            "title": "Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Qiu Ran",
                "Yankai Lin",
                "Peng Li",
                "Jie Zhou"
            ],
            "abstract": "Non-autoregressive neural machine translation (NAT) predicts the entire\ntarget sequence simultaneously and significantly accelerates inference process.\nHowever, NAT discards the dependency information in a sentence, and thus\ninevitably suffers from the multi-modality problem: the target tokens may be\nprovided by different possible translations, often causing token repetitions or\nmissing. To alleviate this problem, we propose a novel semi-autoregressive\nmodel RecoverSAT in this work, which generates a translation as a sequence of\nsegments. The segments are generated simultaneously while each segment is\npredicted token-by-token. By dynamically determining segment length and\ndeleting repetitive segments, RecoverSAT is capable of recovering from\nrepetitive and missing token errors. Experimental results on three widely-used\nbenchmark datasets show that our proposed model achieves more than 4$\\times$\nspeedup while maintaining comparable performance compared with the\ncorresponding autoregressive model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05244v1",
            "date": "2020-06-09 13:28:57+00:00",
            "title": "Knowledge-Aided Open-Domain Question Answering",
            "authors": [
                "Mantong Zhou",
                "Zhouxing Shi",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "Open-domain question answering (QA) aims to find the answer to a question\nfrom a large collection of documents.Though many models for single-document\nmachine comprehension have achieved strong performance, there is still much\nroom for improving open-domain QA systems since document retrieval and answer\nreranking are still unsatisfactory. Golden documents that contain the correct\nanswers may not be correctly scored by the retrieval component, and the correct\nanswers that have been extracted may be wrongly ranked after other candidate\nanswers by the reranking component. One of the reasons is derived from the\nindependent principle in which each candidate document (or answer) is scored\nindependently without considering its relationship to other documents (or\nanswers). In this work, we propose a knowledge-aided open-domain QA (KAQA)\nmethod which targets at improving relevant document retrieval and candidate\nanswer reranking by considering the relationship between a question and the\ndocuments (termed as question-document graph), and the relationship between\ncandidate documents (termed as document-document graph). The graphs are built\nusing knowledge triples from external knowledge resources. During document\nretrieval, a candidate document is scored by considering its relationship to\nthe question and other documents. During answer reranking, a candidate answer\nis reranked using not only its own context but also the clues from other\ndocuments. The experimental results show that our proposed method improves\ndocument retrieval and answer reranking, and thereby enhances the overall\nperformance of open-domain question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05354v2",
            "date": "2020-06-09 15:38:21+00:00",
            "title": "Combination of abstractive and extractive approaches for summarization of long scientific texts",
            "authors": [
                "Vladislav Tretyak",
                "Denis Stepanov"
            ],
            "abstract": "In this research work, we present a method to generate summaries of long\nscientific documents that uses the advantages of both extractive and\nabstractive approaches. Before producing a summary in an abstractive manner, we\nperform the extractive step, which then is used for conditioning the abstractor\nmodule. We used pre-trained transformer-based language models, for both\nextractor and abstractor. Our experiments showed that using extractive and\nabstractive models jointly significantly improves summarization results and\nROUGE scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05489v2",
            "date": "2020-06-09 20:04:02+00:00",
            "title": "Modeling Label Semantics for Predicting Emotional Reactions",
            "authors": [
                "Radhika Gaonkar",
                "Heeyoung Kwon",
                "Mohaddeseh Bastan",
                "Niranjan Balasubramanian",
                "Nathanael Chambers"
            ],
            "abstract": "Predicting how events induce emotions in the characters of a story is\ntypically seen as a standard multi-label classification task, which usually\ntreats labels as anonymous classes to predict. They ignore information that may\nbe conveyed by the emotion labels themselves. We propose that the semantics of\nemotion labels can guide a model's attention when representing the input story.\nFurther, we observe that the emotions evoked by an event are often related: an\nevent that evokes joy is unlikely to also evoke sadness. In this work, we\nexplicitly model label classes via label embeddings, and add mechanisms that\ntrack label-label correlations both during training and inference. We also\nintroduce a new semi-supervision strategy that regularizes for the correlations\non unlabeled data. Our empirical evaluations show that modeling label semantics\nyields consistent benefits, and we advance the state-of-the-art on an emotion\ninference task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05602v1",
            "date": "2020-06-10 01:41:00+00:00",
            "title": "Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis",
            "authors": [
                "Yong Dai",
                "Jian Liu",
                "Xiancong Ren",
                "Zenglin Xu"
            ],
            "abstract": "Multi-source unsupervised domain adaptation (MS-UDA) for sentiment analysis\n(SA) aims to leverage useful information in multiple source domains to help do\nSA in an unlabeled target domain that has no supervised information. Existing\nalgorithms of MS-UDA either only exploit the shared features, i.e., the\ndomain-invariant information, or based on some weak assumption in NLP, e.g.,\nsmoothness assumption. To avoid these problems, we propose two transfer\nlearning frameworks based on the multi-source domain adaptation methodology for\nSA by combining the source hypotheses to derive a good target hypothesis. The\nkey feature of the first framework is a novel Weighting Scheme based\nUnsupervised Domain Adaptation framework (WS-UDA), which combine the source\nclassifiers to acquire pseudo labels for target instances directly. While the\nsecond framework is a Two-Stage Training based Unsupervised Domain Adaptation\nframework (2ST-UDA), which further exploits these pseudo labels to train a\ntarget private extractor. Importantly, the weights assigned to each source\nclassifier are based on the relations between target instances and source\ndomains, which measured by a discriminator through the adversarial training.\nFurthermore, through the same discriminator, we also fulfill the separation of\nshared features and private features. Experimental results on two SA datasets\ndemonstrate the promising performance of our frameworks, which outperforms\nunsupervised state-of-the-art competitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05621v1",
            "date": "2020-06-10 02:42:38+00:00",
            "title": "Understanding Points of Correspondence between Sentences for Abstractive Summarization",
            "authors": [
                "Logan Lebanoff",
                "John Muchovej",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Lidan Wang",
                "Walter Chang",
                "Fei Liu"
            ],
            "abstract": "Fusing sentences containing disparate content is a remarkable human ability\nthat helps create informative and succinct summaries. Such a simple task for\nhumans has remained challenging for modern abstractive summarizers,\nsubstantially restricting their applicability in real-world scenarios. In this\npaper, we present an investigation into fusing sentences drawn from a document\nby introducing the notion of points of correspondence, which are cohesive\ndevices that tie any two sentences together into a coherent text. The types of\npoints of correspondence are delineated by text cohesion theory, covering\npronominal and nominal referencing, repetition and beyond. We create a dataset\ncontaining the documents, source and fusion sentences, and human annotations of\npoints of correspondence between sentences. Our dataset bridges the gap between\ncoreference resolution and summarization. It is publicly shared to serve as a\nbasis for future work to measure the success of sentence fusion systems.\n(https://github.com/ucfnlp/points-of-correspondence)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06110v1",
            "date": "2020-06-10 23:29:05+00:00",
            "title": "Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols",
            "authors": [
                "Sarah E. Finch",
                "Jinho D. Choi"
            ],
            "abstract": "As conversational AI-based dialogue management has increasingly become a\ntrending topic, the need for a standardized and reliable evaluation procedure\ngrows even more pressing. The current state of affairs suggests various\nevaluation protocols to assess chat-oriented dialogue management systems,\nrendering it difficult to conduct fair comparative studies across different\napproaches and gain an insightful understanding of their values. To foster this\nresearch, a more robust evaluation protocol must be set in place. This paper\npresents a comprehensive synthesis of both automated and human evaluation\nmethods on dialogue systems, identifying their shortcomings while accumulating\nevidence towards the most effective evaluation dimensions. A total of 20 papers\nfrom the last two years are surveyed to analyze three types of evaluation\nprotocols: automated, static, and interactive. Finally, the evaluation\ndimensions used in these papers are compared against our expert evaluation on\nthe system-user dialogue data collected from the Alexa Prize 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06202v2",
            "date": "2020-06-11 05:25:18+00:00",
            "title": "A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages",
            "authors": [
                "Pedro Javier Ortiz Su\u00e1rez",
                "Laurent Romary",
                "Beno\u00eet Sagot"
            ],
            "abstract": "We use the multilingual OSCAR corpus, extracted from Common Crawl via\nlanguage classification, filtering and cleaning, to train monolingual\ncontextualized word embeddings (ELMo) for five mid-resource languages. We then\ncompare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for\nthese languages on the part-of-speech tagging and parsing tasks. We show that,\ndespite the noise in the Common-Crawl-based OSCAR data, embeddings trained on\nOSCAR perform much better than monolingual embeddings trained on Wikipedia.\nThey actually equal or improve the current state of the art in tagging and\nparsing for all five languages. In particular, they also improve over\nmultilingual Wikipedia-based contextual embeddings (multilingual BERT), which\nalmost always constitutes the previous state of the art, thereby showing that\nthe benefit of a larger, more diverse corpus surpasses the cross-lingual\nbenefit of multilingual embedding architectures.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association for\n  Computational Linguistics, July 2020, Online",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06226v1",
            "date": "2020-06-11 06:55:13+00:00",
            "title": "Discrete Latent Variable Representations for Low-Resource Text Classification",
            "authors": [
                "Shuning Jin",
                "Sam Wiseman",
                "Karl Stratos",
                "Karen Livescu"
            ],
            "abstract": "While much work on deep latent variable models of text uses continuous latent\nvariables, discrete latent variables are interesting because they are more\ninterpretable and typically more space efficient. We consider several\napproaches to learning discrete latent variable models for text in the case\nwhere exact marginalization over these variables is intractable. We compare the\nperformance of the learned representations as features for low-resource\ndocument and sentence classification. Our best models outperform the previous\nbest reported results with continuous representations in these low-resource\nsettings, while learning significantly more compressed representations.\nInterestingly, we find that an amortized variant of Hard EM performs\nparticularly well in the lowest-resource regimes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06259v1",
            "date": "2020-06-11 09:00:11+00:00",
            "title": "Augmenting Data for Sarcasm Detection with Unlabeled Conversation Context",
            "authors": [
                "Hankyol Lee",
                "Youngjae Yu",
                "Gunhee Kim"
            ],
            "abstract": "We present a novel data augmentation technique, CRA (Contextual Response\nAugmentation), which utilizes conversational context to generate meaningful\nsamples for training. We also mitigate the issues regarding unbalanced context\nlengths by changing the input-output format of the model such that it can deal\nwith varying context lengths effectively. Specifically, our proposed model,\ntrained with the proposed data augmentation technique, participated in the\nsarcasm detection task of FigLang2020, have won and achieves the best\nperformance in both Reddit and Twitter datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06264v2",
            "date": "2020-06-11 09:12:53+00:00",
            "title": "Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics",
            "authors": [
                "Nitika Mathur",
                "Timothy Baldwin",
                "Trevor Cohn"
            ],
            "abstract": "Automatic metrics are fundamental for the development and evaluation of\nmachine translation systems. Judging whether, and to what extent, automatic\nmetrics concur with the gold standard of human evaluation is not a\nstraightforward problem. We show that current methods for judging metrics are\nhighly sensitive to the translations used for assessment, particularly the\npresence of outliers, which often leads to falsely confident conclusions about\na metric's efficacy. Finally, we turn to pairwise system ranking, developing a\nmethod for thresholding performance improvement under an automatic metric\nagainst human judgements, which allows quantification of type I versus type II\nerrors incurred, i.e., insignificant human differences in system quality that\nare accepted, and significant human differences that are rejected. Together,\nthese findings suggest improvements to the protocols for metric evaluation and\nsystem performance evaluation in machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06341v2",
            "date": "2020-06-11 11:30:30+00:00",
            "title": "Provenance for Linguistic Corpora Through Nanopublications",
            "authors": [
                "Timo Lek",
                "Anna de Groot",
                "Tobias Kuhn",
                "Roser Morante"
            ],
            "abstract": "Research in Computational Linguistics is dependent on text corpora for\ntraining and testing new tools and methodologies. While there exists a plethora\nof annotated linguistic information, these corpora are often not interoperable\nwithout significant manual work. Moreover, these annotations might have evolved\ninto different versions, making it challenging for researchers to know the\ndata's provenance. This paper addresses this issue with a case study on event\nannotated corpora and by creating a new, more interoperable representation of\nthis data in the form of nanopublications. We demonstrate how linguistic\nannotations from separate corpora can be reliably linked from the start, and\nthereby be accessed and queried as if they were a single dataset. We describe\nhow such nanopublications can be created and demonstrate how SPARQL queries can\nbe performed to extract interesting content from the new representations. The\nqueries show that information of multiple corpora can be retrieved more easily\nand effectively because the information of different corpora is represented in\na uniform data format.",
            "journal ref": "In Proceedings of the 14th Linguistic Annotation Workshop (LAW),\n  co-located with COLING 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06402v2",
            "date": "2020-06-11 13:15:59+00:00",
            "title": "CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP",
            "authors": [
                "Libo Qin",
                "Minheng Ni",
                "Yue Zhang",
                "Wanxiang Che"
            ],
            "abstract": "Multi-lingual contextualized embeddings, such as multilingual-BERT (mBERT),\nhave shown success in a variety of zero-shot cross-lingual tasks. However,\nthese models are limited by having inconsistent contextualized representations\nof subwords across different languages. Existing work addresses this issue by\nbilingual projection and fine-tuning technique. We propose a data augmentation\nframework to generate multi-lingual code-switching data to fine-tune mBERT,\nwhich encourages model to align representations from source and multiple target\nlanguages once by mixing their context information. Compared with the existing\nwork, our method does not rely on bilingual sentences for training, and\nrequires only one training process for multiple target languages. Experimental\nresults on five tasks with 19 languages show that our method leads to\nsignificantly improved performances for all the tasks compared with mBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06478v2",
            "date": "2020-06-11 14:43:34+00:00",
            "title": "Multi-hop Reading Comprehension across Documents with Path-based Graph Convolutional Network",
            "authors": [
                "Zeyun Tang",
                "Yongliang Shen",
                "Xinyin Ma",
                "Wei Xu",
                "Jiale Yu",
                "Weiming Lu"
            ],
            "abstract": "Multi-hop reading comprehension across multiple documents attracts much\nattention recently. In this paper, we propose a novel approach to tackle this\nmulti-hop reading comprehension problem. Inspired by human reasoning\nprocessing, we construct a path-based reasoning graph from supporting\ndocuments. This graph can combine both the idea of the graph-based and\npath-based approaches, so it is better for multi-hop reasoning. Meanwhile, we\npropose Gated-RGCN to accumulate evidence on the path-based reasoning graph,\nwhich contains a new question-aware gating mechanism to regulate the usefulness\nof information propagating across documents and add question information during\nreasoning. We evaluate our approach on WikiHop dataset, and our approach\nachieves state-of-the-art accuracy against previously published approaches.\nEspecially, our ensemble model surpasses human performance by 4.2%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07015v1",
            "date": "2020-06-12 08:59:10+00:00",
            "title": "Speaker Sensitive Response Evaluation Model",
            "authors": [
                "JinYeong Bak",
                "Alice Oh"
            ],
            "abstract": "Automatic evaluation of open-domain dialogue response generation is very\nchallenging because there are many appropriate responses for a given context.\nExisting evaluation models merely compare the generated response with the\nground truth response and rate many of the appropriate responses as\ninappropriate if they deviate from the ground truth. One approach to resolve\nthis problem is to consider the similarity of the generated response with the\nconversational context. In this paper, we propose an automatic evaluation model\nbased on that idea and learn the model parameters from an unlabeled\nconversation corpus. Our approach considers the speakers in defining the\ndifferent levels of similar context. We use a Twitter conversation corpus that\ncontains many speakers and conversations to test our evaluation model.\nExperiments show that our model outperforms the other existing evaluation\nmetrics in terms of high correlation with human annotation scores. We also show\nthat our model trained on Twitter can be applied to movie dialogues without any\nadditional training. We provide our code and the learned parameters so that\nthey can be used for automatic evaluation of dialogue response generation\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07235v2",
            "date": "2020-06-12 14:39:40+00:00",
            "title": "SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)",
            "authors": [
                "Marcos Zampieri",
                "Preslav Nakov",
                "Sara Rosenthal",
                "Pepa Atanasova",
                "Georgi Karadzhov",
                "Hamdy Mubarak",
                "Leon Derczynski",
                "Zeses Pitenis",
                "\u00c7a\u011fr\u0131 \u00c7\u00f6ltekin"
            ],
            "abstract": "We present the results and main findings of SemEval-2020 Task 12 on\nMultilingual Offensive Language Identification in Social Media (OffensEval\n2020). The task involves three subtasks corresponding to the hierarchical\ntaxonomy of the OLID schema (Zampieri et al., 2019a) from OffensEval 2019. The\ntask featured five languages: English, Arabic, Danish, Greek, and Turkish for\nSubtask A. In addition, English also featured Subtasks B and C. OffensEval 2020\nwas one of the most popular tasks at SemEval-2020 attracting a large number of\nparticipants across all subtasks and also across all languages. A total of 528\nteams signed up to participate in the task, 145 teams submitted systems during\nthe evaluation period, and 70 submitted system description papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07264v1",
            "date": "2020-06-12 15:21:57+00:00",
            "title": "Low-resource Languages: A Review of Past Work and Future Challenges",
            "authors": [
                "Alexandre Magueresse",
                "Vincent Carles",
                "Evan Heetderks"
            ],
            "abstract": "A current problem in NLP is massaging and processing low-resource languages\nwhich lack useful training attributes such as supervised data, number of native\nspeakers or experts, etc. This review paper concisely summarizes previous\ngroundbreaking achievements made towards resolving this problem, and analyzes\npotential improvements in the context of the overall future research direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07296v6",
            "date": "2020-06-12 16:25:45+00:00",
            "title": "Information Extraction of Clinical Trial Eligibility Criteria",
            "authors": [
                "Yitong Tseo",
                "M. I. Salkola",
                "Ahmed Mohamed",
                "Anuj Kumar",
                "Freddy Abnousi"
            ],
            "abstract": "Clinical trials predicate subject eligibility on a diversity of criteria\nranging from patient demographics to food allergies. Trials post their\nrequirements as semantically complex, unstructured free-text. Formalizing trial\ncriteria to a computer-interpretable syntax would facilitate eligibility\ndetermination. In this paper, we investigate an information extraction (IE)\napproach for grounding criteria from trials in ClinicalTrials(dot)gov to a\nshared knowledge base. We frame the problem as a novel knowledge base\npopulation task, and implement a solution combining machine learning and\ncontext free grammar. To our knowledge, this work is the first criteria\nextraction system to apply attention-based conditional random field\narchitecture for named entity recognition (NER), and word2vec embedding\nclustering for named entity linking (NEL). We release the resources and core\ncomponents of our system on GitHub at\nhttps://github.com/facebookresearch/Clinical-Trial-Parser. Finally, we report\nour per module and end to end performances; we conclude that our system is\ncompetitive with Criteria2Query, which we view as the current state-of-the-art\nin criteria extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07425v2",
            "date": "2020-06-12 19:04:10+00:00",
            "title": "Measuring Forecasting Skill from Text",
            "authors": [
                "Shi Zong",
                "Alan Ritter",
                "Eduard Hovy"
            ],
            "abstract": "People vary in their ability to make accurate predictions about the future.\nPrior studies have shown that some individuals can predict the outcome of\nfuture events with consistently better accuracy. This leads to a natural\nquestion: what makes some forecasters better than others? In this paper we\nexplore connections between the language people use to describe their\npredictions and their forecasting skill. Datasets from two different\nforecasting domains are explored: (1) geopolitical forecasts from Good Judgment\nOpen, an online prediction forum and (2) a corpus of company earnings forecasts\nmade by financial analysts. We present a number of linguistic metrics which are\ncomputed over text associated with people's predictions about the future\nincluding: uncertainty, readability, and emotion. By studying linguistic\nfactors associated with predictions, we are able to shed some light on the\napproach taken by skilled forecasters. Furthermore, we demonstrate that it is\npossible to accurately predict forecasting skill using a model that is based\nsolely on language. This could potentially be useful for identifying accurate\npredictions or potentially skilled forecasters earlier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07667v1",
            "date": "2020-06-13 15:48:15+00:00",
            "title": "Words ranking and Hirsch index for identifying the core of the hapaxes in political texts",
            "authors": [
                "Valerio Ficcadenti",
                "Roy Cerqueti",
                "Marcel Ausloos",
                "Gurjeet Dhesi"
            ],
            "abstract": "This paper deals with a quantitative analysis of the content of official\npolitical speeches. We study a set of about one thousand talks pronounced by\nthe US Presidents, ranging from Washington to Trump. In particular, we search\nfor the relevance of the rare words, i.e. those said only once in each speech\n-- the so-called hapaxes. We implement a rank-size procedure of Zipf-Mandelbrot\ntype for discussing the hapaxes' frequencies regularity over the overall set of\nspeeches. Starting from the obtained rank-size law, we define and detect the\ncore of the hapaxes set by means of a procedure based on an Hirsch index\nvariant. We discuss the resulting list of words in the light of the overall US\nPresidents' speeches. We further show that this core of hapaxes itself can be\nwell fitted through a Zipf-Mandelbrot law and that contains elements producing\ndeviations at the low ranks between scatter plots and fitted curve -- the\nso-called king and vice-roy effect. Some socio-political insights are derived\nfrom the obtained findings about the US Presidents messages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07804v1",
            "date": "2020-06-14 05:19:46+00:00",
            "title": "Vietnamese Word Segmentation with SVM: Ambiguity Reduction and Suffix Capture",
            "authors": [
                "Duc-Vu Nguyen",
                "Dang Van Thin",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In this paper, we approach Vietnamese word segmentation as a binary\nclassification by using the Support Vector Machine classifier. We inherit\nfeatures from prior works such as n-gram of syllables, n-gram of syllable\ntypes, and checking conjunction of adjacent syllables in the dictionary. We\npropose two novel ways to feature extraction, one to reduce the overlap\nambiguity and the other to increase the ability to predict unknown words\ncontaining suffixes. Different from UETsegmenter and RDRsegmenter, two\nstate-of-the-art Vietnamese word segmentation methods, we do not employ the\nlongest matching algorithm as an initial processing step or any post-processing\ntechnique. According to experimental results on benchmark Vietnamese datasets,\nour proposed method obtained a better F1-score than the prior state-of-the-art\nmethods UETsegmenter, and RDRsegmenter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07890v1",
            "date": "2020-06-14 12:54:01+00:00",
            "title": "FinEst BERT and CroSloEngual BERT: less is more in multilingual models",
            "authors": [
                "Matej Ul\u010dar",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "Large pretrained masked language models have become state-of-the-art\nsolutions for many NLP problems. The research has been mostly focused on\nEnglish language, though. While massively multilingual models exist, studies\nhave shown that monolingual models produce much better results. We train two\ntrilingual BERT-like models, one for Finnish, Estonian, and English, the other\nfor Croatian, Slovenian, and English. We evaluate their performance on several\ndownstream tasks, NER, POS-tagging, and dependency parsing, using the\nmultilingual BERT and XLM-R as baselines. The newly created FinEst BERT and\nCroSloEngual BERT improve the results on all tasks in most monolingual and\ncross-lingual situations",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08097v2",
            "date": "2020-06-15 02:51:06+00:00",
            "title": "FinBERT: A Pretrained Language Model for Financial Communications",
            "authors": [
                "Yi Yang",
                "Mark Christopher Siy UY",
                "Allen Huang"
            ],
            "abstract": "Contextual pretrained language models, such as BERT (Devlin et al., 2019),\nhave made significant breakthrough in various NLP tasks by training on large\nscale of unlabeled text re-sources.Financial sector also accumulates large\namount of financial communication text.However, there is no pretrained finance\nspecific language models available. In this work,we address the need by\npretraining a financial domain specific BERT models, FinBERT, using a large\nscale of financial communication corpora. Experiments on three financial\nsentiment classification tasks confirm the advantage of FinBERT over generic\ndomain BERT model. The code and pretrained models are available at\nhttps://github.com/yya518/FinBERT. We hope this will be useful for\npractitioners and researchers working on financial NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08101v1",
            "date": "2020-06-15 02:59:52+00:00",
            "title": "Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder",
            "authors": [
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Jian Yin",
                "Daxin Jiang",
                "Ming Zhou"
            ],
            "abstract": "Generating inferential texts about an event in different perspectives\nrequires reasoning over different contexts that the event occurs. Existing\nworks usually ignore the context that is not explicitly provided, resulting in\na context-independent semantic representation that struggles to support the\ngeneration. To address this, we propose an approach that automatically finds\nevidence for an event from a large text corpus, and leverages the evidence to\nguide the generation of inferential texts. Our approach works in an\nencoder-decoder manner and is equipped with a Vector Quantised-Variational\nAutoencoder, where the encoder outputs representations from a distribution over\ndiscrete variables. Such discrete representations enable automatically\nselecting relevant evidence, which not only facilitates evidence-aware\ngeneration, but also provides a natural way to uncover rationales behind the\ngeneration. Our approach provides state-of-the-art performance on both\nEvent2Mind and ATOMIC datasets. More importantly, we find that with discrete\nrepresentations, our model selectively uses evidence to generate different\ninferential texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08185v1",
            "date": "2020-06-15 07:23:58+00:00",
            "title": "Extracting N-ary Cross-sentence Relations using Constrained Subsequence Kernel",
            "authors": [
                "Sachin Pawar",
                "Pushpak Bhattacharyya",
                "Girish K. Palshikar"
            ],
            "abstract": "Most of the past work in relation extraction deals with relations occurring\nwithin a sentence and having only two entity arguments. We propose a new\nformulation of the relation extraction task where the relations are more\ngeneral than intra-sentence relations in the sense that they may span multiple\nsentences and may have more than two arguments. Moreover, the relations are\nmore specific than corpus-level relations in the sense that their scope is\nlimited only within a document and not valid globally throughout the corpus. We\npropose a novel sequence representation to characterize instances of such\nrelations. We then explore various classifiers whose features are derived from\nthis sequence representation. For SVM classifier, we design a Constrained\nSubsequence Kernel which is a variant of Generalized Subsequence Kernel. We\nevaluate our approach on three datasets across two domains: biomedical and\ngeneral domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08297v1",
            "date": "2020-06-15 11:47:00+00:00",
            "title": "Fine-grained Human Evaluation of Transformer and Recurrent Approaches to Neural Machine Translation for English-to-Chinese",
            "authors": [
                "Yuying Ye",
                "Antonio Toral"
            ],
            "abstract": "This research presents a fine-grained human evaluation to compare the\nTransformer and recurrent approaches to neural machine translation (MT), on the\ntranslation direction English-to-Chinese. To this end, we develop an error\ntaxonomy compliant with the Multidimensional Quality Metrics (MQM) framework\nthat is customised to the relevant phenomena of this translation direction. We\nthen conduct an error annotation using this customised error taxonomy on the\noutput of state-of-the-art recurrent- and Transformer-based MT systems on a\nsubset of WMT2019's news test set. The resulting annotation shows that,\ncompared to the best recurrent system, the best Transformer system results in a\n31% reduction of the total number of errors and it produced significantly less\nerrors in 10 out of 22 error categories. We also note that two of the systems\nevaluated do not produce any error for a category that was relevant for this\ntranslation direction prior to the advent of NMT systems: Chinese classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08332v1",
            "date": "2020-06-09 17:01:55+00:00",
            "title": "An Augmented Translation Technique for low Resource language pair: Sanskrit to Hindi translation",
            "authors": [
                "Rashi Kumar",
                "Piyush Jha",
                "Vineet Sahula"
            ],
            "abstract": "Neural Machine Translation (NMT) is an ongoing technique for Machine\nTranslation (MT) using enormous artificial neural network. It has exhibited\npromising outcomes and has shown incredible potential in solving challenging\nmachine translation exercises. One such exercise is the best approach to\nfurnish great MT to language sets with a little preparing information. In this\nwork, Zero Shot Translation (ZST) is inspected for a low resource language\npair. By working on high resource language pairs for which benchmarks are\navailable, namely Spanish to Portuguese, and training on data sets\n(Spanish-English and English-Portuguese) we prepare a state of proof for ZST\nsystem that gives appropriate results on the available data. Subsequently the\nsame architecture is tested for Sanskrit to Hindi translation for which data is\nsparse, by training the model on English-Hindi and Sanskrit-English language\npairs. In order to prepare and decipher with ZST system, we broaden the\npreparation and interpretation pipelines of NMT seq2seq model in tensorflow,\nincorporating ZST features. Dimensionality reduction of word embedding is\nperformed to reduce the memory usage for data storage and to achieve a faster\ntraining and translation cycles. In this work existing helpful technology has\nbeen utilized in an imaginative manner to execute our NLP issue of Sanskrit to\nHindi translation. A Sanskrit-Hindi parallel corpus of 300 is constructed for\ntesting. The data required for the construction of parallel corpus has been\ntaken from the telecasted news, published on Department of Public Information,\nstate government of Madhya Pradesh, India website.",
            "journal ref": "Proceedings of the 2019 2nd International Conference on\n  Algorithms, Computing and Artificial Intelligence, December 2019, pp 377 to\n  383",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08337v2",
            "date": "2020-06-02 04:31:09+00:00",
            "title": "Open-Domain Question Answering with Pre-Constructed Question Spaces",
            "authors": [
                "Jinfeng Xiao",
                "Lidan Wang",
                "Franck Dernoncourt",
                "Trung Bui",
                "Tong Sun",
                "Jiawei Han"
            ],
            "abstract": "Open-domain question answering aims at solving the task of locating the\nanswers to user-generated questions in massive collections of documents. There\nare two families of solutions available: retriever-readers, and\nknowledge-graph-based approaches. A retriever-reader usually first uses\ninformation retrieval methods like TF-IDF to locate some documents or\nparagraphs that are likely to be relevant to the question, and then feeds the\nretrieved text to a neural network reader to extract the answer. Alternatively,\nknowledge graphs can be constructed from the corpus and be queried against to\nanswer user questions. We propose a novel algorithm with a reader-retriever\nstructure that differs from both families. Our reader-retriever first uses an\noffline reader to read the corpus and generate collections of all answerable\nquestions associated with their answers, and then uses an online retriever to\nrespond to user queries by searching the pre-constructed question spaces for\nanswers that are most likely to be asked in the given way. We further combine\nretriever-reader and reader-retriever results into one single answer by\nexamining the consistency between the two components. We claim that our\nalgorithm solves some bottlenecks in existing work, and demonstrate that it\nachieves superior accuracy on real-world datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08342v2",
            "date": "2020-06-02 13:48:14+00:00",
            "title": "Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity",
            "authors": [
                "Lukas Muttenthaler"
            ],
            "abstract": "Understanding subjectivity demands reasoning skills beyond the realm of\ncommon knowledge. It requires a machine learning model to process sentiment and\nto perform opinion mining. In this work, I've exploited a recently released\ndataset for span-selection Question Answering, namely SubjQA. SubjQA is the\nfirst QA dataset that contains questions that ask for subjective opinions\ncorresponding to review paragraphs from six different domains. Hence, to answer\nthese subjective questions, a learner must extract opinions and process\nsentiment for various domains, and additionally, align the knowledge extracted\nfrom a paragraph with the natural language utterances in the corresponding\nquestion, which together enhance the difficulty of a QA task. The primary goal\nof this thesis was to investigate the inner workings (i.e., latent\nrepresentations) of a Transformer-based architecture to contribute to a better\nunderstanding of these not yet well understood \"black-box\" models.\nTransformer's hidden representations, concerning the true answer span, are\nclustered more closely in vector space than those representations corresponding\nto erroneous predictions. This observation holds across the top three\nTransformer layers for both objective and subjective questions and generally\nincreases as a function of layer dimensions. Moreover, the probability to\nachieve a high cosine similarity among hidden representations in latent space\nconcerning the true answer span tokens is significantly higher for correct\ncompared to incorrect answer span predictions. These results have decisive\nimplications for down-stream applications, where it is crucial to know about\nwhy a neural network made mistakes, and in which point, in space and time the\nmistake has happened (e.g., to automatically predict correctness of an answer\nspan prediction without the necessity of labeled data).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08387v2",
            "date": "2020-06-15 13:20:13+00:00",
            "title": "Catplayinginthesnow: Impact of Prior Segmentation on a Model of Visually Grounded Speech",
            "authors": [
                "William N. Havard",
                "Jean-Pierre Chevrot",
                "Laurent Besacier"
            ],
            "abstract": "The language acquisition literature shows that children do not build their\nlexicon by segmenting the spoken input into phonemes and then building up words\nfrom them, but rather adopt a top-down approach and start by segmenting\nword-like units and then break them down into smaller units. This suggests that\nthe ideal way of learning a language is by starting from full semantic units.\nIn this paper, we investigate if this is also the case for a neural model of\nVisually Grounded Speech trained on a speech-image retrieval task. We evaluated\nhow well such a network is able to learn a reliable speech-to-image mapping\nwhen provided with phone, syllable, or word boundary information. We present a\nsimple way to introduce such information into an RNN-based model and\ninvestigate which type of boundary is the most efficient. We also explore at\nwhich level of the network's architecture such information should be introduced\nso as to maximise its performances. Finally, we show that using multiple\nboundary types at once in a hierarchical structure, by which low-level segments\nare used to recompose high-level segments, is beneficial and yields better\nresults than using low-level or high-level segments in isolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08748v1",
            "date": "2020-06-15 20:40:06+00:00",
            "title": "DynE: Dynamic Ensemble Decoding for Multi-Document Summarization",
            "authors": [
                "Chris Hokamp",
                "Demian Gholipour Ghalandari",
                "Nghia The Pham",
                "John Glover"
            ],
            "abstract": "Sequence-to-sequence (s2s) models are the basis for extensive work in natural\nlanguage processing. However, some applications, such as multi-document\nsummarization, multi-modal machine translation, and the automatic post-editing\nof machine translation, require mapping a set of multiple distinct inputs into\na single output sequence. Recent work has introduced bespoke architectures for\nthese multi-input settings, and developed models which can handle increasingly\nlonger inputs; however, the performance of special model architectures is\nlimited by the available in-domain training data. In this work we propose a\nsimple decoding methodology which ensembles the output of multiple instances of\nthe same model on different inputs. Our proposed approach allows models trained\nfor vanilla s2s tasks to be directly used in multi-input settings. This works\nparticularly well when each of the inputs has significant overlap with the\nothers, as when compressing a cluster of news articles about the same event\ninto a single coherent summary, and we obtain state-of-the-art results on\nseveral multi-document summarization datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08881v1",
            "date": "2020-06-16 02:41:46+00:00",
            "title": "Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation",
            "authors": [
                "Kellie Webster",
                "Emily Pitler"
            ],
            "abstract": "Machine translation systems with inadequate document understanding can make\nerrors when translating dropped or neutral pronouns into languages with\ngendered pronouns (e.g., English). Predicting the underlying gender of these\npronouns is difficult since it is not marked textually and must instead be\ninferred from coreferent mentions in the context. We propose a novel\ncross-lingual pivoting technique for automatically producing high-quality\ngender labels, and show that this data can be used to fine-tune a BERT\nclassifier with 92% F1 for Spanish dropped feminine pronouns, compared with\n30-51% for neural machine translation models and 54-71% for a non-fine-tuned\nBERT model. We augment a neural machine translation model with labels from our\nclassifier to improve pronoun translation, while still having parallelizable\ntranslation models that translate a sentence at a time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09035v1",
            "date": "2020-06-16 09:57:40+00:00",
            "title": "The SPPD System for Schema Guided Dialogue State Tracking Challenge",
            "authors": [
                "Miao Li",
                "Haoqi Xiong",
                "Yunbo Cao"
            ],
            "abstract": "This paper introduces one of our group's work on the Dialog System Technology\nChallenges 8 (DSTC8), the SPPD system for Schema Guided dialogue state tracking\nchallenge. This challenge, named as Track 4 in DSTC8, provides a brand new and\nchallenging dataset for developing scalable multi-domain dialogue state\ntracking algorithms for real world dialogue systems. We propose a zero-shot\ndialogue state tracking system for this task. The key components of the system\nis a number of BERT based zero-shot NLU models that can effectively capture\nsemantic relations between natural language descriptions of services' schemas\nand utterances from dialogue turns. We also propose some strategies to make the\nsystem better to exploit information from longer dialogue history and to\novercome the slot carryover problem for multi-domain dialogues. The\nexperimental results show that the proposed system achieves a significant\nimprovement compared with the baseline system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09109v2",
            "date": "2020-06-16 12:37:50+00:00",
            "title": "How to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation",
            "authors": [
                "Steffen Eger",
                "Johannes Daxenberger",
                "Iryna Gurevych"
            ],
            "abstract": "Sentence encoders map sentences to real valued vectors for use in downstream\napplications. To peek into these representations - e.g., to increase\ninterpretability of their results - probing tasks have been designed which\nquery them for linguistic knowledge. However, designing probing tasks for\nlesser-resourced languages is tricky, because these often lack large-scale\nannotated data or (high-quality) dependency parsers as a prerequisite of\nprobing task design in English. To investigate how to probe sentence embeddings\nin such cases, we investigate sensitivity of probing task results to structural\ndesign choices, conducting the first such large scale study. We show that\ndesign choices like size of the annotated probing dataset and type of\nclassifier used for evaluation do (sometimes substantially) influence probing\noutcomes. We then probe embeddings in a multilingual setup with design choices\nthat lie in a 'stable region', as we identify for English, and find that\nresults on English do not transfer to other languages. Fairer and more\ncomprehensive sentence-level probing evaluation should thus be carried out on\nmultiple languages in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09235v1",
            "date": "2020-06-16 15:11:51+00:00",
            "title": "Weakly-supervised Domain Adaption for Aspect Extraction via Multi-level Interaction Transfer",
            "authors": [
                "Tao Liang",
                "Wenya Wang",
                "Fengmao Lv"
            ],
            "abstract": "Fine-grained aspect extraction is an essential sub-task in aspect based\nopinion analysis. It aims to identify the aspect terms (a.k.a. opinion targets)\nof a product or service in each sentence. However, expensive annotation process\nis usually involved to acquire sufficient token-level labels for each domain.\nTo address this limitation, some previous works propose domain adaptation\nstrategies to transfer knowledge from a sufficiently labeled source domain to\nunlabeled target domains. But due to both the difficulty of fine-grained\nprediction problems and the large domain gap between domains, the performance\nremains unsatisfactory. This work conducts a pioneer study on leveraging\nsentence-level aspect category labels that can be usually available in\ncommercial services like review sites to promote token-level transfer for the\nextraction purpose. Specifically, the aspect category information is used to\nconstruct pivot knowledge for transfer with assumption that the interactions\nbetween sentence-level aspect category and token-level aspect terms are\ninvariant across domains. To this end, we propose a novel multi-level\nreconstruction mechanism that aligns both the fine-grained and coarse-grained\ninformation in multiple levels of abstractions. Comprehensive experiments\ndemonstrate that our approach can fully utilize sentence-level aspect category\nlabels to improve cross-domain aspect extraction with a large performance gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09242v3",
            "date": "2020-06-16 15:20:04+00:00",
            "title": "Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs",
            "authors": [
                "Martin Schmitt",
                "Leonardo F. R. Ribeiro",
                "Philipp Dufter",
                "Iryna Gurevych",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "We present Graformer, a novel Transformer-based encoder-decoder architecture\nfor graph-to-text generation. With our novel graph self-attention, the encoding\nof a node relies on all nodes in the input graph - not only direct neighbors -\nfacilitating the detection of global patterns. We represent the relation\nbetween two nodes as the length of the shortest path between them. Graformer\nlearns to weight these node-node relations differently for different attention\nheads, thus virtually learning differently connected views of the input graph.\nWe evaluate Graformer on two popular graph-to-text generation benchmarks,\nAGENDA and WebNLG, where it achieves strong performance while using many fewer\nparameters than other approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09277v1",
            "date": "2020-06-16 16:11:04+00:00",
            "title": "Communicative need modulates competition in language change",
            "authors": [
                "Andres Karjus",
                "Richard A. Blythe",
                "Simon Kirby",
                "Kenny Smith"
            ],
            "abstract": "All living languages change over time. The causes for this are many, one\nbeing the emergence and borrowing of new linguistic elements. Competition\nbetween the new elements and older ones with a similar semantic or grammatical\nfunction may lead to speakers preferring one of them, and leaving the other to\ngo out of use. We introduce a general method for quantifying competition\nbetween linguistic elements in diachronic corpora which does not require\nlanguage-specific resources other than a sufficiently large corpus. This\napproach is readily applicable to a wide range of languages and linguistic\nsubsystems. Here, we apply it to lexical data in five corpora differing in\nlanguage, type, genre, and time span. We find that changes in communicative\nneed are consistently predictive of lexical competition dynamics.\nNear-synonymous words are more likely to directly compete if they belong to a\ntopic of conversation whose importance to language users is constant over time,\npossibly leading to the extinction of one of the competing words. By contrast,\nin topics which are increasing in importance for language users,\nnear-synonymous words tend not to compete directly and can coexist. This\nsuggests that, in addition to direct competition between words, language change\ncan be driven by competition between topics or semantic subspaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09336v2",
            "date": "2020-06-16 17:20:25+00:00",
            "title": "Cross-Cultural Similarity Features for Cross-Lingual Transfer Learning of Pragmatically Motivated Tasks",
            "authors": [
                "Jimin Sun",
                "Hwijeen Ahn",
                "Chan Young Park",
                "Yulia Tsvetkov",
                "David R. Mortensen"
            ],
            "abstract": "Much work in cross-lingual transfer learning explored how to select better\ntransfer languages for multilingual tasks, primarily focusing on typological\nand genealogical similarities between languages. We hypothesize that these\nmeasures of linguistic proximity are not enough when working with\npragmatically-motivated tasks, such as sentiment analysis. As an alternative,\nwe introduce three linguistic features that capture cross-cultural similarities\nthat manifest in linguistic patterns and quantify distinct aspects of language\npragmatics: language context-level, figurative language, and the lexification\nof emotion concepts. Our analyses show that the proposed pragmatic features do\ncapture cross-cultural similarities and align well with existing work in\nsociolinguistics and linguistic anthropology. We further corroborate the\neffectiveness of pragmatically-driven transfer in the downstream task of\nchoosing transfer languages for cross-lingual sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09432v1",
            "date": "2020-06-16 18:23:58+00:00",
            "title": "The Role of Verb Semantics in Hungarian Verb-Object Order",
            "authors": [
                "Dorottya Demszky",
                "L\u00e1szl\u00f3 K\u00e1lm\u00e1n",
                "Dan Jurafsky",
                "Beth Levin"
            ],
            "abstract": "Hungarian is often referred to as a discourse-configurational language, since\nthe structural position of constituents is determined by their logical function\n(topic or comment) rather than their grammatical function (e.g., subject or\nobject). We build on work by Koml\\'osy (1989) and argue that in addition to\ndiscourse context, the lexical semantics of the verb also plays a significant\nrole in determining Hungarian word order. In order to investigate the role of\nlexical semantics in determining Hungarian word order, we conduct a\nlarge-scale, data-driven analysis on the ordering of 380 transitive verbs and\ntheir objects, as observed in hundreds of thousands of examples extracted from\nthe Hungarian Gigaword Corpus. We test the effect of lexical semantics on the\nordering of verbs and their objects by grouping verbs into 11 semantic classes.\nIn addition to the semantic class of the verb, we also include two control\nfeatures related to information structure, object definiteness and object NP\nweight, chosen to allow a comparison of their effect size to that of verb\nsemantics. Our results suggest that all three features have a significant\neffect on verb-object ordering in Hungarian and among these features, the\nsemantic class of the verb has the largest effect. Specifically, we find that\nstative verbs, such as fed \"cover\", jelent \"mean\" and \\\"ovez \"surround\", tend\nto be OV-preferring (with the exception of psych verbs which are strongly\nVO-preferring) and non-stative verbs, such as b\\'ir\\'al \"judge\", cs\\\"okkent\n\"reduce\" and cs\\'okol \"kiss\", verbs tend to be VO-preferring. These findings\nsupport our hypothesis that lexical semantic factors influence word order in\nHungarian.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09589v2",
            "date": "2020-06-17 01:21:19+00:00",
            "title": "Modeling Subjective Assessments of Guilt in Newspaper Crime Narratives",
            "authors": [
                "Elisa Kreiss",
                "Zijian Wang",
                "Christopher Potts"
            ],
            "abstract": "Crime reporting is a prevalent form of journalism with the power to shape\npublic perceptions and social policies. How does the language of these reports\nact on readers? We seek to address this question with the SuspectGuilt Corpus\nof annotated crime stories from English-language newspapers in the U.S. For\nSuspectGuilt, annotators read short crime articles and provided text-level\nratings concerning the guilt of the main suspect as well as span-level\nannotations indicating which parts of the story they felt most influenced their\nratings. SuspectGuilt thus provides a rich picture of how linguistic choices\naffect subjective guilt judgments. In addition, we use SuspectGuilt to train\nand assess predictive models, and show that these models benefit from genre\npretraining and joint supervision from the text-level ratings and span-level\nannotations. Such models might be used as tools for understanding the societal\neffects of crime reporting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09627v2",
            "date": "2020-06-17 03:24:38+00:00",
            "title": "Building Low-Resource NER Models Using Non-Speaker Annotation",
            "authors": [
                "Tatiana Tsygankova",
                "Francesca Marini",
                "Stephen Mayhew",
                "Dan Roth"
            ],
            "abstract": "In low-resource natural language processing (NLP), the key problems are a\nlack of target language training data, and a lack of native speakers to create\nit. Cross-lingual methods have had notable success in addressing these\nconcerns, but in certain common circumstances, such as insufficient\npre-training corpora or languages far from the source language, their\nperformance suffers. In this work we propose a complementary approach to\nbuilding low-resource Named Entity Recognition (NER) models using\n``non-speaker'' (NS) annotations, provided by annotators with no prior\nexperience in the target language. We recruit 30 participants in a carefully\ncontrolled annotation experiment with Indonesian, Russian, and Hindi. We show\nthat use of NS annotators produces results that are consistently on par or\nbetter than cross-lingual methods built on modern contextual representations,\nand have the potential to outperform with additional effort. We conclude with\nobservations of common annotation patterns and recommended implementation\npractices, and motivate how NS annotations can be used in addition to prior\nmethods for improved performance. For more details,\nhttp://cogcomp.org/page/publication_view/941",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09639v1",
            "date": "2020-06-17 03:53:12+00:00",
            "title": "Iterative Edit-Based Unsupervised Sentence Simplification",
            "authors": [
                "Dhruv Kumar",
                "Lili Mou",
                "Lukasz Golab",
                "Olga Vechtomova"
            ],
            "abstract": "We present a novel iterative, edit-based approach to unsupervised sentence\nsimplification. Our model is guided by a scoring function involving fluency,\nsimplicity, and meaning preservation. Then, we iteratively perform word and\nphrase-level edits on the complex sentence. Compared with previous approaches,\nour model does not require a parallel training set, but is more controllable\nand interpretable. Experiments on Newsela and WikiLarge datasets show that our\napproach is nearly as effective as state-of-the-art supervised approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09685v1",
            "date": "2020-06-17 07:02:42+00:00",
            "title": "Exploiting Review Neighbors for Contextualized Helpfulness Prediction",
            "authors": [
                "Jiahua Du",
                "Jia Rong",
                "Hua Wang",
                "Yanchun Zhang"
            ],
            "abstract": "Helpfulness prediction techniques have been widely used to identify and\nrecommend high-quality online reviews to customers. Currently, the vast\nmajority of studies assume that a review's helpfulness is self-contained. In\npractice, however, customers hardly process reviews independently given the\nsequential nature. The perceived helpfulness of a review is likely to be\naffected by its sequential neighbors (i.e., context), which has been largely\nignored. This paper proposes a new methodology to capture the missing\ninteraction between reviews and their neighbors. The first end-to-end neural\narchitecture is developed for neighbor-aware helpfulness prediction (NAP). For\neach review, NAP allows for three types of neighbor selection: its preceding,\nfollowing, and surrounding neighbors. Four weighting schemes are designed to\nlearn context clues from the selected neighbors. A review is then\ncontextualized into the learned clues for neighbor-aware helpfulness\nprediction. NAP is evaluated on six domains of real-world online reviews\nagainst a series of state-of-the-art baselines. Extensive experiments confirm\nthe effectiveness of NAP and the influence of sequential neighbors on a current\nreviews. Further hyperparameter analysis reveals three main findings. (1) On\naverage, eight neighbors treated with uneven importance are engaged for context\nconstruction. (2) The benefit of neighbor-aware prediction mainly results from\ncloser neighbors. (3) Equally considering up to five closest neighbors of a\nreview can usually produce a weaker but tolerable prediction result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09719v1",
            "date": "2020-06-17 08:40:52+00:00",
            "title": "Automatically Ranked Russian Paraphrase Corpus for Text Generation",
            "authors": [
                "Vadim Gudkov",
                "Olga Mitrofanova",
                "Elizaveta Filippskikh"
            ],
            "abstract": "The article is focused on automatic development and ranking of a large corpus\nfor Russian paraphrase generation which proves to be the first corpus of such\ntype in Russian computational linguistics. Existing manually annotated\nparaphrase datasets for Russian are limited to small-sized ParaPhraser corpus\nand ParaPlag which are suitable for a set of NLP tasks, such as paraphrase and\nplagiarism detection, sentence similarity and relatedness estimation, etc. Due\nto size restrictions, these datasets can hardly be applied in end-to-end text\ngeneration solutions. Meanwhile, paraphrase generation requires a large amount\nof training data. In our study we propose a solution to the problem: we\ncollect, rank and evaluate a new publicly available headline paraphrase corpus\n(ParaPhraser Plus), and then perform text generation experiments with manual\nevaluation on automatically ranked corpora using the Universal Transformer\narchitecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09766v1",
            "date": "2020-06-17 10:34:16+00:00",
            "title": "Improving unsupervised neural aspect extraction for online discussions using out-of-domain classification",
            "authors": [
                "Anton Alekseev",
                "Elena Tutubalina",
                "Valentin Malykh",
                "Sergey Nikolenko"
            ],
            "abstract": "Deep learning architectures based on self-attention have recently achieved\nand surpassed state of the art results in the task of unsupervised aspect\nextraction and topic modeling. While models such as neural attention-based\naspect extraction (ABAE) have been successfully applied to user-generated\ntexts, they are less coherent when applied to traditional data sources such as\nnews articles and newsgroup documents. In this work, we introduce a simple\napproach based on sentence filtering in order to improve topical aspects\nlearned from newsgroups-based content without modifying the basic mechanism of\nABAE. We train a probabilistic classifier to distinguish between out-of-domain\ntexts (outer dataset) and in-domain texts (target dataset). Then, during data\npreparation we filter out sentences that have a low probability of being\nin-domain and train the neural model on the remaining sentences. The positive\neffect of sentence filtering on topic coherence is demonstrated in comparison\nto aspect extraction models trained on unfiltered texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10093v1",
            "date": "2020-06-17 18:30:30+00:00",
            "title": "Extensively Matching for Few-shot Learning Event Detection",
            "authors": [
                "Viet Dac Lai",
                "Franck Dernoncourt",
                "Thien Huu Nguyen"
            ],
            "abstract": "Current event detection models under super-vised learning settings fail to\ntransfer to newevent types. Few-shot learning has not beenexplored in event\ndetection even though it al-lows a model to perform well with high\ngener-alization on new event types. In this work, weformulate event detection\nas a few-shot learn-ing problem to enable to extend event detec-tion to new\nevent types. We propose two novelloss factors that matching examples in the\nsup-port set to provide more training signals to themodel. Moreover, these\ntraining signals can beapplied in many metric-based few-shot learn-ing models.\nOur extensive experiments on theACE-2005 dataset (under a few-shot\nlearningsetting) show that the proposed method can im-prove the performance of\nfew-shot learning",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10157v1",
            "date": "2020-06-17 21:02:40+00:00",
            "title": "Is this Dialogue Coherent? Learning from Dialogue Acts and Entities",
            "authors": [
                "Alessandra Cervone",
                "Giuseppe Riccardi"
            ],
            "abstract": "In this work, we investigate the human perception of coherence in open-domain\ndialogues. In particular, we address the problem of annotating and modeling the\ncoherence of next-turn candidates while considering the entire history of the\ndialogue. First, we create the Switchboard Coherence (SWBD-Coh) corpus, a\ndataset of human-human spoken dialogues annotated with turn coherence ratings,\nwhere next-turn candidate utterances ratings are provided considering the full\ndialogue context. Our statistical analysis of the corpus indicates how turn\ncoherence perception is affected by patterns of distribution of entities\npreviously introduced and the Dialogue Acts used. Second, we experiment with\ndifferent architectures to model entities, Dialogue Acts and their combination\nand evaluate their performance in predicting human coherence ratings on\nSWBD-Coh. We find that models combining both DA and entity information yield\nthe best performances both for response selection and turn coherence rating.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10270v2",
            "date": "2020-06-18 04:24:28+00:00",
            "title": "Multi-branch Attentive Transformer",
            "authors": [
                "Yang Fan",
                "Shufang Xie",
                "Yingce Xia",
                "Lijun Wu",
                "Tao Qin",
                "Xiang-Yang Li",
                "Tie-Yan Liu"
            ],
            "abstract": "While the multi-branch architecture is one of the key ingredients to the\nsuccess of computer vision tasks, it has not been well investigated in natural\nlanguage processing, especially sequence learning tasks. In this work, we\npropose a simple yet effective variant of Transformer called multi-branch\nattentive Transformer (briefly, MAT), where the attention layer is the average\nof multiple branches and each branch is an independent multi-head attention\nlayer. We leverage two training techniques to regularize the training:\ndrop-branch, which randomly drops individual branches during training, and\nproximal initialization, which uses a pre-trained Transformer model to\ninitialize multiple branches. Experiments on machine translation, code\ngeneration and natural language understanding demonstrate that such a simple\nvariant of Transformer brings significant improvements. Our code is available\nat \\url{https://github.com/HA-Transformer}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10276v1",
            "date": "2020-06-18 04:53:07+00:00",
            "title": "Octet: Online Catalog Taxonomy Enrichment with Self-Supervision",
            "authors": [
                "Yuning Mao",
                "Tong Zhao",
                "Andrey Kan",
                "Chenwei Zhang",
                "Xin Luna Dong",
                "Christos Faloutsos",
                "Jiawei Han"
            ],
            "abstract": "Taxonomies have found wide applications in various domains, especially online\nfor item categorization, browsing, and search. Despite the prevalent use of\nonline catalog taxonomies, most of them in practice are maintained by humans,\nwhich is labor-intensive and difficult to scale. While taxonomy construction\nfrom scratch is considerably studied in the literature, how to effectively\nenrich existing incomplete taxonomies remains an open yet important research\nquestion. Taxonomy enrichment not only requires the robustness to deal with\nemerging terms but also the consistency between existing taxonomy structure and\nnew term attachment. In this paper, we present a self-supervised end-to-end\nframework, Octet, for Online Catalog Taxonomy EnrichmenT. Octet leverages\nheterogeneous information unique to online catalog taxonomies such as user\nqueries, items, and their relations to the taxonomy nodes while requiring no\nother supervision than the existing taxonomies. We propose to distantly train a\nsequence labeling model for term extraction and employ graph neural networks\n(GNNs) to capture the taxonomy structure as well as the query-item-taxonomy\ninteractions for term attachment. Extensive experiments in different online\ndomains demonstrate the superiority of Octet over state-of-the-art methods via\nboth automatic and human evaluations. Notably, Octet enriches an online catalog\ntaxonomy in production to 2 times larger in the open-world evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10369v3",
            "date": "2020-06-18 09:06:49+00:00",
            "title": "Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation",
            "authors": [
                "Jungo Kasai",
                "Nikolaos Pappas",
                "Hao Peng",
                "James Cross",
                "Noah A. Smith"
            ],
            "abstract": "During the recent years, much effort has been invested in non-autoregressive\nneural machine translation, which appears to be an efficient alternative to\nstate-of-the-art autoregressive machine translation on modern GPUs. In contrast\nto the latter where generation is sequential, the former allows generation to\nbe parallelized across target token positions. Non-autoregressive machine\ntranslation provides a tradeoff between translation quality and inference\nspeed, but some of the latest models have achieved impressive tradeoffs\ncompared to autoregressive baselines. In this work, we reexamine this tradeoff\nand argue that autoregressive baselines can be substantially sped up without\nloss in accuracy. Specifically, we study autoregressive models with encoders\nand decoders of varied depths. Our extensive experiments show that given a\nsufficiently deep encoder, a single-layer autoregressive decoder can\nsubstantially outperform strong non-autoregressive models with comparable\ninference speed. We show that the speed disadvantage for autoregressive\nbaselines compared to non-autoregressive methods has been overestimated in\nthree aspects: suboptimal layer allocation, insufficient speed measurement, and\nlack of knowledge distillation. Our results establish a new protocol for future\nresearch toward fast, accurate machine translation. Our code is available at\nhttps://github.com/jungokasai/deep-shallow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10413v2",
            "date": "2020-06-18 10:40:37+00:00",
            "title": "Are Pretrained Language Models Symbolic Reasoners Over Knowledge?",
            "authors": [
                "Nora Kassner",
                "Benno Krojer",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "How can pretrained language models (PLMs) learn factual knowledge from the\ntraining set? We investigate the two most important mechanisms: reasoning and\nmemorization. Prior work has attempted to quantify the number of facts PLMs\nlearn, but we present, using synthetic data, the first study that investigates\nthe causal relation between facts present in training and facts learned by the\nPLM. For reasoning, we show that PLMs seem to learn to apply some symbolic\nreasoning rules correctly but struggle with others, including two-hop\nreasoning. Further analysis suggests that even the application of learned\nreasoning rules is flawed. For memorization, we identify schema conformity\n(facts systematically supported by other facts) and frequency as key factors\nfor its success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10677v1",
            "date": "2020-06-18 17:05:45+00:00",
            "title": "AMALGUM -- A Free, Balanced, Multilayer English Web Corpus",
            "authors": [
                "Luke Gessler",
                "Siyao Peng",
                "Yang Liu",
                "Yilun Zhu",
                "Shabnam Behzad",
                "Amir Zeldes"
            ],
            "abstract": "We present a freely available, genre-balanced English web corpus totaling 4M\ntokens and featuring a large number of high-quality automatic annotation\nlayers, including dependency trees, non-named entity annotations, coreference\nresolution, and discourse trees in Rhetorical Structure Theory. By tapping open\nonline data sources the corpus is meant to offer a more sizable alternative to\nsmaller manually created annotated data sets, while avoiding pitfalls such as\nimbalanced or unknown composition, licensing problems, and low-quality natural\nlanguage processing. We harness knowledge from multiple annotation layers in\norder to achieve a \"better than NLP\" benchmark and evaluate the accuracy of the\nresulting resource.",
            "journal ref": "In Proceedings of The 12th Language Resources and Evaluation\n  Conference (pp. 5267-5275), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10973v1",
            "date": "2020-06-19 06:07:48+00:00",
            "title": "Sentiment Frames for Attitude Extraction in Russian",
            "authors": [
                "Natalia Loukachevitch",
                "Nicolay Rusnachenko"
            ],
            "abstract": "Texts can convey several types of inter-related information concerning\nopinions and attitudes. Such information includes the author's attitude towards\nmentioned entities, attitudes of the entities towards each other, positive and\nnegative effects on the entities in the described situations. In this paper, we\ndescribed the lexicon RuSentiFrames for Russian, where predicate words and\nexpressions are collected and linked to so-called sentiment frames conveying\nseveral types of presupposed information on attitudes and effects. We applied\nthe created frames in the task of extracting attitudes from a large news\ncollection.",
            "journal ref": "Proceedings of the International Conference on Computational\n  Linguistics and Intellectual Technologies \"Dialogue-2020\", 2020, pp.526-537",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11056v1",
            "date": "2020-06-19 10:21:17+00:00",
            "title": "A Survey of Syntactic-Semantic Parsing Based on Constituent and Dependency Structures",
            "authors": [
                "Meishan Zhang"
            ],
            "abstract": "Syntactic and semantic parsing has been investigated for decades, which is\none primary topic in the natural language processing community. This article\naims for a brief survey on this topic. The parsing community includes many\ntasks, which are difficult to be covered fully. Here we focus on two of the\nmost popular formalizations of parsing: constituent parsing and dependency\nparsing. Constituent parsing is majorly targeted to syntactic analysis, and\ndependency parsing can handle both syntactic and semantic analysis. This\narticle briefly reviews the representative models of constituent parsing and\ndependency parsing, and also dependency graph parsing with rich semantics.\nBesides, we also review the closely-related topics such as cross-domain,\ncross-lingual and joint parsing models, parser application as well as corpus\ndevelopment of parsing in the article.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11063v3",
            "date": "2020-06-19 10:44:06+00:00",
            "title": "Dataset for Automatic Summarization of Russian News",
            "authors": [
                "Ilya Gusev"
            ],
            "abstract": "Automatic text summarization has been studied in a variety of domains and\nlanguages. However, this does not hold for the Russian language. To overcome\nthis issue, we present Gazeta, the first dataset for summarization of Russian\nnews. We describe the properties of this dataset and benchmark several\nextractive and abstractive models. We demonstrate that the dataset is a valid\ntask for methods of text summarization for Russian. Additionally, we prove the\npretrained mBART model to be useful for Russian text summarization.",
            "journal ref": "In: AINL 2020. Communications in Computer and Information Science,\n  vol 1292. Springer, Cham (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11098v2",
            "date": "2020-06-19 12:00:05+00:00",
            "title": "Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans",
            "authors": [
                "Yair Lakretz",
                "Dieuwke Hupkes",
                "Alessandra Vergallito",
                "Marco Marelli",
                "Marco Baroni",
                "Stanislas Dehaene"
            ],
            "abstract": "Recursive processing in sentence comprehension is considered a hallmark of\nhuman linguistic abilities. However, its underlying neural mechanisms remain\nlargely unknown. We studied whether a modern artificial neural network trained\nwith \"deep learning\" methods mimics a central aspect of human sentence\nprocessing, namely the storing of grammatical number and gender information in\nworking memory and its use in long-distance agreement (e.g., capturing the\ncorrect number agreement between subject and verb when they are separated by\nother phrases). Although the network, a recurrent architecture with Long\nShort-Term Memory units, was solely trained to predict the next word in a large\ncorpus, analysis showed the emergence of a very sparse set of specialized units\nthat successfully handled local and long-distance syntactic agreement for\ngrammatical number. However, the simulations also showed that this mechanism\ndoes not support full recursion and fails with some long-range embedded\ndependencies. We tested the model's predictions in a behavioral experiment\nwhere humans detected violations in number agreement in sentences with\nsystematic variations in the singular/plural status of multiple nouns, with or\nwithout embedding. Human and model error patterns were remarkably similar,\nshowing that the model echoes various effects observed in human data. However,\na key difference was that, with embedded long-range dependencies, humans\nremained above chance level, while the model's systematic errors brought it\nbelow chance. Overall, our study shows that exploring the ways in which modern\nartificial neural networks process sentences leads to precise and testable\nhypotheses about human linguistic performance.",
            "journal ref": "Lakretz et al. (2021), Cognition",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11138v2",
            "date": "2020-06-19 13:49:26+00:00",
            "title": "New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles",
            "authors": [
                "Kiet Van Nguyen",
                "Tin Van Huynh",
                "Duc-Vu Nguyen",
                "Anh Gia-Tuan Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Large-scale and high-quality corpora are necessary for evaluating machine\nreading comprehension models on a low-resource language like Vietnamese.\nBesides, machine reading comprehension (MRC) for the health domain offers great\npotential for practical applications; however, there is still very little MRC\nresearch in this domain. This paper presents ViNewsQA as a new corpus for the\nVietnamese language to evaluate healthcare reading comprehension models. The\ncorpus comprises 22,057 human-generated question-answer pairs. Crowd-workers\ncreate the questions and their answers based on a collection of over 4,416\nonline Vietnamese healthcare news articles, where the answers comprise spans\nextracted from the corresponding articles. In particular, we develop a process\nof creating a corpus for the Vietnamese machine reading comprehension.\nComprehensive evaluations demonstrate that our corpus requires abilities beyond\nsimple reasoning, such as word matching and demanding difficult reasoning based\non single-or-multiple-sentence information. We conduct experiments using\ndifferent types of machine reading comprehension methods to achieve the first\nbaseline performances, compared with further models' performances. We also\nmeasure human performance on the corpus and compared it with several powerful\nneural network-based and transfer learning-based models. Our experiments show\nthat the best machine model is ALBERT, which achieves an exact match score of\n65.26% and an F1-score of 84.89% on our corpus. The significant differences\nbetween humans and the best-performance model (14.53% of EM and 10.90% of\nF1-score) on the test set of our corpus indicate that improvements in ViNewsQA\ncould be explored in the future study. Our corpus is publicly available on our\nwebsite for the research purpose to encourage the research community to make\nthese improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11558v1",
            "date": "2020-06-20 11:57:01+00:00",
            "title": "Seq2Seq and Joint Learning Based Unix Command Line Prediction System",
            "authors": [
                "Thoudam Doren Singh",
                "Abdullah Faiz Ur Rahman Khilji",
                "Divyansha",
                "Apoorva Vikram Singh",
                "Surmila Thokchom",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Despite being an open-source operating system pioneered in the early 90s,\nUNIX based platforms have not been able to garner an overwhelming reception\nfrom amateur end users. One of the rationales for under popularity of UNIX\nbased systems is the steep learning curve corresponding to them due to\nextensive use of command line interface instead of usual interactive graphical\nuser interface. In past years, the majority of insights used to explore the\nconcern are eminently centered around the notion of utilizing chronic log\nhistory of the user to make the prediction of successive command. The\napproaches directed at anatomization of this notion are predominantly in\naccordance with Probabilistic inference models. The techniques employed in\npast, however, have not been competent enough to address the predicament as\nlegitimately as anticipated. Instead of deploying usual mechanism of\nrecommendation systems, we have employed a simple yet novel approach of Seq2seq\nmodel by leveraging continuous representations of self-curated exhaustive\nKnowledge Base (KB) to enhance the embedding employed in the model. This work\ndescribes an assistive, adaptive and dynamic way of enhancing UNIX command line\nprediction systems. Experimental methods state that our model has achieved\naccuracy surpassing mixture of other techniques and adaptive command line\ninterface mechanism as acclaimed in the past.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11572v2",
            "date": "2020-06-20 13:24:14+00:00",
            "title": "SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection",
            "authors": [
                "Ekaterina Vylomova",
                "Jennifer White",
                "Elizabeth Salesky",
                "Sabrina J. Mielke",
                "Shijie Wu",
                "Edoardo Ponti",
                "Rowan Hall Maudslay",
                "Ran Zmigrod",
                "Josef Valvoda",
                "Svetlana Toldova",
                "Francis Tyers",
                "Elena Klyachko",
                "Ilya Yegorov",
                "Natalia Krizhanovsky",
                "Paula Czarnowska",
                "Irene Nikkarinen",
                "Andrew Krizhanovsky",
                "Tiago Pimentel",
                "Lucas Torroba Hennigen",
                "Christo Kirov",
                "Garrett Nicolai",
                "Adina Williams",
                "Antonios Anastasopoulos",
                "Hilaria Cruz",
                "Eleanor Chodroff",
                "Ryan Cotterell",
                "Miikka Silfverberg",
                "Mans Hulden"
            ],
            "abstract": "A broad goal in natural language processing (NLP) is to develop a system that\nhas the capacity to process any natural language. Most systems, however, are\ndeveloped using data from just one language such as English. The SIGMORPHON\n2020 shared task on morphological reinflection aims to investigate systems'\nability to generalize across typologically distinct languages, many of which\nare low resource. Systems were developed using data from 45 languages and just\n5 language families, fine-tuned with data from an additional 45 languages and\n10 language families (13 in total), and evaluated on all 90 languages. A total\nof 22 systems (19 neural) from 10 teams were submitted to the task. All four\nwinning systems were neural (two monolingual transformers and two massively\nmultilingual RNN-based models with gated attention). Most teams demonstrate\nutility of data hallucination and augmentation, ensembles, and multilingual\ntraining for low-resource languages. Non-neural learners and manually designed\ngrammars showed competitive and even superior performance on some languages\n(such as Ingrian, Tajik, Tagalog, Zarma, Lingala), especially with very limited\ndata. Some language families (Afro-Asiatic, Niger-Congo, Turkic) were\nrelatively easy for most systems and achieved over 90% mean accuracy while\nothers were more challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11586v1",
            "date": "2020-06-20 14:25:06+00:00",
            "title": "AraDIC: Arabic Document Classification using Image-Based Character Embeddings and Class-Balanced Loss",
            "authors": [
                "Mahmoud Daif",
                "Shunsuke Kitada",
                "Hitoshi Iyatomi"
            ],
            "abstract": "Classical and some deep learning techniques for Arabic text classification\noften depend on complex morphological analysis, word segmentation, and\nhand-crafted feature engineering. These could be eliminated by using\ncharacter-level features. We propose a novel end-to-end Arabic document\nclassification framework, Arabic document image-based classifier (AraDIC),\ninspired by the work on image-based character embeddings. AraDIC consists of an\nimage-based character encoder and a classifier. They are trained in an\nend-to-end fashion using the class balanced loss to deal with the long-tailed\ndata distribution problem. To evaluate the effectiveness of AraDIC, we created\nand published two datasets, the Arabic Wikipedia title (AWT) dataset and the\nArabic poetry (AraP) dataset. To the best of our knowledge, this is the first\nimage-based character embedding framework addressing the problem of Arabic text\nclassification. We also present the first deep learning-based text classifier\nwidely evaluated on modern standard Arabic, colloquial Arabic and classical\nArabic. AraDIC shows performance improvement over classical and deep learning\nbaselines by 12.29% and 23.05% for the micro and macro F-score, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11605v1",
            "date": "2020-06-20 16:09:24+00:00",
            "title": "Studying Attention Models in Sentiment Attitude Extraction Task",
            "authors": [
                "Nicolay Rusnachenko",
                "Natalia Loukachevitch"
            ],
            "abstract": "In the sentiment attitude extraction task, the aim is to identify\n<<attitudes>> -- sentiment relations between entities mentioned in text. In\nthis paper, we provide a study on attention-based context encoders in the\nsentiment attitude extraction task. For this task, we adapt attentive context\nencoders of two types: (i) feature-based; (ii) self-based. Our experiments with\na corpus of Russian analytical texts RuSentRel illustrate that the models\ntrained with attentive encoders outperform ones that were trained without them\nand achieve 1.5-5.9% increase by F1. We also provide the analysis of attention\nweight distributions in dependence on the term type.",
            "journal ref": "M\\'etais E., Meziane F., Horacek H., Cimiano P. (eds) Natural\n  Language Processing and Information Systems. NLDB 2020. Lecture Notes in\n  Computer Science, vol 12089. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11646v1",
            "date": "2020-06-20 20:21:17+00:00",
            "title": "The Importance of Category Labels in Grammar Induction with Child-directed Utterances",
            "authors": [
                "Lifeng Jin",
                "William Schuler"
            ],
            "abstract": "Recent progress in grammar induction has shown that grammar induction is\npossible without explicit assumptions of language-specific knowledge. However,\nevaluation of induced grammars usually has ignored phrasal labels, an essential\npart of a grammar. Experiments in this work using a labeled evaluation metric,\nRH, show that linguistically motivated predictions about grammar sparsity and\nuse of categories can only be revealed through labeled evaluation. Furthermore,\ndepth-bounding as an implementation of human memory constraints in grammar\ninducers is still effective with labeled evaluation on multilingual transcribed\nchild-directed utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11824v1",
            "date": "2020-06-21 15:22:03+00:00",
            "title": "Enriching Large-Scale Eventuality Knowledge Graph with Entailment Relations",
            "authors": [
                "Changlong Yu",
                "Hongming Zhang",
                "Yangqiu Song",
                "Wilfred Ng",
                "Lifeng Shang"
            ],
            "abstract": "Computational and cognitive studies suggest that the abstraction of\neventualities (activities, states, and events) is crucial for humans to\nunderstand daily eventualities. In this paper, we propose a scalable approach\nto model the entailment relations between eventualities (\"eat an apple''\nentails ''eat fruit''). As a result, we construct a large-scale eventuality\nentailment graph (EEG), which has 10 million eventuality nodes and 103 million\nentailment edges. Detailed experiments and analysis demonstrate the\neffectiveness of the proposed approach and quality of the resulting knowledge\ngraph. Our datasets and code are available at\nhttps://github.com/HKUST-KnowComp/ASER-EEG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11834v3",
            "date": "2020-06-21 15:51:54+00:00",
            "title": "AdvAug: Robust Adversarial Augmentation for Neural Machine Translation",
            "authors": [
                "Yong Cheng",
                "Lu Jiang",
                "Wolfgang Macherey",
                "Jacob Eisenstein"
            ],
            "abstract": "In this paper, we propose a new adversarial augmentation method for Neural\nMachine Translation (NMT). The main idea is to minimize the vicinal risk over\nvirtual sentences sampled from two vicinity distributions, of which the crucial\none is a novel vicinity distribution for adversarial sentences that describes a\nsmooth interpolated embedding space centered around observed training sentence\npairs. We then discuss our approach, AdvAug, to train NMT models using the\nembeddings of virtual sentences in sequence-to-sequence learning. Experiments\non Chinese-English, English-French, and English-German translation benchmarks\nshow that AdvAug achieves significant improvements over the Transformer (up to\n4.9 BLEU points), and substantially outperforms other data augmentation\ntechniques (e.g. back-translation) without using extra corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11852v1",
            "date": "2020-06-21 17:18:01+00:00",
            "title": "Labeling Explicit Discourse Relations using Pre-trained Language Models",
            "authors": [
                "Murathan Kurfal\u0131"
            ],
            "abstract": "Labeling explicit discourse relations is one of the most challenging\nsub-tasks of the shallow discourse parsing where the goal is to identify the\ndiscourse connectives and the boundaries of their arguments. The\nstate-of-the-art models achieve slightly above 45% of F-score by using\nhand-crafted features. The current paper investigates the efficacy of the\npre-trained language models in this task. We find that the pre-trained language\nmodels, when finetuned, are powerful enough to replace the linguistic features.\nWe evaluate our model on PDTB 2.0 and report the state-of-the-art results in\nthe extraction of the full relation. This is the first time when a model\noutperforms the knowledge intensive models without employing any linguistic\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12005v1",
            "date": "2020-06-22 04:49:47+00:00",
            "title": "Efficient text generation of user-defined topic using generative adversarial networks",
            "authors": [
                "Chenhan Yuan",
                "Yi-chin Huang",
                "Cheng-Hung Tsai"
            ],
            "abstract": "This study focused on efficient text generation using generative adversarial\nnetworks (GAN). Assuming that the goal is to generate a paragraph of a\nuser-defined topic and sentimental tendency, conventionally the whole network\nhas to be re-trained to obtain new results each time when a user changes the\ntopic. This would be time-consuming and impractical. Therefore, we propose a\nUser-Defined GAN (UD-GAN) with two-level discriminators to solve this problem.\nThe first discriminator aims to guide the generator to learn paragraph-level\ninformation and sentence syntactic structure, which is constructed by\nmultiple-LSTMs. The second one copes with higher-level information, such as the\nuser-defined sentiment and topic for text generation. The cosine similarity\nbased on TF-IDF and length penalty are adopted to determine the relevance of\nthe topic. Then, the second discriminator is re-trained with the generator if\nthe topic or sentiment for text generation is modified. The system evaluations\nare conducted to compare the performance of the proposed method with other\nGAN-based ones. The objective results showed that the proposed method is\ncapable of generating texts with less time than others and the generated text\nis related to the user-defined topic and sentiment. We will further investigate\nthe possibility of incorporating more detailed paragraph information such as\nsemantics into text generation to enhance the result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12040v1",
            "date": "2020-06-22 07:20:20+00:00",
            "title": "Clinical Predictive Keyboard using Statistical and Neural Language Modeling",
            "authors": [
                "John Pavlopoulos",
                "Panagiotis Papapetrou"
            ],
            "abstract": "A language model can be used to predict the next word during authoring, to\ncorrect spelling or to accelerate writing (e.g., in sms or emails). Language\nmodels, however, have only been applied in a very small scale to assist\nphysicians during authoring (e.g., discharge summaries or radiology reports).\nBut along with the assistance to the physician, computer-based systems which\nexpedite the patient's exit also assist in decreasing the hospital infections.\nWe employed statistical and neural language modeling to predict the next word\nof a clinical text and assess all the models in terms of accuracy and keystroke\ndiscount in two datasets with radiology reports. We show that a neural language\nmodel can achieve as high as 51.3% accuracy in radiology reports (one out of\ntwo words predicted correctly). We also show that even when the models are\nemployed only for frequent words, the physician can save valuable time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12106v1",
            "date": "2020-06-22 09:59:39+00:00",
            "title": "Exploiting Non-Taxonomic Relations for Measuring Semantic Similarity and Relatedness in WordNet",
            "authors": [
                "Mohannad AlMousa",
                "Rachid Benlamri",
                "Richard Khoury"
            ],
            "abstract": "Various applications in the areas of computational linguistics and artificial\nintelligence employ semantic similarity to solve challenging tasks, such as\nword sense disambiguation, text classification, information retrieval, machine\ntranslation, and document clustering. Previous work on semantic similarity\nfollowed a mono-relational approach using mostly the taxonomic relation \"ISA\".\nThis paper explores the benefits of using all types of non-taxonomic relations\nin large linked data, such as WordNet knowledge graph, to enhance existing\nsemantic similarity and relatedness measures. We propose a holistic\npoly-relational approach based on a new relation-based information content and\nnon-taxonomic-based weighted paths to devise a comprehensive semantic\nsimilarity and relatedness measure. To demonstrate the benefits of exploiting\nnon-taxonomic relations in a knowledge graph, we used three strategies to\ndeploy non-taxonomic relations at different granularity levels. We conducted\nexperiments on four well-known gold standard datasets, and the results\ndemonstrated the robustness and scalability of the proposed semantic similarity\nand relatedness measure, which significantly improves existing similarity\nmeasures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12146v1",
            "date": "2020-06-22 11:18:26+00:00",
            "title": "ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion",
            "authors": [
                "BingningWang",
                "Ting Yao",
                "Qi Zhang",
                "Jingfang Xu",
                "Xiaochuan Wang"
            ],
            "abstract": "This paper presents the ReCO, a human-curated ChineseReading Comprehension\ndataset on Opinion. The questions in ReCO are opinion based queries issued to\nthe commercial search engine. The passages are provided by the crowdworkers who\nextract the support snippet from the retrieved documents. Finally, an\nabstractive yes/no/uncertain answer was given by the crowdworkers. The release\nof ReCO consists of 300k questions that to our knowledge is the largest in\nChinese reading comprehension. A prominent characteristic of ReCO is that in\naddition to the original context paragraph, we also provided the support\nevidence that could be directly used to answer the question. Quality analysis\ndemonstrates the challenge of ReCO that requires various types of reasoning\nskills, such as causal inference, logical reasoning, etc. Current QA models\nthat perform very well on many question answering problems, such as BERT, only\nachieve 77% accuracy on this dataset, a large margin behind humans nearly 92%\nperformance, indicating ReCO presents a good challenge for machine reading\ncomprehension. The codes, datasets are freely available at\nhttps://github.com/benywon/ReCO.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12234v2",
            "date": "2020-06-22 13:30:35+00:00",
            "title": "Shared Task on Evaluating Accuracy in Natural Language Generation",
            "authors": [
                "Ehud Reiter",
                "Craig Thomson"
            ],
            "abstract": "We propose a shared task on methodologies and algorithms for evaluating the\naccuracy of generated texts. Participants will measure the accuracy of\nbasketball game summaries produced by NLG systems from basketball box score\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12414v1",
            "date": "2020-06-22 16:43:34+00:00",
            "title": "Dirichlet-Smoothed Word Embeddings for Low-Resource Settings",
            "authors": [
                "Jakob Jungmaier",
                "Nora Kassner",
                "Benjamin Roth"
            ],
            "abstract": "Nowadays, classical count-based word embeddings using positive pointwise\nmutual information (PPMI) weighted co-occurrence matrices have been widely\nsuperseded by machine-learning-based methods like word2vec and GloVe. But these\nmethods are usually applied using very large amounts of text data. In many\ncases, however, there is not much text data available, for example for specific\ndomains or low-resource languages. This paper revisits PPMI by adding Dirichlet\nsmoothing to correct its bias towards rare words. We evaluate on standard word\nsimilarity data sets and compare to word2vec and the recent state of the art\nfor low-resource settings: Positive and Unlabeled (PU) Learning for word\nembeddings. The proposed method outperforms PU-Learning for low-resource\nsettings and obtains competitive results for Maltese and Luxembourgish.",
            "journal ref": "LREC 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12418v2",
            "date": "2020-06-22 16:44:26+00:00",
            "title": "A Step Towards Interpretable Authorship Verification",
            "authors": [
                "Oren Halvani",
                "Lukas Graner",
                "Roey Regev"
            ],
            "abstract": "A central problem that has been researched for many years in the field of\ndigital text forensics is the question whether two documents were written by\nthe same author. Authorship verification (AV) is a research branch in this\nfield that deals with this question. Over the years, research activities in the\ncontext of AV have steadily increased, which has led to a variety of approaches\ntrying to solve this problem. Many of these approaches, however, make use of\nfeatures that are related to or influenced by the topic of the documents.\nTherefore, it may accidentally happen that their verification results are based\nnot on the writing style (the actual focus of AV), but on the topic of the\ndocuments. To address this problem, we propose an alternative AV approach that\nconsiders only topic-agnostic features in its classification decision. In\naddition, we present a post-hoc interpretation method that allows to understand\nwhich particular features have contributed to the prediction of the proposed AV\nmethod. To evaluate the performance of our AV method, we compared it with ten\ncompeting baselines (including the current state of the art) on four\nchallenging data sets. The results show that our approach outperforms all\nbaselines in two cases (with a maximum accuracy of 84%), while in the other two\ncases it performs as well as the strongest baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12799v1",
            "date": "2020-06-23 07:15:11+00:00",
            "title": "Keyframe Segmentation and Positional Encoding for Video-guided Machine Translation Challenge 2020",
            "authors": [
                "Tosho Hirasawa",
                "Zhishen Yang",
                "Mamoru Komachi",
                "Naoaki Okazaki"
            ],
            "abstract": "Video-guided machine translation as one of multimodal neural machine\ntranslation tasks targeting on generating high-quality text translation by\ntangibly engaging both video and text. In this work, we presented our\nvideo-guided machine translation system in approaching the Video-guided Machine\nTranslation Challenge 2020. This system employs keyframe-based video feature\nextractions along with the video feature positional encoding. In the evaluation\nphase, our system scored 36.60 corpus-level BLEU-4 and achieved the 1st place\non the Video-guided Machine Translation Challenge 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12816v1",
            "date": "2020-06-23 08:17:48+00:00",
            "title": "Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering",
            "authors": [
                "Xin Cong",
                "Bowen Yu",
                "Tingwen Liu",
                "Shiyao Cui",
                "Hengzhu Tang",
                "Bin Wang"
            ],
            "abstract": "Few-shot classification tends to struggle when it needs to adapt to diverse\ndomains. Due to the non-overlapping label space between domains, the\nperformance of conventional domain adaptation is limited. Previous work tackles\nthe problem in a transductive manner, by assuming access to the full set of\ntest data, which is too restrictive for many real-world applications. In this\npaper, we set out to tackle this issue by introducing a inductive framework,\nDaFeC, to improve Domain adaptation performance for Few-shot classification via\nClustering. We first build a representation extractor to derive features for\nunlabeled data from the target domain (no test data is necessary) and then\ngroup them with a cluster miner. The generated pseudo-labeled data and the\nlabeled source-domain data are used as supervision to update the parameters of\nthe few-shot classifier. In order to derive high-quality pseudo labels, we\npropose a Clustering Promotion Mechanism, to learn better features for the\ntarget domain via Similarity Entropy Minimization and Adversarial Distribution\nAlignment, which are combined with a Cosine Annealing Strategy. Experiments are\nperformed on the FewRel 2.0 dataset. Our approach outperforms previous work\nwith absolute gains (in classification accuracy) of 4.95%, 9.55%, 3.99% and\n11.62%, respectively, under four few-shot settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13071v1",
            "date": "2020-06-23 14:47:41+00:00",
            "title": "Domain Adaptation for Semantic Parsing",
            "authors": [
                "Zechang Li",
                "Yuxuan Lai",
                "Yansong Feng",
                "Dongyan Zhao"
            ],
            "abstract": "Recently, semantic parsing has attracted much attention in the community.\nAlthough many neural modeling efforts have greatly improved the performance, it\nstill suffers from the data scarcity issue. In this paper, we propose a novel\nsemantic parser for domain adaptation, where we have much fewer annotated data\nin the target domain compared to the source domain. Our semantic parser\nbenefits from a two-stage coarse-to-fine framework, thus can provide different\nand accurate treatments for the two stages, i.e., focusing on domain invariant\nand domain specific information, respectively. In the coarse stage, our novel\ndomain discrimination component and domain relevance attention encourage the\nmodel to learn transferable domain general structures. In the fine stage, the\nmodel is guided to concentrate on domain related details. Experiments on a\nbenchmark dataset show that our method consistently outperforms several popular\ndomain adaptation strategies. Additionally, we show that our model can well\nexploit limited target data to capture the difference between the source and\ntarget domain, even when the target domain has far fewer training instances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13200v1",
            "date": "2020-06-23 17:57:25+00:00",
            "title": "Combining Neural Language Models for WordSense Induction",
            "authors": [
                "Nikolay Arefyev",
                "Boris Sheludko",
                "Tatiana Aleksashina"
            ],
            "abstract": "Word sense induction (WSI) is the problem of grouping occurrences of an\nambiguous word according to the expressed sense of this word. Recently a new\napproach to this task was proposed, which generates possible substitutes for\nthe ambiguous word in a particular context using neural language models, and\nthen clusters sparse bag-of-words vectors built from these substitutes. In this\nwork, we apply this approach to the Russian language and improve it in two\nways. First, we propose methods of combining left and right contexts, resulting\nin better substitutes generated. Second, instead of fixed number of clusters\nfor all ambiguous words we propose a technique for selecting individual number\nof clusters for each word. Our approach established new state-of-the-art level,\nimproving current best results of WSI for the Russian language on two RUSSE\n2018 datasets by a large margin.",
            "journal ref": "van der Aalst W. et al. (eds) Analysis of Images, Social Networks\n  and Texts. AIST 2019. Lecture Notes in Computer Science, vol 11832. Springer,\n  Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13327v1",
            "date": "2020-06-23 20:54:25+00:00",
            "title": "Classifying Referential and Non-referential It Using Gaze",
            "authors": [
                "Victoria Yaneva",
                "Le An Ha",
                "Richard Evans",
                "Ruslan Mitkov"
            ],
            "abstract": "When processing a text, humans and machines must disambiguate between\ndifferent uses of the pronoun it, including non-referential, nominal anaphoric\nor clause anaphoric ones. In this paper, we use eye-tracking data to learn how\nhumans perform this disambiguation. We use this knowledge to improve the\nautomatic classification of it. We show that by using gaze data and a\nPOS-tagger we are able to significantly outperform a common baseline and\nclassify between three categories of it with an accuracy comparable to that of\nlinguisticbased approaches. In addition, the discriminatory power of specific\ngaze features informs the way humans process the pronoun, which, to the best of\nour knowledge, has not been explored using data from a natural reading task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13343v1",
            "date": "2020-06-23 21:28:28+00:00",
            "title": "One Model to Pronounce Them All: Multilingual Grapheme-to-Phoneme Conversion With a Transformer Ensemble",
            "authors": [
                "Kaili Vesik",
                "Muhammad Abdul-Mageed",
                "Miikka Silfverberg"
            ],
            "abstract": "The task of grapheme-to-phoneme (G2P) conversion is important for both speech\nrecognition and synthesis. Similar to other speech and language processing\ntasks, in a scenario where only small-sized training data are available,\nlearning G2P models is challenging. We describe a simple approach of exploiting\nmodel ensembles, based on multilingual Transformers and self-training, to\ndevelop a highly effective G2P solution for 15 languages. Our models are\ndeveloped as part of our participation in the SIGMORPHON 2020 Shared Task 1\nfocused at G2P. Our best models achieve 14.99 word error rate (WER) and 3.30\nphoneme error rate (PER), a sizeable improvement over the shared task\ncompetitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13425v1",
            "date": "2020-06-24 02:08:44+00:00",
            "title": "A High-Quality Multilingual Dataset for Structured Documentation Translation",
            "authors": [
                "Kazuma Hashimoto",
                "Raffaella Buschiazzo",
                "James Bradbury",
                "Teresa Marshall",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "This paper presents a high-quality multilingual dataset for the documentation\ndomain to advance research on localization of structured text. Unlike\nwidely-used datasets for translation of plain text, we collect XML-structured\nparallel text segments from the online documentation for an enterprise software\nplatform. These Web pages have been professionally translated from English into\n16 languages and maintained by domain experts, and around 100,000 text segments\nare available for each language pair. We build and evaluate translation models\nfor seven target languages from English, with several different copy mechanisms\nand an XML-constrained beam search. We also experiment with a non-English pair\nto show that our dataset has the potential to explicitly enable $17 \\times 16$\ntranslation settings. Our experiments show that learning to translate with the\nXML tags improves translation accuracy, and the beam search accurately\ngenerates XML structures. We also discuss trade-offs of using the copy\nmechanisms by focusing on translation of numerical words and named entities. We\nfurther provide a detailed human analysis of gaps between the model output and\nhuman translations for real-world applications, including suitability for\npost-editing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13730v2",
            "date": "2020-06-23 13:54:48+00:00",
            "title": "Attention-Based Neural Networks for Sentiment Attitude Extraction using Distant Supervision",
            "authors": [
                "Nicolay Rusnachenko",
                "Natalia Loukachevitch"
            ],
            "abstract": "In the sentiment attitude extraction task, the aim is to identify\n<<attitudes>> -- sentiment relations between entities mentioned in text. In\nthis paper, we provide a study on attention-based context encoders in the\nsentiment attitude extraction task. For this task, we adapt attentive context\nencoders of two types: (1) feature-based; (2) self-based. In our study, we\nutilize the corpus of Russian analytical texts RuSentRel and automatically\nconstructed news collection RuAttitudes for enriching the training set. We\nconsider the problem of attitude extraction as two-class (positive, negative)\nand three-class (positive, negative, neutral) classification tasks for whole\ndocuments. Our experiments with the RuSentRel corpus show that the three-class\nclassification models, which employ the RuAttitudes corpus for training, result\nin 10% increase and extra 3% by F1, when model architectures include the\nattention mechanism. We also provide the analysis of attention weight\ndistributions in dependence on the term type.",
            "journal ref": "The 10th International Conference on Web Intelligence, Mining and\n  Semantics (WIMS 2020), June 30-July 3, 2020, Biarritz, France",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14017v1",
            "date": "2020-06-24 19:42:54+00:00",
            "title": "XREF: Entity Linking for Chinese News Comments with Supplementary Article Reference",
            "authors": [
                "Xinyu Hua",
                "Lei Li",
                "Lifeng Hua",
                "Lu Wang"
            ],
            "abstract": "Automatic identification of mentioned entities in social media posts\nfacilitates quick digestion of trending topics and popular opinions.\nNonetheless, this remains a challenging task due to limited context and diverse\nname variations. In this paper, we study the problem of entity linking for\nChinese news comments given mentions' spans. We hypothesize that comments often\nrefer to entities in the corresponding news article, as well as topics\ninvolving the entities. We therefore propose a novel model, XREF, that\nleverages attention mechanisms to (1) pinpoint relevant context within\ncomments, and (2) detect supporting entities from the news article. To improve\ntraining, we make two contributions: (a) we propose a supervised attention loss\nin addition to the standard cross entropy, and (b) we develop a weakly\nsupervised training scheme to utilize the large-scale unlabeled corpus. Two new\ndatasets in entertainment and product domains are collected and annotated for\nexperiments. Our proposed method outperforms previous methods on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14116v1",
            "date": "2020-06-25 00:42:39+00:00",
            "title": "Normalizing Text using Language Modelling based on Phonetics and String Similarity",
            "authors": [
                "Fenil Doshi",
                "Jimit Gandhi",
                "Deep Gosalia",
                "Sudhir Bagul"
            ],
            "abstract": "Social media networks and chatting platforms often use an informal version of\nnatural text. Adversarial spelling attacks also tend to alter the input text by\nmodifying the characters in the text. Normalizing these texts is an essential\nstep for various applications like language translation and text to speech\nsynthesis where the models are trained over clean regular English language. We\npropose a new robust model to perform text normalization.\n  Our system uses the BERT language model to predict the masked words that\ncorrespond to the unnormalized words. We propose two unique masking strategies\nthat try to replace the unnormalized words in the text with their root form\nusing a unique score based on phonetic and string similarity metrics.We use\nhuman-centric evaluations where volunteers were asked to rank the normalized\ntext. Our strategies yield an accuracy of 86.7% and 83.2% which indicates the\neffectiveness of our system in dealing with text normalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14194v2",
            "date": "2020-06-25 06:16:29+00:00",
            "title": "Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion",
            "authors": [
                "Alex Sokolov",
                "Tracy Rohlin",
                "Ariya Rastrow"
            ],
            "abstract": "Grapheme-to-phoneme (G2P) models are a key component in Automatic Speech\nRecognition (ASR) systems, such as the ASR system in Alexa, as they are used to\ngenerate pronunciations for out-of-vocabulary words that do not exist in the\npronunciation lexicons (mappings like \"e c h o\" to \"E k oU\"). Most G2P systems\nare monolingual and based on traditional joint-sequence based n-gram models\n[1,2]. As an alternative, we present a single end-to-end trained neural G2P\nmodel that shares same encoder and decoder across multiple languages. This\nallows the model to utilize a combination of universal symbol inventories of\nLatin-like alphabets and cross-linguistically shared feature representations.\nSuch model is especially useful in the scenarios of low resource languages and\ncode switching/foreign words, where the pronunciations in one language need to\nbe adapted to other locales or accents. We further experiment with word\nlanguage distribution vector as an additional training target in order to\nimprove system performance by helping the model decouple pronunciations across\na variety of languages in the parameter space. We show 7.2% average improvement\nin phoneme error rate over low resource languages and no degradation over high\nresource ones compared to monolingual baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14198v2",
            "date": "2020-06-25 06:28:09+00:00",
            "title": "A Simple Approach to Case-Based Reasoning in Knowledge Bases",
            "authors": [
                "Rajarshi Das",
                "Ameya Godbole",
                "Shehzaad Dhuliawala",
                "Manzil Zaheer",
                "Andrew McCallum"
            ],
            "abstract": "We present a surprisingly simple yet accurate approach to reasoning in\nknowledge graphs (KGs) that requires \\emph{no training}, and is reminiscent of\ncase-based reasoning in classical artificial intelligence (AI). Consider the\ntask of finding a target entity given a source entity and a binary relation.\nOur non-parametric approach derives crisp logical rules for each query by\nfinding multiple \\textit{graph path patterns} that connect similar source\nentities through the given relation. Using our method, we obtain new\nstate-of-the-art accuracy, outperforming all previous models, on NELL-995 and\nFB-122. We also demonstrate that our model is robust in low data settings,\noutperforming recently proposed meta-learning approaches",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14209v1",
            "date": "2020-06-25 07:11:07+00:00",
            "title": "Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes",
            "authors": [
                "Marina Sedinkina",
                "Nikolas Breitkopf",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "In this paper, we automatically create sentiment dictionaries for predicting\nfinancial outcomes. We compare three approaches: (I) manual adaptation of the\ndomain-general dictionary H4N, (ii) automatic adaptation of H4N and (iii) a\ncombination consisting of first manual, then automatic adaptation. In our\nexperiments, we demonstrate that the automatically adapted sentiment dictionary\noutperforms the previous state of the art in predicting the financial outcomes\nexcess return and volatility. In particular, automatic adaptation performs\nbetter than manual adaptation. In our analysis, we find that annotation based\non an expert's a priori belief about a word's meaning can be incorrect --\nannotation should be performed based on the word's contexts in the target\ndomain instead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14320v1",
            "date": "2020-06-25 11:51:08+00:00",
            "title": "Analyzing Effect of Repeated Reading on Oral Fluency and Narrative Production for Computer-Assisted Language Learning",
            "authors": [
                "Santosh Kumar Barnwal",
                "Uma Shanker Tiwary"
            ],
            "abstract": "Repeated reading (RR) helps learners, who have little to no experience with\nreading fluently to gain confidence, speed and process words automatically. The\nbenefits of repeated readings include helping all learners with fact recall,\naiding identification of learners' main ideas and vocabulary, increasing\ncomprehension, leading to faster reading as well as increasing word recognition\naccuracy, and assisting struggling learners as they transition from\nword-by-word reading to more meaningful phrasing. Thus, RR ultimately helps in\nimprovements of learners' oral fluency and narrative production. However, there\nare no open audio datasets available on oral responses of learners based on\ntheir RR practices. Therefore, in this paper, we present our dataset, discuss\nits properties, and propose a method to assess oral fluency and narrative\nproduction for learners of English using acoustic, prosodic, lexical and\nsyntactical characteristics. The results show that a CALL system can be\ndeveloped for assessing the improvements in learners' oral fluency and\nnarrative production.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14405v1",
            "date": "2020-06-25 13:43:11+00:00",
            "title": "Learning Source Phrase Representations for Neural Machine Translation",
            "authors": [
                "Hongfei Xu",
                "Josef van Genabith",
                "Deyi Xiong",
                "Qiuhui Liu",
                "Jingyi Zhang"
            ],
            "abstract": "The Transformer translation model (Vaswani et al., 2017) based on a\nmulti-head attention mechanism can be computed effectively in parallel and has\nsignificantly pushed forward the performance of Neural Machine Translation\n(NMT). Though intuitively the attentional network can connect distant words via\nshorter network paths than RNNs, empirical analysis demonstrates that it still\nhas difficulty in fully capturing long-distance dependencies (Tang et al.,\n2018). Considering that modeling phrases instead of words has significantly\nimproved the Statistical Machine Translation (SMT) approach through the use of\nlarger translation blocks (\"phrases\") and its reordering ability, modeling NMT\nat phrase level is an intuitive proposal to help the model capture\nlong-distance relationships. In this paper, we first propose an attentive\nphrase representation generation mechanism which is able to generate phrase\nrepresentations from corresponding token representations. In addition, we\nincorporate the generated phrase representations into the Transformer\ntranslation model to enhance its ability to capture long-distance\nrelationships. In our experiments, we obtain significant improvements on the\nWMT 14 English-German and English-French tasks on top of the strong Transformer\nbaseline, which shows the effectiveness of our approach. Our approach helps\nTransformer Base models perform at the level of Transformer Big models, and\neven significantly better for long sentences, but with substantially fewer\nparameters and training steps. The fact that phrase representations help even\nin the big setting further supports our conjecture that they make a valuable\ncontribution to long-distance relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14465v3",
            "date": "2020-06-25 14:59:47+00:00",
            "title": "IIT Gandhinagar at SemEval-2020 Task 9: Code-Mixed Sentiment Classification Using Candidate Sentence Generation and Selection",
            "authors": [
                "Vivek Srivastava",
                "Mayank Singh"
            ],
            "abstract": "Code-mixing is the phenomenon of using multiple languages in the same\nutterance of a text or speech. It is a frequently used pattern of communication\non various platforms such as social media sites, online gaming, product\nreviews, etc. Sentiment analysis of the monolingual text is a well-studied\ntask. Code-mixing adds to the challenge of analyzing the sentiment of the text\ndue to the non-standard writing style. We present a candidate sentence\ngeneration and selection based approach on top of the Bi-LSTM based neural\nclassifier to classify the Hinglish code-mixed text into one of the three\nsentiment classes positive, negative, or neutral. The proposed approach shows\nan improvement in the system performance as compared to the Bi-LSTM based\nneural classifier. The results present an opportunity to understand various\nother nuances of code-mixing in the textual data, such as humor-detection,\nintent classification, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14668v1",
            "date": "2020-06-25 19:24:57+00:00",
            "title": "THEaiTRE: Artificial Intelligence to Write a Theatre Play",
            "authors": [
                "Rudolf Rosa",
                "Ond\u0159ej Du\u0161ek",
                "Tom Kocmi",
                "David Mare\u010dek",
                "Tom\u00e1\u0161 Musil",
                "Patr\u00edcia Schmidtov\u00e1",
                "Dominik Jurko",
                "Ond\u0159ej Bojar",
                "Daniel Hrbek",
                "David Ko\u0161\u0165\u00e1k",
                "Martina Kinsk\u00e1",
                "Josef Dole\u017eal",
                "Kl\u00e1ra Voseck\u00e1"
            ],
            "abstract": "We present THEaiTRE, a starting project aimed at automatic generation of\ntheatre play scripts. This paper reviews related work and drafts an approach we\nintend to follow. We plan to adopt generative neural language models and\nhierarchical generation approaches, supported by summarization and machine\ntranslation methods, and complemented with a human-in-the-loop approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14767v1",
            "date": "2020-06-26 03:08:33+00:00",
            "title": "Dialog as a Vehicle for Lifelong Learning",
            "authors": [
                "Aishwarya Padmakumar",
                "Raymond J. Mooney"
            ],
            "abstract": "Dialog systems research has primarily been focused around two main types of\napplications - task-oriented dialog systems that learn to use clarification to\naid in understanding a goal, and open-ended dialog systems that are expected to\ncarry out unconstrained \"chit chat\" conversations. However, dialog interactions\ncan also be used to obtain various types of knowledge that can be used to\nimprove an underlying language understanding system, or other machine learning\nsystems that the dialog acts over. In this position paper, we present the\nproblem of designing dialog systems that enable lifelong learning as an\nimportant challenge problem, in particular for applications involving\nphysically situated robots. We include examples of prior work in this\ndirection, and discuss challenges that remain to be addressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15435v1",
            "date": "2020-06-27 20:06:15+00:00",
            "title": "Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization",
            "authors": [
                "Beliz Gunel",
                "Chenguang Zhu",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "Neural models have become successful at producing abstractive summaries that\nare human-readable and fluent. However, these models have two critical\nshortcomings: they often don't respect the facts that are either included in\nthe source article or are known to humans as commonsense knowledge, and they\ndon't produce coherent summaries when the source article is long. In this work,\nwe propose a novel architecture that extends Transformer encoder-decoder\narchitecture in order to improve on these shortcomings. First, we incorporate\nentity-level knowledge from the Wikidata knowledge graph into the\nencoder-decoder architecture. Injecting structural world knowledge from\nWikidata helps our abstractive summarization model to be more fact-aware.\nSecond, we utilize the ideas used in Transformer-XL language model in our\nproposed encoder-decoder architecture. This helps our model with producing\ncoherent summaries even when the source article is long. We test our model on\nCNN/Daily Mail summarization dataset and show improvements on ROUGE scores over\nthe baseline Transformer model. We also include model predictions for which our\nmodel accurately conveys the facts, while the baseline Transformer model\ndoesn't.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15454v1",
            "date": "2020-06-27 21:51:38+00:00",
            "title": "A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards",
            "authors": [
                "Zi-Yi Dou",
                "Sachin Kumar",
                "Yulia Tsvetkov"
            ],
            "abstract": "Cross-lingual text summarization aims at generating a document summary in one\nlanguage given input in another language. It is a practically important but\nunder-explored task, primarily due to the dearth of available data. Existing\nmethods resort to machine translation to synthesize training data, but such\npipeline approaches suffer from error propagation. In this work, we propose an\nend-to-end cross-lingual text summarization model. The model uses reinforcement\nlearning to directly optimize a bilingual semantic similarity metric between\nthe summaries generated in a target language and gold summaries in a source\nlanguage. We also introduce techniques to pre-train the model leveraging\nmonolingual summarization and machine translation objectives. Experimental\nresults in both English--Chinese and English--German cross-lingual\nsummarization settings demonstrate the effectiveness of our methods. In\naddition, we find that reinforcement learning models with bilingual semantic\nsimilarity as rewards generate more fluent sentences than strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15585v1",
            "date": "2020-06-28 12:19:15+00:00",
            "title": "Self-Attention Networks for Intent Detection",
            "authors": [
                "Sevinj Yolchuyeva",
                "G\u00e9za N\u00e9meth",
                "B\u00e1lint Gyires-T\u00f3th"
            ],
            "abstract": "Self-attention networks (SAN) have shown promising performance in various\nNatural Language Processing (NLP) scenarios, especially in machine translation.\nOne of the main points of SANs is the strength of capturing long-range and\nmulti-scale dependencies from the data. In this paper, we present a novel\nintent detection system which is based on a self-attention network and a\nBi-LSTM. Our approach shows improvement by using a transformer model and deep\naveraging network-based universal sentence encoder compared to previous\nsolutions. We evaluate the system on Snips, Smart Speaker, Smart Lights, and\nATIS datasets by different evaluation metrics. The performance of the proposed\nmodel is compared with LSTM with the same datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15795v1",
            "date": "2020-06-29 03:36:04+00:00",
            "title": "Combine Convolution with Recurrent Networks for Text Classification",
            "authors": [
                "Shengfei Lyu",
                "Jiaqi Liu"
            ],
            "abstract": "Convolutional neural network (CNN) and recurrent neural network (RNN) are two\npopular architectures used in text classification. Traditional methods to\ncombine the strengths of the two networks rely on streamlining them or\nconcatenating features extracted from them. In this paper, we propose a novel\nmethod to keep the strengths of the two networks to a great extent. In the\nproposed model, a convolutional neural network is applied to learn a 2D weight\nmatrix where each row reflects the importance of each word from different\naspects. Meanwhile, we use a bi-directional RNN to process each word and employ\na neural tensor layer that fuses forward and backward hidden states to get word\nrepresentations. In the end, the weight matrix and word representations are\ncombined to obtain the representation in a 2D matrix form for the text. We\ncarry out experiments on a number of datasets for text classification. The\nexperimental results confirm the effectiveness of the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15854v1",
            "date": "2020-06-29 07:56:22+00:00",
            "title": "A Framework for Pre-processing of Social Media Feeds based on Integrated Local Knowledge Base",
            "authors": [
                "Taiwo Kolajo",
                "Olawande Daramola",
                "Ayodele Adebiyi",
                "Seth Aaditeshwar"
            ],
            "abstract": "Most of the previous studies on the semantic analysis of social media feeds\nhave not considered the issue of ambiguity that is associated with slangs,\nabbreviations, and acronyms that are embedded in social media posts. These\nnoisy terms have implicit meanings and form part of the rich semantic context\nthat must be analysed to gain complete insights from social media feeds. This\npaper proposes an improved framework for pre-processing of social media feeds\nfor better performance. To do this, the use of an integrated knowledge base\n(ikb) which comprises a local knowledge source (Naijalingo), urban dictionary\nand internet slang was combined with the adapted Lesk algorithm to facilitate\nsemantic analysis of social media feeds. Experimental results showed that the\nproposed approach performed better than existing methods when it was tested on\nthree machine learning models, which are support vector machines, multilayer\nperceptron, and convolutional neural networks. The framework had an accuracy of\n94.07% on a standardized dataset, and 99.78% on localised dataset when used to\nextract sentiments from tweets. The improved performance on the localised\ndataset reveals the advantage of integrating the use of local knowledge sources\ninto the process of analysing social media feeds particularly in interpreting\nslangs/acronyms/abbreviations that have contextually rooted meanings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15942v2",
            "date": "2020-06-29 11:20:13+00:00",
            "title": "Hinting Semantic Parsing with Statistical Word Sense Disambiguation",
            "authors": [
                "Ritwik Bose",
                "Siddharth Vashishtha",
                "James Allen"
            ],
            "abstract": "The task of Semantic Parsing can be approximated as a transformation of an\nutterance into a logical form graph where edges represent semantic roles and\nnodes represent word senses. The resulting representation should be capture the\nmeaning of the utterance and be suitable for reasoning. Word senses and\nsemantic roles are interdependent, meaning errors in assigning word senses can\ncause errors in assigning semantic roles and vice versa. While statistical\napproaches to word sense disambiguation outperform logical, rule-based semantic\nparsers for raw word sense assignment, these statistical word sense\ndisambiguation systems do not produce the rich role structure or detailed\nsemantic representation of the input. In this work, we provide hints from a\nstatistical WSD system to guide a logical semantic parser to produce better\nsemantic type assignments while maintaining the soundness of the resulting\nlogical forms. We observe an improvement of up to 10.5% in F-score, however we\nfind that this improvement comes at a cost to the structural integrity of the\nparse",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15994v4",
            "date": "2020-06-29 12:39:44+00:00",
            "title": "Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models",
            "authors": [
                "Viet Bui The",
                "Oanh Tran Thi",
                "Phuong Le-Hong"
            ],
            "abstract": "This paper describes our study on using mutilingual BERT embeddings and some\nnew neural models for improving sequence tagging tasks for the Vietnamese\nlanguage. We propose new model architectures and evaluate them extensively on\ntwo named entity recognition datasets of VLSP 2016 and VLSP 2018, and on two\npart-of-speech tagging datasets of VLSP 2010 and VLSP 2013. Our proposed models\noutperform existing methods and achieve new state-of-the-art results. In\nparticular, we have pushed the accuracy of part-of-speech tagging to 95.40% on\nthe VLSP 2010 corpus, to 96.77% on the VLSP 2013 corpus; and the F1 score of\nnamed entity recognition to 94.07% on the VLSP 2016 corpus, to 90.31% on the\nVLSP 2018 corpus. Our code and pre-trained models viBERT and vELECTRA are\nreleased as open source to facilitate adoption and further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16082v1",
            "date": "2020-06-29 14:35:42+00:00",
            "title": "Measuring Memorization Effect in Word-Level Neural Networks Probing",
            "authors": [
                "Rudolf Rosa",
                "Tom\u00e1\u0161 Musil",
                "David Mare\u010dek"
            ],
            "abstract": "Multiple studies have probed representations emerging in neural networks\ntrained for end-to-end NLP tasks and examined what word-level linguistic\ninformation may be encoded in the representations. In classical probing, a\nclassifier is trained on the representations to extract the target linguistic\ninformation. However, there is a threat of the classifier simply memorizing the\nlinguistic labels for individual words, instead of extracting the linguistic\nabstractions from the representations, thus reporting false positive results.\nWhile considerable efforts have been made to minimize the memorization problem,\nthe task of actually measuring the amount of memorization happening in the\nclassifier has been understudied so far. In our work, we propose a simple\ngeneral method for measuring the memorization effect, based on a symmetric\nselection of comparable sets of test words seen versus unseen in training. Our\nmethod can be used to explicitly quantify the amount of memorization happening\nin a probing setup, so that an adequate setup can be chosen and the results of\nthe probing can be interpreted with a reliability estimate. We exemplify this\nby showcasing our method on a case study of probing for part of speech in a\ntrained neural machine translation encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16146v1",
            "date": "2020-06-29 16:10:27+00:00",
            "title": "Want to Identify, Extract and Normalize Adverse Drug Reactions in Tweets? Use RoBERTa",
            "authors": [
                "Katikapalli Subramanyam Kalyan",
                "S. Sangeetha"
            ],
            "abstract": "This paper presents our approach for task 2 and task 3 of Social Media Mining\nfor Health (SMM4H) 2020 shared tasks. In task 2, we have to differentiate\nadverse drug reaction (ADR) tweets from nonADR tweets and is treated as binary\nclassification. Task3 involves extracting ADR mentions and then mapping them to\nMedDRA codes. Extracting ADR mentions is treated as sequence labeling and\nnormalizing ADR mentions is treated as multi-class classification. Our system\nis based on pre-trained language model RoBERTa and it achieves a) F1-score of\n58% in task2 which is 12% more than the average score b) relaxed F1-score of\n70.1% in ADR extraction of task 3 which is 13.7% more than the average score\nand relaxed F1-score of 35% in ADR extraction + normalization of task3 which is\n5.8% more than the average score. Overall, our models achieve promising results\nin both the tasks with significant improvements over average scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16212v1",
            "date": "2020-06-29 17:24:09+00:00",
            "title": "Towards the Study of Morphological Processing of the Tangkhul Language",
            "authors": [
                "Mirinso Shadang",
                "Navanath Saharia",
                "Thoudam Doren Singh"
            ],
            "abstract": "There is no or little work on natural language processing of Tangkhul\nlanguage. The current work is a humble beginning of morphological processing of\nthis language using an unsupervised approach. We use a small corpus collected\nfrom different sources of text books, short stories and articles of other\ntopics. Based on the experiments carried out, the morpheme identification task\nusing morphessor gives reasonable and interesting output despite using a small\ncorpus.",
            "journal ref": "In proceeding of Regional International Conference on Natural\n  Language Processing (regICON) 2017, 3rd and 4th November 2017, IIIT Senapati,\n  Manipur, India",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16403v1",
            "date": "2020-06-29 21:44:51+00:00",
            "title": "ANA at SemEval-2020 Task 4: mUlti-task learNIng for cOmmonsense reasoNing (UNION)",
            "authors": [
                "Anandh Perumal",
                "Chenyang Huang",
                "Amine Trabelsi",
                "Osmar R. Za\u00efane"
            ],
            "abstract": "In this paper, we describe our mUlti-task learNIng for cOmmonsense reasoNing\n(UNION) system submitted for Task C of the SemEval2020 Task 4, which is to\ngenerate a reason explaining why a given false statement is non-sensical.\nHowever, we found in the early experiments that simple adaptations such as\nfine-tuning GPT2 often yield dull and non-informative generations (e.g. simple\nnegations). In order to generate more meaningful explanations, we propose\nUNION, a unified end-to-end framework, to utilize several existing commonsense\ndatasets so that it allows a model to learn more dynamics under the scope of\ncommonsense reasoning. In order to perform model selection efficiently,\naccurately and promptly, we also propose a couple of auxiliary automatic\nevaluation metrics so that we can extensively compare the models from different\nperspectives. Our submitted system not only results in a good performance in\nthe proposed metrics but also outperforms its competitors with the highest\nachieved score of 2.10 for human evaluation while remaining a BLEU score of\n15.7. Our code is made publicly available at GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16779v4",
            "date": "2020-06-30 13:36:10+00:00",
            "title": "PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning",
            "authors": [
                "Siqi Bao",
                "Huang He",
                "Fan Wang",
                "Hua Wu",
                "Haifeng Wang",
                "Wenquan Wu",
                "Zhen Guo",
                "Zhibin Liu",
                "Xinchao Xu"
            ],
            "abstract": "To build a high-quality open-domain chatbot, we introduce the effective\ntraining process of PLATO-2 via curriculum learning. There are two stages\ninvolved in the learning process. In the first stage, a coarse-grained\ngeneration model is trained to learn response generation under the simplified\nframework of one-to-one mapping. In the second stage, a fine-grained generative\nmodel augmented with latent variables and an evaluation model are further\ntrained to generate diverse responses and to select the best response,\nrespectively. PLATO-2 was trained on both Chinese and English data, whose\neffectiveness and superiority are verified through comprehensive evaluations,\nachieving new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00217v4",
            "date": "2020-07-01 04:05:48+00:00",
            "title": "Transferability of Natural Language Inference to Biomedical Question Answering",
            "authors": [
                "Minbyul Jeong",
                "Mujeen Sung",
                "Gangwoo Kim",
                "Donghyeon Kim",
                "Wonjin Yoon",
                "Jaehyo Yoo",
                "Jaewoo Kang"
            ],
            "abstract": "Biomedical question answering (QA) is a challenging task due to the scarcity\nof data and the requirement of domain expertise. Pre-trained language models\nhave been used to address these issues. Recently, learning relationships\nbetween sentence pairs has been proved to improve performance in general QA. In\nthis paper, we focus on applying BioBERT to transfer the knowledge of natural\nlanguage inference (NLI) to biomedical QA. We observe that BioBERT trained on\nthe NLI dataset obtains better performance on Yes/No (+5.59%), Factoid\n(+0.53%), List type (+13.58%) questions compared to performance obtained in a\nprevious challenge (BioASQ 7B Phase B). We present a sequential transfer\nlearning method that significantly performed well in the 8th BioASQ Challenge\n(Phase B). In sequential transfer learning, the order in which tasks are\nfine-tuned is important. We measure an unanswerable rate of the extractive QA\nsetting when the formats of factoid and list type questions are converted to\nthe format of the Stanford Question Answering Dataset (SQuAD).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00257v2",
            "date": "2020-07-01 05:40:16+00:00",
            "title": "So What's the Plan? Mining Strategic Planning Documents",
            "authors": [
                "Ekaterina Artemova",
                "Tatiana Batura",
                "Anna Golenkovskaya",
                "Vitaly Ivanin",
                "Vladimir Ivanov",
                "Veronika Sarkisyan",
                "Ivan Smurov",
                "Elena Tutubalina"
            ],
            "abstract": "In this paper we present a corpus of Russian strategic planning documents,\nRuREBus. This project is grounded both from language technology and\ne-government perspectives. Not only new language sources and tools are being\ndeveloped, but also their applications to e-goverment research. We demonstrate\nthe pipeline for creating a text corpus from scratch. First, the annotation\nschema is designed. Next texts are marked up using human-in-the-loop strategy,\nso that preliminary annotations are derived from a machine learning model and\nare manually corrected. The amount of annotated texts is large enough to\nshowcase what insights can be gained from RuREBus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00320v1",
            "date": "2020-07-01 08:33:44+00:00",
            "title": "Iterative Paraphrastic Augmentation with Discriminative Span Alignment",
            "authors": [
                "Ryan Culkin",
                "J. Edward Hu",
                "Elias Stengel-Eskin",
                "Guanghui Qin",
                "Benjamin Van Durme"
            ],
            "abstract": "We introduce a novel paraphrastic augmentation strategy based on\nsentence-level lexically constrained paraphrasing and discriminative span\nalignment. Our approach allows for the large-scale expansion of existing\nresources, or the rapid creation of new resources from a small,\nmanually-produced seed corpus. We illustrate our framework on the Berkeley\nFrameNet Project, a large-scale language understanding effort spanning more\nthan two decades of human labor. Based on roughly four days of collecting\ntraining data for the alignment model and approximately one day of parallel\ncompute, we automatically generate 495,300 unique (Frame, Trigger) combinations\nannotated in context, a roughly 50x expansion atop FrameNet v1.7.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00916v1",
            "date": "2020-07-02 06:50:30+00:00",
            "title": "Fact-based Text Editing",
            "authors": [
                "Hayate Iso",
                "Chao Qiao",
                "Hang Li"
            ],
            "abstract": "We propose a novel text editing task, referred to as \\textit{fact-based text\nediting}, in which the goal is to revise a given document to better describe\nthe facts in a knowledge base (e.g., several triples). The task is important in\npractice because reflecting the truth is a common requirement in text editing.\nFirst, we propose a method for automatically generating a dataset for research\non fact-based text editing, where each instance consists of a draft text, a\nrevised text, and several facts represented in triples. We apply the method\ninto two public table-to-text datasets, obtaining two new datasets consisting\nof 233k and 37k instances, respectively. Next, we propose a new neural network\narchitecture for fact-based text editing, called \\textsc{FactEditor}, which\nedits a draft text by referring to given facts using a buffer, a stream, and a\nmemory. A straightforward approach to address the problem would be to employ an\nencoder-decoder model. Our experimental results on the two datasets show that\n\\textsc{FactEditor} outperforms the encoder-decoder approach in terms of\nfidelity and fluency. The results also show that \\textsc{FactEditor} conducts\ninference faster than the encoder-decoder approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00924v1",
            "date": "2020-07-02 06:59:53+00:00",
            "title": "IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVE",
            "authors": [
                "Luxi Xing",
                "Yuqiang Xie",
                "Yue Hu",
                "Wei Peng"
            ],
            "abstract": "This paper introduces our systems for the first two subtasks of SemEval\nTask4: Commonsense Validation and Explanation. To clarify the intention for\njudgment and inject contrastive information for selection, we propose the input\nreconstruction strategy with prompt templates. Specifically, we formalize the\nsubtasks into the multiple-choice question answering format and construct the\ninput with the prompt templates, then, the final prediction of question\nanswering is considered as the result of subtasks. Experimental results show\nthat our approaches achieve significant performance compared with the baseline\nsystems. Our approaches secure the third rank on both official test sets of the\nfirst two subtasks with an accuracy of 96.4 and an accuracy of 94.3\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00968v1",
            "date": "2020-07-02 08:59:15+00:00",
            "title": "Project PIAF: Building a Native French Question-Answering Dataset",
            "authors": [
                "Rachel Keraron",
                "Guillaume Lancrenon",
                "Mathilde Bras",
                "Fr\u00e9d\u00e9ric Allary",
                "Gilles Moyse",
                "Thomas Scialom",
                "Edmundo-Pavel Soriano-Morales",
                "Jacopo Staiano"
            ],
            "abstract": "Motivated by the lack of data for non-English languages, in particular for\nthe evaluation of downstream tasks such as Question Answering, we present a\nparticipatory effort to collect a native French Question Answering Dataset.\nFurthermore, we describe and publicly release the annotation tool developed for\nour collection effort, along with the data obtained and preliminary baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01176v1",
            "date": "2020-07-02 14:57:28+00:00",
            "title": "Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset",
            "authors": [
                "Brian Roark",
                "Lawrence Wolf-Sonkin",
                "Christo Kirov",
                "Sabrina J. Mielke",
                "Cibu Johny",
                "Isin Demirsahin",
                "Keith Hall"
            ],
            "abstract": "This paper describes the Dakshina dataset, a new resource consisting of text\nin both the Latin and native scripts for 12 South Asian languages. The dataset\nincludes, for each language: 1) native script Wikipedia text; 2) a romanization\nlexicon; and 3) full sentence parallel data in both a native script of the\nlanguage and the basic Latin alphabet. We document the methods used for\npreparation and selection of the Wikipedia text in each language; collection of\nattested romanizations for sampled lexicons; and manual romanization of\nheld-out sentences from the native script collections. We additionally provide\nbaseline results on several tasks made possible by the dataset, including\nsingle word transliteration, full sentence transliteration, and language\nmodeling of native script and romanized text. Keywords: romanization,\ntransliteration, South Asian languages",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01189v3",
            "date": "2020-07-02 15:21:56+00:00",
            "title": "Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis",
            "authors": [
                "Avinash Madasu",
                "Vijjini Anvesh Rao"
            ],
            "abstract": "Elastic Weight Consolidation (EWC) is a technique used in overcoming\ncatastrophic forgetting between successive tasks trained on a neural network.\nWe use this phenomenon of information sharing between tasks for domain\nadaptation. Training data for tasks such as sentiment analysis (SA) may not be\nfairly represented across multiple domains. Domain Adaptation (DA) aims to\nbuild algorithms that leverage information from source domains to facilitate\nperformance on an unseen target domain. We propose a model-independent\nframework - Sequential Domain Adaptation (SDA). SDA draws on EWC for training\non successive source domains to move towards a general domain solution, thereby\nsolving the problem of domain adaptation. We test SDA on convolutional,\nrecurrent, and attention-based architectures. Our experiments show that the\nproposed framework enables simple architectures such as CNNs to outperform\ncomplex state-of-the-art models in domain adaptation of SA. In addition, we\nobserve that the effectiveness of a harder first Anti-Curriculum ordering of\nsource domains leads to maximum performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01359v2",
            "date": "2020-07-02 19:55:08+00:00",
            "title": "Bayesian multilingual topic model for zero-shot cross-lingual topic identification",
            "authors": [
                "Santosh Kesiraju",
                "Sangeet Sagar",
                "Ond\u0159ej Glembek",
                "Luk\u00e1\u0161 Burget",
                "Suryakanth V Gangashetty"
            ],
            "abstract": "This paper presents a Bayesian multilingual topic model for learning\nlanguage-independent document embeddings. Our model learns to represent the\ndocuments in the form of Gaussian distributions, thereby encoding the\nuncertainty in its covariance. We propagate the learned uncertainties through\nlinear classifiers for zero-shot cross-lingual topic identification. Our\nexperiments on 5 language Europarl and Reuters (MLDoc) corpora show that the\nproposed model outperforms multi-lingual word embedding and BiLSTM sentence\nencoder based systems with significant margins in the majority of the transfer\ndirections. Moreover, our system trained under a single day on a single GPU\nwith much lower amounts of data performs competitively as compared to the\nstate-of-the-art universal BiLSTM sentence encoder trained on 93 languages. Our\nexperimental analysis shows that the amount of parallel data improves the\noverall performance of embeddings. Nonetheless, exploiting the uncertainties is\nalways beneficial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01528v1",
            "date": "2020-07-03 07:31:14+00:00",
            "title": "On-The-Fly Information Retrieval Augmentation for Language Models",
            "authors": [
                "Hai Wang",
                "David McAllester"
            ],
            "abstract": "Here we experiment with the use of information retrieval as an augmentation\nfor pre-trained language models. The text corpus used in information retrieval\ncan be viewed as form of episodic memory which grows over time. By augmenting\nGPT 2.0 with information retrieval we achieve a zero shot 15% relative\nreduction in perplexity on Gigaword corpus without any re-training. We also\nvalidate our IR augmentation on an event co-reference task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01652v1",
            "date": "2020-07-03 12:47:13+00:00",
            "title": "Generating Informative Dialogue Responses with Keywords-Guided Networks",
            "authors": [
                "Heng-Da Xu",
                "Xian-Ling Mao",
                "Zewen Chi",
                "Jing-Jing Zhu",
                "Fanshu Sun",
                "Heyan Huang"
            ],
            "abstract": "Recently, open-domain dialogue systems have attracted growing attention. Most\nof them use the sequence-to-sequence (Seq2Seq) architecture to generate\nresponses. However, traditional Seq2Seq-based open-domain dialogue models tend\nto generate generic and safe responses, which are less informative, unlike\nhuman responses. In this paper, we propose a simple but effective\nkeywords-guided Sequence-to-Sequence model (KW-Seq2Seq) which uses keywords\ninformation as guidance to generate open-domain dialogue responses.\nSpecifically, KW-Seq2Seq first uses a keywords decoder to predict some topic\nkeywords, and then generates the final response under the guidance of them.\nExtensive experiments demonstrate that the KW-Seq2Seq model produces more\ninformative, coherent and fluent responses, yielding substantive gain in both\nautomatic and human evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01658v1",
            "date": "2020-07-03 12:53:39+00:00",
            "title": "Playing with Words at the National Library of Sweden -- Making a Swedish BERT",
            "authors": [
                "Martin Malmsten",
                "Love B\u00f6rjeson",
                "Chris Haffenden"
            ],
            "abstract": "This paper introduces the Swedish BERT (\"KB-BERT\") developed by the KBLab for\ndata-driven research at the National Library of Sweden (KB). Building on recent\nefforts to create transformer-based BERT models for languages other than\nEnglish, we explain how we used KB's collections to create and train a new\nlanguage-specific BERT model for Swedish. We also present the results of our\nmodel in comparison with existing models - chiefly that produced by the Swedish\nPublic Employment Service, Arbetsf\\\"ormedlingen, and Google's multilingual\nM-BERT - where we demonstrate that KB-BERT outperforms these in a range of NLP\ntasks from named entity recognition (NER) to part-of-speech tagging (POS). Our\ndiscussion highlights the difficulties that continue to exist given the lack of\ntraining data and testbeds for smaller languages like Swedish. We release our\nmodel for further exploration and research here:\nhttps://github.com/Kungbib/swedish-bert-models .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01667v1",
            "date": "2020-07-03 13:09:37+00:00",
            "title": "Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer",
            "authors": [
                "Kate\u0159ina Mackov\u00e1",
                "Milan Straka"
            ],
            "abstract": "Reading comprehension is a well studied task, with huge training datasets in\nEnglish. This work focuses on building reading comprehension systems for Czech,\nwithout requiring any manually annotated Czech training data. First of all, we\nautomatically translated SQuAD 1.1 and SQuAD 2.0 datasets to Czech to create\ntraining and development data, which we release at\nhttp://hdl.handle.net/11234/1-3249. We then trained and evaluated several BERT\nand XLM-RoBERTa baseline models. However, our main focus lies in cross-lingual\ntransfer models. We report that a XLM-RoBERTa model trained on English data and\nevaluated on Czech achieves very competitive performance, only approximately 2\npercent points worse than a~model trained on the translated Czech data. This\nresult is extremely good, considering the fact that the model has not seen any\nCzech data during training. The cross-lingual transfer approach is very\nflexible and provides a reading comprehension in any language, for which we\nhave enough monolingual raw texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01852v1",
            "date": "2020-07-03 17:58:42+00:00",
            "title": "Language-agnostic BERT Sentence Embedding",
            "authors": [
                "Fangxiaoyu Feng",
                "Yinfei Yang",
                "Daniel Cer",
                "Naveen Arivazhagan",
                "Wei Wang"
            ],
            "abstract": "We adapt multilingual BERT to produce language-agnostic sentence embeddings\nfor 109 languages. %The state-of-the-art for numerous monolingual and\nmultilingual NLP tasks is masked language model (MLM) pretraining followed by\ntask specific fine-tuning. While English sentence embeddings have been obtained\nby fine-tuning a pretrained BERT model, such models have not been applied to\nmultilingual sentence embeddings. Our model combines masked language model\n(MLM) and translation language model (TLM) pretraining with a translation\nranking task using bi-directional dual encoders. The resulting multilingual\nsentence embeddings improve average bi-text retrieval accuracy over 112\nlanguages to 83.7%, well above the 65.5% achieved by the prior state-of-the-art\non Tatoeba. Our sentence embeddings also establish new state-of-the-art results\non BUCC and UN bi-text retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01955v1",
            "date": "2020-07-03 22:22:44+00:00",
            "title": "El Departamento de Nosotros: How Machine Translated Corpora Affects Language Models in MRC Tasks",
            "authors": [
                "Maria Khvalchik",
                "Mikhail Galkin"
            ],
            "abstract": "Pre-training large-scale language models (LMs) requires huge amounts of text\ncorpora. LMs for English enjoy ever growing corpora of diverse language\nresources. However, less resourced languages and their mono- and multilingual\nLMs often struggle to obtain bigger datasets. A typical approach in this case\nimplies using machine translation of English corpora to a target language. In\nthis work, we study the caveats of applying directly translated corpora for\nfine-tuning LMs for downstream natural language processing tasks and\ndemonstrate that careful curation along with post-processing lead to improved\nperformance and overall LMs robustness. In the empirical evaluation, we perform\na comparison of directly translated against curated Spanish SQuAD datasets on\nboth user and system levels. Further experimental results on XQuAD and MLQA\ntransfer-learning evaluation question answering tasks show that presumably\nmultilingual LMs exhibit more resilience to machine translation artifacts in\nterms of the exact match score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02038v1",
            "date": "2020-07-04 08:05:40+00:00",
            "title": "Low Rank Fusion based Transformers for Multimodal Sequences",
            "authors": [
                "Saurav Sahay",
                "Eda Okur",
                "Shachi H Kumar",
                "Lama Nachman"
            ],
            "abstract": "Our senses individually work in a coordinated fashion to express our\nemotional intentions. In this work, we experiment with modeling\nmodality-specific sensory signals to attend to our latent multimodal emotional\nintentions and vice versa expressed via low-rank multimodal fusion and\nmultimodal transformers. The low-rank factorization of multimodal fusion\namongst the modalities helps represent approximate multiplicative latent signal\ninteractions. Motivated by the work of~\\cite{tsai2019MULT} and~\\cite{Liu_2018},\nwe present our transformer-based cross-fusion architecture without any\nover-parameterization of the model. The low-rank fusion helps represent the\nlatent signal interactions while the modality-specific attention helps focus on\nrelevant parts of the signal. We present two methods for the Multimodal\nSentiment and Emotion Recognition results on CMU-MOSEI, CMU-MOSI, and IEMOCAP\ndatasets and show that our models have lesser parameters, train faster and\nperform comparably to many larger fusion-based architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02164v1",
            "date": "2020-07-04 18:46:36+00:00",
            "title": "Birds of a Feather Flock Together: Satirical News Detection via Language Model Differentiation",
            "authors": [
                "Yigeng Zhang",
                "Fan Yang",
                "Yifan Zhang",
                "Eduard Dragut",
                "Arjun Mukherjee"
            ],
            "abstract": "Satirical news is regularly shared in modern social media because it is\nentertaining with smartly embedded humor. However, it can be harmful to society\nbecause it can sometimes be mistaken as factual news, due to its deceptive\ncharacter. We found that in satirical news, the lexical and pragmatical\nattributes of the context are the key factors in amusing the readers. In this\nwork, we propose a method that differentiates the satirical news and true news.\nIt takes advantage of satirical writing evidence by leveraging the difference\nbetween the prediction loss of two language models, one trained on true news\nand the other on satirical news, when given a new news article. We compute\nseveral statistical metrics of language model prediction loss as features,\nwhich are then used to conduct downstream classification. The proposed method\nis computationally effective because the language models capture the language\nusage differences between satirical news documents and traditional news\ndocuments, and are sensitive when applied to documents outside their domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02259v1",
            "date": "2020-07-05 07:48:51+00:00",
            "title": "EmotionGIF-Yankee: A Sentiment Classifier with Robust Model Based Ensemble Methods",
            "authors": [
                "Wei-Yao Wang",
                "Kai-Shiang Chang",
                "Yu-Chien Tang"
            ],
            "abstract": "This paper provides a method to classify sentiment with robust model based\nensemble methods. We preprocess tweet data to enhance coverage of tokenizer. To\nreduce domain bias, we first train tweet dataset for pre-trained language\nmodel. Besides, each classifier has its strengths and weakness, we leverage\ndifferent types of models with ensemble methods: average and power weighted\nsum. From the experiments, we show that our approach has achieved positive\neffect for sentiment classification. Our system reached third place among 26\nteams from the evaluation in SocialNLP 2020 EmotionGIF competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02461v2",
            "date": "2020-07-05 22:23:37+00:00",
            "title": "CORD19STS: COVID-19 Semantic Textual Similarity Dataset",
            "authors": [
                "Xiao Guo",
                "Hengameh Mirzaalian",
                "Ekraam Sabir",
                "Ayush Jaiswal",
                "Wael Abd-Almageed"
            ],
            "abstract": "In order to combat the COVID-19 pandemic, society can benefit from various\nnatural language processing applications, such as dialog medical diagnosis\nsystems and information retrieval engines calibrated specifically for COVID-19.\nThese applications rely on the ability to measure semantic textual similarity\n(STS), making STS a fundamental task that can benefit several downstream\napplications. However, existing STS datasets and models fail to translate their\nperformance to a domain-specific environment such as COVID-19. To overcome this\ngap, we introduce CORD19STS dataset which includes 13,710 annotated sentence\npairs collected from COVID-19 open research dataset (CORD-19) challenge. To be\nspecific, we generated one million sentence pairs using different sampling\nstrategies. We then used a finetuned BERT-like language model, which we call\nSen-SCI-CORD19-BERT, to calculate the similarity scores between sentence pairs\nto provide a balanced dataset with respect to the different semantic similarity\nlevels, which gives us a total of 32K sentence pairs. Each sentence pair was\nannotated by five Amazon Mechanical Turk (AMT) crowd workers, where the labels\nrepresent different semantic similarity levels between the sentence pairs (i.e.\nrelated, somewhat-related, and not-related). After employing a rigorous\nqualification tasks to verify collected annotations, our final CORD19STS\ndataset includes 13,710 sentence pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02598v2",
            "date": "2020-07-06 09:17:14+00:00",
            "title": "Reflection-based Word Attribute Transfer",
            "authors": [
                "Yoichi Ishibashi",
                "Katsuhito Sudoh",
                "Koichiro Yoshino",
                "Satoshi Nakamura"
            ],
            "abstract": "Word embeddings, which often represent such analogic relations as king - man\n+ woman = queen, can be used to change a word's attribute, including its\ngender. For transferring king into queen in this analogy-based manner, we\nsubtract a difference vector man - woman based on the knowledge that king is\nmale. However, developing such knowledge is very costly for words and\nattributes. In this work, we propose a novel method for word attribute transfer\nbased on reflection mappings without such an analogy operation. Experimental\nresults show that our proposed method can transfer the word attributes of the\ngiven words without changing the words that do not have the target attributes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02670v1",
            "date": "2020-07-06 12:03:14+00:00",
            "title": "A Broad-Coverage Deep Semantic Lexicon for Verbs",
            "authors": [
                "James Allen",
                "Hannah An",
                "Ritwik Bose",
                "Will de Beaumont",
                "Choh Man Teng"
            ],
            "abstract": "Progress on deep language understanding is inhibited by the lack of a broad\ncoverage lexicon that connects linguistic behavior to ontological concepts and\naxioms. We have developed COLLIE-V, a deep lexical resource for verbs, with the\ncoverage of WordNet and syntactic and semantic details that meet or exceed\nexisting resources. Bootstrapping from a hand-built lexicon and ontology, new\nontological concepts and lexical entries, together with semantic role\npreferences and entailment axioms, are automatically derived by combining\nmultiple constraints from parsing dictionary definitions and examples. We\nevaluated the accuracy of the technique along a number of different dimensions\nand were able to obtain high accuracy in deriving new concepts and lexical\nentries. COLLIE-V is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02671v1",
            "date": "2020-07-06 12:05:27+00:00",
            "title": "Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences",
            "authors": [
                "Xiangyu Duan",
                "Baijun Ji",
                "Hao Jia",
                "Min Tan",
                "Min Zhang",
                "Boxing Chen",
                "Weihua Luo",
                "Yue Zhang"
            ],
            "abstract": "In this paper, we propose a new task of machine translation (MT), which is\nbased on no parallel sentences but can refer to a ground-truth bilingual\ndictionary. Motivated by the ability of a monolingual speaker learning to\ntranslate via looking up the bilingual dictionary, we propose the task to see\nhow much potential an MT system can attain using the bilingual dictionary and\nlarge scale monolingual corpora, while is independent on parallel sentences. We\npropose anchored training (AT) to tackle the task. AT uses the bilingual\ndictionary to establish anchoring points for closing the gap between source\nlanguage and target language. Experiments on various language pairs show that\nour approaches are significantly better than various baselines, including\ndictionary-based word-by-word translation, dictionary-supervised cross-lingual\nword embedding transformation, and unsupervised MT. On distant language pairs\nthat are hard for unsupervised MT to perform well, AT performs remarkably\nbetter, achieving performances comparable to supervised SMT trained on more\nthan 4M parallel sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02758v1",
            "date": "2020-07-06 13:58:51+00:00",
            "title": "Sentiment Polarity Detection on Bengali Book Reviews Using Multinomial Naive Bayes",
            "authors": [
                "Eftekhar Hossain",
                "Omar Sharif",
                "Mohammed Moshiul Hoque"
            ],
            "abstract": "Recently, sentiment polarity detection has increased attention to NLP\nresearchers due to the massive availability of customer's opinions or reviews\nin the online platform. Due to the continued expansion of e-commerce sites, the\nrate of purchase of various products, including books, are growing enormously\namong the people. Reader's opinions/reviews affect the buying decision of a\ncustomer in most cases. This work introduces a machine learning-based technique\nto determine sentiment polarities (either positive or negative category) from\nBengali book reviews. To assess the effectiveness of the proposed technique, a\ncorpus with 2000 reviews on Bengali books is developed. A comparative analysis\nwith various approaches (such as logistic regression, naive Bayes, SVM, and\nSGD) also performed by taking into consideration of the unigram, bigram, and\ntrigram features, respectively. Experimental result reveals that the\nmultinomial Naive Bayes with unigram feature outperforms the other techniques\nwith 84% accuracy on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02871v2",
            "date": "2020-07-06 16:35:30+00:00",
            "title": "DART: Open-Domain Structured Data Record to Text Generation",
            "authors": [
                "Linyong Nan",
                "Dragomir Radev",
                "Rui Zhang",
                "Amrit Rau",
                "Abhinand Sivaprasad",
                "Chiachun Hsieh",
                "Xiangru Tang",
                "Aadit Vyas",
                "Neha Verma",
                "Pranav Krishna",
                "Yangxiaokang Liu",
                "Nadia Irwanto",
                "Jessica Pan",
                "Faiaz Rahman",
                "Ahmad Zaidi",
                "Mutethia Mutuma",
                "Yasin Tarabar",
                "Ankit Gupta",
                "Tao Yu",
                "Yi Chern Tan",
                "Xi Victoria Lin",
                "Caiming Xiong",
                "Richard Socher",
                "Nazneen Fatema Rajani"
            ],
            "abstract": "We present DART, an open domain structured DAta Record to Text generation\ndataset with over 82k instances (DARTs). Data-to-Text annotations can be a\ncostly process, especially when dealing with tables which are the major source\nof structured data and contain nontrivial structures. To this end, we propose a\nprocedure of extracting semantic triples from tables that encodes their\nstructures by exploiting the semantic dependencies among table headers and the\ntable title. Our dataset construction framework effectively merged\nheterogeneous sources from open domain semantic parsing and dialogue-act-based\nmeaning representation tasks by utilizing techniques such as: tree ontology\nannotation, question-answer pair to declarative sentence conversion, and\npredicate unification, all with minimum post-editing. We present systematic\nevaluation on DART as well as new state-of-the-art results on WebNLG 2017 to\nshow that DART (1) poses new challenges to existing data-to-text datasets and\n(2) facilitates out-of-domain generalization. Our data and code can be found at\nhttps://github.com/Yale-LILY/dart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02880v2",
            "date": "2020-07-06 16:48:11+00:00",
            "title": "Contextualized Spoken Word Representations from Convolutional Autoencoders",
            "authors": [
                "Prakamya Mishra",
                "Pranav Mathur"
            ],
            "abstract": "A lot of work has been done to build text-based language models for\nperforming different NLP tasks, but not much research has been done in the case\nof audio-based language models. This paper proposes a Convolutional Autoencoder\nbased neural architecture to model syntactically and semantically adequate\ncontextualized representations of varying length spoken words. The use of such\nrepresentations can not only lead to great advances in the audio-based NLP\ntasks but can also curtail the loss of information like tone, expression,\naccent, etc while converting speech to text to perform these tasks. The\nperformance of the proposed model is validated by (1) examining the generated\nvector space, and (2) evaluating its performance on three benchmark datasets\nfor measuring word similarities, against existing widely used text-based\nlanguage models that are trained on the transcriptions. The proposed model was\nable to demonstrate its robustness when compared to the other two\nlanguage-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03006v1",
            "date": "2020-07-06 18:48:11+00:00",
            "title": "Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords",
            "authors": [
                "Tom Kocmi",
                "Martin Popel",
                "Ondrej Bojar"
            ],
            "abstract": "We present a new release of the Czech-English parallel corpus CzEng 2.0\nconsisting of over 2 billion words (2 \"gigawords\") in each language. The corpus\ncontains document-level information and is filtered with several techniques to\nlower the amount of noise. In addition to the data in the previous version of\nCzEng, it contains new authentic and also high-quality synthetic parallel data.\nCzEng is freely available for research and educational purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03140v1",
            "date": "2020-07-07 00:19:47+00:00",
            "title": "Research on Annotation Rules and Recognition Algorithm Based on Phrase Window",
            "authors": [
                "Guang Liu",
                "Gang Tu",
                "Zheng Li",
                "Yi-Jian Liu"
            ],
            "abstract": "At present, most Natural Language Processing technology is based on the\nresults of Word Segmentation for Dependency Parsing, which mainly uses an\nend-to-end method based on supervised learning. There are two main problems\nwith this method: firstly, the la-beling rules are complex and the data is too\ndifficult to label, the workload of which is large; secondly, the algorithm\ncannot recognize the multi-granularity and diversity of language components. In\norder to solve these two problems, we propose labeling rules based on phrase\nwindows, and designed corresponding phrase recognition algorithms. The labeling\nrule uses phrases as the minimum unit, di-vides sentences into 7 types of\nnestable phrase types, and marks the grammatical dependencies between phrases.\nThe corresponding algorithm, drawing on the idea of identifying the target area\nin the image field, can find the start and end positions of various phrases in\nthe sentence, and realize the synchronous recognition of nested phrases and\ngrammatical dependencies. The results of the experiment shows that the labeling\nrule is convenient and easy to use, and there is no ambiguity; the algorithm is\nmore grammatically multi-granular and diverse than the end-to-end algorithm.\nExperiments on the CPWD dataset improve the accuracy of the end-to-end method\nby about 1 point. The corresponding method was applied to the CCL2018\ncompetition, and the first place in the Chinese Metaphor Sentiment Analysis\nTask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03541v1",
            "date": "2020-07-07 15:14:32+00:00",
            "title": "scb-mt-en-th-2020: A Large English-Thai Parallel Corpus",
            "authors": [
                "Lalita Lowphansirikul",
                "Charin Polpanumas",
                "Attapol T. Rutherford",
                "Sarana Nutanong"
            ],
            "abstract": "The primary objective of our work is to build a large-scale English-Thai\ndataset for machine translation. We construct an English-Thai machine\ntranslation dataset with over 1 million segment pairs, curated from various\nsources, namely news, Wikipedia articles, SMS messages, task-based dialogs,\nweb-crawled data and government documents. Methodology for gathering data,\nbuilding parallel texts and removing noisy sentence pairs are presented in a\nreproducible manner. We train machine translation models based on this dataset.\nOur models' performance are comparable to that of Google Translation API (as of\nMay 2020) for Thai-English and outperform Google when the Open Parallel Corpus\n(OPUS) is included in the training data for both Thai-English and English-Thai\ntranslation. The dataset, pre-trained models, and source code to reproduce our\nwork are available for public use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03765v1",
            "date": "2020-07-07 20:01:42+00:00",
            "title": "Evaluating German Transformer Language Models with Syntactic Agreement Tests",
            "authors": [
                "Karolina Zaczynska",
                "Nils Feldhus",
                "Robert Schwarzenberg",
                "Aleksandra Gabryszak",
                "Sebastian M\u00f6ller"
            ],
            "abstract": "Pre-trained transformer language models (TLMs) have recently refashioned\nnatural language processing (NLP): Most state-of-the-art NLP models now operate\non top of TLMs to benefit from contextualization and knowledge induction. To\nexplain their success, the scientific community conducted numerous analyses.\nBesides other methods, syntactic agreement tests were utilized to analyse TLMs.\nMost of the studies were conducted for the English language, however. In this\nwork, we analyse German TLMs. To this end, we design numerous agreement tasks,\nsome of which consider peculiarities of the German language. Our experimental\nresults show that state-of-the-art German TLMs generally perform well on\nagreement tasks, but we also identify and discuss syntactic structures that\npush them to their limits.",
            "journal ref": "Proceedings of the 5th Swiss Text Analytics Conference and the\n  16th Conference on Natural Language Processing, SwissText/KONVENS 2020,\n  Zurich, Switzerland, June 23-25, 2020 [online only]. CEUR Workshop\n  Proceedings 2624",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03860v1",
            "date": "2020-07-08 02:30:00+00:00",
            "title": "Research on multi-dimensional end-to-end phrase recognition algorithm based on background knowledge",
            "authors": [
                "Zheng Li",
                "Gang Tu",
                "Guang Liu",
                "Zhi-Qiang Zhan",
                "Yi-Jian Liu"
            ],
            "abstract": "At present, the deep end-to-end method based on supervised learning is used\nin entity recognition and dependency analysis. There are two problems in this\nmethod: firstly, background knowledge cannot be introduced; secondly, multi\ngranularity and nested features of natural language cannot be recognized. In\norder to solve these problems, the annotation rules based on phrase window are\nproposed, and the corresponding multi-dimensional end-to-end phrase recognition\nalgorithm is designed. This annotation rule divides sentences into seven types\nof nested phrases, and indicates the dependency between phrases. The algorithm\ncan not only introduce background knowledge, recognize all kinds of nested\nphrases in sentences, but also recognize the dependency between phrases. The\nexperimental results show that the annotation rule is easy to use and has no\nambiguity; the matching algorithm is more consistent with the multi granularity\nand diversity characteristics of syntax than the traditional end-to-end\nalgorithm. The experiment on CPWD dataset, by introducing background knowledge,\nthe new algorithm improves the accuracy of the end-to-end method by more than\none point. The corresponding method was applied to the CCL 2018 competition and\nwon the first place in the task of Chinese humor type recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03875v2",
            "date": "2020-07-08 03:28:04+00:00",
            "title": "KQA Pro: A Large-Scale Dataset with Interpretable Programs and Accurate SPARQLs for Complex Question Answering over Knowledge Base",
            "authors": [
                "Jiaxin Shi",
                "Shulin Cao",
                "Liangming Pan",
                "Yutong Xiang",
                "Lei Hou",
                "Juanzi Li",
                "Hanwang Zhang",
                "Bin He"
            ],
            "abstract": "Complex question answering over knowledge base (Complex KBQA) is challenging\nbecause it requires various compositional reasoning capabilities, such as\nmulti-hop inference, attribute comparison, set operation, and etc. Existing\nbenchmarks have some shortcomings that limit the development of Complex KBQA:\n1) they only provide QA pairs without explicit reasoning processes; 2)\nquestions are either generated by templates, leading to poor diversity, or on a\nsmall scale. To this end, we introduce KQA Pro, a large-scale dataset for\nComplex KBQA. We define a compositional and highly-interpretable formal format,\nnamed Program, to represent the reasoning process of complex questions. We\npropose compositional strategies to generate questions, corresponding SPARQLs,\nand Programs with a small number of templates, and then paraphrase the\ngenerated questions to natural language questions (NLQ) by crowdsourcing,\ngiving rise to around 120K diverse instances. SPARQL and Program depict two\ncomplementary solutions to answer complex questions, which can benefit a large\nspectrum of QA methods. Besides the QA task, KQA Pro can also serves for the\nsemantic parsing task. As far as we know, it is currently the largest corpus of\nNLQ-to-SPARQL and NLQ-to-Program. We conduct extensive experiments to evaluate\nwhether machines can learn to answer our complex questions in different cases,\nthat is, with only QA supervision or with intermediate SPARQL/Program\nsupervision. We find that state-of-the-art KBQA methods learnt from only QA\npairs perform very poor on our dataset, implying our questions are more\nchallenging than previous datasets. However, pretrained models learnt from our\nNLQ-to-SPARQL and NLQ-to-Program annotations surprisingly achieve about 90\\%\nanswering accuracy, which is even close to the human expert performance...",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03876v1",
            "date": "2020-07-08 03:31:03+00:00",
            "title": "Audio-Visual Understanding of Passenger Intents for In-Cabin Conversational Agents",
            "authors": [
                "Eda Okur",
                "Shachi H Kumar",
                "Saurav Sahay",
                "Lama Nachman"
            ],
            "abstract": "Building multimodal dialogue understanding capabilities situated in the\nin-cabin context is crucial to enhance passenger comfort in autonomous vehicle\n(AV) interaction systems. To this end, understanding passenger intents from\nspoken interactions and vehicle vision systems is a crucial component for\ndeveloping contextual and visually grounded conversational agents for AV.\nTowards this goal, we explore AMIE (Automated-vehicle Multimodal In-cabin\nExperience), the in-cabin agent responsible for handling multimodal\npassenger-vehicle interactions. In this work, we discuss the benefits of a\nmultimodal understanding of in-cabin utterances by incorporating\nverbal/language input together with the non-verbal/acoustic and visual clues\nfrom inside and outside the vehicle. Our experimental results outperformed\ntext-only baselines as we achieved improved performances for intent detection\nwith a multimodal approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03988v1",
            "date": "2020-07-08 09:49:38+00:00",
            "title": "Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases",
            "authors": [
                "Yu Liu",
                "Quanming Yao",
                "Yong Li"
            ],
            "abstract": "With the rapid development of knowledge bases (KBs), link prediction task,\nwhich completes KBs with missing facts, has been broadly studied in especially\nbinary relational KBs (a.k.a knowledge graph) with powerful tensor\ndecomposition related methods. However, the ubiquitous n-ary relational KBs\nwith higher-arity relational facts are paid less attention, in which existing\ntranslation based and neural network based approaches have weak expressiveness\nand high complexity in modeling various relations. Tensor decomposition has not\nbeen considered for n-ary relational KBs, while directly extending tensor\ndecomposition related methods of binary relational KBs to the n-ary case does\nnot yield satisfactory results due to exponential model complexity and their\nstrong assumptions on binary relations. To generalize tensor decomposition for\nn-ary relational KBs, in this work, we propose GETD, a generalized model based\non Tucker decomposition and Tensor Ring decomposition. The existing negative\nsampling technique is also generalized to the n-ary case for GETD. In addition,\nwe theoretically prove that GETD is fully expressive to completely represent\nany KBs. Extensive evaluations on two representative n-ary relational KB\ndatasets demonstrate the superior performance of GETD, significantly improving\nthe state-of-the-art methods by over 15\\%. Moreover, GETD further obtains the\nstate-of-the-art results on the benchmark binary relational KB datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04070v1",
            "date": "2020-07-08 12:38:50+00:00",
            "title": "Learning Neural Textual Representations for Citation Recommendation",
            "authors": [
                "Binh Thanh Kieu",
                "Inigo Jauregi Unanue",
                "Son Bao Pham",
                "Hieu Xuan Phan",
                "Massimo Piccardi"
            ],
            "abstract": "With the rapid growth of the scientific literature, manually selecting\nappropriate citations for a paper is becoming increasingly challenging and\ntime-consuming. While several approaches for automated citation recommendation\nhave been proposed in the recent years, effective document representations for\ncitation recommendation are still elusive to a large extent. For this reason,\nin this paper we propose a novel approach to citation recommendation which\nleverages a deep sequential representation of the documents (Sentence-BERT)\ncascaded with Siamese and triplet networks in a submodular scoring function. To\nthe best of our knowledge, this is the first approach to combine deep\nrepresentations and submodular selection for a task of citation recommendation.\nExperiments have been carried out using a popular benchmark dataset - the ACL\nAnthology Network corpus - and evaluated against baselines and a\nstate-of-the-art approach using metrics such as the MRR and F1-at-k score. The\nresults show that the proposed approach has been able to outperform all the\ncompared approaches in every measured metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04297v2",
            "date": "2020-06-27 21:01:52+00:00",
            "title": "Open Domain Suggestion Mining Leveraging Fine-Grained Analysis",
            "authors": [
                "Shreya Singal",
                "Tanishq Goel",
                "Shivang Chopra",
                "Sonika Dahiya"
            ],
            "abstract": "Suggestion mining tasks are often semantically complex and lack sophisticated\nmethodologies that can be applied to real-world data. The presence of\nsuggestions across a large diversity of domains and the absence of large\nlabelled and balanced datasets render this task particularly challenging to\ndeal with. In an attempt to overcome these challenges, we propose a two-tier\npipeline that leverages Discourse Marker based oversampling and fine-grained\nsuggestion mining techniques to retrieve suggestions from online forums.\nThrough extensive comparison on a real-world open-domain suggestion dataset, we\ndemonstrate how the oversampling technique combined with transformer based\nfine-grained analysis can beat the state of the art. Additionally, we perform\nextensive qualitative and qualitative analysis to give construct validity to\nour proposed pipeline. Finally, we discuss the practical, computational and\nreproducibility aspects of the deployment of our pipeline across the web.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04300v2",
            "date": "2020-06-27 20:24:35+00:00",
            "title": "Normalizador Neural de Datas e Endere\u00e7os",
            "authors": [
                "Gustavo Plensack",
                "Paulo Finardi"
            ],
            "abstract": "Documents of any kind present a wide variety of date and address formats, in\nsome cases dates can be written entirely in full or even have different types\nof separators. The pattern disorder in addresses is even greater due to the\ngreater possibility of interchanging between streets, neighborhoods, cities and\nstates. In the context of natural language processing, problems of this nature\nare handled by rigid tools such as ReGex or DateParser, which are efficient as\nlong as the expected input is pre-configured. When these algorithms are given\nan unexpected format, errors and unwanted outputs happen. To circumvent this\nchallenge, we present a solution with deep neural networks state of art T5 that\ntreats non-preconfigured formats of dates and addresses with accuracy above 90%\nin some cases. With this model, our proposal brings generalization to the task\nof normalizing dates and addresses. We also deal with this problem with noisy\ndata that simulates possible errors in the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04301v1",
            "date": "2020-06-30 16:44:28+00:00",
            "title": "Segmentation Approach for Coreference Resolution Task",
            "authors": [
                "Aref Jafari",
                "Ali Ghodsi"
            ],
            "abstract": "In coreference resolution, it is important to consider all members of a\ncoreference cluster and decide about all of them at once. This technique can\nhelp to avoid losing precision and also in finding long-distance relations. The\npresented paper is a report of an ongoing study on an idea which proposes a new\napproach for coreference resolution which can resolve all coreference mentions\nto a given mention in the document in one pass. This has been accomplished by\ndefining an embedding method for the position of all members of a coreference\ncluster in a document and resolving all of them for a given mention. In the\nproposed method, the BERT model has been used for encoding the documents and a\nhead network designed to capture the relations between the embedded tokens.\nThese are then converted to the proposed span position embedding matrix which\nembeds the position of all coreference mentions in the document. We tested this\nidea on CoNLL 2012 dataset and although the preliminary results from this\nmethod do not quite meet the state-of-the-art results, they are promising and\nthey can capture features like long-distance relations better than the other\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04302v1",
            "date": "2020-07-02 06:07:29+00:00",
            "title": "A Novel BGCapsule Network for Text Classification",
            "authors": [
                "Akhilesh Kumar Gangwar",
                "Vadlamani Ravi"
            ],
            "abstract": "Several text classification tasks such as sentiment analysis, news\ncategorization, multi-label classification and opinion classification are\nchallenging problems even for modern deep learning networks. Recently, Capsule\nNetworks (CapsNets) are proposed for image classification. It has been shown\nthat CapsNets have several advantages over Convolutional Neural Networks\n(CNNs), while their validity in the domain of text has been less explored. In\nthis paper, we propose a novel hybrid architecture viz., BGCapsule, which is a\nCapsule model preceded by an ensemble of Bidirectional Gated Recurrent Units\n(BiGRU) for several text classification tasks. We employed an ensemble of\nBidirectional GRUs for feature extraction layer preceding the primary capsule\nlayer. The hybrid architecture, after performing basic pre-processing steps,\nconsists of five layers: an embedding layer based on GloVe, a BiGRU based\nensemble layer, a primary capsule layer, a flatten layer and fully connected\nReLU layer followed by a fully connected softmax layer. In order to evaluate\nthe effectiveness of BGCapsule, we conducted extensive experiments on five\nbenchmark datasets (ranging from 10,000 records to 700,000 records) including\nMovie Review (MR Imdb 2005), AG News dataset, Dbpedia ontology dataset, Yelp\nReview Full dataset and Yelp review polarity dataset. These benchmarks cover\nseveral text classification tasks such as news categorization, sentiment\nanalysis, multiclass classification, multi-label classification and opinion\nclassification. We found that our proposed architecture (BGCapsule) achieves\nbetter accuracy compared to the existing methods without the help of any\nexternal linguistic knowledge such as positive sentiment keywords and negative\nsentiment keywords. Further, BGCapsule converged faster compared to other\nextant techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04303v1",
            "date": "2020-07-05 08:10:30+00:00",
            "title": "Tweets Sentiment Analysis via Word Embeddings and Machine Learning Techniques",
            "authors": [
                "Aditya Sharma",
                "Alex Daniels"
            ],
            "abstract": "Sentiment analysis of social media data consists of attitudes, assessments,\nand emotions which can be considered a way human think. Understanding and\nclassifying the large collection of documents into positive and negative\naspects are a very difficult task. Social networks such as Twitter, Facebook,\nand Instagram provide a platform in order to gather information about peoples\nsentiments and opinions. Considering the fact that people spend hours daily on\nsocial media and share their opinion on various different topics helps us\nanalyze sentiments better. More and more companies are using social media tools\nto provide various services and interact with customers. Sentiment Analysis\n(SA) classifies the polarity of given tweets to positive and negative tweets in\norder to understand the sentiments of the public. This paper aims to perform\nsentiment analysis of real-time 2019 election twitter data using the feature\nselection model word2vec and the machine learning algorithm random forest for\nsentiment classification. Word2vec with Random Forest improves the accuracy of\nsentiment analysis significantly compared to traditional methods such as BOW\nand TF-IDF. Word2vec improves the quality of features by considering contextual\nsemantics of words in a text hence improving the accuracy of machine learning\nand sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04428v1",
            "date": "2020-07-08 20:53:14+00:00",
            "title": "Discourse Coherence, Reference Grounding and Goal Oriented Dialogue",
            "authors": [
                "Baber Khalid",
                "Malihe Alikhani",
                "Michael Fellner",
                "Brian McMahan",
                "Matthew Stone"
            ],
            "abstract": "Prior approaches to realizing mixed-initiative human--computer referential\ncommunication have adopted information-state or collaborative problem-solving\napproaches. In this paper, we argue for a new approach, inspired by\ncoherence-based models of discourse such as SDRT \\cite{asher-lascarides:2003a},\nin which utterances attach to an evolving discourse structure and the\nassociated knowledge graph of speaker commitments serves as an interface to\nreal-world reasoning and conversational strategy. As first steps towards\nimplementing the approach, we describe a simple dialogue system in a\nreferential communication domain that accumulates constraints across discourse,\ninterprets them using a learned probabilistic model, and plans clarification\nusing reinforcement learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04626v2",
            "date": "2020-07-09 08:26:22+00:00",
            "title": "DISCO PAL: Diachronic Spanish Sonnet Corpus with Psychological and Affective Labels",
            "authors": [
                "Alberto Barbado",
                "V\u00edctor Fresno",
                "\u00c1ngeles Manjarr\u00e9s Riesco",
                "Salvador Ros"
            ],
            "abstract": "Nowadays, there are many applications of text mining over corpora from\ndifferent languages. However, most of them are based on texts in prose, lacking\napplications that work with poetry texts. An example of an application of text\nmining in poetry is the usage of features derived from their individual words\nin order to capture the lexical, sublexical and interlexical meaning, and infer\nthe General Affective Meaning (GAM) of the text. However, though this proposal\nhas been proved as useful for poetry in some languages, there is a lack of\nstudies for both Spanish poetry and for highly-structured poetic compositions\nsuch as sonnets. This article presents a study over an annotated corpus of\nSpanish sonnets, in order to analyse if it is possible to build features from\ntheir individual words for predicting their GAM. The purpose of this is to\nmodel sonnets at an affective level. The article also analyses the relationship\nbetween the GAM of the sonnets and the content itself. For this, we consider\nthe content from a psychological perspective, identifying with tags when a\nsonnet is related to a specific term. Then, we study how GAM changes according\nto each of those psychological terms. The corpus used contains 274 Spanish\nsonnets from authors of different centuries, from 15th to 19th. This corpus was\nannotated by different domain experts. The experts annotated the poems with\naffective and lexico-semantic features, as well as with domain concepts that\nbelong to psychology. Thanks to this, the corpus of sonnets can be used in\ndifferent applications, such as poetry recommender systems, personality text\nmining studies of the authors, or the usage of poetry for therapeutic purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04629v1",
            "date": "2020-07-09 08:29:57+00:00",
            "title": "Principal Word Vectors",
            "authors": [
                "Ali Basirat",
                "Christian Hardmeier",
                "Joakim Nivre"
            ],
            "abstract": "We generalize principal component analysis for embedding words into a vector\nspace. The generalization is made in two major levels. The first is to\ngeneralize the concept of the corpus as a counting process which is defined by\nthree key elements vocabulary set, feature (annotation) set, and context. This\ngeneralization enables the principal word embedding method to generate word\nvectors with regard to different types of contexts and different types of\nannotations provided for a corpus. The second is to generalize the\ntransformation step used in most of the word embedding methods. To this end, we\ndefine two levels of transformations. The first is a quadratic transformation,\nwhich accounts for different types of weighting over the vocabulary units and\ncontextual features. Second is an adaptive non-linear transformation, which\nreshapes the data distribution to be meaningful to principal component\nanalysis. The effect of these generalizations on the word vectors is\nintrinsically studied with regard to the spread and the discriminability of the\nword vectors. We also provide an extrinsic evaluation of the contribution of\nthe principal word vectors on a word similarity benchmark and the task of\ndependency parsing. Our experiments are finalized by a comparison between the\nprincipal word vectors and other sets of word vectors generated with popular\nword embedding methods. The results obtained from our intrinsic evaluation\nmetrics show that the spread and the discriminability of the principal word\nvectors are higher than that of other word embedding methods. The results\nobtained from the extrinsic evaluation metrics show that the principal word\nvectors are better than some of the word embedding methods and on par with\npopular methods of word embedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04686v1",
            "date": "2020-07-09 10:29:19+00:00",
            "title": "Greedy Transition-Based Dependency Parsing with Discrete and Continuous Supertag Features",
            "authors": [
                "Ali Basirat",
                "Joakim Nivre"
            ],
            "abstract": "We study the effect of rich supertag features in greedy transition-based\ndependency parsing. While previous studies have shown that sparse boolean\nfeatures representing the 1-best supertag of a word can improve parsing\naccuracy, we show that we can get further improvements by adding a continuous\nvector representation of the entire supertag distribution for a word. In this\nway, we achieve the best results for greedy transition-based parsing with\nsupertag features with $88.6\\%$ LAS and $90.9\\%$ UASon the English Penn\nTreebank converted to Stanford Dependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04792v1",
            "date": "2020-07-07 19:28:18+00:00",
            "title": "Targeting the Benchmark: On Methodology in Current Natural Language Processing Research",
            "authors": [
                "David Schlangen"
            ],
            "abstract": "It has become a common pattern in our field: One group introduces a language\ntask, exemplified by a dataset, which they argue is challenging enough to serve\nas a benchmark. They also provide a baseline model for it, which then soon is\nimproved upon by other groups. Often, research efforts then move on, and the\npattern repeats itself. What is typically left implicit is the argumentation\nfor why this constitutes progress, and progress towards what. In this paper, we\ntry to step back for a moment from this pattern and work out possible\nargumentations and their parts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04874v1",
            "date": "2020-07-09 15:21:59+00:00",
            "title": "CompRes: A Dataset for Narrative Structure in News",
            "authors": [
                "Effi Levi",
                "Guy Mor",
                "Shaul Shenhav",
                "Tamir Sheafer"
            ],
            "abstract": "This paper addresses the task of automatically detecting narrative structures\nin raw texts. Previous works have utilized the oral narrative theory by Labov\nand Waletzky to identify various narrative elements in personal stories texts.\nInstead, we direct our focus to news articles, motivated by their growing\nsocial impact as well as their role in creating and shaping public opinion.\n  We introduce CompRes -- the first dataset for narrative structure in news\nmedia. We describe the process in which the dataset was constructed: first, we\ndesigned a new narrative annotation scheme, better suited for news media, by\nadapting elements from the narrative theory of Labov and Waletzky (Complication\nand Resolution) and adding a new narrative element of our own (Success); then,\nwe used that scheme to annotate a set of 29 English news articles (containing\n1,099 sentences) collected from news and partisan websites. We use the\nannotated dataset to train several supervised models to identify the different\nnarrative elements, achieving an $F_1$ score of up to 0.7. We conclude by\nsuggesting several promising directions for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05044v2",
            "date": "2020-07-09 19:34:18+00:00",
            "title": "Advances of Transformer-Based Models for News Headline Generation",
            "authors": [
                "Alexey Bukhtiyarov",
                "Ilya Gusev"
            ],
            "abstract": "Pretrained language models based on Transformer architecture are the reason\nfor recent breakthroughs in many areas of NLP, including sentiment analysis,\nquestion answering, named entity recognition. Headline generation is a special\nkind of text summarization task. Models need to have strong natural language\nunderstanding that goes beyond the meaning of individual words and sentences\nand an ability to distinguish essential information to succeed in it. In this\npaper, we fine-tune two pretrained Transformer-based models (mBART and\nBertSumAbs) for that task and achieve new state-of-the-art results on the RIA\nand Lenta datasets of Russian news. BertSumAbs increases ROUGE on average by\n2.9 and 2.0 points respectively over previous best score achieved by\nPhrase-Based Attentional Transformer and CopyNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05194v2",
            "date": "2020-07-10 06:36:13+00:00",
            "title": "What Can We Learn From Almost a Decade of Food Tweets",
            "authors": [
                "Uga Spro\u0123is",
                "Mat\u012bss Rikters"
            ],
            "abstract": "We present the Latvian Twitter Eater Corpus - a set of tweets in the narrow\ndomain related to food, drinks, eating and drinking. The corpus has been\ncollected over time-span of over 8 years and includes over 2 million tweets\nentailed with additional useful data. We also separate two sub-corpora of\nquestion and answer tweets and sentiment annotated tweets. We analyse contents\nof the corpus and demonstrate use-cases for the sub-corpora by training\ndomain-specific question-answering and sentiment-analysis models using data\nfrom the corpus.",
            "journal ref": "In Proceedings of the 9th Conference Human Language Technologies -\n  The Baltic Perspective (Baltic HLT 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05290v1",
            "date": "2020-07-10 10:28:54+00:00",
            "title": "Learn to Use Future Information in Simultaneous Translation",
            "authors": [
                "Xueqing Wu",
                "Yingce Xia",
                "Lijun Wu",
                "Shufang Xie",
                "Weiqing Liu",
                "Jiang Bian",
                "Tao Qin",
                "Tie-Yan Liu"
            ],
            "abstract": "Simultaneous neural machine translation (briefly, NMT) has attracted much\nattention recently. In contrast to standard NMT, where the NMT system can\nutilize the full input sentence, simultaneous NMT is formulated as a\nprefix-to-prefix problem, where the system can only utilize the prefix of the\ninput sentence and more uncertainty is introduced to decoding. Wait-$k$ is a\nsimple yet effective strategy for simultaneous NMT, where the decoder generates\nthe output sequence $k$ words behind the input words. We observed that training\nsimultaneous NMT systems with future information (i.e., trained with a larger\n$k$) generally outperforms the standard ones (i.e., trained with the given\n$k$). Based on this observation, we propose a framework that automatically\nlearns how much future information to use in training for simultaneous NMT. We\nfirst build a series of tasks where each one is associated with a different\n$k$, and then learn a model on these tasks guided by a controller. The\ncontroller is jointly trained with the translation model through bi-level\noptimization. We conduct experiments on four datasets to demonstrate the\neffectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05374v1",
            "date": "2020-07-10 13:26:37+00:00",
            "title": "SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics",
            "authors": [
                "Daniel Deutsch",
                "Dan Roth"
            ],
            "abstract": "We present SacreROUGE, an open-source library for using and developing\nsummarization evaluation metrics. SacreROUGE removes many obstacles that\nresearchers face when using or developing metrics: (1) The library provides\nPython wrappers around the official implementations of existing evaluation\nmetrics so they share a common, easy-to-use interface; (2) it provides\nfunctionality to evaluate how well any metric implemented in the library\ncorrelates to human-annotated judgments, so no additional code needs to be\nwritten for a new evaluation metric; and (3) it includes scripts for loading\ndatasets that contain human judgments so they can easily be used for\nevaluation. This work describes the design of the library, including the core\nMetric interface, the command-line API for evaluating summarization models and\nmetrics, and the scripts to load and reformat publicly available datasets. The\ndevelopment of SacreROUGE is ongoing and open to contributions from the\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05772v1",
            "date": "2020-07-11 13:34:44+00:00",
            "title": "I3rab: A New Arabic Dependency Treebank Based on Arabic Grammatical Theory",
            "authors": [
                "Dana Halabi",
                "Ebaa Fayyoumi",
                "Arafat Awajan"
            ],
            "abstract": "Treebanks are valuable linguistic resources that include the syntactic\nstructure of a language sentence in addition to POS-tags and morphological\nfeatures. They are mainly utilized in modeling statistical parsers. Although\nthe statistical natural language parser has recently become more accurate for\nlanguages such as English, those for the Arabic language still have low\naccuracy. The purpose of this paper is to construct a new Arabic dependency\ntreebank based on the traditional Arabic grammatical theory and the\ncharacteristics of the Arabic language, to investigate their effects on the\naccuracy of statistical parsers. The proposed Arabic dependency treebank,\ncalled I3rab, contrasts with existing Arabic dependency treebanks in two main\nconcepts. The first concept is the approach of determining the main word of the\nsentence, and the second concept is the representation of the joined and covert\npronouns. To evaluate I3rab, we compared its performance against a subset of\nPrague Arabic Dependency Treebank that shares a comparable level of details.\nThe conducted experiments show that the percentage improvement reached up to\n7.5% in UAS and 18.8% in LAS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06104v1",
            "date": "2020-07-12 21:48:52+00:00",
            "title": "Neural disambiguation of lemma and part of speech in morphologically rich languages",
            "authors": [
                "Jos\u00e9 Mar\u00eda Hoya Quecedo",
                "Maximilian W. Koppatz",
                "Giacomo Furlan",
                "Roman Yangarber"
            ],
            "abstract": "We consider the problem of disambiguating the lemma and part of speech of\nambiguous words in morphologically rich languages. We propose a method for\ndisambiguating ambiguous words in context, using a large un-annotated corpus of\ntext, and a morphological analyser -- with no manual disambiguation or data\nannotation. We assume that the morphological analyser produces multiple\nanalyses for ambiguous words. The idea is to train recurrent neural networks on\nthe output that the morphological analyser produces for unambiguous words. We\npresent performance on POS and lemma disambiguation that reaches or surpasses\nthe state of the art -- including supervised models -- using no manually\nannotated data. We evaluate the method on several morphologically rich\nlanguages.",
            "journal ref": "Proceedings of LREC-2020: the 12th Conference on Language\n  Resources and Evaluation",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06162v1",
            "date": "2020-07-13 02:53:03+00:00",
            "title": "Do You Have the Right Scissors? Tailoring Pre-trained Language Models via Monte-Carlo Methods",
            "authors": [
                "Ning Miao",
                "Yuxuan Song",
                "Hao Zhou",
                "Lei Li"
            ],
            "abstract": "It has been a common approach to pre-train a language model on a large corpus\nand fine-tune it on task-specific data. In practice, we observe that\nfine-tuning a pre-trained model on a small dataset may lead to over- and/or\nunder-estimation problem. In this paper, we propose MC-Tailor, a novel method\nto alleviate the above issue in text generation tasks by truncating and\ntransferring the probability mass from over-estimated regions to\nunder-estimated ones. Experiments on a variety of text generation datasets show\nthat MC-Tailor consistently and significantly outperforms the fine-tuning\napproach. Our code is available at this url.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06174v1",
            "date": "2020-07-13 03:56:37+00:00",
            "title": "Generating Fluent Adversarial Examples for Natural Languages",
            "authors": [
                "Huangzhao Zhang",
                "Hao Zhou",
                "Ning Miao",
                "Lei Li"
            ],
            "abstract": "Efficiently building an adversarial attacker for natural language processing\n(NLP) tasks is a real challenge. Firstly, as the sentence space is discrete, it\nis difficult to make small perturbations along the direction of gradients.\nSecondly, the fluency of the generated examples cannot be guaranteed. In this\npaper, we propose MHA, which addresses both problems by performing\nMetropolis-Hastings sampling, whose proposal is designed with the guidance of\ngradients. Experiments on IMDB and SNLI show that our proposed MHA outperforms\nthe baseline model on attacking capability. Adversarial training with MAH also\nleads to better robustness and performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06257v1",
            "date": "2020-07-13 09:19:34+00:00",
            "title": "Transformer with Depth-Wise LSTM",
            "authors": [
                "Hongfei Xu",
                "Qiuhui Liu",
                "Deyi Xiong",
                "Josef van Genabith"
            ],
            "abstract": "Increasing the depth of models allows neural models to model complicated\nfunctions but may also lead to optimization issues. The Transformer translation\nmodel employs the residual connection to ensure its convergence. In this paper,\nwe suggest that the residual connection has its drawbacks, and propose to train\nTransformers with the depth-wise LSTM which regards outputs of layers as steps\nin time series instead of residual connections, under the motivation that the\nvanishing gradient problem suffered by deep networks is the same as recurrent\nnetworks applied to long sequences, while LSTM (Hochreiter and Schmidhuber,\n1997) has been proven of good capability in capturing long-distance\nrelationship, and its design may alleviate some drawbacks of residual\nconnections while ensuring the convergence. We integrate the computation of\nmulti-head attention networks and feed-forward networks with the depth-wise\nLSTM for the Transformer, which shows how to utilize the depth-wise LSTM like\nthe residual connection. Our experiment with the 6-layer Transformer shows that\nour approach can bring about significant BLEU improvements in both WMT 14\nEnglish-German and English-French tasks, and our deep Transformer experiment\ndemonstrates the effectiveness of the depth-wise LSTM on the convergence of\ndeep Transformers. Additionally, we propose to measure the impacts of the\nlayer's non-linearity on the performance by distilling the analyzing layer of\nthe trained model into a linear transformation and observing the performance\ndegradation with the replacement. Our analysis results support the more\nefficient use of per-layer non-linearity with depth-wise LSTM than with\nresidual connections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06400v2",
            "date": "2020-07-13 14:25:49+00:00",
            "title": "GGPONC: A Corpus of German Medical Text with Rich Metadata Based on Clinical Practice Guidelines",
            "authors": [
                "Florian Borchert",
                "Christina Lohr",
                "Luise Modersohn",
                "Thomas Langer",
                "Markus Follmann",
                "Jan Philipp Sachs",
                "Udo Hahn",
                "Matthieu-P. Schapranow"
            ],
            "abstract": "The lack of publicly accessible text corpora is a major obstacle for progress\nin natural language processing. For medical applications, unfortunately, all\nlanguage communities other than English are low-resourced. In this work, we\npresent GGPONC (German Guideline Program in Oncology NLP Corpus), a freely\ndistributable German language corpus based on clinical practice guidelines for\noncology. This corpus is one of the largest ever built from German medical\ndocuments. Unlike clinical documents, clinical guidelines do not contain any\npatient-related information and can therefore be used without data protection\nrestrictions. Moreover, GGPONC is the first corpus for the German language\ncovering diverse conditions in a large medical subfield and provides a variety\nof metadata, such as literature references and evidence levels. By applying and\nevaluating existing medical information extraction pipelines for German text,\nwe are able to draw comparisons for the use of medical language to other\ncorpora, medical and non-medical ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06493v1",
            "date": "2020-07-13 16:43:14+00:00",
            "title": "HSD Shared Task in VLSP Campaign 2019:Hate Speech Detection for Social Good",
            "authors": [
                "Xuan-Son Vu",
                "Thanh Vu",
                "Mai-Vu Tran",
                "Thanh Le-Cong",
                "Huyen T M. Nguyen"
            ],
            "abstract": "The paper describes the organisation of the \"HateSpeech Detection\" (HSD) task\nat the VLSP workshop 2019 on detecting the fine-grained presence of hate speech\nin Vietnamese textual items (i.e., messages) extracted from Facebook, which is\nthe most popular social network site (SNS) in Vietnam. The task is organised as\na multi-class classification task and based on a large-scale dataset containing\n25,431 Vietnamese textual items from Facebook. The task participants were\nchallenged to build a classification model that is capable of classifying an\nitem to one of 3 classes, i.e., \"HATE\", \"OFFENSIVE\" and \"CLEAN\". HSD attracted\na large number of participants and was a popular task at VLSP 2019. In\nparticular, there were 71 teams signed up for the task, 14 of them submitted\nresults with 380 valid submissions from 20th September 2019 to 4th October\n2019.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06761v2",
            "date": "2020-07-14 01:41:25+00:00",
            "title": "Can neural networks acquire a structural bias from raw linguistic data?",
            "authors": [
                "Alex Warstadt",
                "Samuel R. Bowman"
            ],
            "abstract": "We evaluate whether BERT, a widely used neural network for sentence\nprocessing, acquires an inductive bias towards forming structural\ngeneralizations through pretraining on raw data. We conduct four experiments\ntesting its preference for structural vs. linear generalizations in different\nstructure-dependent phenomena. We find that BERT makes a structural\ngeneralization in 3 out of 4 empirical domains---subject-auxiliary inversion,\nreflexive binding, and verb tense detection in embedded clauses---but makes a\nlinear generalization when tested on NPI licensing. We argue that these results\nare the strongest evidence so far from artificial learners supporting the\nproposition that a structural bias can be acquired from raw data. If this\nconclusion is correct, it is tentative evidence that some linguistic universals\ncan be acquired by learners without innate biases. However, the precise\nimplications for human language acquisition are unclear, as humans learn\nlanguage from significantly less data than BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07389v1",
            "date": "2020-07-14 22:41:20+00:00",
            "title": "Emoji Prediction: Extensions and Benchmarking",
            "authors": [
                "Weicheng Ma",
                "Ruibo Liu",
                "Lili Wang",
                "Soroush Vosoughi"
            ],
            "abstract": "Emojis are a succinct form of language which can express concrete meanings,\nemotions, and intentions. Emojis also carry signals that can be used to better\nunderstand communicative intent. They have become a ubiquitous part of our\ndaily lives, making them an important part of understanding user-generated\ncontent. The emoji prediction task aims at predicting the proper set of emojis\nassociated with a piece of text. Through emoji prediction, models can learn\nrich representations of the communicative intent of the written text. While\nexisting research on the emoji prediction task focus on a small subset of emoji\ntypes closely related to certain emotions, this setting oversimplifies the task\nand wastes the expressive power of emojis. In this paper, we extend the\nexisting setting of the emoji prediction task to include a richer set of emojis\nand to allow multi-label classification on the task. We propose novel models\nfor multi-class and multi-label emoji prediction based on Transformer networks.\nWe also construct multiple emoji prediction datasets from Twitter using\nheuristics. The BERT models achieve state-of-the-art performances on all our\ndatasets under all the settings, with relative improvements of 27.21% to\n236.36% in accuracy, 2.01% to 88.28% in top-5 accuracy and 65.19% to 346.79% in\nF-1 score, compared to the prior state-of-the-art. Our results demonstrate the\nefficacy of deep Transformer-based models on the emoji prediction task. We also\nrelease our datasets at\nhttps://github.com/hikari-NYU/Emoji_Prediction_Datasets_MMS for future\nresearchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07403v1",
            "date": "2020-07-14 23:47:08+00:00",
            "title": "Modeling Coherency in Generated Emails by Leveraging Deep Neural Learners",
            "authors": [
                "Avisha Das",
                "Rakesh M. Verma"
            ],
            "abstract": "Advanced machine learning and natural language techniques enable attackers to\nlaunch sophisticated and targeted social engineering-based attacks. To counter\nthe active attacker issue, researchers have since resorted to proactive methods\nof detection. Email masquerading using targeted emails to fool the victim is an\nadvanced attack method. However automatic text generation requires controlling\nthe context and coherency of the generated content, which has been identified\nas an increasingly difficult problem. The method used leverages a hierarchical\ndeep neural model which uses a learned representation of the sentences in the\ninput document to generate structured written emails. We demonstrate the\ngeneration of short and targeted text messages using the deep model. The global\ncoherency of the synthesized text is evaluated using a qualitative study as\nwell as multiple quantitative measures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07683v1",
            "date": "2020-07-15 13:46:50+00:00",
            "title": "UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data",
            "authors": [
                "Qianhui Wu",
                "Zijia Lin",
                "B\u00f6rje F. Karlsson",
                "Biqing Huang",
                "Jian-Guang Lou"
            ],
            "abstract": "Prior works in cross-lingual named entity recognition (NER) with no/little\nlabeled data fall into two primary categories: model transfer based and data\ntransfer based methods. In this paper we find that both method types can\ncomplement each other, in the sense that, the former can exploit context\ninformation via language-independent features but sees no task-specific\ninformation in the target language; while the latter generally generates pseudo\ntarget-language training data via translation but its exploitation of context\ninformation is weakened by inaccurate translations. Moreover, prior works\nrarely leverage unlabeled data in the target language, which can be\neffortlessly collected and potentially contains valuable information for\nimproved results. To handle both problems, we propose a novel approach termed\nUniTrans to Unify both model and data Transfer for cross-lingual NER, and\nfurthermore, to leverage the available information from unlabeled\ntarget-language data via enhanced knowledge distillation. We evaluate our\nproposed UniTrans over 4 target languages on benchmark datasets. Our\nexperimental results show that it substantially outperforms the existing\nstate-of-the-art methods.",
            "journal ref": "In IJCAI, pages 3926-3932, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07691v1",
            "date": "2020-07-15 14:00:18+00:00",
            "title": "A Multilingual Parallel Corpora Collection Effort for Indian Languages",
            "authors": [
                "Shashank Siripragada",
                "Jerin Philip",
                "Vinay P. Namboodiri",
                "C V Jawahar"
            ],
            "abstract": "We present sentence aligned parallel corpora across 10 Indian Languages -\nHindi, Telugu, Tamil, Malayalam, Gujarati, Urdu, Bengali, Oriya, Marathi,\nPunjabi, and English - many of which are categorized as low resource. The\ncorpora are compiled from online sources which have content shared across\nlanguages. The corpora presented significantly extends present resources that\nare either not large enough or are restricted to a specific domain (such as\nhealth). We also provide a separate test corpus compiled from an independent\nonline source that can be independently used for validating the performance in\n10 Indian languages. Alongside, we report on the methods of constructing such\ncorpora using tools enabled by recent advances in machine translation and\ncross-lingual retrieval using deep neural network based methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07728v2",
            "date": "2020-07-15 14:52:24+00:00",
            "title": "Dual Past and Future for Neural Machine Translation",
            "authors": [
                "Jianhao Yan",
                "Fandong Meng",
                "Jie Zhou"
            ],
            "abstract": "Though remarkable successes have been achieved by Neural Machine Translation\n(NMT) in recent years, it still suffers from the inadequate-translation\nproblem. Previous studies show that explicitly modeling the Past and Future\ncontents of the source sentence is beneficial for translation performance.\nHowever, it is not clear whether the commonly used heuristic objective is good\nenough to guide the Past and Future. In this paper, we present a novel dual\nframework that leverages both source-to-target and target-to-source NMT models\nto provide a more direct and accurate supervision signal for the Past and\nFuture modules. Experimental results demonstrate that our proposed method\nsignificantly improves the adequacy of NMT predictions and surpasses previous\nmethods in two well-studied translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07779v3",
            "date": "2020-07-15 15:56:05+00:00",
            "title": "AdapterHub: A Framework for Adapting Transformers",
            "authors": [
                "Jonas Pfeiffer",
                "Andreas R\u00fcckl\u00e9",
                "Clifton Poth",
                "Aishwarya Kamath",
                "Ivan Vuli\u0107",
                "Sebastian Ruder",
                "Kyunghyun Cho",
                "Iryna Gurevych"
            ],
            "abstract": "The current modus operandi in NLP involves downloading and fine-tuning\npre-trained models consisting of millions or billions of parameters. Storing\nand sharing such large trained models is expensive, slow, and time-consuming,\nwhich impedes progress towards more general and versatile NLP methods that\nlearn from and for many tasks. Adapters -- small learnt bottleneck layers\ninserted within each layer of a pre-trained model -- ameliorate this issue by\navoiding full fine-tuning of the entire model. However, sharing and integrating\nadapter layers is not straightforward. We propose AdapterHub, a framework that\nallows dynamic \"stitching-in\" of pre-trained adapters for different tasks and\nlanguages. The framework, built on top of the popular HuggingFace Transformers\nlibrary, enables extremely easy and quick adaptations of state-of-the-art\npre-trained models (e.g., BERT, RoBERTa, XLM-R) across tasks and languages.\nDownloading, sharing, and training adapters is as seamless as possible using\nminimal changes to the training scripts and a specialized infrastructure. Our\nframework enables scalable and easy access to sharing of task-specific models,\nparticularly in low-resource scenarios. AdapterHub includes all recent adapter\narchitectures and can be found at https://AdapterHub.ml.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07803v2",
            "date": "2020-07-15 17:09:17+00:00",
            "title": "Fine-Tune Longformer for Jointly Predicting Rumor Stance and Veracity",
            "authors": [
                "Anant Khandelwal"
            ],
            "abstract": "Increased usage of social media caused the popularity of news and events\nwhich are not even verified, resulting in spread of rumors allover the web. Due\nto widely available social media platforms and increased usage caused the data\nto be available in huge amounts.The manual methods to process such large data\nis costly and time-taking, so there has been an increased attention to process\nand verify such content automatically for the presence of rumors. A lot of\nresearch studies reveal that to identify the stances of posts in the discussion\nthread of such events and news is an important preceding step before identify\nthe rumor veracity. In this paper,we propose a multi-task learning framework\nfor jointly predicting rumor stance and veracity on the dataset released at\nSemEval 2019 RumorEval: Determining rumor veracity and support for\nrumors(SemEval 2019 Task 7), which includes social media rumors stem from a\nvariety of breaking news stories from Reddit as well as Twit-ter. Our framework\nconsists of two parts: a) The bottom part of our framework classifies the\nstance for each post in the conversation thread discussing a rumor via\nmodelling the multi-turn conversation and make each post aware of its\nneighboring posts. b) The upper part predicts the rumor veracity of the\nconversation thread with stance evolution obtained from the bottom part.\nExperimental results on SemEval 2019 Task 7 dataset show that our method\noutperforms previous methods on both rumor stance classification and veracity\nprediction",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07834v2",
            "date": "2020-07-15 16:58:01+00:00",
            "title": "InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training",
            "authors": [
                "Zewen Chi",
                "Li Dong",
                "Furu Wei",
                "Nan Yang",
                "Saksham Singhal",
                "Wenhui Wang",
                "Xia Song",
                "Xian-Ling Mao",
                "Heyan Huang",
                "Ming Zhou"
            ],
            "abstract": "In this work, we present an information-theoretic framework that formulates\ncross-lingual language model pre-training as maximizing mutual information\nbetween multilingual-multi-granularity texts. The unified view helps us to\nbetter understand the existing methods for learning cross-lingual\nrepresentations. More importantly, inspired by the framework, we propose a new\npre-training task based on contrastive learning. Specifically, we regard a\nbilingual sentence pair as two views of the same meaning and encourage their\nencoded representations to be more similar than the negative examples. By\nleveraging both monolingual and parallel corpora, we jointly train the pretext\ntasks to improve the cross-lingual transferability of pre-trained models.\nExperimental results on several benchmarks show that our approach achieves\nconsiderably better performance. The code and pre-trained models are available\nat https://aka.ms/infoxlm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07841v1",
            "date": "2020-07-15 17:03:34+00:00",
            "title": "Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation",
            "authors": [
                "Paul Tardy",
                "David Janiszek",
                "Yannick Est\u00e8ve",
                "Vincent Nguyen"
            ],
            "abstract": "Summarizing texts is not a straightforward task. Before even considering text\nsummarization, one should determine what kind of summary is expected. How much\nshould the information be compressed? Is it relevant to reformulate or should\nthe summary stick to the original phrasing? State-of-the-art on automatic text\nsummarization mostly revolves around news articles. We suggest that considering\na wider variety of tasks would lead to an improvement in the field, in terms of\ngeneralization and robustness. We explore meeting summarization: generating\nreports from automatic transcriptions. Our work consists in segmenting and\naligning transcriptions with respect to reports, to get a suitable dataset for\nneural summarization. Using a bootstrapping approach, we provide pre-alignments\nthat are corrected by human annotators, making a validation set against which\nwe evaluate automatic models. This consistently reduces annotators' efforts by\nproviding iteratively better pre-alignment and maximizes the corpus size by\nusing annotations from our automatic alignment models. Evaluation is conducted\non \\publicmeetings, a novel corpus of aligned public meetings. We report\nautomatic alignment and summarization performances on this corpus and show that\nautomatic alignment is relevant for data annotation since it leads to large\nimprovement of almost +4 on all ROUGE scores on the summarization task.",
            "journal ref": "LREC 2020 -- Proceedings of The 12th Language Resources and\n  Evaluation Conference, 2020, pp. 6718--6724",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08124v1",
            "date": "2020-07-16 05:52:16+00:00",
            "title": "LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning",
            "authors": [
                "Jian Liu",
                "Leyang Cui",
                "Hanmeng Liu",
                "Dandan Huang",
                "Yile Wang",
                "Yue Zhang"
            ],
            "abstract": "Machine reading is a fundamental task for testing the capability of natural\nlanguage understanding, which is closely related to human cognition in many\naspects. With the rising of deep learning techniques, algorithmic models rival\nhuman performances on simple QA, and thus increasingly challenging machine\nreading datasets have been proposed. Though various challenges such as evidence\nintegration and commonsense knowledge have been integrated, one of the\nfundamental capabilities in human reading, namely logical reasoning, is not\nfully investigated. We build a comprehensive dataset, named LogiQA, which is\nsourced from expert-written questions for testing human Logical reasoning. It\nconsists of 8,678 QA instances, covering multiple types of deductive reasoning.\nResults show that state-of-the-art neural models perform by far worse than\nhuman ceiling. Our dataset can also serve as a benchmark for reinvestigating\nlogical AI under the deep learning NLP setting. The dataset is freely available\nat https://github.com/lgw863/LogiQA-dataset",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08416v1",
            "date": "2020-07-16 15:53:02+00:00",
            "title": "SLK-NER: Exploiting Second-order Lexicon Knowledge for Chinese NER",
            "authors": [
                "Dou Hu",
                "Lingwei Wei"
            ],
            "abstract": "Although character-based models using lexicon have achieved promising results\nfor Chinese named entity recognition (NER) task, some lexical words would\nintroduce erroneous information due to wrongly matched words. Existing\nresearches proposed many strategies to integrate lexicon knowledge. However,\nthey performed with simple first-order lexicon knowledge, which provided\ninsufficient word information and still faced the challenge of matched word\nboundary conflicts; or explored the lexicon knowledge with graph where\nhigher-order information introducing negative words may disturb the\nidentification. To alleviate the above limitations, we present new insight into\nsecond-order lexicon knowledge (SLK) of each character in the sentence to\nprovide more lexical word information including semantic and word boundary\nfeatures. Based on these, we propose a SLK-based model with a novel strategy to\nintegrate the above lexicon knowledge. The proposed model can exploit more\ndiscernible lexical words information with the help of global context.\nExperimental results on three public datasets demonstrate the validity of SLK.\nThe proposed model achieves more excellent performance than the\nstate-of-the-art comparison methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08426v2",
            "date": "2020-07-16 16:05:34+00:00",
            "title": "Investigating Pretrained Language Models for Graph-to-Text Generation",
            "authors": [
                "Leonardo F. R. Ribeiro",
                "Martin Schmitt",
                "Hinrich Sch\u00fctze",
                "Iryna Gurevych"
            ],
            "abstract": "Graph-to-text generation aims to generate fluent texts from graph-based data.\nIn this paper, we investigate two recently proposed pretrained language models\n(PLMs) and analyze the impact of different task-adaptive pretraining strategies\nfor PLMs in graph-to-text generation. We present a study across three graph\ndomains: meaning representations, Wikipedia knowledge graphs (KGs) and\nscientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art\nresults and that task-adaptive pretraining strategies improve their performance\neven further. In particular, we report new state-of-the-art BLEU scores of\n49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative\nimprovement of 31.8%, 4.5%, and 42.4%, respectively. In an extensive analysis,\nwe identify possible reasons for the PLMs' success on graph-to-text tasks. We\nfind evidence that their knowledge about true facts helps them perform well\neven when the input graph representation is reduced to a simple bag of node and\nedge labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08445v2",
            "date": "2020-07-16 16:27:38+00:00",
            "title": "Hierarchical Interaction Networks with Rethinking Mechanism for Document-level Sentiment Analysis",
            "authors": [
                "Lingwei Wei",
                "Dou Hu",
                "Wei Zhou",
                "Xuehai Tang",
                "Xiaodan Zhang",
                "Xin Wang",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "Document-level Sentiment Analysis (DSA) is more challenging due to vague\nsemantic links and complicate sentiment information. Recent works have been\ndevoted to leveraging text summarization and have achieved promising results.\nHowever, these summarization-based methods did not take full advantage of the\nsummary including ignoring the inherent interactions between the summary and\ndocument. As a result, they limited the representation to express major points\nin the document, which is highly indicative of the key sentiment. In this\npaper, we study how to effectively generate a discriminative representation\nwith explicit subject patterns and sentiment contexts for DSA. A Hierarchical\nInteraction Networks (HIN) is proposed to explore bidirectional interactions\nbetween the summary and document at multiple granularities and learn\nsubject-oriented document representations for sentiment classification.\nFurthermore, we design a Sentiment-based Rethinking mechanism (SR) by refining\nthe HIN with sentiment label information to learn a more sentiment-aware\ndocument representation. We extensively evaluate our proposed models on three\npublic datasets. The experimental results consistently demonstrate the\neffectiveness of our proposed models and show that HIN-SR outperforms various\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08742v1",
            "date": "2020-07-17 04:06:09+00:00",
            "title": "A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation",
            "authors": [
                "Yongjing Yin",
                "Fandong Meng",
                "Jinsong Su",
                "Chulun Zhou",
                "Zhengyuan Yang",
                "Jie Zhou",
                "Jiebo Luo"
            ],
            "abstract": "Multi-modal neural machine translation (NMT) aims to translate source\nsentences into a target language paired with images. However, dominant\nmulti-modal NMT models do not fully exploit fine-grained semantic\ncorrespondences between semantic units of different modalities, which have\npotential to refine multi-modal representation learning. To deal with this\nissue, in this paper, we propose a novel graph-based multi-modal fusion encoder\nfor NMT. Specifically, we first represent the input sentence and image using a\nunified multi-modal graph, which captures various semantic relationships\nbetween multi-modal semantic units (words and visual objects). We then stack\nmultiple graph-based multi-modal fusion layers that iteratively perform\nsemantic interactions to learn node representations. Finally, these\nrepresentations provide an attention-based context vector for the decoder. We\nevaluate our proposed encoder on the Multi30K datasets. Experimental results\nand in-depth analysis show the superiority of our multi-modal NMT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09076v1",
            "date": "2020-07-17 15:56:54+00:00",
            "title": "Constructing a Family Tree of Ten Indo-European Languages with Delexicalized Cross-linguistic Transfer Patterns",
            "authors": [
                "Yuanyuan Zhao",
                "Weiwei Sun",
                "Xiaojun Wan"
            ],
            "abstract": "It is reasonable to hypothesize that the divergence patterns formulated by\nhistorical linguists and typologists reflect constraints on human languages,\nand are thus consistent with Second Language Acquisition (SLA) in a certain\nway. In this paper, we validate this hypothesis on ten Indo-European languages.\nWe formalize the delexicalized transfer as interpretable tree-to-string and\ntree-to-tree patterns which can be automatically induced from web data by\napplying neural syntactic parsing and grammar induction technologies. This\nallows us to quantitatively probe cross-linguistic transfer and extend\ninquiries of SLA. We extend existing works which utilize mixed features and\nsupport the agreement between delexicalized cross-linguistic transfer and the\nphylogenetic structure resulting from the historical-comparative paradigm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09495v1",
            "date": "2020-07-18 18:40:40+00:00",
            "title": "A novel approach to sentiment analysis in Persian using discourse and external semantic information",
            "authors": [
                "Rahim Dehkharghani",
                "Hojjat Emami"
            ],
            "abstract": "Sentiment analysis attempts to identify, extract and quantify affective\nstates and subjective information from various types of data such as text,\naudio, and video. Many approaches have been proposed to extract the sentiment\nof individuals from documents written in natural languages in recent years. The\nmajority of these approaches have focused on English, while resource-lean\nlanguages such as Persian suffer from the lack of research work and language\nresources. Due to this gap in Persian, the current work is accomplished to\nintroduce new methods for sentiment analysis which have been applied on\nPersian. The proposed approach in this paper is two-fold: The first one is\nbased on classifier combination, and the second one is based on deep neural\nnetworks which benefits from word embedding vectors. Both approaches takes\nadvantage of local discourse information and external knowledge bases, and also\ncover several language issues such as negation and intensification,\nandaddresses different granularity levels, namely word, aspect, sentence,\nphrase and document-levels. To evaluate the performance of the proposed\napproach, a Persian dataset is collected from Persian hotel reviews referred as\nhotel reviews. The proposed approach has been compared to counterpart methods\nbased on the benchmark dataset. The experimental results approve the\neffectiveness of the proposed approach when compared to related works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09557v1",
            "date": "2020-07-19 02:11:53+00:00",
            "title": "From Spatial Relations to Spatial Configurations",
            "authors": [
                "Soham Dan",
                "Parisa Kordjamshidi",
                "Julia Bonn",
                "Archna Bhatia",
                "Jon Cai",
                "Martha Palmer",
                "Dan Roth"
            ],
            "abstract": "Spatial Reasoning from language is essential for natural language\nunderstanding. Supporting it requires a representation scheme that can capture\nspatial phenomena encountered in language as well as in images and videos.\nExisting spatial representations are not sufficient for describing spatial\nconfigurations used in complex tasks. This paper extends the capabilities of\nexisting spatial representation languages and increases coverage of the\nsemantic aspects that are needed to ground the spatial meaning of natural\nlanguage text in the world. Our spatial relation language is able to represent\na large, comprehensive set of spatial concepts crucial for reasoning and is\ndesigned to support the composition of static and dynamic spatial\nconfigurations. We integrate this language with the Abstract Meaning\nRepresentation(AMR) annotation schema and present a corpus annotated by this\nextended AMR. To exhibit the applicability of our representation scheme, we\nannotate text taken from diverse datasets and show how we extend the\ncapabilities of existing spatial representation languages with the fine-grained\ndecomposition of semantics and blend it seamlessly with AMRs of sentences and\ndiscourse representations as a whole.",
            "journal ref": "LREC 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09774v1",
            "date": "2020-07-19 20:15:55+00:00",
            "title": "An Overview of Natural Language State Representation for Reinforcement Learning",
            "authors": [
                "Brielen Madureira",
                "David Schlangen"
            ],
            "abstract": "A suitable state representation is a fundamental part of the learning process\nin Reinforcement Learning. In various tasks, the state can either be described\nby natural language or be natural language itself. This survey outlines the\nstrategies used in the literature to build natural language state\nrepresentations. We appeal for more linguistically interpretable and grounded\nrepresentations, careful justification of design decisions and evaluation of\nthe effectiveness of different approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09903v1",
            "date": "2020-07-20 06:23:18+00:00",
            "title": "Multimodal Dialogue State Tracking By QA Approach with Data Augmentation",
            "authors": [
                "Xiangyang Mou",
                "Brandyn Sigouin",
                "Ian Steenstra",
                "Hui Su"
            ],
            "abstract": "Recently, a more challenging state tracking task, Audio-Video Scene-Aware\nDialogue (AVSD), is catching an increasing amount of attention among\nresearchers. Different from purely text-based dialogue state tracking, the\ndialogue in AVSD contains a sequence of question-answer pairs about a video and\nthe final answer to the given question requires additional understanding of the\nvideo. This paper interprets the AVSD task from an open-domain Question\nAnswering (QA) point of view and proposes a multimodal open-domain QA system to\ndeal with the problem. The proposed QA system uses common encoder-decoder\nframework with multimodal fusion and attention. Teacher forcing is applied to\ntrain a natural language generator. We also propose a new data augmentation\napproach specifically under QA assumption. Our experiments show that our model\nand techniques bring significant improvements over the baseline model on the\nDSTC7-AVSD dataset and demonstrate the potentials of our data augmentation\ntechniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10286v4",
            "date": "2020-07-13 15:57:26+00:00",
            "title": "COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model",
            "authors": [
                "Jingqi Wang",
                "Noor Abu-el-rub",
                "Josh Gray",
                "Huy Anh Pham",
                "Yujia Zhou",
                "Frank Manion",
                "Mei Liu",
                "Xing Song",
                "Hua Xu",
                "Masoud Rouhizadeh",
                "Yaoyun Zhang"
            ],
            "abstract": "The COVID-19 pandemic swept across the world rapidly, infecting millions of\npeople. An efficient tool that can accurately recognize important clinical\nconcepts of COVID-19 from free text in electronic health records (EHRs) will be\nvaluable to accelerate COVID-19 clinical research. To this end, this study aims\nat adapting the existing CLAMP natural language processing tool to quickly\nbuild COVID-19 SignSym, which can extract COVID-19 signs/symptoms and their 8\nattributes (body location, severity, temporal expression, subject, condition,\nuncertainty, negation, and course) from clinical text. The extracted\ninformation is also mapped to standard concepts in the Observational Medical\nOutcomes Partnership common data model. A hybrid approach of combining deep\nlearning-based models, curated lexicons, and pattern-based rules was applied to\nquickly build the COVID-19 SignSym from CLAMP, with optimized performance. Our\nextensive evaluation using 3 external sites with clinical notes of COVID-19\npatients, as well as the online medical dialogues of COVID-19, shows COVID-19\nSign-Sym can achieve high performance across data sources. The workflow used\nfor this study can be generalized to other use cases, where existing clinical\nnatural language processing tools need to be customized for specific\ninformation needs within a short time. COVID-19 SignSym is freely accessible to\nthe research community as a downloadable package\n(https://clamp.uth.edu/covid/nlp.php) and has been used by 16 healthcare\norganizations to support clinical research of COVID-19.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10681v1",
            "date": "2020-07-21 09:41:07+00:00",
            "title": "Neural Machine Translation with Error Correction",
            "authors": [
                "Kaitao Song",
                "Xu Tan",
                "Jianfeng Lu"
            ],
            "abstract": "Neural machine translation (NMT) generates the next target token given as\ninput the previous ground truth target tokens during training while the\nprevious generated target tokens during inference, which causes discrepancy\nbetween training and inference as well as error propagation, and affects the\ntranslation accuracy. In this paper, we introduce an error correction mechanism\ninto NMT, which corrects the error information in the previous generated tokens\nto better predict the next token. Specifically, we introduce two-stream\nself-attention from XLNet into NMT decoder, where the query stream is used to\npredict the next token, and meanwhile the content stream is used to correct the\nerror information from the previous predicted tokens. We leverage scheduled\nsampling to simulate the prediction errors during training. Experiments on\nthree IWSLT translation datasets and two WMT translation datasets demonstrate\nthat our method achieves improvements over Transformer baseline and scheduled\nsampling. Further experimental analyses also verify the effectiveness of our\nproposed error correction mechanism to improve the translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10945v1",
            "date": "2020-07-21 17:03:00+00:00",
            "title": "XD at SemEval-2020 Task 12: Ensemble Approach to Offensive Language Identification in Social Media Using Transformer Encoders",
            "authors": [
                "Xiangjue Dong",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents six document classification models using the latest\ntransformer encoders and a high-performing ensemble model for a task of\noffensive language identification in social media. For the individual models,\ndeep transformer layers are applied to perform multi-head attentions. For the\nensemble model, the utterance representations taken from those individual\nmodels are concatenated and fed into a linear decoder to make the final\ndecisions. Our ensemble model outperforms the individual models and shows up to\n8.6% improvement over the individual models on the development set. On the test\nset, it achieves macro-F1 of 90.9% and becomes one of the high performing\nsystems among 85 participants in the sub-task A of this shared task. Our\nanalysis shows that although the ensemble model significantly improves the\naccuracy on the development set, the improvement is not as evident on the test\nset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11314v1",
            "date": "2020-07-22 10:09:26+00:00",
            "title": "Better Early than Late: Fusing Topics with Word Embeddings for Neural Question Paraphrase Identification",
            "authors": [
                "Nicole Peinelt",
                "Dong Nguyen",
                "Maria Liakata"
            ],
            "abstract": "Question paraphrase identification is a key task in Community Question\nAnswering (CQA) to determine if an incoming question has been previously asked.\nMany current models use word embeddings to identify duplicate questions, but\nthe use of topic models in feature-engineered systems suggests that they can be\nhelpful for this task, too. We therefore propose two ways of merging topics\nwith word embeddings (early vs. late fusion) in a new neural architecture for\nquestion paraphrase identification. Our results show that our system\noutperforms neural baselines on multiple CQA datasets, while an ablation study\nhighlights the importance of topics and especially early topic-embedding fusion\nin our architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11348v1",
            "date": "2020-07-22 11:22:57+00:00",
            "title": "Massive Multi-Document Summarization of Product Reviews with Weak Supervision",
            "authors": [
                "Ori Shapira",
                "Ran Levy"
            ],
            "abstract": "Product reviews summarization is a type of Multi-Document Summarization (MDS)\ntask in which the summarized document sets are often far larger than in\ntraditional MDS (up to tens of thousands of reviews). We highlight this\ndifference and coin the term \"Massive Multi-Document Summarization\" (MMDS) to\ndenote an MDS task that involves hundreds of documents or more. Prior work on\nproduct reviews summarization considered small samples of the reviews, mainly\ndue to the difficulty of handling massive document sets. We show that\nsummarizing small samples can result in loss of important information and\nprovide misleading evaluation results. We propose a schema for summarizing a\nmassive set of reviews on top of a standard summarization algorithm. Since\nwriting large volumes of reference summaries needed for advanced neural network\nmodels is impractical, our solution relies on weak supervision. Finally, we\npropose an evaluation scheme that is based on multiple crowdsourced reference\nsummaries and aims to capture the massive review collection. We show that an\ninitial implementation of our schema significantly improves over several\nbaselines in ROUGE scores, and exhibits strong coherence in a manual linguistic\nquality assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11381v1",
            "date": "2020-07-22 12:47:11+00:00",
            "title": "To Be or Not To Be a Verbal Multiword Expression: A Quest for Discriminating Features",
            "authors": [
                "Caroline Pasquer",
                "Agata Savary",
                "Jean-Yves Antoine",
                "Carlos Ramisch",
                "Nicolas Labroche",
                "Arnaud Giacometti"
            ],
            "abstract": "Automatic identification of mutiword expressions (MWEs) is a pre-requisite\nfor semantically-oriented downstream applications. This task is challenging\nbecause MWEs, especially verbal ones (VMWEs), exhibit surface variability.\nHowever, this variability is usually more restricted than in regular (non-VMWE)\nconstructions, which leads to various variability profiles. We use this fact to\ndetermine the optimal set of features which could be used in a supervised\nclassification setting to solve a subproblem of VMWE identification: the\nidentification of occurrences of previously seen VMWEs. Surprisingly, a simple\ncustom frequency-based feature selection method proves more efficient than\nother standard methods such as Chi-squared test, information gain or decision\ntrees. An SVM classifier using the optimal set of only 6 features outperforms\nthe best systems from a recent shared task on the French seen data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11464v2",
            "date": "2020-07-22 14:37:42+00:00",
            "title": "SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection",
            "authors": [
                "Dominik Schlechtweg",
                "Barbara McGillivray",
                "Simon Hengchen",
                "Haim Dubossarsky",
                "Nina Tahmasebi"
            ],
            "abstract": "Lexical Semantic Change detection, i.e., the task of identifying words that\nchange meaning over time, is a very active research area, with applications in\nNLP, lexicography, and linguistics. Evaluation is currently the most pressing\nproblem in Lexical Semantic Change detection, as no gold standards are\navailable to the community, which hinders progress. We present the results of\nthe first shared task that addresses this gap by providing researchers with an\nevaluation framework and manually annotated, high-quality datasets for English,\nGerman, Latin, and Swedish. 33 teams submitted 186 systems, which were\nevaluated on two subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11648v1",
            "date": "2020-07-22 19:52:34+00:00",
            "title": "Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models",
            "authors": [
                "Mittul Singh",
                "Peter Smit",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "Character-based Neural Network Language Models (NNLM) have the advantage of\nsmaller vocabulary and thus faster training times in comparison to NNLMs based\non multi-character units. However, in low-resource scenarios, both the\ncharacter and multi-character NNLMs suffer from data sparsity. In such\nscenarios, cross-lingual transfer has improved multi-character NNLM performance\nby allowing information transfer from a source to the target language. In the\nsame vein, we propose to use cross-lingual transfer for character NNLMs applied\nto low-resource Automatic Speech Recognition (ASR). However, applying\ncross-lingual transfer to character NNLMs is not as straightforward. We observe\nthat relatedness of the source language plays an important role in\ncross-lingual pretraining of character NNLMs. We evaluate this aspect on ASR\ntasks for two target languages: Finnish (with English and Estonian as source)\nand Swedish (with Danish, Norwegian, and English as source). Prior work has\nobserved no difference between using the related or unrelated language for\nmulti-character NNLMs. We, however, show that for character-based NNLMs, only\npretraining with a related language improves the ASR performance, and using an\nunrelated language may deteriorate it. We also observe that the benefits are\nlarger when there is much lesser target data than source data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11865v1",
            "date": "2020-07-23 08:48:06+00:00",
            "title": "AI4D -- African Language Dataset Challenge",
            "authors": [
                "Kathleen Siminyu",
                "Sackey Freshia",
                "Jade Abbott",
                "Vukosi Marivate"
            ],
            "abstract": "As language and speech technologies become more advanced, the lack of\nfundamental digital resources for African languages, such as data, spell\ncheckers and Part of Speech taggers, means that the digital divide between\nthese languages and others keeps growing. This work details the organisation of\nthe AI4D - African Language Dataset Challenge, an effort to incentivize the\ncreation, organization and discovery of African language datasets through a\ncompetitive challenge. We particularly encouraged the submission of annotated\ndatasets which can be used for training task-specific supervised machine\nlearning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11949v1",
            "date": "2020-07-23 12:02:40+00:00",
            "title": "Deep Learning based, end-to-end metaphor detection in Greek language with Recurrent and Convolutional Neural Networks",
            "authors": [
                "Konstantinos Perifanos",
                "Eirini Florou",
                "Dionysis Goutsos"
            ],
            "abstract": "This paper presents and benchmarks a number of end-to-end Deep Learning based\nmodels for metaphor detection in Greek. We combine Convolutional Neural\nNetworks and Recurrent Neural Networks with representation learning to bear on\nthe metaphor detection problem for the Greek language. The models presented\nachieve exceptional accuracy scores, significantly improving the previous state\nof the art results, which had already achieved accuracy 0.82. Furthermore, no\nspecial preprocessing, feature engineering or linguistic knowledge is used in\nthis work. The methods presented achieve accuracy of 0.92 and F-score 0.92 with\nConvolutional Neural Networks (CNNs) and bidirectional Long Short Term Memory\nnetworks (LSTMs). Comparable results of 0.91 accuracy and 0.91 F-score are also\nachieved with bidirectional Gated Recurrent Units (GRUs) and Convolutional\nRecurrent Neural Nets (CRNNs). The models are trained and evaluated only on the\nbasis of the training tuples, the sentences and their labels. The outcome is a\nstate of the art collection of metaphor detection models, trained on limited\nlabelled resources, which can be extended to other languages and similar tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12081v2",
            "date": "2020-07-23 15:45:12+00:00",
            "title": "NITS-Hinglish-SentiMix at SemEval-2020 Task 9: Sentiment Analysis For Code-Mixed Social Media Text Using an Ensemble Model",
            "authors": [
                "Subhra Jyoti Baroi",
                "Nivedita Singh",
                "Ringki Das",
                "Thoudam Doren Singh"
            ],
            "abstract": "Sentiment Analysis is the process of deciphering what a sentence emotes and\nclassifying them as either positive, negative, or neutral. In recent times,\nIndia has seen a huge influx in the number of active social media users and\nthis has led to a plethora of unstructured text data. Since the Indian\npopulation is generally fluent in both Hindi and English, they end up\ngenerating code-mixed Hinglish social media text i.e. the expressions of Hindi\nlanguage, written in the Roman script alongside other English words. The\nability to adequately comprehend the notions in these texts is truly necessary.\nOur team, rns2020 participated in Task 9 at SemEval2020 intending to design a\nsystem to carry out the sentiment analysis of code-mixed social media text.\nThis work proposes a system named NITS-Hinglish-SentiMix to viably complete the\nsentiment analysis of such code-mixed Hinglish text. The proposed framework has\nrecorded an F-Score of 0.617 on the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12390v1",
            "date": "2020-07-24 07:28:47+00:00",
            "title": "IDS at SemEval-2020 Task 10: Does Pre-trained Language Model Know What to Emphasize?",
            "authors": [
                "Jaeyoul Shin",
                "Taeuk Kim",
                "Sang-goo Lee"
            ],
            "abstract": "We propose a novel method that enables us to determine words that deserve to\nbe emphasized from written text in visual media, relying only on the\ninformation from the self-attention distributions of pre-trained language\nmodels (PLMs). With extensive experiments and analyses, we show that 1) our\nzero-shot approach is superior to a reasonable baseline that adopts TF-IDF and\nthat 2) there exist several attention heads in PLMs specialized for emphasis\nselection, confirming that PLMs are capable of recognizing important words in\nsentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12432v1",
            "date": "2020-07-24 09:50:26+00:00",
            "title": "MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning",
            "authors": [
                "Aina Gar\u00ed Soler",
                "Marianna Apidianaki"
            ],
            "abstract": "We present the MULTISEM systems submitted to SemEval 2020 Task 3: Graded Word\nSimilarity in Context (GWSC). We experiment with injecting semantic knowledge\ninto pre-trained BERT models through fine-tuning on lexical semantic tasks\nrelated to GWSC. We use existing semantically annotated datasets and propose to\napproximate similarity through automatically generated lexical substitutes in\ncontext. We participate in both GWSC subtasks and address two languages,\nEnglish and Finnish. Our best English models occupy the third and fourth\npositions in the ranking for the two subtasks. Performance is lower for the\nFinnish models which are mid-ranked in the respective subtasks, highlighting\nthe important role of data availability for fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12544v3",
            "date": "2020-07-24 14:48:27+00:00",
            "title": "FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings",
            "authors": [
                "Bertelt Braaksma",
                "Richard Scholtens",
                "Stan van Suijlekom",
                "Remy Wang",
                "Ahmet \u00dcst\u00fcn"
            ],
            "abstract": "In this paper, we present our approach for sentiment classification on\nSpanish-English code-mixed social media data in the SemEval-2020 Task 9. We\ninvestigate performance of various pre-trained Transformer models by using\ndifferent fine-tuning strategies. We explore both monolingual and multilingual\nmodels with the standard fine-tuning method. Additionally, we propose a custom\nmodel that we fine-tune in two steps: once with a language modeling objective,\nand once with a task-specific objective. Although two-step fine-tuning improves\nsentiment classification performance over the base model, the large\nmultilingual XLM-RoBERTa model achieves best weighted F1-score with 0.537 on\ndevelopment data and 0.739 on test data. With this score, our team jupitter\nplaced tenth overall in the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12561v2",
            "date": "2020-07-24 15:06:48+00:00",
            "title": "JUNLP@SemEval-2020 Task 9:Sentiment Analysis of Hindi-English code mixed data using Grid Search Cross Validation",
            "authors": [
                "Avishek Garain",
                "Sainik Kumar Mahata",
                "Dipankar Das"
            ],
            "abstract": "Code-mixing is a phenomenon which arises mainly in multilingual societies.\nMultilingual people, who are well versed in their native languages and also\nEnglish speakers, tend to code-mix using English-based phonetic typing and the\ninsertion of anglicisms in their main language. This linguistic phenomenon\nposes a great challenge to conventional NLP domains such as Sentiment Analysis,\nMachine Translation, and Text Summarization, to name a few. In this work, we\nfocus on working out a plausible solution to the domain of Code-Mixed Sentiment\nAnalysis. This work was done as participation in the SemEval-2020 Sentimix\nTask, where we focused on the sentiment analysis of English-Hindi code-mixed\nsentences. our username for the submission was \"sainik.mahata\" and team name\nwas \"JUNLP\". We used feature extraction algorithms in conjunction with\ntraditional machine learning algorithms such as SVR and Grid Search in an\nattempt to solve the task. Our approach garnered an f1-score of 66.2\\% when\ntested using metrics prepared by the organizers of the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12569v2",
            "date": "2020-07-24 15:23:45+00:00",
            "title": "Named entity recognition in chemical patents using ensemble of contextual language models",
            "authors": [
                "Jenny Copara",
                "Nona Naderi",
                "Julien Knafou",
                "Patrick Ruch",
                "Douglas Teodoro"
            ],
            "abstract": "Chemical patent documents describe a broad range of applications holding key\nreaction and compound information, such as chemical structure, reaction\nformulas, and molecular properties. These informational entities should be\nfirst identified in text passages to be utilized in downstream tasks. Text\nmining provides means to extract relevant information from chemical patents\nthrough information extraction techniques. As part of the Information\nExtraction task of the Cheminformatics Elsevier Melbourne University challenge,\nin this work we study the effectiveness of contextualized language models to\nextract reaction information in chemical patents. We assess transformer\narchitectures trained on a generic and specialised corpora to propose a new\nensemble model. Our best model, based on a majority ensemble approach, achieves\nan exact F1-score of 92.30% and a relaxed F1-score of 96.24%. The results show\nthat ensemble of contextualized language models can provide an effective method\nto extract information from chemical patents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12626v4",
            "date": "2020-07-24 16:25:19+00:00",
            "title": "SummEval: Re-evaluating Summarization Evaluation",
            "authors": [
                "Alexander R. Fabbri",
                "Wojciech Kry\u015bci\u0144ski",
                "Bryan McCann",
                "Caiming Xiong",
                "Richard Socher",
                "Dragomir Radev"
            ],
            "abstract": "The scarcity of comprehensive up-to-date studies on evaluation metrics for\ntext summarization and the lack of consensus regarding evaluation protocols\ncontinue to inhibit progress. We address the existing shortcomings of\nsummarization evaluation methods along five dimensions: 1) we re-evaluate 14\nautomatic evaluation metrics in a comprehensive and consistent fashion using\nneural summarization model outputs along with expert and crowd-sourced human\nannotations, 2) we consistently benchmark 23 recent summarization models using\nthe aforementioned automatic evaluation metrics, 3) we assemble the largest\ncollection of summaries generated by models trained on the CNN/DailyMail news\ndataset and share it in a unified format, 4) we implement and share a toolkit\nthat provides an extensible and unified API for evaluating summarization models\nacross a broad range of automatic metrics, 5) we assemble and share the largest\nand most diverse, in terms of model types, collection of human judgments of\nmodel-generated summaries on the CNN/Daily Mail dataset annotated by both\nexpert judges and crowd-source workers. We hope that this work will help\npromote a more complete evaluation protocol for text summarization as well as\nadvance research in developing evaluation metrics that better correlate with\nhuman judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12733v1",
            "date": "2020-07-24 18:48:37+00:00",
            "title": "IUST at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text using Deep Neural Networks and Linear Baselines",
            "authors": [
                "Soroush Javdan",
                "Taha Shangipour ataei",
                "Behrouz Minaei-Bidgoli"
            ],
            "abstract": "Sentiment Analysis is a well-studied field of Natural Language Processing.\nHowever, the rapid growth of social media and noisy content within them poses\nsignificant challenges in addressing this problem with well-established methods\nand tools. One of these challenges is code-mixing, which means using different\nlanguages to convey thoughts in social media texts. Our group, with the name of\nIUST(username: TAHA), participated at the SemEval-2020 shared task 9 on\nSentiment Analysis for Code-Mixed Social Media Text, and we have attempted to\ndevelop a system to predict the sentiment of a given code-mixed tweet. We used\ndifferent preprocessing techniques and proposed to use different methods that\nvary from NBSVM to more complicated deep neural network models. Our best\nperforming method obtains an F1 score of 0.751 for the Spanish-English sub-task\nand 0.706 over the Hindi-English sub-task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12741v2",
            "date": "2020-07-24 19:17:26+00:00",
            "title": "Consistent Transcription and Translation of Speech",
            "authors": [
                "Matthias Sperber",
                "Hendra Setiawan",
                "Christian Gollan",
                "Udhyakumar Nallasamy",
                "Matthias Paulik"
            ],
            "abstract": "The conventional paradigm in speech translation starts with a speech\nrecognition step to generate transcripts, followed by a translation step with\nthe automatic transcripts as input. To address various shortcomings of this\nparadigm, recent work explores end-to-end trainable direct models that\ntranslate without transcribing. However, transcripts can be an indispensable\noutput in practical applications, which often display transcripts alongside the\ntranslations to users.\n  We make this common requirement explicit and explore the task of jointly\ntranscribing and translating speech. While high accuracy of transcript and\ntranslation are crucial, even highly accurate systems can suffer from\ninconsistencies between both outputs that degrade the user experience. We\nintroduce a methodology to evaluate consistency and compare several modeling\napproaches, including the traditional cascaded approach and end-to-end models.\nWe find that direct models are poorly suited to the joint\ntranscription/translation task, but that end-to-end models that feature a\ncoupled inference procedure are able to achieve strong consistency. We further\nintroduce simple techniques for directly optimizing for consistency, and\nanalyze the resulting trade-offs between consistency, transcription accuracy,\nand translation accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12913v1",
            "date": "2020-07-25 11:35:57+00:00",
            "title": "NoPropaganda at SemEval-2020 Task 11: A Borrowed Approach to Sequence Tagging and Text Classification",
            "authors": [
                "Ilya Dimov",
                "Vladislav Korzun",
                "Ivan Smurov"
            ],
            "abstract": "This paper describes our contribution to SemEval-2020 Task 11: Detection Of\nPropaganda Techniques In News Articles. We start with simple LSTM baselines and\nmove to an autoregressive transformer decoder to predict long continuous\npropaganda spans for the first subtask. We also adopt an approach from relation\nextraction by enveloping spans mentioned above with special tokens for the\nsecond subtask of propaganda technique classification. Our models report an\nF-score of 44.6% and a micro-averaged F-score of 58.2% for those tasks\naccordingly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12916v1",
            "date": "2020-07-25 12:02:26+00:00",
            "title": "Bollyrics: Automatic Lyrics Generator for Romanised Hindi",
            "authors": [
                "Naman Jain",
                "Ankush Chauhan",
                "Atharva Chewale",
                "Ojas Mithbavkar",
                "Ujjaval Shah",
                "Mayank Singh"
            ],
            "abstract": "Song lyrics convey a meaningful story in a creative manner with complex\nrhythmic patterns. Researchers have been successful in generating and analyisng\nlyrics for poetry and songs in English and Chinese. But there are no works\nwhich explore the Hindi language datasets. Given the popularity of Hindi songs\nacross the world and the ambiguous nature of romanized Hindi script, we propose\nBollyrics, an automatic lyric generator for romanized Hindi songs. We propose\nsimple techniques to capture rhyming patterns before and during the model\ntraining process in Hindi language. The dataset and codes are available\npublicly at https://github.com/lingo-iitgn/Bollyrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12946v1",
            "date": "2020-07-25 14:49:31+00:00",
            "title": "Duluth at SemEval-2020 Task 12: Offensive Tweet Identification in English with Logistic Regression",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth systems that participated in SemEval--2020\nTask 12, Multilingual Offensive Language Identification in Social Media\n(OffensEval--2020). We participated in the three English language tasks. Our\nsystems provide a simple Machine Learning baseline using logistic regression.\nWe trained our models on the distantly supervised training data made available\nby the task organizers and used no other resources. As might be expected we did\nnot rank highly in the comparative evaluation: 79th of 85 in Task A, 34th of 43\nin Task B, and 24th of 39 in Task C. We carried out a qualitative analysis of\nour results and found that the class labels in the gold standard data are\nsomewhat noisy. We hypothesize that the extremely high accuracy (> 90%) of the\ntop ranked systems may reflect methods that learn the training data very well\nbut may not generalize to the task of identifying offensive language in\nEnglish. This analysis includes examples of tweets that despite being mildly\nredacted are still offensive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12949v1",
            "date": "2020-07-25 14:56:10+00:00",
            "title": "Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets",
            "authors": [
                "Ted Pedersen"
            ],
            "abstract": "This paper describes the Duluth systems that participated in SemEval--2019\nTask 6, Identifying and Categorizing Offensive Language in Social Media\n(OffensEval). For the most part these systems took traditional Machine Learning\napproaches that built classifiers from lexical features found in manually\nlabeled training data. However, our most successful system for classifying a\ntweet as offensive (or not) was a rule-based black--list approach, and we also\nexperimented with combining the training data from two different but related\nSemEval tasks. Our best systems in each of the three OffensEval tasks placed in\nthe middle of the comparative evaluation, ranking 57th of 103 in task A, 39th\nof 75 in task B, and 44th of 65 in task C.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13061v2",
            "date": "2020-07-26 05:48:46+00:00",
            "title": "Reed at SemEval-2020 Task 9: Fine-Tuning and Bag-of-Words Approaches to Code-Mixed Sentiment Analysis",
            "authors": [
                "Vinay Gopalan",
                "Mark Hopkins"
            ],
            "abstract": "We explore the task of sentiment analysis on Hinglish (code-mixed\nHindi-English) tweets as participants of Task 9 of the SemEval-2020\ncompetition, known as the SentiMix task. We had two main approaches: 1)\napplying transfer learning by fine-tuning pre-trained BERT models and 2)\ntraining feedforward neural networks on bag-of-words representations. During\nthe evaluation phase of the competition, we obtained an F-score of 71.3% with\nour best model, which placed $4^{th}$ out of 62 entries in the official system\nrankings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13184v1",
            "date": "2020-07-26 17:26:20+00:00",
            "title": "KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media",
            "authors": [
                "Ali Safaya",
                "Moutasem Abdullatif",
                "Deniz Yuret"
            ],
            "abstract": "In this paper, we describe our approach to utilize pre-trained BERT models\nwith Convolutional Neural Networks for sub-task A of the Multilingual Offensive\nLanguage Identification shared task (OffensEval 2020), which is a part of the\nSemEval 2020. We show that combining CNN with BERT is better than using BERT on\nits own, and we emphasize the importance of utilizing pre-trained language\nmodels for downstream tasks. Our system, ranked 4th with macro averaged\nF1-Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with\nscore of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of\npre-trained transformer language models for Arabic that we share with the\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13798v1",
            "date": "2020-07-27 18:25:01+00:00",
            "title": "Linguistic Taboos and Euphemisms in Nepali",
            "authors": [
                "Nobal B. Niraula",
                "Saurab Dulal",
                "Diwa Koirala"
            ],
            "abstract": "Languages across the world have words, phrases, and behaviors -- the taboos\n-- that are avoided in public communication considering them as obscene or\ndisturbing to the social, religious, and ethical values of society. However,\npeople deliberately use these linguistic taboos and other language constructs\nto make hurtful, derogatory, and obscene comments. It is nearly impossible to\nconstruct a universal set of offensive or taboo terms because offensiveness is\ndetermined entirely by different factors such as socio-physical setting,\nspeaker-listener relationship, and word choices. In this paper, we present a\ndetailed corpus-based study of offensive language in Nepali. We identify and\ndescribe more than 18 different categories of linguistic offenses including\npolitics, religion, race, and sex. We discuss 12 common euphemisms such as\nsynonym, metaphor and circumlocution. In addition, we introduce a manually\nconstructed data set of over 1000 offensive and taboo terms popular among\ncontemporary speakers. This in-depth study of offensive language and resource\nwill provide a foundation for several downstream tasks such as offensive\nlanguage detection and language learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13968v1",
            "date": "2020-07-28 03:20:31+00:00",
            "title": "YNU-HPCC at SemEval-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis",
            "authors": [
                "Li Yuan",
                "Jin Wang",
                "Xuejie Zhang"
            ],
            "abstract": "In recent years, the growing ubiquity of Internet memes on social media\nplatforms, such as Facebook, Instagram, and Twitter, has become a topic of\nimmense interest. However, the classification and recognition of memes is much\nmore complicated than that of social text since it involves visual cues and\nlanguage understanding. To address this issue, this paper proposed a\nparallel-channel model to process the textual and visual information in memes\nand then analyze the sentiment polarity of memes. In the shared task of\nidentifying and categorizing memes, we preprocess the dataset according to the\nlanguage behaviors on social media. Then, we adapt and fine-tune the\nBidirectional Encoder Representations from Transformers (BERT), and two types\nof convolutional neural network models (CNNs) were used to extract the features\nfrom the pictures. We applied an ensemble model that combined the BiLSTM,\nBIGRU, and Attention models to perform cross domain suggestion mining. The\nofficially released results show that our system performs better than the\nbaseline algorithm. Our team won nineteenth place in subtask A (Sentiment\nClassification). The code of this paper is availabled at :\nhttps://github.com/YuanLi95/Semveal2020-Task8-emotion-analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14074v1",
            "date": "2020-07-28 09:04:47+00:00",
            "title": "Preparation of Sentiment tagged Parallel Corpus and Testing its effect on Machine Translation",
            "authors": [
                "Sainik Kumar Mahata",
                "Amrita Chandra",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "In the current work, we explore the enrichment in the machine translation\noutput when the training parallel corpus is augmented with the introduction of\nsentiment analysis. The paper discusses the preparation of the same sentiment\ntagged English-Bengali parallel corpus. The preparation of raw parallel corpus,\nsentiment analysis of the sentences and the training of a Character Based\nNeural Machine Translation model using the same has been discussed extensively\nin this paper. The output of the translation model has been compared with a\nbase-line translation model using automated metrics such as BLEU and TER as\nwell as manually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14200v1",
            "date": "2020-07-28 13:30:46+00:00",
            "title": "ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation",
            "authors": [
                "Qian Zhao",
                "Siyu Tao",
                "Jie Zhou",
                "Linlin Wang",
                "Xin Lin",
                "Liang He"
            ],
            "abstract": "This paper describes our system for SemEval-2020 Task 4: Commonsense\nValidation and Explanation (Wang et al., 2020). We propose a novel\nKnowledge-enhanced Graph Attention Network (KEGAT) architecture for this task,\nleveraging heterogeneous knowledge from both the structured knowledge base\n(i.e. ConceptNet) and unstructured text to better improve the ability of a\nmachine in commonsense understanding. This model has a powerful commonsense\ninference capability via utilizing suitable commonsense incorporation methods\nand upgraded data augmentation techniques. Besides, an internal sharing\nmechanism is cooperated to prohibit our model from insufficient and excessive\nreasoning for commonsense. As a result, this model performs quite well in both\nvalidation and explanation. For instance, it achieves state-of-the-art accuracy\nin the subtask called Commonsense Explanation (Multi-Choice). We officially\nname the system as ECNU-SenseMaker. Code is publicly available at\nhttps://github.com/ECNU-ICA/ECNU-SenseMaker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14222v1",
            "date": "2020-07-28 13:50:17+00:00",
            "title": "Word embedding and neural network on grammatical gender -- A case study of Swedish",
            "authors": [
                "Marc Allassonni\u00e8re-Tang",
                "Ali Basirat"
            ],
            "abstract": "We analyze the information provided by the word embeddings about the\ngrammatical gender in Swedish. We wish that this paper may serve as one of the\nbridges to connect the methods of computational linguistics and general\nlinguistics. Taking nominal classification in Swedish as a case study, we first\nshow how the information about grammatical gender in language can be captured\nby word embedding models and artificial neural networks. Then, we match our\nresults with previous linguistic hypotheses on assignment and usage of\ngrammatical gender in Swedish and analyze the errors made by the computational\nmodel from a linguistic perspective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14310v1",
            "date": "2020-07-28 15:29:19+00:00",
            "title": "Improving Results on Russian Sentiment Datasets",
            "authors": [
                "Anton Golubev",
                "Natalia Loukachevitch"
            ],
            "abstract": "In this study, we test standard neural network architectures (CNN, LSTM,\nBiLSTM) and recently appeared BERT architectures on previous Russian sentiment\nevaluation datasets. We compare two variants of Russian BERT and show that for\nall sentiment tasks in this study the conversational variant of Russian BERT\nperforms better. The best results were achieved by BERT-NLI model, which treats\nsentiment classification tasks as a natural language inference task. On one of\nthe datasets, this model practically achieves the human level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14435v1",
            "date": "2020-07-28 19:09:11+00:00",
            "title": "Towards Ecologically Valid Research on Language User Interfaces",
            "authors": [
                "Harm de Vries",
                "Dzmitry Bahdanau",
                "Christopher Manning"
            ],
            "abstract": "Language User Interfaces (LUIs) could improve human-machine interaction for a\nwide variety of tasks, such as playing music, getting insights from databases,\nor instructing domestic robots. In contrast to traditional hand-crafted\napproaches, recent work attempts to build LUIs in a data-driven way using\nmodern deep learning methods. To satisfy the data needs of such learning\nalgorithms, researchers have constructed benchmarks that emphasize the quantity\nof collected data at the cost of its naturalness and relevance to real-world\nLUI use cases. As a consequence, research findings on such benchmarks might not\nbe relevant for developing practical LUIs. The goal of this paper is to\nbootstrap the discussion around this issue, which we refer to as the\nbenchmarks' low ecological validity. To this end, we describe what we deem an\nideal methodology for machine learning research on LUIs and categorize five\ncommon ways in which recent benchmarks deviate from it. We give concrete\nexamples of the five kinds of deviations and their consequences. Lastly, we\noffer a number of recommendations as to how to increase the ecological validity\nof machine learning research on LUIs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14454v1",
            "date": "2020-07-28 19:52:21+00:00",
            "title": "Measuring prominence of scientific work in online news as a proxy for impact",
            "authors": [
                "James Ravenscroft",
                "Amanda Clare",
                "Maria Liakata"
            ],
            "abstract": "The impact made by a scientific paper on the work of other academics has many\nestablished metrics, including metrics based on citation counts and social\nmedia commenting. However, determination of the impact of a scientific paper on\nthe wider society is less well established. For example, is it important for\nscientific work to be newsworthy? Here we present a new corpus of newspaper\narticles linked to the scientific papers that they describe. We find that\nImpact Case studies submitted to the UK Research Excellence Framework (REF)\n2014 that refer to scientific papers mentioned in newspaper articles were\nawarded a higher score in the REF assessment. The papers associated with these\ncase studies also feature prominently in the newspaper articles. We hypothesise\nthat such prominence can be a useful proxy for societal impact. We therefore\nprovide a novel baseline approach for measuring the prominence of scientific\npapers mentioned within news articles. Our measurement of prominence is based\non semantic similarity through a graph-based ranking algorithm. We find that\nscientific papers with an associated REF case study are more likely to have a\nstronger prominence score. This supports our hypothesis that linguistic\nprominence in news can be used to suggest the wider non-academic impact of\nscientific work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14477v1",
            "date": "2020-07-28 20:45:43+00:00",
            "title": "GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection",
            "authors": [
                "Sajad Sotudeh",
                "Tong Xiang",
                "Hao-Ren Yao",
                "Sean MacAvaney",
                "Eugene Yang",
                "Nazli Goharian",
                "Ophir Frieder"
            ],
            "abstract": "Offensive language detection is an important and challenging task in natural\nlanguage processing. We present our submissions to the OffensEval 2020 shared\ntask, which includes three English sub-tasks: identifying the presence of\noffensive language (Sub-task A), identifying the presence of target in\noffensive language (Sub-task B), and identifying the categories of the target\n(Sub-task C). Our experiments explore using a domain-tuned contextualized\nlanguage model (namely, BERT) for this task. We also experiment with different\ncomponents and configurations (e.g., a multi-view SVM) stacked upon BERT models\nfor specific sub-tasks. Our submissions achieve F1 scores of 91.7% in Sub-task\nA, 66.5% in Sub-task B, and 63.2% in Sub-task C. We perform an ablation study\nwhich reveals that domain tuning considerably improves the classification\nperformance. Furthermore, error analysis shows common misclassification errors\nmade by our model and outlines research directions for future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14576v1",
            "date": "2020-07-29 03:42:07+00:00",
            "title": "Development of POS tagger for English-Bengali Code-Mixed data",
            "authors": [
                "Tathagata Raha",
                "Sainik Kumar Mahata",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Code-mixed texts are widespread nowadays due to the advent of social media.\nSince these texts combine two languages to formulate a sentence, it gives rise\nto various research problems related to Natural Language Processing. In this\npaper, we try to excavate one such problem, namely, Parts of Speech tagging of\ncode-mixed texts. We have built a system that can POS tag English-Bengali\ncode-mixed data where the Bengali words were written in Roman script. Our\napproach initially involves the collection and cleaning of English-Bengali\ncode-mixed tweets. These tweets were used as a development dataset for building\nour system. The proposed system is a modular approach that starts by tagging\nindividual tokens with their respective languages and then passes them to\ndifferent POS taggers, designed for different languages (English and Bengali,\nin our case). Tags given by the two systems are later joined together and the\nfinal result is then mapped to a universal POS tag set. Our system was checked\nusing 100 manually POS tagged code-mixed sentences and it returned an accuracy\nof 75.29%",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14640v1",
            "date": "2020-07-29 07:27:41+00:00",
            "title": "Biomedical and Clinical English Model Packages in the Stanza Python NLP Library",
            "authors": [
                "Yuhao Zhang",
                "Yuhui Zhang",
                "Peng Qi",
                "Christopher D. Manning",
                "Curtis P. Langlotz"
            ],
            "abstract": "We introduce biomedical and clinical English model packages for the Stanza\nPython NLP library. These packages offer accurate syntactic analysis and named\nentity recognition capabilities for biomedical and clinical text, by combining\nStanza's fully neural architecture with a wide variety of open datasets as well\nas large-scale unsupervised biomedical and clinical text data. We show via\nextensive experiments that our packages achieve syntactic analysis and named\nentity recognition performance that is on par with or surpasses\nstate-of-the-art results. We further show that these models do not compromise\nspeed compared to existing toolkits when GPU acceleration is available, and are\nmade easy to download and use with Stanza's Python interface. A demonstration\nof our packages is available at: http://stanza.run/bio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14936v1",
            "date": "2020-07-29 16:19:02+00:00",
            "title": "#Brexit: Leave or Remain? The Role of User's Community and Diachronic Evolution on Stance Detection",
            "authors": [
                "Mirko Lai",
                "Viviana Patti",
                "Giancarlo Ruffo",
                "Paolo Rosso"
            ],
            "abstract": "Interest has grown around the classification of stance that users assume\nwithin online debates in recent years. Stance has been usually addressed by\nconsidering users posts in isolation, while social studies highlight that\nsocial communities may contribute to influence users' opinion. Furthermore,\nstance should be studied in a diachronic perspective, since it could help to\nshed light on users' opinion shift dynamics that can be recorded during the\ndebate. We analyzed the political discussion in UK about the BREXIT referendum\non Twitter, proposing a novel approach and annotation schema for stance\ndetection, with the main aim of investigating the role of features related to\nsocial network community and diachronic stance evolution. Classification\nexperiments show that such features provide very useful clues for detecting\nstance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15066v1",
            "date": "2020-07-16 08:02:36+00:00",
            "title": "An Experimental Study of The Effects of Position Bias on Emotion CauseExtraction",
            "authors": [
                "Jiayuan Ding",
                "Mayank Kejriwal"
            ],
            "abstract": "Emotion Cause Extraction (ECE) aims to identify emotion causes from a\ndocument after annotating the emotion keywords. Some baselines have been\nproposed to address this problem, such as rule-based, commonsense based and\nmachine learning methods. We show, however, that a simple random selection\napproach toward ECE that does not require observing the text achieves similar\nperformance compared to the baselines. We utilized only position information\nrelative to the emotion cause to accomplish this goal. Since position\ninformation alone without observing the text resulted in higher F-measure, we\ntherefore uncovered a bias in the ECE single genre Sina-news benchmark. Further\nanalysis showed that an imbalance of emotional cause location exists in the\nbenchmark, with a majority of cause clauses immediately preceding the central\nemotion clause. We examine the bias from a linguistic perspective, and show\nthat high accuracy rate of current state-of-art deep learning models that\nutilize location information is only evident in datasets that contain such\nposition biases. The accuracy drastically reduced when a dataset with balanced\nlocation distribution is introduced. We therefore conclude that it is the\ninnate bias in this benchmark that caused high accuracy rate of these deep\nlearning models in ECE. We hope that the case study in this paper presents both\na cautionary lesson, as well as a template for further studies, in interpreting\nthe superior fit of deep learning models without checking for bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15072v1",
            "date": "2020-07-29 19:38:35+00:00",
            "title": "Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification",
            "authors": [
                "Xin Dong",
                "Yaxin Zhu",
                "Yupeng Zhang",
                "Zuohui Fu",
                "Dongkuan Xu",
                "Sen Yang",
                "Gerard de Melo"
            ],
            "abstract": "In cross-lingual text classification, one seeks to exploit labeled data from\none language to train a text classification model that can then be applied to a\ncompletely different language. Recent multilingual representation models have\nmade it much easier to achieve this. Still, there may still be subtle\ndifferences between languages that are neglected when doing so. To address\nthis, we present a semi-supervised adversarial training process that minimizes\nthe maximal loss for label-preserving input perturbations. The resulting model\nthen serves as a teacher to induce labels for unlabeled target language samples\nthat can be used during further adversarial training, allowing us to gradually\nadapt our model to the target language. Compared with a number of strong\nbaselines, we observe significant gains in effectiveness on document and intent\nclassification for a diverse set of languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15207v1",
            "date": "2020-07-30 03:33:46+00:00",
            "title": "MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering",
            "authors": [
                "Shayne Longpre",
                "Yi Lu",
                "Joachim Daiber"
            ],
            "abstract": "Progress in cross-lingual modeling depends on challenging, realistic, and\ndiverse evaluation sets. We introduce Multilingual Knowledge Questions and\nAnswers (MKQA), an open-domain question answering evaluation set comprising 10k\nquestion-answer pairs aligned across 26 typologically diverse languages (260k\nquestion-answer pairs in total). The goal of this dataset is to provide a\nchallenging benchmark for question answering quality across a wide set of\nlanguages. Answers are based on a language-independent data representation,\nmaking results comparable across languages and independent of language-specific\npassages. With 26 languages, this dataset supplies the widest range of\nlanguages to-date for evaluating question answering. We benchmark\nstate-of-the-art extractive question answering baselines, trained on Natural\nQuestions, including Multilingual BERT, and XLM-RoBERTa, in zero shot and\ntranslation settings. Results indicate this dataset is challenging, especially\nin low-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15296v2",
            "date": "2020-07-30 08:22:47+00:00",
            "title": "Leverage Unlabeled Data for Abstractive Speech Summarization with Self-Supervised Learning and Back-Summarization",
            "authors": [
                "Paul Tardy",
                "Louis de Seynes",
                "Fran\u00e7ois Hernandez",
                "Vincent Nguyen",
                "David Janiszek",
                "Yannick Est\u00e8ve"
            ],
            "abstract": "Supervised approaches for Neural Abstractive Summarization require large\nannotated corpora that are costly to build. We present a French meeting\nsummarization task where reports are predicted based on the automatic\ntranscription of the meeting audio recordings. In order to build a corpus for\nthis task, it is necessary to obtain the (automatic or manual) transcription of\neach meeting, and then to segment and align it with the corresponding manual\nreport to produce training examples suitable for training. On the other hand,\nwe have access to a very large amount of unaligned data, in particular reports\nwithout corresponding transcription. Reports are professionally written and\nwell formatted making pre-processing straightforward. In this context, we study\nhow to take advantage of this massive amount of unaligned data using two\napproaches (i) self-supervised pre-training using a target-side denoising\nencoder-decoder model; (ii) back-summarization i.e. reversing the summarization\nprocess by learning to predict the transcription given the report, in order to\nalign single reports with generated transcription, and use this synthetic\ndataset for further training. We report large improvements compared to the\nprevious baseline (trained on aligned data only) for both approaches on two\nevaluation sets. Moreover, combining the two gives even better results,\noutperforming the baseline by a large margin of +6 ROUGE-1 and ROUGE-L and +5\nROUGE-2 on two evaluation sets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15620v2",
            "date": "2020-07-30 17:43:14+00:00",
            "title": "Neural Modeling for Named Entities and Morphology (NEMO^2)",
            "authors": [
                "Dan Bareket",
                "Reut Tsarfaty"
            ],
            "abstract": "Named Entity Recognition (NER) is a fundamental NLP task, commonly formulated\nas classification over a sequence of tokens. Morphologically-Rich Languages\n(MRLs) pose a challenge to this basic formulation, as the boundaries of Named\nEntities do not necessarily coincide with token boundaries, rather, they\nrespect morphological boundaries. To address NER in MRLs we then need to answer\ntwo fundamental questions, namely, what are the basic units to be labeled, and\nhow can these units be detected and classified in realistic settings, i.e.,\nwhere no gold morphology is available. We empirically investigate these\nquestions on a novel NER benchmark, with parallel tokenlevel and morpheme-level\nNER annotations, which we develop for Modern Hebrew, a morphologically\nrich-and-ambiguous language. Our results show that explicitly modeling\nmorphological boundaries leads to improved NER performance, and that a novel\nhybrid architecture, in which NER precedes and prunes morphological\ndecomposition, greatly outperforms the standard pipeline, where morphological\ndecomposition strictly precedes NER, setting a new performance bar for both\nHebrew NER and Hebrew morphological decomposition tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15700v1",
            "date": "2020-07-30 19:25:00+00:00",
            "title": "The Unreasonable Effectiveness of Machine Learning in Moldavian versus Romanian Dialect Identification",
            "authors": [
                "Mihaela G\u0103man",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this work, we provide a follow-up on the Moldavian versus Romanian\nCross-Dialect Topic Identification (MRC) shared task of the VarDial 2019\nEvaluation Campaign. The shared task included two sub-task types: one that\nconsisted in discriminating between the Moldavian and the Romanian dialects and\none that consisted in classifying documents by topic across the two dialects of\nRomanian. Participants achieved impressive scores, e.g. the top model for\nMoldavian versus Romanian dialect identification obtained a macro F1 score of\n0.895. We conduct a subjective evaluation by human annotators, showing that\nhumans attain much lower accuracy rates compared to machine learning (ML)\nmodels. Hence, it remains unclear why the methods proposed by participants\nattain such high accuracy rates. Our goal is to understand (i) why the proposed\nmethods work so well (by visualizing the discriminative features) and (ii) to\nwhat extent these methods can keep their high accuracy levels, e.g. when we\nshorten the text samples to single sentences or when use tweets at inference\ntime. A secondary goal of our work is to propose an improved ML model using\nensemble learning. Our experiments show that ML models can accurately identify\nthe dialects, even at the sentence level and across different domains (news\narticles versus tweets). We also analyze the most discriminative features of\nthe best performing models, providing some explanations behind the decisions\ntaken by these models. Interestingly, we learn new dialectal patterns\npreviously unknown to us or to our human annotators. Furthermore, we conduct\nexperiments showing that the machine learning performance on the MRC shared\ntask can be improved through an ensemble based on classifier stacking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15960v4",
            "date": "2020-07-31 10:58:39+00:00",
            "title": "On Learning Universal Representations Across Languages",
            "authors": [
                "Xiangpeng Wei",
                "Rongxiang Weng",
                "Yue Hu",
                "Luxi Xing",
                "Heng Yu",
                "Weihua Luo"
            ],
            "abstract": "Recent studies have demonstrated the overwhelming advantage of cross-lingual\npre-trained models (PTMs), such as multilingual BERT and XLM, on cross-lingual\nNLP tasks. However, existing approaches essentially capture the co-occurrence\namong tokens through involving the masked language model (MLM) objective with\ntoken-level cross entropy. In this work, we extend these approaches to learn\nsentence-level representations and show the effectiveness on cross-lingual\nunderstanding and generation. Specifically, we propose a Hierarchical\nContrastive Learning (HiCTL) method to (1) learn universal representations for\nparallel sentences distributed in one or multiple languages and (2) distinguish\nthe semantically-related words from a shared cross-lingual vocabulary for each\nsentence. We conduct evaluations on two challenging cross-lingual tasks, XTREME\nand machine translation. Experimental results show that the HiCTL outperforms\nthe state-of-the-art XLM-R by an absolute gain of 4.2% accuracy on the XTREME\nbenchmark as well as achieves substantial improvements on both of the\nhigh-resource and low-resource English-to-X translation tasks over strong\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16006v1",
            "date": "2020-07-23 16:03:50+00:00",
            "title": "Word Embeddings: Stability and Semantic Change",
            "authors": [
                "Lucas Rettenmeier"
            ],
            "abstract": "Word embeddings are computed by a class of techniques within natural language\nprocessing (NLP), that create continuous vector representations of words in a\nlanguage from a large text corpus. The stochastic nature of the training\nprocess of most embedding techniques can lead to surprisingly strong\ninstability, i.e. subsequently applying the same technique to the same data\ntwice, can produce entirely different results. In this work, we present an\nexperimental study on the instability of the training process of three of the\nmost influential embedding techniques of the last decade: word2vec, GloVe and\nfastText. Based on the experimental results, we propose a statistical model to\ndescribe the instability of embedding techniques and introduce a novel metric\nto measure the instability of the representation of an individual word.\nFinally, we propose a method to minimize the instability - by computing a\nmodified average over multiple runs - and apply it to a specific linguistic\nproblem: The detection and quantification of semantic change, i.e. measuring\nchanges in the meaning and usage of words over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16193v1",
            "date": "2020-07-31 17:44:41+00:00",
            "title": "SimulEval: An Evaluation Toolkit for Simultaneous Translation",
            "authors": [
                "Xutai Ma",
                "Mohammad Javad Dousti",
                "Changhan Wang",
                "Jiatao Gu",
                "Juan Pino"
            ],
            "abstract": "Simultaneous translation on both text and speech focuses on a real-time and\nlow-latency scenario where the model starts translating before reading the\ncomplete source input. Evaluating simultaneous translation models is more\ncomplex than offline models because the latency is another factor to consider\nin addition to translation quality. The research community, despite its growing\nfocus on novel modeling approaches to simultaneous translation, currently lacks\na universal evaluation procedure. Therefore, we present SimulEval, an\neasy-to-use and general evaluation toolkit for both simultaneous text and\nspeech translation. A server-client scheme is introduced to create a\nsimultaneous translation scenario, where the server sends source input and\nreceives predictions for evaluation and the client executes customized\npolicies. Given a policy, it automatically performs simultaneous decoding and\ncollectively reports several popular latency metrics. We also adapt latency\nmetrics from text simultaneous translation to the speech task. Additionally,\nSimulEval is equipped with a visualization interface to provide better\nunderstanding of the simultaneous decoding process of a system. SimulEval has\nalready been extensively used for the IWSLT 2020 shared task on simultaneous\nspeech translation. Code will be released upon publication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00293v1",
            "date": "2020-08-01 16:28:47+00:00",
            "title": "The test set for the TransCoder system",
            "authors": [
                "Ernest Davis"
            ],
            "abstract": "The TransCoder system translates source code between Java, C++, and Python 3.\nThe test set that was used to evaluate its quality is missing important\nfeatures of Java, including the ability to define and use classes and the\nability to call user-defined functions other than recursively. Therefore, the\naccuracy of TransCoder over programs with those features remains unknown.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00343v1",
            "date": "2020-08-01 21:22:53+00:00",
            "title": "Extracting actionable information from microtexts",
            "authors": [
                "Ali H\u00fcrriyeto\u011flu"
            ],
            "abstract": "Microblogs such as Twitter represent a powerful source of information. Part\nof this information can be aggregated beyond the level of individual posts.\nSome of this aggregated information is referring to events that could or should\nbe acted upon in the interest of e-governance, public safety, or other levels\nof public interest. Moreover, a significant amount of this information, if\naggregated, could complement existing information networks in a non-trivial\nway. This dissertation proposes a semi-automatic method for extracting\nactionable information that serves this purpose. First, we show that predicting\ntime to event is possible for both in-domain and cross-domain scenarios.\nSecond, we suggest a method which facilitates the definition of relevance for\nan analyst's context and the use of this definition to analyze new data.\nFinally, we propose a method to integrate the machine learning based relevant\ninformation classification method with a rule-based information classification\ntechnique to classify microtexts. Fully automatizing microtext analysis has\nbeen our goal since the first day of this research project. Our efforts in this\ndirection informed us about the extent this automation can be realized. We\nmostly first developed an automated approach, then we extended and improved it\nby integrating human intervention at various steps of the automated approach.\nOur experience confirms previous work that states that a well-designed human\nintervention or contribution in design, realization, or evaluation of an\ninformation system either improves its performance or enables its realization.\nAs our studies and results directed us toward its necessity and value, we were\ninspired from previous studies in designing human involvement and customized\nour approaches to benefit from human input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00345v1",
            "date": "2020-08-01 21:39:54+00:00",
            "title": "Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting",
            "authors": [
                "Ali H\u00fcrriyeto\u011flu",
                "Erdem Y\u00f6r\u00fck",
                "Deniz Y\u00fcret",
                "\u00c7a\u011fr\u0131 Yoltar",
                "Burak G\u00fcrel",
                "F\u0131rat Duru\u015fan",
                "Osman Mutlu",
                "Arda Akdemir"
            ],
            "abstract": "We present an overview of the CLEF-2019 Lab ProtestNews on Extracting\nProtests from News in the context of generalizable natural language processing.\nThe lab consists of document, sentence, and token level information\nclassification and extraction tasks that were referred as task 1, task 2, and\ntask 3 respectively in the scope of this lab. The tasks required the\nparticipants to identify protest relevant information from English local news\nat one or more aforementioned levels in a cross-context setting, which is\ncross-country in the scope of this lab. The training and development data were\ncollected from India and test data was collected from India and China. The lab\nattracted 58 teams to participate in the lab. 12 and 9 of these teams submitted\nresults and working notes respectively. We have observed neural networks yield\nthe best results and the performance drops significantly for majority of the\nsubmissions in the cross-country setting, which is China.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00351v1",
            "date": "2020-08-01 22:20:48+00:00",
            "title": "Cross-context News Corpus for Protest Events related Knowledge Base Construction",
            "authors": [
                "Ali H\u00fcrriyeto\u011flu",
                "Erdem Y\u00f6r\u00fck",
                "Deniz Y\u00fcret",
                "Osman Mutlu",
                "\u00c7a\u011fr\u0131 Yoltar",
                "F\u0131rat Duru\u015fan",
                "Burak G\u00fcrel"
            ],
            "abstract": "We describe a gold standard corpus of protest events that comprise of various\nlocal and international sources from various countries in English. The corpus\ncontains document, sentence, and token level annotations. This corpus\nfacilitates creating machine learning models that automatically classify news\narticles and extract protest event-related information, constructing knowledge\nbases which enable comparative social and political science studies. For each\nnews source, the annotation starts on random samples of news articles and\ncontinues with samples that are drawn using active learning. Each batch of\nsamples was annotated by two social and political scientists, adjudicated by an\nannotation supervisor, and was improved by identifying annotation errors\nsemi-automatically. We found that the corpus has the variety and quality to\ndevelop and benchmark text classification and event extraction systems in a\ncross-context setting, which contributes to the generalizability and robustness\nof automated text processing systems. This corpus and the reported results will\nset the currently lacking common ground in automated protest event collection\nstudies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00364v5",
            "date": "2020-08-02 00:09:03+00:00",
            "title": "A Survey on Text Classification: From Shallow to Deep Learning",
            "authors": [
                "Qian Li",
                "Hao Peng",
                "Jianxin Li",
                "Congying Xia",
                "Renyu Yang",
                "Lichao Sun",
                "Philip S. Yu",
                "Lifang He"
            ],
            "abstract": "Text classification is the most fundamental and essential task in natural\nlanguage processing. The last decade has seen a surge of research in this area\ndue to the unprecedented success of deep learning. Numerous methods, datasets,\nand evaluation metrics have been proposed in the literature, raising the need\nfor a comprehensive and updated survey. This paper fills the gap by reviewing\nthe state of the art approaches from 1961 to 2020, focusing on models from\nshallow to deep learning. We create a taxonomy for text classification\naccording to the text involved and the models used for feature extraction and\nclassification. We then discuss each of these categories in detail, dealing\nwith both the technical developments and benchmark datasets that support tests\nof predictions. A comprehensive comparison between different techniques, as\nwell as identifying the pros and cons of various evaluation metrics are also\nprovided in this survey. Finally, we conclude by summarizing key implications,\nfuture research directions, and the challenges facing the research area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00401v1",
            "date": "2020-08-02 05:36:55+00:00",
            "title": "Multilingual Translation with Extensible Multilingual Pretraining and Finetuning",
            "authors": [
                "Yuqing Tang",
                "Chau Tran",
                "Xian Li",
                "Peng-Jen Chen",
                "Naman Goyal",
                "Vishrav Chaudhary",
                "Jiatao Gu",
                "Angela Fan"
            ],
            "abstract": "Recent work demonstrates the potential of multilingual pretraining of\ncreating one model that can be used for various tasks in different languages.\nPrevious work in multilingual pretraining has demonstrated that machine\ntranslation systems can be created by finetuning on bitext. In this work, we\nshow that multilingual translation models can be created through multilingual\nfinetuning. Instead of finetuning on one direction, a pretrained model is\nfinetuned on many directions at the same time. Compared to multilingual models\ntrained from scratch, starting from pretrained models incorporates the benefits\nof large quantities of unlabeled monolingual data, which is particularly\nimportant for low resource languages where bitext is not available. We\ndemonstrate that pretrained models can be extended to incorporate additional\nlanguages without loss of performance. We double the number of languages in\nmBART to support multilingual machine translation models of 50 languages.\nFinally, we create the ML50 benchmark, covering low, mid, and high resource\nlanguages, to facilitate reproducible research by standardizing training and\nevaluation data. On ML50, we demonstrate that multilingual finetuning improves\non average 1 BLEU over the strongest baselines (being either multilingual from\nscratch or bilingual finetuning) while improving 9.3 BLEU on average over\nbilingual baselines from scratch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00441v2",
            "date": "2020-08-02 09:52:58+00:00",
            "title": "Relation Extraction with Self-determined Graph Convolutional Network",
            "authors": [
                "Sunil Kumar Sahu",
                "Derek Thomas",
                "Billy Chiu",
                "Neha Sengupta",
                "Mohammady Mahdy"
            ],
            "abstract": "Relation Extraction is a way of obtaining the semantic relationship between\nentities in text. The state-of-the-art methods use linguistic tools to build a\ngraph for the text in which the entities appear and then a Graph Convolutional\nNetwork (GCN) is employed to encode the pre-built graphs. Although their\nperformance is promising, the reliance on linguistic tools results in a non\nend-to-end process. In this work, we propose a novel model, the Self-determined\nGraph Convolutional Network (SGCN), which determines a weighted graph using a\nself-attention mechanism, rather using any linguistic tool. Then, the\nself-determined graph is encoded using a GCN. We test our model on the TACRED\ndataset and achieve the state-of-the-art result. Our experiments show that SGCN\noutperforms the traditional GCN, which uses dependency parsing tools to build\nthe graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00482v1",
            "date": "2020-08-02 13:50:47+00:00",
            "title": "Investigating the Effect of Emoji in Opinion Classification of Uzbek Movie Review Comments",
            "authors": [
                "Ilyos Rabbimov",
                "Iosif Mporas",
                "Vasiliki Simaki",
                "Sami Kobilov"
            ],
            "abstract": "Opinion mining on social media posts has become more and more popular. Users\noften express their opinion on a topic not only with words but they also use\nimage symbols such as emoticons and emoji. In this paper, we investigate the\neffect of emoji-based features in opinion classification of Uzbek texts, and\nmore specifically movie review comments from YouTube. Several classification\nalgorithms are tested, and feature ranking is performed to evaluate the\ndiscriminative ability of the emoji-based features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00563v1",
            "date": "2020-08-02 20:32:19+00:00",
            "title": "SemEval-2020 Task 5: Counterfactual Recognition",
            "authors": [
                "Xiaoyu Yang",
                "Stephen Obadinma",
                "Huasha Zhao",
                "Qiong Zhang",
                "Stan Matwin",
                "Xiaodan Zhu"
            ],
            "abstract": "We present a counterfactual recognition (CR) task, the shared Task 5 of\nSemEval-2020. Counterfactuals describe potential outcomes (consequents)\nproduced by actions or circumstances that did not happen or cannot happen and\nare counter to the facts (antecedent). Counterfactual thinking is an important\ncharacteristic of the human cognitive system; it connects antecedents and\nconsequents with causal relations. Our task provides a benchmark for\ncounterfactual recognition in natural language with two subtasks. Subtask-1\naims to determine whether a given sentence is a counterfactual statement or\nnot. Subtask-2 requires the participating systems to extract the antecedent and\nconsequent in a given counterfactual statement. During the SemEval-2020\nofficial evaluation period, we received 27 submissions to Subtask-1 and 11 to\nSubtask-2. The data, baseline code, and leaderboard can be found at\nhttps://competitions.codalab.org/competitions/21691. The data and baseline code\nare also available at https://zenodo.org/record/3932442.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00670v1",
            "date": "2020-08-03 06:43:11+00:00",
            "title": "Deep Learning based Topic Analysis on Financial Emerging Event Tweets",
            "authors": [
                "Shaan Aryaman",
                "Nguwi Yok Yen"
            ],
            "abstract": "Financial analyses of stock markets rely heavily on quantitative approaches\nin an attempt to predict subsequent or market movements based on historical\nprices and other measurable metrics. These quantitative analyses might have\nmissed out on un-quantifiable aspects like sentiment and speculation that also\nimpact the market. Analyzing vast amounts of qualitative text data to\nunderstand public opinion on social media platform is one approach to address\nthis gap. This work carried out topic analysis on 28264 financial tweets [1]\nvia clustering to discover emerging events in the stock market. Three main\ntopics were discovered to be discussed frequently within the period. First, the\nfinancial ratio EPS is a measure that has been discussed frequently by\ninvestors. Secondly, short selling of shares were discussed heavily, it was\noften mentioned together with Morgan Stanley. Thirdly, oil and energy sectors\nwere often discussed together with policy. These tweets were semantically\nclustered by a method consisting of word2vec algorithm to obtain word\nembeddings that map words to vectors. Semantic word clusters were then formed.\nEach tweet was then vectorized using the Term Frequency-Inverse Document\nFrequency (TF-IDF) values of the words it consisted of and based on which\nclusters its words were in. Tweet vectors were then converted to compressed\nrepresentations by training a deep-autoencoder. K-means clusters were then\nformed. This method reduces dimensionality and produces dense vectors, in\ncontrast to the usual Vector Space Model. Topic modelling with Latent Dirichlet\nAllocation (LDA) and top frequent words were used to analyze clusters and\nreveal emerging events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00805v1",
            "date": "2020-08-03 12:03:17+00:00",
            "title": "LT@Helsinki at SemEval-2020 Task 12: Multilingual or language-specific BERT?",
            "authors": [
                "Marc P\u00e0mies",
                "Emily \u00d6hman",
                "Kaisla Kajava",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "This paper presents the different models submitted by the LT@Helsinki team\nfor the SemEval 2020 Shared Task 12. Our team participated in sub-tasks A and\nC; titled offensive language identification and offense target identification,\nrespectively. In both cases we used the so-called Bidirectional Encoder\nRepresentation from Transformer (BERT), a model pre-trained by Google and\nfine-tuned by us on the OLID and SOLID datasets. The results show that\noffensive tweet classification is one of several language-based tasks where\nBERT can achieve state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00853v2",
            "date": "2020-08-03 13:05:42+00:00",
            "title": "Predicting the Humorousness of Tweets Using Gaussian Process Preference Learning",
            "authors": [
                "Tristan Miller",
                "Erik-L\u00e2n Do Dinh",
                "Edwin Simpson",
                "Iryna Gurevych"
            ],
            "abstract": "Most humour processing systems to date make at best discrete, coarse-grained\ndistinctions between the comical and the conventional, yet such notions are\nbetter conceptualized as a broad spectrum. In this paper, we present a\nprobabilistic approach, a variant of Gaussian process preference learning\n(GPPL), that learns to rank and rate the humorousness of short texts by\nexploiting human preference judgments and automatically sourced linguistic\nannotations. We apply our system, which is similar to one that had previously\nshown good performance on English-language one-liners annotated with pairwise\nhumorousness annotations, to the Spanish-language data set of the\nHAHA@IberLEF2019 evaluation campaign. We report system performance for the\ncampaign's two subtasks, humour detection and funniness score prediction, and\ndiscuss some issues arising from the conversion between the numeric scores used\nin the HAHA@IberLEF2019 data and the pairwise judgment annotations required for\nour method.",
            "journal ref": "Procesamiento del Lenguaje Natural, 64:37-44, March 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01354v1",
            "date": "2020-08-04 06:20:50+00:00",
            "title": "NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer",
            "authors": [
                "Hwijeen Ahn",
                "Jimin Sun",
                "Chan Young Park",
                "Jungyun Seo"
            ],
            "abstract": "This paper describes our approach to the task of identifying offensive\nlanguages in a multilingual setting. We investigate two data augmentation\nstrategies: using additional semi-supervised labels with different thresholds\nand cross-lingual transfer with data selection. Leveraging the semi-supervised\ndataset resulted in performance improvements compared to the baseline trained\nsolely with the manually-annotated dataset. We propose a new metric,\nTranslation Embedding Distance, to measure the transferability of instances for\ncross-lingual data selection. We also introduce various preprocessing steps\ntailored for social media text along with methods to fine-tune the pre-trained\nmultilingual BERT (mBERT) for offensive language identification. Our\nmultilingual systems achieved competitive results in Greek, Danish, and Turkish\nat OffensEval 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01391v1",
            "date": "2020-08-04 07:59:02+00:00",
            "title": "A Survey of Orthographic Information in Machine Translation",
            "authors": [
                "Bharathi Raja Chakravarthi",
                "Priya Rani",
                "Mihael Arcan",
                "John P. McCrae"
            ],
            "abstract": "Machine translation is one of the applications of natural language processing\nwhich has been explored in different languages. Recently researchers started\npaying attention towards machine translation for resource-poor languages and\nclosely related languages. A widespread and underlying problem for these\nmachine translation systems is the variation in orthographic conventions which\ncauses many issues to traditional approaches. Two languages written in two\ndifferent orthographies are not easily comparable, but orthographic information\ncan also be used to improve the machine translation system. This article offers\na survey of research regarding orthography's influence on machine translation\nof under-resourced languages. It introduces under-resourced languages in terms\nof machine translation and how orthographic information can be utilised to\nimprove machine translation. We describe previous work in this area, discussing\nwhat underlying assumptions were made, and showing how orthographic knowledge\nimproves the performance of machine translation of under-resourced languages.\nWe discuss different types of machine translation and demonstrate a recent\ntrend that seeks to link orthographic information with well-established machine\ntranslation methods. Considerable attention is given to current efforts of\ncognates information at different levels of machine translation and the lessons\nthat can be drawn from this. Additionally, multilingual neural machine\ntranslation of closely related languages is given a particular focus in this\nsurvey. This article ends with a discussion of the way forward in machine\ntranslation with orthographic information, focusing on multilingual settings\nand bilingual lexicon induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01466v2",
            "date": "2020-08-04 11:25:09+00:00",
            "title": "Taking Notes on the Fly Helps BERT Pre-training",
            "authors": [
                "Qiyu Wu",
                "Chen Xing",
                "Yatao Li",
                "Guolin Ke",
                "Di He",
                "Tie-Yan Liu"
            ],
            "abstract": "How to make unsupervised language pre-training more efficient and less\nresource-intensive is an important research direction in NLP. In this paper, we\nfocus on improving the efficiency of language pre-training methods through\nproviding better data utilization. It is well-known that in language data\ncorpus, words follow a heavy-tail distribution. A large proportion of words\nappear only very few times and the embeddings of rare words are usually poorly\noptimized. We argue that such embeddings carry inadequate semantic signals,\nwhich could make the data utilization inefficient and slow down the\npre-training of the entire model. To mitigate this problem, we propose Taking\nNotes on the Fly (TNF), which takes notes for rare words on the fly during\npre-training to help the model understand them when they occur next time.\nSpecifically, TNF maintains a note dictionary and saves a rare word's\ncontextual information in it as notes when the rare word occurs in a sentence.\nWhen the same rare word occurs again during training, the note information\nsaved beforehand can be employed to enhance the semantics of the current\nsentence. By doing so, TNF provides better data utilization since\ncross-sentence information is employed to cover the inadequate semantics caused\nby rare words in the sentences. We implement TNF on both BERT and ELECTRA to\ncheck its efficiency and effectiveness. Experimental results show that TNF's\ntraining time is $60\\%$ less than its backbone pre-training models when\nreaching the same performance. When trained with the same number of iterations,\nTNF outperforms its backbone methods on most of downstream tasks and the\naverage GLUE score. Source code is attached in the supplementary material.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01474v3",
            "date": "2020-08-04 11:52:20+00:00",
            "title": "SARG: A Novel Semi Autoregressive Generator for Multi-turn Incomplete Utterance Restoration",
            "authors": [
                "Mengzuo Huang",
                "Feng Li",
                "Wuhe Zou",
                "Weidong Zhang"
            ],
            "abstract": "Dialogue systems in open domain have achieved great success due to the easily\nobtained single-turn corpus and the development of deep learning, but the\nmulti-turn scenario is still a challenge because of the frequent coreference\nand information omission. In this paper, we investigate the incomplete\nutterance restoration which has brought general improvement over multi-turn\ndialogue systems in recent studies. Meanwhile, jointly inspired by the\nautoregression for text generation and the sequence labeling for text editing,\nwe propose a novel semi autoregressive generator (SARG) with the high\nefficiency and flexibility. Moreover, experiments on two benchmarks show that\nour proposed model significantly outperforms the state-of-the-art models in\nterms of quality and inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01523v2",
            "date": "2020-07-28 01:33:54+00:00",
            "title": "A System for Worldwide COVID-19 Information Aggregation",
            "authors": [
                "Akiko Aizawa",
                "Frederic Bergeron",
                "Junjie Chen",
                "Fei Cheng",
                "Katsuhiko Hayashi",
                "Kentaro Inui",
                "Hiroyoshi Ito",
                "Daisuke Kawahara",
                "Masaru Kitsuregawa",
                "Hirokazu Kiyomaru",
                "Masaki Kobayashi",
                "Takashi Kodama",
                "Sadao Kurohashi",
                "Qianying Liu",
                "Masaki Matsubara",
                "Yusuke Miyao",
                "Atsuyuki Morishima",
                "Yugo Murawaki",
                "Kazumasa Omura",
                "Haiyue Song",
                "Eiichiro Sumita",
                "Shinji Suzuki",
                "Ribeka Tanaka",
                "Yu Tanaka",
                "Masashi Toyoda",
                "Nobuhiro Ueda",
                "Honai Ueoka",
                "Masao Utiyama",
                "Ying Zhong"
            ],
            "abstract": "The global pandemic of COVID-19 has made the public pay close attention to\nrelated news, covering various domains, such as sanitation, treatment, and\neffects on education. Meanwhile, the COVID-19 condition is very different among\nthe countries (e.g., policies and development of the epidemic), and thus\ncitizens would be interested in news in foreign countries. We build a system\nfor worldwide COVID-19 information aggregation containing reliable articles\nfrom 10 regions in 7 languages sorted by topics. Our reliable COVID-19 related\nwebsite dataset collected through crowdsourcing ensures the quality of the\narticles. A neural machine translation module translates articles in other\nlanguages into Japanese and English. A BERT-based topic-classifier trained on\nour article-topic pair dataset helps users find their interested information\nefficiently by putting articles into different categories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01545v1",
            "date": "2020-07-27 23:58:54+00:00",
            "title": "ULD@NUIG at SemEval-2020 Task 9: Generative Morphemes with an Attention Model for Sentiment Analysis in Code-Mixed Text",
            "authors": [
                "Koustava Goswami",
                "Priya Rani",
                "Bharathi Raja Chakravarthi",
                "Theodorus Fransen",
                "John P. McCrae"
            ],
            "abstract": "Code mixing is a common phenomena in multilingual societies where people\nswitch from one language to another for various reasons. Recent advances in\npublic communication over different social media sites have led to an increase\nin the frequency of code-mixed usage in written language. In this paper, we\npresent the Generative Morphemes with Attention (GenMA) Model sentiment\nanalysis system contributed to SemEval 2020 Task 9 SentiMix. The system aims to\npredict the sentiments of the given English-Hindi code-mixed tweets without\nusing word-level language tags instead inferring this automatically using a\nmorphological model. The system is based on a novel deep neural network (DNN)\narchitecture, which has outperformed the baseline F1-score on the test data-set\nas well as the validation data-set. Our results can be found under the user\nname \"koustava\" on the \"Sentimix Hindi English\" page",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01546v1",
            "date": "2020-07-27 20:45:13+00:00",
            "title": "Next word prediction based on the N-gram model for Kurdish Sorani and Kurmanji",
            "authors": [
                "Hozan K. Hamarashid",
                "Soran A. Saeed",
                "Tarik A. Rashid"
            ],
            "abstract": "Next word prediction is an input technology that simplifies the process of\ntyping by suggesting the next word to a user to select, as typing in a\nconversation consumes time. A few previous studies have focused on the Kurdish\nlanguage, including the use of next word prediction. However, the lack of a\nKurdish text corpus presents a challenge. Moreover, the lack of a sufficient\nnumber of N-grams for the Kurdish language, for instance, five grams, is the\nreason for the rare use of next Kurdish word prediction. Furthermore, the\nimproper display of several Kurdish letters in the Rstudio software is another\nproblem. This paper provides a Kurdish corpus, creates five, and presents a\nunique research work on next word prediction for Kurdish Sorani and Kurmanji.\nThe N-gram model has been used for next word prediction to reduce the amount of\ntime while typing in the Kurdish language. In addition, little work has been\nconducted on next Kurdish word prediction; thus, the N-gram model is utilized\nto suggest text accurately. To do so, R programming and RStudio are used to\nbuild the application. The model is 96.3% accurate.",
            "journal ref": "Neural Computing and Applications, NCAA-D-19-02773R1, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01555v1",
            "date": "2020-08-01 19:22:51+00:00",
            "title": "Tense, aspect and mood based event extraction for situation analysis and crisis management",
            "authors": [
                "Ali H\u00fcrriyeto\u011flu"
            ],
            "abstract": "Nowadays event extraction systems mainly deal with a relatively small amount\nof information about temporal and modal qualifications of situations, primarily\nprocessing assertive sentences in the past tense. However, systems with a wider\ncoverage of tense, aspect and mood can provide better analyses and can be used\nin a wider range of text analysis applications. This thesis develops such a\nsystem for Turkish language. This is accomplished by extending Open Source\nInformation Mining and Analysis (OPTIMA) research group's event extraction\nsoftware, by implementing appropriate extensions in the semantic representation\nformat, by adding a partial grammar which improves the TAM (Tense, Aspect and\nMood) marker, adverb analysis and matching functions of ExPRESS, and by\nconstructing an appropriate lexicon in the standard of CORLEONE. These\nextensions are based on iv the theory of anchoring relations (Tem\\\"urc\\\"u,\n2007, 2011) which is a crosslinguistically applicable semantic framework for\nanalyzing tense, aspect and mood related categories. The result is a system\nwhich can, in addition to extracting basic event structures, classify sentences\ngiven in news reports according to their temporal, modal and\nvolitional/illocutionary values. Although the focus is on news reports of\nnatural disasters, disease outbreaks and man-made disasters in Turkish\nlanguage, the approach can be adapted to other languages, domains and genres.\nThis event extraction and classification system, with further developments, can\nprovide a basis for automated browsing systems for preventing environmental and\nhumanitarian risk.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01739v1",
            "date": "2020-08-04 18:00:07+00:00",
            "title": "Select, Extract and Generate: Neural Keyphrase Generation with Syntactic Guidance",
            "authors": [
                "Wasi Uddin Ahmad",
                "Xiao Bai",
                "Soomin Lee",
                "Kai-Wei Chang"
            ],
            "abstract": "In recent years, deep neural sequence-to-sequence framework has demonstrated\npromising results in keyphrase generation. However, processing long documents\nusing such deep neural networks requires high computational resources. To\nreduce the computational cost, the documents are typically truncated before\ngiven as inputs. As a result, the models may miss essential points conveyed in\na document. Moreover, most of the existing methods are either extractive\n(identify important phrases from the document) or generative (generate phrases\nword by word), and hence they do not benefit from the advantages of both\nmodeling techniques. To address these challenges, we propose \\emph{SEG-Net}, a\nneural keyphrase generation model that is composed of two major components, (1)\na selector that selects the salient sentences in a document, and (2) an\nextractor-generator that jointly extracts and generates keyphrases from the\nselected sentences. SEG-Net uses a self-attentive architecture, known as,\n\\emph{Transformer} as the building block with a couple of uniqueness. First,\nSEG-Net incorporates a novel \\emph{layer-wise} coverage attention to summarize\nmost of the points discussed in the target document. Second, it uses an\n\\emph{informed} copy attention mechanism to encourage focusing on different\nsegments of the document during keyphrase extraction and generation. Besides,\nSEG-Net jointly learns keyphrase generation and their part-of-speech tag\nprediction, where the later provides syntactic supervision to the former. The\nexperimental results on seven keyphrase generation benchmarks from scientific\nand web documents demonstrate that SEG-Net outperforms the state-of-the-art\nneural generative methods by a large margin in both domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01809v1",
            "date": "2020-08-04 20:13:51+00:00",
            "title": "Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring",
            "authors": [
                "Haoran Zhang",
                "Diane Litman"
            ],
            "abstract": "While automated essay scoring (AES) can reliably grade essays at scale,\nautomated writing evaluation (AWE) additionally provides formative feedback to\nguide essay revision. However, a neural AES typically does not provide useful\nfeature representations for supporting AWE. This paper presents a method for\nlinking AWE and neural AES, by extracting Topical Components (TCs) representing\nevidence from a source text using the intermediate output of attention layers.\nWe evaluate performance using a feature-based AES requiring TCs. Results show\nthat performance is comparable whether using automatically or manually\nconstructed TCs for 1) representing essays as rubric-based features, 2) grading\nessays.",
            "journal ref": "The 58th Annual Meeting of the Association for Computational\n  Linguistics, pp. 8569-8584. 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01940v1",
            "date": "2020-08-05 05:19:44+00:00",
            "title": "Designing the Business Conversation Corpus",
            "authors": [
                "Mat\u012bss Rikters",
                "Ryokan Ri",
                "Tong Li",
                "Toshiaki Nakazawa"
            ],
            "abstract": "While the progress of machine translation of written text has come far in the\npast several years thanks to the increasing availability of parallel corpora\nand corpora-based training technologies, automatic translation of spoken text\nand dialogues remains challenging even for modern systems. In this paper, we\naim to boost the machine translation quality of conversational texts by\nintroducing a newly constructed Japanese-English business conversation parallel\ncorpus. A detailed analysis of the corpus is provided along with challenging\nexamples for automatic translation. We also experiment with adding the corpus\nin a machine translation training scenario and show how the resulting system\nbenefits from its use.",
            "journal ref": "Published in proceedings of the 6th Workshop on Asian Translation,\n  2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01946v2",
            "date": "2020-08-05 06:01:46+00:00",
            "title": "An exploration of the encoding of grammatical gender in word embeddings",
            "authors": [
                "Hartger Veeman",
                "Ali Basirat"
            ],
            "abstract": "The vector representation of words, known as word embeddings, has opened a\nnew research approach in linguistic studies. These representations can capture\ndifferent types of information about words. The grammatical gender of nouns is\na typical classification of nouns based on their formal and semantic\nproperties. The study of grammatical gender based on word embeddings can give\ninsight into discussions on how grammatical genders are determined. In this\nstudy, we compare different sets of word embeddings according to the accuracy\nof a neural classifier determining the grammatical gender of nouns. It is found\nthat there is an overlap in how grammatical gender is encoded in Swedish,\nDanish, and Dutch embeddings. Our experimental results on the contextualized\nembeddings pointed out that adding more contextual information to embeddings is\ndetrimental to the classifier's performance. We also observed that removing\nmorpho-syntactic features such as articles from the training corpora of\nembeddings decreases the classification performance dramatically, indicating a\nlarge portion of the information is encoded in the relationship between nouns\nand articles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02047v1",
            "date": "2020-08-05 11:11:55+00:00",
            "title": "Multiple Texts as a Limiting Factor in Online Learning: Quantifying (Dis-)similarities of Knowledge Networks across Languages",
            "authors": [
                "Alexander Mehler",
                "Wahed Hemati",
                "Pascal Welke",
                "Maxim Konca",
                "Tolga Uslu"
            ],
            "abstract": "We test the hypothesis that the extent to which one obtains information on a\ngiven topic through Wikipedia depends on the language in which it is consulted.\nControlling the size factor, we investigate this hypothesis for a number of 25\nsubject areas. Since Wikipedia is a central part of the web-based information\nlandscape, this indicates a language-related, linguistic bias. The article\ntherefore deals with the question of whether Wikipedia exhibits this kind of\nlinguistic relativity or not. From the perspective of educational science, the\narticle develops a computational model of the information landscape from which\nmultiple texts are drawn as typical input of web-based reading. For this\npurpose, it develops a hybrid model of intra- and intertextual similarity of\ndifferent parts of the information landscape and tests this model on the\nexample of 35 languages and corresponding Wikipedias. In this way the article\nbuilds a bridge between reading research, educational science, Wikipedia\nresearch and computational linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02096v1",
            "date": "2020-08-05 12:56:16+00:00",
            "title": "Computational linguistic assessment of textbook and online learning media by means of threshold concepts in business education",
            "authors": [
                "Andy L\u00fccking",
                "Sebastian Br\u00fcckner",
                "Giuseppe Abrami",
                "Tolga Uslu",
                "Alexander Mehler"
            ],
            "abstract": "Threshold concepts are key terms in domain-based knowledge acquisition. They\nare regarded as building blocks of the conceptual development of domain\nknowledge within particular learners. From a linguistic perspective, however,\nthreshold concepts are instances of specialized vocabularies, exhibiting\nparticular linguistic features. Threshold concepts are typically used in\nspecialized texts such as textbooks -- that is, within a formal learning\nenvironment. However, they also occur in informal learning environments like\nnewspapers. In this article, a first approach is taken to combine both lines\ninto an overarching research program - that is, to provide a computational\nlinguistic assessment of different resources, including in particular online\nresources, by means of threshold concepts. To this end, the distributive\nprofiles of 63 threshold concepts from business education (which have been\ncollected from threshold concept research) has been investigated in three kinds\nof (German) resources, namely textbooks, newspapers, and Wikipedia. Wikipedia\nis (one of) the largest and most widely used online resources. We looked at the\nthreshold concepts' frequency distribution, their compound distribution, and\ntheir network structure within the three kind of resources. The two main\nfindings can be summarized as follows: Firstly, the three kinds of resources\ncan indeed be distinguished in terms of their threshold concepts' profiles.\nSecondly, Wikipedia definitely appears to be a formal learning resource.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02270v1",
            "date": "2020-08-05 17:52:25+00:00",
            "title": "Contextualized Translation of Automatically Segmented Speech",
            "authors": [
                "Marco Gaido",
                "Mattia Antonino Di Gangi",
                "Matteo Negri",
                "Mauro Cettolo",
                "Marco Turchi"
            ],
            "abstract": "Direct speech-to-text translation (ST) models are usually trained on corpora\nsegmented at sentence level, but at inference time they are commonly fed with\naudio split by a voice activity detector (VAD). Since VAD segmentation is not\nsyntax-informed, the resulting segments do not necessarily correspond to\nwell-formed sentences uttered by the speaker but, most likely, to fragments of\none or more sentences. This segmentation mismatch degrades considerably the\nquality of ST models' output. So far, researchers have focused on improving\naudio segmentation towards producing sentence-like splits. In this paper,\ninstead, we address the issue in the model, making it more robust to a\ndifferent, potentially sub-optimal segmentation. To this aim, we train our\nmodels on randomly segmented data and compare two approaches: fine-tuning and\nadding the previous segment as context. We show that our context-aware solution\nis more robust to VAD-segmented input, outperforming a strong base model and\nthe fine-tuning on different VAD segmentations of an English-German test set by\nup to 4.25 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02385v1",
            "date": "2020-08-05 22:21:03+00:00",
            "title": "Efficient MDI Adaptation for n-gram Language Models",
            "authors": [
                "Ruizhe Huang",
                "Ke Li",
                "Ashish Arora",
                "Dan Povey",
                "Sanjeev Khudanpur"
            ],
            "abstract": "This paper presents an efficient algorithm for n-gram language model\nadaptation under the minimum discrimination information (MDI) principle, where\nan out-of-domain language model is adapted to satisfy the constraints of\nmarginal probabilities of the in-domain data. The challenge for MDI language\nmodel adaptation is its computational complexity. By taking advantage of the\nbackoff structure of n-gram model and the idea of hierarchical training method,\noriginally proposed for maximum entropy (ME) language models, we show that MDI\nadaptation can be computed in linear-time complexity to the inputs in each\niteration. The complexity remains the same as ME models, although MDI is more\ngeneral than ME. This makes MDI adaptation practical for large corpus and\nvocabulary. Experimental results confirm the scalability of our algorithm on\nvery large datasets, while MDI adaptation gets slightly worse perplexity but\nbetter word error rate results compared to simple linear interpolation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02496v3",
            "date": "2020-08-06 07:43:19+00:00",
            "title": "ConvBERT: Improving BERT with Span-based Dynamic Convolution",
            "authors": [
                "Zihang Jiang",
                "Weihao Yu",
                "Daquan Zhou",
                "Yunpeng Chen",
                "Jiashi Feng",
                "Shuicheng Yan"
            ],
            "abstract": "Pre-trained language models like BERT and its variants have recently achieved\nimpressive performance in various natural language understanding tasks.\nHowever, BERT heavily relies on the global self-attention block and thus\nsuffers large memory footprint and computation cost. Although all its attention\nheads query on the whole input sequence for generating the attention map from a\nglobal perspective, we observe some heads only need to learn local\ndependencies, which means the existence of computation redundancy. We therefore\npropose a novel span-based dynamic convolution to replace these self-attention\nheads to directly model local dependencies. The novel convolution heads,\ntogether with the rest self-attention heads, form a new mixed attention block\nthat is more efficient at both global and local context learning. We equip BERT\nwith this mixed attention design and build a ConvBERT model. Experiments have\nshown that ConvBERT significantly outperforms BERT and its variants in various\ndownstream tasks, with lower training cost and fewer model parameters.\nRemarkably, ConvBERTbase model achieves 86.4 GLUE score, 0.7 higher than\nELECTRAbase, while using less than 1/4 training cost. Code and pre-trained\nmodels will be released.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02879v1",
            "date": "2020-08-06 21:28:36+00:00",
            "title": "Efficient Neural Query Auto Completion",
            "authors": [
                "Sida Wang",
                "Weiwei Guo",
                "Huiji Gao",
                "Bo Long"
            ],
            "abstract": "Query Auto Completion (QAC), as the starting point of information retrieval\ntasks, is critical to user experience. Generally it has two steps: generating\ncompleted query candidates according to query prefixes, and ranking them based\non extracted features. Three major challenges are observed for a query auto\ncompletion system: (1) QAC has a strict online latency requirement. For each\nkeystroke, results must be returned within tens of milliseconds, which poses a\nsignificant challenge in designing sophisticated language models for it. (2)\nFor unseen queries, generated candidates are of poor quality as contextual\ninformation is not fully utilized. (3) Traditional QAC systems heavily rely on\nhandcrafted features such as the query candidate frequency in search logs,\nlacking sufficient semantic understanding of the candidate.\n  In this paper, we propose an efficient neural QAC system with effective\ncontext modeling to overcome these challenges. On the candidate generation\nside, this system uses as much information as possible in unseen prefixes to\ngenerate relevant candidates, increasing the recall by a large margin. On the\ncandidate ranking side, an unnormalized language model is proposed, which\neffectively captures deep semantics of queries. This approach presents better\nranking performance over state-of-the-art neural ranking methods and reduces\n$\\sim$95\\% latency compared to neural language modeling methods. The empirical\nresults on public datasets show that our model achieves a good balance between\naccuracy and efficiency. This system is served in LinkedIn job search with\nsignificant product impact observed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03020v1",
            "date": "2020-08-07 07:16:40+00:00",
            "title": "A Context-based Disambiguation Model for Sentiment Concepts Using a Bag-of-concepts Approach",
            "authors": [
                "Zeinab Rajabi",
                "MohammadReza Valavi",
                "Maryam Hourali"
            ],
            "abstract": "With the widespread dissemination of user-generated content on different\nsocial networks, and online consumer systems such as Amazon, the quantity of\nopinionated information available on the Internet has been increased. One of\nthe main tasks of the sentiment analysis is to detect polarity within a text.\nThe existing polarity detection methods mainly focus on keywords and their\nnaive frequency counts; however, they less regard the meanings and implicit\ndimensions of the natural concepts. Although background knowledge plays a\ncritical role in determining the polarity of concepts, it has been disregarded\nin polarity detection methods. This study presents a context-based model to\nsolve ambiguous polarity concepts using commonsense knowledge. First, a model\nis presented to generate a source of ambiguous sentiment concepts based on\nSenticNet by computing the probability distribution. Then the model uses a\nbag-of-concepts approach to remove ambiguities and semantic augmentation with\nthe ConceptNet handling to overcome lost knowledge. ConceptNet is a large-scale\nsemantic network with a large number of commonsense concepts. In this paper,\nthe point mutual information (PMI) measure is used to select the contextual\nconcepts having strong relationships with ambiguous concepts. The polarity of\nthe ambiguous concepts is precisely detected using positive/negative contextual\nconcepts and the relationship of the concepts in the semantic knowledge base.\nThe text representation scheme is semantically enriched using Numberbatch,\nwhich is a word embedding model based on the concepts from the ConceptNet\nsemantic network. The proposed model is evaluated by applying a corpus of\nproduct reviews, called Semeval. The experimental results revealed an accuracy\nrate of 82.07%, representing the effectiveness of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03101v1",
            "date": "2020-08-07 12:06:42+00:00",
            "title": "Privacy Guarantees for De-identifying Text Transformations",
            "authors": [
                "David Ifeoluwa Adelani",
                "Ali Davody",
                "Thomas Kleinbauer",
                "Dietrich Klakow"
            ],
            "abstract": "Machine Learning approaches to Natural Language Processing tasks benefit from\na comprehensive collection of real-life user data. At the same time, there is a\nclear need for protecting the privacy of the users whose data is collected and\nprocessed. For text collections, such as, e.g., transcripts of voice\ninteractions or patient records, replacing sensitive parts with benign\nalternatives can provide de-identification. However, how much privacy is\nactually guaranteed by such text transformations, and are the resulting texts\nstill useful for machine learning? In this paper, we derive formal privacy\nguarantees for general text transformation-based de-identification methods on\nthe basis of Differential Privacy. We also measure the effect that different\nways of masking private information in dialog transcripts have on a subsequent\nmachine learning task. To this end, we formulate different masking strategies\nand compare their privacy-utility trade-offs. In particular, we compare a\nsimple redact approach with more sophisticated word-by-word replacement using\ndeep learning models on multiple natural language understanding tasks like\nnamed entity recognition, intent detection, and dialog act classification. We\nfind that only word-by-word replacement is robust against performance drops in\nvarious tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03164v1",
            "date": "2020-08-07 13:16:28+00:00",
            "title": "IMS at SemEval-2020 Task 1: How low can you go? Dimensionality in Lexical Semantic Change Detection",
            "authors": [
                "Jens Kaiser",
                "Dominik Schlechtweg",
                "Sean Papay",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We present the results of our system for SemEval-2020 Task 1 that exploits a\ncommonly used lexical semantic change detection model based on Skip-Gram with\nNegative Sampling. Our system focuses on Vector Initialization (VI) alignment,\ncompares VI to the currently top-ranking models for Subtask 2 and demonstrates\nthat these can be outperformed if we optimize VI dimensionality. We demonstrate\nthat differences in performance can largely be attributed to model-specific\nsources of noise, and we reveal a strong relationship between dimensionality\nand frequency-induced noise in VI alignment. Our results suggest that lexical\nsemantic change models integrating vector space alignment should pay more\nattention to the role of the dimensionality parameter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03232v2",
            "date": "2020-08-07 15:48:58+00:00",
            "title": "Quran Intelligent Ontology Construction Approach Using Association Rules Mining",
            "authors": [
                "Fouzi Harrag",
                "Abdullah Al-Nasser",
                "Abdullah Al-Musnad",
                "Rayan Al-Shaya"
            ],
            "abstract": "Ontology can be seen as a formal representation of knowledge. They have been\ninvestigated in many artificial intelligence studies including semantic web,\nsoftware engineering, and information retrieval. The aim of ontology is to\ndevelop knowledge representations that can be shared and reused. This research\nproject is concerned with the use of association rules to extract the Quran\nontology. The manual acquisition of ontologies from Quran verses can be very\ncostly; therefore, we need an intelligent system for Quran ontology\nconstruction using patternbased schemes and associations rules to discover\nQuran concepts and semantics relations from Quran verses. Our system is based\non the combination of statistics and linguistics methods to extract concepts\nand conceptual relations from Quran. In particular, a linguistic pattern-based\napproach is exploited to extract specific concepts from the Quran, while the\nconceptual relations are found based on association rules technique. The Quran\nontology will offer a new and powerful representation of Quran knowledge, and\nthe association rules will help to represent the relations between all classes\nof connected concepts in the Quran ontology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03277v3",
            "date": "2020-08-07 17:28:32+00:00",
            "title": "Learning a natural-language to LTL executable semantic parser for grounded robotics",
            "authors": [
                "Christopher Wang",
                "Candace Ross",
                "Yen-Ling Kuo",
                "Boris Katz",
                "Andrei Barbu"
            ],
            "abstract": "Children acquire their native language with apparent ease by observing how\nlanguage is used in context and attempting to use it themselves. They do so\nwithout laborious annotations, negative examples, or even direct corrections.\nWe take a step toward robots that can do the same by training a grounded\nsemantic parser, which discovers latent linguistic representations that can be\nused for the execution of natural-language commands. In particular, we focus on\nthe difficult domain of commands with a temporal aspect, whose semantics we\ncapture with Linear Temporal Logic, LTL. Our parser is trained with pairs of\nsentences and executions as well as an executor. At training time, the parser\nhypothesizes a meaning representation for the input as a formula in LTL. Three\ncompeting pressures allow the parser to discover meaning from language. First,\nany hypothesized meaning for a sentence must be permissive enough to reflect\nall the annotated execution trajectories. Second, the executor -- a pretrained\nend-to-end LTL planner -- must find that the observe trajectories are likely\nexecutions of the meaning. Finally, a generator, which reconstructs the\noriginal input, encourages the model to find representations that conserve\nknowledge about the command. Together these ensure that the meaning is neither\ntoo general nor too specific. Our model generalizes well, being able to parse\nand execute both machine-generated and human-generated commands, with\nnear-equal accuracy, despite the fact that the human-generated sentences are\nmuch more varied and complex with an open lexicon. The approach presented here\nis not specific to LTL: it can be applied to any domain where sentence meanings\ncan be hypothesized and an executor can verify these meanings, thus opening the\ndoor to many applications for robotic agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03340v1",
            "date": "2020-08-07 19:11:51+00:00",
            "title": "Retrofitting Vector Representations of Adverse Event Reporting Data to Structured Knowledge to Improve Pharmacovigilance Signal Detection",
            "authors": [
                "Xiruo Ding",
                "Trevor Cohen"
            ],
            "abstract": "Adverse drug events (ADE) are prevalent and costly. Clinical trials are\nconstrained in their ability to identify potential ADEs, motivating the\ndevelopment of spontaneous reporting systems for post-market surveillance.\nStatistical methods provide a convenient way to detect signals from these\nreports but have limitations in leveraging relationships between drugs and ADEs\ngiven their discrete count-based nature. A previously proposed method, aer2vec,\ngenerates distributed vector representations of ADE report entities that\ncapture patterns of similarity but cannot utilize lexical knowledge. We address\nthis limitation by retrofitting aer2vec drug embeddings to knowledge from\nRxNorm and developing a novel retrofitting variant using vector rescaling to\npreserve magnitude. When evaluated in the context of a pharmacovigilance signal\ndetection task, aer2vec with retrofitting consistently outperforms\ndisproportionality metrics when trained on minimally preprocessed data.\nRetrofitting with rescaling results in further improvements in the larger and\nmore challenging of two pharmacovigilance reference sets used for evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03417v1",
            "date": "2020-08-08 02:15:25+00:00",
            "title": "Point or Generate Dialogue State Tracker",
            "authors": [
                "Song Xiaohui",
                "Hu Songlin"
            ],
            "abstract": "Dialogue state tracking is a key part of a task-oriented dialogue system,\nwhich estimates the user's goal at each turn of the dialogue. In this paper, we\npropose the Point-Or-Generate Dialogue State Tracker (POGD). POGD solves the\ndialogue state tracking task in two perspectives: 1) point out explicitly\nexpressed slot values from the user's utterance, and 2) generate implicitly\nexpressed ones based on slot-specific contexts. It also shares parameters\nacross all slots, which achieves knowledge sharing and gains scalability to\nlarge-scale across-domain dialogues. Moreover, the training process of its\nsubmodules is formulated as a multi-task learning procedure to further promote\nits capability of generalization. Experiments show that POGD not only obtains\nstate-of-the-art results on both WoZ 2.0 and MultiWoZ 2.0 datasets but also has\ngood generalization on unseen values and new slots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03736v1",
            "date": "2020-08-09 14:38:48+00:00",
            "title": "Fast and Accurate Neural CRF Constituency Parsing",
            "authors": [
                "Yu Zhang",
                "Houquan Zhou",
                "Zhenghua Li"
            ],
            "abstract": "Estimating probability distribution is one of the core issues in the NLP\nfield. However, in both deep learning (DL) and pre-DL eras, unlike the vast\napplications of linear-chain CRF in sequence labeling tasks, very few works\nhave applied tree-structure CRF to constituency parsing, mainly due to the\ncomplexity and inefficiency of the inside-outside algorithm. This work presents\na fast and accurate neural CRF constituency parser. The key idea is to batchify\nthe inside algorithm for loss computation by direct large tensor operations on\nGPU, and meanwhile avoid the outside algorithm for gradient computation via\nefficient back-propagation. We also propose a simple two-stage\nbracketing-then-labeling parsing approach to improve efficiency further. To\nimprove the parsing performance, inspired by recent progress in dependency\nparsing, we introduce a new scoring architecture based on boundary\nrepresentation and biaffine attention, and a beneficial dropout strategy.\nExperiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF parser\nachieves new state-of-the-art performance on both settings of w/o and w/ BERT,\nand can parse over 1,000 sentences per second. We release our code at\nhttps://github.com/yzhangcs/crfpar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03945v2",
            "date": "2020-08-10 08:12:34+00:00",
            "title": "Does BERT Solve Commonsense Task via Commonsense Knowledge?",
            "authors": [
                "Leyang Cui",
                "Sijie Cheng",
                "Yu Wu",
                "Yue Zhang"
            ],
            "abstract": "BERT has been used for solving commonsense tasks such as CommonsenseQA. While\nprior research has found that BERT does contain commonsense information to some\nextent, there has been work showing that pre-trained models can rely on\nspurious associations (e.g., data bias) rather than key cues in solving\nsentiment classification and other problems. We quantitatively investigate the\npresence of structural commonsense cues in BERT when solving commonsense tasks,\nand the importance of such cues for the model prediction. Using two different\nmeasures, we find that BERT does use relevant knowledge for solving the task,\nand the presence of commonsense knowledge is positively correlated to the model\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03946v1",
            "date": "2020-08-10 08:12:49+00:00",
            "title": "A Large-Scale Chinese Short-Text Conversation Dataset",
            "authors": [
                "Yida Wang",
                "Pei Ke",
                "Yinhe Zheng",
                "Kaili Huang",
                "Yong Jiang",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "The advancements of neural dialogue generation models show promising results\non modeling short-text conversations. However, training such models usually\nneeds a large-scale high-quality dialogue corpus, which is hard to access. In\nthis paper, we present a large-scale cleaned Chinese conversation dataset,\nLCCC, which contains a base version (6.8million dialogues) and a large version\n(12.0 million dialogues). The quality of our dataset is ensured by a rigorous\ndata cleaning pipeline, which is built based on a set of rules and a classifier\nthat is trained on manually annotated 110K dialogue pairs. We also release\npre-training dialogue models which are trained on LCCC-base and LCCC-large\nrespectively. The cleaned dataset and the pre-training models will facilitate\nthe research of short-text conversation modeling. All the models and datasets\nare available at https://github.com/thu-coai/CDial-GPT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03979v2",
            "date": "2020-08-10 09:26:00+00:00",
            "title": "KR-BERT: A Small-Scale Korean-Specific Language Model",
            "authors": [
                "Sangah Lee",
                "Hansol Jang",
                "Yunmee Baik",
                "Suzi Park",
                "Hyopil Shin"
            ],
            "abstract": "Since the appearance of BERT, recent works including XLNet and RoBERTa\nutilize sentence embedding models pre-trained by large corpora and a large\nnumber of parameters. Because such models have large hardware and a huge amount\nof data, they take a long time to pre-train. Therefore it is important to\nattempt to make smaller models that perform comparatively. In this paper, we\ntrained a Korean-specific model KR-BERT, utilizing a smaller vocabulary and\ndataset. Since Korean is one of the morphologically rich languages with poor\nresources using non-Latin alphabets, it is also important to capture\nlanguage-specific linguistic phenomena that the Multilingual BERT model missed.\nWe tested several tokenizers including our BidirectionalWordPiece Tokenizer and\nadjusted the minimal span of tokens for tokenization ranging from sub-character\nlevel to character-level to construct a better vocabulary for our model. With\nthose adjustments, our KR-BERT model performed comparably and even better than\nother existing pre-trained models using a corpus about 1/10 of the size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04276v1",
            "date": "2020-08-10 17:17:21+00:00",
            "title": "A Bootstrapped Model to Detect Abuse and Intent in White Supremacist Corpora",
            "authors": [
                "B. Simons",
                "D. B. Skillicorn"
            ],
            "abstract": "Intelligence analysts face a difficult problem: distinguishing extremist\nrhetoric from potential extremist violence. Many are content to express abuse\nagainst some target group, but only a few indicate a willingness to engage in\nviolence. We address this problem by building a predictive model for intent,\nbootstrapping from a seed set of intent words, and language templates\nexpressing intent. We design both an n-gram and attention-based deep learner\nfor intent and use them as colearners to improve both the basis for prediction\nand the predictions themselves. They converge to stable predictions in a few\nrounds. We merge predictions of intent with predictions of abusive language to\ndetect posts that indicate a desire for violent action. We validate the\npredictions by comparing them to crowd-sourced labelling. The methodology can\nbe applied to other linguistic properties for which a plausible starting point\ncan be defined.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04277v1",
            "date": "2020-08-10 17:17:52+00:00",
            "title": "SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets",
            "authors": [
                "Parth Patwa",
                "Gustavo Aguilar",
                "Sudipta Kar",
                "Suraj Pandey",
                "Srinivas PYKL",
                "Bj\u00f6rn Gamb\u00e4ck",
                "Tanmoy Chakraborty",
                "Thamar Solorio",
                "Amitava Das"
            ],
            "abstract": "In this paper, we present the results of the SemEval-2020 Task 9 on Sentiment\nAnalysis of Code-Mixed Tweets (SentiMix 2020). We also release and describe our\nHinglish (Hindi-English) and Spanglish (Spanish-English) corpora annotated with\nword-level language identification and sentence-level sentiment labels. These\ncorpora are comprised of 20K and 19K examples, respectively. The sentiment\nlabels are - Positive, Negative, and Neutral. SentiMix attracted 89 submissions\nin total including 61 teams that participated in the Hinglish contest and 28\nsubmitted systems to the Spanglish competition. The best performance achieved\nwas 75.0% F1 score for Hinglish and 80.6% F1 for Spanglish. We observe that\nBERT-like models and ensemble methods are the most common and successful\napproaches among the participants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04550v2",
            "date": "2020-08-11 06:50:23+00:00",
            "title": "A Parallel Evaluation Data Set of Software Documentation with Document Structure Annotation",
            "authors": [
                "Bianka Buschbeck",
                "Miriam Exel"
            ],
            "abstract": "This paper accompanies the software documentation data set for machine\ntranslation, a parallel evaluation data set of data originating from the SAP\nHelp Portal, that we released to the machine translation community for research\npurposes. It offers the possibility to tune and evaluate machine translation\nsystems in the domain of corporate software documentation and contributes to\nthe availability of a wider range of evaluation scenarios. The data set\ncomprises of the language pairs English to Hindi, Indonesian, Malay and Thai,\nand thus also increases the test coverage for the many low-resource language\npairs. Unlike most evaluation data sets that consist of plain parallel text,\nthe segments in this data set come with additional metadata that describes\nstructural information of the document context. We provide insights into the\norigin and creation, the particularities and characteristics of the data set as\nwell as machine translation results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04702v1",
            "date": "2020-08-11 13:54:11+00:00",
            "title": "A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings",
            "authors": [
                "Lixing Zhu",
                "Yulan He",
                "Deyu Zhou"
            ],
            "abstract": "We propose a novel generative model to explore both local and global context\nfor joint learning topics and topic-specific word embeddings. In particular, we\nassume that global latent topics are shared across documents, a word is\ngenerated by a hidden semantic vector encoding its contextual semantic meaning,\nand its context words are generated conditional on both the hidden semantic\nvector and global latent topics. Topics are trained jointly with the word\nembeddings. The trained model maps words to topic-dependent embeddings, which\nnaturally addresses the issue of word polysemy. Experimental results show that\nthe proposed model outperforms the word-level embedding methods in both word\nsimilarity evaluation and word sense disambiguation. Furthermore, the model\nalso extracts more coherent topics compared with existing neural topic models\nor other models for joint learning of topics and word embeddings. Finally, the\nmodel can be easily integrated with existing deep contextualized word embedding\nlearning methods to further improve the performance of downstream tasks such as\nsentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04759v1",
            "date": "2020-08-11 15:01:52+00:00",
            "title": "Hybrid Ranking Network for Text-to-SQL",
            "authors": [
                "Qin Lyu",
                "Kaushik Chakrabarti",
                "Shobhit Hathi",
                "Souvik Kundu",
                "Jianwen Zhang",
                "Zheng Chen"
            ],
            "abstract": "In this paper, we study how to leverage pre-trained language models in\nText-to-SQL. We argue that previous approaches under utilize the base language\nmodels by concatenating all columns together with the NL question and feeding\nthem into the base language model in the encoding stage. We propose a neat\napproach called Hybrid Ranking Network (HydraNet) which breaks down the problem\ninto column-wise ranking and decoding and finally assembles the column-wise\noutputs into a SQL query by straightforward rules. In this approach, the\nencoder is given a NL question and one individual column, which perfectly\naligns with the original tasks BERT/RoBERTa is trained on, and hence we avoid\nany ad-hoc pooling or additional encoding layers which are necessary in prior\napproaches. Experiments on the WikiSQL dataset show that the proposed approach\nis very effective, achieving the top place on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04860v2",
            "date": "2020-08-11 17:05:13+00:00",
            "title": "Revisiting Low Resource Status of Indian Languages in Machine Translation",
            "authors": [
                "Jerin Philip",
                "Shashank Siripragada",
                "Vinay P. Namboodiri",
                "C. V. Jawahar"
            ],
            "abstract": "Indian language machine translation performance is hampered due to the lack\nof large scale multi-lingual sentence aligned corpora and robust benchmarks.\nThrough this paper, we provide and analyse an automated framework to obtain\nsuch a corpus for Indian language neural machine translation (NMT) systems. Our\npipeline consists of a baseline NMT system, a retrieval module, and an\nalignment module that is used to work with publicly available websites such as\npress releases by the government. The main contribution towards this effort is\nto obtain an incremental method that uses the above pipeline to iteratively\nimprove the size of the corpus as well as improve each of the components of our\nsystem. Through our work, we also evaluate the design choices such as the\nchoice of pivoting language and the effect of iterative incremental increase in\ncorpus size. Our work in addition to providing an automated framework also\nresults in generating a relatively larger corpus as compared to existing\ncorpora that are available for Indian languages. This corpus helps us obtain\nsubstantially improved results on the publicly available WAT evaluation\nbenchmark and other standard evaluation benchmarks.",
            "journal ref": "8th ACM IKDD CODS and 26th COMAD (CODS COMAD 2021), January 2--4,\n  2021, Bangalore, India",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04885v1",
            "date": "2020-08-11 17:42:26+00:00",
            "title": "The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020",
            "authors": [
                "Tobias Domhan",
                "Michael Denkowski",
                "David Vilar",
                "Xing Niu",
                "Felix Hieber",
                "Kenneth Heafield"
            ],
            "abstract": "We present Sockeye 2, a modernized and streamlined version of the Sockeye\nneural machine translation (NMT) toolkit. New features include a simplified\ncode base through the use of MXNet's Gluon API, a focus on state of the art\nmodel architectures, distributed mixed precision training, and efficient CPU\ndecoding with 8-bit quantization. These improvements result in faster training\nand inference, higher automatic metric scores, and a shorter path from research\nto production.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04935v2",
            "date": "2020-08-11 18:05:34+00:00",
            "title": "Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity",
            "authors": [
                "Brian Thompson",
                "Matt Post"
            ],
            "abstract": "Recent work has shown that a multilingual neural machine translation (NMT)\nmodel can be used to judge how well a sentence paraphrases another sentence in\nthe same language (Thompson and Post, 2020); however, attempting to generate\nparaphrases from such a model using standard beam search produces trivial\ncopies or near copies. We introduce a simple paraphrase generation algorithm\nwhich discourages the production of n-grams that are present in the input. Our\napproach enables paraphrase generation in many languages from a single\nmultilingual NMT model. Furthermore, the amount of lexical diversity between\nthe input and output can be controlled at generation time. We conduct a human\nevaluation to compare our method to a paraphraser trained on the large English\nsynthetic paraphrase database ParaBank 2 (Hu et al., 2019c) and find that our\nmethod produces paraphrases that better preserve meaning and are more\ngramatical, for the same level of lexical diversity. Additional smaller human\nassessments demonstrate our approach also works in two non-English languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05055v1",
            "date": "2020-08-12 01:16:45+00:00",
            "title": "The Annotation Guideline of LST20 Corpus",
            "authors": [
                "Prachya Boonkwan",
                "Vorapon Luantangsrisuk",
                "Sitthaa Phaholphinyo",
                "Kanyanat Kriengket",
                "Dhanon Leenoi",
                "Charun Phrombut",
                "Monthika Boriboon",
                "Krit Kosawat",
                "Thepchai Supnithi"
            ],
            "abstract": "This report presents the annotation guideline for LST20, a large-scale corpus\nwith multiple layers of linguistic annotation for Thai language processing. Our\nguideline consists of five layers of linguistic annotation: word segmentation,\nPOS tagging, named entities, clause boundaries, and sentence boundaries. The\ndataset complies to the CoNLL-2003-style format for ease of use. LST20 Corpus\noffers five layers of linguistic annotation as aforementioned. At a large\nscale, it consists of 3,164,864 words, 288,020 named entities, 248,962 clauses,\nand 74,180 sentences, while it is annotated with 16 distinct POS tags. All\n3,745 documents are also annotated with 15 news genres. Regarding its sheer\nsize, this dataset is considered large enough for developing joint neural\nmodels for NLP. With the existence of this publicly available corpus, Thai has\nbecome a linguistically rich language for the first time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05122v1",
            "date": "2020-08-12 06:07:44+00:00",
            "title": "The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models",
            "authors": [
                "Ian Tenney",
                "James Wexler",
                "Jasmijn Bastings",
                "Tolga Bolukbasi",
                "Andy Coenen",
                "Sebastian Gehrmann",
                "Ellen Jiang",
                "Mahima Pushkarna",
                "Carey Radebaugh",
                "Emily Reif",
                "Ann Yuan"
            ],
            "abstract": "We present the Language Interpretability Tool (LIT), an open-source platform\nfor visualization and understanding of NLP models. We focus on core questions\nabout model behavior: Why did my model make this prediction? When does it\nperform poorly? What happens under a controlled change in the input? LIT\nintegrates local explanations, aggregate analysis, and counterfactual\ngeneration into a streamlined, browser-based interface to enable rapid\nexploration and error analysis. We include case studies for a diverse set of\nworkflows, including exploring counterfactuals for sentiment analysis,\nmeasuring gender bias in coreference systems, and exploring local behavior in\ntext generation. LIT supports a wide range of models--including classification,\nseq2seq, and structured prediction--and is highly extensible through a\ndeclarative, framework-agnostic API. LIT is under active development, with code\nand full documentation available at https://github.com/pair-code/lit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05179v1",
            "date": "2020-08-12 08:50:09+00:00",
            "title": "Modeling Inter-Aspect Dependencies with a Non-temporal Mechanism for Aspect-Based Sentiment Analysis",
            "authors": [
                "Yunlong Liang",
                "Fandong Meng",
                "Jinchao Zhang",
                "Yufeng Chen",
                "Jinan Xu",
                "Jie Zhou"
            ],
            "abstract": "For multiple aspects scenario of aspect-based sentiment analysis (ABSA),\nexisting approaches typically ignore inter-aspect relations or rely on temporal\ndependencies to process aspect-aware representations of all aspects in a\nsentence. Although multiple aspects of a sentence appear in a non-adjacent\nsequential order, they are not in a strict temporal relationship as natural\nlanguage sequence, thus the aspect-aware sentence representations should not be\ntreated as temporal dependency processing. In this paper, we propose a novel\nnon-temporal mechanism to enhance the ABSA task through modeling inter-aspect\ndependencies. Furthermore, we focus on the well-known class imbalance issue on\nthe ABSA task and address it by down-weighting the loss assigned to\nwell-classified instances. Experiments on two distinct domains of SemEval 2014\ntask 4 demonstrate the effectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05348v2",
            "date": "2020-08-12 14:40:51+00:00",
            "title": "Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task",
            "authors": [
                "Pinzhen Chen",
                "Kenneth Heafield"
            ],
            "abstract": "Supervised Chinese word segmentation has entered the deep learning era which\nreduces the hassle of feature engineering. Recently, some researchers attempted\nto treat it as character-level translation which further simplified model\ndesigning and building, but there is still a performance gap between the\ntranslation-based approach and other methods. In this work, we apply the best\npractices from low-resource neural machine translation to Chinese word\nsegmentation. We build encoder-decoder models with attention, and examine a\nseries of techniques including regularization, data augmentation, objective\nweighting, transfer learning and ensembling. Our method is generic for word\nsegmentation, without the need for feature engineering or model implementation.\nIn the closed test with constrained data, our method ties with the state of the\nart on the MSR dataset and is comparable to other methods on the PKU dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05658v1",
            "date": "2020-08-13 02:59:52+00:00",
            "title": "Cognitive Representation Learning of Self-Media Online Article Quality",
            "authors": [
                "Yiru Wang",
                "Shen Huang",
                "Gongfu Li",
                "Qiang Deng",
                "Dongliang Liao",
                "Pengda Si",
                "Yujiu Yang",
                "Jin Xu"
            ],
            "abstract": "The automatic quality assessment of self-media online articles is an urgent\nand new issue, which is of great value to the online recommendation and search.\nDifferent from traditional and well-formed articles, self-media online articles\nare mainly created by users, which have the appearance characteristics of\ndifferent text levels and multi-modal hybrid editing, along with the potential\ncharacteristics of diverse content, different styles, large semantic spans and\ngood interactive experience requirements. To solve these challenges, we\nestablish a joint model CoQAN in combination with the layout organization,\nwriting characteristics and text semantics, designing different representation\nlearning subnetworks, especially for the feature learning process and\ninteractive reading habits on mobile terminals. It is more consistent with the\ncognitive style of expressing an expert's evaluation of articles. We have also\nconstructed a large scale real-world assessment dataset. Extensive experimental\nresults show that the proposed framework significantly outperforms\nstate-of-the-art methods, and effectively learns and integrates different\nfactors of the online article quality assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05666v1",
            "date": "2020-08-13 03:14:25+00:00",
            "title": "Dialogue State Induction Using Neural Latent Variable Models",
            "authors": [
                "Qingkai Min",
                "Libo Qin",
                "Zhiyang Teng",
                "Xiao Liu",
                "Yue Zhang"
            ],
            "abstract": "Dialogue state modules are a useful component in a task-oriented dialogue\nsystem. Traditional methods find dialogue states by manually labeling training\ncorpora, upon which neural models are trained. However, the labeling process\ncan be costly, slow, error-prone, and more importantly, cannot cover the vast\nrange of domains in real-world dialogues for customer service. We propose the\ntask of dialogue state induction, building two neural latent variable models\nthat mine dialogue states automatically from unlabeled customer service\ndialogue records. Results show that the models can effectively find meaningful\nslots. In addition, equipped with induced dialogue states, a state-of-the-art\ndialogue system gives better performance compared with not using a dialogue\nstate module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06408v1",
            "date": "2020-08-13 16:07:00+00:00",
            "title": "ANDES at SemEval-2020 Task 12: A jointly-trained BERT multilingual model for offensive language detection",
            "authors": [
                "Juan Manuel P\u00e9rez",
                "Aym\u00e9 Arango",
                "Franco Luque"
            ],
            "abstract": "This paper describes our participation in SemEval-2020 Task 12: Multilingual\nOffensive Language Detection. We jointly-trained a single model by fine-tuning\nMultilingual BERT to tackle the task across all the proposed languages:\nEnglish, Danish, Turkish, Greek and Arabic. Our single model had competitive\nresults, with a performance close to top-performing systems in spite of sharing\nthe same parameters across all languages. Zero-shot and few-shot experiments\nwere also conducted to analyze the transference performance among these\nlanguages. We make our code public for further research",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06606v3",
            "date": "2020-08-14 23:44:11+00:00",
            "title": "Quantification of BERT Diagnosis Generalizability Across Medical Specialties Using Semantic Dataset Distance",
            "authors": [
                "Mihir P. Khambete",
                "William Su",
                "Juan Garcia",
                "Marcus A. Badgeley"
            ],
            "abstract": "Deep learning models in healthcare may fail to generalize on data from unseen\ncorpora. Additionally, no quantitative metric exists to tell how existing\nmodels will perform on new data. Previous studies demonstrated that NLP models\nof medical notes generalize variably between institutions, but ignored other\nlevels of healthcare organization. We measured SciBERT diagnosis sentiment\nclassifier generalizability between medical specialties using EHR sentences\nfrom MIMIC-III. Models trained on one specialty performed better on internal\ntest sets than mixed or external test sets (mean AUCs 0.92, 0.87, and 0.83,\nrespectively; p = 0.016). When models are trained on more specialties, they\nhave better test performances (p < 1e-4). Model performance on new corpora is\ndirectly correlated to the similarity between train and test sentence content\n(p < 1e-4). Future studies should assess additional axes of generalization to\nensure deep learning models fulfil their intended purpose across institutions,\nspecialties, and practices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06695v1",
            "date": "2020-08-15 10:34:27+00:00",
            "title": "Label-Wise Document Pre-Training for Multi-Label Text Classification",
            "authors": [
                "Han Liu",
                "Caixia Yuan",
                "Xiaojie Wang"
            ],
            "abstract": "A major challenge of multi-label text classification (MLTC) is to\nstimulatingly exploit possible label differences and label correlations. In\nthis paper, we tackle this challenge by developing Label-Wise Pre-Training\n(LW-PT) method to get a document representation with label-aware information.\nThe basic idea is that, a multi-label document can be represented as a\ncombination of multiple label-wise representations, and that, correlated labels\nalways cooccur in the same or similar documents. LW-PT implements this idea by\nconstructing label-wise document classification tasks and trains label-wise\ndocument encoders. Finally, the pre-trained label-wise encoder is fine-tuned\nwith the downstream MLTC task. Extensive experimental results validate that the\nproposed method has significant advantages over the previous state-of-the-art\nmodels and is able to discover reasonable label relationship. The code is\nreleased to facilitate other researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06788v2",
            "date": "2020-08-15 21:03:36+00:00",
            "title": "Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation",
            "authors": [
                "Goran Glava\u0161",
                "Ivan Vuli\u0107"
            ],
            "abstract": "Traditional NLP has long held (supervised) syntactic parsing necessary for\nsuccessful higher-level semantic language understanding (LU). The recent advent\nof end-to-end neural models, self-supervised via language modeling (LM), and\ntheir success on a wide range of LU tasks, however, questions this belief. In\nthis work, we empirically investigate the usefulness of supervised parsing for\nsemantic LU in the context of LM-pretrained transformer networks. Relying on\nthe established fine-tuning paradigm, we first couple a pretrained transformer\nwith a biaffine parsing head, aiming to infuse explicit syntactic knowledge\nfrom Universal Dependencies treebanks into the transformer. We then fine-tune\nthe model for LU tasks and measure the effect of the intermediate parsing\ntraining (IPT) on downstream LU task performance. Results from both monolingual\nEnglish and zero-shot language transfer experiments (with intermediate\ntarget-language parsing) show that explicit formalized syntax, injected into\ntransformers through IPT, has very limited and inconsistent effect on\ndownstream LU performance. Our results, coupled with our analysis of\ntransformers' representation spaces before and after intermediate parsing, make\na significant step towards providing answers to an essential question: how\n(un)availing is supervised parsing for high-level semantic natural language\nunderstanding in the era of large neural models?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06865v1",
            "date": "2020-08-16 09:28:37+00:00",
            "title": "Discovering Lexical Similarity Through Articulatory Feature-based Phonetic Edit Distance",
            "authors": [
                "Tafseer Ahmed",
                "Muhammad Suffian Nizami",
                "Muhammad Yaseen Khan"
            ],
            "abstract": "Lexical Similarity (LS) between two languages uncovers many interesting\nlinguistic insights such as genetic relationship, mutual intelligibility, and\nthe usage of one's vocabulary into other. There are various methods through\nwhich LS is evaluated. In the same regard, this paper presents a method of\nPhonetic Edit Distance (PED) that uses a soft comparison of letters using the\narticulatory features associated with them. The system converts the words into\nthe corresponding International Phonetic Alphabet (IPA), followed by the\nconversion of IPA into its set of articulatory features. Later, the lists of\nthe set of articulatory features are compared using the proposed method. As an\nexample, PED gives edit distance of German word vater and Persian word pidar as\n0.82; and similarly, Hebrew word shalom and Arabic word salaam as 0.93, whereas\nfor a juxtapose comparison, their IPA based edit distances are 4 and 2\nrespectively. Experiments are performed with six languages (Arabic, Hindi,\nMarathi, Persian, Sanskrit, and Urdu). In this regard, we extracted part of\nspeech wise word-lists from the Universal Dependency corpora and evaluated the\nLS for every pair of language. Thus, with the proposed approach, we find the\ngenetic affinity, similarity, and borrowing/loan-words despite having script\ndifferences and sound variation phenomena among these languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06914v1",
            "date": "2020-08-16 14:13:32+00:00",
            "title": "DCR-Net: A Deep Co-Interactive Relation Network for Joint Dialog Act Recognition and Sentiment Classification",
            "authors": [
                "Libo Qin",
                "Wanxiang Che",
                "Yangming Li",
                "Minheng Ni",
                "Ting Liu"
            ],
            "abstract": "In dialog system, dialog act recognition and sentiment classification are two\ncorrelative tasks to capture speakers intentions, where dialog act and\nsentiment can indicate the explicit and the implicit intentions separately.\nMost of the existing systems either treat them as separate tasks or just\njointly model the two tasks by sharing parameters in an implicit way without\nexplicitly modeling mutual interaction and relation. To address this problem,\nwe propose a Deep Co-Interactive Relation Network (DCR-Net) to explicitly\nconsider the cross-impact and model the interaction between the two tasks by\nintroducing a co-interactive relation layer. In addition, the proposed relation\nlayer can be stacked to gradually capture mutual knowledge with multiple steps\nof interaction. Especially, we thoroughly study different relation layers and\ntheir effects. Experimental results on two public datasets (Mastodon and\nDailydialog) show that our model outperforms the state-of-the-art joint model\nby 4.3% and 3.4% in terms of F1 score on dialog act recognition task, 5.7% and\n12.4% on sentiment classification respectively. Comprehensive analysis\nempirically verifies the effectiveness of explicitly modeling the relation\nbetween the two tasks and the multi-steps interaction mechanism. Finally, we\nemploy the Bidirectional Encoder Representation from Transformer (BERT) in our\nframework, which can further boost our performance in both tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07027v1",
            "date": "2020-08-16 23:19:30+00:00",
            "title": "Adding Recurrence to Pretrained Transformers for Improved Efficiency and Context Size",
            "authors": [
                "Davis Yoshida",
                "Allyson Ettinger",
                "Kevin Gimpel"
            ],
            "abstract": "Fine-tuning a pretrained transformer for a downstream task has become a\nstandard method in NLP in the last few years. While the results from these\nmodels are impressive, applying them can be extremely computationally\nexpensive, as is pretraining new models with the latest architectures. We\npresent a novel method for applying pretrained transformer language models\nwhich lowers their memory requirement both at training and inference time. An\nadditional benefit is that our method removes the fixed context size constraint\nthat most transformer models have, allowing for more flexible use. When applied\nto the GPT-2 language model, we find that our method attains better perplexity\nthan an unmodified GPT-2 model on the PG-19 and WikiText-103 corpora, for a\ngiven amount of computation or memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07138v1",
            "date": "2020-08-17 08:04:11+00:00",
            "title": "Logical Semantics, Dialogical Argumentation, and Textual Entailment",
            "authors": [
                "Davide Catta",
                "Richard Moot",
                "Christian Retor\u00e9"
            ],
            "abstract": "In this chapter, we introduce a new dialogical system for first order\nclassical logic which is close to natural language argumentation, and we prove\nits completeness with respect to usual classical validity. We combine our\ndialogical system with the Grail syntactic and semantic parser developed by the\nsecond author in order to address automated textual entailment, that is, we use\nit for deciding whether or not a sentence is a consequence of a short text.\nThis work-which connects natural language semantics and argumentation with\ndialogical logic-can be viewed as a step towards an inferentialist view of\nnatural language semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07189v1",
            "date": "2020-08-17 10:07:23+00:00",
            "title": "Comparison of Syntactic Parsers on Biomedical Texts",
            "authors": [
                "Maria Biryukov"
            ],
            "abstract": "Syntactic parsing is an important step in the automated text analysis which\naims at information extraction. Quality of the syntactic parsing determines to\na large extent the recall and precision of the text mining results. In this\npaper we evaluate the performance of several popular syntactic parsers in\napplication to the biomedical text mining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07259v2",
            "date": "2020-08-17 12:45:39+00:00",
            "title": "BUT-FIT at SemEval-2020 Task 4: Multilingual commonsense",
            "authors": [
                "Josef Jon",
                "Martin Faj\u010d\u00edk",
                "Martin Do\u010dekal",
                "Pavel Smr\u017e"
            ],
            "abstract": "This paper describes work of the BUT-FIT's team at SemEval 2020 Task 4 -\nCommonsense Validation and Explanation. We participated in all three subtasks.\nIn subtasks A and B, our submissions are based on pretrained language\nrepresentation models (namely ALBERT) and data augmentation. We experimented\nwith solving the task for another language, Czech, by means of multilingual\nmodels and machine translated dataset, or translated model inputs. We show that\nwith a strong machine translation system, our system can be used in another\nlanguage with a small accuracy loss. In subtask C, our submission, which is\nbased on pretrained sequence-to-sequence model (BART), ranked 1st in BLEU score\nranking, however, we show that the correlation between BLEU and human\nevaluation, in which our submission ended up 4th, is low. We analyse the\nmetrics used in the evaluation and we propose an additional score based on\nmodel from subtask B, which correlates well with our manual ranking, as well as\nreranking method based on the same principle. We performed an error and dataset\nanalysis for all subtasks and we present our findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07347v2",
            "date": "2020-08-17 14:16:15+00:00",
            "title": "HunFlair: An Easy-to-Use Tool for State-of-the-Art Biomedical Named Entity Recognition",
            "authors": [
                "Leon Weber",
                "Mario S\u00e4nger",
                "Jannes M\u00fcnchmeyer",
                "Maryam Habibi",
                "Ulf Leser",
                "Alan Akbik"
            ],
            "abstract": "Summary: Named Entity Recognition (NER) is an important step in biomedical\ninformation extraction pipelines. Tools for NER should be easy to use, cover\nmultiple entity types, highly accurate, and robust towards variations in text\ngenre and style. To this end, we propose HunFlair, an NER tagger covering\nmultiple entity types integrated into the widely used NLP framework Flair.\nHunFlair outperforms other state-of-the-art standalone NER tools with an\naverage gain of 7.26 pp over the next best tool, can be installed with a single\ncommand and is applied with only four lines of code. Availability: HunFlair is\nfreely available through the Flair framework under an MIT license:\nhttps://github.com/flairNLP/flair and is compatible with all major operating\nsystems. Contact:{weberple,saengema,alan.akbik}@informatik.hu-berlin.de",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07466v1",
            "date": "2020-08-17 16:45:50+00:00",
            "title": "Narrative Interpolation for Generating and Understanding Stories",
            "authors": [
                "Su Wang",
                "Greg Durrett",
                "Katrin Erk"
            ],
            "abstract": "We propose a method for controlled narrative/story generation where we are\nable to guide the model to produce coherent narratives with user-specified\ntarget endings by interpolation: for example, we are told that Jim went hiking\nand at the end Jim needed to be rescued, and we want the model to incrementally\ngenerate steps along the way. The core of our method is an interpolation model\nbased on GPT-2 which conditions on a previous sentence and a next sentence in a\nnarrative and fills in the gap. Additionally, a reranker helps control for\ncoherence of the generated text. With human evaluation, we show that\nending-guided generation results in narratives which are coherent, faithful to\nthe given ending guide, and require less manual effort on the part of the human\nguide writer than past approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07481v1",
            "date": "2020-08-17 17:16:08+00:00",
            "title": "Emotion Carrier Recognition from Personal Narratives",
            "authors": [
                "Aniruddha Tammewar",
                "Alessandra Cervone",
                "Giuseppe Riccardi"
            ],
            "abstract": "Personal Narratives (PN) - recollections of facts, events, and thoughts from\none's own experience - are often used in everyday conversations. So far, PNs\nhave mainly been explored for tasks such as valence prediction or emotion\nclassification (i.e. happy, sad). However, these tasks might overlook more\nfine-grained information that could nevertheless prove relevant for\nunderstanding PNs. In this work, we propose a novel task for Narrative\nUnderstanding: Emotion Carrier Recognition (ECR). We argue that automatic\nrecognition of emotion carriers, the text fragments that carry the emotions of\nthe narrator (i.e. 'loss of a grandpa', 'high school reunion'), from PNs,\nprovides a deeper level of emotion analysis needed, for instance, in the mental\nhealthcare domain. In this work, we explore the task of ECR using a corpus of\nPNs manually annotated with emotion carriers and investigate different baseline\nmodels for the task. Furthermore, we propose several evaluation strategies for\nthe task. Based on the inter-annotator agreement, the task in itself was found\nto be complex and subjective for humans. Nevertheless, we discuss evaluation\nmetrics that could be suitable for applications based on ECR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07605v1",
            "date": "2020-08-17 20:22:56+00:00",
            "title": "Stock Index Prediction with Multi-task Learning and Word Polarity Over Time",
            "authors": [
                "Yue Zhou",
                "Kerstin Voigt"
            ],
            "abstract": "Sentiment-based stock prediction systems aim to explore sentiment or event\nsignals from online corpora and attempt to relate the signals to stock price\nvariations. Both the feature-based and neural-networks-based approaches have\ndelivered promising results. However, the frequently minor fluctuations of the\nstock prices restrict learning the sentiment of text from price patterns, and\nlearning market sentiment from text can be biased if the text is irrelevant to\nthe underlying market. In addition, when using discrete word features, the\npolarity of a certain term can change over time according to different events.\nTo address these issues, we propose a two-stage system that consists of a\nsentiment extractor to extract the opinion on the market trend and a summarizer\nthat predicts the direction of the index movement of following week given the\nopinions of the news over the current week. We adopt BERT with multitask\nlearning which additionally predicts the worthiness of the news and propose a\nmetric called Polarity-Over-Time to extract the word polarity among different\nevent periods. A Weekly-Monday prediction framework and a new dataset, the\n10-year Reuters financial news dataset, are also proposed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07723v1",
            "date": "2020-08-18 03:34:09+00:00",
            "title": "NASE: Learning Knowledge Graph Embedding for Link Prediction via Neural Architecture Search",
            "authors": [
                "Xiaoyu Kou",
                "Bingfeng Luo",
                "Huang Hu",
                "Yan Zhang"
            ],
            "abstract": "Link prediction is the task of predicting missing connections between\nentities in the knowledge graph (KG). While various forms of models are\nproposed for the link prediction task, most of them are designed based on a few\nknown relation patterns in several well-known datasets. Due to the diversity\nand complexity nature of the real-world KGs, it is inherently difficult to\ndesign a model that fits all datasets well. To address this issue, previous\nwork has tried to use Automated Machine Learning (AutoML) to search for the\nbest model for a given dataset. However, their search space is limited only to\nbilinear model families. In this paper, we propose a novel Neural Architecture\nSearch (NAS) framework for the link prediction task. First, the embeddings of\nthe input triplet are refined by the Representation Search Module. Then, the\nprediction score is searched within the Score Function Search Module. This\nframework entails a more general search space, which enables us to take\nadvantage of several mainstream model families, and thus it can potentially\nachieve better performance. We relax the search space to be continuous so that\nthe architecture can be optimized efficiently using gradient-based search\nstrategies. Experimental results on several benchmark datasets demonstrate the\neffectiveness of our method compared with several state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07772v2",
            "date": "2020-08-18 07:14:54+00:00",
            "title": "Very Deep Transformers for Neural Machine Translation",
            "authors": [
                "Xiaodong Liu",
                "Kevin Duh",
                "Liyuan Liu",
                "Jianfeng Gao"
            ],
            "abstract": "We explore the application of very deep Transformer models for Neural Machine\nTranslation (NMT). Using a simple yet effective initialization technique that\nstabilizes training, we show that it is feasible to build standard\nTransformer-based models with up to 60 encoder layers and 12 decoder layers.\nThese deep models outperform their baseline 6-layer counterparts by as much as\n2.5 BLEU, and achieve new state-of-the-art benchmark results on WMT14\nEnglish-French (43.8 BLEU and 46.4 BLEU with back-translation) and WMT14\nEnglish-German (30.1 BLEU).The code and trained models will be publicly\navailable at: https://github.com/namisan/exdeep-nmt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07905v3",
            "date": "2020-08-18 13:04:03+00:00",
            "title": "Glancing Transformer for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Lihua Qian",
                "Hao Zhou",
                "Yu Bao",
                "Mingxuan Wang",
                "Lin Qiu",
                "Weinan Zhang",
                "Yong Yu",
                "Lei Li"
            ],
            "abstract": "Recent work on non-autoregressive neural machine translation (NAT) aims at\nimproving the efficiency by parallel decoding without sacrificing the quality.\nHowever, existing NAT methods are either inferior to Transformer or require\nmultiple decoding passes, leading to reduced speedup. We propose the Glancing\nLanguage Model (GLM), a method to learn word interdependency for single-pass\nparallel generation models. With GLM, we develop Glancing Transformer (GLAT)\nfor machine translation. With only single-pass parallel decoding, GLAT is able\nto generate high-quality translation with 8-15 times speedup. Experiments on\nmultiple WMT language directions show that GLAT outperforms all previous single\npass non-autoregressive methods, and is nearly comparable to Transformer,\nreducing the gap to 0.25-0.9 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08225v2",
            "date": "2020-08-19 02:18:53+00:00",
            "title": "Victim or Perpetrator? Analysis of Violent Characters Portrayals from Movie Scripts",
            "authors": [
                "Victor R Martinez",
                "Krishna Somandepalli",
                "Karan Singla",
                "Anil Ramanakrishna",
                "Yalda T. Uhls",
                "Shrikanth Narayanan"
            ],
            "abstract": "Violent content in the media can influence viewers' perception of the\nsociety. For example, frequent depictions of certain demographics as victims or\nperpetrators of violence can shape stereotyped attitudes. We propose that\ncomputational methods can aid in the large-scale analysis of violence in\nmovies. The method we develop characterizes aspects of violent content solely\nfrom the language used in the scripts. Thus, our method is applicable to a\nmovie in the earlier stages of content creation even before it is produced.\nThis is complementary to previous works which rely on audio or video post\nproduction. In this work, we identify stereotypes in character roles (i.e.,\nvictim, perpetrator and narrator) based on the demographics of the actor casted\nfor that role. Our results highlight two significant differences in the\nfrequency of portrayals as well as the demographics of the interaction between\nvictims and perpetrators : (1) female characters appear more often as victims,\nand (2) perpetrators are more likely to be White if the victim is Black or\nLatino. To date, we are the first to show that language used in movie scripts\nis a strong indicator of violent content, and that there are systematic\nportrayals of certain demographics as victims and perpetrators in a large\ndataset. This offers novel computational tools to assist in creating awareness\nof representations in storytelling",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08315v1",
            "date": "2020-08-19 07:58:16+00:00",
            "title": "FinChat: Corpus and evaluation setup for Finnish chat conversations on everyday topics",
            "authors": [
                "Katri Leino",
                "Juho Leinonen",
                "Mittul Singh",
                "Sami Virpioja",
                "Mikko Kurimo"
            ],
            "abstract": "Creating open-domain chatbots requires large amounts of conversational data\nand related benchmark tasks to evaluate them. Standardized evaluation tasks are\ncrucial for creating automatic evaluation metrics for model development;\notherwise, comparing the models would require resource-expensive human\nevaluation. While chatbot challenges have recently managed to provide a\nplethora of such resources for English, resources in other languages are not\nyet available. In this work, we provide a starting point for Finnish\nopen-domain chatbot research. We describe our collection efforts to create the\nFinnish chat conversation corpus FinChat, which is made available publicly.\nFinChat includes unscripted conversations on seven topics from people of\ndifferent ages. Using this corpus, we also construct a retrieval-based\nevaluation task for Finnish chatbot development. We observe that off-the-shelf\nchatbot models trained on conversational corpora do not perform better than\nchance at choosing the right answer based on automatic metrics, while humans\ncan do the same task almost perfectly. Similarly, in a human evaluation,\nresponses to questions from the evaluation set generated by the chatbots are\npredominantly marked as incoherent. Thus, FinChat provides a challenging\nevaluation set, meant to encourage chatbot development in Finnish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08439v1",
            "date": "2020-08-19 13:46:37+00:00",
            "title": "BabelEnconding at SemEval-2020 Task 3: Contextual Similarity as a Combination of Multilingualism and Language Models",
            "authors": [
                "Lucas R. C. Pessutto",
                "Tiago de Melo",
                "Viviane P. Moreira",
                "Altigran da Silva"
            ],
            "abstract": "This paper describes the system submitted by our team (BabelEnconding) to\nSemEval-2020 Task 3: Predicting the Graded Effect of Context in Word\nSimilarity. We propose an approach that relies on translation and multilingual\nlanguage models in order to compute the contextual similarity between pairs of\nwords. Our hypothesis is that evidence from additional languages can leverage\nthe correlation with the human generated scores. BabelEnconding was applied to\nboth subtasks and ranked among the top-3 in six out of eight task/language\ncombinations and was the highest scoring system three times.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08547v1",
            "date": "2020-08-19 16:47:15+00:00",
            "title": "UoB at SemEval-2020 Task 12: Boosting BERT with Corpus Level Information",
            "authors": [
                "Wah Meng Lim",
                "Harish Tayyar Madabushi"
            ],
            "abstract": "Pre-trained language model word representation, such as BERT, have been\nextremely successful in several Natural Language Processing tasks significantly\nimproving on the state-of-the-art. This can largely be attributed to their\nability to better capture semantic information contained within a sentence.\nSeveral tasks, however, can benefit from information available at a corpus\nlevel, such as Term Frequency-Inverse Document Frequency (TF-IDF). In this work\nwe test the effectiveness of integrating this information with BERT on the task\nof identifying abuse on social media and show that integrating this information\nwith BERT does indeed significantly improve performance. We participate in\nSub-Task A (abuse detection) wherein we achieve a score within two points of\nthe top performing team and in Sub-Task B (target detection) wherein we are\nranked 4 of the 44 participating teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08612v2",
            "date": "2020-08-19 18:12:33+00:00",
            "title": "A Survey on Text Simplification",
            "authors": [
                "Punardeep Sikka",
                "Vijay Mago"
            ],
            "abstract": "Text Simplification (TS) aims to reduce the linguistic complexity of content\nto make it easier to understand. Research in TS has been of keen interest,\nespecially as approaches to TS have shifted from manual, hand-crafted rules to\nautomated simplification. This survey seeks to provide a comprehensive overview\nof TS, including a brief description of earlier approaches used, discussion of\nvarious aspects of simplification (lexical, semantic and syntactic), and latest\ntechniques being utilized in the field. We note that the research in the field\nhas clearly shifted towards utilizing deep learning techniques to perform TS,\nwith a specific focus on developing solutions to combat the lack of data\navailable for simplification. We also include a discussion of datasets and\nevaluations metrics commonly used, along with discussion of related fields\nwithin Natural Language Processing (NLP), like semantic similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08769v1",
            "date": "2020-08-20 04:31:03+00:00",
            "title": "Lite Training Strategies for Portuguese-English and English-Portuguese Translation",
            "authors": [
                "Alexandre Lopes",
                "Rodrigo Nogueira",
                "Roberto Lotufo",
                "Helio Pedrini"
            ],
            "abstract": "Despite the widespread adoption of deep learning for machine translation, it\nis still expensive to develop high-quality translation models. In this work, we\ninvestigate the use of pre-trained models, such as T5 for Portuguese-English\nand English-Portuguese translation tasks using low-cost hardware. We explore\nthe use of Portuguese and English pre-trained language models and propose an\nadaptation of the English tokenizer to represent Portuguese characters, such as\ndiaeresis, acute and grave accents. We compare our models to the Google\nTranslate API and MarianMT on a subset of the ParaCrawl dataset, as well as to\nthe winning submission to the WMT19 Biomedical Translation Shared Task. We also\ndescribe our submission to the WMT20 Biomedical Translation Shared Task. Our\nresults show that our models have a competitive performance to state-of-the-art\nmodels while being trained on modest hardware (a single 8GB gaming GPU for nine\ndays). Our data, models and code are available at\nhttps://github.com/unicamp-dl/Lite-T5-Translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08810v4",
            "date": "2020-08-20 07:29:14+00:00",
            "title": "An Experimental Study of Deep Neural Network Models for Vietnamese Multiple-Choice Reading Comprehension",
            "authors": [
                "Son T. Luu",
                "Kiet Van Nguyen",
                "Anh Gia-Tuan Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Machine reading comprehension (MRC) is a challenging task in natural language\nprocessing that makes computers understanding natural language texts and answer\nquestions based on those texts. There are many techniques for solving this\nproblems, and word representation is a very important technique that impact\nmost to the accuracy of machine reading comprehension problem in the popular\nlanguages like English and Chinese. However, few studies on MRC have been\nconducted in low-resource languages such as Vietnamese. In this paper, we\nconduct several experiments on neural network-based model to understand the\nimpact of word representation to the Vietnamese multiple-choice machine reading\ncomprehension. Our experiments include using the Co-match model on six\ndifferent Vietnamese word embeddings and the BERT model for multiple-choice\nreading comprehension. On the ViMMRC corpus, the accuracy of BERT model is\n61.28% on test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08854v1",
            "date": "2020-08-20 09:30:05+00:00",
            "title": "Checkworthiness in Automatic Claim Detection Models: Definitions and Analysis of Datasets",
            "authors": [
                "Liesbeth Allein",
                "Marie-Francine Moens"
            ],
            "abstract": "Public, professional and academic interest in automated fact-checking has\ndrastically increased over the past decade, with many aiming to automate one of\nthe first steps in a fact-check procedure: the selection of so-called\ncheckworthy claims. However, there is little agreement on the definition and\ncharacteristics of checkworthiness among fact-checkers, which is consequently\nreflected in the datasets used for training and testing checkworthy claim\ndetection models. After elaborate analysis of checkworthy claim selection\nprocedures in fact-check organisations and analysis of state-of-the-art claim\ndetection datasets, checkworthiness is defined as the concept of having a\nspatiotemporal and context-dependent worth and need to have the correctness of\nthe objectivity it conveys verified. This is irrespective of the claim's\nperceived veracity judgement by an individual based on prior knowledge and\nbeliefs. Concerning the characteristics of current datasets, it is argued that\nthe data is not only highly imbalanced and noisy, but also too limited in scope\nand language. Furthermore, we believe that the subjective concept of\ncheckworthiness might not be a suitable filter for claim detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08896v3",
            "date": "2020-08-20 11:25:26+00:00",
            "title": "Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR",
            "authors": [
                "Juri Opitz",
                "Anette Frank"
            ],
            "abstract": "Systems that generate natural language text from abstract meaning\nrepresentations such as AMR are typically evaluated using automatic surface\nmatching metrics that compare the generated texts to reference texts from which\nthe input meaning representations were constructed. We show that besides\nwell-known issues from which such metrics suffer, an additional problem arises\nwhen applying these metrics for AMR-to-text evaluation, since an abstract\nmeaning representation allows for numerous surface realizations. In this work\nwe aim to alleviate these issues by proposing $\\mathcal{M}\\mathcal{F}_\\beta$, a\ndecomposable metric that builds on two pillars. The first is the principle of\nmeaning preservation $\\mathcal{M}$: it measures to what extent a given AMR can\nbe reconstructed from the generated sentence using SOTA AMR parsers and\napplying (fine-grained) AMR evaluation metrics to measure the distance between\nthe original and the reconstructed AMR. The second pillar builds on a principle\nof (grammatical) form $\\mathcal{F}$ that measures the linguistic quality of the\ngenerated text, which we implement using SOTA language models. In two extensive\npilot studies we show that fulfillment of both principles offers benefits for\nAMR-to-text evaluation, including explainability of scores. Since\n$\\mathcal{M}\\mathcal{F}_\\beta$ does not necessarily rely on gold AMRs, it may\nextend to other text generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08995v2",
            "date": "2020-08-20 14:30:33+00:00",
            "title": "AutoKG: Constructing Virtual Knowledge Graphs from Unstructured Documents for Question Answering",
            "authors": [
                "Seunghak Yu",
                "Tianxing He",
                "James Glass"
            ],
            "abstract": "Knowledge graphs (KGs) have the advantage of providing fine-grained detail\nfor question-answering systems. Unfortunately, building a reliable KG is\ntime-consuming and expensive as it requires human intervention. To overcome\nthis issue, we propose a novel framework to automatically construct a KG from\nunstructured documents that does not require external alignment. We first\nextract surface-form knowledge tuples from unstructured documents and encode\nthem with contextual information. Entities with similar context semantics are\nthen linked through internal alignment to form a graph structure. This allows\nus to extract the desired information from multiple documents by traversing the\ngenerated KG without a manual process. We examine its performance in retrieval\nbased QA systems by reformulating the WikiMovies and MetaQA datasets into a\ntuple-level retrieval task. The experimental results show that our method\noutperforms traditional retrieval methods by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09036v2",
            "date": "2020-08-20 15:39:36+00:00",
            "title": "Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries",
            "authors": [
                "Benjamin Heinzerling",
                "Kentaro Inui"
            ],
            "abstract": "Pretrained language models have been suggested as a possible alternative or\ncomplement to structured knowledge bases. However, this emerging LM-as-KB\nparadigm has so far only been considered in a very limited setting, which only\nallows handling 21k entities whose single-token name is found in common LM\nvocabularies. Furthermore, the main benefit of this paradigm, namely querying\nthe KB using a variety of natural language paraphrases, is underexplored so\nfar. Here, we formulate two basic requirements for treating LMs as KBs: (i) the\nability to store a large number facts involving a large number of entities and\n(ii) the ability to query stored facts. We explore three entity representations\nthat allow LMs to represent millions of entities and present a detailed case\nstudy on paraphrased querying of world knowledge in LMs, thereby providing a\nproof-of-concept that language models can indeed serve as knowledge bases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09084v2",
            "date": "2020-08-20 17:17:38+00:00",
            "title": "Do Syntax Trees Help Pre-trained Transformers Extract Information?",
            "authors": [
                "Devendra Singh Sachan",
                "Yuhao Zhang",
                "Peng Qi",
                "William Hamilton"
            ],
            "abstract": "Much recent work suggests that incorporating syntax information from\ndependency trees can improve task-specific transformer models. However, the\neffect of incorporating dependency tree information into pre-trained\ntransformer models (e.g., BERT) remains unclear, especially given recent\nstudies highlighting how these models implicitly encode syntax. In this work,\nwe systematically study the utility of incorporating dependency trees into\npre-trained transformers on three representative information extraction tasks:\nsemantic role labeling (SRL), named entity recognition, and relation\nextraction.\n  We propose and investigate two distinct strategies for incorporating\ndependency structure: a late fusion approach, which applies a graph neural\nnetwork on the output of a transformer, and a joint fusion approach, which\ninfuses syntax structure into the transformer attention layers. These\nstrategies are representative of prior work, but we introduce additional model\ndesign elements that are necessary for obtaining improved performance. Our\nempirical analysis demonstrates that these syntax-infused transformers obtain\nstate-of-the-art results on SRL and relation extraction tasks. However, our\nanalysis also reveals a critical shortcoming of these models: we find that\ntheir performance gains are highly contingent on the availability of\nhuman-annotated dependency parses, which raises important questions regarding\nthe viability of syntax-augmented transformers in real-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09094v2",
            "date": "2020-08-20 17:34:15+00:00",
            "title": "Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes",
            "authors": [
                "Nicholas Lourie",
                "Ronan Le Bras",
                "Yejin Choi"
            ],
            "abstract": "As AI systems become an increasing part of people's everyday lives, it\nbecomes ever more important that they understand people's ethical norms.\nMotivated by descriptive ethics, a field of study that focuses on people's\ndescriptive judgments rather than theoretical prescriptions on morality, we\ninvestigate a novel, data-driven approach to machine ethics.\n  We introduce Scruples, the first large-scale dataset with 625,000 ethical\njudgments over 32,000 real-life anecdotes. Each anecdote recounts a complex\nethical situation, often posing moral dilemmas, paired with a distribution of\njudgments contributed by the community members. Our dataset presents a major\nchallenge to state-of-the-art neural language models, leaving significant room\nfor improvement. However, when presented with simplified moral situations, the\nresults are considerably more promising, suggesting that neural models can\neffectively learn simpler ethical building blocks.\n  A key take-away of our empirical analysis is that norms are not always\nclean-cut; many situations are naturally divisive. We present a new method to\nestimate the best possible performance on such tasks with inherently diverse\nlabel distributions, and explore likelihood functions that separate intrinsic\nfrom model uncertainty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09112v1",
            "date": "2020-08-20 17:58:56+00:00",
            "title": "Inducing Language-Agnostic Multilingual Representations",
            "authors": [
                "Wei Zhao",
                "Steffen Eger",
                "Johannes Bjerva",
                "Isabelle Augenstein"
            ],
            "abstract": "Multilingual representations have the potential to make cross-lingual systems\navailable to the vast majority of languages in the world. However, they\ncurrently require large pretraining corpora, or assume access to typologically\nsimilar languages. In this work, we address these obstacles by removing\nlanguage identity signals from multilingual embeddings. We examine three\napproaches for this: 1) re-aligning the vector spaces of target languages (all\ntogether) to a pivot source language; 2) removing languages-specific means and\nvariances, which yields better discriminativeness of embeddings as a\nby-product; and 3) normalizing input texts by removing morphological\ncontractions and sentence reordering, thus yielding language-agnostic\nrepresentations. We evaluate on the tasks of XNLI and reference-free MT\nevaluation of varying difficulty across 19 selected languages. Our experiments\ndemonstrate the language-agnostic behavior of our multilingual representations,\nwhich manifest the potential of zero-shot cross-lingual transfer to distant and\nlow-resource languages, and decrease the performance gap by 8.9 points (M-BERT)\nand 18.2 points (XLM-R) on average across all tasks and languages. We make our\ncodes and models available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09144v2",
            "date": "2020-08-20 18:10:13+00:00",
            "title": "PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data",
            "authors": [
                "Diedre Carmo",
                "Marcos Piau",
                "Israel Campiotti",
                "Rodrigo Nogueira",
                "Roberto Lotufo"
            ],
            "abstract": "In natural language processing (NLP), there is a need for more resources in\nPortuguese, since much of the data used in the state-of-the-art research is in\nother languages. In this paper, we pretrain a T5 model on the BrWac corpus, an\nextensive collection of web pages in Portuguese, and evaluate its performance\nagainst other Portuguese pretrained models and multilingual models on three\ndifferent tasks. We show that our Portuguese pretrained models have\nsignificantly better performance over the original T5 models. Moreover, we\ndemonstrate the positive impact of using a Portuguese vocabulary. Our code and\nmodels are available at https://github.com/unicamp-dl/PTT5.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09236v1",
            "date": "2020-08-21 00:05:08+00:00",
            "title": "Spatial Language Representation with Multi-Level Geocoding",
            "authors": [
                "Sayali Kulkarni",
                "Shailee Jain",
                "Mohammad Javad Hosseini",
                "Jason Baldridge",
                "Eugene Ie",
                "Li Zhang"
            ],
            "abstract": "We present a multi-level geocoding model (MLG) that learns to associate texts\nto geographic locations. The Earth's surface is represented using space-filling\ncurves that decompose the sphere into a hierarchy of similarly sized,\nnon-overlapping cells. MLG balances generalization and accuracy by combining\nlosses across multiple levels and predicting cells at each level\nsimultaneously. Without using any dataset-specific tuning, we show that MLG\nobtains state-of-the-art results for toponym resolution on three English\ndatasets. Furthermore, it obtains large gains without any knowledge base\nmetadata, demonstrating that it can effectively learn the connection between\ntext spans and coordinates - and thus can be extended to toponymns not present\nin knowledge bases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09249v2",
            "date": "2020-08-21 01:07:36+00:00",
            "title": "GRIT: Generative Role-filler Transformers for Document-level Event Entity Extraction",
            "authors": [
                "Xinya Du",
                "Alexander M. Rush",
                "Claire Cardie"
            ],
            "abstract": "We revisit the classic problem of document-level role-filler entity\nextraction (REE) for template filling. We argue that sentence-level approaches\nare ill-suited to the task and introduce a generative transformer-based\nencoder-decoder framework (GRIT) that is designed to model context at the\ndocument level: it can make extraction decisions across sentence boundaries; is\nimplicitly aware of noun phrase coreference structure, and has the capacity to\nrespect cross-role dependencies in the template structure. We evaluate our\napproach on the MUC-4 dataset, and show that our model performs substantially\nbetter than prior work. We also show that our modeling choices contribute to\nmodel performance, e.g., by implicitly capturing linguistic knowledge such as\nrecognizing coreferent entity mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09266v1",
            "date": "2020-08-21 02:12:32+00:00",
            "title": "Adapting Event Extractors to Medical Data: Bridging the Covariate Shift",
            "authors": [
                "Aakanksha Naik",
                "Jill Lehman",
                "Carolyn Rose"
            ],
            "abstract": "We tackle the task of adapting event extractors to new domains without\nlabeled data, by aligning the marginal distributions of source and target\ndomains. As a testbed, we create two new event extraction datasets using\nEnglish texts from two medical domains: (i) clinical notes, and (ii)\ndoctor-patient conversations. We test the efficacy of three marginal alignment\ntechniques: (i) adversarial domain adaptation (ADA), (ii) domain adaptive\nfine-tuning (DAFT), and (iii) a novel instance weighting technique based on\nlanguage model likelihood scores (LIW). LIW and DAFT improve over a no-transfer\nBERT baseline on both domains, but ADA only improves on clinical notes. Deeper\nanalysis of performance under different types of shifts (e.g., lexical shift,\nsemantic shift) reveals interesting variations among models. Our\nbest-performing models reach F1 scores of 70.0 and 72.9 on notes and\nconversations respectively, using no labeled data from target domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09290v2",
            "date": "2020-08-21 03:31:50+00:00",
            "title": "Don't Change Me! User-Controllable Selective Paraphrase Generation",
            "authors": [
                "Mohan Zhang",
                "Luchen Tan",
                "Zhengkai Tu",
                "Zihang Fu",
                "Kun Xiong",
                "Ming Li",
                "Jimmy Lin"
            ],
            "abstract": "In the paraphrase generation task, source sentences often contain phrases\nthat should not be altered. Which phrases, however, can be context dependent\nand can vary by application. Our solution to this challenge is to provide the\nuser with explicit tags that can be placed around any arbitrary segment of text\nto mean \"don't change me!\" when generating a paraphrase; the model learns to\nexplicitly copy these phrases to the output. The contribution of this work is a\nnovel data generation technique using distant supervision that allows us to\nstart with a pretrained sequence-to-sequence model and fine-tune a paraphrase\ngenerator that exhibits this behavior, allowing user-controllable paraphrase\ngeneration. Additionally, we modify the loss during fine-tuning to explicitly\nencourage diversity in model output. Our technique is language agnostic, and we\nreport experiments in English and Chinese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09394v1",
            "date": "2020-08-21 09:52:35+00:00",
            "title": "A Variational Approach to Unsupervised Sentiment Analysis",
            "authors": [
                "Ziqian Zeng",
                "Wenxuan Zhou",
                "Xin Liu",
                "Zizheng Lin",
                "Yangqin Song",
                "Michael David Kuo",
                "Wan Hang Keith Chiu"
            ],
            "abstract": "In this paper, we propose a variational approach to unsupervised sentiment\nanalysis. Instead of using ground truth provided by domain experts, we use\ntarget-opinion word pairs as a supervision signal. For example, in a document\nsnippet \"the room is big,\" (room, big) is a target-opinion word pair. These\nword pairs can be extracted by using dependency parsers and simple rules. Our\nobjective function is to predict an opinion word given a target word while our\nultimate goal is to learn a sentiment classifier. By introducing a latent\nvariable, i.e., the sentiment polarity, to the objective function, we can\ninject the sentiment classifier to the objective function via the evidence\nlower bound. We can learn a sentiment classifier by optimizing the lower bound.\nWe also impose sophisticated constraints on opinion words as regularization\nwhich encourages that if two documents have similar (dissimilar) opinion words,\nthe sentiment classifiers should produce similar (different) probability\ndistribution. We apply our method to sentiment analysis on customer reviews and\nclinical narratives. The experiment results show our method can outperform\nunsupervised baselines in sentiment analysis task on both domains, and our\nmethod obtains comparable results to the supervised method with hundreds of\nlabels per aspect in customer reviews domain, and obtains comparable results to\nsupervised methods in clinical narratives domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09513v1",
            "date": "2020-08-21 14:42:33+00:00",
            "title": "Keywords lie far from the mean of all words in local vector space",
            "authors": [
                "Eirini Papagiannopoulou",
                "Grigorios Tsoumakas",
                "Apostolos N. Papadopoulos"
            ],
            "abstract": "Keyword extraction is an important document process that aims at finding a\nsmall set of terms that concisely describe a document's topics. The most\npopular state-of-the-art unsupervised approaches belong to the family of the\ngraph-based methods that build a graph-of-words and use various centrality\nmeasures to score the nodes (candidate keywords). In this work, we follow a\ndifferent path to detect the keywords from a text document by modeling the main\ndistribution of the document's words using local word vector representations.\nThen, we rank the candidates based on their position in the text and the\ndistance between the corresponding local vectors and the main distribution's\ncenter. We confirm the high performance of our approach compared to strong\nbaselines and state-of-the-art unsupervised keyword extraction methods, through\nan extended experimental study, investigating the properties of the local\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09703v1",
            "date": "2020-08-21 22:35:57+00:00",
            "title": "Team DoNotDistribute at SemEval-2020 Task 11: Features, Finetuning, and Data Augmentation in Neural Models for Propaganda Detection in News Articles",
            "authors": [
                "Michael Kranzlein",
                "Shabnam Behzad",
                "Nazli Goharian"
            ],
            "abstract": "This paper presents our systems for SemEval 2020 Shared Task 11: Detection of\nPropaganda Techniques in News Articles. We participate in both the span\nidentification and technique classification subtasks and report on experiments\nusing different BERT-based models along with handcrafted features. Our models\nperform well above the baselines for both tasks, and we contribute ablation\nstudies and discussion of our results to dissect the effectiveness of different\nfeatures and techniques with the goal of aiding future studies in propaganda\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09740v1",
            "date": "2020-08-22 03:02:57+00:00",
            "title": "Applications of BERT Based Sequence Tagging Models on Chinese Medical Text Attributes Extraction",
            "authors": [
                "Gang Zhao",
                "Teng Zhang",
                "Chenxiao Wang",
                "Ping Lv",
                "Ji Wu"
            ],
            "abstract": "We convert the Chinese medical text attributes extraction task into a\nsequence tagging or machine reading comprehension task. Based on BERT\npre-trained models, we have not only tried the widely used LSTM-CRF sequence\ntagging model, but also other sequence models, such as CNN, UCNN, WaveNet,\nSelfAttention, etc, which reaches similar performance as LSTM+CRF. This sheds a\nlight on the traditional sequence tagging models. Since the aspect of emphasis\nfor different sequence tagging models varies substantially, ensembling these\nmodels adds diversity to the final system. By doing so, our system achieves\ngood performance on the task of Chinese medical text attributes extraction\n(subtask 2 of CCKS 2019 task 1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09894v2",
            "date": "2020-08-22 18:18:02+00:00",
            "title": "DUTH at SemEval-2020 Task 11: BERT with Entity Mapping for Propaganda Classification",
            "authors": [
                "Anastasios Bairaktaris",
                "Symeon Symeonidis",
                "Avi Arampatzis"
            ],
            "abstract": "This report describes the methods employed by the Democritus University of\nThrace (DUTH) team for participating in SemEval-2020 Task 11: Detection of\nPropaganda Techniques in News Articles. Our team dealt with Subtask 2:\nTechnique Classification. We used shallow Natural Language Processing (NLP)\npreprocessing techniques to reduce the noise in the dataset, feature selection\nmethods, and common supervised machine learning algorithms. Our final model is\nbased on using the BERT system with entity mapping. To improve our model's\naccuracy, we mapped certain words into five distinct categories by employing\nword-classes and entity recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10129v1",
            "date": "2020-08-23 23:19:17+00:00",
            "title": "Predicting Helpfulness of Online Reviews",
            "authors": [
                "Abdalraheem Alsmadi",
                "Shadi AlZu'bi",
                "Mahmoud Al-Ayyoub",
                "Yaser Jararweh"
            ],
            "abstract": "E-commerce dominates a large part of the world's economy with many websites\ndedicated to online selling products. The vast majority of e-commerce websites\nprovide their customers with the ability to express their opinions about the\nproducts/services they purchase. These feedback in the form of reviews\nrepresent a rich source of information about the users' experiences and level\nof satisfaction, which is of great benefit to both the producer and the\nconsumer. However, not all of these reviews are helpful/useful. The traditional\nway of determining the helpfulness of a review is through the feedback from\nhuman users. However, such a method does not necessarily cover all reviews.\nMoreover, it has many issues like bias, high cost, etc. Thus, there is a need\nto automate this process. This paper presents a set of machine learning (ML)\nmodels to predict the helpfulness online reviews. Mainly, three approaches are\nused: a supervised learning approach (using ML as well as deep learning (DL)\nmodels), a semi-supervised approach (that combines DL models with word\nembeddings), and pre-trained word embedding models that uses transfer learning\n(TL). The latter two approaches are among the unique aspects of this paper as\nthey follow the recent trend of utilizing unlabeled text. The results show that\nthe proposed DL approaches have superiority over the traditional existing ones.\nMoreover, the semi-supervised has a remarkable performance compared with the\nother ones.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10163v1",
            "date": "2020-08-24 02:15:29+00:00",
            "title": "syrapropa at SemEval-2020 Task 11: BERT-based Models Design For Propagandistic Technique and Span Detection",
            "authors": [
                "Jinfen Li",
                "Lu Xiao"
            ],
            "abstract": "This paper describes the BERT-based models proposed for two subtasks in\nSemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. We\nfirst build the model for Span Identification (SI) based on SpanBERT, and\nfacilitate the detection by a deeper model and a sentence-level representation.\nWe then develop a hybrid model for the Technique Classification (TC). The\nhybrid model is composed of three submodels including two BERT models with\ndifferent training methods, and a feature-based Logistic Regression model. We\nendeavor to deal with imbalanced dataset by adjusting cost function. We are in\nthe seventh place in SI subtask (0.4711 of F1-measure), and in the third place\nin TC subtask (0.6783 of F1-measure) on the development set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10284v1",
            "date": "2020-08-24 09:37:45+00:00",
            "title": "Cross-lingual Semantic Role Labeling with Model Transfer",
            "authors": [
                "Hao Fei",
                "Meishan Zhang",
                "Fei Li",
                "Donghong Ji"
            ],
            "abstract": "Prior studies show that cross-lingual semantic role labeling (SRL) can be\nachieved by model transfer under the help of universal features. In this paper,\nwe fill the gap of cross-lingual SRL by proposing an end-to-end SRL model that\nincorporates a variety of universal features and transfer methods. We study\nboth the bilingual transfer and multi-source transfer, under gold or\nmachine-generated syntactic inputs, pre-trained high-order abstract features,\nand contextualized multilingual word representations. Experimental results on\nthe Universal Proposition Bank corpus indicate that performances of the\ncross-lingual SRL can vary by leveraging different cross-lingual features. In\naddition, whether the features are gold-standard also has an impact on\nperformances. Precisely, we find that gold syntax features are much more\ncrucial for cross-lingual SRL, compared with the automatically-generated ones.\nMoreover, universal dependency structure features are able to give the best\nhelp, and both pre-trained high-order features and contextualized word\nrepresentations can further bring significant improvements.",
            "journal ref": "TASLP, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10392v1",
            "date": "2020-08-24 12:43:08+00:00",
            "title": "End to End Dialogue Transformer",
            "authors": [
                "Ond\u0159ej M\u011bkota",
                "Memduh G\u00f6k\u0131rmak",
                "Petr Laitoch"
            ],
            "abstract": "Dialogue systems attempt to facilitate conversations between humans and\ncomputers, for purposes as diverse as small talk to booking a vacation. We are\nhere inspired by the performance of the recurrent neural network-based model\nSequicity, which when conducting a dialogue uses a sequence-to-sequence\narchitecture to first produce a textual representation of what is going on in\nthe dialogue, and in a further step use this along with database findings to\nproduce a reply to the user. We here propose a dialogue system based on the\nTransformer architecture instead of Sequicity's RNN-based architecture, that\nworks similarly in an end-to-end, sequence-to-sequence fashion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10648v2",
            "date": "2020-08-24 18:38:42+00:00",
            "title": "A Baseline Analysis for Podcast Abstractive Summarization",
            "authors": [
                "Chujie Zheng",
                "Harry Jiannan Wang",
                "Kunpeng Zhang",
                "Ling Fan"
            ],
            "abstract": "Podcast summary, an important factor affecting end-users' listening\ndecisions, has often been considered a critical feature in podcast\nrecommendation systems, as well as many downstream applications. Existing\nabstractive summarization approaches are mainly built on fine-tuned models on\nprofessionally edited texts such as CNN and DailyMail news. Different from\nnews, podcasts are often longer, more colloquial and conversational, and\nnoisier with contents on commercials and sponsorship, which makes automatic\npodcast summarization extremely challenging. This paper presents a baseline\nanalysis of podcast summarization using the Spotify Podcast Dataset provided by\nTREC 2020. It aims to help researchers understand current state-of-the-art\npre-trained models and hence build a foundation for creating better models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10762v1",
            "date": "2020-08-25 00:34:28+00:00",
            "title": "Contextualized moral inference",
            "authors": [
                "Jing Yi Xie",
                "Graeme Hirst",
                "Yang Xu"
            ],
            "abstract": "Developing moral awareness in intelligent systems has shifted from a topic of\nphilosophical inquiry to a critical and practical issue in artificial\nintelligence over the past decades. However, automated inference of everyday\nmoral situations remains an under-explored problem. We present a text-based\napproach that predicts people's intuitive judgment of moral vignettes. Our\nmethodology builds on recent work in contextualized language models and textual\ninference of moral sentiment. We show that a contextualized representation\noffers a substantial advantage over alternative representations based on word\nembeddings and emotion sentiment in inferring human moral judgment, evaluated\nand reflected in three independent datasets from moral psychology. We discuss\nthe promise and limitations of our approach toward automated textual moral\nreasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10813v1",
            "date": "2020-08-25 04:41:35+00:00",
            "title": "Conceptualized Representation Learning for Chinese Biomedical Text Mining",
            "authors": [
                "Ningyu Zhang",
                "Qianghuai Jia",
                "Kangping Yin",
                "Liang Dong",
                "Feng Gao",
                "Nengwei Hua"
            ],
            "abstract": "Biomedical text mining is becoming increasingly important as the number of\nbiomedical documents and web data rapidly grows. Recently, word representation\nmodels such as BERT has gained popularity among researchers. However, it is\ndifficult to estimate their performance on datasets containing biomedical texts\nas the word distributions of general and biomedical corpora are quite\ndifferent. Moreover, the medical domain has long-tail concepts and\nterminologies that are difficult to be learned via language models. For the\nChinese biomedical text, it is more difficult due to its complex structure and\nthe variety of phrase combinations. In this paper, we investigate how the\nrecently introduced pre-trained language model BERT can be adapted for Chinese\nbiomedical corpora and propose a novel conceptualized representation learning\napproach. We also release a new Chinese Biomedical Language Understanding\nEvaluation benchmark (\\textbf{ChineseBLUE}). We examine the effectiveness of\nChinese pre-trained models: BERT, BERT-wwm, RoBERTa, and our approach.\nExperimental results on the benchmark show that our approach could bring\nsignificant gain. We release the pre-trained model on GitHub:\nhttps://github.com/alibaba-research/ChineseBLUE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10820v1",
            "date": "2020-08-25 04:58:07+00:00",
            "title": "Simple Unsupervised Similarity-Based Aspect Extraction",
            "authors": [
                "Danny Suarez Vargas",
                "Lucas R. C. Pessutto",
                "Viviane Pereira Moreira"
            ],
            "abstract": "In the context of sentiment analysis, there has been growing interest in\nperforming a finer granularity analysis focusing on the specific aspects of the\nentities being evaluated. This is the goal of Aspect-Based Sentiment Analysis\n(ABSA) which basically involves two tasks: aspect extraction and polarity\ndetection. The first task is responsible for discovering the aspects mentioned\nin the review text and the second task assigns a sentiment orientation\n(positive, negative, or neutral) to that aspect. Currently, the\nstate-of-the-art in ABSA consists of the application of deep learning methods\nsuch as recurrent, convolutional and attention neural networks. The limitation\nof these techniques is that they require a lot of training data and are\ncomputationally expensive. In this paper, we propose a simple approach called\nSUAEx for aspect extraction. SUAEx is unsupervised and relies solely on the\nsimilarity of word embeddings. Experimental results on datasets from three\ndifferent domains have shown that SUAEx achieves results that can outperform\nthe state-of-the-art attention-based approach at a fraction of the time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10856v1",
            "date": "2020-08-25 07:32:09+00:00",
            "title": "TabSim: A Siamese Neural Network for Accurate Estimation of Table Similarity",
            "authors": [
                "Maryam Habibi",
                "Johannes Starlinger",
                "Ulf Leser"
            ],
            "abstract": "Tables are a popular and efficient means of presenting structured\ninformation. They are used extensively in various kinds of documents including\nweb pages. Tables display information as a two-dimensional matrix, the\nsemantics of which is conveyed by a mixture of structure (rows, columns),\nheaders, caption, and content. Recent research has started to consider tables\nas first class objects, not just as an addendum to texts, yielding interesting\nresults for problems like table matching, table completion, or value\nimputation. All of these problems inherently rely on an accurate measure for\nthe semantic similarity of two tables. We present TabSim, a novel method to\ncompute table similarity scores using deep neural networks. Conceptually,\nTabSim represents a table as a learned concatenation of embeddings of its\ncaption, its content, and its structure. Given two tables in this\nrepresentation, a Siamese neural network is trained to compute a score\ncorrelating with the tables' semantic similarity. To train and evaluate our\nmethod, we created a gold standard corpus consisting of 1500 table pairs\nextracted from biomedical articles and manually scored regarding their degree\nof similarity, and adopted two other corpora originally developed for a\ndifferent yet similar task. Our evaluation shows that TabSim outperforms other\ntable similarity measures on average by app. 7% pp F1-score in a binary\nsimilarity classification setting and by app. 1.5% pp in a ranking scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10875v2",
            "date": "2020-08-25 08:22:38+00:00",
            "title": "ETC-NLG: End-to-end Topic-Conditioned Natural Language Generation",
            "authors": [
                "Ginevra Carbone",
                "Gabriele Sarti"
            ],
            "abstract": "Plug-and-play language models (PPLMs) enable topic-conditioned natural\nlanguage generation by pairing large pre-trained generators with attribute\nmodels used to steer the predicted token distribution towards the selected\ntopic. Despite their computational efficiency, PPLMs require large amounts of\nlabeled texts to effectively balance generation fluency and proper\nconditioning, making them unsuitable for low-resource settings. We present\nETC-NLG, an approach leveraging topic modeling annotations to enable\nfully-unsupervised End-to-end Topic-Conditioned Natural Language Generation\nover emergent topics in unlabeled document collections. We first test the\neffectiveness of our approach in a low-resource setting for Italian, evaluating\nthe conditioning for both topic models and gold annotations. We then perform a\ncomparative evaluation of ETC-NLG for Italian and English using a parallel\ncorpus. Finally, we propose an automatic approach to estimate the effectiveness\nof conditioning on the generated utterances.",
            "journal ref": "Italian Journal of Computational Linguistics (IJCoL 2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10889v1",
            "date": "2020-08-25 08:56:40+00:00",
            "title": "Query Understanding via Intent Description Generation",
            "authors": [
                "Ruqing Zhang",
                "Jiafeng Guo",
                "Yixing Fan",
                "Yanyan Lan",
                "Xueqi Cheng"
            ],
            "abstract": "Query understanding is a fundamental problem in information retrieval (IR),\nwhich has attracted continuous attention through the past decades. Many\ndifferent tasks have been proposed for understanding users' search queries,\ne.g., query classification or query clustering. However, it is not that precise\nto understand a search query at the intent class/cluster level due to the loss\nof many detailed information. As we may find in many benchmark datasets, e.g.,\nTREC and SemEval, queries are often associated with a detailed description\nprovided by human annotators which clearly describes its intent to help\nevaluate the relevance of the documents. If a system could automatically\ngenerate a detailed and precise intent description for a search query, like\nhuman annotators, that would indicate much better query understanding has been\nachieved. In this paper, therefore, we propose a novel\nQuery-to-Intent-Description (Q2ID) task for query understanding. Unlike those\nexisting ranking tasks which leverage the query and its description to compute\nthe relevance of documents, Q2ID is a reverse task which aims to generate a\nnatural language intent description based on both relevant and irrelevant\ndocuments of a given query. To address this new task, we propose a novel\nContrastive Generation model, namely CtrsGen for short, to generate the intent\ndescription by contrasting the relevant documents with the irrelevant documents\ngiven a query. We demonstrate the effectiveness of our model by comparing with\nseveral state-of-the-art generation models on the Q2ID task. We discuss the\npotential usage of such Q2ID technique through an example application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10906v1",
            "date": "2020-08-25 09:37:06+00:00",
            "title": "Comparative Computational Analysis of Global Structure in Canonical, Non-Canonical and Non-Literary Texts",
            "authors": [
                "Mahdi Mohseni",
                "Volker Gast",
                "Christoph Redies"
            ],
            "abstract": "This study investigates global properties of literary and non-literary texts.\nWithin the literary texts, a distinction is made between canonical and\nnon-canonical works. The central hypothesis of the study is that the three text\ntypes (non-literary, literary/canonical and literary/non-canonical) exhibit\nsystematic differences with respect to structural design features as correlates\nof aesthetic responses in readers. To investigate these differences, we\ncompiled a corpus containing texts of the three categories of interest, the\nJena Textual Aesthetics Corpus. Two aspects of global structure are\ninvestigated, variability and self-similar (fractal) patterns, which reflect\nlong-range correlations along texts. We use four types of basic observations,\n(i) the frequency of POS-tags per sentence, (ii) sentence length, (iii) lexical\ndiversity in chunks of text, and (iv) the distribution of topic probabilities\nin chunks of texts. These basic observations are grouped into two more general\ncategories, (a) the low-level properties (i) and (ii), which are observed at\nthe level of the sentence (reflecting linguistic decoding), and (b) the\nhigh-level properties (iii) and (iv), which are observed at the textual level\n(reflecting comprehension). The basic observations are transformed into time\nseries, and these time series are subject to multifractal detrended fluctuation\nanalysis (MFDFA). Our results show that low-level properties of texts are\nbetter discriminators than high-level properties, for the three text types\nunder analysis. Canonical literary texts differ from non-canonical ones\nprimarily in terms of variability. Fractality seems to be a universal feature\nof text, more pronounced in non-literary than in literary texts. Beyond the\nspecific results of the study, we intend to open up new perspectives on the\nexperimental study of textual aesthetics.",
            "journal ref": "https://www.frontiersin.org/articles/10.3389/fpsyg.2021.599063/full?&utm_source=Email_to_authors_&utm_medium=Email&utm_content=T1_11.5e1_author&utm_campaign=Email_publication&field=&journalName=Frontiers_in_Psychology&id=599063",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11053v1",
            "date": "2020-08-25 14:27:58+00:00",
            "title": "JokeMeter at SemEval-2020 Task 7: Convolutional humor",
            "authors": [
                "Martin Docekal",
                "Martin Fajcik",
                "Josef Jon",
                "Pavel Smrz"
            ],
            "abstract": "This paper describes our system that was designed for Humor evaluation within\nthe SemEval-2020 Task 7. The system is based on convolutional neural network\narchitecture. We investigate the system on the official dataset, and we provide\nmore insight to model itself to see how the learned inner features look.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11257v1",
            "date": "2020-08-25 20:30:21+00:00",
            "title": "The Impact of Indirect Machine Translation on Sentiment Classification",
            "authors": [
                "Alberto Poncelas",
                "Pintu Lohar",
                "Andy Way",
                "James Hadley"
            ],
            "abstract": "Sentiment classification has been crucial for many natural language\nprocessing (NLP) applications, such as the analysis of movie reviews, tweets,\nor customer feedback. A sufficiently large amount of data is required to build\na robust sentiment classification system. However, such resources are not\nalways available for all domains or for all languages.\n  In this work, we propose employing a machine translation (MT) system to\ntranslate customer feedback into another language to investigate in which cases\ntranslated sentences can have a positive or negative impact on an automatic\nsentiment classifier. Furthermore, as performing a direct translation is not\nalways possible, we explore the performance of automatic classifiers on\nsentences that have been translated using a pivot MT system.\n  We conduct several experiments using the above approaches to analyse the\nperformance of our proposed sentiment classification system and discuss the\nadvantages and drawbacks of classifying translated sentences.",
            "journal ref": "Proceedings of Association for Machine Translation in the\n  Americas, AMTA (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11293v2",
            "date": "2020-08-25 22:22:50+00:00",
            "title": "Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization",
            "authors": [
                "Byron C. Wallace",
                "Sayantan Saha",
                "Frank Soboczenski",
                "Iain J. Marshall"
            ],
            "abstract": "We consider the problem of automatically generating a narrative biomedical\nevidence summary from multiple trial reports. We evaluate modern neural models\nfor abstractive summarization of relevant article abstracts from systematic\nreviews previously conducted by members of the Cochrane collaboration, using\nthe authors conclusions section of the review abstract as our target. We enlist\nmedical professionals to evaluate generated summaries, and we find that modern\nsummarization systems yield consistently fluent and relevant synopses, but that\nthey are not always factual. We propose new approaches that capitalize on\ndomain-specific models to inform summarization, e.g., by explicitly demarcating\nsnippets of inputs that convey key findings, and emphasizing the reports of\nlarge and high-quality trials. We find that these strategies modestly improve\nthe factual accuracy of generated summaries. Finally, we propose a new method\nfor automatically evaluating the factuality of generated narrative evidence\nsyntheses using models that infer the directionality of reported findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11295v1",
            "date": "2020-08-25 22:28:14+00:00",
            "title": "Concept Extraction Using Pointer-Generator Networks",
            "authors": [
                "Alexander Shvets",
                "Leo Wanner"
            ],
            "abstract": "Concept extraction is crucial for a number of downstream applications.\nHowever, surprisingly enough, straightforward single token/nominal\nchunk-concept alignment or dictionary lookup techniques such as DBpedia\nSpotlight still prevail. We propose a generic open-domain OOV-oriented\nextractive model that is based on distant supervision of a pointer-generator\nnetwork leveraging bidirectional LSTMs and a copy mechanism. The model has been\ntrained on a large annotated corpus compiled specifically for this task from\n250K Wikipedia pages, and tested on regular pages, where the pointers to other\npages are considered as ground truth concepts. The outcome of the experiments\nshows that our model significantly outperforms standard techniques and, when\nused on top of DBpedia Spotlight, further improves its performance. The\nexperiments furthermore show that the model can be readily ported to other\ndatasets on which it equally achieves a state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11398v1",
            "date": "2020-08-26 06:30:43+00:00",
            "title": "Decision Tree J48 at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text (Hinglish)",
            "authors": [
                "Gaurav Singh"
            ],
            "abstract": "This paper discusses the design of the system used for providing a solution\nfor the problem given at SemEval-2020 Task 9 where sentiment analysis of\ncode-mixed language Hindi and English needed to be performed. This system uses\nWeka as a tool for providing the classifier for the classification of tweets\nand python is used for loading the data from the files provided and cleaning\nit. Only part of the training data was provided to the system for classifying\nthe tweets in the test data set on which evaluation of the system was done. The\nsystem performance was assessed using the official competition evaluation\nmetric F1-score. Classifier was trained on two sets of training data which\nresulted in F1 scores of 0.4972 and 0.5316.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11488v1",
            "date": "2020-08-26 11:01:13+00:00",
            "title": "Machine learning approach of Japanese composition scoring and writing aided system's design",
            "authors": [
                "Wanhong Huang"
            ],
            "abstract": "Automatic scoring system is extremely complex for any language. Because\nnatural language itself is a complex model. When we evaluate articles generated\nby natural language, we need to view the articles from many dimensions such as\nword features, grammatical features, semantic features, text structure and so\non. Even human beings sometimes can't accurately grade a composition because\ndifferent people have different opinions about the same article. But a\ncomposition scoring system can greatly assist language learners. It can make\nlanguage leaner improve themselves in the process of output something. Though\nit is still difficult for machines to directly evaluate a composition at the\nsemantic and pragmatic levels, especially for Japanese, Chinese and other\nlanguage in high context cultures, we can make machine evaluate a passage in\nword and grammar levels, which can as an assistance of composition rater or\nlanguage learner. Especially for foreign language learners, lexical and\nsyntactic content are usually what they are more concerned about. In our\nexperiments, we did the follows works: 1) We use word segmentation tools and\ndictionaries to achieve word segmentation of an article, and extract word\nfeatures, as well as generate a words' complexity feature of an article. And\nBow technique are used to extract the theme features. 2) We designed a\nTuring-complete automata model and create 300+ automatons for the grammars that\nappear in the JLPT examination. And extract grammars features by using these\nautomatons. 3) We propose a statistical approach for scoring a specify theme of\ncomposition, the final score will depend on all the writings that submitted to\nthe system. 4) We design an grammar hint function for language leaner, so that\nthey can know currently what grammars they can use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11584v2",
            "date": "2020-08-26 14:42:14+00:00",
            "title": "Inno at SemEval-2020 Task 11: Leveraging Pure Transformer for Multi-Class Propaganda Detection",
            "authors": [
                "Dmitry Grigorev",
                "Vladimir Ivanov"
            ],
            "abstract": "The paper presents the solution of team \"Inno\" to a SEMEVAL 2020 task 11\n\"Detection of propaganda techniques in news articles\". The goal of the second\nsubtask is to classify textual segments that correspond to one of the 18 given\npropaganda techniques in news articles dataset. We tested a pure\nTransformer-based model with an optimized learning scheme on the ability to\ndistinguish propaganda techniques between each other. Our model showed 0.6 and\n0.58 overall F1 score on validation set and test set accordingly and non-zero\nF1 score on each class on both sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11608v3",
            "date": "2020-08-26 15:07:07+00:00",
            "title": "Analysis and Evaluation of Language Models for Word Sense Disambiguation",
            "authors": [
                "Daniel Loureiro",
                "Kiamehr Rezaee",
                "Mohammad Taher Pilehvar",
                "Jose Camacho-Collados"
            ],
            "abstract": "Transformer-based language models have taken many fields in NLP by storm.\nBERT and its derivatives dominate most of the existing evaluation benchmarks,\nincluding those for Word Sense Disambiguation (WSD), thanks to their ability in\ncapturing context-sensitive semantic nuances. However, there is still little\nknowledge about their capabilities and potential limitations in encoding and\nrecovering word senses. In this article, we provide an in-depth quantitative\nand qualitative analysis of the celebrated BERT model with respect to lexical\nambiguity. One of the main conclusions of our analysis is that BERT can\naccurately capture high-level sense distinctions, even when a limited number of\nexamples is available for each word sense. Our analysis also reveals that in\nsome cases language models come close to solving coarse-grained noun\ndisambiguation under ideal conditions in terms of availability of training data\nand computing resources. However, this scenario rarely occurs in real-world\nsettings and, hence, many practical challenges remain even in the\ncoarse-grained setting. We also perform an in-depth comparison of the two main\nlanguage model based WSD strategies, i.e., fine-tuning and feature extraction,\nfinding that the latter approach is more robust with respect to sense bias and\nit can better exploit limited available training data. In fact, the simple\nfeature extraction strategy of averaging contextualized embeddings proves\nrobust even using only three training sentences per word sense, with minimal\nimprovements obtained by increasing the size of this training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11841v2",
            "date": "2020-08-26 21:57:25+00:00",
            "title": "On the Optimality of Vagueness: \"Around\", \"Between\", and the Gricean Maxims",
            "authors": [
                "Paul Egr\u00e9",
                "Benjamin Spector",
                "Ad\u00e8le Mortier",
                "Steven Verheyen"
            ],
            "abstract": "Why is our language vague? We argue that in contexts in which a cooperative\nspeaker is not perfectly informed about the world, the use of vague expressions\ncan offer an optimal tradeoff between truthfulness (Gricean Quality) and\ninformativeness (Gricean Quantity). Focusing on expressions of approximation\nsuch as \"around\", which are semantically vague, we show that they allow the\nspeaker to convey indirect probabilistic information, in a way that gives the\nlistener a more accurate representation of the information available to the\nspeaker than any more precise expression would (intervals of the form\n\"between\"). We give a probabilistic treatment of the interpretation of\n\"around\", and offer a model for the interpretation and use of\n\"around\"-statements within the Rational Speech Act (RSA) framework. Our model\ndiffers in substantive ways from the Lexical Uncertainty model often used\nwithin the RSA framework for vague predicates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11940v1",
            "date": "2020-08-27 06:42:18+00:00",
            "title": "Relation/Entity-Centric Reading Comprehension",
            "authors": [
                "Takeshi Onishi"
            ],
            "abstract": "Constructing a machine that understands human language is one of the most\nelusive and long-standing challenges in artificial intelligence. This thesis\naddresses this challenge through studies of reading comprehension with a focus\non understanding entities and their relationships. More specifically, we focus\non question answering tasks designed to measure reading comprehension. We focus\non entities and relations because they are typically used to represent the\nsemantics of natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11970v1",
            "date": "2020-08-27 07:53:47+00:00",
            "title": "Improvement of a dedicated model for open domain persona-aware dialogue generation",
            "authors": [
                "Qiang Han"
            ],
            "abstract": "This paper analyzes some speed and performance improvement methods of\nTransformer architecture in recent years, mainly its application in dedicated\nmodel training. The dedicated model studied here refers to the open domain\npersona-aware dialogue generation model, and the dataset is multi turn short\ndialogue, The total length of a single input sequence is no more than 105\ntokens. Therefore, many improvements in the architecture and attention\nmechanism of transformer architecture for long sequence processing are not\ndiscussed in this paper. The source code of the experiments has been open\nsourced: https://github.com/ghosthamlet/persona",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11986v1",
            "date": "2020-08-27 08:31:13+00:00",
            "title": "Query Focused Multi-document Summarisation of Biomedical Texts",
            "authors": [
                "Diego Molla",
                "Christopher Jones",
                "Vincent Nguyen"
            ],
            "abstract": "This paper presents the participation of Macquarie University and the\nAustralian National University for Task B Phase B of the 2020 BioASQ Challenge\n(BioASQ8b). Our overall framework implements Query focused multi-document\nextractive summarisation by applying either a classification or a regression\nlayer to the candidate sentence embeddings and to the comparison between the\nquestion and sentence embeddings. We experiment with variants using BERT and\nBioBERT, Siamese architectures, and reinforcement learning. We observe the best\nresults when BERT is used to obtain the word embeddings, followed by an LSTM\nlayer to obtain sentence embeddings. Variants using Siamese architectures or\nBioBERT did not improve the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12009v2",
            "date": "2020-08-27 09:25:05+00:00",
            "title": "A Survey of Evaluation Metrics Used for NLG Systems",
            "authors": [
                "Ananya B. Sai",
                "Akash Kumar Mohankumar",
                "Mitesh M. Khapra"
            ],
            "abstract": "The success of Deep Learning has created a surge in interest in a wide a\nrange of Natural Language Generation (NLG) tasks. Deep Learning has not only\npushed the state of the art in several existing NLG tasks but has also\nfacilitated researchers to explore various newer NLG tasks such as image\ncaptioning. Such rapid progress in NLG has necessitated the development of\naccurate automatic evaluation metrics that would allow us to track the progress\nin the field of NLG. However, unlike classification tasks, automatically\nevaluating NLG systems in itself is a huge challenge. Several works have shown\nthat early heuristic-based metrics such as BLEU, ROUGE are inadequate for\ncapturing the nuances in the different NLG tasks. The expanding number of NLG\nmodels and the shortcomings of the current metrics has led to a rapid surge in\nthe number of evaluation metrics proposed since 2014. Moreover, various\nevaluation metrics have shifted from using pre-determined heuristic-based\nformulae to trained transformer models. This rapid change in a relatively short\ntime has led to the need for a survey of the existing NLG metrics to help\nexisting and new researchers to quickly come up to speed with the developments\nthat have happened in NLG evaluation in the last few years. Through this\nsurvey, we first wish to highlight the challenges and difficulties in\nautomatically evaluating NLG systems. Then, we provide a coherent taxonomy of\nthe evaluation metrics to organize the existing metrics and to better\nunderstand the developments in the field. We also describe the different\nmetrics in detail and highlight their key contributions. Later, we discuss the\nmain shortcomings identified in the existing metrics and describe the\nmethodology used to evaluate evaluation metrics. Finally, we discuss our\nsuggestions and recommendations on the next steps forward to improve the\nautomatic evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12014v2",
            "date": "2020-08-27 09:36:14+00:00",
            "title": "GREEK-BERT: The Greeks visiting Sesame Street",
            "authors": [
                "John Koutsikakis",
                "Ilias Chalkidis",
                "Prodromos Malakasiotis",
                "Ion Androutsopoulos"
            ],
            "abstract": "Transformer-based language models, such as BERT and its variants, have\nachieved state-of-the-art performance in several downstream natural language\nprocessing (NLP) tasks on generic benchmark datasets (e.g., GLUE, SQUAD, RACE).\nHowever, these models have mostly been applied to the resource-rich English\nlanguage. In this paper, we present GREEK-BERT, a monolingual BERT-based\nlanguage model for modern Greek. We evaluate its performance in three NLP\ntasks, i.e., part-of-speech tagging, named entity recognition, and natural\nlanguage inference, obtaining state-of-the-art performance. Interestingly, in\ntwo of the benchmarks GREEK-BERT outperforms two multilingual Transformer-based\nmodels (M-BERT, XLM-R), as well as shallower neural baselines operating on\npre-trained word embeddings, by a large margin (5%-10%). Most importantly, we\nmake both GREEK-BERT and our training code publicly available, along with code\nillustrating how GREEK-BERT can be fine-tuned for downstream NLP tasks. We\nexpect these resources to boost NLP research and applications for modern Greek.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12169v1",
            "date": "2020-08-27 14:57:01+00:00",
            "title": "Uralic Language Identification (ULI) 2020 shared task dataset and the Wanca 2017 corpus",
            "authors": [
                "Tommi Jauhiainen",
                "Heidi Jauhiainen",
                "Niko Partanen",
                "Krister Lind\u00e9n"
            ],
            "abstract": "This article introduces the Wanca 2017 corpus of texts crawled from the\ninternet from which the sentences in rare Uralic languages for the use of the\nUralic Language Identification (ULI) 2020 shared task were collected. We\ndescribe the ULI dataset and how it was constructed using the Wanca 2017 corpus\nand texts in different languages from the Leipzig corpora collection. We also\nprovide baseline language identification experiments conducted using the ULI\n2020 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12281v3",
            "date": "2020-08-27 17:34:40+00:00",
            "title": "Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check",
            "authors": [
                "Minh Nguyen",
                "Gia H. Ngo",
                "Nancy F. Chen"
            ],
            "abstract": "Spell check is a useful application which processes noisy human-generated\ntext. Spell check for Chinese poses unresolved problems due to the large number\nof characters, the sparse distribution of errors, and the dearth of resources\nwith sufficient coverage of heterogeneous and shifting error domains. For\nChinese spell check, filtering using confusion sets narrows the search space\nand makes finding corrections easier. However, most, if not all, confusion sets\nused to date are fixed and thus do not include new, shifting error domains. We\npropose a scalable adaptable filter that exploits hierarchical character\nembeddings to (1) obviate the need to handcraft confusion sets, and (2) resolve\nsparsity problems related to infrequent errors. Our approach compares favorably\nwith competitive baselines and obtains SOTA results on the 2014 and 2015\nChinese Spelling Check Bake-off datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12283v1",
            "date": "2020-08-27 17:41:23+00:00",
            "title": "Entity and Evidence Guided Relation Extraction for DocRED",
            "authors": [
                "Kevin Huang",
                "Guangtao Wang",
                "Tengyu Ma",
                "Jing Huang"
            ],
            "abstract": "Document-level relation extraction is a challenging task which requires\nreasoning over multiple sentences in order to predict relations in a document.\nIn this paper, we pro-pose a joint training frameworkE2GRE(Entity and Evidence\nGuided Relation Extraction)for this task. First, we introduce entity-guided\nsequences as inputs to a pre-trained language model (e.g. BERT, RoBERTa). These\nentity-guided sequences help a pre-trained language model (LM) to focus on\nareas of the document related to the entity. Secondly, we guide the fine-tuning\nof the pre-trained language model by using its internal attention probabilities\nas additional features for evidence prediction.Our new approach encourages the\npre-trained language model to focus on the entities and supporting/evidence\nsentences. We evaluate our E2GRE approach on DocRED, a recently released\nlarge-scale dataset for relation extraction. Our approach is able to achieve\nstate-of-the-art results on the public leaderboard across all metrics, showing\nthat our E2GRE is both effective and synergistic on relation extraction and\nevidence prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12360v1",
            "date": "2020-08-27 20:04:30+00:00",
            "title": "Language Models as Emotional Classifiers for Textual Conversations",
            "authors": [
                "Connor T. Heaton",
                "David M. Schwartz"
            ],
            "abstract": "Emotions play a critical role in our everyday lives by altering how we\nperceive, process and respond to our environment. Affective computing aims to\ninstill in computers the ability to detect and act on the emotions of human\nactors. A core aspect of any affective computing system is the classification\nof a user's emotion. In this study we present a novel methodology for\nclassifying emotion in a conversation. At the backbone of our proposed\nmethodology is a pre-trained Language Model (LM), which is supplemented by a\nGraph Convolutional Network (GCN) that propagates information over the\npredicate-argument structure identified in an utterance. We apply our proposed\nmethodology on the IEMOCAP and Friends data sets, achieving state-of-the-art\nperformance on the former and a higher accuracy on certain emotional labels on\nthe latter. Furthermore, we examine the role context plays in our methodology\nby altering how much of the preceding conversation the model has access to when\nmaking a classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12548v1",
            "date": "2020-08-28 09:27:43+00:00",
            "title": "Cost-Quality Adaptive Active Learning for Chinese Clinical Named Entity Recognition",
            "authors": [
                "Tingting Cai",
                "Yangming Zhou",
                "Hong Zheng"
            ],
            "abstract": "Clinical Named Entity Recognition (CNER) aims to automatically identity\nclinical terminologies in Electronic Health Records (EHRs), which is a\nfundamental and crucial step for clinical research. To train a high-performance\nmodel for CNER, it usually requires a large number of EHRs with high-quality\nlabels. However, labeling EHRs, especially Chinese EHRs, is time-consuming and\nexpensive. One effective solution to this is active learning, where a model\nasks labelers to annotate data which the model is uncertain of. Conventional\nactive learning assumes a single labeler that always replies noiseless answers\nto queried labels. However, in real settings, multiple labelers provide diverse\nquality of annotation with varied costs and labelers with low overall\nannotation quality can still assign correct labels for some specific instances.\nIn this paper, we propose a Cost-Quality Adaptive Active Learning (CQAAL)\napproach for CNER in Chinese EHRs, which maintains a balance between the\nannotation quality, labeling costs, and the informativeness of selected\ninstances. Specifically, CQAAL selects cost-effective instance-labeler pairs to\nachieve better annotation quality with lower costs in an adaptive manner.\nComputational results on the CCKS-2017 Task 2 benchmark dataset demonstrate the\nsuperiority and effectiveness of the proposed CQAAL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12704v1",
            "date": "2020-08-28 15:30:08+00:00",
            "title": "Two Step Joint Model for Drug Drug Interaction Extraction",
            "authors": [
                "Siliang Tang",
                "Qi Zhang",
                "Tianpeng Zheng",
                "Mengdi Zhou",
                "Zhan Chen",
                "Lixing Shen",
                "Xiang Ren",
                "Yueting Zhuang",
                "Shiliang Pu",
                "Fei Wu"
            ],
            "abstract": "When patients need to take medicine, particularly taking more than one kind\nof drug simultaneously, they should be alarmed that there possibly exists\ndrug-drug interaction. Interaction between drugs may have a negative impact on\npatients or even cause death. Generally, drugs that conflict with a specific\ndrug (or label drug) are usually described in its drug label or package insert.\nSince more and more new drug products come into the market, it is difficult to\ncollect such information by manual. We take part in the Drug-Drug Interaction\n(DDI) Extraction from Drug Labels challenge of Text Analysis Conference (TAC)\n2018, choosing task1 and task2 to automatically extract DDI related mentions\nand DDI relations respectively. Instead of regarding task1 as named entity\nrecognition (NER) task and regarding task2 as relation extraction (RE) task\nthen solving it in a pipeline, we propose a two step joint model to detect DDI\nand it's related mentions jointly. A sequence tagging system (CNN-GRU\nencoder-decoder) finds precipitants first and search its fine-grained Trigger\nand determine the DDI for each precipitant in the second step. Moreover, a rule\nbased model is built to determine the sub-type for pharmacokinetic interation.\nOur system achieved best result in both task1 and task2. F-measure reaches 0.46\nin task1 and 0.40 in task2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12878v1",
            "date": "2020-08-28 23:32:33+00:00",
            "title": "Knowledge Efficient Deep Learning for Natural Language Processing",
            "authors": [
                "Hai Wang"
            ],
            "abstract": "Deep learning has become the workhorse for a wide range of natural language\nprocessing applications. But much of the success of deep learning relies on\nannotated examples. Annotation is time-consuming and expensive to produce at\nscale. Here we are interested in methods for reducing the required quantity of\nannotated data -- by making the learning methods more knowledge efficient so as\nto make them more applicable in low annotation (low resource) settings. There\nare various classical approaches to making the models more knowledge efficient\nsuch as multi-task learning, transfer learning, weakly supervised and\nunsupervised learning etc. This thesis focuses on adapting such classical\nmethods to modern deep learning models and algorithms.\n  This thesis describes four works aimed at making machine learning models more\nknowledge efficient. First, we propose a knowledge rich deep learning model\n(KRDL) as a unifying learning framework for incorporating prior knowledge into\ndeep models. In particular, we apply KRDL built on Markov logic networks to\ndenoise weak supervision. Second, we apply a KRDL model to assist the machine\nreading models to find the correct evidence sentences that can support their\ndecision. Third, we investigate the knowledge transfer techniques in\nmultilingual setting, where we proposed a method that can improve pre-trained\nmultilingual BERT based on the bilingual dictionary. Fourth, we present an\nepisodic memory network for language modelling, in which we encode the large\nexternal knowledge for the pre-trained GPT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12918v2",
            "date": "2020-08-29 05:48:32+00:00",
            "title": "Zero-Resource Knowledge-Grounded Dialogue Generation",
            "authors": [
                "Linxiao Li",
                "Can Xu",
                "Wei Wu",
                "Yufan Zhao",
                "Xueliang Zhao",
                "Chongyang Tao"
            ],
            "abstract": "While neural conversation models have shown great potentials towards\ngenerating informative and engaging responses via introducing external\nknowledge, learning such a model often requires knowledge-grounded dialogues\nthat are difficult to obtain. To overcome the data challenge and reduce the\ncost of building a knowledge-grounded dialogue system, we explore the problem\nunder a zero-resource setting by assuming no context-knowledge-response triples\nare needed for training. To this end, we propose representing the knowledge\nthat bridges a context and a response and the way that the knowledge is\nexpressed as latent variables, and devise a variational approach that can\neffectively estimate a generation model from a dialogue corpus and a knowledge\ncorpus that are independent with each other. Evaluation results on three\nbenchmarks of knowledge-grounded dialogue generation indicate that our model\ncan achieve comparable performance with state-of-the-art methods that rely on\nknowledge-grounded dialogues for training, and exhibits a good generalization\nability over different topics and different datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12988v4",
            "date": "2020-08-29 14:58:26+00:00",
            "title": "Efficient Computation of Expectations under Spanning Tree Distributions",
            "authors": [
                "Ran Zmigrod",
                "Tim Vieira",
                "Ryan Cotterell"
            ],
            "abstract": "We give a general framework for inference in spanning tree models. We propose\nunified algorithms for the important cases of first-order expectations and\nsecond-order expectations in edge-factored, non-projective spanning-tree\nmodels. Our algorithms exploit a fundamental connection between gradients and\nexpectations, which allows us to derive efficient algorithms. These algorithms\nare easy to implement with or without automatic differentiation software. We\nmotivate the development of our framework with several \\emph{cautionary tales}\nof previous research, which has developed numerous inefficient algorithms for\ncomputing expectations and their gradients. We demonstrate how our framework\nefficiently computes several quantities with known algorithms, including the\nexpected attachment score, entropy, and generalized expectation criteria. As a\nbonus, we give algorithms for quantities that are missing in the literature,\nincluding the KL divergence. In all cases, our approach matches the efficiency\nof existing algorithms and, in several cases, reduces the runtime complexity by\na factor of the sentence length. We validate the implementation of our\nframework through runtime experiments. We find our algorithms are up to 15 and\n9 times faster than previous algorithms for computing the Shannon entropy and\nthe gradient of the generalized expectation objective, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13339v2",
            "date": "2020-08-31 03:28:18+00:00",
            "title": "BiTT: Bidirectional Tree Tagging for Joint Extraction of Overlapping Entities and Relations",
            "authors": [
                "Xukun Luo",
                "Weijie Liu",
                "Meng Ma",
                "Ping Wang"
            ],
            "abstract": "Joint extraction refers to extracting triples, composed of entities and\nrelations, simultaneously from the text with a single model. However, most\nexisting methods fail to extract all triples accurately and efficiently from\nsentences with overlapping issue, i.e., the same entity is included in multiple\ntriples. In this paper, we propose a novel scheme called Bidirectional Tree\nTagging (BiTT) to label overlapping triples in text. In BiTT, the triples with\nthe same relation category in a sentence are especially represented as two\nbinary trees, each of which is converted into a word-level tags sequence to\nlabel each word. Based on BiTT scheme, we develop an end-to-end extraction\nframework to predict the BiTT tags and further extract triples efficiently. We\nadopt the Bi-LSTM and the BERT as the encoder in our framework respectively,\nand obtain promising results in public English as well as Chinese datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13609v1",
            "date": "2020-08-31 13:57:31+00:00",
            "title": "Detecting Generic Music Features with Single Layer Feedforward Network using Unsupervised Hebbian Computation",
            "authors": [
                "Sourav Das",
                "Anup Kumar Kolya"
            ],
            "abstract": "With the ever-increasing number of digital music and vast music track\nfeatures through popular online music streaming software and apps, feature\nrecognition using the neural network is being used for experimentation to\nproduce a wide range of results across a variety of experiments recently.\nThrough this work, the authors extract information on such features from a\npopular open-source music corpus and explored new recognition techniques, by\napplying unsupervised Hebbian learning techniques on their single-layer neural\nnetwork using the same dataset. The authors show the detailed empirical\nfindings to simulate how such an algorithm can help a single layer feedforward\nnetwork in training for music feature learning as patterns. The unsupervised\ntraining algorithm enhances their proposed neural network to achieve an\naccuracy of 90.36% for successful music feature detection. For comparative\nanalysis against similar tasks, authors put their results with the likes of\nseveral previous benchmark works. They further discuss the limitations and\nthorough error analysis of their work. The authors hope to discover and gather\nnew information about this particular classification technique and its\nperformance, and further understand future potential directions and prospects\nthat could improve the art of computational music feature recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13694v1",
            "date": "2020-08-31 15:55:24+00:00",
            "title": "SemEval-2020 Task 6: Definition extraction from free text with the DEFT corpus",
            "authors": [
                "Sasha Spala",
                "Nicholas A Miller",
                "Franck Dernoncourt",
                "Carl Dockhorn"
            ],
            "abstract": "Research on definition extraction has been conducted for well over a decade,\nlargely with significant constraints on the type of definitions considered. In\nthis work, we present DeftEval, a SemEval shared task in which participants\nmust extract definitions from free text using a term-definition pair corpus\nthat reflects the complex reality of definitions in natural language.\nDefinitions and glosses in free text often appear without explicit indicators,\nacross sentences boundaries, or in an otherwise complex linguistic manner.\nDeftEval involved 3 distinct subtasks: 1)Sentence classification, 2) sequence\nlabeling, and 3) relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00331v2",
            "date": "2020-09-01 10:19:18+00:00",
            "title": "Extracting Semantic Concepts and Relations from Scientific Publications by Using Deep Learning",
            "authors": [
                "Fatima N. AL-Aswadi",
                "Huah Yong Chan",
                "Keng Hoon Gan"
            ],
            "abstract": "With the large volume of unstructured data that increases constantly on the\nweb, the motivation of representing the knowledge in this data in the\nmachine-understandable form is increased. Ontology is one of the major\ncornerstones of representing the information in a more meaningful way on the\nsemantic Web. The current ontology repositories are quite limited either for\ntheir scope or for currentness. In addition, the current ontology extraction\nsystems have many shortcomings and drawbacks, such as using a small dataset,\ndepending on a large amount predefined patterns to extract semantic relations,\nand extracting a very few types of relations. The aim of this paper is to\nintroduce a proposal of automatically extracting semantic concepts and\nrelations from scientific publications. This paper suggests new types of\nsemantic relations and points out of using deep learning (DL) models for\nsemantic relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00590v1",
            "date": "2020-09-01 17:27:12+00:00",
            "title": "SuperPAL: Supervised Proposition ALignment for Multi-Document Summarization and Derivative Sub-Tasks",
            "authors": [
                "Ori Ernst",
                "Ori Shapira",
                "Ramakanth Pasunuru",
                "Michael Lepioshkin",
                "Jacob Goldberger",
                "Mohit Bansal",
                "Ido Dagan"
            ],
            "abstract": "Multi-document summarization (MDS) is a challenging task, often decomposed to\nsubtasks of salience and redundancy detection, followed by generation. While\nalignment of spans between reference summaries and source documents has been\nleveraged for training component tasks, the underlying alignment step was never\nindependently addressed or evaluated. We advocate developing high quality\nsource-reference alignment algorithms, that can be applied to recent\nlarge-scale datasets to obtain useful \"silver\", i.e. approximate, training\ndata. As a first step, we present an annotation methodology by which we create\ngold standard development and test sets for summary-source alignment, and\nsuggest its utility for tuning and evaluating effective alignment algorithms,\nas well as for properly evaluating MDS subtasks. Second, we introduce a new\nlarge-scale alignment dataset for training, with which an automatic alignment\nmodel was trained. This aligner achieves higher coherency with the reference\nsummary than previous aligners used for summarization, and gets significantly\nhigher ROUGE results when replacing a simpler aligner in a competitive\nsummarization model. Finally, we release three additional datasets (for\nsalience, clustering and generation), naturally derived from our alignment\ndatasets. Furthermore, these datasets can be derived from any summarization\ndataset automatically after extracting alignments with our trained aligner.\nHence, they can be utilized for training summarization sub-tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00694v2",
            "date": "2020-09-01 20:57:41+00:00",
            "title": "Automatic Assignment of Radiology Examination Protocols Using Pre-trained Language Models with Knowledge Distillation",
            "authors": [
                "Wilson Lau",
                "Laura Aaltonen",
                "Martin Gunn",
                "Meliha Yetisgen"
            ],
            "abstract": "Selecting radiology examination protocol is a repetitive, and time-consuming\nprocess. In this paper, we present a deep learning approach to automatically\nassign protocols to computer tomography examinations, by pre-training a\ndomain-specific BERT model ($BERT_{rad}$). To handle the high data imbalance\nacross exam protocols, we used a knowledge distillation approach that\nup-sampled the minority classes through data augmentation. We compared\nclassification performance of the described approach with the statistical\nn-gram models using Support Vector Machine (SVM) and Random Forest (RF)\nclassifiers, as well as the Google's $BERT_{base}$ model. SVM and RF achieved\nmacro-averaged F1 scores of 0.45 and 0.6 while $BERT_{base}$ and $BERT_{rad}$\nachieved 0.61 and 0.63. Knowledge distillation improved overall performance on\nthe minority classes, achieving a F1 score of 0.66.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01126v2",
            "date": "2020-09-02 15:05:43+00:00",
            "title": "Too good to be true? Predicting author profiles from abusive language",
            "authors": [
                "Isabelle van der Vegt",
                "Bennett Kleinberg",
                "Paul Gill"
            ],
            "abstract": "The problem of online threats and abuse could potentially be mitigated with a\ncomputational approach, where sources of abuse are better understood or\nidentified through author profiling. However, abusive language constitutes a\nspecific domain of language for which it has not yet been tested whether\ndifferences emerge based on a text author's personality, age, or gender. This\nstudy examines statistical relationships between author demographics and\nabusive vs normal language, and performs prediction experiments for\npersonality, age, and gender. Although some statistical relationships were\nestablished between author characteristics and language use, these patterns did\nnot translate to high prediction performance. Personality traits were predicted\nwithin 15% of their actual value, age was predicted with an error margin of 10\nyears, and gender was classified correctly in 70% of the cases. These results\nare poor when compared to previous research on author profiling, therefore we\nurge caution in applying this within the context of abusive language and threat\nassessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01134v1",
            "date": "2020-09-02 15:18:55+00:00",
            "title": "An exploratory study of L1-specific non-words",
            "authors": [
                "David Alfter"
            ],
            "abstract": "In this paper, we explore L1-specific non-words, i.e. non-words in a target\nlanguage (in this case Swedish) that are re-ranked by a different-language\nlanguage model. We surmise that speakers of a certain L1 will react different\nto L1-specific non-words than to general non-words. We present the results from\ntwo small case studies exploring whether re-ranking non-words with different\nlanguage models leads to a perceived difference in `Swedishness' (pilot study\n1) and whether German and English native speakers have longer reaction times in\na lexical decision task when presented with their respective L1-specific\nnon-words (pilot study 2). Tentative results seem to indicate that L1-specific\nnon-words are processed second-slowest, after purely Swedish-looking non-words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01195v1",
            "date": "2020-09-02 17:09:29+00:00",
            "title": "Garain at SemEval-2020 Task 12: Sequence based Deep Learning for Categorizing Offensive Language in Social Media",
            "authors": [
                "Avishek Garain"
            ],
            "abstract": "SemEval-2020 Task 12 was OffenseEval: Multilingual Offensive Language\nIdentification in Social Media (Zampieri et al., 2020). The task was subdivided\ninto multiple languages and datasets were provided for each one. The task was\nfurther divided into three sub-tasks: offensive language identification,\nautomatic categorization of offense types, and offense target identification. I\nhave participated in the task-C, that is, offense target identification. For\npreparing the proposed system, I have made use of Deep Learning networks like\nLSTMs and frameworks like Keras which combine the bag of words model with\nautomatically generated sequence based features and manually extracted features\nfrom the given dataset. My system on training on 25% of the whole dataset\nachieves macro averaged f1 score of 47.763%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01303v1",
            "date": "2020-09-02 19:07:34+00:00",
            "title": "Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading",
            "authors": [
                "Sasi Kiran Gaddipati",
                "Deebul Nair",
                "Paul G. Pl\u00f6ger"
            ],
            "abstract": "Automatic Short Answer Grading (ASAG) is the process of grading the student\nanswers by computational approaches given a question and the desired answer.\nPrevious works implemented the methods of concept mapping, facet mapping, and\nsome used the conventional word embeddings for extracting semantic features.\nThey extracted multiple features manually to train on the corresponding\ndatasets. We use pretrained embeddings of the transfer learning models, ELMo,\nBERT, GPT, and GPT-2 to assess their efficiency on this task. We train with a\nsingle feature, cosine similarity, extracted from the embeddings of these\nmodels. We compare the RMSE scores and correlation measurements of the four\nmodels with previous works on Mohler dataset. Our work demonstrates that ELMo\noutperformed the other three models. We also, briefly describe the four\ntransfer learning models and conclude with the possible causes of poor results\nof transfer learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01312v2",
            "date": "2020-09-02 19:28:40+00:00",
            "title": "A Simple Global Neural Discourse Parser",
            "authors": [
                "Yichu Zhou",
                "Omri Koshorek",
                "Vivek Srikumar",
                "Jonathan Berant"
            ],
            "abstract": "Discourse parsing is largely dominated by greedy parsers with\nmanually-designed features, while global parsing is rare due to its\ncomputational expense. In this paper, we propose a simple chart-based neural\ndiscourse parser that does not require any manually-crafted features and is\nbased on learned span representations only. To overcome the computational\nchallenge, we propose an independence assumption between the label assigned to\na node in the tree and the splitting point that separates its children, which\nresults in tractable decoding. We empirically demonstrate that our model\nachieves the best performance among global parsers, and comparable performance\nto state-of-art greedy parsers, using only learned span representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01460v1",
            "date": "2020-09-03 05:51:27+00:00",
            "title": "orgFAQ: A New Dataset and Analysis on Organizational FAQs and User Questions",
            "authors": [
                "Guy Lev",
                "Michal Shmueli-Scheuer",
                "Achiya Jerbi",
                "David Konopnicki"
            ],
            "abstract": "Frequently Asked Questions (FAQ) webpages are created by organizations for\ntheir users. FAQs are used in several scenarios, e.g., to answer user\nquestions. On the other hand, the content of FAQs is affected by user questions\nby definition. In order to promote research in this field, several FAQ datasets\nexist. However, we claim that being collected from community websites, they do\nnot correctly represent challenges associated with FAQs in an organizational\ncontext. Thus, we release orgFAQ, a new dataset composed of $6988$ user\nquestions and $1579$ corresponding FAQs that were extracted from organizations'\nFAQ webpages in the Jobs domain. In this paper, we provide an analysis of the\nproperties of such FAQs, and demonstrate the usefulness of our new dataset by\nutilizing it in a relevant task from the Jobs domain. We also show the value of\nthe orgFAQ dataset in a task of a different domain - the COVID-19 pandemic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01560v2",
            "date": "2020-09-03 10:10:20+00:00",
            "title": "Biomedical named entity recognition using BERT in the machine reading comprehension framework",
            "authors": [
                "Cong Sun",
                "Zhihao Yang",
                "Lei Wang",
                "Yin Zhang",
                "Hongfei Lin",
                "Jian Wang"
            ],
            "abstract": "Recognition of biomedical entities from literature is a challenging research\nfocus, which is the foundation for extracting a large amount of biomedical\nknowledge existing in unstructured texts into structured formats. Using the\nsequence labeling framework to implement biomedical named entity recognition\n(BioNER) is currently a conventional method. This method, however, often cannot\ntake full advantage of the semantic information in the dataset, and the\nperformance is not always satisfactory. In this work, instead of treating the\nBioNER task as a sequence labeling problem, we formulate it as a machine\nreading comprehension (MRC) problem. This formulation can introduce more prior\nknowledge utilizing well-designed queries, and no longer need decoding\nprocesses such as conditional random fields (CRF). We conduct experiments on\nsix BioNER datasets, and the experimental results demonstrate the effectiveness\nof our method. Our method achieves state-of-the-art (SOTA) performance on the\nBC4CHEMD, BC5CDR-Chem, BC5CDR-Disease, NCBI-Disease, BC2GM and JNLPBA datasets,\nachieving F1-scores of 92.92%, 94.19%, 87.83%, 90.04%, 85.48% and 78.93%,\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01712v1",
            "date": "2020-09-03 14:43:04+00:00",
            "title": "The ADAPT Enhanced Dependency Parser at the IWPT 2020 Shared Task",
            "authors": [
                "James Barry",
                "Joachim Wagner",
                "Jennifer Foster"
            ],
            "abstract": "We describe the ADAPT system for the 2020 IWPT Shared Task on parsing\nenhanced Universal Dependencies in 17 languages. We implement a pipeline\napproach using UDPipe and UDPipe-future to provide initial levels of\nannotation. The enhanced dependency graph is either produced by a graph-based\nsemantic dependency parser or is built from the basic tree using a small set of\nheuristics. Our results show that, for the majority of languages, a semantic\ndependency parser can be successfully applied to the task of parsing enhanced\ndependencies.\n  Unfortunately, we did not ensure a connected graph as part of our pipeline\napproach and our competition submission relied on a last-minute fix to pass the\nvalidation script which harmed our official evaluation scores significantly.\nOur submission ranked eighth in the official evaluation with a macro-averaged\ncoarse ELAS F1 of 67.23 and a treebank average of 67.49. We later implemented\nour own graph-connecting fix which resulted in a score of 79.53 (language\naverage) or 79.76 (treebank average), which would have placed fourth in the\ncompetition evaluation.",
            "journal ref": "Proceedings of the 16th International Conference on Parsing\n  Technologies and the IWPT 2020 Shared Task (2020) 227-235",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01826v2",
            "date": "2020-09-03 17:44:44+00:00",
            "title": "A Python Library for Exploratory Data Analysis on Twitter Data based on Tokens and Aggregated Origin-Destination Information",
            "authors": [
                "Mario Graff",
                "Daniela Moctezuma",
                "Sabino Miranda-Jim\u00e9nez",
                "Eric S. Tellez"
            ],
            "abstract": "Twitter is perhaps the social media more amenable for research. It requires\nonly a few steps to obtain information, and there are plenty of libraries that\ncan help in this regard. Nonetheless, knowing whether a particular event is\nexpressed on Twitter is a challenging task that requires a considerable\ncollection of tweets. This proposal aims to facilitate, to a researcher\ninterested, the process of mining events on Twitter. The events could be\nrelated to natural disasters, health issues, and people's mobility, among other\nstudies that can be pursued with the library proposed. Different applications\nare presented in this contribution to illustrate the library's capabilities: an\nexploratory analysis of the topics discovered in tweets, a study on similarity\namong dialects of the Spanish language, and a mobility report on different\ncountries. In summary, the Python library presented is applied to different\ndomains and retrieves a plethora of information processed from Twitter (since\nDecember 2015) in terms of words, bi-grams of words, and their frequencies by\nday for Arabic, English, Spanish, and Russian languages. The mobility\ninformation is related to the number of travels among locations for more than\n200 countries or territories; our library also provides access to this\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02070v2",
            "date": "2020-09-04 08:46:22+00:00",
            "title": "AutoTrans: Automating Transformer Design via Reinforced Architecture Search",
            "authors": [
                "Wei Zhu",
                "Xiaoling Wang",
                "Xipeng Qiu",
                "Yuan Ni",
                "Guotong Xie"
            ],
            "abstract": "Though the transformer architectures have shown dominance in many natural\nlanguage understanding tasks, there are still unsolved issues for the training\nof transformer models, especially the need for a principled way of warm-up\nwhich has shown importance for stable training of a transformer, as well as\nwhether the task at hand prefer to scale the attention product or not. In this\npaper, we empirically explore automating the design choices in the transformer\nmodel, i.e., how to set layer-norm, whether to scale, number of layers, number\nof heads, activation function, etc, so that one can obtain a transformer\narchitecture that better suits the tasks at hand. RL is employed to navigate\nalong search space, and special parameter sharing strategies are designed to\naccelerate the search. It is shown that sampling a proportion of training data\nper epoch during search help to improve the search quality. Experiments on the\nCoNLL03, Multi-30k, IWSLT14 and WMT-14 shows that the searched transformer\nmodel can outperform the standard transformers. In particular, we show that our\nlearned model can be trained more robustly with large learning rates without\nwarm-up.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02073v1",
            "date": "2020-09-04 08:58:42+00:00",
            "title": "Linguistically inspired morphological inflection with a sequence to sequence model",
            "authors": [
                "Eleni Metheniti",
                "Guenter Neumann",
                "Josef van Genabith"
            ],
            "abstract": "Inflection is an essential part of every human language's morphology, yet\nlittle effort has been made to unify linguistic theory and computational\nmethods in recent years. Methods of string manipulation are used to infer\ninflectional changes; our research question is whether a neural network would\nbe capable of learning inflectional morphemes for inflection production in a\nsimilar way to a human in early stages of language acquisition. We are using an\ninflectional corpus (Metheniti and Neumann, 2020) and a single layer seq2seq\nmodel to test this hypothesis, in which the inflectional affixes are learned\nand predicted as a block and the word stem is modelled as a character sequence\nto account for infixation. Our character-morpheme-based model creates\ninflection by predicting the stem character-to-character and the inflectional\naffixes as character blocks. We conducted three experiments on creating an\ninflected form of a word given the lemma and a set of input and target\nfeatures, comparing our architecture to a mainstream character-based model with\nthe same hyperparameters, training and test sets. Overall for 17 languages, we\nnoticed small improvements on inflecting known lemmas (+0.68%) but steadily\nbetter performance of our model in predicting inflected forms of unknown words\n(+3.7%) and small improvements on predicting in a low-resource scenario\n(+1.09%)",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02431v1",
            "date": "2020-09-05 01:44:11+00:00",
            "title": "Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models",
            "authors": [
                "Evan Williams",
                "Paul Rodrigues",
                "Valerie Novak"
            ],
            "abstract": "We introduce the strategies used by the Accenture Team for the CLEF2020\nCheckThat! Lab, Task 1, on English and Arabic. This shared task evaluated\nwhether a claim in social media text should be professionally fact checked. To\na journalist, a statement presented as fact, which would be of interest to a\nlarge audience, requires professional fact-checking before dissemination. We\nutilized BERT and RoBERTa models to identify claims in social media text a\nprofessional fact-checker should review, and rank these in priority order for\nthe fact-checker. For the English challenge, we fine-tuned a RoBERTa model and\nadded an extra mean pooling layer and a dropout layer to enhance\ngeneralizability to unseen text. For the Arabic task, we fine-tuned\nArabic-language BERT models and demonstrate the use of back-translation to\namplify the minority class and balance the dataset. The work presented here was\nscored 1st place in the English track, and 1st, 2nd, 3rd, and 4th place in the\nArabic track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02619v1",
            "date": "2020-09-06 00:15:33+00:00",
            "title": "MIDAS at SemEval-2020 Task 10: Emphasis Selection using Label Distribution Learning and Contextual Embeddings",
            "authors": [
                "Sarthak Anand",
                "Pradyumna Gupta",
                "Hemant Yadav",
                "Debanjan Mahata",
                "Rakesh Gosangi",
                "Haimin Zhang",
                "Rajiv Ratn Shah"
            ],
            "abstract": "This paper presents our submission to the SemEval 2020 - Task 10 on emphasis\nselection in written text. We approach this emphasis selection problem as a\nsequence labeling task where we represent the underlying text with various\ncontextual embedding models. We also employ label distribution learning to\naccount for annotator disagreements. We experiment with the choice of model\narchitectures, trainability of layers, and different contextual embeddings. Our\nbest performing architecture is an ensemble of different models, which achieved\nan overall matching score of 0.783, placing us 15th out of 31 participating\nteams. Lastly, we analyze the results in terms of parts of speech tags,\nsentence lengths, and word ordering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02649v1",
            "date": "2020-09-06 05:46:24+00:00",
            "title": "Once Upon A Time In Visualization: Understanding the Use of Textual Narratives for Causality",
            "authors": [
                "Arjun Choudhry",
                "Mandar Sharma",
                "Pramod Chundury",
                "Thomas Kapler",
                "Derek W. S. Gray",
                "Naren Ramakrishnan",
                "Niklas Elmqvist"
            ],
            "abstract": "Causality visualization can help people understand temporal chains of events,\nsuch as messages sent in a distributed system, cause and effect in a historical\nconflict, or the interplay between political actors over time. However, as the\nscale and complexity of these event sequences grows, even these visualizations\ncan become overwhelming to use. In this paper, we propose the use of textual\nnarratives as a data-driven storytelling method to augment causality\nvisualization. We first propose a design space for how textual narratives can\nbe used to describe causal data. We then present results from a crowdsourced\nuser study where participants were asked to recover causality information from\ntwo causality visualizations--causal graphs and Hasse diagrams--with and\nwithout an associated textual narrative. Finally, we describe CAUSEWORKS, a\ncausality visualization system for understanding how specific interventions\ninfluence a causal model. The system incorporates an automatic textual\nnarrative mechanism based on our design space. We validate CAUSEWORKS through\ninterviews with experts who used the system for understanding complex events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02685v1",
            "date": "2020-09-06 09:28:44+00:00",
            "title": "Automatic Dialect Adaptation in Finnish and its Effect on Perceived Creativity",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Niko Partanen",
                "Khalid Alnajjar",
                "Jack Rueter",
                "Thierry Poibeau"
            ],
            "abstract": "We present a novel approach for adapting text written in standard Finnish to\ndifferent dialects. We experiment with character level NMT models both by using\na multi-dialectal and transfer learning approaches. The models are tested with\nover 20 different dialects. The results seem to favor transfer learning,\nalthough not strongly over the multi-dialectal approach. We study the influence\ndialectal adaptation has on perceived creativity of computer generated poetry.\nOur results suggest that the more the dialect deviates from the standard\nFinnish, the lower scores people tend to give on an existing evaluation metric.\nHowever, on a word association test, people associate creativity and\noriginality more with dialect and fluency more with standard Finnish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02779v2",
            "date": "2020-09-06 17:17:41+00:00",
            "title": "UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis",
            "authors": [
                "George-Alexandru Vlad",
                "George-Eduard Zaharia",
                "Dumitru-Clementin Cercel",
                "Costin-Gabriel Chiru",
                "Stefan Trausan-Matu"
            ],
            "abstract": "Users from the online environment can create different ways of expressing\ntheir thoughts, opinions, or conception of amusement. Internet memes were\ncreated specifically for these situations. Their main purpose is to transmit\nideas by using combinations of images and texts such that they will create a\ncertain state for the receptor, depending on the message the meme has to send.\nThese posts can be related to various situations or events, thus adding a funny\nside to any circumstance our world is situated in. In this paper, we describe\nthe system developed by our team for SemEval-2020 Task 8: Memotion Analysis.\nMore specifically, we introduce a novel system to analyze these posts, a\nmultimodal multi-task learning architecture that combines ALBERT for text\nencoding with VGG-16 for image representation. In this manner, we show that the\ninformation behind them can be properly revealed. Our approach achieves good\nperformance on each of the three subtasks of the current competition, ranking\n11th for Subtask A (0.3453 macro F1-score), 1st for Subtask B (0.5183 macro\nF1-score), and 3rd for Subtask C (0.3171 macro F1-score) while exceeding the\nofficial baseline results by high margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02780v1",
            "date": "2020-09-06 17:19:18+00:00",
            "title": "UPB at SemEval-2020 Task 9: Identifying Sentiment in Code-Mixed Social Media Texts using Transformers and Multi-Task Learning",
            "authors": [
                "George-Eduard Zaharia",
                "George-Alexandru Vlad",
                "Dumitru-Clementin Cercel",
                "Traian Rebedea",
                "Costin-Gabriel Chiru"
            ],
            "abstract": "Sentiment analysis is a process widely used in opinion mining campaigns\nconducted today. This phenomenon presents applications in a variety of fields,\nespecially in collecting information related to the attitude or satisfaction of\nusers concerning a particular subject. However, the task of managing such a\nprocess becomes noticeably more difficult when it is applied in cultures that\ntend to combine two languages in order to express ideas and thoughts. By\ninterleaving words from two languages, the user can express with ease, but at\nthe cost of making the text far less intelligible for those who are not\nfamiliar with this technique, but also for standard opinion mining algorithms.\nIn this paper, we describe the systems developed by our team for SemEval-2020\nTask 9 that aims to cover two well-known code-mixed languages: Hindi-English\nand Spanish-English.\n  We intend to solve this issue by introducing a solution that takes advantage\nof several neural network approaches, as well as pre-trained word embeddings.\nOur approach (multlingual BERT) achieves promising performance on the\nHindi-English task, with an average F1-score of 0.6850, registered on the\ncompetition leaderboard, ranking our team 16th out of 62 participants. For the\nSpanish-English task, we obtained an average F1-score of 0.7064 ranking our\nteam 17th out of 29 participants by using another multilingual\nTransformer-based model, XLM-RoBERTa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02795v1",
            "date": "2020-09-06 18:34:54+00:00",
            "title": "Duluth at SemEval-2020 Task 7: Using Surprise as a Key to Unlock Humorous Headlines",
            "authors": [
                "Shuning Jin",
                "Yue Yin",
                "XianE Tang",
                "Ted Pedersen"
            ],
            "abstract": "We use pretrained transformer-based language models in SemEval-2020 Task 7:\nAssessing the Funniness of Edited News Headlines. Inspired by the incongruity\ntheory of humor, we use a contrastive approach to capture the surprise in the\nedited headlines. In the official evaluation, our system gets 0.531 RMSE in\nSubtask 1, 11th among 49 submissions. In Subtask 2, our system gets 0.632\naccuracy, 9th among 32 submissions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02902v2",
            "date": "2020-09-07 06:11:56+00:00",
            "title": "TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis",
            "authors": [
                "Zilong Wang",
                "Zhaohong Wan",
                "Xiaojun Wan"
            ],
            "abstract": "Multimodal sentiment analysis is an important research area that predicts\nspeaker's sentiment tendency through features extracted from textual, visual\nand acoustic modalities. The central challenge is the fusion method of the\nmultimodal information. A variety of fusion methods have been proposed, but few\nof them adopt end-to-end translation models to mine the subtle correlation\nbetween modalities. Enlightened by recent success of Transformer in the area of\nmachine translation, we propose a new fusion method, TransModality, to address\nthe task of multimodal sentiment analysis. We assume that translation between\nmodalities contributes to a better joint representation of speaker's utterance.\nWith Transformer, the learned features embody the information both from the\nsource modality and the target modality. We validate our model on multiple\nmultimodal datasets: CMU-MOSI, MELD, IEMOCAP. The experiments show that our\nproposed method achieves the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02935v3",
            "date": "2020-09-07 08:20:31+00:00",
            "title": "UIT-HSE at WNUT-2020 Task 2: Exploiting CT-BERT for Identifying COVID-19 Information on the Twitter Social Network",
            "authors": [
                "Khiem Vinh Tran",
                "Hao Phu Phan",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Recently, COVID-19 has affected a variety of real-life aspects of the world\nand led to dreadful consequences. More and more tweets about COVID-19 has been\nshared publicly on Twitter. However, the plurality of those Tweets are\nuninformative, which is challenging to build automatic systems to detect the\ninformative ones for useful AI applications. In this paper, we present our\nresults at the W-NUT 2020 Shared Task 2: Identification of Informative COVID-19\nEnglish Tweets. In particular, we propose our simple but effective approach\nusing the transformer-based models based on COVID-Twitter-BERT (CT-BERT) with\ndifferent fine-tuning techniques. As a result, we achieve the F1-Score of\n90.94\\% with the third place on the leaderboard of this task which attracted 56\nsubmitted teams in total.",
            "journal ref": "https://www.aclweb.org/anthology/2020.wnut-1.53/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03116v2",
            "date": "2020-09-07 14:07:12+00:00",
            "title": "Why Not Simply Translate? A First Swedish Evaluation Benchmark for Semantic Similarity",
            "authors": [
                "Tim Isbister",
                "Magnus Sahlgren"
            ],
            "abstract": "This paper presents the first Swedish evaluation benchmark for textual\nsemantic similarity. The benchmark is compiled by simply running the English\nSTS-B dataset through the Google machine translation API. This paper discusses\npotential problems with using such a simple approach to compile a Swedish\nevaluation benchmark, including translation errors, vocabulary variation, and\nproductive compounding. Despite some obvious problems with the resulting\ndataset, we use the benchmark to compare the majority of the currently existing\nSwedish text representations, demonstrating that native models outperform\nmultilingual ones, and that simple bag of words performs remarkably well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03191v1",
            "date": "2020-09-07 15:54:23+00:00",
            "title": "COVCOR20 at WNUT-2020 Task 2: An Attempt to Combine Deep Learning and Expert rules",
            "authors": [
                "Ali H\u00fcrriyeto\u011flu",
                "Ali Safaya",
                "Nelleke Oostdijk",
                "Osman Mutlu",
                "Erdem Y\u00f6r\u00fck"
            ],
            "abstract": "In the scope of WNUT-2020 Task 2, we developed various text classification\nsystems, using deep learning models and one using linguistically informed\nrules. While both of the deep learning systems outperformed the system using\nthe linguistically informed rules, we found that through the integration of\n(the output of) the three systems a better performance could be achieved than\nthe standalone performance of each approach in a cross-validation setting.\nHowever, on the test data the performance of the integration was slightly lower\nthan our best performing deep learning model. These results hardly indicate any\nprogress in line of integrating machine learning and expert rules driven\nsystems. We expect that the release of the annotation manuals and gold labels\nof the test data after this workshop will shed light on these perplexing\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03673v2",
            "date": "2020-09-08 12:20:04+00:00",
            "title": "kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification",
            "authors": [
                "Jiaxiang Liu",
                "Xuyi Chen",
                "Shikun Feng",
                "Shuohuan Wang",
                "Xuan Ouyang",
                "Yu Sun",
                "Zhengjie Huang",
                "Weiyue Su"
            ],
            "abstract": "Code switching is a linguistic phenomenon that may occur within a\nmultilingual setting where speakers share more than one language. With the\nincreasing communication between groups with different languages, this\nphenomenon is more and more popular. However, there are little research and\ndata in this area, especially in code-mixing sentiment classification. In this\nwork, the domain transfer learning from state-of-the-art uni-language model\nERNIE is tested on the code-mixing dataset, and surprisingly, a strong baseline\nis achieved. Furthermore, the adversarial training with a multi-lingual model\nis used to achieve 1st place of SemEval-2020 Task 9 Hindi-English sentiment\nclassification competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03695v1",
            "date": "2020-09-08 12:39:47+00:00",
            "title": "Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification",
            "authors": [
                "Samuel Louvan",
                "Bernardo Magnini"
            ],
            "abstract": "Neural-based models have achieved outstanding performance on slot filling and\nintent classification, when fairly large in-domain training data are available.\nHowever, as new domains are frequently added, creating sizeable data is\nexpensive. We show that lightweight augmentation, a set of augmentation methods\ninvolving word span and sentence level operations, alleviates data scarcity\nproblems. Our experiments on limited data settings show that lightweight\naugmentation yields significant performance improvement on slot filling on the\nATIS and SNIPS datasets, and achieves competitive performance with respect to\nmore complex, state-of-the-art, augmentation approaches. Furthermore,\nlightweight augmentation is also beneficial when combined with pre-trained\nLM-based models, as it improves BERT-based joint intent and slot filling\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03706v1",
            "date": "2020-09-08 12:51:22+00:00",
            "title": "ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model",
            "authors": [
                "Zhengjie Huang",
                "Shikun Feng",
                "Weiyue Su",
                "Xuyi Chen",
                "Shuohuan Wang",
                "Jiaxiang Liu",
                "Xuan Ouyang",
                "Yu Sun"
            ],
            "abstract": "This paper describes the system designed by ERNIE Team which achieved the\nfirst place in SemEval-2020 Task 10: Emphasis Selection For Written Text in\nVisual Media. Given a sentence, we are asked to find out the most important\nwords as the suggestion for automated design. We leverage the unsupervised\npre-training model and finetune these models on our task. After our\ninvestigation, we found that the following models achieved an excellent\nperformance in this task: ERNIE 2.0, XLM-ROBERTA, ROBERTA and ALBERT. We\ncombine a pointwise regression loss and a pairwise ranking loss which is more\nclose to the final M atchm metric to finetune our models. And we also find that\nadditional feature engineering and data augmentation can help improve the\nperformance. Our best model achieves the highest score of 0.823 and ranks first\nfor all kinds of metrics",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03897v1",
            "date": "2020-09-08 18:00:00+00:00",
            "title": "Quantifying the Causal Effects of Conversational Tendencies",
            "authors": [
                "Justine Zhang",
                "Sendhil Mullainathan",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "Understanding what leads to effective conversations can aid the design of\nbetter computer-mediated communication platforms. In particular, prior\nobservational work has sought to identify behaviors of individuals that\ncorrelate to their conversational efficiency. However, translating such\ncorrelations to causal interpretations is a necessary step in using them in a\nprescriptive fashion to guide better designs and policies.\n  In this work, we formally describe the problem of drawing causal links\nbetween conversational behaviors and outcomes. We focus on the task of\ndetermining a particular type of policy for a text-based crisis counseling\nplatform: how best to allocate counselors based on their behavioral tendencies\nexhibited in their past conversations. We apply arguments derived from causal\ninference to underline key challenges that arise in conversational settings\nwhere randomized trials are hard to implement. Finally, we show how to\ncircumvent these inference challenges in our particular domain, and illustrate\nthe potential benefits of an allocation policy informed by the resulting\nprescriptive information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04007v1",
            "date": "2020-09-08 21:55:22+00:00",
            "title": "Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function",
            "authors": [
                "Devendra Singh Sachan",
                "Manzil Zaheer",
                "Ruslan Salakhutdinov"
            ],
            "abstract": "In this paper, we study bidirectional LSTM network for the task of text\nclassification using both supervised and semi-supervised approaches. Several\nprior works have suggested that either complex pretraining schemes using\nunsupervised methods such as language modeling (Dai and Le 2015; Miyato, Dai,\nand Goodfellow 2016) or complicated models (Johnson and Zhang 2017) are\nnecessary to achieve a high classification accuracy. However, we develop a\ntraining strategy that allows even a simple BiLSTM model, when trained with\ncross-entropy loss, to achieve competitive results compared with more complex\napproaches. Furthermore, in addition to cross-entropy loss, by using a\ncombination of entropy minimization, adversarial, and virtual adversarial\nlosses for both labeled and unlabeled data, we report state-of-the-art results\nfor text classification task on several benchmark datasets. In particular, on\nthe ACL-IMDB sentiment analysis and AG-News topic classification datasets, our\nmethod outperforms current approaches by a substantial margin. We also show the\ngenerality of the mixed objective function by improving the performance on\nrelation extraction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04087v1",
            "date": "2020-09-09 03:11:43+00:00",
            "title": "Central Yup'ik and Machine Translation of Low-Resource Polysynthetic Languages",
            "authors": [
                "Christopher Liu",
                "Laura Domin\u00e9",
                "Kevin Chavez",
                "Richard Socher"
            ],
            "abstract": "Machine translation tools do not yet exist for the Yup'ik language, a\npolysynthetic language spoken by around 8,000 people who live primarily in\nSouthwest Alaska. We compiled a parallel text corpus for Yup'ik and English and\ndeveloped a morphological parser for Yup'ik based on grammar rules. We trained\na seq2seq neural machine translation model with attention to translate Yup'ik\ninput into English. We then compared the influence of different tokenization\nmethods, namely rule-based, unsupervised (byte pair encoding), and unsupervised\nmorphological (Morfessor) parsing, on BLEU score accuracy for Yup'ik to English\ntranslation. We find that using tokenized input increases the translation\naccuracy compared to that of unparsed input. Although overall Morfessor did\nbest with a vocabulary size of 30k, our first experiments show that BPE\nperformed best with a reduced vocabulary size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04095v1",
            "date": "2020-09-09 04:31:44+00:00",
            "title": "Comparative Study of Language Models on Cross-Domain Data with Model Agnostic Explainability",
            "authors": [
                "Mayank Chhipa",
                "Hrushikesh Mahesh Vazurkar",
                "Abhijeet Kumar",
                "Mridul Mishra"
            ],
            "abstract": "With the recent influx of bidirectional contextualized transformer language\nmodels in the NLP, it becomes a necessity to have a systematic comparative\nstudy of these models on variety of datasets. Also, the performance of these\nlanguage models has not been explored on non-GLUE datasets. The study presented\nin paper compares the state-of-the-art language models - BERT, ELECTRA and its\nderivatives which include RoBERTa, ALBERT and DistilBERT. We conducted\nexperiments by finetuning these models for cross domain and disparate data and\npenned an in-depth analysis of model's performances. Moreover, an\nexplainability of language models coherent with pretraining is presented which\nverifies the context capturing capabilities of these models through a model\nagnostic approach. The experimental results establish new state-of-the-art for\nYelp 2013 rating classification task and Financial Phrasebank sentiment\ndetection task with 69% accuracy and 88.2% accuracy respectively. Finally, the\nstudy conferred here can greatly assist industry researchers in choosing the\nlanguage model effectively in terms of performance or compute efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04202v1",
            "date": "2020-09-09 10:38:48+00:00",
            "title": "Impact of News on the Commodity Market: Dataset and Results",
            "authors": [
                "Ankur Sinha",
                "Tanmay Khandait"
            ],
            "abstract": "Over the last few years, machine learning based methods have been applied to\nextract information from news flow in the financial domain. However, this\ninformation has mostly been in the form of the financial sentiments contained\nin the news headlines, primarily for the stock prices. In our current work, we\npropose that various other dimensions of information can be extracted from news\nheadlines, which will be of interest to investors, policy-makers and other\npractitioners. We propose a framework that extracts information such as past\nmovements and expected directionality in prices, asset comparison and other\ngeneral information that the news is referring to. We apply this framework to\nthe commodity \"Gold\" and train the machine learning models using a dataset of\n11,412 human-annotated news headlines (released with this study), collected\nfrom the period 2000-2019. We experiment to validate the causal effect of news\nflow on gold prices and observe that the information produced from our\nframework significantly impacts the future gold price.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04639v1",
            "date": "2020-09-10 02:22:21+00:00",
            "title": "Improving Coreference Resolution by Leveraging Entity-Centric Features with Graph Neural Networks and Second-order Inference",
            "authors": [
                "Lu Liu",
                "Zhenqiao Song",
                "Xiaoqing Zheng"
            ],
            "abstract": "One of the major challenges in coreference resolution is how to make use of\nentity-level features defined over clusters of mentions rather than mention\npairs. However, coreferent mentions usually spread far apart in an entire text,\nwhich makes it extremely difficult to incorporate entity-level features. We\npropose a graph neural network-based coreference resolution method that can\ncapture the entity-centric information by encouraging the sharing of features\nacross all mentions that probably refer to the same real-world entity. Mentions\nare linked to each other via the edges modeling how likely two linked mentions\npoint to the same entity. Modeling by such graphs, the features between\nmentions can be shared by message passing operations in an entity-centric\nmanner. A global inference algorithm up to second-order features is also\npresented to optimally cluster mentions into consistent groups. Experimental\nresults show our graph neural network-based method combing with the\nsecond-order decoding algorithm (named GNNCR) achieved close to\nstate-of-the-art performance on the English CoNLL-2012 Shared Task dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04656v1",
            "date": "2020-09-10 03:53:18+00:00",
            "title": "Learning Universal Representations from Word to Sentence",
            "authors": [
                "Yian Li",
                "Hai Zhao"
            ],
            "abstract": "Despite the well-developed cut-edge representation learning for language,\nmost language representation models usually focus on specific level of\nlinguistic unit, which cause great inconvenience when being confronted with\nhandling multiple layers of linguistic objects in a unified way. Thus this work\nintroduces and explores the universal representation learning, i.e., embeddings\nof different levels of linguistic unit in a uniform vector space through a\ntask-independent evaluation. We present our approach of constructing analogy\ndatasets in terms of words, phrases and sentences and experiment with multiple\nrepresentation models to examine geometric properties of the learned vector\nspace. Then we empirically verify that well pre-trained Transformer models\nincorporated with appropriate training settings may effectively yield universal\nrepresentation. Especially, our implementation of fine-tuning ALBERT on NLI and\nPPDB datasets achieves the highest accuracy on analogy tasks in different\nlanguage levels. Further experiments on the insurance FAQ task show\neffectiveness of universal representation models in real-world applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04707v1",
            "date": "2020-09-10 07:47:01+00:00",
            "title": "On Target Segmentation for Direct Speech Translation",
            "authors": [
                "Mattia Antonino Di Gangi",
                "Marco Gaido",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Recent studies on direct speech translation show continuous improvements by\nmeans of data augmentation techniques and bigger deep learning models. While\nthese methods are helping to close the gap between this new approach and the\nmore traditional cascaded one, there are many incongruities among different\nstudies that make it difficult to assess the state of the art. Surprisingly,\none point of discussion is the segmentation of the target text. Character-level\nsegmentation has been initially proposed to obtain an open vocabulary, but it\nresults on long sequences and long training time. Then, subword-level\nsegmentation became the state of the art in neural machine translation as it\nproduces shorter sequences that reduce the training time, while being superior\nto word-level models. As such, recent works on speech translation started using\ntarget subwords despite the initial use of characters and some recent claims of\nbetter results at the character level. In this work, we perform an extensive\ncomparison of the two methods on three benchmarks covering 8 language\ndirections and multilingual training. Subword-level segmentation compares\nfavorably in all settings, outperforming its character-level counterpart in a\nrange of 1 to 3 BLEU points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04732v1",
            "date": "2020-09-10 08:55:25+00:00",
            "title": "Analyze the Effects of Weighting Functions on Cost Function in the Glove Model",
            "authors": [
                "Trieu Hai Nguyen"
            ],
            "abstract": "When dealing with the large vocabulary size and corpus size, the run-time for\ntraining Glove model is long, it can even be up to several dozen hours for\ndata, which is approximately 500MB in size. As a result, finding and selecting\nthe optimal parameters for the weighting function create many difficulties for\nweak hardware. Of course, to get the best results, we need to test benchmarks\nmany times. In order to solve this problem, we derive a weighting function,\nwhich can save time for choosing parameters and making benchmarks. It also\nallows one to obtain nearly similar accuracy at the same given time without\nconcern for experimentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04798v1",
            "date": "2020-09-10 12:06:48+00:00",
            "title": "The Grievance Dictionary: Understanding Threatening Language Use",
            "authors": [
                "Isabelle van der Vegt",
                "Maximilian Mozes",
                "Bennett Kleinberg",
                "Paul Gill"
            ],
            "abstract": "This paper introduces the Grievance Dictionary, a psycholinguistic dictionary\nwhich can be used to automatically understand language use in the context of\ngrievance-fuelled violence threat assessment. We describe the development the\ndictionary, which was informed by suggestions from experienced threat\nassessment practitioners. These suggestions and subsequent human and\ncomputational word list generation resulted in a dictionary of 20,502 words\nannotated by 2,318 participants. The dictionary was validated by applying it to\ntexts written by violent and non-violent individuals, showing strong evidence\nfor a difference between populations in several dictionary categories. Further\nclassification tasks showed promising performance, but future improvements are\nstill needed. Finally, we provide instructions and suggestions for the use of\nthe Grievance Dictionary by security professionals and (violence) researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04953v1",
            "date": "2020-09-10 15:49:24+00:00",
            "title": "Classification of descriptions and summary using multiple passes of statistical and natural language toolkits",
            "authors": [
                "Saumya Banthia",
                "Anantha Sharma"
            ],
            "abstract": "This document describes a possible approach that can be used to check the\nrelevance of a summary / definition of an entity with respect to its name. This\nclassifier focuses on the relevancy of an entity's name to its summary /\ndefinition, in other words, it is a name relevance check. The percentage score\nobtained from this approach can be used either on its own or used to supplement\nscores obtained from other metrics to arrive upon a final classification; at\nthe end of the document, potential improvements have also been outlined. The\ndataset that this document focuses on achieving an objective score is a list of\npackage names and their respective summaries (sourced from pypi.org).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05021v1",
            "date": "2020-09-10 17:38:32+00:00",
            "title": "Investigating Gender Bias in BERT",
            "authors": [
                "Rishabh Bhardwaj",
                "Navonil Majumder",
                "Soujanya Poria"
            ],
            "abstract": "Contextual language models (CLMs) have pushed the NLP benchmarks to a new\nheight. It has become a new norm to utilize CLM provided word embeddings in\ndownstream tasks such as text classification. However, unless addressed, CLMs\nare prone to learn intrinsic gender-bias in the dataset. As a result,\npredictions of downstream NLP models can vary noticeably by varying gender\nwords, such as replacing \"he\" to \"she\", or even gender-neutral words. In this\npaper, we focus our analysis on a popular CLM, i.e., BERT. We analyse the\ngender-bias it induces in five downstream tasks related to emotion and\nsentiment intensity prediction. For each task, we train a simple regressor\nutilizing BERT's word embeddings. We then evaluate the gender-bias in\nregressors using an equity evaluation corpus. Ideally and from the specific\ndesign, the models should discard gender informative features from the input.\nHowever, the results show a significant dependence of the system's predictions\non gender-particular words and phrases. We claim that such biases can be\nreduced by removing genderspecific features from word embedding. Hence, for\neach layer in BERT, we identify directions that primarily encode gender\ninformation. The space formed by such directions is referred to as the gender\nsubspace in the semantic space of word embeddings. We propose an algorithm that\nfinds fine-grained gender directions, i.e., one primary direction for each BERT\nlayer. This obviates the need of realizing gender subspace in multiple\ndimensions and prevents other crucial information from being omitted.\nExperiments show that removing embedding components in such directions achieves\ngreat success in reducing BERT-induced bias in the downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05092v2",
            "date": "2020-09-10 18:51:48+00:00",
            "title": "Dialogue Relation Extraction with Document-level Heterogeneous Graph Attention Networks",
            "authors": [
                "Hui Chen",
                "Pengfei Hong",
                "Wei Han",
                "Navonil Majumder",
                "Soujanya Poria"
            ],
            "abstract": "Dialogue relation extraction (DRE) aims to detect the relation between two\nentities mentioned in a multi-party dialogue. It plays an important role in\nconstructing knowledge graphs from conversational data increasingly abundant on\nthe internet and facilitating intelligent dialogue system development. The\nprior methods of DRE do not meaningfully leverage speaker information-they just\nprepend the utterances with the respective speaker names. Thus, they fail to\nmodel the crucial inter-speaker relations that may give additional context to\nrelevant argument entities through pronouns and triggers. We, however, present\na graph attention network-based method for DRE where a graph, that contains\nmeaningfully connected speaker, entity, entity-type, and utterance nodes, is\nconstructed. This graph is fed to a graph attention network for context\npropagation among relevant nodes, which effectively captures the dialogue\ncontext. We empirically show that this graph-based approach quite effectively\ncaptures the relations between different entity pairs in a dialogue as it\noutperforms the state-of-the-art approaches by a significant margin on the\nbenchmark dataset DialogRE. Our code is released at:\nhttps://github.com/declare-lab/dialog-HGAT",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05128v1",
            "date": "2020-09-10 19:59:08+00:00",
            "title": "RadLex Normalization in Radiology Reports",
            "authors": [
                "Surabhi Datta",
                "Jordan Godfrey-Stovall",
                "Kirk Roberts"
            ],
            "abstract": "Radiology reports have been widely used for extraction of various clinically\nsignificant information about patients' imaging studies. However, limited\nresearch has focused on standardizing the entities to a common\nradiology-specific vocabulary. Further, no study to date has attempted to\nleverage RadLex for standardization. In this paper, we aim to normalize a\ndiverse set of radiological entities to RadLex terms. We manually construct a\nnormalization corpus by annotating entities from three types of reports. This\ncontains 1706 entity mentions. We propose two deep learning-based NLP methods\nbased on a pre-trained language model (BERT) for automatic normalization.\nFirst, we employ BM25 to retrieve candidate concepts for the BERT-based models\n(re-ranker and span detector) to predict the normalized concept. The results\nare promising, with the best accuracy (78.44%) obtained by the span detector.\nAdditionally, we discuss the challenges involved in corpus construction and\npropose new RadLex terms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05166v3",
            "date": "2020-09-10 22:42:15+00:00",
            "title": "FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding",
            "authors": [
                "Yuwei Fang",
                "Shuohang Wang",
                "Zhe Gan",
                "Siqi Sun",
                "Jingjing Liu"
            ],
            "abstract": "Large-scale cross-lingual language models (LM), such as mBERT, Unicoder and\nXLM, have achieved great success in cross-lingual representation learning.\nHowever, when applied to zero-shot cross-lingual transfer tasks, most existing\nmethods use only single-language input for LM finetuning, without leveraging\nthe intrinsic cross-lingual alignment between different languages that proves\nessential for multilingual tasks. In this paper, we propose FILTER, an enhanced\nfusion method that takes cross-lingual data as input for XLM finetuning.\nSpecifically, FILTER first encodes text input in the source language and its\ntranslation in the target language independently in the shallow layers, then\nperforms cross-language fusion to extract multilingual knowledge in the\nintermediate layers, and finally performs further language-specific encoding.\nDuring inference, the model makes predictions based on the text input in the\ntarget language and its translation in the source language. For simple tasks\nsuch as classification, translated text in the target language shares the same\nlabel as the source language. However, this shared label becomes less accurate\nor even unavailable for more complex tasks such as question answering, NER and\nPOS tagging. To tackle this issue, we further propose an additional\nKL-divergence self-teaching loss for model training, based on auto-generated\nsoft pseudo-labels for translated text in the target language. Extensive\nexperiments demonstrate that FILTER achieves new state of the art on two\nchallenging multilingual multi-task benchmarks, XTREME and XGLUE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05167v1",
            "date": "2020-09-10 22:44:29+00:00",
            "title": "Accelerating Real-Time Question Answering via Question Generation",
            "authors": [
                "Yuwei Fang",
                "Shuohang Wang",
                "Zhe Gan",
                "Siqi Sun",
                "Jingjing Liu"
            ],
            "abstract": "Existing approaches to real-time question answering (RTQA) rely on learning\nthe representations of only key phrases in the documents, then matching them\nwith the question representation to derive answer. However, such approach is\nbottlenecked by the encoding time of real-time questions, thus suffering from\ndetectable latency in deployment for large-volume traffic. To accelerate RTQA\nfor practical use, we present Ocean-Q (an Ocean of Questions), a novel approach\nthat leverages question generation (QG) for RTQA. Ocean-Q introduces a QG model\nto generate a large pool of question-answer (QA) pairs offline, then in real\ntime matches an input question with the candidate QA pool to predict the answer\nwithout question encoding. To further improve QG quality, we propose a new data\naugmentation method and leverage multi-task learning and diverse beam search to\nboost RTQA performance. Experiments on SQuAD(-open) and HotpotQA benchmarks\ndemonstrate that Ocean-Q is able to accelerate the fastest state-of-the-art\nRTQA system by 4X times, with only a 3+% accuracy drop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05289v1",
            "date": "2020-09-11 08:44:14+00:00",
            "title": "UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT",
            "authors": [
                "Andrei Paraschiv",
                "Dumitru-Clementin Cercel",
                "Mihai Dascalu"
            ],
            "abstract": "Manipulative and misleading news have become a commodity for some online news\noutlets and these news have gained a significant impact on the global mindset\nof people. Propaganda is a frequently employed manipulation method having as\ngoal to influence readers by spreading ideas meant to distort or manipulate\ntheir opinions. This paper describes our participation in the SemEval-2020,\nTask 11: Detection of Propaganda Techniques in News Articles competition. Our\napproach considers specializing a pre-trained BERT model on propagandistic and\nhyperpartisan news articles, enabling it to create more adequate\nrepresentations for the two subtasks, namely propaganda Span Identification\n(SI) and propaganda Technique Classification (TC). Our proposed system achieved\na F1-score of 46.060% in subtask SI, ranking 5th in the leaderboard from 36\nteams and a micro-averaged F1 score of 54.302% for subtask TC, ranking 19th\nfrom 32 teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05387v3",
            "date": "2020-09-11 12:21:41+00:00",
            "title": "IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding",
            "authors": [
                "Bryan Wilie",
                "Karissa Vincentio",
                "Genta Indra Winata",
                "Samuel Cahyawijaya",
                "Xiaohong Li",
                "Zhi Yuan Lim",
                "Sidik Soleman",
                "Rahmad Mahendra",
                "Pascale Fung",
                "Syafri Bahar",
                "Ayu Purwarianti"
            ],
            "abstract": "Although Indonesian is known to be the fourth most frequently used language\nover the internet, the research progress on this language in the natural\nlanguage processing (NLP) is slow-moving due to a lack of available resources.\nIn response, we introduce the first-ever vast resource for the training,\nevaluating, and benchmarking on Indonesian natural language understanding\n(IndoNLU) tasks. IndoNLU includes twelve tasks, ranging from single sentence\nclassification to pair-sentences sequence labeling with different levels of\ncomplexity. The datasets for the tasks lie in different domains and styles to\nensure task diversity. We also provide a set of Indonesian pre-trained models\n(IndoBERT) trained from a large and clean Indonesian dataset Indo4B collected\nfrom publicly available sources such as social media texts, blogs, news, and\nwebsites. We release baseline models for all twelve tasks, as well as the\nframework for benchmark evaluation, and thus it enables everyone to benchmark\ntheir system performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05426v4",
            "date": "2020-09-11 13:21:28+00:00",
            "title": "Semantic Relations and Deep Learning",
            "authors": [
                "Vivi Nastase",
                "Stan Szpakowicz"
            ],
            "abstract": "The second edition of \"Semantic Relations Between Nominals\" by Vivi Nastase,\nStan Szpakowicz, Preslav Nakov and Diarmuid \\'O S\\'eaghdha has been published\nin April 2021 by Morgan & Claypool\n(www.morganclaypoolpublishers.com/catalog_Orig/product_info.php?products_id=1627).\nA new Chapter 5 of the book, by Vivi Nastase and Stan Szpakowicz, discusses\nrelation classification/extraction in the deep-learning paradigm which arose\nafter the first edition appeared. This is Chapter 5, made public by the kind\npermission of Morgan & Claypool.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05460v2",
            "date": "2020-09-11 14:12:54+00:00",
            "title": "Robust Neural Machine Translation: Modeling Orthographic and Interpunctual Variation",
            "authors": [
                "Toms Bergmanis",
                "Art\u016brs Stafanovi\u010ds",
                "M\u0101rcis Pinnis"
            ],
            "abstract": "Neural machine translation systems typically are trained on curated corpora\nand break when faced with non-standard orthography or punctuation. Resilience\nto spelling mistakes and typos, however, is crucial as machine translation\nsystems are used to translate texts of informal origins, such as chat\nconversations, social media posts and web pages. We propose a simple generative\nnoise model to generate adversarial examples of ten different types. We use\nthese to augment machine translation systems' training data and show that, when\ntested on noisy data, systems trained using adversarial examples perform almost\nas well as when translating clean data, while baseline systems' performance\ndrops by 2-3 BLEU points. To measure the robustness and noise invariance of\nmachine translation systems' outputs, we use the average translation edit rate\nbetween the translation of the original sentence and its noised variants. Using\nthis measure, we show that systems trained on adversarial examples on average\nyield 50% consistency improvements when compared to baselines trained on clean\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05639v2",
            "date": "2020-09-11 19:48:42+00:00",
            "title": "Solving Arithmetic Word Problems by Scoring Equations with Recursive Neural Networks",
            "authors": [
                "Klim Zaporojets",
                "Giannis Bekoulis",
                "Johannes Deleu",
                "Thomas Demeester",
                "Chris Develder"
            ],
            "abstract": "Solving arithmetic word problems is a cornerstone task in assessing language\nunderstanding and reasoning capabilities in NLP systems. Recent works use\nautomatic extraction and ranking of candidate solution equations providing the\nanswer to arithmetic word problems. In this work, we explore novel approaches\nto score such candidate solution equations using tree-structured recursive\nneural network (Tree-RNN) configurations. The advantage of this Tree-RNN\napproach over using more established sequential representations, is that it can\nnaturally capture the structure of the equations. Our proposed method consists\nof transforming the mathematical expression of the equation into an expression\ntree. Further, we encode this tree into a Tree-RNN by using different Tree-LSTM\narchitectures. Experimental results show that our proposed method (i) improves\noverall performance with more than 3% accuracy points compared to previous\nstate-of-the-art, and with over 15% points on a subset of problems that require\nmore complex reasoning, and (ii) outperforms sequential LSTMs by 4% accuracy\npoints on such more complex problems.",
            "journal ref": "Expert Systems with Applications, 174 (2021) 114704",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05737v1",
            "date": "2020-09-12 07:01:12+00:00",
            "title": "Syntax Role for Neural Semantic Role Labeling",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Shexia He",
                "Jiaxun Cai"
            ],
            "abstract": "Semantic role labeling (SRL) is dedicated to recognizing the semantic\npredicate-argument structure of a sentence. Previous studies in terms of\ntraditional models have shown syntactic information can make remarkable\ncontributions to SRL performance; however, the necessity of syntactic\ninformation was challenged by a few recent neural SRL studies that demonstrate\nimpressive performance without syntactic backbones and suggest that syntax\ninformation becomes much less important for neural semantic role labeling,\nespecially when paired with recent deep neural network and large-scale\npre-trained language models. Despite this notion, the neural SRL field still\nlacks a systematic and full investigation on the relevance of syntactic\ninformation in SRL, for both dependency and both monolingual and multilingual\nsettings. This paper intends to quantify the importance of syntactic\ninformation for neural SRL in the deep learning framework. We introduce three\ntypical SRL frameworks (baselines), sequence-based, tree-based, and\ngraph-based, which are accompanied by two categories of exploiting syntactic\ninformation: syntax pruning-based and syntax feature-based. Experiments are\nconducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages\navailable, and results show that neural SRL models can still benefit from\nsyntactic information under certain conditions. Furthermore, we show the\nquantitative significance of syntax to neural SRL models together with a\nthorough empirical survey using existing models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05781v2",
            "date": "2020-09-12 12:53:47+00:00",
            "title": "Intent Detection with WikiHow",
            "authors": [
                "Li Zhang",
                "Qing Lyu",
                "Chris Callison-Burch"
            ],
            "abstract": "Modern task-oriented dialog systems need to reliably understand users'\nintents. Intent detection is most challenging when moving to new domains or new\nlanguages, since there is little annotated data. To address this challenge, we\npresent a suite of pretrained intent detection models. Our models are able to\npredict a broad range of intended goals from many actions because they are\ntrained on wikiHow, a comprehensive instructional website. Our models achieve\nstate-of-the-art results on the Snips dataset, the Schema-Guided Dialogue\ndataset, and all 3 languages of the Facebook multilingual dialog datasets. Our\nmodels also demonstrate strong zero- and few-shot performance, reaching over\n75% accuracy using only 100 training examples in all datasets.",
            "journal ref": "Proceedings of the 1st Conference of the Asia-Pacific Chapter of\n  the Association for Computational Linguistics and the 10th International\n  Joint Conference on Natural Language Processing (2020) 328-333",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05782v1",
            "date": "2020-09-12 12:59:54+00:00",
            "title": "CIA_NITT at WNUT-2020 Task 2: Classification of COVID-19 Tweets Using Pre-trained Language Models",
            "authors": [
                "Yandrapati Prakash Babu",
                "Rajagopal Eswari"
            ],
            "abstract": "This paper presents our models for WNUT 2020 shared task2. The shared task2\ninvolves identification of COVID-19 related informative tweets. We treat this\nas binary text classification problem and experiment with pre-trained language\nmodels. Our first model which is based on CT-BERT achieves F1-score of 88.7%\nand second model which is an ensemble of CT-BERT, RoBERTa and SVM achieves\nF1-score of 88.52%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05831v2",
            "date": "2020-09-12 17:20:01+00:00",
            "title": "Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge",
            "authors": [
                "Kai Sun",
                "Dian Yu",
                "Jianshu Chen",
                "Dong Yu",
                "Claire Cardie"
            ],
            "abstract": "In this paper, we aim to extract commonsense knowledge to improve machine\nreading comprehension. We propose to represent relations implicitly by\nsituating structured knowledge in a context instead of relying on a pre-defined\nset of relations, and we call it contextualized knowledge. Each piece of\ncontextualized knowledge consists of a pair of interrelated verbal and\nnonverbal messages extracted from a script and the scene in which they occur as\ncontext to implicitly represent the relation between the verbal and nonverbal\nmessages, which are originally conveyed by different modalities within the\nscript. We propose a two-stage fine-tuning strategy to use the large-scale\nweakly-labeled data based on a single type of contextualized knowledge and\nemploy a teacher-student paradigm to inject multiple types of contextualized\nknowledge into a student machine reader. Experimental results demonstrate that\nour method outperforms a state-of-the-art baseline by a 4.3% improvement in\naccuracy on the machine reading comprehension dataset C^3, wherein most of the\nquestions require unstated prior knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05935v1",
            "date": "2020-09-13 06:53:40+00:00",
            "title": "Combining Word and Character Vector Representation on Neural Machine Translation",
            "authors": [
                "K. M. Shahih",
                "Ayu Purwarianti"
            ],
            "abstract": "This paper describes combinations of word vector representation and character\nvector representation in English-Indonesian neural machine translation (NMT).\nSix configurations of NMT models were built with different input vector\nrepresentations: word-based, combination of word and character representation\nusing bidirectional LSTM(bi-LSTM), combination of word and character\nrepresentation using CNN, combination of word and character representation by\ncombining bi-LSTM and CNN by three different vector operations: addition,\npointwise multiplication, and averaging. The experiment results showed that NMT\nmodels with concatenation of word and character representation obtained BLEU\nscore higher than baseline model, ranging from 9.14 points to 11.65 points, for\nall models that combining both word and character representation, except the\nmodel that combining word and character representation using both bi-LSTM and\nCNN by addition operation. The highest BLEU score achieved was 42.48 compared\nto the 30.83 of the baseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06040v1",
            "date": "2020-09-13 16:42:18+00:00",
            "title": "Span-based Semantic Parsing for Compositional Generalization",
            "authors": [
                "Jonathan Herzig",
                "Jonathan Berant"
            ],
            "abstract": "Despite the success of sequence-to-sequence (seq2seq) models in semantic\nparsing, recent work has shown that they fail in compositional generalization,\ni.e., the ability to generalize to new structures built of components observed\nduring training. In this work, we posit that a span-based parser should lead to\nbetter compositional generalization. we propose SpanBasedSP, a parser that\npredicts a span tree over an input utterance, explicitly encoding how partial\nprograms compose over spans in the input. SpanBasedSP extends Pasupat et al.\n(2019) to be comparable to seq2seq models by (i) training from programs,\nwithout access to gold trees, treating trees as latent variables, (ii) parsing\na class of non-projective trees through an extension to standard CKY. On\nGeoQuery, SCAN and CLOSURE datasets, SpanBasedSP performs similarly to strong\nseq2seq baselines on random splits, but dramatically improves performance\ncompared to baselines on splits that require compositional generalization: from\n$69.8 \\rightarrow 95.3$ average accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06097v1",
            "date": "2020-09-13 22:09:30+00:00",
            "title": "Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding",
            "authors": [
                "Shuohang Wang",
                "Luowei Zhou",
                "Zhe Gan",
                "Yen-Chun Chen",
                "Yuwei Fang",
                "Siqi Sun",
                "Yu Cheng",
                "Jingjing Liu"
            ],
            "abstract": "Transformer has become ubiquitous in the deep learning field. One of the key\ningredients that destined its success is the self-attention mechanism, which\nallows fully-connected contextual encoding over input tokens. However, despite\nits effectiveness in modeling short sequences, self-attention suffers when\nhandling inputs with extreme long-range dependencies, as its complexity grows\nquadratically with respect to the sequence length. Therefore, long sequences\nare often encoded by Transformer in chunks using a sliding window. In this\npaper, we propose Cluster-Former, a novel clustering-based sparse Transformer\nto perform attention across chunked sequences. Our proposed method allows\ninformation integration beyond local windows, which is especially beneficial\nfor question answering (QA) and language modeling tasks that rely on long-range\ndependencies. Experiments show that Cluster-Former achieves state-of-the-art\nperformance on several major QA benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06206v3",
            "date": "2020-09-14 05:24:28+00:00",
            "title": "On Robustness and Bias Analysis of BERT-based Relation Extraction",
            "authors": [
                "Luoqiu Li",
                "Xiang Chen",
                "Hongbin Ye",
                "Zhen Bi",
                "Shumin Deng",
                "Ningyu Zhang",
                "Huajun Chen"
            ],
            "abstract": "Fine-tuning pre-trained models have achieved impressive performance on\nstandard natural language processing benchmarks. However, the resultant model\ngeneralizability remains poorly understood. We do not know, for example, how\nexcellent performance can lead to the perfection of generalization models. In\nthis study, we analyze a fine-tuned BERT model from different perspectives\nusing relation extraction. We also characterize the differences in\ngeneralization techniques according to our proposed improvements. From\nempirical experimentation, we find that BERT suffers a bottleneck in terms of\nrobustness by way of randomizations, adversarial and counterfactual tests, and\nbiases (i.e., selection and semantic). These findings highlight opportunities\nfor future improvements. Our open-sourced testbed DiagnoseRE is available in\n\\url{https://github.com/zjunlp/DiagnoseRE}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06207v7",
            "date": "2020-09-14 05:29:24+00:00",
            "title": "Contrastive Triple Extraction with Generative Transformer",
            "authors": [
                "Hongbin Ye",
                "Ningyu Zhang",
                "Shumin Deng",
                "Mosha Chen",
                "Chuanqi Tan",
                "Fei Huang",
                "Huajun Chen"
            ],
            "abstract": "Triple extraction is an essential task in information extraction for natural\nlanguage processing and knowledge graph construction. In this paper, we revisit\nthe end-to-end triple extraction task for sequence generation. Since generative\ntriple extraction may struggle to capture long-term dependencies and generate\nunfaithful triples, we introduce a novel model, contrastive triple extraction\nwith a generative transformer. Specifically, we introduce a single shared\ntransformer module for encoder-decoder-based generation. To generate faithful\nresults, we propose a novel triplet contrastive training object. Moreover, we\nintroduce two mechanisms to further improve model performance (i.e., batch-wise\ndynamic attention-masking and triple-wise calibration). Experimental results on\nthree datasets (i.e., NYT, WebNLG, and MIE) show that our approach achieves\nbetter performance than that of baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06265v1",
            "date": "2020-09-14 08:44:46+00:00",
            "title": "Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues",
            "authors": [
                "Ruijian Xu",
                "Chongyang Tao",
                "Daxin Jiang",
                "Xueliang Zhao",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Building an intelligent dialogue system with the ability to select a proper\nresponse according to a multi-turn context is a great challenging task.\nExisting studies focus on building a context-response matching model with\nvarious neural architectures or PLMs and typically learning with a single\nresponse prediction task. These approaches overlook many potential training\nsignals contained in dialogue data, which might be beneficial for context\nunderstanding and produce better features for response prediction. Besides, the\nresponse retrieved from existing dialogue systems supervised by the\nconventional way still faces some critical challenges, including incoherence\nand inconsistency. To address these issues, in this paper, we propose learning\na context-response matching model with auxiliary self-supervised tasks designed\nfor the dialogue data based on pre-trained language models. Specifically, we\nintroduce four self-supervised tasks including next session prediction,\nutterance restoration, incoherence detection and consistency discrimination,\nand jointly train the PLM-based response selection model with these auxiliary\ntasks in a multi-task manner. By this means, the auxiliary tasks can guide the\nlearning of the matching model to achieve a better local optimum and select a\nmore proper response. Experiment results on two benchmarks indicate that the\nproposed auxiliary self-supervised tasks bring significant improvement for\nmulti-turn response selection in retrieval-based dialogues, and our model\nachieves new state-of-the-art results on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06376v1",
            "date": "2020-09-05 19:07:17+00:00",
            "title": "Analysis and representation of Igbo text document for a text-based system",
            "authors": [
                "Ifeanyi-Reuben Nkechi J.",
                "Ugwu Chidiebere",
                "Adegbola Tunde"
            ],
            "abstract": "The advancement in Information Technology (IT) has assisted in inculcating\nthe three Nigeria major languages in text-based application such as text\nmining, information retrieval and natural language processing. The interest of\nthis paper is the Igbo language, which uses compounding as a common type of\nword formation and as well has many vocabularies of compound words. The issues\nof collocation, word ordering and compounding play high role in Igbo language.\nThe ambiguity in dealing with these compound words has made the representation\nof Igbo language text document very difficult because this cannot be addressed\nusing the most common and standard approach of the Bag-Of-Words (BOW) model of\ntext representation, which ignores the word order and relation. However, this\ncause for a concern and the need to develop an improved model to capture this\nsituation. This paper presents the analysis of Igbo language text document,\nconsidering its compounding nature and describes its representation with the\nWord-based N-gram model to properly prepare it for any text-based application.\nThe result shows that Bigram and Trigram n-gram text representation models\nprovide more semantic information as well addresses the issues of compounding,\nword ordering and collocations which are the major language peculiarities in\nIgbo. They are likely to give better performance when used in any Igbo\ntext-based system.",
            "journal ref": "International Journal of Data Mining Techniques and Applications\n  (IJDMTA). Volume 06 Issue 01, June 2017, Page No 26-32",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06451v1",
            "date": "2020-09-14 14:07:50+00:00",
            "title": "Development of a Dataset and a Deep Learning Baseline Named Entity Recognizer for Three Low Resource Languages: Bhojpuri, Maithili and Magahi",
            "authors": [
                "Rajesh Kumar Mundotiya",
                "Shantanu Kumar",
                "Ajeet kumar",
                "Umesh Chandra Chaudhary",
                "Supriya Chauhan",
                "Swasti Mishra",
                "Praveen Gatla",
                "Anil Kumar Singh"
            ],
            "abstract": "In Natural Language Processing (NLP) pipelines, Named Entity Recognition\n(NER) is one of the preliminary problems, which marks proper nouns and other\nnamed entities such as Location, Person, Organization, Disease etc. Such\nentities, without a NER module, adversely affect the performance of a machine\ntranslation system. NER helps in overcoming this problem by recognising and\nhandling such entities separately, although it can be useful in Information\nExtraction systems also. Bhojpuri, Maithili and Magahi are low resource\nlanguages, usually known as Purvanchal languages. This paper focuses on the\ndevelopment of a NER benchmark dataset for the Machine Translation systems\ndeveloped to translate from these languages to Hindi by annotating parts of\ntheir available corpora. Bhojpuri, Maithili and Magahi corpora of sizes 228373,\n157468 and 56190 tokens, respectively, were annotated using 22 entity labels.\nThe annotation considers coarse-grained annotation labels followed by the\ntagset used in one of the Hindi NER datasets. We also report a Deep Learning\nbased baseline that uses an LSTM-CNNs-CRF model. The lower baseline F1-scores\nfrom the NER tool obtained by using Conditional Random Fields models are 96.73\nfor Bhojpuri, 93.33 for Maithili and 95.04 for Magahi. The Deep Learning-based\ntechnique (LSTM-CNNs-CRF) achieved 96.25 for Bhojpuri, 93.33 for Maithili and\n95.44 for Magahi.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06819v2",
            "date": "2020-09-15 01:52:47+00:00",
            "title": "MatScIE: An automated tool for the generation of databases of methods and parameters used in the computational materials science literature",
            "authors": [
                "Souradip Guha",
                "Ankan Mullick",
                "Jatin Agrawal",
                "Swetarekha Ram",
                "Samir Ghui",
                "Seung-Cheol Lee",
                "Satadeep Bhattacharjee",
                "Pawan Goyal"
            ],
            "abstract": "The number of published articles in the field of materials science is growing\nrapidly every year. This comparatively unstructured data source, which contains\na large amount of information, has a restriction on its re-usability, as the\ninformation needed to carry out further calculations using the data in it must\nbe extracted manually. It is very important to obtain valid and contextually\ncorrect information from the online (offline) data, as it can be useful not\nonly to generate inputs for further calculations, but also to incorporate them\ninto a querying framework. Retaining this context as a priority, we have\ndeveloped an automated tool, MatScIE (Material Scince Information Extractor)\nthat can extract relevant information from material science literature and make\na structured database that is much easier to use for material simulations.\nSpecifically, we extract the material details, methods, code, parameters, and\nstructure from the various research articles. Finally, we created a web\napplication where users can upload published articles and view/download the\ninformation obtained from this tool and can create their own databases for\ntheir personal uses.",
            "journal ref": "Computational Material Science, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06891v3",
            "date": "2020-09-15 07:09:26+00:00",
            "title": "Attention-aware Inference: Generating Summarization Texts According to Attention Distributions",
            "authors": [
                "Ye Ma",
                "Zixun Lan",
                "Lu Zong"
            ],
            "abstract": "This paper proposes a novel inference algorithm for seq-to-seq models. It\ncorrects beam search step-by-step via the optimal attention distribution to\nmake the generated text attend to source tokens in a controlled way.\nExperiments show the proposed attention-aware inference produces summaries\nrather differently from the beam search, and achieves promising improvements of\nhigher scores and greater conciseness. The algorithm is also proven robust as\nit remains to outperform beam search significantly even with corrupted\nattention distributions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06957v1",
            "date": "2020-09-15 10:01:27+00:00",
            "title": "High-order Refining for End-to-end Chinese Semantic Role Labeling",
            "authors": [
                "Hao Fei",
                "Yafeng Ren",
                "Donghong Ji"
            ],
            "abstract": "Current end-to-end semantic role labeling is mostly accomplished via\ngraph-based neural models. However, these all are first-order models, where\neach decision for detecting any predicate-argument pair is made in isolation\nwith local features. In this paper, we present a high-order refining mechanism\nto perform interaction between all predicate-argument pairs. Based on the\nbaseline graph model, our high-order refining module learns higher-order\nfeatures between all candidate pairs via attention calculation, which are later\nused to update the original token representations. After several iterations of\nrefinement, the underlying token representations can be enriched with globally\ninteracted features. Our high-order model achieves state-of-the-art results on\nChinese SRL data, including CoNLL09 and Universal Proposition Bank, meanwhile\nrelieving the long-range dependency issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06978v1",
            "date": "2020-09-15 10:50:05+00:00",
            "title": "Dialogue Response Ranking Training with Large-Scale Human Feedback Data",
            "authors": [
                "Xiang Gao",
                "Yizhe Zhang",
                "Michel Galley",
                "Chris Brockett",
                "Bill Dolan"
            ],
            "abstract": "Existing open-domain dialog models are generally trained to minimize the\nperplexity of target human responses. However, some human replies are more\nengaging than others, spawning more followup interactions. Current\nconversational models are increasingly capable of producing turns that are\ncontext-relevant, but in order to produce compelling agents, these models need\nto be able to predict and optimize for turns that are genuinely engaging. We\nleverage social media feedback data (number of replies and upvotes) to build a\nlarge-scale training dataset for feedback prediction. To alleviate possible\ndistortion between the feedback and engagingness, we convert the ranking\nproblem to a comparison of response pairs which involve few confounding\nfactors. We trained DialogRPT, a set of GPT-2 based models on 133M pairs of\nhuman feedback data and the resulting ranker outperformed several baselines.\nParticularly, our ranker outperforms the conventional dialog perplexity\nbaseline with a large margin on predicting Reddit feedback. We finally combine\nthe feedback prediction models and a human-like scoring model to rank the\nmachine-generated dialog responses. Crowd-sourced human evaluation shows that\nour ranking method correlates better with real human preferences than baseline\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07032v1",
            "date": "2020-09-15 12:53:09+00:00",
            "title": "Noisy Self-Knowledge Distillation for Text Summarization",
            "authors": [
                "Yang Liu",
                "Sheng Shen",
                "Mirella Lapata"
            ],
            "abstract": "In this paper we apply self-knowledge distillation to text summarization\nwhich we argue can alleviate problems with maximum-likelihood training on\nsingle reference and noisy datasets. Instead of relying on one-hot annotation\nlabels, our student summarization model is trained with guidance from a teacher\nwhich generates smoothed labels to help regularize training. Furthermore, to\nbetter model uncertainty during training, we introduce multiple noise signals\nfor both teacher and student models. We demonstrate experimentally on three\nbenchmarks that our framework boosts the performance of both pretrained and\nnon-pretrained summarizers achieving state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07117v2",
            "date": "2020-09-15 14:17:53+00:00",
            "title": "Multi-Referenced Training for Dialogue Response Generation",
            "authors": [
                "Tianyu Zhao",
                "Tatsuya Kawahara"
            ],
            "abstract": "In open-domain dialogue response generation, a dialogue context can be\ncontinued with diverse responses, and the dialogue models should capture such\none-to-many relations. In this work, we first analyze the training objective of\ndialogue models from the view of Kullback-Leibler divergence (KLD) and show\nthat the gap between the real world probability distribution and the\nsingle-referenced data's probability distribution prevents the model from\nlearning the one-to-many relations efficiently. Then we explore approaches to\nmulti-referenced training in two aspects. Data-wise, we generate diverse pseudo\nreferences from a powerful pretrained model to build multi-referenced data that\nprovides a better approximation of the real-world distribution. Model-wise, we\npropose to equip variational models with an expressive prior, named linear\nGaussian model (LGM). Experimental results of automated evaluation and human\nevaluation show that the methods yield significant improvements over baselines.\nWe will release our code and data in\nhttps://github.com/ZHAOTING/dialog-processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07119v1",
            "date": "2020-09-15 14:20:04+00:00",
            "title": "Improving Joint Layer RNN based Keyphrase Extraction by Using Syntactical Features",
            "authors": [
                "Miftahul Mahfuzh",
                "Sidik Soleman",
                "Ayu Purwarianti"
            ],
            "abstract": "Keyphrase extraction as a task to identify important words or phrases from a\ntext, is a crucial process to identify main topics when analyzing texts from a\nsocial media platform. In our study, we focus on text written in Indonesia\nlanguage taken from Twitter. Different from the original joint layer recurrent\nneural network (JRNN) with output of one sequence of keywords and using only\nword embedding, here we propose to modify the input layer of JRNN to extract\nmore than one sequence of keywords by additional information of syntactical\nfeatures, namely part of speech, named entity types, and dependency structures.\nSince JRNN in general requires a large amount of data as the training examples\nand creating those examples is expensive, we used a data augmentation method to\nincrease the number of training examples. Our experiment had shown that our\nmethod outperformed the baseline methods. Our method achieved .9597 in accuracy\nand .7691 in F1.",
            "journal ref": "2019 International Conference of Advanced Informatics: Concepts,\n  Theory and Applications (ICAICTA)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07148v2",
            "date": "2020-09-15 15:02:28+00:00",
            "title": "Cascaded Semantic and Positional Self-Attention Network for Document Classification",
            "authors": [
                "Juyong Jiang",
                "Jie Zhang",
                "Kai Zhang"
            ],
            "abstract": "Transformers have shown great success in learning representations for\nlanguage modelling. However, an open challenge still remains on how to\nsystematically aggregate semantic information (word embedding) with positional\n(or temporal) information (word orders). In this work, we propose a new\narchitecture to aggregate the two sources of information using cascaded\nsemantic and positional self-attention network (CSPAN) in the context of\ndocument classification. The CSPAN uses a semantic self-attention layer\ncascaded with Bi-LSTM to process the semantic and positional information in a\nsequential manner, and then adaptively combine them together through a residue\nconnection. Compared with commonly used positional encoding schemes, CSPAN can\nexploit the interaction between semantics and word positions in a more\ninterpretable and adaptive manner, and the classification performance can be\nnotably improved while simultaneously preserving a compact model size and high\nconvergence rate. We evaluate the CSPAN model on several benchmark data sets\nfor document classification with careful ablation studies, and demonstrate the\nencouraging results compared with state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07162v1",
            "date": "2020-09-15 15:10:51+00:00",
            "title": "Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product",
            "authors": [
                "Tiangang Zhu",
                "Yue Wang",
                "Haoran Li",
                "Youzheng Wu",
                "Xiaodong He",
                "Bowen Zhou"
            ],
            "abstract": "Product attribute values are essential in many e-commerce scenarios, such as\ncustomer service robots, product recommendations, and product retrieval. While\nin the real world, the attribute values of a product are usually incomplete and\nvary over time, which greatly hinders the practical applications. In this\npaper, we propose a multimodal method to jointly predict product attributes and\nextract values from textual product descriptions with the help of the product\nimages. We argue that product attributes and values are highly correlated,\ne.g., it will be easier to extract the values on condition that the product\nattributes are given. Thus, we jointly model the attribute prediction and value\nextraction tasks from multiple aspects towards the interactions between\nattributes and values. Moreover, product images have distinct effects on our\ntasks for different product attributes and values. Thus, we selectively draw\nuseful visual information from product images to enhance our model. We annotate\na multimodal product attribute value dataset that contains 87,194 instances,\nand the experimental results on this dataset demonstrate that explicitly\nmodeling the relationship between attributes and values facilitates our method\nto establish the correspondence between them, and selectively utilizing visual\nproduct information is necessary for the task. Our code and dataset will be\nreleased to the public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07177v1",
            "date": "2020-09-15 15:30:14+00:00",
            "title": "Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Jason Lee",
                "Raphael Shu",
                "Kyunghyun Cho"
            ],
            "abstract": "We propose an efficient inference procedure for non-autoregressive machine\ntranslation that iteratively refines translation purely in the continuous\nspace. Given a continuous latent variable model for machine translation (Shu et\nal., 2020), we train an inference network to approximate the gradient of the\nmarginal log probability of the target sentence, using only the latent variable\nas input. This allows us to use gradient-based optimization to find the target\nsentence at inference time that approximately maximizes its marginal\nprobability. As each refinement step only involves computation in the latent\nspace of low dimensionality (we use 8 in our experiments), we avoid\ncomputational overhead incurred by existing non-autoregressive inference\nprocedures that often refine in token space. We compare our approach to a\nrecently proposed EM-like inference procedure (Shu et al., 2020) that optimizes\nin a hybrid space, consisting of both discrete and continuous variables. We\nevaluate our approach on WMT'14 En-De, WMT'16 Ro-En and IWSLT'16 De-En, and\nobserve two advantages over the EM-like inference: (1) it is computationally\nefficient, i.e. each refinement step is twice as fast, and (2) it is more\neffective, resulting in higher marginal probabilities and BLEU scores with the\nsame number of refinement steps. On WMT'14 En-De, for instance, our approach is\nable to decode 6.2 times faster than the autoregressive model with minimal\ndegradation to translation quality (0.9 BLEU).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07188v1",
            "date": "2020-09-15 15:52:21+00:00",
            "title": "Event Presence Prediction Helps Trigger Detection Across Languages",
            "authors": [
                "Parul Awasthy",
                "Tahira Naseem",
                "Jian Ni",
                "Taesun Moon",
                "Radu Florian"
            ],
            "abstract": "The task of event detection and classification is central to most information\nretrieval applications. We show that a Transformer based architecture can\neffectively model event extraction as a sequence labeling task. We propose a\ncombination of sentence level and token level training objectives that\nsignificantly boosts the performance of a BERT based event extraction model.\nOur approach achieves a new state-of-the-art performance on ACE 2005 data for\nEnglish and Chinese. We also test our model on ERE Spanish, achieving an\naverage gain of 2 absolute F1 points over prior best performing model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07310v3",
            "date": "2020-09-15 18:19:11+00:00",
            "title": "Simultaneous Machine Translation with Visual Context",
            "authors": [
                "Ozan Caglayan",
                "Julia Ive",
                "Veneta Haralampieva",
                "Pranava Madhyastha",
                "Lo\u00efc Barrault",
                "Lucia Specia"
            ],
            "abstract": "Simultaneous machine translation (SiMT) aims to translate a continuous input\ntext stream into another language with the lowest latency and highest quality\npossible. The translation thus has to start with an incomplete source text,\nwhich is read progressively, creating the need for anticipation. In this paper,\nwe seek to understand whether the addition of visual information can compensate\nfor the missing source context. To this end, we analyse the impact of different\nmultimodal approaches and visual features on state-of-the-art SiMT frameworks.\nOur results show that visual context is helpful and that visually-grounded\nmodels based on explicit object region information are much better than\ncommonly used global features, reaching up to 3 BLEU points improvement under\nlow latency scenarios. Our qualitative analysis illustrates cases where only\nthe multimodal systems are able to translate correctly from English into\ngender-marked languages, as well as deal with differences in word order, such\nas adjective-noun placement between English and French.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07317v1",
            "date": "2020-09-15 18:41:29+00:00",
            "title": "Cascaded Models for Better Fine-Grained Named Entity Recognition",
            "authors": [
                "Parul Awasthy",
                "Taesun Moon",
                "Jian Ni",
                "Radu Florian"
            ],
            "abstract": "Named Entity Recognition (NER) is an essential precursor task for many\nnatural language applications, such as relation extraction or event extraction.\nMuch of the NER research has been done on datasets with few classes of entity\ntypes (e.g. PER, LOC, ORG, MISC), but many real world applications (disaster\nrelief, complex event extraction, law enforcement) can benefit from a larger\nNER typeset. More recently, datasets were created that have hundreds to\nthousands of types of entities, sparking new lines of research (Sekine,\n2008;Ling and Weld, 2012; Gillick et al., 2014; Choiet al., 2018). In this\npaper we present a cascaded approach to labeling fine-grained NER, applying to\na newly released fine-grained NER dataset that was used in the TAC KBP 2019\nevaluation (Ji et al., 2019), inspired by the fact that training data is\navailable for some of the coarse labels. Using a combination of transformer\nnetworks, we show that performance can be improved by about 20 F1 absolute, as\ncompared with the straightforward model built on the full fine-grained types,\nand show that, surprisingly, using course-labeled data in three languages leads\nto an improvement in the English data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07364v2",
            "date": "2020-09-15 21:52:16+00:00",
            "title": "An information theoretic view on selecting linguistic probes",
            "authors": [
                "Zining Zhu",
                "Frank Rudzicz"
            ],
            "abstract": "There is increasing interest in assessing the linguistic knowledge encoded in\nneural representations. A popular approach is to attach a diagnostic classifier\n-- or \"probe\" -- to perform supervised classification from internal\nrepresentations. However, how to select a good probe is in debate. Hewitt and\nLiang (2019) showed that a high performance on diagnostic classification itself\nis insufficient, because it can be attributed to either \"the representation\nbeing rich in knowledge\", or \"the probe learning the task\", which Pimentel et\nal. (2020) challenged. We show this dichotomy is valid\ninformation-theoretically. In addition, we find that the methods to construct\nand select good probes proposed by the two papers, *control task* (Hewitt and\nLiang, 2019) and *control function* (Pimentel et al., 2020), are equivalent --\nthe errors of their approaches are identical (modulo irrelevant terms).\nEmpirically, these two selection criteria lead to results that highly agree\nwith each other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07365v2",
            "date": "2020-09-15 21:54:01+00:00",
            "title": "Fast semantic parsing with well-typedness guarantees",
            "authors": [
                "Matthias Lindemann",
                "Jonas Groschwitz",
                "Alexander Koller"
            ],
            "abstract": "AM dependency parsing is a linguistically principled method for neural\nsemantic parsing with high accuracy across multiple graphbanks. It relies on a\ntype system that models semantic valency but makes existing parsers slow. We\ndescribe an A* parser and a transition-based parser for AM dependency parsing\nwhich guarantee well-typedness and improve parsing speed by up to 3 orders of\nmagnitude, while maintaining or improving accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07382v2",
            "date": "2020-09-15 23:06:48+00:00",
            "title": "Multi-span Style Extraction for Generative Reading Comprehension",
            "authors": [
                "Junjie Yang",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "Generative machine reading comprehension (MRC) requires a model to generate\nwell-formed answers. For this type of MRC, answer generation method is crucial\nto the model performance. However, generative models, which are supposed to be\nthe right model for the task, in generally perform poorly. At the same time,\nsingle-span extraction models have been proven effective for extractive MRC,\nwhere the answer is constrained to a single span in the passage. Nevertheless,\nthey generally suffer from generating incomplete answers or introducing\nredundant words when applied to the generative MRC. Thus, we extend the\nsingle-span extraction method to multi-span, proposing a new framework which\nenables generative MRC to be smoothly solved as multi-span extraction. Thorough\nexperiments demonstrate that this novel approach can alleviate the dilemma\nbetween generative models and single-span models and produce answers with\nbetter-formed syntax and semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07397v1",
            "date": "2020-09-16 00:21:56+00:00",
            "title": "Arabic Opinion Mining Using a Hybrid Recommender System Approach",
            "authors": [
                "Fouzi Harrag",
                "Abdulmalik Salman Al-Salman",
                "Alaa Alquahtani"
            ],
            "abstract": "Recommender systems nowadays are playing an important role in the delivery of\nservices and information to users. Sentiment analysis (also known as opinion\nmining) is the process of determining the attitude of textual opinions, whether\nthey are positive, negative or neutral. Data sparsity is representing a big\nissue for recommender systems because of the insufficiency of user rating or\nabsence of data about users or items. This research proposed a hybrid approach\ncombining sentiment analysis and recommender systems to tackle the problem of\ndata sparsity problems by predicting the rating of products from users reviews\nusing text mining and NLP techniques. This research focuses especially on\nArabic reviews, where the model is evaluated using Opinion Corpus for Arabic\n(OCA) dataset. Our system was efficient, and it showed a good accuracy of\nnearly 85 percent in predicting rating from reviews",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07408v1",
            "date": "2020-09-16 01:07:07+00:00",
            "title": "Retrofitting Structure-aware Transformer Language Model for End Tasks",
            "authors": [
                "Hao Fei",
                "Yafeng Ren",
                "Donghong Ji"
            ],
            "abstract": "We consider retrofitting structure-aware Transformer-based language model for\nfacilitating end tasks by proposing to exploit syntactic distance to encode\nboth the phrasal constituency and dependency connection into the language\nmodel. A middle-layer structural learning strategy is leveraged for structure\nintegration, accomplished with main semantic task training under multi-task\nlearning scheme. Experimental results show that the retrofitted structure-aware\nTransformer language model achieves improved perplexity, meanwhile inducing\naccurate syntactic phrases. By performing structure-aware fine-tuning, our\nmodel achieves significant improvements for both semantic- and\nsyntactic-dependent tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07411v1",
            "date": "2020-09-16 01:30:21+00:00",
            "title": "Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP",
            "authors": [
                "Hao Fei",
                "Yafeng Ren",
                "Donghong Ji"
            ],
            "abstract": "Syntax has been shown useful for various NLP tasks, while existing work\nmostly encodes singleton syntactic tree using one hierarchical neural network.\nIn this paper, we investigate a simple and effective method, Knowledge\nDistillation, to integrate heterogeneous structure knowledge into a unified\nsequential LSTM encoder. Experimental results on four typical syntax-dependent\ntasks show that our method outperforms tree encoders by effectively integrating\nrich heterogeneous structure syntax, meanwhile reducing error propagation, and\nalso outperforms ensemble methods, in terms of both the efficiency and\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07465v5",
            "date": "2020-09-16 04:31:38+00:00",
            "title": "Answering Any-hop Open-domain Questions with Iterative Document Reranking",
            "authors": [
                "Ping Nie",
                "Yuyu Zhang",
                "Arun Ramamurthy",
                "Le Song"
            ],
            "abstract": "Existing approaches for open-domain question answering (QA) are typically\ndesigned for questions that require either single-hop or multi-hop reasoning,\nwhich make strong assumptions of the complexity of questions to be answered.\nAlso, multi-step document retrieval often incurs higher number of relevant but\nnon-supporting documents, which dampens the downstream noise-sensitive reader\nmodule for answer extraction. To address these challenges, we propose a unified\nQA framework to answer any-hop open-domain questions, which iteratively\nretrieves, reranks and filters documents, and adaptively determines when to\nstop the retrieval process. To improve the retrieval accuracy, we propose a\ngraph-based reranking model that perform multi-document interaction as the core\nof our iterative reranking framework. Our method consistently achieves\nperformance comparable to or better than the state-of-the-art on both\nsingle-hop and multi-hop open-domain QA datasets, including Natural Questions\nOpen, SQuAD Open, and HotpotQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07481v1",
            "date": "2020-09-16 05:58:00+00:00",
            "title": "Unsupervised Summarization by Jointly Extracting Sentences and Keywords",
            "authors": [
                "Zongyi Li",
                "Xiaoqing Zheng"
            ],
            "abstract": "We present RepRank, an unsupervised graph-based ranking model for extractive\nmulti-document summarization in which the similarity between words, sentences,\nand word-to-sentence can be estimated by the distances between their vector\nrepresentations in a unified vector space. In order to obtain desirable\nrepresentations, we propose a self-attention based learning method that\nrepresent a sentence by the weighted sum of its word embeddings, and the\nweights are concentrated to those words hopefully better reflecting the content\nof a document. We show that salient sentences and keywords can be extracted in\na joint and mutual reinforcement process using our learned representations, and\nprove that this process always converges to a unique solution leading to\nimprovement in performance. A variant of absorbing random walk and the\ncorresponding sampling-based algorithm are also described to avoid redundancy\nand increase diversity in the summaries. Experiment results with multiple\nbenchmark datasets show that RepRank achieved the best or comparable\nperformance in ROUGE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07489v1",
            "date": "2020-09-16 06:28:58+00:00",
            "title": "Graph-to-Sequence Neural Machine Translation",
            "authors": [
                "Sufeng Duan",
                "Hai Zhao",
                "Rui Wang"
            ],
            "abstract": "Neural machine translation (NMT) usually works in a seq2seq learning way by\nviewing either source or target sentence as a linear sequence of words, which\ncan be regarded as a special case of graph, taking words in the sequence as\nnodes and relationships between words as edges. In the light of the current NMT\nmodels more or less capture graph information among the sequence in a latent\nway, we present a graph-to-sequence model facilitating explicit graph\ninformation capturing. In detail, we propose a graph-based SAN-based NMT model\ncalled Graph-Transformer by capturing information of subgraphs of different\norders in every layers. Subgraphs are put into different groups according to\ntheir orders, and every group of subgraphs respectively reflect different\nlevels of dependency between words. For fusing subgraph representations, we\nempirically explore three methods which weight different groups of subgraphs of\ndifferent orders. Results of experiments on WMT14 English-German and IWSLT14\nGerman-English show that our method can effectively boost the Transformer with\nan improvement of 1.1 BLEU points on WMT14 English-German dataset and 1.0 BLEU\npoints on IWSLT14 German-English dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07502v2",
            "date": "2020-09-16 06:53:15+00:00",
            "title": "Contextualized Perturbation for Textual Adversarial Attack",
            "authors": [
                "Dianqi Li",
                "Yizhe Zhang",
                "Hao Peng",
                "Liqun Chen",
                "Chris Brockett",
                "Ming-Ting Sun",
                "Bill Dolan"
            ],
            "abstract": "Adversarial examples expose the vulnerabilities of natural language\nprocessing (NLP) models, and can be used to evaluate and improve their\nrobustness. Existing techniques of generating such examples are typically\ndriven by local heuristic rules that are agnostic to the context, often\nresulting in unnatural and ungrammatical outputs. This paper presents CLARE, a\nContextuaLized AdversaRial Example generation model that produces fluent and\ngrammatical outputs through a mask-then-infill procedure. CLARE builds on a\npre-trained masked language model and modifies the inputs in a context-aware\nmanner. We propose three contextualized perturbations, Replace, Insert and\nMerge, allowing for generating outputs of varied lengths. With a richer range\nof available strategies, CLARE is able to attack a victim model more\nefficiently with fewer edits. Extensive experiments and human evaluation\ndemonstrate that CLARE outperforms the baselines in terms of attack success\nrate, textual similarity, fluency and grammaticality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07602v1",
            "date": "2020-09-16 11:01:46+00:00",
            "title": "UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation",
            "authors": [
                "Jian Guan",
                "Minlie Huang"
            ],
            "abstract": "Despite the success of existing referenced metrics (e.g., BLEU and\nMoverScore), they correlate poorly with human judgments for open-ended text\ngeneration including story or dialog generation because of the notorious\none-to-many issue: there are many plausible outputs for the same input, which\nmay differ substantially in literal or semantics from the limited number of\ngiven references. To alleviate this issue, we propose UNION, a learnable\nunreferenced metric for evaluating open-ended story generation, which measures\nthe quality of a generated story without any reference. Built on top of BERT,\nUNION is trained to distinguish human-written stories from negative samples and\nrecover the perturbation in negative stories. We propose an approach of\nconstructing negative samples by mimicking the errors commonly observed in\nexisting NLG models, including repeated plots, conflicting logic, and\nlong-range incoherence. Experiments on two story datasets demonstrate that\nUNION is a reliable measure for evaluating the quality of generated stories,\nwhich correlates better with human judgments and is more generalizable than\nexisting state-of-the-art metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07610v3",
            "date": "2020-09-16 11:37:10+00:00",
            "title": "Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT",
            "authors": [
                "Alexandra Chronopoulou",
                "Dario Stojanovski",
                "Alexander Fraser"
            ],
            "abstract": "Using a language model (LM) pretrained on two languages with large\nmonolingual data in order to initialize an unsupervised neural machine\ntranslation (UNMT) system yields state-of-the-art results. When limited data is\navailable for one language, however, this method leads to poor translations. We\npresent an effective approach that reuses an LM that is pretrained only on the\nhigh-resource language. The monolingual LM is fine-tuned on both languages and\nis then used to initialize a UNMT model. To reuse the pretrained LM, we have to\nmodify its predefined vocabulary, to account for the new language. We therefore\npropose a novel vocabulary extension method. Our approach, RE-LM, outperforms a\ncompetitive cross-lingual pretraining model (XLM) in English-Macedonian (En-Mk)\nand English-Albanian (En-Sq), yielding more than +8.3 BLEU points for all four\ntranslation directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07615v2",
            "date": "2020-09-16 11:53:00+00:00",
            "title": "Neural Dialogue State Tracking with Temporally Expressive Networks",
            "authors": [
                "Junfan Chen",
                "Richong Zhang",
                "Yongyi Mao",
                "Jie Xu"
            ],
            "abstract": "Dialogue state tracking (DST) is an important part of a spoken dialogue\nsystem. Existing DST models either ignore temporal feature dependencies across\ndialogue turns or fail to explicitly model temporal state dependencies in a\ndialogue. In this work, we propose Temporally Expressive Networks (TEN) to\njointly model the two types of temporal dependencies in DST. The TEN model\nutilizes the power of recurrent networks and probabilistic graphical models.\nEvaluating on standard datasets, TEN is demonstrated to be effective in\nimproving the accuracy of turn-level-state prediction and the state\naggregation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07616v3",
            "date": "2020-09-16 11:54:15+00:00",
            "title": "Parallel Interactive Networks for Multi-Domain Dialogue State Generation",
            "authors": [
                "Junfan Chen",
                "Richong Zhang",
                "Yongyi Mao",
                "Jie Xu"
            ],
            "abstract": "The dependencies between system and user utterances in the same turn and\nacross different turns are not fully considered in existing multidomain\ndialogue state tracking (MDST) models. In this study, we argue that the\nincorporation of these dependencies is crucial for the design of MDST and\npropose Parallel Interactive Networks (PIN) to model these dependencies.\nSpecifically, we integrate an interactive encoder to jointly model the in-turn\ndependencies and cross-turn dependencies. The slot-level context is introduced\nto extract more expressive features for different slots. And a distributed copy\nmechanism is utilized to selectively copy words from historical system\nutterances or historical user utterances. Empirical studies demonstrated the\nsuperiority of the proposed PIN model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07690v2",
            "date": "2020-09-16 13:50:09+00:00",
            "title": "Reasoning about Goals, Steps, and Temporal Ordering with WikiHow",
            "authors": [
                "Li Zhang",
                "Qing Lyu",
                "Chris Callison-Burch"
            ],
            "abstract": "We propose a suite of reasoning tasks on two types of relations between\nprocedural events: goal-step relations (\"learn poses\" is a step in the larger\ngoal of \"doing yoga\") and step-step temporal relations (\"buy a yoga mat\"\ntypically precedes \"learn poses\"). We introduce a dataset targeting these two\nrelations based on wikiHow, a website of instructional how-to articles. Our\nhuman-validated test set serves as a reliable benchmark for commonsense\ninference, with a gap of about 10% to 20% between the performance of\nstate-of-the-art transformer models and human performance. Our\nautomatically-generated training set allows models to effectively transfer to\nout-of-domain tasks requiring knowledge of procedural events, with greatly\nimproved performances on SWAG, Snips, and the Story Cloze Test in zero- and\nfew-shot settings.",
            "journal ref": "Proceedings of the 2020 Conference on Empirical Methods in Natural\n  Language Processing (EMNLP) (2020) 4630-4639",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07715v1",
            "date": "2020-09-16 14:36:41+00:00",
            "title": "Knowledge Graphs for Multilingual Language Translation and Generation",
            "authors": [
                "Diego Moussallem"
            ],
            "abstract": "The Natural Language Processing (NLP) community has recently seen outstanding\nprogress, catalysed by the release of different Neural Network (NN)\narchitectures. Neural-based approaches have proven effective by significantly\nincreasing the output quality of a large number of automated solutions for NLP\ntasks (Belinkov and Glass, 2019). Despite these notable advancements, dealing\nwith entities still poses a difficult challenge as they are rarely seen in\ntraining data. Entities can be classified into two groups, i.e., proper nouns\nand common nouns. Proper nouns are also known as Named Entities (NE) and\ncorrespond to the name of people, organizations, or locations, e.g., John, WHO,\nor Canada. Common nouns describe classes of objects, e.g., spoon or cancer.\nBoth types of entities can be found in a Knowledge Graph (KG). Recent work has\nsuccessfully exploited the contribution of KGs in NLP tasks, such as Natural\nLanguage Inference (NLI) (KM et al.,2018) and Question Answering (QA) (Sorokin\nand Gurevych, 2018). Only a few works had exploited the benefits of KGs in\nNeural Machine Translation (NMT) when the work presented herein began.\nAdditionally, few works had studied the contribution of KGs to Natural Language\nGeneration (NLG) tasks. Moreover, the multilinguality also remained an open\nresearch area in these respective tasks (Young et al., 2018). In this thesis,\nwe focus on the use of KGs for machine translation and the generation of texts\nto deal with the problems caused by entities and consequently enhance the\nquality of automatically generated texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07728v2",
            "date": "2020-09-16 14:59:06+00:00",
            "title": "NABU $\\mathrm{-}$ Multilingual Graph-based Neural RDF Verbalizer",
            "authors": [
                "Diego Moussallem",
                "Dwaraknath Gnaneshwar",
                "Thiago Castro Ferreira",
                "Axel-Cyrille Ngonga Ngomo"
            ],
            "abstract": "The RDF-to-text task has recently gained substantial attention due to\ncontinuous growth of Linked Data. In contrast to traditional pipeline models,\nrecent studies have focused on neural models, which are now able to convert a\nset of RDF triples into text in an end-to-end style with promising results.\nHowever, English is the only language widely targeted. We address this research\ngap by presenting NABU, a multilingual graph-based neural model that verbalizes\nRDF data to German, Russian, and English. NABU is based on an encoder-decoder\narchitecture, uses an encoder inspired by Graph Attention Networks and a\nTransformer as decoder. Our approach relies on the fact that knowledge graphs\nare language-agnostic and they hence can be used to generate multilingual text.\nWe evaluate NABU in monolingual and multilingual settings on standard\nbenchmarking WebNLG datasets. Our results show that NABU outperforms\nstate-of-the-art approaches on English with 66.21 BLEU, and achieves consistent\nresults across all languages on the multilingual scenario with 56.04 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07783v3",
            "date": "2020-09-16 16:23:17+00:00",
            "title": "Generative Language-Grounded Policy in Vision-and-Language Navigation with Bayes' Rule",
            "authors": [
                "Shuhei Kurita",
                "Kyunghyun Cho"
            ],
            "abstract": "Vision-and-language navigation (VLN) is a task in which an agent is embodied\nin a realistic 3D environment and follows an instruction to reach the goal\nnode. While most of the previous studies have built and investigated a\ndiscriminative approach, we notice that there are in fact two possible\napproaches to building such a VLN agent: discriminative \\textit{and}\ngenerative. In this paper, we design and investigate a generative\nlanguage-grounded policy which uses a language model to compute the\ndistribution over all possible instructions i.e. all possible sequences of\nvocabulary tokens given action and the transition history. In experiments, we\nshow that the proposed generative approach outperforms the discriminative\napproach in the Room-2-Room (R2R) and Room-4-Room (R4R) datasets, especially in\nthe unseen environments. We further show that the combination of the generative\nand discriminative policies achieves close to the state-of-the art results in\nthe R2R dataset, demonstrating that the generative and discriminative policies\ncapture the different aspects of VLN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07936v1",
            "date": "2020-09-16 21:11:23+00:00",
            "title": "How to marry a star: probabilistic constraints for meaning in context",
            "authors": [
                "Katrin Erk",
                "Aurelie Herbelot"
            ],
            "abstract": "In this paper, we derive a notion of word meaning in context from Fillmore's\n'semantics of understanding', in which a listener draws on their knowledge of\nboth language and the world to 'envision' the situation described in an\nutterance. We characterize utterance understanding as a combination of\ncognitive semantics and Discourse Representation Theory, formalized as a\nsituation description system:a probabilistic model which takes utterance\nunderstanding to be the mental process of describing one or more situations\nthat would account for an observed utterance. Our model captures the interplay\nof local and global contexts and their joint influence upon the lexical\nrepresentation of sentence constituents. We implement the system using a\ndirected graphical model, and apply it to examples containing various\ncontextualization phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07968v2",
            "date": "2020-09-16 22:52:46+00:00",
            "title": "SKIM : Few-Shot Conversational Semantic Parsers with Formal Dialogue Contexts",
            "authors": [
                "Giovanni Campagna",
                "Sina J. Semnani",
                "Ryan Kearns",
                "Lucas Jun Koba Sato",
                "Silei Xu",
                "Monica S. Lam"
            ],
            "abstract": "The traditional dialogue state tracking (DST) task tracks the dialogue state\ngiven the past history of user and agent utterances. This paper proposes to\nreplace the utterances before the current turn with a formal representation,\nwhich is used as the context in a semantic parser mapping the current user\nutterance to its formal meaning. In addition, we propose TOC (Task-Oriented\nContext), a formal dialogue state representation. This approach eliminates the\nneed to parse a long history of natural language utterances; however, it adds\ncomplexity to the dialogue annotations. We propose Skim, a contextual semantic\nparser, trained with a sample-efficient training strategy: (1) a novel abstract\ndialogue state machine to synthesize training sets with TOC annotations; (2)\ndata augmentation with automatic paraphrasing, (3) few-shot training, and (4)\nself-training. This paper also presents MultiWOZ 2.4, which consists of the\nfull test set and a partial validation set of MultiWOZ 2.1, reannotated with\nthe TOC representation. Skim achieves 78% turn-by-turn exact match accuracy and\n85% slot accuracy, while our annotation effort amounts to only 2% of the\ntraining data used in MultiWOZ 2.1. The MultiWOZ 2.4 dataset will be released\nupon publication.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08034v2",
            "date": "2020-09-17 03:09:10+00:00",
            "title": "Towards Fully 8-bit Integer Inference for the Transformer Model",
            "authors": [
                "Ye Lin",
                "Yanyang Li",
                "Tengbo Liu",
                "Tong Xiao",
                "Tongran Liu",
                "Jingbo Zhu"
            ],
            "abstract": "8-bit integer inference, as a promising direction in reducing both the\nlatency and storage of deep neural networks, has made great progress recently.\nOn the other hand, previous systems still rely on 32-bit floating point for\ncertain functions in complex models (e.g., Softmax in Transformer), and make\nheavy use of quantization and de-quantization. In this work, we show that after\na principled modification on the Transformer architecture, dubbed Integer\nTransformer, an (almost) fully 8-bit integer inference algorithm Scale\nPropagation could be derived. De-quantization is adopted when necessary, which\nmakes the network more efficient. Our experiments on WMT16 En<->Ro, WMT14\nEn<->De and En->Fr translation tasks as well as the WikiText-103 language\nmodelling task show that the fully 8-bit Transformer system achieves comparable\nperformance with the floating point baseline but requires nearly 4x less memory\nfootprint.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08043v2",
            "date": "2020-09-17 03:37:37+00:00",
            "title": "Self-supervised pre-training and contrastive representation learning for multiple-choice video QA",
            "authors": [
                "Seonhoon Kim",
                "Seohyeong Jeong",
                "Eunbyul Kim",
                "Inho Kang",
                "Nojun Kwak"
            ],
            "abstract": "Video Question Answering (Video QA) requires fine-grained understanding of\nboth video and language modalities to answer the given questions. In this\npaper, we propose novel training schemes for multiple-choice video question\nanswering with a self-supervised pre-training stage and a supervised\ncontrastive learning in the main stage as an auxiliary learning. In the\nself-supervised pre-training stage, we transform the original problem format of\npredicting the correct answer into the one that predicts the relevant question\nto provide a model with broader contextual inputs without any further dataset\nor annotation. For contrastive learning in the main stage, we add a masking\nnoise to the input corresponding to the ground-truth answer, and consider the\noriginal input of the ground-truth answer as a positive sample, while treating\nthe rest as negative samples. By mapping the positive sample closer to the\nmasked input, we show that the model performance is improved. We further employ\nlocally aligned attention to focus more effectively on the video frames that\nare particularly relevant to the given corresponding subtitle sentences. We\nevaluate our proposed model on highly competitive benchmark datasets related to\nmultiple-choice video QA: TVQA, TVQA+, and DramaQA. Experimental results show\nthat our model achieves state-of-the-art performance on all datasets. We also\nvalidate our approaches through further analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08070v2",
            "date": "2020-09-17 04:58:39+00:00",
            "title": "On the Transferability of Minimal Prediction Preserving Inputs in Question Answering",
            "authors": [
                "Shayne Longpre",
                "Yi Lu",
                "Christopher DuBois"
            ],
            "abstract": "Recent work (Feng et al., 2018) establishes the presence of short,\nuninterpretable input fragments that yield high confidence and accuracy in\nneural models. We refer to these as Minimal Prediction Preserving Inputs\n(MPPIs). In the context of question answering, we investigate competing\nhypotheses for the existence of MPPIs, including poor posterior calibration of\nneural models, lack of pretraining, and \"dataset bias\" (where a model learns to\nattend to spurious, non-generalizable cues in the training data). We discover a\nperplexing invariance of MPPIs to random training seed, model architecture,\npretraining, and training domain. MPPIs demonstrate remarkable transferability\nacross domains achieving significantly higher performance than comparably short\nqueries. Additionally, penalizing over-confidence on MPPIs fails to improve\neither generalization or adversarial robustness. These results suggest the\ninterpretability of MPPIs is insufficient to characterize generalization\ncapacity of these models. We hope this focused investigation encourages more\nsystematic analysis of model behavior outside of the human interpretable\ndistribution of examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08088v1",
            "date": "2020-09-17 06:10:07+00:00",
            "title": "Code-switching pre-training for neural machine translation",
            "authors": [
                "Zhen Yang",
                "Bojie Hu",
                "Ambyera Han",
                "Shen Huang",
                "Qi Ju"
            ],
            "abstract": "This paper proposes a new pre-training method, called Code-Switching\nPre-training (CSP for short) for Neural Machine Translation (NMT). Unlike\ntraditional pre-training method which randomly masks some fragments of the\ninput sentence, the proposed CSP randomly replaces some words in the source\nsentence with their translation words in the target language. Specifically, we\nfirstly perform lexicon induction with unsupervised word embedding mapping\nbetween the source and target languages, and then randomly replace some words\nin the input sentence with their translation words according to the extracted\ntranslation lexicons. CSP adopts the encoder-decoder framework: its encoder\ntakes the code-mixed sentence as input, and its decoder predicts the replaced\nfragment of the input sentence. In this way, CSP is able to pre-train the NMT\nmodel by explicitly making the most of the cross-lingual alignment information\nextracted from the source and target monolingual corpus. Additionally, we\nrelieve the pretrain-finetune discrepancy caused by the artificial symbols like\n[mask]. To verify the effectiveness of the proposed method, we conduct\nextensive experiments on unsupervised and supervised NMT. Experimental results\nshow that CSP achieves significant improvements over baselines without\npre-training or with other pre-training methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08153v1",
            "date": "2020-09-17 09:00:59+00:00",
            "title": "End-to-End Neural Event Coreference Resolution",
            "authors": [
                "Yaojie Lu",
                "Hongyu Lin",
                "Jialong Tang",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "Traditional event coreference systems usually rely on pipeline framework and\nhand-crafted features, which often face error propagation problem and have poor\ngeneralization ability. In this paper, we propose an End-to-End Event\nCoreference approach -- E3C neural network, which can jointly model event\ndetection and event coreference resolution tasks, and learn to extract features\nfrom raw text automatically. Furthermore, because event mentions are highly\ndiversified and event coreference is intricately governed by long-distance,\nsemantic-dependent decisions, a type-guided event coreference mechanism is\nfurther proposed in our E3C neural network. Experiments show that our method\nachieves new state-of-the-art performance on two standard datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08171v1",
            "date": "2020-09-17 09:28:07+00:00",
            "title": "ISCAS at SemEval-2020 Task 5: Pre-trained Transformers for Counterfactual Statement Modeling",
            "authors": [
                "Yaojie Lu",
                "Annan Li",
                "Hongyu Lin",
                "Xianpei Han",
                "Le Sun"
            ],
            "abstract": "ISCAS participated in two subtasks of SemEval 2020 Task 5: detecting\ncounterfactual statements and detecting antecedent and consequence. This paper\ndescribes our system which is based on pre-trained transformers. For the first\nsubtask, we train several transformer-based classifiers for detecting\ncounterfactual statements. For the second subtask, we formulate antecedent and\nconsequence extraction as a query-based question answering problem. The two\nsubsystems both achieved third place in the evaluation. Our system is openly\nreleased at https://github.com/casnlu/ISCAS-SemEval2020Task5.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08180v1",
            "date": "2020-09-17 09:48:59+00:00",
            "title": "DSC IIT-ISM at SemEval-2020 Task 6: Boosting BERT with Dependencies for Definition Extraction",
            "authors": [
                "Aadarsh Singh",
                "Priyanshu Kumar",
                "Aman Sinha"
            ],
            "abstract": "We explore the performance of Bidirectional Encoder Representations from\nTransformers (BERT) at definition extraction. We further propose a joint model\nof BERT and Text Level Graph Convolutional Network so as to incorporate\ndependencies into the model. Our proposed model produces better results than\nBERT and achieves comparable results to BERT with fine tuned language model in\nDeftEval (Task 6 of SemEval 2020), a shared task of classifying whether a\nsentence contains a definition or not (Subtask 1).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08240v1",
            "date": "2020-09-17 12:45:46+00:00",
            "title": "What if we had no Wikipedia? Domain-independent Term Extraction from a Large News Corpus",
            "authors": [
                "Yonatan Bilu",
                "Shai Gretz",
                "Edo Cohen",
                "Noam Slonim"
            ],
            "abstract": "One of the most impressive human endeavors of the past two decades is the\ncollection and categorization of human knowledge in the free and accessible\nformat that is Wikipedia. In this work we ask what makes a term worthy of\nentering this edifice of knowledge, and having a page of its own in Wikipedia?\nTo what extent is this a natural product of on-going human discourse and\ndiscussion rather than an idiosyncratic choice of Wikipedia editors?\nSpecifically, we aim to identify such \"wiki-worthy\" terms in a massive news\ncorpus, and see if this can be done with no, or minimal, dependency on actual\nWikipedia entries. We suggest a five-step pipeline for doing so, providing\nbaseline results for all five, and the relevant datasets for benchmarking them.\nOur work sheds new light on the domain-specific Automatic Term Extraction\nproblem, with the problem at hand being a domain-independent variant of it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08380v1",
            "date": "2020-09-17 15:48:13+00:00",
            "title": "Evaluating Interactive Summarization: an Expansion-Based Framework",
            "authors": [
                "Ori Shapira",
                "Ramakanth Pasunuru",
                "Hadar Ronen",
                "Mohit Bansal",
                "Yael Amsterdamer",
                "Ido Dagan"
            ],
            "abstract": "Allowing users to interact with multi-document summarizers is a promising\ndirection towards improving and customizing summary results. Different ideas\nfor interactive summarization have been proposed in previous work but these\nsolutions are highly divergent and incomparable. In this paper, we develop an\nend-to-end evaluation framework for expansion-based interactive summarization,\nwhich considers the accumulating information along an interactive session. Our\nframework includes a procedure of collecting real user sessions and evaluation\nmeasures relying on standards, but adapted to reflect interaction. All of our\nsolutions are intended to be released publicly as a benchmark, allowing\ncomparison of future developments in interactive summarization. We demonstrate\nthe use of our framework by evaluating and comparing baseline implementations\nthat we developed for this purpose, which will serve as part of our benchmark.\nOur extensive experimentation and analysis of these systems motivate our design\nchoices and support the viability of our framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08478v2",
            "date": "2020-09-17 18:00:43+00:00",
            "title": "PhenoTagger: A Hybrid Method for Phenotype Concept Recognition using Human Phenotype Ontology",
            "authors": [
                "Ling Luo",
                "Shankai Yan",
                "Po-Ting Lai",
                "Daniel Veltri",
                "Andrew Oler",
                "Sandhya Xirasagar",
                "Rajarshi Ghosh",
                "Morgan Similuk",
                "Peter N. Robinson",
                "Zhiyong Lu"
            ],
            "abstract": "Automatic phenotype concept recognition from unstructured text remains a\nchallenging task in biomedical text mining research. Previous works that\naddress the task typically use dictionary-based matching methods, which can\nachieve high precision but suffer from lower recall. Recently, machine\nlearning-based methods have been proposed to identify biomedical concepts,\nwhich can recognize more unseen concept synonyms by automatic feature learning.\nHowever, most methods require large corpora of manually annotated data for\nmodel training, which is difficult to obtain due to the high cost of human\nannotation. In this paper, we propose PhenoTagger, a hybrid method that\ncombines both dictionary and machine learning-based methods to recognize Human\nPhenotype Ontology (HPO) concepts in unstructured biomedical text. We first use\nall concepts and synonyms in HPO to construct a dictionary, which is then used\nto automatically build a distantly supervised training dataset for machine\nlearning. Next, a cutting-edge deep learning model is trained to classify each\ncandidate phrase (n-gram from input sentence) into a corresponding concept\nlabel. Finally, the dictionary and machine learning-based prediction results\nare combined for improved performance. Our method is validated with two HPO\ncorpora, and the results show that PhenoTagger compares favorably to previous\nmethods. In addition, to demonstrate the generalizability of our method, we\nretrained PhenoTagger using the disease ontology MEDIC for disease concept\nrecognition to investigate the effect of training on different ontologies.\nExperimental results on the NCBI disease corpus show that PhenoTagger without\nrequiring manually annotated training data achieves competitive performance as\ncompared with state-of-the-art supervised methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08560v2",
            "date": "2020-09-17 23:37:33+00:00",
            "title": "Small but Mighty: New Benchmarks for Split and Rephrase",
            "authors": [
                "Li Zhang",
                "Huaiyu Zhu",
                "Siddhartha Brahma",
                "Yunyao Li"
            ],
            "abstract": "Split and Rephrase is a text simplification task of rewriting a complex\nsentence into simpler ones. As a relatively new task, it is paramount to ensure\nthe soundness of its evaluation benchmark and metric. We find that the widely\nused benchmark dataset universally contains easily exploitable syntactic cues\ncaused by its automatic generation process. Taking advantage of such cues, we\nshow that even a simple rule-based model can perform on par with the\nstate-of-the-art model. To remedy such limitations, we collect and release two\ncrowdsourced benchmark datasets. We not only make sure that they contain\nsignificantly more diverse syntax, but also carefully control for their quality\naccording to a well-defined set of criteria. While no satisfactory automatic\nmetric exists, we apply fine-grained manual evaluation based on these criteria\nusing crowdsourcing, showing that our datasets better represent the task and\nare significantly more challenging for the models.",
            "journal ref": "Proceedings of the 2020 Conference on Empirical Methods in Natural\n  Language Processing (EMNLP) (2020) 1198-1205",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08595v1",
            "date": "2020-09-18 02:38:01+00:00",
            "title": "Unsupervised Parallel Corpus Mining on Web Data",
            "authors": [
                "Guokun Lai",
                "Zihang Dai",
                "Yiming Yang"
            ],
            "abstract": "With a large amount of parallel data, neural machine translation systems are\nable to deliver human-level performance for sentence-level translation.\nHowever, it is costly to label a large amount of parallel data by humans. In\ncontrast, there is a large-scale of parallel corpus created by humans on the\nInternet. The major difficulty to utilize them is how to filter them out from\nthe noise website environments. Current parallel data mining methods all\nrequire labeled parallel data as the training source. In this paper, we present\na pipeline to mine the parallel corpus from the Internet in an unsupervised\nmanner. On the widely used WMT'14 English-French and WMT'16 English-German\nbenchmarks, the machine translator trained with the data extracted by our\npipeline achieves very close performance to the supervised results. On the\nWMT'16 English-Romanian and Romanian-English benchmarks, our system produces\nnew state-of-the-art results, 39.81 and 38.95 BLEU scores, even compared with\nsupervised approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08633v2",
            "date": "2020-09-18 05:41:52+00:00",
            "title": "fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP",
            "authors": [
                "Zhichao Geng",
                "Hang Yan",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "We present fastHan, an open-source toolkit for four basic tasks in Chinese\nnatural language processing: Chinese word segmentation (CWS), Part-of-Speech\n(POS) tagging, named entity recognition (NER), and dependency parsing. The\nbackbone of fastHan is a multi-task model based on a pruned BERT, which uses\nthe first 8 layers in BERT. We also provide a 4-layer base model compressed\nfrom the 8-layer model. The joint-model is trained and evaluated on 13 corpora\nof four tasks, yielding near state-of-the-art (SOTA) performance in dependency\nparsing and NER, achieving SOTA performance in CWS and POS. Besides, fastHan's\ntransferability is also strong, performing much better than popular\nsegmentation tools on a non-training corpus. To better meet the need of\npractical application, we allow users to use their own labeled data to further\nfine-tune fastHan. In addition to its small size and excellent performance,\nfastHan is user-friendly. Implemented as a python package, fastHan isolates\nusers from the internal technical details and is convenient to use. The project\nis released on Github.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08636v1",
            "date": "2020-09-18 05:55:37+00:00",
            "title": "Hierarchical GPT with Congruent Transformers for Multi-Sentence Language Models",
            "authors": [
                "Jihyeon Roh",
                "Huiseong Gim",
                "Soo-Young Lee"
            ],
            "abstract": "We report a GPT-based multi-sentence language model for dialogue generation\nand document understanding. First, we propose a hierarchical GPT which consists\nof three blocks, i.e., a sentence encoding block, a sentence generating block,\nand a sentence decoding block. The sentence encoding and decoding blocks are\nbasically the encoder-decoder blocks of the standard Transformers, which work\non each sentence independently. The sentence generating block is inserted\nbetween the encoding and decoding blocks, and generates the next sentence\nembedding vector from the previous sentence embedding vectors. We believe it is\nthe way human make conversation and understand paragraphs and documents. Since\neach sentence may consist of fewer words, the sentence encoding and decoding\nTransformers can use much smaller dimensional embedding vectors. Secondly, we\nnote the attention in the Transformers utilizes the inner-product similarity\nmeasure. Therefore, to compare the two vectors in the same space, we set the\ntransform matrices for queries and keys to be the same. Otherwise, the\nsimilarity concept is incongruent. We report experimental results to show that\nthese two modifications increase the language model performance for tasks with\nmultiple sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08712v1",
            "date": "2020-09-18 09:30:48+00:00",
            "title": "The birth of Romanian BERT",
            "authors": [
                "Stefan Daniel Dumitrescu",
                "Andrei-Marius Avram",
                "Sampo Pyysalo"
            ],
            "abstract": "Large-scale pretrained language models have become ubiquitous in Natural\nLanguage Processing. However, most of these models are available either in\nhigh-resource languages, in particular English, or as multilingual models that\ncompromise performance on individual languages for coverage. This paper\nintroduces Romanian BERT, the first purely Romanian transformer-based language\nmodel, pretrained on a large text corpus. We discuss corpus composition and\ncleaning, the model training process, as well as an extensive evaluation of the\nmodel on various Romanian datasets. We open source not only the model itself,\nbut also a repository that contains information on how to obtain the corpus,\nfine-tune and use this model in production (with practical examples), and how\nto fully replicate the evaluation process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08775v1",
            "date": "2020-09-16 19:43:29+00:00",
            "title": "Document-level Neural Machine Translation with Document Embeddings",
            "authors": [
                "Shu Jiang",
                "Hai Zhao",
                "Zuchao Li",
                "Bao-Liang Lu"
            ],
            "abstract": "Standard neural machine translation (NMT) is on the assumption of\ndocument-level context independent. Most existing document-level NMT methods\nare satisfied with a smattering sense of brief document-level information,\nwhile this work focuses on exploiting detailed document-level context in terms\nof multiple forms of document embeddings, which is capable of sufficiently\nmodeling deeper and richer document-level context. The proposed document-aware\nNMT is implemented to enhance the Transformer baseline by introducing both\nglobal and local document-level clues on the source end. Experiments show that\nthe proposed method significantly improves the translation performance over\nstrong baselines and other related studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08820v1",
            "date": "2020-09-18 13:04:04+00:00",
            "title": "FarsTail: A Persian Natural Language Inference Dataset",
            "authors": [
                "Hossein Amirkhani",
                "Mohammad Azari Jafari",
                "Azadeh Amirak",
                "Zohreh Pourjafari",
                "Soroush Faridan Jahromi",
                "Zeinab Kouhkan"
            ],
            "abstract": "Natural language inference (NLI) is known as one of the central tasks in\nnatural language processing (NLP) which encapsulates many fundamental aspects\nof language understanding. With the considerable achievements of data-hungry\ndeep learning methods in NLP tasks, a great amount of effort has been devoted\nto develop more diverse datasets for different languages. In this paper, we\npresent a new dataset for the NLI task in the Persian language, also known as\nFarsi, which is one of the dominant languages in the Middle East. This dataset,\nnamed FarsTail, includes 10,367 samples which are provided in both the Persian\nlanguage as well as the indexed format to be useful for non-Persian\nresearchers. The samples are generated from 3,539 multiple-choice questions\nwith the least amount of annotator interventions in a way similar to the\nSciTail dataset. A carefully designed multi-step process is adopted to ensure\nthe quality of the dataset. We also present the results of traditional and\nstate-of-the-art methods on FarsTail including different embedding methods such\nas word2vec, fastText, ELMo, BERT, and LASER, as well as different modeling\napproaches such as DecompAtt, ESIM, HBMP, ULMFiT, and cross-lingual transfer\napproach to provide a solid baseline for the future research. The best obtained\ntest accuracy is 78.13% which shows that there is a big room for improving the\ncurrent methods to be useful for real-world NLP applications in different\nlanguages. The dataset is available at https://github.com/dml-qom/FarsTail.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09016v1",
            "date": "2020-09-18 18:37:03+00:00",
            "title": "Presenting Simultaneous Translation in Limited Space",
            "authors": [
                "Dominik Mach\u00e1\u010dek",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Some methods of automatic simultaneous translation of a long-form speech\nallow revisions of outputs, trading accuracy for low latency. Deploying these\nsystems for users faces the problem of presenting subtitles in a limited space,\nsuch as two lines on a television screen. The subtitles must be shown promptly,\nincrementally, and with adequate time for reading. We provide an algorithm for\nsubtitling. Furthermore, we propose a way how to estimate the overall usability\nof the combination of automatic translation and subtitling by measuring the\nquality, latency, and stability on a test set, and propose an improved measure\nfor translation latency.",
            "journal ref": "ITAT WAFNL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09025v2",
            "date": "2020-09-18 18:54:15+00:00",
            "title": "COMET: A Neural Framework for MT Evaluation",
            "authors": [
                "Ricardo Rei",
                "Craig Stewart",
                "Ana C Farinha",
                "Alon Lavie"
            ],
            "abstract": "We present COMET, a neural framework for training multilingual machine\ntranslation evaluation models which obtains new state-of-the-art levels of\ncorrelation with human judgements. Our framework leverages recent breakthroughs\nin cross-lingual pretrained language modeling resulting in highly multilingual\nand adaptable MT evaluation models that exploit information from both the\nsource input and a target-language reference translation in order to more\naccurately predict MT quality. To showcase our framework, we train three models\nwith different types of human judgements: Direct Assessments, Human-mediated\nTranslation Edit Rate and Multidimensional Quality Metrics. Our models achieve\nnew state-of-the-art performance on the WMT 2019 Metrics shared task and\ndemonstrate robustness to high-performing systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09120v1",
            "date": "2020-09-18 23:39:15+00:00",
            "title": "Tradeoffs in Sentence Selection Techniques for Open-Domain Question Answering",
            "authors": [
                "Shih-Ting Lin",
                "Greg Durrett"
            ],
            "abstract": "Current methods in open-domain question answering (QA) usually employ a\npipeline of first retrieving relevant documents, then applying strong reading\ncomprehension (RC) models to that retrieved text. However, modern RC models are\ncomplex and expensive to run, so techniques to prune the space of retrieved\ntext are critical to allow this approach to scale. In this paper, we focus on\napproaches which apply an intermediate sentence selection step to address this\nissue, and investigate the best practices for this approach. We describe two\ngroups of models for sentence selection: QA-based approaches, which run a\nfull-fledged QA system to identify answer candidates, and retrieval-based\nmodels, which find parts of each passage specifically related to each question.\nWe examine trade-offs between processing speed and task performance in these\ntwo approaches, and demonstrate an ensemble module that represents a hybrid of\nthe two. From experiments on Open-SQuAD and TriviaQA, we show that very\nlightweight QA models can do well at this task, but retrieval-based models are\nfaster still. An ensemble module we describe balances between the two and\ngeneralizes well cross-domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09132v1",
            "date": "2020-09-19 01:16:18+00:00",
            "title": "Prior Art Search and Reranking for Generated Patent Text",
            "authors": [
                "Jieh-Sheng Lee",
                "Jieh Hsiang"
            ],
            "abstract": "Generative models, such as GPT-2, have demonstrated impressive results\nrecently. A fundamental question we'd like to address is: where did the\ngenerated text come from? This work is our initial effort toward answering the\nquestion by using prior art search. The purpose of the prior art search is to\nfind the most similar prior text in the training data of GPT-2. We take a\nreranking approach and apply it to the patent domain. Specifically, we\npre-train GPT-2 models from scratch by using the patent data from the USPTO.\nThe input for the prior art search is the patent text generated by the GPT-2\nmodel. We also pre-trained BERT models from scratch for converting patent text\nto embeddings. The steps of reranking are: (1) search the most similar text in\nthe training data of GPT-2 by taking a bag-of-word ranking approach (BM25), (2)\nconvert the search results in text format to BERT embeddings, and (3) provide\nthe final result by ranking the BERT embeddings based on their similarities\nwith the patent text generated by GPT-2. The experiments in this work show that\nsuch reranking is better than ranking with embeddings alone. However, our mixed\nresults also indicate that calculating the semantic similarities among long\ntext spans is still challenging. To our knowledge, this work is the first to\nimplement a reranking system to identify retrospectively the most similar\ninputs to a GPT model based on its output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09147v1",
            "date": "2020-09-19 02:41:04+00:00",
            "title": "Enhancing Dialogue Generation via Multi-Level Contrastive Learning",
            "authors": [
                "Xin Li",
                "Piji Li",
                "Yan Wang",
                "Xiaojiang Liu",
                "Wai Lam"
            ],
            "abstract": "Most of the existing works for dialogue generation are data-driven models\ntrained directly on corpora crawled from websites. They mainly focus on\nimproving the model architecture to produce better responses but pay little\nattention to considering the quality of the training data contrastively. In\nthis paper, we propose a multi-level contrastive learning paradigm to model the\nfine-grained quality of the responses with respect to the query. A Rank-aware\nCalibration (RC) network is designed to construct the multi-level contrastive\noptimization objectives. Since these objectives are calculated based on the\nsentence level, which may erroneously encourage/suppress the generation of\nuninformative/informative words. To tackle this incidental issue, on one hand,\nwe design an exquisite token-level strategy for estimating the instance loss\nmore accurately. On the other hand, we build a Knowledge Inference (KI)\ncomponent to capture the keyword knowledge from the reference during training\nand exploit such information to encourage the generation of informative words.\nWe evaluate the proposed model on a carefully annotated dialogue dataset and\nthe results suggest that our model can generate more relevant and diverse\nresponses compared to the baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09152v2",
            "date": "2020-09-19 03:23:26+00:00",
            "title": "Weight Distillation: Transferring the Knowledge in Neural Network Parameters",
            "authors": [
                "Ye Lin",
                "Yanyang Li",
                "Ziyang Wang",
                "Bei Li",
                "Quan Du",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "Knowledge distillation has been proven to be effective in model acceleration\nand compression. It allows a small network to learn to generalize in the same\nway as a large network. Recent successes in pre-training suggest the\neffectiveness of transferring model parameters. Inspired by this, we\ninvestigate methods of model acceleration and compression in another line of\nresearch. We propose Weight Distillation to transfer the knowledge in the large\nnetwork parameters through a parameter generator. Our experiments on WMT16\nEn-Ro, NIST12 Zh-En, and WMT14 En-De machine translation tasks show that weight\ndistillation can train a small network that is 1.88~2.94x faster than the large\nnetwork but with competitive performance. With the same sized small network,\nweight distillation can outperform knowledge distillation by 0.51~1.82 BLEU\npoints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09154v2",
            "date": "2020-09-19 03:32:37+00:00",
            "title": "CLEVR Parser: A Graph Parser Library for Geometric Learning on Language Grounded Image Scenes",
            "authors": [
                "Raeid Saqur",
                "Ameet Deshpande"
            ],
            "abstract": "The CLEVR dataset has been used extensively in language grounded visual\nreasoning in Machine Learning (ML) and Natural Language Processing (NLP)\ndomains. We present a graph parser library for CLEVR, that provides\nfunctionalities for object-centric attributes and relationships extraction, and\nconstruction of structural graph representations for dual modalities.\nStructural order-invariant representations enable geometric learning and can\naid in downstream tasks like language grounding to vision, robotics,\ncompositionality, interpretability, and computational grammar construction. We\nprovide three extensible main components - parser, embedder, and visualizer\nthat can be tailored to suit specific learning setups. We also provide\nout-of-the-box functionality for seamless integration with popular deep graph\nneural network (GNN) libraries. Additionally, we discuss downstream usage and\napplications of the library, and how it accelerates research for the NLP\nresearch community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09162v1",
            "date": "2020-09-19 04:37:33+00:00",
            "title": "Extracting Summary Knowledge Graphs from Long Documents",
            "authors": [
                "Zeqiu Wu",
                "Rik Koncel-Kedziorski",
                "Mari Ostendorf",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Knowledge graphs capture entities and relations from long documents and can\nfacilitate reasoning in many downstream applications. Extracting compact\nknowledge graphs containing only salient entities and relations is important\nbut challenging for understanding and summarizing long documents. We introduce\na new text-to-graph task of predicting summarized knowledge graphs from long\ndocuments. We develop a dataset of 200k document/graph pairs using automatic\nand human annotations. We also develop strong baselines for this task based on\ngraph learning and text summarization, and provide quantitative and qualitative\nstudies of their effect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09173v1",
            "date": "2020-09-19 06:20:37+00:00",
            "title": "Nominal Compound Chain Extraction: A New Task for Semantic-enriched Lexical Chain",
            "authors": [
                "Bobo Li",
                "Hao Fei",
                "Yafeng Ren",
                "Donghong Ji"
            ],
            "abstract": "Lexical chain consists of cohesion words in a document, which implies the\nunderlying structure of a text, and thus facilitates downstream NLP tasks.\nNevertheless, existing work focuses on detecting the simple surface lexicons\nwith shallow syntax associations, ignoring the semantic-aware lexical compounds\nas well as the latent semantic frames, (e.g., topic), which can be much more\ncrucial for real-world NLP applications. In this paper, we introduce a novel\ntask, Nominal Compound Chain Extraction (NCCE), extracting and clustering all\nthe nominal compounds that share identical semantic topics. In addition, we\nmodel the task as a two-stage prediction (i.e., compound extraction and chain\ndetection), which is handled via a proposed joint framework. The model employs\nthe BERT encoder to yield contextualized document representation. Also, HowNet\nis exploited as external resources for offering rich sememe information. The\nexperiments are based on our manually annotated corpus, and the results prove\nthe necessity of the NCCE task as well as the effectiveness of our joint\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09174v1",
            "date": "2020-09-19 06:26:07+00:00",
            "title": "Aggressive Language Detection with Joint Text Normalization via Adversarial Multi-task Learning",
            "authors": [
                "Shengqiong Wu",
                "Hao Fei",
                "Donghong Ji"
            ],
            "abstract": "Aggressive language detection (ALD), detecting the abusive and offensive\nlanguage in texts, is one of the crucial applications in NLP community. Most\nexisting works treat ALD as regular classification with neural models, while\nignoring the inherent conflicts of social media text that they are quite\nunnormalized and irregular. In this work, we target improving the ALD by\njointly performing text normalization (TN), via an adversarial multi-task\nlearning framework. The private encoders for ALD and TN focus on the\ntask-specific features retrieving, respectively, and the shared encoder learns\nthe underlying common features over two tasks. During adversarial training, a\ntask discriminator distinguishes the separate learning of ALD or TN.\nExperimental results on four ALD datasets show that our model outperforms all\nbaselines under differing settings by large margins, demonstrating the\nnecessity of joint learning the TN with ALD. Further analysis is conducted for\na better understanding of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09223v1",
            "date": "2020-09-19 12:58:47+00:00",
            "title": "BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition",
            "authors": [
                "Usman Naseem",
                "Matloob Khushi",
                "Vinay Reddy",
                "Sakthivel Rajendran",
                "Imran Razzak",
                "Jinman Kim"
            ],
            "abstract": "In recent years, with the growing amount of biomedical documents, coupled\nwith advancement in natural language processing algorithms, the research on\nbiomedical named entity recognition (BioNER) has increased exponentially.\nHowever, BioNER research is challenging as NER in the biomedical domain are:\n(i) often restricted due to limited amount of training data, (ii) an entity can\nrefer to multiple types and concepts depending on its context and, (iii) heavy\nreliance on acronyms that are sub-domain specific. Existing BioNER approaches\noften neglect these issues and directly adopt the state-of-the-art (SOTA)\nmodels trained in general corpora which often yields unsatisfactory results. We\npropose biomedical ALBERT (A Lite Bidirectional Encoder Representations from\nTransformers for Biomedical Text Mining) bioALBERT, an effective\ndomain-specific language model trained on large-scale biomedical corpora\ndesigned to capture biomedical context-dependent NER. We adopted a\nself-supervised loss used in ALBERT that focuses on modelling inter-sentence\ncoherence to better learn context-dependent representations and incorporated\nparameter reduction techniques to lower memory consumption and increase the\ntraining speed in BioNER. In our experiments, BioALBERT outperformed\ncomparative SOTA BioNER models on eight biomedical NER benchmark datasets with\nfour different entity types. We trained four different variants of BioALBERT\nmodels which are available for the research community to be used in future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09241v1",
            "date": "2020-09-19 14:41:50+00:00",
            "title": "Word class flexibility: A deep contextualized approach",
            "authors": [
                "Bai Li",
                "Guillaume Thomas",
                "Yang Xu",
                "Frank Rudzicz"
            ],
            "abstract": "Word class flexibility refers to the phenomenon whereby a single word form is\nused across different grammatical categories. Extensive work in linguistic\ntypology has sought to characterize word class flexibility across languages,\nbut quantifying this phenomenon accurately and at scale has been fraught with\ndifficulties. We propose a principled methodology to explore regularity in word\nclass flexibility. Our method builds on recent work in contextualized word\nembeddings to quantify semantic shift between word classes (e.g., noun-to-verb,\nverb-to-noun), and we apply this method to 37 languages. We find that\ncontextualized embeddings not only capture human judgment of class variation\nwithin words in English, but also uncover shared tendencies in class\nflexibility across languages. Specifically, we find greater semantic variation\nwhen flexible lemmas are used in their dominant word class, supporting the view\nthat word class flexibility is a directional process. Our work highlights the\nutility of deep contextualized models in linguistic typology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09309v1",
            "date": "2020-09-19 22:13:27+00:00",
            "title": "Towards Computational Linguistics in Minangkabau Language: Studies on Sentiment Analysis and Machine Translation",
            "authors": [
                "Fajri Koto",
                "Ikhwan Koto"
            ],
            "abstract": "Although some linguists (Rusmali et al., 1985; Crouch, 2009) have fairly\nattempted to define the morphology and syntax of Minangkabau, information\nprocessing in this language is still absent due to the scarcity of the\nannotated resource. In this work, we release two Minangkabau corpora: sentiment\nanalysis and machine translation that are harvested and constructed from\nTwitter and Wikipedia. We conduct the first computational linguistics in\nMinangkabau language employing classic machine learning and\nsequence-to-sequence models such as LSTM and Transformer. Our first experiments\nshow that the classification performance over Minangkabau text significantly\ndrops when tested with the model trained in Indonesian. Whereas, in the machine\ntranslation experiment, a simple word-to-word translation using a bilingual\ndictionary outperforms LSTM and Transformer model in terms of BLEU score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09359v2",
            "date": "2020-09-20 06:06:27+00:00",
            "title": "Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation",
            "authors": [
                "Tahmid Hasan",
                "Abhik Bhattacharjee",
                "Kazi Samin",
                "Masum Hasan",
                "Madhusudan Basak",
                "M. Sohel Rahman",
                "Rifat Shahriyar"
            ],
            "abstract": "Despite being the seventh most widely spoken language in the world, Bengali\nhas received much less attention in machine translation literature due to being\nlow in resources. Most publicly available parallel corpora for Bengali are not\nlarge enough; and have rather poor quality, mostly because of incorrect\nsentence alignments resulting from erroneous sentence segmentation, and also\nbecause of a high volume of noise present in them. In this work, we build a\ncustomized sentence segmenter for Bengali and propose two novel methods for\nparallel corpus creation on low-resource setups: aligner ensembling and batch\nfiltering. With the segmenter and the two methods combined, we compile a\nhigh-quality Bengali-English parallel corpus comprising of 2.75 million\nsentence pairs, more than 2 million of which were not available before.\nTraining on neural models, we achieve an improvement of more than 9 BLEU score\nover previous approaches to Bengali-English machine translation. We also\nevaluate on a new test set of 1000 pairs made with extensive quality control.\nWe release the segmenter, parallel corpus, and the evaluation set, thus\nelevating Bengali from its low-resource status. To the best of our knowledge,\nthis is the first ever large scale study on Bengali-English machine\ntranslation. We believe our study will pave the way for future research on\nBengali-English machine translation as well as other low-resource languages.\nOur data and code are available at https://github.com/csebuetnlp/banglanmt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09363v1",
            "date": "2020-09-20 06:30:45+00:00",
            "title": "Understanding Mention Detector-Linker Interaction for Neural Coreference Resolution",
            "authors": [
                "Zhaofeng Wu",
                "Matt Gardner"
            ],
            "abstract": "Coreference resolution is an important task for discourse-level natural\nlanguage understanding. However, despite significant recent progress, the\nquality of current state-of-the-art systems still considerably trails behind\nhuman-level performance. Using the CoNLL-2012 and PreCo datasets, we dissect\nthe best instantiation of the mainstream end-to-end coreference resolution\nmodel that underlies most current best-performing coreference systems, and\nempirically analyze the behavior of its two components: the mention detector\nand mention linker. While the detector traditionally focuses heavily on recall\nas a design decision, we demonstrate the importance of precision, calling for\ntheir balance. However, we point out the difficulty in building a precise\ndetector due to its inability to make important anaphoricity decisions. We also\nhighlight the enormous room for improving the linker and that the rest of its\nerrors mainly involve pronoun resolution. We hope our findings will help future\nresearch in building coreference resolution systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09427v2",
            "date": "2020-09-20 13:06:38+00:00",
            "title": "Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data",
            "authors": [
                "Rongsheng Zhang",
                "Yinhe Zheng",
                "Jianzhi Shao",
                "Xiaoxi Mao",
                "Yadong Xi",
                "Minlie Huang"
            ],
            "abstract": "Recent advances in open-domain dialogue systems rely on the success of neural\nmodels that are trained on large-scale data. However, collecting large-scale\ndialogue data is usually time-consuming and labor-intensive. To address this\ndata dilemma, we propose a novel data augmentation method for training\nopen-domain dialogue models by utilizing unpaired data. Specifically, a\ndata-level distillation process is first proposed to construct augmented\ndialogues where both post and response are retrieved from the unpaired data. A\nranking module is employed to filter out low-quality dialogues. Further, a\nmodel-level distillation process is employed to distill a teacher model trained\non high-quality paired data to augmented dialogue pairs, thereby preventing\ndialogue models from being affected by the noise in the augmented data.\nAutomatic and manual evaluation indicates that our method can produce\nhigh-quality dialogue pairs with diverse contents, and the proposed data-level\nand model-level dialogue distillation can improve the performance of\ncompetitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09509v1",
            "date": "2020-09-20 19:50:28+00:00",
            "title": "Relation Extraction from Biomedical and Clinical Text: Unified Multitask Learning Framework",
            "authors": [
                "Shweta Yadav",
                "Srivatsa Ramesh",
                "Sriparna Saha",
                "Asif Ekbal"
            ],
            "abstract": "To minimize the accelerating amount of time invested in the biomedical\nliterature search, numerous approaches for automated knowledge extraction have\nbeen proposed. Relation extraction is one such task where semantic relations\nbetween the entities are identified from the free text. In the biomedical\ndomain, extraction of regulatory pathways, metabolic processes, adverse drug\nreaction or disease models necessitates knowledge from the individual\nrelations, for example, physical or regulatory interactions between genes,\nproteins, drugs, chemical, disease or phenotype. In this paper, we study the\nrelation extraction task from three major biomedical and clinical tasks, namely\ndrug-drug interaction, protein-protein interaction, and medical concept\nrelation extraction. Towards this, we model the relation extraction problem in\nmulti-task learning (MTL) framework and introduce for the first time the\nconcept of structured self-attentive network complemented with the adversarial\nlearning approach for the prediction of relationships from the biomedical and\nclinical text. The fundamental notion of MTL is to simultaneously learn\nmultiple problems together by utilizing the concepts of the shared\nrepresentation. Additionally, we also generate the highly efficient single task\nmodel which exploits the shortest dependency path embedding learned over the\nattentive gated recurrent unit to compare our proposed MTL models. The\nframework we propose significantly improves overall the baselines (deep\nlearning techniques) and single-task models for predicting the relationships,\nwithout compromising on the performance of all the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09568v2",
            "date": "2020-09-21 01:52:32+00:00",
            "title": "Vector Projection Network for Few-shot Slot Tagging in Natural Language Understanding",
            "authors": [
                "Su Zhu",
                "Ruisheng Cao",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Few-shot slot tagging becomes appealing for rapid domain transfer and\nadaptation, motivated by the tremendous development of conversational dialogue\nsystems. In this paper, we propose a vector projection network for few-shot\nslot tagging, which exploits projections of contextual word embeddings on each\ntarget label vector as word-label similarities. Essentially, this approach is\nequivalent to a normalized linear model with an adaptive bias. The contrastive\nexperiment demonstrates that our proposed vector projection based similarity\nmetric can significantly surpass other variants. Specifically, in the five-shot\nsetting on benchmarks SNIPS and NER, our method outperforms the strongest\nfew-shot learning baseline by $6.30$ and $13.79$ points on F$_1$ score,\nrespectively. Our code will be released at\nhttps://github.com/sz128/few_shot_slot_tagging_and_NER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09587v1",
            "date": "2020-09-21 02:48:46+00:00",
            "title": "Improving Robustness and Generality of NLP Models Using Disentangled Representations",
            "authors": [
                "Jiawei Wu",
                "Xiaoya Li",
                "Xiang Ao",
                "Yuxian Meng",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Supervised neural networks, which first map an input $x$ to a single\nrepresentation $z$, and then map $z$ to the output label $y$, have achieved\nremarkable success in a wide range of natural language processing (NLP) tasks.\nDespite their success, neural models lack for both robustness and generality:\nsmall perturbations to inputs can result in absolutely different outputs; the\nperformance of a model trained on one domain drops drastically when tested on\nanother domain.\n  In this paper, we present methods to improve robustness and generality of NLP\nmodels from the standpoint of disentangled representation learning. Instead of\nmapping $x$ to a single representation $z$, the proposed strategy maps $x$ to a\nset of representations $\\{z_1,z_2,...,z_K\\}$ while forcing them to be\ndisentangled. These representations are then mapped to different logits $l$s,\nthe ensemble of which is used to make the final prediction $y$. We propose\ndifferent methods to incorporate this idea into currently widely-used models,\nincluding adding an $L$2 regularizer on $z$s or adding Total Correlation (TC)\nunder the framework of variational information bottleneck (VIB). We show that\nmodels trained with the proposed criteria provide better robustness and domain\nadaptation ability in a wide range of supervised learning tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09600v1",
            "date": "2020-09-21 03:45:14+00:00",
            "title": "Assessing the Severity of Health States based on Social Media Posts",
            "authors": [
                "Shweta Yadav",
                "Joy Prakash Sain",
                "Amit Sheth",
                "Asif Ekbal",
                "Sriparna Saha",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "The unprecedented growth of Internet users has resulted in an abundance of\nunstructured information on social media including health forums, where\npatients request health-related information or opinions from other users.\nPrevious studies have shown that online peer support has limited effectiveness\nwithout expert intervention. Therefore, a system capable of assessing the\nseverity of health state from the patients' social media posts can help health\nprofessionals (HP) in prioritizing the user's post. In this study, we inspect\nthe efficacy of different aspects of Natural Language Understanding (NLU) to\nidentify the severity of the user's health state in relation to two\nperspectives(tasks) (a) Medical Condition (i.e., Recover, Exist, Deteriorate,\nOther) and (b) Medication (i.e., Effective, Ineffective, Serious Adverse\nEffect, Other) in online health communities. We propose a multiview learning\nframework that models both the textual content as well as\ncontextual-information to assess the severity of the user's health state.\nSpecifically, our model utilizes the NLU views such as sentiment, emotions,\npersonality, and use of figurative language to extract the contextual\ninformation. The diverse NLU views demonstrate its effectiveness on both the\ntasks and as well as on the individual disease to assess a user's health.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09629v3",
            "date": "2020-09-21 06:10:39+00:00",
            "title": "Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition",
            "authors": [
                "Wenliang Dai",
                "Zihan Liu",
                "Tiezheng Yu",
                "Pascale Fung"
            ],
            "abstract": "Despite the recent achievements made in the multi-modal emotion recognition\ntask, two problems still exist and have not been well investigated: 1) the\nrelationship between different emotion categories are not utilized, which leads\nto sub-optimal performance; and 2) current models fail to cope well with\nlow-resource emotions, especially for unseen emotions. In this paper, we\npropose a modality-transferable model with emotion embeddings to tackle the\naforementioned issues. We use pre-trained word embeddings to represent emotion\ncategories for textual data. Then, two mapping functions are learned to\ntransfer these embeddings into visual and acoustic spaces. For each modality,\nthe model calculates the representation distance between the input sequence and\ntarget emotions and makes predictions based on the distances. By doing so, our\nmodel can directly adapt to the unseen emotions in any modality since we have\ntheir pre-trained embeddings and modality mapping functions. Experiments show\nthat our model achieves state-of-the-art performance on most of the emotion\ncategories. In addition, our model also outperforms existing baselines in the\nzero-shot and few-shot scenarios for unseen emotions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09672v1",
            "date": "2020-09-21 08:14:30+00:00",
            "title": "Alleviating the Inequality of Attention Heads for Neural Machine Translation",
            "authors": [
                "Zewei Sun",
                "Shujian Huang",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "Recent studies show that the attention heads in Transformer are not equal. We\nrelate this phenomenon to the imbalance training of multi-head attention and\nthe model dependence on specific heads. To tackle this problem, we propose a\nsimple masking method: HeadMask, in two specific ways. Experiments show that\ntranslation improvements are achieved on multiple language pairs. Subsequent\nempirical analyses also support our assumption and confirm the effectiveness of\nthe method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09680v5",
            "date": "2020-09-21 08:38:23+00:00",
            "title": "Profile Consistency Identification for Open-domain Dialogue Agents",
            "authors": [
                "Haoyu Song",
                "Yan Wang",
                "Wei-Nan Zhang",
                "Zhengyu Zhao",
                "Ting Liu",
                "Xiaojiang Liu"
            ],
            "abstract": "Maintaining a consistent attribute profile is crucial for dialogue agents to\nnaturally converse with humans. Existing studies on improving attribute\nconsistency mainly explored how to incorporate attribute information in the\nresponses, but few efforts have been made to identify the consistency relations\nbetween response and attribute profile. To facilitate the study of profile\nconsistency identification, we create a large-scale human-annotated dataset\nwith over 110K single-turn conversations and their key-value attribute\nprofiles. Explicit relation between response and profile is manually labeled.\nWe also propose a key-value structure information enriched BERT model to\nidentify the profile consistency, and it gained improvements over strong\nbaselines. Further evaluations on downstream tasks demonstrate that the profile\nconsistency identification model is conducive for improving dialogue\nconsistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09708v2",
            "date": "2020-09-21 09:21:52+00:00",
            "title": "Towards Empathetic Dialogue Generation over Multi-type Knowledge",
            "authors": [
                "Qintong Li",
                "Piji Li",
                "Zhumin Chen",
                "Zhaochun Ren"
            ],
            "abstract": "Enabling the machines with empathetic abilities to provide context-consistent\nresponses is crucial on both semantic and emotional levels. The task of\nempathetic dialogue generation is proposed to address this problem. However,\nlacking external knowledge makes it difficult to perceive implicit emotions\nfrom limited dialogue history. To address the above challenges, we propose to\nleverage multi-type knowledge, i.e, the commonsense knowledge and emotional\nlexicon, to explicitly understand and express emotions in empathetic dialogue\ngeneration. We first enrich the dialogue history by jointly interacting with\ntwo-type knowledge and construct an emotional context graph. Then we introduce\na multi-type knowledge-aware context encoder to learn emotional context\nrepresentations and distill emotional signals, which are the prerequisites to\npredicate emotions expressed in responses. Finally, we propose an emotional\ncross-attention mechanism to exploit the emotional dependencies between the\nemotional context graph and the target empathetic response. Conducted on a\nbenchmark dataset, extensive experimental results show that our proposed\nframework outperforms state-of-the-art baselines in terms of automatic metrics\nand human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09730v1",
            "date": "2020-09-21 10:04:07+00:00",
            "title": "Multitask Pointer Network for Multi-Representational Parsing",
            "authors": [
                "Daniel Fern\u00e1ndez-Gonz\u00e1lez",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We propose a transition-based approach that, by training a single model, can\nefficiently parse any input sentence with both constituent and dependency\ntrees, supporting both continuous/projective and discontinuous/non-projective\nsyntactic structures. To that end, we develop a Pointer Network architecture\nwith two separate task-specific decoders and a common encoder, and follow a\nmultitask learning strategy to jointly train them. The resulting quadratic\nsystem, not only becomes the first parser that can jointly produce both\nunrestricted constituent and dependency trees from a single model, but also\nproves that both syntactic formalisms can benefit from each other during\ntraining, achieving state-of-the-art accuracies in several widely-used\nbenchmarks such as the continuous English and Chinese Penn Treebanks, as well\nas the discontinuous German NEGRA and TIGER datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09961v1",
            "date": "2020-09-21 15:38:45+00:00",
            "title": "Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference",
            "authors": [
                "Galen Weld",
                "Peter West",
                "Maria Glenski",
                "David Arbour",
                "Ryan Rossi",
                "Tim Althoff"
            ],
            "abstract": "Leveraging text, such as social media posts, for causal inferences requires\nthe use of NLP models to 'learn' and adjust for confounders, which could\notherwise impart bias. However, evaluating such models is challenging, as\nground truth is almost never available. We demonstrate the need for empirical\nevaluation frameworks for causal inference in natural language by showing that\nexisting, commonly used models regularly disagree with one another on real\nworld tasks. We contribute the first such framework, generalizing several\nchallenges across these real world tasks. Using this framework, we evaluate a\nlarge set of commonly used causal inference models based on propensity scores\nand identify their strengths and weaknesses to inform future improvements. We\nmake all tasks, data, and models public to inform applications and encourage\nadditional research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10047v2",
            "date": "2020-09-21 17:39:00+00:00",
            "title": "UCD-CS at W-NUT 2020 Shared Task-3: A Text to Text Approach for COVID-19 Event Extraction on Social Media",
            "authors": [
                "Congcong Wang",
                "David Lillis"
            ],
            "abstract": "In this paper, we describe our approach in the shared task: COVID-19 event\nextraction from Twitter. The objective of this task is to extract answers from\nCOVID-related tweets to a set of predefined slot-filling questions. Our\napproach treats the event extraction task as a question answering task by\nleveraging the transformer-based T5 text-to-text model.\n  According to the official evaluation scores returned, namely F1, our\nsubmitted run achieves competitive performance compared to other participating\nruns (Top 3). However, we argue that this evaluation may underestimate the\nactual performance of runs based on text-generation. Although some such runs\nmay answer the slot questions well, they may not be an exact string match for\nthe gold standard answers. To measure the extent of this underestimation, we\nadopt a simple exact-answer transformation method aiming at converting the\nwell-answered predictions to exactly-matched predictions. The results show that\nafter this transformation our run overall reaches the same level of performance\nas the best participating run and state-of-the-art F1 scores in three of five\nCOVID-related events. Our code is publicly available to aid reproducibility",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10053v1",
            "date": "2020-09-21 17:47:44+00:00",
            "title": "Latin BERT: A Contextual Language Model for Classical Philology",
            "authors": [
                "David Bamman",
                "Patrick J. Burns"
            ],
            "abstract": "We present Latin BERT, a contextual language model for the Latin language,\ntrained on 642.7 million words from a variety of sources spanning the Classical\nera to the 21st century. In a series of case studies, we illustrate the\naffordances of this language-specific model both for work in natural language\nprocessing for Latin and in using computational methods for traditional\nscholarship: we show that Latin BERT achieves a new state of the art for\npart-of-speech tagging on all three Universal Dependency datasets for Latin and\ncan be used for predicting missing text (including critical emendations); we\ncreate a new dataset for assessing word sense disambiguation for Latin and\ndemonstrate that Latin BERT outperforms static word embeddings; and we show\nthat it can be used for semantically-informed search by querying contextual\nnearest neighbors. We publicly release trained models to help drive future work\nin this space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10056v1",
            "date": "2020-09-21 17:48:43+00:00",
            "title": "Composed Variational Natural Language Generation for Few-shot Intents",
            "authors": [
                "Congying Xia",
                "Caiming Xiong",
                "Philip Yu",
                "Richard Socher"
            ],
            "abstract": "In this paper, we focus on generating training examples for few-shot intents\nin the realistic imbalanced scenario. To build connections between existing\nmany-shot intents and few-shot intents, we consider an intent as a combination\nof a domain and an action, and propose a composed variational natural language\ngenerator (CLANG), a transformer-based conditional variational autoencoder.\nCLANG utilizes two latent variables to represent the utterances corresponding\nto two different independent parts (domain and action) in the intent, and the\nlatent variables are composed together to generate natural examples.\nAdditionally, to improve the generator learning, we adopt the contrastive\nregularization loss that contrasts the in-class with the out-of-class utterance\ngeneration given the intent. To evaluate the quality of the generated\nutterances, experiments are conducted on the generalized few-shot intent\ndetection task. Empirical results show that our proposed model achieves\nstate-of-the-art performances on two real-world intent detection datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10155v1",
            "date": "2020-09-21 19:54:42+00:00",
            "title": "\"When they say weed causes depression, but it's your fav antidepressant\": Knowledge-aware Attention Framework for Relationship Extraction",
            "authors": [
                "Shweta Yadav",
                "Usha Lokala",
                "Raminta Daniulaityte",
                "Krishnaprasad Thirunarayan",
                "Francois Lamy",
                "Amit Sheth"
            ],
            "abstract": "With the increasing legalization of medical and recreational use of cannabis,\nmore research is needed to understand the association between depression and\nconsumer behavior related to cannabis consumption. Big social media data has\npotential to provide deeper insights about these associations to public health\nanalysts. In this interdisciplinary study, we demonstrate the value of\nincorporating domain-specific knowledge in the learning process to identify the\nrelationships between cannabis use and depression. We develop an end-to-end\nknowledge infused deep learning framework (Gated-K-BERT) that leverages the\npre-trained BERT language representation model and domain-specific declarative\nknowledge source (Drug Abuse Ontology (DAO)) to jointly extract entities and\ntheir relationship using gated fusion sharing mechanism. Our model is further\ntailored to provide more focus to the entities mention in the sentence through\nentity-position aware attention layer, where ontology is used to locate the\ntarget entities position. Experimental results show that inclusion of the\nknowledge-aware attentive representation in association with BERT can extract\nthe cannabis-depression relationship with better coverage in comparison to the\nstate-of-the-art relation extractor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10205v2",
            "date": "2020-09-21 22:34:13+00:00",
            "title": "The Persian Dependency Treebank Made Universal",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Pegah Safari",
                "Amirsaeid Moloodi",
                "Alireza Nourian"
            ],
            "abstract": "We describe an automatic method for converting the Persian Dependency\nTreebank (Rasooli et al, 2013) to Universal Dependencies. This treebank\ncontains 29107 sentences. Our experiments along with manual linguistic analysis\nshow that our data is more compatible with Universal Dependencies than the\nUppsala Persian Universal Dependency Treebank (Seraji et al., 2016), and is\nlarger in size and more diverse in vocabulary. Our data brings in a labeled\nattachment F-score of 85.2 in supervised parsing. Our delexicalized\nPersian-to-English parser transfer experiments show that a parsing model\ntrained on our data is ~2% absolutely more accurate than that of Seraji et al.\n(2016) in terms of labeled attachment score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10229v3",
            "date": "2020-09-22 00:11:32+00:00",
            "title": "An Empirical Study on Neural Keyphrase Generation",
            "authors": [
                "Rui Meng",
                "Xingdi Yuan",
                "Tong Wang",
                "Sanqiang Zhao",
                "Adam Trischler",
                "Daqing He"
            ],
            "abstract": "Recent years have seen a flourishing of neural keyphrase generation (KPG)\nworks, including the release of several large-scale datasets and a host of new\nmodels to tackle them. Model performance on KPG tasks has increased\nsignificantly with evolving deep learning research. However, there lacks a\ncomprehensive comparison among different model designs, and a thorough\ninvestigation on related factors that may affect a KPG system's generalization\nperformance. In this empirical study, we aim to fill this gap by providing\nextensive experimental results and analyzing the most crucial factors impacting\nthe generalizability of KPG models. We hope this study can help clarify some of\nthe uncertainties surrounding the KPG task and facilitate future research on\nthis topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10288v2",
            "date": "2020-09-22 02:35:45+00:00",
            "title": "Towards Causal Explanation Detection with Pyramid Salient-Aware Network",
            "authors": [
                "Xinyu Zuo",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao"
            ],
            "abstract": "Causal explanation analysis (CEA) can assist us to understand the reasons\nbehind daily events, which has been found very helpful for understanding the\ncoherence of messages. In this paper, we focus on Causal Explanation Detection,\nan important subtask of causal explanation analysis, which determines whether a\ncausal explanation exists in one message. We design a Pyramid Salient-Aware\nNetwork (PSAN) to detect causal explanations on messages. PSAN can assist in\ncausal explanation detection via capturing the salient semantics of discourses\ncontained in their keywords with a bottom graph-based word-level salient\nnetwork. Furthermore, PSAN can modify the dominance of discourses via a top\nattention-based discourse-level salient network to enhance explanatory\nsemantics of messages. The experiments on the commonly used dataset of CEA\nshows that the PSAN outperforms the state-of-the-art method by 1.8% F1 value on\nthe Causal Explanation Detection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10290v1",
            "date": "2020-09-22 02:48:48+00:00",
            "title": "Event Coreference Resolution via a Multi-loss Neural Network without Using Argument Information",
            "authors": [
                "Xinyu Zuo",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao"
            ],
            "abstract": "Event coreference resolution(ECR) is an important task in Natural Language\nProcessing (NLP) and nearly all the existing approaches to this task rely on\nevent argument information. However, these methods tend to suffer from error\npropagation from the stage of event argument extraction. Besides, not every\nevent mention contains all arguments of an event, and argument information may\nconfuse the model that events have arguments to detect event coreference in\nreal text. Furthermore, the context information of an event is useful to infer\nthe coreference between events. Thus, in order to reduce the errors propagated\nfrom event argument extraction and use context information effectively, we\npropose a multi-loss neural network model that does not need any argument\ninformation to do the within-document event coreference resolution task and\nachieve a significant performance than the state-of-the-art methods.",
            "journal ref": "SCIENCE CHINA Information Sciences, Volume 62, Issue\n  11:212101(2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10321v1",
            "date": "2020-09-22 05:08:48+00:00",
            "title": "Deep Reinforcement Learning for On-line Dialogue State Tracking",
            "authors": [
                "Zhi Chen",
                "Lu Chen",
                "Xiang Zhou",
                "Kai Yu"
            ],
            "abstract": "Dialogue state tracking (DST) is a crucial module in dialogue management. It\nis usually cast as a supervised training problem, which is not convenient for\non-line optimization. In this paper, a novel companion teaching based deep\nreinforcement learning (DRL) framework for on-line DST optimization is\nproposed. To the best of our knowledge, this is the first effort to optimize\nthe DST module within DRL framework for on-line task-oriented spoken dialogue\nsystems. In addition, dialogue policy can be further jointly updated.\nExperiments show that on-line DST optimization can effectively improve the\ndialogue manager performance while keeping the flexibility of using predefined\npolicy. Joint training of both DST and policy can further improve the\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10326v1",
            "date": "2020-09-22 05:39:31+00:00",
            "title": "Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management",
            "authors": [
                "Zhi Chen",
                "Lu Chen",
                "Xiaoyuan Liu",
                "Kai Yu"
            ],
            "abstract": "The task-oriented spoken dialogue system (SDS) aims to assist a human user in\naccomplishing a specific task (e.g., hotel booking). The dialogue management is\na core part of SDS. There are two main missions in dialogue management:\ndialogue belief state tracking (summarising conversation history) and dialogue\ndecision-making (deciding how to reply to the user). In this work, we only\nfocus on devising a policy that chooses which dialogue action to respond to the\nuser. The sequential system decision-making process can be abstracted into a\npartially observable Markov decision process (POMDP). Under this framework,\nreinforcement learning approaches can be used for automated policy\noptimization. In the past few years, there are many deep reinforcement learning\n(DRL) algorithms, which use neural networks (NN) as function approximators,\ninvestigated for dialogue policy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10355v1",
            "date": "2020-09-22 07:23:02+00:00",
            "title": "Structured Hierarchical Dialogue Policy with Graph Neural Networks",
            "authors": [
                "Zhi Chen",
                "Xiaoyuan Liu",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Dialogue policy training for composite tasks, such as restaurant reservation\nin multiple places, is a practically important and challenging problem.\nRecently, hierarchical deep reinforcement learning (HDRL) methods have achieved\ngood performance in composite tasks. However, in vanilla HDRL, both top-level\nand low-level policies are all represented by multi-layer perceptrons (MLPs)\nwhich take the concatenation of all observations from the environment as the\ninput for predicting actions. Thus, traditional HDRL approach often suffers\nfrom low sampling efficiency and poor transferability. In this paper, we\naddress these problems by utilizing the flexibility of graph neural networks\n(GNNs). A novel ComNet is proposed to model the structure of a hierarchical\nagent. The performance of ComNet is tested on composited tasks of the PyDial\nbenchmark. Experiments show that ComNet outperforms vanilla HDRL systems with\nperformance close to the upper bound. It not only achieves sample efficiency\nbut also is more robust to noise while maintaining the transferability to other\ncomposite tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10430v1",
            "date": "2020-09-22 10:15:09+00:00",
            "title": "Dual Learning for Dialogue State Tracking",
            "authors": [
                "Zhi Chen",
                "Lu Chen",
                "Yanbin Zhao",
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "In task-oriented multi-turn dialogue systems, dialogue state refers to a\ncompact representation of the user goal in the context of dialogue history.\nDialogue state tracking (DST) is to estimate the dialogue state at each turn.\nDue to the dependency on complicated dialogue history contexts, DST data\nannotation is more expensive than single-sentence language understanding, which\nmakes the task more challenging. In this work, we formulate DST as a sequence\ngeneration problem and propose a novel dual-learning framework to make full use\nof unlabeled data. In the dual-learning framework, there are two agents: the\nprimal tracker agent (utterance-to-state generator) and the dual utterance\ngenerator agent (state-to-utterance genera-tor). Compared with traditional\nsupervised learning framework, dual learning can iteratively update both agents\nthrough the reconstruction error and reward signal respectively without labeled\ndata. Reward sparsity problem is hard to solve in previous DST methods. In this\nwork, the reformulation of DST as a sequence generation model effectively\nalleviates this problem. We call this primal tracker agent dual-DST.\nExperimental results on MultiWOZ2.1 dataset show that the proposed dual-DST\nworks very well, especially when labelled data is limited. It achieves\ncomparable performance to the system where labeled data is fully used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10435v1",
            "date": "2020-09-22 10:27:18+00:00",
            "title": "CREDIT: Coarse-to-Fine Sequence Generation for Dialogue State Tracking",
            "authors": [
                "Zhi Chen",
                "Lu Chen",
                "Zihan Xu",
                "Yanbin Zhao",
                "Su Zhu",
                "Kai Yu"
            ],
            "abstract": "In dialogue systems, a dialogue state tracker aims to accurately find a\ncompact representation of the current dialogue status, based on the entire\ndialogue history. While previous approaches often define dialogue states as a\ncombination of separate triples ({\\em domain-slot-value}), in this paper, we\nemploy a structured state representation and cast dialogue state tracking as a\nsequence generation problem. Based on this new formulation, we propose a {\\bf\nC}oa{\\bf R}s{\\bf E}-to-fine {\\bf DI}alogue state {\\bf T}racking ({\\bf CREDIT})\napproach. Taking advantage of the structured state representation, which is a\nmarked language sequence, we can further fine-tune the pre-trained model (by\nsupervised learning) by optimizing natural language metrics with the policy\ngradient method. Like all generative state tracking methods, CREDIT does not\nrely on pre-defined dialogue ontology enumerating all possible slot values.\nExperiments demonstrate our tracker achieves encouraging joint goal accuracy\nfor the five domains in MultiWOZ 2.0 and MultiWOZ 2.1 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10447v2",
            "date": "2020-09-22 11:02:21+00:00",
            "title": "SUMBT+LaRL: End-to-end Neural Task-oriented Dialog System with Reinforcement Learning",
            "authors": [
                "Hwaran Lee",
                "Seokhwan Jo",
                "HyungJun Kim",
                "Sangkeun Jung",
                "Tae-Yoon Kim"
            ],
            "abstract": "The recent advent of neural approaches for developing each dialog component\nin task-oriented dialog systems has remarkably improved, yet optimizing the\noverall system performance remains a challenge. In this paper, we propose an\nend-to-end trainable neural dialog system with reinforcement learning, named\nSUMBT+LaRL. The SUMBT+ estimates user-acts as well as dialog belief states, and\nthe LaRL models latent system action spaces and generates responses given the\nestimated contexts. We experimentally demonstrate that the training framework\nin which the SUMBT+ and LaRL are separately pretrained and then the entire\nsystem is fine-tuned significantly increases dialog success rates. We propose\nnew success criteria for reinforcement learning to the end-to-end dialog system\nas well as provide experimental analysis on a different result aspect depending\non the success criteria and evaluation methods. Consequently, our model\nachieved the new state-of-the-art success rate of 85.4% on corpus-based\nevaluation, and a comparable success rate of 81.40% on simulator-based\nevaluation provided by the DSTC8 challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10542v1",
            "date": "2020-09-22 13:31:37+00:00",
            "title": "Context-theoretic Semantics for Natural Language: an Algebraic Framework",
            "authors": [
                "Daoud Clarke"
            ],
            "abstract": "Techniques in which words are represented as vectors have proved useful in\nmany applications in computational linguistics, however there is currently no\ngeneral semantic formalism for representing meaning in terms of vectors. We\npresent a framework for natural language semantics in which words, phrases and\nsentences are all represented as vectors, based on a theoretical analysis which\nassumes that meaning is determined by context.\n  In the theoretical analysis, we define a corpus model as a mathematical\nabstraction of a text corpus. The meaning of a string of words is assumed to be\na vector representing the contexts it occurs in in the corpus model. Based on\nthis assumption, we can show that the vector representations of words can be\nconsidered as elements of an algebra over a field. We note that in applications\nof vector spaces to representing meanings of words there is an underlying\nlattice structure; we interpret the partial ordering of the lattice as\ndescribing entailment between meanings. We also define the context-theoretic\nprobability of a string, and, based on this and the lattice structure, a degree\nof entailment between strings.\n  Together these properties form guidelines as to how to construct semantic\nrepresentations within the framework. A context theory is an implementation of\nthe framework; in an implementation strings are represented as vectors with the\nproperties deduced from the theoretical analysis.\n  We show how to incorporate logical semantics into context theories; this\nenables us to represent statistical information about uncertainty by taking\nweighted sums of individual representations. We also use the framework to\nanalyse approaches to the task of recognising textual entailment, to\nontological representations of meaning and to representing syntactic structure.\nFor the latter, we give new algebraic descriptions of link grammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10557v2",
            "date": "2020-09-22 13:55:34+00:00",
            "title": "GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis",
            "authors": [
                "Huaishao Luo",
                "Lei Ji",
                "Tianrui Li",
                "Nan Duan",
                "Daxin Jiang"
            ],
            "abstract": "In this paper, we focus on the imbalance issue, which is rarely studied in\naspect term extraction and aspect sentiment classification when regarding them\nas sequence labeling tasks. Besides, previous works usually ignore the\ninteraction between aspect terms when labeling polarities. We propose a\nGRadient hArmonized and CascadEd labeling model (GRACE) to solve these\nproblems. Specifically, a cascaded labeling module is developed to enhance the\ninterchange between aspect terms and improve the attention of sentiment tokens\nwhen labeling sentiment polarities. The polarities sequence is designed to\ndepend on the generated aspect terms labels. To alleviate the imbalance issue,\nwe extend the gradient harmonized mechanism used in object detection to the\naspect-based sentiment analysis by adjusting the weight of each label\ndynamically. The proposed GRACE adopts a post-pretraining BERT as its backbone.\nExperimental results demonstrate that the proposed model achieves consistency\nimprovement on multiple benchmark datasets and generates state-of-the-art\nresults.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10680v2",
            "date": "2020-09-22 16:55:49+00:00",
            "title": "AutoRC: Improving BERT Based Relation Classification Models via Architecture Search",
            "authors": [
                "Wei Zhu",
                "Xipeng Qiu",
                "Yuan Ni",
                "Guotong Xie"
            ],
            "abstract": "Although BERT based relation classification (RC) models have achieved\nsignificant improvements over the traditional deep learning models, it seems\nthat no consensus can be reached on what is the optimal architecture. Firstly,\nthere are multiple alternatives for entity span identification. Second, there\nare a collection of pooling operations to aggregate the representations of\nentities and contexts into fixed length vectors. Third, it is difficult to\nmanually decide which feature vectors, including their interactions, are\nbeneficial for classifying the relation types. In this work, we design a\ncomprehensive search space for BERT based RC models and employ neural\narchitecture search (NAS) method to automatically discover the design choices\nmentioned above. Experiments on seven benchmark RC tasks show that our method\nis efficient and effective in finding better architectures than the baseline\nBERT based RC model. Ablation study demonstrates the necessity of our search\nspace design and the effectiveness of our search method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10792v1",
            "date": "2020-09-22 20:13:48+00:00",
            "title": "Ghmerti at SemEval-2019 Task 6: A Deep Word- and Character-based Approach to Offensive Language Identification",
            "authors": [
                "Ehsan Doostmohammadi",
                "Hossein Sameti",
                "Ali Saffar"
            ],
            "abstract": "This paper presents the models submitted by Ghmerti team for subtasks A and B\nof the OffensEval shared task at SemEval 2019. OffensEval addresses the problem\nof identifying and categorizing offensive language in social media in three\nsubtasks; whether or not a content is offensive (subtask A), whether it is\ntargeted (subtask B) towards an individual, a group, or other entities (subtask\nC). The proposed approach includes character-level Convolutional Neural\nNetwork, word-level Recurrent Neural Network, and some preprocessing. The\nperformance achieved by the proposed model for subtask A is 77.93%\nmacro-averaged F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10794v1",
            "date": "2020-09-22 20:17:45+00:00",
            "title": "Investigating Machine Learning Methods for Language and Dialect Identification of Cuneiform Texts",
            "authors": [
                "Ehsan Doostmohammadi",
                "Minoo Nassajian"
            ],
            "abstract": "Identification of the languages written using cuneiform symbols is a\ndifficult task due to the lack of resources and the problem of tokenization.\nThe Cuneiform Language Identification task in VarDial 2019 addresses the\nproblem of identifying seven languages and dialects written in cuneiform;\nSumerian and six dialects of Akkadian language: Old Babylonian, Middle\nBabylonian Peripheral, Standard Babylonian, Neo-Babylonian, Late Babylonian,\nand Neo-Assyrian. This paper describes the approaches taken by SharifCL team to\nthis problem in VarDial 2019. The best result belongs to an ensemble of Support\nVector Machines and a naive Bayes classifier, both working on character-level\nfeatures, with macro-averaged F1-score of 72.10%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10795v2",
            "date": "2020-09-22 20:19:41+00:00",
            "title": "Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics",
            "authors": [
                "Swabha Swayamdipta",
                "Roy Schwartz",
                "Nicholas Lourie",
                "Yizhong Wang",
                "Hannaneh Hajishirzi",
                "Noah A. Smith",
                "Yejin Choi"
            ],
            "abstract": "Large datasets have become commonplace in NLP research. However, the\nincreased emphasis on data quantity has made it challenging to assess the\nquality of data. We introduce Data Maps---a model-based tool to characterize\nand diagnose datasets. We leverage a largely ignored source of information: the\nbehavior of the model on individual instances during training (training\ndynamics) for building data maps. This yields two intuitive measures for each\nexample---the model's confidence in the true class, and the variability of this\nconfidence across epochs---obtained in a single run of training. Experiments\nacross four datasets show that these model-dependent measures reveal three\ndistinct regions in the data map, each with pronounced characteristics. First,\nour data maps show the presence of \"ambiguous\" regions with respect to the\nmodel, which contribute the most towards out-of-distribution generalization.\nSecond, the most populous regions in the data are \"easy to learn\" for the\nmodel, and play an important role in model optimization. Finally, data maps\nuncover a region with instances that the model finds \"hard to learn\"; these\noften correspond to labeling errors. Our results indicate that a shift in focus\nfrom quantity to quality of data could lead to robust models and improved\nout-of-distribution generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10815v2",
            "date": "2020-09-22 21:02:14+00:00",
            "title": "Keeping Up Appearances: Computational Modeling of Face Acts in Persuasion Oriented Discussions",
            "authors": [
                "Ritam Dutt",
                "Rishabh Joshi",
                "Carolyn Penstein Rose"
            ],
            "abstract": "The notion of face refers to the public self-image of an individual that\nemerges both from the individual's own actions as well as from the interaction\nwith others. Modeling face and understanding its state changes throughout a\nconversation is critical to the study of maintenance of basic human needs in\nand through interaction. Grounded in the politeness theory of Brown and\nLevinson (1978), we propose a generalized framework for modeling face acts in\npersuasion conversations, resulting in a reliable coding manual, an annotated\ncorpus, and computational models. The framework reveals insights about\ndifferences in face act utilization between asymmetric roles in persuasion\nconversations. Using computational models, we are able to successfully identify\nface acts as well as predict a key conversational outcome (e.g. donation\nsuccess). Finally, we model a latent representation of the conversational state\nto analyze the impact of predicted face acts on the probability of a positive\nconversational outcome and observe several correlations that corroborate\nprevious findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10855v1",
            "date": "2020-09-22 23:21:04+00:00",
            "title": "Controlling Style in Generated Dialogue",
            "authors": [
                "Eric Michael Smith",
                "Diana Gonzalez-Rico",
                "Emily Dinan",
                "Y-Lan Boureau"
            ],
            "abstract": "Open-domain conversation models have become good at generating\nnatural-sounding dialogue, using very large architectures with billions of\ntrainable parameters. The vast training data required to train these\narchitectures aggregates many different styles, tones, and qualities. Using\nthat data to train a single model makes it difficult to use the model as a\nconsistent conversational agent, e.g. with a stable set of persona traits and a\ntypical style of expression. Several architectures affording control mechanisms\nover generation architectures have been proposed, each with different\ntrade-offs. However, it remains unclear whether their use in dialogue is\nviable, and what the trade-offs look like with the most recent state-of-the-art\nconversational architectures. In this work, we adapt three previously proposed\ncontrollable generation architectures to open-domain dialogue generation,\ncontrolling the style of the generation to match one among about 200 possible\nstyles. We compare their respective performance and tradeoffs, and show how\nthey can be used to provide insights into existing conversational datasets, and\ngenerate a varied set of styled conversation replies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11005v3",
            "date": "2020-09-23 08:49:39+00:00",
            "title": "Exploiting Vietnamese Social Media Characteristics for Textual Emotion Recognition in Vietnamese",
            "authors": [
                "Khang Phuoc-Quy Nguyen",
                "Kiet Van Nguyen"
            ],
            "abstract": "Textual emotion recognition has been a promising research topic in recent\nyears. Many researchers aim to build more accurate and robust emotion detection\nsystems. In this paper, we conduct several experiments to indicate how data\npre-processing affects a machine learning method on textual emotion\nrecognition. These experiments are performed on the Vietnamese Social Media\nEmotion Corpus (UIT-VSMEC) as the benchmark dataset. We explore Vietnamese\nsocial media characteristics to propose different pre-processing techniques,\nand key-clause extraction with emotional context to improve the machine\nperformance on UIT-VSMEC. Our experimental evaluation shows that with\nappropriate pre-processing techniques based on Vietnamese social media\ncharacteristics, Multinomial Logistic Regression (MLR) achieves the best\nF1-score of 64.40%, a significant improvement of 4.66% over the CNN model built\nby the authors of UIT-VSMEC (59.74%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11023v2",
            "date": "2020-09-23 09:45:23+00:00",
            "title": "The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets",
            "authors": [
                "Oana-Maria Camburu",
                "Eleonora Giunchiglia",
                "Jakob Foerster",
                "Thomas Lukasiewicz",
                "Phil Blunsom"
            ],
            "abstract": "For neural models to garner widespread public trust and ensure fairness, we\nmust have human-intelligible explanations for their predictions. Recently, an\nincreasing number of works focus on explaining the predictions of neural models\nin terms of the relevance of the input features. In this work, we show that\nfeature-based explanations pose problems even for explaining trivial models. We\nshow that, in certain cases, there exist at least two ground-truth\nfeature-based explanations, and that, sometimes, neither of them is enough to\nprovide a complete view of the decision-making process of the model. Moreover,\nwe show that two popular classes of explainers, Shapley explainers and minimal\nsufficient subsets explainers, target fundamentally different types of\nground-truth explanations, despite the apparently implicit assumption that\nexplainers should look for one specific feature-based explanation. These\nfindings bring an additional dimension to consider in both developing and\nchoosing explainers.",
            "journal ref": "Explainable Agency in Artificial Intelligence Workshop at AAAI\n  2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11027v1",
            "date": "2020-09-23 09:52:28+00:00",
            "title": "KoBE: Knowledge-Based Machine Translation Evaluation",
            "authors": [
                "Zorik Gekhman",
                "Roee Aharoni",
                "Genady Beryozkin",
                "Markus Freitag",
                "Wolfgang Macherey"
            ],
            "abstract": "We propose a simple and effective method for machine translation evaluation\nwhich does not require reference translations. Our approach is based on (1)\ngrounding the entity mentions found in each source sentence and candidate\ntranslation against a large-scale multilingual knowledge base, and (2)\nmeasuring the recall of the grounded entities found in the candidate vs. those\nfound in the source. Our approach achieves the highest correlation with human\njudgements on 9 out of the 18 language pairs from the WMT19 benchmark for\nevaluation without references, which is the largest number of wins for a single\nevaluation method on this task. On 4 language pairs, we also achieve higher\ncorrelation with human judgements than BLEU. To foster further research, we\nrelease a dataset containing 1.8 million grounded entity mentions across 18\nlanguage pairs from the WMT19 metrics track data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11032v3",
            "date": "2020-09-23 10:02:10+00:00",
            "title": "Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling",
            "authors": [
                "Arie Cattan",
                "Alon Eirew",
                "Gabriel Stanovsky",
                "Mandar Joshi",
                "Ido Dagan"
            ],
            "abstract": "Recent evaluation protocols for Cross-document (CD) coreference resolution\nhave often been inconsistent or lenient, leading to incomparable results across\nworks and overestimation of performance. To facilitate proper future research\non this task, our primary contribution is proposing a pragmatic evaluation\nmethodology which assumes access to only raw text -- rather than assuming gold\nmentions, disregards singleton prediction, and addresses typical targeted\nsettings in CD coreference resolution. Aiming to set baseline results for\nfuture research that would follow our evaluation methodology, we build the\nfirst end-to-end model for this task. Our model adapts and extends recent\nneural models for within-document coreference resolution to address the CD\ncoreference setting, which outperforms state-of-the-art results by a\nsignificant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11136v1",
            "date": "2020-09-23 13:28:38+00:00",
            "title": "Seq2Edits: Sequence Transduction Using Span-level Edit Operations",
            "authors": [
                "Felix Stahlberg",
                "Shankar Kumar"
            ],
            "abstract": "We propose Seq2Edits, an open-vocabulary approach to sequence editing for\nnatural language processing (NLP) tasks with a high degree of overlap between\ninput and output texts. In this approach, each sequence-to-sequence\ntransduction is represented as a sequence of edit operations, where each\noperation either replaces an entire source span with target tokens or keeps it\nunchanged. We evaluate our method on five NLP tasks (text normalization,\nsentence fusion, sentence splitting & rephrasing, text simplification, and\ngrammatical error correction) and report competitive results across the board.\nFor grammatical error correction, our method speeds up inference by up to 5.2x\ncompared to full sequence models because inference time depends on the number\nof edits rather than the number of target tokens. For text normalization,\nsentence fusion, and grammatical error correction, our approach improves\nexplainability by associating each edit operation with a human-readable tag.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11144v1",
            "date": "2020-09-23 13:41:27+00:00",
            "title": "Evolution of Part-of-Speech in Classical Chinese",
            "authors": [
                "Bai Li"
            ],
            "abstract": "Classical Chinese is a language notable for its word class flexibility: the\nsame word may often be used as a noun or a verb. Bisang (2008) claimed that\nClassical Chinese is a precategorical language, where the syntactic position of\na word determines its part-of-speech category. In this paper, we apply\nentropy-based metrics to evaluate these claims on historical corpora. We\nfurther explore differences between nouns and verbs in Classical Chinese: using\npsycholinguistic norms, we find a positive correlation between concreteness and\nnoun usage. Finally, we align character embeddings from Classical and Modern\nChinese, and find that verbs undergo more semantic change than nouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11201v2",
            "date": "2020-09-23 15:07:33+00:00",
            "title": "Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages",
            "authors": [
                "Xavier Garcia",
                "Aditya Siddhant",
                "Orhan Firat",
                "Ankur P. Parikh"
            ],
            "abstract": "Unsupervised translation has reached impressive performance on resource-rich\nlanguage pairs such as English-French and English-German. However, early\nstudies have shown that in more realistic settings involving low-resource, rare\nlanguages, unsupervised translation performs poorly, achieving less than 3.0\nBLEU. In this work, we show that multilinguality is critical to making\nunsupervised systems practical for low-resource settings. In particular, we\npresent a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali,\nSinhala, and Turkish) to and from English directions, which leverages\nmonolingual and auxiliary parallel data from other high-resource language pairs\nvia a three-stage training scheme. We outperform all current state-of-the-art\nunsupervised baselines for these languages, achieving gains of up to 14.4 BLEU.\nAdditionally, we outperform a large collection of supervised WMT submissions\nfor various language pairs as well as match the performance of the current\nstate-of-the-art supervised model for Nepali-English. We conduct a series of\nablation studies to establish the robustness of our model under different\ndegrees of data quality, as well as to analyze the factors which led to the\nsuperior performance of the proposed approach over traditional unsupervised\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11260v1",
            "date": "2020-09-23 17:12:06+00:00",
            "title": "A Token-wise CNN-based Method for Sentence Compression",
            "authors": [
                "Weiwei Hou",
                "Hanna Suominen",
                "Piotr Koniusz",
                "Sabrina Caldwell",
                "Tom Gedeon"
            ],
            "abstract": "Sentence compression is a Natural Language Processing (NLP) task aimed at\nshortening original sentences and preserving their key information. Its\napplications can benefit many fields e.g. one can build tools for language\neducation. However, current methods are largely based on Recurrent Neural\nNetwork (RNN) models which suffer from poor processing speed. To address this\nissue, in this paper, we propose a token-wise Convolutional Neural Network, a\nCNN-based model along with pre-trained Bidirectional Encoder Representations\nfrom Transformers (BERT) features for deletion-based sentence compression. We\nalso compare our model with RNN-based models and fine-tuned BERT. Although one\nof the RNN-based models outperforms marginally other models given the same\ninput, our CNN-based model was ten times faster than the RNN-based approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11321v1",
            "date": "2020-09-23 18:06:52+00:00",
            "title": "Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining",
            "authors": [
                "Ananya B. Sai",
                "Akash Kumar Mohankumar",
                "Siddhartha Arora",
                "Mitesh M. Khapra"
            ],
            "abstract": "There is an increasing focus on model-based dialog evaluation metrics such as\nADEM, RUBER, and the more recent BERT-based metrics. These models aim to assign\na high score to all relevant responses and a low score to all irrelevant\nresponses. Ideally, such models should be trained using multiple relevant and\nirrelevant responses for any given context. However, no such data is publicly\navailable, and hence existing models are usually trained using a single\nrelevant response and multiple randomly selected responses from other contexts\n(random negatives). To allow for better training and robust evaluation of\nmodel-based metrics, we introduce the DailyDialog++ dataset, consisting of (i)\nfive relevant responses for each context and (ii) five adversarially crafted\nirrelevant responses for each context. Using this dataset, we first show that\neven in the presence of multiple correct references, n-gram based metrics and\nembedding based metrics do not perform well at separating relevant responses\nfrom even random negatives. While model-based metrics perform better than\nn-gram and embedding based metrics on random negatives, their performance drops\nsubstantially when evaluated on adversarial examples. To check if large scale\npretraining could help, we propose a new BERT-based evaluation metric called\nDEB, which is pretrained on 727M Reddit conversations and then finetuned on our\ndataset. DEB significantly outperforms existing models, showing better\ncorrelation with human judgements and better performance on random negatives\n(88.27% accuracy). However, its performance again drops substantially, when\nevaluated on adversarial responses, thereby highlighting that even large-scale\npretrained evaluation models are not robust to the adversarial examples in our\ndataset. The dataset and code are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11340v2",
            "date": "2020-09-23 19:03:58+00:00",
            "title": "The importance of fillers for text representations of speech transcripts",
            "authors": [
                "Tanvi Dinkar",
                "Pierre Colombo",
                "Matthieu Labeau",
                "Chlo\u00e9 Clavel"
            ],
            "abstract": "While being an essential component of spoken language, fillers (e.g.\"um\" or\n\"uh\") often remain overlooked in Spoken Language Understanding (SLU) tasks. We\nexplore the possibility of representing them with deep contextualised\nembeddings, showing improvements on modelling spoken language and two\ndownstream tasks - predicting a speaker's stance and expressed confidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11382v1",
            "date": "2020-09-23 21:22:15+00:00",
            "title": "Multi-Pass Transformer for Machine Translation",
            "authors": [
                "Peng Gao",
                "Chiori Hori",
                "Shijie Geng",
                "Takaaki Hori",
                "Jonathan Le Roux"
            ],
            "abstract": "In contrast with previous approaches where information flows only towards\ndeeper layers of a stack, we consider a multi-pass transformer (MPT)\narchitecture in which earlier layers are allowed to process information in\nlight of the output of later layers. To maintain a directed acyclic graph\nstructure, the encoder stack of a transformer is repeated along a new\nmulti-pass dimension, keeping the parameters tied, and information is allowed\nto proceed unidirectionally both towards deeper layers within an encoder stack\nand towards any layer of subsequent stacks. We consider both soft (i.e.,\ncontinuous) and hard (i.e., discrete) connections between parallel encoder\nstacks, relying on a neural architecture search to find the best connection\npattern in the hard case. We perform an extensive ablation study of the\nproposed MPT architecture and compare it with other state-of-the-art\ntransformer architectures. Surprisingly, Base Transformer equipped with MPT can\nsurpass the performance of Large Transformer on the challenging machine\ntranslation En-De and En-Fr datasets. In the hard connection case, the optimal\nconnection pattern found for En-De also leads to improved performance for\nEn-Fr.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11423v3",
            "date": "2020-09-24 00:35:26+00:00",
            "title": "Task-Oriented Dialogue as Dataflow Synthesis",
            "authors": [
                "Semantic Machines",
                "Jacob Andreas",
                "John Bufe",
                "David Burkett",
                "Charles Chen",
                "Josh Clausman",
                "Jean Crawford",
                "Kate Crim",
                "Jordan DeLoach",
                "Leah Dorner",
                "Jason Eisner",
                "Hao Fang",
                "Alan Guo",
                "David Hall",
                "Kristin Hayes",
                "Kellie Hill",
                "Diana Ho",
                "Wendy Iwaszuk",
                "Smriti Jha",
                "Dan Klein",
                "Jayant Krishnamurthy",
                "Theo Lanman",
                "Percy Liang",
                "Christopher H Lin",
                "Ilya Lintsbakh",
                "Andy McGovern",
                "Aleksandr Nisnevich",
                "Adam Pauls",
                "Dmitrij Petters",
                "Brent Read",
                "Dan Roth",
                "Subhro Roy",
                "Jesse Rusak",
                "Beth Short",
                "Div Slomin",
                "Ben Snyder",
                "Stephon Striplin",
                "Yu Su",
                "Zachary Tellman",
                "Sam Thomson",
                "Andrei Vorobev",
                "Izabela Witoszko",
                "Jason Wolfe",
                "Abby Wray",
                "Yuchen Zhang",
                "Alexander Zotov"
            ],
            "abstract": "We describe an approach to task-oriented dialogue in which dialogue state is\nrepresented as a dataflow graph. A dialogue agent maps each user utterance to a\nprogram that extends this graph. Programs include metacomputation operators for\nreference and revision that reuse dataflow fragments from previous turns. Our\ngraph-based state enables the expression and manipulation of complex user\nintents, and explicit metacomputation makes these intents easier for learned\nmodels to predict. We introduce a new dataset, SMCalFlow, featuring complex\ndialogues about events, weather, places, and people. Experiments show that\ndataflow graphs and metacomputation substantially improve representability and\npredictability in these natural dialogues. Additional experiments on the\nMultiWOZ dataset show that our dataflow representation enables an otherwise\noff-the-shelf sequence-to-sequence model to match the best existing\ntask-specific state tracking model. The SMCalFlow dataset and code for\nreplicating experiments are available at\nhttps://www.microsoft.com/en-us/research/project/dataflow-based-dialogue-semantic-machines.",
            "journal ref": "Transactions of the Association for Computational Linguistics 2020\n  Vol. 8, 556-571",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11462v2",
            "date": "2020-09-24 03:17:19+00:00",
            "title": "RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models",
            "authors": [
                "Samuel Gehman",
                "Suchin Gururangan",
                "Maarten Sap",
                "Yejin Choi",
                "Noah A. Smith"
            ],
            "abstract": "Pretrained neural language models (LMs) are prone to generating racist,\nsexist, or otherwise toxic language which hinders their safe deployment. We\ninvestigate the extent to which pretrained LMs can be prompted to generate\ntoxic language, and the effectiveness of controllable text generation\nalgorithms at preventing such toxic degeneration. We create and release\nRealToxicityPrompts, a dataset of 100K naturally occurring, sentence-level\nprompts derived from a large corpus of English web text, paired with toxicity\nscores from a widely-used toxicity classifier. Using RealToxicityPrompts, we\nfind that pretrained LMs can degenerate into toxic text even from seemingly\ninnocuous prompts. We empirically assess several controllable generation\nmethods, and find that while data- or compute-intensive methods (e.g., adaptive\npretraining on non-toxic data) are more effective at steering away from\ntoxicity than simpler solutions (e.g., banning \"bad\" words), no current method\nis failsafe against neural toxic degeneration. To pinpoint the potential cause\nof such persistent toxic degeneration, we analyze two web text corpora used to\npretrain several LMs (including GPT-2; Radford et. al, 2019), and find a\nsignificant amount of offensive, factually unreliable, and otherwise toxic\ncontent. Our work provides a test bed for evaluating toxic generations by LMs\nand stresses the need for better data selection processes for pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11473v2",
            "date": "2020-09-24 03:41:13+00:00",
            "title": "AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation",
            "authors": [
                "Huishuang Tian",
                "Kexin Yang",
                "Dayiheng Liu",
                "Jiancheng Lv"
            ],
            "abstract": "Ancient Chinese is the essence of Chinese culture. There are several natural\nlanguage processing tasks of ancient Chinese domain, such as ancient-modern\nChinese translation, poem generation, and couplet generation. Previous studies\nusually use the supervised models which deeply rely on parallel data. However,\nit is difficult to obtain large-scale parallel data of ancient Chinese. In\norder to make full use of the more easily available monolingual ancient Chinese\ncorpora, we release AnchiBERT, a pre-trained language model based on the\narchitecture of BERT, which is trained on large-scale ancient Chinese corpora.\nWe evaluate AnchiBERT on both language understanding and generation tasks,\nincluding poem classification, ancient-modern Chinese translation, poem\ngeneration, and couplet generation. The experimental results show that\nAnchiBERT outperforms BERT as well as the non-pretrained models and achieves\nstate-of-the-art results in all cases.",
            "journal ref": "ijcnn 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11506v2",
            "date": "2020-09-24 06:17:10+00:00",
            "title": "Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems",
            "authors": [
                "Wei Zhao",
                "Mingyue Shang",
                "Yang Liu",
                "Liang Wang",
                "Jingming Liu"
            ],
            "abstract": "Automatic math word problem solving has attracted growing attention in recent\nyears. The evaluation datasets used by previous works have serious limitations\nin terms of scale and diversity. In this paper, we release a new large-scale\nand template-rich math word problem dataset named Ape210K. It consists of 210K\nChinese elementary school-level math problems, which is 9 times the size of the\nlargest public dataset Math23K. Each problem contains both the gold answer and\nthe equations needed to derive the answer. Ape210K is also of greater diversity\nwith 56K templates, which is 25 times more than Math23K. Our analysis shows\nthat solving Ape210K requires not only natural language understanding but also\ncommonsense knowledge. We expect Ape210K to be a benchmark for math word\nproblem solving systems. Experiments indicate that state-of-the-art models on\nthe Math23K dataset perform poorly on Ape210K. We propose a copy-augmented and\nfeature-enriched sequence to sequence (seq2seq) model, which outperforms\nexisting models by 3.2% on the Math23K dataset and serves as a strong baseline\nof the Ape210K dataset. The gap is still significant between human and our\nbaseline model, calling for further research efforts. We make Ape210K dataset\npublicly available at https://github.com/yuantiku/ape210k",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11523v2",
            "date": "2020-09-24 07:21:14+00:00",
            "title": "Grounded Compositional Outputs for Adaptive Language Modeling",
            "authors": [
                "Nikolaos Pappas",
                "Phoebe Mulcaire",
                "Noah A. Smith"
            ],
            "abstract": "Language models have emerged as a central component across NLP, and a great\ndeal of progress depends on the ability to cheaply adapt them (e.g., through\nfinetuning) to new domains and tasks. A language model's vocabulary$-$typically\nselected before training and permanently fixed later$-$affects its size and is\npart of what makes it resistant to such adaptation. Prior work has used\ncompositional input embeddings based on surface forms to ameliorate this issue.\nIn this work, we go one step beyond and propose a fully compositional output\nembedding layer for language models, which is further grounded in information\nfrom a structured lexicon (WordNet), namely semantically related words and\nfree-text definitions. To our knowledge, the result is the first word-level\nlanguage model with a size that does not depend on the training vocabulary. We\nevaluate the model on conventional language modeling as well as challenging\ncross-domain settings with an open vocabulary, finding that it matches or\noutperforms previous state-of-the-art output embedding methods and adaptation\napproaches. Our analysis attributes the improvements to sample efficiency: our\nmodel is more accurate for low-frequency words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11538v3",
            "date": "2020-09-24 08:04:37+00:00",
            "title": "Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training",
            "authors": [
                "Hai Ye",
                "Qingyu Tan",
                "Ruidan He",
                "Juntao Li",
                "Hwee Tou Ng",
                "Lidong Bing"
            ],
            "abstract": "Adapting pre-trained language models (PrLMs) (e.g., BERT) to new domains has\ngained much attention recently. Instead of fine-tuning PrLMs as done in most\nprevious work, we investigate how to adapt the features of PrLMs to new domains\nwithout fine-tuning. We explore unsupervised domain adaptation (UDA) in this\npaper. With the features from PrLMs, we adapt the models trained with labeled\ndata from the source domain to the unlabeled target domain. Self-training is\nwidely used for UDA which predicts pseudo labels on the target domain data for\ntraining. However, the predicted pseudo labels inevitably include noise, which\nwill negatively affect training a robust model. To improve the robustness of\nself-training, in this paper we present class-aware feature self-distillation\n(CFd) to learn discriminative features from PrLMs, in which PrLM features are\nself-distilled into a feature adaptation module and the features from the same\nclass are more tightly clustered. We further extend CFd to a cross-language\nsetting, in which language discrepancy is studied. Experiments on two\nmonolingual and multilingual Amazon review datasets show that CFd can\nconsistently improve the performance of self-training in cross-domain and\ncross-language settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11616v3",
            "date": "2020-09-24 11:45:39+00:00",
            "title": "N-LTP: A Open-source Neural Chinese Language Technology Platform with Pretrained Models",
            "authors": [
                "Wanxiang Che",
                "Yunlong Feng",
                "Libo Qin",
                "Ting Liu"
            ],
            "abstract": "We introduce N-LTP, an open-source neural language technology platform\nsupporting six fundamental Chinese NLP tasks: lexical analysis (Chinese word\nsegmentation, part-of-speech tagging, and named entity recognition), {syntactic\nparsing} (dependency parsing), and {semantic parsing} (semantic dependency\nparsing and semantic role labeling). Unlike the existing state-of-the-art\ntoolkits, such as Stanza, that adopt an independent model for each task, N-LTP\nadopts the multi-task framework by using a shared pre-trained model, which has\nthe advantage of capturing the shared knowledge across relevant Chinese tasks.\nIn addition, knowledge distillation where the single-task model teaches the\nmulti-task model is further introduced to encourage the multi-task model to\nsurpass its single-task teacher. Finally, we provide a collection of\neasy-to-use APIs and a visualization tool to make users easier to use and view\nthe processing results directly. To the best of our knowledge, this is the\nfirst toolkit to support six Chinese NLP fundamental tasks. Source code,\ndocumentation, and pre-trained models are available at \\url{https://ltp.ai/}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11692v1",
            "date": "2020-09-24 13:55:32+00:00",
            "title": "Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph",
            "authors": [
                "Haozhe Ji",
                "Pei Ke",
                "Shaohan Huang",
                "Furu Wei",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "Despite the success of generative pre-trained language models on a series of\ntext generation tasks, they still suffer in cases where reasoning over\nunderlying commonsense knowledge is required during generation. Existing\napproaches that integrate commonsense knowledge into generative pre-trained\nlanguage models simply transfer relational knowledge by post-training on\nindividual knowledge triples while ignoring rich connections within the\nknowledge graph. We argue that exploiting both the structural and semantic\ninformation of the knowledge graph facilitates commonsense-aware text\ngeneration. In this paper, we propose Generation with Multi-Hop Reasoning Flow\n(GRF) that enables pre-trained models with dynamic multi-hop reasoning on\nmulti-relational paths extracted from the external commonsense knowledge graph.\nWe empirically show that our model outperforms existing baselines on three text\ngeneration tasks that require reasoning over commonsense knowledge. We also\ndemonstrate the effectiveness of the dynamic multi-hop reasoning module with\nreasoning paths inferred by the model that provide rationale to the generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11753v1",
            "date": "2020-09-24 15:27:20+00:00",
            "title": "Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths",
            "authors": [
                "Haozhe Ji",
                "Pei Ke",
                "Shaohan Huang",
                "Furu Wei",
                "Minlie Huang"
            ],
            "abstract": "Commonsense explanation generation aims to empower the machine's sense-making\ncapability by generating plausible explanations to statements against\ncommonsense. While this task is easy to human, the machine still struggles to\ngenerate reasonable and informative explanations. In this work, we propose a\nmethod that first extracts the underlying concepts which are served as\n\\textit{bridges} in the reasoning chain and then integrates these concepts to\ngenerate the final explanation. To facilitate the reasoning process, we utilize\nexternal commonsense knowledge to build the connection between a statement and\nthe bridge concepts by extracting and pruning multi-hop paths to build a\nsubgraph. We design a bridge concept extraction model that first scores the\ntriples, routes the paths in the subgraph, and further selects bridge concepts\nwith weak supervision at both the triple level and the concept level. We\nconduct experiments on the commonsense explanation generation task and our\nmodel outperforms the state-of-the-art baselines in both automatic and human\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11832v1",
            "date": "2020-09-24 17:28:59+00:00",
            "title": "Novel Keyword Extraction and Language Detection Approaches",
            "authors": [
                "Malgorzata Pikies",
                "Andronicus Riyono",
                "Junade Ali"
            ],
            "abstract": "Fuzzy string matching and language classification are important tools in\nNatural Language Processing pipelines, this paper provides advances in both\nareas. We propose a fast novel approach to string tokenisation for fuzzy\nlanguage matching and experimentally demonstrate an 83.6% decrease in\nprocessing time with an estimated improvement in recall of 3.1% at the cost of\na 2.6% decrease in precision. This approach is able to work even where keywords\nare subdivided into multiple words, without needing to scan\ncharacter-to-character. So far there has been little work considering using\nmetadata to enhance language classification algorithms. We provide\nobservational data and find the Accept-Language header is 14% more likely to\nmatch the classification than the IP Address.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12046v1",
            "date": "2020-09-25 06:31:06+00:00",
            "title": "Controllable Text Generation with Focused Variation",
            "authors": [
                "Lei Shu",
                "Alexandros Papangelis",
                "Yi-Chia Wang",
                "Gokhan Tur",
                "Hu Xu",
                "Zhaleh Feizollahi",
                "Bing Liu",
                "Piero Molino"
            ],
            "abstract": "This work introduces Focused-Variation Network (FVN), a novel model to\ncontrol language generation. The main problems in previous controlled language\ngeneration models range from the difficulty of generating text according to the\ngiven attributes, to the lack of diversity of the generated texts. FVN\naddresses these issues by learning disjoint discrete latent spaces for each\nattribute inside codebooks, which allows for both controllability and\ndiversity, while at the same time generating fluent text. We evaluate FVN on\ntwo text generation datasets with annotated content and style, and show\nstate-of-the-art performance as assessed by automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12102v1",
            "date": "2020-09-25 09:38:59+00:00",
            "title": "Focus-Constrained Attention Mechanism for CVAE-based Response Generation",
            "authors": [
                "Zhi Cui",
                "Yanran Li",
                "Jiayi Zhang",
                "Jianwei Cui",
                "Chen Wei",
                "Bin Wang"
            ],
            "abstract": "To model diverse responses for a given post, one promising way is to\nintroduce a latent variable into Seq2Seq models. The latent variable is\nsupposed to capture the discourse-level information and encourage the\ninformativeness of target responses. However, such discourse-level information\nis often too coarse for the decoder to be utilized. To tackle it, our idea is\nto transform the coarse-grained discourse-level information into fine-grained\nword-level information. Specifically, we firstly measure the semantic\nconcentration of corresponding target response on the post words by introducing\na fine-grained focus signal. Then, we propose a focus-constrained attention\nmechanism to take full advantage of focus in well aligning the input to the\ntarget response. The experimental results demonstrate that by exploiting the\nfine-grained signal, our model can generate more diverse and informative\nresponses compared with several state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12269v1",
            "date": "2020-09-25 14:36:41+00:00",
            "title": "PerKey: A Persian News Corpus for Keyphrase Extraction and Generation",
            "authors": [
                "Ehsan Doostmohammadi",
                "Mohammad Hadi Bokaei",
                "Hossein Sameti"
            ],
            "abstract": "Keyphrases provide an extremely dense summary of a text. Such information can\nbe used in many Natural Language Processing tasks, such as information\nretrieval and text summarization. Since previous studies on Persian keyword or\nkeyphrase extraction have not published their data, the field suffers from the\nlack of a human extracted keyphrase dataset. In this paper, we introduce\nPerKey, a corpus of 553k news articles from six Persian news websites and\nagencies with relatively high quality author extracted keyphrases, which is\nthen filtered and cleaned to achieve higher quality keyphrases. The resulted\ndata was put into human assessment to ensure the quality of the keyphrases. We\nalso measured the performance of different supervised and unsupervised\ntechniques, e.g. TFIDF, MultipartiteRank, KEA, etc. on the dataset using\nprecision, recall, and F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12271v1",
            "date": "2020-09-25 14:40:14+00:00",
            "title": "Persian Keyphrase Generation Using Sequence-to-Sequence Models",
            "authors": [
                "Ehsan Doostmohammadi",
                "Mohammad Hadi Bokaei",
                "Hossein Sameti"
            ],
            "abstract": "Keyphrases are a very short summary of an input text and provide the main\nsubjects discussed in the text. Keyphrase extraction is a useful upstream task\nand can be used in various natural language processing problems, for example,\ntext summarization and information retrieval, to name a few. However, not all\nthe keyphrases are explicitly mentioned in the body of the text. In real-world\nexamples there are always some topics that are discussed implicitly. Extracting\nsuch keyphrases requires a generative approach, which is adopted here. In this\npaper, we try to tackle the problem of keyphrase generation and extraction from\nnews articles using deep sequence-to-sequence models. These models\nsignificantly outperform the conventional methods such as Topic Rank, KPMiner,\nand KEA in the task of keyphrase extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12319v2",
            "date": "2020-09-25 16:18:52+00:00",
            "title": "Empirical Study of Text Augmentation on Social Media Text in Vietnamese",
            "authors": [
                "Son T. Luu",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In the text classification problem, the imbalance of labels in datasets\naffect the performance of the text-classification models. Practically, the data\nabout user comments on social networking sites not altogether appeared - the\nadministrators often only allow positive comments and hide negative comments.\nThus, when collecting the data about user comments on the social network, the\ndata is usually skewed about one label, which leads the dataset to become\nimbalanced and deteriorate the model's ability. The data augmentation\ntechniques are applied to solve the imbalance problem between classes of the\ndataset, increasing the prediction model's accuracy. In this paper, we\nperformed augmentation techniques on the VLSP2019 Hate Speech Detection on\nVietnamese social texts and the UIT - VSFC: Vietnamese Students' Feedback\nCorpus for Sentiment Analysis. The result of augmentation increases by about\n1.5% in the F1-macro score on both corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12341v1",
            "date": "2020-09-25 17:01:19+00:00",
            "title": "Developing FB Chatbot Based on Deep Learning Using RASA Framework for University Enquiries",
            "authors": [
                "Yurio Windiatmoko",
                "Ahmad Fathan Hidayatullah",
                "Ridho Rahmadi"
            ],
            "abstract": "Smart systems for Universities powered by Artificial Intelligence have been\nmassively developed to help humans in various tasks. The chatbot concept is not\nsomething new in today society which is developing with recent technology.\nCollege students or candidates of college students often need actual\ninformation like asking for something to customer service, especially during\nthis pandemic, when it is difficult to have an immediate face-to-face meeting.\nChatbots are functionally helping in several things such as curriculum\ninformation, admission for new students, schedule info for any lecture courses,\nstudents grade information, and some adding features for Muslim worships\nschedule, also weather forecast information. This Chatbot is developed by Deep\nLearning models, which was adopted by an artificial intelligence model that\nreplicates human intelligence with some specific training schemes. This kind of\nDeep Learning is based on RNN which has some specific memory savings scheme for\nthe Deep Learning Model, specifically this chatbot using LSTM which already\nintegrates by RASA framework. LSTM is also known as Long Short Term Memory\nwhich efficiently saves some required memory but will remove some memory that\nis not needed. This Chatbot uses the FB platform because of the FB users have\nalready reached up to 60.8% of its entire population in Indonesia. Here's the\nchatbot only focuses on case studies at campus of the Magister Informatics FTI\nUniversity of Islamic Indonesia. This research is a first stage development\nwithin fairly sufficient simulate data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12344v2",
            "date": "2020-09-25 17:04:17+00:00",
            "title": "A little goes a long way: Improving toxic language classification despite data scarcity",
            "authors": [
                "Mika Juuti",
                "Tommi Gr\u00f6ndahl",
                "Adrian Flanagan",
                "N. Asokan"
            ],
            "abstract": "Detection of some types of toxic language is hampered by extreme scarcity of\nlabeled training data. Data augmentation - generating new synthetic data from a\nlabeled seed dataset - can help. The efficacy of data augmentation on toxic\nlanguage classification has not been fully explored. We present the first\nsystematic study on how data augmentation techniques impact performance across\ntoxic language classifiers, ranging from shallow logistic regression\narchitectures to BERT - a state-of-the-art pre-trained Transformer network. We\ncompare the performance of eight techniques on very scarce seed datasets. We\nshow that while BERT performed the best, shallow classifiers performed\ncomparably when trained on data augmented with a combination of three\ntechniques, including GPT-2-generated sentences. We discuss the interplay of\nperformance and computational overhead, which can inform the choice of\ntechniques under different constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12431v1",
            "date": "2020-09-25 20:49:07+00:00",
            "title": "XTE: Explainable Text Entailment",
            "authors": [
                "Vivian S. Silva",
                "Andr\u00e9 Freitas",
                "Siegfried Handschuh"
            ],
            "abstract": "Text entailment, the task of determining whether a piece of text logically\nfollows from another piece of text, is a key component in NLP, providing input\nfor many semantic applications such as question answering, text summarization,\ninformation extraction, and machine translation, among others. Entailment\nscenarios can range from a simple syntactic variation to more complex semantic\nrelationships between pieces of text, but most approaches try a\none-size-fits-all solution that usually favors some scenario to the detriment\nof another. Furthermore, for entailments requiring world knowledge, most\nsystems still work as a \"black box\", providing a yes/no answer that does not\nexplain the underlying reasoning process. In this work, we introduce XTE -\nExplainable Text Entailment - a novel composite approach for recognizing text\nentailment which analyzes the entailment pair to decide whether it must be\nresolved syntactically or semantically. Also, if a semantic matching is\ninvolved, we make the answer interpretable, using external knowledge bases\ncomposed of structured lexical definitions to generate natural language\njustifications that explain the semantic relationship holding between the\npieces of text. Besides outperforming well-established entailment algorithms,\nour composite approach gives an important step towards Explainable AI, allowing\nthe inference model interpretation, making the semantic reasoning process\nexplicit and understandable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12452v1",
            "date": "2020-09-25 22:06:06+00:00",
            "title": "BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context",
            "authors": [
                "Jean-Philippe Corbeil",
                "Hadi Abdi Ghadivel"
            ],
            "abstract": "Newly-introduced deep learning architectures, namely BERT, XLNet, RoBERTa and\nALBERT, have been proved to be robust on several NLP tasks. However, the\ndatasets trained on these architectures are fixed in terms of size and\ngeneralizability. To relieve this issue, we apply one of the most inexpensive\nsolutions to update these datasets. We call this approach BET by which we\nanalyze the backtranslation data augmentation on the transformer-based\narchitectures. Using the Google Translate API with ten intermediary languages\nfrom ten different language families, we externally evaluate the results in the\ncontext of automatic paraphrase identification in a transformer-based\nframework. Our findings suggest that BET improves the paraphrase identification\nperformance on the Microsoft Research Paraphrase Corpus (MRPC) to more than 3%\non both accuracy and F1 score. We also analyze the augmentation in the low-data\nregime with downsampled versions of MRPC, Twitter Paraphrase Corpus (TPC) and\nQuora Question Pairs. In many low-data cases, we observe a switch from a\nfailing model on the test set to reasonable performances. The results\ndemonstrate that BET is a highly promising data augmentation technique: to push\nthe current state-of-the-art of existing datasets and to bootstrap the\nutilization of deep learning architectures in the low-data regime of a hundred\nsamples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12506v2",
            "date": "2020-09-26 02:31:42+00:00",
            "title": "Learning to Plan and Realize Separately for Open-Ended Dialogue Systems",
            "authors": [
                "Sashank Santhanam",
                "Zhuo Cheng",
                "Brodie Mather",
                "Bonnie Dorr",
                "Archna Bhatia",
                "Bryanna Hebenstreit",
                "Alan Zemel",
                "Adam Dalton",
                "Tomek Strzalkowski",
                "Samira Shaikh"
            ],
            "abstract": "Achieving true human-like ability to conduct a conversation remains an\nelusive goal for open-ended dialogue systems. We posit this is because extant\napproaches towards natural language generation (NLG) are typically construed as\nend-to-end architectures that do not adequately model human generation\nprocesses. To investigate, we decouple generation into two separate phases:\nplanning and realization. In the planning phase, we train two planners to\ngenerate plans for response utterances. The realization phase uses response\nplans to produce an appropriate response. Through rigorous evaluations, both\nautomated and human, we demonstrate that decoupling the process into planning\nand realization performs better than an end-to-end approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12534v2",
            "date": "2020-09-26 08:21:32+00:00",
            "title": "iNLTK: Natural Language Toolkit for Indic Languages",
            "authors": [
                "Gaurav Arora"
            ],
            "abstract": "We present iNLTK, an open-source NLP library consisting of pre-trained\nlanguage models and out-of-the-box support for Data Augmentation, Textual\nSimilarity, Sentence Embeddings, Word Embeddings, Tokenization and Text\nGeneration in 13 Indic Languages. By using pre-trained models from iNLTK for\ntext classification on publicly available datasets, we significantly outperform\npreviously reported results. On these datasets, we also show that by using\npre-trained models and data augmentation from iNLTK, we can achieve more than\n95% of the previous best performance by using less than 10% of the training\ndata. iNLTK is already being widely used by the community and has 40,000+\ndownloads, 600+ stars and 100+ forks on GitHub. The library is available at\nhttps://github.com/goru001/inltk.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12539v2",
            "date": "2020-09-26 08:43:06+00:00",
            "title": "Topic-Aware Multi-turn Dialogue Modeling",
            "authors": [
                "Yi Xu",
                "Hai Zhao",
                "Zhuosheng Zhang"
            ],
            "abstract": "In the retrieval-based multi-turn dialogue modeling, it remains a challenge\nto select the most appropriate response according to extracting salient\nfeatures in context utterances. As a conversation goes on, topic shift at\ndiscourse-level naturally happens through the continuous multi-turn dialogue\ncontext. However, all known retrieval-based systems are satisfied with\nexploiting local topic words for context utterance representation but fail to\ncapture such essential global topic-aware clues at discourse-level. Instead of\ntaking topic-agnostic n-gram utterance as processing unit for matching purpose\nin existing systems, this paper presents a novel topic-aware solution for\nmulti-turn dialogue modeling, which segments and extracts topic-aware\nutterances in an unsupervised way, so that the resulted model is capable of\ncapturing salient topic shift at discourse-level in need and thus effectively\ntrack topic flow during multi-turn conversation. Our topic-aware modeling is\nimplemented by a newly proposed unsupervised topic-aware segmentation algorithm\nand Topic-Aware Dual-attention Matching (TADAM) Network, which matches each\ntopic segment with the response in a dual cross-attention way. Experimental\nresults on three public datasets show TADAM can outperform the state-of-the-art\nmethod, especially by 3.3% on E-commerce dataset that has an obvious topic\nshift.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12615v1",
            "date": "2020-09-26 14:56:57+00:00",
            "title": "ARPA: Armenian Paraphrase Detection Corpus and Models",
            "authors": [
                "Arthur Malajyan",
                "Karen Avetisyan",
                "Tsolak Ghukasyan"
            ],
            "abstract": "In this work, we employ a semi-automatic method based on back translation to\ngenerate a sentential paraphrase corpus for the Armenian language. The initial\ncollection of sentences is translated from Armenian to English and back twice,\nresulting in pairs of lexically distant but semantically similar sentences. The\ngenerated paraphrases are then manually reviewed and annotated. Using the\nmethod train and test datasets are created, containing 2360 paraphrases in\ntotal. In addition, the datasets are used to train and evaluate BERTbased\nmodels for detecting paraphrase in Armenian, achieving results comparable to\nthe state-of-the-art of other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12626v2",
            "date": "2020-09-26 15:53:22+00:00",
            "title": "DWIE: an entity-centric dataset for multi-task document-level information extraction",
            "authors": [
                "Klim Zaporojets",
                "Johannes Deleu",
                "Chris Develder",
                "Thomas Demeester"
            ],
            "abstract": "This paper presents DWIE, the 'Deutsche Welle corpus for Information\nExtraction', a newly created multi-task dataset that combines four main\nInformation Extraction (IE) annotation subtasks: (i) Named Entity Recognition\n(NER), (ii) Coreference Resolution, (iii) Relation Extraction (RE), and (iv)\nEntity Linking. DWIE is conceived as an entity-centric dataset that describes\ninteractions and properties of conceptual entities on the level of the complete\ndocument. This contrasts with currently dominant mention-driven approaches that\nstart from the detection and classification of named entity mentions in\nindividual sentences. Further, DWIE presented two main challenges when building\nand evaluating IE models for it. First, the use of traditional mention-level\nevaluation metrics for NER and RE tasks on entity-centric DWIE dataset can\nresult in measurements dominated by predictions on more frequently mentioned\nentities. We tackle this issue by proposing a new entity-driven metric that\ntakes into account the number of mentions that compose each of the predicted\nand ground truth entities. Second, the document-level multi-task annotations\nrequire the models to transfer information between entity mentions located in\ndifferent parts of the document, as well as between different tasks, in a joint\nlearning setting. To realize this, we propose to use graph-based neural message\npassing techniques between document-level mention spans. Our experiments show\nan improvement of up to 5.5 F1 percentage points when incorporating neural\ngraph propagation into our joint model. This demonstrates DWIE's potential to\nstimulate further research in graph neural networks for representation learning\nin multi-task IE. We make DWIE publicly available at\nhttps://github.com/klimzaporojets/DWIE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12643v2",
            "date": "2020-09-26 17:06:29+00:00",
            "title": "Recurrent Inference in Text Editing",
            "authors": [
                "Ning Shi",
                "Ziheng Zeng",
                "Haotian Zhang",
                "Yichen Gong"
            ],
            "abstract": "In neural text editing, prevalent sequence-to-sequence based approaches\ndirectly map the unedited text either to the edited text or the editing\noperations, in which the performance is degraded by the limited source text\nencoding and long, varying decoding steps. To address this problem, we propose\na new inference method, Recurrence, that iteratively performs editing actions,\nsignificantly narrowing the problem space. In each iteration, encoding the\npartially edited text, Recurrence decodes the latent representation, generates\nan action of short, fixed-length, and applies the action to complete a single\nedit. For a comprehensive comparison, we introduce three types of text editing\ntasks: Arithmetic Operators Restoration (AOR), Arithmetic Equation\nSimplification (AES), Arithmetic Equation Correction (AEC). Extensive\nexperiments on these tasks with varying difficulties demonstrate that\nRecurrence achieves improvements over conventional inference methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12695v1",
            "date": "2020-09-26 21:56:22+00:00",
            "title": "Techniques to Improve Q&A Accuracy with Transformer-based models on Large Complex Documents",
            "authors": [
                "Chejui Liao",
                "Tabish Maniar",
                "Sravanajyothi N",
                "Anantha Sharma"
            ],
            "abstract": "This paper discusses the effectiveness of various text processing techniques,\ntheir combinations, and encodings to achieve a reduction of complexity and size\nin a given text corpus. The simplified text corpus is sent to BERT (or similar\ntransformer based models) for question and answering and can produce more\nrelevant responses to user queries. This paper takes a scientific approach to\ndetermine the benefits and effectiveness of various techniques and concludes a\nbest-fit combination that produces a statistically significant improvement in\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12719v2",
            "date": "2020-09-27 01:04:06+00:00",
            "title": "Stylized Dialogue Response Generation Using Stylized Unpaired Texts",
            "authors": [
                "Yinhe Zheng",
                "Zikai Chen",
                "Rongsheng Zhang",
                "Shilei Huang",
                "Xiaoxi Mao",
                "Minlie Huang"
            ],
            "abstract": "Generating stylized responses is essential to build intelligent and engaging\ndialogue systems. However, this task is far from well-explored due to the\ndifficulties of rendering a particular style in coherent responses, especially\nwhen the target style is embedded only in unpaired texts that cannot be\ndirectly used to train the dialogue model. This paper proposes a stylized\ndialogue generation method that can capture stylistic features embedded in\nunpaired texts. Specifically, our method can produce dialogue responses that\nare both coherent to the given context and conform to the target style. In this\nstudy, an inverse dialogue model is first introduced to predict possible posts\nfor the input responses, and then this inverse model is used to generate\nstylized pseudo dialogue pairs based on these stylized unpaired texts. Further,\nthese pseudo pairs are employed to train the stylized dialogue model with a\njoint training process, and a style routing approach is proposed to intensify\nstylistic features in the decoder. Automatic and manual evaluations on two\ndatasets demonstrate that our method outperforms competitive baselines in\nproducing coherent and style-intensive dialogue responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12721v1",
            "date": "2020-09-27 01:40:01+00:00",
            "title": "A Brief Survey and Comparative Study of Recent Development of Pronoun Coreference Resolution",
            "authors": [
                "Hongming Zhang",
                "Xinran Zhao",
                "Yangqiu Song"
            ],
            "abstract": "Pronoun Coreference Resolution (PCR) is the task of resolving pronominal\nexpressions to all mentions they refer to. Compared with the general\ncoreference resolution task, the main challenge of PCR is the coreference\nrelation prediction rather than the mention detection. As one important natural\nlanguage understanding (NLU) component, pronoun resolution is crucial for many\ndownstream tasks and still challenging for existing models, which motivates us\nto survey existing approaches and think about how to do better. In this survey,\nwe first introduce representative datasets and models for the ordinary pronoun\ncoreference resolution task. Then we focus on recent progress on hard pronoun\ncoreference resolution problems (e.g., Winograd Schema Challenge) to analyze\nhow well current models can understand commonsense. We conduct extensive\nexperiments to show that even though current models are achieving good\nperformance on the standard evaluation set, they are still not ready to be used\nin real applications (e.g., all SOTA models struggle on correctly resolving\npronouns to infrequent objects). All experiment codes are available at\nhttps://github.com/HKUST-KnowComp/PCR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12756v2",
            "date": "2020-09-27 06:12:29+00:00",
            "title": "Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval",
            "authors": [
                "Wenhan Xiong",
                "Xiang Lorraine Li",
                "Srini Iyer",
                "Jingfei Du",
                "Patrick Lewis",
                "William Yang Wang",
                "Yashar Mehdad",
                "Wen-tau Yih",
                "Sebastian Riedel",
                "Douwe Kiela",
                "Barlas O\u011fuz"
            ],
            "abstract": "We propose a simple and efficient multi-hop dense retrieval approach for\nanswering complex open-domain questions, which achieves state-of-the-art\nperformance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.\nContrary to previous work, our method does not require access to any\ncorpus-specific information, such as inter-document hyperlinks or\nhuman-annotated entity markers, and can be applied to any unstructured text\ncorpus. Our system also yields a much better efficiency-accuracy trade-off,\nmatching the best published accuracy on HotpotQA while being 10 times faster at\ninference time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12862v1",
            "date": "2020-09-27 15:00:52+00:00",
            "title": "What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties",
            "authors": [
                "Rochelle Choenni",
                "Ekaterina Shutova"
            ],
            "abstract": "Multilingual sentence encoders have seen much success in cross-lingual model\ntransfer for downstream NLP tasks. Yet, we know relatively little about the\nproperties of individual languages or the general patterns of linguistic\nvariation that they encode. We propose methods for probing sentence\nrepresentations from state-of-the-art multilingual encoders (LASER, M-BERT, XLM\nand XLM-R) with respect to a range of typological properties pertaining to\nlexical, morphological and syntactic structure. In addition, we investigate how\nthis information is distributed across all layers of the models. Our results\nshow interesting differences in encoding linguistic variation associated with\ndifferent pretraining strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12952v1",
            "date": "2020-09-27 21:07:51+00:00",
            "title": "Unsupervised Pre-training for Biomedical Question Answering",
            "authors": [
                "Vaishnavi Kommaraju",
                "Karthick Gunasekaran",
                "Kun Li",
                "Trapit Bansal",
                "Andrew McCallum",
                "Ivana Williams",
                "Ana-Maria Istrate"
            ],
            "abstract": "We explore the suitability of unsupervised representation learning methods on\nbiomedical text -- BioBERT, SciBERT, and BioSentVec -- for biomedical question\nanswering. To further improve unsupervised representations for biomedical QA,\nwe introduce a new pre-training task from unlabeled data designed to reason\nabout biomedical entities in the context. Our pre-training method consists of\ncorrupting a given context by randomly replacing some mention of a biomedical\nentity with a random entity mention and then querying the model with the\ncorrect entity mention in order to locate the corrupted part of the context.\nThis de-noising task enables the model to learn good representations from\nabundant, unlabeled biomedical text that helps QA tasks and minimizes the\ntrain-test mismatch between the pre-training task and the downstream QA tasks\nby requiring the model to predict spans. Our experiments show that pre-training\nBioBERT on the proposed pre-training task significantly boosts performance and\noutperforms the previous best model from the 7th BioASQ Task 7b-Phase B\nchallenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13060v2",
            "date": "2020-09-28 04:28:48+00:00",
            "title": "A Simple and Efficient Ensemble Classifier Combining Multiple Neural Network Models on Social Media Datasets in Vietnamese",
            "authors": [
                "Huy Duc Huynh",
                "Hang Thi-Thuy Do",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Text classification is a popular topic of natural language processing, which\nhas currently attracted numerous research efforts worldwide. The significant\nincrease of data in social media requires the vast attention of researchers to\nanalyze such data. There are various studies in this field in many languages\nbut limited to the Vietnamese language. Therefore, this study aims to classify\nVietnamese texts on social media from three different Vietnamese benchmark\ndatasets. Advanced deep learning models are used and optimized in this study,\nincluding CNN, LSTM, and their variants. We also implement the BERT, which has\nnever been applied to the datasets. Our experiments find a suitable model for\nclassification tasks on each specific dataset. To take advantage of single\nmodels, we propose an ensemble model, combining the highest-performance models.\nOur single models reach positive results on each dataset. Moreover, our\nensemble model achieves the best performance on all three datasets. We reach\n86.96% of F1- score for the HSD-VLSP dataset, 65.79% of F1-score for the\nUIT-VSMEC dataset, 92.79% and 89.70% for sentiments and topics on the UIT-VSFC\ndataset, respectively. Therefore, our models achieve better performances as\ncompared to previous studies on these datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13282v1",
            "date": "2020-09-28 12:47:59+00:00",
            "title": "Graph-based Multi-hop Reasoning for Long Text Generation",
            "authors": [
                "Liang Zhao",
                "Jingjing Xu",
                "Junyang Lin",
                "Yichang Zhang",
                "Hongxia Yang",
                "Xu Sun"
            ],
            "abstract": "Long text generation is an important but challenging task.The main problem\nlies in learning sentence-level semantic dependencies which traditional\ngenerative models often suffer from. To address this problem, we propose a\nMulti-hop Reasoning Generation (MRG) approach that incorporates multi-hop\nreasoning over a knowledge graph to learn semantic dependencies among\nsentences. MRG consists of twoparts, a graph-based multi-hop reasoning module\nand a path-aware sentence realization module. The reasoning module is\nresponsible for searching skeleton paths from a knowledge graph to imitate the\nimagination process in the human writing for semantic transfer. Based on the\ninferred paths, the sentence realization module then generates a complete\nsentence. Unlike previous black-box models, MRG explicitly infers the skeleton\npath, which provides explanatory views tounderstand how the proposed model\nworks. We conduct experiments on three representative tasks, including story\ngeneration, review generation, and product description generation. Automatic\nand manual evaluation show that our proposed method can generate more\ninformative and coherentlong text than strong baselines, such as pre-trained\nmodels(e.g. GPT-2) and knowledge-enhanced models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13312v1",
            "date": "2020-09-28 13:32:59+00:00",
            "title": "Reducing Quantity Hallucinations in Abstractive Summarization",
            "authors": [
                "Zheng Zhao",
                "Shay B. Cohen",
                "Bonnie Webber"
            ],
            "abstract": "It is well-known that abstractive summaries are subject to\nhallucination---including material that is not supported by the original text.\nWhile summaries can be made hallucination-free by limiting them to general\nphrases, such summaries would fail to be very informative. Alternatively, one\ncan try to avoid hallucinations by verifying that any specific entities in the\nsummary appear in the original text in a similar context. This is the approach\ntaken by our system, Herman. The system learns to recognize and verify quantity\nentities (dates, numbers, sums of money, etc.) in a beam-worth of abstractive\nsummaries produced by state-of-the-art models, in order to up-rank those\nsummaries whose quantity terms are supported by the original text. Experimental\nresults demonstrate that the ROUGE scores of such up-ranked summaries have a\nhigher Precision than summaries that have not been up-ranked, without a\ncomparable loss in Recall, resulting in higher F$_1$. Preliminary human\nevaluation of up-ranked vs. original summaries shows people's preference for\nthe former.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13398v1",
            "date": "2020-09-28 15:19:23+00:00",
            "title": "Aspects of Terminological and Named Entity Knowledge within Rule-Based Machine Translation Models for Under-Resourced Neural Machine Translation Scenarios",
            "authors": [
                "Daniel Torregrosa",
                "Nivranshu Pasricha",
                "Maraim Masoud",
                "Bharathi Raja Chakravarthi",
                "Juan Alonso",
                "Noe Casas",
                "Mihael Arcan"
            ],
            "abstract": "Rule-based machine translation is a machine translation paradigm where\nlinguistic knowledge is encoded by an expert in the form of rules that\ntranslate text from source to target language. While this approach grants\nextensive control over the output of the system, the cost of formalising the\nneeded linguistic knowledge is much higher than training a corpus-based system,\nwhere a machine learning approach is used to automatically learn to translate\nfrom examples. In this paper, we describe different approaches to leverage the\ninformation contained in rule-based machine translation systems to improve a\ncorpus-based one, namely, a neural machine translation model, with a focus on a\nlow-resource scenario. Three different kinds of information were used:\nmorphological information, named entities and terminology. In addition to\nevaluating the general performance of the system, we systematically analysed\nthe performance of the proposed approaches when dealing with the targeted\nphenomena. Our results suggest that the proposed models have limited ability to\nlearn from external information, and most approaches do not significantly alter\nthe results of the automatic evaluation, but our preliminary qualitative\nevaluation shows that in certain cases the hypothesis generated by our system\nexhibit favourable behaviour such as keeping the use of passive voice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13431v1",
            "date": "2020-09-28 15:59:31+00:00",
            "title": "PIN: A Novel Parallel Interactive Network for Spoken Language Understanding",
            "authors": [
                "Peilin Zhou",
                "Zhiqi Huang",
                "Fenglin Liu",
                "Yuexian Zou"
            ],
            "abstract": "Spoken Language Understanding (SLU) is an essential part of the spoken\ndialogue system, which typically consists of intent detection (ID) and slot\nfilling (SF) tasks. Recently, recurrent neural networks (RNNs) based methods\nachieved the state-of-the-art for SLU. It is noted that, in the existing\nRNN-based approaches, ID and SF tasks are often jointly modeled to utilize the\ncorrelation information between them. However, we noted that, so far, the\nefforts to obtain better performance by supporting bidirectional and explicit\ninformation exchange between ID and SF are not well studied.In addition, few\nstudies attempt to capture the local context information to enhance the\nperformance of SF. Motivated by these findings, in this paper, Parallel\nInteractive Network (PIN) is proposed to model the mutual guidance between ID\nand SF. Specifically, given an utterance, a Gaussian self-attentive encoder is\nintroduced to generate the context-aware feature embedding of the utterance\nwhich is able to capture local context information. Taking the feature\nembedding of the utterance, Slot2Intent module and Intent2Slot module are\ndeveloped to capture the bidirectional information flow for ID and SF tasks.\nFinally, a cooperation mechanism is constructed to fuse the information\nobtained from Slot2Intent and Intent2Slot modules to further reduce the\nprediction bias.The experiments on two benchmark datasets, i.e., SNIPS and\nATIS, demonstrate the effectiveness of our approach, which achieves a\ncompetitive result with state-of-the-art models. More encouragingly, by using\nthe feature embedding of the utterance generated by the pre-trained language\nmodel BERT, our method achieves the state-of-the-art among all comparison\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13658v1",
            "date": "2020-09-28 22:18:58+00:00",
            "title": "Improve Transformer Models with Better Relative Position Embeddings",
            "authors": [
                "Zhiheng Huang",
                "Davis Liang",
                "Peng Xu",
                "Bing Xiang"
            ],
            "abstract": "Transformer architectures rely on explicit position encodings in order to\npreserve a notion of word order. In this paper, we argue that existing work\ndoes not fully utilize position information. For example, the initial proposal\nof a sinusoid embedding is fixed and not learnable. In this paper, we first\nreview absolute position embeddings and existing methods for relative position\nembeddings. We then propose new techniques that encourage increased interaction\nbetween query, key and relative position embeddings in the self-attention\nmechanism. Our most promising approach is a generalization of the absolute\nposition embedding, improving results on SQuAD1.1 compared to previous position\nembeddings approaches. In addition, we address the inductive property of\nwhether a position embedding can be robust enough to handle long sequences. We\ndemonstrate empirically that our relative position embedding method is\nreasonably generalized and robust from the inductive perspective. Finally, we\nshow that our proposed method can be adopted as a near drop-in replacement for\nimproving the accuracy of large models with a small computational budget.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13699v2",
            "date": "2020-09-29 00:25:24+00:00",
            "title": "Leader: Prefixing a Length for Faster Word Vector Serialization",
            "authors": [
                "Brian Lester"
            ],
            "abstract": "Two competing file formats have become the de facto standards for\ndistributing pre-trained word embeddings. Both are named after the most popular\npre-trained embeddings that are distributed in that format. The GloVe format is\nan entirely text based format that suffers from huge file sizes and slow reads,\nand the word2vec format is a smaller binary format that mixes a textual\nrepresentation of words with a binary representation of the vectors themselves.\nBoth formats have problems that we solve with a new format we call the Leader\nformat. We include a word length prefix for faster reads while maintaining the\nsmaller file size a binary format offers. We also created a minimalist library\nto facilitate the reading and writing of various word vector formats, as well\nas tools for converting pre-trained embeddings to our new Leader format.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13815v1",
            "date": "2020-09-29 07:02:19+00:00",
            "title": "Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation",
            "authors": [
                "Yinfei Yang",
                "Ning Jin",
                "Kuo Lin",
                "Mandy Guo",
                "Daniel Cer"
            ],
            "abstract": "Neural models that independently project questions and answers into a shared\nembedding space allow for efficient continuous space retrieval from large\ncorpora. Independently computing embeddings for questions and answers results\nin late fusion of information related to matching questions to their answers.\nWhile critical for efficient retrieval, late fusion underperforms models that\nmake use of early fusion (e.g., a BERT based classifier with cross-attention\nbetween question-answer pairs). We present a supervised data mining method\nusing an accurate early fusion model to improve the training of an efficient\nlate fusion retrieval model. We first train an accurate classification model\nwith cross-attention between questions and answers. The accurate\ncross-attention model is then used to annotate additional passages in order to\ngenerate weighted training examples for a neural retrieval model. The resulting\nretrieval model with additional data significantly outperforms retrieval models\ndirectly trained with gold annotations on Precision at $N$ (P@N) and Mean\nReciprocal Rank (MRR).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13833v2",
            "date": "2020-09-29 07:44:37+00:00",
            "title": "HINT3: Raising the bar for Intent Detection in the Wild",
            "authors": [
                "Gaurav Arora",
                "Chirag Jain",
                "Manas Chaturvedi",
                "Krupal Modi"
            ],
            "abstract": "Intent Detection systems in the real world are exposed to complexities of\nimbalanced datasets containing varying perception of intent, unintended\ncorrelations and domain-specific aberrations. To facilitate benchmarking which\ncan reflect near real-world scenarios, we introduce 3 new datasets created from\nlive chatbots in diverse domains. Unlike most existing datasets that are\ncrowdsourced, our datasets contain real user queries received by the chatbots\nand facilitates penalising unwanted correlations grasped during the training\nprocess. We evaluate 4 NLU platforms and a BERT based classifier and find that\nperformance saturates at inadequate levels on test sets because all systems\nlatch on to unintended patterns in training data.",
            "journal ref": "Proceedings of the First Workshop on Insights from Negative\n  Results in NLP @ EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13859v1",
            "date": "2020-09-29 08:32:32+00:00",
            "title": "Fake News Spreader Detection on Twitter using Character N-Grams. Notebook for PAN at CLEF 2020",
            "authors": [
                "Inna Vogel",
                "Meghana Meghana"
            ],
            "abstract": "The authors of fake news often use facts from verified news sources and mix\nthem with misinformation to create confusion and provoke unrest among the\nreaders. The spread of fake news can thereby have serious implications on our\nsociety. They can sway political elections, push down the stock price or crush\nreputations of corporations or public figures. Several websites have taken on\nthe mission of checking rumors and allegations, but are often not fast enough\nto check the content of all the news being disseminated. Especially social\nmedia websites have offered an easy platform for the fast propagation of\ninformation. Towards limiting fake news from being propagated among social\nmedia users, the task of this year's PAN 2020 challenge lays the focus on the\nfake news spreaders. The aim of the task is to determine whether it is possible\nto discriminate authors that have shared fake news in the past from those that\nhave never done it. In this notebook, we describe our profiling system for the\nfake news detection task on Twitter. For this, we conduct different feature\nextraction techniques and learning experiments from a multilingual perspective,\nnamely English and Spanish. Our final submitted systems use character n-grams\nas features in combination with a linear SVM for English and Logistic\nRegression for the Spanish language. Our submitted models achieve an overall\naccuracy of 73% and 79% on the English and Spanish official test set,\nrespectively. Our experiments show that it is difficult to differentiate\nsolidly fake news spreaders on Twitter from users who share credible\ninformation leaving room for further investigations. Our model ranked 3rd out\nof 72 competitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13889v1",
            "date": "2020-09-29 09:25:54+00:00",
            "title": "Sequence-to-Sequence Learning for Indonesian Automatic Question Generator",
            "authors": [
                "Ferdiant Joshua Muis",
                "Ayu Purwarianti"
            ],
            "abstract": "Automatic question generation is defined as the task of automating the\ncreation of question given a various of textual data. Research in automatic\nquestion generator (AQG) has been conducted for more than 10 years, mainly\nfocused on factoid question. In all these studies, the state-of-the-art is\nattained using sequence-to-sequence approach. However, AQG system for\nIndonesian has not ever been researched intensely. In this work we construct an\nIndonesian automatic question generator, adapting the architecture from some\nprevious works. In summary, we used sequence-to-sequence approach using BiGRU,\nBiLSTM, and Transformer with additional linguistic features, copy mechanism,\nand coverage mechanism. Since there is no public large dan popular Indonesian\ndataset for question generation, we translated SQuAD v2.0 factoid question\nanswering dataset, with additional Indonesian TyDiQA dev set for testing. The\nsystem achieved BLEU1, BLEU2, BLEU3, BLEU4, and ROUGE-L score at 38,35, 20,96,\n10,68, 5,78, and 43,4 for SQuAD, and 39.9, 20.78, 10.26, 6.31, 44.13 for\nTyDiQA, respectively. The system performed well when the expected answers are\nnamed entities and are syntactically close with the context explaining them.\nAdditionally, from native Indonesian perspective, the best questions generated\nby our best models on their best cases are acceptable and reasonably useful.",
            "journal ref": "2020 International Conference on Advanced Informatics: Concept,\n  Theory and Application (ICAICTA)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13902v5",
            "date": "2020-09-29 09:50:21+00:00",
            "title": "Utterance-level Dialogue Understanding: An Empirical Study",
            "authors": [
                "Deepanway Ghosal",
                "Navonil Majumder",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "The recent abundance of conversational data on the Web and elsewhere calls\nfor effective NLP systems for dialog understanding. Complete utterance-level\nunderstanding often requires context understanding, defined by nearby\nutterances. In recent years, a number of approaches have been proposed for\nvarious utterance-level dialogue understanding tasks. Most of these approaches\naccount for the context for effective understanding. In this paper, we explore\nand quantify the role of context for different aspects of a dialogue, namely\nemotion, intent, and dialogue act identification, using state-of-the-art dialog\nunderstanding methods as baselines. Specifically, we employ various\nperturbations to distort the context of a given utterance and study its impact\non the different tasks and baselines. This provides us with insights into the\nfundamental contextual controlling factors of different aspects of a dialogue.\nSuch insights can inspire more effective dialogue understanding models, and\nprovide support for future text generation approaches. The implementation\npertaining to this work is available at\nhttps://github.com/declare-lab/dialogue-understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13905v1",
            "date": "2020-09-29 09:55:04+00:00",
            "title": "Aligning Intraobserver Agreement by Transitivity",
            "authors": [
                "Jacopo Amidei"
            ],
            "abstract": "Annotation reproducibility and accuracy rely on good consistency within\nannotators. We propose a novel method for measuring within annotator\nconsistency or annotator Intraobserver Agreement (IA). The proposed approach is\nbased on transitivity, a measure that has been thoroughly studied in the\ncontext of rational decision-making. The transitivity measure, in contrast with\nthe commonly used test-retest strategy for annotator IA, is less sensitive to\nthe several types of bias introduced by the test-retest strategy. We present a\nrepresentation theorem to the effect that relative judgement data that meet\ntransitivity can be mapped to a scale (in terms of measurement theory). We also\ndiscuss a further application of transitivity as part of data collection design\nfor addressing the problem of the quadratic complexity of data collection of\nrelative judgements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13971v1",
            "date": "2020-09-29 12:41:27+00:00",
            "title": "Neural Topic Modeling with Cycle-Consistent Adversarial Training",
            "authors": [
                "Xuemeng Hu",
                "Rui Wang",
                "Deyu Zhou",
                "Yuxuan Xiong"
            ],
            "abstract": "Advances on deep generative models have attracted significant research\ninterest in neural topic modeling. The recently proposed Adversarial-neural\nTopic Model models topics with an adversarially trained generator network and\nemploys Dirichlet prior to capture the semantic patterns in latent topics. It\nis effective in discovering coherent topics but unable to infer topic\ndistributions for given documents or utilize available document labels. To\novercome such limitations, we propose Topic Modeling with Cycle-consistent\nAdversarial Training (ToMCAT) and its supervised version sToMCAT. ToMCAT\nemploys a generator network to interpret topics and an encoder network to infer\ndocument topics. Adversarial training and cycle-consistent constraints are used\nto encourage the generator and the encoder to produce realistic samples that\ncoordinate with each other. sToMCAT extends ToMCAT by incorporating document\nlabels into the topic modeling process to help discover more coherent topics.\nThe effectiveness of the proposed models is evaluated on\nunsupervised/supervised topic modeling and text classification. The\nexperimental results show that our models can produce both coherent and\ninformative topics, outperforming a number of competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13972v1",
            "date": "2020-09-29 12:45:55+00:00",
            "title": "Neural Topic Modeling by Incorporating Document Relationship Graph",
            "authors": [
                "Deyu Zhou",
                "Xuemeng Hu",
                "Rui Wang"
            ],
            "abstract": "Graph Neural Networks (GNNs) that capture the relationships between graph\nnodes via message passing have been a hot research direction in the natural\nlanguage processing community. In this paper, we propose Graph Topic Model\n(GTM), a GNN based neural topic model that represents a corpus as a document\nrelationship graph. Documents and words in the corpus become nodes in the graph\nand are connected based on document-word co-occurrences. By introducing the\ngraph structure, the relationships between documents are established through\ntheir shared words and thus the topical representation of a document is\nenriched by aggregating information from its neighboring nodes using graph\nconvolution. Extensive experiments on three datasets were conducted and the\nresults demonstrate the effectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14083v1",
            "date": "2020-09-29 15:10:59+00:00",
            "title": "Building Legal Case Retrieval Systems with Lexical Matching and Summarization using A Pre-Trained Phrase Scoring Model",
            "authors": [
                "Vu Tran",
                "Minh Le Nguyen",
                "Ken Satoh"
            ],
            "abstract": "We present our method for tackling the legal case retrieval task of the\nCompetition on Legal Information Extraction/Entailment 2019. Our approach is\nbased on the idea that summarization is important for retrieval. On one hand,\nwe adopt a summarization based model called encoded summarization which encodes\na given document into continuous vector space which embeds the summary\nproperties of the document. We utilize the resource of COLIEE 2018 on which we\ntrain the document representation model. On the other hand, we extract lexical\nfeatures on different parts of a given query and its candidates. We observe\nthat by comparing different parts of the query and its candidates, we can\nachieve better performance. Furthermore, the combination of the lexical\nfeatures with latent features by the summarization-based method achieves even\nbetter performance. We have achieved the state-of-the-art result for the task\non the benchmark of the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14109v2",
            "date": "2020-09-29 15:48:58+00:00",
            "title": "Improving Low Compute Language Modeling with In-Domain Embedding Initialisation",
            "authors": [
                "Charles Welch",
                "Rada Mihalcea",
                "Jonathan K. Kummerfeld"
            ],
            "abstract": "Many NLP applications, such as biomedical data and technical support, have\n10-100 million tokens of in-domain data and limited computational resources for\nlearning from it. How should we train a language model in this scenario? Most\nlanguage modeling research considers either a small dataset with a closed\nvocabulary (like the standard 1 million token Penn Treebank), or the whole web\nwith byte-pair encoding. We show that for our target setting in English,\ninitialising and freezing input embeddings using in-domain data can improve\nlanguage model performance by providing a useful representation of rare words,\nand this pattern holds across several different domains. In the process, we\nshow that the standard convention of tying input and output embeddings does not\nimprove perplexity when initializing with embeddings trained on in-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14124v2",
            "date": "2020-09-29 16:12:52+00:00",
            "title": "Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank",
            "authors": [
                "Ethan C. Chau",
                "Lucy H. Lin",
                "Noah A. Smith"
            ],
            "abstract": "Pretrained multilingual contextual representations have shown great success,\nbut due to the limits of their pretraining data, their benefits do not apply\nequally to all language varieties. This presents a challenge for language\nvarieties unfamiliar to these models, whose labeled \\emph{and unlabeled} data\nis too limited to train a monolingual model effectively. We propose the use of\nadditional language-specific pretraining and vocabulary augmentation to adapt\nmultilingual models to low-resource settings. Using dependency parsing of four\ndiverse low-resource language varieties as a case study, we show that these\nmethods significantly improve performance over baselines, especially in the\nlowest-resource cases, and demonstrate the importance of the relationship\nbetween such models' pretraining data and target language varieties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14304v1",
            "date": "2020-09-29 20:56:57+00:00",
            "title": "Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study",
            "authors": [
                "Saurabh Kulshreshtha",
                "Jos\u00e9 Luis Redondo-Garc\u00eda",
                "Ching-Yun Chang"
            ],
            "abstract": "Multilingual BERT (mBERT) has shown reasonable capability for zero-shot\ncross-lingual transfer when fine-tuned on downstream tasks. Since mBERT is not\npre-trained with explicit cross-lingual supervision, transfer performance can\nfurther be improved by aligning mBERT with cross-lingual signal. Prior work\nproposes several approaches to align contextualised embeddings. In this paper\nwe analyse how different forms of cross-lingual supervision and various\nalignment methods influence the transfer capability of mBERT in zero-shot\nsetting. Specifically, we compare parallel corpora vs. dictionary-based\nsupervision and rotational vs. fine-tuning based alignment methods. We evaluate\nthe performance of different alignment methodologies across eight languages on\ntwo tasks: Name Entity Recognition and Semantic Slot Filling. In addition, we\npropose a novel normalisation method which consistently improves the\nperformance of rotation-based alignment including a notable 3% F1 improvement\nfor distant and typologically dissimilar languages. Importantly we identify the\nbiases of the alignment methods to the type of task and proximity to the\ntransfer language. We also find that supervision from parallel corpus is\ngenerally superior to dictionary alignments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14306v2",
            "date": "2020-09-29 21:03:44+00:00",
            "title": "INSPIRED: Toward Sociable Recommendation Dialog Systems",
            "authors": [
                "Shirley Anugrah Hayati",
                "Dongyeop Kang",
                "Qingxiaoyang Zhu",
                "Weiyan Shi",
                "Zhou Yu"
            ],
            "abstract": "In recommendation dialogs, humans commonly disclose their preference and make\nrecommendations in a friendly manner. However, this is a challenge when\ndeveloping a sociable recommendation dialog system, due to the lack of dialog\ndataset annotated with such sociable strategies. Therefore, we present\nINSPIRED, a new dataset of 1,001 human-human dialogs for movie recommendation\nwith measures for successful recommendations. To better understand how humans\nmake recommendations in communication, we design an annotation scheme related\nto recommendation strategies based on social science theories and annotate\nthese dialogs. Our analysis shows that sociable recommendation strategies, such\nas sharing personal opinions or communicating with encouragement, more\nfrequently lead to successful recommendations. Based on our dataset, we train\nend-to-end recommendation dialog systems with and without our strategy labels.\nIn both automatic and human evaluation, our model with strategy incorporation\noutperforms the baseline model. This work is a first step for building sociable\nrecommendation dialog systems with a basis of social science theories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14335v1",
            "date": "2020-09-29 22:49:15+00:00",
            "title": "Natcat: Weakly Supervised Text Classification with Naturally Annotated Datasets",
            "authors": [
                "Zewei Chu",
                "Karl Stratos",
                "Kevin Gimpel"
            ],
            "abstract": "We seek to improve text classification by leveraging naturally annotated\ndata. In particular, we construct a general purpose text categorization dataset\n(NatCat) from three online resources: Wikipedia, Reddit, and Stack Exchange.\nThese datasets consist of document-category pairs derived from manual curation\nthat occurs naturally by their communities. We build general purpose text\nclassifiers by training on NatCat and evaluate them on a suite of 11 text\nclassification tasks (CatEval). We benchmark different modeling choices and\ndataset combinations, and show how each task benefits from different NatCat\ntraining resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14348v1",
            "date": "2020-09-29 23:53:50+00:00",
            "title": "MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension",
            "authors": [
                "Huaishao Luo",
                "Yu Shi",
                "Ming Gong",
                "Linjun Shou",
                "Tianrui Li"
            ],
            "abstract": "Span extraction is an essential problem in machine reading comprehension.\nMost of the existing algorithms predict the start and end positions of an\nanswer span in the given corresponding context by generating two probability\nvectors. In this paper, we propose a novel approach that extends the\nprobability vector to a probability matrix. Such a matrix can cover more\nstart-end position pairs. Precisely, to each possible start index, the method\nalways generates an end probability vector. Besides, we propose a\nsampling-based training strategy to address the computational cost and memory\nissue in the matrix training phase. We evaluate our method on SQuAD 1.1 and\nthree other question answering benchmarks. Leveraging the most competitive\nmodels BERT and BiDAF as the backbone, our proposed approach can get consistent\nimprovements in all datasets, demonstrating the effectiveness of the proposed\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14375v1",
            "date": "2020-09-30 01:22:58+00:00",
            "title": "Generation of lyrics lines conditioned on music audio clips",
            "authors": [
                "Olga Vechtomova",
                "Gaurav Sahu",
                "Dhruv Kumar"
            ],
            "abstract": "We present a system for generating novel lyrics lines conditioned on music\naudio. A bimodal neural network model learns to generate lines conditioned on\nany given short audio clip. The model consists of a spectrogram variational\nautoencoder (VAE) and a text VAE. Both automatic and human evaluations\ndemonstrate effectiveness of our model in generating lines that have an\nemotional impact matching a given audio clip. The system is intended to serve\nas a creativity tool for songwriters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14384v1",
            "date": "2020-09-30 01:52:00+00:00",
            "title": "Development of Word Embeddings for Uzbek Language",
            "authors": [
                "B. Mansurov",
                "A. Mansurov"
            ],
            "abstract": "In this paper, we share the process of developing word embeddings for the\nCyrillic variant of the Uzbek language. The result of our work is the first\npublicly available set of word vectors trained on the word2vec, GloVe, and\nfastText algorithms using a high-quality web crawl corpus developed in-house.\nThe developed word embeddings can be used in many natural language processing\ndownstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14394v2",
            "date": "2020-09-30 02:33:09+00:00",
            "title": "Multiple Word Embeddings for Increased Diversity of Representation",
            "authors": [
                "Brian Lester",
                "Daniel Pressel",
                "Amy Hemmeter",
                "Sagnik Ray Choudhury",
                "Srinivas Bangalore"
            ],
            "abstract": "Most state-of-the-art models in natural language processing (NLP) are neural\nmodels built on top of large, pre-trained, contextual language models that\ngenerate representations of words in context and are fine-tuned for the task at\nhand. The improvements afforded by these \"contextual embeddings\" come with a\nhigh computational cost. In this work, we explore a simple technique that\nsubstantially and consistently improves performance over a strong baseline with\nnegligible increase in run time. We concatenate multiple pre-trained embeddings\nto strengthen our representation of words. We show that this concatenation\ntechnique works across many tasks, datasets, and model types. We analyze\naspects of pre-trained embedding similarity and vocabulary coverage and find\nthat the representational diversity between different pre-trained embeddings is\nthe driving force of why this technique works. We provide open source\nimplementations of our models in both TensorFlow and PyTorch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14395v1",
            "date": "2020-09-30 02:34:19+00:00",
            "title": "Can Automatic Post-Editing Improve NMT?",
            "authors": [
                "Shamil Chollampatt",
                "Raymond Hendy Susanto",
                "Liling Tan",
                "Ewa Szymanska"
            ],
            "abstract": "Automatic post-editing (APE) aims to improve machine translations, thereby\nreducing human post-editing effort. APE has had notable success when used with\nstatistical machine translation (SMT) systems but has not been as successful\nover neural machine translation (NMT) systems. This has raised questions on the\nrelevance of APE task in the current scenario. However, the training of APE\nmodels has been heavily reliant on large-scale artificial corpora combined with\nonly limited human post-edited data. We hypothesize that APE models have been\nunderperforming in improving NMT translations due to the lack of adequate\nsupervision. To ascertain our hypothesis, we compile a larger corpus of human\npost-edits of English to German NMT. We empirically show that a state-of-art\nneural APE model trained on this corpus can significantly improve a strong\nin-domain NMT system, challenging the current understanding in the field. We\nfurther investigate the effects of varying training data sizes, using\nartificial training data, and domain specificity for the APE task. We release\nthis new corpus under CC BY-NC-SA 4.0 license at\nhttps://github.com/shamilcm/pedra.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14463v1",
            "date": "2020-09-30 06:00:37+00:00",
            "title": "Neural RST-based Evaluation of Discourse Coherence",
            "authors": [
                "Grigorii Guz",
                "Peyman Bateni",
                "Darius Muglich",
                "Giuseppe Carenini"
            ],
            "abstract": "This paper evaluates the utility of Rhetorical Structure Theory (RST) trees\nand relations in discourse coherence evaluation. We show that incorporating\nsilver-standard RST features can increase accuracy when classifying coherence.\nWe demonstrate this through our tree-recursive neural model, namely\nRST-Recursive, which takes advantage of the text's RST features produced by a\nstate of the art RST parser. We evaluate our approach on the Grammarly Corpus\nfor Discourse Coherence (GCDC) and show that when ensembled with the current\nstate of the art, we can achieve the new state of the art accuracy on this\nbenchmark. Furthermore, when deployed alone, RST-Recursive achieves competitive\naccuracy while having 62% fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14658v1",
            "date": "2020-09-30 13:18:57+00:00",
            "title": "Learning Hard Retrieval Cross Attention for Transformer",
            "authors": [
                "Hongfei Xu",
                "Qiuhui Liu"
            ],
            "abstract": "The Transformer translation model that based on the multi-head attention\nmechanism can be parallelized easily and lead to competitive performance in\nmachine translation. The multi-head attention network performs the scaled\ndot-product attention function in parallel, empowering the model by jointly\nattending to information from different representation subspaces at different\npositions. Though its advantages in parallelization, many previous works\nsuggest the computation of the attention mechanism is not sufficiently\nefficient, especially when processing long sequences, and propose approaches to\nimprove its efficiency with long sentences. In this paper, we accelerate the\ninference of the scaled dot-product attention in another perspective.\nSpecifically, instead of squeezing the sequence to attend, we simplify the\ncomputation of the scaled dot-product attention by learning a hard retrieval\nattention which only attends to one token in the sentence rather than all\ntokens. Since the hard attention mechanism only attends to one position, the\nmatrix multiplication between attention probabilities and the value sequence in\nthe standard scaled dot-product attention can be replaced by a simple and\nefficient retrieval operation. As a result, our hard retrieval attention\nmechanism can empirically accelerate the scaled dot-product attention for both\nlong and short sequences by 66.5%, while performing competitively in a wide\nrange of machine translation tasks when using for cross attention networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14725v3",
            "date": "2020-09-30 15:06:56+00:00",
            "title": "A Vietnamese Dataset for Evaluating Machine Reading Comprehension",
            "authors": [
                "Kiet Van Nguyen",
                "Duc-Vu Nguyen",
                "Anh Gia-Tuan Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "Over 97 million people speak Vietnamese as their native language in the\nworld. However, there are few research studies on machine reading comprehension\n(MRC) for Vietnamese, the task of understanding a text and answering questions\nrelated to it. Due to the lack of benchmark datasets for Vietnamese, we present\nthe Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the\nlow-resource language as Vietnamese to evaluate MRC models. This dataset\ncomprises over 23,000 human-generated question-answer pairs based on 5,109\npassages of 174 Vietnamese articles from Wikipedia. In particular, we propose a\nnew process of dataset creation for Vietnamese MRC. Our in-depth analyses\nillustrate that our dataset requires abilities beyond simple reasoning like\nword matching and demands single-sentence and multiple-sentence inferences.\nBesides, we conduct experiments on state-of-the-art MRC methods for English and\nChinese as the first experimental models on UIT-ViQuAD. We also estimate human\nperformance on the dataset and compare it to the experimental results of\npowerful machine learning models. As a result, the substantial differences\nbetween human performance and the best model performance on the dataset\nindicate that improvements can be made on UIT-ViQuAD in future research. Our\ndataset is freely available on our website to encourage the research community\nto overcome challenges in Vietnamese MRC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14780v2",
            "date": "2020-09-30 16:34:27+00:00",
            "title": "Bridging Information-Seeking Human Gaze and Machine Reading Comprehension",
            "authors": [
                "Jonathan Malmaud",
                "Roger Levy",
                "Yevgeni Berzak"
            ],
            "abstract": "In this work, we analyze how human gaze during reading comprehension is\nconditioned on the given reading comprehension question, and whether this\nsignal can be beneficial for machine reading comprehension. To this end, we\ncollect a new eye-tracking dataset with a large number of participants engaging\nin a multiple choice reading comprehension task. Our analysis of this data\nreveals increased fixation times over parts of the text that are most relevant\nfor answering the question. Motivated by this finding, we propose making\nautomated reading comprehension more human-like by mimicking human\ninformation-seeking reading behavior during reading comprehension. We\ndemonstrate that this approach leads to performance gains on multiple choice\nquestion answering in English for a state-of-the-art reading comprehension\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14790v1",
            "date": "2020-09-30 17:00:10+00:00",
            "title": "BERT for Monolingual and Cross-Lingual Reverse Dictionary",
            "authors": [
                "Hang Yan",
                "Xiaonan Li",
                "Xipeng Qiu"
            ],
            "abstract": "Reverse dictionary is the task to find the proper target word given the word\ndescription. In this paper, we tried to incorporate BERT into this task.\nHowever, since BERT is based on the byte-pair-encoding (BPE) subword encoding,\nit is nontrivial to make BERT generate a word given the description. We propose\na simple but effective method to make BERT generate the target word for this\nspecific task. Besides, the cross-lingual reverse dictionary is the task to\nfind the proper target word described in another language. Previous models have\nto keep two different word embeddings and learn to align these embeddings.\nNevertheless, by using the Multilingual BERT (mBERT), we can efficiently\nconduct the cross-lingual reverse dictionary with one subword embedding, and\nthe alignment between languages is not necessary. More importantly, mBERT can\nachieve remarkable cross-lingual reverse dictionary performance even without\nthe parallel corpus, which means it can conduct the cross-lingual reverse\ndictionary with only corresponding monolingual data. Code is publicly available\nat https://github.com/yhcc/BertForRD.git.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14809v2",
            "date": "2020-09-30 17:32:27+00:00",
            "title": "A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing",
            "authors": [
                "Sanxing Chen",
                "Aidan San",
                "Xiaodong Liu",
                "Yangfeng Ji"
            ],
            "abstract": "In Text-to-SQL semantic parsing, selecting the correct entities (tables and\ncolumns) for the generated SQL query is both crucial and challenging; the\nparser is required to connect the natural language (NL) question and the SQL\nquery to the structured knowledge in the database. We formulate two linking\nprocesses to address this challenge: schema linking which links explicit NL\nmentions to the database and structural linking which links the entities in the\noutput SQL with their structural relationships in the database schema.\nIntuitively, the effectiveness of these two linking processes changes based on\nthe entity being generated, thus we propose to dynamically choose between them\nusing a gating mechanism. Integrating the proposed method with two graph neural\nnetwork-based semantic parsers together with BERT representations demonstrates\nsubstantial gains in parsing accuracy on the challenging Spider dataset.\nAnalyses show that our proposed method helps to enhance the structure of the\nmodel output when generating complicated SQL queries and offers more\nexplainable predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14824v1",
            "date": "2020-09-30 17:54:56+00:00",
            "title": "On Romanization for Model Transfer Between Scripts in Neural Machine Translation",
            "authors": [
                "Chantal Amrhein",
                "Rico Sennrich"
            ],
            "abstract": "Transfer learning is a popular strategy to improve the quality of\nlow-resource machine translation. For an optimal transfer of the embedding\nlayer, the child and parent model should share a substantial part of the\nvocabulary. This is not the case when transferring to languages with a\ndifferent script. We explore the benefit of romanization in this scenario. Our\nresults show that romanization entails information loss and is thus not always\nsuperior to simpler vocabulary transfer methods, but can improve the transfer\nbetween related languages with different scripts. We compare two romanization\ntools and find that they exhibit different degrees of information loss, which\naffects translation quality. Finally, we extend romanization to the target\nside, showing that this can be a successful strategy when coupled with a simple\nderomanization model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00150v1",
            "date": "2020-09-30 23:54:38+00:00",
            "title": "Learning from Mistakes: Combining Ontologies via Self-Training for Dialogue Generation",
            "authors": [
                "Lena Reed",
                "Vrindavan Harrison",
                "Shereen Oraby",
                "Dilek Hakkani-Tur",
                "Marilyn Walker"
            ],
            "abstract": "Natural language generators (NLGs) for task-oriented dialogue typically take\na meaning representation (MR) as input. They are trained end-to-end with a\ncorpus of MR/utterance pairs, where the MRs cover a specific set of dialogue\nacts and domain attributes. Creation of such datasets is labor-intensive and\ntime-consuming. Therefore, dialogue systems for new domain ontologies would\nbenefit from using data for pre-existing ontologies. Here we explore, for the\nfirst time, whether it is possible to train an NLG for a new larger ontology\nusing existing training sets for the restaurant domain, where each set is based\non a different ontology. We create a new, larger combined ontology, and then\ntrain an NLG to produce utterances covering it. For example, if one dataset has\nattributes for family-friendly and rating information, and the other has\nattributes for decor and service, our aim is an NLG for the combined ontology\nthat can produce utterances that realize values for family-friendly, rating,\ndecor and service. Initial experiments with a baseline neural\nsequence-to-sequence model show that this task is surprisingly challenging. We\nthen develop a novel self-training method that identifies (errorful) model\noutputs, automatically constructs a corrected MR input to form a new (MR,\nutterance) training pair, and then repeatedly adds these new instances back\ninto the training data. We then test the resulting model on a new test set. The\nresult is a self-trained model whose performance is an absolute 75.4%\nimprovement over the baseline model. We also report a human qualitative\nevaluation of the final model showing that it achieves high naturalness,\nsemantic coherence and grammaticality",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00153v2",
            "date": "2020-10-01 00:18:43+00:00",
            "title": "Examining the rhetorical capacities of neural language models",
            "authors": [
                "Zining Zhu",
                "Chuer Pan",
                "Mohamed Abdalla",
                "Frank Rudzicz"
            ],
            "abstract": "Recently, neural language models (LMs) have demonstrated impressive abilities\nin generating high-quality discourse. While many recent papers have analyzed\nthe syntactic aspects encoded in LMs, there has been no analysis to date of the\ninter-sentential, rhetorical knowledge. In this paper, we propose a method that\nquantitatively evaluates the rhetorical capacities of neural LMs. We examine\nthe capacities of neural LMs understanding the rhetoric of discourse by\nevaluating their abilities to encode a set of linguistic features derived from\nRhetorical Structure Theory (RST). Our experiments show that BERT-based LMs\noutperform other Transformer LMs, revealing the richer discourse knowledge in\ntheir intermediate layer representations. In addition, GPT-2 and XLNet\napparently encode less rhetorical knowledge, and we suggest an explanation\ndrawing from linguistic philosophy. Our method shows an avenue towards\nquantifying the rhetorical capacities of neural LMs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00190v1",
            "date": "2020-10-01 03:44:44+00:00",
            "title": "A Compare Aggregate Transformer for Understanding Document-grounded Dialogue",
            "authors": [
                "Longxuan Ma",
                "Weinan Zhang",
                "Runxin Sun",
                "Ting Liu"
            ],
            "abstract": "Unstructured documents serving as external knowledge of the dialogues help to\ngenerate more informative responses. Previous research focused on knowledge\nselection (KS) in the document with dialogue. However, dialogue history that is\nnot related to the current dialogue may introduce noise in the KS processing.\nIn this paper, we propose a Compare Aggregate Transformer (CAT) to jointly\ndenoise the dialogue context and aggregate the document information for\nresponse generation. We designed two different comparison mechanisms to reduce\nnoise (before and during decoding). In addition, we propose two metrics for\nevaluating document utilization efficiency based on word overlap. Experimental\nresults on the CMUDoG dataset show that the proposed CAT model outperforms the\nstate-of-the-art approach and strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00198v1",
            "date": "2020-10-01 05:21:32+00:00",
            "title": "Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models",
            "authors": [
                "Thai Binh Nguyen",
                "Quang Minh Nguyen",
                "Thi Thu Hien Nguyen",
                "Quoc Truong Do",
                "Chi Mai Luong"
            ],
            "abstract": "Studies on the Named Entity Recognition (NER) task have shown outstanding\nresults that reach human parity on input texts with correct text formattings,\nsuch as with proper punctuation and capitalization. However, such conditions\nare not available in applications where the input is speech, because the text\nis generated from a speech recognition system (ASR), and that the system does\nnot consider the text formatting. In this paper, we (1) presented the first\nVietnamese speech dataset for NER task, and (2) the first pre-trained public\nlarge-scale monolingual language model for Vietnamese that achieved the new\nstate-of-the-art for the Vietnamese NER task by 1.3% absolute F1 score\ncomparing to the latest study. And finally, (3) we proposed a new pipeline for\nNER task from speech that overcomes the text formatting problem by introducing\na text capitalization and punctuation recovery model (CaPu) into the pipeline.\nThe model takes input text from an ASR system and performs two tasks at the\nsame time, producing proper text formatting that helps to improve NER\nperformance. Experimental results indicated that the CaPu model helps to\nimprove by nearly 4% of F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00287v2",
            "date": "2020-10-01 10:32:17+00:00",
            "title": "Joint Persian Word Segmentation Correction and Zero-Width Non-Joiner Recognition Using BERT",
            "authors": [
                "Ehsan Doostmohammadi",
                "Minoo Nassajian",
                "Adel Rahimi"
            ],
            "abstract": "Words are properly segmented in the Persian writing system; in practice,\nhowever, these writing rules are often neglected, resulting in single words\nbeing written disjointedly and multiple words written without any white spaces\nbetween them. This paper addresses the problems of word segmentation and\nzero-width non-joiner (ZWNJ) recognition in Persian, which we approach jointly\nas a sequence labeling problem. We achieved a macro-averaged F1-score of 92.40%\non a carefully collected corpus of 500 sentences with a high level of\ndifficulty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00294v3",
            "date": "2020-10-01 10:54:54+00:00",
            "title": "Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting",
            "authors": [
                "Anshul Wadhawan"
            ],
            "abstract": "This paper presents the approach that we employed to tackle the EMNLP\nWNUT-2020 Shared Task 2 : Identification of informative COVID-19 English\nTweets. The task is to develop a system that automatically identifies whether\nan English Tweet related to the novel coronavirus (COVID-19) is informative or\nnot. We solve the task in three stages. The first stage involves pre-processing\nthe dataset by filtering only relevant information. This is followed by\nexperimenting with multiple deep learning models like CNNs, RNNs and\nTransformer based models. In the last stage, we propose an ensemble of the best\nmodel trained on different subsets of the provided dataset. Our final approach\nachieved an F1-score of 0.9037 and we were ranked sixth overall with F1-score\nas the evaluation criteria.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00310v3",
            "date": "2020-10-01 11:41:44+00:00",
            "title": "\"Did you really mean what you said?\" : Sarcasm Detection in Hindi-English Code-Mixed Data using Bilingual Word Embeddings",
            "authors": [
                "Akshita Aggarwal",
                "Anshul Wadhawan",
                "Anshima Chaudhary",
                "Kavita Maurya"
            ],
            "abstract": "With the increased use of social media platforms by people across the world,\nmany new interesting NLP problems have come into existence. One such being the\ndetection of sarcasm in the social media texts. We present a corpus of tweets\nfor training custom word embeddings and a Hinglish dataset labelled for sarcasm\ndetection. We propose a deep learning based approach to address the issue of\nsarcasm detection in Hindi-English code mixed tweets using bilingual word\nembeddings derived from FastText and Word2Vec approaches. We experimented with\nvarious deep learning models, including CNNs, LSTMs, Bi-directional LSTMs (with\nand without attention). We were able to outperform all state-of-the-art\nperformances with our deep learning models, with attention based Bi-directional\nLSTMs giving the best performance exhibiting an accuracy of 78.49%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00363v1",
            "date": "2020-10-01 12:49:01+00:00",
            "title": "How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text",
            "authors": [
                "Chihiro Shibata",
                "Kei Uchiumi",
                "Daichi Mochihashi"
            ],
            "abstract": "Long Short-Term Memory recurrent neural network (LSTM) is widely used and\nknown to capture informative long-term syntactic dependencies. However, how\nsuch information are reflected in its internal vectors for natural text has not\nyet been sufficiently investigated. We analyze them by learning a language\nmodel where syntactic structures are implicitly given. We empirically show that\nthe context update vectors, i.e. outputs of internal gates, are approximately\nquantized to binary or ternary values to help the language model to count the\ndepth of nesting accurately, as Suzgun et al. (2019) recently show for\nsynthetic Dyck languages. For some dimensions in the context vector, we show\nthat their activations are highly correlated with the depth of phrase\nstructures, such as VP and NP. Moreover, with an $L_1$ regularization, we also\nfound that it can accurately predict whether a word is inside a phrase\nstructure or not from a small number of components of the context vector. Even\nfor the case of learning from raw text, context vectors are shown to still\ncorrelate well with the phrase structures. Finally, we show that natural\nclusters of the functional words and the part of speeches that trigger phrases\nare represented in a small but principal subspace of the context-update vector\nof LSTM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00372v1",
            "date": "2020-10-01 13:10:03+00:00",
            "title": "Citation Sentiment Changes Analysis",
            "authors": [
                "Haixia Liu"
            ],
            "abstract": "Metrics for measuring the citation sentiment changes were introduced.\nCitation sentiment changes can be observed from global citation sentiment\nsequences (GCSSs). With respect to a cited paper, the citation sentiment\nsequences were analysed across a collection of citing papers ordered by the\npublished time. For analysing GCSSs, Eddy Dissipation Rate (EDR) was adopted,\nwith the hypothesis that the GCSSs pattern differences can be spotted by EDR\nbased method. Preliminary evidence showed that EDR based method holds the\npotential for analysing a publication's impact in a time series fashion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00454v2",
            "date": "2020-10-01 14:48:31+00:00",
            "title": "Evaluating Multilingual BERT for Estonian",
            "authors": [
                "Claudia Kittask",
                "Kirill Milintsevich",
                "Kairit Sirts"
            ],
            "abstract": "Recently, large pre-trained language models, such as BERT, have reached\nstate-of-the-art performance in many natural language processing tasks, but for\nmany languages, including Estonian, BERT models are not yet available. However,\nthere exist several multilingual BERT models that can handle multiple languages\nsimultaneously and that have been trained also on Estonian data. In this paper,\nwe evaluate four multilingual models -- multilingual BERT, multilingual\ndistilled BERT, XLM and XLM-RoBERTa -- on several NLP tasks including POS and\nmorphological tagging, NER and text classification. Our aim is to establish a\ncomparison between these multilingual BERT models and the existing baseline\nneural models for these tasks. Our results show that multilingual BERT models\ncan generalise well on different Estonian NLP tasks outperforming all baselines\nmodels for POS and morphological tagging and text classification, and reaching\nthe comparable level with the best baseline for NER, with XLM-RoBERTa achieving\nthe highest results compared with other multilingual models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00490v2",
            "date": "2020-10-01 15:33:09+00:00",
            "title": "Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary",
            "authors": [
                "Daniel Deutsch",
                "Tania Bedrax-Weiss",
                "Dan Roth"
            ],
            "abstract": "A desirable property of a reference-based evaluation metric that measures the\ncontent quality of a summary is that it should estimate how much information\nthat summary has in common with a reference. Traditional text overlap based\nmetrics such as ROUGE fail to achieve this because they are limited to matching\ntokens, either lexically or via embeddings. In this work, we propose a metric\nto evaluate the content quality of a summary using question-answering (QA).\nQA-based methods directly measure a summary's information overlap with a\nreference, making them fundamentally different than text overlap metrics. We\ndemonstrate the experimental benefits of QA-based metrics through an analysis\nof our proposed metric, QAEval. QAEval out-performs current state-of-the-art\nmetrics on most evaluations using benchmark datasets, while being competitive\non others due to limitations of state-of-the-art models. Through a careful\nanalysis of each component of QAEval, we identify its performance bottlenecks\nand estimate that its potential upper-bound performance surpasses all other\nautomatic metrics, approaching that of the gold-standard Pyramid Method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00526v1",
            "date": "2020-10-01 16:18:51+00:00",
            "title": "LiveQA: A Question Answering Dataset over Sports Live",
            "authors": [
                "Qianying Liu",
                "Sicong Jiang",
                "Yizhong Wang",
                "Sujian Li"
            ],
            "abstract": "In this paper, we introduce LiveQA, a new question answering dataset\nconstructed from play-by-play live broadcast. It contains 117k multiple-choice\nquestions written by human commentators for over 1,670 NBA games, which are\ncollected from the Chinese Hupu (https://nba.hupu.com/games) website. Derived\nfrom the characteristics of sports games, LiveQA can potentially test the\nreasoning ability across timeline-based live broadcasts, which is challenging\ncompared to the existing datasets. In LiveQA, the questions require\nunderstanding the timeline, tracking events or doing mathematical computations.\nOur preliminary experiments show that the dataset introduces a challenging\nproblem for question answering models, and a strong baseline model only\nachieves the accuracy of 53.1\\% and cannot beat the dominant option rule. We\nrelease the code and data of this paper for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00633v1",
            "date": "2020-10-01 18:17:58+00:00",
            "title": "Discontinuous Constituent Parsing as Sequence Labeling",
            "authors": [
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "This paper reduces discontinuous parsing to sequence labeling. It first shows\nthat existing reductions for constituent parsing as labeling do not support\ndiscontinuities. Second, it fills this gap and proposes to encode tree\ndiscontinuities as nearly ordered permutations of the input sequence. Third, it\nstudies whether such discontinuous representations are learnable. The\nexperiments show that despite the architectural simplicity, under the right\nrepresentation, the models are fast and accurate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00710v1",
            "date": "2020-10-01 22:24:46+00:00",
            "title": "Nearest Neighbor Machine Translation",
            "authors": [
                "Urvashi Khandelwal",
                "Angela Fan",
                "Dan Jurafsky",
                "Luke Zettlemoyer",
                "Mike Lewis"
            ],
            "abstract": "We introduce $k$-nearest-neighbor machine translation ($k$NN-MT), which\npredicts tokens with a nearest neighbor classifier over a large datastore of\ncached examples, using representations from a neural translation model for\nsimilarity search. This approach requires no additional training and scales to\ngive the decoder direct access to billions of examples at test time, resulting\nin a highly expressive model that consistently improves performance across many\nsettings. Simply adding nearest neighbor search improves a state-of-the-art\nGerman-English translation model by 1.5 BLEU. $k$NN-MT allows a single model to\nbe adapted to diverse domains by using a domain-specific datastore, improving\nresults by an average of 9.2 BLEU over zero-shot transfer, and achieving new\nstate-of-the-art results---without training on these domains. A massively\nmultilingual model can also be specialized for particular language pairs, with\nimprovements of 3 BLEU for translating from English into German and Chinese.\nQualitatively, $k$NN-MT is easily interpretable; it combines source and target\ncontext to retrieve highly relevant examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00760v1",
            "date": "2020-10-02 03:09:26+00:00",
            "title": "STIL -- Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++",
            "authors": [
                "Jack G. M. FitzGerald"
            ],
            "abstract": "Slot-filling, Translation, Intent classification, and Language\nidentification, or STIL, is a newly-proposed task for multilingual Natural\nLanguage Understanding (NLU). By performing simultaneous slot filling and\ntranslation into a single output language (English in this case), some portion\nof downstream system components can be monolingual, reducing development and\nmaintenance cost. Results are given using the multilingual BART model (Liu et\nal., 2020) fine-tuned on 7 languages using the MultiATIS++ dataset. When no\ntranslation is performed, mBART's performance is comparable to the current\nstate of the art system (Cross-Lingual BERT by Xu et al. (2020)) for the\nlanguages tested, with better average intent classification accuracy (96.07%\nversus 95.50%) but worse average slot F1 (89.87% versus 90.81%). When\nsimultaneous translation is performed, average intent classification accuracy\ndegrades by only 1.7% relative and average slot F1 degrades by only 1.2%\nrelative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00761v1",
            "date": "2020-10-02 03:15:03+00:00",
            "title": "Enriching Word Embeddings with Temporal and Spatial Information",
            "authors": [
                "Hongyu Gong",
                "Suma Bhat",
                "Pramod Viswanath"
            ],
            "abstract": "The meaning of a word is closely linked to sociocultural factors that can\nchange over time and location, resulting in corresponding meaning changes.\nTaking a global view of words and their meanings in a widely used language,\nsuch as English, may require us to capture more refined semantics for use in\ntime-specific or location-aware situations, such as the study of cultural\ntrends or language use. However, popular vector representations for words do\nnot adequately include temporal or spatial information. In this work, we\npresent a model for learning word representation conditioned on time and\nlocation. In addition to capturing meaning changes over time and location, we\nrequire that the resulting word embeddings retain salient semantic and\ngeometric properties. We train our model on time- and location-stamped corpora,\nand show using both quantitative and qualitative evaluations that it can\ncapture semantics across time and locations. We note that our model compares\nfavorably with the state-of-the-art for time-specific embedding, and serves as\na new benchmark for location-specific embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00767v3",
            "date": "2020-10-02 03:54:37+00:00",
            "title": "Enhancing Fine-grained Sentiment Classification Exploiting Local Context Embedding",
            "authors": [
                "Heng Yang",
                "Biqing Zeng"
            ],
            "abstract": "Target-oriented sentiment classification is a fine-grained task of natural\nlanguage processing to analyze the sentiment polarity of the targets. To\nimprove the performance of sentiment classification, many approaches proposed\nvarious attention mechanisms to capture the important context words of a\ntarget. However, previous approaches ignored the significant relatedness of a\ntarget's sentiment and its local context. This paper proposes a local\ncontext-aware network (LCA-Net), equipped with the local context embedding and\nlocal context prediction loss, to strengthen the model by emphasizing the\nsentiment information of the local context. The experimental results on three\ncommon datasets show that local context-aware network performs superior to\nexisting approaches in extracting local context features. Besides, the local\ncontext-aware framework is easy to adapt to many models, with the potential to\nimprove other target-level tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00784v1",
            "date": "2020-10-01 09:20:18+00:00",
            "title": "An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training",
            "authors": [
                "Kristjan Arumae",
                "Qing Sun",
                "Parminder Bhatia"
            ],
            "abstract": "Pre-training large language models has become a standard in the natural\nlanguage processing community. Such models are pre-trained on generic data\n(e.g. BookCorpus and English Wikipedia) and often fine-tuned on tasks in the\nsame domain. However, in order to achieve state-of-the-art performance on out\nof domain tasks such as clinical named entity recognition and relation\nextraction, additional in domain pre-training is required. In practice, staged\nmulti-domain pre-training presents performance deterioration in the form of\ncatastrophic forgetting (CF) when evaluated on a generic benchmark such as\nGLUE. In this paper we conduct an empirical investigation into known methods to\nmitigate CF. We find that elastic weight consolidation provides best overall\nscores yielding only a 0.33% drop in performance across seven generic tasks\nwhile remaining competitive in bio-medical tasks. Furthermore, we explore\ngradient and latent clustering based data selection techniques to improve\ncoverage when using elastic weight consolidation and experience replay methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00796v1",
            "date": "2020-10-02 05:53:36+00:00",
            "title": "JAKET: Joint Pre-training of Knowledge Graph and Language Understanding",
            "authors": [
                "Donghan Yu",
                "Chenguang Zhu",
                "Yiming Yang",
                "Michael Zeng"
            ],
            "abstract": "Knowledge graphs (KGs) contain rich information about world knowledge,\nentities and relations. Thus, they can be great supplements to existing\npre-trained language models. However, it remains a challenge to efficiently\nintegrate information from KG into language modeling. And the understanding of\na knowledge graph requires related context. We propose a novel joint\npre-training framework, JAKET, to model both the knowledge graph and language.\nThe knowledge module and language module provide essential information to\nmutually assist each other: the knowledge module produces embeddings for\nentities in text while the language module generates context-aware initial\nembeddings for entities and relations in the graph. Our design enables the\npre-trained model to easily adapt to unseen knowledge graphs in new domains.\nExperimental results on several knowledge-aware NLP tasks show that our\nproposed framework achieves superior performance by effectively leveraging\nknowledge in language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00840v1",
            "date": "2020-10-02 08:07:12+00:00",
            "title": "MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models",
            "authors": [
                "Peng Xu",
                "Mostofa Patwary",
                "Mohammad Shoeybi",
                "Raul Puri",
                "Pascale Fung",
                "Anima Anandkumar",
                "Bryan Catanzaro"
            ],
            "abstract": "Existing pre-trained large language models have shown unparalleled generative\ncapabilities. However, they are not controllable. In this paper, we propose\nMEGATRON-CNTRL, a novel framework that uses large-scale language models and\nadds control to text generation by incorporating an external knowledge base.\nOur framework consists of a keyword predictor, a knowledge retriever, a\ncontextual knowledge ranker, and a conditional text generator. As we do not\nhave access to ground-truth supervision for the knowledge ranker, we make use\nof weak supervision from sentence embedding. The empirical results show that\nour model generates more fluent, consistent, and coherent stories with less\nrepetition and higher diversity compared to prior work on the ROC story\ndataset. We showcase the controllability of our model by replacing the keywords\nused to generate stories and re-running the generation process. Human\nevaluation results show that 77.5% of these stories are successfully controlled\nby the new keywords. Furthermore, by scaling our model from 124 million to 8.3\nbillion parameters we demonstrate that larger models improve both the quality\nof generation (from 74.5% to 93.0% for consistency) and controllability (from\n77.5% to 91.5%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01054v1",
            "date": "2020-10-02 15:33:42+00:00",
            "title": "Unsupervised Text Style Transfer with Padded Masked Language Models",
            "authors": [
                "Eric Malmi",
                "Aliaksei Severyn",
                "Sascha Rothe"
            ],
            "abstract": "We propose Masker, an unsupervised text-editing method for style transfer. To\ntackle cases when no parallel source-target pairs are available, we train\nmasked language models (MLMs) for both the source and the target domain. Then\nwe find the text spans where the two models disagree the most in terms of\nlikelihood. This allows us to identify the source tokens to delete to transform\nthe source text to match the style of the target domain. The deleted tokens are\nreplaced with the target MLM, and by using a padded MLM variant, we avoid\nhaving to predetermine the number of inserted tokens. Our experiments on\nsentence fusion and sentiment transfer demonstrate that Masker performs\ncompetitively in a fully unsupervised setting. Moreover, in low-resource\nsettings, it improves supervised methods' accuracy by over 10 percentage points\nwhen pre-training them on silver training data generated by Masker.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01063v1",
            "date": "2020-10-02 15:44:58+00:00",
            "title": "Syntax Representation in Word Embeddings and Neural Networks -- A Survey",
            "authors": [
                "Tomasz Limisiewicz",
                "David Mare\u010dek"
            ],
            "abstract": "Neural networks trained on natural language processing tasks capture syntax\neven though it is not provided as a supervision signal. This indicates that\nsyntactic analysis is essential to the understating of language in artificial\nintelligence systems. This overview paper covers approaches of evaluating the\namount of syntactic information included in the representations of words for\ndifferent neural network architectures. We mainly summarize re-search on\nEnglish monolingual data on language modeling tasks and multilingual data for\nneural machine translation systems and multilingual language models. We\ndescribe which pre-trained models and representations of language are best\nsuited for transfer to syntactic tasks.",
            "journal ref": "Proceedings of the 20th Conference ITAT 2020: Automata, Formal and\n  Natural Languages Workshop",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01080v1",
            "date": "2020-10-02 16:20:30+00:00",
            "title": "HUMAN: Hierarchical Universal Modular ANnotator",
            "authors": [
                "Moritz Wolf",
                "Dana Ruiter",
                "Ashwin Geet D'Sa",
                "Liane Reiners",
                "Jan Alexandersson",
                "Dietrich Klakow"
            ],
            "abstract": "A lot of real-world phenomena are complex and cannot be captured by single\ntask annotations. This causes a need for subsequent annotations, with\ninterdependent questions and answers describing the nature of the subject at\nhand. Even in the case a phenomenon is easily captured by a single task, the\nhigh specialisation of most annotation tools can result in having to switch to\nanother tool if the task only slightly changes.\n  We introduce HUMAN, a novel web-based annotation tool that addresses the\nabove problems by a) covering a variety of annotation tasks on both textual and\nimage data, and b) the usage of an internal deterministic state machine,\nallowing the researcher to chain different annotation tasks in an\ninterdependent manner. Further, the modular nature of the tool makes it easy to\ndefine new annotation tasks and integrate machine learning algorithms e.g., for\nactive learning. HUMAN comes with an easy-to-use graphical user interface that\nsimplifies the annotation task and management.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01108v1",
            "date": "2020-10-02 17:09:47+00:00",
            "title": "Cross-Lingual Transfer Learning for Complex Word Identification",
            "authors": [
                "George-Eduard Zaharia",
                "Dumitru-Clementin Cercel",
                "Mihai Dascalu"
            ],
            "abstract": "Complex Word Identification (CWI) is a task centered on detecting\nhard-to-understand words, or groups of words, in texts from different areas of\nexpertise. The purpose of CWI is to highlight problematic structures that\nnon-native speakers would usually find difficult to understand. Our approach\nuses zero-shot, one-shot, and few-shot learning techniques, alongside\nstate-of-the-art solutions for Natural Language Processing (NLP) tasks (i.e.,\nTransformers). Our aim is to provide evidence that the proposed models can\nlearn the characteristics of complex words in a multilingual environment by\nrelying on the CWI shared task 2018 dataset available for four different\nlanguages (i.e., English, German, Spanish, and also French). Our approach\nsurpasses state-of-the-art cross-lingual results in terms of macro F1-score on\nEnglish (0.774), German (0.782), and Spanish (0.734) languages, for the\nzero-shot learning scenario. At the same time, our model also outperforms the\nstate-of-the-art monolingual result for German (0.795 macro F1-score).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01150v1",
            "date": "2020-10-02 18:06:31+00:00",
            "title": "Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media",
            "authors": [
                "Xiang Dai",
                "Sarvnaz Karimi",
                "Ben Hachey",
                "Cecile Paris"
            ],
            "abstract": "Recent studies on domain-specific BERT models show that effectiveness on\ndownstream tasks can be improved when models are pretrained on in-domain data.\nOften, the pretraining data used in these models are selected based on their\nsubject matter, e.g., biology or computer science. Given the range of\napplications using social media text, and its unique language variety, we\npretrain two models on tweets and forum text respectively, and empirically\ndemonstrate the effectiveness of these two resources. In addition, we\ninvestigate how similarity measures can be used to nominate in-domain\npretraining data. We publicly release our pretrained models at\nhttps://bit.ly/35RpTf0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01160v2",
            "date": "2020-10-02 18:31:45+00:00",
            "title": "Automatic Extraction of Rules Governing Morphological Agreement",
            "authors": [
                "Aditi Chaudhary",
                "Antonios Anastasopoulos",
                "Adithya Pratapa",
                "David R. Mortensen",
                "Zaid Sheikh",
                "Yulia Tsvetkov",
                "Graham Neubig"
            ],
            "abstract": "Creating a descriptive grammar of a language is an indispensable step for\nlanguage documentation and preservation. However, at the same time it is a\ntedious, time-consuming task. In this paper, we take steps towards automating\nthis process by devising an automated framework for extracting a first-pass\ngrammatical specification from raw text in a concise, human- and\nmachine-readable format. We focus on extracting rules describing agreement, a\nmorphosyntactic phenomenon at the core of the grammars of many of the world's\nlanguages. We apply our framework to all languages included in the Universal\nDependencies project, with promising results. Using cross-lingual transfer,\neven with no expert annotations in the language of interest, our framework\nextracts a grammatical specification which is nearly equivalent to those\ncreated with large amounts of gold-standard annotated data. We confirm this\nfinding with human expert evaluations of the rules that our framework produces,\nwhich have an average accuracy of 78%. We release an interface demonstrating\nthe extracted rules at https://neulab.github.io/lase/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01169v2",
            "date": "2020-10-02 19:06:29+00:00",
            "title": "DocuBot : Generating financial reports using natural language interactions",
            "authors": [
                "Vineeth Ravi",
                "Selim Amrouni",
                "Andrea Stefanucci",
                "Armineh Nourbakhsh",
                "Prashant Reddy",
                "Manuela Veloso"
            ],
            "abstract": "The financial services industry perpetually processes an overwhelming amount\nof complex data. Digital reports are often created based on tedious manual\nanalysis as well as visualization of the underlying trends and characteristics\nof data. Often, the accruing costs of human computation errors in creating\nthese reports are very high. We present DocuBot, a novel AI-powered virtual\nassistant for creating and modifying content in digital documents by modeling\nnatural language interactions as \"skills\" and using them to transform\nunderlying data. DocuBot has the ability to agglomerate saved skills for reuse,\nenabling humans to automatically generate recurrent reports. DocuBot also has\nthe capability to continuously learn domain-specific and user-specific\nvocabulary by interacting with the user. We present evidence that DocuBot adds\nvalue to the financial industry and demonstrate its impact with experiments\ninvolving real and simulated users tasked with creating PowerPoint\npresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01239v1",
            "date": "2020-10-03 00:45:01+00:00",
            "title": "Mining Knowledge for Natural Language Inference from Wikipedia Categories",
            "authors": [
                "Mingda Chen",
                "Zewei Chu",
                "Karl Stratos",
                "Kevin Gimpel"
            ],
            "abstract": "Accurate lexical entailment (LE) and natural language inference (NLI) often\nrequire large quantities of costly annotations. To alleviate the need for\nlabeled data, we introduce WikiNLI: a resource for improving model performance\non NLI and LE tasks. It contains 428,899 pairs of phrases constructed from\nnaturally annotated category hierarchies in Wikipedia. We show that we can\nimprove strong baselines such as BERT and RoBERTa by pretraining them on\nWikiNLI and transferring the models on downstream tasks. We conduct systematic\ncomparisons with phrases extracted from other knowledge bases such as WordNet\nand Wikidata to find that pretraining on WikiNLI gives the best performance. In\naddition, we construct WikiNLI in other languages, and show that pretraining on\nthem improves performance on NLI tasks of corresponding languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01263v1",
            "date": "2020-10-03 02:52:28+00:00",
            "title": "Multilevel Text Alignment with Cross-Document Attention",
            "authors": [
                "Xuhui Zhou",
                "Nikolaos Pappas",
                "Noah A. Smith"
            ],
            "abstract": "Text alignment finds application in tasks such as citation recommendation and\nplagiarism detection. Existing alignment methods operate at a single,\npredefined level and cannot learn to align texts at, for example, sentence and\ndocument levels. We propose a new learning approach that equips previously\nestablished hierarchical attention encoders for representing documents with a\ncross-document attention component, enabling structural comparisons across\ndifferent levels (document-to-document and sentence-to-document). Our component\nis weakly supervised from document pairs and can align at multiple levels. Our\nevaluation on predicting document-to-document relationships and\nsentence-to-document relationships on the tasks of citation recommendation and\nplagiarism detection shows that our approach outperforms previously established\nhierarchical, attention encoders based on recurrent and transformer\ncontextualization that are unaware of structural correspondence between\ndocuments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01268v1",
            "date": "2020-10-03 03:18:52+00:00",
            "title": "Partially-Aligned Data-to-Text Generation with Distant Supervision",
            "authors": [
                "Zihao Fu",
                "Bei Shi",
                "Wai Lam",
                "Lidong Bing",
                "Zhiyuan Liu"
            ],
            "abstract": "The Data-to-Text task aims to generate human-readable text for describing\nsome given structured data enabling more interpretability. However, the typical\ngeneration task is confined to a few particular domains since it requires\nwell-aligned data which is difficult and expensive to obtain. Using\npartially-aligned data is an alternative way of solving the dataset scarcity\nproblem. This kind of data is much easier to obtain since it can be produced\nautomatically. However, using this kind of data induces the over-generation\nproblem posing difficulties for existing models, which tends to add unrelated\nexcerpts during the generation procedure. In order to effectively utilize\nautomatically annotated partially-aligned datasets, we extend the traditional\ngeneration task to a refined task called Partially-Aligned Data-to-Text\nGeneration (PADTG) which is more practical since it utilizes automatically\nannotated data for training and thus considerably expands the application\ndomains. To tackle this new task, we propose a novel distant supervision\ngeneration framework. It firstly estimates the input data's supportiveness for\neach target word with an estimator and then applies a supportiveness adaptor\nand a rebalanced beam search to harness the over-generation problem in the\ntraining and generation phases respectively. We also contribute a\npartially-aligned dataset (The data and source code of this paper can be\nobtained from https://github.com/fuzihaofzh/distant_supervision_nlg by sampling\nsentences from Wikipedia and automatically extracting corresponding KB triples\nfor each sentence from Wikidata. The experimental results show that our\nframework outperforms all baseline models as well as verify the feasibility of\nutilizing partially-aligned data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01272v1",
            "date": "2020-10-03 04:03:52+00:00",
            "title": "Towards Interpretable Reasoning over Paragraph Effects in Situation",
            "authors": [
                "Mucheng Ren",
                "Xiubo Geng",
                "Tao Qin",
                "Heyan Huang",
                "Daxin Jiang"
            ],
            "abstract": "We focus on the task of reasoning over paragraph effects in situation, which\nrequires a model to understand the cause and effect described in a background\nparagraph, and apply the knowledge to a novel situation. Existing works ignore\nthe complicated reasoning process and solve it with a one-step \"black box\"\nmodel. Inspired by human cognitive processes, in this paper we propose a\nsequential approach for this task which explicitly models each step of the\nreasoning process with neural network modules. In particular, five reasoning\nmodules are designed and learned in an end-to-end manner, which leads to a more\ninterpretable model. Experimental results on the ROPES dataset demonstrate the\neffectiveness and explainability of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01288v2",
            "date": "2020-10-03 06:14:06+00:00",
            "title": "Unsupervised Cross-lingual Image Captioning",
            "authors": [
                "Jiahui Gao",
                "Yi Zhou",
                "Philip L. H. Yu",
                "Shafiq Joty",
                "Jiuxiang Gu"
            ],
            "abstract": "Research in image captioning has mostly focused on English because of the\navailability of image-caption paired datasets in this language. However,\nbuilding vision-language systems only for English deprives a large part of the\nworld population of AI technologies' benefit. On the other hand, creating\nimage-caption paired datasets for every target language is expensive. In this\nwork, we present a novel unsupervised cross-lingual method to generate image\ncaptions in a target language without using any image-caption corpus in the\nsource or target languages. Our method relies on (i) a cross-lingual scene\ngraph to sentence translation process, which learns to decode sentences in the\ntarget language from a cross-lingual encoding space of scene graphs using a\nsentence parallel (bitext) corpus, and (ii) an unsupervised cross-modal feature\nmapping which seeks to map an encoded scene graph features from image modality\nto language modality. We verify the effectiveness of our proposed method on the\nChinese image caption generation task. The comparisons against several existing\nmethods demonstrate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01345v1",
            "date": "2020-10-03 12:58:47+00:00",
            "title": "A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples",
            "authors": [
                "Zhao Meng",
                "Roger Wattenhofer"
            ],
            "abstract": "Generating adversarial examples for natural language is hard, as natural\nlanguage consists of discrete symbols, and examples are often of variable\nlengths. In this paper, we propose a geometry-inspired attack for generating\nnatural language adversarial examples. Our attack generates adversarial\nexamples by iteratively approximating the decision boundary of Deep Neural\nNetworks (DNNs). Experiments on two datasets with two different models show\nthat our attack fools natural language models with high success rates, while\nonly replacing a few words. Human evaluation shows that adversarial examples\ngenerated by our attack are hard for humans to recognize. Further experiments\nshow that adversarial training can improve model robustness against our attack.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01454v1",
            "date": "2020-10-04 00:35:47+00:00",
            "title": "MIME: MIMicking Emotions for Empathetic Response Generation",
            "authors": [
                "Navonil Majumder",
                "Pengfei Hong",
                "Shanshan Peng",
                "Jiankun Lu",
                "Deepanway Ghosal",
                "Alexander Gelbukh",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "Current approaches to empathetic response generation view the set of emotions\nexpressed in the input text as a flat structure, where all the emotions are\ntreated uniformly. We argue that empathetic responses often mimic the emotion\nof the user to a varying degree, depending on its positivity or negativity and\ncontent. We show that the consideration of this polarity-based emotion clusters\nand emotional mimicry results in improved empathy and contextual relevance of\nthe response as compared to the state-of-the-art. Also, we introduce\nstochasticity into the emotion mixture that yields emotionally more varied\nempathetic responses than the previous work. We demonstrate the importance of\nthese factors to empathetic response generation using both automatic- and\nhuman-based evaluations. The implementation of MIME is publicly available at\nhttps://github.com/declare-lab/MIME.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01461v1",
            "date": "2020-10-04 01:23:17+00:00",
            "title": "Sentence Constituent-Aware Aspect-Category Sentiment Analysis with Graph Attention Networks",
            "authors": [
                "Yuncong Li",
                "Cunxiang Yin",
                "Sheng-hua Zhong"
            ],
            "abstract": "Aspect category sentiment analysis (ACSA) aims to predict the sentiment\npolarities of the aspect categories discussed in sentences. Since a sentence\nusually discusses one or more aspect categories and expresses different\nsentiments toward them, various attention-based methods have been developed to\nallocate the appropriate sentiment words for the given aspect category and\nobtain promising results. However, most of these methods directly use the given\naspect category to find the aspect category-related sentiment words, which may\ncause mismatching between the sentiment words and the aspect categories when an\nunrelated sentiment word is semantically meaningful for the given aspect\ncategory. To mitigate this problem, we propose a Sentence Constituent-Aware\nNetwork (SCAN) for aspect-category sentiment analysis. SCAN contains two graph\nattention modules and an interactive loss function. The graph attention modules\ngenerate representations of the nodes in sentence constituency parse trees for\nthe aspect category detection (ACD) task and the ACSA task, respectively. ACD\naims to detect aspect categories discussed in sentences and is a auxiliary\ntask. For a given aspect category, the interactive loss function helps the ACD\ntask to find the nodes which can predict the aspect category but can't predict\nother aspect categories. The sentiment words in the nodes then are used to\npredict the sentiment polarity of the aspect category by the ACSA task. The\nexperimental results on five public datasets demonstrate the effectiveness of\nSCAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01475v1",
            "date": "2020-10-04 03:13:46+00:00",
            "title": "Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space",
            "authors": [
                "Dayiheng Liu",
                "Yeyun Gong",
                "Jie Fu",
                "Yu Yan",
                "Jiusheng Chen",
                "Jiancheng Lv",
                "Nan Duan",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we propose a novel data augmentation method, referred to as\nControllable Rewriting based Question Data Augmentation (CRQDA), for machine\nreading comprehension (MRC), question generation, and question-answering\nnatural language inference tasks. We treat the question data augmentation task\nas a constrained question rewriting problem to generate context-relevant,\nhigh-quality, and diverse question data samples. CRQDA utilizes a Transformer\nautoencoder to map the original discrete question into a continuous embedding\nspace. It then uses a pre-trained MRC model to revise the question\nrepresentation iteratively with gradient-based optimization. Finally, the\nrevised question representations are mapped back into the discrete space, which\nserve as additional question data. Comprehensive experiments on SQuAD 2.0,\nSQuAD 1.1 question generation, and QNLI tasks demonstrate the effectiveness of\nCRQDA",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01502v1",
            "date": "2020-10-04 08:00:19+00:00",
            "title": "Multi-turn Response Selection using Dialogue Dependency Relations",
            "authors": [
                "Qi Jia",
                "Yizhu Liu",
                "Siyu Ren",
                "Kenny Q. Zhu",
                "Haifeng Tang"
            ],
            "abstract": "Multi-turn response selection is a task designed for developing dialogue\nagents. The performance on this task has a remarkable improvement with\npre-trained language models. However, these models simply concatenate the turns\nin dialogue history as the input and largely ignore the dependencies between\nthe turns. In this paper, we propose a dialogue extraction algorithm to\ntransform a dialogue history into threads based on their dependency relations.\nEach thread can be regarded as a self-contained sub-dialogue. We also propose\nThread-Encoder model to encode threads and candidates into compact\nrepresentations by pre-trained Transformers and finally get the matching score\nthrough an attention layer. The experiments show that dependency relations are\nhelpful for dialogue context understanding, and our model outperforms the\nstate-of-the-art baselines on both DSTC7 and DSTC8*, with competitive results\non UbuntuV2.",
            "journal ref": "Proceedings of the 2020 Conference on Empirical Methods in Natural\n  Language Processing (EMNLP)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01512v2",
            "date": "2020-10-04 08:31:54+00:00",
            "title": "A Multi-task Learning Framework for Opinion Triplet Extraction",
            "authors": [
                "Chen Zhang",
                "Qiuchi Li",
                "Dawei Song",
                "Benyou Wang"
            ],
            "abstract": "The state-of-the-art Aspect-based Sentiment Analysis (ABSA) approaches are\nmainly based on either detecting aspect terms and their corresponding sentiment\npolarities, or co-extracting aspect and opinion terms. However, the extraction\nof aspect-sentiment pairs lacks opinion terms as a reference, while\nco-extraction of aspect and opinion terms would not lead to meaningful pairs\nwithout determining their sentiment dependencies. To address the issue, we\npresent a novel view of ABSA as an opinion triplet extraction task, and propose\na multi-task learning framework to jointly extract aspect terms and opinion\nterms, and simultaneously parses sentiment dependencies between them with a\nbiaffine scorer. At inference phase, the extraction of triplets is facilitated\nby a triplet decoding method based on the above outputs. We evaluate the\nproposed framework on four SemEval benchmarks for ASBA. The results demonstrate\nthat our approach significantly outperforms a range of strong baselines and\nstate-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01535v1",
            "date": "2020-10-04 10:51:22+00:00",
            "title": "A Survey of Unsupervised Dependency Parsing",
            "authors": [
                "Wenjuan Han",
                "Yong Jiang",
                "Hwee Tou Ng",
                "Kewei Tu"
            ],
            "abstract": "Syntactic dependency parsing is an important task in natural language\nprocessing. Unsupervised dependency parsing aims to learn a dependency parser\nfrom sentences that have no annotation of their correct parse trees. Despite\nits difficulty, unsupervised parsing is an interesting research direction\nbecause of its capability of utilizing almost unlimited unannotated text data.\nIt also serves as the basis for other research in low-resource parsing. In this\npaper, we survey existing approaches to unsupervised dependency parsing,\nidentify two major classes of approaches, and discuss recent trends. We hope\nthat our survey can provide insights for researchers and facilitate future\nresearch on this topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01554v1",
            "date": "2020-10-04 11:52:50+00:00",
            "title": "Leveraging Multilingual News Websites for Building a Kurdish Parallel Corpus",
            "authors": [
                "Sina Ahmadi",
                "Hossein Hassani",
                "Daban Q. Jaff"
            ],
            "abstract": "Machine translation has been a major motivation of development in natural\nlanguage processing. Despite the burgeoning achievements in creating more\nefficient machine translation systems thanks to deep learning methods, parallel\ncorpora have remained indispensable for progress in the field. In an attempt to\ncreate parallel corpora for the Kurdish language, in this paper, we describe\nour approach in retrieving potentially-alignable news articles from\nmulti-language websites and manually align them across dialects and languages\nbased on lexical similarity and transliteration of scripts. We present a corpus\ncontaining 12,327 translation pairs in the two major dialects of Kurdish,\nSorani and Kurmanji. We also provide 1,797 and 650 translation pairs in\nEnglish-Kurmanji and English-Sorani. The corpus is publicly available under the\nCC BY-NC-SA 4.0 license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01556v1",
            "date": "2020-10-04 11:59:59+00:00",
            "title": "Reverse Operation based Data Augmentation for Solving Math Word Problems",
            "authors": [
                "Qianying Liu",
                "Wenyu Guan",
                "Sujian Li",
                "Fei Cheng",
                "Daisuke Kawahara",
                "Sadao Kurohashi"
            ],
            "abstract": "Automatically solving math word problems is a critical task in the field of\nnatural language processing. Recent models have reached their performance\nbottleneck and require more high-quality data for training. Inspired by human\ndouble-checking mechanism, we propose a reverse operation based data\naugmentation method that makes use of mathematical logic to produce new\nhigh-quality math problems and introduce new knowledge points that can give\nsupervision for new mathematical reasoning logic. We apply the augmented data\non two SOTA math word problem solving models. Experimental results show the\neffectiveness of our approach\\footnote{We will release our code and data after\nthe paper is accepted.}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01610v2",
            "date": "2020-10-04 15:54:03+00:00",
            "title": "Adversarial Attack and Defense of Structured Prediction Models",
            "authors": [
                "Wenjuan Han",
                "Liwen Zhang",
                "Yong Jiang",
                "Kewei Tu"
            ],
            "abstract": "Building an effective adversarial attacker and elaborating on countermeasures\nfor adversarial attacks for natural language processing (NLP) have attracted a\nlot of research in recent years. However, most of the existing approaches focus\non classification problems. In this paper, we investigate attacks and defenses\nfor structured prediction tasks in NLP. Besides the difficulty of perturbing\ndiscrete words and the sentence fluency problem faced by attackers in any NLP\ntasks, there is a specific challenge to attackers of structured prediction\nmodels: the structured output of structured prediction models is sensitive to\nsmall perturbations in the input. To address these problems, we propose a novel\nand unified framework that learns to attack a structured prediction model using\na sequence-to-sequence model with feedbacks from multiple reference models of\nthe same structured prediction task. Based on the proposed attack, we further\nreinforce the victim model with adversarial training, making its prediction\nmore robust and accurate. We evaluate the proposed framework in dependency\nparsing and part-of-speech tagging. Automatic and human evaluations show that\nour proposed framework succeeds in both attacking state-of-the-art structured\nprediction models and boosting them with adversarial training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01620v1",
            "date": "2020-10-04 16:28:13+00:00",
            "title": "Meta Sequence Learning and Its Applications",
            "authors": [
                "Cheng Zhang",
                "Jie Wang"
            ],
            "abstract": "We present a meta-sequence representation of sentences and demonstrate how to\nuse meta sequence learning to generate adequate question-answer pairs (QAPs)\nover a given article. A meta sequence is a sequence of vectors of semantic and\nsyntactic tags. On a given declarative sentence, a trained model converts it to\na meta sequence, finds a matched meta sequence in its learned database, and\nuses the corresponding meta sequence for interrogative sentence to generate\nQAPs. We show that, trained on a small dataset, our method generates\nefficiently, on the official SAT practice reading tests, a large number of\nsyntactically and semantically correct QAPs with high accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01653v1",
            "date": "2020-10-04 18:55:47+00:00",
            "title": "An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels",
            "authors": [
                "Ilias Chalkidis",
                "Manos Fergadiotis",
                "Sotiris Kotitsas",
                "Prodromos Malakasiotis",
                "Nikolaos Aletras",
                "Ion Androutsopoulos"
            ],
            "abstract": "Large-scale Multi-label Text Classification (LMTC) has a wide range of\nNatural Language Processing (NLP) applications and presents interesting\nchallenges. First, not all labels are well represented in the training set, due\nto the very large label set and the skewed label distributions of LMTC\ndatasets. Also, label hierarchies and differences in human labelling guidelines\nmay affect graph-aware annotation proximity. Finally, the label hierarchies are\nperiodically updated, requiring LMTC models capable of zero-shot\ngeneralization. Current state-of-the-art LMTC models employ Label-Wise\nAttention Networks (LWANs), which (1) typically treat LMTC as flat multi-label\nclassification; (2) may use the label hierarchy to improve zero-shot learning,\nalthough this practice is vastly understudied; and (3) have not been combined\nwith pre-trained Transformers (e.g. BERT), which have led to state-of-the-art\nresults in several NLP benchmarks. Here, for the first time, we empirically\nevaluate a battery of LMTC methods from vanilla LWANs to hierarchical\nclassification approaches and transfer learning, on frequent, few, and\nzero-shot learning on three datasets from different domains. We show that\nhierarchical methods based on Probabilistic Label Trees (PLTs) outperform\nLWANs. Furthermore, we show that Transformer-based approaches outperform the\nstate-of-the-art in two of the datasets, and we propose a new state-of-the-art\nmethod which combines BERT with LWANs. Finally, we propose new models that\nleverage the label hierarchy to improve few and zero-shot learning, considering\non each dataset a graph-aware annotation proximity measure that we introduce.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01657v1",
            "date": "2020-10-04 19:03:39+00:00",
            "title": "Inquisitive Question Generation for High Level Text Comprehension",
            "authors": [
                "Wei-Jen Ko",
                "Te-Yuan Chen",
                "Yiyan Huang",
                "Greg Durrett",
                "Junyi Jessy Li"
            ],
            "abstract": "Inquisitive probing questions come naturally to humans in a variety of\nsettings, but is a challenging task for automatic systems. One natural type of\nquestion to ask tries to fill a gap in knowledge during text comprehension,\nlike reading a news article: we might ask about background information, deeper\nreasons behind things occurring, or more. Despite recent progress with\ndata-driven approaches, generating such questions is beyond the range of models\ntrained on existing datasets.\n  We introduce INQUISITIVE, a dataset of ~19K questions that are elicited while\na person is reading through a document. Compared to existing datasets,\nINQUISITIVE questions target more towards high-level (semantic and discourse)\ncomprehension of text. We show that readers engage in a series of pragmatic\nstrategies to seek information. Finally, we evaluate question generation models\nbased on GPT-2 and show that our model is able to generate reasonable questions\nalthough the task is challenging, and highlight the importance of context to\ngenerate INQUISITIVE questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01658v1",
            "date": "2020-10-04 19:06:16+00:00",
            "title": "Generating Dialogue Responses from a Semantic Latent Space",
            "authors": [
                "Wei-Jen Ko",
                "Avik Ray",
                "Yilin Shen",
                "Hongxia Jin"
            ],
            "abstract": "Existing open-domain dialogue generation models are usually trained to mimic\nthe gold response in the training set using cross-entropy loss on the\nvocabulary. However, a good response does not need to resemble the gold\nresponse, since there are multiple possible responses to a given prompt. In\nthis work, we hypothesize that the current models are unable to integrate\ninformation from multiple semantically similar valid responses of a prompt,\nresulting in the generation of generic and uninformative responses. To address\nthis issue, we propose an alternative to the end-to-end classification on\nvocabulary. We learn the pair relationship between the prompts and responses as\na regression task on a latent space instead. In our novel dialog generation\nmodel, the representations of semantically related sentences are close to each\nother on the latent space. Human evaluation showed that learning the task on a\ncontinuous space can generate responses that are both relevant and informative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01667v1",
            "date": "2020-10-04 19:42:40+00:00",
            "title": "Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation",
            "authors": [
                "Luyu Gao",
                "Xinyi Wang",
                "Graham Neubig"
            ],
            "abstract": "To improve the performance of Neural Machine Translation~(NMT) for\nlow-resource languages~(LRL), one effective strategy is to leverage parallel\ndata from a related high-resource language~(HRL). However, multilingual data\nhas been found more beneficial for NMT models that translate from the LRL to a\ntarget language than the ones that translate into the LRLs. In this paper, we\naim to improve the effectiveness of multilingual transfer for NMT models that\ntranslate \\emph{into} the LRL, by designing a better decoder word embedding.\nExtending upon a general-purpose multilingual encoding method Soft Decoupled\nEncoding~\\citep{SDE}, we propose DecSDE, an efficient character n-gram based\nembedding specifically designed for the NMT decoder. Our experiments show that\nDecSDE leads to consistent gains of up to 1.8 BLEU on translation from English\nto four different languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01672v1",
            "date": "2020-10-04 20:12:44+00:00",
            "title": "Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization",
            "authors": [
                "Jiaao Chen",
                "Diyi Yang"
            ],
            "abstract": "Text summarization is one of the most challenging and interesting problems in\nNLP. Although much attention has been paid to summarizing structured text like\nnews reports or encyclopedia articles, summarizing conversations---an essential\npart of human-human/machine interaction where most important pieces of\ninformation are scattered across various utterances of different\nspeakers---remains relatively under-investigated. This work proposes a\nmulti-view sequence-to-sequence model by first extracting conversational\nstructures of unstructured daily chats from different views to represent\nconversations and then utilizing a multi-view decoder to incorporate different\nviews to generate dialogue summaries. Experiments on a large-scale dialogue\nsummarization corpus demonstrated that our methods significantly outperformed\nprevious state-of-the-art models via both automatic evaluations and human\njudgment. We also discussed specific challenges that current approaches faced\nwith this task. We have publicly released our code at\nhttps://github.com/GT-SALT/Multi-View-Seq2Seq.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01677v1",
            "date": "2020-10-04 20:46:26+00:00",
            "title": "Local Additivity Based Data Augmentation for Semi-supervised NER",
            "authors": [
                "Jiaao Chen",
                "Zhenghui Wang",
                "Ran Tian",
                "Zichao Yang",
                "Diyi Yang"
            ],
            "abstract": "Named Entity Recognition (NER) is one of the first stages in deep language\nunderstanding yet current NER models heavily rely on human-annotated data. In\nthis work, to alleviate the dependence on labeled data, we propose a Local\nAdditivity based Data Augmentation (LADA) method for semi-supervised NER, in\nwhich we create virtual samples by interpolating sequences close to each other.\nOur approach has two variations: Intra-LADA and Inter-LADA, where Intra-LADA\nperforms interpolations among tokens within one sentence, and Inter-LADA\nsamples different sentences to interpolate. Through linear additions between\nsampled training data, LADA creates an infinite amount of labeled data and\nimproves both entity and context learning. We further extend LADA to the\nsemi-supervised setting by designing a novel consistency loss for unlabeled\ndata. Experiments conducted on two NER benchmarks demonstrate the effectiveness\nof our methods over several strong baselines. We have publicly released our\ncode at https://github.com/GT-SALT/LADA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01683v1",
            "date": "2020-10-04 21:06:45+00:00",
            "title": "Weakly-supervised Fine-grained Event Recognition on Social Media Texts for Disaster Management",
            "authors": [
                "Wenlin Yao",
                "Cheng Zhang",
                "Shiva Saravanan",
                "Ruihong Huang",
                "Ali Mostafavi"
            ],
            "abstract": "People increasingly use social media to report emergencies, seek help or\nshare information during disasters, which makes social networks an important\ntool for disaster management. To meet these time-critical needs, we present a\nweakly supervised approach for rapidly building high-quality classifiers that\nlabel each individual Twitter message with fine-grained event categories. Most\nimportantly, we propose a novel method to create high-quality labeled data in a\ntimely manner that automatically clusters tweets containing an event keyword\nand asks a domain expert to disambiguate event word senses and label clusters\nquickly. In addition, to process extremely noisy and often rather short\nuser-generated messages, we enrich tweet representations using preceding\ncontext tweets and reply tweets in building event recognition classifiers. The\nevaluation on two hurricanes, Harvey and Florence, shows that using only 1-2\nperson-hours of human supervision, the rapidly trained weakly supervised\nclassifiers outperform supervised classifiers trained using more than ten\nthousand annotated tweets created in over 50 person-hours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01737v1",
            "date": "2020-10-05 01:33:58+00:00",
            "title": "Transformer-Based Neural Text Generation with Syntactic Guidance",
            "authors": [
                "Yinghao Li",
                "Rui Feng",
                "Isaac Rehg",
                "Chao Zhang"
            ],
            "abstract": "We study the problem of using (partial) constituency parse trees as syntactic\nguidance for controlled text generation. Existing approaches to this problem\nuse recurrent structures, which not only suffer from the long-term dependency\nproblem but also falls short in modeling the tree structure of the syntactic\nguidance. We propose to leverage the parallelism of Transformer to better\nincorporate parse trees. Our method first expands a partial template\nconstituency parse tree to a full-fledged parse tree tailored for the input\nsource text, and then uses the expanded tree to guide text generation. The\neffectiveness of our model in this process hinges upon two new attention\nmechanisms: 1) a path attention mechanism that forces one node to attend to\nonly other nodes located in its path in the syntax tree to better incorporate\nsyntax guidance; 2) a multi-encoder attention mechanism that allows the decoder\nto dynamically attend to information from multiple encoders. Our experiments in\nthe controlled paraphrasing task show that our method outperforms SOTA models\nboth semantically and syntactically, improving the best baseline's BLEU score\nfrom 11.83 to 26.27.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01739v1",
            "date": "2020-10-05 01:49:47+00:00",
            "title": "Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models",
            "authors": [
                "Thuy-Trang Vu",
                "Dinh Phung",
                "Gholamreza Haffari"
            ],
            "abstract": "Recent work has shown the importance of adaptation of broad-coverage\ncontextualised embedding models on the domain of the target task of interest.\nCurrent self-supervised adaptation methods are simplistic, as the training\nsignal comes from a small percentage of \\emph{randomly} masked-out tokens. In\nthis paper, we show that careful masking strategies can bridge the knowledge\ngap of masked language models (MLMs) about the domains more effectively by\nallocating self-supervision where it is needed. Furthermore, we propose an\neffective training strategy by adversarially masking out those tokens which are\nharder to reconstruct by the underlying MLM. The adversarial objective leads to\na challenging combinatorial optimisation problem over \\emph{subsets} of tokens,\nwhich we tackle efficiently through relaxation to a variational lowerbound and\ndynamic programming. On six unsupervised domain adaptation tasks involving\nnamed entity recognition, our method strongly outperforms the random masking\nstrategy and achieves up to +1.64 F1 score improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01770v2",
            "date": "2020-10-05 04:32:38+00:00",
            "title": "Second-Order NLP Adversarial Examples",
            "authors": [
                "John X. Morris"
            ],
            "abstract": "Adversarial example generation methods in NLP rely on models like language\nmodels or sentence encoders to determine if potential adversarial examples are\nvalid. In these methods, a valid adversarial example fools the model being\nattacked, and is determined to be semantically or syntactically valid by a\nsecond model. Research to date has counted all such examples as errors by the\nattacked model. We contend that these adversarial examples may not be flaws in\nthe attacked model, but flaws in the model that determines validity. We term\nsuch invalid inputs second-order adversarial examples. We propose the\nconstraint robustness curve and associated metric ACCS as tools for evaluating\nthe robustness of a constraint to second-order adversarial examples. To\ngenerate this curve, we design an adversarial attack to run directly on the\nsemantic similarity models. We test on two constraints, the Universal Sentence\nEncoder (USE) and BERTScore. Our findings indicate that such second-order\nexamples exist, but are typically less common than first-order adversarial\nexamples in state-of-the-art models. They also indicate that USE is effective\nas constraint on NLP adversarial examples, while BERTScore is nearly\nineffectual. Code for running the experiments in this paper is available at\nhttps://github.com/jxmorris12/second-order-adversarial-examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01771v1",
            "date": "2020-10-05 04:32:47+00:00",
            "title": "Improving AMR Parsing with Sequence-to-Sequence Pre-training",
            "authors": [
                "Dongqin Xu",
                "Junhui Li",
                "Muhua Zhu",
                "Min Zhang",
                "Guodong Zhou"
            ],
            "abstract": "In the literature, the research on abstract meaning representation (AMR)\nparsing is much restricted by the size of human-curated dataset which is\ncritical to build an AMR parser with good performance. To alleviate such data\nsize restriction, pre-trained models have been drawing more and more attention\nin AMR parsing. However, previous pre-trained models, like BERT, are\nimplemented for general purpose which may not work as expected for the specific\ntask of AMR parsing. In this paper, we focus on sequence-to-sequence (seq2seq)\nAMR parsing and propose a seq2seq pre-training approach to build pre-trained\nmodels in both single and joint way on three relevant tasks, i.e., machine\ntranslation, syntactic parsing, and AMR parsing itself. Moreover, we extend the\nvanilla fine-tuning method to a multi-task learning fine-tuning method that\noptimizes for the performance of AMR parsing while endeavors to preserve the\nresponse of pre-trained models. Extensive experimental results on two English\nbenchmark datasets show that both the single and joint pre-trained models\nsignificantly improve the performance (e.g., from 71.5 to 80.2 on AMR 2.0),\nwhich reaches the state of the art. The result is very encouraging since we\nachieve this with seq2seq models rather than complex models. We make our code\nand model available at https://github.com/xdqkid/S2S-AMR-Parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01781v1",
            "date": "2020-10-05 05:04:14+00:00",
            "title": "Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning",
            "authors": [
                "Hanlu Wu",
                "Tengfei Ma",
                "Lingfei Wu",
                "Tariro Manyumwa",
                "Shouling Ji"
            ],
            "abstract": "Evaluation of a document summarization system has been a critical factor to\nimpact the success of the summarization task. Previous approaches, such as\nROUGE, mainly consider the informativeness of the assessed summary and require\nhuman-generated references for each test summary. In this work, we propose to\nevaluate the summary qualities without reference summaries by unsupervised\ncontrastive learning. Specifically, we design a new metric which covers both\nlinguistic qualities and semantic informativeness based on BERT. To learn the\nmetric, for each summary, we construct different types of negative samples with\nrespect to different aspects of the summary qualities, and train our model with\na ranking loss. Experiments on Newsroom and CNN/Daily Mail demonstrate that our\nnew evaluation method outperforms other metrics even without reference\nsummaries. Furthermore, we show that our method is general and transferable\nacross datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01791v1",
            "date": "2020-10-05 05:40:56+00:00",
            "title": "Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior",
            "authors": [
                "Zi Lin",
                "Jeremiah Zhe Liu",
                "Zi Yang",
                "Nan Hua",
                "Dan Roth"
            ],
            "abstract": "Traditional (unstructured) pruning methods for a Transformer model focus on\nregularizing the individual weights by penalizing them toward zero. In this\nwork, we explore spectral-normalized identity priors (SNIP), a structured\npruning approach that penalizes an entire residual module in a Transformer\nmodel toward an identity mapping. Our method identifies and discards\nunimportant non-linear mappings in the residual connections by applying a\nthresholding operator on the function norm. It is applicable to any structured\nmodule, including a single attention head, an entire attention block, or a\nfeed-forward subnetwork. Furthermore, we introduce spectral normalization to\nstabilize the distribution of the post-activation values of the Transformer\nlayers, further improving the pruning effectiveness of the proposed\nmethodology. We conduct experiments with BERT on 5 GLUE benchmark tasks to\ndemonstrate that SNIP achieves effective pruning results while maintaining\ncomparable performance. Specifically, we improve the performance over the\nstate-of-the-art by 0.5 to 1.0% on average at 50% compression ratio.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01830v3",
            "date": "2020-10-05 07:40:35+00:00",
            "title": "On the Frailty of Universal POS Tags for Neural UD Parsers",
            "authors": [
                "Mark Anderson",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present an analysis on the effect UPOS accuracy has on parsing\nperformance. Results suggest that leveraging UPOS tags as features for neural\nparsers requires a prohibitively high tagging accuracy and that the use of gold\ntags offers a non-linear increase in performance, suggesting some sort of\nexceptionality. We also investigate what aspects of predicted UPOS tags impact\nparsing accuracy the most, highlighting some potentially meaningful linguistic\nfacets of the problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01893v2",
            "date": "2020-10-05 10:10:19+00:00",
            "title": "Regularizing Dialogue Generation by Imitating Implicit Scenarios",
            "authors": [
                "Shaoxiong Feng",
                "Xuancheng Ren",
                "Hongshen Chen",
                "Bin Sun",
                "Kan Li",
                "Xu Sun"
            ],
            "abstract": "Human dialogues are scenario-based and appropriate responses generally relate\nto the latent context knowledge entailed by the specific scenario. To enable\nresponses that are more meaningful and context-specific, we propose to improve\ngenerative dialogue systems from the scenario perspective, where both dialogue\nhistory and future conversation are taken into account to implicitly\nreconstruct the scenario knowledge. More importantly, the conversation\nscenarios are further internalized using imitation learning framework, where\nthe conventional dialogue model that has no access to future conversations is\neffectively regularized by transferring the scenario knowledge contained in\nhierarchical supervising signals from the scenario-based dialogue model, so\nthat the future conversation is not required in actual inference. Extensive\nevaluations show that our approach significantly outperforms state-of-the-art\nbaselines on diversity and relevance, and expresses scenario-specific\nknowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01897v1",
            "date": "2020-10-05 10:25:29+00:00",
            "title": "PUM at SemEval-2020 Task 12: Aggregation of Transformer-based models' features for offensive language recognition",
            "authors": [
                "Piotr Janiszewski",
                "Mateusz Skiba",
                "Urszula Wali\u0144ska"
            ],
            "abstract": "In this paper, we describe the PUM team's entry to the SemEval-2020 Task 12.\nCreating our solution involved leveraging two well-known pretrained models used\nin natural language processing: BERT and XLNet, which achieve state-of-the-art\nresults in multiple NLP tasks. The models were fine-tuned for each subtask\nseparately and features taken from their hidden layers were combined and fed\ninto a fully connected neural network. The model using aggregated Transformer\nfeatures can serve as a powerful tool for offensive language identification\nproblem. Our team was ranked 7th out of 40 in Sub-task C - Offense target\nidentification with 64.727% macro F1-score and 64th out of 85 in Sub-task A -\nOffensive language identification (89.726% F1-score).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01899v1",
            "date": "2020-10-05 10:28:03+00:00",
            "title": "Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph",
            "authors": [
                "Xin Lv",
                "Xu Han",
                "Lei Hou",
                "Juanzi Li",
                "Zhiyuan Liu",
                "Wei Zhang",
                "Yichi Zhang",
                "Hao Kong",
                "Suhui Wu"
            ],
            "abstract": "Multi-hop reasoning has been widely studied in recent years to seek an\neffective and interpretable method for knowledge graph (KG) completion. Most\nprevious reasoning methods are designed for dense KGs with enough paths between\nentities, but cannot work well on those sparse KGs that only contain sparse\npaths for reasoning. On the one hand, sparse KGs contain less information,\nwhich makes it difficult for the model to choose correct paths. On the other\nhand, the lack of evidential paths to target entities also makes the reasoning\nprocess difficult. To solve these problems, we propose a multi-hop reasoning\nmodel named DacKGR over sparse KGs, by applying novel dynamic anticipation and\ncompletion strategies: (1) The anticipation strategy utilizes the latent\nprediction of embedding-based models to make our model perform more potential\npath search over sparse KGs. (2) Based on the anticipation information, the\ncompletion strategy dynamically adds edges as additional actions during the\npath search, which further alleviates the sparseness problem of KGs. The\nexperimental results on five datasets sampled from Freebase, NELL and Wikidata\nshow that our method outperforms state-of-the-art baselines. Our codes and\ndatasets can be obtained from https://github.com/THU-KEG/DacKGR",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01908v2",
            "date": "2020-10-02 13:28:47+00:00",
            "title": "Exploiting Unsupervised Data for Emotion Recognition in Conversations",
            "authors": [
                "Wenxiang Jiao",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Emotion Recognition in Conversations (ERC) aims to predict the emotional\nstate of speakers in conversations, which is essentially a text classification\ntask. Unlike the sentence-level text classification problem, the available\nsupervised data for the ERC task is limited, which potentially prevents the\nmodels from playing their maximum effect. In this paper, we propose a novel\napproach to leverage unsupervised conversation data, which is more accessible.\nSpecifically, we propose the Conversation Completion (ConvCom) task, which\nattempts to select the correct answer from candidate answers to fill a masked\nutterance in a conversation. Then, we Pre-train a basic COntext- Dependent\nEncoder (Pre-CODE) on the ConvCom task. Finally, we fine-tune the Pre-CODE on\nthe datasets of ERC. Experimental results demonstrate that pre-training on\nunsupervised data achieves significant improvement of performance on the ERC\ndatasets, particularly on the minority emotion classes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01923v2",
            "date": "2020-10-05 11:21:59+00:00",
            "title": "Learning from Context or Names? An Empirical Study on Neural Relation Extraction",
            "authors": [
                "Hao Peng",
                "Tianyu Gao",
                "Xu Han",
                "Yankai Lin",
                "Peng Li",
                "Zhiyuan Liu",
                "Maosong Sun",
                "Jie Zhou"
            ],
            "abstract": "Neural models have achieved remarkable success on relation extraction (RE)\nbenchmarks. However, there is no clear understanding which type of information\naffects existing RE models to make decisions and how to further improve the\nperformance of these models. To this end, we empirically study the effect of\ntwo main information sources in text: textual context and entity mentions\n(names). We find that (i) while context is the main source to support the\npredictions, RE models also heavily rely on the information from entity\nmentions, most of which is type information, and (ii) existing datasets may\nleak shallow heuristics via entity mentions and thus contribute to the high\nperformance on RE benchmarks. Based on the analyses, we propose an\nentity-masked contrastive pre-training framework for RE to gain a deeper\nunderstanding on both textual context and type information while avoiding rote\nmemorization of entities or use of superficial cues in mentions. We carry out\nextensive experiments to support our views, and show that our framework can\nimprove the effectiveness and robustness of neural models in different RE\nscenarios. All the code and datasets are released at\nhttps://github.com/thunlp/RE-Context-or-Names.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01998v1",
            "date": "2020-10-05 13:34:20+00:00",
            "title": "X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset",
            "authors": [
                "Angel Daza",
                "Anette Frank"
            ],
            "abstract": "Even though SRL is researched for many languages, major improvements have\nmostly been obtained for English, for which more resources are available. In\nfact, existing multilingual SRL datasets contain disparate annotation styles or\ncome from different domains, hampering generalization in multilingual learning.\nIn this work, we propose a method to automatically construct an SRL corpus that\nis parallel in four languages: English, French, German, Spanish, with unified\npredicate and role annotations that are fully comparable across languages. We\napply high-quality machine translation to the English CoNLL-09 dataset and use\nmultilingual BERT to project its high-quality annotations to the target\nlanguages. We include human-validated test sets that we use to measure the\nprojection quality, and show that projection is denser and more precise than a\nstrong baseline. Finally, we train different SOTA models on our novel corpus\nfor mono- and multilingual SRL, showing that the multilingual annotations\nimprove performance especially for the weaker languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02053v1",
            "date": "2020-10-05 14:42:56+00:00",
            "title": "A Fully Hyperbolic Neural Model for Hierarchical Multi-Class Classification",
            "authors": [
                "Federico L\u00f3pez",
                "Michael Strube"
            ],
            "abstract": "Label inventories for fine-grained entity typing have grown in size and\ncomplexity. Nonetheless, they exhibit a hierarchical structure. Hyperbolic\nspaces offer a mathematically appealing approach for learning hierarchical\nrepresentations of symbolic data. However, it is not clear how to integrate\nhyperbolic components into downstream tasks. This is the first work that\nproposes a fully hyperbolic model for multi-class multi-label classification,\nwhich performs all operations in hyperbolic space. We evaluate the proposed\nmodel on two challenging datasets and compare to different baselines that\noperate under Euclidean assumptions. Our hyperbolic model infers the latent\nhierarchy from the class distribution, captures implicit hyponymic relations in\nthe inventory, and shows performance on par with state-of-the-art methods on\nfine-grained classification with remarkable reduction of the parameter size. A\nthorough analysis sheds light on the impact of each component in the final\nprediction and showcases its ease of integration with Euclidean layers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02094v2",
            "date": "2020-10-05 15:25:47+00:00",
            "title": "Gauravarora@HASOC-Dravidian-CodeMix-FIRE2020: Pre-training ULMFiT on Synthetically Generated Code-Mixed Data for Hate Speech Detection",
            "authors": [
                "Gaurav Arora"
            ],
            "abstract": "This paper describes the system submitted to Dravidian-Codemix-HASOC2020:\nHate Speech and Offensive Content Identification in Dravidian languages\n(Tamil-English and Malayalam-English). The task aims to identify offensive\nlanguage in code-mixed dataset of comments/posts in Dravidian languages\ncollected from social media. We participated in both Sub-task A, which aims to\nidentify offensive content in mixed-script (mixture of Native and Roman script)\nand Sub-task B, which aims to identify offensive content in Roman script, for\nDravidian languages. In order to address these tasks, we proposed pre-training\nULMFiT on synthetically generated code-mixed data, generated by modelling\ncode-mixed data generation as a Markov process using Markov chains. Our model\nachieved 0.88 weighted F1-score for code-mixed Tamil-English language in\nSub-task B and got 2nd rank on the leader-board. Additionally, our model\nachieved 0.91 weighted F1-score (4th Rank) for mixed-script Malayalam-English\nin Sub-task A and 0.74 weighted F1-score (5th Rank) for code-mixed\nMalayalam-English language in Sub-task B.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02142v2",
            "date": "2020-10-05 16:45:30+00:00",
            "title": "PublishInCovid19 at WNUT 2020 Shared Task-1: Entity Recognition in Wet Lab Protocols using Structured Learning Ensemble and Contextualised Embeddings",
            "authors": [
                "Janvijay Singh",
                "Anshul Wadhawan"
            ],
            "abstract": "In this paper, we describe the approach that we employed to address the task\nof Entity Recognition over Wet Lab Protocols -- a shared task in EMNLP\nWNUT-2020 Workshop. Our approach is composed of two phases. In the first phase,\nwe experiment with various contextualised word embeddings (like Flair,\nBERT-based) and a BiLSTM-CRF model to arrive at the best-performing\narchitecture. In the second phase, we create an ensemble composed of eleven\nBiLSTM-CRF models. The individual models are trained on random train-validation\nsplits of the complete dataset. Here, we also experiment with different output\nmerging schemes, including Majority Voting and Structured Learning Ensembling\n(SLE). Our final submission achieved a micro F1-score of 0.8175 and 0.7757 for\nthe partial and exact match of the entity spans, respectively. We were ranked\nfirst and second, in terms of partial and exact match, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02172v3",
            "date": "2020-10-05 17:19:10+00:00",
            "title": "Speakers Fill Lexical Semantic Gaps with Context",
            "authors": [
                "Tiago Pimentel",
                "Rowan Hall Maudslay",
                "Dami\u00e1n Blasi",
                "Ryan Cotterell"
            ],
            "abstract": "Lexical ambiguity is widespread in language, allowing for the reuse of\neconomical word forms and therefore making language more efficient. If\nambiguous words cannot be disambiguated from context, however, this gain in\nefficiency might make language less clear -- resulting in frequent\nmiscommunication. For a language to be clear and efficiently encoded, we posit\nthat the lexical ambiguity of a word type should correlate with how much\ninformation context provides about it, on average. To investigate whether this\nis the case, we operationalise the lexical ambiguity of a word as the entropy\nof meanings it can take, and provide two ways to estimate this -- one which\nrequires human annotation (using WordNet), and one which does not (using BERT),\nmaking it readily applicable to a large number of languages. We validate these\nmeasures by showing that, on six high-resource languages, there are significant\nPearson correlations between our BERT-based estimate of ambiguity and the\nnumber of synonyms a word has in WordNet (e.g. $\\rho = 0.40$ in English). We\nthen test our main hypothesis -- that a word's lexical ambiguity should\nnegatively correlate with its contextual uncertainty -- and find significant\ncorrelations on all 18 typologically diverse languages we analyse. This\nsuggests that, in the presence of ambiguity, speakers compensate by making\ncontexts more informative.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02179v1",
            "date": "2020-10-05 17:24:57+00:00",
            "title": "Assessing the Helpfulness of Learning Materials with Inference-Based Learner-Like Agent",
            "authors": [
                "Yun-Hsuan Jen",
                "Chieh-Yang Huang",
                "Mei-Hua Chen",
                "Ting-Hao 'Kenneth' Huang",
                "Lun-Wei Ku"
            ],
            "abstract": "Many English-as-a-second language learners have trouble using near-synonym\nwords (e.g., small vs.little; briefly vs.shortly) correctly, and often look for\nexample sentences to learn how two nearly synonymous terms differ. Prior work\nuses hand-crafted scores to recommend sentences but has difficulty in adopting\nsuch scores to all the near-synonyms as near-synonyms differ in various ways.\nWe notice that the helpfulness of the learning material would reflect on the\nlearners' performance. Thus, we propose the inference-based learner-like agent\nto mimic learner behavior and identify good learning materials by examining the\nagent's performance. To enable the agent to behave like a learner, we leverage\nentailment modeling's capability of inferring answers from the provided\nmaterials. Experimental results show that the proposed agent is equipped with\ngood learner-like behavior to achieve the best performance in both\nfill-in-the-blank (FITB) and good example sentence selection tasks. We further\nconduct a classroom user study with college ESL learners. The results of the\nuser study show that the proposed agent can find out example sentences that\nhelp students learn more easily and efficiently. Compared to other models, the\nproposed agent improves the score of more than 17% of students after learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02194v1",
            "date": "2020-10-05 17:52:25+00:00",
            "title": "Self-training Improves Pre-training for Natural Language Understanding",
            "authors": [
                "Jingfei Du",
                "Edouard Grave",
                "Beliz Gunel",
                "Vishrav Chaudhary",
                "Onur Celebi",
                "Michael Auli",
                "Ves Stoyanov",
                "Alexis Conneau"
            ],
            "abstract": "Unsupervised pre-training has led to much recent progress in natural language\nunderstanding. In this paper, we study self-training as another way to leverage\nunlabeled data through semi-supervised learning. To obtain additional data for\na specific task, we introduce SentAugment, a data augmentation method which\ncomputes task-specific query embeddings from labeled data to retrieve sentences\nfrom a bank of billions of unlabeled sentences crawled from the web. Unlike\nprevious semi-supervised methods, our approach does not require in-domain\nunlabeled data and is therefore more generally applicable. Experiments show\nthat self-training is complementary to strong RoBERTa baselines on a variety of\ntasks. Our augmentation approach leads to scalable and effective self-training\nwith improvements of up to 2.6% on standard text classification benchmarks.\nFinally, we also show strong gains on knowledge-distillation and few-shot\nlearning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02239v1",
            "date": "2020-10-05 18:00:15+00:00",
            "title": "Acrostic Poem Generation",
            "authors": [
                "Rajat Agarwal",
                "Katharina Kann"
            ],
            "abstract": "We propose a new task in the area of computational creativity: acrostic poem\ngeneration in English. Acrostic poems are poems that contain a hidden message;\ntypically, the first letter of each line spells out a word or short phrase. We\ndefine the task as a generation task with multiple constraints: given an input\nword, 1) the initial letters of each line should spell out the provided word,\n2) the poem's semantics should also relate to it, and 3) the poem should\nconform to a rhyming scheme. We further provide a baseline model for the task,\nwhich consists of a conditional neural language model in combination with a\nneural rhyming model. Since no dedicated datasets for acrostic poem generation\nexist, we create training data for our task by first training a separate topic\nprediction model on a small set of topic-annotated poems and then predicting\ntopics for additional poems. Our experiments show that the acrostic poems\ngenerated by our baseline are received well by humans and do not lose much\nquality due to the additional constraints. Last, we confirm that poems\ngenerated by our model are indeed closely related to the provided prompts, and\nthat pretraining on Wikipedia can boost performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02295v3",
            "date": "2020-10-05 19:29:49+00:00",
            "title": "SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding",
            "authors": [
                "Yu-An Chung",
                "Chenguang Zhu",
                "Michael Zeng"
            ],
            "abstract": "Spoken language understanding (SLU) requires a model to analyze input\nacoustic signal to understand its linguistic content and make predictions. To\nboost the models' performance, various pre-training methods have been proposed\nto learn rich representations from large-scale unannotated speech and text.\nHowever, the inherent disparities between the two modalities necessitate a\nmutual analysis. In this paper, we propose a novel semi-supervised learning\nframework, SPLAT, to jointly pre-train the speech and language modules. Besides\nconducting a self-supervised masked language modeling task on the two\nindividual modules using unpaired speech and text, SPLAT aligns representations\nfrom the two modules in a shared latent space using a small amount of paired\nspeech and text. Thus, during fine-tuning, the speech module alone can produce\nrepresentations carrying both acoustic information and contextual semantic\nknowledge of an input acoustic signal. Experimental results verify the\neffectiveness of our approach on various SLU tasks. For example, SPLAT improves\nthe previous state-of-the-art performance on the Spoken SQuAD dataset by more\nthan 10%.",
            "journal ref": "North American Chapter of the Association for Computational\n  Linguistics (NAACL), Mexico City, Mexico, 2021. North American Chapter of the\n  Association for Computational Linguistics (NAACL), Mexico City, Mexico, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02301v1",
            "date": "2020-10-05 19:45:03+00:00",
            "title": "PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation",
            "authors": [
                "Xinyu Hua",
                "Lu Wang"
            ],
            "abstract": "Pre-trained Transformers have enabled impressive breakthroughs in generating\nlong and fluent text, yet their outputs are often \"rambling\" without coherently\narranged content. In this work, we present a novel content-controlled text\ngeneration framework, PAIR, with planning and iterative refinement, which is\nbuilt upon a large model, BART. We first adapt the BERT model to automatically\nconstruct the content plans, consisting of keyphrase assignments and their\ncorresponding sentence-level positions. The BART model is employed for\ngeneration without modifying its structure. We then propose a refinement\nalgorithm to gradually enhance the generation quality within the\nsequence-to-sequence framework. Evaluation with automatic metrics shows that\nadding planning consistently improves the generation quality on three distinct\ndomains, with an average of 20 BLEU points and 12 METEOR points improvements.\nIn addition, human judges rate our system outputs to be more relevant and\ncoherent than comparisons without planning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02305v1",
            "date": "2020-10-05 19:53:41+00:00",
            "title": "Conversational Document Prediction to Assist Customer Care Agents",
            "authors": [
                "Jatin Ganhotra",
                "Haggai Roitman",
                "Doron Cohen",
                "Nathaniel Mills",
                "Chulaka Gunasekara",
                "Yosi Mass",
                "Sachindra Joshi",
                "Luis Lastras",
                "David Konopnicki"
            ],
            "abstract": "A frequent pattern in customer care conversations is the agents responding\nwith appropriate webpage URLs that address users' needs. We study the task of\npredicting the documents that customer care agents can use to facilitate users'\nneeds. We also introduce a new public dataset which supports the aforementioned\nproblem. Using this dataset and two others, we investigate state-of-the art\ndeep learning (DL) and information retrieval (IR) models for the task.\nAdditionally, we analyze the practicality of such systems in terms of inference\ntime complexity. Our show that an hybrid IR+DL approach provides the best of\nboth worlds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02316v1",
            "date": "2020-10-05 20:15:51+00:00",
            "title": "Sentiment Analysis for Reinforcement Learning",
            "authors": [
                "Ameet Deshpande",
                "Eve Fleisig"
            ],
            "abstract": "While reinforcement learning (RL) has been successful in natural language\nprocessing (NLP) domains such as dialogue generation and text-based games, it\ntypically faces the problem of sparse rewards that leads to slow or no\nconvergence. Traditional methods that use text descriptions to extract only a\nstate representation ignore the feedback inherently present in them. In\ntext-based games, for example, descriptions like \"Good Job! You ate the food}\"\nindicate progress, and descriptions like \"You entered a new room\" indicate\nexploration. Positive and negative cues like these can be converted to rewards\nthrough sentiment analysis. This technique converts the sparse reward problem\ninto a dense one, which is easier to solve. Furthermore, this can enable\nreinforcement learning without rewards, in which the agent learns entirely from\nthese intrinsic sentiment rewards. This framework is similar to intrinsic\nmotivation, where the environment does not necessarily provide the rewards, but\nthe agent analyzes and realizes them by itself. We find that providing dense\nrewards in text-based games using sentiment analysis improves performance under\nsome conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02338v1",
            "date": "2020-10-05 21:07:45+00:00",
            "title": "CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation",
            "authors": [
                "Tianlu Wang",
                "Xuezhi Wang",
                "Yao Qin",
                "Ben Packer",
                "Kang Li",
                "Jilin Chen",
                "Alex Beutel",
                "Ed Chi"
            ],
            "abstract": "NLP models are shown to suffer from robustness issues, i.e., a model's\nprediction can be easily changed under small perturbations to the input. In\nthis work, we present a Controlled Adversarial Text Generation (CAT-Gen) model\nthat, given an input text, generates adversarial texts through controllable\nattributes that are known to be invariant to task labels. For example, in order\nto attack a model for sentiment classification over product reviews, we can use\nthe product categories as the controllable attribute which would not change the\nsentiment of the reviews. Experiments on real-world NLP datasets demonstrate\nthat our method can generate more diverse and fluent adversarial texts,\ncompared to many existing adversarial text generation approaches. We further\nuse our generated adversarial examples to improve models through adversarial\ntraining, and we demonstrate that our generated attacks are more robust against\nmodel re-training and different model architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02352v2",
            "date": "2020-10-05 21:50:09+00:00",
            "title": "Inference Strategies for Machine Translation with Conditional Masking",
            "authors": [
                "Julia Kreutzer",
                "George Foster",
                "Colin Cherry"
            ],
            "abstract": "Conditional masked language model (CMLM) training has proven successful for\nnon-autoregressive and semi-autoregressive sequence generation tasks, such as\nmachine translation. Given a trained CMLM, however, it is not clear what the\nbest inference strategy is. We formulate masked inference as a factorization of\nconditional probabilities of partial sequences, show that this does not harm\nperformance, and investigate a number of simple heuristics motivated by this\nperspective. We identify a thresholding strategy that has advantages over the\nstandard \"mask-predict\" algorithm, and provide analyses of its behavior on\nmachine translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02375v2",
            "date": "2020-10-05 22:39:08+00:00",
            "title": "Investigating representations of verb bias in neural language models",
            "authors": [
                "Robert D. Hawkins",
                "Takateru Yamakoshi",
                "Thomas L. Griffiths",
                "Adele E. Goldberg"
            ],
            "abstract": "Languages typically provide more than one grammatical construction to express\ncertain types of messages. A speaker's choice of construction is known to\ndepend on multiple factors, including the choice of main verb -- a phenomenon\nknown as \\emph{verb bias}. Here we introduce DAIS, a large benchmark dataset\ncontaining 50K human judgments for 5K distinct sentence pairs in the English\ndative alternation. This dataset includes 200 unique verbs and systematically\nvaries the definiteness and length of arguments. We use this dataset, as well\nas an existing corpus of naturally occurring data, to evaluate how well recent\nneural language models capture human preferences. Results show that larger\nmodels perform better than smaller models, and transformer architectures (e.g.\nGPT-2) tend to out-perform recurrent architectures (e.g. LSTMs) even under\ncomparable parameter and training settings. Additional analyses of internal\nfeature representations suggest that transformers may better integrate specific\nlexical information with grammatical constructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02384v1",
            "date": "2020-10-05 23:06:24+00:00",
            "title": "Fine-Grained Grounding for Multimodal Speech Recognition",
            "authors": [
                "Tejas Srinivasan",
                "Ramon Sanabria",
                "Florian Metze",
                "Desmond Elliott"
            ],
            "abstract": "Multimodal automatic speech recognition systems integrate information from\nimages to improve speech recognition quality, by grounding the speech in the\nvisual context. While visual signals have been shown to be useful for\nrecovering entities that have been masked in the audio, these models should be\ncapable of recovering a broader range of word types. Existing systems rely on\nglobal visual features that represent the entire image, but localizing the\nrelevant regions of the image will make it possible to recover a larger set of\nwords, such as adjectives and verbs. In this paper, we propose a model that\nuses finer-grained visual information from different parts of the image, using\nautomatic object proposals. In experiments on the Flickr8K Audio Captions\nCorpus, we find that our model improves over approaches that use global visual\nfeatures, that the proposals enable the model to recover entities and other\nrelated words, such as adjectives, and that improvements are due to the model's\nability to localize the correct proposals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02399v1",
            "date": "2020-10-06 00:04:08+00:00",
            "title": "Guiding Attention for Self-Supervised Learning with Transformers",
            "authors": [
                "Ameet Deshpande",
                "Karthik Narasimhan"
            ],
            "abstract": "In this paper, we propose a simple and effective technique to allow for\nefficient self-supervised learning with bi-directional Transformers. Our\napproach is motivated by recent studies demonstrating that self-attention\npatterns in trained models contain a majority of non-linguistic regularities.\nWe propose a computationally efficient auxiliary loss function to guide\nattention heads to conform to such patterns. Our method is agnostic to the\nactual pre-training objective and results in faster convergence of models as\nwell as better performance on downstream tasks compared to the baselines,\nachieving state of the art results in low-resource settings. Surprisingly, we\nalso find that linguistic properties of attention heads are not necessarily\ncorrelated with language modeling performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02405v1",
            "date": "2020-10-06 00:25:50+00:00",
            "title": "Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning",
            "authors": [
                "Yi Yang",
                "Arzoo Katiyar"
            ],
            "abstract": "We present a simple few-shot named entity recognition (NER) system based on\nnearest neighbor learning and structured inference. Our system uses a\nsupervised NER model trained on the source domain, as a feature extractor.\nAcross several test domains, we show that a nearest neighbor classifier in this\nfeature-space is far more effective than the standard meta-learning approaches.\nWe further propose a cheap but effective method to capture the label\ndependencies between entity tags without expensive CRF training. We show that\nour method of combining structured decoding with nearest neighbor learning\nachieves state-of-the-art performance on standard few-shot NER evaluation\ntasks, improving F1 scores by $6\\%$ to $16\\%$ absolute points over prior\nmeta-learning based systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02423v2",
            "date": "2020-10-06 01:34:58+00:00",
            "title": "On the Role of Supervision in Unsupervised Constituency Parsing",
            "authors": [
                "Haoyue Shi",
                "Karen Livescu",
                "Kevin Gimpel"
            ],
            "abstract": "We analyze several recent unsupervised constituency parsing models, which are\ntuned with respect to the parsing $F_1$ score on the Wall Street Journal (WSJ)\ndevelopment set (1,700 sentences). We introduce strong baselines for them, by\ntraining an existing supervised parsing model (Kitaev and Klein, 2018) on the\nsame labeled examples they access. When training on the 1,700 examples, or even\nwhen using only 50 examples for training and 5 for development, such a few-shot\nparsing approach can outperform all the unsupervised parsing methods by a\nsignificant margin. Few-shot parsing can be further improved by a simple data\naugmentation method and self-training. This suggests that, in order to arrive\nat fair conclusions, we should carefully consider the amount of labeled data\nused for model development. We propose two protocols for future work on\nunsupervised parsing: (i) use fully unsupervised criteria for hyperparameter\ntuning and model selection; (ii) use as few labeled examples as possible for\nmodel development, and compare to few-shot parsing trained on the same labeled\nexamples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02428v3",
            "date": "2020-10-06 01:49:52+00:00",
            "title": "UnQovering Stereotyping Biases via Underspecified Questions",
            "authors": [
                "Tao Li",
                "Tushar Khot",
                "Daniel Khashabi",
                "Ashish Sabharwal",
                "Vivek Srikumar"
            ],
            "abstract": "While language embeddings have been shown to have stereotyping biases, how\nthese biases affect downstream question answering (QA) models remains\nunexplored. We present UNQOVER, a general framework to probe and quantify\nbiases through underspecified questions. We show that a naive use of model\nscores can lead to incorrect bias estimates due to two forms of reasoning\nerrors: positional dependence and question independence. We design a formalism\nthat isolates the aforementioned errors. As case studies, we use this metric to\nanalyze four important classes of stereotypes: gender, nationality, ethnicity,\nand religion. We probe five transformer-based QA models trained on two QA\ndatasets, along with their underlying language models. Our broad study reveals\nthat (1) all these models, with and without fine-tuning, have notable\nstereotyping biases in these classes; (2) larger models often have higher bias;\nand (3) the effect of fine-tuning on bias varies strongly with the dataset and\nthe model size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02429v3",
            "date": "2020-10-06 01:52:34+00:00",
            "title": "Modeling Preconditions in Text with a Crowd-sourced Dataset",
            "authors": [
                "Heeyoung Kwon",
                "Mahnaz Koupaee",
                "Pratyush Singh",
                "Gargi Sawhney",
                "Anmol Shukla",
                "Keerthi Kumar Kallur",
                "Nathanael Chambers",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Preconditions provide a form of logical connection between events that\nexplains why some events occur together and information that is complementary\nto the more widely studied relations such as causation, temporal ordering,\nentailment, and discourse relations. Modeling preconditions in text has been\nhampered in part due to the lack of large scale labeled data grounded in text.\nThis paper introduces PeKo, a crowd-sourced annotation of preconditions between\nevent pairs in newswire, an order of magnitude larger than prior text\nannotations. To complement this new corpus, we also introduce two challenge\ntasks aimed at modeling preconditions: (i) Precondition Identification -- a\nstandard classification task defined over pairs of event mentions, and (ii)\nPrecondition Generation -- a generative task aimed at testing a more general\nability to reason about a given event. Evaluation on both tasks shows that\nmodeling preconditions is challenging even for today's large language models\n(LM). This suggests that precondition knowledge is not easily accessible in\nLM-derived representations alone. Our generation results show that fine-tuning\nan LM on PeKo yields better conditional relations than when trained on raw text\nor temporally-ordered corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02443v1",
            "date": "2020-10-06 02:51:02+00:00",
            "title": "Multi-Fact Correction in Abstractive Text Summarization",
            "authors": [
                "Yue Dong",
                "Shuohang Wang",
                "Zhe Gan",
                "Yu Cheng",
                "Jackie Chi Kit Cheung",
                "Jingjing Liu"
            ],
            "abstract": "Pre-trained neural abstractive summarization systems have dominated\nextractive strategies on news summarization performance, at least in terms of\nROUGE. However, system-generated abstractive summaries often face the pitfall\nof factual inconsistency: generating incorrect facts with respect to the source\ntext. To address this challenge, we propose Span-Fact, a suite of two factual\ncorrection models that leverages knowledge learned from question answering\nmodels to make corrections in system-generated summaries via span selection.\nOur models employ single or multi-masking strategies to either iteratively or\nauto-regressively replace entities in order to ensure semantic consistency\nw.r.t. the source text, while retaining the syntactic structure of summaries\ngenerated by abstractive summarization models. Experiments show that our models\nsignificantly boost the factual consistency of system-generated summaries\nwithout sacrificing summary quality in terms of both automatic metrics and\nhuman evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02448v1",
            "date": "2020-10-06 03:09:14+00:00",
            "title": "On the Branching Bias of Syntax Extracted from Pre-trained Language Models",
            "authors": [
                "Huayang Li",
                "Lemao Liu",
                "Guoping Huang",
                "Shuming Shi"
            ],
            "abstract": "Many efforts have been devoted to extracting constituency trees from\npre-trained language models, often proceeding in two stages: feature definition\nand parsing. However, this kind of methods may suffer from the branching bias\nissue, which will inflate the performances on languages with the same branch it\nbiases to. In this work, we propose quantitatively measuring the branching bias\nby comparing the performance gap on a language and its reversed language, which\nis agnostic to both language models and extracting methods. Furthermore, we\nanalyze the impacts of three factors on the branching bias, namely parsing\nalgorithms, feature definitions, and language models. Experiments show that\nseveral existing works exhibit branching biases, and some implementations of\nthese three factors can introduce the branching bias.",
            "journal ref": "EMNLP 2020 Findings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02466v1",
            "date": "2020-10-06 04:12:28+00:00",
            "title": "Are Words Commensurate with Actions? Quantifying Commitment to a Cause from Online Public Messaging",
            "authors": [
                "Zhao Wang",
                "Jennifer Cutler",
                "Aron Culotta"
            ],
            "abstract": "Public entities such as companies and politicians increasingly use online\nsocial networks to communicate directly with their constituencies. Often, this\npublic messaging is aimed at aligning the entity with a particular cause or\nissue, such as the environment or public health. However, as a consumer or\nvoter, it can be difficult to assess an entity's true commitment to a cause\nbased on public messaging. In this paper, we present a text classification\napproach to categorize a message according to its commitment level toward a\ncause. We then compare the volume of such messages with external ratings based\non entities' actions (e.g., a politician's voting record with respect to the\nenvironment or a company's rating from environmental non-profits). We find that\nby distinguishing between low- and high- level commitment messages, we can more\nreliably identify truly committed entities. Furthermore, by measuring the\ndiscrepancy between classified messages and external ratings, we can identify\nentities whose public messaging does not align with their actions, thereby\nproviding a methodology to identify potentially \"inauthentic\" messaging\ncampaigns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02473v1",
            "date": "2020-10-06 04:38:09+00:00",
            "title": "Iterative Domain-Repaired Back-Translation",
            "authors": [
                "Hao-Ran Wei",
                "Zhirui Zhang",
                "Boxing Chen",
                "Weihua Luo"
            ],
            "abstract": "In this paper, we focus on the domain-specific translation with low\nresources, where in-domain parallel corpora are scarce or nonexistent. One\ncommon and effective strategy for this case is exploiting in-domain monolingual\ndata with the back-translation method. However, the synthetic parallel data is\nvery noisy because they are generated by imperfect out-of-domain systems,\nresulting in the poor performance of domain adaptation. To address this issue,\nwe propose a novel iterative domain-repaired back-translation framework, which\nintroduces the Domain-Repair (DR) model to refine translations in synthetic\nbilingual data. To this end, we construct corresponding data for the DR model\ntraining by round-trip translating the monolingual sentences, and then design\nthe unified training framework to optimize paired DR and NMT models jointly.\nExperiments on adapting NMT models between specific domains and from the\ngeneral domain to specific domains demonstrate the effectiveness of our\nproposed approach, achieving 15.79 and 4.47 BLEU improvements on average over\nunadapted models and back-translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02480v2",
            "date": "2020-10-06 05:15:39+00:00",
            "title": "Pretrained Language Model Embryology: The Birth of ALBERT",
            "authors": [
                "Cheng-Han Chiang",
                "Sung-Feng Huang",
                "Hung-yi Lee"
            ],
            "abstract": "While behaviors of pretrained language models (LMs) have been thoroughly\nexamined, what happened during pretraining is rarely studied. We thus\ninvestigate the developmental process from a set of randomly initialized\nparameters to a totipotent language model, which we refer to as the embryology\nof a pretrained language model. Our results show that ALBERT learns to\nreconstruct and predict tokens of different parts of speech (POS) in different\nlearning speeds during pretraining. We also find that linguistic knowledge and\nworld knowledge do not generally improve as pretraining proceeds, nor do\ndownstream tasks' performance. These findings suggest that knowledge of a\npretrained model varies during pretraining, and having more pretrain steps does\nnot necessarily provide a model with more comprehensive knowledge. We will\nprovide source codes and pretrained models to reproduce our results at\nhttps://github.com/d223302/albert-embryology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02494v1",
            "date": "2020-10-06 05:49:03+00:00",
            "title": "Help! Need Advice on Identifying Advice",
            "authors": [
                "Venkata Subrahmanyan Govindarajan",
                "Benjamin T Chen",
                "Rebecca Warholic",
                "Katrin Erk",
                "Junyi Jessy Li"
            ],
            "abstract": "Humans use language to accomplish a wide variety of tasks - asking for and\ngiving advice being one of them. In online advice forums, advice is mixed in\nwith non-advice, like emotional support, and is sometimes stated explicitly,\nsometimes implicitly. Understanding the language of advice would equip systems\nwith a better grasp of language pragmatics; practically, the ability to\nidentify advice would drastically increase the efficiency of advice-seeking\nonline, as well as advice-giving in natural language generation systems.\n  We present a dataset in English from two Reddit advice forums - r/AskParents\nand r/needadvice - annotated for whether sentences in posts contain advice or\nnot. Our analysis reveals rich linguistic phenomena in advice discourse. We\npresent preliminary models showing that while pre-trained language models are\nable to capture advice better than rule-based systems, advice identification is\nchallenging, and we identify directions for future research.\n  Comments: To be presented at EMNLP 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02498v1",
            "date": "2020-10-06 05:59:25+00:00",
            "title": "GRUEN for Evaluating Linguistic Quality of Generated Text",
            "authors": [
                "Wanzheng Zhu",
                "Suma Bhat"
            ],
            "abstract": "Automatic evaluation metrics are indispensable for evaluating generated text.\nTo date, these metrics have focused almost exclusively on the content selection\naspect of the system output, ignoring the linguistic quality aspect altogether.\nWe bridge this gap by proposing GRUEN for evaluating Grammaticality,\nnon-Redundancy, focUs, structure and coherENce of generated text. GRUEN\nutilizes a BERT-based model and a class of syntactic, semantic, and contextual\nfeatures to examine the system output. Unlike most existing evaluation metrics\nwhich require human references as an input, GRUEN is reference-less and\nrequires only the system output. Besides, it has the advantage of being\nunsupervised, deterministic, and adaptable to various tasks. Experiments on\nseven datasets over four language generation tasks show that the proposed\nmetric correlates highly with human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02534v1",
            "date": "2020-10-06 07:20:41+00:00",
            "title": "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks",
            "authors": [
                "Kyubyong Park",
                "Joohong Lee",
                "Seongbo Jang",
                "Dawoon Jung"
            ],
            "abstract": "Typically, tokenization is the very first step in most text processing works.\nAs a token serves as an atomic unit that embeds the contextual information of\ntext, how to define a token plays a decisive role in the performance of a\nmodel.Even though Byte Pair Encoding (BPE) has been considered the de facto\nstandard tokenization method due to its simplicity and universality, it still\nremains unclear whether BPE works best across all languages and tasks. In this\npaper, we test several tokenization strategies in order to answer our primary\nresearch question, that is, \"What is the best tokenization strategy for Korean\nNLP tasks?\" Experimental results demonstrate that a hybrid approach of\nmorphological segmentation followed by BPE works best in Korean to/from English\nmachine translation and natural language understanding tasks such as KorNLI,\nKorSTS, NSMC, and PAWS-X. As an exception, for KorQuAD, the Korean extension of\nSQuAD, BPE segmentation turns out to be the most effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02537v1",
            "date": "2020-10-06 07:43:17+00:00",
            "title": "Do Explicit Alignments Robustly Improve Multilingual Encoders?",
            "authors": [
                "Shijie Wu",
                "Mark Dredze"
            ],
            "abstract": "Multilingual BERT (mBERT), XLM-RoBERTa (XLMR) and other unsupervised\nmultilingual encoders can effectively learn cross-lingual representation.\nExplicit alignment objectives based on bitexts like Europarl or MultiUN have\nbeen shown to further improve these representations. However, word-level\nalignments are often suboptimal and such bitexts are unavailable for many\nlanguages. In this paper, we propose a new contrastive alignment objective that\ncan better utilize such signal, and examine whether these previous alignment\nmethods can be adapted to noisier sources of aligned data: a randomly sampled 1\nmillion pair subset of the OPUS collection. Additionally, rather than report\nresults on a single dataset with a single model run, we report the mean and\nstandard derivation of multiple runs with different seeds, on four datasets and\ntasks. Our more extensive analysis finds that, while our new objective\noutperforms previous work, overall these methods do not improve performance\nwith a more robust evaluation framework. Furthermore, the gains from using a\nbetter underlying model eclipse any benefits from alignment training. These\nnegative results dictate more care in evaluating these methods and suggest\nlimitations in applying explicit alignment objectives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02550v2",
            "date": "2020-10-06 08:31:14+00:00",
            "title": "Please Mind the Root: Decoding Arborescences for Dependency Parsing",
            "authors": [
                "Ran Zmigrod",
                "Tim Vieira",
                "Ryan Cotterell"
            ],
            "abstract": "The connection between dependency trees and spanning trees is exploited by\nthe NLP community to train and to decode graph-based dependency parsers.\nHowever, the NLP literature has missed an important difference between the two\nstructures: only one edge may emanate from the root in a dependency tree. We\nanalyzed the output of state-of-the-art parsers on many languages from the\nUniversal Dependency Treebank: although these parsers are often able to learn\nthat trees which violate the constraint should be assigned lower probabilities,\ntheir ability to do so unsurprisingly de-grades as the size of the training set\ndecreases. In fact, the worst constraint-violation rate we observe is 24%.\nPrior work has proposed an inefficient algorithm to enforce the constraint,\nwhich adds a factor of n to the decoding runtime. We adapt an algorithm due to\nGabow and Tarjan (1984) to dependency parsing, which satisfies the constraint\nwithout compromising the original runtime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02552v1",
            "date": "2020-10-06 08:57:31+00:00",
            "title": "Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation",
            "authors": [
                "Wenxiang Jiao",
                "Xing Wang",
                "Shilin He",
                "Irwin King",
                "Michael R. Lyu",
                "Zhaopeng Tu"
            ],
            "abstract": "Large-scale training datasets lie at the core of the recent success of neural\nmachine translation (NMT) models. However, the complex patterns and potential\nnoises in the large-scale data make training NMT models difficult. In this\nwork, we explore to identify the inactive training examples which contribute\nless to the model performance, and show that the existence of inactive examples\ndepends on the data distribution. We further introduce data rejuvenation to\nimprove the training of NMT models on large-scale datasets by exploiting\ninactive examples. The proposed framework consists of three phases. First, we\ntrain an identification model on the original training data, and use it to\ndistinguish inactive examples and active examples by their sentence-level\noutput probabilities. Then, we train a rejuvenation model on the active\nexamples, which is used to re-label the inactive examples with\nforward-translation. Finally, the rejuvenated examples and the active examples\nare combined to train the final NMT model. Experimental results on WMT14\nEnglish-German and English-French datasets show that the proposed data\nrejuvenation consistently and significantly improves performance for several\nstrong NMT models. Extensive analyses reveal that our approach stabilizes and\naccelerates the training process of NMT models, resulting in final models with\nbetter generalization capability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02557v1",
            "date": "2020-10-06 09:04:58+00:00",
            "title": "PolicyQA: A Reading Comprehension Dataset for Privacy Policies",
            "authors": [
                "Wasi Uddin Ahmad",
                "Jianfeng Chi",
                "Yuan Tian",
                "Kai-Wei Chang"
            ],
            "abstract": "Privacy policy documents are long and verbose. A question answering (QA)\nsystem can assist users in finding the information that is relevant and\nimportant to them. Prior studies in this domain frame the QA task as retrieving\nthe most relevant text segment or a list of sentences from the policy document\ngiven a question. On the contrary, we argue that providing users with a short\ntext span from policy documents reduces the burden of searching the target\ninformation from a lengthy text segment. In this paper, we present PolicyQA, a\ndataset that contains 25,017 reading comprehension style examples curated from\nan existing corpus of 115 website privacy policies. PolicyQA provides 714\nhuman-annotated questions written for a wide range of privacy practices. We\nevaluate two existing neural QA models and perform rigorous analysis to reveal\nthe advantages and challenges offered by PolicyQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02559v1",
            "date": "2020-10-06 09:06:07+00:00",
            "title": "LEGAL-BERT: The Muppets straight out of Law School",
            "authors": [
                "Ilias Chalkidis",
                "Manos Fergadiotis",
                "Prodromos Malakasiotis",
                "Nikolaos Aletras",
                "Ion Androutsopoulos"
            ],
            "abstract": "BERT has achieved impressive performance in several NLP tasks. However, there\nhas been limited investigation on its adaptation guidelines in specialised\ndomains. Here we focus on the legal domain, where we explore several approaches\nfor applying BERT models to downstream legal tasks, evaluating on multiple\ndatasets. Our findings indicate that the previous guidelines for pre-training\nand fine-tuning, often blindly followed, do not always generalize well in the\nlegal domain. Thus we propose a systematic investigation of the available\nstrategies when applying BERT in specialised domains. These are: (a) use the\noriginal BERT out of the box, (b) adapt BERT by additional pre-training on\ndomain-specific corpora, and (c) pre-train BERT from scratch on domain-specific\ncorpora. We also propose a broader hyper-parameter search space when\nfine-tuning for downstream tasks and we release LEGAL-BERT, a family of BERT\nmodels intended to assist legal NLP research, computational law, and legal\ntechnology applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02562v1",
            "date": "2020-10-06 09:11:02+00:00",
            "title": "Cross-Lingual Text Classification with Minimal Resources by Transferring a Sparse Teacher",
            "authors": [
                "Giannis Karamanolakis",
                "Daniel Hsu",
                "Luis Gravano"
            ],
            "abstract": "Cross-lingual text classification alleviates the need for manually labeled\ndocuments in a target language by leveraging labeled documents from other\nlanguages. Existing approaches for transferring supervision across languages\nrequire expensive cross-lingual resources, such as parallel corpora, while less\nexpensive cross-lingual representation learning approaches train classifiers\nwithout target labeled documents. In this work, we propose a cross-lingual\nteacher-student method, CLTS, that generates \"weak\" supervision in the target\nlanguage using minimal cross-lingual resources, in the form of a small number\nof word translations. Given a limited translation budget, CLTS extracts and\ntransfers only the most important task-specific seed words across languages and\ninitializes a teacher classifier based on the translated seed words. Then, CLTS\niteratively trains a more powerful student that also exploits the context of\nthe seed words in unlabeled target documents and outperforms the teacher. CLTS\nis simple and surprisingly effective in 18 diverse languages: by transferring\njust 20 seed words, even a bag-of-words logistic regression student outperforms\nstate-of-the-art cross-lingual methods (e.g., based on multilingual BERT).\nMoreover, CLTS can accommodate any type of student classifier: leveraging a\nmonolingual BERT student leads to further improvements and outperforms even\nmore expensive approaches by up to 12% in accuracy. Finally, CLTS addresses\nemerging tasks in low-resource languages using just a small number of word\ntranslations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02568v1",
            "date": "2020-10-06 09:26:55+00:00",
            "title": "SupMMD: A Sentence Importance Model for Extractive Summarization using Maximum Mean Discrepancy",
            "authors": [
                "Umanga Bista",
                "Alexander Patrick Mathews",
                "Aditya Krishna Menon",
                "Lexing Xie"
            ],
            "abstract": "Most work on multi-document summarization has focused on generic\nsummarization of information present in each individual document set. However,\nthe under-explored setting of update summarization, where the goal is to\nidentify the new information present in each set, is of equal practical\ninterest (e.g., presenting readers with updates on an evolving news topic). In\nthis work, we present SupMMD, a novel technique for generic and update\nsummarization based on the maximum mean discrepancy from kernel two-sample\ntesting. SupMMD combines both supervised learning for salience and unsupervised\nlearning for coverage and diversity. Further, we adapt multiple kernel learning\nto make use of similarity across multiple information sources (e.g., text\nfeatures and knowledge based concepts). We show the efficacy of SupMMD in both\ngeneric and update summarization tasks by meeting or exceeding the current\nstate-of-the-art on the DUC-2004 and TAC-2009 datasets.",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02569v1",
            "date": "2020-10-06 09:29:50+00:00",
            "title": "StyleDGPT: Stylized Response Generation with Pre-trained Language Models",
            "authors": [
                "Ze Yang",
                "Wei Wu",
                "Can Xu",
                "Xinnian Liang",
                "Jiaqi Bai",
                "Liran Wang",
                "Wei Wang",
                "Zhoujun Li"
            ],
            "abstract": "Generating responses following a desired style has great potentials to extend\napplications of open-domain dialogue systems, yet is refrained by lacking of\nparallel data for training. In this work, we explore the challenging task with\npre-trained language models that have brought breakthrough to various natural\nlanguage tasks. To this end, we introduce a KL loss and a style classifier to\nthe fine-tuning step in order to steer response generation towards the target\nstyle in both a word-level and a sentence-level. Comprehensive empirical\nstudies with two public datasets indicate that our model can significantly\noutperform state-of-the-art methods in terms of both style consistency and\ncontextual coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02570v1",
            "date": "2020-10-06 09:29:51+00:00",
            "title": "Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution",
            "authors": [
                "Yordan Yordanov",
                "Oana-Maria Camburu",
                "Vid Kocijan",
                "Thomas Lukasiewicz"
            ],
            "abstract": "Hard cases of pronoun resolution have been used as a long-standing benchmark\nfor commonsense reasoning. In the recent literature, pre-trained language\nmodels have been used to obtain state-of-the-art results on pronoun resolution.\nOverall, four categories of training and evaluation objectives have been\nintroduced. The variety of training datasets and pre-trained language models\nused in these works makes it unclear whether the choice of training objective\nis critical. In this work, we make a fair comparison of the performance and\nseed-wise stability of four models that represent the four categories of\nobjectives. Our experiments show that the objective of sequence ranking\nperforms the best in-domain, while the objective of semantic similarity between\ncandidates and pronoun performs the best out-of-domain. We also observe a\nseed-wise instability of the model using sequence ranking, which is not the\ncase when the other objectives are used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02584v1",
            "date": "2020-10-06 09:50:25+00:00",
            "title": "Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start",
            "authors": [
                "Wenpeng Yin",
                "Nazneen Fatema Rajani",
                "Dragomir Radev",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "A standard way to address different NLP problems is by first constructing a\nproblem-specific dataset, then building a model to fit this dataset. To build\nthe ultimate artificial intelligence, we desire a single machine that can\nhandle diverse new problems, for which task-specific annotations are limited.\nWe bring up textual entailment as a unified solver for such NLP problems.\nHowever, current research of textual entailment has not spilled much ink on the\nfollowing questions: (i) How well does a pretrained textual entailment system\ngeneralize across domains with only a handful of domain-specific examples? and\n(ii) When is it worth transforming an NLP task into textual entailment? We\nargue that the transforming is unnecessary if we can obtain rich annotations\nfor this task. Textual entailment really matters particularly when the target\nNLP task has insufficient annotations.\n  Universal NLP can be probably achieved through different routines. In this\nwork, we introduce Universal Few-shot textual Entailment (UFO-Entail). We\ndemonstrate that this framework enables a pretrained entailment model to work\nwell on new entailment domains in a few-shot setting, and show its\neffectiveness as a unified solver for several downstream NLP tasks such as\nquestion answering and coreference resolution when the end-task annotations are\nlimited. Code: https://github.com/salesforce/UniversalFewShotNLP",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02587v1",
            "date": "2020-10-06 09:55:00+00:00",
            "title": "Dissecting Span Identification Tasks with Performance Prediction",
            "authors": [
                "Sean Papay",
                "Roman Klinger",
                "Sebastian Pad\u00f3"
            ],
            "abstract": "Span identification (in short, span ID) tasks such as chunking, NER, or\ncode-switching detection, ask models to identify and classify relevant spans in\na text. Despite being a staple of NLP, and sharing a common structure, there is\nlittle insight on how these tasks' properties influence their difficulty, and\nthus little guidance on what model families work well on span ID tasks, and\nwhy. We analyze span ID tasks via performance prediction, estimating how well\nneural architectures do on different tasks. Our contributions are: (a) we\nidentify key properties of span ID tasks that can inform performance\nprediction; (b) we carry out a large-scale experiment on English data, building\na model to predict performance for unseen span ID tasks that can support\narchitecture choices; (c), we investigate the parameters of the meta model,\nyielding new insights on how model and task properties interact to affect span\nID performance. We find, e.g., that span frequency is especially important for\nLSTMs, and that CRFs help when spans are infrequent and boundaries\nnon-distinctive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02588v1",
            "date": "2020-10-06 09:55:36+00:00",
            "title": "CoRefi: A Crowd Sourcing Suite for Coreference Annotation",
            "authors": [
                "Aaron Bornstein",
                "Arie Cattan",
                "Ido Dagan"
            ],
            "abstract": "Coreference annotation is an important, yet expensive and time consuming,\ntask, which often involved expert annotators trained on complex decision\nguidelines. To enable cheaper and more efficient annotation, we present CoRefi,\na web-based coreference annotation suite, oriented for crowdsourcing. Beyond\nthe core coreference annotation tool, CoRefi provides guided onboarding for the\ntask as well as a novel algorithm for a reviewing phase. CoRefi is open source\nand directly embeds into any website, including popular crowdsourcing\nplatforms.\n  CoRefi Demo: aka.ms/corefi Video Tour: aka.ms/corefivideo Github Repo:\nhttps://github.com/aribornstein/corefi",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02591v1",
            "date": "2020-10-06 10:01:19+00:00",
            "title": "Scene Graph Modification Based on Natural Language Commands",
            "authors": [
                "Xuanli He",
                "Quan Hung Tran",
                "Gholamreza Haffari",
                "Walter Chang",
                "Trung Bui",
                "Zhe Lin",
                "Franck Dernoncourt",
                "Nhan Dam"
            ],
            "abstract": "Structured representations like graphs and parse trees play a crucial role in\nmany Natural Language Processing systems. In recent years, the advancements in\nmulti-turn user interfaces necessitate the need for controlling and updating\nthese structured representations given new sources of information. Although\nthere have been many efforts focusing on improving the performance of the\nparsers that map text to graphs or parse trees, very few have explored the\nproblem of directly manipulating these representations. In this paper, we\nexplore the novel problem of graph modification, where the systems need to\nlearn how to update an existing scene graph given a new user's command. Our\nnovel models based on graph-based sparse transformer and cross attention\ninformation fusion outperform previous systems adapted from the machine\ntranslation and graph generation literature. We further contribute our large\ngraph modification datasets to the research community to encourage future\nresearch for this new problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02609v3",
            "date": "2020-10-06 10:40:34+00:00",
            "title": "Position-Aware Tagging for Aspect Sentiment Triplet Extraction",
            "authors": [
                "Lu Xu",
                "Hao Li",
                "Wei Lu",
                "Lidong Bing"
            ],
            "abstract": "Aspect Sentiment Triplet Extraction (ASTE) is the task of extracting the\ntriplets of target entities, their associated sentiment, and opinion spans\nexplaining the reason for the sentiment. Existing research efforts mostly solve\nthis problem using pipeline approaches, which break the triplet extraction\nprocess into several stages. Our observation is that the three elements within\na triplet are highly related to each other, and this motivates us to build a\njoint model to extract such triplets using a sequence tagging approach.\nHowever, how to effectively design a tagging approach to extract the triplets\nthat can capture the rich interactions among the elements is a challenging\nresearch question. In this work, we propose the first end-to-end model with a\nnovel position-aware tagging scheme that is capable of jointly extracting the\ntriplets. Our experimental results on several existing datasets show that\njointly capturing elements in the triplet using our approach leads to improved\nperformance over the existing approaches. We also conducted extensive\nexperiments to investigate the model effectiveness and robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02636v1",
            "date": "2020-10-06 11:37:46+00:00",
            "title": "Neural Speech Synthesis for Estonian",
            "authors": [
                "Liisa R\u00e4tsep",
                "Liisi Piits",
                "Hille Pajupuu",
                "Indrek Hein",
                "Mark Fi\u0161el"
            ],
            "abstract": "This technical report describes the results of a collaboration between the\nNLP research group at the University of Tartu and the Institute of Estonian\nLanguage on improving neural speech synthesis for Estonian. The report (written\nin Estonian) describes the project results, the summary of which is: (1) Speech\nsynthesis data from 6 speakers for a total of 92.4 hours is collected and\nopenly released (CC-BY-4.0). Data available at https://konekorpus.tartunlp.ai\nand https://www.eki.ee/litsents/. (2) software and models for neural speech\nsynthesis is released open-source (MIT license). Available at\nhttps://koodivaramu.eesti.ee/tartunlp/text-to-speech . (3) We ran evaluations\nof the new models and compared them to other existing solutions (HMM-based HTS\nmodels from EKI, http://www.eki.ee/heli/, and Google's speech synthesis for\nEstonian, accessed via https://translate.google.com). Evaluation includes voice\nacceptability MOS scores for sentence-level and longer excerpts, detailed error\nanalysis and evaluation of the pre-processing module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02646v1",
            "date": "2020-10-06 11:47:20+00:00",
            "title": "On the Sparsity of Neural Machine Translation Models",
            "authors": [
                "Yong Wang",
                "Longyue Wang",
                "Victor O. K. Li",
                "Zhaopeng Tu"
            ],
            "abstract": "Modern neural machine translation (NMT) models employ a large number of\nparameters, which leads to serious over-parameterization and typically causes\nthe underutilization of computational resources. In response to this problem,\nwe empirically investigate whether the redundant parameters can be reused to\nachieve better performance. Experiments and analyses are systematically\nconducted on different datasets and NMT architectures. We show that: 1) the\npruned parameters can be rejuvenated to improve the baseline model by up to\n+0.8 BLEU points; 2) the rejuvenated parameters are reallocated to enhance the\nability of modeling low-level lexical information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02649v1",
            "date": "2020-10-06 11:53:23+00:00",
            "title": "Context Modeling with Evidence Filter for Multiple Choice Question Answering",
            "authors": [
                "Sicheng Yu",
                "Hao Zhang",
                "Wei Jing",
                "Jing Jiang"
            ],
            "abstract": "Multiple-Choice Question Answering (MCQA) is a challenging task in machine\nreading comprehension. The main challenge in MCQA is to extract \"evidence\" from\nthe given context that supports the correct answer. In the OpenbookQA dataset,\nthe requirement of extracting \"evidence\" is particularly important due to the\nmutual independence of sentences in the context. Existing work tackles this\nproblem by annotated evidence or distant supervision with rules which overly\nrely on human efforts. To address the challenge, we propose a simple yet\neffective approach termed evidence filtering to model the relationships between\nthe encoded contexts with respect to different options collectively and to\npotentially highlight the evidence sentences and filter out unrelated\nsentences. In addition to the effective reduction of human efforts of our\napproach compared, through extensive experiments on OpenbookQA, we show that\nthe proposed approach outperforms the models that use the same backbone and\nmore training data; and our parameter analysis also demonstrates the\ninterpretability of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02650v2",
            "date": "2020-10-06 11:57:03+00:00",
            "title": "If beam search is the answer, what was the question?",
            "authors": [
                "Clara Meister",
                "Tim Vieira",
                "Ryan Cotterell"
            ],
            "abstract": "Quite surprisingly, exact maximum a posteriori (MAP) decoding of neural\nlanguage generators frequently leads to low-quality results. Rather, most\nstate-of-the-art results on language generation tasks are attained using beam\nsearch despite its overwhelmingly high search error rate. This implies that the\nMAP objective alone does not express the properties we desire in text, which\nmerits the question: if beam search is the answer, what was the question? We\nframe beam search as the exact solution to a different decoding objective in\norder to gain insights into why high probability under a model alone may not\nindicate adequacy. We find that beam search enforces uniform information\ndensity in text, a property motivated by cognitive science. We suggest a set of\ndecoding objectives that explicitly enforce this property and find that exact\ndecoding with these objectives alleviates the problems encountered when\ndecoding poorly calibrated language generation models. Additionally, we analyze\nthe text produced using various decoding strategies and see that, in our neural\nmachine translation experiments, the extent to which this property is adhered\nto strongly correlates with BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02654v1",
            "date": "2020-10-06 12:03:47+00:00",
            "title": "Extracting Implicitly Asserted Propositions in Argumentation",
            "authors": [
                "Yohan Jo",
                "Jacky Visser",
                "Chris Reed",
                "Eduard Hovy"
            ],
            "abstract": "Argumentation accommodates various rhetorical devices, such as questions,\nreported speech, and imperatives. These rhetorical tools usually assert\nargumentatively relevant propositions rather implicitly, so understanding their\ntrue meaning is key to understanding certain arguments properly. However, most\nargument mining systems and computational linguistics research have paid little\nattention to implicitly asserted propositions in argumentation. In this paper,\nwe examine a wide range of computational methods for extracting propositions\nthat are implicitly asserted in questions, reported speech, and imperatives in\nargumentation. By evaluating the models on a corpus of 2016 U.S. presidential\ndebates and online commentary, we demonstrate the effectiveness and limitations\nof the computational models. Our study may inform future research on argument\nmining and the semantics of these rhetorical devices in argumentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02656v1",
            "date": "2020-10-06 12:07:54+00:00",
            "title": "Multi-Instance Multi-Label Learning Networks for Aspect-Category Sentiment Analysis",
            "authors": [
                "Yuncong Li",
                "Cunxiang Yin",
                "Sheng-hua Zhong",
                "Xu Pan"
            ],
            "abstract": "Aspect-category sentiment analysis (ACSA) aims to predict sentiment\npolarities of sentences with respect to given aspect categories. To detect the\nsentiment toward a particular aspect category in a sentence, most previous\nmethods first generate an aspect category-specific sentence representation for\nthe aspect category, then predict the sentiment polarity based on the\nrepresentation. These methods ignore the fact that the sentiment of an aspect\ncategory mentioned in a sentence is an aggregation of the sentiments of the\nwords indicating the aspect category in the sentence, which leads to suboptimal\nperformance. In this paper, we propose a Multi-Instance Multi-Label Learning\nNetwork for Aspect-Category sentiment analysis (AC-MIMLLN), which treats\nsentences as bags, words as instances, and the words indicating an aspect\ncategory as the key instances of the aspect category. Given a sentence and the\naspect categories mentioned in the sentence, AC-MIMLLN first predicts the\nsentiments of the instances, then finds the key instances for the aspect\ncategories, finally obtains the sentiments of the sentence toward the aspect\ncategories by aggregating the key instance sentiments. Experimental results on\nthree public datasets demonstrate the effectiveness of AC-MIMLLN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02660v1",
            "date": "2020-10-06 12:13:00+00:00",
            "title": "Detecting Attackable Sentences in Arguments",
            "authors": [
                "Yohan Jo",
                "Seojin Bang",
                "Emaad Manzoor",
                "Eduard Hovy",
                "Chris Reed"
            ],
            "abstract": "Finding attackable sentences in an argument is the first step toward\nsuccessful refutation in argumentation. We present a first large-scale analysis\nof sentence attackability in online arguments. We analyze driving reasons for\nattacks in argumentation and identify relevant characteristics of sentences. We\ndemonstrate that a sentence's attackability is associated with many of these\ncharacteristics regarding the sentence's content, proposition types, and tone,\nand that an external knowledge source can provide useful information about\nattackability. Building on these findings, we demonstrate that machine learning\nmodels can automatically detect attackable sentences in arguments,\nsignificantly better than several baselines and comparably well to laypeople.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02665v1",
            "date": "2020-10-06 12:35:13+00:00",
            "title": "Automatic Metaphor Interpretation Using Word Embeddings",
            "authors": [
                "Kfir Bar",
                "Nachum Dershowitz",
                "Lena Dankin"
            ],
            "abstract": "We suggest a model for metaphor interpretation using word embeddings trained\nover a relatively large corpus. Our system handles nominal metaphors, like\n\"time is money\". It generates a ranked list of potential interpretations of\ngiven metaphors. Candidate meanings are drawn from collocations of the topic\n(\"time\") and vehicle (\"money\") components, automatically extracted from a\ndependency-parsed corpus. We explore adding candidates derived from word\nassociation norms (common human responses to cues). Our ranking procedure\nconsiders similarity between candidate interpretations and metaphor components,\nmeasured in a semantic vector space. Lastly, a clustering algorithm removes\nsemantically related duplicates, thereby allowing other candidate\ninterpretations to attain higher rank. We evaluate using a set of annotated\nmetaphors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02686v1",
            "date": "2020-10-06 13:05:47+00:00",
            "title": "BERT Knows Punta Cana is not just beautiful, it's gorgeous: Ranking Scalar Adjectives with Contextualised Representations",
            "authors": [
                "Aina Gar\u00ed Soler",
                "Marianna Apidianaki"
            ],
            "abstract": "Adjectives like pretty, beautiful and gorgeous describe positive properties\nof the nouns they modify but with different intensity. These differences are\nimportant for natural language understanding and reasoning. We propose a novel\nBERT-based approach to intensity detection for scalar adjectives. We model\nintensity by vectors directly derived from contextualised representations and\nshow they can successfully rank scalar adjectives. We evaluate our models both\nintrinsically, on gold standard datasets, and on an Indirect Question Answering\ntask. Our results demonstrate that BERT encodes rich knowledge about the\nsemantics of scalar adjectives, and is able to provide better quality intensity\nrankings than static embeddings and previous models with access to dedicated\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02695v1",
            "date": "2020-10-06 13:17:38+00:00",
            "title": "Analyzing Individual Neurons in Pre-trained Language Models",
            "authors": [
                "Nadir Durrani",
                "Hassan Sajjad",
                "Fahim Dalvi",
                "Yonatan Belinkov"
            ],
            "abstract": "While a lot of analysis has been carried to demonstrate linguistic knowledge\ncaptured by the representations learned within deep NLP models, very little\nattention has been paid towards individual neurons.We carry outa neuron-level\nanalysis using core linguistic tasks of predicting morphology, syntax and\nsemantics, on pre-trained language models, with questions like: i) do\nindividual neurons in pre-trained models capture linguistic information? ii)\nwhich parts of the network learn more about certain linguistic phenomena? iii)\nhow distributed or focused is the information? and iv) how do various\narchitectures differ in learning these properties? We found small subsets of\nneurons to predict linguistic tasks, with lower level tasks (such as\nmorphology) localized in fewer neurons, compared to higher level task of\npredicting syntax. Our study also reveals interesting cross architectural\ncomparisons. For example, we found neurons in XLNet to be more localized and\ndisjoint when predicting properties compared to BERT and others, where they are\nmore distributed and coupled.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02696v2",
            "date": "2020-10-06 13:18:35+00:00",
            "title": "Aspect Based Sentiment Analysis with Aspect-Specific Opinion Spans",
            "authors": [
                "Lu Xu",
                "Lidong Bing",
                "Wei Lu",
                "Fei Huang"
            ],
            "abstract": "Aspect based sentiment analysis, predicting sentiment polarity of given\naspects, has drawn extensive attention. Previous attention-based models\nemphasize using aspect semantics to help extract opinion features for\nclassification. However, these works are either not able to capture opinion\nspans as a whole, or not able to capture variable-length opinion spans. In this\npaper, we present a neat and effective structured attention model by\naggregating multiple linear-chain CRFs. Such a design allows the model to\nextract aspect-specific opinion spans and then evaluate sentiment polarity by\nexploiting the extracted opinion features. The experimental results on four\ndatasets demonstrate the effectiveness of the proposed model, and our analysis\ndemonstrates that our model can capture aspect-specific opinion spans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02733v1",
            "date": "2020-10-06 13:55:11+00:00",
            "title": "Towards Coalgebras in Stylometry",
            "authors": [
                "Jo\u00ebl A. Doat"
            ],
            "abstract": "The syntactic behaviour of texts can highly vary depending on their contexts\n(e.g. author, genre, etc.). From the standpoint of stylometry, it can be\nhelpful to objectively measure this behaviour. In this paper, we discuss how\ncoalgebras are used to formalise the notion of behaviour by embedding syntactic\nfeatures of a given text into probabilistic transition systems. By introducing\nthe behavioural distance, we are then able to quantitatively measure\ndifferences between points in these systems and thus, comparing features of\ndifferent texts. Furthermore, the behavioural distance of points can be\napproximated by a polynomial-time algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02744v1",
            "date": "2020-10-06 14:12:58+00:00",
            "title": "Stepwise Extractive Summarization and Planning with Structured Transformers",
            "authors": [
                "Shashi Narayan",
                "Joshua Maynez",
                "Jakub Adamek",
                "Daniele Pighin",
                "Bla\u017e Bratani\u010d",
                "Ryan McDonald"
            ],
            "abstract": "We propose encoder-centric stepwise models for extractive summarization using\nstructured transformers -- HiBERT and Extended Transformers. We enable stepwise\nsummarization by injecting the previously generated summary into the structured\ntransformer as an auxiliary sub-structure. Our models are not only efficient in\nmodeling the structure of long inputs, but they also do not rely on\ntask-specific redundancy-aware modeling, making them a general purpose\nextractive content planner for different tasks. When evaluated on CNN/DailyMail\nextractive summarization, stepwise models achieve state-of-the-art performance\nin terms of Rouge without any redundancy aware modeling or sentence filtering.\nThis also holds true for Rotowire table-to-text generation, where our models\nsurpass previously reported metrics for content selection, planning and\nordering, highlighting the strength of stepwise modeling. Amongst the two\nstructured transformers we test, stepwise Extended Transformers provides the\nbest performance across both datasets and sets a new standard for these\nchallenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02789v1",
            "date": "2020-10-06 14:59:16+00:00",
            "title": "An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks",
            "authors": [
                "Lifu Tu",
                "Tianyu Liu",
                "Kevin Gimpel"
            ],
            "abstract": "Many tasks in natural language processing involve predicting structured\noutputs, e.g., sequence labeling, semantic role labeling, parsing, and machine\ntranslation. Researchers are increasingly applying deep representation learning\nto these problems, but the structured component of these approaches is usually\nquite simplistic. In this work, we propose several high-order energy terms to\ncapture complex dependencies among labels in sequence labeling, including\nseveral that consider the entire label sequence. We use neural\nparameterizations for these energy terms, drawing from convolutional,\nrecurrent, and self-attention networks. We use the framework of learning\nenergy-based inference networks (Tu and Gimpel, 2018) for dealing with the\ndifficulties of training and inference with such models. We empirically\ndemonstrate that this approach achieves substantial improvement using a variety\nof high-order energy terms on four sequence labeling tasks, while having the\nsame decoding speed as simple, local classifiers. We also find high-order\nenergies to help in noisy data conditions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02795v1",
            "date": "2020-10-06 15:09:38+00:00",
            "title": "COSMIC: COmmonSense knowledge for eMotion Identification in Conversations",
            "authors": [
                "Deepanway Ghosal",
                "Navonil Majumder",
                "Alexander Gelbukh",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "In this paper, we address the task of utterance level emotion recognition in\nconversations using commonsense knowledge. We propose COSMIC, a new framework\nthat incorporates different elements of commonsense such as mental states,\nevents, and causal relations, and build upon them to learn interactions between\ninterlocutors participating in a conversation. Current state-of-the-art methods\noften encounter difficulties in context propagation, emotion shift detection,\nand differentiating between related emotion classes. By learning distinct\ncommonsense representations, COSMIC addresses these challenges and achieves new\nstate-of-the-art results for emotion recognition on four different benchmark\nconversational datasets. Our code is available at\nhttps://github.com/declare-lab/conv-emotion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02812v1",
            "date": "2020-10-06 15:21:08+00:00",
            "title": "Intrinsic Probing through Dimension Selection",
            "authors": [
                "Lucas Torroba Hennigen",
                "Adina Williams",
                "Ryan Cotterell"
            ],
            "abstract": "Most modern NLP systems make use of pre-trained contextual representations\nthat attain astonishingly high performance on a variety of tasks. Such high\nperformance should not be possible unless some form of linguistic structure\ninheres in these representations, and a wealth of research has sprung up on\nprobing for it. In this paper, we draw a distinction between intrinsic probing,\nwhich examines how linguistic information is structured within a\nrepresentation, and the extrinsic probing popular in prior work, which only\nargues for the presence of such information by showing that it can be\nsuccessfully extracted. To enable intrinsic probing, we propose a novel\nframework based on a decomposable multivariate Gaussian probe that allows us to\ndetermine whether the linguistic information in word embeddings is dispersed or\nfocal. We then probe fastText and BERT for various morphosyntactic attributes\nacross 36 languages. We find that most attributes are reliably encoded by only\na few neurons, with fastText concentrating its linguistic structure more than\nBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02815v1",
            "date": "2020-10-06 15:25:15+00:00",
            "title": "QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines",
            "authors": [
                "Valentina Pyatkin",
                "Ayal Klein",
                "Reut Tsarfaty",
                "Ido Dagan"
            ],
            "abstract": "Discourse relations describe how two propositions relate to one another, and\nidentifying them automatically is an integral part of natural language\nunderstanding. However, annotating discourse relations typically requires\nexpert annotators. Recently, different semantic aspects of a sentence have been\nrepresented and crowd-sourced via question-and-answer (QA) pairs. This paper\nproposes a novel representation of discourse relations as QA pairs, which in\nturn allows us to crowd-source wide-coverage data annotated with discourse\nrelations, via an intuitively appealing interface for composing such questions\nand answers. Based on our proposed representation, we collect a novel and\nwide-coverage QADiscourse dataset, and present baseline algorithms for\npredicting QADiscourse relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02864v1",
            "date": "2020-10-06 16:34:03+00:00",
            "title": "A Novel Challenge Set for Hebrew Morphological Disambiguation and Diacritics Restoration",
            "authors": [
                "Avi Shmidman",
                "Joshua Guedalia",
                "Shaltiel Shmidman",
                "Moshe Koppel",
                "Reut Tsarfaty"
            ],
            "abstract": "One of the primary tasks of morphological parsers is the disambiguation of\nhomographs. Particularly difficult are cases of unbalanced ambiguity, where one\nof the possible analyses is far more frequent than the others. In such cases,\nthere may not exist sufficient examples of the minority analyses in order to\nproperly evaluate performance, nor to train effective classifiers. In this\npaper we address the issue of unbalanced morphological ambiguities in Hebrew.\nWe offer a challenge set for Hebrew homographs -- the first of its kind --\ncontaining substantial attestation of each analysis of 21 Hebrew homographs. We\nshow that the current SOTA of Hebrew disambiguation performs poorly on cases of\nunbalanced ambiguity. Leveraging our new dataset, we achieve a new\nstate-of-the-art for all 21 words, improving the overall average F1 score from\n0.67 to 0.95. Our resulting annotated datasets are made publicly available for\nfurther research.",
            "journal ref": "Findings of EMNLP, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02882v1",
            "date": "2020-10-06 17:06:50+00:00",
            "title": "COD3S: Diverse Generation with Discrete Semantic Signatures",
            "authors": [
                "Nathaniel Weir",
                "Jo\u00e3o Sedoc",
                "Benjamin Van Durme"
            ],
            "abstract": "We present COD3S, a novel method for generating semantically diverse\nsentences using neural sequence-to-sequence (seq2seq) models. Conditioned on an\ninput, seq2seq models typically produce semantically and syntactically\nhomogeneous sets of sentences and thus perform poorly on one-to-many sequence\ngeneration tasks. Our two-stage approach improves output diversity by\nconditioning generation on locality-sensitive hash (LSH)-based semantic\nsentence codes whose Hamming distances highly correlate with human judgments of\nsemantic textual similarity. Though it is generally applicable, we apply COD3S\nto causal generation, the task of predicting a proposition's plausible causes\nor effects. We demonstrate through automatic and human evaluation that\nresponses produced using our method exhibit improved diversity without\ndegrading task performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02903v1",
            "date": "2020-10-06 17:36:29+00:00",
            "title": "Keep CALM and Explore: Language Models for Action Generation in Text-based Games",
            "authors": [
                "Shunyu Yao",
                "Rohan Rao",
                "Matthew Hausknecht",
                "Karthik Narasimhan"
            ],
            "abstract": "Text-based games present a unique challenge for autonomous agents to operate\nin natural language and handle enormous action spaces. In this paper, we\npropose the Contextual Action Language Model (CALM) to generate a compact set\nof action candidates at each game state. Our key insight is to train language\nmodels on human gameplay, where people demonstrate linguistic priors and a\ngeneral game sense for promising actions conditioned on game history. We\ncombine CALM with a reinforcement learning agent which re-ranks the generated\naction candidates to maximize in-game rewards. We evaluate our approach using\nthe Jericho benchmark, on games unseen by CALM during training. Our method\nobtains a 69% relative improvement in average game score over the previous\nstate-of-the-art model. Surprisingly, on half of these games, CALM is\ncompetitive with or better than other models that have access to ground truth\nadmissible actions. Code and data are available at\nhttps://github.com/princeton-nlp/calm-textgame.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02975v1",
            "date": "2020-10-06 19:09:02+00:00",
            "title": "Supervised Seeded Iterated Learning for Interactive Language Learning",
            "authors": [
                "Yuchen Lu",
                "Soumye Singhal",
                "Florian Strub",
                "Olivier Pietquin",
                "Aaron Courville"
            ],
            "abstract": "Language drift has been one of the major obstacles to train language models\nthrough interaction. When word-based conversational agents are trained towards\ncompleting a task, they tend to invent their language rather than leveraging\nnatural language. In recent literature, two general methods partially counter\nthis phenomenon: Supervised Selfplay (S2P) and Seeded Iterated Learning (SIL).\nWhile S2P jointly trains interactive and supervised losses to counter the\ndrift, SIL changes the training dynamics to prevent language drift from\noccurring. In this paper, we first highlight their respective weaknesses, i.e.,\nlate-stage training collapses and higher negative likelihood when evaluated on\nhuman corpus. Given these observations, we introduce Supervised Seeded Iterated\nLearning to combine both methods to minimize their respective weaknesses. We\nthen show the effectiveness of \\algo in the language-drift translation game.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02976v2",
            "date": "2020-10-06 19:09:03+00:00",
            "title": "Are \"Undocumented Workers\" the Same as \"Illegal Aliens\"? Disentangling Denotation and Connotation in Vector Spaces",
            "authors": [
                "Albert Webson",
                "Zhizhong Chen",
                "Carsten Eickhoff",
                "Ellie Pavlick"
            ],
            "abstract": "In politics, neologisms are frequently invented for partisan objectives. For\nexample, \"undocumented workers\" and \"illegal aliens\" refer to the same group of\npeople (i.e., they have the same denotation), but they carry clearly different\nconnotations. Examples like these have traditionally posed a challenge to\nreference-based semantic theories and led to increasing acceptance of\nalternative theories (e.g., Two-Factor Semantics) among philosophers and\ncognitive scientists. In NLP, however, popular pretrained models encode both\ndenotation and connotation as one entangled representation. In this study, we\npropose an adversarial neural network that decomposes a pretrained\nrepresentation as independent denotation and connotation representations. For\nintrinsic interpretability, we show that words with the same denotation but\ndifferent connotations (e.g., \"immigrants\" vs. \"aliens\", \"estate tax\" vs.\n\"death tax\") move closer to each other in denotation space while moving further\napart in connotation space. For extrinsic application, we train an information\nretrieval system with our disentangled representations and show that the\ndenotation vectors improve the viewpoint diversity of document rankings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03001v3",
            "date": "2020-10-06 20:05:43+00:00",
            "title": "A Review on Fact Extraction and Verification",
            "authors": [
                "Giannis Bekoulis",
                "Christina Papagiannopoulou",
                "Nikos Deligiannis"
            ],
            "abstract": "We study the fact checking problem, which aims to identify the veracity of a\ngiven claim. Specifically, we focus on the task of Fact Extraction and\nVERification (FEVER) and its accompanied dataset. The task consists of the\nsubtasks of retrieving the relevant documents (and sentences) from Wikipedia\nand validating whether the information in the documents supports or refutes a\ngiven claim. This task is essential and can be the building block of\napplications such as fake news detection and medical claim verification. In\nthis paper, we aim at a better understanding of the challenges of the task by\npresenting the literature in a structured and comprehensive way. Moreover, we\ndescribe the proposed methods by analyzing the technical perspectives of the\ndifferent approaches and discussing the performance results on the FEVER\ndataset, which is the most well-studied and formally structured dataset on the\nfact extraction and verification task. We also conduct the largest experimental\nstudy to date on identifying beneficial loss functions for the sentence\nretrieval component. Our analysis indicates that sampling negative sentences is\nimportant for improving the performance and decreasing the computational\ncomplexity. Finally, we describe open issues and future challenges, and we\nmotivate future research in the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03009v2",
            "date": "2020-10-06 20:30:35+00:00",
            "title": "GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction",
            "authors": [
                "Wasi Uddin Ahmad",
                "Nanyun Peng",
                "Kai-Wei Chang"
            ],
            "abstract": "Recent progress in cross-lingual relation and event extraction use graph\nconvolutional networks (GCNs) with universal dependency parses to learn\nlanguage-agnostic sentence representations such that models trained on one\nlanguage can be applied to other languages. However, GCNs struggle to model\nwords with long-range dependencies or are not directly connected in the\ndependency tree. To address these challenges, we propose to utilize the\nself-attention mechanism where we explicitly fuse structural information to\nlearn the dependencies between words with different syntactic distances. We\nintroduce GATE, a {\\bf G}raph {\\bf A}ttention {\\bf T}ransformer {\\bf E}ncoder,\nand test its cross-lingual transferability on relation and event extraction\ntasks. We perform experiments on the ACE05 dataset that includes three\ntypologically different languages: English, Chinese, and Arabic. The evaluation\nresults show that GATE outperforms three recently proposed methods by a large\nmargin. Our detailed analysis reveals that due to the reliance on syntactic\ndependencies, GATE produces robust representations that facilitate transfer\nacross languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03010v1",
            "date": "2020-10-06 20:30:59+00:00",
            "title": "Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Priming",
            "authors": [
                "Kanishka Misra",
                "Allyson Ettinger",
                "Julia Taylor Rayz"
            ],
            "abstract": "Models trained to estimate word probabilities in context have become\nubiquitous in natural language processing. How do these models use lexical cues\nin context to inform their word probabilities? To answer this question, we\npresent a case study analyzing the pre-trained BERT model with tests informed\nby semantic priming. Using English lexical stimuli that show priming in humans,\nwe find that BERT too shows \"priming,\" predicting a word with greater\nprobability when the context includes a related word versus an unrelated one.\nThis effect decreases as the amount of information provided by the context\nincreases. Follow-up analysis shows BERT to be increasingly distracted by\nrelated prime words as context becomes more informative, assigning lower\nprobabilities to related words. Our findings highlight the importance of\nconsidering contextual constraint effects when studying word prediction in\nthese models, and highlight possible parallels with human processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03022v1",
            "date": "2020-10-06 21:00:54+00:00",
            "title": "Resource-Enhanced Neural Model for Event Argument Extraction",
            "authors": [
                "Jie Ma",
                "Shuai Wang",
                "Rishita Anubhai",
                "Miguel Ballesteros",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Event argument extraction (EAE) aims to identify the arguments of an event\nand classify the roles that those arguments play. Despite great efforts made in\nprior work, there remain many challenges: (1) Data scarcity. (2) Capturing the\nlong-range dependency, specifically, the connection between an event trigger\nand a distant event argument. (3) Integrating event trigger information into\ncandidate argument representation. For (1), we explore using unlabeled data in\ndifferent ways. For (2), we propose to use a syntax-attending Transformer that\ncan utilize dependency parses to guide the attention mechanism. For (3), we\npropose a trigger-aware sequence encoder with several types of\ntrigger-dependent sequence representations. We also support argument extraction\neither from text annotated with gold entities or from plain text. Experiments\non the English ACE2005 benchmark show that our approach achieves a new\nstate-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03034v1",
            "date": "2020-10-06 21:08:16+00:00",
            "title": "Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers",
            "authors": [
                "Yimeng Wu",
                "Peyman Passban",
                "Mehdi Rezagholizade",
                "Qun Liu"
            ],
            "abstract": "With the growth of computing power neural machine translation (NMT) models\nalso grow accordingly and become better. However, they also become harder to\ndeploy on edge devices due to memory constraints. To cope with this problem, a\ncommon practice is to distill knowledge from a large and accurately-trained\nteacher network (T) into a compact student network (S). Although knowledge\ndistillation (KD) is useful in most cases, our study shows that existing KD\ntechniques might not be suitable enough for deep NMT engines, so we propose a\nnovel alternative. In our model, besides matching T and S predictions we have a\ncombinatorial mechanism to inject layer-level supervision from T to S. In this\npaper, we target low-resource settings and evaluate our translation engines for\nPortuguese--English, Turkish--English, and English--German directions. Students\ntrained using our technique have 50% fewer parameters and can still deliver\ncomparable results to those of 12-layer teachers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03061v1",
            "date": "2020-10-06 22:23:00+00:00",
            "title": "A Survey on Recognizing Textual Entailment as an NLP Evaluation",
            "authors": [
                "Adam Poliak"
            ],
            "abstract": "Recognizing Textual Entailment (RTE) was proposed as a unified evaluation\nframework to compare semantic understanding of different NLP systems. In this\nsurvey paper, we provide an overview of different approaches for evaluating and\nunderstanding the reasoning capabilities of NLP systems. We then focus our\ndiscussion on RTE by highlighting prominent RTE datasets as well as advances in\nRTE dataset that focus on specific linguistic phenomena that can be used to\nevaluate NLP systems on a fine-grained level. We conclude by arguing that when\nevaluating NLP systems, the community should utilize newly introduced RTE\ndatasets that focus on specific linguistic phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03065v1",
            "date": "2020-10-06 22:33:58+00:00",
            "title": "Anubhuti -- An annotated dataset for emotional analysis of Bengali short stories",
            "authors": [
                "Aditya Pal",
                "Bhaskar Karn"
            ],
            "abstract": "Thousands of short stories and articles are being written in many different\nlanguages all around the world today. Bengali, or Bangla, is the second highest\nspoken language in India after Hindi and is the national language of the\ncountry of Bangladesh. This work reports in detail the creation of Anubhuti --\nthe first and largest text corpus for analyzing emotions expressed by writers\nof Bengali short stories. We explain the data collection methods, the manual\nannotation process and the resulting high inter-annotator agreement of the\ndataset due to the linguistic expertise of the annotators and the clear\nmethodology of labelling followed. We also address some of the challenges faced\nin the collection of raw data and annotation process of a low resource language\nlike Bengali. We have verified the performance of our dataset with baseline\nMachine Learning as well as a Deep Learning model for emotion classification\nand have found that these standard models have a high accuracy and relevant\nfeature selection on Anubhuti. In addition, we also explain how this dataset\ncan be of interest to linguists and data analysts to study the flow of emotions\nas expressed by writers of Bengali literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03093v1",
            "date": "2020-10-07 00:28:05+00:00",
            "title": "WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization",
            "authors": [
                "Faisal Ladhak",
                "Esin Durmus",
                "Claire Cardie",
                "Kathleen McKeown"
            ],
            "abstract": "We introduce WikiLingua, a large-scale, multilingual dataset for the\nevaluation of crosslingual abstractive summarization systems. We extract\narticle and summary pairs in 18 languages from WikiHow, a high quality,\ncollaborative resource of how-to guides on a diverse set of topics written by\nhuman authors. We create gold-standard article-summary alignments across\nlanguages by aligning the images that are used to describe each how-to step in\nan article. As a set of baselines for further studies, we evaluate the\nperformance of existing cross-lingual abstractive summarization methods on our\ndataset. We further propose a method for direct crosslingual summarization\n(i.e., without requiring translation at inference time) by leveraging synthetic\ndata and Neural Machine Translation as a pre-training step. Our method\nsignificantly outperforms the baseline approaches, while being more cost\nefficient during inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03099v1",
            "date": "2020-10-07 01:19:23+00:00",
            "title": "DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling",
            "authors": [
                "Jiecao Chen",
                "Liu Yang",
                "Karthik Raman",
                "Michael Bendersky",
                "Jung-Jung Yeh",
                "Yun Zhou",
                "Marc Najork",
                "Danyang Cai",
                "Ehsan Emadzadeh"
            ],
            "abstract": "Pre-trained models like BERT (Devlin et al., 2018) have dominated NLP / IR\napplications such as single sentence classification, text pair classification,\nand question answering. However, deploying these models in real systems is\nhighly non-trivial due to their exorbitant computational costs. A common remedy\nto this is knowledge distillation (Hinton et al., 2015), leading to faster\ninference. However -- as we show here -- existing works are not optimized for\ndealing with pairs (or tuples) of texts. Consequently, they are either not\nscalable or demonstrate subpar performance. In this work, we propose DiPair --\na novel framework for distilling fast and accurate models on text pair tasks.\nCoupled with an end-to-end training strategy, DiPair is both highly scalable\nand offers improved quality-speed tradeoffs. Empirical studies conducted on\nboth academic and real-world e-commerce benchmarks demonstrate the efficacy of\nthe proposed approach with speedups of over 350x and minimal quality drop\nrelative to the cross-attention teacher BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03142v3",
            "date": "2020-10-07 03:57:54+00:00",
            "title": "Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information",
            "authors": [
                "Zehui Lin",
                "Xiao Pan",
                "Mingxuan Wang",
                "Xipeng Qiu",
                "Jiangtao Feng",
                "Hao Zhou",
                "Lei Li"
            ],
            "abstract": "We investigate the following question for machine translation (MT): can we\ndevelop a single universal MT model to serve as the common seed and obtain\nderivative and improved models on arbitrary language pairs? We propose mRASP,\nan approach to pre-train a universal multilingual neural machine translation\nmodel. Our key idea in mRASP is its novel technique of random aligned\nsubstitution, which brings words and phrases with similar meanings across\nmultiple languages closer in the representation space. We pre-train a mRASP\nmodel on 32 language pairs jointly with only public datasets. The model is then\nfine-tuned on downstream language pairs to obtain specialized MT models. We\ncarry out extensive experiments on 42 translation directions across a diverse\nsettings, including low, medium, rich resource, and as well as transferring to\nexotic language pairs. Experimental results demonstrate that mRASP achieves\nsignificant performance improvement compared to directly training on those\ntarget pairs. It is the first time to verify that multiple low-resource\nlanguage pairs can be utilized to improve rich resource MT. Surprisingly, mRASP\nis even able to improve the translation quality on exotic languages that never\noccur in the pre-training corpus. Code, data, and pre-trained models are\navailable at https://github.com/linzehui/mRASP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03147v1",
            "date": "2020-10-07 04:05:37+00:00",
            "title": "OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction",
            "authors": [
                "Keshav Kolluru",
                "Vaibhav Adlakha",
                "Samarth Aggarwal",
                "Mausam",
                "Soumen Chakrabarti"
            ],
            "abstract": "A recent state-of-the-art neural open information extraction (OpenIE) system\ngenerates extractions iteratively, requiring repeated encoding of partial\noutputs. This comes at a significant computational cost. On the other hand,\nsequence labeling approaches for OpenIE are much faster, but worse in\nextraction quality. In this paper, we bridge this trade-off by presenting an\niterative labeling-based system that establishes a new state of the art for\nOpenIE, while extracting 10x faster. This is achieved through a novel Iterative\nGrid Labeling (IGL) architecture, which treats OpenIE as a 2-D grid labeling\ntask. We improve its performance further by applying coverage (soft)\nconstraints on the grid at training time.\n  Moreover, on observing that the best OpenIE systems falter at handling\ncoordination structures, our OpenIE system also incorporates a new coordination\nanalyzer built with the same IGL architecture. This IGL based coordination\nanalyzer helps our OpenIE system handle complicated coordination structures,\nwhile also establishing a new state of the art on the task of coordination\nanalysis, with a 12.3 pts improvement in F1 over previous analyzers. Our OpenIE\nsystem, OpenIE6, beats the previous systems by as much as 4 pts in F1, while\nbeing much faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03154v1",
            "date": "2020-10-07 04:43:48+00:00",
            "title": "Fortifying Toxic Speech Detectors Against Veiled Toxicity",
            "authors": [
                "Xiaochuang Han",
                "Yulia Tsvetkov"
            ],
            "abstract": "Modern toxic speech detectors are incompetent in recognizing disguised\noffensive language, such as adversarial attacks that deliberately avoid known\ntoxic lexicons, or manifestations of implicit bias. Building a large annotated\ndataset for such veiled toxicity can be very expensive. In this work, we\npropose a framework aimed at fortifying existing toxic speech detectors without\na large labeled corpus of veiled toxicity. Just a handful of probing examples\nare used to surface orders of magnitude more disguised offenses. We augment the\ntoxic speech detector's training data with these discovered offensive examples,\nthereby making it more robust to veiled toxicity while preserving its utility\nin detecting overt toxicity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03155v1",
            "date": "2020-10-07 04:45:09+00:00",
            "title": "A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction",
            "authors": [
                "Masato Mita",
                "Shun Kiyono",
                "Masahiro Kaneko",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Existing approaches for grammatical error correction (GEC) largely rely on\nsupervised learning with manually created GEC datasets. However, there has been\nlittle focus on verifying and ensuring the quality of the datasets, and on how\nlower-quality data might affect GEC performance. We indeed found that there is\na non-negligible amount of \"noise\" where errors were inappropriately edited or\nleft uncorrected. To address this, we designed a self-refinement method where\nthe key idea is to denoise these datasets by leveraging the prediction\nconsistency of existing models, and outperformed strong denoising baseline\nmethods. We further applied task-specific techniques and achieved\nstate-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks.\nWe then analyzed the effect of the proposed denoising method, and found that\nour approach leads to improved coverage of corrections and facilitated fluency\nedits which are reflected in higher recall and overall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03224v1",
            "date": "2020-10-07 07:06:09+00:00",
            "title": "Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields",
            "authors": [
                "Jingxuan Yang",
                "Kerui Xu",
                "Jun Xu",
                "Si Li",
                "Sheng Gao",
                "Jun Guo",
                "Ji-Rong Wen",
                "Nianwen Xue"
            ],
            "abstract": "Pronouns are often dropped in Chinese conversations and recovering the\ndropped pronouns is important for NLP applications such as Machine Translation.\nExisting approaches usually formulate this as a sequence labeling task of\npredicting whether there is a dropped pronoun before each token and its type.\nEach utterance is considered to be a sequence and labeled independently.\nAlthough these approaches have shown promise, labeling each utterance\nindependently ignores the dependencies between pronouns in neighboring\nutterances. Modeling these dependencies is critical to improving the\nperformance of dropped pronoun recovery. In this paper, we present a novel\nframework that combines the strength of Transformer network with General\nConditional Random Fields (GCRF) to model the dependencies between pronouns in\nneighboring utterances. Results on three Chinese conversation datasets show\nthat the Transformer-GCRF model outperforms the state-of-the-art dropped\npronoun recovery models. Exploratory analysis also demonstrates that the GCRF\ndid help to capture the dependencies between pronouns in neighboring\nutterances, thus contributes to the performance improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03249v2",
            "date": "2020-10-07 08:03:58+00:00",
            "title": "Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment",
            "authors": [
                "Zhiyuan Liu",
                "Yixin Cao",
                "Liangming Pan",
                "Juanzi Li",
                "Zhiyuan Liu",
                "Tat-Seng Chua"
            ],
            "abstract": "Entity alignment (EA) aims at building a unified Knowledge Graph (KG) of rich\ncontent by linking the equivalent entities from various KGs. GNN-based EA\nmethods present promising performances by modeling the KG structure defined by\nrelation triples. However, attribute triples can also provide crucial alignment\nsignal but have not been well explored yet. In this paper, we propose to\nutilize an attributed value encoder and partition the KG into subgraphs to\nmodel the various types of attribute triples efficiently. Besides, the\nperformances of current EA methods are overestimated because of the name-bias\nof existing EA datasets. To make an objective evaluation, we propose a hard\nexperimental setting where we select equivalent entity pairs with very\ndifferent names as the test set. Under both the regular and hard settings, our\nmethod achieves significant improvements ($5.10\\%$ on average Hits@$1$ in\nDBP$15$k) over $12$ baselines in cross-lingual and monolingual datasets.\nAblation studies on different subgraphs and a case study about attribute types\nfurther demonstrate the effectiveness of our method. Source code and data can\nbe found at https://github.com/thunlp/explore-and-evaluate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03260v1",
            "date": "2020-10-07 08:29:11+00:00",
            "title": "Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction",
            "authors": [
                "Mengyun Chen",
                "Tao Ge",
                "Xingxing Zhang",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "We propose a novel language-independent approach to improve the efficiency\nfor Grammatical Error Correction (GEC) by dividing the task into two subtasks:\nErroneous Span Detection (ESD) and Erroneous Span Correction (ESC). ESD\nidentifies grammatically incorrect text spans with an efficient sequence\ntagging model. Then, ESC leverages a seq2seq model to take the sentence with\nannotated erroneous spans as input and only outputs the corrected text for\nthese spans. Experiments show our approach performs comparably to conventional\nseq2seq approaches in both English and Chinese GEC benchmarks with less than\n50% time cost for inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03276v1",
            "date": "2020-10-07 08:57:34+00:00",
            "title": "ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization",
            "authors": [
                "Tzuf Paz-Argaman",
                "Yuval Atzmon",
                "Gal Chechik",
                "Reut Tsarfaty"
            ],
            "abstract": "We study the problem of recognizing visual entities from the textual\ndescriptions of their classes. Specifically, given birds' images with free-text\ndescriptions of their species, we learn to classify images of previously-unseen\nspecies based on specie descriptions. This setup has been studied in the vision\ncommunity under the name zero-shot learning from text, focusing on learning to\ntransfer knowledge about visual aspects of birds from seen classes to\npreviously-unseen ones. Here, we suggest focusing on the textual description\nand distilling from the description the most relevant information to\neffectively match visual features to the parts of the text that discuss them.\nSpecifically, (1) we propose to leverage the similarity between species,\nreflected in the similarity between text descriptions of the species. (2) we\nderive visual summaries of the texts, i.e., extractive summaries that focus on\nthe visual features that tend to be reflected in images. We propose a simple\nattention-based model augmented with the similarity and visual summaries\ncomponents. Our empirical results consistently and significantly outperform the\nstate-of-the-art on the largest benchmarks for text-based zero-shot learning,\nillustrating the critical importance of texts for zero-shot image-recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03295v2",
            "date": "2020-10-07 09:16:45+00:00",
            "title": "COMETA: A Corpus for Medical Entity Linking in the Social Media",
            "authors": [
                "Marco Basaldella",
                "Fangyu Liu",
                "Ehsan Shareghi",
                "Nigel Collier"
            ],
            "abstract": "Whilst there has been growing progress in Entity Linking (EL) for general\nlanguage, existing datasets fail to address the complex nature of health\nterminology in layman's language. Meanwhile, there is a growing need for\napplications that can understand the public's voice in the health domain. To\naddress this we introduce a new corpus called COMETA, consisting of 20k English\nbiomedical entity mentions from Reddit expert-annotated with links to SNOMED\nCT, a widely-used medical knowledge graph. Our corpus satisfies a combination\nof desirable properties, from scale and coverage to diversity and quality, that\nto the best of our knowledge has not been met by any of the existing resources\nin the field. Through benchmark experiments on 20 EL baselines from string- to\nneural-based models we shed light on the ability of these systems to perform\ncomplex inference on entities and concepts under 2 challenging evaluation\nscenarios. Our experimental results on COMETA illustrate that no golden bullet\nexists and even the best mainstream techniques still have a significant\nperformance gap to fill, while the best solution relies on combining different\nviews of data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03338v1",
            "date": "2020-10-07 11:18:49+00:00",
            "title": "Improving QA Generalization by Concurrent Modeling of Multiple Biases",
            "authors": [
                "Mingzhu Wu",
                "Nafise Sadat Moosavi",
                "Andreas R\u00fcckl\u00e9",
                "Iryna Gurevych"
            ],
            "abstract": "Existing NLP datasets contain various biases that models can easily exploit\nto achieve high performances on the corresponding evaluation sets. However,\nfocusing on dataset-specific biases limits their ability to learn more\ngeneralizable knowledge about the task from more general data patterns. In this\npaper, we investigate the impact of debiasing methods for improving\ngeneralization and propose a general framework for improving the performance on\nboth in-domain and out-of-domain datasets by concurrent modeling of multiple\nbiases in the training data. Our framework weights each example based on the\nbiases it contains and the strength of those biases in the training data. It\nthen uses these weights in the training objective so that the model relies less\non examples with high bias weights. We extensively evaluate our framework on\nextractive question answering with training data from various domains with\nmultiple biases of different strengths. We perform the evaluations in two\ndifferent settings, in which the model is trained on a single domain or\nmultiple domains simultaneously, and show its effectiveness in both settings\ncompared to state-of-the-art debiasing methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03384v1",
            "date": "2020-10-07 12:54:28+00:00",
            "title": "Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision",
            "authors": [
                "Max Glockner",
                "Ivan Habernal",
                "Iryna Gurevych"
            ],
            "abstract": "Evaluating the trustworthiness of a model's prediction is essential for\ndifferentiating between `right for the right reasons' and `right for the wrong\nreasons'. Identifying textual spans that determine the target label, known as\nfaithful rationales, usually relies on pipeline approaches or reinforcement\nlearning. However, such methods either require supervision and thus costly\nannotation of the rationales or employ non-differentiable models. We propose a\ndifferentiable training-framework to create models which output faithful\nrationales on a sentence level, by solely applying supervision on the target\ntask. To achieve this, our model solves the task based on each rationale\nindividually and learns to assign high scores to those which solved the task\nbest. Our evaluation on three different datasets shows competitive results\ncompared to a standard BERT blackbox while exceeding a pipeline counterpart's\nperformance in two cases. We further exploit the transparent decision-making\nprocess of these models to prefer selecting the correct rationales by applying\ndirect supervision, thereby boosting the performance on the rationale-level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03424v2",
            "date": "2020-10-07 14:06:09+00:00",
            "title": "Cross-lingual Extended Named Entity Classification of Wikipedia Articles",
            "authors": [
                "The Viet Bui",
                "Phuong Le-Hong"
            ],
            "abstract": "The FPT.AI team participated in the SHINRA2020-ML subtask of the NTCIR-15\nSHINRA task. This paper describes our method to solving the problem and\ndiscusses the official results. Our method focuses on learning cross-lingual\nrepresentations, both on the word level and document level for page\nclassification. We propose a three-stage approach including multilingual model\npre-training, monolingual model fine-tuning and cross-lingual voting. Our\nsystem is able to achieve the best scores for 25 out of 30 languages; and its\naccuracy gaps to the best performing systems of the other five languages are\nrelatively small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03450v1",
            "date": "2020-10-07 14:41:40+00:00",
            "title": "\"I'd rather just go to bed\": Understanding Indirect Answers",
            "authors": [
                "Annie Louis",
                "Dan Roth",
                "Filip Radlinski"
            ],
            "abstract": "We revisit a pragmatic inference problem in dialog: understanding indirect\nresponses to questions. Humans can interpret 'I'm starving.' in response to\n'Hungry?', even without direct cue words such as 'yes' and 'no'. In dialog\nsystems, allowing natural responses rather than closed vocabularies would be\nsimilarly beneficial. However, today's systems are only as sensitive to these\npragmatic moves as their language model allows. We create and release the first\nlarge-scale English language corpus 'Circa' with 34,268 (polar question,\nindirect answer) pairs to enable progress on this task. The data was collected\nvia elaborate crowdsourcing, and contains utterances with yes/no meaning, as\nwell as uncertain, middle-ground, and conditional responses. We also present\nBERT-based neural models to predict such categories for a question-answer pair.\nWe find that while transfer learning from entailment works reasonably,\nperformance is not yet sufficient for robust dialog. Our models reach 82-88%\naccuracy for a 4-class distinction, and 74-85% for 6 classes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03476v2",
            "date": "2020-10-07 15:25:41+00:00",
            "title": "Learning a Cost-Effective Annotation Policy for Question Answering",
            "authors": [
                "Bernhard Kratzwald",
                "Stefan Feuerriegel",
                "Huan Sun"
            ],
            "abstract": "State-of-the-art question answering (QA) relies upon large amounts of\ntraining data for which labeling is time consuming and thus expensive. For this\nreason, customizing QA systems is challenging. As a remedy, we propose a novel\nframework for annotating QA datasets that entails learning a cost-effective\nannotation policy and a semi-supervised annotation scheme. The latter reduces\nthe human effort: it leverages the underlying QA system to suggest potential\ncandidate annotations. Human annotators then simply provide binary feedback on\nthese candidates. Our system is designed such that past annotations\ncontinuously improve the future performance and thus overall annotation cost.\nTo the best of our knowledge, this is the first paper to address the problem of\nannotating questions with minimal annotation cost. We compare our framework\nagainst traditional manual annotations in an extensive set of experiments. We\nfind that our approach can reduce up to 21.1% of the annotation cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03481v1",
            "date": "2020-10-07 15:34:00+00:00",
            "title": "ELMo and BERT in semantic change detection for Russian",
            "authors": [
                "Julia Rodina",
                "Yuliya Trofimova",
                "Andrey Kutuzov",
                "Ekaterina Artemova"
            ],
            "abstract": "We study the effectiveness of contextualized embeddings for the task of\ndiachronic semantic change detection for Russian language data. Evaluation test\nsets consist of Russian nouns and adjectives annotated based on their\noccurrences in texts created in pre-Soviet, Soviet and post-Soviet time\nperiods. ELMo and BERT architectures are compared on the task of ranking\nRussian words according to the degree of their semantic change over time. We\nuse several methods for aggregation of contextualized embeddings from these\narchitectures and evaluate their performance. Finally, we compare unsupervised\nand supervised techniques in this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03486v1",
            "date": "2020-10-07 15:44:55+00:00",
            "title": "Improving Sentiment Analysis over non-English Tweets using Multilingual Transformers and Automatic Translation for Data-Augmentation",
            "authors": [
                "Valentin Barriere",
                "Alexandra Balahur"
            ],
            "abstract": "Tweets are specific text data when compared to general text. Although\nsentiment analysis over tweets has become very popular in the last decade for\nEnglish, it is still difficult to find huge annotated corpora for non-English\nlanguages. The recent rise of the transformer models in Natural Language\nProcessing allows to achieve unparalleled performances in many tasks, but these\nmodels need a consequent quantity of text to adapt to the tweet domain. We\npropose the use of a multilingual transformer model, that we pre-train over\nEnglish tweets and apply data-augmentation using automatic translation to adapt\nthe model to non-English languages. Our experiments in French, Spanish, German\nand Italian suggest that the proposed technique is an efficient way to improve\nthe results of the transformers over small corpora of tweets in a non-English\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03494v2",
            "date": "2020-10-07 15:58:25+00:00",
            "title": "TeaForN: Teacher-Forcing with N-grams",
            "authors": [
                "Sebastian Goodman",
                "Nan Ding",
                "Radu Soricut"
            ],
            "abstract": "Sequence generation models trained with teacher-forcing suffer from issues\nrelated to exposure bias and lack of differentiability across timesteps. Our\nproposed method, Teacher-Forcing with N-grams (TeaForN), addresses both these\nproblems directly, through the use of a stack of N decoders trained to decode\nalong a secondary time axis that allows model parameter updates based on N\nprediction steps. TeaForN can be used with a wide class of decoder\narchitectures and requires minimal modifications from a standard\nteacher-forcing setup. Empirically, we show that TeaForN boosts generation\nquality on one Machine Translation benchmark, WMT 2014 English-French, and two\nNews Summarization benchmarks, CNN/Dailymail and Gigaword.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03538v1",
            "date": "2020-10-07 17:34:50+00:00",
            "title": "Exploring the Role of Argument Structure in Online Debate Persuasion",
            "authors": [
                "Jialu Li",
                "Esin Durmus",
                "Claire Cardie"
            ],
            "abstract": "Online debate forums provide users a platform to express their opinions on\ncontroversial topics while being exposed to opinions from diverse set of\nviewpoints. Existing work in Natural Language Processing (NLP) has shown that\nlinguistic features extracted from the debate text and features encoding the\ncharacteristics of the audience are both critical in persuasion studies. In\nthis paper, we aim to further investigate the role of discourse structure of\nthe arguments from online debates in their persuasiveness. In particular, we\nuse the factor graph model to obtain features for the argument structure of\ndebates from an online debating platform and incorporate these features to an\nLSTM-based model to predict the debater that makes the most convincing\narguments. We find that incorporating argument structure features play an\nessential role in achieving the better predictive performance in assessing the\npersuasiveness of the arguments in online debates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03542v1",
            "date": "2020-10-07 17:40:19+00:00",
            "title": "Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models",
            "authors": [
                "Shuohuan Wang",
                "Jiaxiang Liu",
                "Xuan Ouyang",
                "Yu Sun"
            ],
            "abstract": "This paper describes Galileo's performance in SemEval-2020 Task 12 on\ndetecting and categorizing offensive language in social media. For Offensive\nLanguage Identification, we proposed a multi-lingual method using Pre-trained\nLanguage Models, ERNIE and XLM-R. For offensive language categorization, we\nproposed a knowledge distillation method trained on soft labels generated by\nseveral supervised models. Our team participated in all three sub-tasks. In\nSub-task A - Offensive Language Identification, we ranked first in terms of\naverage F1 scores in all languages. We are also the only team which ranked\namong the top three across all languages. We also took the first place in\nSub-task B - Automatic Categorization of Offense Types and Sub-task C - Offence\nTarget Identification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03546v1",
            "date": "2020-10-07 17:47:53+00:00",
            "title": "Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing",
            "authors": [
                "Xilun Chen",
                "Asish Ghoshal",
                "Yashar Mehdad",
                "Luke Zettlemoyer",
                "Sonal Gupta"
            ],
            "abstract": "Task-oriented semantic parsing is a critical component of virtual assistants,\nwhich is responsible for understanding the user's intents (set reminder, play\nmusic, etc.). Recent advances in deep learning have enabled several approaches\nto successfully parse more complex queries (Gupta et al., 2018; Rongali et\nal.,2020), but these models require a large amount of annotated training data\nto parse queries on new domains (e.g. reminder, music).\n  In this paper, we focus on adapting task-oriented semantic parsers to\nlow-resource domains, and propose a novel method that outperforms a supervised\nneural model at a 10-fold data reduction. In particular, we identify two\nfundamental factors for low-resource domain adaptation: better representation\nlearning and better training techniques. Our representation learning uses BART\n(Lewis et al., 2019) to initialize our model which outperforms encoder-only\npre-trained representations used in previous work. Furthermore, we train with\noptimization-based meta-learning (Finn et al., 2017) to improve generalization\nto low-resource domains. This approach significantly outperforms all baseline\nmethods in the experiments on a newly collected multi-domain task-oriented\nsemantic parsing dataset (TOPv2), which we release to the public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03548v2",
            "date": "2020-10-07 17:48:12+00:00",
            "title": "Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion",
            "authors": [
                "Rajarshi Das",
                "Ameya Godbole",
                "Nicholas Monath",
                "Manzil Zaheer",
                "Andrew McCallum"
            ],
            "abstract": "A case-based reasoning (CBR) system solves a new problem by retrieving\n`cases' that are similar to the given problem. If such a system can achieve\nhigh accuracy, it is appealing owing to its simplicity, interpretability, and\nscalability. In this paper, we demonstrate that such a system is achievable for\nreasoning in knowledge-bases (KBs). Our approach predicts attributes for an\nentity by gathering reasoning paths from similar entities in the KB. Our\nprobabilistic model estimates the likelihood that a path is effective at\nanswering a query about the given entity. The parameters of our model can be\nefficiently computed using simple path statistics and require no iterative\noptimization. Our model is non-parametric, growing dynamically as new entities\nand relations are added to the KB. On several benchmark datasets our approach\nsignificantly outperforms other rule learning approaches and performs\ncomparably to state-of-the-art embedding-based approaches. Furthermore, we\ndemonstrate the effectiveness of our model in an \"open-world\" setting where new\nentities arrive in an online fashion, significantly outperforming\nstate-of-the-art approaches and nearly matching the best offline method. Code\navailable at https://github.com/ameyagodbole/Prob-CBR",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03550v2",
            "date": "2020-10-07 17:50:58+00:00",
            "title": "Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations",
            "authors": [
                "Benjamin E. Nye",
                "Jay DeYoung",
                "Eric Lehman",
                "Ani Nenkova",
                "Iain J. Marshall",
                "Byron C. Wallace"
            ],
            "abstract": "The best evidence concerning comparative treatment effectiveness comes from\nclinical trials, the results of which are reported in unstructured articles.\nMedical experts must manually extract information from articles to inform\ndecision-making, which is time-consuming and expensive. Here we consider the\nend-to-end task of both (a) extracting treatments and outcomes from full-text\narticles describing clinical trials (entity identification) and, (b) inferring\nthe reported results for the former with respect to the latter (relation\nextraction). We introduce new data for this task, and evaluate models that have\nrecently achieved state-of-the-art results on similar tasks in Natural Language\nProcessing. We then propose a new method motivated by how trial results are\ntypically presented that outperforms these purely data-driven baselines.\nFinally, we run a fielded evaluation of the model with a non-profit seeking to\nidentify existing drugs that might be re-purposed for cancer, showing the\npotential utility of end-to-end evidence extraction systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03604v2",
            "date": "2020-10-07 18:51:17+00:00",
            "title": "SRLGRN: Semantic Role Labeling Graph Reasoning Network",
            "authors": [
                "Chen Zheng",
                "Parisa Kordjamshidi"
            ],
            "abstract": "This work deals with the challenge of learning and reasoning over multi-hop\nquestion answering (QA). We propose a graph reasoning network based on the\nsemantic structure of the sentences to learn cross paragraph reasoning paths\nand find the supporting facts and the answer jointly. The proposed graph is a\nheterogeneous document-level graph that contains nodes of type sentence\n(question, title, and other sentences), and semantic role labeling sub-graphs\nper sentence that contain arguments as nodes and predicates as edges.\nIncorporating the argument types, the argument phrases, and the semantics of\nthe edges originated from SRL predicates into the graph encoder helps in\nfinding and also the explainability of the reasoning paths. Our proposed\napproach shows competitive performance on the HotpotQA distractor setting\nbenchmark compared to the recent state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03614v1",
            "date": "2020-10-07 19:16:45+00:00",
            "title": "Combining Deep Learning and String Kernels for the Localization of Swiss German Tweets",
            "authors": [
                "Mihaela Gaman",
                "Radu Tudor Ionescu"
            ],
            "abstract": "In this work, we introduce the methods proposed by the UnibucKernel team in\nsolving the Social Media Variety Geolocation task featured in the 2020 VarDial\nEvaluation Campaign. We address only the second subtask, which targets a data\nset composed of nearly 30 thousand Swiss German Jodels. The dialect\nidentification task is about accurately predicting the latitude and longitude\nof test samples. We frame the task as a double regression problem, employing a\nvariety of machine learning approaches to predict both latitude and longitude.\nFrom simple models for regression, such as Support Vector Regression, to deep\nneural networks, such as Long Short-Term Memory networks and character-level\nconvolutional neural networks, and, finally, to ensemble models based on\nmeta-learners, such as XGBoost, our interest is focused on approaching the\nproblem from a few different perspectives, in an attempt to minimize the\nprediction error. With the same goal in mind, we also considered many types of\nfeatures, from high-level features, such as BERT embeddings, to low-level\nfeatures, such as characters n-grams, which are known to provide good results\nin dialect identification. Our empirical results indicate that the handcrafted\nmodel based on string kernels outperforms the deep learning approaches.\nNevertheless, our best performance is given by the ensemble model that combines\nboth handcrafted and deep learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03617v1",
            "date": "2020-10-07 19:19:42+00:00",
            "title": "MuSeM: Detecting Incongruent News Headlines using Mutual Attentive Semantic Matching",
            "authors": [
                "Rahul Mishra",
                "Piyush Yadav",
                "Remi Calizzano",
                "Markus Leippold"
            ],
            "abstract": "Measuring the congruence between two texts has several useful applications,\nsuch as detecting the prevalent deceptive and misleading news headlines on the\nweb. Many works have proposed machine learning based solutions such as text\nsimilarity between the headline and body text to detect the incongruence. Text\nsimilarity based methods fail to perform well due to different inherent\nchallenges such as relative length mismatch between the news headline and its\nbody content and non-overlapping vocabulary. On the other hand, more recent\nworks that use headline guided attention to learn a headline derived contextual\nrepresentation of the news body also result in convoluting overall\nrepresentation due to the news body's lengthiness. This paper proposes a method\nthat uses inter-mutual attention-based semantic matching between the original\nand synthetically generated headlines, which utilizes the difference between\nall pairs of word embeddings of words involved. The paper also investigates two\nmore variations of our method, which use concatenation and dot-products of word\nembeddings of the words of original and synthetic headlines. We observe that\nthe proposed method outperforms prior arts significantly for two publicly\navailable datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03640v1",
            "date": "2020-10-07 20:27:12+00:00",
            "title": "Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations",
            "authors": [
                "Emily Allaway",
                "Kathleen McKeown"
            ],
            "abstract": "Stance detection is an important component of understanding hidden influences\nin everyday life. Since there are thousands of potential topics to take a\nstance on, most with little to no training data, we focus on zero-shot stance\ndetection: classifying stance from no training examples. In this paper, we\npresent a new dataset for zero-shot stance detection that captures a wider\nrange of topics and lexical variation than in previous datasets. Additionally,\nwe propose a new model for stance detection that implicitly captures\nrelationships between topics using generalized topic representations and show\nthat this model improves performance on a number of challenging linguistic\nphenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03652v1",
            "date": "2020-10-07 21:02:41+00:00",
            "title": "Cross-Thought for Sentence Encoder Pre-training",
            "authors": [
                "Shuohang Wang",
                "Yuwei Fang",
                "Siqi Sun",
                "Zhe Gan",
                "Yu Cheng",
                "Jing Jiang",
                "Jingjing Liu"
            ],
            "abstract": "In this paper, we propose Cross-Thought, a novel approach to pre-training\nsequence encoder, which is instrumental in building reusable sequence\nembeddings for large-scale NLP tasks such as question answering. Instead of\nusing the original signals of full sentences, we train a Transformer-based\nsequence encoder over a large set of short sequences, which allows the model to\nautomatically select the most useful information for predicting masked words.\nExperiments on question answering and textual entailment tasks demonstrate that\nour pre-trained encoder can outperform state-of-the-art encoders trained with\ncontinuous sentence signals as well as traditional masked language modeling\nbaselines. Our proposed approach also achieves new state of the art on HotpotQA\n(full-wiki setting) by improving intermediate information retrieval\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03656v1",
            "date": "2020-10-07 21:17:25+00:00",
            "title": "Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data",
            "authors": [
                "Shachar Rosenman",
                "Alon Jacovi",
                "Yoav Goldberg"
            ],
            "abstract": "The process of collecting and annotating training data may introduce\ndistribution artifacts which may limit the ability of models to learn correct\ngeneralization behavior. We identify failure modes of SOTA relation extraction\n(RE) models trained on TACRED, which we attribute to limitations in the data\nannotation process. We collect and annotate a challenge-set we call Challenging\nRE (CRE), based on naturally occurring corpus examples, to benchmark this\nbehavior. Our experiments with four state-of-the-art RE models show that they\nhave indeed adopted shallow heuristics that do not generalize to the\nchallenge-set data. Further, we find that alternative question answering\nmodeling performs significantly better than the SOTA models on the\nchallenge-set, despite worse overall TACRED performance. By adding some of the\nchallenge data as training examples, the performance of the model improves.\nFinally, we provide concrete suggestion on how to improve RE data collection to\nalleviate this behavior.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03662v1",
            "date": "2020-10-07 21:26:20+00:00",
            "title": "Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank",
            "authors": [
                "Eleftheria Briakou",
                "Marine Carpuat"
            ],
            "abstract": "Detecting fine-grained differences in content conveyed in different languages\nmatters for cross-lingual NLP and multilingual corpora analysis, but it is a\nchallenging machine learning problem since annotation is expensive and hard to\nscale. This work improves the prediction and annotation of fine-grained\nsemantic divergences. We introduce a training strategy for multilingual BERT\nmodels by learning to rank synthetic divergent examples of varying granularity.\nWe evaluate our models on the Rationalized English-French Semantic Divergences,\na new dataset released with this work, consisting of English-French\nsentence-pairs annotated with semantic divergence classes and token-level\nrationales. Learning to rank helps detect fine-grained sentence-level\ndivergences more accurately than a strong sentence-level similarity model,\nwhile token-level predictions have the potential of further distinguishing\nbetween coarse and fine-grained divergences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03722v1",
            "date": "2020-10-08 01:49:16+00:00",
            "title": "A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion",
            "authors": [
                "Logan Lebanoff",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Walter Chang",
                "Fei Liu"
            ],
            "abstract": "We present an empirical study in favor of a cascade architecture to neural\ntext summarization. Summarization practices vary widely but few other than news\nsummarization can provide a sufficient amount of training data enough to meet\nthe requirement of end-to-end neural abstractive systems which perform content\nselection and surface realization jointly to generate abstracts. Such systems\nalso pose a challenge to summarization evaluation, as they force content\nselection to be evaluated along with text generation, yet evaluation of the\nlatter remains an unsolved problem. In this paper, we present empirical results\nshowing that the performance of a cascaded pipeline that separately identifies\nimportant content pieces and stitches them together into a coherent text is\ncomparable to or outranks that of end-to-end systems, whereas a pipeline\narchitecture allows for flexible content selection. We finally discuss how we\ncan take advantage of a cascaded pipeline in neural text summarization and shed\nlight on important directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03725v1",
            "date": "2020-10-08 02:01:31+00:00",
            "title": "PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge",
            "authors": [
                "Yun He",
                "Zhuoer Wang",
                "Yin Zhang",
                "Ruihong Huang",
                "James Caverlee"
            ],
            "abstract": "We present a new benchmark dataset called PARADE for paraphrase\nidentification that requires specialized domain knowledge. PARADE contains\nparaphrases that overlap very little at the lexical and syntactic level but are\nsemantically equivalent based on computer science domain knowledge, as well as\nnon-paraphrases that overlap greatly at the lexical and syntactic level but are\nnot semantically equivalent based on this domain knowledge. Experiments show\nthat both state-of-the-art neural models and non-expert human annotators have\npoor performance on PARADE. For example, BERT after fine-tuning achieves an F1\nscore of 0.709, which is much lower than its performance on other paraphrase\nidentification datasets. PARADE can serve as a resource for researchers\ninterested in testing models that incorporate domain knowledge. We make our\ndata and code freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03726v1",
            "date": "2020-10-08 02:01:35+00:00",
            "title": "Learning to Fuse Sentences with Transformers for Summarization",
            "authors": [
                "Logan Lebanoff",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Lidan Wang",
                "Walter Chang",
                "Fei Liu"
            ],
            "abstract": "The ability to fuse sentences is highly attractive for summarization systems\nbecause it is an essential step to produce succinct abstracts. However, to\ndate, summarizers can fail on fusing sentences. They tend to produce few\nsummary sentences by fusion or generate incorrect fusions that lead the summary\nto fail to retain the original meaning. In this paper, we explore the ability\nof Transformers to fuse sentences and propose novel algorithms to enhance their\nability to perform sentence fusion by leveraging the knowledge of points of\ncorrespondence between sentences. Through extensive experiments, we investigate\nthe effects of different design choices on Transformer's performance. Our\nfindings highlight the importance of modeling points of correspondence between\nsentences for effective sentence fusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03732v2",
            "date": "2020-10-08 02:26:22+00:00",
            "title": "Leveraging Discourse Rewards for Document-Level Neural Machine Translation",
            "authors": [
                "Inigo Jauregi Unanue",
                "Nazanin Esmaili",
                "Gholamreza Haffari",
                "Massimo Piccardi"
            ],
            "abstract": "Document-level machine translation focuses on the translation of entire\ndocuments from a source to a target language. It is widely regarded as a\nchallenging task since the translation of the individual sentences in the\ndocument needs to retain aspects of the discourse at document level. However,\ndocument-level translation models are usually not trained to explicitly ensure\ndiscourse quality. Therefore, in this paper we propose a training approach that\nexplicitly optimizes two established discourse metrics, lexical cohesion (LC)\nand coherence (COH), by using a reinforcement learning objective. Experiments\nover four different language pairs and three translation domains have shown\nthat our training approach has been able to achieve more cohesive and coherent\ndocument translations than other competitive approaches, yet without\ncompromising the faithfulness to the reference translation. In the case of the\nZh-En language pair, our method has achieved an improvement of 2.46 percentage\npoints (pp) in LC and 1.17 pp in COH over the runner-up, while at the same time\nimproving 0.63 pp in BLEU score and 0.47 pp in F_BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03737v1",
            "date": "2020-10-08 02:36:07+00:00",
            "title": "Shallow-to-Deep Training for Neural Machine Translation",
            "authors": [
                "Bei Li",
                "Ziyang Wang",
                "Hui Liu",
                "Yufan Jiang",
                "Quan Du",
                "Tong Xiao",
                "Huizhen Wang",
                "Jingbo Zhu"
            ],
            "abstract": "Deep encoders have been proven to be effective in improving neural machine\ntranslation (NMT) systems, but training an extremely deep encoder is time\nconsuming. Moreover, why deep models help NMT is an open question. In this\npaper, we investigate the behavior of a well-tuned deep Transformer system. We\nfind that stacking layers is helpful in improving the representation ability of\nNMT models and adjacent layers perform similarly. This inspires us to develop a\nshallow-to-deep training method that learns deep models by stacking shallow\nmodels. In this way, we successfully train a Transformer system with a 54-layer\nencoder. Experimental results on WMT'16 English-German and WMT'14\nEnglish-French translation tasks show that it is $1.4$ $\\times$ faster than\ntraining from scratch, and achieves a BLEU score of $30.33$ and $43.29$ on two\ntasks. The code is publicly available at\nhttps://github.com/libeineu/SDT-Training/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03738v1",
            "date": "2020-10-08 02:36:39+00:00",
            "title": "Multi-hop Inference for Question-driven Summarization",
            "authors": [
                "Yang Deng",
                "Wenxuan Zhang",
                "Wai Lam"
            ],
            "abstract": "Question-driven summarization has been recently studied as an effective\napproach to summarizing the source document to produce concise but informative\nanswers for non-factoid questions. In this work, we propose a novel\nquestion-driven abstractive summarization method, Multi-hop Selective Generator\n(MSG), to incorporate multi-hop reasoning into question-driven summarization\nand, meanwhile, provide justifications for the generated summaries.\nSpecifically, we jointly model the relevance to the question and the\ninterrelation among different sentences via a human-like multi-hop inference\nmodule, which captures important sentences for justifying the summarized\nanswer. A gated selective pointer generator network with a multi-view coverage\nmechanism is designed to integrate diverse information from different\nperspectives. Experimental results show that the proposed method consistently\noutperforms state-of-the-art methods on two non-factoid QA datasets, namely\nWikiHow and PubMedQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03746v1",
            "date": "2020-10-08 03:14:38+00:00",
            "title": "Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition",
            "authors": [
                "Yun He",
                "Ziwei Zhu",
                "Yin Zhang",
                "Qin Chen",
                "James Caverlee"
            ],
            "abstract": "Knowledge of a disease includes information of various aspects of the\ndisease, such as signs and symptoms, diagnosis and treatment. This disease\nknowledge is critical for many health-related and biomedical tasks, including\nconsumer health question answering, medical language inference and disease name\nrecognition. While pre-trained language models like BERT have shown success in\ncapturing syntactic, semantic, and world knowledge from text, we find they can\nbe further complemented by specific information like knowledge of symptoms,\ndiagnoses, treatments, and other disease aspects. Hence, we integrate BERT with\ndisease knowledge for improving these important tasks. Specifically, we propose\na new disease knowledge infusion training procedure and evaluate it on a suite\nof BERT models including BERT, BioBERT, SciBERT, ClinicalBERT, BlueBERT, and\nALBERT. Experiments over the three tasks show that these models can be enhanced\nin nearly all cases, demonstrating the viability of disease knowledge infusion.\nFor example, accuracy of BioBERT on consumer health question answering is\nimproved from 68.29% to 72.09%, while new SOTA results are observed in two\ndatasets. We make our data and code freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03763v2",
            "date": "2020-10-08 04:59:39+00:00",
            "title": "Assessing Phrasal Representation and Composition in Transformers",
            "authors": [
                "Lang Yu",
                "Allyson Ettinger"
            ],
            "abstract": "Deep transformer models have pushed performance on NLP tasks to new limits,\nsuggesting sophisticated treatment of complex linguistic inputs, such as\nphrases. However, we have limited understanding of how these models handle\nrepresentation of phrases, and whether this reflects sophisticated composition\nof phrase meaning like that done by humans. In this paper, we present\nsystematic analysis of phrasal representations in state-of-the-art pre-trained\ntransformers. We use tests leveraging human judgments of phrase similarity and\nmeaning shift, and compare results before and after control of word overlap, to\ntease apart lexical effects versus composition effects. We find that phrase\nrepresentation in these models relies heavily on word content, with little\nevidence of nuanced composition. We also identify variations in phrase\nrepresentation quality across models, layers, and representation types, and\nmake corresponding recommendations for usage of representations from these\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03766v1",
            "date": "2020-10-08 05:12:52+00:00",
            "title": "Improving Attention Mechanism with Query-Value Interaction",
            "authors": [
                "Chuhan Wu",
                "Fangzhao Wu",
                "Tao Qi",
                "Yongfeng Huang"
            ],
            "abstract": "Attention mechanism has played critical roles in various state-of-the-art NLP\nmodels such as Transformer and BERT. It can be formulated as a ternary function\nthat maps the input queries, keys and values into an output by using a\nsummation of values weighted by the attention weights derived from the\ninteractions between queries and keys. Similar with query-key interactions,\nthere is also inherent relatedness between queries and values, and\nincorporating query-value interactions has the potential to enhance the output\nby learning customized values according to the characteristics of queries.\nHowever, the query-value interactions are ignored by existing attention\nmethods, which may be not optimal. In this paper, we propose to improve the\nexisting attention mechanism by incorporating query-value interactions. We\npropose a query-value interaction function which can learn query-aware\nattention values, and combine them with the original values and attention\nweights to form the final output. Extensive experiments on four datasets for\ndifferent tasks show that our approach can consistently improve the performance\nof many attention-based models by incorporating query-value interactions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03773v2",
            "date": "2020-10-08 05:34:43+00:00",
            "title": "Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention",
            "authors": [
                "Yang Li",
                "Tao Shen",
                "Guodong Long",
                "Jing Jiang",
                "Tianyi Zhou",
                "Chengqi Zhang"
            ],
            "abstract": "Wrong labeling problem and long-tail relations are two main challenges caused\nby distant supervision in relation extraction. Recent works alleviate the wrong\nlabeling by selective attention via multi-instance learning, but cannot well\nhandle long-tail relations even if hierarchies of the relations are introduced\nto share knowledge. In this work, we propose a novel neural network,\nCollaborating Relation-augmented Attention (CoRA), to handle both the wrong\nlabeling and long-tail relations. Particularly, we first propose\nrelation-augmented attention network as base model. It operates on sentence bag\nwith a sentence-to-relation attention to minimize the effect of wrong labeling.\nThen, facilitated by the proposed base model, we introduce collaborating\nrelation features shared among relations in the hierarchies to promote the\nrelation-augmenting process and balance the training data for long-tail\nrelations. Besides the main training objective to predict the relation of a\nsentence bag, an auxiliary objective is utilized to guide the\nrelation-augmenting process for a more accurate bag-level representation. In\nthe experiments on the popular benchmark dataset NYT, the proposed CoRA\nimproves the prior state-of-the-art performance by a large margin in terms of\nPrecision@N, AUC and Hits@K. Further analyses verify its superior capability in\nhandling long-tail relations in contrast to the competitors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03776v2",
            "date": "2020-10-08 05:39:00+00:00",
            "title": "Detect All Abuse! Toward Universal Abusive Language Detection Models",
            "authors": [
                "Kunze Wang",
                "Dong Lu",
                "Soyeon Caren Han",
                "Siqu Long",
                "Josiah Poon"
            ],
            "abstract": "Online abusive language detection (ALD) has become a societal issue of\nincreasing importance in recent years. Several previous works in online ALD\nfocused on solving a single abusive language problem in a single domain, like\nTwitter, and have not been successfully transferable to the general ALD task or\ndomain. In this paper, we introduce a new generic ALD framework, MACAS, which\nis capable of addressing several types of ALD tasks across different domains.\nOur generic framework covers multi-aspect abusive language embeddings that\nrepresent the target and content aspects of abusive language and applies a\ntextual graph embedding that analyses the user's linguistic behaviour. Then, we\npropose and use the cross-attention gate flow mechanism to embrace multiple\naspects of abusive language. Quantitative and qualitative evaluation results\nshow that our ALD algorithm rivals or exceeds the six state-of-the-art ALD\nalgorithms across seven ALD datasets covering multiple aspects of abusive\nlanguage and different online community domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03851v1",
            "date": "2020-10-08 09:10:55+00:00",
            "title": "Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders",
            "authors": [
                "Jue Wang",
                "Wei Lu"
            ],
            "abstract": "Named entity recognition and relation extraction are two important\nfundamental problems. Joint learning algorithms have been proposed to solve\nboth tasks simultaneously, and many of them cast the joint task as a\ntable-filling problem. However, they typically focused on learning a single\nencoder (usually learning representation in the form of a table) to capture\ninformation required for both tasks within the same space. We argue that it can\nbe beneficial to design two distinct encoders to capture such two different\ntypes of information in the learning process. In this work, we propose the\nnovel {\\em table-sequence encoders} where two different encoders -- a table\nencoder and a sequence encoder are designed to help each other in the\nrepresentation learning process. Our experiments confirm the advantages of\nhaving {\\em two} encoders over {\\em one} encoder. On several standard datasets,\nour model shows significant improvements over existing approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03880v3",
            "date": "2020-10-08 10:16:52+00:00",
            "title": "A Co-Interactive Transformer for Joint Slot Filling and Intent Detection",
            "authors": [
                "Libo Qin",
                "Tailu Liu",
                "Wanxiang Che",
                "Bingbing Kang",
                "Sendong Zhao",
                "Ting Liu"
            ],
            "abstract": "Intent detection and slot filling are two main tasks for building a spoken\nlanguage understanding (SLU) system. The two tasks are closely related and the\ninformation of one task can be utilized in the other task. Previous studies\neither model the two tasks separately or only consider the single information\nflow from intent to slot. None of the prior approaches model the bidirectional\nconnection between the two tasks simultaneously. In this paper, we propose a\nCo-Interactive Transformer to consider the cross-impact between the two tasks.\nInstead of adopting the self-attention mechanism in vanilla Transformer, we\npropose a co-interactive module to consider the cross-impact by building a\nbidirectional connection between the two related tasks. In addition, the\nproposed co-interactive module can be stacked to incrementally enhance each\nother with mutual features. The experimental results on two public datasets\n(SNIPS and ATIS) show that our model achieves the state-of-the-art performance\nwith considerable improvements (+3.4% and +0.9% on overall acc). Extensive\nexperiments empirically verify that our model successfully captures the mutual\ninteraction knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03881v1",
            "date": "2020-10-08 10:19:50+00:00",
            "title": "Large Product Key Memory for Pretrained Language Models",
            "authors": [
                "Gyuwan Kim",
                "Tae-Hwan Jung"
            ],
            "abstract": "Product key memory (PKM) proposed by Lample et al. (2019) enables to improve\nprediction accuracy by increasing model capacity efficiently with insignificant\ncomputational overhead. However, their empirical application is only limited to\ncausal language modeling. Motivated by the recent success of pretrained\nlanguage models (PLMs), we investigate how to incorporate large PKM into PLMs\nthat can be finetuned for a wide variety of downstream NLP tasks. We define a\nnew memory usage metric, and careful observation using this metric reveals that\nmost memory slots remain outdated during the training of PKM-augmented models.\nTo train better PLMs by tackling this issue, we propose simple but effective\nsolutions: (1) initialization from the model weights pretrained without memory\nand (2) augmenting PKM by addition rather than replacing a feed-forward\nnetwork. We verify that both of them are crucial for the pretraining of\nPKM-augmented PLMs, enhancing memory utilization and downstream performance.\nCode and pretrained weights are available at\nhttps://github.com/clovaai/pkm-transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03903v2",
            "date": "2020-10-08 11:11:05+00:00",
            "title": "Injecting Word Information with Multi-Level Word Adapter for Chinese Spoken Language Understanding",
            "authors": [
                "Dechuang Teng",
                "Libo Qin",
                "Wanxiang Che",
                "Sendong Zhao",
                "Ting Liu"
            ],
            "abstract": "In this paper, we improve Chinese spoken language understanding (SLU) by\ninjecting word information. Previous studies on Chinese SLU do not consider the\nword information, failing to detect word boundaries that are beneficial for\nintent detection and slot filling. To address this issue, we propose a\nmulti-level word adapter to inject word information for Chinese SLU, which\nconsists of (1) sentence-level word adapter, which directly fuses the sentence\nrepresentations of the word information and character information to perform\nintent detection and (2) character-level word adapter, which is applied at each\ncharacter for selectively controlling weights on word information as well as\ncharacter information. Experimental results on two Chinese SLU datasets show\nthat our model can capture useful word information and achieve state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03982v1",
            "date": "2020-10-08 13:56:09+00:00",
            "title": "Generating Instructions at Different Levels of Abstraction",
            "authors": [
                "Arne K\u00f6hn",
                "Julia Wichlacz",
                "\u00c1lvaro Torralba",
                "Daniel H\u00f6ller",
                "J\u00f6rg Hoffmann",
                "Alexander Koller"
            ],
            "abstract": "When generating technical instructions, it is often convenient to describe\ncomplex objects in the world at different levels of abstraction. A novice user\nmight need an object explained piece by piece, while for an expert, talking\nabout the complex object (e.g. a wall or railing) directly may be more succinct\nand efficient. We show how to generate building instructions at different\nlevels of abstraction in Minecraft. We introduce the use of hierarchical\nplanning to this end, a method from AI planning which can capture the structure\nof complex objects neatly. A crowdsourcing evaluation shows that the choice of\nabstraction level matters to users, and that an abstraction strategy which\nbalances low-level and high-level object descriptions compares favorably to\nones which don't.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03994v1",
            "date": "2020-10-08 14:07:32+00:00",
            "title": "GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems",
            "authors": [
                "Lishan Huang",
                "Zheng Ye",
                "Jinghui Qin",
                "Liang Lin",
                "Xiaodan Liang"
            ],
            "abstract": "Automatically evaluating dialogue coherence is a challenging but high-demand\nability for developing high-quality open-domain dialogue systems. However,\ncurrent evaluation metrics consider only surface features or utterance-level\nsemantics, without explicitly considering the fine-grained topic transition\ndynamics of dialogue flows. Here, we first consider that the graph structure\nconstituted with topics in a dialogue can accurately depict the underlying\ncommunication logic, which is a more natural way to produce persuasive metrics.\nCapitalized on the topic-level dialogue graph, we propose a new evaluation\nmetric GRADE, which stands for Graph-enhanced Representations for Automatic\nDialogue Evaluation. Specifically, GRADE incorporates both coarse-grained\nutterance-level contextualized representations and fine-grained topic-level\ngraph representations to evaluate dialogue coherence. The graph representations\nare obtained by reasoning over topic-level dialogue graphs enhanced with the\nevidence from a commonsense graph, including k-hop neighboring representations\nand hop-attention weights. Experimental results show that our GRADE\nsignificantly outperforms other state-of-the-art metrics on measuring diverse\ndialogue models in terms of the Pearson and Spearman correlations with human\njudgements. Besides, we release a new large-scale human evaluation benchmark to\nfacilitate future research on automatic metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04043v1",
            "date": "2020-10-08 15:10:47+00:00",
            "title": "Precise Task Formalization Matters in Winograd Schema Evaluations",
            "authors": [
                "Haokun Liu",
                "William Huang",
                "Dhara A. Mungra",
                "Samuel R. Bowman"
            ],
            "abstract": "Performance on the Winograd Schema Challenge (WSC), a respected English\ncommonsense reasoning benchmark, recently rocketed from chance accuracy to 89%\non the SuperGLUE leaderboard, with relatively little corroborating evidence of\na correspondingly large improvement in reasoning ability. We hypothesize that\nmuch of this improvement comes from recent changes in task formalization---the\ncombination of input specification, loss function, and reuse of pretrained\nparameters---by users of the dataset, rather than improvements in the\npretrained model's reasoning ability. We perform an ablation on two Winograd\nSchema datasets that interpolates between the formalizations used before and\nafter this surge, and find (i) framing the task as multiple choice improves\nperformance by 2-6 points and (ii) several additional techniques, including the\nreuse of a pretrained language modeling head, can mitigate the model's extreme\nsensitivity to hyperparameters. We urge future benchmark creators to impose\nadditional structure to minimize the impact of formalization decisions on\nreported results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04141v2",
            "date": "2020-10-08 17:36:34+00:00",
            "title": "DART: A Lightweight Quality-Suggestive Data-to-Text Annotation Tool",
            "authors": [
                "Ernie Chang",
                "Jeriah Caplinger",
                "Alex Marin",
                "Xiaoyu Shen",
                "Vera Demberg"
            ],
            "abstract": "We present a lightweight annotation tool, the Data AnnotatoR Tool (DART), for\nthe general task of labeling structured data with textual descriptions. The\ntool is implemented as an interactive application that reduces human efforts in\nannotating large quantities of structured data, e.g. in the format of a table\nor tree structure. By using a backend sequence-to-sequence model, our system\niteratively analyzes the annotated labels in order to better sample unlabeled\ndata. In a simulation experiment performed on annotating large quantities of\nstructured data, DART has been shown to reduce the total number of annotations\nneeded with active learning and automatically suggesting relevant labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04191v2",
            "date": "2020-10-08 18:09:45+00:00",
            "title": "PoinT-5: Pointer Network and T-5 based Financial NarrativeSummarisation",
            "authors": [
                "Abhishek Singh"
            ],
            "abstract": "Companies provide annual reports to their shareholders at the end of the\nfinancial year that describes their operations and financial conditions. The\naverage length of these reports is 80, and it may extend up to 250 pages long.\nIn this paper, we propose our methodology PoinT-5 (the combination of Pointer\nNetwork and T-5 (Test-to-text transfer Transformer) algorithms) that we used in\nthe Financial Narrative Summarisation (FNS) 2020 task. The proposed method uses\npointer networks to extract important narrative sentences from the report, and\nthen T-5 is used to paraphrase extracted sentences into a concise yet\ninformative sentence. We evaluate our method using ROUGE-N (1,2), L, and SU4.\nThe proposed method achieves the highest precision scores in all the metrics\nand highest F1 scores in ROUGE1, and LCS and the only solution to cross the\nMUSE solution baseline in ROUGE-LCS metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04246v2",
            "date": "2020-10-08 20:14:41+00:00",
            "title": "Dual Inference for Improving Language Understanding and Generation",
            "authors": [
                "Shang-Yu Su",
                "Yung-Sung Chuang",
                "Yun-Nung Chen"
            ],
            "abstract": "Natural language understanding (NLU) and Natural language generation (NLG)\ntasks hold a strong dual relationship, where NLU aims at predicting semantic\nlabels based on natural language utterances and NLG does the opposite. The\nprior work mainly focused on exploiting the duality in model training in order\nto obtain the models with better performance. However, regarding the\nfast-growing scale of models in the current NLP area, sometimes we may have\ndifficulty retraining whole NLU and NLG models. To better address the issue,\nthis paper proposes to leverage the duality in the inference stage without the\nneed of retraining. The experiments on three benchmark datasets demonstrate the\neffectiveness of the proposed method in both NLU and NLG, providing the great\npotential of practical usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04249v1",
            "date": "2020-10-08 20:26:34+00:00",
            "title": "Evaluating the Effectiveness of Efficient Neural Architecture Search for Sentence-Pair Tasks",
            "authors": [
                "Ansel MacLaughlin",
                "Jwala Dhamala",
                "Anoop Kumar",
                "Sriram Venkatapathy",
                "Ragav Venkatesan",
                "Rahul Gupta"
            ],
            "abstract": "Neural Architecture Search (NAS) methods, which automatically learn entire\nneural model or individual neural cell architectures, have recently achieved\ncompetitive or state-of-the-art (SOTA) performance on variety of natural\nlanguage processing and computer vision tasks, including language modeling,\nnatural language inference, and image classification. In this work, we explore\nthe applicability of a SOTA NAS algorithm, Efficient Neural Architecture Search\n(ENAS) (Pham et al., 2018) to two sentence pair tasks, paraphrase detection and\nsemantic textual similarity. We use ENAS to perform a micro-level search and\nlearn a task-optimized RNN cell architecture as a drop-in replacement for an\nLSTM. We explore the effectiveness of ENAS through experiments on three\ndatasets (MRPC, SICK, STS-B), with two different models (ESIM, BiLSTM-Max), and\ntwo sets of embeddings (Glove, BERT). In contrast to prior work applying ENAS\nto NLP tasks, our results are mixed -- we find that ENAS architectures\nsometimes, but not always, outperform LSTMs and perform similarly to random\narchitecture search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04288v1",
            "date": "2020-10-08 22:44:19+00:00",
            "title": "On the Role of Style in Parsing Speech with Neural Models",
            "authors": [
                "Trang Tran",
                "Jiahong Yuan",
                "Yang Liu",
                "Mari Ostendorf"
            ],
            "abstract": "The differences in written text and conversational speech are substantial;\nprevious parsers trained on treebanked text have given very poor results on\nspontaneous speech. For spoken language, the mismatch in style also extends to\nprosodic cues, though it is less well understood. This paper re-examines the\nuse of written text in parsing speech in the context of recent advances in\nneural language processing. We show that neural approaches facilitate using\nwritten text to improve parsing of spontaneous speech, and that prosody further\nimproves over this state-of-the-art result. Further, we find an asymmetric\ndegradation from read vs. spontaneous mismatch, with spontaneous speech more\ngenerally useful for training parsers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04293v1",
            "date": "2020-10-08 22:51:25+00:00",
            "title": "Analysis of Disfluency in Children's Speech",
            "authors": [
                "Trang Tran",
                "Morgan Tinkler",
                "Gary Yeung",
                "Abeer Alwan",
                "Mari Ostendorf"
            ],
            "abstract": "Disfluencies are prevalent in spontaneous speech, as shown in many studies of\nadult speech. Less is understood about children's speech, especially in\npre-school children who are still developing their language skills. We present\na novel dataset with annotated disfluencies of spontaneous explanations from 26\nchildren (ages 5--8), interviewed twice over a year-long period. Our\npreliminary analysis reveals significant differences between children's speech\nin our corpus and adult spontaneous speech from two corpora (Switchboard and\nCallHome). Children have higher disfluency and filler rates, tend to use nasal\nfilled pauses more frequently, and on average exhibit longer reparandums than\nrepairs, in contrast to adult speakers. Despite the differences, an automatic\ndisfluency detection system trained on adult (Switchboard) speech transcripts\nperforms reasonably well on children's speech, achieving an F1 score that is\n10\\% higher than the score on an adult out-of-domain dataset (CallHome).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04297v3",
            "date": "2020-10-08 23:16:26+00:00",
            "title": "Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task",
            "authors": [
                "Thibault Sellam",
                "Amy Pu",
                "Hyung Won Chung",
                "Sebastian Gehrmann",
                "Qijun Tan",
                "Markus Freitag",
                "Dipanjan Das",
                "Ankur P. Parikh"
            ],
            "abstract": "The quality of machine translation systems has dramatically improved over the\nlast decade, and as a result, evaluation has become an increasingly challenging\nproblem. This paper describes our contribution to the WMT 2020 Metrics Shared\nTask, the main benchmark for automatic evaluation of translation. We make\nseveral submissions based on BLEURT, a previously published metric based on\ntransfer learning. We extend the metric beyond English and evaluate it on 14\nlanguage pairs for which fine-tuning data is available, as well as 4\n\"zero-shot\" language pairs, for which we have no labelled examples.\nAdditionally, we focus on English to German and demonstrate how to combine\nBLEURT's predictions with those of YiSi and use alternative reference\ntranslations to enhance the performance. Empirical results show that the models\nachieve competitive results on the WMT Metrics 2019 Shared Task, indicating\ntheir promise for the 2020 edition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04314v1",
            "date": "2020-10-09 01:05:32+00:00",
            "title": "Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning",
            "authors": [
                "Xiaomian Kang",
                "Yang Zhao",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "Document-level neural machine translation has yielded attractive\nimprovements. However, majority of existing methods roughly use all context\nsentences in a fixed scope. They neglect the fact that different source\nsentences need different sizes of context. To address this problem, we propose\nan effective approach to select dynamic context so that the document-level\ntranslation model can utilize the more useful selected context sentences to\nproduce better translations. Specifically, we introduce a selection module that\nis independent of the translation module to score each candidate context\nsentence. Then, we propose two strategies to explicitly select a variable\nnumber of context sentences and feed them into the translation module. We train\nthe two modules end-to-end via reinforcement learning. A novel reward is\nproposed to encourage the selection and utilization of dynamic context\nsentences. Experiments demonstrate that our approach can select adaptive\ncontext sentences for different source sentences, and significantly improves\nthe performance of document-level translation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04332v1",
            "date": "2020-10-09 02:35:14+00:00",
            "title": "Langsmith: An Interactive Academic Text Revision System",
            "authors": [
                "Takumi Ito",
                "Tatsuki Kuribayashi",
                "Masatoshi Hidaka",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Despite the current diversity and inclusion initiatives in the academic\ncommunity, researchers with a non-native command of English still face\nsignificant obstacles when writing papers in English. This paper presents the\nLangsmith editor, which assists inexperienced, non-native researchers to write\nEnglish papers, especially in the natural language processing (NLP) field. Our\nsystem can suggest fluent, academic-style sentences to writers based on their\nrough, incomplete phrases or sentences. The system also encourages interaction\nbetween human writers and the computerized revision system. The experimental\nresults demonstrated that Langsmith helps non-native English-speaker students\nwrite papers in English. The system is available at https://emnlp-demo.editor.\nlangsmith.co.jp/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04362v1",
            "date": "2020-10-09 04:07:52+00:00",
            "title": "Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers",
            "authors": [
                "Brian Lester",
                "Daniel Pressel",
                "Amy Hemmeter",
                "Sagnik Ray Choudhury",
                "Srinivas Bangalore"
            ],
            "abstract": "Current state-of-the-art models for named entity recognition (NER) are neural\nmodels with a conditional random field (CRF) as the final layer. Entities are\nrepresented as per-token labels with a special structure in order to decode\nthem into spans. Current work eschews prior knowledge of how the span encoding\nscheme works and relies on the CRF learning which transitions are illegal and\nwhich are not to facilitate global coherence. We find that by constraining the\noutput to suppress illegal transitions we can train a tagger with a\ncross-entropy loss twice as fast as a CRF with differences in F1 that are\nstatistically insignificant, effectively eliminating the need for a CRF. We\nanalyze the dynamics of tag co-occurrence to explain when these constraints are\nmost effective and provide open source implementations of our tagger in both\nPyTorch and TensorFlow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04373v1",
            "date": "2020-10-09 05:03:48+00:00",
            "title": "iobes: A Library for Span-Level Processing",
            "authors": [
                "Brian Lester"
            ],
            "abstract": "Many tasks in natural language processing, such as named entity recognition\nand slot-filling, involve identifying and labeling specific spans of text. In\norder to leverage common models, these tasks are often recast as sequence\nlabeling tasks. Each token is given a label and these labels are prefixed with\nspecial tokens such as B- or I-. After a model assigns labels to each token,\nthese prefixes are used to group the tokens into spans.\n  Properly parsing these annotations is critical for producing fair and\ncomparable metrics; however, despite its importance, there is not an\neasy-to-use, standardized, programmatically integratable library to help work\nwith span labeling. To remedy this, we introduce our open-source library,\niobes. iobes is used for parsing, converting, and processing spans represented\nas token level decisions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04379v1",
            "date": "2020-10-09 05:47:00+00:00",
            "title": "Q-learning with Language Model for Edit-based Unsupervised Summarization",
            "authors": [
                "Ryosuke Kohita",
                "Akifumi Wachi",
                "Yang Zhao",
                "Ryuki Tachibana"
            ],
            "abstract": "Unsupervised methods are promising for abstractive text summarization in that\nthe parallel corpora is not required. However, their performance is still far\nfrom being satisfied, therefore research on promising solutions is on-going. In\nthis paper, we propose a new approach based on Q-learning with an edit-based\nsummarization. The method combines two key modules to form an Editorial Agent\nand Language Model converter (EALM). The agent predicts edit actions (e.t.,\ndelete, keep, and replace), and then the LM converter deterministically\ngenerates a summary on the basis of the action signals. Q-learning is leveraged\nto train the agent to produce proper edit actions. Experimental results show\nthat EALM delivered competitive performance compared with the previous\nencoder-decoder-based methods, even with truly zero paired data (i.e., no\nvalidation set). Defining the task as Q-learning enables us not only to develop\na competitive method but also to make the latest techniques in reinforcement\nlearning available for unsupervised summarization. We also conduct qualitative\nanalysis, providing insights into future study on unsupervised summarizers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04380v1",
            "date": "2020-10-09 05:55:05+00:00",
            "title": "Token-level Adaptive Training for Neural Machine Translation",
            "authors": [
                "Shuhao Gu",
                "Jinchao Zhang",
                "Fandong Meng",
                "Yang Feng",
                "Wanying Xie",
                "Jie Zhou",
                "Dong Yu"
            ],
            "abstract": "There exists a token imbalance phenomenon in natural language as different\ntokens appear with different frequencies, which leads to different learning\ndifficulties for tokens in Neural Machine Translation (NMT). The vanilla NMT\nmodel usually adopts trivial equal-weighted objectives for target tokens with\ndifferent frequencies and tends to generate more high-frequency tokens and less\nlow-frequency tokens compared with the golden token distribution. However,\nlow-frequency tokens may carry critical semantic information that will affect\nthe translation quality once they are neglected. In this paper, we explored\ntarget token-level adaptive objectives based on token frequencies to assign\nappropriate weights for each target token during training. We aimed that those\nmeaningful but relatively low-frequency words could be assigned with larger\nweights in objectives to encourage the model to pay more attention to these\ntokens. Our method yields consistent improvements in translation quality on\nZH-EN, EN-RO, and EN-DE translation tasks, especially on sentences that contain\nmore low-frequency tokens where we can get 1.68, 1.02, and 0.52 BLEU increases\ncompared with baseline, respectively. Further analyses show that our method can\nalso improve the lexical diversity of translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04383v1",
            "date": "2020-10-09 06:03:46+00:00",
            "title": "Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation",
            "authors": [
                "Yan Zhang",
                "Zhijiang Guo",
                "Zhiyang Teng",
                "Wei Lu",
                "Shay B. Cohen",
                "Zuozhu Liu",
                "Lidong Bing"
            ],
            "abstract": "AMR-to-text generation is used to transduce Abstract Meaning Representation\nstructures (AMR) into text. A key challenge in this task is to efficiently\nlearn effective graph representations. Previously, Graph Convolution Networks\n(GCNs) were used to encode input AMRs, however, vanilla GCNs are not able to\ncapture non-local information and additionally, they follow a local\n(first-order) information aggregation scheme. To account for these issues,\nlarger and deeper GCN models are required to capture more complex interactions.\nIn this paper, we introduce a dynamic fusion mechanism, proposing Lightweight\nDynamic Graph Convolutional Networks (LDGCNs) that capture richer non-local\ninteractions by synthesizing higher order information from the input graphs. We\nfurther develop two novel parameter saving strategies based on the group graph\nconvolutions and weight tied convolutions to reduce memory usage and model\ncomplexity. With the help of these strategies, we are able to train a model\nwith fewer parameters while maintaining the model capacity. Experiments\ndemonstrate that LDGCNs outperform state-of-the-art models on two benchmark\ndatasets for AMR-to-text generation with significantly fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04395v1",
            "date": "2020-10-09 07:07:04+00:00",
            "title": "gundapusunil at SemEval-2020 Task 9: Syntactic Semantic LSTM Architecture for SENTIment Analysis of Code-MIXed Data",
            "authors": [
                "Sunil Gundapu",
                "Radhika Mamidi"
            ],
            "abstract": "The phenomenon of mixing the vocabulary and syntax of multiple languages\nwithin the same utterance is called Code-Mixing. This is more evident in\nmultilingual societies. In this paper, we have developed a system for SemEval\n2020: Task 9 on Sentiment Analysis for Code-Mixed Social Media Text. Our system\nfirst generates two types of embeddings for the social media text. In those,\nthe first one is character level embeddings to encode the character level\ninformation and to handle the out-of-vocabulary entries and the second one is\nFastText word embeddings for capturing morphology and semantics. These two\nembeddings were passed to the LSTM network and the system outperformed the\nbaseline model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04411v1",
            "date": "2020-10-09 07:48:09+00:00",
            "title": "Uncertainty-Aware Semantic Augmentation for Neural Machine Translation",
            "authors": [
                "Xiangpeng Wei",
                "Heng Yu",
                "Yue Hu",
                "Rongxiang Weng",
                "Luxi Xing",
                "Weihua Luo"
            ],
            "abstract": "As a sequence-to-sequence generation task, neural machine translation (NMT)\nnaturally contains intrinsic uncertainty, where a single sentence in one\nlanguage has multiple valid counterparts in the other. However, the dominant\nmethods for NMT only observe one of them from the parallel corpora for the\nmodel training but have to deal with adequate variations under the same meaning\nat inference. This leads to a discrepancy of the data distribution between the\ntraining and the inference phases. To address this problem, we propose\nuncertainty-aware semantic augmentation, which explicitly captures the\nuniversal semantic information among multiple semantically-equivalent source\nsentences and enhances the hidden representations with this information for\nbetter translations. Extensive experiments on various translation tasks reveal\nthat our approach significantly outperforms the strong baselines and the\nexisting methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04480v1",
            "date": "2020-10-09 10:12:02+00:00",
            "title": "MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset",
            "authors": [
                "Marina Fomicheva",
                "Shuo Sun",
                "Erick Fonseca",
                "Fr\u00e9d\u00e9ric Blain",
                "Vishrav Chaudhary",
                "Francisco Guzm\u00e1n",
                "Nina Lopatina",
                "Lucia Specia",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "We present MLQE-PE, a new dataset for Machine Translation (MT) Quality\nEstimation (QE) and Automatic Post-Editing (APE). The dataset contains seven\nlanguage pairs, with human labels for 9,000 translations per language pair in\nthe following formats: sentence-level direct assessments and post-editing\neffort, and word-level good/bad labels. It also contains the post-edited\nsentences, as well as titles of the articles where the sentences were extracted\nfrom, and the neural MT models used to translate the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04482v1",
            "date": "2020-10-09 10:15:06+00:00",
            "title": "Word Level Language Identification in English Telugu Code Mixed Data",
            "authors": [
                "Sunil Gundapu",
                "Radhika Mamidi"
            ],
            "abstract": "In a multilingual or sociolingual configuration Intra-sentential Code\nSwitching (ICS) or Code Mixing (CM) is frequently observed nowadays. In the\nworld, most of the people know more than one language. CM usage is especially\napparent in social media platforms. Moreover, ICS is particularly significant\nin the context of technology, health, and law where conveying the upcoming\ndevelopments are difficult in one's native language. In applications like\ndialog systems, machine translation, semantic parsing, shallow parsing, etc. CM\nand Code Switching pose serious challenges. To do any further advancement in\ncode-mixed data, the necessary step is Language Identification. In this paper,\nwe present a study of various models - Nave Bayes Classifier, Random Forest\nClassifier, Conditional Random Field (CRF), and Hidden Markov Model (HMM) for\nLanguage Identification in English - Telugu Code Mixed Data. Considering the\npaucity of resources in code mixed languages, we proposed the CRF model and HMM\nmodel for word level language identification. Our best performing system is\nCRF-based with an f1-score of 0.91.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04520v1",
            "date": "2020-10-09 12:08:14+00:00",
            "title": "Online Back-Parsing for AMR-to-Text Generation",
            "authors": [
                "Xuefeng Bai",
                "Linfeng Song",
                "Yue Zhang"
            ],
            "abstract": "AMR-to-text generation aims to recover a text containing the same meaning as\nan input AMR graph. Current research develops increasingly powerful graph\nencoders to better represent AMR graphs, with decoders based on standard\nlanguage modeling being used to generate outputs. We propose a decoder that\nback predicts projected AMR graphs on the target sentence during text\ngeneration. As the result, our outputs can better preserve the input meaning\nthan standard decoders. Experiments on two AMR benchmarks show the superiority\nof our model over the previous state-of-the-art system based on graph\nTransformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04529v1",
            "date": "2020-10-09 12:39:33+00:00",
            "title": "What Have We Achieved on Text Summarization?",
            "authors": [
                "Dandan Huang",
                "Leyang Cui",
                "Sen Yang",
                "Guangsheng Bao",
                "Kun Wang",
                "Jun Xie",
                "Yue Zhang"
            ],
            "abstract": "Deep learning has led to significant improvement in text summarization with\nvarious methods investigated and improved ROUGE scores reported over the years.\nHowever, gaps still exist between summaries produced by automatic summarizers\nand human professionals. Aiming to gain more understanding of summarization\nsystems with respect to their strengths and limits on a fine-grained syntactic\nand semantic level, we consult the Multidimensional Quality Metric(MQM) and\nquantify 8 major sources of errors on 10 representative summarization models\nmanually. Primarily, we find that 1) under similar settings, extractive\nsummarizers are in general better than their abstractive counterparts thanks to\nstrength in faithfulness and factual-consistency; 2) milestone techniques such\nas copy, coverage and hybrid extractive/abstractive methods do bring specific\nimprovements but also demonstrate limitations; 3) pre-training techniques, and\nin particular sequence-to-sequence pre-training, are highly effective for\nimproving text summarization, with BART giving the best results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04582v1",
            "date": "2020-10-09 13:57:52+00:00",
            "title": "Denoising Multi-Source Weak Supervision for Neural Text Classification",
            "authors": [
                "Wendi Ren",
                "Yinghao Li",
                "Hanting Su",
                "David Kartchner",
                "Cassie Mitchell",
                "Chao Zhang"
            ],
            "abstract": "We study the problem of learning neural text classifiers without using any\nlabeled data, but only easy-to-provide rules as multiple weak supervision\nsources. This problem is challenging because rule-induced weak labels are often\nnoisy and incomplete. To address these two challenges, we design a label\ndenoiser, which estimates the source reliability using a conditional soft\nattention mechanism and then reduces label noise by aggregating rule-annotated\nweak labels. The denoised pseudo labels then supervise a neural classifier to\npredicts soft labels for unmatched samples, which address the rule coverage\nissue. We evaluate our model on five benchmarks for sentiment, topic, and\nrelation classifications. The results show that our model outperforms\nstate-of-the-art weakly-supervised and semi-supervised methods consistently,\nand achieves comparable performance with fully-supervised methods even without\nany labeled data. Our code can be found at\nhttps://github.com/weakrules/Denoise-multi-weak-sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04637v1",
            "date": "2020-10-09 15:30:05+00:00",
            "title": "Recurrent babbling: evaluating the acquisition of grammar from limited input data",
            "authors": [
                "Ludovica Pannitto",
                "Aur\u00e9lie Herbelot"
            ],
            "abstract": "Recurrent Neural Networks (RNNs) have been shown to capture various aspects\nof syntax from raw linguistic input. In most previous experiments, however,\nlearning happens over unrealistic corpora, which do not reflect the type and\namount of data a child would be exposed to. This paper remedies this state of\naffairs by training a Long Short-Term Memory network (LSTM) over a\nrealistically sized subset of child-directed input. The behaviour of the\nnetwork is analysed over time using a novel methodology which consists in\nquantifying the level of grammatical abstraction in the model's generated\noutput (its \"babbling\"), compared to the language it has been exposed to. We\nshow that the LSTM indeed abstracts new structuresas learning proceeds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04640v2",
            "date": "2020-10-09 15:33:50+00:00",
            "title": "Grid Tagging Scheme for Aspect-oriented Fine-grained Opinion Extraction",
            "authors": [
                "Zhen Wu",
                "Chengcan Ying",
                "Fei Zhao",
                "Zhifang Fan",
                "Xinyu Dai",
                "Rui Xia"
            ],
            "abstract": "Aspect-oriented Fine-grained Opinion Extraction (AFOE) aims at extracting\naspect terms and opinion terms from review in the form of opinion pairs or\nadditionally extracting sentiment polarity of aspect term to form opinion\ntriplet. Because of containing several opinion factors, the complete AFOE task\nis usually divided into multiple subtasks and achieved in the pipeline.\nHowever, pipeline approaches easily suffer from error propagation and\ninconvenience in real-world scenarios. To this end, we propose a novel tagging\nscheme, Grid Tagging Scheme (GTS), to address the AFOE task in an end-to-end\nfashion only with one unified grid tagging task. Additionally, we design an\neffective inference strategy on GTS to exploit mutual indication between\ndifferent opinion factors for more accurate extractions. To validate the\nfeasibility and compatibility of GTS, we implement three different GTS models\nrespectively based on CNN, BiLSTM, and BERT, and conduct experiments on the\naspect-oriented opinion pair extraction and opinion triplet extraction\ndatasets. Extensive experimental results indicate that GTS models outperform\nstrong baselines significantly and achieve state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04641v1",
            "date": "2020-10-09 15:33:54+00:00",
            "title": "High-order Semantic Role Labeling",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Rui Wang",
                "Kevin Parnow"
            ],
            "abstract": "Semantic role labeling is primarily used to identify predicates, arguments,\nand their semantic relationships. Due to the limitations of modeling methods\nand the conditions of pre-identified predicates, previous work has focused on\nthe relationships between predicates and arguments and the correlations between\narguments at most, while the correlations between predicates have been\nneglected for a long time. High-order features and structure learning were very\ncommon in modeling such correlations before the neural network era. In this\npaper, we introduce a high-order graph structure for the neural semantic role\nlabeling model, which enables the model to explicitly consider not only the\nisolated predicate-argument pairs but also the interaction between the\npredicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009\nbenchmark show that the high-order structural learning techniques are\nbeneficial to the strong performing SRL models and further boost our baseline\nto achieve new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04658v2",
            "date": "2020-10-09 16:04:51+00:00",
            "title": "Case Study: Deontological Ethics in NLP",
            "authors": [
                "Shrimai Prabhumoye",
                "Brendon Boldt",
                "Ruslan Salakhutdinov",
                "Alan W Black"
            ],
            "abstract": "Recent work in natural language processing (NLP) has focused on ethical\nchallenges such as understanding and mitigating bias in data and algorithms;\nidentifying objectionable content like hate speech, stereotypes and offensive\nlanguage; and building frameworks for better system design and data handling\npractices. However, there has been little discussion about the ethical\nfoundations that underlie these efforts. In this work, we study one ethical\ntheory, namely deontological ethics, from the perspective of NLP. In\nparticular, we focus on the generalization principle and the respect for\nautonomy through informed consent. We provide four case studies to demonstrate\nhow these principles can be used with NLP systems. We also recommend directions\nto avoid the ethical issues in these systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04674v1",
            "date": "2020-10-09 16:48:41+00:00",
            "title": "Learning Context-Free Languages with Nondeterministic Stack RNNs",
            "authors": [
                "Brian DuSell",
                "David Chiang"
            ],
            "abstract": "We present a differentiable stack data structure that simultaneously and\ntractably encodes an exponential number of stack configurations, based on\nLang's algorithm for simulating nondeterministic pushdown automata. We call the\ncombination of this data structure with a recurrent neural network (RNN)\ncontroller a Nondeterministic Stack RNN. We compare our model against existing\nstack RNNs on various formal languages, demonstrating that our model converges\nmore reliably to algorithmic behavior on deterministic tasks, and achieves\nlower cross-entropy on inherently nondeterministic tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04706v1",
            "date": "2020-10-09 17:50:29+00:00",
            "title": "Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty",
            "authors": [
                "Katherine A. Keith",
                "Christoph Teichmann",
                "Brendan O'Connor",
                "Edgar Meij"
            ],
            "abstract": "Methods and applications are inextricably linked in science, and in\nparticular in the domain of text-as-data. In this paper, we examine one such\ntext-as-data application, an established economic index that measures economic\npolicy uncertainty from keyword occurrences in news. This index, which is shown\nto correlate with firm investment, employment, and excess market returns, has\nhad substantive impact in both the private sector and academia. Yet, as we\nrevisit and extend the original authors' annotations and text measurements we\nfind interesting text-as-data methodological research questions: (1) Are\nannotator disagreements a reflection of ambiguity in language? (2) Do\nalternative text measurements correlate with one another and with measures of\nexternal predictive validity? We find for this application (1) some annotator\ndisagreements of economic policy uncertainty can be attributed to ambiguity in\nlanguage, and (2) switching measurements from keyword-matching to supervised\nmachine learning classifiers results in low correlation, a concerning\nimplication for the validity of the index.",
            "journal ref": "2020 Natural Language Processing + Computational Social Science\n  Workshop (NLP+CSS) at EMNLP",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04744v1",
            "date": "2020-10-09 18:03:49+00:00",
            "title": "Learning to Pronounce Chinese Without a Pronunciation Dictionary",
            "authors": [
                "Christopher Chu",
                "Scot Fang",
                "Kevin Knight"
            ],
            "abstract": "We demonstrate a program that learns to pronounce Chinese text in Mandarin,\nwithout a pronunciation dictionary. From non-parallel streams of Chinese\ncharacters and Chinese pinyin syllables, it establishes a many-to-many mapping\nbetween characters and pronunciations. Using unsupervised methods, the program\neffectively deciphers writing into speech. Its token-level\ncharacter-to-syllable accuracy is 89%, which significantly exceeds the 22%\naccuracy of prior work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04746v1",
            "date": "2020-10-09 18:04:05+00:00",
            "title": "Solving Historical Dictionary Codes with a Neural Language Model",
            "authors": [
                "Christopher Chu",
                "Raphael Valenti",
                "Kevin Knight"
            ],
            "abstract": "We solve difficult word-based substitution codes by constructing a decoding\nlattice and searching that lattice with a neural language model. We apply our\nmethod to a set of enciphered letters exchanged between US Army General James\nWilkinson and agents of the Spanish Crown in the late 1700s and early 1800s,\nobtained from the US Library of Congress. We are able to decipher 75.1% of the\ncipher-word tokens correctly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04747v1",
            "date": "2020-10-09 18:04:17+00:00",
            "title": "MEEP: An Open-Source Platform for Human-Human Dialog Collection and End-to-End Agent Training",
            "authors": [
                "Arkady Arkhangorodsky",
                "Amittai Axelrod",
                "Christopher Chu",
                "Scot Fang",
                "Yiqi Huang",
                "Ajay Nagesh",
                "Xing Shi",
                "Boliang Zhang",
                "Kevin Knight"
            ],
            "abstract": "We create a new task-oriented dialog platform (MEEP) where agents are given\nconsiderable freedom in terms of utterances and API calls, but are constrained\nto work within a push-button environment. We include facilities for collecting\nhuman-human dialog corpora, and for training automatic agents in an end-to-end\nfashion. We demonstrate MEEP with a dialog assistant that lets users specify\ntrip destinations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04755v1",
            "date": "2020-10-09 18:27:55+00:00",
            "title": "Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model",
            "authors": [
                "Jun Yen Leung",
                "Guy Emerson",
                "Ryan Cotterell"
            ],
            "abstract": "Across languages, multiple consecutive adjectives modifying a noun (e.g. \"the\nbig red dog\") follow certain unmarked ordering rules. While explanatory\naccounts have been put forward, much of the work done in this area has relied\nprimarily on the intuitive judgment of native speakers, rather than on corpus\ndata. We present the first purely corpus-driven model of multi-lingual\nadjective ordering in the form of a latent-variable model that can accurately\norder adjectives across 24 different languages, even when the training and\ntesting languages are different. We utilize this novel statistical model to\nprovide strong converging evidence for the existence of universal,\ncross-linguistic, hierarchical adjective ordering tendencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04762v1",
            "date": "2020-10-09 18:44:02+00:00",
            "title": "Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data",
            "authors": [
                "William Huang",
                "Haokun Liu",
                "Samuel R. Bowman"
            ],
            "abstract": "A growing body of work shows that models exploit annotation artifacts to\nachieve state-of-the-art performance on standard crowdsourced\nbenchmarks---datasets collected from crowdworkers to create an evaluation\ntask---while still failing on out-of-domain examples for the same task. Recent\nwork has explored the use of counterfactually-augmented data---data built by\nminimally editing a set of seed examples to yield counterfactual labels---to\naugment training data associated with these benchmarks and build more robust\nclassifiers that generalize better. However, Khashabi et al. (2020) find that\nthis type of augmentation yields little benefit on reading comprehension tasks\nwhen controlling for dataset size and cost of collection. We build upon this\nwork by using English natural language inference data to test model\ngeneralization and robustness and find that models trained on a\ncounterfactually-augmented SNLI dataset do not generalize better than\nunaugmented datasets of similar size and that counterfactual augmentation can\nhurt performance, yielding models that are less robust to challenge examples.\nCounterfactual augmentation of natural language understanding data through\nstandard crowdsourcing techniques does not appear to be an effective way of\ncollecting training data and further innovation is required to make this\ngeneral line of work viable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04806v1",
            "date": "2020-10-09 21:06:57+00:00",
            "title": "AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data",
            "authors": [
                "Silei Xu",
                "Sina J. Semnani",
                "Giovanni Campagna",
                "Monica S. Lam"
            ],
            "abstract": "We propose AutoQA, a methodology and toolkit to generate semantic parsers\nthat answer questions on databases, with no manual effort. Given a database\nschema and its data, AutoQA automatically generates a large set of high-quality\nquestions for training that covers different database operations. It uses\nautomatic paraphrasing combined with template-based parsing to find alternative\nexpressions of an attribute in different parts of speech. It also uses a novel\nfiltered auto-paraphraser to generate correct paraphrases of entire sentences.\nWe apply AutoQA to the Schema2QA dataset and obtain an average logical form\naccuracy of 62.9% when tested on natural questions, which is only 6.4% lower\nthan a model trained with expert natural language annotations and paraphrase\ndata collected from crowdworkers. To demonstrate the generality of AutoQA, we\nalso apply it to the Overnight dataset. AutoQA achieves 69.8% answer accuracy,\n16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower\nthan the same model trained with human data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04829v2",
            "date": "2020-10-09 22:23:21+00:00",
            "title": "Relation Classification as Two-way Span-Prediction",
            "authors": [
                "Amir DN Cohen",
                "Shachar Rosenman",
                "Yoav Goldberg"
            ],
            "abstract": "The current supervised relation classification (RC) task uses a single\nembedding to represent the relation between a pair of entities. We argue that a\nbetter approach is to treat the RC task as span-prediction (SP) problem,\nsimilar to Question answering (QA). We present a span-prediction based system\nfor RC and evaluate its performance compared to the embedding based system. We\ndemonstrate that the supervised SP objective works significantly better then\nthe standard classification based objective. We achieve state-of-the-art\nresults on the TACRED and SemEval task 8 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04872v1",
            "date": "2020-10-10 02:09:19+00:00",
            "title": "Self-play for Data Efficient Language Acquisition",
            "authors": [
                "Charles Lovering",
                "Ellie Pavlick"
            ],
            "abstract": "When communicating, people behave consistently across conversational roles:\nPeople understand the words they say and are able to produce the words they\nhear. To date, artificial agents developed for language tasks have lacked such\nsymmetry, meaning agents trained to produce language are unable to understand\nit and vice-versa. In this work, we exploit the symmetric nature of\ncommunication in order to improve both the efficiency and quality of language\nacquisition in learning agents. Specifically, we consider the setting in which\nan agent must learn to both understand and generate words in an existing\nlanguage, but with the assumption that access to interaction with \"oracle\"\nspeakers of the language is very limited. We show that using self-play as a\nsubstitute for direct supervision enables the agent to transfer its knowledge\nacross roles (e.g. training as a listener but testing as a speaker) and make\nbetter inferences about the ground truth lexicon using only a handful of\ninteractions with the oracle.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04887v1",
            "date": "2020-10-10 03:14:00+00:00",
            "title": "Discourse structure interacts with reference but not syntax in neural language models",
            "authors": [
                "Forrest Davis",
                "Marten van Schijndel"
            ],
            "abstract": "Language models (LMs) trained on large quantities of text have been claimed\nto acquire abstract linguistic representations. Our work tests the robustness\nof these abstractions by focusing on the ability of LMs to learn interactions\nbetween different linguistic representations. In particular, we utilized\nstimuli from psycholinguistic studies showing that humans can condition\nreference (i.e. coreference resolution) and syntactic processing on the same\ndiscourse structure (implicit causality). We compared both transformer and long\nshort-term memory LMs to find that, contrary to humans, implicit causality only\ninfluences LM behavior for reference, not syntax, despite model representations\nthat encode the necessary discourse information. Our results further suggest\nthat LM behavior can contradict not only learned representations of discourse\nbut also syntactic agreement, pointing to shortcomings of standard language\nmodeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04897v1",
            "date": "2020-10-10 04:22:07+00:00",
            "title": "Information Extraction from Swedish Medical Prescriptions with Sig-Transformer Encoder",
            "authors": [
                "John Pougue Biyong",
                "Bo Wang",
                "Terry Lyons",
                "Alejo J Nevado-Holgado"
            ],
            "abstract": "Relying on large pretrained language models such as Bidirectional Encoder\nRepresentations from Transformers (BERT) for encoding and adding a simple\nprediction layer has led to impressive performance in many clinical natural\nlanguage processing (NLP) tasks. In this work, we present a novel extension to\nthe Transformer architecture, by incorporating signature transform with the\nself-attention model. This architecture is added between embedding and\nprediction layers. Experiments on a new Swedish prescription data show the\nproposed architecture to be superior in two of the three information extraction\ntasks, comparing to baseline models. Finally, we evaluate two different\nembedding approaches between applying Multilingual BERT and translating the\nSwedish text to English then encode with a BERT model pretrained on clinical\nnotes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04926v1",
            "date": "2020-10-10 07:12:48+00:00",
            "title": "Latent Tree Learning with Ordered Neurons: What Parses Does It Produce?",
            "authors": [
                "Yian Zhang"
            ],
            "abstract": "Recent latent tree learning models can learn constituency parsing without any\nexposure to human-annotated tree structures. One such model is ON-LSTM (Shen et\nal., 2019), which is trained on language modelling and has\nnear-state-of-the-art performance on unsupervised parsing. In order to better\nunderstand the performance and consistency of the model as well as how the\nparses it generates are different from gold-standard PTB parses, we replicate\nthe model with different restarts and examine their parses. We find that (1)\nthe model has reasonably consistent parsing behaviors across different\nrestarts, (2) the model struggles with the internal structures of complex noun\nphrases, (3) the model has a tendency to overestimate the height of the split\npoints right before verbs. We speculate that both problems could potentially be\nsolved by adopting a different training task other than unidirectional language\nmodelling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04941v1",
            "date": "2020-10-10 08:34:19+00:00",
            "title": "When Hearst Is not Enough: Improving Hypernymy Detection from Corpus with Distributional Models",
            "authors": [
                "Changlong Yu",
                "Jialong Han",
                "Peifeng Wang",
                "Yangqiu Song",
                "Hongming Zhang",
                "Wilfred Ng",
                "Shuming Shi"
            ],
            "abstract": "We address hypernymy detection, i.e., whether an is-a relationship exists\nbetween words (x, y), with the help of large textual corpora. Most conventional\napproaches to this task have been categorized to be either pattern-based or\ndistributional. Recent studies suggest that pattern-based ones are superior, if\nlarge-scale Hearst pairs are extracted and fed, with the sparsity of unseen (x,\ny) pairs relieved. However, they become invalid in some specific sparsity\ncases, where x or y is not involved in any pattern. For the first time, this\npaper quantifies the non-negligible existence of those specific cases. We also\ndemonstrate that distributional methods are ideal to make up for pattern-based\nones in such cases. We devise a complementary framework, under which a\npattern-based and a distributional model collaborate seamlessly in cases which\nthey each prefer. On several benchmark datasets, our framework achieves\ncompetitive improvements and the case study shows its better interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04970v1",
            "date": "2020-10-10 10:36:58+00:00",
            "title": "MS-Ranker: Accumulating Evidence from Potentially Correct Candidates for Answer Selection",
            "authors": [
                "Yingxue Zhang",
                "Fandong Meng",
                "Peng Li",
                "Ping Jian",
                "Jie Zhou"
            ],
            "abstract": "As conventional answer selection (AS) methods generally match the question\nwith each candidate answer independently, they suffer from the lack of matching\ninformation between the question and the candidate. To address this problem, we\npropose a novel reinforcement learning (RL) based multi-step ranking model,\nnamed MS-Ranker, which accumulates information from potentially correct\ncandidate answers as extra evidence for matching the question with a candidate.\nIn specific, we explicitly consider the potential correctness of candidates and\nupdate the evidence with a gating mechanism. Moreover, as we use a listwise\nranking reward, our model learns to pay more attention to the overall\nperformance. Experiments on two benchmarks, namely WikiQA and SemEval-2016 CQA,\nshow that our model significantly outperforms existing methods that do not rely\non external resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04989v1",
            "date": "2020-10-10 13:11:41+00:00",
            "title": "Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns",
            "authors": [
                "Lei Zhou",
                "Liang Ding",
                "Koichi Takeda"
            ],
            "abstract": "This paper describes our submission of the WMT 2020 Shared Task on Sentence\nLevel Direct Assessment, Quality Estimation (QE). In this study, we empirically\nreveal the \\textit{mismatching issue} when directly adopting BERTScore to QE.\nSpecifically, there exist lots of mismatching errors between the source\nsentence and translated candidate sentence with token pairwise similarity. In\nresponse to this issue, we propose to expose explicit cross-lingual patterns,\n\\textit{e.g.} word alignments and generation score, to our proposed zero-shot\nmodels. Experiments show that our proposed QE model with explicit cross-lingual\npatterns could alleviate the mismatching issue, thereby improving the\nperformance. Encouragingly, our zero-shot QE method could achieve comparable\nperformance with supervised QE method, and even outperforms the supervised\ncounterpart on 2 out of 6 directions. We expect our work could shed light on\nthe zero-shot QE model improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05002v1",
            "date": "2020-10-10 13:47:55+00:00",
            "title": "Compressing Transformer-Based Semantic Parsing Models using Compositional Code Embeddings",
            "authors": [
                "Prafull Prakash",
                "Saurabh Kumar Shashidhar",
                "Wenlong Zhao",
                "Subendhu Rongali",
                "Haidar Khan",
                "Michael Kayser"
            ],
            "abstract": "The current state-of-the-art task-oriented semantic parsing models use BERT\nor RoBERTa as pretrained encoders; these models have huge memory footprints.\nThis poses a challenge to their deployment for voice assistants such as Amazon\nAlexa and Google Assistant on edge devices with limited memory budgets. We\npropose to learn compositional code embeddings to greatly reduce the sizes of\nBERT-base and RoBERTa-base. We also apply the technique to DistilBERT,\nALBERT-base, and ALBERT-large, three already compressed BERT variants which\nattain similar state-of-the-art performances on semantic parsing with much\nsmaller model sizes. We observe 95.15% ~ 98.46% embedding compression rates and\n20.47% ~ 34.22% encoder compression rates, while preserving greater than 97.5%\nsemantic parsing performances. We provide the recipe for training and analyze\nthe trade-off between code embedding sizes and downstream performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05096v1",
            "date": "2020-10-10 21:35:42+00:00",
            "title": "Leveraging Spatial Information in Radiology Reports for Ischemic Stroke Phenotyping",
            "authors": [
                "Surabhi Datta",
                "Shekhar Khanpara",
                "Roy F. Riascos",
                "Kirk Roberts"
            ],
            "abstract": "Classifying fine-grained ischemic stroke phenotypes relies on identifying\nimportant clinical information. Radiology reports provide relevant information\nwith context to determine such phenotype information. We focus on stroke\nphenotypes with location-specific information: brain region affected,\nlaterality, stroke stage, and lacunarity. We use an existing fine-grained\nspatial information extraction system--Rad-SpatialNet--to identify clinically\nimportant information and apply simple domain rules on the extracted\ninformation to classify phenotypes. The performance of our proposed approach is\npromising (recall of 89.62% for classifying brain region and 74.11% for\nclassifying brain region, side, and stroke stage together). Our work\ndemonstrates that an information extraction system based on a fine-grained\nschema can be utilized to determine complex phenotypes with the inclusion of\nsimple domain rules. These phenotypes have the potential to facilitate stroke\nresearch focusing on post-stroke outcome and treatment planning based on the\nstroke location.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05111v1",
            "date": "2020-10-10 22:27:17+00:00",
            "title": "Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification",
            "authors": [
                "Shyam Subramanian",
                "Kyumin Lee"
            ],
            "abstract": "Automated fact extraction and verification is a challenging task that\ninvolves finding relevant evidence sentences from a reliable corpus to verify\nthe truthfulness of a claim. Existing models either (i) concatenate all the\nevidence sentences, leading to the inclusion of redundant and noisy\ninformation; or (ii) process each claim-evidence sentence pair separately and\naggregate all of them later, missing the early combination of related sentences\nfor more accurate claim verification. Unlike the prior works, in this paper, we\npropose Hierarchical Evidence Set Modeling (HESM), a framework to extract\nevidence sets (each of which may contain multiple evidence sentences), and\nverify a claim to be supported, refuted or not enough info, by encoding and\nattending the claim and evidence sets at different levels of hierarchy. Our\nexperimental results show that HESM outperforms 7 state-of-the-art methods for\nfact extraction and claim verification. Our source code is available at\nhttps://github.com/ShyamSubramanian/HESM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05122v1",
            "date": "2020-10-11 00:40:05+00:00",
            "title": "SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task",
            "authors": [
                "Zuchao Li",
                "Hai Zhao",
                "Rui Wang",
                "Kehai Chen",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "In this paper, we introduced our joint team SJTU-NICT 's participation in the\nWMT 2020 machine translation shared task. In this shared task, we participated\nin four translation directions of three language pairs: English-Chinese,\nEnglish-Polish on supervised machine translation track, German-Upper Sorbian on\nlow-resource and unsupervised machine translation tracks. Based on different\nconditions of language pairs, we have experimented with diverse neural machine\ntranslation (NMT) techniques: document-enhanced NMT, XLM pre-trained language\nmodel enhanced NMT, bidirectional translation as a pre-training, reference\nlanguage based UNMT, data-dependent gaussian prior objective, and BT-BLEU\ncollaborative filtering self-training. We also used the TF-IDF algorithm to\nfilter the training set to obtain a domain more similar set with the test set\nfor finetuning. In our submissions, the primary systems won the first place on\nEnglish to Chinese, Polish to English, and German to Upper Sorbian translation\ndirections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05129v1",
            "date": "2020-10-11 01:16:10+00:00",
            "title": "Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions",
            "authors": [
                "Dongyeop Kang",
                "Andrew Head",
                "Risham Sidhu",
                "Kyle Lo",
                "Daniel S. Weld",
                "Marti A. Hearst"
            ],
            "abstract": "The task of definition detection is important for scholarly papers, because\npapers often make use of technical terminology that may be unfamiliar to\nreaders. Despite prior work on definition detection, current approaches are far\nfrom being accurate enough to use in real-world applications. In this paper, we\nfirst perform in-depth error analysis of the current best performing definition\ndetection system and discover major causes of errors. Based on this analysis,\nwe develop a new definition detection system, HEDDEx, that utilizes syntactic\nfeatures, transformer encoders, and heuristic filters, and evaluate it on a\nstandard sentence-level benchmark. Because current benchmarks evaluate randomly\nsampled sentences, we propose an alternative evaluation that assesses every\nsentence within a document. This allows for evaluating recall in addition to\nprecision. HEDDEx outperforms the leading system on both the sentence-level and\nthe document-level tasks, by 12.7 F1 points and 14.4 F1 points, respectively.\nWe note that performance on the high-recall document-level task is much lower\nthan in the standard evaluation approach, due to the necessity of incorporation\nof document structure as features. We discuss remaining challenges in\ndocument-level definition detection, ideas for improvements, and potential\nissues for the development of reading aid applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05139v2",
            "date": "2020-10-11 02:19:15+00:00",
            "title": "CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems",
            "authors": [
                "Yiran Chen",
                "Pengfei Liu",
                "Ming Zhong",
                "Zi-Yi Dou",
                "Danqing Wang",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Neural network-based models augmented with unsupervised pre-trained knowledge\nhave achieved impressive performance on text summarization. However, most\nexisting evaluation methods are limited to an in-domain setting, where\nsummarizers are trained and evaluated on the same dataset. We argue that this\napproach can narrow our understanding of the generalization ability for\ndifferent summarization systems. In this paper, we perform an in-depth analysis\nof characteristics of different datasets and investigate the performance of\ndifferent summarization models under a cross-dataset setting, in which a\nsummarizer trained on one corpus will be evaluated on a range of out-of-domain\ncorpora. A comprehensive study of 11 representative summarization systems on 5\ndatasets from different domains reveals the effect of model architectures and\ngeneration ways (i.e. abstractive and extractive) on model generalization\nability. Further, experimental results shed light on the limitations of\nexisting summarizers. Brief introduction and supplementary code can be found in\nhttps://github.com/zide05/CDEvalSumm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05141v1",
            "date": "2020-10-11 02:38:21+00:00",
            "title": "Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task",
            "authors": [
                "Dongyeop Kang",
                "Eduard Hovy"
            ],
            "abstract": "Despite the recent success of contextualized language models on various NLP\ntasks, language model itself cannot capture textual coherence of a long,\nmulti-sentence document (e.g., a paragraph). Humans often make structural\ndecisions on what and how to say about before making utterances. Guiding\nsurface realization with such high-level decisions and structuring text in a\ncoherent way is essentially called a planning process. Where can the model\nlearn such high-level coherence? A paragraph itself contains various forms of\ninductive coherence signals called self-supervision in this work, such as\nsentence orders, topical keywords, rhetorical structures, and so on. Motivated\nby that, this work proposes a new paragraph completion task PARCOM; predicting\nmasked sentences in a paragraph. However, the task suffers from predicting and\nselecting appropriate topical content with respect to the given context. To\naddress that, we propose a self-supervised text planner SSPlanner that predicts\nwhat to say first (content prediction), then guides the pretrained language\nmodel (surface realization) using the predicted content. SSPlanner outperforms\nthe baseline generation models on the paragraph completion task in both\nautomatic and human evaluation. We also find that a combination of noun and\nverb types of keywords is the most effective for content selection. As more\nnumber of content keywords are provided, overall generation quality also\nincreases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05143v1",
            "date": "2020-10-11 02:57:11+00:00",
            "title": "PHICON: Improving Generalization of Clinical Text De-identification Models via Data Augmentation",
            "authors": [
                "Xiang Yue",
                "Shuang Zhou"
            ],
            "abstract": "De-identification is the task of identifying protected health information\n(PHI) in the clinical text. Existing neural de-identification models often fail\nto generalize to a new dataset. We propose a simple yet effective data\naugmentation method PHICON to alleviate the generalization issue. PHICON\nconsists of PHI augmentation and Context augmentation, which creates augmented\ntraining corpora by replacing PHI entities with named-entities sampled from\nexternal sources, and by changing background context with synonym replacement\nor random word insertion, respectively. Experimental results on the i2b2 2006\nand 2014 de-identification challenge datasets show that PHICON can help three\nselected de-identification models boost F1-score (by at most 8.6%) on\ncross-dataset test setting. We also discuss how much augmentation to use and\nhow each augmentation method influences the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05164v1",
            "date": "2020-10-11 04:33:03+00:00",
            "title": "A General Model of Conversational Dynamics and an Example Application in Serious Illness Communication",
            "authors": [
                "Laurence A. Clarfeld",
                "Robert Gramling",
                "Donna M. Rizzo",
                "Margaret J. Eppstein"
            ],
            "abstract": "Conversation has been a primary means for the exchange of information since\nancient times. Understanding patterns of information flow in conversations is a\ncritical step in assessing and improving communication quality. In this paper,\nwe describe COnversational DYnamics Model (CODYM) analysis, a novel approach\nfor studying patterns of information flow in conversations. CODYMs are Markov\nModels that capture sequential dependencies in the lengths of speaker turns.\nThe proposed method is automated and scalable, and preserves the privacy of the\nconversational participants. The primary function of CODYM analysis is to\nquantify and visualize patterns of information flow, concisely summarized over\nsequential turns from one or more conversations. Our approach is general and\ncomplements existing methods, providing a new tool for use in the analysis of\nany type of conversation. As an important first application, we demonstrate the\nmodel on transcribed conversations between palliative care clinicians and\nseriously ill patients. These conversations are dynamic and complex, taking\nplace amidst heavy emotions, and include difficult topics such as end-of-life\npreferences and patient values. We perform a versatile set of CODYM analyses\nthat (a) establish the validity of the model by confirming known patterns of\nconversational turn-taking and word usage, (b) identify normative patterns of\ninformation flow in serious illness conversations, and (c) show how these\npatterns vary across narrative time and differ under expressions of anger, fear\nand sadness. Potential applications of CODYMs range from assessment and\ntraining of effective healthcare communication to comparing conversational\ndynamics across language and culture, with the prospect of identifying\nuniversal similarities and unique \"fingerprints\" of information flow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05194v1",
            "date": "2020-10-11 08:46:17+00:00",
            "title": "Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only",
            "authors": [
                "Ziyi Liu",
                "Giannis Karamanolakis",
                "Daniel Hsu",
                "Luis Gravano"
            ],
            "abstract": "Health departments have been deploying text classification systems for the\nearly detection of foodborne illness complaints in social media documents such\nas Yelp restaurant reviews. Current systems have been successfully applied for\ndocuments in English and, as a result, a promising direction is to increase\ncoverage and recall by considering documents in additional languages, such as\nSpanish or Chinese. Training previous systems for more languages, however,\nwould be expensive, as it would require the manual annotation of many documents\nfor each new target language. To address this challenge, we consider\ncross-lingual learning and train multilingual classifiers using only the\nannotations for English-language reviews. Recent zero-shot approaches based on\npre-trained multi-lingual BERT (mBERT) have been shown to effectively align\nlanguages for aspects such as sentiment. Interestingly, we show that those\napproaches are less effective for capturing the nuances of foodborne illness,\nour public health application of interest. To improve performance without extra\nannotations, we create artificial training documents in the target language\nthrough machine translation and train mBERT jointly for the source (English)\nand target language. Furthermore, we show that translating labeled documents to\nmultiple languages leads to additional performance improvements for some target\nlanguages. We demonstrate the benefits of our approach through extensive\nexperiments with Yelp restaurant reviews in seven languages. Our classifiers\nidentify foodborne illness complaints in multilingual reviews from the Yelp\nChallenge dataset, which highlights the potential of our general approach for\ndeployment in health departments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05229v1",
            "date": "2020-10-11 11:59:40+00:00",
            "title": "Machine Translation of Mathematical Text",
            "authors": [
                "Aditya Ohri",
                "Tanya Schmah"
            ],
            "abstract": "We have implemented a machine translation system, the PolyMath Translator,\nfor LaTeX documents containing mathematical text. The current implementation\ntranslates English LaTeX to French LaTeX, attaining a BLEU score of 53.5 on a\nheld-out test corpus of mathematical sentences. It produces LaTeX documents\nthat can be compiled to PDF without further editing. The system first converts\nthe body of an input LaTeX document into English sentences containing math\ntokens, using the pandoc universal document converter to parse LaTeX input. We\nhave trained a Transformer-based translator model, using OpenNMT, on a combined\ncorpus containing a small proportion of domain-specific sentences. Our full\nsystem uses both this Transformer model and Google Translate, the latter being\nused as a backup to better handle linguistic features that do not appear in our\ntraining dataset. If the Transformer model does not have confidence in its\ntranslation, as determined by a high perplexity score, then we use Google\nTranslate with a custom glossary. This backup was used 26% of the time on our\ntest corpus of mathematical sentences. The PolyMath Translator is available as\na web service at www.polymathtrans.ai.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05230v1",
            "date": "2020-10-11 12:05:00+00:00",
            "title": "Controllable Multi-Character Psychology-Oriented Story Generation",
            "authors": [
                "Feifei Xu",
                "Xinpeng Wang",
                "Yunpu Ma",
                "Volker Tresp",
                "Yuyi Wang",
                "Shanlin Zhou",
                "Haizhou Du"
            ],
            "abstract": "Story generation, which aims to generate a long and coherent story\nautomatically based on the title or an input sentence, is an important research\narea in the field of natural language generation. There is relatively little\nwork on story generation with appointed emotions. Most existing works focus on\nusing only one specific emotion to control the generation of a whole story and\nignore the emotional changes in the characters in the course of the story. In\nour work, we aim to design an emotional line for each character that considers\nmultiple emotions common in psychological theories, with the goal of generating\nstories with richer emotional changes in the characters. To the best of our\nknowledge, this work is first to focuses on characters' emotional lines in\nstory generation. We present a novel model-based attention mechanism that we\ncall SoCP (Storytelling of multi-Character Psychology). We show that the\nproposed model can generate stories considering the changes in the\npsychological state of different characters. To take into account the\nparticularity of the model, in addition to commonly used evaluation\nindicators(BLEU, ROUGE, etc.), we introduce the accuracy rate of psychological\nstate control as a novel evaluation metric. The new indicator reflects the\neffect of the model on the psychological state control of story characters.\nExperiments show that with SoCP, the generated stories follow the psychological\nstate for each character according to both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05248v1",
            "date": "2020-10-11 13:44:52+00:00",
            "title": "Towards Accurate and Reliable Energy Measurement of NLP Models",
            "authors": [
                "Qingqing Cao",
                "Aruna Balasubramanian",
                "Niranjan Balasubramanian"
            ],
            "abstract": "Accurate and reliable measurement of energy consumption is critical for\nmaking well-informed design choices when choosing and training large scale NLP\nmodels. In this work, we show that existing software-based energy measurements\nare not accurate because they do not take into account hardware differences and\nhow resource utilization affects energy consumption. We conduct energy\nmeasurement experiments with four different models for a question answering\ntask. We quantify the error of existing software-based energy measurements by\nusing a hardware power meter that provides highly accurate energy measurements.\nOur key takeaway is the need for a more accurate energy estimation model that\ntakes into account hardware variabilities and the non-linear relationship\nbetween resource utilization and energy consumption. We release the code and\ndata at https://github.com/csarron/sustainlp2020-energy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05269v1",
            "date": "2020-10-11 15:21:01+00:00",
            "title": "Automated Prediction of Medieval Arabic Diacritics",
            "authors": [
                "Khalid Alnajjar",
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Niko Partanen",
                "Jack Rueter"
            ],
            "abstract": "This study uses a character level neural machine translation approach trained\non a long short-term memory-based bi-directional recurrent neural network\narchitecture for diacritization of Medieval Arabic. The results improve from\nthe online tool used as a baseline. A diacritization model have been published\nopenly through an easy to use Python package available on PyPi and Zenodo. We\nhave found that context size should be considered when optimizing a feasible\nprediction model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05318v1",
            "date": "2020-10-11 18:53:05+00:00",
            "title": "TransQuest at WMT2020: Sentence-Level Direct Assessment",
            "authors": [
                "Tharindu Ranasinghe",
                "Constantin Orasan",
                "Ruslan Mitkov"
            ],
            "abstract": "This paper presents the team TransQuest's participation in Sentence-Level\nDirect Assessment shared task in WMT 2020. We introduce a simple QE framework\nbased on cross-lingual transformers, and we use it to implement and evaluate\ntwo different neural architectures. The proposed methods achieve\nstate-of-the-art results surpassing the results obtained by OpenKiwi, the\nbaseline used in the shared task. We further fine tune the QE framework by\nperforming ensemble and data augmentation. Our approach is the winning solution\nin all of the language pairs according to the WMT 2020 official results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05330v1",
            "date": "2020-10-11 19:51:21+00:00",
            "title": "Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU",
            "authors": [
                "Brielen Madureira",
                "David Schlangen"
            ],
            "abstract": "While humans process language incrementally, the best language encoders\ncurrently used in NLP do not. Both bidirectional LSTMs and Transformers assume\nthat the sequence that is to be encoded is available in full, to be processed\neither forwards and backwards (BiLSTMs) or as a whole (Transformers). We\ninvestigate how they behave under incremental interfaces, when partial output\nmust be provided based on partial input seen up to a certain time step, which\nmay happen in interactive systems. We test five models on various NLU datasets\nand compare their performance using three incremental evaluation metrics. The\nresults support the possibility of using bidirectional encoders in incremental\nmode while retaining most of their non-incremental quality. The\n\"omni-directional\" BERT model, which achieves better non-incremental\nperformance, is impacted more by the incremental access. This can be alleviated\nby adapting the training regime (truncated training), or the testing procedure,\nby delaying the output until some right context is available or by\nincorporating hypothetical right contexts generated by a language model like\nGPT-2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05332v2",
            "date": "2020-10-11 20:05:42+00:00",
            "title": "Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make It",
            "authors": [
                "Danielle Saunders",
                "Rosie Sallis",
                "Bill Byrne"
            ],
            "abstract": "Neural Machine Translation (NMT) has been shown to struggle with grammatical\ngender that is dependent on the gender of human referents, which can cause\ngender bias effects. Many existing approaches to this problem seek to control\ngender inflection in the target language by explicitly or implicitly adding a\ngender feature to the source sentence, usually at the sentence level.\n  In this paper we propose schemes for incorporating explicit word-level gender\ninflection tags into NMT. We explore the potential of this gender-inflection\ncontrolled translation when the gender feature can be determined from a human\nreference, or when a test sentence can be automatically gender-tagged,\nassessing on English-to-Spanish and English-to-German translation.\n  We find that simple existing approaches can over-generalize a gender-feature\nto multiple entities in a sentence, and suggest effective alternatives in the\nform of tagged coreference adaptation data. We also propose an extension to\nassess translations of gender-neutral entities from English given a\ncorresponding linguistic convention, such as a non-binary inflection, in the\ntarget language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05333v1",
            "date": "2020-10-11 20:09:43+00:00",
            "title": "Addressing Exposure Bias With Document Minimum Risk Training: Cambridge at the WMT20 Biomedical Translation Task",
            "authors": [
                "Danielle Saunders",
                "Bill Byrne"
            ],
            "abstract": "The 2020 WMT Biomedical translation task evaluated Medline abstract\ntranslations. This is a small-domain translation task, meaning limited relevant\ntraining data with very distinct style and vocabulary. Models trained on such\ndata are susceptible to exposure bias effects, particularly when training\nsentence pairs are imperfect translations of each other. This can result in\npoor behaviour during inference if the model learns to neglect the source\nsentence.\n  The UNICAM entry addresses this problem during fine-tuning using a robust\nvariant on Minimum Risk Training. We contrast this approach with data-filtering\nto remove `problem' training examples. Under MRT fine-tuning we obtain good\nresults for both directions of English-German and English-Spanish biomedical\ntranslation. In particular we achieve the best English-to-Spanish translation\nresult and second-best Spanish-to-English result, despite using only single\nmodels with no ensembling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05338v1",
            "date": "2020-10-11 20:27:55+00:00",
            "title": "We Can Detect Your Bias: Predicting the Political Ideology of News Articles",
            "authors": [
                "Ramy Baly",
                "Giovanni Da San Martino",
                "James Glass",
                "Preslav Nakov"
            ],
            "abstract": "We explore the task of predicting the leading political ideology or bias of\nnews articles. First, we collect and release a large dataset of 34,737 articles\nthat were manually annotated for political ideology -left, center, or right-,\nwhich is well-balanced across both topics and media. We further use a\nchallenging experimental setup where the test examples come from media that\nwere not seen during training, which prevents the model from learning to detect\nthe source of the target news article instead of predicting its political\nideology. From a modeling perspective, we propose an adversarial media\nadaptation, as well as a specially adapted triplet loss. We further add\nbackground information about the source, and we show that it is quite helpful\nfor improving article-level prediction. Our experimental results show very\nsizable improvements over using state-of-the-art pre-trained Transformers in\nthis challenging setup.",
            "journal ref": "EMNLP-2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05345v3",
            "date": "2020-10-11 21:11:09+00:00",
            "title": "Do Language Embeddings Capture Scales?",
            "authors": [
                "Xikun Zhang",
                "Deepak Ramachandran",
                "Ian Tenney",
                "Yanai Elazar",
                "Dan Roth"
            ],
            "abstract": "Pretrained Language Models (LMs) have been shown to possess significant\nlinguistic, common sense, and factual knowledge. One form of knowledge that has\nnot been studied yet in this context is information about the scalar magnitudes\nof objects. We show that pretrained language models capture a significant\namount of this information but are short of the capability required for general\ncommon-sense reasoning. We identify contextual information in pre-training and\nnumeracy as two key factors affecting their performance and show that a simple\nmethod of canonicalizing numbers can have a significant effect on the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05358v1",
            "date": "2020-10-11 22:09:27+00:00",
            "title": "Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)",
            "authors": [
                "Alex Warstadt",
                "Yian Zhang",
                "Haau-Sing Li",
                "Haokun Liu",
                "Samuel R. Bowman"
            ],
            "abstract": "One reason pretraining on self-supervised linguistic tasks is effective is\nthat it teaches models features that are helpful for language understanding.\nHowever, we want pretrained models to learn not only to represent linguistic\nfeatures, but also to use those features preferentially during fine-turning.\nWith this goal in mind, we introduce a new English-language diagnostic set\ncalled MSGS (the Mixed Signals Generalization Set), which consists of 20\nambiguous binary classification tasks that we use to test whether a pretrained\nmodel prefers linguistic or surface generalizations during fine-tuning. We\npretrain RoBERTa models from scratch on quantities of data ranging from 1M to\n1B words and compare their performance on MSGS to the publicly available\nRoBERTa-base. We find that models can learn to represent linguistic features\nwith little pretraining data, but require far more data to learn to prefer\nlinguistic generalizations over surface ones. Eventually, with about 30B words\nof pretraining data, RoBERTa-base does demonstrate a linguistic bias with some\nregularity. We conclude that while self-supervised pretraining is an effective\nway to learn helpful inductive biases, there is likely room to improve the rate\nat which models learn which features matter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05369v1",
            "date": "2020-10-11 23:01:51+00:00",
            "title": "Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis",
            "authors": [
                "Roy Bar-Haim",
                "Yoav Kantor",
                "Lilach Eden",
                "Roni Friedman",
                "Dan Lahav",
                "Noam Slonim"
            ],
            "abstract": "When summarizing a collection of views, arguments or opinions on some topic,\nit is often desirable not only to extract the most salient points, but also to\nquantify their prevalence. Work on multi-document summarization has\ntraditionally focused on creating textual summaries, which lack this\nquantitative aspect. Recent work has proposed to summarize arguments by mapping\nthem to a small set of expert-generated key points, where the salience of each\nkey point corresponds to the number of its matching arguments. The current work\nadvances key point analysis in two important respects: first, we develop a\nmethod for automatic extraction of key points, which enables fully automatic\nanalysis, and is shown to achieve performance comparable to a human expert.\nSecond, we demonstrate that the applicability of key point analysis goes well\nbeyond argumentation data. Using models trained on publicly available\nargumentation datasets, we achieve promising results in two additional domains:\nmunicipal surveys and user reviews. An additional contribution is an in-depth\nevaluation of argument-to-key point matching models, where we substantially\noutperform previous results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05444v1",
            "date": "2020-10-12 04:25:48+00:00",
            "title": "OCNLI: Original Chinese Natural Language Inference",
            "authors": [
                "Hai Hu",
                "Kyle Richardson",
                "Liang Xu",
                "Lu Li",
                "Sandra Kuebler",
                "Lawrence S. Moss"
            ],
            "abstract": "Despite the tremendous recent progress on natural language inference (NLI),\ndriven largely by large-scale investment in new datasets (e.g., SNLI, MNLI) and\nadvances in modeling, most progress has been limited to English due to a lack\nof reliable datasets for most of the world's languages. In this paper, we\npresent the first large-scale NLI dataset (consisting of ~56,000 annotated\nsentence pairs) for Chinese called the Original Chinese Natural Language\nInference dataset (OCNLI). Unlike recent attempts at extending NLI to other\nlanguages, our dataset does not rely on any automatic translation or non-expert\nannotation. Instead, we elicit annotations from native speakers specializing in\nlinguistics. We follow closely the annotation protocol used for MNLI, but\ncreate new strategies for eliciting diverse hypotheses. We establish several\nbaseline results on our dataset using state-of-the-art pre-trained models for\nChinese, and find even the best performing models to be far outpaced by human\nperformance (~12% absolute performance gap), making it a challenging new\nresource that we hope will help to accelerate progress in Chinese NLU. To the\nbest of our knowledge, this is the first human-elicited MNLI-style corpus for a\nnon-English language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05445v1",
            "date": "2020-10-12 04:26:46+00:00",
            "title": "Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation",
            "authors": [
                "Fahimeh Saleh",
                "Wray Buntine",
                "Gholamreza Haffari"
            ],
            "abstract": "Scarcity of parallel sentence-pairs poses a significant hurdle for training\nhigh-quality Neural Machine Translation (NMT) models in bilingually\nlow-resource scenarios. A standard approach is transfer learning, which\ninvolves taking a model trained on a high-resource language-pair and\nfine-tuning it on the data of the low-resource MT condition of interest.\nHowever, it is not clear generally which high-resource language-pair offers the\nbest transfer learning for the target MT setting. Furthermore, different\ntransferred models may have complementary semantic and/or syntactic strengths,\nhence using only one model may be sub-optimal. In this paper, we tackle this\nproblem using knowledge distillation, where we propose to distill the knowledge\nof ensemble of teacher models to a single student model. As the quality of\nthese teacher models varies, we propose an effective adaptive knowledge\ndistillation approach to dynamically adjust the contribution of the teacher\nmodels during the distillation process. Experiments on transferring from a\ncollection of six language pairs from IWSLT to five low-resource language-pairs\nfrom TED Talks demonstrate the effectiveness of our approach, achieving up to\n+0.9 BLEU score improvement compared to strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05465v1",
            "date": "2020-10-12 05:45:44+00:00",
            "title": "COGS: A Compositional Generalization Challenge Based on Semantic Interpretation",
            "authors": [
                "Najoung Kim",
                "Tal Linzen"
            ],
            "abstract": "Natural language is characterized by compositionality: the meaning of a\ncomplex expression is constructed from the meanings of its constituent parts.\nTo facilitate the evaluation of the compositional abilities of language\nprocessing architectures, we introduce COGS, a semantic parsing dataset based\non a fragment of English. The evaluation portion of COGS contains multiple\nsystematic gaps that can only be addressed by compositional generalization;\nthese include new combinations of familiar syntactic structures, or new\ncombinations of familiar words and familiar structures. In experiments with\nTransformers and LSTMs, we found that in-distribution accuracy on the COGS test\nset was near-perfect (96--99%), but generalization accuracy was substantially\nlower (16--35%) and showed high sensitivity to random seed ($\\pm$6--8%). These\nfindings indicate that contemporary standard NLP models are limited in their\ncompositional generalization capacity, and position COGS as a good way to\nmeasure progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05478v2",
            "date": "2020-10-12 06:43:10+00:00",
            "title": "Evaluating Factuality in Generation with Dependency-level Entailment",
            "authors": [
                "Tanya Goyal",
                "Greg Durrett"
            ],
            "abstract": "Despite significant progress in text generation models, a serious limitation\nis their tendency to produce text that is factually inconsistent with\ninformation in the input. Recent work has studied whether textual entailment\nsystems can be used to identify factual errors; however, these sentence-level\nentailment models are trained to solve a different problem than generation\nfiltering and they do not localize which part of a generation is non-factual.\nIn this paper, we propose a new formulation of entailment that decomposes it at\nthe level of dependency arcs. Rather than focusing on aggregate decisions, we\ninstead ask whether the semantic relationship manifested by individual\ndependency arcs in the generated output is supported by the input. Human\njudgments on this task are difficult to obtain; we therefore propose a method\nto automatically create data based on existing entailment or paraphrase\ncorpora. Experiments show that our dependency arc entailment model trained on\nthis data can identify factual inconsistencies in paraphrasing and\nsummarization better than sentence-level methods or those based on question\ngeneration, while additionally localizing the erroneous parts of the\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05511v1",
            "date": "2020-10-12 08:06:12+00:00",
            "title": "A Sentiment-Controllable Topic-to-Essay Generator with Topic Knowledge Graph",
            "authors": [
                "Lin Qiao",
                "Jianhao Yan",
                "Fandong Meng",
                "Zhendong Yang",
                "Jie Zhou"
            ],
            "abstract": "Generating a vivid, novel, and diverse essay with only several given topic\nwords is a challenging task of natural language generation. In previous work,\nthere are two problems left unsolved: neglect of sentiment beneath the text and\ninsufficient utilization of topic-related knowledge. Therefore, we propose a\nnovel Sentiment-Controllable topic-to-essay generator with a Topic Knowledge\nGraph enhanced decoder, named SCTKG, which is based on the conditional\nvariational autoencoder (CVAE) framework. We firstly inject the sentiment\ninformation into the generator for controlling sentiment for each sentence,\nwhich leads to various generated essays. Then we design a Topic Knowledge Graph\nenhanced decoder. Unlike existing models that use knowledge entities\nseparately, our model treats the knowledge graph as a whole and encodes more\nstructured, connected semantic information in the graph to generate a more\nrelevant essay. Experimental results show that our SCTKG can generate sentiment\ncontrollable essays and outperform the state-of-the-art approach in terms of\ntopic relevance, fluency, and diversity on both automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05522v1",
            "date": "2020-10-12 08:24:36+00:00",
            "title": "Pre-trained Language Model Based Active Learning for Sentence Matching",
            "authors": [
                "Guirong Bai",
                "Shizhu He",
                "Kang Liu",
                "Jun Zhao",
                "Zaiqing Nie"
            ],
            "abstract": "Active learning is able to significantly reduce the annotation cost for\ndata-driven techniques. However, previous active learning approaches for\nnatural language processing mainly depend on the entropy-based uncertainty\ncriterion, and ignore the characteristics of natural language. In this paper,\nwe propose a pre-trained language model based active learning approach for\nsentence matching. Differing from previous active learning, it can provide\nlinguistic criteria to measure instances and help select more efficient\ninstances for annotation. Experiments demonstrate our approach can achieve\ngreater accuracy with fewer labeled training instances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05533v1",
            "date": "2020-10-12 08:45:28+00:00",
            "title": "Toward Cross-Lingual Definition Generation for Language Learners",
            "authors": [
                "Cunliang Kong",
                "Liner Yang",
                "Tianzuo Zhang",
                "Qinan Fan",
                "Zhenghao Liu",
                "Yun Chen",
                "Erhong Yang"
            ],
            "abstract": "Generating dictionary definitions automatically can prove useful for language\nlearners. However, it's still a challenging task of cross-lingual definition\ngeneration. In this work, we propose to generate definitions in English for\nwords in various languages. To achieve this, we present a simple yet effective\napproach based on publicly available pretrained language models. In this\napproach, models can be directly applied to other languages after trained on\nthe English dataset. We demonstrate the effectiveness of this approach on\nzero-shot definition generation. Experiments and manual analyses on newly\nconstructed datasets show that our models have a strong cross-lingual transfer\nability and can generate fluent English definitions for Chinese words. We\nfurther measure the lexical complexity of generated and reference definitions.\nThe results show that the generated definitions are much simpler, which is more\nsuitable for language learners.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05542v1",
            "date": "2020-10-12 08:57:58+00:00",
            "title": "The National Corpus of Contemporary Welsh: Project Report | Y Corpws Cenedlaethol Cymraeg Cyfoes: Adroddiad y Prosiect",
            "authors": [
                "Dawn Knight",
                "Steve Morris",
                "Tess Fitzpatrick",
                "Paul Rayson",
                "Irena Spasi\u0107",
                "Enlli M\u00f4n Thomas"
            ],
            "abstract": "This report provides an overview of the CorCenCC project and the online\ncorpus resource that was developed as a result of work on the project. The\nreport lays out the theoretical underpinnings of the research, demonstrating\nhow the project has built on and extended this theory. We also raise and\ndiscuss some of the key operational questions that arose during the course of\nthe project, outlining the ways in which they were answered, the impact of\nthese decisions on the resource that has been produced and the longer-term\ncontribution they will make to practices in corpus-building. Finally, we\ndiscuss some of the applications and the utility of the work, outlining the\nimpact that CorCenCC is set to have on a range of different individuals and\nuser groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05549v1",
            "date": "2020-10-12 09:15:12+00:00",
            "title": "Improving Low Resource Code-switched ASR using Augmented Code-switched TTS",
            "authors": [
                "Yash Sharma",
                "Basil Abraham",
                "Karan Taneja",
                "Preethi Jyothi"
            ],
            "abstract": "Building Automatic Speech Recognition (ASR) systems for code-switched speech\nhas recently gained renewed attention due to the widespread use of speech\ntechnologies in multilingual communities worldwide. End-to-end ASR systems are\na natural modeling choice due to their ease of use and superior performance in\nmonolingual settings. However, it is well known that end-to-end systems require\nlarge amounts of labeled speech. In this work, we investigate improving\ncode-switched ASR in low resource settings via data augmentation using\ncode-switched text-to-speech (TTS) synthesis. We propose two targeted\ntechniques to effectively leverage TTS speech samples: 1) Mixup, an existing\ntechnique to create new training samples via linear interpolation of existing\nsamples, applied to TTS and real speech samples, and 2) a new loss function,\nused in conjunction with TTS samples, to encourage code-switched predictions.\nWe report significant improvements in ASR performance achieving absolute word\nerror rate (WER) reductions of up to 5%, and measurable improvement in code\nswitching using our proposed techniques on a Hindi-English code-switched ASR\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05567v1",
            "date": "2020-10-12 09:36:24+00:00",
            "title": "Joint Semantic Analysis with Document-Level Cross-Task Coherence Rewards",
            "authors": [
                "Rahul Aralikatte",
                "Mostafa Abdou",
                "Heather Lent",
                "Daniel Hershcovich",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Coreference resolution and semantic role labeling are NLP tasks that capture\ndifferent aspects of semantics, indicating respectively, which expressions\nrefer to the same entity, and what semantic roles expressions serve in the\nsentence. However, they are often closely interdependent, and both generally\nnecessitate natural language understanding. Do they form a coherent abstract\nrepresentation of documents? We present a neural network architecture for joint\ncoreference resolution and semantic role labeling for English, and train graph\nneural networks to model the 'coherence' of the combined shallow semantic\ngraph. Using the resulting coherence score as a reward for our joint semantic\nanalyzer, we use reinforcement learning to encourage global coherence over the\ndocument and between semantic annotations. This leads to improvements on both\ntasks in multiple datasets from different domains, and across a range of\nencoders of different expressivity, calling, we believe, for a more holistic\napproach to semantics in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05572v1",
            "date": "2020-10-12 09:49:27+00:00",
            "title": "Meta-Context Transformers for Domain-Specific Response Generation",
            "authors": [
                "Debanjana Kar",
                "Suranjana Samanta",
                "Amar Prakash Azad"
            ],
            "abstract": "Despite the tremendous success of neural dialogue models in recent years, it\nsuffers a lack of relevance, diversity, and some times coherence in generated\nresponses. Lately, transformer-based models, such as GPT-2, have revolutionized\nthe landscape of dialogue generation by capturing the long-range structures\nthrough language modeling. Though these models have exhibited excellent\nlanguage coherence, they often lack relevance and terms when used for\ndomain-specific response generation. In this paper, we present DSRNet (Domain\nSpecific Response Network), a transformer-based model for dialogue response\ngeneration by reinforcing domain-specific attributes. In particular, we extract\nmeta attributes from context and infuse them with the context utterances for\nbetter attention over domain-specific key terms and relevance. We study the use\nof DSRNet in a multi-turn multi-interlocutor environment for domain-specific\nresponse generation. In our experiments, we evaluate DSRNet on Ubuntu dialogue\ndatasets, which are mainly composed of various technical domain related\ndialogues for IT domain issue resolutions and also on CamRest676 dataset, which\ncontains restaurant domain conversations. Trained with maximum likelihood\nobjective, our model shows significant improvement over the state-of-the-art\nfor multi-turn dialogue systems supported by better BLEU and semantic\nsimilarity (BertScore) scores. Besides, we also observe that the responses\nproduced by our model carry higher relevance due to the presence of\ndomain-specific key attributes that exhibit better overlap with the attributes\nof the context. Our analysis shows that the performance improvement is mostly\ndue to the infusion of key terms along with dialogues which result in better\nattention over domain-relevant terms. Other contributing factors include joint\nmodeling of dialogue context with the domain-specific meta attributes and\ntopics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05581v1",
            "date": "2020-10-12 10:09:05+00:00",
            "title": "Counterfactual Variable Control for Robust and Interpretable Question Answering",
            "authors": [
                "Sicheng Yu",
                "Yulei Niu",
                "Shuohang Wang",
                "Jing Jiang",
                "Qianru Sun"
            ],
            "abstract": "Deep neural network based question answering (QA) models are neither robust\nnor explainable in many cases. For example, a multiple-choice QA model, tested\nwithout any input of question, is surprisingly \"capable\" to predict the most of\ncorrect options. In this paper, we inspect such spurious \"capability\" of QA\nmodels using causal inference. We find the crux is the shortcut correlation,\ne.g., unrobust word alignment between passage and options learned by the\nmodels. We propose a novel approach called Counterfactual Variable Control\n(CVC) that explicitly mitigates any shortcut correlation and preserves the\ncomprehensive reasoning for robust QA. Specifically, we leverage multi-branch\narchitecture that allows us to disentangle robust and shortcut correlations in\nthe training process of QA. We then conduct two novel CVC inference methods (on\ntrained models) to capture the effect of comprehensive reasoning as the final\nprediction. For evaluation, we conduct extensive experiments using two BERT\nbackbones on both multi-choice and span-extraction QA benchmarks. The results\nshow that our CVC achieves high robustness against a variety of adversarial\nattacks in QA while maintaining good interpretation ability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05587v1",
            "date": "2020-10-12 10:24:40+00:00",
            "title": "Social Commonsense Reasoning with Multi-Head Knowledge Attention",
            "authors": [
                "Debjit Paul",
                "Anette Frank"
            ],
            "abstract": "Social Commonsense Reasoning requires understanding of text, knowledge about\nsocial events and their pragmatic implications, as well as commonsense\nreasoning skills. In this work we propose a novel multi-head knowledge\nattention model that encodes semi-structured commonsense inference rules and\nlearns to incorporate them in a transformer-based reasoning cell. We assess the\nmodel's performance on two tasks that require different reasoning skills:\nAbductive Natural Language Inference and Counterfactual Invariance Prediction\nas a new task. We show that our proposed model improves performance over strong\nstate-of-the-art models (i.e., RoBERTa) across both reasoning tasks. Notably we\nare, to the best of our knowledge, the first to demonstrate that a model that\nlearns to perform counterfactual reasoning helps predicting the best\nexplanation in an abductive reasoning task. We validate the robustness of the\nmodel's reasoning capabilities by perturbing the knowledge and provide\nqualitative analysis on the model's knowledge incorporation capabilities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05594v2",
            "date": "2020-10-12 10:53:19+00:00",
            "title": "MultiWOZ 2.3: A multi-domain task-oriented dataset enhanced with annotation corrections and co-reference annotation",
            "authors": [
                "Ting Han",
                "Ximing Liu",
                "Ryuichi Takanobu",
                "Yixin Lian",
                "Chongxuan Huang",
                "Wei Peng",
                "Minlie Huang"
            ],
            "abstract": "Task-oriented dialogue systems have made unprecedented progress with multiple\nstate-of-the-art (SOTA) models underpinned by a number of publicly available\nMultiWOZ datasets. Dialogue state annotations are error-prone, leading to\nsub-optimal performance. Various efforts have been put in rectifying the\nannotation errors presented in the original MultiWOZ dataset. In this paper, we\nintroduce MultiWOZ 2.3, in which we differentiate incorrect annotations in\ndialogue acts from dialogue states, identifying a lack of co-reference when\npublishing the updated dataset. To ensure consistency between dialogue acts and\ndialogue states, we implement co-reference features and unify annotations of\ndialogue acts and dialogue states. We update the state of the art performance\nof natural language understanding and dialog state tracking on MultiWOZ 2.3,\nwhere the results show significant improvements than on previous versions of\nMultiWOZ datasets (2.0-2.2).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05607v1",
            "date": "2020-10-12 11:27:47+00:00",
            "title": "The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?",
            "authors": [
                "Jasmijn Bastings",
                "Katja Filippova"
            ],
            "abstract": "There is a recent surge of interest in using attention as explanation of\nmodel predictions, with mixed evidence on whether attention can be used as\nsuch. While attention conveniently gives us one weight per input token and is\neasily extracted, it is often unclear toward what goal it is used as\nexplanation. We find that often that goal, whether explicitly stated or not, is\nto find out what input tokens are the most relevant to a prediction, and that\nthe implied user for the explanation is a model developer. For this goal and\nuser, we argue that input saliency methods are better suited, and that there\nare no compelling reasons to use attention, despite the coincidence that it\nprovides a weight for each input. With this position paper, we hope to shift\nsome of the recent focus on attention to saliency methods, and for authors to\nclearly state the goal and user for their explanations.",
            "journal ref": "Proceedings of the 2020 EMNLP Workshop BlackboxNLP: Analyzing and\n  Interpreting Neural Networks for NLP",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05633v1",
            "date": "2020-10-12 12:07:02+00:00",
            "title": "Contextual Modulation for Relation-Level Metaphor Identification",
            "authors": [
                "Omnia Zayed",
                "John P. McCrae",
                "Paul Buitelaar"
            ],
            "abstract": "Identifying metaphors in text is very challenging and requires comprehending\nthe underlying comparison. The automation of this cognitive process has gained\nwide attention lately. However, the majority of existing approaches concentrate\non word-level identification by treating the task as either single-word\nclassification or sequential labelling without explicitly modelling the\ninteraction between the metaphor components. On the other hand, while existing\nrelation-level approaches implicitly model this interaction, they ignore the\ncontext where the metaphor occurs. In this work, we address these limitations\nby introducing a novel architecture for identifying relation-level metaphoric\nexpressions of certain grammatical relations based on contextual modulation. In\na methodology inspired by works in visual reasoning, our approach is based on\nconditioning the neural network computation on the deep contextualised features\nof the candidate expressions using feature-wise linear modulation. We\ndemonstrate that the proposed architecture achieves state-of-the-art results on\nbenchmark datasets. The proposed methodology is generic and could be applied to\nother textual classification problems that benefit from contextual interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05647v1",
            "date": "2020-10-12 12:34:58+00:00",
            "title": "Improving Compositional Generalization in Semantic Parsing",
            "authors": [
                "Inbar Oren",
                "Jonathan Herzig",
                "Nitish Gupta",
                "Matt Gardner",
                "Jonathan Berant"
            ],
            "abstract": "Generalization of models to out-of-distribution (OOD) data has captured\ntremendous attention recently. Specifically, compositional generalization,\ni.e., whether a model generalizes to new structures built of components\nobserved during training, has sparked substantial interest. In this work, we\ninvestigate compositional generalization in semantic parsing, a natural\ntest-bed for compositional generalization, as output programs are constructed\nfrom sub-components. We analyze a wide variety of models and propose multiple\nextensions to the attention module of the semantic parser, aiming to improve\ncompositional generalization. We find that the following factors improve\ncompositional generalization: (a) using contextual representations, such as\nELMo and BERT, (b) informing the decoder what input tokens have previously been\nattended to, (c) training the decoder attention to agree with pre-computed\ntoken alignments, and (d) downsampling examples corresponding to frequent\nprogram templates. While we substantially reduce the gap between\nin-distribution and OOD generalization, performance on OOD compositions is\nstill substantially lower.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05648v2",
            "date": "2020-10-12 12:35:36+00:00",
            "title": "From Hero to Z\u00e9roe: A Benchmark of Low-Level Adversarial Attacks",
            "authors": [
                "Steffen Eger",
                "Yannik Benz"
            ],
            "abstract": "Adversarial attacks are label-preserving modifications to inputs of machine\nlearning classifiers designed to fool machines but not humans. Natural Language\nProcessing (NLP) has mostly focused on high-level attack scenarios such as\nparaphrasing input texts. We argue that these are less realistic in typical\napplication scenarios such as in social media, and instead focus on low-level\nattacks on the character-level. Guided by human cognitive abilities and human\nrobustness, we propose the first large-scale catalogue and benchmark of\nlow-level adversarial attacks, which we dub Z\\'eroe, encompassing nine\ndifferent attack modes including visual and phonetic adversaries. We show that\nRoBERTa, NLP's current workhorse, fails on our attacks. Our dataset provides a\nbenchmark for testing robustness of future more human-like NLP models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05670v1",
            "date": "2020-10-12 13:08:45+00:00",
            "title": "Modelling Lexical Ambiguity with Density Matrices",
            "authors": [
                "Francois Meyer",
                "Martha Lewis"
            ],
            "abstract": "Words can have multiple senses. Compositional distributional models of\nmeaning have been argued to deal well with finer shades of meaning variation\nknown as polysemy, but are not so well equipped to handle word senses that are\netymologically unrelated, or homonymy. Moving from vectors to density matrices\nallows us to encode a probability distribution over different senses of a word,\nand can also be accommodated within a compositional distributional model of\nmeaning. In this paper we present three new neural models for learning density\nmatrices from a corpus, and test their ability to discriminate between word\nsenses on a range of compositional datasets. When paired with a particular\ncomposition method, our best model outperforms existing vector-based\ncompositional models as well as strong sentence encoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05700v1",
            "date": "2020-10-12 13:31:01+00:00",
            "title": "Reformulating Unsupervised Style Transfer as Paraphrase Generation",
            "authors": [
                "Kalpesh Krishna",
                "John Wieting",
                "Mohit Iyyer"
            ],
            "abstract": "Modern NLP defines the task of style transfer as modifying the style of a\ngiven sentence without appreciably changing its semantics, which implies that\nthe outputs of style transfer systems should be paraphrases of their inputs.\nHowever, many existing systems purportedly designed for style transfer\ninherently warp the input's meaning through attribute transfer, which changes\nsemantic properties such as sentiment. In this paper, we reformulate\nunsupervised style transfer as a paraphrase generation problem, and present a\nsimple methodology based on fine-tuning pretrained language models on\nautomatically generated paraphrase data. Despite its simplicity, our method\nsignificantly outperforms state-of-the-art style transfer systems on both human\nand automatic evaluations. We also survey 23 style transfer papers and discover\nthat existing automatic metrics can be easily gamed and propose fixed variants.\nFinally, we pivot to a more real-world style transfer setting by collecting a\nlarge dataset of 15M sentences in 11 diverse styles, which we use for an\nin-depth analysis of our system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05725v1",
            "date": "2020-10-12 14:12:37+00:00",
            "title": "Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models",
            "authors": [
                "Ethan Wilcox",
                "Peng Qian",
                "Richard Futrell",
                "Ryosuke Kohita",
                "Roger Levy",
                "Miguel Ballesteros"
            ],
            "abstract": "Humans can learn structural properties about a word from minimal experience,\nand deploy their learned syntactic representations uniformly in different\ngrammatical contexts. We assess the ability of modern neural language models to\nreproduce this behavior in English and evaluate the effect of structural\nsupervision on learning outcomes. First, we assess few-shot learning\ncapabilities by developing controlled experiments that probe models' syntactic\nnominal number and verbal argument structure generalizations for tokens seen as\nfew as two times during training. Second, we assess invariance properties of\nlearned representation: the ability of a model to transfer syntactic\ngeneralizations from a base context (e.g., a simple declarative active-voice\nsentence) to a transformed context (e.g., an interrogative sentence). We test\nfour models trained on the same dataset: an n-gram baseline, an LSTM, and two\nLSTM-variants trained with explicit structural supervision (Dyer et al.,2016;\nCharniak et al., 2016). We find that in most cases, the neural models are able\nto induce the proper syntactic generalizations after minimal exposure, often\nfrom just two examples during training, and that the two structurally\nsupervised models generalize more accurately than the LSTM model. All neural\nmodels are able to leverage information learned in base contexts to drive\nexpectations in transformed contexts, indicating that they have learned some\ninvariance properties of syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05731v1",
            "date": "2020-10-12 14:24:01+00:00",
            "title": "Probing Pretrained Language Models for Lexical Semantics",
            "authors": [
                "Ivan Vuli\u0107",
                "Edoardo Maria Ponti",
                "Robert Litschko",
                "Goran Glava\u0161",
                "Anna Korhonen"
            ],
            "abstract": "The success of large pretrained language models (LMs) such as BERT and\nRoBERTa has sparked interest in probing their representations, in order to\nunveil what types of knowledge they implicitly capture. While prior research\nfocused on morphosyntactic, semantic, and world knowledge, it remains unclear\nto which extent LMs also derive lexical type-level knowledge from words in\ncontext. In this work, we present a systematic empirical analysis across six\ntypologically diverse languages and five different lexical tasks, addressing\nthe following questions: 1) How do different lexical knowledge extraction\nstrategies (monolingual versus multilingual source LM, out-of-context versus\nin-context encoding, inclusion of special tokens, and layer-wise averaging)\nimpact performance? How consistent are the observed effects across tasks and\nlanguages? 2) Is lexical knowledge stored in few parameters, or is it scattered\nthroughout the network? 3) How do these representations fare against\ntraditional static word vectors in lexical tasks? 4) Does the lexical\ninformation emerging from independently trained monolingual LMs display latent\nsimilarities? Our main results indicate patterns and best practices that hold\nuniversally, but also point to prominent variations across languages and tasks.\nMoreover, we validate the claim that lower Transformer layers carry more\ntype-level lexical knowledge, but also show that this knowledge is distributed\nacross multiple layers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05736v1",
            "date": "2020-10-12 14:28:48+00:00",
            "title": "EFSG: Evolutionary Fooling Sentences Generator",
            "authors": [
                "Marco Di Giovanni",
                "Marco Brambilla"
            ],
            "abstract": "Large pre-trained language representation models (LMs) have recently\ncollected a huge number of successes in many NLP tasks.\n  In 2018 BERT, and later its successors (e.g. RoBERTa), obtained\nstate-of-the-art results in classical benchmark tasks, such as GLUE benchmark.\n  After that, works about adversarial attacks have been published to test their\ngeneralization proprieties and robustness.\n  In this work, we design Evolutionary Fooling Sentences Generator (EFSG), a\nmodel- and task-agnostic adversarial attack algorithm built using an\nevolutionary approach to generate false-positive sentences for binary\nclassification tasks.\n  We successfully apply EFSG to CoLA and MRPC tasks, on BERT and RoBERTa,\ncomparing performances. Results prove the presence of weak spots in\nstate-of-the-art LMs.\n  We finally test adversarial training as a data augmentation defence approach\nagainst EFSG, obtaining stronger improved models with no loss of accuracy when\ntested on the original datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05740v3",
            "date": "2020-10-12 14:34:07+00:00",
            "title": "Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems",
            "authors": [
                "Yanjie Gou",
                "Yinjie Lei",
                "Lingqiao Liu"
            ],
            "abstract": "Recent studies try to build task-oriented dialogue systems in an end-to-end\nmanner and the existing works make great progress on this task. However, there\nis still an issue need to be further considered, i.e., how to effectively\nrepresent the knowledge bases and incorporate that into dialogue systems. To\nsolve this issue, we design a novel Transformer-based Context-aware Memory\nGenerator to model the entities in knowledge bases, which can produce entity\nrepresentations with perceiving all the relevant entities and dialogue history.\nFurthermore, we propose Context-aware Memory Enhanced Transformer (CMET), which\ncan effectively aggregate information from the dialogue history and knowledge\nbases to generate more accurate responses. Through extensive experiments, our\nmethod can achieve superior performance over the state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05763v1",
            "date": "2020-10-12 14:56:22+00:00",
            "title": "Layer-wise Guided Training for BERT: Learning Incrementally Refined Document Representations",
            "authors": [
                "Nikolaos Manginas",
                "Ilias Chalkidis",
                "Prodromos Malakasiotis"
            ],
            "abstract": "Although BERT is widely used by the NLP community, little is known about its\ninner workings. Several attempts have been made to shed light on certain\naspects of BERT, often with contradicting conclusions. A much raised concern\nfocuses on BERT's over-parameterization and under-utilization issues. To this\nend, we propose o novel approach to fine-tune BERT in a structured manner.\nSpecifically, we focus on Large Scale Multilabel Text Classification (LMTC)\nwhere documents are assigned with one or more labels from a large predefined\nset of hierarchically organized labels. Our approach guides specific BERT\nlayers to predict labels from specific hierarchy levels. Experimenting with two\nLMTC datasets we show that this structured fine-tuning approach not only yields\nbetter classification results but also leads to better parameter utilization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05856v1",
            "date": "2020-10-12 17:02:50+00:00",
            "title": "Controllable Paraphrasing and Translation with a Syntactic Exemplar",
            "authors": [
                "Mingda Chen",
                "Sam Wiseman",
                "Kevin Gimpel"
            ],
            "abstract": "Most prior work on exemplar-based syntactically controlled paraphrase\ngeneration relies on automatically-constructed large-scale paraphrase datasets.\nWe sidestep this prerequisite by adapting models from prior work to be able to\nlearn solely from bilingual text (bitext). Despite only using bitext for\ntraining, and in near zero-shot conditions, our single proposed model can\nperform four tasks: controlled paraphrase generation in both languages and\ncontrolled machine translation in both language directions. To evaluate these\ntasks quantitatively, we create three novel evaluation datasets. Our\nexperimental results show that our models achieve competitive results on\ncontrolled paraphrase generation and strong performance on controlled machine\ntranslation. Analysis shows that our models learn to disentangle semantics and\nsyntax in their latent representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05873v1",
            "date": "2020-10-12 17:25:02+00:00",
            "title": "Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data",
            "authors": [
                "Katja Filippova"
            ],
            "abstract": "Neural text generation (data- or text-to-text) demonstrates remarkable\nperformance when training data is abundant which for many applications is not\nthe case. To collect a large corpus of parallel data, heuristic rules are often\nused but they inevitably let noise into the data, such as phrases in the output\nwhich cannot be explained by the input. Consequently, models pick up on the\nnoise and may hallucinate--generate fluent but unsupported text. Our\ncontribution is a simple but powerful technique to treat such hallucinations as\na controllable aspect of the generated text, without dismissing any input and\nwithout modifying the model architecture. On the WikiBio corpus (Lebret et al.,\n2016), a particularly noisy dataset, we demonstrate the efficacy of the\ntechnique both in an automatic and in a human evaluation.",
            "journal ref": "Findings of EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05904v1",
            "date": "2020-10-12 17:57:00+00:00",
            "title": "Multi-Stage Pre-training for Low-Resource Domain Adaptation",
            "authors": [
                "Rong Zhang",
                "Revanth Gangi Reddy",
                "Md Arafat Sultan",
                "Vittorio Castelli",
                "Anthony Ferritto",
                "Radu Florian",
                "Efsun Sarioglu Kayi",
                "Salim Roukos",
                "Avirup Sil",
                "Todd Ward"
            ],
            "abstract": "Transfer learning techniques are particularly useful in NLP tasks where a\nsizable amount of high-quality annotated data is difficult to obtain. Current\napproaches directly adapt a pre-trained language model (LM) on in-domain text\nbefore fine-tuning to downstream tasks. We show that extending the vocabulary\nof the LM with domain-specific terms leads to further gains. To a bigger\neffect, we utilize structure in the unlabeled data to create auxiliary\nsynthetic tasks, which helps the LM transfer to downstream tasks. We apply\nthese approaches incrementally on a pre-trained Roberta-large LM and show\nconsiderable performance gain on three tasks in the IT domain: Extractive\nReading Comprehension, Document Ranking and Duplicate Question Detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05953v1",
            "date": "2020-10-12 18:27:05+00:00",
            "title": "COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs",
            "authors": [
                "Jena D. Hwang",
                "Chandra Bhagavatula",
                "Ronan Le Bras",
                "Jeff Da",
                "Keisuke Sakaguchi",
                "Antoine Bosselut",
                "Yejin Choi"
            ],
            "abstract": "Recent years have brought about a renewed interest in commonsense\nrepresentation and reasoning in the field of natural language understanding.\nThe development of new commonsense knowledge graphs (CSKG) has been central to\nthese advances as their diverse facts can be used and referenced by machine\nlearning models for tackling new and challenging tasks. At the same time, there\nremain questions about the quality and coverage of these resources due to the\nmassive scale required to comprehensively encompass general commonsense\nknowledge.\n  In this work, we posit that manually constructed CSKGs will never achieve the\ncoverage necessary to be applicable in all situations encountered by NLP\nagents. Therefore, we propose a new evaluation framework for testing the\nutility of KGs based on how effectively implicit knowledge representations can\nbe learned from them.\n  With this new goal, we propose ATOMIC 2020, a new CSKG of general-purpose\ncommonsense knowledge containing knowledge that is not readily available in\npretrained language models. We evaluate its properties in comparison with other\nleading CSKGs, performing the first large-scale pairwise study of commonsense\nknowledge resources. Next, we show that ATOMIC 2020 is better suited for\ntraining knowledge models that can generate accurate, representative knowledge\nfor new, unseen entities and events. Finally, through human evaluation, we show\nthat the few-shot performance of GPT-3 (175B parameters), while impressive,\nremains ~12 absolute points lower than a BART-based knowledge model trained on\nATOMIC 2020 despite using over 430x fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05959v1",
            "date": "2020-10-12 18:39:07+00:00",
            "title": "Towards Induction of Structured Phoneme Inventories",
            "authors": [
                "Alexander Gutkin",
                "Martin Jansche",
                "Lucy Skidmore"
            ],
            "abstract": "This extended abstract surveying the work on phonological typology was\nprepared for \"SIGTYP 2020: The Second Workshop on Computational Research in\nLinguistic Typology\" to be held at EMNLP 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05971v1",
            "date": "2020-10-12 19:04:04+00:00",
            "title": "The Extraordinary Failure of Complement Coercion Crowdsourcing",
            "authors": [
                "Yanai Elazar",
                "Victoria Basmov",
                "Shauli Ravfogel",
                "Yoav Goldberg",
                "Reut Tsarfaty"
            ],
            "abstract": "Crowdsourcing has eased and scaled up the collection of linguistic annotation\nin recent years. In this work, we follow known methodologies of collecting\nlabeled data for the complement coercion phenomenon. These are constructions\nwith an implied action -- e.g., \"I started a new book I bought last week\",\nwhere the implied action is reading. We aim to collect annotated data for this\nphenomenon by reducing it to either of two known tasks: Explicit Completion and\nNatural Language Inference. However, in both cases, crowdsourcing resulted in\nlow agreement scores, even though we followed the same methodologies as in\nprevious work. Why does the same process fail to yield high agreement scores?\nWe specify our modeling schemes, highlight the differences with previous work\nand provide some insights about the task and possible explanations for the\nfailure. We conclude that specific phenomena require tailored solutions, not\nonly in specialized algorithms, but also in data collection methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06018v1",
            "date": "2020-10-12 20:42:21+00:00",
            "title": "Gender Coreference and Bias Evaluation at WMT 2020",
            "authors": [
                "Tom Kocmi",
                "Tomasz Limisiewicz",
                "Gabriel Stanovsky"
            ],
            "abstract": "Gender bias in machine translation can manifest when choosing gender\ninflections based on spurious gender correlations. For example, always\ntranslating doctors as men and nurses as women. This can be particularly\nharmful as models become more popular and deployed within commercial systems.\nOur work presents the largest evidence for the phenomenon in more than 19\nsystems submitted to the WMT over four diverse target languages: Czech, German,\nPolish, and Russian. To achieve this, we use WinoMT, a recent automatic test\nsuite which examines gender coreference and bias when translating from English\nto languages with grammatical gender. We extend WinoMT to handle two new\nlanguages tested in WMT: Polish and Czech. We find that all systems\nconsistently use spurious correlations in the data rather than meaningful\ncontextual information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06028v1",
            "date": "2020-10-12 21:10:18+00:00",
            "title": "End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems",
            "authors": [
                "Siamak Shakeri",
                "Cicero Nogueira dos Santos",
                "Henry Zhu",
                "Patrick Ng",
                "Feng Nan",
                "Zhiguo Wang",
                "Ramesh Nallapati",
                "Bing Xiang"
            ],
            "abstract": "We propose an end-to-end approach for synthetic QA data generation. Our model\ncomprises a single transformer-based encoder-decoder network that is trained\nend-to-end to generate both answers and questions. In a nutshell, we feed a\npassage to the encoder and ask the decoder to generate a question and an answer\ntoken-by-token. The likelihood produced in the generation process is used as a\nfiltering score, which avoids the need for a separate filtering model. Our\ngenerator is trained by fine-tuning a pretrained LM using maximum likelihood\nestimation. The experimental results indicate significant improvements in the\ndomain adaptation of QA models outperforming current state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06032v2",
            "date": "2020-10-12 21:15:29+00:00",
            "title": "Measuring and Reducing Gendered Correlations in Pre-trained Models",
            "authors": [
                "Kellie Webster",
                "Xuezhi Wang",
                "Ian Tenney",
                "Alex Beutel",
                "Emily Pitler",
                "Ellie Pavlick",
                "Jilin Chen",
                "Ed Chi",
                "Slav Petrov"
            ],
            "abstract": "Pre-trained models have revolutionized natural language understanding.\nHowever, researchers have found they can encode artifacts undesired in many\napplications, such as professions correlating with one gender more than\nanother. We explore such gendered correlations as a case study for how to\naddress unintended correlations in pre-trained models. We define metrics and\nreveal that it is possible for models with similar accuracy to encode\ncorrelations at very different rates. We show how measured correlations can be\nreduced with general-purpose techniques, and highlight the trade offs different\nstrategies have. With these results, we make recommendations for training\nrobust models: (1) carefully evaluate unintended correlations, (2) be mindful\nof seemingly innocuous configuration differences, and (3) focus on general\nmitigations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06041v1",
            "date": "2020-10-12 21:28:57+00:00",
            "title": "Towards Machine Translation for the Kurdish Language",
            "authors": [
                "Sina Ahmadi",
                "Mariam Masoud"
            ],
            "abstract": "Machine translation is the task of translating texts from one language to\nanother using computers. It has been one of the major tasks in natural language\nprocessing and computational linguistics and has been motivating to facilitate\nhuman communication. Kurdish, an Indo-European language, has received little\nattention in this realm due to the language being less-resourced. Therefore, in\nthis paper, we are addressing the main issues in creating a machine translation\nsystem for the Kurdish language, with a focus on the Sorani dialect. We\ndescribe the available scarce parallel data suitable for training a neural\nmachine translation model for Sorani Kurdish-English translation. We also\ndiscuss some of the major challenges in Kurdish language translation and\ndemonstrate how fundamental text processing tasks, such as tokenization, can\nimprove translation performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06060v2",
            "date": "2020-10-12 22:46:10+00:00",
            "title": "BioMegatron: Larger Biomedical Domain Language Model",
            "authors": [
                "Hoo-Chang Shin",
                "Yang Zhang",
                "Evelina Bakhturina",
                "Raul Puri",
                "Mostofa Patwary",
                "Mohammad Shoeybi",
                "Raghav Mani"
            ],
            "abstract": "There has been an influx of biomedical domain-specific language models,\nshowing language models pre-trained on biomedical text perform better on\nbiomedical domain benchmarks than those trained on general domain text corpora\nsuch as Wikipedia and Books. Yet, most works do not study the factors affecting\neach domain language application deeply. Additionally, the study of model size\non domain-specific models has been mostly missing. We empirically study and\nevaluate several factors that can affect performance on domain language\napplications, such as the sub-word vocabulary set, model size, pre-training\ncorpus, and domain transfer. We show consistent improvements on benchmarks with\nour larger BioMegatron model trained on a larger domain corpus, contributing to\nour understanding of domain language model applications. We demonstrate\nnoticeable improvements over the previous state-of-the-art (SOTA) on standard\nbiomedical NLP benchmarks of named entity recognition, relation extraction, and\nquestion answering. Model checkpoints and code are available at\n[https://ngc.nvidia.com] and [https://github.com/NVIDIA/NeMo].",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06069v2",
            "date": "2020-10-12 23:12:11+00:00",
            "title": "Are Some Words Worth More than Others?",
            "authors": [
                "Shiran Dudy",
                "Steven Bedrick"
            ],
            "abstract": "Current evaluation metrics for language modeling and generation rely heavily\non the accuracy of predicted (or generated) words as compared to a reference\nground truth. While important, token-level accuracy only captures one aspect of\na language model's behavior, and ignores linguistic properties of words that\nmay allow some mis-predicted tokens to be useful in practice. Furthermore,\nstatistics directly tied to prediction accuracy (including perplexity) may be\nconfounded by the Zipfian nature of written language, as the majority of the\nprediction attempts will occur with frequently-occurring types. A model's\nperformance may vary greatly between high- and low-frequency words, which in\npractice could lead to failure modes such as repetitive and dull generated text\nbeing produced by a downstream consumer of a language model. To address this,\nwe propose two new intrinsic evaluation measures within the framework of a\nsimple word prediction task that are designed to give a more holistic picture\nof a language model's performance. We evaluate several commonly-used large\nEnglish language models using our proposed metrics, and demonstrate that our\napproach reveals functional differences in performance between the models that\nare obscured by more traditional metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06115v2",
            "date": "2020-10-13 01:58:29+00:00",
            "title": "Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks",
            "authors": [
                "Yuanhe Tian",
                "Yan Song",
                "Fei Xia"
            ],
            "abstract": "Supertagging is conventionally regarded as an important task for combinatory\ncategorial grammar (CCG) parsing, where effective modeling of contextual\ninformation is highly important to this task. However, existing studies have\nmade limited efforts to leverage contextual features except for applying\npowerful encoders (e.g., bi-LSTM). In this paper, we propose attentive graph\nconvolutional networks to enhance neural CCG supertagging through a novel\nsolution of leveraging contextual information. Specifically, we build the graph\nfrom chunks (n-grams) extracted from a lexicon and apply attention over the\ngraph, so that different word pairs from the contexts within and across chunks\nare weighted in the model and facilitate the supertagging accordingly. The\nexperiments performed on the CCGbank demonstrate that our approach outperforms\nall previous studies in terms of both supertagging and parsing. Further\nanalyses illustrate the effectiveness of each component in our approach to\ndiscriminatively learn from word pairs to enhance CCG supertagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06122v1",
            "date": "2020-10-13 02:27:05+00:00",
            "title": "Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options",
            "authors": [
                "Clara Vania",
                "Ruijie Chen",
                "Samuel R. Bowman"
            ],
            "abstract": "Large-scale natural language inference (NLI) datasets such as SNLI or MNLI\nhave been created by asking crowdworkers to read a premise and write three new\nhypotheses, one for each possible semantic relationships (entailment,\ncontradiction, and neutral). While this protocol has been used to create useful\nbenchmark data, it remains unclear whether the writing-based annotation\nprotocol is optimal for any purpose, since it has not been evaluated directly.\nFurthermore, there is ample evidence that crowdworker writing can introduce\nartifacts in the data. We investigate two alternative protocols which\nautomatically create candidate (premise, hypothesis) pairs for annotators to\nlabel. Using these protocols and a writing-based baseline, we collect several\nnew English NLI datasets of over 3k examples each, each using a fixed amount of\nannotator time, but a varying number of examples to fit that time budget. Our\nexperiments on NLI and transfer learning show negative results: None of the\nalternative protocols outperforms the baseline in evaluations of generalization\nwithin NLI or on transfer to outside target tasks. We conclude that crowdworker\nwriting still the best known option for entailment data, highlighting the need\nfor further data collection work to focus on improving writing-based annotation\nprocesses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06137v1",
            "date": "2020-10-13 03:17:34+00:00",
            "title": "Corruption Is Not All Bad: Incorporating Discourse Structure into Pre-training via Corruption for Essay Scoring",
            "authors": [
                "Farjana Sultana Mim",
                "Naoya Inoue",
                "Paul Reisert",
                "Hiroki Ouchi",
                "Kentaro Inui"
            ],
            "abstract": "Existing approaches for automated essay scoring and document representation\nlearning typically rely on discourse parsers to incorporate discourse structure\ninto text representation. However, the performance of parsers is not always\nadequate, especially when they are used on noisy texts, such as student essays.\nIn this paper, we propose an unsupervised pre-training approach to capture\ndiscourse structure of essays in terms of coherence and cohesion that does not\nrequire any discourse parser or annotation. We introduce several types of\ntoken, sentence and paragraph-level corruption techniques for our proposed\npre-training approach and augment masked language modeling pre-training with\nour pre-training method to leverage both contextualized and discourse\ninformation. Our proposed unsupervised approach achieves new state-of-the-art\nresult on essay Organization scoring task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06138v1",
            "date": "2020-10-13 03:25:15+00:00",
            "title": "Incorporating BERT into Parallel Sequence Decoding with Adapters",
            "authors": [
                "Junliang Guo",
                "Zhirui Zhang",
                "Linli Xu",
                "Hao-Ran Wei",
                "Boxing Chen",
                "Enhong Chen"
            ],
            "abstract": "While large scale pre-trained language models such as BERT have achieved\ngreat success on various natural language understanding tasks, how to\nefficiently and effectively incorporate them into sequence-to-sequence models\nand the corresponding text generation tasks remains a non-trivial problem. In\nthis paper, we propose to address this problem by taking two different BERT\nmodels as the encoder and decoder respectively, and fine-tuning them by\nintroducing simple and lightweight adapter modules, which are inserted between\nBERT layers and tuned on the task-specific dataset. In this way, we obtain a\nflexible and efficient model which is able to jointly leverage the information\ncontained in the source-side and target-side BERT models, while bypassing the\ncatastrophic forgetting problem. Each component in the framework can be\nconsidered as a plug-in unit, making the framework flexible and task agnostic.\nOur framework is based on a parallel sequence decoding algorithm named\nMask-Predict considering the bi-directional and conditional independent nature\nof BERT, and can be adapted to traditional autoregressive decoding easily. We\nconduct extensive experiments on neural machine translation tasks where the\nproposed method consistently outperforms autoregressive baselines while\nreducing the inference latency by half, and achieves $36.49$/$33.57$ BLEU\nscores on IWSLT14 German-English/WMT14 German-English translation. When adapted\nto autoregressive decoding, the proposed method achieves $30.60$/$43.56$ BLEU\nscores on WMT14 English-German/English-French translation, on par with the\nstate-of-the-art baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06185v1",
            "date": "2020-10-13 05:22:30+00:00",
            "title": "The workweek is the best time to start a family -- A Study of GPT-2 Based Claim Generation",
            "authors": [
                "Shai Gretz",
                "Yonatan Bilu",
                "Edo Cohen-Karlik",
                "Noam Slonim"
            ],
            "abstract": "Argument generation is a challenging task whose research is timely\nconsidering its potential impact on social media and the dissemination of\ninformation. Here we suggest a pipeline based on GPT-2 for generating coherent\nclaims, and explore the types of claims that it produces, and their veracity,\nusing an array of manual and automatic assessments. In addition, we explore the\ninterplay between this task and the task of Claim Retrieval, showing how they\ncan complement one another.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06189v3",
            "date": "2020-10-13 05:29:56+00:00",
            "title": "X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models",
            "authors": [
                "Zhengbao Jiang",
                "Antonios Anastasopoulos",
                "Jun Araki",
                "Haibo Ding",
                "Graham Neubig"
            ],
            "abstract": "Language models (LMs) have proven surprisingly successful at capturing\nfactual knowledge by completing cloze-style fill-in-the-blank questions such as\n\"Punta Cana is located in _.\" However, while knowledge is both written and\nqueried in many languages, studies on LMs' factual representation ability have\nalmost invariably been performed on English. To assess factual knowledge\nretrieval in LMs in different languages, we create a multilingual benchmark of\ncloze-style probes for 23 typologically diverse languages. To properly handle\nlanguage variations, we expand probing methods from single- to multi-word\nentities, and develop several decoding algorithms to generate multi-token\npredictions. Extensive experimental results provide insights about how well (or\npoorly) current state-of-the-art LMs perform at this task in languages with\nmore or fewer available resources. We further propose a code-switching-based\nmethod to improve the ability of multilingual LMs to access knowledge, and\nverify its effectiveness on several benchmark languages. Benchmark data and\ncode have been released at https://x-factr.github.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06203v2",
            "date": "2020-10-13 07:07:59+00:00",
            "title": "Mitigating Gender Bias in Machine Translation with Target Gender Annotations",
            "authors": [
                "Art\u016brs Stafanovi\u010ds",
                "Toms Bergmanis",
                "M\u0101rcis Pinnis"
            ],
            "abstract": "When translating \"The secretary asked for details.\" to a language with\ngrammatical gender, it might be necessary to determine the gender of the\nsubject \"secretary\". If the sentence does not contain the necessary\ninformation, it is not always possible to disambiguate. In such cases, machine\ntranslation systems select the most common translation option, which often\ncorresponds to the stereotypical translations, thus potentially exacerbating\nprejudice and marginalisation of certain groups and people. We argue that the\ninformation necessary for an adequate translation can not always be deduced\nfrom the sentence being translated or even might depend on external knowledge.\nTherefore, in this work, we propose to decouple the task of acquiring the\nnecessary information from the task of learning to translate correctly when\nsuch information is available. To that end, we present a method for training\nmachine translation systems to use word-level annotations containing\ninformation about subject's gender. To prepare training data, we annotate\nregular source language words with grammatical gender information of the\ncorresponding target language words. Using such data to train machine\ntranslation systems reduces their reliance on gender stereotypes when\ninformation about the subject's gender is available. Our experiments on five\nlanguage pairs show that this allows improving accuracy on the WinoMT test set\nby up to 25.8 percentage points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06213v1",
            "date": "2020-10-13 07:42:25+00:00",
            "title": "KLearn: Background Knowledge Inference from Summarization Data",
            "authors": [
                "Maxime Peyrard",
                "Robert West"
            ],
            "abstract": "The goal of text summarization is to compress documents to the relevant\ninformation while excluding background information already known to the\nreceiver. So far, summarization researchers have given considerably more\nattention to relevance than to background knowledge. In contrast, this work\nputs background knowledge in the foreground. Building on the realization that\nthe choices made by human summarizers and annotators contain implicit\ninformation about their background knowledge, we develop and compare techniques\nfor inferring background knowledge from summarization data. Based on this\nframework, we define summary scoring functions that explicitly model background\nknowledge, and show that these scoring functions fit human judgments\nsignificantly better than baselines. We illustrate some of the many potential\napplications of our framework. First, we provide insights into human\ninformation importance priors. Second, we demonstrate that averaging the\nbackground knowledge of multiple, potentially biased annotators or corpora\ngreatly improves summary-scoring performance. Finally, we discuss potential\napplications of our framework beyond summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06251v2",
            "date": "2020-10-13 09:22:48+00:00",
            "title": "Annotationsaurus: A Searchable Directory of Annotation Tools",
            "authors": [
                "Mariana Neves",
                "Jurica Seva"
            ],
            "abstract": "Manual annotation of textual documents is a necessary task when constructing\nbenchmark corpora for training and evaluating machine learning algorithms. We\ncreated a comprehensive directory of annotation tools that currently includes\n93 tools. We analyzed the tools over a set of 31 features and implemented\nsimple scripts and a Web application that filters the tools based on chosen\ncriteria. We present two use cases using the directory and propose ideas for\nits maintenance. The directory, source codes for scripts, and link to the Web\napplication are available at: https://github.com/mariananeves/annotation-tools",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06283v1",
            "date": "2020-10-13 10:53:20+00:00",
            "title": "F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering",
            "authors": [
                "Hendrik Schuff",
                "Heike Adel",
                "Ngoc Thang Vu"
            ],
            "abstract": "Explainable question answering systems predict an answer together with an\nexplanation showing why the answer has been selected. The goal is to enable\nusers to assess the correctness of the system and understand its reasoning\nprocess. However, we show that current models and evaluation settings have\nshortcomings regarding the coupling of answer and explanation which might cause\nserious issues in user experience. As a remedy, we propose a hierarchical model\nand a new regularization term to strengthen the answer-explanation coupling as\nwell as two evaluation scores to quantify the coupling. We conduct experiments\non the HOTPOTQA benchmark data set and perform a user study. The user study\nshows that our models increase the ability of the users to judge the\ncorrectness of the system and that scores like F1 are not enough to estimate\nthe usefulness of a model in a practical setting with human users. Our scores\nare better aligned with user experience, making them promising candidates for\nmodel selection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06294v1",
            "date": "2020-10-13 11:19:42+00:00",
            "title": "Extending Implicit Discourse Relation Recognition to the PDTB-3",
            "authors": [
                "Li Liang",
                "Zheng Zhao",
                "Bonnie Webber"
            ],
            "abstract": "The PDTB-3 contains many more Implicit discourse relations than the previous\nPDTB-2. This is in part because implicit relations have now been annotated\nwithin sentences as well as between them. In addition, some now co-occur with\nexplicit discourse relations, instead of standing on their own. Here we show\nthat while this can complicate the problem of identifying the location of\nimplicit discourse relations, it can in turn simplify the problem of\nidentifying their senses. We present data to support this claim, as well as\nmethods that can serve as a non-trivial baseline for future state-of-the-art\nrecognizers for implicit discourse relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06310v2",
            "date": "2020-10-13 11:51:17+00:00",
            "title": "Cross-Supervised Joint-Event-Extraction with Heterogeneous Information Networks",
            "authors": [
                "Yue Wang",
                "Zhuo Xu",
                "Lu Bai",
                "Yao Wan",
                "Lixin Cui",
                "Qian Zhao",
                "Edwin R. Hancock",
                "Philip S. Yu"
            ],
            "abstract": "Joint-event-extraction, which extracts structural information (i.e., entities\nor triggers of events) from unstructured real-world corpora, has attracted more\nand more research attention in natural language processing. Most existing works\ndo not fully address the sparse co-occurrence relationships between entities\nand triggers, which loses this important information and thus deteriorates the\nextraction performance. To mitigate this issue, we first define the\njoint-event-extraction as a sequence-to-sequence labeling task with a tag set\ncomposed of tags of triggers and entities. Then, to incorporate the missing\ninformation in the aforementioned co-occurrence relationships, we propose a\nCross-Supervised Mechanism (CSM) to alternately supervise the extraction of\neither triggers or entities based on the type distribution of each other.\nMoreover, since the connected entities and triggers naturally form a\nheterogeneous information network (HIN), we leverage the latent pattern along\nmeta-paths for a given corpus to further improve the performance of our\nproposed method. To verify the effectiveness of our proposed method, we conduct\nextensive experiments on four real-world datasets as well as compare our method\nwith state-of-the-art methods. Empirical results and analysis show that our\napproach outperforms the state-of-the-art methods in both entity and trigger\nextraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06351v4",
            "date": "2020-10-13 13:08:34+00:00",
            "title": "CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations",
            "authors": [
                "Fuli Luo",
                "Pengcheng Yang",
                "Shicheng Li",
                "Xuancheng Ren",
                "Xu Sun"
            ],
            "abstract": "Pre-trained self-supervised models such as BERT have achieved striking\nsuccess in learning sequence representations, especially for natural language\nprocessing. These models typically corrupt the given sequences with certain\ntypes of noise, such as masking, shuffling, or substitution, and then try to\nrecover the original input. However, such pre-training approaches are prone to\nlearning representations that are covariant with the noise, leading to the\ndiscrepancy between the pre-training and fine-tuning stage. To remedy this, we\npresent ContrAstive Pre-Training (CAPT) to learn noise invariant sequence\nrepresentations. The proposed CAPT encourages the consistency between\nrepresentations of the original sequence and its corrupted version via\nunsupervised instance-wise training signals. In this way, it not only\nalleviates the pretrain-finetune discrepancy induced by the noise of\npre-training, but also aids the pre-trained model in better capturing global\nsemantics of the input via more effective sentence-level supervision. Different\nfrom most prior work that focuses on a particular modality, comprehensive\nempirical evidence on 11 natural language understanding and cross-modal tasks\nillustrates that CAPT is applicable for both language and vision-language\ntasks, and obtains surprisingly consistent improvement, including 0.6\\%\nabsolute gain on GLUE benchmarks and 0.8\\% absolute increment on\n$\\text{NLVR}^2$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06354v1",
            "date": "2020-10-13 13:12:21+00:00",
            "title": "The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT",
            "authors": [
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "This paper describes the development of a new benchmark for machine\ntranslation that provides training and test data for thousands of language\npairs covering over 500 languages and tools for creating state-of-the-art\ntranslation models from that collection. The main goal is to trigger the\ndevelopment of open translation tools and models with a much broader coverage\nof the World's languages. Using the package it is possible to work on realistic\nlow-resource scenarios avoiding artificially reduced setups that are common\nwhen demonstrating zero-shot or few-shot learning. For the first time, this\npackage provides a comprehensive collection of diverse data sets in hundreds of\nlanguages with systematic language and script annotation and data splits to\nextend the narrow coverage of existing benchmarks. Together with the data\nrelease, we also provide a growing number of pre-trained baseline models for\nindividual language pairs and selected language groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06359v2",
            "date": "2020-10-13 13:14:37+00:00",
            "title": "Fine-grained linguistic evaluation for state-of-the-art Machine Translation",
            "authors": [
                "Eleftherios Avramidis",
                "Vivien Macketanz",
                "Ursula Strohriegel",
                "Aljoscha Burchardt",
                "Sebastian M\u00f6ller"
            ],
            "abstract": "This paper describes a test suite submission providing detailed statistics of\nlinguistic performance for the state-of-the-art German-English systems of the\nFifth Conference of Machine Translation (WMT20). The analysis covers 107\nphenomena organized in 14 categories based on about 5,500 test items, including\na manual annotation effort of 45 person hours. Two systems (Tohoku and Huoshan)\nappear to have significantly better test suite accuracy than the others,\nalthough the best system of WMT20 is not significantly better than the one from\nWMT19 in a macro-average. Additionally, we identify some linguistic phenomena\nwhere all systems suffer (such as idioms, resultative predicates and\npluperfect), but we are also able to identify particular weaknesses for\nindividual systems (such as quotation marks, lexical ambiguity and sluicing).\nMost of the systems of WMT19 which submitted new versions this year show\nimprovements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06396v2",
            "date": "2020-10-13 13:51:57+00:00",
            "title": "Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension",
            "authors": [
                "Ekta Sood",
                "Simon Tannert",
                "Diego Frassinelli",
                "Andreas Bulling",
                "Ngoc Thang Vu"
            ],
            "abstract": "While neural networks with attention mechanisms have achieved superior\nperformance on many natural language processing tasks, it remains unclear to\nwhich extent learned attention resembles human visual attention. In this paper,\nwe propose a new method that leverages eye-tracking data to investigate the\nrelationship between human visual attention and neural attention in machine\nreading comprehension. To this end, we introduce a novel 23 participant eye\ntracking dataset - MQA-RC, in which participants read movie plots and answered\npre-defined questions. We compare state of the art networks based on long\nshort-term memory (LSTM), convolutional neural models (CNN) and XLNet\nTransformer architectures. We find that higher similarity to human attention\nand performance significantly correlates to the LSTM and CNN models. However,\nwe show this relationship does not hold true for the XLNet models -- despite\nthe fact that the XLNet performs best on this challenging task. Our results\nsuggest that different architectures seem to learn rather different neural\nattention strategies and similarity of neural to human attention does not\nguarantee best performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06436v1",
            "date": "2020-10-13 14:54:05+00:00",
            "title": "RuSemShift: a dataset of historical lexical semantic change in Russian",
            "authors": [
                "Julia Rodina",
                "Andrey Kutuzov"
            ],
            "abstract": "We present RuSemShift, a large-scale manually annotated test set for the task\nof semantic change modeling in Russian for two long-term time period pairs:\nfrom the pre-Soviet through the Soviet times and from the Soviet through the\npost-Soviet times. Target words were annotated by multiple crowd-source\nworkers. The annotation process was organized following the DURel framework and\nwas based on sentence contexts extracted from the Russian National Corpus.\nAdditionally, we report the performance of several distributional approaches on\nRuSemShift, achieving promising results, which at the same time leave room for\nother researchers to improve.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06447v2",
            "date": "2020-10-13 15:06:07+00:00",
            "title": "Pagsusuri ng RNN-based Transfer Learning Technique sa Low-Resource Language",
            "authors": [
                "Dan John Velasco"
            ],
            "abstract": "Low-resource languages such as Filipino suffer from data scarcity which makes\nit challenging to develop NLP applications for Filipino language. The use of\nTransfer Learning (TL) techniques alleviates this problem in low-resource\nsetting. In recent years, transformer-based models are proven to be effective\nin low-resource tasks but faces challenges in accessibility due to its high\ncompute and memory requirements. For this reason, there's a need for a cheaper\nbut effective alternative. This paper has three contributions. First, release a\npre-trained AWD-LSTM language model for Filipino language. Second, benchmark\nAWD-LSTM in the Hate Speech classification task and show that it performs on\npar with transformer-based models. Third, analyze the the performance of\nAWD-LSTM in low-resource setting using degradation test and compare it with\ntransformer-based models.\n  -----\n  Ang mga low-resource languages tulad ng Filipino ay gipit sa accessible na\ndatos kaya't mahirap gumawa ng mga applications sa wikang ito. Ang mga Transfer\nLearning (TL) techniques ay malaking tulong para sa low-resource setting o mga\npagkakataong gipit sa datos. Sa mga nagdaang taon, nanaig ang mga\ntransformer-based TL techniques pagdating sa low-resource tasks ngunit ito ay\nmataas na compute and memory requirements kaya nangangailangan ng mas mura pero\nepektibong alternatibo. Ang papel na ito ay may tatlong kontribusyon. Una,\nmaglabas ng pre-trained AWD-LSTM language model sa wikang Filipino upang maging\ntuntungan sa pagbuo ng mga NLP applications sa wikang Filipino. Pangalawa, mag\nbenchmark ng AWD-LSTM sa Hate Speech classification task at ipakita na kayang\nnitong makipagsabayan sa mga transformer-based models. Pangatlo, suriin ang\nperformance ng AWD-LSTM sa low-resource setting gamit ang degradation test at\nikumpara ito sa mga transformer-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06472v1",
            "date": "2020-10-13 15:25:33+00:00",
            "title": "Demographic Representation and Collective Storytelling in the Me Too Twitter Hashtag Activism Movement",
            "authors": [
                "Aaron Mueller",
                "Zach Wood-Doughty",
                "Silvio Amir",
                "Mark Dredze",
                "Alicia L. Nobles"
            ],
            "abstract": "The #MeToo movement on Twitter has drawn attention to the pervasive nature of\nsexual harassment and violence. While #MeToo has been praised for providing\nsupport for self-disclosures of harassment or violence and shifting societal\nresponse, it has also been criticized for exemplifying how women of color have\nbeen discounted for their historical contributions to and excluded from\nfeminist movements. Through an analysis of over 600,000 tweets from over\n256,000 unique users, we examine online #MeToo conversations across gender and\nracial/ethnic identities and the topics that each demographic emphasized. We\nfound that tweets authored by white women were overrepresented in the movement\ncompared to other demographics, aligning with criticism of unequal\nrepresentation. We found that intersected identities contributed differing\nnarratives to frame the movement, co-opted the movement to raise visibility in\nparallel ongoing movements, employed the same hashtags both critically and\nsupportively, and revived and created new hashtags in response to pivotal\nmoments. Notably, tweets authored by black women often expressed emotional\nsupport and were critical about differential treatment in the justice system\nand by police. In comparison, tweets authored by white women and men often\nhighlighted sexual harassment and violence by public figures and weaved in more\ngeneral political discussions. We discuss the implications of work for digital\nactivism research and design including suggestions to raise visibility by those\nwho were under-represented in this hashtag activism movement. Content warning:\nthis article discusses issues of sexual harassment and violence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06478v1",
            "date": "2020-10-13 15:32:00+00:00",
            "title": "XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization",
            "authors": [
                "Alessandro Raganato",
                "Tommaso Pasini",
                "Jose Camacho-Collados",
                "Mohammad Taher Pilehvar"
            ],
            "abstract": "The ability to correctly model distinct meanings of a word is crucial for the\neffectiveness of semantic representation techniques. However, most existing\nevaluation benchmarks for assessing this criterion are tied to sense\ninventories (usually WordNet), restricting their usage to a small subset of\nknowledge-based representation techniques. The Word-in-Context dataset (WiC)\naddresses the dependence on sense inventories by reformulating the standard\ndisambiguation task as a binary classification problem; but, it is limited to\nthe English language. We put forward a large multilingual benchmark, XL-WiC,\nfeaturing gold standards in 12 new languages from varied language families and\nwith different degrees of resource availability, opening room for evaluation\nscenarios such as zero-shot cross-lingual transfer. We perform a series of\nexperiments to determine the reliability of the datasets and to set performance\nbaselines for several recent contextualized multilingual models. Experimental\nresults show that even when no tagged instances are available for a target\nlanguage, models trained solely on the English data can attain competitive\nperformance in the task of distinguishing different meanings of a word, even\nfor distant languages. XL-WiC is available at\nhttps://pilehvar.github.io/xlwic/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06640v2",
            "date": "2020-10-13 19:13:07+00:00",
            "title": "Enhancing the Identification of Cyberbullying through Participant Roles",
            "authors": [
                "Gathika Ratnayaka",
                "Thushari Atapattu",
                "Mahen Herath",
                "Georgia Zhang",
                "Katrina Falkner"
            ],
            "abstract": "Cyberbullying is a prevalent social problem that inflicts detrimental\nconsequences to the health and safety of victims such as psychological\ndistress, anti-social behaviour, and suicide. The automation of cyberbullying\ndetection is a recent but widely researched problem, with current research\nhaving a strong focus on a binary classification of bullying versus\nnon-bullying. This paper proposes a novel approach to enhancing cyberbullying\ndetection through role modeling. We utilise a dataset from ASKfm to perform\nmulti-class classification to detect participant roles (e.g. victim, harasser).\nOur preliminary results demonstrate promising performance including 0.83 and\n0.76 of F1-score for cyberbullying and role classification respectively,\noutperforming baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06666v1",
            "date": "2020-10-13 19:56:02+00:00",
            "title": "Probing for Multilingual Numerical Understanding in Transformer-Based Language Models",
            "authors": [
                "Devin Johnson",
                "Denise Mak",
                "Drew Barker",
                "Lexi Loessberg-Zahl"
            ],
            "abstract": "Natural language numbers are an example of compositional structures, where\nlarger numbers are composed of operations on smaller numbers. Given that\ncompositional reasoning is a key to natural language understanding, we propose\nnovel multilingual probing tasks tested on DistilBERT, XLM, and BERT to\ninvestigate for evidence of compositional reasoning over numerical data in\nvarious natural language number systems. By using both grammaticality judgment\nand value comparison classification tasks in English, Japanese, Danish, and\nFrench, we find evidence that the information encoded in these pretrained\nmodels' embeddings is sufficient for grammaticality judgments but generally not\nfor value comparisons. We analyze possible reasons for this and discuss how our\ntasks could be extended in further studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06716v1",
            "date": "2020-10-13 22:08:11+00:00",
            "title": "Sensitivity of BLANC to human-scored qualities of text summaries",
            "authors": [
                "Oleg Vasilyev",
                "Vedant Dharnidharka",
                "Nicholas Egan",
                "Charlene Chambliss",
                "John Bohannon"
            ],
            "abstract": "We explore the sensitivity of a document summary quality estimator, BLANC, to\nhuman assessment of qualities for the same summaries. In our human evaluations,\nwe distinguish five summary qualities, defined by how fluent, understandable,\ninformative, compact, and factually correct the summary is. We make the case\nfor optimal BLANC parameters, at which the BLANC sensitivity to almost all of\nsummary qualities is about as good as the sensitivity of a human annotator.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06778v1",
            "date": "2020-10-14 02:24:04+00:00",
            "title": "Google Crowdsourced Speech Corpora and Related Open-Source Resources for Low-Resource Languages and Dialects: An Overview",
            "authors": [
                "Alena Butryna",
                "Shan-Hui Cathy Chu",
                "Isin Demirsahin",
                "Alexander Gutkin",
                "Linne Ha",
                "Fei He",
                "Martin Jansche",
                "Cibu Johny",
                "Anna Katanova",
                "Oddur Kjartansson",
                "Chenfang Li",
                "Tatiana Merkulova",
                "Yin May Oo",
                "Knot Pipatsrisawat",
                "Clara Rivera",
                "Supheakmungkol Sarin",
                "Pasindu de Silva",
                "Keshan Sodimana",
                "Richard Sproat",
                "Theeraphol Wattanavekin",
                "Jaka Aris Eko Wibawa"
            ],
            "abstract": "This paper presents an overview of a program designed to address the growing\nneed for developing freely available speech resources for under-represented\nlanguages. At present we have released 38 datasets for building text-to-speech\nand automatic speech recognition applications for languages and dialects of\nSouth and Southeast Asia, Africa, Europe and South America. The paper describes\nthe methodology used for developing such corpora and presents some of our\nfindings that could benefit under-represented language communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06786v1",
            "date": "2020-10-14 02:57:10+00:00",
            "title": "A Self-supervised Representation Learning of Sentence Structure for Authorship Attribution",
            "authors": [
                "Fereshteh Jafariakinabad",
                "Kien A. Hua"
            ],
            "abstract": "Syntactic structure of sentences in a document substantially informs about\nits authorial writing style. Sentence representation learning has been widely\nexplored in recent years and it has been shown that it improves the\ngeneralization of different downstream tasks across many domains. Even though\nutilizing probing methods in several studies suggests that these learned\ncontextual representations implicitly encode some amount of syntax, explicit\nsyntactic information further improves the performance of deep neural models in\nthe domain of authorship attribution. These observations have motivated us to\ninvestigate the explicit representation learning of syntactic structure of\nsentences. In this paper, we propose a self-supervised framework for learning\nstructural representations of sentences. The self-supervised network contains\ntwo components; a lexical sub-network and a syntactic sub-network which take\nthe sequence of words and their corresponding structural labels as the input,\nrespectively. Due to the n-to-1 mapping of words to their structural labels,\neach word will be embedded into a vector representation which mainly carries\nstructural information. We evaluate the learned structural representations of\nsentences using different probing tasks, and subsequently utilize them in the\nauthorship attribution task. Our experimental results indicate that the\nstructural embeddings significantly improve the classification tasks when\nconcatenated with the existing pre-trained word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06804v1",
            "date": "2020-10-14 04:21:57+00:00",
            "title": "Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion",
            "authors": [
                "Ankur Goswami",
                "Akshata Bhat",
                "Hadar Ohana",
                "Theodoros Rekatsinas"
            ],
            "abstract": "We show that state-of-the-art self-supervised language models can be readily\nused to extract relations from a corpus without the need to train a fine-tuned\nextractive head. We introduce RE-Flex, a simple framework that performs\nconstrained cloze completion over pretrained language models to perform\nunsupervised relation extraction. RE-Flex uses contextual matching to ensure\nthat language model predictions matches supporting evidence from the input\ncorpus that is relevant to a target relation. We perform an extensive\nexperimental study over multiple relation extraction benchmarks and demonstrate\nthat RE-Flex outperforms competing unsupervised relation extraction methods\nbased on pretrained language models by up to 27.8 $F_1$ points compared to the\nnext-best method. Our results show that constrained inference queries against a\nlanguage model can enable accurate unsupervised relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06823v1",
            "date": "2020-10-14 06:27:07+00:00",
            "title": "Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems",
            "authors": [
                "Jinghui Qin",
                "Lihui Lin",
                "Xiaodan Liang",
                "Rumin Zhang",
                "Liang Lin"
            ],
            "abstract": "A practical automatic textual math word problems (MWPs) solver should be able\nto solve various textual MWPs while most existing works only focused on\none-unknown linear MWPs. Herein, we propose a simple but efficient method\ncalled Universal Expression Tree (UET) to make the first attempt to represent\nthe equations of various MWPs uniformly. Then a semantically-aligned universal\ntree-structured solver (SAU-Solver) based on an encoder-decoder framework is\nproposed to resolve multiple types of MWPs in a unified model, benefiting from\nour UET representation. Our SAU-Solver generates a universal expression tree\nexplicitly by deciding which symbol to generate according to the generated\nsymbols' semantic meanings like human solving MWPs. Besides, our SAU-Solver\nalso includes a novel subtree-level semanticallyaligned regularization to\nfurther enforce the semantic constraints and rationality of the generated\nexpression tree by aligning with the contextual information. Finally, to\nvalidate the universality of our solver and extend the research boundary of\nMWPs, we introduce a new challenging Hybrid Math Word Problems dataset (HMWP),\nconsisting of three types of MWPs. Experimental results on several MWPs\ndatasets show that our model can solve universal types of MWPs and outperforms\nseveral state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06835v1",
            "date": "2020-10-13 06:29:51+00:00",
            "title": "A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering",
            "authors": [
                "Svitlana Vakulenko",
                "Shayne Longpre",
                "Zhucheng Tu",
                "Raviteja Anantha"
            ],
            "abstract": "The dependency between an adequate question formulation and correct answer\nselection is a very intriguing but still underexplored area. In this paper, we\nshow that question rewriting (QR) of the conversational context allows to shed\nmore light on this phenomenon and also use it to evaluate robustness of\ndifferent answer selection approaches. We introduce a simple framework that\nenables an automated analysis of the conversational question answering (QA)\nperformance using question rewrites, and present the results of this analysis\non the TREC CAsT and QuAC (CANARD) datasets. Our experiments uncover\nsensitivity to question formulation of the popular state-of-the-art models for\nreading comprehension and passage ranking. Our results demonstrate that the\nreading comprehension model is insensitive to question formulation, while the\npassage ranking changes dramatically with a little variation in the input\nquestion. The benefit of QR is that it allows us to pinpoint and group such\ncases automatically. We show how to use this methodology to verify whether QA\nmodels are really learning the task or just finding shortcuts in the dataset,\nand better understand the frequent types of error they make.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06857v1",
            "date": "2020-10-14 07:47:33+00:00",
            "title": "Learning Word Representations for Tunisian Sentiment Analysis",
            "authors": [
                "Abir Messaoudi",
                "Hatem Haddad",
                "Moez Ben HajHmida",
                "Chayma Fourati",
                "Abderrazak Ben Hamida"
            ],
            "abstract": "Tunisians on social media tend to express themselves in their local dialect\nusing Latin script (TUNIZI). This raises an additional challenge to the process\nof exploring and recognizing online opinions. To date, very little work has\naddressed TUNIZI sentiment analysis due to scarce resources for training an\nautomated system. In this paper, we focus on the Tunisian dialect sentiment\nanalysis used on social media. Most of the previous work used machine learning\ntechniques combined with handcrafted features. More recently, Deep Neural\nNetworks were widely used for this task, especially for the English language.\nIn this paper, we explore the importance of various unsupervised word\nrepresentations (word2vec, BERT) and we investigate the use of Convolutional\nNeural Networks and Bidirectional Long Short-Term Memory. Without using any\nkind of handcrafted features, our experimental results on two publicly\navailable datasets showed comparable performances to other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06858v1",
            "date": "2020-10-14 07:52:47+00:00",
            "title": "fugashi, a Tool for Tokenizing Japanese in Python",
            "authors": [
                "Paul McCann"
            ],
            "abstract": "Recent years have seen an increase in the number of large-scale multilingual\nNLP projects. However, even in such projects, languages with special processing\nrequirements are often excluded. One such language is Japanese. Japanese is\nwritten without spaces, tokenization is non-trivial, and while high quality\nopen source tokenizers exist they can be hard to use and lack English\ndocumentation. This paper introduces fugashi, a MeCab wrapper for Python, and\ngives an introduction to tokenizing Japanese.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06891v1",
            "date": "2020-10-14 09:03:36+00:00",
            "title": "Memformer: The Memory-Augmented Transformer",
            "authors": [
                "Qingyang Wu",
                "Zhenzhong Lan",
                "Jing Gu",
                "Zhou Yu"
            ],
            "abstract": "Transformer models have obtained remarkable accomplishments in various NLP\ntasks. However, these models have efficiency issues on long sequences, as the\ncomplexity of their self-attention module scales quadratically with the\nsequence length. To remedy the limitation, we present Memformer, a novel\nlanguage model that utilizes a single unified memory to encode and retrieve\npast information. It includes a new optimization scheme, Memory Replay\nBack-Propagation, which promotes long-range back-propagation through time with\na significantly reduced memory requirement. Memformer achieves $\\mathcal{O}(n)$\ntime complexity and $\\mathcal{O}(1)$ space complexity in processing long\nsequences, meaning that the model can handle an infinite length sequence during\ninference. Our model is also compatible with other self-supervised tasks to\nfurther improve the performance on language modeling. Experimental results show\nthat Memformer outperforms the previous long-range sequence models on\nWikiText-103, including Transformer-XL and compressive Transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06925v2",
            "date": "2020-10-14 10:09:01+00:00",
            "title": "DA-Transformer: Distance-aware Transformer",
            "authors": [
                "Chuhan Wu",
                "Fangzhao Wu",
                "Yongfeng Huang"
            ],
            "abstract": "Transformer has achieved great success in the NLP field by composing various\nadvanced models like BERT and GPT. However, Transformer and its existing\nvariants may not be optimal in capturing token distances because the position\nor distance embeddings used by these methods usually cannot keep the precise\ninformation of real distances, which may not be beneficial for modeling the\norders and relations of contexts. In this paper, we propose DA-Transformer,\nwhich is a distance-aware Transformer that can exploit the real distance. We\npropose to incorporate the real distances between tokens to re-scale the raw\nself-attention weights, which are computed by the relevance between attention\nquery and key. Concretely, in different self-attention heads the relative\ndistance between each pair of tokens is weighted by different learnable\nparameters, which control the different preferences on long- or short-term\ninformation of these heads. Since the raw weighted real distances may not be\noptimal for adjusting self-attention weights, we propose a learnable sigmoid\nfunction to map them into re-scaled coefficients that have proper ranges. We\nfirst clip the raw self-attention weights via the ReLU function to keep\nnon-negativity and introduce sparsity, and then multiply them with the\nre-scaled coefficients to encode real distance information into self-attention.\nExtensive experiments on five benchmark datasets show that DA-Transformer can\neffectively improve the performance of many tasks and outperform the vanilla\nTransformer and its several variants.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07048v1",
            "date": "2020-10-14 12:55:36+00:00",
            "title": "Chinese Lexical Simplification",
            "authors": [
                "Jipeng Qiang",
                "Xinyu Lu",
                "Yun Li",
                "Yunhao Yuan",
                "Yang Shi",
                "Xindong Wu"
            ],
            "abstract": "Lexical simplification has attracted much attention in many languages, which\nis the process of replacing complex words in a given sentence with simpler\nalternatives of equivalent meaning. Although the richness of vocabulary in\nChinese makes the text very difficult to read for children and non-native\nspeakers, there is no research work for Chinese lexical simplification (CLS)\ntask. To circumvent difficulties in acquiring annotations, we manually create\nthe first benchmark dataset for CLS, which can be used for evaluating the\nlexical simplification systems automatically. In order to acquire more thorough\ncomparison, we present five different types of methods as baselines to generate\nsubstitute candidates for the complex word that include synonym-based approach,\nword embedding-based approach, pretrained language model-based approach,\nsememe-based approach, and a hybrid approach. Finally, we design the\nexperimental evaluation of these baselines and discuss their advantages and\ndisadvantages. To our best knowledge, this is the first study for CLS task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07074v1",
            "date": "2020-10-14 13:22:20+00:00",
            "title": "Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries",
            "authors": [
                "Xiaofei Sun",
                "Chun Fan",
                "Zijun Sun",
                "Yuxian Meng",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Long-text generation remains a challenge. The difficulty of generating\ncoherent long texts lies in the fact that existing models overwhelmingly focus\non the tasks of local word prediction, and cannot make high level plans on what\nto generate or capture the high-level discourse dependencies between chunks of\ntexts. Inspired by how humans write, where a list of bullet points or a catalog\nis first outlined, and then each bullet point is expanded to form the whole\narticle, we propose {\\it SOE}, a pipelined system that involves of summarizing,\noutlining and elaborating for long text generation: the model first outlines\nthe summaries for different segments of long texts, and then elaborates on each\nbullet point to generate the corresponding segment. To avoid the\nlabor-intensive process of summary soliciting, we propose the {\\it\nreconstruction} strategy, which extracts segment summaries in an unsupervised\nmanner by selecting its most informative part to reconstruct the segment.The\nproposed generation system comes with the following merits: (1) the summary\nprovides high-level guidances for text generation and avoids the local minimum\nof individual word predictions; (2) the high-level discourse dependencies are\ncaptured in the conditional dependencies between summaries and are preserved\nduring the summary expansion process and (3) additionally, we are able to\nconsider significantly more contexts by representing contexts as concise\nsummaries. Extensive experiments demonstrate that SOE produces long texts with\nsignificantly better quality, along with faster convergence speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07101v1",
            "date": "2020-10-14 13:59:07+00:00",
            "title": "Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction",
            "authors": [
                "Xu Zhao",
                "Zihao Wang",
                "Hao Wu",
                "Yong Zhang"
            ],
            "abstract": "Semi-supervision is a promising paradigm for Bilingual Lexicon Induction\n(BLI) with limited annotations. However, previous semisupervised methods do not\nfully utilize the knowledge hidden in annotated and nonannotated data, which\nhinders further improvement of their performance. In this paper, we propose a\nnew semi-supervised BLI framework to encourage the interaction between the\nsupervised signal and unsupervised alignment. We design two message-passing\nmechanisms to transfer knowledge between annotated and non-annotated data,\nnamed prior optimal transport and bi-directional lexicon update respectively.\nThen, we perform semi-supervised learning based on a cyclic or a parallel\nparameter feeding routine to update our models. Our framework is a general\nframework that can incorporate any supervised and unsupervised BLI methods\nbased on optimal transport. Experimental results on MUSE and VecMap datasets\nshow significant improvement of our models. Ablation study also proves that the\ntwo-way interaction between the supervised signal and unsupervised alignment\naccounts for the gain of the overall performance. Results on distant language\npairs further illustrate the advantage and robustness of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07109v1",
            "date": "2020-10-14 14:05:06+00:00",
            "title": "An Investigation on Different Underlying Quantization Schemes for Pre-trained Language Models",
            "authors": [
                "Zihan Zhao",
                "Yuncong Liu",
                "Lu Chen",
                "Qi Liu",
                "Rao Ma",
                "Kai Yu"
            ],
            "abstract": "Recently, pre-trained language models like BERT have shown promising\nperformance on multiple natural language processing tasks. However, the\napplication of these models has been limited due to their huge size. To reduce\nits size, a popular and efficient way is quantization. Nevertheless, most of\nthe works focusing on BERT quantization adapted primary linear clustering as\nthe quantization scheme, and few works try to upgrade it. That limits the\nperformance of quantization significantly. In this paper, we implement k-means\nquantization and compare its performance on the fix-precision quantization of\nBERT with linear quantization. Through the comparison, we verify that the\neffect of the underlying quantization scheme upgrading is underestimated and\nthere is a huge development potential of k-means quantization. Besides, we also\ncompare the two quantization schemes on ALBERT models to explore the robustness\ndifferences between different pre-trained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07174v1",
            "date": "2020-10-14 15:46:17+00:00",
            "title": "The EOS Decision and Length Extrapolation",
            "authors": [
                "Benjamin Newman",
                "John Hewitt",
                "Percy Liang",
                "Christopher D. Manning"
            ],
            "abstract": "Extrapolation to unseen sequence lengths is a challenge for neural generative\nmodels of language. In this work, we characterize the effect on length\nextrapolation of a modeling decision often overlooked: predicting the end of\nthe generative process through the use of a special end-of-sequence (EOS)\nvocabulary item. We study an oracle setting - forcing models to generate to the\ncorrect sequence length at test time - to compare the length-extrapolative\nbehavior of networks trained to predict EOS (+EOS) with networks not trained to\n(-EOS). We find that -EOS substantially outperforms +EOS, for example\nextrapolating well to lengths 10 times longer than those seen at training time\nin a bracket closing task, as well as achieving a 40% improvement over +EOS in\nthe difficult SCAN dataset length generalization task. By comparing the hidden\nstates and dynamics of -EOS and +EOS models, we observe that +EOS models fail\nto generalize because they (1) unnecessarily stratify their hidden states by\ntheir linear position is a sequence (structures we call length manifolds) or\n(2) get stuck in clusters (which we refer to as length attractors) once the EOS\ntoken is the highest-probability prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07279v2",
            "date": "2020-10-14 17:54:42+00:00",
            "title": "Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey",
            "authors": [
                "Khyathi Raghavi Chandu",
                "Alan W Black"
            ],
            "abstract": "Neural text generation metamorphosed into several critical natural language\napplications ranging from text completion to free form narrative generation. In\norder to progress research in text generation, it is critical to absorb the\nexisting research works and position ourselves in this massively growing field.\nSpecifically, this paper surveys the fundamental components of modeling\napproaches relaying task agnostic impacts across various generation tasks such\nas storytelling, summarization, translation etc., In this context, we present\nan abstraction of the imperative techniques with respect to learning paradigms,\npretraining, modeling approaches, decoding and the key challenges outstanding\nin the field in each of them. Thereby, we deliver a one-stop destination for\nresearchers in the field to facilitate a perspective on where to situate their\nwork and how it impacts other closely related generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07375v1",
            "date": "2020-10-14 19:32:56+00:00",
            "title": "Decoding Methods for Neural Narrative Generation",
            "authors": [
                "Alexandra DeLucia",
                "Aaron Mueller",
                "Xiang Lisa Li",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Narrative generation is an open-ended NLP task in which a model generates a\nstory given a prompt. The task is similar to neural response generation for\nchatbots; however, innovations in response generation are often not applied to\nnarrative generation, despite the similarity between these tasks. We aim to\nbridge this gap by applying and evaluating advances in decoding methods for\nneural response generation to neural narrative generation. In particular, we\nemploy GPT-2 and perform ablations across nucleus sampling thresholds and\ndiverse decoding hyperparameters---specifically, maximum mutual\ninformation---analyzing results over multiple criteria with automatic and human\nevaluation. We find that (1) nucleus sampling is generally best within $0.7\n\\leq p \\leq 0.9$; (2) a maximum mutual information objective can improve the\nquality of generated stories; and (3) established automatic metrics do not\ncorrelate well with human judgments of narrative quality on any qualitative\nmetric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07440v1",
            "date": "2020-10-14 23:50:25+00:00",
            "title": "A new approach for extracting the conceptual schema of texts based on the linguistic Thematic Progression theory",
            "authors": [
                "Elena del Olmo Su\u00e1rez",
                "Ana Mar\u00eda Fern\u00e1ndez-Pampill\u00f3n Cesteros"
            ],
            "abstract": "The purpose of this article is to present a new approach for the discovery\nand labelling of the implicit conceptual schema of texts through the\napplication of the Thematic Progression theory. The underlying conceptual\nschema is the core component for the generation of summaries that are genuinely\nconsistent with the semantics of the text.",
            "journal ref": "Del Olmo, E.; Fern\\'andez-Pampill\\'on, A. A new approach for\n  extracting the conceptual schema of texts based on the linguistic Thematic\n  Progression theory. Proceedings of the Workshop on Hybrid Intelligence for\n  NLP Tasks (ECAI-2020), 23-27",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07475v1",
            "date": "2020-10-15 02:35:31+00:00",
            "title": "Neural Deepfake Detection with Factual Structure of Text",
            "authors": [
                "Wanjun Zhong",
                "Duyu Tang",
                "Zenan Xu",
                "Ruize Wang",
                "Nan Duan",
                "Ming Zhou",
                "Jiahai Wang",
                "Jian Yin"
            ],
            "abstract": "Deepfake detection, the task of automatically discriminating\nmachine-generated text, is increasingly critical with recent advances in\nnatural language generative models. Existing approaches to deepfake detection\ntypically represent documents with coarse-grained representations. However,\nthey struggle to capture factual structures of documents, which is a\ndiscriminative factor between machine-generated and human-written text\naccording to our statistical analysis. To address this, we propose a\ngraph-based model that utilizes the factual structure of a document for\ndeepfake detection of text. Our approach represents the factual structure of a\ngiven document as an entity graph, which is further utilized to learn sentence\nrepresentations with a graph neural network. Sentence representations are then\ncomposed to a document representation for making predictions, where consistent\nrelations between neighboring sentences are sequentially modeled. Results of\nexperiments on two public deepfake datasets show that our approach\nsignificantly improves strong base models built with RoBERTa. Model analysis\nfurther indicates that our model can distinguish the difference in the factual\nstructure between machine-generated text and human-written text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07503v1",
            "date": "2020-10-15 04:03:00+00:00",
            "title": "Multi-Task Learning for Cross-Lingual Abstractive Summarization",
            "authors": [
                "Sho Takase",
                "Naoaki Okazaki"
            ],
            "abstract": "We present a multi-task learning framework for cross-lingual abstractive\nsummarization to augment training data. Recent studies constructed pseudo\ncross-lingual abstractive summarization data to train their neural\nencoder-decoders. Meanwhile, we introduce existing genuine data such as\ntranslation pairs and monolingual abstractive summarization data into training.\nOur proposed method, Transum, attaches a special token to the beginning of the\ninput sentence to indicate the target task. The special token enables us to\nincorporate the genuine data into the training data easily. The experimental\nresults show that Transum achieves better performance than the model trained\nwith only pseudo cross-lingual summarization data. In addition, we achieve the\ntop ROUGE score on Chinese-English and Arabic-English abstractive\nsummarization. Moreover, Transum also has a positive effect on machine\ntranslation. Experimental results indicate that Transum improves the\nperformance from the strong baseline, Transformer, in Chinese-English,\nArabic-English, and English-Japanese translation datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07515v1",
            "date": "2020-10-15 04:42:29+00:00",
            "title": "RNNs can generate bounded hierarchical languages with optimal memory",
            "authors": [
                "John Hewitt",
                "Michael Hahn",
                "Surya Ganguli",
                "Percy Liang",
                "Christopher D. Manning"
            ],
            "abstract": "Recurrent neural networks empirically generate natural language with high\nsyntactic fidelity. However, their success is not well-understood\ntheoretically. We provide theoretical insight into this success, proving in a\nfinite-precision setting that RNNs can efficiently generate bounded\nhierarchical languages that reflect the scaffolding of natural language syntax.\nWe introduce Dyck-($k$,$m$), the language of well-nested brackets (of $k$\ntypes) and $m$-bounded nesting depth, reflecting the bounded memory needs and\nlong-distance dependencies of natural language syntax. The best known results\nuse $O(k^{\\frac{m}{2}})$ memory (hidden units) to generate these languages. We\nprove that an RNN with $O(m \\log k)$ hidden units suffices, an exponential\nreduction in memory, by an explicit construction. Finally, we show that no\nalgorithm, even with unbounded computation, can suffice with $o(m \\log k)$\nhidden units.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07522v1",
            "date": "2020-10-15 04:58:23+00:00",
            "title": "Named Entity Recognition and Relation Extraction using Enhanced Table Filling by Contextualized Representations",
            "authors": [
                "Youmi Ma",
                "Tatsuya Hiraoka",
                "Naoaki Okazaki"
            ],
            "abstract": "In this study, a novel method for extracting named entities and relations\nfrom unstructured text based on the table representation is presented. By using\ncontextualized word embeddings, the proposed method computes representations\nfor entity mentions and long-range dependencies without complicated\nhand-crafted features or neural-network architectures. We also adapt a tensor\ndot-product to predict relation labels all at once without resorting to\nhistory-based predictions or search strategies. These advances significantly\nsimplify the model and algorithm for the extraction of named entities and\nrelations. Despite its simplicity, the experimental results demonstrate that\nthe proposed method outperforms the state-of-the-art methods on the CoNLL04 and\nACE05 English datasets. We also confirm that the proposed method achieves a\ncomparable performance with the state-of-the-art NER models on the ACE05\ndatasets when multiple sentences are provided for context aggregation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07543v1",
            "date": "2020-10-15 06:36:39+00:00",
            "title": "Improving Constituency Parsing with Span Attention",
            "authors": [
                "Yuanhe Tian",
                "Yan Song",
                "Fei Xia",
                "Tong Zhang"
            ],
            "abstract": "Constituency parsing is a fundamental and important task for natural language\nunderstanding, where a good representation of contextual information can help\nthis task. N-grams, which is a conventional type of feature for contextual\ninformation, have been demonstrated to be useful in many tasks, and thus could\nalso be beneficial for constituency parsing if they are appropriately modeled.\nIn this paper, we propose span attention for neural chart-based constituency\nparsing to leverage n-gram information. Considering that current chart-based\nparsers with Transformer-based encoder represent spans by subtraction of the\nhidden states at the span boundaries, which may cause information loss\nespecially for long spans, we incorporate n-grams into span representations by\nweighting them according to their contributions to the parsing process.\nMoreover, we propose categorical span attention to further enhance the model by\nweighting n-grams within different length categories, and thus benefit\nlong-sentence parsing. Experimental results on three widely used benchmark\ndatasets demonstrate the effectiveness of our approach in parsing Arabic,\nChinese, and English, where state-of-the-art performance is obtained by our\napproach on all of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07557v3",
            "date": "2020-10-15 07:11:04+00:00",
            "title": "Token Sequence Labeling vs. Clause Classification for English Emotion Stimulus Detection",
            "authors": [
                "Laura Oberl\u00e4nder",
                "Roman Klinger"
            ],
            "abstract": "Emotion stimulus detection is the task of finding the cause of an emotion in\na textual description, similar to target or aspect detection for sentiment\nanalysis. Previous work approached this in three ways, namely (1) as text\nclassification into an inventory of predefined possible stimuli (\"Is the\nstimulus category A or B?\"), (2) as sequence labeling of tokens (\"Which tokens\ndescribe the stimulus?\"), and (3) as clause classification (\"Does this clause\ncontain the emotion stimulus?\"). So far, setting (3) has been evaluated broadly\non Mandarin and (2) on English, but no comparison has been performed.\nTherefore, we aim to answer whether clause classification or sequence labeling\nis better suited for emotion stimulus detection in English. To accomplish that,\nwe propose an integrated framework which enables us to evaluate the two\ndifferent approaches comparably, implement models inspired by state-of-the-art\napproaches in Mandarin, and test them on four English data sets from different\ndomains. Our results show that sequence labeling is superior on three out of\nfour datasets, in both clause-based and sequence-based evaluation. The only\ncase in which clause classification performs better is one data set with a high\ndensity of clause annotations. Our error analysis further confirms\nquantitatively and qualitatively that clauses are not the appropriate stimulus\nunit in English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07574v1",
            "date": "2020-10-15 07:52:01+00:00",
            "title": "Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses",
            "authors": [
                "Simon Flachs",
                "Oph\u00e9lie Lacroix",
                "Helen Yannakoudakis",
                "Marek Rei",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Evaluation of grammatical error correction (GEC) systems has primarily\nfocused on essays written by non-native learners of English, which however is\nonly part of the full spectrum of GEC applications. We aim to broaden the\ntarget domain of GEC and release CWEB, a new benchmark for GEC consisting of\nwebsite text generated by English speakers of varying levels of proficiency.\nWebsite data is a common and important domain that contains far fewer\ngrammatical errors than learner essays, which we show presents a challenge to\nstate-of-the-art GEC systems. We demonstrate that a factor behind this is the\ninability of systems to rely on a strong internal language model in low error\ndensity domains. We hope this work shall facilitate the development of\nopen-domain GEC models that generalize to different topics and genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07576v1",
            "date": "2020-10-15 07:53:28+00:00",
            "title": "Pretrained Language Models for Dialogue Generation with Multiple Input Sources",
            "authors": [
                "Yu Cao",
                "Wei Bi",
                "Meng Fang",
                "Dacheng Tao"
            ],
            "abstract": "Large-scale pretrained language models have achieved outstanding performance\non natural language understanding tasks. However, it is still under\ninvestigating how to apply them to dialogue generation tasks, especially those\nwith responses conditioned on multiple sources. Previous work simply\nconcatenates all input sources or averages information from different input\nsources. In this work, we study dialogue models with multiple input sources\nadapted from the pretrained language model GPT2. We explore various methods to\nfuse multiple separate attention information corresponding to different\nsources. Our experimental results show that proper fusion methods deliver\nhigher relevance with dialogue history than simple fusion baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07606v3",
            "date": "2020-10-15 09:03:38+00:00",
            "title": "Learning Better Representation for Tables by Self-Supervised Tasks",
            "authors": [
                "Liang Li",
                "Can Ma",
                "Yinliang Yue",
                "Linjun Shou",
                "Dayong Hu"
            ],
            "abstract": "Table-to-text generation aims at automatically generating natural text to\nhelp people to conveniently obtain the important information in tables.\nAlthough neural models for table-to-text have achieved remarkable progress,\nsome problems still overlooked. The first is that the values recorded in many\ntables are mostly numbers in practice. The existing approaches do not do\nspecial treatment for these, and still regard these as words in natural\nlanguage text. Secondly, the target texts in training dataset may contain\nredundant information or facts do not exist in the input tables. These may give\nwrong supervision signals to some methods based on content selection and\nplanning and auxiliary supervision. To solve these problems, we propose two\nself-supervised tasks, Number Ordering and Significance Ordering, to help to\nlearn better table representation. The former works on the column dimension to\nhelp to incorporate the size property of numbers into table representation. The\nlatter acts on row dimension and help to learn a significance-aware table\nrepresentation. We test our methods on the widely used dataset ROTOWIRE which\nconsists of NBA game statistic and related news. The experimental results\ndemonstrate that the model trained together with these two self-supervised\ntasks can generate text that contains more salient and well-organized facts,\neven without modeling context selection and planning. And we achieve the\nstate-of-the-art performance on automatic metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07638v1",
            "date": "2020-10-15 10:11:40+00:00",
            "title": "Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses",
            "authors": [
                "Prathyusha Jwalapuram",
                "Shafiq Joty",
                "Youlin Shen"
            ],
            "abstract": "Popular Neural Machine Translation model training uses strategies like\nbacktranslation to improve BLEU scores, requiring large amounts of additional\ndata and training. We introduce a class of conditional\ngenerative-discriminative hybrid losses that we use to fine-tune a trained\nmachine translation model. Through a combination of targeted fine-tuning\nobjectives and intuitive re-use of the training data the model has failed to\nadequately learn from, we improve the model performance of both a\nsentence-level and a contextual model without using any additional data. We\ntarget the improvement of pronoun translations through our fine-tuning and\nevaluate our models on a pronoun benchmark testset. Our sentence-level model\nshows a 0.5 BLEU improvement on both the WMT14 and the IWSLT13 De-En testsets,\nwhile our contextual model achieves the best results, improving from 31.81 to\n32 BLEU on WMT14 De-En testset, and from 32.10 to 33.13 on the IWSLT13 De-En\ntestset, with corresponding improvements in pronoun translation. We further\nshow the generalizability of our method by reproducing the improvements on two\nadditional language pairs, Fr-En and Cs-En. Code available at\n<https://github.com/ntunlp/pronoun-finetuning>.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07665v1",
            "date": "2020-10-15 11:12:26+00:00",
            "title": "Diverse Keyphrase Generation with Neural Unlikelihood Training",
            "authors": [
                "Hareesh Bahuleyan",
                "Layla El Asri"
            ],
            "abstract": "In this paper, we study sequence-to-sequence (S2S) keyphrase generation\nmodels from the perspective of diversity. Recent advances in neural natural\nlanguage generation have made possible remarkable progress on the task of\nkeyphrase generation, demonstrated through improvements on quality metrics such\nas F1-score. However, the importance of diversity in keyphrase generation has\nbeen largely ignored. We first analyze the extent of information redundancy\npresent in the outputs generated by a baseline model trained using maximum\nlikelihood estimation (MLE). Our findings show that repetition of keyphrases is\na major issue with MLE training. To alleviate this issue, we adopt neural\nunlikelihood (UL) objective for training the S2S model. Our version of UL\ntraining operates at (1) the target token level to discourage the generation of\nrepeating tokens; (2) the copy token level to avoid copying repetitive tokens\nfrom the source text. Further, to encourage better model planning during the\ndecoding process, we incorporate K-step ahead token prediction objective that\ncomputes both MLE and UL losses on future tokens as well. Through extensive\nexperiments on datasets from three different domains we demonstrate that the\nproposed approach attains considerably large diversity gains, while maintaining\ncompetitive output quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07711v1",
            "date": "2020-10-15 12:40:56+00:00",
            "title": "Does Chinese BERT Encode Word Structure?",
            "authors": [
                "Yile Wang",
                "Leyang Cui",
                "Yue Zhang"
            ],
            "abstract": "Contextualized representations give significantly improved results for a wide\nrange of NLP tasks. Much work has been dedicated to analyzing the features\ncaptured by representative models such as BERT. Existing work finds that\nsyntactic, semantic and word sense knowledge are encoded in BERT. However,\nlittle work has investigated word features for character-based languages such\nas Chinese. We investigate Chinese BERT using both attention weight\ndistribution statistics and probing tasks, finding that (1) word information is\ncaptured by BERT; (2) word-level features are mostly in the middle\nrepresentation layers; (3) downstream tasks make different use of word features\nin BERT, with POS tagging and chunking relying the most on word features, and\nnatural language inference relying the least on such features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07878v1",
            "date": "2020-10-15 16:55:45+00:00",
            "title": "Tokenization Repair in the Presence of Spelling Errors",
            "authors": [
                "Hannah Bast",
                "Matthias Hertel",
                "Mostafa M. Mohamed"
            ],
            "abstract": "We consider the following tokenization repair problem: Given a natural\nlanguage text with any combination of missing or spurious spaces, correct\nthese. Spelling errors can be present, but it's not part of the problem to\ncorrect them. For example, given: \"Tispa per isabout token izaionrep air\",\ncompute \"Tis paper is about tokenizaion repair\". It is tempting to think of\nthis problem as a special case of spelling correction or to treat the two\nproblems together. We make a case that tokenization repair and spelling\ncorrection should and can be treated as separate problems. We investigate a\nvariety of neural models as well as a number of strong baselines. We identify\nthree main ingredients to high-quality tokenization repair: deep language\nmodels with a bidirectional component, training the models on text with\nspelling errors, and making use of the space information already present. Our\nbest methods can repair all tokenization errors on 97.5% of the correctly\nspelled test sentences and on 96.0% of the misspelled test sentences. With all\nspaces removed from the given text (the scenario from previous work), the\naccuracy falls to 94.5% and 90.1%, respectively. We conduct a detailed error\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07882v1",
            "date": "2020-10-15 16:57:27+00:00",
            "title": "Understanding Neural Abstractive Summarization Models via Uncertainty",
            "authors": [
                "Jiacheng Xu",
                "Shrey Desai",
                "Greg Durrett"
            ],
            "abstract": "An advantage of seq2seq abstractive summarization models is that they\ngenerate text in a free-form manner, but this flexibility makes it difficult to\ninterpret model behavior. In this work, we analyze summarization decoders in\nboth blackbox and whitebox ways by studying on the entropy, or uncertainty, of\nthe model's token-level predictions. For two strong pre-trained models, PEGASUS\nand BART on two summarization datasets, we find a strong correlation between\nlow prediction entropy and where the model copies tokens rather than generating\nnovel text. The decoder's uncertainty also connects to factors like sentence\nposition and syntactic distance between adjacent pairs of tokens, giving a\nsense of what factors make a context particularly selective for the model's\nnext output token. Finally, we study the relationship of decoder uncertainty\nand attention behavior to understand how attention gives rise to these observed\neffects in the model. We show that uncertainty is a useful perspective for\nanalyzing summarization and text generation models more broadly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07886v1",
            "date": "2020-10-15 17:07:10+00:00",
            "title": "Compressive Summarization with Plausibility and Salience Modeling",
            "authors": [
                "Shrey Desai",
                "Jiacheng Xu",
                "Greg Durrett"
            ],
            "abstract": "Compressive summarization systems typically rely on a crafted set of\nsyntactic rules to determine what spans of possible summary sentences can be\ndeleted, then learn a model of what to actually delete by optimizing for\ncontent selection (ROUGE). In this work, we propose to relax the rigid\nsyntactic constraints on candidate spans and instead leave compression\ndecisions to two data-driven criteria: plausibility and salience. Deleting a\nspan is plausible if removing it maintains the grammaticality and factuality of\na sentence, and spans are salient if they contain important information from\nthe summary. Each of these is judged by a pre-trained Transformer model, and\nonly deletions that are both plausible and not salient can be applied. When\nintegrated into a simple extraction-compression pipeline, our method achieves\nstrong in-domain results on benchmark summarization datasets, and human\nevaluation shows that the plausibility model generally selects for grammatical\nand factual deletions. Furthermore, the flexibility of our approach allows it\nto generalize cross-domain: our system fine-tuned on only 500 samples from a\nnew domain can match or exceed an in-domain extractive model trained on much\nmore data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07988v1",
            "date": "2020-10-15 19:12:52+00:00",
            "title": "CXP949 at WNUT-2020 Task 2: Extracting Informative COVID-19 Tweets -- RoBERTa Ensembles and The Continued Relevance of Handcrafted Features",
            "authors": [
                "Calum Perrio",
                "Harish Tayyar Madabushi"
            ],
            "abstract": "This paper presents our submission to Task 2 of the Workshop on Noisy\nUser-generated Text. We explore improving the performance of a pre-trained\ntransformer-based language model fine-tuned for text classification through an\nensemble implementation that makes use of corpus level information and a\nhandcrafted feature. We test the effectiveness of including the aforementioned\nfeatures in accommodating the challenges of a noisy data set centred on a\nspecific subject outside the remit of the pre-training data. We show that\ninclusion of additional features can improve classification results and achieve\na score within 2 points of the top performing team.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08014v3",
            "date": "2020-10-15 20:46:14+00:00",
            "title": "GSum: A General Framework for Guided Neural Abstractive Summarization",
            "authors": [
                "Zi-Yi Dou",
                "Pengfei Liu",
                "Hiroaki Hayashi",
                "Zhengbao Jiang",
                "Graham Neubig"
            ],
            "abstract": "Neural abstractive summarization models are flexible and can produce coherent\nsummaries, but they are sometimes unfaithful and can be difficult to control.\nWhile previous studies attempt to provide different types of guidance to\ncontrol the output and increase faithfulness, it is not clear how these\nstrategies compare and contrast to each other. In this paper, we propose a\ngeneral and extensible guided summarization framework (GSum) that can\neffectively take different kinds of external guidance as input, and we perform\nexperiments across several different varieties. Experiments demonstrate that\nthis model is effective, achieving state-of-the-art performance according to\nROUGE on 4 popular summarization datasets when using highlighted sentences as\nguidance. In addition, we show that our guided model can generate more faithful\nsummaries and demonstrate how different types of guidance generate\nqualitatively different summaries, lending a degree of controllability to the\nlearned models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08067v1",
            "date": "2020-10-15 23:25:01+00:00",
            "title": "Montague Grammar Induction",
            "authors": [
                "Gene Louis Kim",
                "Aaron Steven White"
            ],
            "abstract": "We propose a computational modeling framework for inducing combinatory\ncategorial grammars from arbitrary behavioral data. This framework provides the\nanalyst fine-grained control over the assumptions that the induced grammar\nshould conform to: (i) what the primitive types are; (ii) how complex types are\nconstructed; (iii) what set of combinators can be used to combine types; and\n(iv) whether (and to what) the types of some lexical items should be fixed. In\na proof-of-concept experiment, we deploy our framework for use in\ndistributional analysis. We focus on the relationship between\ns(emantic)-selection and c(ategory)-selection, using as input a lexicon-scale\nacceptability judgment dataset focused on English verbs' syntactic distribution\n(the MegaAcceptability dataset) and enforcing standard assumptions from the\nsemantics literature on the induced grammar.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08090v1",
            "date": "2020-10-16 01:25:01+00:00",
            "title": "Inferring symmetry in natural language",
            "authors": [
                "Chelsea Tanchip",
                "Lei Yu",
                "Aotao Xu",
                "Yang Xu"
            ],
            "abstract": "We present a methodological framework for inferring symmetry of verb\npredicates in natural language. Empirical work on predicate symmetry has taken\ntwo main approaches. The feature-based approach focuses on linguistic features\npertaining to symmetry. The context-based approach denies the existence of\nabsolute symmetry but instead argues that such inference is context dependent.\nWe develop methods that formalize these approaches and evaluate them against a\nnovel symmetry inference sentence (SIS) dataset comprised of 400 naturalistic\nusages of literature-informed verbs spanning the spectrum of\nsymmetry-asymmetry. Our results show that a hybrid transfer learning model that\nintegrates linguistic features with contextualized language models most\nfaithfully predicts the empirical data. Our work integrates existing approaches\nto symmetry in natural language and suggests how symmetry inference can improve\nsystematicity in state-of-the-art language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08197v1",
            "date": "2020-10-16 06:59:34+00:00",
            "title": "Lexicon-constrained Copying Network for Chinese Abstractive Summarization",
            "authors": [
                "Boyan Wan",
                "Zhuo Tang",
                "Li Yang"
            ],
            "abstract": "Copy mechanism allows sequence-to-sequence models to choose words from the\ninput and put them directly into the output, which is finding increasing use in\nabstractive summarization. However, since there is no explicit delimiter in\nChinese sentences, most existing models for Chinese abstractive summarization\ncan only perform character copy, resulting in inefficient. To solve this\nproblem, we propose a lexicon-constrained copying network that models\nmulti-granularity in both encoder and decoder. On the source side, words and\ncharacters are aggregated into the same input memory using a Transformerbased\nencoder. On the target side, the decoder can copy either a character or a\nmulti-character word at each time step, and the decoding process is guided by a\nword-enhanced search algorithm that facilitates the parallel computation and\nencourages the model to copy more words. Moreover, we adopt a word selector to\nintegrate keyword information. Experiments results on a Chinese social media\ndataset show that our model can work standalone or with the word selector. Both\nforms can outperform previous character-based models and achieve competitive\nperformances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08210v1",
            "date": "2020-10-16 07:39:20+00:00",
            "title": "Coarse-to-Fine Pre-training for Named Entity Recognition",
            "authors": [
                "Mengge Xue",
                "Bowen Yu",
                "Zhenyu Zhang",
                "Tingwen Liu",
                "Yue Zhang",
                "Bin Wang"
            ],
            "abstract": "More recently, Named Entity Recognition hasachieved great advances aided by\npre-trainingapproaches such as BERT. However, currentpre-training techniques\nfocus on building lan-guage modeling objectives to learn a gen-eral\nrepresentation, ignoring the named entity-related knowledge. To this end, we\nproposea NER-specific pre-training framework to in-ject coarse-to-fine\nautomatically mined entityknowledge into pre-trained models. Specifi-cally, we\nfirst warm-up the model via an en-tity span identification task by training it\nwithWikipedia anchors, which can be deemed asgeneral-typed entities. Then we\nleverage thegazetteer-based distant supervision strategy totrain the model\nextract coarse-grained typedentities. Finally, we devise a\nself-supervisedauxiliary task to mine the fine-grained namedentity knowledge\nvia clustering.Empiricalstudies on three public NER datasets demon-strate that\nour framework achieves significantimprovements against several pre-trained\nbase-lines, establishing the new state-of-the-art per-formance on three\nbenchmarks. Besides, weshow that our framework gains promising re-sults without\nusing human-labeled trainingdata, demonstrating its effectiveness in label-few\nand low-resource scenarios",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08232v1",
            "date": "2020-10-16 08:28:05+00:00",
            "title": "WNUT-2020 Task 2: Identification of Informative COVID-19 English Tweets",
            "authors": [
                "Dat Quoc Nguyen",
                "Thanh Vu",
                "Afshin Rahimi",
                "Mai Hoang Dao",
                "Linh The Nguyen",
                "Long Doan"
            ],
            "abstract": "In this paper, we provide an overview of the WNUT-2020 shared task on the\nidentification of informative COVID-19 English Tweets. We describe how we\nconstruct a corpus of 10K Tweets and organize the development and evaluation\nphases for this task. In addition, we also present a brief summary of results\nobtained from the final system evaluation submissions of 55 teams, finding that\n(i) many systems obtain very high performance, up to 0.91 F1 score, (ii) the\nmajority of the submissions achieve substantially higher results than the\nbaseline fastText (Joulin et al., 2017), and (iii) fine-tuning pre-trained\nlanguage models on relevant language data followed by supervised training\nperforms well in this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08240v2",
            "date": "2020-10-16 08:43:27+00:00",
            "title": "Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks",
            "authors": [
                "Nandan Thakur",
                "Nils Reimers",
                "Johannes Daxenberger",
                "Iryna Gurevych"
            ],
            "abstract": "There are two approaches for pairwise sentence scoring: Cross-encoders, which\nperform full-attention over the input pair, and Bi-encoders, which map each\ninput independently to a dense vector space. While cross-encoders often achieve\nhigher performance, they are too slow for many practical use cases.\nBi-encoders, on the other hand, require substantial training data and\nfine-tuning over the target task to achieve competitive performance. We present\na simple yet efficient data augmentation strategy called Augmented SBERT, where\nwe use the cross-encoder to label a larger set of input pairs to augment the\ntraining data for the bi-encoder. We show that, in this process, selecting the\nsentence pairs is non-trivial and crucial for the success of the method. We\nevaluate our approach on multiple tasks (in-domain) as well as on a domain\nadaptation task. Augmented SBERT achieves an improvement of up to 6 points for\nin-domain and of up to 37 points for domain adaptation tasks compared to the\noriginal bi-encoder performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08242v1",
            "date": "2020-10-16 08:44:09+00:00",
            "title": "Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers",
            "authors": [
                "Shusheng Xu",
                "Xingxing Zhang",
                "Yi Wu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Unsupervised extractive document summarization aims to select important\nsentences from a document without using labeled summaries during training.\nExisting methods are mostly graph-based with sentences as nodes and edge\nweights measured by sentence similarities. In this work, we find that\ntransformer attentions can be used to rank sentences for unsupervised\nextractive summarization. Specifically, we first pre-train a hierarchical\ntransformer model using unlabeled documents only. Then we propose a method to\nrank sentences using sentence-level self-attentions and pre-training\nobjectives. Experiments on CNN/DailyMail and New York Times datasets show our\nmodel achieves state-of-the-art performance on unsupervised summarization. We\nalso find in experiments that our model is less dependent on sentence\npositions. When using a linear combination of our model and a recent\nunsupervised model explicitly modeling sentence positions, we obtain even\nbetter results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08246v2",
            "date": "2020-10-16 08:47:24+00:00",
            "title": "SIGTYP 2020 Shared Task: Prediction of Typological Features",
            "authors": [
                "Johannes Bjerva",
                "Elizabeth Salesky",
                "Sabrina J. Mielke",
                "Aditi Chaudhary",
                "Giuseppe G. A. Celano",
                "Edoardo M. Ponti",
                "Ekaterina Vylomova",
                "Ryan Cotterell",
                "Isabelle Augenstein"
            ],
            "abstract": "Typological knowledge bases (KBs) such as WALS (Dryer and Haspelmath, 2013)\ncontain information about linguistic properties of the world's languages. They\nhave been shown to be useful for downstream applications, including\ncross-lingual transfer learning and linguistic probing. A major drawback\nhampering broader adoption of typological KBs is that they are sparsely\npopulated, in the sense that most languages only have annotations for some\nfeatures, and skewed, in that few features have wide coverage. As typological\nfeatures often correlate with one another, it is possible to predict them and\nthus automatically populate typological KBs, which is also the focus of this\nshared task. Overall, the task attracted 8 submissions from 5 teams, out of\nwhich the most successful methods make use of such feature correlations.\nHowever, our error analysis reveals that even the strongest submitted systems\nstruggle with predicting feature values for languages where few features are\nknown.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08275v1",
            "date": "2020-10-16 09:49:32+00:00",
            "title": "It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT",
            "authors": [
                "Hila Gonen",
                "Shauli Ravfogel",
                "Yanai Elazar",
                "Yoav Goldberg"
            ],
            "abstract": "Recent works have demonstrated that multilingual BERT (mBERT) learns rich\ncross-lingual representations, that allow for transfer across languages. We\nstudy the word-level translation information embedded in mBERT and present two\nsimple methods that expose remarkable translation capabilities with no\nfine-tuning. The results suggest that most of this information is encoded in a\nnon-linear way, while some of it can also be recovered with purely linear\ntools. As part of our analysis, we test the hypothesis that mBERT learns\nrepresentations which contain both a language-encoding component and an\nabstract, cross-lingual component, and explicitly identify an empirical\nlanguage-identity subspace within mBERT representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08318v2",
            "date": "2020-10-16 11:20:03+00:00",
            "title": "Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification",
            "authors": [
                "Andrew Moore",
                "Jeremy Barnes"
            ],
            "abstract": "The majority of work in targeted sentiment analysis has concentrated on\nfinding better methods to improve the overall results. Within this paper we\nshow that these models are not robust to linguistic phenomena, specifically\nnegation and speculation. In this paper, we propose a multi-task learning\nmethod to incorporate information from syntactic and semantic auxiliary tasks,\nincluding negation and speculation scope detection, to create English-language\nmodels that are more robust to these phenomena. Further we create two challenge\ndatasets to evaluate model performance on negated and speculative samples. We\nfind that multi-task models and transfer learning via language modelling can\nimprove performance on these challenge datasets, but the overall performances\nindicate that there is still much room for improvement. We release both the\ndatasets and the source code at\nhttps://github.com/jerbarnes/multitask_negation_for_targeted_sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08323v1",
            "date": "2020-10-16 11:32:12+00:00",
            "title": "QA2Explanation: Generating and Evaluating Explanations for Question Answering Systems over Knowledge Graph",
            "authors": [
                "Saeedeh Shekarpour",
                "Abhishek Nadgeri",
                "Kuldeep Singh"
            ],
            "abstract": "In the era of Big Knowledge Graphs, Question Answering (QA) systems have\nreached a milestone in their performance and feasibility. However, their\napplicability, particularly in specific domains such as the biomedical domain,\nhas not gained wide acceptance due to their \"black box\" nature, which hinders\ntransparency, fairness, and accountability of QA systems. Therefore, users are\nunable to understand how and why particular questions have been answered,\nwhereas some others fail. To address this challenge, in this paper, we develop\nan automatic approach for generating explanations during various stages of a\npipeline-based QA system. Our approach is a supervised and automatic approach\nwhich considers three classes (i.e., success, no answer, and wrong answer) for\nannotating the output of involved QA components. Upon our prediction, a\ntemplate explanation is chosen and integrated into the output of the\ncorresponding component. To measure the effectiveness of the approach, we\nconducted a user survey as to how non-expert users perceive our generated\nexplanations. The results of our study show a significant increase in the four\ndimensions of the human factor from the Human-computer interaction community.",
            "journal ref": "EMNLP2020 Workshop:IntEx-SemPar",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08422v1",
            "date": "2020-10-16 14:36:38+00:00",
            "title": "Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering",
            "authors": [
                "Wissam Siblini",
                "Mohamed Challal",
                "Charlotte Pasqual"
            ],
            "abstract": "Open Domain Question Answering (ODQA) on a large-scale corpus of documents\n(e.g. Wikipedia) is a key challenge in computer science. Although\ntransformer-based language models such as Bert have shown on SQuAD the ability\nto surpass humans for extracting answers in small passages of text, they suffer\nfrom their high complexity when faced to a much larger search space. The most\ncommon way to tackle this problem is to add a preliminary Information Retrieval\nstep to heavily filter the corpus and only keep the relevant passages. In this\npaper, we propose a more direct and complementary solution which consists in\napplying a generic change in the architecture of transformer-based models to\ndelay the attention between subparts of the input and allow a more efficient\nmanagement of computations. The resulting variants are competitive with the\noriginal models on the extractive task and allow, on the ODQA setting, a\nsignificant speedup and even a performance improvement in many cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08432v1",
            "date": "2020-10-16 14:54:28+00:00",
            "title": "Multi-Adversarial Learning for Cross-Lingual Word Embeddings",
            "authors": [
                "Haozhou Wang",
                "James Henderson",
                "Paola Merlo"
            ],
            "abstract": "Generative adversarial networks (GANs) have succeeded in inducing\ncross-lingual word embeddings -- maps of matching words across languages --\nwithout supervision. Despite these successes, GANs' performance for the\ndifficult case of distant languages is still not satisfactory. These\nlimitations have been explained by GANs' incorrect assumption that source and\ntarget embedding spaces are related by a single linear mapping and are\napproximately isomorphic. We assume instead that, especially across distant\nlanguages, the mapping is only piece-wise linear, and propose a\nmulti-adversarial learning method. This novel method induces the seed\ncross-lingual dictionary through multiple mappings, each induced to fit the\nmapping for one subspace. Our experiments on unsupervised bilingual lexicon\ninduction show that this method improves performance over previous\nsingle-mapping methods, especially for distant languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08540v1",
            "date": "2020-10-16 17:49:59+00:00",
            "title": "Detecting Objectifying Language in Online Professor Reviews",
            "authors": [
                "Angie Waller",
                "Kyle Gorman"
            ],
            "abstract": "Student reviews often make reference to professors' physical appearances.\nUntil recently RateMyProfessors.com, the website of this study's focus, used a\ndesign feature to encourage a \"hot or not\" rating of college professors. In the\nwake of recent #MeToo and #TimesUp movements, social awareness of the\ninappropriateness of these reviews has grown; however, objectifying comments\nremain and continue to be posted in this online context. We describe two\nsupervised text classifiers for detecting objectifying commentary in professor\nreviews. We then ensemble these classifiers and use the resulting model to\ntrack objectifying commentary at scale. We measure correlations between\nobjectifying commentary, changes to the review website interface, and teacher\ngender across a ten-year period.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08566v2",
            "date": "2020-10-16 18:02:07+00:00",
            "title": "Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models",
            "authors": [
                "Peter West",
                "Ximing Lu",
                "Ari Holtzman",
                "Chandra Bhagavatula",
                "Jena Hwang",
                "Yejin Choi"
            ],
            "abstract": "Publicly available, large pretrained LanguageModels (LMs) generate text with\nremarkable quality, but only sequentially from left to right. As a result, they\nare not immediately applicable to generation tasks that break the\nunidirectional assumption, such as paraphrasing or text-infilling,\nnecessitating task-specific supervision. In this paper, we present Reflective\nDecoding, a novel unsupervised algorithm that allows for direct application of\nunidirectional LMs to non-sequential tasks. Our 2-step approach requires no\nsupervision or even parallel corpora, only two off-the-shelf pretrained LMs in\nopposite directions: forward and backward. First, in the contextualization\nstep, we use LMs to generate ensembles of past and future contexts which\ncollectively capture the input (e.g. the source sentence for paraphrasing).\nSecond, in the reflection step, we condition on these \"context ensembles\",\ngenerating outputs that are compatible with them. Comprehensive empirical\nresults demonstrate that Reflective Decoding outperforms strong unsupervised\nbaselines on both paraphrasing and abductive text infilling, significantly\nnarrowing the gap between unsupervised and supervised methods. Reflective\nDecoding surpasses multiple supervised baselines on various metrics including\nhuman evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08570v1",
            "date": "2020-10-16 18:10:47+00:00",
            "title": "Generating Fact Checking Summaries for Web Claims",
            "authors": [
                "Rahul Mishra",
                "Dhruv Gupta",
                "Markus Leippold"
            ],
            "abstract": "We present SUMO, a neural attention-based approach that learns to establish\nthe correctness of textual claims based on evidence in the form of text\ndocuments (e.g., news articles or Web documents). SUMO further generates an\nextractive summary by presenting a diversified set of sentences from the\ndocuments that explain its decision on the correctness of the textual claim.\nPrior approaches to address the problem of fact checking and evidence\nextraction have relied on simple concatenation of claim and document word\nembeddings as an input to claim driven attention weight computation. This is\ndone so as to extract salient words and sentences from the documents that help\nestablish the correctness of the claim. However, this design of claim-driven\nattention does not capture the contextual information in documents properly. We\nimprove on the prior art by using improved claim and title guided hierarchical\nattention to model effective contextual cues. We show the efficacy of our\napproach on datasets concerning political, healthcare, and environmental\nissues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08580v3",
            "date": "2020-10-16 18:23:05+00:00",
            "title": "Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets",
            "authors": [
                "Chuanrong Li",
                "Lin Shengshuo",
                "Leo Z. Liu",
                "Xinyi Wu",
                "Xuhui Zhou",
                "Shane Steinert-Threlkeld"
            ],
            "abstract": "Although large-scale pretrained language models, such as BERT and RoBERTa,\nhave achieved superhuman performance on in-distribution test sets, their\nperformance suffers on out-of-distribution test sets (e.g., on contrast sets).\nBuilding contrast sets often re-quires human-expert annotation, which is\nexpensive and hard to create on a large scale. In this work, we propose a\nLinguistically-Informed Transformation (LIT) method to automatically generate\ncontrast sets, which enables practitioners to explore linguistic phenomena of\ninterests as well as compose different phenomena. Experimenting with our method\non SNLI and MNLI shows that current pretrained language models, although being\nclaimed to contain sufficient linguistic knowledge, struggle on our\nautomatically generated contrast sets. Furthermore, we improve models'\nperformance on the contrast sets by apply-ing LIT to augment the training data,\nwithout affecting performance on the original data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08618v1",
            "date": "2020-10-16 20:26:10+00:00",
            "title": "Substance over Style: Document-Level Targeted Content Transfer",
            "authors": [
                "Allison Hegel",
                "Sudha Rao",
                "Asli Celikyilmaz",
                "Bill Dolan"
            ],
            "abstract": "Existing language models excel at writing from scratch, but many real-world\nscenarios require rewriting an existing document to fit a set of constraints.\nAlthough sentence-level rewriting has been fairly well-studied, little work has\naddressed the challenge of rewriting an entire document coherently. In this\nwork, we introduce the task of document-level targeted content transfer and\naddress it in the recipe domain, with a recipe as the document and a dietary\nrestriction (such as vegan or dairy-free) as the targeted constraint. We\npropose a novel model for this task based on the generative pre-trained\nlanguage model (GPT-2) and train on a large number of roughly-aligned recipe\npairs (https://github.com/microsoft/document-level-targeted-content-transfer).\nBoth automatic and human evaluations show that our model out-performs existing\nmethods by generating coherent and diverse rewrites that obey the constraint\nwhile remaining close to the original document. Finally, we analyze our model's\nrewrites to assess progress toward the goal of making language generation more\nattuned to constraints that are substantive rather than stylistic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08642v1",
            "date": "2020-10-16 21:49:20+00:00",
            "title": "Multimodal Speech Recognition with Unstructured Audio Masking",
            "authors": [
                "Tejas Srinivasan",
                "Ramon Sanabria",
                "Florian Metze",
                "Desmond Elliott"
            ],
            "abstract": "Visual context has been shown to be useful for automatic speech recognition\n(ASR) systems when the speech signal is noisy or corrupted. Previous work,\nhowever, has only demonstrated the utility of visual context in an unrealistic\nsetting, where a fixed set of words are systematically masked in the audio. In\nthis paper, we simulate a more realistic masking scenario during model\ntraining, called RandWordMask, where the masking can occur for any word\nsegment. Our experiments on the Flickr 8K Audio Captions Corpus show that\nmultimodal ASR can generalize to recover different types of masked words in\nthis unstructured masking setting. Moreover, our analysis shows that our models\nare capable of attending to the visual signal when the audio signal is\ncorrupted. These results show that multimodal ASR systems can leverage the\nvisual signal in more generalized noisy scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08670v1",
            "date": "2020-10-16 23:57:03+00:00",
            "title": "CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding",
            "authors": [
                "Yanru Qu",
                "Dinghan Shen",
                "Yelong Shen",
                "Sandra Sajeev",
                "Jiawei Han",
                "Weizhu Chen"
            ],
            "abstract": "Data augmentation has been demonstrated as an effective strategy for\nimproving model generalization and data efficiency. However, due to the\ndiscrete nature of natural language, designing label-preserving transformations\nfor text data tends to be more challenging. In this paper, we propose a novel\ndata augmentation framework dubbed CoDA, which synthesizes diverse and\ninformative augmented examples by integrating multiple transformations\norganically. Moreover, a contrastive regularization objective is introduced to\ncapture the global relationship among all the data samples. A momentum encoder\nalong with a memory bank is further leveraged to better estimate the\ncontrastive loss. To verify the effectiveness of the proposed framework, we\napply CoDA to Transformer-based models on a wide range of natural language\nunderstanding tasks. On the GLUE benchmark, CoDA gives rise to an average\nimprovement of 2.2% while applied to the RoBERTa-large model. More importantly,\nit consistently exhibits stronger results relative to several competitive data\naugmentation and adversarial training base-lines (including the low-resource\nsettings). Extensive experiments show that the proposed contrastive objective\ncan be flexibly combined with various data augmentation approaches to further\nboost their performance, highlighting the wide applicability of the CoDA\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08728v2",
            "date": "2020-10-17 06:47:58+00:00",
            "title": "Incorporate Semantic Structures into Machine Translation Evaluation via UCCA",
            "authors": [
                "Jin Xu",
                "Yinuo Guo",
                "Junfeng Hu"
            ],
            "abstract": "Copying mechanism has been commonly used in neural paraphrasing networks and\nother text generation tasks, in which some important words in the input\nsequence are preserved in the output sequence. Similarly, in machine\ntranslation, we notice that there are certain words or phrases appearing in all\ngood translations of one source text, and these words tend to convey important\nsemantic information. Therefore, in this work, we define words carrying\nimportant semantic meanings in sentences as semantic core words. Moreover, we\npropose an MT evaluation approach named Semantically Weighted Sentence\nSimilarity (SWSS). It leverages the power of UCCA to identify semantic core\nwords, and then calculates sentence similarity scores on the overlap of\nsemantic core words. Experimental results show that SWSS can consistently\nimprove the performance of popular MT evaluation metrics which are based on\nlexical similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08738v1",
            "date": "2020-10-17 08:18:59+00:00",
            "title": "RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling",
            "authors": [
                "Jun Quan",
                "Shian Zhang",
                "Qian Cao",
                "Zizhong Li",
                "Deyi Xiong"
            ],
            "abstract": "In order to alleviate the shortage of multi-domain data and to capture\ndiscourse phenomena for task-oriented dialogue modeling, we propose RiSAWOZ, a\nlarge-scale multi-domain Chinese Wizard-of-Oz dataset with Rich Semantic\nAnnotations. RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn\nsemantically annotated dialogues, with more than 150K utterances spanning over\n12 domains, which is larger than all previous annotated H2H conversational\ndatasets. Both single- and multi-domain dialogues are constructed, accounting\nfor 65% and 35%, respectively. Each dialogue is labeled with comprehensive\ndialogue annotations, including dialogue goal in the form of natural language\ndescription, domain, dialogue states and acts at both the user and system side.\nIn addition to traditional dialogue annotations, we especially provide\nlinguistic annotations on discourse phenomena, e.g., ellipsis and coreference,\nin dialogues, which are useful for dialogue coreference and ellipsis resolution\ntasks. Apart from the fully annotated dataset, we also present a detailed\ndescription of the data collection procedure, statistics and analysis of the\ndataset. A series of benchmark models and results are reported, including\nnatural language understanding (intent detection & slot filling), dialogue\nstate tracking and dialogue context-to-text generation, as well as coreference\nand ellipsis resolution, which facilitate the baseline comparison for future\nresearch on this corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08756v1",
            "date": "2020-10-17 10:11:41+00:00",
            "title": "CUSATNLP@HASOC-Dravidian-CodeMix-FIRE2020:Identifying Offensive Language from ManglishTweets",
            "authors": [
                "Sara Renjit",
                "Sumam Mary Idicula"
            ],
            "abstract": "With the popularity of social media, communications through blogs, Facebook,\nTwitter, and other plat-forms have increased. Initially, English was the only\nmedium of communication. Fortunately, now we can communicate in any language.\nIt has led to people using English and their own native or mother tongue\nlanguage in a mixed form. Sometimes, comments in other languages have English\ntransliterated format or other cases; people use the intended language scripts.\nIdentifying sentiments and offensive content from such code mixed tweets is a\nnecessary task in these times. We present a working model submitted for Task2\nof the sub-track HASOC Offensive Language Identification- DravidianCodeMix in\nForum for Information Retrieval Evaluation, 2020. It is a message level\nclassification task. An embedding model-based classifier identifies offensive\nand not offensive comments in our approach. We applied this method in the\nManglish dataset provided along with the sub-track.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08822v1",
            "date": "2020-10-17 16:40:37+00:00",
            "title": "Consistency and Coherency Enhanced Story Generation",
            "authors": [
                "Wei Wang",
                "Piji Li",
                "Hai-Tao Zheng"
            ],
            "abstract": "Story generation is a challenging task, which demands to maintain consistency\nof the plots and characters throughout the story. Previous works have shown\nthat GPT2, a large-scale language model, has achieved good performance on story\ngeneration. However, we observe that several serious issues still exist in the\nstories generated by GPT2 which can be categorized into two folds: consistency\nand coherency. In terms of consistency, on one hand, GPT2 cannot guarantee the\nconsistency of the plots explicitly. On the other hand, the generated stories\nusually contain coreference errors. In terms of coherency, GPT2 does not take\naccount of the discourse relations between sentences of stories directly. To\nenhance the consistency and coherency of the generated stories, we propose a\ntwo-stage generation framework, where the first stage is to organize the story\noutline which depicts the story plots and events, and the second stage is to\nexpand the outline into a complete story. Therefore the plots consistency can\nbe controlled and guaranteed explicitly. In addition, coreference supervision\nsignals are incorporated to reduce coreference errors and improve the\ncoreference consistency. Moreover, we design an auxiliary task of discourse\nrelation modeling to improve the coherency of the generated stories.\nExperimental results on a story dataset show that our model outperforms the\nbaseline approaches in terms of both automatic metrics and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08824v1",
            "date": "2020-10-17 16:49:43+00:00",
            "title": "Knowledge-Grounded Dialogue Generation with Pre-trained Language Models",
            "authors": [
                "Xueliang Zhao",
                "Wei Wu",
                "Can Xu",
                "Chongyang Tao",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "We study knowledge-grounded dialogue generation with pre-trained language\nmodels. To leverage the redundant external knowledge under capacity constraint,\nwe propose equipping response generation defined by a pre-trained language\nmodel with a knowledge selection module, and an unsupervised approach to\njointly optimizing knowledge selection and response generation with unlabeled\ndialogues. Empirical results on two benchmarks indicate that our model can\nsignificantly outperform state-of-the-art methods in both automatic evaluation\nand human judgment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08892v1",
            "date": "2020-10-18 00:21:53+00:00",
            "title": "Mixed-Lingual Pre-training for Cross-lingual Summarization",
            "authors": [
                "Ruochen Xu",
                "Chenguang Zhu",
                "Yu Shi",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "Cross-lingual Summarization (CLS) aims at producing a summary in the target\nlanguage for an article in the source language. Traditional solutions employ a\ntwo-step approach, i.e. translate then summarize or summarize then translate.\nRecently, end-to-end models have achieved better results, but these approaches\nare mostly limited by their dependence on large-scale labeled data. We propose\na solution based on mixed-lingual pre-training that leverages both\ncross-lingual tasks such as translation and monolingual tasks like masked\nlanguage models. Thus, our model can leverage the massive monolingual data to\nenhance its modeling of language. Moreover, the architecture has no\ntask-specific components, which saves memory and increases optimization\nefficiency. We show in experiments that this pre-training scheme can\neffectively boost the performance of cross-lingual summarization. In Neural\nCross-Lingual Summarization (NCLS) dataset, our model achieves an improvement\nof 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over\nstate-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08961v1",
            "date": "2020-10-18 11:18:29+00:00",
            "title": "Capturing Longer Context for Document-level Neural Machine Translation: A Multi-resolutional Approach",
            "authors": [
                "Zewei Sun",
                "Mingxuan Wang",
                "Hao Zhou",
                "Chengqi Zhao",
                "Shujian Huang",
                "Jiajun Chen",
                "Lei Li"
            ],
            "abstract": "Discourse context has been proven useful when translating documents. It is\nquite a challenge to incorporate long document context in the prevailing neural\nmachine translation models such as Transformer. In this paper, we propose\nmulti-resolutional (MR) Doc2Doc, a method to train a neural\nsequence-to-sequence model for document-level translation. Our trained model\ncan simultaneously translate sentence by sentence as well as a document as a\nwhole. We evaluate our method and several recent approaches on nine\ndocument-level datasets and two sentence-level datasets across six languages.\nExperiments show that MR Doc2Doc outperforms sentence-level models and previous\nmethods in a comprehensive set of metrics, including BLEU, four lexical\nindices, three newly proposed assistant linguistic indicators, and human\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08974v1",
            "date": "2020-10-18 12:21:37+00:00",
            "title": "hinglishNorm -- A Corpus of Hindi-English Code Mixed Sentences for Text Normalization",
            "authors": [
                "Piyush Makhija",
                "Ankit Kumar",
                "Anuj Gupta"
            ],
            "abstract": "We present hinglishNorm -- a human annotated corpus of Hindi-English\ncode-mixed sentences for text normalization task. Each sentence in the corpus\nis aligned to its corresponding human annotated normalized form. To the best of\nour knowledge, there is no corpus of Hindi-English code-mixed sentences for\ntext normalization task that is publicly available. Our work is the first\nattempt in this direction. The corpus contains 13494 parallel segments.\nFurther, we present baseline normalization results on this corpus. We obtain a\nWord Error Rate (WER) of 15.55, BiLingual Evaluation Understudy (BLEU) score of\n71.2, and Metric for Evaluation of Translation with Explicit ORdering (METEOR)\nscore of 0.50.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08980v1",
            "date": "2020-10-18 13:17:09+00:00",
            "title": "Querent Intent in Multi-Sentence Questions",
            "authors": [
                "Laurie Burchell",
                "Jie Chi",
                "Tom Hosking",
                "Nina Markl",
                "Bonnie Webber"
            ],
            "abstract": "Multi-sentence questions (MSQs) are sequences of questions connected by\nrelations which, unlike sequences of standalone questions, need to be answered\nas a unit. Following Rhetorical Structure Theory (RST), we recognise that\ndifferent \"question discourse relations\" between the subparts of MSQs reflect\ndifferent speaker intents, and consequently elicit different answering\nstrategies. Correctly identifying these relations is therefore a crucial step\nin automatically answering MSQs. We identify five different types of MSQs in\nEnglish, and define five novel relations to describe them. We extract over\n162,000 MSQs from Stack Exchange to enable future research. Finally, we\nimplement a high-precision baseline classifier based on surface features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09072v1",
            "date": "2020-10-18 19:27:06+00:00",
            "title": "UoB at SemEval-2020 Task 1: Automatic Identification of Novel Word Senses",
            "authors": [
                "Eleri Sarsfield",
                "Harish Tayyar Madabushi"
            ],
            "abstract": "Much as the social landscape in which languages are spoken shifts, language\ntoo evolves to suit the needs of its users. Lexical semantic change analysis is\na burgeoning field of semantic analysis which aims to trace changes in the\nmeanings of words over time. This paper presents an approach to lexical\nsemantic change detection based on Bayesian word sense induction suitable for\nnovel word sense identification. This approach is used for a submission to\nSemEval-2020 Task 1, which shows the approach to be capable of the SemEval\ntask. The same approach is also applied to a corpus gleaned from 15 years of\nTwitter data, the results of which are then used to identify words which may be\ninstances of slang.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09078v1",
            "date": "2020-10-18 19:37:24+00:00",
            "title": "Incorporating Count-Based Features into Pre-Trained Models for Improved Stance Detection",
            "authors": [
                "Anushka Prakash",
                "Harish Tayyar Madabushi"
            ],
            "abstract": "The explosive growth and popularity of Social Media has revolutionised the\nway we communicate and collaborate. Unfortunately, this same ease of accessing\nand sharing information has led to an explosion of misinformation and\npropaganda. Given that stance detection can significantly aid in veracity\nprediction, this work focuses on boosting automated stance detection, a task on\nwhich pre-trained models have been extremely successful on, as on several other\ntasks. This work shows that the task of stance detection can benefit from\nfeature based information, especially on certain under performing classes,\nhowever, integrating such features into pre-trained models using ensembling is\nchallenging. We propose a novel architecture for integrating features with\npre-trained models that address these challenges and test our method on the\nRumourEval 2019 dataset. This method achieves state-of-the-art results with an\nF1-score of 63.94 on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09190v1",
            "date": "2020-10-19 03:29:21+00:00",
            "title": "SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline",
            "authors": [
                "Jiaxin Ju",
                "Ming Liu",
                "Longxiang Gao",
                "Shirui Pan"
            ],
            "abstract": "The Scholarly Document Processing (SDP) workshop is to encourage more efforts\non natural language understanding of scientific task. It contains three shared\ntasks and we participate in the LongSumm shared task. In this paper, we\ndescribe our text summarization system, SciSummPip, inspired by SummPip (Zhao\net al., 2020) that is an unsupervised text summarization system for\nmulti-document in news domain. Our SciSummPip includes a transformer-based\nlanguage model SciBERT (Beltagy et al., 2019) for contextual sentence\nrepresentation, content selection with PageRank (Page et al., 1999), sentence\ngraph construction with both deep and linguistic information, sentence graph\nclustering and within-graph summary generation. Our work differs from previous\nmethod in that content selection and a summary length constraint is applied to\nadapt to the scientific domain. The experiment results on both training dataset\nand blind test dataset show the effectiveness of our method, and we empirically\nverify the robustness of modules used in SciSummPip with BERTScore (Zhang et\nal., 2019a).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09194v2",
            "date": "2020-10-19 03:38:56+00:00",
            "title": "Infusing Sequential Information into Conditional Masked Translation Model with Self-Review Mechanism",
            "authors": [
                "Pan Xie",
                "Zhi Cui",
                "Xiuyin Chen",
                "Xiaohui Hu",
                "Jianwei Cui",
                "Bin Wang"
            ],
            "abstract": "Non-autoregressive models generate target words in a parallel way, which\nachieve a faster decoding speed but at the sacrifice of translation accuracy.\nTo remedy a flawed translation by non-autoregressive models, a promising\napproach is to train a conditional masked translation model (CMTM), and refine\nthe generated results within several iterations. Unfortunately, such approach\nhardly considers the \\textit{sequential dependency} among target words, which\ninevitably results in a translation degradation. Hence, instead of solely\ntraining a Transformer-based CMTM, we propose a Self-Review Mechanism to infuse\nsequential information into it. Concretely, we insert a left-to-right mask to\nthe same decoder of CMTM, and then induce it to autoregressively review whether\neach generated word from CMTM is supposed to be replaced or kept. The\nexperimental results (WMT14 En$\\leftrightarrow$De and WMT16\nEn$\\leftrightarrow$Ro) demonstrate that our model uses dramatically less\ntraining computations than the typical CMTM, as well as outperforms several\nstate-of-the-art non-autoregressive models by over 1 BLEU. Through knowledge\ndistillation, our model even surpasses a typical left-to-right Transformer\nmodel, while significantly speeding up decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09233v2",
            "date": "2020-10-19 05:57:11+00:00",
            "title": "Auto-Encoding Variational Bayes for Inferring Topics and Visualization",
            "authors": [
                "Dang Pham",
                "Tuan M. V. Le"
            ],
            "abstract": "Visualization and topic modeling are widely used approaches for text\nanalysis. Traditional visualization methods find low-dimensional\nrepresentations of documents in the visualization space (typically 2D or 3D)\nthat can be displayed using a scatterplot. In contrast, topic modeling aims to\ndiscover topics from text, but for visualization, one needs to perform a\npost-hoc embedding using dimensionality reduction methods. Recent approaches\npropose using a generative model to jointly find topics and visualization,\nallowing the semantics to be infused in the visualization space for a\nmeaningful interpretation. A major challenge that prevents these methods from\nbeing used practically is the scalability of their inference algorithms. We\npresent, to the best of our knowledge, the first fast Auto-Encoding Variational\nBayes based inference method for jointly inferring topics and visualization.\nSince our method is black box, it can handle model changes efficiently with\nlittle mathematical rederivation effort. We demonstrate the efficiency and\neffectiveness of our method on real-world large datasets and compare it with\nexisting baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09252v1",
            "date": "2020-10-19 06:36:11+00:00",
            "title": "Dimsum @LaySumm 20: BART-based Approach for Scientific Document Summarization",
            "authors": [
                "Tiezheng Yu",
                "Dan Su",
                "Wenliang Dai",
                "Pascale Fung"
            ],
            "abstract": "Lay summarization aims to generate lay summaries of scientific papers\nautomatically. It is an essential task that can increase the relevance of\nscience for all of society. In this paper, we build a lay summary generation\nsystem based on the BART model. We leverage sentence labels as extra\nsupervision signals to improve the performance of lay summarization. In the\nCL-LaySumm 2020 shared task, our model achieves 46.00\\% Rouge1-F1 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09381v1",
            "date": "2020-10-19 11:08:16+00:00",
            "title": "The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification",
            "authors": [
                "Abdullatif K\u00f6ksal",
                "Arzucan \u00d6zg\u00fcr"
            ],
            "abstract": "Relation classification is one of the key topics in information extraction,\nwhich can be used to construct knowledge bases or to provide useful information\nfor question answering. Current approaches for relation classification are\nmainly focused on the English language and require lots of training data with\nhuman annotations. Creating and annotating a large amount of training data for\nlow-resource languages is impractical and expensive. To overcome this issue, we\npropose two cross-lingual relation classification models: a baseline model\nbased on Multilingual BERT and a new multilingual pretraining setup, which\nsignificantly improves the baseline with distant supervision. For evaluation,\nwe introduce a new public benchmark dataset for cross-lingual relation\nclassification in English, French, German, Spanish, and Turkish, called RELX.\nWe also provide the RELX-Distant dataset, which includes hundreds of thousands\nof sentences with relations from Wikipedia and Wikidata collected by distant\nsupervision for these languages. Our code and data are available at:\nhttps://github.com/boun-tabi/RELX",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09403v1",
            "date": "2020-10-19 11:51:45+00:00",
            "title": "Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation",
            "authors": [
                "Du\u0161an Vari\u0161",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "This work presents our ongoing research of unsupervised pretraining in neural\nmachine translation (NMT). In our method, we initialize the weights of the\nencoder and decoder with two language models that are trained with monolingual\ndata and then fine-tune the model on parallel data using Elastic Weight\nConsolidation (EWC) to avoid forgetting of the original language modeling\ntasks. We compare the regularization by EWC with the previous work that focuses\non regularization by language modeling objectives. The positive result is that\nusing EWC with the decoder achieves BLEU scores similar to the previous work.\nHowever, the model converges 2-3 times faster and does not require the original\nunlabeled training data during the fine-tuning stage. In contrast, the\nregularization using EWC is less effective if the original and new tasks are\nnot closely related. We show that initializing the bidirectional NMT encoder\nwith a left-to-right language model and forcing the model to remember the\noriginal left-to-right language modeling task limits the learning capacity of\nthe encoder for the whole bidirectional context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09517v1",
            "date": "2020-10-19 13:51:40+00:00",
            "title": "Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads",
            "authors": [
                "Bowen Li",
                "Taeuk Kim",
                "Reinald Kim Amplayo",
                "Frank Keller"
            ],
            "abstract": "Transformer-based pre-trained language models (PLMs) have dramatically\nimproved the state of the art in NLP across many tasks. This has led to\nsubstantial interest in analyzing the syntactic knowledge PLMs learn. Previous\napproaches to this question have been limited, mostly using test suites or\nprobes. Here, we propose a novel fully unsupervised parsing approach that\nextracts constituency trees from PLM attention heads. We rank transformer\nattention heads based on their inherent properties, and create an ensemble of\nhigh-ranking heads to produce the final tree. Our method is adaptable to\nlow-resource languages, as it does not rely on development sets, which can be\nexpensive to annotate. Our experiments show that the proposed method often\noutperform existing approaches if there is no development set present. Our\nunsupervised parser can also be used as a tool to analyze the grammars PLMs\nlearn implicitly. For this, we use the parse trees induced by our method to\ntrain a neural PCFG and compare it to a grammar derived from a human-annotated\ntreebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09598v1",
            "date": "2020-10-19 15:23:24+00:00",
            "title": "Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering",
            "authors": [
                "Jeroen Offerijns",
                "Suzan Verberne",
                "Tessa Verhoef"
            ],
            "abstract": "For the field of education, being able to generate semantically correct and\neducationally relevant multiple choice questions (MCQs) could have a large\nimpact. While question generation itself is an active research topic,\ngenerating distractors (the incorrect multiple choice options) receives much\nless attention. A missed opportunity, since there is still a lot of room for\nimprovement in this area. In this work, we train a GPT-2 language model to\ngenerate three distractors for a given question and text context, using the\nRACE dataset. Next, we train a BERT language model to answer MCQs, and use this\nmodel as a filter, to select only questions that can be answered and therefore\npresumably make sense. To evaluate our work, we start by using text generation\nmetrics, which show that our model outperforms earlier work on distractor\ngeneration (DG) and achieves state-of-the-art performance. Also, by calculating\nthe question answering ability, we show that larger base models lead to better\nperformance. Moreover, we conducted a human evaluation study, which confirmed\nthe quality of the generated questions, but showed no statistically significant\neffect of the QA filter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09608v1",
            "date": "2020-10-19 15:44:03+00:00",
            "title": "Incorporating Terminology Constraints in Automatic Post-Editing",
            "authors": [
                "David Wan",
                "Chris Kedzie",
                "Faisal Ladhak",
                "Marine Carpuat",
                "Kathleen McKeown"
            ],
            "abstract": "Users of machine translation (MT) may want to ensure the use of specific\nlexical terminologies. While there exist techniques for incorporating\nterminology constraints during inference for MT, current APE approaches cannot\nensure that they will appear in the final translation. In this paper, we\npresent both autoregressive and non-autoregressive models for lexically\nconstrained APE, demonstrating that our approach enables preservation of 95% of\nthe terminologies and also improves translation quality on English-German\nbenchmarks. Even when applied to lexically constrained MT output, our approach\nis able to improve preservation of the terminologies. However, we show that our\nmodels do not learn to copy constraints systematically and suggest a simple\ndata augmentation technique that leads to improved performance and robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09623v2",
            "date": "2020-10-19 16:02:00+00:00",
            "title": "An Empirical Study for Vietnamese Constituency Parsing with Pre-training",
            "authors": [
                "Tuan-Vi Tran",
                "Xuan-Thien Pham",
                "Duc-Vu Nguyen",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen"
            ],
            "abstract": "In this work, we use a span-based approach for Vietnamese constituency\nparsing. Our method follows the self-attention encoder architecture and a chart\ndecoder using a CKY-style inference algorithm. We present analyses of the\nexperiment results of the comparison of our empirical method using pre-training\nmodels XLM-Roberta and PhoBERT on both Vietnamese datasets VietTreebank and\nNIIVTB1. The results show that our model with XLM-Roberta archived the\nsignificantly F1-score better than other pre-training models, VietTreebank at\n81.19% and NIIVTB1 at 85.70%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09638v1",
            "date": "2020-10-19 16:27:48+00:00",
            "title": "Adaptive Attentional Network for Few-Shot Knowledge Graph Completion",
            "authors": [
                "Jiawei Sheng",
                "Shu Guo",
                "Zhenyu Chen",
                "Juwei Yue",
                "Lihong Wang",
                "Tingwen Liu",
                "Hongbo Xu"
            ],
            "abstract": "Few-shot Knowledge Graph (KG) completion is a focus of current research,\nwhere each task aims at querying unseen facts of a relation given its few-shot\nreference entity pairs. Recent attempts solve this problem by learning static\nrepresentations of entities and references, ignoring their dynamic properties,\ni.e., entities may exhibit diverse roles within task relations, and references\nmay make different contributions to queries. This work proposes an adaptive\nattentional network for few-shot KG completion by learning adaptive entity and\nreference representations. Specifically, entities are modeled by an adaptive\nneighbor encoder to discern their task-oriented roles, while references are\nmodeled by an adaptive query-aware aggregator to differentiate their\ncontributions. Through the attention mechanism, both entities and references\ncan capture their fine-grained semantic meanings, and thus render more\nexpressive representations. This will be more predictive for knowledge\nacquisition in the few-shot scenario. Evaluation in link prediction on two\npublic datasets shows that our approach achieves new state-of-the-art results\nwith different few-shot sizes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09692v1",
            "date": "2020-10-19 17:30:08+00:00",
            "title": "Question Generation for Supporting Informational Query Intents",
            "authors": [
                "Xusen Yin",
                "Jonathan May",
                "Li Zhou",
                "Kevin Small"
            ],
            "abstract": "Users frequently ask simple factoid questions when encountering question\nanswering (QA) systems, attenuating the impact of myriad recent works designed\nto support more complex questions. Prompting users with automatically generated\nsuggested questions (SQs) can improve understanding of QA system capabilities\nand thus facilitate using this technology more effectively. While question\ngeneration (QG) is a well-established problem, existing methods are not\ntargeted at producing SQ guidance for human users seeking more in-depth\ninformation about a specific concept. In particular, existing QG works are\ninsufficient for this task as the generated questions frequently (1) require\naccess to supporting documents as comprehension context (e.g., How many points\ndid LeBron score?) and (2) focus on short answer spans, often producing\nperipheral factoid questions unlikely to attract interest. In this work, we aim\nto generate self-explanatory questions that focus on the main document topics\nand are answerable with variable length passages as appropriate. We satisfy\nthese requirements by using a BERT-based Pointer-Generator Network (BertPGN)\ntrained on the Natural Questions (NQ) dataset. First, we show that the BertPGN\nmodel produces state-of-the-art QG performance for long and short answers for\nin-domain NQ (BLEU-4 for 20.13 and 28.09, respectively). Secondly, we evaluate\nthis QG model on the out-of-domain NewsQA dataset automatically and with human\nevaluation, demonstrating that our method produces better SQs for news\narticles, even those from a different domain than the training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09693v1",
            "date": "2020-10-19 17:32:40+00:00",
            "title": "Subtitles to Segmentation: Improving Low-Resource Speech-to-Text Translation Pipelines",
            "authors": [
                "David Wan",
                "Zhengping Jiang",
                "Chris Kedzie",
                "Elsbeth Turcan",
                "Peter Bell",
                "Kathleen McKeown"
            ],
            "abstract": "In this work, we focus on improving ASR output segmentation in the context of\nlow-resource language speech-to-text translation. ASR output segmentation is\ncrucial, as ASR systems segment the input audio using purely acoustic\ninformation and are not guaranteed to output sentence-like segments. Since most\nMT systems expect sentences as input, feeding in longer unsegmented passages\ncan lead to sub-optimal performance. We explore the feasibility of using\ndatasets of subtitles from TV shows and movies to train better ASR segmentation\nmodels. We further incorporate part-of-speech (POS) tag and dependency label\ninformation (derived from the unsegmented ASR outputs) into our segmentation\nmodel. We show that this noisy syntactic information can improve model\naccuracy. We evaluate our models intrinsically on segmentation quality and\nextrinsically on downstream MT performance, as well as downstream tasks\nincluding cross-lingual information retrieval (CLIR) tasks and human relevance\nassessments. Our model shows improved performance on downstream tasks for\nLithuanian and Bulgarian.",
            "journal ref": "CLSST@LREC 2020 68-73",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09828v1",
            "date": "2020-10-19 20:08:26+00:00",
            "title": "Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking",
            "authors": [
                "Elliot Schumacher",
                "James Mayfield",
                "Mark Dredze"
            ],
            "abstract": "Cross-language entity linking grounds mentions in multiple languages to a\nsingle-language knowledge base. We propose a neural ranking architecture for\nthis task that uses multilingual BERT representations of the mention and the\ncontext in a neural network. We find that the multilingual ability of BERT\nleads to robust performance in monolingual and multilingual settings.\nFurthermore, we explore zero-shot language transfer and find surprisingly\nrobust performance. We investigate the zero-shot degradation and find that it\ncan be partially mitigated by a proposed auxiliary training objective, but that\nthe remaining error can best be attributed to domain shift rather than language\ntransfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09905v2",
            "date": "2020-10-19 22:45:27+00:00",
            "title": "SmartTriage: A system for personalized patient data capture, documentation generation, and decision support",
            "authors": [
                "Ilya Valmianski",
                "Ian M. Finn",
                "Nave Frost",
                "Yang Wang",
                "Baodong Liu",
                "James J. Zhu",
                "Sunil Karumuri",
                "Daniel S. Zisook"
            ],
            "abstract": "Symptom checkers have emerged as an important tool for collecting symptoms\nand diagnosing patients, minimizing the involvement of clinical personnel. We\ndeveloped a machine-learning-backed system, SmartTriage, which goes beyond\nconventional symptom checking through a tight bi-directional integration with\nthe electronic medical record (EMR). Conditioned on EMR-derived patient\nhistory, our system identifies the patient's chief complaint from a free-text\nentry and then asks a series of discrete questions to obtain relevant\nsymptomatology. The patient-specific data are used to predict detailed\nICD-10-CM codes as well as medication, laboratory, and imaging orders. Patient\nresponses and clinical decision support (CDS) predictions are then inserted\nback into the EMR. To train the machine learning components of SmartTriage, we\nemployed novel data sets of over 25 million primary care encounters and 1\nmillion patient free-text reason-for-visit entries. These data sets were used\nto construct: (1) a long short-term memory (LSTM) based patient history\nrepresentation, (2) a fine-tuned transformer model for chief complaint\nextraction, (3) a random forest model for question sequencing, and (4) a\nfeed-forward network for CDS predictions. We also present the full production\narchitecture for the pilot deployment of SmartTriage that covers 337 patient\nchief complaints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09935v1",
            "date": "2020-10-20 00:36:15+00:00",
            "title": "Cue Me In: Content-Inducing Approaches to Interactive Story Generation",
            "authors": [
                "Faeze Brahman",
                "Alexandru Petrusca",
                "Snigdha Chaturvedi"
            ],
            "abstract": "Automatically generating stories is a challenging problem that requires\nproducing causally related and logical sequences of events about a topic.\nPrevious approaches in this domain have focused largely on one-shot generation,\nwhere a language model outputs a complete story based on limited initial input\nfrom a user. Here, we instead focus on the task of interactive story\ngeneration, where the user provides the model mid-level sentence abstractions\nin the form of cue phrases during the generation process. This provides an\ninterface for human users to guide the story generation. We present two\ncontent-inducing approaches to effectively incorporate this additional\ninformation. Experimental results from both automatic and human evaluations\nshow that these methods produce more topically coherent and personalized\nstories compared to baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10035v2",
            "date": "2020-10-20 05:06:23+00:00",
            "title": "Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification",
            "authors": [
                "Neha Srikanth",
                "Junyi Jessy Li"
            ],
            "abstract": "Much of modern-day text simplification research focuses on sentence-level\nsimplification, transforming original, more complex sentences into simplified\nversions. However, adding content can often be useful when difficult concepts\nand reasoning need to be explained. In this work, we present the first\ndata-driven study of content addition in text simplification, which we call\nelaborative simplification. We introduce a new annotated dataset of 1.3K\ninstances of elaborative simplification in the Newsela corpus, and analyze how\nentities, ideas, and concepts are elaborated through the lens of contextual\nspecificity. We establish baselines for elaboration generation using\nlarge-scale pre-trained language models, and demonstrate that considering\ncontextual specificity during generation can improve performance. Our results\nillustrate the complexities of elaborative simplification, suggesting many\ninteresting directions for future work.",
            "journal ref": "Findings of Association of Computational Linguistics 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10042v2",
            "date": "2020-10-20 05:42:47+00:00",
            "title": "Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation",
            "authors": [
                "Yasuhide Miura",
                "Yuhao Zhang",
                "Emily Bao Tsai",
                "Curtis P. Langlotz",
                "Dan Jurafsky"
            ],
            "abstract": "Neural image-to-text radiology report generation systems offer the potential\nto improve radiology reporting by reducing the repetitive process of report\ndrafting and identifying possible medical errors. However, existing report\ngeneration systems, despite achieving high performances on natural language\ngeneration metrics such as CIDEr or BLEU, still suffer from incomplete and\ninconsistent generations. Here we introduce two new simple rewards to encourage\nthe generation of factually complete and consistent radiology reports: one that\nencourages the system to generate radiology domain entities consistent with the\nreference, and one that uses natural language inference to encourage these\nentities to be described in inferentially consistent ways. We combine these\nwith the novel use of an existing semantic equivalence metric (BERTScore). We\nfurther propose a report generation system that optimizes these rewards via\nreinforcement learning. On two open radiology report datasets, our system\nsubstantially improved the F1 score of a clinical information extraction\nperformance by +22.1 (Delta +63.9%). We further show via a human evaluation and\na qualitative analysis that our system leads to generations that are more\nfactually complete and consistent compared to the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10044v1",
            "date": "2020-10-20 05:44:55+00:00",
            "title": "Incorporating Commonsense Knowledge into Abstractive Dialogue Summarization via Heterogeneous Graph Networks",
            "authors": [
                "Xiachong Feng",
                "Xiaocheng Feng",
                "Bing Qin",
                "Ting Liu"
            ],
            "abstract": "Abstractive dialogue summarization is the task of capturing the highlights of\na dialogue and rewriting them into a concise version. In this paper, we present\na novel multi-speaker dialogue summarizer to demonstrate how large-scale\ncommonsense knowledge can facilitate dialogue understanding and summary\ngeneration. In detail, we consider utterance and commonsense knowledge as two\ndifferent types of data and design a Dialogue Heterogeneous Graph Network\n(D-HGN) for modeling both information. Meanwhile, we also add speakers as\nheterogeneous nodes to facilitate information flow. Experimental results on the\nSAMSum dataset show that our model can outperform various methods. We also\nconduct zero-shot setting experiments on the Argumentative Dialogue Summary\nCorpus, the results show that our model can better generalized to the new\ndomain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10077v2",
            "date": "2020-10-20 07:08:00+00:00",
            "title": "Neural Language Modeling for Contextualized Temporal Graph Generation",
            "authors": [
                "Aman Madaan",
                "Yiming Yang"
            ],
            "abstract": "This paper presents the first study on using large-scale pre-trained language\nmodels for automated generation of an event-level temporal graph for a\ndocument. Despite the huge success of neural pre-training methods in NLP tasks,\nits potential for temporal reasoning over event graphs has not been\nsufficiently explored. Part of the reason is the difficulty in obtaining large\ntraining corpora with human-annotated events and temporal links. We address\nthis challenge by using existing IE/NLP tools to automatically generate a large\nquantity (89,000) of system-produced document-graph pairs, and propose a novel\nformulation of the contextualized graph generation problem as a\nsequence-to-sequence mapping task. These strategies enable us to leverage and\nfine-tune pre-trained language models on the system-induced training data for\nthe graph generation task. Our experiments show that our approach is highly\neffective in generating structurally and semantically valid graphs. Further,\nevaluation on a challenging hand-labeled, out-domain corpus shows that our\nmethod outperforms the closest existing method by a large margin on several\nmetrics. Code and pre-trained models are available at\nhttps://github.com/madaan/temporal-graph-gen.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10111v1",
            "date": "2020-10-20 08:10:29+00:00",
            "title": "JUNLP@Dravidian-CodeMix-FIRE2020: Sentiment Classification of Code-Mixed Tweets using Bi-Directional RNN and Language Tags",
            "authors": [
                "Sainik Kumar Mahata",
                "Dipankar Das",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Sentiment analysis has been an active area of research in the past two\ndecades and recently, with the advent of social media, there has been an\nincreasing demand for sentiment analysis on social media texts. Since the\nsocial media texts are not in one language and are largely code-mixed in\nnature, the traditional sentiment classification models fail to produce\nacceptable results. This paper tries to solve this very research problem and\nuses bi-directional LSTMs along with language tagging, to facilitate sentiment\ntagging of code-mixed Tamil texts that have been extracted from social media.\nThe presented algorithm, when evaluated on the test data, garnered precision,\nrecall, and F1 scores of 0.59, 0.66, and 0.58 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10238v1",
            "date": "2020-10-20 13:02:42+00:00",
            "title": "Supertagging-based Parsing with Linear Context-free Rewriting Systems",
            "authors": [
                "Richard M\u00f6rbitz",
                "Thomas Ruprecht"
            ],
            "abstract": "We present the first supertagging-based parser for LCFRS. It utilizes neural\nclassifiers and tremendously outperforms previous LCFRS-based parsers in both\naccuracy and parsing speed. Moreover, our results keep up with the best\n(general) discontinuous parsers, particularly the scores for discontinuous\nconstitutents are excellent. The heart of our approach is an efficient\nlexicalization procedure which induces a lexical LCFRS from any discontinuous\ntreebank. It is an adaptation of previous work by M\\\"orbitz and Ruprecht\n(2020). We also describe a modification to usual chart-based LCFRS parsing that\naccounts for supertagging and introduce a procedure for the transformation of\nlexical LCFRS derivations into equivalent parse trees of the original treebank.\nOur approach is implemented and evaluated on the English Discontinuous Penn\nTreebank and the German corpora NeGra and Tiger.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10286v1",
            "date": "2020-10-20 13:56:30+00:00",
            "title": "Bi-directional Cognitive Thinking Network for Machine Reading Comprehension",
            "authors": [
                "Wei Peng",
                "Yue Hu",
                "Luxi Xing",
                "Yuqiang Xie",
                "Jing Yu",
                "Yajing Sun",
                "Xiangpeng Wei"
            ],
            "abstract": "We propose a novel Bi-directional Cognitive Knowledge Framework (BCKF) for\nreading comprehension from the perspective of complementary learning systems\ntheory. It aims to simulate two ways of thinking in the brain to answer\nquestions, including reverse thinking and inertial thinking. To validate the\neffectiveness of our framework, we design a corresponding Bi-directional\nCognitive Thinking Network (BCTN) to encode the passage and generate a question\n(answer) given an answer (question) and decouple the bi-directional knowledge.\nThe model has the ability to reverse reasoning questions which can assist\ninertial thinking to generate more accurate answers. Competitive improvement is\nobserved in DuReader dataset, confirming our hypothesis that bi-directional\nknowledge helps the QA task. The novel framework shows an interesting\nperspective on machine reading comprehension and cognitive science.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10323v1",
            "date": "2020-10-20 14:45:25+00:00",
            "title": "Topic-Aware Abstractive Text Summarization",
            "authors": [
                "Chujie Zheng",
                "Kunpeng Zhang",
                "Harry Jiannan Wang",
                "Ling Fan"
            ],
            "abstract": "Automatic text summarization aims at condensing a document to a shorter\nversion while preserving the key information. Different from extractive\nsummarization which simply selects text fragments from the document,\nabstractive summarization generates the summary in a word-by-word manner. Most\ncurrent state-of-the-art (SOTA) abstractive summarization methods are based on\nthe Transformer-based encoder-decoder architecture and focus on novel\nself-supervised objectives in pre-training. While these models well capture the\ncontextual information among words in documents, little attention has been paid\nto incorporating global semantics to better fine-tune for the downstream\nabstractive summarization task.\n  In this study, we propose a topic-aware abstractive summarization (TAAS)\nframework by leveraging the underlying semantic structure of documents\nrepresented by their latent topics. Specifically, TAAS seamlessly incorporates\na neural topic modeling into an encoder-decoder based sequence generation\nprocedure via attention for summarization. This design is able to learn and\npreserve global semantics of documents and thus makes summarization effective,\nwhich has been proved by our experiments on real-world datasets. As compared to\nseveral cutting-edge baseline methods, we show that TAAS outperforms BART, a\nwell-recognized SOTA model, by 2%, 8%, and 12% regarding the F measure of\nROUGE-1, ROUGE-2, and ROUGE-L, respectively. TAAS also achieves comparable\nperformance to PEGASUS and ProphetNet, which is difficult to accomplish given\nthat training PEGASUS and ProphetNet requires enormous computing capacity\nbeyond what we used in this study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10392v3",
            "date": "2020-10-20 15:58:53+00:00",
            "title": "CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters",
            "authors": [
                "Hicham El Boukkouri",
                "Olivier Ferret",
                "Thomas Lavergne",
                "Hiroshi Noji",
                "Pierre Zweigenbaum",
                "Junichi Tsujii"
            ],
            "abstract": "Due to the compelling improvements brought by BERT, many recent\nrepresentation models adopted the Transformer architecture as their main\nbuilding block, consequently inheriting the wordpiece tokenization system\ndespite it not being intrinsically linked to the notion of Transformers. While\nthis system is thought to achieve a good balance between the flexibility of\ncharacters and the efficiency of full words, using predefined wordpiece\nvocabularies from the general domain is not always suitable, especially when\nbuilding models for specialized domains (e.g., the medical domain). Moreover,\nadopting a wordpiece tokenization shifts the focus from the word level to the\nsubword level, making the models conceptually more complex and arguably less\nconvenient in practice. For these reasons, we propose CharacterBERT, a new\nvariant of BERT that drops the wordpiece system altogether and uses a\nCharacter-CNN module instead to represent entire words by consulting their\ncharacters. We show that this new model improves the performance of BERT on a\nvariety of medical domain tasks while at the same time producing robust,\nword-level and open-vocabulary representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10472v1",
            "date": "2020-10-20 17:31:07+00:00",
            "title": "Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages",
            "authors": [
                "Yiyuan Li",
                "Antonios Anastasopoulos",
                "Alan W Black"
            ],
            "abstract": "Spelling normalization for low resource languages is a challenging task\nbecause the patterns are hard to predict and large corpora are usually required\nto collect enough examples. This work shows a comparison of a neural model and\ncharacter language models with varying amounts on target language data. Our\nusage scenario is interactive correction with nearly zero amounts of training\nexamples, improving models as more data is collected, for example within a chat\napp. Such models are designed to be incrementally improved as feedback is given\nfrom users. In this work, we design a knowledge-base and prediction model\nembedded system for spelling correction in low-resource languages. Experimental\nresults on multiple languages show that the model could become effective with a\nsmall amount of data. We perform experiments on both natural and synthetic\ndata, as well as on data from two endangered languages (Ainu and Griko). Last,\nwe built a prototype system that was used for a small case study on Hinglish,\nwhich further demonstrated the suitability of our approach in real world\nscenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10501v1",
            "date": "2020-10-20 17:54:16+00:00",
            "title": "Natural Language Inference with Mixed Effects",
            "authors": [
                "William Gantt",
                "Benjamin Kane",
                "Aaron Steven White"
            ],
            "abstract": "There is growing evidence that the prevalence of disagreement in the raw\nannotations used to construct natural language inference datasets makes the\ncommon practice of aggregating those annotations to a single label problematic.\nWe propose a generic method that allows one to skip the aggregation step and\ntrain on the raw annotations directly without subjecting the model to unwanted\nnoise that can arise from annotator response biases. We demonstrate that this\nmethod, which generalizes the notion of a \\textit{mixed effects model} by\nincorporating \\textit{annotator random effects} into any existing neural model,\nimproves performance over models that do not incorporate such effects.",
            "journal ref": "The Ninth Joint Conference on Lexical and Computational Semantics\n  (*SEM2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10566v1",
            "date": "2020-10-20 18:57:42+00:00",
            "title": "Better Highlighting: Creating Sub-Sentence Summary Highlights",
            "authors": [
                "Sangwoo Cho",
                "Kaiqiang Song",
                "Chen Li",
                "Dong Yu",
                "Hassan Foroosh",
                "Fei Liu"
            ],
            "abstract": "Amongst the best means to summarize is highlighting. In this paper, we aim to\ngenerate summary highlights to be overlaid on the original documents to make it\neasier for readers to sift through a large amount of text. The method allows\nsummaries to be understood in context to prevent a summarizer from distorting\nthe original meaning, of which abstractive summarizers usually fall short. In\nparticular, we present a new method to produce self-contained highlights that\nare understandable on their own to avoid confusion. Our method combines\ndeterminantal point processes and deep contextualized representations to\nidentify an optimal set of sub-sentence segments that are both important and\nnon-redundant to form summary highlights. To demonstrate the flexibility and\nmodeling power of our method, we conduct extensive experiments on summarization\ndatasets. Our analysis provides evidence that highlighting is a promising\navenue of research towards future summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10573v1",
            "date": "2020-10-20 19:20:29+00:00",
            "title": "AutoMeTS: The Autocomplete for Medical Text Simplification",
            "authors": [
                "Hoang Van",
                "David Kauchak",
                "Gondy Leroy"
            ],
            "abstract": "The goal of text simplification (TS) is to transform difficult text into a\nversion that is easier to understand and more broadly accessible to a wide\nvariety of readers. In some domains, such as healthcare, fully automated\napproaches cannot be used since information must be accurately preserved.\nInstead, semi-automated approaches can be used that assist a human writer in\nsimplifying text faster and at a higher quality. In this paper, we examine the\napplication of autocomplete to text simplification in the medical domain. We\nintroduce a new parallel medical data set consisting of aligned English\nWikipedia with Simple English Wikipedia sentences and examine the application\nof pretrained neural language models (PNLMs) on this dataset. We compare four\nPNLMs(BERT, RoBERTa, XLNet, and GPT-2), and show how the additional context of\nthe sentence to be simplified can be incorporated to achieve better results\n(6.17% absolute improvement over the best individual model). We also introduce\nan ensemble model that combines the four PNLMs and outperforms the best\nindividual model by 2.1%, resulting in an overall word prediction accuracy of\n64.52%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10649v1",
            "date": "2020-10-20 22:20:49+00:00",
            "title": "Detecting Media Bias in News Articles using Gaussian Bias Distributions",
            "authors": [
                "Wei-Fan Chen",
                "Khalid Al-Khatib",
                "Benno Stein",
                "Henning Wachsmuth"
            ],
            "abstract": "Media plays an important role in shaping public opinion. Biased media can\ninfluence people in undesirable directions and hence should be unmasked as\nsuch. We observe that featurebased and neural text classification approaches\nwhich rely only on the distribution of low-level lexical information fail to\ndetect media bias. This weakness becomes most noticeable for articles on new\nevents, where words appear in new contexts and hence their \"bias\npredictiveness\" is unclear. In this paper, we therefore study how second-order\ninformation about biased statements in an article helps to improve detection\neffectiveness. In particular, we utilize the probability distributions of the\nfrequency, positions, and sequential order of lexical and informational\nsentence-level bias in a Gaussian Mixture Model. On an existing media bias\ndataset, we find that the frequency and positions of biased statements strongly\nimpact article-level bias, whereas their exact sequential order is secondary.\nUsing a standard model for sentence-level bias detection, we provide empirical\nevidence that article-level bias detectors that use second-order information\nclearly outperform those without.",
            "journal ref": "EMNLP 2020 Findings",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10652v1",
            "date": "2020-10-20 22:25:00+00:00",
            "title": "Analyzing Political Bias and Unfairness in News Articles at Different Levels of Granularity",
            "authors": [
                "Wei-Fan Chen",
                "Khalid Al-Khatib",
                "Henning Wachsmuth",
                "Benno Stein"
            ],
            "abstract": "Media organizations bear great reponsibility because of their considerable\ninfluence on shaping beliefs and positions of our society. Any form of media\ncan contain overly biased content, e.g., by reporting on political events in a\nselective or incomplete manner. A relevant question hence is whether and how\nsuch form of imbalanced news coverage can be exposed. The research presented in\nthis paper addresses not only the automatic detection of bias but goes one step\nfurther in that it explores how political bias and unfairness are manifested\nlinguistically. In this regard we utilize a new corpus of 6964 news articles\nwith labels derived from adfontesmedia.com and develop a neural model for bias\nassessment. By analyzing this model on article excerpts, we find insightful\nbias patterns at different levels of text granularity, from single words to the\nwhole article discourse.",
            "journal ref": "NLP+CSS 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10669v1",
            "date": "2020-10-20 23:20:31+00:00",
            "title": "Transition-based Parsing with Stack-Transformers",
            "authors": [
                "Ramon Fernandez Astudillo",
                "Miguel Ballesteros",
                "Tahira Naseem",
                "Austin Blodgett",
                "Radu Florian"
            ],
            "abstract": "Modeling the parser state is key to good performance in transition-based\nparsing. Recurrent Neural Networks considerably improved the performance of\ntransition-based systems by modelling the global state, e.g. stack-LSTM\nparsers, or local state modeling of contextualized features, e.g. Bi-LSTM\nparsers. Given the success of Transformer architectures in recent parsing\nsystems, this work explores modifications of the sequence-to-sequence\nTransformer architecture to model either global or local parser states in\ntransition-based parsing. We show that modifications of the cross attention\nmechanism of the Transformer considerably strengthen performance both on\ndependency and Abstract Meaning Representation (AMR) parsing tasks,\nparticularly for smaller models or limited training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10673v1",
            "date": "2020-10-20 23:45:04+00:00",
            "title": "Pushing the Limits of AMR Parsing with Self-Learning",
            "authors": [
                "Young-Suk Lee",
                "Ramon Fernandez Astudillo",
                "Tahira Naseem",
                "Revanth Gangi Reddy",
                "Radu Florian",
                "Salim Roukos"
            ],
            "abstract": "Abstract Meaning Representation (AMR) parsing has experienced a notable\ngrowth in performance in the last two years, due both to the impact of transfer\nlearning and the development of novel architectures specific to AMR. At the\nsame time, self-learning techniques have helped push the performance boundaries\nof other natural language processing applications, such as machine translation\nor question answering. In this paper, we explore different ways in which\ntrained models can be applied to improve AMR parsing performance, including\ngeneration of synthetic text and AMR annotations as well as refinement of\nactions oracle. We show that, without any additional human annotations, these\ntechniques improve an already performant parser and achieve state-of-the-art\nresults on AMR 1.0 and AMR 2.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10694v2",
            "date": "2020-10-21 00:58:29+00:00",
            "title": "An Investigation of the Relation Between Grapheme Embeddings and Pronunciation for Tacotron-based Systems",
            "authors": [
                "Antoine Perquin",
                "Erica Cooper",
                "Junichi Yamagishi"
            ],
            "abstract": "End-to-end models, particularly Tacotron-based ones, are currently a popular\nsolution for text-to-speech synthesis. They allow the production of\nhigh-quality synthesized speech with little to no text preprocessing. Indeed,\nthey can be trained using either graphemes or phonemes as input directly.\nHowever, in the case of grapheme inputs, little is known concerning the\nrelation between the underlying representations learned by the model and word\npronunciations. This work investigates this relation in the case of a Tacotron\nmodel trained on French graphemes. Our analysis shows that grapheme embeddings\nare related to phoneme information despite no such information being present\nduring training. Thanks to this property, we show that grapheme embeddings\nlearned by Tacotron models can be useful for tasks such as grapheme-to-phoneme\nconversion and control of the pronunciation in synthetic speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10743v2",
            "date": "2020-10-21 03:41:49+00:00",
            "title": "Multi-Unit Transformers for Neural Machine Translation",
            "authors": [
                "Jianhao Yan",
                "Fandong Meng",
                "Jie Zhou"
            ],
            "abstract": "Transformer models achieve remarkable success in Neural Machine Translation.\nMany efforts have been devoted to deepening the Transformer by stacking several\nunits (i.e., a combination of Multihead Attentions and FFN) in a cascade, while\nthe investigation over multiple parallel units draws little attention. In this\npaper, we propose the Multi-Unit Transformers (MUTE), which aim to promote the\nexpressiveness of the Transformer by introducing diverse and complementary\nunits. Specifically, we use several parallel units and show that modeling with\nmultiple units improves model performance and introduces diversity. Further, to\nbetter leverage the advantage of the multi-unit setting, we design biased\nmodule and sequential dependency that guide and encourage complementariness\namong different units. Experimental results on three machine translation tasks,\nthe NIST Chinese-to-English, WMT'14 English-to-German and WMT'18\nChinese-to-English, show that the MUTE models significantly outperform the\nTransformer-Base, by up to +1.52, +1.90 and +1.10 BLEU points, with only a mild\ndrop in inference speed (about 3.1%). In addition, our methods also surpass the\nTransformer-Big model, with only 54\\% of its parameters. These results\ndemonstrate the effectiveness of the MUTE, as well as its efficiency in both\nthe inference process and parameter usage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10757v1",
            "date": "2020-10-21 04:28:42+00:00",
            "title": "RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering",
            "authors": [
                "Srinivasan Iyer",
                "Sewon Min",
                "Yashar Mehdad",
                "Wen-tau Yih"
            ],
            "abstract": "State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain\nQuestion Answering (QA) are typically trained for span selection using\ndistantly supervised positive examples and heuristically retrieved negative\nexamples. This training scheme possibly explains empirical observations that\nthese models achieve a high recall amongst their top few predictions, but a low\noverall accuracy, motivating the need for answer re-ranking. We develop a\nsimple and effective re-ranking approach (RECONSIDER) for span-extraction\ntasks, that improves upon the performance of large pre-trained MRC models.\nRECONSIDER is trained on positive and negative examples extracted from high\nconfidence predictions of MRC models, and uses in-passage span annotations to\nperform span-focused re-ranking over a smaller candidate set. As a result,\nRECONSIDER learns to eliminate close false positive passages, and achieves a\nnew state of the art on four QA tasks, including 45.5% Exact Match accuracy on\nNatural Questions with real user questions, and 61.7% on TriviaQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10801v1",
            "date": "2020-10-21 07:39:55+00:00",
            "title": "Quasi Error-free Text Classification and Authorship Recognition in a large Corpus of English Literature based on a Novel Feature Set",
            "authors": [
                "Arthur M. Jacobs",
                "Annette Kinder"
            ],
            "abstract": "The Gutenberg Literary English Corpus (GLEC) provides a rich source of\ntextual data for research in digital humanities, computational linguistics or\nneurocognitive poetics. However, so far only a small subcorpus, the Gutenberg\nEnglish Poetry Corpus, has been submitted to quantitative text analyses\nproviding predictions for scientific studies of literature. Here we show that\nin the entire GLEC quasi error-free text classification and authorship\nrecognition is possible with a method using the same set of five style and five\ncontent features, computed via style and sentiment analysis, in both tasks. Our\nresults identify two standard and two novel features (i.e., type-token ratio,\nfrequency, sonority score, surprise) as most diagnostic in these tasks. By\nproviding a simple tool applicable to both short poems and long novels\ngenerating quantitative predictions about features that co-determe the\ncognitive and affective processing of specific text categories or authors, our\ndata pave the way for many future computational and empirical studies of\nliterature or experiments in reading psychology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10811v1",
            "date": "2020-10-21 08:09:20+00:00",
            "title": "STN4DST: A Scalable Dialogue State Tracking based on Slot Tagging Navigation",
            "authors": [
                "Puhai Yang",
                "Heyan Huang",
                "Xianling Mao"
            ],
            "abstract": "Scalability for handling unknown slot values is a important problem in\ndialogue state tracking (DST). As far as we know, previous scalable DST\napproaches generally rely on either the candidate generation from slot tagging\noutput or the span extraction in dialogue context. However, the candidate\ngeneration based DST often suffers from error propagation due to its pipelined\ntwo-stage process; meanwhile span extraction based DST has the risk of\ngenerating invalid spans in the lack of semantic constraints between start and\nend position pointers. To tackle the above drawbacks, in this paper, we propose\na novel scalable dialogue state tracking method based on slot tagging\nnavigation, which implements an end-to-end single-step pointer to locate and\nextract slot value quickly and accurately by the joint learning of slot tagging\nand slot value position prediction in the dialogue context, especially for\nunknown slot values. Extensive experiments over several benchmark datasets show\nthat the proposed model performs better than state-of-the-art baselines\ngreatly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10820v2",
            "date": "2020-10-21 08:27:36+00:00",
            "title": "Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipedia",
            "authors": [
                "Chan Young Park",
                "Xinru Yan",
                "Anjalie Field",
                "Yulia Tsvetkov"
            ],
            "abstract": "Specific lexical choices in narrative text reflect both the writer's\nattitudes towards people in the narrative and influence the audience's\nreactions. Prior work has examined descriptions of people in English using\ncontextual affective analysis, a natural language processing (NLP) technique\nthat seeks to analyze how people are portrayed along dimensions of power,\nagency, and sentiment. Our work presents an extension of this methodology to\nmultilingual settings, which is enabled by a new corpus that we collect and a\nnew multilingual model. We additionally show how word connotations differ\nacross languages and cultures, highlighting the difficulty of generalizing\nexisting English datasets and methods. We then demonstrate the usefulness of\nour method by analyzing Wikipedia biography pages of members of the LGBT\ncommunity across three languages: English, Russian, and Spanish. Our results\nshow systematic differences in how the LGBT community is portrayed across\nlanguages, surfacing cultural differences in narratives and signs of social\nbiases. Practically, this model can be used to identify Wikipedia articles for\nfurther manual analysis -- articles that might contain content gaps or an\nimbalanced representation of particular social groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10833v1",
            "date": "2020-10-21 08:44:54+00:00",
            "title": "KnowDis: Knowledge Enhanced Data Augmentation for Event Causality Detection via Distant Supervision",
            "authors": [
                "Xinyu Zuo",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao"
            ],
            "abstract": "Modern models of event causality detection (ECD) are mainly based on\nsupervised learning from small hand-labeled corpora. However, hand-labeled\ntraining data is expensive to produce, low coverage of causal expressions and\nlimited in size, which makes supervised methods hard to detect causal relations\nbetween events. To solve this data lacking problem, we investigate a data\naugmentation framework for ECD, dubbed as Knowledge Enhanced Distant Data\nAugmentation (KnowDis). Experimental results on two benchmark datasets\nEventStoryLine corpus and Causal-TimeBank show that 1) KnowDis can augment\navailable training data assisted with the lexical and causal commonsense\nknowledge for ECD via distant supervision, and 2) our method outperforms\nprevious methods by a large margin assisted with automatically labeled training\ndata.",
            "journal ref": "COLING2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10839v1",
            "date": "2020-10-21 09:02:30+00:00",
            "title": "TMT: A Transformer-based Modal Translator for Improving Multimodal Sequence Representations in Audio Visual Scene-aware Dialog",
            "authors": [
                "Wubo Li",
                "Dongwei Jiang",
                "Wei Zou",
                "Xiangang Li"
            ],
            "abstract": "Audio Visual Scene-aware Dialog (AVSD) is a task to generate responses when\ndiscussing about a given video. The previous state-of-the-art model shows\nsuperior performance for this task using Transformer-based architecture.\nHowever, there remain some limitations in learning better representation of\nmodalities. Inspired by Neural Machine Translation (NMT), we propose the\nTransformer-based Modal Translator (TMT) to learn the representations of the\nsource modal sequence by translating the source modal sequence to the related\ntarget modal sequence in a supervised manner. Based on Multimodal Transformer\nNetworks (MTN), we apply TMT to video and dialog, proposing MTN-TMT for the\nvideo-grounded dialog system. On the AVSD track of the Dialog System Technology\nChallenge 7, MTN-TMT outperforms the MTN and other submission models in both\nVideo and Text task and Text Only task. Compared with MTN, MTN-TMT improves all\nmetrics, especially, achieving relative improvement up to 14.1% on CIDEr. Index\nTerms: multimodal learning, audio-visual scene-aware dialog, neural machine\ntranslation, multi-task learning",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10852v4",
            "date": "2020-10-21 09:25:48+00:00",
            "title": "Gender Prediction Based on Vietnamese Names with Machine Learning Techniques",
            "authors": [
                "Huy Quoc To",
                "Kiet Van Nguyen",
                "Ngan Luu-Thuy Nguyen",
                "Anh Gia-Tuan Nguyen"
            ],
            "abstract": "As biological gender is one of the aspects of presenting individual human,\nmuch work has been done on gender classification based on people names. The\nproposals for English and Chinese languages are tremendous; still, there have\nbeen few works done for Vietnamese so far. We propose a new dataset for gender\nprediction based on Vietnamese names. This dataset comprises over 26,000 full\nnames annotated with genders. This dataset is available on our website for\nresearch purposes. In addition, this paper describes six machine learning\nalgorithms (Support Vector Machine, Multinomial Naive Bayes, Bernoulli Naive\nBayes, Decision Tree, Random Forrest and Logistic Regression) and a deep\nlearning model (LSTM) with fastText word embedding for gender prediction on\nVietnamese names. We create a dataset and investigate the impact of each name\ncomponent on detecting gender. As a result, the best F1-score that we have\nachieved is up to 96% on LSTM model and we generate a web API based on our\ntrained model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10866v2",
            "date": "2020-10-21 09:49:47+00:00",
            "title": "PARENTing via Model-Agnostic Reinforcement Learning to Correct Pathological Behaviors in Data-to-Text Generation",
            "authors": [
                "Cl\u00e9ment Rebuffel",
                "Laure Soulier",
                "Geoffrey Scoutheeten",
                "Patrick Gallinari"
            ],
            "abstract": "In language generation models conditioned by structured data, the classical\ntraining via maximum likelihood almost always leads models to pick up on\ndataset divergence (i.e., hallucinations or omissions), and to incorporate them\nerroneously in their own generations at inference. In this work, we build ontop\nof previous Reinforcement Learning based approaches and show that a\nmodel-agnostic framework relying on the recently introduced PARENT metric is\nefficient at reducing both hallucinations and omissions. Evaluations on the\nwidely used WikiBIO and WebNLG benchmarks demonstrate the effectiveness of this\nframework compared to state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10894v1",
            "date": "2020-10-21 11:07:53+00:00",
            "title": "Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training",
            "authors": [
                "Yingyao Wang",
                "Junwei Bao",
                "Guangyi Liu",
                "Youzheng Wu",
                "Xiaodong He",
                "Bowen Zhou",
                "Tiejun Zhao"
            ],
            "abstract": "This paper aims to enhance the few-shot relation classification especially\nfor sentences that jointly describe multiple relations. Due to the fact that\nsome relations usually keep high co-occurrence in the same context, previous\nfew-shot relation classifiers struggle to distinguish them with few annotated\ninstances. To alleviate the above relation confusion problem, we propose CTEG,\na model equipped with two mechanisms to learn to decouple these easily-confused\nrelations. On the one hand, an Entity-Guided Attention (EGA) mechanism, which\nleverages the syntactic relations and relative positions between each word and\nthe specified entity pair, is introduced to guide the attention to filter out\ninformation causing confusion. On the other hand, a Confusion-Aware Training\n(CAT) method is proposed to explicitly learn to distinguish relations by\nplaying a pushing-away game between classifying a sentence into a true relation\nand its confusing relation. Extensive experiments are conducted on the FewRel\ndataset, and the results show that our proposed model achieves comparable and\neven much better results to strong baselines in terms of accuracy. Furthermore,\nthe ablation test and case study verify the effectiveness of our proposed EGA\nand CAT, especially in addressing the relation confusion problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10907v2",
            "date": "2020-10-21 11:37:27+00:00",
            "title": "Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation",
            "authors": [
                "Elena Voita",
                "Rico Sennrich",
                "Ivan Titov"
            ],
            "abstract": "In Neural Machine Translation (and, more generally, conditional language\nmodeling), the generation of a target token is influenced by two types of\ncontext: the source and the prefix of the target sequence. While many attempts\nto understand the internal workings of NMT models have been made, none of them\nexplicitly evaluates relative source and target contributions to a generation\ndecision. We argue that this relative contribution can be evaluated by adopting\na variant of Layerwise Relevance Propagation (LRP). Its underlying\n'conservation principle' makes relevance propagation unique: differently from\nother methods, it evaluates not an abstract quantity reflecting token\nimportance, but the proportion of each token's influence. We extend LRP to the\nTransformer and conduct an analysis of NMT models which explicitly evaluates\nthe source and target relative contributions to the generation process. We\nanalyze changes in these contributions when conditioning on different types of\nprefixes, when varying the training objective or the amount of training data,\nand during the training process. We find that models trained with more data\ntend to rely on source information more and to have more sharp token\ncontributions; the training process is non-monotonic with several stages of\ndifferent nature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10910v1",
            "date": "2020-10-21 11:44:04+00:00",
            "title": "Complaint Identification in Social Media with Transformer Networks",
            "authors": [
                "Mali Jin",
                "Nikolaos Aletras"
            ],
            "abstract": "Complaining is a speech act extensively used by humans to communicate a\nnegative inconsistency between reality and expectations. Previous work on\nautomatically identifying complaints in social media has focused on using\nfeature-based and task-specific neural network models. Adapting\nstate-of-the-art pre-trained neural language models and their combinations with\nother linguistic information from topics or sentiment for complaint prediction\nhas yet to be explored. In this paper, we evaluate a battery of neural models\nunderpinned by transformer networks which we subsequently combine with\nlinguistic information. Experiments on a publicly available data set of\ncomplaints demonstrate that our models outperform previous state-of-the-art\nmethods by a large margin achieving a macro F1 up to 87.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10921v1",
            "date": "2020-10-21 12:08:02+00:00",
            "title": "LemMED: Fast and Effective Neural Morphological Analysis with Short Context Windows",
            "authors": [
                "Aibek Makazhanov",
                "Sharon Goldwater",
                "Adam Lopez"
            ],
            "abstract": "We present LemMED, a character-level encoder-decoder for contextual\nmorphological analysis (combined lemmatization and tagging). LemMED extends and\nis named after two other attention-based models, namely Lematus, a contextual\nlemmatizer, and MED, a morphological (re)inflection model. Our approach does\nnot require training separate lemmatization and tagging models, nor does it\nneed additional resources and tools, such as morphological dictionaries or\ntransducers. Moreover, LemMED relies solely on character-level representations\nand on local context. Although the model can, in principle, account for global\ncontext on sentence level, our experiments show that using just a single word\nof context around each target word is not only more computationally feasible,\nbut yields better results as well. We evaluate LemMED in the framework of the\nSIMGMORPHON-2019 shared task on combined lemmatization and tagging. In terms of\naverage performance LemMED ranks 5th among 13 systems and is bested only by the\nsubmissions that use contextualized embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10999v1",
            "date": "2020-10-21 13:40:15+00:00",
            "title": "Is Retriever Merely an Approximator of Reader?",
            "authors": [
                "Sohee Yang",
                "Minjoon Seo"
            ],
            "abstract": "The state of the art in open-domain question answering (QA) relies on an\nefficient retriever that drastically reduces the search space for the expensive\nreader. A rather overlooked question in the community is the relationship\nbetween the retriever and the reader, and in particular, if the whole purpose\nof the retriever is just a fast approximation for the reader. Our empirical\nevidence indicates that the answer is no, and that the reader and the retriever\nare complementary to each other even in terms of accuracy only. We make a\ncareful conjecture that the architectural constraint of the retriever, which\nhas been originally intended for enabling approximate search, seems to also\nmake the model more robust in large-scale search. We then propose to distill\nthe reader into the retriever so that the retriever absorbs the strength of the\nreader while keeping its own benefit. Experimental results show that our method\ncan enhance the document recall rate as well as the end-to-end QA accuracy of\noff-the-shelf retrievers in open-domain QA tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11004v3",
            "date": "2020-10-21 13:44:40+00:00",
            "title": "Controllable Text Simplification with Explicit Paraphrasing",
            "authors": [
                "Mounica Maddela",
                "Fernando Alva-Manchego",
                "Wei Xu"
            ],
            "abstract": "Text Simplification improves the readability of sentences through several\nrewriting transformations, such as lexical paraphrasing, deletion, and\nsplitting. Current simplification systems are predominantly\nsequence-to-sequence models that are trained end-to-end to perform all these\noperations simultaneously. However, such systems limit themselves to mostly\ndeleting words and cannot easily adapt to the requirements of different target\naudiences. In this paper, we propose a novel hybrid approach that leverages\nlinguistically-motivated rules for splitting and deletion, and couples them\nwith a neural paraphrasing model to produce varied rewriting styles. We\nintroduce a new data augmentation method to improve the paraphrasing capability\nof our model. Through automatic and manual evaluations, we show that our\nproposed model establishes a new state-of-the-art for the task, paraphrasing\nmore often than the existing systems, and can control the degree of each\nsimplification operation applied to the input texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11018v1",
            "date": "2020-10-21 14:02:27+00:00",
            "title": "Token Drop mechanism for Neural Machine Translation",
            "authors": [
                "Huaao Zhang",
                "Shigui Qiu",
                "Xiangyu Duan",
                "Min Zhang"
            ],
            "abstract": "Neural machine translation with millions of parameters is vulnerable to\nunfamiliar inputs. We propose Token Drop to improve generalization and avoid\noverfitting for the NMT model. Similar to word dropout, whereas we replace\ndropped token with a special token instead of setting zero to words. We further\nintroduce two self-supervised objectives: Replaced Token Detection and Dropped\nToken Prediction. Our method aims to force model generating target translation\nwith less information, in this way the model can learn textual representation\nbetter. Experiments on Chinese-English and English-Romanian benchmark\ndemonstrate the effectiveness of our approach and our model achieves\nsignificant improvements over a strong Transformer baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11019v1",
            "date": "2020-10-21 14:03:16+00:00",
            "title": "LT3 at SemEval-2020 Task 9: Cross-lingual Embeddings for Sentiment Analysis of Hinglish Social Media Text",
            "authors": [
                "Pranaydeep Singh",
                "Els Lefever"
            ],
            "abstract": "This paper describes our contribution to the SemEval-2020 Task 9 on Sentiment\nAnalysis for Code-mixed Social Media Text. We investigated two approaches to\nsolve the task of Hinglish sentiment analysis. The first approach uses\ncross-lingual embeddings resulting from projecting Hinglish and pre-trained\nEnglish FastText word embeddings in the same space. The second approach\nincorporates pre-trained English embeddings that are incrementally retrained\nwith a set of Hinglish tweets. The results show that the second approach\nperforms best, with an F1-score of 70.52% on the held-out test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11032v2",
            "date": "2020-10-21 14:28:22+00:00",
            "title": "Classifying Syntactic Errors in Learner Language",
            "authors": [
                "Leshem Choshen",
                "Dmitry Nikolaev",
                "Yevgeni Berzak",
                "Omri Abend"
            ],
            "abstract": "We present a method for classifying syntactic errors in learner language,\nnamely errors whose correction alters the morphosyntactic structure of a\nsentence.\n  The methodology builds on the established Universal Dependencies syntactic\nrepresentation scheme, and provides complementary information to other\nerror-classification systems.\n  Unlike existing error classification methods, our method is applicable across\nlanguages, which we showcase by producing a detailed picture of syntactic\nerrors in learner English and learner Russian. We further demonstrate the\nutility of the methodology for analyzing the outputs of leading Grammatical\nError Correction (GEC) systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11054v1",
            "date": "2020-10-21 15:03:52+00:00",
            "title": "Deciphering Undersegmented Ancient Scripts Using Phonetic Prior",
            "authors": [
                "Jiaming Luo",
                "Frederik Hartmann",
                "Enrico Santus",
                "Yuan Cao",
                "Regina Barzilay"
            ],
            "abstract": "Most undeciphered lost languages exhibit two characteristics that pose\nsignificant decipherment challenges: (1) the scripts are not fully segmented\ninto words; (2) the closest known language is not determined. We propose a\ndecipherment model that handles both of these challenges by building on rich\nlinguistic constraints reflecting consistent patterns in historical sound\nchange. We capture the natural phonological geometry by learning character\nembeddings based on the International Phonetic Alphabet (IPA). The resulting\ngenerative framework jointly models word segmentation and cognate alignment,\ninformed by phonological constraints. We evaluate the model on both deciphered\nlanguages (Gothic, Ugaritic) and an undeciphered one (Iberian). The experiments\nshow that incorporating phonetic geometry leads to clear and consistent gains.\nAdditionally, we propose a measure for language closeness which correctly\nidentifies related languages for Gothic and Ugaritic. For Iberian, the method\ndoes not show strong evidence supporting Basque as a related language,\nconcurring with the favored position by the current scholarship.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11080v1",
            "date": "2020-10-21 15:43:07+00:00",
            "title": "Online Conversation Disentanglement with Pointer Networks",
            "authors": [
                "Tao Yu",
                "Shafiq Joty"
            ],
            "abstract": "Huge amounts of textual conversations occur online every day, where multiple\nconversations take place concurrently. Interleaved conversations lead to\ndifficulties in not only following the ongoing discussions but also extracting\nrelevant information from simultaneous messages. Conversation disentanglement\naims to separate intermingled messages into detached conversations. However,\nexisting disentanglement methods rely mostly on handcrafted features that are\ndataset specific, which hinders generalization and adaptability. In this work,\nwe propose an end-to-end online framework for conversation disentanglement that\navoids time-consuming domain-specific feature engineering. We design a novel\nway to embed the whole utterance that comprises timestamp, speaker, and message\ntext, and proposes a custom attention mechanism that models disentanglement as\na pointing problem while effectively capturing inter-utterance interactions in\nan end-to-end fashion. We also introduce a joint-learning objective to better\ncapture contextual information. Our experiments on the Ubuntu IRC dataset show\nthat our method achieves state-of-the-art performance in both link and\nconversation prediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11085v1",
            "date": "2020-10-21 15:53:29+00:00",
            "title": "NeuSpell: A Neural Spelling Correction Toolkit",
            "authors": [
                "Sai Muralidhar Jayanthi",
                "Danish Pruthi",
                "Graham Neubig"
            ],
            "abstract": "We introduce NeuSpell, an open-source toolkit for spelling correction in\nEnglish. Our toolkit comprises ten different models, and benchmarks them on\nnaturally occurring misspellings from multiple sources. We find that many\nsystems do not adequately leverage the context around the misspelt token. To\nremedy this, (i) we train neural models using spelling errors in context,\nsynthetically constructed by reverse engineering isolated misspellings; and\n(ii) use contextual representations. By training on our synthetic examples,\ncorrection rates improve by 9% (absolute) compared to the case when models are\ntrained on randomly sampled character perturbations. Using richer contextual\nrepresentations boosts the correction rate by another 3%. Our toolkit enables\npractitioners to use our proposed and existing spelling correction systems,\nboth via a unified command line, as well as a web interface. Among many\npotential applications, we demonstrate the utility of our spell-checkers in\ncombating adversarial misspellings. The toolkit can be accessed at\nneuspell.github.io. Code and pretrained models are available at\nhttp://github.com/neuspell/neuspell.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11089v1",
            "date": "2020-10-16 20:39:57+00:00",
            "title": "Lexicon generation for detecting fake news",
            "authors": [
                "U\u011fur Merto\u011flu",
                "Burkay Gen\u00e7"
            ],
            "abstract": "With the digitization of media, an immense amount of news data has been\ngenerated by online sources, including mainstream media outlets as well as\nsocial networks. However, the ease of production and distribution resulted in\ncirculation of fake news as well as credible, authentic news. The pervasive\ndissemination of fake news has extreme negative impacts on individuals and\nsociety. Therefore, fake news detection has recently become an emerging topic\nas an interdisciplinary research field that is attracting significant attention\nfrom many research disciplines, including social sciences and linguistics. In\nthis study, we propose a method primarily based on lexicons including a scoring\nsystem to facilitate the detection of the fake news in Turkish. We contribute\nto the literature by collecting a novel, large scale, and credible dataset of\nTurkish news, and by constructing the first fake news detection lexicon for\nTurkish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11091v1",
            "date": "2020-10-17 00:45:02+00:00",
            "title": "TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis",
            "authors": [
                "Mohiuddin Md Abdul Qudar",
                "Vijay Mago"
            ],
            "abstract": "Twitter is a well-known microblogging social site where users express their\nviews and opinions in real-time. As a result, tweets tend to contain valuable\ninformation. With the advancements of deep learning in the domain of natural\nlanguage processing, extracting meaningful information from tweets has become a\ngrowing interest among natural language researchers. Applying existing language\nrepresentation models to extract information from Twitter does not often\nproduce good results. Moreover, there is no existing language representation\nmodels for text analysis specific to the social media domain. Hence, in this\narticle, we introduce two TweetBERT models, which are domain specific language\npresentation models, pre-trained on millions of tweets. We show that the\nTweetBERT models significantly outperform the traditional BERT models in\nTwitter text mining tasks by more than 7% on each Twitter dataset. We also\nprovide an extensive analysis by evaluating seven BERT models on 31 different\ndatasets. Our results validate our hypothesis that continuously training\nlanguage models on twitter corpus help performance with Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11119v1",
            "date": "2020-10-21 16:27:49+00:00",
            "title": "DuoRAT: Towards Simpler Text-to-SQL Models",
            "authors": [
                "Torsten Scholak",
                "Raymond Li",
                "Dzmitry Bahdanau",
                "Harm de Vries",
                "Chris Pal"
            ],
            "abstract": "Recent research has shown that neural text-to-SQL models can effectively\ntranslate natural language questions into corresponding SQL queries on unseen\ndatabases. Working mostly on the Spider dataset, researchers have been\nproposing increasingly sophisticated modelling approaches to the problem.\nContrary to this trend, in this paper we identify the aspects in which\ntext-to-SQL models can be simplified. We begin by building DuoRAT, a\nre-implementation of the state-of-the-art RAT-SQL model that unlike RAT-SQL is\nusing only relation-aware or vanilla transformers as the building blocks. We\nperform several ablation experiments using DuoRAT as the baseline model. Our\nexperiments confirm the usefulness of some of the techniques and point out the\nredundancy of others, including structural SQL features and features that link\nthe question with the schema.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11153v2",
            "date": "2020-10-21 17:18:51+00:00",
            "title": "Cascaded Models With Cyclic Feedback For Direct Speech Translation",
            "authors": [
                "Tsz Kin Lam",
                "Shigehiko Schamoni",
                "Stefan Riezler"
            ],
            "abstract": "Direct speech translation describes a scenario where only speech inputs and\ncorresponding translations are available. Such data are notoriously limited. We\npresent a technique that allows cascades of automatic speech recognition (ASR)\nand machine translation (MT) to exploit in-domain direct speech translation\ndata in addition to out-of-domain MT and ASR data. After pre-training MT and\nASR, we use a feedback cycle where the downstream performance of the MT system\nis used as a signal to improve the ASR system by self-training, and the MT\ncomponent is fine-tuned on multiple ASR outputs, making it more tolerant\ntowards spelling variations. A comparison to end-to-end speech translation\nusing components of identical architecture and the same data shows gains of up\nto 3.8 BLEU points on LibriVoxDeEn and up to 5.1 BLEU points on CoVoST for\nGerman-to-English speech translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11247v1",
            "date": "2020-10-21 19:03:06+00:00",
            "title": "Improving Simultaneous Translation with Pseudo References",
            "authors": [
                "Junkun Chen",
                "Renjie Zheng",
                "Atsuhito Kita",
                "Mingbo Ma",
                "Liang Huang"
            ],
            "abstract": "Simultaneous translation is vastly different from full-sentence translation,\nin the sense that it starts translation before the source sentence ends, with\nonly a few words delay. However, due to the lack of large scale and publicly\navailable simultaneous translation datasets, most simultaneous translation\nsystems still train with ordinary full-sentence parallel corpora which are not\nsuitable for the simultaneous scenario due to the existence of unnecessary\nlong-distance reorderings. Instead of expensive, time-consuming annotation, we\npropose a novel method that rewrites the target side of existing full-sentence\ncorpus into simultaneous-style translation. Experiments on Chinese-to-English\ntranslation demonstrate about +2.7 BLEU improvements with the addition of newly\ngenerated pseudo references.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11253v2",
            "date": "2020-10-21 19:16:27+00:00",
            "title": "Clustering-based Inference for Biomedical Entity Linking",
            "authors": [
                "Rico Angell",
                "Nicholas Monath",
                "Sunil Mohan",
                "Nishant Yadav",
                "Andrew McCallum"
            ],
            "abstract": "Due to large number of entities in biomedical knowledge bases, only a small\nfraction of entities have corresponding labelled training data. This\nnecessitates entity linking models which are able to link mentions of unseen\nentities using learned representations of entities. Previous approaches link\neach mention independently, ignoring the relationships within and across\ndocuments between the entity mentions. These relations can be very useful for\nlinking mentions in biomedical text where linking decisions are often difficult\ndue mentions having a generic or a highly specialized form. In this paper, we\nintroduce a model in which linking decisions can be made not merely by linking\nto a knowledge base entity but also by grouping multiple mentions together via\nclustering and jointly making linking predictions. In experiments on the\nlargest publicly available biomedical dataset, we improve the best independent\nprediction for entity linking by 3.0 points of accuracy, and our\nclustering-based inference model further improves entity linking by 2.3 points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11304v3",
            "date": "2020-10-21 20:41:23+00:00",
            "title": "Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling",
            "authors": [
                "Wenxuan Zhou",
                "Kevin Huang",
                "Tengyu Ma",
                "Jing Huang"
            ],
            "abstract": "Document-level relation extraction (RE) poses new challenges compared to its\nsentence-level counterpart. One document commonly contains multiple entity\npairs, and one entity pair occurs multiple times in the document associated\nwith multiple possible relations. In this paper, we propose two novel\ntechniques, adaptive thresholding and localized context pooling, to solve the\nmulti-label and multi-entity problems. The adaptive thresholding replaces the\nglobal threshold for multi-label classification in the prior work with a\nlearnable entities-dependent threshold. The localized context pooling directly\ntransfers attention from pre-trained language models to locate relevant context\nthat is useful to decide the relation. We experiment on three document-level RE\nbenchmark datasets: DocRED, a recently released large-scale RE dataset, and two\ndatasets CDRand GDA in the biomedical domain. Our ATLOP (Adaptive Thresholding\nand Localized cOntext Pooling) model achieves an F1 score of 63.4, and also\nsignificantly outperforms existing models on both CDR and GDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11325v1",
            "date": "2020-10-21 21:48:39+00:00",
            "title": "Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction",
            "authors": [
                "Rui Feng",
                "Jie Yuan",
                "Chao Zhang"
            ],
            "abstract": "We study the problem of event extraction from text data, which requires both\ndetecting target event types and their arguments. Typically, both the event\ndetection and argument detection subtasks are formulated as supervised sequence\nlabeling problems. We argue that the event extraction models so trained are\ninherently label-hungry, and can generalize poorly across domains and text\ngenres.We propose a reading comprehension framework for event\nextraction.Specifically, we formulate event detection as a textual entailment\nprediction problem, and argument detection as a question answer-ing problem. By\nconstructing proper query templates, our approach can effectively distill rich\nknowledge about tasks and label semantics from pretrained reading comprehension\nmodels. Moreover, our model can be fine-tuned with a small amount of data to\nboost its performance. Our experiment results show that our method performs\nstrongly for zero-shot and few-shot event extraction, and it achieves\nstate-of-the-art performance on the ACE 2005 benchmark when trained with full\nsupervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11333v1",
            "date": "2020-10-21 22:07:31+00:00",
            "title": "Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas",
            "authors": [
                "Yogarshi Vyas",
                "Miguel Ballesteros"
            ],
            "abstract": "In entity linking, mentions of named entities in raw text are disambiguated\nagainst a knowledge base (KB). This work focuses on linking to unseen KBs that\ndo not have training data and whose schema is unknown during training. Our\napproach relies on methods to flexibly convert entities from arbitrary KBs with\nseveral attribute-value pairs into flat strings, which we use in conjunction\nwith state-of-the-art models for zero-shot linking. To improve the\ngeneralization of our model, we use two regularization schemes based on\nshuffling of entity attributes and handling of unseen attributes. Experiments\non English datasets where models are trained on the CoNLL dataset, and tested\non the TAC-KBP 2010 dataset show that our models outperform baseline models by\nover 12 points of accuracy. Unlike prior work, our approach also allows for\nseamlessly combining multiple training datasets. We test this ability by adding\nboth a completely different dataset (Wikia), as well as increasing amount of\ntraining data from the TAC-KBP 2010 training set. Our models perform favorably\nacross the board.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11338v2",
            "date": "2020-10-21 22:40:43+00:00",
            "title": "A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks",
            "authors": [
                "Yun Tang",
                "Juan Pino",
                "Changhan Wang",
                "Xutai Ma",
                "Dmitriy Genzel"
            ],
            "abstract": "Attention-based sequence-to-sequence modeling provides a powerful and elegant\nsolution for applications that need to map one sequence to a different\nsequence. Its success heavily relies on the availability of large amounts of\ntraining data. This presents a challenge for speech applications where labelled\nspeech data is very expensive to obtain, such as automatic speech recognition\n(ASR) and speech translation (ST). In this study, we propose a general\nmulti-task learning framework to leverage text data for ASR and ST tasks. Two\nauxiliary tasks, a denoising autoencoder task and machine translation task, are\nproposed to be co-trained with ASR and ST tasks respectively. We demonstrate\nthat representing text input as phoneme sequences can reduce the difference\nbetween speech and text inputs, and enhance the knowledge transfer from text\ncorpora to the speech to text tasks. Our experiments show that the proposed\nmethod achieves a relative 10~15% word error rate reduction on the English\nLibrispeech task compared with our baseline, and improves the speech\ntranslation quality on the MuST-C tasks by 3.6~9.2 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11349v1",
            "date": "2020-10-21 23:40:26+00:00",
            "title": "LSTM-LM with Long-Term History for First-Pass Decoding in Conversational Speech Recognition",
            "authors": [
                "Xie Chen",
                "Sarangarajan Parthasarathy",
                "William Gale",
                "Shuangyu Chang",
                "Michael Zeng"
            ],
            "abstract": "LSTM language models (LSTM-LMs) have been proven to be powerful and yielded\nsignificant performance improvements over count based n-gram LMs in modern\nspeech recognition systems. Due to its infinite history states and\ncomputational load, most previous studies focus on applying LSTM-LMs in the\nsecond-pass for rescoring purpose. Recent work shows that it is feasible and\ncomputationally affordable to adopt the LSTM-LMs in the first-pass decoding\nwithin a dynamic (or tree based) decoder framework. In this work, the LSTM-LM\nis composed with a WFST decoder on-the-fly for the first-pass decoding.\nFurthermore, motivated by the long-term history nature of LSTM-LMs, the use of\ncontext beyond the current utterance is explored for the first-pass decoding in\nconversational speech recognition. The context information is captured by the\nhidden states of LSTM-LMs across utterance and can be used to guide the\nfirst-pass search effectively. The experimental results in our internal meeting\ntranscription system show that significant performance improvements can be\nobtained by incorporating the contextual information with LSTM-LMs in the\nfirst-pass decoding, compared to applying the contextual information in the\nsecond-pass rescoring.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11351v1",
            "date": "2020-10-21 23:45:18+00:00",
            "title": "Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures",
            "authors": [
                "M. Li",
                "H. Bai",
                "L. Tan",
                "K. Xiong",
                "M. Li",
                "J. Lin"
            ],
            "abstract": "Measuring sentence semantic similarity using pre-trained language models such\nas BERT generally yields unsatisfactory zero-shot performance, and one main\nreason is ineffective token aggregation methods such as mean pooling. In this\npaper, we demonstrate under a Bayesian framework that distance between\nprimitive statistics such as the mean of word embeddings are fundamentally\nflawed for capturing sentence-level semantic similarity. To remedy this issue,\nwe propose to learn a categorical variational autoencoder (VAE) based on\noff-the-shelf pre-trained language models. We theoretically prove that\nmeasuring the distance between the latent categorical mixtures, namely\nLatte-Mix, can better reflect the true sentence semantic similarity. In\naddition, our Bayesian framework provides explanations for why models finetuned\non labelled sentence pairs have better zero-shot performance. We also\nempirically demonstrate that these finetuned models could be further improved\nby Latte-Mix. Our method not only yields the state-of-the-art zero-shot\nperformance on semantic similarity datasets such as STS, but also enjoy the\nbenefits of fast training and having small memory footprints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11383v1",
            "date": "2020-10-22 02:14:27+00:00",
            "title": "Exploit Multiple Reference Graphs for Semi-supervised Relation Extraction",
            "authors": [
                "Wanli Li",
                "Tieyun Qian"
            ],
            "abstract": "Manual annotation of the labeled data for relation extraction is\ntime-consuming and labor-intensive. Semi-supervised methods can offer helping\nhands for this problem and have aroused great research interests. Existing work\nfocuses on mapping the unlabeled samples to the classes to augment the labeled\ndataset. However, it is hard to find an overall good mapping function,\nespecially for the samples with complicated syntactic components in one\nsentence.\n  To tackle this limitation, we propose to build the connection between the\nunlabeled data and the labeled ones rather than directly mapping the unlabeled\nsamples to the classes. Specifically, we first use three kinds of information\nto construct reference graphs, including entity reference, verb reference, and\nsemantics reference. The goal is to semantically or lexically connect the\nunlabeled sample(s) to the labeled one(s). Then, we develop a Multiple\nReference Graph (MRefG) model to exploit the reference information for better\nrecognizing high-quality unlabeled samples. The effectiveness of our method is\ndemonstrated by extensive comparison experiments with the state-of-the-art\nbaselines on two public datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11384v1",
            "date": "2020-10-22 02:15:13+00:00",
            "title": "A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews",
            "authors": [
                "Gabriele Pergola",
                "Lin Gui",
                "Yulan He"
            ],
            "abstract": "The flexibility of the inference process in Variational Autoencoders (VAEs)\nhas recently led to revising traditional probabilistic topic models giving rise\nto Neural Topic Models (NTM). Although these approaches have achieved\nsignificant results, surprisingly very little work has been done on how to\ndisentangle the latent topics. Existing topic models when applied to reviews\nmay extract topics associated with writers' subjective opinions mixed with\nthose related to factual descriptions such as plot summaries in movie and book\nreviews. It is thus desirable to automatically separate opinion topics from\nplot/neutral ones enabling a better interpretability. In this paper, we propose\na neural topic model combined with adversarial training to disentangle opinion\ntopics from plot and neutral ones. We conduct an extensive experimental\nassessment introducing a new collection of movie and book reviews paired with\ntheir plots, namely MOBO dataset, showing an improved coherence and variety of\ntopics, a consistent disentanglement rate, and sentiment classification\nperformance superior to other supervised topic models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11445v2",
            "date": "2020-10-22 05:02:06+00:00",
            "title": "MAM: Masked Acoustic Modeling for End-to-End Speech-to-Text Translation",
            "authors": [
                "Junkun Chen",
                "Mingbo Ma",
                "Renjie Zheng",
                "Liang Huang"
            ],
            "abstract": "End-to-end Speech-to-text Translation (E2E-ST), which directly translates\nsource language speech to target language text, is widely useful in practice,\nbut traditional cascaded approaches (ASR+MT) often suffer from error\npropagation in the pipeline. On the other hand, existing end-to-end solutions\nheavily depend on the source language transcriptions for pre-training or\nmulti-task training with Automatic Speech Recognition (ASR). We instead propose\na simple technique to learn a robust speech encoder in a self-supervised\nfashion only on the speech side, which can utilize speech data without\ntranscription. This technique termed Masked Acoustic Modeling (MAM), not only\nprovides an alternative solution to improving E2E-ST, but also can perform\npre-training on any acoustic signals (including non-speech ones) without\nannotation. We conduct our experiments over 8 different translation directions.\nIn the setting without using any transcriptions, our technique achieves an\naverage improvement of +1.1 BLEU, and +2.3 BLEU with MAM pre-training.\nPre-training of MAM with arbitrary acoustic signals also has an average\nimprovement with +1.6 BLEU for those languages. Compared with ASR multi-task\nlearning solution, which replies on transcription during training, our\npre-trained MAM model, which does not use transcription, achieves similar\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11478v2",
            "date": "2020-10-22 06:51:24+00:00",
            "title": "Knowledge Distillation for BERT Unsupervised Domain Adaptation",
            "authors": [
                "Minho Ryu",
                "Kichun Lee"
            ],
            "abstract": "A pre-trained language model, BERT, has brought significant performance\nimprovements across a range of natural language processing tasks. Since the\nmodel is trained on a large corpus of diverse topics, it shows robust\nperformance for domain shift problems in which data distributions at training\n(source data) and testing (target data) differ while sharing similarities.\nDespite its great improvements compared to previous models, it still suffers\nfrom performance degradation due to domain shifts. To mitigate such problems,\nwe propose a simple but effective unsupervised domain adaptation method,\nadversarial adaptation with distillation (AAD), which combines the adversarial\ndiscriminative domain adaptation (ADDA) framework with knowledge distillation.\nWe evaluate our approach in the task of cross-domain sentiment classification\non 30 domain pairs, advancing the state-of-the-art performance for unsupervised\ndomain adaptation in text sentiment classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11490v1",
            "date": "2020-10-22 07:21:17+00:00",
            "title": "On the Effects of Using word2vec Representations in Neural Networks for Dialogue Act Recognition",
            "authors": [
                "Christophe Cerisara",
                "Pavel Kral",
                "Ladislav Lenc"
            ],
            "abstract": "Dialogue act recognition is an important component of a large number of\nnatural language processing pipelines. Many research works have been carried\nout in this area, but relatively few investigate deep neural networks and word\nembeddings. This is surprising, given that both of these techniques have proven\nexceptionally good in most other language-related domains. We propose in this\nwork a new deep neural network that explores recurrent models to capture word\nsequences within sentences, and further study the impact of pretrained word\nembeddings. We validate this model on three languages: English, French and\nCzech. The performance of the proposed approach is consistent across these\nlanguages and it is comparable to the state-of-the-art results in English. More\nimportantly, we confirm that deep neural networks indeed outperform a Maximum\nEntropy classifier, which was expected. However , and this is more surprising,\nwe also found that standard word2vec em-beddings do not seem to bring valuable\ninformation for this task and the proposed model, whatever the size of the\ntraining corpus is. We thus further analyse the resulting embeddings and\nconclude that a possible explanation may be related to the mismatch between the\ntype of lexical-semantic information captured by the word2vec embeddings, and\nthe kind of relations between words that is the most useful for the dialogue\nact recognition task.",
            "journal ref": "Computer Speech and Language, Elsevier, 2018, 47, pp.175 - 193",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11539v1",
            "date": "2020-10-22 09:03:23+00:00",
            "title": "Cross Copy Network for Dialogue Generation",
            "authors": [
                "Changzhen Ji",
                "Xin Zhou",
                "Yating Zhang",
                "Xiaozhong Liu",
                "Changlong Sun",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "In the past few years, audiences from different fields witness the\nachievements of sequence-to-sequence models (e.g., LSTM+attention, Pointer\nGenerator Networks, and Transformer) to enhance dialogue content generation.\nWhile content fluency and accuracy often serve as the major indicators for\nmodel training, dialogue logics, carrying critical information for some\nparticular domains, are often ignored. Take customer service and court debate\ndialogue as examples, compatible logics can be observed across different\ndialogue instances, and this information can provide vital evidence for\nutterance generation. In this paper, we propose a novel network architecture -\nCross Copy Networks(CCN) to explore the current dialog context and similar\ndialogue instances' logical structure simultaneously. Experiments with two\ntasks, court debate and customer service content generation, proved that the\nproposed algorithm is superior to existing state-of-art content generation\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11548v1",
            "date": "2020-10-22 09:20:24+00:00",
            "title": "Method of noun phrase detection in Ukrainian texts",
            "authors": [
                "S. D. Pogorilyy",
                "A. A. Kramov"
            ],
            "abstract": "Introduction. The area of natural language processing considers AI-complete\ntasks that cannot be solved using traditional algorithmic actions. Such tasks\nare commonly implemented with the usage of machine learning methodology and\nmeans of computer linguistics. One of the preprocessing tasks of a text is the\nsearch of noun phrases. The accuracy of this task has implications for the\neffectiveness of many other tasks in the area of natural language processing.\nIn spite of the active development of research in the area of natural language\nprocessing, the investigation of the search for noun phrases within Ukrainian\ntexts are still at an early stage. Results. The different methods of noun\nphrases detection have been analyzed. The expediency of the representation of\nsentences as a tree structure has been justified. The key disadvantage of many\nmethods of noun phrase detection is the severe dependence of the effectiveness\nof their detection from the features of a certain language. Taking into account\nthe unified format of sentence processing and the availability of the trained\nmodel for the building of sentence trees for Ukrainian texts, the Universal\nDependency model has been chosen. The complex method of noun phrases detection\nin Ukrainian texts utilizing Universal Dependencies means and named-entity\nrecognition model has been suggested. Experimental verification of the\neffectiveness of the suggested method on the corpus of Ukrainian news has been\nperformed. Different metrics of method accuracy have been calculated.\nConclusions. The results obtained can indicate that the suggested method can be\nused to find noun phrases in Ukrainian texts. An accuracy increase of the\nmethod can be made with the usage of appropriate named-entity recognition\nmodels according to a subject area.",
            "journal ref": "Control Systems and Computers. 2019. Issue 5. P. 48-59",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11553v1",
            "date": "2020-10-22 09:24:05+00:00",
            "title": "Incorporating Stylistic Lexical Preferences in Generative Language Models",
            "authors": [
                "Hrituraj Singh",
                "Gaurav Verma",
                "Balaji Vasan Srinivasan"
            ],
            "abstract": "While recent advances in language modeling have resulted in powerful\ngeneration models, their generation style remains implicitly dependent on the\ntraining data and can not emulate a specific target style. Leveraging the\ngenerative capabilities of a transformer-based language models, we present an\napproach to induce certain target-author attributes by incorporating continuous\nmulti-dimensional lexical preferences of an author into generative language\nmodels. We introduce rewarding strategies in a reinforcement learning framework\nthat encourages the use of words across multiple categorical dimensions, to\nvarying extents. Our experiments demonstrate that the proposed approach can\ngenerate text that distinctively aligns with a given target author's lexical\nstyle. We conduct quantitative and qualitative comparisons with competitive and\nrelevant baselines to illustrate the benefits of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11574v2",
            "date": "2020-10-22 10:09:10+00:00",
            "title": "Exploiting News Article Structure for Automatic Corpus Generation",
            "authors": [
                "Jan Christian Blaise Cruz",
                "Jose Kristian Resabal",
                "James Lin",
                "Dan John Velasco",
                "Charibeth Cheng"
            ],
            "abstract": "Transformers represent the state-of-the-art in Natural Language Processing\n(NLP) in recent years, proving effective even in tasks done in low-resource\nlanguages. While pretrained transformers for these languages can be made, it is\nchallenging to measure their true performance and capacity due to the lack of\nhard benchmark datasets, as well as the difficulty and cost of producing them.\nIn this paper, we present three contributions: First, we propose a methodology\nfor automatically producing Natural Language Inference (NLI) benchmark datasets\nfor low-resource languages using published news articles. Through this, we\ncreate and release NewsPH-NLI, the first sentence entailment benchmark dataset\nin the low-resource Filipino language. Second, we produce new pretrained\ntransformers based on the ELECTRA technique to further alleviate the resource\nscarcity in Filipino, benchmarking them on our dataset against other\ncommonly-used transfer learning techniques. Lastly, we perform analyses on\ntransfer learning techniques to shed light on their true performance when\noperating in low-data domains through the use of degradation tests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11578v2",
            "date": "2020-10-22 10:16:29+00:00",
            "title": "Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus",
            "authors": [
                "Navita Goyal",
                "Balaji Vasan Srinivasan",
                "Anandhavelu Natarajan",
                "Abhilasha Sancheti"
            ],
            "abstract": "Style transfer has been widely explored in natural language generation with\nnon-parallel corpus by directly or indirectly extracting a notion of style from\nsource and target domain corpus. A common shortcoming of existing approaches is\nthe prerequisite of joint annotations across all the stylistic dimensions under\nconsideration. Availability of such dataset across a combination of styles\nlimits the extension of these setups to multiple style dimensions. While\ncascading single-dimensional models across multiple styles is a possibility, it\nsuffers from content loss, especially when the style dimensions are not\ncompletely independent of each other. In our work, we relax this requirement of\njointly annotated data across multiple styles by using independently acquired\ndata across different style dimensions without any additional annotations. We\ninitialize an encoder-decoder setup with transformer-based language model\npre-trained on a generic corpus and enhance its re-writing capability to\nmultiple target style dimensions by employing multiple style-aware language\nmodels as discriminators. Through quantitative and qualitative evaluation, we\nshow the ability of our model to control styles across multiple style\ndimensions while preserving content of the input text. We compare it against\nbaselines involving cascaded state-of-the-art uni-dimensional style transfer\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11604v2",
            "date": "2020-10-22 11:05:14+00:00",
            "title": "AI-lead Court Debate Case Investigation",
            "authors": [
                "Changzhen Ji",
                "Xin Zhou",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "The multi-role judicial debate composed of the plaintiff, defendant, and\njudge is an important part of the judicial trial. Different from other types of\ndialogue, questions are raised by the judge, The plaintiff, plaintiff's agent\ndefendant, and defendant's agent would be to debating so that the trial can\nproceed in an orderly manner. Question generation is an important task in\nNatural Language Generation. In the judicial trial, it can help the judge raise\nefficient questions so that the judge has a clearer understanding of the case.\nIn this work, we propose an innovative end-to-end question generation\nmodel-Trial Brain Model (TBM) to build a Trial Brain, it can generate the\nquestions the judge wants to ask through the historical dialogue between the\nplaintiff and the defendant. Unlike prior efforts in natural language\ngeneration, our model can learn the judge's questioning intention through\npredefined knowledge. We do experiments on real-world datasets, the\nexperimental results show that our model can provide a more accurate question\nin the multi-role court debate scene.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11639v1",
            "date": "2020-10-22 12:22:50+00:00",
            "title": "Towards Fully Bilingual Deep Language Modeling",
            "authors": [
                "Li-Hsin Chang",
                "Sampo Pyysalo",
                "Jenna Kanerva",
                "Filip Ginter"
            ],
            "abstract": "Language models based on deep neural networks have facilitated great advances\nin natural language processing and understanding tasks in recent years. While\nmodels covering a large number of languages have been introduced, their\nmultilinguality has come at a cost in terms of monolingual performance, and the\nbest-performing models at most tasks not involving cross-lingual transfer\nremain monolingual. In this paper, we consider the question of whether it is\npossible to pre-train a bilingual model for two remotely related languages\nwithout compromising performance at either language. We collect pre-training\ndata, create a Finnish-English bilingual BERT model and evaluate its\nperformance on datasets used to evaluate the corresponding monolingual models.\nOur bilingual model performs on par with Google's original English BERT on GLUE\nand nearly matches the performance of monolingual Finnish BERT on a range of\nFinnish NLP tasks, clearly outperforming multilingual BERT. We find that when\nthe model vocabulary size is increased, the BERT-Base architecture has\nsufficient capacity to learn two remotely related languages to a level where it\nachieves comparable performance with monolingual models, demonstrating the\nfeasibility of training fully bilingual deep language models. The model and all\ntools involved in its creation are freely available at\nhttps://github.com/TurkuNLP/biBERT",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11683v1",
            "date": "2020-10-22 13:21:03+00:00",
            "title": "An Analysis of Simple Data Augmentation for Named Entity Recognition",
            "authors": [
                "Xiang Dai",
                "Heike Adel"
            ],
            "abstract": "Simple yet effective data augmentation techniques have been proposed for\nsentence-level and sentence-pair natural language processing tasks. Inspired by\nthese efforts, we design and compare data augmentation for named entity\nrecognition, which is usually modeled as a token-level sequence labeling\nproblem. Through experiments on two data sets from the biomedical and materials\nscience domains (i2b2-2010 and MaSciP), we show that simple augmentation can\nboost performance for both recurrent and transformer-based models, especially\nfor small training sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11747v1",
            "date": "2020-10-22 14:04:01+00:00",
            "title": "CUNI Systems for the Unsupervised and Very Low Resource Translation Task in WMT20",
            "authors": [
                "Ivana Kvapil\u00edkov\u00e1",
                "Tom Kocmi",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "This paper presents a description of CUNI systems submitted to the WMT20 task\non unsupervised and very low-resource supervised machine translation between\nGerman and Upper Sorbian. We experimented with training on synthetic data and\npre-training on a related language pair. In the fully unsupervised scenario, we\nachieved 25.5 and 23.7 BLEU translating from and into Upper Sorbian,\nrespectively. Our low-resource systems relied on transfer learning from\nGerman-Czech parallel data and achieved 57.4 BLEU and 56.1 BLEU, which is an\nimprovement of 10 BLEU points over the baseline trained only on the available\nsmall German-Upper Sorbian parallel corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11764v1",
            "date": "2020-10-22 14:36:04+00:00",
            "title": "EIGEN: Event Influence GENeration using Pre-trained Language Models",
            "authors": [
                "Aman Madaan",
                "Dheeraj Rajagopal",
                "Yiming Yang",
                "Abhilasha Ravichander",
                "Eduard Hovy",
                "Shrimai Prabhumoye"
            ],
            "abstract": "Reasoning about events and tracking their influences is fundamental to\nunderstanding processes. In this paper, we present EIGEN - a method to leverage\npre-trained language models to generate event influences conditioned on a\ncontext, nature of their influence, and the distance in a reasoning chain. We\nalso derive a new dataset for research and evaluation of methods for event\ninfluence generation. EIGEN outperforms strong baselines both in terms of\nautomated evaluation metrics (by 10 ROUGE points) and human judgments on\ncloseness to reference and relevance of generations. Furthermore, we show that\nthe event influences generated by EIGEN improve the performance on a \"what-if\"\nQuestion Answering (WIQA) benchmark (over 3% F1), especially for questions that\nrequire background knowledge and multi-hop reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11791v1",
            "date": "2020-10-22 15:13:35+00:00",
            "title": "ConVEx: Data-Efficient and Few-Shot Slot Labeling",
            "authors": [
                "Matthew Henderson",
                "Ivan Vuli\u0107"
            ],
            "abstract": "We propose ConVEx (Conversational Value Extractor), an efficient pretraining\nand fine-tuning neural approach for slot-labeling dialog tasks. Instead of\nrelying on more general pretraining objectives from prior work (e.g., language\nmodeling, response selection), ConVEx's pretraining objective, a novel pairwise\ncloze task using Reddit data, is well aligned with its intended usage on\nsequence labeling tasks. This enables learning domain-specific slot labelers by\nsimply fine-tuning decoding layers of the pretrained general-purpose sequence\nlabeling model, while the majority of the pretrained model's parameters are\nkept frozen. We report state-of-the-art performance of ConVEx across a range of\ndiverse domains and data sets for dialog slot-labeling, with the largest gains\nin the most challenging, few-shot setups. We believe that ConVEx's reduced\npretraining times (i.e., only 18 hours on 12 GPUs) and cost, along with its\nefficient fine-tuning and strong performance, promise wider portability and\nscalability for data-efficient sequence-labeling tasks in general.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11818v1",
            "date": "2020-10-22 15:55:15+00:00",
            "title": "Compositional Generalization via Semantic Tagging",
            "authors": [
                "Hao Zheng",
                "Mirella Lapata"
            ],
            "abstract": "Although neural sequence-to-sequence models have been successfully applied to\nsemantic parsing, they struggle to perform well on query-based data splits that\nrequire \\emph{composition generalization}, an ability of systematically\ngeneralizing to unseen composition of seen components. Motivated by the\nexplicitly built-in compositionality in traditional statistical semantic\nparsing, we propose a new decoding framework that preserves the expressivity\nand generality of sequence-to-sequence models while featuring explicit\nlexicon-style alignments and disentangled information processing. Specifically,\nwe decompose decoding into two phases where an input utterance is first tagged\nwith semantic symbols representing the meanings of its individual words, and\nthen a sequence-to-sequence model is used to predict the final meaning\nrepresentation conditioning on the utterance and the predicted tag sequence.\nExperimental results on three semantic parsing datasets with query-based splits\nshow that the proposed approach consistently improves compositional\ngeneralization of sequence-to-sequence models across different model\narchitectures, domains and semantic formalisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11853v1",
            "date": "2020-10-22 16:45:00+00:00",
            "title": "STAR: A Schema-Guided Dialog Dataset for Transfer Learning",
            "authors": [
                "Johannes E. M. Mosig",
                "Shikib Mehri",
                "Thomas Kober"
            ],
            "abstract": "We present STAR, a schema-guided task-oriented dialog dataset consisting of\n127,833 utterances and knowledge base queries across 5,820 task-oriented\ndialogs in 13 domains that is especially designed to facilitate task and domain\ntransfer learning in task-oriented dialog. Furthermore, we propose a scalable\ncrowd-sourcing paradigm to collect arbitrarily large datasets of the same\nquality as STAR. Moreover, we introduce novel schema-guided dialog models that\nuse an explicit description of the task(s) to generalize from known to unknown\ntasks. We demonstrate the effectiveness of these models, particularly for\nzero-shot generalization across tasks and domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11856v3",
            "date": "2020-10-22 16:47:17+00:00",
            "title": "XOR QA: Cross-lingual Open-Retrieval Question Answering",
            "authors": [
                "Akari Asai",
                "Jungo Kasai",
                "Jonathan H. Clark",
                "Kenton Lee",
                "Eunsol Choi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Multilingual question answering tasks typically assume answers exist in the\nsame language as the question. Yet in practice, many languages face both\ninformation scarcity -- where languages have few reference articles -- and\ninformation asymmetry -- where questions reference concepts from other\ncultures. This work extends open-retrieval question answering to a\ncross-lingual setting enabling questions from one language to be answered via\nanswer content from another language. We construct a large-scale dataset built\non questions from TyDi QA lacking same-language answers. Our task formulation,\ncalled Cross-lingual Open Retrieval Question Answering (XOR QA), includes 40k\ninformation-seeking questions from across 7 diverse non-English languages.\nBased on this dataset, we introduce three new tasks that involve cross-lingual\ndocument retrieval using multi-lingual and English resources. We establish\nbaselines with state-of-the-art machine translation systems and cross-lingual\npretrained models. Experimental results suggest that XOR QA is a challenging\ntask that will facilitate the development of novel techniques for multilingual\nquestion answering. Our data and code are available at\nhttps://nlp.cs.washington.edu/xorqa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11859v1",
            "date": "2020-10-22 16:49:18+00:00",
            "title": "Not all parameters are born equal: Attention is mostly what you need",
            "authors": [
                "Nikolay Bogoychev"
            ],
            "abstract": "Transformers are widely used in state-of-the-art machine translation, but the\nkey to their success is still unknown. To gain insight into this, we consider\nthree groups of parameters: embeddings, attention, and feed forward neural\nnetwork (FFN) layers. We examine the relative importance of each by performing\nan ablation study where we initialise them at random and freeze them, so that\ntheir weights do not change over the course of the training. Through this, we\nshow that the attention and FFN are equally important and fulfil the same\nfunctionality in a model. We show that the decision about whether a component\nis frozen or allowed to train is at least as important for the final model\nperformance as its number of parameters. At the same time, the number of\nparameters alone is not indicative of a component's importance. Finally, while\nthe embedding layer is the least essential for machine translation tasks, it is\nthe most important component for language modelling tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11915v1",
            "date": "2020-10-22 17:48:17+00:00",
            "title": "Challenges in Information Seeking QA:Unanswerable Questions and Paragraph Retrieval",
            "authors": [
                "Akari Asai",
                "Eunsol Choi"
            ],
            "abstract": "Recent progress in pretrained language model \"solved\" many reading\ncomprehension benchmark datasets. Yet information-seeking Question Answering\n(QA) datasets, where questions are written without the evidence document,\nremain unsolved. We analyze two such datasets (Natural Questions and TyDi QA)\nto identify remaining headrooms: paragraph selection and answerability\nclassification, i.e. determining whether the paired evidence document contains\nthe answer to the query or not. In other words, given a gold paragraph and\nknowing whether it contains an answer or not, models easily outperform a single\nannotator in both datasets. After identifying unanswerability as a bottleneck,\nwe further inspect what makes questions unanswerable. Our study points to\navenues for future research, both for dataset creation and model development.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11934v3",
            "date": "2020-10-22 17:58:14+00:00",
            "title": "mT5: A massively multilingual pre-trained text-to-text transformer",
            "authors": [
                "Linting Xue",
                "Noah Constant",
                "Adam Roberts",
                "Mihir Kale",
                "Rami Al-Rfou",
                "Aditya Siddhant",
                "Aditya Barua",
                "Colin Raffel"
            ],
            "abstract": "The recent \"Text-to-Text Transfer Transformer\" (T5) leveraged a unified\ntext-to-text format and scale to attain state-of-the-art results on a wide\nvariety of English-language NLP tasks. In this paper, we introduce mT5, a\nmultilingual variant of T5 that was pre-trained on a new Common Crawl-based\ndataset covering 101 languages. We detail the design and modified training of\nmT5 and demonstrate its state-of-the-art performance on many multilingual\nbenchmarks. We also describe a simple technique to prevent \"accidental\ntranslation\" in the zero-shot setting, where a generative model chooses to\n(partially) translate its prediction into the wrong language. All of the code\nand model checkpoints used in this work are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11973v1",
            "date": "2020-10-22 18:18:19+00:00",
            "title": "Rediscovering the Slavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification",
            "authors": [
                "Badr M. Abdullah",
                "Jacek Kudera",
                "Tania Avgustinova",
                "Bernd M\u00f6bius",
                "Dietrich Klakow"
            ],
            "abstract": "Deep neural networks have been employed for various spoken language\nrecognition tasks, including tasks that are multilingual by definition such as\nspoken language identification. In this paper, we present a neural model for\nSlavic language identification in speech signals and analyze its emergent\nrepresentations to investigate whether they reflect objective measures of\nlanguage relatedness and/or non-linguists' perception of language similarity.\nWhile our analysis shows that the language representation space indeed captures\nlanguage relatedness to a great extent, we find perceptual confusability\nbetween languages in our study to be the best predictor of the language\nrepresentation similarity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11988v2",
            "date": "2020-10-22 19:00:36+00:00",
            "title": "Meta-Learning for Domain Generalization in Semantic Parsing",
            "authors": [
                "Bailin Wang",
                "Mirella Lapata",
                "Ivan Titov"
            ],
            "abstract": "The importance of building semantic parsers which can be applied to new\ndomains and generate programs unseen at training has long been acknowledged,\nand datasets testing out-of-domain performance are becoming increasingly\navailable. However, little or no attention has been devoted to learning\nalgorithms or objectives which promote domain generalization, with virtually\nall existing approaches relying on standard supervised learning. In this work,\nwe use a meta-learning framework which targets zero-shot domain generalization\nfor semantic parsing. We apply a model-agnostic training algorithm that\nsimulates zero-shot parsing by constructing virtual train and test sets from\ndisjoint domains. The learning objective capitalizes on the intuition that\ngradient steps that improve source-domain performance should also improve\ntarget-domain performance, thus encouraging a parser to generalize to unseen\ntarget domains. Experimental results on the (English) Spider and Chinese Spider\ndatasets show that the meta-learning objective significantly boosts the\nperformance of a baseline parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12077v1",
            "date": "2020-10-22 21:37:28+00:00",
            "title": "Summarizing Utterances from Japanese Assembly Minutes using Political Sentence-BERT-based Method for QA Lab-PoliInfo-2 Task of NTCIR-15",
            "authors": [
                "Daiki Shirafuji",
                "Hiromichi Kameya",
                "Rafal Rzepka",
                "Kenji Araki"
            ],
            "abstract": "There are many discussions held during political meetings, and a large number\nof utterances for various topics is included in their transcripts. We need to\nread all of them if we want to follow speakers\\' intentions or opinions about a\ngiven topic. To avoid such a costly and time-consuming process to grasp often\nlongish discussions, NLP researchers work on generating concise summaries of\nutterances. Summarization subtask in QA Lab-PoliInfo-2 task of the NTCIR-15\naddresses this problem for Japanese utterances in assembly minutes, and our\nteam (SKRA) participated in this subtask. As a first step for summarizing\nutterances, we created a new pre-trained sentence embedding model, i.e. the\nJapanese Political Sentence-BERT. With this model, we summarize utterances\nwithout labelled data. This paper describes our approach to solving the task\nand discusses its results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12156v1",
            "date": "2020-10-23 04:26:33+00:00",
            "title": "Attention Transfer Network for Aspect-level Sentiment Classification",
            "authors": [
                "Fei Zhao",
                "Zhen Wu",
                "Xinyu Dai"
            ],
            "abstract": "Aspect-level sentiment classification (ASC) aims to detect the sentiment\npolarity of a given opinion target in a sentence. In neural network-based\nmethods for ASC, most works employ the attention mechanism to capture the\ncorresponding sentiment words of the opinion target, then aggregate them as\nevidence to infer the sentiment of the target. However, aspect-level datasets\nare all relatively small-scale due to the complexity of annotation. Data\nscarcity causes the attention mechanism sometimes to fail to focus on the\ncorresponding sentiment words of the target, which finally weakens the\nperformance of neural models. To address the issue, we propose a novel\nAttention Transfer Network (ATN) in this paper, which can successfully exploit\nattention knowledge from resource-rich document-level sentiment classification\ndatasets to improve the attention capability of the aspect-level sentiment\nclassification task. In the ATN model, we design two different methods to\ntransfer attention knowledge and conduct experiments on two ASC benchmark\ndatasets. Extensive experimental results show that our methods consistently\noutperform state-of-the-art works. Further analysis also validates the\neffectiveness of ATN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12183v2",
            "date": "2020-10-23 06:28:25+00:00",
            "title": "Learning Similarity between Movie Characters and Its Potential Implications on Understanding Human Experiences",
            "authors": [
                "Zhilin Wang",
                "Weizhe Lin",
                "Xiaodong Wu"
            ],
            "abstract": "While many different aspects of human experiences have been studied by the\nNLP community, none has captured its full richness. We propose a new task to\ncapture this richness based on an unlikely setting: movie characters. We sought\nto capture theme-level similarities between movie characters that were\ncommunity-curated into 20,000 themes. By introducing a two-step approach that\nbalances performance and efficiency, we managed to achieve 9-27\\% improvement\nover recent paragraph-embedding based methods. Finally, we demonstrate how the\nthematic information learnt from movie characters can potentially be used to\nunderstand themes in the experience of people, as indicated on Reddit posts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12198v2",
            "date": "2020-10-23 07:12:52+00:00",
            "title": "Domain Divergences: a Survey and Empirical Analysis",
            "authors": [
                "Abhinav Ramesh Kashyap",
                "Devamanyu Hazarika",
                "Min-Yen Kan",
                "Roger Zimmermann"
            ],
            "abstract": "Domain divergence plays a significant role in estimating the performance of a\nmodel in new domains. While there is a significant literature on divergence\nmeasures, researchers find it hard to choose an appropriate divergence for a\ngiven NLP application. We address this shortcoming by both surveying the\nliterature and through an empirical study. We develop a taxonomy of divergence\nmeasures consisting of three classes -- Information-theoretic, Geometric, and\nHigher-order measures and identify the relationships between them. Further, to\nunderstand the common use-cases of these measures, we recognise three novel\napplications -- 1) Data Selection, 2) Learning Representation, and 3) Decisions\nin the Wild -- and use it to organise our literature. From this, we identify\nthat Information-theoretic measures are prevalent for 1) and 3), and\nHigher-order measures are more common for 2). To further help researchers\nchoose appropriate measures to predict drop in performance -- an important\naspect of Decisions in the Wild, we perform correlation analysis spanning 130\ndomain adaptation scenarios, 3 varied NLP tasks and 12 divergence measures\nidentified from our survey. To calculate these divergences, we consider the\ncurrent contextual word representations (CWR) and contrast with the older\ndistributed representations. We find that traditional measures over word\ndistributions still serve as strong baselines, while higher-order measures with\nCWR are effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12223v1",
            "date": "2020-10-23 08:13:39+00:00",
            "title": "Proof-theoretic aspects of NL$\u03bb$",
            "authors": [
                "Richard Moot"
            ],
            "abstract": "We present a proof-theoretic analysis of the logic NL$\\lambda$ (Barker \\&\nShan 2014, Barker 2019). We notably introduce a novel calculus of proof nets\nand prove it is sound and complete with respect to the sequent calculus for the\nlogic. We study decidability and complexity of the logic using this new\ncalculus, proving a new upper bound for complexity of the logic (showing it is\nin NP) and a new lower bound for the class of formal language generated by the\nformalism (mildly context-sensitive languages extended with a permutation\nclosure operation). Finally, thanks to this new calculus, we present a novel\ncomparison between NL$\\lambda$ and the hybrid type-logical grammars of Kubota\n\\& Levine (2020). We show there is an unexpected convergence of the natural\nlanguage analyses proposed in the two formalism. In addition to studying the\nproof-theoretic properties of NL$\\lambda$, we greatly extends its linguistic\ncoverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12251v1",
            "date": "2020-10-23 09:23:44+00:00",
            "title": "A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems",
            "authors": [
                "Sunghyun Park",
                "Han Li",
                "Ameen Patel",
                "Sidharth Mudgal",
                "Sungjin Lee",
                "Young-Bum Kim",
                "Spyros Matsoukas",
                "Ruhi Sarikaya"
            ],
            "abstract": "Natural Language Understanding (NLU) is an established component within a\nconversational AI or digital assistant system, and it is responsible for\nproducing semantic understanding of a user request. We propose a scalable and\nautomatic approach for improving NLU in a large-scale conversational AI system\nby leveraging implicit user feedback, with an insight that user interaction\ndata and dialog context have rich information embedded from which user\nsatisfaction and intention can be inferred. In particular, we propose a general\ndomain-agnostic framework for curating new supervision data for improving NLU\nfrom live production traffic. With an extensive set of experiments, we show the\nresults of applying the framework and improving NLU for a large-scale\nproduction system and show its impact across 10 domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12272v2",
            "date": "2020-10-23 10:00:46+00:00",
            "title": "Pre-training with Meta Learning for Chinese Word Segmentation",
            "authors": [
                "Zhen Ke",
                "Liang Shi",
                "Songtao Sun",
                "Erli Meng",
                "Bin Wang",
                "Xipeng Qiu"
            ],
            "abstract": "Recent researches show that pre-trained models (PTMs) are beneficial to\nChinese Word Segmentation (CWS). However, PTMs used in previous works usually\nadopt language modeling as pre-training tasks, lacking task-specific prior\nsegmentation knowledge and ignoring the discrepancy between pre-training tasks\nand downstream CWS tasks. In this paper, we propose a CWS-specific pre-trained\nmodel METASEG, which employs a unified architecture and incorporates meta\nlearning algorithm into a multi-criteria pre-training task. Empirical results\nshow that METASEG could utilize common prior segmentation knowledge from\ndifferent existing criteria and alleviate the discrepancy between pre-trained\nmodels and downstream CWS tasks. Besides, METASEG can achieve new\nstate-of-the-art performance on twelve widely-used CWS datasets and\nsignificantly improve model performance in low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12321v2",
            "date": "2020-10-23 11:57:33+00:00",
            "title": "BARThez: a Skilled Pretrained French Sequence-to-Sequence Model",
            "authors": [
                "Moussa Kamal Eddine",
                "Antoine J. -P. Tixier",
                "Michalis Vazirgiannis"
            ],
            "abstract": "Inductive transfer learning has taken the entire NLP field by storm, with\nmodels such as BERT and BART setting new state of the art on countless NLU\ntasks. However, most of the available models and research have been conducted\nfor English. In this work, we introduce BARThez, the first large-scale\npretrained seq2seq model for French. Being based on BART, BARThez is\nparticularly well-suited for generative tasks. We evaluate BARThez on five\ndiscriminative tasks from the FLUE benchmark and two generative tasks from a\nnovel summarization dataset, OrangeSum, that we created for this research. We\nshow BARThez to be very competitive with state-of-the-art BERT-based French\nlanguage models such as CamemBERT and FlauBERT. We also continue the\npretraining of a multilingual BART on BARThez' corpus, and show our resulting\nmodel, mBARThez, to significantly boost BARThez' generative performance. Code,\ndata and models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12401v1",
            "date": "2020-10-23 13:45:33+00:00",
            "title": "Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets",
            "authors": [
                "Gaurish Thakkar",
                "Marcis Pinnis"
            ],
            "abstract": "In this paper, we present various pre-training strategies that aid in\nim-proving the accuracy of the sentiment classification task. We, at first,\npre-trainlanguage representation models using these strategies and then\nfine-tune them onthe downstream task. Experimental results on a time-balanced\ntweet evaluation setshow the improvement over the previous technique. We\nachieve 76% accuracy forsentiment analysis on Latvian tweets, which is a\nsubstantial improvement over pre-vious work",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12405v2",
            "date": "2020-10-23 13:47:01+00:00",
            "title": "Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond",
            "authors": [
                "Xin Li",
                "Lidong Bing",
                "Wenxuan Zhang",
                "Zheng Li",
                "Wai Lam"
            ],
            "abstract": "Cross-lingual adaptation with multilingual pre-trained language models\n(mPTLMs) mainly consists of two lines of works: zero-shot approach and\ntranslation-based approach, which have been studied extensively on the\nsequence-level tasks. We further verify the efficacy of these cross-lingual\nadaptation approaches by evaluating their performances on more fine-grained\nsequence tagging tasks. After re-examining their strengths and drawbacks, we\npropose a novel framework to consolidate the zero-shot approach and the\ntranslation-based approach for better adaptation performance. Instead of simply\naugmenting the source data with the machine-translated data, we tailor-make a\nwarm-up mechanism to quickly update the mPTLMs with the gradients estimated on\na few translated data. Then, the adaptation approach is applied to the refined\nparameters and the cross-lingual transfer is performed in a warm-start way. The\nexperimental results on nine target languages demonstrate that our method is\nbeneficial to the cross-lingual adaptation of various sequence tagging tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12406v1",
            "date": "2020-10-23 13:53:31+00:00",
            "title": "UNER: Universal Named-Entity RecognitionFramework",
            "authors": [
                "Diego Alves",
                "Tin Kuculo",
                "Gabriel Amaral",
                "Gaurish Thakkar",
                "Marko Tadic"
            ],
            "abstract": "We introduce the Universal Named-Entity Recognition (UNER)framework, a\n4-level classification hierarchy, and the methodology that isbeing adopted to\ncreate the first multilingual UNER corpus: the SETimesparallel corpus annotated\nfor named-entities. First, the English SETimescorpus will be annotated using\nexisting tools and knowledge bases. Afterevaluating the resulting annotations\nthrough crowdsourcing campaigns,they will be propagated automatically to other\nlanguages within the SE-Times corpora. Finally, as an extrinsic evaluation, the\nUNER multilin-gual dataset will be used to train and test available NER tools.\nAs part offuture research directions, we aim to increase the number of\nlanguages inthe UNER corpus and to investigate possible ways of integrating\nUNERwith available knowledge graphs to improve named-entity recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12412v2",
            "date": "2020-10-23 14:02:32+00:00",
            "title": "SmBoP: Semi-autoregressive Bottom-up Semantic Parsing",
            "authors": [
                "Ohad Rubin",
                "Jonathan Berant"
            ],
            "abstract": "The de-facto standard decoding method for semantic parsing in recent years\nhas been to autoregressively decode the abstract syntax tree of the target\nprogram using a top-down depth-first traversal. In this work, we propose an\nalternative approach: a Semi-autoregressive Bottom-up Parser (SmBoP) that\nconstructs at decoding step $t$ the top-$K$ sub-trees of height $\\leq t$. Our\nparser enjoys several benefits compared to top-down autoregressive parsing.\nFrom an efficiency perspective, bottom-up parsing allows to decode all\nsub-trees of a certain height in parallel, leading to logarithmic runtime\ncomplexity rather than linear. From a modeling perspective, a bottom-up parser\nlearns representations for meaningful semantic sub-programs at each step,\nrather than for semantically-vacuous partial trees. We apply SmBoP on Spider, a\nchallenging zero-shot semantic parsing benchmark, and show that SmBoP leads to\na 2.2x speed-up in decoding time and a $\\sim$5x speed-up in training time,\ncompared to a semantic parser that uses autoregressive decoding. SmBoP obtains\n71.1 denotation accuracy on Spider, establishing a new state-of-the-art, and\n69.5 exact match, comparable to the 69.6 exact match of the autoregressive\nRAT-SQL+GraPPa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12428v1",
            "date": "2020-10-23 14:21:03+00:00",
            "title": "Evaluating Language Tools for Fifteen EU-official Under-resourced Languages",
            "authors": [
                "Diego Alves",
                "Gaurish Thakkar",
                "Marko Tadi\u0107"
            ],
            "abstract": "This article presents the results of the evaluation campaign of language\ntools available for fifteen EU-official under-resourced languages. The\nevaluation was conducted within the MSC ITN CLEOPATRA action that aims at\nbuilding the cross-lingual event-centric knowledge processing on top of the\napplication of linguistic processing chains (LPCs) for at least 24 EU-official\nlanguages. In this campaign, we concentrated on three existing NLP platforms\n(Stanford CoreNLP, NLP Cube, UDPipe) that all provide models for\nunder-resourced languages and in this first run we covered 15 under-resourced\nlanguages for which the models were available. We present the design of the\nevaluation campaign and present the results as well as discuss them. We\nconsidered the difference between reported and our tested results within a\nsingle percentage point as being within the limits of acceptable tolerance and\nthus consider this result as reproducible. However, for a number of languages,\nthe results are below what was reported in the literature, and in some cases,\nour testing results are even better than the ones reported previously.\nParticularly problematic was the evaluation of NERC systems. One of the reasons\nis the absence of universally or cross-lingually applicable named entities\nclassification scheme that would serve the NERC task in different languages\nanalogous to the Universal Dependency scheme in parsing task. To build such a\nscheme has become one of our the future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12433v1",
            "date": "2020-10-23 14:26:30+00:00",
            "title": "Natural Language Processing Chains Inside a Cross-lingual Event-Centric Knowledge Pipeline for European Union Under-resourced Languages",
            "authors": [
                "Diego Alves",
                "Gaurish Thakkar",
                "Marko Tadi\u0107"
            ],
            "abstract": "This article presents the strategy for developing a platform containing\nLanguage Processing Chains for European Union languages, consisting of\nTokenization to Parsing, also including Named Entity recognition andwith\naddition ofSentiment Analysis. These chains are part of the first step of an\nevent-centric knowledge processing pipeline whose aim is to process\nmultilingual media information about major events that can cause an impactin\nEurope and the rest of the world. Due to the differences in terms of\navailability of language resources for each language, we have built this\nstrategy in three steps, starting with processing chains for the well-resourced\nlanguages and finishing with the development of new modules for the\nunder-resourced ones. In order to classify all European Union official\nlanguages in terms of resources, we have analysed the size of annotated corpora\nas well as the existence of pre-trained models in mainstream Language\nProcessing tools, and we have combined this information with the proposed\nclassification published at META-NETwhitepaper series.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12472v2",
            "date": "2020-10-23 15:14:14+00:00",
            "title": "HateBERT: Retraining BERT for Abusive Language Detection in English",
            "authors": [
                "Tommaso Caselli",
                "Valerio Basile",
                "Jelena Mitrovi\u0107",
                "Michael Granitzer"
            ],
            "abstract": "In this paper, we introduce HateBERT, a re-trained BERT model for abusive\nlanguage detection in English. The model was trained on RAL-E, a large-scale\ndataset of Reddit comments in English from communities banned for being\noffensive, abusive, or hateful that we have collected and made available to the\npublic. We present the results of a detailed comparison between a general\npre-trained language model and the abuse-inclined version obtained by\nretraining with posts from the banned communities on three English datasets for\noffensive, abusive language and hate speech detection tasks. In all datasets,\nHateBERT outperforms the corresponding general BERT model. We also discuss a\nbattery of experiments comparing the portability of the generic pre-trained\nlanguage model and its corresponding abusive language-inclined counterpart\nacross the datasets, indicating that portability is affected by compatibility\nof the annotated phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12473v1",
            "date": "2020-10-23 15:16:10+00:00",
            "title": "Intrinsic Quality Assessment of Arguments",
            "authors": [
                "Henning Wachsmuth",
                "Till Werner"
            ],
            "abstract": "Several quality dimensions of natural language arguments have been\ninvestigated. Some are likely to be reflected in linguistic features (e.g., an\nargument's arrangement), whereas others depend on context (e.g., relevance) or\ntopic knowledge (e.g., acceptability). In this paper, we study the intrinsic\ncomputational assessment of 15 dimensions, i.e., only learning from an\nargument's text. In systematic experiments with eight feature types on an\nexisting corpus, we observe moderate but significant learning success for most\ndimensions. Rhetorical quality seems hardest to assess, and subjectivity\nfeatures turn out strong, although length bias in the corpus impedes full\nvalidity. We also find that human assessors differ more clearly to each other\nthan to our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12495v1",
            "date": "2020-10-23 15:55:15+00:00",
            "title": "Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries",
            "authors": [
                "Daniel Deutsch",
                "Dan Roth"
            ],
            "abstract": "Reference-based metrics such as ROUGE or BERTScore evaluate the content\nquality of a summary by comparing the summary to a reference. Ideally, this\ncomparison should measure the summary's information quality by calculating how\nmuch information the summaries have in common. In this work, we analyze the\ntoken alignments used by ROUGE and BERTScore to compare summaries and argue\nthat their scores largely cannot be interpreted as measuring information\noverlap, but rather the extent to which they discuss the same topics. Further,\nwe provide evidence that this result holds true for many other summarization\nevaluation metrics. The consequence of this result is that it means the\nsummarization community has not yet found a reliable automatic metric that\naligns with its research goal, to generate summaries with high-quality\ninformation. Then, we propose a simple and interpretable method of evaluating\nsummaries which does directly measure information overlap and demonstrate how\nit can be used to gain insights into model behavior that could not be provided\nby other methods alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12505v2",
            "date": "2020-10-23 16:13:06+00:00",
            "title": "Helping users discover perspectives: Enhancing opinion mining with joint topic models",
            "authors": [
                "Tim Draws",
                "Jody Liu",
                "Nava Tintarev"
            ],
            "abstract": "Support or opposition concerning a debated claim such as abortion should be\nlegal can have different underlying reasons, which we call perspectives. This\npaper explores how opinion mining can be enhanced with joint topic modeling, to\nidentify distinct perspectives within the topic, providing an informative\noverview from unstructured text. We evaluate four joint topic models (TAM, JST,\nVODUM, and LAM) in a user study assessing human understandability of the\nextracted perspectives. Based on the results, we conclude that joint topic\nmodels such as TAM can discover perspectives that align with human judgments.\nMoreover, our results suggest that users are not influenced by their\npre-existing stance on the topic of abortion when interpreting the output of\ntopic models.",
            "journal ref": "2020 International Conference on Data Mining Workshops (ICDMW)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12510v1",
            "date": "2020-10-23 16:22:05+00:00",
            "title": "Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures",
            "authors": [
                "Nafise Sadat Moosavi",
                "Marcel de Boer",
                "Prasetya Ajie Utama",
                "Iryna Gurevych"
            ],
            "abstract": "Existing NLP datasets contain various biases, and models tend to quickly\nlearn those biases, which in turn limits their robustness. Existing approaches\nto improve robustness against dataset biases mostly focus on changing the\ntraining objective so that models learn less from biased examples. Besides,\nthey mostly focus on addressing a specific bias, and while they improve the\nperformance on adversarial evaluation sets of the targeted bias, they may bias\nthe model in other ways, and therefore, hurt the overall robustness. In this\npaper, we propose to augment the input sentences in the training data with\ntheir corresponding predicate-argument structures, which provide a higher-level\nabstraction over different realizations of the same meaning and help the model\nto recognize important parts of sentences. We show that without targeting a\nspecific bias, our sentence augmentation improves the robustness of transformer\nmodels against multiple biases. In addition, we show that models can still be\nvulnerable to the lexical overlap bias, even when the training data does not\ncontain this bias, and that the sentence augmentation also improves the\nrobustness in this scenario. We will release our adversarial datasets to\nevaluate bias in such a scenario as well as our augmentation scripts at\nhttps://github.com/UKPLab/data-augmentation-for-robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12523v1",
            "date": "2020-10-23 16:45:06+00:00",
            "title": "Neural Passage Retrieval with Improved Negative Contrast",
            "authors": [
                "Jing Lu",
                "Gustavo Hernandez Abrego",
                "Ji Ma",
                "Jianmo Ni",
                "Yinfei Yang"
            ],
            "abstract": "In this paper we explore the effects of negative sampling in dual encoder\nmodels used to retrieve passages for automatic question answering. We explore\nfour negative sampling strategies that complement the straightforward random\nsampling of negatives, typically used to train dual encoder models. Out of the\nfour strategies, three are based on retrieval and one on heuristics. Our\nretrieval-based strategies are based on the semantic similarity and the lexical\noverlap between questions and passages. We train the dual encoder models in two\nstages: pre-training with synthetic data and fine tuning with domain-specific\ndata. We apply negative sampling to both stages. The approach is evaluated in\ntwo passage retrieval tasks. Even though it is not evident that there is one\nsingle sampling strategy that works best in all the tasks, it is clear that our\nstrategies contribute to improving the contrast between the response and all\nthe other passages. Furthermore, mixing the negatives from different strategies\nachieve performance on par with the best performing strategy in all tasks. Our\nresults establish a new state-of-the-art level of performance on two of the\nopen-domain question answering datasets that we evaluated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12527v3",
            "date": "2020-10-23 16:51:09+00:00",
            "title": "Retrieve, Read, Rerank, then Iterate: Answering Open-Domain Questions of Varying Reasoning Steps from Text",
            "authors": [
                "Peng Qi",
                "Haejun Lee",
                "Oghenetegiri \"TG\" Sido",
                "Christopher D. Manning"
            ],
            "abstract": "We develop a unified system to answer directly from text open-domain\nquestions that may require a varying number of retrieval steps. We employ a\nsingle multi-task transformer model to perform all the necessary subtasks --\nretrieving supporting facts, reranking them, and predicting the answer from all\nretrieved documents -- in an iterative fashion. We avoid making crucial\nassumptions as previous work that do not transfer well to real-world settings,\nincluding exploiting knowledge of the fixed number of retrieval steps required\nto answer each question or using structured metadata like knowledge bases or\nweb links that have limited availability. Instead, we design a system that\nwould answer open-domain questions on any text collection without prior\nknowledge of reasoning complexity. To emulate this setting, we construct a new\nbenchmark by combining existing one- and two-step datasets with a new\ncollection of 203 questions that require three Wikipedia pages to answer,\nunifying Wikipedia corpora versions in the process. We show that our model\ndemonstrates competitive performance on both existing benchmarks and this new\nbenchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12532v1",
            "date": "2020-10-23 17:00:26+00:00",
            "title": "GiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method",
            "authors": [
                "Nicole Peinelt",
                "Marek Rei",
                "Maria Liakata"
            ],
            "abstract": "Large pre-trained language models such as BERT have been the driving force\nbehind recent improvements across many NLP tasks. However, BERT is only trained\nto predict missing words - either behind masks or in the next sentence - and\nhas no knowledge of lexical, syntactic or semantic information beyond what it\npicks up through unsupervised pre-training. We propose a novel method to\nexplicitly inject linguistic knowledge in the form of word embeddings into any\nlayer of a pre-trained BERT. Our performance improvements on multiple semantic\nsimilarity datasets when injecting dependency-based and counter-fitted\nembeddings indicate that such information is beneficial and currently missing\nfrom the original model. Our qualitative analysis shows that counter-fitted\nembedding injection particularly helps with cases involving synonym pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12547v2",
            "date": "2020-10-23 17:14:41+00:00",
            "title": "Multilingual BERT Post-Pretraining Alignment",
            "authors": [
                "Lin Pan",
                "Chung-Wei Hang",
                "Haode Qi",
                "Abhishek Shah",
                "Saloni Potdar",
                "Mo Yu"
            ],
            "abstract": "We propose a simple method to align multilingual contextual embeddings as a\npost-pretraining step for improved zero-shot cross-lingual transferability of\nthe pretrained models. Using parallel data, our method aligns embeddings on the\nword level through the recently proposed Translation Language Modeling\nobjective as well as on the sentence level via contrastive learning and random\ninput shuffling. We also perform sentence-level code-switching with English\nwhen finetuning on downstream tasks. On XNLI, our best model (initialized from\nmBERT) improves over mBERT by 4.7% in the zero-shot setting and achieves\ncomparable result to XLM for translate-train while using less than 18% of the\nsame parallel data and 31% less model parameters. On MLQA, our model\noutperforms XLM-R_Base that has 57% more parameters than ours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12563v2",
            "date": "2020-10-23 17:47:06+00:00",
            "title": "Concealed Data Poisoning Attacks on NLP Models",
            "authors": [
                "Eric Wallace",
                "Tony Z. Zhao",
                "Shi Feng",
                "Sameer Singh"
            ],
            "abstract": "Adversarial attacks alter NLP model predictions by perturbing test-time\ninputs. However, it is much less understood whether, and how, predictions can\nbe manipulated with small, concealed changes to the training data. In this\nwork, we develop a new data poisoning attack that allows an adversary to\ncontrol model predictions whenever a desired trigger phrase is present in the\ninput. For instance, we insert 50 poison examples into a sentiment model's\ntraining set that causes the model to frequently predict Positive whenever the\ninput contains \"James Bond\". Crucially, we craft these poison examples using a\ngradient-based procedure so that they do not mention the trigger phrase. We\nalso apply our poison attack to language modeling (\"Apple iPhone\" triggers\nnegative generations) and machine translation (\"iced coffee\" mistranslated as\n\"hot coffee\"). We conclude by proposing three defenses that can mitigate our\nattack at some cost in prediction accuracy or extra human annotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12566v1",
            "date": "2020-10-23 17:53:11+00:00",
            "title": "DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries",
            "authors": [
                "Aditi Chaudhary",
                "Karthik Raman",
                "Krishna Srinivasan",
                "Jiecao Chen"
            ],
            "abstract": "Pre-trained multilingual language models such as mBERT have shown immense\ngains for several natural language processing (NLP) tasks, especially in the\nzero-shot cross-lingual setting. Most, if not all, of these pre-trained models\nrely on the masked-language modeling (MLM) objective as the key language\nlearning objective. The principle behind these approaches is that predicting\nthe masked words with the help of the surrounding text helps learn potent\ncontextualized representations. Despite the strong representation learning\ncapability enabled by MLM, we demonstrate an inherent limitation of MLM for\nmultilingual representation learning. In particular, by requiring the model to\npredict the language-specific token, the MLM objective disincentivizes learning\na language-agnostic representation -- which is a key goal of multilingual\npre-training. Therefore to encourage better cross-lingual representation\nlearning we propose the DICT-MLM method. DICT-MLM works by incentivizing the\nmodel to be able to predict not just the original masked word, but potentially\nany of its cross-lingual synonyms as well. Our empirical analysis on multiple\ndownstream tasks spanning 30+ languages, demonstrates the efficacy of the\nproposed approach and its ability to learn better multilingual representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12613v1",
            "date": "2020-10-23 18:57:47+00:00",
            "title": "Ranking Creative Language Characteristics in Small Data Scenarios",
            "authors": [
                "Julia Siekiera",
                "Marius K\u00f6ppel",
                "Edwin Simpson",
                "Kevin Stowe",
                "Iryna Gurevych",
                "Stefan Kramer"
            ],
            "abstract": "The ability to rank creative natural language provides an important general\ntool for downstream language understanding and generation. However, current\ndeep ranking models require substantial amounts of labeled data that are\ndifficult and expensive to obtain for different domains, languages and creative\ncharacteristics. A recent neural approach, the DirectRanker, promises to reduce\nthe amount of training data needed but its application to text isn't fully\nexplored. We therefore adapt the DirectRanker to provide a new deep model for\nranking creative language with small data. We compare DirectRanker with a\nBayesian approach, Gaussian process preference learning (GPPL), which has\npreviously been shown to work well with sparse data. Our experiments with\nsparse training data show that while the performance of standard neural ranking\napproaches collapses with small training datasets, DirectRanker remains\neffective. We find that combining DirectRanker with GPPL increases performance\nacross different settings by leveraging the complementary benefits of both\nmodels. Our combined approach outperforms the previous state-of-the-art on\nhumor and metaphor novelty tasks, increasing Spearman's $\\rho$ by 14% and 16%\non average.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12626v1",
            "date": "2020-10-23 19:16:59+00:00",
            "title": "Topic Modeling with Contextualized Word Representation Clusters",
            "authors": [
                "Laure Thompson",
                "David Mimno"
            ],
            "abstract": "Clustering token-level contextualized word representations produces output\nthat shares many similarities with topic models for English text collections.\nUnlike clusterings of vocabulary-level word embeddings, the resulting models\nmore naturally capture polysemy and can be used as a way of organizing\ndocuments. We evaluate token clusterings trained from several different output\nlayers of popular contextualized language models. We find that BERT and GPT-2\nproduce high quality clusterings, but RoBERTa does not. These cluster models\nare simple, reliable, and can perform as well as, if not better than, LDA topic\nmodels, maintaining high topic quality even when the number of topics is large\nrelative to the size of the local collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12627v1",
            "date": "2020-10-23 19:17:00+00:00",
            "title": "Anchor-based Bilingual Word Embeddings for Low-Resource Languages",
            "authors": [
                "Tobias Eder",
                "Viktor Hangya",
                "Alexander Fraser"
            ],
            "abstract": "Bilingual word embeddings (BWEs) are useful for many cross-lingual\napplications, such as bilingual lexicon induction (BLI) and cross-lingual\ntransfer learning. While recent methods have led to good quality BWEs for\ndifferent language pairs using only weak bilingual signals, they still rely on\nan abundance of monolingual training data in both languages for their\nperformance. This becomes a problem especially in the case of low resource\nlanguages where neither parallel bilingual corpora nor large monolingual\ntraining data are available. This paper proposes a new approach for building\nBWEs in which the vector space of the high resource source language is used as\na starting point for training an embedding space for the low resource target\nlanguage. By using the source vectors as anchors the vector spaces are\nautomatically aligned. We evaluate the resulting BWEs on BLI and show the\nproposed method outperforms previous approaches in the low-resource setting by\na large margin. We show strong results on the standard English-German test pair\n(using German to simulate low resource). We also show we can build useful BWEs\nfor English-Hiligaynon, a true low-resource language, where previous approaches\nfailed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12634v1",
            "date": "2020-10-23 19:36:57+00:00",
            "title": "Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL",
            "authors": [
                "Yusen Zhang",
                "Xiangyu Dong",
                "Shuaichen Chang",
                "Tao Yu",
                "Peng Shi",
                "Rui Zhang"
            ],
            "abstract": "Neural models have achieved significant results on the text-to-SQL task, in\nwhich most current work assumes all the input questions are legal and generates\na SQL query for any input. However, in the real scenario, users can input any\ntext that may not be able to be answered by a SQL query. In this work, we\npropose TriageSQL, the first cross-domain text-to-SQL question intention\nclassification benchmark that requires models to distinguish four types of\nunanswerable questions from answerable questions. The baseline RoBERTa model\nachieves a 60% F1 score on the test set, demonstrating the need for further\nimprovement on this task. Our dataset is available at\nhttps://github.com/chatc/TriageSQL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12643v1",
            "date": "2020-10-23 20:09:01+00:00",
            "title": "Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering",
            "authors": [
                "Arij Riabi",
                "Thomas Scialom",
                "Rachel Keraron",
                "Beno\u00eet Sagot",
                "Djam\u00e9 Seddah",
                "Jacopo Staiano"
            ],
            "abstract": "Coupled with the availability of large scale datasets, deep learning\narchitectures have enabled rapid progress on the Question Answering task.\nHowever, most of those datasets are in English, and the performances of\nstate-of-the-art multilingual models are significantly lower when evaluated on\nnon-English data. Due to high data collection costs, it is not realistic to\nobtain annotated data for each language one desires to support.\n  We propose a method to improve the Cross-lingual Question Answering\nperformance without requiring additional annotated data, leveraging Question\nGeneration models to produce synthetic samples in a cross-lingual fashion. We\nshow that the proposed method allows to significantly outperform the baselines\ntrained on English data only. We report a new state-of-the-art on four\nmultilingual datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12652v1",
            "date": "2020-10-23 20:31:37+00:00",
            "title": "Rapid Domain Adaptation for Machine Translation with Monolingual Data",
            "authors": [
                "Mahdis Mahdieh",
                "Mia Xu Chen",
                "Yuan Cao",
                "Orhan Firat"
            ],
            "abstract": "One challenge of machine translation is how to quickly adapt to unseen\ndomains in face of surging events like COVID-19, in which case timely and\naccurate translation of in-domain information into multiple languages is\ncritical but little parallel data is available yet. In this paper, we propose\nan approach that enables rapid domain adaptation from the perspective of\nunsupervised translation. Our proposed approach only requires in-domain\nmonolingual data and can be quickly applied to a preexisting translation system\ntrained on general domain, reaching significant gains on in-domain translation\nquality with little or no drop on general-domain. We also propose an effective\nprocedure of simultaneous adaptation for multiple domains and languages. To the\nbest of our knowledge, this is the first attempt that aims to address\nunsupervised multilingual domain adaptation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12658v1",
            "date": "2020-10-23 20:47:58+00:00",
            "title": "Generating Adequate Distractors for Multiple-Choice Questions",
            "authors": [
                "Cheng Zhang",
                "Yicheng Sun",
                "Hejia Chen",
                "Jie Wang"
            ],
            "abstract": "This paper presents a novel approach to automatic generation of adequate\ndistractors for a given question-answer pair (QAP) generated from a given\narticle to form an adequate multiple-choice question (MCQ). Our method is a\ncombination of part-of-speech tagging, named-entity tagging, semantic-role\nlabeling, regular expressions, domain knowledge bases, word embeddings, word\nedit distance, WordNet, and other algorithms. We use the US SAT (Scholastic\nAssessment Test) practice reading tests as a dataset to produce QAPs and\ngenerate three distractors for each QAP to form an MCQ. We show that, via\nexperiments and evaluations by human judges, each MCQ has at least one adequate\ndistractor and 84\\% of MCQs have three adequate distractors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12684v2",
            "date": "2020-10-23 22:02:40+00:00",
            "title": "Dynamic Contextualized Word Embeddings",
            "authors": [
                "Valentin Hofmann",
                "Janet B. Pierrehumbert",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Static word embeddings that represent words by a single vector cannot capture\nthe variability of word meaning in different linguistic and extralinguistic\ncontexts. Building on prior work on contextualized and dynamic word embeddings,\nwe introduce dynamic contextualized word embeddings that represent words as a\nfunction of both linguistic and extralinguistic context. Based on a pretrained\nlanguage model (PLM), dynamic contextualized word embeddings model time and\nsocial space jointly, which makes them attractive for a range of NLP tasks\ninvolving semantic variability. We highlight potential applications by means of\nqualitative and quantitative analyses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12688v2",
            "date": "2020-10-23 22:14:50+00:00",
            "title": "Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training",
            "authors": [
                "Oshin Agarwal",
                "Heming Ge",
                "Siamak Shakeri",
                "Rami Al-Rfou"
            ],
            "abstract": "Prior work on Data-To-Text Generation, the task of converting knowledge graph\n(KG) triples into natural text, focused on domain-specific benchmark datasets.\nIn this paper, however, we verbalize the entire English Wikidata KG, and\ndiscuss the unique challenges associated with a broad, open-domain, large-scale\nverbalization. We further show that verbalizing a comprehensive, encyclopedic\nKG like Wikidata can be used to integrate structured KGs and natural language\ncorpora. In contrast to the many architectures that have been developed to\nintegrate these two sources, our approach converts the KG into natural text,\nallowing it to be seamlessly integrated into existing language models. It\ncarries the further advantages of improved factual accuracy and reduced\ntoxicity in the resulting language model. We evaluate this approach by\naugmenting the retrieval corpus in a retrieval language model and showing\nsignificant improvements on the knowledge intensive tasks of open domain QA and\nthe LAMA knowledge probe.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12694v1",
            "date": "2020-10-23 22:38:18+00:00",
            "title": "AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization",
            "authors": [
                "Sayali Kulkarni",
                "Sheide Chammas",
                "Wan Zhu",
                "Fei Sha",
                "Eugene Ie"
            ],
            "abstract": "Summarization is the task of compressing source document(s) into coherent and\nsuccinct passages. This is a valuable tool to present users with concise and\naccurate sketch of the top ranked documents related to their queries.\nQuery-based multi-document summarization (qMDS) addresses this pervasive need,\nbut the research is severely limited due to lack of training and evaluation\ndatasets as existing single-document and multi-document summarization datasets\nare inadequate in form and scale. We propose a scalable approach called\nAQuaMuSe to automatically mine qMDS examples from question answering datasets\nand large document corpora. Our approach is unique in the sense that it can\ngeneral a dual dataset -- for extractive and abstractive summaries both. We\npublicly release a specific instance of an AQuaMuSe dataset with 5,519\nquery-based summaries, each associated with an average of 6 input documents\nselected from an index of 355M documents from Common Crawl. Extensive\nevaluation of the dataset along with baseline summarization model experiments\nare provided.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12699v2",
            "date": "2020-10-23 22:58:26+00:00",
            "title": "Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary",
            "authors": [
                "Stefan Gr\u00fcnewald",
                "Annemarie Friedrich",
                "Jonas Kuhn"
            ],
            "abstract": "The introduction of pre-trained transformer-based contextualized word\nembeddings has led to considerable improvements in the accuracy of graph-based\nparsers for frameworks such as Universal Dependencies (UD). However, previous\nworks differ in various dimensions, including their choice of pre-trained\nlanguage models and whether they use LSTM layers. With the aims of\ndisentangling the effects of these choices and identifying a simple yet widely\napplicable architecture, we introduce STEPS, a new modular graph-based\ndependency parser. Using STEPS, we perform a series of analyses on the UD\ncorpora of a diverse set of languages. We find that the choice of pre-trained\nembeddings has by far the greatest impact on parser performance and identify\nXLM-R as a robust choice across the languages in our study. Adding LSTM layers\nprovides no benefits. A multi-task training setup outputting additional UD\nfeatures may contort results. Taking these insights together, we propose a\nsimple but widely applicable parser architecture and configuration, achieving\nnew state-of-the-art results for 11 out of 12 diverse languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12707v3",
            "date": "2020-10-23 23:25:00+00:00",
            "title": "Learning to Recognize Dialect Features",
            "authors": [
                "Dorottya Demszky",
                "Devyani Sharma",
                "Jonathan H. Clark",
                "Vinodkumar Prabhakaran",
                "Jacob Eisenstein"
            ],
            "abstract": "Building NLP systems that serve everyone requires accounting for dialect\ndifferences. But dialects are not monolithic entities: rather, distinctions\nbetween and within dialects are captured by the presence, absence, and\nfrequency of dozens of dialect features in speech and text, such as the\ndeletion of the copula in \"He {} running\". In this paper, we introduce the task\nof dialect feature detection, and present two multitask learning approaches,\nboth based on pretrained transformers. For most dialects, large-scale annotated\ncorpora for these features are unavailable, making it difficult to train\nrecognizers. We train our models on a small number of minimal pairs, building\non how linguists typically define dialect features. Evaluation on a test set of\n22 dialect features of Indian English demonstrates that these models learn to\nrecognize many features with high accuracy, and that a few minimal pairs can be\nas effective for training as thousands of labeled examples. We also demonstrate\nthe downstream applicability of dialect feature detection both as a measure of\ndialect density and as a dialect classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12712v1",
            "date": "2020-10-23 23:41:51+00:00",
            "title": "A Caption Is Worth A Thousand Images: Investigating Image Captions for Multimodal Named Entity Recognition",
            "authors": [
                "Shuguang Chen",
                "Gustavo Aguilar",
                "Leonardo Neves",
                "Thamar Solorio"
            ],
            "abstract": "Multimodal named entity recognition (MNER) requires to bridge the gap between\nlanguage understanding and visual context. Due to advances in natural language\nprocessing (NLP) and computer vision (CV), many neural techniques have been\nproposed to incorporate images into the NER task. In this work, we conduct a\ndetailed analysis of current state-of-the-art fusion techniques for MNER and\ndescribe scenarios where adding information from the image does not always\nresult in boosts in performance. We also study the use of captions as a way to\nenrich the context for MNER. We provide extensive empirical analysis and an\nablation study on three datasets from popular social platforms to expose the\nsituations where the approach is beneficial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12719v1",
            "date": "2020-10-24 00:14:04+00:00",
            "title": "Word2vec Conjecture and A Limitative Result",
            "authors": [
                "Falcon Z. Dai"
            ],
            "abstract": "Being inspired by the success of \\texttt{word2vec}\n\\citep{mikolov2013distributed} in capturing analogies, we study the conjecture\nthat analogical relations can be represented by vector spaces. Unlike many\nprevious works that focus on the distributional semantic aspect of\n\\texttt{word2vec}, we study the purely \\emph{representational} question: can\n\\emph{all} semantic word-word relations be represented by differences (or\ndirections) of vectors? We call this the word2vec conjecture and point out some\nof its desirable implications. However, we will exhibit a class of relations\nthat cannot be represented in this way, thus falsifying the conjecture and\nestablishing a limitative result for the representability of semantic relations\nby vector spaces over fields of characteristic 0, e.g., real or complex\nnumbers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12723v1",
            "date": "2020-10-24 00:27:44+00:00",
            "title": "Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation",
            "authors": [
                "Yuning Mao",
                "Xiang Ren",
                "Heng Ji",
                "Jiawei Han"
            ],
            "abstract": "Summaries generated by abstractive summarization are supposed to only contain\nstatements entailed by the source documents. However, state-of-the-art\nabstractive methods are still prone to hallucinate content inconsistent with\nthe source documents. In this paper, we propose constrained abstractive\nsummarization (CAS), a general setup that preserves the factual consistency of\nabstractive summarization by specifying tokens as constraints that must be\npresent in the summary. We explore the feasibility of using lexically\nconstrained decoding, a technique applicable to any abstractive method with\nbeam search decoding, to fulfill CAS and conduct experiments in two scenarios:\n(1) Standard summarization without human involvement, where keyphrase\nextraction is used to extract constraints from source documents; (2)\nInteractive summarization with human feedback, which is simulated by taking\nmissing tokens in the reference summaries as constraints. Automatic and human\nevaluations on two benchmark datasets demonstrate that CAS improves the quality\nof abstractive summaries, especially on factual consistency. In particular, we\nobserve up to 11.2 ROUGE-2 gains when several ground-truth tokens are used as\nconstraints in the interactive summarization scenario.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12725v2",
            "date": "2020-10-24 00:38:27+00:00",
            "title": "Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?",
            "authors": [
                "Peter Shaw",
                "Ming-Wei Chang",
                "Panupong Pasupat",
                "Kristina Toutanova"
            ],
            "abstract": "Sequence-to-sequence models excel at handling natural language variation, but\nhave been shown to struggle with out-of-distribution compositional\ngeneralization. This has motivated new specialized architectures with stronger\ncompositional biases, but most of these approaches have only been evaluated on\nsynthetically-generated datasets, which are not representative of natural\nlanguage variation. In this work we ask: can we develop a semantic parsing\napproach that handles both natural language variation and compositional\ngeneralization? To better assess this capability, we propose new train and test\nsplits of non-synthetic datasets. We demonstrate that strong existing\napproaches do not perform well across a broad set of evaluations. We also\npropose NQG-T5, a hybrid model that combines a high-precision grammar-based\napproach with a pre-trained sequence-to-sequence model. It outperforms existing\napproaches across several compositional generalization challenges on\nnon-synthetic data, while also being competitive with the state-of-the-art on\nstandard evaluations. While still far from solving this problem, our study\nhighlights the importance of diverse evaluations and the open challenge of\nhandling both compositional generalization and natural language variation in\nsemantic parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12729v1",
            "date": "2020-10-24 01:03:51+00:00",
            "title": "ANLIzing the Adversarial Natural Language Inference Dataset",
            "authors": [
                "Adina Williams",
                "Tristan Thrush",
                "Douwe Kiela"
            ],
            "abstract": "We perform an in-depth error analysis of Adversarial NLI (ANLI), a recently\nintroduced large-scale human-and-model-in-the-loop natural language inference\ndataset collected over multiple rounds. We propose a fine-grained annotation\nscheme of the different aspects of inference that are responsible for the gold\nclassification labels, and use it to hand-code all three of the ANLI\ndevelopment sets. We use these annotations to answer a variety of interesting\nquestions: which inference types are most common, which models have the highest\nperformance on each reasoning type, and which types are the most challenging\nfor state of-the-art models? We hope that our annotations will enable more\nfine-grained evaluation of models trained on ANLI, provide us with a deeper\nunderstanding of where models fail and succeed, and help us determine how to\ntrain better models in future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12730v2",
            "date": "2020-10-24 01:08:28+00:00",
            "title": "Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality",
            "authors": [
                "Gustavo Aguilar",
                "Bryan McCann",
                "Tong Niu",
                "Nazneen Rajani",
                "Nitish Keskar",
                "Thamar Solorio"
            ],
            "abstract": "Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword\ntokenization process of language models. BPE provides multiple benefits, such\nas handling the out-of-vocabulary problem and reducing vocabulary sparsity.\nHowever, this process is defined from the pre-training data statistics, making\nthe tokenization on different domains susceptible to infrequent spelling\nsequences (e.g., misspellings as in social media or character-level adversarial\nattacks). On the other hand, though robust to misspellings, pure\ncharacter-level models often lead to unreasonably large sequences and make it\nharder for the model to learn meaningful contiguous characters. We propose a\ncharacter-based subword module (char2subword) that learns the subword embedding\ntable in pre-trained models like BERT to alleviate these challenges. Our\nchar2subword module builds representations from characters out of the subword\nvocabulary, and it can be used as a drop-in replacement of the subword\nembedding table. The module is robust to character-level alterations such as\nmisspellings, word inflection, casing, and punctuation. We integrate it further\nwith BERT through pre-training while keeping BERT transformer parameters fixed.\nWe show our method's effectiveness by outperforming mBERT on the linguistic\ncode-switching evaluation (LinCE) benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12741v1",
            "date": "2020-10-24 01:59:49+00:00",
            "title": "An Evaluation Protocol for Generative Conversational Systems",
            "authors": [
                "Seolhwa Lee",
                "Heuiseok Lim",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "There is a multitude of novel generative models for open-domain\nconversational systems; however, there is no systematic evaluation of different\nsystems. Systematic comparisons require consistency in experimental design,\nevaluation sets, conversational systems and their outputs, and statistical\nanalysis. We lay out a protocol for the evaluation of conversational models\nusing head-to-head pairwise comparison. We analyze ten recent models that claim\nstate-of-the-art performance using a paired head-to-head performance\n(win-loss-tie) on five evaluation datasets. Our findings show that DialoGPT and\nBlender are superior systems using Bradley-Terry model and TrueSkill ranking\nmethods. These findings demonstrate the feasibility of our protocol to evaluate\nconversational agents and evaluation sets. Finally, we make all code and\nevaluations publicly available for researchers to compare their model to other\nstate-of-the-art dialog models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12742v2",
            "date": "2020-10-24 02:02:58+00:00",
            "title": "Text Style Transfer: A Review and Experimental Evaluation",
            "authors": [
                "Zhiqiang Hu",
                "Roy Ka-Wei Lee",
                "Charu C. Aggarwal",
                "Aston Zhang"
            ],
            "abstract": "The stylistic properties of text have intrigued computational linguistics\nresearchers in recent years. Specifically, researchers have investigated the\nText Style Transfer (TST) task, which aims to change the stylistic properties\nof the text while retaining its style independent content. Over the last few\nyears, many novel TST algorithms have been developed, while the industry has\nleveraged these algorithms to enable exciting TST applications. The field of\nTST research has burgeoned because of this symbiosis. This article aims to\nprovide a comprehensive review of recent research efforts on text style\ntransfer. More concretely, we create a taxonomy to organize the TST models and\nprovide a comprehensive summary of the state of the art. We review the existing\nevaluation methodologies for TST tasks and conduct a large-scale\nreproducibility study where we experimentally benchmark 19 state-of-the-art TST\nalgorithms on two publicly available datasets. Finally, we expand on current\ntrends and provide new perspectives on the new and exciting developments in the\nTST field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12753v2",
            "date": "2020-10-24 03:12:27+00:00",
            "title": "Temporal Reasoning on Implicit Events from Distant Supervision",
            "authors": [
                "Ben Zhou",
                "Kyle Richardson",
                "Qiang Ning",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Dan Roth"
            ],
            "abstract": "We propose TRACIE, a novel temporal reasoning dataset that evaluates the\ndegree to which systems understand implicit events -- events that are not\nmentioned explicitly in natural language text but can be inferred from it. This\nintroduces a new challenge in temporal reasoning research, where prior work has\nfocused on explicitly mentioned events. Human readers can infer implicit events\nvia commonsense reasoning, resulting in a more comprehensive understanding of\nthe situation and, consequently, better reasoning about time. We find, however,\nthat state-of-the-art models struggle when predicting temporal relationships\nbetween implicit and explicit events. To address this, we propose a\nneuro-symbolic temporal reasoning model, SYMTIME, which exploits distant\nsupervision signals from large-scale text and uses temporal rules to combine\nstart times and durations to infer end times. SYMTIME outperforms strong\nbaseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training\nsetting. Our approach also generalizes to other temporal reasoning tasks, as\nevidenced by a gain of 1%-9% on MATRES, an explicit event benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12757v2",
            "date": "2020-10-24 03:22:43+00:00",
            "title": "Adding Chit-Chat to Enhance Task-Oriented Dialogues",
            "authors": [
                "Kai Sun",
                "Seungwhan Moon",
                "Paul Crook",
                "Stephen Roller",
                "Becka Silvert",
                "Bing Liu",
                "Zhiguang Wang",
                "Honglei Liu",
                "Eunjoon Cho",
                "Claire Cardie"
            ],
            "abstract": "Existing dialogue corpora and models are typically designed under two\ndisjoint motives: while task-oriented systems focus on achieving functional\ngoals (e.g., booking hotels), open-domain chatbots aim at making socially\nengaging conversations. In this work, we propose to integrate both types of\nsystems by Adding Chit-Chat to ENhance Task-ORiented dialogues (ACCENTOR), with\nthe goal of making virtual assistant conversations more engaging and\ninteractive. Specifically, we propose a Human <-> AI collaborative data\ncollection approach for generating diverse chit-chat responses to augment\ntask-oriented dialogues with minimal annotation effort. We then present our new\nchit-chat-based annotations to 23.8K dialogues from two popular task-oriented\ndatasets (Schema-Guided Dialogue and MultiWOZ 2.1) and demonstrate their\nadvantage over the originals via human evaluation. Lastly, we propose three new\nmodels for adding chit-chat to task-oriented dialogues, explicitly trained to\npredict user goals and to generate contextually relevant chit-chat responses.\nAutomatic and human evaluations show that, compared with the state-of-the-art\ntask-oriented baseline, our models can code-switch between task and chit-chat\nto be more engaging, interesting, knowledgeable, and humanlike, while\nmaintaining competitive task performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12758v1",
            "date": "2020-10-24 03:23:14+00:00",
            "title": "NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions",
            "authors": [
                "Zhiyu Chen",
                "Honglei Liu",
                "Hu Xu",
                "Seungwhan Moon",
                "Hao Zhou",
                "Bing Liu"
            ],
            "abstract": "Existing conversational systems are mostly agent-centric, which assumes the\nuser utterances would closely follow the system ontology (for NLU or dialogue\nstate tracking). However, in real-world scenarios, it is highly desirable that\nthe users can speak freely in their own way. It is extremely hard, if not\nimpossible, for the users to adapt to the unknown system ontology. In this\nwork, we attempt to build a user-centric dialogue system. As there is no clean\nmapping for a user's free form utterance to an ontology, we first model the\nuser preferences as estimated distributions over the system ontology and map\nthe users' utterances to such distributions. Learning such a mapping poses new\nchallenges on reasoning over existing knowledge, ranging from factoid\nknowledge, commonsense knowledge to the users' own situations. To this end, we\nbuild a new dataset named NUANCED that focuses on such realistic settings for\nconversational recommendation. Collected via dialogue simulation and\nparaphrasing, NUANCED contains 5.1k dialogues, 26k turns of high-quality user\nresponses. We conduct experiments, showing both the usefulness and challenges\nof our problem setting. We believe NUANCED can serve as a valuable resource to\npush existing research from the agent-centric system to the user-centric\nsystem. The code and data will be made publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12762v2",
            "date": "2020-10-24 03:40:56+00:00",
            "title": "Measuring Association Between Labels and Free-Text Rationales",
            "authors": [
                "Sarah Wiegreffe",
                "Ana Marasovi\u0107",
                "Noah A. Smith"
            ],
            "abstract": "In interpretable NLP, we require faithful rationales that reflect the model's\ndecision-making process for an explained instance. While prior work focuses on\nextractive rationales (a subset of the input words), we investigate their\nless-studied counterpart: free-text natural language rationales. We demonstrate\nthat pipelines, existing models for faithful extractive rationalization on\ninformation-extraction style tasks, do not extend as reliably to \"reasoning\"\ntasks requiring free-text rationales. We turn to models that jointly predict\nand rationalize, a class of widely used high-performance models for free-text\nrationalization whose faithfulness is not yet established. We define\nlabel-rationale association as a necessary property for faithfulness: the\ninternal mechanisms of the model producing the label and the rationale must be\nmeaningfully correlated. We propose two measurements to test this property:\nrobustness equivalence and feature importance agreement. We find that\nstate-of-the-art T5-based joint models exhibit both properties for\nrationalizing commonsense question-answering and natural language inference,\nindicating their potential for producing faithful free-text rationales.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12770v3",
            "date": "2020-10-24 04:10:32+00:00",
            "title": "Conversational Semantic Parsing for Dialog State Tracking",
            "authors": [
                "Jianpeng Cheng",
                "Devang Agrawal",
                "Hector Martinez Alonso",
                "Shruti Bhargava",
                "Joris Driesen",
                "Federico Flego",
                "Shaona Ghosh",
                "Dain Kaplan",
                "Dimitri Kartsaklis",
                "Lin Li",
                "Dhivya Piraviperumal",
                "Jason D Williams",
                "Hong Yu",
                "Diarmuid O Seaghdha",
                "Anders Johannsen"
            ],
            "abstract": "We consider a new perspective on dialog state tracking (DST), the task of\nestimating a user's goal through the course of a dialog. By formulating DST as\na semantic parsing task over hierarchical representations, we can incorporate\nsemantic compositionality, cross-domain knowledge sharing and co-reference. We\npresent TreeDST, a dataset of 27k conversations annotated with tree-structured\ndialog states and system acts. We describe an encoder-decoder framework for DST\nwith hierarchical representations, which leads to 20% improvement over\nstate-of-the-art DST approaches that operate on a flat meaning space of\nslot-value pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12771v2",
            "date": "2020-10-24 04:30:02+00:00",
            "title": "On Learning Text Style Transfer with Direct Rewards",
            "authors": [
                "Yixin Liu",
                "Graham Neubig",
                "John Wieting"
            ],
            "abstract": "In most cases, the lack of parallel corpora makes it impossible to directly\ntrain supervised models for the text style transfer task. In this paper, we\nexplore training algorithms that instead optimize reward functions that\nexplicitly consider different aspects of the style-transferred outputs. In\nparticular, we leverage semantic similarity metrics originally used for\nfine-tuning neural machine translation models to explicitly assess the\npreservation of content between system outputs and input texts. We also\ninvestigate the potential weaknesses of the existing automatic metrics and\npropose efficient strategies of using these metrics for training. The\nexperimental results show that our model provides significant gains in both\nautomatic and human evaluation over strong baselines, indicating the\neffectiveness of our proposed methods and training strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12776v1",
            "date": "2020-10-24 04:41:30+00:00",
            "title": "Improved Synthetic Training for Reading Comprehension",
            "authors": [
                "Yanda Chen",
                "Md Arafat Sultan",
                "Vittorio Castelli"
            ],
            "abstract": "Automatically generated synthetic training examples have been shown to\nimprove performance in machine reading comprehension (MRC). Compared to human\nannotated gold standard data, synthetic training data has unique properties,\nsuch as high availability at the possible expense of quality. In view of such\ndifferences, in this paper, we explore novel applications of synthetic examples\nto MRC. Our proposed pre-training and knowledge distillation strategies show\nsignificant improvements over existing methods. In a particularly surprising\ndiscovery, we observe that synthetic distillation often yields students that\ncan outperform the teacher model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12779v1",
            "date": "2020-10-24 04:51:47+00:00",
            "title": "Fair Hate Speech Detection through Evaluation of Social Group Counterfactuals",
            "authors": [
                "Aida Mostafazadeh Davani",
                "Ali Omrani",
                "Brendan Kennedy",
                "Mohammad Atari",
                "Xiang Ren",
                "Morteza Dehghani"
            ],
            "abstract": "Approaches for mitigating bias in supervised models are designed to reduce\nmodels' dependence on specific sensitive features of the input data, e.g.,\nmentioned social groups. However, in the case of hate speech detection, it is\nnot always desirable to equalize the effects of social groups because of their\nessential role in distinguishing outgroup-derogatory hate, such that particular\ntypes of hateful rhetoric carry the intended meaning only when contextualized\naround certain social group tokens. Counterfactual token fairness for a\nmentioned social group evaluates the model's predictions as to whether they are\nthe same for (a) the actual sentence and (b) a counterfactual instance, which\nis generated by changing the mentioned social group in the sentence. Our\napproach assures robust model predictions for counterfactuals that imply\nsimilar meaning as the actual sentence. To quantify the similarity of a\nsentence and its counterfactual, we compare their likelihood score calculated\nby generative language models. By equalizing model behaviors on each sentence\nand its counterfactuals, we mitigate bias in the proposed model while\npreserving the overall classification performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12784v1",
            "date": "2020-10-24 05:06:29+00:00",
            "title": "Clustering Contextualized Representations of Text for Unsupervised Syntax Induction",
            "authors": [
                "Vikram Gupta",
                "Haoyue Shi",
                "Kevin Gimpel",
                "Mrinmaya Sachan"
            ],
            "abstract": "We explore clustering of contextualized text representations for two\nunsupervised syntax induction tasks: part of speech induction (POSI) and\nconstituency labelling (CoLab). We propose a deep embedded clustering approach\nwhich jointly transforms these representations into a lower dimension cluster\nfriendly space and clusters them. We further enhance these representations by\naugmenting them with task-specific representations. We also explore the\neffectiveness of multilingual representations for different tasks and\nlanguages. With this work, we establish the first strong baselines for\nunsupervised syntax induction using contextualized text representations. We\nreport competitive performance on 45-tag POSI, state-of-the-art performance on\n12-tag POSI across 10 languages, and competitive results on CoLab.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12786v2",
            "date": "2020-10-24 05:16:36+00:00",
            "title": "Measuring the `I don't know' Problem through the Lens of Gricean Quantity",
            "authors": [
                "Huda Khayrallah",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "We consider the intrinsic evaluation of neural generative dialog models\nthrough the lens of Grice's Maxims of Conversation (1975). Based on the maxim\nof Quantity (be informative), we propose Relative Utterance Quantity (RUQ) to\ndiagnose the `I don't know' problem, in which a dialog system produces generic\nresponses. The linguistically motivated RUQ diagnostic compares the model score\nof a generic response to that of the reference response. We find that for\nreasonable baseline models, `I don't know' is preferred over the reference the\nmajority of the time, but this can be reduced to less than 5% with\nhyperparameter tuning. RUQ allows for the direct analysis of the `I don't know'\nproblem, which has been addressed but not analyzed by prior work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12787v3",
            "date": "2020-10-24 05:28:16+00:00",
            "title": "Document-level Event Extraction with Efficient End-to-end Learning of Cross-event Dependencies",
            "authors": [
                "Kung-Hsiang Huang",
                "Nanyun Peng"
            ],
            "abstract": "Fully understanding narratives often requires identifying events in the\ncontext of whole documents and modeling the event relations. However,\ndocument-level event extraction is a challenging task as it requires the\nextraction of event and entity coreference, and capturing arguments that span\nacross different sentences. Existing works on event extraction usually confine\non extracting events from single sentences, which fail to capture the\nrelationships between the event mentions at the scale of a document, as well as\nthe event arguments that appear in a different sentence than the event trigger.\nIn this paper, we propose an end-to-end model leveraging Deep Value Networks\n(DVN), a structured prediction algorithm, to efficiently capture cross-event\ndependencies for document-level event extraction. Experimental results show\nthat our approach achieves comparable performance to CRF-based models on ACE05,\nwhile enjoys significantly higher computational efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12789v3",
            "date": "2020-10-24 05:40:47+00:00",
            "title": "New Approaches for Natural Language Understanding based on the Idea that Natural Language encodes both Information and its Processing Procedures",
            "authors": [
                "Limin Zhang"
            ],
            "abstract": "We must recognize that natural language is a way of information encoding, and\nit encodes not only the information but also the procedures for how information\nis processed. To understand natural language, the same as we conceive and\ndesign computer languages, the first step is to separate information (or data)\nand the processing procedures of information (or data). In natural language,\nsome processing procedures of data are encoded directly as the structure chunk\nand the pointer chunk (this paper has reclassified lexical chunks as the data\nchunk, structure chunk, and the pointer chunk); some processing procedures of\ndata imply in sentences structures; some requests of processing procedures are\nexpressed by information senders and processed by information receivers. For\nthe data parts, the classification encoding system of attribute information and\nthe information organization architecture (including constitutional structures\nof information sets and the hierarchy between the information sets) were\ndiscussed. In section 2, the theoretical part elaborated in section 2 has been\nverified in examples and proofed that the studies in this paper have achieved\nthe goal of enabling machines to understand the information conveyed in the\ndialogue. In section 4, the author summarizes the basic conditions of\n\"Understanding\", rethinks what \"Understanding\" is and how to proceed. The study\nin this paper provides a practical, theoretical basis and research methods for\nNLU. It also can be applied in large-scale and multi-type information\nprocessing in the artificial intelligence (AI) area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12795v1",
            "date": "2020-10-24 06:17:35+00:00",
            "title": "CaM-Gen:Causally-aware Metric-guided Text Generation",
            "authors": [
                "Navita Goyal",
                "Roodram Paneri",
                "Ayush Agarwal",
                "Udit Kalani",
                "Abhilasha Sancheti",
                "Niyati Chhaya"
            ],
            "abstract": "Content is created for a well-defined purpose, often described by a metric or\na signal represented in the form of structured information. The relationship\nbetween the metrics or the goal of a target content and the content itself are\nnon-trivial. While large scale language models show promising text generation\ncapabilities, guiding and informing the generated text with external metrics is\nchallenging. These metrics and the content tend to have inherent relationships\nand not all of them may directly impact the content. We introduce a CaM-Gen:\nCausally-aware Generative Networks guided by user-defined input metrics\nincorporating the causal relationships between the metric and the content\nfeatures. We leverage causal inference techniques to identify the causally\nsignificant aspects of text that leads to the target metric and then explicitly\nguide the generative model towards these by a feedback mechanism. We propose\nthis mechanism for variational autoencoder-based and transformer-based\ngenerative models. The proposed models beat baselines in terms of the target\nmetric accuracy while maintaining the fluency and the language quality of the\ngenerated text. To the best of our knowledge, this is one of the early attempts\nat incorporating a metric-guide using causal inference towards controlled\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12808v1",
            "date": "2020-10-24 06:55:52+00:00",
            "title": "Paired Representation Learning for Event and Entity Coreference",
            "authors": [
                "Xiaodong Yu",
                "Wenpeng Yin",
                "Dan Roth"
            ],
            "abstract": "Co-reference of Events and of Entities are commonly formulated as binary\nclassification problems, given a pair of events or entities as input. Earlier\nwork addressed the main challenge in these problems -- the representation of\neach element in the input pair by: (i) modelling the representation of one\nelement (event or entity) without considering the other element in the pair;\n(ii) encoding all attributes of one element (e.g., arguments of an event) into\na single non-interpretable vector, thus losing the ability to compare\ncross-element attributes. In this work we propose paired representation\nlearning (PairedRL) for coreference resolution. Given a pair of elements\n(Events or Entities) our model treats the pair's sentences as a single sequence\nso that each element in the pair learns its representation by encoding its own\ncontext as well the other element's context. In addition, when representing\nevents, PairedRL is structured in that it represents the event's arguments to\nfacilitate their individual contribution to the final prediction. As we show,\nin both (within-document & cross-document) event and entity coreference\nbenchmarks, our unified approach, PairedRL, outperforms prior state of the art\nsystems with a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12812v2",
            "date": "2020-10-24 07:14:01+00:00",
            "title": "A Frustratingly Easy Approach for Entity and Relation Extraction",
            "authors": [
                "Zexuan Zhong",
                "Danqi Chen"
            ],
            "abstract": "End-to-end relation extraction aims to identify named entities and extract\nrelations between them. Most recent work models these two subtasks jointly,\neither by casting them in one structured prediction framework, or performing\nmulti-task learning through shared representations. In this work, we present a\nsimple pipelined approach for entity and relation extraction, and establish the\nnew state-of-the-art on standard benchmarks (ACE04, ACE05 and SciERC),\nobtaining a 1.7%-2.8% absolute improvement in relation F1 over previous joint\nmodels with the same pre-trained encoders. Our approach essentially builds on\ntwo independent encoders and merely uses the entity model to construct the\ninput for the relation model. Through a series of careful examinations, we\nvalidate the importance of learning distinct contextual representations for\nentities and relations, fusing entity information early in the relation model,\nand incorporating global context. Finally, we also present an efficient\napproximation to our approach which requires only one pass of both entity and\nrelation encoders at inference time, achieving an 8-16$\\times$ speedup with a\nslight reduction in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12813v2",
            "date": "2020-10-24 07:16:21+00:00",
            "title": "Constructing Taxonomies from Pretrained Language Models",
            "authors": [
                "Catherine Chen",
                "Kevin Lin",
                "Dan Klein"
            ],
            "abstract": "We present a method for constructing taxonomic trees (e.g., WordNet) using\npretrained language models. Our approach is composed of two modules, one that\npredicts parenthood relations and another that reconciles those predictions\ninto trees. The parenthood prediction module produces likelihood scores for\neach potential parent-child pair, creating a graph of parent-child relation\nscores. The tree reconciliation module treats the task as a graph optimization\nproblem and outputs the maximum spanning tree of this graph. We train our model\non subtrees sampled from WordNet, and test on non-overlapping WordNet subtrees.\nWe show that incorporating web-retrieved glosses can further improve\nperformance. On the task of constructing subtrees of English WordNet, the model\nachieves 66.7 ancestor F1, a 20.0% relative increase over the previous best\npublished result on this task. In addition, we convert the original English\ndataset into nine other languages using Open Multilingual WordNet and extend\nour results across these languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12820v2",
            "date": "2020-10-24 07:37:49+00:00",
            "title": "\"Nice Try, Kiddo\": Investigating Ad Hominems in Dialogue Responses",
            "authors": [
                "Emily Sheng",
                "Kai-Wei Chang",
                "Premkumar Natarajan",
                "Nanyun Peng"
            ],
            "abstract": "Ad hominem attacks are those that target some feature of a person's character\ninstead of the position the person is maintaining. These attacks are harmful\nbecause they propagate implicit biases and diminish a person's credibility.\nSince dialogue systems respond directly to user input, it is important to study\nad hominems in dialogue responses. To this end, we propose categories of ad\nhominems, compose an annotated dataset, and build a classifier to analyze human\nand dialogue system responses to English Twitter posts. We specifically compare\nresponses to Twitter topics about marginalized communities (#BlackLivesMatter,\n#MeToo) versus other topics (#Vegan, #WFH), because the abusive language of ad\nhominems could further amplify the skew of power away from marginalized\npopulations. Furthermore, we propose a constrained decoding technique that uses\nsalient $n$-gram similarity as a soft constraint for top-$k$ sampling to reduce\nthe amount of ad hominems generated. Our results indicate that 1) responses\nfrom both humans and DialoGPT contain more ad hominems for discussions around\nmarginalized communities, 2) different quantities of ad hominems in the\ntraining data can influence the likelihood of generating ad hominems, and 3) we\ncan use constrained decoding techniques to reduce ad hominems in generated\ndialogue responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12825v2",
            "date": "2020-10-24 07:55:32+00:00",
            "title": "Cross-neutralising: Probing for joint encoding of linguistic information in multilingual models",
            "authors": [
                "Rochelle Choenni",
                "Ekaterina Shutova"
            ],
            "abstract": "Multilingual sentence encoders are widely used to transfer NLP models across\nlanguages. The success of this transfer is, however, dependent on the model's\nability to encode the patterns of cross-lingual similarity and variation. Yet,\nlittle is known as to how these models are able to do this. We propose a simple\nmethod to study how relationships between languages are encoded in two\nstate-of-the-art multilingual models (i.e. M-BERT and XLM-R). The results\nprovide insight into their information sharing mechanisms and suggest that\nlinguistic properties are encoded jointly across typologically-similar\nlanguages in these models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12826v1",
            "date": "2020-10-24 08:00:30+00:00",
            "title": "Text Editing by Command",
            "authors": [
                "Felix Faltings",
                "Michel Galley",
                "Gerold Hintz",
                "Chris Brockett",
                "Chris Quirk",
                "Jianfeng Gao",
                "Bill Dolan"
            ],
            "abstract": "A prevailing paradigm in neural text generation is one-shot generation, where\ntext is produced in a single step. The one-shot setting is inadequate, however,\nwhen the constraints the user wishes to impose on the generated text are\ndynamic, especially when authoring longer documents. We address this limitation\nwith an interactive text generation setting in which the user interacts with\nthe system by issuing commands to edit existing text. To this end, we propose a\nnovel text editing task, and introduce WikiDocEdits, a dataset of\nsingle-sentence edits crawled from Wikipedia. We show that our Interactive\nEditor, a transformer-based model trained on this dataset, outperforms\nbaselines and obtains positive results in both automatic and human evaluations.\nWe present empirical and qualitative analyses of this model's performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12827v1",
            "date": "2020-10-24 08:06:18+00:00",
            "title": "Context-aware Decoder for Neural Machine Translation using a Target-side Document-Level Language Model",
            "authors": [
                "Amane Sugiyama",
                "Naoki Yoshinaga"
            ],
            "abstract": "Although many context-aware neural machine translation models have been\nproposed to incorporate contexts in translation, most of those models are\ntrained end-to-end on parallel documents aligned in sentence-level. Because\nonly a few domains (and language pairs) have such document-level parallel data,\nwe cannot perform accurate context-aware translation in most domains. We\ntherefore present a simple method to turn a sentence-level translation model\ninto a context-aware model by incorporating a document-level language model\ninto the decoder. Our context-aware decoder is built upon only a sentence-level\nparallel corpora and monolingual corpora; thus no document-level parallel data\nis needed. In a theoretical viewpoint, the core part of this work is the novel\nrepresentation of contextual information using point-wise mutual information\nbetween context and the current sentence. We show the effectiveness of our\napproach in three language pairs, English to French, English to Russian, and\nJapanese to English, by evaluation in \\textsc{bleu} and contrastive tests for\ncontext-aware translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12829v4",
            "date": "2020-10-24 08:15:08+00:00",
            "title": "Multilingual Speech Translation with Efficient Finetuning of Pretrained Models",
            "authors": [
                "Xian Li",
                "Changhan Wang",
                "Yun Tang",
                "Chau Tran",
                "Yuqing Tang",
                "Juan Pino",
                "Alexei Baevski",
                "Alexis Conneau",
                "Michael Auli"
            ],
            "abstract": "We present a simple yet effective approach to build multilingual\nspeech-to-text (ST) translation by efficient transfer learning from pretrained\nspeech encoder and text decoder. Our key finding is that a minimalistic LNA\n(LayerNorm and Attention) finetuning can achieve zero-shot crosslingual and\ncross-modality transfer ability by only finetuning less than 10% of the\npretrained parameters. This enables effectively leveraging large pretrained\nmodels with low training cost. Using wav2vec 2.0 for acoustic modeling, and\nmBART for multilingual text generation, our approach advanced the new\nstate-of-the-art for 34 translation directions (and surpassing cascaded ST for\n23 of them) on large-scale multilingual ST benchmark CoVoST 2 (+6.4 BLEU on\naverage across 15 En-X directions and +5.1 BLEU on average across 19 X-En\ndirections). Our approach demonstrates strong zero-shot performance in a\nmany-to-many multilingual model (+5.7 BLEU on average across 18 non-English\ndirections), making it an appealing approach for attaining high-quality speech\ntranslation with improved parameter and data efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12834v1",
            "date": "2020-10-24 08:30:20+00:00",
            "title": "Go Figure! A Meta Evaluation of Factuality in Summarization",
            "authors": [
                "Saadia Gabriel",
                "Asli Celikyilmaz",
                "Rahul Jha",
                "Yejin Choi",
                "Jianfeng Gao"
            ],
            "abstract": "Text generation models can generate factually inconsistent text containing\ndistorted or fabricated facts about the source text. Recent work has focused on\nbuilding evaluation models to verify the factual correctness of semantically\nconstrained text generation tasks such as document summarization. While the\nfield of factuality evaluation is growing fast, we don't have well-defined\ncriteria for measuring the effectiveness, generalizability, reliability, or\nsensitivity of the factuality metrics. Focusing on these aspects, in this\npaper, we introduce a meta-evaluation framework for evaluating factual\nconsistency metrics. We introduce five necessary, common-sense conditions for\neffective factuality metrics and experiment with nine recent factuality metrics\nusing synthetic and human-labeled factuality data from short news, long news\nand dialogue summarization domains. Our framework enables assessing the\nefficiency of any new factual consistency metric on a variety of dimensions\nover multiple summarization domains and can be easily extended with new\nmeta-evaluation criteria. We also present our conclusions towards standardizing\nthe factuality evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12836v2",
            "date": "2020-10-24 08:36:49+00:00",
            "title": "Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation",
            "authors": [
                "Alexander R. Fabbri",
                "Simeng Han",
                "Haoyuan Li",
                "Haoran Li",
                "Marjan Ghazvininejad",
                "Shafiq Joty",
                "Dragomir Radev",
                "Yashar Mehdad"
            ],
            "abstract": "Models pretrained with self-supervised objectives on large text corpora\nachieve state-of-the-art performance on English text summarization tasks.\nHowever, these models are typically fine-tuned on hundreds of thousands of data\npoints, an infeasible requirement when applying summarization to new, niche\ndomains. In this work, we introduce a novel and generalizable method, called\nWikiTransfer, for fine-tuning pretrained models for summarization in an\nunsupervised, dataset-specific manner. WikiTransfer fine-tunes pretrained\nmodels on pseudo-summaries, produced from generic Wikipedia data, which contain\ncharacteristics of the target dataset, such as the length and level of\nabstraction of the desired summaries. WikiTransfer models achieve\nstate-of-the-art, zero-shot abstractive summarization performance on the\nCNN-DailyMail dataset and demonstrate the effectiveness of our approach on\nthree additional diverse datasets. These models are more robust to noisy data\nand also achieve better or comparable few-shot performance using 10 and 100\ntraining examples when compared to few-shot transfer from other summarization\ndatasets. To further boost performance, we employ data augmentation via\nround-trip translation as well as introduce a regularization term for improved\nfew-shot transfer. To understand the role of dataset aspects in transfer\nperformance and the quality of the resulting output summaries, we further study\nthe effect of the components of our unsupervised fine-tuning data and analyze\nfew-shot performance using both automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12850v3",
            "date": "2020-10-24 09:39:35+00:00",
            "title": "CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers",
            "authors": [
                "Shiyang Li",
                "Semih Yavuz",
                "Kazuma Hashimoto",
                "Jia Li",
                "Tong Niu",
                "Nazneen Rajani",
                "Xifeng Yan",
                "Yingbo Zhou",
                "Caiming Xiong"
            ],
            "abstract": "Dialogue state trackers have made significant progress on benchmark datasets,\nbut their generalization capability to novel and realistic scenarios beyond the\nheld-out conversations is less understood. We propose controllable\ncounterfactuals (CoCo) to bridge this gap and evaluate dialogue state tracking\n(DST) models on novel scenarios, i.e., would the system successfully tackle the\nrequest if the user responded differently but still consistently with the\ndialogue flow? CoCo leverages turn-level belief states as counterfactual\nconditionals to produce novel conversation scenarios in two steps: (i)\ncounterfactual goal generation at turn-level by dropping and adding slots\nfollowed by replacing slot values, (ii) counterfactual conversation generation\nthat is conditioned on (i) and consistent with the dialogue flow. Evaluating\nstate-of-the-art DST models on MultiWOZ dataset with CoCo-generated\ncounterfactuals results in a significant performance drop of up to 30.8% (from\n49.4% to 18.6%) in absolute joint goal accuracy. In comparison, widely used\ntechniques like paraphrasing only affect the accuracy by at most 2%. Human\nevaluations show that COCO-generated conversations perfectly reflect the\nunderlying user goal with more than 95% accuracy and are as human-like as the\noriginal conversations, further strengthening its reliability and promise to be\nadopted as part of the robustness evaluation of DST models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12858v2",
            "date": "2020-10-24 10:15:03+00:00",
            "title": "When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models",
            "authors": [
                "Benjamin Muller",
                "Antonis Anastasopoulos",
                "Beno\u00eet Sagot",
                "Djam\u00e9 Seddah"
            ],
            "abstract": "Transfer learning based on pretraining language models on a large amount of\nraw data has become a new norm to reach state-of-the-art performance in NLP.\nStill, it remains unclear how this approach should be applied for unseen\nlanguages that are not covered by any available large-scale multilingual\nlanguage model and for which only a small amount of raw data is generally\navailable. In this work, by comparing multilingual and monolingual models, we\nshow that such models behave in multiple ways on unseen languages. Some\nlanguages greatly benefit from transfer learning and behave similarly to\nclosely related high resource languages whereas others apparently do not.\nFocusing on the latter, we show that this failure to transfer is largely\nrelated to the impact of the script used to write such languages.\nTransliterating those languages improves very significantly the ability of\nlarge-scale multilingual language models on downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12868v2",
            "date": "2020-10-24 11:00:58+00:00",
            "title": "Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation",
            "authors": [
                "Yongchang Hao",
                "Shilin He",
                "Wenxiang Jiao",
                "Zhaopeng Tu",
                "Michael Lyu",
                "Xing Wang"
            ],
            "abstract": "Non-Autoregressive machine Translation (NAT) models have demonstrated\nsignificant inference speedup but suffer from inferior translation accuracy.\nThe common practice to tackle the problem is transferring the Autoregressive\nmachine Translation (AT) knowledge to NAT models, e.g., with knowledge\ndistillation. In this work, we hypothesize and empirically verify that AT and\nNAT encoders capture different linguistic properties of source sentences.\nTherefore, we propose to adopt Multi-Task learning to transfer the AT knowledge\nto NAT models through encoder sharing. Specifically, we take the AT model as an\nauxiliary task to enhance NAT model performance. Experimental results on WMT14\nEnglish-German and WMT16 English-Romanian datasets show that the proposed\nMulti-Task NAT achieves significant improvements over the baseline NAT models.\nFurthermore, the performance on large-scale WMT19 and WMT20 English-German\ndatasets confirm the consistency of our proposed method. In addition,\nexperimental results demonstrate that our Multi-Task NAT is complementary to\nknowledge distillation, the standard knowledge transfer method for NAT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12873v2",
            "date": "2020-10-24 11:09:16+00:00",
            "title": "Learning Contextualized Knowledge Structures for Commonsense Reasoning",
            "authors": [
                "Jun Yan",
                "Mrigank Raman",
                "Aaron Chan",
                "Tianyu Zhang",
                "Ryan Rossi",
                "Handong Zhao",
                "Sungchul Kim",
                "Nedim Lipka",
                "Xiang Ren"
            ],
            "abstract": "Recently, neural-symbolic models have achieved noteworthy success in\nleveraging knowledge graphs (KGs) for commonsense reasoning tasks, like\nquestion answering (QA). However, fact sparsity, inherent in human-annotated\nKGs, can hinder such models from retrieving task-relevant knowledge. To address\nthese issues, we propose Hybrid Graph Network (HGN), a neural-symbolic model\nthat reasons over both extracted (human-labeled) and generated facts within the\nsame learned graph structure. Given a KG subgraph of extracted facts, HGN is\njointly trained to generate complementary facts, encode relational information\nin the resulting \"hybrid\" subgraph, and filter out task-irrelevant facts. We\ndemonstrate HGN's ability to produce contextually pertinent subgraphs by\nshowing considerable performance gains across four commonsense reasoning\nbenchmarks and a user study of fact validness and helpfulness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12881v1",
            "date": "2020-10-24 11:44:41+00:00",
            "title": "Revisiting Neural Language Modelling with Syllables",
            "authors": [
                "Arturo Oncevay",
                "Kervy Rivas Rojas"
            ],
            "abstract": "Language modelling is regularly analysed at word, subword or character units,\nbut syllables are seldom used. Syllables provide shorter sequences than\ncharacters, they can be extracted with rules, and their segmentation typically\nrequires less specialised effort than identifying morphemes. We reconsider\nsyllables for an open-vocabulary generation task in 20 languages. We use\nrule-based syllabification methods for five languages and address the rest with\na hyphenation tool, which behaviour as syllable proxy is validated. With a\ncomparable perplexity, we show that syllables outperform characters, annotated\nmorphemes and unsupervised subwords. Finally, we also study the overlapping of\nsyllables concerning other subword pieces and discuss some limitations and\nopportunities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12882v1",
            "date": "2020-10-24 11:52:05+00:00",
            "title": "FedE: Embedding Knowledge Graphs in Federated Setting",
            "authors": [
                "Mingyang Chen",
                "Wen Zhang",
                "Zonggang Yuan",
                "Yantao Jia",
                "Huajun Chen"
            ],
            "abstract": "Knowledge graphs (KGs) consisting of triples are always incomplete, so it's\nimportant to do Knowledge Graph Completion (KGC) by predicting missing triples.\nMulti-Source KG is a common situation in real KG applications which can be\nviewed as a set of related individual KGs where different KGs contains\nrelations of different aspects of entities. It's intuitive that, for each\nindividual KG, its completion could be greatly contributed by the triples\ndefined and labeled in other ones. However, because of the data privacy and\nsensitivity, a set of relevant knowledge graphs cannot complement each other's\nKGC by just collecting data from different knowledge graphs together.\nTherefore, in this paper, we introduce federated setting to keep their privacy\nwithout triple transferring between KGs and apply it in embedding knowledge\ngraph, a typical method which have proven effective for KGC in the past decade.\nWe propose a Federated Knowledge Graph Embedding framework FedE, focusing on\nlearning knowledge graph embeddings by aggregating locally-computed updates.\nFinally, we conduct extensive experiments on datasets derived from KGE\nbenchmark datasets and results show the effectiveness of our proposed FedE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12884v2",
            "date": "2020-10-24 11:55:22+00:00",
            "title": "NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints",
            "authors": [
                "Ximing Lu",
                "Peter West",
                "Rowan Zellers",
                "Ronan Le Bras",
                "Chandra Bhagavatula",
                "Yejin Choi"
            ],
            "abstract": "Conditional text generation often requires lexical constraints, i.e., which\nwords should or shouldn't be included in the output text. While the dominant\nrecipe for conditional text generation has been large-scale pretrained language\nmodels that are finetuned on the task-specific training data, such models do\nnot learn to follow the underlying constraints reliably, even when supervised\nwith large amounts of task-specific examples.\n  We propose NeuroLogic Decoding, a simple yet effective algorithm that enables\nneural language models -- supervised or not -- to generate fluent text while\nsatisfying complex lexical constraints. Our approach is powerful yet efficient.\nIt handles any set of lexical constraints that is expressible under predicate\nlogic, while its asymptotic runtime is equivalent to conventional beam search.\n  Empirical results on four benchmarks show that NeuroLogic Decoding\noutperforms previous approaches, including algorithms that handle a subset of\nour constraints. Moreover, we find that unsupervised models with NeuroLogic\nDecoding often outperform supervised models with conventional decoding, even\nwhen the latter is based on considerably larger networks. Our results suggest\nthe limit of large-scale neural networks for fine-grained controllable\ngeneration and the promise of inference-time algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12885v1",
            "date": "2020-10-24 11:55:28+00:00",
            "title": "Unsupervised Paraphrase Generation via Dynamic Blocking",
            "authors": [
                "Tong Niu",
                "Semih Yavuz",
                "Yingbo Zhou",
                "Huan Wang",
                "Nitish Shirish Keskar",
                "Caiming Xiong"
            ],
            "abstract": "We propose Dynamic Blocking, a decoding algorithm which enables large-scale\npretrained autoregressive models (such as BART, T5, GPT-2 and XLNet) to\ngenerate high-quality paraphrases in an unsupervised setting. In order to\nobtain an alternative surface form, whenever the language model emits a token\nthat is present in the source sequence, we prevent the model from generating\nthe subsequent source token for the next time step. We show that our approach\nachieves state-of-the-art results on benchmark datasets when compared to\nprevious unsupervised approaches, and is even comparable with strong\nsupervised, in-domain models. We also propose a new automatic metric based on\nself-BLEU and BERTscore which not only discourages the model from copying the\ninput through, but also evaluates text similarity based on distributed\nrepresentations, hence avoiding reliance on exact keyword matching. In\naddition, we demonstrate that our model generalizes across languages without\nany additional training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12937v1",
            "date": "2020-10-24 17:22:44+00:00",
            "title": "A Benchmark Corpus and Neural Approach for Sanskrit Derivative Nouns Analysis",
            "authors": [
                "Arun Kumar Singh",
                "Sushant Dave",
                "Dr. Prathosh A. P.",
                "Prof. Brejesh Lall",
                "Shresth Mehta"
            ],
            "abstract": "This paper presents first benchmark corpus of Sanskrit Pratyaya (suffix) and\ninflectional words (padas) formed due to suffixes along with neural network\nbased approaches to process the formation and splitting of inflectional words.\nInflectional words spans the primary and secondary derivative nouns as the\nscope of current work. Pratyayas are an important dimension of morphological\nanalysis of Sanskrit texts. There have been Sanskrit Computational Linguistics\ntools for processing and analyzing Sanskrit texts. Unfortunately there has not\nbeen any work to standardize & validate these tools specifically for derivative\nnouns analysis. In this work, we prepared a Sanskrit suffix benchmark corpus\ncalled Pratyaya-Kosh to evaluate the performance of tools. We also present our\nown neural approach for derivative nouns analysis while evaluating the same on\nmost prominent Sanskrit Morphological Analysis tools. This benchmark will be\nfreely dedicated and available to researchers worldwide and we hope it will\nmotivate all to improve morphological analysis in Sanskrit Language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12940v1",
            "date": "2020-10-24 18:02:40+00:00",
            "title": "Neural Compound-Word (Sandhi) Generation and Splitting in Sanskrit Language",
            "authors": [
                "Sushant Dave",
                "Arun Kumar Singh",
                "Dr. Prathosh A. P.",
                "Prof. Brejesh Lall"
            ],
            "abstract": "This paper describes neural network based approaches to the process of the\nformation and splitting of word-compounding, respectively known as the Sandhi\nand Vichchhed, in Sanskrit language. Sandhi is an important idea essential to\nmorphological analysis of Sanskrit texts. Sandhi leads to word transformations\nat word boundaries. The rules of Sandhi formation are well defined but complex,\nsometimes optional and in some cases, require knowledge about the nature of the\nwords being compounded. Sandhi split or Vichchhed is an even more difficult\ntask given its non uniqueness and context dependence. In this work, we propose\nthe route of formulating the problem as a sequence to sequence prediction task,\nusing modern deep learning techniques. Being the first fully data driven\ntechnique, we demonstrate that our model has an accuracy better than the\nexisting methods on multiple standard datasets, despite not using any\nadditional lexical or morphological resources. The code is being made available\nat https://github.com/IITD-DataScience/Sandhi_Prakarana",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13031v1",
            "date": "2020-10-25 04:27:43+00:00",
            "title": "Towards Medical Knowmetrics: Representing and Computing Medical Knowledge using Semantic Predications as the Knowledge Unit and the Uncertainty as the Knowledge Context",
            "authors": [
                "Xiaoying Li",
                "Suyuan Peng",
                "Jian Du"
            ],
            "abstract": "In China, Prof. Hongzhou Zhao and Zeyuan Liu are the pioneers of the concept\n\"knowledge unit\" and \"knowmetrics\" for measuring knowledge. However, the\ndefinition of \"computable knowledge object\" remains controversial so far in\ndifferent fields. For example, it is defined as 1) quantitative scientific\nconcept in natural science and engineering, 2) knowledge point in the field of\neducation research, and 3) semantic predications, i.e.,\nSubject-Predicate-Object (SPO) triples in biomedical fields. The Semantic\nMEDLINE Database (SemMedDB), a high-quality public repository of SPO triples\nextracted from medical literature, provides a basic data infrastructure for\nmeasuring medical knowledge. In general, the study of extracting SPO triples as\ncomputable knowledge unit from unstructured scientific text has been\noverwhelmingly focusing on scientific knowledge per se. Since the SPO triples\nwould be possibly extracted from hypothetical, speculative statements or even\nconflicting and contradictory assertions, the knowledge status (i.e., the\nuncertainty), which serves as an integral and critical part of scientific\nknowledge has been largely overlooked. This article aims to put forward a\nframework for Medical Knowmetrics using the SPO triples as the knowledge unit\nand the uncertainty as the knowledge context. The lung cancer publications\ndataset is used to validate the proposed framework. The uncertainty of medical\nknowledge and how its status evolves over time indirectly reflect the strength\nof competing knowledge claims, and the probability of certainty for a given SPO\ntriple. We try to discuss the new insights using the uncertainty-centric\napproaches to detect research fronts, and identify knowledge claims with high\ncertainty level, in order to improve the efficacy of knowledge-driven decision\nsupport.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13040v2",
            "date": "2020-10-25 05:18:14+00:00",
            "title": "Fine-tuning ERNIE for chest abnormal imaging signs extraction",
            "authors": [
                "Zhaoning Li",
                "Jiangtao Ren"
            ],
            "abstract": "Chest imaging reports describe the results of chest radiography procedures.\nAutomatic extraction of abnormal imaging signs from chest imaging reports has a\npivotal role in clinical research and a wide range of downstream medical tasks.\nHowever, there are few studies on information extraction from Chinese chest\nimaging reports. In this paper, we formulate chest abnormal imaging sign\nextraction as a sequence tagging and matching problem. On this basis, we\npropose a transferred abnormal imaging signs extractor with pretrained ERNIE as\nthe backbone, named EASON (fine-tuning ERNIE with CRF for Abnormal Signs\nExtractiON), which can address the problem of data insufficiency. In addition,\nto assign the attributes (the body part and degree) to corresponding abnormal\nimaging signs from the results of the sequence tagging model, we design a\nsimple but effective tag2relation algorithm based on the nature of chest\nimaging report text. We evaluate our method on the corpus provided by a medical\nbig data company, and the experimental results demonstrate that our method\nachieves significant and consistent improvement compared to other baselines.",
            "journal ref": "Journal of Biomedical Informatics, Volume 108, 2020, 103492",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13049v1",
            "date": "2020-10-25 07:17:45+00:00",
            "title": "Commonsense knowledge adversarial dataset that challenges ELECTRA",
            "authors": [
                "Gongqi Lin",
                "Yuan Miao",
                "Xiaoyong Yang",
                "Wenwu Ou",
                "Lizhen Cui",
                "Wei Guo",
                "Chunyan Miao"
            ],
            "abstract": "Commonsense knowledge is critical in human reading comprehension. While\nmachine comprehension has made significant progress in recent years, the\nability in handling commonsense knowledge remains limited. Synonyms are one of\nthe most widely used commonsense knowledge. Constructing adversarial dataset is\nan important approach to find weak points of machine comprehension models and\nsupport the design of solutions. To investigate machine comprehension models'\nability in handling the commonsense knowledge, we created a Question and Answer\nDataset with common knowledge of Synonyms (QADS). QADS are questions generated\nbased on SQuAD 2.0 by applying commonsense knowledge of synonyms. The synonyms\nare extracted from WordNet. Words often have multiple meanings and synonyms. We\nused an enhanced Lesk algorithm to perform word sense disambiguation to\nidentify synonyms for the context. ELECTRA achieves the state-of-art result on\nthe SQuAD 2.0 dataset in 2019. With scale, ELECTRA can achieve similar\nperformance as BERT does. However, QADS shows that ELECTRA has little ability\nto handle commonsense knowledge of synonyms. In our experiment, ELECTRA-small\ncan achieve 70% accuracy on SQuAD 2.0, but only 20% on QADS. ELECTRA-large did\nnot perform much better. Its accuracy on SQuAD 2.0 is 88% but dropped\nsignificantly to 26% on QADS. In our earlier experiments, BERT, although also\nfailed badly on QADS, was not as bad as ELECTRA. The result shows that even\ntop-performing NLP models have little ability to handle commonsense knowledge\nwhich is essential in reading comprehension.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13057v1",
            "date": "2020-10-25 07:56:52+00:00",
            "title": "Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge",
            "authors": [
                "Sathvik Nair",
                "Mahesh Srinivasan",
                "Stephan Meylan"
            ],
            "abstract": "Understanding context-dependent variation in word meanings is a key aspect of\nhuman language comprehension supported by the lexicon. Lexicographic resources\n(e.g., WordNet) capture only some of this context-dependent variation; for\nexample, they often do not encode how closely senses, or discretized word\nmeanings, are related to one another. Our work investigates whether recent\nadvances in NLP, specifically contextualized word embeddings, capture\nhuman-like distinctions between English word senses, such as polysemy and\nhomonymy. We collect data from a behavioral, web-based experiment, in which\nparticipants provide judgments of the relatedness of multiple WordNet senses of\na word in a two-dimensional spatial arrangement task. We find that\nparticipants' judgments of the relatedness between senses are correlated with\ndistances between senses in the BERT embedding space. Homonymous senses (e.g.,\nbat as mammal vs. bat as sports equipment) are reliably more distant from one\nanother in the embedding space than polysemous ones (e.g., chicken as animal\nvs. chicken as meat). Our findings point towards the potential utility of\ncontinuous-space representations of sense meanings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13062v1",
            "date": "2020-10-25 08:13:34+00:00",
            "title": "Transgender Community Sentiment Analysis from Social Media Data: A Natural Language Processing Approach",
            "authors": [
                "Mengzhe Li",
                "Yudan Wang",
                "Ying Zhao",
                "Zhixiang Li"
            ],
            "abstract": "Transgender community is experiencing a huge disparity in mental health\nconditions compared with the general population. Interpreting the social medial\ndata posted by transgender people may help us understand the sentiments of\nthese sexual minority groups better and apply early interventions. In this\nstudy, we manually categorize 300 social media comments posted by transgender\npeople to the sentiment of negative, positive, and neutral. 5 machine learning\nalgorithms and 2 deep neural networks are adopted to build sentiment analysis\nclassifiers based on the annotated data. Results show that our annotations are\nreliable with a high Cohen's Kappa score over 0.8 across all three classes.\nLSTM model yields an optimal performance of accuracy over 0.85 and AUC of\n0.876. Our next step will focus on using advanced natural language processing\nalgorithms on a larger annotated dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13094v2",
            "date": "2020-10-25 11:30:05+00:00",
            "title": "Autoencoding Improves Pre-trained Word Embeddings",
            "authors": [
                "Masahiro Kaneko",
                "Danushka Bollegala"
            ],
            "abstract": "Prior work investigating the geometry of pre-trained word embeddings have\nshown that word embeddings to be distributed in a narrow cone and by centering\nand projecting using principal component vectors one can increase the accuracy\nof a given set of pre-trained word embeddings. However, theoretically, this\npost-processing step is equivalent to applying a linear autoencoder to minimise\nthe squared l2 reconstruction error. This result contradicts prior work (Mu and\nViswanath, 2018) that proposed to remove the top principal components from\npre-trained embeddings. We experimentally verify our theoretical claims and\nshow that retaining the top principal components is indeed useful for improving\npre-trained word embeddings, without requiring access to additional linguistic\nresources or labelled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13192v1",
            "date": "2020-10-25 19:04:03+00:00",
            "title": "The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task",
            "authors": [
                "Alexandra Chronopoulou",
                "Dario Stojanovski",
                "Viktor Hangya",
                "Alexander Fraser"
            ],
            "abstract": "This paper describes the submission of LMU Munich to the WMT 2020\nunsupervised shared task, in two language directions, German<->Upper Sorbian.\nOur core unsupervised neural machine translation (UNMT) system follows the\nstrategy of Chronopoulou et al. (2020), using a monolingual pretrained language\ngeneration model (on German) and fine-tuning it on both German and Upper\nSorbian, before initializing a UNMT model, which is trained with online\nbacktranslation. Pseudo-parallel data obtained from an unsupervised statistical\nmachine translation (USMT) system is used to fine-tune the UNMT model. We also\napply BPE-Dropout to the low resource (Upper Sorbian) data to obtain a more\nrobust system. We additionally experiment with residual adapters and find them\nuseful in the Upper Sorbian->German direction. We explore sampling during\nbacktranslation and curriculum learning to use SMT translations in a more\nprincipled way. Finally, we ensemble our best-performing systems and reach a\nBLEU score of 32.4 on German->Upper Sorbian and 35.2 on Upper Sorbian->German.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13378v1",
            "date": "2020-10-26 07:13:17+00:00",
            "title": "Introducing Syntactic Structures into Target Opinion Word Extraction with Deep Learning",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Nasim Nouri",
                "Franck Dernoncourt",
                "Dejing Dou",
                "Thien Huu Nguyen"
            ],
            "abstract": "Targeted opinion word extraction (TOWE) is a sub-task of aspect based\nsentiment analysis (ABSA) which aims to find the opinion words for a given\naspect-term in a sentence. Despite their success for TOWE, the current deep\nlearning models fail to exploit the syntactic information of the sentences that\nhave been proved to be useful for TOWE in the prior research. In this work, we\npropose to incorporate the syntactic structures of the sentences into the deep\nlearning models for TOWE, leveraging the syntax-based opinion possibility\nscores and the syntactic connections between the words. We also introduce a\nnovel regularization technique to improve the performance of the deep learning\nmodels based on the representation distinctions between the words in TOWE. The\nproposed model is extensively analyzed and achieves the state-of-the-art\nperformance on four benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13382v1",
            "date": "2020-10-26 07:25:15+00:00",
            "title": "FastFormers: Highly Efficient Transformer Models for Natural Language Understanding",
            "authors": [
                "Young Jin Kim",
                "Hany Hassan Awadalla"
            ],
            "abstract": "Transformer-based models are the state-of-the-art for Natural Language\nUnderstanding (NLU) applications. Models are getting bigger and better on\nvarious tasks. However, Transformer models remain computationally challenging\nsince they are not efficient at inference-time compared to traditional\napproaches. In this paper, we present FastFormers, a set of recipes to achieve\nefficient inference-time performance for Transformer-based models on various\nNLU tasks. We show how carefully utilizing knowledge distillation, structured\npruning and numerical optimization can lead to drastic improvements on\ninference efficiency. We provide effective recipes that can guide practitioners\nto choose the best settings for various NLU tasks and pretrained models.\nApplying the proposed recipes to the SuperGLUE benchmark, we achieve from 9.8x\nup to 233.9x speed-up compared to out-of-the-box models on CPU. On GPU, we also\nachieve up to 12.4x speed-up with the presented methods. We show that\nFastFormers can drastically reduce cost of serving 100 million requests from\n4,223 USD to just 18 USD on an Azure F16s_v2 instance. This translates to a\nsustainable runtime by reducing energy consumption 6.9x - 125.8x according to\nthe metrics used in the SustaiNLP 2020 shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13389v1",
            "date": "2020-10-26 07:36:24+00:00",
            "title": "Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Nasim Nour",
                "Franck Dernoncourt",
                "Quan Hung Tran",
                "Dejing Dou",
                "Thien Huu Nguyen"
            ],
            "abstract": "Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment\npolarity of a sentence toward a specific aspect. Recently, it has been shown\nthat dependency trees can be integrated into deep learning models to produce\nthe state-of-the-art performance for ABSA. However, these models tend to\ncompute the hidden/representation vectors without considering the aspect terms\nand fail to benefit from the overall contextual importance scores of the words\nthat can be obtained from the dependency tree for ABSA. In this work, we\npropose a novel graph-based deep learning model to overcome these two issues of\nthe prior work on ABSA. In our model, gate vectors are generated from the\nrepresentation vectors of the aspect terms to customize the hidden vectors of\nthe graph-based models toward the aspect terms. In addition, we propose a\nmechanism to obtain the importance scores for each word in the sentences based\non the dependency trees that are then injected into the model to improve the\nrepresentation vectors for ABSA. The proposed model achieves the\nstate-of-the-art performance on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13391v1",
            "date": "2020-10-26 07:41:40+00:00",
            "title": "Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Tuan Ngo Nguyen",
                "Thien Huu Nguyen"
            ],
            "abstract": "The goal of Event Argument Extraction (EAE) is to find the role of each\nentity mention for a given event trigger word. It has been shown in the\nprevious works that the syntactic structures of the sentences are helpful for\nthe deep learning models for EAE. However, a major problem in such prior works\nis that they fail to exploit the semantic structures of the sentences to induce\neffective representations for EAE. Consequently, in this work, we propose a\nnovel model for EAE that exploits both syntactic and semantic structures of the\nsentences with the Graph Transformer Networks (GTNs) to learn more effective\nsentence structures for EAE. In addition, we introduce a novel inductive bias\nbased on information bottleneck to improve generalization of the EAE models.\nExtensive experiments are performed to demonstrate the benefits of the proposed\nmodel, leading to state-of-the-art performance for EAE on standard datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13415v1",
            "date": "2020-10-26 08:35:06+00:00",
            "title": "TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking",
            "authors": [
                "Yucheng Wang",
                "Bowen Yu",
                "Yueyang Zhang",
                "Tingwen Liu",
                "Hongsong Zhu",
                "Limin Sun"
            ],
            "abstract": "Extracting entities and relations from unstructured text has attracted\nincreasing attention in recent years but remains challenging, due to the\nintrinsic difficulty in identifying overlapping relations with shared entities.\nPrior works show that joint learning can result in a noticeable performance\ngain. However, they usually involve sequential interrelated steps and suffer\nfrom the problem of exposure bias. At training time, they predict with the\nground truth conditions while at inference it has to make extraction from\nscratch. This discrepancy leads to error accumulation. To mitigate the issue,\nwe propose in this paper a one-stage joint extraction model, namely, TPLinker,\nwhich is capable of discovering overlapping relations sharing one or both\nentities while immune from the exposure bias. TPLinker formulates joint\nextraction as a token pair linking problem and introduces a novel handshaking\ntagging scheme that aligns the boundary tokens of entity pairs under each\nrelation type. Experiment results show that TPLinker performs significantly\nbetter on overlapping and multiple relation extraction, and achieves\nstate-of-the-art performance on two public datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13515v1",
            "date": "2020-10-26 12:14:14+00:00",
            "title": "Syllabification of the Divine Comedy",
            "authors": [
                "Andrea Asperti",
                "Stefano Dal Bianco"
            ],
            "abstract": "We provide a syllabification algorithm for the Divine Comedy using techniques\nfrom probabilistic and constraint programming. We particularly focus on the\nsynalephe, addressed in terms of the \"propensity\" of a word to take part in a\nsynalephe with adjacent words. We jointly provide an online vocabulary\ncontaining, for each word, information about its syllabification, the location\nof the tonic accent, and the aforementioned synalephe propensity, on the left\nand right sides. The algorithm is intrinsically nondeterministic, producing\ndifferent possible syllabifications for each verse, with different likelihoods;\nmetric constraints relative to accents on the 10th, 4th and 6th syllables are\nused to further reduce the solution space. The most likely syllabification is\nhence returned as output. We believe that this work could be a major milestone\nfor a lot of different investigations. From the point of view of digital\nhumanities it opens new perspectives on computer assisted analysis of digital\nsources, comprising automated detection of anomalous and problematic cases,\nmetric clustering of verses and their categorization, or more foundational\ninvestigations addressing e.g. the phonetic roles of consonants and vowels.\nFrom the point of view of text processing and deep learning, information about\nsyllabification and the location of accents opens a wide range of exciting\nperspectives, from the possibility of automatic learning syllabification of\nwords and verses, to the improvement of generative models, aware of metric\nissues, and more respectful of the expected musicality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13544v1",
            "date": "2020-10-26 12:52:28+00:00",
            "title": "Meta-Learning for Neural Relation Classification with Distant Supervision",
            "authors": [
                "Zhenzhen Li",
                "Jian-Yun Nie",
                "Benyou Wang",
                "Pan Du",
                "Yuhan Zhang",
                "Lixin Zou",
                "Dongsheng Li"
            ],
            "abstract": "Distant supervision provides a means to create a large number of weakly\nlabeled data at low cost for relation classification. However, the resulting\nlabeled instances are very noisy, containing data with wrong labels. Many\napproaches have been proposed to select a subset of reliable instances for\nneural model training, but they still suffer from noisy labeling problem or\nunderutilization of the weakly-labeled data. To better select more reliable\ntraining instances, we introduce a small amount of manually labeled data as\nreference to guide the selection process. In this paper, we propose a\nmeta-learning based approach, which learns to reweight noisy training data\nunder the guidance of reference data. As the clean reference data is usually\nvery small, we propose to augment it by dynamically distilling the most\nreliable elite instances from the noisy data. Experiments on several datasets\ndemonstrate that the reference data can effectively guide the selection of\ntraining data, and our augmented approach consistently improves the performance\nof relation classification comparing to the existing state-of-the-art methods.",
            "journal ref": "In Proceedings of CIKM, pp. 815-824. 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13588v1",
            "date": "2020-10-26 13:57:20+00:00",
            "title": "Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale",
            "authors": [
                "Ozan Caglayan",
                "Pranava Madhyastha",
                "Lucia Specia"
            ],
            "abstract": "Automatic evaluation of language generation systems is a well-studied problem\nin Natural Language Processing. While novel metrics are proposed every year, a\nfew popular metrics remain as the de facto metrics to evaluate tasks such as\nimage captioning and machine translation, despite their known limitations. This\nis partly due to ease of use, and partly because researchers expect to see them\nand know how to interpret them. In this paper, we urge the community for more\ncareful consideration of how they automatically evaluate their models by\ndemonstrating important failure cases on multiple datasets, language pairs and\ntasks. Our experiments show that metrics (i) usually prefer system outputs to\nhuman-authored texts, (ii) can be insensitive to correct translations of rare\nwords, (iii) can yield surprisingly high scores when given a single sentence as\nsystem output for the entire test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13814v1",
            "date": "2020-10-26 18:01:52+00:00",
            "title": "Is it Great or Terrible? Preserving Sentiment in Neural Machine Translation of Arabic Reviews",
            "authors": [
                "Hadeel Saadany",
                "Constantin Orasan"
            ],
            "abstract": "Since the advent of Neural Machine Translation (NMT) approaches there has\nbeen a tremendous improvement in the quality of automatic translation. However,\nNMT output still lacks accuracy in some low-resource languages and sometimes\nmakes major errors that need extensive post-editing. This is particularly\nnoticeable with texts that do not follow common lexico-grammatical standards,\nsuch as user generated content (UGC). In this paper we investigate the\nchallenges involved in translating book reviews from Arabic into English, with\nparticular focus on the errors that lead to incorrect translation of sentiment\npolarity. Our study points to the special characteristics of Arabic UGC,\nexamines the sentiment transfer errors made by Google Translate of Arabic UGC\nto English, analyzes why the problem occurs, and proposes an error typology\nspecific of the translation of Arabic UGC. Our analysis shows that the output\nof online translation tools of Arabic UGC can either fail to transfer the\nsentiment at all by producing a neutral target text, or completely flips the\nsentiment polarity of the target word or phrase and hence delivers a wrong\naffect message. We address this problem by fine-tuning an NMT model with\nrespect to sentiment polarity showing that this approach can significantly help\nwith correcting sentiment errors detected in the online translation of Arabic\nUGC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13826v1",
            "date": "2020-10-26 18:21:27+00:00",
            "title": "Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretraining",
            "authors": [
                "Cheng-I Lai",
                "Yung-Sung Chuang",
                "Hung-Yi Lee",
                "Shang-Wen Li",
                "James Glass"
            ],
            "abstract": "Much recent work on Spoken Language Understanding (SLU) is limited in at\nleast one of three ways: models were trained on oracle text input and neglected\nASR errors, models were trained to predict only intents without the slot\nvalues, or models were trained on a large amount of in-house data. In this\npaper, we propose a clean and general framework to learn semantics directly\nfrom speech with semi-supervision from transcribed or untranscribed speech to\naddress these issues. Our framework is built upon pretrained end-to-end (E2E)\nASR and self-supervised language models, such as BERT, and fine-tuned on a\nlimited amount of target SLU data. We study two semi-supervised settings for\nthe ASR component: supervised pretraining on transcribed speech, and\nunsupervised pretraining by replacing the ASR encoder with self-supervised\nspeech representations, such as wav2vec. In parallel, we identify two essential\ncriteria for evaluating SLU models: environmental noise-robustness and E2E\nsemantics evaluation. Experiments on ATIS show that our SLU framework with\nspeech as input can perform on par with those using oracle text as input in\nsemantics understanding, even though environmental noise is present and a\nlimited amount of labeled semantics data is available for training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13870v1",
            "date": "2020-10-26 19:51:36+00:00",
            "title": "Word Frequency Does Not Predict Grammatical Knowledge in Language Models",
            "authors": [
                "Charles Yu",
                "Ryan Sie",
                "Nico Tedeschi",
                "Leon Bergen"
            ],
            "abstract": "Neural language models learn, to varying degrees of accuracy, the grammatical\nproperties of natural languages. In this work, we investigate whether there are\nsystematic sources of variation in the language models' accuracy. Focusing on\nsubject-verb agreement and reflexive anaphora, we find that certain nouns are\nsystematically understood better than others, an effect which is robust across\ngrammatical tasks and different language models. Surprisingly, we find that\nacross four orders of magnitude, corpus frequency is unrelated to a noun's\nperformance on grammatical tasks. Finally, we find that a novel noun's\ngrammatical properties can be few-shot learned from various types of training\ndata. The results present a paradox: there should be less variation in\ngrammatical performance than is actually observed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13878v1",
            "date": "2020-10-26 20:10:12+00:00",
            "title": "Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer",
            "authors": [
                "Suyoun Kim",
                "Yuan Shangguan",
                "Jay Mahadeokar",
                "Antoine Bruguier",
                "Christian Fuegen",
                "Michael L. Seltzer",
                "Duc Le"
            ],
            "abstract": "Recurrent Neural Network Transducer (RNN-T), like most end-to-end speech\nrecognition model architectures, has an implicit neural network language model\n(NNLM) and cannot easily leverage unpaired text data during training. Previous\nwork has proposed various fusion methods to incorporate external NNLMs into\nend-to-end ASR to address this weakness. In this paper, we propose extensions\nto these techniques that allow RNN-T to exploit external NNLMs during both\ntraining and inference time, resulting in 13-18% relative Word Error Rate\nimprovement on Librispeech compared to strong baselines. Furthermore, our\nmethods do not incur extra algorithmic latency and allow for flexible\nplug-and-play of different NNLMs without re-training. We also share in-depth\nanalysis to better understand the benefits of the different NNLM fusion\nmethods. Our work provides a reliable technique for leveraging unpaired text\ndata to significantly improve RNN-T while keeping the system streamable,\nflexible, and lightweight.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13944v1",
            "date": "2020-10-26 23:09:09+00:00",
            "title": "Reading Between the Lines: Exploring Infilling in Visual Narratives",
            "authors": [
                "Khyathi Raghavi Chandu",
                "Ruo-Ping Dong",
                "Alan Black"
            ],
            "abstract": "Generating long form narratives such as stories and procedures from multiple\nmodalities has been a long standing dream for artificial intelligence. In this\nregard, there is often crucial subtext that is derived from the surrounding\ncontexts. The general seq2seq training methods render the models shorthanded\nwhile attempting to bridge the gap between these neighbouring contexts. In this\npaper, we tackle this problem by using \\textit{infilling} techniques involving\nprediction of missing steps in a narrative while generating textual\ndescriptions from a sequence of images. We also present a new large scale\n\\textit{visual procedure telling} (ViPT) dataset with a total of 46,200\nprocedures and around 340k pairwise images and textual descriptions that is\nrich in such contextual dependencies. Generating steps using infilling\ntechnique demonstrates the effectiveness in visual procedures with more\ncoherent texts. We conclusively show a METEOR score of 27.51 on procedures\nwhich is higher than the state-of-the-art on visual storytelling. We also\ndemonstrate the effects of interposing new text with missing images during\ninference. The code and the dataset will be publicly available at\nhttps://visual-narratives.github.io/Visual-Narratives/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13991v1",
            "date": "2020-10-27 02:09:06+00:00",
            "title": "Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learning",
            "authors": [
                "Dongwei Jiang",
                "Wubo Li",
                "Miao Cao",
                "Ruixiong Zhang",
                "Wei Zou",
                "Kun Han",
                "Xiangang Li"
            ],
            "abstract": "Self-supervised visual pretraining has shown significant progress recently.\nAmong those methods, SimCLR greatly advanced the state of the art in\nself-supervised and semi-supervised learning on ImageNet. The input feature\nrepresentations for speech and visual tasks are both continuous, so it is\nnatural to consider applying similar objective on speech representation\nlearning. In this paper, we propose Speech SimCLR, a new self-supervised\nobjective for speech representation learning. During training, Speech SimCLR\napplies augmentation on raw speech and its spectrogram. Its objective is the\ncombination of contrastive loss that maximizes agreement between differently\naugmented samples in the latent space and reconstruction loss of input\nrepresentation. The proposed method achieved competitive results on speech\nemotion recognition and speech recognition. When used as feature extractor, our\nbest model achieved 5.89% word error rate on LibriSpeech test-clean set using\nLibriSpeech 960 hours as pretraining data and LibriSpeech train-clean-100 set\nas fine-tuning data, which is the lowest error rate obtained in this setup to\nthe best of our knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14029v1",
            "date": "2020-10-27 03:20:04+00:00",
            "title": "Volctrans Parallel Corpus Filtering System for WMT 2020",
            "authors": [
                "Runxin Xu",
                "Zhuo Zhi",
                "Jun Cao",
                "Mingxuan Wang",
                "Lei Li"
            ],
            "abstract": "In this paper, we describe our submissions to the WMT20 shared task on\nparallel corpus filtering and alignment for low-resource conditions. The task\nrequires the participants to align potential parallel sentence pairs out of the\ngiven document pairs, and score them so that low-quality pairs can be filtered.\nOur system, Volctrans, is made of two modules, i.e., a mining module and a\nscoring module. Based on the word alignment model, the mining module adopts an\niterative mining strategy to extract latent parallel sentences. In the scoring\nmodule, an XLM-based scorer provides scores, followed by reranking mechanisms\nand ensemble. Our submissions outperform the baseline by 3.x/2.x and 2.x/2.x\nfor km-en and ps-en on From Scratch/Fine-Tune conditions, which is the highest\namong all submissions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14042v1",
            "date": "2020-10-27 04:03:47+00:00",
            "title": "To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging",
            "authors": [
                "Kasturi Bhattacharjee",
                "Miguel Ballesteros",
                "Rishita Anubhai",
                "Smaranda Muresan",
                "Jie Ma",
                "Faisal Ladhak",
                "Yaser Al-Onaizan"
            ],
            "abstract": "Leveraging large amounts of unlabeled data using Transformer-like\narchitectures, like BERT, has gained popularity in recent times owing to their\neffectiveness in learning general representations that can then be further\nfine-tuned for downstream tasks to much success. However, training these models\ncan be costly both from an economic and environmental standpoint. In this work,\nwe investigate how to effectively use unlabeled data: by exploring the\ntask-specific semi-supervised approach, Cross-View Training (CVT) and comparing\nit with task-agnostic BERT in multiple settings that include domain and task\nrelevant English data. CVT uses a much lighter model architecture and we show\nthat it achieves similar performance to BERT on a set of sequence tagging\ntasks, with lesser financial and environmental impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14123v1",
            "date": "2020-10-27 08:28:28+00:00",
            "title": "Event Detection: Gate Diversity and Syntactic Importance Scoresfor Graph Convolution Neural Networks",
            "authors": [
                "Viet Dac Lai",
                "Tuan Ngo Nguyen",
                "Thien Huu Nguyen"
            ],
            "abstract": "Recent studies on event detection (ED) haveshown that the syntactic\ndependency graph canbe employed in graph convolution neural net-works (GCN) to\nachieve state-of-the-art per-formance. However, the computation of thehidden\nvectors in such graph-based models isagnostic to the trigger candidate words,\npo-tentially leaving irrelevant information for thetrigger candidate for event\nprediction. In addi-tion, the current models for ED fail to exploitthe overall\ncontextual importance scores of thewords, which can be obtained via the\ndepen-dency tree, to boost the performance. In thisstudy, we propose a novel\ngating mechanismto filter noisy information in the hidden vec-tors of the GCN\nmodels for ED based on theinformation from the trigger candidate. Wealso\nintroduce novel mechanisms to achievethe contextual diversity for the gates and\ntheimportance score consistency for the graphsand models in ED. The experiments\nshow thatthe proposed model achieves state-of-the-artperformance on two ED\ndatasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14318v1",
            "date": "2020-10-27 14:29:28+00:00",
            "title": "Multitask Training with Text Data for End-to-End Speech Recognition",
            "authors": [
                "Peidong Wang",
                "Tara N. Sainath",
                "Ron J. Weiss"
            ],
            "abstract": "We propose a multitask training method for attention-based end-to-end speech\nrecognition models to better incorporate language level information. We\nregularize the decoder in a sequence-to-sequence architecture by multitask\ntraining it on both the speech recognition task and a next-token prediction\nlanguage modeling task. Trained on either the 100 hour subset of LibriSpeech or\nthe full 960 hour dataset, the proposed method leads to an 11% relative\nperformance improvement over the baseline and is comparable to language model\nshallow fusion, without requiring an additional neural network during decoding.\nAnalyses of sample output sentences and the word error rate on rare words\ndemonstrate that the proposed method can incorporate language level information\neffectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14342v1",
            "date": "2020-10-27 14:57:21+00:00",
            "title": "Listener's Social Identity Matters in Personalised Response Generation",
            "authors": [
                "Guanyi Chen",
                "Yinhe Zheng",
                "Yupei Du"
            ],
            "abstract": "Personalised response generation enables generating human-like responses by\nmeans of assigning the generator a social identity. However, pragmatics theory\nsuggests that human beings adjust the way of speaking based on not only who\nthey are but also whom they are talking to. In other words, when modelling\npersonalised dialogues, it might be favourable if we also take the listener's\nsocial identity into consideration. To validate this idea, we use gender as a\ntypical example of a social variable to investigate how the listener's identity\ninfluences the language used in Chinese dialogues on social media. Also, we\nbuild personalised generators. The experiment results demonstrate that the\nlistener's identity indeed matters in the language use of responses and that\nthe response generator can capture such differences in language use. More\ninterestingly, by additionally modelling the listener's identity, the\npersonalised response generator performs better in its own identity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14465v2",
            "date": "2020-10-27 17:24:27+00:00",
            "title": "Evaluating Gender Bias in Speech Translation",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Christine Basta",
                "Gerard I. G\u00e1llego"
            ],
            "abstract": "The scientific community is more and more aware of the necessity to embrace\npluralism and consistently represent major and minor social groups. In this\ndirection, there is an urgent need to provide evaluation sets and protocols to\nmeasure existing biases in our automatic systems. This paper introduces WinoST,\na new freely available challenge set for evaluating gender bias in speech\ntranslation. WinoST is the speech version of WinoMT which is an MT challenge\nset and both follow an evaluation protocol to measure gender accuracy. Using a\nstate-of-the-art end-to-end speech translation system, we report the gender\nbias evaluation on 4 language pairs, and we show that gender accuracy in speech\ntranslation is more than 23% lower than in MT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14534v1",
            "date": "2020-10-27 18:06:09+00:00",
            "title": "Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias",
            "authors": [
                "Marion Bartl",
                "Malvina Nissim",
                "Albert Gatt"
            ],
            "abstract": "Contextualized word embeddings have been replacing standard embeddings as the\nrepresentational knowledge source of choice in NLP systems. Since a variety of\nbiases have previously been found in standard word embeddings, it is crucial to\nassess biases encoded in their replacements as well. Focusing on BERT (Devlin\net al., 2018), we measure gender bias by studying associations between\ngender-denoting target words and names of professions in English and German,\ncomparing the findings with real-world workforce statistics. We mitigate bias\nby fine-tuning BERT on the GAP corpus (Webster et al., 2018), after applying\nCounterfactual Data Substitution (CDS) (Maudslay et al., 2019). We show that\nour method of measuring bias is appropriate for languages such as English, but\nnot for languages with a rich morphology and gender-marking, such as German.\nOur results highlight the importance of investigating bias and mitigation\ntechniques cross-linguistically, especially in view of the current emphasis on\nlarge-scale, multilingual language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14568v1",
            "date": "2020-10-27 19:19:38+00:00",
            "title": "Strongly Incremental Constituency Parsing with Graph Neural Networks",
            "authors": [
                "Kaiyu Yang",
                "Jia Deng"
            ],
            "abstract": "Parsing sentences into syntax trees can benefit downstream applications in\nNLP. Transition-based parsers build trees by executing actions in a state\ntransition system. They are computationally efficient, and can leverage machine\nlearning to predict actions based on partial trees. However, existing\ntransition-based parsers are predominantly based on the shift-reduce transition\nsystem, which does not align with how humans are known to parse sentences.\nPsycholinguistic research suggests that human parsing is strongly incremental:\nhumans grow a single parse tree by adding exactly one token at each step. In\nthis paper, we propose a novel transition system called attach-juxtapose. It is\nstrongly incremental; it represents a partial sentence using a single tree;\neach action adds exactly one token into the partial tree. Based on our\ntransition system, we develop a strongly incremental parser. At each step, it\nencodes the partial tree using a graph neural network and predicts an action.\nWe evaluate our parser on Penn Treebank (PTB) and Chinese Treebank (CTB). On\nPTB, it outperforms existing parsers trained with only constituency trees; and\nit performs on par with state-of-the-art parsers that use dependency trees as\nadditional training data. On CTB, our parser establishes a new state of the\nart. Code is available at\nhttps://github.com/princeton-vl/attach-juxtapose-parser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14576v3",
            "date": "2020-10-27 19:34:53+00:00",
            "title": "WNUT-2020 Task 1 Overview: Extracting Entities and Relations from Wet Lab Protocols",
            "authors": [
                "Jeniya Tabassum",
                "Sydney Lee",
                "Wei Xu",
                "Alan Ritter"
            ],
            "abstract": "This paper presents the results of the wet lab information extraction task at\nWNUT 2020. This task consisted of two sub tasks: (1) a Named Entity Recognition\n(NER) task with 13 participants and (2) a Relation Extraction (RE) task with 2\nparticipants. We outline the task, data annotation process, corpus statistics,\nand provide a high-level overview of the participating systems for each sub\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14584v2",
            "date": "2020-10-27 19:48:23+00:00",
            "title": "Predicting Themes within Complex Unstructured Texts: A Case Study on Safeguarding Reports",
            "authors": [
                "Aleksandra Edwards",
                "David Rogers",
                "Jose Camacho-Collados",
                "H\u00e9l\u00e8ne de Ribaupierre",
                "Alun Preece"
            ],
            "abstract": "The task of text and sentence classification is associated with the need for\nlarge amounts of labelled training data. The acquisition of high volumes of\nlabelled datasets can be expensive or unfeasible, especially for\nhighly-specialised domains for which documents are hard to obtain. Research on\nthe application of supervised classification based on small amounts of training\ndata is limited. In this paper, we address the combination of state-of-the-art\ndeep learning and classification methods and provide an insight into what\ncombination of methods fit the needs of small, domain-specific, and\nterminologically-rich corpora. We focus on a real-world scenario related to a\ncollection of safeguarding reports comprising learning experiences and\nreflections on tackling serious incidents involving children and vulnerable\nadults. The relatively small volume of available reports and their use of\nhighly domain-specific terminology makes the application of automated\napproaches difficult. We focus on the problem of automatically identifying the\nmain themes in a safeguarding report using supervised classification\napproaches. Our results show the potential of deep learning models to simulate\nsubject-expert behaviour even for complex tasks with limited labelled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14587v1",
            "date": "2020-10-27 19:51:04+00:00",
            "title": "On the diminishing return of labeling clinical reports",
            "authors": [
                "Jean-Baptiste Lamare",
                "Tobi Olatunji",
                "Li Yao"
            ],
            "abstract": "Ample evidence suggests that better machine learning models may be steadily\nobtained by training on increasingly larger datasets on natural language\nprocessing (NLP) problems from non-medical domains. Whether the same holds true\nfor medical NLP has by far not been thoroughly investigated. This work shows\nthat this is indeed not always the case. We reveal the somehow\ncounter-intuitive observation that performant medical NLP models may be\nobtained with small amount of labeled data, quite the opposite to the common\nbelief, most likely due to the domain specificity of the problem. We show\nquantitatively the effect of training data size on a fixed test set composed of\ntwo of the largest public chest x-ray radiology report datasets on the task of\nabnormality classification. The trained models not only make use of the\ntraining data efficiently, but also outperform the current state-of-the-art\nrule-based systems by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14678v1",
            "date": "2020-10-28 00:12:36+00:00",
            "title": "What Does This Acronym Mean? Introducing a New Dataset for Acronym Identification and Disambiguation",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Franck Dernoncourt",
                "Quan Hung Tran",
                "Thien Huu Nguyen"
            ],
            "abstract": "Acronyms are the short forms of phrases that facilitate conveying lengthy\nsentences in documents and serve as one of the mainstays of writing. Due to\ntheir importance, identifying acronyms and corresponding phrases (i.e., acronym\nidentification (AI)) and finding the correct meaning of each acronym (i.e.,\nacronym disambiguation (AD)) are crucial for text understanding. Despite the\nrecent progress on this task, there are some limitations in the existing\ndatasets which hinder further improvement. More specifically, limited size of\nmanually annotated AI datasets or noises in the automatically created acronym\nidentification datasets obstruct designing advanced high-performing acronym\nidentification models. Moreover, the existing datasets are mostly limited to\nthe medical domain and ignore other domains. In order to address these two\nlimitations, we first create a manually annotated large AI dataset for\nscientific domain. This dataset contains 17,506 sentences which is\nsubstantially larger than previous scientific AI datasets. Next, we prepare an\nAD dataset for scientific domain with 62,441 samples which is significantly\nlarger than the previous scientific AD dataset. Our experiments show that the\nexisting state-of-the-art models fall far behind human-level performance on\nboth datasets proposed by this work. In addition, we propose a new deep\nlearning model that utilizes the syntactical structure of the sentence to\nexpand an ambiguous acronym in a sentence. The proposed model outperforms the\nstate-of-the-art models on the new AD dataset, providing a strong baseline for\nfuture research on this dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14697v2",
            "date": "2020-10-28 01:53:59+00:00",
            "title": "Character Entropy in Modern and Historical Texts: Comparison Metrics for an Undeciphered Manuscript",
            "authors": [
                "Luke Lindemann",
                "Claire Bowern"
            ],
            "abstract": "This paper outlines the creation of three corpora for multilingual comparison\nand analysis of the Voynich manuscript: a corpus of Voynich texts partitioned\nby Currier language, scribal hand, and transcription system, a corpus of 294\nlanguage samples compiled from Wikipedia, and a corpus of eighteen transcribed\nhistorical texts in eight languages. These corpora will be utilized in\nsubsequent work by the Voynich Working Group at Yale University.\n  We demonstrate the utility of these corpora for studying characteristics of\nthe Voynich script and language, with an analysis of conditional character\nentropy in Voynichese. We discuss the interaction between character entropy and\nlanguage, script size and type, glyph compositionality, scribal conventions and\nabbreviations, positional character variants, and bigram frequency.\n  This analysis characterizes the interaction between script compositionality,\ncharacter size, and predictability. We show that substantial manipulations of\nglyph composition are not sufficient to align conditional entropy levels with\nnatural languages. The unusually predictable nature of the Voynichese script is\nnot attributable to a particular script or transcription system, underlying\nlanguage, or substitution cipher. Voynichese is distinct from every comparison\ntext in our corpora because character placement is highly constrained within\nthe word, and this may indicate the loss of phonemic distinctions from the\nunderlying language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14720v1",
            "date": "2020-10-28 03:01:33+00:00",
            "title": "Second-Order Unsupervised Neural Dependency Parsing",
            "authors": [
                "Songlin Yang",
                "Yong Jiang",
                "Wenjuan Han",
                "Kewei Tu"
            ],
            "abstract": "Most of the unsupervised dependency parsers are based on first-order\nprobabilistic generative models that only consider local parent-child\ninformation. Inspired by second-order supervised dependency parsing, we\nproposed a second-order extension of unsupervised neural dependency models that\nincorporate grandparent-child or sibling information. We also propose a novel\ndesign of the neural parameterization and optimization methods of the\ndependency models. In second-order models, the number of grammar rules grows\ncubically with the increase of vocabulary size, making it difficult to train\nlexicalized models that may contain thousands of words. To circumvent this\nproblem while still benefiting from both second-order parsing and\nlexicalization, we use the agreement-based learning framework to jointly train\na second-order unlexicalized model and a first-order lexicalized model.\nExperiments on multiple datasets show the effectiveness of our second-order\nmodels compared with recent state-of-the-art methods. Our joint model achieves\na 10% improvement over the previous state-of-the-art parser on the full WSJ\ntest set",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14759v2",
            "date": "2020-10-26 22:30:17+00:00",
            "title": "Fine-grained Information Status Classification Using Discourse Context-Aware BERT",
            "authors": [
                "Yufang Hou"
            ],
            "abstract": "Previous work on bridging anaphora recognition (Hou et al., 2013a) casts the\nproblem as a subtask of learning fine-grained information status (IS). However,\nthese systems heavily depend on many hand-crafted linguistic features. In this\npaper, we propose a simple discourse context-aware BERT model for fine-grained\nIS classification. On the ISNotes corpus (Markert et al., 2012), our model\nachieves new state-of-the-art performance on fine-grained IS classification,\nobtaining a 4.8 absolute overall accuracy improvement compared to Hou et al.\n(2013a). More importantly, we also show an improvement of 10.5 F1 points for\nbridging anaphora recognition without using any complex hand-crafted semantic\nfeatures designed for capturing the bridging phenomenon. We further analyze the\ntrained model and find that the most attended signals for each IS category\ncorrespond well to linguistic notions of information status.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14806v2",
            "date": "2020-10-28 08:08:12+00:00",
            "title": "The Volctrans Machine Translation System for WMT20",
            "authors": [
                "Liwei Wu",
                "Xiao Pan",
                "Zehui Lin",
                "Yaoming Zhu",
                "Mingxuan Wang",
                "Lei Li"
            ],
            "abstract": "This paper describes our VolcTrans system on WMT20 shared news translation\ntask. We participated in 8 translation directions. Our basic systems are based\non Transformer, with several variants (wider or deeper Transformers, dynamic\nconvolutions). The final system includes text pre-process, data selection,\nsynthetic data generation, advanced model ensemble, and multilingual\npre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14920v1",
            "date": "2020-10-28 12:33:04+00:00",
            "title": "Bridging the Modality Gap for Speech-to-Text Translation",
            "authors": [
                "Yuchen Liu",
                "Junnan Zhu",
                "Jiajun Zhang",
                "Chengqing Zong"
            ],
            "abstract": "End-to-end speech translation aims to translate speech in one language into\ntext in another language via an end-to-end way. Most existing methods employ an\nencoder-decoder structure with a single encoder to learn acoustic\nrepresentation and semantic information simultaneously, which ignores the\nspeech-and-text modality differences and makes the encoder overloaded, leading\nto great difficulty in learning such a model. To address these issues, we\npropose a Speech-to-Text Adaptation for Speech Translation (STAST) model which\naims to improve the end-to-end model performance by bridging the modality gap\nbetween speech and text. Specifically, we decouple the speech translation\nencoder into three parts and introduce a shrink mechanism to match the length\nof speech representation with that of the corresponding text transcription. To\nobtain better semantic representation, we completely integrate a text-based\ntranslation model into the STAST so that two tasks can be trained in the same\nlatent space. Furthermore, we introduce a cross-modal adaptation method to\nclose the distance between speech and text representation. Experimental results\non English-French and English-German speech translation corpora have shown that\nour model significantly outperforms strong baselines, and achieves the new\nstate-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14952v1",
            "date": "2020-10-28 13:10:24+00:00",
            "title": "Towards Ethics by Design in Online Abusive Content Detection",
            "authors": [
                "Svetlana Kiritchenko",
                "Isar Nejadgholi"
            ],
            "abstract": "To support safety and inclusion in online communications, significant efforts\nin NLP research have been put towards addressing the problem of abusive content\ndetection, commonly defined as a supervised classification task. The research\neffort has spread out across several closely related sub-areas, such as\ndetection of hate speech, toxicity, cyberbullying, etc. There is a pressing\nneed to consolidate the field under a common framework for task formulation,\ndataset design and performance evaluation. Further, despite current\ntechnologies achieving high classification accuracies, several ethical issues\nhave been revealed. We bring ethical issues to forefront and propose a unified\nframework as a two-step process. First, online content is categorized around\npersonal and identity-related subject matters. Second, severity of abuse is\nidentified through comparative annotation within each category. The novel\nframework is guided by the Ethics by Design principle and is a step towards\nbuilding more accurate and trusted models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15036v1",
            "date": "2020-10-28 15:15:13+00:00",
            "title": "A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models",
            "authors": [
                "Usman Naseem",
                "Imran Razzak",
                "Shah Khalid Khan",
                "Mukesh Prasad"
            ],
            "abstract": "Word representation has always been an important research area in the history\nof natural language processing (NLP). Understanding such complex text data is\nimperative, given that it is rich in information and can be used widely across\nvarious applications. In this survey, we explore different word representation\nmodels and its power of expression, from the classical to modern-day\nstate-of-the-art word representation language models (LMS). We describe a\nvariety of text representation methods, and model designs have blossomed in the\ncontext of NLP, including SOTA LMs. These models can transform large volumes of\ntext into effective vector representations capturing the same semantic\ninformation. Further, such representations can be utilized by various machine\nlearning (ML) algorithms for a variety of NLP related tasks. In the end, this\nsurvey briefly discusses the commonly used ML and DL based classifiers,\nevaluation metrics and the applications of these word embeddings in different\nNLP tasks.",
            "journal ref": "ACM Transactions on Asian and Low-Resource Language Information\n  Processing (TALLIP) 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15225v1",
            "date": "2020-10-28 20:47:13+00:00",
            "title": "A Visuospatial Dataset for Naturalistic Verb Learning",
            "authors": [
                "Dylan Ebert",
                "Ellie Pavlick"
            ],
            "abstract": "We introduce a new dataset for training and evaluating grounded language\nmodels. Our data is collected within a virtual reality environment and is\ndesigned to emulate the quality of language data to which a pre-verbal child is\nlikely to have access: That is, naturalistic, spontaneous speech paired with\nrichly grounded visuospatial context. We use the collected data to compare\nseveral distributional semantics models for verb learning. We evaluate neural\nmodels based on 2D (pixel) features as well as feature-engineered models based\non 3D (symbolic, spatial) features, and show that neither modeling approach\nachieves satisfactory performance. Our results are consistent with evidence\nfrom child language acquisition that emphasizes the difficulty of learning\nverbs from naive distributional data. We discuss avenues for future work on\ncognitively-inspired grounded language learning, and release our corpus with\nthe intent of facilitating research on the topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15313v1",
            "date": "2020-10-29 02:07:05+00:00",
            "title": "\"where is this relationship going?\": Understanding Relationship Trajectories in Narrative Text",
            "authors": [
                "Keen You",
                "Dan Goldwasser"
            ],
            "abstract": "We examine a new commonsense reasoning task: given a narrative describing a\nsocial interaction that centers on two protagonists, systems make inferences\nabout the underlying relationship trajectory. Specifically, we propose two\nevaluation tasks: Relationship Outlook Prediction MCQ and Resolution Prediction\nMCQ. In Relationship Outlook Prediction, a system maps an interaction to a\nrelationship outlook that captures how the interaction is expected to change\nthe relationship. In Resolution Prediction, a system attributes a given\nrelationship outlook to a particular resolution that explains the outcome.\nThese two tasks parallel two real-life questions that people frequently ponder\nupon as they navigate different social situations: \"where is this relationship\ngoing?\" and \"how did we end up here?\". To facilitate the investigation of human\nsocial relationships through these two tasks, we construct a new dataset,\nSocial Narrative Tree, which consists of 1250 stories documenting a variety of\ndaily social interactions. The narratives encode a multitude of social elements\nthat interweave to give rise to rich commonsense knowledge of how relationships\nevolve with respect to social interactions. We establish baseline performances\nusing language models and the accuracies are significantly lower than human\nperformance. The results demonstrate that models need to look beyond syntactic\nand semantic signals to comprehend complex human relationships.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15360v1",
            "date": "2020-10-29 05:29:26+00:00",
            "title": "Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection",
            "authors": [
                "Shaolei Wang",
                "Zhongyuan Wang",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "Most existing approaches to disfluency detection heavily rely on\nhuman-annotated corpora, which is expensive to obtain in practice. There have\nbeen several proposals to alleviate this issue with, for instance,\nself-supervised learning techniques, but they still require human-annotated\ncorpora. In this work, we explore the unsupervised learning paradigm which can\npotentially work with unlabeled text corpora that are cheaper and easier to\nobtain. Our model builds upon the recent work on Noisy Student Training, a\nsemi-supervised learning approach that extends the idea of self-training.\nExperimental results on the commonly used English Switchboard test set show\nthat our approach achieves competitive performance compared to the previous\nstate-of-the-art supervised systems using contextualized word embeddings (e.g.\nBERT and ELECTRA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15411v2",
            "date": "2020-10-29 08:23:24+00:00",
            "title": "Conversation Graph: Data Augmentation, Training and Evaluation for Non-Deterministic Dialogue Management",
            "authors": [
                "Milan Gritta",
                "Gerasimos Lampouras",
                "Ignacio Iacobacci"
            ],
            "abstract": "Task-oriented dialogue systems typically rely on large amounts of\nhigh-quality training data or require complex handcrafted rules. However,\nexisting datasets are often limited in size considering the complexity of the\ndialogues. Additionally, conventional training signal inference is not suitable\nfor non-deterministic agent behaviour, i.e. considering multiple actions as\nvalid in identical dialogue states. We propose the Conversation Graph\n(ConvGraph), a graph-based representation of dialogues that can be exploited\nfor data augmentation, multi-reference training and evaluation of\nnon-deterministic agents. ConvGraph generates novel dialogue paths to augment\ndata volume and diversity. Intrinsic and extrinsic evaluation across three\ndatasets shows that data augmentation and/or multi-reference training with\nConvGraph can improve dialogue success rates by up to 6.4%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15423v1",
            "date": "2020-10-29 08:59:37+00:00",
            "title": "Tilde at WMT 2020: News Task Systems",
            "authors": [
                "Rihards Kri\u0161lauks",
                "M\u0101rcis Pinnis"
            ],
            "abstract": "This paper describes Tilde's submission to the WMT2020 shared task on news\ntranslation for both directions of the English-Polish language pair in both the\nconstrained and the unconstrained tracks. We follow our submissions from the\nprevious years and build our baseline systems to be morphologically motivated\nsub-word unit-based Transformer base models that we train using the Marian\nmachine translation toolkit. Additionally, we experiment with different\nparallel and monolingual data selection schemes, as well as sampled\nback-translation. Our final models are ensembles of Transformer base and\nTransformer big models that feature right-to-left re-ranking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15437v1",
            "date": "2020-10-29 09:16:23+00:00",
            "title": "Memory Attentive Fusion: External Language Model Integration for Transformer-based Sequence-to-Sequence Model",
            "authors": [
                "Mana Ihori",
                "Ryo Masumura",
                "Naoki Makishima",
                "Tomohiro Tanaka",
                "Akihiko Takashima",
                "Shota Orihashi"
            ],
            "abstract": "This paper presents a novel fusion method for integrating an external\nlanguage model (LM) into the Transformer based sequence-to-sequence (seq2seq)\nmodel. While paired data are basically required to train the seq2seq model, the\nexternal LM can be trained with only unpaired data. Thus, it is important to\nleverage memorized knowledge in the external LM for building the seq2seq model,\nsince it is hard to prepare a large amount of paired data. However, the\nexisting fusion methods assume that the LM is integrated with recurrent neural\nnetwork-based seq2seq models instead of the Transformer. Therefore, this paper\nproposes a fusion method that can explicitly utilize network structures in the\nTransformer. The proposed method, called {\\bf memory attentive fusion},\nleverages the Transformer-style attention mechanism that repeats source-target\nattention in a multi-hop manner for reading the memorized knowledge in the LM.\nOur experiments on two text-style conversion tasks demonstrate that the\nproposed method performs better than conventional fusion methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15458v1",
            "date": "2020-10-29 10:06:46+00:00",
            "title": "Named Entity Recognition for Social Media Texts with Semantic Augmentation",
            "authors": [
                "Yuyang Nie",
                "Yuanhe Tian",
                "Xiang Wan",
                "Yan Song",
                "Bo Dai"
            ],
            "abstract": "Existing approaches for named entity recognition suffer from data sparsity\nproblems when conducted on short and informal texts, especially user-generated\nsocial media content. Semantic augmentation is a potential way to alleviate\nthis problem. Given that rich semantic information is implicitly preserved in\npre-trained word embeddings, they are potential ideal resources for semantic\naugmentation. In this paper, we propose a neural-based approach to NER for\nsocial media texts where both local (from running text) and augmented semantics\nare taken into account. In particular, we obtain the augmented semantic\ninformation from a large-scale corpus, and propose an attentive semantic\naugmentation module and a gate module to encode and aggregate such information,\nrespectively. Extensive experiments are performed on three benchmark datasets\ncollected from English and Chinese social media platforms, where the results\ndemonstrate the superiority of our approach to previous studies across all\nthree datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15466v1",
            "date": "2020-10-29 10:25:17+00:00",
            "title": "Improving Named Entity Recognition with Attentive Ensemble of Syntactic Information",
            "authors": [
                "Yuyang Nie",
                "Yuanhe Tian",
                "Yan Song",
                "Xiang Ao",
                "Xiang Wan"
            ],
            "abstract": "Named entity recognition (NER) is highly sensitive to sentential syntactic\nand semantic properties where entities may be extracted according to how they\nare used and placed in the running text. To model such properties, one could\nrely on existing resources to providing helpful knowledge to the NER task; some\nexisting studies proved the effectiveness of doing so, and yet are limited in\nappropriately leveraging the knowledge such as distinguishing the important\nones for particular context. In this paper, we improve NER by leveraging\ndifferent types of syntactic information through attentive ensemble, which\nfunctionalizes by the proposed key-value memory networks, syntax attention, and\nthe gate mechanism for encoding, weighting and aggregating such syntactic\ninformation, respectively. Experimental results on six English and Chinese\nbenchmark datasets suggest the effectiveness of the proposed model and show\nthat it outperforms previous studies on all experiment datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15535v1",
            "date": "2020-10-29 12:59:44+00:00",
            "title": "Unbabel's Participation in the WMT20 Metrics Shared Task",
            "authors": [
                "Ricardo Rei",
                "Craig Stewart",
                "Catarina Farinha",
                "Alon Lavie"
            ],
            "abstract": "We present the contribution of the Unbabel team to the WMT 2020 Shared Task\non Metrics. We intend to participate on the segment-level, document-level and\nsystem-level tracks on all language pairs, as well as the 'QE as a Metric'\ntrack. Accordingly, we illustrate results of our models in these tracks with\nreference to test sets from the previous year. Our submissions build upon the\nrecently proposed COMET framework: We train several estimator models to regress\non different human-generated quality scores and a novel ranking model trained\non relative ranks obtained from Direct Assessments. We also propose a simple\ntechnique for converting segment-level predictions into a document-level score.\nOverall, our systems achieve strong results for all language pairs on previous\ntest sets and in many cases set a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15598v1",
            "date": "2020-10-29 13:53:42+00:00",
            "title": "May I Ask Who's Calling? Named Entity Recognition on Call Center Transcripts for Privacy Law Compliance",
            "authors": [
                "Micaela Kaplan"
            ],
            "abstract": "We investigate using Named Entity Recognition on a new type of user-generated\ntext: a call center conversation. These conversations combine problems from\nspontaneous speech with problems novel to conversational Automated Speech\nRecognition, including incorrect recognition, alongside other common problems\nfrom noisy user-generated text. Using our own corpus with new annotations,\ntraining custom contextual string embeddings, and applying a BiLSTM-CRF, we\nmatch state-of-the-art results on our novel task.",
            "journal ref": "Proceedings of the 2020 EMNLP Workshop W-NUT: The Sixth Workshop\n  on Noisy User-generated Text (2020) 1-6",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15778v1",
            "date": "2020-10-29 17:25:20+00:00",
            "title": "Contextual BERT: Conditioning the Language Model Using a Global State",
            "authors": [
                "Timo I. Denk",
                "Ana Peleteiro Ramallo"
            ],
            "abstract": "BERT is a popular language model whose main pre-training task is to fill in\nthe blank, i.e., predicting a word that was masked out of a sentence, based on\nthe remaining words. In some applications, however, having an additional\ncontext can help the model make the right prediction, e.g., by taking the\ndomain or the time of writing into account. This motivates us to advance the\nBERT architecture by adding a global state for conditioning on a fixed-sized\ncontext. We present our two novel approaches and apply them to an industry\nuse-case, where we complete fashion outfits with missing articles, conditioned\non a specific customer. An experimental comparison to other methods from the\nliterature shows that our methods improve personalization significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15909v2",
            "date": "2020-10-29 19:49:17+00:00",
            "title": "Learning as Abduction: Trainable Natural Logic Theorem Prover for Natural Language Inference",
            "authors": [
                "Lasha Abzianidze"
            ],
            "abstract": "Tackling Natural Language Inference with a logic-based method is becoming\nless and less common. While this might have been counterintuitive several\ndecades ago, nowadays it seems pretty obvious. The main reasons for such a\nconception are that (a) logic-based methods are usually brittle when it comes\nto processing wide-coverage texts, and (b) instead of automatically learning\nfrom data, they require much of manual effort for development. We make a step\ntowards to overcome such shortcomings by modeling learning from data as\nabduction: reversing a theorem-proving procedure to abduce semantic relations\nthat serve as the best explanation for the gold label of an inference problem.\nIn other words, instead of proving sentence-level inference relations with the\nhelp of lexical relations, the lexical relations are proved taking into account\nthe sentence-level inference relations. We implement the learning method in a\ntableau theorem prover for natural language and show that it improves the\nperformance of the theorem prover on the SICK dataset by 1.4% while still\nmaintaining high precision (>94%). The obtained results are competitive with\nthe state of the art among logic-based systems.",
            "journal ref": "Proceedings of the Ninth Joint Conference on Lexical and\n  Computational Semantics (*SEM). ACL. pp. 20-31. 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16021v2",
            "date": "2020-10-30 02:06:10+00:00",
            "title": "CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering",
            "authors": [
                "Xiang Yue",
                "Xinliang Frederick Zhang",
                "Ziyu Yao",
                "Simon Lin",
                "Huan Sun"
            ],
            "abstract": "Clinical question answering (QA) aims to automatically answer questions from\nmedical professionals based on clinical texts. Studies show that neural QA\nmodels trained on one corpus may not generalize well to new clinical texts from\na different institute or a different patient group, where large-scale QA pairs\nare not readily available for retraining. To address this challenge, we propose\na simple yet effective framework, CliniQG4QA, which leverages question\ngeneration (QG) to synthesize QA pairs on new clinical contexts and boosts QA\nmodels without requiring manual annotations. In order to generate diverse types\nof questions that are essential for training QA models, we further introduce a\nseq2seq-based question phrase prediction (QPP) module that can be used together\nwith most existing QG models to diversify their generation. Our comprehensive\nexperiment results show that the QA corpus generated by our framework is\nhelpful to improve QA models on the new contexts (up to 8% absolute gain in\nterms of Exact Match), and that the QPP module plays a crucial role in\nachieving the gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16046v2",
            "date": "2020-10-30 03:41:38+00:00",
            "title": "VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation",
            "authors": [
                "Fuli Luo",
                "Wei Wang",
                "Jiahao Liu",
                "Yijia Liu",
                "Bin Bi",
                "Songfang Huang",
                "Fei Huang",
                "Luo Si"
            ],
            "abstract": "Existing work in multilingual pretraining has demonstrated the potential of\ncross-lingual transferability by training a unified Transformer encoder for\nmultiple languages. However, much of this work only relies on the shared\nvocabulary and bilingual contexts to encourage the correlation across\nlanguages, which is loose and implicit for aligning the contextual\nrepresentations between languages. In this paper, we plug a cross-attention\nmodule into the Transformer encoder to explicitly build the interdependence\nbetween languages. It can effectively avoid the degeneration of predicting\nmasked words only conditioned on the context in its own language. More\nimportantly, when fine-tuning on downstream tasks, the cross-attention module\ncan be plugged in or out on-demand, thus naturally benefiting a wider range of\ncross-lingual tasks, from language understanding to generation.\n  As a result, the proposed cross-lingual model delivers new state-of-the-art\nresults on various cross-lingual understanding tasks of the XTREME benchmark,\ncovering text classification, sequence labeling, question answering, and\nsentence retrieval. For cross-lingual generation tasks, it also outperforms all\nexisting cross-lingual models and state-of-the-art Transformer variants on\nWMT14 English-to-German and English-to-French translation datasets, with gains\nof up to 1~2 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16056v1",
            "date": "2020-10-30 04:08:03+00:00",
            "title": "Generating Radiology Reports via Memory-driven Transformer",
            "authors": [
                "Zhihong Chen",
                "Yan Song",
                "Tsung-Hui Chang",
                "Xiang Wan"
            ],
            "abstract": "Medical imaging is frequently used in clinical practice and trials for\ndiagnosis and treatment. Writing imaging reports is time-consuming and can be\nerror-prone for inexperienced radiologists. Therefore, automatically generating\nradiology reports is highly desired to lighten the workload of radiologists and\naccordingly promote clinical automation, which is an essential task to apply\nartificial intelligence to the medical domain. In this paper, we propose to\ngenerate radiology reports with memory-driven Transformer, where a relational\nmemory is designed to record key information of the generation process and a\nmemory-driven conditional layer normalization is applied to incorporating the\nmemory into the decoder of Transformer. Experimental results on two prevailing\nradiology report datasets, IU X-Ray and MIMIC-CXR, show that our proposed\napproach outperforms previous models with respect to both language generation\nmetrics and clinical evaluations. Particularly, this is the first work\nreporting the generation results on MIMIC-CXR to the best of our knowledge.\nFurther analyses also demonstrate that our approach is able to generate long\nreports with necessary medical terms as well as meaningful image-text attention\nmappings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16059v1",
            "date": "2020-10-30 04:18:39+00:00",
            "title": "Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction",
            "authors": [
                "Haiyang Yu",
                "Ningyu Zhang",
                "Shumin Deng",
                "Hongbin Ye",
                "Wei Zhang",
                "Huajun Chen"
            ],
            "abstract": "Current supervised relational triple extraction approaches require huge\namounts of labeled data and thus suffer from poor performance in few-shot\nsettings. However, people can grasp new knowledge by learning a few instances.\nTo this end, we take the first step to study the few-shot relational triple\nextraction, which has not been well understood. Unlike previous single-task\nfew-shot problems, relational triple extraction is more challenging as the\nentities and relations have implicit correlations. In this paper, We propose a\nnovel multi-prototype embedding network model to jointly extract the\ncomposition of relational triples, namely, entity pairs and corresponding\nrelations. To be specific, we design a hybrid prototypical learning mechanism\nthat bridges text and knowledge concerning both entities and relations. Thus,\nimplicit correlations between entities and relations are injected.\nAdditionally, we propose a prototype-aware regularization to learn more\nrepresentative prototypes. Experimental results demonstrate that the proposed\nmethod can improve the performance of the few-shot triple extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16068v1",
            "date": "2020-10-30 04:30:09+00:00",
            "title": "Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification",
            "authors": [
                "Juan Li",
                "Ruoxu Wang",
                "Ningyu Zhang",
                "Wen Zhang",
                "Fan Yang",
                "Huajun Chen"
            ],
            "abstract": "Relation classification aims to extract semantic relations between entity\npairs from the sentences. However, most existing methods can only identify seen\nrelation classes that occurred during training. To recognize unseen relations\nat test time, we explore the problem of zero-shot relation classification.\nPrevious work regards the problem as reading comprehension or textual\nentailment, which have to rely on artificial descriptive information to improve\nthe understandability of relation types. Thus, rich semantic knowledge of the\nrelation labels is ignored. In this paper, we propose a novel logic-guided\nsemantic representation learning model for zero-shot relation classification.\nOur approach builds connections between seen and unseen relations via implicit\nand explicit semantic representations with knowledge graph embeddings and logic\nrules. Extensive experimental results demonstrate that our method can\ngeneralize to unseen relation types and achieve promising improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16097v1",
            "date": "2020-10-30 06:34:44+00:00",
            "title": "Target Word Masking for Location Metonymy Resolution",
            "authors": [
                "Haonan Li",
                "Maria Vasardani",
                "Martin Tomko",
                "Timothy Baldwin"
            ],
            "abstract": "Existing metonymy resolution approaches rely on features extracted from\nexternal resources like dictionaries and hand-crafted lexical resources. In\nthis paper, we propose an end-to-end word-level classification approach based\nonly on BERT, without dependencies on taggers, parsers, curated dictionaries of\nplace names, or other external resources. We show that our approach achieves\nthe state-of-the-art on 5 datasets, surpassing conventional BERT models and\nbenchmarks by a large margin. We also show that our approach generalises well\nto unseen data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16275v1",
            "date": "2020-10-30 13:52:52+00:00",
            "title": "Towards Accurate and Consistent Evaluation: A Dataset for Distantly-Supervised Relation Extraction",
            "authors": [
                "Tong Zhu",
                "Haitao Wang",
                "Junjie Yu",
                "Xiabing Zhou",
                "Wenliang Chen",
                "Wei Zhang",
                "Min Zhang"
            ],
            "abstract": "In recent years, distantly-supervised relation extraction has achieved a\ncertain success by using deep neural networks. Distant Supervision (DS) can\nautomatically generate large-scale annotated data by aligning entity pairs from\nKnowledge Bases (KB) to sentences. However, these DS-generated datasets\ninevitably have wrong labels that result in incorrect evaluation scores during\ntesting, which may mislead the researchers. To solve this problem, we build a\nnew dataset NYTH, where we use the DS-generated data as training data and hire\nannotators to label test data. Compared with the previous datasets, NYT-H has a\nmuch larger test set and then we can perform more accurate and consistent\nevaluation. Finally, we present the experimental results of several widely used\nsystems on NYT-H. The experimental results show that the ranking lists of the\ncomparison systems on the DS-labelled test data and human-annotated test data\nare different. This indicates that our human-annotated data is necessary for\nevaluation of distantly-supervised relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16408v1",
            "date": "2020-10-05 16:19:00+00:00",
            "title": "Sentiment Analysis for Roman Urdu Text over Social Media, a Comparative Study",
            "authors": [
                "Irfan Qutab",
                "Khawar Iqbal Malik",
                "Hira Arooj"
            ],
            "abstract": "In present century, data volume is increasing enormously. The data could be\nin form for image, text, voice, and video. One factor in this huge growth of\ndata is usage of social media where everyone is posting data on daily basis\nduring chatting, exchanging information, and uploading their personal and\nofficial credential. Research of sentiments seeks to uncover abstract knowledge\nin Published texts in which users communicate their emotions and thoughts about\nshared content, including blogs, news and social networks. Roman Urdu is the\none of most dominant language on social networks in Pakistan and India. Roman\nUrdu is among the varieties of the world's third largest Urdu language but yet\nnot sufficient work has been done in this language. In this article we\naddressed the prior concepts and strategies used to examine the sentiment of\nthe roman Urdu text and reported their results as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16411v1",
            "date": "2020-10-09 01:20:09+00:00",
            "title": "Mere account mein kitna balance hai? -- On building voice enabled Banking Services for Multilingual Communities",
            "authors": [
                "Akshat Gupta",
                "Sai Krishna Rallabandi",
                "Alan W Black"
            ],
            "abstract": "Tremendous progress in speech and language processing has brought language\ntechnologies closer to daily human life. Voice technology has the potential to\nact as a horizontal enabling layer across all aspects of digitization. It is\nespecially beneficial to rural communities in scenarios like a pandemic. In\nthis work we present our initial exploratory work towards one such direction --\nbuilding voice enabled banking services for multilingual societies. Speech\ninteraction for typical banking transactions in multilingual communities\ninvolves the presence of filled pauses and is characterized by Code Mixing.\nCode Mixing is a phenomenon where lexical items from one language are embedded\nin the utterance of another. Therefore speech systems deployed for banking\napplications should be able to process such content. In our work we investigate\nvarious training strategies for building speech based intent recognition\nsystems. We present our results using a Naive Bayes classifier on approximate\nacoustic phone units using the Allosaurus library.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00033v1",
            "date": "2020-10-30 18:28:42+00:00",
            "title": "Streaming Simultaneous Speech Translation with Augmented Memory Transformer",
            "authors": [
                "Xutai Ma",
                "Yongqiang Wang",
                "Mohammad Javad Dousti",
                "Philipp Koehn",
                "Juan Pino"
            ],
            "abstract": "Transformer-based models have achieved state-of-the-art performance on speech\ntranslation tasks. However, the model architecture is not efficient enough for\nstreaming scenarios since self-attention is computed over an entire input\nsequence and the computational cost grows quadratically with the length of the\ninput sequence. Nevertheless, most of the previous work on simultaneous speech\ntranslation, the task of generating translations from partial audio input,\nignores the time spent in generating the translation when analyzing the\nlatency. With this assumption, a system may have good latency quality\ntrade-offs but be inapplicable in real-time scenarios. In this paper, we focus\non the task of streaming simultaneous speech translation, where the systems are\nnot only capable of translating with partial input but are also able to handle\nvery long or continuous input. We propose an end-to-end transformer-based\nsequence-to-sequence model, equipped with an augmented memory transformer\nencoder, which has shown great success on the streaming automatic speech\nrecognition task with hybrid or transducer-based models. We conduct an\nempirical evaluation of the proposed model on segment, context and memory sizes\nand we compare our approach to a transformer with a unidirectional mask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00092v1",
            "date": "2020-10-30 20:26:41+00:00",
            "title": "Analyzing Gender Bias within Narrative Tropes",
            "authors": [
                "Dhruvil Gala",
                "Mohammad Omar Khursheed",
                "Hannah Lerner",
                "Brendan O'Connor",
                "Mohit Iyyer"
            ],
            "abstract": "Popular media reflects and reinforces societal biases through the use of\ntropes, which are narrative elements, such as archetypal characters and plot\narcs, that occur frequently across media. In this paper, we specifically\ninvestigate gender bias within a large collection of tropes. To enable our\nstudy, we crawl tvtropes.org, an online user-created repository that contains\n30K tropes associated with 1.9M examples of their occurrences across film,\ntelevision, and literature. We automatically score the \"genderedness\" of each\ntrope in our TVTROPES dataset, which enables an analysis of (1) highly-gendered\ntopics within tropes, (2) the relationship between gender bias and popular\nreception, and (3) how the gender of a work's creator correlates with the types\nof tropes that they use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00169v1",
            "date": "2020-10-31 02:21:43+00:00",
            "title": "Understanding Pre-trained BERT for Aspect-based Sentiment Analysis",
            "authors": [
                "Hu Xu",
                "Lei Shu",
                "Philip S. Yu",
                "Bing Liu"
            ],
            "abstract": "This paper analyzes the pre-trained hidden representations learned from\nreviews on BERT for tasks in aspect-based sentiment analysis (ABSA). Our work\nis motivated by the recent progress in BERT-based language models for ABSA.\nHowever, it is not clear how the general proxy task of (masked) language model\ntrained on unlabeled corpus without annotations of aspects or opinions can\nprovide important features for downstream tasks in ABSA. By leveraging the\nannotated datasets in ABSA, we investigate both the attentions and the learned\nrepresentations of BERT pre-trained on reviews. We found that BERT uses very\nfew self-attention heads to encode context words (such as prepositions or\npronouns that indicating an aspect) and opinion words for an aspect. Most\nfeatures in the representation of an aspect are dedicated to the fine-grained\nsemantics of the domain (or product category) and the aspect itself, instead of\ncarrying summarized opinions from its context. We hope this investigation can\nhelp future research in improving self-supervised learning, unsupervised\nlearning and fine-tuning for ABSA. The pre-trained model and code can be found\nat https://github.com/howardhsu/BERT-for-RRC-ABSA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00244v2",
            "date": "2020-10-31 11:14:16+00:00",
            "title": "Evaluating Bias In Dutch Word Embeddings",
            "authors": [
                "Rodrigo Alejandro Ch\u00e1vez Mulsa",
                "Gerasimos Spanakis"
            ],
            "abstract": "Recent research in Natural Language Processing has revealed that word\nembeddings can encode social biases present in the training data which can\naffect minorities in real world applications. This paper explores the gender\nbias implicit in Dutch embeddings while investigating whether English language\nbased approaches can also be used in Dutch. We implement the Word Embeddings\nAssociation Test (WEAT), Clustering and Sentence Embeddings Association Test\n(SEAT) methods to quantify the gender bias in Dutch word embeddings, then we\nproceed to reduce the bias with Hard-Debias and Sent-Debias mitigation methods\nand finally we evaluate the performance of the debiased embeddings in\ndownstream tasks. The results suggest that, among others, gender bias is\npresent in traditional and contextualized Dutch word embeddings. We highlight\nhow techniques used to measure and reduce bias created for English can be used\nin Dutch embeddings by adequately translating the data and taking into account\nthe unique characteristics of the language. Furthermore, we analyze the effect\nof the debiasing techniques on downstream tasks which show a negligible impact\non traditional embeddings and a 2% decrease in performance in contextualized\nembeddings. Finally, we release the translated Dutch datasets to the public\nalong with the traditional embeddings with mitigated bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00245v1",
            "date": "2020-10-31 11:21:39+00:00",
            "title": "Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution",
            "authors": [
                "Juntao Yu",
                "Nafise Sadat Moosavi",
                "Silviu Paun",
                "Massimo Poesio"
            ],
            "abstract": "Now that the performance of coreference resolvers on the simpler forms of\nanaphoric reference has greatly improved, more attention is devoted to more\ncomplex aspects of anaphora. One limitation of virtually all coreference\nresolution models is the focus on single-antecedent anaphors. Plural anaphors\nwith multiple antecedents-so-called split-antecedent anaphors (as in John met\nMary. They went to the movies) have not been widely studied, because they are\nnot annotated in ONTONOTES and are relatively infrequent in other corpora. In\nthis paper, we introduce the first model for unrestricted resolution of\nsplit-antecedent anaphors. We start with a strong baseline enhanced by BERT\nembeddings, and show that we can substantially improve its performance by\naddressing the sparsity issue. To do this, we experiment with auxiliary corpora\nwhere split-antecedent anaphors were annotated by the crowd, and with transfer\nlearning models using element-of bridging references and single-antecedent\ncoreference as auxiliary tasks. Evaluation on the gold annotated ARRAU corpus\nshows that the out best model uses a combination of three auxiliary corpora\nachieved F1 scores of 70% and 43.6% when evaluated in a lenient and strict\nsetting, respectively, i.e., 11 and 21 percentage points gain when compared\nwith our baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00259v2",
            "date": "2020-10-31 12:29:25+00:00",
            "title": "Rumor Detection on Twitter Using Multiloss Hierarchical BiLSTM with an Attenuation Factor",
            "authors": [
                "Yudianto Sujana",
                "Jiawen Li",
                "Hung-Yu Kao"
            ],
            "abstract": "Social media platforms such as Twitter have become a breeding ground for\nunverified information or rumors. These rumors can threaten people's health,\nendanger the economy, and affect the stability of a country. Many researchers\nhave developed models to classify rumors using traditional machine learning or\nvanilla deep learning models. However, previous studies on rumor detection have\nachieved low precision and are time consuming. Inspired by the hierarchical\nmodel and multitask learning, a multiloss hierarchical BiLSTM model with an\nattenuation factor is proposed in this paper. The model is divided into two\nBiLSTM modules: post level and event level. By means of this hierarchical\nstructure, the model can extract deep in-formation from limited quantities of\ntext. Each module has a loss function that helps to learn bilateral features\nand reduce the training time. An attenuation fac-tor is added at the post level\nto increase the accuracy. The results on two rumor datasets demonstrate that\nour model achieves better performance than that of state-of-the-art machine\nlearning and vanilla deep learning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00286v1",
            "date": "2020-10-31 14:34:43+00:00",
            "title": "Neural Coreference Resolution for Arabic",
            "authors": [
                "Abdulrahman Aloraini",
                "Juntao Yu",
                "Massimo Poesio"
            ],
            "abstract": "No neural coreference resolver for Arabic exists, in fact we are not aware of\nany learning-based coreference resolver for Arabic since (Bjorkelund and Kuhn,\n2014). In this paper, we introduce a coreference resolution system for Arabic\nbased on Lee et al's end to end architecture combined with the Arabic version\nof bert and an external mention detector. As far as we know, this is the first\nneural coreference resolution system aimed specifically to Arabic, and it\nsubstantially outperforms the existing state of the art on OntoNotes 5.0 with a\ngain of 15.2 points conll F1. We also discuss the current limitations of the\ntask for Arabic and possible approaches that can tackle these challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00310v1",
            "date": "2020-10-31 16:48:55+00:00",
            "title": "Method of the coherence evaluation of Ukrainian text",
            "authors": [
                "S. D. Pogorilyy",
                "A. A. Kramov"
            ],
            "abstract": "Due to the growing role of the SEO technologies, it is necessary to perform\nan automated analysis of the article's quality. Such approach helps both to\nreturn the most intelligible pages for the user's query and to raise the web\nsites positions to the top of query results. An automated assessment of a\ncoherence is a part of the complex analysis of the text. In this article, main\nmethods for text coherence measurements for Ukrainian language are analyzed.\nExpediency of using the semantic similarity graph method in comparison with\nother methods are explained. It is suggested the improvement of that method by\nthe pre-training of the neural network for vector representations of sentences.\nExperimental examination of the original method and its modifications is made.\nTraining and examination procedures are made on the corpus of Ukrainian texts,\nwhich were previously retrieved from abstracts and full texts of Ukrainian\nscientific articles. The testing procedure is implemented by performing of two\ntypical tasks for the text coherence assessment: document discrimination task\nand insertion task. Accordingly to the analysis it is defined the most\neffective combination of method's modification and its parameter for the\nmeasurement of the text coherence.",
            "journal ref": "Data Recording, Storage & Processing. 2018. Vol. 20, Issue 4. P.\n  64-75",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00318v1",
            "date": "2020-10-31 17:12:32+00:00",
            "title": "Effective Approach to Develop a Sentiment Annotator For Legal Domain in a Low Resource Setting",
            "authors": [
                "Gathika Ratnayaka",
                "Nisansa de Silva",
                "Amal Shehan Perera",
                "Ramesh Pathirana"
            ],
            "abstract": "Analyzing the sentiments of legal opinions available in Legal Opinion Texts\ncan facilitate several use cases such as legal judgement prediction,\ncontradictory statements identification and party-based sentiment analysis.\nHowever, the task of developing a legal domain specific sentiment annotator is\nchallenging due to resource constraints such as lack of domain specific\nlabelled data and domain expertise. In this study, we propose novel techniques\nthat can be used to develop a sentiment annotator for the legal domain while\nminimizing the need for manual annotations of data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00335v1",
            "date": "2020-10-31 18:44:07+00:00",
            "title": "Pick a Fight or Bite your Tongue: Investigation of Gender Differences in Idiomatic Language Usage",
            "authors": [
                "Ella Rabinovich",
                "Hila Gonen",
                "Suzanne Stevenson"
            ],
            "abstract": "A large body of research on gender-linked language has established\nfoundations regarding cross-gender differences in lexical, emotional, and\ntopical preferences, along with their sociological underpinnings. We compile a\nnovel, large and diverse corpus of spontaneous linguistic productions annotated\nwith speakers' gender, and perform a first large-scale empirical study of\ndistinctions in the usage of \\textit{figurative language} between male and\nfemale authors. Our analyses suggest that (1) idiomatic choices reflect\ngender-specific lexical and semantic preferences in general language, (2) men's\nand women's idiomatic usages express higher emotion than their literal\nlanguage, with detectable, albeit more subtle, differences between male and\nfemale authors along the dimension of dominance compared to similar\ndistinctions in their literal utterances, and (3) contextual analysis of\nidiomatic expressions reveals considerable differences, reflecting subtle\ndivergences in usage environments, shaped by cross-gender communication styles\nand semantic biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00346v1",
            "date": "2020-10-31 19:39:37+00:00",
            "title": "Efficient Arabic emotion recognition using deep neural networks",
            "authors": [
                "Ahmed Ali",
                "Yasser Hifny"
            ],
            "abstract": "Emotion recognition from speech signal based on deep learning is an active\nresearch area. Convolutional neural networks (CNNs) may be the dominant method\nin this area. In this paper, we implement two neural architectures to address\nthis problem. The first architecture is an attention-based CNN-LSTM-DNN model.\nIn this novel architecture, the convolutional layers extract salient features\nand the bi-directional long short-term memory (BLSTM) layers handle the\nsequential phenomena of the speech signal. This is followed by an attention\nlayer, which extracts a summary vector that is fed to the fully connected dense\nlayer (DNN), which finally connects to a softmax output layer. The second\narchitecture is based on a deep CNN model. The results on an Arabic speech\nemotion recognition task show that our innovative approach can lead to\nsignificant improvements (2.2% absolute improvements) over a strong deep CNN\nbaseline system. On the other hand, the deep CNN models are significantly\nfaster than the attention based CNN-LSTM-DNN models in training and\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00387v1",
            "date": "2020-11-01 00:21:59+00:00",
            "title": "Be More with Less: Hypergraph Attention Networks for Inductive Text Classification",
            "authors": [
                "Kaize Ding",
                "Jianling Wang",
                "Jundong Li",
                "Dingcheng Li",
                "Huan Liu"
            ],
            "abstract": "Text classification is a critical research topic with broad applications in\nnatural language processing. Recently, graph neural networks (GNNs) have\nreceived increasing attention in the research community and demonstrated their\npromising results on this canonical task. Despite the success, their\nperformance could be largely jeopardized in practice since they are: (1) unable\nto capture high-order interaction between words; (2) inefficient to handle\nlarge datasets and new documents. To address those issues, in this paper, we\npropose a principled model -- hypergraph attention networks (HyperGAT), which\ncan obtain more expressive power with less computational consumption for text\nrepresentation learning. Extensive experiments on various benchmark datasets\ndemonstrate the efficacy of the proposed approach on the text classification\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00398v1",
            "date": "2020-11-01 01:47:16+00:00",
            "title": "Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism",
            "authors": [
                "Peng Su",
                "K. Vijay-Shanker"
            ],
            "abstract": "With the explosive growth of biomedical literature, designing automatic tools\nto extract information from the literature has great significance in biomedical\nresearch. Recently, transformer-based BERT models adapted to the biomedical\ndomain have produced leading results. However, all the existing BERT models for\nrelation classification only utilize partial knowledge from the last layer. In\nthis paper, we will investigate the method of utilizing the entire layer in the\nfine-tuning process of BERT model. To the best of our knowledge, we are the\nfirst to explore this method. The experimental results illustrate that our\nmethod improves the BERT model performance and outperforms the state-of-the-art\nmethods on three benchmark datasets for different relation extraction tasks. In\naddition, further analysis indicates that the key knowledge about the relations\ncan be learned from the last layer of BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00406v1",
            "date": "2020-11-01 02:48:37+00:00",
            "title": "Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies",
            "authors": [
                "Alexander H. Liu",
                "Yu-An Chung",
                "James Glass"
            ],
            "abstract": "Self-supervised speech representations have been shown to be effective in a\nvariety of speech applications. However, existing representation learning\nmethods generally rely on the autoregressive model and/or observed global\ndependencies while generating the representation. In this work, we propose\nNon-Autoregressive Predictive Coding (NPC), a self-supervised method, to learn\na speech representation in a non-autoregressive manner by relying only on local\ndependencies of speech. NPC has a conceptually simple objective and can be\nimplemented easily with the introduced Masked Convolution Blocks. NPC offers a\nsignificant speedup for inference since it is parallelizable in time and has a\nfixed inference time for each time step regardless of the input sequence\nlength. We discuss and verify the effectiveness of NPC by theoretically and\nempirically comparing it with other methods. We show that the NPC\nrepresentation is comparable to other methods in speech experiments on phonetic\nand speaker classification while being more efficient.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00452v1",
            "date": "2020-11-01 08:56:56+00:00",
            "title": "Fake or Real? A Study of Arabic Satirical Fake News",
            "authors": [
                "Hadeel Saadany",
                "Emad Mohamed",
                "Constantin Orasan"
            ],
            "abstract": "One very common type of fake news is satire which comes in a form of a news\nwebsite or an online platform that parodies reputable real news agencies to\ncreate a sarcastic version of reality. This type of fake news is often\ndisseminated by individuals on their online platforms as it has a much stronger\neffect in delivering criticism than through a straightforward message. However,\nwhen the satirical text is disseminated via social media without mention of its\nsource, it can be mistaken for real news. This study conducts several\nexploratory analyses to identify the linguistic properties of Arabic fake news\nwith satirical content. We exploit these features to build a number of machine\nlearning models capable of identifying satirical fake news with an accuracy of\nup to 98.6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00470v1",
            "date": "2020-11-01 10:44:46+00:00",
            "title": "Seeing Both the Forest and the Trees: Multi-head Attention for Joint Classification on Different Compositional Levels",
            "authors": [
                "Miruna Pislar",
                "Marek Rei"
            ],
            "abstract": "In natural languages, words are used in association to construct sentences.\nIt is not words in isolation, but the appropriate combination of hierarchical\nstructures that conveys the meaning of the whole sentence. Neural networks can\ncapture expressive language features; however, insights into the link between\nwords and sentences are difficult to acquire automatically. In this work, we\ndesign a deep neural network architecture that explicitly wires lower and\nhigher linguistic components; we then evaluate its ability to perform the same\ntask at different hierarchical levels. Settling on broad text classification\ntasks, we show that our model, MHAL, learns to simultaneously solve them at\ndifferent levels of granularity by fluidly transferring knowledge between\nhierarchies. Using a multi-head attention mechanism to tie the representations\nbetween single words and full sentences, MHAL systematically outperforms\nequivalent models that are not incentivized towards developing compositional\nrepresentations. Moreover, we demonstrate that, with the proposed architecture,\nthe sentence information flows naturally to individual words, allowing the\nmodel to behave like a sequence labeller (which is a lower, word-level task)\neven without any word supervision, in a zero-shot fashion.",
            "journal ref": "COLING 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00474v1",
            "date": "2020-11-01 11:00:19+00:00",
            "title": "Opinion Transmission Network for Jointly Improving Aspect-oriented Opinion Words Extraction and Sentiment Classification",
            "authors": [
                "Chengcan Ying",
                "Zhen Wu",
                "Xinyu Dai",
                "Shujian Huang",
                "Jiajun Chen"
            ],
            "abstract": "Aspect-level sentiment classification (ALSC) and aspect oriented opinion\nwords extraction (AOWE) are two highly relevant aspect-based sentiment analysis\n(ABSA) subtasks. They respectively aim to detect the sentiment polarity and\nextract the corresponding opinion words toward a given aspect in a sentence.\nPrevious works separate them and focus on one of them by training neural models\non small-scale labeled data, while neglecting the connections between them. In\nthis paper, we propose a novel joint model, Opinion Transmission Network (OTN),\nto exploit the potential bridge between ALSC and AOWE to achieve the goal of\nfacilitating them simultaneously. Specifically, we design two tailor-made\nopinion transmission mechanisms to control opinion clues flow bidirectionally,\nrespectively from ALSC to AOWE and AOWE to ALSC. Experiment results on two\nbenchmark datasets show that our joint model outperforms strong baselines on\nthe two tasks. Further analysis also validates the effectiveness of opinion\ntransmission mechanisms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00476v1",
            "date": "2020-11-01 11:06:31+00:00",
            "title": "Transformer-based Multi-Aspect Modeling for Multi-Aspect Multi-Sentiment Analysis",
            "authors": [
                "Zhen Wu",
                "Chengcan Ying",
                "Xinyu Dai",
                "Shujian Huang",
                "Jiajun Chen"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) aims at analyzing the sentiment of a\ngiven aspect in a sentence. Recently, neural network-based methods have\nachieved promising results in existing ABSA datasets. However, these datasets\ntend to degenerate to sentence-level sentiment analysis because most sentences\ncontain only one aspect or multiple aspects with the same sentiment polarity.\nTo facilitate the research of ABSA, NLPCC 2020 Shared Task 2 releases a new\nlarge-scale Multi-Aspect Multi-Sentiment (MAMS) dataset. In the MAMS dataset,\neach sentence contains at least two different aspects with different sentiment\npolarities, which makes ABSA more complex and challenging. To address the\nchallenging dataset, we re-formalize ABSA as a problem of multi-aspect\nsentiment analysis, and propose a novel Transformer-based Multi-aspect Modeling\nscheme (TMM), which can capture potential relations between multiple aspects\nand simultaneously detect the sentiment of all aspects in a sentence.\nExperiment results on the MAMS dataset show that our method achieves noticeable\nimprovements compared with strong baselines such as BERT and RoBERTa, and\nfinally ranks the 2nd in NLPCC 2020 Shared Task 2 Evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00519v1",
            "date": "2020-11-01 14:48:49+00:00",
            "title": "CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering",
            "authors": [
                "Junru Lu",
                "Gabriele Pergola",
                "Lin Gui",
                "Binyang Li",
                "Yulan He"
            ],
            "abstract": "We introduce CHIME, a cross-passage hierarchical memory network for question\nanswering (QA) via text generation. It extends XLNet introducing an auxiliary\nmemory module consisting of two components: the context memory collecting\ncross-passage evidences, and the answer memory working as a buffer continually\nrefining the generated answers. Empirically, we show the efficacy of the\nproposed architecture in the multi-passage generative QA, outperforming the\nstate-of-the-art baselines with better syntactically well-formed answers and\nincreased precision in addressing the questions of the AmazonQA review dataset.\nAn additional qualitative analysis revealed the interpretability introduced by\nthe memory module.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00543v1",
            "date": "2020-11-01 15:59:24+00:00",
            "title": "Semantic coordinates analysis reveals language changes in the AI field",
            "authors": [
                "Zining Zhu",
                "Yang Xu",
                "Frank Rudzicz"
            ],
            "abstract": "Semantic shifts can reflect changes in beliefs across hundreds of years, but\nit is less clear whether trends in fast-changing communities across a short\ntime can be detected. We propose semantic coordinates analysis, a method based\non semantic shifts, that reveals changes in language within publications of a\nfield (we use AI as example) across a short time span. We use GloVe-style\nprobability ratios to quantify the shifting directions and extents from\nmultiple viewpoints. We show that semantic coordinates analysis can detect\nshifts echoing changes of research interests (e.g., \"deep\" shifted further from\n\"rigorous\" to \"neural\"), and developments of research activities (e,g.,\n\"collaboration\" contains less \"competition\" than \"collaboration\"), based on\npublications spanning as short as 10 years.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00547v1",
            "date": "2020-11-01 16:11:17+00:00",
            "title": "SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training",
            "authors": [
                "Huda Khayrallah",
                "Jo\u00e3o Sedoc"
            ],
            "abstract": "Non-task-oriented dialog models suffer from poor quality and non-diverse\nresponses. To overcome limited conversational data, we apply Simulated Multiple\nReference Training (SMRT; Khayrallah et al., 2020), and use a paraphraser to\nsimulate multiple responses per training prompt. We find SMRT improves over a\nstrong Transformer baseline as measured by human and automatic quality scores\nand lexical diversity. We also find SMRT is comparable to pretraining in human\nevaluation quality, and outperforms pretraining on automatic quality and\nlexical diversity, without requiring related-domain dialog data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00564v1",
            "date": "2020-11-01 17:15:42+00:00",
            "title": "Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Survey",
            "authors": [
                "Samuel Louvan",
                "Bernardo Magnini"
            ],
            "abstract": "In recent years, fostered by deep learning technologies and by the high\ndemand for conversational AI, various approaches have been proposed that\naddress the capacity to elicit and understand user's needs in task-oriented\ndialogue systems. We focus on two core tasks, slot filling (SF) and intent\nclassification (IC), and survey how neural-based models have rapidly evolved to\naddress natural language understanding in dialogue systems. We introduce three\nneural architectures: independent model, which model SF and IC separately,\njoint models, which exploit the mutual benefit of the two tasks simultaneously,\nand transfer learning models, that scale the model to new domains. We discuss\nthe current state of the research in SF and IC and highlight challenges that\nstill require attention.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00592v1",
            "date": "2020-11-01 18:46:53+00:00",
            "title": "Vec2Sent: Probing Sentence Embeddings with Natural Language Generation",
            "authors": [
                "Martin Kerscher",
                "Steffen Eger"
            ],
            "abstract": "We introspect black-box sentence embeddings by conditionally generating from\nthem with the objective to retrieve the underlying discrete sentence. We\nperceive of this as a new unsupervised probing task and show that it correlates\nwell with downstream task performance. We also illustrate how the language\ngenerated from different encoders differs. We apply our approach to generate\nsentence analogies from sentence embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00596v2",
            "date": "2020-11-01 18:53:32+00:00",
            "title": "Bracketing Encodings for 2-Planar Dependency Parsing",
            "authors": [
                "Michalina Strzyz",
                "David Vilares",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "We present a bracketing-based encoding that can be used to represent any\n2-planar dependency tree over a sentence of length n as a sequence of n labels,\nhence providing almost total coverage of crossing arcs in sequence labeling\nparsing. First, we show that existing bracketing encodings for parsing as\nlabeling can only handle a very mild extension of projective trees. Second, we\novercome this limitation by taking into account the well-known property of\n2-planarity, which is present in the vast majority of dependency syntactic\nstructures in treebanks, i.e., the arcs of a dependency tree can be split into\ntwo planes such that arcs in a given plane do not cross. We take advantage of\nthis property to design a method that balances the brackets and that encodes\nthe arcs belonging to each of those planes, allowing for almost unrestricted\nnon-projectivity (round 99.9% coverage) in sequence labeling parsing. The\nexperiments show that our linearizations improve over the accuracy of the\noriginal bracketing encoding in highly non-projective treebanks (on average by\n0.4 LAS), while achieving a similar speed. Also, they are especially suitable\nwhen PoS tags are not used as input parameters to the models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00615v1",
            "date": "2020-11-01 19:50:34+00:00",
            "title": "Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning",
            "authors": [
                "Jon Ander Campos",
                "Kyunghyun Cho",
                "Arantxa Otegi",
                "Aitor Soroa",
                "Gorka Azkune",
                "Eneko Agirre"
            ],
            "abstract": "The interaction of conversational systems with users poses an exciting\nopportunity for improving them after deployment, but little evidence has been\nprovided of its feasibility. In most applications, users are not able to\nprovide the correct answer to the system, but they are able to provide binary\n(correct, incorrect) feedback. In this paper we propose feedback-weighted\nlearning based on importance sampling to improve upon an initial supervised\nsystem using binary user feedback. We perform simulated experiments on document\nclassification (for development) and Conversational Question Answering datasets\nlike QuAC and DoQA, where binary user feedback is derived from gold\nannotations. The results show that our method is able to improve over the\ninitial supervised system, getting close to a fully-supervised system that has\naccess to the same labeled examples in in-domain experiments (QuAC), and even\nmatching in out-of-domain experiments (DoQA). Our work opens the prospect to\nexploit interactions with real users and improve conversational systems after\ndeployment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00633v1",
            "date": "2020-11-01 21:57:51+00:00",
            "title": "Aspect-Based Argument Mining",
            "authors": [
                "Dietrich Trautmann"
            ],
            "abstract": "Computational Argumentation in general and Argument Mining in particular are\nimportant research fields. In previous works, many of the challenges to\nautomatically extract and to some degree reason over natural language arguments\nwere addressed. The tools to extract argument units are increasingly available\nand further open problems can be addressed. In this work, we are presenting the\ntask of Aspect-Based Argument Mining (ABAM), with the essential subtasks of\nAspect Term Extraction (ATE) and Nested Segmentation (NS). At the first\ninstance, we create and release an annotated corpus with aspect information on\nthe token-level. We consider aspects as the main point(s) argument units are\naddressing. This information is important for further downstream tasks such as\nargument ranking, argument summarization and generation, as well as the search\nfor counter-arguments on the aspect-level. We present several experiments using\nstate-of-the-art supervised architectures and demonstrate their performance for\nboth of the subtasks. The annotated benchmark is available at\nhttps://github.com/trtm/ABAM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00669v1",
            "date": "2020-11-02 01:25:13+00:00",
            "title": "Reasoning Over History: Context Aware Visual Dialog",
            "authors": [
                "Muhammad A. Shah",
                "Shikib Mehri",
                "Tejas Srinivasan"
            ],
            "abstract": "While neural models have been shown to exhibit strong performance on\nsingle-turn visual question answering (VQA) tasks, extending VQA to a\nmulti-turn, conversational setting remains a challenge. One way to address this\nchallenge is to augment existing strong neural VQA models with the mechanisms\nthat allow them to retain information from previous dialog turns. One strong\nVQA model is the MAC network, which decomposes a task into a series of\nattention-based reasoning steps. However, since the MAC network is designed for\nsingle-turn question answering, it is not capable of referring to past dialog\nturns. More specifically, it struggles with tasks that require reasoning over\nthe dialog history, particularly coreference resolution. We extend the MAC\nnetwork architecture with Context-aware Attention and Memory (CAM), which\nattends over control states in past dialog turns to determine the necessary\nreasoning operations for the current question. MAC nets with CAM achieve up to\n98.25% accuracy on the CLEVR-Dialog dataset, beating the existing\nstate-of-the-art by 30% (absolute). Our error analysis indicates that with CAM,\nthe model's performance particularly improved on questions that required\ncoreference resolution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00677v1",
            "date": "2020-11-02 01:54:56+00:00",
            "title": "IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP",
            "authors": [
                "Fajri Koto",
                "Afshin Rahimi",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "Although the Indonesian language is spoken by almost 200 million people and\nthe 10th most spoken language in the world, it is under-represented in NLP\nresearch. Previous work on Indonesian has been hampered by a lack of annotated\ndatasets, a sparsity of language resources, and a lack of resource\nstandardization. In this work, we release the IndoLEM dataset comprising seven\ntasks for the Indonesian language, spanning morpho-syntax, semantics, and\ndiscourse. We additionally release IndoBERT, a new pre-trained language model\nfor Indonesian, and evaluate it over IndoLEM, in addition to benchmarking it\nagainst existing resources. Our experiments show that IndoBERT achieves\nstate-of-the-art performance over most of the tasks in IndoLEM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00679v1",
            "date": "2020-11-02 02:01:12+00:00",
            "title": "Liputan6: A Large-scale Indonesian Dataset for Text Summarization",
            "authors": [
                "Fajri Koto",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "In this paper, we introduce a large-scale Indonesian summarization dataset.\nWe harvest articles from Liputan6.com, an online news portal, and obtain\n215,827 document-summary pairs. We leverage pre-trained language models to\ndevelop benchmark extractive and abstractive summarization methods over the\ndataset with multilingual and monolingual BERT-based models. We include a\nthorough error analysis by examining machine-generated summaries that have low\nROUGE scores, and expose both issues with ROUGE it-self, as well as with\nextractive and abstractive summarization models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00682v1",
            "date": "2020-11-02 02:06:33+00:00",
            "title": "Sequence-to-Sequence Networks Learn the Meaning of Reflexive Anaphora",
            "authors": [
                "Robert Frank",
                "Jackson Petty"
            ],
            "abstract": "Reflexive anaphora present a challenge for semantic interpretation: their\nmeaning varies depending on context in a way that appears to require abstract\nvariables. Past work has raised doubts about the ability of recurrent networks\nto meet this challenge. In this paper, we explore this question in the context\nof a fragment of English that incorporates the relevant sort of contextual\nvariability. We consider sequence-to-sequence architectures with recurrent\nunits and show that such networks are capable of learning semantic\ninterpretations for reflexive anaphora which generalize to novel antecedents.\nWe explore the effect of attention mechanisms and different recurrent unit\ntypes on the type of training data that is needed for success as measured in\ntwo ways: how much lexical support is needed to induce an abstract reflexive\nmeaning (i.e., how many distinct reflexive antecedents must occur during\ntraining) and what contexts must a noun phrase occur in to support\ngeneralization of reflexive interpretation to this noun phrase?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00692v2",
            "date": "2020-11-02 02:33:59+00:00",
            "title": "How Domain Terminology Affects Meeting Summarization Performance",
            "authors": [
                "Jia Jin Koay",
                "Alexander Roustai",
                "Xiaojin Dai",
                "Dillon Burns",
                "Alec Kerrigan",
                "Fei Liu"
            ],
            "abstract": "Meetings are essential to modern organizations. Numerous meetings are held\nand recorded daily, more than can ever be comprehended. A meeting summarization\nsystem that identifies salient utterances from the transcripts to automatically\ngenerate meeting minutes can help. It empowers users to rapidly search and sift\nthrough large meeting collections. To date, the impact of domain terminology on\nthe performance of meeting summarization remains understudied, despite that\nmeetings are rich with domain knowledge. In this paper, we create gold-standard\nannotations for domain terminology on a sizable meeting corpus; they are known\nas jargon terms. We then analyze the performance of a meeting summarization\nsystem with and without jargon terms. Our findings reveal that domain\nterminology can have a substantial impact on summarization performance. We\npublicly release all domain terminology to advance research in meeting\nsummarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00740v1",
            "date": "2020-11-02 04:28:16+00:00",
            "title": "Abstracting Influence Paths for Explaining (Contextualization of) BERT Models",
            "authors": [
                "Kaiji Lu",
                "Zifan Wang",
                "Piotr Mardziel",
                "Anupam Datta"
            ],
            "abstract": "While \"attention is all you need\" may be proving true, we do not yet know\nwhy: attention-based models such as BERT are superior but how they\ncontextualize information even for simple grammatical rules such as\nsubject-verb number agreement (SVA) is uncertain. We introduce multi-partite\npatterns, abstractions of sets of paths through a neural network model.\nPatterns quantify and localize the effect of an input concept (e.g., a\nsubject's number) on an output concept (e.g. corresponding verb's number) to\npaths passing through a sequence of model components, thus surfacing how BERT\ncontextualizes information. We describe guided pattern refinement, an efficient\nsearch procedure for finding patterns representative of concept-critical paths.\nWe discover that patterns generate succinct and meaningful explanations for\nBERT, highlighted by \"copy\" and \"transfer\" operations implemented by skip\nconnections and attention heads, respectively. We also show how pattern\nvisualizations help us understand how BERT contextualizes various grammatical\nconcepts, such as SVA across clauses, and why it makes errors in some cases\nwhile succeeding in others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00758v1",
            "date": "2020-11-02 05:47:08+00:00",
            "title": "\u00daFAL at MRP 2020: Permutation-invariant Semantic Parsing in PERIN",
            "authors": [
                "David Samuel",
                "Milan Straka"
            ],
            "abstract": "We present PERIN, a novel permutation-invariant approach to sentence-to-graph\nsemantic parsing. PERIN is a versatile, cross-framework and language\nindependent architecture for universal modeling of semantic structures. Our\nsystem participated in the CoNLL 2020 shared task, Cross-Framework Meaning\nRepresentation Parsing (MRP 2020), where it was evaluated on five different\nframeworks (AMR, DRG, EDS, PTG and UCCA) across four languages. PERIN was one\nof the winners of the shared task. The source code and pretrained models are\navailable at https://github.com/ufal/perin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00767v2",
            "date": "2020-11-02 06:24:58+00:00",
            "title": "Reducing Confusion in Active Learning for Part-Of-Speech Tagging",
            "authors": [
                "Aditi Chaudhary",
                "Antonios Anastasopoulos",
                "Zaid Sheikh",
                "Graham Neubig"
            ],
            "abstract": "Active learning (AL) uses a data selection algorithm to select useful\ntraining samples to minimize annotation cost. This is now an essential tool for\nbuilding low-resource syntactic analyzers such as part-of-speech (POS) taggers.\nExisting AL heuristics are generally designed on the principle of selecting\nuncertain yet representative training instances, where annotating these\ninstances may reduce a large number of errors. However, in an empirical study\nacross six typologically diverse languages (German, Swedish, Galician, North\nSami, Persian, and Ukrainian), we found the surprising result that even in an\noracle scenario where we know the true uncertainty of predictions, these\ncurrent heuristics are far from optimal. Based on this analysis, we pose the\nproblem of AL as selecting instances which maximally reduce the confusion\nbetween particular pairs of output tags. Extensive experimentation on the\naforementioned languages shows that our proposed AL strategy outperforms other\nAL strategies by a significant margin. We also present auxiliary results\ndemonstrating the importance of proper calibration of models, which we ensure\nthrough cross-view training, and analysis demonstrating how our proposed\nstrategy selects examples that more closely follow the oracle data\ndistribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00770v1",
            "date": "2020-11-02 06:34:33+00:00",
            "title": "Context-Aware Cross-Attention for Non-Autoregressive Translation",
            "authors": [
                "Liang Ding",
                "Longyue Wang",
                "Di Wu",
                "Dacheng Tao",
                "Zhaopeng Tu"
            ],
            "abstract": "Non-autoregressive translation (NAT) significantly accelerates the inference\nprocess by predicting the entire target sequence. However, due to the lack of\ntarget dependency modelling in the decoder, the conditional generation process\nheavily depends on the cross-attention. In this paper, we reveal a localness\nperception problem in NAT cross-attention, for which it is difficult to\nadequately capture source context. To alleviate this problem, we propose to\nenhance signals of neighbour source tokens into conventional cross-attention.\nExperimental results on several representative datasets show that our approach\ncan consistently improve translation quality over strong NAT baselines.\nExtensive analyses demonstrate that the enhanced cross-attention achieves\nbetter exploitation of source contexts by leveraging both local and global\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00777v1",
            "date": "2020-11-02 07:08:19+00:00",
            "title": "COSMO: Conditional SEQ2SEQ-based Mixture Model for Zero-Shot Commonsense Question Answering",
            "authors": [
                "Farhad Moghimifar",
                "Lizhen Qu",
                "Yue Zhuo",
                "Mahsa Baktashmotlagh",
                "Gholamreza Haffari"
            ],
            "abstract": "Commonsense reasoning refers to the ability of evaluating a social situation\nand acting accordingly. Identification of the implicit causes and effects of a\nsocial context is the driving capability which can enable machines to perform\ncommonsense reasoning. The dynamic world of social interactions requires\ncontext-dependent on-demand systems to infer such underlying information.\nHowever, current approaches in this realm lack the ability to perform\ncommonsense reasoning upon facing an unseen situation, mostly due to\nincapability of identifying a diverse range of implicit social relations. Hence\nthey fail to estimate the correct reasoning path. In this paper, we present\nConditional SEQ2SEQ-based Mixture model (COSMO), which provides us with the\ncapabilities of dynamic and diverse content generation. We use COSMO to\ngenerate context-dependent clauses, which form a dynamic Knowledge Graph (KG)\non-the-fly for commonsense reasoning. To show the adaptability of our model to\ncontext-dependant knowledge generation, we address the task of zero-shot\ncommonsense question answering. The empirical results indicate an improvement\nof up to +5.2% over the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00797v1",
            "date": "2020-11-02 07:51:05+00:00",
            "title": "Context Dependent Semantic Parsing: A Survey",
            "authors": [
                "Zhuang Li",
                "Lizhen Qu",
                "Gholamreza Haffari"
            ],
            "abstract": "Semantic parsing is the task of translating natural language utterances into\nmachine-readable meaning representations. Currently, most semantic parsing\nmethods are not able to utilize contextual information (e.g. dialogue and\ncomments history), which has a great potential to boost semantic parsing\nperformance. To address this issue, context dependent semantic parsing has\nrecently drawn a lot of attention. In this survey, we investigate progress on\nthe methods for the context dependent semantic parsing, together with the\ncurrent datasets and tasks. We then point out open problems and challenges for\nfuture research in this area. The collected resources for this topic are\navailable\nat:https://github.com/zhuang-li/Contextual-Semantic-Parsing-Paper-List.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00834v1",
            "date": "2020-11-02 09:03:46+00:00",
            "title": "Comparison by Conversion: Reverse-Engineering UCCA from Syntax and Lexical Semantics",
            "authors": [
                "Daniel Hershcovich",
                "Nathan Schneider",
                "Dotan Dvir",
                "Jakob Prange",
                "Miryam de Lhoneux",
                "Omri Abend"
            ],
            "abstract": "Building robust natural language understanding systems will require a clear\ncharacterization of whether and how various linguistic meaning representations\ncomplement each other. To perform a systematic comparative analysis, we\nevaluate the mapping between meaning representations from different frameworks\nusing two complementary methods: (i) a rule-based converter, and (ii) a\nsupervised delexicalized parser that parses to one framework using only\ninformation from the other as features. We apply these methods to convert the\nSTREUSLE corpus (with syntactic and lexical semantic annotations) to UCCA (a\ngraph-structured full-sentence meaning representation). Both methods yield\nsurprisingly accurate target representations, close to fully supervised UCCA\nparser quality---indicating that UCCA annotations are partially redundant with\nSTREUSLE annotations. Despite this substantial convergence between frameworks,\nwe find several important areas of divergence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00943v2",
            "date": "2020-11-02 12:52:31+00:00",
            "title": "How Far Does BERT Look At:Distance-based Clustering and Analysis of BERT$'$s Attention",
            "authors": [
                "Yue Guan",
                "Jingwen Leng",
                "Chao Li",
                "Quan Chen",
                "Minyi Guo"
            ],
            "abstract": "Recent research on the multi-head attention mechanism, especially that in\npre-trained models such as BERT, has shown us heuristics and clues in analyzing\nvarious aspects of the mechanism. As most of the research focus on probing\ntasks or hidden states, previous works have found some primitive patterns of\nattention head behavior by heuristic analytical methods, but a more systematic\nanalysis specific on the attention patterns still remains primitive. In this\nwork, we clearly cluster the attention heatmaps into significantly different\npatterns through unsupervised clustering on top of a set of proposed features,\nwhich corroborates with previous observations. We further study their\ncorresponding functions through analytical study. In addition, our proposed\nfeatures can be used to explain and calibrate different attention heads in\nTransformer models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00948v2",
            "date": "2020-11-02 13:05:00+00:00",
            "title": "An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution",
            "authors": [
                "Ryuto Konno",
                "Yuichiroh Matsubayashi",
                "Shun Kiyono",
                "Hiroki Ouchi",
                "Ryo Takahashi",
                "Kentaro Inui"
            ],
            "abstract": "One critical issue of zero anaphora resolution (ZAR) is the scarcity of\nlabeled data. This study explores how effectively this problem can be\nalleviated by data augmentation. We adopt a state-of-the-art data augmentation\nmethod, called the contextual data augmentation (CDA), that generates labeled\ntraining instances using a pretrained language model. The CDA has been reported\nto work well for several other natural language processing tasks, including\ntext classification and machine translation. This study addresses two\nunderexplored issues on CDA, that is, how to reduce the computational cost of\ndata augmentation and how to ensure the quality of the generated data. We also\npropose two methods to adapt CDA to ZAR: [MASK]-based augmentation and\nlinguistically-controlled masking. Consequently, the experimental results on\nJapanese ZAR show that our methods contribute to both the accuracy gain and the\ncomputation cost reduction. Our closer analysis reveals that the proposed\nmethod can improve the quality of the augmented training data when compared to\nthe conventional CDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00960v1",
            "date": "2020-11-02 13:25:39+00:00",
            "title": "A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English",
            "authors": [
                "Marius Mosbach",
                "Stefania Degaetano-Ortlieb",
                "Marie-Pauline Krielke",
                "Badr M. Abdullah",
                "Dietrich Klakow"
            ],
            "abstract": "Transformer-based language models achieve high performance on various tasks,\nbut we still lack understanding of the kind of linguistic knowledge they learn\nand rely on. We evaluate three models (BERT, RoBERTa, and ALBERT), testing\ntheir grammatical and semantic knowledge by sentence-level probing, diagnostic\ncases, and masked prediction tasks. We focus on relative clauses (in American\nEnglish) as a complex phenomenon needing contextual information and antecedent\nidentification to be resolved. Based on a naturalistic dataset, probing shows\nthat all three models indeed capture linguistic knowledge about grammaticality,\nachieving high performance. Evaluation on diagnostic cases and masked\nprediction tasks considering fine-grained linguistic knowledge, however, shows\npronounced model-specific weaknesses especially on semantic knowledge, strongly\nimpacting models' performance. Our results highlight the importance of (a)model\ncomparison in evaluation task and (b) building up claims of model performance\nand the linguistic knowledge they capture beyond purely probing-based\nevaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00961v1",
            "date": "2020-11-02 13:27:21+00:00",
            "title": "Combining Event Semantics and Degree Semantics for Natural Language Inference",
            "authors": [
                "Izumi Haruta",
                "Koji Mineshima",
                "Daisuke Bekki"
            ],
            "abstract": "In formal semantics, there are two well-developed semantic frameworks: event\nsemantics, which treats verbs and adverbial modifiers using the notion of\nevent, and degree semantics, which analyzes adjectives and comparatives using\nthe notion of degree. However, it is not obvious whether these frameworks can\nbe combined to handle cases in which the phenomena in question are interacting\nwith each other. Here, we study this issue by focusing on natural language\ninference (NLI). We implement a logic-based NLI system that combines event\nsemantics and degree semantics and their interaction with lexical knowledge. We\nevaluate the system on various NLI datasets containing linguistically\nchallenging problems. The results show that the system achieves high accuracies\non these datasets in comparison with previous logic-based systems and\ndeep-learning-based systems. This suggests that the two semantic frameworks can\nbe combined consistently to handle various combinations of linguistic phenomena\nwithout compromising the advantage of either framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00975v1",
            "date": "2020-11-02 13:50:59+00:00",
            "title": "DNN-Based Semantic Model for Rescoring N-best Speech Recognition List",
            "authors": [
                "Dominique Fohr",
                "Irina Illina"
            ],
            "abstract": "The word error rate (WER) of an automatic speech recognition (ASR) system\nincreases when a mismatch occurs between the training and the testing\nconditions due to the noise, etc. In this case, the acoustic information can be\nless reliable. This work aims to improve ASR by modeling long-term semantic\nrelations to compensate for distorted acoustic features. We propose to perform\nthis through rescoring of the ASR N-best hypotheses list. To achieve this, we\ntrain a deep neural network (DNN). Our DNN rescoring model is aimed at\nselecting hypotheses that have better semantic consistency and therefore lower\nWER. We investigate two types of representations as part of input features to\nour DNN model: static word embeddings (from word2vec) and dynamic contextual\nembeddings (from BERT). Acoustic and linguistic features are also included. We\nperform experiments on the publicly available dataset TED-LIUM mixed with real\nnoise. The proposed rescoring approaches give significant improvement of the\nWER over the ASR system without rescoring models in two noisy conditions and\nwith n-gram and RNNLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01026v1",
            "date": "2020-11-02 15:17:44+00:00",
            "title": "Biased TextRank: Unsupervised Graph-Based Content Extraction",
            "authors": [
                "Ashkan Kazemi",
                "Ver\u00f3nica P\u00e9rez-Rosas",
                "Rada Mihalcea"
            ],
            "abstract": "We introduce Biased TextRank, a graph-based content extraction method\ninspired by the popular TextRank algorithm that ranks text spans according to\ntheir importance for language processing tasks and according to their relevance\nto an input \"focus.\" Biased TextRank enables focused content extraction for\ntext by modifying the random restarts in the execution of TextRank. The random\nrestart probabilities are assigned based on the relevance of the graph nodes to\nthe focus of the task. We present two applications of Biased TextRank: focused\nsummarization and explanation extraction, and show that our algorithm leads to\nimproved performance on two different datasets by significant ROUGE-N score\nmargins. Much like its predecessor, Biased TextRank is unsupervised, easy to\nimplement and orders of magnitude faster and lighter than current\nstate-of-the-art Natural Language Processing methods for similar tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01060v2",
            "date": "2020-11-02 15:42:40+00:00",
            "title": "Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps",
            "authors": [
                "Xanh Ho",
                "Anh-Khoa Duong Nguyen",
                "Saku Sugawara",
                "Akiko Aizawa"
            ],
            "abstract": "A multi-hop question answering (QA) dataset aims to test reasoning and\ninference skills by requiring a model to read multiple paragraphs to answer a\ngiven question. However, current datasets do not provide a complete explanation\nfor the reasoning process from the question to the answer. Further, previous\nstudies revealed that many examples in existing multi-hop datasets do not\nrequire multi-hop reasoning to answer a question. In this study, we present a\nnew multi-hop QA dataset, called 2WikiMultiHopQA, which uses structured and\nunstructured data. In our dataset, we introduce the evidence information\ncontaining a reasoning path for multi-hop questions. The evidence information\nhas two benefits: (i) providing a comprehensive explanation for predictions and\n(ii) evaluating the reasoning skills of a model. We carefully design a pipeline\nand a set of templates when generating a question-answer pair that guarantees\nthe multi-hop steps and the quality of the questions. We also exploit the\nstructured format in Wikidata and use logical rules to create questions that\nare natural but still require multi-hop reasoning. Through experiments, we\ndemonstrate that our dataset is challenging for multi-hop models and it ensures\nthat multi-hop reasoning is required.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01097v1",
            "date": "2020-11-02 16:31:14+00:00",
            "title": "Enabling Zero-shot Multilingual Spoken Language Translation with Language-Specific Encoders and Decoders",
            "authors": [
                "Carlos Escolano",
                "Marta R. Costa-juss\u00e0",
                "Jos\u00e9 A. R. Fonollosa",
                "Carlos Segura"
            ],
            "abstract": "Current end-to-end approaches to Spoken Language Translation (SLT) rely on\nlimited training resources, especially for multilingual settings. On the other\nhand, Multilingual Neural Machine Translation (MultiNMT) approaches rely on\nhigher quality and more massive data sets. Our proposed method extends a\nMultiNMT architecture based on language-specific encoders-decoders to the task\nof Multilingual SLT (MultiSLT) Our experiments on four different languages show\nthat coupling the speech encoder to the MultiNMT architecture produces similar\nquality translations compared to a bilingual baseline ($\\pm 0.2$ BLEU) while\neffectively allowing for zero-shot MultiSLT. Additionally, we propose using\nAdapter networks for SLT that produce consistent improvements of +1 BLEU points\nin all tested languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01139v1",
            "date": "2020-11-02 17:28:36+00:00",
            "title": "Automated Transcription of Non-Latin Script Periodicals: A Case Study in the Ottoman Turkish Print Archive",
            "authors": [
                "Suphan Kirmizialtin",
                "David Wrisley"
            ],
            "abstract": "Our study utilizes deep learning methods for the automated transcription of\nlate nineteenth- and early twentieth-century periodicals written in Arabic\nscript Ottoman Turkish (OT) using the Transkribus platform. We discuss the\nhistorical situation of OT text collections and how they were excluded for the\nmost part from the late twentieth century corpora digitization that took place\nin many Latin script languages. This exclusion has two basic reasons: the\ntechnical challenges of OCR for Arabic script languages, and the rapid\nabandonment of that very script in the Turkish historical context. In the\nspecific case of OT, opening periodical collections to digital tools require\ntraining HTR models to generate transcriptions in the Latin writing system of\ncontemporary readers of Turkish, and not, as some may expect, in right-to-left\nArabic script text. In the paper we discuss the challenges of training such\nmodels where one-to-one correspondence between the writing systems do not\nexist, and we report results based on our HTR experiments with two OT\nperiodicals from the early twentieth century. Finally, we reflect on potential\ndomain bias of HTR models in historical languages exhibiting spatio-temporal\nvariance as well as the significance of working between writing systems for\nlanguage communities that have experienced language reform and script change.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01181v3",
            "date": "2020-11-02 18:17:51+00:00",
            "title": "QMUL-SDS @ SardiStance: Leveraging Network Interactions to Boost Performance on Stance Detection using Knowledge Graphs",
            "authors": [
                "Rabab Alkhalifa",
                "Arkaitz Zubiaga"
            ],
            "abstract": "This paper presents our submission to the SardiStance 2020 shared task,\ndescribing the architecture used for Task A and Task B. While our submission\nfor Task A did not exceed the baseline, retraining our model using all the\ntraining tweets, showed promising results leading to (f-avg 0.601) using\nbidirectional LSTM with BERT multilingual embedding for Task A. For our\nsubmission for Task B, we ranked 6th (f-avg 0.709). With further investigation,\nour best experimented settings increased performance from (f-avg 0.573) to\n(f-avg 0.733) with same architecture and parameter settings and after only\nincorporating social interaction features -- highlighting the impact of social\ninteraction on the model's performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01196v1",
            "date": "2020-11-02 18:41:32+00:00",
            "title": "The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks",
            "authors": [
                "Brihi Joshi",
                "Neil Shah",
                "Francesco Barbieri",
                "Leonardo Neves"
            ],
            "abstract": "Contextual embeddings derived from transformer-based neural language models\nhave shown state-of-the-art performance for various tasks such as question\nanswering, sentiment analysis, and textual similarity in recent years.\nExtensive work shows how accurately such models can represent abstract,\nsemantic information present in text. In this expository work, we explore a\ntangent direction and analyze such models' performance on tasks that require a\nmore granular level of representation. We focus on the problem of textual\nsimilarity from two perspectives: matching documents on a granular level\n(requiring embeddings to capture fine-grained attributes in the text), and an\nabstract level (requiring embeddings to capture overall textual semantics). We\nempirically demonstrate, across two datasets from different domains, that\ndespite high performance in abstract document matching as expected, contextual\nembeddings are consistently (and at times, vastly) outperformed by simple\nbaselines like TF-IDF for more granular tasks. We then propose a simple but\neffective method to incorporate TF-IDF into models that use contextual\nembeddings, achieving relative improvements of up to 36% on granular tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01482v1",
            "date": "2020-11-03 05:06:37+00:00",
            "title": "Layer-Wise Multi-View Learning for Neural Machine Translation",
            "authors": [
                "Qiang Wang",
                "Changliang Li",
                "Yue Zhang",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "Traditional neural machine translation is limited to the topmost encoder\nlayer's context representation and cannot directly perceive the lower encoder\nlayers. Existing solutions usually rely on the adjustment of network\narchitecture, making the calculation more complicated or introducing additional\nstructural restrictions. In this work, we propose layer-wise multi-view\nlearning to solve this problem, circumventing the necessity to change the model\nstructure. We regard each encoder layer's off-the-shelf output, a by-product in\nlayer-by-layer encoding, as the redundant view for the input sentence. In this\nway, in addition to the topmost encoder layer (referred to as the primary\nview), we also incorporate an intermediate encoder layer as the auxiliary view.\nWe feed the two views to a partially shared decoder to maintain independent\nprediction. Consistency regularization based on KL divergence is used to\nencourage the two views to learn from each other. Extensive experimental\nresults on five translation tasks show that our approach yields stable\nimprovements over multiple strong baselines. As another bonus, our method is\nagnostic to network architectures and can maintain the same inference speed as\nthe original model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01513v1",
            "date": "2020-11-03 07:13:06+00:00",
            "title": "CharBERT: Character-aware Pre-trained Language Model",
            "authors": [
                "Wentao Ma",
                "Yiming Cui",
                "Chenglei Si",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "Most pre-trained language models (PLMs) construct word representations at\nsubword level with Byte-Pair Encoding (BPE) or its variations, by which OOV\n(out-of-vocab) words are almost avoidable. However, those methods split a word\ninto subword units and make the representation incomplete and fragile. In this\npaper, we propose a character-aware pre-trained language model named CharBERT\nimproving on the previous methods (such as BERT, RoBERTa) to tackle these\nproblems. We first construct the contextual word embedding for each token from\nthe sequential character representations, then fuse the representations of\ncharacters and the subword representations by a novel heterogeneous interaction\nmodule. We also propose a new pre-training task named NLM (Noisy LM) for\nunsupervised character representation learning. We evaluate our method on\nquestion answering, sequence labeling, and text classification tasks, both on\nthe original datasets and adversarial misspelling test sets. The experimental\nresults show that our method can significantly improve the performance and\nrobustness of PLMs simultaneously. Pretrained models, evaluation sets, and code\nare available at https://github.com/wtma/CharBERT",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01575v1",
            "date": "2020-11-03 09:02:57+00:00",
            "title": "AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings",
            "authors": [
                "Anne Lauscher",
                "Rafik Takieddin",
                "Simone Paolo Ponzetto",
                "Goran Glava\u0161"
            ],
            "abstract": "Recent work has shown that distributional word vector spaces often encode\nhuman biases like sexism or racism. In this work, we conduct an extensive\nanalysis of biases in Arabic word embeddings by applying a range of recently\nintroduced bias tests on a variety of embedding spaces induced from corpora in\nArabic. We measure the presence of biases across several dimensions, namely:\nembedding models (Skip-Gram, CBOW, and FastText) and vector sizes, types of\ntext (encyclopedic text, and news vs. user-generated content), dialects\n(Egyptian Arabic vs. Modern Standard Arabic), and time (diachronic analyses\nover corpora from different time periods). Our analysis yields several\ninteresting findings, e.g., that implicit gender bias in embeddings trained on\nArabic news corpora steadily increases over time (between 2007 and 2017). We\nmake the Arabic bias specifications (AraWEAT) publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01589v1",
            "date": "2020-11-03 09:40:25+00:00",
            "title": "Creating a Domain-diverse Corpus for Theory-based Argument Quality Assessment",
            "authors": [
                "Lily Ng",
                "Anne Lauscher",
                "Joel Tetreault",
                "Courtney Napoles"
            ],
            "abstract": "Computational models of argument quality (AQ) have focused primarily on\nassessing the overall quality or just one specific characteristic of an\nargument, such as its convincingness or its clarity. However, previous work has\nclaimed that assessment based on theoretical dimensions of argumentation could\nbenefit writers, but developing such models has been limited by the lack of\nannotated data. In this work, we describe GAQCorpus, the first large,\ndomain-diverse annotated corpus of theory-based AQ. We discuss how we designed\nthe annotation task to reliably collect a large number of judgments with\ncrowdsourcing, formulating theory-based guidelines that helped make subjective\njudgments of AQ more objective. We demonstrate how to identify arguments and\nadapt the annotation task for three diverse domains. Our work will inform\nresearch on theory-based argumentation annotation and enable the creation of\nmore diverse corpora to support computational AQ assessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01599v2",
            "date": "2020-11-03 10:01:44+00:00",
            "title": "Experiencers, Stimuli, or Targets: Which Semantic Roles Enable Machine Learning to Infer the Emotions?",
            "authors": [
                "Laura Oberl\u00e4nder",
                "Kevin Reich",
                "Roman Klinger"
            ],
            "abstract": "Emotion recognition is predominantly formulated as text classification in\nwhich textual units are assigned to an emotion from a predefined inventory\n(e.g., fear, joy, anger, disgust, sadness, surprise, trust, anticipation). More\nrecently, semantic role labeling approaches have been developed to extract\nstructures from the text to answer questions like: \"who is described to feel\nthe emotion?\" (experiencer), \"what causes this emotion?\" (stimulus), and at\nwhich entity is it directed?\" (target). Though it has been shown that jointly\nmodeling stimulus and emotion category prediction is beneficial for both\nsubtasks, it remains unclear which of these semantic roles enables a classifier\nto infer the emotion. Is it the experiencer, because the identity of a person\nis biased towards a particular emotion (X is always happy)? Is it a particular\ntarget (everybody loves X) or a stimulus (doing X makes everybody sad)? We\nanswer these questions by training emotion classification models on five\navailable datasets annotated with at least one semantic role by masking the\nfillers of these roles in the text in a controlled manner and find that across\nmultiple corpora, stimuli and targets carry emotion information, while the\nexperiencer might be considered a confounder. Further, we analyze if informing\nthe model about the position of the role improves the classification decision.\nParticularly on literature corpora we find that the role information improves\nthe emotion classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01612v2",
            "date": "2020-11-03 10:43:22+00:00",
            "title": "XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection",
            "authors": [
                "Emily \u00d6hman",
                "Marc P\u00e0mies",
                "Kaisla Kajava",
                "J\u00f6rg Tiedemann"
            ],
            "abstract": "We introduce XED, a multilingual fine-grained emotion dataset. The dataset\nconsists of human-annotated Finnish (25k) and English sentences (30k), as well\nas projected annotations for 30 additional languages, providing new resources\nfor many low-resource languages. We use Plutchik's core emotions to annotate\nthe dataset with the addition of neutral to create a multilabel multiclass\ndataset. The dataset is carefully evaluated using language-specific BERT models\nand SVMs to show that XED performs on par with other similar datasets and is\ntherefore a useful tool for sentiment analysis and emotion detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01615v1",
            "date": "2020-11-03 10:52:00+00:00",
            "title": "A Benchmark of Rule-Based and Neural Coreference Resolution in Dutch Novels and News",
            "authors": [
                "Corb\u00e8n Poot",
                "Andreas van Cranenburgh"
            ],
            "abstract": "We evaluate a rule-based (Lee et al., 2013) and neural (Lee et al., 2018)\ncoreference system on Dutch datasets of two domains: literary novels and\nnews/Wikipedia text. The results provide insight into the relative strengths of\ndata-driven and knowledge-driven systems, as well as the influence of domain,\ndocument length, and annotation schemes. The neural system performs best on\nnews/Wikipedia text, while the rule-based system performs best on literature.\nThe neural system shows weaknesses with limited training data and long\ndocuments, while the rule-based system is affected by annotation differences.\nThe code and models used in this paper are available at\nhttps://github.com/andreasvc/crac2020",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01624v1",
            "date": "2020-11-03 11:10:17+00:00",
            "title": "Results of a Single Blind Literary Taste Test with Short Anonymized Novel Fragments",
            "authors": [
                "Andreas van Cranenburgh",
                "Corina Koolen"
            ],
            "abstract": "It is an open question to what extent perceptions of literary quality are\nderived from text-intrinsic versus social factors. While supervised models can\npredict literary quality ratings from textual factors quite successfully, as\nshown in the Riddle of Literary Quality project (Koolen et al., 2020), this\ndoes not prove that social factors are not important, nor can we assume that\nreaders make judgments on literary quality in the same way and based on the\nsame information as machine learning models. We report the results of a pilot\nstudy to gauge the effect of textual features on literary ratings of\nDutch-language novels by participants in a controlled experiment with 48\nparticipants. In an exploratory analysis, we compare the ratings to those from\nthe large reader survey of the Riddle in which social factors were not\nexcluded, and to machine learning predictions of those literary ratings. We\nfind moderate to strong correlations of questionnaire ratings with the survey\nratings, but the predictions are closer to the survey ratings. Code and data:\nhttps://github.com/andreasvc/litquest",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01675v2",
            "date": "2020-11-03 13:04:31+00:00",
            "title": "Joint Entity and Relation Extraction with Set Prediction Networks",
            "authors": [
                "Dianbo Sui",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao",
                "Xiangrong Zeng",
                "Shengping Liu"
            ],
            "abstract": "The joint entity and relation extraction task aims to extract all relational\ntriples from a sentence. In essence, the relational triples contained in a\nsentence are unordered. However, previous seq2seq based models require to\nconvert the set of triples into a sequence in the training phase. To break this\nbottleneck, we treat joint entity and relation extraction as a direct set\nprediction problem, so that the extraction model can get rid of the burden of\npredicting the order of multiple triples. To solve this set prediction problem,\nwe propose networks featured by transformers with non-autoregressive parallel\ndecoding. Unlike autoregressive approaches that generate triples one by one in\na certain order, the proposed networks directly output the final set of triples\nin one shot. Furthermore, we also design a set-based loss that forces unique\npredictions via bipartite matching. Compared with cross-entropy loss that\nhighly penalizes small shifts in triple order, the proposed bipartite matching\nloss is invariant to any permutation of predictions; thus, it can provide the\nproposed networks with a more accurate training signal by ignoring triple order\nand focusing on relation types and entities. Experiments on two benchmark\ndatasets show that our proposed model significantly outperforms current\nstate-of-the-art methods. Training code and trained models will be available at\nhttp://github.com/DianboWork/SPN4RE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01682v1",
            "date": "2020-11-03 13:18:16+00:00",
            "title": "Cross-lingual Word Embeddings beyond Zero-shot Machine Translation",
            "authors": [
                "Shifei Chen",
                "Ali Basirat"
            ],
            "abstract": "We explore the transferability of a multilingual neural machine translation\nmodel to unseen languages when the transfer is grounded solely on the\ncross-lingual word embeddings. Our experimental results show that the\ntranslation knowledge can transfer weakly to other languages and that the\ndegree of transferability depends on the languages' relatedness. We also\ndiscuss the limiting aspects of the multilingual architectures that cause weak\ntranslation transfer and suggest how to mitigate the limitations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01694v2",
            "date": "2020-11-03 13:32:38+00:00",
            "title": "Data-to-Text Generation with Iterative Text Editing",
            "authors": [
                "Zden\u011bk Kasner",
                "Ond\u0159ej Du\u0161ek"
            ],
            "abstract": "We present a novel approach to data-to-text generation based on iterative\ntext editing. Our approach maximizes the completeness and semantic accuracy of\nthe output text while leveraging the abilities of recent pre-trained models for\ntext editing (LaserTagger) and language modeling (GPT-2) to improve the text\nfluency. To this end, we first transform data items to text using trivial\ntemplates, and then we iteratively improve the resulting text by a neural model\ntrained for the sentence fusion task. The output of the model is filtered by a\nsimple heuristic and reranked with an off-the-shelf pre-trained language model.\nWe evaluate our approach on two major data-to-text datasets (WebNLG, Cleaned\nE2E) and analyze its caveats and benefits. Furthermore, we show that our\nformulation of data-to-text generation opens up the possibility for zero-shot\ndomain adaptation using a general-domain dataset for sentence fusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01696v1",
            "date": "2020-11-03 13:34:36+00:00",
            "title": "Towards Automated Anamnesis Summarization: BERT-based Models for Symptom Extraction",
            "authors": [
                "Anton Sch\u00e4fer",
                "Nils Blach",
                "Oliver Rausch",
                "Maximilian Warm",
                "Nils Kr\u00fcger"
            ],
            "abstract": "Professionals in modern healthcare systems are increasingly burdened by\ndocumentation workloads. Documentation of the initial patient anamnesis is\nparticularly relevant, forming the basis of successful further diagnostic\nmeasures. However, manually prepared notes are inherently unstructured and\noften incomplete. In this paper, we investigate the potential of modern NLP\ntechniques to support doctors in this matter. We present a dataset of German\npatient monologues, and formulate a well-defined information extraction task\nunder the constraints of real-world utility and practicality. In addition, we\npropose BERT-based models in order to solve said task. We can demonstrate\npromising performance of the models in both symptom identification and symptom\nattribute extraction, significantly outperforming simpler baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01703v1",
            "date": "2020-11-03 13:45:54+00:00",
            "title": "Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translation",
            "authors": [
                "Annette Rios",
                "Mathias M\u00fcller",
                "Rico Sennrich"
            ],
            "abstract": "Zero-shot neural machine translation is an attractive goal because of the\nhigh cost of obtaining data and building translation systems for new\ntranslation directions. However, previous papers have reported mixed success in\nzero-shot translation. It is hard to predict in which settings it will be\neffective, and what limits performance compared to a fully supervised system.\nIn this paper, we investigate zero-shot performance of a multilingual\nEN$\\leftrightarrow${FR,CS,DE,FI} system trained on WMT data. We find that\nzero-shot performance is highly unstable and can vary by more than 6 BLEU\nbetween training runs, making it difficult to reliably track improvements. We\nobserve a bias towards copying the source in zero-shot translation, and\ninvestigate how the choice of subword segmentation affects this bias. We find\nthat language-specific subword segmentation results in less subword copying at\ntraining time, and leads to better zero-shot performance compared to jointly\ntrained segmentation. A recent trend in multilingual models is to not train on\nparallel data between all language pairs, but have a single bridge language,\ne.g. English. We find that this negatively affects zero-shot translation and\nleads to a failure mode where the model ignores the language tag and instead\nproduces English output in zero-shot directions. We show that this bias towards\nEnglish can be effectively reduced with even a small amount of parallel data in\nsome of the non-English pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01785v1",
            "date": "2020-11-03 15:28:07+00:00",
            "title": "Modeling Event Salience in Narratives via Barthes' Cardinal Functions",
            "authors": [
                "Takaki Otake",
                "Sho Yokoi",
                "Naoya Inoue",
                "Ryo Takahashi",
                "Tatsuki Kuribayashi",
                "Kentaro Inui"
            ],
            "abstract": "Events in a narrative differ in salience: some are more important to the\nstory than others. Estimating event salience is useful for tasks such as story\ngeneration, and as a tool for text analysis in narratology and folkloristics.\nTo compute event salience without any annotations, we adopt Barthes' definition\nof event salience and propose several unsupervised methods that require only a\npre-trained language model. Evaluating the proposed methods on folktales with\nevent salience annotation, we show that the proposed methods outperform\nbaseline methods and find fine-tuning a language model on narrative texts is a\nkey factor in improving the proposed methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01798v1",
            "date": "2020-11-03 15:45:42+00:00",
            "title": "Semi-Supervised Cleansing of Web Argument Corpora",
            "authors": [
                "Jonas Dorsch",
                "Henning Wachsmuth"
            ],
            "abstract": "Debate portals and similar web platforms constitute one of the main text\nsources in computational argumentation research and its applications. While the\ncorpora built upon these sources are rich of argumentatively relevant content\nand structure, they also include text that is irrelevant, or even detrimental,\nto their purpose. In this paper, we present a precision-oriented approach to\ndetecting such irrelevant text in a semi-supervised way. Given a few seed\nexamples, the approach automatically learns basic lexical patterns of relevance\nand irrelevance and then incrementally bootstraps new patterns from sentences\nmatching the patterns. In the existing args.me corpus with 400k argumentative\ntexts, our approach detects almost 87k irrelevant sentences, at a precision of\n0.97 according to manual evaluation. With low effort, the approach can be\nadapted to other web argument corpora, providing a generic way to improve\ncorpus quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01837v3",
            "date": "2020-11-03 16:50:13+00:00",
            "title": "The Gap on GAP: Tackling the Problem of Differing Data Distributions in Bias-Measuring Datasets",
            "authors": [
                "Vid Kocijan",
                "Oana-Maria Camburu",
                "Thomas Lukasiewicz"
            ],
            "abstract": "Diagnostic datasets that can detect biased models are an important\nprerequisite for bias reduction within natural language processing. However,\nundesired patterns in the collected data can make such tests incorrect. For\nexample, if the feminine subset of a gender-bias-measuring coreference\nresolution dataset contains sentences with a longer average distance between\nthe pronoun and the correct candidate, an RNN-based model may perform worse on\nthis subset due to long-term dependencies. In this work, we introduce a\ntheoretically grounded method for weighting test samples to cope with such\npatterns in the test data. We demonstrate the method on the GAP dataset for\ncoreference resolution. We annotate GAP with spans of all personal names and\nshow that examples in the female subset contain more personal names and a\nlonger distance between pronouns and their referents, potentially affecting the\nbias score in an undesired way. Using our weighting method, we find the set of\nweights on the test instances that should be used for coping with these\ncorrelations, and we re-evaluate 16 recently released coreference models.",
            "journal ref": "AAAI 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01846v1",
            "date": "2020-11-03 17:01:44+00:00",
            "title": "Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks",
            "authors": [
                "Denis Emelin",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "Word sense disambiguation is a well-known source of translation errors in\nNMT. We posit that some of the incorrect disambiguation choices are due to\nmodels' over-reliance on dataset artifacts found in training data, specifically\nsuperficial word co-occurrences, rather than a deeper understanding of the\nsource text. We introduce a method for the prediction of disambiguation errors\nbased on statistical data properties, demonstrating its effectiveness across\nseveral domains and model types. Moreover, we develop a simple adversarial\nattack strategy that minimally perturbs sentences in order to elicit\ndisambiguation errors to further probe the robustness of translation models.\nOur findings indicate that disambiguation robustness varies substantially\nbetween domains and that different models trained on the same data are\nvulnerable to different attacks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01856v1",
            "date": "2020-11-03 17:18:03+00:00",
            "title": "Finding Friends and Flipping Frenemies: Automatic Paraphrase Dataset Augmentation Using Graph Theory",
            "authors": [
                "Hannah Chen",
                "Yangfeng Ji",
                "David Evans"
            ],
            "abstract": "Most NLP datasets are manually labeled, so suffer from inconsistent labeling\nor limited size. We propose methods for automatically improving datasets by\nviewing them as graphs with expected semantic properties. We construct a\nparaphrase graph from the provided sentence pair labels, and create an\naugmented dataset by directly inferring labels from the original sentence pairs\nusing a transitivity property. We use structural balance theory to identify\nlikely mislabelings in the graph, and flip their labels. We evaluate our\nmethods on paraphrase models trained using these datasets starting from a\npretrained BERT model, and find that the automatically-enhanced training sets\nresult in more accurate models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01860v1",
            "date": "2020-11-03 17:30:05+00:00",
            "title": "Decoupling entrainment from consistency using deep neural networks",
            "authors": [
                "Andreas Weise",
                "Rivka Levitan"
            ],
            "abstract": "Human interlocutors tend to engage in adaptive behavior known as entrainment\nto become more similar to each other. Isolating the effect of consistency,\ni.e., speakers adhering to their individual styles, is a critical part of the\nanalysis of entrainment. We propose to treat speakers' initial vocal features\nas confounds for the prediction of subsequent outputs. Using two existing\nneural approaches to deconfounding, we define new measures of entrainment that\ncontrol for consistency. These successfully discriminate real interactions from\nfake ones. Interestingly, our stricter methods correlate with social variables\nin opposite direction from previous measures that do not account for\nconsistency. These results demonstrate the advantages of using neural networks\nto model entrainment, and raise questions regarding how to interpret prior\nassociations of conversation quality with entrainment measures that do not\naccount for consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01913v1",
            "date": "2020-11-03 18:43:19+00:00",
            "title": "Towards Code-switched Classification Exploiting Constituent Language Resources",
            "authors": [
                "Tanvi Dadu",
                "Kartikey Pant"
            ],
            "abstract": "Code-switching is a commonly observed communicative phenomenon denoting a\nshift from one language to another within the same speech exchange. The\nanalysis of code-switched data often becomes an assiduous task, owing to the\nlimited availability of data. We propose converting code-switched data into its\nconstituent high resource languages for exploiting both monolingual and\ncross-lingual settings in this work. This conversion allows us to utilize the\nhigher resource availability for its constituent languages for multiple\ndownstream tasks.\n  We perform experiments for two downstream tasks, sarcasm detection and hate\nspeech detection, in the English-Hindi code-switched setting. These experiments\nshow an increase in 22% and 42.5% in F1-score for sarcasm detection and hate\nspeech detection, respectively, compared to the state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02048v1",
            "date": "2020-11-03 22:47:58+00:00",
            "title": "SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation",
            "authors": [
                "Xutai Ma",
                "Juan Pino",
                "Philipp Koehn"
            ],
            "abstract": "Simultaneous text translation and end-to-end speech translation have recently\nmade great progress but little work has combined these tasks together. We\ninvestigate how to adapt simultaneous text translation methods such as wait-k\nand monotonic multihead attention to end-to-end simultaneous speech translation\nby introducing a pre-decision module. A detailed analysis is provided on the\nlatency-quality trade-offs of combining fixed and flexible pre-decision with\nfixed and flexible policies. We also design a novel computation-aware latency\nmetric, adapted from Average Lagging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02050v1",
            "date": "2020-11-03 22:55:40+00:00",
            "title": "Generating Synthetic Data for Task-Oriented Semantic Parsing with Hierarchical Representations",
            "authors": [
                "Ke Tran",
                "Ming Tan"
            ],
            "abstract": "Modern conversational AI systems support natural language understanding for a\nwide variety of capabilities. While a majority of these tasks can be\naccomplished using a simple and flat representation of intents and slots, more\nsophisticated capabilities require complex hierarchical representations\nsupported by semantic parsing. State-of-the-art semantic parsers are trained\nusing supervised learning with data labeled according to a hierarchical schema\nwhich might be costly to obtain or not readily available for a new domain. In\nthis work, we explore the possibility of generating synthetic data for neural\nsemantic parsing using a pretrained denoising sequence-to-sequence model (i.e.,\nBART). Specifically, we first extract masked templates from the existing\nlabeled utterances, and then fine-tune BART to generate synthetic utterances\nconditioning on the extracted templates. Finally, we use an auxiliary parser\n(AP) to filter the generated utterances. The AP guarantees the quality of the\ngenerated data. We show the potential of our approach when evaluating on the\nFacebook TOP dataset for navigation domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02063v1",
            "date": "2020-11-03 23:34:42+00:00",
            "title": "Treebanking User-Generated Content: a UD Based Overview of Guidelines, Corpora and Unified Recommendations",
            "authors": [
                "Manuela Sanguinetti",
                "Lauren Cassidy",
                "Cristina Bosco",
                "\u00d6zlem \u00c7etino\u011flu",
                "Alessandra Teresa Cignarella",
                "Teresa Lynn",
                "Ines Rehbein",
                "Josef Ruppenhofer",
                "Djam\u00e9 Seddah",
                "Amir Zeldes"
            ],
            "abstract": "This article presents a discussion on the main linguistic phenomena which\ncause difficulties in the analysis of user-generated texts found on the web and\nin social media, and proposes a set of annotation guidelines for their\ntreatment within the Universal Dependencies (UD) framework of syntactic\nanalysis. Given on the one hand the increasing number of treebanks featuring\nuser-generated content, and its somewhat inconsistent treatment in these\nresources on the other, the aim of this article is twofold: (1) to provide a\ncondensed, though comprehensive, overview of such treebanks -- based on\navailable literature -- along with their main features and a comparative\nanalysis of their annotation criteria, and (2) to propose a set of tentative\nUD-based annotation guidelines, to promote consistent treatment of the\nparticular phenomena found in these types of texts. The overarching goal of\nthis article is to provide a common framework for researchers interested in\ndeveloping similar resources in UD, thus promoting cross-linguistic\nconsistency, which is a principle that has always been central to the spirit of\nUD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02070v1",
            "date": "2020-11-04 00:03:04+00:00",
            "title": "Probing Multilingual BERT for Genetic and Typological Signals",
            "authors": [
                "Taraka Rama",
                "Lisa Beinborn",
                "Steffen Eger"
            ],
            "abstract": "We probe the layers in multilingual BERT (mBERT) for phylogenetic and\ngeographic language signals across 100 languages and compute language distances\nbased on the mBERT representations. We 1) employ the language distances to\ninfer and evaluate language trees, finding that they are close to the reference\nfamily tree in terms of quartet tree distance, 2) perform distance matrix\nregression analysis, finding that the language distances can be best explained\nby phylogenetic and worst by structural factors and 3) present a novel measure\nfor measuring diachronic meaning stability (based on cross-lingual\nrepresentation variability) which correlates significantly with published\nranked lists based on linguistic approaches. Our results contribute to the\nnascent field of typological interpretability of cross-lingual text\nrepresentations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02093v1",
            "date": "2020-11-04 01:23:30+00:00",
            "title": "Chinese Grammatical Correction Using BERT-based Pre-trained Model",
            "authors": [
                "Hongfei Wang",
                "Michiki Kurosawa",
                "Satoru Katsumata",
                "Mamoru Komachi"
            ],
            "abstract": "In recent years, pre-trained models have been extensively studied, and\nseveral downstream tasks have benefited from their utilization. In this study,\nwe verify the effectiveness of two methods that incorporate a BERT-based\npre-trained model developed by Cui et al. (2020) into an encoder-decoder model\non Chinese grammatical error correction tasks. We also analyze the error type\nand conclude that sentence-level errors are yet to be addressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02121v1",
            "date": "2020-11-04 04:44:47+00:00",
            "title": "PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents",
            "authors": [
                "Ryo Fujii",
                "Masato Mita",
                "Kaori Abe",
                "Kazuaki Hanawa",
                "Makoto Morishita",
                "Jun Suzuki",
                "Kentaro Inui"
            ],
            "abstract": "Neural Machine Translation (NMT) has shown drastic improvement in its quality\nwhen translating clean input, such as text from the news domain. However,\nexisting studies suggest that NMT still struggles with certain kinds of input\nwith considerable noise, such as User-Generated Contents (UGC) on the Internet.\nTo make better use of NMT for cross-cultural communication, one of the most\npromising directions is to develop a model that correctly handles these\nexpressions. Though its importance has been recognized, it is still not clear\nas to what creates the great gap in performance between the translation of\nclean input and that of UGC. To answer the question, we present a new dataset,\nPheMT, for evaluating the robustness of MT systems against specific linguistic\nphenomena in Japanese-English translation. Our experiments with the created\ndataset revealed that not only our in-house models but even widely used\noff-the-shelf systems are greatly disturbed by the presence of certain\nphenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02173v1",
            "date": "2020-11-04 08:24:05+00:00",
            "title": "Neural text normalization leveraging similarities of strings and sounds",
            "authors": [
                "Riku Kawamura",
                "Tatsuya Aoki",
                "Hidetaka Kamigaito",
                "Hiroya Takamura",
                "Manabu Okumura"
            ],
            "abstract": "We propose neural models that can normalize text by considering the\nsimilarities of word strings and sounds. We experimentally compared a model\nthat considers the similarities of both word strings and sounds, a model that\nconsiders only the similarity of word strings or of sounds, and a model without\nthe similarities as a baseline. Results showed that leveraging the word string\nsimilarity succeeded in dealing with misspellings and abbreviations, and taking\ninto account the sound similarity succeeded in dealing with phonetic\nsubstitutions and emphasized characters. So that the proposed models achieved\nhigher F$_1$ scores than the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02207v1",
            "date": "2020-11-04 10:14:31+00:00",
            "title": "Extracting Chemical-Protein Interactions via Calibrated Deep Neural Network and Self-training",
            "authors": [
                "Dongha Choi",
                "Hyunju Lee"
            ],
            "abstract": "The extraction of interactions between chemicals and proteins from several\nbiomedical articles is important in many fields of biomedical research such as\ndrug development and prediction of drug side effects. Several natural language\nprocessing methods, including deep neural network (DNN) models, have been\napplied to address this problem. However, these methods were trained with\nhard-labeled data, which tend to become over-confident, leading to degradation\nof the model reliability. To estimate the data uncertainty and improve the\nreliability, \"calibration\" techniques have been applied to deep learning\nmodels. In this study, to extract chemical--protein interactions, we propose a\nDNN-based approach incorporating uncertainty information and calibration\ntechniques. Our model first encodes the input sequence using a pre-trained\nlanguage-understanding model, following which it is trained using two\ncalibration methods: mixup training and addition of a confidence penalty loss.\nFinally, the model is re-trained with augmented data that are extracted using\nthe estimated uncertainties. Our approach has achieved state-of-the-art\nperformance with regard to the Biocreative VI ChemProt task, while preserving\nhigher calibration abilities than those of previous approaches. Furthermore,\nour approach also presents the possibilities of using uncertainty estimation\nfor performance improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02378v1",
            "date": "2020-11-04 16:12:39+00:00",
            "title": "A BERT-based Dual Embedding Model for Chinese Idiom Prediction",
            "authors": [
                "Minghuan Tan",
                "Jing Jiang"
            ],
            "abstract": "Chinese idioms are special fixed phrases usually derived from ancient\nstories, whose meanings are oftentimes highly idiomatic and non-compositional.\nThe Chinese idiom prediction task is to select the correct idiom from a set of\ncandidate idioms given a context with a blank. We propose a BERT-based dual\nembedding model to encode the contextual words as well as to learn dual\nembeddings of the idioms. Specifically, we first match the embedding of each\ncandidate idiom with the hidden representation corresponding to the blank in\nthe context. We then match the embedding of each candidate idiom with the\nhidden representations of all the tokens in the context thorough context\npooling. We further propose to use two separate idiom embeddings for the two\nkinds of matching. Experiments on a recently released Chinese idiom cloze test\ndataset show that our proposed method performs better than the existing state\nof the art. Ablation experiments also show that both context pooling and dual\nembedding contribute to the improvement of performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02541v1",
            "date": "2020-11-04 21:16:48+00:00",
            "title": "MTLB-STRUCT @PARSEME 2020: Capturing Unseen Multiword Expressions Using Multi-task Learning and Pre-trained Masked Language Models",
            "authors": [
                "Shiva Taslimipoor",
                "Sara Bahaadini",
                "Ekaterina Kochmar"
            ],
            "abstract": "This paper describes a semi-supervised system that jointly learns verbal\nmultiword expressions (VMWEs) and dependency parse trees as an auxiliary task.\nThe model benefits from pre-trained multilingual BERT. BERT hidden layers are\nshared among the two tasks and we introduce an additional linear layer to\nretrieve VMWE tags. The dependency parse tree prediction is modelled by a\nlinear layer and a bilinear one plus a tree CRF on top of BERT. The system has\nparticipated in the open track of the PARSEME shared task 2020 and ranked first\nin terms of F1-score in identifying unseen VMWEs as well as VMWEs in general,\naveraged across all 14 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02566v1",
            "date": "2020-11-04 22:33:05+00:00",
            "title": "MK-SQuIT: Synthesizing Questions using Iterative Template-filling",
            "authors": [
                "Benjamin A. Spiegel",
                "Vincent Cheong",
                "James E. Kaplan",
                "Anthony Sanchez"
            ],
            "abstract": "The aim of this work is to create a framework for synthetically generating\nquestion/query pairs with as little human input as possible. These datasets can\nbe used to train machine translation systems to convert natural language\nquestions into queries, a useful tool that could allow for more natural access\nto database information. Existing methods of dataset generation require human\ninput that scales linearly with the size of the dataset, resulting in small\ndatasets. Aside from a short initial configuration task, no human input is\nrequired during the query generation process of our system. We leverage\nWikiData, a knowledge base of RDF triples, as a source for generating the main\ncontent of questions and queries. Using multiple layers of question templating\nwe are able to sidestep some of the most challenging parts of query generation\nthat have been handled by humans in previous methods; humans never have to\nmodify, aggregate, inspect, annotate, or generate any questions or queries at\nany step in the process. Our system is easily configurable to multiple domains\nand can be modified to generate queries in natural languages other than\nEnglish. We also present an example dataset of 110,000 question/query pairs\nacross four WikiData domains. We then present a baseline model that we train\nusing the dataset which shows promise in a commercial QA setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02686v1",
            "date": "2020-11-05 07:07:40+00:00",
            "title": "Investigating Societal Biases in a Poetry Composition System",
            "authors": [
                "Emily Sheng",
                "David Uthus"
            ],
            "abstract": "There is a growing collection of work analyzing and mitigating societal\nbiases in language understanding, generation, and retrieval tasks, though\nexamining biases in creative tasks remains underexplored. Creative language\napplications are meant for direct interaction with users, so it is important to\nquantify and mitigate societal biases in these applications. We introduce a\nnovel study on a pipeline to mitigate societal biases when retrieving next\nverse suggestions in a poetry composition system. Our results suggest that data\naugmentation through sentiment style transfer has potential for mitigating\nsocietal biases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02687v1",
            "date": "2020-11-05 07:10:08+00:00",
            "title": "Context-Aware Answer Extraction in Question Answering",
            "authors": [
                "Yeon Seonwoo",
                "Ji-Hoon Kim",
                "Jung-Woo Ha",
                "Alice Oh"
            ],
            "abstract": "Extractive QA models have shown very promising performance in predicting the\ncorrect answer to a question for a given passage. However, they sometimes\nresult in predicting the correct answer text but in a context irrelevant to the\ngiven question. This discrepancy becomes especially important as the number of\noccurrences of the answer text in a passage increases. To resolve this issue,\nwe propose \\textbf{BLANC} (\\textbf{BL}ock \\textbf{A}ttentio\\textbf{N} for\n\\textbf{C}ontext prediction) based on two main ideas: context prediction as an\nauxiliary task in multi-task learning manner, and a block attention method that\nlearns the context prediction task. With experiments on reading comprehension,\nwe show that BLANC outperforms the state-of-the-art QA models, and the\nperformance gap increases as the number of answer text occurrences increases.\nWe also conduct an experiment of training the models using SQuAD and predicting\nthe supporting facts on HotpotQA and show that BLANC outperforms all baseline\nmodels in this zero-shot setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02788v2",
            "date": "2020-11-05 12:39:30+00:00",
            "title": "NUAA-QMUL at SemEval-2020 Task 8: Utilizing BERT and DenseNet for Internet Meme Emotion Analysis",
            "authors": [
                "Xiaoyu Guo",
                "Jing Ma",
                "Arkaitz Zubiaga"
            ],
            "abstract": "This paper describes our contribution to SemEval 2020 Task 8: Memotion\nAnalysis. Our system learns multi-modal embeddings from text and images in\norder to classify Internet memes by sentiment. Our model learns text embeddings\nusing BERT and extracts features from images with DenseNet, subsequently\ncombining both features through concatenation. We also compare our results with\nthose produced by DenseNet, ResNet, BERT, and BERT-ResNet. Our results show\nthat image classification models have the potential to help classifying memes,\nwith DenseNet outperforming ResNet. Adding text features is however not always\nhelpful for Memotion Analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02821v3",
            "date": "2020-11-05 13:58:32+00:00",
            "title": "Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translation",
            "authors": [
                "Aloka Fernando",
                "Surangika Ranathunga",
                "Gihan Dias"
            ],
            "abstract": "Out of vocabulary (OOV) is a problem in the context of Machine Translation\n(MT) in low-resourced languages. When source and/or target languages are\nmorphologically rich, it becomes even worse. Bilingual list integration is an\napproach to address the OOV problem. This allows more words to be translated\nthan are in the training data. However, since bilingual lists contain words in\nthe base form, it will not translate inflected forms for morphologically rich\nlanguages such as Sinhala and Tamil. This paper focuses on data augmentation\ntechniques where bilingual lexicon terms are expanded based on case-markers\nwith the objective of generating new words, to be used in Statistical machine\nTranslation (SMT). This data augmentation technique for dictionary terms shows\nimproved BLEU scores for Sinhala-English SMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02930v2",
            "date": "2020-11-04 14:11:35+00:00",
            "title": "Paralinguistic Privacy Protection at the Edge",
            "authors": [
                "Ranya Aloufi",
                "Hamed Haddadi",
                "David Boyle"
            ],
            "abstract": "Voice user interfaces and digital assistants are rapidly entering our lives\nand becoming singular touch points spanning our devices. These always-on\nservices capture and transmit our audio data to powerful cloud services for\nfurther processing and subsequent actions. Our voices and raw audio signals\ncollected through these devices contain a host of sensitive paralinguistic\ninformation that is transmitted to service providers regardless of deliberate\nor false triggers. As our emotional patterns and sensitive attributes like our\nidentity, gender, mental well-being, are easily inferred using deep acoustic\nmodels, we encounter a new generation of privacy risks by using these services.\nOne approach to mitigate the risk of paralinguistic-based privacy breaches is\nto exploit a combination of cloud-based processing with privacy-preserving,\non-device paralinguistic information learning and filtering before transmitting\nvoice data. In this paper we introduce EDGY, a configurable, lightweight,\ndisentangled representation learning framework that transforms and filters\nhigh-dimensional voice data to identify and contain sensitive attributes at the\nedge prior to offloading to the cloud. We evaluate EDGY's on-device performance\nand explore optimization techniques, including model quantization and knowledge\ndistillation, to enable private, accurate and efficient representation learning\non resource-constrained devices. Our results show that EDGY runs in tens of\nmilliseconds with 0.2% relative improvement in ABX score or minimal performance\npenalties in learning linguistic representations from raw voice signals, using\na CPU and a single-core ARM processor without specialized hardware.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02935v2",
            "date": "2020-11-05 16:00:35+00:00",
            "title": "QMUL-SDS @ DIACR-Ita: Evaluating Unsupervised Diachronic Lexical Semantics Classification in Italian",
            "authors": [
                "Rabab Alkhalifa",
                "Adam Tsakalidis",
                "Arkaitz Zubiaga",
                "Maria Liakata"
            ],
            "abstract": "In this paper, we present the results and main findings of our system for the\nDIACR-ITA 2020 Task. Our system focuses on using variations of training sets\nand different semantic detection methods. The task involves training, aligning\nand predicting a word's vector change from two diachronic Italian corpora. We\ndemonstrate that using Temporal Word Embeddings with a Compass C-BOW model is\nmore effective compared to different approaches including Logistic Regression\nand a Feed Forward Neural Network using accuracy. Our model ranked 3rd with an\naccuracy of 83.3%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02944v1",
            "date": "2020-11-05 16:08:50+00:00",
            "title": "Learning Efficient Task-Specific Meta-Embeddings with Word Prisms",
            "authors": [
                "Jingyi He",
                "KC Tsiolis",
                "Kian Kenyon-Dean",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Word embeddings are trained to predict word cooccurrence statistics, which\nleads them to possess different lexical properties (syntactic, semantic, etc.)\ndepending on the notion of context defined at training time. These properties\nmanifest when querying the embedding space for the most similar vectors, and\nwhen used at the input layer of deep neural networks trained to solve\ndownstream NLP problems. Meta-embeddings combine multiple sets of differently\ntrained word embeddings, and have been shown to successfully improve intrinsic\nand extrinsic performance over equivalent models which use just one set of\nsource embeddings. We introduce word prisms: a simple and efficient\nmeta-embedding method that learns to combine source embeddings according to the\ntask at hand. Word prisms learn orthogonal transformations to linearly combine\nthe input source embeddings, which allows them to be very efficient at\ninference time. We evaluate word prisms in comparison to other meta-embedding\nmethods on six extrinsic evaluations and observe that word prisms offer\nimprovements in performance on all tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02947v3",
            "date": "2020-11-05 16:16:49+00:00",
            "title": "CODER: Knowledge infused cross-lingual medical term embedding for term normalization",
            "authors": [
                "Zheng Yuan",
                "Zhengyun Zhao",
                "Haixia Sun",
                "Jiao Li",
                "Fei Wang",
                "Sheng Yu"
            ],
            "abstract": "This paper proposes CODER: contrastive learning on knowledge graphs for\ncross-lingual medical term representation. CODER is designed for medical term\nnormalization by providing close vector representations for different terms\nthat represent the same or similar medical concepts with cross-lingual support.\nWe train CODER via contrastive learning on a medical knowledge graph (KG) named\nthe Unified Medical Language System, where similarities are calculated\nutilizing both terms and relation triplets from KG. Training with relations\ninjects medical knowledge into embeddings and aims to provide potentially\nbetter machine learning features. We evaluate CODER in zero-shot term\nnormalization, semantic similarity, and relation classification benchmarks,\nwhich show that CODERoutperforms various state-of-the-art biomedical word\nembedding, concept embeddings, and contextual embeddings. Our codes and models\nare available at https://github.com/GanjinZero/CODER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02998v1",
            "date": "2020-11-05 17:47:03+00:00",
            "title": "Competence-Level Prediction and Resume & Job Description Matching Using Context-Aware Transformer Models",
            "authors": [
                "Changmao Li",
                "Elaine Fisher",
                "Rebecca Thomas",
                "Steve Pittard",
                "Vicki Hertzberg",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents a comprehensive study on resume classification to reduce\nthe time and labor needed to screen an overwhelming number of applications\nsignificantly, while improving the selection of suitable candidates. A total of\n6,492 resumes are extracted from 24,933 job applications for 252 positions\ndesignated into four levels of experience for Clinical Research Coordinators\n(CRC). Each resume is manually annotated to its most appropriate CRC position\nby experts through several rounds of triple annotation to establish guidelines.\nAs a result, a high Kappa score of 61% is achieved for inter-annotator\nagreement. Given this dataset, novel transformer-based classification models\nare developed for two tasks: the first task takes a resume and classifies it to\na CRC level (T1), and the second task takes both a resume and a job description\nto apply and predicts if the application is suited to the job T2. Our best\nmodels using section encoding and multi-head attention decoding give results of\n73.3% to T1 and 79.2% to T2. Our analysis shows that the prediction errors are\nmostly made among adjacent CRC levels, which are hard for even experts to\ndistinguish, implying the practical value of our models in real HR platforms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03017v1",
            "date": "2020-11-05 18:22:38+00:00",
            "title": "MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision",
            "authors": [
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "The lack of large and diverse discourse treebanks hinders the application of\ndata-driven approaches, such as deep-learning, to RST-style discourse parsing.\nIn this work, we present a novel scalable methodology to automatically generate\ndiscourse treebanks using distant supervision from sentiment-annotated\ndatasets, creating and publishing MEGA-DT, a new large-scale\ndiscourse-annotated corpus. Our approach generates discourse trees\nincorporating structure and nuclearity for documents of arbitrary length by\nrelying on an efficient heuristic beam-search strategy, extended with a\nstochastic component. Experiments on multiple datasets indicate that a\ndiscourse parser trained on our MEGA-DT treebank delivers promising\ninter-domain performance gains when compared to parsers trained on\nhuman-annotated discourse corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03021v1",
            "date": "2020-11-05 18:28:13+00:00",
            "title": "From Sentiment Annotations to Sentiment Prediction through Discourse Augmentation",
            "authors": [
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "Sentiment analysis, especially for long documents, plausibly requires methods\ncapturing complex linguistics structures. To accommodate this, we propose a\nnovel framework to exploit task-related discourse for the task of sentiment\nanalysis. More specifically, we are combining the large-scale,\nsentiment-dependent MEGA-DT treebank with a novel neural architecture for\nsentiment prediction, based on a hybrid TreeLSTM hierarchical attention model.\nExperiments show that our framework using sentiment-related discourse\naugmentations for sentiment prediction enhances the overall performance for\nlong documents, even beyond previous approaches using well-established\ndiscourse parsers trained on human annotated data. We show that a simple\nensemble approach can further enhance performance by selectively using\ndiscourse, depending on the document length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03161v2",
            "date": "2020-11-06 02:23:01+00:00",
            "title": "What's New? Summarizing Contributions in Scientific Literature",
            "authors": [
                "Hiroaki Hayashi",
                "Wojciech Kry\u015bci\u0144ski",
                "Bryan McCann",
                "Nazneen Rajani",
                "Caiming Xiong"
            ],
            "abstract": "With thousands of academic articles shared on a daily basis, it has become\nincreasingly difficult to keep up with the latest scientific findings. To\novercome this problem, we introduce a new task of disentangled paper\nsummarization, which seeks to generate separate summaries for the paper\ncontributions and the context of the work, making it easier to identify the key\nfindings shared in articles. For this purpose, we extend the S2ORC corpus of\nacademic articles, which spans a diverse set of domains ranging from economics\nto psychology, by adding disentangled \"contribution\" and \"context\" reference\nlabels. Together with the dataset, we introduce and analyze three baseline\napproaches: 1) a unified model controlled by input code prefixes, 2) a model\nwith separate generation heads specialized in generating the disentangled\noutputs, and 3) a training strategy that guides the model using additional\nsupervision coming from inbound and outbound citations. We also propose a\ncomprehensive automatic evaluation protocol which reports the relevance,\nnovelty, and disentanglement of generated outputs. Through a human study\ninvolving expert annotators, we show that in 79%, of cases our new task is\nconsidered more helpful than traditional scientific paper summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03203v1",
            "date": "2020-11-06 06:11:26+00:00",
            "title": "Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining",
            "authors": [
                "Grigorii Guz",
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "RST-based discourse parsing is an important NLP task with numerous downstream\napplications, such as summarization, machine translation and opinion mining. In\nthis paper, we demonstrate a simple, yet highly accurate discourse parser,\nincorporating recent contextual language models. Our parser establishes the new\nstate-of-the-art (SOTA) performance for predicting structure and nuclearity on\ntwo key RST datasets, RST-DT and Instr-DT. We further demonstrate that\npretraining our parser on the recently available large-scale \"silver-standard\"\ndiscourse treebank MEGA-DT provides even larger performance benefits,\nsuggesting a novel and promising research direction in the field of discourse\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03258v1",
            "date": "2020-11-06 10:02:12+00:00",
            "title": "OP-IMS @ DIACR-Ita: Back to the Roots: SGNS+OP+CD still rocks Semantic Change Detection",
            "authors": [
                "Jens Kaiser",
                "Dominik Schlechtweg",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We present the results of our participation in the DIACR-Ita shared task on\nlexical semantic change detection for Italian. We exploit one of the earliest\nand most influential semantic change detection models based on Skip-Gram with\nNegative Sampling, Orthogonal Procrustes alignment and Cosine Distance and\nobtain the winning submission of the shared task with near to perfect accuracy\n.94. Our results once more indicate that, within the present task setup in\nlexical semantic change detection, the traditional type-based approaches yield\nexcellent performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03259v1",
            "date": "2020-11-06 10:06:10+00:00",
            "title": "Alquist 2.0: Alexa Prize Socialbot Based on Sub-Dialogue Models",
            "authors": [
                "Jan Pichl",
                "Petr Marek",
                "Jakub Konr\u00e1d",
                "Martin Matul\u00edk",
                "Jan \u0160ediv\u00fd"
            ],
            "abstract": "This paper presents the second version of the dialogue system named Alquist\ncompeting in Amazon Alexa Prize 2018. We introduce a system leveraging\nontology-based topic structure called topic nodes. Each of the nodes consists\nof several sub-dialogues, and each sub-dialogue has its own LSTM-based model\nfor dialogue management. The sub-dialogues can be triggered according to the\ntopic hierarchy or a user intent which allows the bot to create a unique\nexperience during each session.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03261v1",
            "date": "2020-11-06 10:10:02+00:00",
            "title": "Alquist 3.0: Alexa Prize Bot Using Conversational Knowledge Graph",
            "authors": [
                "Jan Pichl",
                "Petr Marek",
                "Jakub Konr\u00e1d",
                "Petr Lorenc",
                "Van Duy Ta",
                "Jan \u0160ediv\u00fd"
            ],
            "abstract": "The third version of the open-domain dialogue system Alquist developed within\nthe Alexa Prize 2020 competition is designed to conduct coherent and engaging\nconversations on popular topics. The main novel contribution is the\nintroduction of a system leveraging an innovative approach based on a\nconversational knowledge graph and adjacency pairs. The conversational\nknowledge graph allows the system to utilize knowledge expressed during the\ndialogue in consequent turns and across conversations. Dialogue adjacency pairs\ndivide the conversation into small conversational structures, which can be\ncombined and allow the system to react to a wide range of user inputs flexibly.\n  We discuss and describe Alquist's pipeline, data acquisition and processing,\ndialogue manager, NLG, knowledge aggregation, and a hierarchy of adjacency\npairs. We present the experimental results of the individual parts of the\nsystem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03286v2",
            "date": "2020-11-06 11:19:47+00:00",
            "title": "Semi-Supervised Low-Resource Style Transfer of Indonesian Informal to Formal Language with Iterative Forward-Translation",
            "authors": [
                "Haryo Akbarianto Wibowo",
                "Tatag Aziz Prawiro",
                "Muhammad Ihsan",
                "Alham Fikri Aji",
                "Radityo Eko Prasojo",
                "Rahmad Mahendra",
                "Suci Fitriany"
            ],
            "abstract": "In its daily use, the Indonesian language is riddled with informality, that\nis, deviations from the standard in terms of vocabulary, spelling, and word\norder. On the other hand, current available Indonesian NLP models are typically\ndeveloped with the standard Indonesian in mind. In this work, we address a\nstyle-transfer from informal to formal Indonesian as a low-resource machine\ntranslation problem. We build a new dataset of parallel sentences of informal\nIndonesian and its formal counterpart. We benchmark several strategies to\nperform style transfer from informal to formal Indonesian. We also explore\naugmenting the training set with artificial forward-translated data. Since we\nare dealing with an extremely low-resource setting, we find that a phrase-based\nmachine translation approach outperforms the Transformer-based approach.\nAlternatively, a pre-trained GPT-2 fined-tuned to this task performed equally\nwell but costs more computational resource. Our findings show a promising step\ntowards leveraging machine translation models for style transfer. Our code and\ndata are available in https://github.com/haryoa/stif-indonesia",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03287v1",
            "date": "2020-11-06 11:23:09+00:00",
            "title": "The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generation",
            "authors": [
                "Yova Kementchedjhieva",
                "Di Lu",
                "Joel Tetreault"
            ],
            "abstract": "News articles, image captions, product reviews and many other texts mention\npeople and organizations whose name recognition could vary for different\naudiences. In such cases, background information about the named entities could\nbe provided in the form of an appositive noun phrase, either written by a human\nor generated automatically. We expand on the previous work in appositive\ngeneration with a new, more realistic, end-to-end definition of the task,\ninstantiated by a dataset that spans four languages (English, Spanish, German\nand Polish), two entity types (person and organization) and two domains\n(Wikipedia and News). We carry out an extensive analysis of the data and the\ntask, pointing to the various modeling challenges it poses. The results we\nobtain with standard language generation methods show that the task is indeed\nnon-trivial, and leaves plenty of room for improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03292v2",
            "date": "2020-11-06 11:33:29+00:00",
            "title": "Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning",
            "authors": [
                "Yufan Jiang",
                "Shuangzhi Wu",
                "Jing Gong",
                "Yahui Cheng",
                "Peng Meng",
                "Weiliang Lin",
                "Zhibo Chen",
                "Mu li"
            ],
            "abstract": "Multi-choice Machine Reading Comprehension (MMRC) aims to select the correct\nanswer from a set of options based on a given passage and question. Due to task\nspecific of MMRC, it is non-trivial to transfer knowledge from other MRC tasks\nsuch as SQuAD, Dream. In this paper, we simply reconstruct multi-choice to\nsingle-choice by training a binary classification to distinguish whether a\ncertain answer is correct. Then select the option with the highest confidence\nscore. We construct our model upon ALBERT-xxlarge model and estimate it on the\nRACE dataset. During training, We adopt AutoML strategy to tune better\nparameters. Experimental results show that the single-choice is better than\nmulti-choice. In addition, by transferring knowledge from other kinds of MRC\ntasks, our model achieves a new state-of-the-art results in both single and\nensemble settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03469v1",
            "date": "2020-11-06 16:47:43+00:00",
            "title": "Understanding Pure Character-Based Neural Machine Translation: The Case of Translating Finnish into English",
            "authors": [
                "Gongbo Tang",
                "Rico Sennrich",
                "Joakim Nivre"
            ],
            "abstract": "Recent work has shown that deeper character-based neural machine translation\n(NMT) models can outperform subword-based models. However, it is still unclear\nwhat makes deeper character-based models successful. In this paper, we conduct\nan investigation into pure character-based models in the case of translating\nFinnish into English, including exploring the ability to learn word senses and\nmorphological inflections and the attention mechanism. We demonstrate that\nword-level information is distributed over the entire character sequence rather\nthan over a single character, and characters at different positions play\ndifferent roles in learning linguistic knowledge. In addition, character-based\nmodels need more layers to encode word senses which explains why only deeper\nmodels outperform subword-based models. The attention distribution pattern\nshows that separators attract a lot of attention and we explore a sparse\nword-level attention to enforce character hidden states to capture the full\nword-level information. Experimental results show that the word-level attention\nwith a single head results in 1.2 BLEU points drop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03492v2",
            "date": "2020-11-06 17:57:18+00:00",
            "title": "Practical and Ethical Considerations in the Effective use of Emotion and Sentiment Lexicons",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "Lexicons of word-emotion associations are widely used in research and\nreal-world applications. As part of my research, I have created several such\nlexicons (e.g., the NRC Emotion Lexicon). This paper outlines some practical\nand ethical considerations involved in the effective use of these lexical\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03588v1",
            "date": "2020-11-06 20:33:12+00:00",
            "title": "Hostility Detection Dataset in Hindi",
            "authors": [
                "Mohit Bhardwaj",
                "Md Shad Akhtar",
                "Asif Ekbal",
                "Amitava Das",
                "Tanmoy Chakraborty"
            ],
            "abstract": "In this paper, we present a novel hostility detection dataset in Hindi\nlanguage. We collect and manually annotate ~8200 online posts. The annotated\ndataset covers four hostility dimensions: fake news, hate speech, offensive,\nand defamation posts, along with a non-hostile label. The hostile posts are\nalso considered for multi-label tags due to a significant overlap among the\nhostile classes. We release this dataset as part of the CONSTRAINT-2021 shared\ntask on hostile post detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03713v1",
            "date": "2020-11-07 06:56:54+00:00",
            "title": "Naturalization of Text by the Insertion of Pauses and Filler Words",
            "authors": [
                "Richa Sharma",
                "Parth Vipul Shah",
                "Ashwini M. Joshi"
            ],
            "abstract": "In this article, we introduce a set of methods to naturalize text based on\nnatural human speech. Voice-based interactions provide a natural way of\ninterfacing with electronic systems and are seeing a widespread adaptation of\nlate. These computerized voices can be naturalized to some degree by inserting\npauses and filler words at appropriate positions. The first proposed text\ntransformation method uses the frequency of bigrams in the training data to\nmake appropriate insertions in the input sentence. It uses a probability\ndistribution to choose the insertions from a set of all possible insertions.\nThis method is fast and can be included before a Text-To-Speech module. The\nsecond method uses a Recurrent Neural Network to predict the next word to be\ninserted. It confirms the insertions given by the bigram method. Additionally,\nthe degree of naturalization can be controlled in both these methods. On the\nconduction of a blind survey, we conclude that the output of these text\ntransformation methods is comparable to natural speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03770v1",
            "date": "2020-11-07 12:58:37+00:00",
            "title": "Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads",
            "authors": [
                "Zhengyan Zhang",
                "Fanchao Qi",
                "Zhiyuan Liu",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Deep pre-trained Transformer models have achieved state-of-the-art results\nover a variety of natural language processing (NLP) tasks. By learning rich\nlanguage knowledge with millions of parameters, these models are usually\noverparameterized and significantly increase the computational overhead in\napplications. It is intuitive to address this issue by model compression. In\nthis work, we propose a method, called Single-Shot Meta-Pruning, to compress\ndeep pre-trained Transformers before fine-tuning. Specifically, we focus on\npruning unnecessary attention heads adaptively for different downstream tasks.\nTo measure the informativeness of attention heads, we train our Single-Shot\nMeta-Pruner (SMP) with a meta-learning paradigm aiming to maintain the\ndistribution of text representations after pruning. Compared with existing\ncompression methods for pre-trained models, our method can reduce the overhead\nof both fine-tuning and inference. Experimental results show that our pruner\ncan selectively prune 50% of attention heads with little impact on the\nperformance on downstream tasks and even provide better text representations.\nThe source code will be released in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03803v1",
            "date": "2020-11-07 16:31:45+00:00",
            "title": "Rethinking the Value of Transformer Components",
            "authors": [
                "Wenxuan Wang",
                "Zhaopeng Tu"
            ],
            "abstract": "Transformer becomes the state-of-the-art translation model, while it is not\nwell studied how each intermediate component contributes to the model\nperformance, which poses significant challenges for designing optimal\narchitectures. In this work, we bridge this gap by evaluating the impact of\nindividual component (sub-layer) in trained Transformer models from different\nperspectives. Experimental results across language pairs, training strategies,\nand model capacities show that certain components are consistently more\nimportant than the others. We also report a number of interesting findings that\nmight help humans better analyze, understand and improve Transformer models.\nBased on these observations, we further propose a new training strategy that\ncan improves translation performance by distinguishing the unimportant\ncomponents in training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03877v1",
            "date": "2020-11-08 00:38:08+00:00",
            "title": "Best Practices for Data-Efficient Modeling in NLG:How to Train Production-Ready Neural Models with Less Data",
            "authors": [
                "Ankit Arun",
                "Soumya Batra",
                "Vikas Bhardwaj",
                "Ashwini Challa",
                "Pinar Donmez",
                "Peyman Heidari",
                "Hakan Inan",
                "Shashank Jain",
                "Anuj Kumar",
                "Shawn Mei",
                "Karthik Mohan",
                "Michael White"
            ],
            "abstract": "Natural language generation (NLG) is a critical component in conversational\nsystems, owing to its role of formulating a correct and natural text response.\nTraditionally, NLG components have been deployed using template-based\nsolutions. Although neural network solutions recently developed in the research\ncommunity have been shown to provide several benefits, deployment of such\nmodel-based solutions has been challenging due to high latency, correctness\nissues, and high data needs. In this paper, we present approaches that have\nhelped us deploy data-efficient neural solutions for NLG in conversational\nsystems to production. We describe a family of sampling and modeling techniques\nto attain production quality with light-weight neural network models using only\na fraction of the data that would be necessary otherwise, and show a thorough\ncomparison between each. Our results show that domain complexity dictates the\nappropriate approach to achieve high data efficiency. Finally, we distill the\nlessons from our experimental findings into a list of best practices for\nproduction-level NLG model development, and present them in a brief runbook.\nImportantly, the end products of all of the techniques are small\nsequence-to-sequence models (2Mb) that we can reliably deploy in production.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03888v1",
            "date": "2020-11-08 02:05:25+00:00",
            "title": "Denoising Relation Extraction from Document-level Distant Supervision",
            "authors": [
                "Chaojun Xiao",
                "Yuan Yao",
                "Ruobing Xie",
                "Xu Han",
                "Zhiyuan Liu",
                "Maosong Sun",
                "Fen Lin",
                "Leyu Lin"
            ],
            "abstract": "Distant supervision (DS) has been widely used to generate auto-labeled data\nfor sentence-level relation extraction (RE), which improves RE performance.\nHowever, the existing success of DS cannot be directly transferred to the more\nchallenging document-level relation extraction (DocRE), since the inherent\nnoise in DS may be even multiplied in document level and significantly harm the\nperformance of RE. To address this challenge, we propose a novel pre-trained\nmodel for DocRE, which denoises the document-level DS data via multiple\npre-training tasks. Experimental results on the large-scale DocRE benchmark\nshow that our model can capture useful information from noisy DS data and\nachieve promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03992v1",
            "date": "2020-11-08 14:49:18+00:00",
            "title": "A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems",
            "authors": [
                "Craig Thomson",
                "Ehud Reiter"
            ],
            "abstract": "Most Natural Language Generation systems need to produce accurate texts. We\npropose a methodology for high-quality human evaluation of the accuracy of\ngenerated texts, which is intended to serve as a gold-standard for accuracy\nevaluations of data-to-text systems. We use our methodology to evaluate the\naccuracy of computer generated basketball summaries. We then show how our gold\nstandard evaluation can be used to validate automated metrics",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04124v1",
            "date": "2020-11-09 01:11:55+00:00",
            "title": "What time is it? Temporal Analysis of Novels",
            "authors": [
                "Allen Kim",
                "Charuta Pethe",
                "Steven Skiena"
            ],
            "abstract": "Recognizing the flow of time in a story is a crucial aspect of understanding\nit. Prior work related to time has primarily focused on identifying temporal\nexpressions or relative sequencing of events, but here we propose\ncomputationally annotating each line of a book with wall clock times, even in\nthe absence of explicit time-descriptive phrases. To do so, we construct a data\nset of hourly time phrases from 52,183 fictional books. We then construct a\ntime-of-day classification model that achieves an average error of 2.27 hours.\nFurthermore, we show that by analyzing a book in whole using dynamic\nprogramming of breakpoints, we can roughly partition a book into segments that\neach correspond to a particular time-of-day. This approach improves upon\nbaselines by over two hours. Finally, we apply our model to a corpus of\nliterature categorized by different periods in history, to show interesting\ntrends of hourly activity throughout the past. Among several observations we\nfind that the fraction of events taking place past 10 P.M jumps past 1880 -\ncoincident with the advent of the electric light bulb and city lights.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04132v2",
            "date": "2020-11-09 01:31:05+00:00",
            "title": "Automatic Summarization of Open-Domain Podcast Episodes",
            "authors": [
                "Kaiqiang Song",
                "Chen Li",
                "Xiaoyang Wang",
                "Dong Yu",
                "Fei Liu"
            ],
            "abstract": "We present implementation details of our abstractive summarizers that achieve\ncompetitive results on the Podcast Summarization task of TREC 2020. A concise\ntextual summary that captures important information is crucial for users to\ndecide whether to listen to the podcast. Prior work focuses primarily on\nlearning contextualized representations. Instead, we investigate several\nless-studied aspects of neural abstractive summarization, including (i) the\nimportance of selecting important segments from transcripts to serve as input\nto the summarizer; (ii) striking a balance between the amount and quality of\ntraining instances; (iii) the appropriate summary length and start/end points.\nWe highlight the design considerations behind our system and offer key insights\ninto the strengths and weaknesses of neural abstractive systems. Our results\nsuggest that identifying important segments from transcripts to use as input to\nan abstractive summarizer is advantageous for summarizing long documents. Our\nbest system achieves a quality rating of 1.559 judged by NIST evaluators---an\nabsolute increase of 0.268 (+21%) over the creator descriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04134v1",
            "date": "2020-11-09 01:39:52+00:00",
            "title": "CxGBERT: BERT meets Construction Grammar",
            "authors": [
                "Harish Tayyar Madabushi",
                "Laurence Romain",
                "Dagmar Divjak",
                "Petar Milin"
            ],
            "abstract": "While lexico-semantic elements no doubt capture a large amount of linguistic\ninformation, it has been argued that they do not capture all information\ncontained in text. This assumption is central to constructionist approaches to\nlanguage which argue that language consists of constructions, learned pairings\nof a form and a function or meaning that are either frequent or have a meaning\nthat cannot be predicted from its component parts. BERT's training objectives\ngive it access to a tremendous amount of lexico-semantic information, and while\nBERTology has shown that BERT captures certain important linguistic dimensions,\nthere have been no studies exploring the extent to which BERT might have access\nto constructional information. In this work we design several probes and\nconduct extensive experiments to answer this question. Our results allow us to\nconclude that BERT does indeed have access to a significant amount of\ninformation, much of which linguists typically call constructional information.\nThe impact of this observation is potentially far-reaching as it provides\ninsights into what deep learning methods learn from text, while also showing\nthat information contained in constructions is redundantly encoded in\nlexico-semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04151v1",
            "date": "2020-11-09 02:14:33+00:00",
            "title": "\"What Do You Mean by That?\" A Parser-Independent Interactive Approach for Enhancing Text-to-SQL",
            "authors": [
                "Yuntao Li",
                "Bei Chen",
                "Qian Liu",
                "Yan Gao",
                "Jian-Guang Lou",
                "Yan Zhang",
                "Dongmei Zhang"
            ],
            "abstract": "In Natural Language Interfaces to Databases systems, the text-to-SQL\ntechnique allows users to query databases by using natural language questions.\nThough significant progress in this area has been made recently, most parsers\nmay fall short when they are deployed in real systems. One main reason stems\nfrom the difficulty of fully understanding the users' natural language\nquestions. In this paper, we include human in the loop and present a novel\nparser-independent interactive approach (PIIA) that interacts with users using\nmulti-choice questions and can easily work with arbitrary parsers. Experiments\nwere conducted on two cross-domain datasets, the WikiSQL and the more complex\nSpider, with five state-of-the-art parsers. These demonstrated that PIIA is\ncapable of enhancing the text-to-SQL performance with limited interaction turns\nby using both simulation and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04163v1",
            "date": "2020-11-09 02:56:54+00:00",
            "title": "Chapter Captor: Text Segmentation in Novels",
            "authors": [
                "Charuta Pethe",
                "Allen Kim",
                "Steven Skiena"
            ],
            "abstract": "Books are typically segmented into chapters and sections, representing\ncoherent subnarratives and topics. We investigate the task of predicting\nchapter boundaries, as a proxy for the general task of segmenting long texts.\nWe build a Project Gutenberg chapter segmentation data set of 9,126 English\nnovels, using a hybrid approach combining neural inference and rule matching to\nrecognize chapter title headers in books, achieving an F1-score of 0.77 on this\ntask. Using this annotated data as ground truth after removing structural cues,\nwe present cut-based and neural methods for chapter segmentation, achieving an\nF1-score of 0.453 on the challenging task of exact break prediction over\nbook-length documents. Finally, we reveal interesting historical trends in the\nchapter structure of novels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04196v2",
            "date": "2020-11-09 05:22:57+00:00",
            "title": "Efficient End-to-End Speech Recognition Using Performers in Conformers",
            "authors": [
                "Peidong Wang",
                "DeLiang Wang"
            ],
            "abstract": "On-device end-to-end speech recognition poses a high requirement on model\nefficiency. Most prior works improve the efficiency by reducing model sizes. We\npropose to reduce the complexity of model architectures in addition to model\nsizes. More specifically, we reduce the floating-point operations in conformer\nby replacing the transformer module with a performer. The proposed\nattention-based efficient end-to-end speech recognition model yields\ncompetitive performance on the LibriSpeech corpus with 10 millions of\nparameters and linear computation complexity. The proposed model also\noutperforms previous lightweight end-to-end models by about 20% relatively in\nword error rate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04241v1",
            "date": "2020-11-09 08:17:17+00:00",
            "title": "Pointing to Subwords for Generating Function Names in Source Code",
            "authors": [
                "Shogo Fujita",
                "Hidetaka Kamigaito",
                "Hiroya Takamura",
                "Manabu Okumura"
            ],
            "abstract": "We tackle the task of automatically generating a function name from source\ncode. Existing generators face difficulties in generating low-frequency or\nout-of-vocabulary subwords. In this paper, we propose two strategies for\ncopying low-frequency or out-of-vocabulary subwords in inputs. Our best\nperforming model showed an improvement over the conventional method in terms of\nour modified F1 and accuracy on the Java-small and Java-large datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04266v1",
            "date": "2020-11-09 09:30:37+00:00",
            "title": "BERT-JAM: Boosting BERT-Enhanced Neural Machine Translation with Joint Attention",
            "authors": [
                "Zhebin Zhang",
                "Sai Wu",
                "Dawei Jiang",
                "Gang Chen"
            ],
            "abstract": "BERT-enhanced neural machine translation (NMT) aims at leveraging\nBERT-encoded representations for translation tasks. A recently proposed\napproach uses attention mechanisms to fuse Transformer's encoder and decoder\nlayers with BERT's last-layer representation and shows enhanced performance.\nHowever, their method doesn't allow for the flexible distribution of attention\nbetween the BERT representation and the encoder/decoder representation. In this\nwork, we propose a novel BERT-enhanced NMT model called BERT-JAM which improves\nupon existing models from two aspects: 1) BERT-JAM uses joint-attention modules\nto allow the encoder/decoder layers to dynamically allocate attention between\ndifferent representations, and 2) BERT-JAM allows the encoder/decoder layers to\nmake use of BERT's intermediate representations by composing them using a gated\nlinear unit (GLU). We train BERT-JAM with a novel three-phase optimization\nstrategy that progressively unfreezes different components of BERT-JAM. Our\nexperiments show that BERT-JAM achieves SOTA BLEU scores on multiple\ntranslation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04308v1",
            "date": "2020-11-09 10:24:12+00:00",
            "title": "Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT",
            "authors": [
                "Rik van Noord",
                "Antonio Toral",
                "Johan Bos"
            ],
            "abstract": "We combine character-level and contextual language model representations to\nimprove performance on Discourse Representation Structure parsing. Character\nrepresentations can easily be added in a sequence-to-sequence model in either\none encoder or as a fully separate encoder, with improvements that are robust\nto different language models, languages and data sets. For English, these\nimprovements are larger than adding individual sources of linguistic\ninformation or adding non-contextual embeddings. A new method of analysis based\non semantic tags demonstrates that the character-level representations improve\nperformance across a subset of selected semantic phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04372v1",
            "date": "2020-11-09 12:13:55+00:00",
            "title": "Low-Resource Adaptation of Neural NLP Models",
            "authors": [
                "Farhad Nooralahzadeh"
            ],
            "abstract": "Real-world applications of natural language processing (NLP) are challenging.\nNLP models rely heavily on supervised machine learning and require large\namounts of annotated data. These resources are often based on language data\navailable in large quantities, such as English newswire. However, in real-world\napplications of NLP, the textual resources vary across several dimensions, such\nas language, dialect, topic, and genre. It is challenging to find annotated\ndata of sufficient amount and quality. The objective of this thesis is to\ninvestigate methods for dealing with such low-resource scenarios in information\nextraction and natural language understanding. To this end, we study distant\nsupervision and sequential transfer learning in various low-resource settings.\nWe develop and adapt neural NLP models to explore a number of research\nquestions concerning NLP tasks with minimal or no training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04393v3",
            "date": "2020-11-09 12:49:39+00:00",
            "title": "Positional Artefacts Propagate Through Masked Language Model Embeddings",
            "authors": [
                "Ziyang Luo",
                "Artur Kulmizev",
                "Xiaoxi Mao"
            ],
            "abstract": "In this work, we demonstrate that the contextualized word vectors derived\nfrom pretrained masked language model-based encoders share a common, perhaps\nundesirable pattern across layers. Namely, we find cases of persistent outlier\nneurons within BERT and RoBERTa's hidden state vectors that consistently bear\nthe smallest or largest values in said vectors. In an attempt to investigate\nthe source of this information, we introduce a neuron-level analysis method,\nwhich reveals that the outliers are closely related to information captured by\npositional embeddings. We also pre-train the RoBERTa-base models from scratch\nand find that the outliers disappear without using positional embeddings. These\noutliers, we find, are the major cause of anisotropy of encoders' raw vector\nspaces, and clipping them leads to increased similarity across vectors. We\ndemonstrate this in practice by showing that clipped vectors can more\naccurately distinguish word senses, as well as lead to better sentence\nembeddings when mean pooling. In three supervised tasks, we find that clipping\ndoes not affect the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04499v1",
            "date": "2020-11-09 15:28:53+00:00",
            "title": "Synonym Knowledge Enhanced Reader for Chinese Idiom Reading Comprehension",
            "authors": [
                "Siyu Long",
                "Ran Wang",
                "Kun Tao",
                "Jiali Zeng",
                "Xin-Yu Dai"
            ],
            "abstract": "Machine reading comprehension (MRC) is the task that asks a machine to answer\nquestions based on a given context. For Chinese MRC, due to the non-literal and\nnon-compositional semantic characteristics, Chinese idioms pose unique\nchallenges for machines to understand. Previous studies tend to treat idioms\nseparately without fully exploiting the relationship among them. In this paper,\nwe first define the concept of literal meaning coverage to measure the\nconsistency between semantics and literal meanings for Chinese idioms. With the\ndefinition, we prove that the literal meanings of many idioms are far from\ntheir semantics, and we also verify that the synonymic relationship can\nmitigate this inconsistency, which would be beneficial for idiom comprehension.\nFurthermore, to fully utilize the synonymic relationship, we propose the\nsynonym knowledge enhanced reader. Specifically, for each idiom, we first\nconstruct a synonym graph according to the annotations from a high-quality\nsynonym dictionary or the cosine similarity between the pre-trained idiom\nembeddings and then incorporate the graph attention network and gate mechanism\nto encode the graph. Experimental results on ChID, a large-scale Chinese idiom\nreading comprehension dataset, show that our model achieves state-of-the-art\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04507v1",
            "date": "2020-11-09 15:37:43+00:00",
            "title": "VisBERT: Hidden-State Visualizations for Transformers",
            "authors": [
                "Betty van Aken",
                "Benjamin Winter",
                "Alexander L\u00f6ser",
                "Felix A. Gers"
            ],
            "abstract": "Explainability and interpretability are two important concepts, the absence\nof which can and should impede the application of well-performing neural\nnetworks to real-world problems. At the same time, they are difficult to\nincorporate into the large, black-box models that achieve state-of-the-art\nresults in a multitude of NLP tasks. Bidirectional Encoder Representations from\nTransformers (BERT) is one such black-box model. It has become a staple\narchitecture to solve many different NLP tasks and has inspired a number of\nrelated Transformer models. Understanding how these models draw conclusions is\ncrucial for both their improvement and application. We contribute to this\nchallenge by presenting VisBERT, a tool for visualizing the contextual token\nrepresentations within BERT for the task of (multi-hop) Question Answering.\nInstead of analyzing attention weights, we focus on the hidden states resulting\nfrom each encoder block within the BERT model. This way we can observe how the\nsemantic representations are transformed throughout the layers of the model.\nVisBERT enables users to get insights about the model's internal state and to\nexplore its inference steps or potential shortcomings. The tool allows us to\nidentify distinct phases in BERT's transformations that are similar to a\ntraditional NLP pipeline and offer insights during failed predictions.",
            "journal ref": "Companion Proceedings of the Web Conference 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04637v2",
            "date": "2020-11-09 18:49:41+00:00",
            "title": "Action State Update Approach to Dialogue Management",
            "authors": [
                "Svetlana Stoyanchev",
                "Simon Keizer",
                "Rama Doddipatla"
            ],
            "abstract": "Utterance interpretation is one of the main functions of a dialogue manager,\nwhich is the key component of a dialogue system. We propose the action state\nupdate approach (ASU) for utterance interpretation, featuring a statistically\ntrained binary classifier used to detect dialogue state update actions in the\ntext of a user utterance. Our goal is to interpret referring expressions in\nuser input without a domain-specific natural language understanding component.\nFor training the model, we use active learning to automatically select\nsimulated training examples. With both user-simulated and interactive human\nevaluations, we show that the ASU approach successfully interprets user\nutterances in a dialogue system, including those with referring expressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04732v1",
            "date": "2020-11-09 20:16:57+00:00",
            "title": "CLAR: A Cross-Lingual Argument Regularizer for Semantic Role Labeling",
            "authors": [
                "Ishan Jindal",
                "Yunyao Li",
                "Siddhartha Brahma",
                "Huaiyu Zhu"
            ],
            "abstract": "Semantic role labeling (SRL) identifies predicate-argument structure(s) in a\ngiven sentence. Although different languages have different argument\nannotations, polyglot training, the idea of training one model on multiple\nlanguages, has previously been shown to outperform monolingual baselines,\nespecially for low resource languages. In fact, even a simple combination of\ndata has been shown to be effective with polyglot training by representing the\ndistant vocabularies in a shared representation space. Meanwhile, despite the\ndissimilarity in argument annotations between languages, certain argument\nlabels do share common semantic meaning across languages (e.g. adjuncts have\nmore or less similar semantic meaning across languages). To leverage such\nsimilarity in annotation space across languages, we propose a method called\nCross-Lingual Argument Regularizer (CLAR). CLAR identifies such linguistic\nannotation similarity across languages and exploits this information to map the\ntarget language arguments using a transformation of the space on which source\nlanguage arguments lie. By doing so, our experimental results show that CLAR\nconsistently improves SRL performance on multiple languages over monolingual\nand polyglot baselines for low resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04784v3",
            "date": "2020-11-09 21:33:53+00:00",
            "title": "EstBERT: A Pretrained Language-Specific BERT for Estonian",
            "authors": [
                "Hasan Tanvir",
                "Claudia Kittask",
                "Sandra Eiche",
                "Kairit Sirts"
            ],
            "abstract": "This paper presents EstBERT, a large pretrained transformer-based\nlanguage-specific BERT model for Estonian. Recent work has evaluated\nmultilingual BERT models on Estonian tasks and found them to outperform the\nbaselines. Still, based on existing studies on other languages, a\nlanguage-specific BERT model is expected to improve over the multilingual ones.\nWe first describe the EstBERT pretraining process and then present the results\nof the models based on finetuned EstBERT for multiple NLP tasks, including POS\nand morphological tagging, named entity recognition and text classification.\nThe evaluation results show that the models based on EstBERT outperform\nmultilingual BERT models on five tasks out of six, providing further evidence\ntowards a view that training language-specific BERT models are still useful,\neven when multilingual models are available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04845v2",
            "date": "2020-11-10 00:40:20+00:00",
            "title": "Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTS",
            "authors": [
                "Katsuhito Sudoh",
                "Takatomo Kano",
                "Sashi Novitasari",
                "Tomoya Yanagita",
                "Sakriani Sakti",
                "Satoshi Nakamura"
            ],
            "abstract": "This paper presents a newly developed, simultaneous neural speech-to-speech\ntranslation system and its evaluation. The system consists of three\nfully-incremental neural processing modules for automatic speech recognition\n(ASR), machine translation (MT), and text-to-speech synthesis (TTS). We\ninvestigated its overall latency in the system's Ear-Voice Span and speaking\nlatency along with module-level performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04864v1",
            "date": "2020-11-10 02:31:31+00:00",
            "title": "Natural Language Inference in Context -- Investigating Contextual Reasoning over Long Texts",
            "authors": [
                "Hanmeng Liu",
                "Leyang Cui",
                "Jian Liu",
                "Yue Zhang"
            ],
            "abstract": "Natural language inference (NLI) is a fundamental NLP task, investigating the\nentailment relationship between two texts. Popular NLI datasets present the\ntask at sentence-level. While adequate for testing semantic representations,\nthey fall short for testing contextual reasoning over long texts, which is a\nnatural part of the human inference process. We introduce ConTRoL, a new\ndataset for ConTextual Reasoning over Long texts. Consisting of 8,325\nexpert-designed \"context-hypothesis\" pairs with gold labels, ConTRoL is a\npassage-level NLI dataset with a focus on complex contextual reasoning types\nsuch as logical reasoning. It is derived from competitive selection and\nrecruitment test (verbal reasoning test) for police recruitment, with expert\nlevel quality. Compared with previous NLI benchmarks, the materials in ConTRoL\nare much more challenging, involving a range of reasoning types. Empirical\nresults show that state-of-the-art language models perform by far worse than\neducated humans. Our dataset can also serve as a testing-set for downstream\ntasks like Checking Factual Correctness of Summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04946v1",
            "date": "2020-11-10 07:16:18+00:00",
            "title": "When Do You Need Billions of Words of Pretraining Data?",
            "authors": [
                "Yian Zhang",
                "Alex Warstadt",
                "Haau-Sing Li",
                "Samuel R. Bowman"
            ],
            "abstract": "NLP is currently dominated by general-purpose pretrained language models like\nRoBERTa, which achieve strong performance on NLU tasks through pretraining on\nbillions of words. But what exact knowledge or skills do Transformer LMs learn\nfrom large-scale pretraining that they cannot learn from less data? We adopt\nfour probing methods---classifier probing, information-theoretic probing,\nunsupervised relative acceptability judgment, and fine-tuning on NLU\ntasks---and draw learning curves that track the growth of these different\nmeasures of linguistic ability with respect to pretraining data volume using\nthe MiniBERTas, a group of RoBERTa models pretrained on 1M, 10M, 100M and 1B\nwords. We find that LMs require only about 10M or 100M words to learn\nrepresentations that reliably encode most syntactic and semantic features we\ntest. A much larger quantity of data is needed in order to acquire enough\ncommonsense knowledge and other skills required to master typical downstream\nNLU tasks. The results suggest that, while the ability to encode linguistic\nfeatures is almost certainly necessary for language understanding, it is likely\nthat other forms of knowledge are the major drivers of recent improvements in\nlanguage understanding among large pretrained models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05103v1",
            "date": "2020-11-10 14:00:42+00:00",
            "title": "Does Social Support Expressed in Post Titles Elicit Comments in Online Substance Use Recovery Forums?",
            "authors": [
                "Anietie Andy",
                "Sharath Guntuku"
            ],
            "abstract": "Individuals recovering from substance use often seek social support\n(emotional and informational) on online recovery forums, where they can both\nwrite and comment on posts, expressing their struggles and successes. A common\nchallenge in these forums is that certain posts (some of which may be support\nseeking) receive no comments. In this work, we use data from two Reddit\nsubstance recovery forums:/r/Leaves and/r/OpiatesRecovery, to determine the\nrelationship between the social supports expressed in the titles of posts and\nthe number of comments they receive. We show that the types of social support\nexpressed in post titles that elicit comments vary from one substance use\nrecovery forum to the other.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05152v3",
            "date": "2020-11-10 15:19:01+00:00",
            "title": "Multi-Task Sequence Prediction For Tunisian Arabizi Multi-Level Annotation",
            "authors": [
                "Elisa Gugliotta",
                "Marco Dinarelli",
                "Olivier Kraif"
            ],
            "abstract": "In this paper we propose a multi-task sequence prediction system, based on\nrecurrent neural networks and used to annotate on multiple levels an Arabizi\nTunisian corpus. The annotation performed are text classification,\ntokenization, PoS tagging and encoding of Tunisian Arabizi into CODA* Arabic\northography. The system is learned to predict all the annotation levels in\ncascade, starting from Arabizi input. We evaluate the system on the TIGER\nGerman corpus, suitably converting data to have a multi-task problem, in order\nto show the effectiveness of our neural architecture. We show also how we used\nthe system in order to annotate a Tunisian Arabizi corpus, which has been\nafterwards manually corrected and used to further evaluate sequence models on\nTunisian data. Our system is developed for the Fairseq framework, which allows\nfor a fast and easy use for any other sequence prediction problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05233v2",
            "date": "2020-11-10 16:45:52+00:00",
            "title": "On the State of Social Media Data for Mental Health Research",
            "authors": [
                "Keith Harrigian",
                "Carlos Aguirre",
                "Mark Dredze"
            ],
            "abstract": "Data-driven methods for mental health treatment and surveillance have become\na major focus in computational science research in the last decade. However,\nprogress in the domain, in terms of both medical understanding and system\nperformance, remains bounded by the availability of adequate data. Prior\nsystematic reviews have not necessarily made it possible to measure the degree\nto which data-related challenges have affected research progress. In this\npaper, we offer an analysis specifically on the state of social media data that\nexists for conducting mental health research. We do so by introducing an\nopen-source directory of mental health datasets, annotated using a standardized\nschema to facilitate meta-analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05249v1",
            "date": "2020-11-10 17:17:23+00:00",
            "title": "Towards Preemptive Detection of Depression and Anxiety in Twitter",
            "authors": [
                "David Owen",
                "Jose Camacho Collados",
                "Luis Espinosa-Anke"
            ],
            "abstract": "Depression and anxiety are psychiatric disorders that are observed in many\nareas of everyday life. For example, these disorders manifest themselves\nsomewhat frequently in texts written by nondiagnosed users in social media.\nHowever, detecting users with these conditions is not a straightforward task as\nthey may not explicitly talk about their mental state, and if they do,\ncontextual cues such as immediacy must be taken into account. When available,\nlinguistic flags pointing to probable anxiety or depression could be used by\nmedical experts to write better guidelines and treatments. In this paper, we\ndevelop a dataset designed to foster research in depression and anxiety\ndetection in Twitter, framing the detection task as a binary tweet\nclassification problem. We then apply state-of-the-art classification models to\nthis dataset, providing a competitive set of baselines alongside qualitative\nerror analysis. Our results show that language models perform reasonably well,\nand better than more traditional baselines. Nonetheless, there is clear room\nfor improvement, particularly with unbalanced training sets and in cases where\nseemingly obvious linguistic cues (keywords) are used counter-intuitively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05257v1",
            "date": "2020-11-10 17:25:27+00:00",
            "title": "Medical Knowledge-enriched Textual Entailment Framework",
            "authors": [
                "Shweta Yadav",
                "Vishal Pallagani",
                "Amit Sheth"
            ],
            "abstract": "One of the cardinal tasks in achieving robust medical question answering\nsystems is textual entailment. The existing approaches make use of an ensemble\nof pre-trained language models or data augmentation, often to clock higher\nnumbers on the validation metrics. However, two major shortcomings impede\nhigher success in identifying entailment: (1) understanding the focus/intent of\nthe question and (2) ability to utilize the real-world background knowledge to\ncapture the context beyond the sentence. In this paper, we present a novel\nMedical Knowledge-Enriched Textual Entailment framework that allows the model\nto acquire a semantic and global representation of the input medical text with\nthe help of a relevant domain-specific knowledge graph. We evaluate our\nframework on the benchmark MEDIQA-RQE dataset and manifest that the use of\nknowledge enriched dual-encoding mechanism help in achieving an absolute\nimprovement of 8.27% over SOTA language models. We have made the source code\navailable here.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05284v1",
            "date": "2020-11-10 18:04:26+00:00",
            "title": "Neural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara",
            "authors": [
                "Allahsera Auguste Tapo",
                "Bakary Coulibaly",
                "S\u00e9bastien Diarra",
                "Christopher Homan",
                "Julia Kreutzer",
                "Sarah Luger",
                "Arthur Nagashima",
                "Marcos Zampieri",
                "Michael Leventhal"
            ],
            "abstract": "Low-resource languages present unique challenges to (neural) machine\ntranslation. We discuss the case of Bambara, a Mande language for which\ntraining data is scarce and requires significant amounts of pre-processing.\nMore than the linguistic situation of Bambara itself, the socio-cultural\ncontext within which Bambara speakers live poses challenges for automated\nprocessing of this language. In this paper, we present the first parallel data\nset for machine translation of Bambara into and from English and French and the\nfirst benchmark results on machine translation to and from Bambara. We discuss\nchallenges in working with low-resource languages and propose strategies to\ncope with data scarcity in low-resource machine translation (MT).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05295v1",
            "date": "2020-11-10 18:32:53+00:00",
            "title": "DoLFIn: Distributions over Latent Features for Interpretability",
            "authors": [
                "Phong Le",
                "Willem Zuidema"
            ],
            "abstract": "Interpreting the inner workings of neural models is a key step in ensuring\nthe robustness and trustworthiness of the models, but work on neural network\ninterpretability typically faces a trade-off: either the models are too\nconstrained to be very useful, or the solutions found by the models are too\ncomplex to interpret. We propose a novel strategy for achieving\ninterpretability that -- in our experiments -- avoids this trade-off. Our\napproach builds on the success of using probability as the central quantity,\nsuch as for instance within the attention mechanism. In our architecture,\nDoLFIn (Distributions over Latent Features for Interpretability), we do no\ndetermine beforehand what each feature represents, and features go altogether\ninto an unordered set. Each feature has an associated probability ranging from\n0 to 1, weighing its importance for further processing. We show that, unlike\nattention and saliency map approaches, this set-up makes it straight-forward to\ncompute the probability with which an input component supports the decision the\nneural model makes. To demonstrate the usefulness of the approach, we apply\nDoLFIn to text classification, and show that DoLFIn not only provides\ninterpretable solutions, but even slightly outperforms the classical CNN and\nBiLSTM text classifiers on the SST2 and AG-news datasets.",
            "journal ref": "COLING 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05402v1",
            "date": "2020-11-10 21:21:08+00:00",
            "title": "OCR Post Correction for Endangered Language Texts",
            "authors": [
                "Shruti Rijhwani",
                "Antonios Anastasopoulos",
                "Graham Neubig"
            ],
            "abstract": "There is little to no data available to build natural language processing\nmodels for most endangered languages. However, textual data in these languages\noften exists in formats that are not machine-readable, such as paper books and\nscanned images. In this work, we address the task of extracting text from these\nresources. We create a benchmark dataset of transcriptions for scanned books in\nthree critically endangered languages and present a systematic analysis of how\ngeneral-purpose OCR tools are not robust to the data-scarce setting of\nendangered languages. We develop an OCR post-correction method tailored to ease\ntraining in this data-scarce setting, reducing the recognition error rate by\n34% on average across the three languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05443v1",
            "date": "2020-11-10 22:47:14+00:00",
            "title": "Multilingual AMR-to-Text Generation",
            "authors": [
                "Angela Fan",
                "Claire Gardent"
            ],
            "abstract": "Generating text from structured data is challenging because it requires\nbridging the gap between (i) structure and natural language (NL) and (ii)\nsemantically underspecified input and fully specified NL output. Multilingual\ngeneration brings in an additional challenge: that of generating into languages\nwith varied word order and morphological properties. In this work, we focus on\nAbstract Meaning Representations (AMRs) as structured input, where previous\nresearch has overwhelmingly focused on generating only into English. We\nleverage advances in cross-lingual embeddings, pretraining, and multilingual\nmodels to create multilingual AMR-to-text models that generate in twenty one\ndifferent languages. For eighteen languages, based on automatic metrics, our\nmultilingual models surpass baselines that generate into a single language. We\nanalyse the ability of our multilingual models to accurately capture morphology\nand word order using human evaluation, and find that native speakers judge our\ngenerations to be fluent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05448v1",
            "date": "2020-11-10 23:02:47+00:00",
            "title": "Generating Fact Checking Briefs",
            "authors": [
                "Angela Fan",
                "Aleksandra Piktus",
                "Fabio Petroni",
                "Guillaume Wenzek",
                "Marzieh Saeidi",
                "Andreas Vlachos",
                "Antoine Bordes",
                "Sebastian Riedel"
            ],
            "abstract": "Fact checking at scale is difficult -- while the number of active fact\nchecking websites is growing, it remains too small for the needs of the\ncontemporary media ecosystem. However, despite good intentions, contributions\nfrom volunteers are often error-prone, and thus in practice restricted to claim\ndetection. We investigate how to increase the accuracy and efficiency of fact\nchecking by providing information about the claim before performing the check,\nin the form of natural language briefs. We investigate passage-based briefs,\ncontaining a relevant passage from Wikipedia, entity-centric ones consisting of\nWikipedia pages of mentioned entities, and Question-Answering Briefs, with\nquestions decomposing the claim, and their answers. To produce QABriefs, we\ndevelop QABriefer, a model that generates a set of questions conditioned on the\nclaim, searches the web for evidence, and generates answers. To train its\ncomponents, we introduce QABriefDataset which we collected via crowdsourcing.\nWe show that fact checking with briefs -- in particular QABriefs -- increases\nthe accuracy of crowdworkers by 10% while slightly decreasing the time taken.\nFor volunteer (unpaid) fact checkers, QABriefs slightly increase accuracy and\nreduce the time required by around 20%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05449v1",
            "date": "2020-11-10 23:03:50+00:00",
            "title": "From Unsupervised Machine Translation To Adversarial Text Generation",
            "authors": [
                "Ahmad Rashid",
                "Alan Do-Omri",
                "Md. Akmal Haidar",
                "Qun Liu",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "We present a self-attention based bilingual adversarial text generator\n(B-GAN) which can learn to generate text from the encoder representation of an\nunsupervised neural machine translation system. B-GAN is able to generate a\ndistributed latent space representation which can be paired with an attention\nbased decoder to generate fluent sentences. When trained on an encoder shared\nbetween two languages and paired with the appropriate decoder, it can generate\nsentences in either language. B-GAN is trained using a combination of\nreconstruction loss for auto-encoder, a cross domain loss for translation and a\nGAN based adversarial loss for text generation. We demonstrate that B-GAN,\ntrained on monolingual corpora only using multiple losses, generates more\nfluent sentences compared to monolingual baselines while effectively using half\nthe number of parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05504v1",
            "date": "2020-11-11 01:44:09+00:00",
            "title": "Morphological Disambiguation from Stemming Data",
            "authors": [
                "Antoine Nzeyimana"
            ],
            "abstract": "Morphological analysis and disambiguation is an important task and a crucial\npreprocessing step in natural language processing of morphologically rich\nlanguages. Kinyarwanda, a morphologically rich language, currently lacks tools\nfor automated morphological analysis. While linguistically curated finite state\ntools can be easily developed for morphological analysis, the morphological\nrichness of the language allows many ambiguous analyses to be produced,\nrequiring effective disambiguation. In this paper, we propose learning to\nmorphologically disambiguate Kinyarwanda verbal forms from a new stemming\ndataset collected through crowd-sourcing. Using feature engineering and a\nfeed-forward neural network based classifier, we achieve about 89%\nnon-contextualized disambiguation accuracy. Our experiments reveal that\ninflectional properties of stems and morpheme association rules are the most\ndiscriminative features for disambiguation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05551v1",
            "date": "2020-11-11 05:20:39+00:00",
            "title": "NIT COVID-19 at WNUT-2020 Task 2: Deep Learning Model RoBERTa for Identify Informative COVID-19 English Tweets",
            "authors": [
                "Jagadeesh M S",
                "Alphonse P J A"
            ],
            "abstract": "This paper presents the model submitted by the NIT_COVID-19 team for\nidentified informative COVID-19 English tweets at WNUT-2020 Task2. This shared\ntask addresses the problem of automatically identifying whether an English\ntweet related to informative (novel coronavirus) or not. These informative\ntweets provide information about recovered, confirmed, suspected, and death\ncases as well as the location or travel history of the cases. The proposed\napproach includes pre-processing techniques and pre-trained RoBERTa with\nsuitable hyperparameters for English coronavirus tweet classification. The\nperformance achieved by the proposed model for shared task WNUT 2020 Task2 is\n89.14% in the F1-score metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05688v1",
            "date": "2020-11-11 10:51:07+00:00",
            "title": "Personal-ITY: A Novel YouTube-based Corpus for Personality Prediction in Italian",
            "authors": [
                "Elisa Bassignana",
                "Malvina Nissim",
                "Viviana Patti"
            ],
            "abstract": "We present a novel corpus for personality prediction in Italian, containing a\nlarger number of authors and a different genre compared to previously available\nresources. The corpus is built exploiting Distant Supervision, assigning\nMyers-Briggs Type Indicator (MBTI) labels to YouTube comments, and can lend\nitself to a variety of experiments. We report on preliminary experiments on\nPersonal-ITY, which can serve as a baseline for future work, showing that some\ntypes are easier to predict than others, and discussing the perks of\ncross-dataset prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05706v1",
            "date": "2020-11-11 11:22:05+00:00",
            "title": "Multilingual Irony Detection with Dependency Syntax and Neural Models",
            "authors": [
                "Alessandra Teresa Cignarella",
                "Valerio Basile",
                "Manuela Sanguinetti",
                "Cristina Bosco",
                "Paolo Rosso",
                "Farah Benamara"
            ],
            "abstract": "This paper presents an in-depth investigation of the effectiveness of\ndependency-based syntactic features on the irony detection task in a\nmultilingual perspective (English, Spanish, French and Italian). It focuses on\nthe contribution from syntactic knowledge, exploiting linguistic resources\nwhere syntax is annotated according to the Universal Dependencies scheme. Three\ndistinct experimental settings are provided. In the first, a variety of\nsyntactic dependency-based features combined with classical machine learning\nclassifiers are explored. In the second scenario, two well-known types of word\nembeddings are trained on parsed data and tested against gold standard\ndatasets. In the third setting, dependency-based syntactic features are\ncombined into the Multilingual BERT architecture. The results suggest that\nfine-grained dependency-based syntactic information is informative for the\ndetection of irony.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05744v1",
            "date": "2020-11-11 12:56:21+00:00",
            "title": "IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generation",
            "authors": [
                "Yitao Cai",
                "Xiaojun Wan"
            ],
            "abstract": "Context-dependent text-to-SQL task has drawn much attention in recent years.\nPrevious models on context-dependent text-to-SQL task only concentrate on\nutilizing historical user inputs. In this work, in addition to using encoders\nto capture historical information of user inputs, we propose a database schema\ninteraction graph encoder to utilize historicalal information of database\nschema items. In decoding phase, we introduce a gate mechanism to weigh the\nimportance of different vocabularies and then make the prediction of SQL\ntokens. We evaluate our model on the benchmark SParC and CoSQL datasets, which\nare two large complex context-dependent cross-domain text-to-SQL datasets. Our\nmodel outperforms previous state-of-the-art model by a large margin and\nachieves new state-of-the-art results on the two datasets. The comparison and\nablation results demonstrate the efficacy of our model and the usefulness of\nthe database schema interaction graph encoder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05788v1",
            "date": "2020-11-11 14:05:32+00:00",
            "title": "Assessment of text coherence based on the cohesion estimation",
            "authors": [
                "S. D. Pogorilyy",
                "A. A. Kramov"
            ],
            "abstract": "In this paper, a graph-based coherence estimation method based on the\ncohesion estimation is suggested. Our method uses a graph-based approach to\nprovide a user with an understanding of the evaluation process. Moreover, it\ncan be applied to different languages, therefore, the effectiveness of this\nmethod is examined on the set of English, Chinese, and Arabic texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05911v1",
            "date": "2020-11-11 17:04:55+00:00",
            "title": "Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research",
            "authors": [
                "Lucy Havens",
                "Melissa Terras",
                "Benjamin Bach",
                "Beatrice Alex"
            ],
            "abstract": "We propose a bias-aware methodology to engage with power relations in natural\nlanguage processing (NLP) research. NLP research rarely engages with bias in\nsocial contexts, limiting its ability to mitigate bias. While researchers have\nrecommended actions, technical methods, and documentation practices, no\nmethodology exists to integrate critical reflections on bias with technical NLP\nmethods. In this paper, after an extensive and interdisciplinary literature\nreview, we contribute a bias-aware methodology for NLP research. We also\ncontribute a definition of biased text, a discussion of the implications of\nbiased NLP systems, and a case study demonstrating how we are executing the\nbias-aware methodology in research on archival metadata descriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05932v1",
            "date": "2020-11-11 17:44:57+00:00",
            "title": "Overview of CAPITEL Shared Tasks at IberLEF 2020: Named Entity Recognition and Universal Dependencies Parsing",
            "authors": [
                "Jordi Porta-Zamorano",
                "Luis Espinosa-Anke"
            ],
            "abstract": "We present the results of the CAPITEL-EVAL shared task, held in the context\nof the IberLEF 2020 competition series. CAPITEL-EVAL consisted on two subtasks:\n(1) Named Entity Recognition and Classification and (2) Universal Dependency\nparsing. For both, the source data was a newly annotated corpus, CAPITEL, a\ncollection of Spanish articles in the newswire domain. A total of seven teams\nparticipated in CAPITEL-EVAL, with a total of 13 runs submitted across all\nsubtasks. Data, results and further information about this task can be found at\nsites.google.com/view/capitel2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06056v1",
            "date": "2020-11-11 20:21:21+00:00",
            "title": "Text Augmentation for Language Models in High Error Recognition Scenario",
            "authors": [
                "Karel Bene\u0161",
                "Luk\u00e1\u0161 Burget"
            ],
            "abstract": "We examine the effect of data augmentation for training of language models\nfor speech recognition. We compare augmentation based on global error\nstatistics with one based on per-word unigram statistics of ASR errors and\nobserve that it is better to only pay attention the global substitution,\ndeletion and insertion rates. This simple scheme also performs consistently\nbetter than label smoothing and its sampled variants. Additionally, we\ninvestigate into the behavior of perplexity estimated on augmented data, but\nconclude that it gives no better prediction of the final error rate. Our best\naugmentation scheme increases the absolute WER improvement from second-pass\nrescoring from 1.1 % to 1.9 % absolute on the CHiMe-6 challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06057v1",
            "date": "2020-11-11 20:23:09+00:00",
            "title": "Exploring the Value of Personalized Word Embeddings",
            "authors": [
                "Charles Welch",
                "Jonathan K. Kummerfeld",
                "Ver\u00f3nica P\u00e9rez-Rosas",
                "Rada Mihalcea"
            ],
            "abstract": "In this paper, we introduce personalized word embeddings, and examine their\nvalue for language modeling. We compare the performance of our proposed\nprediction model when using personalized versus generic word representations,\nand study how these representations can be leveraged for improved performance.\nWe provide insight into what types of words can be more accurately predicted\nwhen building personalized models. Our results show that a subset of words\nbelonging to specific psycholinguistic categories tend to vary more in their\nrepresentations across users and that combining generic and personalized word\nembeddings yields the best performance, with a 4.7% relative reduction in\nperplexity. Additionally, we show that a language model using personalized word\nembeddings can be effectively used for authorship attribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06132v1",
            "date": "2020-11-12 00:32:51+00:00",
            "title": "Incorporating a Local Translation Mechanism into Non-autoregressive Translation",
            "authors": [
                "Xiang Kong",
                "Zhisong Zhang",
                "Eduard Hovy"
            ],
            "abstract": "In this work, we introduce a novel local autoregressive translation (LAT)\nmechanism into non-autoregressive translation (NAT) models so as to capture\nlocal dependencies among tar-get outputs. Specifically, for each target\ndecoding position, instead of only one token, we predict a short sequence of\ntokens in an autoregressive way. We further design an efficient merging\nalgorithm to align and merge the out-put pieces into one final output sequence.\nWe integrate LAT into the conditional masked language model (CMLM;\nGhazvininejad et al.,2019) and similarly adopt iterative decoding. Empirical\nresults on five translation tasks show that compared with CMLM, our method\nachieves comparable or better performance with fewer decoding iterations,\nbringing a 2.5xspeedup. Further analysis indicates that our method reduces\nrepeated translations and performs better at longer sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06149v1",
            "date": "2020-11-12 01:17:49+00:00",
            "title": "Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework",
            "authors": [
                "Shweta Yadav",
                "Jainish Chauhan",
                "Joy Prakash Sain",
                "Krishnaprasad Thirunarayan",
                "Amit Sheth",
                "Jeremiah Schumm"
            ],
            "abstract": "Existing studies on using social media for deriving mental health status of\nusers focus on the depression detection task. However, for case management and\nreferral to psychiatrists, healthcare workers require practical and scalable\ndepressive disorder screening and triage system. This study aims to design and\nevaluate a decision support system (DSS) to reliably determine the depressive\ntriage level by capturing fine-grained depressive symptoms expressed in user\ntweets through the emulation of Patient Health Questionnaire-9 (PHQ-9) that is\nroutinely used in clinical practice. The reliable detection of depressive\nsymptoms from tweets is challenging because the 280-character limit on tweets\nincentivizes the use of creative artifacts in the utterances and figurative\nusage contributes to effective expression. We propose a novel BERT based robust\nmulti-task learning framework to accurately identify the depressive symptoms\nusing the auxiliary task of figurative usage detection. Specifically, our\nproposed novel task sharing mechanism, co-task aware attention, enables\nautomatic selection of optimal information across the BERT layers and tasks by\nsoft-sharing of parameters. Our results show that modeling figurative usage can\ndemonstrably improve the model's robustness and reliability for distinguishing\nthe depression symptoms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06174v6",
            "date": "2020-11-12 03:00:20+00:00",
            "title": "Theoretical Knowledge Graph Reasoning via Ending Anchored Rules",
            "authors": [
                "Canlin Zhang",
                "Yannis Katsis",
                "Yoshiki Vazquez-Baeza",
                "Andrew Bartko",
                "Ho-Cheol Kim",
                "Chun-Nan Hsu"
            ],
            "abstract": "Discovering precise and specific rules from knowledge graphs is regarded as\nan essential challenge, which can improve the performances of many downstream\ntasks and even provide new ways to approach some Natural Language Processing\nresearch topics. In this paper, we provide a fundamental theory for knowledge\ngraph reasoning based on the ending anchored rules. Our theory provides precise\nreasons explaining why or why not a triple is correct. Then, we implement our\ntheory by what we call the EARDict model. Results show that our EARDict model\nsignificantly outperforms all the benchmark models on three large datasets of\nknowledge graph completion. Especially, our model achieves a Hits@10 score of\n96.6 percent on WN18RR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06198v1",
            "date": "2020-11-12 04:41:35+00:00",
            "title": "Enabling Interactive Transcription in an Indigenous Community",
            "authors": [
                "\u00c9ric Le Ferrand",
                "Steven Bird",
                "Laurent Besacier"
            ],
            "abstract": "We propose a novel transcription workflow which combines spoken term\ndetection and human-in-the-loop, together with a pilot experiment. This work is\ngrounded in an almost zero-resource scenario where only a few terms have so far\nbeen identified, involving two endangered languages. We show that in the early\nstages of transcription, when the available data is insufficient to train a\nrobust ASR system, it is possible to take advantage of the transcription of a\nsmall number of isolated words in order to bootstrap the transcription of a\nspeech collection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06226v1",
            "date": "2020-11-12 06:41:27+00:00",
            "title": "Cross-lingual and Multilingual Spoken Term Detection for Low-Resource Indian Languages",
            "authors": [
                "Sanket Shah",
                "Satarupa Guha",
                "Simran Khanuja",
                "Sunayana Sitaram"
            ],
            "abstract": "Spoken Term Detection (STD) is the task of searching for words or phrases\nwithin audio, given either text or spoken input as a query. In this work, we\nuse state-of-the-art Hindi, Tamil and Telugu ASR systems cross-lingually for\nlexical Spoken Term Detection in ten low-resource Indian languages. Since no\npublicly available dataset exists for Spoken Term Detection in these languages,\nwe create a new dataset using a publicly available TTS dataset. We report a\nstandard metric for STD, Mean Term Weighted Value (MTWV) and show that ASR\nsystems built in languages that are phonetically similar to the target\nlanguages have higher accuracy, however, it is also possible to get high MTWV\nscores for dissimilar languages by using a relaxed phone matching algorithm. We\npropose a technique to bootstrap the Grapheme-to-Phoneme (g2p) mapping between\nall the languages under consideration using publicly available resources. Gains\nare obtained when we combine the output of multiple ASR systems and when we use\nlanguage-specific Language Models. We show that it is possible to perform STD\ncross-lingually in a zero-shot manner without the need for any\nlanguage-specific speech data. We plan to make the STD dataset available for\nother researchers interested in cross-lingual STD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06457v1",
            "date": "2020-11-12 15:57:23+00:00",
            "title": "World Trade Center responders in their own words: Predicting PTSD symptom trajectories with AI-based language analyses of interviews",
            "authors": [
                "Youngseo Son",
                "Sean A. P. Clouston",
                "Roman Kotov",
                "Johannes C. Eichstaedt",
                "Evelyn J. Bromet",
                "Benjamin J. Luft",
                "H Andrew Schwartz"
            ],
            "abstract": "Background: Oral histories from 9/11 responders to the World Trade Center\n(WTC) attacks provide rich narratives about distress and resilience. Artificial\nIntelligence (AI) models promise to detect psychopathology in natural language,\nbut they have been evaluated primarily in non-clinical settings using social\nmedia. This study sought to test the ability of AI-based language assessments\nto predict PTSD symptom trajectories among responders. Methods: Participants\nwere 124 responders whose health was monitored at the Stony Brook WTC Health\nand Wellness Program who completed oral history interviews about their initial\nWTC experiences. PTSD symptom severity was measured longitudinally using the\nPTSD Checklist (PCL) for up to 7 years post-interview. AI-based indicators were\ncomputed for depression, anxiety, neuroticism, and extraversion along with\ndictionary-based measures of linguistic and interpersonal style. Linear\nregression and multilevel models estimated associations of AI indicators with\nconcurrent and subsequent PTSD symptom severity (significance adjusted by false\ndiscovery rate). Results: Cross-sectionally, greater depressive language\n(beta=0.32; p=0.043) and first-person singular usage (beta=0.31; p=0.044) were\nassociated with increased symptom severity. Longitudinally, anxious language\npredicted future worsening in PCL scores (beta=0.31; p=0.031), whereas\nfirst-person plural usage (beta=-0.37; p=0.007) and longer words usage\n(beta=-0.36; p=0.007) predicted improvement. Conclusions: This is the first\nstudy to demonstrate the value of AI in understanding PTSD in a vulnerable\npopulation. Future studies should extend this application to other trauma\nexposures and to other demographic groups, especially under-represented\nminorities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06467v1",
            "date": "2020-11-12 16:17:59+00:00",
            "title": "Exploiting Cross-Dialectal Gold Syntax for Low-Resource Historical Languages: Towards a Generic Parser for Pre-Modern Slavic",
            "authors": [
                "Nilo Pedrazzini"
            ],
            "abstract": "This paper explores the possibility of improving the performance of\nspecialized parsers for pre-modern Slavic by training them on data from\ndifferent related varieties. Because of their linguistic heterogeneity,\npre-modern Slavic varieties are treated as low-resource historical languages,\nwhereby cross-dialectal treebank data may be exploited to overcome data\nscarcity and attempt the training of a variety-agnostic parser. Previous\nexperiments on early Slavic dependency parsing are discussed, particularly with\nregard to their ability to tackle different orthographic, regional and\nstylistic features. A generic pre-modern Slavic parser and two specialized\nparsers -- one for East Slavic and one for South Slavic -- are trained using\njPTDP (Nguyen & Verspoor 2018), a neural network model for joint part-of-speech\n(POS) tagging and dependency parsing which had shown promising results on a\nnumber of Universal Dependency (UD) treebanks, including Old Church Slavonic\n(OCS). With these experiments, a new state of the art is obtained for both OCS\n(83.79\\% unlabelled attachment score (UAS) and 78.43\\% labelled attachement\nscore (LAS)) and Old East Slavic (OES) (85.7\\% UAS and 80.16\\% LAS).",
            "journal ref": "Proceedings of the Workshop on Computational Humanities Research,\n  18-20 November 2020 (CEUR Workshop Proceedings, Vol. 2723), 237-247",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06486v1",
            "date": "2020-11-12 16:43:10+00:00",
            "title": "Overview of the Ninth Dialog System Technology Challenge: DSTC9",
            "authors": [
                "Chulaka Gunasekara",
                "Seokhwan Kim",
                "Luis Fernando D'Haro",
                "Abhinav Rastogi",
                "Yun-Nung Chen",
                "Mihail Eric",
                "Behnam Hedayatnia",
                "Karthik Gopalakrishnan",
                "Yang Liu",
                "Chao-Wei Huang",
                "Dilek Hakkani-T\u00fcr",
                "Jinchao Li",
                "Qi Zhu",
                "Lingxiao Luo",
                "Lars Liden",
                "Kaili Huang",
                "Shahin Shayandeh",
                "Runze Liang",
                "Baolin Peng",
                "Zheng Zhang",
                "Swadheen Shukla",
                "Minlie Huang",
                "Jianfeng Gao",
                "Shikib Mehri",
                "Yulan Feng",
                "Carla Gordon",
                "Seyed Hossein Alavi",
                "David Traum",
                "Maxine Eskenazi",
                "Ahmad Beirami",
                "Eunjoon",
                "Cho",
                "Paul A. Crook",
                "Ankita De",
                "Alborz Geramifard",
                "Satwik Kottur",
                "Seungwhan Moon",
                "Shivani Poddar",
                "Rajen Subba"
            ],
            "abstract": "This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9).\nThis edition of the DSTC focuses on applying end-to-end dialog technologies for\nfour distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling\nwith unstructured knowledge access, 2. Multi-domain task-oriented dialog, 3.\nInteractive evaluation of dialog, and 4. Situated interactive multi-modal\ndialog. This paper describes the task definition, provided datasets, baselines\nand evaluation set-up for each track. We also summarize the results of the\nsubmitted systems to highlight the overall trends of the state-of-the-art\ntechnologies for the tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06489v1",
            "date": "2020-11-12 16:59:56+00:00",
            "title": "Natural Language Processing to Detect Cognitive Concerns in Electronic Health Records Using Deep Learning",
            "authors": [
                "Zhuoqiao Hong",
                "Colin G. Magdamo",
                "Yi-han Sheu",
                "Prathamesh Mohite",
                "Ayush Noori",
                "Elissa M. Ye",
                "Wendong Ge",
                "Haoqi Sun",
                "Laura Brenner",
                "Gregory Robbins",
                "Shibani Mukerji",
                "Sahar Zafar",
                "Nicole Benson",
                "Lidia Moura",
                "John Hsu",
                "Bradley T. Hyman",
                "Michael B. Westover",
                "Deborah Blacker",
                "Sudeshna Das"
            ],
            "abstract": "Dementia is under-recognized in the community, under-diagnosed by healthcare\nprofessionals, and under-coded in claims data. Information on cognitive\ndysfunction, however, is often found in unstructured clinician notes within\nmedical records but manual review by experts is time consuming and often prone\nto errors. Automated mining of these notes presents a potential opportunity to\nlabel patients with cognitive concerns who could benefit from an evaluation or\nbe referred to specialist care. In order to identify patients with cognitive\nconcerns in electronic medical records, we applied natural language processing\n(NLP) algorithms and compared model performance to a baseline model that used\nstructured diagnosis codes and medication data only. An attention-based deep\nlearning model outperformed the baseline model and other simpler models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06523v1",
            "date": "2020-11-12 17:36:22+00:00",
            "title": "Inference-only sub-character decomposition improves translation of unseen logographic characters",
            "authors": [
                "Danielle Saunders",
                "Weston Feely",
                "Bill Byrne"
            ],
            "abstract": "Neural Machine Translation (NMT) on logographic source languages struggles\nwhen translating `unseen' characters, which never appear in the training data.\nOne possible approach to this problem uses sub-character decomposition for\ntraining and test sentences. However, this approach involves complete\nretraining, and its effectiveness for unseen character translation to\nnon-logographic languages has not been fully explored.\n  We investigate existing ideograph-based sub-character decomposition\napproaches for Chinese-to-English and Japanese-to-English NMT, for both\nhigh-resource and low-resource domains. For each language pair and domain we\nconstruct a test set where all source sentences contain at least one unseen\nlogographic character. We find that complete sub-character decomposition often\nharms unseen character translation, and gives inconsistent results generally.\n  We offer a simple alternative based on decomposition before inference for\nunseen characters only. Our approach allows flexible application, achieving\ntranslation adequacy improvements and requiring no additional models or\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06623v2",
            "date": "2020-11-12 19:08:44+00:00",
            "title": "doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset",
            "authors": [
                "Song Feng",
                "Hui Wan",
                "Chulaka Gunasekara",
                "Siva Sankalp Patel",
                "Sachindra Joshi",
                "Luis A. Lastras"
            ],
            "abstract": "We introduce doc2dial, a new dataset of goal-oriented dialogues that are\ngrounded in the associated documents. Inspired by how the authors compose\ndocuments for guiding end users, we first construct dialogue flows based on the\ncontent elements that corresponds to higher-level relations across text\nsections as well as lower-level relations between discourse units within a\nsection. Then we present these dialogue flows to crowd contributors to create\nconversational utterances. The dataset includes about 4800 annotated\nconversations with an average of 14 turns that are grounded in over 480\ndocuments from four domains. Compared to the prior document-grounded dialogue\ndatasets, this dataset covers a variety of dialogue scenes in\ninformation-seeking conversations. For evaluating the versatility of the\ndataset, we introduce multiple dialogue modeling tasks and present baseline\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06642v1",
            "date": "2020-11-12 20:34:49+00:00",
            "title": "Context-aware Stand-alone Neural Spelling Correction",
            "authors": [
                "Xiangci Li",
                "Hairong Liu",
                "Liang Huang"
            ],
            "abstract": "Existing natural language processing systems are vulnerable to noisy inputs\nresulting from misspellings. On the contrary, humans can easily infer the\ncorresponding correct words from their misspellings and surrounding context.\nInspired by this, we address the stand-alone spelling correction problem, which\nonly corrects the spelling of each token without additional token insertion or\ndeletion, by utilizing both spelling information and global context\nrepresentations. We present a simple yet powerful solution that jointly detects\nand corrects misspellings as a sequence labeling task by fine-turning a\npre-trained language model. Our solution outperforms the previous\nstate-of-the-art result by 12.8% absolute F0.5 score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06737v1",
            "date": "2020-11-13 02:58:55+00:00",
            "title": "Unsupervised Explanation Generation for Machine Reading Comprehension",
            "authors": [
                "Yiming Cui",
                "Ting Liu",
                "Shijin Wang",
                "Guoping Hu"
            ],
            "abstract": "With the blooming of various Pre-trained Language Models (PLMs), Machine\nReading Comprehension (MRC) has embraced significant improvements on various\nbenchmarks and even surpass human performances. However, the existing works\nonly target on the accuracy of the final predictions and neglect the importance\nof the explanations for the prediction, which is a big obstacle when utilizing\nthese models in real-life applications to convince humans. In this paper, we\npropose a self-explainable framework for the machine reading comprehension\ntask. The main idea is that the proposed system tries to use less passage\ninformation and achieve similar results compared to the system that uses the\nwhole passage, while the filtered passage will be used as explanations. We\ncarried out experiments on three multiple-choice MRC datasets, and found that\nthe proposed system could achieve consistent improvements over baseline\nsystems. To evaluate the explainability, we compared our approach with the\ntraditional attention mechanism in human evaluations and found that the\nproposed system has a notable advantage over the latter one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06754v1",
            "date": "2020-11-13 04:31:51+00:00",
            "title": "Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning",
            "authors": [
                "Morteza Rohanian",
                "Julian Hough"
            ],
            "abstract": "We present a multi-task learning framework to enable the training of one\nuniversal incremental dialogue processing model with four tasks of disfluency\ndetection, language modelling, part-of-speech tagging, and utterance\nsegmentation in a simple deep recurrent setting. We show that these tasks\nprovide positive inductive biases to each other with the optimal contribution\nof each one relying on the severity of the noise from the task. Our live\nmulti-task model outperforms similar individual tasks, delivers competitive\nperformance, and is beneficial for future use in conversational agents in\npsychiatric treatment.",
            "journal ref": "The 28th International Conference on Computational Linguistics\n  (COLING 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06844v2",
            "date": "2020-11-13 10:19:13+00:00",
            "title": "Cross-Domain Learning for Classifying Propaganda in Online Contents",
            "authors": [
                "Liqiang Wang",
                "Xiaoyu Shen",
                "Gerard de Melo",
                "Gerhard Weikum"
            ],
            "abstract": "As news and social media exhibit an increasing amount of manipulative\npolarized content, detecting such propaganda has received attention as a new\ntask for content analysis. Prior work has focused on supervised learning with\ntraining data from the same domain. However, as propaganda can be subtle and\nkeeps evolving, manual identification and proper labeling are very demanding.\nAs a consequence, training data is a major bottleneck. In this paper, we tackle\nthis bottleneck and present an approach to leverage cross-domain learning,\nbased on labeled documents and sentences from news and tweets, as well as\npolitical speeches with a clear difference in their degrees of being\npropagandistic. We devise informative features and build various classifiers\nfor propaganda labeling, using cross-domain learning. Our experiments\ndemonstrate the usefulness of this approach, and identify difficulties and\nlimitations in various configurations of sources and targets for the transfer\nstep. We further analyze the influence of various features, and characterize\nsalient indicators of propaganda.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06854v2",
            "date": "2020-11-13 10:53:27+00:00",
            "title": "Interpretable Multi-dataset Evaluation for Named Entity Recognition",
            "authors": [
                "Jinlan Fu",
                "Pengfei Liu",
                "Graham Neubig"
            ],
            "abstract": "With the proliferation of models for natural language processing tasks, it is\neven harder to understand the differences between models and their relative\nmerits. Simply looking at differences between holistic metrics such as\naccuracy, BLEU, or F1 does not tell us why or how particular methods perform\ndifferently and how diverse datasets influence the model design choices. In\nthis paper, we present a general methodology for interpretable evaluation for\nthe named entity recognition (NER) task. The proposed evaluation method enables\nus to interpret the differences in models and datasets, as well as the\ninterplay between them, identifying the strengths and weaknesses of current\nsystems. By making our analysis tool available, we make it easy for future\nresearchers to run similar analyses and drive progress in this area:\nhttps://github.com/neulab/InterpretEval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06858v2",
            "date": "2020-11-13 11:07:08+00:00",
            "title": "RethinkCWS: Is Chinese Word Segmentation a Solved Task?",
            "authors": [
                "Jinlan Fu",
                "Pengfei Liu",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "The performance of the Chinese Word Segmentation (CWS) systems has gradually\nreached a plateau with the rapid development of deep neural networks,\nespecially the successful use of large pre-trained models. In this paper, we\ntake stock of what we have achieved and rethink what's left in the CWS task.\nMethodologically, we propose a fine-grained evaluation for existing CWS\nsystems, which not only allows us to diagnose the strengths and weaknesses of\nexisting models (under the in-dataset setting), but enables us to quantify the\ndiscrepancy between different criterion and alleviate the negative transfer\nproblem when doing multi-criteria learning. Strategically, despite not aiming\nto propose a novel model in this paper, our comprehensive experiments on eight\nmodels and seven datasets, as well as thorough analysis, could search for some\npromising direction for future research. We make all codes publicly available\nand release an interface that can quickly evaluate and diagnose user's models:\nhttps://github.com/neulab/InterpretEval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06993v2",
            "date": "2020-11-13 16:13:59+00:00",
            "title": "FLERT: Document-Level Features for Named Entity Recognition",
            "authors": [
                "Stefan Schweter",
                "Alan Akbik"
            ],
            "abstract": "Current state-of-the-art approaches for named entity recognition (NER)\ntypically consider text at the sentence-level and thus do not model information\nthat crosses sentence boundaries. However, the use of transformer-based models\nfor NER offers natural options for capturing document-level features. In this\npaper, we perform a comparative evaluation of document-level features in the\ntwo standard NER architectures commonly considered in the literature, namely\n\"fine-tuning\" and \"feature-based LSTM-CRF\". We evaluate different\nhyperparameters for document-level features such as context window size and\nenforcing document-locality. We present experiments from which we derive\nrecommendations for how to model document context and present new\nstate-of-the-art scores on several CoNLL-03 benchmark datasets. Our approach is\nintegrated into the Flair framework to facilitate reproduction of our\nexperiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07009v1",
            "date": "2020-11-11 12:45:33+00:00",
            "title": "Matching Theory and Data with Personal-ITY: What a Corpus of Italian YouTube Comments Reveals About Personality",
            "authors": [
                "Elisa Bassignana",
                "Malvina Nissim",
                "Viviana Patti"
            ],
            "abstract": "As a contribution to personality detection in languages other than English,\nwe rely on distant supervision to create Personal-ITY, a novel corpus of\nYouTube comments in Italian, where authors are labelled with personality\ntraits. The traits are derived from one of the mainstream personality theories\nin psychology research, named MBTI. Using personality prediction experiments,\nwe (i) study the task of personality prediction in itself on our corpus as well\nas on TwiSty, a Twitter dataset also annotated with MBTI labels; (ii) carry out\nan extensive, in-depth analysis of the features used by the classifier, and\nview them specifically under the light of the original theory that we used to\ncreate the corpus in the first place. We observe that no single model is best\nat personality detection, and that while some traits are easier than others to\ndetect, and also to match back to theory, for other, less frequent traits the\npicture is much more blurred.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07109v1",
            "date": "2020-11-13 19:58:38+00:00",
            "title": "The Teacher-Student Chatroom Corpus",
            "authors": [
                "Andrew Caines",
                "Helen Yannakoudakis",
                "Helena Edmondson",
                "Helen Allen",
                "Pascual P\u00e9rez-Paredes",
                "Bill Byrne",
                "Paula Buttery"
            ],
            "abstract": "The Teacher-Student Chatroom Corpus (TSCC) is a collection of written\nconversations captured during one-to-one lessons between teachers and learners\nof English. The lessons took place in an online chatroom and therefore involve\nmore interactive, immediate and informal language than might be found in\nasynchronous exchanges such as email correspondence. The fact that the lessons\nwere one-to-one means that the teacher was able to focus exclusively on the\nlinguistic abilities and errors of the student, and to offer personalised\nexercises, scaffolding and correction. The TSCC contains more than one hundred\nlessons between two teachers and eight students, amounting to 13.5K\nconversational turns and 133K words: it is freely available for research use.\nWe describe the corpus design, data collection procedure and annotations added\nto the text. We perform some preliminary descriptive analyses of the data and\nconsider possible uses of the TSCC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07120v1",
            "date": "2020-11-03 00:43:58+00:00",
            "title": "Streaming Attention-Based Models with Augmented Memory for End-to-End Speech Recognition",
            "authors": [
                "Ching-Feng Yeh",
                "Yongqiang Wang",
                "Yangyang Shi",
                "Chunyang Wu",
                "Frank Zhang",
                "Julian Chan",
                "Michael L. Seltzer"
            ],
            "abstract": "Attention-based models have been gaining popularity recently for their strong\nperformance demonstrated in fields such as machine translation and automatic\nspeech recognition. One major challenge of attention-based models is the need\nof access to the full sequence and the quadratically growing computational cost\nconcerning the sequence length. These characteristics pose challenges,\nespecially for low-latency scenarios, where the system is often required to be\nstreaming. In this paper, we build a compact and streaming speech recognition\nsystem on top of the end-to-end neural transducer architecture with\nattention-based modules augmented with convolution. The proposed system equips\nthe end-to-end models with the streaming capability and reduces the large\nfootprint from the streaming attention-based model using augmented memory. On\nthe LibriSpeech dataset, our proposed system achieves word error rates 2.7% on\ntest-clean and 5.8% on test-other, to our best knowledge the lowest among\nstreaming approaches reported so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07127v1",
            "date": "2020-11-13 20:59:21+00:00",
            "title": "IIRC: A Dataset of Incomplete Information Reading Comprehension Questions",
            "authors": [
                "James Ferguson",
                "Matt Gardner",
                "Hannaneh Hajishirzi",
                "Tushar Khot",
                "Pradeep Dasigi"
            ],
            "abstract": "Humans often have to read multiple documents to address their information\nneeds. However, most existing reading comprehension (RC) tasks only focus on\nquestions for which the contexts provide all the information required to answer\nthem, thus not evaluating a system's performance at identifying a potential\nlack of sufficient information and locating sources for that information. To\nfill this gap, we present a dataset, IIRC, with more than 13K questions over\nparagraphs from English Wikipedia that provide only partial information to\nanswer them, with the missing information occurring in one or more linked\ndocuments. The questions were written by crowd workers who did not have access\nto any of the linked documents, leading to questions that have little lexical\noverlap with the contexts where the answers appear. This process also gave many\nquestions without answers, and those that require discrete reasoning,\nincreasing the difficulty of the task. We follow recent modeling work on\nvarious reading comprehension datasets to construct a baseline model for this\ndataset, finding that it achieves 31.1% F1 on this task, while estimated human\nperformance is 88.4%. The dataset, code for the baseline system, and a\nleaderboard can be found at https://allennlp.org/iirc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07164v1",
            "date": "2020-11-13 23:22:28+00:00",
            "title": "Language Models not just for Pre-training: Fast Online Neural Noisy Channel Modeling",
            "authors": [
                "Shruti Bhosale",
                "Kyra Yee",
                "Sergey Edunov",
                "Michael Auli"
            ],
            "abstract": "Pre-training models on vast quantities of unlabeled data has emerged as an\neffective approach to improving accuracy on many NLP tasks. On the other hand,\ntraditional machine translation has a long history of leveraging unlabeled data\nthrough noisy channel modeling. The same idea has recently been shown to\nachieve strong improvements for neural machine translation. Unfortunately,\nna\\\"{i}ve noisy channel modeling with modern sequence to sequence models is up\nto an order of magnitude slower than alternatives. We address this issue by\nintroducing efficient approximations to make inference with the noisy channel\napproach as fast as strong ensembles while increasing accuracy. We also show\nthat the noisy channel approach can outperform strong pre-training results by\nachieving a new state of the art on WMT Romanian-English translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07247v2",
            "date": "2020-11-14 09:50:35+00:00",
            "title": "CL-IMS @ DIACR-Ita: Volente o Nolente: BERT does not outperform SGNS on Semantic Change Detection",
            "authors": [
                "Severin Laicher",
                "Gioia Baldissin",
                "Enrique Casta\u00f1eda",
                "Dominik Schlechtweg",
                "Sabine Schulte im Walde"
            ],
            "abstract": "We present the results of our participation in the DIACR-Ita shared task on\nlexical semantic change detection for Italian. We exploit Average Pairwise\nDistance of token-based BERT embeddings between time points and rank 5 (of 8)\nin the official ranking with an accuracy of $.72$. While we tune parameters on\nthe English data set of SemEval-2020 Task 1 and reach high performance, this\ndoes not translate to the Italian DIACR-Ita data set. Our results show that we\ndo not manage to find robust ways to exploit BERT embeddings in lexical\nsemantic change detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07251v1",
            "date": "2020-11-14 10:06:57+00:00",
            "title": "DebateSum: A large-scale argument mining and summarization dataset",
            "authors": [
                "Allen Roush",
                "Arvind Balaji"
            ],
            "abstract": "Prior work in Argument Mining frequently alludes to its potential\napplications in automatic debating systems. Despite this focus, almost no\ndatasets or models exist which apply natural language processing techniques to\nproblems found within competitive formal debate. To remedy this, we present the\nDebateSum dataset. DebateSum consists of 187,386 unique pieces of evidence with\ncorresponding argument and extractive summaries. DebateSum was made using data\ncompiled by competitors within the National Speech and Debate Association over\na 7-year period. We train several transformer summarization models to benchmark\nsummarization performance on DebateSum. We also introduce a set of fasttext\nword-vectors trained on DebateSum called debate2vec. Finally, we present a\nsearch engine for this dataset which is utilized extensively by members of the\nNational Speech and Debate Association today. The DebateSum search engine is\navailable to the public here: http://www.debate.cards",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07389v1",
            "date": "2020-11-14 21:14:17+00:00",
            "title": "Words are the Window to the Soul: Language-based User Representations for Fake News Detection",
            "authors": [
                "Marco Del Tredici",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "Cognitive and social traits of individuals are reflected in language use.\nMoreover, individuals who are prone to spread fake news online often share\ncommon traits. Building on these ideas, we introduce a model that creates\nrepresentations of individuals on social media based only on the language they\nproduce, and use them to detect fake news. We show that language-based user\nrepresentations are beneficial for this task. We also present an extended\nanalysis of the language of fake news spreaders, showing that its main features\nare mostly domain independent and consistent across two English datasets.\nFinally, we exploit the relation between language use and connections in the\nsocial graph to assess the presence of the Echo Chamber effect in our data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07398v1",
            "date": "2020-11-14 21:55:46+00:00",
            "title": "Lessons from Computational Modelling of Reference Production in Mandarin and English",
            "authors": [
                "Guanyi Chen",
                "Kees van Deemter"
            ],
            "abstract": "Referring expression generation (REG) algorithms offer computational models\nof the production of referring expressions. In earlier work, a corpus of\nreferring expressions (REs) in Mandarin was introduced. In the present paper,\nwe annotate this corpus, evaluate classic REG algorithms on it, and compare the\nresults with earlier results on the evaluation of REG for English referring\nexpressions. Next, we offer an in-depth analysis of the corpus, focusing on\nissues that arise from the grammar of Mandarin. We discuss shortcomings of\nprevious REG evaluations that came to light during our investigation and we\nhighlight some surprising results. Perhaps most strikingly, we found a much\nhigher proportion of under-specified expressions than previous studies had\nsuggested, not just in Mandarin but in English as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07593v1",
            "date": "2020-11-15 17:54:49+00:00",
            "title": "Morphologically Aware Word-Level Translation",
            "authors": [
                "Paula Czarnowska",
                "Sebastian Ruder",
                "Ryan Cotterell",
                "Ann Copestake"
            ],
            "abstract": "We propose a novel morphologically aware probability model for bilingual\nlexicon induction, which jointly models lexeme translation and inflectional\nmorphology in a structured way. Our model exploits the basic linguistic\nintuition that the lexeme is the key lexical unit of meaning, while\ninflectional morphology provides additional syntactic information. This\napproach leads to substantial performance improvements - 19% average\nimprovement in accuracy across 6 language pairs over the state of the art in\nthe supervised setting and 16% in the weakly supervised setting. As another\ncontribution, we highlight issues associated with modern BLI that stem from\nignoring inflectional morphology, and propose three suggestions for improving\nthe task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07670v1",
            "date": "2020-11-16 00:57:14+00:00",
            "title": "IIT_kgp at FinCausal 2020, Shared Task 1: Causality Detection using Sentence Embeddings in Financial Reports",
            "authors": [
                "Arka Mitra",
                "Harshvardhan Srivastava",
                "Yugam Tiwari"
            ],
            "abstract": "The paper describes the work that the team submitted to FinCausal 2020 Shared\nTask. This work is associated with the first sub-task of identifying causality\nin sentences. The various models used in the experiments tried to obtain a\nlatent space representation for each of the sentences. Linear regression was\nperformed on these representations to classify whether the sentence is causal\nor not. The experiments have shown BERT (Large) performed the best, giving a F1\nscore of 0.958, in the task of detecting the causality of sentences in\nfinancial texts and reports. The class imbalance was dealt with a modified loss\nfunction to give a better metric score for the evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07832v1",
            "date": "2020-11-16 10:02:52+00:00",
            "title": "WikiAsp: A Dataset for Multi-domain Aspect-based Summarization",
            "authors": [
                "Hiroaki Hayashi",
                "Prashant Budania",
                "Peng Wang",
                "Chris Ackerson",
                "Raj Neervannan",
                "Graham Neubig"
            ],
            "abstract": "Aspect-based summarization is the task of generating focused summaries based\non specific points of interest. Such summaries aid efficient analysis of text,\nsuch as quickly understanding reviews or opinions from different angles.\nHowever, due to large differences in the type of aspects for different domains\n(e.g., sentiment, product features), the development of previous models has\ntended to be domain-specific. In this paper, we propose WikiAsp, a large-scale\ndataset for multi-domain aspect-based summarization that attempts to spur\nresearch in the direction of open-domain aspect-based summarization.\nSpecifically, we build the dataset using Wikipedia articles from 20 different\ndomains, using the section titles and boundaries of each article as a proxy for\naspect annotation. We propose several straightforward baseline models for this\ntask and conduct experiments on the dataset. Results highlight key challenges\nthat existing summarization models face in this setting, such as proper pronoun\nhandling of quoted sources and consistent explanation of time-sensitive events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07868v1",
            "date": "2020-11-16 11:13:41+00:00",
            "title": "Evaluating Sentence Segmentation and Word Tokenization Systems on Estonian Web Texts",
            "authors": [
                "Kairit Sirts",
                "Kairit Peekman"
            ],
            "abstract": "Texts obtained from web are noisy and do not necessarily follow the\northographic sentence and word boundary rules. Thus, sentence segmentation and\nword tokenization systems that have been developed on well-formed texts might\nnot perform so well on unedited web texts. In this paper, we first describe the\nmanual annotation of sentence boundaries of an Estonian web dataset and then\npresent the evaluation results of three existing sentence segmentation and word\ntokenization systems on this corpus: EstNLTK, Stanza and UDPipe. While EstNLTK\nobtains the highest performance compared to other systems on sentence\nsegmentation on this dataset, the sentence segmentation performance of Stanza\nand UDPipe remains well below the results obtained on the more well-formed\nEstonian UD test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07933v1",
            "date": "2020-11-16 13:31:33+00:00",
            "title": "Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions",
            "authors": [
                "Muhammad N. ElNokrashy",
                "Amr Hendy",
                "Mohamed Abdelghaffar",
                "Mohamed Afify",
                "Ahmed Tawfik",
                "Hany Hassan Awadalla"
            ],
            "abstract": "This paper describes our submission to the WMT20 sentence filtering task. We\ncombine scores from (1) a custom LASER built for each source language, (2) a\nclassifier built to distinguish positive and negative pairs by semantic\nalignment, and (3) the original scores included in the task devkit. For the\nmBART finetuning setup, provided by the organizers, our method shows 7% and 5%\nrelative improvement over baseline, in sacreBLEU score on the test set for\nPashto and Khmer respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07975v1",
            "date": "2020-11-16 14:11:29+00:00",
            "title": "Datasets and Models for Authorship Attribution on Italian Personal Writings",
            "authors": [
                "Gaetana Ruggiero",
                "Albert Gatt",
                "Malvina Nissim"
            ],
            "abstract": "Existing research on Authorship Attribution (AA) focuses on texts for which a\nlot of data is available (e.g novels), mainly in English. We approach AA via\nAuthorship Verification on short Italian texts in two novel datasets, and\nanalyze the interaction between genre, topic, gender and length. Results show\nthat AV is feasible even with little data, but more evidence helps. Gender and\ntopic can be indicative clues, and if not controlled for, they might overtake\nmore specific aspects of personal style.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07990v1",
            "date": "2020-11-16 14:36:42+00:00",
            "title": "The Person Index Challenge: Extraction of Persons from Messy, Short Texts",
            "authors": [
                "Markus Schr\u00f6der",
                "Christian Jilek",
                "Michael Schulze",
                "Andreas Dengel"
            ],
            "abstract": "When persons are mentioned in texts with their first name, last name and/or\nmiddle names, there can be a high variation which of their names are used, how\ntheir names are ordered and if their names are abbreviated. If multiple persons\nare mentioned consecutively in very different ways, especially short texts can\nbe perceived as \"messy\". Once ambiguous names occur, associations to persons\nmay not be inferred correctly. Despite these eventualities, in this paper we\nask how well an unsupervised algorithm can build a person index from short\ntexts. We define a person index as a structured table that distinctly catalogs\nindividuals by their names. First, we give a formal definition of the problem\nand describe a procedure to generate ground truth data for future evaluations.\nTo give a first solution to this challenge, a baseline approach is implemented.\nBy using our proposed evaluation strategy, we test the performance of the\nbaseline and suggest further improvements. For future research the source code\nis publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07997v1",
            "date": "2020-11-16 14:46:08+00:00",
            "title": "Comparative Probing of Lexical Semantics Theories for Cognitive Plausibility and Technological Usefulness",
            "authors": [
                "Ant\u00f3nio Branco",
                "Jo\u00e3o Rodrigues",
                "Ma\u0142gorzata Salawa",
                "Ruben Branco",
                "Chakaveh Saedi"
            ],
            "abstract": "Lexical semantics theories differ in advocating that the meaning of words is\nrepresented as an inference graph, a feature mapping or a vector space, thus\nraising the question: is it the case that one of these approaches is superior\nto the others in representing lexical semantics appropriately? Or in its non\nantagonistic counterpart: could there be a unified account of lexical semantics\nwhere these approaches seamlessly emerge as (partial) renderings of (different)\naspects of a core semantic knowledge base?\n  In this paper, we contribute to these research questions with a number of\nexperiments that systematically probe different lexical semantics theories for\ntheir levels of cognitive plausibility and of technological usefulness.\n  The empirical findings obtained from these experiments advance our insight on\nlexical semantics as the feature-based approach emerges as superior to the\nother ones, and arguably also move us closer to finding answers to the research\nquestions above.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08067v3",
            "date": "2020-10-24 10:08:52+00:00",
            "title": "Hierarchical Transformer for Task Oriented Dialog Systems",
            "authors": [
                "Bishal Santra",
                "Potnuru Anusha",
                "Pawan Goyal"
            ],
            "abstract": "Generative models for dialog systems have gained much interest because of the\nrecent success of RNN and Transformer based models in tasks like question\nanswering and summarization. Although the task of dialog response generation is\ngenerally seen as a sequence-to-sequence (Seq2Seq) problem, researchers in the\npast have found it challenging to train dialog systems using the standard\nSeq2Seq models. Therefore, to help the model learn meaningful utterance and\nconversation level features, Sordoni et al. (2015b); Serban et al. (2016)\nproposed Hierarchical RNN architecture, which was later adopted by several\nother RNN based dialog systems. With the transformer-based models dominating\nthe seq2seq problems lately, the natural question to ask is the applicability\nof the notion of hierarchy in transformer based dialog systems. In this paper,\nwe propose a generalized framework for Hierarchical Transformer Encoders and\nshow how a standard transformer can be morphed into any hierarchical encoder,\nincluding HRED and HIBERT like models, by using specially designed attention\nmasks and positional encodings. We demonstrate that Hierarchical Encoding helps\nachieve better natural language understanding of the contexts in\ntransformer-based models for task-oriented dialog systems through a wide range\nof experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08074v1",
            "date": "2020-11-04 09:42:54+00:00",
            "title": "Answer Identification in Collaborative Organizational Group Chat",
            "authors": [
                "Naama Tepper",
                "Naama Zwerdling",
                "David Naori",
                "Inbal Ronen"
            ],
            "abstract": "We present a simple unsupervised approach for answer identification in\norganizational group chat. In recent years, organizational group chat is on the\nrise enabling asynchronous text-based collaboration between co-workers in\ndifferent locations and time zones. Finding answers to questions is often\ncritical for work efficiency. However, group chat is characterized by\nintertwined conversations and 'always on' availability, making it hard for\nusers to pinpoint answers to questions they care about in real-time or search\nfor answers in retrospective. In addition, structural and lexical\ncharacteristics differ between chat groups, making it hard to find a 'one model\nfits all' approach. Our Kernel Density Estimation (KDE) based clustering\napproach termed Ans-Chat implicitly learns discussion patterns as a means for\nanswer identification, thus eliminating the need to channel-specific tagging.\nEmpirical evaluation shows that this solution outperforms other approached.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08092v1",
            "date": "2020-10-31 02:33:53+00:00",
            "title": "A Dataset for Tracking Entities in Open Domain Procedural Text",
            "authors": [
                "Niket Tandon",
                "Keisuke Sakaguchi",
                "Bhavana Dalvi Mishra",
                "Dheeraj Rajagopal",
                "Peter Clark",
                "Michal Guerquin",
                "Kyle Richardson",
                "Eduard Hovy"
            ],
            "abstract": "We present the first dataset for tracking state changes in procedural text\nfrom arbitrary domains by using an unrestricted (open) vocabulary. For example,\nin a text describing fog removal using potatoes, a car window may transition\nbetween being foggy, sticky,opaque, and clear. Previous formulations of this\ntask provide the text and entities involved,and ask how those entities change\nfor just a small, pre-defined set of attributes (e.g., location), limiting\ntheir fidelity. Our solution is a new task formulation where given just a\nprocedural text as input, the task is to generate a set of state change\ntuples(entity, at-tribute, before-state, after-state)for each step,where the\nentity, attribute, and state values must be predicted from an open vocabulary.\nUsing crowdsourcing, we create OPENPI1, a high-quality (91.5% coverage as\njudged by humans and completely vetted), and large-scale dataset comprising\n29,928 state changes over 4,050 sentences from 810 procedural real-world\nparagraphs from WikiHow.com. A current state-of-the-art generation model on\nthis task achieves 16.1% F1 based on BLEU metric, leaving enough room for novel\nmodel architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08115v1",
            "date": "2020-11-16 17:25:24+00:00",
            "title": "Learning from Task Descriptions",
            "authors": [
                "Orion Weller",
                "Nicholas Lourie",
                "Matt Gardner",
                "Matthew E. Peters"
            ],
            "abstract": "Typically, machine learning systems solve new tasks by training on thousands\nof examples. In contrast, humans can solve new tasks by reading some\ninstructions, with perhaps an example or two. To take a step toward closing\nthis gap, we introduce a framework for developing NLP systems that solve new\ntasks after reading their descriptions, synthesizing prior work in this area.\nWe instantiate this framework with a new English language dataset, ZEST,\nstructured for task-oriented evaluation on unseen tasks. Formulating task\ndescriptions as questions, we ensure each is general enough to apply to many\npossible inputs, thus comprehensively evaluating a model's ability to solve\neach task. Moreover, the dataset's structure tests specific types of systematic\ngeneralization. We find that the state-of-the-art T5 model achieves a score of\n12% on ZEST, leaving a significant challenge for NLP researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08262v1",
            "date": "2020-11-16 20:30:31+00:00",
            "title": "A Probabilistic Approach in Historical Linguistics Word Order Change in Infinitival Clauses: from Latin to Old French",
            "authors": [
                "Olga Scrivner"
            ],
            "abstract": "This research offers a new interdisciplinary approach to the field of\nLinguistics by using Computational Linguistics, NLP, Bayesian Statistics and\nSociolinguistics methods. This thesis investigates word order change in\ninfinitival clauses from Object-Verb (OV) to Verb-Object (VO) in the history of\nLatin and Old French. By applying a variationist approach, I examine a\nsynchronic word order variation in each stage of language change, from which I\ninfer the character, periodization and constraints of diachronic variation. I\nalso show that in discourse-configurational languages, such as Latin and Early\nOld French, it is possible to identify pragmatically neutral contexts by using\ninformation structure annotation. I further argue that by mapping pragmatic\ncategories into a syntactic structure, we can detect how word order change\nunfolds. For this investigation, the data are extracted from annotated corpora\nspanning several centuries of Latin and Old French and from additional\nresources created by using computational linguistic methods. The data are then\nfurther codified for various pragmatic, semantic, syntactic and sociolinguistic\nfactors. This study also evaluates previous factors proposed to account for\nword order alternation and change. I show how information structure and\nsyntactic constraints change over time and propose a method that allows\nresearchers to differentiate a stable word order alternation from alternation\nindicating a change. Finally, I present a three-stage probabilistic model of\nword order change, which also conforms to traditional language change patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08291v1",
            "date": "2020-11-16 21:31:28+00:00",
            "title": "A Two-Phase Approach for Abstractive Podcast Summarization",
            "authors": [
                "Chujie Zheng",
                "Kunpeng Zhang",
                "Harry Jiannan Wang",
                "Ling Fan"
            ],
            "abstract": "Podcast summarization is different from summarization of other data formats,\nsuch as news, patents, and scientific papers in that podcasts are often longer,\nconversational, colloquial, and full of sponsorship and advertising\ninformation, which imposes great challenges for existing models. In this paper,\nwe focus on abstractive podcast summarization and propose a two-phase approach:\nsentence selection and seq2seq learning. Specifically, we first select\nimportant sentences from the noisy long podcast transcripts. The selection is\nbased on sentence similarity to the reference to reduce the redundancy and the\nassociated latent topics to preserve semantics. Then the selected sentences are\nfed into a pre-trained encoder-decoder framework for the summary generation.\nOur approach achieves promising results regarding both ROUGE-based measures and\nhuman evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08298v1",
            "date": "2020-11-16 21:49:00+00:00",
            "title": "Facebook AI's WMT20 News Translation Task Submission",
            "authors": [
                "Peng-Jen Chen",
                "Ann Lee",
                "Changhan Wang",
                "Naman Goyal",
                "Angela Fan",
                "Mary Williamson",
                "Jiatao Gu"
            ],
            "abstract": "This paper describes Facebook AI's submission to WMT20 shared news\ntranslation task. We focus on the low resource setting and participate in two\nlanguage pairs, Tamil <-> English and Inuktitut <-> English, where there are\nlimited out-of-domain bitext and monolingual data. We approach the low resource\nproblem using two main strategies, leveraging all available data and adapting\nthe system to the target news domain. We explore techniques that leverage\nbitext and monolingual data from all languages, such as self-supervised model\npretraining, multilingual models, data augmentation, and reranking. To better\nadapt the translation system to the test domain, we explore dataset tagging and\nfine-tuning on in-domain data. We observe that different techniques provide\nvaried improvements based on the available data of the language pair. Based on\nthe finding, we integrate these techniques into one training pipeline. For\nEn->Ta, we explore an unconstrained setup with additional Tamil bitext and\nmonolingual data and show that further improvement can be obtained. On the test\nset, our best submitted systems achieve 21.5 and 13.7 BLEU for Ta->En and\nEn->Ta respectively, and 27.9 and 13.0 for Iu->En and En->Iu respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08320v1",
            "date": "2020-11-16 22:45:03+00:00",
            "title": "Don't Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communities",
            "authors": [
                "Carla P\u00e9rez-Almendros",
                "Luis Espinosa-Anke",
                "Steven Schockaert"
            ],
            "abstract": "In this paper, we introduce a new annotated dataset which is aimed at\nsupporting the development of NLP models to identify and categorize language\nthat is patronizing or condescending towards vulnerable communities (e.g.\nrefugees, homeless people, poor families). While the prevalence of such\nlanguage in the general media has long been shown to have harmful effects, it\ndiffers from other types of harmful language, in that it is generally used\nunconsciously and with good intentions. We furthermore believe that the often\nsubtle nature of patronizing and condescending language (PCL) presents an\ninteresting technical challenge for the NLP community. Our analysis of the\nproposed dataset shows that identifying PCL is hard for standard NLP models,\nwith language models such as BERT achieving the best results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08539v1",
            "date": "2020-11-17 10:15:36+00:00",
            "title": "MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining",
            "authors": [
                "Wei Zhu"
            ],
            "abstract": "Despite the development of pre-trained language models (PLMs) significantly\nraise the performances of various Chinese natural language processing (NLP)\ntasks, the vocabulary for these Chinese PLMs remain to be the one provided by\nGoogle Chinese Bert \\cite{devlin2018bert}, which is based on Chinese\ncharacters. Second, the masked language model pre-training is based on a single\nvocabulary, which limits its downstream task performances. In this work, we\nfirst propose a novel method, \\emph{seg\\_tok}, to form the vocabulary of\nChinese BERT, with the help of Chinese word segmentation (CWS) and subword\ntokenization. Then we propose three versions of multi-vocabulary pretraining\n(MVP) to improve the models expressiveness. Experiments show that: (a) compared\nwith char based vocabulary, \\emph{seg\\_tok} does not only improves the\nperformances of Chinese PLMs on sentence level tasks, it can also improve\nefficiency; (b) MVP improves PLMs' downstream performance, especially it can\nimprove \\emph{seg\\_tok}'s performances on sequence labeling tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08626v2",
            "date": "2020-11-17 13:39:05+00:00",
            "title": "Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining",
            "authors": [
                "Zijun Sun",
                "Chun Fan",
                "Xiaofei Sun",
                "Yuxian Meng",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "The goal of semi-supervised learning is to utilize the unlabeled, in-domain\ndataset U to improve models trained on the labeled dataset D. Under the context\nof large-scale language-model (LM) pretraining, how we can make the best use of\nU is poorly understood: is semi-supervised learning still beneficial with the\npresence of large-scale pretraining? should U be used for in-domain LM\npretraining or pseudo-label generation? how should the pseudo-label based\nsemi-supervised model be actually implemented? how different semi-supervised\nstrategies affect performances regarding D of different sizes, U of different\nsizes, etc. In this paper, we conduct comprehensive studies on semi-supervised\nlearning in the task of text classification under the context of large-scale LM\npretraining. Our studies shed important lights on the behavior of\nsemi-supervised learning methods: (1) with the presence of in-domain\npretraining LM on U, open-domain LM pretraining is unnecessary; (2) both the\nin-domain pretraining strategy and the pseudo-label based strategy introduce\nsignificant performance boosts, with the former performing better with larger\nU, the latter performing better with smaller U, and the combination leading to\nthe largest performance boost; (3) self-training (pretraining first on pseudo\nlabels D' and then fine-tuning on D) yields better performances when D is\nsmall, while joint training on the combination of pseudo labels D' and the\noriginal dataset D yields better performances when D is large. Using\nsemi-supervised learning strategies, we are able to achieve a performance of\naround 93.8% accuracy with only 50 training data points on the IMDB dataset,\nand a competitive performance of 96.6% with the full IMDB dataset. Our work\nmarks an initial step in understanding the behavior of semi-supervised learning\nmodels under the context of large-scale pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08835v2",
            "date": "2020-11-17 18:55:00+00:00",
            "title": "Toward Understanding Clinical Context of Medication Change Events in Clinical Narratives",
            "authors": [
                "Diwakar Mahajan",
                "Jennifer J Liang",
                "Ching-Huei Tsou"
            ],
            "abstract": "Understanding medication events in clinical narratives is essential to\nachieving a complete picture of a patient's medication history. While prior\nresearch has explored classification of medication changes from clinical notes,\nstudies to date have not considered the necessary clinical context needed for\ntheir use in real-world applications, such as medication timeline generation\nand medication reconciliation. In this paper, we present the Contextualized\nMedication Event Dataset (CMED), a dataset for capturing relevant context of\nmedication changes documented in clinical notes, which was developed using a\nnovel conceptual framework that organizes context for clinical events into\nvarious orthogonal dimensions. In this process, we define specific contextual\naspects pertinent to medication change events, characterize the dataset, and\nreport the results of preliminary experiments. CMED consists of 9,013\nmedication mentions annotated over 500 clinical notes, and will be released to\nthe community as a shared task in 2021.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08903v2",
            "date": "2020-11-17 19:41:02+00:00",
            "title": "Towards Olfactory Information Extraction from Text: A Case Study on Detecting Smell Experiences in Novels",
            "authors": [
                "Ryan Brate",
                "Paul Groth",
                "Marieke van Erp"
            ],
            "abstract": "Environmental factors determine the smells we perceive, but societal factors\nfactors shape the importance, sentiment and biases we give to them.\nDescriptions of smells in text, or as we call them `smell experiences', offer a\nwindow into these factors, but they must first be identified. To the best of\nour knowledge, no tool exists to extract references to smell experiences from\ntext. In this paper, we present two variations on a semi-supervised approach to\nidentify smell experiences in English literature. The combined set of patterns\nfrom both implementations offer significantly better performance than a\nkeyword-based baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08951v1",
            "date": "2020-11-17 21:21:37+00:00",
            "title": "Exploring Neural Entity Representations for Semantic Information",
            "authors": [
                "Andrew Runge",
                "Eduard Hovy"
            ],
            "abstract": "Neural methods for embedding entities are typically extrinsically evaluated\non downstream tasks and, more recently, intrinsically using probing tasks.\nDownstream task-based comparisons are often difficult to interpret due to\ndifferences in task structure, while probing task evaluations often look at\nonly a few attributes and models. We address both of these issues by evaluating\na diverse set of eight neural entity embedding methods on a set of simple\nprobing tasks, demonstrating which methods are able to remember words used to\ndescribe entities, learn type, relationship and factual information, and\nidentify how frequently an entity is mentioned. We also compare these methods\nin a unified framework on two entity linking tasks and discuss how they\ngeneralize to different model architectures and datasets.",
            "journal ref": "Proceedings of the Third BlackboxNLP Workshop on Analyzing and\n  Interpreting Neural Networks for NLP 2020, p. 204-216",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09031v3",
            "date": "2020-11-18 01:35:01+00:00",
            "title": "Self-training For Pre-training Language Models",
            "authors": [
                "Tong Guo"
            ],
            "abstract": "Language model pre-training has proven to be useful in many language\nunderstanding tasks. In this paper, we investigate whether it is still helpful\nto add the self-training method in the pre-training step and the fine-tuning\nstep. Towards this goal, we propose a learning framework that making best use\nof the unlabel data on the low-resource and high-resource labeled dataset. In\nindustry NLP applications, we have large amounts of data produced by users or\ncustomers. Our learning framework is based on this large amounts of unlabel\ndata. First, We use the model fine-tuned on manually labeled dataset to predict\npseudo labels for the user-generated unlabeled data. Then we use the pseudo\nlabels to supervise the task-specific training on the large amounts of\nuser-generated data. We consider this task-specific training step on pseudo\nlabels as a pre-training step for the next fine-tuning step. At last, we\nfine-tune on the manually labeled dataset upon the pre-trained model. In this\nwork, we first empirically show that our method is able to solidly improve the\nperformance by 3.6%, when the manually labeled fine-tuning dataset is\nrelatively small. Then we also show that our method still is able to improve\nthe performance further by 0.2%, when the manually labeled fine-tuning dataset\nis relatively large enough. We argue that our method make the best use of the\nunlabel data, which is superior to either pre-training or self-training alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09140v1",
            "date": "2020-11-18 07:33:03+00:00",
            "title": "Diverse and Non-redundant Answer Set Extraction on Community QA based on DPPs",
            "authors": [
                "Shogo Fujita",
                "Tomohide Shibata",
                "Manabu Okumura"
            ],
            "abstract": "In community-based question answering (CQA) platforms, it takes time for a\nuser to get useful information from among many answers. Although one solution\nis an answer ranking method, the user still needs to read through the\ntop-ranked answers carefully. This paper proposes a new task of selecting a\ndiverse and non-redundant answer set rather than ranking the answers. Our\nmethod is based on determinantal point processes (DPPs), and it calculates the\nanswer importance and similarity between answers by using BERT. We built a\ndataset focusing on a Japanese CQA site, and the experiments on this dataset\ndemonstrated that the proposed method outperformed several baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09210v1",
            "date": "2020-11-18 10:59:14+00:00",
            "title": "Improving Document-Level Sentiment Analysis with User and Product Context",
            "authors": [
                "Chenyang Lyu",
                "Jennifer Foster",
                "Yvette Graham"
            ],
            "abstract": "Past work that improves document-level sentiment analysis by encoding user\nand product information has been limited to considering only the text of the\ncurrent review. We investigate incorporating additional review text available\nat the time of sentiment prediction that may prove meaningful for guiding\nprediction. Firstly, we incorporate all available historical review text\nbelonging to the author of the review in question. Secondly, we investigate the\ninclusion of historical reviews associated with the current product (written by\nother users). We achieve this by explicitly storing representations of reviews\nwritten by the same user and about the same product and force the model to\nmemorize all reviews for one particular user and product. Additionally, we drop\nthe hierarchical architecture used in previous work to enable words in the text\nto directly attend to each other. Experiment results on IMDB, Yelp 2013 and\nYelp 2014 datasets show improvement to state-of-the-art of more than 2\npercentage points in the best case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09212v1",
            "date": "2020-11-18 11:10:29+00:00",
            "title": "On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition",
            "authors": [
                "Manon Macary",
                "Marie Tahon",
                "Yannick Est\u00e8ve",
                "Anthony Rousseau"
            ],
            "abstract": "Pre-training for feature extraction is an increasingly studied approach to\nget better continuous representations of audio and text content. In the present\nwork, we use wav2vec and camemBERT as self-supervised learned models to\nrepresent our data in order to perform continuous emotion recognition from\nspeech (SER) on AlloSat, a large French emotional database describing the\nsatisfaction dimension, and on the state of the art corpus SEWA focusing on\nvalence, arousal and liking dimensions. To the authors' knowledge, this paper\npresents the first study showing that the joint use of wav2vec and BERT-like\npre-trained features is very relevant to deal with continuous SER task, usually\ncharacterized by a small amount of labeled training data. Evaluated by the\nwell-known concordance correlation coefficient (CCC), our experiments show that\nwe can reach a CCC value of 0.825 instead of 0.592 when using MFCC in\nconjunction with word2vec word embedding on the AlloSat dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09249v1",
            "date": "2020-11-18 12:49:17+00:00",
            "title": "The Ubiqus English-Inuktitut System for WMT20",
            "authors": [
                "Fran\u00e7ois Hernandez",
                "Vincent Nguyen"
            ],
            "abstract": "This paper describes Ubiqus' submission to the WMT20 English-Inuktitut shared\nnews translation task. Our main system, and only submission, is based on a\nmultilingual approach, jointly training a Transformer model on several\nagglutinative languages. The English-Inuktitut translation task is challenging\nat every step, from data selection, preparation and tokenization to quality\nevaluation down the line. Difficulties emerge both because of the peculiarities\nof the Inuktitut language as well as the low-resource context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09289v1",
            "date": "2020-11-18 13:59:36+00:00",
            "title": "Master Thesis: Neural Sign Language Translation by Learning Tokenization",
            "authors": [
                "Alptekin Orbay"
            ],
            "abstract": "In this thesis, we propose a multitask learning based method to improve\nNeural Sign Language Translation (NSLT) consisting of two parts, a tokenization\nlayer and Neural Machine Translation (NMT). The tokenization part focuses on\nhow Sign Language (SL) videos should be represented to be fed into the other\npart. It has not been studied elaborately whereas NMT research has attracted\nseveral researchers contributing enormous advancements. Up to now, there are\ntwo main input tokenization levels, namely frame-level and gloss-level\ntokenization. Glosses are world-like intermediate presentation and unique to\nSLs. Therefore, we aim to develop a generic sign-level tokenization layer so\nthat it is applicable to other domains without further effort. We begin with\ninvestigating current tokenization approaches and explain their weaknesses with\nseveral experiments. To provide a solution, we adapt Transfer Learning,\nMultitask Learning and Unsupervised Domain Adaptation into this research to\nleverage additional supervision. We succeed in enabling knowledge transfer\nbetween SLs and improve translation quality by 5 points in BLEU-4 and 8 points\nin ROUGE scores. Secondly, we show the effects of body parts by extensive\nexperiments in all the tokenization approaches. Apart from these, we adopt\n3D-CNNs to improve efficiency in terms of time and space. Lastly, we discuss\nthe advantages of sign-level tokenization over gloss-level tokenization. To sum\nup, our proposed method eliminates the need for gloss level annotation to\nobtain higher scores by providing additional supervision by utilizing weak\nsupervision sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09378v1",
            "date": "2020-11-18 16:23:30+00:00",
            "title": "LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization",
            "authors": [
                "Nurul Lubis",
                "Christian Geishauser",
                "Michael Heck",
                "Hsien-chin Lin",
                "Marco Moresi",
                "Carel van Niekerk",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Reinforcement learning (RL) can enable task-oriented dialogue systems to\nsteer the conversation towards successful task completion. In an end-to-end\nsetting, a response can be constructed in a word-level sequential decision\nmaking process with the entire system vocabulary as action space. Policies\ntrained in such a fashion do not require expert-defined action spaces, but they\nhave to deal with large action spaces and long trajectories, making RL\nimpractical. Using the latent space of a variational model as action space\nalleviates this problem. However, current approaches use an uninformed prior\nfor training and optimize the latent distribution solely on the context. It is\ntherefore unclear whether the latent representation truly encodes the\ncharacteristics of different actions. In this paper, we explore three ways of\nleveraging an auxiliary task to shape the latent variable distribution: via\npre-training, to obtain an informed prior, and via multitask learning. We\nchoose response auto-encoding as the auxiliary task, as this captures the\ngenerative factors of dialogue responses while requiring low computational cost\nand neither additional data nor labels. Our approach yields a more\naction-characterized latent representations which support end-to-end dialogue\npolicy optimization and achieves state-of-the-art success rates. These results\nwarrant a more wide-spread use of RL in end-to-end dialogue models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09379v1",
            "date": "2020-11-18 16:23:30+00:00",
            "title": "Out-of-Task Training for Dialog State Tracking Models",
            "authors": [
                "Michael Heck",
                "Carel van Niekerk",
                "Nurul Lubis",
                "Christian Geishauser",
                "Hsien-Chin Lin",
                "Marco Moresi",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "Dialog state tracking (DST) suffers from severe data sparsity. While many\nnatural language processing (NLP) tasks benefit from transfer learning and\nmulti-task learning, in dialog these methods are limited by the amount of\navailable data and by the specificity of dialog applications. In this work, we\nsuccessfully utilize non-dialog data from unrelated NLP tasks to train dialog\nstate trackers. This opens the door to the abundance of unrelated NLP corpora\nto mitigate the data sparsity issue inherent to DST.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09413v1",
            "date": "2020-11-18 17:21:51+00:00",
            "title": "Topology of Word Embeddings: Singularities Reflect Polysemy",
            "authors": [
                "Alexander Jakubowski",
                "Milica Ga\u0161i\u0107",
                "Marcus Zibrowius"
            ],
            "abstract": "The manifold hypothesis suggests that word vectors live on a submanifold\nwithin their ambient vector space. We argue that we should, more accurately,\nexpect them to live on a pinched manifold: a singular quotient of a manifold\nobtained by identifying some of its points. The identified, singular points\ncorrespond to polysemous words, i.e. words with multiple meanings. Our point of\nview suggests that monosemous and polysemous words can be distinguished based\non the topology of their neighbourhoods. We present two kinds of empirical\nevidence to support this point of view: (1) We introduce a topological measure\nof polysemy based on persistent homology that correlates well with the actual\nnumber of meanings of a word. (2) We propose a simple, topologically motivated\nsolution to the SemEval-2010 task on Word Sense Induction & Disambiguation that\nproduces competitive results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09463v2",
            "date": "2020-11-18 18:41:27+00:00",
            "title": "EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications",
            "authors": [
                "Minghui Qiu",
                "Peng Li",
                "Hanjie Pan",
                "Chengyu Wang",
                "Ang Wang",
                "Cen Chen",
                "Yaliang Li",
                "Dehong Gao",
                "Jun Huang",
                "Yong Li",
                "Jun Yang",
                "Deng Cai",
                "Wei Lin"
            ],
            "abstract": "The literature has witnessed the success of applying deep Transfer Learning\n(TL) algorithms to many NLP applications, yet it is not easy to build a simple\nand scalable TL toolkit for this purpose. To bridge this gap, the EasyTransfer\nplatform is designed to make it easy to develop deep TL algorithms for NLP\napplications. It is built with rich API abstractions, a scalable architecture\nand comprehensive deep TL algorithms, to make the development of NLP\napplications easier. To be specific, the build-in data and model parallelism\nstrategy shows to be 4x faster than the default distribution strategy of\nTensorflow. EasyTransfer supports the mainstream pre-trained ModelZoo,\nincluding Pre-trained Language Models (PLMs) and multi-modality models. It also\nintegrates various SOTA models for mainstream NLP applications in AppZoo, and\nsupports mainstream TL algorithms as well. The toolkit is convenient for users\nto quickly start model training, evaluation, offline prediction, and online\ndeployment. This system is currently deployed at Alibaba to support a variety\nof business scenarios, including item recommendation, personalized search, and\nconversational question answering. Extensive experiments on real-world datasets\nshow that EasyTransfer is suitable for online production with cutting-edge\nperformance. The source code of EasyTransfer is released at Github\n(https://github.com/alibaba/EasyTransfer).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09567v1",
            "date": "2020-11-18 22:33:09+00:00",
            "title": "Predicting metrical patterns in Spanish poetry with language models",
            "authors": [
                "Javier de la Rosa",
                "Salvador Ros",
                "Elena Gonz\u00e1lez-Blanco"
            ],
            "abstract": "In this paper, we compare automated metrical pattern identification systems\navailable for Spanish against extensive experiments done by fine-tuning\nlanguage models trained on the same task. Despite being initially conceived as\na model suitable for semantic tasks, our results suggest that BERT-based models\nretain enough structural information to perform reasonably well for Spanish\nscansion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09684v1",
            "date": "2020-11-19 06:24:42+00:00",
            "title": "SentiLSTM: A Deep Learning Approach for Sentiment Analysis of Restaurant Reviews",
            "authors": [
                "Eftekhar Hossain",
                "Omar Sharif",
                "Mohammed Moshiul Hoque",
                "Iqbal H. Sarker"
            ],
            "abstract": "The amount of textual data generation has increased enormously due to the\neffortless access of the Internet and the evolution of various web 2.0\napplications. These textual data productions resulted because of the people\nexpress their opinion, emotion or sentiment about any product or service in the\nform of tweets, Facebook post or status, blog write up, and reviews. Sentiment\nanalysis deals with the process of computationally identifying and categorizing\nopinions expressed in a piece of text, especially in order to determine whether\nthe writer's attitude toward a particular topic is positive, negative, or\nneutral. The impact of customer review is significant to perceive the customer\nattitude towards a restaurant. Thus, the automatic detection of sentiment from\nreviews is advantageous for the restaurant owners, or service providers and\ncustomers to make their decisions or services more satisfactory. This paper\nproposes, a deep learning-based technique (i.e., BiLSTM) to classify the\nreviews provided by the clients of the restaurant into positive and negative\npolarities. A corpus consists of 8435 reviews is constructed to evaluate the\nproposed technique. In addition, a comparative analysis of the proposed\ntechnique with other machine learning algorithms presented. The results of the\nevaluation on test dataset show that BiLSTM technique produced in the highest\naccuracy of 91.35%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09708v1",
            "date": "2020-11-19 08:22:49+00:00",
            "title": "Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?",
            "authors": [
                "Yufan Zhao",
                "Wei Wu",
                "Can Xu"
            ],
            "abstract": "We study knowledge-grounded dialogue generation with pre-trained language\nmodels. Instead of pursuing new state-of-the-art on benchmarks, we try to\nunderstand if the knowledge stored in parameters of the pre-trained models is\nalready enough to ground open domain dialogues, and thus allows us to get rid\nof the dependency on external knowledge sources in generation. Through\nextensive experiments on benchmarks, we find that by fine-tuning with a few\ndialogues containing knowledge, the pre-trained language models can outperform\nthe state-of-the-art model that requires external knowledge in automatic\nevaluation and human judgment, suggesting a positive answer to the question we\nraised.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09739v1",
            "date": "2020-11-19 09:29:51+00:00",
            "title": "Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT",
            "authors": [
                "Ruifeng Yuan",
                "Zili Wang",
                "Wenjie Li"
            ],
            "abstract": "Most current extractive summarization models generate summaries by selecting\nsalient sentences. However, one of the problems with sentence-level extractive\nsummarization is that there exists a gap between the human-written gold summary\nand the oracle sentence labels. In this paper, we propose to extract fact-level\nsemantic units for better extractive summarization. We also introduce a\nhierarchical structure, which incorporates the multi-level of granularities of\nthe textual information into the model. In addition, we incorporate our model\nwith BERT using a hierarchical graph mask. This allows us to combine BERT's\nability in natural language understanding and the structural information\nwithout increasing the scale of the model. Experiments on the CNN/DaliyMail\ndataset show that our model achieves state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09817v3",
            "date": "2020-11-19 13:40:03+00:00",
            "title": "Entity Recognition and Relation Extraction from Scientific and Technical Texts in Russian",
            "authors": [
                "Elena Bruches",
                "Alexey Pauls",
                "Tatiana Batura",
                "Vladimir Isachenko"
            ],
            "abstract": "This paper is devoted to the study of methods for information extraction\n(entity recognition and relation classification) from scientific texts on\ninformation technology. Scientific publications provide valuable information\ninto cutting-edge scientific advances, but efficient processing of increasing\namounts of data is a time-consuming task. In this paper, several modifications\nof methods for the Russian language are proposed. It also includes the results\nof experiments comparing a keyword extraction method, vocabulary method, and\nsome methods based on neural networks. Text collections for these tasks exist\nfor the English language and are actively used by the scientific community, but\nat present, such datasets in Russian are not publicly available. In this paper,\nwe present a corpus of scientific texts in Russian, RuSERRC. This dataset\nconsists of 1600 unlabeled documents and 80 labeled with entities and semantic\nrelations (6 relation types were considered). The dataset and models are\navailable at https://github.com/iis-research-team. We hope they can be useful\nfor research purposes and development of information extraction systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09825v1",
            "date": "2020-11-19 13:58:47+00:00",
            "title": "Do We Need Online NLU Tools?",
            "authors": [
                "Petr Lorenc",
                "Petr Marek",
                "Jan Pichl",
                "Jakub Konr\u00e1d",
                "Jan \u0160ediv\u00fd"
            ],
            "abstract": "The intent recognition is an essential algorithm of any conversational AI\napplication. It is responsible for the classification of an input message into\nmeaningful classes. In many bot development platforms, we can configure the NLU\npipeline. Several intent recognition services are currently available as an\nAPI, or we choose from many open-source alternatives. However, there is no\ncomparison of intent recognition services and open-source algorithms. Many\nfactors make the selection of the right approach to the intent recognition\nchallenging in practice. In this paper, we suggest criteria to choose the best\nintent recognition algorithm for an application. We present a dataset for\nevaluation. Finally, we compare selected public NLU services with selected\nopen-source algorithms for intent recognition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10280v1",
            "date": "2020-11-20 08:50:06+00:00",
            "title": "Are Chess Discussions Racist? An Adversarial Hate Speech Data Set",
            "authors": [
                "Rupak Sarkar",
                "Ashiqur R. KhudaBukhsh"
            ],
            "abstract": "On June 28, 2020, while presenting a chess podcast on Grandmaster Hikaru\nNakamura, Antonio Radi\\'c's YouTube handle got blocked because it contained\n\"harmful and dangerous\" content. YouTube did not give further specific reason,\nand the channel got reinstated within 24 hours. However, Radi\\'c speculated\nthat given the current political situation, a referral to \"black against\nwhite\", albeit in the context of chess, earned him this temporary ban. In this\npaper, via a substantial corpus of 681,995 comments, on 8,818 YouTube videos\nhosted by five highly popular chess-focused YouTube channels, we ask the\nfollowing research question: \\emph{how robust are off-the-shelf hate-speech\nclassifiers to out-of-domain adversarial examples?} We release a data set of\n1,000 annotated comments where existing hate speech classifiers misclassified\nbenign chess discussions as hate speech. We conclude with an intriguing analogy\nresult on racial bias with our findings pointing out to the broader challenge\nof color polysemy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10361v1",
            "date": "2020-11-20 12:03:41+00:00",
            "title": "1st AfricaNLP Workshop Proceedings, 2020",
            "authors": [
                "Kathleen Siminyu",
                "Laura Martinus",
                "Vukosi Marivate"
            ],
            "abstract": "Proceedings of the 1st AfricaNLP Workshop held on 26th April alongside ICLR\n2020, Virtual Conference, Formerly Addis Ababa Ethiopia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10428v1",
            "date": "2020-11-20 14:51:07+00:00",
            "title": "Topic modelling discourse dynamics in historical newspapers",
            "authors": [
                "Jani Marjanen",
                "Elaine Zosa",
                "Simon Hengchen",
                "Lidia Pivovarova",
                "Mikko Tolonen"
            ],
            "abstract": "This paper addresses methodological issues in diachronic data analysis for\nhistorical research. We apply two families of topic models (LDA and DTM) on a\nrelatively large set of historical newspapers, with the aim of capturing and\nunderstanding discourse dynamics. Our case study focuses on newspapers and\nperiodicals published in Finland between 1854 and 1917, but our method can\neasily be transposed to any diachronic data. Our main contributions are a) a\ncombined sampling, training and inference procedure for applying topic models\nto huge and imbalanced diachronic text collections; b) a discussion on the\ndifferences between two topic models for this type of data; c) quantifying\ntopic prominence for a period and thus a generalization of document-wise topic\nassignment to a discourse level; and d) a discussion of the role of humanistic\ninterpretation with regard to analysing discourse dynamics through topic\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10683v1",
            "date": "2020-11-21 00:28:34+00:00",
            "title": "Athena: Constructing Dialogues Dynamically with Discourse Constraints",
            "authors": [
                "Vrindavan Harrison",
                "Juraj Juraska",
                "Wen Cui",
                "Lena Reed",
                "Kevin K. Bowden",
                "Jiaqi Wu",
                "Brian Schwarzmann",
                "Abteen Ebrahimi",
                "Rishi Rajasekaran",
                "Nikhil Varghese",
                "Max Wechsler-Azen",
                "Steve Whittaker",
                "Jeffrey Flanigan",
                "Marilyn Walker"
            ],
            "abstract": "This report describes Athena, a dialogue system for spoken conversation on\npopular topics and current events. We develop a flexible topic-agnostic\napproach to dialogue management that dynamically configures dialogue based on\ngeneral principles of entity and topic coherence. Athena's dialogue manager\nuses a contract-based method where discourse constraints are dispatched to\nclusters of response generators. This allows Athena to procure responses from\ndynamic sources, such as knowledge graph traversals and feature-based\non-the-fly response retrieval methods. After describing the dialogue system\narchitecture, we perform an analysis of conversations that Athena participated\nin during the 2019 Alexa Prize Competition. We conclude with a report on\nseveral user studies we carried out to better understand how individual user\ncharacteristics affect system ratings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10819v1",
            "date": "2020-11-21 16:37:28+00:00",
            "title": "Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference",
            "authors": [
                "Ond\u0159ej Du\u0161ek",
                "Zden\u011bk Kasner"
            ],
            "abstract": "A major challenge in evaluating data-to-text (D2T) generation is measuring\nthe semantic accuracy of the generated text, i.e. checking if the output text\ncontains all and only facts supported by the input data. We propose a new\nmetric for evaluating the semantic accuracy of D2T generation based on a neural\nmodel pretrained for natural language inference (NLI). We use the NLI model to\ncheck textual entailment between the input data and the output text in both\ndirections, allowing us to reveal omissions or hallucinations. Input data are\nconverted to text for NLI using trivial templates. Our experiments on two\nrecent D2T datasets show that our metric can achieve high accuracy in\nidentifying erroneous system outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10832v1",
            "date": "2020-11-21 17:53:41+00:00",
            "title": "Sensing Ambiguity in Henry James' \"The Turn of the Screw\"",
            "authors": [
                "Victor Makarenkov",
                "Yael Segalovitz"
            ],
            "abstract": "Fields such as the philosophy of language, continental philosophy, and\nliterary studies have long established that human language is, at its essence,\nambiguous and that this quality, although challenging to communication,\nenriches language and points to the complexity of human thought. On the other\nhand, in the NLP field there have been ongoing efforts aimed at disambiguation\nfor various downstream tasks. This work brings together computational text\nanalysis and literary analysis to demonstrate the extent to which ambiguity in\ncertain texts plays a key role in shaping meaning and thus requires analysis\nrather than elimination. We revisit the discussion, well known in the\nhumanities, about the role ambiguity plays in Henry James' 19th century\nnovella, The Turn of the Screw. We model each of the novella's two competing\ninterpretations as a topic and computationally demonstrate that the duality\nbetween them exists consistently throughout the work and shapes, rather than\nobscures, its meaning. We also demonstrate that cosine similarity and word\nmover's distance are sensitive enough to detect ambiguity in its most subtle\nliterary form, despite doubts to the contrary raised by literary scholars. Our\nanalysis is built on topic word lists and word embeddings from various sources.\nWe first claim, and then empirically show, the interdependence between\ncomputational analysis and close reading performed by a human expert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11090v1",
            "date": "2020-11-22 19:19:33+00:00",
            "title": "Cross-Domain Generalization Through Memorization: A Study of Nearest Neighbors in Neural Duplicate Question Detection",
            "authors": [
                "Yadollah Yaghoobzadeh",
                "Alexandre Rochette",
                "Timothy J. Hazen"
            ],
            "abstract": "Duplicate question detection (DQD) is important to increase efficiency of\ncommunity and automatic question answering systems. Unfortunately, gathering\nsupervised data in a domain is time-consuming and expensive, and our ability to\nleverage annotations across domains is minimal. In this work, we leverage\nneural representations and study nearest neighbors for cross-domain\ngeneralization in DQD. We first encode question pairs of the source and target\ndomain in a rich representation space and then using a k-nearest neighbour\nretrieval-based method, we aggregate the neighbors' labels and distances to\nrank pairs. We observe robust performance of this method in different\ncross-domain scenarios of StackExchange, Spring and Quora datasets,\noutperforming cross-entropy classification in multiple cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11115v1",
            "date": "2020-11-22 21:51:19+00:00",
            "title": "Employing distributional semantics to organize task-focused vocabulary learning",
            "authors": [
                "Haemanth Santhi Ponnusamy",
                "Detmar Meurers"
            ],
            "abstract": "How can a learner systematically prepare for reading a book they are\ninterested in? In this paper,we explore how computational linguistic methods\nsuch as distributional semantics, morphological clustering, and exercise\ngeneration can be combined with graph-based learner models to answer this\nquestion both conceptually and in practice. Based on the highly structured\nlearner model and concepts from network analysis, the learner is guided to\nefficiently explore the targeted lexical space. They practice using multi-gap\nlearning activities generated from the book focused on words that are central\nto the targeted lexical space. As such the approach offers a unique combination\nof computational linguistic methods with concepts from network analysis and the\ntutoring system domain to support learners in achieving their individual,\nreading task-based learning goals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11387v1",
            "date": "2020-11-23 13:29:16+00:00",
            "title": "STEPs-RL: Speech-Text Entanglement for Phonetically Sound Representation Learning",
            "authors": [
                "Prakamya Mishra"
            ],
            "abstract": "In this paper, we present a novel multi-modal deep neural network\narchitecture that uses speech and text entanglement for learning phonetically\nsound spoken-word representations. STEPs-RL is trained in a supervised manner\nto predict the phonetic sequence of a target spoken-word using its contextual\nspoken word's speech and text, such that the model encodes its meaningful\nlatent representations. Unlike existing work, we have used text along with\nspeech for auditory representation learning to capture semantical and\nsyntactical information along with the acoustic and temporal information. The\nlatent representations produced by our model were not only able to predict the\ntarget phonetic sequences with an accuracy of 89.47% but were also able to\nachieve competitive results to textual word representation models, Word2Vec &\nFastText (trained on textual transcripts), when evaluated on four widely used\nword similarity benchmark datasets. In addition, investigation of the generated\nvector space also demonstrated the capability of the proposed model to capture\nthe phonetic structure of the spoken-words. To the best of our knowledge, none\nof the existing works use speech and text entanglement for learning spoken-word\nrepresentation, which makes this work first of its kind.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11465v3",
            "date": "2020-11-23 15:24:27+00:00",
            "title": "Bi-ISCA: Bidirectional Inter-Sentence Contextual Attention Mechanism for Detecting Sarcasm in User Generated Noisy Short Text",
            "authors": [
                "Prakamya Mishra",
                "Saroj Kaushik",
                "Kuntal Dey"
            ],
            "abstract": "Many online comments on social media platforms are hateful, humorous, or\nsarcastic. The sarcastic nature of these comments (especially the short ones)\nalters their actual implied sentiments, which leads to misinterpretations by\nthe existing sentiment analysis models. A lot of research has already been done\nto detect sarcasm in the text using user-based, topical, and conversational\ninformation but not much work has been done to use inter-sentence contextual\ninformation for detecting the same. This paper proposes a new state-of-the-art\ndeep learning architecture that uses a novel Bidirectional Inter-Sentence\nContextual Attention mechanism (Bi-ISCA) to capture inter-sentence dependencies\nfor detecting sarcasm in the user-generated short text using only the\nconversational context. The proposed deep learning model demonstrates the\ncapability to capture explicit, implicit, and contextual incongruous words &\nphrases responsible for invoking sarcasm. Bi-ISCA generates state-of-the-art\nresults on two widely used benchmark datasets for the sarcasm detection task\n(Reddit and Twitter). To the best of our knowledge, none of the existing\nstate-of-the-art models use an inter-sentence contextual attention mechanism to\ndetect sarcasm in the user-generated short text using only conversational\ncontext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11499v1",
            "date": "2020-11-23 16:00:42+00:00",
            "title": "Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model",
            "authors": [
                "Juntao Li",
                "Ruidan He",
                "Hai Ye",
                "Hwee Tou Ng",
                "Lidong Bing",
                "Rui Yan"
            ],
            "abstract": "Recent research indicates that pretraining cross-lingual language models on\nlarge-scale unlabeled texts yields significant performance improvements over\nvarious cross-lingual and low-resource tasks. Through training on one hundred\nlanguages and terabytes of texts, cross-lingual language models have proven to\nbe effective in leveraging high-resource languages to enhance low-resource\nlanguage processing and outperform monolingual models. In this paper, we\nfurther investigate the cross-lingual and cross-domain (CLCD) setting when a\npretrained cross-lingual language model needs to adapt to new domains.\nSpecifically, we propose a novel unsupervised feature decomposition method that\ncan automatically extract domain-specific features and domain-invariant\nfeatures from the entangled pretrained cross-lingual representations, given\nunlabeled raw texts in the source language. Our proposed model leverages mutual\ninformation estimation to decompose the representations computed by a\ncross-lingual model into domain-invariant and domain-specific parts.\nExperimental results show that our proposed method achieves significant\nperformance improvements over the state-of-the-art pretrained cross-lingual\nlanguage model in the CLCD setting. The source code of this paper is publicly\navailable at https://github.com/lijuntaopku/UFD.",
            "journal ref": "IJCAI-PRICAI2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11523v3",
            "date": "2020-11-23 16:33:48+00:00",
            "title": "An Online Multilingual Hate speech Recognition System",
            "authors": [
                "Neeraj Vashistha",
                "Arkaitz Zubiaga",
                "Shanky Sharma"
            ],
            "abstract": "The exponential increase in the use of the Internet and social media over the\nlast two decades has changed human interaction. This has led to many positive\noutcomes, but at the same time it has brought risks and harms. While the volume\nof harmful content online, such as hate speech, is not manageable by humans,\ninterest in the academic community to investigate automated means for hate\nspeech detection has increased. In this study, we analyse six publicly\navailable datasets by combining them into a single homogeneous dataset and\nclassify them into three classes, abusive, hateful or neither. We create a\nbaseline model and we improve model performance scores using various\noptimisation techniques. After attaining a competitive performance score, we\ncreate a tool which identifies and scores a page with effective metric in\nnear-real time and uses the same as feedback to re-train our model. We prove\nthe competitive performance of our multilingual model on two langauges, English\nand Hindi, leading to comparable or superior performance to most monolingual\nmodels.",
            "journal ref": "Information 12, no. 1: 5 (2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11536v1",
            "date": "2020-11-23 16:49:37+00:00",
            "title": "Studying Taxonomy Enrichment on Diachronic WordNet Versions",
            "authors": [
                "Irina Nikishina",
                "Alexander Panchenko",
                "Varvara Logacheva",
                "Natalia Loukachevitch"
            ],
            "abstract": "Ontologies, taxonomies, and thesauri are used in many NLP tasks. However,\nmost studies are focused on the creation of these lexical resources rather than\nthe maintenance of the existing ones. Thus, we address the problem of taxonomy\nenrichment. We explore the possibilities of taxonomy extension in a\nresource-poor setting and present methods which are applicable to a large\nnumber of languages. We create novel English and Russian datasets for training\nand evaluating taxonomy enrichment models and describe a technique of creating\nsuch datasets for other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11673v1",
            "date": "2020-11-23 19:12:31+00:00",
            "title": "Does BERT Understand Sentiment? Leveraging Comparisons Between Contextual and Non-Contextual Embeddings to Improve Aspect-Based Sentiment Models",
            "authors": [
                "Natesh Reddy",
                "Pranaydeep Singh",
                "Muktabh Mayank Srivastava"
            ],
            "abstract": "When performing Polarity Detection for different words in a sentence, we need\nto look at the words around to understand the sentiment. Massively pretrained\nlanguage models like BERT can encode not only just the words in a document but\nalso the context around the words along with them. This begs the questions,\n\"Does a pretrain language model also automatically encode sentiment information\nabout each word?\" and \"Can it be used to infer polarity towards different\naspects?\". In this work we try to answer this question by showing that training\na comparison of a contextual embedding from BERT and a generic word embedding\ncan be used to infer sentiment. We also show that if we finetune a subset of\nweights the model built on comparison of BERT and generic word embedding, it\ncan get state of the art results for Polarity Detection in Aspect Based\nSentiment Classification datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11712v1",
            "date": "2020-11-23 20:33:09+00:00",
            "title": "Using Machine Learning and Natural Language Processing Techniques to Analyze and Support Moderation of Student Book Discussions",
            "authors": [
                "Jernej Vivod"
            ],
            "abstract": "The increasing adoption of technology to augment or even replace traditional\nface-to-face learning has led to the development of a myriad of tools and\nplatforms aimed at engaging the students and facilitating the teacher's ability\nto present new information. The IMapBook project aims at improving the literacy\nand reading comprehension skills of elementary school-aged children by\npresenting them with interactive e-books and letting them take part in\nmoderated book discussions. This study aims to develop and illustrate a machine\nlearning-based approach to message classification that could be used to\nautomatically notify the discussion moderator of a possible need for an\nintervention and also to collect other useful information about the ongoing\ndiscussion. We aim to predict whether a message posted in the discussion is\nrelevant to the discussed book, whether the message is a statement, a question,\nor an answer, and in which broad category it can be classified. We\nincrementally enrich our used feature subsets and compare them using standard\nclassification algorithms as well as the novel Feature stacking method. We use\nstandard classification performance metrics as well as the Bayesian correlated\nt-test to show that the use of described methods in discussion moderation is\nfeasible. Moving forward, we seek to attain better performance by focusing on\nextracting more of the significant information found in the strong temporal\ninterdependence of the messages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11773v1",
            "date": "2020-11-23 22:30:32+00:00",
            "title": "Advancing Humor-Focused Sentiment Analysis through Improved Contextualized Embeddings and Model Architecture",
            "authors": [
                "Felipe Godoy"
            ],
            "abstract": "Humor is a natural and fundamental component of human interactions. When\ncorrectly applied, humor allows us to express thoughts and feelings\nconveniently and effectively, increasing interpersonal affection, likeability,\nand trust. However, understanding the use of humor is a computationally\nchallenging task from the perspective of humor-aware language processing\nmodels. As language models become ubiquitous through virtual-assistants and IOT\ndevices, the need to develop humor-aware models rises exponentially. To further\nimprove the state-of-the-art capacity to perform this particular\nsentiment-analysis task we must explore models that incorporate contextualized\nand nonverbal elements in their design. Ideally, we seek architectures\naccepting non-verbal elements as additional embedded inputs to the model,\nalongside the original sentence-embedded input. This survey thus analyses the\ncurrent state of research in techniques for improved contextualized embedding\nincorporating nonverbal information, as well as newly proposed deep\narchitectures to improve context retention on top of popular word-embeddings\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11807v1",
            "date": "2020-11-24 00:28:22+00:00",
            "title": "Acoustic span embeddings for multilingual query-by-example search",
            "authors": [
                "Yushi Hu",
                "Shane Settle",
                "Karen Livescu"
            ],
            "abstract": "Query-by-example (QbE) speech search is the task of matching spoken queries\nto utterances within a search collection. In low- or zero-resource settings,\nQbE search is often addressed with approaches based on dynamic time warping\n(DTW). Recent work has found that methods based on acoustic word embeddings\n(AWEs) can improve both performance and search speed. However, prior work on\nAWE-based QbE has primarily focused on English data and with single-word\nqueries. In this work, we generalize AWE training to spans of words, producing\nacoustic span embeddings (ASE), and explore the application of ASE to QbE with\narbitrary-length queries in multiple unseen languages. We consider the commonly\nused setting where we have access to labeled data in other languages (in our\ncase, several low-resource languages) distinct from the unseen test languages.\nWe evaluate our approach on the QUESST 2015 QbE tasks, finding that\nmultilingual ASE-based search is much faster than DTW-based search and\noutperforms the best previously published results on this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11928v3",
            "date": "2020-11-24 06:59:45+00:00",
            "title": "GLGE: A New General Language Generation Evaluation Benchmark",
            "authors": [
                "Dayiheng Liu",
                "Yu Yan",
                "Yeyun Gong",
                "Weizhen Qi",
                "Hang Zhang",
                "Jian Jiao",
                "Weizhu Chen",
                "Jie Fu",
                "Linjun Shou",
                "Ming Gong",
                "Pengcheng Wang",
                "Jiusheng Chen",
                "Daxin Jiang",
                "Jiancheng Lv",
                "Ruofei Zhang",
                "Winnie Wu",
                "Ming Zhou",
                "Nan Duan"
            ],
            "abstract": "Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress\nof pretraining and transfer learning in Natural Language Processing (NLP).\nThese benchmarks mostly focus on a range of Natural Language Understanding\n(NLU) tasks, without considering the Natural Language Generation (NLG) models.\nIn this paper, we present the General Language Generation Evaluation (GLGE), a\nnew multi-task benchmark for evaluating the generalization capabilities of NLG\nmodels across eight language generation tasks. For each task, we continue to\ndesign three subtasks in terms of task difficulty (GLGE-Easy, GLGE-Medium, and\nGLGE-Hard). This introduces 24 subtasks to comprehensively compare model\nperformance. To encourage research on pretraining and transfer learning on NLG\nmodels, we make GLGE publicly available and build a leaderboard with strong\nbaselines including MASS, BART, and ProphetNet (The source code and dataset are\npublicly available at https://github.com/microsoft/glge).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12073v1",
            "date": "2020-11-24 13:19:06+00:00",
            "title": "Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis",
            "authors": [
                "Michael A. Lepori",
                "R. Thomas McCoy"
            ],
            "abstract": "As the name implies, contextualized representations of language are typically\nmotivated by their ability to encode context. Which aspects of context are\ncaptured by such representations? We introduce an approach to address this\nquestion using Representational Similarity Analysis (RSA). As case studies, we\ninvestigate the degree to which a verb embedding encodes the verb's subject, a\npronoun embedding encodes the pronoun's antecedent, and a full-sentence\nrepresentation encodes the sentence's head word (as determined by a dependency\nparse). In all cases, we show that BERT's contextualized embeddings reflect the\nlinguistic dependency being studied, and that BERT encodes these dependencies\nto a greater degree than it encodes less linguistically-salient controls. These\nresults demonstrate the ability of our approach to adjudicate between\nhypotheses about which aspects of context are encoded in representations of\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12086v1",
            "date": "2020-11-24 13:45:14+00:00",
            "title": "Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysis",
            "authors": [
                "Michael A. Lepori"
            ],
            "abstract": "We present a new approach for detecting human-like social biases in word\nembeddings using representational similarity analysis. Specifically, we probe\ncontextualized and non-contextualized embeddings for evidence of intersectional\nbiases against Black women. We show that these embeddings represent Black women\nas simultaneously less feminine than White women, and less Black than Black\nmen. This finding aligns with intersectionality theory, which argues that\nmultiple identity categories (such as race or sex) layer on top of each other\nin order to create unique modes of discrimination that are not shared by any\nindividual category.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12170v1",
            "date": "2020-11-24 15:45:52+00:00",
            "title": "Domain-Transferable Method for Named Entity Recognition Task",
            "authors": [
                "Vladislav Mikhailov",
                "Tatiana Shavrina"
            ],
            "abstract": "Named Entity Recognition (NER) is a fundamental task in the fields of natural\nlanguage processing and information extraction. NER has been widely used as a\nstandalone tool or an essential component in a variety of applications such as\nquestion answering, dialogue assistants and knowledge graphs development.\nHowever, training reliable NER models requires a large amount of labelled data\nwhich is expensive to obtain, particularly in specialized domains. This paper\ndescribes a method to learn a domain-specific NER model for an arbitrary set of\nnamed entities when domain-specific supervision is not available. We assume\nthat the supervision can be obtained with no human effort, and neural models\ncan learn from each other. The code, data and models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12183v1",
            "date": "2020-11-24 16:02:36+00:00",
            "title": "Generating Intelligible Plumitifs Descriptions: Use Case Application with Ethical Considerations",
            "authors": [
                "David Beauchemin",
                "Nicolas Garneau",
                "Eve Gaumond",
                "Pierre-Luc D\u00e9ziel",
                "Richard Khoury",
                "Luc Lamontagne"
            ],
            "abstract": "Plumitifs (dockets) were initially a tool for law clerks. Nowadays, they are\nused as summaries presenting all the steps of a judicial case. Information\nconcerning parties' identity, jurisdiction in charge of administering the case,\nand some information relating to the nature and the course of the preceding are\navailable through plumitifs. They are publicly accessible but barely\nunderstandable; they are written using abbreviations and referring to\nprovisions from the Criminal Code of Canada, which makes them hard to reason\nabout. In this paper, we propose a simple yet efficient multi-source language\ngeneration architecture that leverages both the plumitif and the Criminal\nCode's content to generate intelligible plumitifs descriptions. It goes without\nsaying that ethical considerations rise with these sensitive documents made\nreadable and available at scale, legitimate concerns that we address in this\npaper.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12184v1",
            "date": "2020-11-24 16:07:18+00:00",
            "title": "Neural Text Classification by Jointly Learning to Cluster and Align",
            "authors": [
                "Yekun Chai",
                "Haidong Zhang",
                "Shuo Jin"
            ],
            "abstract": "Distributional text clustering delivers semantically informative\nrepresentations and captures the relevance between each word and semantic\nclustering centroids. We extend the neural text clustering approach to text\nclassification tasks by inducing cluster centers via a latent variable model\nand interacting with distributional word embeddings, to enrich the\nrepresentation of tokens and measure the relatedness between tokens and each\nlearnable cluster centroid. The proposed method jointly learns word clustering\ncentroids and clustering-token alignments, achieving the state of the art\nresults on multiple benchmark datasets and proving that the proposed\ncluster-token alignment mechanism is indeed favorable to text classification.\nNotably, our qualitative analysis has conspicuously illustrated that text\nrepresentations learned by the proposed model are in accord well with our\nintuition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12249v1",
            "date": "2020-11-24 17:45:03+00:00",
            "title": "Cross-Document Event Coreference Resolution Beyond Corpus-Tailored Systems",
            "authors": [
                "Michael Bugert",
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "Cross-document event coreference resolution (CDCR) is an NLP task in which\nmentions of events need to be identified and clustered throughout a collection\nof documents. CDCR aims to benefit downstream multi-document applications, but\ndespite recent progress on corpora and model development, downstream\nimprovements from applying CDCR have not been shown yet. The reason lies in the\nfact that every CDCR system released to date was developed, trained, and tested\nonly on a single respective corpus. This raises strong concerns on their\ngeneralizability --- a must-have for downstream applications where the\nmagnitude of domains or event mentions is likely to exceed those found in a\ncurated corpus. To approach this issue, we define a uniform evaluation setup\ninvolving three CDCR corpora: ECB+, the Gun Violence Corpus and the Football\nCoreference Corpus (which we reannotate on token level to make our analysis\npossible). We compare a corpus-independent, feature-based system against a\nrecent neural system developed for ECB+. Whilst being inferior in absolute\nnumbers, the feature-based system shows more consistent performance across all\ncorpora whereas the neural system is hit-and-miss. Via model introspection, we\nfind that the importance of event actions, event time, etc. for resolving\ncoreference in practice varies greatly between the corpora. Additional analysis\nshows that several systems overfit on the structure of the ECB+ corpus. We\nconclude with recommendations on how to move beyond corpus-tailored CDCR\nsystems in the future -- the most important being that evaluation on multiple\nCDCR corpora is strongly necessary. To facilitate future research, we release\nour dataset, annotation guidelines, and model implementation to the public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12265v1",
            "date": "2020-11-24 18:20:24+00:00",
            "title": "A Pattern-mining Driven Study on Differences of Newspapers in Expressing Temporal Information",
            "authors": [
                "Yingxue Fu",
                "Elaine Ui Dhonnchadha"
            ],
            "abstract": "This paper studies the differences between different types of newspapers in\nexpressing temporal information, which is a topic that has not received much\nattention. Techniques from the fields of temporal processing and pattern mining\nare employed to investigate this topic. First, a corpus annotated with temporal\ninformation is created by the author. Then, sequences of temporal information\ntags mixed with part-of-speech tags are extracted from the corpus. The TKS\nalgorithm is used to mine skip-gram patterns from the sequences. With these\npatterns, the signatures of the four newspapers are obtained. In order to make\nthe signatures uniquely characterize the newspapers, we revise the signatures\nby removing reference patterns. Through examining the number of patterns in the\nsignatures and revised signatures, the proportion of patterns containing\ntemporal information tags and the specific patterns containing temporal\ninformation tags, it is found that newspapers differ in ways of expressing\ntemporal information.",
            "journal ref": "David C. Wyld et al. (Eds): NLP, JSE, MLTEC, DMS, NeTIOT, ITCS,\n  SIP, CST, ARIA - 2020 pp. 111-129, 2020. CS & IT - CSCP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12432v1",
            "date": "2020-11-24 22:35:44+00:00",
            "title": "Enhancing deep neural networks with morphological information",
            "authors": [
                "Matej Klemen",
                "Luka Krsnik",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "Currently, deep learning approaches are superior in natural language\nprocessing due to their ability to extract informative features and patterns\nfrom languages. Two most successful neural architectures are LSTM and\ntransformers, the latter mostly used in the form of large pretrained language\nmodels such as BERT. While cross-lingual approaches are on the rise, a vast\nmajority of current natural language processing techniques is designed and\napplied to English, and less-resourced languages are lagging behind. In\nmorphologically rich languages, plenty of information is conveyed through\nchanges in morphology, e.g., through different prefixes and suffixes modifying\nstems of words. The existing neural approaches do not explicitly use the\ninformation on word morphology. We analyze the effect of adding morphological\nfeatures to LSTM and BERT models. We use three tasks available in many\nless-resourced languages: named entity recognition (NER), dependency parsing\n(DP), and comment filtering (CF). We construct sensible baselines involving\nLSTM and BERT models, which we adjust by adding additional input in the form of\npart of speech (POS) tags and universal features. We compare the obtained\nmodels across subsets of eight languages. Our results suggest that adding\nmorphological features has mixed effects depending on the quality of features\nand the task. The features improve the performance of LSTM-based models on the\nNER and DP tasks, while they do not benefit the performance on the CF task. For\nBERT-based models, the added morphological features only improve the\nperformance on DP when they are of high quality, while they do not show any\npractical improvement when they are predicted. As in NER and CF datasets\nmanually checked features are not available, we only experiment with the\npredicted morphological features and find that they do not cause any practical\nimprovement in performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12631v2",
            "date": "2020-11-25 10:45:38+00:00",
            "title": "A Panoramic Survey of Natural Language Processing in the Arab World",
            "authors": [
                "Kareem Darwish",
                "Nizar Habash",
                "Mourad Abbas",
                "Hend Al-Khalifa",
                "Huseein T. Al-Natsheh",
                "Samhaa R. El-Beltagy",
                "Houda Bouamor",
                "Karim Bouzoubaa",
                "Violetta Cavalli-Sforza",
                "Wassim El-Hajj",
                "Mustafa Jarrar",
                "Hamdy Mubarak"
            ],
            "abstract": "The term natural language refers to any system of symbolic communication\n(spoken, signed or written) without intentional human planning and design. This\ndistinguishes natural languages such as Arabic and Japanese from artificially\nconstructed languages such as Esperanto or Python. Natural language processing\n(NLP) is the sub-field of artificial intelligence (AI) focused on modeling\nnatural languages to build applications such as speech recognition and\nsynthesis, machine translation, optical character recognition (OCR), sentiment\nanalysis (SA), question answering, dialogue systems, etc. NLP is a highly\ninterdisciplinary field with connections to computer science, linguistics,\ncognitive science, psychology, mathematics and others. Some of the earliest AI\napplications were in NLP (e.g., machine translation); and the last decade\n(2010-2020) in particular has witnessed an incredible increase in quality,\nmatched with a rise in public awareness, use, and expectations of what may have\nseemed like science fiction in the past. NLP researchers pride themselves on\ndeveloping language independent models and tools that can be applied to all\nhuman languages, e.g. machine translation systems can be built for a variety of\nlanguages using the same basic mechanisms and models. However, the reality is\nthat some languages do get more attention (e.g., English and Chinese) than\nothers (e.g., Hindi and Swahili). Arabic, the primary language of the Arab\nworld and the religious language of millions of non-Arab Muslims is somewhere\nin the middle of this continuum. Though Arabic NLP has many challenges, it has\nseen many successes and developments. Next we discuss Arabic's main challenges\nas a necessary background, and we present a brief history of Arabic NLP. We\nthen survey a number of its research areas, and close with a critical\ndiscussion of the future of Arabic NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12771v1",
            "date": "2020-11-25 14:33:18+00:00",
            "title": "Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations",
            "authors": [
                "Haojie Pan",
                "Cen Chen",
                "Minghui Qiu",
                "Liu Yang",
                "Feng Ji",
                "Jun Huang",
                "Haiqing Chen"
            ],
            "abstract": "Intelligent personal assistant systems for information-seeking conversations\nare increasingly popular in real-world applications, especially for e-commerce\ncompanies. With the development of research in such conversation systems, the\npseudo-relevance feedback (PRF) has demonstrated its effectiveness in\nincorporating relevance signals from external documents. However, the existing\nstudies are either based on heuristic rules or require heavy manual labeling.\nIn this work, we treat the PRF selection as a learning task and proposed a\nreinforced learning based method that can be trained in an end-to-end manner\nwithout any human annotations. More specifically, we proposed a reinforced\nselector to extract useful PRF terms to enhance response candidates and a BERT\nbased response ranker to rank the PRF-enhanced responses. The performance of\nthe ranker serves as rewards to guide the selector to extract useful PRF terms,\nand thus boost the task performance. Extensive experiments on both standard\nbenchmark and commercial datasets show the superiority of our reinforced PRF\nterm selector compared with other potential soft or hard selection methods.\nBoth qualitative case studies and quantitative analysis show that our model can\nnot only select meaningful PRF terms to expand response candidates but also\nachieve the best results compared with all the baseline methods on a variety of\nevaluation metrics. We have also deployed our method on online production in an\ne-commerce company, which shows a significant improvement over the existing\nonline ranking system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13100v1",
            "date": "2020-11-26 02:58:24+00:00",
            "title": "Automatic Distractor Generation for Multiple Choice Questions in Standard Tests",
            "authors": [
                "Zhaopeng Qiu",
                "Xian Wu",
                "Wei Fan"
            ],
            "abstract": "To assess the knowledge proficiency of a learner, multiple choice question is\nan efficient and widespread form in standard tests. However, the composition of\nthe multiple choice question, especially the construction of distractors is\nquite challenging. The distractors are required to both incorrect and plausible\nenough to confuse the learners who did not master the knowledge. Currently, the\ndistractors are generated by domain experts which are both expensive and\ntime-consuming. This urges the emergence of automatic distractor generation,\nwhich can benefit various standard tests in a wide range of domains. In this\npaper, we propose a question and answer guided distractor generation (EDGE)\nframework to automate distractor generation. EDGE consists of three major\nmodules: (1) the Reforming Question Module and the Reforming Passage Module\napply gate layers to guarantee the inherent incorrectness of the generated\ndistractors; (2) the Distractor Generator Module applies attention mechanism to\ncontrol the level of plausibility. Experimental results on a large-scale public\ndataset demonstrate that our model significantly outperforms existing models\nand achieves a new state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13115v1",
            "date": "2020-11-26 03:57:56+00:00",
            "title": "Learning Causal Bayesian Networks from Text",
            "authors": [
                "Farhad Moghimifar",
                "Afshin Rahimi",
                "Mahsa Baktashmotlagh",
                "Xue Li"
            ],
            "abstract": "Causal relationships form the basis for reasoning and decision-making in\nArtificial Intelligence systems. To exploit the large volume of textual data\navailable today, the automatic discovery of causal relationships from text has\nemerged as a significant challenge in recent years. Existing approaches in this\nrealm are limited to the extraction of low-level relations among individual\nevents. To overcome the limitations of the existing approaches, in this paper,\nwe propose a method for automatic inference of causal relationships from human\nwritten language at conceptual level. To this end, we leverage the\ncharacteristics of hierarchy of concepts and linguistic variables created from\ntext, and represent the extracted causal relationships in the form of a Causal\nBayesian Network. Our experiments demonstrate superiority of our approach over\nthe existing approaches in inferring complex causal reasoning from the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13187v2",
            "date": "2020-11-26 09:04:07+00:00",
            "title": "Transformer-Based Models for Automatic Identification of Argument Relations: A Cross-Domain Evaluation",
            "authors": [
                "Ramon Ruiz-Dolz",
                "Stella Heras",
                "Jose Alemany",
                "Ana Garc\u00eda-Fornes"
            ],
            "abstract": "Argument Mining is defined as the task of automatically identifying and\nextracting argumentative components (e.g., premises, claims, etc.) and\ndetecting the existing relations among them (i.e., support, attack, rephrase,\nno relation). One of the main issues when approaching this problem is the lack\nof data, and the size of the publicly available corpora. In this work, we use\nthe recently annotated US2016 debate corpus. US2016 is the largest existing\nargument annotated corpus, which allows exploring the benefits of the most\nrecent advances in Natural Language Processing in a complex domain like\nArgument (relation) Mining. We present an exhaustive analysis of the behavior\nof transformer-based models (i.e., BERT, XLNET, RoBERTa, DistilBERT and ALBERT)\nwhen predicting argument relations. Finally, we evaluate the models in five\ndifferent domains, with the objective of finding the less domain dependent\nmodel. We obtain a macro F1-score of 0.70 with the US2016 evaluation corpus,\nand a macro F1-score of 0.61 with the Moral Maze cross-domain corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13354v3",
            "date": "2020-11-26 15:36:06+00:00",
            "title": "Braid: Weaving Symbolic and Neural Knowledge into Coherent Logical Explanations",
            "authors": [
                "Aditya Kalyanpur",
                "Tom Breloff",
                "David Ferrucci",
                "Adam Lally",
                "John Jantos"
            ],
            "abstract": "Traditional symbolic reasoning engines, while attractive for their precision\nand explicability, have a few major drawbacks: the use of brittle inference\nprocedures that rely on exact matching (unification) of logical terms, an\ninability to deal with uncertainty, and the need for a precompiled rule-base of\nknowledge (the \"knowledge acquisition\" problem). These issues are particularly\nsevere for the Natural Language Understanding (NLU) task, where we often use\nimplicit background knowledge to understand and reason about text, resort to\nfuzzy alignment of concepts and relations during reasoning, and constantly deal\nwith ambiguity in representations. To address these issues, we devise a novel\nFOL-based reasoner, called Braid, that supports probabilistic rules, and uses\nthe notion of custom unification functions and dynamic rule generation to\novercome the brittle matching and knowledge-gap problem prevalent in\ntraditional reasoners. In this paper, we describe the reasoning algorithms used\nin Braid-BC (the backchaining component of Braid), and their implementation in\na distributed task-based framework that builds proof/explanation graphs for an\ninput query in a scalable manner. We use a simple QA example from a children's\nstory to motivate Braid-BC's design and explain how the various components work\ntogether to produce a coherent logical explanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13549v1",
            "date": "2020-11-27 04:14:55+00:00",
            "title": "Domain Adaptative Causality Encoder",
            "authors": [
                "Farhad Moghimifar",
                "Gholamreza Haffari",
                "Mahsa Baktashmotlagh"
            ],
            "abstract": "Current approaches which are mainly based on the extraction of low-level\nrelations among individual events are limited by the shortage of publicly\navailable labelled data. Therefore, the resulting models perform poorly when\napplied to a distributionally different domain for which labelled data did not\nexist at the time of training. To overcome this limitation, in this paper, we\nleverage the characteristics of dependency trees and adversarial learning to\naddress the tasks of adaptive causality identification and localisation. The\nterm adaptive is used since the training and test data come from two\ndistributionally different datasets, which to the best of our knowledge, this\nwork is the first to address. Moreover, we present a new causality dataset,\nnamely MedCaus, which integrates all types of causality in the text. Our\nexperiments on four different benchmark causality datasets demonstrate the\nsuperiority of our approach over the existing baselines, by up to 7%\nimprovement, on the tasks of identification and localisation of the causal\nrelations from the text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13573v1",
            "date": "2020-11-27 06:13:56+00:00",
            "title": "Chinese Medical Question Answer Matching Based on Interactive Sentence Representation Learning",
            "authors": [
                "Xiongtao Cui",
                "Jungang Han"
            ],
            "abstract": "Chinese medical question-answer matching is more challenging than the\nopen-domain question answer matching in English. Even though the deep learning\nmethod has performed well in improving the performance of question answer\nmatching, these methods only focus on the semantic information inside\nsentences, while ignoring the semantic association between questions and\nanswers, thus resulting in performance deficits. In this paper, we design a\nseries of interactive sentence representation learning models to tackle this\nproblem. To better adapt to Chinese medical question-answer matching and take\nthe advantages of different neural network structures, we propose the Crossed\nBERT network to extract the deep semantic information inside the sentence and\nthe semantic association between question and answer, and then combine with the\nmulti-scale CNNs network or BiGRU network to take the advantage of different\nstructure of neural networks to learn more semantic features into the sentence\nrepresentation. The experiments on the cMedQA V2.0 and cMedQA V1.0 dataset show\nthat our model significantly outperforms all the existing state-of-the-art\nmodels of Chinese medical question answer matching.",
            "journal ref": "pp. 93-109, 2020. CS & IT - CSCP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13633v2",
            "date": "2020-11-27 09:49:37+00:00",
            "title": "CoRe: An Efficient Coarse-refined Training Framework for BERT",
            "authors": [
                "Cheng Yang",
                "Shengnan Wang",
                "Yuechuan Li",
                "Chao Yang",
                "Ming Yan",
                "Jingqiao Zhang",
                "Fangquan Lin"
            ],
            "abstract": "In recent years, BERT has made significant breakthroughs on many natural\nlanguage processing tasks and attracted great attentions. Despite its accuracy\ngains, the BERT model generally involves a huge number of parameters and needs\nto be trained on massive datasets, so training such a model is computationally\nvery challenging and time-consuming. Hence, training efficiency should be a\ncritical issue. In this paper, we propose a novel coarse-refined training\nframework named CoRe to speed up the training of BERT. Specifically, we\ndecompose the training process of BERT into two phases. In the first phase, by\nintroducing fast attention mechanism and decomposing the large parameters in\nthe feed-forward network sub-layer, we construct a relaxed BERT model which has\nmuch less parameters and much lower model complexity than the original BERT, so\nthe relaxed model can be quickly trained. In the second phase, we transform the\ntrained relaxed BERT model into the original BERT and further retrain the\nmodel. Thanks to the desired initialization provided by the relaxed model, the\nretraining phase requires much less training steps, compared with training an\noriginal BERT model from scratch with a random initialization. Experimental\nresults show that the proposed CoRe framework can greatly reduce the training\ntime without reducing the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13635v1",
            "date": "2020-11-27 10:00:22+00:00",
            "title": "Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup",
            "authors": [
                "Cheng Yang",
                "Shengnan Wang",
                "Chao Yang",
                "Yuechuan Li",
                "Ru He",
                "Jingqiao Zhang"
            ],
            "abstract": "Pre-trained language models, such as BERT, have achieved significant accuracy\ngain in many natural language processing tasks. Despite its effectiveness, the\nhuge number of parameters makes training a BERT model computationally very\nchallenging. In this paper, we propose an efficient multi-stage layerwise\ntraining (MSLT) approach to reduce the training time of BERT. We decompose the\nwhole training process into several stages. The training is started from a\nsmall model with only a few encoder layers and we gradually increase the depth\nof the model by adding new encoder layers. At each stage, we only train the top\n(near the output layer) few encoder layers which are newly added. The\nparameters of the other layers which have been trained in the previous stages\nwill not be updated in the current stage. In BERT training, the backward\ncomputation is much more time-consuming than the forward computation,\nespecially in the distributed training setting in which the backward\ncomputation time further includes the communication time for gradient\nsynchronization. In the proposed training strategy, only top few layers\nparticipate in backward computation, while most layers only participate in\nforward computation. Hence both the computation and communication efficiencies\nare greatly improved. Experimental results show that the proposed method can\nachieve more than 110% training speedup without significant performance\ndegradation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13662v1",
            "date": "2020-11-27 10:57:18+00:00",
            "title": "FFCI: A Framework for Interpretable Automatic Evaluation of Summarization",
            "authors": [
                "Fajri Koto",
                "Jey Han Lau",
                "Timothy Baldwin"
            ],
            "abstract": "In this paper, we propose FFCI, a framework for automatic summarization\nevaluation that comprises four elements: Faithfulness, Focus, Coverage, and\nInter-sentential coherence. We design FFCI by comprehensively studying\ntraditional evaluation metrics and model-based evaluations, including question\nanswering (QA) approaches, STS, next-sentence prediction (NSP), and scores from\n19 pre-trained language models. Our study reveals three key findings: (1)\ncalculating BertSCORE between the summary and article sentences yields a higher\ncorrelation score than recently-proposed QA-based evaluation methods for\nfaithfulness evaluation; (2) GPT2Score has the best Pearson's correlation for\nfocus and coverage; and (3) a simple NSP model is effective at evaluating\ninter-sentential coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14037v1",
            "date": "2020-11-28 00:44:35+00:00",
            "title": "Text Mining for Processing Interview Data in Computational Social Science",
            "authors": [
                "Jussi Karlgren",
                "Renee Li",
                "Eva M Meyersson Milgrom"
            ],
            "abstract": "We use commercially available text analysis technology to process interview\ntext data from a computational social science study. We find that topical\nclustering and terminological enrichment provide for convenient exploration and\nquantification of the responses. This makes it possible to generate and test\nhypotheses and to compare textual and non-textual variables, and saves analyst\neffort. We encourage studies in social science to use text analysis, especially\nfor exploratory open-ended studies. We discuss how replicability requirements\nare met by text analysis technology. We note that the most recent learning\nmodels are not designed with transparency in mind, and that research requires a\nmodel to be editable and its decisions to be explainable. The tools available\ntoday, such as the one used in the present study, are not built for processing\ninterview texts. While many of the variables under consideration are\nquantifiable using lexical statistics, we find that some interesting and\npotentially valuable features are difficult or impossible to automatise\nreliably at present. We note that there are some potentially interesting\napplications for traditional natural language processing mechanisms such as\nnamed entity recognition and anaphora resolution in this application area. We\nconclude with a suggestion for language technologists to investigate the\nchallenge of processing interview data comprehensively, especially the\ninterplay between question and response, and we encourage social science\nresearchers not to hesitate to use text analysis tools, especially for the\nexploratory phase of processing interview data.?",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14039v1",
            "date": "2020-11-28 00:46:43+00:00",
            "title": "Understanding How BERT Learns to Identify Edits",
            "authors": [
                "Samuel Stevens",
                "Yu Su"
            ],
            "abstract": "Pre-trained transformer language models such as BERT are ubiquitous in NLP\nresearch, leading to work on understanding how and why these models work.\nAttention mechanisms have been proposed as a means of interpretability with\nvarying conclusions. We propose applying BERT-based models to a sequence\nclassification task and using the data set's labeling schema to measure each\nmodel's interpretability. We find that classification performance scores do not\nalways correlate with interpretability. Despite this, BERT's attention weights\nare interpretable for over 70% of examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14190v1",
            "date": "2020-11-28 18:44:52+00:00",
            "title": "Using Multiple Subwords to Improve English-Esperanto Automated Literary Translation Quality",
            "authors": [
                "Alberto Poncelas",
                "Jan Buts",
                "James Hadley",
                "Andy Way"
            ],
            "abstract": "Building Machine Translation (MT) systems for low-resource languages remains\nchallenging. For many language pairs, parallel data are not widely available,\nand in such cases MT models do not achieve results comparable to those seen\nwith high-resource languages.\n  When data are scarce, it is of paramount importance to make optimal use of\nthe limited material available. To that end, in this paper we propose employing\nthe same parallel sentences multiple times, only changing the way the words are\nsplit each time. For this purpose we use several Byte Pair Encoding models,\nwith various merge operations used in their configuration.\n  In our experiments, we use this technique to expand the available data and\nimprove an MT system involving a low-resource language pair, namely\nEnglish-Esperanto.\n  As an additional contribution, we made available a set of English-Esperanto\nparallel data in the literary domain.",
            "journal ref": "The 3rd Workshop on Technologies for MT of Low Resource Languages\n  (LoResMT 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14277v1",
            "date": "2020-11-29 04:34:39+00:00",
            "title": "Intrinsic Knowledge Evaluation on Chinese Language Models",
            "authors": [
                "Zhiruo Wang",
                "Renfen Hu"
            ],
            "abstract": "Recent NLP tasks have benefited a lot from pre-trained language models (LM)\nsince they are able to encode knowledge of various aspects. However, current LM\nevaluations focus on downstream performance, hence lack to comprehensively\ninspect in which aspect and to what extent have they encoded knowledge. This\npaper addresses both queries by proposing four tasks on syntactic, semantic,\ncommonsense, and factual knowledge, aggregating to a total of $39,308$\nquestions covering both linguistic and world knowledge in Chinese. Throughout\nexperiments, our probes and knowledge data prove to be a reliable benchmark for\nevaluating pre-trained Chinese LMs. Our work is publicly available at\nhttps://github.com/ZhiruoWang/ChnEval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14293v1",
            "date": "2020-11-29 05:54:03+00:00",
            "title": "Inflating Topic Relevance with Ideology: A Case Study of Political Ideology Bias in Social Topic Detection Models",
            "authors": [
                "Meiqi Guo",
                "Rebecca Hwa",
                "Yu-Ru Lin",
                "Wen-Ting Chung"
            ],
            "abstract": "We investigate the impact of political ideology biases in training data.\nThrough a set of comparison studies, we examine the propagation of biases in\nseveral widely-used NLP models and its effect on the overall retrieval\naccuracy. Our work highlights the susceptibility of large, complex models to\npropagating the biases from human-selected input, which may lead to a\ndeterioration of retrieval accuracy, and the importance of controlling for\nthese biases. Finally, as a way to mitigate the bias, we propose to learn a\ntext representation that is invariant to political ideology while still judging\ntopic relevance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14344v1",
            "date": "2020-11-29 11:36:13+00:00",
            "title": "Generative Pre-training for Paraphrase Generation by Representing and Predicting Spans in Exemplars",
            "authors": [
                "Tien-Cuong Bui",
                "Van-Duc Le",
                "Hai-Thien To",
                "Sang Kyun Cha"
            ],
            "abstract": "Paraphrase generation is a long-standing problem and serves an essential role\nin many natural language processing problems. Despite some encouraging results,\nrecent methods either confront the problem of favoring generic utterance or\nneed to retrain the model from scratch for each new dataset. This paper\npresents a novel approach to paraphrasing sentences, extended from the GPT-2\nmodel. We develop a template masking technique, named first-order masking, to\nmasked out irrelevant words in exemplars utilizing POS taggers. So that, the\nparaphrasing task is changed to predicting spans in masked templates. Our\nproposed approach outperforms competitive baselines, especially in the semantic\npreservation aspect. To prevent the model from being biased towards a given\ntemplate, we introduce a technique, referred to as second-order masking, which\nutilizes Bernoulli distribution to control the visibility of the\nfirst-order-masked template's tokens. Moreover, this technique allows the model\nto provide various paraphrased sentences in testing by adjusting the\nsecond-order-masking level. For scale-up objectives, we compare the performance\nof two alternatives template-selection methods, which shows that they were\nequivalent in preserving semantic information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14459v1",
            "date": "2020-11-29 22:51:25+00:00",
            "title": "Improved Semantic Role Labeling using Parameterized Neighborhood Memory Adaptation",
            "authors": [
                "Ishan Jindal",
                "Ranit Aharonov",
                "Siddhartha Brahma",
                "Huaiyu Zhu",
                "Yunyao Li"
            ],
            "abstract": "Deep neural models achieve some of the best results for semantic role\nlabeling. Inspired by instance-based learning that utilizes nearest neighbors\nto handle low-frequency context-specific training samples, we investigate the\nuse of memory adaptation techniques in deep neural models. We propose a\nparameterized neighborhood memory adaptive (PNMA) method that uses a\nparameterized representation of the nearest neighbors of tokens in a memory of\nactivations and makes predictions based on the most similar samples in the\ntraining data. We empirically show that PNMA consistently improves the SRL\nperformance of the base model irrespective of types of word embeddings. Coupled\nwith contextualized word embeddings derived from BERT, PNMA improves over\nexisting models for both span and dependency semantic parsing datasets,\nespecially on out-of-domain text, reaching F1 scores of 80.2, and 84.97 on\nCoNLL2005, and CoNLL2009 datasets, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14489v2",
            "date": "2020-11-30 01:22:05+00:00",
            "title": "Modelling Verbal Morphology in Nen",
            "authors": [
                "Saliha Murado\u011flu",
                "Nicholas Evans",
                "Ekaterina Vylomova"
            ],
            "abstract": "Nen verbal morphology is remarkably complex; a transitive verb can take up to\n1,740 unique forms. The combined effect of having a large combinatoric space\nand a low-resource setting amplifies the need for NLP tools. Nen morphology\nutilises distributed exponence - a non-trivial means of mapping form to\nmeaning. In this paper, we attempt to model Nen verbal morphology using\nstate-of-the-art machine learning models for morphological reinflection. We\nexplore and categorise the types of errors these systems generate. Our results\nshow sensitivity to training data composition; different distributions of verb\ntype yield different accuracies (patterning with E-complexity). We also\ndemonstrate the types of patterns that can be inferred from the training data\nthrough the case study of syncretism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14608v1",
            "date": "2020-11-30 08:13:41+00:00",
            "title": "Dynamic Curriculum Learning for Low-Resource Neural Machine Translation",
            "authors": [
                "Chen Xu",
                "Bojie Hu",
                "Yufan Jiang",
                "Kai Feng",
                "Zeyang Wang",
                "Shen Huang",
                "Qi Ju",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "Large amounts of data has made neural machine translation (NMT) a big success\nin recent years. But it is still a challenge if we train these models on\nsmall-scale corpora. In this case, the way of using data appears to be more\nimportant. Here, we investigate the effective use of training data for\nlow-resource NMT. In particular, we propose a dynamic curriculum learning (DCL)\nmethod to reorder training samples in training. Unlike previous work, we do not\nuse a static scoring function for reordering. Instead, the order of training\nsamples is dynamically determined in two ways - loss decline and model\ncompetence. This eases training by highlighting easy samples that the current\nmodel has enough competence to learn. We test our DCL method in a\nTransformer-based system. Experimental results show that DCL outperforms\nseveral strong baselines on three low-resource machine translation benchmarks\nand different sized data of WMT' 16 En-De.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14678v1",
            "date": "2020-11-30 10:41:50+00:00",
            "title": "UWB @ DIACR-Ita: Lexical Semantic Change Detection with CCA and Orthogonal Transformation",
            "authors": [
                "Ond\u0159ej Pra\u017e\u00e1k",
                "Pavel P\u0159ib\u00e1\u0148",
                "Stephen Taylor"
            ],
            "abstract": "In this paper, we describe our method for detection of lexical semantic\nchange (i.e., word sense changes over time) for the DIACR-Ita shared task,\nwhere we ranked $1^{st}$. We examine semantic differences between specific\nwords in two Italian corpora, chosen from different time periods. Our method is\nfully unsupervised and language independent. It consists of preparing a\nsemantic vector space for each corpus, earlier and later. Then we compute a\nlinear transformation between earlier and later spaces, using CCA and\nOrthogonal Transformation. Finally, we measure the cosines between the\ntransformed vectors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14874v1",
            "date": "2020-11-30 15:11:51+00:00",
            "title": "A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction",
            "authors": [
                "Yanyang Li",
                "Yingfeng Luo",
                "Ye Lin",
                "Quan Du",
                "Huizhen Wang",
                "Shujian Huang",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "Unsupervised Bilingual Dictionary Induction methods based on the\ninitialization and the self-learning have achieved great success in similar\nlanguage pairs, e.g., English-Spanish. But they still fail and have an accuracy\nof 0% in many distant language pairs, e.g., English-Japanese. In this work, we\nshow that this failure results from the gap between the actual initialization\nperformance and the minimum initialization performance for the self-learning to\nsucceed. We propose Iterative Dimension Reduction to bridge this gap. Our\nexperiments show that this simple method does not hamper the performance of\nsimilar language pairs and achieves an accuracy of 13.64~55.53% between English\nand four distant languages, i.e., Chinese, Japanese, Vietnamese and Thai.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14979v1",
            "date": "2020-11-30 16:51:08+00:00",
            "title": "Machine Translation of Novels in the Age of Transformer",
            "authors": [
                "Antonio Toral",
                "Antoni Oliver",
                "Pau Ribas Ballest\u00edn"
            ],
            "abstract": "In this chapter we build a machine translation (MT) system tailored to the\nliterary domain, specifically to novels, based on the state-of-the-art\narchitecture in neural MT (NMT), the Transformer (Vaswani et al., 2017), for\nthe translation direction English-to-Catalan. Subsequently, we assess to what\nextent such a system can be useful by evaluating its translations, by comparing\nthis MT system against three other systems (two domain-specific systems under\nthe recurrent and phrase-based paradigms and a popular generic on-line system)\non three evaluations. The first evaluation is automatic and uses the\nmost-widely used automatic evaluation metric, BLEU. The two remaining\nevaluations are manual and they assess, respectively, preference and amount of\npost-editing required to make the translation error-free. As expected, the\ndomain-specific Transformer-based system outperformed the three other systems\nin all the three evaluations conducted, in all cases by a large margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00004v1",
            "date": "2020-11-30 10:47:45+00:00",
            "title": "UWB at SemEval-2020 Task 1: Lexical Semantic Change Detection",
            "authors": [
                "Ond\u0159ej Pra\u017e\u00e1k",
                "Pavel P\u0159ib\u00e1\u0148",
                "Stephen Taylor",
                "Jakub Sido"
            ],
            "abstract": "In this paper, we describe our method for the detection of lexical semantic\nchange, i.e., word sense changes over time. We examine semantic differences\nbetween specific words in two corpora, chosen from different time periods, for\nEnglish, German, Latin, and Swedish. Our method was created for the SemEval\n2020 Task 1: \\textit{Unsupervised Lexical Semantic Change Detection.} We ranked\n$1^{st}$ in Sub-task 1: binary change detection, and $4^{th}$ in Sub-task 2:\nranked change detection. Our method is fully unsupervised and language\nindependent. It consists of preparing a semantic vector space for each corpus,\nearlier and later; computing a linear transformation between earlier and later\nspaces, using Canonical Correlation Analysis and Orthogonal Transformation; and\nmeasuring the cosines between the transformed vector for the target word from\nthe earlier corpus and the vector for the target word in the later corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00052v1",
            "date": "2020-11-30 19:07:27+00:00",
            "title": "Systematically Exploring Redundancy Reduction in Summarizing Long Documents",
            "authors": [
                "Wen Xiao",
                "Giuseppe Carenini"
            ],
            "abstract": "Our analysis of large summarization datasets indicates that redundancy is a\nvery serious problem when summarizing long documents. Yet, redundancy reduction\nhas not been thoroughly investigated in neural summarization. In this work, we\nsystematically explore and compare different ways to deal with redundancy when\nsummarizing long documents. Specifically, we organize the existing methods into\ncategories based on when and how the redundancy is considered. Then, in the\ncontext of these categories, we propose three additional methods balancing\nnon-redundancy and importance in a general and flexible way. In a series of\nexperiments, we show that our proposed methods achieve the state-of-the-art\nwith respect to ROUGE scores on two scientific paper datasets, Pubmed and\narXiv, while reducing redundancy significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00209v1",
            "date": "2020-12-01 01:39:38+00:00",
            "title": "High Quality Real-Time Structured Debate Generation",
            "authors": [
                "Eric Bolton",
                "Alex Calderwood",
                "Niles Christensen",
                "Jerome Kafrouni",
                "Iddo Drori"
            ],
            "abstract": "Automatically generating debates is a challenging task that requires an\nunderstanding of arguments and how to negate or support them. In this work we\ndefine debate trees and paths for generating debates while enforcing a high\nlevel structure and grammar. We leverage a large corpus of tree-structured\ndebates that have metadata associated with each argument. We develop a\nframework for generating plausible debates which is agnostic to the sentence\nembedding model. Our results demonstrate the ability to generate debates in\nreal-time on complex topics at a quality that is close to humans, as evaluated\nby the style, content, and strategy metrics used for judging competitive human\ndebates. In the spirit of reproducible research we make our data, models, and\ncode publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00288v1",
            "date": "2020-12-01 06:09:44+00:00",
            "title": "BAN-ABSA: An Aspect-Based Sentiment Analysis dataset for Bengali and it's baseline evaluation",
            "authors": [
                "Mahfuz Ahmed Masum",
                "Sheikh Junayed Ahmed",
                "Ayesha Tasnim",
                "Md Saiful Islam"
            ],
            "abstract": "Due to the breathtaking growth of social media or newspaper user comments,\nonline product reviews comments, sentiment analysis (SA) has captured\nsubstantial interest from the researchers. With the fast increase of domain, SA\nwork aims not only to predict the sentiment of a sentence or document but also\nto give the necessary detail on different aspects of the sentence or document\n(i.e. aspect-based sentiment analysis). A considerable number of datasets for\nSA and aspect-based sentiment analysis (ABSA) have been made available for\nEnglish and other well-known European languages. In this paper, we present a\nmanually annotated Bengali dataset of high quality, BAN-ABSA, which is\nannotated with aspect and its associated sentiment by 3 native Bengali\nspeakers. The dataset consists of 2,619 positive, 4,721 negative and 1,669\nneutral data samples from 9,009 unique comments gathered from some famous\nBengali news portals. In addition, we conducted a baseline evaluation with a\nfocus on deep learning model, achieved an accuracy of 78.75% for aspect term\nextraction and accuracy of 71.08% for sentiment classification. Experiments on\nthe BAN-ABSA dataset show that the CNN model is better in terms of accuracy\nthough Bi-LSTM significantly outperforms CNN model in terms of average\nF1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00366v1",
            "date": "2020-12-01 09:51:23+00:00",
            "title": "An Enhanced Knowledge Injection Model for Commonsense Generation",
            "authors": [
                "Zhihao Fan",
                "Yeyun Gong",
                "Zhongyu Wei",
                "Siyuan Wang",
                "Yameng Huang",
                "Jian Jiao",
                "Xuanjing Huang",
                "Nan Duan",
                "Ruofei Zhang"
            ],
            "abstract": "Commonsense generation aims at generating plausible everyday scenario\ndescription based on a set of provided concepts. Digging the relationship of\nconcepts from scratch is non-trivial, therefore, we retrieve prototypes from\nexternal knowledge to assist the understanding of the scenario for better\ndescription generation. We integrate two additional modules, namely position\nindicator and scaling module, into the pretrained encoder-decoder model for\nprototype modeling to enhance the knowledge injection procedure. We conduct\nexperiment on CommonGen benchmark, and experimental results show that our\nmethod significantly improves the performance on all the metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00413v1",
            "date": "2020-12-01 11:32:56+00:00",
            "title": "CPM: A Large-scale Generative Chinese Pre-trained Language Model",
            "authors": [
                "Zhengyan Zhang",
                "Xu Han",
                "Hao Zhou",
                "Pei Ke",
                "Yuxian Gu",
                "Deming Ye",
                "Yujia Qin",
                "Yusheng Su",
                "Haozhe Ji",
                "Jian Guan",
                "Fanchao Qi",
                "Xiaozhi Wang",
                "Yanan Zheng",
                "Guoyang Zeng",
                "Huanqi Cao",
                "Shengqi Chen",
                "Daixuan Li",
                "Zhenbo Sun",
                "Zhiyuan Liu",
                "Minlie Huang",
                "Wentao Han",
                "Jie Tang",
                "Juanzi Li",
                "Xiaoyan Zhu",
                "Maosong Sun"
            ],
            "abstract": "Pre-trained Language Models (PLMs) have proven to be beneficial for various\ndownstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB\ntraining data, drew a lot of attention due to the capacity of few-shot (even\nzero-shot) learning. However, applying GPT-3 to address Chinese NLP tasks is\nstill challenging, as the training corpus of GPT-3 is primarily English, and\nthe parameters are not publicly available. In this technical report, we release\nthe Chinese Pre-trained Language Model (CPM) with generative pre-training on\nlarge-scale Chinese training data. To the best of our knowledge, CPM, with 2.6\nbillion parameters and 100GB Chinese training data, is the largest Chinese\npre-trained language model, which could facilitate several downstream Chinese\nNLP tasks, such as conversation, essay generation, cloze test, and language\nunderstanding. Extensive experiments demonstrate that CPM achieves strong\nperformance on many NLP tasks in the settings of few-shot (even zero-shot)\nlearning. The code and parameters are available at\nhttps://github.com/TsinghuaAI/CPM-Generate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00571v1",
            "date": "2020-12-01 15:25:47+00:00",
            "title": "Denoising Pre-Training and Data Augmentation Strategies for Enhanced RDF Verbalization with Transformers",
            "authors": [
                "Sebastien Montella",
                "Betty Fabre",
                "Tanguy Urvoy",
                "Johannes Heinecke",
                "Lina Rojas-Barahona"
            ],
            "abstract": "The task of verbalization of RDF triples has known a growth in popularity due\nto the rising ubiquity of Knowledge Bases (KBs). The formalism of RDF triples\nis a simple and efficient way to store facts at a large scale. However, its\nabstract representation makes it difficult for humans to interpret. For this\npurpose, the WebNLG challenge aims at promoting automated RDF-to-text\ngeneration. We propose to leverage pre-trainings from augmented data with the\nTransformer model using a data augmentation strategy. Our experiment results\nshow a minimum relative increases of 3.73%, 126.05% and 88.16% in BLEU score\nfor seen categories, unseen entities and unseen categories respectively over\nthe standard training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00728v2",
            "date": "2020-12-01 18:40:42+00:00",
            "title": "Intrinsic analysis for dual word embedding space models",
            "authors": [
                "Mohit Mayank"
            ],
            "abstract": "Recent word embeddings techniques represent words in a continuous vector\nspace, moving away from the atomic and sparse representations of the past. Each\nsuch technique can further create multiple varieties of embeddings based on\ndifferent settings of hyper-parameters like embedding dimension size, context\nwindow size and training method. One additional variety appears when we\nespecially consider the Dual embedding space techniques which generate not one\nbut two-word embeddings as output. This gives rise to an interesting question -\n\"is there one or a combination of the two word embeddings variety, which works\nbetter for a specific task?\". This paper tries to answer this question by\nconsidering all of these variations. Herein, we compare two classical embedding\nmethods belonging to two different methodologies - Word2Vec from window-based\nand Glove from count-based. For an extensive evaluation after considering all\nvariations, a total of 84 different models were compared against semantic,\nassociation and analogy evaluations tasks which are made up of 9 open-source\nlinguistics datasets. The final Word2vec reports showcase the preference of\nnon-default model for 2 out of 3 tasks. In case of Glove, non-default models\noutperform in all 3 evaluation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00902v1",
            "date": "2020-12-02 00:03:07+00:00",
            "title": "Automatic Extraction of Ranked SNP-Phenotype Associations from Literature through Detecting Neural Candidates, Negation and Modality Markers",
            "authors": [
                "Behrouz Bokharaeian",
                "Alberto Diaz"
            ],
            "abstract": "Genome-wide association (GWA) constitutes a prominent portion of studies\nwhich have been conducted on personalized medicine and pharmacogenomics.\nRecently, very few methods have been developed for extracting mutation-diseases\nassociations. However, there is no available method for extracting the\nassociation of SNP-phenotype from text which considers degree of confidence in\nassociations. In this study, first a relation extraction method relying on\nlinguistic-based negation detection and neutral candidates is proposed. The\nexperiments show that negation cues and scope as well as detecting neutral\ncandidates can be employed for implementing a superior relation extraction\nmethod which outperforms the kernel-based counterparts due to a uniform innate\npolarity of sentences and small number of complex sentences in the corpus.\nMoreover, a modality based approach is proposed to estimate the confidence\nlevel of the extracted association which can be used to assess the reliability\nof the reported association. Keywords: SNP, Phenotype, Biomedical Relation\nExtraction, Negation Detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00955v2",
            "date": "2020-12-02 03:53:13+00:00",
            "title": "How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering",
            "authors": [
                "Zhengbao Jiang",
                "Jun Araki",
                "Haibo Ding",
                "Graham Neubig"
            ],
            "abstract": "Recent works have shown that language models (LM) capture different types of\nknowledge regarding facts or common sense. However, because no model is\nperfect, they still fail to provide appropriate answers in many cases. In this\npaper, we ask the question \"how can we know when language models know, with\nconfidence, the answer to a particular query?\" We examine this question from\nthe point of view of calibration, the property of a probabilistic model's\npredicted probabilities actually being well correlated with the probabilities\nof correctness. We examine three strong generative models -- T5, BART, and\nGPT-2 -- and study whether their probabilities on QA tasks are well calibrated,\nfinding the answer is a relatively emphatic no. We then examine methods to\ncalibrate such models to make their confidence scores correlate better with the\nlikelihood of correctness through fine-tuning, post-hoc probability\nmodification, or adjustment of the predicted outputs or inputs. Experiments on\na diverse range of datasets demonstrate the effectiveness of our methods. We\nalso perform analysis to study the strengths and limitations of these methods,\nshedding light on further improvements that may be made in methods for\ncalibrating LMs. We have released the code at\nhttps://github.com/jzbjyb/lm-calibration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00958v1",
            "date": "2020-12-02 04:08:49+00:00",
            "title": "Interactive Teaching for Conversational AI",
            "authors": [
                "Qing Ping",
                "Feiyang Niu",
                "Govind Thattai",
                "Joel Chengottusseriyil",
                "Qiaozi Gao",
                "Aishwarya Reganti",
                "Prashanth Rajagopal",
                "Gokhan Tur",
                "Dilek Hakkani-Tur",
                "Prem Nataraja"
            ],
            "abstract": "Current conversational AI systems aim to understand a set of pre-designed\nrequests and execute related actions, which limits them to evolve naturally and\nadapt based on human interactions. Motivated by how children learn their first\nlanguage interacting with adults, this paper describes a new Teachable AI\nsystem that is capable of learning new language nuggets called concepts,\ndirectly from end users using live interactive teaching sessions. The proposed\nsetup uses three models to: a) Identify gaps in understanding automatically\nduring live conversational interactions, b) Learn the respective\ninterpretations of such unknown concepts from live interactions with users, and\nc) Manage a classroom sub-dialogue specifically tailored for interactive\nteaching sessions. We propose state-of-the-art transformer based neural\narchitectures of models, fine-tuned on top of pre-trained models, and show\naccuracy improvements on the respective components. We demonstrate that this\nmethod is very promising in leading way to build more adaptive and personalized\nlanguage understanding models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01254v1",
            "date": "2020-11-16 07:00:35+00:00",
            "title": "Retrieving and ranking short medical questions with two stages neural matching model",
            "authors": [
                "Xiang Li",
                "Xinyu Fu",
                "Zheng Lu",
                "Ruibin Bai",
                "Uwe Aickelin",
                "Peiming Ge",
                "Gong Liu"
            ],
            "abstract": "Internet hospital is a rising business thanks to recent advances in mobile\nweb technology and high demand of health care services. Online medical services\nbecome increasingly popular and active. According to US data in 2018, 80\npercent of internet users have asked health-related questions online. Numerous\ndata is generated in unprecedented speed and scale. Those representative\nquestions and answers in medical fields are valuable raw data sources for\nmedical data mining. Automated machine interpretation on those sheer amount of\ndata gives an opportunity to assist doctors to answer frequently asked\nmedical-related questions from the perspective of information retrieval and\nmachine learning approaches. In this work, we propose a novel two-stage\nframework for the semantic matching of query-level medical questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01266v1",
            "date": "2020-12-02 15:18:37+00:00",
            "title": "Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains",
            "authors": [
                "Haojie Pan",
                "Chengyu Wang",
                "Minghui Qiu",
                "Yichang Zhang",
                "Yaliang Li",
                "Jun Huang"
            ],
            "abstract": "Pre-trained language models have been applied to various NLP tasks with\nconsiderable performance gains. However, the large model sizes, together with\nthe long inference time, limit the deployment of such models in real-time\napplications. Typical approaches consider knowledge distillation to distill\nlarge teacher models into small student models. However, most of these studies\nfocus on single-domain only, which ignores the transferable knowledge from\nother domains. We argue that training a teacher with transferable knowledge\ndigested across domains can achieve better generalization capability to help\nknowledge distillation. To this end, we propose a Meta-Knowledge Distillation\n(Meta-KD) framework to build a meta-teacher model that captures transferable\nknowledge across domains inspired by meta-learning and use it to pass knowledge\nto students. Specifically, we first leverage a cross-domain learning process to\ntrain the meta-teacher on multiple domains, and then propose a\nmeta-distillation algorithm to learn single-domain student models with guidance\nfrom the meta-teacher. Experiments on two public multi-domain NLP tasks show\nthe effectiveness and superiority of the proposed Meta-KD framework. We also\ndemonstrate the capability of Meta-KD in both few-shot and zero-shot learning\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01285v2",
            "date": "2020-12-02 15:51:36+00:00",
            "title": "Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories",
            "authors": [
                "Jakob Prange",
                "Nathan Schneider",
                "Vivek Srikumar"
            ],
            "abstract": "Although current CCG supertaggers achieve high accuracy on the standard WSJ\ntest set, few systems make use of the categories' internal structure that will\ndrive the syntactic derivation during parsing. The tagset is traditionally\ntruncated, discarding the many rare and complex category types in the long\ntail. However, supertags are themselves trees. Rather than give up on rare\ntags, we investigate constructive models that account for their internal\nstructure, including novel methods for tree-structured prediction. Our best\ntagger is capable of recovering a sizeable fraction of the long-tail supertags\nand even generates CCG categories that have never been seen in training, while\napproximating the prior state of the art in overall tag accuracy with fewer\nparameters. We further investigate how well different approaches generalize to\nout-of-domain evaluation sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01288v1",
            "date": "2020-12-02 15:52:38+00:00",
            "title": "A Computational Approach to Measuring the Semantic Divergence of Cognates",
            "authors": [
                "Ana-Sabina Uban",
                "Alina-Maria Ciobanu",
                "Liviu P. Dinu"
            ],
            "abstract": "Meaning is the foundation stone of intercultural communication. Languages are\ncontinuously changing, and words shift their meanings for various reasons.\nSemantic divergence in related languages is a key concern of historical\nlinguistics. In this paper we investigate semantic divergence across languages\nby measuring the semantic similarity of cognate sets in multiple languages. The\nmethod that we propose is based on cross-lingual word embeddings. In this paper\nwe implement and evaluate our method on English and five Romance languages, but\nit can be extended easily to any language pair, requiring only large\nmonolingual corpora for the involved languages and a small bilingual dictionary\nfor the pair. This language-agnostic method facilitates a quantitative analysis\nof cognates divergence -- by computing degrees of semantic similarity between\ncognate pairs -- and provides insights for identifying false friends. As a\nsecond contribution, we formulate a straightforward method for detecting false\nfriends, and introduce the notion of \"soft false friend\" and \"hard false\nfriend\", as well as a measure of the degree of \"falseness\" of a false friends\npair. Additionally, we propose an algorithm that can output suggestions for\ncorrecting false friends, which could result in a very helpful tool for\nlanguage learning or translation.",
            "journal ref": "20th International Conference on Computational Linguistics and\n  Intelligent Text Processing (CICLing 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01305v1",
            "date": "2020-12-02 16:16:46+00:00",
            "title": "Analyzing Stylistic Variation across Different Political Regimes",
            "authors": [
                "Liviu P. Dinu",
                "Ana-Sabina Uban"
            ],
            "abstract": "In this article we propose a stylistic analysis of texts written across two\ndifferent periods, which differ not only temporally, but politically and\nculturally: communism and democracy in Romania. We aim to analyze the stylistic\nvariation between texts written during these two periods, and determine at what\nlevels the variation is more apparent (if any): at the stylistic level, at the\ntopic level etc. We take a look at the stylistic profile of these texts\ncomparatively, by performing clustering and classification experiments on the\ntexts, using traditional authorship attribution methods and features. To\nconfirm the stylistic variation is indeed an effect of the change in political\nand cultural environment, and not merely reflective of a natural change in the\nauthor's style with time, we look at various stylistic metrics over time and\nshow that the change in style between the two periods is statistically\nsignificant. We also perform an analysis of the variation in topic between the\ntwo epochs, to compare with the variation at the style level. These analyses\nshow that texts from the two periods can indeed be distinguished, both from the\npoint of view of style and from that of semantic content (topic).",
            "journal ref": "19th International Conference on Computational Linguistics and\n  Intelligent Text Processing (CICLing 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01524v1",
            "date": "2020-12-02 20:58:04+00:00",
            "title": "TAN-NTM: Topic Attention Networks for Neural Topic Modeling",
            "authors": [
                "Madhur Panwar",
                "Shashank Shailabh",
                "Milan Aggarwal",
                "Balaji Krishnamurthy"
            ],
            "abstract": "Topic models have been widely used to learn representations from text and\ngain insight into document corpora. To perform topic discovery, existing neural\nmodels use document bag-of-words (BoW) representation as input followed by\nvariational inference and learn topic-word distribution through reconstructing\nBoW. Such methods have mainly focused on analysing the effect of enforcing\nsuitable priors on document distribution. However, little importance has been\ngiven to encoding improved document features for capturing document semantics\nbetter. In this work, we propose a novel framework: TAN-NTM which models\ndocument as a sequence of tokens instead of BoW at the input layer and\nprocesses it through an LSTM whose output is used to perform variational\ninference followed by BoW decoding. We apply attention on LSTM outputs to\nempower the model to attend on relevant words which convey topic related cues.\nWe hypothesise that attention can be performed effectively if done in a topic\nguided manner and establish this empirically through ablations. We factor in\ntopic-word distribution to perform topic aware attention achieving\nstate-of-the-art results with ~9-15 percentage improvement over score of\nexisting SOTA topic models in NPMI coherence metric on four benchmark datasets\n- 20NewsGroup, Yelp, AGNews, DBpedia. TAN-NTM also obtains better document\nclassification accuracy owing to learning improved document-topic features. We\nqualitatively discuss that attention mechanism enables unsupervised discovery\nof keywords. Motivated by this, we further show that our proposed framework\nachieves state-of-the-art performance on topic aware supervised generation of\nkeyphrases on StackExchange and Weibo datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01675v1",
            "date": "2020-12-03 03:24:24+00:00",
            "title": "Federated Learning with Diversified Preference for Humor Recognition",
            "authors": [
                "Xu Guo",
                "Pengwei Xing",
                "Siwei Feng",
                "Boyang Li",
                "Chunyan Miao"
            ],
            "abstract": "Understanding humor is critical to creative language modeling with many\napplications in human-AI interaction. However, due to differences in the\ncognitive systems of the audience, the perception of humor can be highly\nsubjective. Thus, a given passage can be regarded as funny to different degrees\nby different readers. This makes training humorous text recognition models that\ncan adapt to diverse humor preferences highly challenging. In this paper, we\npropose the FedHumor approach to recognize humorous text contents in a\npersonalized manner through federated learning (FL). It is a federated BERT\nmodel capable of jointly considering the overall distribution of humor scores\nwith humor labels by individuals for given texts. Extensive experiments\ndemonstrate significant advantages of FedHumor in recognizing humor contents\naccurately for people with diverse humor preferences compared to 9\nstate-of-the-art humor recognition approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01704v1",
            "date": "2020-12-03 05:03:38+00:00",
            "title": "Multilingual Neural RST Discourse Parsing",
            "authors": [
                "Zhengyuan Liu",
                "Ke Shi",
                "Nancy F. Chen"
            ],
            "abstract": "Text discourse parsing plays an important role in understanding information\nflow and argumentative structure in natural language. Previous research under\nthe Rhetorical Structure Theory (RST) has mostly focused on inducing and\nevaluating models from the English treebank. However, the parsing tasks for\nother languages such as German, Dutch, and Portuguese are still challenging due\nto the shortage of annotated data. In this work, we investigate two approaches\nto establish a neural, cross-lingual discourse parser via: (1) utilizing\nmultilingual vector representations; and (2) adopting segment-level translation\nof the source content. Experiment results show that both methods are effective\neven with limited training data, and achieve state-of-the-art performance on\ncross-lingual, document-level discourse parsing on all sub-tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01786v2",
            "date": "2020-12-03 09:32:05+00:00",
            "title": "Self-Explaining Structures Improve NLP Models",
            "authors": [
                "Zijun Sun",
                "Chun Fan",
                "Qinghong Han",
                "Xiaofei Sun",
                "Yuxian Meng",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Existing approaches to explaining deep learning models in NLP usually suffer\nfrom two major drawbacks: (1) the main model and the explaining model are\ndecoupled: an additional probing or surrogate model is used to interpret an\nexisting model, and thus existing explaining tools are not self-explainable;\n(2) the probing model is only able to explain a model's predictions by\noperating on low-level features by computing saliency scores for individual\nwords but are clumsy at high-level text units such as phrases, sentences, or\nparagraphs. To deal with these two issues, in this paper, we propose a simple\nyet general and effective self-explaining framework for deep learning models in\nNLP. The key point of the proposed framework is to put an additional layer, as\nis called by the interpretation layer, on top of any existing NLP model. This\nlayer aggregates the information for each text span, which is then associated\nwith a specific weight, and their weighted combination is fed to the softmax\nfunction for the final prediction. The proposed model comes with the following\nmerits: (1) span weights make the model self-explainable and do not require an\nadditional probing model for interpretation; (2) the proposed model is general\nand can be adapted to any existing deep learning structures in NLP; (3) the\nweight associated with each text span provides direct importance scores for\nhigher-level text units such as phrases and sentences. We for the first time\nshow that interpretability does not come at the cost of performance: a neural\nmodel of self-explaining features obtains better performances than its\ncounterpart without the self-explaining nature, achieving a new SOTA\nperformance of 59.1 on SST-5 and a new SOTA performance of 92.3 on SNLI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01878v1",
            "date": "2020-12-03 12:49:22+00:00",
            "title": "Label Enhanced Event Detection with Heterogeneous Graph Attention Networks",
            "authors": [
                "Shiyao Cui",
                "Bowen Yu",
                "Xin Cong",
                "Tingwen Liu",
                "Quangang Li",
                "Jinqiao Shi"
            ],
            "abstract": "Event Detection (ED) aims to recognize instances of specified types of event\ntriggers in text. Different from English ED, Chinese ED suffers from the\nproblem of word-trigger mismatch due to the uncertain word boundaries. Existing\napproaches injecting word information into character-level models have achieved\npromising progress to alleviate this problem, but they are limited by two\nissues. First, the interaction between characters and lexicon words is not\nfully exploited. Second, they ignore the semantic information provided by event\nlabels. We thus propose a novel architecture named Label enhanced Heterogeneous\nGraph Attention Networks (L-HGAT). Specifically, we transform each sentence\ninto a graph, where character nodes and word nodes are connected with different\ntypes of edges, so that the interaction between words and characters is fully\nreserved. A heterogeneous graph attention networks is then introduced to\npropagate relational message and enrich information interaction. Furthermore,\nwe convert each label into a trigger-prototype-based embedding, and design a\nmargin loss to guide the model distinguish confusing event labels. Experiments\non two benchmark datasets show that our model achieves significant improvement\nover a range of competitive baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02015v1",
            "date": "2020-12-03 15:50:20+00:00",
            "title": "Context in Informational Bias Detection",
            "authors": [
                "Esther van den Berg",
                "Katja Markert"
            ],
            "abstract": "Informational bias is bias conveyed through sentences or clauses that provide\ntangential, speculative or background information that can sway readers'\nopinions towards entities. By nature, informational bias is context-dependent,\nbut previous work on informational bias detection has not explored the role of\ncontext beyond the sentence. In this paper, we explore four kinds of context\nfor informational bias in English news articles: neighboring sentences, the\nfull article, articles on the same event from other news publishers, and\narticles from the same domain (but potentially different events). We find that\nintegrating event context improves classification performance over a very\nstrong baseline. In addition, we perform the first error analysis of models on\nthis task. We find that the best-performing context-inclusive model outperforms\nthe baseline on longer sentences, and sentences from politically centrist\narticles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02028v1",
            "date": "2020-11-18 20:15:01+00:00",
            "title": "Ontology-based and User-focused Automatic Text Summarization (OATS): Using COVID-19 Risk Factors as an Example",
            "authors": [
                "Po-Hsu Allen Chen",
                "Amy Leibrand",
                "Jordan Vasko",
                "Mitch Gauthier"
            ],
            "abstract": "This paper proposes a novel Ontology-based and user-focused Automatic Text\nSummarization (OATS) system, in the setting where the goal is to automatically\ngenerate text summarization from unstructured text by extracting sentences\ncontaining the information that aligns to the user's focus. OATS consists of\ntwo modules: ontology-based topic identification and user-focused text\nsummarization; it first utilizes an ontology-based approach to identify\nrelevant documents to user's interest, and then takes advantage of the answers\nextracted from a question answering model using questions specified from users\nfor the generation of text summarization. To support the fight against the\nCOVID-19 pandemic, we used COVID-19 risk factors as an example to demonstrate\nthe proposed OATS system with the aim of helping the medical community\naccurately identify relevant scientific literature and efficiently review the\ninformation that addresses risk factors related to COVID-19.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02030v2",
            "date": "2020-11-20 13:58:21+00:00",
            "title": "Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural Networks",
            "authors": [
                "Ileana Rugina",
                "Rumen Dangovski",
                "Li Jing",
                "Preslav Nakov",
                "Marin Solja\u010di\u0107"
            ],
            "abstract": "The attention mechanism is a key component of the neural revolution in\nNatural Language Processing (NLP). As the size of attention-based models has\nbeen scaling with the available computational resources, a number of pruning\ntechniques have been developed to detect and to exploit sparseness in such\nmodels in order to make them more efficient. The majority of such efforts have\nfocused on looking for attention patterns and then hard-coding them to achieve\nsparseness, or pruning the weights of the attention mechanisms based on\nstatistical information from the training data. Here, we marry these two lines\nof research by proposing Attention Pruning (AP): a novel pruning framework that\ncollects observations about the attention patterns in a fixed dataset and then\ninduces a global sparseness mask for the model. This can save 90% of the\nattention computation for language modelling and about 50% for machine\ntranslation and for solving GLUE tasks, while maintaining the quality of the\nresults. Moreover, using our method, we discovered important distinctions\nbetween self- and cross-attention patterns, which could guide future NLP\nresearch in attention-based modelling. Our framework can in principle speed up\nany model that uses attention mechanism, thus helping develop better models for\nexisting or for new NLP applications. Our implementation is available at\nhttps://github.com/irugina/AP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02038v2",
            "date": "2020-11-22 17:28:50+00:00",
            "title": "Modelling Compositionality and Structure Dependence in Natural Language",
            "authors": [
                "Karthikeya Ramesh Kaushik",
                "Andrea E. Martin"
            ],
            "abstract": "Human beings possess the most sophisticated computational machinery in the\nknown universe. We can understand language of rich descriptive power, and\ncommunicate in the same environment with astonishing clarity. Two of the many\ncontributors to the interest in natural language - the properties of\nCompositionality and Structure Dependence, are well documented, and offer a\nvast space to ask interesting modelling questions. The first step to begin\nanswering these questions is to ground verbal theory in formal terms. Drawing\non linguistics and set theory, a formalisation of these ideas is presented in\nthe first half of this thesis. We see how cognitive systems that process\nlanguage need to have certain functional constraints, viz. time based,\nincremental operations that rely on a structurally defined domain. The\nobservations that result from analysing this formal setup are examined as part\nof a modelling exercise. Using the advances of word embedding techniques, a\nmodel of relational learning is simulated with a custom dataset to demonstrate\nhow a time based role-filler binding mechanism satisfies some of the\nconstraints described in the first section. The model's ability to map\nstructure, along with its symbolic-connectionist architecture makes for a\ncognitively plausible implementation. The formalisation and simulation are\ntogether an attempt to recognise the constraints imposed by linguistic theory,\nand explore the opportunities presented by a cognitive model of relation\nlearning to realise these constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02144v1",
            "date": "2020-12-03 18:23:21+00:00",
            "title": "Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !",
            "authors": [
                "Wen Xiao",
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "The multi-head self-attention of popular transformer models is widely used\nwithin Natural Language Processing (NLP), including for the task of extractive\nsummarization. With the goal of analyzing and pruning the parameter-heavy\nself-attention mechanism, there are multiple approaches proposing more\nparameter-light self-attention alternatives. In this paper, we present a novel\nparameter-lean self-attention mechanism using discourse priors. Our new tree\nself-attention is based on document-level discourse information, extending the\nrecently proposed \"Synthesizer\" framework with another lightweight alternative.\nWe show empirical results that our tree self-attention approach achieves\ncompetitive ROUGE-scores on the task of extractive summarization. When compared\nto the original single-head transformer model, the tree attention approach\nreaches similar performance on both, EDU and sentence level, despite the\nsignificant reduction of parameters in the attention component. We further\nsignificantly outperform the 8-head transformer model on sentence level when\napplying a more balanced hyper-parameter setting, requiring an order of\nmagnitude less parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02353v2",
            "date": "2020-12-04 01:11:13+00:00",
            "title": "Few-Shot Event Detection with Prototypical Amortized Conditional Random Field",
            "authors": [
                "Xin Cong",
                "Shiyao Cui",
                "Bowen Yu",
                "Tingwen Liu",
                "Yubin Wang",
                "Bin Wang"
            ],
            "abstract": "Event detection tends to struggle when it needs to recognize novel event\ntypes with a few samples. The previous work attempts to solve this problem in\nthe identify-then-classify manner but ignores the trigger discrepancy between\nevent types, thus suffering from the error propagation. In this paper, we\npresent a novel unified model which converts the task to a few-shot tagging\nproblem with a double-part tagging scheme. To this end, we first propose the\nPrototypical Amortized Conditional Random Field (PA-CRF) to model the label\ndependency in the few-shot scenario, which approximates the transition scores\nbetween labels based on the label prototypes. Then Gaussian distribution is\nintroduced for modeling of the transition scores to alleviate the uncertain\nestimation resulting from insufficient data. Experimental results show that the\nunified models work better than existing identify-then-classify models and our\nPA-CRF further achieves the best results on the benchmark dataset FewEvent. Our\ncode and data are available at http://github.com/congxin95/PA-CRF.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02462v1",
            "date": "2020-12-04 08:34:39+00:00",
            "title": "Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning",
            "authors": [
                "Daniel Grie\u00dfhaber",
                "Johannes Maucher",
                "Ngoc Thang Vu"
            ],
            "abstract": "Recently, leveraging pre-trained Transformer based language models in down\nstream, task specific models has advanced state of the art results in natural\nlanguage understanding tasks. However, only a little research has explored the\nsuitability of this approach in low resource settings with less than 1,000\ntraining data points. In this work, we explore fine-tuning methods of BERT -- a\npre-trained Transformer based language model -- by utilizing pool-based active\nlearning to speed up training while keeping the cost of labeling new data\nconstant. Our experimental results on the GLUE data set show an advantage in\nmodel performance by maximizing the approximate knowledge gain of the model\nwhen querying from the pool of unlabeled data. Finally, we demonstrate and\nanalyze the benefits of freezing layers of the language model during\nfine-tuning to reduce the number of trainable parameters, making it more\nsuitable for low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02498v1",
            "date": "2020-12-04 09:58:47+00:00",
            "title": "Data Processing and Annotation Schemes for FinCausal Shared Task",
            "authors": [
                "Dominique Mariko",
                "Estelle Labidurie",
                "Yagmur Ozturk",
                "Hanna Abi Akl",
                "Hugues de Mazancourt"
            ],
            "abstract": "This document explains the annotation schemes used to label the data for the\nFinCausal Shared Task (Mariko et al., 2020). This task is associated to the\nJoint Workshop on Financial Narrative Processing and MultiLing Financial\nSummarisation (FNP-FNS 2020), to be held at The 28th International Conference\non Computational Linguistics (COLING'2020), on December 12, 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02535v2",
            "date": "2020-12-04 11:32:55+00:00",
            "title": "CUED_speech at TREC 2020 Podcast Summarisation Track",
            "authors": [
                "Potsawee Manakul",
                "Mark Gales"
            ],
            "abstract": "In this paper, we describe our approach for the Podcast Summarisation\nchallenge in TREC 2020. Given a podcast episode with its transcription, the\ngoal is to generate a summary that captures the most important information in\nthe content. Our approach consists of two steps: (1) Filtering redundant or\nless informative sentences in the transcription using the attention of a\nhierarchical model; (2) Applying a state-of-the-art text summarisation system\n(BART) fine-tuned on the Podcast data using a sequence-level reward function.\nFurthermore, we perform ensembles of three and nine models for our submission\nruns. We also fine-tune the BART model on the Podcast data as our baseline. The\nhuman evaluation by NIST shows that our best submission achieves 1.777 in the\nEGFB scale, while the score of creator-provided description is 1.291. Our\nsystem won the Spotify Podcast Summarisation Challenge in the TREC2020 Podcast\nTrack in both human and automatic evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02553v1",
            "date": "2020-12-04 12:30:31+00:00",
            "title": "DDRel: A New Dataset for Interpersonal Relation Classification in Dyadic Dialogues",
            "authors": [
                "Qi Jia",
                "Hongru Huang",
                "Kenny Q. Zhu"
            ],
            "abstract": "Interpersonal language style shifting in dialogues is an interesting and\nalmost instinctive ability of human. Understanding interpersonal relationship\nfrom language content is also a crucial step toward further understanding\ndialogues. Previous work mainly focuses on relation extraction between named\nentities in texts. In this paper, we propose the task of relation\nclassification of interlocutors based on their dialogues. We crawled movie\nscripts from IMSDb, and annotated the relation labels for each session\naccording to 13 pre-defined relationships. The annotated dataset DDRel consists\nof 6300 dyadic dialogue sessions between 694 pair of speakers with 53,126\nutterances in total. We also construct session-level and pair-level relation\nclassification tasks with widely-accepted baselines. The experimental results\nshow that this task is challenging for existing models and the dataset will be\nuseful for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02565v1",
            "date": "2020-12-04 13:12:31+00:00",
            "title": "Automated Detection of Cyberbullying Against Women and Immigrants and Cross-domain Adaptability",
            "authors": [
                "Thushari Atapattu",
                "Mahen Herath",
                "Georgia Zhang",
                "Katrina Falkner"
            ],
            "abstract": "Cyberbullying is a prevalent and growing social problem due to the surge of\nsocial media technology usage. Minorities, women, and adolescents are among the\ncommon victims of cyberbullying. Despite the advancement of NLP technologies,\nthe automated cyberbullying detection remains challenging. This paper focuses\non advancing the technology using state-of-the-art NLP techniques. We use a\nTwitter dataset from SemEval 2019 - Task 5(HatEval) on hate speech against\nwomen and immigrants. Our best performing ensemble model based on DistilBERT\nhas achieved 0.73 and 0.74 of F1 score in the task of classifying hate speech\n(Task A) and aggressiveness and target (Task B) respectively. We adapt the\nensemble model developed for Task A to classify offensive language in external\ndatasets and achieved ~0.7 of F1 score using three benchmark datasets, enabling\npromising results for cross-domain adaptability. We conduct a qualitative\nanalysis of misclassified tweets to provide insightful recommendations for\nfuture cyberbullying research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02578v1",
            "date": "2020-12-04 13:36:29+00:00",
            "title": "Ve'rdd. Narrowing the Gap between Paper Dictionaries, Low-Resource NLP and Community Involvement",
            "authors": [
                "Khalid Alnajjar",
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Jack Rueter",
                "Niko Partanen"
            ],
            "abstract": "We present an open-source online dictionary editing system, Ve'rdd, that\noffers a chance to re-evaluate and edit grassroots dictionaries that have been\nexposed to multiple amateur editors. The idea is to incorporate community\nactivities into a state-of-the-art finite-state language description of a\nseriously endangered minority language, Skolt Sami. Problems involve getting\nthe community to take part in things above the pencil-and-paper level. At\ntimes, it seems that the native speakers and the dictionary oriented are\nlacking technical understanding to utilize the infrastructures which might make\ntheir work more meaningful in the future, i.e. multiple reuse of all of their\ninput. Therefore, our system integrates with the existing tools and\ninfrastructures for Uralic language masking the technical complexities behind a\nuser-friendly UI.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02613v1",
            "date": "2020-12-04 14:17:46+00:00",
            "title": "FinnSentiment -- A Finnish Social Media Corpus for Sentiment Polarity Annotation",
            "authors": [
                "Krister Lind\u00e9n",
                "Tommi Jauhiainen",
                "Sam Hardwick"
            ],
            "abstract": "Sentiment analysis and opinion mining is an important task with obvious\napplication areas in social media, e.g. when indicating hate speech and fake\nnews. In our survey of previous work, we note that there is no large-scale\nsocial media data set with sentiment polarity annotations for Finnish. This\npublications aims to remedy this shortcoming by introducing a 27,000 sentence\ndata set annotated independently with sentiment polarity by three native\nannotators. We had the same three annotators for the whole data set, which\nprovides a unique opportunity for further studies of annotator behaviour over\ntime. We analyse their inter-annotator agreement and provide two baselines to\nvalidate the usefulness of the data set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02721v1",
            "date": "2020-12-04 17:11:04+00:00",
            "title": "Event Guided Denoising for Multilingual Relation Learning",
            "authors": [
                "Amith Ananthram",
                "Emily Allaway",
                "Kathleen McKeown"
            ],
            "abstract": "General purpose relation extraction has recently seen considerable gains in\npart due to a massively data-intensive distant supervision technique from\nSoares et al. (2019) that produces state-of-the-art results across many\nbenchmarks. In this work, we present a methodology for collecting high quality\ntraining data for relation extraction from unlabeled text that achieves a\nnear-recreation of their zero-shot and few-shot results at a fraction of the\ntraining cost. Our approach exploits the predictable distributional structure\nof date-marked news articles to build a denoised corpus -- the extraction\nprocess filters out low quality examples. We show that a smaller multilingual\nencoder trained on this corpus performs comparably to the current\nstate-of-the-art (when both receive little to no fine-tuning) on few-shot and\nstandard relation benchmarks in English and Spanish despite using many fewer\nexamples (50k vs. 300mil+).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02819v1",
            "date": "2020-12-04 19:51:24+00:00",
            "title": "On-Device Sentence Similarity for SMS Dataset",
            "authors": [
                "Arun D Prabhu",
                "Nikhil Arora",
                "Shubham Vatsal",
                "Gopi Ramena",
                "Sukumar Moharana",
                "Naresh Purre"
            ],
            "abstract": "Determining the sentence similarity between Short Message Service (SMS)\ntexts/sentences plays a significant role in mobile device industry. Gauging the\nsimilarity between SMS data is thus necessary for various applications like\nenhanced searching and navigation, clubbing together SMS of similar type when\ngiven a custom label or tag is provided by user irrespective of their sender\netc. The problem faced with SMS data is its incomplete structure and\ngrammatical inconsistencies. In this paper, we propose a unique pipeline for\nevaluating the text similarity between SMS texts. We use Part of Speech (POS)\nmodel for keyword extraction by taking advantage of the partial structure\nembedded in SMS texts and similarity comparisons are carried out using\nstatistical methods. The proposed pipeline deals with major semantic variations\nacross SMS data as well as makes it effective for its application on-device\n(mobile phone). To showcase the capabilities of our work, our pipeline has been\ndesigned with an inclination towards one of the possible applications of SMS\ntext similarity discussed in one of the following sections but nonetheless\nguarantees scalability for other applications as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02875v1",
            "date": "2020-12-04 22:20:55+00:00",
            "title": "Inductive Bias and Language Expressivity in Emergent Communication",
            "authors": [
                "Shangmin Guo",
                "Yi Ren",
                "Agnieszka S\u0142owik",
                "Kory Mathewson"
            ],
            "abstract": "Referential games and reconstruction games are the most common game types for\nstudying emergent languages. We investigate how the type of the language game\naffects the emergent language in terms of: i) language compositionality and ii)\ntransfer of an emergent language to a task different from its origin, which we\nrefer to as language expressivity. With empirical experiments on a handcrafted\nsymbolic dataset, we show that languages emerged from different games have\ndifferent compositionality and further different expressivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02929v1",
            "date": "2020-12-05 02:51:09+00:00",
            "title": "Data-Efficient Methods for Dialogue Systems",
            "authors": [
                "Igor Shalyminov"
            ],
            "abstract": "Conversational User Interface (CUI) has become ubiquitous in everyday life,\nin consumer-focused products like Siri and Alexa or business-oriented\nsolutions. Deep learning underlies many recent breakthroughs in dialogue\nsystems but requires very large amounts of training data, often annotated by\nexperts. Trained with smaller data, these methods end up severely lacking\nrobustness (e.g. to disfluencies and out-of-domain input), and often just have\ntoo little generalisation power. In this thesis, we address the above issues by\nintroducing a series of methods for training robust dialogue systems from\nminimal data. Firstly, we study two orthogonal approaches to dialogue:\nlinguistically informed and machine learning-based - from the data efficiency\nperspective. We outline the steps to obtain data-efficient solutions with\neither approach. We then introduce two data-efficient models for dialogue\nresponse generation: the Dialogue Knowledge Transfer Network based on latent\nvariable dialogue representations, and the hybrid Generative-Retrieval\nTransformer model (ranked first at the DSTC 8 Fast Domain Adaptation task).\nNext, we address the problem of robustness given minimal data. As such, propose\na multitask LSTM-based model for domain-general disfluency detection. For the\nproblem of out-of-domain input, we present Turn Dropout, a data augmentation\ntechnique for anomaly detection only using in-domain data, and introduce\nautoencoder-augmented models for efficient training with Turn Dropout. Finally,\nwe focus on social dialogue and introduce a neural model for response ranking\nin social conversation used in Alana, the 3rd place winner in the Amazon Alexa\nPrize 2017 and 2018. We employ a novel technique of predicting the dialogue\nlength as the main ranking objective and show that this approach improves upon\nthe ratings-based counterpart in terms of data efficiency while matching it in\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02983v1",
            "date": "2020-12-05 09:18:43+00:00",
            "title": "On-Device Tag Generation for Unstructured Text",
            "authors": [
                "Manish Chugani",
                "Shubham Vatsal",
                "Gopi Ramena",
                "Sukumar Moharana",
                "Naresh Purre"
            ],
            "abstract": "With the overwhelming transition to smart phones, storing important\ninformation in the form of unstructured text has become habitual to users of\nmobile devices. From grocery lists to drafts of emails and important speeches,\nusers store a lot of data in the form of unstructured text (for eg: in the\nNotes application) on their devices, leading to cluttering of data. This not\nonly prevents users from efficient navigation in the applications but also\nprecludes them from perceiving the relations that could be present across data\nin those applications. This paper proposes a novel pipeline to generate a set\nof tags using world knowledge based on the keywords and concepts present in\nunstructured textual data. These tags can then be used to summarize, categorize\nor search for the desired information thus enhancing user experience by\nallowing them to have a holistic outlook of the kind of information stored in\nthe form of unstructured text. In the proposed system, we use an on-device\n(mobile phone) efficient CNN model with pruned ConceptNet resource to achieve\nour goal. The architecture also presents a novel ranking algorithm to extract\nthe top n tags from any given text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02990v1",
            "date": "2020-12-05 10:00:06+00:00",
            "title": "Codeswitched Sentence Creation using Dependency Parsing",
            "authors": [
                "Dhruval Jain",
                "Arun D Prabhu",
                "Shubham Vatsal",
                "Gopi Ramena",
                "Naresh Purre"
            ],
            "abstract": "Codeswitching has become one of the most common occurrences across\nmultilingual speakers of the world, especially in countries like India which\nencompasses around 23 official languages with the number of bilingual speakers\nbeing around 300 million. The scarcity of Codeswitched data becomes a\nbottleneck in the exploration of this domain with respect to various Natural\nLanguage Processing (NLP) tasks. We thus present a novel algorithm which\nharnesses the syntactic structure of English grammar to develop grammatically\nsensible Codeswitched versions of English-Hindi, English-Marathi and\nEnglish-Kannada data. Apart from maintaining the grammatical sanity to a great\nextent, our methodology also guarantees abundant generation of data from a\nminuscule snapshot of given data. We use multiple datasets to showcase the\ncapabilities of our algorithm while at the same time we assess the quality of\ngenerated Codeswitched data using some qualitative metrics along with providing\nbaseline results for couple of NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03118v1",
            "date": "2020-12-05 20:50:53+00:00",
            "title": "Modeling and Utilizing User's Internal State in Movie Recommendation Dialogue",
            "authors": [
                "Takashi Kodama",
                "Ribeka Tanaka",
                "Sadao Kurohashi"
            ],
            "abstract": "Intelligent dialogue systems are expected as a new interface between humans\nand machines. Such an intelligent dialogue system should estimate the user's\ninternal state (UIS) in dialogues and change its response appropriately\naccording to the estimation result. In this paper, we model the UIS in\ndialogues, taking movie recommendation dialogues as examples, and construct a\ndialogue system that changes its response based on the UIS. Based on the\ndialogue data analysis, we model the UIS as three elements: knowledge,\ninterest, and engagement. We train the UIS estimators on a dialogue corpus with\nthe modeled UIS's annotations. The estimators achieved high estimation\naccuracy. We also design response change rules that change the system's\nresponses according to each UIS. We confirmed that response changes using the\nresult of the UIS estimators improved the system utterances' naturalness in\nboth dialogue-wise evaluation and utterance-wise evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03201v1",
            "date": "2020-12-06 07:33:45+00:00",
            "title": "A Two-Systems Perspective for Computational Thinking",
            "authors": [
                "Arvind W Kiwelekar",
                "Swanand Navandar",
                "Dharmendra K. Yadav"
            ],
            "abstract": "Computational Thinking (CT) has emerged as one of the vital thinking skills\nin recent times, especially for Science, Technology, Engineering and Management\n(STEM) graduates. Educators are in search of underlying cognitive models\nagainst which CT can be analyzed and evaluated. This paper suggests adopting\nKahneman's two-systems model as a framework to understand the computational\nthought process. Kahneman's two-systems model postulates that human thinking\nhappens at two levels, i.e. fast and slow thinking. This paper illustrates\nthrough examples that CT activities can be represented and analyzed using\nKahneman's two-systems model. The potential benefits of adopting Kahneman's\ntwo-systems perspective are that it helps us to fix the biases that cause\nerrors in our reasoning. Further, it also provides a set of heuristics to speed\nup reasoning activities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03477v2",
            "date": "2020-12-07 06:48:59+00:00",
            "title": "Document Graph for Neural Machine Translation",
            "authors": [
                "Mingzhou Xu",
                "Liangyou Li",
                "Derek. F. Wong",
                "Qun Liu",
                "Lidia S. Chao"
            ],
            "abstract": "Previous works have shown that contextual information can improve the\nperformance of neural machine translation (NMT). However, most existing\ndocument-level NMT methods failed to leverage contexts beyond a few set of\nprevious sentences. How to make use of the whole document as global contexts is\nstill a challenge. To address this issue, we hypothesize that a document can be\nrepresented as a graph that connects relevant contexts regardless of their\ndistances. We employ several types of relations, including adjacency, syntactic\ndependency, lexical consistency, and coreference, to construct the document\ngraph. Then, we incorporate both source and target graphs into the conventional\nTransformer architecture with graph convolutional networks. Experiments on\nvarious NMT benchmarks, including IWSLT English-French, Chinese-English, WMT\nEnglish-German and Opensubtitle English-Russian, demonstrate that using\ndocument graphs can significantly improve the translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03502v2",
            "date": "2020-12-07 07:51:38+00:00",
            "title": "Dialogue Discourse-Aware Graph Model and Data Augmentation for Meeting Summarization",
            "authors": [
                "Xiachong Feng",
                "Xiaocheng Feng",
                "Bing Qin",
                "Xinwei Geng"
            ],
            "abstract": "Meeting summarization is a challenging task due to its dynamic interaction\nnature among multiple speakers and lack of sufficient training data. Existing\nmethods view the meeting as a linear sequence of utterances while ignoring the\ndiverse relations between each utterance. Besides, the limited labeled data\nfurther hinders the ability of data-hungry neural models. In this paper, we try\nto mitigate the above challenges by introducing dialogue-discourse relations.\nFirst, we present a Dialogue Discourse-Dware Meeting Summarizer (DDAMS) to\nexplicitly model the interaction between utterances in a meeting by modeling\ndifferent discourse relations. The core module is a relational graph encoder,\nwhere the utterances and discourse relations are modeled in a graph interaction\nmanner. Moreover, we devise a Dialogue Discourse-Aware Data Augmentation\n(DDADA) strategy to construct a pseudo-summarization corpus from existing input\nmeetings, which is 20 times larger than the original dataset and can be used to\npretrain DDAMS. Experimental results on AMI and ICSI meeting datasets show that\nour full system can achieve SOTA performance. Our codes will be available at:\nhttps://github.com/xcfcode/DDAMS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03536v2",
            "date": "2020-12-07 08:58:09+00:00",
            "title": "H-FND: Hierarchical False-Negative Denoising for Distant Supervision Relation Extraction",
            "authors": [
                "Jhih-Wei Chen",
                "Tsu-Jui Fu",
                "Chen-Kang Lee",
                "Wei-Yun Ma"
            ],
            "abstract": "Although distant supervision automatically generates training data for\nrelation extraction, it also introduces false-positive (FP) and false-negative\n(FN) training instances to the generated datasets. Whereas both types of errors\ndegrade the final model performance, previous work on distant supervision\ndenoising focuses more on suppressing FP noise and less on resolving the FN\nproblem. We here propose H-FND, a hierarchical false-negative denoising\nframework for robust distant supervision relation extraction, as an FN\ndenoising solution. H-FND uses a hierarchical policy which first determines\nwhether non-relation (NA) instances should be kept, discarded, or revised\nduring the training process. For those learning instances which are to be\nrevised, the policy further reassigns them appropriate relations, making them\nbetter training inputs. Experiments on SemEval-2010 and TACRED were conducted\nwith controlled FN ratios that randomly turn the relations of training and\nvalidation instances into negatives to generate FN instances. In this setting,\nH-FND can revise FN instances correctly and maintains high F1 scores even when\n50% of the instances have been turned into negatives. Experiment on NYT10 is\nfurther conducted to shows that H-FND is applicable in a realistic setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03539v2",
            "date": "2020-12-07 09:08:16+00:00",
            "title": "UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2",
            "authors": [
                "Yunyi Yang",
                "Yunhao Li",
                "Xiaojun Quan"
            ],
            "abstract": "This paper presents our task-oriented dialog system UBAR which models\ntask-oriented dialogs on a dialog session level. Specifically, UBAR is acquired\nby fine-tuning the large pre-trained unidirectional language model GPT-2 on the\nsequence of the entire dialog session which is composed of user utterance,\nbelief state, database result, system act, and system response of every dialog\nturn. Additionally, UBAR is evaluated in a more realistic setting, where its\ndialog context has access to user utterances and all content it generated such\nas belief states, system acts, and system responses. Experimental results on\nthe MultiWOZ datasets show that UBAR achieves state-of-the-art performances in\nmultiple settings, improving the combined score of response generation, policy\noptimization, and end-to-end modeling by 4.7, 3.5, and 9.4 points respectively.\nThorough analyses demonstrate that the session-level training sequence\nformulation and the generated dialog context are essential for UBAR to operate\nas a fully end-to-end task-oriented dialog system in real life. We also examine\nthe transfer ability of UBAR to new domains with limited data and provide\nvisualization and a case study to illustrate the advantages of UBAR in modeling\non a dialog session level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03619v2",
            "date": "2020-12-07 12:09:37+00:00",
            "title": "Structural Text Segmentation of Legal Documents",
            "authors": [
                "Dennis Aumiller",
                "Satya Almasian",
                "Sebastian Lackner",
                "Michael Gertz"
            ],
            "abstract": "The growing complexity of legal cases has lead to an increasing interest in\nlegal information retrieval systems that can effectively satisfy user-specific\ninformation needs. However, such downstream systems typically require documents\nto be properly formatted and segmented, which is often done with relatively\nsimple pre-processing steps, disregarding topical coherence of segments.\nSystems generally rely on representations of individual sentences or\nparagraphs, which may lack crucial context, or document-level representations,\nwhich are too long for meaningful search results. To address this issue, we\npropose a segmentation system that can predict topical coherence of sequential\ntext segments spanning several paragraphs, effectively segmenting a document\nand providing a more balanced representation for downstream applications. We\nbuild our model on top of popular transformer networks and formulate structural\ntext segmentation as topical change detection, by performing a series of\nindependent classifications that allow for efficient fine-tuning on\ntask-specific data. We crawl a novel dataset consisting of roughly $74,000$\nonline Terms-of-Service documents, including hierarchical topic annotations,\nwhich we use for training. Results show that our proposed system significantly\noutperforms baselines, and adapts well to structural peculiarities of legal\ndocuments. We release both data and trained models to the research community\nfor future work.https://github.com/dennlinger/TopicalChange",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03763v1",
            "date": "2020-12-07 15:00:18+00:00",
            "title": "Using previous acoustic context to improve Text-to-Speech synthesis",
            "authors": [
                "Pilar Oplustil-Gallegos",
                "Simon King"
            ],
            "abstract": "Many speech synthesis datasets, especially those derived from audiobooks,\nnaturally comprise sequences of utterances. Nevertheless, such data are\ncommonly treated as individual, unordered utterances both when training a model\nand at inference time. This discards important prosodic phenomena above the\nutterance level. In this paper, we leverage the sequential nature of the data\nusing an acoustic context encoder that produces an embedding of the previous\nutterance audio. This is input to the decoder in a Tacotron 2 model. The\nembedding is also used for a secondary task, providing additional supervision.\nWe compare two secondary tasks: predicting the ordering of utterance pairs, and\npredicting the embedding of the current utterance audio. Results show that the\nrelation between consecutive utterances is informative: our proposed model\nsignificantly improves naturalness over a Tacotron 2 baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03833v1",
            "date": "2020-12-07 16:33:23+00:00",
            "title": "What Meaning-Form Correlation Has to Compose With",
            "authors": [
                "Timothee Mickus",
                "Timoth\u00e9e Bernard",
                "Denis Paperno"
            ],
            "abstract": "Compositionality is a widely discussed property of natural languages,\nalthough its exact definition has been elusive. We focus on the proposal that\ncompositionality can be assessed by measuring meaning-form correlation. We\nanalyze meaning-form correlation on three sets of languages: (i) artificial toy\nlanguages tailored to be compositional, (ii) a set of English dictionary\ndefinitions, and (iii) a set of English sentences drawn from literature. We\nfind that linguistic phenomena such as synonymy and ungrounded stop-words weigh\non MFC measurements, and that straightforward methods to mitigate their effects\nhave widely varying results depending on the dataset they are applied to. Data\nand code are made publicly available.",
            "journal ref": "Proceedings of the 28th International Conference on Computational\n  Linguistics (2020) 3737-3749",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03845v1",
            "date": "2020-12-07 16:48:34+00:00",
            "title": "Stylometry for Noisy Medieval Data: Evaluating Paul Meyer's Hagiographic Hypothesis",
            "authors": [
                "Jean-Baptiste Camps",
                "Thibault Cl\u00e9rice",
                "Ariane Pinche"
            ],
            "abstract": "Stylometric analysis of medieval vernacular texts is still a significant\nchallenge: the importance of scribal variation, be it spelling or more\nsubstantial, as well as the variants and errors introduced in the tradition,\ncomplicate the task of the would-be stylometrist. Basing the analysis on the\nstudy of the copy from a single hand of several texts can partially mitigate\nthese issues (Camps and Cafiero, 2013), but the limited availability of\ncomplete diplomatic transcriptions might make this difficult. In this paper, we\nuse a workflow combining handwritten text recognition and stylometric analysis,\napplied to the case of the hagiographic works contained in MS BnF, fr. 412. We\nseek to evaluate Paul Meyer's hypothesis about the constitution of groups of\nhagiographic works, as well as to examine potential authorial groupings in a\nvastly anonymous corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03855v1",
            "date": "2020-12-07 17:02:00+00:00",
            "title": "The Lab vs The Crowd: An Investigation into Data Quality for Neural Dialogue Models",
            "authors": [
                "Jos\u00e9 Lopes",
                "Francisco J. Chiyah Garcia",
                "Helen Hastie"
            ],
            "abstract": "Challenges around collecting and processing quality data have hampered\nprogress in data-driven dialogue models. Previous approaches are moving away\nfrom costly, resource-intensive lab settings, where collection is slow but\nwhere the data is deemed of high quality. The advent of crowd-sourcing\nplatforms, such as Amazon Mechanical Turk, has provided researchers with an\nalternative cost-effective and rapid way to collect data. However, the\ncollection of fluid, natural spoken or textual interaction can be challenging,\nparticularly between two crowd-sourced workers. In this study, we compare the\nperformance of dialogue models for the same interaction task but collected in\ntwo different settings: in the lab vs. crowd-sourced. We find that fewer lab\ndialogues are needed to reach similar accuracy, less than half the amount of\nlab data as crowd-sourced data. We discuss the advantages and disadvantages of\neach data collection method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03929v2",
            "date": "2020-12-07 18:58:57+00:00",
            "title": "Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations",
            "authors": [
                "Haode Qi",
                "Lin Pan",
                "Atin Sood",
                "Abhishek Shah",
                "Ladislav Kunc",
                "Mo Yu",
                "Saloni Potdar"
            ],
            "abstract": "Intent detection is a key component of modern goal-oriented dialog systems\nthat accomplish a user task by predicting the intent of users' text input.\nThere are three primary challenges in designing robust and accurate intent\ndetection models. First, typical intent detection models require a large amount\nof labeled data to achieve high accuracy. Unfortunately, in practical scenarios\nit is more common to find small, unbalanced, and noisy datasets. Secondly, even\nwith large training data, the intent detection models can see a different\ndistribution of test data when being deployed in the real world, leading to\npoor accuracy. Finally, a practical intent detection model must be\ncomputationally efficient in both training and single query inference so that\nit can be used continuously and re-trained frequently. We benchmark intent\ndetection methods on a variety of datasets. Our results show that Watson\nAssistant's intent detection model outperforms other commercial solutions and\nis comparable to large pretrained language models while requiring only a\nfraction of computational resources and training data. Watson Assistant\ndemonstrates a higher degree of robustness when the training and test\ndistributions differ.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03942v1",
            "date": "2020-12-07 05:37:32+00:00",
            "title": "CX DB8: A queryable extractive summarizer and semantic search engine",
            "authors": [
                "Allen Roush"
            ],
            "abstract": "Competitive Debate's increasingly technical nature has left competitors\nlooking for tools to accelerate evidence production. We find that the unique\ntype of extractive summarization performed by competitive debaters -\nsummarization with a bias towards a particular target meaning - can be\nperformed using the latest innovations in unsupervised pre-trained text\nvectorization models. We introduce CX_DB8, a queryable word-level extractive\nsummarizer and evidence creation framework, which allows for rapid, biasable\nsummarization of arbitarily sized texts. CX_DB8s usage of the embedding\nframework Flair means that as the underlying models improve, CX_DB8 will also\nimprove. We observe that CX_DB8 also functions as a semantic search engine, and\nhas application as a supplement to traditional \"find\" functionality in programs\nand webpages. CX_DB8 is currently used by competitive debaters and is made\navailable to the public at https://github.com/Hellisotherpeople/CX_DB8",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04080v1",
            "date": "2020-12-07 21:56:45+00:00",
            "title": "A Taxonomy of Empathetic Response Intents in Human Social Conversations",
            "authors": [
                "Anuradha Welivita",
                "Pearl Pu"
            ],
            "abstract": "Open-domain conversational agents or chatbots are becoming increasingly\npopular in the natural language processing community. One of the challenges is\nenabling them to converse in an empathetic manner. Current neural response\ngeneration methods rely solely on end-to-end learning from large scale\nconversation data to generate dialogues. This approach can produce socially\nunacceptable responses due to the lack of large-scale quality data used to\ntrain the neural models. However, recent work has shown the promise of\ncombining dialogue act/intent modelling and neural response generation. This\nhybrid method improves the response quality of chatbots and makes them more\ncontrollable and interpretable. A key element in dialog intent modelling is the\ndevelopment of a taxonomy. Inspired by this idea, we have manually labeled 500\nresponse intents using a subset of a sizeable empathetic dialogue dataset (25K\ndialogues). Our goal is to produce a large-scale taxonomy for empathetic\nresponse intents. Furthermore, using lexical and machine learning methods, we\nautomatically analysed both speaker and listener utterances of the entire\ndataset with identified response intents and 32 emotion categories. Finally, we\nuse information visualization methods to summarize emotional dialogue exchange\npatterns and their temporal progression. These results reveal novel and\nimportant empathy patterns in human-human open-domain conversations and can\nserve as heuristics for hybrid approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04169v1",
            "date": "2020-12-08 02:22:09+00:00",
            "title": "Improving Human-Labeled Data through Dynamic Automatic Conflict Resolution",
            "authors": [
                "David Q. Sun",
                "Hadas Kotek",
                "Christopher Klein",
                "Mayank Gupta",
                "William Li",
                "Jason D. Williams"
            ],
            "abstract": "This paper develops and implements a scalable methodology for (a) estimating\nthe noisiness of labels produced by a typical crowdsourcing semantic annotation\ntask, and (b) reducing the resulting error of the labeling process by as much\nas 20-30% in comparison to other common labeling strategies. Importantly, this\nnew approach to the labeling process, which we name Dynamic Automatic Conflict\nResolution (DACR), does not require a ground truth dataset and is instead based\non inter-project annotation inconsistencies. This makes DACR not only more\naccurate but also available to a broad range of labeling tasks. In what follows\nwe present results from a text classification task performed at scale for a\ncommercial personal assistant, and evaluate the inherent ambiguity uncovered by\nthis annotation strategy as compared to other common labeling strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04194v1",
            "date": "2020-12-08 03:37:50+00:00",
            "title": "Unsupervised Label Refinement Improves Dataless Text Classification",
            "authors": [
                "Zewei Chu",
                "Karl Stratos",
                "Kevin Gimpel"
            ],
            "abstract": "Dataless text classification is capable of classifying documents into\npreviously unseen labels by assigning a score to any document paired with a\nlabel description. While promising, it crucially relies on accurate\ndescriptions of the label set for each downstream task. This reliance causes\ndataless classifiers to be highly sensitive to the choice of label descriptions\nand hinders the broader application of dataless classification in practice. In\nthis paper, we ask the following question: how can we improve dataless text\nclassification using the inputs of the downstream task dataset? Our primary\nsolution is a clustering based approach. Given a dataless classifier, our\napproach refines its set of predictions using k-means clustering. We\ndemonstrate the broad applicability of our approach by improving the\nperformance of two widely used classifier architectures, one that encodes\ntext-category pairs with two independent encoders and one with a single joint\nencoder. Experiments show that our approach consistently improves dataless\nclassification across different datasets and makes the classifier more robust\nto the choice of label descriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04281v1",
            "date": "2020-12-08 08:54:36+00:00",
            "title": "CTRLsum: Towards Generic Controllable Text Summarization",
            "authors": [
                "Junxian He",
                "Wojciech Kry\u015bci\u0144ski",
                "Bryan McCann",
                "Nazneen Rajani",
                "Caiming Xiong"
            ],
            "abstract": "Current summarization systems yield generic summaries that are disconnected\nfrom users' preferences and expectations. To address this limitation, we\npresent CTRLsum, a novel framework for controllable summarization. Our approach\nenables users to control multiple aspects of generated summaries by interacting\nwith the summarization system through textual input in the form of a set of\nkeywords or descriptive prompts. Using a single unified model, CTRLsum is able\nto achieve a broad scope of summary manipulation at inference time without\nrequiring additional human annotations or pre-defining a set of control aspects\nduring training. We quantitatively demonstrate the effectiveness of our\napproach on three domains of summarization datasets and five control aspects:\n1) entity-centric and 2) length-controllable summarization, 3) contribution\nsummarization on scientific papers, 4) invention purpose summarization on\npatent filings, and 5) question-guided summarization on news articles in a\nreading comprehension setting. Moreover, when used in a standard, uncontrolled\nsummarization setting, CTRLsum achieves state-of-the-art results on the\nCNN/DailyMail dataset. Code and model checkpoints are available at\nhttps://github.com/salesforce/ctrl-sum",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04332v1",
            "date": "2020-12-08 10:14:29+00:00",
            "title": "Facts2Story: Controlling Text Generation by Key Facts",
            "authors": [
                "Eyal Orbach",
                "Yoav Goldberg"
            ],
            "abstract": "Recent advancements in self-attention neural network architectures have\nraised the bar for open-ended text generation. Yet, while current methods are\ncapable of producing a coherent text which is several hundred words long,\nattaining control over the content that is being generated -- as well as\nevaluating it -- are still open questions. We propose a controlled generation\ntask which is based on expanding a sequence of facts, expressed in natural\nlanguage, into a longer narrative. We introduce human-based evaluation metrics\nfor this task, as well as a method for deriving a large training dataset. We\nevaluate three methods on this task, based on fine-tuning pre-trained models.\nWe show that while auto-regressive, unidirectional Language Models such as GPT2\nproduce better fluency, they struggle to adhere to the requested facts. We\npropose a plan-and-cloze model (using fine-tuned XLNet) which produces\ncompetitive fluency while adhering to the requested content.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04334v2",
            "date": "2020-12-08 10:16:27+00:00",
            "title": "From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension",
            "authors": [
                "Lingyong Yan",
                "Xianpei Han",
                "Le Sun",
                "Fangchao Liu",
                "Ning Bian"
            ],
            "abstract": "Distant supervision (DS) is a promising approach for relation extraction but\noften suffers from the noisy label problem. Traditional DS methods usually\nrepresent an entity pair as a bag of sentences and denoise labels using\nmulti-instance learning techniques. The bag-based paradigm, however, fails to\nleverage the inter-sentence-level and the entity-level evidence for relation\nextraction, and their denoising algorithms are often specialized and\ncomplicated. In this paper, we propose a new DS paradigm--document-based\ndistant supervision, which models relation extraction as a document-based\nmachine reading comprehension (MRC) task. By re-organizing all sentences about\nan entity as a document and extracting relations via querying the document with\nrelation-specific questions, the document-based DS paradigm can simultaneously\nencode and exploit all sentence-level, inter-sentence-level, and entity-level\nevidence. Furthermore, we design a new loss function--DSLoss (distant\nsupervision loss), which can effectively train MRC models using only\n$\\langle$document, question, answer$\\rangle$ tuples, therefore noisy label\nproblem can be inherently resolved. Experiments show that our method achieves\nnew state-of-the-art DS performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04395v1",
            "date": "2020-12-08 12:24:36+00:00",
            "title": "End-to-End Chinese Parsing Exploiting Lexicons",
            "authors": [
                "Yuan Zhang",
                "Zhiyang Teng",
                "Yue Zhang"
            ],
            "abstract": "Chinese parsing has traditionally been solved by three pipeline systems\nincluding word-segmentation, part-of-speech tagging and dependency parsing\nmodules. In this paper, we propose an end-to-end Chinese parsing model based on\ncharacter inputs which jointly learns to output word segmentation,\npart-of-speech tags and dependency structures. In particular, our parsing model\nrelies on word-char graph attention networks, which can enrich the character\ninputs with external word knowledge. Experiments on three Chinese parsing\nbenchmark datasets show the effectiveness of our models, achieving the\nstate-of-the-art results on end-to-end Chinese parsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04443v1",
            "date": "2020-12-08 14:23:46+00:00",
            "title": "Extractive Opinion Summarization in Quantized Transformer Spaces",
            "authors": [
                "Stefanos Angelidis",
                "Reinald Kim Amplayo",
                "Yoshihiko Suhara",
                "Xiaolan Wang",
                "Mirella Lapata"
            ],
            "abstract": "We present the Quantized Transformer (QT), an unsupervised system for\nextractive opinion summarization. QT is inspired by Vector-Quantized\nVariational Autoencoders, which we repurpose for popularity-driven\nsummarization. It uses a clustering interpretation of the quantized space and a\nnovel extraction algorithm to discover popular opinions among hundreds of\nreviews, a significant step towards opinion summarization of practical scope.\nIn addition, QT enables controllable summarization without further training, by\nutilizing properties of the quantized space to extract aspect-specific\nsummaries. We also make publicly available SPACE, a large-scale evaluation\nbenchmark for opinion summarizers, comprising general and aspect-specific\nsummaries for 50 hotels. Experiments demonstrate the promise of our approach,\nwhich is validated by human studies where judges showed clear preference for\nour method over competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04538v1",
            "date": "2020-12-07 06:38:53+00:00",
            "title": "Big Green at WNUT 2020 Shared Task-1: Relation Extraction as Contextualized Sequence Classification",
            "authors": [
                "Chris Miller",
                "Soroush Vosoughi"
            ],
            "abstract": "Relation and event extraction is an important task in natural language\nprocessing. We introduce a system which uses contextualized knowledge graph\ncompletion to classify relations and events between known entities in a noisy\ntext environment. We report results which show that our system is able to\neffectively extract relations and events from a dataset of wet lab protocols.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04575v1",
            "date": "2020-12-08 17:20:20+00:00",
            "title": "The Role of Interpretable Patterns in Deep Learning for Morphology",
            "authors": [
                "Judit Acs",
                "Andras Kornai"
            ],
            "abstract": "We examine the role of character patterns in three tasks: morphological\nanalysis, lemmatization and copy. We use a modified version of the standard\nsequence-to-sequence model, where the encoder is a pattern matching network.\nEach pattern scores all possible N character long subwords (substrings) on the\nsource side, and the highest scoring subword's score is used to initialize the\ndecoder as well as the input to the attention mechanism. This method allows\nlearning which subwords of the input are important for generating the output.\nBy training the models on the same source but different target, we can compare\nwhat subwords are important for different tasks and how they relate to each\nother. We define a similarity metric, a generalized form of the Jaccard\nsimilarity, and assign a similarity score to each pair of the three tasks that\nwork on the same source but may differ in target. We examine how these three\ntasks are related to each other in 12 languages. Our code is publicly\navailable.",
            "journal ref": "XVI. Magyar Sz\\'am\\'it\\'og\\'epes Nyelv\\'eszeti Konferencia, 2020,\n  page 171-179 (MSZNY2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04778v2",
            "date": "2020-12-08 22:54:35+00:00",
            "title": "Fact-Enhanced Synthetic News Generation",
            "authors": [
                "Kai Shu",
                "Yichuan Li",
                "Kaize Ding",
                "Huan Liu"
            ],
            "abstract": "The advanced text generation methods have witnessed great success in text\nsummarization, language translation, and synthetic news generation. However,\nthese techniques can be abused to generate disinformation and fake news. To\nbetter understand the potential threats of synthetic news, we develop a new\ngeneration method FactGen to generate high-quality news content. The existing\ntext generation methods either afford limited supplementary information or lose\nconsistency between the input and output which makes the synthetic news less\ntrustworthy. To address these issues, FactGen retrieves external facts to\nenrich the output and reconstructs the input claim from the generated content\nto improve the consistency among the input and the output. Experiment results\non real-world datasets show that the generated news contents of FactGen are\nconsistent and contain rich facts. We also discuss the possible defending\nmethod to identify these synthetic news pieces if FactGen is used to generate\nsynthetic news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04796v1",
            "date": "2020-12-09 00:10:57+00:00",
            "title": "On an Unknown Ancestor of Burrows' Delta Measure",
            "authors": [
                "Petr Plechac"
            ],
            "abstract": "This article points out some surprising similarities between a 1944 study by\nGeorgy Udny Yule and modern approaches to authorship attribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04808v2",
            "date": "2020-12-09 00:57:49+00:00",
            "title": "Fusing Context Into Knowledge Graph for Commonsense Question Answering",
            "authors": [
                "Yichong Xu",
                "Chenguang Zhu",
                "Ruochen Xu",
                "Yang Liu",
                "Michael Zeng",
                "Xuedong Huang"
            ],
            "abstract": "Commonsense question answering (QA) requires a model to grasp commonsense and\nfactual knowledge to answer questions about world events. Many prior methods\ncouple language modeling with knowledge graphs (KG). However, although a KG\ncontains rich structural information, it lacks the context to provide a more\nprecise understanding of the concepts. This creates a gap when fusing knowledge\ngraphs into language modeling, especially when there is insufficient labeled\ndata. Thus, we propose to employ external entity descriptions to provide\ncontextual information for knowledge understanding. We retrieve descriptions of\nrelated concepts from Wiktionary and feed them as additional input to\npre-trained language models. The resulting model achieves state-of-the-art\nresult in the CommonsenseQA dataset and the best result among non-generative\nmodels in OpenBookQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04812v1",
            "date": "2020-12-09 01:08:13+00:00",
            "title": "Improving Relation Extraction by Leveraging Knowledge Graph Link Prediction",
            "authors": [
                "George Stoica",
                "Emmanouil Antonios Platanios",
                "Barnab\u00e1s P\u00f3czos"
            ],
            "abstract": "Relation extraction (RE) aims to predict a relation between a subject and an\nobject in a sentence, while knowledge graph link prediction (KGLP) aims to\npredict a set of objects, O, given a subject and a relation from a knowledge\ngraph. These two problems are closely related as their respective objectives\nare intertwined: given a sentence containing a subject and an object o, a RE\nmodel predicts a relation that can then be used by a KGLP model together with\nthe subject, to predict a set of objects O. Thus, we expect object o to be in\nset O. In this paper, we leverage this insight by proposing a multi-task\nlearning approach that improves the performance of RE models by jointly\ntraining on RE and KGLP tasks. We illustrate the generality of our approach by\napplying it on several existing RE models and empirically demonstrate how it\nhelps them achieve consistent performance gains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04821v1",
            "date": "2020-12-09 02:05:00+00:00",
            "title": "Complex Relation Extraction: Challenges and Opportunities",
            "authors": [
                "Haiyun Jiang",
                "Qiaoben Bao",
                "Qiao Cheng",
                "Deqing Yang",
                "Li Wang",
                "Yanghua Xiao"
            ],
            "abstract": "Relation extraction aims to identify the target relations of entities in\ntexts. Relation extraction is very important for knowledge base construction\nand text understanding. Traditional binary relation extraction, including\nsupervised, semi-supervised and distant supervised ones, has been extensively\nstudied and significant results are achieved. In recent years, many complex\nrelation extraction tasks, i.e., the variants of simple binary relation\nextraction, are proposed to meet the complex applications in practice. However,\nthere is no literature to fully investigate and summarize these complex\nrelation extraction works so far. In this paper, we first report the recent\nprogress in traditional simple binary relation extraction. Then we summarize\nthe existing complex relation extraction tasks and present the definition,\nrecent progress, challenges and opportunities for each task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04882v1",
            "date": "2020-12-09 06:09:31+00:00",
            "title": "Infusing Multi-Source Knowledge with Heterogeneous Graph Neural Network for Emotional Conversation Generation",
            "authors": [
                "Yunlong Liang",
                "Fandong Meng",
                "Ying Zhang",
                "Jinan Xu",
                "Yufeng Chen",
                "Jie Zhou"
            ],
            "abstract": "The success of emotional conversation systems depends on sufficient\nperception and appropriate expression of emotions. In a real-world\nconversation, we firstly instinctively perceive emotions from multi-source\ninformation, including the emotion flow of dialogue history, facial\nexpressions, and personalities of speakers, and then express suitable emotions\naccording to our personalities, but these multiple types of information are\ninsufficiently exploited in emotional conversation fields. To address this\nissue, we propose a heterogeneous graph-based model for emotional conversation\ngeneration. Specifically, we design a Heterogeneous Graph-Based Encoder to\nrepresent the conversation content (i.e., the dialogue history, its emotion\nflow, facial expressions, and speakers' personalities) with a heterogeneous\ngraph neural network, and then predict suitable emotions for feedback. After\nthat, we employ an Emotion-Personality-Aware Decoder to generate a response not\nonly relevant to the conversation context but also with appropriate emotions,\nby taking the encoded graph representations, the predicted emotions from the\nencoder and the personality of the current speaker as inputs. Experimental\nresults show that our model can effectively perceive emotions from multi-source\nknowledge and generate a satisfactory response, which significantly outperforms\nprevious state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04946v2",
            "date": "2020-12-09 10:02:09+00:00",
            "title": "Multidimensional scaling and linguistic theory",
            "authors": [
                "Martijn van der Klis",
                "Jos Tellings"
            ],
            "abstract": "This paper reports on the state-of-the-art in the application of\nmultidimensional scaling (MDS) techniques to create semantic maps in linguistic\nresearch. MDS refers to a statistical technique that represents objects\n(lexical items, linguistic contexts, languages, etc.) as points in a space so\nthat close similarity between the objects corresponds to close distances\nbetween the corresponding points in the representation. We focus on the recent\ntrend to apply MDS to parallel corpus data in order to investigate a certain\nlinguistic phenomenon from a cross-linguistic perspective.\n  We first introduce the mathematical foundations of MDS, intended for\nnon-experts, so that readers understand notions such as 'eigenvalues',\n'dimensionality reduction', 'stress values', etc. as they appear in linguistic\nMDS writing.\n  We then give an exhaustive overview of past research that employs MDS\ntechniques in combination with parallel corpus data, and propose a set of\nterminology to succinctly describe the key parameters of a particular MDS\napplication. We go over various research questions that have been answered with\nthe aid of MDS maps, showing that the methodology covers topics in a spectrum\nranging from classic typology (e.g. language classification) to formal\nlinguistics (e.g. study of a phenomenon in a single language).\n  We finally identify two lines of future research that build on the insights\nof earlier MDS research described in the paper. First, we envisage the use of\nMDS in the investigation of cross-linguistic variation of compositional\nstructures, an important area in variation research that has not been\napproached by parallel corpus work yet. Second, we discuss how MDS can be\ncomplemented and compared with other dimensionality reduction techniques that\nhave seen little use in the linguistic domain so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04955v1",
            "date": "2020-12-09 10:18:03+00:00",
            "title": "Breeding Gender-aware Direct Speech Translation Systems",
            "authors": [
                "Marco Gaido",
                "Beatrice Savoldi",
                "Luisa Bentivogli",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "In automatic speech translation (ST), traditional cascade approaches\ninvolving separate transcription and translation steps are giving ground to\nincreasingly competitive and more robust direct solutions. In particular, by\ntranslating speech audio data without intermediate transcription, direct ST\nmodels are able to leverage and preserve essential information present in the\ninput (e.g. speaker's vocal characteristics) that is otherwise lost in the\ncascade framework. Although such ability proved to be useful for gender\ntranslation, direct ST is nonetheless affected by gender bias just like its\ncascade counterpart, as well as machine translation and numerous other natural\nlanguage processing applications. Moreover, direct ST systems that exclusively\nrely on vocal biometric features as a gender cue can be unsuitable and\npotentially harmful for certain users. Going beyond speech signals, in this\npaper we compare different approaches to inform direct ST models about the\nspeaker's gender and test their ability to handle gender translation from\nEnglish into Italian and French. To this aim, we manually annotated large\ndatasets with speakers' gender information and used them for experiments\nreflecting different possible real-world scenarios. Our results show that\ngender-aware direct ST solutions can significantly outperform strong - but\ngender-unaware - direct ST models. In particular, the translation of\ngender-marked words can increase up to 30 points in accuracy while preserving\noverall translation quality.",
            "journal ref": "In Proceedings of the 28th International Conference on\n  Computational Linguistics, Dec 2020, 3951-3964. Online",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04964v1",
            "date": "2020-12-09 10:33:13+00:00",
            "title": "On Knowledge Distillation for Direct Speech Translation",
            "authors": [
                "Marco Gaido",
                "Mattia A. Di Gangi",
                "Matteo Negri",
                "Marco Turchi"
            ],
            "abstract": "Direct speech translation (ST) has shown to be a complex task requiring\nknowledge transfer from its sub-tasks: automatic speech recognition (ASR) and\nmachine translation (MT). For MT, one of the most promising techniques to\ntransfer knowledge is knowledge distillation. In this paper, we compare the\ndifferent solutions to distill knowledge in a sequence-to-sequence task like\nST. Moreover, we analyze eventual drawbacks of this approach and how to\nalleviate them maintaining the benefits in terms of translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04995v1",
            "date": "2020-12-09 11:59:58+00:00",
            "title": "Tracking Interaction States for Multi-Turn Text-to-SQL Semantic Parsing",
            "authors": [
                "Run-Ze Wang",
                "Zhen-Hua Ling",
                "Jing-Bo Zhou",
                "Yu Hu"
            ],
            "abstract": "The task of multi-turn text-to-SQL semantic parsing aims to translate natural\nlanguage utterances in an interaction into SQL queries in order to answer them\nusing a database which normally contains multiple table schemas. Previous\nstudies on this task usually utilized contextual information to enrich\nutterance representations and to further influence the decoding process. While\nthey ignored to describe and track the interaction states which are determined\nby history SQL queries and are related with the intent of current utterance. In\nthis paper, two kinds of interaction states are defined based on schema items\nand SQL keywords separately. A relational graph neural network and a non-linear\nlayer are designed to update the representations of these two states\nrespectively. The dynamic schema-state and SQL-state representations are then\nutilized to decode the SQL query corresponding to current utterance.\nExperimental results on the challenging CoSQL dataset demonstrate the\neffectiveness of our proposed method, which achieves better performance than\nother published methods on the task leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05300v1",
            "date": "2020-12-09 20:22:11+00:00",
            "title": "Cross-lingual Word Sense Disambiguation using mBERT Embeddings with Syntactic Dependencies",
            "authors": [
                "Xingran Zhu"
            ],
            "abstract": "Cross-lingual word sense disambiguation (WSD) tackles the challenge of\ndisambiguating ambiguous words across languages given context. The pre-trained\nBERT embedding model has been proven to be effective in extracting contextual\ninformation of words, and have been incorporated as features into many\nstate-of-the-art WSD systems. In order to investigate how syntactic information\ncan be added into the BERT embeddings to result in both semantics- and\nsyntax-incorporated word embeddings, this project proposes the concatenated\nembeddings by producing dependency parse tress and encoding the relative\nrelationships of words into the input embeddings. Two methods are also proposed\nto reduce the size of the concatenated embeddings. The experimental results\nshow that the high dimensionality of the syntax-incorporated embeddings\nconstitute an obstacle for the classification task, which needs to be further\naddressed in future studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05318v1",
            "date": "2020-12-09 20:59:31+00:00",
            "title": "Normalization of Different Swedish Dialects Spoken in Finland",
            "authors": [
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Niko Partanen",
                "Khalid Alnajjar"
            ],
            "abstract": "Our study presents a dialect normalization method for different Finland\nSwedish dialects covering six regions. We tested 5 different models, and the\nbest model improved the word error rate from 76.45 to 28.58. Contrary to\nresults reported in earlier research on Finnish dialects, we found that\ntraining the model with one word at a time gave best results. We believe this\nis due to the size of the training data available for the model. Our models are\naccessible as a Python package. The study provides important information about\nthe adaptability of these methods in different contexts, and gives important\nbaselines for further study.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05331v1",
            "date": "2020-12-09 21:41:40+00:00",
            "title": "Speech Recognition for Endangered and Extinct Samoyedic languages",
            "authors": [
                "Niko Partanen",
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Tiina Klooster"
            ],
            "abstract": "Our study presents a series of experiments on speech recognition with\nendangered and extinct Samoyedic languages, spoken in Northern and Southern\nSiberia. To best of our knowledge, this is the first time a functional ASR\nsystem is built for an extinct language. We achieve with Kamas language a Label\nError Rate of 15\\%, and conclude through careful error analysis that this\nquality is already very useful as a starting point for refined human\ntranscriptions. Our results with related Nganasan language are more modest,\nwith best model having the error rate of 33\\%. We show, however, through\nexperiments where Kamas training data is enlarged incrementally, that Nganasan\nresults are in line with what is expected under low-resource circumstances of\nthe language. Based on this, we provide recommendations for scenarios in which\nfurther language documentation or archive processing activities could benefit\nfrom modern ASR technology. All training data and processing scripts haven been\npublished on Zenodo with clear licences to ensure further work in this\nimportant topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05395v4",
            "date": "2020-12-10 01:27:24+00:00",
            "title": "Infusing Finetuning with Semantic Dependencies",
            "authors": [
                "Zhaofeng Wu",
                "Hao Peng",
                "Noah A. Smith"
            ],
            "abstract": "For natural language processing systems, two kinds of evidence support the\nuse of text representations from neural language models \"pretrained\" on large\nunannotated corpora: performance on application-inspired benchmarks (Peters et\nal., 2018, inter alia), and the emergence of syntactic abstractions in those\nrepresentations (Tenney et al., 2019, inter alia). On the other hand, the lack\nof grounded supervision calls into question how well these representations can\never capture meaning (Bender and Koller, 2020). We apply novel probes to recent\nlanguage models -- specifically focusing on predicate-argument structure as\noperationalized by semantic dependencies (Ivanova et al., 2012) -- and find\nthat, unlike syntax, semantics is not brought to the surface by today's\npretrained models. We then use convolutional graph encoders to explicitly\nincorporate semantic parses into task-specific finetuning, yielding benefits to\nnatural language understanding (NLU) tasks in the GLUE benchmark. This approach\ndemonstrates the potential for general-purpose (rather than task-specific)\nlinguistic supervision, above and beyond conventional pretraining and\nfinetuning. Several diagnostics help to localize the benefits of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05414v4",
            "date": "2020-12-10 02:21:34+00:00",
            "title": "Rewriter-Evaluator Architecture for Neural Machine Translation",
            "authors": [
                "Yangming Li",
                "Kaisheng Yao"
            ],
            "abstract": "Encoder-decoder has been widely used in neural machine translation (NMT). A\nfew methods have been proposed to improve it with multiple passes of decoding.\nHowever, their full potential is limited by a lack of appropriate termination\npolicies. To address this issue, we present a novel architecture,\nRewriter-Evaluator. It consists of a rewriter and an evaluator. Translating a\nsource sentence involves multiple passes. At every pass, the rewriter produces\na new translation to improve the past translation and the evaluator estimates\nthe translation quality to decide whether to terminate the rewriting process.\nWe also propose prioritized gradient descent (PGD) that facilitates training\nthe rewriter and the evaluator jointly. Though incurring multiple passes of\ndecoding, Rewriter-Evaluator with the proposed PGD method can be trained with a\nsimilar time to that of training encoder-decoder models. We apply the proposed\narchitecture to improve the general NMT models (e.g., Transformer). We conduct\nextensive experiments on two translation tasks, Chinese-English and\nEnglish-German, and show that the proposed architecture notably improves the\nperformances of NMT models and significantly outperforms previous baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05418v3",
            "date": "2020-12-10 02:31:52+00:00",
            "title": "Segmenting Natural Language Sentences via Lexical Unit Analysis",
            "authors": [
                "Yangming Li",
                "Lemao Liu",
                "Shuming Shi"
            ],
            "abstract": "In this work, we present Lexical Unit Analysis (LUA), a framework for general\nsequence segmentation tasks. Given a natural language sentence, LUA scores all\nthe valid segmentation candidates and utilizes dynamic programming (DP) to\nextract the maximum scoring one. LUA enjoys a number of appealing properties\nsuch as inherently guaranteeing the predicted segmentation to be valid and\nfacilitating globally optimal training and inference. Besides, the practical\ntime complexity of LUA can be reduced to linear time, which is very efficient.\nWe have conducted extensive experiments on 5 tasks, including syntactic\nchunking, named entity recognition (NER), slot filling, Chinese word\nsegmentation, and Chinese part-of-speech (POS) tagging, across 15 datasets. Our\nmodels have achieved the state-of-the-art performances on 13 of them. The\nresults also show that the F1 score of identifying long-length segments is\nnotably improved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05426v5",
            "date": "2020-12-10 02:53:59+00:00",
            "title": "Empirical Analysis of Unlabeled Entity Problem in Named Entity Recognition",
            "authors": [
                "Yangming Li",
                "Lemao Liu",
                "Shuming Shi"
            ],
            "abstract": "In many scenarios, named entity recognition (NER) models severely suffer from\nunlabeled entity problem, where the entities of a sentence may not be fully\nannotated. Through empirical studies performed on synthetic datasets, we find\ntwo causes of performance degradation. One is the reduction of annotated\nentities and the other is treating unlabeled entities as negative instances.\nThe first cause has less impact than the second one and can be mitigated by\nadopting pretraining language models. The second cause seriously misguides a\nmodel in training and greatly affects its performances. Based on the above\nobservations, we propose a general approach, which can almost eliminate the\nmisguidance brought by unlabeled entities. The key idea is to use negative\nsampling that, to a large extent, avoids training NER models with unlabeled\nentities. Experiments on synthetic datasets and real-world datasets show that\nour model is robust to unlabeled entity problem and surpasses prior baselines.\nOn well-annotated datasets, our model is competitive with the state-of-the-art\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05628v2",
            "date": "2020-12-10 12:27:16+00:00",
            "title": "As Good as New. How to Successfully Recycle English GPT-2 to Make Models for Other Languages",
            "authors": [
                "Wietse de Vries",
                "Malvina Nissim"
            ],
            "abstract": "Large generative language models have been very successful for English, but\nother languages lag behind, in part due to data and computational limitations.\nWe propose a method that may overcome these problems by adapting existing\npre-trained models to new languages. Specifically, we describe the adaptation\nof English GPT-2 to Italian and Dutch by retraining lexical embeddings without\ntuning the Transformer layers. As a result, we obtain lexical embeddings for\nItalian and Dutch that are aligned with the original English lexical\nembeddings. Additionally, we scale up complexity by transforming relearned\nlexical embeddings of GPT-2 small to the GPT-2 medium embedding space. This\nmethod minimises the amount of training and prevents losing information during\nadaptation that was learned by GPT-2. English GPT-2 models with relearned\nlexical embeddings can generate realistic sentences in Italian and Dutch.\nThough on average these sentences are still identifiable as artificial by\nhumans, they are assessed on par with sentences generated by a GPT-2 model\nfully trained from scratch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05776v1",
            "date": "2020-12-10 16:06:05+00:00",
            "title": "Multi-Sense Language Modelling",
            "authors": [
                "Andrea Lekkas",
                "Peter Schneider-Kamp",
                "Isabelle Augenstein"
            ],
            "abstract": "The effectiveness of a language model is influenced by its token\nrepresentations, which must encode contextual information and handle the same\nword form having a plurality of meanings (polysemy). Currently, none of the\ncommon language modelling architectures explicitly model polysemy. We propose a\nlanguage model which not only predicts the next word, but also its sense in\ncontext. We argue that this higher prediction granularity may be useful for end\ntasks such as assistive writing, and allow for more a precise linking of\nlanguage models with knowledge bases. We find that multi-sense language\nmodelling requires architectures that go beyond standard language models, and\nhere propose a structured prediction framework that decomposes the task into a\nword followed by a sense prediction task. For sense prediction, we utilise a\nGraph Attention Network, which encodes definitions and example uses of word\nsenses. Overall, we find that multi-sense language modelling is a highly\nchallenging task, and suggest that future work focus on the creation of more\nannotated training datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05786v1",
            "date": "2020-12-10 16:22:36+00:00",
            "title": "Exploring Pair-Wise NMT for Indian Languages",
            "authors": [
                "Kartheek Akella",
                "Sai Himal Allu",
                "Sridhar Suresh Ragupathi",
                "Aman Singhal",
                "Zeeshan Khan",
                "Vinay P. Namboodiri",
                "C V Jawahar"
            ],
            "abstract": "In this paper, we address the task of improving pair-wise machine translation\nfor specific low resource Indian languages. Multilingual NMT models have\ndemonstrated a reasonable amount of effectiveness on resource-poor languages.\nIn this work, we show that the performance of these models can be significantly\nimproved upon by using back-translation through a filtered back-translation\nprocess and subsequent fine-tuning on the limited pair-wise language corpora.\nThe analysis in this paper suggests that this method can significantly improve\na multilingual model's performance over its baseline, yielding state-of-the-art\nresults for various Indian languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05879v1",
            "date": "2020-12-10 18:39:26+00:00",
            "title": "Automatic Standardization of Colloquial Persian",
            "authors": [
                "Mohammad Sadegh Rasooli",
                "Farzane Bakhtyari",
                "Fatemeh Shafiei",
                "Mahsa Ravanbakhsh",
                "Chris Callison-Burch"
            ],
            "abstract": "The Iranian Persian language has two varieties: standard and colloquial. Most\nnatural language processing tools for Persian assume that the text is in\nstandard form: this assumption is wrong in many real applications especially\nweb content. This paper describes a simple and effective standardization\napproach based on sequence-to-sequence translation. We design an algorithm for\ngenerating artificial parallel colloquial-to-standard data for learning a\nsequence-to-sequence model. Moreover, we annotate a publicly available\nevaluation data consisting of 1912 sentences from a diverse set of domains. Our\nintrinsic evaluation shows a higher BLEU score of 62.8 versus 61.7 compared to\nan off-the-shelf rule-based standardization model in which the original text\nhas a BLEU score of 46.4. We also show that our model improves\nEnglish-to-Persian machine translation in scenarios for which the training data\nis from colloquial Persian with 1.4 absolute BLEU score difference in the\ndevelopment data, and 0.8 in the test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05958v1",
            "date": "2020-12-10 20:29:34+00:00",
            "title": "Multilingual Transfer Learning for QA Using Translation as Data Augmentation",
            "authors": [
                "Mihaela Bornea",
                "Lin Pan",
                "Sara Rosenthal",
                "Radu Florian",
                "Avirup Sil"
            ],
            "abstract": "Prior work on multilingual question answering has mostly focused on using\nlarge multilingual pre-trained language models (LM) to perform zero-shot\nlanguage-wise learning: train a QA model on English and test on other\nlanguages. In this work, we explore strategies that improve cross-lingual\ntransfer by bringing the multilingual embeddings closer in the semantic space.\nOur first strategy augments the original English training data with machine\ntranslation-generated data. This results in a corpus of multilingual\nsilver-labeled QA pairs that is 14 times larger than the original training set.\nIn addition, we propose two novel strategies, language adversarial training and\nlanguage arbitration framework, which significantly improve the (zero-resource)\ncross-lingual transfer performance and result in LM embeddings that are less\nlanguage-variant. Empirically, we show that the proposed models outperform the\nprevious zero-shot baseline on the recently introduced multilingual MLQA and\nTyDiQA datasets.",
            "journal ref": "AAAI 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06106v1",
            "date": "2020-12-11 03:52:17+00:00",
            "title": "EQG-RACE: Examination-Type Question Generation",
            "authors": [
                "Xin Jia",
                "Wenjie Zhou",
                "Xu Sun",
                "Yunfang Wu"
            ],
            "abstract": "Question Generation (QG) is an essential component of the automatic\nintelligent tutoring systems, which aims to generate high-quality questions for\nfacilitating the reading practice and assessments. However, existing QG\ntechnologies encounter several key issues concerning the biased and unnatural\nlanguage sources of datasets which are mainly obtained from the Web (e.g.\nSQuAD). In this paper, we propose an innovative Examination-type Question\nGeneration approach (EQG-RACE) to generate exam-like questions based on a\ndataset extracted from RACE. Two main strategies are employed in EQG-RACE for\ndealing with discrete answer information and reasoning among long contexts. A\nRough Answer and Key Sentence Tagging scheme is utilized to enhance the\nrepresentations of input. An Answer-guided Graph Convolutional Network (AG-GCN)\nis designed to capture structure information in revealing the inter-sentences\nand intra-sentence relations. Experimental results show a state-of-the-art\nperformance of EQG-RACE, which is apparently superior to the baselines. In\naddition, our work has established a new QG prototype with a reshaped dataset\nand QG method, which provides an important benchmark for related research in\nfuture work. We will make our data and code publicly available for further\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06143v1",
            "date": "2020-12-11 06:03:33+00:00",
            "title": "Document-aligned Japanese-English Conversation Parallel Corpus",
            "authors": [
                "Mat\u012bss Rikters",
                "Ryokan Ri",
                "Tong Li",
                "Toshiaki Nakazawa"
            ],
            "abstract": "Sentence-level (SL) machine translation (MT) has reached acceptable quality\nfor many high-resourced languages, but not document-level (DL) MT, which is\ndifficult to 1) train with little amount of DL data; and 2) evaluate, as the\nmain methods and data sets focus on SL evaluation. To address the first issue,\nwe present a document-aligned Japanese-English conversation corpus, including\nbalanced, high-quality business conversation data for tuning and testing. As\nfor the second issue, we manually identify the main areas where SL MT fails to\nproduce adequate translations in lack of context. We then create an evaluation\nset where these phenomena are annotated to alleviate automatic evaluation of DL\nsystems. We train MT models using our corpus to demonstrate how using context\nleads to improvements.",
            "journal ref": "Proceedings of the Fifth Conference on Machine Translation (2020),\n  pages 637-643",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06153v1",
            "date": "2020-12-11 06:29:58+00:00",
            "title": "Improving Task-Agnostic BERT Distillation with Layer Mapping Search",
            "authors": [
                "Xiaoqi Jiao",
                "Huating Chang",
                "Yichun Yin",
                "Lifeng Shang",
                "Xin Jiang",
                "Xiao Chen",
                "Linlin Li",
                "Fang Wang",
                "Qun Liu"
            ],
            "abstract": "Knowledge distillation (KD) which transfers the knowledge from a large\nteacher model to a small student model, has been widely used to compress the\nBERT model recently. Besides the supervision in the output in the original KD,\nrecent works show that layer-level supervision is crucial to the performance of\nthe student BERT model. However, previous works designed the layer mapping\nstrategy heuristically (e.g., uniform or last-layer), which can lead to\ninferior performance. In this paper, we propose to use the genetic algorithm\n(GA) to search for the optimal layer mapping automatically. To accelerate the\nsearch process, we further propose a proxy setting where a small portion of the\ntraining corpus are sampled for distillation, and three representative tasks\nare chosen for evaluation. After obtaining the optimal layer mapping, we\nperform the task-agnostic BERT distillation with it on the whole corpus to\nbuild a compact student model, which can be directly fine-tuned on downstream\ntasks. Comprehensive experiments on the evaluation benchmarks demonstrate that\n1) layer mapping strategy has a significant effect on task-agnostic BERT\ndistillation and different layer mappings can result in quite different\nperformances; 2) the optimal layer mapping strategy from the proposed search\nprocess consistently outperforms the other heuristic ones; 3) with the optimal\nlayer mapping, our student model achieves state-of-the-art performance on the\nGLUE tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06236v1",
            "date": "2020-12-11 10:52:04+00:00",
            "title": "Improving Zero Shot Learning Baselines with Commonsense Knowledge",
            "authors": [
                "Abhinaba Roy",
                "Deepanway Ghosal",
                "Erik Cambria",
                "Navonil Majumder",
                "Rada Mihalcea",
                "Soujanya Poria"
            ],
            "abstract": "Zero shot learning -- the problem of training and testing on a completely\ndisjoint set of classes -- relies greatly on its ability to transfer knowledge\nfrom train classes to test classes. Traditionally semantic embeddings\nconsisting of human defined attributes (HA) or distributed word embeddings\n(DWE) are used to facilitate this transfer by improving the association between\nvisual and semantic embeddings. In this paper, we take advantage of explicit\nrelations between nodes defined in ConceptNet, a commonsense knowledge graph,\nto generate commonsense embeddings of the class labels by using a graph\nconvolution network-based autoencoder. Our experiments performed on three\nstandard benchmark datasets surpass the strong baselines when we fuse our\ncommonsense embeddings with existing semantic embeddings i.e. HA and DWE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06262v1",
            "date": "2020-12-11 11:55:55+00:00",
            "title": "Morphology Matters: A Multilingual Language Modeling Analysis",
            "authors": [
                "Hyunji Hayley Park",
                "Katherine J. Zhang",
                "Coleman Haley",
                "Kenneth Steimel",
                "Han Liu",
                "Lane Schwartz"
            ],
            "abstract": "Prior studies in multilingual language modeling (e.g., Cotterell et al.,\n2018; Mielke et al., 2019) disagree on whether or not inflectional morphology\nmakes languages harder to model. We attempt to resolve the disagreement and\nextend those studies. We compile a larger corpus of 145 Bible translations in\n92 languages and a larger number of typological features. We fill in missing\ntypological data for several languages and consider corpus-based measures of\nmorphological complexity in addition to expert-produced typological features.\nWe find that several morphological measures are significantly associated with\nhigher surprisal when LSTM models are trained with BPE-segmented data. We also\ninvestigate linguistically-motivated subword segmentation strategies like\nMorfessor and Finite-State Transducers (FSTs) and find that these segmentation\nstrategies yield better performance and reduce the impact of a language's\nmorphology on language modeling.",
            "journal ref": "Transactions of the Association for Computational Linguistics 9\n  (2021) 261-276",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06431v1",
            "date": "2020-12-11 15:46:15+00:00",
            "title": "Discriminating Between Similar Nordic Languages",
            "authors": [
                "Ren\u00e9 Haas",
                "Leon Derczynski"
            ],
            "abstract": "Automatic language identification is a challenging problem. Discriminating\nbetween closely related languages is especially difficult. This paper presents\na machine learning approach for automatic language identification for the\nNordic languages, which often suffer miscategorisation by existing\nstate-of-the-art tools. Concretely we will focus on discrimination between six\nNordic languages: Danish, Swedish, Norwegian (Nynorsk), Norwegian (Bokm{\\aa}l),\nFaroese and Icelandic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06460v1",
            "date": "2020-12-11 16:32:41+00:00",
            "title": "Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer",
            "authors": [
                "Marko Vidoni",
                "Ivan Vuli\u0107",
                "Goran Glava\u0161"
            ],
            "abstract": "Adapter modules, additional trainable parameters that enable efficient\nfine-tuning of pretrained transformers, have recently been used for language\nspecialization of multilingual transformers, improving downstream zero-shot\ncross-lingual transfer. In this work, we propose orthogonal language and task\nadapters (dubbed orthoadapters) for cross-lingual transfer. They are trained to\nencode language- and task-specific information that is complementary (i.e.,\northogonal) to the knowledge already stored in the pretrained transformer's\nparameters. Our zero-shot cross-lingual transfer experiments, involving three\ntasks (POS-tagging, NER, NLI) and a set of 10 diverse languages, 1) point to\nthe usefulness of orthoadapters in cross-lingual transfer, especially for the\nmost complex NLI task, but also 2) indicate that the optimal adapter\nconfiguration highly depends on the task and the target language. We hope that\nour work will motivate a wider investigation of usefulness of orthogonality\nconstraints in language- and task-specific fine-tuning of pretrained\ntransformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06606v1",
            "date": "2020-12-11 19:23:28+00:00",
            "title": "TF-CR: Weighting Embeddings for Text Classification",
            "authors": [
                "Arkaitz Zubiaga"
            ],
            "abstract": "Text classification, as the task consisting in assigning categories to\ntextual instances, is a very common task in information science. Methods\nlearning distributed representations of words, such as word embeddings, have\nbecome popular in recent years as the features to use for text classification\ntasks. Despite the increasing use of word embeddings for text classification,\nthese are generally used in an unsupervised manner, i.e. information derived\nfrom class labels in the training data are not exploited. While word embeddings\ninherently capture the distributional characteristics of words, and contexts\nobserved around them in a large dataset, they aren't optimised to consider the\ndistributions of words across categories in the classification dataset at hand.\nTo optimise text representations based on word embeddings by incorporating\nclass distributions in the training data, we propose the use of weighting\nschemes that assign a weight to embeddings of each word based on its saliency\nin each class. To achieve this, we introduce a novel weighting scheme, Term\nFrequency-Category Ratio (TF-CR), which can weight high-frequency,\ncategory-exclusive words higher when computing word embeddings. Our experiments\non 16 classification datasets show the effectiveness of TF-CR, leading to\nimproved performance scores over existing weighting schemes, with a performance\ngap that increases as the size of the training data grows.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06717v2",
            "date": "2020-12-12 03:52:15+00:00",
            "title": "Mapping the Timescale Organization of Neural Language Models",
            "authors": [
                "Hsiang-Yun Sherry Chien",
                "Jinhan Zhang",
                "Christopher. J. Honey"
            ],
            "abstract": "In the human brain, sequences of language input are processed within a\ndistributed and hierarchical architecture, in which higher stages of processing\nencode contextual information over longer timescales. In contrast, in recurrent\nneural networks which perform natural language processing, we know little about\nhow the multiple timescales of contextual information are functionally\norganized. Therefore, we applied tools developed in neuroscience to map the\n\"processing timescales\" of individual units within a word-level LSTM language\nmodel. This timescale-mapping method assigned long timescales to units\npreviously found to track long-range syntactic dependencies. Additionally, the\nmapping revealed a small subset of the network (less than 15% of units) with\nlong timescales and whose function had not previously been explored. We next\nprobed the functional organization of the network by examining the relationship\nbetween the processing timescale of units and their network connectivity. We\nidentified two classes of long-timescale units: \"controller\" units composed a\ndensely interconnected subnetwork and strongly projected to the rest of the\nnetwork, while \"integrator\" units showed the longest timescales in the network,\nand expressed projection profiles closer to the mean projection profile.\nAblating integrator and controller units affected model performance at\ndifferent positions within a sentence, suggesting distinctive functions of\nthese two sets of units. Finally, we tested the generalization of these results\nto a character-level LSTM model and models with different architectures. In\nsummary, we demonstrated a model-free technique for mapping the timescale\norganization in recurrent neural networks, and we applied this method to reveal\nthe timescale and functional organization of neural language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06754v1",
            "date": "2020-12-12 08:21:08+00:00",
            "title": "SenSeNet: Neural Keyphrase Generation with Document Structure",
            "authors": [
                "Yichao Luo",
                "Zhengyan Li",
                "Bingning Wang",
                "Xiaoyu Xing",
                "Qi Zhang",
                "Xuanjing Huang"
            ],
            "abstract": "Keyphrase Generation (KG) is the task of generating central topics from a\ngiven document or literary work, which captures the crucial information\nnecessary to understand the content. Documents such as scientific literature\ncontain rich meta-sentence information, which represents the logical-semantic\nstructure of the documents. However, previous approaches ignore the constraints\nof document logical structure, and hence they mistakenly generate keyphrases\nfrom unimportant sentences. To address this problem, we propose a new method\ncalled Sentence Selective Network (SenSeNet) to incorporate the meta-sentence\ninductive bias into KG. In SenSeNet, we use a straight-through estimator for\nend-to-end training and incorporate weak supervision in the training of the\nsentence selection module. Experimental results show that SenSeNet can\nconsistently improve the performance of major KG models based on seq2seq\nframework, which demonstrate the effectiveness of capturing structural\ninformation and distinguishing the significance of sentences in KG task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06780v1",
            "date": "2020-12-12 10:43:41+00:00",
            "title": "GDPNet: Refining Latent Multi-View Graph for Relation Extraction",
            "authors": [
                "Fuzhao Xue",
                "Aixin Sun",
                "Hao Zhang",
                "Eng Siong Chng"
            ],
            "abstract": "Relation Extraction (RE) is to predict the relation type of two entities that\nare mentioned in a piece of text, e.g., a sentence or a dialogue. When the\ngiven text is long, it is challenging to identify indicative words for the\nrelation prediction. Recent advances on RE task are from BERT-based sequence\nmodeling and graph-based modeling of relationships among the tokens in the\nsequence. In this paper, we propose to construct a latent multi-view graph to\ncapture various possible relationships among tokens. We then refine this graph\nto select important words for relation prediction. Finally, the representation\nof the refined graph and the BERT-based sequence representation are\nconcatenated for relation extraction. Specifically, in our proposed GDPNet\n(Gaussian Dynamic Time Warping Pooling Net), we utilize Gaussian Graph\nGenerator (GGG) to generate edges of the multi-view graph. The graph is then\nrefined by Dynamic Time Warping Pooling (DTWPool). On DialogRE and TACRED, we\nshow that GDPNet achieves the best performance on dialogue-level RE, and\ncomparable performance with the state-of-the-arts on sentence-level RE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06847v1",
            "date": "2020-12-12 16:02:08+00:00",
            "title": "AffectON: Incorporating Affect Into Dialog Generation",
            "authors": [
                "Zana Bucinca",
                "Yucel Yemez",
                "Engin Erzin",
                "Metin Sezgin"
            ],
            "abstract": "Due to its expressivity, natural language is paramount for explicit and\nimplicit affective state communication among humans. The same linguistic\ninquiry (e.g., How are you?) might induce responses with different affects\ndepending on the affective state of the conversational partner(s) and the\ncontext of the conversation. Yet, most dialog systems do not consider affect as\nconstitutive aspect of response generation. In this paper, we introduce\nAffectON, an approach for generating affective responses during inference. For\ngenerating language in a targeted affect, our approach leverages a\nprobabilistic language model and an affective space. AffectON is language model\nagnostic, since it can work with probabilities generated by any language model\n(e.g., sequence-to-sequence models, neural language models, n-grams). Hence, it\ncan be employed for both affective dialog and affective language generation. We\nexperimented with affective dialog generation and evaluated the generated text\nobjectively and subjectively. For the subjective part of the evaluation, we\ndesigned a custom user interface for rating and provided recommendations for\nthe design of such interfaces. The results, both subjective and objective\ndemonstrate that our approach is successful in pulling the generated language\ntoward the targeted affect, with little sacrifice in syntactic coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06971v1",
            "date": "2020-12-13 05:52:07+00:00",
            "title": "Syntactic representation learning for neural network based TTS with syntactic parse tree traversal",
            "authors": [
                "Changhe Song",
                "Jingbei Li",
                "Yixuan Zhou",
                "Zhiyong Wu",
                "Helen Meng"
            ],
            "abstract": "Syntactic structure of a sentence text is correlated with the prosodic\nstructure of the speech that is crucial for improving the prosody and\nnaturalness of a text-to-speech (TTS) system. Nowadays TTS systems usually try\nto incorporate syntactic structure information with manually designed features\nbased on expert knowledge. In this paper, we propose a syntactic representation\nlearning method based on syntactic parse tree traversal to automatically\nutilize the syntactic structure information. Two constituent label sequences\nare linearized through left-first and right-first traversals from constituent\nparse tree. Syntactic representations are then extracted at word level from\neach constituent label sequence by a corresponding uni-directional gated\nrecurrent unit (GRU) network. Meanwhile, nuclear-norm maximization loss is\nintroduced to enhance the discriminability and diversity of the embeddings of\nconstituent labels. Upsampled syntactic representations and phoneme embeddings\nare concatenated to serve as the encoder input of Tacotron2. Experimental\nresults demonstrate the effectiveness of our proposed approach, with mean\nopinion score (MOS) increasing from 3.70 to 3.82 and ABX preference exceeding\nby 17% compared with the baseline. In addition, for sentences with multiple\nsyntactic parse trees, prosodic differences can be clearly perceived from the\nsynthesized speeches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07019v1",
            "date": "2020-12-13 09:46:24+00:00",
            "title": "Iterative Utterance Segmentation for Neural Semantic Parsing",
            "authors": [
                "Yinuo Guo",
                "Zeqi Lin",
                "Jian-Guang Lou",
                "Dongmei Zhang"
            ],
            "abstract": "Neural semantic parsers usually fail to parse long and complex utterances\ninto correct meaning representations, due to the lack of exploiting the\nprinciple of compositionality. To address this issue, we present a novel\nframework for boosting neural semantic parsers via iterative utterance\nsegmentation. Given an input utterance, our framework iterates between two\nneural modules: a segmenter for segmenting a span from the utterance, and a\nparser for mapping the span into a partial meaning representation. Then, these\nintermediate parsing results are composed into the final meaning\nrepresentation. One key advantage is that this framework does not require any\nhandcraft templates or additional labeled data for utterance segmentation: we\nachieve this through proposing a novel training method, in which the parser\nprovides pseudo supervision for the segmenter. Experiments on Geo,\nComplexWebQuestions, and Formulas show that our framework can consistently\nimprove performances of neural semantic parsers in different domains. On data\nsplits that require compositional generalization, our framework brings\nsignificant accuracy gains: Geo 63.1 to 81.2, Formulas 59.7 to 72.7,\nComplexWebQuestions 27.1 to 56.3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07073v1",
            "date": "2020-12-13 14:45:01+00:00",
            "title": "SPARTA: Speaker Profiling for ARabic TAlk",
            "authors": [
                "Wael Farhan",
                "Muhy Eddin Za'ter",
                "Qusai Abu Obaidah",
                "Hisham al Bataineh",
                "Zyad Sober",
                "Hussein T. Al-Natsheh"
            ],
            "abstract": "This paper proposes a novel approach to an automatic estimation of three\nspeaker traits from Arabic speech: gender, emotion, and dialect. After showing\npromising results on different text classification tasks, the multi-task\nlearning (MTL) approach is used in this paper for Arabic speech classification\ntasks. The dataset was assembled from six publicly available datasets. First,\nThe datasets were edited and thoroughly divided into train, development, and\ntest sets (open to the public), and a benchmark was set for each task and\ndataset throughout the paper. Then, three different networks were explored:\nLong Short Term Memory (LSTM), Convolutional Neural Network (CNN), and\nFully-Connected Neural Network (FCNN) on five different types of features: two\nraw features (MFCC and MEL) and three pre-trained vectors (i-vectors,\nd-vectors, and x-vectors). LSTM and CNN networks were implemented using raw\nfeatures: MFCC and MEL, where FCNN was explored on the pre-trained vectors\nwhile varying the hyper-parameters of these networks to obtain the best results\nfor each dataset and task. MTL was evaluated against the single task learning\n(STL) approach for the three tasks and six datasets, in which the MTL and\npre-trained vectors almost constantly outperformed STL. All the data and\npre-trained models used in this paper are available and can be acquired by the\npublic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07162v2",
            "date": "2020-12-13 21:44:29+00:00",
            "title": "Mask-Align: Self-Supervised Neural Word Alignment",
            "authors": [
                "Chi Chen",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "Word alignment, which aims to align translationally equivalent words between\nsource and target sentences, plays an important role in many natural language\nprocessing tasks. Current unsupervised neural alignment methods focus on\ninducing alignments from neural machine translation models, which does not\nleverage the full context in the target sequence. In this paper, we propose\nMask-Align, a self-supervised word alignment model that takes advantage of the\nfull context on the target side. Our model masks out each target token and\npredicts it conditioned on both source and the remaining target tokens. This\ntwo-step process is based on the assumption that the source token contributing\nmost to recovering the masked target token should be aligned. We also introduce\nan attention variant called leaky attention, which alleviates the problem of\nunexpected high cross-attention weights on special tokens such as periods.\nExperiments on four language pairs show that our model outperforms previous\nunsupervised neural aligners and obtains new state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07280v6",
            "date": "2020-12-14 06:20:27+00:00",
            "title": "Contrastive Learning with Adversarial Perturbations for Conditional Text Generation",
            "authors": [
                "Seanie Lee",
                "Dong Bok Lee",
                "Sung Ju Hwang"
            ],
            "abstract": "Recently, sequence-to-sequence (seq2seq) models with the Transformer\narchitecture have achieved remarkable performance on various conditional text\ngeneration tasks, such as machine translation. However, most of them are\ntrained with teacher forcing with the ground truth label given at each time\nstep, without being exposed to incorrectly generated tokens during training,\nwhich hurts its generalization to unseen inputs, that is known as the \"exposure\nbias\" problem. In this work, we propose to mitigate the conditional text\ngeneration problem by contrasting positive pairs with negative pairs, such that\nthe model is exposed to various valid or incorrect perturbations of the inputs,\nfor improved generalization. However, training the model with naive contrastive\nlearning framework using random non-target sequences as negative examples is\nsuboptimal, since they are easily distinguishable from the correct output,\nespecially so with models pretrained with large text corpora. Also, generating\npositive examples requires domain-specific augmentation heuristics which may\nnot generalize over diverse domains. To tackle this problem, we propose a\nprincipled method to generate positive and negative samples for contrastive\nlearning of seq2seq models. Specifically, we generate negative examples by\nadding small perturbations to the input sequence to minimize its conditional\nlikelihood, and positive examples by adding large perturbations while enforcing\nit to have a high conditional likelihood. Such \"hard\" positive and negative\npairs generated using our method guides the model to better distinguish correct\noutputs from incorrect ones. We empirically show that our proposed method\nsignificantly improves the generalization of the seq2seq on three text\ngeneration tasks - machine translation, text summarization, and question\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07300v1",
            "date": "2020-12-14 07:31:17+00:00",
            "title": "Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders",
            "authors": [
                "Yicheng Zou",
                "Jun Lin",
                "Lujun Zhao",
                "Yangyang Kang",
                "Zhuoren Jiang",
                "Changlong Sun",
                "Qi Zhang",
                "Xuanjing Huang",
                "Xiaozhong Liu"
            ],
            "abstract": "Automatic chat summarization can help people quickly grasp important\ninformation from numerous chat messages. Unlike conventional documents, chat\nlogs usually have fragmented and evolving topics. In addition, these logs\ncontain a quantity of elliptical and interrogative sentences, which make the\nchat summarization highly context dependent. In this work, we propose a novel\nunsupervised framework called RankAE to perform chat summarization without\nemploying manually labeled data. RankAE consists of a topic-oriented ranking\nstrategy that selects topic utterances according to centrality and diversity\nsimultaneously, as well as a denoising auto-encoder that is carefully designed\nto generate succinct but context-informative summaries based on the selected\nutterances. To evaluate the proposed method, we collect a large-scale dataset\nof chat logs from a customer service environment and build an annotated set\nonly for model evaluation. Experimental results show that RankAE significantly\noutperforms other unsupervised methods and is able to generate high-quality\nsummaries in terms of relevance and topic coverage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07311v1",
            "date": "2020-12-14 07:50:25+00:00",
            "title": "Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling",
            "authors": [
                "Yicheng Zou",
                "Lujun Zhao",
                "Yangyang Kang",
                "Jun Lin",
                "Minlong Peng",
                "Zhuoren Jiang",
                "Changlong Sun",
                "Qi Zhang",
                "Xuanjing Huang",
                "Xiaozhong Liu"
            ],
            "abstract": "In a customer service system, dialogue summarization can boost service\nefficiency by automatically creating summaries for long spoken dialogues in\nwhich customers and agents try to address issues about specific topics. In this\nwork, we focus on topic-oriented dialogue summarization, which generates highly\nabstractive summaries that preserve the main ideas from dialogues. In spoken\ndialogues, abundant dialogue noise and common semantics could obscure the\nunderlying informative content, making the general topic modeling approaches\ndifficult to apply. In addition, for customer service, role-specific\ninformation matters and is an indispensable part of a summary. To effectively\nperform topic modeling on dialogues and capture multi-role information, in this\nwork we propose a novel topic-augmented two-stage dialogue summarizer (TDS)\njointly with a saliency-aware neural topic model (SATM) for topic-oriented\nsummarization of customer service dialogues. Comprehensive studies on a\nreal-world Chinese customer service dataset demonstrated the superiority of our\nmethod against several strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07379v1",
            "date": "2020-12-14 10:02:11+00:00",
            "title": "Generating Math Word Problems from Equations with Topic Controlling and Commonsense Enforcement",
            "authors": [
                "Tianyang Cao",
                "Shuang Zeng",
                "Songge Zhao",
                "Mairgup Mansur",
                "Baobao Chang"
            ],
            "abstract": "Recent years have seen significant advancement in text generation tasks with\nthe help of neural language models. However, there exists a challenging task:\ngenerating math problem text based on mathematical equations, which has made\nlittle progress so far. In this paper, we present a novel equation-to-problem\ntext generation model. In our model, 1) we propose a flexible scheme to\neffectively encode math equations, we then enhance the equation encoder by a\nVaritional Autoen-coder (VAE) 2) given a math equation, we perform topic\nselection, followed by which a dynamic topic memory mechanism is introduced to\nrestrict the topic distribution of the generator 3) to avoid commonsense\nviolation in traditional generation model, we pretrain word embedding with\nbackground knowledge graph (KG), and we link decoded words to related words in\nKG, targeted at injecting background knowledge into our model. We evaluate our\nmodel through both automatic metrices and human evaluation, experiments\ndemonstrate our model outperforms baseline and previous models in both accuracy\nand richness of generated problem text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07410v1",
            "date": "2020-12-14 10:58:01+00:00",
            "title": "Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension",
            "authors": [
                "Xiuying Chen",
                "Zhi Cui",
                "Jiayi Zhang",
                "Chen Wei",
                "Jianwei Cui",
                "Bin Wang",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "In multi-turn dialog, utterances do not always take the full form of\nsentences \\cite{Carbonell1983DiscoursePA}, which naturally makes understanding\nthe dialog context more difficult. However, it is essential to fully grasp the\ndialog context to generate a reasonable response. Hence, in this paper, we\npropose to improve the response generation performance by examining the model's\nability to answer a reading comprehension question, where the question is\nfocused on the omitted information in the dialog. Enlightened by the multi-task\nlearning scheme, we propose a joint framework that unifies these two tasks,\nsharing the same encoder to extract the common and task-invariant features with\ndifferent decoders to learn task-specific features. To better fusing\ninformation from the question and the dialog history in the encoding part, we\npropose to augment the Transformer architecture with a memory updater, which is\ndesigned to selectively store and update the history dialog information so as\nto support downstream tasks. For the experiment, we employ human annotators to\nwrite and examine a large-scale dialog reading comprehension dataset. Extensive\nexperiments are conducted on this dataset, and the results show that the\nproposed model brings substantial improvements over several strong baselines on\nboth tasks. In this way, we demonstrate that reasoning can indeed help better\nresponse generation and vice versa. We release our large-scale dataset for\nfurther research.",
            "journal ref": "AAAI 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07419v1",
            "date": "2020-12-14 11:11:43+00:00",
            "title": "The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement",
            "authors": [
                "Mingzhe Li",
                "Xiuying Chen",
                "Min Yang",
                "Shen Gao",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Eye-catching headlines function as the first device to trigger more clicks,\nbringing reciprocal effect between producers and viewers. Producers can obtain\nmore traffic and profits, and readers can have access to outstanding articles.\nWhen generating attractive headlines, it is important to not only capture the\nattractive content but also follow an eye-catching written style. In this\npaper, we propose a Disentanglement-based Attractive Headline Generator (DAHG)\nthat generates headline which captures the attractive content following the\nattractive style. Concretely, we first devise a disentanglement module to\ndivide the style and content of an attractive prototype headline into latent\nspaces, with two auxiliary constraints to ensure the two spaces are indeed\ndisentangled. The latent content information is then used to further polish the\ndocument representation and help capture the salient part. Finally, the\ngenerator takes the polished document as input to generate headline under the\nguidance of the attractive style. Extensive experiments on the public Kuaibao\ndataset show that DAHG achieves state-of-the-art performance. Human evaluation\nalso demonstrates that DAHG triggers 22% more clicks than existing models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07510v1",
            "date": "2020-12-02 16:47:20+00:00",
            "title": "Exploiting BERT to improve aspect-based sentiment analysis performance on Persian language",
            "authors": [
                "H. Jafarian",
                "A. H. Taghavi",
                "A. Javaheri",
                "R. Rawassizadeh"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) is a more detailed task in sentiment\nanalysis, by identifying opinion polarity toward a certain aspect in a text.\nThis method is attracting more attention from the community, due to the fact\nthat it provides more thorough and useful information. However, there are few\nlanguage-specific researches on Persian language. The present research aims to\nimprove the ABSA on the Persian Pars-ABSA dataset. This research shows the\npotential of using pre-trained BERT model and taking advantage of using\nsentence-pair input on an ABSA task. The results indicate that employing\nPars-BERT pre-trained model along with natural language inference auxiliary\nsentence (NLI-M) could boost the ABSA task accuracy up to 91% which is 5.5%\n(absolute) higher than state-of-the-art studies on Pars-ABSA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07516v1",
            "date": "2020-11-30 18:53:30+00:00",
            "title": "Meta learning to classify intent and slot labels with noisy few shot examples",
            "authors": [
                "Shang-Wen Li",
                "Jason Krone",
                "Shuyan Dong",
                "Yi Zhang",
                "Yaser Al-onaizan"
            ],
            "abstract": "Recently deep learning has dominated many machine learning areas, including\nspoken language understanding (SLU). However, deep learning models are\nnotorious for being data-hungry, and the heavily optimized models are usually\nsensitive to the quality of the training examples provided and the consistency\nbetween training and inference conditions. To improve the performance of SLU\nmodels on tasks with noisy and low training resources, we propose a new SLU\nbenchmarking task: few-shot robust SLU, where SLU comprises two core problems,\nintent classification (IC) and slot labeling (SL). We establish the task by\ndefining few-shot splits on three public IC/SL datasets, ATIS, SNIPS, and TOP,\nand adding two types of natural noises (adaptation example missing/replacing\nand modality mismatch) to the splits. We further propose a novel noise-robust\nfew-shot SLU model based on prototypical networks. We show the model\nconsistently outperforms the conventional fine-tuning baseline and another\npopular meta-learning method, Model-Agnostic Meta-Learning (MAML), in terms of\nachieving better IC accuracy and SL F1, and yielding smaller performance\nvariation when noises are present.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07517v1",
            "date": "2020-11-30 16:41:04+00:00",
            "title": "Fake News Detection in Social Media using Graph Neural Networks and NLP Techniques: A COVID-19 Use-case",
            "authors": [
                "Abdullah Hamid",
                "Nasrullah Shiekh",
                "Naina Said",
                "Kashif Ahmad",
                "Asma Gul",
                "Laiq Hassan",
                "Ala Al-Fuqaha"
            ],
            "abstract": "The paper presents our solutions for the MediaEval 2020 task namely FakeNews:\nCorona Virus and 5G Conspiracy Multimedia Twitter-Data-Based Analysis. The task\naims to analyze tweets related to COVID-19 and 5G conspiracy theories to detect\nmisinformation spreaders. The task is composed of two sub-tasks namely (i)\ntext-based, and (ii) structure-based fake news detection. For the first task,\nwe propose six different solutions relying on Bag of Words (BoW) and BERT\nembedding. Three of the methods aim at binary classification task by\ndifferentiating in 5G conspiracy and the rest of the COVID-19 related tweets\nwhile the rest of them treat the task as ternary classification problem. In the\nternary classification task, our BoW and BERT based methods obtained an\nF1-score of .606% and .566% on the development set, respectively. On the binary\nclassification, the BoW and BERT based solutions obtained an average F1-score\nof .666% and .693%, respectively. On the other hand, for structure-based fake\nnews detection, we rely on Graph Neural Networks (GNNs) achieving an average\nROC of .95% on the development set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07521v1",
            "date": "2020-11-30 07:46:21+00:00",
            "title": "Procode: the Swiss Multilingual Solution for Automatic Coding and Recoding of Occupations and Economic Activities",
            "authors": [
                "Nenad Savic",
                "Nicolas Bovio",
                "Fabian Gilbert",
                "Irina Guseva Canu"
            ],
            "abstract": "Objective. Epidemiological studies require data that are in alignment with\nthe classifications established for occupations or economic activities. The\nclassifications usually include hundreds of codes and titles. Manual coding of\nraw data may result in misclassification and be time consuming. The goal was to\ndevelop and test a web-tool, named Procode, for coding of free-texts against\nclassifications and recoding between different classifications. Methods. Three\ntext classifiers, i.e. Complement Naive Bayes (CNB), Support Vector Machine\n(SVM) and Random Forest Classifier (RFC), were investigated using a k-fold\ncross-validation. 30 000 free-texts with manually assigned classification codes\nof French classification of occupations (PCS) and French classification of\nactivities (NAF) were available. For recoding, Procode integrated a workflow\nthat converts codes of one classification to another according to existing\ncrosswalks. Since this is a straightforward operation, only the recoding time\nwas measured. Results. Among the three investigated text classifiers, CNB\nresulted in the best performance, where the classifier predicted accurately\n57-81% and 63-83% classification codes for PCS and NAF, respectively. SVM lead\nto somewhat lower results (by 1-2%), while RFC coded accurately up to 30% of\nthe data. The coding operation required one minute per 10 000 records, while\nthe recoding was faster, i.e. 5-10 seconds. Conclusion. The algorithm\nintegrated in Procode showed satisfactory performance, since the tool had to\nassign the right code by choosing between 500-700 different choices. Based on\nthe results, the authors decided to implement CNB in Procode. In future, if\nanother classifier shows a superior performance, an update will include the\nrequired modifications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07528v1",
            "date": "2020-11-28 12:12:17+00:00",
            "title": "Disentangling Homophemes in Lip Reading using Perplexity Analysis",
            "authors": [
                "Souheil Fenghour",
                "Daqing Chen",
                "Kun Guo",
                "Perry Xiao"
            ],
            "abstract": "The performance of automated lip reading using visemes as a classification\nschema has achieved less success compared with the use of ASCII characters and\nwords largely due to the problem of different words sharing identical visemes.\nThe Generative Pre-Training transformer is an effective autoregressive language\nmodel used for many tasks in Natural Language Processing, including sentence\nprediction and text classification.\n  This paper proposes a new application for this model and applies it in the\ncontext of lip reading, where it serves as a language model to convert visual\nspeech in the form of visemes, to language in the form of words and sentences.\nThe network uses the search for optimal perplexity to perform the\nviseme-to-word mapping and is thus a solution to the one-to-many mapping\nproblem that exists whereby various words that sound different when spoken look\nidentical. This paper proposes a method to tackle the one-to-many mapping\nproblem when performing automated lip reading using solely visual cues in two\nseparate scenarios: the first scenario is where the word boundary, that is, the\nbeginning and the ending of a word, is unknown; and the second scenario is\nwhere the boundary is known.\n  Sentences from the benchmark BBC dataset \"Lip Reading Sentences in the\nWild\"(LRS2), are classified with a character error rate of 10.7% and a word\nerror rate of 18.0%. The main contribution of this paper is to propose a method\nof predicting words through the use of perplexity analysis when only visual\ncues are present, using an autoregressive language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07538v1",
            "date": "2020-12-03 10:21:11+00:00",
            "title": "Sentiment analysis in Bengali via transfer learning using multi-lingual BERT",
            "authors": [
                "Khondoker Ittehadul Islam",
                "Md. Saiful Islam",
                "Md Ruhul Amin"
            ],
            "abstract": "Sentiment analysis (SA) in Bengali is challenging due to this Indo-Aryan\nlanguage's highly inflected properties with more than 160 different inflected\nforms for verbs and 36 different forms for noun and 24 different forms for\npronouns. The lack of standard labeled datasets in the Bengali domain makes the\ntask of SA even harder. In this paper, we present manually tagged 2-class and\n3-class SA datasets in Bengali. We also demonstrate that the multi-lingual BERT\nmodel with relevant extensions can be trained via the approach of transfer\nlearning over those novel datasets to improve the state-of-the-art performance\nin sentiment classification tasks. This deep learning model achieves an\naccuracy of 71\\% for 2-class sentiment classification compared to the current\nstate-of-the-art accuracy of 68\\%. We also present the very first Bengali SA\nclassifier for the 3-class manually tagged dataset, and our proposed model\nachieves an accuracy of 60\\%. We further use this model to analyze the\nsentiment of public comments in the online daily newspaper. Our analysis shows\nthat people post negative comments for political or sports news more often,\nwhile the religious article comments represent positive sentiment. The dataset\nand code is publicly available at\nhttps://github.com/KhondokerIslam/Bengali\\_Sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07637v1",
            "date": "2020-12-11 10:32:35+00:00",
            "title": "The Complexity of Comparative Text Analysis -- \"The Gardener is always the Murderer\" says the Fourth Machine",
            "authors": [
                "Marcus Weber",
                "Konstantin Fackeldey"
            ],
            "abstract": "There is a heated debate about how far computers can map the complexity of\ntext analysis compared to the abilities of the whole team of human researchers.\nA \"deep\" analysis of a given text is still beyond the possibilities of modern\ncomputers.\n  In the heart of the existing computational text analysis algorithms there are\noperations with real numbers, such as additions and multiplications according\nto the rules of algebraic fields. However, the process of \"comparing\" has a\nvery precise mathematical structure, which is different from the structure of\nan algebraic field. The mathematical structure of \"comparing\" can be expressed\nby using Boolean rings. We build on this structure and define the corresponding\nalgebraic equations lifting algorithms of comparative text analysis onto the\n\"correct\" algebraic basis. From this point of view, we can investigate the\nquestion of {\\em computational} complexity of comparative text analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07808v1",
            "date": "2020-12-14 18:41:58+00:00",
            "title": "Unsupervised Opinion Summarization with Content Planning",
            "authors": [
                "Reinald Kim Amplayo",
                "Stefanos Angelidis",
                "Mirella Lapata"
            ],
            "abstract": "The recent success of deep learning techniques for abstractive summarization\nis predicated on the availability of large-scale datasets. When summarizing\nreviews (e.g., for products or movies), such training data is neither available\nnor can be easily sourced, motivating the development of methods which rely on\nsynthetic datasets for supervised training. We show that explicitly\nincorporating content planning in a summarization model not only yields output\nof higher quality, but also allows the creation of synthetic datasets which are\nmore natural, resembling real world document-summary pairs. Our content plans\ntake the form of aspect and sentiment distributions which we induce from data\nwithout access to expensive annotations. Synthetic datasets are created by\nsampling pseudo-reviews from a Dirichlet distribution parametrized by our\ncontent planner, while our model generates summaries based on input reviews and\ninduced content plans. Experimental results on three domains show that our\napproach outperforms competitive models in generating informative, coherent,\nand fluent summaries that capture opinion consensus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07978v1",
            "date": "2020-12-14 22:27:18+00:00",
            "title": "Model Choices Influence Attributive Word Associations: A Semi-supervised Analysis of Static Word Embeddings",
            "authors": [
                "Geetanjali Bihani",
                "Julia Taylor Rayz"
            ],
            "abstract": "Static word embeddings encode word associations, extensively utilized in\ndownstream NLP tasks. Although prior studies have discussed the nature of such\nword associations in terms of biases and lexical regularities captured, the\nvariation in word associations based on the embedding training procedure\nremains in obscurity. This work aims to address this gap by assessing\nattributive word associations across five different static word embedding\narchitectures, analyzing the impact of the choice of the model architecture,\ncontext learning flavor and training corpora. Our approach utilizes a\nsemi-supervised clustering method to cluster annotated proper nouns and\nadjectives, based on their word embedding features, revealing underlying\nattributive word associations formed in the embedding space, without\nintroducing any confirmation bias. Our results reveal that the choice of the\ncontext learning flavor during embedding training (CBOW vs skip-gram) impacts\nthe word association distinguishability and word embeddings' sensitivity to\ndeviations in the training corpora. Moreover, it is empirically shown that even\nwhen trained over the same corpora, there is significant inter-model disparity\nand intra-model similarity in the encoded word associations across different\nword embedding models, portraying specific patterns in the way the embedding\nspace is created for each embedding architecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08012v1",
            "date": "2020-12-14 23:50:20+00:00",
            "title": "Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision",
            "authors": [
                "Faeze Brahman",
                "Vered Shwartz",
                "Rachel Rudinger",
                "Yejin Choi"
            ],
            "abstract": "The black-box nature of neural models has motivated a line of research that\naims to generate natural language rationales to explain why a model made\ncertain predictions. Such rationale generation models, to date, have been\ntrained on dataset-specific crowdsourced rationales, but this approach is\ncostly and is not generalizable to new tasks and domains. In this paper, we\ninvestigate the extent to which neural models can reason about natural language\nrationales that explain model predictions, relying only on distant supervision\nwith no additional annotation cost for human-written rationales. We investigate\nmultiple ways to automatically generate rationales using pre-trained language\nmodels, neural knowledge models, and distant supervision from related tasks,\nand train generative models capable of composing explanatory rationales for\nunseen instances. We demonstrate our approach on the defeasible inference task,\na nonmonotonic reasoning task in which an inference may be strengthened or\nweakened when new information (an update) is introduced. Our model shows\npromises at generating post-hoc rationales explaining why an inference is more\nor less likely given the additional information, however, it mostly generates\ntrivial rationales reflecting the fundamental limitations of neural language\nmodels. Conversely, the more realistic setup of jointly predicting the update\nor its type and generating rationale is more challenging, suggesting an\nimportant future direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08117v1",
            "date": "2020-12-15 06:39:54+00:00",
            "title": "Writing Polishment with Simile: Task, Dataset and A Neural Approach",
            "authors": [
                "Jiayi Zhang",
                "Zhi Cui",
                "Xiaoqiang Xia",
                "Yalong Guo",
                "Yanran Li",
                "Chen Wei",
                "Jianwei Cui"
            ],
            "abstract": "A simile is a figure of speech that directly makes a comparison, showing\nsimilarities between two different things, e.g. \"Reading papers can be dull\nsometimes,like watching grass grow\". Human writers often interpolate\nappropriate similes into proper locations of the plain text to vivify their\nwritings. However, none of existing work has explored neural simile\ninterpolation, including both locating and generation. In this paper, we\npropose a new task of Writing Polishment with Simile (WPS) to investigate\nwhether machines are able to polish texts with similes as we human do.\nAccordingly, we design a two-staged Locate&Gen model based on transformer\narchitecture. Our model firstly locates where the simile interpolation should\nhappen, and then generates a location-specific simile. We also release a\nlarge-scale Chinese Simile (CS) dataset containing 5 million similes with\ncontext. The experimental results demonstrate the feasibility of WPS task and\nshed light on the future research directions towards better automatic text\npolishment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08150v1",
            "date": "2020-12-15 08:43:07+00:00",
            "title": "Learning to Check Contract Inconsistencies",
            "authors": [
                "Shuo Zhang",
                "Junzhou Zhao",
                "Pinghui Wang",
                "Nuo Xu",
                "Yang Yang",
                "Yiting Liu",
                "Yi Huang",
                "Junlan Feng"
            ],
            "abstract": "Contract consistency is important in ensuring the legal validity of the\ncontract. In many scenarios, a contract is written by filling the blanks in a\nprecompiled form. Due to carelessness, two blanks that should be filled with\nthe same (or different)content may be incorrectly filled with different (or\nsame) content. This will result in the issue of contract inconsistencies, which\nmay severely impair the legal validity of the contract. Traditional methods to\naddress this issue mainly rely on manual contract review, which is\nlabor-intensive and costly. In this work, we formulate a novel Contract\nInconsistency Checking (CIC) problem, and design an end-to-end framework,\ncalled Pair-wise Blank Resolution (PBR), to solve the CIC problem with high\naccuracy. Our PBR model contains a novel BlankCoder to address the challenge of\nmodeling meaningless blanks. BlankCoder adopts a two-stage attention mechanism\nthat adequately associates a meaningless blank with its relevant descriptions\nwhile avoiding the incorporation of irrelevant context words. Experiments\nconducted on real-world datasets show the promising performance of our method\nwith a balanced accuracy of 94.05% and an F1 score of 90.90% in the CIC\nproblem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08396v1",
            "date": "2020-12-15 16:12:04+00:00",
            "title": "Modeling Homophone Noise for Robust Neural Machine Translation",
            "authors": [
                "Wenjie Qin",
                "Xiang Li",
                "Yuhui Sun",
                "Deyi Xiong",
                "Jianwei Cui",
                "Bin Wang"
            ],
            "abstract": "In this paper, we propose a robust neural machine translation (NMT)\nframework. The framework consists of a homophone noise detector and a\nsyllable-aware NMT model to homophone errors. The detector identifies potential\nhomophone errors in a textual sentence and converts them into syllables to form\na mixed sequence that is then fed into the syllable-aware NMT. Extensive\nexperiments on Chinese->English translation demonstrate that our proposed\nmethod not only significantly outperforms baselines on noisy test sets with\nhomophone noise, but also achieves a substantial improvement on clean text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08549v1",
            "date": "2020-12-15 19:02:15+00:00",
            "title": "Exploring Transfer Learning For End-to-End Spoken Language Understanding",
            "authors": [
                "Subendhu Rongali",
                "Beiye Liu",
                "Liwei Cai",
                "Konstantine Arkoudas",
                "Chengwei Su",
                "Wael Hamza"
            ],
            "abstract": "Voice Assistants such as Alexa, Siri, and Google Assistant typically use a\ntwo-stage Spoken Language Understanding pipeline; first, an Automatic Speech\nRecognition (ASR) component to process customer speech and generate text\ntranscriptions, followed by a Natural Language Understanding (NLU) component to\nmap transcriptions to an actionable hypothesis. An end-to-end (E2E) system that\ngoes directly from speech to a hypothesis is a more attractive option. These\nsystems were shown to be smaller, faster, and better optimized. However, they\nrequire massive amounts of end-to-end training data and in addition, don't take\nadvantage of the already available ASR and NLU training data.\n  In this work, we propose an E2E system that is designed to jointly train on\nmultiple speech-to-text tasks, such as ASR (speech-transcription) and SLU\n(speech-hypothesis), and text-to-text tasks, such as NLU (text-hypothesis). We\ncall this the Audio-Text All-Task (AT-AT) Model and we show that it beats the\nperformance of E2E models trained on individual tasks, especially ones trained\non limited data. We show this result on an internal music dataset and two\npublic datasets, FluentSpeech and SNIPS Audio, where we achieve\nstate-of-the-art results. Since our model can process both speech and text\ninput sequences and learn to predict a target sequence, it also allows us to do\nzero-shot E2E SLU by training on only text-hypothesis data (without any speech)\nfrom a new domain. We evaluate this ability of our model on the Facebook TOP\ndataset and set a new benchmark for zeroshot E2E performance. We will soon\nrelease the audio data collected for the TOP dataset for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08561v1",
            "date": "2020-12-15 19:17:33+00:00",
            "title": "Pre-Training Transformers as Energy-Based Cloze Models",
            "authors": [
                "Kevin Clark",
                "Minh-Thang Luong",
                "Quoc V. Le",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce Electric, an energy-based cloze model for representation\nlearning over text. Like BERT, it is a conditional generative model of tokens\ngiven their contexts. However, Electric does not use masking or output a full\ndistribution over tokens that could occur in a context. Instead, it assigns a\nscalar energy score to each input token indicating how likely it is given its\ncontext. We train Electric using an algorithm based on noise-contrastive\nestimation and elucidate how this learning objective is closely related to the\nrecently proposed ELECTRA pre-training method. Electric performs well when\ntransferred to downstream tasks and is particularly effective at producing\nlikelihood scores for text: it re-ranks speech recognition n-best lists better\nthan language models and much faster than masked language models. Furthermore,\nit offers a clearer and more principled view of what ELECTRA learns during\npre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08789v1",
            "date": "2020-12-16 08:21:51+00:00",
            "title": "Focusing More on Conflicts with Mis-Predictions Helps Language Pre-Training",
            "authors": [
                "Chen Xing",
                "Wencong Xiao",
                "Yong Li",
                "Wei Lin"
            ],
            "abstract": "In this work, we propose to improve the effectiveness of language\npre-training methods with the help of mis-predictions during pre-training.\nNeglecting words in the input sentence that have conflicting semantics with\nmis-predictions is likely to be the reason of generating mis-predictions at\npre-training. Therefore, we hypothesis that mis-predictions during pre-training\ncan act as detectors of the ill focuses of the model. If we train the model to\nfocus more on the conflicts with the mis-predictions while focus less on the\nrest words in the input sentence, the mis-predictions can be more easily\ncorrected and the entire model could be better trained. Towards this end, we\nintroduce Focusing Less on Context of Mis-predictions(McMisP). In McMisP, we\nrecord the co-occurrence information between words to detect the conflicting\nwords with mis-predictions in an unsupervised way. Then McMisP uses such\ninformation to guide the attention modules when a mis-prediction occurs.\nSpecifically, several attention modules in the Transformer are optimized to\nfocus more on words in the input sentence that have co-occurred rarely with the\nmis-predictions and vice versa. Results show that McMisP significantly\nexpedites BERT and ELECTRA and improves their performances on downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08844v2",
            "date": "2020-12-16 10:34:37+00:00",
            "title": "A Lightweight Neural Model for Biomedical Entity Linking",
            "authors": [
                "Lihu Chen",
                "Ga\u00ebl Varoquaux",
                "Fabian M. Suchanek"
            ],
            "abstract": "Biomedical entity linking aims to map biomedical mentions, such as diseases\nand drugs, to standard entities in a given knowledge base. The specific\nchallenge in this context is that the same biomedical entity can have a wide\nrange of names, including synonyms, morphological variations, and names with\ndifferent word orderings. Recently, BERT-based methods have advanced the\nstate-of-the-art by allowing for rich representations of word sequences.\nHowever, they often have hundreds of millions of parameters and require heavy\ncomputing resources, which limits their applications in resource-limited\nscenarios. Here, we propose a lightweight neural method for biomedical entity\nlinking, which needs just a fraction of the parameters of a BERT model and much\nless computing resources. Our method uses a simple alignment layer with\nattention mechanisms to capture the variations between mention and entity\nnames. Yet, we show that our model is competitive with previous work on\nstandard evaluation benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08958v2",
            "date": "2020-12-16 13:57:31+00:00",
            "title": "Costs to Consider in Adopting NLP for Your Business",
            "authors": [
                "Made Nindyatama Nityasya",
                "Haryo Akbarianto Wibowo",
                "Radityo Eko Prasojo",
                "Alham Fikri Aji"
            ],
            "abstract": "Recent advances in Natural Language Processing (NLP) have largely pushed deep\ntransformer-based models as the go-to state-of-the-art technique without much\nregard to the production and utilization cost. Companies planning to adopt\nthese methods into their business face difficulties because of the lack of\nmachine, data, and human resources to build them. We compare both the\nperformance and the cost of classical learning algorithms to the latest ones in\ncommon sequence and text labeling tasks. In our industrial datasets, we find\nthat classical models often perform on par with deep neural ones despite the\nlower cost. We show the trade-off between performance gain and the cost across\nthe models to give more insights for AI-pivoting business. Further, we call for\nmore research into low-cost models, especially for under-resourced languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09035v1",
            "date": "2020-12-16 15:53:02+00:00",
            "title": "Show or Tell? Demonstration is More Robust to Changes in Shared Perception than Explanation",
            "authors": [
                "Theodore R. Sumers",
                "Mark K. Ho",
                "Thomas L. Griffiths"
            ],
            "abstract": "Successful teaching entails a complex interaction between a teacher and a\nlearner. The teacher must select and convey information based on what they\nthink the learner perceives and believes. Teaching always involves misaligned\nbeliefs, but studies of pedagogy often focus on situations where teachers and\nlearners share perceptions. Nonetheless, a teacher and learner may not always\nexperience or attend to the same aspects of the environment. Here, we study how\nmisaligned perceptions influence communication. We hypothesize that the\nefficacy of different forms of communication depends on the shared perceptual\nstate between teacher and learner. We develop a cooperative teaching game to\ntest whether concrete mediums (demonstrations, or \"showing\") are more robust\nthan abstract ones (language, or \"telling\") when the teacher and learner are\nnot perceptually aligned. We find evidence that (1) language-based teaching is\nmore affected by perceptual misalignment, but (2) demonstration-based teaching\nis less likely to convey nuanced information. We discuss implications for human\npedagogy and machine learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09118v2",
            "date": "2020-12-16 18:01:04+00:00",
            "title": "Exploring Thematic Coherence in Fake News",
            "authors": [
                "Martins Samuel Dogo",
                "Deepak P",
                "Anna Jurek-Loughrey"
            ],
            "abstract": "The spread of fake news remains a serious global issue; understanding and\ncurtailing it is paramount. One way of differentiating between deceptive and\ntruthful stories is by analyzing their coherence. This study explores the use\nof topic models to analyze the coherence of cross-domain news shared online.\nExperimental results on seven cross-domain datasets demonstrate that fake news\nshows a greater thematic deviation between its opening sentences and its\nremainder.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09370v1",
            "date": "2020-12-17 02:49:46+00:00",
            "title": "InSRL: A Multi-view Learning Framework Fusing Multiple Information Sources for Distantly-supervised Relation Extraction",
            "authors": [
                "Zhendong Chu",
                "Haiyun Jiang",
                "Yanghua Xiao",
                "Wei Wang"
            ],
            "abstract": "Distant supervision makes it possible to automatically label bags of\nsentences for relation extraction by leveraging knowledge bases, but suffers\nfrom the sparse and noisy bag issues. Additional information sources are\nurgently needed to supplement the training data and overcome these issues. In\nthis paper, we introduce two widely-existing sources in knowledge bases, namely\nentity descriptions, and multi-grained entity types to enrich the distantly\nsupervised data. We see information sources as multiple views and fusing them\nto construct an intact space with sufficient information. An end-to-end\nmulti-view learning framework is proposed for relation extraction via Intact\nSpace Representation Learning (InSRL), and the representations of single views\nare jointly learned simultaneously. Moreover, inner-view and cross-view\nattention mechanisms are used to highlight important information on different\nlevels on an entity-pair basis. The experimental results on a popular benchmark\ndataset demonstrate the necessity of additional information sources and the\neffectiveness of our framework. We will release the implementation of our model\nand dataset with multiple information sources after the anonymized review\nphase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09411v1",
            "date": "2020-12-17 06:38:04+00:00",
            "title": "Interactive Question Clarification in Dialogue via Reinforcement Learning",
            "authors": [
                "Xiang Hu",
                "Zujie Wen",
                "Yafang Wang",
                "Xiaolong Li",
                "Gerard de Melo"
            ],
            "abstract": "Coping with ambiguous questions has been a perennial problem in real-world\ndialogue systems. Although clarification by asking questions is a common form\nof human interaction, it is hard to define appropriate questions to elicit more\nspecific intents from a user. In this work, we propose a reinforcement model to\nclarify ambiguous questions by suggesting refinements of the original query. We\nfirst formulate a collection partitioning problem to select a set of labels\nenabling us to distinguish potential unambiguous intents. We list the chosen\nlabels as intent phrases to the user for further confirmation. The selected\nlabel along with the original user query then serves as a refined query, for\nwhich a suitable response can more easily be identified. The model is trained\nusing reinforcement learning with a deep policy network. We evaluate our model\nbased on real-world user clicks and demonstrate significant improvements across\nseveral different experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09647v2",
            "date": "2020-12-17 14:54:59+00:00",
            "title": "Ultra-Fast, Low-Storage, Highly Effective Coarse-grained Selection in Retrieval-based Chatbot by Using Deep Semantic Hashing",
            "authors": [
                "Tian Lan",
                "Xian-Ling Mao",
                "Xiaoyan Gao",
                "Wei Wei",
                "Heyan Huang"
            ],
            "abstract": "We study the coarse-grained selection module in retrieval-based chatbot.\nCoarse-grained selection is a basic module in a retrieval-based chatbot, which\nconstructs a rough candidate set from the whole database to speed up the\ninteraction with customers. So far, there are two kinds of approaches for\ncoarse-grained selection module: (1) sparse representation; (2) dense\nrepresentation. To the best of our knowledge, there is no systematic comparison\nbetween these two approaches in retrieval-based chatbots, and which kind of\nmethod is better in real scenarios is still an open question. In this paper, we\nfirst systematically compare these two methods from four aspects: (1)\neffectiveness; (2) index stoarge; (3) search time cost; (4) human evaluation.\nExtensive experiment results demonstrate that dense representation method\nsignificantly outperforms the sparse representation, but costs more time and\nstorage occupation. In order to overcome these fatal weaknesses of dense\nrepresentation method, we propose an ultra-fast, low-storage, and highly\neffective Deep Semantic Hashing Coarse-grained selection method, called DSHC\nmodel. Specifically, in our proposed DSHC model, a hashing optimizing module\nthat consists of two autoencoder models is stacked on a trained dense\nrepresentation model, and three loss functions are designed to optimize it. The\nhash codes provided by hashing optimizing module effectively preserve the rich\nsemantic and similarity information in dense vectors. Extensive experiment\nresults prove that, our proposed DSHC model can achieve much faster speed and\nlower storage than sparse representation, with limited performance loss\ncompared with dense representation. Besides, our source codes have been\npublicly released for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09686v1",
            "date": "2020-12-17 15:53:54+00:00",
            "title": "Hate Speech detection in the Bengali language: A dataset and its baseline evaluation",
            "authors": [
                "Nauros Romim",
                "Mosahed Ahmed",
                "Hriteshwar Talukder",
                "Md Saiful Islam"
            ],
            "abstract": "Social media sites such as YouTube and Facebook have become an integral part\nof everyone's life and in the last few years, hate speech in the social media\ncomment section has increased rapidly. Detection of hate speech on social media\nwebsites faces a variety of challenges including small imbalanced data sets,\nthe findings of an appropriate model and also the choice of feature analysis\nmethod. further more, this problem is more severe for the Bengali speaking\ncommunity due to the lack of gold standard labelled datasets. This paper\npresents a new dataset of 30,000 user comments tagged by crowd sourcing and\nvarified by experts. All the comments are collected from YouTube and Facebook\ncomment section and classified into seven categories: sports, entertainment,\nreligion, politics, crime, celebrity and TikTok & meme. A total of 50\nannotators annotated each comment three times and the majority vote was taken\nas the final annotation. Nevertheless, we have conducted base line experiments\nand several deep learning models along with extensive pre-trained Bengali word\nembedding such as Word2Vec, FastText and BengFastText on this dataset to\nfacilitate future research opportunities. The experiment illustrated that\nalthough all deep learning models performed well, SVM achieved the best result\nwith 87.5% accuracy. Our core contribution is to make this benchmark dataset\navailable and accessible to facilitate further research in the field of in the\nfield of Bengali hate speech detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09692v4",
            "date": "2020-12-17 16:00:08+00:00",
            "title": "Benchmarking Automatic Detection of Psycholinguistic Characteristics for Better Human-Computer Interaction",
            "authors": [
                "Sanja \u0160tajner",
                "Seren Yenikent",
                "Marc Franco-Salvador"
            ],
            "abstract": "When two people pay attention to each other and are interested in what the\nother has to say or write, they almost instantly adapt their writing/speaking\nstyle to match the other. For a successful interaction with a user, chatbots\nand dialogue systems should be able to do the same. We propose a framework\nconsisting of five psycholinguistic textual characteristics for better\nhuman-computer interaction. We describe the annotation processes used for\ncollecting the data, and benchmark five binary classification tasks,\nexperimenting with different training sizes and model architectures. We perform\nexperiments in English, Spanish, German, Chinese, and Arabic. The best\narchitectures noticeably outperform several baselines and achieve\nmacro-averaged F1-scores between 72% and 96% depending on the language and the\ntask. Similar results are achieved even with a small amount of training data.\nThe proposed framework proved to be fairly easy to model for various languages\neven with small amount of manually annotated data if right architectures are\nused. At the same time, it showed potential for improving user satisfaction if\napplied in existing commercial chatbots.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09766v2",
            "date": "2020-12-17 17:22:30+00:00",
            "title": "MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering",
            "authors": [
                "Sofian Chaybouti",
                "Achraf Saghe",
                "Aymen Shabou"
            ],
            "abstract": "In this paper, we introduce MIX : a multi-task deep learning approach to\nsolve Open-Domain Question Answering. First, we design our system as a\nmulti-stage pipeline made of 3 building blocks : a BM25-based Retriever, to\nreduce the search space; RoBERTa based Scorer and Extractor, to rank retrieved\nparagraphs and extract relevant spans of text respectively. Eventually, we\nfurther improve computational efficiency of our system to deal with the\nscalability challenge : thanks to multi-task learning, we parallelize the close\ntasks solved by the Scorer and the Extractor. Our system is on par with\nstate-of-the-art performances on the squad-open benchmark while being simpler\nconceptually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10120v1",
            "date": "2020-12-18 09:24:01+00:00",
            "title": "Technical Progress Analysis Using a Dynamic Topic Model for Technical Terms to Revise Patent Classification Codes",
            "authors": [
                "Mana Iwata",
                "Yoshiro Matsuda",
                "Yoshimasa Utsumi",
                "Yoshitoshi Tanaka",
                "Kazuhide Nakata"
            ],
            "abstract": "Japanese patents are assigned a patent classification code, FI (File Index),\nthat is unique to Japan. FI is a subdivision of the IPC, an international\npatent classification code, that is related to Japanese technology. FIs are\nrevised to keep up with technological developments. These revisions have\nalready established more than 30,000 new FIs since 2006. However, these\nrevisions require a lot of time and workload. Moreover, these revisions are not\nautomated and are thus inefficient. Therefore, using machine learning to assist\nin the revision of patent classification codes (FI) will lead to improved\naccuracy and efficiency. This study analyzes patent documents from this new\nperspective of assisting in the revision of patent classification codes with\nmachine learning. To analyze time-series changes in patents, we used the\ndynamic topic model (DTM), which is an extension of the latent Dirichlet\nallocation (LDA). Also, unlike English, the Japanese language requires\nmorphological analysis. Patents contain many technical words that are not used\nin everyday life, so morphological analysis using a common dictionary is not\nsufficient. Therefore, we used a technique for extracting technical terms from\ntext. After extracting technical terms, we applied them to DTM. In this study,\nwe determined the technological progress of the lighting class F21 for 14 years\nand compared it with the actual revision of patent classification codes. In\nother words, we extracted technical terms from Japanese patents and applied DTM\nto determine the progress of Japanese technology. Then, we analyzed the results\nfrom the new perspective of revising patent classification codes with machine\nlearning. As a result, it was found that those whose topics were on the rise\nwere judged to be new technologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10187v1",
            "date": "2020-12-18 12:17:08+00:00",
            "title": "Regularized Attentive Capsule Network for Overlapped Relation Extraction",
            "authors": [
                "Tianyi Liu",
                "Xiangyu Lin",
                "Weijia Jia",
                "Mingliang Zhou",
                "Wei Zhao"
            ],
            "abstract": "Distantly supervised relation extraction has been widely applied in knowledge\nbase construction due to its less requirement of human efforts. However, the\nautomatically established training datasets in distant supervision contain\nlow-quality instances with noisy words and overlapped relations, introducing\ngreat challenges to the accurate extraction of relations. To address this\nproblem, we propose a novel Regularized Attentive Capsule Network (RA-CapNet)\nto better identify highly overlapped relations in each informal sentence. To\ndiscover multiple relation features in an instance, we embed multi-head\nattention into the capsule network as the low-level capsules, where the\nsubtraction of two entities acts as a new form of relation query to select\nsalient features regardless of their positions. To further discriminate\noverlapped relation features, we devise disagreement regularization to\nexplicitly encourage the diversity among both multiple attention heads and\nlow-level capsules. Extensive experiments conducted on widely used datasets\nshow that our model achieves significant improvements in relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10209v5",
            "date": "2020-12-18 13:05:11+00:00",
            "title": "Deep Open Intent Classification with Adaptive Decision Boundary",
            "authors": [
                "Hanlei Zhang",
                "Hua Xu",
                "Ting-En Lin"
            ],
            "abstract": "Open intent classification is a challenging task in dialogue systems. On the\none hand, it should ensure the quality of known intent identification. On the\nother hand, it needs to detect the open (unknown) intent without prior\nknowledge. Current models are limited in finding the appropriate decision\nboundary to balance the performances of both known intents and the open intent.\nIn this paper, we propose a post-processing method to learn the adaptive\ndecision boundary (ADB) for open intent classification. We first utilize the\nlabeled known intent samples to pre-train the model. Then, we automatically\nlearn the adaptive spherical decision boundary for each known class with the\naid of well-trained features. Specifically, we propose a new loss function to\nbalance both the empirical risk and the open space risk. Our method does not\nneed open intent samples and is free from modifying the model architecture.\nMoreover, our approach is surprisingly insensitive with less labeled data and\nfewer known intents. Extensive experiments on three benchmark datasets show\nthat our method yields significant improvements compared with the\nstate-of-the-art methods. The codes are released at\nhttps://github.com/thuiar/Adaptive-Decision-Boundary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10235v1",
            "date": "2020-12-18 13:50:17+00:00",
            "title": "AdvExpander: Generating Natural Language Adversarial Examples by Expanding Text",
            "authors": [
                "Zhihong Shao",
                "Zitao Liu",
                "Jiyong Zhang",
                "Zhongqin Wu",
                "Minlie Huang"
            ],
            "abstract": "Adversarial examples are vital to expose the vulnerability of machine\nlearning models. Despite the success of the most popular substitution-based\nmethods which substitutes some characters or words in the original examples,\nonly substitution is insufficient to uncover all robustness issues of models.\nIn this paper, we present AdvExpander, a method that crafts new adversarial\nexamples by expanding text, which is complementary to previous\nsubstitution-based methods. We first utilize linguistic rules to determine\nwhich constituents to expand and what types of modifiers to expand with. We\nthen expand each constituent by inserting an adversarial modifier searched from\na CVAE-based generative model which is pre-trained on a large scale corpus. To\nsearch adversarial modifiers, we directly search adversarial latent codes in\nthe latent space without tuning the pre-trained parameters. To ensure that our\nadversarial examples are label-preserving for text matching, we also constrain\nthe modifications with a heuristic rule. Experiments on three classification\ntasks verify the effectiveness of AdvExpander and the validity of our\nadversarial examples. AdvExpander crafts a new type of adversarial examples by\ntext expansion, thereby promising to reveal new robustness issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10267v1",
            "date": "2020-12-18 14:33:08+00:00",
            "title": "ReINTEL Challenge 2020: A Multimodal Ensemble Model for Detecting Unreliable Information on Vietnamese SNS",
            "authors": [
                "Nguyen Manh Duc Tuan",
                "Pham Quang Nhat Minh"
            ],
            "abstract": "In this paper, we present our methods for unrealiable information\nidentification task at VLSP 2020 ReINTEL Challenge. The task is to classify a\npiece of information into reliable or unreliable category. We propose a novel\nmultimodal ensemble model which combines two multimodal models to solve the\ntask. In each multimodal model, we combined feature representations acquired\nfrom three different data types: texts, images, and metadata. Multimodal\nfeatures are derived from three neural networks and fused for classification.\nExperimental results showed that our proposed multimodal ensemble model\nimproved against single models in term of ROC AUC score. We obtained 0.9445 AUC\nscore on the private test of the challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10275v2",
            "date": "2020-12-18 14:53:49+00:00",
            "title": "An Empirical Study of Using Pre-trained BERT Models for Vietnamese Relation Extraction Task at VLSP 2020",
            "authors": [
                "Pham Quang Nhat Minh"
            ],
            "abstract": "In this paper, we present an empirical study of using pre-trained BERT models\nfor the relation extraction task at the VLSP 2020 Evaluation Campaign. We\napplied two state-of-the-art BERT-based models: R-BERT and BERT model with\nentity starts. For each model, we compared two pre-trained BERT models:\nFPTAI/vibert and NlpHUST/vibert4news. We found that NlpHUST/vibert4news model\nsignificantly outperforms FPTAI/vibert for the Vietnamese relation extraction\ntask. Finally, we proposed an ensemble model that combines R-BERT and BERT with\nentity starts. Our proposed ensemble model slightly improved against two single\nmodels on the development data and the test data provided by the task\norganizers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10309v1",
            "date": "2020-12-18 15:53:50+00:00",
            "title": "Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training",
            "authors": [
                "Peng Shi",
                "Patrick Ng",
                "Zhiguo Wang",
                "Henghui Zhu",
                "Alexander Hanbo Li",
                "Jun Wang",
                "Cicero Nogueira dos Santos",
                "Bing Xiang"
            ],
            "abstract": "Most recently, there has been significant interest in learning contextual\nrepresentations for various NLP tasks, by leveraging large scale text corpora\nto train large neural language models with self-supervised learning objectives,\nsuch as Masked Language Model (MLM). However, based on a pilot study, we\nobserve three issues of existing general-purpose language models when they are\napplied to text-to-SQL semantic parsers: fail to detect column mentions in the\nutterances, fail to infer column mentions from cell values, and fail to compose\ncomplex SQL queries. To mitigate these issues, we present a model pre-training\nframework, Generation-Augmented Pre-training (GAP), that jointly learns\nrepresentations of natural language utterances and table schemas by leveraging\ngeneration models to generate pre-train data. GAP MODEL is trained on 2M\nutterance-schema pairs and 30K utterance-schema-SQL triples, whose utterances\nare produced by generative models. Based on experimental results, neural\nsemantic parsers that leverage GAP MODEL as a representation encoder obtain new\nstate-of-the-art results on both SPIDER and CRITERIA-TO-SQL benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10608v1",
            "date": "2020-12-19 06:56:59+00:00",
            "title": "Uncertainty-Aware Label Refinement for Sequence Labeling",
            "authors": [
                "Tao Gui",
                "Jiacheng Ye",
                "Qi Zhang",
                "Zhengyan Li",
                "Zichu Fei",
                "Yeyun Gong",
                "Xuanjing Huang"
            ],
            "abstract": "Conditional random fields (CRF) for label decoding has become ubiquitous in\nsequence labeling tasks. However, the local label dependencies and inefficient\nViterbi decoding have always been a problem to be solved. In this work, we\nintroduce a novel two-stage label decoding framework to model long-term label\ndependencies, while being much more computationally efficient. A base model\nfirst predicts draft labels, and then a novel two-stream self-attention model\nmakes refinements on these draft predictions based on long-range label\ndependencies, which can achieve parallel decoding for a faster prediction. In\naddition, in order to mitigate the side effects of incorrect draft labels,\nBayesian neural networks are used to indicate the labels with a high\nprobability of being wrong, which can greatly assist in preventing error\npropagation. The experimental results on three sequence labeling benchmarks\ndemonstrated that the proposed method not only outperformed the CRF-based\nmethods but also greatly accelerated the inference process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10668v1",
            "date": "2020-12-19 11:55:46+00:00",
            "title": "FraCaS: Temporal Analysis",
            "authors": [
                "Jean-Philippe Bernardy",
                "Stergios Chatzikyriakidis"
            ],
            "abstract": "In this paper, we propose an implementation of temporal semantics which is\nsuitable for inference problems. This implementation translates syntax trees to\nlogical formulas, suitable for consumption by the Coq proof assistant. We\nsupport several phenomena including: temporal references, temporal adverbs,\naspectual classes and progressives. We apply these semantics to the complete\nFraCaS testsuite. We obtain an accuracy of 81 percent overall and 73 percent\nfor problems explicitly marked as related to temporal reference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10813v1",
            "date": "2020-12-19 23:23:40+00:00",
            "title": "Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection",
            "authors": [
                "Yikang Li",
                "Pulkit Goel",
                "Varsha Kuppur Rajendra",
                "Har Simrat Singh",
                "Jonathan Francis",
                "Kaixin Ma",
                "Eric Nyberg",
                "Alessandro Oltramari"
            ],
            "abstract": "Conditional text generation has been a challenging task that is yet to see\nhuman-level performance from state-of-the-art models. In this work, we\nspecifically focus on the Commongen benchmark, wherein the aim is to generate a\nplausible sentence for a given set of input concepts. Despite advances in other\ntasks, large pre-trained language models that are fine-tuned on this dataset\noften produce sentences that are syntactically correct but qualitatively\ndeviate from a human understanding of common sense. Furthermore, generated\nsequences are unable to fulfill such lexical requirements as matching\npart-of-speech and full concept coverage. In this paper, we explore how\ncommonsense knowledge graphs can enhance model performance, with respect to\ncommonsense reasoning and lexically-constrained decoding. We propose strategies\nfor enhancing the semantic correctness of the generated text, which we\naccomplish through: extracting commonsense relations from Conceptnet, injecting\nthese relations into the Unified Language Model (UniLM) through attention\nmechanisms, and enforcing the aforementioned lexical requirements through\noutput constraints. By performing several ablations, we find that commonsense\ninjection enables the generation of sentences that are more aligned with human\nunderstanding, while remaining compliant with lexical requirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11145v1",
            "date": "2020-12-21 06:54:38+00:00",
            "title": "Domain specific BERT representation for Named Entity Recognition of lab protocol",
            "authors": [
                "Tejas Vaidhya",
                "Ayush Kaushal"
            ],
            "abstract": "Supervised models trained to predict properties from representations have\nbeen achieving high accuracy on a variety of tasks. For instance, the BERT\nfamily seems to work exceptionally well on the downstream task from NER tagging\nto the range of other linguistic tasks. But the vocabulary used in the medical\nfield contains a lot of different tokens used only in the medical industry such\nas the name of different diseases, devices, organisms, medicines, etc. that\nmakes it difficult for traditional BERT model to create contextualized\nembedding. In this paper, we are going to illustrate the System for Named\nEntity Tagging based on Bio-Bert. Experimental results show that our model\ngives substantial improvements over the baseline and stood the fourth runner up\nin terms of F1 score, and first runner up in terms of Recall with just 2.21 F1\nscore behind the best one.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11169v1",
            "date": "2020-12-21 08:01:04+00:00",
            "title": "An End-to-End Document-Level Neural Discourse Parser Exploiting Multi-Granularity Representations",
            "authors": [
                "Ke Shi",
                "Zhengyuan Liu",
                "Nancy F. Chen"
            ],
            "abstract": "Document-level discourse parsing, in accordance with the Rhetorical Structure\nTheory (RST), remains notoriously challenging. Challenges include the deep\nstructure of document-level discourse trees, the requirement of subtle semantic\njudgments, and the lack of large-scale training corpora. To address such\nchallenges, we propose to exploit robust representations derived from multiple\nlevels of granularity across syntax and semantics, and in turn incorporate such\nrepresentations in an end-to-end encoder-decoder neural architecture for more\nresourceful discourse processing. In particular, we first use a pre-trained\ncontextual language model that embodies high-order and long-range dependency to\nenable finer-grain semantic, syntactic, and organizational representations. We\nfurther encode such representations with boundary and hierarchical information\nto obtain more refined modeling for document-level discourse processing.\nExperimental results show that our parser achieves the state-of-the-art\nperformance, approaching human-level performance on the benchmarked RST\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11204v1",
            "date": "2020-12-21 09:35:52+00:00",
            "title": "Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization",
            "authors": [
                "Mehrdad Farahani",
                "Mohammad Gharachorloo",
                "Mohammad Manthouri"
            ],
            "abstract": "Text summarization is one of the most critical Natural Language Processing\n(NLP) tasks. More and more researches are conducted in this field every day.\nPre-trained transformer-based encoder-decoder models have begun to gain\npopularity for these tasks. This paper proposes two methods to address this\ntask and introduces a novel dataset named pn-summary for Persian abstractive\ntext summarization. The models employed in this paper are mT5 and an\nencoder-decoder version of the ParsBERT model (i.e., a monolingual BERT model\nfor Persian). These models are fine-tuned on the pn-summary dataset. The\ncurrent work is the first of its kind and, by achieving promising results, can\nserve as a baseline for any future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11357v1",
            "date": "2020-12-21 14:10:42+00:00",
            "title": "Self-attention Comparison Module for Boosting Performance on Retrieval-based Open-Domain Dialog Systems",
            "authors": [
                "Tian Lan",
                "Xian-Ling Mao",
                "Zhipeng Zhao",
                "Wei Wei",
                "Heyan Huang"
            ],
            "abstract": "Since the pre-trained language models are widely used, retrieval-based\nopen-domain dialog systems, have attracted considerable attention from\nresearchers recently. Most of the previous works select a suitable response\nonly according to the matching degree between the query and each individual\ncandidate response. Although good performance has been achieved, these recent\nworks ignore the comparison among the candidate responses, which could provide\nrich information for selecting the most appropriate response. Intuitively,\nbetter decisions could be made when the models can get access to the comparison\ninformation among all the candidate responses. In order to leverage the\ncomparison information among the candidate responses, in this paper, we propose\na novel and plug-in Self-attention Comparison Module for retrieval-based\nopen-domain dialog systems, called SCM. Extensive experiment results\ndemonstrate that our proposed self-attention comparison module effectively\nboosts the performance of the existing retrieval-based open-domain dialog\nsystems. Besides, we have publicly released our source codes for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11384v1",
            "date": "2020-12-21 14:29:31+00:00",
            "title": "Document-Level Relation Extraction with Reconstruction",
            "authors": [
                "Wang Xu",
                "Kehai Chen",
                "Tiejun Zhao"
            ],
            "abstract": "In document-level relation extraction (DocRE), graph structure is generally\nused to encode relation information in the input document to classify the\nrelation category between each entity pair, and has greatly advanced the DocRE\ntask over the past several years. However, the learned graph representation\nuniversally models relation information between all entity pairs regardless of\nwhether there are relationships between these entity pairs. Thus, those entity\npairs without relationships disperse the attention of the encoder-classifier\nDocRE for ones with relationships, which may further hind the improvement of\nDocRE. To alleviate this issue, we propose a novel\nencoder-classifier-reconstructor model for DocRE. The reconstructor manages to\nreconstruct the ground-truth path dependencies from the graph representation,\nto ensure that the proposed DocRE model pays more attention to encode entity\npairs with relationships in the training. Furthermore, the reconstructor is\nregarded as a relationship indicator to assist relation classification in the\ninference, which can further improve the performance of DocRE model.\nExperimental results on a large-scale DocRE dataset show that the proposed\nmodel can significantly improve the accuracy of relation extraction on a strong\nheterogeneous graph-based baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11420v1",
            "date": "2020-12-21 15:22:47+00:00",
            "title": "TechTexC: Classification of Technical Texts using Convolution and Bidirectional Long Short Term Memory Network",
            "authors": [
                "Omar Sharif",
                "Eftekhar Hossain",
                "Mohammed Moshiul Hoque"
            ],
            "abstract": "This paper illustrates the details description of technical text\nclassification system and its results that developed as a part of participation\nin the shared task TechDofication 2020. The shared task consists of two\nsub-tasks: (i) first task identify the coarse-grained technical domain of given\ntext in a specified language and (ii) the second task classify a text of\ncomputer science domain into fine-grained sub-domains. A classification system\n(called 'TechTexC') is developed to perform the classification task using three\ntechniques: convolution neural network (CNN), bidirectional long short term\nmemory (BiLSTM) network, and combined CNN with BiLSTM. Results show that CNN\nwith BiLSTM model outperforms the other techniques concerning task-1 of\nsub-tasks (a, b, c and g) and task-2a. This combined model obtained f1 scores\nof 82.63 (sub-task a), 81.95 (sub-task b), 82.39 (sub-task c), 84.37 (sub-task\ng), and 67.44 (task-2a) on the development dataset. Moreover, in the case of\ntest set, the combined CNN with BiLSTM approach achieved that higher accuracy\nfor the subtasks 1a (70.76%), 1b (79.97%), 1c (65.45%), 1g (49.23%) and 2a\n(70.14%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11468v1",
            "date": "2020-12-21 16:36:32+00:00",
            "title": "Pattern-aware Data Augmentation for Query Rewriting in Voice Assistant Systems",
            "authors": [
                "Yunmo Chen",
                "Sixing Lu",
                "Fan Yang",
                "Xiaojiang Huang",
                "Xing Fan",
                "Chenlei Guo"
            ],
            "abstract": "Query rewriting (QR) systems are widely used to reduce the friction caused by\nerrors in a spoken language understanding pipeline. However, the underlying\nsupervised models require a large number of labeled pairs, and these pairs are\nhard and costly to be collected. Therefore, We propose an augmentation\nframework that learns patterns from existing training pairs and generates\nrewrite candidates from rewrite labels inversely to compensate for insufficient\nQR training data. The proposed framework casts the augmentation problem as a\nsequence-to-sequence generation task and enforces the optimization process with\na policy gradient technique for controllable rewarding. This approach goes\nbeyond the traditional heuristics or rule-based augmentation methods and is not\nconstrained to generate predefined patterns of swapping/replacing words. Our\nexperimental results show its effectiveness compared with a fully trained QR\nbaseline and demonstrate its potential application in boosting the QR\nperformance on low-resource domains or locales.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11657v1",
            "date": "2020-12-21 19:47:04+00:00",
            "title": "Subword Sampling for Low Resource Word Alignment",
            "authors": [
                "Ehsaneddin Asgari",
                "Masoud Jalili Sabet",
                "Philipp Dufter",
                "Christopher Ringlstetter",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Annotation projection is an important area in NLP that can greatly contribute\nto creating language resources for low-resource languages. Word alignment plays\na key role in this setting. However, most of the existing word alignment\nmethods are designed for a high resource setting in machine translation where\nmillions of parallel sentences are available. This amount reduces to a few\nthousands of sentences when dealing with low-resource languages failing the\nexisting established IBM models. In this paper, we propose subword\nsampling-based alignment of text units. This method's hypothesis is that the\naggregation of different granularities of text for certain language pairs can\nhelp word-level alignment. For certain languages for which gold-standard\nalignments exist, we propose an iterative Bayesian optimization framework to\noptimize selecting possible subwords from the space of possible subword\nrepresentations of the source and target sentences. We show that the subword\nsampling method consistently outperforms word-level alignment on six language\npairs: English-German, English-French, English-Romanian, English-Persian,\nEnglish-Hindi, and English-Inuktitut. In addition, we show that the\nhyperparameters learned for certain language pairs can be applied to other\nlanguages at no supervision and consistently improve the alignment results. We\nobserve that using $5K$ parallel sentences together with our proposed subword\nsampling approach, we obtain similar F1 scores to the use of $100K$'s of\nparallel sentences in existing word-level fast-align/eflomal alignment methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11760v4",
            "date": "2020-12-22 00:29:15+00:00",
            "title": "Acronym Identification and Disambiguation Shared Tasks for Scientific Document Understanding",
            "authors": [
                "Amir Pouran Ben Veyseh",
                "Franck Dernoncourt",
                "Thien Huu Nguyen",
                "Walter Chang",
                "Leo Anthony Celi"
            ],
            "abstract": "Acronyms are the short forms of longer phrases and they are frequently used\nin writing, especially scholarly writing, to save space and facilitate the\ncommunication of information. As such, every text understanding tool should be\ncapable of recognizing acronyms in text (i.e., acronym identification) and also\nfinding their correct meaning (i.e., acronym disambiguation). As most of the\nprior works on these tasks are restricted to the biomedical domain and use\nunsupervised methods or models trained on limited datasets, they fail to\nperform well for scientific document understanding. To push forward research in\nthis direction, we have organized two shared task for acronym identification\nand acronym disambiguation in scientific documents, named AI@SDU and AD@SDU,\nrespectively. The two shared tasks have attracted 52 and 43 participants,\nrespectively. While the submitted systems make substantial improvements\ncompared to the existing baselines, there are still far from the human-level\nperformance. This paper reviews the two shared tasks and the prominent\nparticipating systems for each of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11808v2",
            "date": "2020-12-22 03:03:50+00:00",
            "title": "Improved Biomedical Word Embeddings in the Transformer Era",
            "authors": [
                "Jiho Noh",
                "Ramakanth Kavuluru"
            ],
            "abstract": "Biomedical word embeddings are usually pre-trained on free text corpora with\nneural methods that capture local and global distributional properties. They\nare leveraged in downstream tasks using various neural architectures that are\ndesigned to optimize task-specific objectives that might further tune such\nembeddings. Since 2018, however, there is a marked shift from these static\nembeddings to contextual embeddings motivated by language models (e.g., ELMo,\ntransformers such as BERT, and ULMFiT). These dynamic embeddings have the added\nbenefit of being able to distinguish homonyms and acronyms given their context.\nHowever, static embeddings are still relevant in low resource settings (e.g.,\nsmart devices, IoT elements) and to study lexical semantics from a\ncomputational linguistics perspective. In this paper, we jointly learn word and\nconcept embeddings by first using the skip-gram method and further fine-tuning\nthem with correlational information manifesting in co-occurring Medical Subject\nHeading (MeSH) concepts in biomedical citations. This fine-tuning is\naccomplished with the BERT transformer architecture in the two-sentence input\nmode with a classification objective that captures MeSH pair co-occurrence. In\nessence, we repurpose a transformer architecture (typically used to generate\ndynamic embeddings) to improve static embeddings using concept correlations. We\nconduct evaluations of these tuned static embeddings using multiple datasets\nfor word relatedness developed by previous efforts. Without selectively culling\nconcepts and terms (as was pursued by previous efforts), we believe we offer\nthe most exhaustive evaluation of static embeddings to date with clear\nperformance improvements across the board. We provide our code and embeddings\nfor public use for downstream applications and research endeavors:\nhttps://github.com/bionlproc/BERT-CRel-Embeddings",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11820v2",
            "date": "2020-12-22 03:51:35+00:00",
            "title": "Recognizing Emotion Cause in Conversations",
            "authors": [
                "Soujanya Poria",
                "Navonil Majumder",
                "Devamanyu Hazarika",
                "Deepanway Ghosal",
                "Rishabh Bhardwaj",
                "Samson Yu Bai Jian",
                "Romila Ghosh",
                "Niyati Chhaya",
                "Alexander Gelbukh",
                "Rada Mihalcea"
            ],
            "abstract": "Recognizing the cause behind emotions in text is a fundamental yet\nunder-explored area of research in NLP. Advances in this area hold the\npotential to improve interpretability and performance in affect-based models.\nIdentifying emotion causes at the utterance level in conversations is\nparticularly challenging due to the intermingling dynamic among the\ninterlocutors. To this end, we introduce the task of recognizing emotion cause\nin conversations with an accompanying dataset named RECCON. Furthermore, we\ndefine different cause types based on the source of the causes and establish\nstrong transformer-based baselines to address two different sub-tasks of\nRECCON: 1) Causal Span Extraction and 2) Causal Emotion Entailment. The dataset\nis available at https://github.com/declare-lab/RECCON.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11995v1",
            "date": "2020-12-22 13:38:06+00:00",
            "title": "Pre-Training a Language Model Without Human Language",
            "authors": [
                "Cheng-Han Chiang",
                "Hung-yi Lee"
            ],
            "abstract": "In this paper, we study how the intrinsic nature of pre-training data\ncontributes to the fine-tuned downstream performance. To this end, we pre-train\ndifferent transformer-based masked language models on several corpora with\ncertain features, and we fine-tune those language models on GLUE benchmarks. We\nfind that models pre-trained on unstructured data beat those trained directly\nfrom scratch on downstream tasks. Our results also show that pre-training on\nstructured data does not always make the model acquire ability that can be\ntransferred to natural language downstream tasks. To our great astonishment, we\nuncover that pre-training on certain non-human language data gives GLUE\nperformance close to performance pre-trained on another non-English language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12121v2",
            "date": "2020-12-22 15:59:44+00:00",
            "title": "Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages",
            "authors": [
                "Cheng Yi",
                "Jianzhong Wang",
                "Ning Cheng",
                "Shiyu Zhou",
                "Bo Xu"
            ],
            "abstract": "There are several domains that own corresponding widely used feature\nextractors, such as ResNet, BERT, and GPT-x. These models are usually\npre-trained on large amounts of unlabeled data by self-supervision and can be\neffectively applied to downstream tasks. In the speech domain, wav2vec2.0\nstarts to show its powerful representation ability and feasibility of ultra-low\nresource speech recognition on the Librispeech corpus, which belongs to the\naudiobook domain. However, wav2vec2.0 has not been examined on real spoken\nscenarios and languages other than English. To verify its universality over\nlanguages, we apply pre-trained models to solve low-resource speech recognition\ntasks in various spoken languages. We achieve more than 20% relative\nimprovements in six languages compared with previous work. Among these\nlanguages, English achieves a gain of 52.4%. Moreover, using coarse-grained\nmodeling units, such as subword or character, achieves better results than\nfine-grained modeling units, such as phone or letter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12366v1",
            "date": "2020-12-22 21:34:02+00:00",
            "title": "Multi-Head Self-Attention with Role-Guided Masks",
            "authors": [
                "Dongsheng Wang",
                "Casper Hansen",
                "Lucas Chaves Lima",
                "Christian Hansen",
                "Maria Maistro",
                "Jakob Grue Simonsen",
                "Christina Lioma"
            ],
            "abstract": "The state of the art in learning meaningful semantic representations of words\nis the Transformer model and its attention mechanisms. Simply put, the\nattention mechanisms learn to attend to specific parts of the input dispensing\nrecurrence and convolutions. While some of the learned attention heads have\nbeen found to play linguistically interpretable roles, they can be redundant or\nprone to errors. We propose a method to guide the attention heads towards roles\nidentified in prior work as important. We do this by defining role-specific\nmasks to constrain the heads to attend to specific parts of the input, such\nthat different heads are designed to play different roles. Experiments on text\nclassification and machine translation using 7 different datasets show that our\nmethod outperforms competitive attention-based, CNN, and RNN baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12382v1",
            "date": "2020-12-22 22:02:37+00:00",
            "title": "Simple-QE: Better Automatic Quality Estimation for Text Simplification",
            "authors": [
                "Reno Kriz",
                "Marianna Apidianaki",
                "Chris Callison-Burch"
            ],
            "abstract": "Text simplification systems generate versions of texts that are easier to\nunderstand for a broader audience. The quality of simplified texts is generally\nestimated using metrics that compare to human references, which can be\ndifficult to obtain. We propose Simple-QE, a BERT-based quality estimation (QE)\nmodel adapted from prior summarization QE work, and show that it correlates\nwell with human quality judgments. Simple-QE does not require human references,\nwhich makes the model useful in a practical setting where users would need to\nbe informed about the quality of generated simplifications. We also show that\nwe can adapt this approach to accurately predict the complexity of\nhuman-written texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12458v2",
            "date": "2020-12-23 02:43:37+00:00",
            "title": "TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems",
            "authors": [
                "Bill Byrne",
                "Karthik Krishnamoorthi",
                "Saravanan Ganesh",
                "Mihir Sanjay Kale"
            ],
            "abstract": "We present a data-driven, end-to-end approach to transaction-based dialog\nsystems that performs at near-human levels in terms of verbal response quality\nand factual grounding accuracy. We show that two essential components of the\nsystem produce these results: a sufficiently large and diverse, in-domain\nlabeled dataset, and a neural network-based, pre-trained model that generates\nboth verbal responses and API call predictions. In terms of data, we introduce\nTicketTalk, a movie ticketing dialog dataset with 23,789 annotated\nconversations. The movie ticketing conversations range from completely\nopen-ended and unrestricted to more structured, both in terms of their\nknowledge base, discourse features, and number of turns. In qualitative human\nevaluations, model-generated responses trained on just 10,000 TicketTalk\ndialogs were rated to \"make sense\" 86.5 percent of the time, almost the same as\nhuman responses in the same contexts. Our simple, API-focused annotation schema\nresults in a much easier labeling task making it faster and more cost\neffective. It is also the key component for being able to predict API calls\naccurately. We handle factual grounding by incorporating API calls in the\ntraining data, allowing our model to learn which actions to take and when.\nTrained on the same 10,000-dialog set, the model's API call predictions were\nrated to be correct 93.9 percent of the time in our evaluations, surpassing the\nratings for the corresponding human labels. We show how API prediction and\nresponse generation scores improve as the dataset size incrementally increases\nfrom 5000 to 21,000 dialogs. Our analysis also clearly illustrates the benefits\nof pre-training. We are publicly releasing the TicketTalk dataset with this\npaper to facilitate future work on transaction-based dialogs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12624v3",
            "date": "2020-12-23 12:28:17+00:00",
            "title": "Learning Dense Representations of Phrases at Scale",
            "authors": [
                "Jinhyuk Lee",
                "Mujeen Sung",
                "Jaewoo Kang",
                "Danqi Chen"
            ],
            "abstract": "Open-domain question answering can be reformulated as a phrase retrieval\nproblem, without the need for processing documents on-demand during inference\n(Seo et al., 2019). However, current phrase retrieval models heavily depend on\nsparse representations and still underperform retriever-reader approaches. In\nthis work, we show for the first time that we can learn dense representations\nof phrases alone that achieve much stronger performance in open-domain QA. We\npresent an effective method to learn phrase representations from the\nsupervision of reading comprehension tasks, coupled with novel negative\nsampling methods. We also propose a query-side fine-tuning strategy, which can\nsupport transfer learning and reduce the discrepancy between training and\ninference. On five popular open-domain QA datasets, our model DensePhrases\nimproves over previous phrase retrieval models by 15%-25% absolute accuracy and\nmatches the performance of state-of-the-art retriever-reader models. Our model\nis easy to parallelize due to pure dense representations and processes more\nthan 10 questions per second on CPUs. Finally, we directly use our pre-indexed\ndense phrase representations for two slot filling tasks, showing the promise of\nutilizing DensePhrases as a dense knowledge base for downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12641v1",
            "date": "2020-12-23 13:22:53+00:00",
            "title": "Negation in Cognitive Reasoning",
            "authors": [
                "Claudia Schon",
                "Sophie Siebert",
                "Frieder Stolzenburg"
            ],
            "abstract": "Negation is both an operation in formal logic and in natural language by\nwhich a proposition is replaced by one stating the opposite, as by the addition\nof \"not\" or another negation cue. Treating negation in an adequate way is\nrequired for cognitive reasoning, which comprises commonsense reasoning and\ntext comprehension. One task of cognitive reasoning is answering questions\ngiven by sentences in natural language. There are tools based on discourse\nrepresentation theory to convert sentences automatically into a formal logical\nrepresentation. However, since the knowledge in logical databases in practice\nalways is incomplete, forward reasoning of automated reasoning systems alone\ndoes not suffice to derive answers to questions because, instead of complete\nproofs, often only partial positive knowledge can be derived. In consequence,\nnegative information from negated expressions does not help in this context,\nbecause only negative knowledge can be derived from this. Therefore, we aim at\nreducing syntactic negation, strictly speaking, the negated event or property,\nto its inverse. This lays the basis of cognitive reasoning employing both logic\nand machine learning for general question answering. In this paper, we describe\nan effective procedure to determine the negated event or property in order to\nreplace it with it inverse and our overall system for cognitive reasoning. We\ndemonstrate the procedure with examples and evaluate it with several\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12799v1",
            "date": "2020-12-23 16:59:43+00:00",
            "title": "Automatic Scansion of Spanish Poetry without Syllabification",
            "authors": [
                "Guillermo Marco Rem\u00f3n",
                "Julio Gonzalo"
            ],
            "abstract": "In recent years, several systems of automated metric analysis of Spanish\npoetry have emerged. These systems rely on complex methods of syllabification\nand stress assignment, which use PoS-tagging libraries, whose computational\ncost is high. This cost increases with the calculation of metric ambiguities.\nFurthermore, they do not consider determining issues in syllabic count such as\nthe phenomena of compensation between hemistichs of verses of more than eleven\nsyllables. However, it is possible to carry out an informative and accurate\nmetric analysis without using these costly methods. We propose an algorithm\nthat performs accurate scansion (number of syllables, stress pattern and type\nof verse) without syllabification. It addresses metric ambiguities and takes\ninto account the hemistichs compensation. Our algorithm outperforms the current\nstate of the art by 2% in fixed-metre poetry, and 25% in mixed-metre poetry. It\nalso runs 21 and 25 times faster, respectively. Finally, a desktop application\nis offered as a tool for researchers of Spanish poetry.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13163v2",
            "date": "2020-12-24 08:14:36+00:00",
            "title": "Cross-lingual Universal Dependency Parsing Only from One Monolingual Treebank",
            "authors": [
                "Kailai Sun",
                "Zuchao Li",
                "Hai Zhao"
            ],
            "abstract": "Syntactic parsing is a highly linguistic processing task whose parser\nrequires training on treebanks from the expensive human annotation. As it is\nunlikely to obtain a treebank for every human language, in this work, we\npropose an effective cross-lingual UD parsing framework for transferring parser\nfrom only one source monolingual treebank to any other target languages without\ntreebank available. To reach satisfactory parsing accuracy among quite\ndifferent languages, we introduce two language modeling tasks into dependency\nparsing as multi-tasking. Assuming only unlabeled data from target languages\nplus the source treebank can be exploited together, we adopt a self-training\nstrategy for further performance improvement in terms of our multi-task\nframework. Our proposed cross-lingual parsers are implemented for English,\nChinese, and 22 UD treebanks. The empirical study shows that our cross-lingual\nparsers yield promising results for all target languages, for the first time,\napproaching the parser performance which is trained in its own target treebank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13176v1",
            "date": "2020-12-24 09:27:52+00:00",
            "title": "Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters",
            "authors": [
                "Marta R. Costa-juss\u00e0",
                "Carlos Escolano",
                "Christine Basta",
                "Javier Ferrando",
                "Roser Batlle",
                "Ksenia Kharitonova"
            ],
            "abstract": "Multilingual Neural Machine Translation architectures mainly differ in the\namount of sharing modules and parameters among languages. In this paper, and\nfrom an algorithmic perspective, we explore if the chosen architecture, when\ntrained with the same data, influences the gender bias accuracy. Experiments in\nfour language pairs show that Language-Specific encoders-decoders exhibit less\nbias than the Shared encoder-decoder architecture. Further interpretability\nanalysis of source embeddings and the attention shows that, in the\nLanguage-Specific case, the embeddings encode more gender information, and its\nattention is more diverted. Both behaviors help in mitigating gender bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13260v1",
            "date": "2020-12-24 14:10:24+00:00",
            "title": "Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification",
            "authors": [
                "Libo Qin",
                "Zhouyang Li",
                "Wanxiang Che",
                "Minheng Ni",
                "Ting Liu"
            ],
            "abstract": "In a dialog system, dialog act recognition and sentiment classification are\ntwo correlative tasks to capture speakers intentions, where dialog act and\nsentiment can indicate the explicit and the implicit intentions separately. The\ndialog context information (contextual information) and the mutual interaction\ninformation are two key factors that contribute to the two related tasks.\nUnfortunately, none of the existing approaches consider the two important\nsources of information simultaneously. In this paper, we propose a\nCo-Interactive Graph Attention Network (Co-GAT) to jointly perform the two\ntasks. The core module is a proposed co-interactive graph interaction layer\nwhere a cross-utterances connection and a cross-tasks connection are\nconstructed and iteratively updated with each other, achieving to consider the\ntwo types of information simultaneously. Experimental results on two public\ndatasets show that our model successfully captures the two sources of\ninformation and achieve the state-of-the-art performance.\n  In addition, we find that the contributions from the contextual and mutual\ninteraction information do not fully overlap with contextualized word\nrepresentations (BERT, Roberta, XLNet).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13339v1",
            "date": "2020-12-24 17:24:54+00:00",
            "title": "A Context Aware Approach for Generating Natural Language Attacks",
            "authors": [
                "Rishabh Maheshwary",
                "Saket Maheshwary",
                "Vikram Pudi"
            ],
            "abstract": "We study an important task of attacking natural language processing models in\na black box setting. We propose an attack strategy that crafts semantically\nsimilar adversarial examples on text classification and entailment tasks. Our\nproposed attack finds candidate words by considering the information of both\nthe original word and its surrounding context. It jointly leverages masked\nlanguage modelling and next sentence prediction for context understanding. In\ncomparison to attacks proposed in prior literature, we are able to generate\nhigh quality adversarial examples that do significantly better both in terms of\nsuccess rate and word perturbation percentage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13354v1",
            "date": "2020-12-24 17:40:06+00:00",
            "title": "To what extent do human explanations of model behavior align with actual model behavior?",
            "authors": [
                "Grusha Prasad",
                "Yixin Nie",
                "Mohit Bansal",
                "Robin Jia",
                "Douwe Kiela",
                "Adina Williams"
            ],
            "abstract": "Given the increasingly prominent role NLP models (will) play in our lives, it\nis important to evaluate models on their alignment with human expectations of\nhow models behave. Using Natural Language Inference (NLI) as a case study, we\ninvestigated the extent to which human-generated explanations of models'\ninference decisions align with how models actually make these decisions. More\nspecifically, we defined two alignment metrics that quantify how well natural\nlanguage human explanations align with model sensitivity to input words, as\nmeasured by integrated gradients. Then, we evaluated six different transformer\nmodels (the base and large versions of BERT, RoBERTa and ELECTRA), and found\nthat the BERT-base model has the highest alignment with human-generated\nexplanations, for both alignment metrics. Additionally, the base versions of\nthe models we surveyed tended to have higher alignment with human-generated\nexplanations than their larger counterparts, suggesting that increasing the\nnumber model parameters could result in worse alignment with human\nexplanations. Finally, we find that a model's alignment with human explanations\nis not predicted by the model's accuracy on NLI, suggesting that accuracy and\nalignment are orthogonal, and both are important ways to evaluate models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13436v1",
            "date": "2020-12-24 20:20:50+00:00",
            "title": "ThamizhiUDp: A Dependency Parser for Tamil",
            "authors": [
                "Kengatharaiyer Sarveswaran",
                "Gihan Dias"
            ],
            "abstract": "This paper describes how we developed a neural-based dependency parser,\nnamely ThamizhiUDp, which provides a complete pipeline for the dependency\nparsing of the Tamil language text using Universal Dependency formalism. We\nhave considered the phases of the dependency parsing pipeline and identified\ntools and resources in each of these phases to improve the accuracy and to\ntackle data scarcity. ThamizhiUDp uses Stanza for tokenisation and\nlemmatisation, ThamizhiPOSt and ThamizhiMorph for generating Part of Speech\n(POS) and Morphological annotations, and uuparser with multilingual training\nfor dependency parsing. ThamizhiPOSt is our POS tagger, which is based on the\nStanza, trained with Amrita POS-tagged corpus. It is the current\nstate-of-the-art in Tamil POS tagging with an F1 score of 93.27. Our\nmorphological analyzer, ThamizhiMorph is a rule-based system with a very good\ncoverage of Tamil. Our dependency parser ThamizhiUDp was trained using\nmultilingual data. It shows a Labelled Assigned Score (LAS) of 62.39, 4 points\nhigher than the current best achieved for Tamil dependency parsing. Therefore,\nwe show that breaking up the dependency parsing pipeline to accommodate\nexisting tools and resources is a viable approach for low-resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13454v1",
            "date": "2020-12-24 22:25:22+00:00",
            "title": "Why Neural Machine Translation Prefers Empty Outputs",
            "authors": [
                "Xing Shi",
                "Yijun Xiao",
                "Kevin Knight"
            ],
            "abstract": "We investigate why neural machine translation (NMT) systems assign high\nprobability to empty translations. We find two explanations. First, label\nsmoothing makes correct-length translations less confident, making it easier\nfor the empty translation to finally outscore them. Second, NMT systems use the\nsame, high-frequency EoS word to end all target sentences, regardless of\nlength. This creates an implicit smoothing that increases zero-length\ntranslations. Using different EoS types in target sentences of different\nlengths exposes and eliminates this implicit smoothing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13624v1",
            "date": "2020-12-25 20:29:56+00:00",
            "title": "Fine-grained Emotion and Intent Learning in Movie Dialogues",
            "authors": [
                "Anuradha Welivita",
                "Yubo Xie",
                "Pearl Pu"
            ],
            "abstract": "We propose a novel large-scale emotional dialogue dataset, consisting of 1M\ndialogues retrieved from the OpenSubtitles corpus and annotated with 32\nemotions and 9 empathetic response intents using a BERT-based fine-grained\ndialogue emotion classifier. This work explains the complex pipeline used to\npreprocess movie subtitles and select good movie dialogues to annotate. We also\ndescribe the semi-supervised learning process followed to train a fine-grained\nemotion classifier to annotate these dialogues. Despite the large set of\nlabels, our dialogue emotion classifier achieved an accuracy of $65\\%$ and was\nused to annotate 1M emotional movie dialogues from OpenSubtitles. This scale of\nemotional dialogue classification has never been attempted before, both in\nterms of dataset size and fine-grained emotion and intent categories.\nVisualization techniques used to analyze the quality of the resultant dataset\nsuggest that it conforms to the patterns of human social interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13866v1",
            "date": "2020-12-27 05:33:21+00:00",
            "title": "Learning Light-Weight Translation Models from Deep Transformer",
            "authors": [
                "Bei Li",
                "Ziyang Wang",
                "Hui Liu",
                "Quan Du",
                "Tong Xiao",
                "Chunliang Zhang",
                "Jingbo Zhu"
            ],
            "abstract": "Recently, deep models have shown tremendous improvements in neural machine\ntranslation (NMT). However, systems of this kind are computationally expensive\nand memory intensive. In this paper, we take a natural step towards learning\nstrong but light-weight NMT systems. We proposed a novel group-permutation\nbased knowledge distillation approach to compressing the deep Transformer model\ninto a shallow model. The experimental results on several benchmarks validate\nthe effectiveness of our method. Our compressed model is 8X shallower than the\ndeep model, with almost no loss in BLEU. To further enhance the teacher model,\nwe present a Skipping Sub-Layer method to randomly omit sub-layers to introduce\nperturbation into training, which achieves a BLEU score of 30.63 on\nEnglish-German newstest2014. The code is publicly available at\nhttps://github.com/libeineu/GPKD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13873v1",
            "date": "2020-12-27 06:22:23+00:00",
            "title": "An Embarrassingly Simple Model for Dialogue Relation Extraction",
            "authors": [
                "Fuzhao Xue",
                "Aixin Sun",
                "Hao Zhang",
                "Eng Siong Chng"
            ],
            "abstract": "Dialogue relation extraction (RE) is to predict the relation type of two\nentities mentioned in a dialogue. In this paper, we model Dialogue RE as a\nmulti-label classification task and propose a simple yet effective model named\nSimpleRE. SimpleRE captures the interrelations among multiple relations in a\ndialogue through a novel input format, BERT Relation Token Sequence (BRS). In\nBRS, multiple [CLS] tokens are used to capture different relations between\ndifferent pairs of entities. A Relation Refinement Gate (RRG) is designed to\nextract relation-specific semantic representation adaptively. Experiments on\nDialogRE show that SimpleRE achieves the best performance with much shorter\ntraining time. SimpleRE outperforms all direct baselines on sentence-level RE\nwithout using external resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13939v1",
            "date": "2020-12-27 13:26:11+00:00",
            "title": "Adaptive Convolution for Semantic Role Labeling",
            "authors": [
                "Kashif Munir",
                "Hai Zhao",
                "Zuchao Li"
            ],
            "abstract": "Semantic role labeling (SRL) aims at elaborating the meaning of a sentence by\nforming a predicate-argument structure. Recent researches depicted that the\neffective use of syntax can improve SRL performance. However, syntax is a\ncomplicated linguistic clue and is hard to be effectively applied in a\ndownstream task like SRL. This work effectively encodes syntax using adaptive\nconvolution which endows strong flexibility to existing convolutional networks.\nThe existing CNNs may help in encoding a complicated structure like syntax for\nSRL, but it still has shortcomings. Contrary to traditional convolutional\nnetworks that use same filters for different inputs, adaptive convolution uses\nadaptively generated filters conditioned on syntactically informed inputs. We\nachieve this with the integration of a filter generation network which\ngenerates the input specific filters. This helps the model to focus on\nimportant syntactic features present inside the input, thus enlarging the gap\nbetween syntax-aware and syntax-agnostic SRL systems. We further study a\nhashing technique to compress the size of the filter generation network for SRL\nin terms of trainable parameters. Experiments on CoNLL-2009 dataset confirm\nthat the proposed model substantially outperforms most previous SRL systems for\nboth English and Chinese languages",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14022v1",
            "date": "2020-12-27 22:30:13+00:00",
            "title": "ALP-KD: Attention-Based Layer Projection for Knowledge Distillation",
            "authors": [
                "Peyman Passban",
                "Yimeng Wu",
                "Mehdi Rezagholizadeh",
                "Qun Liu"
            ],
            "abstract": "Knowledge distillation is considered as a training and compression strategy\nin which two neural networks, namely a teacher and a student, are coupled\ntogether during training. The teacher network is supposed to be a trustworthy\npredictor and the student tries to mimic its predictions. Usually, a student\nwith a lighter architecture is selected so we can achieve compression and yet\ndeliver high-quality results. In such a setting, distillation only happens for\nfinal predictions whereas the student could also benefit from teacher's\nsupervision for internal components.\n  Motivated by this, we studied the problem of distillation for intermediate\nlayers. Since there might not be a one-to-one alignment between student and\nteacher layers, existing techniques skip some teacher layers and only distill\nfrom a subset of them. This shortcoming directly impacts quality, so we instead\npropose a combinatorial technique which relies on attention. Our model fuses\nteacher-side information and takes each layer's significance into\nconsideration, then performs distillation between combined teacher layers and\nthose of the student. Using our technique, we distilled a 12-layer BERT (Devlin\net al. 2019) into 6-, 4-, and 2-layer counterparts and evaluated them on GLUE\ntasks (Wang et al. 2018). Experimental results show that our combinatorial\napproach is able to outperform other existing techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14094v1",
            "date": "2020-12-28 04:38:45+00:00",
            "title": "Pivot Through English: Reliably Answering Multilingual Questions without Document Retrieval",
            "authors": [
                "Ivan Montero",
                "Shayne Longpre",
                "Ni Lao",
                "Andrew J. Frank",
                "Christopher DuBois"
            ],
            "abstract": "Existing methods for open-retrieval question answering in lower resource\nlanguages (LRLs) lag significantly behind English. They not only suffer from\nthe shortcomings of non-English document retrieval, but are reliant on\nlanguage-specific supervision for either the task or translation. We formulate\na task setup more realistic to available resources, that circumvents document\nretrieval to reliably transfer knowledge from English to lower resource\nlanguages. Assuming a strong English question answering model or database, we\ncompare and analyze methods that pivot through English: to map foreign queries\nto English and then English answers back to target language answers. Within\nthis task setup we propose Reranked Multilingual Maximal Inner Product Search\n(RM-MIPS), akin to semantic similarity retrieval over the English training set\nwith reranking, which outperforms the strongest baselines by 2.7% on XQuAD and\n6.2% on MKQA. Analysis demonstrates the particular efficacy of this strategy\nover state-of-the-art alternatives in challenging settings: low-resource\nlanguages, with extensive distractor data and query distribution misalignment.\nCircumventing retrieval, our analysis shows this approach offers rapid answer\ngeneration to almost any language off-the-shelf, without the need for any\nadditional training data in the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14116v2",
            "date": "2020-12-28 06:48:04+00:00",
            "title": "Syntax-Enhanced Pre-trained Model",
            "authors": [
                "Zenan Xu",
                "Daya Guo",
                "Duyu Tang",
                "Qinliang Su",
                "Linjun Shou",
                "Ming Gong",
                "Wanjun Zhong",
                "Xiaojun Quan",
                "Nan Duan",
                "Daxin Jiang"
            ],
            "abstract": "We study the problem of leveraging the syntactic structure of text to enhance\npre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of\ntext either in the pre-training stage or in the fine-tuning stage, so that they\nsuffer from discrepancy between the two stages. Such a problem would lead to\nthe necessity of having human-annotated syntactic information, which limits the\napplication of existing methods to broader scenarios. To address this, we\npresent a model that utilizes the syntax of text in both pre-training and\nfine-tuning stages. Our model is based on Transformer with a syntax-aware\nattention layer that considers the dependency tree of the text. We further\nintroduce a new pre-training task of predicting the syntactic distance among\ntokens in the dependency tree. We evaluate the model on three downstream tasks,\nincluding relation classification, entity typing, and question answering.\nResults show that our model achieves state-of-the-art performance on six public\nbenchmark datasets. We have two major findings. First, we demonstrate that\ninfusing automatically produced syntax of text improves pre-trained models.\nSecond, global syntactic distances among tokens bring larger performance gains\ncompared to local head relations between contiguous tokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14136v1",
            "date": "2020-12-28 08:10:28+00:00",
            "title": "On Generating Extended Summaries of Long Documents",
            "authors": [
                "Sajad Sotudeh",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "Prior work in document summarization has mainly focused on generating short\nsummaries of a document. While this type of summary helps get a high-level view\nof a given document, it is desirable in some cases to know more detailed\ninformation about its salient points that can't fit in a short summary. This is\ntypically the case for longer documents such as a research paper, legal\ndocument, or a book. In this paper, we present a new method for generating\nextended summaries of long papers. Our method exploits hierarchical structure\nof the documents and incorporates it into an extractive summarization model\nthrough a multi-task learning approach. We then present our results on three\nlong summarization datasets, arXiv-Long, PubMed-Long, and Longsumm. Our method\noutperforms or matches the performance of strong baselines. Furthermore, we\nperform a comprehensive analysis over the generated results, shedding insights\non future research for long-form summary generation task. Our analysis shows\nthat our multi-tasking approach can adjust extraction probability distribution\nto the favor of summary-worthy sentences across diverse sections. Our datasets,\nand codes are publicly available at\nhttps://github.com/Georgetown-IR-Lab/ExtendedSumm",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14271v3",
            "date": "2020-12-28 15:20:52+00:00",
            "title": "Towards Fully Automated Manga Translation",
            "authors": [
                "Ryota Hinami",
                "Shonosuke Ishiwatari",
                "Kazuhiko Yasuda",
                "Yusuke Matsui"
            ],
            "abstract": "We tackle the problem of machine translation of manga, Japanese comics. Manga\ntranslation involves two important problems in machine translation:\ncontext-aware and multimodal translation. Since text and images are mixed up in\nan unstructured fashion in Manga, obtaining context from the image is essential\nfor manga translation. However, it is still an open problem how to extract\ncontext from image and integrate into MT models. In addition, corpus and\nbenchmarks to train and evaluate such model is currently unavailable. In this\npaper, we make the following four contributions that establishes the foundation\nof manga translation research. First, we propose multimodal context-aware\ntranslation framework. We are the first to incorporate context information\nobtained from manga image. It enables us to translate texts in speech bubbles\nthat cannot be translated without using context information (e.g., texts in\nother speech bubbles, gender of speakers, etc.). Second, for training the\nmodel, we propose the approach to automatic corpus construction from pairs of\noriginal manga and their translations, by which large parallel corpus can be\nconstructed without any manual labeling. Third, we created a new benchmark to\nevaluate manga translation. Finally, on top of our proposed methods, we devised\na first comprehensive system for fully automated manga translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14312v1",
            "date": "2020-12-28 15:47:45+00:00",
            "title": "Panarchy: ripples of a boundary concept",
            "authors": [
                "Juan Rocha",
                "Linda Luvuno",
                "Jesse Rieb",
                "Erin Crockett",
                "Katja Malmborg",
                "Michael Schoon",
                "Garry Peterson"
            ],
            "abstract": "How do social-ecological systems change over time? In 2002 Holling and\ncolleagues proposed the concept of Panarchy, which presented social-ecological\nsystems as an interacting set of adaptive cycles, each of which is produced by\nthe dynamic tensions between novelty and efficiency at multiple scales.\nInitially introduced as a conceptual framework and set of metaphors, panarchy\nhas gained the attention of scholars across many disciplines and its ideas\ncontinue to inspire further conceptual developments. Almost twenty years after\nthis concept was introduced we review how it has been used, tested, extended\nand revised. We do this by combining qualitative methods and machine learning.\nDocument analysis was used to code panarchy features that are commonly used in\nthe scientific literature (N = 42), a qualitative analysis that was\ncomplemented with topic modeling of 2177 documents. We find that the adaptive\ncycle is the feature of panarchy that has attracted the most attention.\nChallenges remain in empirically grounding the metaphor, but recent theoretical\nand empirical work offers some avenues for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14320v2",
            "date": "2020-12-28 16:02:28+00:00",
            "title": "BURT: BERT-inspired Universal Representation from Learning Meaningful Segment",
            "authors": [
                "Yian Li",
                "Hai Zhao"
            ],
            "abstract": "Although pre-trained contextualized language models such as BERT achieve\nsignificant performance on various downstream tasks, current language\nrepresentation still only focuses on linguistic objective at a specific\ngranularity, which may not applicable when multiple levels of linguistic units\nare involved at the same time. Thus this work introduces and explores the\nuniversal representation learning, i.e., embeddings of different levels of\nlinguistic unit in a uniform vector space. We present a universal\nrepresentation model, BURT (BERT-inspired Universal Representation from\nlearning meaningful segmenT), to encode different levels of linguistic unit\ninto the same vector space. Specifically, we extract and mask meaningful\nsegments based on point-wise mutual information (PMI) to incorporate different\ngranular objectives into the pre-training stage. We conduct experiments on\ndatasets for English and Chinese including the GLUE and CLUE benchmarks, where\nour model surpasses its baselines and alternatives on a wide range of\ndownstream tasks. We present our approach of constructing analogy datasets in\nterms of words, phrases and sentences and experiment with multiple\nrepresentation models to examine geometric properties of the learned vector\nspace through a task-independent evaluation. Finally, we verify the\neffectiveness of our unified pre-training strategy in two real-world text\nmatching scenarios. As a result, our model significantly outperforms existing\ninformation retrieval (IR) methods and yields universal representations that\ncan be directly applied to retrieval-based question-answering and natural\nlanguage generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14388v2",
            "date": "2020-12-28 18:06:37+00:00",
            "title": "Universal Sentence Representation Learning with Conditional Masked Language Model",
            "authors": [
                "Ziyi Yang",
                "Yinfei Yang",
                "Daniel Cer",
                "Jax Law",
                "Eric Darve"
            ],
            "abstract": "This paper presents a novel training method, Conditional Masked Language\nModeling (CMLM), to effectively learn sentence representations on large scale\nunlabeled corpora. CMLM integrates sentence representation learning into MLM\ntraining by conditioning on the encoded vectors of adjacent sentences. Our\nEnglish CMLM model achieves state-of-the-art performance on SentEval, even\noutperforming models learned using (semi-)supervised signals. As a fully\nunsupervised learning method, CMLM can be conveniently extended to a broad\nrange of languages and domains. We find that a multilingual CMLM model\nco-trained with bitext retrieval~(BR) and natural language inference~(NLI)\ntasks outperforms the previous state-of-the-art multilingual models by a large\nmargin. We explore the same language bias of the learned representations, and\npropose a principle component based approach to remove the language identifying\ninformation from the representation while still retaining sentence semantics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14500v2",
            "date": "2020-12-28 21:51:31+00:00",
            "title": "A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification",
            "authors": [
                "Xiangci Li",
                "Gully Burns",
                "Nanyun Peng"
            ],
            "abstract": "Even for domain experts, it is a non-trivial task to verify a scientific\nclaim by providing supporting or refuting evidence rationales. The situation\nworsens as misinformation is proliferated on social media or news websites,\nmanually or programmatically, at every moment. As a result, an automatic\nfact-verification tool becomes crucial for combating the spread of\nmisinformation. In this work, we propose a novel, paragraph-level, multi-task\nlearning model for the SciFact task by directly computing a sequence of\ncontextualized sentence embeddings from a BERT model and jointly training the\nmodel on rationale selection and stance prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14535v1",
            "date": "2020-12-29 00:05:35+00:00",
            "title": "Robust Dialogue Utterance Rewriting as Sequence Tagging",
            "authors": [
                "Jie Hao",
                "Linfeng Song",
                "Liwei Wang",
                "Kun Xu",
                "Zhaopeng Tu",
                "Dong Yu"
            ],
            "abstract": "The task of dialogue rewriting aims to reconstruct the latest dialogue\nutterance by copying the missing content from the dialogue context. Until now,\nthe existing models for this task suffer from the robustness issue, i.e.,\nperformances drop dramatically when testing on a different domain. We address\nthis robustness issue by proposing a novel sequence-tagging-based model so that\nthe search space is significantly reduced, yet the core of this task is still\nwell covered. As a common issue of most tagging models for text generation, the\nmodel's outputs may lack fluency. To alleviate this issue, we inject the loss\nsignal from BLEU or GPT-2 under a REINFORCE framework. Experiments show huge\nimprovements of our model over the current state-of-the-art systems on domain\ntransfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14583v2",
            "date": "2020-12-29 03:18:50+00:00",
            "title": "Understanding and Improving Lexical Choice in Non-Autoregressive Translation",
            "authors": [
                "Liang Ding",
                "Longyue Wang",
                "Xuebo Liu",
                "Derek F. Wong",
                "Dacheng Tao",
                "Zhaopeng Tu"
            ],
            "abstract": "Knowledge distillation (KD) is essential for training non-autoregressive\ntranslation (NAT) models by reducing the complexity of the raw data with an\nautoregressive teacher model. In this study, we empirically show that as a side\neffect of this training, the lexical choice errors on low-frequency words are\npropagated to the NAT model from the teacher model. To alleviate this problem,\nwe propose to expose the raw data to NAT models to restore the useful\ninformation of low-frequency words, which are missed in the distilled data. To\nthis end, we introduce an extra Kullback-Leibler divergence term derived by\ncomparing the lexical choice of NAT model and that embedded in the raw data.\nExperimental results across language pairs and model architectures demonstrate\nthe effectiveness and universality of the proposed approach. Extensive analyses\nconfirm our claim that our approach improves performance by reducing the\nlexical choice errors on low-frequency words. Encouragingly, our approach\npushes the SOTA NAT performance on the WMT14 English-German and WMT16\nRomanian-English datasets up to 27.8 and 33.8 BLEU points, respectively. The\nsource code will be released.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14602v1",
            "date": "2020-12-29 04:48:52+00:00",
            "title": "Is human scoring the best criteria for summary evaluation?",
            "authors": [
                "Oleg Vasilyev",
                "John Bohannon"
            ],
            "abstract": "Normally, summary quality measures are compared with quality scores produced\nby human annotators. A higher correlation with human scores is considered to be\na fair indicator of a better measure. We discuss observations that cast doubt\non this view. We attempt to show a possibility of an alternative indicator.\nGiven a family of measures, we explore a criterion of selecting the best\nmeasure not relying on correlations with human scores. Our observations for the\nBLANC family of measures suggest that the criterion is universal across very\ndifferent styles of summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14610v1",
            "date": "2020-12-29 05:14:08+00:00",
            "title": "Unified Open-Domain Question Answering with Structured and Unstructured Knowledge",
            "authors": [
                "Barlas Oguz",
                "Xilun Chen",
                "Vladimir Karpukhin",
                "Stan Peshterliev",
                "Dmytro Okhonko",
                "Michael Schlichtkrull",
                "Sonal Gupta",
                "Yashar Mehdad",
                "Scott Yih"
            ],
            "abstract": "We study open-domain question answering (ODQA) with structured, unstructured\nand semi-structured knowledge sources, including text, tables, lists, and\nknowledge bases. Our approach homogenizes all sources by reducing them to text,\nand applies recent, powerful retriever-reader models which have so far been\nlimited to text sources only. We show that knowledge-base QA can be greatly\nimproved when reformulated in this way. Contrary to previous work, we find that\ncombining sources always helps, even for datasets which target a single source\nby construction. As a result, our unified model produces state-of-the-art\nresults on 3 popular ODQA benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14642v1",
            "date": "2020-12-29 07:30:03+00:00",
            "title": "Multiple Structural Priors Guided Self Attention Network for Language Understanding",
            "authors": [
                "Le Qi",
                "Yu Zhang",
                "Qingyu Yin",
                "Ting Liu"
            ],
            "abstract": "Self attention networks (SANs) have been widely utilized in recent NLP\nstudies. Unlike CNNs or RNNs, standard SANs are usually position-independent,\nand thus are incapable of capturing the structural priors between sequences of\nwords. Existing studies commonly apply one single mask strategy on SANs for\nincorporating structural priors while failing at modeling more abundant\nstructural information of texts. In this paper, we aim at introducing multiple\ntypes of structural priors into SAN models, proposing the Multiple Structural\nPriors Guided Self Attention Network (MS-SAN) that transforms different\nstructural priors into different attention heads by using a novel multi-mask\nbased multi-head attention mechanism. In particular, we integrate two\ncategories of structural priors, including the sequential order and the\nrelative position of words. For the purpose of capturing the latent\nhierarchical structure of the texts, we extract these information not only from\nthe word contexts but also from the dependency syntax trees. Experimental\nresults on two tasks show that MS-SAN achieves significant improvements against\nother strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14645v2",
            "date": "2020-12-29 07:41:48+00:00",
            "title": "Interpretable NLG for Task-oriented Dialogue Systems with Heterogeneous Rendering Machines",
            "authors": [
                "Yangming Li",
                "Kaisheng Yao"
            ],
            "abstract": "End-to-end neural networks have achieved promising performances in natural\nlanguage generation (NLG). However, they are treated as black boxes and lack\ninterpretability. To address this problem, we propose a novel framework,\nheterogeneous rendering machines (HRM), that interprets how neural generators\nrender an input dialogue act (DA) into an utterance. HRM consists of a renderer\nset and a mode switcher. The renderer set contains multiple decoders that vary\nin both structure and functionality. For every generation step, the mode\nswitcher selects an appropriate decoder from the renderer set to generate an\nitem (a word or a phrase). To verify the effectiveness of our method, we have\nconducted extensive experiments on 5 benchmark datasets. In terms of automatic\nmetrics (e.g., BLEU), our model is competitive with the current\nstate-of-the-art method. The qualitative analysis shows that our model can\ninterpret the rendering process of neural generators well. Human evaluation\nalso confirms the interpretability of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14660v4",
            "date": "2020-12-29 08:51:47+00:00",
            "title": "A Theoretical Analysis of the Repetition Problem in Text Generation",
            "authors": [
                "Zihao Fu",
                "Wai Lam",
                "Anthony Man-Cho So",
                "Bei Shi"
            ],
            "abstract": "Text generation tasks, including translation, summarization, language models,\nand etc. see rapid growth during recent years. Despite the remarkable\nachievements, the repetition problem has been observed in nearly all text\ngeneration models undermining the generation performance extensively. To solve\nthe repetition problem, many methods have been proposed, but there is no\nexisting theoretical analysis to show why this problem happens and how it is\nresolved. In this paper, we propose a new framework for theoretical analysis\nfor the repetition problem. We first define the Average Repetition Probability\n(ARP) to characterize the repetition problem quantitatively. Then, we conduct\nan extensive analysis of the Markov generation model and derive several upper\nbounds of the average repetition probability with intuitive understanding. We\nshow that most of the existing methods are essentially minimizing the upper\nbounds explicitly or implicitly. Grounded on our theory, we show that the\nrepetition problem is, unfortunately, caused by the traits of our language\nitself. One major reason is attributed to the fact that there exist too many\nwords predicting the same word as the subsequent word with high probability.\nConsequently, it is easy to go back to that word and form repetitions and we\ndub it as the high inflow problem. Furthermore, we derive a concentration bound\nof the average repetition probability for a general generation model. Finally,\nbased on the theoretical upper bounds, we propose a novel rebalanced encoding\napproach to alleviate the high inflow problem. The experimental results show\nthat our theoretical framework is applicable in general generation models and\nour proposed rebalanced encoding approach alleviates the repetition problem\nsignificantly. The source code of this paper can be obtained from\nhttps://github.com/fuzihaofzh/repetition-problem-nlg.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14682v1",
            "date": "2020-12-29 09:43:50+00:00",
            "title": "Accelerating Pre-trained Language Models via Calibrated Cascade",
            "authors": [
                "Lei Li",
                "Yankai Lin",
                "Shuhuai Ren",
                "Deli Chen",
                "Xuancheng Ren",
                "Peng Li",
                "Jie Zhou",
                "Xu Sun"
            ],
            "abstract": "Dynamic early exiting aims to accelerate pre-trained language models' (PLMs)\ninference by exiting in shallow layer without passing through the entire model.\nIn this paper, we analyze the working mechanism of dynamic early exiting and\nfind it cannot achieve a satisfying trade-off between inference speed and\nperformance. On one hand, the PLMs' representations in shallow layers are not\nsufficient for accurate prediction. One the other hand, the internal off-ramps\ncannot provide reliable exiting decisions. To remedy this, we instead propose\nCascadeBERT, which dynamically selects a proper-sized, complete model in a\ncascading manner. To obtain more reliable model selection, we further devise a\ndifficulty-aware objective, encouraging the model output class probability to\nreflect the real difficulty of each instance. Extensive experimental results\ndemonstrate the superiority of our proposal over strong baseline models of\nPLMs' acceleration including both dynamic early exiting and knowledge\ndistillation methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14710v2",
            "date": "2020-12-29 11:37:43+00:00",
            "title": "Code Summarization with Structure-induced Transformer",
            "authors": [
                "Hongqiu Wu",
                "Hai Zhao",
                "Min Zhang"
            ],
            "abstract": "Code summarization (CS) is becoming a promising area in recent language\nunderstanding, which aims to generate sensible human language automatically for\nprogramming language in the format of source code, serving in the most\nconvenience of programmer developing. It is well known that programming\nlanguages are highly structured. Thus previous works attempt to apply\nstructure-based traversal (SBT) or non-sequential models like Tree-LSTM and\ngraph neural network (GNN) to learn structural program semantics. However, it\nis surprising that incorporating SBT into advanced encoder like Transformer\ninstead of LSTM has been shown no performance gain, which lets GNN become the\nonly rest means modeling such necessary structural clue in source code. To\nrelease such inconvenience, we propose structure-induced Transformer, which\nencodes sequential code inputs with multi-view structural clues in terms of a\nnewly-proposed structure-induced self-attention mechanism. Extensive\nexperiments show that our proposed structure-induced Transformer helps achieve\nnew state-of-the-art results on benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14740v3",
            "date": "2020-12-29 13:01:52+00:00",
            "title": "LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding",
            "authors": [
                "Yang Xu",
                "Yiheng Xu",
                "Tengchao Lv",
                "Lei Cui",
                "Furu Wei",
                "Guoxin Wang",
                "Yijuan Lu",
                "Dinei Florencio",
                "Cha Zhang",
                "Wanxiang Che",
                "Min Zhang",
                "Lidong Zhou"
            ],
            "abstract": "Pre-training of text and layout has proved effective in a variety of\nvisually-rich document understanding tasks due to its effective model\narchitecture and the advantage of large-scale unlabeled scanned/digital-born\ndocuments. In this paper, we present \\textbf{LayoutLMv2} by pre-training text,\nlayout and image in a multi-modal framework, where new model architectures and\npre-training tasks are leveraged. Specifically, LayoutLMv2 not only uses the\nexisting masked visual-language modeling task but also the new text-image\nalignment and text-image matching tasks in the pre-training stage, where\ncross-modality interaction is better learned. Meanwhile, it also integrates a\nspatial-aware self-attention mechanism into the Transformer architecture, so\nthat the model can fully understand the relative positional relationship among\ndifferent text blocks. Experiment results show that LayoutLMv2 outperforms\nstrong baselines and achieves new state-of-the-art results on a wide variety of\ndownstream visually-rich document understanding tasks, including FUNSD (0.7895\n-> 0.8420), CORD (0.9493 -> 0.9601), SROIE (0.9524 -> 0.9781), Kleister-NDA\n(0.834 -> 0.852), RVL-CDIP (0.9443 -> 0.9564), and DocVQA (0.7295 -> 0.8672).\nThe pre-trained LayoutLMv2 model is publicly available at\nhttps://aka.ms/layoutlmv2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14756v2",
            "date": "2020-12-29 14:06:41+00:00",
            "title": "Dialogue Response Selection with Hierarchical Curriculum Learning",
            "authors": [
                "Yixuan Su",
                "Deng Cai",
                "Qingyu Zhou",
                "Zibo Lin",
                "Simon Baker",
                "Yunbo Cao",
                "Shuming Shi",
                "Nigel Collier",
                "Yan Wang"
            ],
            "abstract": "We study the learning of a matching model for dialogue response selection.\nMotivated by the recent finding that models trained with random negative\nsamples are not ideal in real-world scenarios, we propose a hierarchical\ncurriculum learning framework that trains the matching model in an\n\"easy-to-difficult\" scheme. Our learning framework consists of two\ncomplementary curricula: (1) corpus-level curriculum (CC); and (2)\ninstance-level curriculum (IC). In CC, the model gradually increases its\nability in finding the matching clues between the dialogue context and a\nresponse candidate. As for IC, it progressively strengthens the model's ability\nin identifying the mismatching information between the dialogue context and a\nresponse candidate. Empirical studies on three benchmark datasets with three\nstate-of-the-art matching models demonstrate that the proposed learning\nframework significantly improves the model performance across various\nevaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14763v2",
            "date": "2020-12-29 14:23:50+00:00",
            "title": "CMV-BERT: Contrastive multi-vocab pretraining of BERT",
            "authors": [
                "Wei Zhu",
                "Daniel Cheung"
            ],
            "abstract": "In this work, we represent CMV-BERT, which improves the pretraining of a\nlanguage model via two ingredients: (a) contrastive learning, which is well\nstudied in the area of computer vision; (b) multiple vocabularies, one of which\nis fine-grained and the other is coarse-grained. The two methods both provide\ndifferent views of an original sentence, and both are shown to be beneficial.\nDownstream tasks demonstrate our proposed CMV-BERT are effective in improving\nthe pretrained language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14769v1",
            "date": "2020-12-29 14:28:07+00:00",
            "title": "Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces",
            "authors": [
                "Linyang Li",
                "Yunfan Shao",
                "Demin Song",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "Adversarial attacks in texts are mostly substitution-based methods that\nreplace words or characters in the original texts to achieve success attacks.\nRecent methods use pre-trained language models as the substitutes generator.\nWhile in Chinese, such methods are not applicable since words in Chinese\nrequire segmentations first. In this paper, we propose a pre-train language\nmodel as the substitutes generator using sentence-pieces to craft adversarial\nexamples in Chinese. The substitutions in the generated adversarial examples\nare not characters or words but \\textit{'pieces'}, which are more natural to\nChinese readers. Experiments results show that the generated adversarial\nsamples can mislead strong target models and remain fluent and semantically\npreserved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14781v1",
            "date": "2020-12-29 14:47:35+00:00",
            "title": "A Hierarchical Transformer with Speaker Modeling for Emotion Recognition in Conversation",
            "authors": [
                "Jiangnan Li",
                "Zheng Lin",
                "Peng Fu",
                "Qingyi Si",
                "Weiping Wang"
            ],
            "abstract": "Emotion Recognition in Conversation (ERC) is a more challenging task than\nconventional text emotion recognition. It can be regarded as a personalized and\ninteractive emotion recognition task, which is supposed to consider not only\nthe semantic information of text but also the influences from speakers. The\ncurrent method models speakers' interactions by building a relation between\nevery two speakers. However, this fine-grained but complicated modeling is\ncomputationally expensive, hard to extend, and can only consider local context.\nTo address this problem, we simplify the complicated modeling to a binary\nversion: Intra-Speaker and Inter-Speaker dependencies, without identifying\nevery unique speaker for the targeted speaker. To better achieve the simplified\ninteraction modeling of speakers in Transformer, which shows excellent ability\nto settle long-distance dependency, we design three types of masks and\nrespectively utilize them in three independent Transformer blocks. The designed\nmasks respectively model the conventional context modeling, Intra-Speaker\ndependency, and Inter-Speaker dependency. Furthermore, different speaker-aware\ninformation extracted by Transformer blocks diversely contributes to the\nprediction, and therefore we utilize the attention mechanism to automatically\nweight them. Experiments on two ERC datasets indicate that our model is\nefficacious to achieve better performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14837v1",
            "date": "2020-12-29 16:36:49+00:00",
            "title": "DRS at MRP 2020: Dressing up Discourse Representation Structures as Graphs",
            "authors": [
                "Lasha Abzianidze",
                "Johan Bos",
                "Stephan Oepen"
            ],
            "abstract": "Discourse Representation Theory (DRT) is a formal account for representing\nthe meaning of natural language discourse. Meaning in DRT is modeled via a\nDiscourse Representation Structure (DRS), a meaning representation with a\nmodel-theoretic interpretation, which is usually depicted as nested boxes. In\ncontrast, a directed labeled graph is a common data structure used to encode\nsemantics of natural language texts. The paper describes the procedure of\ndressing up DRSs as directed labeled graphs to include DRT as a new framework\nin the 2020 shared task on Cross-Framework and Cross-Lingual Meaning\nRepresentation Parsing. Since one of the goals of the shared task is to\nencourage unified models for several semantic graph frameworks, the conversion\nprocedure was biased towards making the DRT graph framework somewhat similar to\nother graph-based meaning representation frameworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14854v1",
            "date": "2020-12-29 17:04:10+00:00",
            "title": "The Parallel Meaning Bank: A Framework for Semantically Annotating Multiple Languages",
            "authors": [
                "Lasha Abzianidze",
                "Rik van Noord",
                "Chunliu Wang",
                "Johan Bos"
            ],
            "abstract": "This paper gives a general description of the ideas behind the Parallel\nMeaning Bank, a framework with the aim to provide an easy way to annotate\ncompositional semantics for texts written in languages other than English. The\nannotation procedure is semi-automatic, and comprises seven layers of\nlinguistic information: segmentation, symbolisation, semantic tagging, word\nsense disambiguation, syntactic structure, thematic role labelling, and\nco-reference. New languages can be added to the meaning bank as long as the\ndocuments are based on translations from English, but also introduce new\ninteresting challenges on the linguistics assumptions underlying the Parallel\nMeaning Bank.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14913v1",
            "date": "2020-12-29 19:12:05+00:00",
            "title": "Transformer Feed-Forward Layers Are Key-Value Memories",
            "authors": [
                "Mor Geva",
                "Roei Schuster",
                "Jonathan Berant",
                "Omer Levy"
            ],
            "abstract": "Feed-forward layers constitute two-thirds of a transformer model's\nparameters, yet their role in the network remains under-explored. We show that\nfeed-forward layers in transformer-based language models operate as key-value\nmemories, where each key correlates with textual patterns in the training\nexamples, and each value induces a distribution over the output vocabulary. Our\nexperiments show that the learned patterns are human-interpretable, and that\nlower layers tend to capture shallow patterns, while upper layers learn more\nsemantic ones. The values complement the keys' input patterns by inducing\noutput distributions that concentrate probability mass on tokens likely to\nappear immediately after each pattern, particularly in the upper layers.\nFinally, we demonstrate that the output of a feed-forward layer is a\ncomposition of its memories, which is subsequently refined throughout the\nmodel's layers via residual connections to produce the final output\ndistribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14919v2",
            "date": "2020-12-29 19:35:34+00:00",
            "title": "WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections",
            "authors": [
                "Mingda Chen",
                "Sam Wiseman",
                "Kevin Gimpel"
            ],
            "abstract": "Datasets for data-to-text generation typically focus either on multi-domain,\nsingle-sentence generation or on single-domain, long-form generation. In this\nwork, we cast generating Wikipedia sections as a data-to-text generation task\nand create a large-scale dataset, WikiTableT, that pairs Wikipedia sections\nwith their corresponding tabular data and various metadata. WikiTableT contains\nmillions of instances, covering a broad range of topics, as well as a variety\nof flavors of generation tasks with different levels of flexibility. We\nbenchmark several training and decoding strategies on WikiTableT. Our\nqualitative analysis shows that the best approaches can generate fluent and\nhigh quality texts but they struggle with coherence and factuality, showing the\npotential for our dataset to inspire future work on long-form generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14956v2",
            "date": "2020-12-29 22:01:38+00:00",
            "title": "Generating Natural Language Attacks in a Hard Label Black Box Setting",
            "authors": [
                "Rishabh Maheshwary",
                "Saket Maheshwary",
                "Vikram Pudi"
            ],
            "abstract": "We study an important and challenging task of attacking natural language\nprocessing models in a hard label black box setting. We propose a\ndecision-based attack strategy that crafts high quality adversarial examples on\ntext classification and entailment tasks. Our proposed attack strategy\nleverages population-based optimization algorithm to craft plausible and\nsemantically similar adversarial examples by observing only the top label\npredicted by the target model. At each iteration, the optimization procedure\nallow word replacements that maximizes the overall semantic similarity between\nthe original and the adversarial text. Further, our approach does not rely on\nusing substitute models or any kind of training data. We demonstrate the\nefficacy of our proposed approach through extensive experimentation and\nablation studies on five state-of-the-art target models across seven benchmark\ndatasets. In comparison to attacks proposed in prior literature, we are able to\nachieve a higher success rate with lower word perturbation percentage that too\nin a highly restricted setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15015v2",
            "date": "2020-12-30 03:02:50+00:00",
            "title": "OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts",
            "authors": [
                "Yuxian Meng",
                "Shuhe Wang",
                "Qinghong Han",
                "Xiaofei Sun",
                "Fei Wu",
                "Rui Yan",
                "Jiwei Li"
            ],
            "abstract": "When humans converse, what a speaker will say next significantly depends on\nwhat he sees. Unfortunately, existing dialogue models generate dialogue\nutterances only based on preceding textual contexts, and visual contexts are\nrarely considered. This is due to a lack of a large-scale multi-module dialogue\ndataset with utterances paired with visual contexts. In this paper, we release\n{\\bf OpenViDial}, a large-scale multi-module dialogue dataset. The dialogue\nturns and visual contexts are extracted from movies and TV series, where each\ndialogue turn is paired with the corresponding visual context in which it takes\nplace. OpenViDial contains a total number of 1.1 million dialogue turns, and\nthus 1.1 million visual contexts stored in images. Based on this dataset, we\npropose a family of encoder-decoder models leveraging both textual and visual\ncontexts, from coarse-grained image features extracted from CNNs to\nfine-grained object features extracted from Faster R-CNNs. We observe that\nvisual information significantly improves dialogue generation qualities,\nverifying the necessity of integrating multi-modal features for dialogue\nlearning. Our work marks an important step towards large-scale multi-modal\ndialogue learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15023v1",
            "date": "2020-12-30 03:36:18+00:00",
            "title": "Language Identification of Devanagari Poems",
            "authors": [
                "Priyankit Acharya",
                "Aditya Ku. Pathak",
                "Rakesh Ch. Balabantaray",
                "Anil Ku. Singh"
            ],
            "abstract": "Language Identification is a very important part of several text processing\npipelines. Extensive research has been done in this field. This paper proposes\na procedure for automatic language identification of poems for poem analysis\ntask, consisting of 10 Devanagari based languages of India i.e. Angika, Awadhi,\nBraj, Bhojpuri, Chhattisgarhi, Garhwali, Haryanvi, Hindi, Magahi, and Maithili.\nWe collated corpora of poems of varying length and studied the similarity of\npoems among the 10 languages at the lexical level. Finally, various language\nidentification systems based on supervised machine learning and deep learning\ntechniques are applied and evaluated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15045v2",
            "date": "2020-12-30 05:20:16+00:00",
            "title": "Reservoir Transformers",
            "authors": [
                "Sheng Shen",
                "Alexei Baevski",
                "Ari S. Morcos",
                "Kurt Keutzer",
                "Michael Auli",
                "Douwe Kiela"
            ],
            "abstract": "We demonstrate that transformers obtain impressive performance even when some\nof the layers are randomly initialized and never updated. Inspired by old and\nwell-established ideas in machine learning, we explore a variety of non-linear\n\"reservoir\" layers interspersed with regular transformer layers, and show\nimprovements in wall-clock compute time until convergence, as well as overall\nperformance, on various machine translation and (masked) language modelling\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15070v1",
            "date": "2020-12-30 07:49:00+00:00",
            "title": "Enhancing Pre-trained Language Model with Lexical Simplification",
            "authors": [
                "Rongzhou Bao",
                "Jiayi Wang",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "For both human readers and pre-trained language models (PrLMs), lexical\ndiversity may lead to confusion and inaccuracy when understanding the\nunderlying semantic meanings of given sentences. By substituting complex words\nwith simple alternatives, lexical simplification (LS) is a recognized method to\nreduce such lexical diversity, and therefore to improve the understandability\nof sentences. In this paper, we leverage LS and propose a novel approach which\ncan effectively improve the performance of PrLMs in text classification. A\nrule-based simplification process is applied to a given sentence. PrLMs are\nencouraged to predict the real label of the given sentence with auxiliary\ninputs from the simplified version. Using strong PrLMs (BERT and ELECTRA) as\nbaselines, our approach can still further improve the performance in various\ntext classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15075v1",
            "date": "2020-12-30 08:19:02+00:00",
            "title": "Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA",
            "authors": [
                "Ana Valeria Gonzalez",
                "Gagan Bansal",
                "Angela Fan",
                "Robin Jia",
                "Yashar Mehdad",
                "Srinivasan Iyer"
            ],
            "abstract": "While research on explaining predictions of open-domain QA systems (ODQA) to\nusers is gaining momentum, most works have failed to evaluate the extent to\nwhich explanations improve user trust. While few works evaluate explanations\nusing user studies, they employ settings that may deviate from the end-user's\nusage in-the-wild: ODQA is most ubiquitous in voice-assistants, yet current\nresearch only evaluates explanations using a visual display, and may\nerroneously extrapolate conclusions about the most performant explanations to\nother modalities. To alleviate these issues, we conduct user studies that\nmeasure whether explanations help users correctly decide when to accept or\nreject an ODQA system's answer. Unlike prior work, we control for explanation\nmodality, e.g., whether they are communicated to users through a spoken or\nvisual interface, and contrast effectiveness across modalities. Our results\nshow that explanations derived from retrieved evidence passages can outperform\nstrong baselines (calibrated confidence) across modalities but the best\nexplanation strategy in fact changes with the modality. We show common failure\ncases of current explanations, emphasize end-to-end evaluation of explanations,\nand caution against evaluating them in proxy modalities that are different from\ndeployment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15127v1",
            "date": "2020-12-30 12:20:41+00:00",
            "title": "Improving Zero-Shot Translation by Disentangling Positional Information",
            "authors": [
                "Danni Liu",
                "Jan Niehues",
                "James Cross",
                "Francisco Guzm\u00e1n",
                "Xian Li"
            ],
            "abstract": "Multilingual neural machine translation has shown the capability of directly\ntranslating between language pairs unseen in training, i.e. zero-shot\ntranslation. Despite being conceptually attractive, it often suffers from low\noutput quality. The difficulty of generalizing to new translation directions\nsuggests the model representations are highly specific to those language pairs\nseen in training. We demonstrate that a main factor causing the\nlanguage-specific representations is the positional correspondence to input\ntokens. We show that this can be easily alleviated by removing residual\nconnections in an encoder layer. With this modification, we gain up to 18.5\nBLEU points on zero-shot translation while retaining quality on supervised\ndirections. The improvements are particularly prominent between related\nlanguages, where our proposed model outperforms pivot-based translation.\nMoreover, our approach allows easy integration of new languages, which\nsubstantially expands translation coverage. By thorough inspections of the\nhidden layer outputs, we show that our approach indeed leads to more\nlanguage-independent representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15150v2",
            "date": "2020-12-30 13:29:58+00:00",
            "title": "Improving BERT with Syntax-aware Local Attention",
            "authors": [
                "Zhongli Li",
                "Qingyu Zhou",
                "Chao Li",
                "Ke Xu",
                "Yunbo Cao"
            ],
            "abstract": "Pre-trained Transformer-based neural language models, such as BERT, have\nachieved remarkable results on varieties of NLP tasks. Recent works have shown\nthat attention-based models can benefit from more focused attention over local\nregions. Most of them restrict the attention scope within a linear span, or\nconfine to certain tasks such as machine translation and question answering. In\nthis paper, we propose a syntax-aware local attention, where the attention\nscopes are restrained based on the distances in the syntactic structure. The\nproposed syntax-aware local attention can be integrated with pretrained\nlanguage models, such as BERT, to render the model to focus on syntactically\nrelevant words. We conduct experiments on various single-sentence benchmarks,\nincluding sentence classification and sequence labeling tasks. Experimental\nresults show consistent gains over BERT on all benchmark datasets. The\nextensive studies verify that our model achieves better performance owing to\nmore focused attention over syntactically relevant words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15156v1",
            "date": "2020-12-30 13:46:06+00:00",
            "title": "A Memory Efficient Baseline for Open Domain Question Answering",
            "authors": [
                "Gautier Izacard",
                "Fabio Petroni",
                "Lucas Hosseini",
                "Nicola De Cao",
                "Sebastian Riedel",
                "Edouard Grave"
            ],
            "abstract": "Recently, retrieval systems based on dense representations have led to\nimportant improvements in open-domain question answering, and related tasks.\nWhile very effective, this approach is also memory intensive, as the dense\nvectors for the whole knowledge source need to be kept in memory. In this\npaper, we study how the memory footprint of dense retriever-reader systems can\nbe reduced. We consider three strategies to reduce the index size: dimension\nreduction, vector quantization and passage filtering. We evaluate our approach\non two question answering benchmarks: TriviaQA and NaturalQuestions, showing\nthat it is possible to get competitive systems using less than 6Gb of memory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15228v1",
            "date": "2020-12-30 17:14:25+00:00",
            "title": "Introducing Orthogonal Constraint in Structural Probes",
            "authors": [
                "Tomasz Limisiewicz",
                "David Mare\u010dek"
            ],
            "abstract": "With the recent success of pre-trained models in NLP, a significant focus was\nput on interpreting their representations. One of the most prominent approaches\nis structural probing (Hewitt and Manning, 2019), where a linear projection of\nlanguage vector space is performed in order to approximate the topology of\nlinguistic structures. In this work, we decompose this mapping into 1.\nisomorphic space rotation; 2. linear scaling that identifies and scales the\nmost relevant directions. We introduce novel structural tasks to exam our\nmethod's ability to disentangle information hidden in the embeddings. We\nexperimentally show that our approach can be performed in a multitask setting.\nMoreover, the orthogonal constraint identifies embedding subspaces encoding\nspecific linguistic features and make the probe less vulnerable to\nmemorization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15229v2",
            "date": "2020-12-30 17:16:33+00:00",
            "title": "Can Sequence-to-Sequence Models Crack Substitution Ciphers?",
            "authors": [
                "Nada Aldarrab",
                "Jonathan May"
            ],
            "abstract": "Decipherment of historical ciphers is a challenging problem. The language of\nthe target plaintext might be unknown, and ciphertext can have a lot of noise.\nState-of-the-art decipherment methods use beam search and a neural language\nmodel to score candidate plaintext hypotheses for a given cipher, assuming the\nplaintext language is known. We propose an end-to-end multilingual model for\nsolving simple substitution ciphers. We test our model on synthetic and real\nhistorical ciphers and show that our proposed method can decipher text without\nexplicit language identification while still being robust to noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15243v2",
            "date": "2020-12-30 17:47:24+00:00",
            "title": "Unsupervised Label-aware Event Trigger and Argument Classification",
            "authors": [
                "Hongming Zhang",
                "Haoyu Wang",
                "Dan Roth"
            ],
            "abstract": "Identifying events and mapping them to pre-defined event types has long been\nan important natural language processing problem. Most previous work has been\nheavily relying on labor-intensive and domain-specific annotations while\nignoring the semantic meaning contained in the labels of the event types. As a\nresult, the learned models cannot effectively generalize to new domains, where\nnew event types could be introduced. In this paper, we propose an unsupervised\nevent extraction pipeline, which first identifies events with available tools\n(e.g., SRL) and then automatically maps them to pre-defined event types with\nour proposed unsupervised classification model. Rather than relying on\nannotated data, our model matches the semantics of identified events with those\nof event type labels. Specifically, we leverage pre-trained language models to\ncontextually represent pre-defined types for both event triggers and arguments.\nAfter we map identified events to the target types via representation\nsimilarity, we use the event ontology (e.g., argument type \"Victim\" can only\nappear as the argument of event type \"Attack\") as global constraints to\nregularize the prediction. The proposed approach is shown to be very effective\nwhen tested on the ACE-2005 dataset, which has 33 trigger and 22 argument\ntypes. Without using any annotation, we successfully map 83% of the triggers\nand 54% of the arguments to the correct types, almost doubling the performance\nof previous zero-shot approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15263v1",
            "date": "2020-12-30 18:21:55+00:00",
            "title": "Predicting cross-linguistic adjective order with information gain",
            "authors": [
                "William Dyer",
                "Richard Futrell",
                "Zoey Liu",
                "Gregory Scontras"
            ],
            "abstract": "Languages vary in their placement of multiple adjectives before, after, or\nsurrounding the noun, but they typically exhibit strong intra-language\ntendencies on the relative order of those adjectives (e.g., the preference for\n`big blue box' in English, `grande bo\\^{i}te bleue' in French, and\n`alsund\\={u}q al'azraq alkab\\={\\i}r' in Arabic). We advance a new quantitative\naccount of adjective order across typologically-distinct languages based on\nmaximizing information gain. Our model addresses the left-right asymmetry of\nFrench-type ANA sequences with the same approach as AAN and NAA orderings,\nwithout appeal to other mechanisms. We find that, across 32 languages, the\npreferred order of adjectives largely mirrors an efficient algorithm of\nmaximizing information gain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15283v1",
            "date": "2020-12-30 18:57:16+00:00",
            "title": "DEER: A Data Efficient Language Model for Event Temporal Reasoning",
            "authors": [
                "Rujun Han",
                "Xiang Ren",
                "Nanyun Peng"
            ],
            "abstract": "Pretrained language models (LMs) such as BERT, RoBERTa, and ELECTRA are\neffective at improving the performances of a variety of downstream NLP tasks.\nRecently, researchers have incorporated domain and task-specific knowledge in\nthese LMs' training objectives and further enhanced models' capability of\nhandling downstream tasks. However, none of these LMs are designed specifically\nfor event temporal reasoning. We propose DEER, a language model that is trained\nto focus on event temporal relations and performs better under low-resource\nsettings than original LMs. More specifically, we create a large number of\ntraining samples to simulate the machine reading comprehension and information\nextraction tasks for event temporal understanding and leverage a\ngenerator-discriminator structure to reinforce the LMs' capability of event\ntemporal reasoning. Our experimental results show that DEER can achieve SOTA\nresults and works particularly well in low-resource settings across 5 widely\nused datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15349v1",
            "date": "2020-12-30 22:38:21+00:00",
            "title": "DynaSent: A Dynamic Benchmark for Sentiment Analysis",
            "authors": [
                "Christopher Potts",
                "Zhengxuan Wu",
                "Atticus Geiger",
                "Douwe Kiela"
            ],
            "abstract": "We introduce DynaSent ('Dynamic Sentiment'), a new English-language benchmark\ntask for ternary (positive/negative/neutral) sentiment analysis. DynaSent\ncombines naturally occurring sentences with sentences created using the\nopen-source Dynabench Platform, which facilities human-and-model-in-the-loop\ndataset creation. DynaSent has a total of 121,634 sentences, each validated by\nfive crowdworkers, and its development and test splits are designed to produce\nchance performance for even the best models we have been able to develop; when\nfuture models solve this task, we will use them to create DynaSent version 2,\ncontinuing the dynamic evolution of this benchmark. Here, we report on the\ndataset creation effort, focusing on the steps we took to increase quality and\nreduce artifacts. We also present evidence that DynaSent's Neutral category is\nmore coherent than the comparable category in other benchmarks, and we motivate\ntraining models from scratch for each round over successive fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15353v1",
            "date": "2020-12-30 22:52:29+00:00",
            "title": "Deriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings",
            "authors": [
                "Jacob Turton",
                "David Vinson",
                "Robert Elliott Smith"
            ],
            "abstract": "Models based on the transformer architecture, such as BERT, have marked a\ncrucial step forward in the field of Natural Language Processing. Importantly,\nthey allow the creation of word embeddings that capture important semantic\ninformation about words in context. However, as single entities, these\nembeddings are difficult to interpret and the models used to create them have\nbeen described as opaque. Binder and colleagues proposed an intuitive embedding\nspace where each dimension is based on one of 65 core semantic features.\nUnfortunately, the space only exists for a small dataset of 535 words, limiting\nits uses. Previous work (Utsumi, 2018, 2020, Turton, Vinson & Smith, 2020) has\nshown that Binder features can be derived from static embeddings and\nsuccessfully extrapolated to a large new vocabulary. Taking the next step, this\npaper demonstrates that Binder features can be derived from the BERT embedding\nspace. This provides contextualised Binder embeddings, which can aid in\nunderstanding semantic differences between words in context. It additionally\nprovides insights into how semantic features are represented across the\ndifferent layers of the BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15409v3",
            "date": "2020-12-31 02:46:47+00:00",
            "title": "UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning",
            "authors": [
                "Wei Li",
                "Can Gao",
                "Guocheng Niu",
                "Xinyan Xiao",
                "Hao Liu",
                "Jiachen Liu",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Existed pre-training methods either focus on single-modal tasks or\nmulti-modal tasks, and cannot effectively adapt to each other. They can only\nutilize single-modal data (i.e. text or image) or limited multi-modal data\n(i.e. image-text pairs). In this work, we propose a unified-modal pre-training\narchitecture, namely UNIMO, which can effectively adapt to both single-modal\nand multi-modal understanding and generation tasks. Large scale of free text\ncorpus and image collections can be utilized to improve the capability of\nvisual and textual understanding, and cross-modal contrastive learning (CMCL)\nis leveraged to align the textual and visual information into a unified\nsemantic space over a corpus of image-text pairs. As the non-paired\nsingle-modal data is very rich, our model can utilize much larger scale of data\nto learn more generalizable representations. Moreover, the textual knowledge\nand visual knowledge can enhance each other in the unified semantic space. The\nexperimental results show that UNIMO significantly improves the performance of\nseveral single-modal and multi-modal downstream tasks. Our code and pre-trained\nmodels are public at\nhttps://github.com/PaddlePaddle/Research/tree/master/NLP/UNIMO",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15421v1",
            "date": "2020-12-31 03:24:34+00:00",
            "title": "Verb Knowledge Injection for Multilingual Event Processing",
            "authors": [
                "Olga Majewska",
                "Ivan Vuli\u0107",
                "Goran Glava\u0161",
                "Edoardo M. Ponti",
                "Anna Korhonen"
            ],
            "abstract": "In parallel to their overwhelming success across NLP tasks, language ability\nof deep Transformer networks, pretrained via language modeling (LM) objectives\nhas undergone extensive scrutiny. While probing revealed that these models\nencode a range of syntactic and semantic properties of a language, they are\nstill prone to fall back on superficial cues and simple heuristics to solve\ndownstream tasks, rather than leverage deeper linguistic knowledge. In this\npaper, we target one such area of their deficiency, verbal reasoning. We\ninvestigate whether injecting explicit information on verbs' semantic-syntactic\nbehaviour improves the performance of LM-pretrained Transformers in event\nextraction tasks -- downstream tasks for which accurate verb processing is\nparamount. Concretely, we impart the verb knowledge from curated lexical\nresources into dedicated adapter modules (dubbed verb adapters), allowing it to\ncomplement, in downstream tasks, the language knowledge obtained during\nLM-pretraining. We first demonstrate that injecting verb knowledge leads to\nperformance gains in English event extraction. We then explore the utility of\nverb adapters for event extraction in other languages: we investigate (1)\nzero-shot language transfer with multilingual Transformers as well as (2)\ntransfer via (noisy automatic) translation of English verb-based lexical\nconstraints. Our results show that the benefits of verb knowledge injection\nindeed extend to other languages, even when verb adapters are trained on\nnoisily translated constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15425v1",
            "date": "2020-12-31 03:32:58+00:00",
            "title": "The jsRealB Text Realizer: Organization and Use Cases",
            "authors": [
                "Guy Lapalme"
            ],
            "abstract": "This paper describes the design principles behind jsRealB, a surface realizer\nwritten in JavaScript for English or French sentences from a specification\ninspired by the constituent syntax formalism. It can be used either within a\nweb page or as a node .js module. We show that the seemingly simple process of\ntext realization involves many interesting implementation challenges in order\nto take into account the specifics of each language. jsRealB has a large\ncoverage of English and French and has been used to develop realistic\ndata-to-text applications and to reproduce existing literary texts and\nsentences with Universal Dependency annotations. Its source code and that of\nits applications are available on GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15455v1",
            "date": "2020-12-31 05:28:42+00:00",
            "title": "Exploring Monolingual Data for Neural Machine Translation with Knowledge Distillation",
            "authors": [
                "Alham Fikri Aji",
                "Kenneth Heafield"
            ],
            "abstract": "We explore two types of monolingual data that can be included in knowledge\ndistillation training for neural machine translation (NMT). The first is the\nsource-side monolingual data. Second, is the target-side monolingual data that\nis used as back-translation data. Both datasets are (forward-)translated by a\nteacher model from source-language to target-language, which are then combined\ninto a dataset for smaller student models. We find that source-side monolingual\ndata improves model performance when evaluated by test-set originated from\nsource-side. Likewise, target-side data has a positive effect on the test-set\nin the opposite direction. We also show that it is not required to train the\nstudent model with the same data used by the teacher, as long as the domains\nare the same. Finally, we find that combining source-side and target-side\nyields in better performance than relying on just one side of the monolingual\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15466v1",
            "date": "2020-12-31 06:40:13+00:00",
            "title": "CLEAR: Contrastive Learning for Sentence Representation",
            "authors": [
                "Zhuofeng Wu",
                "Sinong Wang",
                "Jiatao Gu",
                "Madian Khabsa",
                "Fei Sun",
                "Hao Ma"
            ],
            "abstract": "Pre-trained language models have proven their unique powers in capturing\nimplicit language features. However, most pre-training approaches focus on the\nword-level training objective, while sentence-level objectives are rarely\nstudied. In this paper, we propose Contrastive LEArning for sentence\nRepresentation (CLEAR), which employs multiple sentence-level augmentation\nstrategies in order to learn a noise-invariant sentence representation. These\naugmentations include word and span deletion, reordering, and substitution.\nFurthermore, we investigate the key reasons that make contrastive learning\neffective through numerous experiments. We observe that different sentence\naugmentations during pre-training lead to different performance improvements on\nvarious downstream tasks. Our approach is shown to outperform multiple existing\nmethods on both SentEval and GLUE benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15482v1",
            "date": "2020-12-31 07:22:15+00:00",
            "title": "FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation",
            "authors": [
                "Kushal Lakhotia",
                "Bhargavi Paranjape",
                "Asish Ghoshal",
                "Wen-tau Yih",
                "Yashar Mehdad",
                "Srinivasan Iyer"
            ],
            "abstract": "Natural language (NL) explanations of model predictions are gaining\npopularity as a means to understand and verify decisions made by large\nblack-box pre-trained models, for NLP tasks such as Question Answering (QA) and\nFact Verification. Recently, pre-trained sequence to sequence (seq2seq) models\nhave proven to be very effective in jointly making predictions, as well as\ngenerating NL explanations. However, these models have many shortcomings; they\ncan fabricate explanations even for incorrect predictions, they are difficult\nto adapt to long input documents, and their training requires a large amount of\nlabeled data. In this paper, we develop FiD-Ex, which addresses these\nshortcomings for seq2seq models by: 1) introducing sentence markers to\neliminate explanation fabrication by encouraging extractive generation, 2)\nusing the fusion-in-decoder architecture to handle long input contexts, and 3)\nintermediate fine-tuning on re-structured open domain QA datasets to improve\nfew-shot performance. FiD-Ex significantly improves over prior work in terms of\nexplanation metrics and task accuracy, on multiple tasks from the ERASER\nexplainability benchmark, both in the fully supervised and in the few-shot\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15515v1",
            "date": "2020-12-31 09:35:27+00:00",
            "title": "Neural Machine Translation: A Review of Methods, Resources, and Tools",
            "authors": [
                "Zhixing Tan",
                "Shuo Wang",
                "Zonghan Yang",
                "Gang Chen",
                "Xuancheng Huang",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "Machine translation (MT) is an important sub-field of natural language\nprocessing that aims to translate natural languages using computers. In recent\nyears, end-to-end neural machine translation (NMT) has achieved great success\nand has become the new mainstream method in practical MT systems. In this\narticle, we first provide a broad review of the methods for NMT and focus on\nmethods relating to architectures, decoding, and data augmentation. Then we\nsummarize the resources and tools that are useful for researchers. Finally, we\nconclude with a discussion of possible future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15516v2",
            "date": "2020-12-31 09:35:39+00:00",
            "title": "AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding",
            "authors": [
                "Wissam Antoun",
                "Fady Baly",
                "Hazem Hajj"
            ],
            "abstract": "Advances in English language representation enabled a more sample-efficient\npre-training task by Efficiently Learning an Encoder that Classifies Token\nReplacements Accurately (ELECTRA). Which, instead of training a model to\nrecover masked tokens, it trains a discriminator model to distinguish true\ninput tokens from corrupted tokens that were replaced by a generator network.\nOn the other hand, current Arabic language representation approaches rely only\non pretraining via masked language modeling. In this paper, we develop an\nArabic language representation model, which we name AraELECTRA. Our model is\npretrained using the replaced token detection objective on large Arabic text\ncorpora. We evaluate our model on multiple Arabic NLP tasks, including reading\ncomprehension, sentiment analysis, and named-entity recognition and we show\nthat AraELECTRA outperforms current state-of-the-art Arabic language\nrepresentation models, given the same pretraining data and with even a smaller\nmodel size.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15520v2",
            "date": "2020-12-31 09:48:05+00:00",
            "title": "AraGPT2: Pre-Trained Transformer for Arabic Language Generation",
            "authors": [
                "Wissam Antoun",
                "Fady Baly",
                "Hazem Hajj"
            ],
            "abstract": "Recently, pre-trained transformer-based architectures have proven to be very\nefficient at language modeling and understanding, given that they are trained\non a large enough corpus. Applications in language generation for Arabic are\nstill lagging in comparison to other NLP advances primarily due to the lack of\nadvanced Arabic language generation models. In this paper, we develop the first\nadvanced Arabic language generation model, AraGPT2, trained from scratch on a\nlarge Arabic corpus of internet text and news articles. Our largest model,\nAraGPT2-mega, has 1.46 billion parameters, which makes it the largest Arabic\nlanguage model available. The Mega model was evaluated and showed success on\ndifferent tasks including synthetic news generation, and zero-shot question\nanswering. For text generation, our best model achieves a perplexity of 29.8 on\nheld-out Wikipedia articles. A study conducted with human evaluators showed the\nsignificant success of AraGPT2-mega in generating news articles that are\ndifficult to distinguish from articles written by humans. We thus develop and\nrelease an automatic discriminator model with a 98% percent accuracy in\ndetecting model-generated text. The models are also publicly available, hoping\nto encourage new research directions and applications for Arabic NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15524v1",
            "date": "2020-12-31 10:01:29+00:00",
            "title": "Linear-Time WordPiece Tokenization",
            "authors": [
                "Xinying Song",
                "Alex Salcianu",
                "Yang Song",
                "Dave Dopson",
                "Denny Zhou"
            ],
            "abstract": "WordPiece tokenization is a subword-based tokenization schema adopted by\nBERT: it segments the input text via a longest-match-first tokenization\nstrategy, known as Maximum Matching or MaxMatch. To the best of our knowledge,\nall published MaxMatch algorithms are quadratic (or higher). In this paper, we\npropose LinMaxMatch, a novel linear-time algorithm for MaxMatch and WordPiece\ntokenization. Inspired by the Aho-Corasick algorithm, we introduce additional\nlinkages on top of the trie built from the vocabulary, allowing smart\ntransitions when the trie matching cannot continue. Experimental results show\nthat our algorithm is 3x faster on average than two production systems by\nHuggingFace and TensorFlow Text. Regarding long-tail inputs, our algorithm is\n4.5x faster at the 95 percentile. This work has immediate practical value\n(reducing inference latency, saving compute resources, etc.) and is of\ntheoretical interest by providing an optimal complexity solution to the\ndecades-old MaxMatch problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15525v2",
            "date": "2020-12-31 10:09:29+00:00",
            "title": "BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining",
            "authors": [
                "Weizhen Qi",
                "Yeyun Gong",
                "Jian Jiao",
                "Yu Yan",
                "Dayiheng Liu",
                "Weizhu Chen",
                "Kewen Tang",
                "Houqiang Li",
                "Jiusheng Chen",
                "Ruofei Zhang",
                "Ming Zhou",
                "Nan Duan"
            ],
            "abstract": "In this paper, we propose BANG, a new pretraining model to Bridge the gap\nbetween Autoregressive (AR) and Non-autoregressive (NAR) Generation. AR and NAR\ngeneration can be uniformly regarded as to what extent previous tokens can be\nattended, and BANG bridges AR and NAR generation by designing a novel model\nstructure for large-scale pretraining. The pretrained BANG model can\nsimultaneously support AR, NAR and semi-NAR generation to meet different\nrequirements. Experiments on question generation (SQuAD 1.1), summarization\n(XSum) and dialogue generation (PersonaChat) show that BANG improves NAR and\nsemi-NAR performance significantly as well as attaining comparable performance\nwith strong AR pretrained models. Compared with the semi-NAR strong baselines,\nBANG achieves absolute improvements of 14.01 and 5.24 in the overall scores of\nSQuAD 1.1 and XSum, respectively. In addition, BANG achieves absolute\nimprovements of 10.73, 6.39 and 5.90 in the overall scores of SQuAD, XSUM and\nPersonaChat respectively compared with the strong NAR baselines. Our code will\nbe made publicly available at https://github.com/microsoft/BANG .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15534v1",
            "date": "2020-12-31 10:36:01+00:00",
            "title": "HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions",
            "authors": [
                "Shaobo Li",
                "Xiaoguang Li",
                "Lifeng Shang",
                "Xin Jiang",
                "Qun Liu",
                "Chengjie Sun",
                "Zhenzhou Ji",
                "Bingquan Liu"
            ],
            "abstract": "Collecting supporting evidence from large corpora of text (e.g., Wikipedia)\nis of great challenge for open-domain Question Answering (QA). Especially, for\nmulti-hop open-domain QA, scattered evidence pieces are required to be gathered\ntogether to support the answer extraction. In this paper, we propose a new\nretrieval target, hop, to collect the hidden reasoning evidence from Wikipedia\nfor complex question answering. Specifically, the hop in this paper is defined\nas the combination of a hyperlink and the corresponding outbound link document.\nThe hyperlink is encoded as the mention embedding which models the structured\nknowledge of how the outbound link entity is mentioned in the textual context,\nand the corresponding outbound link document is encoded as the document\nembedding representing the unstructured knowledge within it. Accordingly, we\nbuild HopRetriever which retrieves hops over Wikipedia to answer complex\nquestions. Experiments on the HotpotQA dataset demonstrate that HopRetriever\noutperforms previously published evidence retrieval methods by large margins.\nMoreover, our approach also yields quantifiable interpretations of the evidence\ncollection process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15547v1",
            "date": "2020-12-31 11:16:51+00:00",
            "title": "XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders",
            "authors": [
                "Shuming Ma",
                "Jian Yang",
                "Haoyang Huang",
                "Zewen Chi",
                "Li Dong",
                "Dongdong Zhang",
                "Hany Hassan Awadalla",
                "Alexandre Muzio",
                "Akiko Eriguchi",
                "Saksham Singhal",
                "Xia Song",
                "Arul Menezes",
                "Furu Wei"
            ],
            "abstract": "Multilingual machine translation enables a single model to translate between\ndifferent languages. Most existing multilingual machine translation systems\nadopt a randomly initialized Transformer backbone. In this work, inspired by\nthe recent success of language model pre-training, we present XLM-T, which\ninitializes the model with an off-the-shelf pretrained cross-lingual\nTransformer encoder and fine-tunes it with multilingual parallel data. This\nsimple method achieves significant improvements on a WMT dataset with 10\nlanguage pairs and the OPUS-100 corpus with 94 pairs. Surprisingly, the method\nis also effective even upon the strong baseline with back-translation.\nMoreover, extensive analysis of XLM-T on unsupervised syntactic parsing, word\nalignment, and multilingual classification explains its effectiveness for\nmachine translation. The code will be at https://aka.ms/xlm-t.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15562v2",
            "date": "2020-12-31 11:37:28+00:00",
            "title": "UNKs Everywhere: Adapting Multilingual Language Models to New Scripts",
            "authors": [
                "Jonas Pfeiffer",
                "Ivan Vuli\u0107",
                "Iryna Gurevych",
                "Sebastian Ruder"
            ],
            "abstract": "Massively multilingual language models such as multilingual BERT offer\nstate-of-the-art cross-lingual transfer performance on a range of NLP tasks.\nHowever, due to limited capacity and large differences in pretraining data\nsizes, there is a profound performance gap between resource-rich and\nresource-poor target languages. The ultimate challenge is dealing with\nunder-resourced languages not covered at all by the models and written in\nscripts unseen during pretraining. In this work, we propose a series of novel\ndata-efficient methods that enable quick and effective adaptation of pretrained\nmultilingual models to such low-resource languages and unseen scripts. Relying\non matrix factorization, our methods capitalize on the existing latent\nknowledge about multiple languages already available in the pretrained model's\nembedding matrix. Furthermore, we show that learning of the new dedicated\nembedding matrix in the target language can be improved by leveraging a small\nnumber of vocabulary items (i.e., the so-called lexically overlapping tokens)\nshared between mBERT's and target language vocabulary. Our adaptation\ntechniques offer substantial performance gains for languages with unseen\nscripts. We also demonstrate that they can yield improvements for low-resource\nlanguages written in scripts covered by the pretrained model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15573v1",
            "date": "2020-12-31 12:18:41+00:00",
            "title": "Coreference Reasoning in Machine Reading Comprehension",
            "authors": [
                "Mingzhu Wu",
                "Nafise Sadat Moosavi",
                "Dan Roth",
                "Iryna Gurevych"
            ],
            "abstract": "The ability to reason about multiple references to a given entity is\nessential for natural language understanding and has been long studied in NLP.\nIn recent years, as the format of Question Answering (QA) became a standard for\nmachine reading comprehension (MRC), there have been data collection efforts,\ne.g., Dasigi et al. (2019), that attempt to evaluate the ability of MRC models\nto reason about coreference. However, as we show, coreference reasoning in MRC\nis a greater challenge than was earlier thought; MRC datasets do not reflect\nthe natural distribution and, consequently, the challenges of coreference\nreasoning. Specifically, success on these datasets does not reflect a model's\nproficiency in coreference reasoning. We propose a methodology for creating\nreading comprehension datasets that better reflect the challenges of\ncoreference reasoning and use it to show that state-of-the-art models still\nstruggle with these phenomena. Furthermore, we develop an effective way to use\nnaturally occurring coreference phenomena from annotated coreference resolution\ndatasets when training MRC models. This allows us to show an improvement in the\ncoreference reasoning abilities of state-of-the-art models across various MRC\ndatasets. We will release all the code and the resulting dataset at\nhttps://github.com/UKPLab/coref-reasoning-in-qa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15606v2",
            "date": "2020-12-31 13:44:56+00:00",
            "title": "HateCheck: Functional Tests for Hate Speech Detection Models",
            "authors": [
                "Paul R\u00f6ttger",
                "Bertram Vidgen",
                "Dong Nguyen",
                "Zeerak Waseem",
                "Helen Margetts",
                "Janet B. Pierrehumbert"
            ],
            "abstract": "Detecting online hate is a difficult task that even state-of-the-art models\nstruggle with. Typically, hate speech detection models are evaluated by\nmeasuring their performance on held-out test data using metrics such as\naccuracy and F1 score. However, this approach makes it difficult to identify\nspecific model weak points. It also risks overestimating generalisable model\nperformance due to increasingly well-evidenced systematic gaps and biases in\nhate speech datasets. To enable more targeted diagnostic insights, we introduce\nHateCheck, a suite of functional tests for hate speech detection models. We\nspecify 29 model functionalities motivated by a review of previous research and\na series of interviews with civil society stakeholders. We craft test cases for\neach functionality and validate their quality through a structured annotation\nprocess. To illustrate HateCheck's utility, we test near-state-of-the-art\ntransformer models as well as two popular commercial models, revealing critical\nmodel weaknesses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15613v2",
            "date": "2020-12-31 14:11:00+00:00",
            "title": "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models",
            "authors": [
                "Phillip Rust",
                "Jonas Pfeiffer",
                "Ivan Vuli\u0107",
                "Sebastian Ruder",
                "Iryna Gurevych"
            ],
            "abstract": "In this work, we provide a systematic and comprehensive empirical comparison\nof pretrained multilingual language models versus their monolingual\ncounterparts with regard to their monolingual task performance. We study a set\nof nine typologically diverse languages with readily available pretrained\nmonolingual models on a set of five diverse monolingual downstream tasks. We\nfirst aim to establish, via fair and controlled comparisons, if a gap between\nthe multilingual and the corresponding monolingual representation of that\nlanguage exists, and subsequently investigate the reason for any performance\ndifference. To disentangle conflating factors, we train new monolingual models\non the same data, with monolingually and multilingually trained tokenizers. We\nfind that while the pretraining data size is an important factor, a designated\nmonolingual tokenizer plays an equally important role in the downstream\nperformance. Our results show that languages that are adequately represented in\nthe multilingual model's vocabulary exhibit negligible performance decreases\nover their monolingual counterparts. We further find that replacing the\noriginal multilingual tokenizer with the specialized monolingual tokenizer\nimproves the downstream performance of the multilingual model for almost every\ntask and language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15621v1",
            "date": "2020-12-31 14:23:55+00:00",
            "title": "Open Korean Corpora: A Practical Report",
            "authors": [
                "Won Ik Cho",
                "Sangwhan Moon",
                "Youngsook Song"
            ],
            "abstract": "Korean is often referred to as a low-resource language in the research\ncommunity. While this claim is partially true, it is also because the\navailability of resources is inadequately advertised and curated. This work\ncurates and reviews a list of Korean corpora, first describing\ninstitution-level resource development, then further iterate through a list of\ncurrent open datasets for different types of tasks. We then propose a direction\non how open-source dataset construction and releases should be done for\nless-resourced languages to promote research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15639v1",
            "date": "2020-12-31 14:58:01+00:00",
            "title": "TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis",
            "authors": [
                "Haisong Zhang",
                "Lemao Liu",
                "Haiyun Jiang",
                "Yangming Li",
                "Enbo Zhao",
                "Kun Xu",
                "Linfeng Song",
                "Suncong Zheng",
                "Botong Zhou",
                "Jianchen Zhu",
                "Xiao Feng",
                "Tao Chen",
                "Tao Yang",
                "Dong Yu",
                "Feng Zhang",
                "Zhanhui Kang",
                "Shuming Shi"
            ],
            "abstract": "This technique report introduces TexSmart, a text understanding system that\nsupports fine-grained named entity recognition (NER) and enhanced semantic\nanalysis functionalities. Compared to most previous publicly available text\nunderstanding systems and tools, TexSmart holds some unique features. First,\nthe NER function of TexSmart supports over 1,000 entity types, while most other\npublic tools typically support several to (at most) dozens of entity types.\nSecond, TexSmart introduces new semantic analysis functions like semantic\nexpansion and deep semantic representation, that are absent in most previous\nsystems. Third, a spectrum of algorithms (from very fast algorithms to those\nthat are relatively slow but more accurate) are implemented for one function in\nTexSmart, to fulfill the requirements of different academic and industrial\napplications. The adoption of unsupervised or weakly-supervised algorithms is\nespecially emphasized, with the goal of easily updating our models to include\nfresh data with less human annotation efforts.\n  The main contents of this report include major functions of TexSmart,\nalgorithms for achieving these functions, how to use the TexSmart toolkit and\nWeb APIs, and evaluation results of some key algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15643v1",
            "date": "2020-12-31 15:05:36+00:00",
            "title": "CoCoLM: COmplex COmmonsense Enhanced Language Model",
            "authors": [
                "Changlong Yu",
                "Hongming Zhang",
                "Yangqiu Song",
                "Wilfred Ng"
            ],
            "abstract": "Large-scale pre-trained language models have demonstrated strong knowledge\nrepresentation ability. However, recent studies suggest that even though these\ngiant models contains rich simple commonsense knowledge (e.g., bird can fly and\nfish can swim.), they often struggle with the complex commonsense knowledge\nthat involves multiple eventualities (verb-centric phrases, e.g., identifying\nthe relationship between ``Jim yells at Bob'' and ``Bob is upset'').To address\nthis problem, in this paper, we propose to help pre-trained language models\nbetter incorporate complex commonsense knowledge. Different from existing\nfine-tuning approaches, we do not focus on a specific task and propose a\ngeneral language model named CoCoLM. Through the careful training over a\nlarge-scale eventuality knowledge graphs ASER, we successfully teach\npre-trained language models (i.e., BERT and RoBERTa) rich complex commonsense\nknowledge among eventualities. Experiments on multiple downstream commonsense\ntasks that requires the correct understanding of eventualities demonstrate the\neffectiveness of CoCoLM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15671v1",
            "date": "2020-12-31 15:49:49+00:00",
            "title": "VOLT: Improving Vocabularization via Optimal Transport for Machine Translation",
            "authors": [
                "Jingjing Xu",
                "Hao Zhou",
                "Chun Gan",
                "Zaixiang Zheng",
                "Lei Li"
            ],
            "abstract": "It is well accepted that the choice of token vocabulary largely affects the\nperformance of machine translation. However, due to expensive trial costs, most\nstudies only conduct simple trials with dominant approaches (e.g BPE) and\ncommonly used vocabulary sizes. In this paper, we find an exciting relation\nbetween an information-theoretic feature and BLEU scores. With this\nobservation, we formulate the quest of vocabularization -- finding the best\ntoken dictionary with a proper size -- as an optimal transport problem. We then\npropose VOLT, a simple and efficient vocabularization solution without the full\nand costly trial training. We evaluate our approach on multiple machine\ntranslation tasks, including WMT-14 English-German translation, TED bilingual\ntranslation, and TED multilingual translation. Empirical results show that VOLT\nbeats widely-used vocabularies on diverse scenarios. For example, VOLT achieves\n70% vocabulary size reduction and 0.6 BLEU gain on English-German translation.\nAlso, one advantage of VOLT lies in its low resource consumption. Compared to\nnaive BPE-search, VOLT reduces the search time from 288 GPU hours to 0.5 CPU\nhours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15674v3",
            "date": "2020-12-31 15:52:27+00:00",
            "title": "ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora",
            "authors": [
                "Xuan Ouyang",
                "Shuohuan Wang",
                "Chao Pang",
                "Yu Sun",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Recent studies have demonstrated that pre-trained cross-lingual models\nachieve impressive performance on downstream cross-lingual tasks. This\nimprovement stems from the learning of a large amount of monolingual and\nparallel corpora. While it is generally acknowledged that parallel corpora are\ncritical for improving the model performance, existing methods are often\nconstrained by the size of parallel corpora, especially for the low-resource\nlanguages. In this paper, we propose ERNIE-M, a new training method that\nencourages the model to align the representation of multiple languages with\nmonolingual corpora, to break the constraint of parallel corpus size on the\nmodel performance. Our key insight is to integrate the idea of back translation\nin the pre-training process. We generate pseudo-parallel sentences pairs on a\nmonolingual corpus to enable the learning of semantic alignment between\ndifferent languages, which enhances the semantic modeling of cross-lingual\nmodels. Experimental results show that ERNIE-M outperforms existing\ncross-lingual models and delivers new state-of-the-art results on various\ncross-lingual downstream tasks. The codes and pre-trained models will be made\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15682v2",
            "date": "2020-12-31 16:03:48+00:00",
            "title": "A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters",
            "authors": [
                "Mengjie Zhao",
                "Yi Zhu",
                "Ehsan Shareghi",
                "Ivan Vuli\u0107",
                "Roi Reichart",
                "Anna Korhonen",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Few-shot crosslingual transfer has been shown to outperform its zero-shot\ncounterpart with pretrained encoders like multilingual BERT. Despite its\ngrowing popularity, little to no attention has been paid to standardizing and\nanalyzing the design of few-shot experiments. In this work, we highlight a\nfundamental risk posed by this shortcoming, illustrating that the model\nexhibits a high degree of sensitivity to the selection of few shots. We conduct\na large-scale experimental study on 40 sets of sampled few shots for six\ndiverse NLP tasks across up to 40 languages. We provide an analysis of success\nand failure cases of few-shot transfer, which highlights the role of lexical\nfeatures. Additionally, we show that a straightforward full model finetuning\napproach is quite effective for few-shot transfer, outperforming several\nstate-of-the-art few-shot approaches. As a step towards standardizing few-shot\ncrosslingual experimental designs, we make our sampled few shots publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15688v2",
            "date": "2020-12-31 16:12:48+00:00",
            "title": "ERNIE-Doc: A Retrospective Long-Document Modeling Transformer",
            "authors": [
                "Siyu Ding",
                "Junyuan Shang",
                "Shuohuan Wang",
                "Yu Sun",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Transformers are not suited for processing long documents, due to their\nquadratically increasing memory and time consumption. Simply truncating a long\ndocument or applying the sparse attention mechanism will incur the context\nfragmentation problem or lead to an inferior modeling capability against\ncomparable model sizes. In this paper, we propose ERNIE-Doc, a document-level\nlanguage pretraining model based on Recurrence Transformers. Two well-designed\ntechniques, namely the retrospective feed mechanism and the enhanced recurrence\nmechanism, enable ERNIE-Doc, which has a much longer effective context length,\nto capture the contextual information of a complete document. We pretrain\nERNIE-Doc to explicitly learn the relationships among segments with an\nadditional document-aware segment-reordering objective. Various experiments\nwere conducted on both English and Chinese document-level tasks. ERNIE-Doc\nimproved the state-of-the-art language modeling result of perplexity to 16.8 on\nWikiText-103. Moreover, it outperformed competitive pretraining models by a\nlarge margin on most language understanding tasks, such as text classification\nand question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15699v2",
            "date": "2020-12-31 16:28:07+00:00",
            "title": "Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning",
            "authors": [
                "Chenglei Si",
                "Zhengyan Zhang",
                "Fanchao Qi",
                "Zhiyuan Liu",
                "Yasheng Wang",
                "Qun Liu",
                "Maosong Sun"
            ],
            "abstract": "Pretrained language models (PLMs) perform poorly under adversarial attacks.\nTo improve the adversarial robustness, adversarial data augmentation (ADA) has\nbeen widely adopted to cover more search space of adversarial attacks by adding\ntextual adversarial examples during training. However, the number of\nadversarial examples for text augmentation is still extremely insufficient due\nto the exponentially large attack search space. In this work, we propose a\nsimple and effective method to cover a much larger proportion of the attack\nsearch space, called Adversarial and Mixup Data Augmentation (AMDA).\nSpecifically, AMDA linearly interpolates the representations of pairs of\ntraining samples to form new virtual samples, which are more abundant and\ndiverse than the discrete text adversarial examples in conventional ADA.\nMoreover, to fairly evaluate the robustness of different models, we adopt a\nchallenging evaluation setup, which generates a new set of adversarial examples\ntargeting each model. In text classification experiments of BERT and RoBERTa,\nAMDA achieves significant robustness gains under two strong adversarial attacks\nand alleviates the performance degradation of ADA on the clean data. Our code\nis released at: https://github.com/thunlp/MixADA .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15701v1",
            "date": "2020-12-31 16:34:54+00:00",
            "title": "BinaryBERT: Pushing the Limit of BERT Quantization",
            "authors": [
                "Haoli Bai",
                "Wei Zhang",
                "Lu Hou",
                "Lifeng Shang",
                "Jing Jin",
                "Xin Jiang",
                "Qun Liu",
                "Michael Lyu",
                "Irwin King"
            ],
            "abstract": "The rapid development of large pre-trained language models has greatly\nincreased the demand for model compression techniques, among which quantization\nis a popular solution. In this paper, we propose BinaryBERT, which pushes BERT\nquantization to the limit with weight binarization. We find that a binary BERT\nis hard to be trained directly than a ternary counterpart due to its complex\nand irregular loss landscapes. Therefore, we propose ternary weight splitting,\nwhich initializes the binary model by equivalent splitting from a half-sized\nternary network. The binary model thus inherits the good performance of the\nternary model, and can be further enhanced by fine-tuning the new architecture\nafter splitting. Empirical results show that BinaryBERT has negligible\nperformance drop compared to the full-precision BERT-base while being\n$24\\times$ smaller, achieving the state-of-the-art results on GLUE and SQuAD\nbenchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15710v1",
            "date": "2020-12-31 16:55:05+00:00",
            "title": "Revisiting Robust Neural Machine Translation: A Transformer Case Study",
            "authors": [
                "Peyman Passban",
                "Puneeth S. M. Saladi",
                "Qun Liu"
            ],
            "abstract": "Transformers (Vaswani et al., 2017) have brought a remarkable improvement in\nthe performance of neural machine translation (NMT) systems, but they could be\nsurprisingly vulnerable to noise. Accordingly, we tried to investigate how\nnoise breaks Transformers and if there exist solutions to deal with such\nissues. There is a large body of work in the NMT literature on analyzing the\nbehaviour of conventional models for the problem of noise but it seems\nTransformers are understudied in this context.\n  Therefore, we introduce a novel data-driven technique to incorporate noise\nduring training. This idea is comparable to the well-known fine-tuning\nstrategy. Moreover, we propose two new extensions to the original Transformer,\nthat modify the neural architecture as well as the training process to handle\nnoise. We evaluated our techniques to translate the English--German pair in\nboth directions. Experimental results show that our models have a higher\ntolerance to noise. More specifically, they perform with no deterioration where\nup to 10% of entire test words are infected by noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15717v1",
            "date": "2020-12-31 17:15:09+00:00",
            "title": "FDMT: A Benchmark Dataset for Fine-grained Domain Adaptation in Machine Translation",
            "authors": [
                "Wenhao Zhu",
                "Shujian Huang",
                "Tong Pu",
                "Xu Zhang",
                "Jian Yu",
                "Wei Chen",
                "Yanfeng Wang",
                "Jiajun Chen"
            ],
            "abstract": "Previous domain adaptation research usually neglect the diversity in\ntranslation within a same domain, which is a core problem for adapting a\ngeneral neural machine translation (NMT) model into a specific domain in\nreal-world scenarios. One representative of such challenging scenarios is to\ndeploy a translation system for a conference with a specific topic, e.g.\ncomputer networks or natural language processing, where there is usually\nextremely less resources due to the limited time schedule. To motivate a wide\ninvestigation in such settings, we present a real-world fine-grained domain\nadaptation task in machine translation (FDMT). The FDMT dataset (Zh-En)\nconsists of four sub-domains of information technology: autonomous vehicles, AI\neducation, real-time networks and smart phone. To be closer to reality, FDMT\ndoes not employ any in-domain bilingual training data. Instead, each sub-domain\nis equipped with monolingual data, bilingual dictionary and knowledge base, to\nencourage in-depth exploration of these available resources. Corresponding\ndevelopment set and test set are provided for evaluation purpose. We make\nquantitative experiments and deep analyses in this new setting, which\nbenchmarks the fine-grained domain adaptation task and reveals several\nchallenging problems that need to be addressed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15784v1",
            "date": "2020-12-31 18:07:07+00:00",
            "title": "Understanding Politics via Contextualized Discourse Processing",
            "authors": [
                "Rajkumar Pujari",
                "Dan Goldwasser"
            ],
            "abstract": "Politicians often have underlying agendas when reacting to events. Arguments\nin contexts of various events reflect a fairly consistent set of agendas for a\ngiven entity. In spite of recent advances in Pretrained Language Models (PLMs),\nthose text representations are not designed to capture such nuanced patterns.\nIn this paper, we propose a Compositional Reader model consisting of encoder\nand composer modules, that attempts to capture and leverage such information to\ngenerate more effective representations for entities, issues, and events. These\nrepresentations are contextualized by tweets, press releases, issues, news\narticles, and participating entities. Our model can process several documents\nat once and generate composed representations for multiple entities over\nseveral issues or events. Via qualitative and quantitative empirical analysis,\nwe show that these representations are meaningful and effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15786v1",
            "date": "2020-12-31 18:10:18+00:00",
            "title": "Conditional Generation of Temporally-ordered Event Sequences",
            "authors": [
                "Shih-Ting Lin",
                "Nathanael Chambers",
                "Greg Durrett"
            ],
            "abstract": "Models encapsulating narrative schema knowledge have proven to be useful for\na range of event-related tasks, but these models typically do not engage with\ntemporal relationships between events. We present a a BART-based conditional\ngeneration model capable of capturing event cooccurrence as well as temporality\nof event sequences. This single model can address both temporal ordering,\nsorting a given sequence of events into the order they occurred, and event\ninfilling, predicting new events which fit into a temporally-ordered sequence\nof existing ones. Our model is trained as a denoising autoencoder: we take\ntemporally-ordered event sequences, shuffle them, delete some events, and then\nattempting to recover the original event sequence. In this fashion, the model\nlearns to make inferences given incomplete knowledge about the events in an\nunderlying scenario. On the temporal ordering task, we show that our model is\nable to unscramble event sequences from existing datasets without access to\nexplicitly labeled temporal training data, outperforming both a BERT-based\npairwise model and a BERT-based pointer network. On event infilling, human\nevaluation shows that our model is able to generate events that fit better\ntemporally into the input events when compared to GPT-2 story completion\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15793v1",
            "date": "2020-12-31 18:17:57+00:00",
            "title": "Promoting Graph Awareness in Linearized Graph-to-Text Generation",
            "authors": [
                "Alexander Hoyle",
                "Ana Marasovi\u0107",
                "Noah Smith"
            ],
            "abstract": "Generating text from structured inputs, such as meaning representations or\nRDF triples, has often involved the use of specialized graph-encoding neural\nnetworks. However, recent applications of pretrained transformers to\nlinearizations of graph inputs have yielded state-of-the-art generation results\non graph-to-text tasks. Here, we explore the ability of these linearized models\nto encode local graph structures, in particular their invariance to the graph\nlinearization strategy and their ability to reconstruct corrupted inputs. Our\nfindings motivate solutions to enrich the quality of models' implicit graph\nencodings via scaffolding. Namely, we use graph-denoising objectives\nimplemented in a multi-task text-to-text framework. We find that these\ndenoising scaffolds lead to substantial improvements in downstream generation\nin low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15810v1",
            "date": "2020-12-31 18:34:29+00:00",
            "title": "UCCA's Foundational Layer: Annotation Guidelines v2.1",
            "authors": [
                "Omri Abend",
                "Nathan Schneider",
                "Dotan Dvir",
                "Jakob Prange",
                "Ari Rappoport"
            ],
            "abstract": "This is the annotation manual for Universal Conceptual Cognitive Annotation\n(UCCA; Abend and Rappoport, 2013), specifically the Foundational Layer. UCCA is\na graph-based semantic annotation scheme based on typological linguistic\nprinciples. It has been applied to several languages; for ease of exposition\nthese guidelines give examples mainly in English. New annotators may wish to\nstart with the tutorial on the UCCA framework (Abend et al., 2020). Further\nresources are available at the project homepage:\nhttps://universalconceptualcognitiveannotation.github.io",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15828v1",
            "date": "2020-12-31 18:51:26+00:00",
            "title": "MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers",
            "authors": [
                "Wenhui Wang",
                "Hangbo Bao",
                "Shaohan Huang",
                "Li Dong",
                "Furu Wei"
            ],
            "abstract": "We generalize deep self-attention distillation in MiniLM (Wang et al., 2020)\nby only using self-attention relation distillation for task-agnostic\ncompression of pretrained Transformers. In particular, we define multi-head\nself-attention relations as scaled dot-product between the pairs of query, key,\nand value vectors within each self-attention module. Then we employ the above\nrelational knowledge to train the student model. Besides its simplicity and\nunified principle, more favorably, there is no restriction in terms of the\nnumber of student's attention heads, while most previous work has to guarantee\nthe same head number between teacher and student. Moreover, the fine-grained\nself-attention relations tend to fully exploit the interaction knowledge\nlearned by Transformer. In addition, we thoroughly examine the layer selection\nstrategy for teacher models, rather than just relying on the last layer as in\nMiniLM. Experimental results demonstrate that our models distilled from\nbase-size and large-size teachers (BERT, and RoBERTa) outperform the state of\nthe art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15832v2",
            "date": "2020-12-31 18:52:59+00:00",
            "title": "Shortformer: Better Language Modeling using Shorter Inputs",
            "authors": [
                "Ofir Press",
                "Noah A. Smith",
                "Mike Lewis"
            ],
            "abstract": "Increasing the input length has been a driver of progress in language\nmodeling with transformers. We identify conditions where shorter inputs are not\nharmful, and achieve perplexity and efficiency improvements through two new\nmethods that decrease input length. First, we show that initially training a\nmodel on short subsequences before moving on to longer ones both reduces\noverall training time and, surprisingly, substantially improves perplexity.\nSecond, we show how to improve the efficiency of recurrence methods in\ntransformers, which let models condition on previously processed tokens when\ngenerating sequences that exceed the maximal length the transformer can handle\nat once. Existing methods require computationally expensive relative position\nembeddings; we introduce a simple alternative of adding absolute position\nembeddings to queries and keys instead of to word embeddings, which efficiently\nproduces superior results. We show that these recurrent models also benefit\nfrom short input lengths. Combining these techniques speeds up training by a\nfactor of 1.65, reduces memory usage, and substantially improves perplexity on\nWikiText-103, without adding any parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15833v1",
            "date": "2020-12-31 18:52:59+00:00",
            "title": "Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade",
            "authors": [
                "Jiatao Gu",
                "Xiang Kong"
            ],
            "abstract": "Fully non-autoregressive neural machine translation (NAT) is proposed to\nsimultaneously predict tokens with single forward of neural networks, which\nsignificantly reduces the inference latency at the expense of quality drop\ncompared to the Transformer baseline. In this work, we target on closing the\nperformance gap while maintaining the latency advantage. We first inspect the\nfundamental issues of fully NAT models, and adopt dependency reduction in the\nlearning space of output tokens as the basic guidance. Then, we revisit methods\nin four different aspects that have been proven effective for improving NAT\nmodels, and carefully combine these techniques with necessary modifications.\nOur extensive experiments on three translation benchmarks show that the\nproposed system achieves the new state-of-the-art results for fully NAT models,\nand obtains comparable performance with the autoregressive and iterative NAT\nsystems. For instance, one of the proposed models achieves 27.49 BLEU points on\nWMT14 En-De with approximately 16.5X speed up at inference time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15837v1",
            "date": "2020-12-31 18:55:30+00:00",
            "title": "Using Natural Language Relations between Answer Choices for Machine Comprehension",
            "authors": [
                "Rajkumar Pujari",
                "Dan Goldwasser"
            ],
            "abstract": "When evaluating an answer choice for Reading Comprehension task, other answer\nchoices available for the question and the answers of related questions about\nthe same paragraph often provide valuable information. In this paper, we\npropose a method to leverage the natural language relations between the answer\nchoices, such as entailment and contradiction, to improve the performance of\nmachine comprehension. We use a stand-alone question answering (QA) system to\nperform QA task and a Natural Language Inference (NLI) system to identify the\nrelations between the choice pairs. Then we perform inference using an Integer\nLinear Programming (ILP)-based relational framework to re-evaluate the\ndecisions made by the standalone QA system in light of the relations identified\nby the NLI system. We also propose a multitask learning model that learns both\nthe tasks jointly.",
            "journal ref": "Proceedings of the 2019 Conference of the North {A}merican Chapter\n  of the Association for Computational Linguistics: Human Language\n  Technologies, Volume 1 (Long and Short Papers)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15859v3",
            "date": "2020-12-31 18:59:44+00:00",
            "title": "Intrinsic Bias Metrics Do Not Correlate with Application Bias",
            "authors": [
                "Seraphina Goldfarb-Tarrant",
                "Rebecca Marchant",
                "Ricardo Mu\u00f1oz Sanchez",
                "Mugdha Pandya",
                "Adam Lopez"
            ],
            "abstract": "Natural Language Processing (NLP) systems learn harmful societal biases that\ncause them to amplify inequality as they are deployed in more and more\nsituations. To guide efforts at debiasing these systems, the NLP community\nrelies on a variety of metrics that quantify bias in models. Some of these\nmetrics are intrinsic, measuring bias in word embedding spaces, and some are\nextrinsic, measuring bias in downstream tasks that the word embeddings enable.\nDo these intrinsic and extrinsic metrics correlate with each other? We compare\nintrinsic and extrinsic metrics across hundreds of trained models covering\ndifferent tasks and experimental conditions. Our results show no reliable\ncorrelation between these metrics that holds in all scenarios across tasks and\nlanguages. We urge researchers working on debiasing to focus on extrinsic\nmeasures of bias, and to make using these measures more feasible via creation\nof new challenge sets and annotated test data. To aid this effort, we release\ncode, a new intrinsic metric, and an annotated test set focused on gender bias\nin hate speech.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00027v1",
            "date": "2020-12-31 19:00:10+00:00",
            "title": "The Pile: An 800GB Dataset of Diverse Text for Language Modeling",
            "authors": [
                "Leo Gao",
                "Stella Biderman",
                "Sid Black",
                "Laurence Golding",
                "Travis Hoppe",
                "Charles Foster",
                "Jason Phang",
                "Horace He",
                "Anish Thite",
                "Noa Nabeshima",
                "Shawn Presser",
                "Connor Leahy"
            ],
            "abstract": "Recent work has demonstrated that increased training dataset diversity\nimproves general cross-domain knowledge and downstream generalization\ncapability for large-scale language models. With this in mind, we present\n\\textit{the Pile}: an 825 GiB English text corpus targeted at training\nlarge-scale language models. The Pile is constructed from 22 diverse\nhigh-quality subsets -- both existing and newly constructed -- many of which\nderive from academic or professional sources. Our evaluation of the untuned\nperformance of GPT-2 and GPT-3 on the Pile shows that these models struggle on\nmany of its components, such as academic writing. Conversely, models trained on\nthe Pile improve significantly over both Raw CC and CC-100 on all components of\nthe Pile, while improving performance on downstream evaluations. Through an\nin-depth exploratory analysis, we document potentially concerning aspects of\nthe data for prospective users. We make publicly available the code used in its\nconstruction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00036v1",
            "date": "2020-12-31 19:06:18+00:00",
            "title": "KART: Privacy Leakage Framework of Language Models Pre-trained with Clinical Records",
            "authors": [
                "Yuta Nakamura",
                "Shouhei Hanaoka",
                "Yukihiro Nomura",
                "Naoto Hayashi",
                "Osamu Abe",
                "Shuntaro Yada",
                "Shoko Wakamiya",
                "Eiji Aramaki"
            ],
            "abstract": "Nowadays, mainstream natural language pro-cessing (NLP) is empowered by\npre-trained language models. In the biomedical domain, only models pre-trained\nwith anonymized data have been published. This policy is acceptable, but there\nare two questions: Can the privacy policy of language models be different from\nthat of data? What happens if private language models are accidentally made\npublic? We empirically evaluated the privacy risk of language models, using\nseveral BERT models pre-trained with MIMIC-III corpus in different data\nanonymity and corpus sizes. We simulated model inversion attacks to obtain the\nclinical information of target individuals, whose full names are already known\nto attackers. The BERT models were probably low-risk because the Top-100\naccuracy of each attack was far below expected by chance. Moreover, most\nprivacy leakage situations have several common primary factors; therefore, we\nformalized various privacy leakage scenarios under a universal novel framework\nnamed Knowledge, Anonymization, Resource, and Target (KART) framework. The KART\nframework helps parameterize complex privacy leakage scenarios and simplifies\nthe comprehensive evaluation. Since the concept of the KART framework is domain\nagnostic, it can contribute to the establishment of privacy guidelines of\nlanguage models beyond the biomedical domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00056v1",
            "date": "2020-12-31 20:18:29+00:00",
            "title": "Towards Modelling Coherence in Spoken Discourse",
            "authors": [
                "Rajaswa Patil",
                "Yaman Kumar Singla",
                "Rajiv Ratn Shah",
                "Mika Hama",
                "Roger Zimmermann"
            ],
            "abstract": "While there has been significant progress towards modelling coherence in\nwritten discourse, the work in modelling spoken discourse coherence has been\nquite limited. Unlike the coherence in text, coherence in spoken discourse is\nalso dependent on the prosodic and acoustic patterns in speech. In this paper,\nwe model coherence in spoken discourse with audio-based coherence models. We\nperform experiments with four coherence-related tasks with spoken discourses.\nIn our experiments, we evaluate machine-generated speech against the speech\ndelivered by expert human speakers. We also compare the spoken discourses\ngenerated by human language learners of varying language proficiency levels.\nOur results show that incorporating the audio modality along with the text\nbenefits the coherence models in performing downstream coherence related tasks\nwith spoken discourses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00078v1",
            "date": "2020-12-31 21:27:12+00:00",
            "title": "Controlled Analyses of Social Biases in Wikipedia Bios",
            "authors": [
                "Anjalie Field",
                "Chan Young Park",
                "Yulia Tsvetkov"
            ],
            "abstract": "Social biases on Wikipedia, a widely-read global platform, could greatly\ninfluence public opinion. While prior research has examined man/woman gender\nbias in biography articles, possible influences of confounding variables limit\nconclusions. In this work, we present a methodology for reducing the effects of\nconfounding variables in analyses of Wikipedia biography pages. Given a target\ncorpus for analysis (e.g. biography pages about women), we present a method for\nconstructing a comparison corpus that matches the target corpus in as many\nattributes as possible, except the target attribute (e.g. the gender of the\nsubject). We evaluate our methodology by developing metrics to measure how well\nthe comparison corpus aligns with the target corpus. We then examine how\narticles about gender and racial minorities (cisgender women, non-binary\npeople, transgender women, and transgender men; African American, Asian\nAmerican, and Hispanic/Latinx American people) differ from other articles,\nincluding analyses driven by social theories like intersectionality. In\naddition to identifying suspect social biases, our results show that failing to\ncontrol for confounding variables can result in different conclusions and mask\nbiases. Our contributions include methodology that facilitates further analyses\nof bias in Wikipedia articles, findings that can aid Wikipedia editors in\nreducing biases, and framework and evaluation metrics to guide future work in\nthis area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.01785v2",
            "date": "2020-12-27 06:32:55+00:00",
            "title": "ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic",
            "authors": [
                "Muhammad Abdul-Mageed",
                "AbdelRahim Elmadany",
                "El Moatez Billah Nagoudi"
            ],
            "abstract": "Pre-trained language models (LMs) are currently integral to many natural\nlanguage processing systems. Although multilingual LMs were also introduced to\nserve many languages, these have limitations such as being costly at inference\ntime and the size and diversity of non-English data involved in their\npre-training. We remedy these issues for a collection of diverse Arabic\nvarieties by introducing two powerful deep bidirectional transformer-based\nmodels, ARBERT and MARBERT. To evaluate our models, we also introduce ARLUE, a\nnew benchmark for multi-dialectal Arabic language understanding evaluation.\nARLUE is built using $42$ datasets targeting six different task clusters,\nallowing us to offer a series of standardized experiments under rich\nconditions. When fine-tuned on ARLUE, our models collectively achieve new\nstate-of-the-art results across the majority of tasks (37 out of 48\nclassification tasks, on the 42 datasets). Our best model acquires the highest\nARLUE score (77.40) across all six task clusters, outperforming all other\nmodels including XLM-R Large (~ 3.4 x larger size). Our models are publicly\navailable at https://github.com/UBC-NLP/marbert and ARLUE will be released\nthrough the same repository.",
            "journal ref": "ACL-IJCNLP 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.03028v1",
            "date": "2020-12-15 10:42:14+00:00",
            "title": "MeisterMorxrc at SemEval-2020 Task 9: Fine-Tune Bert and Multitask Learning for Sentiment Analysis of Code-Mixed Tweets",
            "authors": [
                "Qi Wu",
                "Peng Wang",
                "Chenghao Huang"
            ],
            "abstract": "Natural language processing (NLP) has been applied to various fields\nincluding text classification and sentiment analysis. In the shared task of\nsentiment analysis of code-mixed tweets, which is a part of the SemEval-2020\ncompetition~\\cite{patwa2020sentimix}, we preprocess datasets by replacing emoji\nand deleting uncommon characters and so on, and then fine-tune the\nBidirectional Encoder Representation from Transformers(BERT) to perform the\nbest. After exhausting top3 submissions, Our team MeisterMorxrc achieves an\naveraged F1 score of 0.730 in this task, and and our codalab username is\nMeisterMorxrc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.03216v2",
            "date": "2020-12-19 11:19:11+00:00",
            "title": "Breaking Writer's Block: Low-cost Fine-tuning of Natural Language Generation Models",
            "authors": [
                "Alexandre Duval",
                "Thomas Lamson",
                "Gael de Leseleuc de Kerouara",
                "Matthias Gall\u00e9"
            ],
            "abstract": "It is standard procedure these days to solve Information Extraction task by\nfine-tuning large pre-trained language models. This is not the case for\ngeneration task, which relies on a variety of techniques for controlled\nlanguage generation. In this paper, we describe a system that fine-tunes a\nnatural language generation model for the problem of solving Writer's Block.\nThe fine-tuning changes the conditioning to also include the right context in\naddition to the left context, as well as an optional list of entities, the\nsize, the genre and a summary of the paragraph that the human author wishes to\ngenerate. Our proposed fine-tuning obtains excellent results, even with a small\nnumber of epochs and a total cost of USD 150. The system can be accessed as a\nweb-service, and all the code is released. A video showcasing the interface and\nthe model is also available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00137v2",
            "date": "2020-01-01 04:49:23+00:00",
            "title": "Stacked DeBERT: All Attention in Incomplete Data for Text Classification",
            "authors": [
                "Gwenaelle Cunha Sergio",
                "Minho Lee"
            ],
            "abstract": "In this paper, we propose Stacked DeBERT, short for Stacked Denoising\nBidirectional Encoder Representations from Transformers. This novel model\nimproves robustness in incomplete data, when compared to existing systems, by\ndesigning a novel encoding scheme in BERT, a powerful language representation\nmodel solely based on attention mechanisms. Incomplete data in natural language\nprocessing refer to text with missing or incorrect words, and its presence can\nhinder the performance of current models that were not implemented to withstand\nsuch noises, but must still perform well even under duress. This is due to the\nfact that current approaches are built for and trained with clean and complete\ndata, and thus are not able to extract features that can adequately represent\nincomplete data. Our proposed approach consists of obtaining intermediate input\nrepresentations by applying an embedding layer to the input tokens followed by\nvanilla transformers. These intermediate features are given as input to novel\ndenoising transformers which are responsible for obtaining richer input\nrepresentations. The proposed approach takes advantage of stacks of multilayer\nperceptrons for the reconstruction of missing words' embeddings by extracting\nmore abstract and meaningful hidden feature vectors, and bidirectional\ntransformers for improved embedding representation. We consider two datasets\nfor training and evaluation: the Chatbot Natural Language Understanding\nEvaluation Corpus and Kaggle's Twitter Sentiment Corpus. Our model shows\nimproved F1-scores and better robustness in informal/incorrect texts present in\ntweets and in texts with Speech-to-Text error in the sentiment and intent\nclassification tasks.",
            "journal ref": "Neural Networks 136 (2021) 87-96",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00572v2",
            "date": "2020-01-03 03:43:35+00:00",
            "title": "Read Beyond the Lines: Understanding the Implied Textual Meaning via a Skim and Intensive Reading Model",
            "authors": [
                "Guoxiu He",
                "Zhe Gao",
                "Zhuoren Jiang",
                "Yangyang Kang",
                "Changlong Sun",
                "Xiaozhong Liu",
                "Wei Lu"
            ],
            "abstract": "The nonliteral interpretation of a text is hard to be understood by machine\nmodels due to its high context-sensitivity and heavy usage of figurative\nlanguage. In this study, inspired by human reading comprehension, we propose a\nnovel, simple, and effective deep neural framework, called Skim and Intensive\nReading Model (SIRM), for figuring out implied textual meaning. The proposed\nSIRM consists of two main components, namely the skim reading component and\nintensive reading component. N-gram features are quickly extracted from the\nskim reading component, which is a combination of several convolutional neural\nnetworks, as skim (entire) information. An intensive reading component enables\na hierarchical investigation for both local (sentence) and global (paragraph)\nrepresentation, which encapsulates the current embedding and the contextual\ninformation with a dense connection. More specifically, the contextual\ninformation includes the near-neighbor information and the skim information\nmentioned above. Finally, besides the normal training loss function, we employ\nan adversarial loss function as a penalty over the skim reading component to\neliminate noisy information arisen from special figurative words in the\ntraining data. To verify the effectiveness, robustness, and efficiency of the\nproposed architecture, we conduct extensive comparative experiments on several\nsarcasm benchmarks and an industrial spam dataset with metaphors. Experimental\nresults indicate that (1) the proposed model, which benefits from context\nmodeling and consideration of figurative language, outperforms existing\nstate-of-the-art solutions, with comparable parameter scale and training speed;\n(2) the SIRM yields superior robustness in terms of parameter size sensitivity;\n(3) compared with ablation and addition variants of the SIRM, the final\nframework is efficient enough.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00575v1",
            "date": "2020-01-01 20:39:22+00:00",
            "title": "Smart Summarizer for Blind People",
            "authors": [
                "Mona teja K",
                "Mohan Sai. S",
                "H S S S Raviteja D",
                "Sai Kushagra P V"
            ],
            "abstract": "In today's world, time is a very important resource. In our busy lives, most\nof us hardly have time to read the complete news so what we have to do is just\ngo through the headlines and satisfy ourselves with that. As a result, we might\nmiss a part of the news or misinterpret the complete thing. The situation is\neven worse for the people who are visually impaired or have lost their ability\nto see. The inability of these people to read text has a huge impact on their\nlives. There are a number of methods for blind people to read the text. Braille\nscript, in particular, is one of the examples, but it is a highly inefficient\nmethod as it is really time taking and requires a lot of practice. So, we\npresent a method for visually impaired people based on the sense of sound which\nis obviously better and more accurate than the sense of touch. This paper deals\nwith an efficient method to summarize news into important keywords so as to\nsave the efforts to go through the complete text every single time. This paper\ndeals with many API's and modules like the tesseract, GTTS, and many algorithms\nthat have been discussed and implemented in detail such as Luhn's Algorithm,\nLatent Semantic Analysis Algorithm, Text Ranking Algorithm. And the other\nfunctionality that this paper deals with is converting the summarized text to\nspeech so that the system can aid even the blind people.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2001.00623v1",
            "date": "2020-01-02 21:01:02+00:00",
            "title": "Mining Disinformation and Fake News: Concepts, Methods, and Recent Advancements",
            "authors": [
                "Kai Shu",
                "Suhang Wang",
                "Dongwon Lee",
                "Huan Liu"
            ],
            "abstract": "In recent years, disinformation including fake news, has became a global\nphenomenon due to its explosive growth, particularly on social media. The wide\nspread of disinformation and fake news can cause detrimental societal effects.\nDespite the recent progress in detecting disinformation and fake news, it is\nstill non-trivial due to its complexity, diversity, multi-modality, and costs\nof fact-checking or annotation. The goal of this chapter is to pave the way for\nappreciating the challenges and advancements via: (1) introducing the types of\ninformation disorder on social media and examine their differences and\nconnections; (2) describing important and emerging tasks to combat\ndisinformation for characterization, detection and attribution; and (3)\ndiscussing a weak supervision approach to detect disinformation with limited\nlabeled data. We then provide an overview of the chapters in this book that\nrepresent the recent advancements in three related parts: (1) user engagements\nin the dissemination of information disorder; (2) techniques on detecting and\nmitigating disinformation; and (3) trending issues such as ethics, blockchain,\nclickbaits, etc. We hope this book to be a convenient entry point for\nresearchers, practitioners, and students to understand the problems and\nchallenges, learn state-of-the-art solutions for their specific needs, and\nquickly identify new research problems in their domains.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2001.00733v1",
            "date": "2020-01-03 05:56:13+00:00",
            "title": "\"Love is as Complex as Math\": Metaphor Generation System for Social Chatbot",
            "authors": [
                "Danning Zheng",
                "Ruihua Song",
                "Tianran Hu",
                "Hao Fu",
                "Jin Zhou"
            ],
            "abstract": "As the wide adoption of intelligent chatbot in human daily life, user demands\nfor such systems evolve from basic task-solving conversations to more casual\nand friend-like communication. To meet the user needs and build emotional bond\nwith users, it is essential for social chatbots to incorporate more human-like\nand advanced linguistic features. In this paper, we investigate the usage of a\ncommonly used rhetorical device by human -- metaphor for social chatbot. Our\nwork first designs a metaphor generation framework, which generates topic-aware\nand novel figurative sentences. By embedding the framework into a chatbot\nsystem, we then enables the chatbot to communicate with users using figurative\nlanguage. Human annotators validate the novelty and properness of the generated\nmetaphors. More importantly, we evaluate the effects of employing metaphors in\nhuman-chatbot conversations. Experiments indicate that our system effectively\narouses user interests in communicating with our chatbot, resulting in\nsignificantly longer human-chatbot conversations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00862v1",
            "date": "2020-01-03 15:28:52+00:00",
            "title": "Meaning updating of density matrices",
            "authors": [
                "Bob Coecke",
                "Konstantinos Meichanetzidis"
            ],
            "abstract": "The DisCoCat model of natural language meaning assigns meaning to a sentence\ngiven: (i) the meanings of its words, and, (ii) its grammatical structure. The\nrecently introduced DisCoCirc model extends this to text consisting of multiple\nsentences. While in DisCoCat all meanings are fixed, in DisCoCirc each sentence\nupdates meanings of words. In this paper we explore different update mechanisms\nfor DisCoCirc, in the case where meaning is encoded in density matrices---which\ncome with several advantages as compared to vectors.\n  Our starting point are two non-commutative update mechanisms, borrowing one\nfrom quantum foundations research, from Leifer and Spekkens. Unfortunately,\nneither of these satisfies any desirable algebraic properties, nor are internal\nto the meaning category. By passing to double density matrices we do get an\nelegant internal diagrammatic update mechanism.\n  We also show that (commutative) spiders can be cast as an instance of the\nLeifer-Spekkens update mechanism. This result is of interest to quantum\nfoundations, as it bridges the work in Categorical Quantum Mechanics (CQM) with\nthat on conditional quantum states. Our work also underpins implementation of\ntext-level natural language processing on quantum hardware (a.k.a. QNLP), for\nwhich exponential space-gain and quadratic speed-up have previously been\nidentified.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/2001.01243v1",
            "date": "2020-01-05 14:19:11+00:00",
            "title": "Automatic Business Process Structure Discovery using Ordered Neurons LSTM: A Preliminary Study",
            "authors": [
                "Xue Han",
                "Lianxue Hu",
                "Yabin Dang",
                "Shivali Agarwal",
                "Lijun Mei",
                "Shaochun Li",
                "Xin Zhou"
            ],
            "abstract": "Automatic process discovery from textual process documentations is highly\ndesirable to reduce time and cost of Business Process Management (BPM)\nimplementation in organizations. However, existing automatic process discovery\napproaches mainly focus on identifying activities out of the documentations.\nDeriving the structural relationships between activities, which is important in\nthe whole process discovery scope, is still a challenge. In fact, a business\nprocess has latent semantic hierarchical structure which defines different\nlevels of detail to reflect the complex business logic. Recent findings in\nneural machine learning area show that the meaningful linguistic structure can\nbe induced by joint language modeling and structure learning. Inspired by these\nfindings, we propose to retrieve the latent hierarchical structure present in\nthe textual business process documents by building a neural network that\nleverages a novel recurrent architecture, Ordered Neurons LSTM (ON-LSTM), with\nprocess-level language model objective. We tested the proposed approach on data\nset of Process Description Documents (PDD) from our practical Robotic Process\nAutomation (RPA) projects. Preliminary experiments showed promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01595v1",
            "date": "2020-01-02 15:23:11+00:00",
            "title": "Why Moli\u00e8re most likely did write his plays",
            "authors": [
                "Florian Cafiero",
                "Jean-Baptiste Camps"
            ],
            "abstract": "As for Shakespeare, a hard-fought debate has emerged about Moli\\`ere, a\nsupposedly uneducated actor who, according to some, could not have written the\nmasterpieces attributed to him. In the past decades, the century-old thesis\naccording to which Pierre Corneille would be their actual author has become\npopular, mostly because of new works in computational linguistics. These\nresults are reassessed here through state-of-the-art attribution methods. We\nstudy a corpus of comedies in verse by major authors of Moli\\`ere and\nCorneille's time. Analysis of lexicon, rhymes, word forms, affixes,\nmorphosyntactic sequences, and function words do not give any clue that another\nauthor among the major playwrights of the time would have written the plays\nsigned under the name Moli\\`ere.",
            "journal ref": "Science Advances, 27 Nov 2019: Vol. 5, no. 11, eaax5489",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01622v1",
            "date": "2020-01-06 15:11:59+00:00",
            "title": "Exploring Benefits of Transfer Learning in Neural Machine Translation",
            "authors": [
                "Tom Kocmi"
            ],
            "abstract": "Neural machine translation is known to require large numbers of parallel\ntraining sentences, which generally prevent it from excelling on low-resource\nlanguage pairs. This thesis explores the use of cross-lingual transfer learning\non neural networks as a way of solving the problem with the lack of resources.\nWe propose several transfer learning approaches to reuse a model pretrained on\na high-resource language pair. We pay particular attention to the simplicity of\nthe techniques. We study two scenarios: (a) when we reuse the high-resource\nmodel without any prior modifications to its training process and (b) when we\ncan prepare the first-stage high-resource model for transfer learning in\nadvance. For the former scenario, we present a proof-of-concept method by\nreusing a model trained by other researchers. In the latter scenario, we\npresent a method which reaches even larger improvements in translation\nperformance. Apart from proposed techniques, we focus on an in-depth analysis\nof transfer learning techniques and try to shed some light on transfer learning\nimprovements. We show how our techniques address specific problems of\nlow-resource languages and are suitable even in high-resource transfer\nlearning. We evaluate the potential drawbacks and behavior by studying transfer\nlearning in various situations, for example, under artificially damaged\ntraining corpora, or with fixed various model parts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01871v2",
            "date": "2020-01-07 03:10:42+00:00",
            "title": "Attention over Parameters for Dialogue Systems",
            "authors": [
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Chien-Sheng Wu",
                "Jamin Shin",
                "Pascale Fung"
            ],
            "abstract": "Dialogue systems require a great deal of different but complementary\nexpertise to assist, inform, and entertain humans. For example, different\ndomains (e.g., restaurant reservation, train ticket booking) of goal-oriented\ndialogue systems can be viewed as different skills, and so does ordinary\nchatting abilities of chit-chat dialogue systems. In this paper, we propose to\nlearn a dialogue system that independently parameterizes different dialogue\nskills, and learns to select and combine each of them through Attention over\nParameters (AoP). The experimental results show that this approach achieves\ncompetitive performance on a combined dataset of MultiWOZ, In-Car Assistant,\nand Persona-Chat. Finally, we demonstrate that each dialogue skill is\neffectively learned and can be combined with other skills to produce selective\nresponses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01941v1",
            "date": "2020-01-07 09:22:58+00:00",
            "title": "Paraphrase Generation with Latent Bag of Words",
            "authors": [
                "Yao Fu",
                "Yansong Feng",
                "John P. Cunningham"
            ],
            "abstract": "Paraphrase generation is a longstanding important problem in natural language\nprocessing.\n  In addition, recent progress in deep generative models has shown promising\nresults on discrete latent variables for text generation.\n  Inspired by variational autoencoders with discrete latent structures, in this\nwork, we propose a latent bag of words (BOW) model for paraphrase generation.\n  We ground the semantics of a discrete latent variable by the BOW from the\ntarget sentences.\n  We use this latent variable to build a fully differentiable content planning\nand surface realization model.\n  Specifically, we use source words to predict their neighbors and model the\ntarget BOW with a mixture of softmax.\n  We use Gumbel top-k reparameterization to perform differentiable subset\nsampling from the predicted BOW distribution.\n  We retrieve the sampled word embeddings and use them to augment the decoder\nand guide its generation search space.\n  Our latent BOW model not only enhances the decoder, but also exhibits clear\ninterpretability.\n  We show the model interpretability with regard to \\emph{(i)} unsupervised\nlearning of word neighbors \\emph{(ii)} the step-by-step generation procedure.\n  Extensive experiments demonstrate the transparent and effective generation\nprocess of this model.\\footnote{Our code can be found at\n\\url{https://github.com/FranxYao/dgm_latent_bow}}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02178v1",
            "date": "2020-01-07 17:05:16+00:00",
            "title": "Heaps' law and Heaps functions in tagged texts: Evidences of their linguistic relevance",
            "authors": [
                "Andr\u00e9s Chacoma",
                "Dami\u00e1n H. Zanette"
            ],
            "abstract": "We study the relationship between vocabulary size and text length in a corpus\nof $75$ literary works in English, authored by six writers, distinguishing\nbetween the contributions of three grammatical classes (or ``tags,'' namely,\n{\\it nouns}, {\\it verbs}, and {\\it others}), and analyze the progressive\nappearance of new words of each tag along each individual text. While the\npower-law relation prescribed by Heaps' law is satisfactorily fulfilled by\ntotal vocabulary sizes and text lengths, the appearance of new words in each\ntext is on the whole well described by the average of random shufflings of the\ntext, which does not obey a power law. Deviations from this average, however,\nare statistically significant and show a systematic trend across the corpus.\nSpecifically, they reveal that the appearance of new words along each text is\npredominantly retarded with respect to the average of random shufflings.\nMoreover, different tags are shown to add systematically distinct contributions\nto this tendency, with {\\it verbs} and {\\it others} being respectively more and\nless retarded than the mean trend, and {\\it nouns} following instead this\noverall mean. These statistical systematicities are likely to point to the\nexistence of linguistically relevant information stored in the different\nvariants of Heaps' law, a feature that is still in need of extensive\nassessment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02284v2",
            "date": "2020-01-07 21:47:37+00:00",
            "title": "Multipurpose Intelligent Process Automation via Conversational Assistant",
            "authors": [
                "Alena Moiseeva",
                "Dietrich Trautmann",
                "Michael Heimann",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Intelligent Process Automation (IPA) is an emerging technology with a primary\ngoal to assist the knowledge worker by taking care of repetitive, routine and\nlow-cognitive tasks. Conversational agents that can interact with users in a\nnatural language are potential application for IPA systems. Such intelligent\nagents can assist the user by answering specific questions and executing\nroutine tasks that are ordinarily performed in a natural language (i.e.,\ncustomer support). In this work, we tackle a challenge of implementing an IPA\nconversational assistant in a real-world industrial setting with a lack of\nstructured training data. Our proposed system brings two significant benefits:\nFirst, it reduces repetitive and time-consuming activities and, therefore,\nallows workers to focus on more intelligent processes. Second, by interacting\nwith users, it augments the resources with structured and to some extent\nlabeled training data. We showcase the usage of the latter by re-implementing\nseveral components of our system with Transfer Learning (TL) methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02462v1",
            "date": "2020-01-08 11:44:47+00:00",
            "title": "From Natural Language Instructions to Complex Processes: Issues in Chaining Trigger Action Rules",
            "authors": [
                "Nobuhiro Ito",
                "Yuya Suzuki",
                "Akiko Aizawa"
            ],
            "abstract": "Automation services for complex business processes usually require a high\nlevel of information technology literacy. There is a strong demand for a\nsmartly assisted process automation (IPA: intelligent process automation)\nservice that enables even general users to easily use advanced automation. A\nnatural language interface for such automation is expected as an elemental\ntechnology for the IPA realization. The workflow targeted by IPA is generally\ncomposed of a combination of multiple tasks. However, semantic parsing, one of\nthe natural language processing methods, for such complex workflows has not yet\nbeen fully studied. The reasons are that (1) the formal expression and grammar\nof the workflow required for semantic analysis have not been sufficiently\nexamined and (2) the dataset of the workflow formal expression with its\ncorresponding natural language description required for learning workflow\nsemantics did not exist. This paper defines a new grammar for complex workflows\nwith chaining machine-executable meaning representations for semantic parsing.\nThe representations are at a high abstraction level. Additionally, an approach\nto creating datasets is proposed based on this grammar.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2001.02660v2",
            "date": "2020-01-08 18:04:52+00:00",
            "title": "REST: A Thread Embedding Approach for Identifying and Classifying User-specified Information in Security Forums",
            "authors": [
                "Joobin Gharibshah",
                "Evangelos E. Papalexakis",
                "Michalis Faloutsos"
            ],
            "abstract": "How can we extract useful information from a security forum? We focus on\nidentifying threads of interest to a security professional: (a) alerts of\nworrisome events, such as attacks, (b) offering of malicious services and\nproducts, (c) hacking information to perform malicious acts, and (d) useful\nsecurity-related experiences. The analysis of security forums is in its infancy\ndespite several promising recent works. Novel approaches are needed to address\nthe challenges in this domain: (a) the difficulty in specifying the \"topics\" of\ninterest efficiently, and (b) the unstructured and informal nature of the text.\nWe propose, REST, a systematic methodology to: (a) identify threads of interest\nbased on a, possibly incomplete, bag of words, and (b) classify them into one\nof the four classes above. The key novelty of the work is a multi-step weighted\nembedding approach: we project words, threads and classes in appropriate\nembedding spaces and establish relevance and similarity there. We evaluate our\nmethod with real data from three security forums with a total of 164k posts and\n21K threads. First, REST robustness to initial keyword selection can extend the\nuser-provided keyword set and thus, it can recover from missing keywords.\nSecond, REST categorizes the threads into the classes of interest with superior\naccuracy compared to five other methods: REST exhibits an accuracy between\n63.3-76.9%. We see our approach as a first step for harnessing the wealth of\ninformation of online forums in a user-friendly way, since the user can loosely\nspecify her keywords of interest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.02731v1",
            "date": "2020-01-08 20:36:17+00:00",
            "title": "SirenLess: reveal the intention behind news",
            "authors": [
                "Xumeng Chen",
                "Leo Yu-Ho Lo",
                "Huamin Qu"
            ],
            "abstract": "News articles tend to be increasingly misleading nowadays, preventing readers\nfrom making subjective judgments towards certain events. While some machine\nlearning approaches have been proposed to detect misleading news, most of them\nare black boxes that provide limited help for humans in decision making. In\nthis paper, we present SirenLess, a visual analytical system for misleading\nnews detection by linguistic features. The system features article explorer, a\nnovel interactive tool that integrates news metadata and linguistic features to\nreveal semantic structures of news articles and facilitate textual analysis. We\nuse SirenLess to analyze 18 news articles from different sources and summarize\nsome helpful patterns for misleading news detection. A user study with\njournalism professionals and university students is conducted to confirm the\nusefulness and effectiveness of our system.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2001.03041v1",
            "date": "2020-01-09 15:07:32+00:00",
            "title": "Open Challenge for Correcting Errors of Speech Recognition Systems",
            "authors": [
                "Marek Kubis",
                "Zygmunt Vetulani",
                "Miko\u0142aj Wypych",
                "Tomasz Zi\u0119tkiewicz"
            ],
            "abstract": "The paper announces the new long-term challenge for improving the performance\nof automatic speech recognition systems. The goal of the challenge is to\ninvestigate methods of correcting the recognition results on the basis of\npreviously made errors by the speech processing system. The dataset prepared\nfor the task is described and evaluation criteria are presented.",
            "journal ref": "Vetulani, Zygmunt, Paroubek, Patrick (eds.): Proceedings of the\n  9th Language and Technology Conference, pp. 219-223, Wydawnictwo Nauka i\n  Innowacje, Pozna\\'n, Poland, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03294v1",
            "date": "2020-01-10 03:12:28+00:00",
            "title": "Learning to Multi-Task Learn for Better Neural Machine Translation",
            "authors": [
                "Poorya Zaremoodi",
                "Gholamreza Haffari"
            ],
            "abstract": "Scarcity of parallel sentence pairs is a major challenge for training high\nquality neural machine translation (NMT) models in bilingually low-resource\nscenarios, as NMT is data-hungry. Multi-task learning is an elegant approach to\ninject linguistic-related inductive biases into NMT, using auxiliary syntactic\nand semantic tasks, to improve generalisation. The challenge, however, is to\ndevise effective training schedules, prescribing when to make use of the\nauxiliary tasks during the training process to fill the knowledge gaps of the\nmain translation task, a setting referred to as biased-MTL. Current approaches\nfor the training schedule are based on hand-engineering heuristics, whose\neffectiveness vary in different MTL settings. We propose a novel framework for\nlearning the training schedule, ie learning to multi-task learn, for the MTL\nsetting of interest. We formulate the training schedule as a Markov decision\nprocess which paves the way to employ policy learning methods to learn the\nscheduling policy. We effectively and efficiently learn the training schedule\npolicy within the imitation learning framework using an oracle policy algorithm\nthat dynamically sets the importance weights of auxiliary tasks based on their\ncontributions to the generalisability of the main NMT task. Experiments on\nlow-resource NMT settings show the resulting automatically learned training\nschedulers are competitive with the best heuristics, and lead to up to +1.1\nBLEU score improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03303v1",
            "date": "2020-01-10 04:39:44+00:00",
            "title": "Linking Social Media Posts to News with Siamese Transformers",
            "authors": [
                "Jacob Danovitch"
            ],
            "abstract": "Many computational social science projects examine online discourse\nsurrounding a specific trending topic. These works often involve the\nacquisition of large-scale corpora relevant to the event in question to analyze\naspects of the response to the event. Keyword searches present a\nprecision-recall trade-off and crowd-sourced annotations, while effective, are\ncostly. This work aims to enable automatic and accurate ad-hoc retrieval of\ncomments discussing a trending topic from a large corpus, using only a handful\nof seed news articles.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2001.03521v1",
            "date": "2020-01-10 15:45:59+00:00",
            "title": "Towards Minimal Supervision BERT-based Grammar Error Correction",
            "authors": [
                "Yiyuan Li",
                "Antonios Anastasopoulos",
                "Alan W Black"
            ],
            "abstract": "Current grammatical error correction (GEC) models typically consider the task\nas sequence generation, which requires large amounts of annotated data and\nlimit the applications in data-limited settings. We try to incorporate\ncontextual information from pre-trained language model to leverage annotation\nand benefit multilingual scenarios. Results show strong potential of\nBidirectional Encoder Representations from Transformers (BERT) in grammatical\nerror correction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.03897v1",
            "date": "2020-01-12 09:40:11+00:00",
            "title": "Stochastic Natural Language Generation Using Dependency Information",
            "authors": [
                "Elham Seifossadat",
                "Hossein Sameti"
            ],
            "abstract": "This article presents a stochastic corpus-based model for generating natural\nlanguage text. Our model first encodes dependency relations from training data\nthrough a feature set, then concatenates these features to produce a new\ndependency tree for a given meaning representation, and finally generates a\nnatural language utterance from the produced dependency tree. We test our model\non nine domains from tabular, dialogue act and RDF format. Our model\noutperforms the corpus-based state-of-the-art methods trained on tabular\ndatasets and also achieves comparable results with neural network-based\napproaches trained on dialogue act, E2E and WebNLG datasets for BLEU and ERR\nevaluation metrics. Also, by reporting Human Evaluation results, we show that\nour model produces high-quality utterances in aspects of informativeness and\nnaturalness as well as quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04192v1",
            "date": "2020-01-13 12:47:49+00:00",
            "title": "A logic-based relational learning approach to relation extraction: The OntoILPER system",
            "authors": [
                "Rinaldo Lima",
                "Bernard Espinasse",
                "Fred Freitas"
            ],
            "abstract": "Relation Extraction (RE), the task of detecting and characterizing semantic\nrelations between entities in text, has gained much importance in the last two\ndecades, mainly in the biomedical domain. Many papers have been published on\nRelation Extraction using supervised machine learning techniques. Most of these\ntechniques rely on statistical methods, such as feature-based and\ntree-kernels-based methods. Such statistical learning techniques are usually\nbased on a propositional hypothesis space for representing examples, i.e., they\nemploy an attribute-value representation of features. This kind of\nrepresentation has some drawbacks, particularly in the extraction of complex\nrelations which demand more contextual information about the involving\ninstances, i.e., it is not able to effectively capture structural information\nfrom parse trees without loss of information. In this work, we present\nOntoILPER, a logic-based relational learning approach to Relation Extraction\nthat uses Inductive Logic Programming for generating extraction models in the\nform of symbolic extraction rules. OntoILPER takes profit of a rich relational\nrepresentation of examples, which can alleviate the aforementioned drawbacks.\nThe proposed relational approach seems to be more suitable for Relation\nExtraction than statistical ones for several reasons that we argue. Moreover,\nOntoILPER uses a domain ontology that guides the background knowledge\ngeneration process and is used for storing the extracted relation instances.\nThe induced extraction rules were evaluated on three protein-protein\ninteraction datasets from the biomedical domain. The performance of OntoILPER\nextraction models was compared with other state-of-the-art RE systems. The\nencouraging results seem to demonstrate the effectiveness of the proposed\nsolution.",
            "journal ref": "Engineering Applications of Artificial Intelligence, Elsevier,\n  2019, 78, pp.142-157",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2001.04200v1",
            "date": "2020-01-13 13:01:14+00:00",
            "title": "Mining customer product reviews for product development: A summarization process",
            "authors": [
                "Tianjun Hou",
                "Bernard Yannou",
                "Yann Leroy",
                "Emilie Poirson"
            ],
            "abstract": "This research set out to identify and structure from online reviews the words\nand expressions related to customers' likes and dislikes to guide product\ndevelopment. Previous methods were mainly focused on product features. However,\nreviewers express their preference not only on product features. In this paper,\nbased on an extensive literature review in design science, the authors propose\na summarization model containing multiples aspects of user preference, such as\nproduct affordances, emotions, usage conditions. Meanwhile, the linguistic\npatterns describing these aspects of preference are discovered and drafted as\nannotation guidelines. A case study demonstrates that with the proposed model\nand the annotation guidelines, human annotators can structure the online\nreviews with high inter-agreement. As high inter-agreement human annotation\nresults are essential for automatizing the online review summarization process\nwith the natural language processing, this study provides materials for the\nfuture study of automatization.",
            "journal ref": "Expert Systems with Applications, Elsevier, 2019, 132, pp.141-150",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04246v2",
            "date": "2020-01-13 14:03:26+00:00",
            "title": "AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search",
            "authors": [
                "Daoyuan Chen",
                "Yaliang Li",
                "Minghui Qiu",
                "Zhen Wang",
                "Bofang Li",
                "Bolin Ding",
                "Hongbo Deng",
                "Jun Huang",
                "Wei Lin",
                "Jingren Zhou"
            ],
            "abstract": "Large pre-trained language models such as BERT have shown their effectiveness\nin various natural language processing tasks. However, the huge parameter size\nmakes them difficult to be deployed in real-time applications that require\nquick inference with limited resources. Existing methods compress BERT into\nsmall models while such compression is task-independent, i.e., the same\ncompressed BERT for all different downstream tasks. Motivated by the necessity\nand benefits of task-oriented BERT compression, we propose a novel compression\nmethod, AdaBERT, that leverages differentiable Neural Architecture Search to\nautomatically compress BERT into task-adaptive small models for specific tasks.\nWe incorporate a task-oriented knowledge distillation loss to provide search\nhints and an efficiency-aware loss as search constraints, which enables a good\ntrade-off between efficiency and effectiveness for task-adaptive BERT\ncompression. We evaluate AdaBERT on several NLP tasks, and the results\ndemonstrate that those task-adaptive compressed models are 12.7x to 29.3x\nfaster than BERT in inference time and 11.5x to 17.0x smaller in terms of\nparameter size, while comparable performance is maintained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04586v1",
            "date": "2020-01-14 02:05:14+00:00",
            "title": "Bi-Decoder Augmented Network for Neural Machine Translation",
            "authors": [
                "Boyuan Pan",
                "Yazheng Yang",
                "Zhou Zhao",
                "Yueting Zhuang",
                "Deng Cai"
            ],
            "abstract": "Neural Machine Translation (NMT) has become a popular technology in recent\nyears, and the encoder-decoder framework is the mainstream among all the\nmethods. It's obvious that the quality of the semantic representations from\nencoding is very crucial and can significantly affect the performance of the\nmodel. However, existing unidirectional source-to-target architectures may\nhardly produce a language-independent representation of the text because they\nrely heavily on the specific relations of the given language pairs. To\nalleviate this problem, in this paper, we propose a novel Bi-Decoder Augmented\nNetwork (BiDAN) for the neural machine translation task. Besides the original\ndecoder which generates the target language sequence, we add an auxiliary\ndecoder to generate back the source language sequence at the training time.\nSince each decoder transforms the representations of the input text into its\ncorresponding language, jointly training with two target ends can make the\nshared encoder has the potential to produce a language-independent semantic\nspace. We conduct extensive experiments on several NMT benchmark datasets and\nthe results demonstrate the effectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.04693v1",
            "date": "2020-01-14 10:12:50+00:00",
            "title": "Balancing the composition of word embeddings across heterogenous data sets",
            "authors": [
                "Stephanie Brandl",
                "David Lassner",
                "Maximilian Alber"
            ],
            "abstract": "Word embeddings capture semantic relationships based on contextual\ninformation and are the basis for a wide variety of natural language processing\napplications. Notably these relationships are solely learned from the data and\nsubsequently the data composition impacts the semantic of embeddings which\narguably can lead to biased word vectors. Given qualitatively different data\nsubsets, we aim to align the influence of single subsets on the resulting word\nvectors, while retaining their quality. In this regard we propose a criteria to\nmeasure the shift towards a single data subset and develop approaches to meet\nboth objectives. We find that a weighted average of the two subset embeddings\nbalances the influence of those subsets while word similarity performance\ndecreases. We further propose a promising optimization approach to balance\ninfluences and quality of word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05315v1",
            "date": "2020-01-11 14:50:57+00:00",
            "title": "A Continuous Space Neural Language Model for Bengali Language",
            "authors": [
                "Hemayet Ahmed Chowdhury",
                "Md. Azizul Haque Imon",
                "Anisur Rahman",
                "Aisha Khatun",
                "Md. Saiful Islam"
            ],
            "abstract": "Language models are generally employed to estimate the probability\ndistribution of various linguistic units, making them one of the fundamental\nparts of natural language processing. Applications of language models include a\nwide spectrum of tasks such as text summarization, translation and\nclassification. For a low resource language like Bengali, the research in this\narea so far can be considered to be narrow at the very least, with some\ntraditional count based models being proposed. This paper attempts to address\nthe issue and proposes a continuous-space neural language model, or more\nspecifically an ASGD weight dropped LSTM language model, along with techniques\nto efficiently train it for Bengali Language. The performance analysis with\nsome currently existing count based models illustrated in this paper also shows\nthat the proposed architecture outperforms its counterparts by achieving an\ninference perplexity as low as 51.2 on the held out data set for Bengali.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05467v1",
            "date": "2020-01-15 18:32:06+00:00",
            "title": "AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses",
            "authors": [
                "Tong Niu",
                "Mohit Bansal"
            ],
            "abstract": "Many sequence-to-sequence dialogue models tend to generate safe,\nuninformative responses. There have been various useful efforts on trying to\neliminate them. However, these approaches either improve decoding algorithms\nduring inference, rely on hand-crafted features, or employ complex models. In\nour work, we build dialogue models that are dynamically aware of what\nutterances or tokens are dull without any feature-engineering. Specifically, we\nstart with a simple yet effective automatic metric, AvgOut, which calculates\nthe average output probability distribution of all time steps on the decoder\nside during training. This metric directly estimates which tokens are more\nlikely to be generated, thus making it a faithful evaluation of the model\ndiversity (i.e., for diverse models, the token probabilities should be more\nevenly distributed rather than peaked at a few dull tokens). We then leverage\nthis novel metric to propose three models that promote diversity without losing\nrelevance. The first model, MinAvgOut, directly maximizes the diversity score\nthrough the output distributions of each batch; the second model, Label\nFine-Tuning (LFT), prepends to the source sequence a label continuously scaled\nby the diversity score to control the diversity level; the third model, RL,\nadopts Reinforcement Learning and treats the diversity score as a reward\nsignal. Moreover, we experiment with a hybrid model by combining the loss terms\nof MinAvgOut and RL. All four models outperform their base LSTM-RNN model on\nboth diversity and relevance by a large margin, and are comparable to or better\nthan competitive baselines (also verified via human evaluation). Moreover, our\napproaches are orthogonal to the base model, making them applicable as an\nadd-on to other emerging better dialogue models in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05493v2",
            "date": "2020-01-15 17:06:29+00:00",
            "title": "A Unified System for Aggression Identification in English Code-Mixed and Uni-Lingual Texts",
            "authors": [
                "Anant Khandelwal",
                "Niraj Kumar"
            ],
            "abstract": "Wide usage of social media platforms has increased the risk of aggression,\nwhich results in mental stress and affects the lives of people negatively like\npsychological agony, fighting behavior, and disrespect to others. Majority of\nsuch conversations contains code-mixed languages[28]. Additionally, the way\nused to express thought or communication style also changes from one social\nmedia plat-form to another platform (e.g., communication styles are different\nin twitter and Facebook). These all have increased the complexity of the\nproblem. To solve these problems, we have introduced a unified and robust\nmulti-modal deep learning architecture which works for English code-mixed\ndataset and uni-lingual English dataset both.The devised system, uses\npsycho-linguistic features and very ba-sic linguistic features. Our multi-modal\ndeep learning architecture contains, Deep Pyramid CNN, Pooled BiLSTM, and\nDisconnected RNN(with Glove and FastText embedding, both). Finally, the system\ntakes the decision based on model averaging. We evaluated our system on English\nCode-Mixed TRAC 2018 dataset and uni-lingual English dataset obtained from\nKaggle. Experimental results show that our proposed system outperforms all the\nprevious approaches on English code-mixed dataset and uni-lingual English\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.05614v3",
            "date": "2020-01-16 02:18:27+00:00",
            "title": "Delving Deeper into the Decoder for Video Captioning",
            "authors": [
                "Haoran Chen",
                "Jianmin Li",
                "Xiaolin Hu"
            ],
            "abstract": "Video captioning is an advanced multi-modal task which aims to describe a\nvideo clip using a natural language sentence. The encoder-decoder framework is\nthe most popular paradigm for this task in recent years. However, there exist\nsome problems in the decoder of a video captioning model. We make a thorough\ninvestigation into the decoder and adopt three techniques to improve the\nperformance of the model. First of all, a combination of variational dropout\nand layer normalization is embedded into a recurrent unit to alleviate the\nproblem of overfitting. Secondly, a new online method is proposed to evaluate\nthe performance of a model on a validation set so as to select the best\ncheckpoint for testing. Finally, a new training strategy called professional\nlearning is proposed which uses the strengths of a captioning model and\nbypasses its weaknesses. It is demonstrated in the experiments on Microsoft\nResearch Video Description Corpus (MSVD) and MSR-Video to Text (MSR-VTT)\ndatasets that our model has achieved the best results evaluated by BLEU, CIDEr,\nMETEOR and ROUGE-L metrics with significant gains of up to 18% on MSVD and 3.5%\non MSR-VTT compared with the previous state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2001.05727v1",
            "date": "2020-01-16 10:16:37+00:00",
            "title": "Document Network Projection in Pretrained Word Embedding Space",
            "authors": [
                "Antoine Gourru",
                "Adrien Guille",
                "Julien Velcin",
                "Julien Jacques"
            ],
            "abstract": "We present Regularized Linear Embedding (RLE), a novel method that projects a\ncollection of linked documents (e.g. citation network) into a pretrained word\nembedding space. In addition to the textual content, we leverage a matrix of\npairwise similarities providing complementary information (e.g., the network\nproximity of two documents in a citation graph). We first build a simple word\nvector average for each document, and we use the similarities to alter this\naverage representation. The document representations can help to solve many\ninformation retrieval tasks, such as recommendation, classification and\nclustering. We demonstrate that our approach outperforms or matches existing\ndocument network embedding methods on node classification and link prediction\ntasks. Furthermore, we show that it helps identifying relevant keywords to\ndescribe document classes.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2001.06286v2",
            "date": "2020-01-17 13:25:44+00:00",
            "title": "RobBERT: a Dutch RoBERTa-based Language Model",
            "authors": [
                "Pieter Delobelle",
                "Thomas Winters",
                "Bettina Berendt"
            ],
            "abstract": "Pre-trained language models have been dominating the field of natural\nlanguage processing in recent years, and have led to significant performance\ngains for various complex natural language tasks. One of the most prominent\npre-trained language models is BERT, which was released as an English as well\nas a multilingual version. Although multilingual BERT performs well on many\ntasks, recent studies show that BERT models trained on a single language\nsignificantly outperform the multilingual version. Training a Dutch BERT model\nthus has a lot of potential for a wide range of Dutch NLP tasks. While previous\napproaches have used earlier implementations of BERT to train a Dutch version\nof BERT, we used RoBERTa, a robustly optimized BERT approach, to train a Dutch\nlanguage model called RobBERT. We measured its performance on various tasks as\nwell as the importance of the fine-tuning dataset size. We also evaluated the\nimportance of language-specific tokenizers and the model's fairness. We found\nthat RobBERT improves state-of-the-art results for various tasks, and\nespecially significantly outperforms other models when dealing with smaller\ndatasets. These results indicate that it is a powerful pre-trained model for a\nlarge variety of Dutch language tasks. The pre-trained and fine-tuned models\nare publicly available to support further downstream Dutch NLP applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.06350v1",
            "date": "2020-01-14 22:37:21+00:00",
            "title": "A Hybrid Solution to Learn Turn-Taking in Multi-Party Service-based Chat Groups",
            "authors": [
                "Maira Gatti de Bayser",
                "Melina Alberio Guerra",
                "Paulo Cavalin",
                "Claudio Pinhanez"
            ],
            "abstract": "To predict the next most likely participant to interact in a multi-party\nconversation is a difficult problem. In a text-based chat group, the only\ninformation available is the sender, the content of the text and the dialogue\nhistory. In this paper we present our study on how these information can be\nused on the prediction task through a corpus and architecture that integrates\nturn-taking classifiers based on Maximum Likelihood Expectation (MLE),\nConvolutional Neural Networks (CNN) and Finite State Automata (FSA). The corpus\nis a synthetic adaptation of the Multi-Domain Wizard-of-Oz dataset (MultiWOZ)\nto a multiple travel service-based bots scenario with dialogue errors and was\ncreated to simulate user's interaction and evaluate the architecture. We\npresent experimental results which show that the CNN approach achieves better\nperformance than the baseline with an accuracy of 92.34%, but the integrated\nsolution with MLE, CNN and FSA achieves performance even better, with 95.65%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.06674v1",
            "date": "2020-01-18 14:47:10+00:00",
            "title": "Ranking Significant Discrepancies in Clinical Reports",
            "authors": [
                "Sean MacAvaney",
                "Arman Cohan",
                "Nazli Goharian",
                "Ross Filice"
            ],
            "abstract": "Medical errors are a major public health concern and a leading cause of death\nworldwide. Many healthcare centers and hospitals use reporting systems where\nmedical practitioners write a preliminary medical report and the report is\nlater reviewed, revised, and finalized by a more experienced physician. The\nrevisions range from stylistic to corrections of critical errors or\nmisinterpretations of the case. Due to the large quantity of reports written\ndaily, it is often difficult to manually and thoroughly review all the\nfinalized reports to find such errors and learn from them. To address this\nchallenge, we propose a novel ranking approach, consisting of textual and\nontological overlaps between the preliminary and final versions of reports. The\napproach learns to rank the reports based on the degree of discrepancy between\nthe versions. This allows medical practitioners to easily identify and learn\nfrom the reports in which their interpretation most substantially differed from\nthat of the attending physician (who finalized the report). This is a crucial\nstep towards uncovering potential errors and helping medical practitioners to\nlearn from such errors, thus improving patient-care in the long run. We\nevaluate our model on a dataset of radiology reports and show that our approach\noutperforms both previously-proposed approaches and more recent language models\nby 4.5% to 15.4%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2001.06944v1",
            "date": "2020-01-20 02:19:13+00:00",
            "title": "Nested-Wasserstein Self-Imitation Learning for Sequence Generation",
            "authors": [
                "Ruiyi Zhang",
                "Changyou Chen",
                "Zhe Gan",
                "Zheng Wen",
                "Wenlin Wang",
                "Lawrence Carin"
            ],
            "abstract": "Reinforcement learning (RL) has been widely studied for improving\nsequence-generation models. However, the conventional rewards used for RL\ntraining typically cannot capture sufficient semantic information and therefore\nrender model bias. Further, the sparse and delayed rewards make RL exploration\ninefficient. To alleviate these issues, we propose the concept of\nnested-Wasserstein distance for distributional semantic matching. To further\nexploit it, a novel nested-Wasserstein self-imitation learning framework is\ndeveloped, encouraging the model to exploit historical high-rewarded sequences\nfor enhanced exploration and better semantic matching. Our solution can be\nunderstood as approximately executing proximal policy optimization with\nWasserstein trust-regions. Experiments on a variety of unconditional and\nconditional sequence-generation tasks demonstrate the proposed approach\nconsistently leads to improved performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07098v2",
            "date": "2020-01-20 13:10:01+00:00",
            "title": "Audio Summarization with Audio Features and Probability Distribution Divergence",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Romain Deveaud",
                "Eric SanJuan",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "The automatic summarization of multimedia sources is an important task that\nfacilitates the understanding of an individual by condensing the source while\nmaintaining relevant information. In this paper we focus on audio summarization\nbased on audio features and the probability of distribution divergence. Our\nmethod, based on an extractive summarization approach, aims to select the most\nrelevant segments until a time threshold is reached. It takes into account the\nsegment's length, position and informativeness value. Informativeness of each\nsegment is obtained by mapping a set of audio features issued from its\nMel-frequency Cepstral Coefficients and their corresponding Jensen-Shannon\ndivergence score. Results over a multi-evaluator scheme shows that our approach\nprovides understandable and informative summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07263v3",
            "date": "2020-01-20 22:03:42+00:00",
            "title": "Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard",
            "authors": [
                "Zolt\u00e1n T\u00fcske",
                "George Saon",
                "Kartik Audhkhasi",
                "Brian Kingsbury"
            ],
            "abstract": "It is generally believed that direct sequence-to-sequence (seq2seq) speech\nrecognition models are competitive with hybrid models only when a large amount\nof data, at least a thousand hours, is available for training. In this paper,\nwe show that state-of-the-art recognition performance can be achieved on the\nSwitchboard-300 database using a single headed attention, LSTM based model.\nUsing a cross-utterance language model, our single-pass speaker independent\nsystem reaches 6.4% and 12.5% word error rate (WER) on the Switchboard and\nCallHome subsets of Hub5'00, without a pronunciation lexicon. While careful\nregularization and data augmentation are crucial in achieving this level of\nperformance, experiments on Switchboard-2000 show that nothing is more useful\nthan more data. Overall, the combination of various regularizations and a\nsimple but fairly large model results in a new state of the art, 4.7% and 7.8%\nWER on the Switchboard and CallHome sets, using SWB-2000 without any external\ndata resources.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2001.07526v1",
            "date": "2020-01-21 13:41:09+00:00",
            "title": "Domain-Aware Dialogue State Tracker for Multi-Domain Dialogue Systems",
            "authors": [
                "Vevake Balaraman",
                "Bernardo Magnini"
            ],
            "abstract": "In task-oriented dialogue systems the dialogue state tracker (DST) component\nis responsible for predicting the state of the dialogue based on the dialogue\nhistory. Current DST approaches rely on a predefined domain ontology, a fact\nthat limits their effective usage for large scale conversational agents, where\nthe DST constantly needs to be interfaced with ever-increasing services and\nAPIs. Focused towards overcoming this drawback, we propose a domain-aware\ndialogue state tracker, that is completely data-driven and it is modeled to\npredict for dynamic service schemas. The proposed model utilizes domain and\nslot information to extract both domain and slot specific representations for a\ngiven dialogue, and then uses such representations to predict the values of the\ncorresponding slot. Integrating this mechanism with a pretrained language model\n(i.e. BERT), our approach can effectively learn semantic relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07820v3",
            "date": "2020-01-22 00:05:45+00:00",
            "title": "Elephant in the Room: An Evaluation Framework for Assessing Adversarial Examples in NLP",
            "authors": [
                "Ying Xu",
                "Xu Zhong",
                "Antonio Jose Jimeno Yepes",
                "Jey Han Lau"
            ],
            "abstract": "An adversarial example is an input transformed by small perturbations that\nmachine learning models consistently misclassify. While there are a number of\nmethods proposed to generate adversarial examples for text data, it is not\ntrivial to assess the quality of these adversarial examples, as minor\nperturbations (such as changing a word in a sentence) can lead to a significant\nshift in their meaning, readability and classification label. In this paper, we\npropose an evaluation framework consisting of a set of automatic evaluation\nmetrics and human evaluation guidelines, to rigorously assess the quality of\nadversarial examples based on the aforementioned properties. We experiment with\nsix benchmark attacking methods and found that some methods generate\nadversarial examples with poor readability and content preservation. We also\nlearned that multiple factors could influence the attacking performance, such\nas the length of the text inputs and architecture of the classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.07885v2",
            "date": "2020-01-22 05:34:45+00:00",
            "title": "Normalization of Input-output Shared Embeddings in Text Generation Models",
            "authors": [
                "Jinyang Liu",
                "Yujia Zhai",
                "Zizhong Chen"
            ],
            "abstract": "Neural Network based models have been state-of-the-art models for various\nNatural Language Processing tasks, however, the input and output dimension\nproblem in the networks has still not been fully resolved, especially in text\ngeneration tasks (e.g. Machine Translation, Text Summarization), in which input\nand output both have huge sizes of vocabularies. Therefore, input-output\nembedding weight sharing has been introduced and adopted widely, which remains\nto be improved. Based on linear algebra and statistical theories, this paper\nlocates the shortcoming of existed input-output embedding weight sharing\nmethod, then raises methods for improving input-output weight shared embedding,\namong which methods of normalization of embedding weight matrices show best\nperformance. These methods are nearly computational cost-free, can get combined\nwith other embedding techniques, and show good effectiveness when applied on\nstate-of-the-art Neural Network models. For Transformer-big models, the\nnormalization techniques can get at best 0.6 BLEU improvement compared to the\noriginal version of model on WMT'16 En-De dataset, and similar BLEU\nimprovements on IWSLT 14' datasets. For DynamicConv models, 0.5 BLEU\nimprovement can be attained on WMT'16 En-De dataset, and 0.41 BLEU improvement\non IWSLT 14' De-En translation task is achieved.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08010v1",
            "date": "2020-01-22 13:45:34+00:00",
            "title": "ARAACOM: ARAbic Algerian Corpus for Opinion Mining",
            "authors": [
                "Zitouni Abdelhafid",
                "Hichem Rahab",
                "Abdelhafid Zitouni",
                "Mahieddine Djoudi"
            ],
            "abstract": "Nowadays, it is no more needed to do an enormous effort to distribute a lot\nof forms to thousands of people and collect them, then convert this from into\nelectronic format to track people opinion about some subjects. A lot of web\nsites can today reach a large spectrum with less effort. The majority of web\nsites suggest to their visitors to leave backups about their feeling of the\nsite or events. So, this makes for us a lot of data which need powerful mean to\nexploit. Opinion mining in the web becomes more and more an attracting task,\ndue the increasing need for individuals and societies to track the mood of\npeople against several subjects of daily life (sports, politics,\ntelevision,...). A lot of works in opinion mining was developed in western\nlanguages especially English, such works in Arabic language still very scarce.\nIn this paper, we propose our approach, for opinion mining in Arabic Algerian\nnews paper. CCS CONCEPTS $\\bullet$Information systems~Sentiment analysis\n$\\bullet$ Computing methodologies~Natural language processing",
            "journal ref": "ICCES '17: Proceedings of the International Conference on\n  Computing for Engineering and Sciences, Jul 2017, Istanbul, France. pp.35-39",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08053v1",
            "date": "2020-01-22 15:15:34+00:00",
            "title": "Contextualized Embeddings in Named-Entity Recognition: An Empirical Study on Generalization",
            "authors": [
                "Bruno Taill\u00e9",
                "Vincent Guigue",
                "Patrick Gallinari"
            ],
            "abstract": "Contextualized embeddings use unsupervised language model pretraining to\ncompute word representations depending on their context. This is intuitively\nuseful for generalization, especially in Named-Entity Recognition where it is\ncrucial to detect mentions never seen during training. However, standard\nEnglish benchmarks overestimate the importance of lexical over contextual\nfeatures because of an unrealistic lexical overlap between train and test\nmentions. In this paper, we perform an empirical analysis of the generalization\ncapabilities of state-of-the-art contextualized embeddings by separating\nmentions by novelty and with out-of-domain evaluation. We show that they are\nparticularly beneficial for unseen mentions detection, especially\nout-of-domain. For models trained on CoNLL03, language model contextualization\nleads to a +1.2% maximal relative micro-F1 score increase in-domain against\n+13% out-of-domain on the WNUT dataset",
            "journal ref": "ECIR 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08140v2",
            "date": "2020-01-22 16:42:06+00:00",
            "title": "A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation",
            "authors": [
                "Di Jin",
                "Zhijing Jin",
                "Joey Tianyi Zhou",
                "Peter Szolovits"
            ],
            "abstract": "State-of-the-art neural machine translation (NMT) systems are data-hungry and\nperform poorly on new domains with no supervised data. As data collection is\nexpensive and infeasible in many cases, domain adaptation methods are needed.\nIn this work, we propose a simple but effect approach to the semi-supervised\ndomain adaptation scenario of NMT, where the aim is to improve the performance\nof a translation model on the target domain consisting of only non-parallel\ndata with the help of supervised source domain data. This approach iteratively\ntrains a Transformer-based NMT model via three training objectives: language\nmodeling, back-translation, and supervised translation. We evaluate this method\non two adaptation settings: adaptation between specific domains and adaptation\nfrom a general domain to specific domains, and on two language pairs: German to\nEnglish and Romanian to English. With substantial performance improvement\nachieved---up to +19.31 BLEU over the strongest baseline, and +47.69 BLEU\nimprovement over the unadapted model---we present this method as a simple but\ntough-to-beat baseline in the field of semi-supervised domain adaptation for\nNMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08353v1",
            "date": "2020-01-23 02:47:39+00:00",
            "title": "Pre-training via Leveraging Assisting Languages and Data Selection for Neural Machine Translation",
            "authors": [
                "Haiyue Song",
                "Raj Dabre",
                "Zhuoyuan Mao",
                "Fei Cheng",
                "Sadao Kurohashi",
                "Eiichiro Sumita"
            ],
            "abstract": "Sequence-to-sequence (S2S) pre-training using large monolingual data is known\nto improve performance for various S2S NLP tasks in low-resource settings.\nHowever, large monolingual corpora might not always be available for the\nlanguages of interest (LOI). To this end, we propose to exploit monolingual\ncorpora of other languages to complement the scarcity of monolingual corpora\nfor the LOI. A case study of low-resource Japanese-English neural machine\ntranslation (NMT) reveals that leveraging large Chinese and French monolingual\ncorpora can help overcome the shortage of Japanese and English monolingual\ncorpora, respectively, for S2S pre-training. We further show how to utilize\nscript mapping (Chinese to Japanese) to increase the similarity between the two\nmonolingual corpora leading to further improvements in translation quality.\nAdditionally, we propose simple data-selection techniques to be used prior to\npre-training that significantly impact the quality of S2S pre-training. An\nempirical comparison of our proposed methods reveals that leveraging assisting\nlanguage monolingual corpora, data selection and script mapping are extremely\nimportant for NMT pre-training in low-resource scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08604v3",
            "date": "2020-01-23 15:34:56+00:00",
            "title": "Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation",
            "authors": [
                "Kang Min Yoo",
                "Hanbit Lee",
                "Franck Dernoncourt",
                "Trung Bui",
                "Walter Chang",
                "Sang-goo Lee"
            ],
            "abstract": "Recent works have shown that generative data augmentation, where synthetic\nsamples generated from deep generative models complement the training dataset,\nbenefit NLP tasks. In this work, we extend this approach to the task of dialog\nstate tracking for goal-oriented dialogs. Due to the inherent hierarchical\nstructure of goal-oriented dialogs over utterances and related annotations, the\ndeep generative model must be capable of capturing the coherence among\ndifferent hierarchies and types of dialog features. We propose the Variational\nHierarchical Dialog Autoencoder (VHDA) for modeling the complete aspects of\ngoal-oriented dialogs, including linguistic features and underlying structured\nannotations, namely speaker information, dialog acts, and goals. The proposed\narchitecture is designed to model each aspect of goal-oriented dialogs using\ninter-connected latent variables and learns to generate coherent goal-oriented\ndialogs from the latent spaces. To overcome training issues that arise from\ntraining complex variational models, we propose appropriate training\nstrategies. Experiments on various dialog datasets show that our model improves\nthe downstream dialog trackers' robustness via generative data augmentation. We\nalso discover additional benefits of our unified approach to modeling\ngoal-oriented dialogs: dialog response generation and user simulation, where\nour model outperforms previous strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08665v1",
            "date": "2020-01-23 17:04:00+00:00",
            "title": "Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model",
            "authors": [
                "Qing Wan",
                "Yoonsuck Choe"
            ],
            "abstract": "Consider a natural language sentence describing a specific step in a food\nrecipe. In such instructions, recognizing actions (such as press, bake, etc.)\nand the resulting changes in the state of the ingredients (shape molded,\ncustard cooked, temperature hot, etc.) is a challenging task. One way to cope\nwith this challenge is to explicitly model a simulator module that applies\nactions to entities and predicts the resulting outcome (Bosselut et al. 2018).\nHowever, such a model can be unnecessarily complex. In this paper, we propose a\nsimplified neural network model that separates action recognition and state\nchange prediction, while coupling the two through a novel loss function. This\nallows learning to indirectly influence each other. Our model, although\nsimpler, achieves higher state change prediction performance (67% average\naccuracy for ours vs. 55% in (Bosselut et al. 2018)) and takes fewer samples to\ntrain (10K ours vs. 65K+ by (Bosselut et al. 2018)).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08845v1",
            "date": "2020-01-23 23:08:24+00:00",
            "title": "Linguistic Fingerprints of Internet Censorship: the Case of SinaWeibo",
            "authors": [
                "Kei Yin Ng",
                "Anna Feldman",
                "Jing Peng"
            ],
            "abstract": "This paper studies how the linguistic components of blogposts collected from\nSina Weibo, a Chinese microblogging platform, might affect the blogposts'\nlikelihood of being censored. Our results go along with King et al. (2013)'s\nCollective Action Potential (CAP) theory, which states that a blogpost's\npotential of causing riot or assembly in real life is the key determinant of it\ngetting censored. Although there is not a definitive measure of this construct,\nthe linguistic features that we identify as discriminatory go along with the\nCAP theory. We build a classifier that significantly outperforms non-expert\nhumans in predicting whether a blogpost will be censored. The crowdsourcing\nresults suggest that while humans tend to see censored blogposts as more\ncontroversial and more likely to trigger action in real life than the\nuncensored counterparts, they in general cannot make a better guess than our\nmodel when it comes to `reading the mind' of the censors in deciding whether a\nblogpost should be censored. We do not claim that censorship is only determined\nby the linguistic features. There are many other factors contributing to\ncensorship decisions. The focus of the present paper is on the linguistic form\nof blogposts. Our work suggests that it is possible to use linguistic\nproperties of social media posts to automatically predict if they are going to\nbe censored.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.08868v2",
            "date": "2020-01-24 03:03:51+00:00",
            "title": "Exploration Based Language Learning for Text-Based Games",
            "authors": [
                "Andrea Madotto",
                "Mahdi Namazifar",
                "Joost Huizinga",
                "Piero Molino",
                "Adrien Ecoffet",
                "Huaixiu Zheng",
                "Alexandros Papangelis",
                "Dian Yu",
                "Chandra Khatri",
                "Gokhan Tur"
            ],
            "abstract": "This work presents an exploration and imitation-learning-based agent capable\nof state-of-the-art performance in playing text-based computer games.\nText-based computer games describe their world to the player through natural\nlanguage and expect the player to interact with the game using text. These\ngames are of interest as they can be seen as a testbed for language\nunderstanding, problem-solving, and language generation by artificial agents.\nMoreover, they provide a learning environment in which these skills can be\nacquired through interactions with an environment rather than using fixed\ncorpora. One aspect that makes these games particularly challenging for\nlearning agents is the combinatorially large action space. Existing methods for\nsolving text-based games are limited to games that are either very simple or\nhave an action space restricted to a predetermined set of admissible actions.\nIn this work, we propose to use the exploration approach of Go-Explore for\nsolving text-based games. More specifically, in an initial exploration phase,\nwe first extract trajectories with high rewards, after which we train a policy\nto solve the game by imitating these trajectories. Our experiments show that\nthis approach outperforms existing solutions in solving text-based games, and\nit is more sample efficient in terms of the number of interactions with the\nenvironment. Moreover, we show that the learned policy can generalize better\nthan existing solutions to unseen games without using any restriction on the\naction space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09215v2",
            "date": "2020-01-24 22:23:22+00:00",
            "title": "An Iterative Approach for Identifying Complaint Based Tweets in Social Media Platforms",
            "authors": [
                "Gyanesh Anand",
                "Akash Gautam",
                "Puneet Mathur",
                "Debanjan Mahata",
                "Rajiv Ratn Shah",
                "Ramit Sawhney"
            ],
            "abstract": "Twitter is a social media platform where users express opinions over a\nvariety of issues. Posts offering grievances or complaints can be utilized by\nprivate/ public organizations to improve their service and promptly gauge a\nlow-cost assessment. In this paper, we propose an iterative methodology which\naims to identify complaint based posts pertaining to the transport domain. We\nperform comprehensive evaluations along with releasing a novel dataset for the\nresearch purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.09309v2",
            "date": "2020-01-25 13:35:34+00:00",
            "title": "BERT's output layer recognizes all hidden layers? Some Intriguing Phenomena and a simple way to boost BERT",
            "authors": [
                "Wei-Tsung Kao",
                "Tsung-Han Wu",
                "Po-Han Chi",
                "Chun-Cheng Hsieh",
                "Hung-Yi Lee"
            ],
            "abstract": "Although Bidirectional Encoder Representations from Transformers (BERT) have\nachieved tremendous success in many natural language processing (NLP) tasks, it\nremains a black box. A variety of previous works have tried to lift the veil of\nBERT and understand each layer's functionality. In this paper, we found that\nsurprisingly the output layer of BERT can reconstruct the input sentence by\ndirectly taking each layer of BERT as input, even though the output layer has\nnever seen the input other than the final hidden layer. This fact remains true\nacross a wide variety of BERT-based models, even when some layers are\nduplicated. Based on this observation, we propose a quite simple method to\nboost the performance of BERT. By duplicating some layers in the BERT-based\nmodels to make it deeper (no extra training required in this step), they obtain\nbetter performance in the downstream tasks after fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10071v1",
            "date": "2020-01-27 20:39:32+00:00",
            "title": "SemClinBr -- a multi institutional and multi specialty semantically annotated corpus for Portuguese clinical NLP tasks",
            "authors": [
                "Lucas Emanuel Silva e Oliveira",
                "Ana Carolina Peters",
                "Adalniza Moura Pucca da Silva",
                "Caroline P. Gebeluca",
                "Yohan Bonescki Gumiel",
                "Lilian Mie Mukai Cintho",
                "Deborah Ribeiro Carvalho",
                "Sadid A. Hasan",
                "Claudia Maria Cabral Moro"
            ],
            "abstract": "The high volume of research focusing on extracting patient's information from\nelectronic health records (EHR) has led to an increase in the demand for\nannotated corpora, which are a very valuable resource for both the development\nand evaluation of natural language processing (NLP) algorithms. The absence of\na multi-purpose clinical corpus outside the scope of the English language,\nespecially in Brazilian Portuguese, is glaring and severely impacts scientific\nprogress in the biomedical NLP field. In this study, we developed a\nsemantically annotated corpus using clinical texts from multiple medical\nspecialties, document types, and institutions. We present the following: (1) a\nsurvey listing common aspects and lessons learned from previous research, (2) a\nfine-grained annotation schema which could be replicated and guide other\nannotation initiatives, (3) a web-based annotation tool focusing on an\nannotation suggestion feature, and (4) both intrinsic and extrinsic evaluation\nof the annotations. The result of this work is the SemClinBr, a corpus that has\n1,000 clinical notes, labeled with 65,117 entities and 11,263 relations, and\ncan support a variety of clinical NLP tasks and boost the EHR's secondary use\nfor the Portuguese language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.10161v1",
            "date": "2020-01-28 04:13:05+00:00",
            "title": "Bringing Stories Alive: Generating Interactive Fiction Worlds",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Wesley Cheung",
                "Dan Tu",
                "William Broniec",
                "Mark O. Riedl"
            ],
            "abstract": "World building forms the foundation of any task that requires narrative\nintelligence. In this work, we focus on procedurally generating interactive\nfiction worlds---text-based worlds that players \"see\" and \"talk to\" using\nnatural language. Generating these worlds requires referencing everyday and\nthematic commonsense priors in addition to being semantically consistent,\ninteresting, and coherent throughout. Using existing story plots as\ninspiration, we present a method that first extracts a partial knowledge graph\nencoding basic information regarding world structure such as locations and\nobjects. This knowledge graph is then automatically completed utilizing\nthematic knowledge and used to guide a neural language generation model that\nfleshes out the rest of the world. We perform human participant-based\nevaluations, testing our neural model's ability to extract and fill-in a\nknowledge graph and to generate language conditioned on it against rule-based\nand human-made baselines. Our code is available at\nhttps://github.com/rajammanabrolu/WorldGeneration.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2001.10590v1",
            "date": "2020-01-26 05:59:57+00:00",
            "title": "An Effective Automatic Image Annotation Model Via Attention Model and Data Equilibrium",
            "authors": [
                "Amir Vatani",
                "Milad Taleby Ahvanooey",
                "Mostafa Rahimi"
            ],
            "abstract": "Nowadays, a huge number of images are available. However, retrieving a\nrequired image for an ordinary user is a challenging task in computer vision\nsystems. During the past two decades, many types of research have been\nintroduced to improve the performance of the automatic annotation of images,\nwhich are traditionally focused on content-based image retrieval. Although,\nrecent research demonstrates that there is a semantic gap between content-based\nimage retrieval and image semantics understandable by humans. As a result,\nexisting research in this area has caused to bridge the semantic gap between\nlow-level image features and high-level semantics. The conventional method of\nbridging the semantic gap is through the automatic image annotation (AIA) that\nextracts semantic features using machine learning techniques. In this paper, we\npropose a novel AIA model based on the deep learning feature extraction method.\nThe proposed model has three phases, including a feature extractor, a tag\ngenerator, and an image annotator. First, the proposed model extracts\nautomatically the high and low-level features based on dual-tree continues\nwavelet transform (DT-CWT), singular value decomposition, distribution of color\nton, and the deep neural network. Moreover, the tag generator balances the\ndictionary of the annotated keywords by a new log-entropy auto-encoder (LEAE)\nand then describes these keywords by word embedding. Finally, the annotator\nworks based on the long-short-term memory (LSTM) network in order to obtain the\nimportance degree of specific features of the image. The experiments conducted\non two benchmark datasets confirm that the superiority of the proposed model\ncompared to the previous models in terms of performance criteria.",
            "journal ref": "Int. J. Adv. Comput. Sci. Appl, 9(3), pp.269-277 (2018)",
            "category": "cs.MM"
        },
        {
            "url": "http://arxiv.org/abs/2001.10929v2",
            "date": "2020-01-29 16:19:44+00:00",
            "title": "AMR Similarity Metrics from Principles",
            "authors": [
                "Juri Opitz",
                "Letitia Parcalabescu",
                "Anette Frank"
            ],
            "abstract": "Different metrics have been proposed to compare Abstract Meaning\nRepresentation (AMR) graphs. The canonical Smatch metric (Cai and Knight, 2013)\naligns the variables of two graphs and assesses triple matches. The recent\nSemBleu metric (Song and Gildea, 2019) is based on the machine-translation\nmetric Bleu (Papineni et al., 2002) and increases computational efficiency by\nablating the variable-alignment.\n  In this paper, i) we establish criteria that enable researchers to perform a\nprincipled assessment of metrics comparing meaning representations like AMR;\nii) we undertake a thorough analysis of Smatch and SemBleu where we show that\nthe latter exhibits some undesirable properties. For example, it does not\nconform to the identity of indiscernibles rule and introduces biases that are\nhard to control; iii) we propose a novel metric S$^2$match that is more\nbenevolent to only very slight meaning deviations and targets the fulfilment of\nall established criteria. We assess its suitability and show its advantages\nover Smatch and SemBleu.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11121v1",
            "date": "2020-01-29 22:44:05+00:00",
            "title": "ABSent: Cross-Lingual Sentence Representation Mapping with Bidirectional GANs",
            "authors": [
                "Zuohui Fu",
                "Yikun Xian",
                "Shijie Geng",
                "Yingqiang Ge",
                "Yuting Wang",
                "Xin Dong",
                "Guang Wang",
                "Gerard de Melo"
            ],
            "abstract": "A number of cross-lingual transfer learning approaches based on neural\nnetworks have been proposed for the case when large amounts of parallel text\nare at our disposal. However, in many real-world settings, the size of parallel\nannotated training data is restricted. Additionally, prior cross-lingual\nmapping research has mainly focused on the word level. This raises the question\nof whether such techniques can also be applied to effortlessly obtain\ncross-lingually aligned sentence representations. To this end, we propose an\nAdversarial Bi-directional Sentence Embedding Mapping (ABSent) framework, which\nlearns mappings of cross-lingual sentence representations from limited\nquantities of parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11224v1",
            "date": "2020-01-30 09:17:32+00:00",
            "title": "Introducing the diagrammatic mode",
            "authors": [
                "Tuomo Hiippala",
                "John A. Bateman"
            ],
            "abstract": "In this article, we propose a multimodal perspective to diagrammatic\nrepresentations by sketching a description of what may be tentatively termed\nthe diagrammatic mode. We consider diagrammatic representations in the light of\ncontemporary multimodality theory and explicate what enables diagrammatic\nrepresentations to integrate natural language, various forms of graphics,\ndiagrammatic elements such as arrows, lines and other expressive resources into\ncoherent organisations. We illustrate the proposed approach using two recent\ndiagram corpora and show how a multimodal approach supports the empirical\nanalysis of diagrammatic representations, especially in identifying\ndiagrammatic constituents and describing their interrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11268v1",
            "date": "2020-01-30 11:45:59+00:00",
            "title": "Data Mining in Clinical Trial Text: Transformers for Classification and Question Answering Tasks",
            "authors": [
                "Lena Schmidt",
                "Julie Weeds",
                "Julian P. T. Higgins"
            ],
            "abstract": "This research on data extraction methods applies recent advances in natural\nlanguage processing to evidence synthesis based on medical texts. Texts of\ninterest include abstracts of clinical trials in English and in multilingual\ncontexts. The main focus is on information characterized via the Population,\nIntervention, Comparator, and Outcome (PICO) framework, but data extraction is\nnot limited to these fields. Recent neural network architectures based on\ntransformers show capacities for transfer learning and increased performance on\ndownstream natural language processing tasks such as universal reading\ncomprehension, brought forward by this architecture's use of contextualized\nword embeddings and self-attention mechanisms. This paper contributes to\nsolving problems related to ambiguity in PICO sentence prediction tasks, as\nwell as highlighting how annotations for training named entity recognition\nsystems are used to train a high-performing, but nevertheless flexible\narchitecture for question answering in systematic review automation.\nAdditionally, it demonstrates how the problem of insufficient amounts of\ntraining annotations for PICO entity extraction is tackled by augmentation. All\nmodels in this paper were created with the aim to support systematic review\n(semi)automation. They achieve high F1 scores, and demonstrate the feasibility\nof applying transformer-based classification methods to support data mining in\nthe biomedical literature.",
            "journal ref": "HEALTHINF 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11314v3",
            "date": "2020-01-26 02:54:49+00:00",
            "title": "ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation",
            "authors": [
                "Dongling Xiao",
                "Han Zhang",
                "Yukun Li",
                "Yu Sun",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Current pre-training works in natural language generation pay little\nattention to the problem of exposure bias on downstream tasks. To address this\nissue, we propose an enhanced multi-flow sequence to sequence pre-training and\nfine-tuning framework named ERNIE-GEN, which bridges the discrepancy between\ntraining and inference with an infilling generation mechanism and a noise-aware\ngeneration method. To make generation closer to human writing patterns, this\nframework introduces a span-by-span generation flow that trains the model to\npredict semantically-complete spans consecutively rather than predicting word\nby word. Unlike existing pre-training methods, ERNIE-GEN incorporates\nmulti-granularity target sampling to construct pre-training data, which\nenhances the correlation between encoder and decoder. Experimental results\ndemonstrate that ERNIE-GEN achieves state-of-the-art results with a much\nsmaller amount of pre-training data and parameters on a range of language\ngeneration tasks, including abstractive summarization (Gigaword and\nCNN/DailyMail), question generation (SQuAD), dialogue generation (Persona-Chat)\nand generative question answering (CoQA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11382v1",
            "date": "2020-01-16 08:38:40+00:00",
            "title": "Intweetive Text Summarization",
            "authors": [
                "Jean Val\u00e8re Cossu",
                "Juan-Manuel Torres-Moreno",
                "Eric SanJuan",
                "Marc El-B\u00e8ze"
            ],
            "abstract": "The amount of user generated contents from various social medias allows\nanalyst to handle a wide view of conversations on several topics related to\ntheir business. Nevertheless keeping up-to-date with this amount of information\nis not humanly feasible. Automatic Summarization then provides an interesting\nmean to digest the dynamics and the mass volume of contents. In this paper, we\naddress the issue of tweets summarization which remains scarcely explored. We\npropose to automatically generated summaries of Micro-Blogs conversations\ndealing with public figures E-Reputation. These summaries are generated using\nkey-word queries or sample tweet and offer a focused view of the whole\nMicro-Blog network. Since state-of-the-art is lacking on this point we conduct\nand evaluate our experiments over the multilingual CLEF RepLab Topic-Detection\ndataset according to an experimental evaluation process.",
            "journal ref": "International Journal of Computational Linguistics and\n  Applications vol. 7, no. 1, 2016, pp. 67-83",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11384v1",
            "date": "2020-01-20 06:12:12+00:00",
            "title": "Unsupervised Sentiment Analysis for Code-mixed Data",
            "authors": [
                "Siddharth Yadav",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Code-mixing is the practice of alternating between two or more languages.\nMostly observed in multilingual societies, its occurrence is increasing and\ntherefore its importance. A major part of sentiment analysis research has been\nmonolingual, and most of them perform poorly on code-mixed text. In this work,\nwe introduce methods that use different kinds of multilingual and cross-lingual\nembeddings to efficiently transfer knowledge from monolingual text to\ncode-mixed text for sentiment analysis of code-mixed text. Our methods can\nhandle code-mixed text through a zero-shot learning. Our methods beat\nstate-of-the-art on English-Spanish code-mixed sentiment analysis by absolute\n3\\% F1-score. We are able to achieve 0.58 F1-score (without parallel corpus)\nand 0.62 F1-score (with parallel corpus) on the same benchmark in a zero-shot\nway as compared to 0.68 F1-score in supervised settings. Our code is publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11691v2",
            "date": "2020-01-31 07:50:25+00:00",
            "title": "Self-Adversarial Learning with Comparative Discrimination for Text Generation",
            "authors": [
                "Wangchunshu Zhou",
                "Tao Ge",
                "Ke Xu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "Conventional Generative Adversarial Networks (GANs) for text generation tend\nto have issues of reward sparsity and mode collapse that affect the quality and\ndiversity of generated samples. To address the issues, we propose a novel\nself-adversarial learning (SAL) paradigm for improving GANs' performance in\ntext generation. In contrast to standard GANs that use a binary classifier as\nits discriminator to predict whether a sample is real or generated, SAL employs\na comparative discriminator which is a pairwise classifier for comparing the\ntext quality between a pair of samples. During training, SAL rewards the\ngenerator when its currently generated sentence is found to be better than its\npreviously generated samples. This self-improvement reward mechanism allows the\nmodel to receive credits more easily and avoid collapsing towards the limited\nnumber of real samples, which not only helps alleviate the reward sparsity\nissue but also reduces the risk of mode collapse. Experiments on text\ngeneration benchmark datasets show that our proposed approach substantially\nimproves both the quality and the diversity, and yields more stable performance\ncompared to the previous GANs for text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.11694v3",
            "date": "2020-01-31 07:55:39+00:00",
            "title": "Pseudo-Bidirectional Decoding for Local Sequence Transduction",
            "authors": [
                "Wangchunshu Zhou",
                "Tao Ge",
                "Ke Xu"
            ],
            "abstract": "Local sequence transduction (LST) tasks are sequence transduction tasks where\nthere exists massive overlapping between the source and target sequences, such\nas Grammatical Error Correction (GEC) and spell or OCR correction. Previous\nwork generally tackles LST tasks with standard sequence-to-sequence (seq2seq)\nmodels that generate output tokens from left to right and suffer from the issue\nof unbalanced outputs. Motivated by the characteristic of LST tasks, in this\npaper, we propose a simple but versatile approach named Pseudo-Bidirectional\nDecoding (PBD) for LST tasks. PBD copies the corresponding representation of\nsource tokens to the decoder as pseudo future context to enable the decoder to\nattends to its bi-directional context. In addition, the bidirectional decoding\nscheme and the characteristic of LST tasks motivate us to share the encoder and\nthe decoder of seq2seq models. The proposed PBD approach provides right side\ncontext information for the decoder and models the inductive bias of LST tasks,\nreducing the number of parameters by half and providing good regularization\neffects. Experimental results on several benchmark datasets show that our\napproach consistently improves the performance of standard seq2seq models on\nLST tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00119v1",
            "date": "2020-02-01 01:22:44+00:00",
            "title": "Improving Domain-Adapted Sentiment Classification by Deep Adversarial Mutual Learning",
            "authors": [
                "Qianming Xue",
                "Wei Zhang",
                "Hongyuan Zha"
            ],
            "abstract": "Domain-adapted sentiment classification refers to training on a labeled\nsource domain to well infer document-level sentiment on an unlabeled target\ndomain. Most existing relevant models involve a feature extractor and a\nsentiment classifier, where the feature extractor works towards learning\ndomain-invariant features from both domains, and the sentiment classifier is\ntrained only on the source domain to guide the feature extractor. As such, they\nlack a mechanism to use sentiment polarity lying in the target domain. To\nimprove domain-adapted sentiment classification by learning sentiment from the\ntarget domain as well, we devise a novel deep adversarial mutual learning\napproach involving two groups of feature extractors, domain discriminators,\nsentiment classifiers, and label probers. The domain discriminators enable the\nfeature extractors to obtain domain-invariant features. Meanwhile, the label\nprober in each group explores document sentiment polarity of the target domain\nthrough the sentiment prediction generated by the classifier in the peer group,\nand guides the learning of the feature extractor in its own group. The proposed\napproach achieves the mutual learning of the two groups in an end-to-end\nmanner. Experiments on multiple public datasets indicate our method obtains the\nstate-of-the-art performance, validating the effectiveness of mutual learning\nthrough label probers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00206v1",
            "date": "2020-02-01 13:24:03+00:00",
            "title": "Novel Entity Discovery from Web Tables",
            "authors": [
                "Shuo Zhang",
                "Edgar Meij",
                "Krisztian Balog",
                "Ridho Reinanda"
            ],
            "abstract": "When working with any sort of knowledge base (KB) one has to make sure it is\nas complete and also as up-to-date as possible. Both tasks are non-trivial as\nthey require recall-oriented efforts to determine which entities and\nrelationships are missing from the KB. As such they require a significant\namount of labor. Tables on the Web, on the other hand, are abundant and have\nthe distinct potential to assist with these tasks. In particular, we can\nleverage the content in such tables to discover new entities, properties, and\nrelationships. Because web tables typically only contain raw textual content we\nfirst need to determine which cells refer to which known entities---a task we\ndub table-to-KB matching. This first task aims to infer table semantics by\nlinking table cells and heading columns to elements of a KB. Then second task\nbuilds upon these linked entities and properties to not only identify novel\nones in the same table but also to bootstrap their type and additional\nrelationships. We refer to this process as novel entity discovery and, to the\nbest of our knowledge, it is the first endeavor on mining the unlinked cells in\nweb tables. Our method identifies not only out-of-KB (``novel'') information\nbut also novel aliases for in-KB (``known'') entities. When evaluated using\nthree purpose-built test collections, we find that our proposed approaches\nobtain a marked improvement in terms of precision over our baselines whilst\nkeeping recall stable.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.00388v4",
            "date": "2020-02-02 13:17:31+00:00",
            "title": "A Survey on Knowledge Graphs: Representation, Acquisition and Applications",
            "authors": [
                "Shaoxiong Ji",
                "Shirui Pan",
                "Erik Cambria",
                "Pekka Marttinen",
                "Philip S. Yu"
            ],
            "abstract": "Human knowledge provides a formal understanding of the world. Knowledge\ngraphs that represent structural relations between entities have become an\nincreasingly popular research direction towards cognition and human-level\nintelligence. In this survey, we provide a comprehensive review of knowledge\ngraph covering overall research topics about 1) knowledge graph representation\nlearning, 2) knowledge acquisition and completion, 3) temporal knowledge graph,\nand 4) knowledge-aware applications, and summarize recent breakthroughs and\nperspective directions to facilitate future research. We propose a full-view\ncategorization and new taxonomies on these topics. Knowledge graph embedding is\norganized from four aspects of representation space, scoring function, encoding\nmodels, and auxiliary information. For knowledge acquisition, especially\nknowledge graph completion, embedding methods, path inference, and logical rule\nreasoning, are reviewed. We further explore several emerging topics, including\nmeta relational learning, commonsense reasoning, and temporal knowledge graphs.\nTo facilitate future research on knowledge graphs, we also provide a curated\ncollection of datasets and open-source libraries on different tasks. In the\nend, we have a thorough outlook on several promising research directions.",
            "journal ref": "IEEE Transactions on Neural Networks and Learning Systems, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00527v2",
            "date": "2020-02-03 01:23:41+00:00",
            "title": "Phylogenetic signal in phonotactics",
            "authors": [
                "Jayden L. Macklin-Cordes",
                "Claire Bowern",
                "Erich R. Round"
            ],
            "abstract": "Phylogenetic methods have broad potential in linguistics beyond tree\ninference. Here, we show how a phylogenetic approach opens the possibility of\ngaining historical insights from entirely new kinds of linguistic data--in this\ninstance, statistical phonotactics. We extract phonotactic data from 111\nPama-Nyungan vocabularies and apply tests for phylogenetic signal, quantifying\nthe degree to which the data reflect phylogenetic history. We test three\ndatasets: (1) binary variables recording the presence or absence of biphones\n(two-segment sequences) in a lexicon (2) frequencies of transitions between\nsegments, and (3) frequencies of transitions between natural sound classes.\nAustralian languages have been characterized as having a high degree of\nphonotactic homogeneity. Nevertheless, we detect phylogenetic signal in all\ndatasets. Phylogenetic signal is greater in finer-grained frequency data than\nin binary data, and greatest in natural-class-based data. These results\ndemonstrate the viability of employing a new source of readily extractable data\nin historical and comparative linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00571v1",
            "date": "2020-02-03 05:59:52+00:00",
            "title": "IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems",
            "authors": [
                "Liu Yang",
                "Minghui Qiu",
                "Chen Qu",
                "Cen Chen",
                "Jiafeng Guo",
                "Yongfeng Zhang",
                "W. Bruce Croft",
                "Haiqing Chen"
            ],
            "abstract": "Personal assistant systems, such as Apple Siri, Google Assistant, Amazon\nAlexa, and Microsoft Cortana, are becoming ever more widely used. Understanding\nuser intent such as clarification questions, potential answers and user\nfeedback in information-seeking conversations is critical for retrieving good\nresponses. In this paper, we analyze user intent patterns in\ninformation-seeking conversations and propose an intent-aware neural response\nranking model \"IART\", which refers to \"Intent-Aware Ranking with Transformers\".\nIART is built on top of the integration of user intent modeling and language\nrepresentation learning with the Transformer architecture, which relies\nentirely on a self-attention mechanism instead of recurrent nets. It\nincorporates intent-aware utterance attention to derive an importance weighting\nscheme of utterances in conversation context with the aim of better\nconversation history understanding. We conduct extensive experiments with three\ninformation-seeking conversation data sets including both standard benchmarks\nand commercial data. Our proposed model outperforms all baseline methods with\nrespect to a variety of metrics. We also perform case studies and analysis of\nlearned user intent and its impact on response ranking in information-seeking\nconversations to provide interpretation of results.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.00583v1",
            "date": "2020-02-03 07:15:29+00:00",
            "title": "CoTK: An Open-Source Toolkit for Fast Development and Fair Evaluation of Text Generation",
            "authors": [
                "Fei Huang",
                "Dazhen Wan",
                "Zhihong Shao",
                "Pei Ke",
                "Jian Guan",
                "Yilin Niu",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "In text generation evaluation, many practical issues, such as inconsistent\nexperimental settings and metric implementations, are often ignored but lead to\nunfair evaluation and untenable conclusions. We present CoTK, an open-source\ntoolkit aiming to support fast development and fair evaluation of text\ngeneration. In model development, CoTK helps handle the cumbersome issues, such\nas data processing, metric implementation, and reproduction. It standardizes\nthe development steps and reduces human errors which may lead to inconsistent\nexperimental settings. In model evaluation, CoTK provides implementation for\nmany commonly used metrics and benchmark models across different experimental\nsettings. As a unique feature, CoTK can signify when and which metric cannot be\nfairly compared. We demonstrate that it is convenient to use CoTK for model\ndevelopment and evaluation, particularly across different experimental\nsettings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00652v2",
            "date": "2020-02-03 11:28:10+00:00",
            "title": "How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context",
            "authors": [
                "Qian Liu",
                "Bei Chen",
                "Jiaqi Guo",
                "Jian-Guang Lou",
                "Bin Zhou",
                "Dongmei Zhang"
            ],
            "abstract": "Recently semantic parsing in context has received considerable attention,\nwhich is challenging since there are complex contextual phenomena. Previous\nworks verified their proposed methods in limited scenarios, which motivates us\nto conduct an exploratory study on context modeling methods under real-world\nsemantic parsing in context. We present a grammar-based decoding semantic\nparser and adapt typical context modeling methods on top of it. We evaluate 13\ncontext modeling methods on two large complex cross-domain datasets, and our\nbest model achieves state-of-the-art performances on both datasets with\nsignificant improvements. Furthermore, we summarize the most frequent\ncontextual phenomena, with a fine-grained analysis on representative models,\nwhich may shed light on potential research directions. Our code is available at\nhttps://github.com/microsoft/ContextualSP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00725v1",
            "date": "2020-01-23 18:23:03+00:00",
            "title": "Traduction des Grammaires Cat\u00e9gorielles de Lambek dans les Grammaires Cat\u00e9gorielles Abstraites",
            "authors": [
                "Valentin D. Richard"
            ],
            "abstract": "Lambek Grammars (LG) are a computational modelling of natural language, based\non non-commutative compositional types. It has been widely studied, especially\nfor languages where the syntax plays a major role (like English). The goal of\nthis internship report is to demonstrate that every Lambek Grammar can be, not\nentirely but efficiently, expressed in Abstract Categorial Grammars (ACG). The\nlatter is a novel modelling based on higher-order signature homomorphisms\n(using $\\lambda$-calculus), aiming at uniting the currently used models. The\nmain idea is to transform the type rewriting system of LGs into that of\nContext-Free Grammars (CFG) by erasing introduction and elimination rules and\ngenerating enough axioms so that the cut rule suffices. This iterative approach\npreserves the derivations and enables us to stop the possible infinite\ngenerative process at any step. Although the underlying algorithm was not fully\nimplemented, this proof provides another argument in favour of the relevance of\nACGs in Natural Language Processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00734v1",
            "date": "2020-01-20 12:30:17+00:00",
            "title": "Analysis of the quotation corpus of the Russian Wiktionary",
            "authors": [
                "A. Smirnov",
                "T. Levashova",
                "A. Karpov",
                "I. Kipyatkova",
                "A. Ronzhin",
                "A. Krizhanovsky",
                "N. Krizhanovsky"
            ],
            "abstract": "The quantitative evaluation of quotations in the Russian Wiktionary was\nperformed using the developed Wiktionary parser. It was found that the number\nof quotations in the dictionary is growing fast (51.5 thousands in 2011, 62\nthousands in 2012). These quotations were extracted and saved in the relational\ndatabase of a machine-readable dictionary. For this database, tables related to\nthe quotations were designed. A histogram of distribution of quotations of\nliterary works written in different years was built. It was made an attempt to\nexplain the characteristics of the histogram by associating it with the years\nof the most popular and cited (in the Russian Wiktionary) writers of the\nnineteenth century. It was found that more than one-third of all the quotations\n(the example sentences) contained in the Russian Wiktionary are taken by the\neditors of a Wiktionary entry from the Russian National Corpus.",
            "journal ref": "Research in Computing Science, Vol. 56, pp. 101-112, 2012",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00748v2",
            "date": "2020-01-27 05:27:09+00:00",
            "title": "Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus",
            "authors": [
                "Bang Liu",
                "Haojie Wei",
                "Di Niu",
                "Haolan Chen",
                "Yancheng He"
            ],
            "abstract": "The ability to ask questions is important in both human and machine\nintelligence. Learning to ask questions helps knowledge acquisition, improves\nquestion-answering and machine reading comprehension tasks, and helps a chatbot\nto keep the conversation flowing with a human. Existing question generation\nmodels are ineffective at generating a large amount of high-quality\nquestion-answer pairs from unstructured text, since given an answer and an\ninput passage, question generation is inherently a one-to-many mapping. In this\npaper, we propose Answer-Clue-Style-aware Question Generation (ACS-QG), which\naims at automatically generating high-quality and diverse question-answer pairs\nfrom unlabeled text corpus at scale by imitating the way a human asks\nquestions. Our system consists of: i) an information extractor, which samples\nfrom the text multiple types of assistive information to guide question\ngeneration; ii) neural question generators, which generate diverse and\ncontrollable questions, leveraging the extracted assistive information; and\niii) a neural quality controller, which removes low-quality generated data\nbased on text entailment. We compare our question generation models with\nexisting approaches and resort to voluntary human evaluation to assess the\nquality of the generated question-answer pairs. The evaluation results suggest\nthat our system dramatically outperforms state-of-the-art neural question\ngeneration models in terms of the generation quality, while being scalable in\nthe meantime. With models trained on a relatively smaller amount of data, we\ncan generate 2.8 million quality-assured question-answer pairs from a million\nsentences found in Wikipedia.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.00760v1",
            "date": "2020-01-31 07:39:45+00:00",
            "title": "FastWordBug: A Fast Method To Generate Adversarial Text Against NLP Applications",
            "authors": [
                "Dou Goodman",
                "Lv Zhonghou",
                "Wang minghua"
            ],
            "abstract": "In this paper, we present a novel algorithm, FastWordBug, to efficiently\ngenerate small text perturbations in a black-box setting that forces a\nsentiment analysis or text classification mode to make an incorrect prediction.\nBy combining the part of speech attributes of words, we propose a scoring\nmethod that can quickly identify important words that affect text\nclassification. We evaluate FastWordBug on three real-world text datasets and\ntwo state-of-the-art machine learning models under black-box setting. The\nresults show that our method can significantly reduce the accuracy of the\nmodel, and at the same time, we can call the model as little as possible, with\nthe highest attack efficiency. We also attack two popular real-world cloud\nservices of NLP, and the results show that our method works as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01030v1",
            "date": "2020-02-03 22:13:07+00:00",
            "title": "Detecting Fake News with Capsule Neural Networks",
            "authors": [
                "Mohammad Hadi Goldani",
                "Saeedeh Momtazi",
                "Reza Safabakhsh"
            ],
            "abstract": "Fake news is dramatically increased in social media in recent years. This has\nprompted the need for effective fake news detection algorithms. Capsule neural\nnetworks have been successful in computer vision and are receiving attention\nfor use in Natural Language Processing (NLP). This paper aims to use capsule\nneural networks in the fake news detection task. We use different embedding\nmodels for news items of different lengths. Static word embedding is used for\nshort news items, whereas non-static word embeddings that allow incremental\nup-training and updating in the training phase are used for medium length or\nlarge news statements. Moreover, we apply different levels of n-grams for\nfeature extraction. Our proposed architectures are evaluated on two recent\nwell-known datasets in the field, namely ISOT and LIAR. The results show\nencouraging performance, outperforming the state-of-the-art methods by 7.8% on\nISOT and 3.1% on the validation set, and 1% on the test set of the LIAR\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01065v2",
            "date": "2020-02-04 00:28:38+00:00",
            "title": "Fake News Detection by means of Uncertainty Weighted Causal Graphs",
            "authors": [
                "Eduardo C. Garrido-Merch\u00e1n",
                "Cristina Puente",
                "Rafael Palacios"
            ],
            "abstract": "Society is experimenting changes in information consumption, as new\ninformation channels such as social networks let people share news that do not\nnecessarily be trust worthy. Sometimes, these sources of information produce\nfake news deliberately with doubtful purposes and the consumers of that\ninformation share it to other users thinking that the information is accurate.\nThis transmission of information represents an issue in our society, as can\ninfluence negatively the opinion of people about certain figures, groups or\nideas. Hence, it is desirable to design a system that is able to detect and\nclassify information as fake and categorize a source of information as trust\nworthy or not. Current systems experiment difficulties performing this task, as\nit is complicated to design an automatic procedure that can classify this\ninformation independent on the context. In this work, we propose a mechanism to\ndetect fake news through a classifier based on weighted causal graphs. These\ngraphs are specific hybrid models that are built through causal relations\nretrieved from texts and consider the uncertainty of causal relations. We take\nadvantage of this representation to use the probability distributions of this\ngraph and built a fake news classifier based on the entropy and KL divergence\nof learned and new information. We believe that the problem of fake news is\naccurately tackled by this model due to its hybrid nature between a symbolic\nand quantitative methodology. We describe the methodology of this classifier\nand add empirical evidence of the usefulness of our proposed approach in the\nform of synthetic experiments and a real experiment involving lung cancer.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2002.01207v1",
            "date": "2020-02-04 10:09:42+00:00",
            "title": "Arabic Diacritic Recovery Using a Feature-Rich biLSTM Model",
            "authors": [
                "Kareem Darwish",
                "Ahmed Abdelali",
                "Hamdy Mubarak",
                "Mohamed Eldesouki"
            ],
            "abstract": "Diacritics (short vowels) are typically omitted when writing Arabic text, and\nreaders have to reintroduce them to correctly pronounce words. There are two\ntypes of Arabic diacritics: the first are core-word diacritics (CW), which\nspecify the lexical selection, and the second are case endings (CE), which\ntypically appear at the end of the word stem and generally specify their\nsyntactic roles. Recovering CEs is relatively harder than recovering core-word\ndiacritics due to inter-word dependencies, which are often distant. In this\npaper, we use a feature-rich recurrent neural network model that uses a variety\nof linguistic and surface-level features to recover both core word diacritics\nand case endings. Our model surpasses all previous state-of-the-art systems\nwith a CW error rate (CWER) of 2.86\\% and a CE error rate (CEER) of 3.7% for\nModern Standard Arabic (MSA) and CWER of 2.2% and CEER of 2.5% for Classical\nArabic (CA). When combining diacritized word cores with case endings, the\nresultant word error rate is 6.0% and 4.3% for MSA and CA respectively. This\nhighlights the effectiveness of feature engineering for such deep neural\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01214v1",
            "date": "2020-02-04 10:30:49+00:00",
            "title": "On Stochastic Automata over Monoids",
            "authors": [
                "Karl-Heinz Zimmermann",
                "Merve Nur Cakir"
            ],
            "abstract": "Stochastic automata over monoids as input sets are studied. The\nwell-definedness of these automata requires an extension postulate that\nreplaces the inherent universal property of free monoids. As a generalization\nof Turakainen's result, it will be shown that the generalized automata over\nmonoids have the same acceptance power as their stochastic counterparts. The\nkey to homomorphisms is a commuting property between the monoid homomorphism of\ninput states and the monoid homomorphism of transition matrices. Closure\nproperties of the languages accepted by stochastic automata over monoids are\ninvestigated. matrices. Closure properties of the languages accepted by\nstochastic automata over monoids are investigated.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01412v1",
            "date": "2020-02-04 17:12:43+00:00",
            "title": "Iterative Data Programming for Expanding Text Classification Corpora",
            "authors": [
                "Neil Mallinar",
                "Abhishek Shah",
                "Tin Kam Ho",
                "Rajendra Ugrani",
                "Ayush Gupta"
            ],
            "abstract": "Real-world text classification tasks often require many labeled training\nexamples that are expensive to obtain. Recent advancements in machine teaching,\nspecifically the data programming paradigm, facilitate the creation of training\ndata sets quickly via a general framework for building weak models, also known\nas labeling functions, and denoising them through ensemble learning techniques.\nWe present a fast, simple data programming method for augmenting text data sets\nby generating neighborhood-based weak models with minimal supervision.\nFurthermore, our method employs an iterative procedure to identify sparsely\ndistributed examples from large volumes of unlabeled data. The iterative data\nprogramming techniques improve newer weak models as more labeled data is\nconfirmed with human-in-loop. We show empirical results on sentence\nclassification tasks, including those from a task of improving intent\nrecognition in conversational agents.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2002.01462v4",
            "date": "2020-02-04 18:40:38+00:00",
            "title": "Semantic Search of Memes on Twitter",
            "authors": [
                "Jesus Perez-Martin",
                "Benjamin Bustos",
                "Magdalena Saldana"
            ],
            "abstract": "Memes are becoming a useful source of data for analyzing behavior on social\nmedia. However, a problem to tackle is how to correctly identify a meme. As the\nnumber of memes published every day on social media is huge, there is a need\nfor automatic methods for classifying and searching in large meme datasets.\nThis paper proposes and compares several methods for automatically classifying\nimages as memes. Also, we propose a method that allows us to implement a system\nfor retrieving memes from a dataset using a textual query. We experimentally\nevaluate the methods using a large dataset of memes collected from Twitter\nusers in Chile, which was annotated by a group of experts. Though some of the\nevaluated methods are effective, there is still room for improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01510v2",
            "date": "2020-02-04 19:30:55+00:00",
            "title": "Generalizing meanings from partners to populations: Hierarchical inference supports convention formation on networks",
            "authors": [
                "Robert D. Hawkins",
                "Noah D. Goodman",
                "Adele E. Goldberg",
                "Thomas L. Griffiths"
            ],
            "abstract": "A key property of linguistic conventions is that they hold over an entire\ncommunity of speakers, allowing us to communicate efficiently even with people\nwe have never met before. At the same time, much of our language use is\npartner-specific: we know that words may be understood differently by different\npeople based on our shared history. This poses a challenge for accounts of\nconvention formation. Exactly how do agents make the inferential leap to\ncommunity-wide expectations while maintaining partner-specific knowledge? We\npropose a hierarchical Bayesian model to explain how speakers and listeners\nsolve this inductive problem. To evaluate our model's predictions, we conducted\nan experiment where participants played an extended natural-language\ncommunication game with different partners in a small community. We examine\nseveral measures of generalization and find key signatures of both\npartner-specificity and community convergence that distinguish our model from\nalternatives. These results suggest that partner-specificity is not only\ncompatible with the formation of community-wide conventions, but may facilitate\nit when coupled with a powerful inductive mechanism.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01535v1",
            "date": "2020-02-04 21:02:11+00:00",
            "title": "Lightweight Convolutional Representations for On-Device Natural Language Processing",
            "authors": [
                "Shrey Desai",
                "Geoffrey Goh",
                "Arun Babu",
                "Ahmed Aly"
            ],
            "abstract": "The increasing computational and memory complexities of deep neural networks\nhave made it difficult to deploy them on low-resource electronic devices (e.g.,\nmobile phones, tablets, wearables). Practitioners have developed numerous model\ncompression methods to address these concerns, but few have condensed input\nrepresentations themselves. In this work, we propose a fast, accurate, and\nlightweight convolutional representation that can be swapped into any neural\nmodel and compressed significantly (up to 32x) with a negligible reduction in\nperformance. In addition, we show gains over recurrent representations when\nconsidering resource-centric metrics (e.g., model file size, latency, memory\nusage) on a Samsung Galaxy S9.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01685v1",
            "date": "2020-02-05 08:43:02+00:00",
            "title": "Parsing as Pretraining",
            "authors": [
                "David Vilares",
                "Michalina Strzyz",
                "Anders S\u00f8gaard",
                "Carlos G\u00f3mez-Rodr\u00edguez"
            ],
            "abstract": "Recent analyses suggest that encoders pretrained for language modeling\ncapture certain morpho-syntactic structure. However, probing frameworks for\nword vectors still do not report results on standard setups such as constituent\nand dependency parsing. This paper addresses this problem and does full parsing\n(on English) relying only on pretraining architectures -- and no decoding. We\nfirst cast constituent and dependency parsing as sequence tagging. We then use\na single feed-forward layer to directly map word vectors to labels that encode\na linearized tree. This is used to: (i) see how far we can reach on syntax\nmodelling with just pretrained encoders, and (ii) shed some light about the\nsyntax-sensitivity of different word vectors (by freezing the weights of the\npretraining network during training). For evaluation, we use bracketing\nF1-score and LAS, and analyze in-depth differences across representations for\nspan lengths and dependency displacements. The overall results surpass existing\nsequence tagging parsers on the PTB (93.5%) and end-to-end EN-EWT UD (78.8%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01761v1",
            "date": "2020-02-05 12:44:01+00:00",
            "title": "Multi-Fusion Chinese WordNet (MCW) : Compound of Machine Learning and Manual Correction",
            "authors": [
                "Mingchen Li",
                "Zili Zhou",
                "Yanna Wang"
            ],
            "abstract": "Princeton WordNet (PWN) is a lexicon-semantic network based on cognitive\nlinguistics, which promotes the development of natural language processing.\nBased on PWN, five Chinese wordnets have been developed to solve the problems\nof syntax and semantics. They include: Northeastern University Chinese WordNet\n(NEW), Sinica Bilingual Ontological WordNet (BOW), Southeast University Chinese\nWordNet (SEW), Taiwan University Chinese WordNet (CWN), Chinese Open WordNet\n(COW). By using them, we found that these word networks have low accuracy and\ncoverage, and cannot completely portray the semantic network of PWN. So we\ndecided to make a new Chinese wordnet called Multi-Fusion Chinese Wordnet (MCW)\nto make up those shortcomings. The key idea is to extend the SEW with the help\nof Oxford bilingual dictionary and Xinhua bilingual dictionary, and then\ncorrect it. More specifically, we used machine learning and manual adjustment\nin our corrections. Two standards were formulated to help our work. We\nconducted experiments on three tasks including relatedness calculation, word\nsimilarity and word sense disambiguation for the comparison of lemma's\naccuracy, at the same time, coverage also was compared. The results indicate\nthat MCW can benefit from coverage and accuracy via our method. However, it\nstill has room for improvement, especially with lemmas. In the future, we will\ncontinue to enhance the accuracy of MCW and expand the concepts in it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01808v5",
            "date": "2020-02-05 14:30:49+00:00",
            "title": "K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters",
            "authors": [
                "Ruize Wang",
                "Duyu Tang",
                "Nan Duan",
                "Zhongyu Wei",
                "Xuanjing Huang",
                "Jianshu ji",
                "Guihong Cao",
                "Daxin Jiang",
                "Ming Zhou"
            ],
            "abstract": "We study the problem of injecting knowledge into large pre-trained models\nlike BERT and RoBERTa. Existing methods typically update the original\nparameters of pre-trained models when injecting knowledge. However, when\nmultiple kinds of knowledge are injected, the historically injected knowledge\nwould be flushed away. To address this, we propose K-Adapter, a framework that\nretains the original parameters of the pre-trained model fixed and supports the\ndevelopment of versatile knowledge-infused model. Taking RoBERTa as the\nbackbone model, K-Adapter has a neural adapter for each kind of infused\nknowledge, like a plug-in connected to RoBERTa. There is no information flow\nbetween different adapters, thus multiple adapters can be efficiently trained\nin a distributed way. As a case study, we inject two kinds of knowledge in this\nwork, including (1) factual knowledge obtained from automatically aligned\ntext-triplets on Wikipedia and Wikidata and (2) linguistic knowledge obtained\nvia dependency parsing. Results on three knowledge-driven tasks, including\nrelation classification, entity typing, and question answering, demonstrate\nthat each adapter improves the performance and the combination of both adapters\nbrings further improvements. Further analysis indicates that K-Adapter captures\nversatile knowledge than RoBERTa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.01984v1",
            "date": "2020-02-05 20:43:14+00:00",
            "title": "UNCC Biomedical Semantic Question Answering Systems. BioASQ: Task-7B, Phase-B",
            "authors": [
                "Sai Krishna Telukuntla",
                "Aditya Kapri",
                "Wlodek Zadrozny"
            ],
            "abstract": "In this paper, we detail our submission to the 2019, 7th year, BioASQ\ncompetition. We present our approach for Task-7b, Phase B, Exact Answering\nTask. These Question Answering (QA) tasks include Factoid, Yes/No, List Type\nQuestion answering. Our system is based on a contextual word embedding model.\nWe have used a Bidirectional Encoder Representations from Transformers(BERT)\nbased system, fined tuned for biomedical question answering task using BioBERT.\nIn the third test batch set, our system achieved the highest MRR score for\nFactoid Question Answering task. Also, for List type question answering task\nour system achieved the highest recall score in the fourth test batch set.\nAlong with our detailed approach, we present the results for our submissions,\nand also highlight identified downsides for our current approach and ways to\nimprove them in our future experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02000v1",
            "date": "2020-02-05 21:40:50+00:00",
            "title": "Aligning the Pretraining and Finetuning Objectives of Language Models",
            "authors": [
                "Nuo Wang Pierse",
                "Jingwen Lu"
            ],
            "abstract": "We demonstrate that explicitly aligning the pretraining objectives to the\nfinetuning objectives in language model training significantly improves the\nfinetuning task performance and reduces the minimum amount of finetuning\nexamples required. The performance margin gained from objective alignment\nallows us to build language models with smaller sizes for tasks with less\navailable training data. We provide empirical evidence of these claims by\napplying objective alignment to concept-of-interest tagging and acronym\ndetection tasks. We found that, with objective alignment, our 768 by 3 and 512\nby 3 transformer language models can reach accuracy of 83.9%/82.5% for\nconcept-of-interest tagging and 73.8%/70.2% for acronym detection using only\n200 finetuning examples per task, outperforming the 768 by 3 model pretrained\nwithout objective alignment by +4.8%/+3.4% and +9.9%/+6.3%. We name finetuning\nsmall language models in the presence of hundreds of training examples or less\n\"Few Example learning\". In practice, Few Example Learning enabled by objective\nalignment not only saves human labeling costs, but also makes it possible to\nleverage language models in more real-time applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02031v1",
            "date": "2020-02-05 22:56:11+00:00",
            "title": "Stimulating Creativity with FunLines: A Case Study of Humor Generation in Headlines",
            "authors": [
                "Nabil Hossain",
                "John Krumm",
                "Tanvir Sajed",
                "Henry Kautz"
            ],
            "abstract": "Building datasets of creative text, such as humor, is quite challenging. We\nintroduce FunLines, a competitive game where players edit news headlines to\nmake them funny, and where they rate the funniness of headlines edited by\nothers. FunLines makes the humor generation process fun, interactive,\ncollaborative, rewarding and educational, keeping players engaged and providing\nhumor data at a very low cost compared to traditional crowdsourcing approaches.\nFunLines offers useful performance feedback, assisting players in getting\nbetter over time at generating and assessing humor, as our analysis shows. This\nhelps to further increase the quality of the generated dataset. We show the\neffectiveness of this data by training humor classification models that\noutperform a previous benchmark, and we release this dataset to the public.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2002.02070v1",
            "date": "2020-02-06 02:10:33+00:00",
            "title": "Understanding Car-Speak: Replacing Humans in Dealerships",
            "authors": [
                "Habeeb Hooshmand",
                "James Caverlee"
            ],
            "abstract": "A large portion of the car-buying experience in the United States involves\ninteractions at a car dealership. At the dealership, the car-buyer relays their\nneeds to a sales representative. However, most car-buyers are only have an\nabstract description of the vehicle they need. Therefore, they are only able to\ndescribe their ideal car in \"car-speak\". Car-speak is abstract language that\npertains to a car's physical attributes. In this paper, we define car-speak. We\nalso aim to curate a reasonable data set of car-speak language. Finally, we\ntrain several classifiers in order to classify car-speak.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.02095v1",
            "date": "2020-02-06 04:37:44+00:00",
            "title": "Attractive or Faithful? Popularity-Reinforced Learning for Inspired Headline Generation",
            "authors": [
                "Yun-Zhu Song",
                "Hong-Han Shuai",
                "Sung-Lin Yeh",
                "Yi-Lun Wu",
                "Lun-Wei Ku",
                "Wen-Chih Peng"
            ],
            "abstract": "With the rapid proliferation of online media sources and published news,\nheadlines have become increasingly important for attracting readers to news\narticles, since users may be overwhelmed with the massive information. In this\npaper, we generate inspired headlines that preserve the nature of news articles\nand catch the eye of the reader simultaneously. The task of inspired headline\ngeneration can be viewed as a specific form of Headline Generation (HG) task,\nwith the emphasis on creating an attractive headline from a given news article.\nTo generate inspired headlines, we propose a novel framework called\nPOpularity-Reinforced Learning for inspired Headline Generation (PORL-HG).\nPORL-HG exploits the extractive-abstractive architecture with 1) Popular Topic\nAttention (PTA) for guiding the extractor to select the attractive sentence\nfrom the article and 2) a popularity predictor for guiding the abstractor to\nrewrite the attractive sentence. Moreover, since the sentence selection of the\nextractor is not differentiable, techniques of reinforcement learning (RL) are\nutilized to bridge the gap with rewards obtained from a popularity score\npredictor. Through quantitative and qualitative experiments, we show that the\nproposed PORL-HG significantly outperforms the state-of-the-art headline\ngeneration models in terms of attractiveness evaluated by both human (71.03%)\nand the predictor (at least 27.60%), while the faithfulness of PORL-HG is also\ncomparable to the state-of-the-art generation model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02109v2",
            "date": "2020-02-06 05:53:41+00:00",
            "title": "Multilingual acoustic word embedding models for processing zero-resource languages",
            "authors": [
                "Herman Kamper",
                "Yevgen Matusevych",
                "Sharon Goldwater"
            ],
            "abstract": "Acoustic word embeddings are fixed-dimensional representations of\nvariable-length speech segments. In settings where unlabelled speech is the\nonly available resource, such embeddings can be used in \"zero-resource\" speech\nsearch, indexing and discovery systems. Here we propose to train a single\nsupervised embedding model on labelled data from multiple well-resourced\nlanguages and then apply it to unseen zero-resource languages. For this\ntransfer learning approach, we consider two multilingual recurrent neural\nnetwork models: a discriminative classifier trained on the joint vocabularies\nof all training languages, and a correspondence autoencoder trained to\nreconstruct word pairs. We test these using a word discrimination task on six\ntarget zero-resource languages. When trained on seven well-resourced languages,\nboth models perform similarly and outperform unsupervised models trained on the\nzero-resource languages. With just a single training language, the second model\nworks better, but performance depends more on the particular training--testing\nlanguage pair.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02238v1",
            "date": "2020-02-06 13:11:46+00:00",
            "title": "Towards Semantic Noise Cleansing of Categorical Data based on Semantic Infusion",
            "authors": [
                "Rishabh Gupta",
                "Rajesh N Rao"
            ],
            "abstract": "Semantic Noise affects text analytics activities for the domain-specific\nindustries significantly. It impedes the text understanding which holds prime\nimportance in the critical decision making tasks. In this work, we formalize\nsemantic noise as a sequence of terms that do not contribute to the narrative\nof the text. We look beyond the notion of standard statistically-based stop\nwords and consider the semantics of terms to exclude the semantic noise. We\npresent a novel Semantic Infusion technique to associate meta-data with the\ncategorical corpus text and demonstrate its near-lossless nature. Based on this\ntechnique, we propose an unsupervised text-preprocessing framework to filter\nthe semantic noise using the context of the terms. Later we present the\nevaluation results of the proposed framework using a web forum dataset from the\nautomobile-domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02427v1",
            "date": "2020-02-06 18:23:27+00:00",
            "title": "Irony Detection in a Multilingual Context",
            "authors": [
                "Bilal Ghanem",
                "Jihen Karoui",
                "Farah Benamara",
                "Paolo Rosso",
                "V\u00e9ronique Moriceau"
            ],
            "abstract": "This paper proposes the first multilingual (French, English and Arabic) and\nmulticultural (Indo-European languages vs. less culturally close languages)\nirony detection system. We employ both feature-based models and neural\narchitectures using monolingual word representation. We compare the performance\nof these systems with state-of-the-art systems to identify their capabilities.\nWe show that these monolingual models trained separately on different languages\nusing multilingual word representation or text-based features can open the door\nto irony detection in languages that lack of annotated data for irony.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02631v1",
            "date": "2020-02-07 05:52:06+00:00",
            "title": "Translating Web Search Queries into Natural Language Questions",
            "authors": [
                "Adarsh Kumar",
                "Sandipan Dandapat",
                "Sushil Chordia"
            ],
            "abstract": "Users often query a search engine with a specific question in mind and often\nthese queries are keywords or sub-sentential fragments. For example, if the\nusers want to know the answer for \"What's the capital of USA\", they will most\nprobably query \"capital of USA\" or \"USA capital\" or some keyword-based\nvariation of this. For example, for the user entered query \"capital of USA\",\nthe most probable question intent is \"What's the capital of USA?\". In this\npaper, we are proposing a method to generate well-formed natural language\nquestion from a given keyword-based query, which has the same question intent\nas the query. Conversion of keyword-based web query into a well-formed question\nhas lots of applications, with some of them being in search engines, Community\nQuestion Answering (CQA) website and bots communication. We found a synergy\nbetween query-to-question problem with standard machine translation(MT) task.\nWe have used both Statistical MT (SMT) and Neural MT (NMT) models to generate\nthe questions from the query. We have observed that MT models perform well in\nterms of both automatic and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.02800v1",
            "date": "2020-02-07 14:18:53+00:00",
            "title": "Depressed individuals express more distorted thinking on social media",
            "authors": [
                "Krishna C. Bathina",
                "Marijn ten Thij",
                "Lorenzo Lorenzo-Luaces",
                "Lauren A. Rutter",
                "Johan Bollen"
            ],
            "abstract": "Depression is a leading cause of disability worldwide, but is often\nunder-diagnosed and under-treated. One of the tenets of cognitive-behavioral\ntherapy (CBT) is that individuals who are depressed exhibit distorted modes of\nthinking, so-called cognitive distortions, which can negatively affect their\nemotions and motivation. Here, we show that individuals with a self-reported\ndiagnosis of depression on social media express higher levels of distorted\nthinking than a random sample. Some types of distorted thinking were found to\nbe more than twice as prevalent in our depressed cohort, in particular\nPersonalizing and Emotional Reasoning. This effect is specific to the distorted\ncontent of the expression and can not be explained by the presence of specific\ntopics, sentiment, or first-person pronouns. Our results point towards the\ndetection, and possibly mitigation, of patterns of online language that are\ngenerally deemed depressogenic. They may also provide insight into recent\nobservations that social media usage can have a negative impact on mental\nhealth.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2002.02925v4",
            "date": "2020-02-07 17:52:16+00:00",
            "title": "BERT-of-Theseus: Compressing BERT by Progressive Module Replacing",
            "authors": [
                "Canwen Xu",
                "Wangchunshu Zhou",
                "Tao Ge",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we propose a novel model compression approach to effectively\ncompress BERT by progressive module replacing. Our approach first divides the\noriginal BERT into several modules and builds their compact substitutes. Then,\nwe randomly replace the original modules with their substitutes to train the\ncompact modules to mimic the behavior of the original modules. We progressively\nincrease the probability of replacement through the training. In this way, our\napproach brings a deeper level of interaction between the original and compact\nmodels. Compared to the previous knowledge distillation approaches for BERT\ncompression, our approach does not introduce any additional loss function. Our\napproach outperforms existing knowledge distillation approaches on GLUE\nbenchmark, showing a new perspective of model compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03049v1",
            "date": "2020-02-07 23:54:23+00:00",
            "title": "Snippext: Semi-supervised Opinion Mining with Augmented Data",
            "authors": [
                "Zhengjie Miao",
                "Yuliang Li",
                "Xiaolan Wang",
                "Wang-Chiew Tan"
            ],
            "abstract": "Online services are interested in solutions to opinion mining, which is the\nproblem of extracting aspects, opinions, and sentiments from text. One method\nto mine opinions is to leverage the recent success of pre-trained language\nmodels which can be fine-tuned to obtain high-quality extractions from reviews.\nHowever, fine-tuning language models still requires a non-trivial amount of\ntraining data. In this paper, we study the problem of how to significantly\nreduce the amount of labeled training data required in fine-tuning language\nmodels for opinion mining. We describe Snippext, an opinion mining system\ndeveloped over a language model that is fine-tuned through semi-supervised\nlearning with augmented data. A novelty of Snippext is its clever use of a\ntwo-prong approach to achieve state-of-the-art (SOTA) performance with little\nlabeled training data through: (1) data augmentation to automatically generate\nmore labeled training data from existing ones, and (2) a semi-supervised\nlearning technique to leverage the massive amount of unlabeled data in addition\nto the (limited amount of) labeled data. We show with extensive experiments\nthat Snippext performs comparably and can even exceed previous SOTA results on\nseveral opinion mining tasks with only half the training data required.\nFurthermore, it achieves new SOTA results when all training data are leveraged.\nBy comparison to a baseline pipeline, we found that Snippext extracts\nsignificantly more fine-grained opinions which enable new opportunities of\ndownstream applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03056v1",
            "date": "2020-02-08 00:42:21+00:00",
            "title": "autoNLP: NLP Feature Recommendations for Text Analytics Applications",
            "authors": [
                "Janardan Misra"
            ],
            "abstract": "While designing machine learning based text analytics applications, often,\nNLP data scientists manually determine which NLP features to use based upon\ntheir knowledge and experience with related problems. This results in increased\nefforts during feature engineering process and renders automated reuse of\nfeatures across semantically related applications inherently difficult. In this\npaper, we argue for standardization in feature specification by outlining\nstructure of a language for specifying NLP features and present an approach for\ntheir reuse across applications to increase likelihood of identifying optimal\nfeatures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03079v2",
            "date": "2020-02-08 03:41:37+00:00",
            "title": "Blank Language Models",
            "authors": [
                "Tianxiao Shen",
                "Victor Quach",
                "Regina Barzilay",
                "Tommi Jaakkola"
            ],
            "abstract": "We propose Blank Language Model (BLM), a model that generates sequences by\ndynamically creating and filling in blanks. The blanks control which part of\nthe sequence to expand, making BLM ideal for a variety of text editing and\nrewriting tasks. The model can start from a single blank or partially completed\ntext with blanks at specified locations. It iteratively determines which word\nto place in a blank and whether to insert new blanks, and stops generating when\nno blanks are left to fill. BLM can be efficiently trained using a lower bound\nof the marginal data likelihood. On the task of filling missing text snippets,\nBLM significantly outperforms all other baselines in terms of both accuracy and\nfluency. Experiments on style transfer and damaged ancient text restoration\ndemonstrate the potential of this framework for a wide range of applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03246v1",
            "date": "2020-02-08 23:15:06+00:00",
            "title": "SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning",
            "authors": [
                "Andrew Best",
                "Sahil Narang",
                "Dinesh Manocha"
            ],
            "abstract": "We present a novel approach for generating plausible verbal interactions\nbetween virtual human-like agents and user avatars in shared virtual\nenvironments. Sense-Plan-Ask, or SPA, extends prior work in propositional\nplanning and natural language processing to enable agents to plan with\nuncertain information, and leverage question and answer dialogue with other\nagents and avatars to obtain the needed information and complete their goals.\nThe agents are additionally able to respond to questions from the avatars and\nother agents using natural-language enabling real-time multi-agent multi-avatar\ncommunication environments.\n  Our algorithm can simulate tens of virtual agents at interactive rates\ninteracting, moving, communicating, planning, and replanning. We find that our\nalgorithm creates a small runtime cost and enables agents to complete their\ngoals more effectively than agents without the ability to leverage\nnatural-language communication. We demonstrate quantitative results on a set of\nsimulated benchmarks and detail the results of a preliminary user-study\nconducted to evaluate the plausibility of the virtual interactions generated by\nSPA. Overall, we find that participants prefer SPA to prior techniques in 84\\%\nof responses including significant benefits in terms of the plausibility of\nnatural-language interactions and the positive impact of those interactions.",
            "journal ref": "None",
            "category": "cs.MA"
        },
        {
            "url": "http://arxiv.org/abs/2002.03259v1",
            "date": "2020-02-09 01:03:25+00:00",
            "title": "Rough Set based Aggregate Rank Measure & its Application to Supervised Multi Document Summarization",
            "authors": [
                "Nidhika Yadav",
                "Niladri Chatterjee"
            ],
            "abstract": "Most problems in Machine Learning cater to classification and the objects of\nuniverse are classified to a relevant class. Ranking of classified objects of\nuniverse per decision class is a challenging problem. We in this paper propose\na novel Rough Set based membership called Rank Measure to solve to this\nproblem. It shall be utilized for ranking the elements to a particular class.\nIt differs from Pawlak Rough Set based membership function which gives an\nequivalent characterization of the Rough Set based approximations. It becomes\nparamount to look beyond the traditional approach of computing memberships\nwhile handling inconsistent, erroneous and missing data that is typically\npresent in real world problems. This led us to propose the aggregate Rank\nMeasure. The contribution of the paper is three fold. Firstly, it proposes a\nRough Set based measure to be utilized for numerical characterization of within\nclass ranking of objects. Secondly, it proposes and establish the properties of\nRank Measure and aggregate Rank Measure based membership. Thirdly, we apply the\nconcept of membership and aggregate ranking to the problem of supervised Multi\nDocument Summarization wherein first the important class of sentences are\ndetermined using various supervised learning techniques and are post processed\nusing the proposed ranking measure. The results proved to have significant\nimprovement in accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03405v3",
            "date": "2020-02-09 17:46:22+00:00",
            "title": "Attend to the beginning: A study on using bidirectional attention for extractive summarization",
            "authors": [
                "Ahmed Magooda",
                "Cezary Marcjan"
            ],
            "abstract": "Forum discussion data differ in both structure and properties from generic\nform of textual data such as news. Henceforth, summarization techniques should,\nin turn, make use of such differences, and craft models that can benefit from\nthe structural nature of discussion data. In this work, we propose attending to\nthe beginning of a document, to improve the performance of extractive\nsummarization models when applied to forum discussion data. Evaluations\ndemonstrated that with the help of bidirectional attention mechanism, attending\nto the beginning of a document (initial comment/post) in a discussion thread,\ncan introduce a consistent boost in ROUGE scores, as well as introducing a new\nState Of The Art (SOTA) ROUGE scores on the forum discussions dataset.\nAdditionally, we explored whether this hypothesis is extendable to other\ngeneric forms of textual data. We make use of the tendency of introducing\nimportant information early in the text, by attending to the first few\nsentences in generic textual data. Evaluations demonstrated that attending to\nintroductory sentences using bidirectional attention, improves the performance\nof extractive summarization models when even applied to more generic form of\ntextual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03407v1",
            "date": "2020-02-09 17:49:08+00:00",
            "title": "Abstractive Summarization for Low Resource Data using Domain Transfer and Data Synthesis",
            "authors": [
                "Ahmed Magooda",
                "Diane Litman"
            ],
            "abstract": "Training abstractive summarization models typically requires large amounts of\ndata, which can be a limitation for many domains. In this paper we explore\nusing domain transfer and data synthesis to improve the performance of recent\nabstractive summarization methods when applied to small corpora of student\nreflections. First, we explored whether tuning state of the art model trained\non newspaper data could boost performance on student reflection data.\nEvaluations demonstrated that summaries produced by the tuned model achieved\nhigher ROUGE scores compared to model trained on just student reflection data\nor just newspaper data. The tuned model also achieved higher scores compared to\nextractive summarization baselines, and additionally was judged to produce more\ncoherent and readable summaries in human evaluations. Second, we explored\nwhether synthesizing summaries of student data could additionally boost\nperformance. We proposed a template-based model to synthesize new data, which\nwhen incorporated into training further increased ROUGE scores. Finally, we\nshowed that combining data synthesis with domain transfer achieved higher ROUGE\nscores compared to only using one of the two approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03518v2",
            "date": "2020-02-10 03:27:21+00:00",
            "title": "Multilingual Alignment of Contextual Word Representations",
            "authors": [
                "Steven Cao",
                "Nikita Kitaev",
                "Dan Klein"
            ],
            "abstract": "We propose procedures for evaluating and strengthening contextual embedding\nalignment and show that they are useful in analyzing and improving multilingual\nBERT. In particular, after our proposed alignment procedure, BERT exhibits\nsignificantly improved zero-shot performance on XNLI compared to the base\nmodel, remarkably matching pseudo-fully-supervised translate-train models for\nBulgarian and Greek. Further, to measure the degree of alignment, we introduce\na contextual version of word retrieval and show that it correlates well with\ndownstream zero-shot transfer. Using this word retrieval task, we also analyze\nBERT and find that it exhibits systematic deficiencies, e.g. worse alignment\nfor open-class parts-of-speech and word pairs written in different scripts,\nthat are corrected by the alignment procedure. These results support contextual\nalignment as a useful concept for understanding large multilingual pre-trained\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.03531v1",
            "date": "2020-02-10 04:00:07+00:00",
            "title": "A Novel Kuhnian Ontology for Epistemic Classification of STM Scholarly Articles",
            "authors": [
                "Khalid M. Saqr",
                "Abdelrahman Elsharawy"
            ],
            "abstract": "Thomas Kuhn proposed his paradigmatic view of scientific discovery five\ndecades ago. The concept of paradigm has not only explained the progress of\nscience, but has also become the central epistemic concept among STM\nscientists. Here, we adopt the principles of Kuhnian philosophy to construct a\nnovel ontology aims at classifying and evaluating the impact of STM scholarly\narticles. First, we explain how the Kuhnian cycle of science describes research\nat different epistemic stages. Second, we show how the Kuhnian cycle could be\nreconstructed into modular ontologies which classify scholarly articles\naccording to their contribution to paradigm-centred knowledge. The proposed\nontology and its scenarios are discussed. To the best of the authors knowledge,\nthis is the first attempt for creating an ontology for describing scholarly\narticles based on the Kuhnian paradigmatic view of science.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2002.03552v1",
            "date": "2020-02-10 05:23:38+00:00",
            "title": "Automating App Review Response Generation",
            "authors": [
                "Cuiyun Gao",
                "Jichuan Zeng",
                "Xin Xia",
                "David Lo",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Previous studies showed that replying to a user review usually has a positive\neffect on the rating that is given by the user to the app. For example, Hassan\net al. found that responding to a review increases the chances of a user\nupdating their given rating by up to six times compared to not responding. To\nalleviate the labor burden in replying to the bulk of user reviews, developers\nusually adopt a template-based strategy where the templates can express\nappreciation for using the app or mention the company email address for users\nto follow up. However, reading a large number of user reviews every day is not\nan easy task for developers. Thus, there is a need for more automation to help\ndevelopers respond to user reviews.\n  Addressing the aforementioned need, in this work we propose a novel approach\nRRGen that automatically generates review responses by learning knowledge\nrelations between reviews and their responses. RRGen explicitly incorporates\nreview attributes, such as user rating and review length, and learns the\nrelations between reviews and corresponding responses in a supervised way from\nthe available training data. Experiments on 58 apps and 309,246 review-response\npairs highlight that RRGen outperforms the baselines by at least 67.4% in terms\nof BLEU-4 (an accuracy measure that is widely used to evaluate dialogue\nresponse generation systems). Qualitative analysis also confirms the\neffectiveness of RRGen in generating relevant and accurate responses.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2002.03912v3",
            "date": "2020-02-10 16:20:49+00:00",
            "title": "A Probabilistic Formulation of Unsupervised Text Style Transfer",
            "authors": [
                "Junxian He",
                "Xinyi Wang",
                "Graham Neubig",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We present a deep generative model for unsupervised text style transfer that\nunifies previously proposed non-generative techniques. Our probabilistic\napproach models non-parallel data from two domains as a partially observed\nparallel corpus. By hypothesizing a parallel latent sequence that generates\neach observed sequence, our model learns to transform sequences from one domain\nto another in a completely unsupervised fashion. In contrast with traditional\ngenerative sequence models (e.g. the HMM), our model makes few assumptions\nabout the data it generates: it uses a recurrent language model as a prior and\nan encoder-decoder as a transduction distribution. While computation of\nmarginal data likelihood is intractable in this model class, we show that\namortized variational inference admits a practical surrogate. Further, by\ndrawing connections between our variational objective and other recent\nunsupervised style transfer and machine translation techniques, we show how our\nprobabilistic view can unify some known non-generative objectives such as\nbacktranslation and adversarial loss. Finally, we demonstrate the effectiveness\nof our method on a wide range of unsupervised style transfer tasks, including\nsentiment transfer, formality transfer, word decipherment, author imitation,\nand related language translation. Across all style transfer tasks, our approach\nyields substantial gains over state-of-the-art non-generative baselines,\nincluding the state-of-the-art unsupervised machine translation techniques that\nour approach generalizes. Further, we conduct experiments on a standard\nunsupervised machine translation task and find that our unified approach\nmatches the current state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04494v2",
            "date": "2020-02-11 15:49:32+00:00",
            "title": "The Rumour Mill: Making the Spread of Misinformation Explicit and Tangible",
            "authors": [
                "Nanna Inie",
                "Jeanette Falk Olesen",
                "Leon Derczynski"
            ],
            "abstract": "Misinformation spread presents a technological and social threat to society.\nWith the advance of AI-based language models, automatically generated texts\nhave become difficult to identify and easy to create at scale. We present \"The\nRumour Mill\", a playful art piece, designed as a commentary on the spread of\nrumours and automatically-generated misinformation. The mill is a tabletop\ninteractive machine, which invites a user to experience the process of creating\nbelievable text by interacting with different tangible controls on the mill.\nThe user manipulates visible parameters to adjust the genre and type of an\nautomatically generated text rumour. The Rumour Mill is a physical\ndemonstration of the state of current technology and its ability to generate\nand manipulate natural language text, and of the act of starting and spreading\nrumours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04513v2",
            "date": "2020-02-03 17:55:19+00:00",
            "title": "An experiment exploring the theoretical and methodological challenges in developing a semi-automated approach to analysis of small-N qualitative data",
            "authors": [
                "Sandro Tsang"
            ],
            "abstract": "This paper experiments with designing a semi-automated qualitative data\nanalysis (QDA) algorithm to analyse 20 transcripts by using freeware.\nText-mining (TM) and QDA were guided by frequency and association measures,\nbecause these statistics remain robust when the sample size is small. The\nrefined TM algorithm split the text into various sizes based on a manually\nrevised dictionary. This lemmatisation approach may reflect the context of the\ntext better than uniformly tokenising the text into one single size. TM results\nwere used for initial coding. Code repacking was guided by association measures\nand external data to implement a general inductive QDA approach. The\ninformation retrieved by TM and QDA was depicted in subgraphs for comparisons.\nThe analyses were completed in 6-7 days. Both algorithms retrieved contextually\nconsistent and relevant information. However, the QDA algorithm retrieved more\nspecific information than TM alone. The QDA algorithm does not strictly comply\nwith the convention of TM or of QDA, but becomes a more efficient, systematic\nand transparent text analysis approach than a conventional QDA approach.\nScaling up QDA to reliably discover knowledge from text was exactly the\nresearch purpose. This paper also sheds light on understanding the relations\nbetween information technologies, theory and methodologies.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.04689v1",
            "date": "2020-02-11 21:17:29+00:00",
            "title": "Two Huge Title and Keyword Generation Corpora of Research Articles",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Recent developments in sequence-to-sequence learning with neural networks\nhave considerably improved the quality of automatically generated text\nsummaries and document keywords, stipulating the need for even bigger training\ncorpora. Metadata of research articles are usually easy to find online and can\nbe used to perform research on various tasks. In this paper, we introduce two\nhuge datasets for text summarization (OAGSX) and keyword generation (OAGKX)\nresearch, containing 34 million and 23 million records, respectively. The data\nwere retrieved from the Open Academic Graph which is a network of research\nprofiles and publications. We carefully processed each record and also tried\nseveral extractive and abstractive methods of both tasks to create performance\nbaselines for other researchers. We further illustrate the performance of those\nmethods previewing their outputs. In the near future, we would like to apply\ntopic modeling on the two sets to derive subsets of research articles from more\nspecific disciplines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04793v2",
            "date": "2020-02-12 04:31:40+00:00",
            "title": "ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems",
            "authors": [
                "Qi Zhu",
                "Zheng Zhang",
                "Yan Fang",
                "Xiang Li",
                "Ryuichi Takanobu",
                "Jinchao Li",
                "Baolin Peng",
                "Jianfeng Gao",
                "Xiaoyan Zhu",
                "Minlie Huang"
            ],
            "abstract": "We present ConvLab-2, an open-source toolkit that enables researchers to\nbuild task-oriented dialogue systems with state-of-the-art models, perform an\nend-to-end evaluation, and diagnose the weakness of systems. As the successor\nof ConvLab (Lee et al., 2019b), ConvLab-2 inherits ConvLab's framework but\nintegrates more powerful dialogue models and supports more datasets. Besides,\nwe have developed an analysis tool and an interactive tool to assist\nresearchers in diagnosing dialogue systems. The analysis tool presents rich\nstatistics and summarizes common mistakes from simulated dialogues, which\nfacilitates error analysis and system improvement. The interactive tool\nprovides a user interface that allows developers to diagnose an assembled\ndialogue system by interacting with the system and modifying the output of each\nsystem component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04815v1",
            "date": "2020-02-12 06:11:48+00:00",
            "title": "Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference",
            "authors": [
                "Youwei Song",
                "Jiahai Wang",
                "Zhiwei Liang",
                "Zhiyue Liu",
                "Tao Jiang"
            ],
            "abstract": "Aspect based sentiment analysis aims to identify the sentimental tendency\ntowards a given aspect in text. Fine-tuning of pretrained BERT performs\nexcellent on this task and achieves state-of-the-art performances. Existing\nBERT-based works only utilize the last output layer of BERT and ignore the\nsemantic knowledge in the intermediate layers. This paper explores the\npotential of utilizing BERT intermediate layers to enhance the performance of\nfine-tuning of BERT. To the best of our knowledge, no existing work has been\ndone on this research. To show the generality, we also apply this approach to a\nnatural language inference task. Experimental results demonstrate the\neffectiveness and generality of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.04936v1",
            "date": "2020-02-12 12:06:32+00:00",
            "title": "Joint Embedding in Named Entity Linking on Sentence Level",
            "authors": [
                "Wei Shi",
                "Siyuan Zhang",
                "Zhiwei Zhang",
                "Hong Cheng",
                "Jeffrey Xu Yu"
            ],
            "abstract": "Named entity linking is to map an ambiguous mention in documents to an entity\nin a knowledge base. The named entity linking is challenging, given the fact\nthat there are multiple candidate entities for a mention in a document. It is\ndifficult to link a mention when it appears multiple times in a document, since\nthere are conflicts by the contexts around the appearances of the mention. In\naddition, it is difficult since the given training dataset is small due to the\nreason that it is done manually to link a mention to its mapping entity. In the\nliterature, there are many reported studies among which the recent embedding\nmethods learn vectors of entities from the training dataset at document level.\nTo address these issues, we focus on how to link entity for mentions at a\nsentence level, which reduces the noises introduced by different appearances of\nthe same mention in a document at the expense of insufficient information to be\nused. We propose a new unified embedding method by maximizing the relationships\nlearned from knowledge graphs. We confirm the effectiveness of our method in\nour experimental studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05058v1",
            "date": "2020-02-12 15:52:21+00:00",
            "title": "Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models",
            "authors": [
                "Wangchunshu Zhou",
                "Ke Xu"
            ],
            "abstract": "Automated evaluation of open domain natural language generation (NLG) models\nremains a challenge and widely used metrics such as BLEU and Perplexity can be\nmisleading in some cases. In our paper, we propose to evaluate natural language\ngeneration models by learning to compare a pair of generated sentences by\nfine-tuning BERT, which has been shown to have good natural language\nunderstanding ability. We also propose to evaluate the model-level quality of\nNLG models with sample-level comparison results with skill rating system. While\nable to be trained in a fully self-supervised fashion, our model can be further\nfine-tuned with a little amount of human preference annotation to better\nimitate human judgment. In addition to evaluating trained models, we propose to\napply our model as a performance indicator during training for better\nhyperparameter tuning and early-stopping. We evaluate our approach on both\nstory generation and chit-chat dialogue response generation. Experimental\nresults show that our model correlates better with human preference compared\nwith previous automated evaluation approaches. Training with the proposed\nmetric yields better performance in human evaluation, which further\ndemonstrates the effectiveness of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05162v2",
            "date": "2020-02-12 17:28:30+00:00",
            "title": "A Combined Stochastic and Physical Framework for Modeling Indoor 5G Millimeter Wave Propagation",
            "authors": [
                "Georges Nassif",
                "Catherine Gloaguen",
                "Philippe Martins"
            ],
            "abstract": "Indoor coverage is a major challenge for 5G millimeter waves (mmWaves). In\nthis paper, we address this problem through a novel theoretical framework that\ncombines stochastic indoor environment modeling with advanced physical\npropagation simulation. This approach is particularly adapted to investigate\nindoor-to-indoor 5G mmWave propagation. Its system implementation, so-called\niGeoStat, generates parameterized typical environments that account for the\nindoor spatial variations, then simulates radio propagation based on the\nphysical interaction between electromagnetic waves and material properties.\nThis framework is not dedicated to a particular environment, material,\nfrequency or use case and aims to statistically understand the influence of\nindoor environment parameters on mmWave propagation properties, especially\ncoverage and path loss. Its implementation raises numerous computational\nchallenges that we solve by formulating an adapted link budget and designing\nnew memory optimization algorithms. The first simulation results for two major\n5G applications are validated with measurement data and show the efficiency of\niGeoStat to simulate multiple diffusion in realistic environments, within a\nreasonable amount of time and memory resources. Generated output maps confirm\nthat diffusion has a critical impact on indoor mmWave propagation and that\nproper physical modeling is of the utmost importance to generate relevant\npropagation models.",
            "journal ref": "None",
            "category": "cs.NI"
        },
        {
            "url": "http://arxiv.org/abs/2002.05201v2",
            "date": "2020-02-12 19:56:58+00:00",
            "title": "Deep compositional robotic planners that follow natural language commands",
            "authors": [
                "Yen-Ling Kuo",
                "Boris Katz",
                "Andrei Barbu"
            ],
            "abstract": "We demonstrate how a sampling-based robotic planner can be augmented to learn\nto understand a sequence of natural language commands in a continuous\nconfiguration space to move and manipulate objects. Our approach combines a\ndeep network structured according to the parse of a complex command that\nincludes objects, verbs, spatial relations, and attributes, with a\nsampling-based planner, RRT. A recurrent hierarchical deep network controls how\nthe planner explores the environment, determines when a planned path is likely\nto achieve a goal, and estimates the confidence of each move to trade off\nexploitation and exploration between the network and the planner. Planners are\ndesigned to have near-optimal behavior when information about the task is\nmissing, while networks learn to exploit observations which are available from\nthe environment, making the two naturally complementary. Combining the two\nenables generalization to new maps, new kinds of obstacles, and more complex\nsentences that do not occur in the training set. Little data is required to\ntrain the model despite it jointly acquiring a CNN that extracts features from\nthe environment as it learns the meanings of words. The model provides a level\nof interpretability through the use of attention maps allowing users to see its\nreasoning steps despite being an end-to-end model. This end-to-end model allows\nrobots to learn to follow natural language commands in challenging continuous\nenvironments.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/2002.05527v1",
            "date": "2020-02-12 04:01:57+00:00",
            "title": "Unsupervised Separation of Native and Loanwords for Malayalam and Telugu",
            "authors": [
                "Sridhama Prakhya",
                "Deepak P"
            ],
            "abstract": "Quite often, words from one language are adopted within a different language\nwithout translation; these words appear in transliterated form in text written\nin the latter language. This phenomenon is particularly widespread within\nIndian languages where many words are loaned from English. In this paper, we\naddress the task of identifying loanwords automatically and in an unsupervised\nmanner, from large datasets of words from agglutinative Dravidian languages. We\ntarget two specific languages from the Dravidian family, viz., Malayalam and\nTelugu. Based on familiarity with the languages, we outline an observation that\nnative words in both these languages tend to be characterized by a much more\nversatile stem - stem being a shorthand to denote the subword sequence formed\nby the first few characters of the word - than words that are loaned from other\nlanguages. We harness this observation to build an objective function and an\niterative optimization formulation to optimize for it, yielding a scoring of\neach word's nativeness in the process. Through an extensive empirical analysis\nover real-world datasets from both Malayalam and Telugu, we illustrate the\neffectiveness of our method in quantifying nativeness effectively over\navailable baselines for the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05556v1",
            "date": "2020-02-13 15:08:12+00:00",
            "title": "Sparse and Structured Visual Attention",
            "authors": [
                "Pedro Henrique Martins",
                "Vlad Niculae",
                "Zita Marinho",
                "Andr\u00e9 Martins"
            ],
            "abstract": "Visual attention mechanisms are widely used in multimodal tasks, such as\nimage captioning and visual question answering (VQA). One drawback of\nsoftmax-based attention mechanisms is that they assign probability mass to all\nimage regions, regardless of their adjacency structure and of their relevance\nto the text. In this paper, to better link the image structure with the text,\nwe replace the traditional softmax attention mechanism with two alternative\nsparsity-promoting transformations: sparsemax, which is able to select the\nrelevant regions only (assigning zero weight to the rest), and a newly proposed\nTotal-Variation Sparse Attention (TVmax), which further encourages the joint\nselection of adjacent spatial locations. Experiments in image captioning and\nVQA, using both LSTM and Transformer architectures, show gains in terms of\nhuman-rated caption quality, attention relevance, and VQA accuracy, with\nimproved interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05606v1",
            "date": "2020-02-13 16:30:34+00:00",
            "title": "Sentiment Analysis Using Averaged Weighted Word Vector Features",
            "authors": [
                "Ali Erkan",
                "Tunga Gungor"
            ],
            "abstract": "People use the world wide web heavily to share their experience with entities\nsuch as products, services, or travel destinations. Texts that provide online\nfeedback in the form of reviews and comments are essential to make consumer\ndecisions. These comments create a valuable source that may be used to measure\nsatisfaction related to products or services. Sentiment analysis is the task of\nidentifying opinions expressed in such text fragments. In this work, we develop\ntwo methods that combine different types of word vectors to learn and estimate\npolarity of reviews. We develop average review vectors from word vectors and\nadd weights to this review vectors using word frequencies in positive and\nnegative sensitivity-tagged reviews. We applied the methods to several datasets\nfrom different domains that are used as standard benchmarks for sentiment\nanalysis. We ensemble the techniques with each other and existing methods, and\nwe make a comparison with the approaches in the literature. The results show\nthat the performances of our approaches outperform the state-of-the-art success\nrates.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05607v1",
            "date": "2020-02-13 16:31:50+00:00",
            "title": "Pre-Training for Query Rewriting in A Spoken Language Understanding System",
            "authors": [
                "Zheng Chen",
                "Xing Fan",
                "Yuan Ling",
                "Lambert Mathias",
                "Chenlei Guo"
            ],
            "abstract": "Query rewriting (QR) is an increasingly important technique to reduce\ncustomer friction caused by errors in a spoken language understanding pipeline,\nwhere the errors originate from various sources such as speech recognition\nerrors, language understanding errors or entity resolution errors. In this\nwork, we first propose a neural-retrieval based approach for query rewriting.\nThen, inspired by the wide success of pre-trained contextual language\nembeddings, and also as a way to compensate for insufficient QR training data,\nwe propose a language-modeling (LM) based approach to pre-train query\nembeddings on historical user conversation data with a voice assistant. In\naddition, we propose to use the NLU hypotheses generated by the language\nunderstanding system to augment the pre-training. Our experiments show\npre-training provides rich prior information and help the QR task achieve\nstrong performance. We also show joint pre-training with NLU hypotheses has\nfurther benefit. Finally, after pre-training, we find a small set of rewrite\npairs is enough to fine-tune the QR model to outperform a strong baseline by\nfull training on all QR training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05867v2",
            "date": "2020-02-14 04:23:28+00:00",
            "title": "Transformers as Soft Reasoners over Language",
            "authors": [
                "Peter Clark",
                "Oyvind Tafjord",
                "Kyle Richardson"
            ],
            "abstract": "Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of\nproviding a system with explicit, general knowledge and having the system\nreason over that knowledge. However, expressing the knowledge in a formal\n(logical or probabilistic) representation has been a major obstacle to this\nresearch. This paper investigates a modern approach to this problem where the\nfacts and rules are provided as natural language sentences, thus bypassing a\nformal representation. We train transformers to reason (or emulate reasoning)\nover these sentences using synthetically generated data. Our models, that we\ncall RuleTakers, provide the first empirical demonstration that this kind of\nsoft reasoning over language is learnable, can achieve high (99%) accuracy, and\ngeneralizes to test data requiring substantially deeper chaining than seen\nduring training (95%+ scores). We also demonstrate that the models transfer\nwell to two hand-authored rulebases, and to rulebases paraphrased into more\nnatural language. These findings are significant as it suggests a new role for\ntransformers, namely as limited \"soft theorem provers\" operating over explicit\ntheories in language. This in turn suggests new possibilities for\nexplainability, correctability, and counterfactual reasoning in\nquestion-answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.05923v2",
            "date": "2020-02-14 09:04:18+00:00",
            "title": "Zero-Resource Cross-Domain Named Entity Recognition",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Pascale Fung"
            ],
            "abstract": "Existing models for cross-domain named entity recognition (NER) rely on\nnumerous unlabeled corpus or labeled NER training data in target domains.\nHowever, collecting data for low-resource target domains is not only expensive\nbut also time-consuming. Hence, we propose a cross-domain NER model that does\nnot use any external resources. We first introduce a Multi-Task Learning (MTL)\nby adding a new objective function to detect whether tokens are named entities\nor not. We then introduce a framework called Mixture of Entity Experts (MoEE)\nto improve the robustness for zero-resource domain adaptation. Finally,\nexperimental results show that our model outperforms strong unsupervised\ncross-domain sequence labeling models, and the performance of our model is\nclose to that of the state-of-the-art model which leverages extensive\nresources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06170v1",
            "date": "2020-02-14 18:41:58+00:00",
            "title": "Transformer on a Diet",
            "authors": [
                "Chenguang Wang",
                "Zihao Ye",
                "Aston Zhang",
                "Zheng Zhang",
                "Alexander J. Smola"
            ],
            "abstract": "Transformer has been widely used thanks to its ability to capture sequence\ninformation in an efficient way. However, recent developments, such as BERT and\nGPT-2, deliver only heavy architectures with a focus on effectiveness. In this\npaper, we explore three carefully-designed light Transformer architectures to\nfigure out whether the Transformer with less computations could produce\ncompetitive results. Experimental results on language model benchmark datasets\nhint that such trade-off is promising, and the light Transformer reduces 70%\nparameters at best, while obtains competitive perplexity compared to standard\nTransformer. The source code is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06305v1",
            "date": "2020-02-15 02:40:10+00:00",
            "title": "Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping",
            "authors": [
                "Jesse Dodge",
                "Gabriel Ilharco",
                "Roy Schwartz",
                "Ali Farhadi",
                "Hannaneh Hajishirzi",
                "Noah Smith"
            ],
            "abstract": "Fine-tuning pretrained contextual word embedding models to supervised\ndownstream tasks has become commonplace in natural language processing. This\nprocess, however, is often brittle: even with the same hyperparameter values,\ndistinct random seeds can lead to substantially different results. To better\nunderstand this phenomenon, we experiment with four datasets from the GLUE\nbenchmark, fine-tuning BERT hundreds of times on each while varying only the\nrandom seeds. We find substantial performance increases compared to previously\nreported results, and we quantify how the performance of the best-found model\nvaries as a function of the number of fine-tuning trials. Further, we examine\ntwo factors influenced by the choice of random seed: weight initialization and\ntraining data order. We find that both contribute comparably to the variance of\nout-of-sample performance, and that some weight initializations perform well\nacross all tasks explored. On small datasets, we observe that many fine-tuning\ntrials diverge part of the way through training, and we offer best practices\nfor practitioners to stop training less promising runs early. We publicly\nrelease all of our experimental data, including training and validation scores\nfor 2,100 trials, to encourage further analysis of training dynamics during\nfine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06525v1",
            "date": "2020-02-16 07:10:45+00:00",
            "title": "Learning to Generate Multiple Style Transfer Outputs for an Input Sentence",
            "authors": [
                "Kevin Lin",
                "Ming-Yu Liu",
                "Ming-Ting Sun",
                "Jan Kautz"
            ],
            "abstract": "Text style transfer refers to the task of rephrasing a given text in a\ndifferent style. While various methods have been proposed to advance the state\nof the art, they often assume the transfer output follows a delta distribution,\nand thus their models cannot generate different style transfer results for a\ngiven input text. To address the limitation, we propose a one-to-many text\nstyle transfer framework. In contrast to prior works that learn a one-to-one\nmapping that converts an input sentence to one output sentence, our approach\nlearns a one-to-many mapping that can convert an input sentence to multiple\ndifferent output sentences, while preserving the input content. This is\nachieved by applying adversarial training with a latent decomposition scheme.\nSpecifically, we decompose the latent representation of the input sentence to a\nstyle code that captures the language style variation and a content code that\nencodes the language style-independent content. We then combine the content\ncode with the style code for generating a style transfer output. By combining\nthe same content code with a different style code, we generate a different\nstyle transfer output. Extensive experimental results with comparisons to\nseveral text style transfer approaches on multiple public datasets using a\ndiverse set of performance metrics validate effectiveness of the proposed\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06544v1",
            "date": "2020-02-16 09:22:32+00:00",
            "title": "Exploring Neural Models for Parsing Natural Language into First-Order Logic",
            "authors": [
                "Hrituraj Singh",
                "Milan Aggrawal",
                "Balaji Krishnamurthy"
            ],
            "abstract": "Semantic parsing is the task of obtaining machine-interpretable\nrepresentations from natural language text. We consider one such formal\nrepresentation - First-Order Logic (FOL) and explore the capability of neural\nmodels in parsing English sentences to FOL. We model FOL parsing as a sequence\nto sequence mapping task where given a natural language sentence, it is encoded\ninto an intermediate representation using an LSTM followed by a decoder which\nsequentially generates the predicates in the corresponding FOL formula. We\nimprove the standard encoder-decoder model by introducing a variable alignment\nmechanism that enables it to align variables across predicates in the predicted\nFOL. We further show the effectiveness of predicting the category of FOL entity\n- Unary, Binary, Variables and Scoped Entities, at each decoder step as an\nauxiliary task on improving the consistency of generated FOL. We perform\nrigorous evaluations and extensive ablations. We also aim to release our code\nas well as large scale FOL dataset along with models to aid further research in\nlogic-based parsing and inference in NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06670v1",
            "date": "2020-02-16 20:20:38+00:00",
            "title": "The Utility of General Domain Transfer Learning for Medical Language Tasks",
            "authors": [
                "Daniel Ranti",
                "Katie Hanss",
                "Shan Zhao",
                "Varun Arvind",
                "Joseph Titano",
                "Anthony Costa",
                "Eric Oermann"
            ],
            "abstract": "The purpose of this study is to analyze the efficacy of transfer learning\ntechniques and transformer-based models as applied to medical natural language\nprocessing (NLP) tasks, specifically radiological text classification. We used\n1,977 labeled head CT reports, from a corpus of 96,303 total reports, to\nevaluate the efficacy of pretraining using general domain corpora and a\ncombined general and medical domain corpus with a bidirectional representations\nfrom transformers (BERT) model for the purpose of radiological text\nclassification. Model performance was benchmarked to a logistic regression\nusing bag-of-words vectorization and a long short-term memory (LSTM)\nmulti-label multi-class classification model, and compared to the published\nliterature in medical text classification. The BERT models using either set of\npretrained checkpoints outperformed the logistic regression model, achieving\nsample-weighted average F1-scores of 0.87 and 0.87 for the general domain model\nand the combined general and biomedical-domain model. General text transfer\nlearning may be a viable technique to generate state-of-the-art results within\nmedical NLP tasks on radiological corpora, outperforming other deep models such\nas LSTMs. The efficacy of pretraining and transformer-based models could serve\nto facilitate the creation of groundbreaking NLP models in the uniquely\nchallenging data environment of medical text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.06854v1",
            "date": "2020-02-17 09:30:52+00:00",
            "title": "HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset",
            "authors": [
                "Diego Antognini",
                "Boi Faltings"
            ],
            "abstract": "Today, recommender systems are an inevitable part of everyone's daily digital\nroutine and are present on most internet platforms. State-of-the-art deep\nlearning-based models require a large number of data to achieve their best\nperformance. Many datasets fulfilling this criterion have been proposed for\nmultiple domains, such as Amazon products, restaurants, or beers. However,\nworks and datasets in the hotel domain are limited: the largest hotel review\ndataset is below the million samples. Additionally, the hotel domain suffers\nfrom a higher data sparsity than traditional recommendation datasets and\ntherefore, traditional collaborative-filtering approaches cannot be applied to\nsuch data. In this paper, we propose HotelRec, a very large-scale hotel\nrecommendation dataset, based on TripAdvisor, containing 50 million reviews. To\nthe best of our knowledge, HotelRec is the largest publicly available dataset\nin the hotel domain (50M versus 0.9M) and additionally, the largest\nrecommendation dataset in a single domain and with textual reviews (50M versus\n22M). We release HotelRec for further research:\nhttps://github.com/Diego999/HotelRec.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.07338v1",
            "date": "2020-02-18 02:22:31+00:00",
            "title": "Conditional Self-Attention for Query-based Summarization",
            "authors": [
                "Yujia Xie",
                "Tianyi Zhou",
                "Yi Mao",
                "Weizhu Chen"
            ],
            "abstract": "Self-attention mechanisms have achieved great success on a variety of NLP\ntasks due to its flexibility of capturing dependency between arbitrary\npositions in a sequence. For problems such as query-based summarization (Qsumm)\nand knowledge graph reasoning where each input sequence is associated with an\nextra query, explicitly modeling such conditional contextual dependencies can\nlead to a more accurate solution, which however cannot be captured by existing\nself-attention mechanisms. In this paper, we propose \\textit{conditional\nself-attention} (CSA), a neural network module designed for conditional\ndependency modeling. CSA works by adjusting the pairwise attention between\ninput tokens in a self-attention module with the matching score of the inputs\nto the given query. Thereby, the contextual dependencies modeled by CSA will be\nhighly relevant to the query. We further studied variants of CSA defined by\ndifferent types of attention. Experiments on Debatepedia and HotpotQA benchmark\ndatasets show CSA consistently outperforms vanilla Transformer and previous\nmodels for the Qsumm problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07397v1",
            "date": "2020-02-18 06:29:01+00:00",
            "title": "Improving Multi-Turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting",
            "authors": [
                "Kun Zhou",
                "Wayne Xin Zhao",
                "Yutao Zhu",
                "Ji-Rong Wen",
                "Jingsong Yu"
            ],
            "abstract": "Open-domain retrieval-based dialogue systems require a considerable amount of\ntraining data to learn their parameters. However, in practice, the negative\nsamples of training data are usually selected from an unannotated conversation\ndata set at random. The generated training data is likely to contain noise and\naffect the performance of the response selection models. To address this\ndifficulty, we consider utilizing the underlying correlation in the data\nresource itself to derive different kinds of supervision signals and reduce the\ninfluence of noisy data. More specially, we consider a main-complementary task\npair. The main task (\\ie our focus) selects the correct response given the last\nutterance and context, and the complementary task selects the last utterance\ngiven the response and context. The key point is that the output of the\ncomplementary task is used to set instance weights for the main task. We\nconduct extensive experiments in two public datasets and obtain significant\nimprovement in both datasets. We also investigate the variant of our approach\nin multiple aspects, and the results have verified the effectiveness of our\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07563v3",
            "date": "2020-02-18 13:57:10+00:00",
            "title": "A Model to Measure the Spread Power of Rumors",
            "authors": [
                "Zoleikha Jahanbakhsh-Nagadeh",
                "Mohammad-Reza Feizi-Derakhshi",
                "Majid Ramezani",
                "Taymaz Rahkar-Farshi",
                "Meysam Asgari-Chenaghlu",
                "Narjes Nikzad-Khasmakhi",
                "Ali-Reza Feizi-Derakhshi",
                "Mehrdad Ranjbar-Khadivi",
                "Elnaz Zafarani-Moattar",
                "Mohammad-Ali Balafar"
            ],
            "abstract": "Nowadays, a significant portion of daily interacted posts in social media are\ninfected by rumors. This study investigates the problem of rumor analysis in\ndifferent areas from other researches. It tackles the unaddressed problem\nrelated to calculating the Spread Power of Rumor (SPR) for the first time and\nseeks to examine the spread power as the function of multi-contextual features.\nFor this purpose, the theory of Allport and Postman will be adopted. In which\nit claims that there are two key factors determinant to the spread power of\nrumors, namely importance and ambiguity. The proposed Rumor Spread Power\nMeasurement Model (RSPMM) computes SPR by utilizing a textual-based approach,\nwhich entails contextual features to compute the spread power of the rumors in\ntwo categories: False Rumor (FR) and True Rumor (TR). Totally 51 contextual\nfeatures are introduced to measure SPR and their impact on classification are\ninvestigated, then 42 features in two categories \"importance\" (28 features) and\n\"ambiguity\" (14 features) are selected to compute SPR. The proposed RSPMM is\nverified on two labelled datasets, which are collected from Twitter and\nTelegram. The results show that (i) the proposed new features are effective and\nefficient to discriminate between FRs and TRs. (ii) the proposed RSPMM approach\nfocused only on contextual features while existing techniques are based on\nStructure and Content features, but RSPMM achieves considerably outstanding\nresults (F-measure=83%). (iii) The result of T-Test shows that SPR criteria can\nsignificantly distinguish between FR and TR, besides it can be useful as a new\nmethod to verify the trueness of rumors.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2002.07591v1",
            "date": "2020-02-18 14:40:20+00:00",
            "title": "Text Classification with Lexicon from PreAttention Mechanism",
            "authors": [
                "QingBiao LI",
                "Chunhua Wu",
                "Kangfeng Zheng"
            ],
            "abstract": "A comprehensive and high-quality lexicon plays a crucial role in traditional\ntext classification approaches. And it improves the utilization of the\nlinguistic knowledge. Although it is helpful for the task, the lexicon has got\nlittle attention in recent neural network models. Firstly, getting a\nhigh-quality lexicon is not easy. We lack an effective automated lexicon\nextraction method, and most lexicons are hand crafted, which is very\ninefficient for big data. What's more, there is no an effective way to use a\nlexicon in a neural network. To address those limitations, we propose a\nPre-Attention mechanism for text classification in this paper, which can learn\nattention of different words according to their effects in the classification\ntasks. The words with different attention can form a domain lexicon.\nExperiments on three benchmark text classification tasks show that our models\nget competitive result comparing with the state-of-the-art methods. We get\n90.5% accuracy on Stanford Large Movie Review dataset, 82.3% on Subjectivity\ndataset, 93.7% on Movie Reviews. And compared with the text classification\nmodel without Pre-Attention mechanism, those with Pre-Attention mechanism\nimprove by 0.9%-2.4% accuracy, which proves the validity of the Pre-Attention\nmechanism. In addition, the Pre-Attention mechanism performs well followed by\ndifferent types of neural networks (e.g., convolutional neural networks and\nLong Short-Term Memory networks). For the same dataset, when we use\nPre-Attention mechanism to get attention value followed by different neural\nnetworks, those words with high attention values have a high degree of\ncoincidence, which proves the versatility and portability of the Pre-Attention\nmechanism. we can get stable lexicons by attention values, which is an\ninspiring method of information extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07660v2",
            "date": "2020-02-18 15:47:14+00:00",
            "title": "Decidability of cutpoint isolation for probabilistic finite automata on letter-bounded inputs",
            "authors": [
                "Paul C. Bell",
                "Pavel Semukhin"
            ],
            "abstract": "We show the surprising result that the cutpoint isolation problem is\ndecidable for Probabilistic Finite Automata (PFA) where input words are taken\nfrom a letter-bounded context-free language. A context-free language\n$\\mathcal{L}$ is letter-bounded when $\\mathcal{L} \\subseteq a_1^*a_2^* \\cdots\na_\\ell^*$ for some finite $\\ell > 0$ where each letter is distinct. A cutpoint\nis isolated when it cannot be approached arbitrarily closely. The decidability\nof this problem is in marked contrast to the situation for the (strict)\nemptiness problem for PFA which is undecidable under the even more severe\nrestrictions of PFA with polynomial ambiguity, commutative matrices and input\nover a letter-bounded language as well as to the injectivity problem which is\nundecidable for PFA over letter-bounded languages. We provide a constructive\nnondeterministic algorithm to solve the cutpoint isolation problem, which holds\neven when the PFA is exponentially ambiguous. We also show that the problem is\nat least NP-hard and use our decision procedure to solve several related\nproblems.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07715v3",
            "date": "2020-02-18 16:41:24+00:00",
            "title": "Neural Relation Prediction for Simple Question Answering over Knowledge Graph",
            "authors": [
                "Amin Abolghasemi",
                "Saeedeh Momtazi"
            ],
            "abstract": "Knowledge graphs are widely used as a typical resource to provide answers to\nfactoid questions. In simple question answering over knowledge graphs, relation\nextraction aims to predict the relation of a factoid question from a set of\npredefined relation types. Most recent methods take advantage of neural\nnetworks to match a question with all predefined relations. In this paper, we\npropose an instance-based method to capture the underlying relation of question\nand to this aim, we detect matching paraphrases of a new question which share\nthe same relation, and their corresponding relation is selected as our\nprediction. The idea of our model roots in the fact that a relation can be\nexpressed with various forms of questions while these forms share lexically or\nsemantically similar terms and concepts. Our experiments on the SimpleQuestions\ndataset show that the proposed model achieves better accuracy compared to the\nstate-of-the-art relation extraction models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07725v2",
            "date": "2020-02-18 16:51:05+00:00",
            "title": "Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims",
            "authors": [
                "Kevin Meng",
                "Damian Jimenez",
                "Fatma Arslan",
                "Jacob Daniel Devasier",
                "Daniel Obembe",
                "Chengkai Li"
            ],
            "abstract": "We present a study on the efficacy of adversarial training on transformer\nneural network models, with respect to the task of detecting check-worthy\nclaims. In this work, we introduce the first adversarially-regularized,\ntransformer-based claim spotter model that achieves state-of-the-art results on\nmultiple challenging benchmarks. We obtain a 4.70 point F1-score improvement\nover current state-of-the-art models on the ClaimBuster Dataset and CLEF2019\nDataset, respectively. In the process, we propose a method to apply adversarial\ntraining to transformer models, which has the potential to be generalized to\nmany similar text classification tasks. Along with our results, we are\nreleasing our codebase and manually labeled datasets. We also showcase our\nmodels' real world usage via a live public API.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.07927v2",
            "date": "2020-02-18 23:52:39+00:00",
            "title": "Studying the Effects of Cognitive Biases in Evaluation of Conversational Agents",
            "authors": [
                "Sashank Santhanam",
                "Alireza Karduni",
                "Samira Shaikh"
            ],
            "abstract": "Humans quite frequently interact with conversational agents. The rapid\nadvancement in generative language modeling through neural networks has helped\nadvance the creation of intelligent conversational agents. Researchers\ntypically evaluate the output of their models through crowdsourced judgments,\nbut there are no established best practices for conducting such studies.\nMoreover, it is unclear if cognitive biases in decision-making are affecting\ncrowdsourced workers' judgments when they undertake these tasks. To\ninvestigate, we conducted a between-subjects study with 77 crowdsourced workers\nto understand the role of cognitive biases, specifically anchoring bias, when\nhumans are asked to evaluate the output of conversational agents. Our results\nprovide insight into how best to evaluate conversational agents. We find\nincreased consistency in ratings across two experimental conditions may be a\nresult of anchoring bias. We also determine that external factors such as time\nand prior experience in similar tasks have effects on inter-rater consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08046v1",
            "date": "2020-02-19 08:17:00+00:00",
            "title": "Tree-structured Attention with Hierarchical Accumulation",
            "authors": [
                "Xuan-Phi Nguyen",
                "Shafiq Joty",
                "Steven C. H. Hoi",
                "Richard Socher"
            ],
            "abstract": "Incorporating hierarchical structures like constituency trees has been shown\nto be effective for various natural language processing (NLP) tasks. However,\nit is evident that state-of-the-art (SOTA) sequence-based models like the\nTransformer struggle to encode such structures inherently. On the other hand,\ndedicated models like the Tree-LSTM, while explicitly modeling hierarchical\nstructures, do not perform as efficiently as the Transformer. In this paper, we\nattempt to bridge this gap with \"Hierarchical Accumulation\" to encode parse\ntree structures into self-attention at constant time complexity. Our approach\noutperforms SOTA methods in four IWSLT translation tasks and the WMT'14\nEnglish-German translation task. It also yields improvements over Transformer\nand Tree-LSTM on three text classification tasks. We further demonstrate that\nusing hierarchical priors can compensate for data shortage, and that our model\nprefers phrase-level attentions over token-level attentions.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2002.08131v1",
            "date": "2020-02-19 12:22:46+00:00",
            "title": "Hierarchical models vs. transfer learning for document-level sentiment classification",
            "authors": [
                "Jeremy Barnes",
                "Vinit Ravishankar",
                "Lilja \u00d8vrelid",
                "Erik Velldal"
            ],
            "abstract": "Documents are composed of smaller pieces - paragraphs, sentences, and tokens\n- that have complex relationships between one another. Sentiment classification\nmodels that take into account the structure inherent in these documents have a\ntheoretical advantage over those that do not. At the same time, transfer\nlearning models based on language model pretraining have shown promise for\ndocument classification. However, these two paradigms have not been\nsystematically compared and it is not clear under which circumstances one\napproach is better than the other. In this work we empirically compare\nhierarchical models and transfer learning for document-level sentiment\nclassification. We show that non-trivial hierarchical models outperform\nprevious baselines and transfer learning on document-level sentiment\nclassification in five languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08155v4",
            "date": "2020-02-19 13:09:07+00:00",
            "title": "CodeBERT: A Pre-Trained Model for Programming and Natural Languages",
            "authors": [
                "Zhangyin Feng",
                "Daya Guo",
                "Duyu Tang",
                "Nan Duan",
                "Xiaocheng Feng",
                "Ming Gong",
                "Linjun Shou",
                "Bing Qin",
                "Ting Liu",
                "Daxin Jiang",
                "Ming Zhou"
            ],
            "abstract": "We present CodeBERT, a bimodal pre-trained model for programming language\n(PL) and nat-ural language (NL). CodeBERT learns general-purpose\nrepresentations that support downstream NL-PL applications such as natural\nlanguage codesearch, code documentation generation, etc. We develop CodeBERT\nwith Transformer-based neural architecture, and train it with a hybrid\nobjective function that incorporates the pre-training task of replaced token\ndetection, which is to detect plausible alternatives sampled from generators.\nThis enables us to utilize both bimodal data of NL-PL pairs and unimodal data,\nwhere the former provides input tokens for model training while the latter\nhelps to learn better generators. We evaluate CodeBERT on two NL-PL\napplications by fine-tuning model parameters. Results show that CodeBERT\nachieves state-of-the-art performance on both natural language code search and\ncode documentation generation tasks. Furthermore, to investigate what type of\nknowledge is learned in CodeBERT, we construct a dataset for NL-PL probing, and\nevaluate in a zero-shot setting where parameters of pre-trained models are\nfixed. Results show that CodeBERT performs better than previous pre-trained\nmodels on NL-PL probing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08325v2",
            "date": "2020-02-19 17:57:46+00:00",
            "title": "VQA-LOL: Visual Question Answering under the Lens of Logic",
            "authors": [
                "Tejas Gokhale",
                "Pratyay Banerjee",
                "Chitta Baral",
                "Yezhou Yang"
            ],
            "abstract": "Logical connectives and their implications on the meaning of a natural\nlanguage sentence are a fundamental aspect of understanding. In this paper, we\ninvestigate whether visual question answering (VQA) systems trained to answer a\nquestion about an image, are able to answer the logical composition of multiple\nsuch questions. When put under this \\textit{Lens of Logic}, state-of-the-art\nVQA models have difficulty in correctly answering these logically composed\nquestions. We construct an augmentation of the VQA dataset as a benchmark, with\nquestions containing logical compositions and linguistic transformations\n(negation, disjunction, conjunction, and antonyms). We propose our {Lens of\nLogic (LOL)} model which uses question-attention and logic-attention to\nunderstand logical connectives in the question, and a novel\nFr\\'echet-Compatibility Loss, which ensures that the answers of the component\nquestions and the composed question are consistent with the inferred logical\noperation. Our model shows substantial improvement in learning logical\ncompositions while retaining performance on VQA. We suggest this work as a move\ntowards robustness by embedding logical connectives in visual understanding.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2002.08562v1",
            "date": "2020-02-20 04:14:35+00:00",
            "title": "Federated pretraining and fine tuning of BERT using clinical notes from multiple silos",
            "authors": [
                "Dianbo Liu",
                "Tim Miller"
            ],
            "abstract": "Large scale contextual representation models, such as BERT, have\nsignificantly advanced natural language processing (NLP) in recently years.\nHowever, in certain area like healthcare, accessing diverse large scale text\ndata from multiple institutions is extremely challenging due to privacy and\nregulatory reasons. In this article, we show that it is possible to both\npretrain and fine tune BERT models in a federated manner using clinical texts\nfrom different silos without moving the data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08608v3",
            "date": "2020-02-20 08:01:28+00:00",
            "title": "FrameAxis: Characterizing Framing Bias and Intensity with Word Embedding",
            "authors": [
                "Haewoon Kwak",
                "Jisun An",
                "Elise Jing",
                "Yong-Yeol Ahn"
            ],
            "abstract": "Framing is a process of emphasizing a certain aspect of an issue over the\nothers, nudging readers or listeners towards different positions on the issue\neven without making a biased argument. Here, we propose FrameAxis, a method for\nautomatically characterizing framing of a given text by identifying the most\nrelevant semantic axes (\"microframes\") that are overrepresented in the text\nusing word embedding. In contrast to the traditional framing analysis, which\ntends to be constrained by a small number of manually annotated general frames,\nour unsupervised approach can be readily applied to large datasets because it\ndoes not require manual annotations. It can also provide more nuanced insights\nby considering a host of semantic axes. Our method is designed to\nquantitatively tease out two important dimensions of framing: \\textit{framing\nbias} -- how biased an argument is -- and \\textit{framing intensity} -- how\nmuch a particular aspect over another is highlighted -- from the text, offering\na nuanced characterization of framing. We demonstrate how FrameAxis\nsuccessfully captures framing bias and intensity in a variety of text data from\nrestaurant reviews to news media. The existing domain knowledge can be directly\nincorporated into FrameAxis by guiding candidate microframes to test and\nfine-tune automatically discovered microframes. We propose methods for\nexplaining the results of FrameAxis at the level of individual words and\ndocuments. Our method may accelerate scalable and nuanced computational\nanalyses of framing across disciplines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08614v1",
            "date": "2020-02-20 08:20:52+00:00",
            "title": "Balancing Cost and Benefit with Tied-Multi Transformers",
            "authors": [
                "Raj Dabre",
                "Raphael Rubino",
                "Atsushi Fujita"
            ],
            "abstract": "We propose and evaluate a novel procedure for training multiple Transformers\nwith tied parameters which compresses multiple models into one enabling the\ndynamic choice of the number of encoder and decoder layers during decoding. In\nsequence-to-sequence modeling, typically, the output of the last layer of the\nN-layer encoder is fed to the M-layer decoder, and the output of the last\ndecoder layer is used to compute loss. Instead, our method computes a single\nloss consisting of NxM losses, where each loss is computed from the output of\none of the M decoder layers connected to one of the N encoder layers. Such a\nmodel subsumes NxM models with different number of encoder and decoder layers,\nand can be used for decoding with fewer than the maximum number of encoder and\ndecoder layers. We then propose a mechanism to choose a priori the number of\nencoder and decoder layers for faster decoding, and also explore recurrent\nstacking of layers and knowledge distillation for model compression. We present\na cost-benefit analysis of applying the proposed approaches for neural machine\ntranslation and show that they reduce decoding costs while preserving\ntranslation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08801v2",
            "date": "2020-02-20 15:25:20+00:00",
            "title": "Guiding attention in Sequence-to-sequence models for Dialogue Act prediction",
            "authors": [
                "Pierre Colombo",
                "Emile Chapuis",
                "Matteo Manica",
                "Emmanuel Vignon",
                "Giovanna Varni",
                "Chloe Clavel"
            ],
            "abstract": "The task of predicting dialog acts (DA) based on conversational dialog is a\nkey component in the development of conversational agents. Accurately\npredicting DAs requires a precise modeling of both the conversation and the\nglobal tag dependencies. We leverage seq2seq approaches widely adopted in\nNeural Machine Translation (NMT) to improve the modelling of tag sequentiality.\nSeq2seq models are known to learn complex global dependencies while currently\nproposed approaches using linear conditional random fields (CRF) only model\nlocal tag dependencies. In this work, we introduce a seq2seq model tailored for\nDA classification using: a hierarchical encoder, a novel guided attention\nmechanism and beam search applied to both training and inference. Compared to\nthe state of the art our model does not require handcrafted features and is\ntrained end-to-end. Furthermore, the proposed approach achieves an unmatched\naccuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on\nMRDA.",
            "journal ref": "AAAI 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08866v1",
            "date": "2020-02-20 17:06:27+00:00",
            "title": "Contextual Lensing of Universal Sentence Representations",
            "authors": [
                "Jamie Kiros"
            ],
            "abstract": "What makes a universal sentence encoder universal? The notion of a generic\nencoder of text appears to be at odds with the inherent contextualization and\nnon-permanence of language use in a dynamic world. However, mapping sentences\ninto generic fixed-length vectors for downstream similarity and retrieval tasks\nhas been fruitful, particularly for multilingual applications. How do we manage\nthis dilemma? In this work we propose Contextual Lensing, a methodology for\ninducing context-oriented universal sentence vectors. We break the construction\nof universal sentence vectors into a core, variable length, sentence matrix\nrepresentation equipped with an adaptable `lens' from which fixed-length\nvectors can be induced as a function of the lens context. We show that it is\npossible to focus notions of language similarity into a small number of lens\nparameters given a core universal matrix representation. For example, we\ndemonstrate the ability to encode translation similarity of sentences across\nseveral languages into a single weight matrix, even when the core encoder has\nnot seen parallel data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08901v1",
            "date": "2020-02-07 13:14:58+00:00",
            "title": "Identifying physical health comorbidities in a cohort of individuals with severe mental illness: An application of SemEHR",
            "authors": [
                "Rebecca Bendayan",
                "Honghan Wu",
                "Zeljko Kraljevic",
                "Robert Stewart",
                "Tom Searle",
                "Jaya Chaturvedi",
                "Jayati Das-Munshi",
                "Zina Ibrahim",
                "Aurelie Mascio",
                "Angus Roberts",
                "Daniel Bean",
                "Richard Dobson"
            ],
            "abstract": "Multimorbidity research in mental health services requires data from physical\nhealth conditions which is traditionally limited in mental health care\nelectronic health records. In this study, we aimed to extract data from\nphysical health conditions from clinical notes using SemEHR. Data was extracted\nfrom Clinical Record Interactive Search (CRIS) system at South London and\nMaudsley Biomedical Research Centre (SLaM BRC) and the cohort consisted of all\nindividuals who had received a primary or secondary diagnosis of severe mental\nillness between 2007 and 2018. Three pairs of annotators annotated 2403\ndocuments with an average Cohen's Kappa of 0.757. Results show that the NLP\nperformance varies across different diseases areas (F1 0.601 - 0.954)\nsuggesting that the language patterns or terminologies of different condition\ngroups entail different technical challenges to the same NLP task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08909v1",
            "date": "2020-02-10 18:40:59+00:00",
            "title": "REALM: Retrieval-Augmented Language Model Pre-Training",
            "authors": [
                "Kelvin Guu",
                "Kenton Lee",
                "Zora Tung",
                "Panupong Pasupat",
                "Ming-Wei Chang"
            ],
            "abstract": "Language model pre-training has been shown to capture a surprising amount of\nworld knowledge, crucial for NLP tasks such as question answering. However,\nthis knowledge is stored implicitly in the parameters of a neural network,\nrequiring ever-larger networks to cover more facts.\n  To capture knowledge in a more modular and interpretable way, we augment\nlanguage model pre-training with a latent knowledge retriever, which allows the\nmodel to retrieve and attend over documents from a large corpus such as\nWikipedia, used during pre-training, fine-tuning and inference. For the first\ntime, we show how to pre-train such a knowledge retriever in an unsupervised\nmanner, using masked language modeling as the learning signal and\nbackpropagating through a retrieval step that considers millions of documents.\n  We demonstrate the effectiveness of Retrieval-Augmented Language Model\npre-training (REALM) by fine-tuning on the challenging task of Open-domain\nQuestion Answering (Open-QA). We compare against state-of-the-art models for\nboth explicit and implicit knowledge storage on three popular Open-QA\nbenchmarks, and find that we outperform all previous methods by a significant\nmargin (4-16% absolute accuracy), while also providing qualitative benefits\nsuch as interpretability and modularity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.08911v1",
            "date": "2020-02-20 17:54:46+00:00",
            "title": "Measuring Social Biases in Grounded Vision and Language Embeddings",
            "authors": [
                "Candace Ross",
                "Boris Katz",
                "Andrei Barbu"
            ],
            "abstract": "We generalize the notion of social biases from language embeddings to\ngrounded vision and language embeddings. Biases are present in grounded\nembeddings, and indeed seem to be equally or more significant than for\nungrounded embeddings. This is despite the fact that vision and language can\nsuffer from different biases, which one might hope could attenuate the biases\nin both. Multiple ways exist to generalize metrics measuring bias in word\nembeddings to this new setting. We introduce the space of generalizations\n(Grounded-WEAT and Grounded-SEAT) and demonstrate that three generalizations\nanswer different yet important questions about how biases, language, and vision\ninteract. These metrics are used on a new dataset, the first for grounded bias,\ncreated by augmenting extending standard linguistic bias benchmarks with 10,228\nimages from COCO, Conceptual Captions, and Google Images. Dataset construction\nis challenging because vision datasets are themselves very biased. The presence\nof these biases in systems will begin to have real-world consequences as they\nare deployed, making carefully measuring bias and then mitigating it critical\nto building a fair society.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09127v4",
            "date": "2020-02-21 04:38:37+00:00",
            "title": "Learning Dynamic Belief Graphs to Generalize on Text-Based Games",
            "authors": [
                "Ashutosh Adhikari",
                "Xingdi Yuan",
                "Marc-Alexandre C\u00f4t\u00e9",
                "Mikul\u00e1\u0161 Zelinka",
                "Marc-Antoine Rondeau",
                "Romain Laroche",
                "Pascal Poupart",
                "Jian Tang",
                "Adam Trischler",
                "William L. Hamilton"
            ],
            "abstract": "Playing text-based games requires skills in processing natural language and\nsequential decision making. Achieving human-level performance on text-based\ngames remains an open challenge, and prior research has largely relied on\nhand-crafted structured representations and heuristics. In this work, we\ninvestigate how an agent can plan and generalize in text-based games using\ngraph-structured representations learned end-to-end from raw text. We propose a\nnovel graph-aided transformer agent (GATA) that infers and updates latent\nbelief graphs during planning to enable effective action selection by capturing\nthe underlying game dynamics. GATA is trained using a combination of\nreinforcement and self-supervised learning. Our work demonstrates that the\nlearned graph-based representations help agents converge to better policies\nthan their text-only counterparts and facilitate effective generalization\nacross game configurations. Experiments on 500+ unique games from the TextWorld\nsuite show that our best agent outperforms text-based baselines by an average\nof 24.2%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09213v1",
            "date": "2020-02-21 10:39:53+00:00",
            "title": "Refinement of Unsupervised Cross-Lingual Word Embeddings",
            "authors": [
                "Magdalena Biesialska",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "Cross-lingual word embeddings aim to bridge the gap between high-resource and\nlow-resource languages by allowing to learn multilingual word representations\neven without using any direct bilingual signal. The lion's share of the methods\nare projection-based approaches that map pre-trained embeddings into a shared\nlatent space. These methods are mostly based on the orthogonal transformation,\nwhich assumes language vector spaces to be isomorphic. However, this criterion\ndoes not necessarily hold, especially for morphologically-rich languages. In\nthis paper, we propose a self-supervised method to refine the alignment of\nunsupervised bilingual word embeddings. The proposed model moves vectors of\nwords and their corresponding translations closer to each other as well as\nenforces length- and center-invariance, thus allowing to better align\ncross-lingual embeddings. The experimental results demonstrate the\neffectiveness of our approach, as in most cases it outperforms state-of-the-art\nmethods in a bilingual lexicon induction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09361v1",
            "date": "2020-02-21 15:33:53+00:00",
            "title": "Crowdsourced Collective Entity Resolution with Relational Match Propagation",
            "authors": [
                "Jiacheng Huang",
                "Wei Hu",
                "Zhifeng Bao",
                "Yuzhong Qu"
            ],
            "abstract": "Knowledge bases (KBs) store rich yet heterogeneous entities and facts. Entity\nresolution (ER) aims to identify entities in KBs which refer to the same\nreal-world object. Recent studies have shown significant benefits of involving\nhumans in the loop of ER. They often resolve entities with pairwise similarity\nmeasures over attribute values and resort to the crowds to label uncertain\nones. However, existing methods still suffer from high labor costs and\ninsufficient labeling to some extent. In this paper, we propose a novel\napproach called crowdsourced collective ER, which leverages the relationships\nbetween entities to infer matches jointly rather than independently.\nSpecifically, it iteratively asks human workers to label picked entity pairs\nand propagates the labeling information to their neighbors in distance. During\nthis process, we address the problems of candidate entity pruning,\nprobabilistic propagation, optimal question selection and error-tolerant truth\ninference. Our experiments on real-world datasets demonstrate that, compared\nwith state-of-the-art methods, our approach achieves superior accuracy with\nmuch less labeling.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/2002.09599v1",
            "date": "2020-02-22 01:49:27+00:00",
            "title": "Training Question Answering Models From Synthetic Data",
            "authors": [
                "Raul Puri",
                "Ryan Spring",
                "Mostofa Patwary",
                "Mohammad Shoeybi",
                "Bryan Catanzaro"
            ],
            "abstract": "Question and answer generation is a data augmentation method that aims to\nimprove question answering (QA) models given the limited amount of human\nlabeled data. However, a considerable gap remains between synthetic and\nhuman-generated question-answer pairs. This work aims to narrow this gap by\ntaking advantage of large language models and explores several factors such as\nmodel size, quality of pretrained models, scale of data synthesized, and\nalgorithmic choices. On the SQuAD1.1 question answering task, we achieve higher\naccuracy using solely synthetic questions and answers than when using the\nSQuAD1.1 training set questions alone. Removing access to real Wikipedia data,\nwe synthesize questions and answers from a synthetic corpus generated by an 8.3\nbillion parameter GPT-2 model. With no access to human supervision and only\naccess to other models, we are able to train state of the art question\nanswering networks on entirely model-generated data that achieve 88.4 Exact\nMatch (EM) and 93.9 F1 score on the SQuAD1.1 dev set. We further apply our\nmethodology to SQuAD2.0 and show a 2.8 absolute gain on EM score compared to\nprior work using synthetic data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09604v1",
            "date": "2020-02-22 02:34:51+00:00",
            "title": "Emergent Communication with World Models",
            "authors": [
                "Alexander I. Cowen-Rivers",
                "Jason Naradowsky"
            ],
            "abstract": "We introduce Language World Models, a class of language-conditional\ngenerative model which interpret natural language messages by predicting latent\ncodes of future observations. This provides a visual grounding of the message,\nsimilar to an enhanced observation of the world, which may include objects\noutside of the listening agent's field-of-view. We incorporate this\n\"observation\" into a persistent memory state, and allow the listening agent's\npolicy to condition on it, akin to the relationship between memory and\ncontroller in a World Model. We show this improves effective communication and\ntask success in 2D gridworld speaker-listener navigation tasks. In addition, we\ndevelop two losses framed specifically for our model-based formulation to\npromote positive signalling and positive listening. Finally, because messages\nare interpreted in a generative model, we can visualize the model beliefs to\ngain insight into how the communication channel is utilized.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09877v1",
            "date": "2020-02-23 09:52:20+00:00",
            "title": "Automata for Hyperlanguages",
            "authors": [
                "Borzoo Bonakdarpour",
                "Sarai Sheinvald"
            ],
            "abstract": "Hyperproperties lift conventional trace properties from a set of execution\ntraces to a set of sets of execution traces. Hyperproperties have been shown to\nbe a powerful formalism for expressing and reasoning about information-flow\nsecurity policies and important properties of cyber-physical systems such as\nsensitivity and robustness, as well as consistency conditions in distributed\ncomputing such as linearizability. Although there is an extensive body of work\non automata-based representation of trace properties, we currently lack such\ncharacterization for hyperproperties. We introduce hyperautomata for em\nhyperlanguages, which are languages over sets of words. Essentially,\nhyperautomata allow running multiple quantified words over an automaton. We\npropose a specific type of hyperautomata called nondeterministic finite\nhyperautomata (NFH), which accept regular hyperlanguages. We demonstrate the\nability of regular hyperlanguages to express hyperproperties for finite traces.\nWe then explore the fundamental properties of NFH and show their closure under\nthe Boolean operations. We show that while nonemptiness is undecidable in\ngeneral, it is decidable for several fragments of NFH. We further show the\ndecidability of the membership problem for finite sets and regular languages\nfor NFH, as well as the containment problem for several fragments of NFH.\nFinally, we introduce learning algorithms based on Angluin's L-star algorithm\nfor the fragments NFH in which the quantification is either strictly universal\nor strictly existential.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09901v1",
            "date": "2020-02-23 13:33:04+00:00",
            "title": "A Nepali Rule Based Stemmer and its performance on different NLP applications",
            "authors": [
                "Pravesh Koirala",
                "Aman Shakya"
            ],
            "abstract": "Stemming is an integral part of Natural Language Processing (NLP). It's a\npreprocessing step in almost every NLP application. Arguably, the most\nimportant usage of stemming is in Information Retrieval (IR). While there are\nlots of work done on stemming in languages like English, Nepali stemming has\nonly a few works. This study focuses on creating a Rule Based stemmer for\nNepali text. Specifically, it is an affix stripping system that identifies two\ndifferent class of suffixes in Nepali grammar and strips them separately. Only\na single negativity prefix (Na) is identified and stripped. This study focuses\non a number of techniques like exception word identification, morphological\nnormalization and word transformation to increase stemming performance. The\nstemmer is tested intrinsically using Paice's method and extrinsically on a\nbasic tf-idf based IR system and an elementary news topic classifier using\nMultinomial Naive Bayes Classifier. The difference in performance of these\nsystems with and without using the stemmer is analysed.",
            "journal ref": "Proceedings of the 4th International IT Conference on ICT with\n  Smart Computing and 9th National Students' Conference on Information\n  Technology, (NaSCoIT 2018), Kathmandu, Nepal, ISSN No 2505-1075, pp. 16\n  (December 2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.09919v2",
            "date": "2020-02-23 15:16:43+00:00",
            "title": "Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?",
            "authors": [
                "Yixuan Tang",
                "Hwee Tou Ng",
                "Anthony K. H. Tung"
            ],
            "abstract": "Multi-hop question answering (QA) requires a model to retrieve and integrate\ninformation from different parts of a long text to answer a question. Humans\nanswer this kind of complex questions via a divide-and-conquer approach. In\nthis paper, we investigate whether top-performing models for multi-hop\nquestions understand the underlying sub-questions like humans. We adopt a\nneural decomposition model to generate sub-questions for a multi-hop complex\nquestion, followed by extracting the corresponding sub-answers. We show that\nmultiple state-of-the-art multi-hop QA models fail to correctly answer a large\nportion of sub-questions, although their corresponding multi-hop questions are\ncorrectly answered. This indicates that these models manage to answer the\nmulti-hop questions using some partial clues, instead of truly understanding\nthe reasoning paths. We also propose a new model which significantly improves\nthe performance on answering the sub-questions. Our work takes a step forward\ntowards building a more explainable multi-hop QA system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10116v1",
            "date": "2020-02-24 08:34:33+00:00",
            "title": "A Hybrid Approach to Dependency Parsing: Combining Rules and Morphology with Deep Learning",
            "authors": [
                "\u015eaziye Bet\u00fcl \u00d6zate\u015f",
                "Arzucan \u00d6zg\u00fcr",
                "Tunga G\u00fcng\u00f6r",
                "Balk\u0131z \u00d6zt\u00fcrk"
            ],
            "abstract": "Fully data-driven, deep learning-based models are usually designed as\nlanguage-independent and have been shown to be successful for many natural\nlanguage processing tasks. However, when the studied language is low-resourced\nand the amount of training data is insufficient, these models can benefit from\nthe integration of natural language grammar-based information. We propose two\napproaches to dependency parsing especially for languages with restricted\namount of training data. Our first approach combines a state-of-the-art deep\nlearning-based parser with a rule-based approach and the second one\nincorporates morphological information into the parser. In the rule-based\napproach, the parsing decisions made by the rules are encoded and concatenated\nwith the vector representations of the input words as additional information to\nthe deep network. The morphology-based approach proposes different methods to\ninclude the morphological structure of words into the parser network.\nExperiments are conducted on the IMST-UD Treebank and the results suggest that\nintegration of explicit knowledge about the target language to a neural parser\nthrough a rule-based parsing system and morphological analysis leads to more\naccurate annotations and hence, increases the parsing performance in terms of\nattachment scores. The proposed methods are developed for Turkish, but can be\nadapted to other languages as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10284v1",
            "date": "2020-02-06 23:45:50+00:00",
            "title": "Word Embeddings Inherently Recover the Conceptual Organization of the Human Mind",
            "authors": [
                "Victor Swift"
            ],
            "abstract": "Machine learning is a means to uncover deep patterns from rich sources of\ndata. Here, we find that machine learning can recover the conceptual\norganization of the human mind when applied to the natural language use of\nmillions of people. Utilizing text from billions of webpages, we recover most\nof the concepts contained in English, Dutch, and Japanese, as represented in\nlarge scale Word Association networks. Our results justify machine learning as\na means to probe the human mind, at a depth and scale that has been\nunattainable using self-report and observational methods. Beyond direct\npsychological applications, our methods may prove useful for projects concerned\nwith defining, assessing, relating, or uncovering concepts in any scientific\nfield.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10329v1",
            "date": "2020-02-24 15:57:41+00:00",
            "title": "KBSET -- Knowledge-Based Support for Scholarly Editing and Text Processing with Declarative LaTeX Markup and a Core Written in SWI-Prolog",
            "authors": [
                "Jana Kittelmann",
                "Christoph Wernhard"
            ],
            "abstract": "KBSET is an environment that provides support for scholarly editing in two\nflavors: First, as a practical tool KBSET/Letters that accompanies the\ndevelopment of editions of correspondences (in particular from the 18th and\n19th century), completely from source documents to PDF and HTML presentations.\nSecond, as a prototypical tool KBSET/NER for experimentally investigating novel\nforms of working on editions that are centered around automated named entity\nrecognition. KBSET can process declarative application-specific markup that is\nexpressed in LaTeX notation and incorporate large external fact bases that are\ntypically provided in RDF. KBSET includes specially developed LaTeX styles and\na core system that is written in SWI-Prolog, which is used there in many roles,\nutilizing that it realizes the potential of Prolog as a unifying language.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2002.10345v1",
            "date": "2020-02-24 16:17:12+00:00",
            "title": "Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation",
            "authors": [
                "Yige Xu",
                "Xipeng Qiu",
                "Ligao Zhou",
                "Xuanjing Huang"
            ],
            "abstract": "Fine-tuning pre-trained language models like BERT has become an effective way\nin NLP and yields state-of-the-art results on many downstream tasks. Recent\nstudies on adapting BERT to new tasks mainly focus on modifying the model\nstructure, re-designing the pre-train tasks, and leveraging external data and\nknowledge. The fine-tuning strategy itself has yet to be fully explored. In\nthis paper, we improve the fine-tuning of BERT with two effective mechanisms:\nself-ensemble and self-distillation. The experiments on text classification and\nnatural language inference tasks show our proposed methods can significantly\nimprove the adaption of BERT without any external data or knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10375v2",
            "date": "2020-02-24 17:07:32+00:00",
            "title": "Discriminative Adversarial Search for Abstractive Summarization",
            "authors": [
                "Thomas Scialom",
                "Paul-Alexis Dray",
                "Sylvain Lamprier",
                "Benjamin Piwowarski",
                "Jacopo Staiano"
            ],
            "abstract": "We introduce a novel approach for sequence decoding, Discriminative\nAdversarial Search (DAS), which has the desirable properties of alleviating the\neffects of exposure bias without requiring external metrics. Inspired by\nGenerative Adversarial Networks (GANs), wherein a discriminator is used to\nimprove the generator, our method differs from GANs in that the generator\nparameters are not updated at training time and the discriminator is only used\nto drive sequence generation at inference time.\n  We investigate the effectiveness of the proposed approach on the task of\nAbstractive Summarization: the results obtained show that a naive application\nof DAS improves over the state-of-the-art methods, with further gains obtained\nvia discriminator retraining. Moreover, we show how DAS can be effective for\ncross-domain adaptation. Finally, all results reported are obtained without\nadditional rule-based filtering strategies, commonly used by the best\nperforming systems available: this indicates that DAS can effectively be\ndeployed without relying on post-hoc modifications of the generated outputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10582v1",
            "date": "2020-02-24 23:07:38+00:00",
            "title": "Automating Discovery of Dominance in Synchronous Computer-Mediated Communication",
            "authors": [
                "Jim Samuel",
                "Richard Holowczak",
                "Raquel Benbunan-Fich",
                "Ilan Levine"
            ],
            "abstract": "With the advent of electronic interaction, dominance (or the assertion of\ncontrol over others) has acquired new dimensions. This study investigates the\ndynamics and characteristics of dominance in virtual interaction by analyzing\nelectronic chat transcripts of groups solving a hidden profile task. We\ninvestigate computer-mediated communication behavior patterns that demonstrate\ndominance and identify a number of relevant variables. These indicators are\ncalculated with automatic and manual coding of text transcripts. A comparison\nof both sets of variables indicates that automatic text analysis methods yield\nsimilar conclusions than manual coding. These findings are encouraging to\nadvance research in text analysis methods in general, and in the study of\nvirtual team dominance in particular.",
            "journal ref": "47th Hawaii International Conference on System Sciences, 2014, pp.\n  1804-1812",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2002.10640v1",
            "date": "2020-02-25 03:13:32+00:00",
            "title": "Differentiable Reasoning over a Virtual Knowledge Base",
            "authors": [
                "Bhuwan Dhingra",
                "Manzil Zaheer",
                "Vidhisha Balachandran",
                "Graham Neubig",
                "Ruslan Salakhutdinov",
                "William W. Cohen"
            ],
            "abstract": "We consider the task of answering complex multi-hop questions using a corpus\nas a virtual knowledge base (KB). In particular, we describe a neural module,\nDrKIT, that traverses textual data like a KB, softly following paths of\nrelations between mentions of entities in the corpus. At each step the module\nuses a combination of sparse-matrix TFIDF indices and a maximum inner product\nsearch (MIPS) on a special index of contextual representations of the mentions.\nThis module is differentiable, so the full system can be trained end-to-end\nusing gradient based methods, starting from natural language inputs. We also\ndescribe a pretraining scheme for the contextual representation encoder by\ngenerating hard negative examples using existing knowledge bases. We show that\nDrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset,\ncutting the gap between text-based and KB-based state-of-the-art by 70%. On\nHotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking\napproach to retrieving the relevant passages required to answer a question.\nDrKIT is also very efficient, processing 10-100x more queries per second than\nexisting multi-hop systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10665v1",
            "date": "2020-02-25 04:56:47+00:00",
            "title": "Declarative Memory-based Structure for the Representation of Text Data",
            "authors": [
                "Sumant Pushp",
                "Pragya Kashmira",
                "Shyamanta M Hazarika"
            ],
            "abstract": "In the era of intelligent computing, computational progress in text\nprocessing is an essential consideration. Many systems have been developed to\nprocess text over different languages. Though, there is considerable\ndevelopment, they still lack in understanding of the text, i.e., instead of\nkeeping text as knowledge, many treat text as a data. In this work we introduce\na text representation scheme which is influenced by human memory\ninfrastructure. Since texts are declarative in nature, a structural\norganization would foster efficient computation over text. We exploit long term\nepisodic memory to keep text information observed over time. This not only keep\nfragments of text in an organized fashion but also reduces redundancy and\nstores the temporal relation among them. Wordnet has been used to imitate\nsemantic memory, which works at word level to facilitate the understanding\nabout individual words within text. Experimental results of various operation\nperformed over episodic memory and growth of knowledge infrastructure over time\nis reported.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2002.10772v1",
            "date": "2020-02-25 10:05:56+00:00",
            "title": "Label-guided Learning for Text Classification",
            "authors": [
                "Xien Liu",
                "Song Wang",
                "Xiao Zhang",
                "Xinxin You",
                "Ji Wu",
                "Dejing Dou"
            ],
            "abstract": "Text classification is one of the most important and fundamental tasks in\nnatural language processing. Performance of this task mainly dependents on text\nrepresentation learning. Currently, most existing learning frameworks mainly\nfocus on encoding local contextual information between words. These methods\nalways neglect to exploit global clues, such as label information, for encoding\ntext information. In this study, we propose a label-guided learning framework\nLguidedLearn for text representation and classification. Our method is novel\nbut simple that we only insert a label-guided encoding layer into the commonly\nused text representation learning schemas. That label-guided layer performs\nlabel-based attentive encoding to map the universal text embedding (encoded by\na contextual information learner) into different label spaces, resulting in\nlabel-wise embeddings. In our proposed framework, the label-guided layer can be\neasily and directly applied with a contextual encoding method to perform\njointly learning. Text information is encoded based on both the local\ncontextual information and the global label clues. Therefore, the obtained text\nembeddings are more robust and discriminative for text classification.\nExtensive experiments are conducted on benchmark datasets to illustrate the\neffectiveness of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.10782v2",
            "date": "2020-02-25 10:36:17+00:00",
            "title": "Abstractive Snippet Generation",
            "authors": [
                "Wei-Fan Chen",
                "Shahbaz Syed",
                "Benno Stein",
                "Matthias Hagen",
                "Martin Potthast"
            ],
            "abstract": "An abstractive snippet is an originally created piece of text to summarize a\nweb page on a search engine results page. Compared to the conventional\nextractive snippets, which are generated by extracting phrases and sentences\nverbatim from a web page, abstractive snippets circumvent copyright issues;\neven more interesting is the fact that they open the door for personalization.\nAbstractive snippets have been evaluated as equally powerful in terms of user\nacceptance and expressiveness---but the key question remains: Can abstractive\nsnippets be automatically generated with sufficient quality?\n  This paper introduces a new approach to abstractive snippet generation: We\nidentify the first two large-scale sources for distant supervision, namely\nanchor contexts and web directories. By mining the entire ClueWeb09 and\nClueWeb12 for anchor contexts and by utilizing the DMOZ Open Directory Project,\nwe compile the Webis Abstractive Snippet Corpus 2020, comprising more than 3.5\nmillion triples of the form $\\langle$query, snippet, document$\\rangle$ as\ntraining examples, where the snippet is either an anchor context or a web\ndirectory description in lieu of a genuine query-biased abstractive snippet of\nthe web document. We propose a bidirectional abstractive snippet generation\nmodel and assess the quality of both our corpus and the generated abstractive\nsnippets with standard measures, crowdsourcing, and in comparison to the state\nof the art. The evaluation shows that our novel data sources along with the\nproposed model allow for producing usable query-biased abstractive snippets\nwhile minimizing text reuse.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2002.11296v1",
            "date": "2020-02-26 04:18:01+00:00",
            "title": "Sparse Sinkhorn Attention",
            "authors": [
                "Yi Tay",
                "Dara Bahri",
                "Liu Yang",
                "Donald Metzler",
                "Da-Cheng Juan"
            ],
            "abstract": "We propose Sparse Sinkhorn Attention, a new efficient and sparse method for\nlearning to attend. Our method is based on differentiable sorting of internal\nrepresentations. Concretely, we introduce a meta sorting network that learns to\ngenerate latent permutations over sequences. Given sorted sequences, we are\nthen able to compute quasi-global attention with only local windows, improving\nthe memory efficiency of the attention module. To this end, we propose new\nalgorithmic innovations such as Causal Sinkhorn Balancing and SortCut, a\ndynamic sequence truncation method for tailoring Sinkhorn Attention for\nencoding and/or decoding purposes. Via extensive experiments on algorithmic\nseq2seq sorting, language modeling, pixel-wise image generation, document\nclassification and natural language inference, we demonstrate that our memory\nefficient Sinkhorn Attention method is competitive with vanilla attention and\nconsistently outperforms recently proposed efficient Transformer models such as\nSparse Transformers.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2002.11566v1",
            "date": "2020-02-26 15:34:52+00:00",
            "title": "Object Relational Graph with Teacher-Recommended Learning for Video Captioning",
            "authors": [
                "Ziqi Zhang",
                "Yaya Shi",
                "Chunfeng Yuan",
                "Bing Li",
                "Peijin Wang",
                "Weiming Hu",
                "Zhengjun Zha"
            ],
            "abstract": "Taking full advantage of the information from both vision and language is\ncritical for the video captioning task. Existing models lack adequate visual\nrepresentation due to the neglect of interaction between object, and sufficient\ntraining for content-related words due to long-tailed problems. In this paper,\nwe propose a complete video captioning system including both a novel model and\nan effective training strategy. Specifically, we propose an object relational\ngraph (ORG) based encoder, which captures more detailed interaction features to\nenrich visual representation. Meanwhile, we design a teacher-recommended\nlearning (TRL) method to make full use of the successful external language\nmodel (ELM) to integrate the abundant linguistic knowledge into the caption\nmodel. The ELM generates more semantically similar word proposals which extend\nthe ground-truth words used for training to deal with the long-tailed problem.\nExperimental evaluations on three benchmarks: MSVD, MSR-VTT and VATEX show the\nproposed ORG-TRL system achieves state-of-the-art performance. Extensive\nablation studies and visualizations illustrate the effectiveness of our system.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2002.11623v2",
            "date": "2020-02-14 09:51:24+00:00",
            "title": "Trends of digitalization and adoption of big data & analytics among UK SMEs: Analysis and lessons drawn from a case study of 53 SMEs",
            "authors": [
                "Muhidin Mohamed",
                "Philip Weber"
            ],
            "abstract": "Small and Medium Enterprises (SMEs) now generate digital data at an\nunprecedented rate from online transactions, social media marketing and\nassociated customer interactions, online product or service reviews and\nfeedback, clinical diagnosis, Internet of Things (IoT) sensors, and production\nprocesses. All these forms of data can be transformed into monetary value if\nput into a proper data value chain. This requires both skills and IT\ninvestments for the long-term benefit of businesses. However, such spending is\nbeyond the capacity of most SMEs due to their limited resources and restricted\naccess to finances. This paper presents lessons learned from a case study of 53\nUK SMEs, mostly from the West Midlands region of England, supported as part of\na 3-year ERDF project, Big Data Corridor, in the areas of big data management,\nanalytics and related IT issues. Based on our study's sample companies, several\nperspectives including the digital technology trends, challenges facing the UK\nSMEs, and the state of their adoption in data analytics and big data, are\npresented in the paper.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2002.11768v3",
            "date": "2020-02-19 04:18:45+00:00",
            "title": "Attacking Neural Text Detectors",
            "authors": [
                "Max Wolff",
                "Stuart Wolff"
            ],
            "abstract": "Machine learning based language models have recently made significant\nprogress, which introduces a danger to spread misinformation. To combat this\npotential danger, several methods have been proposed for detecting text written\nby these language models. This paper presents two classes of black-box attacks\non these detectors, one which randomly replaces characters with homoglyphs, and\nthe other a simple scheme to purposefully misspell words. The homoglyph and\nmisspelling attacks decrease a popular neural text detector's recall on neural\ntext from 97.44% to 0.26% and 22.68%, respectively. Results also indicate that\nthe attacks are transferable to other neural text detectors.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/2002.11794v2",
            "date": "2020-02-26 21:17:13+00:00",
            "title": "Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers",
            "authors": [
                "Zhuohan Li",
                "Eric Wallace",
                "Sheng Shen",
                "Kevin Lin",
                "Kurt Keutzer",
                "Dan Klein",
                "Joseph E. Gonzalez"
            ],
            "abstract": "Since hardware resources are limited, the objective of training deep learning\nmodels is typically to maximize accuracy subject to the time and memory\nconstraints of training and inference. We study the impact of model size in\nthis setting, focusing on Transformer models for NLP tasks that are limited by\ncompute: self-supervised pretraining and high-resource machine translation. We\nfirst show that even though smaller Transformer models execute faster per\niteration, wider and deeper models converge in significantly fewer steps.\nMoreover, this acceleration in convergence typically outpaces the additional\ncomputational overhead of using larger models. Therefore, the most\ncompute-efficient training strategy is to counterintuitively train extremely\nlarge models but stop after a small number of iterations.\n  This leads to an apparent trade-off between the training efficiency of large\nTransformer models and the inference efficiency of small Transformer models.\nHowever, we show that large models are more robust to compression techniques\nsuch as quantization and pruning than small models. Consequently, one can get\nthe best of both worlds: heavily compressed, large models achieve higher\naccuracy than lightly compressed, small models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.11848v1",
            "date": "2020-02-27 00:09:25+00:00",
            "title": "Analysis of diversity-accuracy tradeoff in image captioning",
            "authors": [
                "Ruotian Luo",
                "Gregory Shakhnarovich"
            ],
            "abstract": "We investigate the effect of different model architectures, training\nobjectives, hyperparameter settings and decoding procedures on the diversity of\nautomatically generated image captions. Our results show that 1) simple\ndecoding by naive sampling, coupled with low temperature is a competitive and\nfast method to produce diverse and accurate caption sets; 2) training with\nCIDEr-based reward using Reinforcement learning harms the diversity properties\nof the resulting generator, which cannot be mitigated by manipulating decoding\nparameters. In addition, we propose a new metric AllSPICE for evaluating both\naccuracy and diversity of a set of captions by a single value.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12005v2",
            "date": "2020-02-27 09:50:41+00:00",
            "title": "Squashed Shifted PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces",
            "authors": [
                "Zhenisbek Assylbekov",
                "Alibi Jangeldin"
            ],
            "abstract": "We show that removing sigmoid transformation in the skip-gram with negative\nsampling (SGNS) objective does not harm the quality of word vectors\nsignificantly and at the same time is related to factorizing a squashed shifted\nPMI matrix which, in turn, can be treated as a connection probabilities matrix\nof a random graph. Empirically, such graph is a complex network, i.e. it has\nstrong clustering and scale-free degree distribution, and is tightly connected\nwith hyperbolic spaces. In short, we show the connection between static word\nembeddings and hyperbolic spaces through the squashed shifted PMI matrix using\nanalytical and empirical methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12196v3",
            "date": "2020-02-27 15:42:39+00:00",
            "title": "Annotation of Emotion Carriers in Personal Narratives",
            "authors": [
                "Aniruddha Tammewar",
                "Alessandra Cervone",
                "Eva-Maria Messner",
                "Giuseppe Riccardi"
            ],
            "abstract": "We are interested in the problem of understanding personal narratives (PN) -\nspoken or written - recollections of facts, events, and thoughts. In PN,\nemotion carriers are the speech or text segments that best explain the\nemotional state of the user. Such segments may include entities, verb or noun\nphrases. Advanced automatic understanding of PNs requires not only the\nprediction of the user emotional state but also to identify which events (e.g.\n\"the loss of relative\" or \"the visit of grandpa\") or people ( e.g. \"the old\ngroup of high school mates\") carry the emotion manifested during the personal\nrecollection. This work proposes and evaluates an annotation model for\nidentifying emotion carriers in spoken personal narratives. Compared to other\ntext genres such as news and microblogs, spoken PNs are particularly\nchallenging because a narrative is usually unstructured, involving multiple\nsub-events and characters as well as thoughts and associated emotions perceived\nby the narrator. In this work, we experiment with annotating emotion carriers\nfrom speech transcriptions in the Ulm State-of-Mind in Speech (USoMS) corpus, a\ndataset of German PNs. We believe this resource could be used for experiments\nin the automatic extraction of emotion carriers from PN, a task that could\nprovide further advancements in narrative understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12457v1",
            "date": "2020-02-27 21:44:41+00:00",
            "title": "Comment Ranking Diversification in Forum Discussions",
            "authors": [
                "Curtis G. Northcutt",
                "Kimberly A. Leon",
                "Naichun Chen"
            ],
            "abstract": "Viewing consumption of discussion forums with hundreds or more comments\ndepends on ranking because most users only view top-ranked comments. When\ncomments are ranked by an ordered score (e.g. number of replies or up-votes)\nwithout adjusting for semantic similarity of near-ranked comments, top-ranked\ncomments are more likely to emphasize the majority opinion and incur\nredundancy. In this paper, we propose a top K comment diversification\nre-ranking model using Maximal Marginal Relevance (MMR) and evaluate its impact\nin three categories: (1) semantic diversity, (2) inclusion of the semantics of\nlower-ranked comments, and (3) redundancy, within the context of a HarvardX\ncourse discussion forum. We conducted a double-blind, small-scale evaluation\nexperiment requiring subjects to select between the top 5 comments of a\ndiversified ranking and a baseline ranking ordered by score. For three\nsubjects, across 100 trials, subjects selected the diversified (75% score, 25%\ndiversification) ranking as significantly (1) more diverse, (2) more inclusive,\nand (3) less redundant. Within each category, inter-rater reliability showed\nmoderate consistency, with typical Cohen-Kappa scores near 0.2. Our findings\nsuggest that our model improves (1) diversification, (2) inclusion, and (3)\nredundancy, among top K ranked comments in online discussion forums.",
            "journal ref": "Proceedings of the Sixth {ACM} Conference on Learning @ Scale, L@S\n  2017",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2002.12530v2",
            "date": "2020-02-28 03:53:31+00:00",
            "title": "Temporal Convolutional Attention-based Network For Sequence Modeling",
            "authors": [
                "Hongyan Hao",
                "Yan Wang",
                "Yudi Xia",
                "Jian Zhao",
                "Furao Shen"
            ],
            "abstract": "With the development of feed-forward models, the default model for sequence\nmodeling has gradually evolved to replace recurrent networks. Many powerful\nfeed-forward models based on convolutional networks and attention mechanism\nwere proposed and show more potential to handle sequence modeling tasks. We\nwonder that is there an architecture that can not only achieve an approximate\nsubstitution of recurrent network, but also absorb the advantages of\nfeed-forward models. So we propose an exploratory architecture referred to\nTemporal Convolutional Attention-based Network (TCAN) which combines temporal\nconvolutional network and attention mechanism. TCAN includes two parts, one is\nTemporal Attention (TA) which captures relevant features inside the sequence,\nthe other is Enhanced Residual (ER) which extracts shallow layer's important\ninformation and transfers to deep layers. We improve the state-of-the-art\nresults of bpc/perplexity to 26.92 on word-level PTB, 1.043 on character-level\nPTB, and 6.66 on WikiText-2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2002.12798v1",
            "date": "2020-02-27 05:06:19+00:00",
            "title": "Optimizing Memory-Access Patterns for Deep Learning Accelerators",
            "authors": [
                "Hongbin Zheng",
                "Sejong Oh",
                "Huiqing Wang",
                "Preston Briggs",
                "Jiading Gai",
                "Animesh Jain",
                "Yizhi Liu",
                "Rich Heaton",
                "Randy Huang",
                "Yida Wang"
            ],
            "abstract": "Deep learning (DL) workloads are moving towards accelerators for faster\nprocessing and lower cost. Modern DL accelerators are good at handling the\nlarge-scale multiply-accumulate operations that dominate DL workloads; however,\nit is challenging to make full use of the compute power of an accelerator since\nthe data must be properly staged in a software-managed scratchpad memory.\nFailing to do so can result in significant performance loss. This paper\nproposes a systematic approach which leverages the polyhedral model to analyze\nall operators of a DL model together to minimize the number of memory accesses.\nExperiments show that our approach can substantially reduce the impact of\nmemory accesses required by common neural-network models on a homegrown AWS\nmachine-learning inference chip named Inferentia, which is available through\nAmazon EC2 Inf1 instances.",
            "journal ref": "None",
            "category": "cs.PF"
        },
        {
            "url": "http://arxiv.org/abs/2002.12867v1",
            "date": "2020-02-28 17:05:55+00:00",
            "title": "Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Noe Casas",
                "Eneko Agirre"
            ],
            "abstract": "Back-translation provides a simple yet effective approach to exploit\nmonolingual corpora in Neural Machine Translation (NMT). Its iterative variant,\nwhere two opposite NMT models are jointly trained by alternately using a\nsynthetic parallel corpus generated by the reverse model, plays a central role\nin unsupervised machine translation. In order to start producing sound\ntranslations and provide a meaningful training signal to each other, existing\napproaches rely on either a separate machine translation system to warm up the\niterative procedure, or some form of pre-training to initialize the weights of\nthe model. In this paper, we analyze the role that such initialization plays in\niterative back-translation. Is the behavior of the final system heavily\ndependent on it? Or does iterative back-translation converge to a similar\nsolution given any reasonable initialization? Through a series of empirical\nexperiments over a diverse set of warmup systems, we show that, although the\nquality of the initial system does affect final performance, its effect is\nrelatively small, as iterative back-translation has a strong tendency to\nconvergence to a similar solution. As such, the margin of improvement left for\nthe initialization method is narrow, suggesting that future research should\nfocus more on improving the iterative mechanism itself.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.00739v1",
            "date": "2020-03-02 10:03:14+00:00",
            "title": "Long Short-Term Sample Distillation",
            "authors": [
                "Liang Jiang",
                "Zujie Wen",
                "Zhongping Liang",
                "Yafang Wang",
                "Gerard de Melo",
                "Zhe Li",
                "Liangzhuang Ma",
                "Jiaxing Zhang",
                "Xiaolong Li",
                "Yuan Qi"
            ],
            "abstract": "In the past decade, there has been substantial progress at training\nincreasingly deep neural networks. Recent advances within the teacher--student\ntraining paradigm have established that information about past training updates\nshow promise as a source of guidance during subsequent training steps. Based on\nthis notion, in this paper, we propose Long Short-Term Sample Distillation, a\nnovel training policy that simultaneously leverages multiple phases of the\nprevious training process to guide the later training updates to a neural\nnetwork, while efficiently proceeding in just one single generation pass. With\nLong Short-Term Sample Distillation, the supervision signal for each sample is\ndecomposed into two parts: a long-term signal and a short-term one. The\nlong-term teacher draws on snapshots from several epochs ago in order to\nprovide steadfast guidance and to guarantee teacher--student differences, while\nthe short-term one yields more up-to-date cues with the goal of enabling\nhigher-quality updates. Moreover, the teachers for each sample are unique, such\nthat, overall, the model learns from a very diverse set of teachers.\nComprehensive experimental results across a range of vision and NLP tasks\ndemonstrate the effectiveness of this new training method.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.00744v3",
            "date": "2020-03-02 10:21:17+00:00",
            "title": "PhoBERT: Pre-trained language models for Vietnamese",
            "authors": [
                "Dat Quoc Nguyen",
                "Anh Tuan Nguyen"
            ],
            "abstract": "We present PhoBERT with two versions, PhoBERT-base and PhoBERT-large, the\nfirst public large-scale monolingual language models pre-trained for\nVietnamese. Experimental results show that PhoBERT consistently outperforms the\nrecent best pre-trained multilingual model XLM-R (Conneau et al., 2020) and\nimproves the state-of-the-art in multiple Vietnamese-specific NLP tasks\nincluding Part-of-speech tagging, Dependency parsing, Named-entity recognition\nand Natural language inference. We release PhoBERT to facilitate future\nresearch and downstream applications for Vietnamese NLP. Our PhoBERT models are\navailable at https://github.com/VinAIResearch/PhoBERT",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01338v1",
            "date": "2020-03-03 05:10:13+00:00",
            "title": "Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion",
            "authors": [
                "Jingyuan Yang",
                "Guang Liu",
                "Yuzhao Mao",
                "Zhiwei Zhao",
                "Weiguo Gao",
                "Xuan Li",
                "Haiqin Yang",
                "Jianping Shen"
            ],
            "abstract": "Task 1 of the DSTC8-track1 challenge aims to develop an end-to-end\nmulti-domain dialogue system to accomplish complex users' goals under tourist\ninformation desk settings. This paper describes our submitted solution,\nHierarchical Context Enhanced Dialogue System (HCEDS), for this task. The main\nmotivation of our system is to comprehensively explore the potential of\nhierarchical context for sufficiently understanding complex dialogues. More\nspecifically, we apply BERT to capture token-level information and employ the\nattention mechanism to capture sentence-level information. The results listed\nin the leaderboard show that our system achieves first place in automatic\nevaluation and the second place in human evaluation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.01345v1",
            "date": "2020-03-03 05:49:55+00:00",
            "title": "Benchmark Performance of Machine And Deep Learning Based Methodologies for Urdu Text Document Classification",
            "authors": [
                "Muhammad Nabeel Asim",
                "Muhammad Usman Ghani",
                "Muhammad Ali Ibrahim",
                "Sheraz Ahmad",
                "Waqar Mahmood",
                "Andreas Dengel"
            ],
            "abstract": "In order to provide benchmark performance for Urdu text document\nclassification, the contribution of this paper is manifold. First, it pro-vides\na publicly available benchmark dataset manually tagged against 6 classes.\nSecond, it investigates the performance impact of traditional machine learning\nbased Urdu text document classification methodologies by embedding 10\nfilter-based feature selection algorithms which have been widely used for other\nlanguages. Third, for the very first time, it as-sesses the performance of\nvarious deep learning based methodologies for Urdu text document\nclassification. In this regard, for experimentation, we adapt 10 deep learning\nclassification methodologies which have pro-duced best performance figures for\nEnglish text classification. Fourth, it also investigates the performance\nimpact of transfer learning by utiliz-ing Bidirectional Encoder Representations\nfrom Transformers approach for Urdu language. Fifth, it evaluates the integrity\nof a hybrid approach which combines traditional machine learning based feature\nengineering and deep learning based automated feature engineering. Experimental\nresults show that feature selection approach named as Normalised Dif-ference\nMeasure along with Support Vector Machine outshines state-of-the-art\nperformance on two closed source benchmark datasets CLE Urdu Digest 1000k, and\nCLE Urdu Digest 1Million with a significant margin of 32%, and 13%\nrespectively. Across all three datasets, Normalised Differ-ence Measure\noutperforms other filter based feature selection algorithms as it significantly\nuplifts the performance of all adopted machine learning, deep learning, and\nhybrid approaches. The source code and presented dataset are available at\nGithub repository.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.01797v1",
            "date": "2020-03-03 21:13:12+00:00",
            "title": "Discover Your Social Identity from What You Tweet: a Content Based Approach",
            "authors": [
                "Binxuan Huang",
                "Kathleen M. Carley"
            ],
            "abstract": "An identity denotes the role an individual or a group plays in highly\ndifferentiated contemporary societies. In this paper, our goal is to classify\nTwitter users based on their role identities. We first collect a coarse-grained\npublic figure dataset automatically, then manually label a more fine-grained\nidentity dataset. We propose a hierarchical self-attention neural network for\nTwitter user role identity classification. Our experiments demonstrate that the\nproposed model significantly outperforms multiple baselines. We further propose\na transfer learning scheme that improves our model's performance by a large\nmargin. Such transfer learning also greatly reduces the need for a large amount\nof human labeled data.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2003.01857v1",
            "date": "2020-03-04 02:00:57+00:00",
            "title": "SeMemNN: A Semantic Matrix-Based Memory Neural Network for Text Classification",
            "authors": [
                "Changzeng Fu",
                "Chaoran Liu",
                "Carlos Toshinori Ishi",
                "Yuichiro Yoshikawa",
                "Hiroshi Ishiguro"
            ],
            "abstract": "Text categorization is the task of assigning labels to documents written in a\nnatural language, and it has numerous real-world applications including\nsentiment analysis as well as traditional topic assignment tasks. In this\npaper, we propose 5 different configurations for the semantic matrix-based\nmemory neural network with end-to-end learning manner and evaluate our proposed\nmethod on two corpora of news articles (AG news, Sogou news). The best\nperformance of our proposed method outperforms the baseline VDCNN models on the\ntext classification task and gives a faster speed for learning semantics.\nMoreover, we also evaluate our model on small scale datasets. The results show\nthat our proposed method can still achieve better results in comparison to\nVDCNN on the small scale dataset. This paper is to appear in the Proceedings of\nthe 2020 IEEE 14th International Conference on Semantic Computing (ICSC 2020),\nSan Diego, California, 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02245v2",
            "date": "2020-03-04 18:35:19+00:00",
            "title": "Data Augmentation using Pre-trained Transformer Models",
            "authors": [
                "Varun Kumar",
                "Ashutosh Choudhary",
                "Eunah Cho"
            ],
            "abstract": "Language model based pre-trained models such as BERT have provided\nsignificant gains across different NLP tasks. In this paper, we study different\ntypes of transformer based pre-trained models such as auto-regressive models\n(GPT-2), auto-encoder models (BERT), and seq2seq models (BART) for conditional\ndata augmentation. We show that prepending the class labels to text sequences\nprovides a simple yet effective way to condition the pre-trained models for\ndata augmentation. Additionally, on three classification benchmarks,\npre-trained Seq2Seq model outperforms other data augmentation methods in a\nlow-resource setting. Further, we explore how different pre-trained model based\ndata augmentation differs in-terms of data diversity, and how well such methods\npreserve the class-label information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02599v2",
            "date": "2020-03-05 13:22:23+00:00",
            "title": "An Incremental Explanation of Inference in Hybrid Bayesian Networks for Increasing Model Trustworthiness and Supporting Clinical Decision Making",
            "authors": [
                "Evangelia Kyrimi",
                "Somayyeh Mossadegh",
                "Nigel Tai",
                "William Marsh"
            ],
            "abstract": "Various AI models are increasingly being considered as part of clinical\ndecision-support tools. However, the trustworthiness of such models is rarely\nconsidered. Clinicians are more likely to use a model if they can understand\nand trust its predictions. Key to this is if its underlying reasoning can be\nexplained. A Bayesian network (BN) model has the advantage that it is not a\nblack-box and its reasoning can be explained. In this paper, we propose an\nincremental explanation of inference that can be applied to hybrid BNs, i.e.\nthose that contain both discrete and continuous nodes. The key questions that\nwe answer are: (1) which important evidence supports or contradicts the\nprediction, and (2) through which intermediate variables does the information\nflow. The explanation is illustrated using a real clinical case study. A small\nevaluation study is also conducted.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.02639v1",
            "date": "2020-03-04 15:07:03+00:00",
            "title": "Phase transitions in a decentralized graph-based approach to human language",
            "authors": [
                "Javier Vera",
                "Felipe Urbina",
                "Wenceslao Palma"
            ],
            "abstract": "Zipf's law establishes a scaling behavior for word-frequencies in large text\ncorpora. The appearance of Zipfian properties in human language has been\npreviously explained as an optimization problem for the interests of speakers\nand hearers. On the other hand, human-like vocabularies can be viewed as\nbipartite graphs. The aim here is double: within a bipartite-graph approach to\nhuman vocabularies, to propose a decentralized language game model for the\nformation of Zipfian properties. To do this, we define a language game, in\nwhich a population of artificial agents is involved in idealized linguistic\ninteractions. Numerical simulations show the appearance of a phase transition\nfrom an initially disordered state to three possible phases for language\nformation. Our results suggest that Zipfian properties in language seem to\narise partly from decentralized linguistic interactions between agents endowed\nwith bipartite word-meaning mappings.",
            "journal ref": "Phys. Rev. E 103, 022129 (2021)",
            "category": "physics.soc-ph"
        },
        {
            "url": "http://arxiv.org/abs/2003.02736v2",
            "date": "2020-03-05 16:06:07+00:00",
            "title": "Claim Check-Worthiness Detection as Positive Unlabelled Learning",
            "authors": [
                "Dustin Wright",
                "Isabelle Augenstein"
            ],
            "abstract": "As the first step of automatic fact checking, claim check-worthiness\ndetection is a critical component of fact checking systems. There are multiple\nlines of research which study this problem: check-worthiness ranking from\npolitical speeches and debates, rumour detection on Twitter, and citation\nneeded detection from Wikipedia. To date, there has been no structured\ncomparison of these various tasks to understand their relatedness, and no\ninvestigation into whether or not a unified approach to all of them is\nachievable. In this work, we illuminate a central challenge in claim\ncheck-worthiness detection underlying all of these tasks, being that they hinge\nupon detecting both how factual a sentence is, as well as how likely a sentence\nis to be believed without verification. As such, annotators only mark those\ninstances they judge to be clear-cut check-worthy. Our best performing method\nis a unified approach which automatically corrects for this using a variant of\npositive unlabelled learning that finds instances which were incorrectly\nlabelled as not check-worthy. In applying this, we out-perform the state of the\nart in two of the three tasks studied for claim check-worthiness detection in\nEnglish.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.02958v1",
            "date": "2020-03-05 23:09:24+00:00",
            "title": "EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems",
            "authors": [
                "Rohola Zandie",
                "Mohammad H. Mahoor"
            ],
            "abstract": "Understanding emotions and responding accordingly is one of the biggest\nchallenges of dialog systems. This paper presents EmpTransfo, a multi-head\nTransformer architecture for creating an empathetic dialog system. EmpTransfo\nutilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) for language\ngeneration, though models with different sizes can be used. We show that\nutilizing the history of emotions and other metadata can improve the quality of\ngenerated conversations by the dialog system. Our experimental results using a\nchallenging language corpus show that the proposed approach outperforms other\nmodels in terms of Hit@1 and PPL (Perplexity).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03069v1",
            "date": "2020-03-06 08:18:13+00:00",
            "title": "Parsing Thai Social Data: A New Challenge for Thai NLP",
            "authors": [
                "Sattaya Singkul",
                "Borirat Khampingyot",
                "Nattasit Maharattamalai",
                "Supawat Taerungruang",
                "Tawunrat Chalothorn"
            ],
            "abstract": "Dependency parsing (DP) is a task that analyzes text for syntactic structure\nand relationship between words. DP is widely used to improve natural language\nprocessing (NLP) applications in many languages such as English. Previous works\non DP are generally applicable to formally written languages. However, they do\nnot apply to informal languages such as the ones used in social networks.\nTherefore, DP has to be researched and explored with such social network data.\nIn this paper, we explore and identify a DP model that is suitable for Thai\nsocial network data. After that, we will identify the appropriate linguistic\nunit as an input. The result showed that, the transition based model called,\nimprove Elkared dependency parser outperform the others at UAS of 81.42%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03072v1",
            "date": "2020-03-06 08:29:37+00:00",
            "title": "Improving Neural Named Entity Recognition with Gazetteers",
            "authors": [
                "Chan Hee Song",
                "Dawn Lawrie",
                "Tim Finin",
                "James Mayfield"
            ],
            "abstract": "The goal of this work is to improve the performance of a neural named entity\nrecognition system by adding input features that indicate a word is part of a\nname included in a gazetteer. This article describes how to generate gazetteers\nfrom the Wikidata knowledge graph as well as how to integrate the information\ninto a neural NER system. Experiments reveal that the approach yields\nperformance gains in two distinct languages: a high-resource, word-based\nlanguage, English and a high-resource, character-based language, Chinese.\nExperiments were also performed in a low-resource language, Russian on a newly\nannotated Russian NER corpus from Reddit tagged with four core types and twelve\nextended types. This article reports a baseline score. It is a longer version\nof a paper in the 33rd FLAIRS conference (Song et al. 2020).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03204v2",
            "date": "2020-03-06 13:47:30+00:00",
            "title": "Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing?",
            "authors": [
                "Houquan Zhou",
                "Yu Zhang",
                "Zhenghua Li",
                "Min Zhang"
            ],
            "abstract": "In the pre deep learning era, part-of-speech tags have been considered as\nindispensable ingredients for feature engineering in dependency parsing. But\nquite a few works focus on joint tagging and parsing models to avoid error\npropagation. In contrast, recent studies suggest that POS tagging becomes much\nless important or even useless for neural parsing, especially when using\ncharacter-based word representations. Yet there are not enough investigations\nfocusing on this issue, both empirically and linguistically. To answer this, we\ndesign and compare three typical multi-task learning framework, i.e.,\nShare-Loose, Share-Tight, and Stack, for joint tagging and parsing based on the\nstate-of-the-art biaffine parser. Considering that it is much cheaper to\nannotate POS tags than parse trees, we also investigate the utilization of\nlarge-scale heterogeneous POS tag data. We conduct experiments on both English\nand Chinese datasets, and the results clearly show that POS tagging (both\nhomogeneous and heterogeneous) can still significantly improve parsing\nperformance when using the Stack joint framework. We conduct detailed analysis\nand gain more insights from the linguistic aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03239v2",
            "date": "2020-03-06 14:35:20+00:00",
            "title": "On the Role of Conceptualization in Commonsense Knowledge Graph Construction",
            "authors": [
                "Mutian He",
                "Yangqiu Song",
                "Kun Xu",
                "Dong Yu"
            ],
            "abstract": "Commonsense knowledge graphs (CKGs) like Atomic and ASER are substantially\ndifferent from conventional KGs as they consist of much larger number of nodes\nformed by loosely-structured text, which, though, enables them to handle highly\ndiverse queries in natural language related to commonsense, leads to unique\nchallenges for automatic KG construction methods. Besides identifying relations\nabsent from the KG between nodes, such methods are also expected to explore\nabsent nodes represented by text, in which different real-world things, or\nentities, may appear. To deal with the innumerable entities involved with\ncommonsense in the real world, we introduce to CKG construction methods\nconceptualization, i.e., to view entities mentioned in text as instances of\nspecific concepts or vice versa. We build synthetic triples by\nconceptualization, and further formulate the task as triple classification,\nhandled by a discriminatory model with knowledge transferred from pretrained\nlanguage models and fine-tuned by negative sampling. Experiments demonstrate\nthat our methods can effectively identify plausible triples and expand the KG\nby triples of both new nodes and edges of high diversity and novelty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03350v1",
            "date": "2020-03-06 18:27:39+00:00",
            "title": "Distributional semantic modeling: a revised technique to train term/word vector space models applying the ontology-related approach",
            "authors": [
                "Oleksandr Palagin",
                "Vitalii Velychko",
                "Kyrylo Malakhov",
                "Oleksandr Shchurov"
            ],
            "abstract": "We design a new technique for the distributional semantic modeling with a\nneural network-based approach to learn distributed term representations (or\nterm embeddings) - term vector space models as a result, inspired by the recent\nontology-related approach (using different types of contextual knowledge such\nas syntactic knowledge, terminological knowledge, semantic knowledge, etc.) to\nthe identification of terms (term extraction) and relations between them\n(relation extraction) called semantic pre-processing technology - SPT. Our\nmethod relies on automatic term extraction from the natural language texts and\nsubsequent formation of the problem-oriented or application-oriented (also\ndeeply annotated) text corpora where the fundamental entity is the term\n(includes non-compositional and compositional terms). This gives us an\nopportunity to changeover from distributed word representations (or word\nembeddings) to distributed term representations (or term embeddings). This\ntransition will allow to generate more accurate semantic maps of different\nsubject domains (also, of relations between input terms - it is useful to\nexplore clusters and oppositions, or to test your hypotheses about them). The\nsemantic map can be represented as a graph using Vec2graph - a Python library\nfor visualizing word embeddings (term embeddings in our case) as dynamic and\ninteractive graphs. The Vec2graph library coupled with term embeddings will not\nonly improve accuracy in solving standard NLP tasks, but also update the\nconventional concept of automated ontology development. The main practical\nresult of our work is the development kit (set of toolkits represented as web\nservice APIs and web application), which provides all necessary routines for\nthe basic linguistic pre-processing and the semantic pre-processing of the\nnatural language texts in Ukrainian for future training of term vector space\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03504v1",
            "date": "2020-03-07 03:29:01+00:00",
            "title": "A Post-processing Method for Detecting Unknown Intent of Dialogue System via Pre-trained Deep Neural Network Classifier",
            "authors": [
                "Ting-En Lin",
                "Hua Xu"
            ],
            "abstract": "With the maturity and popularity of dialogue systems, detecting user's\nunknown intent in dialogue systems has become an important task. It is also one\nof the most challenging tasks since we can hardly get examples, prior knowledge\nor the exact numbers of unknown intents. In this paper, we propose SofterMax\nand deep novelty detection (SMDN), a simple yet effective post-processing\nmethod for detecting unknown intent in dialogue systems based on pre-trained\ndeep neural network classifiers. Our method can be flexibly applied on top of\nany classifiers trained in deep neural networks without changing the model\narchitecture. We calibrate the confidence of the softmax outputs to compute the\ncalibrated confidence score (i.e., SofterMax) and use it to calculate the\ndecision boundary for unknown intent detection. Furthermore, we feed the\nfeature representations learned by the deep neural networks into traditional\nnovelty detection algorithm to detect unknown intents from different\nperspectives. Finally, we combine the methods above to perform the joint\nprediction. Our method classifies examples that differ from known intents as\nunknown and does not require any examples or prior knowledge of it. We have\nconducted extensive experiments on three benchmark dialogue datasets. The\nresults show that our method can yield significant improvements compared with\nthe state-of-the-art baselines",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.03612v1",
            "date": "2020-03-07 17:22:52+00:00",
            "title": "Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text",
            "authors": [
                "Katherine Van Koevering",
                "Austin R. Benson",
                "Jon Kleinberg"
            ],
            "abstract": "There is inherent information captured in the order in which we write words\nin a list. The orderings of binomials --- lists of two words separated by `and'\nor `or' --- has been studied for more than a century. These binomials are\ncommon across many areas of speech, in both formal and informal text. In the\nlast century, numerous explanations have been given to describe what order\npeople use for these binomials, from differences in semantics to differences in\nphonology. These rules describe primarily `frozen' binomials that exist in\nexactly one ordering and have lacked large-scale trials to determine efficacy.\n  Online text provides a unique opportunity to study these lists in the context\nof informal text at a very large scale. In this work, we expand the view of\nbinomials to include a large-scale analysis of both frozen and non-frozen\nbinomials in a quantitative way. Using this data, we then demonstrate that most\npreviously proposed rules are ineffective at predicting binomial ordering. By\ntracking the order of these binomials across time and communities we are able\nto establish additional, unexplored dimensions central to these predictions.\n  Expanding beyond the question of individual binomials, we also explore the\nglobal structure of binomials in various communities, establishing a new model\nfor these lists and analyzing this structure for non-frozen and frozen\nbinomials. Additionally, novel analysis of trinomials --- lists of length three\n--- suggests that none of the binomials analysis applies in these cases.\nFinally, we demonstrate how large data sets gleaned from the web can be used in\nconjunction with older theories to expand and improve on old questions.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2003.03623v1",
            "date": "2020-03-07 18:09:18+00:00",
            "title": "Knowledge Graphs and Knowledge Networks: The Story in Brief",
            "authors": [
                "Amit Sheth",
                "Swati Padhee",
                "Amelie Gyrard"
            ],
            "abstract": "Knowledge Graphs (KGs) represent real-world noisy raw information in a\nstructured form, capturing relationships between entities. However, for dynamic\nreal-world applications such as social networks, recommender systems,\ncomputational biology, relational knowledge representation has emerged as a\nchallenging research problem where there is a need to represent the changing\nnodes, attributes, and edges over time. The evolution of search engine\nresponses to user queries in the last few years is partly because of the role\nof KGs such as Google KG. KGs are significantly contributing to various AI\napplications from link prediction, entity relations prediction, node\nclassification to recommendation and question answering systems. This article\nis an attempt to summarize the journey of KG for AI.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.03875v1",
            "date": "2020-03-09 00:32:13+00:00",
            "title": "Overview of the CCKS 2019 Knowledge Graph Evaluation Track: Entity, Relation, Event and QA",
            "authors": [
                "Xianpei Han",
                "Zhichun Wang",
                "Jiangtao Zhang",
                "Qinghua Wen",
                "Wenqi Li",
                "Buzhou Tang",
                "Qi Wang",
                "Zhifan Feng",
                "Yang Zhang",
                "Yajuan Lu",
                "Haitao Wang",
                "Wenliang Chen",
                "Hao Shao",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao",
                "Taifeng Wang",
                "Kezun Zhang",
                "Meng Wang",
                "Yinlin Jiang",
                "Guilin Qi",
                "Lei Zou",
                "Sen Hu",
                "Minhao Zhang",
                "Yinnian Lin"
            ],
            "abstract": "Knowledge graph models world knowledge as concepts, entities, and the\nrelationships between them, which has been widely used in many real-world\ntasks. CCKS 2019 held an evaluation track with 6 tasks and attracted more than\n1,600 teams. In this paper, we give an overview of the knowledge graph\nevaluation tract at CCKS 2019. By reviewing the task definition, successful\nmethods, useful resources, good strategies and research challenges associated\nwith each task in CCKS 2019, this paper can provide a helpful reference for\ndeveloping knowledge graph applications and conducting future knowledge graph\nresearches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.04195v1",
            "date": "2020-03-09 15:20:21+00:00",
            "title": "An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation",
            "authors": [
                "Piji Li"
            ],
            "abstract": "We present an empirical investigation of pre-trained Transformer-based\nauto-regressive language models for the task of open-domain dialogue\ngeneration. Training paradigm of pre-training and fine-tuning is employed to\nconduct the parameter learning. Corpora of News and Wikipedia in Chinese and\nEnglish are collected for the pre-training stage respectively. Dialogue context\nand response are concatenated into a single sequence utilized as the input of\nthe models during the fine-tuning stage. A weighted joint prediction paradigm\nfor both context and response is designed to evaluate the performance of models\nwith or without the loss term for context prediction. Various of decoding\nstrategies such as greedy search, beam search, top-k sampling, etc. are\nemployed to conduct the response text generation. Extensive experiments are\nconducted on the typical single-turn and multi-turn dialogue corpora such as\nWeibo, Douban, Reddit, DailyDialog, and Persona-Chat. Detailed numbers of\nautomatic evaluation metrics on relevance and diversity of the generated\nresults for the languages models as well as the baseline approaches are\nreported.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04360v2",
            "date": "2020-03-03 20:35:36+00:00",
            "title": "GenNet : Reading Comprehension with Multiple Choice Questions using Generation and Selection model",
            "authors": [
                "Vaishali Ingale",
                "Pushpender Singh"
            ],
            "abstract": "Multiple-choice machine reading comprehension is difficult task as its\nrequired machines to select the correct option from a set of candidate or\npossible options using the given passage and question.Reading Comprehension\nwith Multiple Choice Questions task,required a human (or machine) to read a\ngiven passage, question pair and select the best one option from n given\noptions. There are two different ways to select the correct answer from the\ngiven passage. Either by selecting the best match answer to by eliminating the\nworst match answer. Here we proposed GenNet model, a neural network-based\nmodel. In this model first we will generate the answer of the question from the\npassage and then will matched the generated answer with given answer, the best\nmatched option will be our answer. For answer generation we used S-net (Tan et\nal., 2017) model trained on SQuAD and to evaluate our model we used Large-scale\nRACE (ReAding Comprehension Dataset From Examinations) (Lai et al.,2017).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04419v3",
            "date": "2020-03-09 21:30:55+00:00",
            "title": "Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages",
            "authors": [
                "Machel Reid",
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "The contrast between the need for large amounts of data for current Natural\nLanguage Processing (NLP) techniques, and the lack thereof, is accentuated in\nthe case of African languages, most of which are considered low-resource. To\nhelp circumvent this issue, we explore techniques exploiting the qualities of\nmorphologically rich languages (MRLs), while leveraging pretrained word vectors\nin well-resourced languages. In our exploration, we show that a meta-embedding\napproach combining both pretrained and morphologically-informed word embeddings\nperforms best in the downstream task of Xhosa-English translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04748v1",
            "date": "2020-03-10 14:06:55+00:00",
            "title": "On the coexistence of competing languages",
            "authors": [
                "Jean-Marc Luck",
                "Anita Mehta"
            ],
            "abstract": "We investigate the evolution of competing languages, a subject where much\nprevious literature suggests that the outcome is always the domination of one\nlanguage over all the others. Since coexistence of languages is observed in\nreality, we here revisit the question of language competition, with an emphasis\non uncovering the ways in which coexistence might emerge. We find that this\nemergence is related to symmetry breaking, and explore two particular scenarios\n-- the first relating to an imbalance in the population dynamics of language\nspeakers in a single geographical area, and the second to do with spatial\nheterogeneity, where language preferences are specific to different\ngeographical regions. For each of these, the investigation of paradigmatic\nsituations leads us to a quantitative understanding of the conditions leading\nto language coexistence. We also obtain predictions of the number of surviving\nlanguages as a function of various model parameters.",
            "journal ref": "Eur. Phys. J. B (2020) 93, 73",
            "category": "cond-mat.stat-mech"
        },
        {
            "url": "http://arxiv.org/abs/2003.04865v1",
            "date": "2020-03-10 17:15:48+00:00",
            "title": "Video Caption Dataset for Describing Human Actions in Japanese",
            "authors": [
                "Yutaro Shigeto",
                "Yuya Yoshikawa",
                "Jiaqing Lin",
                "Akikazu Takeuchi"
            ],
            "abstract": "In recent years, automatic video caption generation has attracted\nconsiderable attention. This paper focuses on the generation of Japanese\ncaptions for describing human actions. While most currently available video\ncaption datasets have been constructed for English, there is no equivalent\nJapanese dataset. To address this, we constructed a large-scale Japanese video\ncaption dataset consisting of 79,822 videos and 399,233 captions. Each caption\nin our dataset describes a video in the form of \"who does what and where.\" To\ndescribe human actions, it is important to identify the details of a person,\nplace, and action. Indeed, when we describe human actions, we usually mention\nthe scene, person, and action. In our experiments, we evaluated two caption\ngeneration methods to obtain benchmark results. Further, we investigated\nwhether those generation methods could specify \"who does what and where.\"",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04970v1",
            "date": "2020-02-14 12:38:18+00:00",
            "title": "A Dataset Independent Set of Baselines for Relation Prediction in Argument Mining",
            "authors": [
                "Oana Cocarascu",
                "Elena Cabrio",
                "Serena Villata",
                "Francesca Toni"
            ],
            "abstract": "Argument Mining is the research area which aims at extracting argument\ncomponents and predicting argumentative relations (i.e.,support and attack)\nfrom text. In particular, numerous approaches have been proposed in the\nliterature to predict the relations holding between the arguments, and\napplication-specific annotated resources were built for this purpose. Despite\nthe fact that these resources have been created to experiment on the same task,\nthe definition of a single relation prediction method to be successfully\napplied to a significant portion of these datasets is an open research problem\nin Argument Mining. This means that none of the methods proposed in the\nliterature can be easily ported from one resource to another. In this paper, we\naddress this problem by proposing a set of dataset independent strong neural\nbaselines which obtain homogeneous results on all the datasets proposed in the\nliterature for the argumentative relation prediction task. Thus, our baselines\ncan be employed by the Argument Mining community to compare more effectively\nhow well a method performs on the argumentative relation prediction task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04973v1",
            "date": "2020-02-10 20:17:34+00:00",
            "title": "Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning",
            "authors": [
                "Neha Singh",
                "Nirmalya Roy",
                "Aryya Gangopadhyay"
            ],
            "abstract": "Social media generates an enormous amount of data on a daily basis but it is\nvery challenging to effectively utilize the data without annotating or labeling\nit according to the target application. We investigate the problem of localized\nflood detection using the social sensing model (Twitter) in order to provide an\nefficient, reliable and accurate flood text classification model with minimal\nlabeled data. This study is important since it can immensely help in providing\nthe flood-related updates and notifications to the city officials for emergency\ndecision making, rescue operations, and early warnings, etc. We propose to\nperform the text classification using the inductive transfer learning method\ni.e pre-trained language model ULMFiT and fine-tune it in order to effectively\nclassify the flood-related feeds in any new location. Finally, we show that\nusing very little new labeled data in the target domain we can successfully\nbuild an efficient and high performing model for flood detection and analysis\nwith human-generated facts and observations from Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.04992v1",
            "date": "2020-02-26 16:32:25+00:00",
            "title": "Multi-task Learning with Multi-head Attention for Multi-choice Reading Comprehension",
            "authors": [
                "Hui Wan"
            ],
            "abstract": "Multiple-choice Machine Reading Comprehension (MRC) is an important and\nchallenging Natural Language Understanding (NLU) task, in which a machine must\nchoose the answer to a question from a set of choices, with the question placed\nin context of text passages or dialog. In the last a couple of years the NLU\nfield has been revolutionized with the advent of models based on the\nTransformer architecture, which are pretrained on massive amounts of\nunsupervised data and then fine-tuned for various supervised learning NLU\ntasks. Transformer models have come to dominate a wide variety of leader-boards\nin the NLU field; in the area of MRC, the current state-of-the-art model on the\nDREAM dataset (see[Sunet al., 2019]) fine tunes Albert, a large pretrained\nTransformer-based model, and addition-ally combines it with an extra layer of\nmulti-head attention between context and question-answer[Zhuet al., 2020].The\npurpose of this note is to document a new state-of-the-art result in the DREAM\ntask, which is accomplished by, additionally, performing multi-task learning on\ntwo MRC multi-choice reading comprehension tasks (RACE and DREAM).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05002v1",
            "date": "2020-03-10 21:11:53+00:00",
            "title": "TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages",
            "authors": [
                "Jonathan H. Clark",
                "Eunsol Choi",
                "Michael Collins",
                "Dan Garrette",
                "Tom Kwiatkowski",
                "Vitaly Nikolaev",
                "Jennimaria Palomaki"
            ],
            "abstract": "Confidently making progress on multilingual modeling requires challenging,\ntrustworthy evaluations. We present TyDi QA---a question answering dataset\ncovering 11 typologically diverse languages with 204K question-answer pairs.\nThe languages of TyDi QA are diverse with regard to their typology---the set of\nlinguistic features each language expresses---such that we expect models\nperforming well on this set to generalize across a large number of the world's\nlanguages. We present a quantitative analysis of the data quality and\nexample-level qualitative linguistic analyses of observed language phenomena\nthat would not be found in English-only corpora. To provide a realistic\ninformation-seeking task and avoid priming effects, questions are written by\npeople who want to know the answer, but don't know the answer yet, and the data\nis collected directly in each language without the use of translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05146v1",
            "date": "2020-03-11 07:48:46+00:00",
            "title": "Entity Extraction from Wikipedia List Pages",
            "authors": [
                "Nicolas Heist",
                "Heiko Paulheim"
            ],
            "abstract": "When it comes to factual knowledge about a wide range of domains, Wikipedia\nis often the prime source of information on the web. DBpedia and YAGO, as large\ncross-domain knowledge graphs, encode a subset of that knowledge by creating an\nentity for each page in Wikipedia, and connecting them through edges. It is\nwell known, however, that Wikipedia-based knowledge graphs are far from\ncomplete. Especially, as Wikipedia's policies permit pages about subjects only\nif they have a certain popularity, such graphs tend to lack information about\nless well-known entities. Information about these entities is oftentimes\navailable in the encyclopedia, but not represented as an individual page. In\nthis paper, we present a two-phased approach for the extraction of entities\nfrom Wikipedia's list pages, which have proven to serve as a valuable source of\ninformation. In the first phase, we build a large taxonomy from categories and\nlist pages with DBpedia as a backbone. With distant supervision, we extract\ntraining data for the identification of new entities in list pages that we use\nin the second phase to train a classification model. With this approach we\nextract over 700k new entities and extend DBpedia with 7.5M new type statements\nand 3.8M new facts of high precision.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.05162v3",
            "date": "2020-03-11 08:42:57+00:00",
            "title": "Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning",
            "authors": [
                "Zhiyuan Fang",
                "Tejas Gokhale",
                "Pratyay Banerjee",
                "Chitta Baral",
                "Yezhou Yang"
            ],
            "abstract": "Captioning is a crucial and challenging task for video understanding. In\nvideos that involve active agents such as humans, the agent's actions can bring\nabout myriad changes in the scene. Observable changes such as movements,\nmanipulations, and transformations of the objects in the scene, are reflected\nin conventional video captioning. Unlike images, actions in videos are also\ninherently linked to social aspects such as intentions (why the action is\ntaking place), effects (what changes due to the action), and attributes that\ndescribe the agent. Thus for video understanding, such as when captioning\nvideos or when answering questions about videos, one must have an understanding\nof these commonsense aspects. We present the first work on generating\ncommonsense captions directly from videos, to describe latent aspects such as\nintentions, effects, and attributes. We present a new dataset\n\"Video-to-Commonsense (V2C)\" that contains $\\sim9k$ videos of human agents\nperforming various actions, annotated with 3 types of commonsense descriptions.\nAdditionally we explore the use of open-ended video-based commonsense question\nanswering (V2C-QA) as a way to enrich our captions. Both the generation task\nand the QA task can be used to enrich video captions.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.05171v2",
            "date": "2020-03-11 09:07:02+00:00",
            "title": "Vector symbolic architectures for context-free grammars",
            "authors": [
                "Peter beim Graben",
                "Markus Huber",
                "Werner Meyer",
                "Ronald R\u00f6mer",
                "Matthias Wolff"
            ],
            "abstract": "Background / introduction. Vector symbolic architectures (VSA) are a viable\napproach for the hyperdimensional representation of symbolic data, such as\ndocuments, syntactic structures, or semantic frames. Methods. We present a\nrigorous mathematical framework for the representation of phrase structure\ntrees and parse trees of context-free grammars (CFG) in Fock space, i.e.\ninfinite-dimensional Hilbert space as being used in quantum field theory. We\ndefine a novel normal form for CFG by means of term algebras. Using a recently\ndeveloped software toolbox, called FockBox, we construct Fock space\nrepresentations for the trees built up by a CFG left-corner (LC) parser.\nResults. We prove a universal representation theorem for CFG term algebras in\nFock space and illustrate our findings through a low-dimensional principal\ncomponent projection of the LC parser states. Conclusions. Our approach could\nleverage the development of VSA for explainable artificial intelligence (XAI)\nby means of hyperdimensional deep neural computation. It could be of\nsignificance for the improvement of cognitive user interfaces and other\napplications of VSA in machine learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05473v1",
            "date": "2020-03-11 18:23:00+00:00",
            "title": "Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking",
            "authors": [
                "Samuel Broscheit"
            ],
            "abstract": "A typical architecture for end-to-end entity linking systems consists of\nthree steps: mention detection, candidate generation and entity disambiguation.\nIn this study we investigate the following questions: (a) Can all those steps\nbe learned jointly with a model for contextualized text-representations, i.e.\nBERT (Devlin et al., 2019)? (b) How much entity knowledge is already contained\nin pretrained BERT? (c) Does additional entity knowledge improve BERT's\nperformance in downstream tasks? To this end, we propose an extreme\nsimplification of the entity linking setup that works surprisingly well: simply\ncast it as a per token classification over the entire entity vocabulary (over\n700K classes in our case). We show on an entity linking benchmark that (i) this\nmodel improves the entity representations over plain BERT, (ii) that it\noutperforms entity linking architectures that optimize the tasks separately and\n(iii) that it only comes second to the current state-of-the-art that does\nmention detection and entity disambiguation jointly. Additionally, we\ninvestigate the usefulness of entity-aware token-representations in the\ntext-understanding benchmark GLUE, as well as the question answering benchmarks\nSQUAD V2 and SWAG and also the EN-DE WMT14 machine translation benchmark. To\nour surprise, we find that most of those benchmarks do not benefit from\nadditional entity knowledge, except for a task with very small training data,\nthe RTE task in GLUE, which improves by 2%.",
            "journal ref": "Proceedings of the 23rd Conference on Computational Natural\n  Language Learning (CoNLL), 2019, 677-685",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05573v1",
            "date": "2020-03-12 02:18:19+00:00",
            "title": "Learning word-referent mappings and concepts from raw inputs",
            "authors": [
                "Wai Keen Vong",
                "Brenden M. Lake"
            ],
            "abstract": "How do children learn correspondences between the language and the world from\nnoisy, ambiguous, naturalistic input? One hypothesis is via cross-situational\nlearning: tracking words and their possible referents across multiple\nsituations allows learners to disambiguate correct word-referent mappings (Yu &\nSmith, 2007). However, previous models of cross-situational word learning\noperate on highly simplified representations, side-stepping two important\naspects of the actual learning problem. First, how can word-referent mappings\nbe learned from raw inputs such as images? Second, how can these learned\nmappings generalize to novel instances of a known word? In this paper, we\npresent a neural network model trained from scratch via self-supervision that\ntakes in raw images and words as inputs, and show that it can learn\nword-referent mappings from fully ambiguous scenes and utterances through\ncross-situational learning. In addition, the model generalizes to novel word\ninstances, locates referents of words in a scene, and shows a preference for\nmutual exclusivity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05758v2",
            "date": "2020-03-12 12:55:50+00:00",
            "title": "It Means More if It Sounds Good: Yet Another Hypothesis Concerning the Evolution of Polysemous Words",
            "authors": [
                "Ivan P. Yamshchikov",
                "Cyrille Merleau Nono Saha",
                "Igor Samenko",
                "J\u00fcrgen Jost"
            ],
            "abstract": "This position paper looks into the formation of language and shows ties\nbetween structural properties of the words in the English language and their\npolysemy. Using Ollivier-Ricci curvature over a large graph of synonyms to\nestimate polysemy it shows empirically that the words that arguably are easier\nto pronounce also tend to have multiple meanings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05826v1",
            "date": "2020-03-12 14:51:41+00:00",
            "title": "Regular Intersection Emptiness of Graph Problems: Finding a Needle in a Haystack of Graphs with the Help of Automata",
            "authors": [
                "Petra Wolf",
                "Henning Fernau"
            ],
            "abstract": "The Int_reg-problem of a combinatorial problem P asks, given a\nnondeterministic automaton M as input, whether the language L(M) accepted by M\ncontains any positive instance of the problem P. We consider the\nInt_reg-problem for a number of different graph problems and give general\ncriteria that give decision procedures for these Int_reg-problems. To achieve\nthis goal, we consider a natural graph encoding so that the language of all\ngraph encodings is regular. Then, we draw the connection between classical\npumping- and interchange-arguments from the field of formal language theory\nwith the graph operations induced on the encoded graph. Our techniques apply\namong others to the Int_reg-problem of well-known graph problems like Vertex\nCover and Independent Set, as well as to subgraph problems, graph-edit problems\nand graph-partitioning problems, including coloring problems.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2003.05895v1",
            "date": "2020-03-03 22:05:37+00:00",
            "title": "Investigating the influence Brexit had on Financial Markets, in particular the GBP/EUR exchange rate",
            "authors": [
                "Michael Filletti"
            ],
            "abstract": "On 23rd June 2016, 51.9% of British voters voted to leave the European Union,\ntriggering a process and events that have led to the United Kingdom leaving the\nEU, an event that has become known as 'Brexit'. In this piece of research, we\ninvestigate the effects of this entire process on the currency markets,\nspecifically the GBP/EUR exchange rate. Financial markets are known to be\nsensitive to news articles and media, and the aim of this research is to\nevaluate the magnitude of impact of relevant events, as well as whether the\nimpact was positive or negative for the GBP.",
            "journal ref": "None",
            "category": "q-fin.ST"
        },
        {
            "url": "http://arxiv.org/abs/2003.05995v1",
            "date": "2020-03-12 19:47:29+00:00",
            "title": "CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues",
            "authors": [
                "Francisco J. Chiyah Garcia",
                "Jos\u00e9 Lopes",
                "Xingkun Liu",
                "Helen Hastie"
            ],
            "abstract": "Large corpora of task-based and open-domain conversational dialogues are\nhugely valuable in the field of data-driven dialogue systems. Crowdsourcing\nplatforms, such as Amazon Mechanical Turk, have been an effective method for\ncollecting such large amounts of data. However, difficulties arise when\ntask-based dialogues require expert domain knowledge or rapid access to\ndomain-relevant information, such as databases for tourism. This will become\neven more prevalent as dialogue systems become increasingly ambitious,\nexpanding into tasks with high levels of complexity that require collaboration\nand forward planning, such as in our domain of emergency response. In this\npaper, we propose CRWIZ: a framework for collecting real-time Wizard of Oz\ndialogues through crowdsourcing for collaborative, complex tasks. This\nframework uses semi-guided dialogue to avoid interactions that breach\nprocedures and processes only known to experts, while enabling the capture of a\nwide variety of interactions. The framework is available at\nhttps://github.com/JChiyah/crwiz",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2003.06279v1",
            "date": "2020-03-13 13:35:44+00:00",
            "title": "Using word embeddings to improve the discriminability of co-occurrence text networks",
            "authors": [
                "Laura V. C. Quispe",
                "Jorge A. V. Tohalino",
                "Diego R. Amancio"
            ],
            "abstract": "Word co-occurrence networks have been employed to analyze texts both in the\npractical and theoretical scenarios. Despite the relative success in several\napplications, traditional co-occurrence networks fail in establishing links\nbetween similar words whenever they appear distant in the text. Here we\ninvestigate whether the use of word embeddings as a tool to create virtual\nlinks in co-occurrence networks may improve the quality of classification\nsystems. Our results revealed that the discriminability in the stylometry task\nis improved when using Glove, Word2Vec and FastText. In addition, we found that\noptimized results are obtained when stopwords are not disregarded and a simple\nglobal thresholding strategy is used to establish virtual links. Because the\nproposed approach is able to improve the representation of texts as complex\nnetworks, we believe that it could be extended to study other natural language\nprocessing tasks. Likewise, theoretical languages studies could benefit from\nthe adopted enriched representation of word co-occurrence networks.",
            "journal ref": "Physica A 562, 125344, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06514v1",
            "date": "2020-03-13 23:56:37+00:00",
            "title": "DAN: Dual-View Representation Learning for Adapting Stance Classifiers to New Domains",
            "authors": [
                "Chang Xu",
                "Cecile Paris",
                "Surya Nepal",
                "Ross Sparks",
                "Chong Long",
                "Yafang Wang"
            ],
            "abstract": "We address the issue of having a limited number of annotations for stance\nclassification in a new domain, by adapting out-of-domain classifiers with\ndomain adaptation. Existing approaches often align different domains in a\nsingle, global feature space (or view), which may fail to fully capture the\nrichness of the languages used for expressing stances, leading to reduced\nadaptability on stance data. In this paper, we identify two major types of\nstance expressions that are linguistically distinct, and we propose a tailored\ndual-view adaptation network (DAN) to adapt these expressions across domains.\nThe proposed model first learns a separate view for domain transfer in each\nexpression channel and then selects the best adapted parts of both views for\noptimal transfer. We find that the learned view features can be more easily\naligned and more stance-discriminative in either or both views, leading to more\ntransferable overall features after combining the views. Results from extensive\nexperiments show that our method can enhance the state-of-the-art single-view\nmethods in matching stance data across different domains, and that it\nconsistently improves those methods on various adaptation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.06561v1",
            "date": "2020-03-14 06:16:30+00:00",
            "title": "Semantically-Enriched Search Engine for Geoportals: A Case Study with ArcGIS Online",
            "authors": [
                "Gengchen Mai",
                "Krzysztof Janowicz",
                "Sathya Prasad",
                "Meilin Shi",
                "Ling Cai",
                "Rui Zhu",
                "Blake Regalia",
                "Ni Lao"
            ],
            "abstract": "Many geoportals such as ArcGIS Online are established with the goal of\nimproving geospatial data reusability and achieving intelligent knowledge\ndiscovery. However, according to previous research, most of the existing\ngeoportals adopt Lucene-based techniques to achieve their core search\nfunctionality, which has a limited ability to capture the user's search\nintentions. To better understand a user's search intention, query expansion can\nbe used to enrich the user's query by adding semantically similar terms. In the\ncontext of geoportals and geographic information retrieval, we advocate the\nidea of semantically enriching a user's query from both geospatial and thematic\nperspectives. In the geospatial aspect, we propose to enrich a query by using\nboth place partonomy and distance decay. In terms of the thematic aspect,\nconcept expansion and embedding-based document similarity are used to infer the\nimplicit information hidden in a user's query. This semantic query expansion 1\n2 G. Mai et al. framework is implemented as a semantically-enriched search\nengine using ArcGIS Online as a case study. A benchmark dataset is constructed\nto evaluate the proposed framework. Our evaluation results show that the\nproposed semantic query expansion framework is very effective in capturing a\nuser's search intention and significantly outperforms a well-established\nbaseline-Lucene's practical scoring function-with more than 3.0 increments in\nDCG@K (K=3,5,10).",
            "journal ref": "AGILE 2020, Jun. 16 - 19, 2020, Chania, Crete, Greece",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.06745v1",
            "date": "2020-03-15 03:08:06+00:00",
            "title": "Vision-Dialog Navigation by Exploring Cross-modal Memory",
            "authors": [
                "Yi Zhu",
                "Fengda Zhu",
                "Zhaohuan Zhan",
                "Bingqian Lin",
                "Jianbin Jiao",
                "Xiaojun Chang",
                "Xiaodan Liang"
            ],
            "abstract": "Vision-dialog navigation posed as a new holy-grail task in vision-language\ndisciplinary targets at learning an agent endowed with the capability of\nconstant conversation for help with natural language and navigating according\nto human responses. Besides the common challenges faced in visual language\nnavigation, vision-dialog navigation also requires to handle well with the\nlanguage intentions of a series of questions about the temporal context from\ndialogue history and co-reasoning both dialogs and visual scenes. In this\npaper, we propose the Cross-modal Memory Network (CMN) for remembering and\nunderstanding the rich information relevant to historical navigation actions.\nOur CMN consists of two memory modules, the language memory module (L-mem) and\nthe visual memory module (V-mem). Specifically, L-mem learns latent\nrelationships between the current language interaction and a dialog history by\nemploying a multi-head attention mechanism. V-mem learns to associate the\ncurrent visual views and the cross-modal memory about the previous navigation\nactions. The cross-modal memory is generated via a vision-to-language attention\nand a language-to-vision attention. Benefiting from the collaborative learning\nof the L-mem and the V-mem, our CMN is able to explore the memory about the\ndecision making of historical navigation actions which is for the current step.\nExperiments on the CVDN dataset show that our CMN outperforms the previous\nstate-of-the-art model by a significant margin on both seen and unseen\nenvironments.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.07385v1",
            "date": "2020-03-16 18:08:52+00:00",
            "title": "A Formal Analysis of Multimodal Referring Strategies Under Common Ground",
            "authors": [
                "Nikhil Krishnaswamy",
                "James Pustejovsky"
            ],
            "abstract": "In this paper, we present an analysis of computationally generated\nmixed-modality definite referring expressions using combinations of gesture and\nlinguistic descriptions. In doing so, we expose some striking formal semantic\nproperties of the interactions between gesture and language, conditioned on the\nintroduction of content into the common ground between the (computational)\nspeaker and (human) viewer, and demonstrate how these formal features can\ncontribute to training better models to predict viewer judgment of referring\nexpressions, and potentially to the generation of more natural and informative\nreferring expressions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07444v3",
            "date": "2020-03-16 21:16:00+00:00",
            "title": "A Label Proportions Estimation Technique for Adversarial Domain Adaptation in Text Classification",
            "authors": [
                "Zhuohao Chen",
                "Singla Karan",
                "David C. Atkins",
                "Zac E Imel",
                "Shrikanth Narayanan"
            ],
            "abstract": "Many text classification tasks are domain-dependent, and various domain\nadaptation approaches have been proposed to predict unlabeled data in a new\ndomain. Domain-adversarial neural networks (DANN) and their variants have been\nused widely recently and have achieved promising results for this problem.\nHowever, most of these approaches assume that the label proportions of the\nsource and target domains are similar, which rarely holds in most real-world\nscenarios. Sometimes the label shift can be large and the DANN fails to learn\ndomain-invariant features. In this study, we focus on unsupervised domain\nadaptation of text classification with label shift and introduce a domain\nadversarial network with label proportions estimation (DAN-LPE) framework. The\nDAN-LPE simultaneously trains a domain adversarial net and processes label\nproportions estimation by the confusion of the source domain and the\npredictions of the target domain. Experiments show the DAN-LPE achieves a good\nestimate of the target label distributions and reduces the label shift to\nimprove the classification performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07683v1",
            "date": "2020-02-29 04:30:28+00:00",
            "title": "Who Wins the Game of Thrones? How Sentiments Improve the Prediction of Candidate Choice",
            "authors": [
                "Chaehan So"
            ],
            "abstract": "This paper analyzes how candidate choice prediction improves by different\npsychological predictors. To investigate this question, it collected an\noriginal survey dataset featuring the popular TV series \"Game of Thrones\". The\nrespondents answered which character they anticipated to win in the final\nepisode of the series, and explained their choice of the final candidate in\nfree text from which sentiments were extracted. These sentiments were compared\nto feature sets derived from candidate likeability and candidate personality\nratings. In our benchmarking of 10-fold cross-validation in 100 repetitions,\nall feature sets except the likeability ratings yielded a 10-11% improvement in\naccuracy on the holdout set over the base model. Treating the class imbalance\nwith synthetic minority oversampling (SMOTE) increased holdout set performance\nby 20-34% but surprisingly not testing set performance. Taken together, our\nstudy provides a quantified estimation of the additional predictive value of\npsychological predictors. Likeability ratings were clearly outperformed by the\nfeature sets based on personality, emotional valence, and basic emotions.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2003.07845v2",
            "date": "2020-03-17 17:50:26+00:00",
            "title": "PowerNorm: Rethinking Batch Normalization in Transformers",
            "authors": [
                "Sheng Shen",
                "Zhewei Yao",
                "Amir Gholami",
                "Michael W. Mahoney",
                "Kurt Keutzer"
            ],
            "abstract": "The standard normalization method for neural network (NN) models used in\nNatural Language Processing (NLP) is layer normalization (LN). This is\ndifferent than batch normalization (BN), which is widely-adopted in Computer\nVision. The preferred use of LN in NLP is principally due to the empirical\nobservation that a (naive/vanilla) use of BN leads to significant performance\ndegradation for NLP tasks; however, a thorough understanding of the underlying\nreasons for this is not always evident. In this paper, we perform a systematic\nstudy of NLP transformer models to understand why BN has a poor performance, as\ncompared to LN. We find that the statistics of NLP data across the batch\ndimension exhibit large fluctuations throughout training. This results in\ninstability, if BN is naively implemented. To address this, we propose Power\nNormalization (PN), a novel normalization scheme that resolves this issue by\n(i) relaxing zero-mean normalization in BN, (ii) incorporating a running\nquadratic mean instead of per batch statistics to stabilize fluctuations, and\n(iii) using an approximate backpropagation for incorporating the running\nstatistics in the forward pass. We show theoretically, under mild assumptions,\nthat PN leads to a smaller Lipschitz constant for the loss, compared with BN.\nFurthermore, we prove that the approximate backpropagation scheme leads to\nbounded gradients. We extensively test PN for transformers on a range of NLP\ntasks, and we show that it significantly outperforms both LN and BN. In\nparticular, PN outperforms LN by 0.4/0.6 BLEU on IWSLT14/WMT14 and 5.6/3.0 PPL\non PTB/WikiText-103. We make our code publicly available at\n\\url{https://github.com/sIncerass/powernorm}.",
            "journal ref": "ICML 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.07892v3",
            "date": "2020-03-17 18:58:44+00:00",
            "title": "Calibration of Pre-trained Transformers",
            "authors": [
                "Shrey Desai",
                "Greg Durrett"
            ],
            "abstract": "Pre-trained Transformers are now ubiquitous in natural language processing,\nbut despite their high end-task performance, little is known empirically about\nwhether they are calibrated. Specifically, do these models' posterior\nprobabilities provide an accurate empirical measure of how likely the model is\nto be correct on a given example? We focus on BERT and RoBERTa in this work,\nand analyze their calibration across three tasks: natural language inference,\nparaphrase detection, and commonsense reasoning. For each task, we consider\nin-domain as well as challenging out-of-domain settings, where models face more\nexamples they should be uncertain about. We show that: (1) when used\nout-of-the-box, pre-trained models are calibrated in-domain, and compared to\nbaselines, their calibration error out-of-domain can be as much as 3.5x lower;\n(2) temperature scaling is effective at further reducing calibration error\nin-domain, and using label smoothing to deliberately increase empirical\nuncertainty helps calibrate posteriors out-of-domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08001v1",
            "date": "2020-03-18 01:18:09+00:00",
            "title": "Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study",
            "authors": [
                "Farahnaz Akrami",
                "Mohammed Samiul Saeef",
                "Qingheng Zhang",
                "Wei Hu",
                "Chengkai Li"
            ],
            "abstract": "In the active research area of employing embedding models for knowledge graph\ncompletion, particularly for the task of link prediction, most prior studies\nused two benchmark datasets FB15k and WN18 in evaluating such models. Most\ntriples in these and other datasets in such studies belong to reverse and\nduplicate relations which exhibit high data redundancy due to semantic\nduplication, correlation or data incompleteness. This is a case of excessive\ndata leakage---a model is trained using features that otherwise would not be\navailable when the model needs to be applied for real prediction. There are\nalso Cartesian product relations for which every triple formed by the Cartesian\nproduct of applicable subjects and objects is a true fact. Link prediction on\nthe aforementioned relations is easy and can be achieved with even better\naccuracy using straightforward rules instead of sophisticated embedding models.\nA more fundamental defect of these models is that the link prediction scenario,\ngiven such data, is non-existent in the real-world. This paper is the first\nsystematic study with the main objective of assessing the true effectiveness of\nembedding models when the unrealistic triples are removed. Our experiment\nresults show these models are much less accurate than what we used to perceive.\nTheir poor accuracy renders link prediction a task without truly effective\nautomated solution. Hence, we call for re-investigation of possible effective\napproaches.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.08004v1",
            "date": "2020-03-18 01:30:02+00:00",
            "title": "Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization",
            "authors": [
                "Haiyang Xu",
                "Yun Wang",
                "Kun Han",
                "Baochang Ma",
                "Junwen Chen",
                "Xiangang Li"
            ],
            "abstract": "Abstractive text summarization is a challenging task, and one need to design\na mechanism to effectively extract salient information from the source text and\nthen generate a summary. A parsing process of the source text contains critical\nsyntactic or semantic structures, which is useful to generate more accurate\nsummary. However, modeling a parsing tree for text summarization is not trivial\ndue to its non-linear structure and it is harder to deal with a document that\nincludes multiple sentences and their parsing trees. In this paper, we propose\nto use a graph to connect the parsing trees from the sentences in a document\nand utilize the stacked graph convolutional networks (GCNs) to learn the\nsyntactic representation for a document. The selective attention mechanism is\nused to extract salient information in semantic and structural aspect and\ngenerate an abstractive summary. We evaluate our approach on the CNN/Daily Mail\ntext summarization dataset. The experimental results show that the proposed\nGCNs based selective attention approach outperforms the baselines and achieves\nthe state-of-the-art performance on the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08271v3",
            "date": "2020-03-18 15:22:51+00:00",
            "title": "Pre-trained Models for Natural Language Processing: A Survey",
            "authors": [
                "Xipeng Qiu",
                "Tianxiang Sun",
                "Yige Xu",
                "Yunfan Shao",
                "Ning Dai",
                "Xuanjing Huang"
            ],
            "abstract": "Recently, the emergence of pre-trained models (PTMs) has brought natural\nlanguage processing (NLP) to a new era. In this survey, we provide a\ncomprehensive review of PTMs for NLP. We first briefly introduce language\nrepresentation learning and its research progress. Then we systematically\ncategorize existing PTMs based on a taxonomy with four perspectives. Next, we\ndescribe how to adapt the knowledge of PTMs to the downstream tasks. Finally,\nwe outline some potential directions of PTMs for future research. This survey\nis purposed to be a hands-on guide for understanding, using, and developing\nPTMs for various NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08370v2",
            "date": "2020-03-18 17:48:35+00:00",
            "title": "Distant Supervision and Noisy Label Learning for Low Resource Named Entity Recognition: A Study on Hausa and Yor\u00f9b\u00e1",
            "authors": [
                "David Ifeoluwa Adelani",
                "Michael A. Hedderich",
                "Dawei Zhu",
                "Esther van den Berg",
                "Dietrich Klakow"
            ],
            "abstract": "The lack of labeled training data has limited the development of natural\nlanguage processing tools, such as named entity recognition, for many languages\nspoken in developing countries. Techniques such as distant and weak supervision\ncan be used to create labeled data in a (semi-) automatic way. Additionally, to\nalleviate some of the negative effects of the errors in automatic annotation,\nnoise-handling methods can be integrated. Pretrained word embeddings are\nanother key component of most neural named entity classifiers. With the advent\nof more complex contextual word embeddings, an interesting trade-off between\nmodel size and performance arises. While these techniques have been shown to\nwork well in high-resource settings, we want to study how they perform in\nlow-resource scenarios. In this work, we perform named entity recognition for\nHausa and Yor\\`ub\\'a, two languages that are widely spoken in several\ndeveloping countries. We evaluate different embedding approaches and show that\ndistant supervision can be successfully leveraged in a realistic low-resource\nscenario where it can more than double a classifier's performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08380v1",
            "date": "2020-03-18 17:56:07+00:00",
            "title": "TTTTTackling WinoGrande Schemas",
            "authors": [
                "Sheng-Chieh Lin",
                "Jheng-Hong Yang",
                "Rodrigo Nogueira",
                "Ming-Feng Tsai",
                "Chuan-Ju Wang",
                "Jimmy Lin"
            ],
            "abstract": "We applied the T5 sequence-to-sequence model to tackle the AI2 WinoGrande\nChallenge by decomposing each example into two input text strings, each\ncontaining a hypothesis, and using the probabilities assigned to the\n\"entailment\" token as a score of the hypothesis. Our first (and only)\nsubmission to the official leaderboard yielded 0.7673 AUC on March 13, 2020,\nwhich is the best known result at this time and beats the previous state of the\nart by over five points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08437v1",
            "date": "2020-03-18 18:59:55+00:00",
            "title": "A Corpus of Adpositional Supersenses for Mandarin Chinese",
            "authors": [
                "Siyao Peng",
                "Yang Liu",
                "Yilun Zhu",
                "Austin Blodgett",
                "Yushi Zhao",
                "Nathan Schneider"
            ],
            "abstract": "Adpositions are frequent markers of semantic relations, but they are highly\nambiguous and vary significantly from language to language. Moreover, there is\na dearth of annotated corpora for investigating the cross-linguistic variation\nof adposition semantics, or for building multilingual disambiguation systems.\nThis paper presents a corpus in which all adpositions have been semantically\nannotated in Mandarin Chinese; to the best of our knowledge, this is the first\nChinese corpus to be broadly annotated with adposition semantics. Our approach\nadapts a framework that defined a general set of supersenses according to\nostensibly language-independent semantic criteria, though its development\nfocused primarily on English prepositions (Schneider et al., 2018). We find\nthat the supersense categories are well-suited to Chinese adpositions despite\nsyntactic differences from English. On a Mandarin translation of The Little\nPrince, we achieve high inter-annotator agreement and analyze semantic\ncorrespondences of adposition tokens in bitext.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08553v1",
            "date": "2020-03-19 03:32:03+00:00",
            "title": "QnAMaker: Data to Bot in 2 Minutes",
            "authors": [
                "Parag Agrawal",
                "Tulasi Menon",
                "Aya Kamel",
                "Michel Naim",
                "Chaikesh Chouragade",
                "Gurvinder Singh",
                "Rohan Kulkarni",
                "Anshuman Suri",
                "Sahithi Katakam",
                "Vineet Pratik",
                "Prakul Bansal",
                "Simerpreet Kaur",
                "Neha Rajput",
                "Anand Duggal",
                "Achraf Chalabi",
                "Prashant Choudhari",
                "Reddy Satti",
                "Niranjan Nayak"
            ],
            "abstract": "Having a bot for seamless conversations is a much-desired feature that\nproducts and services today seek for their websites and mobile apps. These bots\nhelp reduce traffic received by human support significantly by handling\nfrequent and directly answerable known questions. Many such services have huge\nreference documents such as FAQ pages, which makes it hard for users to browse\nthrough this data. A conversation layer over such raw data can lower traffic to\nhuman support by a great margin. We demonstrate QnAMaker, a service that\ncreates a conversational layer over semi-structured data such as FAQ pages,\nproduct manuals, and support documents. QnAMaker is the popular choice for\nExtraction and Question-Answering as a service and is used by over 15,000 bots\nin production. It is also used by search interfaces and not just bots.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.08811v1",
            "date": "2020-03-19 14:23:12+00:00",
            "title": "Temporal Embeddings and Transformer Models for Narrative Text Understanding",
            "authors": [
                "Vani K",
                "Simone Mellace",
                "Alessandro Antonucci"
            ],
            "abstract": "We present two deep learning approaches to narrative text understanding for\ncharacter relationship modelling. The temporal evolution of these relations is\ndescribed by dynamic word embeddings, that are designed to learn semantic\nchanges over time. An empirical analysis of the corresponding character\ntrajectories shows that such approaches are effective in depicting dynamic\nevolution. A supervised learning approach based on the state-of-the-art\ntransformer model BERT is used instead to detect static relations between\ncharacters. The empirical validation shows that such events (e.g., two\ncharacters belonging to the same family) might be spotted with good accuracy,\neven when using automatically annotated data. This provides a deeper\nunderstanding of narrative plots based on the identification of key facts.\nStandard clustering techniques are finally used for character de-aliasing, a\nnecessary pre-processing step for both approaches. Overall, deep learning\nmodels appear to be suitable for narrative text understanding, while also\nproviding a challenging and unexploited benchmark for general natural language\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08875v1",
            "date": "2020-03-19 15:55:21+00:00",
            "title": "Beheshti-NER: Persian Named Entity Recognition Using BERT",
            "authors": [
                "Ehsan Taher",
                "Seyed Abbas Hoseini",
                "Mehrnoush Shamsfard"
            ],
            "abstract": "Named entity recognition is a natural language processing task to recognize\nand extract spans of text associated with named entities and classify them in\nsemantic Categories.\n  Google BERT is a deep bidirectional language model, pre-trained on large\ncorpora that can be fine-tuned to solve many NLP tasks such as question\nanswering, named entity recognition, part of speech tagging and etc. In this\npaper, we use the pre-trained deep bidirectional network, BERT, to make a model\nfor named entity recognition in Persian.\n  We also compare the results of our model with the previous state of the art\nresults achieved on Persian NER. Our evaluation metric is CONLL 2003 score in\ntwo levels of word and phrase. This model achieved second place in NSURL-2019\ntask 7 competition which associated with NER for the Persian language. our\nresults in this competition are 83.5 and 88.4 f1 CONLL score respectively in\nphrase and word level evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.08964v3",
            "date": "2020-03-19 18:15:05+00:00",
            "title": "The value of text for small business default prediction: A deep learning approach",
            "authors": [
                "Matthew Stevenson",
                "Christophe Mues",
                "Cristi\u00e1n Bravo"
            ],
            "abstract": "Compared to consumer lending, Micro, Small and Medium Enterprise (mSME)\ncredit risk modelling is particularly challenging, as, often, the same sources\nof information are not available. Therefore, it is standard policy for a loan\nofficer to provide a textual loan assessment to mitigate limited data\navailability. In turn, this statement is analysed by a credit expert alongside\nany available standard credit data. In our paper, we exploit recent advances\nfrom the field of Deep Learning and Natural Language Processing (NLP),\nincluding the BERT (Bidirectional Encoder Representations from Transformers)\nmodel, to extract information from 60000 textual assessments provided by a\nlender. We consider the performance in terms of the AUC (Area Under the\nreceiver operating characteristic Curve) and Brier Score metrics and find that\nthe text alone is surprisingly effective for predicting default. However, when\ncombined with traditional data, it yields no additional predictive capability,\nwith performance dependent on the text's length. Our proposed deep learning\nmodel does, however, appear to be robust to the quality of the text and\ntherefore suitable for partly automating the mSME lending process. We also\ndemonstrate how the content of loan assessments influences performance, leading\nus to a series of recommendations on a new strategy for collecting future mSME\nloan assessments.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2003.09024v1",
            "date": "2020-03-19 21:24:45+00:00",
            "title": "Techniques for Vocabulary Expansion in Hybrid Speech Recognition Systems",
            "authors": [
                "Nikolay Malkovsky",
                "Vladimir Bataev",
                "Dmitrii Sviridkin",
                "Natalia Kizhaeva",
                "Aleksandr Laptev",
                "Ildar Valiev",
                "Oleg Petrov"
            ],
            "abstract": "The problem of out of vocabulary words (OOV) is typical for any speech\nrecognition system, hybrid systems are usually constructed to recognize a fixed\nset of words and rarely can include all the words that will be encountered\nduring exploitation of the system. One of the popular approach to cover OOVs is\nto use subword units rather then words. Such system can potentially recognize\nany previously unseen word if the word can be constructed from present subword\nunits, but also non-existing words can be recognized. The other popular\napproach is to modify HMM part of the system so that it can be easily and\neffectively expanded with custom set of words we want to add to the system. In\nthis paper we explore different existing methods of this solution on both graph\nconstruction and search method levels. We also present a novel vocabulary\nexpansion techniques which solve some common internal subroutine problems\nregarding recognition graph processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09180v1",
            "date": "2020-03-20 10:35:35+00:00",
            "title": "Detecting Mismatch between Text Script and Voice-over Using Utterance Verification Based on Phoneme Recognition Ranking",
            "authors": [
                "Yoonjae Jeong",
                "Hoon-Young Cho"
            ],
            "abstract": "The purpose of this study is to detect the mismatch between text script and\nvoice-over. For this, we present a novel utterance verification (UV) method,\nwhich calculates the degree of correspondence between a voice-over and the\nphoneme sequence of a script. We found that the phoneme recognition\nprobabilities of exaggerated voice-overs decrease compared to ordinary\nutterances, but their rankings do not demonstrate any significant change. The\nproposed method, therefore, uses the recognition ranking of each phoneme\nsegment corresponding to a phoneme sequence for measuring the confidence of a\nvoice-over utterance for its corresponding script. The experimental results\nshow that the proposed UV method outperforms a state-of-the-art approach using\ncross modal attention used for detecting mismatch between speech and\ntranscription.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2003.09520v2",
            "date": "2020-03-20 22:29:42+00:00",
            "title": "TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus",
            "authors": [
                "Elisa Gugliotta",
                "Marco Dinarelli"
            ],
            "abstract": "This article describes the constitution process of the first\nmorpho-syntactically annotated Tunisian Arabish Corpus (TArC). Arabish, also\nknown as Arabizi, is a spontaneous coding of Arabic dialects in Latin\ncharacters and arithmographs (numbers used as letters). This code-system was\ndeveloped by Arabic-speaking users of social media in order to facilitate the\nwriting in the Computer-Mediated Communication (CMC) and text messaging\ninformal frameworks. There is variety in the realization of Arabish amongst\ndialects, and each Arabish code-system is under-resourced, in the same way as\nmost of the Arabic dialects. In the last few years, the focus on Arabic\ndialects in the NLP field has considerably increased. Taking this into\nconsideration, TArC will be a useful support for different types of analyses,\ncomputational and linguistic, as well as for NLP tools training. In this\narticle we will describe preliminary work on the TArC semi-automatic\nconstruction process and some of the first analyses we developed on TArC. In\naddition, in order to provide a complete overview of the challenges faced\nduring the building process, we will present the main Tunisian dialect\ncharacteristics and their encoding in Tunisian Arabish.",
            "journal ref": "Proceedings of the 12th Language Resources and Evaluation\n  Conference, 2020, (6279-6286)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09530v2",
            "date": "2020-03-20 23:32:08+00:00",
            "title": "A Framework for Generating Explanations from Temporal Personal Health Data",
            "authors": [
                "Jonathan J. Harris",
                "Ching-Hua Chen",
                "Mohammed J. Zaki"
            ],
            "abstract": "Whereas it has become easier for individuals to track their personal health\ndata (e.g., heart rate, step count, food log), there is still a wide chasm\nbetween the collection of data and the generation of meaningful explanations to\nhelp users better understand what their data means to them. With an increased\ncomprehension of their data, users will be able to act upon the newfound\ninformation and work towards striving closer to their health goals. We aim to\nbridge the gap between data collection and explanation generation by mining the\ndata for interesting behavioral findings that may provide hints about a user's\ntendencies. Our focus is on improving the explainability of temporal personal\nhealth data via a set of informative summary templates, or \"protoforms.\" These\nprotoforms span both evaluation-based summaries that help users evaluate their\nhealth goals and pattern-based summaries that explain their implicit behaviors.\nIn addition to individual users, the protoforms we use are also designed for\npopulation-level summaries. We apply our approach to generate summaries (both\nunivariate and multivariate) from real user data and show that our system can\ngenerate interesting and useful explanations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09831v1",
            "date": "2020-03-22 07:27:07+00:00",
            "title": "Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding",
            "authors": [
                "Su Zhu",
                "Zijian Zhao",
                "Rao Ma",
                "Kai Yu"
            ],
            "abstract": "Traditional slot filling in natural language understanding (NLU) predicts a\none-hot vector for each word. This form of label representation lacks semantic\ncorrelation modelling, which leads to severe data sparsity problem, especially\nwhen adapting an NLU model to a new domain. To address this issue, a novel\nlabel embedding based slot filling framework is proposed in this paper. Here,\ndistributed label embedding is constructed for each slot using prior knowledge.\nThree encoding methods are investigated to incorporate different kinds of prior\nknowledge about slots: atomic concepts, slot descriptions, and slot exemplars.\nThe proposed label embeddings tend to share text patterns and reuses data with\ndifferent slot labels. This makes it useful for adaptive NLU with limited data.\nAlso, since label embedding is independent of NLU model, it is compatible with\nalmost all deep learning based slot filling models. The proposed approaches are\nevaluated on three datasets. Experiments on single domain and domain adaptation\ntasks show that label embedding achieves significant performance improvement\nover traditional one-hot label representation as well as advanced zero-shot\napproaches.",
            "journal ref": "TASLP, vol. 28, pp. 1440-1451, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09833v3",
            "date": "2020-03-22 07:58:44+00:00",
            "title": "SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection",
            "authors": [
                "Xiaoya Li",
                "Yuxian Meng",
                "Mingxin Zhou",
                "Qinghong Han",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "While the self-attention mechanism has been widely used in a wide variety of\ntasks, it has the unfortunate property of a quadratic cost with respect to the\ninput length, which makes it difficult to deal with long inputs. In this paper,\nwe present a method for accelerating and structuring self-attentions: Sparse\nAdaptive Connection (SAC). In SAC, we regard the input sequence as a graph and\nattention operations are performed between linked nodes. In contrast with\nprevious self-attention models with pre-defined structures (edges), the model\nlearns to construct attention edges to improve task-specific performances. In\nthis way, the model is able to select the most salient nodes and reduce the\nquadratic complexity regardless of the sequence length. Based on SAC, we show\nthat previous variants of self-attention models are its special cases. Through\nextensive experiments on neural machine translation, language modeling, graph\nrepresentation learning and image classification, we demonstrate SAC is\ncompetitive with state-of-the-art models while significantly reducing memory\ncost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.09853v1",
            "date": "2020-03-22 10:26:08+00:00",
            "title": "Visual Question Answering for Cultural Heritage",
            "authors": [
                "Pietro Bongini",
                "Federico Becattini",
                "Andrew D. Bagdanov",
                "Alberto Del Bimbo"
            ],
            "abstract": "Technology and the fruition of cultural heritage are becoming increasingly\nmore entwined, especially with the advent of smart audio guides, virtual and\naugmented reality, and interactive installations. Machine learning and computer\nvision are important components of this ongoing integration, enabling new\ninteraction modalities between user and museum. Nonetheless, the most frequent\nway of interacting with paintings and statues still remains taking pictures.\nYet images alone can only convey the aesthetics of the artwork, lacking is\ninformation which is often required to fully understand and appreciate it.\nUsually this additional knowledge comes both from the artwork itself (and\ntherefore the image depicting it) and from an external source of knowledge,\nsuch as an information sheet. While the former can be inferred by computer\nvision algorithms, the latter needs more structured data to pair visual content\nwith relevant information. Regardless of its source, this information still\nmust be be effectively transmitted to the user. A popular emerging trend in\ncomputer vision is Visual Question Answering (VQA), in which users can interact\nwith a neural network by posing questions in natural language and receiving\nanswers about the visual content. We believe that this will be the evolution of\nsmart audio guides for museum visits and simple image browsing on personal\nsmartphones. This will turn the classic audio guide into a smart personal\ninstructor with which the visitor can interact by asking for explanations\nfocused on specific interests. The advantages are twofold: on the one hand the\ncognitive burden of the visitor will decrease, limiting the flow of information\nto what the user actually wants to hear; and on the other hand it proposes the\nmost natural way of interacting with a guide, favoring engagement.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.09971v2",
            "date": "2020-03-22 19:04:25+00:00",
            "title": "A Better Variant of Self-Critical Sequence Training",
            "authors": [
                "Ruotian Luo"
            ],
            "abstract": "In this work, we present a simple yet better variant of Self-Critical\nSequence Training. We make a simple change in the choice of baseline function\nin REINFORCE algorithm. The new baseline can bring better performance with no\nextra cost, compared to the greedy decoding baseline.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.09986v1",
            "date": "2020-03-22 20:18:20+00:00",
            "title": "Toward Tag-free Aspect Based Sentiment Analysis: A Multiple Attention Network Approach",
            "authors": [
                "Yao Qiang",
                "Xin Li",
                "Dongxiao Zhu"
            ],
            "abstract": "Existing aspect based sentiment analysis (ABSA) approaches leverage various\nneural network models to extract the aspect sentiments via learning\naspect-specific feature representations. However, these approaches heavily rely\non manual tagging of user reviews according to the predefined aspects as the\ninput, a laborious and time-consuming process. Moreover, the underlying methods\ndo not explain how and why the opposing aspect level polarities in a user\nreview lead to the overall polarity. In this paper, we tackle these two\nproblems by designing and implementing a new Multiple-Attention Network (MAN)\napproach for more powerful ABSA without the need for aspect tags using two new\ntag-free data sets crawled directly from TripAdvisor\n({https://www.tripadvisor.com}). With the Self- and Position-Aware attention\nmechanism, MAN is capable of extracting both aspect level and overall\nsentiments from the text reviews using the aspect level and overall customer\nratings, and it can also detect the vital aspect(s) leading to the overall\nsentiment polarity among different aspects via a new aspect ranking scheme. We\ncarry out extensive experiments to demonstrate the strong performance of MAN\ncompared to other state-of-the-art ABSA approaches and the explainability of\nour approach by visualizing and interpreting attention weights in case studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10066v1",
            "date": "2020-03-23 03:44:56+00:00",
            "title": "Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments",
            "authors": [
                "Koichiro Yoshino",
                "Kohei Wakimoto",
                "Yuta Nishimura",
                "Satoshi Nakamura"
            ],
            "abstract": "Bridging robot action sequences and their natural language captions is an\nimportant task to increase explainability of human assisting robots in their\nrecently evolving field. In this paper, we propose a system for generating\nnatural language captions that describe behaviors of human assisting robots.\nThe system describes robot actions by using robot observations; histories from\nactuator systems and cameras, toward end-to-end bridging between robot actions\nand natural language captions. Two reasons make it challenging to apply\nexisting sequence-to-sequence models to this mapping: 1) it is hard to prepare\na large-scale dataset for any kind of robots and their environment, and 2)\nthere is a gap between the number of samples obtained from robot action\nobservations and generated word sequences of captions. We introduced\nunsupervised segmentation based on K-means clustering to unify typical robot\nobservation patterns into a class. This method makes it possible for the\nnetwork to learn the relationship from a small amount of data. Moreover, we\nutilized a chunking method based on byte-pair encoding (BPE) to fill in the gap\nbetween the number of samples of robot action observations and words in a\ncaption. We also applied an attention mechanism to the segmentation task.\nExperimental results show that the proposed model based on unsupervised\nlearning can generate better descriptions than other methods. We also show that\nthe attention mechanism did not work well in our low-resource setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10286v1",
            "date": "2020-03-07 17:55:41+00:00",
            "title": "PathVQA: 30000+ Questions for Medical Visual Question Answering",
            "authors": [
                "Xuehai He",
                "Yichen Zhang",
                "Luntian Mou",
                "Eric Xing",
                "Pengtao Xie"
            ],
            "abstract": "Is it possible to develop an \"AI Pathologist\" to pass the board-certified\nexamination of the American Board of Pathology? To achieve this goal, the first\nstep is to create a visual question answering (VQA) dataset where the AI agent\nis presented with a pathology image together with a question and is asked to\ngive the correct answer. Our work makes the first attempt to build such a\ndataset. Different from creating general-domain VQA datasets where the images\nare widely accessible and there are many crowdsourcing workers available and\ncapable of generating question-answer pairs, developing a medical VQA dataset\nis much more challenging. First, due to privacy concerns, pathology images are\nusually not publicly available. Second, only well-trained pathologists can\nunderstand pathology images, but they barely have time to help create datasets\nfor AI research. To address these challenges, we resort to pathology textbooks\nand online digital libraries. We develop a semi-automated pipeline to extract\npathology images and captions from textbooks and generate question-answer pairs\nfrom captions using natural language processing. We collect 32,799 open-ended\nquestions from 4,998 pathology images where each question is manually checked\nto ensure correctness. To our best knowledge, this is the first dataset for\npathology VQA. Our dataset will be released publicly to promote research in\nmedical VQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10606v1",
            "date": "2020-03-24 01:31:14+00:00",
            "title": "Video Object Grounding using Semantic Roles in Language Description",
            "authors": [
                "Arka Sadhu",
                "Kan Chen",
                "Ram Nevatia"
            ],
            "abstract": "We explore the task of Video Object Grounding (VOG), which grounds objects in\nvideos referred to in natural language descriptions. Previous methods apply\nimage grounding based algorithms to address VOG, fail to explore the object\nrelation information and suffer from limited generalization. Here, we\ninvestigate the role of object relations in VOG and propose a novel framework\nVOGNet to encode multi-modal object relations via self-attention with relative\nposition encoding. To evaluate VOGNet, we propose novel contrasting sampling\nmethods to generate more challenging grounding input samples, and construct a\nnew dataset called ActivityNet-SRL (ASRL) based on existing caption and\ngrounding datasets. Experiments on ASRL validate the need of encoding object\nrelations in VOG, and our VOGNet outperforms competitive baselines by a\nsignificant margin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.10715v1",
            "date": "2020-03-24 08:38:36+00:00",
            "title": "Investigating Software Usage in the Social Sciences: A Knowledge Graph Approach",
            "authors": [
                "David Schindler",
                "Benjamin Zapilko",
                "Frank Kr\u00fcger"
            ],
            "abstract": "Knowledge about the software used in scientific investigations is necessary\nfor different reasons, including provenance of the results, measuring software\nimpact to attribute developers, and bibliometric software citation analysis in\ngeneral. Additionally, providing information about whether and how the software\nand the source code are available allows an assessment about the state and role\nof open source software in science in general. While such analyses can be done\nmanually, large scale analyses require the application of automated methods of\ninformation extraction and linking. In this paper, we present SoftwareKG - a\nknowledge graph that contains information about software mentions from more\nthan 51,000 scientific articles from the social sciences. A silver standard\ncorpus, created by a distant and weak supervision approach, and a gold standard\ncorpus, created by manual annotation, were used to train an LSTM based neural\nnetwork to identify software mentions in scientific articles. The model\nachieves a recognition rate of .82 F-score in exact matches. As a result, we\nidentified more than 133,000 software mentions. For entity disambiguation, we\nused the public domain knowledge base DBpedia. Furthermore, we linked the\nentities of the knowledge graph to other knowledge bases such as the Microsoft\nAcademic Knowledge Graph, the Software Ontology, and Wikidata. Finally, we\nillustrate, how SoftwareKG can be used to assess the role of software in the\nsocial sciences.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.10773v1",
            "date": "2020-03-24 11:17:20+00:00",
            "title": "Generating Chinese Poetry from Images via Concrete and Abstract Information",
            "authors": [
                "Yusen Liu",
                "Dayiheng Liu",
                "Jiancheng Lv",
                "Yongsheng Sang"
            ],
            "abstract": "In recent years, the automatic generation of classical Chinese poetry has\nmade great progress. Besides focusing on improving the quality of the generated\npoetry, there is a new topic about generating poetry from an image. However,\nthe existing methods for this topic still have the problem of topic drift and\nsemantic inconsistency, and the image-poem pairs dataset is hard to be built\nwhen training these models. In this paper, we extract and integrate the\nConcrete and Abstract information from images to address those issues. We\nproposed an infilling-based Chinese poetry generation model which can infill\nthe Concrete keywords into each line of poems in an explicit way, and an\nabstract information embedding to integrate the Abstract information into\ngenerated poems. In addition, we use non-parallel data during training and\nconstruct separate image datasets and poem datasets to train the different\ncomponents in our framework. Both automatic and human evaluation results show\nthat our approach can generate poems which have better consistency with images\nwithout losing the quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.10925v1",
            "date": "2020-03-24 15:31:05+00:00",
            "title": "Learning Compact Reward for Image Captioning",
            "authors": [
                "Nannan Li",
                "Zhenzhong Chen"
            ],
            "abstract": "Adversarial learning has shown its advances in generating natural and diverse\ndescriptions in image captioning. However, the learned reward of existing\nadversarial methods is vague and ill-defined due to the reward ambiguity\nproblem. In this paper, we propose a refined Adversarial Inverse Reinforcement\nLearning (rAIRL) method to handle the reward ambiguity problem by disentangling\nreward for each word in a sentence, as well as achieve stable adversarial\ntraining by refining the loss function to shift the generator towards Nash\nequilibrium. In addition, we introduce a conditional term in the loss function\nto mitigate mode collapse and to increase the diversity of the generated\ndescriptions. Our experiments on MS COCO and Flickr30K show that our method can\nlearn compact reward for image captioning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.11080v5",
            "date": "2020-03-24 19:09:37+00:00",
            "title": "XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization",
            "authors": [
                "Junjie Hu",
                "Sebastian Ruder",
                "Aditya Siddhant",
                "Graham Neubig",
                "Orhan Firat",
                "Melvin Johnson"
            ],
            "abstract": "Much recent progress in applications of machine learning models to NLP has\nbeen driven by benchmarks that evaluate models across a wide variety of tasks.\nHowever, these broad-coverage benchmarks have been mostly limited to English,\nand despite an increasing interest in multilingual models, a benchmark that\nenables the comprehensive evaluation of such methods on a diverse range of\nlanguages and tasks is still missing. To this end, we introduce the\nCross-lingual TRansfer Evaluation of Multilingual Encoders XTREME benchmark, a\nmulti-task benchmark for evaluating the cross-lingual generalization\ncapabilities of multilingual representations across 40 languages and 9 tasks.\nWe demonstrate that while models tested on English reach human performance on\nmany tasks, there is still a sizable gap in the performance of cross-lingually\ntransferred models, particularly on syntactic and sentence retrieval tasks.\nThere is also a wide spread of results across languages. We release the\nbenchmark to encourage research on cross-lingual learning methods that transfer\nlinguistic knowledge across a diverse and representative set of languages and\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11105v1",
            "date": "2020-03-19 03:32:04+00:00",
            "title": "EQL -- an extremely easy to learn knowledge graph query language, achieving highspeed and precise search",
            "authors": [
                "Han Liu",
                "Shantao Liu"
            ],
            "abstract": "EQL, also named as Extremely Simple Query Language, can be widely used in the\nfield of knowledge graph, precise search, strong artificial intelligence,\ndatabase, smart speaker ,patent search and other fields. EQL adopt the\nprinciple of minimalism in design and pursues simplicity and easy to learn so\nthat everyone can master it quickly. EQL language and lambda calculus are\ninterconvertible, that reveals the mathematical nature of EQL language, and\nlays a solid foundation for rigor and logical integrity of EQL language. The\nEQL language and a comprehensive knowledge graph system with the world's\ncommonsense can together form the foundation of strong AI in the future, and\nmake up for the current lack of understanding of world's commonsense by current\nAI system. EQL language can be used not only by humans, but also as a basic\nlanguage for data query and data exchange between robots.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/2003.11173v1",
            "date": "2020-03-25 01:29:38+00:00",
            "title": "Learning Syntactic and Dynamic Selective Encoding for Document Summarization",
            "authors": [
                "Haiyang Xu",
                "Yahao He",
                "Kun Han",
                "Junwen Chen",
                "Xiangang Li"
            ],
            "abstract": "Text summarization aims to generate a headline or a short summary consisting\nof the major information of the source text. Recent studies employ the\nsequence-to-sequence framework to encode the input with a neural network and\ngenerate abstractive summary. However, most studies feed the encoder with the\nsemantic word embedding but ignore the syntactic information of the text.\nFurther, although previous studies proposed the selective gate to control the\ninformation flow from the encoder to the decoder, it is static during the\ndecoding and cannot differentiate the information based on the decoder states.\nIn this paper, we propose a novel neural architecture for document\nsummarization. Our approach has the following contributions: first, we\nincorporate syntactic information such as constituency parsing trees into the\nencoding sequence to learn both the semantic and syntactic information from the\ndocument, resulting in more accurate summary; second, we propose a dynamic gate\nnetwork to select the salient information based on the context of the decoder\nstate, which is essential to document summarization. The proposed model has\nbeen evaluated on CNN/Daily Mail summarization datasets and the experimental\nresults show that the proposed approach outperforms baseline approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11184v1",
            "date": "2020-03-25 02:19:17+00:00",
            "title": "Adversarial Multi-Binary Neural Network for Multi-class Classification",
            "authors": [
                "Haiyang Xu",
                "Junwen Chen",
                "Kun Han",
                "Xiangang Li"
            ],
            "abstract": "Multi-class text classification is one of the key problems in machine\nlearning and natural language processing. Emerging neural networks deal with\nthe problem using a multi-output softmax layer and achieve substantial\nprogress, but they do not explicitly learn the correlation among classes. In\nthis paper, we use a multi-task framework to address multi-class\nclassification, where a multi-class classifier and multiple binary classifiers\nare trained together. Moreover, we employ adversarial training to distinguish\nthe class-specific features and the class-agnostic features. The model benefits\nfrom better feature representation. We conduct experiments on two large-scale\nmulti-class text classification tasks and demonstrate that the proposed\narchitecture outperforms baseline approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11517v1",
            "date": "2020-03-11 20:55:01+00:00",
            "title": "From Algebraic Word Problem to Program: A Formalized Approach",
            "authors": [
                "Adam Wiemerslage",
                "Shafiuddin Rehan Ahmed"
            ],
            "abstract": "In this paper, we propose a pipeline to convert grade school level algebraic\nword problem into program of a formal languageA-IMP. Using natural language\nprocessing tools, we break the problem into sentence fragments which can then\nbe reduced to functions. The functions are categorized by the head verb of the\nsentence and its structure, as defined by (Hosseini et al., 2014). We define\nthe function signature and extract its arguments from the text using dependency\nparsing. We have a working implementation of the entire pipeline which can be\nfound on our github repository.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11518v2",
            "date": "2020-03-10 13:05:52+00:00",
            "title": "Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction",
            "authors": [
                "Yan Xiao",
                "Yaochu Jin",
                "Ran Cheng",
                "Kuangrong Hao"
            ],
            "abstract": "With an exponential explosive growth of various digital text information, it\nis challenging to efficiently obtain specific knowledge from massive\nunstructured text information. As one basic task for natural language\nprocessing (NLP), relation extraction aims to extract the semantic relation\nbetween entity pairs based on the given text. To avoid manual labeling of\ndatasets, distant supervision relation extraction (DSRE) has been widely used,\naiming to utilize knowledge base to automatically annotate datasets.\nUnfortunately, this method heavily suffers from wrong labelling due to the\nunderlying strong assumptions. To address this issue, we propose a new\nframework using hybrid attention-based Transformer block with multi-instance\nlearning to perform the DSRE task. More specifically, the Transformer block is\nfirstly used as the sentence encoder to capture syntactic information of\nsentences, which mainly utilizes multi-head self-attention to extract features\nfrom word level. Then, a more concise sentence-level attention mechanism is\nadopted to constitute the bag representation, aiming to incorporate valid\ninformation of each sentence to effectively represent the bag. Experimental\nresults on the public dataset New York Times (NYT) demonstrate that the\nproposed approach can outperform the state-of-the-art algorithms on the\nevaluation dataset, which verifies the effectiveness of our model for the DSRE\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.11519v3",
            "date": "2020-03-09 20:56:36+00:00",
            "title": "Vector logic allows counterfactual virtualization by The Square Root of NOT",
            "authors": [
                "Eduardo Mizraji"
            ],
            "abstract": "In this work we investigate the representation of counterfactual conditionals\nusing the vector logic, a matrix-vectors formalism for logical functions and\ntruth values. Inside this formalism, the counterfactuals can be transformed in\ncomplex matrices preprocessing an implication matrix with one of the square\nroots of NOT, a complex matrix. This mathematical approach puts in evidence the\nvirtual character of the counterfactuals. This happens because this\nrepresentation produces a valuation of a counterfactual that is the\nsuperposition of the two opposite truth values weighted, respectively, by two\ncomplex conjugated coefficients. This result shows that this procedure gives an\nuncertain evaluation projected on the complex domain. After this basic\nrepresentation, the judgment of the plausibility of a given counterfactual\nallows us to shift the decision towards an acceptance or a refusal. This shift\nis the result of applying for a second time one of the two square roots of NOT.",
            "journal ref": "Logic Journal of IGPL-Published online on July 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12145v2",
            "date": "2020-03-13 22:11:39+00:00",
            "title": "Knowledge Graph Alignment using String Edit Distance",
            "authors": [
                "Navdeep Kaur",
                "Gautam Kunapuli",
                "Sriraam Natarajan"
            ],
            "abstract": "In this work, we propose a novel knowledge graph alignment technique based\nupon string edit distance that exploits the type information between entities\nand can find similarity between relations of any arity",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.12462v2",
            "date": "2020-03-24 02:38:35+00:00",
            "title": "TextCaps: a Dataset for Image Captioning with Reading Comprehension",
            "authors": [
                "Oleksii Sidorov",
                "Ronghang Hu",
                "Marcus Rohrbach",
                "Amanpreet Singh"
            ],
            "abstract": "Image descriptions can help visually impaired people to quickly understand\nthe image content. While we made significant progress in automatically\ndescribing images and optical character recognition, current approaches are\nunable to include written text in their descriptions, although text is\nomnipresent in human environments and frequently critical to understand our\nsurroundings. To study how to comprehend text in the context of an image we\ncollect a novel dataset, TextCaps, with 145k captions for 28k images. Our\ndataset challenges a model to recognize text, relate it to its visual context,\nand decide what part of the text to copy or paraphrase, requiring spatial,\nsemantic, and visual reasoning between multiple text tokens and visual\nentities, such as objects. We study baselines and adapt existing approaches to\nthis new task, which we refer to as image captioning with reading\ncomprehension. Our analysis with automatic and human studies shows that our new\nTextCaps dataset provides many new technical challenges over previous datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2003.12660v1",
            "date": "2020-03-27 22:40:01+00:00",
            "title": "Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgin",
            "authors": [
                "Orevaoghene Ahia",
                "Kelechi Ogueji"
            ],
            "abstract": "Nigerian Pidgin is arguably the most widely spoken language in Nigeria.\nVariants of this language are also spoken across West and Central Africa,\nmaking it a very important language. This work aims to establish supervised and\nunsupervised neural machine translation (NMT) baselines between English and\nNigerian Pidgin. We implement and compare NMT models with different\ntokenization methods, creating a solid foundation for future works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12694v3",
            "date": "2020-03-28 03:45:31+00:00",
            "title": "Countering Language Drift with Seeded Iterated Learning",
            "authors": [
                "Yuchen Lu",
                "Soumye Singhal",
                "Florian Strub",
                "Olivier Pietquin",
                "Aaron Courville"
            ],
            "abstract": "Pretraining on human corpus and then finetuning in a simulator has become a\nstandard pipeline for training a goal-oriented dialogue agent. Nevertheless, as\nsoon as the agents are finetuned to maximize task completion, they suffer from\nthe so-called language drift phenomenon: they slowly lose syntactic and\nsemantic properties of language as they only focus on solving the task. In this\npaper, we propose a generic approach to counter language drift called Seeded\niterated learning (SIL). We periodically refine a pretrained student agent by\nimitating data sampled from a newly generated teacher agent. At each time step,\nthe teacher is created by copying the student agent, before being finetuned to\nmaximize task completion. SIL does not require external syntactic constraint\nnor semantic knowledge, making it a valuable task-agnostic finetuning protocol.\nWe evaluate SIL in a toy-setting Lewis Game, and then scale it up to the\ntranslation game with natural language. In both settings, SIL helps counter\nlanguage drift as well as it improves the task completion compared to\nbaselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.12718v3",
            "date": "2020-03-28 05:47:33+00:00",
            "title": "Mining Implicit Entity Preference from User-Item Interaction Data for Knowledge Graph Completion via Adversarial Learning",
            "authors": [
                "Gaole He",
                "Junyi Li",
                "Wayne Xin Zhao",
                "Peiju Liu",
                "Ji-Rong Wen"
            ],
            "abstract": "The task of Knowledge Graph Completion (KGC) aims to automatically infer the\nmissing fact information in Knowledge Graph (KG). In this paper, we take a new\nperspective that aims to leverage rich user-item interaction data (user\ninteraction data for short) for improving the KGC task. Our work is inspired by\nthe observation that many KG entities correspond to online items in application\nsystems. However, the two kinds of data sources have very different intrinsic\ncharacteristics, and it is likely to hurt the original performance using simple\nfusion strategy. To address this challenge, we propose a novel adversarial\nlearning approach by leveraging user interaction data for the KGC task. Our\ngenerator is isolated from user interaction data, and serves to improve the\nperformance of the discriminator. The discriminator takes the learned useful\ninformation from user interaction data as input, and gradually enhances the\nevaluation capacity in order to identify the fake samples generated by the\ngenerator. To discover implicit entity preference of users, we design an\nelaborate collaborative learning algorithms based on graph neural networks,\nwhich will be jointly optimized with the discriminator. Such an approach is\neffective to alleviate the issues about data heterogeneity and semantic\ncomplexity for the KGC task. Extensive experiments on three real-world datasets\nhave demonstrated the effectiveness of our approach on the KGC task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2003.12799v1",
            "date": "2020-03-28 14:31:01+00:00",
            "title": "Unsupervised feature learning for speech using correspondence and Siamese networks",
            "authors": [
                "Petri-Johan Last",
                "Herman A. Engelbrecht",
                "Herman Kamper"
            ],
            "abstract": "In zero-resource settings where transcribed speech audio is unavailable,\nunsupervised feature learning is essential for downstream speech processing\ntasks. Here we compare two recent methods for frame-level acoustic feature\nlearning. For both methods, unsupervised term discovery is used to find pairs\nof word examples of the same unknown type. Dynamic programming is then used to\nalign the feature frames between each word pair, serving as weak top-down\nsupervision for the two models. For the correspondence autoencoder (CAE),\nmatching frames are presented as input-output pairs. The Triamese network uses\na contrastive loss to reduce the distance between frames of the same predicted\nword type while increasing the distance between negative examples. For the\nfirst time, these feature extractors are compared on the same discrimination\ntasks using the same weak supervision pairs. We find that, on the two datasets\nconsidered here, the CAE outperforms the Triamese network. However, we show\nthat a new hybrid correspondence-Triamese approach (CTriamese), consistently\noutperforms both the CAE and Triamese models in terms of average precision and\nABX error rates on both English and Xitsonga evaluation data.",
            "journal ref": "IEEE Signal Processing Letters 27 (2020) 421-425",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.12900v1",
            "date": "2020-03-28 22:10:48+00:00",
            "title": "Orchestrating NLP Services for the Legal Domain",
            "authors": [
                "Juli\u00e1n Moreno-Schneider",
                "Georg Rehm",
                "Elena Montiel-Ponsoda",
                "V\u00edctor Rodriguez-Doncel",
                "Artem Revenko",
                "Sotirios Karampatakis",
                "Maria Khvalchik",
                "Christian Sageder",
                "Jorge Gracia",
                "Filippo Maganza"
            ],
            "abstract": "Legal technology is currently receiving a lot of attention from various\nangles. In this contribution we describe the main technical components of a\nsystem that is currently under development in the European innovation project\nLynx, which includes partners from industry and research. The key contribution\nof this paper is a workflow manager that enables the flexible orchestration of\nworkflows based on a portfolio of Natural Language Processing and Content\nCuration services as well as a Multilingual Legal Knowledge Graph that contains\nsemantic information and meaningful references to legal documents. We also\ndescribe different use cases with which we experiment and develop prototypical\nsolutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13003v2",
            "date": "2020-03-29 11:27:10+00:00",
            "title": "Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining",
            "authors": [
                "Chengyu Wang",
                "Minghui Qiu",
                "Jun Huang",
                "Xiaofeng He"
            ],
            "abstract": "Pre-trained neural language models bring significant improvement for various\nNLP tasks, by fine-tuning the models on task-specific training sets. During\nfine-tuning, the parameters are initialized from pre-trained models directly,\nwhich ignores how the learning process of similar NLP tasks in different\ndomains is correlated and mutually reinforced. In this paper, we propose an\neffective learning procedure named Meta Fine-Tuning (MFT), served as a\nmeta-learner to solve a group of similar NLP tasks for neural language models.\nInstead of simply multi-task training over all the datasets, MFT only learns\nfrom typical instances of various domains to acquire highly transferable\nknowledge. It further encourages the language model to encode domain-invariant\nrepresentations by optimizing a series of novel domain corruption loss\nfunctions. After MFT, the model can be fine-tuned for each domain with better\nparameter initializations and higher generalization ability. We implement MFT\nupon BERT to solve several multi-domain text mining tasks. Experimental results\nconfirm the effectiveness of MFT and its usefulness for few-shot learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13016v1",
            "date": "2020-03-29 13:20:43+00:00",
            "title": "A Dataset of German Legal Documents for Named Entity Recognition",
            "authors": [
                "Elena Leitner",
                "Georg Rehm",
                "Juli\u00e1n Moreno-Schneider"
            ],
            "abstract": "We describe a dataset developed for Named Entity Recognition in German\nfederal court decisions. It consists of approx. 67,000 sentences with over 2\nmillion tokens. The resource contains 54,000 manually annotated entities,\nmapped to 19 fine-grained semantic classes: person, judge, lawyer, country,\ncity, street, landscape, organization, company, institution, court, brand, law,\nordinance, European legal norm, regulation, contract, court decision, and legal\nliterature. The legal documents were, furthermore, automatically annotated with\nmore than 35,000 TimeML-based time expressions. The dataset, which is available\nunder a CC-BY 4.0 license in the CoNNL-2002 format, was developed for training\nan NER service for German legal documents in the EU project Lynx.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13027v1",
            "date": "2020-03-29 14:00:17+00:00",
            "title": "Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling",
            "authors": [
                "Dmitrii Aksenov",
                "Juli\u00e1n Moreno-Schneider",
                "Peter Bourgonje",
                "Robert Schwarzenberg",
                "Leonhard Hennig",
                "Georg Rehm"
            ],
            "abstract": "We explore to what extent knowledge about the pre-trained language model that\nis used is beneficial for the task of abstractive summarization. To this end,\nwe experiment with conditioning the encoder and decoder of a Transformer-based\nneural model on the BERT language model. In addition, we propose a new method\nof BERT-windowing, which allows chunk-wise processing of texts longer than the\nBERT window size. We also explore how locality modelling, i.e., the explicit\nrestriction of calculations to the local context, can affect the summarization\nability of the Transformer. This is done by introducing 2-dimensional\nconvolutional self-attention into the first layers of the encoder. The results\nof our models are compared to a baseline and the state-of-the-art models on the\nCNN/Daily Mail dataset. We additionally train our model on the SwissText\ndataset to demonstrate usability on German. Both models outperform the baseline\nin ROUGE scores on two datasets and show its superiority in a manual\nqualitative analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13118v2",
            "date": "2020-03-29 19:25:32+00:00",
            "title": "Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement",
            "authors": [
                "Alireza Mohammadshahi",
                "James Henderson"
            ],
            "abstract": "We propose the Recursive Non-autoregressive Graph-to-Graph Transformer\narchitecture (RNGTr) for the iterative refinement of arbitrary graphs through\nthe recursive application of a non-autoregressive Graph-to-Graph Transformer\nand apply it to syntactic dependency parsing. We demonstrate the power and\neffectiveness of RNGTr on several dependency corpora, using a refinement model\npre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a\nnon-recursive parser similar to our refinement model. RNGTr can improve the\naccuracy of a variety of initial parsers on 13 languages from the Universal\nDependencies Treebanks, English and Chinese Penn Treebanks, and the German\nCoNLL2009 corpus, even improving over the new state-of-the-art results achieved\nby SynTr, significantly improving the state-of-the-art for all corpora tested.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13198v4",
            "date": "2020-03-30 03:13:22+00:00",
            "title": "InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining",
            "authors": [
                "Junyang Lin",
                "An Yang",
                "Yichang Zhang",
                "Jie Liu",
                "Jingren Zhou",
                "Hongxia Yang"
            ],
            "abstract": "Multi-modal pretraining for learning high-level multi-modal representation is\na further step towards deep learning and artificial intelligence. In this work,\nwe propose a novel model, namely InterBERT (BERT for Interaction), which is the\nfirst model of our series of multimodal pretraining methods M6\n(MultiModality-to-MultiModality Multitask Mega-transformer). The model owns\nstrong capability of modeling interaction between the information flows of\ndifferent modalities. The single-stream interaction module is capable of\neffectively processing information of multiple modalilties, and the two-stream\nmodule on top preserves the independence of each modality to avoid performance\ndowngrade in single-modal tasks. We pretrain the model with three pretraining\ntasks, including masked segment modeling (MSM), masked region modeling (MRM)\nand image-text matching (ITM); and finetune the model on a series of\nvision-and-language downstream tasks. Experimental results demonstrate that\nInterBERT outperforms a series of strong baselines, including the most recent\nmulti-modal pretraining methods, and the analysis shows that MSM and MRM are\neffective for pretraining and our method can achieve performances comparable to\nBERT in single-modal tasks. Besides, we propose a large-scale dataset for\nmulti-modal pretraining in Chinese, and we develop the Chinese InterBERT which\nis the first Chinese multi-modal pretrained model. We pretrain the Chinese\nInterBERT on our proposed dataset of 3.1M image-text pairs from the mobile\nTaobao, the largest Chinese e-commerce platform. We finetune the model for\ntext-based image retrieval, and recently we deployed the model online for\ntopic-based recommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13230v1",
            "date": "2020-03-30 05:42:03+00:00",
            "title": "AliCoCo: Alibaba E-commerce Cognitive Concept Net",
            "authors": [
                "Xusheng Luo",
                "Luxin Liu",
                "Yonghua Yang",
                "Le Bo",
                "Yuanpeng Cao",
                "Jinhang Wu",
                "Qiang Li",
                "Keping Yang",
                "Kenny Q. Zhu"
            ],
            "abstract": "One of the ultimate goals of e-commerce platforms is to satisfy various\nshopping needs for their customers. Much efforts are devoted to creating\ntaxonomies or ontologies in e-commerce towards this goal. However, user needs\nin e-commerce are still not well defined, and none of the existing ontologies\nhas the enough depth and breadth for universal user needs understanding. The\nsemantic gap in-between prevents shopping experience from being more\nintelligent. In this paper, we propose to construct a large-scale e-commerce\ncognitive concept net named \"AliCoCo\", which is practiced in Alibaba, the\nlargest Chinese e-commerce platform in the world. We formally define user needs\nin e-commerce, then conceptualize them as nodes in the net. We present details\non how AliCoCo is constructed semi-automatically and its successful, ongoing\nand potential applications in e-commerce.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.13236v1",
            "date": "2020-03-30 06:46:35+00:00",
            "title": "Making Metadata Fit for Next Generation Language Technology Platforms: The Metadata Schema of the European Language Grid",
            "authors": [
                "Penny Labropoulou",
                "Katerina Gkirtzou",
                "Maria Gavriilidou",
                "Miltos Deligiannis",
                "Dimitrios Galanis",
                "Stelios Piperidis",
                "Georg Rehm",
                "Maria Berger",
                "Val\u00e9rie Mapelli",
                "Micka\u00ebl Rigault",
                "Victoria Arranz",
                "Khalid Choukri",
                "Gerhard Backfried",
                "Jos\u00e9 Manuel G\u00f3mez P\u00e9rez",
                "Andres Garcia Silva"
            ],
            "abstract": "The current scientific and technological landscape is characterised by the\nincreasing availability of data resources and processing tools and services. In\nthis setting, metadata have emerged as a key factor facilitating management,\nsharing and usage of such digital assets. In this paper we present ELG-SHARE, a\nrich metadata schema catering for the description of Language Resources and\nTechnologies (processing and generation services and tools, models, corpora,\nterm lists, etc.), as well as related entities (e.g., organizations, projects,\nsupporting documents, etc.). The schema powers the European Language Grid\nplatform that aims to be the primary hub and marketplace for industry-relevant\nLanguage Technology in Europe. ELG-SHARE has been based on various metadata\nschemas, vocabularies, and ontologies, as well as related recommendations and\nguidelines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13316v1",
            "date": "2020-03-30 10:06:36+00:00",
            "title": "How human judgment impairs automated deception detection performance",
            "authors": [
                "Bennett Kleinberg",
                "Bruno Verschuere"
            ],
            "abstract": "Background: Deception detection is a prevalent problem for security\npractitioners. With a need for more large-scale approaches, automated methods\nusing machine learning have gained traction. However, detection performance\nstill implies considerable error rates. Findings from other domains suggest\nthat hybrid human-machine integrations could offer a viable path in deception\ndetection tasks. Method: We collected a corpus of truthful and deceptive\nanswers about participants' autobiographical intentions (n=1640) and tested\nwhether a combination of supervised machine learning and human judgment could\nimprove deception detection accuracy. Human judges were presented with the\noutcome of the automated credibility judgment of truthful and deceptive\nstatements. They could either fully overrule it (hybrid-overrule condition) or\nadjust it within a given boundary (hybrid-adjust condition). Results: The data\nsuggest that in neither of the hybrid conditions did the human judgment add a\nmeaningful contribution. Machine learning in isolation identified truth-tellers\nand liars with an overall accuracy of 69%. Human involvement through\nhybrid-overrule decisions brought the accuracy back to the chance level. The\nhybrid-adjust condition did not deception detection performance. The\ndecision-making strategies of humans suggest that the truth bias - the tendency\nto assume the other is telling the truth - could explain the detrimental\neffect. Conclusion: The current study does not support the notion that humans\ncan meaningfully add to the deception detection performance of a machine\nlearning system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13519v1",
            "date": "2020-03-30 14:45:51+00:00",
            "title": "QRMine: A python package for triangulation in Grounded Theory",
            "authors": [
                "Bell Raj Eapen",
                "Norm Archer",
                "Kamran Sartipi"
            ],
            "abstract": "Grounded theory (GT) is a qualitative research method for building theory\ngrounded in data. GT uses textual and numeric data and follows various stages\nof coding or tagging data for sense-making, such as open coding and selective\ncoding. Machine Learning (ML) techniques, including natural language processing\n(NLP), can assist the researchers in the coding process. Triangulation is the\nprocess of combining various types of data. ML can facilitate deriving insights\nfrom numerical data for corroborating findings from the textual interview\ntranscripts. We present an open-source python package (QRMine) that\nencapsulates various ML and NLP libraries to support coding and triangulation\nin GT. QRMine enables researchers to use these methods on their data with\nminimal effort. Researchers can install QRMine from the python package index\n(PyPI) and can contribute to its development. We believe that the concept of\ncomputational triangulation will make GT relevant in the realm of big data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13600v2",
            "date": "2020-03-30 16:30:47+00:00",
            "title": "AriEL: volume coding for sentence generation",
            "authors": [
                "Luca Celotti",
                "Simon Brodeur",
                "Jean Rouat"
            ],
            "abstract": "Mapping sequences of discrete data to a point in a continuous space makes it\ndifficult to retrieve those sequences via random sampling. Mapping the input to\na volume would make it easier to retrieve at test time, and that's the strategy\nfollowed by the family of approaches based on Variational Autoencoder. However\nthe fact that they are at the same time optimizing for prediction and for\nsmoothness of representation, forces them to trade-off between the two. We\nimprove on the performance of some of the standard methods in deep learning to\ngenerate sentences by uniformly sampling a continuous space. We do it by\nproposing AriEL, that constructs volumes in a continuous space, without the\nneed of encouraging the creation of volumes through the loss function. We first\nbenchmark on a toy grammar, that allows to automatically evaluate the language\nlearned and generated by the models. Then, we benchmark on a real dataset of\nhuman dialogues. Our results indicate that the random access to the stored\ninformation is dramatically improved, and our method AriEL is able to generate\na wider variety of correct language by randomly sampling the latent space. VAE\nfollows in performance for the toy dataset while, AE and Transformer follow for\nthe real dataset. This partially supports to the hypothesis that encoding\ninformation into volumes instead of into points, can lead to improved retrieval\nof learned information with random sampling. This can lead to better generators\nand we also discuss potential disadvantages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13657v3",
            "date": "2020-03-30 17:44:42+00:00",
            "title": "Analysing the Extent of Misinformation in Cancer Related Tweets",
            "authors": [
                "Rakesh Bal",
                "Sayan Sinha",
                "Swastika Dutta",
                "Rishabh Joshi",
                "Sayan Ghosh",
                "Ritam Dutt"
            ],
            "abstract": "Twitter has become one of the most sought after places to discuss a wide\nvariety of topics, including medically relevant issues such as cancer. This\nhelps spread awareness regarding the various causes, cures and prevention\nmethods of cancer. However, no proper analysis has been performed, which\ndiscusses the validity of such claims. In this work, we aim to tackle the\nmisinformation spread in such platforms. We collect and present a dataset\nregarding tweets which talk specifically about cancer and propose an\nattention-based deep learning model for automated detection of misinformation\nalong with its spread. We then do a comparative analysis of the linguistic\nvariation in the text corresponding to misinformation and truth. This analysis\nhelps us gather relevant insights on various social aspects related to\nmisinformed tweets.",
            "journal ref": "ICWSM 2020, 14, 924-928",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2003.13721v1",
            "date": "2020-03-30 18:15:32+00:00",
            "title": "Amharic Abstractive Text Summarization",
            "authors": [
                "Amr M. Zaki",
                "Mahmoud I. Khalil",
                "Hazem M. Abbas"
            ],
            "abstract": "Text Summarization is the task of condensing long text into just a handful of\nsentences. Many approaches have been proposed for this task, some of the very\nfirst were building statistical models (Extractive Methods) capable of\nselecting important words and copying them to the output, however these models\nlacked the ability to paraphrase sentences, as they simply select important\nwords without actually understanding their contexts nor understanding their\nmeaning, here comes the use of Deep Learning based architectures (Abstractive\nMethods), which effectively tries to understand the meaning of sentences to\nbuild meaningful summaries. In this work we discuss one of these new novel\napproaches which combines curriculum learning with Deep Learning, this model is\ncalled Scheduled Sampling. We apply this work to one of the most widely spoken\nAfrican languages which is the Amharic Language, as we try to enrich the\nAfrican NLP community with top-notch Deep Learning architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.13841v1",
            "date": "2020-03-30 22:07:22+00:00",
            "title": "A Hierarchical Transformer for Unsupervised Parsing",
            "authors": [
                "Ashok Thillaisundaram"
            ],
            "abstract": "The underlying structure of natural language is hierarchical; words combine\ninto phrases, which in turn form clauses. An awareness of this hierarchical\nstructure can aid machine learning models in performing many linguistic tasks.\nHowever, most such models just process text sequentially and there is no bias\ntowards learning hierarchical structure encoded into their architecture. In\nthis paper, we extend the recent transformer model (Vaswani et al., 2017) by\nenabling it to learn hierarchical representations. To achieve this, we adapt\nthe ordering mechanism introduced in Shen et al., 2018, to the self-attention\nmodule of the transformer architecture. We train our new model on language\nmodelling and then apply it to the task of unsupervised parsing. We achieve\nreasonable results on the freely available subset of the WSJ10 dataset with an\nF1-score of about 50%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2003.13956v1",
            "date": "2020-03-31 05:12:31+00:00",
            "title": "SPARQA: Skeleton-based Semantic Parsing for Complex Questions over Knowledge Bases",
            "authors": [
                "Yawei Sun",
                "Lingling Zhang",
                "Gong Cheng",
                "Yuzhong Qu"
            ],
            "abstract": "Semantic parsing transforms a natural language question into a formal query\nover a knowledge base. Many existing methods rely on syntactic parsing like\ndependencies. However, the accuracy of producing such expressive formalisms is\nnot satisfying on long complex questions. In this paper, we propose a novel\nskeleton grammar to represent the high-level structure of a complex question.\nThis dedicated coarse-grained formalism with a BERT-based parsing algorithm\nhelps to improve the accuracy of the downstream fine-grained semantic parsing.\nBesides, to align the structure of a question with the structure of a knowledge\nbase, our multi-strategy method combines sentence-level and word-level\nsemantics. Our approach shows promising performance on several datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14056v3",
            "date": "2020-03-31 09:48:25+00:00",
            "title": "Understanding Cross-Lingual Syntactic Transfer in Multilingual Recurrent Neural Networks",
            "authors": [
                "Prajit Dhar",
                "Arianna Bisazza"
            ],
            "abstract": "It is now established that modern neural language models can be successfully\ntrained on multiple languages simultaneously without changes to the underlying\narchitecture. But what kind of knowledge is really shared among languages\nwithin these models? Does multilingual training mostly lead to an alignment of\nthe lexical representation spaces or does it also enable the sharing of purely\ngrammatical knowledge? In this paper we dissect different forms of\ncross-lingual transfer and look for its most determining factors, using a\nvariety of models and probing tasks. We find that exposing our LMs to a related\nlanguage does not always increase grammatical knowledge in the target language,\nand that optimal conditions for lexical-semantic transfer may not be optimal\nfor syntactic transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2003.14292v1",
            "date": "2020-03-31 15:27:31+00:00",
            "title": "Graph Enhanced Representation Learning for News Recommendation",
            "authors": [
                "Suyu Ge",
                "Chuhan Wu",
                "Fangzhao Wu",
                "Tao Qi",
                "Yongfeng Huang"
            ],
            "abstract": "With the explosion of online news, personalized news recommendation becomes\nincreasingly important for online news platforms to help their users find\ninteresting information. Existing news recommendation methods achieve\npersonalization by building accurate news representations from news content and\nuser representations from their direct interactions with news (e.g., click),\nwhile ignoring the high-order relatedness between users and news. Here we\npropose a news recommendation method which can enhance the representation\nlearning of users and news by modeling their relatedness in a graph setting. In\nour method, users and news are both viewed as nodes in a bipartite graph\nconstructed from historical user click behaviors. For news representations, a\ntransformer architecture is first exploited to build news semantic\nrepresentations. Then we combine it with the information from neighbor news in\nthe graph via a graph attention network. For user representations, we not only\nrepresent users from their historically clicked news, but also attentively\nincorporate the representations of their neighbor users in the graph. Improved\nperformances on a large-scale real-world dataset validate the effectiveness of\nour proposed method.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2003.14324v1",
            "date": "2020-03-31 16:03:38+00:00",
            "title": "On the Integration of LinguisticFeatures into Statistical and Neural Machine Translation",
            "authors": [
                "Eva Vanmassenhove"
            ],
            "abstract": "New machine translations (MT) technologies are emerging rapidly and with\nthem, bold claims of achieving human parity such as: (i) the results produced\napproach \"accuracy achieved by average bilingual human translators\" (Wu et al.,\n2017b) or (ii) the \"translation quality is at human parity when compared to\nprofessional human translators\" (Hassan et al., 2018) have seen the light of\nday (Laubli et al., 2018). Aside from the fact that many of these papers craft\ntheir own definition of human parity, these sensational claims are often not\nsupported by a complete analysis of all aspects involved in translation.\nEstablishing the discrepancies between the strengths of statistical approaches\nto MT and the way humans translate has been the starting point of our research.\nBy looking at MT output and linguistic theory, we were able to identify some\nremaining issues. The problems range from simple number and gender agreement\nerrors to more complex phenomena such as the correct translation of aspectual\nvalues and tenses. Our experiments confirm, along with other studies\n(Bentivogli et al., 2016), that neural MT has surpassed statistical MT in many\naspects. However, some problems remain and others have emerged. We cover a\nseries of problems related to the integration of specific linguistic features\ninto statistical and neural MT, aiming to analyse and provide a solution to\nsome of them. Our work focuses on addressing three main research questions that\nrevolve around the complex relationship between linguistics and MT in general.\nWe identify linguistic information that is lacking in order for automatic\ntranslation systems to produce more accurate translations and integrate\nadditional features into the existing pipelines. We identify overgeneralization\nor 'algorithmic bias' as a potential drawback of neural MT and link it to many\nof the remaining linguistic issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00088v1",
            "date": "2020-03-31 20:21:55+00:00",
            "title": "A Clustering Framework for Lexical Normalization of Roman Urdu",
            "authors": [
                "Abdul Rafae Khan",
                "Asim Karim",
                "Hassan Sajjad",
                "Faisal Kamiran",
                "Jia Xu"
            ],
            "abstract": "Roman Urdu is an informal form of the Urdu language written in Roman script,\nwhich is widely used in South Asia for online textual content. It lacks\nstandard spelling and hence poses several normalization challenges during\nautomatic language processing. In this article, we present a feature-based\nclustering framework for the lexical normalization of Roman Urdu corpora, which\nincludes a phonetic algorithm UrduPhone, a string matching component, a\nfeature-based similarity function, and a clustering algorithm Lex-Var.\nUrduPhone encodes Roman Urdu strings to their pronunciation-based\nrepresentations. The string matching component handles character-level\nvariations that occur when writing Urdu using Roman script.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00104v1",
            "date": "2020-03-31 20:58:14+00:00",
            "title": "Improvement of electronic Governance and mobile Governance in Multilingual Countries with Digital Etymology using Sanskrit Grammar",
            "authors": [
                "Arijit Das",
                "Diganta Saha"
            ],
            "abstract": "With huge improvement of digital connectivity (Wifi,3G,4G) and digital\ndevices access to internet has reached in the remotest corners now a days.\nRural people can easily access web or apps from PDAs, laptops, smartphones etc.\nThis is an opportunity of the Government to reach to the citizen in large\nnumber, get their feedback, associate them in policy decision with e governance\nwithout deploying huge man, material or resourses. But the Government of\nmultilingual countries face a lot of problem in successful implementation of\nGovernment to Citizen (G2C) and Citizen to Government (C2G) governance as the\nrural people tend and prefer to interact in their native languages. Presenting\nequal experience over web or app to different language group of speakers is a\nreal challenge. In this research we have sorted out the problems faced by Indo\nAryan speaking netizens which is in general also applicable to any language\nfamily groups or subgroups. Then we have tried to give probable solutions using\nEtymology. Etymology is used to correlate the words using their ROOT forms. In\n5th century BC Panini wrote Astadhyayi where he depicted sutras or rules -- how\na word is changed according to person,tense,gender,number etc. Later this book\nwas followed in Western countries also to derive their grammar of comparatively\nnew languages. We have trained our system for automatic root extraction from\nthe surface level or morphed form of words using Panian Gramatical rules. We\nhave tested our system over 10000 bengali Verbs and extracted the root form\nwith 98% accuracy. We are now working to extend the program to successfully\nlemmatize any words of any language and correlate them by applying those rule\nsets in Artificial Neural Network.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2004.00248v1",
            "date": "2020-04-01 06:19:56+00:00",
            "title": "Adversarial Transfer Learning for Punctuation Restoration",
            "authors": [
                "Jiangyan Yi",
                "Jianhua Tao",
                "Ye Bai",
                "Zhengkun Tian",
                "Cunhang Fan"
            ],
            "abstract": "Previous studies demonstrate that word embeddings and part-of-speech (POS)\ntags are helpful for punctuation restoration tasks. However, two drawbacks\nstill exist. One is that word embeddings are pre-trained by unidirectional\nlanguage modeling objectives. Thus the word embeddings only contain\nleft-to-right context information. The other is that POS tags are provided by\nan external POS tagger. So computation cost will be increased and incorrect\npredicted tags may affect the performance of restoring punctuation marks during\ndecoding. This paper proposes adversarial transfer learning to address these\nproblems. A pre-trained bidirectional encoder representations from transformers\n(BERT) model is used to initialize a punctuation model. Thus the transferred\nmodel parameters carry both left-to-right and right-to-left representations.\nFurthermore, adversarial multi-task learning is introduced to learn task\ninvariant knowledge for punctuation prediction. We use an extra POS tagging\ntask to help the training of the punctuation predicting task. Adversarial\ntraining is utilized to prevent the shared parameters from containing task\nspecific information. We only use the punctuation predicting task to restore\nmarks during decoding stage. Therefore, it will not need extra computation and\nnot introduce incorrect tags from the POS tagger. Experiments are conducted on\nIWSLT2011 datasets. The results demonstrate that the punctuation predicting\nmodels obtain further performance improvement with task invariant knowledge\nfrom the POS tagging task. Our best model outperforms the previous\nstate-of-the-art model trained only with lexical features by up to 9.2%\nabsolute overall F_1-score on test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00390v1",
            "date": "2020-04-01 12:42:06+00:00",
            "title": "More Grounded Image Captioning by Distilling Image-Text Matching Model",
            "authors": [
                "Yuanen Zhou",
                "Meng Wang",
                "Daqing Liu",
                "Zhenzhen Hu",
                "Hanwang Zhang"
            ],
            "abstract": "Visual attention not only improves the performance of image captioners, but\nalso serves as a visual interpretation to qualitatively measure the caption\nrationality and model transparency. Specifically, we expect that a captioner\ncan fix its attentive gaze on the correct objects while generating the\ncorresponding words. This ability is also known as grounded image captioning.\nHowever, the grounding accuracy of existing captioners is far from\nsatisfactory. To improve the grounding accuracy while retaining the captioning\nquality, it is expensive to collect the word-region alignment as strong\nsupervision. To this end, we propose a Part-of-Speech (POS) enhanced image-text\nmatching model (SCAN \\cite{lee2018stacked}): POS-SCAN, as the effective\nknowledge distillation for more grounded image captioning. The benefits are\ntwo-fold: 1) given a sentence and an image, POS-SCAN can ground the objects\nmore accurately than SCAN; 2) POS-SCAN serves as a word-region alignment\nregularization for the captioner's visual attention module. By showing\nbenchmark experimental results, we demonstrate that conventional image\ncaptioners equipped with POS-SCAN can significantly improve the grounding\naccuracy without strong supervision. Last but not the least, we explore the\nindispensable Self-Critical Sequence Training (SCST) \\cite{Rennie_2017_CVPR} in\nthe context of grounded image captioning and show that the image-text matching\nscore can serve as a reward for more grounded captioning\n\\footnote{https://github.com/YuanEZhou/Grounded-Image-Captioning}.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.00499v3",
            "date": "2020-02-23 12:13:48+00:00",
            "title": "Unique Chinese Linguistic Phenomena",
            "authors": [
                "Shengbin Jia"
            ],
            "abstract": "Linguistics holds unique characteristics of generality, stability, and\nnationality, which will affect the formulation of extraction strategies and\nshould be incorporated into the relation extraction. Chinese open relation\nextraction is not well-established, because of the complexity of Chinese\nlinguistics makes it harder to operate, and the methods for English are not\ncompatible with that for Chinese. The diversities between Chinese and English\nlinguistics are mainly reflected in morphology and syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00584v3",
            "date": "2020-04-01 17:14:10+00:00",
            "title": "Deep Entity Matching with Pre-Trained Language Models",
            "authors": [
                "Yuliang Li",
                "Jinfeng Li",
                "Yoshihiko Suhara",
                "AnHai Doan",
                "Wang-Chiew Tan"
            ],
            "abstract": "We present Ditto, a novel entity matching system based on pre-trained\nTransformer-based language models. We fine-tune and cast EM as a sequence-pair\nclassification problem to leverage such models with a simple architecture. Our\nexperiments show that a straightforward application of language models such as\nBERT, DistilBERT, or RoBERTa pre-trained on large text corpora already\nsignificantly improves the matching quality and outperforms previous\nstate-of-the-art (SOTA), by up to 29% of F1 score on benchmark datasets. We\nalso developed three optimization techniques to further improve Ditto's\nmatching capability. Ditto allows domain knowledge to be injected by\nhighlighting important pieces of input information that may be of interest when\nmaking matching decisions. Ditto also summarizes strings that are too long so\nthat only the essential information is retained and used for EM. Finally, Ditto\nadapts a SOTA technique on data augmentation for text to EM to augment the\ntraining data with (difficult) examples. This way, Ditto is forced to learn\n\"harder\" to improve the model's matching capability. The optimizations we\ndeveloped further boost the performance of Ditto by up to 9.8%. Perhaps more\nsurprisingly, we establish that Ditto can achieve the previous SOTA results\nwith at most half the number of labeled data. Finally, we demonstrate Ditto's\neffectiveness on a real-world large-scale EM task. On matching two company\ndatasets consisting of 789K and 412K records, Ditto achieves a high F1 score of\n96.5%.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/2004.00648v1",
            "date": "2020-04-01 18:06:21+00:00",
            "title": "Igbo-English Machine Translation: An Evaluation Benchmark",
            "authors": [
                "Ignatius Ezeani",
                "Paul Rayson",
                "Ikechukwu Onyenwe",
                "Chinedu Uchechukwu",
                "Mark Hepple"
            ],
            "abstract": "Although researchers and practitioners are pushing the boundaries and\nenhancing the capacities of NLP tools and methods, works on African languages\nare lagging. A lot of focus on well resourced languages such as English,\nJapanese, German, French, Russian, Mandarin Chinese etc. Over 97% of the\nworld's 7000 languages, including African languages, are low resourced for NLP\ni.e. they have little or no data, tools, and techniques for NLP research. For\ninstance, only 5 out of 2965, 0.19% authors of full text papers in the ACL\nAnthology extracted from the 5 major conferences in 2018 ACL, NAACL, EMNLP,\nCOLING and CoNLL, are affiliated to African institutions. In this work, we\ndiscuss our effort toward building a standard machine translation benchmark\ndataset for Igbo, one of the 3 major Nigerian languages. Igbo is spoken by more\nthan 50 million people globally with over 50% of the speakers are in\nsoutheastern Nigeria. Igbo is low resourced although there have been some\nefforts toward developing IgboNLP such as part of speech tagging and diacritic\nrestoration",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.00998v1",
            "date": "2020-03-31 22:43:29+00:00",
            "title": "DeepSumm -- Deep Code Summaries using Neural Transformer Architecture",
            "authors": [
                "Vivek Gupta"
            ],
            "abstract": "Source code summarizing is a task of writing short, natural language\ndescriptions of source code behavior during run time. Such summaries are\nextremely useful for software development and maintenance but are expensive to\nmanually author,hence it is done for small fraction of the code that is\nproduced and is often ignored. Automatic code documentation can possibly solve\nthis at a low cost. This is thus an emerging research field with further\napplications to program comprehension, and software maintenance. Traditional\nmethods often relied on cognitive models that were built in the form of\ntemplates and by heuristics and had varying degree of adoption by the developer\ncommunity. But with recent advancements, end to end data-driven approaches\nbased on neural techniques have largely overtaken the traditional techniques.\nMuch of the current landscape employs neural translation based architectures\nwith recurrence and attention which is resource and time intensive training\nprocedure. In this paper, we employ neural techniques to solve the task of\nsource code summarizing and specifically compare NMT based techniques to more\nsimplified and appealing Transformer architecture on a dataset of Java methods\nand comments. We bring forth an argument to dispense the need of recurrence in\nthe training procedure. To the best of our knowledge, transformer based models\nhave not been used for the task before. With supervised samples of more than\n2.1m comments and code, we reduce the training time by more than 50% and\nachieve the BLEU score of 17.99 for the test set of examples.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2004.01095v1",
            "date": "2020-04-02 16:00:10+00:00",
            "title": "MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model",
            "authors": [
                "Han Fu",
                "Rui Wu",
                "Chenghao Liu",
                "Jianling Sun"
            ],
            "abstract": "Nowadays, driven by the increasing concern on diet and health, food computing\nhas attracted enormous attention from both industry and research community. One\nof the most popular research topics in this domain is Food Retrieval, due to\nits profound influence on health-oriented applications. In this paper, we focus\non the task of cross-modal retrieval between food images and cooking recipes.\nWe present Modality-Consistent Embedding Network (MCEN) that learns\nmodality-invariant representations by projecting images and texts to the same\nembedding space. To capture the latent alignments between modalities, we\nincorporate stochastic latent variables to explicitly exploit the interactions\nbetween textual and visual features. Importantly, our method learns the\ncross-modal alignments during training but computes embeddings of different\nmodalities independently at inference time for the sake of efficiency.\nExtensive experimental results clearly demonstrate that the proposed MCEN\noutperforms all existing approaches on the benchmark Recipe1M dataset and\nrequires less computational cost.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.01251v1",
            "date": "2020-04-02 20:39:12+00:00",
            "title": "R3: A Reading Comprehension Benchmark Requiring Reasoning Processes",
            "authors": [
                "Ran Wang",
                "Kun Tao",
                "Dingjie Song",
                "Zhilong Zhang",
                "Xiao Ma",
                "Xi'ao Su",
                "Xinyu Dai"
            ],
            "abstract": "Existing question answering systems can only predict answers without explicit\nreasoning processes, which hinder their explainability and make us overestimate\ntheir ability of understanding and reasoning over natural language. In this\nwork, we propose a novel task of reading comprehension, in which a model is\nrequired to provide final answers and reasoning processes. To this end, we\nintroduce a formalism for reasoning over unstructured text, namely Text\nReasoning Meaning Representation (TRMR). TRMR consists of three phrases, which\nis expressive enough to characterize the reasoning process to answer reading\ncomprehension questions. We develop an annotation platform to facilitate TRMR's\nannotation, and release the R3 dataset, a \\textbf{R}eading comprehension\nbenchmark \\textbf{R}equiring \\textbf{R}easoning processes. R3 contains over 60K\npairs of question-answer pairs and their TRMRs. Our dataset is available at:\n\\url{http://anonymous}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01422v1",
            "date": "2020-04-03 08:09:07+00:00",
            "title": "Learning synchronous context-free grammars with multiple specialised non-terminals for hierarchical phrase-based translation",
            "authors": [
                "Felipe S\u00e1nchez-Mart\u00ednez",
                "Juan Antonio P\u00e9rez-Ortiz",
                "Rafael C. Carrasco"
            ],
            "abstract": "Translation models based on hierarchical phrase-based statistical machine\ntranslation (HSMT) have shown better performances than the non-hierarchical\nphrase-based counterparts for some language pairs. The standard approach to\nHSMT learns and apply a synchronous context-free grammar with a single\nnon-terminal. The hypothesis behind the grammar refinement algorithm presented\nin this work is that this single non-terminal is overloaded, and insufficiently\ndiscriminative, and therefore, an adequate split of it into more specialised\nsymbols could lead to improved models. This paper presents a method to learn\nsynchronous context-free grammars with a huge number of initial non-terminals,\nwhich are then grouped via a clustering algorithm. Our experiments show that\nthe resulting smaller set of non-terminals correctly capture the contextual\ninformation that makes it possible to statistically significantly improve the\nBLEU score of the standard HSMT approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01694v1",
            "date": "2020-04-03 17:49:56+00:00",
            "title": "A Set of Recommendations for Assessing Human-Machine Parity in Language Translation",
            "authors": [
                "Samuel L\u00e4ubli",
                "Sheila Castilho",
                "Graham Neubig",
                "Rico Sennrich",
                "Qinlan Shen",
                "Antonio Toral"
            ],
            "abstract": "The quality of machine translation has increased remarkably over the past\nyears, to the degree that it was found to be indistinguishable from\nprofessional human translation in a number of empirical investigations. We\nreassess Hassan et al.'s 2018 investigation into Chinese to English news\ntranslation, showing that the finding of human-machine parity was owed to\nweaknesses in the evaluation design - which is currently considered best\npractice in the field. We show that the professional human translations\ncontained significantly fewer errors, and that perceived quality in human\nevaluation depends on the choice of raters, the availability of linguistic\ncontext, and the creation of reference translations. Our results call for\nrevisiting current best practices to assess strong machine translation systems\nin general and human-machine parity in particular, for which we offer a set of\nrecommendations based on our empirical findings.",
            "journal ref": "Journal of Artificial Intelligence Research 67 (2020) 653-672",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.01820v1",
            "date": "2020-04-04 00:35:16+00:00",
            "title": "Aggressive, Repetitive, Intentional, Visible, and Imbalanced: Refining Representations for Cyberbullying Classification",
            "authors": [
                "Caleb Ziems",
                "Ymir Vigfusson",
                "Fred Morstatter"
            ],
            "abstract": "Cyberbullying is a pervasive problem in online communities. To identify\ncyberbullying cases in large-scale social networks, content moderators depend\non machine learning classifiers for automatic cyberbullying detection. However,\nexisting models remain unfit for real-world applications, largely due to a\nshortage of publicly available training data and a lack of standard criteria\nfor assigning ground truth labels. In this study, we address the need for\nreliable data using an original annotation framework. Inspired by social\nsciences research into bullying behavior, we characterize the nuanced problem\nof cyberbullying using five explicit factors to represent its social and\nlinguistic aspects. We model this behavior using social network and\nlanguage-based features, which improve classifier performance. These results\ndemonstrate the importance of representing and modeling cyberbullying as a\nsocial phenomenon.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.01881v1",
            "date": "2020-04-04 07:31:59+00:00",
            "title": "CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection",
            "authors": [
                "Congying Xia",
                "Chenwei Zhang",
                "Hoang Nguyen",
                "Jiawei Zhang",
                "Philip Yu"
            ],
            "abstract": "In this paper, we formulate a more realistic and difficult problem setup for\nthe intent detection task in natural language understanding, namely Generalized\nFew-Shot Intent Detection (GFSID). GFSID aims to discriminate a joint label\nspace consisting of both existing intents which have enough labeled data and\nnovel intents which only have a few examples for each class. To approach this\nproblem, we propose a novel model, Conditional Text Generation with BERT\n(CG-BERT). CG-BERT effectively leverages a large pre-trained language model to\ngenerate text conditioned on the intent label. By modeling the utterance\ndistribution with variational inference, CG-BERT can generate diverse\nutterances for the novel intents even with only a few utterances available.\nExperimental results show that CG-BERT achieves state-of-the-art performance on\nthe GFSID task with 1-shot and 5-shot settings on two real-world datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02002v3",
            "date": "2020-04-04 19:19:55+00:00",
            "title": "Talk to Papers: Bringing Neural Question Answering to Academic Search",
            "authors": [
                "Tianchang Zhao",
                "Kyusong Lee"
            ],
            "abstract": "We introduce Talk to Papers, which exploits the recent open-domain question\nanswering (QA) techniques to improve the current experience of academic search.\nIt's designed to enable researchers to use natural language queries to find\nprecise answers and extract insights from a massive amount of academic papers.\nWe present a large improvement over classic search engine baseline on several\nstandard QA datasets and provide the community a collaborative data collection\ntool to curate the first natural language processing research QA dataset via a\ncommunity effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02135v5",
            "date": "2020-04-05 09:34:52+00:00",
            "title": "Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation",
            "authors": [
                "Xingyuan Chen",
                "Ping Cai",
                "Peng Jin",
                "Hongjun Wang",
                "Xinyu Dai",
                "Jiajun Chen"
            ],
            "abstract": "The autoregressive language model (ALM) trained with maximum likelihood\nestimation (MLE) is widely used in unconditional text generation. Due to\nexposure bias, the generated texts still suffer from low quality and diversity.\nThis presents statistically as a discrepancy between the real text and\ngenerated text. Some research shows a discriminator can detect this\ndiscrepancy. Because the discriminator can encode more information than the\ngenerator, discriminator has the potentiality to improve generator. To\nalleviate the exposure bias, generative adversarial networks (GAN) use the\ndiscriminator to update the generator's parameters directly, but they fail by\nbeing evaluated precisely. A critical reason for the failure is the difference\nbetween the discriminator input and the ALM input. We propose a novel mechanism\nby adding a filter which has the same input as the discriminator. First,\ndiscriminator detects the discrepancy signals and passes to filter directly (or\nby learning). Then, we use the filter to reject some generated samples with a\nsampling-based method. Thus, the original generative distribution is revised to\nreduce the discrepancy. Two ALMs, RNN-based and Transformer-based, are\nexperimented. Evaluated precisely by three metrics, our mechanism consistently\noutperforms the ALMs and all kinds of GANs across two benchmark data sets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.02199v1",
            "date": "2020-04-05 13:32:58+00:00",
            "title": "Understanding Learning Dynamics for Neural Machine Translation",
            "authors": [
                "Conghui Zhu",
                "Guanlin Li",
                "Lemao Liu",
                "Tiejun Zhao",
                "Shuming Shi"
            ],
            "abstract": "Despite the great success of NMT, there still remains a severe challenge: it\nis hard to interpret the internal dynamics during its training process. In this\npaper we propose to understand learning dynamics of NMT by using a recent\nproposed technique named Loss Change Allocation\n(LCA)~\\citep{lan-2019-loss-change-allocation}. As LCA requires calculating the\ngradient on an entire dataset for each update, we instead present an\napproximate to put it into practice in NMT scenario. %motivated by the lesson\nfrom sgd. Our simulated experiment shows that such approximate calculation is\nefficient and is empirically proved to deliver consistent results to the\nbrute-force implementation. In particular, extensive experiments on two\nstandard translation benchmark datasets reveal some valuable findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02288v2",
            "date": "2020-04-05 19:31:44+00:00",
            "title": "Continual Domain-Tuning for Pretrained Language Models",
            "authors": [
                "Subendhu Rongali",
                "Abhyuday Jagannatha",
                "Bhanu Pratap Singh Rawat",
                "Hong Yu"
            ],
            "abstract": "Pre-trained language models (LM) such as BERT, DistilBERT, and RoBERTa can be\ntuned for different domains (domain-tuning) by continuing the pre-training\nphase on a new target domain corpus. This simple domain tuning (SDT) technique\nhas been widely used to create domain-tuned models such as BioBERT, SciBERT and\nClinicalBERT. However, during the pretraining phase on the target domain, the\nLM models may catastrophically forget the patterns learned from their source\ndomain. In this work, we study the effects of catastrophic forgetting on\ndomain-tuned LM models and investigate methods that mitigate its negative\neffects. We propose continual learning (CL) based alternatives for SDT, that\naim to reduce catastrophic forgetting. We show that these methods may increase\nthe performance of LM models on downstream target domain tasks. Additionally,\nwe also show that constraining the LM model from forgetting the source domain\nleads to downstream task models that are more robust to domain shifts. We\nanalyze the computational cost of using our proposed CL methods and provide\nrecommendations for computationally lightweight and effective CL domain-tuning\nprocedures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02363v2",
            "date": "2020-04-06 00:49:20+00:00",
            "title": "Exploring Early Prediction of Buyer-Seller Negotiation Outcomes",
            "authors": [
                "Kushal Chawla",
                "Gale Lucas",
                "Jonathan May",
                "Jonathan Gratch"
            ],
            "abstract": "Agents that negotiate with humans find broad applications in pedagogy and\nconversational AI. Most efforts in human-agent negotiations rely on restrictive\nmenu-driven interfaces for communication. To advance the research in\nlanguage-based negotiation systems, we explore a novel task of early prediction\nof buyer-seller negotiation outcomes, by varying the fraction of utterances\nthat the model can access. We explore the feasibility of early prediction by\nusing traditional feature-based methods, as well as by incorporating the\nnon-linguistic task context into a pretrained language model using sentence\ntemplates. We further quantify the extent to which linguistic features help in\nmaking better predictions apart from the task-specific price information.\nFinally, probing the pretrained model helps us to identify specific features,\nsuch as trust and agreement, that contribute to the prediction performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02438v2",
            "date": "2020-04-06 07:23:17+00:00",
            "title": "SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction",
            "authors": [
                "Xuming Hu",
                "Chenwei Zhang",
                "Yusong Xu",
                "Lijie Wen",
                "Philip S. Yu"
            ],
            "abstract": "Open relation extraction is the task of extracting open-domain relation facts\nfrom natural language sentences. Existing works either utilize heuristics or\ndistant-supervised annotations to train a supervised classifier over\npre-defined relations, or adopt unsupervised methods with additional\nassumptions that have less discriminative power. In this work, we proposed a\nself-supervised framework named SelfORE, which exploits weak, self-supervised\nsignals by leveraging large pretrained language model for adaptive clustering\non contextualized relational features, and bootstraps the self-supervised\nsignals by improving contextualized features in relation classification.\nExperimental results on three datasets show the effectiveness and robustness of\nSelfORE on open-domain Relation Extraction when comparing with competitive\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02557v3",
            "date": "2020-04-06 11:09:44+00:00",
            "title": "Distinguish Confusing Law Articles for Legal Judgment Prediction",
            "authors": [
                "Nuo Xu",
                "Pinghui Wang",
                "Long Chen",
                "Li Pan",
                "Xiaoyan Wang",
                "Junzhou Zhao"
            ],
            "abstract": "Legal Judgment Prediction (LJP) is the task of automatically predicting a law\ncase's judgment results given a text describing its facts, which has excellent\nprospects in judicial assistance systems and convenient services for the\npublic. In practice, confusing charges are frequent, because law cases\napplicable to similar law articles are easily misjudged. For addressing this\nissue, the existing method relies heavily on domain experts, which hinders its\napplication in different law systems. In this paper, we present an end-to-end\nmodel, LADAN, to solve the task of LJP. To distinguish confusing charges, we\npropose a novel graph neural network to automatically learn subtle differences\nbetween confusing law articles and design a novel attention mechanism that\nfully exploits the learned differences to extract compelling discriminative\nfeatures from fact descriptions attentively. Experiments conducted on\nreal-world datasets demonstrate the superiority of our LADAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02705v2",
            "date": "2020-04-06 14:37:39+00:00",
            "title": "Quantum Inspired Word Representation and Computation",
            "authors": [
                "Shen Li",
                "Renfen Hu",
                "Jinshan Wu"
            ],
            "abstract": "Word meaning has different aspects, while the existing word representation\n\"compresses\" these aspects into a single vector, and it needs further analysis\nto recover the information in different dimensions. Inspired by quantum\nprobability, we represent words as density matrices, which are inherently\ncapable of representing mixed states. The experiment shows that the density\nmatrix representation can effectively capture different aspects of word meaning\nwhile maintaining comparable reliability with the vector representation.\nFurthermore, we propose a novel method to combine the coherent summation and\nincoherent summation in the computation of both vectors and density matrices.\nIt achieves consistent improvement on word analogy task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02814v1",
            "date": "2020-04-06 16:55:41+00:00",
            "title": "Leveraging the Inherent Hierarchy of Vacancy Titles for Automated Job Ontology Expansion",
            "authors": [
                "Jeroen Van Hautte",
                "Vincent Schelstraete",
                "Mika\u00ebl Wornoo"
            ],
            "abstract": "Machine learning plays an ever-bigger part in online recruitment, powering\nintelligent matchmaking and job recommendations across many of the world's\nlargest job platforms. However, the main text is rarely enough to fully\nunderstand a job posting: more often than not, much of the required information\nis condensed into the job title. Several organised efforts have been made to\nmap job titles onto a hand-made knowledge base as to provide this information,\nbut these only cover around 60\\% of online vacancies. We introduce a novel,\npurely data-driven approach towards the detection of new job titles. Our method\nis conceptually simple, extremely efficient and competitive with traditional\nNER-based approaches. Although the standalone application of our method does\nnot outperform a finetuned BERT model, it can be applied as a preprocessing\nstep as well, substantially boosting accuracy across several architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02843v2",
            "date": "2020-04-06 17:36:42+00:00",
            "title": "Improved Code Summarization via a Graph Neural Network",
            "authors": [
                "Alexander LeClair",
                "Sakib Haque",
                "Lingfei Wu",
                "Collin McMillan"
            ],
            "abstract": "Automatic source code summarization is the task of generating natural\nlanguage descriptions for source code. Automatic code summarization is a\nrapidly expanding research area, especially as the community has taken greater\nadvantage of advances in neural network and AI technologies. In general, source\ncode summarization techniques use the source code as input and outputs a\nnatural language description. Yet a strong consensus is developing that using\nstructural information as input leads to improved performance. The first\napproaches to use structural information flattened the AST into a sequence.\nRecently, more complex approaches based on random AST paths or graph neural\nnetworks have improved on the models using flattened ASTs. However, the\nliterature still does not describe the using a graph neural network together\nwith source code sequence as separate inputs to a model. Therefore, in this\npaper, we present an approach that uses a graph-based neural architecture that\nbetter matches the default structure of the AST to generate these summaries. We\nevaluate our technique using a data set of 2.1 million Java method-comment\npairs and show improvement over four baseline techniques, two from the software\nengineering literature, and two from machine learning literature.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2004.02913v2",
            "date": "2020-04-06 18:03:06+00:00",
            "title": "Speaker-change Aware CRF for Dialogue Act Classification",
            "authors": [
                "Guokan Shang",
                "Antoine Jean-Pierre Tixier",
                "Michalis Vazirgiannis",
                "Jean-Pierre Lorr\u00e9"
            ],
            "abstract": "Recent work in Dialogue Act (DA) classification approaches the task as a\nsequence labeling problem, using neural network models coupled with a\nConditional Random Field (CRF) as the last layer. CRF models the conditional\nprobability of the target DA label sequence given the input utterance sequence.\nHowever, the task involves another important input sequence, that of speakers,\nwhich is ignored by previous work. To address this limitation, this paper\nproposes a simple modification of the CRF layer that takes speaker-change into\naccount. Experiments on the SwDA corpus show that our modified CRF layer\noutperforms the original one, with very wide margins for some DA labels.\nFurther, visualizations demonstrate that our CRF layer can learn meaningful,\nsophisticated transition patterns between DA label pairs conditioned on\nspeaker-change in an end-to-end way. Code is publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.02984v2",
            "date": "2020-04-06 20:20:58+00:00",
            "title": "MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices",
            "authors": [
                "Zhiqing Sun",
                "Hongkun Yu",
                "Xiaodan Song",
                "Renjie Liu",
                "Yiming Yang",
                "Denny Zhou"
            ],
            "abstract": "Natural Language Processing (NLP) has recently achieved great success by\nusing huge pre-trained models with hundreds of millions of parameters. However,\nthese models suffer from heavy model sizes and high latency such that they\ncannot be deployed to resource-limited mobile devices. In this paper, we\npropose MobileBERT for compressing and accelerating the popular BERT model.\nLike the original BERT, MobileBERT is task-agnostic, that is, it can be\ngenerically applied to various downstream NLP tasks via simple fine-tuning.\nBasically, MobileBERT is a thin version of BERT_LARGE, while equipped with\nbottleneck structures and a carefully designed balance between self-attentions\nand feed-forward networks. To train MobileBERT, we first train a specially\ndesigned teacher model, an inverted-bottleneck incorporated BERT_LARGE model.\nThen, we conduct knowledge transfer from this teacher to MobileBERT. Empirical\nstudies show that MobileBERT is 4.3x smaller and 5.5x faster than BERT_BASE\nwhile achieving competitive results on well-known benchmarks. On the natural\nlanguage inference tasks of GLUE, MobileBERT achieves a GLUEscore o 77.7 (0.6\nlower than BERT_BASE), and 62 ms latency on a Pixel 4 phone. On the SQuAD\nv1.1/v2.0 question answering task, MobileBERT achieves a dev F1 score of\n90.0/79.2 (1.5/2.1 higher than BERT_BASE).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03061v2",
            "date": "2020-04-07 01:06:36+00:00",
            "title": "Information-Theoretic Probing for Linguistic Structure",
            "authors": [
                "Tiago Pimentel",
                "Josef Valvoda",
                "Rowan Hall Maudslay",
                "Ran Zmigrod",
                "Adina Williams",
                "Ryan Cotterell"
            ],
            "abstract": "The success of neural networks on a diverse set of NLP tasks has led\nresearchers to question how much these networks actually ``know'' about natural\nlanguage. Probes are a natural way of assessing this. When probing, a\nresearcher chooses a linguistic task and trains a supervised model to predict\nannotations in that linguistic task from the network's learned representations.\nIf the probe does well, the researcher may conclude that the representations\nencode knowledge related to the task. A commonly held belief is that using\nsimpler models as probes is better; the logic is that simpler models will\nidentify linguistic structure, but not learn the task itself. We propose an\ninformation-theoretic operationalization of probing as estimating mutual\ninformation that contradicts this received wisdom: one should always select the\nhighest performing probe one can, even if it is more complex, since it will\nresult in a tighter estimate, and thus reveal more of the linguistic\ninformation inherent in the representation. The experimental portion of our\npaper focuses on empirically estimating the mutual information between a\nlinguistic property and BERT, comparing these estimates to several baselines.\nWe evaluate on a set of ten typologically diverse languages often\nunderrepresented in NLP research---plus English---totalling eleven languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03070v2",
            "date": "2020-04-07 01:49:18+00:00",
            "title": "Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning",
            "authors": [
                "Daya Guo",
                "Akari Asai",
                "Duyu Tang",
                "Nan Duan",
                "Ming Gong",
                "Linjun Shou",
                "Daxin Jiang",
                "Jian Yin",
                "Ming Zhou"
            ],
            "abstract": "We study the problem of generating inferential texts of events for a variety\nof commonsense like \\textit{if-else} relations. Existing approaches typically\nuse limited evidence from training examples and learn for each relation\nindividually. In this work, we use multiple knowledge sources as fuels for the\nmodel. Existing commonsense knowledge bases like ConceptNet are dominated by\ntaxonomic knowledge (e.g., \\textit{isA} and \\textit{relatedTo} relations),\nhaving a limited number of inferential knowledge. We use not only structured\ncommonsense knowledge bases, but also natural language snippets from\nsearch-engine results. These sources are incorporated into a generative base\nmodel via key-value memory network. In addition, we introduce a meta-learning\nbased multi-task learning algorithm. For each targeted commonsense relation, we\nregard the learning of examples from other relations as the meta-training\nprocess, and the evaluation on examples from the targeted relation as the\nmeta-test process. We conduct experiments on Event2Mind and ATOMIC datasets.\nResults show that both the integration of multiple knowledge sources and the\nuse of the meta-learning algorithm improve the performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03212v4",
            "date": "2020-04-07 09:04:43+00:00",
            "title": "Text-Guided Neural Image Inpainting",
            "authors": [
                "Lisai Zhang",
                "Qingcai Chen",
                "Baotian Hu",
                "Shuoran Jiang"
            ],
            "abstract": "Image inpainting task requires filling the corrupted image with contents\ncoherent with the context. This research field has achieved promising progress\nby using neural image inpainting methods. Nevertheless, there is still a\ncritical challenge in guessing the missed content with only the context pixels.\nThe goal of this paper is to fill the semantic information in corrupted images\naccording to the provided descriptive text. Unique from existing text-guided\nimage generation works, the inpainting models are required to compare the\nsemantic content of the given text and the remaining part of the image, then\nfind out the semantic content that should be filled for missing part. To\nfulfill such a task, we propose a novel inpainting model named Text-Guided Dual\nAttention Inpainting Network (TDANet). Firstly, a dual multimodal attention\nmechanism is designed to extract the explicit semantic information about the\ncorrupted regions, which is done by comparing the descriptive text and\ncomplementary image areas through reciprocal attention. Secondly, an image-text\nmatching loss is applied to maximize the semantic similarity of the generated\nimage and the text. Experiments are conducted on two open datasets. Results\nshow that the proposed TDANet model reaches new state-of-the-art on both\nquantitative and qualitative measures. Result analysis suggests that the\ngenerated images are consistent with the guidance text, enabling the generation\nof various results by providing different descriptions. Codes are available at\nhttps://github.com/idealwhite/TDANet",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.03386v4",
            "date": "2020-04-07 13:46:39+00:00",
            "title": "Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking",
            "authors": [
                "Su Zhu",
                "Jieyu Li",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Dialogue state tracking (DST) aims at estimating the current dialogue state\ngiven all the preceding conversation. For multi-domain DST, the data sparsity\nproblem is a major obstacle due to increased numbers of state candidates and\ndialogue lengths. To encode the dialogue context efficiently, we utilize the\nprevious dialogue state (predicted) and the current dialogue utterance as the\ninput for DST. To consider relations among different domain-slots, the schema\ngraph involving prior knowledge is exploited. In this paper, a novel context\nand schema fusion network is proposed to encode the dialogue context and schema\ngraph by using internal and external attention mechanisms. Experiment results\nshow that our approach can obtain new state-of-the-art performance of the\nopen-vocabulary DST on both MultiWOZ 2.0 and MultiWOZ 2.1 benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03420v2",
            "date": "2020-04-07 14:10:27+00:00",
            "title": "Emergent Language Generalization and Acquisition Speed are not tied to Compositionality",
            "authors": [
                "Eugene Kharitonov",
                "Marco Baroni"
            ],
            "abstract": "Studies of discrete languages emerging when neural agents communicate to\nsolve a joint task often look for evidence of compositional structure. This\nstems for the expectation that such a structure would allow languages to be\nacquired faster by the agents and enable them to generalize better. We argue\nthat these beneficial properties are only loosely connected to\ncompositionality. In two experiments, we demonstrate that, depending on the\ntask, non-compositional languages might show equal, or better, generalization\nperformance and acquisition speed than compositional ones. Further research in\nthe area should be clearer about what benefits are expected from\ncompositionality, and how the latter would lead to them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03485v1",
            "date": "2020-04-07 15:35:55+00:00",
            "title": "A Few Topical Tweets are Enough for Effective User-Level Stance Detection",
            "authors": [
                "Younes Samih",
                "Kareem Darwish"
            ],
            "abstract": "Stance detection entails ascertaining the position of a user towards a\ntarget, such as an entity, topic, or claim. Recent work that employs\nunsupervised classification has shown that performing stance detection on vocal\nTwitter users, who have many tweets on a target, can yield very high accuracy\n(+98%). However, such methods perform poorly or fail completely for less vocal\nusers, who may have authored only a few tweets about a target. In this paper,\nwe tackle stance detection for such users using two approaches. In the first\napproach, we improve user-level stance detection by representing tweets using\ncontextualized embeddings, which capture latent meanings of words in context.\nWe show that this approach outperforms two strong baselines and achieves 89.6%\naccuracy and 91.3% macro F-measure on eight controversial topics. In the second\napproach, we expand the tweets of a given user using their Twitter timeline\ntweets, and then we perform unsupervised classification of the user, which\nentails clustering a user with other users in the training set. This approach\nachieves 95.6% accuracy and 93.1% macro F-measure.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.03685v3",
            "date": "2020-04-07 20:15:28+00:00",
            "title": "Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?",
            "authors": [
                "Alon Jacovi",
                "Yoav Goldberg"
            ],
            "abstract": "With the growing popularity of deep-learning based NLP models, comes a need\nfor interpretable systems. But what is interpretability, and what constitutes a\nhigh-quality interpretation? In this opinion piece we reflect on the current\nstate of interpretability evaluation research. We call for more clearly\ndifferentiating between different desired criteria an interpretation should\nsatisfy, and focus on the faithfulness criteria. We survey the literature with\nrespect to faithfulness evaluation, and arrange the current approaches around\nthree assumptions, providing an explicit form to how faithfulness is \"defined\"\nby the community. We provide concrete guidelines on how evaluation of\ninterpretation methods should and should not be conducted. Finally, we claim\nthat the current binary definition for faithfulness sets a potentially\nunrealistic bar for being considered faithful. We call for discarding the\nbinary notion of faithfulness in favor of a more graded one, which we believe\nwill be of greater practical utility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03742v1",
            "date": "2020-04-07 23:02:37+00:00",
            "title": "Towards Evaluating the Robustness of Chinese BERT Classifiers",
            "authors": [
                "Boxin Wang",
                "Boyuan Pan",
                "Xin Li",
                "Bo Li"
            ],
            "abstract": "Recent advances in large-scale language representation models such as BERT\nhave improved the state-of-the-art performances in many NLP tasks. Meanwhile,\ncharacter-level Chinese NLP models, including BERT for Chinese, have also\ndemonstrated that they can outperform the existing models. In this paper, we\nshow that, however, such BERT-based models are vulnerable under character-level\nadversarial attacks. We propose a novel Chinese char-level attack method\nagainst BERT-based classifiers. Essentially, we generate \"small\" perturbation\non the character level in the embedding space and guide the character\nsubstitution procedure. Extensive experiments show that the classification\naccuracy on a Chinese news dataset drops from 91.8% to 0% by manipulating less\nthan 2 characters on average based on the proposed attack. Human evaluations\nalso confirm that our generated Chinese adversarial examples barely affect\nhuman performance on these NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03762v1",
            "date": "2020-04-08 01:05:19+00:00",
            "title": "Generating Narrative Text in a Switching Dynamical System",
            "authors": [
                "Noah Weber",
                "Leena Shekhar",
                "Heeyoung Kwon",
                "Niranjan Balasubramanian",
                "Nathanael Chambers"
            ],
            "abstract": "Early work on narrative modeling used explicit plans and goals to generate\nstories, but the language generation itself was restricted and inflexible.\nModern methods use language models for more robust generation, but often lack\nan explicit representation of the scaffolding and dynamics that guide a\ncoherent narrative. This paper introduces a new model that integrates explicit\nnarrative structure with neural language models, formalizing narrative modeling\nas a Switching Linear Dynamical System (SLDS). A SLDS is a dynamical system in\nwhich the latent dynamics of the system (i.e. how the state vector transforms\nover time) is controlled by top-level discrete switching variables. The\nswitching variables represent narrative structure (e.g., sentiment or discourse\nstates), while the latent state vector encodes information on the current state\nof the narrative. This probabilistic formulation allows us to control\ngeneration, and can be learned in a semi-supervised fashion using both labeled\nand unlabeled data. Additionally, we derive a Gibbs sampler for our model that\ncan fill in arbitrary parts of the narrative, guided by the switching\nvariables. Our filled-in (English language) narratives outperform several\nbaselines on both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03786v1",
            "date": "2020-04-08 03:16:06+00:00",
            "title": "Downstream Model Design of Pre-trained Language Model for Relation Extraction Task",
            "authors": [
                "Cheng Li",
                "Ye Tian"
            ],
            "abstract": "Supervised relation extraction methods based on deep neural network play an\nimportant role in the recent information extraction field. However, at present,\ntheir performance still fails to reach a good level due to the existence of\ncomplicated relations. On the other hand, recently proposed pre-trained\nlanguage models (PLMs) have achieved great success in multiple tasks of natural\nlanguage processing through fine-tuning when combined with the model of\ndownstream tasks. However, original standard tasks of PLM do not include the\nrelation extraction task yet. We believe that PLMs can also be used to solve\nthe relation extraction problem, but it is necessary to establish a specially\ndesigned downstream task model or even loss function for dealing with\ncomplicated relations. In this paper, a new network architecture with a special\nloss function is designed to serve as a downstream model of PLMs for supervised\nrelation extraction. Experiments have shown that our method significantly\nexceeded the current optimal baseline models across multiple public datasets of\nrelation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03807v2",
            "date": "2020-04-08 04:43:37+00:00",
            "title": "SciWING -- A Software Toolkit for Scientific Document Processing",
            "authors": [
                "Abhinav Ramesh Kashyap",
                "Min-Yen Kan"
            ],
            "abstract": "We introduce SciWING, an open-source software toolkit which provides access\nto pre-trained models for scientific document processing tasks, inclusive of\ncitation string parsing and logical structure recovery. SciWING enables\nresearchers to rapidly experiment with different models by swapping and\nstacking different modules. It also enables them declare and run models from a\nconfiguration file. It enables researchers to perform production-ready transfer\nlearning from general, pre-trained transformers (i.e., BERT, SciBERT etc), and\naids development of end-user applications. It includes ready-to-use web and\nterminal-based applications and demonstrations (Available from\nhttp://sciwing.io).",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03808v3",
            "date": "2020-04-08 04:48:44+00:00",
            "title": "Improving BERT with Self-Supervised Attention",
            "authors": [
                "Xiaoyu Kou",
                "Yaming Yang",
                "Yujing Wang",
                "Ce Zhang",
                "Yiren Chen",
                "Yunhai Tong",
                "Yan Zhang",
                "Jing Bai"
            ],
            "abstract": "One of the most popular paradigms of applying large, pre-trained NLP models\nsuch as BERT is to fine-tune it on a smaller dataset. However, one challenge\nremains as the fine-tuned model often overfits on smaller datasets. A symptom\nof this phenomenon is that irrelevant words in the sentences, even when they\nare obvious to humans, can substantially degrade the performance of these\nfine-tuned BERT models. In this paper, we propose a novel technique, called\nSelf-Supervised Attention (SSA) to help facilitate this generalization\nchallenge. Specifically, SSA automatically generates weak, token-level\nattention labels iteratively by \"probing\" the fine-tuned model from the\nprevious iteration. We investigate two different ways of integrating SSA into\nBERT and propose a hybrid approach to combine their benefits. Empirically, on a\nvariety of public datasets, we illustrate significant performance improvement\nusing our SSA-enhanced BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03809v2",
            "date": "2020-04-08 04:51:40+00:00",
            "title": "Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition",
            "authors": [
                "Ryuichi Takanobu",
                "Runze Liang",
                "Minlie Huang"
            ],
            "abstract": "Many studies have applied reinforcement learning to train a dialog policy and\nshow great promise these years. One common approach is to employ a user\nsimulator to obtain a large number of simulated user experiences for\nreinforcement learning algorithms. However, modeling a realistic user simulator\nis challenging. A rule-based simulator requires heavy domain expertise for\ncomplex tasks, and a data-driven simulator requires considerable data and it is\neven unclear how to evaluate a simulator. To avoid explicitly building a user\nsimulator beforehand, we propose Multi-Agent Dialog Policy Learning, which\nregards both the system and the user as the dialog agents. Two agents interact\nwith each other and are jointly learned simultaneously. The method uses the\nactor-critic framework to facilitate pretraining and improve scalability. We\nalso propose Hybrid Value Network for the role-aware reward decomposition to\nintegrate role-specific domain knowledge of each agent in the task-oriented\ndialog. Results show that our method can successfully build a system policy and\na user policy simultaneously, and two agents can achieve a high task success\nrate through conversational interaction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03818v1",
            "date": "2020-04-08 05:28:46+00:00",
            "title": "Explicit Reordering for Neural Machine Translation",
            "authors": [
                "Kehai Chen",
                "Rui Wang",
                "Masao Utiyama",
                "Eiichiro Sumita"
            ],
            "abstract": "In Transformer-based neural machine translation (NMT), the positional\nencoding mechanism helps the self-attention networks to learn the source\nrepresentation with order dependency, which makes the Transformer-based NMT\nachieve state-of-the-art results for various translation tasks. However,\nTransformer-based NMT only adds representations of positions sequentially to\nword vectors in the input sentence and does not explicitly consider reordering\ninformation in this sentence. In this paper, we first empirically investigate\nthe relationship between source reordering information and translation\nperformance. The empirical findings show that the source input with the target\norder learned from the bilingual parallel dataset can substantially improve\ntranslation performance. Thus, we propose a novel reordering method to\nexplicitly model this reordering information for the Transformer-based NMT. The\nempirical results on the WMT14 English-to-German, WAT ASPEC\nJapanese-to-English, and WMT17 Chinese-to-English translation tasks show the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03822v1",
            "date": "2020-04-08 05:44:55+00:00",
            "title": "SIA: A Scalable Interoperable Annotation Server for Biomedical Named Entities",
            "authors": [
                "Johannes Kirschnick",
                "Philippe Thomas",
                "Roland Roller",
                "Leonhard Hennig"
            ],
            "abstract": "Recent years showed a strong increase in biomedical sciences and an inherent\nincrease in publication volume. Extraction of specific information from these\nsources requires highly sophisticated text mining and information extraction\ntools. However, the integration of freely available tools into customized\nworkflows is often cumbersome and difficult. We describe SIA (Scalable\nInteroperable Annotation Server), our contribution to the BeCalm-Technical\ninteroperability and performance of annotation servers (BeCalm-TIPS) task, a\nscalable, extensible, and robust annotation service. The system currently\ncovers six named entity types (i.e., Chemicals, Diseases, Genes, miRNA,\nMutations, and Organisms) and is freely available under Apache 2.0 license at\nhttps://github.com/Erechtheus/sia.",
            "journal ref": "J Cheminform 10, 63 (2018)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03844v2",
            "date": "2020-04-08 07:09:59+00:00",
            "title": "On the Effect of Dropping Layers of Pre-trained Transformer Models",
            "authors": [
                "Hassan Sajjad",
                "Fahim Dalvi",
                "Nadir Durrani",
                "Preslav Nakov"
            ],
            "abstract": "Transformer-based NLP models are trained using hundreds of millions or even\nbillions of parameters, limiting their applicability in computationally\nconstrained environments. While the number of parameters generally correlates\nwith performance, it is not clear whether the entire network is required for a\ndownstream task. Motivated by the recent work on pruning and distilling\npre-trained models, we explore strategies to drop layers in pre-trained models,\nand observe the effect of pruning on downstream GLUE tasks. We were able to\nprune BERT, RoBERTa and XLNet models up to 40%, while maintaining up to 98% of\ntheir original performance. Additionally we show that our pruned models are on\npar with those built using knowledge distillation, both in terms of size and\nperformance. Our experiments yield interesting observations such as, (i) the\nlower layers are most critical to maintain downstream task performance, (ii)\nsome tasks such as paraphrase detection and sentence similarity are more robust\nto the dropping of layers, and (iii) models trained using a different objective\nfunction exhibit different learning patterns and w.r.t the layer dropping.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.03868v3",
            "date": "2020-04-08 08:12:41+00:00",
            "title": "Internal and external pressures on language emergence: least effort, object constancy and frequency",
            "authors": [
                "Diana Rodr\u00edguez Luna",
                "Edoardo Maria Ponti",
                "Dieuwke Hupkes",
                "Elia Bruni"
            ],
            "abstract": "In previous work, artificial agents were shown to achieve almost perfect\naccuracy in referential games where they have to communicate to identify\nimages. Nevertheless, the resulting communication protocols rarely display\nsalient features of natural languages, such as compositionality. In this paper,\nwe propose some realistic sources of pressure on communication that avert this\noutcome. More specifically, we formalise the principle of least effort through\nan auxiliary objective. Moreover, we explore several game variants, inspired by\nthe principle of object constancy, in which we alter the frequency, position,\nand luminosity of the objects in the images. We perform an extensive analysis\non their effect through compositionality metrics, diagnostic classifiers, and\nzero-shot evaluation. Our findings reveal that the proposed sources of pressure\nresult in emerging languages with less redundancy, more focus on high-level\nconceptual information, and better abilities of generalisation. Overall, our\ncontributions reduce the gap between emergent and natural languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04010v2",
            "date": "2020-04-08 14:29:23+00:00",
            "title": "Analyzing Redundancy in Pretrained Transformer Models",
            "authors": [
                "Fahim Dalvi",
                "Hassan Sajjad",
                "Nadir Durrani",
                "Yonatan Belinkov"
            ],
            "abstract": "Transformer-based deep NLP models are trained using hundreds of millions of\nparameters, limiting their applicability in computationally constrained\nenvironments. In this paper, we study the cause of these limitations by\ndefining a notion of Redundancy, which we categorize into two classes: General\nRedundancy and Task-specific Redundancy. We dissect two popular pretrained\nmodels, BERT and XLNet, studying how much redundancy they exhibit at a\nrepresentation-level and at a more fine-grained neuron-level. Our analysis\nreveals interesting insights, such as: i) 85% of the neurons across the network\nare redundant and ii) at least 92% of them can be removed when optimizing\ntowards a downstream task. Based on our analysis, we present an efficient\nfeature-based transfer learning procedure, which maintains 97% performance\nwhile using at-most 10% of the original neurons.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04037v2",
            "date": "2020-04-08 15:06:28+00:00",
            "title": "DynaBERT: Dynamic BERT with Adaptive Width and Depth",
            "authors": [
                "Lu Hou",
                "Zhiqi Huang",
                "Lifeng Shang",
                "Xin Jiang",
                "Xiao Chen",
                "Qun Liu"
            ],
            "abstract": "The pre-trained language models like BERT, though powerful in many natural\nlanguage processing tasks, are both computation and memory expensive. To\nalleviate this problem, one approach is to compress them for specific tasks\nbefore deployment. However, recent works on BERT compression usually compress\nthe large BERT model to a fixed smaller size. They can not fully satisfy the\nrequirements of different edge devices with various hardware performances. In\nthis paper, we propose a novel dynamic BERT model (abbreviated as DynaBERT),\nwhich can flexibly adjust the size and latency by selecting adaptive width and\ndepth. The training process of DynaBERT includes first training a\nwidth-adaptive BERT and then allowing both adaptive width and depth, by\ndistilling knowledge from the full-sized model to small sub-networks. Network\nrewiring is also used to keep the more important attention heads and neurons\nshared by more sub-networks. Comprehensive experiments under various efficiency\nconstraints demonstrate that our proposed dynamic BERT (or RoBERTa) at its\nlargest size has comparable performance as BERT-base (or RoBERTa-base), while\nat smaller widths and depths consistently outperforms existing BERT compression\nmethods. Code is available at\nhttps://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04100v1",
            "date": "2020-04-08 16:25:39+00:00",
            "title": "KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation",
            "authors": [
                "Hao Zhou",
                "Chujie Zheng",
                "Kaili Huang",
                "Minlie Huang",
                "Xiaoyan Zhu"
            ],
            "abstract": "The research of knowledge-driven conversational systems is largely limited\ndue to the lack of dialog data which consist of multi-turn conversations on\nmultiple topics and with knowledge annotations. In this paper, we propose a\nChinese multi-domain knowledge-driven conversation dataset, KdConv, which\ngrounds the topics in multi-turn conversations to knowledge graphs. Our corpus\ncontains 4.5K conversations from three domains (film, music, and travel), and\n86K utterances with an average turn number of 19.0. These conversations contain\nin-depth discussions on related topics and natural transition between multiple\ntopics. To facilitate the following research on this corpus, we provide several\nbenchmark models. Comparative results show that the models can be enhanced by\nintroducing background knowledge, yet there is still a large space for\nleveraging knowledge to model multi-turn conversations for further research.\nResults also show that there are obvious performance differences between\ndifferent domains, indicating that it is worth to further explore transfer\nlearning and domain adaptation. The corpus and benchmark models are publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04124v2",
            "date": "2020-04-08 17:18:56+00:00",
            "title": "LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression",
            "authors": [
                "Yihuan Mao",
                "Yujing Wang",
                "Chufan Wu",
                "Chen Zhang",
                "Yang Wang",
                "Yaming Yang",
                "Quanlu Zhang",
                "Yunhai Tong",
                "Jing Bai"
            ],
            "abstract": "BERT is a cutting-edge language representation model pre-trained by a large\ncorpus, which achieves superior performances on various natural language\nunderstanding tasks. However, a major blocking issue of applying BERT to online\nservices is that it is memory-intensive and leads to unsatisfactory latency of\nuser requests, raising the necessity of model compression. Existing solutions\nleverage the knowledge distillation framework to learn a smaller model that\nimitates the behaviors of BERT. However, the training procedure of knowledge\ndistillation is expensive itself as it requires sufficient training data to\nimitate the teacher model. In this paper, we address this issue by proposing a\nhybrid solution named LadaBERT (Lightweight adaptation of BERT through hybrid\nmodel compression), which combines the advantages of different model\ncompression methods, including weight pruning, matrix factorization and\nknowledge distillation. LadaBERT achieves state-of-the-art accuracy on various\npublic datasets while the training overheads can be reduced by an order of\nmagnitude.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04305v2",
            "date": "2020-04-09 00:10:54+00:00",
            "title": "Conversation Learner -- A Machine Teaching Tool for Building Dialog Managers for Task-Oriented Dialog Systems",
            "authors": [
                "Swadheen Shukla",
                "Lars Liden",
                "Shahin Shayandeh",
                "Eslam Kamal",
                "Jinchao Li",
                "Matt Mazzola",
                "Thomas Park",
                "Baolin Peng",
                "Jianfeng Gao"
            ],
            "abstract": "Traditionally, industry solutions for building a task-oriented dialog system\nhave relied on helping dialog authors define rule-based dialog managers,\nrepresented as dialog flows. While dialog flows are intuitively interpretable\nand good for simple scenarios, they fall short of performance in terms of the\nflexibility needed to handle complex dialogs. On the other hand, purely\nmachine-learned models can handle complex dialogs, but they are considered to\nbe black boxes and require large amounts of training data. In this\ndemonstration, we showcase Conversation Learner, a machine teaching tool for\nbuilding dialog managers. It combines the best of both approaches by enabling\ndialog authors to create a dialog flow using familiar tools, converting the\ndialog flow into a parametric model (e.g., neural networks), and allowing\ndialog authors to improve the dialog manager (i.e., the parametric model) over\ntime by leveraging user-system dialog logs as training data through a machine\nteaching interface.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04312v2",
            "date": "2020-04-09 01:03:44+00:00",
            "title": "Learning to Scale Multilingual Representations for Vision-Language Tasks",
            "authors": [
                "Andrea Burns",
                "Donghyun Kim",
                "Derry Wijaya",
                "Kate Saenko",
                "Bryan A. Plummer"
            ],
            "abstract": "Current multilingual vision-language models either require a large number of\nadditional parameters for each supported language, or suffer performance\ndegradation as languages are added. In this paper, we propose a Scalable\nMultilingual Aligned Language Representation (SMALR) that supports many\nlanguages with few model parameters without sacrificing downstream task\nperformance. SMALR learns a fixed size language-agnostic representation for\nmost words in a multilingual vocabulary, keeping language-specific features for\njust a few. We use a masked cross-language modeling loss to align features with\ncontext from other languages. Additionally, we propose a cross-lingual\nconsistency module that ensures predictions made for a query and its machine\ntranslation are comparable. The effectiveness of SMALR is demonstrated with ten\ndiverse languages, over twice the number supported in vision-language tasks to\ndate. We evaluate on multilingual image-sentence retrieval and outperform prior\nwork by 3-4% with less than 1/5th the training parameters compared to other\nword embedding methods.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.04315v2",
            "date": "2020-04-09 01:07:12+00:00",
            "title": "Large Arabic Twitter Dataset on COVID-19",
            "authors": [
                "Sarah Alqurashi",
                "Ahmad Alhindi",
                "Eisa Alanazi"
            ],
            "abstract": "The 2019 coronavirus disease (COVID-19), emerged late December 2019 in China,\nis now rapidly spreading across the globe. At the time of writing this paper,\nthe number of global confirmed cases has passed two millions and half with over\n180,000 fatalities. Many countries have enforced strict social distancing\npolicies to contain the spread of the virus. This have changed the daily life\nof tens of millions of people, and urged people to turn their discussions\nonline, e.g., via online social media sites like Twitter. In this work, we\ndescribe the first Arabic tweets dataset on COVID-19 that we have been\ncollecting since January 1st, 2020. The dataset would help researchers and\npolicy makers in studying different societal issues related to the pandemic.\nMany other tasks related to behavioral change, information sharing,\nmisinformation and rumors spreading can also be analyzed.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.04343v1",
            "date": "2020-04-08 17:56:58+00:00",
            "title": "Pruning and Sparsemax Methods for Hierarchical Attention Networks",
            "authors": [
                "Jo\u00e3o G. Ribeiro",
                "Frederico S. Felisberto",
                "Isabel C. Neto"
            ],
            "abstract": "This paper introduces and evaluates two novel Hierarchical Attention Network\nmodels [Yang et al., 2016] - i) Hierarchical Pruned Attention Networks, which\nremove the irrelevant words and sentences from the classification process in\norder to reduce potential noise in the document classification accuracy and ii)\nHierarchical Sparsemax Attention Networks, which replace the Softmax function\nused in the attention mechanism with the Sparsemax [Martins and Astudillo,\n2016], capable of better handling importance distributions where a lot of words\nor sentences have very low probabilities. Our empirical evaluation on the IMDB\nReview for sentiment analysis datasets shows both approaches to be able to\nmatch the results obtained by the current state-of-the-art (without, however,\nany significant benefits). All our source code is made available\nathttps://github.com/jmribeiro/dsl-project.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04361v2",
            "date": "2020-04-09 04:14:46+00:00",
            "title": "Calibrating Structured Output Predictors for Natural Language Processing",
            "authors": [
                "Abhyuday Jagannatha",
                "Hong Yu"
            ],
            "abstract": "We address the problem of calibrating prediction confidence for output\nentities of interest in natural language processing (NLP) applications. It is\nimportant that NLP applications such as named entity recognition and question\nanswering produce calibrated confidence scores for their predictions,\nespecially if the system is to be deployed in a safety-critical domain such as\nhealthcare. However, the output space of such structured prediction models is\noften too large to adapt binary or multi-class calibration methods directly. In\nthis study, we propose a general calibration scheme for output entities of\ninterest in neural-network based structured prediction models. Our proposed\nmethod can be used with any binary class calibration scheme and a neural\nnetwork model. Additionally, we show that our calibration method can also be\nused as an uncertainty-aware, entity-specific decoding step to improve the\nperformance of the underlying model at no additional training cost or data\nrequirements. We show that our method outperforms current calibration\ntechniques for named-entity-recognition, part-of-speech and question answering.\nWe also improve our model's performance from our decoding step across several\ntasks and benchmark datasets. Our method improves the calibration and model\nperformance on out-of-domain test scenarios as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04418v2",
            "date": "2020-04-09 08:25:02+00:00",
            "title": "On Optimal Transformer Depth for Low-Resource Language Translation",
            "authors": [
                "Elan van Biljon",
                "Arnu Pretorius",
                "Julia Kreutzer"
            ],
            "abstract": "Transformers have shown great promise as an approach to Neural Machine\nTranslation (NMT) for low-resource languages. However, at the same time,\ntransformer models remain difficult to optimize and require careful tuning of\nhyper-parameters to be useful in this setting. Many NMT toolkits come with a\nset of default hyper-parameters, which researchers and practitioners often\nadopt for the sake of convenience and avoiding tuning. These configurations,\nhowever, have been optimized for large-scale machine translation data sets with\nseveral millions of parallel sentences for European languages like English and\nFrench. In this work, we find that the current trend in the field to use very\nlarge models is detrimental for low-resource languages, since it makes training\nmore difficult and hurts overall performance, confirming previous observations.\nWe see our work as complementary to the Masakhane project (\"Masakhane\" means\n\"We Build Together\" in isiZulu.) In this spirit, low-resource NMT systems are\nnow being built by the community who needs them the most. However, many in the\ncommunity still have very limited access to the type of computational resources\nrequired for building extremely large models promoted by industrial research.\nTherefore, by showing that transformer models perform well (and often best) at\nlow-to-moderate depth, we hope to convince fellow researchers to devote less\ncomputational resources, as well as time, to exploring overly large models\nduring the development of these systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04596v1",
            "date": "2020-04-09 15:34:51+00:00",
            "title": "Global Public Health Surveillance using Media Reports: Redesigning GPHIN",
            "authors": [
                "Dave Carter",
                "Marta Stojanovic",
                "Philip Hachey",
                "Kevin Fournier",
                "Simon Rodier",
                "Yunli Wang",
                "Berry de Bruijn"
            ],
            "abstract": "Global public health surveillance relies on reporting structures and\ntransmission of trustworthy health reports. But in practice, these processes\nmay not always be fast enough, or are hindered by procedural, technical, or\npolitical barriers. GPHIN, the Global Public Health Intelligence Network, was\ndesigned in the late 1990s to scour mainstream news for health events, as that\ntravels faster and more freely. This paper outlines the next generation of\nGPHIN, which went live in 2017, and reports on design decisions underpinning\nits new functions and innovations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04598v1",
            "date": "2020-04-09 15:35:26+00:00",
            "title": "Violent music vs violence and music: Drill rap and violent crime in London",
            "authors": [
                "Bennett Kleinberg",
                "Paul McFarlane"
            ],
            "abstract": "The current policy of removing drill music videos from social media platforms\nsuch as YouTube remains controversial because it risks conflating the\nco-occurrence of drill rap and violence with a causal chain of the two.\nEmpirically, we revisit the question of whether there is evidence to support\nthe conjecture that drill music and gang violence are linked. We provide new\nempirical insights suggesting that: i) drill music lyrics have not become more\nnegative over time if anything they have become more positive; ii) individual\ndrill artists have similar sentiment trajectories to other artists in the drill\ngenre, and iii) there is no meaningful relationship between drill music and\nreal-life violence when compared to three kinds of police-recorded violent\ncrime data in London. We suggest ideas for new work that can help build a\nmuch-needed evidence base around the problem.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.04721v4",
            "date": "2020-04-09 17:54:30+00:00",
            "title": "Translation Artifacts in Cross-lingual Transfer Learning",
            "authors": [
                "Mikel Artetxe",
                "Gorka Labaka",
                "Eneko Agirre"
            ],
            "abstract": "Both human and machine translation play a central role in cross-lingual\ntransfer learning: many multilingual datasets have been created through\nprofessional translation services, and using machine translation to translate\neither the test set or the training set is a widely used transfer technique. In\nthis paper, we show that such translation process can introduce subtle\nartifacts that have a notable impact in existing cross-lingual models. For\ninstance, in natural language inference, translating the premise and the\nhypothesis independently can reduce the lexical overlap between them, which\ncurrent models are highly sensitive to. We show that some previous findings in\ncross-lingual transfer learning need to be reconsidered in the light of this\nphenomenon. Based on the gained insights, we also improve the state-of-the-art\nin XNLI for the translate-test and zero-shot approaches by 4.3 and 2.8 points,\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04803v1",
            "date": "2020-04-09 20:47:15+00:00",
            "title": "FST Morphology for the Endangered Skolt Sami Language",
            "authors": [
                "Jack Rueter",
                "Mika H\u00e4m\u00e4l\u00e4inen"
            ],
            "abstract": "We present advances in the development of a FST-based morphological analyzer\nand generator for Skolt Sami. Like other minority Uralic languages, Skolt Sami\nexhibits a rich morphology, on the one hand, and there is little golden\nstandard material for it, on the other. This makes NLP approaches for its study\ndifficult without a solid morphological analysis. The language is severely\nendangered and the work presented in this paper forms a part of a greater whole\nin its revitalization efforts. Furthermore, we intersperse our description with\nfacilitation and description practices not well documented in the\ninfrastructure. Currently, the analyzer covers over 30,000 Skolt Sami words in\n148 inflectional paradigms and over 12 derivational forms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04938v2",
            "date": "2020-04-10 08:13:07+00:00",
            "title": "Identifying Distributional Perspective Differences from Colingual Groups",
            "authors": [
                "Yufei Tian",
                "Tuhin Chakrabarty",
                "Fred Morstatter",
                "Nanyun Peng"
            ],
            "abstract": "Perspective differences exist among different cultures or languages. A lack\nof mutual understanding among different groups about their perspectives on\nspecific values or events may lead to uninformed decisions or biased opinions.\nAutomatically understanding the group perspectives can provide essential\nbackground for many downstream applications of natural language processing\ntechniques. In this paper, we study colingual groups and use language corpora\nas a proxy to identify their distributional perspectives. We present a novel\ncomputational approach to learn shared understandings, and benchmark our method\nby building culturally-aware models for the English, Chinese, and Japanese\nlanguages. On a held out set of diverse topics including marriage, corruption,\ndemocracy, our model achieves high correlation with human judgements regarding\nintra-group values and inter-group differences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.04945v1",
            "date": "2020-04-10 08:30:50+00:00",
            "title": "The Effect of Sociocultural Variables on Sarcasm Communication Online",
            "authors": [
                "Silviu Vlad Oprea",
                "Walid Magdy"
            ],
            "abstract": "Online social networks (OSN) play an essential role for connecting people and\nallowing them to communicate online. OSN users share their thoughts, moments,\nand news with their network. The messages they share online can include\nsarcastic posts, where the intended meaning expressed by the written text is\ndifferent from the literal one. This could result in miscommunication. Previous\nresearch in psycholinguistics has studied the sociocultural factors the might\nlead to sarcasm misunderstanding between speakers and listeners. However, there\nis a lack of such studies in the context of OSN. In this paper we fill this gap\nby performing a quantitative analysis on the influence of sociocultural\nvariables, including gender, age, country, and English language nativeness, on\nthe effectiveness of sarcastic communication online. We collect examples of\nsarcastic tweets directly from the authors who posted them. Further, we ask\nthird-party annotators of different sociocultural backgrounds to label these\ntweets for sarcasm. Our analysis indicates that age, English language\nnativeness, and country are significantly influential and should be considered\nin the design of future social analysis tools that either study sarcasm\ndirectly, or look at related phenomena where sarcasm may have an influence. We\nalso make observations about the social ecology surrounding sarcastic exchanges\non OSNs. We conclude by suggesting ways in which our findings can be included\nin future work.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.04980v1",
            "date": "2020-04-10 10:38:33+00:00",
            "title": "Negation Detection for Clinical Text Mining in Russian",
            "authors": [
                "Anastasia Funkner",
                "Ksenia Balabaeva",
                "Sergey Kovalchuk"
            ],
            "abstract": "Developing predictive modeling in medicine requires additional features from\nunstructured clinical texts. In Russia, there are no instruments for natural\nlanguage processing to cope with problems of medical records. This paper is\ndevoted to a module of negation detection. The corpus-free machine learning\nmethod is based on gradient boosting classifier is used to detect whether a\ndisease is denied, not mentioned or presented in the text. The detector\nclassifies negations for five diseases and shows average F-score from 0.81 to\n0.93. The benefits of negation detection have been demonstrated by predicting\nthe presence of surgery for patients with the acute coronary syndrome.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05388v1",
            "date": "2020-04-11 12:51:07+00:00",
            "title": "You Impress Me: Dialogue Generation via Mutual Persona Perception",
            "authors": [
                "Qian Liu",
                "Yihong Chen",
                "Bei Chen",
                "Jian-Guang Lou",
                "Zixuan Chen",
                "Bin Zhou",
                "Dongmei Zhang"
            ],
            "abstract": "Despite the continuing efforts to improve the engagingness and consistency of\nchit-chat dialogue systems, the majority of current work simply focus on\nmimicking human-like responses, leaving understudied the aspects of modeling\nunderstanding between interlocutors. The research in cognitive science,\ninstead, suggests that understanding is an essential signal for a high-quality\nchit-chat conversation. Motivated by this, we propose P^2 Bot, a\ntransmitter-receiver based framework with the aim of explicitly modeling\nunderstanding. Specifically, P^2 Bot incorporates mutual persona perception to\nenhance the quality of personalized dialogue generation. Experiments on a large\npublic dataset, Persona-Chat, demonstrate the effectiveness of our approach,\nwith a considerable boost over the state-of-the-art baselines across both\nautomatic metrics and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05438v2",
            "date": "2020-04-11 16:19:02+00:00",
            "title": "Annotating Social Determinants of Health Using Active Learning, and Characterizing Determinants Using Neural Event Extraction",
            "authors": [
                "Kevin Lybarger",
                "Mari Ostendorf",
                "Meliha Yetisgen"
            ],
            "abstract": "Social determinants of health (SDOH) affect health outcomes, and knowledge of\nSDOH can inform clinical decision-making. Automatically extracting SDOH\ninformation from clinical text requires data-driven information extraction\nmodels trained on annotated corpora that are heterogeneous and frequently\ninclude critical SDOH. This work presents a new corpus with SDOH annotations, a\nnovel active learning framework, and the first extraction results on the new\ncorpus. The Social History Annotation Corpus (SHAC) includes 4,480 social\nhistory sections with detailed annotation for 12 SDOH characterizing the\nstatus, extent, and temporal information of 18K distinct events. We introduce a\nnovel active learning framework that selects samples for annotation using a\nsurrogate text classification task as a proxy for a more complex event\nextraction task. The active learning framework successfully increases the\nfrequency of health risk factors and improves automatic extraction of these\nevents over undirected annotation. An event extraction model trained on SHAC\nachieves high extraction performance for substance use status (0.82-0.93 F1),\nemployment status (0.81-0.86 F1), and living status type (0.81-0.93 F1) on data\nfrom three institutions.",
            "journal ref": "Journal of Biomedical Informatics 113 (2021) 103631",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05484v1",
            "date": "2020-04-11 20:51:11+00:00",
            "title": "LAReQA: Language-agnostic answer retrieval from a multilingual pool",
            "authors": [
                "Uma Roy",
                "Noah Constant",
                "Rami Al-Rfou",
                "Aditya Barua",
                "Aaron Phillips",
                "Yinfei Yang"
            ],
            "abstract": "We present LAReQA, a challenging new benchmark for language-agnostic answer\nretrieval from a multilingual candidate pool. Unlike previous cross-lingual\ntasks, LAReQA tests for \"strong\" cross-lingual alignment, requiring\nsemantically related cross-language pairs to be closer in representation space\nthan unrelated same-language pairs. Building on multilingual BERT (mBERT), we\nstudy different strategies for achieving strong alignment. We find that\naugmenting training data via machine translation is effective, and improves\nsignificantly over using mBERT out-of-the-box. Interestingly, the embedding\nbaseline that performs the best on LAReQA falls short of competing baselines on\nzero-shot variants of our task that only target \"weak\" alignment. This finding\nunderscores our claim that languageagnostic retrieval is a substantively new\nkind of cross-lingual evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05572v2",
            "date": "2020-04-12 09:15:21+00:00",
            "title": "AMR Parsing via Graph-Sequence Iterative Inference",
            "authors": [
                "Deng Cai",
                "Wai Lam"
            ],
            "abstract": "We propose a new end-to-end model that treats AMR parsing as a series of dual\ndecisions on the input sequence and the incrementally constructed graph. At\neach time step, our model performs multiple rounds of attention, reasoning, and\ncomposition that aim to answer two critical questions: (1) which part of the\ninput \\textit{sequence} to abstract; and (2) where in the output \\textit{graph}\nto construct the new concept. We show that the answers to these two questions\nare mutually causalities. We design a model based on iterative inference that\nhelps achieve better answers in both perspectives, leading to greatly improved\nparsing accuracy. Our experimental results significantly outperform all\npreviously reported \\textsc{Smatch} scores by large margins. Remarkably,\nwithout the help of any large-scale pre-trained language model (e.g., BERT),\nour model already surpasses previous state-of-the-art using BERT. With the help\nof BERT, we can push the state-of-the-art results to 80.2\\% on LDC2017T10 (AMR\n2.0) and 75.4\\% on LDC2014T12 (AMR 1.0).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05686v2",
            "date": "2020-04-12 19:49:27+00:00",
            "title": "XtremeDistil: Multi-stage Distillation for Massive Multilingual Models",
            "authors": [
                "Subhabrata Mukherjee",
                "Ahmed Awadallah"
            ],
            "abstract": "Deep and large pre-trained language models are the state-of-the-art for\nvarious natural language processing tasks. However, the huge size of these\nmodels could be a deterrent to use them in practice. Some recent and concurrent\nworks use knowledge distillation to compress these huge models into shallow\nones. In this work we study knowledge distillation with a focus on\nmulti-lingual Named Entity Recognition (NER). In particular, we study several\ndistillation strategies and propose a stage-wise optimization scheme leveraging\nteacher internal representations that is agnostic of teacher architecture and\nshow that it outperforms strategies employed in prior works. Additionally, we\ninvestigate the role of several factors like the amount of unlabeled data,\nannotation resources, model architecture and inference latency to name a few.\nWe show that our approach leads to massive compression of MBERT-like teacher\nmodels by upto 35x in terms of parameters and 51x in terms of latency for batch\ninference while retaining 95% of its F1-score for NER over 41 languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05757v1",
            "date": "2020-04-13 03:40:44+00:00",
            "title": "Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models",
            "authors": [
                "Mingjun Zhao",
                "Haijiang Wu",
                "Di Niu",
                "Xiaoli Wang"
            ],
            "abstract": "The competitive performance of neural machine translation (NMT) critically\nrelies on large amounts of training data. However, acquiring high-quality\ntranslation pairs requires expert knowledge and is costly. Therefore, how to\nbest utilize a given dataset of samples with diverse quality and\ncharacteristics becomes an important yet understudied question in NMT.\nCurriculum learning methods have been introduced to NMT to optimize a model's\nperformance by prescribing the data input order, based on heuristics such as\nthe assessment of noise and difficulty levels. However, existing methods\nrequire training from scratch, while in practice most NMT models are\npre-trained on big data already. Moreover, as heuristics, they do not\ngeneralize well. In this paper, we aim to learn a curriculum for improving a\npre-trained NMT model by re-selecting influential data samples from the\noriginal training set and formulate this task as a reinforcement learning\nproblem. Specifically, we propose a data selection framework based on\nDeterministic Actor-Critic, in which a critic network predicts the expected\nchange of model performance due to a certain sample, while an actor network\nlearns to select the best sample out of a random batch of samples presented to\nit. Experiments on several translation datasets show that our method can\nfurther improve the performance of NMT when original batch training reaches its\nceiling, without using additional new training data, and significantly\noutperforms several strong baseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05801v2",
            "date": "2020-04-13 07:31:31+00:00",
            "title": "ProFormer: Towards On-Device LSH Projection Based Transformers",
            "authors": [
                "Chinnadhurai Sankar",
                "Sujith Ravi",
                "Zornitsa Kozareva"
            ],
            "abstract": "At the heart of text based neural models lay word representations, which are\npowerful but occupy a lot of memory making it challenging to deploy to devices\nwith memory constraints such as mobile phones, watches and IoT. To surmount\nthese challenges, we introduce ProFormer -- a projection based transformer\narchitecture that is faster and lighter making it suitable to deploy to memory\nconstraint devices and preserve user privacy. We use LSH projection layer to\ndynamically generate word representations on-the-fly without embedding lookup\ntables leading to significant memory footprint reduction from O(V.d) to O(T),\nwhere V is the vocabulary size, d is the embedding dimension size and T is the\ndimension of the LSH projection representation.\n  We also propose a local projection attention (LPA) layer, which uses\nself-attention to transform the input sequence of N LSH word projections into a\nsequence of N/K representations reducing the computations quadratically by\nO(K^2). We evaluate ProFormer on multiple text classification tasks and\nobserved improvements over prior state-of-the-art on-device approaches for\nshort text classification and comparable performance for long text\nclassification tasks. In comparison with a 2-layer BERT model, ProFormer\nreduced the embedding memory footprint from 92.16 MB to 1.3 KB and requires 16\ntimes less computation overhead, which is very impressive making it the fastest\nand smallest on-device model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.05916v2",
            "date": "2020-04-10 01:36:41+00:00",
            "title": "Telling BERT's full story: from Local Attention to Global Aggregation",
            "authors": [
                "Damian Pascual",
                "Gino Brunner",
                "Roger Wattenhofer"
            ],
            "abstract": "We take a deep look into the behavior of self-attention heads in the\ntransformer architecture. In light of recent work discouraging the use of\nattention distributions for explaining a model's behavior, we show that\nattention distributions can nevertheless provide insights into the local\nbehavior of attention heads. This way, we propose a distinction between local\npatterns revealed by attention and global patterns that refer back to the\ninput, and analyze BERT from both angles. We use gradient attribution to\nanalyze how the output of an attention attention head depends on the input\ntokens, effectively extending the local attention-based analysis to account for\nthe mixing of information throughout the transformer layers. We find that there\nis a significant discrepancy between attention and attribution distributions,\ncaused by the mixing of context inside the model. We quantify this discrepancy\nand observe that interestingly, there are some patterns that persist across all\nlayers despite the mixing.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2004.05986v3",
            "date": "2020-04-13 15:02:29+00:00",
            "title": "CLUE: A Chinese Language Understanding Evaluation Benchmark",
            "authors": [
                "Liang Xu",
                "Hai Hu",
                "Xuanwei Zhang",
                "Lu Li",
                "Chenjie Cao",
                "Yudong Li",
                "Yechen Xu",
                "Kai Sun",
                "Dian Yu",
                "Cong Yu",
                "Yin Tian",
                "Qianqian Dong",
                "Weitang Liu",
                "Bo Shi",
                "Yiming Cui",
                "Junyi Li",
                "Jun Zeng",
                "Rongzhao Wang",
                "Weijian Xie",
                "Yanting Li",
                "Yina Patterson",
                "Zuoyu Tian",
                "Yiwen Zhang",
                "He Zhou",
                "Shaoweihua Liu",
                "Zhe Zhao",
                "Qipeng Zhao",
                "Cong Yue",
                "Xinrui Zhang",
                "Zhengliang Yang",
                "Kyle Richardson",
                "Zhenzhong Lan"
            ],
            "abstract": "The advent of natural language understanding (NLU) benchmarks for English,\nsuch as GLUE and SuperGLUE allows new NLU models to be evaluated across a\ndiverse set of tasks. These comprehensive benchmarks have facilitated a broad\nrange of research and applications in natural language processing (NLP). The\nproblem, however, is that most such benchmarks are limited to English, which\nhas made it difficult to replicate many of the successes in English NLU for\nother languages. To help remedy this issue, we introduce the first large-scale\nChinese Language Understanding Evaluation (CLUE) benchmark. CLUE is an\nopen-ended, community-driven project that brings together 9 tasks spanning\nseveral well-established single-sentence/sentence-pair classification tasks, as\nwell as machine reading comprehension, all on original Chinese text. To\nestablish results on these tasks, we report scores using an exhaustive set of\ncurrent state-of-the-art pre-trained Chinese models (9 in total). We also\nintroduce a number of supplementary datasets and additional tools to help\nfacilitate further progress on Chinese NLU. Our benchmark is released at\nhttps://www.CLUEbenchmarks.com",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06100v2",
            "date": "2020-04-13 17:58:56+00:00",
            "title": "Pretrained Transformers Improve Out-of-Distribution Robustness",
            "authors": [
                "Dan Hendrycks",
                "Xiaoyuan Liu",
                "Eric Wallace",
                "Adam Dziedzic",
                "Rishabh Krishnan",
                "Dawn Song"
            ],
            "abstract": "Although pretrained Transformers such as BERT achieve high accuracy on\nin-distribution examples, do they generalize to new distributions? We\nsystematically measure out-of-distribution (OOD) generalization for seven NLP\ndatasets by constructing a new robustness benchmark with realistic distribution\nshifts. We measure the generalization of previous models including bag-of-words\nmodels, ConvNets, and LSTMs, and we show that pretrained Transformers'\nperformance declines are substantially smaller. Pretrained transformers are\nalso more effective at detecting anomalous or OOD examples, while many previous\nmodels are frequently worse than chance. We examine which factors affect\nrobustness, finding that larger models are not necessarily more robust,\ndistillation can be harmful, and more diverse pretraining data can enhance\nrobustness. Finally, we show where future work can improve OOD robustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06153v2",
            "date": "2020-04-13 18:46:55+00:00",
            "title": "Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification",
            "authors": [
                "Ming Jiang",
                "Jennifer D'Souza",
                "S\u00f6ren Auer",
                "J. Stephen Downie"
            ],
            "abstract": "With the rapid growth of research publications, there is a vast amount of\nscholarly knowledge that needs to be organized in digital libraries. To deal\nwith this challenge, techniques relying on knowledge-graph structures are being\nadvocated. Within such graph-based pipelines, inferring relation types between\nrelated scientific concepts is a crucial step. Recently, advanced techniques\nrelying on language models pre-trained on the large corpus have been popularly\nexplored for automatic relation classification. Despite remarkable\ncontributions that have been made, many of these methods were evaluated under\ndifferent scenarios, which limits their comparability. To this end, we present\na thorough empirical evaluation on eight Bert-based classification models by\nfocusing on two key factors: 1) Bert model variants, and 2) classification\nstrategies. Experiments on three corpora show that domain-specific pre-training\ncorpus benefits the Bert-based classification model to identify the type of\nscientific relations. Although the strategy of predicting a single relation\neach time achieves a higher classification accuracy than the strategy of\nidentifying multiple relation types simultaneously in general, the latter\nstrategy demonstrates a more consistent performance in the corpus with either a\nlarge or small size of annotations. Our study aims to offer recommendations to\nthe stakeholders of digital libraries for selecting the appropriate technique\nto build knowledge-graph-based systems for enhanced scholarly information\norganization.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06216v1",
            "date": "2020-04-13 22:01:38+00:00",
            "title": "Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction",
            "authors": [
                "Hong Guan",
                "Jianfu Li",
                "Hua Xu",
                "Murthy Devarakonda"
            ],
            "abstract": "Background: Identifying relationships between clinical events and temporal\nexpressions is a key challenge in meaningfully analyzing clinical text for use\nin advanced AI applications. While previous studies exist, the state-of-the-art\nperformance has significant room for improvement.\n  Methods: We studied several variants of BERT (Bidirectional Encoder\nRepresentations using Transformers) some involving clinical domain\ncustomization and the others involving improved architecture and/or training\nstrategies. We evaluated these methods using a direct temporal relations\ndataset which is a semantically focused subset of the 2012 i2b2 temporal\nrelations challenge dataset.\n  Results: Our results show that RoBERTa, which employs better pre-training\nstrategies including using 10x larger corpus, has improved overall F measure by\n0.0864 absolute score (on the 1.00 scale) and thus reducing the error rate by\n24% relative to the previous state-of-the-art performance achieved with an SVM\n(support vector machine) model.\n  Conclusion: Modern contextual language modeling neural networks, pre-trained\non a large corpus, achieve impressive performance even on highly-nuanced\nclinical temporal relation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06384v2",
            "date": "2020-04-14 09:39:35+00:00",
            "title": "Incorporating Uncertain Segmentation Information into Chinese NER for Social Media Text",
            "authors": [
                "Shengbin Jia",
                "Ling Ding",
                "Xiaojun Chen",
                "Shijia E",
                "Yang Xiang"
            ],
            "abstract": "Chinese word segmentation is necessary to provide word-level information for\nChinese named entity recognition (NER) systems. However, segmentation error\npropagation is a challenge for Chinese NER while processing colloquial data\nlike social media text. In this paper, we propose a model (UIcwsNN) that\nspecializes in identifying entities from Chinese social media text, especially\nby leveraging ambiguous information of word segmentation. Such uncertain\ninformation contains all the potential segmentation states of a sentence that\nprovides a channel for the model to infer deep word-level characteristics. We\npropose a trilogy (i.e., candidate position embedding -> position selective\nattention -> adaptive word convolution) to encode uncertain word segmentation\ninformation and acquire appropriate word-level representation. Experiments\nresults on the social media corpus show that our model alleviates the\nsegmentation error cascading trouble effectively, and achieves a significant\nperformance improvement of more than 2% over previous state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06427v1",
            "date": "2020-04-14 11:27:30+00:00",
            "title": "Jointly Modeling Aspect and Sentiment with Dynamic Heterogeneous Graph Neural Networks",
            "authors": [
                "Shu Liu",
                "Wei Li",
                "Yunfang Wu",
                "Qi Su",
                "Xu Sun"
            ],
            "abstract": "Target-Based Sentiment Analysis aims to detect the opinion aspects (aspect\nextraction) and the sentiment polarities (sentiment detection) towards them.\nBoth the previous pipeline and integrated methods fail to precisely model the\ninnate connection between these two objectives. In this paper, we propose a\nnovel dynamic heterogeneous graph to jointly model the two objectives in an\nexplicit way. Both the ordinary words and sentiment labels are treated as nodes\nin the heterogeneous graph, so that the aspect words can interact with the\nsentiment information. The graph is initialized with multiple types of\ndependencies, and dynamically modified during real-time prediction. Experiments\non the benchmark datasets show that our model outperforms the state-of-the-art\nmodels. Further analysis demonstrates that our model obtains significant\nperformance gain on the challenging instances under multiple-opinion aspects\nand no-opinion aspect situations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06465v3",
            "date": "2020-04-14 13:14:27+00:00",
            "title": "Deep Learning Models for Multilingual Hate Speech Detection",
            "authors": [
                "Sai Saketh Aluru",
                "Binny Mathew",
                "Punyajoy Saha",
                "Animesh Mukherjee"
            ],
            "abstract": "Hate speech detection is a challenging problem with most of the datasets\navailable in only one language: English. In this paper, we conduct a large\nscale analysis of multilingual hate speech in 9 languages from 16 different\nsources. We observe that in low resource setting, simple models such as LASER\nembedding with logistic regression performs the best, while in high resource\nsetting BERT based models perform better. In case of zero-shot classification,\nlanguages such as Italian and Portuguese achieve good results. Our proposed\nframework could be used as an efficient solution for low-resource languages.\nThese models could also act as good baselines for future multilingual hate\nspeech detection tasks. We have made our code and experimental settings public\nfor other researchers at https://github.com/punyajoy/DE-LIMIT.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.06519v1",
            "date": "2020-04-13 15:46:47+00:00",
            "title": "Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora",
            "authors": [
                "Federico Bianchi",
                "Valerio Di Carlo",
                "Paolo Nicoli",
                "Matteo Palmonari"
            ],
            "abstract": "Word2vec is one of the most used algorithms to generate word embeddings\nbecause of a good mix of efficiency, quality of the generated representations\nand cognitive grounding. However, word meaning is not static and depends on the\ncontext in which words are used. Differences in word meaning that depends on\ntime, location, topic, and other factors, can be studied by analyzing\nembeddings generated from different corpora in collections that are\nrepresentative of these factors. For example, language evolution can be studied\nusing a collection of news articles published in different time periods. In\nthis paper, we present a general framework to support cross-corpora language\nstudies with word embeddings, where embeddings generated from different corpora\ncan be compared to find correspondences and differences in meaning across the\ncorpora. CADE is the core component of our framework and solves the key problem\nof aligning the embeddings generated from different corpora. In particular, we\nfocus on providing solid evidence about the effectiveness, generality, and\nrobustness of CADE. To this end, we conduct quantitative and qualitative\nexperiments in different domains, from temporal word embeddings to language\nlocalization and topical analysis. The results of our experiments suggest that\nCADE achieves state-of-the-art or superior performance on tasks where several\ncompeting approaches are available, yet providing a general method that can be\nused in a variety of domains. Finally, our experiments shed light on the\nconditions under which the alignment is reliable, which substantially depends\non the degree of cross-corpora vocabulary overlap.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.06608v2",
            "date": "2020-04-14 15:51:02+00:00",
            "title": "Multi-source Attention for Unsupervised Domain Adaptation",
            "authors": [
                "Xia Cui",
                "Danushka Bollegala"
            ],
            "abstract": "Domain adaptation considers the problem of generalising a model learnt using\ndata from a particular source domain to a different target domain. Often it is\ndifficult to find a suitable single source to adapt from, and one must consider\nmultiple sources. Using an unrelated source can result in sub-optimal\nperformance, known as the \\emph{negative transfer}. However, it is challenging\nto select the appropriate source(s) for classifying a given target instance in\nmulti-source unsupervised domain adaptation (UDA). We model source-selection as\nan attention-learning problem, where we learn attention over sources for a\ngiven target instance. For this purpose, we first independently learn\nsource-specific classification models, and a relatedness map between sources\nand target domains using pseudo-labelled target domain instances. Next, we\nlearn attention-weights over the sources for aggregating the predictions of the\nsource-specific models. Experimental results on cross-domain sentiment\nclassification benchmarks show that the proposed method outperforms prior\nproposals in multi-source UDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06747v1",
            "date": "2020-04-14 18:22:48+00:00",
            "title": "Extending Text Informativeness Measures to Passage Interestingness Evaluation (Language Model vs. Word Embedding)",
            "authors": [
                "Carlos-Emiliano Gonz\u00e1lez-Gallardo",
                "Eric SanJuan",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Standard informativeness measures used to evaluate Automatic Text\nSummarization mostly rely on n-gram overlapping between the automatic summary\nand the reference summaries. These measures differ from the metric they use\n(cosine, ROUGE, Kullback-Leibler, Logarithm Similarity, etc.) and the bag of\nterms they consider (single words, word n-grams, entities, nuggets, etc.).\nRecent word embedding approaches offer a continuous alternative to discrete\napproaches based on the presence/absence of a text unit. Informativeness\nmeasures have been extended to Focus Information Retrieval evaluation involving\na user's information need represented by short queries. In particular for the\ntask of CLEF-INEX Tweet Contextualization, tweet contents have been considered\nas queries. In this paper we define the concept of Interestingness as a\ngeneralization of Informativeness, whereby the information need is diverse and\nformalized as an unknown set of implicit queries. We then study the ability of\nstate of the art Informativeness measures to cope with this generalization.\nLately we show that with this new framework, standard word embeddings\noutperforms discrete measures only on uni-grams, however bi-grams seems to be a\nkey point of interestingness evaluation. Lastly we prove that the CLEF-INEX\nTweet Contextualization 2012 Logarithm Similarity measure provides best\nresults.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.06866v1",
            "date": "2020-04-15 03:37:47+00:00",
            "title": "On the Linguistic Capacity of Real-Time Counter Automata",
            "authors": [
                "William Merrill"
            ],
            "abstract": "Counter machines have achieved a newfound relevance to the field of natural\nlanguage processing (NLP): recent work suggests some strong-performing\nrecurrent neural networks utilize their memory as counters. Thus, one potential\nway to understand the success of these networks is to revisit the theory of\ncounter computation. Therefore, we study the abilities of real-time counter\nmachines as formal grammars, focusing on formal properties that are relevant\nfor NLP models. We first show that several variants of the counter machine\nconverge to express the same class of formal languages. We also prove that\ncounter languages are closed under complement, union, intersection, and many\nother common set operations. Next, we show that counter machines cannot\nevaluate boolean expressions, even though they can weakly validate their\nsyntax. This has implications for the interpretability and evaluation of neural\nnetwork systems: successfully matching syntactic patterns does not guarantee\nthat counter memory accurately encodes compositional semantics. Finally, we\nconsider whether counter languages are semilinear. This work makes general\ncontributions to the theory of formal languages that are of potential interest\nfor understanding recurrent neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.06986v2",
            "date": "2020-04-15 10:14:15+00:00",
            "title": "Framing COVID-19: How we conceptualize and discuss the pandemic on Twitter",
            "authors": [
                "Philipp Wicke",
                "Marianna M. Bolognesi"
            ],
            "abstract": "Doctors and nurses in these weeks are busy in the trenches, fighting against\na new invisible enemy: Covid-19. Cities are locked down and civilians are\nbesieged in their own homes, to prevent the spreading of the virus. War-related\nterminology is commonly used to frame the discourse around epidemics and\ndiseases. Arguably the discourse around the current epidemic will make use of\nwar-related metaphors too,not only in public discourse and the media, but also\nin the tweets written by non-experts of mass communication. We hereby present\nan analysis of the discourse around #Covid-19, based on a corpus of 200k tweets\nposted on Twitter during March and April 2020. Using topic modelling we first\nanalyze the topics around which the discourse can be classified. Then, we show\nthat the WAR framing is used to talk about specific topics, such as the virus\ntreatment, but not others, such as the effects of social distancing on the\npopulation. We then measure and compare the popularity of the WAR frame to\nthree alternative figurative frames (MONSTER, STORM and TSUNAMI) and a literal\nframe used as control (FAMILY). The results show that while the FAMILY literal\nframe covers a wider portion of the corpus, among the figurative framings WAR\nis the most frequently used, and thus arguably the most conventional one.\nHowever, we conclude, this frame is not apt to elaborate the discourse around\nmany aspects involved in the current situation. Therefore, we conclude, in line\nwith previous suggestions, a plethora of framing options, or a metaphor menu,\nmay facilitate the communication of various aspects involved in the\nCovid-19-related discourse on the social media, and thus support civilians in\nthe expression of their feelings, opinions and ideas during the current\npandemic.",
            "journal ref": "PLOS ONE 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07202v2",
            "date": "2020-04-15 17:00:05+00:00",
            "title": "Entities as Experts: Sparse Memory Access with Entity Supervision",
            "authors": [
                "Thibault F\u00e9vry",
                "Livio Baldini Soares",
                "Nicholas FitzGerald",
                "Eunsol Choi",
                "Tom Kwiatkowski"
            ],
            "abstract": "We focus on the problem of capturing declarative knowledge about entities in\nthe learned parameters of a language model. We introduce a new model - Entities\nas Experts (EAE) - that can access distinct memories of the entities mentioned\nin a piece of text. Unlike previous efforts to integrate entity knowledge into\nsequence models, EAE's entity representations are learned directly from text.\nWe show that EAE's learned representations capture sufficient knowledge to\nanswer TriviaQA questions such as \"Which Dr. Who villain has been played by\nRoger Delgado, Anthony Ainley, Eric Roberts?\", outperforming an\nencoder-generator Transformer model with 10x the parameters. According to the\nLAMA knowledge probes, EAE contains more factual knowledge than a similarly\nsized BERT, as well as previous approaches that integrate external sources of\nentity knowledge. Because EAE associates parameters with specific entities, it\nonly needs to access a fraction of its parameters at inference time, and we\nshow that the correct identification and representation of entities is\nessential to EAE's performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07347v3",
            "date": "2020-04-15 21:18:15+00:00",
            "title": "HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data",
            "authors": [
                "Wenhu Chen",
                "Hanwen Zha",
                "Zhiyu Chen",
                "Wenhan Xiong",
                "Hong Wang",
                "William Wang"
            ],
            "abstract": "Existing question answering datasets focus on dealing with homogeneous\ninformation, based either only on text or KB/Table information alone. However,\nas human knowledge is distributed over heterogeneous forms, using homogeneous\ninformation alone might lead to severe coverage problems. To fill in the gap,\nwe present HybridQA https://github.com/wenhuchen/HybridQA, a new large-scale\nquestion-answering dataset that requires reasoning on heterogeneous\ninformation. Each question is aligned with a Wikipedia table and multiple\nfree-form corpora linked with the entities in the table. The questions are\ndesigned to aggregate both tabular information and text information, i.e., lack\nof either form would render the question unanswerable. We test with three\ndifferent models: 1) a table-only model. 2) text-only model. 3) a hybrid model\nthat combines heterogeneous information to find the answer. The experimental\nresults show that the EM scores obtained by two baselines are below 20\\%, while\nthe hybrid model can achieve an EM over 40\\%. This gap suggests the necessity\nto aggregate heterogeneous information in HybridQA. However, the hybrid model's\nscore is still far behind human performance. Hence, HybridQA can serve as a\nchallenging benchmark to study question answering with heterogeneous\ninformation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07437v3",
            "date": "2020-04-16 03:45:56+00:00",
            "title": "Non-Autoregressive Machine Translation with Latent Alignments",
            "authors": [
                "Chitwan Saharia",
                "William Chan",
                "Saurabh Saxena",
                "Mohammad Norouzi"
            ],
            "abstract": "This paper presents two strong methods, CTC and Imputer, for\nnon-autoregressive machine translation that model latent alignments with\ndynamic programming. We revisit CTC for machine translation and demonstrate\nthat a simple CTC model can achieve state-of-the-art for single-step\nnon-autoregressive machine translation, contrary to what prior work indicates.\nIn addition, we adapt the Imputer model for non-autoregressive machine\ntranslation and demonstrate that Imputer with just 4 generation steps can match\nthe performance of an autoregressive Transformer baseline. Our latent alignment\nmodels are simpler than many existing non-autoregressive translation baselines;\nfor example, we do not require target length prediction or re-scoring with an\nautoregressive model. On the competitive WMT'14 En$\\rightarrow$De task, our CTC\nmodel achieves 25.7 BLEU with a single generation step, while Imputer achieves\n27.5 BLEU with 2 generation steps, and 28.0 BLEU with 4 generation steps. This\ncompares favourably to the autoregressive Transformer baseline at 27.8 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07453v2",
            "date": "2020-04-16 04:28:08+00:00",
            "title": "The Right Tool for the Job: Matching Model and Instance Complexities",
            "authors": [
                "Roy Schwartz",
                "Gabriel Stanovsky",
                "Swabha Swayamdipta",
                "Jesse Dodge",
                "Noah A. Smith"
            ],
            "abstract": "As NLP models become larger, executing a trained model requires significant\ncomputational resources incurring monetary and environmental costs. To better\nrespect a given inference budget, we propose a modification to contextual\nrepresentation fine-tuning which, during inference, allows for an early (and\nfast) \"exit\" from neural network calculations for simple instances, and late\n(and accurate) exit for hard instances. To achieve this, we add classifiers to\ndifferent layers of BERT and use their calibrated confidence scores to make\nearly exit decisions. We test our proposed modification on five different\ndatasets in two tasks: three text classification datasets and two natural\nlanguage inference benchmarks. Our method presents a favorable speed/accuracy\ntradeoff in almost all cases, producing models which are up to five times\nfaster than the state of the art, while preserving their accuracy. Our method\nalso requires almost no additional training resources (in either time or\nparameters) compared to the baseline BERT model. Finally, our method alleviates\nthe need for costly retraining of multiple models at different levels of\nefficiency; we allow users to control the inference speed/accuracy tradeoff\nusing a single trained model, by setting a single variable at inference time.\nWe publicly release our code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07462v2",
            "date": "2020-04-16 05:12:36+00:00",
            "title": "Paraphrase Augmented Task-Oriented Dialog Generation",
            "authors": [
                "Silin Gao",
                "Yichi Zhang",
                "Zhijian Ou",
                "Zhou Yu"
            ],
            "abstract": "Neural generative models have achieved promising performance on dialog\ngeneration tasks if given a huge data set. However, the lack of high-quality\ndialog data and the expensive data annotation process greatly limit their\napplication in real-world settings. We propose a paraphrase augmented response\ngeneration (PARG) framework that jointly trains a paraphrase model and a\nresponse generation model to improve the dialog generation performance. We also\ndesign a method to automatically construct paraphrase training data set based\non dialog state and dialog act labels. PARG is applicable to various dialog\ngeneration models, such as TSCP (Lei et al., 2018) and DAMD (Zhang et al.,\n2019). Experimental results show that the proposed framework improves these\nstate-of-the-art dialog models further on CamRest676 and MultiWOZ. PARG also\nsignificantly outperforms other data augmentation methods in dialog generation\ntasks, especially under low resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07623v2",
            "date": "2020-04-04 14:19:15+00:00",
            "title": "Recognizing Long Grammatical Sequences Using Recurrent Networks Augmented With An External Differentiable Stack",
            "authors": [
                "Ankur Mali",
                "Alexander Ororbia",
                "Daniel Kifer",
                "Clyde Lee Giles"
            ],
            "abstract": "Recurrent neural networks (RNNs) are a widely used deep architecture for\nsequence modeling, generation, and prediction. Despite success in applications\nsuch as machine translation and voice recognition, these stateful models have\nseveral critical shortcomings. Specifically, RNNs generalize poorly over very\nlong sequences, which limits their applicability to many important temporal\nprocessing and time series forecasting problems. For example, RNNs struggle in\nrecognizing complex context free languages (CFLs), never reaching 100% accuracy\non training. One way to address these shortcomings is to couple an RNN with an\nexternal, differentiable memory structure, such as a stack. However,\ndifferentiable memories in prior work have neither been extensively studied on\nCFLs nor tested on sequences longer than those seen in training. The few\nefforts that have studied them have shown that continuous differentiable memory\nstructures yield poor generalization for complex CFLs, making the RNN less\ninterpretable. In this paper, we improve the memory-augmented RNN with\nimportant architectural and state updating mechanisms that ensure that the\nmodel learns to properly balance the use of its latent states with external\nmemory. Our improved RNN models exhibit better generalization performance and\nare able to classify long strings generated by complex hierarchical context\nfree grammars (CFGs). We evaluate our models on CGGs, including the Dyck\nlanguages, as well as on the Penn Treebank language modelling task, and achieve\nstable, robust performance across these benchmarks. Furthermore, we show that\nonly our memory-augmented networks are capable of retaining memory for a longer\nduration up to strings of length 160.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07667v2",
            "date": "2020-04-16 14:02:50+00:00",
            "title": "Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection",
            "authors": [
                "Shauli Ravfogel",
                "Yanai Elazar",
                "Hila Gonen",
                "Michael Twiton",
                "Yoav Goldberg"
            ],
            "abstract": "The ability to control for the kinds of information encoded in neural\nrepresentation has a variety of use cases, especially in light of the challenge\nof interpreting these models. We present Iterative Null-space Projection\n(INLP), a novel method for removing information from neural representations.\nOur method is based on repeated training of linear classifiers that predict a\ncertain property we aim to remove, followed by projection of the\nrepresentations on their null-space. By doing so, the classifiers become\noblivious to that target property, making it hard to linearly separate the data\naccording to it. While applicable for multiple uses, we evaluate our method on\nbias and fairness use-cases, and show that our method is able to mitigate bias\nin word embeddings, as well as to increase fairness in a setting of multi-class\nclassification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.07683v2",
            "date": "2020-04-16 14:43:27+00:00",
            "title": "Do sequence-to-sequence VAEs learn global features of sentences?",
            "authors": [
                "Tom Bosc",
                "Pascal Vincent"
            ],
            "abstract": "Autoregressive language models are powerful and relatively easy to train.\nHowever, these models are usually trained without explicit conditioning labels\nand do not offer easy ways to control global aspects such as sentiment or topic\nduring generation. Bowman & al. (2016) adapted the Variational Autoencoder\n(VAE) for natural language with the sequence-to-sequence architecture and\nclaimed that the latent vector was able to capture such global features in an\nunsupervised manner. We question this claim. We measure which words benefit\nmost from the latent information by decomposing the reconstruction loss per\nposition in the sentence. Using this method, we find that VAEs are prone to\nmemorizing the first words and the sentence length, producing local features of\nlimited usefulness. To alleviate this, we investigate alternative architectures\nbased on bag-of-words assumptions and language model pretraining. These\nvariants learn latent variables that are more global, i.e., more predictive of\ntopic or sentiment labels. Moreover, using reconstructions, we observe that\nthey decrease memorization: the first word and the sentence length are not\nrecovered as accurately than with the baselines, consequently yielding more\ndiverse reconstructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08022v2",
            "date": "2020-04-17 01:40:18+00:00",
            "title": "SongNet: Rigid Formats Controlled Text Generation",
            "authors": [
                "Piji Li",
                "Haisong Zhang",
                "Xiaojiang Liu",
                "Shuming Shi"
            ],
            "abstract": "Neural text generation has made tremendous progress in various tasks. One\ncommon characteristic of most of the tasks is that the texts are not restricted\nto some rigid formats when generating. However, we may confront some special\ntext paradigms such as Lyrics (assume the music score is given), Sonnet, SongCi\n(classical Chinese poetry of the Song dynasty), etc. The typical\ncharacteristics of these texts are in three folds: (1) They must comply fully\nwith the rigid predefined formats. (2) They must obey some rhyming schemes. (3)\nAlthough they are restricted to some formats, the sentence integrity must be\nguaranteed. To the best of our knowledge, text generation based on the\npredefined rigid formats has not been well investigated. Therefore, we propose\na simple and elegant framework named SongNet to tackle this problem. The\nbackbone of the framework is a Transformer-based auto-regressive language\nmodel. Sets of symbols are tailor-designed to improve the modeling performance\nespecially on format, rhyme, and sentence integrity. We improve the attention\nmechanism to impel the model to capture some future information on the format.\nA pre-training and fine-tuning framework is designed to further improve the\ngeneration quality. Extensive experiments conducted on two collected corpora\ndemonstrate that our proposed framework generates significantly better results\nin terms of both automatic metrics and the human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08046v2",
            "date": "2020-04-17 03:12:34+00:00",
            "title": "Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space",
            "authors": [
                "Dongyu Ru",
                "Jiangtao Feng",
                "Lin Qiu",
                "Hao Zhou",
                "Mingxuan Wang",
                "Weinan Zhang",
                "Yong Yu",
                "Lei Li"
            ],
            "abstract": "Active learning for sentence understanding aims at discovering informative\nunlabeled data for annotation and therefore reducing the demand for labeled\ndata. We argue that the typical uncertainty sampling method for active learning\nis time-consuming and can hardly work in real-time, which may lead to\nineffective sample selection. We propose adversarial uncertainty sampling in\ndiscrete space (AUSDS) to retrieve informative unlabeled samples more\nefficiently. AUSDS maps sentences into latent space generated by the popular\npre-trained language models, and discover informative unlabeled text samples\nfor annotation via adversarial attack. The proposed approach is extremely\nefficient compared with traditional uncertainty sampling with more than 10x\nspeedup. Experimental results on five datasets show that AUSDS outperforms\nstrong baselines on effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08070v2",
            "date": "2020-04-17 05:44:37+00:00",
            "title": "Transform and Tell: Entity-Aware News Image Captioning",
            "authors": [
                "Alasdair Tran",
                "Alexander Mathews",
                "Lexing Xie"
            ],
            "abstract": "We propose an end-to-end model which generates captions for images embedded\nin news articles. News images present two key challenges: they rely on\nreal-world knowledge, especially about named entities; and they typically have\nlinguistically rich captions that include uncommon words. We address the first\nchallenge by associating words in the caption with faces and objects in the\nimage, via a multi-modal, multi-head attention mechanism. We tackle the second\nchallenge with a state-of-the-art transformer language model that uses\nbyte-pair-encoding to generate captions as a sequence of word parts. On the\nGoodNews dataset, our model outperforms the previous state of the art by a\nfactor of four in CIDEr score (13 to 54). This performance gain comes from a\nunique combination of language models, word representation, image embeddings,\nface embeddings, object embeddings, and improvements in neural network design.\nWe also introduce the NYTimes800k dataset which is 70% larger than GoodNews,\nhas higher article quality, and includes the locations of images within\narticles as an additional contextual cue.",
            "journal ref": "The IEEE Conference on Computer Vision and Pattern Recognition\n  (CVPR), 2020, pp. 13035-13045",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.08123v1",
            "date": "2020-04-17 08:59:13+00:00",
            "title": "Batch Clustering for Multilingual News Streaming",
            "authors": [
                "Mathis Linger",
                "Mhamed Hajaiej"
            ],
            "abstract": "Nowadays, digital news articles are widely available, published by various\neditors and often written in different languages. This large volume of diverse\nand unorganized information makes human reading very difficult or almost\nimpossible. This leads to a need for algorithms able to arrange high amount of\nmultilingual news into stories. To this purpose, we extend previous works on\nTopic Detection and Tracking, and propose a new system inspired from newsLens.\nWe process articles per batch, looking for monolingual local topics which are\nthen linked across time and languages. Here, we introduce a novel \"replaying\"\nstrategy to link monolingual local topics into stories. Besides, we propose new\nfine tuned multilingual embedding using SBERT to create crosslingual stories.\nOur system gives monolingual state-of-the-art results on dataset of Spanish and\nGerman news and crosslingual state-of-the-art results on English, Spanish and\nGerman news.",
            "journal ref": "Proceedings of Text2Story - Third Workshop on Narrative Extraction\n  From Texts co-located with 42nd European Conference on Information Retrieval\n  (ECIR 2020) Lisbon, Portugal, April 14th, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08178v5",
            "date": "2020-04-17 11:25:07+00:00",
            "title": "Highway Transformer: Self-Gating Enhanced Self-Attentive Networks",
            "authors": [
                "Yekun Chai",
                "Shuo Jin",
                "Xinwen Hou"
            ],
            "abstract": "Self-attention mechanisms have made striking state-of-the-art (SOTA) progress\nin various sequence learning tasks, standing on the multi-headed dot product\nattention by attending to all the global contexts at different locations.\nThrough a pseudo information highway, we introduce a gated component\nself-dependency units (SDU) that incorporates LSTM-styled gating units to\nreplenish internal semantic importance within the multi-dimensional latent\nspace of individual representations. The subsidiary content-based SDU gates\nallow for the information flow of modulated latent embeddings through skipped\nconnections, leading to a clear margin of convergence speed with gradient\ndescent algorithms. We may unveil the role of gating mechanism to aid in the\ncontext-based Transformer modules, with hypothesizing that SDU gates,\nespecially on shallow layers, could push it faster to step towards suboptimal\npoints during the optimization process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08202v2",
            "date": "2020-04-17 12:23:46+00:00",
            "title": "Women worry about family, men about the economy: Gender differences in emotional responses to COVID-19",
            "authors": [
                "Isabelle van der Vegt",
                "Bennett Kleinberg"
            ],
            "abstract": "Among the critical challenges around the COVID-19 pandemic is dealing with\nthe potentially detrimental effects on people's mental health. Designing\nappropriate interventions and identifying the concerns of those most at risk\nrequires methods that can extract worries, concerns and emotional responses\nfrom text data. We examine gender differences and the effect of document length\non worries about the ongoing COVID-19 situation. Our findings suggest that i)\nshort texts do not offer as adequate insights into psychological processes as\nlonger texts. We further find ii) marked gender differences in topics\nconcerning emotional responses. Women worried more about their loved ones and\nsevere health concerns while men were more occupied with effects on the economy\nand society. This paper adds to the understanding of general gender differences\nin language found elsewhere, and shows that the current unique circumstances\nlikely amplified these effects. We close this paper with a call for more\nhigh-quality datasets due to the limitations of Tweet-sized data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08205v1",
            "date": "2020-04-16 14:23:13+00:00",
            "title": "Large-scale analysis of grooming in modern social networks",
            "authors": [
                "Nikolaos Lykousas",
                "Constantinos Patsakis"
            ],
            "abstract": "Social networks are evolving to engage their users more by providing them\nwith more functionalities. One of the most attracting ones is streaming. Users\nmay broadcast part of their daily lives to thousands of others world-wide and\ninteract with them in real-time. Unfortunately, this feature is reportedly\nexploited for grooming. In this work, we provide the first in-depth analysis of\nthis problem for social live streaming services. More precisely, using a\ndataset that we collected, we identify predatory behaviours and grooming on\nchats that bypassed the moderation mechanisms of the LiveMe, the service under\ninvestigation. Beyond the traditional text approaches, we also investigate the\nrelevance of emojis in this context, as well as the user interactions through\nthe gift mechanisms of LiveMe. Finally, our analysis indicates the possibility\nof grooming towards minors, showing the extent of the problem in such\nplatforms.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.08299v1",
            "date": "2020-04-01 07:10:08+00:00",
            "title": "DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator",
            "authors": [
                "Hwanhee Lee",
                "Seunghyun Yoon",
                "Franck Dernoncourt",
                "Doo Soon Kim",
                "Trung Bui",
                "Kyomin Jung"
            ],
            "abstract": "Audio Visual Scene-aware Dialog (AVSD) is the task of generating a response\nfor a question with a given scene, video, audio, and the history of previous\nturns in the dialog. Existing systems for this task employ the transformers or\nrecurrent neural network-based architecture with the encoder-decoder framework.\nEven though these techniques show superior performance for this task, they have\nsignificant limitations: the model easily overfits only to memorize the\ngrammatical patterns; the model follows the prior distribution of the\nvocabularies in a dataset. To alleviate the problems, we propose a Multimodal\nSemantic Transformer Network. It employs a transformer-based architecture with\nan attention-based word embedding layer that generates words by querying word\nembeddings. With this design, our model keeps considering the meaning of the\nwords at the generation stage. The empirical results demonstrate the\nsuperiority of our proposed model that outperforms most of the previous works\nfor the AVSD task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08355v1",
            "date": "2020-04-17 17:22:52+00:00",
            "title": "Towards an Interoperable Ecosystem of AI and LT Platforms: A Roadmap for the Implementation of Different Levels of Interoperability",
            "authors": [
                "Georg Rehm",
                "Dimitrios Galanis",
                "Penny Labropoulou",
                "Stelios Piperidis",
                "Martin Wel\u00df",
                "Ricardo Usbeck",
                "Joachim K\u00f6hler",
                "Miltos Deligiannis",
                "Katerina Gkirtzou",
                "Johannes Fischer",
                "Christian Chiarcos",
                "Nils Feldhus",
                "Juli\u00e1n Moreno-Schneider",
                "Florian Kintzel",
                "Elena Montiel",
                "V\u00edctor Rodr\u00edguez Doncel",
                "John P. McCrae",
                "David Laqua",
                "Irina Patricia Theile",
                "Christian Dittmar",
                "Kalina Bontcheva",
                "Ian Roberts",
                "Andrejs Vasiljevs",
                "Andis Lagzdi\u0146\u0161"
            ],
            "abstract": "With regard to the wider area of AI/LT platform interoperability, we\nconcentrate on two core aspects: (1) cross-platform search and discovery of\nresources and services; (2) composition of cross-platform service workflows. We\ndevise five different levels (of increasing complexity) of platform\ninteroperability that we suggest to implement in a wider federation of AI/LT\nplatforms. We illustrate the approach using the five emerging AI/LT platforms\nAI4EU, ELG, Lynx, QURATOR and SPEAKER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08371v1",
            "date": "2020-04-17 17:49:45+00:00",
            "title": "Exploring the Combination of Contextual Word Embeddings and Knowledge Graph Embeddings",
            "authors": [
                "Lea Dieudonat",
                "Kelvin Han",
                "Phyllicia Leavitt",
                "Esteban Marquer"
            ],
            "abstract": "``Classical'' word embeddings, such as Word2Vec, have been shown to capture\nthe semantics of words based on their distributional properties. However, their\nability to represent the different meanings that a word may have is limited.\nSuch approaches also do not explicitly encode relations between entities, as\ndenoted by words. Embeddings of knowledge bases (KB) capture the explicit\nrelations between entities denoted by words, but are not able to directly\ncapture the syntagmatic properties of these words. To our knowledge, recent\nresearch have focused on representation learning that augment the strengths of\none with the other. In this work, we begin exploring another approach using\ncontextual and KB embeddings jointly at the same level and propose two tasks --\nan entity typing and a relation typing task -- that evaluate the performance of\ncontextual and KB embeddings. We also evaluated a concatenated model of\ncontextual and KB embeddings with these two tasks, and obtain conclusive\nresults on the first task. We hope our work may contribute as a basis for\nmodels and datasets that develop in the direction of this approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08385v1",
            "date": "2020-04-17 02:06:26+00:00",
            "title": "Knowledge-Based Visual Question Answering in Videos",
            "authors": [
                "Noa Garcia",
                "Mayu Otani",
                "Chenhui Chu",
                "Yuta Nakashima"
            ],
            "abstract": "We propose a novel video understanding task by fusing knowledge-based and\nvideo question answering. First, we introduce KnowIT VQA, a video dataset with\n24,282 human-generated question-answer pairs about a popular sitcom. The\ndataset combines visual, textual and temporal coherence reasoning together with\nknowledge-based questions, which need of the experience obtained from the\nviewing of the series to be answered. Second, we propose a video understanding\nmodel by combining the visual and textual video content with specific knowledge\nabout the show. Our main findings are: (i) the incorporation of knowledge\nproduces outstanding improvements for VQA in video, and (ii) the performance on\nKnowIT VQA still lags well behind human accuracy, indicating its usefulness for\nstudying current video modelling limitations.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.08500v4",
            "date": "2020-04-18 00:57:54+00:00",
            "title": "A Formal Hierarchy of RNN Architectures",
            "authors": [
                "William Merrill",
                "Gail Weiss",
                "Yoav Goldberg",
                "Roy Schwartz",
                "Noah A. Smith",
                "Eran Yahav"
            ],
            "abstract": "We develop a formal hierarchy of the expressive capacity of RNN\narchitectures. The hierarchy is based on two formal properties: space\ncomplexity, which measures the RNN's memory, and rational recurrence, defined\nas whether the recurrent update can be described by a weighted finite-state\nmachine. We place several RNN variants within this hierarchy. For example, we\nprove the LSTM is not rational, which formally separates it from the related\nQRNN (Bradbury et al., 2016). We also show how these models' expressive\ncapacity is expanded by stacking multiple layers or composing them with\ndifferent pooling functions. Our results build on the theory of \"saturated\"\nRNNs (Merrill, 2019). While formally extending these findings to unsaturated\nRNNs is left to future work, we hypothesize that the practical learnable\ncapacity of unsaturated RNNs obeys a similar hierarchy. Experimental findings\nfrom training unsaturated networks on formal languages support this conjecture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08694v3",
            "date": "2020-04-18 19:57:39+00:00",
            "title": "Syn-QG: Syntactic and Shallow Semantic Rules for Question Generation",
            "authors": [
                "Kaustubh D. Dhole",
                "Christopher D. Manning"
            ],
            "abstract": "Question Generation (QG) is fundamentally a simple syntactic transformation;\nhowever, many aspects of semantics influence what questions are good to form.\nWe implement this observation by developing Syn-QG, a set of transparent\nsyntactic rules leveraging universal dependencies, shallow semantic parsing,\nlexical resources, and custom rules which transform declarative sentences into\nquestion-answer pairs. We utilize PropBank argument descriptions and VerbNet\nstate predicates to incorporate shallow semantic content, which helps generate\nquestions of a descriptive nature and produce inferential and semantically\nricher questions than existing systems. In order to improve syntactic fluency\nand eliminate grammatically incorrect questions, we employ back-translation\nover the output of these syntactic rules. A set of crowd-sourced evaluations\nshows that our system can generate a larger number of highly grammatical and\nrelevant questions than previous QG systems and that back-translation\ndrastically improves grammaticality at a slight cost of generating irrelevant\nquestions.",
            "journal ref": "Association for Computational Linguistics, 2020.acl-main.69",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.08744v1",
            "date": "2020-04-19 01:55:19+00:00",
            "title": "Are we pretraining it right? Digging deeper into visio-linguistic pretraining",
            "authors": [
                "Amanpreet Singh",
                "Vedanuj Goswami",
                "Devi Parikh"
            ],
            "abstract": "Numerous recent works have proposed pretraining generic visio-linguistic\nrepresentations and then finetuning them for downstream vision and language\ntasks. While architecture and objective function design choices have received\nattention, the choice of pretraining datasets has received little attention. In\nthis work, we question some of the default choices made in literature. For\ninstance, we systematically study how varying similarity between the\npretraining dataset domain (textual and visual) and the downstream domain\naffects performance. Surprisingly, we show that automatically generated data in\na domain closer to the downstream task (e.g., VQA v2) is a better choice for\npretraining than \"natural\" data but of a slightly different domain (e.g.,\nConceptual Captions). On the other hand, some seemingly reasonable choices of\npretraining datasets were found to be entirely ineffective for some downstream\ntasks. This suggests that despite the numerous recent efforts, vision &\nlanguage pretraining does not quite work \"out of the box\" yet. Overall, as a\nby-product of our study, we find that simple design choices in pretraining can\nhelp us achieve close to state-of-art results on downstream tasks without any\narchitectural changes.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.08814v1",
            "date": "2020-04-19 11:00:30+00:00",
            "title": "Graph-Structured Referring Expression Reasoning in The Wild",
            "authors": [
                "Sibei Yang",
                "Guanbin Li",
                "Yizhou Yu"
            ],
            "abstract": "Grounding referring expressions aims to locate in an image an object referred\nto by a natural language expression. The linguistic structure of a referring\nexpression provides a layout of reasoning over the visual contents, and it is\noften crucial to align and jointly understand the image and the referring\nexpression. In this paper, we propose a scene graph guided modular network\n(SGMN), which performs reasoning over a semantic graph and a scene graph with\nneural modules under the guidance of the linguistic structure of the\nexpression. In particular, we model the image as a structured semantic graph,\nand parse the expression into a language scene graph. The language scene graph\nnot only decodes the linguistic structure of the expression, but also has a\nconsistent representation with the image semantic graph. In addition to\nexploring structured solutions to grounding referring expressions, we also\npropose Ref-Reasoning, a large-scale real-world dataset for structured\nreferring expression reasoning. We automatically generate referring expressions\nover the scene graphs of images using diverse expression templates and\nfunctional programs. This dataset is equipped with real-world visual contents\nas well as semantically rich expressions with different reasoning layouts.\nExperimental results show that our SGMN not only significantly outperforms\nexisting state-of-the-art algorithms on the new Ref-Reasoning dataset, but also\nsurpasses state-of-the-art structured methods on commonly used benchmark\ndatasets. It can also provide interpretable visual evidences of reasoning. Data\nand code are available at https://github.com/sibeiyang/sgmn",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.09272v2",
            "date": "2020-04-20 13:26:45+00:00",
            "title": "A Revised Generative Evaluation of Visual Dialogue",
            "authors": [
                "Daniela Massiceti",
                "Viveka Kulharia",
                "Puneet K. Dokania",
                "N. Siddharth",
                "Philip H. S. Torr"
            ],
            "abstract": "Evaluating Visual Dialogue, the task of answering a sequence of questions\nrelating to a visual input, remains an open research challenge. The current\nevaluation scheme of the VisDial dataset computes the ranks of ground-truth\nanswers in predefined candidate sets, which Massiceti et al. (2018) show can be\nsusceptible to the exploitation of dataset biases. This scheme also does little\nto account for the different ways of expressing the same answer--an aspect of\nlanguage that has been well studied in NLP. We propose a revised evaluation\nscheme for the VisDial dataset leveraging metrics from the NLP literature to\nmeasure consensus between answers generated by the model and a set of relevant\nanswers. We construct these relevant answer sets using a simple and effective\nsemi-supervised method based on correlation, which allows us to automatically\nextend and scale sparse relevance annotations from humans to the entire\ndataset. We release these sets and code for the revised evaluation scheme as\nDenseVisDial, and intend them to be an improvement to the dataset in the face\nof its existing constraints and design choices.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.09297v2",
            "date": "2020-04-20 13:54:12+00:00",
            "title": "MPNet: Masked and Permuted Pre-training for Language Understanding",
            "authors": [
                "Kaitao Song",
                "Xu Tan",
                "Tao Qin",
                "Jianfeng Lu",
                "Tie-Yan Liu"
            ],
            "abstract": "BERT adopts masked language modeling (MLM) for pre-training and is one of the\nmost successful pre-training models. Since BERT neglects dependency among\npredicted tokens, XLNet introduces permuted language modeling (PLM) for\npre-training to address this problem. However, XLNet does not leverage the full\nposition information of a sentence and thus suffers from position discrepancy\nbetween pre-training and fine-tuning. In this paper, we propose MPNet, a novel\npre-training method that inherits the advantages of BERT and XLNet and avoids\ntheir limitations. MPNet leverages the dependency among predicted tokens\nthrough permuted language modeling (vs. MLM in BERT), and takes auxiliary\nposition information as input to make the model see a full sentence and thus\nreducing the position discrepancy (vs. PLM in XLNet). We pre-train MPNet on a\nlarge-scale dataset (over 160GB text corpora) and fine-tune on a variety of\ndown-streaming tasks (GLUE, SQuAD, etc). Experimental results show that MPNet\noutperforms MLM and PLM by a large margin, and achieves better results on these\ntasks compared with previous state-of-the-art pre-trained methods (e.g., BERT,\nXLNet, RoBERTa) under the same model setting. The code and the pre-trained\nmodels are available at: https://github.com/microsoft/MPNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09662v1",
            "date": "2020-04-20 22:08:08+00:00",
            "title": "The Panacea Threat Intelligence and Active Defense Platform",
            "authors": [
                "Adam Dalton",
                "Ehsan Aghaei",
                "Ehab Al-Shaer",
                "Archna Bhatia",
                "Esteban Castillo",
                "Zhuo Cheng",
                "Sreekar Dhaduvai",
                "Qi Duan",
                "Md Mazharul Islam",
                "Younes Karimi",
                "Amir Masoumzadeh",
                "Brodie Mather",
                "Sashank Santhanam",
                "Samira Shaikh",
                "Tomek Strzalkowski",
                "Bonnie J. Dorr"
            ],
            "abstract": "We describe Panacea, a system that supports natural language processing (NLP)\ncomponents for active defenses against social engineering attacks. We deploy a\npipeline of human language technology, including Ask and Framing Detection,\nNamed Entity Recognition, Dialogue Engineering, and Stylometry. Panacea\nprocesses modern message formats through a plug-in architecture to accommodate\ninnovative approaches for message analysis, knowledge representation and\ndialogue generation. The novelty of the Panacea system is that uses NLP for\ncyber defense and engages the attacker using bots to elicit evidence to\nattribute to the attacker and to waste the attacker's time and resources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09719v1",
            "date": "2020-04-21 02:43:31+00:00",
            "title": "Word Embedding-based Text Processing for Comprehensive Summarization and Distinct Information Extraction",
            "authors": [
                "Xiangpeng Wan",
                "Hakim Ghazzai",
                "Yehia Massoud"
            ],
            "abstract": "In this paper, we propose two automated text processing frameworks\nspecifically designed to analyze online reviews. The objective of the first\nframework is to summarize the reviews dataset by extracting essential sentence.\nThis is performed by converting sentences into numerical vectors and clustering\nthem using a community detection algorithm based on their similarity levels.\nAfterwards, a correlation score is measured for each sentence to determine its\nimportance level in each cluster and assign it as a tag for that community. The\nsecond framework is based on a question-answering neural network model trained\nto extract answers to multiple different questions. The collected answers are\neffectively clustered to find multiple distinct answers to a single question\nthat might be asked by a customer. The proposed frameworks are shown to be more\ncomprehensive than existing reviews processing solutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09739v2",
            "date": "2020-04-21 03:39:15+00:00",
            "title": "Neural Abstractive Summarization with Structural Attention",
            "authors": [
                "Tanya Chowdhury",
                "Sachin Kumar",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Attentional, RNN-based encoder-decoder architectures have achieved impressive\nperformance on abstractive summarization of news articles. However, these\nmethods fail to account for long term dependencies within the sentences of a\ndocument. This problem is exacerbated in multi-document summarization tasks\nsuch as summarizing the popular opinion in threads present in community\nquestion answering (CQA) websites such as Yahoo! Answers and Quora. These\nthreads contain answers which often overlap or contradict each other. In this\nwork, we present a hierarchical encoder based on structural attention to model\nsuch inter-sentence and inter-document dependencies. We set the popular\npointer-generator architecture and some of the architectures derived from it as\nour baselines and show that they fail to generate good summaries in a\nmulti-document setting. We further illustrate that our proposed model achieves\nsignificant improvement over the baselines in both single and multi-document\nsummarization settings -- in the former setting, it beats the best baseline by\n1.31 and 7.8 ROUGE-1 points on CNN and CQA datasets, respectively; in the\nlatter setting, the performance is further improved by 1.6 ROUGE-1 points on\nthe CQA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09853v3",
            "date": "2020-04-21 09:29:50+00:00",
            "title": "Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questions",
            "authors": [
                "Siyu Ren",
                "Kenny Q. Zhu"
            ],
            "abstract": "In this paper, we propose a novel configurable framework to automatically\ngenerate distractive choices for open-domain cloze-style multiple-choice\nquestions, which incorporates a general-purpose knowledge base to effectively\ncreate a small distractor candidate set, and a feature-rich learning-to-rank\nmodel to select distractors that are both plausible and reliable. Experimental\nresults on datasets across four domains show that our framework yields\ndistractors that are more plausible and reliable than previous methods. This\ndataset can also be used as a benchmark for distractor generation in the\nfuture.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09890v1",
            "date": "2020-04-21 10:37:44+00:00",
            "title": "Considering Likelihood in NLP Classification Explanations with Occlusion and Language Modeling",
            "authors": [
                "David Harbecke",
                "Christoph Alt"
            ],
            "abstract": "Recently, state-of-the-art NLP models gained an increasing syntactic and\nsemantic understanding of language, and explanation methods are crucial to\nunderstand their decisions. Occlusion is a well established method that\nprovides explanations on discrete language data, e.g. by removing a language\nunit from an input and measuring the impact on a model's decision. We argue\nthat current occlusion-based methods often produce invalid or syntactically\nincorrect language data, neglecting the improved abilities of recent NLP\nmodels. Furthermore, gradient-based explanation methods disregard the discrete\ndistribution of data in NLP. Thus, we propose OLM: a novel explanation method\nthat combines occlusion and language models to sample valid and syntactically\ncorrect replacements with high likelihood, given the context of the original\ninput. We lay out a theoretical foundation that alleviates these weaknesses of\nother explanation methods in NLP and provide results that underline the\nimportance of considering data likelihood in occlusion-based explanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.09974v1",
            "date": "2020-04-21 13:09:50+00:00",
            "title": "Learning to Encode Evolutionary Knowledge for Automatic Commenting Long Novels",
            "authors": [
                "Canxiang Yan",
                "Jianhao Yan",
                "Yangyin Xu",
                "Cheng Niu",
                "Jie Zhou"
            ],
            "abstract": "Static knowledge graph has been incorporated extensively into\nsequence-to-sequence framework for text generation. While effectively\nrepresenting structured context, static knowledge graph failed to represent\nknowledge evolution, which is required in modeling dynamic events. In this\npaper, an automatic commenting task is proposed for long novels, which involves\nunderstanding context of more than tens of thousands of words. To model the\ndynamic storyline, especially the transitions of the characters and their\nrelations, Evolutionary Knowledge Graph(EKG) is proposed and learned within a\nmulti-task framework. Given a specific passage to comment, sequential modeling\nis used to incorporate historical and future embedding for context\nrepresentation. Further, a graph-to-sequence model is designed to utilize the\nEKG for comment generation. Extensive experimental results show that our\nEKG-based method is superior to several strong baselines on both automatic and\nhuman evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10035v1",
            "date": "2020-04-21 14:13:33+00:00",
            "title": "Leveraging Cognitive Search Patterns to Enhance Automated Natural Language Retrieval Performance",
            "authors": [
                "Bhawani Selvaretnam",
                "Mohammed Belkhatir"
            ],
            "abstract": "The search of information in large text repositories has been plagued by the\nso-called document-query vocabulary gap, i.e. the semantic discordance between\nthe contents in the stored document entities on the one hand and the human\nquery on the other hand. Over the past two decades, a significant body of works\nhas advanced technical retrieval prowess while several studies have shed light\non issues pertaining to human search behavior. We believe that these efforts\nshould be conjoined, in the sense that automated retrieval systems have to\nfully emulate human search behavior and thus consider the procedure according\nto which users incrementally enhance their initial query. To this end,\ncognitive reformulation patterns that mimic user search behaviour are\nhighlighted and enhancement terms which are statistically collocated with or\nlexical-semantically related to the original terms adopted in the retrieval\nprocess. We formalize the application of these patterns by considering a query\nconceptual representation and introducing a set of operations allowing to\noperate modifications on the initial query. A genetic algorithm-based weighting\nprocess allows placing emphasis on terms according to their conceptual\nrole-type. An experimental evaluation on real-world datasets against relevance,\nlanguage, conceptual and knowledge-based models is conducted. We also show,\nwhen compared to language and relevance models, a better performance in terms\nof mean average precision than a word embedding-based model instantiation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.10037v2",
            "date": "2020-04-21 14:19:43+00:00",
            "title": "LineaRE: Simple but Powerful Knowledge Graph Embedding for Link Prediction",
            "authors": [
                "Yanhui Peng",
                "Jing Zhang"
            ],
            "abstract": "The task of link prediction for knowledge graphs is to predict missing\nrelationships between entities. Knowledge graph embedding, which aims to\nrepresent entities and relations of a knowledge graph as low dimensional\nvectors in a continuous vector space, has achieved promising predictive\nperformance. If an embedding model can cover different types of connectivity\npatterns and mapping properties of relations as many as possible, it will\npotentially bring more benefits for link prediction tasks. In this paper, we\npropose a novel embedding model, namely LineaRE, which is capable of modeling\nfour connectivity patterns (i.e., symmetry, antisymmetry, inversion, and\ncomposition) and four mapping properties (i.e., one-to-one, one-to-many,\nmany-to-one, and many-to-many) of relations. Specifically, we regard knowledge\ngraph embedding as a simple linear regression task, where a relation is modeled\nas a linear function of two low-dimensional vector-presented entities with two\nweight vectors and a bias vector. Since the vectors are defined in a real\nnumber space and the scoring function of the model is linear, our model is\nsimple and scalable to large knowledge graphs. Experimental results on multiple\nwidely used real-world datasets show that the proposed LineaRE model\nsignificantly outperforms existing state-of-the-art models for link prediction\ntasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.10270v2",
            "date": "2020-04-21 20:03:57+00:00",
            "title": "Learnings from Technological Interventions in a Low Resource Language: A Case-Study on Gondi",
            "authors": [
                "Devansh Mehta",
                "Sebastin Santy",
                "Ramaravind Kommiya Mothilal",
                "Brij Mohan Lal Srivastava",
                "Alok Sharma",
                "Anurag Shukla",
                "Vishnu Prasad",
                "Venkanna U",
                "Amit Sharma",
                "Kalika Bali"
            ],
            "abstract": "The primary obstacle to developing technologies for low-resource languages is\nthe lack of usable data. In this paper, we report the adoption and deployment\nof 4 technology-driven methods of data collection for Gondi, a low-resource\nvulnerable language spoken by around 2.3 million tribal people in south and\ncentral India. In the process of data collection, we also help in its revival\nby expanding access to information in Gondi through the creation of linguistic\nresources that can be used by the community, such as a dictionary, children's\nstories, an app with Gondi content from multiple sources and an Interactive\nVoice Response (IVR) based mass awareness platform. At the end of these\ninterventions, we collected a little less than 12,000 translated words and/or\nsentences and identified more than 650 community members whose help can be\nsolicited for future translation efforts. The larger goal of the project is\ncollecting enough data in Gondi to build and deploy viable language\ntechnologies like machine translation and speech to text systems that can help\ntake the language onto the internet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10349v1",
            "date": "2020-04-21 23:58:16+00:00",
            "title": "Textual Visual Semantic Dataset for Text Spotting",
            "authors": [
                "Ahmed Sabir",
                "Francesc Moreno-Noguer",
                "Llu\u00eds Padr\u00f3"
            ],
            "abstract": "Text Spotting in the wild consists of detecting and recognizing text\nappearing in images (e.g. signboards, traffic signals or brands in clothing or\nobjects). This is a challenging problem due to the complexity of the context\nwhere texts appear (uneven backgrounds, shading, occlusions, perspective\ndistortions, etc.). Only a few approaches try to exploit the relation between\ntext and its surrounding environment to better recognize text in the scene. In\nthis paper, we propose a visual context dataset for Text Spotting in the wild,\nwhere the publicly available dataset COCO-text [Veit et al. 2016] has been\nextended with information about the scene (such as objects and places appearing\nin the image) to enable researchers to include semantic relations between texts\nand scene in their Text Spotting systems, and to offer a common framework for\nsuch approaches. For each text in an image, we extract three kinds of context\ninformation: objects in the scene, image location label and a textual image\ndescription (caption). We use state-of-the-art out-of-the-box available tools\nto extract this additional information. Since this information has textual\nform, it can be used to leverage text similarity or semantic relation methods\ninto Text Spotting systems, either as a post-processing or in an end-to-end\ntraining strategy. Our data is publicly available at https://git.io/JeZTb.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.10361v2",
            "date": "2020-04-22 01:37:18+00:00",
            "title": "Testing Machine Translation via Referential Transparency",
            "authors": [
                "Pinjia He",
                "Clara Meister",
                "Zhendong Su"
            ],
            "abstract": "Machine translation software has seen rapid progress in recent years due to\nthe advancement of deep neural networks. People routinely use machine\ntranslation software in their daily lives, such as ordering food in a foreign\nrestaurant, receiving medical diagnosis and treatment from foreign doctors, and\nreading international political news online. However, due to the complexity and\nintractability of the underlying neural networks, modern machine translation\nsoftware is still far from robust and can produce poor or incorrect\ntranslations; this can lead to misunderstanding, financial loss, threats to\npersonal safety and health, and political conflicts. To address this problem,\nwe introduce referentially transparent inputs (RTIs), a simple, widely\napplicable methodology for validating machine translation software. A\nreferentially transparent input is a piece of text that should have similar\ntranslations when used in different contexts. Our practical implementation,\nPurity, detects when this property is broken by a translation. To evaluate RTI,\nwe use Purity to test Google Translate and Bing Microsoft Translator with 200\nunlabeled sentences, which detected 123 and 142 erroneous translations with\nhigh precision (79.3% and 78.3%). The translation errors are diverse, including\nexamples of under-translation, over-translation, word/phrase mistranslation,\nincorrect modification, and unclear logic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10404v2",
            "date": "2020-04-22 06:03:10+00:00",
            "title": "Logical Natural Language Generation from Open-Domain Tables",
            "authors": [
                "Wenhu Chen",
                "Jianshu Chen",
                "Yu Su",
                "Zhiyu Chen",
                "William Yang Wang"
            ],
            "abstract": "Neural natural language generation (NLG) models have recently shown\nremarkable progress in fluency and coherence. However, existing studies on\nneural NLG are primarily focused on surface-level realizations with limited\nemphasis on logical inference, an important aspect of human thinking and\nlanguage. In this paper, we suggest a new NLG task where a model is tasked with\ngenerating natural language statements that can be \\emph{logically entailed} by\nthe facts in an open-domain semi-structured table. To facilitate the study of\nthe proposed logical NLG problem, we use the existing TabFact dataset\n\\cite{chen2019tabfact} featured with a wide range of logical/symbolic\ninferences as our testbed, and propose new automatic metrics to evaluate the\nfidelity of generation models w.r.t.\\ logical inference. The new task poses\nchallenges to the existing monotonic generation frameworks due to the mismatch\nbetween sequence order and logical order. In our experiments, we\ncomprehensively survey different generation architectures (LSTM, Transformer,\nPre-Trained LM) trained with different algorithms (RL, Adversarial Training,\nCoarse-to-Fine) on the dataset and made following observations: 1) Pre-Trained\nLM can significantly boost both the fluency and logical fidelity metrics, 2) RL\nand Adversarial Training are trading fluency for fidelity, 3) Coarse-to-Fine\ngeneration can help partially alleviate the fidelity issue while maintaining\nhigh language fluency. The code and data are available at\n\\url{https://github.com/wenhuchen/LogicNLG}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10624v1",
            "date": "2020-04-22 15:04:52+00:00",
            "title": "Contextualised Graph Attention for Improved Relation Extraction",
            "authors": [
                "Angrosh Mandya",
                "Danushka Bollegala",
                "Frans Coenen"
            ],
            "abstract": "This paper presents a contextualized graph attention network that combines\nedge features and multiple sub-graphs for improving relation extraction. A\nnovel method is proposed to use multiple sub-graphs to learn rich node\nrepresentations in graph-based networks. To this end multiple sub-graphs are\nobtained from a single dependency tree. Two types of edge features are\nproposed, which are effectively combined with GAT and GCN models to apply for\nrelation extraction. The proposed model achieves state-of-the-art performance\non Semeval 2010 Task 8 dataset, achieving an F1-score of 86.3.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10645v2",
            "date": "2020-04-22 15:42:13+00:00",
            "title": "AmbigQA: Answering Ambiguous Open-domain Questions",
            "authors": [
                "Sewon Min",
                "Julian Michael",
                "Hannaneh Hajishirzi",
                "Luke Zettlemoyer"
            ],
            "abstract": "Ambiguity is inherent to open-domain question answering; especially when\nexploring new topics, it can be difficult to ask questions that have a single,\nunambiguous answer. In this paper, we introduce AmbigQA, a new open-domain\nquestion answering task which involves finding every plausible answer, and then\nrewriting the question for each one to resolve the ambiguity. To study this\ntask, we construct AmbigNQ, a dataset covering 14,042 questions from NQ-open,\nan existing open-domain QA benchmark. We find that over half of the questions\nin NQ-open are ambiguous, with diverse sources of ambiguity such as event and\nentity references. We also present strong baseline models for AmbigQA which we\nshow benefit from weakly supervised learning that incorporates NQ-open,\nstrongly suggesting our new task and data will support significant future\nresearch effort. Our data and baselines are available at\nhttps://nlp.cs.washington.edu/ambigqa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10706v4",
            "date": "2020-04-22 17:10:18+00:00",
            "title": "CORD-19: The COVID-19 Open Research Dataset",
            "authors": [
                "Lucy Lu Wang",
                "Kyle Lo",
                "Yoganand Chandrasekhar",
                "Russell Reas",
                "Jiangjiang Yang",
                "Doug Burdick",
                "Darrin Eide",
                "Kathryn Funk",
                "Yannis Katsis",
                "Rodney Kinney",
                "Yunyao Li",
                "Ziyang Liu",
                "William Merrill",
                "Paul Mooney",
                "Dewey Murdick",
                "Devvret Rishi",
                "Jerry Sheehan",
                "Zhihong Shen",
                "Brandon Stilson",
                "Alex Wade",
                "Kuansan Wang",
                "Nancy Xin Ru Wang",
                "Chris Wilhelm",
                "Boya Xie",
                "Douglas Raymond",
                "Daniel S. Weld",
                "Oren Etzioni",
                "Sebastian Kohlmeier"
            ],
            "abstract": "The COVID-19 Open Research Dataset (CORD-19) is a growing resource of\nscientific papers on COVID-19 and related historical coronavirus research.\nCORD-19 is designed to facilitate the development of text mining and\ninformation retrieval systems over its rich collection of metadata and\nstructured full text papers. Since its release, CORD-19 has been downloaded\nover 200K times and has served as the basis of many COVID-19 text mining and\ndiscovery systems. In this article, we describe the mechanics of dataset\nconstruction, highlighting challenges and key design decisions, provide an\noverview of how CORD-19 has been used, and describe several shared tasks built\naround the dataset. We hope this resource will continue to bring together the\ncomputing community, biomedical experts, and policy makers in the search for\neffective treatments and management policies for COVID-19.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2004.10796v3",
            "date": "2020-04-22 19:02:20+00:00",
            "title": "VisualCOMET: Reasoning about the Dynamic Context of a Still Image",
            "authors": [
                "Jae Sung Park",
                "Chandra Bhagavatula",
                "Roozbeh Mottaghi",
                "Ali Farhadi",
                "Yejin Choi"
            ],
            "abstract": "Even from a single frame of a still image, people can reason about the\ndynamic story of the image before, after, and beyond the frame. For example,\ngiven an image of a man struggling to stay afloat in water, we can reason that\nthe man fell into the water sometime in the past, the intent of that man at the\nmoment is to stay alive, and he will need help in the near future or else he\nwill get washed away. We propose VisualComet, the novel framework of visual\ncommonsense reasoning tasks to predict events that might have happened before,\nevents that might happen next, and the intents of the people at present. To\nsupport research toward visual commonsense reasoning, we introduce the first\nlarge-scale repository of Visual Commonsense Graphs that consists of over 1.4\nmillion textual descriptions of visual commonsense inferences carefully\nannotated over a diverse set of 60,000 images, each paired with short video\nsummaries of before and after. In addition, we provide person-grounding (i.e.,\nco-reference links) between people appearing in the image and people mentioned\nin the textual commonsense descriptions, allowing for tighter integration\nbetween images and text. We establish strong baseline performances on this task\nand demonstrate that integration between visual and textual commonsense\nreasoning is the key and wins over non-integrative alternatives.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.10919v1",
            "date": "2020-04-23 01:02:15+00:00",
            "title": "TCNN: Triple Convolutional Neural Network Models for Retrieval-based Question Answering System in E-commerce",
            "authors": [
                "Shuangyong Song",
                "Chao Wang"
            ],
            "abstract": "Automatic question-answering (QA) systems have boomed during last few years,\nand commonly used techniques can be roughly categorized into Information\nRetrieval (IR)-based and generation-based. A key solution to the IR based\nmodels is to retrieve the most similar knowledge entries of a given query from\na QA knowledge base, and then rerank those knowledge entries with semantic\nmatching models. In this paper, we aim to improve an IR based e-commerce QA\nsystem-AliMe with proposed text matching models, including a basic Triple\nConvolutional Neural Network (TCNN) model and two Attention-based TCNN (ATCNN)\nmodels. Experimental results show their effect.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2004.10964v3",
            "date": "2020-04-23 04:21:19+00:00",
            "title": "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks",
            "authors": [
                "Suchin Gururangan",
                "Ana Marasovi\u0107",
                "Swabha Swayamdipta",
                "Kyle Lo",
                "Iz Beltagy",
                "Doug Downey",
                "Noah A. Smith"
            ],
            "abstract": "Language models pretrained on text from a wide variety of sources form the\nfoundation of today's NLP. In light of the success of these broad-coverage\nmodels, we investigate whether it is still helpful to tailor a pretrained model\nto the domain of a target task. We present a study across four domains\n(biomedical and computer science publications, news, and reviews) and eight\nclassification tasks, showing that a second phase of pretraining in-domain\n(domain-adaptive pretraining) leads to performance gains, under both high- and\nlow-resource settings. Moreover, adapting to the task's unlabeled data\n(task-adaptive pretraining) improves performance even after domain-adaptive\npretraining. Finally, we show that adapting to a task corpus augmented using\nsimple data selection strategies is an effective alternative, especially when\nresources for domain-adaptive pretraining might be unavailable. Overall, we\nconsistently find that multi-phase adaptive pretraining offers large gains in\ntask performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11045v1",
            "date": "2020-04-23 09:41:37+00:00",
            "title": "Distilling Knowledge for Fast Retrieval-based Chat-bots",
            "authors": [
                "Amir Vakili Tahami",
                "Kamyar Ghajar",
                "Azadeh Shakery"
            ],
            "abstract": "Response retrieval is a subset of neural ranking in which a model selects a\nsuitable response from a set of candidates given a conversation history.\nRetrieval-based chat-bots are typically employed in information seeking\nconversational systems such as customer support agents. In order to make\npairwise comparisons between a conversation history and a candidate response,\ntwo approaches are common: cross-encoders performing full self-attention over\nthe pair and bi-encoders encoding the pair separately. The former gives better\nprediction quality but is too slow for practical use. In this paper, we propose\na new cross-encoder architecture and transfer knowledge from this model to a\nbi-encoder model using distillation. This effectively boosts bi-encoder\nperformance at no cost during inference time. We perform a detailed analysis of\nthis approach on three response retrieval datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.11081v1",
            "date": "2020-04-23 11:21:30+00:00",
            "title": "Coupling semantic and statistical techniques for dynamically enriching web ontologies",
            "authors": [
                "Mohammed Maree",
                "Mohammed Belkhatir"
            ],
            "abstract": "With the development of the Semantic Web technology, the use of ontologies to\nstore and retrieve information covering several domains has increased. However,\nvery few ontologies are able to cope with the ever-growing need of frequently\nupdated semantic information or specific user requirements in specialized\ndomains. As a result, a critical issue is related to the unavailability of\nrelational information between concepts, also coined missing background\nknowledge. One solution to address this issue relies on the manual enrichment\nof ontologies by domain experts which is however a time consuming and costly\nprocess, hence the need for dynamic ontology enrichment. In this paper we\npresent an automatic coupled statistical/semantic framework for dynamically\nenriching large-scale generic ontologies from the World Wide Web. Using the\nmassive amount of information encoded in texts on the Web as a corpus, missing\nbackground knowledge can therefore be discovered through a combination of\nsemantic relatedness measures and pattern acquisition techniques and\nsubsequently exploited. The benefits of our approach are: (i) proposing the\ndynamic enrichment of large-scale generic ontologies with missing background\nknowledge, and thus, enabling the reuse of such knowledge, (ii) dealing with\nthe issue of costly ontological manual enrichment by domain experts.\nExperimental results in a precision-based evaluation setting demonstrate the\neffectiveness of the proposed techniques.",
            "journal ref": "Journal Intelligent Information Systems 2013",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11083v1",
            "date": "2020-04-23 11:22:38+00:00",
            "title": "Coupled intrinsic and extrinsic human language resource-based query expansion",
            "authors": [
                "Bhawani Selvaretnam",
                "Mohammed Belkhatir"
            ],
            "abstract": "Poor information retrieval performance has often been attributed to the\nquery-document vocabulary mismatch problem which is defined as the difficulty\nfor human users to formulate precise natural language queries that are in line\nwith the vocabulary of the documents deemed relevant to a specific search goal.\nTo alleviate this problem, query expansion processes are applied in order to\nspawn and integrate additional terms to an initial query. This requires\naccurate identification of main query concepts to ensure the intended search\ngoal is duly emphasized and relevant expansion concepts are extracted and\nincluded in the enriched query. Natural language queries have intrinsic\nlinguistic properties such as parts-of-speech labels and grammatical relations\nwhich can be utilized in determining the intended search goal. Additionally,\nextrinsic language-based resources such as ontologies are needed to suggest\nexpansion concepts semantically coherent with the query content. We present\nhere a query expansion framework which capitalizes on both linguistic\ncharacteristics of user queries and ontology resources for query constituent\nencoding, expansion concept extraction and concept weighting. A thorough\nempirical evaluation on real-world datasets validates our approach against\nunigram language model, relevance model and a sequential dependence based\ntechnique.",
            "journal ref": "Knowledge & Information Systems 2018",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11093v1",
            "date": "2020-04-23 11:39:07+00:00",
            "title": "Natural language technology and query expansion: issues, state-of-the-art and perspectives",
            "authors": [
                "Bhawani Selvaretnam",
                "Mohammed Belkhatir"
            ],
            "abstract": "The availability of an abundance of knowledge sources has spurred a large\namount of effort in the development and enhancement of Information Retrieval\ntechniques. Users information needs are expressed in natural language and\nsuccessful retrieval is very much dependent on the effective communication of\nthe intended purpose. Natural language queries consist of multiple linguistic\nfeatures which serve to represent the intended search goal. Linguistic\ncharacteristics that cause semantic ambiguity and misinterpretation of queries\nas well as additional factors such as the lack of familiarity with the search\nenvironment affect the users ability to accurately represent their information\nneeds, coined by the concept intention gap. The latter directly affects the\nrelevance of the returned search results which may not be to the users\nsatisfaction and therefore is a major issue impacting the effectiveness of\ninformation retrieval systems. Central to our discussion is the identification\nof the significant constituents that characterize the query intent and their\nenrichment through the addition of meaningful terms, phrases or even latent\nrepresentations, either manually or automatically to capture their intended\nmeaning. Specifically, we discuss techniques to achieve the enrichment and in\nparticular those utilizing the information gathered from statistical processing\nof term dependencies within a document corpus or from external knowledge\nsources such as ontologies. We lay down the anatomy of a generic linguistic\nbased query expansion framework and propose its module-based decomposition,\ncovering topical issues from query processing, information retrieval,\ncomputational linguistics and ontology engineering. For each of the modules we\nreview state-of-the-art solutions in the literature categorized and analyzed\nunder the light of the techniques used.",
            "journal ref": "J Intell Inf Syst 2012",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.11157v1",
            "date": "2020-04-23 13:46:11+00:00",
            "title": "On Adversarial Examples for Biomedical NLP Tasks",
            "authors": [
                "Vladimir Araujo",
                "Andres Carvallo",
                "Carlos Aspillaga",
                "Denis Parra"
            ],
            "abstract": "The success of pre-trained word embeddings has motivated its use in tasks in\nthe biomedical domain. The BERT language model has shown remarkable results on\nstandard performance metrics in tasks such as Named Entity Recognition (NER)\nand Semantic Textual Similarity (STS), which has brought significant progress\nin the field of NLP. However, it is unclear whether these systems work\nseemingly well in critical domains, such as legal or medical. For that reason,\nin this work, we propose an adversarial evaluation scheme on two well-known\ndatasets for medical NER and STS. We propose two types of attacks inspired by\nnatural spelling errors and typos made by humans. We also propose another type\nof attack that uses synonyms of medical terms. Under these adversarial\nsettings, the accuracy of the models drops significantly, and we quantify the\nextent of this performance loss. We also show that we can significantly improve\nthe robustness of the models by training them with adversarial examples. We\nhope our work will motivate the use of adversarial examples to evaluate and\ndevelop models with increased robustness for medical tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11163v1",
            "date": "2020-04-23 13:54:31+00:00",
            "title": "Same Side Stance Classification Task: Facilitating Argument Stance Classification by Fine-tuning a BERT Model",
            "authors": [
                "Stefan Ollinger",
                "Lorik Dumani",
                "Premtim Sahitaj",
                "Ralph Bergmann",
                "Ralf Schenkel"
            ],
            "abstract": "Research on computational argumentation is currently being intensively\ninvestigated. The goal of this community is to find the best pro and con\narguments for a user given topic either to form an opinion for oneself, or to\npersuade others to adopt a certain standpoint. While existing argument mining\nmethods can find appropriate arguments for a topic, a correct classification\ninto pro and con is not yet reliable. The same side stance classification task\nprovides a dataset of argument pairs classified by whether or not both\narguments share the same stance and does not need to distinguish between\ntopic-specific pro and con vocabulary but only the argument similarity within a\nstance needs to be assessed. The results of our contribution to the task are\nbuild on a setup based on the BERT architecture. We fine-tuned a pre-trained\nBERT model for three epochs and used the first 512 tokens of each argument to\npredict if two arguments share the same stance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11327v1",
            "date": "2020-04-23 17:22:38+00:00",
            "title": "Adaptive Forgetting Curves for Spaced Repetition Language Learning",
            "authors": [
                "Ahmed Zaidi",
                "Andrew Caines",
                "Russell Moore",
                "Paula Buttery",
                "Andrew Rice"
            ],
            "abstract": "The forgetting curve has been extensively explored by psychologists,\neducationalists and cognitive scientists alike. In the context of Intelligent\nTutoring Systems, modelling the forgetting curve for each user and knowledge\ncomponent (e.g. vocabulary word) should enable us to develop optimal revision\nstrategies that counteract memory decay and ensure long-term retention. In this\nstudy we explore a variety of forgetting curve models incorporating\npsychological and linguistic features, and we use these models to predict the\nprobability of word recall by learners of English as a second language. We\nevaluate the impact of the models and their features using data from an online\nvocabulary teaching platform and find that word complexity is a highly\ninformative feature which may be successfully learned by a neural network\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11480v1",
            "date": "2020-04-23 22:25:48+00:00",
            "title": "Characterising User Content on a Multi-lingual Social Network",
            "authors": [
                "Pushkal Agarwal",
                "Kiran Garimella",
                "Sagar Joglekar",
                "Nishanth Sastry",
                "Gareth Tyson"
            ],
            "abstract": "Social media has been on the vanguard of political information diffusion in\nthe 21st century. Most studies that look into disinformation, political\ninfluence and fake-news focus on mainstream social media platforms. This has\ninevitably made English an important factor in our current understanding of\npolitical activity on social media. As a result, there has only been a limited\nnumber of studies into a large portion of the world, including the largest,\nmultilingual and multi-cultural democracy: India. In this paper we present our\ncharacterisation of a multilingual social network in India called ShareChat. We\ncollect an exhaustive dataset across 72 weeks before and during the Indian\ngeneral elections of 2019, across 14 languages. We investigate the cross\nlingual dynamics by clustering visually similar images together, and exploring\nhow they move across language barriers. We find that Telugu, Malayalam, Tamil\nand Kannada languages tend to be dominant in soliciting political images (often\nreferred to as memes), and posts from Hindi have the largest cross-lingual\ndiffusion across ShareChat (as well as images containing text in English). In\nthe case of images containing text that cross language barriers, we see that\nlanguage translation is used to widen the accessibility. That said, we find\ncases where the same image is associated with very different text (and\ntherefore meanings). This initial characterisation paves the way for more\nadvanced pipelines to understand the dynamics of fake and political content in\na multi-lingual and non-textual setting.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2004.11695v1",
            "date": "2020-04-24 16:29:13+00:00",
            "title": "Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions: NLP Using LSTM Recurrent Neural Network Approach",
            "authors": [
                "Hamed Jelodar",
                "Yongli Wang",
                "Rita Orji",
                "Hucheng Huang"
            ],
            "abstract": "Internet forums and public social media, such as online healthcare forums,\nprovide a convenient channel for users (people/patients) concerned about health\nissues to discuss and share information with each other. In late December 2019,\nan outbreak of a novel coronavirus (infection from which results in the disease\nnamed COVID-19) was reported, and, due to the rapid spread of the virus in\nother parts of the world, the World Health Organization declared a state of\nemergency. In this paper, we used automated extraction of COVID-19 related\ndiscussions from social media and a natural language process (NLP) method based\non topic modeling to uncover various issues related to COVID-19 from public\nopinions. Moreover, we also investigate how to use LSTM recurrent neural\nnetwork for sentiment classification of COVID-19 comments. Our findings shed\nlight on the importance of using public opinions and suitable computational\ntechniques to understand issues surrounding COVID-19 and to guide related\ndecision-making.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.11714v1",
            "date": "2020-04-22 23:19:55+00:00",
            "title": "Residual Energy-Based Models for Text Generation",
            "authors": [
                "Yuntian Deng",
                "Anton Bakhtin",
                "Myle Ott",
                "Arthur Szlam",
                "Marc'Aurelio Ranzato"
            ],
            "abstract": "Text generation is ubiquitous in many NLP tasks, from summarization, to\ndialogue and machine translation. The dominant parametric approach is based on\nlocally normalized models which predict one word at a time. While these work\nremarkably well, they are plagued by exposure bias due to the greedy nature of\nthe generation process. In this work, we investigate un-normalized energy-based\nmodels (EBMs) which operate not at the token but at the sequence level. In\norder to make training tractable, we first work in the residual of a pretrained\nlocally normalized language model and second we train using noise contrastive\nestimation. Furthermore, since the EBM works at the sequence level, we can\nleverage pretrained bi-directional contextual representations, such as BERT and\nRoBERTa. Our experiments on two large language modeling datasets show that\nresidual EBMs yield lower perplexity compared to locally normalized baselines.\nMoreover, generation via importance sampling is very efficient and of higher\nquality than the baseline models according to human evaluation.",
            "journal ref": "ICLR 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11861v2",
            "date": "2020-04-24 17:11:37+00:00",
            "title": "Event-QA: A Dataset for Event-Centric Question Answering over Knowledge Graphs",
            "authors": [
                "Tarc\u00edsio Souza Costa",
                "Simon Gottschalk",
                "Elena Demidova"
            ],
            "abstract": "Semantic Question Answering (QA) is a crucial technology to facilitate\nintuitive user access to semantic information stored in knowledge graphs.\nWhereas most of the existing QA systems and datasets focus on entity-centric\nquestions, very little is known about these systems' performance in the context\nof events. As new event-centric knowledge graphs emerge, datasets for such\nquestions gain importance. In this paper, we present the Event-QA dataset for\nanswering event-centric questions over knowledge graphs. Event-QA contains 1000\nsemantic queries and the corresponding English, German and Portuguese\nverbalizations for EventKG - an event-centric knowledge graph with more than\n970 thousand events.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.11980v4",
            "date": "2020-04-24 20:35:11+00:00",
            "title": "Question Answering over Curated and Open Web Sources",
            "authors": [
                "Rishiraj Saha Roy",
                "Avishek Anand"
            ],
            "abstract": "The last few years have seen an explosion of research on the topic of\nautomated question answering (QA), spanning the communities of information\nretrieval, natural language processing, and artificial intelligence. This\ntutorial would cover the highlights of this really active period of growth for\nQA to give the audience a grasp over the families of algorithms that are\ncurrently being used. We partition research contributions by the underlying\nsource from where answers are retrieved: curated knowledge graphs, unstructured\ntext, or hybrid corpora. We choose this dimension of partitioning as it is the\nmost discriminative when it comes to algorithm design. Other key dimensions are\ncovered within each sub-topic: like the complexity of questions addressed, and\ndegrees of explainability and interactivity introduced in the systems. We would\nconclude the tutorial with the most promising emerging trends in the expanse of\nQA, that would help new entrants into this field make the best decisions to\ntake the community forward. Much has changed in the community since the last\ntutorial on QA in SIGIR 2016, and we believe that this timely overview will\nindeed benefit a large number of conference participants.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.12073v3",
            "date": "2020-04-25 07:38:08+00:00",
            "title": "All Word Embeddings from One Embedding",
            "authors": [
                "Sho Takase",
                "Sosuke Kobayashi"
            ],
            "abstract": "In neural network-based models for natural language processing (NLP), the\nlargest part of the parameters often consists of word embeddings. Conventional\nmodels prepare a large embedding matrix whose size depends on the vocabulary\nsize. Therefore, storing these models in memory and disk storage is costly. In\nthis study, to reduce the total number of parameters, the embeddings for all\nwords are represented by transforming a shared embedding. The proposed method,\nALONE (all word embeddings from one), constructs the embedding of a word by\nmodifying the shared embedding with a filter vector, which is word-specific but\nnon-trainable. Then, we input the constructed embedding into a feed-forward\nneural network to increase its expressiveness. Naively, the filter vectors\noccupy the same memory size as the conventional embedding matrix, which depends\non the vocabulary size. To solve this issue, we also introduce a\nmemory-efficient filter construction approach. We indicate our ALONE can be\nused as word representation sufficiently through an experiment on the\nreconstruction of pre-trained word embeddings. In addition, we also conduct\nexperiments on NLP application tasks: machine translation and summarization. We\ncombined ALONE with the current state-of-the-art encoder-decoder model, the\nTransformer, and achieved comparable scores on WMT 2014 English-to-German\ntranslation and DUC 2004 very short summarization with less parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12184v1",
            "date": "2020-04-25 16:37:26+00:00",
            "title": "A Named Entity Based Approach to Model Recipes",
            "authors": [
                "Nirav Diwan",
                "Devansh Batra",
                "Ganesh Bagler"
            ],
            "abstract": "Traditional cooking recipes follow a structure which can be modelled very\nwell if the rules and semantics of the different sections of the recipe text\nare analyzed and represented accurately. We propose a structure that can\naccurately represent the recipe as well as a pipeline to infer the best\nrepresentation of the recipe in this uniform structure. The Ingredients section\nin a recipe typically lists down the ingredients required and corresponding\nattributes such as quantity, temperature, and processing state. This can be\nmodelled by defining these attributes and their values. The physical entities\nwhich make up a recipe can be broadly classified into utensils, ingredients and\ntheir combinations that are related by cooking techniques. The instruction\nsection lists down a series of events in which a cooking technique or process\nis applied upon these utensils and ingredients. We model these relationships in\nthe form of tuples. Thus, using a combination of these methods we model cooking\nrecipe in the dataset RecipeDB to show the efficacy of our method. This mined\ninformation model can have several applications which include translating\nrecipes between languages, determining similarity between recipes, generation\nof novel recipes and estimation of the nutritional profile of recipes. For the\npurpose of recognition of ingredient attributes, we train the Named Entity\nRelationship (NER) models and analyze the inferences with the help of K-Means\nclustering. Our model presented with an F1 score of 0.95 across all datasets.\nWe use a similar NER tagging model for labelling cooking techniques (F1 score =\n0.88) and utensils (F1 score = 0.90) within the instructions section. Finally,\nwe determine the temporal sequence of relationships between ingredients,\nutensils and cooking techniques for modeling the instruction steps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12239v1",
            "date": "2020-04-25 21:37:36+00:00",
            "title": "MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification",
            "authors": [
                "Jiaao Chen",
                "Zichao Yang",
                "Diyi Yang"
            ],
            "abstract": "This paper presents MixText, a semi-supervised learning method for text\nclassification, which uses our newly designed data augmentation method called\nTMix. TMix creates a large amount of augmented training samples by\ninterpolating text in hidden space. Moreover, we leverage recent advances in\ndata augmentation to guess low-entropy labels for unlabeled data, hence making\nthem as easy to use as labeled data.By mixing labeled, unlabeled and augmented\ndata, MixText significantly outperformed current pre-trained and fined-tuned\nmodels and other state-of-the-art semi-supervised learning methods on several\ntext classification benchmarks. The improvement is especially prominent when\nsupervision is extremely limited. We have publicly released our code at\nhttps://github.com/GT-SALT/MixText.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12297v2",
            "date": "2020-04-26 07:04:08+00:00",
            "title": "Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching",
            "authors": [
                "Liu Yang",
                "Mingyang Zhang",
                "Cheng Li",
                "Michael Bendersky",
                "Marc Najork"
            ],
            "abstract": "Many natural language processing and information retrieval problems can be\nformalized as the task of semantic matching. Existing work in this area has\nbeen largely focused on matching between short texts (e.g., question\nanswering), or between a short and a long text (e.g., ad-hoc retrieval).\nSemantic matching between long-form documents, which has many important\napplications like news recommendation, related article recommendation and\ndocument clustering, is relatively less explored and needs more research\neffort. In recent years, self-attention based models like Transformers and BERT\nhave achieved state-of-the-art performance in the task of text matching. These\nmodels, however, are still limited to short text like a few sentences or one\nparagraph due to the quadratic computational complexity of self-attention with\nrespect to input text length. In this paper, we address the issue by proposing\nthe Siamese Multi-depth Transformer-based Hierarchical (SMITH) Encoder for\nlong-form document matching. Our model contains several innovations to adapt\nself-attention models for longer text input. In order to better capture\nsentence level semantic relations within a document, we pre-train the model\nwith a novel masked sentence block language modeling task in addition to the\nmasked word language modeling task used by BERT. Our experimental results on\nseveral benchmark datasets for long-form document matching show that our\nproposed SMITH model outperforms the previous state-of-the-art models including\nhierarchical attention, multi-depth attention-based hierarchical recurrent\nneural network, and BERT. Comparing to BERT based baselines, our model is able\nto increase maximum input text length from 512 to 2048. We will open source a\nWikipedia based benchmark dataset, code and a pre-trained checkpoint to\naccelerate future research on long-form document matching.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.12299v4",
            "date": "2020-04-26 07:17:48+00:00",
            "title": "Dual Learning for Semi-Supervised Natural Language Understanding",
            "authors": [
                "Su Zhu",
                "Ruisheng Cao",
                "Kai Yu"
            ],
            "abstract": "Natural language understanding (NLU) converts sentences into structured\nsemantic forms. The paucity of annotated training samples is still a\nfundamental challenge of NLU. To solve this data sparsity problem, previous\nwork based on semi-supervised learning mainly focuses on exploiting unlabeled\nsentences. In this work, we introduce a dual task of NLU, semantic-to-sentence\ngeneration (SSG), and propose a new framework for semi-supervised NLU with the\ncorresponding dual model. The framework is composed of dual pseudo-labeling and\ndual learning method, which enables an NLU model to make full use of data\n(labeled and unlabeled) through a closed-loop of the primal and dual tasks. By\nincorporating the dual task, the framework can exploit pure semantic forms as\nwell as unlabeled sentences, and further improve the NLU and SSG models\niteratively in the closed-loop. The proposed approaches are evaluated on two\npublic datasets (ATIS and SNIPS). Experiments in the semi-supervised setting\nshow that our methods can outperform various baselines significantly, and\nextensive ablation studies are conducted to verify the effectiveness of our\nframework. Finally, our method can also achieve the state-of-the-art\nperformance on the two datasets in the supervised setting. Our code is\navailable at \\url{https://github.com/rhythmcao/slu-dual-learning.git}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12303v1",
            "date": "2020-04-26 07:43:30+00:00",
            "title": "Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System",
            "authors": [
                "Xinyue Zheng",
                "Peng Wang",
                "Qigang Wang",
                "Zhongchao Shi"
            ],
            "abstract": "Prior work in standardized science exams requires support from large text\ncorpus, such as targeted science corpus fromWikipedia or SimpleWikipedia.\nHowever, retrieving knowledge from the large corpus is time-consuming and\nquestions embedded in complex semantic representation may interfere with\nretrieval. Inspired by the dual process theory in cognitive science, we propose\na MetaQA framework, where system 1 is an intuitive meta-classifier and system 2\nis a reasoning module. Specifically, our method based on meta-learning method\nand large language model BERT, which can efficiently solve science problems by\nlearning from related example questions without relying on external knowledge\nbases. We evaluate our method on AI2 Reasoning Challenge (ARC), and the\nexperimental results show that meta-classifier yields considerable\nclassification performance on emerging question types. The information provided\nby meta-classifier significantly improves the accuracy of reasoning module from\n46.6% to 64.2%, which has a competitive advantage over retrieval-based QA\nmethods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.12330v1",
            "date": "2020-04-26 09:34:32+00:00",
            "title": "Detecting fake news for the new coronavirus by reasoning on the Covid-19 ontology",
            "authors": [
                "Adrian Groza"
            ],
            "abstract": "In the context of the Covid-19 pandemic, many were quick to spread deceptive\ninformation. I investigate here how reasoning in Description Logics (DLs) can\ndetect inconsistencies between trusted medical sources and not trusted ones.\nThe not-trusted information comes in natural language (e.g. \"Covid-19 affects\nonly the elderly\"). To automatically convert into DLs, I used the FRED\nconverter. Reasoning in Description Logics is then performed with the Racer\ntool.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.12332v1",
            "date": "2020-04-26 09:45:45+00:00",
            "title": "Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds",
            "authors": [
                "Kawin Ethayarajh"
            ],
            "abstract": "Most NLP datasets are not annotated with protected attributes such as gender,\nmaking it difficult to measure classification bias using standard measures of\nfairness (e.g., equal opportunity). However, manually annotating a large\ndataset with a protected attribute is slow and expensive. Instead of annotating\nall the examples, can we annotate a subset of them and use that sample to\nestimate the bias? While it is possible to do so, the smaller this annotated\nsample is, the less certain we are that the estimate is close to the true bias.\nIn this work, we propose using Bernstein bounds to represent this uncertainty\nabout the bias estimate as a confidence interval. We provide empirical evidence\nthat a 95% confidence interval derived this way consistently bounds the true\nbias. In quantifying this uncertainty, our method, which we call\nBernstein-bounded unfairness, helps prevent classifiers from being deemed\nbiased or unbiased when there is insufficient evidence to make either claim.\nOur findings suggest that the datasets currently used to measure specific\nbiases are too small to conclusively identify bias except in the most egregious\ncases. For example, consider a co-reference resolution system that is 5% more\naccurate on gender-stereotypical sentences -- to claim it is biased with 95%\nconfidence, we need a bias-specific dataset that is 3.8 times larger than\nWinoBias, the largest available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12362v1",
            "date": "2020-04-26 12:21:04+00:00",
            "title": "Relational Graph Attention Network for Aspect-based Sentiment Analysis",
            "authors": [
                "Kai Wang",
                "Weizhou Shen",
                "Yunyi Yang",
                "Xiaojun Quan",
                "Rui Wang"
            ],
            "abstract": "Aspect-based sentiment analysis aims to determine the sentiment polarity\ntowards a specific aspect in online reviews. Most recent efforts adopt\nattention-based neural network models to implicitly connect aspects with\nopinion words. However, due to the complexity of language and the existence of\nmultiple aspects in a single sentence, these models often confuse the\nconnections. In this paper, we address this problem by means of effective\nencoding of syntax information. Firstly, we define a unified aspect-oriented\ndependency tree structure rooted at a target aspect by reshaping and pruning an\nordinary dependency parse tree. Then, we propose a relational graph attention\nnetwork (R-GAT) to encode the new tree structure for sentiment prediction.\nExtensive experiments are conducted on the SemEval 2014 and Twitter datasets,\nand the experimental results confirm that the connections between aspects and\nopinion words can be better established with our approach, and the performance\nof the graph attention network (GAT) is significantly improved as a\nconsequence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12363v1",
            "date": "2020-04-26 12:21:17+00:00",
            "title": "Multi-Domain Dialogue Acts and Response Co-Generation",
            "authors": [
                "Kai Wang",
                "Junfeng Tian",
                "Rui Wang",
                "Xiaojun Quan",
                "Jianxing Yu"
            ],
            "abstract": "Generating fluent and informative responses is of critical importance for\ntask-oriented dialogue systems. Existing pipeline approaches generally predict\nmultiple dialogue acts first and use them to assist response generation. There\nare at least two shortcomings with such approaches. First, the inherent\nstructures of multi-domain dialogue acts are neglected. Second, the semantic\nassociations between acts and responses are not taken into account for response\ngeneration. To address these issues, we propose a neural co-generation model\nthat generates dialogue acts and responses concurrently. Unlike those pipeline\napproaches, our act generation module preserves the semantic structures of\nmulti-domain dialogue acts and our response generation module dynamically\nattends to different acts as needed. We train the two modules jointly using an\nuncertainty loss to adjust their task weights adaptively. Extensive experiments\nare conducted on the large-scale MultiWOZ dataset and the results show that our\nmodel achieves very favorable improvement over several state-of-the-art models\nin both automatic and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12585v2",
            "date": "2020-04-27 05:20:01+00:00",
            "title": "A Batch Normalized Inference Network Keeps the KL Vanishing Away",
            "authors": [
                "Qile Zhu",
                "Jianlin Su",
                "Wei Bi",
                "Xiaojiang Liu",
                "Xiyao Ma",
                "Xiaolin Li",
                "Dapeng Wu"
            ],
            "abstract": "Variational Autoencoder (VAE) is widely used as a generative model to\napproximate a model's posterior on latent variables by combining the amortized\nvariational inference and deep neural networks. However, when paired with\nstrong autoregressive decoders, VAE often converges to a degenerated local\noptimum known as \"posterior collapse\". Previous approaches consider the\nKullback Leibler divergence (KL) individual for each datapoint. We propose to\nlet the KL follow a distribution across the whole dataset, and analyze that it\nis sufficient to prevent posterior collapse by keeping the expectation of the\nKL's distribution positive. Then we propose Batch Normalized-VAE (BN-VAE), a\nsimple but effective approach to set a lower bound of the expectation by\nregularizing the distribution of the approximate posterior's parameters.\nWithout introducing any new model component or modifying the objective, our\napproach can avoid the posterior collapse effectively and efficiently. We\nfurther show that the proposed BN-VAE can be extended to conditional VAE\n(CVAE). Empirically, our approach surpasses strong autoregressive baselines on\nlanguage modeling, text classification and dialogue generation, and rivals more\ncomplex approaches while keeping almost the same training time as VAE.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2004.12726v3",
            "date": "2020-04-27 11:51:28+00:00",
            "title": "BLEU Neighbors: A Reference-less Approach to Automatic Evaluation",
            "authors": [
                "Kawin Ethayarajh",
                "Dorsa Sadigh"
            ],
            "abstract": "Evaluation is a bottleneck in the development of natural language generation\n(NLG) models. Automatic metrics such as BLEU rely on references, but for tasks\nsuch as open-ended generation, there are no references to draw upon. Although\nlanguage diversity can be estimated using statistical measures such as\nperplexity, measuring language quality requires human evaluation. However,\nbecause human evaluation at scale is slow and expensive, it is used sparingly;\nit cannot be used to rapidly iterate on NLG models, in the way BLEU is used for\nmachine translation. To this end, we propose BLEU Neighbors, a nearest\nneighbors model for estimating language quality by using the BLEU score as a\nkernel function. On existing datasets for chitchat dialogue and open-ended\nsentence generation, we find that -- on average -- the quality estimation from\na BLEU Neighbors model has a lower mean squared error and higher Spearman\ncorrelation with the ground truth than individual human annotators. Despite its\nsimplicity, BLEU Neighbors even outperforms state-of-the-art models on\nautomatically grading essays, including models that have access to a\ngold-standard reference essay.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12765v5",
            "date": "2020-04-27 13:10:11+00:00",
            "title": "ColBERT: Using BERT Sentence Embedding for Humor Detection",
            "authors": [
                "Issa Annamoradnejad",
                "Gohar Zoghi"
            ],
            "abstract": "Automatic humor detection has interesting use cases in modern technologies,\nsuch as chatbots and virtual assistants. In this paper, we propose a novel\napproach for detecting humor in short texts based on the general linguistic\nstructure of humor. Our proposed method uses BERT to generate embeddings for\nsentences of a given text and uses these embeddings as inputs of parallel lines\nof hidden layers in a neural network. These lines are finally concatenated to\npredict the target value. For evaluation purposes, we created a new dataset for\nhumor detection consisting of 200k formal short texts (100k positive and 100k\nnegative). Experimental results show that our proposed method can determine\nhumor in short texts with accuracy and an F1-score of 98.2 percent. Our 8-layer\nmodel with 110M parameters outperforms the baseline models with a large margin,\nshowing the importance of utilizing linguistic structure of texts in machine\nlearning models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12817v1",
            "date": "2020-04-27 14:00:09+00:00",
            "title": "LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning",
            "authors": [
                "Kaitao Song",
                "Hao Sun",
                "Xu Tan",
                "Tao Qin",
                "Jianfeng Lu",
                "Hongzhi Liu",
                "Tie-Yan Liu"
            ],
            "abstract": "While pre-training and fine-tuning, e.g., BERT~\\citep{devlin2018bert},\nGPT-2~\\citep{radford2019language}, have achieved great success in language\nunderstanding and generation tasks, the pre-trained models are usually too big\nfor online deployment in terms of both memory cost and inference speed, which\nhinders them from practical online usage. In this paper, we propose LightPAFF,\na Lightweight Pre-training And Fine-tuning Framework that leverages two-stage\nknowledge distillation to transfer knowledge from a big teacher model to a\nlightweight student model in both pre-training and fine-tuning stages. In this\nway the lightweight model can achieve similar accuracy as the big teacher\nmodel, but with much fewer parameters and thus faster online inference speed.\nLightPAFF can support different pre-training methods (such as BERT, GPT-2 and\nMASS~\\citep{song2019mass}) and be applied to many downstream tasks. Experiments\non three language understanding tasks, three language modeling tasks and three\nsequence to sequence generation tasks demonstrate that while achieving similar\naccuracy with the big BERT, GPT-2 and MASS models, LightPAFF reduces the model\nsize by nearly 5x and improves online inference speed by 5x-7x.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.12832v2",
            "date": "2020-04-27 14:21:03+00:00",
            "title": "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT",
            "authors": [
                "Omar Khattab",
                "Matei Zaharia"
            ],
            "abstract": "Recent progress in Natural Language Understanding (NLU) is driving fast-paced\nadvances in Information Retrieval (IR), largely owed to fine-tuning deep\nlanguage models (LMs) for document ranking. While remarkably effective, the\nranking models based on these LMs increase computational cost by orders of\nmagnitude over prior approaches, particularly as they must feed each\nquery-document pair through a massive neural network to compute a single\nrelevance score. To tackle this, we present ColBERT, a novel ranking model that\nadapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT\nintroduces a late interaction architecture that independently encodes the query\nand the document using BERT and then employs a cheap yet powerful interaction\nstep that models their fine-grained similarity. By delaying and yet retaining\nthis fine-granular interaction, ColBERT can leverage the expressiveness of deep\nLMs while simultaneously gaining the ability to pre-compute document\nrepresentations offline, considerably speeding up query processing. Beyond\nreducing the cost of re-ranking the documents retrieved by a traditional model,\nColBERT's pruning-friendly interaction mechanism enables leveraging\nvector-similarity indexes for end-to-end retrieval directly from a large\ndocument collection. We extensively evaluate ColBERT using two recent passage\nsearch datasets. Results show that ColBERT's effectiveness is competitive with\nexisting BERT-based models (and outperforms every non-BERT baseline), while\nexecuting two orders-of-magnitude faster and requiring four orders-of-magnitude\nfewer FLOPs per query.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.12993v1",
            "date": "2020-04-27 17:58:05+00:00",
            "title": "DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference",
            "authors": [
                "Ji Xin",
                "Raphael Tang",
                "Jaejun Lee",
                "Yaoliang Yu",
                "Jimmy Lin"
            ],
            "abstract": "Large-scale pre-trained language models such as BERT have brought significant\nimprovements to NLP applications. However, they are also notorious for being\nslow in inference, which makes them difficult to deploy in real-time\napplications. We propose a simple but effective method, DeeBERT, to accelerate\nBERT inference. Our approach allows samples to exit earlier without passing\nthrough the entire model. Experiments show that DeeBERT is able to save up to\n~40% inference time with minimal degradation in model quality. Further analyses\nshow different behaviors in the BERT transformer layers and also reveal their\nredundancy. Our work provides new ideas to efficiently apply deep\ntransformer-based models to downstream tasks. Code is available at\nhttps://github.com/castorini/DeeBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13078v1",
            "date": "2020-04-27 18:19:26+00:00",
            "title": "Context-aware Helpfulness Prediction for Online Product Reviews",
            "authors": [
                "Iyiola E. Olatunji",
                "Xin Li",
                "Wai Lam"
            ],
            "abstract": "Modeling and prediction of review helpfulness has become more predominant due\nto proliferation of e-commerce websites and online shops. Since the\nfunctionality of a product cannot be tested before buying, people often rely on\ndifferent kinds of user reviews to decide whether or not to buy a product.\nHowever, quality reviews might be buried deep in the heap of a large amount of\nreviews. Therefore, recommending reviews to customers based on the review\nquality is of the essence. Since there is no direct indication of review\nquality, most reviews use the information that ''X out of Y'' users found the\nreview helpful for obtaining the review quality. However, this approach\nundermines helpfulness prediction because not all reviews have statistically\nabundant votes. In this paper, we propose a neural deep learning model that\npredicts the helpfulness score of a review. This model is based on\nconvolutional neural network (CNN) and a context-aware encoding mechanism which\ncan directly capture relationships between words irrespective of their distance\nin a long sequence. We validated our model on human annotated dataset and the\nresult shows that our model significantly outperforms existing models for\nhelpfulness prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13117v3",
            "date": "2020-04-27 19:30:47+00:00",
            "title": "Conversational Question Answering over Passages by Leveraging Word Proximity Networks",
            "authors": [
                "Magdalena Kaiser",
                "Rishiraj Saha Roy",
                "Gerhard Weikum"
            ],
            "abstract": "Question answering (QA) over text passages is a problem of long-standing\ninterest in information retrieval. Recently, the conversational setting has\nattracted attention, where a user asks a sequence of questions to satisfy her\ninformation needs around a topic. While this setup is a natural one and similar\nto humans conversing with each other, it introduces two key research\nchallenges: understanding the context left implicit by the user in follow-up\nquestions, and dealing with ad hoc question formulations. In this work, we\ndemonstrate CROWN (Conversational passage ranking by Reasoning Over Word\nNetworks): an unsupervised yet effective system for conversational QA with\npassage responses, that supports several modes of context propagation over\nmultiple turns. To this end, CROWN first builds a word proximity network (WPN)\nfrom large corpora to store statistically significant term co-occurrences. At\nanswering time, passages are ranked by a combination of their similarity to the\nquestion, and coherence of query terms within: these factors are measured by\nreading off node and edge weights from the WPN. CROWN provides an interface\nthat is both intuitive for end-users, and insightful for experts for\nreconfiguration to individual setups. CROWN was evaluated on TREC CAsT data,\nwhere it achieved above-median performance in a pool of neural methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.13161v1",
            "date": "2020-04-27 20:34:26+00:00",
            "title": "PuzzLing Machines: A Challenge on Learning From Small Data",
            "authors": [
                "G\u00f6zde G\u00fcl \u015eahin",
                "Yova Kementchedjhieva",
                "Phillip Rust",
                "Iryna Gurevych"
            ],
            "abstract": "Deep neural models have repeatedly proved excellent at memorizing surface\npatterns from large datasets for various ML and NLP benchmarks. They struggle\nto achieve human-like thinking, however, because they lack the skill of\niterative reasoning upon knowledge. To expose this problem in a new light, we\nintroduce a challenge on learning from small data, PuzzLing Machines, which\nconsists of Rosetta Stone puzzles from Linguistic Olympiads for high school\nstudents. These puzzles are carefully designed to contain only the minimal\namount of parallel text necessary to deduce the form of unseen expressions.\nSolving them does not require external information (e.g., knowledge bases,\nvisual signals) or linguistic expertise, but meta-linguistic awareness and\ndeductive skills. Our challenge contains around 100 puzzles covering a wide\nrange of linguistic phenomena from 81 languages. We show that both simple\nstatistical algorithms and state-of-the-art deep neural models perform\ninadequately on this challenge, as expected. We hope that this benchmark,\navailable at https://ukplab.github.io/PuzzLing-Machines/, inspires further\nefforts towards a new paradigm in NLP---one that is grounded in human-like\nreasoning and understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13240v2",
            "date": "2020-04-28 01:47:37+00:00",
            "title": "MultiMix: A Robust Data Augmentation Framework for Cross-Lingual NLP",
            "authors": [
                "M Saiful Bari",
                "Tasnim Mohiuddin",
                "Shafiq Joty"
            ],
            "abstract": "Transfer learning has yielded state-of-the-art (SoTA) results in many\nsupervised natural language processing tasks. However, annotated data for every\ntarget task in every target language is rare, especially for low-resource\nlanguages. We propose MultiMix, a novel data augmentation framework for\nself-supervised learning in zero-resource transfer learning scenarios. In\nparticular, MultiMix targets to solve cross-lingual adaptation problems from a\nsource language distribution to an unknown target language distribution,\nassuming no training labels are available for the target language task. At its\ncore, MultiMix performs simultaneous self-training with data augmentation and\nunsupervised sample selection. To show its effectiveness, we conduct extensive\nexperiments on zero-resource cross-lingual transfer tasks for Named Entity\nRecognition and Natural Language Inference. MultiMix achieves SoTA results in\nboth tasks, outperforming the baselines by a good margin. With an in-depth\nmodel dissection, we demonstrate the cumulative contributions of different\ncomponents to MultiMix's success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13270v1",
            "date": "2020-04-28 03:44:34+00:00",
            "title": "Assessing the Bilingual Knowledge Learned by Neural Machine Translation Models",
            "authors": [
                "Shilin He",
                "Xing Wang",
                "Shuming Shi",
                "Michael R. Lyu",
                "Zhaopeng Tu"
            ],
            "abstract": "Machine translation (MT) systems translate text between different languages\nby automatically learning in-depth knowledge of bilingual lexicons, grammar and\nsemantics from the training examples. Although neural machine translation (NMT)\nhas led the field of MT, we have a poor understanding on how and why it works.\nIn this paper, we bridge the gap by assessing the bilingual knowledge learned\nby NMT models with phrase table -- an interpretable table of bilingual\nlexicons. We extract the phrase table from the training examples that an NMT\nmodel correctly predicts. Extensive experiments on widely-used datasets show\nthat the phrase table is reasonable and consistent against language pairs and\nrandom seeds. Equipped with the interpretable phrase table, we find that NMT\nmodels learn patterns from simple to complex and distill essential bilingual\nknowledge from the training examples. We also revisit some advances that\npotentially affect the learning of bilingual knowledge (e.g.,\nback-translation), and report some interesting findings. We believe this work\nopens a new angle to interpret NMT with statistic models, and provides\nempirical supports for recent advances in improving NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13278v3",
            "date": "2020-04-28 04:08:46+00:00",
            "title": "VD-BERT: A Unified Vision and Dialog Transformer with BERT",
            "authors": [
                "Yue Wang",
                "Shafiq Joty",
                "Michael R. Lyu",
                "Irwin King",
                "Caiming Xiong",
                "Steven C. H. Hoi"
            ],
            "abstract": "Visual dialog is a challenging vision-language task, where a dialog agent\nneeds to answer a series of questions through reasoning on the image content\nand dialog history. Prior work has mostly focused on various attention\nmechanisms to model such intricate interactions. By contrast, in this work, we\npropose VD-BERT, a simple yet effective framework of unified vision-dialog\nTransformer that leverages the pretrained BERT language models for Visual\nDialog tasks. The model is unified in that (1) it captures all the interactions\nbetween the image and the multi-turn dialog using a single-stream Transformer\nencoder, and (2) it supports both answer ranking and answer generation\nseamlessly through the same architecture. More crucially, we adapt BERT for the\neffective fusion of vision and dialog contents via visually grounded training.\nWithout the need of pretraining on external vision-language data, our model\nyields new state of the art, achieving the top position in both single-model\nand ensemble settings (74.54 and 75.35 NDCG scores) on the visual dialog\nleaderboard. Our code and pretrained models are released at\nhttps://github.com/salesforce/VD-BERT.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2004.13342v2",
            "date": "2020-04-28 07:33:14+00:00",
            "title": "Scheduled DropHead: A Regularization Method for Transformer Models",
            "authors": [
                "Wangchunshu Zhou",
                "Tao Ge",
                "Ke Xu",
                "Furu Wei",
                "Ming Zhou"
            ],
            "abstract": "In this paper, we introduce DropHead, a structured dropout method\nspecifically designed for regularizing the multi-head attention mechanism,\nwhich is a key component of transformer, a state-of-the-art model for various\nNLP tasks. In contrast to the conventional dropout mechanisms which randomly\ndrop units or connections, the proposed DropHead is a structured dropout\nmethod. It drops entire attention-heads during training and It prevents the\nmulti-head attention model from being dominated by a small portion of attention\nheads while also reduces the risk of overfitting the training data, thus making\nuse of the multi-head attention mechanism more efficiently. Motivated by recent\nstudies about the learning dynamic of the multi-head attention mechanism, we\npropose a specific dropout rate schedule to adaptively adjust the dropout rate\nof DropHead and achieve better regularization effect. Experimental results on\nboth machine translation and text classification benchmark datasets demonstrate\nthe effectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13432v2",
            "date": "2020-04-28 11:27:24+00:00",
            "title": "Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection",
            "authors": [
                "Wenliang Dai",
                "Tiezheng Yu",
                "Zihan Liu",
                "Pascale Fung"
            ],
            "abstract": "Nowadays, offensive content in social media has become a serious problem, and\nautomatically detecting offensive language is an essential task. In this paper,\nwe build an offensive language detection system, which combines multi-task\nlearning with BERT-based models. Using a pre-trained language model such as\nBERT, we can effectively learn the representations for noisy text in social\nmedia. Besides, to boost the performance of offensive language detection, we\nleverage the supervision signals from other related tasks. In the\nOffensEval-2020 competition, our model achieves 91.51% F1 score in English\nSub-task A, which is comparable to the first place (92.23%F1). An empirical\nanalysis is provided to explain the effectiveness of our approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13481v1",
            "date": "2020-04-25 01:43:00+00:00",
            "title": "A Linguistically Driven Framework for Query Expansion via Grammatical Constituent Highlighting and Role-Based Concept Weighting",
            "authors": [
                "Bhawani Selvaretnam",
                "Mohammed Belkhatir"
            ],
            "abstract": "In this paper, we propose a linguistically-motivated query expansion\nframework that recognizes and en-codes significant query constituents that\ncharacterize query intent in order to improve retrieval performance.\nConcepts-of-Interest are recognized as the core concepts that represent the\ngist of the search goal whilst the remaining query constituents which serve to\nspecify the search goal and complete the query structure are classified as\ndescriptive, relational or structural. Acknowledging the need to form\nsemantically-associated base pairs for the purpose of extracting related\npotential expansion concepts, an algorithm which capitalizes on syntactical\ndependencies to capture relationships between adjacent and non-adjacent query\nconcepts is proposed. Lastly, a robust weighting scheme that duly emphasizes\nthe importance of query constituents based on their linguistic role within the\nexpanded query is presented. We demonstrate improvements in retrieval\neffectiveness in terms of increased mean average precision (MAP) garnered by\nthe proposed linguistic-based query expansion framework through experimentation\non the TREC ad hoc test collections.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.13530v2",
            "date": "2020-04-28 13:57:21+00:00",
            "title": "Introducing a framework to assess newly created questions with Natural Language Processing",
            "authors": [
                "Luca Benedetto",
                "Andrea Cappelli",
                "Roberto Turrin",
                "Paolo Cremonesi"
            ],
            "abstract": "Statistical models such as those derived from Item Response Theory (IRT)\nenable the assessment of students on a specific subject, which can be useful\nfor several purposes (e.g., learning path customization, drop-out prediction).\nHowever, the questions have to be assessed as well and, although it is possible\nto estimate with IRT the characteristics of questions that have already been\nanswered by several students, this technique cannot be used on newly generated\nquestions. In this paper, we propose a framework to train and evaluate models\nfor estimating the difficulty and discrimination of newly created Multiple\nChoice Questions by extracting meaningful features from the text of the\nquestion and of the possible choices. We implement one model using this\nframework and test it on a real-world dataset provided by CloudAcademy, showing\nthat it outperforms previously proposed models, reducing by 6.7% the RMSE for\ndifficulty estimation and by 10.8% the RMSE for discrimination estimation. We\nalso present the results of an ablation study performed to support our features\nchoice and to show the effects of different characteristics of the questions'\ntext on difficulty and discrimination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13606v2",
            "date": "2020-04-28 15:41:12+00:00",
            "title": "The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions",
            "authors": [
                "Xiang Zhou",
                "Yixin Nie",
                "Hao Tan",
                "Mohit Bansal"
            ],
            "abstract": "We find that the performance of state-of-the-art models on Natural Language\nInference (NLI) and Reading Comprehension (RC) analysis/stress sets can be\nhighly unstable. This raises three questions: (1) How will the instability\naffect the reliability of the conclusions drawn based on these analysis sets?\n(2) Where does this instability come from? (3) How should we handle this\ninstability and what are some potential solutions? For the first question, we\nconduct a thorough empirical study over analysis sets and find that in addition\nto the unstable final performance, the instability exists all along the\ntraining curve. We also observe lower-than-expected correlations between the\nanalysis validation set and standard validation set, questioning the\neffectiveness of the current model-selection routine. Next, to answer the\nsecond question, we give both theoretical explanations and empirical evidence\nregarding the source of the instability, demonstrating that the instability\nmainly comes from high inter-example correlations within analysis sets.\nFinally, for the third question, we discuss an initial attempt to mitigate the\ninstability and suggest guidelines for future work such as reporting the\ndecomposed variance for more interpretable results and fair comparison across\nmodels. Our code is publicly available at:\nhttps://github.com/owenzx/InstabilityAnalysis",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13637v2",
            "date": "2020-04-28 16:33:25+00:00",
            "title": "Recipes for building an open-domain chatbot",
            "authors": [
                "Stephen Roller",
                "Emily Dinan",
                "Naman Goyal",
                "Da Ju",
                "Mary Williamson",
                "Yinhan Liu",
                "Jing Xu",
                "Myle Ott",
                "Kurt Shuster",
                "Eric M. Smith",
                "Y-Lan Boureau",
                "Jason Weston"
            ],
            "abstract": "Building open-domain chatbots is a challenging area for machine learning\nresearch. While prior work has shown that scaling neural models in the number\nof parameters and the size of the data they are trained on gives improved\nresults, we show that other ingredients are important for a high-performing\nchatbot. Good conversation requires a number of skills that an expert\nconversationalist blends in a seamless way: providing engaging talking points\nand listening to their partners, and displaying knowledge, empathy and\npersonality appropriately, while maintaining a consistent persona. We show that\nlarge scale models can learn these skills when given appropriate training data\nand choice of generation strategy. We build variants of these recipes with 90M,\n2.7B and 9.4B parameter models, and make our models and code publicly\navailable. Human evaluations show our best models are superior to existing\napproaches in multi-turn dialogue in terms of engagingness and humanness\nmeasurements. We then discuss the limitations of this work by analyzing failure\ncases of our models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13702v2",
            "date": "2020-04-28 17:57:08+00:00",
            "title": "Entity Type Prediction in Knowledge Graphs using Embeddings",
            "authors": [
                "Russa Biswas",
                "Radina Sofronova",
                "Mehwish Alam",
                "Harald Sack"
            ],
            "abstract": "Open Knowledge Graphs (such as DBpedia, Wikidata, YAGO) have been recognized\nas the backbone of diverse applications in the field of data mining and\ninformation retrieval. Hence, the completeness and correctness of the Knowledge\nGraphs (KGs) are vital. Most of these KGs are mostly created either via an\nautomated information extraction from Wikipedia snapshots or information\naccumulation provided by the users or using heuristics. However, it has been\nobserved that the type information of these KGs is often noisy, incomplete, and\nincorrect. To deal with this problem a multi-label classification approach is\nproposed in this work for entity typing using KG embeddings. We compare our\napproach with the current state-of-the-art type prediction method and report on\nexperiments with the KGs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13705v1",
            "date": "2020-04-28 17:59:01+00:00",
            "title": "Showing Your Work Doesn't Always Work",
            "authors": [
                "Raphael Tang",
                "Jaejun Lee",
                "Ji Xin",
                "Xinyu Liu",
                "Yaoliang Yu",
                "Jimmy Lin"
            ],
            "abstract": "In natural language processing, a recently popular line of work explores how\nto best report the experimental results of neural networks. One exemplar\npublication, titled \"Show Your Work: Improved Reporting of Experimental\nResults,\" advocates for reporting the expected validation effectiveness of the\nbest-tuned model, with respect to the computational budget. In the present\nwork, we critically examine this paper. As far as statistical generalizability\nis concerned, we find unspoken pitfalls and caveats with this approach. We\nanalytically show that their estimator is biased and uses error-prone\nassumptions. We find that the estimator favors negative errors and yields poor\nbootstrapped confidence intervals. We derive an unbiased alternative and\nbolster our claims with empirical evidence from statistical simulation. Our\ncodebase is at http://github.com/castorini/meanmax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13717v1",
            "date": "2020-04-28 14:26:12+00:00",
            "title": "Informational Space of Meaning for Scientific Texts",
            "authors": [
                "Neslihan Suzen",
                "Evgeny M. Mirkes",
                "Alexander N. Gorban"
            ],
            "abstract": "In Natural Language Processing, automatic extracting the meaning of texts\nconstitutes an important problem. Our focus is the computational analysis of\nmeaning of short scientific texts (abstracts or brief reports). In this paper,\na vector space model is developed for quantifying the meaning of words and\ntexts. We introduce the Meaning Space, in which the meaning of a word is\nrepresented by a vector of Relative Information Gain (RIG) about the subject\ncategories that the text belongs to, which can be obtained from observing the\nword in the text. This new approach is applied to construct the Meaning Space\nbased on Leicester Scientific Corpus (LSC) and Leicester Scientific\nDictionary-Core (LScDC). The LSC is a scientific corpus of 1,673,350 abstracts\nand the LScDC is a scientific dictionary which words are extracted from the\nLSC. Each text in the LSC belongs to at least one of 252 subject categories of\nWeb of Science (WoS). These categories are used in construction of vectors of\ninformation gains. The Meaning Space is described and statistically analysed\nfor the LSC with the LScDC. The usefulness of the proposed representation model\nis evaluated through top-ranked words in each category. The most informative n\nwords are ordered. We demonstrated that RIG-based word ranking is much more\nuseful than ranking based on raw word frequency in determining the\nscience-specific meaning and importance of a word. The proposed model based on\nRIG is shown to have ability to stand out topic-specific words in categories.\nThe most informative words are presented for 252 categories. The new scientific\ndictionary and the 103,998 x 252 Word-Category RIG Matrix are available online.\nAnalysis of the Meaning Space provides us with a tool to further explore\nquantifying the meaning of a text using more complex and context-dependent\nmeaning models that use co-occurrence of words and their combinations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13783v1",
            "date": "2020-04-28 19:27:48+00:00",
            "title": "Conspiracy in the Time of Corona: Automatic detection of Covid-19 Conspiracy Theories in Social Media and the News",
            "authors": [
                "Shadi Shahsavari",
                "Pavan Holur",
                "Timothy R. Tangherlini",
                "Vwani Roychowdhury"
            ],
            "abstract": "Rumors and conspiracy theories thrive in environments of low confidence and\nlow trust. Consequently, it is not surprising that ones related to the Covid-19\npandemic are proliferating given the lack of any authoritative scientific\nconsensus on the virus, its spread and containment, or on the long term social\nand economic ramifications of the pandemic. Among the stories currently\ncirculating are ones suggesting that the 5G network activates the virus, that\nthe pandemic is a hoax perpetrated by a global cabal, that the virus is a\nbio-weapon released deliberately by the Chinese, or that Bill Gates is using it\nas cover to launch a global surveillance regime. While some may be quick to\ndismiss these stories as having little impact on real-world behavior, recent\nevents including the destruction of property, racially fueled attacks against\nAsian Americans, and demonstrations espousing resistance to public health\norders countermand such conclusions. Inspired by narrative theory, we crawl\nsocial media sites and news reports and, through the application of automated\nmachine-learning methods, discover the underlying narrative frameworks\nsupporting the generation of these stories. We show how the various narrative\nframeworks fueling rumors and conspiracy theories rely on the alignment of\notherwise disparate domains of knowledge, and consider how they attach to the\nbroader reporting on the pandemic. These alignments and attachments, which can\nbe monitored in near real-time, may be useful for identifying areas in the news\nthat are particularly vulnerable to reinterpretation by conspiracy theorists.\nUnderstanding the dynamics of storytelling on social media and the narrative\nframeworks that provide the generative basis for these stories may also be\nhelpful for devising methods to disrupt their spread.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13796v4",
            "date": "2020-04-07 00:24:35+00:00",
            "title": "TextGAIL: Generative Adversarial Imitation Learning for Text Generation",
            "authors": [
                "Qingyang Wu",
                "Lei Li",
                "Zhou Yu"
            ],
            "abstract": "Generative Adversarial Networks (GANs) for text generation have recently\nreceived many criticisms, as they perform worse than their MLE counterparts. We\nsuspect previous text GANs' inferior performance is due to the lack of a\nreliable guiding signal in their discriminators. To address this problem, we\npropose a generative adversarial imitation learning framework for text\ngeneration that uses large pre-trained language models to provide more reliable\nreward guidance. Our approach uses contrastive discriminator, and proximal\npolicy optimization (PPO) to stabilize and improve text generation performance.\nFor evaluation, we conduct experiments on a diverse set of unconditional and\nconditional text generation tasks. Experimental results show that TextGAIL\nachieves better performance in terms of both quality and diversity than the MLE\nbaseline. We also validate our intuition that TextGAIL's discriminator\ndemonstrates the capability of providing reasonable rewards with an additional\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13805v3",
            "date": "2020-04-08 05:42:26+00:00",
            "title": "Multilingual Chart-based Constituency Parse Extraction from Pre-trained Language Models",
            "authors": [
                "Taeuk Kim",
                "Bowen Li",
                "Sang-goo Lee"
            ],
            "abstract": "As it has been unveiled that pre-trained language models (PLMs) are to some\nextent capable of recognizing syntactic concepts in natural language, much\neffort has been made to develop a method for extracting complete (binary)\nparses from PLMs without training separate parsers. We improve upon this\nparadigm by proposing a novel chart-based method and an effective top-K\nensemble technique. Moreover, we demonstrate that we can broaden the scope of\napplication of the approach into multilingual settings. Specifically, we show\nthat by applying our method on multilingual PLMs, it becomes possible to induce\nnon-trivial parses for sentences from nine languages in an integrated and\nlanguage-agnostic manner, attaining performance superior or comparable to that\nof unsupervised PCFGs. We also verify that our approach is robust to\ncross-lingual transfer. Finally, we provide analyses on the inner workings of\nour method. For instance, we discover universal attention heads which are\nconsistently sensitive to syntactic information irrespective of the input\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13820v2",
            "date": "2020-04-19 22:07:39+00:00",
            "title": "Evolution of Semantic Similarity -- A Survey",
            "authors": [
                "Dhivya Chandrasekaran",
                "Vijay Mago"
            ],
            "abstract": "Estimating the semantic similarity between text data is one of the\nchallenging and open research problems in the field of Natural Language\nProcessing (NLP). The versatility of natural language makes it difficult to\ndefine rule-based methods for determining semantic similarity measures. In\norder to address this issue, various semantic similarity methods have been\nproposed over the years. This survey article traces the evolution of such\nmethods, categorizing them based on their underlying principles as\nknowledge-based, corpus-based, deep neural network-based methods, and hybrid\nmethods. Discussing the strengths and weaknesses of each method, this survey\nprovides a comprehensive view of existing systems in place, for new researchers\nto experiment and develop innovative ideas to address the issue of semantic\nsimilarity.",
            "journal ref": "ACM Computing Surveys 54(2):1-37 (2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13822v1",
            "date": "2020-04-20 13:35:47+00:00",
            "title": "A Baseline for the Commands For Autonomous Vehicles Challenge",
            "authors": [
                "Simon Vandenhende",
                "Thierry Deruyttere",
                "Dusan Grujicic"
            ],
            "abstract": "The Commands For Autonomous Vehicles (C4AV) challenge requires participants\nto solve an object referral task in a real-world setting. More specifically, we\nconsider a scenario where a passenger can pass free-form natural language\ncommands to a self-driving car. This problem is particularly challenging, as\nthe language is much less constrained compared to existing benchmarks, and\nobject references are often implicit. The challenge is based on the recent\n\\texttt{Talk2Car} dataset. This document provides a technical overview of a\nmodel that we released to help participants get started in the competition. The\ncode can be found at https://github.com/talk2car/Talk2Car.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13823v1",
            "date": "2020-04-21 02:26:06+00:00",
            "title": "Leveraging Personal Navigation Assistant Systems Using Automated Social Media Traffic Reporting",
            "authors": [
                "Xiangpeng Wan",
                "Hakim Ghazzai",
                "Yehia Massoud"
            ],
            "abstract": "Modern urbanization is demanding smarter technologies to improve a variety of\napplications in intelligent transportation systems to relieve the increasing\namount of vehicular traffic congestion and incidents. Existing incident\ndetection techniques are limited to the use of sensors in the transportation\nnetwork and hang on human-inputs. Despite of its data abundance, social media\nis not well-exploited in such context. In this paper, we develop an automated\ntraffic alert system based on Natural Language Processing (NLP) that filters\nthis flood of information and extract important traffic-related bullets. To\nthis end, we employ the fine-tuning Bidirectional Encoder Representations from\nTransformers (BERT) language embedding model to filter the related traffic\ninformation from social media. Then, we apply a question-answering model to\nextract necessary information characterizing the report event such as its exact\nlocation, occurrence time, and nature of the events. We demonstrate the adopted\nNLP approaches outperform other existing approach and, after effectively\ntraining them, we focus on real-world situation and show how the developed\napproach can, in real-time, extract traffic-related information and\nautomatically convert them into alerts for navigation assistance applications\nsuch as navigation apps.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13831v1",
            "date": "2020-04-23 08:40:11+00:00",
            "title": "A Review of Winograd Schema Challenge Datasets and Approaches",
            "authors": [
                "Vid Kocijan",
                "Thomas Lukasiewicz",
                "Ernest Davis",
                "Gary Marcus",
                "Leora Morgenstern"
            ],
            "abstract": "The Winograd Schema Challenge is both a commonsense reasoning and natural\nlanguage understanding challenge, introduced as an alternative to the Turing\ntest. A Winograd schema is a pair of sentences differing in one or two words\nwith a highly ambiguous pronoun, resolved differently in the two sentences,\nthat appears to require commonsense knowledge to be resolved correctly. The\nexamples were designed to be easily solvable by humans but difficult for\nmachines, in principle requiring a deep understanding of the content of the\ntext and the situation it describes. This paper reviews existing Winograd\nSchema Challenge benchmark datasets and approaches that have been published\nsince its introduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13835v1",
            "date": "2020-04-24 09:25:45+00:00",
            "title": "A Tailored Pre-Training Model for Task-Oriented Dialog Generation",
            "authors": [
                "Jing Gu",
                "Qingyang Wu",
                "Chongruo Wu",
                "Weiyan Shi",
                "Zhou Yu"
            ],
            "abstract": "The recent success of large pre-trained language models such as BERT and\nGPT-2 has suggested the effectiveness of incorporating language priors in\ndownstream dialog generation tasks. However, the performance of pre-trained\nmodels on the dialog task is not as optimal as expected. In this paper, we\npropose a Pre-trained Role Alternating Language model (PRAL), designed\nspecifically for task-oriented conversational systems. We adopted (Wu et al.,\n2019) that models two speakers separately. We also design several techniques,\nsuch as start position randomization, knowledge distillation, and history\ndiscount to improve pre-training performance. We introduce a task-oriented\ndialog pretraining dataset by cleaning 13 existing data sets. We test PRAL on\nthree different downstream tasks. The results show that PRAL performs better or\non par with state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13841v1",
            "date": "2020-03-29 22:05:30+00:00",
            "title": "Neurals Networks for Projecting Named Entities from English to Ewondo",
            "authors": [
                "Michael Franklin Mbouopda",
                "Paulin Melatagia Yonta",
                "Guy Stephane B. Fedim Lombo"
            ],
            "abstract": "Named entity recognition is an important task in natural language processing.\nIt is very well studied for rich language, but still under explored for\nlow-resource languages. The main reason is that the existing techniques\nrequired a lot of annotated data to reach good performance. Recently, a new\ndistributional representation of words has been proposed to project named\nentities from a rich language to a low-resource one. This representation has\nbeen coupled to a neural network in order to project named entities from\nEnglish to Ewondo, a Bantu language spoken in Cameroon. Although the proposed\nmethod reached appreciable results, the size of the used neural network was too\nlarge compared to the size of the dataset. Furthermore the impact of the model\nparameters has not been studied. In this paper, we show experimentally that the\nsame results can be obtained using a smaller neural network. We also emphasize\nthe parameters that are highly correlated to the network performance. This work\nis a step forward to build a reliable and robust network architecture for named\nentity projection in low resource languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13889v2",
            "date": "2020-04-28 23:28:26+00:00",
            "title": "LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space",
            "authors": [
                "Tasnim Mohiuddin",
                "M Saiful Bari",
                "Shafiq Joty"
            ],
            "abstract": "Most of the successful and predominant methods for bilingual lexicon\ninduction (BLI) are mapping-based, where a linear mapping function is learned\nwith the assumption that the word embedding spaces of different languages\nexhibit similar geometric structures (i.e., approximately isomorphic). However,\nseveral recent studies have criticized this simplified assumption showing that\nit does not hold in general even for closely related languages. In this work,\nwe propose a novel semi-supervised method to learn cross-lingual word\nembeddings for BLI. Our model is independent of the isomorphic assumption and\nuses nonlinear mapping in the latent space of two independently trained\nauto-encoders. Through extensive experiments on fifteen (15) different language\npairs (in both directions) comprising resource-rich and low-resource languages\nfrom two different datasets, we demonstrate that our method outperforms\nexisting models by a good margin. Ablation studies show the importance of\ndifferent model components and the necessity of non-linear mapping.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13931v2",
            "date": "2020-04-29 02:47:04+00:00",
            "title": "Span-based Localizing Network for Natural Language Video Localization",
            "authors": [
                "Hao Zhang",
                "Aixin Sun",
                "Wei Jing",
                "Joey Tianyi Zhou"
            ],
            "abstract": "Given an untrimmed video and a text query, natural language video\nlocalization (NLVL) is to locate a matching span from the video that\nsemantically corresponds to the query. Existing solutions formulate NLVL either\nas a ranking task and apply multimodal matching architecture, or as a\nregression task to directly regress the target video span. In this work, we\naddress NLVL task with a span-based QA approach by treating the input video as\ntext passage. We propose a video span localizing network (VSLNet), on top of\nthe standard span-based QA framework, to address NLVL. The proposed VSLNet\ntackles the differences between NLVL and span-based QA through a simple yet\neffective query-guided highlighting (QGH) strategy. The QGH guides VSLNet to\nsearch for matching video span within a highlighted region. Through extensive\nexperiments on three benchmark datasets, we show that the proposed VSLNet\noutperforms the state-of-the-art methods; and adopting span-based QA framework\nis a promising direction to solve NLVL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13937v1",
            "date": "2020-04-29 03:20:22+00:00",
            "title": "Revisiting Round-Trip Translation for Quality Estimation",
            "authors": [
                "Jihyung Moon",
                "Hyunchang Cho",
                "Eunjeong L. Park"
            ],
            "abstract": "Quality estimation (QE) is the task of automatically evaluating the quality\nof translations without human-translated references. Calculating BLEU between\nthe input sentence and round-trip translation (RTT) was once considered as a\nmetric for QE, however, it was found to be a poor predictor of translation\nquality. Recently, various pre-trained language models have made breakthroughs\nin NLP tasks by providing semantically meaningful word and sentence embeddings.\nIn this paper, we employ semantic embeddings to RTT-based QE. Our method\nachieves the highest correlations with human judgments, compared to previous\nWMT 2019 quality estimation metric task submissions. While backward translation\nmodels can be a drawback when using RTT, we observe that with semantic-level\nmetrics, RTT-based QE is robust to the choice of the backward translation\nsystem. Additionally, the proposed method shows consistent performance for both\nSMT and NMT forward translation systems, implying the method does not penalize\na certain type of model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.13980v2",
            "date": "2020-04-29 06:41:26+00:00",
            "title": "Measuring Information Propagation in Literary Social Networks",
            "authors": [
                "Matthew Sims",
                "David Bamman"
            ],
            "abstract": "We present the task of modeling information propagation in literature, in\nwhich we seek to identify pieces of information passing from character A to\ncharacter B to character C, only given a description of their activity in text.\nWe describe a new pipeline for measuring information propagation in this domain\nand publish a new dataset for speaker attribution, enabling the evaluation of\nan important component of this pipeline on a wider range of literary texts than\npreviously studied. Using this pipeline, we analyze the dynamics of information\npropagation in over 5,000 works of fiction, finding that information flows\nthrough characters that fill structural holes connecting different communities,\nand that characters who are women are depicted as filling this role much more\nfrequently than characters who are men.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14025v3",
            "date": "2020-04-29 08:46:38+00:00",
            "title": "Multi-View Attention Network for Visual Dialog",
            "authors": [
                "Sungjin Park",
                "Taesun Whang",
                "Yeochan Yoon",
                "Heuiseok Lim"
            ],
            "abstract": "Visual dialog is a challenging vision-language task in which a series of\nquestions visually grounded by a given image are answered. To resolve the\nvisual dialog task, a high-level understanding of various multimodal inputs\n(e.g., question, dialog history, and image) is required. Specifically, it is\nnecessary for an agent to 1) determine the semantic intent of question and 2)\nalign question-relevant textual and visual contents among heterogeneous\nmodality inputs. In this paper, we propose Multi-View Attention Network (MVAN),\nwhich leverages multiple views about heterogeneous inputs based on attention\nmechanisms. MVAN effectively captures the question-relevant information from\nthe dialog history with two complementary modules (i.e., Topic Aggregation and\nContext Matching), and builds multimodal representations through sequential\nalignment processes (i.e., Modality Alignment). Experimental results on VisDial\nv1.0 dataset show the effectiveness of our proposed model, which outperforms\nthe previous state-of-the-art methods with respect to all evaluation metrics.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.14054v1",
            "date": "2020-04-29 10:06:00+00:00",
            "title": "Topic Propagation in Conversational Search",
            "authors": [
                "I. Mele",
                "C. I. Muntean",
                "F. M. Nardini",
                "R. Perego",
                "N. Tonellotto",
                "O. Frieder"
            ],
            "abstract": "In a conversational context, a user expresses her multi-faceted information\nneed as a sequence of natural-language questions, i.e., utterances. Starting\nfrom a given topic, the conversation evolves through user utterances and system\nreplies. The retrieval of documents relevant to a given utterance in a\nconversation is challenging due to ambiguity of natural language and to the\ndifficulty of detecting possible topic shifts and semantic relationships among\nutterances. We adopt the 2019 TREC Conversational Assistant Track (CAsT)\nframework to experiment with a modular architecture performing: (i) topic-aware\nutterance rewriting, (ii) retrieval of candidate passages for the rewritten\nutterances, and (iii) neural-based re-ranking of candidate passages. We present\na comprehensive experimental evaluation of the architecture assessed in terms\nof traditional IR metrics at small cutoffs. Experimental results show the\neffectiveness of our techniques that achieve an improvement up to 0.28 (+93%)\nfor P@1 and 0.19 (+89.9%) for nDCG@3 w.r.t. the CAsT baseline.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.14069v2",
            "date": "2020-04-29 10:44:00+00:00",
            "title": "Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension",
            "authors": [
                "Fei Yuan",
                "Linjun Shou",
                "Xuanyu Bai",
                "Ming Gong",
                "Yaobo Liang",
                "Nan Duan",
                "Yan Fu",
                "Daxin Jiang"
            ],
            "abstract": "Multilingual pre-trained models could leverage the training data from a rich\nsource language (such as English) to improve performance on low resource\nlanguages. However, the transfer quality for multilingual Machine Reading\nComprehension (MRC) is significantly worse than sentence classification tasks\nmainly due to the requirement of MRC to detect the word level answer boundary.\nIn this paper, we propose two auxiliary tasks in the fine-tuning stage to\ncreate additional phrase boundary supervision: (1) A mixed MRC task, which\ntranslates the question or passage to other languages and builds cross-lingual\nquestion-passage pairs; (2) A language-agnostic knowledge masking task by\nleveraging knowledge phrases mined from web. Besides, extensive experiments on\ntwo cross-lingual MRC datasets show the effectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14074v1",
            "date": "2020-04-29 10:54:40+00:00",
            "title": "Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning",
            "authors": [
                "Alexandre Tamborrino",
                "Nicola Pellicano",
                "Baptiste Pannier",
                "Pascal Voitot",
                "Louise Naudin"
            ],
            "abstract": "Fine-tuning of pre-trained transformer models has become the standard\napproach for solving common NLP tasks. Most of the existing approaches rely on\na randomly initialized classifier on top of such networks. We argue that this\nfine-tuning procedure is sub-optimal as the pre-trained model has no prior on\nthe specific classifier labels, while it might have already learned an\nintrinsic textual representation of the task. In this paper, we introduce a new\nscoring method that casts a plausibility ranking task in a full-text format and\nleverages the masked language modeling head tuned during the pre-training\nphase. We study commonsense reasoning tasks where the model must rank a set of\nhypotheses given a premise, focusing on the COPA, Swag, HellaSwag and\nCommonsenseQA datasets. By exploiting our scoring method without fine-tuning,\nwe are able to produce strong baselines (e.g. 80% test accuracy on COPA) that\nare comparable to supervised approaches. Moreover, when fine-tuning directly on\nthe proposed scoring function, we show that our method provides a much more\nstable training phase across random restarts (e.g $\\times 10$ standard\ndeviation reduction on COPA test accuracy) and requires less annotated data\nthan the standard classifier approach to reach equivalent performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14096v1",
            "date": "2020-04-29 11:37:53+00:00",
            "title": "Do Neural Language Models Show Preferences for Syntactic Formalisms?",
            "authors": [
                "Artur Kulmizev",
                "Vinit Ravishankar",
                "Mostafa Abdou",
                "Joakim Nivre"
            ],
            "abstract": "Recent work on the interpretability of deep neural language models has\nconcluded that many properties of natural language syntax are encoded in their\nrepresentational spaces. However, such studies often suffer from limited scope\nby focusing on a single language and a single linguistic formalism. In this\nstudy, we aim to investigate the extent to which the semblance of syntactic\nstructure captured by language models adheres to a surface-syntactic or deep\nsyntactic style of analysis, and whether the patterns are consistent across\ndifferent languages. We apply a probe for extracting directed dependency trees\nto BERT and ELMo models trained on 13 different languages, probing for two\ndifferent syntactic annotation styles: Universal Dependencies (UD),\nprioritizing deep syntactic relations, and Surface-Syntactic Universal\nDependencies (SUD), focusing on surface structure. We find that both models\nexhibit a preference for UD over SUD - with interesting variations across\nlanguages and layers - and that the strength of this preference is correlated\nwith differences in tree shape.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14118v1",
            "date": "2020-04-29 12:18:14+00:00",
            "title": "Analysing Lexical Semantic Change with Contextualised Word Representations",
            "authors": [
                "Mario Giulianelli",
                "Marco Del Tredici",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "This paper presents the first unsupervised approach to lexical semantic\nchange that makes use of contextualised word representations. We propose a\nnovel method that exploits the BERT neural language model to obtain\nrepresentations of word usages, clusters these representations into usage\ntypes, and measures change along time with three proposed metrics. We create a\nnew evaluation dataset and show that the model representations and the detected\nsemantic shifts are positively correlated with human judgements. Our extensive\nqualitative analysis demonstrates that our method captures a variety of\nsynchronic and diachronic linguistic phenomena. We expect our work to inspire\nfurther research in this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14130v1",
            "date": "2020-04-16 21:23:28+00:00",
            "title": "A Workflow Manager for Complex NLP and Content Curation Pipelines",
            "authors": [
                "Juli\u00e1n Moreno-Schneider",
                "Peter Bourgonje",
                "Florian Kintzel",
                "Georg Rehm"
            ],
            "abstract": "We present a workflow manager for the flexible creation and customisation of\nNLP processing pipelines. The workflow manager addresses challenges in\ninteroperability across various different NLP tasks and hardware-based resource\nusage. Based on the four key principles of generality, flexibility, scalability\nand efficiency, we present the first version of the workflow manager by\nproviding details on its custom definition language, explaining the\ncommunication components and the general system architecture and setup. We\ncurrently implement the system, which is grounded and motivated by real-world\nindustry use cases in several innovation and transfer projects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14135v1",
            "date": "2020-03-29 20:23:14+00:00",
            "title": "BERT Fine-tuning For Arabic Text Summarization",
            "authors": [
                "Khalid N. Elmadani",
                "Mukhtar Elgezouli",
                "Anas Showk"
            ],
            "abstract": "Fine-tuning a pretrained BERT model is the state of the art method for\nextractive/abstractive text summarization, in this paper we showcase how this\nfine-tuning method can be applied to the Arabic language to both construct the\nfirst documented model for abstractive Arabic text summarization and show its\nperformance in Arabic extractive summarization. Our model works with\nmultilingual BERT (as Arabic language does not have a pretrained BERT of its\nown). We show its performance in English corpus first before applying it to\nArabic corpora in both extractive and abstractive tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14218v2",
            "date": "2020-04-29 14:07:18+00:00",
            "title": "Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Recently, fine-tuning pre-trained language models (e.g., multilingual BERT)\nto downstream cross-lingual tasks has shown promising results. However, the\nfine-tuning process inevitably changes the parameters of the pre-trained model\nand weakens its cross-lingual ability, which leads to sub-optimal performance.\nTo alleviate this problem, we leverage continual learning to preserve the\noriginal cross-lingual ability of the pre-trained model when we fine-tune it to\ndownstream tasks. The experimental result shows that our fine-tuning methods\ncan better preserve the cross-lingual ability of the pre-trained model in a\nsentence retrieval task. Our methods also achieve better performance than other\nfine-tuning baselines on the zero-shot cross-lingual part-of-speech tagging and\nnamed entity recognition tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14224v1",
            "date": "2020-04-29 14:22:42+00:00",
            "title": "Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning",
            "authors": [
                "Tao Shen",
                "Yi Mao",
                "Pengcheng He",
                "Guodong Long",
                "Adam Trischler",
                "Weizhu Chen"
            ],
            "abstract": "In this work, we aim at equipping pre-trained language models with structured\nknowledge. We present two self-supervised tasks learning over raw text with the\nguidance from knowledge graphs. Building upon entity-level masked language\nmodels, our first contribution is an entity masking scheme that exploits\nrelational knowledge underlying the text. This is fulfilled by using a linked\nknowledge graph to select informative entities and then masking their mentions.\nIn addition we use knowledge graphs to obtain distractors for the masked\nentities, and propose a novel distractor-suppressed ranking objective which is\noptimized jointly with masked language model. In contrast to existing\nparadigms, our approach uses knowledge graphs implicitly, only during\npre-training, to inject language models with structured knowledge via learning\nfrom raw text. It is more efficient than retrieval-based methods that perform\nentity linking and integration during finetuning and inference, and generalizes\nmore effectively than the methods that directly learn from concatenated graph\ntriples. Experiments show that our proposed model achieves improved performance\non five benchmark datasets, including question answering and knowledge base\ncompletion tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14254v1",
            "date": "2020-04-29 15:02:41+00:00",
            "title": "Task-oriented Dialogue System for Automatic Disease Diagnosis via Hierarchical Reinforcement Learning",
            "authors": [
                "Kangenbei Liao",
                "Qianlong Liu",
                "Zhongyu Wei",
                "Baolin Peng",
                "Qin Chen",
                "Weijian Sun",
                "Xuanjing Huang"
            ],
            "abstract": "In this paper, we focus on automatic disease diagnosis with reinforcement\nlearning (RL) methods in task-oriented dialogues setting. Different from\nconventional RL tasks, the action space for disease diagnosis (i.e., symptoms)\nis inevitably large, especially when the number of diseases increases. However,\nexisting approaches to this problem employ a flat RL policy, which typically\nworks well in simple tasks but has significant challenges in complex scenarios\nlike disease diagnosis. Towards this end, we propose to integrate a\nhierarchical policy of two levels into the dialogue policy learning. The high\nlevel policy consists of a model named master that is responsible for\ntriggering a model in low level, the low level policy consists of several\nsymptom checkers and a disease classifier. Experimental results on both\nself-constructed real-world and synthetic datasets demonstrate that our\nhierarchical framework achieves higher accuracy in disease diagnosis compared\nwith existing systems. Besides, the datasets\n(http://www.sdspeople.fudan.edu.cn/zywei/data/Fudan-Medical-Dialogue2.0) and\ncodes (https://github.com/nnbay/MeicalChatbot-HRL) are all available now.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.14283v3",
            "date": "2020-04-29 15:59:30+00:00",
            "title": "SubjQA: A Dataset for Subjectivity and Review Comprehension",
            "authors": [
                "Johannes Bjerva",
                "Nikita Bhutani",
                "Behzad Golshan",
                "Wang-Chiew Tan",
                "Isabelle Augenstein"
            ],
            "abstract": "Subjectivity is the expression of internal opinions or beliefs which cannot\nbe objectively observed or verified, and has been shown to be important for\nsentiment analysis and word-sense disambiguation. Furthermore, subjectivity is\nan important aspect of user-generated data. In spite of this, subjectivity has\nnot been investigated in contexts where such data is widespread, such as in\nquestion answering (QA). We therefore investigate the relationship between\nsubjectivity and QA, while developing a new dataset. We compare and contrast\nwith analyses from previous work, and verify that findings regarding\nsubjectivity still hold when using recently developed NLP architectures. We\nfind that subjectivity is also an important feature in the case of QA, albeit\nwith more intricate interactions between subjectivity and QA performance. For\ninstance, a subjective question may or may not be associated with a subjective\nanswer. We release an English QA dataset (SubjQA) based on customer reviews,\ncontaining subjectivity annotations for questions and answer spans across 6\ndistinct domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14299v1",
            "date": "2020-04-29 16:17:49+00:00",
            "title": "Detecting Perceived Emotions in Hurricane Disasters",
            "authors": [
                "Shrey Desai",
                "Cornelia Caragea",
                "Junyi Jessy Li"
            ],
            "abstract": "Natural disasters (e.g., hurricanes) affect millions of people each year,\ncausing widespread destruction in their wake. People have recently taken to\nsocial media websites (e.g., Twitter) to share their sentiments and feelings\nwith the larger community. Consequently, these platforms have become\ninstrumental in understanding and perceiving emotions at scale. In this paper,\nwe introduce HurricaneEmo, an emotion dataset of 15,000 English tweets spanning\nthree hurricanes: Harvey, Irma, and Maria. We present a comprehensive study of\nfine-grained emotions and propose classification tasks to discriminate between\ncoarse-grained emotion groups. Our best BERT model, even after task-guided\npre-training which leverages unlabeled Twitter data, achieves only 68% accuracy\n(averaged across all groups). HurricaneEmo serves not only as a challenging\nbenchmark for models but also as a valuable resource for analyzing emotions in\ndisaster-centric domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14303v1",
            "date": "2020-04-29 16:24:02+00:00",
            "title": "TUNIZI: a Tunisian Arabizi sentiment analysis Dataset",
            "authors": [
                "Chayma Fourati",
                "Abir Messaoudi",
                "Hatem Haddad"
            ],
            "abstract": "On social media, Arabic people tend to express themselves in their own local\ndialects. More particularly, Tunisians use the informal way called \"Tunisian\nArabizi\". Analytical studies seek to explore and recognize online opinions\naiming to exploit them for planning and prediction purposes such as measuring\nthe customer satisfaction and establishing sales and marketing strategies.\nHowever, analytical studies based on Deep Learning are data hungry. On the\nother hand, African languages and dialects are considered low resource\nlanguages. For instance, to the best of our knowledge, no annotated Tunisian\nArabizi dataset exists. In this paper, we introduce TUNIZI a sentiment analysis\nTunisian Arabizi Dataset, collected from social networks, preprocessed for\nanalytical studies and annotated manually by Tunisian native speakers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14307v2",
            "date": "2020-04-29 16:28:22+00:00",
            "title": "UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues",
            "authors": [
                "Hung Le",
                "Doyen Sahoo",
                "Chenghao Liu",
                "Nancy F. Chen",
                "Steven C. H. Hoi"
            ],
            "abstract": "Building an end-to-end conversational agent for multi-domain task-oriented\ndialogues has been an open challenge for two main reasons. First, tracking\ndialogue states of multiple domains is non-trivial as the dialogue agent must\nobtain complete states from all relevant domains, some of which might have\nshared slots among domains as well as unique slots specifically for one domain\nonly. Second, the dialogue agent must also process various types of information\nacross domains, including dialogue context, dialogue states, and database, to\ngenerate natural responses to users. Unlike the existing approaches that are\noften designed to train each module separately, we propose \"UniConv\" -- a novel\nunified neural architecture for end-to-end conversational systems in\nmulti-domain task-oriented dialogues, which is designed to jointly train (i) a\nBi-level State Tracker which tracks dialogue states by learning signals at both\nslot and domain level independently, and (ii) a Joint Dialogue Act and Response\nGenerator which incorporates information from various input components and\nmodels dialogue acts and target responses simultaneously. We conduct\ncomprehensive experiments in dialogue state tracking, context-to-text, and\nend-to-end settings on the MultiWOZ2.1 benchmark, achieving superior\nperformance over competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14338v2",
            "date": "2020-04-29 17:10:10+00:00",
            "title": "Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube",
            "authors": [
                "Jack Hessel",
                "Zhenhai Zhu",
                "Bo Pang",
                "Radu Soricut"
            ],
            "abstract": "Pretraining from unlabelled web videos has quickly become the de-facto means\nof achieving high performance on many video understanding tasks. Features are\nlearned via prediction of grounded relationships between visual content and\nautomatic speech recognition (ASR) tokens. However, prior pretraining work has\nbeen limited to only instructional videos; a priori, we expect this domain to\nbe relatively \"easy:\" speakers in instructional videos will often reference the\nliteral objects/actions being depicted. We ask: can similar models be trained\non more diverse video corpora? And, if so, what types of videos are \"grounded\"\nand what types are not? We fit a representative pretraining model to the\ndiverse YouTube8M dataset, and study its success and failure cases. We find\nthat visual-textual grounding is indeed possible across previously unexplored\nvideo categories, and that pretraining on a more diverse set results in\nrepresentations that generalize to both non-instructional and instructional\ndomains.",
            "journal ref": "Published in EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14353v2",
            "date": "2020-04-29 17:31:11+00:00",
            "title": "End-to-End Slot Alignment and Recognition for Cross-Lingual NLU",
            "authors": [
                "Weijia Xu",
                "Batool Haider",
                "Saab Mansour"
            ],
            "abstract": "Natural language understanding (NLU) in the context of goal-oriented dialog\nsystems typically includes intent classification and slot labeling tasks.\nExisting methods to expand an NLU system to new languages use machine\ntranslation with slot label projection from source to the translated\nutterances, and thus are sensitive to projection errors. In this work, we\npropose a novel end-to-end model that learns to align and predict target slot\nlabels jointly for cross-lingual transfer. We introduce MultiATIS++, a new\nmultilingual NLU corpus that extends the Multilingual ATIS corpus to nine\nlanguages across four language families, and evaluate our method using the\ncorpus. Results show that our method outperforms a simple label projection\nmethod using fast-align on most languages, and achieves competitive performance\nto the more complex, state-of-the-art projection method with only half of the\ntraining time. We release our MultiATIS++ corpus to the community to continue\nfuture research on cross-lingual NLU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14355v3",
            "date": "2020-04-29 17:33:31+00:00",
            "title": "Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation",
            "authors": [
                "Nithin Holla",
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "The success of deep learning methods hinges on the availability of large\ntraining datasets annotated for the task of interest. In contrast to human\nintelligence, these methods lack versatility and struggle to learn and adapt\nquickly to new tasks, where labeled data is scarce. Meta-learning aims to solve\nthis problem by training a model on a large number of few-shot tasks, with an\nobjective to learn new tasks quickly from a small number of examples. In this\npaper, we propose a meta-learning framework for few-shot word sense\ndisambiguation (WSD), where the goal is to learn to disambiguate unseen words\nfrom only a few labeled instances. Meta-learning approaches have so far been\ntypically tested in an $N$-way, $K$-shot classification setting where each task\nhas $N$ classes with $K$ examples per class. Owing to its nature, WSD deviates\nfrom this controlled setup and requires the models to handle a large number of\nhighly unbalanced classes. We extend several popular meta-learning approaches\nto this scenario, and analyze their strengths and weaknesses in this new\nchallenging setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14356v1",
            "date": "2020-04-29 17:33:41+00:00",
            "title": "AxCell: Automatic Extraction of Results from Machine Learning Papers",
            "authors": [
                "Marcin Kardas",
                "Piotr Czapla",
                "Pontus Stenetorp",
                "Sebastian Ruder",
                "Sebastian Riedel",
                "Ross Taylor",
                "Robert Stojnic"
            ],
            "abstract": "Tracking progress in machine learning has become increasingly difficult with\nthe recent explosion in the number of papers. In this paper, we present AxCell,\nan automatic machine learning pipeline for extracting results from papers.\nAxCell uses several novel components, including a table segmentation subtask,\nto learn relevant structural knowledge that aids extraction. When compared with\nexisting methods, our approach significantly improves the state of the art for\nresults extraction. We also release a structured, annotated dataset for\ntraining models for results extraction, and a dataset for evaluating the\nperformance of models on this task. Lastly, we show the viability of our\napproach enables it to be used for semi-automated results extraction in\nproduction, suggesting our improvements make this task practically viable for\nthe first time. Code is available on GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14364v1",
            "date": "2020-04-29 17:43:24+00:00",
            "title": "Generating Safe Diversity in NLG via Imitation Learning",
            "authors": [
                "Giulio Zhou",
                "Gerasimos Lampouras"
            ],
            "abstract": "Deep-learning models for language generation tasks tend to produce repetitive\noutput. Various methods have been proposed to encourage lexical diversity\nduring decoding, but this often comes at a cost to the perceived fluency and\nadequacy of the output. In this work, we propose to ameliorate this cost by\nusing an Imitation Learning approach to explore the level of diversity that a\nlanguage generation model can safely produce. Specifically, we augment the\ndecoding process with a meta-classifier trained to distinguish which words at\nany given timestep will lead to high-quality output. We focus our experiments\non concept-to-text generation where models are sensitive to the inclusion of\nirrelevant words due to the strict relation between input and output. Our\nanalysis shows that previous methods for diversity underperform in this\nsetting, while human evaluation suggests that our proposed method achieves a\nhigh level of diversity with minimal effect to the output's fluency and\nadequacy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14366v2",
            "date": "2020-04-29 17:45:12+00:00",
            "title": "Elastic weight consolidation for better bias inoculation",
            "authors": [
                "James Thorne",
                "Andreas Vlachos"
            ],
            "abstract": "The biases present in training datasets have been shown to affect models for\nsentence pair classification tasks such as natural language inference (NLI) and\nfact verification. While fine-tuning models on additional data has been used to\nmitigate them, a common issue is that of catastrophic forgetting of the\noriginal training dataset. In this paper, we show that elastic weight\nconsolidation (EWC) allows fine-tuning of models to mitigate biases while being\nless susceptible to catastrophic forgetting. In our evaluation on fact\nverification and NLI stress tests, we show that fine-tuning with EWC dominates\nstandard fine-tuning, yielding models with lower levels of forgetting on the\noriginal (biased) dataset for equivalent gains in accuracy on the fine-tuning\n(unbiased) dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14373v3",
            "date": "2020-04-29 17:53:45+00:00",
            "title": "ToTTo: A Controlled Table-To-Text Generation Dataset",
            "authors": [
                "Ankur P. Parikh",
                "Xuezhi Wang",
                "Sebastian Gehrmann",
                "Manaal Faruqui",
                "Bhuwan Dhingra",
                "Diyi Yang",
                "Dipanjan Das"
            ],
            "abstract": "We present ToTTo, an open-domain English table-to-text dataset with over\n120,000 training examples that proposes a controlled generation task: given a\nWikipedia table and a set of highlighted table cells, produce a one-sentence\ndescription. To obtain generated targets that are natural but also faithful to\nthe source table, we introduce a dataset construction process where annotators\ndirectly revise existing candidate sentences from Wikipedia. We present\nsystematic analyses of our dataset and annotation process as well as results\nachieved by several state-of-the-art baselines. While usually fluent, existing\nmethods often hallucinate phrases that are not supported by the table,\nsuggesting that this dataset can serve as a useful research benchmark for\nhigh-precision conditional text generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14451v2",
            "date": "2020-04-29 20:00:53+00:00",
            "title": "Pragmatic Issue-Sensitive Image Captioning",
            "authors": [
                "Allen Nie",
                "Reuben Cohn-Gordon",
                "Christopher Potts"
            ],
            "abstract": "Image captioning systems have recently improved dramatically, but they still\ntend to produce captions that are insensitive to the communicative goals that\ncaptions should meet. To address this, we propose Issue-Sensitive Image\nCaptioning (ISIC). In ISIC, a captioning system is given a target image and an\nissue, which is a set of images partitioned in a way that specifies what\ninformation is relevant. The goal of the captioner is to produce a caption that\nresolves this issue. To model this task, we use an extension of the Rational\nSpeech Acts model of pragmatic language use. Our extension is built on top of\nstate-of-the-art pretrained neural image captioners and explicitly reasons\nabout issues in our sense. We establish experimentally that these models\ngenerate captions that are both highly descriptive and issue-sensitive, and we\nshow how ISIC can complement and enrich the related task of Visual Question\nAnswering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14503v3",
            "date": "2020-04-29 22:21:31+00:00",
            "title": "Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation",
            "authors": [
                "Ji Ma",
                "Ivan Korotkov",
                "Yinfei Yang",
                "Keith Hall",
                "Ryan McDonald"
            ],
            "abstract": "A major obstacle to the wide-spread adoption of neural retrieval models is\nthat they require large supervised training sets to surpass traditional\nterm-based techniques, which are constructed from raw corpora. In this paper,\nwe propose an approach to zero-shot learning for passage retrieval that uses\nsynthetic question generation to close this gap. The question generation system\nis trained on general domain data, but is applied to documents in the targeted\ndomain. This allows us to create arbitrarily large, yet noisy, question-passage\nrelevance pairs that are domain specific. Furthermore, when this is coupled\nwith a simple hybrid term-neural model, first-stage retrieval performance can\nbe improved further. Empirically, we show that this is an effective strategy\nfor building neural passage retrieval models in the absence of large training\ncorpora. Depending on the domain, this technique can even approach the accuracy\nof supervised models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2004.14514v1",
            "date": "2020-04-29 23:32:42+00:00",
            "title": "Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition",
            "authors": [
                "Hiroki Ouchi",
                "Jun Suzuki",
                "Sosuke Kobayashi",
                "Sho Yokoi",
                "Tatsuki Kuribayashi",
                "Ryuto Konno",
                "Kentaro Inui"
            ],
            "abstract": "Interpretable rationales for model predictions play a critical role in\npractical applications. In this study, we develop models possessing\ninterpretable inference process for structured prediction. Specifically, we\npresent a method of instance-based learning that learns similarities between\nspans. At inference time, each span is assigned a class label based on its\nsimilar spans in the training set, where it is easy to understand how much each\ntraining instance contributes to the predictions. Through empirical analysis on\nnamed entity recognition, we demonstrate that our method enables to build\nmodels that have high interpretability without sacrificing performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14546v1",
            "date": "2020-04-30 02:20:14+00:00",
            "title": "WT5?! Training Text-to-Text Models to Explain their Predictions",
            "authors": [
                "Sharan Narang",
                "Colin Raffel",
                "Katherine Lee",
                "Adam Roberts",
                "Noah Fiedel",
                "Karishma Malkan"
            ],
            "abstract": "Neural networks have recently achieved human-level performance on various\nchallenging natural language processing (NLP) tasks, but it is notoriously\ndifficult to understand why a neural network produced a particular prediction.\nIn this paper, we leverage the text-to-text framework proposed by Raffel et\nal.(2019) to train language models to output a natural text explanation\nalongside their prediction. Crucially, this requires no modifications to the\nloss function or training and decoding procedures -- we simply train the model\nto output the explanation after generating the (natural text) prediction. We\nshow that this approach not only obtains state-of-the-art results on\nexplainability benchmarks, but also permits learning from a limited set of\nlabeled explanations and transferring rationalization abilities across\ndatasets. To facilitate reproducibility and future work, we release our code\nuse to train the models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14554v1",
            "date": "2020-04-30 03:13:28+00:00",
            "title": "Indirect Identification of Psychosocial Risks from Natural Language",
            "authors": [
                "Kristen C. Allen",
                "Alex Davis",
                "Tamar Krishnamurti"
            ],
            "abstract": "During the perinatal period, psychosocial health risks, including depression\nand intimate partner violence, are associated with serious adverse health\noutcomes for parents and children. To appropriately intervene, healthcare\nprofessionals must first identify those at risk, yet stigma often prevents\npeople from directly disclosing the information needed to prompt an assessment.\nWe examine indirect methods of eliciting and analyzing information that could\nindicate psychosocial risks. Short diary entries by peripartum women exhibit\nthematic patterns, extracted by topic modeling, and emotional perspective,\ndrawn from dictionary-informed sentiment features. Using these features, we use\nregularized regression to predict screening measures of depression and\npsychological aggression by an intimate partner. Journal text entries\nquantified through topic models and sentiment features show promise for\ndepression prediction, with performance almost as good as closed-form\nquestions. Text-based features were less useful for prediction of intimate\npartner violence, but moderately indirect multiple-choice questioning allowed\nfor detection without explicit disclosure. Both methods may serve as an initial\nor complementary screening approach to detecting stigmatized risks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14589v2",
            "date": "2020-04-30 05:31:31+00:00",
            "title": "Improved Natural Language Generation via Loss Truncation",
            "authors": [
                "Daniel Kang",
                "Tatsunori Hashimoto"
            ],
            "abstract": "Neural language models are usually trained to match the distributional\nproperties of a large-scale corpus by minimizing the log loss. While\nstraightforward to optimize, this approach forces the model to reproduce all\nvariations in the dataset, including noisy and invalid references (e.g.,\nmisannotation and hallucinated facts). Worse, the commonly used log loss is\noverly sensitive to such phenomena and even a small fraction of noisy data can\ndegrade performance. In this work, we show that the distinguishability of the\nmodels and reference serves as a principled and robust alternative for handling\ninvalid references. To optimize distinguishability, we propose loss truncation,\nwhich adaptively removes high loss examples during training. We show this is as\neasy to optimize as log loss and tightly bounds distinguishability under noise.\nEmpirically, we demonstrate that loss truncation outperforms existing baselines\non distinguishability on a summarization task, and show that samples generated\nby the loss truncation model have factual accuracy ratings that exceed those of\nbaselines and match human references.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14592v1",
            "date": "2020-04-30 05:59:12+00:00",
            "title": "EnsembleGAN: Adversarial Learning for Retrieval-Generation Ensemble Model on Short-Text Conversation",
            "authors": [
                "Jiayi Zhang",
                "Chongyang Tao",
                "Zhenjing Xu",
                "Qiaojing Xie",
                "Wei Chen",
                "Rui Yan"
            ],
            "abstract": "Generating qualitative responses has always been a challenge for\nhuman-computer dialogue systems. Existing dialogue systems generally derive\nfrom either retrieval-based or generative-based approaches, both of which have\ntheir own pros and cons. Despite the natural idea of an ensemble model of the\ntwo, existing ensemble methods only focused on leveraging one approach to\nenhance another, we argue however that they can be further mutually enhanced\nwith a proper training strategy. In this paper, we propose ensembleGAN, an\nadversarial learning framework for enhancing a retrieval-generation ensemble\nmodel in open-domain conversation scenario. It consists of a\nlanguage-model-like generator, a ranker generator, and one ranker\ndiscriminator. Aiming at generating responses that approximate the ground-truth\nand receive high ranking scores from the discriminator, the two generators\nlearn to generate improved highly relevant responses and competitive unobserved\ncandidates respectively, while the discriminative ranker is trained to identify\ntrue responses from adversarial ones, thus featuring the merits of both\ngenerator counterparts. The experimental results on a large short-text\nconversation data demonstrate the effectiveness of the ensembleGAN by the\namelioration on both human and automatic evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14648v3",
            "date": "2020-04-30 09:10:57+00:00",
            "title": "Robust Question Answering Through Sub-part Alignment",
            "authors": [
                "Jifan Chen",
                "Greg Durrett"
            ],
            "abstract": "Current textual question answering models achieve strong performance on\nin-domain test sets, but often do so by fitting surface-level patterns in the\ndata, so they fail to generalize to out-of-distribution settings. To make a\nmore robust and understandable QA system, we model question answering as an\nalignment problem. We decompose both the question and context into smaller\nunits based on off-the-shelf semantic representations (here, semantic roles),\nand align the question to a subgraph of the context in order to find the\nanswer. We formulate our model as a structured SVM, with alignment scores\ncomputed via BERT, and we can train end-to-end despite using beam search for\napproximate inference. Our explicit use of alignments allows us to explore a\nset of constraints with which we can prohibit certain types of bad model\nbehavior arising in cross-domain settings. Furthermore, by investigating\ndifferences in scores across different potential answers, we can seek to\nunderstand what particular aspects of the input lead the model to choose the\nanswer without relying on post-hoc explanation techniques. We train our model\non SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.\nThe results show that our model is more robust cross-domain than the standard\nBERT QA model, and constraints derived from alignment scores allow us to\neffectively trade off coverage and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14667v2",
            "date": "2020-04-30 10:11:33+00:00",
            "title": "NUBIA: NeUral Based Interchangeability Assessor for Text Generation",
            "authors": [
                "Hassan Kane",
                "Muhammed Yusuf Kocyigit",
                "Ali Abdalla",
                "Pelkins Ajanoh",
                "Mohamed Coulibali"
            ],
            "abstract": "We present NUBIA, a methodology to build automatic evaluation metrics for\ntext generation using only machine learning models as core components. A\ntypical NUBIA model is composed of three modules: a neural feature extractor,\nan aggregator and a calibrator. We demonstrate an implementation of NUBIA which\noutperforms metrics currently used to evaluate machine translation, summaries\nand slightly exceeds/matches state of the art metrics on correlation with human\njudgement on the WMT segment-level Direct Assessment task, sentence-level\nranking and image captioning evaluation. The model implemented is modular,\nexplainable and set to continuously improve over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14677v1",
            "date": "2020-04-30 10:33:40+00:00",
            "title": "AMPERSAND: Argument Mining for PERSuAsive oNline Discussions",
            "authors": [
                "Tuhin Chakrabarty",
                "Christopher Hidey",
                "Smaranda Muresan",
                "Kathy Mckeown",
                "Alyssa Hwang"
            ],
            "abstract": "Argumentation is a type of discourse where speakers try to persuade their\naudience about the reasonableness of a claim by presenting supportive\narguments. Most work in argument mining has focused on modeling arguments in\nmonologues. We propose a computational model for argument mining in online\npersuasive discussion forums that brings together the micro-level (argument as\nproduct) and macro-level (argument as process) models of argumentation.\nFundamentally, this approach relies on identifying relations between components\nof arguments in a discussion thread. Our approach for relation prediction uses\ncontextual information in terms of fine-tuning a pre-trained language model and\nleveraging discourse relations based on Rhetorical Structure Theory. We\nadditionally propose a candidate selection method to automatically predict what\nparts of one's argument will be targeted by other participants in the\ndiscussion. Our models obtain significant improvements compared to recent\nstate-of-the-art approaches using pointer networks and a pre-trained language\nmodel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14704v2",
            "date": "2020-04-30 11:36:33+00:00",
            "title": "A Span-based Linearization for Constituent Trees",
            "authors": [
                "Yang Wei",
                "Yuanbin Wu",
                "Man Lan"
            ],
            "abstract": "We propose a novel linearization of a constituent tree, together with a new\nlocally normalized model. For each split point in a sentence, our model\ncomputes the normalizer on all spans ending with that split point, and then\npredicts a tree span from them. Compared with global models, our model is fast\nand parallelizable. Different from previous local models, our linearization\nmethod is tied on the spans directly and considers more local features when\nperforming span prediction, which is more interpretable and effective.\nExperiments on PTB (95.8 F1) and CTB (92.4 F1) show that our model\nsignificantly outperforms existing local models and efficiently achieves\ncompetitive results with global models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14754v2",
            "date": "2020-04-30 13:20:18+00:00",
            "title": "Self-Supervised and Controlled Multi-Document Opinion Summarization",
            "authors": [
                "Hady Elsahar",
                "Maximin Coavoux",
                "Matthias Gall\u00e9",
                "Jos Rozen"
            ],
            "abstract": "We address the problem of unsupervised abstractive summarization of\ncollections of user generated reviews with self-supervision and control. We\npropose a self-supervised setup that considers an individual document as a\ntarget summary for a set of similar documents. This setting makes training\nsimpler than previous approaches by relying only on standard log-likelihood\nloss. We address the problem of hallucinations through the use of control\ncodes, to steer the generation towards more coherent and relevant\nsummaries.Finally, we extend the Transformer architecture to allow for multiple\nreviews as input. Our benchmarks on two datasets against graph-based and recent\nneural abstractive unsupervised models show that our proposed method generates\nsummaries with a superior quality and relevance.This is confirmed in our human\nevaluation which focuses explicitly on the faithfulness of generated summaries\nWe also provide an ablation study, which shows the importance of the control\nsetup in controlling hallucinations and achieve high sentiment and topic\nalignment of the summaries with the input reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14837v2",
            "date": "2020-04-30 14:47:05+00:00",
            "title": "Accurate Word Alignment Induction from Neural Machine Translation",
            "authors": [
                "Yun Chen",
                "Yang Liu",
                "Guanhua Chen",
                "Xin Jiang",
                "Qun Liu"
            ],
            "abstract": "Despite its original goal to jointly learn to align and translate, prior\nresearches suggest that Transformer captures poor word alignments through its\nattention mechanism. In this paper, we show that attention weights DO capture\naccurate word alignments and propose two novel word alignment induction methods\nShift-Att and Shift-AET. The main idea is to induce alignments at the step when\nthe to-be-aligned target token is the decoder input rather than the decoder\noutput as in previous work. Shift-Att is an interpretation method that induces\nalignments from the attention weights of Transformer and does not require\nparameter update or architecture change. Shift-AET extracts alignments from an\nadditional alignment module which is tightly integrated into Transformer and\ntrained in isolation with supervision from symmetrized Shift-Att alignments.\nExperiments on three publicly available datasets demonstrate that both methods\nperform better than their corresponding neural baselines and Shift-AET\nsignificantly outperforms GIZA++ by 1.4-4.8 AER points.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14839v2",
            "date": "2020-04-30 14:48:39+00:00",
            "title": "Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?",
            "authors": [
                "Hitomi Yanaka",
                "Koji Mineshima",
                "Daisuke Bekki",
                "Kentaro Inui"
            ],
            "abstract": "Despite the success of language models using neural networks, it remains\nunclear to what extent neural models have the generalization ability to perform\ninferences. In this paper, we introduce a method for evaluating whether neural\nmodels can learn systematicity of monotonicity inference in natural language,\nnamely, the regularity for performing arbitrary inferences with generalization\non composition. We consider four aspects of monotonicity inferences and test\nwhether the models can systematically interpret lexical and logical phenomena\non different training/test splits. A series of experiments show that three\nneural models systematically draw inferences on unseen combinations of lexical\nand logical phenomena when the syntactic structures of the sentences are\nsimilar between the training and test sets. However, the performance of the\nmodels significantly decreases when the structures are slightly changed in the\ntest set while retaining all vocabularies and constituents already appearing in\nthe training set. This indicates that the generalization ability of neural\nmodels is limited to cases where the syntactic structures are nearly the same\nas those in the training set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14843v1",
            "date": "2020-04-30 14:55:09+00:00",
            "title": "Knowledge Graph Embeddings and Explainable AI",
            "authors": [
                "Federico Bianchi",
                "Gaetano Rossiello",
                "Luca Costabello",
                "Matteo Palmonari",
                "Pasquale Minervini"
            ],
            "abstract": "Knowledge graph embeddings are now a widely adopted approach to knowledge\nrepresentation in which entities and relationships are embedded in vector\nspaces. In this chapter, we introduce the reader to the concept of knowledge\ngraph embeddings by explaining what they are, how they can be generated and how\nthey can be evaluated. We summarize the state-of-the-art in this field by\ndescribing the approaches that have been introduced to represent knowledge in\nthe vector space. In relation to knowledge representation, we consider the\nproblem of explainability, and discuss models and methods for explaining\npredictions obtained via knowledge graph embeddings.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2004.14905v1",
            "date": "2020-04-30 16:03:06+00:00",
            "title": "Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation",
            "authors": [
                "David Wilmot",
                "Frank Keller"
            ],
            "abstract": "Suspense is a crucial ingredient of narrative fiction, engaging readers and\nmaking stories compelling. While there is a vast theoretical literature on\nsuspense, it is computationally not well understood. We compare two ways for\nmodelling suspense: surprise, a backward-looking measure of how unexpected the\ncurrent state is given the story so far; and uncertainty reduction, a\nforward-looking measure of how unexpected the continuation of the story is.\nBoth can be computed either directly over story representations or over their\nprobability distributions. We propose a hierarchical language model that\nencodes stories and computes surprise and uncertainty reduction. Evaluating\nagainst short stories annotated with human suspense judgements, we find that\nuncertainty reduction over representations is the best predictor, resulting in\nnear-human accuracy. We also show that uncertainty reduction can be used to\npredict suspenseful events in movie synopses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14992v3",
            "date": "2020-04-30 17:36:14+00:00",
            "title": "How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking",
            "authors": [
                "Nicola De Cao",
                "Michael Schlichtkrull",
                "Wilker Aziz",
                "Ivan Titov"
            ],
            "abstract": "Attribution methods assess the contribution of inputs to the model\nprediction. One way to do so is erasure: a subset of inputs is considered\nirrelevant if it can be removed without affecting the prediction. Though\nconceptually simple, erasure's objective is intractable and approximate search\nremains expensive with modern deep NLP models. Erasure is also susceptible to\nthe hindsight bias: the fact that an input can be dropped does not mean that\nthe model `knows' it can be dropped. The resulting pruning is over-aggressive\nand does not reflect how the model arrives at the prediction. To deal with\nthese challenges, we introduce Differentiable Masking. DiffMask learns to\nmask-out subsets of the input while maintaining differentiability. The decision\nto include or disregard an input token is made with a simple model based on\nintermediate hidden layers of the analyzed model. First, this makes the\napproach efficient because we predict rather than search. Second, as with\nprobing classifiers, this reveals what the network `knows' at the corresponding\nlayers. This lets us not only plot attribution heatmaps but also analyze how\ndecisions are formed across network layers. We use DiffMask to study BERT\nmodels on sentiment classification and question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.14996v2",
            "date": "2020-04-30 17:38:27+00:00",
            "title": "Segatron: Segment-Aware Transformer for Language Modeling and Understanding",
            "authors": [
                "He Bai",
                "Peng Shi",
                "Jimmy Lin",
                "Yuqing Xie",
                "Luchen Tan",
                "Kun Xiong",
                "Wen Gao",
                "Ming Li"
            ],
            "abstract": "Transformers are powerful for sequence modeling. Nearly all state-of-the-art\nlanguage models and pre-trained language models are based on the Transformer\narchitecture. However, it distinguishes sequential tokens only with the token\nposition index. We hypothesize that better contextual representations can be\ngenerated from the Transformer with richer positional information. To verify\nthis, we propose a segment-aware Transformer (Segatron), by replacing the\noriginal token position encoding with a combined position encoding of\nparagraph, sentence, and token. We first introduce the segment-aware mechanism\nto Transformer-XL, which is a popular Transformer-based language model with\nmemory extension and relative position encoding. We find that our method can\nfurther improve the Transformer-XL base model and large model, achieving 17.1\nperplexity on the WikiText-103 dataset. We further investigate the pre-training\nmasked language modeling task with Segatron. Experimental results show that\nBERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla\nTransformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence\nrepresentation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2004.15001v2",
            "date": "2020-04-30 17:47:17+00:00",
            "title": "Don't Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings",
            "authors": [
                "Phillip Keung",
                "Yichao Lu",
                "Julian Salazar",
                "Vikas Bhardwaj"
            ],
            "abstract": "Multilingual contextual embeddings have demonstrated state-of-the-art\nperformance in zero-shot cross-lingual transfer learning, where multilingual\nBERT is fine-tuned on one source language and evaluated on a different target\nlanguage. However, published results for mBERT zero-shot accuracy vary as much\nas 17 points on the MLDoc classification task across four papers. We show that\nthe standard practice of using English dev accuracy for model selection in the\nzero-shot setting makes it difficult to obtain reproducible results on the\nMLDoc and XNLI tasks. English dev accuracy is often uncorrelated (or even\nanti-correlated) with target language accuracy, and zero-shot performance\nvaries greatly at different points in the same fine-tuning run and between\ndifferent fine-tuning runs. These reproducibility issues are also present for\nother tasks with different pre-trained embeddings (e.g., MLQA with XLM-R). We\nrecommend providing oracle scores alongside zero-shot results: still fine-tune\nusing English data, but choose a checkpoint with the target dev set. Reporting\nthis upper bound makes results more consistent by avoiding arbitrarily bad\ncheckpoints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00042v1",
            "date": "2020-04-30 18:28:42+00:00",
            "title": "Method for Customizable Automated Tagging: Addressing the Problem of Over-tagging and Under-tagging Text Documents",
            "authors": [
                "Maharshi R. Pandya",
                "Jessica Reyes",
                "Bob Vanderheyden"
            ],
            "abstract": "Using author provided tags to predict tags for a new document often results\nin the overgeneration of tags. In the case where the author doesn't provide any\ntags, our documents face the severe under-tagging issue. In this paper, we\npresent a method to generate a universal set of tags that can be applied widely\nto a large document corpus. Using IBM Watson's NLU service, first, we collect\nkeywords/phrases that we call \"complex document tags\" from 8,854 popular\nreports in the corpus. We apply LDA model over these complex document tags to\ngenerate a set of 765 unique \"simple tags\". In applying the tags to a corpus of\ndocuments, we run each document through the IBM Watson NLU and apply\nappropriate simple tags. Using only 765 simple tags, our method allows us to\ntag 87,397 out of 88,583 total documents in the corpus with at least one tag.\nAbout 92.1% of the total 87,397 documents are also determined to be\nsufficiently-tagged. In the end, we discuss the performance of our method and\nits limitations.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.00048v1",
            "date": "2020-04-30 18:39:25+00:00",
            "title": "Context based Text-generation using LSTM networks",
            "authors": [
                "Sivasurya Santhanam"
            ],
            "abstract": "Long short-term memory(LSTM) units on sequence-based models are being used in\ntranslation, question-answering systems, classification tasks due to their\ncapability of learning long-term dependencies. In Natural language generation,\nLSTM networks are providing impressive results on text generation models by\nlearning language models with grammatically stable syntaxes. But the downside\nis that the network does not learn about the context. The network only learns\nthe input-output function and generates text given a set of input words\nirrespective of pragmatics. As the model is trained without any such context,\nthere is no semantic consistency among the generated sentences. The proposed\nmodel is trained to generate text for a given set of input words along with a\ncontext vector. A context vector is similar to a paragraph vector that grasps\nthe semantic meaning(context) of the sentence. Several methods of extracting\nthe context vectors are proposed in this work. While training a language model,\nin addition to the input-output sequences, context vectors are also trained\nalong with the inputs. Due to this structure, the model learns the relation\namong the input words, context vector and the target word. Given a set of\ncontext terms, a well trained model will generate text around the provided\ncontext. Based on the nature of computing context vectors, the model has been\ntried out with two variations (word importance and word clustering). In the\nword clustering method, the suitable embeddings among various domains are also\nexplored. The results are evaluated based on the semantic closeness of the\ngenerated text to the given context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00062v1",
            "date": "2020-04-30 19:19:37+00:00",
            "title": "Attribution Analysis of Grammatical Dependencies in LSTMs",
            "authors": [
                "Yiding Hao"
            ],
            "abstract": "LSTM language models have been shown to capture syntax-sensitive grammatical\ndependencies such as subject-verb agreement with a high degree of accuracy\n(Linzen et al., 2016, inter alia). However, questions remain regarding whether\nthey do so using spurious correlations, or whether they are truly able to match\nverbs with their subjects. This paper argues for the latter hypothesis. Using\nlayer-wise relevance propagation (Bach et al., 2015), a technique that\nquantifies the contributions of input features to model behavior, we show that\nLSTM performance on number agreement is directly correlated with the model's\nability to distinguish subjects from other nouns. Our results suggest that LSTM\nlanguage models are able to infer robust representations of syntactic\ndependencies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00100v2",
            "date": "2020-04-30 21:00:53+00:00",
            "title": "Linguistic Typology Features from Text: Inferring the Sparse Features of World Atlas of Language Structures",
            "authors": [
                "Alexander Gutkin",
                "Tatiana Merkulova",
                "Martin Jansche"
            ],
            "abstract": "The use of linguistic typological resources in natural language processing\nhas been steadily gaining more popularity. It has been observed that the use of\ntypological information, often combined with distributed language\nrepresentations, leads to significantly more powerful models. While linguistic\ntypology representations from various resources have mostly been used for\nconditioning the models, there has been relatively little attention on\npredicting features from these resources from the input data. In this paper we\ninvestigate whether the various linguistic features from World Atlas of\nLanguage Structures (WALS) can be reliably inferred from multi-lingual text.\nSuch a predictor can be used to infer structural features for a language never\nobserved in training data. We frame this task as a multi-label classification\ninvolving predicting the set of non-mutually exclusive and extremely sparse\nmulti-valued labels (WALS features). We construct a recurrent neural network\npredictor based on byte embeddings and convolutional layers and test its\nperformance on 556 languages, providing analysis for various linguistic types,\nmacro-areas, language families and individual features. We show that some\nfeatures from various linguistic types can be predicted reliably.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00129v2",
            "date": "2020-04-30 22:34:34+00:00",
            "title": "Structure-Tags Improve Text Classification for Scholarly Document Quality Prediction",
            "authors": [
                "Gideon Maillette de Buy Wenniger",
                "Thomas van Dongen",
                "Eleri Aedmaa",
                "Herbert Teun Kruitbosch",
                "Edwin A. Valentijn",
                "Lambert Schomaker"
            ],
            "abstract": "Training recurrent neural networks on long texts, in particular scholarly\ndocuments, causes problems for learning. While hierarchical attention networks\n(HANs) are effective in solving these problems, they still lose important\ninformation about the structure of the text. To tackle these problems, we\npropose the use of HANs combined with structure-tags which mark the role of\nsentences in the document. Adding tags to sentences, marking them as\ncorresponding to title, abstract or main body text, yields improvements over\nthe state-of-the-art for scholarly document quality prediction. The proposed\nsystem is applied to the task of accept/reject prediction on the PeerRead\ndataset and compared against a recent BiLSTM-based model and joint\ntextual+visual model as well as against plain HANs. Compared to plain HANs,\naccuracy increases on all three domains. On the computation and language domain\nour new model works best overall, and increases accuracy 4.7% over the best\nliterature result. We also obtain improvements when introducing the tags for\nprediction of the number of citations for 88k scientific publications that we\ncompiled from the Allen AI S2ORC dataset. For our HAN-system with\nstructure-tags we reach 28.5% explained variance, an improvement of 1.8% over\nour reimplementation of the BiLSTM-based model as well as 1.0% improvement over\nplain HANs.",
            "journal ref": "Proceedings of the First Workshop on Scholarly Document\n  Processing. Association for Computational Linguistics. (2020) 158-167.\n  EMNLP|SDP 2020 https://www.aclweb.org/anthology/2020.sdp-1.18",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00136v1",
            "date": "2020-04-30 23:01:12+00:00",
            "title": "Contextual Text Style Transfer",
            "authors": [
                "Yu Cheng",
                "Zhe Gan",
                "Yizhe Zhang",
                "Oussama Elachqar",
                "Dianqi Li",
                "Jingjing Liu"
            ],
            "abstract": "We introduce a new task, Contextual Text Style Transfer - translating a\nsentence into a desired style with its surrounding context taken into account.\nThis brings two key challenges to existing style transfer approaches: ($i$) how\nto preserve the semantic meaning of target sentence and its consistency with\nsurrounding context during transfer; ($ii$) how to train a robust model with\nlimited labeled data accompanied with context. To realize high-quality style\ntransfer with natural context preservation, we propose a Context-Aware Style\nTransfer (CAST) model, which uses two separate encoders for each input sentence\nand its surrounding context. A classifier is further trained to ensure\ncontextual consistency of the generated sentence. To compensate for the lack of\nparallel data, additional self-reconstruction and back-translation losses are\nintroduced to leverage non-parallel data in a semi-supervised fashion. Two new\nbenchmarks, Enron-Context and Reddit-Context, are introduced for formality and\noffensiveness style transfer. Experimental results on these datasets\ndemonstrate the effectiveness of the proposed CAST model over state-of-the-art\nmethods across style accuracy, content preservation and contextual consistency\nmetrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00147v2",
            "date": "2020-04-30 23:58:03+00:00",
            "title": "Interpretable Entity Representations through Large-Scale Typing",
            "authors": [
                "Yasumasa Onoe",
                "Greg Durrett"
            ],
            "abstract": "In standard methodology for natural language processing, entities in text are\ntypically embedded in dense vector spaces with pre-trained models. The\nembeddings produced this way are effective when fed into downstream models, but\nthey require end-task fine-tuning and are fundamentally difficult to interpret.\nIn this paper, we present an approach to creating entity representations that\nare human readable and achieve high performance on entity-related tasks out of\nthe box. Our representations are vectors whose values correspond to posterior\nprobabilities over fine-grained entity types, indicating the confidence of a\ntyping model's decision that the entity belongs to the corresponding type. We\nobtain these representations using a fine-grained entity typing model, trained\neither on supervised ultra-fine entity typing data (Choi et al. 2018) or\ndistantly-supervised examples from Wikipedia. On entity probing tasks involving\nrecognizing entity identity, our embeddings used in parameter-free downstream\nmodels achieve competitive performance with ELMo- and BERT-based embeddings in\ntrained models. We also show that it is possible to reduce the size of our type\nset in a learning-based way for particular domains. Finally, we show that these\nembeddings can be post-hoc modified through a small number of rules to\nincorporate domain knowledge and improve performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00159v2",
            "date": "2020-05-01 00:47:37+00:00",
            "title": "Why and when should you pool? Analyzing Pooling in Recurrent Architectures",
            "authors": [
                "Pratyush Maini",
                "Keshav Kolluru",
                "Danish Pruthi",
                "Mausam"
            ],
            "abstract": "Pooling-based recurrent neural architectures consistently outperform their\ncounterparts without pooling. However, the reasons for their enhanced\nperformance are largely unexamined. In this work, we examine three commonly\nused pooling techniques (mean-pooling, max-pooling, and attention), and propose\nmax-attention, a novel variant that effectively captures interactions among\npredictive tokens in a sentence. We find that pooling-based architectures\nsubstantially differ from their non-pooling equivalents in their learning\nability and positional biases--which elucidate their performance benefits. By\nanalyzing the gradient propagation, we discover that pooling facilitates better\ngradient flow compared to BiLSTMs. Further, we expose how BiLSTMs are\npositionally biased towards tokens in the beginning and the end of a sequence.\nPooling alleviates such biases. Consequently, we identify settings where\npooling offers large benefits: (i) in low resource scenarios, and (ii) when\nimportant words lie towards the middle of the sentence. Among the pooling\ntechniques studied, max-attention is the most effective, resulting in\nsignificant performance gains on several text classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00162v2",
            "date": "2020-05-01 01:03:16+00:00",
            "title": "Recurrent Interaction Network for Jointly Extracting Entities and Classifying Relations",
            "authors": [
                "Kai Sun",
                "Richong Zhang",
                "Samuel Mensah",
                "Yongyi Mao",
                "Xudong Liu"
            ],
            "abstract": "The idea of using multi-task learning approaches to address the joint\nextraction of entity and relation is motivated by the relatedness between the\nentity recognition task and the relation classification task. Existing methods\nusing multi-task learning techniques to address the problem learn interactions\namong the two tasks through a shared network, where the shared information is\npassed into the task-specific networks for prediction. However, such an\napproach hinders the model from learning explicit interactions between the two\ntasks to improve the performance on the individual tasks. As a solution, we\ndesign a multi-task learning model which we refer to as recurrent interaction\nnetwork which allows the learning of interactions dynamically, to effectively\nmodel task-specific features for classification. Empirical studies on two\nreal-world datasets confirm the superiority of the proposed model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00174v2",
            "date": "2020-05-01 01:58:24+00:00",
            "title": "Universal Adversarial Attacks with Natural Triggers for Text Classification",
            "authors": [
                "Liwei Song",
                "Xinwei Yu",
                "Hsuan-Tung Peng",
                "Karthik Narasimhan"
            ],
            "abstract": "Recent work has demonstrated the vulnerability of modern text classifiers to\nuniversal adversarial attacks, which are input-agnostic sequences of words\nadded to text processed by classifiers. Despite being successful, the word\nsequences produced in such attacks are often ungrammatical and can be easily\ndistinguished from natural text. We develop adversarial attacks that appear\ncloser to natural English phrases and yet confuse classification systems when\nadded to benign inputs. We leverage an adversarially regularized autoencoder\n(ARAE) to generate triggers and propose a gradient-based search that aims to\nmaximize the downstream classifier's prediction loss. Our attacks effectively\nreduce model accuracy on classification tasks while being less identifiable\nthan prior models as per automatic detection metrics and human-subject studies.\nOur aim is to demonstrate that adversarial attacks can be made harder to detect\nthan previously thought and to enable the development of appropriate defenses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00206v1",
            "date": "2020-05-01 04:03:58+00:00",
            "title": "TransOMCS: From Linguistic Graphs to Commonsense Knowledge",
            "authors": [
                "Hongming Zhang",
                "Daniel Khashabi",
                "Yangqiu Song",
                "Dan Roth"
            ],
            "abstract": "Commonsense knowledge acquisition is a key problem for artificial\nintelligence. Conventional methods of acquiring commonsense knowledge generally\nrequire laborious and costly human annotations, which are not feasible on a\nlarge scale. In this paper, we explore a practical way of mining commonsense\nknowledge from linguistic graphs, with the goal of transferring cheap knowledge\nobtained with linguistic patterns into expensive commonsense knowledge. The\nresult is a conversion of ASER [Zhang et al., 2020], a large-scale selectional\npreference knowledge resource, into TransOMCS, of the same representation as\nConceptNet [Liu and Singh, 2004] but two orders of magnitude larger.\nExperimental results demonstrate the transferability of linguistic knowledge to\ncommonsense knowledge and the effectiveness of the proposed approach in terms\nof quantity, novelty, and quality. TransOMCS is publicly available at:\nhttps://github.com/HKUST-KnowComp/TransOMCS.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.00239v1",
            "date": "2020-05-01 06:20:36+00:00",
            "title": "Biomedical Entity Representations with Synonym Marginalization",
            "authors": [
                "Mujeen Sung",
                "Hwisang Jeon",
                "Jinhyuk Lee",
                "Jaewoo Kang"
            ],
            "abstract": "Biomedical named entities often play important roles in many biomedical text\nmining tools. However, due to the incompleteness of provided synonyms and\nnumerous variations in their surface forms, normalization of biomedical\nentities is very challenging. In this paper, we focus on learning\nrepresentations of biomedical entities solely based on the synonyms of\nentities. To learn from the incomplete synonyms, we use a model-based candidate\nselection and maximize the marginal likelihood of the synonyms present in top\ncandidates. Our model-based candidates are iteratively updated to contain more\ndifficult negative samples as our model evolves. In this way, we avoid the\nexplicit pre-selection of negative samples from more than 400K candidates. On\nfour biomedical entity normalization datasets having three different entity\ntypes (disease, chemical, adverse reaction), our model BioSyn consistently\noutperforms previous state-of-the-art models almost reaching the upper bound on\neach dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00278v2",
            "date": "2020-05-01 09:20:48+00:00",
            "title": "Unsupervised Transfer of Semantic Role Models from Verbal to Nominal Domain",
            "authors": [
                "Yanpeng Zhao",
                "Ivan Titov"
            ],
            "abstract": "Semantic role labeling (SRL) is an NLP task involving the assignment of\npredicate arguments to types, called semantic roles. Though research on SRL has\nprimarily focused on verbal predicates and many resources available for SRL\nprovide annotations only for verbs, semantic relations are often triggered by\nother linguistic constructions, e.g., nominalizations. In this work, we\ninvestigate a transfer scenario where we assume role-annotated data for the\nsource verbal domain but only unlabeled data for the target nominal domain. Our\nkey assumption, enabling the transfer between the two domains, is that\nselectional preferences of a role (i.e., preferences or constraints on the\nadmissible arguments) do not strongly depend on whether the relation is\ntriggered by a verb or a noun. For example, the same set of arguments can fill\nthe Acquirer role for the verbal predicate `acquire' and its nominal form\n`acquisition'. We approach the transfer task from the variational autoencoding\nperspective. The labeler serves as an encoder (predicting role labels given a\nsentence), whereas selectional preferences are captured in the decoder\ncomponent (generating arguments for the predicting roles). Nominal roles are\nnot labeled in the training data, and the learning objective instead pushes the\nlabeler to assign roles predictive of the arguments. Sharing the decoder\nparameters across the domains encourages consistency between labels predicted\nfor both domains and facilitates the transfer. The method substantially\noutperforms baselines, such as unsupervised and `direct transfer' methods, on\nthe English CoNLL-2009 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00295v1",
            "date": "2020-05-01 10:16:40+00:00",
            "title": "Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing",
            "authors": [
                "Manikandan Ravikiran",
                "Amin Ekant Muljibhai",
                "Toshinori Miyoshi",
                "Hiroaki Ozaki",
                "Yuta Koreeda",
                "Sakata Masayuki"
            ],
            "abstract": "In this paper, we present our participation in SemEval-2020 Task-12 Subtask-A\n(English Language) which focuses on offensive language identification from\nnoisy labels. To this end, we developed a hybrid system with the BERT\nclassifier trained with tweets selected using Statistical Sampling Algorithm\n(SA) and Post-Processed (PP) using an offensive wordlist. Our developed system\nachieved 34 th position with Macro-averaged F1-score (Macro-F1) of 0.90913 over\nboth offensive and non-offensive classes. We further show comprehensive results\nand error analysis to assist future research in offensive language\nidentification with noisy labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00311v2",
            "date": "2020-05-01 10:57:02+00:00",
            "title": "Language (Re)modelling: Towards Embodied Language Understanding",
            "authors": [
                "Ronen Tamari",
                "Chen Shani",
                "Tom Hope",
                "Miriam R. L. Petruck",
                "Omri Abend",
                "Dafna Shahaf"
            ],
            "abstract": "While natural language understanding (NLU) is advancing rapidly, today's\ntechnology differs from human-like language understanding in fundamental ways,\nnotably in its inferior efficiency, interpretability, and generalization. This\nwork proposes an approach to representation and learning based on the tenets of\nembodied cognitive linguistics (ECL). According to ECL, natural language is\ninherently executable (like programming languages), driven by mental simulation\nand metaphoric mappings over hierarchical compositions of structures and\nschemata learned through embodied interaction. This position paper argues that\nthe use of grounding by metaphoric inference and simulation will greatly\nbenefit NLU systems, and proposes a system architecture along with a roadmap\ntowards realizing this vision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00318v1",
            "date": "2020-05-01 11:29:23+00:00",
            "title": "Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi",
            "authors": [
                "Benjamin Muller",
                "Benoit Sagot",
                "Djam\u00e9 Seddah"
            ],
            "abstract": "Building natural language processing systems for non standardized and low\nresource languages is a difficult challenge. The recent success of large-scale\nmultilingual pretrained language models provides new modeling tools to tackle\nthis. In this work, we study the ability of multilingual language models to\nprocess an unseen dialect. We take user generated North-African Arabic as our\ncase study, a resource-poor dialectal variety of Arabic with frequent\ncode-mixing with French and written in Arabizi, a non-standardized\ntransliteration of Arabic to Latin script. Focusing on two tasks,\npart-of-speech tagging and dependency parsing, we show in zero-shot and\nunsupervised adaptation scenarios that multilingual language models are able to\ntransfer to such an unseen dialect, specifically in two extreme cases: (i)\nacross scripts, using Modern Standard Arabic as a source language, and (ii)\nfrom a distantly related language, unseen during pretraining, namely Maltese.\nOur results constitute the first successful transfer experiments on this\ndialect, paving thus the way for the development of an NLP ecosystem for\nresource-scarce, non-standardized and highly variable vernacular languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00352v2",
            "date": "2020-05-01 12:54:30+00:00",
            "title": "MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases",
            "authors": [
                "Louis Martin",
                "Angela Fan",
                "\u00c9ric de la Clergerie",
                "Antoine Bordes",
                "Beno\u00eet Sagot"
            ],
            "abstract": "Progress in sentence simplification has been hindered by a lack of labeled\nparallel simplification data, particularly in languages other than English. We\nintroduce MUSS, a Multilingual Unsupervised Sentence Simplification system that\ndoes not require labeled simplification data. MUSS uses a novel approach to\nsentence simplification that trains strong models using sentence-level\nparaphrase data instead of proper simplification data. These models leverage\nunsupervised pretraining and controllable generation mechanisms to flexibly\nadjust attributes such as length and lexical complexity at inference time. We\nfurther present a method to mine such paraphrase data in any language from\nCommon Crawl using semantic sentence embeddings, thus removing the need for\nlabeled data. We evaluate our approach on English, French, and Spanish\nsimplification benchmarks and closely match or outperform the previous best\nsupervised results, despite not using any labeled simplification data. We push\nthe state of the art further by incorporating labeled simplification data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00357v5",
            "date": "2020-05-01 13:05:23+00:00",
            "title": "Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research",
            "authors": [
                "Soujanya Poria",
                "Devamanyu Hazarika",
                "Navonil Majumder",
                "Rada Mihalcea"
            ],
            "abstract": "Sentiment analysis as a field has come a long way since it was first\nintroduced as a task nearly 20 years ago. It has widespread commercial\napplications in various domains like marketing, risk management, market\nresearch, and politics, to name a few. Given its saturation in specific\nsubtasks -- such as sentiment polarity classification -- and datasets, there is\nan underlying perception that this field has reached its maturity. In this\narticle, we discuss this perception by pointing out the shortcomings and\nunder-explored, yet key aspects of this field that are necessary to attain true\nsentiment understanding. We analyze the significant leaps responsible for its\ncurrent relevance. Further, we attempt to chart a possible course for this\nfield that covers many overlooked and unanswered questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00436v1",
            "date": "2020-05-01 15:14:22+00:00",
            "title": "Bipartite Flat-Graph Network for Nested Named Entity Recognition",
            "authors": [
                "Ying Luo",
                "Hai Zhao"
            ],
            "abstract": "In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for\nnested named entity recognition (NER), which contains two subgraph modules: a\nflat NER module for outermost entities and a graph module for all the entities\nlocated in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional\nnetwork (GCN) are adopted to jointly learn flat entities and their inner\ndependencies. Different from previous models, which only consider the\nunidirectional delivery of information from innermost layers to outer ones (or\noutside-to-inside), our model effectively captures the bidirectional\ninteraction between them. We first use the entities recognized by the flat NER\nmodule to construct an entity graph, which is fed to the next graph module. The\nricher representation learned from graph module carries the dependencies of\ninner entities and can be exploited to improve outermost entity predictions.\nExperimental results on three standard nested NER datasets demonstrate that our\nBiFlaG outperforms previous state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.00446v2",
            "date": "2020-05-01 15:28:43+00:00",
            "title": "Defense of Word-level Adversarial Attacks via Random Substitution Encoding",
            "authors": [
                "Zhaoyang Wang",
                "Hongtao Wang"
            ],
            "abstract": "The adversarial attacks against deep neural networks on computer vision tasks\nhave spawned many new technologies that help protect models from avoiding false\npredictions. Recently, word-level adversarial attacks on deep models of Natural\nLanguage Processing (NLP) tasks have also demonstrated strong power, e.g.,\nfooling a sentiment classification neural network to make wrong decisions.\nUnfortunately, few previous literatures have discussed the defense of such\nword-level synonym substitution based attacks since they are hard to be\nperceived and detected. In this paper, we shed light on this problem and\npropose a novel defense framework called Random Substitution Encoding (RSE),\nwhich introduces a random substitution encoder into the training process of\noriginal neural networks. Extensive experiments on text classification tasks\ndemonstrate the effectiveness of our framework on defense of word-level\nadversarial attacks, under various base and attack models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00456v1",
            "date": "2020-05-01 15:50:50+00:00",
            "title": "USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation",
            "authors": [
                "Shikib Mehri",
                "Maxine Eskenazi"
            ],
            "abstract": "The lack of meaningful automatic evaluation metrics for dialog has impeded\nopen-domain dialog research. Standard language generation metrics have been\nshown to be ineffective for evaluating dialog models. To this end, this paper\npresents USR, an UnSupervised and Reference-free evaluation metric for dialog.\nUSR is a reference-free metric that trains unsupervised models to measure\nseveral desirable qualities of dialog. USR is shown to strongly correlate with\nhuman judgment on both Topical-Chat (turn-level: 0.42, system-level: 1.0) and\nPersonaChat (turn-level: 0.48 and system-level: 1.0). USR additionally produces\ninterpretable measures for several desirable properties of dialog.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00468v1",
            "date": "2020-05-01 16:03:37+00:00",
            "title": "Automatic Discourse Segmentation: Review and Perspectives",
            "authors": [
                "Iria da Cunha",
                "Juan-Manuel Torres-Moreno"
            ],
            "abstract": "Multilingual discourse parsing is a very prominent research topic. The first\nstage for discourse parsing is discourse segmentation. The study reported in\nthis article addresses a review of two on-line available discourse segmenters\n(for English and Portuguese). We evaluate the possibility of developing similar\ndiscourse segmenters for Spanish, French and African languages.",
            "journal ref": "International Workshop on African Human Language Technologies.\n  17-20 Jan 2010",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.00480v1",
            "date": "2020-05-01 16:43:06+00:00",
            "title": "Knowledge Base Inference for Regular Expression Queries",
            "authors": [
                "Vaibhav Adlakha",
                "Parth Shah",
                "Srikanta Bedathur",
                "Mausam"
            ],
            "abstract": "Two common types of tasks on Knowledge Bases have been studied -- single link\nprediction (Knowledge Base Completion) and path query answering. However, our\nanalysis of user queries on a real-world knowledge base reveals that a\nsignificant fraction of queries specify paths using regular expressions(regex).\nSuch regex queries cannot be handled by any of the existing link prediction or\npath query answering models. In response, we present Regex Query Answering, the\nnovel task of answering regex queries on incomplete KBs. We contribute two\ndatasets for the task, including one where test queries are harvested from\nactual user querylogs. We train baseline neural models for our new task and\npropose novel ways to handle disjunction and Kleene plus regex operators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00496v2",
            "date": "2020-05-01 17:12:20+00:00",
            "title": "Structured Tuning for Semantic Role Labeling",
            "authors": [
                "Tao Li",
                "Parth Anand Jawale",
                "Martha Palmer",
                "Vivek Srikumar"
            ],
            "abstract": "Recent neural network-driven semantic role labeling (SRL) systems have shown\nimpressive improvements in F1 scores. These improvements are due to expressive\ninput representations, which, at least at the surface, are orthogonal to\nknowledge-rich constrained decoding mechanisms that helped linear SRL models.\nIntroducing the benefits of structure to inform neural models presents a\nmethodological challenge. In this paper, we present a structured tuning\nframework to improve models using softened constraints only at training time.\nOur framework leverages the expressiveness of neural networks and provides\nsupervision with structured loss components. We start with a strong baseline\n(RoBERTa) to validate the impact of our approach, and show that our framework\noutperforms the baseline by learning to comply with declarative constraints.\nAdditionally, our experiments with smaller training sizes show that we can\nachieve consistent improvements under low-resource scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00524v1",
            "date": "2020-05-01 17:56:01+00:00",
            "title": "Why Overfitting Isn't Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries",
            "authors": [
                "Mozhi Zhang",
                "Yoshinari Fujinuma",
                "Michael J. Paul",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Cross-lingual word embeddings (CLWE) are often evaluated on bilingual lexicon\ninduction (BLI). Recent CLWE methods use linear projections, which underfit the\ntraining dictionary, to generalize on BLI. However, underfitting can hinder\ngeneralization to other downstream tasks that rely on words from the training\ndictionary. We address this limitation by retrofitting CLWE to the training\ndictionary, which pulls training translation pairs closer in the embedding\nspace and overfits the training dictionary. This simple post-processing step\noften improves accuracy on two downstream tasks, despite lowering BLI test\naccuracy. We also retrofit to both the training dictionary and a synthetic\ndictionary induced from CLWE, which sometimes generalizes even better on\ndownstream tasks. Our results confirm the importance of fully exploiting\ntraining dictionary in downstream tasks and explains why BLI is a flawed CLWE\nevaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00561v2",
            "date": "2020-05-01 18:24:42+00:00",
            "title": "When BERT Plays the Lottery, All Tickets Are Winning",
            "authors": [
                "Sai Prasanna",
                "Anna Rogers",
                "Anna Rumshisky"
            ],
            "abstract": "Large Transformer-based models were shown to be reducible to a smaller number\nof self-attention heads and layers. We consider this phenomenon from the\nperspective of the lottery ticket hypothesis, using both structured and\nmagnitude pruning. For fine-tuned BERT, we show that (a) it is possible to find\nsubnetworks achieving performance that is comparable with that of the full\nmodel, and (b) similarly-sized subnetworks sampled from the rest of the model\nperform worse. Strikingly, with structured pruning even the worst possible\nsubnetworks remain highly trainable, indicating that most pre-trained BERT\nweights are potentially useful. We also study the \"good\" subnetworks to see if\ntheir success can be attributed to superior linguistic knowledge, but find them\nunstable, and not explained by meaningful self-attention patterns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00572v1",
            "date": "2020-05-01 19:00:57+00:00",
            "title": "Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition",
            "authors": [
                "Hu Hu",
                "Rui Zhao",
                "Jinyu Li",
                "Liang Lu",
                "Yifan Gong"
            ],
            "abstract": "Recently, the recurrent neural network transducer (RNN-T) architecture has\nbecome an emerging trend in end-to-end automatic speech recognition research\ndue to its advantages of being capable for online streaming speech recognition.\nHowever, RNN-T training is made difficult by the huge memory requirements, and\ncomplicated neural structure. A common solution to ease the RNN-T training is\nto employ connectionist temporal classification (CTC) model along with RNN\nlanguage model (RNNLM) to initialize the RNN-T parameters. In this work, we\nconversely leverage external alignments to seed the RNN-T model. Two different\npre-training solutions are explored, referred to as encoder pre-training, and\nwhole-network pre-training respectively. Evaluated on Microsoft 65,000 hours\nanonymized production data with personally identifiable information removed,\nour proposed methods can obtain significant improvement. In particular, the\nencoder pre-training solution achieved a 10% and a 8% relative word error rate\nreduction when compared with random initialization and the widely used\nCTC+RNNLM initialization strategy, respectively. Our solutions also\nsignificantly reduce the RNN-T model latency from the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00581v1",
            "date": "2020-05-01 19:58:56+00:00",
            "title": "Multi-scale Transformer Language Models",
            "authors": [
                "Sandeep Subramanian",
                "Ronan Collobert",
                "Marc'Aurelio Ranzato",
                "Y-Lan Boureau"
            ],
            "abstract": "We investigate multi-scale transformer language models that learn\nrepresentations of text at multiple scales, and present three different\narchitectures that have an inductive bias to handle the hierarchical nature of\nlanguage. Experiments on large-scale language modeling benchmarks empirically\ndemonstrate favorable likelihood vs memory footprint trade-offs, e.g. we show\nthat it is possible to train a hierarchical variant with 30 layers that has 23%\nsmaller memory footprint and better perplexity, compared to a vanilla\ntransformer with less than half the number of layers, on the Toronto\nBookCorpus. We analyze the advantages of learned representations at multiple\nscales in terms of memory footprint, compute time, and perplexity, which are\nparticularly appealing given the quadratic scaling of transformers' run time\nand memory usage with respect to sequence length.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00583v1",
            "date": "2020-05-01 20:01:39+00:00",
            "title": "Learning an Unreferenced Metric for Online Dialogue Evaluation",
            "authors": [
                "Koustuv Sinha",
                "Prasanna Parthasarathi",
                "Jasmine Wang",
                "Ryan Lowe",
                "William L. Hamilton",
                "Joelle Pineau"
            ],
            "abstract": "Evaluating the quality of a dialogue interaction between two agents is a\ndifficult task, especially in open-domain chit-chat style dialogue. There have\nbeen recent efforts to develop automatic dialogue evaluation metrics, but most\nof them do not generalize to unseen datasets and/or need a human-generated\nreference response during inference, making it infeasible for online\nevaluation. Here, we propose an unreferenced automated evaluation metric that\nuses large pre-trained language models to extract latent representations of\nutterances, and leverages the temporal transitions that exist between them. We\nshow that our model achieves higher correlation with human annotations in an\nonline setting, while not requiring true responses for comparison during\ninference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00619v5",
            "date": "2020-05-01 21:28:28+00:00",
            "title": "Probing Contextual Language Models for Common Ground with Visual Representations",
            "authors": [
                "Gabriel Ilharco",
                "Rowan Zellers",
                "Ali Farhadi",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "The success of large-scale contextual language models has attracted great\ninterest in probing what is encoded in their representations. In this work, we\nconsider a new question: to what extent contextual representations of concrete\nnouns are aligned with corresponding visual representations? We design a\nprobing model that evaluates how effective are text-only representations in\ndistinguishing between matching and non-matching visual representations. Our\nfindings show that language representations alone provide a strong signal for\nretrieving image patches from the correct object categories. Moreover, they are\neffective in retrieving specific instances of image patches; textual context\nplays an important role in this process. Visually grounded language models\nslightly outperform text-only language models in instance retrieval, but\ngreatly under-perform humans. We hope our analyses inspire future research in\nunderstanding and improving the visual capabilities of language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00624v2",
            "date": "2020-05-01 21:42:32+00:00",
            "title": "Minimally Supervised Categorization of Text with Metadata",
            "authors": [
                "Yu Zhang",
                "Yu Meng",
                "Jiaxin Huang",
                "Frank F. Xu",
                "Xuan Wang",
                "Jiawei Han"
            ],
            "abstract": "Document categorization, which aims to assign a topic label to each document,\nplays a fundamental role in a wide variety of applications. Despite the success\nof existing studies in conventional supervised document classification, they\nare less concerned with two real problems: (1) \\textit{the presence of\nmetadata}: in many domains, text is accompanied by various additional\ninformation such as authors and tags. Such metadata serve as compelling topic\nindicators and should be leveraged into the categorization framework; (2)\n\\textit{label scarcity}: labeled training samples are expensive to obtain in\nsome cases, where categorization needs to be performed using only a small set\nof annotated data. In recognition of these two challenges, we propose\n\\textsc{MetaCat}, a minimally supervised framework to categorize text with\nmetadata. Specifically, we develop a generative process describing the\nrelationships between words, documents, labels, and metadata. Guided by the\ngenerative model, we embed text and metadata into the same semantic space to\nencode heterogeneous signals. Then, based on the same generative process, we\nsynthesize training samples to address the bottleneck of label scarcity. We\nconduct a thorough evaluation on a wide range of datasets. Experimental results\nprove the effectiveness of \\textsc{MetaCat} over many competitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00636v3",
            "date": "2020-05-01 22:14:16+00:00",
            "title": "We Need to Talk About Random Splits",
            "authors": [
                "Anders S\u00f8gaard",
                "Sebastian Ebert",
                "Jasmijn Bastings",
                "Katja Filippova"
            ],
            "abstract": "Gorman and Bedrick (2019) argued for using random splits rather than standard\nsplits in NLP experiments. We argue that random splits, like standard splits,\nlead to overly optimistic performance estimates. We can also split data in\nbiased or adversarial ways, e.g., training on short sentences and evaluating on\nlong ones. Biased sampling has been used in domain adaptation to simulate\nreal-world drift; this is known as the covariate shift assumption. In NLP,\nhowever, even worst-case splits, maximizing bias, often under-estimate the\nerror observed on new samples of in-domain data, i.e., the data that models\nshould minimally generalize to at test time. This invalidates the covariate\nshift assumption. Instead of using multiple random splits, future benchmarks\nshould ideally include multiple, independent test sets instead; if infeasible,\nwe argue that multiple biased splits leads to more realistic performance\nestimates than multiple random splits.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00642v2",
            "date": "2020-05-01 22:59:56+00:00",
            "title": "Spatial Dependency Parsing for Semi-Structured Document Information Extraction",
            "authors": [
                "Wonseok Hwang",
                "Jinyeong Yim",
                "Seunghyun Park",
                "Sohee Yang",
                "Minjoon Seo"
            ],
            "abstract": "Information Extraction (IE) for semi-structured document images is often\napproached as a sequence tagging problem by classifying each recognized input\ntoken into one of the IOB (Inside, Outside, and Beginning) categories. However,\nsuch problem setup has two inherent limitations that (1) it cannot easily\nhandle complex spatial relationships and (2) it is not suitable for highly\nstructured information, which are nevertheless frequently observed in\nreal-world document images. To tackle these issues, we first formulate the IE\ntask as spatial dependency parsing problem that focuses on the relationship\namong text segment nodes in the documents. Under this setup, we then propose\nSPADE (SPAtial DEpendency parser) that models highly complex spatial\nrelationships and an arbitrary number of information layers in the documents in\nan end-to-end manner. We evaluate it on various kinds of documents such as\nreceipts, name cards, forms, and invoices, and show that it achieves a similar\nor better performance compared to strong baselines including BERT-based IOB\ntaggger, with up to 37.7% improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00644v1",
            "date": "2020-05-01 23:05:55+00:00",
            "title": "Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing",
            "authors": [
                "Wonseok Hwang",
                "Jinyeong Yim",
                "Seunghyun Park",
                "Minjoon Seo"
            ],
            "abstract": "Deep learning approaches to semantic parsing require a large amount of\nlabeled data, but annotating complex logical forms is costly. Here, we propose\nSyntactic Question Abstraction and Retrieval (SQAR), a method to build a neural\nsemantic parser that translates a natural language (NL) query to a SQL logical\nform (LF) with less than 1,000 annotated examples. SQAR first retrieves a\nlogical pattern from the train data by computing the similarity between NL\nqueries and then grounds a lexical information on the retrieved pattern in\norder to generate the final LF. We validate SQAR by training models using\nvarious small subsets of WikiSQL train data achieving up to 4.9% higher LF\naccuracy compared to the previous state-of-the-art models on WikiSQL test set.\nWe also show that by using query-similarity to retrieve logical pattern, SQAR\ncan leverage a paraphrasing dataset achieving up to 5.9% higher LF accuracy\ncompared to the case where SQAR is trained by using only WikiSQL data. In\ncontrast to a simple pattern classification approach, SQAR can generate unseen\nlogical patterns upon the addition of new examples without re-training the\nmodel. We also discuss an ideal way to create cost efficient and robust train\ndatasets when the data distribution can be approximated under a data-hungry\nsetting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00646v2",
            "date": "2020-05-01 23:10:26+00:00",
            "title": "Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering",
            "authors": [
                "Yanlin Feng",
                "Xinyue Chen",
                "Bill Yuchen Lin",
                "Peifeng Wang",
                "Jun Yan",
                "Xiang Ren"
            ],
            "abstract": "Existing work on augmenting question answering (QA) models with external\nknowledge (e.g., knowledge graphs) either struggle to model multi-hop relations\nefficiently, or lack transparency into the model's prediction rationale. In\nthis paper, we propose a novel knowledge-aware approach that equips pre-trained\nlanguage models (PTLMs) with a multi-hop relational reasoning module, named\nmulti-hop graph relation network (MHGRN). It performs multi-hop,\nmulti-relational reasoning over subgraphs extracted from external knowledge\ngraphs. The proposed reasoning module unifies path-based reasoning methods and\ngraph neural networks to achieve better interpretability and scalability. We\nalso empirically show its effectiveness and scalability on CommonsenseQA and\nOpenbookQA datasets, and interpret its behaviors with case studies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00652v3",
            "date": "2020-05-01 23:26:41+00:00",
            "title": "An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction",
            "authors": [
                "Bhargavi Paranjape",
                "Mandar Joshi",
                "John Thickstun",
                "Hannaneh Hajishirzi",
                "Luke Zettlemoyer"
            ],
            "abstract": "Decisions of complex language understanding models can be rationalized by\nlimiting their inputs to a relevant subsequence of the original text. A\nrationale should be as concise as possible without significantly degrading task\nperformance, but this balance can be difficult to achieve in practice. In this\npaper, we show that it is possible to better manage this trade-off by\noptimizing a bound on the Information Bottleneck (IB) objective. Our fully\nunsupervised approach jointly learns an explainer that predicts sparse binary\nmasks over sentences, and an end-task predictor that considers only the\nextracted rationale. Using IB, we derive a learning objective that allows\ndirect control of mask sparsity levels through a tunable sparse prior.\nExperiments on ERASER benchmark tasks demonstrate significant gains over\nnorm-minimization techniques for both task performance and agreement with human\nrationales. Furthermore, we find that in the semi-supervised setting, a modest\namount of gold rationales (25% of training examples) closes the gap with a\nmodel that uses the full input.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00660v1",
            "date": "2020-05-02 00:08:42+00:00",
            "title": "GenericsKB: A Knowledge Base of Generic Statements",
            "authors": [
                "Sumithra Bhakthavatsalam",
                "Chloe Anastasiades",
                "Peter Clark"
            ],
            "abstract": "We present a new resource for the NLP community, namely a large (3.5M+\nsentence) knowledge base of *generic statements*, e.g., \"Trees remove carbon\ndioxide from the atmosphere\", collected from multiple corpora. This is the\nfirst large resource to contain *naturally occurring* generic sentences, as\nopposed to extracted or crowdsourced triples, and thus is rich in high-quality,\ngeneral, semantically complete statements. All GenericsKB sentences are\nannotated with their topical term, surrounding context (sentences), and a\n(learned) confidence. We also release GenericsKB-Best (1M+ sentences),\ncontaining the best-quality generics in GenericsKB augmented with selected,\nsynthesized generics from WordNet and ConceptNet. In tests on two existing\ndatasets requiring multihop reasoning (OBQA and QASC), we find using GenericsKB\ncan result in higher scores and better explanations than using a much larger\ncorpus. This demonstrates that GenericsKB can be a useful resource for NLP\napplications, as well as providing data for linguistic studies of generics and\ntheir semantics. GenericsKB is available at\nhttps://allenai.org/data/genericskb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00669v1",
            "date": "2020-05-02 00:39:09+00:00",
            "title": "Contrastive Self-Supervised Learning for Commonsense Reasoning",
            "authors": [
                "Tassilo Klein",
                "Moin Nabi"
            ],
            "abstract": "We propose a self-supervised method to solve Pronoun Disambiguation and\nWinograd Schema Challenge problems. Our approach exploits the characteristic\nstructure of training corpora related to so-called \"trigger\" words, which are\nresponsible for flipping the answer in pronoun disambiguation. We achieve such\ncommonsense reasoning by constructing pair-wise contrastive auxiliary\npredictions. To this end, we leverage a mutual exclusive loss regularized by a\ncontrastive margin. Our architecture is based on the recently introduced\ntransformer networks, BERT, that exhibits strong performance on many NLP\nbenchmarks. Empirical results show that our method alleviates the limitation of\ncurrent supervised approaches for commonsense reasoning. This study opens up\navenues for exploiting inexpensive self-supervision to achieve performance gain\nin commonsense reasoning tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00683v2",
            "date": "2020-05-02 02:47:02+00:00",
            "title": "Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models",
            "authors": [
                "Bill Yuchen Lin",
                "Seyeon Lee",
                "Rahul Khanna",
                "Xiang Ren"
            ],
            "abstract": "Recent works show that pre-trained language models (PTLMs), such as BERT,\npossess certain commonsense and factual knowledge. They suggest that it is\npromising to use PTLMs as \"neural knowledge bases\" via predicting masked words.\nSurprisingly, we find that this may not work for numerical commonsense\nknowledge (e.g., a bird usually has two legs). In this paper, we investigate\nwhether and to what extent we can induce numerical commonsense knowledge from\nPTLMs as well as the robustness of this process. To study this, we introduce a\nnovel probing task with a diagnostic dataset, NumerSense, containing 13.6k\nmasked-word-prediction probes (10.5k for fine-tuning and 3.1k for testing). Our\nanalysis reveals that: (1) BERT and its stronger variant RoBERTa perform poorly\non the diagnostic dataset prior to any fine-tuning; (2) fine-tuning with\ndistant supervision brings some improvement; (3) the best supervised model\nstill performs poorly as compared to human performance (54.06% vs 96.3% in\naccuracy).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00689v3",
            "date": "2020-05-02 03:30:49+00:00",
            "title": "An Imitation Game for Learning Semantic Parsers from User Interaction",
            "authors": [
                "Ziyu Yao",
                "Yiqi Tang",
                "Wen-tau Yih",
                "Huan Sun",
                "Yu Su"
            ],
            "abstract": "Despite the widely successful applications, bootstrapping and fine-tuning\nsemantic parsers are still a tedious process with challenges such as costly\ndata annotation and privacy risks. In this paper, we suggest an alternative,\nhuman-in-the-loop methodology for learning semantic parsers directly from\nusers. A semantic parser should be introspective of its uncertainties and\nprompt for user demonstration when uncertain. In doing so it also gets to\nimitate the user behavior and continue improving itself autonomously with the\nhope that eventually it may become as good as the user in interpreting their\nquestions. To combat the sparsity of demonstration, we propose a novel\nannotation-efficient imitation learning algorithm, which iteratively collects\nnew datasets by mixing demonstrated states and confident predictions and\nre-trains the semantic parser in a Dataset Aggregation fashion (Ross et al.,\n2011). We provide a theoretical analysis of its cost bound and also empirically\ndemonstrate its promising performance on the text-to-SQL problem. Code will be\navailable at https://github.com/sunlab-osu/MISP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00691v2",
            "date": "2020-05-02 03:53:21+00:00",
            "title": "Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering",
            "authors": [
                "Peifeng Wang",
                "Nanyun Peng",
                "Filip Ilievski",
                "Pedro Szekely",
                "Xiang Ren"
            ],
            "abstract": "Commonsense question answering (QA) requires background knowledge which is\nnot explicitly stated in a given context. Prior works use commonsense knowledge\ngraphs (KGs) to obtain this knowledge for reasoning. However, relying entirely\non these KGs may not suffice, considering their limited coverage and the\ncontextual dependence of their knowledge. In this paper, we augment a general\ncommonsense QA framework with a knowledgeable path generator. By extrapolating\nover existing paths in a KG with a state-of-the-art language model, our\ngenerator learns to connect a pair of entities in text with a dynamic, and\npotentially novel, multi-hop relational path. Such paths can provide structured\nevidence for solving commonsense questions without fine-tuning the path\ngenerator. Experiments on two datasets show the superiority of our method over\nprevious works which fully rely on knowledge from KGs (with up to 6%\nimprovement in accuracy), across various amounts of training data. Further\nevaluation suggests that the generated paths are typically interpretable,\nnovel, and relevant to the task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00700v3",
            "date": "2020-05-02 04:42:14+00:00",
            "title": "UnifiedQA: Crossing Format Boundaries With a Single QA System",
            "authors": [
                "Daniel Khashabi",
                "Sewon Min",
                "Tushar Khot",
                "Ashish Sabharwal",
                "Oyvind Tafjord",
                "Peter Clark",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Question answering (QA) tasks have been posed using a variety of formats,\nsuch as extractive span selection, multiple choice, etc. This has led to\nformat-specialized models, and even to an implicit division in the QA\ncommunity. We argue that such boundaries are artificial and perhaps\nunnecessary, given the reasoning abilities we seek to teach are not governed by\nthe format. As evidence, we use the latest advances in language modeling to\nbuild a single pre-trained QA model, UnifiedQA, that performs surprisingly well\nacross 17 QA datasets spanning 4 diverse formats. UnifiedQA performs on par\nwith 9 different models that were trained on individual datasets themselves.\nEven when faced with 12 unseen datasets of observed formats, UnifiedQA performs\nsurprisingly well, showing strong generalization from its out-of-format\ntraining data. Finally, simply fine-tuning this pre-trained QA model into\nspecialized models results in a new state of the art on 6 datasets,\nestablishing UnifiedQA as a strong starting point for building QA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00706v2",
            "date": "2020-05-02 05:15:20+00:00",
            "title": "A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos",
            "authors": [
                "Frank F. Xu",
                "Lei Ji",
                "Botian Shi",
                "Junyi Du",
                "Graham Neubig",
                "Yonatan Bisk",
                "Nan Duan"
            ],
            "abstract": "Watching instructional videos are often used to learn about procedures. Video\ncaptioning is one way of automatically collecting such knowledge. However, it\nprovides only an indirect, overall evaluation of multimodal models with no\nfiner-grained quantitative measure of what they have learned. We propose\ninstead, a benchmark of structured procedural knowledge extracted from cooking\nvideos. This work is complementary to existing tasks, but requires models to\nproduce interpretable structured knowledge in the form of verb-argument tuples.\nOur manually annotated open-vocabulary resource includes 356 instructional\ncooking videos and 15,523 video clip/sentence-level annotations. Our analysis\nshows that the proposed task is challenging and standard modeling approaches\nlike unsupervised segmentation, semantic role labeling, and visual action\ndetection perform poorly when forced to predict every action of a procedure in\na structured form.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00730v2",
            "date": "2020-05-02 07:03:06+00:00",
            "title": "ESPRIT: Explaining Solutions to Physical Reasoning Tasks",
            "authors": [
                "Nazneen Fatema Rajani",
                "Rui Zhang",
                "Yi Chern Tan",
                "Stephan Zheng",
                "Jeremy Weiss",
                "Aadit Vyas",
                "Abhijit Gupta",
                "Caiming XIong",
                "Richard Socher",
                "Dragomir Radev"
            ],
            "abstract": "Neural networks lack the ability to reason about qualitative physics and so\ncannot generalize to scenarios and tasks unseen during training. We propose\nESPRIT, a framework for commonsense reasoning about qualitative physics in\nnatural language that generates interpretable descriptions of physical events.\nWe use a two-step approach of first identifying the pivotal physical events in\nan environment and then generating natural language descriptions of those\nevents using a data-to-text approach. Our framework learns to generate\nexplanations of how the physical simulation will causally evolve so that an\nagent or a human can easily reason about a solution using those interpretable\ndescriptions. Human evaluations indicate that ESPRIT produces crucial\nfine-grained details and has high coverage of physical concepts compared to\neven human annotations. Dataset, code and documentation are available at\nhttps://github.com/salesforce/esprit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00806v3",
            "date": "2020-05-02 11:54:34+00:00",
            "title": "Teaching Machine Comprehension with Compositional Explanations",
            "authors": [
                "Qinyuan Ye",
                "Xiao Huang",
                "Elizabeth Boschee",
                "Xiang Ren"
            ],
            "abstract": "Advances in machine reading comprehension (MRC) rely heavily on the\ncollection of large scale human-annotated examples in the form of (question,\nparagraph, answer) triples. In contrast, humans are typically able to\ngeneralize with only a few examples, relying on deeper underlying world\nknowledge, linguistic sophistication, and/or simply superior deductive powers.\nIn this paper, we focus on \"teaching\" machines reading comprehension, using a\nsmall number of semi-structured explanations that explicitly inform machines\nwhy answer spans are correct. We extract structured variables and rules from\nexplanations and compose neural module teachers that annotate instances for\ntraining downstream MRC models. We use learnable neural modules and soft logic\nto handle linguistic variation and overcome sparse coverage; the modules are\njointly optimized with the MRC model to improve final performance. On the SQuAD\ndataset, our proposed method achieves 70.14% F1 score with supervision from 26\nexplanations, comparable to plain supervised learning using 1,100 labeled\ninstances, yielding a 12x speed up.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00820v2",
            "date": "2020-05-02 12:46:28+00:00",
            "title": "Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing",
            "authors": [
                "Clara Meister",
                "Elizabeth Salesky",
                "Ryan Cotterell"
            ],
            "abstract": "Prior work has explored directly regularizing the output distributions of\nprobabilistic models to alleviate peaky (i.e. over-confident) predictions, a\ncommon sign of overfitting. This class of techniques, of which label smoothing\nis one, has a connection to entropy regularization. Despite the consistent\nsuccess of label smoothing across architectures and data sets in language\ngeneration tasks, two problems remain open: (1) there is little understanding\nof the underlying effects entropy regularizers have on models, and (2) the full\nspace of entropy regularization techniques is largely unexplored. We introduce\na parametric family of entropy regularizers, which includes label smoothing as\na special case, and use it to gain a better understanding of the relationship\nbetween the entropy of a model and its performance on language generation\ntasks. We also find that variance in model performance can be explained largely\nby the resulting entropy of the model. Lastly, we find that label smoothing\nprovably does not allow for sparsity in an output distribution, an undesirable\nproperty for language generation models, and therefore advise the use of other\nentropy regularization methods in its place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00850v2",
            "date": "2020-05-02 15:06:47+00:00",
            "title": "ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation",
            "authors": [
                "Lifu Tu",
                "Richard Yuanzhe Pang",
                "Sam Wiseman",
                "Kevin Gimpel"
            ],
            "abstract": "We propose to train a non-autoregressive machine translation model to\nminimize the energy defined by a pretrained autoregressive model. In\nparticular, we view our non-autoregressive translation system as an inference\nnetwork (Tu and Gimpel, 2018) trained to minimize the autoregressive teacher\nenergy. This contrasts with the popular approach of training a\nnon-autoregressive model on a distilled corpus consisting of the beam-searched\noutputs of such a teacher model. Our approach, which we call ENGINE\n(ENerGy-based Inference NEtworks), achieves state-of-the-art non-autoregressive\nresults on the IWSLT 2014 DE-EN and WMT 2016 RO-EN datasets, approaching the\nperformance of autoregressive models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00879v1",
            "date": "2020-05-02 16:23:47+00:00",
            "title": "Single Model Ensemble using Pseudo-Tags and Distinct Vectors",
            "authors": [
                "Ryosuke Kuwabara",
                "Jun Suzuki",
                "Hideki Nakayama"
            ],
            "abstract": "Model ensemble techniques often increase task performance in neural networks;\nhowever, they require increased time, memory, and management effort. In this\nstudy, we propose a novel method that replicates the effects of a model\nensemble with a single model. Our approach creates K-virtual models within a\nsingle parameter space using K-distinct pseudo-tags and K-distinct vectors.\nExperiments on text classification and sequence labeling tasks on several\ndatasets demonstrate that our method emulates or outperforms a traditional\nmodel ensemble with 1/K-times fewer parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00908v1",
            "date": "2020-05-02 19:28:52+00:00",
            "title": "Clue: Cross-modal Coherence Modeling for Caption Generation",
            "authors": [
                "Malihe Alikhani",
                "Piyush Sharma",
                "Shengjie Li",
                "Radu Soricut",
                "Matthew Stone"
            ],
            "abstract": "We use coherence relations inspired by computational models of discourse to\nstudy the information needs and goals of image captioning. Using an annotation\nprotocol specifically devised for capturing image--caption coherence relations,\nwe annotate 10,000 instances from publicly-available image--caption pairs. We\nintroduce a new task for learning inferences in imagery and text, coherence\nrelation prediction, and show that these coherence annotations can be exploited\nto learn relation classifiers as an intermediary step, and also train\ncoherence-aware, controllable image captioning models. The results show a\ndramatic improvement in the consistency and quality of the generated captions\nwith respect to information needs specified via coherence relations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00912v1",
            "date": "2020-05-02 20:01:59+00:00",
            "title": "Examining Citations of Natural Language Processing Literature",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "We extracted information from the ACL Anthology (AA) and Google Scholar (GS)\nto examine trends in citations of NLP papers. We explore questions such as: how\nwell cited are papers of different types (journal articles, conference papers,\ndemo papers, etc.)? how well cited are papers from different areas of within\nNLP? etc. Notably, we show that only about 56\\% of the papers in AA are cited\nten or more times. CL Journal has the most cited papers, but its citation\ndominance has lessened in recent years. On average, long papers get almost\nthree times as many citations as short papers; and papers on sentiment\nclassification, anaphora resolution, and entity recognition have the highest\nmedian citations. The analyses presented here, and the associated dataset of\nNLP papers mapped to citations, have a number of uses including: understanding\nhow the field is growing and quantifying the impact of different types of\npapers.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association of\n  Computational Linguistics (ACL 2020). July 2020. Seattle, USA",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00932v3",
            "date": "2020-05-02 22:24:52+00:00",
            "title": "Improving Non-autoregressive Neural Machine Translation with Monolingual Data",
            "authors": [
                "Jiawei Zhou",
                "Phillip Keung"
            ],
            "abstract": "Non-autoregressive (NAR) neural machine translation is usually done via\nknowledge distillation from an autoregressive (AR) model. Under this framework,\nwe leverage large monolingual corpora to improve the NAR model's performance,\nwith the goal of transferring the AR model's generalization ability while\npreventing overfitting. On top of a strong NAR baseline, our experimental\nresults on the WMT14 En-De and WMT16 En-Ro news translation tasks confirm that\nmonolingual data augmentation consistently improves the performance of the NAR\nmodel to approach the teacher AR model's performance, yields comparable or\nbetter results than the best non-iterative NAR methods in the literature and\nhelps reduce overfitting in the training process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00944v1",
            "date": "2020-05-02 23:43:52+00:00",
            "title": "Understanding and Improving Information Transfer in Multi-Task Learning",
            "authors": [
                "Sen Wu",
                "Hongyang R. Zhang",
                "Christopher R\u00e9"
            ],
            "abstract": "We investigate multi-task learning approaches that use a shared feature\nrepresentation for all tasks. To better understand the transfer of task\ninformation, we study an architecture with a shared module for all tasks and a\nseparate output module for each task. We study the theory of this setting on\nlinear and ReLU-activated models. Our key observation is that whether or not\ntasks' data are well-aligned can significantly affect the performance of\nmulti-task learning. We show that misalignment between task data can cause\nnegative transfer (or hurt performance) and provide sufficient conditions for\npositive transfer. Inspired by the theoretical insights, we show that aligning\ntasks' embedding layers leads to performance gains for multi-task training and\ntransfer learning on the GLUE benchmark and sentiment analysis tasks; for\nexample, we obtain a 2.35% GLUE score average improvement on 5 GLUE tasks over\nBERT-LARGE using our alignment method. We also design an SVD-based task\nreweighting scheme and show that it improves the robustness of multi-task\ntraining on a multi-label image dataset.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2005.00950v1",
            "date": "2020-05-03 00:06:01+00:00",
            "title": "Extracting Entities and Topics from News and Connecting Criminal Records",
            "authors": [
                "Quang Pham",
                "Marija Stanojevic",
                "Zoran Obradovic"
            ],
            "abstract": "The goal of this paper is to summarize methodologies used in extracting\nentities and topics from a database of criminal records and from a database of\nnewspapers. Statistical models had successfully been used in studying the\ntopics of roughly 300,000 New York Times articles. In addition, these models\nhad also been used to successfully analyze entities related to people,\norganizations, and places (D Newman, 2006). Additionally, analytical\napproaches, especially in hotspot mapping, were used in some researches with an\naim to predict crime locations and circumstances in the future, and those\napproaches had been tested quite successfully (S Chainey, 2008). Based on the\ntwo above notions, this research was performed with the intention to apply data\nscience techniques in analyzing a big amount of data, selecting valuable\nintelligence, clustering violations depending on their types of crime, and\ncreating a crime graph that changes through time. In this research, the task\nwas to download criminal datasets from Kaggle and a collection of news articles\nfrom Kaggle and EAGER project databases, and then to merge these datasets into\none general dataset. The most important goal of this project was performing\nstatistical and natural language processing methods to extract entities and\ntopics as well as to group similar data points into correct clusters, in order\nto understand public data about U.S related crimes better.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.00962v2",
            "date": "2020-05-03 01:31:12+00:00",
            "title": "Gender Gap in Natural Language Processing Research: Disparities in Authorship and Citations",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "Disparities in authorship and citations across gender can have substantial\nadverse consequences not just on the disadvantaged genders, but also on the\nfield of study as a whole. Measuring gender gaps is a crucial step towards\naddressing them. In this work, we examine female first author percentages and\nthe citations to their papers in Natural Language Processing (1965 to 2019). We\ndetermine aggregate-level statistics using an existing manually curated\nauthor--gender list as well as first names strongly associated with a gender.\nWe find that only about 29% of first authors are female and only about 25% of\nlast authors are female. Notably, this percentage has not improved since the\nmid 2000s. We also show that, on average, female first authors are cited less\nthan male first authors, even when controlling for experience and area of\nresearch. Finally, we discuss the ethical considerations involved in automatic\ndemographic analysis.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association of\n  Computational Linguistics (ACL 2020). July 2020. Seattle, USA",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00965v1",
            "date": "2020-05-03 02:33:20+00:00",
            "title": "Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation",
            "authors": [
                "Tianlu Wang",
                "Xi Victoria Lin",
                "Nazneen Fatema Rajani",
                "Bryan McCann",
                "Vicente Ordonez",
                "Caiming Xiong"
            ],
            "abstract": "Word embeddings derived from human-generated corpora inherit strong gender\nbias which can be further amplified by downstream models. Some commonly adopted\ndebiasing approaches, including the seminal Hard Debias algorithm, apply\npost-processing procedures that project pre-trained word embeddings into a\nsubspace orthogonal to an inferred gender subspace. We discover that\nsemantic-agnostic corpus regularities such as word frequency captured by the\nword embeddings negatively impact the performance of these algorithms. We\npropose a simple but effective technique, Double Hard Debias, which purifies\nthe word embeddings against such corpus regularities prior to inferring and\nremoving the gender subspace. Experiments on three bias mitigation benchmarks\nshow that our approach preserves the distributional semantics of the\npre-trained word embeddings while reducing gender bias to a significantly\nlarger degree than prior approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.00979v1",
            "date": "2020-05-03 04:18:44+00:00",
            "title": "How Does Selective Mechanism Improve Self-Attention Networks?",
            "authors": [
                "Xinwei Geng",
                "Longyue Wang",
                "Xing Wang",
                "Bing Qin",
                "Ting Liu",
                "Zhaopeng Tu"
            ],
            "abstract": "Self-attention networks (SANs) with selective mechanism has produced\nsubstantial improvements in various NLP tasks by concentrating on a subset of\ninput words. However, the underlying reasons for their strong performance have\nnot been well explained. In this paper, we bridge the gap by assessing the\nstrengths of selective SANs (SSANs), which are implemented with a flexible and\nuniversal Gumbel-Softmax. Experimental results on several representative NLP\ntasks, including natural language inference, semantic role labelling, and\nmachine translation, show that SSANs consistently outperform the standard SANs.\nThrough well-designed probing experiments, we empirically validate that the\nimprovement of SSANs can be attributed in part to mitigating two commonly-cited\nweaknesses of SANs: word order encoding and structure modeling. Specifically,\nthe selective mechanism improves SANs by paying more attention to content words\nthat contribute to the meaning of the sentence. The code and data are released\nat https://github.com/xwgeng/SSAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01006v3",
            "date": "2020-05-03 06:48:35+00:00",
            "title": "An Accurate Model for Predicting the (Graded) Effect of Context in Word Similarity Based on Bert",
            "authors": [
                "Wei Bao",
                "Hongshu Che",
                "Jiandong Zhang"
            ],
            "abstract": "Natural Language Processing (NLP) has been widely used in the semantic\nanalysis in recent years. Our paper mainly discusses a methodology to analyze\nthe effect that context has on human perception of similar words, which is the\nthird task of SemEval 2020. We apply several methods in calculating the\ndistance between two embedding vector generated by Bidirectional Encoder\nRepresentation from Transformer (BERT). Our team will_go won the 1st place in\nFinnish language track of subtask1, the second place in English track of\nsubtask1.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01096v1",
            "date": "2020-05-03 14:28:28+00:00",
            "title": "Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence",
            "authors": [
                "Xiaoyu Shen",
                "Ernie Chang",
                "Hui Su",
                "Jie Zhou",
                "Dietrich Klakow"
            ],
            "abstract": "The neural attention model has achieved great success in data-to-text\ngeneration tasks. Though usually excelling at producing fluent text, it suffers\nfrom the problem of information missing, repetition and \"hallucination\". Due to\nthe black-box nature of the neural attention architecture, avoiding these\nproblems in a systematic way is non-trivial. To address this concern, we\npropose to explicitly segment target text into fragment units and align them\nwith their data correspondences. The segmentation and correspondence are\njointly learned as latent variables without any human annotations. We further\nimpose a soft statistical constraint to regularize the segmental granularity.\nThe resulting architecture maintains the same expressive power as neural\nattention models, while being able to generate fully interpretable outputs with\nseveral times less computational cost. On both E2E and WebNLG benchmarks, we\nshow the proposed model consistently outperforms its neural attention\ncounterparts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01151v1",
            "date": "2020-05-03 17:36:17+00:00",
            "title": "Let Me Choose: From Verbal Context to Font Selection",
            "authors": [
                "Amirreza Shirani",
                "Franck Dernoncourt",
                "Jose Echevarria",
                "Paul Asente",
                "Nedim Lipka",
                "Thamar Solorio"
            ],
            "abstract": "In this paper, we aim to learn associations between visual attributes of\nfonts and the verbal context of the texts they are typically applied to.\nCompared to related work leveraging the surrounding visual context, we choose\nto focus only on the input text as this can enable new applications for which\nthe text is the only visual element in the document. We introduce a new\ndataset, containing examples of different topics in social media posts and ads,\nlabeled through crowd-sourcing. Due to the subjective nature of the task,\nmultiple fonts might be perceived as acceptable for an input text, which makes\nthis problem challenging. To this end, we investigate different end-to-end\nmodels to learn label distributions on crowd-sourced data and capture\ninter-subjectivity across all annotations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01177v1",
            "date": "2020-05-03 20:08:39+00:00",
            "title": "Tailoring and Evaluating the Wikipedia for in-Domain Comparable Corpora Extraction",
            "authors": [
                "Cristina Espa\u00f1a-Bonet",
                "Alberto Barr\u00f3n-Cede\u00f1o",
                "Llu\u00eds M\u00e0rquez"
            ],
            "abstract": "We propose an automatic language-independent graph-based method to build\n\\`a-la-carte article collections on user-defined domains from the Wikipedia.\nThe core model is based on the exploration of the encyclopaedia's category\ngraph and can produce both monolingual and multilingual comparable collections.\nWe run thorough experiments to assess the quality of the obtained corpora in 10\nlanguages and 743 domains. According to an extensive manual evaluation, our\ngraph-based model outperforms a retrieval-based approach and reaches an average\nprecision of 84% on in-domain articles. As manual evaluations are costly, we\nintroduce the concept of \"domainness\" and design several automatic metrics to\naccount for the quality of the collections. Our best metric for domainness\nshows a strong correlation with the human-judged precision, representing a\nreasonable automatic alternative to assess the quality of domain-specific\ncorpora. We release the WikiTailor toolkit with the implementation of the\nextraction methods, the evaluation measures and several utilities. WikiTailor\nmakes obtaining multilingual in-domain data from the Wikipedia easy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01218v1",
            "date": "2020-05-04 00:19:48+00:00",
            "title": "Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering",
            "authors": [
                "Vikas Yadav",
                "Steven Bethard",
                "Mihai Surdeanu"
            ],
            "abstract": "Evidence retrieval is a critical stage of question answering (QA), necessary\nnot only to improve performance, but also to explain the decisions of the\ncorresponding QA method. We introduce a simple, fast, and unsupervised\niterative evidence retrieval method, which relies on three ideas: (a) an\nunsupervised alignment approach to soft-align questions and answers with\njustification sentences using only GloVe embeddings, (b) an iterative process\nthat reformulates queries focusing on terms that are not covered by existing\njustifications, which (c) a stopping criterion that terminates retrieval when\nthe terms in the given question and candidate answers are covered by the\nretrieved justifications. Despite its simplicity, our approach outperforms all\nthe previous methods (including supervised methods) on the evidence selection\ntask on two datasets: MultiRC and QASC. When these evidence sentences are fed\ninto a RoBERTa answer classification component, we achieve state-of-the-art QA\nperformance on these two datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01259v2",
            "date": "2020-05-04 04:06:53+00:00",
            "title": "Noise Pollution in Hospital Readmission Prediction: Long Document Classification with Reinforcement Learning",
            "authors": [
                "Liyan Xu",
                "Julien Hogan",
                "Rachel E. Patzer",
                "Jinho D. Choi"
            ],
            "abstract": "This paper presents a reinforcement learning approach to extract noise in\nlong clinical documents for the task of readmission prediction after kidney\ntransplant. We face the challenges of developing robust models on a small\ndataset where each document may consist of over 10K tokens with full of noise\nincluding tabular text and task-irrelevant sentences. We first experiment four\ntypes of encoders to empirically decide the best document representation, and\nthen apply reinforcement learning to remove noisy text from the long documents,\nwhich models the noise extraction process as a sequential decision problem. Our\nresults show that the old bag-of-words encoder outperforms deep learning-based\nencoders on this task, and reinforcement learning is able to improve upon\nbaseline while pruning out 25% text segments. Our analysis depicts that\nreinforcement learning is able to identify both typical noisy tokens and\ntask-specific noisy text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01279v1",
            "date": "2020-05-04 05:45:13+00:00",
            "title": "Improving Adversarial Text Generation by Modeling the Distant Future",
            "authors": [
                "Ruiyi Zhang",
                "Changyou Chen",
                "Zhe Gan",
                "Wenlin Wang",
                "Dinghan Shen",
                "Guoyin Wang",
                "Zheng Wen",
                "Lawrence Carin"
            ],
            "abstract": "Auto-regressive text generation models usually focus on local fluency, and\nmay cause inconsistent semantic meaning in long text generation. Further,\nautomatically generating words with similar semantics is challenging, and\nhand-crafted linguistic rules are difficult to apply. We consider a text\nplanning scheme and present a model-based imitation-learning approach to\nalleviate the aforementioned issues. Specifically, we propose a novel guider\nnetwork to focus on the generative process over a longer horizon, which can\nassist next-word prediction and provide intermediate rewards for generator\noptimization. Extensive experiments demonstrate that the proposed method leads\nto improved performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01348v2",
            "date": "2020-05-04 09:44:54+00:00",
            "title": "The Sensitivity of Language Models and Humans to Winograd Schema Perturbations",
            "authors": [
                "Mostafa Abdou",
                "Vinit Ravishankar",
                "Maria Barrett",
                "Yonatan Belinkov",
                "Desmond Elliott",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Large-scale pretrained language models are the major driving force behind\nrecent improvements in performance on the Winograd Schema Challenge, a widely\nemployed test of common sense reasoning ability. We show, however, with a new\ndiagnostic dataset, that these models are sensitive to linguistic perturbations\nof the Winograd examples that minimally affect human understanding. Our results\nhighlight interesting differences between humans and language models: language\nmodels are more sensitive to number or gender alternations and synonym\nreplacements than humans, and humans are more stable and consistent in their\npredictions, maintain a much higher absolute performance, and perform better on\nnon-associative instances than associative ones. Overall, humans are correct\nmore often than out-of-the-box models, and the models are sometimes right for\nthe wrong reasons. Finally, we show that fine-tuning on a large, task-specific\ndataset can offer a solution to these issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01355v1",
            "date": "2020-05-04 09:55:38+00:00",
            "title": "On Systematically Building a Controlled Natural Language for Functional Requirements",
            "authors": [
                "Alvaro Veizaga",
                "Mauricio Alferez",
                "Damiano Torre",
                "Mehrdad Sabetzadeh",
                "Lionel Briand"
            ],
            "abstract": "[Context] Natural language (NL) is pervasive in software requirements\nspecifications (SRSs). However, despite its popularity and widespread use, NL\nis highly prone to quality issues such as vagueness, ambiguity, and\nincompleteness. Controlled natural languages (CNLs) have been proposed as a way\nto prevent quality problems in requirements documents, while maintaining the\nflexibility to write and communicate requirements in an intuitive and\nuniversally understood manner. [Objective] In collaboration with an industrial\npartner from the financial domain, we systematically develop and evaluate a\nCNL, named Rimay, intended at helping analysts write functional requirements.\n[Method] We rely on Grounded Theory for building Rimay and follow well-known\nguidelines for conducting and reporting industrial case study research.\n[Results] Our main contributions are: (1) a qualitative methodology to\nsystematically define a CNL for functional requirements; this methodology is\ngeneral and applicable to information systems beyond the financial domain, (2)\na CNL grammar to represent functional requirements; this grammar is derived\nfrom our experience in the financial domain, but should be applicable, possibly\nwith adaptations, to other information-system domains, and (3) an empirical\nevaluation of our CNL (Rimay) through an industrial case study. Our\ncontributions draw on 15 representative SRSs, collectively containing 3215 NL\nrequirements statements from the financial domain. [Conclusion] Our evaluation\nshows that Rimay is expressive enough to capture, on average, 88% (405 out of\n460) of the NL requirements statements in four previously unseen SRSs from the\nfinancial domain.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2005.01525v2",
            "date": "2020-05-04 14:36:07+00:00",
            "title": "To Test Machine Comprehension, Start by Defining Comprehension",
            "authors": [
                "Jesse Dunietz",
                "Gregory Burnham",
                "Akash Bharadwaj",
                "Owen Rambow",
                "Jennifer Chu-Carroll",
                "David Ferrucci"
            ],
            "abstract": "Many tasks aim to measure machine reading comprehension (MRC), often focusing\non question types presumed to be difficult. Rarely, however, do task designers\nstart by considering what systems should in fact comprehend. In this paper we\nmake two key contributions. First, we argue that existing approaches do not\nadequately define comprehension; they are too unsystematic about what content\nis tested. Second, we present a detailed definition of comprehension -- a\n\"Template of Understanding\" -- for a widely useful class of texts, namely short\nnarratives. We then conduct an experiment that strongly suggests existing\nsystems are not up to the task of narrative understanding as we define it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01556v1",
            "date": "2020-05-04 15:16:10+00:00",
            "title": "Compose Like Humans: Jointly Improving the Coherence and Novelty for Modern Chinese Poetry Generation",
            "authors": [
                "Lei Shen",
                "Xiaoyu Guo",
                "Meng Chen"
            ],
            "abstract": "Chinese poetry is an important part of worldwide culture, and classical and\nmodern sub-branches are quite different. The former is a unique genre and has\nstrict constraints, while the latter is very flexible in length, optional to\nhave rhymes, and similar to modern poetry in other languages. Thus, it requires\nmore to control the coherence and improve the novelty. In this paper, we\npropose a generate-retrieve-then-refine paradigm to jointly improve the\ncoherence and novelty. In the first stage, a draft is generated given keywords\n(i.e., topics) only. The second stage produces a \"refining vector\" from\nretrieval lines. At last, we take into consideration both the draft and the\n\"refining vector\" to generate a new poem. The draft provides future\nsentence-level information for a line to be generated. Meanwhile, the \"refining\nvector\" points out the direction of refinement based on impressive words\ndetection mechanism which can learn good patterns from references and then\ncreate new ones via insertion operation. Experimental results on a collected\nlarge-scale modern Chinese poetry dataset show that our proposed approach can\nnot only generate more coherent poems, but also improve the diversity and\nnovelty.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01646v1",
            "date": "2020-05-04 17:01:11+00:00",
            "title": "A Probabilistic Generative Model for Typographical Analysis of Early Modern Printing",
            "authors": [
                "Kartik Goyal",
                "Chris Dyer",
                "Christopher Warren",
                "Max G'Sell",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We propose a deep and interpretable probabilistic generative model to analyze\nglyph shapes in printed Early Modern documents. We focus on clustering\nextracted glyph images into underlying templates in the presence of multiple\nconfounding sources of variance. Our approach introduces a neural editor model\nthat first generates well-understood printing phenomena like spatial\nperturbations from template parameters via interpertable latent variables, and\nthen modifies the result by generating a non-interpretable latent vector\nresponsible for inking variations, jitter, noise from the archiving process,\nand other unforeseen phenomena associated with Early Modern printing.\nCritically, by introducing an inference network whose input is restricted to\nthe visual residual between the observation and the interpretably-modified\ntemplate, we are able to control and isolate what the vector-valued latent\nvariable captures. We show that our approach outperforms rigid interpretable\nclustering baselines (Ocular) and overly-flexible deep generative models (VAE)\nalike on the task of completely unsupervised discovery of typefaces in\nmixed-font documents.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2005.01655v1",
            "date": "2020-05-04 17:09:15+00:00",
            "title": "Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressions",
            "authors": [
                "Arjun R Akula",
                "Spandana Gella",
                "Yaser Al-Onaizan",
                "Song-Chun Zhu",
                "Siva Reddy"
            ],
            "abstract": "Visual referring expression recognition is a challenging task that requires\nnatural language understanding in the context of an image. We critically\nexamine RefCOCOg, a standard benchmark for this task, using a human study and\nshow that 83.7% of test instances do not require reasoning on linguistic\nstructure, i.e., words are enough to identify the target object, the word order\ndoesn't matter. To measure the true progress of existing models, we split the\ntest set into two sets, one which requires reasoning on linguistic structure\nand the other which doesn't. Additionally, we create an out-of-distribution\ndataset Ref-Adv by asking crowdworkers to perturb in-domain examples such that\nthe target object changes. Using these datasets, we empirically show that\nexisting methods fail to exploit linguistic structure and are 12% to 23% lower\nin performance than the established progress for this task. We also propose two\nmethods, one based on contrastive learning and the other based on multi-task\nlearning, to increase the robustness of ViLBERT, the current state-of-the-art\nmodel for this task. Our datasets are publicly available at\nhttps://github.com/aws/aws-refcocog-adv",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01777v1",
            "date": "2020-05-04 18:27:58+00:00",
            "title": "ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents",
            "authors": [
                "Chia-Yu Li",
                "Daniel Ortega",
                "Dirk V\u00e4th",
                "Florian Lux",
                "Lindsey Vanderlyn",
                "Maximilian Schmidt",
                "Michael Neumann",
                "Moritz V\u00f6lkel",
                "Pavel Denisov",
                "Sabrina Jenne",
                "Zorica Kacarevic",
                "Ngoc Thang Vu"
            ],
            "abstract": "We present ADVISER - an open-source, multi-domain dialog system toolkit that\nenables the development of multi-modal (incorporating speech, text and vision),\nsocially-engaged (e.g. emotion recognition, engagement level prediction and\nbackchanneling) conversational agents. The final Python-based implementation of\nour toolkit is flexible, easy to use, and easy to extend not only for\ntechnically experienced users, such as machine learning researchers, but also\nfor less technically experienced users, such as linguists or cognitive\nscientists, thereby providing a flexible platform for collaborative research.\nLink to open-source code: https://github.com/DigitalPhonetics/adviser",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01803v1",
            "date": "2020-05-04 19:25:39+00:00",
            "title": "A Systematic Media Frame Analysis of 1.5 Million New York Times Articles from 2000 to 2017",
            "authors": [
                "Haewoon Kwak",
                "Jisun An",
                "Yong-Yeol Ahn"
            ],
            "abstract": "Framing is an indispensable narrative device for news media because even the\nsame facts may lead to conflicting understandings if deliberate framing is\nemployed. Therefore, identifying media framing is a crucial step to\nunderstanding how news media influence the public. Framing is, however,\ndifficult to operationalize and detect, and thus traditional media framing\nstudies had to rely on manual annotation, which is challenging to scale up to\nmassive news datasets. Here, by developing a media frame classifier that\nachieves state-of-the-art performance, we systematically analyze the media\nframes of 1.5 million New York Times articles published from 2000 to 2017. By\nexamining the ebb and flow of media frames over almost two decades, we show\nthat short-term frame abundance fluctuation closely corresponds to major\nevents, while there also exist several long-term trends, such as the gradually\nincreasing prevalence of the ``Cultural identity'' frame. By examining specific\ntopics and sentiments, we identify characteristics and dynamics of each frame.\nFinally, as a case study, we delve into the framing of mass shootings,\nrevealing three major framing patterns. Our scalable, computational approach to\nmassive news datasets opens up new pathways for systematic media framing\nstudies.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2005.01810v1",
            "date": "2020-05-04 19:34:46+00:00",
            "title": "Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words",
            "authors": [
                "Josef Klafka",
                "Allyson Ettinger"
            ],
            "abstract": "Although models using contextual word embeddings have achieved\nstate-of-the-art results on a host of NLP tasks, little is known about exactly\nwhat information these embeddings encode about the context words that they are\nunderstood to reflect. To address this question, we introduce a suite of\nprobing tasks that enable fine-grained testing of contextual embeddings for\nencoding of information about surrounding words. We apply these tasks to\nexamine the popular BERT, ELMo and GPT contextual encoders, and find that each\nof our tested information types is indeed encoded as contextual information\nacross tokens, often with near-perfect recoverability-but the encoders vary in\nwhich features they distribute to which tokens, how nuanced their distributions\nare, and how robust the encoding of each feature is to distance. We discuss\nimplications of these results for how different types of models breakdown and\nprioritize word-level context information when constructing token embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01854v2",
            "date": "2020-05-04 21:32:12+00:00",
            "title": "Data Augmentation for Hypernymy Detection",
            "authors": [
                "Thomas Kober",
                "Julie Weeds",
                "Lorenzo Bertolini",
                "David Weir"
            ],
            "abstract": "The automatic detection of hypernymy relationships represents a challenging\nproblem in NLP. The successful application of state-of-the-art supervised\napproaches using distributed representations has generally been impeded by the\nlimited availability of high quality training data. We have developed two novel\ndata augmentation techniques which generate new training examples from existing\nones. First, we combine the linguistic principles of hypernym transitivity and\nintersective modifier-noun composition to generate additional pairs of vectors,\nsuch as \"small dog - dog\" or \"small dog - animal\", for which a hypernymy\nrelationship can be assumed. Second, we use generative adversarial networks\n(GANs) to generate pairs of vectors for which the hypernymy relation can also\nbe assumed. We furthermore present two complementary strategies for extending\nan existing dataset by leveraging linguistic resources such as WordNet. Using\nan evaluation across 3 different datasets for hypernymy detection and 2\ndifferent vector spaces, we demonstrate that both of the proposed automatic\ndata augmentation and dataset extension strategies substantially improve\nclassifier performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01897v3",
            "date": "2020-05-05 01:07:07+00:00",
            "title": "Fine-grained Financial Opinion Mining: A Survey and Research Agenda",
            "authors": [
                "Chung-Chi Chen",
                "Hen-Hsen Huang",
                "Hsin-Hsi Chen"
            ],
            "abstract": "Opinion mining is a prevalent research issue in many domains. In the\nfinancial domain, however, it is still in the early stages. Most of the\nresearches on this topic only focus on the coarse-grained market sentiment\nanalysis, i.e., 2-way classification for bullish/bearish. Thanks to the recent\nfinancial technology (FinTech) development, some interdisciplinary researchers\nstart to involve in the in-depth analysis of investors' opinions. In this\nposition paper, we first define the financial opinions from both coarse-grained\nand fine-grained points of views, and then provide an overview on the issues\nalready tackled. In addition to listing research issues of the existing topics,\nwe further propose a road map of fine-grained financial opinion mining for\nfuture researches, and point out several challenges yet to explore. Moreover,\nwe provide possible directions to deal with the proposed research issues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.01898v1",
            "date": "2020-05-05 01:08:36+00:00",
            "title": "Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering",
            "authors": [
                "Hao Cheng",
                "Ming-Wei Chang",
                "Kenton Lee",
                "Kristina Toutanova"
            ],
            "abstract": "We address the problem of extractive question answering using document-level\ndistant super-vision, pairing questions and relevant documents with answer\nstrings. We compare previously used probability space and distant super-vision\nassumptions (assumptions on the correspondence between the weak answer string\nlabels and possible answer mention spans). We show that these assumptions\ninteract, and that different configurations provide complementary benefits. We\ndemonstrate that a multi-objective model can efficiently combine the advantages\nof multiple assumptions and out-perform the best individual formulation. Our\napproach outperforms previous state-of-the-art models by 4.3 points in F1 on\nTriviaQA-Wiki and 1.7 points in Rouge-L on NarrativeQA summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02008v1",
            "date": "2020-05-05 08:47:34+00:00",
            "title": "Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change",
            "authors": [
                "Hongfei Xu",
                "Josef van Genabith",
                "Deyi Xiong",
                "Qiuhui Liu"
            ],
            "abstract": "The choice of hyper-parameters affects the performance of neural models.\nWhile much previous research (Sutskever et al., 2013; Duchi et al., 2011;\nKingma and Ba, 2015) focuses on accelerating convergence and reducing the\neffects of the learning rate, comparatively few papers concentrate on the\neffect of batch size. In this paper, we analyze how increasing batch size\naffects gradient direction, and propose to evaluate the stability of gradients\nwith their angle change. Based on our observations, the angle change of\ngradient direction first tends to stabilize (i.e. gradually decrease) while\naccumulating mini-batches, and then starts to fluctuate. We propose to\nautomatically and dynamically determine batch sizes by accumulating gradients\nof mini-batches and performing an optimization step at just the time when the\ndirection of gradients starts to fluctuate. To improve the efficiency of our\napproach for large models, we propose a sampling approach to select gradients\nof parameters sensitive to the batch size. Our approach dynamically determines\nproper and efficient batch sizes during training. In our experiments on the WMT\n14 English to German and English to French tasks, our approach improves the\nTransformer with a fixed 25k batch size by +0.73 and +0.82 BLEU respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02049v1",
            "date": "2020-05-05 10:24:28+00:00",
            "title": "Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer",
            "authors": [
                "Chulun Zhou",
                "Liangyu Chen",
                "Jiachen Liu",
                "Xinyan Xiao",
                "Jinsong Su",
                "Sheng Guo",
                "Hua Wu"
            ],
            "abstract": "Unsupervised style transfer aims to change the style of an input sentence\nwhile preserving its original content without using parallel training data. In\ncurrent dominant approaches, owing to the lack of fine-grained control on the\ninfluence from the target style,they are unable to yield desirable output\nsentences. In this paper, we propose a novel attentional sequence-to-sequence\n(Seq2seq) model that dynamically exploits the relevance of each output word to\nthe target style for unsupervised style transfer. Specifically, we first\npretrain a style classifier, where the relevance of each input word to the\noriginal style can be quantified via layer-wise relevance propagation. In a\ndenoising auto-encoding manner, we train an attentional Seq2seq model to\nreconstruct input sentences and repredict word-level previously-quantified\nstyle relevance simultaneously. In this way, this model is endowed with the\nability to automatically predict the style relevance of each output word. Then,\nwe equip the decoder of this model with a neural style component to exploit the\npredicted wordlevel style relevance for better style transfer. Particularly, we\nfine-tune this model using a carefully-designed objective function involving\nstyle transfer, style relevance consistency, content preservation and fluency\nmodeling loss terms. Experimental results show that our proposed model achieves\nstate-of-the-art performance in terms of both transfer accuracy and content\npreservation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02087v1",
            "date": "2020-05-05 12:01:47+00:00",
            "title": "Self-organizing Pattern in Multilayer Network for Words and Syllables",
            "authors": [
                "Li-Min Wang",
                "Sun-Ting Tsai",
                "Shan-Jyun Wu",
                "Meng-Xue Tsai",
                "Daw-Wei Wang",
                "Yi-Ching Su",
                "Tzay-Ming Hong"
            ],
            "abstract": "One of the ultimate goals for linguists is to find universal properties in\nhuman languages. Although words are generally considered as representing\narbitrary mapping between linguistic forms and meanings, we propose a new\nuniversal law that highlights the equally important role of syllables, which is\ncomplementary to Zipf's. By plotting rank-rank frequency distribution of word\nand syllable for English and Chinese corpora, visible lines appear and can be\nfit to a master curve. We discover the multi-layer network for words and\nsyllables based on this analysis exhibits the feature of self-organization\nwhich relies heavily on the inclusion of syllables and their connections.\nAnalytic form for the scaling structure is derived and used to quantify how\nInternet slang becomes fashionable, which demonstrates its usefulness as a new\ntool to evolutionary linguistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02171v3",
            "date": "2020-05-02 23:17:08+00:00",
            "title": "Neural Computing for Online Arabic Handwriting Character Recognition using Hard Stroke Features Mining",
            "authors": [
                "Amjad Rehman"
            ],
            "abstract": "Online Arabic cursive character recognition is still a big challenge due to\nthe existing complexities including Arabic cursive script styles, writing\nspeed, writer mood and so forth. Due to these unavoidable constraints, the\naccuracy of online Arabic character's recognition is still low and retain space\nfor improvement. In this research, an enhanced method of detecting the desired\ncritical points from vertical and horizontal direction-length of handwriting\nstroke features of online Arabic script recognition is proposed. Each extracted\nstroke feature divides every isolated character into some meaningful pattern\nknown as tokens. A minimum feature set is extracted from these tokens for\nclassification of characters using a multilayer perceptron with a\nback-propagation learning algorithm and modified sigmoid function-based\nactivation function. In this work, two milestones are achieved; firstly, attain\na fixed number of tokens, secondly, minimize the number of the most repetitive\ntokens. For experiments, handwritten Arabic characters are selected from the\nOHASD benchmark dataset to test and evaluate the proposed method. The proposed\nmethod achieves an average accuracy of 98.6% comparable in state of art\ncharacter recognition techniques.",
            "journal ref": "IJICIC 2021",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2005.02178v2",
            "date": "2020-05-02 11:49:09+00:00",
            "title": "IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization",
            "authors": [
                "Wenxuan Zhou",
                "Bill Yuchen Lin",
                "Xiang Ren"
            ],
            "abstract": "Fine-tuning pre-trained language models (PTLMs), such as BERT and its better\nvariant RoBERTa, has been a common practice for advancing performance in\nnatural language understanding (NLU) tasks. Recent advance in representation\nlearning shows that isotropic (i.e., unit-variance and uncorrelated) embeddings\ncan significantly improve performance on downstream tasks with faster\nconvergence and better generalization. The isotropy of the pre-trained\nembeddings in PTLMs, however, is relatively under-explored. In this paper, we\nanalyze the isotropy of the pre-trained [CLS] embeddings of PTLMs with\nstraightforward visualization, and point out two major issues: high variance in\ntheir standard deviation, and high correlation between different dimensions. We\nalso propose a new network regularization method, isotropic batch normalization\n(IsoBN) to address the issues, towards learning more isotropic representations\nin fine-tuning by dynamically penalizing dominating principal components. This\nsimple yet effective fine-tuning method yields about 1.0 absolute increment on\nthe average of seven NLU tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02365v3",
            "date": "2020-05-05 17:51:27+00:00",
            "title": "SLEDGE: A Simple Yet Effective Baseline for COVID-19 Scientific Knowledge Search",
            "authors": [
                "Sean MacAvaney",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "With worldwide concerns surrounding the Severe Acute Respiratory Syndrome\nCoronavirus 2 (SARS-CoV-2), there is a rapidly growing body of literature on\nthe virus. Clinicians, researchers, and policy-makers need a way to effectively\nsearch these articles. In this work, we present a search system called SLEDGE,\nwhich utilizes SciBERT to effectively re-rank articles. We train the model on a\ngeneral-domain answer ranking dataset, and transfer the relevance signals to\nSARS-CoV-2 for evaluation. We observe SLEDGE's effectiveness as a strong\nbaseline on the TREC-COVID challenge (topping the learderboard with an nDCG@10\nof 0.6844). Insights provided by a detailed analysis provide some potential\nfuture directions to explore, including the importance of filtering by date and\nthe potential of neural methods that rely more heavily on count signals. We\nrelease the code to facilitate future work on this critical task at\nhttps://github.com/Georgetown-IR-Lab/covid-neural-ir",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.02367v5",
            "date": "2020-05-05 17:51:42+00:00",
            "title": "CODA-19: Using a Non-Expert Crowd to Annotate Research Aspects on 10,000+ Abstracts in the COVID-19 Open Research Dataset",
            "authors": [
                "Ting-Hao 'Kenneth' Huang",
                "Chieh-Yang Huang",
                "Chien-Kuang Cornelia Ding",
                "Yen-Chia Hsu",
                "C. Lee Giles"
            ],
            "abstract": "This paper introduces CODA-19, a human-annotated dataset that codes the\nBackground, Purpose, Method, Finding/Contribution, and Other sections of 10,966\nEnglish abstracts in the COVID-19 Open Research Dataset. CODA-19 was created by\n248 crowd workers from Amazon Mechanical Turk within 10 days, and achieved\nlabeling quality comparable to that of experts. Each abstract was annotated by\nnine different workers, and the final labels were acquired by majority vote.\nThe inter-annotator agreement (Cohen's kappa) between the crowd and the\nbiomedical expert (0.741) is comparable to inter-expert agreement (0.788).\nCODA-19's labels have an accuracy of 82.2% when compared to the biomedical\nexpert's labels, while the accuracy between experts was 85.0%. Reliable human\nannotations help scientists access and integrate the rapidly accelerating\ncoronavirus literature, and also serve as the battery of AI/NLP research, but\nobtaining expert annotations can be slow. We demonstrated that a non-expert\ncrowd can be rapidly employed at scale to join the fight against COVID-19.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02431v2",
            "date": "2020-05-05 18:30:08+00:00",
            "title": "Automated Personalized Feedback Improves Learning Gains in an Intelligent Tutoring System",
            "authors": [
                "Ekaterina Kochmar",
                "Dung Do Vu",
                "Robert Belfer",
                "Varun Gupta",
                "Iulian Vlad Serban",
                "Joelle Pineau"
            ],
            "abstract": "We investigate how automated, data-driven, personalized feedback in a\nlarge-scale intelligent tutoring system (ITS) improves student learning\noutcomes. We propose a machine learning approach to generate personalized\nfeedback, which takes individual needs of students into account. We utilize\nstate-of-the-art machine learning and natural language processing techniques to\nprovide the students with personalized hints, Wikipedia-based explanations, and\nmathematical hints. Our model is used in Korbit, a large-scale dialogue-based\nITS with thousands of students launched in 2019, and we demonstrate that the\npersonalized feedback leads to considerable improvement in student learning\noutcomes and in the subjective evaluation of the feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02470v1",
            "date": "2020-05-05 20:20:25+00:00",
            "title": "Russian Natural Language Generation: Creation of a Language Modelling Dataset and Evaluation with Modern Neural Architectures",
            "authors": [
                "Zein Shaheen",
                "Gerhard Wohlgenannt",
                "Bassel Zaity",
                "Dmitry Mouromtsev",
                "Vadim Pak"
            ],
            "abstract": "Generating coherent, grammatically correct, and meaningful text is very\nchallenging, however, it is crucial to many modern NLP systems. So far,\nresearch has mostly focused on English language, for other languages both\nstandardized datasets, as well as experiments with state-of-the-art models, are\nrare. In this work, we i) provide a novel reference dataset for Russian\nlanguage modeling, ii) experiment with popular modern methods for text\ngeneration, namely variational autoencoders, and generative adversarial\nnetworks, which we trained on the new dataset. We evaluate the generated text\nregarding metrics such as perplexity, grammatical correctness and lexical\ndiversity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02507v1",
            "date": "2020-05-05 21:30:16+00:00",
            "title": "MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models",
            "authors": [
                "Mandy Guo",
                "Yinfei Yang",
                "Daniel Cer",
                "Qinlan Shen",
                "Noah Constant"
            ],
            "abstract": "Retrieval question answering (ReQA) is the task of retrieving a\nsentence-level answer to a question from an open corpus (Ahmad et\nal.,2019).This paper presents MultiReQA, anew multi-domain ReQA evaluation\nsuite com-posed of eight retrieval QA tasks drawn from publicly available QA\ndatasets. We provide the first systematic retrieval based evaluation over these\ndatasets using two supervised neural models, based on fine-tuning BERT\nandUSE-QA models respectively, as well as a surprisingly strong information\nretrieval baseline,BM25. Five of these tasks contain both train-ing and test\ndata, while three contain test data only. Performance on the five tasks with\ntrain-ing data shows that while a general model covering all domains is\nachievable, the best performance is often obtained by training exclusively on\nin-domain data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02557v2",
            "date": "2020-05-06 01:59:13+00:00",
            "title": "Crossing Variational Autoencoders for Answer Retrieval",
            "authors": [
                "Wenhao Yu",
                "Lingfei Wu",
                "Qingkai Zeng",
                "Shu Tao",
                "Yu Deng",
                "Meng Jiang"
            ],
            "abstract": "Answer retrieval is to find the most aligned answer from a large set of\ncandidates given a question. Learning vector representations of\nquestions/answers is the key factor. Question-answer alignment and\nquestion/answer semantics are two important signals for learning the\nrepresentations. Existing methods learned semantic representations with dual\nencoders or dual variational auto-encoders. The semantic information was\nlearned from language models or question-to-question (answer-to-answer)\ngenerative processes. However, the alignment and semantics were too separate to\ncapture the aligned semantics between question and answer. In this work, we\npropose to cross variational auto-encoders by generating questions with aligned\nanswers and generating answers with aligned questions. Experiments show that\nour method outperforms the state-of-the-art answer retrieval method on SQuAD.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02771v1",
            "date": "2020-05-06 12:39:45+00:00",
            "title": "Unsupervised Neural Aspect Search with Related Terms Extraction",
            "authors": [
                "Timur Sokhin",
                "Maria Khodorchenko",
                "Nikolay Butakov"
            ],
            "abstract": "The tasks of aspect identification and term extraction remain challenging in\nnatural language processing. While supervised methods achieve high scores, it\nis hard to use them in real-world applications due to the lack of labelled\ndatasets. Unsupervised approaches outperform these methods on several tasks,\nbut it is still a challenge to extract both an aspect and a corresponding term,\nparticularly in the multi-aspect setting. In this work, we present a novel\nunsupervised neural network with convolutional multi-attention mechanism, that\nallows extracting pairs (aspect, term) simultaneously, and demonstrate the\neffectiveness on the real-world dataset. We apply a special loss aimed to\nimprove the quality of multi-aspect extraction. The experimental study\ndemonstrates, what with this loss we increase the precision not only on this\njoint setting but also on aspect prediction only.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02780v1",
            "date": "2020-04-25 10:45:48+00:00",
            "title": "A Large-scale Industrial and Professional Occupation Dataset",
            "authors": [
                "Junhua Liu",
                "Yung Chuen Ng",
                "Kwan Hui Lim"
            ],
            "abstract": "There has been growing interest in utilizing occupational data mining and\nanalysis. In today's job market, occupational data mining and analysis is\ngrowing in importance as it enables companies to predict employee turnover,\nmodel career trajectories, screen through resumes and perform other human\nresource tasks. A key requirement to facilitate these tasks is the need for an\noccupation-related dataset. However, most research use proprietary datasets or\ndo not make their dataset publicly available, thus impeding development in this\narea. To solve this issue, we present the Industrial and Professional\nOccupation Dataset (IPOD), which comprises 192k job titles belonging to 56k\nLinkedIn users. In addition to making IPOD publicly available, we also: (i)\nmanually annotate each job title with its associated level of seniority, domain\nof work and location; and (ii) provide embedding for job titles and discuss\nvarious use cases. This dataset is publicly available at\nhttps://github.com/junhua/ipod.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.02843v1",
            "date": "2020-05-06 14:13:49+00:00",
            "title": "Graph-Embedding Empowered Entity Retrieval",
            "authors": [
                "Emma J. Gerritse",
                "Faegheh Hasibi",
                "Arjen P. de Vries"
            ],
            "abstract": "In this research, we improve upon the current state of the art in entity\nretrieval by re-ranking the result list using graph embeddings. The paper shows\nthat graph embeddings are useful for entity-oriented search tasks. We\ndemonstrate empirically that encoding information from the knowledge graph into\n(graph) embeddings contributes to a higher increase in effectiveness of entity\nretrieval results than using plain word embeddings. We analyze the impact of\nthe accuracy of the entity linker on the overall retrieval effectiveness. Our\nanalysis further deploys the cluster hypothesis to explain the observed\nadvantages of graph embeddings over the more widely used word embeddings, for\nuser tasks involving ranking entities.",
            "journal ref": "Advances in Information Retrieval. ECIR 2020. Lecture Notes in\n  Computer Science, vol 12035. Springer,",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.02914v3",
            "date": "2020-05-06 15:35:53+00:00",
            "title": "Review of Text Style Transfer Based on Deep Learning",
            "authors": [
                "Xiangyang Li",
                "Guo Pu",
                "Keyu Ming",
                "Pu Li",
                "Jie Wang",
                "Yuxuan Wang"
            ],
            "abstract": "Text style transfer is a hot issue in recent natural language\nprocessing,which mainly studies the text to adapt to different specific\nsituations, audiences and purposes by making some changes. The style of the\ntext usually includes many aspects such as morphology, grammar, emotion,\ncomplexity, fluency, tense, tone and so on. In the traditional text style\ntransfer model, the text style is generally relied on by experts knowledge and\nhand-designed rules, but with the application of deep learning in the field of\nnatural language processing, the text style transfer method based on deep\nlearning Started to be heavily researched. In recent years, text style transfer\nis becoming a hot issue in natural language processing research. This article\nsummarizes the research on the text style transfer model based on deep learning\nin recent years, and summarizes, analyzes and compares the main research\ndirections and progress. In addition, the article also introduces public data\nsets and evaluation indicators commonly used for text style transfer. Finally,\nthe existing characteristics of the text style transfer model are summarized,\nand the future development trend of the text style transfer model based on deep\nlearning is analyzed and forecasted.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02954v1",
            "date": "2020-05-06 17:00:46+00:00",
            "title": "Multitask Models for Supervised Protests Detection in Texts",
            "authors": [
                "Benjamin J. Radford"
            ],
            "abstract": "The CLEF 2019 ProtestNews Lab tasks participants to identify text relating to\npolitical protests within larger corpora of news data. Three tasks include\narticle classification, sentence detection, and event extraction. I apply\nmultitask neural networks capable of producing predictions for two and three of\nthese tasks simultaneously. The multitask framework allows the model to learn\nrelevant features from the training data of all three tasks. This paper\ndemonstrates performance near or above the reported state-of-the-art for\nautomated political event coding though noted differences in research design\nmake direct comparisons difficult.",
            "journal ref": "Working Notes of CLEF 2019 (2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02966v1",
            "date": "2020-05-06 17:20:14+00:00",
            "title": "Seeing the Forest and the Trees: Detection and Cross-Document Coreference Resolution of Militarized Interstate Disputes",
            "authors": [
                "Benjamin J. Radford"
            ],
            "abstract": "Previous efforts to automate the detection of social and political events in\ntext have primarily focused on identifying events described within single\nsentences or documents. Within a corpus of documents, these automated systems\nare unable to link event references -- recognize singular events across\nmultiple sentences or documents. A separate literature in computational\nlinguistics on event coreference resolution attempts to link known events to\none another within (and across) documents. I provide a data set for evaluating\nmethods to identify certain political events in text and to link related texts\nto one another based on shared events. The data set, Headlines of War, is built\non the Militarized Interstate Disputes data set and offers headlines classified\nby dispute status and headline pairs labeled with coreference indicators.\nAdditionally, I introduce a model capable of accomplishing both tasks. The\nmulti-task convolutional neural network is shown to be capable of recognizing\nevents and event coreferences given the headlines' texts and publication dates.",
            "journal ref": "Workshop Proceedings of AESPEN at LREC 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.02990v1",
            "date": "2020-05-06 17:45:35+00:00",
            "title": "PeTra: A Sparsely Supervised Memory Model for People Tracking",
            "authors": [
                "Shubham Toshniwal",
                "Allyson Ettinger",
                "Kevin Gimpel",
                "Karen Livescu"
            ],
            "abstract": "We propose PeTra, a memory-augmented neural network designed to track\nentities in its memory slots. PeTra is trained using sparse annotation from the\nGAP pronoun resolution dataset and outperforms a prior memory model on the task\nwhile using a simpler architecture. We empirically compare key modeling\nchoices, finding that we can simplify several aspects of the design of the\nmemory module while retaining strong performance. To measure the people\ntracking capability of memory models, we (a) propose a new diagnostic\nevaluation based on counting the number of unique entities in text, and (b)\nconduct a small scale human evaluation to compare evidence of people tracking\nin the memory logs of PeTra relative to a previous approach. PeTra is highly\neffective in both evaluations, demonstrating its ability to track people in its\nmemory despite being trained with limited annotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03008v1",
            "date": "2020-05-06 08:38:20+00:00",
            "title": "Evaluating text coherence based on the graph of the consistency of phrases to identify symptoms of schizophrenia",
            "authors": [
                "Artem Kramov"
            ],
            "abstract": "Different state-of-the-art methods of the detection of schizophrenia symptoms\nbased on the estimation of text coherence have been analyzed. The analysis of a\ntext at the level of phrases has been suggested. The method based on the graph\nof the consistency of phrases has been proposed to evaluate the semantic\ncoherence and the cohesion of a text. The semantic coherence, cohesion, and\nother linguistic features (lexical diversity, lexical density) have been taken\ninto account to form feature vectors for the training of a model-classifier.\nThe training of the classifier has been performed on the set of\nEnglish-language interviews. According to the retrieved results, the impact of\neach feature on the output of the model has been analyzed. The results obtained\ncan indicate that the proposed method based on the graph of the consistency of\nphrases may be used in the different tasks of the detection of mental illness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03074v3",
            "date": "2020-05-06 18:58:21+00:00",
            "title": "Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality",
            "authors": [
                "Lachlan McPheat",
                "Mehrnoosh Sadrzadeh",
                "Hadi Wazni",
                "Gijs Wijnholds"
            ],
            "abstract": "We develop a categorical compositional distributional semantics for Lambek\nCalculus with a Relevant Modality !L*, which has a limited edition of the\ncontraction and permutation rules. The categorical part of the semantics is a\nmonoidal biclosed category with a coalgebra modality, very similar to the\nstructure of a Differential Category. We instantiate this category to finite\ndimensional vector spaces and linear maps via \"quantisation\" functors and work\nwith three concrete interpretations of the coalgebra modality. We apply the\nmodel to construct categorical and concrete semantic interpretations for the\nmotivating example of !L*: the derivation of a phrase with a parasitic gap. The\neffectiveness of the concrete interpretations are evaluated via a\ndisambiguation task, on an extension of a sentence disambiguation dataset to\nparasitic gap phrases, using BERT, Word2Vec, and FastText vectors and\nRelational tensors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03119v1",
            "date": "2020-05-06 20:11:46+00:00",
            "title": "Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting",
            "authors": [
                "Po-Yao Huang",
                "Junjie Hu",
                "Xiaojun Chang",
                "Alexander Hauptmann"
            ],
            "abstract": "Unsupervised machine translation (MT) has recently achieved impressive\nresults with monolingual corpora only. However, it is still challenging to\nassociate source-target sentences in the latent space. As people speak\ndifferent languages biologically share similar visual systems, the potential of\nachieving better alignment through visual content is promising yet\nunder-explored in unsupervised multimodal MT (MMT). In this paper, we\ninvestigate how to utilize visual content for disambiguation and promoting\nlatent space alignment in unsupervised MMT. Our model employs multimodal\nback-translation and features pseudo visual pivoting in which we learn a shared\nmultilingual visual-semantic embedding space and incorporate visually-pivoted\ncaptioning as additional weak supervision. The experimental results on the\nwidely used Multi30K dataset show that the proposed model significantly\nimproves over the state-of-the-art methods and generalizes well when the images\nare not available at the testing time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03271v3",
            "date": "2020-05-07 06:24:47+00:00",
            "title": "RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions",
            "authors": [
                "Chung-Cheng Chiu",
                "Arun Narayanan",
                "Wei Han",
                "Rohit Prabhavalkar",
                "Yu Zhang",
                "Navdeep Jaitly",
                "Ruoming Pang",
                "Tara N. Sainath",
                "Patrick Nguyen",
                "Liangliang Cao",
                "Yonghui Wu"
            ],
            "abstract": "In recent years, all-neural end-to-end approaches have obtained\nstate-of-the-art results on several challenging automatic speech recognition\n(ASR) tasks. However, most existing works focus on building ASR models where\ntrain and test data are drawn from the same domain. This results in poor\ngeneralization characteristics on mismatched-domains: e.g., end-to-end models\ntrained on short segments perform poorly when evaluated on longer utterances.\nIn this work, we analyze the generalization properties of streaming and\nnon-streaming recurrent neural network transducer (RNN-T) based end-to-end\nmodels in order to identify model components that negatively affect\ngeneralization performance. We propose two solutions: combining multiple\nregularization techniques during training, and using dynamic overlapping\ninference. On a long-form YouTube test set, when the nonstreaming RNN-T model\nis trained with shorter segments of data, the proposed combination improves\nword error rate (WER) from 22.3% to 14.8%; when the streaming RNN-T model\ntrained on short Search queries, the proposed techniques improve WER on the\nYouTube set from 67.0% to 25.3%. Finally, when trained on Librispeech, we find\nthat dynamic overlapping inference improves WER on YouTube from 99.8% to 33.0%.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.03299v1",
            "date": "2020-05-07 07:43:43+00:00",
            "title": "Adaptive Dialog Policy Learning with Hindsight and User Modeling",
            "authors": [
                "Yan Cao",
                "Keting Lu",
                "Xiaoping Chen",
                "Shiqi Zhang"
            ],
            "abstract": "Reinforcement learning methods have been used to compute dialog policies from\nlanguage-based interaction experiences. Efficiency is of particular importance\nin dialog policy learning, because of the considerable cost of interacting with\npeople, and the very poor user experience from low-quality conversations.\nAiming at improving the efficiency of dialog policy learning, we develop\nalgorithm LHUA (Learning with Hindsight, User modeling, and Adaptation) that,\nfor the first time, enables dialog agents to adaptively learn with hindsight\nfrom both simulated and real users. Simulation and hindsight provide the dialog\nagent with more experience and more (positive) reinforcements respectively.\nExperimental results suggest that, in success rate and policy quality, LHUA\noutperforms competitive baselines from the literature, including its\nno-simulation, no-adaptation, and no-hindsight counterparts.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.03454v2",
            "date": "2020-05-04 15:17:28+00:00",
            "title": "Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture",
            "authors": [
                "Christopher Brix",
                "Parnia Bahar",
                "Hermann Ney"
            ],
            "abstract": "Sparse models require less memory for storage and enable a faster inference\nby reducing the necessary number of FLOPs. This is relevant both for\ntime-critical and on-device computations using neural networks. The stabilized\nlottery ticket hypothesis states that networks can be pruned after none or few\ntraining iterations, using a mask computed based on the unpruned converged\nmodel. On the transformer architecture and the WMT 2014 English-to-German and\nEnglish-to-French tasks, we show that stabilized lottery ticket pruning\nperforms similar to magnitude pruning for sparsity levels of up to 85%, and\npropose a new combination of pruning techniques that outperforms all other\ntechniques for even higher levels of sparsity. Furthermore, we confirm that the\nparameter's initial sign and not its specific value is the primary factor for\nsuccessful training, and show that magnitude pruning could be used to find\nwinning lottery tickets.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2005.03545v3",
            "date": "2020-05-07 15:13:23+00:00",
            "title": "MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis",
            "authors": [
                "Devamanyu Hazarika",
                "Roger Zimmermann",
                "Soujanya Poria"
            ],
            "abstract": "Multimodal Sentiment Analysis is an active area of research that leverages\nmultimodal signals for affective understanding of user-generated videos. The\npredominant approach, addressing this task, has been to develop sophisticated\nfusion techniques. However, the heterogeneous nature of the signals creates\ndistributional modality gaps that pose significant challenges. In this paper,\nwe aim to learn effective modality representations to aid the process of\nfusion. We propose a novel framework, MISA, which projects each modality to two\ndistinct subspaces. The first subspace is modality-invariant, where the\nrepresentations across modalities learn their commonalities and reduce the\nmodality gap. The second subspace is modality-specific, which is private to\neach modality and captures their characteristic features. These representations\nprovide a holistic view of the multimodal data, which is used for fusion that\nleads to task predictions. Our experiments on popular sentiment analysis\nbenchmarks, MOSI and MOSEI, demonstrate significant gains over state-of-the-art\nmodels. We also consider the task of Multimodal Humor Detection and experiment\non the recently proposed UR_FUNNY dataset. Here too, our model fares better\nthan strong baselines, establishing MISA as a useful multimodal framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03588v2",
            "date": "2020-05-07 16:16:24+00:00",
            "title": "Learning Implicit Text Generation via Feature Matching",
            "authors": [
                "Inkit Padhi",
                "Pierre Dognin",
                "Ke Bai",
                "Cicero Nogueira dos Santos",
                "Vijil Chenthamarakshan",
                "Youssef Mroueh",
                "Payel Das"
            ],
            "abstract": "Generative feature matching network (GFMN) is an approach for training\nimplicit generative models for images by performing moment matching on features\nfrom pre-trained neural networks. In this paper, we present new GFMN\nformulations that are effective for sequential data. Our experimental results\nshow the effectiveness of the proposed method, SeqGFMN, for three distinct\ngeneration tasks in English: unconditional text generation, class-conditional\ntext generation, and unsupervised text style transfer. SeqGFMN is stable to\ntrain and outperforms various adversarial approaches for text generation and\ntext style transfer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03640v1",
            "date": "2020-05-07 17:42:13+00:00",
            "title": "Where is Linked Data in Question Answering over Linked Data?",
            "authors": [
                "Tommaso Soru",
                "Edgard Marx",
                "Andr\u00e9 Valdestilhas",
                "Diego Moussallem",
                "Gustavo Publio",
                "Muhammad Saleem"
            ],
            "abstract": "We argue that \"Question Answering with Knowledge Base\" and \"Question\nAnswering over Linked Data\" are currently two instances of the same problem,\ndespite one explicitly declares to deal with Linked Data. We point out the lack\nof existing methods to evaluate question answering on datasets which exploit\nexternal links to the rest of the cloud or share common schema. To this end, we\npropose the creation of new evaluation settings to leverage the advantages of\nthe Semantic Web to achieve AI-complete question answering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03684v2",
            "date": "2020-05-07 18:03:57+00:00",
            "title": "Learning to Segment Actions from Observation and Narration",
            "authors": [
                "Daniel Fried",
                "Jean-Baptiste Alayrac",
                "Phil Blunsom",
                "Chris Dyer",
                "Stephen Clark",
                "Aida Nematzadeh"
            ],
            "abstract": "We apply a generative segmental model of task structure, guided by narration,\nto action segmentation in video. We focus on unsupervised and weakly-supervised\nsettings where no action labels are known during training. Despite its\nsimplicity, our model performs competitively with previous work on a dataset of\nnaturalistic instructional videos. Our model allows us to vary the sources of\nsupervision used in training, and we find that both task structure and\nnarrative language provide large benefits in segmentation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03724v1",
            "date": "2020-05-07 19:54:24+00:00",
            "title": "SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization",
            "authors": [
                "Yang Gao",
                "Wei Zhao",
                "Steffen Eger"
            ],
            "abstract": "We study unsupervised multi-document summarization evaluation metrics, which\nrequire neither human-written reference summaries nor human annotations (e.g.\npreferences, ratings, etc.). We propose SUPERT, which rates the quality of a\nsummary by measuring its semantic similarity with a pseudo reference summary,\ni.e. selected salient sentences from the source documents, using contextualized\nembeddings and soft token alignment techniques. Compared to the\nstate-of-the-art unsupervised evaluation metrics, SUPERT correlates better with\nhuman ratings by 18-39%. Furthermore, we use SUPERT as rewards to guide a\nneural-based reinforcement learning summarizer, yielding favorable performance\ncompared to the state-of-the-art unsupervised summarizers. All source code is\navailable at https://github.com/yg211/acl20-ref-free-eval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03776v2",
            "date": "2020-05-07 21:41:40+00:00",
            "title": "Mapping Natural Language Instructions to Mobile UI Action Sequences",
            "authors": [
                "Yang Li",
                "Jiacong He",
                "Xin Zhou",
                "Yuan Zhang",
                "Jason Baldridge"
            ],
            "abstract": "We present a new problem: grounding natural language instructions to mobile\nuser interface actions, and create three new datasets for it. For full task\nevaluation, we create PIXELHELP, a corpus that pairs English instructions with\nactions performed by people on a mobile UI emulator. To scale training, we\ndecouple the language and action data by (a) annotating action phrase spans in\nHowTo instructions and (b) synthesizing grounded descriptions of actions for\nmobile user interfaces. We use a Transformer to extract action phrase tuples\nfrom long-range natural language instructions. A grounding Transformer then\ncontextually represents UI objects using both their content and screen position\nand connects them to object descriptions. Given a starting screen and\ninstruction, our model achieves 70.59% accuracy on predicting complete\nground-truth action sequences in PIXELHELP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03812v1",
            "date": "2020-05-08 01:16:03+00:00",
            "title": "Comparative Analysis of Word Embeddings for Capturing Word Similarities",
            "authors": [
                "Martina Toshevska",
                "Frosina Stojanovska",
                "Jovan Kalajdjieski"
            ],
            "abstract": "Distributed language representation has become the most widely used technique\nfor language representation in various natural language processing tasks. Most\nof the natural language processing models that are based on deep learning\ntechniques use already pre-trained distributed word representations, commonly\ncalled word embeddings. Determining the most qualitative word embeddings is of\ncrucial importance for such models. However, selecting the appropriate word\nembeddings is a perplexing task since the projected embedding space is not\nintuitive to humans. In this paper, we explore different approaches for\ncreating distributed word representations. We perform an intrinsic evaluation\nof several state-of-the-art word embedding methods. Their performance on\ncapturing word similarities is analysed with existing benchmark datasets for\nword pairs similarities. The research in this paper conducts a correlation\nanalysis between ground truth word similarities and similarities obtained by\ndifferent word embedding methods.",
            "journal ref": "6th International Conference on Natural Language Processing (NATP\n  2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03846v4",
            "date": "2020-05-08 04:19:57+00:00",
            "title": "Synchronous Bidirectional Learning for Multilingual Lip Reading",
            "authors": [
                "Mingshuang Luo",
                "Shuang Yang",
                "Xilin Chen",
                "Zitao Liu",
                "Shiguang Shan"
            ],
            "abstract": "Lip reading has received increasing attention in recent years. This paper\nfocuses on the synergy of multilingual lip reading. There are about as many as\n7000 languages in the world, which implies that it is impractical to train\nseparate lip reading models with large-scale data for each language. Although\neach language has its own linguistic and pronunciation rules, the lip movements\nof all languages share similar patterns due to the common structures of human\norgans. Based on this idea, we try to explore the synergized learning of\nmultilingual lip reading in this paper, and further propose a synchronous\nbidirectional learning (SBL) framework for effective synergy of multilingual\nlip reading. We firstly introduce phonemes as our modeling units for the\nmultilingual setting here. Phonemes are more closely related with the lip\nmovements than the alphabet letters. At the same time, similar phonemes always\nlead to similar visual patterns no matter which type the target language is.\nThen, a novel SBL block is proposed to learn the rules for each language in a\nfill-in-the-blank way. Specifically, the model has to learn to infer the target\nunit given its bidirectional context, which could represent the composition\nrules of phonemes for each language. To make the learning process more targeted\nat each particular language, an extra task of predicting the language identity\nis introduced in the learning process. Finally, a thorough comparison on LRW\n(English) and LRW-1000 (Mandarin) is performed, which shows the promising\nbenefits from the synergized learning of different languages and also reports a\nnew state-of-the-art result on both datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2005.03954v3",
            "date": "2020-05-08 11:01:21+00:00",
            "title": "Towards Conversational Recommendation over Multi-Type Dialogs",
            "authors": [
                "Zeming Liu",
                "Haifeng Wang",
                "Zheng-Yu Niu",
                "Hua Wu",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "We propose a new task of conversational recommendation over multi-type\ndialogs, where the bots can proactively and naturally lead a conversation from\na non-recommendation dialog (e.g., QA) to a recommendation dialog, taking into\naccount user's interests and feedback. To facilitate the study of this task, we\ncreate a human-to-human Chinese dialog dataset \\emph{DuRecDial} (about 10k\ndialogs, 156k utterances), which contains multiple sequential dialogs for every\npair of a recommendation seeker (user) and a recommender (bot). In each dialog,\nthe recommender proactively leads a multi-type dialog to approach\nrecommendation targets and then makes multiple recommendations with rich\ninteraction behavior. This dataset allows us to systematically investigate\ndifferent parts of the overall problem, e.g., how to naturally lead a dialog,\nhow to interact with users for recommendation. Finally we establish baseline\nresults on DuRecDial for future studies. Dataset and codes are publicly\navailable at\nhttps://github.com/PaddlePaddle/models/tree/develop/PaddleNLP/Research/ACL2020-DuRecDial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.03959v1",
            "date": "2020-05-08 11:16:58+00:00",
            "title": "On Vocabulary Reliance in Scene Text Recognition",
            "authors": [
                "Zhaoyi Wan",
                "Jielei Zhang",
                "Liang Zhang",
                "Jiebo Luo",
                "Cong Yao"
            ],
            "abstract": "The pursuit of high performance on public benchmarks has been the driving\nforce for research in scene text recognition, and notable progress has been\nachieved. However, a close investigation reveals a startling fact that the\nstate-of-the-art methods perform well on images with words within vocabulary\nbut generalize poorly to images with words outside vocabulary. We call this\nphenomenon \"vocabulary reliance\". In this paper, we establish an analytical\nframework to conduct an in-depth study on the problem of vocabulary reliance in\nscene text recognition. Key findings include: (1) Vocabulary reliance is\nubiquitous, i.e., all existing algorithms more or less exhibit such\ncharacteristic; (2) Attention-based decoders prove weak in generalizing to\nwords outside vocabulary and segmentation-based decoders perform well in\nutilizing visual features; (3) Context modeling is highly coupled with the\nprediction layers. These findings provide new insights and can benefit future\nresearch in scene text recognition. Furthermore, we propose a simple yet\neffective mutual learning strategy to allow models of two families\n(attention-based and segmentation-based) to learn collaboratively. This remedy\nalleviates the problem of vocabulary reliance and improves the overall scene\ntext recognition performance.",
            "journal ref": "EEE/CVF Conferences on Computer Vision and Pattern Recognition\n  (CVPR), Seattle, WA, June 2020",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2005.04118v1",
            "date": "2020-05-08 15:48:31+00:00",
            "title": "Beyond Accuracy: Behavioral Testing of NLP models with CheckList",
            "authors": [
                "Marco Tulio Ribeiro",
                "Tongshuang Wu",
                "Carlos Guestrin",
                "Sameer Singh"
            ],
            "abstract": "Although measuring held-out accuracy has been the primary approach to\nevaluate generalization, it often overestimates the performance of NLP models,\nwhile alternative approaches for evaluating models either focus on individual\ntasks or on specific behaviors. Inspired by principles of behavioral testing in\nsoftware engineering, we introduce CheckList, a task-agnostic methodology for\ntesting NLP models. CheckList includes a matrix of general linguistic\ncapabilities and test types that facilitate comprehensive test ideation, as\nwell as a software tool to generate a large and diverse number of test cases\nquickly. We illustrate the utility of CheckList with tests for three tasks,\nidentifying critical failures in both commercial and state-of-art models. In a\nuser study, a team responsible for a commercial sentiment analysis model found\nnew and actionable bugs in an extensively tested model. In another user study,\nNLP practitioners with CheckList created twice as many tests, and found almost\nthree times as many bugs as users without it.",
            "journal ref": "Association for Computational Linguistics (ACL), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04147v1",
            "date": "2020-05-08 16:42:54+00:00",
            "title": "Quantum Natural Language Processing on Near-Term Quantum Computers",
            "authors": [
                "Konstantinos Meichanetzidis",
                "Stefano Gogioso",
                "Giovanni De Felice",
                "Nicol\u00f2 Chiappori",
                "Alexis Toumi",
                "Bob Coecke"
            ],
            "abstract": "In this work, we describe a full-stack pipeline for natural language\nprocessing on near-term quantum computers, aka QNLP. The language modelling\nframework we employ is that of compositional distributional semantics\n(DisCoCat), which extends and complements the compositional structure of\npregroup grammars. Within this model, the grammatical reduction of a sentence\nis interpreted as a diagram, encoding a specific interaction of words according\nto the grammar. It is this interaction which, together with a specific choice\nof word embedding, realises the meaning (or \"semantics\") of a sentence.\nBuilding on the formal quantum-like nature of such interactions, we present a\nmethod for mapping DisCoCat diagrams to quantum circuits. Our methodology is\ncompatible both with NISQ devices and with established Quantum Machine Learning\ntechniques, paving the way to near-term applications of quantum technology to\nnatural language processing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04246v1",
            "date": "2020-05-08 18:00:28+00:00",
            "title": "ConvoKit: A Toolkit for the Analysis of Conversations",
            "authors": [
                "Jonathan P. Chang",
                "Caleb Chiam",
                "Liye Fu",
                "Andrew Z. Wang",
                "Justine Zhang",
                "Cristian Danescu-Niculescu-Mizil"
            ],
            "abstract": "This paper describes the design and functionality of ConvoKit, an open-source\ntoolkit for analyzing conversations and the social interactions embedded\nwithin. ConvoKit provides an unified framework for representing and\nmanipulating conversational data, as well as a large and diverse collection of\nconversational datasets. By providing an intuitive interface for exploring and\ninteracting with conversational data, this toolkit lowers the technical\nbarriers for the broad adoption of computational methods for conversational\nanalysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04277v2",
            "date": "2020-05-08 20:19:26+00:00",
            "title": "Adversarial Learning for Supervised and Semi-supervised Relation Extraction in Biomedical Literature",
            "authors": [
                "Peng Su",
                "K. Vijay-Shanker"
            ],
            "abstract": "Adversarial training is a technique of improving model performance by\ninvolving adversarial examples in the training process. In this paper, we\ninvestigate adversarial training with multiple adversarial examples to benefit\nthe relation extraction task. We also apply adversarial training technique in\nsemi-supervised scenarios to utilize unlabeled data. The evaluation results on\nprotein-protein interaction and protein subcellular localization task\nillustrate adversarial training provides improvement on the supervised model,\nand is also effective on involving unlabeled data in the semi-supervised\ntraining case. In addition, our method achieves state-of-the-art performance on\ntwo benchmarking datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04379v1",
            "date": "2020-05-09 06:28:44+00:00",
            "title": "Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation",
            "authors": [
                "Xinting Huang",
                "Jianzhong Qi",
                "Yu Sun",
                "Rui Zhang"
            ],
            "abstract": "Dialogue policy optimization often obtains feedback until task completion in\ntask-oriented dialogue systems. This is insufficient for training intermediate\ndialogue turns since supervision signals (or rewards) are only provided at the\nend of dialogues. To address this issue, reward learning has been introduced to\nlearn from state-action pairs of an optimal policy to provide turn-by-turn\nrewards. This approach requires complete state-action annotations of\nhuman-to-human dialogues (i.e., expert demonstrations), which is labor\nintensive. To overcome this limitation, we propose a novel reward learning\napproach for semi-supervised policy learning. The proposed approach learns a\ndynamics model as the reward function which models dialogue progress (i.e.,\nstate-action sequences) based on expert demonstrations, either with or without\nannotations. The dynamics model computes rewards by predicting whether the\ndialogue progress is consistent with expert demonstrations. We further propose\nto learn action embeddings for a better generalization of the reward function.\nThe proposed approach outperforms competitive policy learning baselines on\nMultiWOZ, a benchmark multi-domain dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04511v2",
            "date": "2020-05-09 20:46:02+00:00",
            "title": "Finding Universal Grammatical Relations in Multilingual BERT",
            "authors": [
                "Ethan A. Chi",
                "John Hewitt",
                "Christopher D. Manning"
            ],
            "abstract": "Recent work has found evidence that Multilingual BERT (mBERT), a\ntransformer-based multilingual masked language model, is capable of zero-shot\ncross-lingual transfer, suggesting that some aspects of its representations are\nshared cross-lingually. To better understand this overlap, we extend recent\nwork on finding syntactic trees in neural networks' internal representations to\nthe multilingual setting. We show that subspaces of mBERT representations\nrecover syntactic tree distances in languages other than English, and that\nthese subspaces are approximately shared across languages. Motivated by these\nresults, we present an unsupervised analysis method that provides evidence\nmBERT learns representations of syntactic dependency labels, in the form of\nclusters which largely agree with the Universal Dependencies taxonomy. This\nevidence suggests that even without explicit supervision, multilingual masked\nlanguage models learn certain linguistic universals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04588v2",
            "date": "2020-05-10 06:12:53+00:00",
            "title": "Transformer Based Language Models for Similar Text Retrieval and Ranking",
            "authors": [
                "Javed Qadrud-Din",
                "Ashraf Bah Rabiou",
                "Ryan Walker",
                "Ravi Soni",
                "Martin Gajek",
                "Gabriel Pack",
                "Akhil Rangaraj"
            ],
            "abstract": "Most approaches for similar text retrieval and ranking with long natural\nlanguage queries rely at some level on queries and responses having words in\ncommon with each other. Recent applications of transformer-based neural\nlanguage models to text retrieval and ranking problems have been very\npromising, but still involve a two-step process in which result candidates are\nfirst obtained through bag-of-words-based approaches, and then reranked by a\nneural transformer. In this paper, we introduce novel approaches for\neffectively applying neural transformer models to similar text retrieval and\nranking without an initial bag-of-words-based step. By eliminating the\nbag-of-words-based step, our approach is able to accurately retrieve and rank\nresults even when they have no non-stopwords in common with the query. We\naccomplish this by using bidirectional encoder representations from\ntransformers (BERT) to create vectorized representations of sentence-length\ntexts, along with a vector nearest neighbor search index. We demonstrate both\nsupervised and unsupervised means of using BERT to accomplish this task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.04684v1",
            "date": "2020-05-10 14:59:36+00:00",
            "title": "From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information",
            "authors": [
                "Shen Gao",
                "Xiuying Chen",
                "Zhaochun Ren",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "Text summarization is the research area aiming at creating a short and\ncondensed version of the original document, which conveys the main idea of the\ndocument in a few words. This research topic has started to attract the\nattention of a large community of researchers, and it is nowadays counted as\none of the most promising research areas. In general, text summarization\nalgorithms aim at using a plain text document as input and then output a\nsummary. However, in real-world applications, most of the data is not in a\nplain text format. Instead, there is much manifold information to be\nsummarized, such as the summary for a web page based on a query in the search\nengine, extreme long document (e.g., academic paper), dialog history and so on.\nIn this paper, we focus on the survey of these new summarization tasks and\napproaches in the real-world application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04712v3",
            "date": "2020-05-10 16:48:23+00:00",
            "title": "CTC-synchronous Training for Monotonic Attention Model",
            "authors": [
                "Hirofumi Inaguma",
                "Masato Mimura",
                "Tatsuya Kawahara"
            ],
            "abstract": "Monotonic chunkwise attention (MoChA) has been studied for the online\nstreaming automatic speech recognition (ASR) based on a sequence-to-sequence\nframework. In contrast to connectionist temporal classification (CTC), backward\nprobabilities cannot be leveraged in the alignment marginalization process\nduring training due to left-to-right dependency in the decoder. This results in\nthe error propagation of alignments to subsequent token generation. To address\nthis problem, we propose CTC-synchronous training (CTC-ST), in which MoChA uses\nCTC alignments to learn optimal monotonic alignments. Reference CTC alignments\nare extracted from a CTC branch sharing the same encoder with the decoder. The\nentire model is jointly optimized so that the expected boundaries from MoChA\nare synchronized with the alignments. Experimental evaluations of the TEDLIUM\nrelease-2 and Librispeech corpora show that the proposed method significantly\nimproves recognition, especially for long utterances. We also show that CTC-ST\ncan bring out the full potential of SpecAugment for MoChA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04732v2",
            "date": "2020-05-10 17:56:10+00:00",
            "title": "Towards Robustifying NLI Models Against Lexical Dataset Biases",
            "authors": [
                "Xiang Zhou",
                "Mohit Bansal"
            ],
            "abstract": "While deep learning models are making fast progress on the task of Natural\nLanguage Inference, recent studies have also shown that these models achieve\nhigh accuracy by exploiting several dataset biases, and without deep\nunderstanding of the language semantics. Using contradiction-word bias and\nword-overlapping bias as our two bias examples, this paper explores both\ndata-level and model-level debiasing methods to robustify models against\nlexical dataset biases. First, we debias the dataset through data augmentation\nand enhancement, but show that the model bias cannot be fully removed via this\nmethod. Next, we also compare two ways of directly debiasing the model without\nknowing what the dataset biases are in advance. The first approach aims to\nremove the label bias at the embedding level. The second approach employs a\nbag-of-words sub-model to capture the features that are likely to exploit the\nbias and prevents the original model from learning these biased features by\nforcing orthogonality between these two sub-models. We performed evaluations on\nnew balanced datasets extracted from the original MNLI dataset as well as the\nNLI stress tests, and show that the orthogonality approach is better at\ndebiasing the model while maintaining competitive overall accuracy. Our code\nand data are available at: https://github.com/owenzx/LexicalDebias-ACL2020",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04816v1",
            "date": "2020-05-11 00:20:33+00:00",
            "title": "Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation",
            "authors": [
                "Aditya Siddhant",
                "Ankur Bapna",
                "Yuan Cao",
                "Orhan Firat",
                "Mia Chen",
                "Sneha Kudugunta",
                "Naveen Arivazhagan",
                "Yonghui Wu"
            ],
            "abstract": "Over the last few years two promising research directions in low-resource\nneural machine translation (NMT) have emerged. The first focuses on utilizing\nhigh-resource languages to improve the quality of low-resource languages via\nmultilingual NMT. The second direction employs monolingual data with\nself-supervision to pre-train translation models, followed by fine-tuning on\nsmall amounts of supervised data. In this work, we join these two lines of\nresearch and demonstrate the efficacy of monolingual data with self-supervision\nin multilingual NMT. We offer three major results: (i) Using monolingual data\nsignificantly boosts the translation quality of low-resource languages in\nmultilingual models. (ii) Self-supervision improves zero-shot translation\nquality in multilingual models. (iii) Leveraging monolingual data with\nself-supervision provides a viable path towards adding new languages to\nmultilingual models, getting up to 33 BLEU on ro-en translation without any\nparallel data or back-translation.",
            "journal ref": "ACL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04862v4",
            "date": "2020-05-11 04:45:02+00:00",
            "title": "Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition",
            "authors": [
                "Ye Bai",
                "Jiangyan Yi",
                "Jianhua Tao",
                "Zhengkun Tian",
                "Zhengqi Wen",
                "Shuai Zhang"
            ],
            "abstract": "Although attention based end-to-end models have achieved promising\nperformance in speech recognition, the multi-pass forward computation in\nbeam-search increases inference time cost, which limits their practical\napplications. To address this issue, we propose a non-autoregressive end-to-end\nspeech recognition system called LASO (listen attentively, and spell once).\nBecause of the non-autoregressive property, LASO predicts a textual token in\nthe sequence without the dependence on other tokens. Without beam-search, the\none-pass propagation much reduces inference time cost of LASO. And because the\nmodel is based on the attention based feedforward structure, the computation\ncan be implemented in parallel efficiently. We conduct experiments on publicly\navailable Chinese dataset AISHELL-1. LASO achieves a character error rate of\n6.4%, which outperforms the state-of-the-art autoregressive transformer model\n(6.7%). The average inference latency is 21 ms, which is 1/50 of the\nautoregressive transformer model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.04929v1",
            "date": "2020-05-11 08:51:30+00:00",
            "title": "Towards logical negation for compositional distributional semantics",
            "authors": [
                "Martha Lewis"
            ],
            "abstract": "The categorical compositional distributional model of meaning gives the\ncomposition of words into phrases and sentences pride of place. However, it has\nso far lacked a model of logical negation. This paper gives some steps towards\nproviding this operator, modelling it as a version of projection onto the\nsubspace orthogonal to a word. We give a small demonstration of the operators\nperformance in a sentence entailment task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.04938v1",
            "date": "2020-05-11 09:07:46+00:00",
            "title": "A Deep Learning Approach for Automatic Detection of Fake News",
            "authors": [
                "Tanik Saikh",
                "Arkadipta De",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Fake news detection is a very prominent and essential task in the field of\njournalism. This challenging problem is seen so far in the field of politics,\nbut it could be even more challenging when it is to be determined in the\nmulti-domain platform. In this paper, we propose two effective models based on\ndeep learning for solving fake news detection problem in online news contents\nof multiple domains. We evaluate our techniques on the two recently released\ndatasets, namely FakeNews AMT and Celebrity for fake news detection. The\nproposed systems yield encouraging performance, outperforming the current\nhandcrafted feature engineering based state-of-the-art system with a\nsignificant margin of 3.08% and 9.3% by the two models, respectively. In order\nto exploit the datasets, available for the related tasks, we perform\ncross-domain analysis (i.e. model trained on FakeNews AMT and tested on\nCelebrity and vice versa) to explore the applicability of our systems across\nthe domains.",
            "journal ref": "Proceedings of the 16th International Conference on Natural\n  Language Processing (ICON 2019)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05114v1",
            "date": "2020-05-11 13:56:58+00:00",
            "title": "Evaluating Sparse Interpretable Word Embeddings for Biomedical Domain",
            "authors": [
                "Mohammad Amin Samadi",
                "Mohammad Sadegh Akhondzadeh",
                "Sayed Jalal Zahabi",
                "Mohammad Hossein Manshaei",
                "Zeinab Maleki",
                "Payman Adibi"
            ],
            "abstract": "Word embeddings have found their way into a wide range of natural language\nprocessing tasks including those in the biomedical domain. While these vector\nrepresentations successfully capture semantic and syntactic word relations,\nhidden patterns and trends in the data, they fail to offer interpretability.\nInterpretability is a key means to justification which is an integral part when\nit comes to biomedical applications. We present an inclusive study on\ninterpretability of word embeddings in the medical domain, focusing on the role\nof sparse methods. Qualitative and quantitative measurements and metrics for\ninterpretability of word vector representations are provided. For the\nquantitative evaluation, we introduce an extensive categorized dataset that can\nbe used to quantify interpretability based on category theory. Intrinsic and\nextrinsic evaluation of the studied methods are also presented. As for the\nlatter, we propose datasets which can be utilized for effective extrinsic\nevaluation of word vectors in the biomedical domain. Based on our experiments,\nit is seen that sparse word vectors show far more interpretability while\npreserving the performance of their original vectors in downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05240v1",
            "date": "2020-05-11 16:31:08+00:00",
            "title": "Commonsense Evidence Generation and Injection in Reading Comprehension",
            "authors": [
                "Ye Liu",
                "Tao Yang",
                "Zeyu You",
                "Wei Fan",
                "Philip S. Yu"
            ],
            "abstract": "Human tackle reading comprehension not only based on the given context itself\nbut often rely on the commonsense beyond. To empower the machine with\ncommonsense reasoning, in this paper, we propose a Commonsense Evidence\nGeneration and Injection framework in reading comprehension, named CEGI. The\nframework injects two kinds of auxiliary commonsense evidence into\ncomprehensive reading to equip the machine with the ability of rational\nthinking. Specifically, we build two evidence generators: the first generator\naims to generate textual evidence via a language model; the other generator\naims to extract factual evidence (automatically aligned text-triples) from a\ncommonsense knowledge graph after graph completion. Those evidences incorporate\ncontextual commonsense and serve as the additional inputs to the model.\nThereafter, we propose a deep contextual encoder to extract semantic\nrelationships among the paragraph, question, option, and evidence. Finally, we\nemploy a capsule network to extract different linguistic units (word and\nphrase) from the relations, and dynamically predict the optimal option based on\nthe extracted units. Experiments on the CosmosQA dataset demonstrate that the\nproposed CEGI model outperforms the current state-of-the-art approaches and\nachieves the accuracy (83.6%) on the leaderboard.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.05442v2",
            "date": "2020-05-11 21:23:43+00:00",
            "title": "On the Generation of Medical Dialogues for COVID-19",
            "authors": [
                "Wenmian Yang",
                "Guangtao Zeng",
                "Bowen Tan",
                "Zeqian Ju",
                "Subrato Chakravorty",
                "Xuehai He",
                "Shu Chen",
                "Xingyi Yang",
                "Qingyang Wu",
                "Zhou Yu",
                "Eric Xing",
                "Pengtao Xie"
            ],
            "abstract": "Under the pandemic of COVID-19, people experiencing COVID19-related symptoms\nor exposed to risk factors have a pressing need to consult doctors. Due to\nhospital closure, a lot of consulting services have been moved online. Because\nof the shortage of medical professionals, many people cannot receive online\nconsultations timely. To address this problem, we aim to develop a medical\ndialogue system that can provide COVID19-related consultations. We collected\ntwo dialogue datasets -- CovidDialog -- (in English and Chinese respectively)\ncontaining conversations between doctors and patients about COVID-19. On these\ntwo datasets, we train several dialogue generation models based on Transformer,\nGPT, and BERT-GPT. Since the two COVID-19 dialogue datasets are small in size,\nwhich bear high risk of overfitting, we leverage transfer learning to mitigate\ndata deficiency. Specifically, we take the pretrained models of Transformer,\nGPT, and BERT-GPT on dialog datasets and other large-scale texts, then finetune\nthem on our CovidDialog tasks. We perform both automatic and human evaluation\nof responses generated by these models. The results show that the generated\nresponses are promising in being doctor-like, relevant to the conversation\nhistory, and clinically informative. The data and code are available at\nhttps://github.com/UCSD-AI4H/COVID-Dialogue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05447v1",
            "date": "2020-05-11 21:33:33+00:00",
            "title": "Luganda Text-to-Speech Machine",
            "authors": [
                "Irene Nandutu",
                "Ernest Mwebaze"
            ],
            "abstract": "In Uganda, Luganda is the most spoken native language. It is used for\ncommunication in informal as well as formal business transactions. The\ndevelopment of technology startups globally related to TTS has mainly been with\nlanguages like English, French, etc. These are added in TTS engines by Google,\nMicrosoft among others, allowing developers in these regions to innovate TTS\nproducts. Luganda is not supported because the language is not built and\ntrained on these engines. In this study, we analyzed the Luganda language\nstructure and constructions and then proposed and developed a Luganda TTS. The\nsystem was built and trained using locally sourced Luganda language text and\naudio. The engine is now able to capture text and reads it aloud. We tested the\naccuracy using MRT and MOS. MRT and MOS tests results are quite good with MRT\nhaving better results. The results general score was 71%. This study will\nenhance previous solutions to NLP gaps in Uganda, as well as provide raw data\nsuch that other research in this area can take place.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05570v1",
            "date": "2020-05-12 06:34:42+00:00",
            "title": "Simultaneous paraphrasing and translation by fine-tuning Transformer models",
            "authors": [
                "Rakesh Chada"
            ],
            "abstract": "This paper describes the third place submission to the shared task on\nsimultaneous translation and paraphrasing for language education at the 4th\nworkshop on Neural Generation and Translation (WNGT) for ACL 2020. The final\nsystem leverages pre-trained translation models and uses a Transformer\narchitecture combined with an oversampling strategy to achieve a competitive\nperformance. This system significantly outperforms the baseline on Hungarian\n(27% absolute improvement in Weighted Macro F1 score) and Portuguese (33%\nabsolute improvement) languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05672v1",
            "date": "2020-05-12 10:32:03+00:00",
            "title": "Learning and Evaluating Emotion Lexicons for 91 Languages",
            "authors": [
                "Sven Buechel",
                "Susanna R\u00fccker",
                "Udo Hahn"
            ],
            "abstract": "Emotion lexicons describe the affective meaning of words and thus constitute\na centerpiece for advanced sentiment and emotion analysis. Yet, manually\ncurated lexicons are only available for a handful of languages, leaving most\nlanguages of the world without such a precious resource for downstream\napplications. Even worse, their coverage is often limited both in terms of the\nlexical units they contain and the emotional variables they feature. In order\nto break this bottleneck, we here introduce a methodology for creating almost\narbitrarily large emotion lexicons for any target language. Our approach\nrequires nothing but a source language emotion lexicon, a bilingual word\ntranslation model, and a target language embedding model. Fulfilling these\nrequirements for 91 languages, we are able to generate representationally rich\nhigh-coverage lexicons comprising eight emotional variables with more than 100k\nlexical entries each. We evaluated the automatically generated lexicons against\nhuman judgment from 26 datasets, spanning 12 typologically diverse languages,\nand found that our approach produces results in line with state-of-the-art\nmonolingual approaches to lexicon creation and even surpasses human reliability\nfor some languages and variables. Code and data are available at\nhttps://github.com/JULIELab/MEmoLon archived under DOI\nhttps://doi.org/10.5281/zenodo.3779901.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05727v1",
            "date": "2020-05-12 12:41:14+00:00",
            "title": "Dynamic Memory Induction Networks for Few-Shot Text Classification",
            "authors": [
                "Ruiying Geng",
                "Binhua Li",
                "Yongbin Li",
                "Jian Sun",
                "Xiaodan Zhu"
            ],
            "abstract": "This paper proposes Dynamic Memory Induction Networks (DMIN) for few-shot\ntext classification. The model utilizes dynamic routing to provide more\nflexibility to memory-based few-shot learning in order to better adapt the\nsupport sets, which is a critical capacity of few-shot classification models.\nBased on that, we further develop induction models with query information,\naiming to enhance the generalization ability of meta-learning. The proposed\nmodel achieves new state-of-the-art results on the miniRCV1 and ODIC dataset,\nimproving the best performance (accuracy) by 2~4%. Detailed analysis is further\nperformed to show the effectiveness of each component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05754v1",
            "date": "2020-05-12 13:29:38+00:00",
            "title": "Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answering",
            "authors": [
                "Angrosh Mandya",
                "James O'Neill",
                "Danushka Bollegala",
                "Frans Coenen"
            ],
            "abstract": "The Conversational Question Answering (CoQA) task involves answering a\nsequence of inter-related conversational questions about a contextual\nparagraph. Although existing approaches employ human-written ground-truth\nanswers for answering conversational questions at test time, in a realistic\nscenario, the CoQA model will not have any access to ground-truth answers for\nthe previous questions, compelling the model to rely upon its own previously\npredicted answers for answering the subsequent questions. In this paper, we\nfind that compounding errors occur when using previously predicted answers at\ntest time, significantly lowering the performance of CoQA systems. To solve\nthis problem, we propose a sampling strategy that dynamically selects between\ntarget answers and model predictions during training, thereby closely\nsimulating the situation at test time. Further, we analyse the severity of this\nphenomena as a function of the question type, conversation length and domain\ntype.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.05763v1",
            "date": "2020-05-12 13:40:06+00:00",
            "title": "WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge",
            "authors": [
                "Hongming Zhang",
                "Xinran Zhao",
                "Yangqiu Song"
            ],
            "abstract": "In this paper, we present the first comprehensive categorization of essential\ncommonsense knowledge for answering the Winograd Schema Challenge (WSC). For\neach of the questions, we invite annotators to first provide reasons for making\ncorrect decisions and then categorize them into six major knowledge categories.\nBy doing so, we better understand the limitation of existing methods (i.e.,\nwhat kind of knowledge cannot be effectively represented or inferred with\nexisting methods) and shed some light on the commonsense knowledge that we need\nto acquire in the future for better commonsense reasoning. Moreover, to\ninvestigate whether current WSC models can understand the commonsense or they\nsimply solve the WSC questions based on the statistical bias of the dataset, we\nleverage the collected reasons to develop a new task called WinoWhy, which\nrequires models to distinguish plausible reasons from very similar but wrong\nreasons for all WSC questions. Experimental results prove that even though\npre-trained language representation models have achieved promising progress on\nthe original WSC dataset, they are still struggling at WinoWhy. Further\nexperiments show that even though supervised models can achieve better\nperformance, the performance of these models can be sensitive to the dataset\ndistribution. WinoWhy and all codes are available at:\nhttps://github.com/HKUST-KnowComp/WinoWhy.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.05814v2",
            "date": "2020-05-12 14:27:19+00:00",
            "title": "A Report on the 2020 Sarcasm Detection Shared Task",
            "authors": [
                "Debanjan Ghosh",
                "Avijit Vajpayee",
                "Smaranda Muresan"
            ],
            "abstract": "Detecting sarcasm and verbal irony is critical for understanding people's\nactual sentiments and beliefs. Thus, the field of sarcasm analysis has become a\npopular research problem in natural language processing. As the community\nworking on computational approaches for sarcasm detection is growing, it is\nimperative to conduct benchmarking studies to analyze the current\nstate-of-the-art, facilitating progress in this area. We report on the shared\ntask on sarcasm detection we conducted as a part of the 2nd Workshop on\nFigurative Language Processing (FigLang 2020) at ACL 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05921v3",
            "date": "2020-05-12 16:58:48+00:00",
            "title": "Intersectional Bias in Hate Speech and Abusive Language Datasets",
            "authors": [
                "Jae Yeon Kim",
                "Carlos Ortiz",
                "Sarah Nam",
                "Sarah Santiago",
                "Vivek Datta"
            ],
            "abstract": "Algorithms are widely applied to detect hate speech and abusive language in\nsocial media. We investigated whether the human-annotated data used to train\nthese algorithms are biased. We utilized a publicly available annotated Twitter\ndataset (Founta et al. 2018) and classified the racial, gender, and party\nidentification dimensions of 99,996 tweets. The results showed that African\nAmerican tweets were up to 3.7 times more likely to be labeled as abusive, and\nAfrican American male tweets were up to 77% more likely to be labeled as\nhateful compared to the others. These patterns were statistically significant\nand robust even when party identification was added as a control variable. This\nstudy provides the first systematic evidence on intersectional bias in datasets\nof hate speech and abusive language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.05927v1",
            "date": "2020-05-12 17:10:13+00:00",
            "title": "Semantic Scaffolds for Pseudocode-to-Code Generation",
            "authors": [
                "Ruiqi Zhong",
                "Mitchell Stern",
                "Dan Klein"
            ],
            "abstract": "We propose a method for program generation based on semantic scaffolds,\nlightweight structures representing the high-level semantic and syntactic\ncomposition of a program. By first searching over plausible scaffolds then\nusing these as constraints for a beam search over programs, we achieve better\ncoverage of the search space when compared with existing techniques. We apply\nour hierarchical search method to the SPoC dataset for pseudocode-to-code\ngeneration, in which we are given line-level natural language pseudocode\nannotations and aim to produce a program satisfying execution-based test cases.\nBy using semantic scaffolds during inference, we achieve a 10% absolute\nimprovement in top-100 accuracy over the previous state-of-the-art.\nAdditionally, we require only 11 candidates to reach the top-3000 performance\nof the previous best approach when tested against unseen problems,\ndemonstrating a substantial improvement in efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06012v4",
            "date": "2020-05-02 10:23:27+00:00",
            "title": "Mega-COV: A Billion-Scale Dataset of 100+ Languages for COVID-19",
            "authors": [
                "Muhammad Abdul-Mageed",
                "AbdelRahim Elmadany",
                "El Moatez Billah Nagoudi",
                "Dinesh Pabbi",
                "Kunal Verma",
                "Rannie Lin"
            ],
            "abstract": "We describe Mega-COV, a billion-scale dataset from Twitter for studying\nCOVID-19. The dataset is diverse (covers 268 countries), longitudinal (goes as\nback as 2007), multilingual (comes in 100+ languages), and has a significant\nnumber of location-tagged tweets (~169M tweets). We release tweet IDs from the\ndataset. We also develop and release two powerful models, one for identifying\nwhether or not a tweet is related to the pandemic (best F1=97%) and another for\ndetecting misinformation about COVID-19 (best F1=92%). A human annotation study\nreveals the utility of our models on a subset of Mega-COV. Our data and models\ncan be useful for studying a wide host of phenomena related to the pandemic.\nMega-COV and our models are publicly available.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.06117v1",
            "date": "2020-05-13 02:07:54+00:00",
            "title": "INFOTABS: Inference on Tables as Semi-structured Data",
            "authors": [
                "Vivek Gupta",
                "Maitrey Mehta",
                "Pegah Nokhiz",
                "Vivek Srikumar"
            ],
            "abstract": "In this paper, we observe that semi-structured tabulated text is ubiquitous;\nunderstanding them requires not only comprehending the meaning of text\nfragments, but also implicit relationships between them. We argue that such\ndata can prove as a testing ground for understanding how we reason about\ninformation. To study this, we introduce a new dataset called INFOTABS,\ncomprising of human-written textual hypotheses based on premises that are\ntables extracted from Wikipedia info-boxes. Our analysis shows that the\nsemi-structured, multi-domain and heterogeneous nature of the premises admits\ncomplex, multi-faceted reasoning. Experiments reveal that, while human\nannotators agree on the relationships between a table-hypothesis pair, several\nstandard modeling strategies are unsuccessful at the task, suggesting that\nreasoning about tables can pose a difficult modeling challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06166v1",
            "date": "2020-05-13 06:06:23+00:00",
            "title": "Parallel Corpus Filtering via Pre-trained Language Models",
            "authors": [
                "Boliang Zhang",
                "Ajay Nagesh",
                "Kevin Knight"
            ],
            "abstract": "Web-crawled data provides a good source of parallel corpora for training\nmachine translation models. It is automatically obtained, but extremely noisy,\nand recent work shows that neural machine translation systems are more\nsensitive to noise than traditional statistical machine translation methods. In\nthis paper, we propose a novel approach to filter out noisy sentence pairs from\nweb-crawled corpora via pre-trained language models. We measure sentence\nparallelism by leveraging the multilingual capability of BERT and use the\nGenerative Pre-training (GPT) language model as a domain filter to balance data\ndomains. We evaluate the proposed method on the WMT 2018 Parallel Corpus\nFiltering shared task, and on our own web-crawled Japanese-Chinese parallel\ncorpus. Our method significantly outperforms baselines and achieves a new\nstate-of-the-art. In an unsupervised setting, our method achieves comparable\nperformance to the top-1 supervised method. We also evaluate on a web-crawled\nJapanese-Chinese parallel corpus that we make publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06420v3",
            "date": "2020-05-13 16:51:02+00:00",
            "title": "The Unstoppable Rise of Computational Linguistics in Deep Learning",
            "authors": [
                "James Henderson"
            ],
            "abstract": "In this paper, we trace the history of neural networks applied to natural\nlanguage understanding tasks, and identify key contributions which the nature\nof language has made to the development of neural network architectures. We\nfocus on the importance of variable binding and its instantiation in\nattention-based models, and argue that Transformer is not a sequence model but\nan induced-structure model. This perspective leads to predictions of the\nchallenges facing research in deep learning architectures for natural language\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06517v1",
            "date": "2020-05-13 18:39:00+00:00",
            "title": "Towards Automatic building of Human-Machine Conversational System to support Maintenance Processes",
            "authors": [
                "Elena Coli",
                "Nicola Melluso",
                "Gualtiero Fantoni",
                "Daniele Mazzei"
            ],
            "abstract": "Companies are dealing with many cognitive changes with the introduction of\nthe Industry 4.0 paradigm. In this constantly changing environment, knowledge\nmanagement is a key factor. Dialog systems, being able to hold a conversation\nwith humans, could support the knowledge management in business environment.\nAlthough, these systems are currently hand-coded and need the intervention of a\nhuman being in writing all the possible questions and answers, and then\nplanning the interactions. This process, besides being time-consuming, is not\nscalable. Conversely, a dialog system, also referred to as chatbot, can be\nbuilt from scratch by simply extracting rules from technical documentation. So,\nthe goal of this research is designing a methodology for automatic building of\nhuman-machine conversational system, able to interact in an industrial\nenvironment. An initial taxonomy, containing entities expected to be found in\nmaintenance manuals, is used to identify the relevant sentences of a manual\nprovided by the company BOBST SA and applying text mining techniques, it is\nautomatically expanded. The final result is a taxonomy network representing the\nentities and their relation, that will be used in future works for managing the\ninteractions of a maintenance chatbot.",
            "journal ref": "R&D Management Conference 2019",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2005.06540v1",
            "date": "2020-05-13 19:14:37+00:00",
            "title": "Deep Learning for Political Science",
            "authors": [
                "Kakia Chatsiou",
                "Slava Jankin Mikhaylov"
            ],
            "abstract": "Political science, and social science in general, have traditionally been\nusing computational methods to study areas such as voting behavior, policy\nmaking, international conflict, and international development. More recently,\nincreasingly available quantities of data are being combined with improved\nalgorithms and affordable computational resources to predict, learn, and\ndiscover new insights from data that is large in volume and variety. New\ndevelopments in the areas of machine learning, deep learning, natural language\nprocessing (NLP), and, more generally, artificial intelligence (AI) are opening\nup new opportunities for testing theories and evaluating the impact of\ninterventions and programs in a more dynamic and effective way. Applications\nusing large volumes of structured and unstructured data are becoming common in\ngovernment and industry, and increasingly also in social science research. This\nchapter offers an introduction to such methods drawing examples from political\nscience. Focusing on the areas where the strengths of the methods coincide with\nchallenges in these fields, the chapter first presents an introduction to AI\nand its core technology - machine learning, with its rapidly developing\nsubfield of deep learning. The discussion of deep neural networks is\nillustrated with the NLP tasks that are relevant to political science. The\nlatest advances in deep learning methods for NLP are also reviewed, together\nwith their potential for improving information extraction and pattern\nrecognition from political science texts.",
            "journal ref": "The SAGE Handbook of Research Methods in Political Science and\n  International Relations, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06600v1",
            "date": "2020-05-02 04:46:52+00:00",
            "title": "A Comprehensive Survey of Grammar Error Correction",
            "authors": [
                "Yu Wang",
                "Yuelin Wang",
                "Jie Liu",
                "Zhuo Liu"
            ],
            "abstract": "Grammar error correction (GEC) is an important application aspect of natural\nlanguage processing techniques. The past decade has witnessed significant\nprogress achieved in GEC for the sake of increasing popularity of machine\nlearning and deep learning, especially in late 2010s when near human-level GEC\nsystems are available. However, there is no prior work focusing on the whole\nrecapitulation of the progress. We present the first survey in GEC for a\ncomprehensive retrospect of the literature in this area. We first give the\nintroduction of five public datasets, data annotation schema, two important\nshared tasks and four standard evaluation metrics. More importantly, we discuss\nfour kinds of basic approaches, including statistical machine translation based\napproach, neural machine translation based approach, classification based\napproach and language model based approach, six commonly applied performance\nboosting techniques for GEC systems and two data augmentation methods. Since\nGEC is typically viewed as a sister task of machine translation, many GEC\nsystems are based on neural machine translation (NMT) approaches, where the\nneural sequence-to-sequence model is applied. Similarly, some performance\nboosting techniques are adapted from machine translation and are successfully\ncombined with GEC systems for enhancement on the final performance.\nFurthermore, we conduct an analysis in level of basic approaches, performance\nboosting techniques and integrated GEC systems based on their experiment\nresults respectively for more clear patterns and conclusions. Finally, we\ndiscuss five prospective directions for future GEC researches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06601v1",
            "date": "2020-04-30 03:09:17+00:00",
            "title": "Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification",
            "authors": [
                "Tengteng Zhang",
                "Yiqin Yu",
                "Jing Mei",
                "Zefang Tang",
                "Xiang Zhang",
                "Shaochun Li"
            ],
            "abstract": "The PICO framework (Population, Intervention, Comparison, and Outcome) is\nusually used to formulate evidence in the medical domain. The major task of\nPICO extraction is to extract sentences from medical literature and classify\nthem into each class. However, in most circumstances, there will be more than\none evidences in an extracted sentence even it has been categorized to a\ncertain class. In order to address this problem, we propose a step-wise disease\nNamed Entity Recognition (DNER) extraction and PICO identification method. With\nour method, sentences in paper title and abstract are first classified into\ndifferent classes of PICO, and medical entities are then identified and\nclassified into P and O. Different kinds of deep learning frameworks are used\nand experimental results show that our method will achieve high performance and\nfine-grained extraction results comparing with conventional PICO extraction\nworks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06602v3",
            "date": "2020-04-30 13:18:29+00:00",
            "title": "CIRCE at SemEval-2020 Task 1: Ensembling Context-Free and Context-Dependent Word Representations",
            "authors": [
                "Martin P\u00f6msl",
                "Roman Lyapin"
            ],
            "abstract": "This paper describes the winning contribution to SemEval-2020 Task 1:\nUnsupervised Lexical Semantic Change Detection (Subtask 2) handed in by team UG\nStudent Intern. We present an ensemble model that makes predictions based on\ncontext-free and context-dependent word representations. The key findings are\nthat (1) context-free word representations are a powerful and robust baseline,\n(2) a sentence classification objective can be used to obtain useful\ncontext-dependent word representations, and (3) combining those representations\nincreases performance on some datasets while decreasing performance on others.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06606v2",
            "date": "2020-05-03 05:00:50+00:00",
            "title": "Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation",
            "authors": [
                "Xuanli He",
                "Gholamreza Haffari",
                "Mohammad Norouzi"
            ],
            "abstract": "This paper introduces Dynamic Programming Encoding (DPE), a new segmentation\nalgorithm for tokenizing sentences into subword units. We view the subword\nsegmentation of output sentences as a latent variable that should be\nmarginalized out for learning and inference. A mixed character-subword\ntransformer is proposed, which enables exact log marginal likelihood estimation\nand exact MAP inference to find target segmentations with maximum posterior\nprobability. DPE uses a lightweight mixed character-subword transformer as a\nmeans of pre-processing parallel data to segment output sentences using dynamic\nprogramming. Empirical results on machine translation suggest that DPE is\neffective for segmenting output sentences and can be combined with BPE dropout\nfor stochastic segmentation of source sentences. DPE achieves an average\nimprovement of 0.9 BLEU over BPE (Sennrich et al., 2016) and an average\nimprovement of 0.55 BLEU over BPE dropout (Provilkov et al., 2019) on several\nWMT datasets including English <=> (German, Romanian, Estonian, Finnish,\nHungarian).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06609v1",
            "date": "2020-05-08 08:36:59+00:00",
            "title": "Neural Machine Translation for South Africa's Official Languages",
            "authors": [
                "Laura Martinus",
                "Jason Webster",
                "Joanne Moonsamy",
                "Moses Shaba Jnr",
                "Ridha Moosa",
                "Robert Fairon"
            ],
            "abstract": "Recent advances in neural machine translation (NMT) have led to\nstate-of-the-art results for many European-based translation tasks. However,\ndespite these advances, there is has been little focus in applying these\nmethods to African languages. In this paper, we seek to address this gap by\ncreating an NMT benchmark BLEU score between English and the ten remaining\nofficial languages in South Africa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06624v1",
            "date": "2020-05-08 14:04:18+00:00",
            "title": "Comparative Analysis of Text Classification Approaches in Electronic Health Records",
            "authors": [
                "Aurelie Mascio",
                "Zeljko Kraljevic",
                "Daniel Bean",
                "Richard Dobson",
                "Robert Stewart",
                "Rebecca Bendayan",
                "Angus Roberts"
            ],
            "abstract": "Text classification tasks which aim at harvesting and/or organizing\ninformation from electronic health records are pivotal to support clinical and\ntranslational research. However these present specific challenges compared to\nother classification tasks, notably due to the particular nature of the medical\nlexicon and language used in clinical records. Recent advances in embedding\nmethods have shown promising results for several clinical tasks, yet there is\nno exhaustive comparison of such approaches with other commonly used word\nrepresentations and classification models. In this work, we analyse the impact\nof various word representations, text pre-processing and classification\nalgorithms on the performance of four different text classification tasks. The\nresults show that traditional approaches, when tailored to the specific\nlanguage and structure of the text inherent to the classification task, can\nachieve or exceed the performance of more recent ones based on contextual\nembeddings such as BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06628v1",
            "date": "2020-05-09 21:56:04+00:00",
            "title": "schuBERT: Optimizing Elements of BERT",
            "authors": [
                "Ashish Khetan",
                "Zohar Karnin"
            ],
            "abstract": "Transformers \\citep{vaswani2017attention} have gradually become a key\ncomponent for many state-of-the-art natural language representation models. A\nrecent Transformer based model- BERT \\citep{devlin2018bert} achieved\nstate-of-the-art results on various natural language processing tasks,\nincluding GLUE, SQuAD v1.1, and SQuAD v2.0. This model however is\ncomputationally prohibitive and has a huge number of parameters. In this work\nwe revisit the architecture choices of BERT in efforts to obtain a lighter\nmodel. We focus on reducing the number of parameters yet our methods can be\napplied towards other objectives such FLOPs or latency. We show that much\nefficient light BERT models can be obtained by reducing algorithmically chosen\ncorrect architecture design dimensions rather than reducing the number of\nTransformer encoder layers. In particular, our schuBERT gives $6.6\\%$ higher\naverage accuracy on GLUE and SQuAD datasets as compared to BERT with three\nencoder layers while having the same number of parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06632v3",
            "date": "2020-05-11 19:04:31+00:00",
            "title": "SCAT: Second Chance Autoencoder for Textual Data",
            "authors": [
                "Somaieh Goudarzvand",
                "Gharib Gharibi",
                "Yugyung Lee"
            ],
            "abstract": "We present a k-competitive learning approach for textual autoencoders named\nSecond Chance Autoencoder (SCAT). SCAT selects the $k$ largest and smallest\npositive activations as the winner neurons, which gain the activation values of\nthe loser neurons during the learning process, and thus focus on retrieving\nwell-representative features for topics. Our experiments show that SCAT\nachieves outstanding performance in classification, topic modeling, and\ndocument visualization compared to LDA, K-Sparse, NVCTM, and KATE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06641v1",
            "date": "2020-05-13 22:04:27+00:00",
            "title": "A Rate-Distortion view of human pragmatic reasoning",
            "authors": [
                "Noga Zaslavsky",
                "Jennifer Hu",
                "Roger P. Levy"
            ],
            "abstract": "What computational principles underlie human pragmatic reasoning? A prominent\napproach to pragmatics is the Rational Speech Act (RSA) framework, which\nformulates pragmatic reasoning as probabilistic speakers and listeners\nrecursively reasoning about each other. While RSA enjoys broad empirical\nsupport, it is not yet clear whether the dynamics of such recursive reasoning\nmay be governed by a general optimization principle. Here, we present a novel\nanalysis of the RSA framework that addresses this question. First, we show that\nRSA recursion implements an alternating maximization for optimizing a tradeoff\nbetween expected utility and communicative effort. On that basis, we study the\ndynamics of RSA recursion and disconfirm the conjecture that expected utility\nis guaranteed to improve with recursion depth. Second, we show that RSA can be\ngrounded in Rate-Distortion theory, while maintaining a similar ability to\naccount for human behavior and avoiding a bias of RSA toward random utterance\nproduction. This work furthers the mathematical understanding of RSA models,\nand suggests that general information-theoretic principles may give rise to\nhuman pragmatic reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06676v1",
            "date": "2020-05-14 00:45:23+00:00",
            "title": "Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions",
            "authors": [
                "Xiaochuang Han",
                "Byron C. Wallace",
                "Yulia Tsvetkov"
            ],
            "abstract": "Modern deep learning models for NLP are notoriously opaque. This has\nmotivated the development of methods for interpreting such models, e.g., via\ngradient-based saliency maps or the visualization of attention weights. Such\napproaches aim to provide explanations for a particular model prediction by\nhighlighting important words in the corresponding input text. While this might\nbe useful for tasks where decisions are explicitly influenced by individual\ntokens in the input, we suspect that such highlighting is not suitable for\ntasks where model decisions should be driven by more complex reasoning. In this\nwork, we investigate the use of influence functions for NLP, providing an\nalternative approach to interpreting neural text classifiers. Influence\nfunctions explain the decisions of a model by identifying influential training\nexamples. Despite the promise of this approach, influence functions have not\nyet been extensively evaluated in the context of NLP, a gap addressed by this\nwork. We conduct a comparison between influence functions and common\nword-saliency methods on representative tasks. As suspected, we find that\ninfluence functions are particularly useful for natural language inference, a\ntask in which 'saliency maps' may not have clear interpretation. Furthermore,\nwe develop a new quantitative measure based on influence functions that can\nreveal artifacts in training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.06898v2",
            "date": "2020-05-14 12:06:02+00:00",
            "title": "Mitigating Gender Bias in Machine Learning Data Sets",
            "authors": [
                "Susan Leavy",
                "Gerardine Meaney",
                "Karen Wade",
                "Derek Greene"
            ],
            "abstract": "Artificial Intelligence has the capacity to amplify and perpetuate societal\nbiases and presents profound ethical implications for society. Gender bias has\nbeen identified in the context of employment advertising and recruitment tools,\ndue to their reliance on underlying language processing and recommendation\nalgorithms. Attempts to address such issues have involved testing learned\nassociations, integrating concepts of fairness to machine learning and\nperforming more rigorous analysis of training data. Mitigating bias when\nalgorithms are trained on textual data is particularly challenging given the\ncomplex way gender ideology is embedded in language. This paper proposes a\nframework for the identification of gender bias in training data for machine\nlearning.The work draws upon gender theory and sociolinguistics to\nsystematically indicate levels of bias in textual training data and associated\nneural word embedding models, thus highlighting pathways for both removing bias\nfrom training data and critically assessing its impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07105v1",
            "date": "2020-05-14 16:15:58+00:00",
            "title": "ZeroShotCeres: Zero-Shot Relation Extraction from Semi-Structured Webpages",
            "authors": [
                "Colin Lockard",
                "Prashant Shiralkar",
                "Xin Luna Dong",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "In many documents, such as semi-structured webpages, textual semantics are\naugmented with additional information conveyed using visual elements including\nlayout, font size, and color. Prior work on information extraction from\nsemi-structured websites has required learning an extraction model specific to\na given template via either manually labeled or distantly supervised data from\nthat template. In this work, we propose a solution for \"zero-shot\" open-domain\nrelation extraction from webpages with a previously unseen template, including\nfrom websites with little overlap with existing sources of knowledge for\ndistant supervision and websites in entirely new subject verticals. Our model\nuses a graph neural network-based approach to build a rich representation of\ntext fields on a webpage and the relationships between them, enabling\ngeneralization to new templates. Experiments show this approach provides a 31%\nF1 gain over a baseline for zero-shot extraction in a new subject vertical.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07174v1",
            "date": "2020-05-14 17:42:25+00:00",
            "title": "Estimating predictive uncertainty for rumour verification models",
            "authors": [
                "Elena Kochkina",
                "Maria Liakata"
            ],
            "abstract": "The inability to correctly resolve rumours circulating online can have\nharmful real-world consequences. We present a method for incorporating model\nand data uncertainty estimates into natural language processing models for\nautomatic rumour verification. We show that these estimates can be used to\nfilter out model predictions likely to be erroneous, so that these difficult\ninstances can be prioritised by a human fact-checker. We propose two methods\nfor uncertainty-based instance rejection, supervised and unsupervised. We also\nshow how uncertainty estimates can be used to interpret model performance as a\nrumour unfolds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07310v2",
            "date": "2020-05-15 01:06:54+00:00",
            "title": "Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models",
            "authors": [
                "Jize Cao",
                "Zhe Gan",
                "Yu Cheng",
                "Licheng Yu",
                "Yen-Chun Chen",
                "Jingjing Liu"
            ],
            "abstract": "Recent Transformer-based large-scale pre-trained models have revolutionized\nvision-and-language (V+L) research. Models such as ViLBERT, LXMERT and UNITER\nhave significantly lifted state of the art across a wide range of V+L\nbenchmarks with joint image-text pre-training. However, little is known about\nthe inner mechanisms that destine their impressive success. To reveal the\nsecrets behind the scene of these powerful models, we present VALUE\n(Vision-And-Language Understanding Evaluation), a set of meticulously designed\nprobing tasks (e.g., Visual Coreference Resolution, Visual Relation Detection,\nLinguistic Probing Tasks) generalizable to standard pre-trained V+L models,\naiming to decipher the inner workings of multimodal pre-training (e.g., the\nimplicit knowledge garnered in individual attention heads, the inherent\ncross-modal alignment learned through contextualized multimodal embeddings).\nThrough extensive analysis of each archetypal model architecture via these\nprobing tasks, our key observations are: (i) Pre-trained models exhibit a\npropensity for attending over text rather than images during inference. (ii)\nThere exists a subset of attention heads that are tailored for capturing\ncross-modal interactions. (iii) Learned attention matrix in pre-trained models\ndemonstrates patterns coherent with the latent alignment between image regions\nand textual words. (iv) Plotted attention patterns reveal\nvisually-interpretable relations among image regions. (v) Pure linguistic\nknowledge is also effectively encoded in the attention heads. These are\nvaluable insights serving to guide future work towards designing better model\narchitecture and objectives for multimodal pre-training.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2005.07362v1",
            "date": "2020-05-15 05:20:06+00:00",
            "title": "Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluation",
            "authors": [
                "Ryuichi Takanobu",
                "Qi Zhu",
                "Jinchao Li",
                "Baolin Peng",
                "Jianfeng Gao",
                "Minlie Huang"
            ],
            "abstract": "There is a growing interest in developing goal-oriented dialog systems which\nserve users in accomplishing complex tasks through multi-turn conversations.\nAlthough many methods are devised to evaluate and improve the performance of\nindividual dialog components, there is a lack of comprehensive empirical study\non how different components contribute to the overall performance of a dialog\nsystem. In this paper, we perform a system-wise evaluation and present an\nempirical analysis on different types of dialog systems which are composed of\ndifferent modules in different settings. Our results show that (1) a pipeline\ndialog system trained using fine-grained supervision signals at different\ncomponent levels often obtains better performance than the systems that use\njoint or end-to-end models trained on coarse-grained labels, (2)\ncomponent-wise, single-turn evaluation results are not always consistent with\nthe overall performance of a dialog system, and (3) despite the discrepancy\nbetween simulators and human users, simulated evaluation is still a valid\nalternative to the costly human evaluation especially in the early stage of\ndevelopment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07421v1",
            "date": "2020-05-15 09:02:38+00:00",
            "title": "Spelling Error Correction with Soft-Masked BERT",
            "authors": [
                "Shaohua Zhang",
                "Haoran Huang",
                "Jicong Liu",
                "Hang Li"
            ],
            "abstract": "Spelling error correction is an important yet challenging task because a\nsatisfactory solution of it essentially needs human-level language\nunderstanding ability. Without loss of generality we consider Chinese spelling\nerror correction (CSC) in this paper. A state-of-the-art method for the task\nselects a character from a list of candidates for correction (including\nnon-correction) at each position of the sentence on the basis of BERT, the\nlanguage representation model. The accuracy of the method can be sub-optimal,\nhowever, because BERT does not have sufficient capability to detect whether\nthere is an error at each position, apparently due to the way of pre-training\nit using mask language modeling. In this work, we propose a novel neural\narchitecture to address the aforementioned issue, which consists of a network\nfor error detection and a network for error correction based on BERT, with the\nformer being connected to the latter with what we call soft-masking technique.\nOur method of using `Soft-Masked BERT' is general, and it may be employed in\nother language detection-correction problems. Experimental results on two\ndatasets demonstrate that the performance of our proposed method is\nsignificantly better than the baselines including the one solely based on BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07456v3",
            "date": "2020-05-15 10:15:27+00:00",
            "title": "Cross-lingual Transfer of Sentiment Classifiers",
            "authors": [
                "Marko Robnik-Sikonja",
                "Kristjan Reba",
                "Igor Mozetic"
            ],
            "abstract": "Word embeddings represent words in a numeric space so that semantic relations\nbetween words are represented as distances and directions in the vector space.\nCross-lingual word embeddings transform vector spaces of different languages so\nthat similar words are aligned. This is done by constructing a mapping between\nvector spaces of two languages or learning a joint vector space for multiple\nlanguages. Cross-lingual embeddings can be used to transfer machine learning\nmodels between languages, thereby compensating for insufficient data in\nless-resourced languages. We use cross-lingual word embeddings to transfer\nmachine learning prediction models for Twitter sentiment between 13 languages.\nWe focus on two transfer mechanisms that recently show superior transfer\nperformance. The first mechanism uses the trained models whose input is the\njoint numerical space for many languages as implemented in the LASER library.\nThe second mechanism uses large pretrained multilingual BERT language models.\nOur experiments show that the transfer of models between similar languages is\nsensible, even with no target language data. The performance of cross-lingual\nmodels obtained with the multilingual BERT and LASER library is comparable, and\nthe differences are language-dependent. The transfer with CroSloEngual BERT,\npretrained on only three languages, is superior on these and some closely\nrelated languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07486v3",
            "date": "2020-05-15 12:12:57+00:00",
            "title": "Adaptive Transformers for Learning Multimodal Representations",
            "authors": [
                "Prajjwal Bhargava"
            ],
            "abstract": "The usage of transformers has grown from learning about language semantics to\nforming meaningful visiolinguistic representations. These architectures are\noften over-parametrized, requiring large amounts of computation. In this work,\nwe extend adaptive approaches to learn more about model interpretability and\ncomputational efficiency. Specifically, we study attention spans, sparse, and\nstructured dropout methods to help understand how their attention mechanism\nextends for vision and language tasks. We further show that these approaches\ncan help us learn more about how the network perceives the complexity of input\nsequences, sparsity preferences for different modalities, and other related\nphenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07604v2",
            "date": "2020-05-15 15:47:02+00:00",
            "title": "Neural Entity Linking on Technical Service Tickets",
            "authors": [
                "Nadja Kurz",
                "Felix Hamann",
                "Adrian Ulges"
            ],
            "abstract": "Entity linking, the task of mapping textual mentions to known entities, has\nrecently been tackled using contextualized neural networks. We address the\nquestion whether these results -- reported for large, high-quality datasets\nsuch as Wikipedia -- transfer to practical business use cases, where labels are\nscarce, text is low-quality, and terminology is highly domain-specific. Using\nan entity linking model based on BERT, a popular transformer network in natural\nlanguage processing, we show that a neural approach outperforms and complements\nhand-coded heuristics, with improvements of about 20% top-1 accuracy. Also, the\nbenefits of transfer learning on a large corpus are demonstrated, while\nfine-tuning proves difficult. Finally, we compare different BERT-based\narchitectures and show that a simple sentence-wise encoding (Bi-Encoder) offers\na fast yet efficient search in practice.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07655v2",
            "date": "2020-05-15 17:17:01+00:00",
            "title": "Analyzing Temporal Relationships between Trending Terms on Twitter and Urban Dictionary Activity",
            "authors": [
                "Steven R. Wilson",
                "Walid Magdy",
                "Barbara McGillivray",
                "Gareth Tyson"
            ],
            "abstract": "As an online, crowd-sourced, open English-language slang dictionary, the\nUrban Dictionary platform contains a wealth of opinions, jokes, and definitions\nof terms, phrases, acronyms, and more. However, it is unclear exactly how\nactivity on this platform relates to larger conversations happening elsewhere\non the web, such as discussions on larger, more popular social media platforms.\nIn this research, we study the temporal activity trends on Urban Dictionary and\nprovide the first analysis of how this activity relates to content being\ndiscussed on a major social network: Twitter. By collecting the whole of Urban\nDictionary, as well as a large sample of tweets over seven years, we explore\nthe connections between the words and phrases that are defined and searched for\non Urban Dictionary and the content that is talked about on Twitter. Through a\nseries of cross-correlation calculations, we identify cases in which Urban\nDictionary activity closely reflects the larger conversation happening on\nTwitter. Then, we analyze the types of terms that have a stronger connection to\ndiscussions on Twitter, finding that Urban Dictionary activity that is\npositively correlated with Twitter is centered around terms related to memes,\npopular public figures, and offline events. Finally, We explore the\nrelationship between periods of time when terms are trending on Twitter and the\ncorresponding activity on Urban Dictionary, revealing that new definitions are\nmore likely to be added to Urban Dictionary for terms that are currently\ntrending on Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07683v2",
            "date": "2020-05-15 17:54:15+00:00",
            "title": "Movement Pruning: Adaptive Sparsity by Fine-Tuning",
            "authors": [
                "Victor Sanh",
                "Thomas Wolf",
                "Alexander M. Rush"
            ],
            "abstract": "Magnitude pruning is a widely used strategy for reducing model size in pure\nsupervised learning; however, it is less effective in the transfer learning\nregime that has become standard for state-of-the-art natural language\nprocessing applications. We propose the use of movement pruning, a simple,\ndeterministic first-order weight pruning method that is more adaptive to\npretrained model fine-tuning. We give mathematical foundations to the method\nand compare it to existing zeroth- and first-order pruning methods. Experiments\nshow that when pruning large pretrained language models, movement pruning shows\nsignificant improvements in high-sparsity regimes. When combined with\ndistillation, the approach achieves minimal accuracy loss with down to only 3%\nof the model parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07692v2",
            "date": "2020-05-14 06:54:07+00:00",
            "title": "An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognition",
            "authors": [
                "Gizem Aras",
                "Didem Makaroglu",
                "Seniz Demir",
                "Altan Cakir"
            ],
            "abstract": "Named entity recognition (NER) is an extensively studied task that extracts\nand classifies named entities in a text. NER is crucial not only in downstream\nlanguage processing applications such as relation extraction and question\nanswering but also in large scale big data operations such as real-time\nanalysis of online digital media content. Recent research efforts on Turkish, a\nless studied language with morphologically rich nature, have demonstrated the\neffectiveness of neural architectures on well-formed texts and yielded\nstate-of-the art results by formulating the task as a sequence tagging problem.\nIn this work, we empirically investigate the use of recent neural architectures\n(Bidirectional long short-term memory and Transformer-based networks) proposed\nfor Turkish NER tagging in the same setting. Our results demonstrate that\ntransformer-based networks which can model long-range context overcome the\nlimitations of BiLSTM networks where different input features at the character,\nsubword, and word levels are utilized. We also propose a transformer-based\nnetwork with a conditional random field (CRF) layer that leads to the\nstate-of-the-art result (95.95\\% f-measure) on a common dataset. Our study\ncontributes to the literature that quantifies the impact of transfer learning\non processing morphologically rich languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07820v1",
            "date": "2020-05-15 23:11:03+00:00",
            "title": "KEIS@JUST at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT",
            "authors": [
                "Saja Khaled Tawalbeh",
                "Mahmoud Hammad",
                "Mohammad AL-Smadi"
            ],
            "abstract": "This research presents our team KEIS@JUST participation at SemEval-2020 Task\n12 which represents shared task on multilingual offensive language. We\nparticipated in all the provided languages for all subtasks except sub-task-A\nfor the English language. Two main approaches have been developed the first is\nperformed to tackle both languages Arabic and English, a weighted ensemble\nconsists of Bi-GRU and CNN followed by Gaussian noise and global pooling layer\nmultiplied by weights to improve the overall performance. The second is\nperformed for other languages, a transfer learning from BERT beside the\nrecurrent neural networks such as Bi-LSTM and Bi-GRU followed by a global\naverage pooling layer. Word embedding and contextual embedding have been used\nas features, moreover, data augmentation has been used only for the Arabic\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07877v1",
            "date": "2020-05-16 05:42:57+00:00",
            "title": "MicroNet for Efficient Language Modeling",
            "authors": [
                "Zhongxia Yan",
                "Hanrui Wang",
                "Demi Guo",
                "Song Han"
            ],
            "abstract": "It is important to design compact language models for efficient deployment.\nWe improve upon recent advances in both the language modeling domain and the\nmodel-compression domain to construct parameter and computation efficient\nlanguage models. We use an efficient transformer-based architecture with\nadaptive embedding and softmax, differentiable non-parametric cache, Hebbian\nsoftmax, knowledge distillation, network pruning, and low-bit quantization. In\nthis paper, we provide the winning solution to the NeurIPS 2019 MicroNet\nChallenge in the language modeling track. Compared to the baseline language\nmodel provided by the MicroNet Challenge, our model is 90 times more\nparameter-efficient and 36 times more computation-efficient while achieving the\nrequired test perplexity of 35 on the Wikitext-103 dataset. We hope that this\nwork will aid future research into efficient language models, and we have\nreleased our full source code at\nhttps://github.com/mit-han-lab/neurips-micronet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07923v1",
            "date": "2020-05-16 09:47:19+00:00",
            "title": "Sequential Sentence Matching Network for Multi-turn Response Selection in Retrieval-based Chatbots",
            "authors": [
                "Chao Xiong",
                "Che Liu",
                "Zijun Xu",
                "Junfeng Jiang",
                "Jieping Ye"
            ],
            "abstract": "Recently, open domain multi-turn chatbots have attracted much interest from\nlots of researchers in both academia and industry. The dominant retrieval-based\nmethods use context-response matching mechanisms for multi-turn response\nselection. Specifically, the state-of-the-art methods perform the\ncontext-response matching by word or segment similarity. However, these models\nlack a full exploitation of the sentence-level semantic information, and make\nsimple mistakes that humans can easily avoid. In this work, we propose a\nmatching network, called sequential sentence matching network (S2M), to use the\nsentence-level semantic information to address the problem. Firstly and most\nimportantly, we find that by using the sentence-level semantic information, the\nnetwork successfully addresses the problem and gets a significant improvement\non matching, resulting in a state-of-the-art performance. Furthermore, we\nintegrate the sentence matching we introduced here and the usual word\nsimilarity matching reported in the current literature, to match at different\nsemantic levels. Experiments on three public data sets show that such\nintegration further improves the model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.07979v1",
            "date": "2020-05-16 13:05:47+00:00",
            "title": "Unsupervised Embedding-based Detection of Lexical Semantic Changes",
            "authors": [
                "Ehsaneddin Asgari",
                "Christoph Ringlstetter",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "This paper describes EmbLexChange, a system introduced by the \"Life-Language\"\nteam for SemEval-2020 Task 1, on unsupervised detection of lexical-semantic\nchanges. EmbLexChange is defined as the divergence between the embedding based\nprofiles of word w (calculated with respect to a set of reference words) in the\nsource and the target domains (source and target domains can be simply two time\nframes t1 and t2). The underlying assumption is that the lexical-semantic\nchange of word w would affect its co-occurring words and subsequently alters\nthe neighborhoods in the embedding spaces. We show that using a resampling\nframework for the selection of reference words, we can reliably detect\nlexical-semantic changes in English, German, Swedish, and Latin. EmbLexChange\nachieved second place in the binary detection of semantic changes in the\nSemEval-2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08025v2",
            "date": "2020-05-16 15:47:53+00:00",
            "title": "IntelliCode Compose: Code Generation Using Transformer",
            "authors": [
                "Alexey Svyatkovskiy",
                "Shao Kun Deng",
                "Shengyu Fu",
                "Neel Sundaresan"
            ],
            "abstract": "In software development through integrated development environments (IDEs),\ncode completion is one of the most widely used features. Nevertheless, majority\nof integrated development environments only support completion of methods and\nAPIs, or arguments.\n  In this paper, we introduce IntelliCode Compose $-$ a general-purpose\nmultilingual code completion tool which is capable of predicting sequences of\ncode tokens of arbitrary types, generating up to entire lines of syntactically\ncorrect code. It leverages state-of-the-art generative transformer model\ntrained on 1.2 billion lines of source code in Python, $C\\#$, JavaScript and\nTypeScript programming languages. IntelliCode Compose is deployed as a\ncloud-based web service. It makes use of client-side tree-based caching,\nefficient parallel implementation of the beam search decoder, and compute graph\noptimizations to meet edit-time completion suggestion requirements in the\nVisual Studio Code IDE and Azure Notebook.\n  Our best model yields an average edit similarity of $86.7\\%$ and a perplexity\nof 1.82 for Python programming language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08042v1",
            "date": "2020-05-16 16:54:52+00:00",
            "title": "Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory",
            "authors": [
                "Chunyang Wu",
                "Yongqiang Wang",
                "Yangyang Shi",
                "Ching-Feng Yeh",
                "Frank Zhang"
            ],
            "abstract": "Transformer-based acoustic modeling has achieved great suc-cess for both\nhybrid and sequence-to-sequence speech recogni-tion. However, it requires\naccess to the full sequence, and thecomputational cost grows quadratically with\nrespect to the in-put sequence length. These factors limit its adoption for\nstream-ing applications. In this work, we proposed a novel augmentedmemory\nself-attention, which attends on a short segment of theinput sequence and a\nbank of memories. The memory bankstores the embedding information for all the\nprocessed seg-ments. On the librispeech benchmark, our proposed\nmethodoutperforms all the existing streamable transformer methods bya large\nmargin and achieved over 15% relative error reduction,compared with the widely\nused LC-BLSTM baseline. Our find-ings are also confirmed on some large internal\ndatasets.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.08056v2",
            "date": "2020-05-16 18:08:58+00:00",
            "title": "Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension",
            "authors": [
                "Hongyu Gong",
                "Yelong Shen",
                "Dian Yu",
                "Jianshu Chen",
                "Dong Yu"
            ],
            "abstract": "In this paper, we study machine reading comprehension (MRC) on long texts,\nwhere a model takes as inputs a lengthy document and a question and then\nextracts a text span from the document as an answer. State-of-the-art models\ntend to use a pretrained transformer model (e.g., BERT) to encode the joint\ncontextual information of document and question. However, these\ntransformer-based models can only take a fixed-length (e.g., 512) text as its\ninput. To deal with even longer text inputs, previous approaches usually chunk\nthem into equally-spaced segments and predict answers based on each segment\nindependently without considering the information from other segments. As a\nresult, they may form segments that fail to cover the correct answer span or\nretain insufficient contexts around it, which significantly degrades the\nperformance. Moreover, they are less capable of answering questions that need\ncross-segment information.\n  We propose to let a model learn to chunk in a more flexible way via\nreinforcement learning: a model can decide the next segment that it wants to\nprocess in either direction. We also employ recurrent mechanisms to enable\ninformation to flow across segments. Experiments on three MRC datasets -- CoQA,\nQuAC, and TriviaQA -- demonstrate the effectiveness of our proposed recurrent\nchunking mechanisms: we can obtain segments that are more likely to contain\ncomplete answers and at the same time provide sufficient contexts around the\nground truth answers for better predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08081v4",
            "date": "2020-05-16 20:00:39+00:00",
            "title": "Layer-Wise Multi-View Decoding for Natural Language Generation",
            "authors": [
                "Fenglin Liu",
                "Xuancheng Ren",
                "Guangxiang Zhao",
                "Xu Sun"
            ],
            "abstract": "In sequence-to-sequence learning, e.g., natural language generation, the\ndecoder relies on the attention mechanism to efficiently extract information\nfrom the encoder. While it is common practice to draw information from only the\nlast encoder layer, recent work has proposed to use representations from\ndifferent encoder layers for diversified levels of information. Nonetheless,\nthe decoder still obtains only a single view of the source sequences, which\nmight lead to insufficient training of the encoder layer stack due to the\nhierarchy bypassing problem. In this work, we propose layer-wise multi-view\ndecoding, where for each decoder layer, together with the representations from\nthe last encoder layer, which serve as a global view, those from other encoder\nlayers are supplemented for a stereoscopic view of the source sequences.\nSystematic experiments and analyses show that we successfully address the\nhierarchy bypassing problem and substantially improve the performance of\nsequence-to-sequence learning with deep representations on diverse tasks, i.e.,\nmachine translation, abstractive summarization and image captioning. In\nparticular, our approach surpasses the previous state-of-the-art models on\nthree benchmark machine translation datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08146v2",
            "date": "2020-05-17 02:01:43+00:00",
            "title": "Semi-Automating Knowledge Base Construction for Cancer Genetics",
            "authors": [
                "Somin Wadhwa",
                "Kanhua Yin",
                "Kevin S. Hughes",
                "Byron C. Wallace"
            ],
            "abstract": "In this work, we consider the exponentially growing subarea of genetics in\ncancer. The need to synthesize and centralize this evidence for dissemination\nhas motivated a team of physicians to manually construct and maintain a\nknowledge base that distills key results reported in the literature. This is a\nlaborious process that entails reading through full-text articles to understand\nthe study design, assess study quality, and extract the reported cancer risk\nestimates associated with particular hereditary cancer genes (i.e.,\npenetrance). In this work, we propose models to automatically surface key\nelements from full-text cancer genetics articles, with the ultimate aim of\nexpediting the manual workflow currently in place.\n  We propose two challenging tasks that are critical for characterizing the\nfindings reported cancer genetics studies: (i) Extracting snippets of text that\ndescribe \\emph{ascertainment mechanisms}, which in turn inform whether the\npopulation studied may introduce bias owing to deviations from the target\npopulation; (ii) Extracting reported risk estimates (e.g., odds or hazard\nratios) associated with specific germline mutations. The latter task may be\nviewed as a joint entity tagging and relation extraction problem. To train\nmodels for these tasks, we induce distant supervision over tokens and snippets\nin full-text articles using the manually constructed knowledge base. We propose\nand evaluate several model variants, including a transformer-based joint entity\nand relation extraction model to extract <germline mutation, risk-estimate>}\npairs. We observe strong empirical performance, highlighting the practical\npotential for such models to aid KB construction in this space. We ablate\ncomponents of our model, observing, e.g., that a joint model for <germline\nmutation, risk-estimate> fares substantially better than a pipelined approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08177v1",
            "date": "2020-05-17 06:41:32+00:00",
            "title": "Encodings of Source Syntax: Similarities in NMT Representations Across Target Languages",
            "authors": [
                "Tyler A. Chang",
                "Anna N. Rafferty"
            ],
            "abstract": "We train neural machine translation (NMT) models from English to six target\nlanguages, using NMT encoder representations to predict ancestor constituent\nlabels of source language words. We find that NMT encoders learn similar source\nsyntax regardless of NMT target language, relying on explicit morphosyntactic\ncues to extract syntactic features from source sentences. Furthermore, the NMT\nencoders outperform RNNs trained directly on several of the constituent label\nprediction tasks, suggesting that NMT encoder representations can be used\neffectively for natural language tasks involving syntax. However, both the NMT\nencoders and the directly-trained RNNs learn substantially different syntactic\ninformation from a probabilistic context-free grammar (PCFG) parser. Despite\nlower overall accuracy scores, the PCFG often performs well on sentences for\nwhich the RNN-based models perform poorly, suggesting that RNN architectures\nare constrained in the types of syntax they can learn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08199v2",
            "date": "2020-05-17 09:13:28+00:00",
            "title": "How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?",
            "authors": [
                "Gantavya Bhatt",
                "Hritik Bansal",
                "Rishubh Singh",
                "Sumeet Agarwal"
            ],
            "abstract": "Long short-term memory (LSTM) networks and their variants are capable of\nencapsulating long-range dependencies, which is evident from their performance\non a variety of linguistic tasks. On the other hand, simple recurrent networks\n(SRNs), which appear more biologically grounded in terms of synaptic\nconnections, have generally been less successful at capturing long-range\ndependencies as well as the loci of grammatical errors in an unsupervised\nsetting. In this paper, we seek to develop models that bridge the gap between\nbiological plausibility and linguistic competence. We propose a new\narchitecture, the Decay RNN, which incorporates the decaying nature of neuronal\nactivations and models the excitatory and inhibitory connections in a\npopulation of neurons. Besides its biological inspiration, our model also shows\ncompetitive performance relative to LSTMs on subject-verb agreement, sentence\ngrammaticality, and language modeling tasks. These results provide some\npointers towards probing the nature of the inductive biases required for RNN\narchitectures to model linguistic phenomena successfully.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08223v2",
            "date": "2020-05-17 11:14:30+00:00",
            "title": "LiSSS: A toy corpus of Spanish Literary Sentences for Emotions detection",
            "authors": [
                "Juan-Manuel Torres-Moreno",
                "Luis-Gil Moreno-Jim\u00e9nez"
            ],
            "abstract": "In this work we present a new small data-set in Computational Creativity (CC)\nfield, the Spanish Literary Sentences for emotions detection corpus (LISSS). We\naddress this corpus of literary sentences in order to evaluate or design\nalgorithms of emotions classification and detection. We have constitute this\ncorpus by manually classifying the sentences in a set of emotions: Love, Fear,\nHappiness, Anger and Sadness/Pain. We also present some baseline classification\nalgorithms applied on our corpus. The LISSS corpus will be available to the\ncommunity as a free resource to evaluate or create CC-like algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08224v1",
            "date": "2020-05-17 11:15:50+00:00",
            "title": "#Coronavirus or #Chinesevirus?!: Understanding the negative sentiment reflected in Tweets with racist hashtags across the development of COVID-19",
            "authors": [
                "Xin Pei",
                "Deval Mehta"
            ],
            "abstract": "Situated in the global outbreak of COVID-19, our study enriches the\ndiscussion concerning the emergent racism and xenophobia on social media. With\nbig data extracted from Twitter, we focus on the analysis of negative sentiment\nreflected in tweets marked with racist hashtags, as racism and xenophobia are\nmore likely to be delivered via the negative sentiment. Especially, we propose\na stage-based approach to capture how the negative sentiment changes along with\nthe three development stages of COVID-19, under which it transformed from a\ndomestic epidemic into an international public health emergency and later, into\nthe global pandemic. At each stage, sentiment analysis enables us to recognize\nthe negative sentiment from tweets with racist hashtags, and keyword extraction\nallows for the discovery of themes in the expression of negative sentiment by\nthese tweets. Under this public health crisis of human beings, this stage-based\napproach enables us to provide policy suggestions for the enactment of\nstage-specific intervention strategies to combat racism and xenophobia on\nsocial media in a more effective way.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.08259v1",
            "date": "2020-05-17 14:18:04+00:00",
            "title": "On the Combined Use of Extrinsic Semantic Resources for Medical Information Search",
            "authors": [
                "Mohammed Maree",
                "Israa Noor",
                "Khaled Rabayah",
                "Mohammed Belkhatir",
                "Saadat M. Alhashmi"
            ],
            "abstract": "Semantic concepts and relations encoded in domain-specific ontologies and\nother medical semantic resources play a crucial role in deciphering terms in\nmedical queries and documents. The exploitation of these resources for tackling\nthe semantic gap issue has been widely studied in the literature. However,\nthere are challenges that hinder their widespread use in real-world\napplications. Among these challenges is the insufficient knowledge individually\nencoded in existing medical ontologies, which is magnified when users express\ntheir information needs using long-winded natural language queries. In this\ncontext, many of the users query terms are either unrecognized by the used\nontologies, or cause retrieving false positives that degrade the quality of\ncurrent medical information search approaches. In this article, we explore the\ncombination of multiple extrinsic semantic resources in the development of a\nfull-fledged medical information search framework to: i) highlight and expand\nhead medical concepts in verbose medical queries (i.e. concepts among query\nterms that significantly contribute to the informativeness and intent of a\ngiven query), ii) build semantically enhanced inverted index documents, iii)\ncontribute to a heuristical weighting technique in the query document matching\nprocess. To demonstrate the effectiveness of the proposed approach, we\nconducted several experiments over the CLEF eHealth 2014 dataset. Findings\nindicate that the proposed method combining several extrinsic semantic\nresources proved to be more effective than related approaches in terms of\nprecision measure.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.08314v1",
            "date": "2020-05-17 17:26:40+00:00",
            "title": "TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data",
            "authors": [
                "Pengcheng Yin",
                "Graham Neubig",
                "Wen-tau Yih",
                "Sebastian Riedel"
            ],
            "abstract": "Recent years have witnessed the burgeoning of pretrained language models\n(LMs) for text-based natural language (NL) understanding tasks. Such models are\ntypically trained on free-form NL text, hence may not be suitable for tasks\nlike semantic parsing over structured data, which require reasoning over both\nfree-form NL questions and structured tabular data (e.g., database tables). In\nthis paper we present TaBERT, a pretrained LM that jointly learns\nrepresentations for NL sentences and (semi-)structured tables. TaBERT is\ntrained on a large corpus of 26 million tables and their English contexts. In\nexperiments, neural semantic parsers using TaBERT as feature representation\nlayers achieve new best results on the challenging weakly-supervised semantic\nparsing benchmark WikiTableQuestions, while performing competitively on the\ntext-to-SQL dataset Spider. Implementation of the model will be available at\nhttp://fburl.com/TaBERT .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08519v2",
            "date": "2020-05-18 08:19:18+00:00",
            "title": "Yseop at SemEval-2020 Task 5: Cascaded BERT Language Model for Counterfactual Statement Analysis",
            "authors": [
                "Hanna Abi Akl",
                "Dominique Mariko",
                "Estelle Labidurie"
            ],
            "abstract": "In this paper, we explore strategies to detect and evaluate counterfactual\nsentences. We describe our system for SemEval-2020 Task 5: Modeling Causal\nReasoning in Language: Detecting Counterfactuals. We use a BERT base model for\nthe classification task and build a hybrid BERT Multi-Layer Perceptron system\nto handle the sequence identification task. Our experiments show that while\nintroducing syntactic and semantic features does little in improving the system\nin the classification task, using these types of features as cascaded linear\ninputs to fine-tune the sequence-delimiting ability of the model ensures it\noutperforms other similar-purpose complex systems like BiLSTM-CRF in the second\ntask. Our system achieves an F1 score of 85.00% in Task 1 and 83.90% in Task 2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08601v1",
            "date": "2020-05-18 11:32:14+00:00",
            "title": "Design Choices for X-vector Based Speaker Anonymization",
            "authors": [
                "Brij Mohan Lal Srivastava",
                "Natalia Tomashenko",
                "Xin Wang",
                "Emmanuel Vincent",
                "Junichi Yamagishi",
                "Mohamed Maouche",
                "Aur\u00e9lien Bellet",
                "Marc Tommasi"
            ],
            "abstract": "The recently proposed x-vector based anonymization scheme converts any input\nvoice into that of a random pseudo-speaker. In this paper, we present a\nflexible pseudo-speaker selection technique as a baseline for the first\nVoicePrivacy Challenge. We explore several design choices for the distance\nmetric between speakers, the region of x-vector space where the pseudo-speaker\nis picked, and gender selection. To assess the strength of anonymization\nachieved, we consider attackers using an x-vector based speaker verification\nsystem who may use original or anonymized speech for enrollment, depending on\ntheir knowledge of the anonymization scheme. The Equal Error Rate (EER)\nachieved by the attackers and the decoding Word Error Rate (WER) over\nanonymized data are reported as the measures of privacy and utility.\nExperiments are performed using datasets derived from LibriSpeech to find the\noptimal combination of design choices in terms of privacy and utility.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.08746v1",
            "date": "2020-05-18 14:21:22+00:00",
            "title": "Improving Named Entity Recognition in Tor Darknet with Local Distance Neighbor Feature",
            "authors": [
                "Mhd Wesam Al-Nabki",
                "Francisco Ja\u00f1ez-Martino",
                "Roberto A. Vasco-Carofilis",
                "Eduardo Fidalgo",
                "Javier Velasco-Mata"
            ],
            "abstract": "Name entity recognition in noisy user-generated texts is a difficult task\nusually enhanced by incorporating an external resource of information, such as\ngazetteers. However, gazetteers are task-specific, and they are expensive to\nbuild and maintain. This paper adopts and improves the approach of Aguilar et\nal. by presenting a novel feature, called Local Distance Neighbor, which\nsubstitutes gazetteers. We tested the new approach on the W-NUT-2017 dataset,\nobtaining state-of-the-art results for the Group, Person and Product categories\nof Named Entities. Next, we added 851 manually labeled samples to the\nW-NUT-2017 dataset to account for named entities in the Tor Darknet related to\nweapons and drug selling. Finally, our proposal achieved an entity and surface\nF1 scores of 52.96% and 50.57% on this extended dataset, demonstrating its\nusefulness for Law Enforcement Agencies to detect named entities in the Tor\nhidden services.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08773v2",
            "date": "2020-05-18 14:41:22+00:00",
            "title": "Classification of Spam Emails through Hierarchical Clustering and Supervised Learning",
            "authors": [
                "Francisco J\u00e1\u00f1ez-Martino",
                "Eduardo Fidalgo",
                "Santiago Gonz\u00e1lez-Mart\u00ednez",
                "Javier Velasco-Mata"
            ],
            "abstract": "Spammers take advantage of email popularity to send indiscriminately\nunsolicited emails. Although researchers and organizations continuously develop\nanti-spam filters based on binary classification, spammers bypass them through\nnew strategies, like word obfuscation or image-based spam. For the first time\nin literature, we propose to classify spam email in categories to improve the\nhandle of already detected spam emails, instead of just using a binary model.\nFirst, we applied a hierarchical clustering algorithm to create SPEMC-$11$K\n(SPam EMail Classification), the first multi-class dataset, which contains\nthree types of spam emails: Health and Technology, Personal Scams, and Sexual\nContent. Then, we used SPEMC-$11$K to evaluate the combination of TF-IDF and\nBOW encodings with Na\\\"ive Bayes, Decision Trees and SVM classifiers. Finally,\nwe recommend for the task of multi-class spam classification the use of (i)\nTF-IDF combined with SVM for the best micro F1 score performance, $95.39\\%$,\nand (ii) TD-IDF along with NB for the fastest spam classification, analyzing an\nemail in $2.13$ms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.08820v1",
            "date": "2020-05-18 15:53:13+00:00",
            "title": "Critical Impact of Social Networks Infodemic on Defeating Coronavirus COVID-19 Pandemic: Twitter-Based Study and Research Directions",
            "authors": [
                "Azzam Mourad",
                "Ali Srour",
                "Haidar Harmanani",
                "Cathia Jenainatiy",
                "Mohamad Arafeh"
            ],
            "abstract": "News creation and consumption has been changing since the advent of social\nmedia. An estimated 2.95 billion people in 2019 used social media worldwide.\nThe widespread of the Coronavirus COVID-19 resulted with a tsunami of social\nmedia. Most platforms were used to transmit relevant news, guidelines and\nprecautions to people. According to WHO, uncontrolled conspiracy theories and\npropaganda are spreading faster than the COVID-19 pandemic itself, creating an\ninfodemic and thus causing psychological panic, misleading medical advises, and\neconomic disruption. Accordingly, discussions have been initiated with the\nobjective of moderating all COVID-19 communications, except those initiated\nfrom trusted sources such as the WHO and authorized governmental entities. This\npaper presents a large-scale study based on data mined from Twitter. Extensive\nanalysis has been performed on approximately one million COVID-19 related\ntweets collected over a period of two months. Furthermore, the profiles of\n288,000 users were analyzed including unique users profiles, meta-data and\ntweets context. The study noted various interesting conclusions including the\ncritical impact of the (1) exploitation of the COVID-19 crisis to redirect\nreaders to irrelevant topics and (2) widespread of unauthentic medical\nprecautions and information. Further data analysis revealed the importance of\nusing social networks in a global pandemic crisis by relying on credible users\nwith variety of occupations, content developers and influencers in specific\nfields. In this context, several insights and findings have been provided while\nelaborating computing and non-computing implications and research directions\nfor potential solutions and social networks management strategies during crisis\nperiods.",
            "journal ref": "https://ieeexplore.ieee.org/document/9223699 (2020)",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.09069v1",
            "date": "2020-05-18 20:41:12+00:00",
            "title": "P-SIF: Document Embeddings Using Partition Averaging",
            "authors": [
                "Vivek Gupta",
                "Ankit Saw",
                "Pegah Nokhiz",
                "Praneeth Netrapalli",
                "Piyush Rai",
                "Partha Talukdar"
            ],
            "abstract": "Simple weighted averaging of word vectors often yields effective\nrepresentations for sentences which outperform sophisticated seq2seq neural\nmodels in many tasks. While it is desirable to use the same method to represent\ndocuments as well, unfortunately, the effectiveness is lost when representing\nlong documents involving multiple sentences. One of the key reasons is that a\nlonger document is likely to contain words from many different topics; hence,\ncreating a single vector while ignoring all the topical structure is unlikely\nto yield an effective document representation. This problem is less acute in\nsingle sentences and other short text fragments where the presence of a single\ntopic is most likely. To alleviate this problem, we present P-SIF, a\npartitioned word averaging model to represent long documents. P-SIF retains the\nsimplicity of simple weighted word averaging while taking a document's topical\nstructure into account. In particular, P-SIF learns topic-specific vectors from\na document and finally concatenates them all to represent the overall document.\nWe provide theoretical justifications on the correctness of P-SIF. Through a\ncomprehensive set of experiments, we demonstrate P-SIF's effectiveness compared\nto simple weighted averaging and many other baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09117v1",
            "date": "2020-05-18 22:20:17+00:00",
            "title": "Contextual Embeddings: When Are They Worth It?",
            "authors": [
                "Simran Arora",
                "Avner May",
                "Jian Zhang",
                "Christopher R\u00e9"
            ],
            "abstract": "We study the settings for which deep contextual embeddings (e.g., BERT) give\nlarge improvements in performance relative to classic pretrained embeddings\n(e.g., GloVe), and an even simpler baseline---random word embeddings---focusing\non the impact of the training set size and the linguistic properties of the\ntask. Surprisingly, we find that both of these simpler baselines can match\ncontextual embeddings on industry-scale data, and often perform within 5 to 10%\naccuracy (absolute) on benchmark tasks. Furthermore, we identify properties of\ndata for which contextual embeddings give particularly large gains: language\ncontaining complex structure, ambiguous word usage, and words unseen in\ntraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09123v2",
            "date": "2020-05-18 22:50:26+00:00",
            "title": "GPT-too: A language-model-first approach for AMR-to-text generation",
            "authors": [
                "Manuel Mager",
                "Ramon Fernandez Astudillo",
                "Tahira Naseem",
                "Md Arafat Sultan",
                "Young-Suk Lee",
                "Radu Florian",
                "Salim Roukos"
            ],
            "abstract": "Meaning Representations (AMRs) are broad-coverage sentence-level semantic\ngraphs. Existing approaches to generating text from AMR have focused on\ntraining sequence-to-sequence or graph-to-sequence models on AMR annotated data\nonly. In this paper, we propose an alternative approach that combines a strong\npre-trained language model with cycle consistency-based re-scoring. Despite the\nsimplicity of the approach, our experimental results show these models\noutperform all previous techniques on the English LDC2017T10dataset, including\nthe recent use of transformer architectures. In addition to the standard\nevaluation metrics, we provide human evaluation experiments that further\nsubstantiate the strength of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09125v2",
            "date": "2020-05-18 22:51:34+00:00",
            "title": "On the Power of Unambiguity in B\u00fcchi Complementation",
            "authors": [
                "Yong Li",
                "Moshe Y. Vardi",
                "Lijun Zhang"
            ],
            "abstract": "In this work, we exploit the power of \\emph{unambiguity} for the\ncomplementation problem of B\\\"uchi automata by utilizing reduced run directed\nacyclic graphs (DAGs) over infinite words, in which each vertex has at most one\npredecessor. We then show how to use this type of reduced run DAGs as a\n\\emph{unified tool} to optimize \\emph{both} rank-based and slice-based\ncomplementation constructions for B\\\"uchi automata with a finite degree of\nambiguity. As a result, given a B\\\"uchi automaton with $n$ states and a finite\ndegree of ambiguity, the number of states in the complementary B\\\"uchi\nautomaton constructed by the classical rank-based and slice-based\ncomplementation constructions can be improved, respectively, to $2^{O(n)}$ from\n$2^{O(n\\log n)}$ and to $O(4^n)$ from $O((3n)^n)$.",
            "journal ref": "EPTCS 326, 2020, pp. 182-198",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09133v1",
            "date": "2020-05-18 23:25:15+00:00",
            "title": "NEJM-enzh: A Parallel Corpus for English-Chinese Translation in the Biomedical Domain",
            "authors": [
                "Boxiang Liu",
                "Liang Huang"
            ],
            "abstract": "Machine translation requires large amounts of parallel text. While such\ndatasets are abundant in domains such as newswire, they are less accessible in\nthe biomedical domain. Chinese and English are two of the most widely spoken\nlanguages, yet to our knowledge a parallel corpus in the biomedical domain does\nnot exist for this language pair. In this study, we develop an effective\npipeline to acquire and process an English-Chinese parallel corpus, consisting\nof about 100,000 sentence pairs and 3,000,000 tokens on each side, from the New\nEngland Journal of Medicine (NEJM). We show that training on out-of-domain data\nand fine-tuning with as few as 4,000 NEJM sentence pairs improve translation\nquality by 25.3 (13.4) BLEU for en$\\to$zh (zh$\\to$en) directions. Translation\nquality continues to improve at a slower pace on larger in-domain datasets,\nwith an increase of 33.0 (24.3) BLEU for en$\\to$zh (zh$\\to$en) directions on\nthe full dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09137v1",
            "date": "2020-05-18 23:49:40+00:00",
            "title": "Weak-Attention Suppression For Transformer Based Speech Recognition",
            "authors": [
                "Yangyang Shi",
                "Yongqiang Wang",
                "Chunyang Wu",
                "Christian Fuegen",
                "Frank Zhang",
                "Duc Le",
                "Ching-Feng Yeh",
                "Michael L. Seltzer"
            ],
            "abstract": "Transformers, originally proposed for natural language processing (NLP)\ntasks, have recently achieved great success in automatic speech recognition\n(ASR). However, adjacent acoustic units (i.e., frames) are highly correlated,\nand long-distance dependencies between them are weak, unlike text units. It\nsuggests that ASR will likely benefit from sparse and localized attention. In\nthis paper, we propose Weak-Attention Suppression (WAS), a method that\ndynamically induces sparsity in attention probabilities. We demonstrate that\nWAS leads to consistent Word Error Rate (WER) improvement over strong\ntransformer baselines. On the widely used LibriSpeech benchmark, our proposed\nmethod reduced WER by 10%$ on test-clean and 5% on test-other for streamable\ntransformers, resulting in a new state-of-the-art among streaming models.\nFurther analysis shows that WAS learns to suppress attention of non-critical\nand redundant continuous acoustic frames, and is more likely to suppress past\nframes rather than future ones. It indicates the importance of lookahead in\nattention-based ASR models.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.09150v2",
            "date": "2020-05-19 00:43:17+00:00",
            "title": "Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces",
            "authors": [
                "Frank Zhang",
                "Yongqiang Wang",
                "Xiaohui Zhang",
                "Chunxi Liu",
                "Yatharth Saraf",
                "Geoffrey Zweig"
            ],
            "abstract": "In this work, we first show that on the widely used LibriSpeech benchmark,\nour transformer-based context-dependent connectionist temporal classification\n(CTC) system produces state-of-the-art results. We then show that using\nwordpieces as modeling units combined with CTC training, we can greatly\nsimplify the engineering pipeline compared to conventional frame-based\ncross-entropy training by excluding all the GMM bootstrapping, decision tree\nbuilding and force alignment steps, while still achieving very competitive\nword-error-rate. Additionally, using wordpieces as modeling units can\nsignificantly improve runtime efficiency since we can use larger stride without\nlosing accuracy. We further confirm these findings on two internal VideoASR\ndatasets: German, which is similar to English as a fusional language, and\nTurkish, which is an agglutinative language.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.09256v2",
            "date": "2020-05-19 07:35:14+00:00",
            "title": "Generative Adversarial Training Data Adaptation for Very Low-resource Automatic Speech Recognition",
            "authors": [
                "Kohei Matsuura",
                "Masato Mimura",
                "Shinsuke Sakai",
                "Tatsuya Kawahara"
            ],
            "abstract": "It is important to transcribe and archive speech data of endangered languages\nfor preserving heritages of verbal culture and automatic speech recognition\n(ASR) is a powerful tool to facilitate this process. However, since endangered\nlanguages do not generally have large corpora with many speakers, the\nperformance of ASR models trained on them are considerably poor in general.\nNevertheless, we are often left with a lot of recordings of spontaneous speech\ndata that have to be transcribed. In this work, for mitigating this speaker\nsparsity problem, we propose to convert the whole training speech data and make\nit sound like the test speaker in order to develop a highly accurate ASR system\nfor this speaker. For this purpose, we utilize a CycleGAN-based non-parallel\nvoice conversion technology to forge a labeled training data that is close to\nthe test speaker's speech. We evaluated this speaker adaptation approach on two\nlow-resource corpora, namely, Ainu and Mboshi. We obtained 35-60% relative\nimprovement in phone error rate on the Ainu corpus, and 40% relative\nimprovement was attained on the Mboshi corpus. This approach outperformed two\nconventional methods namely unsupervised adaptation and multilingual training\nwith these two corpora.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.09260v2",
            "date": "2020-05-19 07:44:48+00:00",
            "title": "Cross-lingual Approaches for Task-specific Dialogue Act Recognition",
            "authors": [
                "Ji\u0159\u00ed Mart\u00ednek",
                "Christophe Cerisara",
                "Pavel Kr\u00e1l",
                "Ladislav Lenc"
            ],
            "abstract": "In this paper we exploit cross-lingual models to enable dialogue act\nrecognition for specific tasks with a small number of annotations. We design a\ntransfer learning approach for dialogue act recognition and validate it on two\ndifferent target languages and domains. We compute dialogue turn embeddings\nwith both a CNN and multi-head self-attention model and show that the best\nresults are obtained by combining all sources of transferred information. We\nfurther demonstrate that the proposed methods significantly outperform related\ncross-lingual DA recognition approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09389v1",
            "date": "2020-05-19 12:32:20+00:00",
            "title": "On the Choice of Auxiliary Languages for Improved Sequence Tagging",
            "authors": [
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "Recent work showed that embeddings from related languages can improve the\nperformance of sequence tagging, even for monolingual models. In this analysis\npaper, we investigate whether the best auxiliary language can be predicted\nbased on language distances and show that the most related language is not\nalways the best auxiliary language. Further, we show that attention-based\nmeta-embeddings can effectively combine pre-trained embeddings from different\nlanguages for sequence tagging and set new state-of-the-art results for\npart-of-speech tagging in five languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09392v1",
            "date": "2020-05-19 12:37:04+00:00",
            "title": "Adversarial Alignment of Multilingual Models for Extracting Temporal Expressions from Text",
            "authors": [
                "Lukas Lange",
                "Anastasiia Iurshina",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "Although temporal tagging is still dominated by rule-based systems, there\nhave been recent attempts at neural temporal taggers. However, all of them\nfocus on monolingual settings. In this paper, we explore multilingual methods\nfor the extraction of temporal expressions from text and investigate\nadversarial training for aligning embedding spaces to one common space. With\nthis, we create a single multilingual model that can also be transferred to\nunseen languages and set the new state of the art in those cross-lingual\ntransfer experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09397v1",
            "date": "2020-05-19 12:44:41+00:00",
            "title": "Closing the Gap: Joint De-Identification and Concept Extraction in the Clinical Domain",
            "authors": [
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "Exploiting natural language processing in the clinical domain requires\nde-identification, i.e., anonymization of personal information in texts.\nHowever, current research considers de-identification and downstream tasks,\nsuch as concept extraction, only in isolation and does not study the effects of\nde-identification on other tasks. In this paper, we close this gap by reporting\nconcept extraction performance on automatically anonymized data and\ninvestigating joint models for de-identification and concept extraction. In\nparticular, we propose a stacked model with restricted access to\nprivacy-sensitive information and a multitask model. We set the new state of\nthe art on benchmark datasets in English (96.1% F1 for de-identification and\n88.9% F1 for concept extraction) and Spanish (91.4% F1 for concept extraction).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09406v1",
            "date": "2020-05-19 13:04:02+00:00",
            "title": "Embeddings as representation for symbolic music",
            "authors": [
                "Sebastian Garcia-Valencia"
            ],
            "abstract": "A representation technique that allows encoding music in a way that contains\nmusical meaning would improve the results of any model trained for computer\nmusic tasks like generation of melodies and harmonies of better quality. The\nfield of natural language processing has done a lot of work in finding a way to\ncapture the semantic meaning of words and sentences, and word embeddings have\nsuccessfully shown the capabilities for such a task. In this paper, we\nexperiment with embeddings to represent musical notes from 3 different\nvariations of a dataset and analyze if the model can capture useful musical\npatterns. To do this, the resulting embeddings are visualized in projections\nusing the t-SNE technique.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09409v2",
            "date": "2020-05-19 13:06:17+00:00",
            "title": "Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge",
            "authors": [
                "Benjamin van Niekerk",
                "Leanne Nortje",
                "Herman Kamper"
            ],
            "abstract": "In this paper, we explore vector quantization for acoustic unit discovery.\nLeveraging unlabelled data, we aim to learn discrete representations of speech\nthat separate phonetic content from speaker-specific details. We propose two\nneural models to tackle this challenge - both use vector quantization to map\ncontinuous features to a finite set of codes. The first model is a type of\nvector-quantized variational autoencoder (VQ-VAE). The VQ-VAE encodes speech\ninto a sequence of discrete units before reconstructing the audio waveform. Our\nsecond model combines vector quantization with contrastive predictive coding\n(VQ-CPC). The idea is to learn a representation of speech by predicting future\nacoustic units. We evaluate the models on English and Indonesian data for the\nZeroSpeech 2020 challenge. In ABX phone discrimination tests, both models\noutperform all submissions to the 2019 and 2020 challenges, with a relative\nimprovement of more than 30%. The models also perform competitively on a\ndownstream voice conversion task. Of the two, VQ-CPC performs slightly better\nin general and is simpler and faster to train. Finally, probing experiments\nshow that vector quantization is an effective bottleneck, forcing the models to\ndiscard speaker information.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.09439v2",
            "date": "2020-05-19 13:35:13+00:00",
            "title": "Functorial Language Games for Question Answering",
            "authors": [
                "Giovanni de Felice",
                "Elena Di Lavore",
                "Mario Rom\u00e1n",
                "Alexis Toumi"
            ],
            "abstract": "We present some categorical investigations into Wittgenstein's\nlanguage-games, with applications to game-theoretic pragmatics and\nquestion-answering in natural language processing.",
            "journal ref": "EPTCS 333, 2021, pp. 311-321",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09740v1",
            "date": "2020-05-19 20:24:02+00:00",
            "title": "GLEAKE: Global and Local Embedding Automatic Keyphrase Extraction",
            "authors": [
                "Javad Rafiei Asl",
                "Juan M. Banda"
            ],
            "abstract": "Automated methods for granular categorization of large corpora of text\ndocuments have become increasingly more important with the rate scientific,\nnews, medical, and web documents are growing in the last few years. Automatic\nkeyphrase extraction (AKE) aims to automatically detect a small set of single\nor multi-words from within a single textual document that captures the main\ntopics of the document. AKE plays an important role in various NLP and\ninformation retrieval tasks such as document summarization and categorization,\nfull-text indexing, and article recommendation. Due to the lack of sufficient\nhuman-labeled data in different textual contents, supervised learning\napproaches are not ideal for automatic detection of keyphrases from the content\nof textual bodies. With the state-of-the-art advances in text embedding\ntechniques, NLP researchers have focused on developing unsupervised methods to\nobtain meaningful insights from raw datasets. In this work, we introduce Global\nand Local Embedding Automatic Keyphrase Extractor (GLEAKE) for the task of AKE.\nGLEAKE utilizes single and multi-word embedding techniques to explore the\nsyntactic and semantic aspects of the candidate phrases and then combines them\ninto a series of embedding-based graphs. Moreover, GLEAKE applies network\nanalysis techniques on each embedding-based graph to refine the most\nsignificant phrases as a final set of keyphrases. We demonstrate the high\nperformance of GLEAKE by evaluating its results on five standard AKE datasets\nfrom different domains and writing styles and by showing its superiority with\nregards to other state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.09803v2",
            "date": "2020-05-20 00:44:38+00:00",
            "title": "A Computational Analysis of Polarization on Indian and Pakistani Social Media",
            "authors": [
                "Aman Tyagi",
                "Anjalie Field",
                "Priyank Lathwal",
                "Yulia Tsvetkov",
                "Kathleen M. Carley"
            ],
            "abstract": "Between February 14, 2019 and March 4, 2019, a terrorist attack in Pulwama,\nKashmir followed by retaliatory airstrikes led to rising tensions between India\nand Pakistan, two nuclear-armed countries. In this work, we examine polarizing\nmessaging on Twitter during these events, particularly focusing on the\npositions of Indian and Pakistani politicians. We use a label propagation\ntechnique focused on hashtag co-occurrences to find polarizing tweets and\nusers. Our analysis reveals that politicians in the ruling political party in\nIndia (BJP) used polarized hashtags and called for escalation of conflict more\nso than politicians from other parties. Our work offers the first analysis of\nhow escalating tensions between India and Pakistan manifest on Twitter and\nprovides a framework for studying polarizing messages.",
            "journal ref": "Social Informatics - 12th International Conference, SocInfo 2020,\n  Pisa, Italy",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.09837v1",
            "date": "2020-05-20 03:34:20+00:00",
            "title": "Positive emotions help rank negative reviews in e-commerce",
            "authors": [
                "Di Weng",
                "Jichang Zhao"
            ],
            "abstract": "Negative reviews, the poor ratings in postpurchase evaluation, play an\nindispensable role in e-commerce, especially in shaping future sales and firm\nequities. However, extant studies seldom examine their potential value for\nsellers and producers in enhancing capabilities of providing better services\nand products. For those who exploited the helpfulness of reviews in the view of\ne-commerce keepers, the ranking approaches were developed for customers\ninstead. To fill this gap, in terms of combining description texts and emotion\npolarities, the aim of the ranking method in this study is to provide the most\nhelpful negative reviews under a certain product attribute for online sellers\nand producers. By applying a more reasonable evaluating procedure, experts with\nrelated backgrounds are hired to vote for the ranking approaches. Our ranking\nmethod turns out to be more reliable for ranking negative reviews for sellers\nand producers, demonstrating a better performance than the baselines like BM25\nwith a result of 8% higher. In this paper, we also enrich the previous\nunderstandings of emotions in valuing reviews. Specifically, it is surprisingly\nfound that positive emotions are more helpful rather than negative emotions in\nranking negative reviews. The unexpected strengthening from positive emotions\nin ranking suggests that less polarized reviews on negative experience in fact\noffer more rational feedbacks and thus more helpfulness to the sellers and\nproducers. The presented ranking method could provide e-commerce practitioners\nwith an efficient and effective way to leverage negative reviews from online\nconsumers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.09946v1",
            "date": "2020-05-20 10:14:01+00:00",
            "title": "GM-CTSC at SemEval-2020 Task 1: Gaussian Mixtures Cross Temporal Similarity Clustering",
            "authors": [
                "Pierluigi Cassotti",
                "Annalina Caputo",
                "Marco Polignano",
                "Pierpaolo Basile"
            ],
            "abstract": "This paper describes the system proposed for the SemEval-2020 Task 1:\nUnsupervised Lexical Semantic Change Detection. We focused our approach on the\ndetection problem. Given the semantics of words captured by temporal word\nembeddings in different time periods, we investigate the use of unsupervised\nmethods to detect when the target word has gained or loosed senses. To this\nend, we defined a new algorithm based on Gaussian Mixture Models to cluster the\ntarget similarities computed over the two periods. We compared the proposed\napproach with a number of similarity-based thresholds. We found that, although\nthe performance of the detection methods varies across the word embedding\nalgorithms, the combination of Gaussian Mixture with Temporal Referencing\nresulted in our best system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10048v1",
            "date": "2020-05-20 13:45:49+00:00",
            "title": "Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources",
            "authors": [
                "Magdalena Biesialska",
                "Bardia Rafieian",
                "Marta R. Costa-juss\u00e0"
            ],
            "abstract": "In this work, we present an effective method for semantic specialization of\nword vector representations. To this end, we use traditional word embeddings\nand apply specialization methods to better capture semantic relations between\nwords. In our approach, we leverage external knowledge from rich lexical\nresources such as BabelNet. We also show that our proposed post-specialization\nmethod based on an adversarial neural network with the Wasserstein distance\nallows to gain improvements over state-of-the-art methods on two tasks: word\nsimilarity and dialog state tracking.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10200v2",
            "date": "2020-05-20 17:05:57+00:00",
            "title": "BERTweet: A pre-trained language model for English Tweets",
            "authors": [
                "Dat Quoc Nguyen",
                "Thanh Vu",
                "Anh Tuan Nguyen"
            ],
            "abstract": "We present BERTweet, the first public large-scale pre-trained language model\nfor English Tweets. Our BERTweet, having the same architecture as BERT-base\n(Devlin et al., 2019), is trained using the RoBERTa pre-training procedure (Liu\net al., 2019). Experiments show that BERTweet outperforms strong baselines\nRoBERTa-base and XLM-R-base (Conneau et al., 2020), producing better\nperformance results than the previous state-of-the-art models on three Tweet\nNLP tasks: Part-of-speech tagging, Named-entity recognition and text\nclassification. We release BERTweet under the MIT License to facilitate future\nresearch and applications on Tweet data. Our BERTweet is available at\nhttps://github.com/VinAIResearch/BERTweet",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10219v1",
            "date": "2020-05-20 17:31:35+00:00",
            "title": "BlaBla: Linguistic Feature Extraction for Clinical Analysis in Multiple Languages",
            "authors": [
                "Abhishek Shivkumar",
                "Jack Weston",
                "Raphael Lenain",
                "Emil Fristed"
            ],
            "abstract": "We introduce BlaBla, an open-source Python library for extracting linguistic\nfeatures with proven clinical relevance to neurological and psychiatric\ndiseases across many languages. BlaBla is a unifying framework for accelerating\nand simplifying clinical linguistic research. The library is built on\nstate-of-the-art NLP frameworks and supports multithreaded/GPU-enabled feature\nextraction via both native Python calls and a command line interface. We\ndescribe BlaBla's architecture and clinical validation of its features across\n12 diseases. We further demonstrate the application of BlaBla to a task\nvisualizing and classifying language disorders in three languages on real\nclinical data from the AphasiaBank dataset. We make the codebase freely\navailable to researchers with the hope of providing a consistent,\nwell-validated foundation for the next generation of clinical linguistic\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10450v1",
            "date": "2020-05-21 03:40:02+00:00",
            "title": "MTSS: Learn from Multiple Domain Teachers and Become a Multi-domain Dialogue Expert",
            "authors": [
                "Shuke Peng",
                "Feng Ji",
                "Zehao Lin",
                "Shaobo Cui",
                "Haiqing Chen",
                "Yin Zhang"
            ],
            "abstract": "How to build a high-quality multi-domain dialogue system is a challenging\nwork due to its complicated and entangled dialogue state space among each\ndomain, which seriously limits the quality of dialogue policy, and further\naffects the generated response. In this paper, we propose a novel method to\nacquire a satisfying policy and subtly circumvent the knotty dialogue state\nrepresentation problem in the multi-domain setting. Inspired by real school\nteaching scenarios, our method is composed of multiple domain-specific teachers\nand a universal student. Each individual teacher only focuses on one specific\ndomain and learns its corresponding domain knowledge and dialogue policy based\non a precisely extracted single domain dialogue state representation. Then,\nthese domain-specific teachers impart their domain knowledge and policies to a\nuniversal student model and collectively make this student model a multi-domain\ndialogue expert. Experiment results show that our method reaches competitive\nresults with SOTAs in both multi-domain and single domain setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.10786v2",
            "date": "2020-05-21 17:08:39+00:00",
            "title": "SafeComp: Protocol For Certifying Cloud Computations Integrity",
            "authors": [
                "Evgeny Shishkin",
                "Evgeny Kislitsyn"
            ],
            "abstract": "We define a problem of certifying computation integrity performed by some\nremote party we do not necessarily trust. We present a multi-party interactive\nprotocol called SafeComp that solves this problem under specified constraints.\nComparing to the nearest related work, our protocol reduces a proof\nconstruction complexity from $O(n \\log{n})$ to $O(n)$, turning a communication\ncomplexity to exactly one round using a certificate of a comparable length.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/2005.10899v1",
            "date": "2020-05-21 20:55:22+00:00",
            "title": "Extracting Daily Dosage from Medication Instructions in EHRs: An Automated Approach and Lessons Learned",
            "authors": [
                "Diwakar Mahajan",
                "Jennifer J. Liang",
                "Ching-Huei Tsou"
            ],
            "abstract": "Understanding a patient's medication history is essential for physicians to\nprovide appropriate treatment recommendations. A medication's prescribed daily\ndosage is a key element of the medication history; however, it is generally not\nprovided as a discrete quantity and needs to be derived from free text\nmedication instructions (Sigs) in the structured electronic health record\n(EHR). Existing works in daily dosage extraction are narrow in scope, dealing\nwith dosage extraction for a single drug from clinical notes. Here, we present\nan automated approach to calculate daily dosage for all medications in EHR\nstructured data. We describe and characterize the variable language used in\nSigs, and present our hybrid system for calculating daily dosage combining deep\nlearning-based named entity extractor with lexicon dictionaries and regular\nexpressions. Our system achieves 0.98 precision and 0.95 recall on an\nexpert-generated dataset of 1000 Sigs, demonstrating its effectiveness on the\ngeneral purpose daily dosage calculation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11153v1",
            "date": "2020-05-19 19:10:49+00:00",
            "title": "Prototypical Q Networks for Automatic Conversational Diagnosis and Few-Shot New Disease Adaption",
            "authors": [
                "Hongyin Luo",
                "Shang-Wen Li",
                "James Glass"
            ],
            "abstract": "Spoken dialog systems have seen applications in many domains, including\nmedical for automatic conversational diagnosis. State-of-the-art dialog\nmanagers are usually driven by deep reinforcement learning models, such as deep\nQ networks (DQNs), which learn by interacting with a simulator to explore the\nentire action space since real conversations are limited. However, the\nDQN-based automatic diagnosis models do not achieve satisfying performances\nwhen adapted to new, unseen diseases with only a few training samples. In this\nwork, we propose the Prototypical Q Networks (ProtoQN) as the dialog manager\nfor the automatic diagnosis systems. The model calculates prototype embeddings\nwith real conversations between doctors and patients, learning from them and\nsimulator-augmented dialogs more efficiently. We create both supervised and\nfew-shot learning tasks with the Muzhi corpus. Experiments showed that the\nProtoQN significantly outperformed the baseline DQN model in both supervised\nand few-shot learning scenarios, and achieves state-of-the-art few-shot\nlearning performances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11176v1",
            "date": "2020-05-22 13:30:37+00:00",
            "title": "RUSSE'2020: Findings of the First Taxonomy Enrichment Task for the Russian language",
            "authors": [
                "Irina Nikishina",
                "Varvara Logacheva",
                "Alexander Panchenko",
                "Natalia Loukachevitch"
            ],
            "abstract": "This paper describes the results of the first shared task on taxonomy\nenrichment for the Russian language. The participants were asked to extend an\nexisting taxonomy with previously unseen words: for each new word their systems\nshould provide a ranked list of possible (candidate) hypernyms. In comparison\nto the previous tasks for other languages, our competition has a more realistic\ntask setting: new words were provided without definitions. Instead, we provided\na textual corpus where these new terms occurred. For this evaluation campaign,\nwe developed a new evaluation dataset based on unpublished RuWordNet data. The\nshared task features two tracks: \"nouns\" and \"verbs\". 16 teams participated in\nthe task demonstrating high results with more than half of them outperforming\nthe provided baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11223v1",
            "date": "2020-05-22 15:01:23+00:00",
            "title": "L2R2: Leveraging Ranking for Abductive Reasoning",
            "authors": [
                "Yunchang Zhu",
                "Liang Pang",
                "Yanyan Lan",
                "Xueqi Cheng"
            ],
            "abstract": "The abductive natural language inference task ($\\alpha$NLI) is proposed to\nevaluate the abductive reasoning ability of a learning system. In the\n$\\alpha$NLI task, two observations are given and the most plausible hypothesis\nis asked to pick out from the candidates. Existing methods simply formulate it\nas a classification problem, thus a cross-entropy log-loss objective is used\nduring training. However, discriminating true from false does not measure the\nplausibility of a hypothesis, for all the hypotheses have a chance to happen,\nonly the probabilities are different. To fill this gap, we switch to a ranking\nperspective that sorts the hypotheses in order of their plausibilities. With\nthis new perspective, a novel $L2R^2$ approach is proposed under the\nlearning-to-rank framework. Firstly, training samples are reorganized into a\nranking form, where two observations and their hypotheses are treated as the\nquery and a set of candidate documents respectively. Then, an ESIM model or\npre-trained language model, e.g. BERT or RoBERTa, is obtained as the scoring\nfunction. Finally, the loss functions for the ranking task can be either\npair-wise or list-wise for training. The experimental results on the ART\ndataset reach the state-of-the-art in the public leaderboard.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.11347v1",
            "date": "2020-05-22 18:32:35+00:00",
            "title": "SentPWNet: A Unified Sentence Pair Weighting Network for Task-specific Sentence Embedding",
            "authors": [
                "Li Zhang",
                "Han Wang",
                "Lingxiao Li"
            ],
            "abstract": "Pair-based metric learning has been widely adopted to learn sentence\nembedding in many NLP tasks such as semantic text similarity due to its\nefficiency in computation. Most existing works employed a sequence encoder\nmodel and utilized limited sentence pairs with a pair-based loss to learn\ndiscriminating sentence representation. However, it is known that the sentence\nrepresentation can be biased when the sampled sentence pairs deviate from the\ntrue distribution of all sentence pairs. In this paper, our theoretical\nanalysis shows that existing works severely suffered from a good pair sampling\nand instance weighting strategy. Instead of one time pair selection and\nlearning on equal weighted pairs, we propose a unified locality weighting and\nlearning framework to learn task-specific sentence embedding. Our model,\nSentPWNet, exploits the neighboring spatial distribution of each sentence as\nlocality weight to indicate the informative level of sentence pair. Such weight\nis updated along with pair-loss optimization in each round, ensuring the model\nkeep learning the most informative sentence pairs. Extensive experiments on\nfour public available datasets and a self-collected place search benchmark with\n1.4 million places clearly demonstrate that our model consistently outperforms\nexisting sentence embedding methods with comparable efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11364v1",
            "date": "2020-05-22 19:39:50+00:00",
            "title": "Open-Retrieval Conversational Question Answering",
            "authors": [
                "Chen Qu",
                "Liu Yang",
                "Cen Chen",
                "Minghui Qiu",
                "W. Bruce Croft",
                "Mohit Iyyer"
            ],
            "abstract": "Conversational search is one of the ultimate goals of information retrieval.\nRecent research approaches conversational search by simplified settings of\nresponse ranking and conversational question answering, where an answer is\neither selected from a given candidate set or extracted from a given passage.\nThese simplifications neglect the fundamental role of retrieval in\nconversational search. To address this limitation, we introduce an\nopen-retrieval conversational question answering (ORConvQA) setting, where we\nlearn to retrieve evidence from a large collection before extracting answers,\nas a further step towards building functional conversational search systems. We\ncreate a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an\nend-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader\nthat are all based on Transformers. Our extensive experiments on OR-QuAC\ndemonstrate that a learnable retriever is crucial for ORConvQA. We further show\nthat our system can make a substantial improvement when we enable history\nmodeling in all system components. Moreover, we show that the reranker\ncomponent contributes to the model performance by providing a regularization\neffect. Finally, further in-depth analyses are performed to provide new\ninsights into ORConvQA.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.11401v4",
            "date": "2020-05-22 21:34:34+00:00",
            "title": "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks",
            "authors": [
                "Patrick Lewis",
                "Ethan Perez",
                "Aleksandra Piktus",
                "Fabio Petroni",
                "Vladimir Karpukhin",
                "Naman Goyal",
                "Heinrich K\u00fcttler",
                "Mike Lewis",
                "Wen-tau Yih",
                "Tim Rockt\u00e4schel",
                "Sebastian Riedel",
                "Douwe Kiela"
            ],
            "abstract": "Large pre-trained language models have been shown to store factual knowledge\nin their parameters, and achieve state-of-the-art results when fine-tuned on\ndownstream NLP tasks. However, their ability to access and precisely manipulate\nknowledge is still limited, and hence on knowledge-intensive tasks, their\nperformance lags behind task-specific architectures. Additionally, providing\nprovenance for their decisions and updating their world knowledge remain open\nresearch problems. Pre-trained models with a differentiable access mechanism to\nexplicit non-parametric memory can overcome this issue, but have so far been\nonly investigated for extractive downstream tasks. We explore a general-purpose\nfine-tuning recipe for retrieval-augmented generation (RAG) -- models which\ncombine pre-trained parametric and non-parametric memory for language\ngeneration. We introduce RAG models where the parametric memory is a\npre-trained seq2seq model and the non-parametric memory is a dense vector index\nof Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG\nformulations, one which conditions on the same retrieved passages across the\nwhole generated sequence, the other can use different passages per token. We\nfine-tune and evaluate our models on a wide range of knowledge-intensive NLP\ntasks and set the state-of-the-art on three open domain QA tasks, outperforming\nparametric seq2seq models and task-specific retrieve-and-extract architectures.\nFor language generation tasks, we find that RAG models generate more specific,\ndiverse and factual language than a state-of-the-art parametric-only seq2seq\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11490v3",
            "date": "2020-05-23 08:29:51+00:00",
            "title": "Summarizing and Exploring Tabular Data in Conversational Search",
            "authors": [
                "Shuo Zhang",
                "Zhuyun Dai",
                "Krisztian Balog",
                "Jamie Callan"
            ],
            "abstract": "Tabular data provide answers to a significant portion of search queries.\nHowever, reciting an entire result table is impractical in conversational\nsearch systems. We propose to generate natural language summaries as answers to\ndescribe the complex information contained in a table. Through crowdsourcing\nexperiments, we build a new conversation-oriented, open-domain table\nsummarization dataset. It includes annotated table summaries, which not only\nanswer questions but also help people explore other information in the table.\nWe utilize this dataset to develop automatic table summarization systems as\nSOTA baselines. Based on the experimental results, we identify challenges and\npoint out future research directions that this resource will support.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.11640v3",
            "date": "2020-05-24 02:26:13+00:00",
            "title": "Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding",
            "authors": [
                "Chen Liu",
                "Su Zhu",
                "Zijian Zhao",
                "Ruisheng Cao",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Spoken Language Understanding (SLU) converts hypotheses from automatic speech\nrecognizer (ASR) into structured semantic representations. ASR recognition\nerrors can severely degenerate the performance of the subsequent SLU module. To\naddress this issue, word confusion networks (WCNs) have been used to encode the\ninput for SLU, which contain richer information than 1-best or n-best\nhypotheses list. To further eliminate ambiguity, the last system act of\ndialogue context is also utilized as additional input. In this paper, a novel\nBERT based SLU model (WCN-BERT SLU) is proposed to encode WCNs and the dialogue\ncontext jointly. It can integrate both structural information and ASR posterior\nprobabilities of WCNs in the BERT architecture. Experiments on DSTC2, a\nbenchmark of SLU, show that the proposed method is effective and can outperform\nprevious state-of-the-art models significantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11665v1",
            "date": "2020-05-24 07:09:04+00:00",
            "title": "A Question Type Driven and Copy Loss Enhanced Frameworkfor Answer-Agnostic Neural Question Generation",
            "authors": [
                "Xiuyu Wu",
                "Nan Jiang",
                "Yunfang Wu"
            ],
            "abstract": "The answer-agnostic question generation is a significant and challenging\ntask, which aims to automatically generate questions for a given sentence but\nwithout an answer. In this paper, we propose two new strategies to deal with\nthis task: question type prediction and copy loss mechanism. The question type\nmodule is to predict the types of questions that should be asked, which allows\nour model to generate multiple types of questions for the same source sentence.\nThe new copy loss enhances the original copy mechanism to make sure that every\nimportant word in the source sentence has been copied when generating\nquestions. Our integrated model outperforms the state-of-the-art approach in\nanswer-agnostic question generation, achieving a BLEU-4 score of 13.9 on SQuAD.\nHuman evaluation further validates the high quality of our generated questions.\nWe will make our code public available for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11723v1",
            "date": "2020-05-24 11:37:22+00:00",
            "title": "Query Resolution for Conversational Search with Limited Supervision",
            "authors": [
                "Nikos Voskarides",
                "Dan Li",
                "Pengjie Ren",
                "Evangelos Kanoulas",
                "Maarten de Rijke"
            ],
            "abstract": "In this work we focus on multi-turn passage retrieval as a crucial component\nof conversational search. One of the key challenges in multi-turn passage\nretrieval comes from the fact that the current turn query is often\nunderspecified due to zero anaphora, topic change, or topic return. Context\nfrom the conversational history can be used to arrive at a better expression of\nthe current turn query, defined as the task of query resolution. In this paper,\nwe model the query resolution task as a binary term classification problem: for\neach term appearing in the previous turns of the conversation decide whether to\nadd it to the current turn query or not. We propose QuReTeC (Query Resolution\nby Term Classification), a neural query resolution model based on bidirectional\ntransformers. We propose a distant supervision method to automatically generate\ntraining data by using query-passage relevance labels. Such labels are often\nreadily available in a collection either as human annotations or inferred from\nuser interactions. We show that QuReTeC outperforms state-of-the-art models,\nand furthermore, that our distant supervision method can be used to\nsubstantially reduce the amount of human-curated data required to train\nQuReTeC. We incorporate QuReTeC in a multi-turn, multi-stage passage retrieval\narchitecture and demonstrate its effectiveness on the TREC CAsT dataset.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.11729v2",
            "date": "2020-05-24 12:14:19+00:00",
            "title": "GoChat: Goal-oriented Chatbots with Hierarchical Reinforcement Learning",
            "authors": [
                "Jianfeng Liu",
                "Feiyang Pan",
                "Ling Luo"
            ],
            "abstract": "A chatbot that converses like a human should be goal-oriented (i.e., be\npurposeful in conversation), which is beyond language generation. However,\nexisting dialogue systems often heavily rely on cumbersome hand-crafted rules\nor costly labelled datasets to reach the goals. In this paper, we propose\nGoal-oriented Chatbots (GoChat), a framework for end-to-end training chatbots\nto maximize the longterm return from offline multi-turn dialogue datasets. Our\nframework utilizes hierarchical reinforcement learning (HRL), where the\nhigh-level policy guides the conversation towards the final goal by determining\nsome sub-goals, and the low-level policy fulfills the sub-goals by generating\nthe corresponding utterance for response. In our experiments on a real-world\ndialogue dataset for anti-fraud in financial, our approach outperforms previous\nmethods on both the quality of response generation as well as the success rate\nof accomplishing the goal.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11739v1",
            "date": "2020-05-24 13:02:57+00:00",
            "title": "Adversarial NLI for Factual Correctness in Text Summarisation Models",
            "authors": [
                "Mario Barrantes",
                "Benedikt Herudek",
                "Richard Wang"
            ],
            "abstract": "We apply the Adversarial NLI dataset to train the NLI model and show that the\nmodel has the potential to enhance factual correctness in abstract\nsummarization. We follow the work of Falke et al. (2019), which rank multiple\ngenerated summaries based on the entailment probabilities between an source\ndocument and summaries and select the summary that has the highest entailment\nprobability. The authors' earlier study concluded that current NLI models are\nnot sufficiently accurate for the ranking task. We show that the Transformer\nmodels fine-tuned on the new dataset achieve significantly higher accuracy and\nhave the potential of selecting a coherent summary.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11861v1",
            "date": "2020-05-24 23:44:45+00:00",
            "title": "ON-TRAC Consortium for End-to-End and Simultaneous Speech Translation Challenge Tasks at IWSLT 2020",
            "authors": [
                "Maha Elbayad",
                "Ha Nguyen",
                "Fethi Bougares",
                "Natalia Tomashenko",
                "Antoine Caubri\u00e8re",
                "Benjamin Lecouteux",
                "Yannick Est\u00e8ve",
                "Laurent Besacier"
            ],
            "abstract": "This paper describes the ON-TRAC Consortium translation systems developed for\ntwo challenge tracks featured in the Evaluation Campaign of IWSLT 2020, offline\nspeech translation and simultaneous speech translation. ON-TRAC Consortium is\ncomposed of researchers from three French academic laboratories: LIA (Avignon\nUniversit\\'e), LIG (Universit\\'e Grenoble Alpes), and LIUM (Le Mans\nUniversit\\'e). Attention-based encoder-decoder models, trained end-to-end, were\nused for our submissions to the offline speech translation track. Our\ncontributions focused on data augmentation and ensembling of multiple models.\nIn the simultaneous speech translation track, we build on Transformer-based\nwait-k models for the text-to-text subtask. For speech-to-text simultaneous\ntranslation, we attach a wait-k MT system to a hybrid ASR system. We propose an\nalgorithm to control the latency of the ASR+MT cascade and achieve a good\nlatency-quality trade-off on both subtasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.11888v1",
            "date": "2020-05-25 02:20:18+00:00",
            "title": "AutoSUM: Automating Feature Extraction and Multi-user Preference Simulation for Entity Summarization",
            "authors": [
                "Dongjun Wei",
                "Yaxin Liu",
                "Fuqing Zhu",
                "Liangjun Zang",
                "Wei Zhou",
                "Yijun Lu",
                "Songlin Hu"
            ],
            "abstract": "Withthegrowthofknowledgegraphs, entity descriptions are becoming extremely\nlengthy. Entity summarization task, aiming to generate diverse, comprehensive,\nand representative summaries for entities, has received increasing interest\nrecently. In most previous methods, features are usually extracted by the\nhandcrafted templates. Then the feature selection and multi-user preference\nsimulation take place, depending too much on human expertise. In this paper, a\nnovel integration method called AutoSUM is proposed for automatic feature\nextraction and multi-user preference simulation to overcome the drawbacks of\nprevious methods. There are two modules in AutoSUM: extractor and simulator.\nThe extractor module operates automatic feature extraction based on a BiLSTM\nwith a combined input representation including word embeddings and graph\nembeddings. Meanwhile, the simulator module automates multi-user preference\nsimulation based on a well-designed two-phase attention mechanism (i.e.,\nentity-phase attention and user-phase attention). Experimental results\ndemonstrate that AutoSUM produces state-of-the-art performance on two widely\nused datasets (i.e., DBpedia and LinkedMDB) in both F-measure and MAP.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.11988v1",
            "date": "2020-05-25 09:12:37+00:00",
            "title": "Deep Learning Models for Automatic Summarization",
            "authors": [
                "Pirmin Lemberger"
            ],
            "abstract": "Text summarization is an NLP task which aims to convert a textual document\ninto a shorter one while keeping as much meaning as possible. This pedagogical\narticle reviews a number of recent Deep Learning architectures that have helped\nto advance research in this field. We will discuss in particular applications\nof pointer networks, hierarchical Transformers and Reinforcement Learning. We\nassume basic knowledge of Seq2Seq architecture and Transformer networks within\nNLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12132v1",
            "date": "2020-05-25 14:15:49+00:00",
            "title": "Degree-Aware Alignment for Entities in Tail",
            "authors": [
                "Weixin Zeng",
                "Xiang Zhao",
                "Wei Wang",
                "Jiuyang Tang",
                "Zhen Tan"
            ],
            "abstract": "Entity alignment (EA) is to discover equivalent entities in knowledge graphs\n(KGs), which bridges heterogeneous sources of information and facilitates the\nintegration of knowledge. Existing EA solutions mainly rely on structural\ninformation to align entities, typically through KG embedding. Nonetheless, in\nreal-life KGs, only a few entities are densely connected to others, and the\nrest majority possess rather sparse neighborhood structure. We refer to the\nlatter as long-tail entities, and observe that such phenomenon arguably limits\nthe use of structural information for EA. To mitigate the issue, we revisit and\ninvestigate into the conventional EA pipeline in pursuit of elegant\nperformance. For pre-alignment, we propose to amplify long-tail entities, which\nare of relatively weak structural information, with entity name information\nthat is generally available (but overlooked) in the form of concatenated power\nmean word embeddings. For alignment, under a novel complementary framework of\nconsolidating structural and name signals, we identify entity's degree as\nimportant guidance to effectively fuse two different sources of information. To\nthis end, a degree-aware co-attention network is conceived, which dynamically\nadjusts the significance of features in a degree-aware manner. For\npost-alignment, we propose to complement original KGs with facts from their\ncounterparts by using confident EA results as anchors via iterative training.\nComprehensive experimental evaluations validate the superiority of our proposed\ntechniques.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.12187v2",
            "date": "2020-05-25 15:58:00+00:00",
            "title": "AMR Quality Rating with a Lightweight CNN",
            "authors": [
                "Juri Opitz"
            ],
            "abstract": "Structured semantic sentence representations such as Abstract Meaning\nRepresentations (AMRs) are potentially useful in various NLP tasks. However,\nthe quality of automatic parses can vary greatly and jeopardizes their\nusefulness. This can be mitigated by models that can accurately rate AMR\nquality in the absence of costly gold data, allowing us to inform downstream\nsystems about an incorporated parse's trustworthiness or select among different\ncandidate parses.\n  In this work, we propose to transfer the AMR graph to the domain of images.\nThis allows us to create a simple convolutional neural network (CNN) that\nimitates a human judge tasked with rating graph quality. Our experiments show\nthat the method can rate quality more accurately than strong baselines, in\nseveral quality dimensions. Moreover, the method proves to be efficient and\nreduces the incurred energy consumption.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12385v1",
            "date": "2020-05-08 13:26:52+00:00",
            "title": "Personalized Early Stage Alzheimer's Disease Detection: A Case Study of President Reagan's Speeches",
            "authors": [
                "Ning Wang",
                "Fan Luo",
                "Vishal Peddagangireddy",
                "K. P. Subbalakshmi",
                "R. Chandramouli"
            ],
            "abstract": "Alzheimer`s disease (AD)-related global healthcare cost is estimated to be $1\ntrillion by 2050. Currently, there is no cure for this disease; however,\nclinical studies show that early diagnosis and intervention helps to extend the\nquality of life and inform technologies for personalized mental healthcare.\nClinical research indicates that the onset and progression of Alzheimer`s\ndisease lead to dementia and other mental health issues. As a result, the\nlanguage capabilities of patient start to decline. In this paper, we show that\nmachine learning-based unsupervised clustering of and anomaly detection with\nlinguistic biomarkers are promising approaches for intuitive visualization and\npersonalized early stage detection of Alzheimer`s disease. We demonstrate this\napproach on 10 year`s (1980 to 1989) of President Ronald Reagan`s speech data\nset. Key linguistic biomarkers that indicate early-stage AD are identified.\nExperimental results show that Reagan had early onset of Alzheimer`s sometime\nbetween 1983 and 1987. This finding is corroborated by prior work that analyzed\nhis interviews using a statistical technique. The proposed technique also\nidentifies the exact speeches that reflect linguistic biomarkers for early\nstage AD.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2005.12411v1",
            "date": "2020-05-25 21:23:52+00:00",
            "title": "The IMS-CUBoulder System for the SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion",
            "authors": [
                "Manuel Mager",
                "Katharina Kann"
            ],
            "abstract": "In this paper, we present the systems of the University of Stuttgart IMS and\nthe University of Colorado Boulder (IMS-CUBoulder) for SIGMORPHON 2020 Task 2\non unsupervised morphological paradigm completion (Kann et al., 2020). The task\nconsists of generating the morphological paradigms of a set of lemmas, given\nonly the lemmas themselves and unlabeled text. Our proposed system is a\nmodified version of the baseline introduced together with the task. In\nparticular, we experiment with substituting the inflection generation component\nwith an LSTM sequence-to-sequence model and an LSTM pointer-generator network.\nOur pointer-generator system obtains the best score of all seven submitted\nsystems on average over all languages, and outperforms the official baseline,\nwhich was best overall, on Bulgarian and Kannada.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12503v1",
            "date": "2020-05-26 03:34:01+00:00",
            "title": "BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection",
            "authors": [
                "Jihyung Moon",
                "Won Ik Cho",
                "Junbum Lee"
            ],
            "abstract": "Toxic comments in online platforms are an unavoidable social issue under the\ncloak of anonymity. Hate speech detection has been actively done for languages\nsuch as English, German, or Italian, where manually labeled corpus has been\nreleased. In this work, we first present 9.4K manually labeled entertainment\nnews comments for identifying Korean toxic speech, collected from a widely used\nonline news platform in Korea. The comments are annotated regarding social bias\nand hate speech since both aspects are correlated. The inter-annotator\nagreement Krippendorff's alpha score is 0.492 and 0.496, respectively. We\nprovide benchmarks using CharCNN, BiLSTM, and BERT, where BERT achieves the\nhighest score on all tasks. The models generally display better performance on\nbias identification, since the hate speech detection is a more subjective\nissue. Additionally, when BERT is trained with bias label for hate speech\ndetection, the prediction score increases, implying that bias and hate are\nintertwined. We make our dataset publicly available and open competitions with\nthe corpus and benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12529v4",
            "date": "2020-05-26 06:09:57+00:00",
            "title": "Policy-Driven Neural Response Generation for Knowledge-Grounded Dialogue Systems",
            "authors": [
                "Behnam Hedayatnia",
                "Karthik Gopalakrishnan",
                "Seokhwan Kim",
                "Yang Liu",
                "Mihail Eric",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Open-domain dialogue systems aim to generate relevant, informative and\nengaging responses. Seq2seq neural response generation approaches do not have\nexplicit mechanisms to control the content or style of the generated response,\nand frequently result in uninformative utterances. In this paper, we propose\nusing a dialogue policy to plan the content and style of target responses in\nthe form of an action plan, which includes knowledge sentences related to the\ndialogue context, targeted dialogue acts, topic information, etc. The\nattributes within the action plan are obtained by automatically annotating the\npublicly released Topical-Chat dataset. We condition neural response generators\non the action plan which is then realized as target utterances at the turn and\nsentence levels. We also investigate different dialogue policy models to\npredict an action plan given the dialogue context. Through automated and human\nevaluation, we measure the appropriateness of the generated responses and check\nif the generation models indeed learn to realize the given action plans. We\ndemonstrate that a basic dialogue policy that operates at the sentence level\ngenerates better responses in comparison to turn level generation as well as\nbaseline models with no action plan. Additionally the basic dialogue policy has\nthe added effect of controllability.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.12533v1",
            "date": "2020-05-26 06:18:47+00:00",
            "title": "Guiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities",
            "authors": [
                "Ben Goertzel",
                "Andres Suarez Madrigal",
                "Gino Yu"
            ],
            "abstract": "A novel approach to automated learning of syntactic rules governing natural\nlanguages is proposed, based on using probabilities assigned to sentences (and\npotentially longer word sequences) by transformer neural network language\nmodels to guide symbolic learning processes like clustering and rule induction.\nThis method exploits the learned linguistic knowledge in transformers, without\nany reference to their inner representations; hence, the technique is readily\nadaptable to the continuous appearance of more powerful language models. We\nshow a proof-of-concept example of our proposed technique, using it to guide\nunsupervised symbolic link-grammar induction methods drawn from our prior\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12535v1",
            "date": "2020-05-26 06:20:32+00:00",
            "title": "Embedding Vector Differences Can Be Aligned With Uncertain Intensional Logic Differences",
            "authors": [
                "Ben Goertzel",
                "Mike Duncan",
                "Debbie Duong",
                "Nil Geisweiller",
                "Hedra Seid",
                "Abdulrahman Semrie",
                "Man Hin Leung",
                "Matthew Ikle'"
            ],
            "abstract": "The DeepWalk algorithm is used to assign embedding vectors to nodes in the\nAtomspace weighted, labeled hypergraph that is used to represent knowledge in\nthe OpenCog AGI system, in the context of an application to probabilistic\ninference regarding the causes of longevity based on data from biological\nontologies and genomic analyses. It is shown that vector difference operations\nbetween embedding vectors are, in appropriate conditions, approximately\nalignable with \"intensional difference\" operations between the hypergraph nodes\ncorresponding to the embedding vectors. This relationship hints at a broader\nfunctorial mapping between uncertain intensional logic and vector arithmetic,\nand opens the door for using embedding vector algebra to guide intensional\ninference control.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2005.12565v1",
            "date": "2020-05-26 08:15:32+00:00",
            "title": "A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction",
            "authors": [
                "Saadullah Amin",
                "Katherine Ann Dunfield",
                "Anna Vechkaeva",
                "G\u00fcnter Neumann"
            ],
            "abstract": "Fact triples are a common form of structured knowledge used within the\nbiomedical domain. As the amount of unstructured scientific texts continues to\ngrow, manual annotation of these texts for the task of relation extraction\nbecomes increasingly expensive. Distant supervision offers a viable approach to\ncombat this by quickly producing large amounts of labeled, but considerably\nnoisy, data. We aim to reduce such noise by extending an entity-enriched\nrelation classification BERT model to the problem of multiple instance\nlearning, and defining a simple data encoding scheme that significantly reduces\nnoise, reaching state-of-the-art performance for distantly-supervised\nbiomedical relation extraction. Our approach further encodes knowledge about\nthe direction of relation triples, allowing for increased focus on relation\nlearning by reducing noise and alleviating the need for joint learning with\nknowledge graph completion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12592v2",
            "date": "2020-05-26 09:33:02+00:00",
            "title": "GECToR -- Grammatical Error Correction: Tag, Not Rewrite",
            "authors": [
                "Kostiantyn Omelianchuk",
                "Vitaliy Atrasevych",
                "Artem Chernodub",
                "Oleksandr Skurzhanskyi"
            ],
            "abstract": "In this paper, we present a simple and efficient GEC sequence tagger using a\nTransformer encoder. Our system is pre-trained on synthetic data and then\nfine-tuned in two stages: first on errorful corpora, and second on a\ncombination of errorful and error-free parallel corpora. We design custom\ntoken-level transformations to map input tokens to target corrections. Our best\nsingle-model/ensemble GEC tagger achieves an $F_{0.5}$ of 65.3/66.5 on\nCoNLL-2014 (test) and $F_{0.5}$ of 72.4/73.6 on BEA-2019 (test). Its inference\nspeed is up to 10 times as fast as a Transformer-based seq2seq GEC system. The\ncode and trained models are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12762v2",
            "date": "2020-05-26 14:34:07+00:00",
            "title": "Exploring aspects of similarity between spoken personal narratives by disentangling them into narrative clause types",
            "authors": [
                "Belen Saldias",
                "Deb Roy"
            ],
            "abstract": "Sharing personal narratives is a fundamental aspect of human social behavior\nas it helps share our life experiences. We can tell stories and rely on our\nbackground to understand their context, similarities, and differences. A\nsubstantial effort has been made towards developing storytelling machines or\ninferring characters' features. However, we don't usually find models that\ncompare narratives. This task is remarkably challenging for machines since\nthey, as sometimes we do, lack an understanding of what similarity means. To\naddress this challenge, we first introduce a corpus of real-world spoken\npersonal narratives comprising 10,296 narrative clauses from 594 video\ntranscripts. Second, we ask non-narrative experts to annotate those clauses\nunder Labov's sociolinguistic model of personal narratives (i.e., action,\norientation, and evaluation clause types) and train a classifier that reaches\n84.7% F-score for the highest-agreed clauses. Finally, we match stories and\nexplore whether people implicitly rely on Labov's framework to compare\nnarratives. We show that actions followed by the narrator's evaluation of these\nare the aspects non-experts consider the most. Our approach is intended to help\ninform machine learning methods aimed at studying or representing personal\nnarratives.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.12816v1",
            "date": "2020-05-26 15:47:42+00:00",
            "title": "Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistants",
            "authors": [
                "Christophe Van Gysel",
                "Manos Tsagkias",
                "Ernest Pusateri",
                "Ilya Oparin"
            ],
            "abstract": "We focus on improving the effectiveness of a Virtual Assistant (VA) in\nrecognizing emerging entities in spoken queries. We introduce a method that\nuses historical user interactions to forecast which entities will gain in\npopularity and become trending, and it subsequently integrates the predictions\nwithin the Automated Speech Recognition (ASR) component of the VA. Experiments\nshow that our proposed approach results in a 20% relative reduction in errors\non emerging entity name utterances without degrading the overall recognition\nquality of the system.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.12994v1",
            "date": "2020-05-26 19:21:57+00:00",
            "title": "A Study of Neural Matching Models for Cross-lingual IR",
            "authors": [
                "Puxuan Yu",
                "James Allan"
            ],
            "abstract": "In this study, we investigate interaction-based neural matching models for\nad-hoc cross-lingual information retrieval (CLIR) using cross-lingual word\nembeddings (CLWEs). With experiments conducted on the CLEF collection over four\nlanguage pairs, we evaluate and provide insight into different neural model\narchitectures, different ways to represent query-document interactions and\nword-pair similarity distributions in CLIR. This study paves the way for\nlearning an end-to-end CLIR system using CLWEs.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.13041v1",
            "date": "2020-05-26 21:02:43+00:00",
            "title": "Examining Racial Bias in an Online Abuse Corpus with Structural Topic Modeling",
            "authors": [
                "Thomas Davidson",
                "Debasmita Bhattacharya"
            ],
            "abstract": "We use structural topic modeling to examine racial bias in data collected to\ntrain models to detect hate speech and abusive language in social media posts.\nWe augment the abusive language dataset by adding an additional feature\nindicating the predicted probability of the tweet being written in\nAfrican-American English. We then use structural topic modeling to examine the\ncontent of the tweets and how the prevalence of different topics is related to\nboth abusiveness annotation and dialect prediction. We find that certain topics\nare disproportionately racialized and considered abusive. We discuss how topic\nmodeling may be a useful approach for identifying bias in annotated data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13125v1",
            "date": "2020-05-27 02:02:57+00:00",
            "title": "Counterfactual Detection meets Transfer Learning",
            "authors": [
                "Kelechi Nwaike",
                "Licheng Jiao"
            ],
            "abstract": "We can consider Counterfactuals as belonging in the domain of Discourse\nstructure and semantics, A core area in Natural Language Understanding and in\nthis paper, we introduce an approach to resolving counterfactual detection as\nwell as the indexing of the antecedents and consequents of Counterfactual\nstatements. While Transfer learning is already being applied to several NLP\ntasks, It has the characteristics to excel in a novel number of tasks. We show\nthat detecting Counterfactuals is a straightforward Binary Classification Task\nthat can be implemented with minimal adaptation on already existing model\nArchitectures, thanks to a well annotated training data set,and we introduce a\nnew end to end pipeline to process antecedents and consequents as an entity\nrecognition task, thus adapting them into Token Classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13170v1",
            "date": "2020-05-27 05:34:12+00:00",
            "title": "Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models",
            "authors": [
                "Haochen Liu",
                "Zhiwei Wang",
                "Tyler Derr",
                "Jiliang Tang"
            ],
            "abstract": "Recently, neural network based dialogue systems have become ubiquitous in our\nincreasingly digitalized society. However, due to their inherent opaqueness,\nsome recently raised concerns about using neural models are starting to be\ntaken seriously. In fact, intentional or unintentional behaviors could lead to\na dialogue system to generate inappropriate responses. Thus, in this paper, we\ninvestigate whether we can learn to craft input sentences that result in a\nblack-box neural dialogue model being manipulated into having its outputs\ncontain target words or match target sentences. We propose a reinforcement\nlearning based model that can generate such desired inputs automatically.\nExtensive experiments on a popular well-trained state-of-the-art neural\ndialogue model show that our method can successfully seek out desired inputs\nthat lead to the target outputs in a considerable portion of cases.\nConsequently, our work reveals the potential of neural dialogue models to be\nmanipulated, which inspires and opens the door towards developing strategies to\ndefend them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13263v2",
            "date": "2020-05-27 09:59:34+00:00",
            "title": "Catching Attention with Automatic Pull Quote Selection",
            "authors": [
                "Tanner Bohn",
                "Charles X. Ling"
            ],
            "abstract": "To advance understanding on how to engage readers, we advocate the novel task\nof automatic pull quote selection. Pull quotes are a component of articles\nspecifically designed to catch the attention of readers with spans of text\nselected from the article and given more salient presentation. This task\ndiffers from related tasks such as summarization and clickbait identification\nby several aspects. We establish a spectrum of baseline approaches to the task,\nranging from handcrafted features to a neural mixture-of-experts to cross-task\nmodels. By examining the contributions of individual features and embedding\ndimensions from these models, we uncover unexpected properties of pull quotes\nto help answer the important question of what engages readers. Human evaluation\nalso supports the uniqueness of this task and the suitability of our selection\nmodels. The benefits of exploring this problem further are clear: pull quotes\nincrease enjoyment and readability, shape reader perceptions, and facilitate\nlearning. Code to reproduce this work is available at\nhttps://github.com/tannerbohn/AutomaticPullQuoteSelection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13342v1",
            "date": "2020-05-27 13:16:19+00:00",
            "title": "Towards an Open Platform for Legal Information",
            "authors": [
                "Malte Ostendorff",
                "Till Blume",
                "Saskia Ostendorff"
            ],
            "abstract": "Recent advances in the area of legal information systems have led to a\nvariety of applications that promise support in processing and accessing legal\ndocuments. Unfortunately, these applications have various limitations, e.g.,\nregarding scope or extensibility. Furthermore, we do not observe a trend\ntowards open access in digital libraries in the legal domain as we observe in\nother domains, e.g., economics of computer science. To improve open access in\nthe legal domain, we present our approach for an open source platform to\ntransparently process and access Legal Open Data. This enables the sustainable\ndevelopment of legal applications by offering a single technology stack.\nMoreover, the approach facilitates the development and deployment of new\ntechnologies. As proof of concept, we implemented six technologies and\ngenerated metadata for more than 250,000 German laws and court decisions. Thus,\nwe can provide users of our platform not only access to legal documents, but\nalso the contained information.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13362v2",
            "date": "2020-05-27 13:46:11+00:00",
            "title": "A Multi-modal Approach to Fine-grained Opinion Mining on Video Reviews",
            "authors": [
                "Edison Marrese-Taylor",
                "Cristian Rodriguez-Opazo",
                "Jorge A. Balazs",
                "Stephen Gould",
                "Yutaka Matsuo"
            ],
            "abstract": "Despite the recent advances in opinion mining for written reviews, few works\nhave tackled the problem on other sources of reviews. In light of this issue,\nwe propose a multi-modal approach for mining fine-grained opinions from video\nreviews that is able to determine the aspects of the item under review that are\nbeing discussed and the sentiment orientation towards them. Our approach works\nat the sentence level without the need for time annotations and uses features\nderived from the audio, video and language transcriptions of its contents. We\nevaluate our approach on two datasets and show that leveraging the video and\naudio modalities consistently provides increased performance over text-only\nbaselines, providing evidence these extra modalities are key in better\nunderstanding video reviews.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13485v3",
            "date": "2020-05-27 16:47:44+00:00",
            "title": "Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing",
            "authors": [
                "Ruisheng Cao",
                "Su Zhu",
                "Chenyu Yang",
                "Chen Liu",
                "Rao Ma",
                "Yanbin Zhao",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "One daunting problem for semantic parsing is the scarcity of annotation.\nAiming to reduce nontrivial human labor, we propose a two-stage semantic\nparsing framework, where the first stage utilizes an unsupervised paraphrase\nmodel to convert an unlabeled natural language utterance into the canonical\nutterance. The downstream naive semantic parser accepts the intermediate output\nand returns the target logical form. Furthermore, the entire training process\nis split into two phases: pre-training and cycle learning. Three tailored\nself-supervised tasks are introduced throughout training to activate the\nunsupervised paraphrase model. Experimental results on benchmarks Overnight and\nGeoGranno demonstrate that our framework is effective and compatible with\nsupervised training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13486v1",
            "date": "2020-05-27 16:49:04+00:00",
            "title": "Neural Temporal Opinion Modelling for Opinion Prediction on Twitter",
            "authors": [
                "Lixing Zhu",
                "Yulan He",
                "Deyu Zhou"
            ],
            "abstract": "Opinion prediction on Twitter is challenging due to the transient nature of\ntweet content and neighbourhood context. In this paper, we model users' tweet\nposting behaviour as a temporal point process to jointly predict the posting\ntime and the stance label of the next tweet given a user's historical tweet\nsequence and tweets posted by their neighbours. We design a topic-driven\nattention mechanism to capture the dynamic topic shifts in the neighbourhood\ncontext. Experimental results show that the proposed model predicts both the\nposting time and the stance labels of future tweets more accurately compared to\na number of competitive baselines.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2005.13575v1",
            "date": "2020-05-27 18:10:46+00:00",
            "title": "In search of isoglosses: continuous and discrete language embeddings in Slavic historical phonology",
            "authors": [
                "Chundra A. Cathcart",
                "Florian Wandl"
            ],
            "abstract": "This paper investigates the ability of neural network architectures to\neffectively learn diachronic phonological generalizations in a multilingual\nsetting. We employ models using three different types of language embedding\n(dense, sigmoid, and straight-through). We find that the Straight-Through model\noutperforms the other two in terms of accuracy, but the Sigmoid model's\nlanguage embeddings show the strongest agreement with the traditional\nsubgrouping of the Slavic languages. We find that the Straight-Through model\nhas learned coherent, semi-interpretable information about sound change, and\noutline directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.13783v2",
            "date": "2020-05-28 05:20:00+00:00",
            "title": "JointMap: Joint Query Intent Understanding For Modeling Intent Hierarchies in E-commerce Search",
            "authors": [
                "Ali Ahmadvand",
                "Surya Kallumadi",
                "Faizan Javed",
                "Eugene Agichtein"
            ],
            "abstract": "An accurate understanding of a user's query intent can help improve the\nperformance of downstream tasks such as query scoping and ranking. In the\ne-commerce domain, recent work in query understanding focuses on the query to\nproduct-category mapping. But, a small yet significant percentage of queries\n(in our website 1.5% or 33M queries in 2019) have non-commercial intent\nassociated with them. These intents are usually associated with non-commercial\ninformation seeking needs such as discounts, store hours, installation guides,\netc. In this paper, we introduce Joint Query Intent Understanding (JointMap), a\ndeep learning model to simultaneously learn two different high-level user\nintent tasks: 1) identifying a query's commercial vs. non-commercial intent,\nand 2) associating a set of relevant product categories in taxonomy to a\nproduct query. JointMap model works by leveraging the transfer bias that exists\nbetween these two related tasks through a joint-learning process. As curating a\nlabeled data set for these tasks can be expensive and time-consuming, we\npropose a distant supervision approach in conjunction with an active learning\nmodel to generate high-quality training data sets. To demonstrate the\neffectiveness of JointMap, we use search queries collected from a large\ncommercial website. Our results show that JointMap significantly improves both\n\"commercial vs. non-commercial\" intent prediction and product category mapping\nby 2.3% and 10% on average over state-of-the-art deep learning methods. Our\nfindings suggest a promising direction to model the intent hierarchies in an\ne-commerce search engine.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.13808v2",
            "date": "2020-05-28 07:04:42+00:00",
            "title": "User Intent Inference for Web Search and Conversational Agents",
            "authors": [
                "Ali Ahmadvand"
            ],
            "abstract": "User intent understanding is a crucial step in designing both conversational\nagents and search engines. Detecting or inferring user intent is challenging,\nsince the user utterances or queries can be short, ambiguous, and contextually\ndependent. To address these research challenges, my thesis work focuses on: 1)\nUtterance topic and intent classification for conversational agents 2) Query\nintent mining and classification for Web search engines, focusing on the\ne-commerce domain. To address the first topic, I proposed novel models to\nincorporate entity information and conversation-context clues to predict both\ntopic and intent of the user's utterances. For the second research topic, I\nplan to extend the existing state of the art methods in Web search intent\nprediction to the e-commerce domain, via: 1) Developing a joint learning model\nto predict search queries' intents and the product categories associated with\nthem, 2) Discovering new hidden users' intents. All the models will be\nevaluated on the real queries available from a major e-commerce site search\nengine. The results from these studies can be leveraged to improve performance\nof various tasks such as natural language understanding, query scoping, query\nsuggestion, and ranking, resulting in an enriched user experience.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.13837v5",
            "date": "2020-05-28 08:26:06+00:00",
            "title": "Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs",
            "authors": [
                "Dong Bok Lee",
                "Seanie Lee",
                "Woo Tae Jeong",
                "Donghwan Kim",
                "Sung Ju Hwang"
            ],
            "abstract": "One of the most crucial challenges in question answering (QA) is the scarcity\nof labeled data, since it is costly to obtain question-answer (QA) pairs for a\ntarget text domain with human annotation. An alternative approach to tackle the\nproblem is to use automatically generated QA pairs from either the problem\ncontext or from large amount of unstructured texts (e.g. Wikipedia). In this\nwork, we propose a hierarchical conditional variational autoencoder (HCVAE) for\ngenerating QA pairs given unstructured texts as contexts, while maximizing the\nmutual information between generated QA pairs to ensure their consistency. We\nvalidate our Information Maximizing Hierarchical Conditional Variational\nAutoEncoder (Info-HCVAE) on several benchmark datasets by evaluating the\nperformance of the QA model (BERT-base) using only the generated QA pairs\n(QA-based evaluation) or by using both the generated and human-labeled pairs\n(semi-supervised learning) for training, against state-of-the-art baseline\nmodels. The results show that our model obtains impressive performance gains\nover all baselines on both tasks, using only a fraction of data for training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14028v1",
            "date": "2020-05-28 14:08:40+00:00",
            "title": "Joint Modelling of Emotion and Abusive Language Detection",
            "authors": [
                "Santhosh Rajamanickam",
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "The rise of online communication platforms has been accompanied by some\nundesirable effects, such as the proliferation of aggressive and abusive\nbehaviour online. Aiming to tackle this problem, the natural language\nprocessing (NLP) community has experimented with a range of techniques for\nabuse detection. While achieving substantial success, these methods have so far\nonly focused on modelling the linguistic properties of the comments and the\nonline communities of users, disregarding the emotional state of the users and\nhow this might affect their language. The latter is, however, inextricably\nlinked to abusive behaviour. In this paper, we present the first joint model of\nemotion and abusive language detection, experimenting in a multi-task learning\nframework that allows one task to inform the other. Our results demonstrate\nthat incorporating affective features leads to significant improvements in\nabuse detection performance across datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14050v2",
            "date": "2020-05-28 14:32:08+00:00",
            "title": "Language (Technology) is Power: A Critical Survey of \"Bias\" in NLP",
            "authors": [
                "Su Lin Blodgett",
                "Solon Barocas",
                "Hal Daum\u00e9 III",
                "Hanna Wallach"
            ],
            "abstract": "We survey 146 papers analyzing \"bias\" in NLP systems, finding that their\nmotivations are often vague, inconsistent, and lacking in normative reasoning,\ndespite the fact that analyzing \"bias\" is an inherently normative process. We\nfurther find that these papers' proposed quantitative techniques for measuring\nor mitigating \"bias\" are poorly matched to their motivations and do not engage\nwith the relevant literature outside of NLP. Based on these findings, we\ndescribe the beginnings of a path forward by proposing three recommendations\nthat should guide work analyzing \"bias\" in NLP systems. These recommendations\nrest on a greater recognition of the relationships between language and social\nhierarchies, encouraging researchers and practitioners to articulate their\nconceptualizations of \"bias\"---i.e., what kinds of system behaviors are\nharmful, in what ways, to whom, and why, as well as the normative reasoning\nunderlying these statements---and to center work around the lived experiences\nof members of communities affected by NLP systems, while interrogating and\nreimagining the power relations between technologists and such communities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14134v2",
            "date": "2020-05-28 16:31:59+00:00",
            "title": "Cats climb entails mammals move: preserving hyponymy in compositional distributional semantics",
            "authors": [
                "Gemma De las Cuevas",
                "Andreas Klingler",
                "Martha Lewis",
                "Tim Netzer"
            ],
            "abstract": "To give vector-based representations of meaning more structure, one approach\nis to use positive semidefinite (psd) matrices. These allow us to model\nsimilarity of words as well as the hyponymy or is-a relationship. Psd matrices\ncan be learnt relatively easily in a given vector space $M\\otimes M^*$, but to\ncompose words to form phrases and sentences, we need representations in larger\nspaces. In this paper, we introduce a generic way of composing the psd matrices\ncorresponding to words. We propose that psd matrices for verbs, adjectives, and\nother functional words be lifted to completely positive (CP) maps that match\ntheir grammatical type. This lifting is carried out by our composition rule\ncalled Compression, Compr. In contrast to previous composition rules like Fuzz\nand Phaser (a.k.a. KMult and BMult), Compr preserves hyponymy. Mathematically,\nCompr is itself a CP map, and is therefore linear and generally\nnon-commutative. We give a number of proposals for the structure of Compr,\nbased on spiders, cups and caps, and generate a range of composition rules. We\ntest these rules on a small sentence entailment dataset, and see some\nimprovements over the performance of Fuzz and Phaser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14253v1",
            "date": "2020-05-28 19:32:52+00:00",
            "title": "Empirical Evaluation of Pretraining Strategies for Supervised Entity Linking",
            "authors": [
                "Thibault F\u00e9vry",
                "Nicholas FitzGerald",
                "Livio Baldini Soares",
                "Tom Kwiatkowski"
            ],
            "abstract": "In this work, we present an entity linking model which combines a Transformer\narchitecture with large scale pretraining from Wikipedia links. Our model\nachieves the state-of-the-art on two commonly used entity linking datasets:\n96.7% on CoNLL and 94.9% on TAC-KBP. We present detailed analyses to understand\nwhat design choices are important for entity linking, including choices of\nnegative entity candidates, Transformer architecture, and input perturbations.\nLastly, we present promising results on more challenging settings such as\nend-to-end entity linking and entity linking without in-domain training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14299v1",
            "date": "2020-05-28 21:17:43+00:00",
            "title": "What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP",
            "authors": [
                "Oskar Wysocki",
                "Malina Florea",
                "Andre Freitas"
            ],
            "abstract": "SemEval is the primary venue in the NLP community for the proposal of new\nchallenges and for the systematic empirical evaluation of NLP systems. This\npaper provides a systematic quantitative analysis of SemEval aiming to evidence\nthe patterns of the contributions behind SemEval. By understanding the\ndistribution of task types, metrics, architectures, participation and citations\nover time we aim to answer the question on what is being evaluated by SemEval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14327v2",
            "date": "2020-05-28 22:30:57+00:00",
            "title": "On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition",
            "authors": [
                "Jinyu Li",
                "Yu Wu",
                "Yashesh Gaur",
                "Chengyi Wang",
                "Rui Zhao",
                "Shujie Liu"
            ],
            "abstract": "Recently, there has been a strong push to transition from hybrid models to\nend-to-end (E2E) models for automatic speech recognition. Currently, there are\nthree promising E2E methods: recurrent neural network transducer (RNN-T), RNN\nattention-based encoder-decoder (AED), and Transformer-AED. In this study, we\nconduct an empirical comparison of RNN-T, RNN-AED, and Transformer-AED models,\nin both non-streaming and streaming modes. We use 65 thousand hours of\nMicrosoft anonymized training data to train these models. As E2E models are\nmore data hungry, it is better to compare their effectiveness with large amount\nof training data. To the best of our knowledge, no such comprehensive study has\nbeen conducted yet. We show that although AED models are stronger than RNN-T in\nthe non-streaming mode, RNN-T is very competitive in streaming mode if its\nencoder can be properly initialized. Among all three E2E models,\ntransformer-AED achieved the best accuracy in both streaming and non-streaming\nmode. We show that both streaming RNN-T and transformer-AED models can obtain\nbetter accuracy than a highly-optimized hybrid model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.14386v1",
            "date": "2020-05-29 05:03:15+00:00",
            "title": "Controlling Length in Image Captioning",
            "authors": [
                "Ruotian Luo",
                "Greg Shakhnarovich"
            ],
            "abstract": "We develop and evaluate captioning models that allow control of caption\nlength. Our models can leverage this control to generate captions of different\nstyle and descriptiveness.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2005.14576v1",
            "date": "2020-05-29 13:56:31+00:00",
            "title": "Harbsafe-162. A Domain-Specific Data Set for the Intrinsic Evaluation of Semantic Representations for Terminological Data",
            "authors": [
                "Susanne Arndt",
                "Dieter Schn\u00e4pp"
            ],
            "abstract": "The article presents Harbsafe-162, a domain-specific data set for evaluating\ndistributional semantic models. It originates from a cooperation by Technische\nUniversit\\\"at Braunschweig and the German Commission for Electrical, Electronic\n& Information Technologies of DIN and VDE, the Harbsafe project. One objective\nof the project is to apply distributional semantic models to terminological\nentries, that is, complex lexical data comprising of at least one or several\nterms, term phrases and a definition. This application is needed to solve a\nmore complex problem: the harmonization of terminologies of standards and\nstandards bodies (i.e. resolution of doublettes and inconsistencies). Due to a\nlack of evaluation data sets for terminological entries, the creation of\nHarbsafe-162 was a necessary step towards harmonization assistance.\nHarbsafe-162 covers data from nine electrotechnical standards in the domain of\nfunctional safety, IT security, and dependability. An intrinsic evaluation\nmethod in the form of a similarity rating task has been applied in which two\nlinguists and three domain experts from standardization participated. The data\nset is used to evaluate a specific implementation of an established sentence\nembedding model. This implementation proves to be satisfactory for the\ndomain-specific data so that further implementations for harmonization\nassistance may be brought forward by the project. Considering recent criticism\non intrinsic evaluation methods, the article concludes with an evaluation of\nHarbsafe-162 and joins a more general discussion about the nature of similarity\nrating tasks. Harbsafe-162 has been made available for the community.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2005.14578v1",
            "date": "2020-05-29 13:58:36+00:00",
            "title": "Improving Unsupervised Sparsespeech Acoustic Models with Categorical Reparameterization",
            "authors": [
                "Benjamin Milde",
                "Chris Biemann"
            ],
            "abstract": "The Sparsespeech model is an unsupervised acoustic model that can generate\ndiscrete pseudo-labels for untranscribed speech. We extend the Sparsespeech\nmodel to allow for sampling over a random discrete variable, yielding\npseudo-posteriorgrams. The degree of sparsity in this posteriorgram can be\nfully controlled after the model has been trained. We use the Gumbel-Softmax\ntrick to approximately sample from a discrete distribution in the neural\nnetwork and this allows us to train the network efficiently with standard\nbackpropagation. The new and improved model is trained and evaluated on the\nLibri-Light corpus, a benchmark for ASR with limited or no supervision. The\nmodel is trained on 600h and 6000h of English read speech. We evaluate the\nimproved model using the ABX error measure and a semi-supervised setting with\n10h of transcribed speech. We observe a relative improvement of up to 31.4% on\nABX error rates across speakers on the test set with the improved Sparsespeech\nmodel on 600h of speech data and further improvements when we scale the model\nto 6000h.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2005.14613v1",
            "date": "2020-05-29 14:52:33+00:00",
            "title": "Using Large Pretrained Language Models for Answering User Queries from Product Specifications",
            "authors": [
                "Kalyani Roy",
                "Smit Shah",
                "Nithish Pai",
                "Jaidam Ramtej",
                "Prajit Prashant Nadkarn",
                "Jyotirmoy Banerjee",
                "Pawan Goyal",
                "Surender Kumar"
            ],
            "abstract": "While buying a product from the e-commerce websites, customers generally have\na plethora of questions. From the perspective of both the e-commerce service\nprovider as well as the customers, there must be an effective question\nanswering system to provide immediate answers to the user queries. While\ncertain questions can only be answered after using the product, there are many\nquestions which can be answered from the product specification itself. Our work\ntakes a first step in this direction by finding out the relevant product\nspecifications, that can help answering the user questions. We propose an\napproach to automatically create a training dataset for this problem. We\nutilize recently proposed XLNet and BERT architectures for this problem and\nfind that they provide much better performance than the Siamese model,\npreviously applied for this problem. Our model gives a good performance even\nwhen trained on one vertical and tested across different verticals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2005.14662v1",
            "date": "2020-05-29 16:40:27+00:00",
            "title": "SLAM-Inspired Simultaneous Contextualization and Interpreting for Incremental Conversation Sentences",
            "authors": [
                "Yusuke Takimoto",
                "Yosuke Fukuchi",
                "Shoya Matsumori",
                "Michita Imai"
            ],
            "abstract": "Distributed representation of words has improved the performance for many\nnatural language tasks. In many methods, however, only one meaning is\nconsidered for one label of a word, and multiple meanings of polysemous words\ndepending on the context are rarely handled. Although research works have dealt\nwith polysemous words, they determine the meanings of such words according to a\nbatch of large documents. Hence, there are two problems with applying these\nmethods to sequential sentences, as in a conversation that contains ambiguous\nexpressions. The first problem is that the methods cannot sequentially deal\nwith the interdependence between context and word interpretation, in which\ncontext is decided by word interpretations and the word interpretations are\ndecided by the context. Context estimation must thus be performed in parallel\nto pursue multiple interpretations. The second problem is that the previous\nmethods use large-scale sets of sentences for offline learning of new\ninterpretations, and the steps of learning and inference are clearly separated.\nSuch methods using offline learning cannot obtain new interpretations during a\nconversation. Hence, to dynamically estimate the conversation context and\ninterpretations of polysemous words in sequential sentences, we propose a\nmethod of Simultaneous Contextualization And INterpreting (SCAIN) based on the\ntraditional Simultaneous Localization And Mapping (SLAM) algorithm. By using\nthe SCAIN algorithm, we can sequentially optimize the interdependence between\ncontext and word interpretation while obtaining new interpretations online. For\nexperimental evaluation, we created two datasets: one from Wikipedia's\ndisambiguation pages and the other from real conversations. For both datasets,\nthe results confirmed that SCAIN could effectively achieve sequential\noptimization of the interdependence and acquisition of new interpretations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00052v1",
            "date": "2020-05-29 19:54:07+00:00",
            "title": "Stance Prediction for Contemporary Issues: Data and Experiments",
            "authors": [
                "Marjan Hosseinia",
                "Eduard Dragut",
                "Arjun Mukherjee"
            ],
            "abstract": "We investigate whether pre-trained bidirectional transformers with sentiment\nand emotion information improve stance detection in long discussions of\ncontemporary issues. As a part of this work, we create a novel stance detection\ndataset covering 419 different controversial issues and their related pros and\ncons collected by procon.org in nonpartisan format. Experimental results show\nthat a shallow recurrent neural network with sentiment or emotion information\ncan reach competitive results compared to fine-tuned BERT with 20x fewer\nparameters. We also use a simple approach that explains which input phrases\ncontribute to stance detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00075v1",
            "date": "2020-05-16 04:11:44+00:00",
            "title": "iCapsNets: Towards Interpretable Capsule Networks for Text Classification",
            "authors": [
                "Zhengyang Wang",
                "Xia Hu",
                "Shuiwang Ji"
            ],
            "abstract": "Many text classification applications require models with satisfying\nperformance as well as good interpretability. Traditional machine learning\nmethods are easy to interpret but have low accuracies. The development of deep\nlearning models boosts the performance significantly. However, deep learning\nmodels are typically hard to interpret. In this work, we propose interpretable\ncapsule networks (iCapsNets) to bridge this gap. iCapsNets use capsules to\nmodel semantic meanings and explore novel methods to increase interpretability.\nThe design of iCapsNets is consistent with human intuition and enables it to\nproduce human-understandable interpretation results. Notably, iCapsNets can be\ninterpreted both locally and globally. In terms of local interpretability,\niCapsNets offer a simple yet effective method to explain the predictions for\neach data sample. On the other hand, iCapsNets explore a novel way to explain\nthe model's general behavior, achieving global interpretability. Experimental\nstudies show that our iCapsNets yield meaningful local and global\ninterpretation results, without suffering from significant performance loss\ncompared to non-interpretable methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00113v1",
            "date": "2020-05-29 22:56:25+00:00",
            "title": "A frame semantics based approach to comparative study of digitized corpus",
            "authors": [
                "Abdelaziz Lakhfif",
                "Mohamed Tayeb Laskri"
            ],
            "abstract": "in this paper, we present a corpus linguistics based approach applied to\nanalyzing digitized classical multilingual novels and narrative texts, from a\nsemantic point of view. Digitized novels such as \"the hobbit (Tolkien J. R. R.,\n1937)\" and \"the hound of the Baskervilles (Doyle A. C. 1901-1902)\", which were\nwidely translated to dozens of languages, provide rich materials for analyzing\nlanguages differences from several perspectives and within a number of\ndisciplines like linguistics, philosophy and cognitive science. Taking motion\nevents conceptualization as a case study, this paper, focus on the morphologic,\nsyntactic, and semantic annotation process of English-Arabic aligned corpus\ncreated from a digitized novels, in order to re-examine the linguistic\nencodings of motion events in English and Arabic in terms of Frame Semantics.\nThe present study argues that differences in motion events conceptualization\nacross languages can be described with frame structure and frame-to-frame\nrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00377v1",
            "date": "2020-05-30 22:14:46+00:00",
            "title": "Linguistic Features for Readability Assessment",
            "authors": [
                "Tovly Deutsch",
                "Masoud Jasbi",
                "Stuart Shieber"
            ],
            "abstract": "Readability assessment aims to automatically classify text by the level\nappropriate for learning readers. Traditional approaches to this task utilize a\nvariety of linguistically motivated features paired with simple machine\nlearning models. More recent methods have improved performance by discarding\nthese features and utilizing deep learning models. However, it is unknown\nwhether augmenting deep learning models with linguistically motivated features\nwould improve performance further. This paper combines these two approaches\nwith the goal of improving overall model performance and addressing this\nquestion. Evaluating on two large readability corpora, we find that, given\nsufficient training data, augmenting deep learning models with linguistically\nmotivated features does not improve state-of-the-art performance. Our results\nprovide preliminary evidence for the hypothesis that the state-of-the-art deep\nlearning models represent linguistic features of the text related to\nreadability. Future research on the nature of representations formed in these\nmodels can shed light on the learned features and their relations to\nlinguistically motivated ones hypothesized in traditional approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00417v1",
            "date": "2020-05-31 02:44:36+00:00",
            "title": "Variational Reward Estimator Bottleneck: Learning Robust Reward Estimator for Multi-Domain Task-Oriented Dialog",
            "authors": [
                "Jeiyoon Park",
                "Chanhee Lee",
                "Kuekyeng Kim",
                "Heuiseok Lim"
            ],
            "abstract": "Despite its notable success in adversarial learning approaches to\nmulti-domain task-oriented dialog system, training the dialog policy via\nadversarial inverse reinforcement learning often fails to balance the\nperformance of the policy generator and reward estimator. During optimization,\nthe reward estimator often overwhelms the policy generator and produces\nexcessively uninformative gradients. We proposes the Variational Reward\nestimator Bottleneck (VRB), which is an effective regularization method that\naims to constrain unproductive information flows between inputs and the reward\nestimator. The VRB focuses on capturing discriminative features, by exploiting\ninformation bottleneck on mutual information. Empirical results on a\nmulti-domain task-oriented dialog dataset demonstrate that the VRB\nsignificantly outperforms previous methods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2006.00459v1",
            "date": "2020-05-31 08:02:23+00:00",
            "title": "SANA : Sentiment Analysis on Newspapers comments in Algeria",
            "authors": [
                "Hichem Rahab",
                "Abdelhafid Zitouni",
                "Mahieddine Djoudi"
            ],
            "abstract": "It is very current in today life to seek for tracking the people opinion from\ntheir interaction with occurring events. A very common way to do that is\ncomments in articles published in newspapers web sites dealing with\ncontemporary events. Sentiment analysis or opinion mining is an emergent field\nwho is the purpose is finding the behind phenomenon masked in opinionated\ntexts. We are interested in our work by comments in Algerian newspaper\nwebsites. For this end, two corpora were used SANA and OCA. SANA corpus is\ncreated by collection of comments from three Algerian newspapers, and annotated\nby two Algerian Arabic native speakers, while OCA is a freely available corpus\nfor sentiment analysis. For the classification we adopt Supports vector\nmachines, naive Bayes and knearest neighbors. Obtained results are very\npromising and show the different effects of stemming in such domain, also\nknearest neighbors give important improvement comparing to other classifiers\nunlike similar works where SVM is the most dominant. From this study we observe\nthe importance of dedicated resources and methods the newspaper comments\nsentiment analysis which we look forward in future works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00464v1",
            "date": "2020-05-31 08:13:00+00:00",
            "title": "Recognizing Chinese Judicial Named Entity using BiLSTM-CRF",
            "authors": [
                "Pin Tang",
                "Pinli Yang",
                "Yuang Shi",
                "Yi Zhou",
                "Feng Lin",
                "Yan Wang"
            ],
            "abstract": "Named entity recognition (NER) plays an essential role in natural language\nprocessing systems. Judicial NER is a fundamental component of judicial\ninformation retrieval, entity relation extraction, and knowledge map building.\nHowever, Chinese judicial NER remains to be more challenging due to the\ncharacteristics of Chinese and high accuracy requirements in the judicial\nfiled. Thus, in this paper, we propose a deep learning-based method named\nBiLSTM-CRF which consists of bi-directional long short-term memory (BiLSTM) and\nconditional random fields (CRF). For further accuracy promotion, we propose to\nuse Adaptive moment estimation (Adam) for optimization of the model. To\nvalidate our method, we perform experiments on judgment documents including\ncommutation, parole and temporary service outside prison, which is acquired\nfrom China Judgments Online. Experimental results achieve the accuracy of\n0.876, recall of 0.856 and F1 score of 0.855, which suggests the superiority of\nthe proposed BiLSTM-CRF with Adam optimizer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00492v2",
            "date": "2020-05-31 11:13:13+00:00",
            "title": "BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis",
            "authors": [
                "Wei Li",
                "Wei Shao",
                "Shaoxiong Ji",
                "Erik Cambria"
            ],
            "abstract": "Sentiment analysis in conversations has gained increasing attention in recent\nyears for the growing amount of applications it can serve, e.g., sentiment\nanalysis, recommender systems, and human-robot interaction. The main difference\nbetween conversational sentiment analysis and single sentence sentiment\nanalysis is the existence of context information which may influence the\nsentiment of an utterance in a dialogue. How to effectively encode contextual\ninformation in dialogues, however, remains a challenge. Existing approaches\nemploy complicated deep learning structures to distinguish different parties in\na conversation and then model the context information. In this paper, we\npropose a fast, compact and parameter-efficient party-ignorant framework named\nbidirectional emotional recurrent unit for conversational sentiment analysis.\nIn our system, a generalized neural tensor block followed by a two-channel\nclassifier is designed to perform context compositionality and sentiment\nclassification, respectively. Extensive experiments on three standard datasets\ndemonstrate that our model outperforms the state of the art in most cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00691v2",
            "date": "2020-06-01 03:30:08+00:00",
            "title": "Stance in Replies and Quotes (SRQ): A New Dataset For Learning Stance in Twitter Conversations",
            "authors": [
                "Ramon Villa-Cox",
                "Sumeet Kumar",
                "Matthew Babcock",
                "Kathleen M. Carley"
            ],
            "abstract": "Automated ways to extract stance (denying vs. supporting opinions) from\nconversations on social media are essential to advance opinion mining research.\nRecently, there is a renewed excitement in the field as we see new models\nattempting to improve the state-of-the-art. However, for training and\nevaluating the models, the datasets used are often small. Additionally, these\nsmall datasets have uneven class distributions, i.e., only a tiny fraction of\nthe examples in the dataset have favoring or denying stances, and most other\nexamples have no clear stance. Moreover, the existing datasets do not\ndistinguish between the different types of conversations on social media (e.g.,\nreplying vs. quoting on Twitter). Because of this, models trained on one event\ndo not generalize to other events.\n  In the presented work, we create a new dataset by labeling stance in\nresponses to posts on Twitter (both replies and quotes) on controversial\nissues. To the best of our knowledge, this is currently the largest\nhuman-labeled stance dataset for Twitter conversations with over 5200 stance\nlabels. More importantly, we designed a tweet collection methodology that\nfavors the selection of denial-type responses. This class is expected to be\nmore useful in the identification of rumors and determining antagonistic\nrelationships between users. Moreover, we include many baseline models for\nlearning the stance in conversations and compare the performance of various\nmodels. We show that combining data from replies and quotes decreases the\naccuracy of models indicating that the two modalities behave differently when\nit comes to stance learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00850v1",
            "date": "2020-06-01 10:52:35+00:00",
            "title": "Sarcasm Detection using Context Separators in Online Discourse",
            "authors": [
                "Kartikey Pant",
                "Tanvi Dadu"
            ],
            "abstract": "Sarcasm is an intricate form of speech, where meaning is conveyed implicitly.\nBeing a convoluted form of expression, detecting sarcasm is an assiduous\nproblem. The difficulty in recognition of sarcasm has many pitfalls, including\nmisunderstandings in everyday communications, which leads us to an increasing\nfocus on automated sarcasm detection. In the second edition of the Figurative\nLanguage Processing (FigLang 2020) workshop, the shared task of sarcasm\ndetection released two datasets, containing responses along with their context\nsampled from Twitter and Reddit.\n  In this work, we use RoBERTa_large to detect sarcasm in both the datasets. We\nfurther assert the importance of context in improving the performance of\ncontextual word embedding based models by using three different types of inputs\n- Response-only, Context-Response, and Context-Response (Separated). We show\nthat our proposed architecture performs competitively for both the datasets. We\nalso show that the addition of a separation token between context and target\nresponse results in an improvement of 5.13% in the F1-score in the Reddit\ndataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.00885v3",
            "date": "2020-05-22 19:08:14+00:00",
            "title": "CoAID: COVID-19 Healthcare Misinformation Dataset",
            "authors": [
                "Limeng Cui",
                "Dongwon Lee"
            ],
            "abstract": "As the COVID-19 virus quickly spreads around the world, unfortunately,\nmisinformation related to COVID-19 also gets created and spreads like wild\nfire. Such misinformation has caused confusion among people, disruptions in\nsociety, and even deadly consequences in health problems. To be able to\nunderstand, detect, and mitigate such COVID-19 misinformation, therefore, has\nnot only deep intellectual values but also huge societal impacts. To help\nresearchers combat COVID-19 health misinformation, therefore, we present CoAID\n(Covid-19 heAlthcare mIsinformation Dataset), with diverse COVID-19 healthcare\nmisinformation, including fake news on websites and social platforms, along\nwith users' social engagement about such news. CoAID includes 4,251 news,\n296,000 related user engagements, 926 social platform posts about COVID-19, and\nground truth labels. The dataset is available at:\nhttps://github.com/cuilimeng/CoAID.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.00988v1",
            "date": "2020-06-01 14:47:48+00:00",
            "title": "Attention Word Embedding",
            "authors": [
                "Shashank Sonkar",
                "Andrew E. Waters",
                "Richard G. Baraniuk"
            ],
            "abstract": "Word embedding models learn semantically rich vector representations of words\nand are widely used to initialize natural processing language (NLP) models. The\npopular continuous bag-of-words (CBOW) model of word2vec learns a vector\nembedding by masking a given word in a sentence and then using the other words\nas a context to predict it. A limitation of CBOW is that it equally weights the\ncontext words when making a prediction, which is inefficient, since some words\nhave higher predictive value than others. We tackle this inefficiency by\nintroducing the Attention Word Embedding (AWE) model, which integrates the\nattention mechanism into the CBOW model. We also propose AWE-S, which\nincorporates subword information. We demonstrate that AWE and AWE-S outperform\nthe state-of-the-art word embedding models both on a variety of word similarity\ndatasets and when used for initialization of NLP models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01067v3",
            "date": "2020-06-01 16:45:38+00:00",
            "title": "Aligning Faithful Interpretations with their Social Attribution",
            "authors": [
                "Alon Jacovi",
                "Yoav Goldberg"
            ],
            "abstract": "We find that the requirement of model interpretations to be faithful is vague\nand incomplete. With interpretation by textual highlights as a case-study, we\npresent several failure cases. Borrowing concepts from social science, we\nidentify that the problem is a misalignment between the causal chain of\ndecisions (causal attribution) and the attribution of human behavior to the\ninterpretation (social attribution). We re-formulate faithfulness as an\naccurate attribution of causality to the model, and introduce the concept of\naligned faithfulness: faithful causal chains that are aligned with their\nexpected social behavior. The two steps of causal attribution and social\nattribution together complete the process of explaining behavior. With this\nformalization, we characterize various failures of misaligned faithful\nhighlight interpretations, and propose an alternative causal chain to remedy\nthe issues. Finally, we implement highlight explanations of the proposed causal\nformat using contrastive explanations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01095v4",
            "date": "2020-06-01 17:23:44+00:00",
            "title": "Emergence of Separable Manifolds in Deep Language Representations",
            "authors": [
                "Jonathan Mamou",
                "Hang Le",
                "Miguel Del Rio",
                "Cory Stephenson",
                "Hanlin Tang",
                "Yoon Kim",
                "SueYeon Chung"
            ],
            "abstract": "Deep neural networks (DNNs) have shown much empirical success in solving\nperceptual tasks across various cognitive modalities. While they are only\nloosely inspired by the biological brain, recent studies report considerable\nsimilarities between representations extracted from task-optimized DNNs and\nneural populations in the brain. DNNs have subsequently become a popular model\nclass to infer computational principles underlying complex cognitive functions,\nand in turn, they have also emerged as a natural testbed for applying methods\noriginally developed to probe information in neural populations. In this work,\nwe utilize mean-field theoretic manifold analysis, a recent technique from\ncomputational neuroscience that connects geometry of feature representations\nwith linear separability of classes, to analyze language representations from\nlarge-scale contextual embedding models. We explore representations from\ndifferent model families (BERT, RoBERTa, GPT, etc.) and find evidence for\nemergence of linguistic manifolds across layer depth (e.g., manifolds for\npart-of-speech tags), especially in ambiguous data (i.e, words with multiple\npart-of-speech tags, or part-of-speech classes including many words). In\naddition, we find that the emergence of linear separability in these manifolds\nis driven by a combined reduction of manifolds' radius, dimensionality and\ninter-manifold correlations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01110v2",
            "date": "2020-06-01 17:50:20+00:00",
            "title": "Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas",
            "authors": [
                "Yen-Ling Kuo",
                "Boris Katz",
                "Andrei Barbu"
            ],
            "abstract": "We demonstrate a reinforcement learning agent which uses a compositional\nrecurrent neural network that takes as input an LTL formula and determines\nsatisfying actions. The input LTL formulas have never been seen before, yet the\nnetwork performs zero-shot generalization to satisfy them. This is a novel form\nof multi-task learning for RL agents where agents learn from one diverse set of\ntasks and generalize to a new set of diverse tasks. The formulation of the\nnetwork enables this capacity to generalize. We demonstrate this ability in two\ndomains. In a symbolic domain, the agent finds a sequence of letters that is\naccepted. In a Minecraft-like environment, the agent finds a sequence of\nactions that conform to the formula. While prior work could learn to execute\none formula reliably given examples of that formula, we demonstrate how to\nencode all formulas reliably. This could form the basis of new multitask agents\nthat discover sub-tasks and execute them without any additional training, as\nwell as the agents which follow more complex linguistic commands. The\nstructures required for this generalization are specific to LTL formulas, which\nopens up an interesting theoretical question: what structures are required in\nneural networks for zero-shot generalization to different logics?",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/2006.01131v1",
            "date": "2020-05-31 17:12:37+00:00",
            "title": "NLP Scholar: An Interactive Visual Explorer for Natural Language Processing Literature",
            "authors": [
                "Saif M. Mohammad"
            ],
            "abstract": "As part of the NLP Scholar project, we created a single unified dataset of\nNLP papers and their meta-information (including citation numbers), by\nextracting and aligning information from the ACL Anthology and Google Scholar.\nIn this paper, we describe several interconnected interactive visualizations\n(dashboards) that present various aspects of the data. Clicking on an item\nwithin a visualization or entering query terms in the search boxes filters the\ndata in all visualizations in the dashboard. This allows users to search for\npapers in the area of their interest, published within specific time periods,\npublished by specified authors, etc. The interactive visualizations presented\nhere, and the associated dataset of papers mapped to citations, have additional\nuses as well including understanding how the field is growing (both overall and\nacross sub-areas), as well as quantifying the impact of different types of\npapers on subsequent publications.",
            "journal ref": "Proceedings of the 58th Annual Meeting of the Association of\n  Computational Linguistics (ACL-2020), July 2020",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01203v1",
            "date": "2020-06-01 19:09:13+00:00",
            "title": "Hybrid Improved Document-level Embedding (HIDE)",
            "authors": [
                "Satanik Mitra",
                "Mamata Jenamani"
            ],
            "abstract": "In recent times, word embeddings are taking a significant role in sentiment\nanalysis. As the generation of word embeddings needs huge corpora, many\napplications use pretrained embeddings. In spite of the success, word\nembeddings suffers from certain drawbacks such as it does not capture sentiment\ninformation of a word, contextual information in terms of parts of speech tags\nand domain-specific information. In this work we propose HIDE a Hybrid Improved\nDocument level Embedding which incorporates domain information, parts of speech\ninformation and sentiment information into existing word embeddings such as\nGloVe and Word2Vec. It combine improved word embeddings into document level\nembeddings. Further, Latent Semantic Analysis (LSA) has been used to represent\ndocuments as a vectors. HIDE is generated, combining LSA and document level\nembeddings, which is computed from improved word embeddings. We test HIDE with\nsix different datasets and shown considerable improvement over the accuracy of\nexisting pretrained word vectors such as GloVe and Word2Vec. We further compare\nour work with two existing document level sentiment analysis approaches. HIDE\nperforms better than existing systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01204v1",
            "date": "2020-05-16 01:55:31+00:00",
            "title": "Automatic Dialogic Instruction Detection for K-12 Online One-on-one Classes",
            "authors": [
                "Shiting Xu",
                "Wenbiao Ding",
                "Zitao Liu"
            ],
            "abstract": "Online one-on-one class is created for highly interactive and immersive\nlearning experience. It demands a large number of qualified online instructors.\nIn this work, we develop six dialogic instructions and help teachers achieve\nthe benefits of one-on-one learning paradigm. Moreover, we utilize neural\nlanguage models, i.e., long short-term memory (LSTM), to detect above six\ninstructions automatically. Experiments demonstrate that the LSTM approach\nachieves AUC scores from 0.840 to 0.979 among all six types of instructions on\nour real-world educational dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01209v1",
            "date": "2020-05-23 18:31:30+00:00",
            "title": "Learning Constraints for Structured Prediction Using Rectifier Networks",
            "authors": [
                "Xingyuan Pan",
                "Maitrey Mehta",
                "Vivek Srikumar"
            ],
            "abstract": "Various natural language processing tasks are structured prediction problems\nwhere outputs are constructed with multiple interdependent decisions. Past work\nhas shown that domain knowledge, framed as constraints over the output space,\ncan help improve predictive accuracy. However, designing good constraints often\nrelies on domain expertise. In this paper, we study the problem of learning\nsuch constraints. We frame the problem as that of training a two-layer\nrectifier network to identify valid structures or substructures, and show a\nconstruction for converting a trained network into a system of linear\nconstraints over the inference variables. Our experiments on several NLP tasks\nshow that the learned constraints can improve the prediction accuracy,\nespecially when the number of training examples is small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01222v1",
            "date": "2020-06-01 19:52:01+00:00",
            "title": "BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text",
            "authors": [
                "Tanvi Dadu",
                "Kartikey Pant",
                "Radhika Mamidi"
            ],
            "abstract": "There is a growing interest in understanding how humans initiate and hold\nconversations. The affective understanding of conversations focuses on the\nproblem of how speakers use emotions to react to a situation and to each other.\nIn the CL-Aff Shared Task, the organizers released Get it #OffMyChest dataset,\nwhich contains Reddit comments from casual and confessional conversations,\nlabeled for their disclosure and supportiveness characteristics. In this paper,\nwe introduce a predictive ensemble model exploiting the finetuned\ncontextualized word embeddings, RoBERTa and ALBERT. We show that our model\noutperforms the base models in all considered metrics, achieving an improvement\nof $3\\%$ in the F1 score. We further conduct statistical analysis and outline\ndeeper insights into the given dataset while providing a new characterization\nof impact for the dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01245v1",
            "date": "2020-06-01 20:35:46+00:00",
            "title": "An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results",
            "authors": [
                "Enrique Amig\u00f3",
                "Julio Gonzalo",
                "Stefano Mizzaro",
                "Jorge Carrillo-de-Albornoz"
            ],
            "abstract": "In Ordinal Classification tasks, items have to be assigned to classes that\nhave a relative ordering, such as positive, neutral, negative in sentiment\nanalysis. Remarkably, the most popular evaluation metrics for ordinal\nclassification tasks either ignore relevant information (for instance,\nprecision/recall on each of the classes ignores their relative ordering) or\nassume additional information (for instance, Mean Average Error assumes\nabsolute distances between classes). In this paper we propose a new metric for\nOrdinal Classification, Closeness Evaluation Measure, that is rooted on\nMeasurement Theory and Information Theory. Our theoretical analysis and\nexperimental results over both synthetic data and data from NLP shared tasks\nindicate that the proposed metric captures quality aspects from different\ntraditional tasks simultaneously. In addition, it generalizes some popular\nclassification (nominal scale) and error minimization (interval scale) metrics,\ndepending on the measurement scale in which it is instantiated.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01285v1",
            "date": "2020-06-01 21:52:19+00:00",
            "title": "Context-based Transformer Models for Answer Sentence Selection",
            "authors": [
                "Ivano Lauriola",
                "Alessandro Moschitti"
            ],
            "abstract": "An important task for the design of Question Answering systems is the\nselection of the sentence containing (or constituting) the answer from\ndocuments relevant to the asked question. Most previous work has only used the\ntarget sentence to compute its score with the question as the models were not\npowerful enough to also effectively encode additional contextual information.\nIn this paper, we analyze the role of the contextual information in the\nsentence selection task, proposing a Transformer based architecture that\nleverages two types of contexts, local and global. The former describes the\nparagraph containing the sentence, aiming at solving implicit references,\nwhereas the latter describes the entire document containing the candidate\nsentence, providing content-based information. The results on three different\nbenchmarks show that the combination of local and global contexts in a\nTransformer model significantly improves the accuracy in Answer Sentence\nSelection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01346v1",
            "date": "2020-06-02 02:12:19+00:00",
            "title": "A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension",
            "authors": [
                "Jie Cai",
                "Zhengzhou Zhu",
                "Ping Nie",
                "Qian Liu"
            ],
            "abstract": "Pre-trained models have brought significant improvements to many NLP tasks\nand have been extensively analyzed. But little is known about the effect of\nfine-tuning on specific tasks. Intuitively, people may agree that a pre-trained\nmodel already learns semantic representations of words (e.g. synonyms are\ncloser to each other) and fine-tuning further improves its capabilities which\nrequire more complicated reasoning (e.g. coreference resolution, entity\nboundary detection, etc). However, how to verify these arguments analytically\nand quantitatively is a challenging task and there are few works focus on this\ntopic. In this paper, inspired by the observation that most probing tasks\ninvolve identifying matched pairs of phrases (e.g. coreference requires\nmatching an entity and a pronoun), we propose a pairwise probe to understand\nBERT fine-tuning on the machine reading comprehension (MRC) task. Specifically,\nwe identify five phenomena in MRC. According to pairwise probing tasks, we\ncompare the performance of each layer's hidden representation of pre-trained\nand fine-tuned BERT. The proposed pairwise probe alleviates the problem of\ndistraction from inaccurate model training and makes a robust and quantitative\ncomparison. Our experimental analysis leads to highly confident conclusions:\n(1) Fine-tuning has little effect on the fundamental and low-level information\nand general semantic tasks. (2) For specific abilities required for downstream\ntasks, fine-tuned BERT is better than pre-trained BERT and such gaps are\nobvious after the fifth layer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01414v3",
            "date": "2020-06-02 06:42:22+00:00",
            "title": "Enhanced Universal Dependency Parsing with Second-Order Inference and Mixture of Training Data",
            "authors": [
                "Xinyu Wang",
                "Yong Jiang",
                "Kewei Tu"
            ],
            "abstract": "This paper presents the system used in our submission to the \\textit{IWPT\n2020 Shared Task}. Our system is a graph-based parser with second-order\ninference. For the low-resource Tamil corpus, we specially mixed the training\ndata of Tamil with other languages and significantly improved the performance\nof Tamil. Due to our misunderstanding of the submission requirements, we\nsubmitted graphs that are not connected, which makes our system only rank\n\\textbf{6th} over 10 teams. However, after we fixed this problem, our system is\n0.6 ELAS higher than the team that ranked \\textbf{1st} in the official results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01538v1",
            "date": "2020-06-02 11:57:53+00:00",
            "title": "WikiBERT models: deep transfer learning for many languages",
            "authors": [
                "Sampo Pyysalo",
                "Jenna Kanerva",
                "Antti Virtanen",
                "Filip Ginter"
            ],
            "abstract": "Deep neural language models such as BERT have enabled substantial recent\nadvances in many natural language processing tasks. Due to the effort and\ncomputational cost involved in their pre-training, language-specific models are\ntypically introduced only for a small number of high-resource languages such as\nEnglish. While multilingual models covering large numbers of languages are\navailable, recent work suggests monolingual training can produce better models,\nand our understanding of the tradeoffs between mono- and multilingual training\nis incomplete. In this paper, we introduce a simple, fully automated pipeline\nfor creating language-specific BERT models from Wikipedia data and introduce 42\nnew such models, most for languages up to now lacking dedicated deep neural\nlanguage models. We assess the merits of these models using the\nstate-of-the-art UDify parser on Universal Dependencies data, contrasting\nperformance with results using the multilingual BERT model. We find that UDify\nusing WikiBERT models outperforms the parser using mBERT on average, with the\nlanguage-specific models showing substantially improved performance for some\nlanguages, yet limited improvement or a decrease in performance for others. We\nalso present preliminary results as first steps toward an understanding of the\nconditions under which language-specific models are most beneficial. All of the\nmethods and models introduced in this work are available under open licenses\nfrom https://github.com/turkunlp/wikibert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01854v1",
            "date": "2020-06-02 18:05:34+00:00",
            "title": "Event Arguments Extraction via Dilate Gated Convolutional Neural Network with Enhanced Local Features",
            "authors": [
                "Zhigang Kan",
                "Linbo Qiao",
                "Sen Yang",
                "Feng Liu",
                "Feng Huang"
            ],
            "abstract": "Event Extraction plays an important role in information-extraction to\nunderstand the world. Event extraction could be split into two subtasks: one is\nevent trigger extraction, the other is event arguments extraction. However, the\nF-Score of event arguments extraction is much lower than that of event trigger\nextraction, i.e. in the most recent work, event trigger extraction achieves\n80.7%, while event arguments extraction achieves only 58%. In pipelined\nstructures, the difficulty of event arguments extraction lies in its lack of\nclassification feature, and the much higher computation consumption. In this\nwork, we proposed a novel Event Extraction approach based on multi-layer Dilate\nGated Convolutional Neural Network (EE-DGCNN) which has fewer parameters. In\naddition, enhanced local information is incorporated into word features, to\nassign event arguments roles for triggers predicted by the first subtask. The\nnumerical experiments demonstrated significant performance improvement beyond\nstate-of-art event extraction approaches on real-world datasets. Further\nanalysis of extraction procedure is presented, as well as experiments are\nconducted to analyze impact factors related to the performance improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01938v1",
            "date": "2020-06-02 20:50:43+00:00",
            "title": "Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings",
            "authors": [
                "Vaibhav Kumar",
                "Tenzin Singhay Bhotia",
                "Vaibhav Kumar",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Word embeddings are the standard model for semantic and syntactic\nrepresentations of words. Unfortunately, these models have been shown to\nexhibit undesirable word associations resulting from gender, racial, and\nreligious biases. Existing post-processing methods for debiasing word\nembeddings are unable to mitigate gender bias hidden in the spatial arrangement\nof word vectors. In this paper, we propose RAN-Debias, a novel gender debiasing\nmethodology which not only eliminates the bias present in a word vector but\nalso alters the spatial distribution of its neighbouring vectors, achieving a\nbias-free setting while maintaining minimal semantic offset. We also propose a\nnew bias evaluation metric - Gender-based Illicit Proximity Estimate (GIPE),\nwhich measures the extent of undue proximity in word vectors resulting from the\npresence of gender-based predilections. Experiments based on a suite of\nevaluation metrics show that RAN-Debias significantly outperforms the\nstate-of-the-art in reducing proximity bias (GIPE) by at least 42.02%. It also\nreduces direct bias, adding minimal semantic disturbance, and achieves the best\nperformance in a downstream application task (coreference resolution).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.01969v1",
            "date": "2020-06-02 22:51:17+00:00",
            "title": "REL: An Entity Linker Standing on the Shoulders of Giants",
            "authors": [
                "Johannes M. van Hulst",
                "Faegheh Hasibi",
                "Koen Dercksen",
                "Krisztian Balog",
                "Arjen P. de Vries"
            ],
            "abstract": "Entity linking is a standard component in modern retrieval system that is\noften performed by third-party toolkits. Despite the plethora of open source\noptions, it is difficult to find a single system that has a modular\narchitecture where certain components may be replaced, does not depend on\nexternal sources, can easily be updated to newer Wikipedia versions, and, most\nimportant of all, has state-of-the-art performance. The REL system presented in\nthis paper aims to fill that gap. Building on state-of-the-art neural\ncomponents from natural language processing research, it is provided as a\nPython package as well as a web API. We also report on an experimental\ncomparison against both well-established systems and the current\nstate-of-the-art on standard entity linking benchmarks.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.01997v1",
            "date": "2020-06-03 00:54:44+00:00",
            "title": "Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2",
            "authors": [
                "Virapat Kieuvongngam",
                "Bowen Tan",
                "Yiming Niu"
            ],
            "abstract": "With the COVID-19 pandemic, there is a growing urgency for medical community\nto keep up with the accelerating growth in the new coronavirus-related\nliterature. As a result, the COVID-19 Open Research Dataset Challenge has\nreleased a corpus of scholarly articles and is calling for machine learning\napproaches to help bridging the gap between the researchers and the rapidly\ngrowing publications. Here, we take advantage of the recent advances in\npre-trained NLP models, BERT and OpenAI GPT-2, to solve this challenge by\nperforming text summarization on this dataset. We evaluate the results using\nROUGE scores and visual inspection. Our model provides abstractive and\ncomprehensive information based on keywords extracted from the original\narticles. Our work can help the the medical community, by providing succinct\nsummaries of articles for which the abstract are not already available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02014v1",
            "date": "2020-06-03 02:22:00+00:00",
            "title": "Norm-Based Curriculum Learning for Neural Machine Translation",
            "authors": [
                "Xuebo Liu",
                "Houtim Lai",
                "Derek F. Wong",
                "Lidia S. Chao"
            ],
            "abstract": "A neural machine translation (NMT) system is expensive to train, especially\nwith high-resource settings. As the NMT architectures become deeper and wider,\nthis issue gets worse and worse. In this paper, we aim to improve the\nefficiency of training an NMT by introducing a novel norm-based curriculum\nlearning method. We use the norm (aka length or module) of a word embedding as\na measure of 1) the difficulty of the sentence, 2) the competence of the model,\nand 3) the weight of the sentence. The norm-based sentence difficulty takes the\nadvantages of both linguistically motivated and model-based sentence\ndifficulties. It is easy to determine and contains learning-dependent features.\nThe norm-based model competence makes NMT learn the curriculum in a fully\nautomated way, while the norm-based sentence weight further enhances the\nlearning of the vector representation of the NMT. Experimental results for the\nWMT'14 English-German and WMT'17 Chinese-English translation tasks demonstrate\nthat the proposed method outperforms strong baselines in terms of BLEU score\n(+1.17/+1.56) and training speedup (2.22x/3.33x).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02104v2",
            "date": "2020-06-03 08:53:40+00:00",
            "title": "Exploiting Class Labels to Boost Performance on Embedding-based Text Classification",
            "authors": [
                "Arkaitz Zubiaga"
            ],
            "abstract": "Text classification is one of the most frequent tasks for processing textual\ndata, facilitating among others research from large-scale datasets. Embeddings\nof different kinds have recently become the de facto standard as features used\nfor text classification. These embeddings have the capacity to capture meanings\nof words inferred from occurrences in large external collections. While they\nare built out of external collections, they are unaware of the distributional\ncharacteristics of words in the classification dataset at hand, including most\nimportantly the distribution of words across classes in training data. To make\nthe most of these embeddings as features and to boost the performance of\nclassifiers using them, we introduce a weighting scheme, Term\nFrequency-Category Ratio (TF-CR), which can weight high-frequency,\ncategory-exclusive words higher when computing word embeddings. Our experiments\non eight datasets show the effectiveness of TF-CR, leading to improved\nperformance scores over the well-known weighting schemes TF-IDF and KLD as well\nas over the absence of a weighting scheme in most cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02163v4",
            "date": "2020-06-03 10:57:21+00:00",
            "title": "Cross-model Back-translated Distillation for Unsupervised Machine Translation",
            "authors": [
                "Xuan-Phi Nguyen",
                "Shafiq Joty",
                "Thanh-Tung Nguyen",
                "Wu Kui",
                "Ai Ti Aw"
            ],
            "abstract": "Recent unsupervised machine translation (UMT) systems usually employ three\nmain principles: initialization, language modeling and iterative\nback-translation, though they may apply them differently. Crucially, iterative\nback-translation and denoising auto-encoding for language modeling provide data\ndiversity to train the UMT systems. However, the gains from these\ndiversification processes has seemed to plateau. We introduce a novel component\nto the standard UMT framework called Cross-model Back-translated Distillation\n(CBD), that is aimed to induce another level of data diversification that\nexisting principles lack. CBD is applicable to all previous UMT approaches. In\nour experiments, CBD achieves the state of the art in the WMT'14\nEnglish-French, WMT'16 English-German and English-Romanian bilingual\nunsupervised translation tasks, with 38.2, 30.1, and 36.3 BLEU respectively. It\nalso yields 1.5-3.3 BLEU improvements in IWSLT English-French and\nEnglish-German tasks. Through extensive experimental analyses, we show that CBD\nis effective because it embraces data diversity while other similar variants do\nnot.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02419v2",
            "date": "2020-06-03 17:50:16+00:00",
            "title": "Emergent Multi-Agent Communication in the Deep Learning Era",
            "authors": [
                "Angeliki Lazaridou",
                "Marco Baroni"
            ],
            "abstract": "The ability to cooperate through language is a defining feature of humans. As\nthe perceptual, motory and planning capabilities of deep artificial networks\nincrease, researchers are studying whether they also can develop a shared\nlanguage to interact. From a scientific perspective, understanding the\nconditions under which language evolves in communities of deep agents and its\nemergent features can shed light on human language evolution. From an applied\nperspective, endowing deep networks with the ability to solve problems\ninteractively by communicating with each other and with us should make them\nmore flexible and useful in everyday life.\n  This article surveys representative recent language emergence studies from\n  both of these two angles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02567v2",
            "date": "2020-06-03 22:39:24+00:00",
            "title": "Extracting COVID-19 Events from Twitter",
            "authors": [
                "Shi Zong",
                "Ashutosh Baheti",
                "Wei Xu",
                "Alan Ritter"
            ],
            "abstract": "We present a corpus of 7,500 tweets annotated with COVID-19 events, including\npositive test results, denied access to testing, and more. We show that our\ncorpus enables automatic identification of COVID-19 events mentioned in Twitter\nwith text spans that fill a set of pre-defined slots for each event. We also\npresent analyses on the self-reporting cases and user's demographic\ninformation. We will make our annotated corpus and extraction tools available\nfor the research community to use upon publication at\nhttps://github.com/viczong/extract_COVID19_events_from_Twitter",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02588v1",
            "date": "2020-06-03 23:53:06+00:00",
            "title": "Meta Dialogue Policy Learning",
            "authors": [
                "Yumo Xu",
                "Chenguang Zhu",
                "Baolin Peng",
                "Michael Zeng"
            ],
            "abstract": "Dialog policy determines the next-step actions for agents and hence is\ncentral to a dialogue system. However, when migrated to novel domains with\nlittle data, a policy model can fail to adapt due to insufficient interactions\nwith the new environment. We propose Deep Transferable Q-Network (DTQN) to\nutilize shareable low-level signals between domains, such as dialogue acts and\nslots. We decompose the state and action representation space into feature\nsubspaces corresponding to these low-level components to facilitate\ncross-domain knowledge transfer. Furthermore, we embed DTQN in a meta-learning\nframework and introduce Meta-DTQN with a dual-replay mechanism to enable\neffective off-policy training and adaptation. In experiments, our model\noutperforms baseline models in terms of both success rate and dialogue\nefficiency on the multi-domain dialogue dataset MultiWOZ 2.0.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02633v1",
            "date": "2020-06-04 03:52:59+00:00",
            "title": "Stopwords in Technical Language Processing",
            "authors": [
                "Serhad Sarica",
                "Jianxi Luo"
            ],
            "abstract": "There are increasingly applications of natural language processing techniques\nfor information retrieval, indexing and topic modelling in the engineering\ncontexts. A standard component of such tasks is the removal of stopwords, which\nare uninformative components of the data. While researchers use readily\navailable stopword lists which are derived for general English language, the\ntechnical jargon of engineering fields contains their own highly frequent and\nuninformative words and there exists no standard stopword list for technical\nlanguage processing applications. Here we address this gap by rigorously\nidentifying generic, insignificant, uninformative stopwords in engineering\ntexts beyond the stopwords in general texts, based on the synthesis of\nalternative data-driven approaches, and curating a stopword list ready for\ntechnical language processing applications.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.02635v4",
            "date": "2020-06-04 03:54:29+00:00",
            "title": "M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training",
            "authors": [
                "Minheng Ni",
                "Haoyang Huang",
                "Lin Su",
                "Edward Cui",
                "Taroon Bharti",
                "Lijuan Wang",
                "Jianfeng Gao",
                "Dongdong Zhang",
                "Nan Duan"
            ],
            "abstract": "We present M3P, a Multitask Multilingual Multimodal Pre-trained model that\ncombines multilingual pre-training and multimodal pre-training into a unified\nframework via multitask pre-training. Our goal is to learn universal\nrepresentations that can map objects occurred in different modalities or texts\nexpressed in different languages into a common semantic space. In addition, to\nexplicitly encourage fine-grained alignment between images and non-English\nlanguages, we also propose Multimodal Code-switched Training (MCT) to combine\nmonolingual pre-training and multimodal pre-training via a code-switch\nstrategy. Experiments are performed on the multilingual image retrieval task\nacross two benchmark datasets, including MSCOCO and Multi30K. M3P can achieve\ncomparable results for English and new state-of-the-art results for non-English\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02876v3",
            "date": "2020-06-04 14:19:52+00:00",
            "title": "Enhanced back-translation for low resource neural machine translation using self-training",
            "authors": [
                "Idris Abdulmumin",
                "Bashir Shehu Galadanci",
                "Abubakar Isa"
            ],
            "abstract": "Improving neural machine translation (NMT) models using the back-translations\nof the monolingual target data (synthetic parallel data) is currently the\nstate-of-the-art approach for training improved translation systems. The\nquality of the backward system - which is trained on the available parallel\ndata and used for the back-translation - has been shown in many studies to\naffect the performance of the final NMT model. In low resource conditions, the\navailable parallel data is usually not enough to train a backward model that\ncan produce the qualitative synthetic data needed to train a standard\ntranslation model. This work proposes a self-training strategy where the output\nof the backward model is used to improve the model itself through the forward\ntranslation technique. The technique was shown to improve baseline low resource\nIWSLT'14 English-German and IWSLT'15 English-Vietnamese backward translation\nmodels by 11.06 and 1.5 BLEUs respectively. The synthetic data generated by the\nimproved English-German backward model was used to train a forward model which\nout-performed another forward model trained using standard back-translation by\n2.7 BLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.02951v2",
            "date": "2020-06-04 15:33:55+00:00",
            "title": "CiwGAN and fiwGAN: Encoding information in acoustic data to model lexical learning with Generative Adversarial Networks",
            "authors": [
                "Ga\u0161per Begu\u0161"
            ],
            "abstract": "How can deep neural networks encode information that corresponds to words in\nhuman speech into raw acoustic data? This paper proposes two neural network\narchitectures for modeling unsupervised lexical learning from raw acoustic\ninputs, ciwGAN (Categorical InfoWaveGAN) and fiwGAN (Featural InfoWaveGAN),\nthat combine a Deep Convolutional GAN architecture for audio data (WaveGAN;\narXiv:1705.07904) with an information theoretic extension of GAN -- InfoGAN\n(arXiv:1606.03657), and propose a new latent space structure that can model\nfeatural learning simultaneously with a higher level classification and allows\nfor a very low-dimension vector representation of lexical items. Lexical\nlearning is modeled as emergent from an architecture that forces a deep neural\nnetwork to output data such that unique information is retrievable from its\nacoustic outputs. The networks trained on lexical items from TIMIT learn to\nencode unique information corresponding to lexical items in the form of\ncategorical variables in their latent space. By manipulating these variables,\nthe network outputs specific lexical items. The network occasionally outputs\ninnovative lexical items that violate training data, but are linguistically\ninterpretable and highly informative for cognitive modeling and neural network\ninterpretability. Innovative outputs suggest that phonetic and phonological\nrepresentations learned by the network can be productively recombined and\ndirectly paralleled to productivity in human speech: a fiwGAN network trained\non `suit' and `dark' outputs innovative `start', even though it never saw\n`start' or even a [st] sequence in the training data. We also argue that\nsetting latent featural codes to values well beyond training range results in\nalmost categorical generation of prototypical lexical items and reveals\nunderlying values of each latent code.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03022v1",
            "date": "2020-06-04 17:16:22+00:00",
            "title": "Response to LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts",
            "authors": [
                "Hao Wu",
                "Gareth J. F. Jones",
                "Francois Pitie"
            ],
            "abstract": "Live video commenting systems are an emerging feature of online video sites.\nRecently the Chinese video sharing platform Bilibili, has popularised a novel\ncaptioning system where user comments are displayed as streams of moving\nsubtitles overlaid on the video playback screen and broadcast to all viewers in\nreal-time. LiveBot was recently introduced as a novel Automatic Live Video\nCommenting (ALVC) application. This enables the automatic generation of live\nvideo comments from both the existing video stream and existing viewers\ncomments. In seeking to reproduce the baseline results reported in the original\nLivebot paper, we found differences between the reproduced results using the\nproject codebase and the numbers reported in the paper. Further examination of\nthis situation suggests that this may be caused by a number of small issues in\nthe project code, including a non-obvious overlap between the training and test\nsets. In this paper, we study these discrepancies in detail and propose an\nalternative baseline implementation as a reference for other researchers in\nthis field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03039v1",
            "date": "2020-06-04 17:49:34+00:00",
            "title": "The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain",
            "authors": [
                "Annemarie Friedrich",
                "Heike Adel",
                "Federico Tomazic",
                "Johannes Hingerl",
                "Renou Benteau",
                "Anika Maruscyk",
                "Lukas Lange"
            ],
            "abstract": "This paper presents a new challenging information extraction task in the\ndomain of materials science. We develop an annotation scheme for marking\ninformation on experiments related to solid oxide fuel cells in scientific\npublications, such as involved materials and measurement conditions. With this\npaper, we publish our annotation guidelines, as well as our SOFC-Exp corpus\nconsisting of 45 open-access scholarly articles annotated by domain experts. A\ncorpus and an inter-annotator agreement study demonstrate the complexity of the\nsuggested named entity recognition and slot filling tasks as well as high\nannotation quality. We also present strong neural-network based models for a\nvariety of tasks that can be addressed on the basis of our new data set. On all\ntasks, using BERT embeddings leads to large performance gains, but with\nincreasing task complexity, adding a recurrent neural network on top seems\nbeneficial. Our models will serve as competitive baselines in future work, and\nanalysis of their performance highlights difficult cases when modeling the data\nand suggests promising research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03096v1",
            "date": "2020-06-04 18:46:02+00:00",
            "title": "SOLO: A Corpus of Tweets for Examining the State of Being Alone",
            "authors": [
                "Svetlana Kiritchenko",
                "Will E. Hipson",
                "Robert J. Coplan",
                "Saif M. Mohammad"
            ],
            "abstract": "The state of being alone can have a substantial impact on our lives, though\nexperiences with time alone diverge significantly among individuals.\nPsychologists distinguish between the concept of solitude, a positive state of\nvoluntary aloneness, and the concept of loneliness, a negative state of\ndissatisfaction with the quality of one's social interactions. Here, for the\nfirst time, we conduct a large-scale computational analysis to explore how the\nterms associated with the state of being alone are used in online language. We\npresent SOLO (State of Being Alone), a corpus of over 4 million tweets\ncollected with query terms 'solitude', 'lonely', and 'loneliness'. We use SOLO\nto analyze the language and emotions associated with the state of being alone.\nWe show that the term 'solitude' tends to co-occur with more positive,\nhigh-dominance words (e.g., enjoy, bliss) while the terms 'lonely' and\n'loneliness' frequently co-occur with negative, low-dominance words (e.g.,\nscared, depressed), which confirms the conceptual distinctions made in\npsychology. We also show that women are more likely to report on negative\nfeelings of being lonely as compared to men, and there are more teenagers among\nthe tweeters that use the word 'lonely' than among the tweeters that use the\nword 'solitude'.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03189v1",
            "date": "2020-06-05 00:57:52+00:00",
            "title": "Human or Machine: Automating Human Likeliness Evaluation of NLG Texts",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Automatic evaluation of various text quality criteria produced by data-driven\nintelligent methods is very common and useful because it is cheap, fast, and\nusually yields repeatable results. In this paper, we present an attempt to\nautomate the human likeliness evaluation of the output text samples coming from\nnatural language generation methods used to solve several tasks. We propose to\nuse a human likeliness score that shows the percentage of the output samples\nfrom a method that look as if they were written by a human. Instead of having\nhuman participants label or rate those samples, we completely automate the\nprocess by using a discrimination procedure based on large pretrained language\nmodels and their probability distributions. As follow up, we plan to perform an\nempirical analysis of human-written and machine-generated texts to find the\noptimal setup of this evaluation approach. A validation procedure involving\nhuman participants will also check how the automatic evaluation correlates with\nhuman judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03221v1",
            "date": "2020-06-05 03:31:49+00:00",
            "title": "Evaluating Text Coherence at Sentence and Paragraph Levels",
            "authors": [
                "Sennan Liu",
                "Shuang Zeng",
                "Sujian Li"
            ],
            "abstract": "In this paper, to evaluate text coherence, we propose the paragraph ordering\ntask as well as conducting sentence ordering. We collected four distinct\ncorpora from different domains on which we investigate the adaptation of\nexisting sentence ordering methods to a paragraph ordering task. We also\ncompare the learnability and robustness of existing models by artificially\ncreating mini datasets and noisy datasets respectively and verifying the\nefficiency of established models under these circumstances. Furthermore, we\ncarry out human evaluation on the rearranged passages from two competitive\nmodels and confirm that WLCS-l is a better metric performing significantly\nhigher correlations with human rating than tau, the most prevalent metric used\nbefore. Results from these evaluations show that except for certain extreme\nconditions, the recurrent graph neural network-based model is an optimal choice\nfor coherence modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03256v1",
            "date": "2020-06-05 06:59:22+00:00",
            "title": "\"To Target or Not to Target\": Identification and Analysis of Abusive Text Using Ensemble of Classifiers",
            "authors": [
                "Gaurav Verma",
                "Niyati Chhaya",
                "Vishwa Vinay"
            ],
            "abstract": "With rising concern around abusive and hateful behavior on social media\nplatforms, we present an ensemble learning method to identify and analyze the\nlinguistic properties of such content. Our stacked ensemble comprises of three\nmachine learning models that capture different aspects of language and provide\ndiverse and coherent insights about inappropriate language. The proposed\napproach provides comparable results to the existing state-of-the-art on the\nTwitter Abusive Behavior dataset (Founta et al. 2018) without using any user or\nnetwork-related information; solely relying on textual properties. We believe\nthat the presented insights and discussion of shortcomings of current\napproaches will highlight potential directions for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03292v1",
            "date": "2020-06-05 08:21:26+00:00",
            "title": "SEAL: Scientific Keyphrase Extraction and Classification",
            "authors": [
                "Ayush Garg",
                "Sammed Shantinath Kagi",
                "Mayank Singh"
            ],
            "abstract": "Automatic scientific keyphrase extraction is a challenging problem\nfacilitating several downstream scholarly tasks like search, recommendation,\nand ranking. In this paper, we introduce SEAL, a scholarly tool for automatic\nkeyphrase extraction and classification. The keyphrase extraction module\ncomprises two-stage neural architecture composed of Bidirectional Long\nShort-Term Memory cells augmented with Conditional Random Fields. The\nclassification module comprises of a Random Forest classifier. We extensively\nexperiment to showcase the robustness of the system. We evaluate multiple\nstate-of-the-art baselines and show a significant improvement. The current\nsystem is hosted at http://lingo.iitgn.ac.in:5000/.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.03353v1",
            "date": "2020-06-05 10:24:43+00:00",
            "title": "Topic Detection from Conversational Dialogue Corpus with Parallel Dirichlet Allocation Model and Elbow Method",
            "authors": [
                "Haider Khalid",
                "Vincent Wade"
            ],
            "abstract": "A conversational system needs to know how to switch between topics to\ncontinue the conversation for a more extended period. For this topic detection\nfrom dialogue corpus has become an important task for a conversation and\naccurate prediction of conversation topics is important for creating coherent\nand engaging dialogue systems. In this paper, we proposed a topic detection\napproach with Parallel Latent Dirichlet Allocation (PLDA) Model by clustering a\nvocabulary of known similar words based on TF-IDF scores and Bag of Words (BOW)\ntechnique. In the experiment, we use K-mean clustering with Elbow Method for\ninterpretation and validation of consistency within-cluster analysis to select\nthe optimal number of clusters. We evaluate our approach by comparing it with\ntraditional LDA and clustering technique. The experimental results show that\ncombining PLDA with Elbow method selects the optimal number of clusters and\nrefine the topics for the conversation.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.03511v3",
            "date": "2020-06-05 15:28:01+00:00",
            "title": "Unsupervised Translation of Programming Languages",
            "authors": [
                "Marie-Anne Lachaux",
                "Baptiste Roziere",
                "Lowik Chanussot",
                "Guillaume Lample"
            ],
            "abstract": "A transcompiler, also known as source-to-source translator, is a system that\nconverts source code from a high-level programming language (such as C++ or\nPython) to another. Transcompilers are primarily used for interoperability, and\nto port codebases written in an obsolete or deprecated language (e.g. COBOL,\nPython 2) to a modern one. They typically rely on handcrafted rewrite rules,\napplied to the source code abstract syntax tree. Unfortunately, the resulting\ntranslations often lack readability, fail to respect the target language\nconventions, and require manual modifications in order to work properly. The\noverall translation process is timeconsuming and requires expertise in both the\nsource and target languages, making code-translation projects expensive.\nAlthough neural models significantly outperform their rule-based counterparts\nin the context of natural language translation, their applications to\ntranscompilation have been limited due to the scarcity of parallel data in this\ndomain. In this paper, we propose to leverage recent approaches in unsupervised\nmachine translation to train a fully unsupervised neural transcompiler. We\ntrain our model on source code from open source GitHub projects, and show that\nit can translate functions between C++, Java, and Python with high accuracy.\nOur method relies exclusively on monolingual source code, requires no expertise\nin the source or target languages, and can easily be generalized to other\nprogramming languages. We also build and release a test set composed of 852\nparallel functions, along with unit tests to check the correctness of\ntranslations. We show that our model outperforms rule-based commercial\nbaselines by a significant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03533v1",
            "date": "2020-06-05 16:12:18+00:00",
            "title": "Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access",
            "authors": [
                "Seokhwan Kim",
                "Mihail Eric",
                "Karthik Gopalakrishnan",
                "Behnam Hedayatnia",
                "Yang Liu",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "Most prior work on task-oriented dialogue systems are restricted to a limited\ncoverage of domain APIs, while users oftentimes have domain related requests\nthat are not covered by the APIs. In this paper, we propose to expand coverage\nof task-oriented dialogue systems by incorporating external unstructured\nknowledge sources. We define three sub-tasks: knowledge-seeking turn detection,\nknowledge selection, and knowledge-grounded response generation, which can be\nmodeled individually or jointly. We introduce an augmented version of MultiWOZ\n2.1, which includes new out-of-API-coverage turns and responses grounded on\nexternal knowledge sources. We present baselines for each sub-task using both\nconventional and neural approaches. Our experimental results demonstrate the\nneed for further research in this direction to enable more informative\nconversational systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03564v1",
            "date": "2020-06-05 17:19:15+00:00",
            "title": "Spoken dialect identification in Twitter using a multi-filter architecture",
            "authors": [
                "Mohammadreza Banaei",
                "R\u00e9mi Lebret",
                "Karl Aberer"
            ],
            "abstract": "This paper presents our approach for SwissText & KONVENS 2020 shared task 2,\nwhich is a multi-stage neural model for Swiss German (GSW) identification on\nTwitter. Our model outputs either GSW or non-GSW and is not meant to be used as\na generic language identifier. Our architecture consists of two independent\nfilters where the first one favors recall, and the second one filter favors\nprecision (both towards GSW). Moreover, we do not use binary models (GSW vs.\nnot-GSW) in our filters but rather a multi-class classifier with GSW being one\nof the possible labels. Our model reaches F1-score of 0.982 on the test set of\nthe shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03644v5",
            "date": "2020-06-05 19:24:16+00:00",
            "title": "Stance Detection on Social Media: State of the Art and Trends",
            "authors": [
                "Abeer AlDayel",
                "Walid Magdy"
            ],
            "abstract": "Stance detection on social media is an emerging opinion mining paradigm for\nvarious social and political applications in which sentiment analysis may be\nsub-optimal. There has been a growing research interest for developing\neffective methods for stance detection methods varying among multiple\ncommunities including natural language processing, web science, and social\ncomputing. This paper surveys the work on stance detection within those\ncommunities and situates its usage within current opinion mining techniques in\nsocial media. It presents an exhaustive review of stance detection techniques\non social media, including the task definition, different types of targets in\nstance detection, features set used, and various machine learning approaches\napplied. The survey reports state-of-the-art results on the existing benchmark\ndatasets on stance detection, and discusses the most effective approaches. In\naddition, this study explores the emerging trends and different applications of\nstance detection on social media. The study concludes by discussing the gaps in\nthe current existing research and highlights the possible future directions for\nstance detection on social media.",
            "journal ref": "Abeer ALDayel, Walid Magdy, Stance detection on social media:\n  State of the art and trends, Information Processing & Management, Volume 58,\n  Issue 4, 2021, 102597, ISSN 0306-4573",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.03652v2",
            "date": "2020-06-05 19:53:30+00:00",
            "title": "Filtered Inner Product Projection for Crosslingual Embedding Alignment",
            "authors": [
                "Vin Sachidananda",
                "Ziyi Yang",
                "Chenguang Zhu"
            ],
            "abstract": "Due to widespread interest in machine translation and transfer learning,\nthere are numerous algorithms for mapping multiple embeddings to a shared\nrepresentation space. Recently, these algorithms have been studied in the\nsetting of bilingual dictionary induction where one seeks to align the\nembeddings of a source and a target language such that translated word pairs\nlie close to one another in a common representation space. In this paper, we\npropose a method, Filtered Inner Product Projection (FIPP), for mapping\nembeddings to a common representation space and evaluate FIPP in the context of\nbilingual dictionary induction. As semantic shifts are pervasive across\nlanguages and domains, FIPP first identifies the common geometric structure in\nboth embeddings and then, only on the common structure, aligns the Gram\nmatrices of these embeddings. Unlike previous approaches, FIPP is applicable\neven when the source and target embeddings are of differing dimensionalities.\nWe show that our approach outperforms existing methods on the MUSE dataset for\nvarious language pairs. Furthermore, FIPP provides computational benefits both\nin ease of implementation and scalability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03654v4",
            "date": "2020-06-05 19:54:34+00:00",
            "title": "DeBERTa: Decoding-enhanced BERT with Disentangled Attention",
            "authors": [
                "Pengcheng He",
                "Xiaodong Liu",
                "Jianfeng Gao",
                "Weizhu Chen"
            ],
            "abstract": "Recent progress in pre-trained neural language models has significantly\nimproved the performance of many natural language processing (NLP) tasks. In\nthis paper we propose a new model architecture DeBERTa (Decoding-enhanced BERT\nwith disentangled attention) that improves the BERT and RoBERTa models using\ntwo novel techniques. The first is the disentangled attention mechanism, where\neach word is represented using two vectors that encode its content and\nposition, respectively, and the attention weights among words are computed\nusing disentangled matrices on their contents and relative positions,\nrespectively. Second, an enhanced mask decoder is used to incorporate absolute\npositions in the decoding layer to predict the masked tokens in model\npre-training. In addition, a new virtual adversarial training method is used\nfor fine-tuning to improve models' generalization. We show that these\ntechniques significantly improve the efficiency of model pre-training and the\nperformance of both natural language understanding (NLU) and natural langauge\ngeneration (NLG) downstream tasks. Compared to RoBERTa-Large, a DeBERTa model\ntrained on half of the training data performs consistently better on a wide\nrange of NLP tasks, achieving improvements on MNLI by +0.9% (90.2% vs. 91.1%),\non SQuAD v2.0 by +2.3% (88.4% vs. 90.7%) and RACE by +3.6% (83.2% vs. 86.8%).\nNotably, we scale up DeBERTa by training a larger version that consists of 48\nTransform layers with 1.5 billion parameters. The significant performance boost\nmakes the single DeBERTa model surpass the human performance on the SuperGLUE\nbenchmark (Wang et al., 2019a) for the first time in terms of macro-average\nscore (89.9 versus 89.8), and the ensemble DeBERTa model sits atop the\nSuperGLUE leaderboard as of January 6, 2021, out performing the human baseline\nby a decent margin (90.3 versus 89.8).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03659v4",
            "date": "2020-06-05 20:00:28+00:00",
            "title": "DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations",
            "authors": [
                "John Giorgi",
                "Osvald Nitski",
                "Bo Wang",
                "Gary Bader"
            ],
            "abstract": "Sentence embeddings are an important component of many natural language\nprocessing (NLP) systems. Like word embeddings, sentence embeddings are\ntypically learned on large text corpora and then transferred to various\ndownstream tasks, such as clustering and retrieval. Unlike word embeddings, the\nhighest performing solutions for learning sentence embeddings require labelled\ndata, limiting their usefulness to languages and domains where labelled data is\nabundant. In this paper, we present DeCLUTR: Deep Contrastive Learning for\nUnsupervised Textual Representations. Inspired by recent advances in deep\nmetric learning (DML), we carefully design a self-supervised objective for\nlearning universal sentence embeddings that does not require labelled training\ndata. When used to extend the pretraining of transformer-based language models,\nour approach closes the performance gap between unsupervised and supervised\npretraining for universal sentence encoders. Importantly, our experiments\nsuggest that the quality of the learned embeddings scale with both the number\nof trainable parameters and the amount of unlabelled training data. Our code\nand pretrained models are publicly available and can be easily adapted to new\ndomains or used to embed unseen text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03701v1",
            "date": "2020-06-05 21:36:33+00:00",
            "title": "Accelerating Natural Language Understanding in Task-Oriented Dialog",
            "authors": [
                "Ojas Ahuja",
                "Shrey Desai"
            ],
            "abstract": "Task-oriented dialog models typically leverage complex neural architectures\nand large-scale, pre-trained Transformers to achieve state-of-the-art\nperformance on popular natural language understanding benchmarks. However,\nthese models frequently have in excess of tens of millions of parameters,\nmaking them impossible to deploy on-device where resource-efficiency is a major\nconcern. In this work, we show that a simple convolutional model compressed\nwith structured pruning achieves largely comparable results to BERT on ATIS and\nSnips, with under 100K parameters. Moreover, we perform acceleration\nexperiments on CPUs, where we observe our multi-task model predicts intents and\nslots nearly 63x faster than even DistilBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03719v2",
            "date": "2020-06-05 22:25:27+00:00",
            "title": "Relation of the Relations: A New Paradigm of the Relation Extraction Problem",
            "authors": [
                "Zhijing Jin",
                "Yongyi Yang",
                "Xipeng Qiu",
                "Zheng Zhang"
            ],
            "abstract": "In natural language, often multiple entities appear in the same text.\nHowever, most previous works in Relation Extraction (RE) limit the scope to\nidentifying the relation between two entities at a time. Such an approach\ninduces a quadratic computation time, and also overlooks the interdependency\nbetween multiple relations, namely the relation of relations (RoR). Due to the\nsignificance of RoR in existing datasets, we propose a new paradigm of RE that\nconsiders as a whole the predictions of all relations in the same context.\nAccordingly, we develop a data-driven approach that does not require\nhand-crafted rules but learns by itself the RoR, using Graph Neural Networks\nand a relation matrix transformer. Experiments show that our model outperforms\nthe state-of-the-art approaches by +1.12\\% on the ACE05 dataset and +2.55\\% on\nSemEval 2018 Task 7.2, which is a substantial improvement on the two\ncompetitive benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.03773v1",
            "date": "2020-06-06 03:43:15+00:00",
            "title": "Challenges and Thrills of Legal Arguments",
            "authors": [
                "Anurag Pallaprolu",
                "Radha Vaidya",
                "Aditya Swaroop Attawar"
            ],
            "abstract": "State-of-the-art attention based models, mostly centered around the\ntransformer architecture, solve the problem of sequence-to-sequence translation\nusing the so-called scaled dot-product attention. While this technique is\nhighly effective for estimating inter-token attention, it does not answer the\nquestion of inter-sequence attention when we deal with conversation-like\nscenarios. We propose an extension, HumBERT, that attempts to perform\ncontinuous contextual argument generation using locally trained transformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04016v1",
            "date": "2020-06-07 01:20:40+00:00",
            "title": "A Multitask Learning Approach for Diacritic Restoration",
            "authors": [
                "Sawsan Alqahtani",
                "Ajay Mishra",
                "Mona Diab"
            ],
            "abstract": "In many languages like Arabic, diacritics are used to specify pronunciations\nas well as meanings. Such diacritics are often omitted in written text,\nincreasing the number of possible pronunciations and meanings for a word. This\nresults in a more ambiguous text making computational processing on such text\nmore difficult. Diacritic restoration is the task of restoring missing\ndiacritics in the written text. Most state-of-the-art diacritic restoration\nmodels are built on character level information which helps generalize the\nmodel to unseen data, but presumably lose useful information at the word level.\nThus, to compensate for this loss, we investigate the use of multi-task\nlearning to jointly optimize diacritic restoration with related NLP problems\nnamely word segmentation, part-of-speech tagging, and syntactic diacritization.\nWe use Arabic as a case study since it has sufficient data resources for tasks\nthat we consider in our joint modeling. Our joint models significantly\noutperform the baselines and are comparable to the state-of-the-art models that\nare more complex relying on morphological analyzers and/or a lot more data\n(e.g. dialectal data).",
            "journal ref": "ACL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04031v2",
            "date": "2020-06-07 03:12:38+00:00",
            "title": "Semantic Loss Application to Entity Relation Recognition",
            "authors": [
                "Venkata Sasank Pagolu"
            ],
            "abstract": "Usually, entity relation recognition systems either use a pipe-lined model\nthat treats the entity tagging and relation identification as separate tasks or\na joint model that simultaneously identifies the relation and entities. This\npaper compares these two general approaches for the entity relation\nrecognition. State-of-the-art entity relation recognition systems are built\nusing deep recurrent neural networks which often does not capture the symbolic\nknowledge or the logical constraints in the problem. The main contribution of\nthis paper is an end-to-end neural model for joint entity relation extraction\nwhich incorporates a novel loss function. This novel loss function encodes the\nconstraint information in the problem to guide the model training effectively.\nWe show that addition of this loss function to the existing typical loss\nfunctions has a positive impact over the performance of the models. This model\nis truly end-to-end, requires no feature engineering and easily extensible.\nExtensive experimentation has been conducted to evaluate the significance of\ncapturing symbolic knowledge for natural language understanding. Models using\nthis loss function are observed to be outperforming their counterparts and\nconverging faster. Experimental results in this work suggest the use of this\nmethodology for other language understanding applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04102v2",
            "date": "2020-06-07 09:52:05+00:00",
            "title": "Language Models as Fact Checkers?",
            "authors": [
                "Nayeon Lee",
                "Belinda Z. Li",
                "Sinong Wang",
                "Wen-tau Yih",
                "Hao Ma",
                "Madian Khabsa"
            ],
            "abstract": "Recent work has suggested that language models (LMs) store both common-sense\nand factual knowledge learned from pre-training data. In this paper, we\nleverage this implicit knowledge to create an effective end-to-end fact checker\nusing a solely a language model, without any external knowledge or explicit\nretrieval components. While previous work on extracting knowledge from LMs have\nfocused on the task of open-domain question answering, to the best of our\nknowledge, this is the first work to examine the use of language models as fact\ncheckers. In a closed-book setting, we show that our zero-shot LM approach\noutperforms a random baseline on the standard FEVER task, and that our\nfine-tuned LM compares favorably with standard baselines. Though we do not\nultimately outperform methods which use explicit knowledge bases, we believe\nour exploration shows that this method is viable and has much room for\nexploration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04136v1",
            "date": "2020-06-07 12:21:16+00:00",
            "title": "Analysis and Synthesis of Hypo and Hyperarticulated Speech",
            "authors": [
                "Benjamin Picart",
                "Thomas Drugman",
                "Thierry Dutoit"
            ],
            "abstract": "This paper focuses on the analysis and synthesis of hypo and hyperarticulated\nspeech in the framework of HMM-based speech synthesis. First of all, a new\nFrench database matching our needs was created, which contains three identical\nsets, pronounced with three different degrees of articulation: neutral, hypo\nand hyperarticulated speech. On that basis, acoustic and phonetic analyses were\nperformed. It is shown that the degrees of articulation significantly\ninfluence, on one hand, both vocal tract and glottal characteristics, and on\nthe other hand, speech rate, phone durations, phone variations and the presence\nof glottal stops. Finally, neutral, hypo and hyperarticulated speech are\nsynthesized using HMM-based speech synthesis and both objective and subjective\ntests aiming at assessing the generated speech quality are performed. These\ntests show that synthesized hypoarticulated speech seems to be less naturally\nrendered than neutral and hyperarticulated speech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2006.04148v1",
            "date": "2020-06-07 13:26:32+00:00",
            "title": "Interactive Extractive Search over Biomedical Corpora",
            "authors": [
                "Hillel Taub-Tabib",
                "Micah Shlain",
                "Shoval Sadde",
                "Dan Lahav",
                "Matan Eyal",
                "Yaara Cohen",
                "Yoav Goldberg"
            ],
            "abstract": "We present a system that allows life-science researchers to search a\nlinguistically annotated corpus of scientific texts using patterns over\ndependency graphs, as well as using patterns over token sequences and a\npowerful variant of boolean keyword queries. In contrast to previous attempts\nto dependency-based search, we introduce a light-weight query language that\ndoes not require the user to know the details of the underlying linguistic\nrepresentations, and instead to query the corpus by providing an example\nsentence coupled with simple markup. Search is performed at an interactive\nspeed due to efficient linguistic graph-indexing and retrieval engine. This\nallows for rapid exploration, development and refinement of user queries. We\ndemonstrate the system using example workflows over two corpora: the PubMed\ncorpus including 14,446,243 PubMed abstracts and the CORD-19 dataset, a\ncollection of over 45,000 research papers focused on COVID-19 research. The\nsystem is publicly available at https://allenai.github.io/spike",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04152v3",
            "date": "2020-06-07 13:38:32+00:00",
            "title": "BERT Loses Patience: Fast and Robust Inference with Early Exit",
            "authors": [
                "Wangchunshu Zhou",
                "Canwen Xu",
                "Tao Ge",
                "Julian McAuley",
                "Ke Xu",
                "Furu Wei"
            ],
            "abstract": "In this paper, we propose Patience-based Early Exit, a straightforward yet\neffective inference method that can be used as a plug-and-play technique to\nsimultaneously improve the efficiency and robustness of a pretrained language\nmodel (PLM). To achieve this, our approach couples an internal-classifier with\neach layer of a PLM and dynamically stops inference when the intermediate\npredictions of the internal classifiers remain unchanged for a pre-defined\nnumber of steps. Our approach improves inference efficiency as it allows the\nmodel to make a prediction with fewer layers. Meanwhile, experimental results\nwith an ALBERT model show that our method can improve the accuracy and\nrobustness of the model by preventing it from overthinking and exploiting\nmultiple classifiers for prediction, yielding a better accuracy-speed trade-off\ncompared to existing early exit methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04232v1",
            "date": "2020-06-07 18:52:58+00:00",
            "title": "Tensors over Semirings for Latent-Variable Weighted Logic Programs",
            "authors": [
                "Esma Balkir",
                "Daniel Gildea",
                "Shay Cohen"
            ],
            "abstract": "Semiring parsing is an elegant framework for describing parsers by using\nsemiring weighted logic programs. In this paper we present a generalization of\nthis concept: latent-variable semiring parsing. With our framework, any\nsemiring weighted logic program can be latentified by transforming weights from\nscalar values of a semiring to rank-n arrays, or tensors, of semiring values,\nallowing the modelling of latent variables within the semiring parsing\nframework. Semiring is too strong a notion when dealing with tensors, and we\nhave to resort to a weaker structure: a partial semiring. We prove that this\ngeneralization preserves all the desired properties of the original semiring\nframework while strictly increasing its expressiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04315v4",
            "date": "2020-06-08 01:49:27+00:00",
            "title": "Counterfactual VQA: A Cause-Effect Look at Language Bias",
            "authors": [
                "Yulei Niu",
                "Kaihua Tang",
                "Hanwang Zhang",
                "Zhiwu Lu",
                "Xian-Sheng Hua",
                "Ji-Rong Wen"
            ],
            "abstract": "VQA models may tend to rely on language bias as a shortcut and thus fail to\nsufficiently learn the multi-modal knowledge from both vision and language.\nRecent debiasing methods proposed to exclude the language prior during\ninference. However, they fail to disentangle the \"good\" language context and\n\"bad\" language bias from the whole. In this paper, we investigate how to\nmitigate language bias in VQA. Motivated by causal effects, we proposed a novel\ncounterfactual inference framework, which enables us to capture the language\nbias as the direct causal effect of questions on answers and reduce the\nlanguage bias by subtracting the direct language effect from the total causal\neffect. Experiments demonstrate that our proposed counterfactual inference\nframework 1) is general to various VQA backbones and fusion strategies, 2)\nachieves competitive performance on the language-bias sensitive VQA-CP dataset\nwhile performs robustly on the balanced VQA v2 dataset without any augmented\ndata. The code is available at https://github.com/yuleiniu/cfvqa.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.04334v3",
            "date": "2020-06-08 03:05:28+00:00",
            "title": "Characterizing Sociolinguistic Variation in the Competing Vaccination Communities",
            "authors": [
                "Shahan Ali Memon",
                "Aman Tyagi",
                "David R. Mortensen",
                "Kathleen M. Carley"
            ],
            "abstract": "Public health practitioners and policy makers grapple with the challenge of\ndevising effective message-based interventions for debunking public health\nmisinformation in cyber communities. \"Framing\" and \"personalization\" of the\nmessage is one of the key features for devising a persuasive messaging\nstrategy. For an effective health communication, it is imperative to focus on\n\"preference-based framing\" where the preferences of the target sub-community\nare taken into consideration. To achieve that, it is important to understand\nand hence characterize the target sub-communities in terms of their social\ninteractions. In the context of health-related misinformation, vaccination\nremains to be the most prevalent topic of discord. Hence, in this paper, we\nconduct a sociolinguistic analysis of the two competing vaccination communities\non Twitter: \"pro-vaxxers\" or individuals who believe in the effectiveness of\nvaccinations, and \"anti-vaxxers\" or individuals who are opposed to\nvaccinations. Our data analysis show significant linguistic variation between\nthe two communities in terms of their usage of linguistic intensifiers,\npronouns, and uncertainty words. Our network-level analysis show significant\ndifferences between the two communities in terms of their network density,\necho-chamberness, and the EI index. We hypothesize that these sociolinguistic\ndifferences can be used as proxies to characterize and understand these\ncommunities to devise better message interventions.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.04513v1",
            "date": "2020-06-08 12:30:25+00:00",
            "title": "Combining word embeddings and convolutional neural networks to detect duplicated questions",
            "authors": [
                "Yoan Dimitrov"
            ],
            "abstract": "Detecting semantic similarities between sentences is still a challenge today\ndue to the ambiguity of natural languages. In this work, we propose a simple\napproach to identifying semantically similar questions by combining the\nstrengths of word embeddings and Convolutional Neural Networks (CNNs). In\naddition, we demonstrate how the cosine similarity metric can be used to\neffectively compare feature vectors. Our network is trained on the Quora\ndataset, which contains over 400k question pairs. We experiment with different\nembedding approaches such as Word2Vec, Fasttext, and Doc2Vec and investigate\nthe effects these approaches have on model performance. Our model achieves\ncompetitive results on the Quora dataset and complements the well-established\nevidence that CNNs can be utilized for paraphrase detection tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04611v1",
            "date": "2020-06-08 14:07:58+00:00",
            "title": "A Comprehensive Survey on Aspect Based Sentiment Analysis",
            "authors": [
                "Kaustubh Yadav"
            ],
            "abstract": "Aspect Based Sentiment Analysis (ABSA) is the sub-field of Natural Language\nProcessing that deals with essentially splitting our data into aspects ad\nfinally extracting the sentiment information. ABSA is known to provide more\ninformation about the context than general sentiment analysis. In this study,\nour aim is to explore the various methodologies practiced while performing\nABSA, and providing a comparative study. This survey paper discusses various\nsolutions in-depth and gives a comparison between them. And is conveniently\ndivided into sections to get a holistic view on the process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04643v1",
            "date": "2020-06-08 14:48:14+00:00",
            "title": "ColdGANs: Taming Language GANs with Cautious Sampling Strategies",
            "authors": [
                "Thomas Scialom",
                "Paul-Alexis Dray",
                "Sylvain Lamprier",
                "Benjamin Piwowarski",
                "Jacopo Staiano"
            ],
            "abstract": "Training regimes based on Maximum Likelihood Estimation (MLE) suffer from\nknown limitations, often leading to poorly generated text sequences. At the\nroot of these limitations is the mismatch between training and inference, i.e.\nthe so-called exposure bias, exacerbated by considering only the reference\ntexts as correct, while in practice several alternative formulations could be\nas good. Generative Adversarial Networks (GANs) can mitigate those limitations\nbut the discrete nature of text has hindered their application to language\ngeneration: the approaches proposed so far, based on Reinforcement Learning,\nhave been shown to underperform MLE. Departing from previous works, we analyze\nthe exploration step in GANs applied to text generation, and show how classical\nsampling results in unstable training. We propose to consider alternative\nexploration strategies in a GAN framework that we name ColdGANs, where we force\nthe sampling to be close to the distribution modes to get smoother learning\ndynamics. For the first time, to the best of our knowledge, the proposed\nlanguage GANs compare favorably to MLE, and obtain improvements over the\nstate-of-the-art on three generative tasks, namely unconditional text\ngeneration, question generation, and abstractive summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.04660v2",
            "date": "2020-06-08 15:03:38+00:00",
            "title": "Read what you need: Controllable Aspect-based Opinion Summarization of Tourist Reviews",
            "authors": [
                "Rajdeep Mukherjee",
                "Hari Chandana Peruri",
                "Uppada Vishnu",
                "Pawan Goyal",
                "Sourangshu Bhattacharya",
                "Niloy Ganguly"
            ],
            "abstract": "Manually extracting relevant aspects and opinions from large volumes of\nuser-generated text is a time-consuming process. Summaries, on the other hand,\nhelp readers with limited time budgets to quickly consume the key ideas from\nthe data. State-of-the-art approaches for multi-document summarization,\nhowever, do not consider user preferences while generating summaries. In this\nwork, we argue the need and propose a solution for generating personalized\naspect-based opinion summaries from large collections of online tourist\nreviews. We let our readers decide and control several attributes of the\nsummary such as the length and specific aspects of interest among others.\nSpecifically, we take an unsupervised approach to extract coherent aspects from\ntourist reviews posted on TripAdvisor. We then propose an Integer Linear\nProgramming (ILP) based extractive technique to select an informative subset of\nopinions around the identified aspects while respecting the user-specified\nvalues for various control parameters. Finally, we evaluate and compare our\nsummaries using crowdsourcing and ROUGE-based metrics and obtain competitive\nresults.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.05014v2",
            "date": "2020-06-09 02:08:03+00:00",
            "title": "HausaMT v1.0: Towards English-Hausa Neural Machine Translation",
            "authors": [
                "Adewale Akinfaderin"
            ],
            "abstract": "Neural Machine Translation (NMT) for low-resource languages suffers from low\nperformance because of the lack of large amounts of parallel data and language\ndiversity. To contribute to ameliorating this problem, we built a baseline\nmodel for English-Hausa machine translation, which is considered a task for\nlow-resource language. The Hausa language is the second largest Afro-Asiatic\nlanguage in the world after Arabic and it is the third largest language for\ntrading across a larger swath of West Africa countries, after English and\nFrench. In this paper, we curated different datasets containing Hausa-English\nparallel corpus for our translation. We trained baseline models and evaluated\nthe performance of our models using the Recurrent and Transformer\nencoder-decoder architecture with two tokenization approaches: standard\nword-level tokenization and Byte Pair Encoding (BPE) subword tokenization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05281v1",
            "date": "2020-06-09 14:15:33+00:00",
            "title": "Extensive Error Analysis and a Learning-Based Evaluation of Medical Entity Recognition Systems to Approximate User Experience",
            "authors": [
                "Isar Nejadgholi",
                "Kathleen C. Fraser",
                "Berry De Bruijn"
            ],
            "abstract": "When comparing entities extracted by a medical entity recognition system with\ngold standard annotations over a test set, two types of mismatches might occur,\nlabel mismatch or span mismatch. Here we focus on span mismatch and show that\nits severity can vary from a serious error to a fully acceptable entity\nextraction due to the subjectivity of span annotations. For a domain-specific\nBERT-based NER system, we showed that 25% of the errors have the same labels\nand overlapping span with gold standard entities. We collected expert judgement\nwhich shows more than 90% of these mismatches are accepted or partially\naccepted by the user. Using the training set of the NER system, we built a fast\nand lightweight entity classifier to approximate the user experience of such\nmismatches through accepting or rejecting them. The decisions made by this\nclassifier are used to calculate a learning-based F-score which is shown to be\na better approximation of a forgiving user's experience than the relaxed\nF-score. We demonstrated the results of applying the proposed evaluation metric\nfor a variety of deep learning medical entity recognition models trained with\ntwo datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05469v1",
            "date": "2020-06-09 19:29:41+00:00",
            "title": "Examination and Extension of Strategies for Improving Personalized Language Modeling via Interpolation",
            "authors": [
                "Liqun Shao",
                "Sahitya Mantravadi",
                "Tom Manzini",
                "Alejandro Buendia",
                "Manon Knoertzer",
                "Soundar Srinivasan",
                "Chris Quirk"
            ],
            "abstract": "In this paper, we detail novel strategies for interpolating personalized\nlanguage models and methods to handle out-of-vocabulary (OOV) tokens to improve\npersonalized language models. Using publicly available data from Reddit, we\ndemonstrate improvements in offline metrics at the user level by interpolating\na global LSTM-based authoring model with a user-personalized n-gram model. By\noptimizing this approach with a back-off to uniform OOV penalty and the\ninterpolation coefficient, we observe that over 80% of users receive a lift in\nperplexity, with an average of 5.2% in perplexity lift per user. In doing this\nresearch we extend previous work in building NLIs and improve the robustness of\nmetrics for downstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05477v1",
            "date": "2020-06-09 19:40:19+00:00",
            "title": "Unsupervised Paraphrase Generation using Pre-trained Language Models",
            "authors": [
                "Chaitra Hegde",
                "Shrikumar Patil"
            ],
            "abstract": "Large scale Pre-trained Language Models have proven to be very powerful\napproach in various Natural language tasks. OpenAI's GPT-2\n\\cite{radford2019language} is notable for its capability to generate fluent,\nwell formulated, grammatically consistent text and for phrase completions. In\nthis paper we leverage this generation capability of GPT-2 to generate\nparaphrases without any supervision from labelled data. We examine how the\nresults compare with other supervised and unsupervised approaches and the\neffect of using paraphrases for data augmentation on downstream tasks such as\nclassification. Our experiments show that paraphrases generated with our model\nare of good quality, are diverse and improves the downstream task performance\nwhen used for data augmentation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05702v1",
            "date": "2020-06-10 07:50:44+00:00",
            "title": "Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network",
            "authors": [
                "Yutai Hou",
                "Wanxiang Che",
                "Yongkui Lai",
                "Zhihan Zhou",
                "Yijia Liu",
                "Han Liu",
                "Ting Liu"
            ],
            "abstract": "In this paper, we explore the slot tagging with only a few labeled support\nsentences (a.k.a. few-shot). Few-shot slot tagging faces a unique challenge\ncompared to the other few-shot classification problems as it calls for modeling\nthe dependencies between labels. But it is hard to apply previously learned\nlabel dependencies to an unseen domain, due to the discrepancy of label sets.\nTo tackle this, we introduce a collapsed dependency transfer mechanism into the\nconditional random field (CRF) to transfer abstract label dependency patterns\nas transition scores. In the few-shot setting, the emission score of CRF can be\ncalculated as a word's similarity to the representation of each label. To\ncalculate such similarity, we propose a Label-enhanced Task-Adaptive Projection\nNetwork (L-TapNet) based on the state-of-the-art few-shot classification model\n-- TapNet, by leveraging label name semantics in representing labels.\nExperimental results show that our model significantly outperforms the\nstrongest few-shot learning baseline by 14.64 F1 scores in the one-shot\nsetting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05726v2",
            "date": "2020-06-10 08:32:56+00:00",
            "title": "Estimating semantic structure for the VQA answer space",
            "authors": [
                "Corentin Kervadec",
                "Grigory Antipov",
                "Moez Baccouche",
                "Christian Wolf"
            ],
            "abstract": "Since its appearance, Visual Question Answering (VQA, i.e. answering a\nquestion posed over an image), has always been treated as a classification\nproblem over a set of predefined answers. Despite its convenience, this\nclassification approach poorly reflects the semantics of the problem limiting\nthe answering to a choice between independent proposals, without taking into\naccount the similarity between them (e.g. equally penalizing for answering cat\nor German shepherd instead of dog). We address this issue by proposing (1) two\nmeasures of proximity between VQA classes, and (2) a corresponding loss which\ntakes into account the estimated proximity. This significantly improves the\ngeneralization of VQA models by reducing their language bias. In particular, we\nshow that our approach is completely model-agnostic since it allows consistent\nimprovements with three different VQA models. Finally, by combining our method\nwith a language bias reduction approach, we report SOTA-level performance on\nthe challenging VQAv2-CP dataset.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.05744v2",
            "date": "2020-06-10 09:22:19+00:00",
            "title": "MC-BERT: Efficient Language Pre-Training via a Meta Controller",
            "authors": [
                "Zhenhui Xu",
                "Linyuan Gong",
                "Guolin Ke",
                "Di He",
                "Shuxin Zheng",
                "Liwei Wang",
                "Jiang Bian",
                "Tie-Yan Liu"
            ],
            "abstract": "Pre-trained contextual representations (e.g., BERT) have become the\nfoundation to achieve state-of-the-art results on many NLP tasks. However,\nlarge-scale pre-training is computationally expensive. ELECTRA, an early\nattempt to accelerate pre-training, trains a discriminative model that predicts\nwhether each input token was replaced by a generator. Our studies reveal that\nELECTRA's success is mainly due to its reduced complexity of the pre-training\ntask: the binary classification (replaced token detection) is more efficient to\nlearn than the generation task (masked language modeling). However, such a\nsimplified task is less semantically informative. To achieve better efficiency\nand effectiveness, we propose a novel meta-learning framework, MC-BERT. The\npre-training task is a multi-choice cloze test with a reject option, where a\nmeta controller network provides training input and candidates. Results over\nGLUE natural language understanding benchmark demonstrate that our proposed\nmethod is both efficient and effective: it outperforms baselines on GLUE\nsemantic tasks given the same computational budget.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.05908v4",
            "date": "2020-06-10 15:52:52+00:00",
            "title": "Embed2Detect: Temporally Clustered Embedded Words for Event Detection in Social Media",
            "authors": [
                "Hansi Hettiarachchi",
                "Mariam Adedoyin-Olowe",
                "Jagdev Bhogal",
                "Mohamed Medhat Gaber"
            ],
            "abstract": "Social media is becoming a primary medium to discuss what is happening around\nthe world. Therefore, the data generated by social media platforms contain rich\ninformation which describes the ongoing events. Further, the timeliness\nassociated with these data is capable of facilitating immediate insights.\nHowever, considering the dynamic nature and high volume of data production in\nsocial media data streams, it is impractical to filter the events manually and\ntherefore, automated event detection mechanisms are invaluable to the\ncommunity. Apart from a few notable exceptions, most previous research on\nautomated event detection have focused only on statistical and syntactical\nfeatures in data and lacked the involvement of underlying semantics which are\nimportant for effective information retrieval from text since they represent\nthe connections between words and their meanings. In this paper, we propose a\nnovel method termed Embed2Detect for event detection in social media by\ncombining the characteristics in word embeddings and hierarchical agglomerative\nclustering. The adoption of word embeddings gives Embed2Detect the capability\nto incorporate powerful semantical features into event detection and overcome a\nmajor limitation inherent in previous approaches. We experimented our method on\ntwo recent real social media data sets which represent the sports and political\ndomain and also compared the results to several state-of-the-art methods. The\nobtained results show that Embed2Detect is capable of effective and efficient\nevent detection and it outperforms the recent event detection methods. For the\nsports data set, Embed2Detect achieved 27% higher F-measure than the\nbest-performed baseline and for the political data set, it was an increase of\n29%.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.05987v3",
            "date": "2020-06-10 17:57:03+00:00",
            "title": "Revisiting Few-sample BERT Fine-tuning",
            "authors": [
                "Tianyi Zhang",
                "Felix Wu",
                "Arzoo Katiyar",
                "Kilian Q. Weinberger",
                "Yoav Artzi"
            ],
            "abstract": "This paper is a study of fine-tuning of BERT contextual representations, with\nfocus on commonly observed instabilities in few-sample scenarios. We identify\nseveral factors that cause this instability: the common use of a non-standard\noptimization method with biased gradient estimation; the limited applicability\nof significant parts of the BERT network for down-stream tasks; and the\nprevalent practice of using a pre-determined, and small number of training\niterations. We empirically test the impact of these factors, and identify\nalternative practices that resolve the commonly observed instability of the\nprocess. In light of these observations, we re-visit recently proposed methods\nto improve few-sample fine-tuning with BERT and re-evaluate their\neffectiveness. Generally, we observe the impact of these methods diminishes\nsignificantly with our modified process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06026v1",
            "date": "2020-06-10 18:32:35+00:00",
            "title": "Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges",
            "authors": [
                "Maxine Eskenazi",
                "Tiancheng Zhao"
            ],
            "abstract": "This USER Workshop was convened with the goal of defining future research\ndirections for the burgeoning intelligent agent research community and to\ncommunicate them to the National Science Foundation. It took place in\nPittsburgh Pennsylvania on October 24 and 25, 2019 and was sponsored by\nNational Science Foundation Grant Number IIS-1934222. Any opinions, findings\nand conclusions or future directions expressed in this document are those of\nthe authors and do not necessarily reflect the views of the National Science\nFoundation. The 27 participants presented their individual research interests\nand their personal research goals. In the breakout sessions that followed, the\nparticipants defined the main research areas within the domain of intelligent\nagents and they discussed the major future directions that the research in each\narea of this domain should take",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06114v2",
            "date": "2020-06-10 23:40:11+00:00",
            "title": "Consolidating Commonsense Knowledge",
            "authors": [
                "Filip Ilievski",
                "Pedro Szekely",
                "Jingwei Cheng",
                "Fu Zhang",
                "Ehsan Qasemi"
            ],
            "abstract": "Commonsense reasoning is an important aspect of building robust AI systems\nand is receiving significant attention in the natural language understanding,\ncomputer vision, and knowledge graphs communities. At present, a number of\nvaluable commonsense knowledge sources exist, with different foci, strengths,\nand weaknesses. In this paper, we list representative sources and their\nproperties. Based on this survey, we propose principles and a representation\nmodel in order to consolidate them into a Common Sense Knowledge Graph (CSKG).\nWe apply this approach to consolidate seven separate sources into a first\nintegrated CSKG. We present statistics of CSKG, present initial investigations\nof its utility on four QA datasets, and list learned lessons.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2006.06143v1",
            "date": "2020-06-11 01:31:17+00:00",
            "title": "Emora STDM: A Versatile Framework for Innovative Dialogue System Development",
            "authors": [
                "James D. Finch",
                "Jinho D. Choi"
            ],
            "abstract": "This demo paper presents Emora STDM (State Transition Dialogue Manager), a\ndialogue system development framework that provides novel workflows for rapid\nprototyping of chat-based dialogue managers as well as collaborative\ndevelopment of complex interactions. Our framework caters to a wide range of\nexpertise levels by supporting interoperability between two popular approaches,\nstate machine and information state, to dialogue management. Our Natural\nLanguage Expression package allows seamless integration of pattern matching,\ncustom NLP modules, and database querying, that makes the workflows much more\nefficient. As a user study, we adopt this framework to an interdisciplinary\nundergraduate course where students with both technical and non-technical\nbackgrounds are able to develop creative dialogue managers in a short period of\ntime.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06295v1",
            "date": "2020-06-11 10:01:59+00:00",
            "title": "Transparency in Language Generation: Levels of Automation",
            "authors": [
                "Justin Edwards",
                "Allison Perrone",
                "Philip R. Doyle"
            ],
            "abstract": "Language models and conversational systems are growing increasingly advanced,\ncreating outputs that may be mistaken for humans. Consumers may thus be misled\nby advertising, media reports, or vagueness regarding the role of automation in\nthe production of language. We propose a taxonomy of language automation, based\non the SAE levels of driving automation, to establish a shared set of terms for\ndescribing automated language. It is our hope that the proposed taxonomy can\nincrease transparency in this rapidly advancing field.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2006.06328v1",
            "date": "2020-06-11 11:03:49+00:00",
            "title": "See what I'm saying? Comparing Intelligent Personal Assistant use for Native and Non-Native Language Speakers",
            "authors": [
                "Yunhan Wu",
                "Daniel Rough",
                "Anna Bleakley",
                "Justin Edwards",
                "Orla Cooney",
                "Philip R. Doyle",
                "Leigh Clark",
                "Benjamin R. Cowan"
            ],
            "abstract": "Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means\nthat many interact in a non-native language. Yet we know little about how IPAs\ncurrently support or hinder these users. Through native (L1) and non-native\n(L2) English speakers interacting with Google Assistant on a smartphone and\nsmart speaker, we aim to understand this more deeply. Interviews revealed that\nL2 speakers prioritised utterance planning around perceived linguistic\nlimitations, as opposed to L1 speakers prioritising succinctness because of\nsystem limitations. L2 speakers see IPAs as insensitive to linguistic needs\nresulting in failed interaction. L2 speakers clearly preferred using\nsmartphones, as visual feedback supported diagnoses of communication breakdowns\nwhilst allowing time to process query results. Conversely, L1 speakers\npreferred smart speakers, with audio feedback being seen as sufficient. We\ndiscuss the need to tailor the IPA experience for L2 users, emphasising visual\nfeedback whilst reducing the burden of language production.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2006.06331v1",
            "date": "2020-06-11 11:06:42+00:00",
            "title": "Mental Workload and Language Production in Non-Native Speaker IPA Interaction",
            "authors": [
                "Yunhan Wu",
                "Justin Edwards",
                "Orla Cooney",
                "Anna Bleakley",
                "Philip R. Doyle",
                "Leigh Clark",
                "Daniel Rough",
                "Benjamin R. Cowan"
            ],
            "abstract": "Through proliferation on smartphones and smart speakers, intelligent personal\nassistants (IPAs) have made speech a common interaction modality. Yet, due to\nlinguistic coverage and varying levels of functionality, many speakers engage\nwith IPAs using a non-native language. This may impact the mental workload and\npattern of language production displayed by non-native speakers. We present a\nmixed-design experiment, wherein native (L1) and non-native (L2) English\nspeakers completed tasks with IPAs through smartphones and smart speakers. We\nfound significantly higher mental workload for L2 speakers during IPA\ninteractions. Contrary to our hypotheses, we found no significant differences\nbetween L1 and L2 speakers in terms of number of turns, lexical complexity,\ndiversity, or lexical adaptation when encountering errors. These findings are\ndiscussed in relation to language production and processing load increases for\nL2 speakers in IPA interaction.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2006.06436v1",
            "date": "2020-06-11 13:48:04+00:00",
            "title": "A Probabilistic Model with Commonsense Constraints for Pattern-based Temporal Fact Extraction",
            "authors": [
                "Yang Zhou",
                "Tong Zhao",
                "Meng Jiang"
            ],
            "abstract": "Textual patterns (e.g., Country's president Person) are specified and/or\ngenerated for extracting factual information from unstructured data.\nPattern-based information extraction methods have been recognized for their\nefficiency and transferability. However, not every pattern is reliable: A major\nchallenge is to derive the most complete and accurate facts from diverse and\nsometimes conflicting extractions. In this work, we propose a probabilistic\ngraphical model which formulates fact extraction in a generative process. It\nautomatically infers true facts and pattern reliability without any\nsupervision. It has two novel designs specially for temporal facts: (1) it\nmodels pattern reliability on two types of time signals, including temporal tag\nin text and text generation time; (2) it models commonsense constraints as\nobservable variables. Experimental results demonstrate that our model\nsignificantly outperforms existing methods on extracting true temporal facts\nfrom news data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.06462v2",
            "date": "2020-06-11 14:18:35+00:00",
            "title": "Learning advanced mathematical computations from examples",
            "authors": [
                "Fran\u00e7ois Charton",
                "Amaury Hayat",
                "Guillaume Lample"
            ],
            "abstract": "Using transformers over large generated datasets, we train models to learn\nmathematical properties of differential systems, such as local stability,\nbehavior at infinity and controllability. We achieve near perfect prediction of\nqualitative characteristics, and good approximations of numerical features of\nthe system. This demonstrates that neural networks can learn to perform complex\ncomputations, grounded in advanced theory, from examples, without built-in\nmathematical knowledge.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2006.06637v2",
            "date": "2020-06-11 17:30:07+00:00",
            "title": "Exploring Weaknesses of VQA Models through Attribution Driven Insights",
            "authors": [
                "Shaunak Halbe"
            ],
            "abstract": "Deep Neural Networks have been successfully used for the task of Visual\nQuestion Answering for the past few years owing to the availability of relevant\nlarge scale datasets. However these datasets are created in artificial settings\nand rarely reflect the real world scenario. Recent research effectively applies\nthese VQA models for answering visual questions for the blind. Despite\nachieving high accuracy these models appear to be susceptible to variation in\ninput questions.We analyze popular VQA models through the lens of attribution\n(input's influence on predictions) to gain valuable insights. Further, We use\nthese insights to craft adversarial attacks which inflict significant damage to\nthese systems with negligible change in meaning of the input questions. We\nbelieve this will enhance development of systems more robust to the possible\nvariations in inputs when deployed to assist the visually impaired.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.06666v2",
            "date": "2020-06-11 17:58:48+00:00",
            "title": "VirTex: Learning Visual Representations from Textual Annotations",
            "authors": [
                "Karan Desai",
                "Justin Johnson"
            ],
            "abstract": "The de-facto approach to many vision tasks is to start from pretrained visual\nrepresentations, typically learned via supervised training on ImageNet. Recent\nmethods have explored unsupervised pretraining to scale to vast quantities of\nunlabeled images. In contrast, we aim to learn high-quality visual\nrepresentations from fewer images. To this end, we revisit supervised\npretraining, and seek data-efficient alternatives to classification-based\npretraining. We propose VirTex -- a pretraining approach using semantically\ndense captions to learn visual representations. We train convolutional networks\nfrom scratch on COCO Captions, and transfer them to downstream recognition\ntasks including image classification, object detection, and instance\nsegmentation. On all tasks, VirTex yields features that match or exceed those\nlearned on ImageNet -- supervised or unsupervised -- despite using up to ten\ntimes fewer images.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.06877v1",
            "date": "2020-06-11 23:48:27+00:00",
            "title": "High-Precision Extraction of Emerging Concepts from Scientific Literature",
            "authors": [
                "Daniel King",
                "Doug Downey",
                "Daniel S. Weld"
            ],
            "abstract": "Identification of new concepts in scientific literature can help power\nfaceted search, scientific trend analysis, knowledge-base construction, and\nmore, but current methods are lacking. Manual identification cannot keep up\nwith the torrent of new publications, while the precision of existing automatic\ntechniques is too low for many applications. We present an unsupervised concept\nextraction method for scientific literature that achieves much higher precision\nthan previous work. Our approach relies on a simple but novel intuition: each\nscientific concept is likely to be introduced or popularized by a single paper\nthat is disproportionately cited by subsequent papers mentioning the concept.\nFrom a corpus of computer science papers on arXiv, we find that our method\nachieves a Precision@1000 of 99%, compared to 86% for prior work, and a\nsubstantially better precision-yield trade-off across the top 15,000\nextractions. To stimulate research in this area, we release our code and data\n(https://github.com/allenai/ForeCite).",
            "journal ref": "Proceedings of the 43rd International ACM SIGIR Conference on\n  Research and Development in Information Retrieval (2020) 1549-1552",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.07398v1",
            "date": "2020-06-12 18:15:59+00:00",
            "title": "Evaluating a Multi-sense Definition Generation Model for Multiple Languages",
            "authors": [
                "Arman Kabiri",
                "Paul Cook"
            ],
            "abstract": "Most prior work on definition modeling has not accounted for polysemy, or has\ndone so by considering definition modeling for a target word in a given\ncontext. In contrast, in this study, we propose a context-agnostic approach to\ndefinition modeling, based on multi-sense word embeddings, that is capable of\ngenerating multiple definitions for a target word. In further, contrast to most\nprior work, which has primarily focused on English, we evaluate our proposed\napproach on fifteen different datasets covering nine languages from several\nlanguage families. To evaluate our approach we consider several variations of\nBLEU. Our results demonstrate that our proposed multi-sense model outperforms a\nsingle-sense model on all fifteen datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07499v1",
            "date": "2020-06-12 22:38:55+00:00",
            "title": "A Generative Model for Joint Natural Language Understanding and Generation",
            "authors": [
                "Bo-Hsiang Tseng",
                "Jianpeng Cheng",
                "Yimai Fang",
                "David Vandyke"
            ],
            "abstract": "Natural language understanding (NLU) and natural language generation (NLG)\nare two fundamental and related tasks in building task-oriented dialogue\nsystems with opposite objectives: NLU tackles the transformation from natural\nlanguage to formal representations, whereas NLG does the reverse. A key to\nsuccess in either task is parallel training data which is expensive to obtain\nat a large scale. In this work, we propose a generative model which couples NLU\nand NLG through a shared latent variable. This approach allows us to explore\nboth spaces of natural language and formal representations, and facilitates\ninformation sharing through the latent space to eventually benefit NLU and NLG.\nOur model achieves state-of-the-art performance on two dialogue datasets with\nboth flat and tree-structured formal representations. We also show that the\nmodel can be trained in a semi-supervised fashion by utilising unlabelled data\nto boost its performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07510v1",
            "date": "2020-06-12 23:34:05+00:00",
            "title": "Do Dogs have Whiskers? A New Knowledge Base of hasPart Relations",
            "authors": [
                "Sumithra Bhakthavatsalam",
                "Kyle Richardson",
                "Niket Tandon",
                "Peter Clark"
            ],
            "abstract": "We present a new knowledge-base of hasPart relationships, extracted from a\nlarge corpus of generic statements. Complementary to other resources available,\nit is the first which is all three of: accurate (90% precision), salient\n(covers relationships a person may mention), and has high coverage of common\nterms (approximated as within a 10 year old's vocabulary), as well as having\nseveral times more hasPart entries than in the popular ontologies ConceptNet\nand WordNet. In addition, it contains information about quantifiers, argument\nmodifiers, and links the entities to appropriate concepts in Wikipedia and\nWordNet. The knowledge base is available at https://allenai.org/data/haspartkb",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07519v2",
            "date": "2020-06-13 00:27:14+00:00",
            "title": "Conversational User Interfaces for Blind Knowledge Workers: A Case Study",
            "authors": [
                "Kyle Dent",
                "Kalai Ramea"
            ],
            "abstract": "Modern trends in interface design for office equipment using controls on\ntouch surfaces create greater obstacles for blind and visually impaired users\nand contribute to an environment of dependency in work settings. We believe\nthat \\textit{conversational user interfaces} (CUIs) offer a reasonable\nalternative to touchscreen interactions enabling more access and most\nimportantly greater independence for blind knowledge workers. We present a case\nstudy of our work to develop a conversational user interface for accessibility\nfor multifunction printers. We also describe our approach to conversational\ninterfaces in general, which emphasizes task-based collaborative interactions\nbetween people and intelligent agents, and we detail the specifics of the\nsolution we created for multifunction printers. To guide our design, we worked\nwith a group of blind and visually impaired individuals starting with focus\ngroup sessions to ascertain the challenges our target users face in their\nprofessional lives. We followed our technology development with a user study to\nassess the solution and direct our future efforts. We present our findings and\nconclusions from the study.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2006.07581v2",
            "date": "2020-06-13 07:02:08+00:00",
            "title": "Mining Implicit Relevance Feedback from User Behavior for Web Question Answering",
            "authors": [
                "Linjun Shou",
                "Shining Bo",
                "Feixiang Cheng",
                "Ming Gong",
                "Jian Pei",
                "Daxin Jiang"
            ],
            "abstract": "Training and refreshing a web-scale Question Answering (QA) system for a\nmulti-lingual commercial search engine often requires a huge amount of training\nexamples. One principled idea is to mine implicit relevance feedback from user\nbehavior recorded in search engine logs. All previous works on mining implicit\nrelevance feedback target at relevance of web documents rather than passages.\nDue to several unique characteristics of QA tasks, the existing user behavior\nmodels for web documents cannot be applied to infer passage relevance. In this\npaper, we make the first study to explore the correlation between user behavior\nand passage relevance, and propose a novel approach for mining training data\nfor Web QA. We conduct extensive experiments on four test datasets and the\nresults show our approach significantly improves the accuracy of passage\nranking without extra human labeled data. In practice, this work has proved\neffective to substantially reduce the human labeling cost for the QA service in\na global commercial search engine, especially for languages with low resources.\nOur techniques have been deployed in multi-language services.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.07698v2",
            "date": "2020-06-13 18:53:22+00:00",
            "title": "Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya",
            "authors": [
                "Abrhalei Tela",
                "Abraham Woubie",
                "Ville Hautamaki"
            ],
            "abstract": "In recent years, transformer models have achieved great success in natural\nlanguage processing (NLP) tasks. Most of the current state-of-the-art NLP\nresults are achieved by using monolingual transformer models, where the model\nis pre-trained using a single language unlabelled text corpus. Then, the model\nis fine-tuned to the specific downstream task. However, the cost of\npre-training a new transformer model is high for most languages. In this work,\nwe propose a cost-effective transfer learning method to adopt a strong source\nlanguage model, trained from a large monolingual corpus to a low-resource\nlanguage. Thus, using XLNet language model, we demonstrate competitive\nperformance with mBERT and a pre-trained target language model on the\ncross-lingual sentiment (CLS) dataset and on a new sentiment analysis dataset\nfor low-resourced language Tigrinya. With only 10k examples of the given\nTigrinya sentiment analysis dataset, English XLNet has achieved 78.88% F1-Score\noutperforming BERT and mBERT by 10% and 7%, respectively. More interestingly,\nfine-tuning (English) XLNet model on the CLS dataset has promising results\ncompared to mBERT and even outperformed mBERT for one dataset of the Japanese\nlanguage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07732v1",
            "date": "2020-06-13 22:34:01+00:00",
            "title": "Through the Twitter Glass: Detecting Questions in Micro-Text",
            "authors": [
                "Kyle Dent",
                "Sharoda Paul"
            ],
            "abstract": "In a separate study, we were interested in understanding people's Q&A habits\non Twitter. Finding questions within Twitter turned out to be a difficult\nchallenge, so we considered applying some traditional NLP approaches to the\nproblem. On the one hand, Twitter is full of idiosyncrasies, which make\nprocessing it difficult. On the other, it is very restricted in length and\ntends to employ simple syntactic constructions, which could help the\nperformance of NLP processing. In order to find out the viability of NLP and\nTwitter, we built a pipeline of tools to work specifically with Twitter input\nfor the task of finding questions in tweets. This work is still preliminary,\nbut in this paper we discuss the techniques we used and the lessons we learned.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07849v1",
            "date": "2020-06-14 09:32:13+00:00",
            "title": "Pot, kettle: Nonliteral titles aren't (natural) science",
            "authors": [
                "Mike Thelwall"
            ],
            "abstract": "Researchers may be tempted to attract attention through poetic titles for\ntheir publications, but would this be mistaken in some fields? Whilst poetic\ntitles are known to be common in medicine, it is not clear whether the practice\nis widespread elsewhere. This article investigates the prevalence of poetic\nexpressions in journal article titles 1996-2019 in 3.3 million articles from\nall 27 Scopus broad fields. Expressions were identified by manually checking\nall phrases with at least 5 words that occurred at least 25 times, finding 149\nstock phrases, idioms, sayings, literary allusions, film names and song titles\nor lyrics. The expressions found are most common in the social sciences and the\nhumanities. They are also relatively common in medicine, but almost absent from\nengineering and the natural and formal sciences. The differences may reflect\nthe less hierarchical and more varied nature of the social sciences and\nhumanities, where interesting titles may attract an audience. In engineering,\nnatural science and formal science fields, authors should take extra care with\npoetic expressions, in case their choice is judged inappropriate. This includes\ninterdisciplinary research overlapping these areas. Conversely, reviewers of\ninterdisciplinary research involving the social sciences should be more\ntolerant of poetic license.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2006.07887v1",
            "date": "2020-06-14 12:48:43+00:00",
            "title": "Examining the Role of Mood Patterns in Predicting Self-Reported Depressive symptoms",
            "authors": [
                "Lucia Lushi Chen",
                "Walid Magdy",
                "Heather Whalley",
                "Maria Wolters"
            ],
            "abstract": "Depression is the leading cause of disability worldwide. Initial efforts to\ndetect depression signals from social media posts have shown promising results.\nGiven the high internal validity, results from such analyses are potentially\nbeneficial to clinical judgment. The existing models for automatic detection of\ndepressive symptoms learn proxy diagnostic signals from social media data, such\nas help-seeking behavior for mental health or medication names. However, in\nreality, individuals with depression typically experience depressed mood, loss\nof pleasure nearly in all the activities, feeling of worthlessness or guilt,\nand diminished ability to think. Therefore, a lot of the proxy signals used in\nthese models lack the theoretical underpinnings for depressive symptoms. It is\nalso reported that social media posts from many patients in the clinical\nsetting do not contain these signals. Based on this research gap, we propose to\nmonitor a type of signal that is well-established as a class of symptoms in\naffective disorders -- mood. The mood is an experience of feeling that can last\nfor hours, days, or even weeks. In this work, we attempt to enrich current\ntechnology for detecting symptoms of potential depression by constructing a\n'mood profile' for social media users.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.07926v2",
            "date": "2020-06-14 15:22:12+00:00",
            "title": "UWSpeech: Speech to Speech Translation for Unwritten Languages",
            "authors": [
                "Chen Zhang",
                "Xu Tan",
                "Yi Ren",
                "Tao Qin",
                "Kejun Zhang",
                "Tie-Yan Liu"
            ],
            "abstract": "Existing speech to speech translation systems heavily rely on the text of\ntarget language: they usually translate source language either to target text\nand then synthesize target speech from text, or directly to target speech with\ntarget text for auxiliary training. However, those methods cannot be applied to\nunwritten target languages, which have no written text or phoneme available. In\nthis paper, we develop a translation system for unwritten languages, named as\nUWSpeech, which converts target unwritten speech into discrete tokens with a\nconverter, and then translates source-language speech into target discrete\ntokens with a translator, and finally synthesizes target speech from target\ndiscrete tokens with an inverter. We propose a method called XL-VAE, which\nenhances vector quantized variational autoencoder (VQ-VAE) with cross-lingual\n(XL) speech recognition, to train the converter and inverter of UWSpeech\njointly. Experiments on Fisher Spanish-English conversation translation dataset\nshow that UWSpeech outperforms direct translation and VQ-VAE baseline by about\n16 and 10 BLEU points respectively, which demonstrate the advantages and\npotentials of UWSpeech.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2006.08281v1",
            "date": "2020-06-15 11:07:52+00:00",
            "title": "On the Multi-Property Extraction and Beyond",
            "authors": [
                "Tomasz Dwojak",
                "Micha\u0142 Pietruszka",
                "\u0141ukasz Borchmann",
                "Filip Grali\u0144ski",
                "Jakub Ch\u0142\u0119dowski"
            ],
            "abstract": "In this paper, we investigate the Dual-source Transformer architecture on the\nWikiReading information extraction and machine reading comprehension dataset.\nThe proposed model outperforms the current state-of-the-art by a large margin.\nNext, we introduce WikiReading Recycled - a newly developed public dataset,\nsupporting the task of multiple property extraction. It keeps the spirit of the\noriginal WikiReading but does not inherit the identified disadvantages of its\npredecessor.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08334v1",
            "date": "2020-06-06 06:51:11+00:00",
            "title": "StackOverflow vs Kaggle: A Study of Developer Discussions About Data Science",
            "authors": [
                "David Hin"
            ],
            "abstract": "Software developers are increasingly required to understand fundamental Data\nscience (DS) concepts. Recently, the presence of machine learning (ML) and deep\nlearning (DL) has dramatically increased in the development of user\napplications, whether they are leveraged through frameworks or implemented from\nscratch. These topics attract much discussion on online platforms. This paper\nconducts large-scale qualitative and quantitative experiments to study the\ncharacteristics of 197836 posts from StackOverflow and Kaggle. Latent Dirichlet\nAllocation topic modelling is used to extract twenty-four DS discussion topics.\nThe main findings include that TensorFlow-related topics were most prevalent in\nStackOverflow, while meta discussion topics were the prevalent ones on Kaggle.\nStackOverflow tends to include lower-level troubleshooting, while Kaggle\nfocuses on practicality and optimising leaderboard performance. In addition,\nacross both communities, DS discussion is increasing at a dramatic rate. While\nTensorFlow discussion on StackOverflow is slowing, interest in Keras is rising.\nFinally, ensemble algorithms are the most mentioned ML/DL algorithms in Kaggle\nbut are rarely discussed on StackOverflow. These findings can help educators\nand researchers to more effectively tailor and prioritise efforts in\nresearching and communicating DS concepts towards different developer\ncommunities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08338v1",
            "date": "2020-06-05 04:39:34+00:00",
            "title": "DeepVar: An End-to-End Deep Learning Approach for Genomic Variant Recognition in Biomedical Literature",
            "authors": [
                "Chaoran Cheng",
                "Fei Tan",
                "Zhi Wei"
            ],
            "abstract": "We consider the problem of Named Entity Recognition (NER) on biomedical\nscientific literature, and more specifically the genomic variants recognition\nin this work. Significant success has been achieved for NER on canonical tasks\nin recent years where large data sets are generally available. However, it\nremains a challenging problem on many domain-specific areas, especially the\ndomains where only small gold annotations can be obtained. In addition, genomic\nvariant entities exhibit diverse linguistic heterogeneity, differing much from\nthose that have been characterized in existing canonical NER tasks. The\nstate-of-the-art machine learning approaches in such tasks heavily rely on\narduous feature engineering to characterize those unique patterns. In this\nwork, we present the first successful end-to-end deep learning approach to\nbridge the gap between generic NER algorithms and low-resource applications\nthrough genomic variants recognition. Our proposed model can result in\npromising performance without any hand-crafted features or post-processing\nrules. Our extensive experiments and results may shed light on other similar\nlow-resource NER applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08339v1",
            "date": "2020-06-02 06:53:21+00:00",
            "title": "Graph-Stega: Semantic Controllable Steganographic Text Generation Guided by Knowledge Graph",
            "authors": [
                "Zhongliang Yang",
                "Baitao Gong",
                "Yamin Li",
                "Jinshuai Yang",
                "Zhiwen Hu",
                "Yongfeng Huang"
            ],
            "abstract": "Most of the existing text generative steganographic methods are based on\ncoding the conditional probability distribution of each word during the\ngeneration process, and then selecting specific words according to the secret\ninformation, so as to achieve information hiding. Such methods have their\nlimitations which may bring potential security risks. Firstly, with the\nincrease of embedding rate, these models will choose words with lower\nconditional probability, which will reduce the quality of the generated\nsteganographic texts; secondly, they can not control the semantic expression of\nthe final generated steganographic text. This paper proposes a new text\ngenerative steganography method which is quietly different from the existing\nmodels. We use a Knowledge Graph (KG) to guide the generation of steganographic\nsentences. On the one hand, we hide the secret information by coding the path\nin the knowledge graph, but not the conditional probability of each generated\nword; on the other hand, we can control the semantic expression of the\ngenerated steganographic text to a certain extent. The experimental results\nshow that the proposed model can guarantee both the quality of the generated\ntext and its semantic expression, which is a supplement and improvement to the\ncurrent text generation steganography.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08367v2",
            "date": "2020-06-09 19:17:30+00:00",
            "title": "Tamil Vowel Recognition With Augmented MNIST-like Data Set",
            "authors": [
                "Muthiah Annamalai"
            ],
            "abstract": "We report generation of a MNIST [4] compatible data set [1] for Tamil vowels\nto enable building a classification DNN or other such ML/AI deep learning [2]\nmodels for Tamil OCR/Handwriting applications. We report the capability of the\n60,000 grayscale, 28x28 pixel dataset to build a 92% accuracy (training) and\n82% cross-validation 4-layer CNN, with 100,000+ parameters, in TensorFlow. We\nalso report a top-1 classification accuracy of 70% and top-2 classification\naccuracy of 92% on handwritten vowels showing, for the same network.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.08506v2",
            "date": "2020-06-15 16:04:16+00:00",
            "title": "Regularized Forward-Backward Decoder for Attention Models",
            "authors": [
                "Tobias Watzel",
                "Ludwig K\u00fcrzinger",
                "Lujun Li",
                "Gerhard Rigoll"
            ],
            "abstract": "Nowadays, attention models are one of the popular candidates for speech\nrecognition. So far, many studies mainly focus on the encoder structure or the\nattention module to enhance the performance of these models. However, mostly\nignore the decoder. In this paper, we propose a novel regularization technique\nincorporating a second decoder during the training phase. This decoder is\noptimized on time-reversed target labels beforehand and supports the standard\ndecoder during training by adding knowledge from future context. Since it is\nonly added during training, we are not changing the basic structure of the\nnetwork or adding complexity during decoding. We evaluate our approach on the\nsmaller TEDLIUMv2 and the larger LibriSpeech dataset, achieving consistent\nimprovements on both of them.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2006.08779v3",
            "date": "2020-06-15 21:31:05+00:00",
            "title": "Automatic Validation of Textual Attribute Values in E-commerce Catalog by Learning with Limited Labeled Data",
            "authors": [
                "Yaqing Wang",
                "Yifan Ethan Xu",
                "Xian Li",
                "Xin Luna Dong",
                "Jing Gao"
            ],
            "abstract": "Product catalogs are valuable resources for eCommerce website. In the\ncatalog, a product is associated with multiple attributes whose values are\nshort texts, such as product name, brand, functionality and flavor. Usually\nindividual retailers self-report these key values, and thus the catalog\ninformation unavoidably contains noisy facts. Although existing deep neural\nnetwork models have shown success in conducting cross-checking between two\npieces of texts, their success has to be dependent upon a large set of quality\nlabeled data, which are hard to obtain in this validation task: products span a\nvariety of categories. To address the aforementioned challenges, we propose a\nnovel meta-learning latent variable approach, called MetaBridge, which can\nlearn transferable knowledge from a subset of categories with limited labeled\ndata and capture the uncertainty of never-seen categories with unlabeled data.\nMore specifically, we make the following contributions. (1) We formalize the\nproblem of validating the textual attribute values of products from a variety\nof categories as a natural language inference task in the few-shot learning\nsetting, and propose a meta-learning latent variable model to jointly process\nthe signals obtained from product profiles and textual attribute values. (2) We\npropose to integrate meta learning and latent variable in a unified model to\neffectively capture the uncertainty of various categories. (3) We propose a\nnovel objective function based on latent variable model in the few-shot\nlearning setting, which ensures distribution consistency between unlabeled and\nlabeled data and prevents overfitting by sampling from the learned\ndistribution. Extensive experiments on real eCommerce datasets from hundreds of\ncategories demonstrate the effectiveness of MetaBridge on textual attribute\nvalidation and its outstanding performance compared with state-of-the-art\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.08952v1",
            "date": "2020-06-16 07:03:28+00:00",
            "title": "Manipulating emotions for ground truth emotion analysis",
            "authors": [
                "Bennett Kleinberg"
            ],
            "abstract": "Text data are being used as a lens through which human cognition can be\nstudied at a large scale. Methods like emotion analysis are now in the standard\ntoolkit of computational social scientists but typically rely on third-person\nannotation with unknown validity. As an alternative, this paper introduces\nonline emotion induction techniques from experimental behavioural research as a\nmethod for text-based emotion analysis. Text data were collected from\nparticipants who were randomly allocated to a happy, neutral or sad condition.\nThe findings support the mood induction procedure. We then examined how well\nlexicon approaches can retrieve the induced emotion. All approaches resulted in\nstatistical differences between the true emotion conditions. Overall, only up\nto one-third of the variance in emotion was captured by text-based\nmeasurements. Pretrained classifiers performed poorly on detecting true\nemotions. The paper concludes with limitations and suggestions for future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09075v1",
            "date": "2020-06-16 11:14:06+00:00",
            "title": "PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models",
            "authors": [
                "Eyal Ben-David",
                "Carmel Rabinovitz",
                "Roi Reichart"
            ],
            "abstract": "Pivot-based neural representation models have lead to significant progress in\ndomain adaptation for NLP. However, previous works that follow this approach\nutilize only labeled data from the source domain and unlabeled data from the\nsource and target domains, but neglect to incorporate massive unlabeled corpora\nthat are not necessarily drawn from these domains. To alleviate this, we\npropose PERL: A representation learning model that extends contextualized word\nembedding models such as BERT with pivot-based fine-tuning. PERL outperforms\nstrong baselines across 22 sentiment classification domain adaptation setups,\nimproves in-domain model performance, yields effective reduced-size models and\nincreases model stability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09213v2",
            "date": "2020-06-15 00:50:41+00:00",
            "title": "A Hybrid Natural Language Generation System Integrating Rules and Deep Learning Algorithms",
            "authors": [
                "Wei Wei",
                "Bei Zhou",
                "Georgios Leontidis"
            ],
            "abstract": "This paper proposes an enhanced natural language generation system combining\nthe merits of both rule-based approaches and modern deep learning algorithms,\nboosting its performance to the extent where the generated textual content is\ncapable of exhibiting agile human-writing styles and the content logic of which\nis highly controllable. We also come up with a novel approach called HMCU to\nmeasure the performance of the natural language processing comprehensively and\nprecisely.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09217v1",
            "date": "2020-06-14 04:27:12+00:00",
            "title": "FFR v1.1: Fon-French Neural Machine Translation",
            "authors": [
                "Bonaventure F. P. Dossou",
                "Chris C. Emezue"
            ],
            "abstract": "All over the world and especially in Africa, researchers are putting efforts\ninto building Neural Machine Translation (NMT) systems to help tackle the\nlanguage barriers in Africa, a continent of over 2000 different languages.\nHowever, the low-resourceness, diacritical, and tonal complexities of African\nlanguages are major issues being faced. The FFR project is a major step towards\ncreating a robust translation model from Fon, a very low-resource and tonal\nlanguage, to French, for research and public use. In this paper, we introduce\nFFR Dataset, a corpus of Fon-to-French translations, describe the diacritical\nencoding process, and introduce our FFR v1.1 model, trained on the dataset. The\ndataset and model are made publicly available at https://github.com/\nbonaventuredossou/ffr-v1, to promote collaboration and reproducibility.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09462v1",
            "date": "2020-06-16 19:13:21+00:00",
            "title": "Selective Question Answering under Domain Shift",
            "authors": [
                "Amita Kamath",
                "Robin Jia",
                "Percy Liang"
            ],
            "abstract": "To avoid giving wrong answers, question answering (QA) models need to know\nwhen to abstain from answering. Moreover, users often ask questions that\ndiverge from the model's training data, making errors more likely and thus\nabstention more critical. In this work, we propose the setting of selective\nquestion answering under domain shift, in which a QA model is tested on a\nmixture of in-domain and out-of-domain data, and must answer (i.e., not abstain\non) as many questions as possible while maintaining high accuracy. Abstention\npolicies based solely on the model's softmax probabilities fare poorly, since\nmodels are overconfident on out-of-domain inputs. Instead, we train a\ncalibrator to identify inputs on which the QA model errs, and abstain when it\npredicts an error is likely. Crucially, the calibrator benefits from observing\nthe model's behavior on out-of-domain data, even if from a different domain\nthan the test data. We combine this method with a SQuAD-trained QA model and\nevaluate on mixtures of SQuAD and five other QA datasets. Our method answers\n56% of questions while maintaining 80% accuracy; in contrast, directly using\nthe model's probabilities only answers 48% at 80% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09479v1",
            "date": "2020-06-16 19:43:30+00:00",
            "title": "EPIE Dataset: A Corpus For Possible Idiomatic Expressions",
            "authors": [
                "Prateek Saxena",
                "Soma Paul"
            ],
            "abstract": "Idiomatic expressions have always been a bottleneck for language\ncomprehension and natural language understanding, specifically for tasks like\nMachine Translation(MT). MT systems predominantly produce literal translations\nof idiomatic expressions as they do not exhibit generic and linguistically\ndeterministic patterns which can be exploited for comprehension of the\nnon-compositional meaning of the expressions. These expressions occur in\nparallel corpora used for training, but due to the comparatively high\noccurrences of the constituent words of idiomatic expressions in literal\ncontext, the idiomatic meaning gets overpowered by the compositional meaning of\nthe expression. State of the art Metaphor Detection Systems are able to detect\nnon-compositional usage at word level but miss out on idiosyncratic phrasal\nidiomatic expressions. This creates a dire need for a dataset with a wider\ncoverage and higher occurrence of commonly occurring idiomatic expressions, the\nspans of which can be used for Metaphor Detection. With this in mind, we\npresent our English Possible Idiomatic Expressions(EPIE) corpus containing\n25206 sentences labelled with lexical instances of 717 idiomatic expressions.\nThese spans also cover literal usages for the given set of idiomatic\nexpressions. We also present the utility of our dataset by using it to train a\nsequence labelling module and testing on three independent datasets with high\naccuracy, precision and recall scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09526v2",
            "date": "2020-06-16 21:30:51+00:00",
            "title": "Cross-lingual Retrieval for Iterative Self-Supervised Training",
            "authors": [
                "Chau Tran",
                "Yuqing Tang",
                "Xian Li",
                "Jiatao Gu"
            ],
            "abstract": "Recent studies have demonstrated the cross-lingual alignment ability of\nmultilingual pretrained language models. In this work, we found that the\ncross-lingual alignment can be further improved by training seq2seq models on\nsentence pairs mined using their own encoder outputs. We utilized these\nfindings to develop a new approach -- cross-lingual retrieval for iterative\nself-supervised training (CRISS), where mining and training processes are\napplied iteratively, improving cross-lingual alignment and translation ability\nat the same time. Using this method, we achieved state-of-the-art unsupervised\nmachine translation results on 9 language directions with an average\nimprovement of 2.4 BLEU, and on the Tatoeba sentence retrieval task in the\nXTREME benchmark on 16 languages with an average improvement of 21.5% in\nabsolute accuracy. Furthermore, CRISS also brings an additional 1.8 BLEU\nimprovement on average compared to mBART, when finetuned on supervised machine\ntranslation downstream tasks.",
            "journal ref": "NeurIPS 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09610v1",
            "date": "2020-06-17 02:32:36+00:00",
            "title": "Canonicalizing Open Knowledge Bases with Multi-Layered Meta-Graph Neural Network",
            "authors": [
                "Tianwen Jiang",
                "Tong Zhao",
                "Bing Qin",
                "Ting Liu",
                "Nitesh V. Chawla",
                "Meng Jiang"
            ],
            "abstract": "Noun phrases and relational phrases in Open Knowledge Bases are often not\ncanonical, leading to redundant and ambiguous facts. In this work, we integrate\nstructural information (from which tuple, which sentence) and semantic\ninformation (semantic similarity) to do the canonicalization. We represent the\ntwo types of information as a multi-layered graph: the structural information\nforms the links across the sentence, relational phrase, and noun phrase layers;\nthe semantic information forms weighted intra-layer links for each layer. We\npropose a graph neural network model to aggregate the representations of noun\nphrases and relational phrases through the multi-layered meta-graph structure.\nExperiments show that our model outperforms existing approaches on a public\ndatasets in general domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09873v1",
            "date": "2020-06-17 13:55:31+00:00",
            "title": "An Exploratory Study of Argumentative Writing by Young Students: A Transformer-based Approach",
            "authors": [
                "Debanjan Ghosh",
                "Beata Beigman Klebanov",
                "Yi Song"
            ],
            "abstract": "We present a computational exploration of argument critique writing by young\nstudents. Middle school students were asked to criticize an argument presented\nin the prompt, focusing on identifying and explaining the reasoning flaws. This\ntask resembles an established college-level argument critique task. Lexical and\ndiscourse features that utilize detailed domain knowledge to identify critiques\nexist for the college task but do not perform well on the young students data.\nInstead, transformer-based architecture (e.g., BERT) fine-tuned on a large\ncorpus of critique essays from the college task performs much better (over 20%\nimprovement in F1 score). Analysis of the performance of various configurations\nof the system suggests that while children's writing does not exhibit the\nstandard discourse structure of an argumentative essay, it does share basic\nlocal sequential structures with the more mature writers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.09891v1",
            "date": "2020-06-17 14:17:58+00:00",
            "title": "Fine-grained Sentiment Controlled Text Generation",
            "authors": [
                "Bidisha Samanta",
                "Mohit Agarwal",
                "Niloy Ganguly"
            ],
            "abstract": "Controlled text generation techniques aim to regulate specific attributes\n(e.g. sentiment) while preserving the attribute independent content. The\nstate-of-the-art approaches model the specified attribute as a structured or\ndiscrete representation while making the content representation independent of\nit to achieve a better control. However, disentangling the text representation\ninto separate latent spaces overlooks complex dependencies between content and\nattribute, leading to generation of poorly constructed and not so meaningful\nsentences. Moreover, such an approach fails to provide a finer control on the\ndegree of attribute change. To address these problems of controlled text\ngeneration, in this paper, we propose DE-VAE, a hierarchical framework which\ncaptures both information enriched entangled representation and attribute\nspecific disentangled representation in different hierarchies. DE-VAE achieves\nbetter control of sentiment as an attribute while preserving the content by\nlearning a suitable lossless transformation network from the disentangled\nsentiment space to the desired entangled representation. Through feature\nsupervision on a single dimension of the disentangled representation, DE-VAE\nmaps the variation of sentiment to a continuous space which helps in smoothly\nregulating sentiment from positive to negative and vice versa. Detailed\nexperiments on three publicly available review datasets show the superiority of\nDE-VAE over recent state-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10145v1",
            "date": "2020-06-17 20:41:57+00:00",
            "title": "Using Sentiment Information for Preemptive Detection of Toxic Comments in Online Conversations",
            "authors": [
                "\u00c9loi Brassard-Gourdeau",
                "Richard Khoury"
            ],
            "abstract": "The challenge of automatic detection of toxic comments online has been the\nsubject of a lot of research recently, but the focus has been mostly on\ndetecting it in individual messages after they have been posted. Some authors\nhave tried to predict if a conversation will derail into toxicity using the\nfeatures of the first few messages. In this paper, we combine that approach\nwith previous work on toxicity detection using sentiment information, and show\nhow the sentiments expressed in the first messages of a conversation can help\npredict upcoming toxicity. Our results show that adding sentiment features does\nhelp improve the accuracy of toxicity prediction, and also allow us to make\nimportant observations on the general task of preemptive toxicity detection.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.10217v1",
            "date": "2020-06-18 00:32:53+00:00",
            "title": "STEAM: Self-Supervised Taxonomy Expansion with Mini-Paths",
            "authors": [
                "Yue Yu",
                "Yinghao Li",
                "Jiaming Shen",
                "Hao Feng",
                "Jimeng Sun",
                "Chao Zhang"
            ],
            "abstract": "Taxonomies are important knowledge ontologies that underpin numerous\napplications on a daily basis, but many taxonomies used in practice suffer from\nthe low coverage issue. We study the taxonomy expansion problem, which aims to\nexpand existing taxonomies with new concept terms. We propose a self-supervised\ntaxonomy expansion model named STEAM, which leverages natural supervision in\nthe existing taxonomy for expansion. To generate natural self-supervision\nsignals, STEAM samples mini-paths from the existing taxonomy, and formulates a\nnode attachment prediction task between anchor mini-paths and query terms. To\nsolve the node attachment task, it learns feature representations for\nquery-anchor pairs from multiple views and performs multi-view co-training for\nprediction. Extensive experiments show that STEAM outperforms state-of-the-art\nmethods for taxonomy expansion by 11.6\\% in accuracy and 7.0\\% in mean\nreciprocal rank on three public benchmarks. The implementation of STEAM can be\nfound at \\url{https://github.com/yueyu1030/STEAM}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10334v1",
            "date": "2020-06-18 07:42:45+00:00",
            "title": "Extraction and Evaluation of Formulaic Expressions Used in Scholarly Papers",
            "authors": [
                "Kenichi Iwatsuki",
                "Florian Boudin",
                "Akiko Aizawa"
            ],
            "abstract": "Formulaic expressions, such as 'in this paper we propose', are helpful for\nauthors of scholarly papers because they convey communicative functions; in the\nabove, it is showing the aim of this paper'. Thus, resources of formulaic\nexpressions, such as a dictionary, that could be looked up easily would be\nuseful. However, forms of formulaic expressions can often vary to a great\nextent. For example, 'in this paper we propose', 'in this study we propose' and\n'in this paper we propose a new method to' are all regarded as formulaic\nexpressions. Such a diversity of spans and forms causes problems in both\nextraction and evaluation of formulaic expressions. In this paper, we propose a\nnew approach that is robust to variation of spans and forms of formulaic\nexpressions. Our approach regards a sentence as consisting of a formulaic part\nand non-formulaic part. Then, instead of trying to extract formulaic\nexpressions from a whole corpus, by extracting them from each sentence,\ndifferent forms can be dealt with at once. Based on this formulation, to avoid\nthe diversity problem, we propose evaluating extraction methods by how much\nthey convey specific communicative functions rather than by comparing extracted\nexpressions to an existing lexicon. We also propose a new extraction method\nthat utilises named entities and dependency structures to remove the\nnon-formulaic part from a sentence. Experimental results show that the proposed\nextraction method achieved the best performance compared to other existing\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.10627v2",
            "date": "2020-06-18 15:50:57+00:00",
            "title": "Compositional Generalization by Learning Analytical Expressions",
            "authors": [
                "Qian Liu",
                "Shengnan An",
                "Jian-Guang Lou",
                "Bei Chen",
                "Zeqi Lin",
                "Yan Gao",
                "Bin Zhou",
                "Nanning Zheng",
                "Dongmei Zhang"
            ],
            "abstract": "Compositional generalization is a basic and essential intellective capability\nof human beings, which allows us to recombine known parts readily. However,\nexisting neural network based models have been proven to be extremely deficient\nin such a capability. Inspired by work in cognition which argues\ncompositionality can be captured by variable slots with symbolic functions, we\npresent a refreshing view that connects a memory-augmented neural model with\nanalytical expressions, to achieve compositional generalization. Our model\nconsists of two cooperative neural modules, Composer and Solver, fitting well\nwith the cognitive argument while being able to be trained in an end-to-end\nmanner via a hierarchical reinforcement learning algorithm. Experiments on the\nwell-known benchmark SCAN demonstrate that our model seizes a great ability of\ncompositional generalization, solving all challenges addressed by previous\nworks with 100% accuracies.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2006.11159v1",
            "date": "2020-06-19 14:43:12+00:00",
            "title": "Graphs with Multiple Sources per Vertex",
            "authors": [
                "Martin van Harmelen",
                "Jonas Groschwitz"
            ],
            "abstract": "Several attempts have been made at constructing Abstract Meaning\nRepresentations (AMRs) compositionally, and recently the idea of using s-graphs\nwith the HR-algebra (Koller, 2015) has been simplified to reduce the number of\noptions when parsing (Groschwitz et al., 2017). This apply-modify algebra\n(AM-algebra) is a linguistically plausible graph algebra with two classes of\noperations, both of rank two: the apply operation is used to combine a\npredicate with its argument; the modify operation is used to modify a\npredicate. While the AM-algebra correctly handles relative clauses and complex\ncases of coordination, it cannot parse reflexive sentences like: \"The raven\nwashes herself.\" To facilitate processing of such reflexive sentences, this\npaper proposes to change the definition of s-graphs underlying the AM-algebra\nto allow vertices with multiple sources, and additionally proposes an adaption\nto the type system of the algebra to correctly handle such vertices.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/2006.11511v2",
            "date": "2020-06-20 07:35:22+00:00",
            "title": "Improving Query Safety at Pinterest",
            "authors": [
                "Abhijit Mahabal",
                "Yinrui Li",
                "Rajat Raina",
                "Daniel Sun",
                "Revati Mahajan",
                "Jure Leskovec"
            ],
            "abstract": "Query recommendations in search engines is a double edged sword, with\nundeniable benefits but potential of harm. Identifying unsafe queries is\nnecessary to protect users from inappropriate query suggestions. However,\nidentifying these is non-trivial because of the linguistic diversity resulting\nfrom large vocabularies, social-group-specific slang and typos, and because the\ninappropriateness of a term depends on the context. Here we formulate the\nproblem as query-set expansion, where we are given a small and potentially\nbiased seed set and the aim is to identify a diverse set of semantically\nrelated queries. We present PinSets, a system for query-set expansion, which\napplies a simple yet powerful mechanism to search user sessions, expanding a\ntiny seed set into thousands of related queries at nearly perfect precision,\ndeep into the tail, along with explanations that are easy to interpret. PinSets\nowes its high quality expansion to using a hybrid of textual and behavioral\ntechniques (i.e., treating queries both as compositional and as black boxes).\nExperiments show that, for the domain of drugs-related queries, PinSets expands\n20 seed queries into 15,670 positive training examples at over 99\\% precision.\nThe generated expansions have diverse vocabulary and correctly handles words\nwith ambiguous safety. PinSets decreased unsafe query suggestions at Pinterest\nby 90\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11512v1",
            "date": "2020-06-20 07:36:06+00:00",
            "title": "Sarcasm Detection in Tweets with BERT and GloVe Embeddings",
            "authors": [
                "Akshay Khatri",
                "Pranav P",
                "Dr. Anand Kumar M"
            ],
            "abstract": "Sarcasm is a form of communication in whichthe person states opposite of what\nhe actually means. It is ambiguous in nature. In this paper, we propose using\nmachine learning techniques with BERT and GloVe embeddings to detect sarcasm in\ntweets. The dataset is preprocessed before extracting the embeddings. The\nproposed model also uses the context in which the user is reacting to along\nwith his actual response.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11534v3",
            "date": "2020-06-20 10:02:20+00:00",
            "title": "IQA: Interactive Query Construction in Semantic Question Answering Systems",
            "authors": [
                "Hamid Zafar",
                "Mohnish Dubey",
                "Jens Lehmann",
                "Elena Demidova"
            ],
            "abstract": "Semantic Question Answering (SQA) systems automatically interpret user\nquestions expressed in a natural language in terms of semantic queries. This\nprocess involves uncertainty, such that the resulting queries do not always\naccurately match the user intent, especially for more complex and less common\nquestions. In this article, we aim to empower users in guiding SQA systems\ntowards the intended semantic queries through interaction. We introduce IQA -\nan interaction scheme for SQA pipelines. This scheme facilitates seamless\nintegration of user feedback in the question answering process and relies on\nOption Gain - a novel metric that enables efficient and intuitive user\ninteraction. Our evaluation shows that using the proposed scheme, even a small\nnumber of user interactions can lead to significant improvements in the\nperformance of SQA systems.",
            "journal ref": "Journal of Web Semantics Volume 64, October 2020, 100586",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.11548v1",
            "date": "2020-06-20 11:09:04+00:00",
            "title": "Named Entity Extraction with Finite State Transducers",
            "authors": [
                "Diego Alexander Hu\u00e9rfano Villalba",
                "Elizabeth Le\u00f3n Guzm\u00e1n"
            ],
            "abstract": "We describe a named entity tagging system that requires minimal linguistic\nknowledge and can be applied to more target languages without substantial\nchanges. The system is based on the ideas of the Brill's tagger which makes it\nreally simple. Using supervised machine learning, we construct a series of\nautomatons (or transducers) in order to tag a given text. The final model is\ncomposed entirely of automatons and it requires a lineal time for tagging. It\nwas tested with the Spanish data set provided in the CoNLL-$2002$ attaining an\noverall $F_{\\beta = 1}$ measure of $60\\%.$ Also, we present an algorithm for\nthe construction of the final transducer used to encode all the learned\ncontextual rules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11578v1",
            "date": "2020-06-20 13:57:55+00:00",
            "title": "Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy",
            "authors": [
                "Antonio H. O. Fonseca",
                "David van Dijk"
            ],
            "abstract": "Word translation is an integral part of language translation. In machine\ntranslation, each language is considered a domain with its own word embedding.\nThe alignment between word embeddings allows linking semantically equivalent\nwords in multilingual contexts. Moreover, it offers a way to infer\ncross-lingual meaning for words without a direct translation. Current methods\nfor word embedding alignment are either supervised, i.e. they require known\nword pairs, or learn a cross-domain transformation on fixed embeddings in an\nunsupervised way. Here we propose an end-to-end approach for word embedding\nalignment that does not require known word pairs. Our method, termed Word\nAlignment through MMD (WAM), learns embeddings that are aligned during sentence\ntranslation training using a localized Maximum Mean Discrepancy (MMD)\nconstraint between the embeddings. We show that our method not only\nout-performs unsupervised methods, but also supervised methods that train on\nknown word translations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11627v1",
            "date": "2020-06-20 18:01:16+00:00",
            "title": "Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble",
            "authors": [
                "Yi Zhou",
                "Xiaoqing Zheng",
                "Cho-Jui Hsieh",
                "Kai-wei Chang",
                "Xuanjing Huang"
            ],
            "abstract": "Despite neural networks have achieved prominent performance on many natural\nlanguage processing (NLP) tasks, they are vulnerable to adversarial examples.\nIn this paper, we propose Dirichlet Neighborhood Ensemble (DNE), a randomized\nsmoothing method for training a robust model to defense substitution-based\nattacks. During training, DNE forms virtual sentences by sampling embedding\nvectors for each word in an input sentence from a convex hull spanned by the\nword and its synonyms, and it augments them with the training data. In such a\nway, the model is robust to adversarial attacks while maintaining the\nperformance on the original clean data. DNE is agnostic to the network\narchitectures and scales to large models for NLP applications. We demonstrate\nthrough extensive experimentation that our method consistently outperforms\nrecently proposed defense methods by a significant margin across different\nnetwork architectures and multiple data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11642v1",
            "date": "2020-06-20 20:03:07+00:00",
            "title": "MDR Cluster-Debias: A Nonlinear WordEmbedding Debiasing Pipeline",
            "authors": [
                "Yuhao Du",
                "Kenneth Joseph"
            ],
            "abstract": "Existing methods for debiasing word embeddings often do so only\nsuperficially, in that words that are stereotypically associated with, e.g., a\nparticular gender in the original embedding space can still be clustered\ntogether in the debiased space. However, there has yet to be a study that\nexplores why this residual clustering exists, and how it might be addressed.\nThe present work fills this gap. We identify two potential reasons for which\nresidual bias exists and develop a new pipeline, MDR Cluster-Debias, to\nmitigate this bias. We explore the strengths and weaknesses of our method,\nfinding that it significantly outperforms other existing debiasing approaches\non a variety of upstream bias tests but achieves limited improvement on\ndecreasing gender bias in a downstream task. This indicates that word\nembeddings encode gender bias in still other ways, not necessarily captured by\nupstream tests.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11719v1",
            "date": "2020-06-21 05:59:34+00:00",
            "title": "Match$^2$: A Matching over Matching Model for Similar Question Identification",
            "authors": [
                "Zizhen Wang",
                "Yixing Fan",
                "Jiafeng Guo",
                "Liu Yang",
                "Ruqing Zhang",
                "Yanyan Lan",
                "Xueqi Cheng",
                "Hui Jiang",
                "Xiaozhao Wang"
            ],
            "abstract": "Community Question Answering (CQA) has become a primary means for people to\nacquire knowledge, where people are free to ask questions or submit answers. To\nenhance the efficiency of the service, similar question identification becomes\na core task in CQA which aims to find a similar question from the archived\nrepository whenever a new question is asked. However, it has long been a\nchallenge to properly measure the similarity between two questions due to the\ninherent variation of natural language, i.e., there could be different ways to\nask a same question or different questions sharing similar expressions. To\nalleviate this problem, it is natural to involve the existing answers for the\nenrichment of the archived questions. Traditional methods typically take a\none-side usage, which leverages the answer as some expanded representation of\nthe corresponding question. Unfortunately, this may introduce unexpected noises\ninto the similarity computation since answers are often long and diverse,\nleading to inferior performance. In this work, we propose a two-side usage,\nwhich leverages the answer as a bridge of the two questions. The key idea is\nbased on our observation that similar questions could be addressed by similar\nparts of the answer while different questions may not. In other words, we can\ncompare the matching patterns of the two questions over the same answer to\nmeasure their similarity. In this way, we propose a novel matching over\nmatching model, namely Match$^2$, which compares the matching patterns between\ntwo question-answer pairs for similar question identification. Empirical\nexperiments on two benchmark datasets demonstrate that our model can\nsignificantly outperform previous state-of-the-art methods on the similar\nquestion identification task.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.11807v1",
            "date": "2020-06-21 14:10:47+00:00",
            "title": "Improving Image Captioning with Better Use of Captions",
            "authors": [
                "Zhan Shi",
                "Xu Zhou",
                "Xipeng Qiu",
                "Xiaodan Zhu"
            ],
            "abstract": "Image captioning is a multimodal problem that has drawn extensive attention\nin both the natural language processing and computer vision community. In this\npaper, we present a novel image captioning architecture to better explore\nsemantics available in captions and leverage that to enhance both image\nrepresentation and caption generation. Our models first construct\ncaption-guided visual relationship graphs that introduce beneficial inductive\nbias using weakly supervised multi-instance learning. The representation is\nthen enhanced with neighbouring and contextual nodes with their textual and\nvisual features. During generation, the model further incorporates visual\nrelationships using multi-task learning for jointly predicting word and\nobject/predicate tag sequences. We perform extensive experiments on the MSCOCO\ndataset, showing that the proposed framework significantly outperforms the\nbaselines, resulting in the state-of-the-art performance under a wide range of\nevaluation metrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.11830v1",
            "date": "2020-06-21 15:41:58+00:00",
            "title": "The NYU-CUBoulder Systems for SIGMORPHON 2020 Task 0 and Task 2",
            "authors": [
                "Assaf Singer",
                "Katharina Kann"
            ],
            "abstract": "We describe the NYU-CUBoulder systems for the SIGMORPHON 2020 Task 0 on\ntypologically diverse morphological inflection and Task 2 on unsupervised\nmorphological paradigm completion. The former consists of generating\nmorphological inflections from a lemma and a set of morphosyntactic features\ndescribing the target form. The latter requires generating entire paradigms for\na set of given lemmas from raw text alone. We model morphological inflection as\na sequence-to-sequence problem, where the input is the sequence of the lemma's\ncharacters with morphological tags, and the output is the sequence of the\ninflected form's characters. First, we apply a transformer model to the task.\nSecond, as inflected forms share most characters with the lemma, we further\npropose a pointer-generator transformer model to allow easy copying of input\ncharacters. Our best performing system for Task 0 is placed 6th out of 23\nsystems. We further use our inflection systems as subcomponents of approaches\nfor Task 2. Our best performing system for Task 2 is the 2nd best out of 7\nsubmissions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.11991v1",
            "date": "2020-06-22 03:39:00+00:00",
            "title": "Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage",
            "authors": [
                "Shijing Si",
                "Rui Wang",
                "Jedrek Wosik",
                "Hao Zhang",
                "David Dov",
                "Guoyin Wang",
                "Ricardo Henao",
                "Lawrence Carin"
            ],
            "abstract": "Small and imbalanced datasets commonly seen in healthcare represent a\nchallenge when training classifiers based on deep learning models. So\nmotivated, we propose a novel framework based on BioBERT (Bidirectional Encoder\nRepresentations from Transformers forBiomedical TextMining). Specifically, (i)\nwe introduce Label Embeddings for Self-Attention in each layer of BERT, which\nwe call LESA-BERT, and (ii) by distilling LESA-BERT to smaller variants, we aim\nto reduce overfitting and model size when working on small datasets. As an\napplication, our framework is utilized to build a model for patient portal\nmessage triage that classifies the urgency of a message into three categories:\nnon-urgent, medium and urgent. Experiments demonstrate that our approach can\noutperform several strong baseline classifiers by a significant margin of 4.3%\nin terms of macro F1 score. The code for this project is publicly available at\n\\url{https://github.com/shijing001/text_classifiers}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12185v2",
            "date": "2020-06-22 12:30:42+00:00",
            "title": "Examination of Community Sentiment Dynamics due to COVID-19 Pandemic: A Case Study from A State in Australia",
            "authors": [
                "Jianlong Zhou",
                "Shuiqiao Yang",
                "Chun Xiao",
                "Fang Chen"
            ],
            "abstract": "The outbreak of the novel Coronavirus Disease 2019 (COVID-19) has caused\nunprecedented impacts to people's daily life around the world. Various measures\nand policies such as lockdown and social-distancing are implemented by\ngovernments to combat the disease during the pandemic period. These measures\nand policies as well as virus itself may cause different mental health issues\nto people such as depression, anxiety, sadness, etc. In this paper, we exploit\nthe massive text data posted by Twitter users to analyse the sentiment dynamics\nof people living in the state of New South Wales (NSW) in Australia during the\npandemic period. Different from the existing work that mostly focuses the\ncountry-level and static sentiment analysis, we analyse the sentiment dynamics\nat the fine-grained local government areas (LGAs). Based on the analysis of\naround 94 million tweets that posted by around 183 thousand users located at\ndifferent LGAs in NSW in five months, we found that people in NSW showed an\noverall positive sentimental polarity and the COVID-19 pandemic decreased the\noverall positive sentimental polarity during the pandemic period. The\nfine-grained analysis of sentiment in LGAs found that despite the dominant\npositive sentiment most of days during the study period, some LGAs experienced\nsignificant sentiment changes from positive to negative. This study also\nanalysed the sentimental dynamics delivered by the hot topics in Twitter such\nas government policies (e.g. the Australia's JobKeeper program, lockdown,\nsocial-distancing) as well as the focused social events (e.g. the Ruby Princess\nCruise). The results showed that the policies and events did affect people's\noverall sentiment, and they affected people's overall sentiment differently at\ndifferent stages.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.12289v1",
            "date": "2020-06-22 14:22:47+00:00",
            "title": "MedLatin1 and MedLatin2: Two Datasets for the Computational Authorship Analysis of Medieval Latin Texts",
            "authors": [
                "Silvia Corbara",
                "Alejandro Moreo",
                "Fabrizio Sebastiani",
                "Mirko Tavoni"
            ],
            "abstract": "We present and make available MedLatin1 and MedLatin2, two datasets of\nmedieval Latin texts to be used in research on computational authorship\nanalysis. MedLatin1 and MedLatin2 consist of 294 and 30 curated texts,\nrespectively, labelled by author, with MedLatin1 texts being of an epistolary\nnature and MedLatin2 texts consisting of literary comments and treatises about\nvarious subjects. As such, these two datasets lend themselves to supporting\nresearch in authorship analysis tasks, such as authorship attribution,\nauthorship verification, or same-author verification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12442v2",
            "date": "2020-06-22 17:23:47+00:00",
            "title": "Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions",
            "authors": [
                "Stephen Roller",
                "Y-Lan Boureau",
                "Jason Weston",
                "Antoine Bordes",
                "Emily Dinan",
                "Angela Fan",
                "David Gunning",
                "Da Ju",
                "Margaret Li",
                "Spencer Poff",
                "Pratik Ringshia",
                "Kurt Shuster",
                "Eric Michael Smith",
                "Arthur Szlam",
                "Jack Urbanek",
                "Mary Williamson"
            ],
            "abstract": "We present our view of what is necessary to build an engaging open-domain\nconversational agent: covering the qualities of such an agent, the pieces of\nthe puzzle that have been built so far, and the gaping holes we have not filled\nyet. We present a biased view, focusing on work done by our own group, while\nciting related work in each area. In particular, we discuss in detail the\nproperties of continual learning, providing engaging content, and being\nwell-behaved -- and how to measure success in providing them. We end with a\ndiscussion of our experience and learnings, and our recommendations to the\ncommunity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.12958v3",
            "date": "2020-06-23 12:55:02+00:00",
            "title": "Can you tell? SSNet -- a Sagittal Stratum-inspired Neural Network Framework for Sentiment Analysis",
            "authors": [
                "Apostol Vassilev",
                "Munawar Hasan",
                "Honglan Jin"
            ],
            "abstract": "When people try to understand nuanced language they typically process\nmultiple input sensor modalities to complete this cognitive task. It turns out\nthe human brain has even a specialized neuron formation, called sagittal\nstratum, to help us understand sarcasm. We use this biological formation as the\ninspiration for designing a neural network architecture that combines\npredictions of different models on the same text to construct robust, accurate\nand computationally efficient classifiers for sentiment analysis and study\nseveral different realizations. Among them, we propose a systematic new\napproach to combining multiple predictions based on a dedicated neural network\nand develop mathematical analysis of it along with state-of-the-art\nexperimental results. We also propose a heuristic-hybrid technique for\ncombining models and back it up with experimental results on a representative\nbenchmark dataset and comparisons to other methods to show the advantages of\nthe new approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13268v1",
            "date": "2020-06-23 18:48:33+00:00",
            "title": "Automating Text Naturalness Evaluation of NLG Systems",
            "authors": [
                "Erion \u00c7ano",
                "Ond\u0159ej Bojar"
            ],
            "abstract": "Automatic methods and metrics that assess various quality criteria of\nautomatically generated texts are important for developing NLG systems because\nthey produce repeatable results and allow for a fast development cycle. We\npresent here an attempt to automate the evaluation of text naturalness which is\na very important characteristic of natural language generation methods. Instead\nof relying on human participants for scoring or labeling the text samples, we\npropose to automate the process by using a human likeliness metric we define\nand a discrimination procedure based on large pretrained language models with\ntheir probability distributions. We analyze the text probability fractions and\nobserve how they are influenced by the size of the generative and\ndiscriminative models involved in the process. Based on our results, bigger\ngenerators and larger pretrained discriminators are more appropriate for a\nbetter evaluation of text naturalness. A comprehensive validation procedure\nwith human participants is required as follow up to check how well this\nautomatic evaluation scheme correlates with human judgments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13299v1",
            "date": "2020-06-23 20:13:42+00:00",
            "title": "Supervised Understanding of Word Embeddings",
            "authors": [
                "Halid Ziya Yerebakan",
                "Parmeet Bhatia",
                "Yoshihisa Shinagawa"
            ],
            "abstract": "Pre-trained word embeddings are widely used for transfer learning in natural\nlanguage processing. The embeddings are continuous and distributed\nrepresentations of the words that preserve their similarities in compact\nEuclidean spaces. However, the dimensions of these spaces do not provide any\nclear interpretation. In this study, we have obtained supervised projections in\nthe form of the linear keyword-level classifiers on word embeddings. We have\nshown that the method creates interpretable projections of original embedding\ndimensions. Activations of the trained classifier nodes correspond to a subset\nof the words in the vocabulary. Thus, they behave similarly to the dictionary\nfeatures while having the merit of continuous value output. Additionally, such\ndictionaries can be grown iteratively with multiple rounds by adding expert\nlabels on top-scoring words to an initial collection of the keywords. Also, the\nsame classifiers can be applied to aligned word embeddings in other languages\nto obtain corresponding dictionaries. In our experiments, we have shown that\ninitializing higher-order networks with these classifier weights gives more\naccurate models for downstream NLP tasks. We further demonstrate the usefulness\nof supervised dimensions in revealing the polysemous nature of a keyword of\ninterest by projecting it's embedding using learned classifiers in different\nsub-spaces.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13507v1",
            "date": "2020-06-24 06:34:25+00:00",
            "title": "On Analyzing Annotation Consistency in Online Abusive Behavior Datasets",
            "authors": [
                "Md Rabiul Awal",
                "Rui Cao",
                "Roy Ka-Wei Lee",
                "Sandra Mitrovi\u0107"
            ],
            "abstract": "Online abusive behavior is an important issue that breaks the cohesiveness of\nonline social communities and even raises public safety concerns in our\nsocieties. Motivated by this rising issue, researchers have proposed,\ncollected, and annotated online abusive content datasets. These datasets play a\ncritical role in facilitating the research on online hate speech and abusive\nbehaviors. However, the annotation of such datasets is a difficult task; it is\noften contentious on what should be the true label of a given text as the\nsemantic difference of the labels may be blurred (e.g., abusive and hate) and\noften subjective. In this study, we proposed an analytical framework to study\nthe annotation consistency in online hate and abusive content datasets. We\napplied our proposed framework to evaluate the consistency of the annotation in\nthree popular datasets that are widely used in online hate speech and abusive\nbehavior studies. We found that there is still a substantial amount of\nannotation inconsistency in the existing datasets, particularly when the labels\nare semantically similar.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2006.13557v1",
            "date": "2020-06-24 08:29:09+00:00",
            "title": "Efficient Constituency Parsing by Pointing",
            "authors": [
                "Thanh-Tung Nguyen",
                "Xuan-Phi Nguyen",
                "Shafiq Joty",
                "Xiaoli Li"
            ],
            "abstract": "We propose a novel constituency parsing model that casts the parsing problem\ninto a series of pointing tasks. Specifically, our model estimates the\nlikelihood of a span being a legitimate tree constituent via the pointing score\ncorresponding to the boundary words of the span. Our parsing model supports\nefficient top-down decoding and our learning objective is able to enforce\nstructural consistency without resorting to the expensive CKY inference. The\nexperiments on the standard English Penn Treebank parsing task show that our\nmethod achieves 92.78 F1 without using pre-trained models, which is higher than\nall the existing methods with similar time complexity. Using pre-trained BERT,\nour model achieves 95.48 F1, which is competitive with the state-of-the-art\nwhile being faster. Our approach also establishes new state-of-the-art in\nBasque and Swedish in the SPMRL shared tasks on multilingual constituency\nparsing.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.13774v1",
            "date": "2020-06-24 14:47:33+00:00",
            "title": "Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings",
            "authors": [
                "David Chang",
                "Ivana Balazevic",
                "Carl Allen",
                "Daniel Chawla",
                "Cynthia Brandt",
                "Richard Andrew Taylor"
            ],
            "abstract": "Much of biomedical and healthcare data is encoded in discrete, symbolic form\nsuch as text and medical codes. There is a wealth of expert-curated biomedical\ndomain knowledge stored in knowledge bases and ontologies, but the lack of\nreliable methods for learning knowledge representation has limited their\nusefulness in machine learning applications. While text-based representation\nlearning has significantly improved in recent years through advances in natural\nlanguage processing, attempts to learn biomedical concept embeddings so far\nhave been lacking. A recent family of models called knowledge graph embeddings\nhave shown promising results on general domain knowledge graphs, and we explore\ntheir capabilities in the biomedical domain. We train several state-of-the-art\nknowledge graph embedding models on the SNOMED-CT knowledge graph, provide a\nbenchmark with comparison to existing methods and in-depth discussion on best\npractices, and make a case for the importance of leveraging the\nmulti-relational nature of knowledge graphs for learning biomedical knowledge\nrepresentation. The embeddings, code, and materials will be made available to\nthe communitY.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2006.13816v2",
            "date": "2020-06-15 20:03:28+00:00",
            "title": "Document Classification for COVID-19 Literature",
            "authors": [
                "Bernal Jim\u00e9nez Guti\u00e9rrez",
                "Juncheng Zeng",
                "Dongdong Zhang",
                "Ping Zhang",
                "Yu Su"
            ],
            "abstract": "The global pandemic has made it more important than ever to quickly and\naccurately retrieve relevant scientific literature for effective consumption by\nresearchers in a wide range of fields. We provide an analysis of several\nmulti-label document classification models on the LitCovid dataset, a growing\ncollection of 23,000 research papers regarding the novel 2019 coronavirus. We\nfind that pre-trained language models fine-tuned on this dataset outperform all\nother baselines and that BioBERT surpasses the others by a small margin with\nmicro-F1 and accuracy scores of around 86% and 75% respectively on the test\nset. We evaluate the data efficiency and generalizability of these models as\nessential features of any system prepared to deal with an urgent situation like\nthe current health crisis. Finally, we explore 50 errors made by the best\nperforming models on LitCovid documents and find that they often (1) correlate\ncertain labels too closely together and (2) fail to focus on discriminative\nsections of the articles; both of which are important issues to address in\nfuture work. Both data and code are available on GitHub.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.14135v2",
            "date": "2020-06-25 02:10:38+00:00",
            "title": "Explainable CNN-attention Networks (C-Attention Network) for Automated Detection of Alzheimer's Disease",
            "authors": [
                "Ning Wang",
                "Mingxuan Chen",
                "K. P. Subbalakshmi"
            ],
            "abstract": "In this work, we propose three explainable deep learning architectures to\nautomatically detect patients with Alzheimer`s disease based on their language\nabilities. The architectures use: (1) only the part-of-speech features; (2)\nonly language embedding features and (3) both of these feature classes via a\nunified architecture. We use self-attention mechanisms and interpretable\n1-dimensional ConvolutionalNeural Network (CNN) to generate two types of\nexplanations of the model`s action: intra-class explanation and inter-class\nexplanation. The inter-class explanation captures the relative importance of\neach of the different features in that class, while the inter-class explanation\ncaptures the relative importance between the classes. Note that although we\nhave considered two classes of features in this paper, the architecture is\neasily expandable to more classes because of its modularity. Extensive\nexperimentation and comparison with several recent models show that our method\noutperforms these methods with an accuracy of 92.2% and F1 score of 0.952on the\nDementiaBank dataset while being able to generate explanations. We show by\nexamples, how to generate these explanations using attention values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14799v2",
            "date": "2020-06-26 04:52:48+00:00",
            "title": "Evaluation of Text Generation: A Survey",
            "authors": [
                "Asli Celikyilmaz",
                "Elizabeth Clark",
                "Jianfeng Gao"
            ],
            "abstract": "The paper surveys evaluation methods of natural language generation (NLG)\nsystems that have been developed in the last few years. We group NLG evaluation\nmethods into three categories: (1) human-centric evaluation metrics, (2)\nautomatic metrics that require no training, and (3) machine-learned metrics.\nFor each category, we discuss the progress that has been made and the\nchallenges still being faced, with a focus on the evaluation of recently\nproposed NLG tasks and neural NLG models. We then present two examples for\ntask-specific NLG evaluations for automatic text summarization and long text\ngeneration, and conclude the paper by proposing future research directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14806v2",
            "date": "2020-06-26 05:44:54+00:00",
            "title": "TURL: Table Understanding through Representation Learning",
            "authors": [
                "Xiang Deng",
                "Huan Sun",
                "Alyssa Lees",
                "You Wu",
                "Cong Yu"
            ],
            "abstract": "Relational tables on the Web store a vast amount of knowledge. Owing to the\nwealth of such tables, there has been tremendous progress on a variety of tasks\nin the area of table understanding. However, existing work generally relies on\nheavily-engineered task-specific features and model architectures. In this\npaper, we present TURL, a novel framework that introduces the\npre-training/fine-tuning paradigm to relational Web tables. During\npre-training, our framework learns deep contextualized representations on\nrelational tables in an unsupervised manner. Its universal model design with\npre-trained representations can be applied to a wide range of tasks with\nminimal task-specific fine-tuning. Specifically, we propose a structure-aware\nTransformer encoder to model the row-column structure of relational tables, and\npresent a new Masked Entity Recovery (MER) objective for pre-training to\ncapture the semantics and knowledge in large-scale unlabeled data. We\nsystematically evaluate TURL with a benchmark consisting of 6 different tasks\nfor table understanding (e.g., relation extraction, cell filling). We show that\nTURL generalizes well to all tasks and substantially outperforms existing\nmethods in almost all instances.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2006.14939v1",
            "date": "2020-06-25 09:15:42+00:00",
            "title": "LSBert: A Simple Framework for Lexical Simplification",
            "authors": [
                "Jipeng Qiang",
                "Yun Li",
                "Yi Zhu",
                "Yunhao Yuan",
                "Xindong Wu"
            ],
            "abstract": "Lexical simplification (LS) aims to replace complex words in a given sentence\nwith their simpler alternatives of equivalent meaning, to simplify the\nsentence. Recently unsupervised lexical simplification approaches only rely on\nthe complex word itself regardless of the given sentence to generate candidate\nsubstitutions, which will inevitably produce a large number of spurious\ncandidates. In this paper, we propose a lexical simplification framework LSBert\nbased on pretrained representation model Bert, that is capable of (1) making\nuse of the wider context when both detecting the words in need of\nsimplification and generating substitue candidates, and (2) taking five\nhigh-quality features into account for ranking candidates, including Bert\nprediction order, Bert-based language model, and the paraphrase database PPDB,\nin addition to the word frequency and word similarity commonly used in other LS\nmethods. We show that our system outputs lexical simplifications that are\ngrammatically correct and semantically appropriate, and obtains obvious\nimprovement compared with these baselines, outperforming the state-of-the-art\nby 29.8 Accuracy points on three well-known benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.14994v2",
            "date": "2020-06-26 14:03:18+00:00",
            "title": "ProVe -- Self-supervised pipeline for automated product replacement and cold-starting based on neural language models",
            "authors": [
                "Andrei Ionut Damian",
                "Laurentiu Piciu",
                "Cosmin Mihai Marinescu"
            ],
            "abstract": "In retail vertical industries, businesses are dealing with human limitation\nof quickly understanding and adapting to new purchasing behaviors. Moreover,\nretail businesses need to overcome the human limitation of properly managing a\nmassive selection of products/brands/categories. These limitations lead to\ndeficiencies from both commercial (e.g. loss of sales, decrease in customer\nsatisfaction) and operational perspective (e.g. out-of-stock, over-stock). In\nthis paper, we propose a pipeline approach based on Natural Language\nUnderstanding, for recommending the most suitable replacements for products\nthat are out-of-stock. Moreover, we will propose a solution for managing\nproducts that were newly introduced in a retailer's portfolio with almost no\ntransactional history. This solution will help businesses: automatically assign\nthe new products to the right category; recommend complementary products for\ncross-sell from day 1; perform sales predictions even with almost no\ntransactional history. Finally, the vector space model resulted by applying the\npipeline presented in this paper is directly used as semantic information in\ndeep learning-based demand forecasting solutions, leading to more accurate\npredictions. The whole research and experimentation process have been done\nusing real-life private transactional data, however the source code is\navailable on https://github.com/Lummetry/ProVe",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15315v1",
            "date": "2020-06-27 08:13:58+00:00",
            "title": "Uncertainty-aware Self-training for Text Classification with Few Labels",
            "authors": [
                "Subhabrata Mukherjee",
                "Ahmed Hassan Awadallah"
            ],
            "abstract": "Recent success of large-scale pre-trained language models crucially hinge on\nfine-tuning them on large amounts of labeled data for the downstream task, that\nare typically expensive to acquire. In this work, we study self-training as one\nof the earliest semi-supervised learning approaches to reduce the annotation\nbottleneck by making use of large-scale unlabeled data for the target task.\nStandard self-training mechanism randomly samples instances from the unlabeled\npool to pseudo-label and augment labeled data. In this work, we propose an\napproach to improve self-training by incorporating uncertainty estimates of the\nunderlying neural network leveraging recent advances in Bayesian deep learning.\nSpecifically, we propose (i) acquisition functions to select instances from the\nunlabeled pool leveraging Monte Carlo (MC) Dropout, and (ii) learning mechanism\nleveraging model confidence for self-training. As an application, we focus on\ntext classification on five benchmark datasets. We show our methods leveraging\nonly 20-30 labeled samples per class for each task for training and for\nvalidation can perform within 3% of fully supervised pre-trained language\nmodels fine-tuned on thousands of labeled instances with an aggregate accuracy\nof 91% and improving by upto 12% over baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15411v1",
            "date": "2020-06-27 17:42:15+00:00",
            "title": "String-based methods for tonal harmony: A corpus study of Haydn's string quartets",
            "authors": [
                "David R. W. Sears"
            ],
            "abstract": "This chapter considers how string-based methods might be adapted to address\nmusic-analytic questions related to the discovery of musical organization, with\nparticular attention devoted to the analysis of tonal harmony. I begin by\napplying the taxonomy of mental organization proposed by Mandler (1979) to the\nconcept of musical organization. Using this taxonomy as a guide, I then present\nevidence for three principles of tonal harmony -- recurrence, syntax, and\nrecursion -- using a corpus of Haydn string quartets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15595v4",
            "date": "2020-06-28 13:11:02+00:00",
            "title": "Rethinking Positional Encoding in Language Pre-training",
            "authors": [
                "Guolin Ke",
                "Di He",
                "Tie-Yan Liu"
            ],
            "abstract": "In this work, we investigate the positional encoding methods used in language\npre-training (e.g., BERT) and identify several problems in the existing\nformulations. First, we show that in the absolute positional encoding, the\naddition operation applied on positional embeddings and word embeddings brings\nmixed correlations between the two heterogeneous information resources. It may\nbring unnecessary randomness in the attention and further limit the\nexpressiveness of the model. Second, we question whether treating the position\nof the symbol \\texttt{[CLS]} the same as other words is a reasonable design,\nconsidering its special role (the representation of the entire sentence) in the\ndownstream tasks. Motivated from above analysis, we propose a new positional\nencoding method called \\textbf{T}ransformer with \\textbf{U}ntied\n\\textbf{P}ositional \\textbf{E}ncoding (TUPE). In the self-attention module,\nTUPE computes the word contextual correlation and positional correlation\nseparately with different parameterizations and then adds them together. This\ndesign removes the mixed and noisy correlations over heterogeneous embeddings\nand offers more expressiveness by using different projection matrices.\nFurthermore, TUPE unties the \\texttt{[CLS]} symbol from other positions, making\nit easier to capture information from all positions. Extensive experiments and\nablation studies on GLUE benchmark demonstrate the effectiveness of the\nproposed method. Codes and models are released at\nhttps://github.com/guolinke/TUPE.",
            "journal ref": "International Conference on Learning Representations (ICLR) 2021,\n  https://openreview.net/forum?id=09-528y2Fgf",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15720v2",
            "date": "2020-06-28 21:23:05+00:00",
            "title": "Progressive Generation of Long Text with Pretrained Language Models",
            "authors": [
                "Bowen Tan",
                "Zichao Yang",
                "Maruan AI-Shedivat",
                "Eric P. Xing",
                "Zhiting Hu"
            ],
            "abstract": "Large-scale language models (LMs) pretrained on massive corpora of text, such\nas GPT-2, are powerful open-domain text generators. However, as our systematic\nexamination reveals, it is still challenging for such models to generate\ncoherent long passages of text (e.g., 1000 tokens), especially when the models\nare fine-tuned to the target domain on a small corpus. Previous\nplanning-then-generation methods also fall short of producing such long text in\nvarious domains. To overcome the limitations, we propose a simple but effective\nmethod of generating text in a progressive manner, inspired by generating\nimages from low to high resolution. Our method first produces domain-specific\ncontent keywords and then progressively refines them into complete passages in\nmultiple stages. The simple design allows our approach to take advantage of\npretrained LMs at each stage and effectively adapt to any target domain given\nonly a small set of examples. We conduct a comprehensive empirical study with a\nbroad set of evaluation metrics, and show that our approach significantly\nimproves upon the fine-tuned large LMs and various planning-then-generation\nmethods in terms of quality and sample efficiency. Human evaluation also\nvalidates that our model generations are more coherent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15732v2",
            "date": "2020-06-28 22:23:03+00:00",
            "title": "Mapping Topic Evolution Across Poetic Traditions",
            "authors": [
                "Petr Plechac",
                "Thomas N. Haider"
            ],
            "abstract": "Poetic traditions across languages evolved differently, but we find that\ncertain semantic topics occur in several of them, albeit sometimes with\ntemporal delay, or with diverging trajectories over time. We apply Latent\nDirichlet Allocation (LDA) to poetry corpora of four languages, i.e. German\n(52k poems), English (85k poems), Russian (18k poems), and Czech (80k poems).\nWe align and interpret salient topics, their trend over time (1600--1925 A.D.),\nshowing similarities and disparities across poetic traditions with a few select\ntopics, and use their trajectories over time to pinpoint specific literary\nepochs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.15935v2",
            "date": "2020-06-29 11:07:10+00:00",
            "title": "Is Japanese gendered language used on Twitter ? A large scale study",
            "authors": [
                "Tiziana Carpi",
                "Stefano Maria Iacus"
            ],
            "abstract": "This study analyzes the usage of Japanese gendered language on Twitter.\nStarting from a collection of 408 million Japanese tweets from 2015 till 2019\nand an additional sample of 2355 manually classified Twitter accounts timelines\ninto gender and categories (politicians, musicians, etc). A large scale textual\nanalysis is performed on this corpus to identify and examine sentence-final\nparticles (SFPs) and first-person pronouns appearing in the texts. It turns out\nthat gendered language is in fact used also on Twitter, in about 6% of the\ntweets, and that the prescriptive classification into \"male\" and \"female\"\nlanguage does not always meet the expectations, with remarkable exceptions.\nFurther, SFPs and pronouns show increasing or decreasing trends, indicating an\nevolution of the language used on Twitter.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16152v3",
            "date": "2020-06-29 16:14:27+00:00",
            "title": "Leveraging Subword Embeddings for Multinational Address Parsing",
            "authors": [
                "Marouane Yassine",
                "David Beauchemin",
                "Fran\u00e7ois Laviolette",
                "Luc Lamontagne"
            ],
            "abstract": "Address parsing consists of identifying the segments that make up an address\nsuch as a street name or a postal code. Because of its importance for tasks\nlike record linkage, address parsing has been approached with many techniques.\nNeural network methods defined a new state-of-the-art for address parsing.\nWhile this approach yielded notable results, previous work has only focused on\napplying neural networks to achieve address parsing of addresses from one\nsource country. We propose an approach in which we employ subword embeddings\nand a Recurrent Neural Network architecture to build a single model capable of\nlearning to parse addresses from multiple countries at the same time while\ntaking into account the difference in languages and address formatting systems.\nWe achieved accuracies around 99 % on the countries used for training with no\npre-processing nor post-processing needed. We explore the possibility of\ntransferring the address parsing knowledge obtained by training on some\ncountries' addresses to others with no further training in a zero-shot transfer\nlearning setting. We achieve good results for 80 % of the countries (33 out of\n41), almost 50 % of which (20 out of 41) is near state-of-the-art performance.\nIn addition, we propose an open-source Python implementation of our trained\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16174v1",
            "date": "2020-06-29 16:37:51+00:00",
            "title": "Multichannel CNN with Attention for Text Classification",
            "authors": [
                "Zhenyu Liu",
                "Haiwei Huang",
                "Chaohong Lu",
                "Shengfei Lyu"
            ],
            "abstract": "Recent years, the approaches based on neural networks have shown remarkable\npotential for sentence modeling. There are two main neural network structures:\nrecurrent neural network (RNN) and convolution neural network (CNN). RNN can\ncapture long term dependencies and store the semantics of the previous\ninformation in a fixed-sized vector. However, RNN is a biased model and its\nability to extract global semantics is restricted by the fixed-sized vector.\nAlternatively, CNN is able to capture n-gram features of texts by utilizing\nconvolutional filters. But the width of convolutional filters restricts its\nperformance. In order to combine the strengths of the two kinds of networks and\nalleviate their shortcomings, this paper proposes Attention-based Multichannel\nConvolutional Neural Network (AMCNN) for text classification. AMCNN utilizes a\nbi-directional long short-term memory to encode the history and future\ninformation of words into high dimensional representations, so that the\ninformation of both the front and back of the sentence can be fully expressed.\nThen the scalar attention and vectorial attention are applied to obtain\nmultichannel representations. The scalar attention can calculate the word-level\nimportance and the vectorial attention can calculate the feature-level\nimportance. In the classification task, AMCNN uses a CNN structure to cpture\nword relations on the representations generated by the scalar and vectorial\nattention mechanism instead of calculating the weighted sums. It can\neffectively extract the n-gram features of the text. The experimental results\non the benchmark datasets demonstrate that AMCNN achieves better performance\nthan state-of-the-art methods. In addition, the visualization results verify\nthe semantic richness of multichannel representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16324v1",
            "date": "2020-06-29 19:15:10+00:00",
            "title": "Universal linguistic inductive biases via meta-learning",
            "authors": [
                "R. Thomas McCoy",
                "Erin Grant",
                "Paul Smolensky",
                "Thomas L. Griffiths",
                "Tal Linzen"
            ],
            "abstract": "How do learners acquire languages from the limited data available to them?\nThis process must involve some inductive biases - factors that affect how a\nlearner generalizes - but it is unclear which inductive biases can explain\nobserved patterns in language acquisition. To facilitate computational modeling\naimed at addressing this question, we introduce a framework for giving\nparticular linguistic inductive biases to a neural network model; such a model\ncan then be used to empirically explore the effects of those inductive biases.\nThis framework disentangles universal inductive biases, which are encoded in\nthe initial values of a neural network's parameters, from non-universal\nfactors, which the neural network must learn from data in a given language. The\ninitial state that encodes the inductive biases is found with meta-learning, a\ntechnique through which a model discovers how to acquire new languages more\neasily via exposure to many possible languages. By controlling the properties\nof the languages that are used during meta-learning, we can control the\ninductive biases that meta-learning imparts. We demonstrate this framework with\na case study based on syllable structure. First, we specify the inductive\nbiases that we intend to give our model, and then we translate those inductive\nbiases into a space of languages from which a model can meta-learn. Finally,\nusing existing analysis techniques, we verify that our approach has imparted\nthe linguistic inductive biases that it was intended to impart.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16336v2",
            "date": "2020-06-29 19:41:26+00:00",
            "title": "Learning Sparse Prototypes for Text Generation",
            "authors": [
                "Junxian He",
                "Taylor Berg-Kirkpatrick",
                "Graham Neubig"
            ],
            "abstract": "Prototype-driven text generation uses non-parametric models that first choose\nfrom a library of sentence \"prototypes\" and then modify the prototype to\ngenerate the output text. While effective, these methods are inefficient at\ntest time as a result of needing to store and index the entire training corpus.\nFurther, existing methods often require heuristics to identify which prototypes\nto reference at training time. In this paper, we propose a novel generative\nmodel that automatically learns a sparse prototype support set that,\nnonetheless, achieves strong language modeling performance. This is achieved by\n(1) imposing a sparsity-inducing prior on the prototype selection distribution,\nand (2) utilizing amortized variational inference to learn a prototype\nretrieval function. In experiments, our model outperforms previous\nprototype-driven language models while achieving up to a 1000x memory\nreduction, as well as a 1000x speed-up at test time. More interestingly, we\nshow that the learned prototypes are able to capture semantics and syntax at\ndifferent granularity as we vary the sparsity of prototype selection, and that\ncertain sentence attributes can be controlled by specifying the prototype for\ngeneration.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16402v1",
            "date": "2020-06-29 21:40:55+00:00",
            "title": "Reading Between the Demographic Lines: Resolving Sources of Bias in Toxicity Classifiers",
            "authors": [
                "Elizabeth Reichert",
                "Helen Qiu",
                "Jasmine Bayrooti"
            ],
            "abstract": "The censorship of toxic comments is often left to the judgment of imperfect\nmodels. Perspective API, a creation of Google technology incubator Jigsaw, is\nperhaps the most widely used toxicity classifier in industry; the model is\nemployed by several online communities including The New York Times to identify\nand filter out toxic comments with the goal of preserving online safety.\nUnfortunately, Google's model tends to unfairly assign higher toxicity scores\nto comments containing words referring to the identities of commonly targeted\ngroups (e.g., \"woman,'' \"gay,'' etc.) because these identities are frequently\nreferenced in a disrespectful manner in the training data. As a result,\ncomments generated by marginalized groups referencing their identities are\noften mistakenly censored. It is important to be cognizant of this unintended\nbias and strive to mitigate its effects. To address this issue, we have\nconstructed several toxicity classifiers with the intention of reducing\nunintended bias while maintaining strong classification performance.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2006.16722v1",
            "date": "2020-06-30 12:24:35+00:00",
            "title": "Correction of Faulty Background Knowledge based on Condition Aware and Revise Transformer for Question Answering",
            "authors": [
                "Xinyan Zhao",
                "Xiao Feng",
                "Haoming Zhong",
                "Jun Yao",
                "Huanhuan Chen"
            ],
            "abstract": "The study of question answering has received increasing attention in recent\nyears. This work focuses on providing an answer that compatible with both user\nintent and conditioning information corresponding to the question, such as\ndelivery status and stock information in e-commerce. However, these conditions\nmay be wrong or incomplete in real-world applications. Although existing\nquestion answering systems have considered the external information, such as\ncategorical attributes and triples in knowledge base, they all assume that the\nexternal information is correct and complete. To alleviate the effect of\ndefective condition values, this paper proposes condition aware and revise\nTransformer (CAR-Transformer). CAR-Transformer (1) revises each condition value\nbased on the whole conversation and original conditions values, and (2) it\nencodes the revised conditions and utilizes the conditions embedding to select\nan answer. Experimental results on a real-world customer service dataset\ndemonstrate that the CAR-Transformer can still select an appropriate reply when\nconditions corresponding to the question exist wrong or missing values, and\nsubstantially outperforms baseline models on automatic and human evaluations.\nThe proposed CAR-Transformer can be extended to other NLP tasks which need to\nconsider conditioning information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2006.16934v3",
            "date": "2020-06-30 16:03:12+00:00",
            "title": "ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph",
            "authors": [
                "Fei Yu",
                "Jiji Tang",
                "Weichong Yin",
                "Yu Sun",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "We propose a knowledge-enhanced approach, ERNIE-ViL, which incorporates\nstructured knowledge obtained from scene graphs to learn joint representations\nof vision-language. ERNIE-ViL tries to build the detailed semantic connections\n(objects, attributes of objects and relationships between objects) across\nvision and language, which are essential to vision-language cross-modal tasks.\nUtilizing scene graphs of visual scenes, ERNIE-ViL constructs Scene Graph\nPrediction tasks, i.e., Object Prediction, Attribute Prediction and\nRelationship Prediction tasks in the pre-training phase. Specifically, these\nprediction tasks are implemented by predicting nodes of different types in the\nscene graph parsed from the sentence. Thus, ERNIE-ViL can learn the joint\nrepresentations characterizing the alignments of the detailed semantics across\nvision and language. After pre-training on large scale image-text aligned\ndatasets, we validate the effectiveness of ERNIE-ViL on 5 cross-modal\ndownstream tasks. ERNIE-ViL achieves state-of-the-art performances on all these\ntasks and ranks the first place on the VCR leaderboard with an absolute\nimprovement of 3.7%.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2006.16967v1",
            "date": "2020-06-25 19:43:13+00:00",
            "title": "Lest We Forget: A Dataset of Coronavirus-Related News Headlines in Swiss Media",
            "authors": [
                "Alireza Ghasemi",
                "Amina Chebira"
            ],
            "abstract": "We release our COVID-19 news dataset, containing more than 10,000 links to\nnews articles related to the Coronavirus pandemic published in the Swiss media\nsince early January 2020. This collection can prove beneficial in mining and\nanalysis of the reaction of the Swiss media and the COVID-19 pandemic and\nextracting insightful information for further research. We hope this dataset\nhelps researchers and the public deliver results that will help analyse the\npandemic and potentially lead to a better understanding of the events.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00236v2",
            "date": "2020-07-01 04:41:05+00:00",
            "title": "SemEval-2020 Task 4: Commonsense Validation and Explanation",
            "authors": [
                "Cunxiang Wang",
                "Shuailong Liang",
                "Yili Jin",
                "Yilong Wang",
                "Xiaodan Zhu",
                "Yue Zhang"
            ],
            "abstract": "In this paper, we present SemEval-2020 Task 4, Commonsense Validation and\nExplanation (ComVE), which includes three subtasks, aiming to evaluate whether\na system can distinguish a natural language statement that makes sense to\nhumans from one that does not, and provide the reasons. Specifically, in our\nfirst subtask, the participating systems are required to choose from two\nnatural language statements of similar wording the one that makes sense and the\none does not. The second subtask additionally asks a system to select the key\nreason from three options why a given statement does not make sense. In the\nthird subtask, a participating system needs to generate the reason. We finally\nattracted 39 teams participating at least one of the three subtasks. For\nSubtask A and Subtask B, the performances of top-ranked systems are close to\nthat of humans. However, for Subtask C, there is still a relatively large gap\nbetween systems and human performance. The dataset used in our task can be\nfound at https://github.com/wangcunxiang/SemEval2020-\nTask4-Commonsense-Validation-and-Explanation; The leaderboard can be found at\nhttps://competitions.codalab.org/competitions/21080#results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00380v1",
            "date": "2020-07-01 10:54:27+00:00",
            "title": "Unsupervised Semantic Hashing with Pairwise Reconstruction",
            "authors": [
                "Casper Hansen",
                "Christian Hansen",
                "Jakob Grue Simonsen",
                "Stephen Alstrup",
                "Christina Lioma"
            ],
            "abstract": "Semantic Hashing is a popular family of methods for efficient similarity\nsearch in large-scale datasets. In Semantic Hashing, documents are encoded as\nshort binary vectors (i.e., hash codes), such that semantic similarity can be\nefficiently computed using the Hamming distance. Recent state-of-the-art\napproaches have utilized weak supervision to train better performing hashing\nmodels. Inspired by this, we present Semantic Hashing with Pairwise\nReconstruction (PairRec), which is a discrete variational autoencoder based\nhashing model. PairRec first encodes weakly supervised training pairs (a query\ndocument and a semantically similar document) into two hash codes, and then\nlearns to reconstruct the same query document from both of these hash codes\n(i.e., pairwise reconstruction). This pairwise reconstruction enables our model\nto encode local neighbourhood structures within the hash code directly through\nthe decoder. We experimentally compare PairRec to traditional and\nstate-of-the-art approaches, and obtain significant performance improvements in\nthe task of document similarity search.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.00576v6",
            "date": "2020-07-01 16:03:20+00:00",
            "title": "COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation",
            "authors": [
                "Qingyun Wang",
                "Manling Li",
                "Xuan Wang",
                "Nikolaus Parulian",
                "Guangxing Han",
                "Jiawei Ma",
                "Jingxuan Tu",
                "Ying Lin",
                "Haoran Zhang",
                "Weili Liu",
                "Aabhas Chauhan",
                "Yingjun Guan",
                "Bangzheng Li",
                "Ruisong Li",
                "Xiangchen Song",
                "Yi R. Fung",
                "Heng Ji",
                "Jiawei Han",
                "Shih-Fu Chang",
                "James Pustejovsky",
                "Jasmine Rah",
                "David Liem",
                "Ahmed Elsayed",
                "Martha Palmer",
                "Clare Voss",
                "Cynthia Schneider",
                "Boyan Onyshkevych"
            ],
            "abstract": "To combat COVID-19, both clinicians and scientists need to digest vast\namounts of relevant biomedical knowledge in scientific literature to understand\nthe disease mechanism and related biological functions. We have developed a\nnovel and comprehensive knowledge discovery framework, COVID-KG to extract\nfine-grained multimedia knowledge elements (entities and their visual chemical\nstructures, relations, and events) from scientific literature. We then exploit\nthe constructed multimedia knowledge graphs (KGs) for question answering and\nreport generation, using drug repurposing as a case study. Our framework also\nprovides detailed contextual sentences, subfigures, and knowledge subgraphs as\nevidence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00709v2",
            "date": "2020-07-01 19:21:10+00:00",
            "title": "Computing Conceptual Distances between Breast Cancer Screening Guidelines: An Implementation of a Near-Peer Epistemic Model of Medical Disagreement",
            "authors": [
                "Hossein Hematialam",
                "Luciana Garbayo",
                "Seethalakshmi Gopalakrishnan",
                "Wlodek Zadrozny"
            ],
            "abstract": "Using natural language processing tools, we investigate the differences of\nrecommendations in medical guidelines for the same decision problem -- breast\ncancer screening. We show that these differences arise from knowledge brought\nto the problem by different medical societies, as reflected in the conceptual\nvocabularies used by the different groups of authors.The computational models\nwe build and analyze agree with the near-peer epistemic model of expert\ndisagreement proposed by Garbayo. Even though the article is a case study\nfocused on one set of guidelines, the proposed methodology is broadly\napplicable. In addition to proposing a novel graph-based similarity model for\ncomparing collections of documents, we perform an extensive analysis of the\nmodel performance. In a series of a few dozen experiments, in three broad\ncategories, we show, at a very high statistical significance level of 3-4\nstandard deviations for our best models, that the high similarity between\nexpert annotated model and our concept based, automatically created,\ncomputational models is not accidental. Our best model achieves roughly 70%\nsimilarity. We also describe possible extensions of this work.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.00750v1",
            "date": "2020-07-01 20:55:41+00:00",
            "title": "Legends: Folklore on Reddit",
            "authors": [
                "Caitrin Armstrong",
                "Derek Ruths"
            ],
            "abstract": "In this paper we introduce Reddit legends, a collection of venerated old\nposts that have become famous on Reddit. To establish the utility of Reddit\nlegends for both computational science/HCI and folkloristics, we investigate\ntwo main questions: (1) whether they can be considered folklore, i.e. if they\nhave consistent form, cultural significance, and undergo spontaneous\ntransmission, and (2) whether they can be studied in a systematic manner.\nThrough several subtasks, including the creation of a typology, an analysis of\nreferences to Reddit legends, and an examination of some of the textual\ncharacteristics of referencing behaviour, we show that Reddit legends can\nindeed be considered as folklore and that they are amendable to systematic\ntext-based approaches. We discuss how these results will enable future analyses\nof folklore on Reddit, including tracking subreddit-wide and individual-user\nbehaviour, and the relationship of this behaviour to other cultural markers.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2007.00814v1",
            "date": "2020-07-01 23:50:58+00:00",
            "title": "Relevance-guided Supervision for OpenQA with ColBERT",
            "authors": [
                "Omar Khattab",
                "Christopher Potts",
                "Matei Zaharia"
            ],
            "abstract": "Systems for Open-Domain Question Answering (OpenQA) generally depend on a\nretriever for finding candidate passages in a large corpus and a reader for\nextracting answers from those passages. In much recent work, the retriever is a\nlearned component that uses coarse-grained vector representations of questions\nand passages. We argue that this modeling choice is insufficiently expressive\nfor dealing with the complexity of natural language questions. To address this,\nwe define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT\nto OpenQA. ColBERT creates fine-grained interactions between questions and\npassages. We propose a weak supervision strategy that iteratively uses ColBERT\nto create its own training data. This greatly improves OpenQA retrieval on both\nNatural Questions and TriviaQA, and the resulting end-to-end OpenQA system\nattains state-of-the-art performance on both of those datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.00824v2",
            "date": "2020-07-02 01:25:22+00:00",
            "title": "Lightme: Analysing Language in Internet Support Groups for Mental Health",
            "authors": [
                "Gabriela Ferraro",
                "Brendan Loo Gee",
                "Shenjia Ji",
                "Luis Salvador-Carulla"
            ],
            "abstract": "Background: Assisting moderators to triage harmful posts in Internet Support\nGroups is relevant to ensure its safe use. Automated text classification\nmethods analysing the language expressed in posts of online forums is a\npromising solution. Methods: Natural Language Processing and Machine Learning\ntechnologies were used to build a triage post classifier using a dataset from\nReachout mental health forum for young people. Results: When comparing with the\nstate-of-the-art, a solution mainly based on features from lexical resources,\nreceived the best classification performance for the crisis posts (52%), which\nis the most severe class. Six salient linguistic characteristics were found\nwhen analysing the crisis post; 1) posts expressing hopelessness, 2) short\nposts expressing concise negative emotional responses, 3) long posts expressing\nvariations of emotions, 4) posts expressing dissatisfaction with available\nhealth services, 5) posts utilising storytelling, and 6) posts expressing users\nseeking advice from peers during a crisis. Conclusion: It is possible to build\na competitive triage classifier using features derived only from the textual\ncontent of the post. Further research needs to be done in order to translate\nour quantitative and qualitative findings into features, as it may improve\noverall performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01022v1",
            "date": "2020-07-02 11:17:16+00:00",
            "title": "NLNDE: Enhancing Neural Sequence Taggers with Attention and Noisy Channel for Robust Pharmacological Entity Detection",
            "authors": [
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "Named entity recognition has been extensively studied on English news texts.\nHowever, the transfer to other domains and languages is still a challenging\nproblem. In this paper, we describe the system with which we participated in\nthe first subtrack of the PharmaCoNER competition of the BioNLP Open Shared\nTasks 2019. Aiming at pharmacological entity detection in Spanish texts, the\ntask provides a non-standard domain and language setting. However, we propose\nan architecture that requires neither language nor domain expertise. We treat\nthe task as a sequence labeling task and experiment with attention-based\nembedding selection and the training on automatically annotated data to further\nimprove our system's performance. Our system achieves promising results,\nespecially by combining the different techniques, and reaches up to 88.6% F1 in\nthe competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01030v1",
            "date": "2020-07-02 11:30:32+00:00",
            "title": "NLNDE: The Neither-Language-Nor-Domain-Experts' Way of Spanish Medical Document De-Identification",
            "authors": [
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "Natural language processing has huge potential in the medical domain which\nrecently led to a lot of research in this field. However, a prerequisite of\nsecure processing of medical documents, e.g., patient notes and clinical\ntrials, is the proper de-identification of privacy-sensitive information. In\nthis paper, we describe our NLNDE system, with which we participated in the\nMEDDOCAN competition, the medical document anonymization task of IberLEF 2019.\nWe address the task of detecting and classifying protected health information\nfrom Spanish data as a sequence-labeling problem and investigate different\nembedding methods for our neural network. Despite dealing in a non-standard\nlanguage and domain setting, the NLNDE system achieves promising results in the\ncompetition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01127v1",
            "date": "2020-07-02 14:23:57+00:00",
            "title": "Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey",
            "authors": [
                "Shivaji Alaparthi",
                "Manit Mishra"
            ],
            "abstract": "The purpose of the study is to investigate the relative effectiveness of four\ndifferent sentiment analysis techniques: (1) unsupervised lexicon-based model\nusing Sent WordNet; (2) traditional supervised machine learning model using\nlogistic regression; (3) supervised deep learning model using Long Short-Term\nMemory (LSTM); and, (4) advanced supervised deep learning models using\nBidirectional Encoder Representations from Transformers (BERT). We use publicly\navailable labeled corpora of 50,000 movie reviews originally posted on internet\nmovie database (IMDB) for analysis using Sent WordNet lexicon, logistic\nregression, LSTM, and BERT. The first three models were run on CPU based system\nwhereas BERT was run on GPU based system. The sentiment classification\nperformance was evaluated based on accuracy, precision, recall, and F1 score.\nThe study puts forth two key insights: (1) relative efficacy of four highly\nadvanced and widely used sentiment analysis techniques; (2) undisputed\nsuperiority of pre-trained advanced supervised deep learning BERT model in\nsentiment analysis from text data. This study provides professionals in\nanalytics industry and academicians working on text analysis key insight\nregarding comparative classification performance evaluation of key sentiment\nanalysis techniques, including the recently developed BERT. This is the first\nresearch endeavor to compare the advanced pre-trained supervised deep learning\nmodel of BERT vis-\\`a-vis other sentiment analysis models of LSTM, logistic\nregression, and Sent WordNet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01282v2",
            "date": "2020-07-02 17:44:57+00:00",
            "title": "Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering",
            "authors": [
                "Gautier Izacard",
                "Edouard Grave"
            ],
            "abstract": "Generative models for open domain question answering have proven to be\ncompetitive, without resorting to external knowledge. While promising, this\napproach requires to use models with billions of parameters, which are\nexpensive to train and query. In this paper, we investigate how much these\nmodels can benefit from retrieving text passages, potentially containing\nevidence. We obtain state-of-the-art results on the Natural Questions and\nTriviaQA open benchmarks. Interestingly, we observe that the performance of\nthis method significantly improves when increasing the number of retrieved\npassages. This is evidence that generative models are good at aggregating and\ncombining evidence from multiple passages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01379v2",
            "date": "2020-07-02 20:44:05+00:00",
            "title": "Detecting Ongoing Events Using Contextual Word and Sentence Embeddings",
            "authors": [
                "Mariano Maisonnave",
                "Fernando Delbianco",
                "Fernando Tohm\u00e9",
                "Ana Maguitman",
                "Evangelos Milios"
            ],
            "abstract": "This paper introduces the Ongoing Event Detection (OED) task, which is a\nspecific Event Detection task where the goal is to detect ongoing event\nmentions only, as opposed to historical, future, hypothetical, or other forms\nor events that are neither fresh nor current. Any application that needs to\nextract structured information about ongoing events from unstructured texts can\ntake advantage of an OED system. The main contribution of this paper are the\nfollowing: (1) it introduces the OED task along with a dataset manually labeled\nfor the task; (2) it presents the design and implementation of an RNN model for\nthe task that uses BERT embeddings to define contextual word and contextual\nsentence embeddings as attributes, which to the best of our knowledge were\nnever used before for detecting ongoing events in news; (3) it presents an\nextensive empirical evaluation that includes (i) the exploration of different\narchitectures and hyperparameters, (ii) an ablation test to study the impact of\neach attribute, and (iii) a comparison with a replication of a state-of-the-art\nmodel. The results offer several insights into the importance of contextual\nembeddings and indicate that the proposed approach is effective in the OED\ntask, outperforming the baseline models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.01918v1",
            "date": "2020-07-03 19:30:28+00:00",
            "title": "Abstractive and mixed summarization for long-single documents",
            "authors": [
                "Roger Barrull",
                "Jugal Kalita"
            ],
            "abstract": "The lack of diversity in the datasets available for automatic summarization\nof documents has meant that the vast majority of neural models for automatic\nsummarization have been trained with news articles. These datasets are\nrelatively small, with an average size of about 600 words, and the models\ntrained with such data sets see their performance limited to short documents.\nIn order to surmount this problem, this paper uses scientific papers as the\ndataset on which different models are trained. These models have been chosen\nbased on their performance on the CNN/Daily Mail data set, so that the highest\nranked model of each architectural variant is selected. In this work, six\ndifferent models are compared, two with an RNN architecture, one with a CNN\narchitecture, two with a Transformer architecture and one with a Transformer\narchitecture combined with reinforcement learning. The results from this work\nshow that those models that use a hierarchical encoder to model the structure\nof the document has a better performance than the rest.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02100v2",
            "date": "2020-07-04 14:03:48+00:00",
            "title": "Pynsett: A programmable relation extractor",
            "authors": [
                "Alberto Cetoli"
            ],
            "abstract": "This paper proposes a programmable relation extraction method for the English\nlanguage by parsing texts into semantic graphs. A person can define rules in\nplain English that act as matching patterns onto the graph representation.\nThese rules are designed to capture the semantic content of the documents,\nallowing for flexibility and ad-hoc entities. Relation extraction is a complex\ntask that typically requires sizable training corpora. The method proposed here\nis ideal for extracting specialized ontologies in a limited collection of\ndocuments.",
            "journal ref": "The Fourteenth International Conference on Advances in Semantic\n  Processing (2020), Pages 45-48",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02282v2",
            "date": "2020-07-05 10:17:02+00:00",
            "title": "Proving Non-Inclusion of B\u00fcchi Automata based on Monte Carlo Sampling",
            "authors": [
                "Yong Li",
                "Andrea Turrini",
                "Xuechao Sun",
                "Lijun Zhang"
            ],
            "abstract": "The search for a proof of correctness and the search for counterexamples\n(bugs) are complementary aspects of verification. In order to maximize the\npractical use of verification tools it is better to pursue them at the same\ntime. While this is well-understood in the termination analysis of programs,\nthis is not the case for the language inclusion analysis of B\\\"uchi automata,\nwhere research mainly focused on improving algorithms for proving language\ninclusion, with the search for counterexamples left to the expensive\ncomplementation operation.\n  In this paper, we present $\\mathsf{IMC}^2$, a specific algorithm for proving\nB\\\"uchi automata non-inclusion $\\mathcal{L}(\\mathcal{A}) \\not\\subseteq\n\\mathcal{L}(\\mathcal{B})$, based on Grosu and Smolka's algorithm\n$\\mathsf{MC}^2$ developed for Monte Carlo model checking against LTL formulas.\nThe algorithm we propose takes $M = \\lceil \\ln \\delta / \\ln (1-\\epsilon)\n\\rceil$ random lasso-shaped samples from $\\mathcal{A}$ to decide whether to\nreject the hypothesis $\\mathcal{L}(\\mathcal{A}) \\not\\subseteq\n\\mathcal{L}(\\mathcal{B})$, for given error probability $\\epsilon$ and\nconfidence level $1 - \\delta$. With such a number of samples, $\\mathsf{IMC}^2$\nensures that the probability of witnessing $\\mathcal{L}(\\mathcal{A})\n\\not\\subseteq \\mathcal{L}(\\mathcal{B})$ via further sampling is less than\n$\\delta$, under the assumption that the probability of finding a lasso\ncounterexample is larger than $\\epsilon$. Extensive experimental evaluation\nshows that $\\mathsf{IMC}^2$ is a fast and reliable way to find counterexamples\nto B\\\"uchi automata inclusion.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02342v1",
            "date": "2020-07-05 13:55:19+00:00",
            "title": "Improving Chinese Segmentation-free Word Embedding With Unsupervised Association Measure",
            "authors": [
                "Yifan Zhang",
                "Maohua Wang",
                "Yongjian Huang",
                "Qianrong Gu"
            ],
            "abstract": "Recent work on segmentation-free word embedding(sembei) developed a new\npipeline of word embedding for unsegmentated language while avoiding\nsegmentation as a preprocessing step. However, too many noisy n-grams existing\nin the embedding vocabulary that do not have strong association strength\nbetween characters would limit the quality of learned word embedding. To deal\nwith this problem, a new version of segmentation-free word embedding model is\nproposed by collecting n-grams vocabulary via a novel unsupervised association\nmeasure called pointwise association with times information(PATI). Comparing\nwith the commonly used n-gram filtering method like frequency used in sembei\nand pointwise mutual information(PMI), the proposed method leverages more\nlatent information from the corpus and thus is able to collect more valid\nn-grams that have stronger cohesion as embedding targets in unsegmented\nlanguage data, such as Chinese texts. Further experiments on Chinese SNS data\nshow that the proposed model improves performance of word embedding in\ndownstream tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02366v1",
            "date": "2020-07-05 15:37:44+00:00",
            "title": "Starfish: A Prototype for Universal Preprocessing and Text-Embedded Programming",
            "authors": [
                "Vlado Keselj"
            ],
            "abstract": "We present a novel concept of universal text preprocessing and text-embedded\nprogramming (PTEP). Preprocessing and text-embedded programming has been widely\nused in programming languages and frameworks in a fragmented and mutually\nisolated way. The PTEP ideas can be found in the implementation of the \\TeX\\\ntypesetting system; they are prominent in PHP and similar web languages, and\nfinally they are used in the Jupyter data science framework. This paper\npresents this area of research and related work in a more unified framework,\nand we describe the implemented system Starfish that satisfies the following\nnovel principles of PTEP: universality, update and replace modes, flexiblity,\nconfigurability, and transparency. We describe the operating model and design\nof Starfish, which is an open-source system implementing universal\npreprocessing and text-embedded programming in Perl. The system is transparent\nand its design allows direct implementation in other programming languages as\nwell.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02375v1",
            "date": "2020-07-05 16:11:57+00:00",
            "title": "Auto-captions on GIF: A Large-scale Video-sentence Dataset for Vision-language Pre-training",
            "authors": [
                "Yingwei Pan",
                "Yehao Li",
                "Jianjie Luo",
                "Jun Xu",
                "Ting Yao",
                "Tao Mei"
            ],
            "abstract": "In this work, we present Auto-captions on GIF, which is a new large-scale\npre-training dataset for generic video understanding. All video-sentence pairs\nare created by automatically extracting and filtering video caption annotations\nfrom billions of web pages. Auto-captions on GIF dataset can be utilized to\npre-train the generic feature representation or encoder-decoder structure for\nvideo captioning, and other downstream tasks (e.g., sentence localization in\nvideos, video question answering, etc.) as well. We present a detailed analysis\nof Auto-captions on GIF dataset in comparison to existing video-sentence\ndatasets. We also provide an evaluation of a Transformer-based encoder-decoder\nstructure for vision-language pre-training, which is further adapted to video\ncaptioning downstream task and yields the compelling generalizability on\nMSR-VTT. The dataset is available at\n\\url{http://www.auto-video-captions.top/2020/dataset}.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.02540v1",
            "date": "2020-07-06 05:51:10+00:00",
            "title": "LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model",
            "authors": [
                "Shilei Liu",
                "Yu Guo",
                "Bochao Li",
                "Feiliang Ren"
            ],
            "abstract": "This paper describes our submission to subtask a and b of SemEval-2020 Task\n4. For subtask a, we use a ALBERT based model with improved input form to pick\nout the common sense statement from two statement candidates. For subtask b, we\nuse a multiple choice model enhanced by hint sentence mechanism to select the\nreason from given options about why a statement is against common sense.\nBesides, we propose a novel transfer learning strategy between subtasks which\nhelp improve the performance. The accuracy scores of our system are 95.6 / 94.9\non official test set and rank 7$^{th}$ / 2$^{nd}$ on Post-Evaluation\nleaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02609v2",
            "date": "2020-07-06 09:54:17+00:00",
            "title": "Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback",
            "authors": [
                "Carlos Gemmell",
                "Federico Rossetto",
                "Jeffrey Dalton"
            ],
            "abstract": "Tools capable of automatic code generation have the potential to augment\nprogrammer's capabilities. While straightforward code retrieval is incorporated\ninto many IDEs, an emerging area is explicit code generation. Code generation\nis currently approached as a Machine Translation task, with Recurrent Neural\nNetwork (RNN) based encoder-decoder architectures trained on code-description\npairs. In this work we introduce and study modern Transformer architectures for\nthis task. We further propose a new model called the Relevance Transformer that\nincorporates external knowledge using pseudo-relevance feedback. The Relevance\nTransformer biases the decoding process to be similar to existing retrieved\ncode while enforcing diversity. We perform experiments on multiple standard\nbenchmark datasets for code generation including Django, Hearthstone, and\nCoNaLa. The results show improvements over state-of-the-art methods based on\nBLEU evaluation. The Relevance Transformer model shows the potential of\nTransformer-based architectures for code generation and introduces a method of\nincorporating pseudo-relevance feedback during inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.02699v1",
            "date": "2020-06-27 10:18:14+00:00",
            "title": "On the Evolution of Programming Languages",
            "authors": [
                "K. R. Chowdhary"
            ],
            "abstract": "This paper attempts to connects the evolution of computer languages with the\nevolution of life, where the later has been dictated by \\emph{theory of\nevolution of species}, and tries to give supportive evidence that the new\nlanguages are more robust than the previous, carry-over the mixed features of\nolder languages, such that strong features gets added into them and weak\nfeatures of older languages gets removed. In addition, an analysis of most\nprominent programming languages is presented, emphasizing on how the features\nof existing languages have influenced the development of new programming\nlanguages. At the end, it suggests a set of experimental languages, which may\nrule the world of programming languages in the time of new multi-core\narchitectures.\n  Index terms- Programming languages' evolution, classifications of languages,\nfuture languages, scripting-languages.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03028v3",
            "date": "2020-07-06 19:22:44+00:00",
            "title": "Labeling of Multilingual Breast MRI Reports",
            "authors": [
                "Chen-Han Tsai",
                "Nahum Kiryati",
                "Eli Konen",
                "Miri Sklair-Levy",
                "Arnaldo Mayer"
            ],
            "abstract": "Medical reports are an essential medium in recording a patient's condition\nthroughout a clinical trial. They contain valuable information that can be\nextracted to generate a large labeled dataset needed for the development of\nclinical tools. However, the majority of medical reports are stored in an\nunregularized format, and a trained human annotator (typically a doctor) must\nmanually assess and label each case, resulting in an expensive and time\nconsuming procedure. In this work, we present a framework for developing a\nmultilingual breast MRI report classifier using a custom-built language\nrepresentation called LAMBR. Our proposed method overcomes practical challenges\nfaced in clinical settings, and we demonstrate improved performance in\nextracting labels from medical reports when compared with conventional\napproaches.",
            "journal ref": "Lecture Notes in Computer Science 12446 (2020) 233-241",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.03310v1",
            "date": "2020-07-07 09:49:47+00:00",
            "title": "DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue",
            "authors": [
                "Xiaoze Jiang",
                "Jing Yu",
                "Yajing Sun",
                "Zengchang Qin",
                "Zihao Zhu",
                "Yue Hu",
                "Qi Wu"
            ],
            "abstract": "Visual Dialogue task requires an agent to be engaged in a conversation with\nhuman about an image. The ability of generating detailed and non-repetitive\nresponses is crucial for the agent to achieve human-like conversation. In this\npaper, we propose a novel generative decoding architecture to generate\nhigh-quality responses, which moves away from decoding the whole encoded\nsemantics towards the design that advocates both transparency and flexibility.\nIn this architecture, word generation is decomposed into a series of\nattention-based information selection steps, performed by the novel recurrent\nDeliberation, Abandon and Memory (DAM) module. Each DAM module performs an\nadaptive combination of the response-level semantics captured from the encoder\nand the word-level semantics specifically selected for generating each word.\nTherefore, the responses contain more detailed and non-repetitive descriptions\nwhile maintaining the semantic accuracy. Furthermore, DAM is flexible to\ncooperate with existing visual dialogue encoders and adaptive to the encoder\nstructures by constraining the information selection mode in DAM. We apply DAM\nto three typical encoders and verify the performance on the VisDial v1.0\ndataset. Experimental results show that the proposed models achieve new\nstate-of-the-art performance with high-quality responses. The code is available\nat https://github.com/JXZe/DAM.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.03405v2",
            "date": "2020-07-07 13:16:19+00:00",
            "title": "Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature",
            "authors": [
                "Jong Won Park"
            ],
            "abstract": "The scientific community continues to publish an overwhelming amount of new\nresearch related to COVID-19 on a daily basis, leading to much literature\nwithout little to no attention. To aid the community in understanding the\nrapidly flowing array of COVID-19 literature, we propose a novel BERT\narchitecture that provides a brief yet original summarization of lengthy\npapers. The model continually learns on new data in online fashion while\nminimizing catastrophic forgetting, thus fitting to the need of the community.\nBenchmark and manual examination of its performance show that the model provide\na sound summary of new scientific literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03500v3",
            "date": "2020-07-07 14:37:27+00:00",
            "title": "The Go Transformer: Natural Language Modeling for Game Play",
            "authors": [
                "Matthew Ciolino",
                "David Noever",
                "Josh Kalin"
            ],
            "abstract": "This work applies natural language modeling to generate plausible strategic\nmoves in the ancient game of Go. We train the Generative Pretrained Transformer\n(GPT-2) to mimic the style of Go champions as archived in Smart Game Format\n(SGF), which offers a text description of move sequences. The trained model\nfurther generates valid but previously unseen strategies for Go. Because GPT-2\npreserves punctuation and spacing, the raw output of the text generator\nprovides inputs to game visualization and creative patterns, such as the Sabaki\nproject's game engine using auto-replays. Results demonstrate that language\nmodeling can capture both the sequencing format of championship Go games and\ntheir strategic formations. Compared to random game boards, the GPT-2\nfine-tuning shows efficient opening move sequences favoring corner play over\nless advantageous center and side play. Game generation as a language modeling\ntask offers novel approaches to more than 40 other board games where historical\ntext annotation provides training data (e.g., Amazons & Connect 4/6).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03771v1",
            "date": "2020-07-07 20:10:31+00:00",
            "title": "Cross-lingual Inductive Transfer to Detect Offensive Language",
            "authors": [
                "Kartikey Pant",
                "Tanvi Dadu"
            ],
            "abstract": "With the growing use of social media and its availability, many instances of\nthe use of offensive language have been observed across multiple languages and\ndomains. This phenomenon has given rise to the growing need to detect the\noffensive language used in social media cross-lingually. In OffensEval 2020,\nthe organizers have released the \\textit{multilingual Offensive Language\nIdentification Dataset} (mOLID), which contains tweets in five different\nlanguages, to detect offensive language. In this work, we introduce a\ncross-lingual inductive approach to identify the offensive language in tweets\nusing the contextual word embedding \\textit{XLM-RoBERTa} (XLM-R). We show that\nour model performs competitively on all five languages, obtaining the fourth\nposition in the English task with an F1-score of $0.919$ and eighth position in\nthe Turkish task with an F1-score of $0.781$. Further experimentation proves\nthat our model works competitively in a zero-shot learning environment, and is\nextensible to other languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.03848v2",
            "date": "2020-07-08 02:00:22+00:00",
            "title": "Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers",
            "authors": [
                "Shijie Geng",
                "Peng Gao",
                "Moitreya Chatterjee",
                "Chiori Hori",
                "Jonathan Le Roux",
                "Yongfeng Zhang",
                "Hongsheng Li",
                "Anoop Cherian"
            ],
            "abstract": "Given an input video, its associated audio, and a brief caption, the\naudio-visual scene aware dialog (AVSD) task requires an agent to indulge in a\nquestion-answer dialog with a human about the audio-visual content. This task\nthus poses a challenging multi-modal representation learning and reasoning\nscenario, advancements into which could influence several human-machine\ninteraction applications. To solve this task, we introduce a\nsemantics-controlled multi-modal shuffled Transformer reasoning framework,\nconsisting of a sequence of Transformer modules, each taking a modality as\ninput and producing representations conditioned on the input question. Our\nproposed Transformer variant uses a shuffling scheme on their multi-head\noutputs, demonstrating better regularization. To encode fine-grained visual\ninformation, we present a novel dynamic scene graph representation learning\npipeline that consists of an intra-frame reasoning layer producing\nspatio-semantic graph representations for every frame, and an inter-frame\naggregation module capturing temporal cues. Our entire pipeline is trained\nend-to-end. We present experiments on the benchmark AVSD dataset, both on\nanswer generation and selection tasks. Our results demonstrate state-of-the-art\nperformances on all evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.03909v3",
            "date": "2020-07-08 05:56:01+00:00",
            "title": "Best-First Beam Search",
            "authors": [
                "Clara Meister",
                "Tim Vieira",
                "Ryan Cotterell"
            ],
            "abstract": "Decoding for many NLP tasks requires an effective heuristic algorithm for\napproximating exact search since the problem of searching the full output space\nis often intractable, or impractical in many settings. The default algorithm\nfor this job is beam search -- a pruned version of breadth-first search. Quite\nsurprisingly, beam search often returns better results than exact inference due\nto beneficial search bias for NLP tasks. In this work, we show that the\nstandard implementation of beam search can be made up to 10x faster in\npractice. Our method assumes that the scoring function is monotonic in the\nsequence length, which allows us to safely prune hypotheses that cannot be in\nthe final set of hypotheses early on. We devise effective monotonic\napproximations to popular nonmonontic scoring functions, including length\nnormalization and mutual information decoding. Lastly, we propose a\nmemory-reduced variant of Best-First Beam Search, which has a similar\nbeneficial search bias in terms of downstream performance, but runs in a\nfraction of the time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04205v1",
            "date": "2020-07-08 15:46:13+00:00",
            "title": "Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets",
            "authors": [
                "Mar\u00eda Andrea Cruz Bland\u00f3n",
                "Okko R\u00e4s\u00e4nen"
            ],
            "abstract": "Neural network models using predictive coding are interesting from the\nviewpoint of computational modelling of human language acquisition, where the\nobjective is to understand how linguistic units could be learned from speech\nwithout any labels. Even though several promising predictive coding -based\nlearning algorithms have been proposed in the literature, it is currently\nunclear how well they generalise to different languages and training dataset\nsizes. In addition, despite that such models have shown to be effective\nphonemic feature learners, it is unclear whether minimisation of the predictive\nloss functions of these models also leads to optimal phoneme-like\nrepresentations. The present study investigates the behaviour of two predictive\ncoding models, Autoregressive Predictive Coding and Contrastive Predictive\nCoding, in a phoneme discrimination task (ABX task) for two languages with\ndifferent dataset sizes. Our experiments show a strong correlation between the\nautoregressive loss and the phoneme discrimination scores with the two\ndatasets. However, to our surprise, the CPC model shows rapid convergence\nalready after one pass over the training data, and, on average, its\nrepresentations outperform those of APC on both languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04247v1",
            "date": "2020-06-23 13:47:58+00:00",
            "title": "Neural relation extraction: a survey",
            "authors": [
                "Mehmet Aydar",
                "Ozge Bozal",
                "Furkan Ozbay"
            ],
            "abstract": "Neural relation extraction discovers semantic relations between entities from\nunstructured text using deep learning methods. In this study, we present a\ncomprehensive review of methods on neural network based relation extraction. We\ndiscuss advantageous and incompetent sides of existing studies and investigate\nadditional research directions and improvement ideas in this field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04248v1",
            "date": "2020-06-19 14:47:21+00:00",
            "title": "Chatbot: A Conversational Agent employed with Named Entity Recognition Model using Artificial Neural Network",
            "authors": [
                "Nazakat Ali"
            ],
            "abstract": "Chatbot is a technology that is used to mimic human behavior using natural\nlanguage. There are different types of Chatbot that can be used as\nconversational agent in various business domains in order to increase the\ncustomer service and satisfaction. For any business domain, it requires a\nknowledge base to be built for that domain and design an information retrieval\nbased system that can respond the user with a piece of documentation or\ngenerated sentences. The core component of a Chatbot is Natural Language\nUnderstanding (NLU) which has been impressively improved by deep learning\nmethods. But we often lack such properly built NLU modules and requires more\ntime to build it from scratch for high quality conversations. This may\nencourage fresh learners to build a Chatbot from scratch with simple\narchitecture and using small dataset, although it may have reduced\nfunctionality, rather than building high quality data driven methods. This\nresearch focuses on Named Entity Recognition (NER) and Intent Classification\nmodels which can be integrated into NLU service of a Chatbot. Named entities\nwill be inserted manually in the knowledge base and automatically detected in a\ngiven sentence. The NER model in the proposed architecture is based on\nartificial neural network which is trained on manually created entities and\nevaluated using CoNLL-2003 dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04298v2",
            "date": "2020-06-29 10:26:50+00:00",
            "title": "Building Interpretable Interaction Trees for Deep NLP Models",
            "authors": [
                "Die Zhang",
                "Huilin Zhou",
                "Hao Zhang",
                "Xiaoyi Bao",
                "Da Huo",
                "Ruizhao Chen",
                "Xu Cheng",
                "Mengyue Wu",
                "Quanshi Zhang"
            ],
            "abstract": "This paper proposes a method to disentangle and quantify interactions among\nwords that are encoded inside a DNN for natural language processing. We\nconstruct a tree to encode salient interactions extracted by the DNN. Six\nmetrics are proposed to analyze properties of interactions between constituents\nin a sentence. The interaction is defined based on Shapley values of words,\nwhich are considered as an unbiased estimation of word contributions to the\nnetwork prediction. Our method is used to quantify word interactions encoded\ninside the BERT, ELMo, LSTM, CNN, and Transformer networks. Experimental\nresults have provided a new perspective to understand these DNNs, and have\ndemonstrated the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.04422v1",
            "date": "2020-07-08 20:41:52+00:00",
            "title": "IQ-VQA: Intelligent Visual Question Answering",
            "authors": [
                "Vatsal Goel",
                "Mohit Chandak",
                "Ashish Anand",
                "Prithwijit Guha"
            ],
            "abstract": "Even though there has been tremendous progress in the field of Visual\nQuestion Answering, models today still tend to be inconsistent and brittle. To\nthis end, we propose a model-independent cyclic framework which increases\nconsistency and robustness of any VQA architecture. We train our models to\nanswer the original question, generate an implication based on the answer and\nthen also learn to answer the generated implication correctly. As a part of the\ncyclic framework, we propose a novel implication generator which can generate\nimplied questions from any question-answer pair. As a baseline for future works\non consistency, we provide a new human annotated VQA-Implications dataset. The\ndataset consists of ~30k questions containing implications of 3 types - Logical\nEquivalence, Necessary Condition and Mutual Exclusion - made from the VQA v2.0\nvalidation dataset. We show that our framework improves consistency of VQA\nmodels by ~15% on the rule-based dataset, ~7% on VQA-Implications dataset and\nrobustness by ~2%, without degrading their performance. In addition, we also\nquantitatively show improvement in attention maps which highlights better\nmulti-modal understanding of vision and language.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.05234v1",
            "date": "2020-07-10 08:15:59+00:00",
            "title": "Pragmatic information in translation: a corpus-based study of tense and mood in English and German",
            "authors": [
                "Anita Ramm",
                "Ekaterina Lapshinova-Koltunski",
                "Alexander Fraser"
            ],
            "abstract": "Grammatical tense and mood are important linguistic phenomena to consider in\nnatural language processing (NLP) research. We consider the correspondence\nbetween English and German tense and mood in translation. Human translators do\nnot find this correspondence easy, and as we will show through careful\nanalysis, there are no simplistic ways to map tense and mood from one language\nto another. Our observations about the challenges of human translation of tense\nand mood have important implications for multilingual NLP. Of particular\nimportance is the challenge of modeling tense and mood in rule-based,\nphrase-based statistical and neural machine translation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05302v2",
            "date": "2020-07-10 11:05:42+00:00",
            "title": "Topic Modeling on User Stories using Word Mover's Distance",
            "authors": [
                "Kim Julian G\u00fclle",
                "Nicholas Ford",
                "Patrick Ebel",
                "Florian Brokhausen",
                "Andreas Vogelsang"
            ],
            "abstract": "Requirements elicitation has recently been complemented with crowd-based\ntechniques, which continuously involve large, heterogeneous groups of users who\nexpress their feedback through a variety of media. Crowd-based elicitation has\ngreat potential for engaging with (potential) users early on but also results\nin large sets of raw and unstructured feedback. Consolidating and analyzing\nthis feedback is a key challenge for turning it into sensible user\nrequirements. In this paper, we focus on topic modeling as a means to identify\ntopics within a large set of crowd-generated user stories and compare three\napproaches: (1) a traditional approach based on Latent Dirichlet Allocation,\n(2) a combination of word embeddings and principal component analysis, and (3)\na combination of word embeddings and Word Mover's Distance. We evaluate the\napproaches on a publicly available set of 2,966 user stories written and\ncategorized by crowd workers. We found that a combination of word embeddings\nand Word Mover's Distance is most promising. Depending on the word embeddings\nwe use in our approaches, we manage to cluster the user stories in two ways:\none that is closer to the original categorization and another that allows new\ninsights into the dataset, e.g. to find potentially new categories.\nUnfortunately, no measure exists to rate the quality of our results\nobjectively. Still, our findings provide a basis for future work towards\nanalyzing crowd-sourced user stories.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05612v1",
            "date": "2020-07-10 21:11:46+00:00",
            "title": "Multi-Dialect Arabic BERT for Country-Level Dialect Identification",
            "authors": [
                "Bashar Talafha",
                "Mohammad Ali",
                "Muhy Eddin Za'ter",
                "Haitham Seelawi",
                "Ibraheem Tuffaha",
                "Mostafa Samir",
                "Wael Farhan",
                "Hussein T. Al-Natsheh"
            ],
            "abstract": "Arabic dialect identification is a complex problem for a number of inherent\nproperties of the language itself. In this paper, we present the experiments\nconducted, and the models developed by our competing team, Mawdoo3 AI, along\nthe way to achieving our winning solution to subtask 1 of the Nuanced Arabic\nDialect Identification (NADI) shared task. The dialect identification subtask\nprovides 21,000 country-level labeled tweets covering all 21 Arab countries. An\nunlabeled corpus of 10M tweets from the same domain is also presented by the\ncompetition organizers for optional use. Our winning solution itself came in\nthe form of an ensemble of different training iterations of our pre-trained\nBERT model, which achieved a micro-averaged F1-score of 26.78% on the subtask\nat hand. We publicly release the pre-trained language model component of our\nwinning solution under the name of Multi-dialect-Arabic-BERT model, for any\ninterested researcher out there.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05618v1",
            "date": "2020-07-10 21:35:17+00:00",
            "title": "GloVeInit at SemEval-2020 Task 1: Using GloVe Vector Initialization for Unsupervised Lexical Semantic Change Detection",
            "authors": [
                "Vaibhav Jain"
            ],
            "abstract": "This paper presents a vector initialization approach for the SemEval2020 Task\n1: Unsupervised Lexical Semantic Change Detection. Given two corpora belonging\nto different time periods and a set of target words, this task requires us to\nclassify whether a word gained or lost a sense over time (subtask 1) and to\nrank them on the basis of the changes in their word senses (subtask 2). The\nproposed approach is based on using Vector Initialization method to align GloVe\nembeddings. The idea is to consecutively train GloVe embeddings for both\ncorpora, while using the first model to initialize the second one. This paper\nis based on the hypothesis that GloVe embeddings are more suited for the Vector\nInitialization method than SGNS embeddings. It presents an intuitive reasoning\nbehind this hypothesis, and also talks about the impact of various factors and\nhyperparameters on the performance of the proposed approach. Our model ranks\n13th and 10th among 33 teams in the two subtasks. The implementation has been\nshared publicly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05727v1",
            "date": "2020-07-11 09:22:01+00:00",
            "title": "Feature Selection on Noisy Twitter Short Text Messages for Language Identification",
            "authors": [
                "Mohd Zeeshan Ansari",
                "Tanvir Ahmad",
                "Ana Fatima"
            ],
            "abstract": "The task of written language identification involves typically the detection\nof the languages present in a sample of text. Moreover, a sequence of text may\nnot belong to a single inherent language but also may be mixture of text\nwritten in multiple languages. This kind of text is generated in large volumes\nfrom social media platforms due to its flexible and user friendly environment.\nSuch text contains very large number of features which are essential for\ndevelopment of statistical, probabilistic as well as other kinds of language\nmodels. The large number of features have rich as well as irrelevant and\nredundant features which have diverse effect over the performance of the\nlearning model. Therefore, feature selection methods are significant in\nchoosing feature that are most relevant for an efficient model. In this\narticle, we basically consider the Hindi-English language identification task\nas Hindi and English are often two most widely spoken languages of India. We\napply different feature selection algorithms across various learning algorithms\nin order to analyze the effect of the algorithm as well as the number of\nfeatures on the performance of the task. The methodology focuses on the word\nlevel language identification using a novel dataset of 6903 tweets extracted\nfrom Twitter. Various n-gram profiles are examined with different feature\nselection algorithms over many classifiers. Finally, an exhaustive comparative\nanalysis is put forward with respect to the overall experiments conducted for\nthe task.",
            "journal ref": "International Journal of Recent Technology and Engineering,\n  Volume-8, Issue-4, Nov 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05872v1",
            "date": "2020-07-11 22:56:37+00:00",
            "title": "Is Machine Learning Speaking my Language? A Critical Look at the NLP-Pipeline Across 8 Human Languages",
            "authors": [
                "Esma Wali",
                "Yan Chen",
                "Christopher Mahoney",
                "Thomas Middleton",
                "Marzieh Babaeianjelodar",
                "Mariama Njie",
                "Jeanna Neefe Matthews"
            ],
            "abstract": "Natural Language Processing (NLP) is increasingly used as a key ingredient in\ncritical decision-making systems such as resume parsers used in sorting a list\nof job candidates. NLP systems often ingest large corpora of human text,\nattempting to learn from past human behavior and decisions in order to produce\nsystems that will make recommendations about our future world. Over 7000 human\nlanguages are being spoken today and the typical NLP pipeline underrepresents\nspeakers of most of them while amplifying the voices of speakers of other\nlanguages. In this paper, a team including speakers of 8 languages - English,\nChinese, Urdu, Farsi, Arabic, French, Spanish, and Wolof - takes a critical\nlook at the typical NLP pipeline and how even when a language is technically\nsupported, substantial caveats remain to prevent full participation. Despite\nhuge and admirable investments in multilingual support in many tools and\nresources, we are still making NLP-guided decisions that systematically and\ndramatically underrepresent the voices of much of the world.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.05976v1",
            "date": "2020-07-12 12:39:35+00:00",
            "title": "Stance Detection in Web and Social Media: A Comparative Study",
            "authors": [
                "Shalmoli Ghosh",
                "Prajwal Singhania",
                "Siddharth Singh",
                "Koustav Rudra",
                "Saptarshi Ghosh"
            ],
            "abstract": "Online forums and social media platforms are increasingly being used to\ndiscuss topics of varying polarities where different people take different\nstances. Several methodologies for automatic stance detection from text have\nbeen proposed in literature. To our knowledge, there has not been any\nsystematic investigation towards their reproducibility, and their comparative\nperformances. In this work, we explore the reproducibility of several existing\nstance detection models, including both neural models and classical\nclassifier-based models. Through experiments on two datasets -- (i)~the popular\nSemEval microblog dataset, and (ii)~a set of health-related online news\narticles -- we also perform a detailed comparative analysis of various methods\nand explore their shortcomings. Implementations of all algorithms discussed in\nthis paper are available at\nhttps://github.com/prajwal1210/Stance-Detection-in-Web-and-Social-Media.",
            "journal ref": "Proceedings of Conference and Labs of the Evaluation Forum (CLEF)\n  2019; Lecture Notes in Computer Science, vol 11696, pp. 75-87",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06198v2",
            "date": "2020-07-13 05:36:36+00:00",
            "title": "Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder",
            "authors": [
                "Gouthaman KV",
                "Anurag Mittal"
            ],
            "abstract": "Recent studies have shown that current VQA models are heavily biased on the\nlanguage priors in the train set to answer the question, irrespective of the\nimage. E.g., overwhelmingly answer \"what sport is\" as \"tennis\" or \"what color\nbanana\" as \"yellow.\" This behavior restricts them from real-world application\nscenarios. In this work, we propose a novel model-agnostic question encoder,\nVisually-Grounded Question Encoder (VGQE), for VQA that reduces this effect.\nVGQE utilizes both visual and language modalities equally while encoding the\nquestion. Hence the question representation itself gets sufficient\nvisual-grounding, and thus reduces the dependency of the model on the language\npriors. We demonstrate the effect of VGQE on three recent VQA models and\nachieve state-of-the-art results on the bias-sensitive split of the VQAv2\ndataset; VQA-CPv2. Further, unlike the existing bias-reduction techniques, on\nthe standard VQAv2 benchmark, our approach does not drop the accuracy; instead,\nit improves the performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.06258v1",
            "date": "2020-07-13 09:22:59+00:00",
            "title": "A theory of interaction semantics",
            "authors": [
                "Johannes Reich"
            ],
            "abstract": "The aim of this article is to delineate a theory of interaction semantics and\nthereby provide a proper understanding of the \"meaning\" of the exchanged\ncharacters within an interaction. The idea is to describe the interaction\n(between discrete systems) by a mechanism that depends on information exchange,\nthat is, on the identical naming of the \"exchanged\" characters -- by a\nprotocol. Complementing a nondeterministic protocol with decisions to a game in\nits interactive form (GIF) makes it interpretable in the sense of an execution.\nThe consistency of such a protocol depends on the particular choice of its sets\nof characters. Thus, assigning a protocol its sets of charaacters makes it\nconsistent or not, creating a fulfillment relation. The interpretation of the\ncharacters during GIF execution results in their meaning. The proposed theory\nof interaction semantics is consistent with the model of information transport\nand processing, it has a clear relation to models of formal semantics, it\naccounts for the fact that the meaning of a character is invariant against\nrenaming and locates the concept of meaning in the technical description of\ninteractions. It defines when two different characters have the same meaning\nand what an \"interpretation\" and what an \"interpretation context\" is as well as\nunder which conditions meaning is compositional.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2007.06351v1",
            "date": "2020-07-13 12:42:43+00:00",
            "title": "A Label Attention Model for ICD Coding from Clinical Text",
            "authors": [
                "Thanh Vu",
                "Dat Quoc Nguyen",
                "Anthony Nguyen"
            ],
            "abstract": "ICD coding is a process of assigning the International Classification of\nDisease diagnosis codes to clinical/medical notes documented by health\nprofessionals (e.g. clinicians). This process requires significant human\nresources, and thus is costly and prone to error. To handle the problem,\nmachine learning has been utilized for automatic ICD coding. Previous\nstate-of-the-art models were based on convolutional neural networks, using a\nsingle/several fixed window sizes. However, the lengths and interdependence\nbetween text fragments related to ICD codes in clinical text vary\nsignificantly, leading to the difficulty of deciding what the best window sizes\nare. In this paper, we propose a new label attention model for automatic ICD\ncoding, which can handle both the various lengths and the interdependence of\nthe ICD code related text fragments. Furthermore, as the majority of ICD codes\nare not frequently used, leading to the extremely imbalanced data issue, we\nadditionally propose a hierarchical joint learning mechanism extending our\nlabel attention model to handle the issue, using the hierarchical relationships\namong the codes. Our label attention model achieves new state-of-the-art\nresults on three benchmark MIMIC datasets, and the joint learning mechanism\nhelps improve the performances for infrequent codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06511v2",
            "date": "2020-07-13 17:04:44+00:00",
            "title": "An Enhanced Text Classification to Explore Health based Indian Government Policy Tweets",
            "authors": [
                "Aarzoo Dhiman",
                "Durga Toshniwal"
            ],
            "abstract": "Government-sponsored policy-making and scheme generations is one of the means\nof protecting and promoting the social, economic, and personal development of\nthe citizens. The evaluation of effectiveness of these schemes done by\ngovernment only provide the statistical information in terms of facts and\nfigures which do not include the in-depth knowledge of public perceptions,\nexperiences and views on the topic. In this research work, we propose an\nimproved text classification framework that classifies the Twitter data of\ndifferent health-based government schemes. The proposed framework leverages the\nlanguage representation models (LR models) BERT, ELMO, and USE. However, these\nLR models have less real-time applicability due to the scarcity of the ample\nannotated data. To handle this, we propose a novel GloVe word embeddings and\nclass-specific sentiments based text augmentation approach (named Mod-EDA)\nwhich boosts the performance of text classification task by increasing the size\nof labeled data. Furthermore, the trained model is leveraged to identify the\nlevel of engagement of citizens towards these policies in different communities\nsuch as middle-income and low-income groups.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06778v3",
            "date": "2020-07-14 02:34:59+00:00",
            "title": "An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models",
            "authors": [
                "Lifu Tu",
                "Garima Lalwani",
                "Spandana Gella",
                "He He"
            ],
            "abstract": "Recent work has shown that pre-trained language models such as BERT improve\nrobustness to spurious correlations in the dataset. Intrigued by these results,\nwe find that the key to their success is generalization from a small amount of\ncounterexamples where the spurious correlations do not hold. When such minority\nexamples are scarce, pre-trained models perform as poorly as models trained\nfrom scratch. In the case of extreme minority, we propose to use multi-task\nlearning (MTL) to improve generalization. Our experiments on natural language\ninference and paraphrase identification show that MTL with the right auxiliary\ntasks significantly improves performance on challenging examples without\nhurting the in-distribution performance. Further, we show that the gain from\nMTL mainly comes from improved generalization from the minority examples. Our\nresults highlight the importance of data diversity for overcoming spurious\ncorrelations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06796v3",
            "date": "2020-07-14 03:49:43+00:00",
            "title": "Calling Out Bluff: Attacking the Robustness of Automatic Scoring Systems with Simple Adversarial Testing",
            "authors": [
                "Yaman Kumar",
                "Mehar Bhatia",
                "Anubha Kabra",
                "Jessy Junyi Li",
                "Di Jin",
                "Rajiv Ratn Shah"
            ],
            "abstract": "A significant progress has been made in deep-learning based Automatic Essay\nScoring (AES) systems in the past two decades. The performance commonly\nmeasured by the standard performance metrics like Quadratic Weighted Kappa\n(QWK), and accuracy points to the same. However, testing on common-sense\nadversarial examples of these AES systems reveal their lack of natural language\nunderstanding capability. Inspired by common student behaviour during\nexaminations, we propose a task agnostic adversarial evaluation scheme for AES\nsystems to test their natural language understanding capabilities and overall\nrobustness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06897v1",
            "date": "2020-07-14 08:14:00+00:00",
            "title": "What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?",
            "authors": [
                "Sriram Balasubramanian",
                "Naman Jain",
                "Gaurav Jindal",
                "Abhijeet Awasthi",
                "Sunita Sarawagi"
            ],
            "abstract": "We evaluate named entity representations of BERT-based NLP models by\ninvestigating their robustness to replacements from the same typed class in the\ninput. We highlight that on several tasks while such perturbations are natural,\nstate of the art trained models are surprisingly brittle. The brittleness\ncontinues even with the recent entity-aware BERT models. We also try to discern\nthe cause of this non-robustness, considering factors such as tokenization and\nfrequency of occurrence. Then we provide a simple method that ensembles\npredictions from multiple replacements while jointly modeling the uncertainty\nof type annotations and label predictions. Experiments on three NLP tasks show\nthat our method enhances robustness and increases accuracy on both natural and\nadversarial datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06908v1",
            "date": "2020-07-14 08:38:37+00:00",
            "title": "Language, communication and society: a gender based linguistics analysis",
            "authors": [
                "P. Cutugno",
                "D. Chiarella",
                "R. Lucentini",
                "L. Marconi",
                "G. Morgavi"
            ],
            "abstract": "The purpose of this study is to find evidence for supporting the hypothesis\nthat language is the mirror of our thinking, our prejudices and cultural\nstereotypes. In this analysis, a questionnaire was administered to 537 people.\nThe answers have been analysed to see if gender stereotypes were present such\nas the attribution of psychological and behavioural characteristics. In\nparticular, the aim was to identify, if any, what are the stereotyped images,\nwhich emerge in defining the roles of men and women in modern society.\nMoreover, the results given can be a good starting point to understand if\ngender stereotypes, and the expectations they produce, can result in\npenalization or inequality. If so, the language and its use would create\ninherently a gender bias, which influences evaluations both in work settings\nboth in everyday life.",
            "journal ref": "Recent Advances in Electrical Engineering Series Volume 50, pag.\n  154-160, 2015, WSEAS Press. Athens ISBN: 9781618043184 ISSN: 1790-5117",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06915v1",
            "date": "2020-07-14 08:52:55+00:00",
            "title": "Questionnaire analysis to define the most suitable survey for port-noise investigation",
            "authors": [
                "Andrea Cerniglia",
                "Davide Chiarella",
                "Paola Cutugno",
                "Lucia Marconi",
                "Anna Magrini",
                "Gelsomina Di Feo",
                "Melissa Ferretti"
            ],
            "abstract": "The high level of noise pollution affecting the areas between ports and\nlogistic platforms represents a problem that can be faced from different points\nof view. Acoustic monitoring, mapping, short-term measurements, port and road\ntraffic flows analyses can give useful indications on the strategies to be\nproposed for a better management of the problem. A survey campaign through the\npreparation of questionnaires to be submitted to the population exposed to\nnoise in the back-port areas will help to better understand the subjective\npoint of view. The paper analyses a sample of questions suitable for the\nspecific research, chosen as part of the wide database of questionnaires\ninternationally proposed for subjective investigations. The preliminary results\nof a first data collection campaign are considered to verify the adequacy of\nthe number, the type of questions, and the type of sample noise used for the\nsurvey. The questionnaire will be optimized to be distributed in the TRIPLO\nproject (TRansports and Innovative sustainable connections between Ports and\nLOgistic platforms). The results of this survey will be the starting point for\nthe linguistic investigation carried out in combination with the acoustic\nmonitoring, to improve understanding the connections between personal feeling\nand technical aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06943v1",
            "date": "2020-07-14 09:59:38+00:00",
            "title": "Modeling Voting for System Combination in Machine Translation",
            "authors": [
                "Xuancheng Huang",
                "Jiacheng Zhang",
                "Zhixing Tan",
                "Derek F. Wong",
                "Huanbo Luan",
                "Jingfang Xu",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "System combination is an important technique for combining the hypotheses of\ndifferent machine translation systems to improve translation performance.\nAlthough early statistical approaches to system combination have been proven\neffective in analyzing the consensus between hypotheses, they suffer from the\nerror propagation problem due to the use of pipelines. While this problem has\nbeen alleviated by end-to-end training of multi-source sequence-to-sequence\nmodels recently, these neural models do not explicitly analyze the relations\nbetween hypotheses and fail to capture their agreement because the attention to\na word in a hypothesis is calculated independently, ignoring the fact that the\nword might occur in multiple hypotheses. In this work, we propose an approach\nto modeling voting for system combination in machine translation. The basic\nidea is to enable words in hypotheses from different systems to vote on words\nthat are representative and should get involved in the generation process. This\ncan be done by quantifying the influence of each voter and its preference for\neach candidate. Our approach combines the advantages of statistical and neural\nmethods since it can not only analyze the relations between hypotheses but also\nallow for end-to-end training. Experiments show that our approach is capable of\nbetter taking advantage of the consensus between hypotheses and achieves\nsignificant improvements over state-of-the-art baselines on Chinese-English and\nEnglish-German machine translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.06949v3",
            "date": "2020-07-14 10:22:05+00:00",
            "title": "Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR",
            "authors": [
                "Bal\u00e1zs Tarj\u00e1n",
                "Gy\u00f6rgy Szasz\u00e1k",
                "Tibor Fegy\u00f3",
                "P\u00e9ter Mihajlik"
            ],
            "abstract": "Recently Deep Transformer models have proven to be particularly powerful in\nlanguage modeling tasks for ASR. Their high complexity, however, makes them\nvery difficult to apply in the first (single) pass of an online system. Recent\nstudies showed that a considerable part of the knowledge of neural network\nLanguage Models (LM) can be transferred to traditional n-grams by using neural\ntext generation based data augmentation. In our paper, we pre-train a GPT-2\nTransformer LM on a general text corpus and fine-tune it on our Hungarian\nconversational call center ASR task. We show that although data augmentation\nwith Transformer-generated text works well for isolating languages, it causes a\nvocabulary explosion in a morphologically rich language. Therefore, we propose\na new method called subword-based neural text augmentation, where we retokenize\nthe generated text into statistically derived subwords. We compare Morfessor\nand BPE statistical subword tokenizers and show that both methods can\nsignificantly improve the WER while greatly reducing vocabulary size and memory\nrequirements. Finally, we also demonstrate that subword-based neural text\naugmentation outperforms the word-based approach not only in terms of overall\nWER but also in recognition of OOV words.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2007.06954v6",
            "date": "2020-07-14 10:30:47+00:00",
            "title": "Global Reactions to COVID-19 on Twitter: A Labelled Dataset with Latent Topic, Sentiment and Emotion Attributes",
            "authors": [
                "Raj Kumar Gupta",
                "Ajay Vishwanath",
                "Yinping Yang"
            ],
            "abstract": "This paper presents a large, labelled dataset on people's responses and\nexpressions related to the COVID-19 pandemic over the Twitter platform. From 28\nJanuary 2020 to 1 Jan 2021, we retrieved over 132 million public Twitter posts\n(i.e., tweets) from more than 20 million unique users using four keywords:\n\"corona\", \"wuhan\", \"nCov\" and \"covid\". Leveraging natural language processing\ntechniques and pre-trained machine learning-based emotion analytic algorithms,\nwe labelled each tweet with seventeen latent semantic attributes, including a)\nten binary attributes indicating the tweet's relevance or irrelevance to the\ntop ten detected topics, b) five quantitative emotion intensity attributes\nindicating the degree of intensity of the valence or sentiment (from extremely\nnegative to extremely positive), and the degree of intensity of fear, of anger,\nof sadness and of joy emotions (from barely noticeable to extremely high\nintensity), and c) two qualitative attributes indicating the sentiment category\nand the dominant emotion category the tweet is mainly expressing. We report the\ndescriptive statistics around the topic, sentiment and emotion attributes, and\ntheir temporal distributions, and discuss the dataset's possible usage in\ncommunication, psychology, public health, economics, and epidemiology research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07196v1",
            "date": "2020-07-11 16:04:30+00:00",
            "title": "Investigation of Sentiment Controllable Chatbot",
            "authors": [
                "Hung-yi Lee",
                "Cheng-Hao Ho",
                "Chien-Fu Lin",
                "Chiung-Chih Chang",
                "Chih-Wei Lee",
                "Yau-Shian Wang",
                "Tsung-Yuan Hsu",
                "Kuan-Yu Chen"
            ],
            "abstract": "Conventional seq2seq chatbot models attempt only to find sentences with the\nhighest probabilities conditioned on the input sequences, without considering\nthe sentiment of the output sentences. In this paper, we investigate four\nmodels to scale or adjust the sentiment of the chatbot response: a\npersona-based model, reinforcement learning, a plug and play model, and\nCycleGAN, all based on the seq2seq model. We also develop machine-evaluated\nmetrics to estimate whether the responses are reasonable given the input. These\nmetrics, together with human evaluation, are used to analyze the performance of\nthe four models in terms of different aspects; reinforcement learning and\nCycleGAN are shown to be very attractive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07287v1",
            "date": "2020-07-14 18:29:49+00:00",
            "title": "Using Holographically Compressed Embeddings in Question Answering",
            "authors": [
                "Salvador E. Barbosa"
            ],
            "abstract": "Word vector representations are central to deep learning natural language\nprocessing models. Many forms of these vectors, known as embeddings, exist,\nincluding word2vec and GloVe. Embeddings are trained on large corpora and learn\nthe word's usage in context, capturing the semantic relationship between words.\nHowever, the semantics from such training are at the level of distinct words\n(known as word types), and can be ambiguous when, for example, a word type can\nbe either a noun or a verb. In question answering, parts-of-speech and named\nentity types are important, but encoding these attributes in neural models\nexpands the size of the input. This research employs holographic compression of\npre-trained embeddings, to represent a token, its part-of-speech, and named\nentity type, in the same dimension as representing only the token. The\nimplementation, in a modified question answering recurrent deep learning\nnetwork, shows that semantic relationships are preserved, and yields strong\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07318v2",
            "date": "2020-07-14 19:31:25+00:00",
            "title": "Deep learning models for representing out-of-vocabulary words",
            "authors": [
                "Johannes V. Lochter",
                "Renato M. Silva",
                "Tiago A. Almeida"
            ],
            "abstract": "Communication has become increasingly dynamic with the popularization of\nsocial networks and applications that allow people to express themselves and\ncommunicate instantly. In this scenario, distributed representation models have\ntheir quality impacted by new words that appear frequently or that are derived\nfrom spelling errors. These words that are unknown by the models, known as\nout-of-vocabulary (OOV) words, need to be properly handled to not degrade the\nquality of the natural language processing (NLP) applications, which depend on\nthe appropriate vector representation of the texts. To better understand this\nproblem and finding the best techniques to handle OOV words, in this study, we\npresent a comprehensive performance evaluation of deep learning models for\nrepresenting OOV words. We performed an intrinsic evaluation using a benchmark\ndataset and an extrinsic evaluation using different NLP tasks: text\ncategorization, named entity recognition, and part-of-speech tagging. Although\nthe results indicated that the best technique for handling OOV words is\ndifferent for each task, Comick, a deep learning method that infers the\nembedding based on the context and the morphological structure of the OOV word,\nobtained promising results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07670v1",
            "date": "2020-07-15 13:01:44+00:00",
            "title": "Logic Constrained Pointer Networks for Interpretable Textual Similarity",
            "authors": [
                "Subhadeep Maji",
                "Rohan Kumar",
                "Manish Bansal",
                "Kalyani Roy",
                "Pawan Goyal"
            ],
            "abstract": "Systematically discovering semantic relationships in text is an important and\nextensively studied area in Natural Language Processing, with various tasks\nsuch as entailment, semantic similarity, etc. Decomposability of sentence-level\nscores via subsequence alignments has been proposed as a way to make models\nmore interpretable. We study the problem of aligning components of sentences\nleading to an interpretable model for semantic textual similarity. In this\npaper, we introduce a novel pointer network based model with a sentinel gating\nfunction to align constituent chunks, which are represented using BERT. We\nimprove this base model with a loss function to equally penalize misalignments\nin both sentences, ensuring the alignments are bidirectional. Finally, to guide\nthe network with structured external knowledge, we introduce first-order logic\nconstraints based on ConceptNet and syntactic knowledge. The model achieves an\nF1 score of 97.73 and 96.32 on the benchmark SemEval datasets for the chunk\nalignment task, showing large improvements over the existing solutions. Source\ncode is available at\nhttps://github.com/manishb89/interpretable_sentence_similarity",
            "journal ref": "IJCAI 2020, Pages 2405-2411",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07758v2",
            "date": "2020-07-15 15:34:35+00:00",
            "title": "Multimodal Word Sense Disambiguation in Creative Practice",
            "authors": [
                "Manuel Ladron de Guevara",
                "Christopher George",
                "Akshat Gupta",
                "Daragh Byrne",
                "Ramesh Krishnamurti"
            ],
            "abstract": "Language is ambiguous; many terms and expressions can convey the same idea.\nThis is especially true in creative practice, where ideas and design intents\nare highly subjective. We present a dataset, Ambiguous Descriptions of Art\nImages (ADARI), of contemporary workpieces, which aims to provide a\nfoundational resource for subjective image description and multimodal word\ndisambiguation in the context of creative practice. The dataset contains a\ntotal of 240k images labeled with 260k descriptive sentences. It is\nadditionally organized into sub-domains of architecture, art, design, fashion,\nfurniture, product design and technology. In subjective image description,\nlabels are not deterministic: for example, the ambiguous label dynamic might\ncorrespond to hundreds of different images. To understand this complexity, we\nanalyze the ambiguity and relevance of text with respect to images using the\nstate-of-the-art pre-trained BERT model for sentence classification. We provide\na baseline for multi-label classification tasks and demonstrate the potential\nof multimodal approaches for understanding ambiguity in design intentions. We\nhope that ADARI dataset and baselines constitute a first step towards\nsubjective label classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.07825v1",
            "date": "2020-07-14 17:53:01+00:00",
            "title": "Intelligent requirements engineering from natural language and their chaining toward CAD models",
            "authors": [
                "Alain-J\u00e9r\u00f4me Foug\u00e8res",
                "Egon Ostrosi"
            ],
            "abstract": "This paper assumes that design language plays an important role in how\ndesigners design and on the creativity of designers. Designers use and develop\nmodels as an aid to thinking, a focus for discussion and decision-making and a\nmeans of evaluating the reliability of the proposals. This paper proposes an\nintelligent method for requirements engineering from natural language and their\nchaining toward CAD models. The transition from linguistic analysis to the\nrepresentation of engineering requirements consists of the translation of the\nsyntactic structure into semantic form represented by conceptual graphs. Based\non the isomorphism between conceptual graphs and predicate logic, a formal\nlanguage of the specification is proposed. The outcome of this language is\nchained and translated in Computer Aided Three-Dimensional Interactive\nApplication (CATIA) models. The tool (EGEON: Engineering desiGn sEmantics\nelabOration and applicatioN) is developed to represent the semantic network of\nengineering requirements. A case study on the design of a car door hinge is\npresented to illustrates the proposed method.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2007.07846v1",
            "date": "2020-07-14 16:26:01+00:00",
            "title": "Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset",
            "authors": [
                "Edwin Zhang",
                "Nikhil Gupta",
                "Raphael Tang",
                "Xiao Han",
                "Ronak Pradeep",
                "Kuang Lu",
                "Yue Zhang",
                "Rodrigo Nogueira",
                "Kyunghyun Cho",
                "Hui Fang",
                "Jimmy Lin"
            ],
            "abstract": "We present Covidex, a search engine that exploits the latest neural ranking\nmodels to provide information access to the COVID-19 Open Research Dataset\ncurated by the Allen Institute for AI. Our system has been online and serving\nusers since late March 2020. The Covidex is the user application component of\nour three-pronged strategy to develop technologies for helping domain experts\ntackle the ongoing global pandemic. In addition, we provide robust and\neasy-to-use keyword search infrastructure that exploits mature fusion-based\nmethods as well as standalone neural ranking models that can be incorporated\ninto other applications. These techniques have been evaluated in the ongoing\nTREC-COVID challenge: Our infrastructure and baselines have been adopted by\nmany participants, including some of the highest-scoring runs in rounds 1, 2,\nand 3. In round 3, we report the highest-scoring run that takes advantage of\nprevious training data and the second-highest fully automatic run.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.07860v2",
            "date": "2020-07-15 17:24:41+00:00",
            "title": "Dialect Diversity in Text Summarization on Twitter",
            "authors": [
                "Vijay Keswani",
                "L. Elisa Celis"
            ],
            "abstract": "Discussions on Twitter involve participation from different communities with\ndifferent dialects and it is often necessary to summarize a large number of\nposts into a representative sample to provide a synopsis. Yet, any such\nrepresentative sample should sufficiently portray the underlying dialect\ndiversity to present the voices of different participating communities\nrepresenting the dialects. Extractive summarization algorithms perform the task\nof constructing subsets that succinctly capture the topic of any given set of\nposts. However, we observe that there is dialect bias in the summaries\ngenerated by common summarization approaches, i.e., they often return summaries\nthat under-represent certain dialects.\n  The vast majority of existing \"fair\" summarization approaches require\nsocially salient attribute labels (in this case, dialect) to ensure that the\ngenerated summary is fair with respect to the socially salient attribute.\nNevertheless, in many applications, these labels do not exist. Furthermore, due\nto the ever-evolving nature of dialects in social media, it is unreasonable to\nlabel or accurately infer the dialect of every social media post. To correct\nfor the dialect bias, we employ a framework that takes an existing text\nsummarization algorithm as a blackbox and, using a small set of dialect-diverse\nsentences, returns a summary that is relatively more dialect-diverse.\nCrucially, this approach does not need the posts being summarized to have\ndialect labels, ensuring that the diversification process is independent of\ndialect classification/identification models. We show the efficacy of our\napproach on Twitter datasets containing posts written in dialects used by\ndifferent social groups defined by race or gender; in all cases, our approach\nleads to improved dialect diversity compared to standard text summarization\napproaches.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2007.07884v1",
            "date": "2020-07-15 17:57:56+00:00",
            "title": "Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook",
            "authors": [
                "Yudhanjaya Wijeratne",
                "Nisansa de Silva"
            ],
            "abstract": "This paper presents two colloquial Sinhala language corpora from the language\nefforts of the Data, Analysis and Policy team of LIRNEasia, as well as a list\nof algorithmically derived stopwords. The larger of the two corpora spans 2010\nto 2020 and contains 28,825,820 to 29,549,672 words of multilingual text posted\nby 533 Sri Lankan Facebook pages, including politics, media, celebrities, and\nother categories; the smaller corpus amounts to 5,402,76 words of only Sinhala\ntext extracted from the larger. Both corpora have markers for their date of\ncreation, page of origin, and content type.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08100v1",
            "date": "2020-07-16 04:22:30+00:00",
            "title": "Towards Debiasing Sentence Representations",
            "authors": [
                "Paul Pu Liang",
                "Irene Mengze Li",
                "Emily Zheng",
                "Yao Chong Lim",
                "Ruslan Salakhutdinov",
                "Louis-Philippe Morency"
            ],
            "abstract": "As natural language processing methods are increasingly deployed in\nreal-world scenarios such as healthcare, legal systems, and social science, it\nbecomes necessary to recognize the role they potentially play in shaping social\nbiases and stereotypes. Previous work has revealed the presence of social\nbiases in widely used word embeddings involving gender, race, religion, and\nother social constructs. While some methods were proposed to debias these\nword-level embeddings, there is a need to perform debiasing at the\nsentence-level given the recent shift towards new contextualized sentence\nrepresentations such as ELMo and BERT. In this paper, we investigate the\npresence of social biases in sentence-level representations and propose a new\nmethod, Sent-Debias, to reduce these biases. We show that Sent-Debias is\neffective in removing biases, and at the same time, preserves performance on\nsentence-level downstream tasks such as sentiment analysis, linguistic\nacceptability, and natural language understanding. We hope that our work will\ninspire future research on characterizing and removing social biases from\nwidely adopted sentence representations for fairer NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08186v2",
            "date": "2020-07-16 08:54:17+00:00",
            "title": "Coupling Distant Annotation and Adversarial Training for Cross-Domain Chinese Word Segmentation",
            "authors": [
                "Ning Ding",
                "Dingkun Long",
                "Guangwei Xu",
                "Muhua Zhu",
                "Pengjun Xie",
                "Xiaobin Wang",
                "Hai-Tao Zheng"
            ],
            "abstract": "Fully supervised neural approaches have achieved significant progress in the\ntask of Chinese word segmentation (CWS). Nevertheless, the performance of\nsupervised models tends to drop dramatically when they are applied to\nout-of-domain data. Performance degradation is caused by the distribution gap\nacross domains and the out of vocabulary (OOV) problem. In order to\nsimultaneously alleviate these two issues, this paper proposes to couple\ndistant annotation and adversarial training for cross-domain CWS. For distant\nannotation, we rethink the essence of \"Chinese words\" and design an automatic\ndistant annotation mechanism that does not need any supervision or pre-defined\ndictionaries from the target domain. The approach could effectively explore\ndomain-specific words and distantly annotate the raw texts for the target\ndomain. For adversarial training, we develop a sentence-level training\nprocedure to perform noise reduction and maximum utilization of the source\ndomain information. Experiments on multiple real-world datasets across various\ndomains show the superiority and robustness of our model, significantly\noutperforming previous state-of-the-art cross-domain CWS methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08749v3",
            "date": "2020-07-17 04:19:30+00:00",
            "title": "Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations",
            "authors": [
                "Benjamin Schloss",
                "Sandeep Konam"
            ],
            "abstract": "Summaries generated from medical conversations can improve recall and\nunderstanding of care plans for patients and reduce documentation burden for\ndoctors. Recent advancements in automatic speech recognition (ASR) and natural\nlanguage understanding (NLU) offer potential solutions to generate these\nsummaries automatically, but rigorous quantitative baselines for benchmarking\nresearch in this domain are lacking. In this paper, we bridge this gap for two\ntasks: classifying utterances from medical conversations according to (i) the\nSOAP section and (ii) the speaker role. Both are fundamental building blocks\nalong the path towards an end-to-end, automated SOAP note for medical\nconversations. We provide details on a dataset that contains human and ASR\ntranscriptions of medical conversations and corresponding machine learning\noptimized SOAP notes. We then present a systematic analysis in which we adapt\nan existing deep learning architecture to the two aforementioned tasks. The\nresults suggest that modelling context in a hierarchical manner, which captures\nboth word and utterance level context, yields substantial improvements on both\nclassification tasks. Additionally, we develop and analyze a modular method for\nadapting our model to ASR output.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08751v1",
            "date": "2020-07-17 04:26:38+00:00",
            "title": "Knowledge-Based Video Question Answering with Unsupervised Scene Descriptions",
            "authors": [
                "Noa Garcia",
                "Yuta Nakashima"
            ],
            "abstract": "To understand movies, humans constantly reason over the dialogues and actions\nshown in specific scenes and relate them to the overall storyline already seen.\nInspired by this behaviour, we design ROLL, a model for knowledge-based video\nstory question answering that leverages three crucial aspects of movie\nunderstanding: dialog comprehension, scene reasoning, and storyline recalling.\nIn ROLL, each of these tasks is in charge of extracting rich and diverse\ninformation by 1) processing scene dialogues, 2) generating unsupervised video\nscene descriptions, and 3) obtaining external knowledge in a weakly supervised\nfashion. To answer a given question correctly, the information generated by\neach inspired-cognitive task is encoded via Transformers and fused through a\nmodality weighting mechanism, which balances the information from the different\nsources. Exhaustive evaluation demonstrates the effectiveness of our approach,\nwhich yields a new state-of-the-art on two challenging video question answering\ndatasets: KnowIT VQA and TVQA+.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.08772v1",
            "date": "2020-07-17 06:06:54+00:00",
            "title": "Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation",
            "authors": [
                "Jinglin Liu",
                "Yi Ren",
                "Xu Tan",
                "Chen Zhang",
                "Tao Qin",
                "Zhou Zhao",
                "Tie-Yan Liu"
            ],
            "abstract": "Non-autoregressive translation (NAT) achieves faster inference speed but at\nthe cost of worse accuracy compared with autoregressive translation (AT). Since\nAT and NAT can share model structure and AT is an easier task than NAT due to\nthe explicit dependency on previous target-side tokens, a natural idea is to\ngradually shift the model training from the easier AT task to the harder NAT\ntask. To smooth the shift from AT training to NAT training, in this paper, we\nintroduce semi-autoregressive translation (SAT) as intermediate tasks. SAT\ncontains a hyperparameter k, and each k value defines a SAT task with different\ndegrees of parallelism. Specially, SAT covers AT and NAT as its special cases:\nit reduces to AT when k = 1 and to NAT when k = N (N is the length of target\nsentence). We design curriculum schedules to gradually shift k from 1 to N,\nwith different pacing functions and number of tasks trained at the same time.\nWe called our method as task-level curriculum learning for NAT (TCL-NAT).\nExperiments on IWSLT14 De-En, IWSLT16 En-De, WMT14 En-De and De-En datasets\nshow that TCL-NAT achieves significant accuracy improvements over previous NAT\nbaselines and reduces the performance gap between NAT and AT models to 1-2 BLEU\npoints, demonstrating the effectiveness of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.08970v2",
            "date": "2020-07-17 13:34:49+00:00",
            "title": "Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures",
            "authors": [
                "Daniel Furrer",
                "Marc van Zee",
                "Nathan Scales",
                "Nathanael Sch\u00e4rli"
            ],
            "abstract": "While mainstream machine learning methods are known to have limited ability\nto compositionally generalize, new architectures and techniques continue to be\nproposed to address this limitation. We investigate state-of-the-art techniques\nand architectures in order to assess their effectiveness in improving\ncompositional generalization in semantic parsing tasks based on the SCAN and\nCFQ datasets. We show that masked language model (MLM) pre-training rivals\nSCAN-inspired architectures on primitive holdout splits. On a more complex\ncompositional task, we show that pre-training leads to significant improvements\nin performance vs. comparable non-pre-trained models, whereas architectures\nproposed to encourage compositional generalization on SCAN or in the area of\nalgorithm learning fail to lead to significant improvements. We establish a new\nstate of the art on the CFQ compositional generalization benchmark using MLM\npre-training together with an intermediate representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09513v1",
            "date": "2020-07-18 20:13:58+00:00",
            "title": "Feature-level Rating System using Customer Reviews and Review Votes",
            "authors": [
                "Koteswar Rao Jerripothula",
                "Ankit Rai",
                "Kanu Garg",
                "Yashvardhan Singh Rautela"
            ],
            "abstract": "This work studies how we can obtain feature-level ratings of the mobile\nproducts from the customer reviews and review votes to influence decision\nmaking, both for new customers and manufacturers. Such a rating system gives a\nmore comprehensive picture of the product than what a product-level rating\nsystem offers. While product-level ratings are too generic, feature-level\nratings are particular; we exactly know what is good or bad about the product.\nThere has always been a need to know which features fall short or are doing\nwell according to the customer's perception. It keeps both the manufacturer and\nthe customer well-informed in the decisions to make in improving the product\nand buying, respectively. Different customers are interested in different\nfeatures. Thus, feature-level ratings can make buying decisions personalized.\nWe analyze the customer reviews collected on an online shopping site (Amazon)\nabout various mobile products and the review votes. Explicitly, we carry out a\nfeature-focused sentiment analysis for this purpose. Eventually, our analysis\nyields ratings to 108 features for 4k+ mobiles sold online. It helps in\ndecision making on how to improve the product (from the manufacturer's\nperspective) and in making the personalized buying decisions (from the buyer's\nperspective) a possibility. Our analysis has applications in recommender\nsystems, consumer research, etc.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09554v2",
            "date": "2020-07-19 01:45:02+00:00",
            "title": "Referring Expression Comprehension: A Survey of Methods and Datasets",
            "authors": [
                "Yanyuan Qiao",
                "Chaorui Deng",
                "Qi Wu"
            ],
            "abstract": "Referring expression comprehension (REC) aims to localize a target object in\nan image described by a referring expression phrased in natural language.\nDifferent from the object detection task that queried object labels have been\npre-defined, the REC problem only can observe the queries during the test. It\nthus more challenging than a conventional computer vision problem. This task\nhas attracted a lot of attention from both computer vision and natural language\nprocessing community, and several lines of work have been proposed, from\nCNN-RNN model, modular network to complex graph-based model. In this survey, we\nfirst examine the state of the art by comparing modern approaches to the\nproblem. We classify methods by their mechanism to encode the visual and\ntextual modalities. In particular, we examine the common approach of joint\nembedding images and expressions to a common feature space. We also discuss\nmodular architectures and graph-based models that interface with structured\ngraph representation. In the second part of this survey, we review the datasets\navailable for training and evaluating REC systems. We then group results\naccording to the datasets, backbone models, settings so that they can be fairly\ncompared. Finally, we discuss promising future directions for the field, in\nparticular the compositional referring expression comprehension that requires\nlonger reasoning chain to address.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.09604v1",
            "date": "2020-07-19 06:36:41+00:00",
            "title": "Meta-learning for Few-shot Natural Language Processing: A Survey",
            "authors": [
                "Wenpeng Yin"
            ],
            "abstract": "Few-shot natural language processing (NLP) refers to NLP tasks that are\naccompanied with merely a handful of labeled examples. This is a real-world\nchallenge that an AI system must learn to handle. Usually we rely on collecting\nmore auxiliary information or developing a more efficient learning algorithm.\nHowever, the general gradient-based optimization in high capacity models, if\ntraining from scratch, requires many parameter-updating steps over a large\nnumber of labeled examples to perform well (Snell et al., 2017). If the target\ntask itself cannot provide more information, how about collecting more tasks\nequipped with rich annotations to help the model learning? The goal of\nmeta-learning is to train a model on a variety of tasks with rich annotations,\nsuch that it can solve a new task using only a few labeled samples. The key\nidea is to train the model's initial parameters such that the model has maximal\nperformance on a new task after the parameters have been updated through zero\nor a couple of gradient steps. There are already some surveys for\nmeta-learning, such as (Vilalta and Drissi, 2002; Vanschoren, 2018; Hospedales\net al., 2020). Nevertheless, this paper focuses on NLP domain, especially\nfew-shot applications. We try to provide clearer definitions, progress summary\nand some common datasets of applying meta-learning to few-shot NLP.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09655v1",
            "date": "2020-07-19 12:00:25+00:00",
            "title": "Political Framing: US COVID19 Blame Game",
            "authors": [
                "Chereen Shurafa",
                "Kareem Darwish",
                "Wajdi Zaghouani"
            ],
            "abstract": "Through the use of Twitter, framing has become a prominent presidential\ncampaign tool for politically active users. Framing is used to influence\nthoughts by evoking a particular perspective on an event. In this paper, we\nshow that the COVID19 pandemic rather than being viewed as a public health\nissue, political rhetoric surrounding it is mostly shaped through a blame frame\n(blame Trump, China, or conspiracies) and a support frame (support candidates)\nbacking the agenda of Republican and Democratic users in the lead up to the\n2020 presidential campaign. We elucidate the divergences between supporters of\nboth parties on Twitter via the use of frames. Additionally, we show how\nframing is used to positively or negatively reinforce users' thoughts. We look\nat how Twitter can efficiently be used to identify frames for topics through a\nreproducible pipeline.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2007.09679v1",
            "date": "2020-07-19 14:33:03+00:00",
            "title": "One-Shot Learning for Language Modelling",
            "authors": [
                "Talip Ucar",
                "Adrian Gonzalez-Martin",
                "Matthew Lee",
                "Adrian Daniel Szwarc"
            ],
            "abstract": "Humans can infer a great deal about the meaning of a word, using the syntax\nand semantics of surrounding words even if it is their first time reading or\nhearing it. We can also generalise the learned concept of the word to new\ntasks. Despite great progress in achieving human-level performance in certain\ntasks (Silver et al., 2016), learning from one or few examples remains a key\nchallenge in machine learning, and has not thoroughly been explored in Natural\nLanguage Processing (NLP).\n  In this work we tackle the problem of oneshot learning for an NLP task by\nemploying ideas from recent developments in machine learning: embeddings,\nattention mechanisms (softmax) and similarity measures (cosine, Euclidean,\nPoincare, and Minkowski). We adapt the framework suggested in matching networks\n(Vinyals et al., 2016), and explore the effectiveness of the aforementioned\nmethods in one, two and three-shot learning problems on the task of predicting\nmissing word explored in (Vinyals et al., 2016) by using the WikiText-2\ndataset. Our work contributes in two ways: Our first contribution is that we\nexplore the effectiveness of different distance metrics on k-shot learning, and\nshow that there is no single best distance metric for k-shot learning, which\nchallenges common belief. We found that the performance of a distance metric\ndepends on the number of shots used during training. The second contribution of\nour work is that we establish a benchmark for one, two, and three-shot learning\non a language task with a publicly available dataset that can be used to\nbenchmark against in future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09757v1",
            "date": "2020-07-19 19:13:20+00:00",
            "title": "Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks",
            "authors": [
                "Diego de Vargas Feijo",
                "Viviane Pereira Moreira"
            ],
            "abstract": "BERT (Bidirectional Encoder Representations from Transformers) and ALBERT (A\nLite BERT) are methods for pre-training language models which can later be\nfine-tuned for a variety of Natural Language Understanding tasks. These methods\nhave been applied to a number of such tasks (mostly in English), achieving\nresults that outperform the state-of-the-art. In this paper, our contribution\nis twofold. First, we make available our trained BERT and Albert model for\nPortuguese. Second, we compare our monolingual and the standard multilingual\nmodels using experiments in semantic textual similarity, recognizing textual\nentailment, textual category classification, sentiment analysis, offensive\ncomment detection, and fake news detection, to assess the effectiveness of the\ngenerated language representations. The results suggest that both monolingual\nand multilingual models are able to achieve state-of-the-art and the advantage\nof training a single language model, if any, is small.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09878v1",
            "date": "2020-07-20 04:10:08+00:00",
            "title": "Frustratingly Hard Evidence Retrieval for QA Over Books",
            "authors": [
                "Xiangyang Mou",
                "Mo Yu",
                "Bingsheng Yao",
                "Chenghao Yang",
                "Xiaoxiao Guo",
                "Saloni Potdar",
                "Hui Su"
            ],
            "abstract": "A lot of progress has been made to improve question answering (QA) in recent\nyears, but the special problem of QA over narrative book stories has not been\nexplored in-depth. We formulate BookQA as an open-domain QA task given its\nsimilar dependency on evidence retrieval. We further investigate how\nstate-of-the-art open-domain QA approaches can help BookQA. Besides achieving\nstate-of-the-art on the NarrativeQA benchmark, our study also reveals the\ndifficulty of evidence retrieval in books with a wealth of experiments and\nanalysis - which necessitates future effort on novel solutions for evidence\nretrieval in BookQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.09970v1",
            "date": "2020-07-20 09:43:11+00:00",
            "title": "How are you? Introducing stress-based text tailoring",
            "authors": [
                "Simone Balloccu",
                "Ehud Reiter",
                "Alexandra Johnstone",
                "Claire Fyfe"
            ],
            "abstract": "Can stress affect not only your life but also how you read and interpret a\ntext? Healthcare has shown evidence of such dynamics and in this short paper we\ndiscuss customising texts based on user stress level, as it could represent a\ncritical factor when it comes to user engagement and behavioural change. We\nfirst show a real-world example in which user behaviour is influenced by\nstress, then, after discussing which tools can be employed to assess and\nmeasure it, we propose an initial method for tailoring the document by\nexploiting complexity reduction and affect enforcement. The result is a short\nand encouraging text which requires less commitment to be read and understood.\nWe believe this work in progress can raise some interesting questions on a\ntopic that is often overlooked in NLG.",
            "journal ref": "IntelLang 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10021v3",
            "date": "2020-07-20 11:54:43+00:00",
            "title": "Voice@SRIB at SemEval-2020 Task 9 and 12: Stacked Ensembling method for Sentiment and Offensiveness detection in Social Media",
            "authors": [
                "Abhishek Singh",
                "Surya Pratap Singh Parmar"
            ],
            "abstract": "In social-media platforms such as Twitter, Facebook, and Reddit, people\nprefer to use code-mixed language such as Spanish-English, Hindi-English to\nexpress their opinions. In this paper, we describe different models we used,\nusing the external dataset to train embeddings, ensembling methods for\nSentimix, and OffensEval tasks. The use of pre-trained embeddings usually helps\nin multiple tasks such as sentence classification, and machine translation. In\nthis experiment, we haveused our trained code-mixed embeddings and twitter\npre-trained embeddings to SemEval tasks. We evaluate our models on macro\nF1-score, precision, accuracy, and recall on the datasets. We intend to show\nthat hyper-parameter tuning and data pre-processing steps help a lot in\nimproving the scores. In our experiments, we are able to achieve 0.886 F1-Macro\non OffenEval Greek language subtask post-evaluation, whereas the highest is\n0.852 during the Evaluation Period. We stood third in Spanglish competition\nwith our best F1-score of 0.756. Codalab username is asking28.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10040v1",
            "date": "2020-07-20 12:23:39+00:00",
            "title": "Knowledge Graph Extraction from Videos",
            "authors": [
                "Louis Mahon",
                "Eleonora Giunchiglia",
                "Bowen Li",
                "Thomas Lukasiewicz"
            ],
            "abstract": "Nearly all existing techniques for automated video annotation (or captioning)\ndescribe videos using natural language sentences. However, this has several\nshortcomings: (i) it is very hard to then further use the generated natural\nlanguage annotations in automated data processing, (ii) generating natural\nlanguage annotations requires to solve the hard subtask of generating\nsemantically precise and syntactically correct natural language sentences,\nwhich is actually unrelated to the task of video annotation, (iii) it is\ndifficult to quantitatively measure performance, as standard metrics (e.g.,\naccuracy and F1-score) are inapplicable, and (iv) annotations are\nlanguage-specific. In this paper, we propose the new task of knowledge graph\nextraction from videos, i.e., producing a description in the form of a\nknowledge graph of the contents of a given video. Since no datasets exist for\nthis task, we also include a method to automatically generate them, starting\nfrom datasets where videos are annotated with natural language. We then\ndescribe an initial deep-learning model for knowledge graph extraction from\nvideos, and report results on MSVD* and MSR-VTT*, two datasets obtained from\nMSVD and MSR-VTT using our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10055v2",
            "date": "2020-07-20 12:47:36+00:00",
            "title": "Morphological Skip-Gram: Using morphological knowledge to improve word representation",
            "authors": [
                "Fl\u00e1vio Santos",
                "Hendrik Macedo",
                "Thiago Bispo",
                "Cleber Zanchettin"
            ],
            "abstract": "Natural language processing models have attracted much interest in the deep\nlearning community. This branch of study is composed of some applications such\nas machine translation, sentiment analysis, named entity recognition, question\nand answer, and others. Word embeddings are continuous word representations,\nthey are an essential module for those applications and are generally used as\ninput word representation to the deep learning models. Word2Vec and GloVe are\ntwo popular methods to learn word embeddings. They achieve good word\nrepresentations, however, they learn representations with limited information\nbecause they ignore the morphological information of the words and consider\nonly one representation vector for each word. This approach implies that\nWord2Vec and GloVe are unaware of the word inner structure. To mitigate this\nproblem, the FastText model represents each word as a bag of characters\nn-grams. Hence, each n-gram has a continuous vector representation, and the\nfinal word representation is the sum of its characters n-grams vectors.\nNevertheless, the use of all n-grams character of a word is a poor approach\nsince some n-grams have no semantic relation with their words and increase the\namount of potentially useless information. This approach also increases the\ntraining phase time. In this work, we propose a new method for training word\nembeddings, and its goal is to replace the FastText bag of character n-grams\nfor a bag of word morphemes through the morphological analysis of the word.\nThus, words with similar context and morphemes are represented by vectors close\nto each other. To evaluate our new approach, we performed intrinsic evaluations\nconsidering 15 different tasks, and the results show a competitive performance\ncompared to FastText.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10276v2",
            "date": "2020-07-20 16:56:46+00:00",
            "title": "Characterizing drug mentions in COVID-19 Twitter Chatter",
            "authors": [
                "Ramya Tekumalla",
                "Juan M. Banda"
            ],
            "abstract": "Since the classification of COVID-19 as a global pandemic, there have been\nmany attempts to treat and contain the virus. Although there is no specific\nantiviral treatment recommended for COVID-19, there are several drugs that can\npotentially help with symptoms. In this work, we mined a large twitter dataset\nof 424 million tweets of COVID-19 chatter to identify discourse around drug\nmentions. While seemingly a straightforward task, due to the informal nature of\nlanguage use in Twitter, we demonstrate the need of machine learning alongside\ntraditional automated methods to aid in this task. By applying these\ncomplementary methods, we are able to recover almost 15% additional data,\nmaking misspelling handling a needed task as a pre-processing step when dealing\nwith social media data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.10287v1",
            "date": "2020-07-04 03:55:31+00:00",
            "title": "Coronavirus Knowledge Graph: A Case Study",
            "authors": [
                "Chongyan Chen",
                "Islam Akef Ebeid",
                "Yi Bu",
                "Ying Ding"
            ],
            "abstract": "The emergence of the novel COVID-19 pandemic has had a significant impact on\nglobal healthcare and the economy over the past few months. The virus's rapid\nwidespread has led to a proliferation in biomedical research addressing the\npandemic and its related topics. One of the essential Knowledge Discovery tools\nthat could help the biomedical research community understand and eventually\nfind a cure for COVID-19 are Knowledge Graphs. The CORD-19 dataset is a\ncollection of publicly available full-text research articles that have been\nrecently published on COVID-19 and coronavirus topics. Here, we use several\nMachine Learning, Deep Learning, and Knowledge Graph construction and mining\ntechniques to formalize and extract insights from the PubMed dataset and the\nCORD-19 dataset to identify COVID-19 related experts and bio-entities. Besides,\nwe suggest possible techniques to predict related diseases, drug candidates,\ngene, gene mutations, and related compounds as part of a systematic effort to\napply Knowledge Discovery methods to help biomedical researchers tackle the\npandemic.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2007.10534v2",
            "date": "2020-07-21 00:07:17+00:00",
            "title": "Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Features",
            "authors": [
                "Gullal S. Cheema",
                "Sherzod Hakimov",
                "Ralph Ewerth"
            ],
            "abstract": "In this digital age of news consumption, a news reader has the ability to\nreact, express and share opinions with others in a highly interactive and fast\nmanner. As a consequence, fake news has made its way into our daily life\nbecause of very limited capacity to verify news on the Internet by large\ncompanies as well as individuals. In this paper, we focus on solving two\nproblems which are part of the fact-checking ecosystem that can help to\nautomate fact-checking of claims in an ever increasing stream of content on\nsocial media. For the first problem, claim check-worthiness prediction, we\nexplore the fusion of syntactic features and deep transformer Bidirectional\nEncoder Representations from Transformers (BERT) embeddings, to classify\ncheck-worthiness of a tweet, i.e. whether it includes a claim or not. We\nconduct a detailed feature analysis and present our best performing models for\nEnglish and Arabic tweets. For the second problem, claim retrieval, we explore\nthe pre-trained embeddings from a Siamese network transformer model\n(sentence-transformers) specifically trained for semantic textual similarity,\nand perform KD-search to retrieve verified claims with respect to a query\ntweet.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10633v1",
            "date": "2020-07-21 07:28:46+00:00",
            "title": "Analysis and Optimization of Service Delay for Multi-quality Videos in Multi-tier Heterogeneous Network with Random Caching",
            "authors": [
                "Xuewei Zhang",
                "Tiejun Lv",
                "Yuan Ren",
                "Wei Ni",
                "Norman C. Beaulieu"
            ],
            "abstract": "Aiming to minimize service delay, we propose a new random caching scheme in\ndevice-to-device (D2D)-assisted heterogeneous network. To support diversified\nviewing qualities of multimedia video services, each video file is encoded into\na base layer (BL) and multiple enhancement layers (ELs) by scalable video\ncoding (SVC). A super layer, including the BL and several ELs, is transmitted\nto every user. We define and quantify the service delay of multi-quality videos\nby deriving successful transmission probabilities when a user is served by a\nD2D helper, a small-cell base station (SBS) and a macro-cell base station\n(MBS). We formulate a delay minimization problem subject to the limited cache\nsizes of D2D helpers and SBSs. The structure of the optimal solutions to the\nproblem is revealed, and then an improved standard gradient projection method\nis designed to effectively obtain the solutions. Both theoretical analysis and\nMonte-Carlo simulations validate the successful transmission probabilities.\nCompared with three benchmark caching policies, the proposed SVC-based random\ncaching scheme is superior in terms of reducing the service delay.",
            "journal ref": "None",
            "category": "cs.NI"
        },
        {
            "url": "http://arxiv.org/abs/2007.10712v1",
            "date": "2020-07-21 11:11:35+00:00",
            "title": "On Analyzing Antisocial Behaviors Amid COVID-19 Pandemic",
            "authors": [
                "Md Rabiul Awal",
                "Rui Cao",
                "Sandra Mitrovic",
                "Roy Ka-Wei Lee"
            ],
            "abstract": "The COVID-19 pandemic has developed to be more than a bio-crisis as global\nnews has reported a sharp rise in xenophobia and discrimination in both online\nand offline communities. Such toxic behaviors take a heavy toll on society,\nespecially during these daunting times. Despite the gravity of the issue, very\nfew studies have studied online antisocial behaviors amid the COVID-19\npandemic. In this paper, we fill the research gap by collecting and annotating\na large dataset of over 40 million COVID-19 related tweets. Specially, we\npropose an annotation framework to annotate the antisocial behavior tweets\nautomatically. We also conduct an empirical analysis of our annotated dataset\nand found that new abusive lexicons are introduced amid the COVID-19 pandemic.\nOur study also identified the vulnerable targets of antisocial behaviors and\nthe factors that influence the spreading of online antisocial content.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2007.10718v1",
            "date": "2020-07-21 11:21:26+00:00",
            "title": "Human Abnormality Detection Based on Bengali Text",
            "authors": [
                "M. F. Mridha",
                "Md. Saifur Rahman",
                "Abu Quwsar Ohi"
            ],
            "abstract": "In the field of natural language processing and human-computer interaction,\nhuman attitudes and sentiments have attracted the researchers. However, in the\nfield of human-computer interaction, human abnormality detection has not been\ninvestigated extensively and most works depend on image-based information. In\nnatural language processing, effective meaning can potentially convey by all\nwords. Each word may bring out difficult encounters because of their semantic\nconnection with ideas or categories. In this paper, an efficient and effective\nhuman abnormality detection model is introduced, that only uses Bengali text.\nThis proposed model can recognize whether the person is in a normal or abnormal\nstate by analyzing their typed Bengali text. To the best of our knowledge, this\nis the first attempt in developing a text based human abnormality detection\nsystem. We have created our Bengali dataset (contains 2000 sentences) that is\ngenerated by voluntary conversations. We have performed the comparative\nanalysis by using Naive Bayes and Support Vector Machine as classifiers. Two\ndifferent feature extraction techniques count vector, and TF-IDF is used to\nexperiment on our constructed dataset. We have achieved a maximum 89% accuracy\nand 92% F1-score with our constructed dataset in our experiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10819v1",
            "date": "2020-07-21 14:05:51+00:00",
            "title": "BAKSA at SemEval-2020 Task 9: Bolstering CNN with Self-Attention for Sentiment Analysis of Code Mixed Text",
            "authors": [
                "Ayush Kumar",
                "Harsh Agarwal",
                "Keshav Bansal",
                "Ashutosh Modi"
            ],
            "abstract": "Sentiment Analysis of code-mixed text has diversified applications in opinion\nmining ranging from tagging user reviews to identifying social or political\nsentiments of a sub-population. In this paper, we present an ensemble\narchitecture of convolutional neural net (CNN) and self-attention based LSTM\nfor sentiment analysis of code-mixed tweets. While the CNN component helps in\nthe classification of positive and negative tweets, the self-attention based\nLSTM, helps in the classification of neutral tweets, because of its ability to\nidentify correct sentiment among multiple sentiment bearing units. We achieved\nF1 scores of 0.707 (ranked 5th) and 0.725 (ranked 13th) on Hindi-English\n(Hinglish) and Spanish-English (Spanglish) datasets, respectively. The\nsubmissions for Hinglish and Spanglish tasks were made under the usernames\nayushk and harsh_6 respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10827v1",
            "date": "2020-07-21 14:06:59+00:00",
            "title": "newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification",
            "authors": [
                "Paramansh Singh",
                "Siraj Sandhu",
                "Subham Kumar",
                "Ashutosh Modi"
            ],
            "abstract": "This paper describes our submissions to SemEval 2020 Task 11: Detection of\nPropaganda Techniques in News Articles for each of the two subtasks of Span\nIdentification and Technique Classification. We make use of pre-trained BERT\nlanguage model enhanced with tagging techniques developed for the task of Named\nEntity Recognition (NER), to develop a system for identifying propaganda spans\nin the text. For the second subtask, we incorporate contextual features in a\npre-trained RoBERTa model for the classification of propaganda techniques. We\nwere ranked 5th in the propaganda technique classification subtask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10830v1",
            "date": "2020-07-21 14:08:02+00:00",
            "title": "CS-NET at SemEval-2020 Task 4: Siamese BERT for ComVE",
            "authors": [
                "Soumya Ranjan Dash",
                "Sandeep Routray",
                "Prateek Varshney",
                "Ashutosh Modi"
            ],
            "abstract": "In this paper, we describe our system for Task 4 of SemEval 2020, which\ninvolves differentiating between natural language statements that confirm to\ncommon sense and those that do not. The organizers propose three subtasks -\nfirst, selecting between two sentences, the one which is against common sense.\nSecond, identifying the most crucial reason why a statement does not make\nsense. Third, generating novel reasons for explaining the against common sense\nstatement. Out of the three subtasks, this paper reports the system description\nof subtask A and subtask B. This paper proposes a model based on transformer\nneural network architecture for addressing the subtasks. The novelty in work\nlies in the architecture design, which handles the logical implication of\ncontradicting statements and simultaneous information extraction from both\nsentences. We use a parallel instance of transformers, which is responsible for\na boost in the performance. We achieved an accuracy of 94.8% in subtask A and\n89% in subtask B on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10873v1",
            "date": "2020-07-21 15:00:01+00:00",
            "title": "Connecting Embeddings for Knowledge Graph Entity Typing",
            "authors": [
                "Yu Zhao",
                "Anxiang Zhang",
                "Ruobing Xie",
                "Kang Liu",
                "Xiaojie Wang"
            ],
            "abstract": "Knowledge graph (KG) entity typing aims at inferring possible missing entity\ntype instances in KG, which is a very significant but still under-explored\nsubtask of knowledge graph completion. In this paper, we propose a novel\napproach for KG entity typing which is trained by jointly utilizing local\ntyping knowledge from existing entity type assertions and global triple\nknowledge from KGs. Specifically, we present two distinct knowledge-driven\neffective mechanisms of entity type inference. Accordingly, we build two novel\nembedding models to realize the mechanisms. Afterward, a joint model with them\nis used to infer missing entity type instances, which favors inferences that\nagree with both entity type instances and triple knowledge in KGs. Experimental\nresults on two real-world datasets (Freebase and YAGO) demonstrate the\neffectiveness of our proposed mechanisms and models for improving KG entity\ntyping. The source code and data of this paper can be obtained from:\nhttps://github.com/ Adam1679/ConnectE",
            "journal ref": "Association for Computational Linguistics, 2020, 6419--6428",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.10877v1",
            "date": "2020-07-21 15:06:58+00:00",
            "title": "problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches",
            "authors": [
                "Karishma Laud",
                "Jagriti Singh",
                "Randeep Kumar Sahu",
                "Ashutosh Modi"
            ],
            "abstract": "In this paper, we present various systems submitted by our team\nproblemConquero for SemEval-2020 Shared Task 12 Multilingual Offensive Language\nIdentification in Social Media. We participated in all the three sub-tasks of\nOffensEval-2020, and our final submissions during the evaluation phase included\ntransformer-based approaches and a soft label-based approach. BERT based\nfine-tuned models were submitted for each language of sub-task A (offensive\ntweet identification). RoBERTa based fine-tuned model for sub-task B (automatic\ncategorization of offense types) was submitted. We submitted two models for\nsub-task C (offense target identification), one using soft labels and the other\nusing BERT based fine-tuned model. Our ranks for sub-task A were Greek-19 out\nof 37, Turkish-22 out of 46, Danish-26 out of 39, Arabic-39 out of 53, and\nEnglish-20 out of 85. We achieved a rank of 28 out of 43 for sub-task B. Our\nbest rank for sub-task C was 20 out of 39 using BERT based fine-tuned model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11053v1",
            "date": "2020-07-21 19:29:26+00:00",
            "title": "Curriculum Vitae Recommendation Based on Text Mining",
            "authors": [
                "Honorio Apaza Alanoca",
                "Americo A. Rubin de Celis Vidal",
                "Josimar Edinson Chire Saire"
            ],
            "abstract": "During the last years, the development in diverse areas related to computer\nscience and internet, allowed to generate new alternatives for decision making\nin the selection of personnel for state and private companies. In order to\noptimize this selection process, the recommendation systems are the most\nsuitable for working with explicit information related to the likes and\ndislikes of employers or end users, since this information allows to generate\nlists of recommendations based on collaboration or similarity of content.\nTherefore, this research takes as a basis these characteristics contained in\nthe database of curricula and job offers, which correspond to the Peruvian\nambit, which highlights the experience, knowledge and skills of each candidate,\nwhich are described in textual terms or words. This research focuses on the\nproblem: how we can take advantage from the growth of unstructured information\nabout job offers and curriculum vitae on different websites for CV\nrecommendation. So, we use the techniques from Text Mining and Natural Language\nProcessing. Then, as a relevant technique for the present study, we emphasize\nthe technique frequency of the Term - Inverse Frequency of the documents\n(TF-IDF), which allows identifying the most relevant CVs in relation to a job\noffer of website through the average values (TF-IDF). So, the weighted value\ncan be used as a qualification value of the relevant curriculum vitae for the\nrecommendation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11057v1",
            "date": "2020-07-21 19:35:39+00:00",
            "title": "Explainable Rumor Detection using Inter and Intra-feature Attention Networks",
            "authors": [
                "Mingxuan Chen",
                "Ning Wang",
                "K. P. Subbalakshmi"
            ],
            "abstract": "With social media becoming ubiquitous, information consumption from this\nmedia has also increased. However, one of the serious problems that have\nemerged with this increase, is the propagation of rumors. Therefore, rumor\nidentification is a very critical task with significant implications to\neconomy, democracy as well as public health and safety. We tackle the problem\nof automated detection of rumors in social media in this paper by designing a\nmodular explainable architecture that uses both latent and handcrafted features\nand can be expanded to as many new classes of features as desired. This\napproach will allow the end user to not only determine whether the piece of\ninformation on the social media is real of a rumor, but also give explanations\non why the algorithm arrived at its conclusion. Using attention mechanisms, we\nare able to interpret the relative importance of each of these features as well\nas the relative importance of the feature classes themselves. The advantage of\nthis approach is that the architecture is expandable to more handcrafted\nfeatures as they become available and also to conduct extensive testing to\ndetermine the relative influences of these features in the final decision.\nExtensive experimentation on popular datasets and benchmarking against eleven\ncontemporary algorithms, show that our approach performs significantly better\nin terms of F-score and accuracy while also being interpretable.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2007.11073v1",
            "date": "2020-07-21 20:11:18+00:00",
            "title": "Will Your Forthcoming Book be Successful? Predicting Book Success with CNN and Readability Scores",
            "authors": [
                "Muhammad Khalifa",
                "Aminul Islam"
            ],
            "abstract": "Predicting the potential success of a book in advance is vital in many\napplications. This could help both publishers and readers in their decision\nmaking process whether or not a book is worth publishing and reading,\nrespectively. This prediction could also help authors decide whether a book\ndraft is good enough to send to a publisher. We propose a model that leverages\nConvolutional Neural Networks along with readability indices. Unlike previous\nmethods, our method includes no count-based, lexical, or syntactic hand-crafted\nfeatures. Instead, we make use of a pre-trained sentence encoder to encode the\nbook sentences. We highlight the connection between this task and book genre\nidentification by showing that embeddings that are good at capturing the\nseparability of book genres are better for the book success prediction task. We\nalso show that only the first 1K sentences are good enough to predict the\nsuccessability of books. Our proposed model outperforms strong baselines on\nthis task by as large as 6.4% F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11171v1",
            "date": "2020-07-22 02:42:01+00:00",
            "title": "When Classical Chinese Meets Machine Learning: Explaining the Relative Performances of Word and Sentence Segmentation Tasks",
            "authors": [
                "Chao-Lin Liu",
                "Chang-Ting Chu",
                "Wei-Ting Chang",
                "Ti-Yong Zheng"
            ],
            "abstract": "We consider three major text sources about the Tang Dynasty of China in our\nexperiments that aim to segment text written in classical Chinese. These\ncorpora include a collection of Tang Tomb Biographies, the New Tang Book, and\nthe Old Tang Book. We show that it is possible to achieve satisfactory\nsegmentation results with the deep learning approach. More interestingly, we\nfound that some of the relative superiority that we observed among different\ndesigns of experiments may be explainable. The relative relevance among the\ntraining corpora provides hints/explanation for the observed differences in\nsegmentation results that were achieved when we employed different combinations\nof corpora to train the classifiers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11189v2",
            "date": "2020-07-22 03:41:32+00:00",
            "title": "Predicting Job-Hopping Motive of Candidates Using Answers to Open-ended Interview Questions",
            "authors": [
                "Madhura Jayaratne",
                "Buddhi Jayatilleke"
            ],
            "abstract": "A significant proportion of voluntary employee turnover includes people who\nfrequently move from job to job, known as job-hopping. Our work shows that\nlanguage used in responding to interview questions on past behaviour and\nsituational judgement is predictive of job-hopping motive as measured by the\nJob-Hopping Motives (JHM) Scale. The study is based on responses from over\n45,000 job applicants who completed an online chat interview and self-rated\nthemselves on JHM Scale. Five different methods of text representation were\nevaluated, namely four open-vocabulary approaches (TF-IDF, LDA, Glove word\nembeddings and Doc2Vec document embeddings) and one closed-vocabulary approach\n(LIWC). The Glove embeddings provided the best results with a correlation of r\n= 0.35 between sequences of words used and the JHM Scale. Further analysis also\nshowed a correlation of r = 0.25 between language-based job-hopping motive and\nthe personality trait Openness to experience and a correlation of r = -0.09\nwith the trait Agreeableness.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2007.11198v1",
            "date": "2020-07-22 04:46:54+00:00",
            "title": "Exploratory Search with Sentence Embeddings",
            "authors": [
                "Austin Silveria"
            ],
            "abstract": "Exploratory search aims to guide users through a corpus rather than\npinpointing exact information. We propose an exploratory search system based on\nhierarchical clusters and document summaries using sentence embeddings. With\nsentence embeddings, we represent documents as the mean of their embedded\nsentences, extract summaries containing sentences close to this document\nrepresentation and extract keyphrases close to the document representation. To\nevaluate our search system, we scrape our personal search history over the past\nyear and report our experience with the system. We then discuss motivating use\ncases of an exploratory search system of this nature and conclude with possible\ndirections of future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11756v1",
            "date": "2020-07-23 02:18:05+00:00",
            "title": "Clustering of Social Media Messages for Humanitarian Aid Response during Crisis",
            "authors": [
                "Swati Padhee",
                "Tanay Kumar Saha",
                "Joel Tetreault",
                "Alejandro Jaimes"
            ],
            "abstract": "Social media has quickly grown into an essential tool for people to\ncommunicate and express their needs during crisis events. Prior work in\nanalyzing social media data for crisis management has focused primarily on\nautomatically identifying actionable (or, informative) crisis-related messages.\nIn this work, we show that recent advances in Deep Learning and Natural\nLanguage Processing outperform prior approaches for the task of classifying\ninformativeness and encourage the field to adopt them for their research or\neven deployment. We also extend these methods to two sub-tasks of\ninformativeness and find that the Deep Learning methods are effective here as\nwell.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2007.11768v1",
            "date": "2020-07-23 03:15:19+00:00",
            "title": "Product Title Generation for Conversational Systems using BERT",
            "authors": [
                "Mansi Ranjit Mane",
                "Shashank Kedia",
                "Aditya Mantha",
                "Stephen Guo",
                "Kannan Achan"
            ],
            "abstract": "Through recent advancements in speech technology and introduction of smart\ndevices, such as Amazon Alexa and Google Home, increasing number of users are\ninteracting with applications through voice. E-commerce companies typically\ndisplay short product titles on their webpages, either human-curated or\nalgorithmically generated, when brevity is required, but these titles are\ndissimilar from natural spoken language. For example, \"Lucky Charms Gluten Free\nBreak-fast Cereal, 20.5 oz a box Lucky Charms Gluten Free\" is acceptable to\ndisplay on a webpage, but \"a 20.5 ounce box of lucky charms gluten free cereal\"\nis easier to comprehend over a conversational system. As compared to display\ndevices, where images and detailed product information can be presented to\nusers, short titles for products are necessary when interfacing with voice\nassistants. We propose a sequence-to-sequence approach using BERT to generate\nshort, natural, spoken language titles from input web titles. Our extensive\nexperiments on a real-world industry dataset and human evaluation of model\noutputs, demonstrate that BERT summarization outperforms comparable baseline\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.11794v1",
            "date": "2020-07-23 05:15:14+00:00",
            "title": "Applying GPGPU to Recurrent Neural Network Language Model based Fast Network Search in the Real-Time LVCSR",
            "authors": [
                "Kyungmin Lee",
                "Chiyoun Park",
                "Ilhwan Kim",
                "Namhoon Kim",
                "Jaewon Lee"
            ],
            "abstract": "Recurrent Neural Network Language Models (RNNLMs) have started to be used in\nvarious fields of speech recognition due to their outstanding performance.\nHowever, the high computational complexity of RNNLMs has been a hurdle in\napplying the RNNLM to a real-time Large Vocabulary Continuous Speech\nRecognition (LVCSR). In order to accelerate the speed of RNNLM-based network\nsearches during decoding, we apply the General Purpose Graphic Processing Units\n(GPGPUs). This paper proposes a novel method of applying GPGPUs to RNNLM-based\ngraph traversals. We have achieved our goal by reducing redundant computations\non CPUs and amount of transfer between GPGPUs and CPUs. The proposed approach\nwas evaluated on both WSJ corpus and in-house data. Experiments shows that the\nproposed approach achieves the real-time speed in various circumstances while\nmaintaining the Word Error Rate (WER) to be relatively 10% lower than that of\nn-gram models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12053v3",
            "date": "2020-07-23 15:11:32+00:00",
            "title": "Revealing semantic and emotional structure of suicide notes with cognitive network science",
            "authors": [
                "Andreia Sofia Teixeira",
                "Szymon Talaga",
                "Trevor James Swanson",
                "Massimo Stella"
            ],
            "abstract": "Understanding the cognitive and emotional perceptions of people who commit\nsuicide is one of the most sensitive scientific challenges. There are\ncircumstances where people feel the need to leave something written, an\nartifact where they express themselves, registering their last words and\nfeelings. These suicide notes are of utmost importance for better understanding\nthe psychology of suicidal ideation. This work gives structure to the\nlinguistic content of suicide notes, revealing interconnections between\ncognitive and emotional states of people who committed suicide. We build upon\ncognitive network science, psycholinguistics and semantic frame theory to\nintroduce a network representation of the mindset expressed in suicide notes.\nOur cognitive network representation enables the quantitative analysis of the\nlanguage in suicide notes through structural balance theory, semantic\nprominence and emotional profiling. Our results indicate that the emotional\nsyntax connecting positively- and negatively-valenced terms gives rise to a\ndegree of structural balance that is significantly higher than null models\nwhere the affective structure was randomized. We show that suicide notes are\naffectively compartmentalized such that positive concepts tend to cluster\ntogether and dominate the overall network structure. A key positive concept is\n\"love\", which integrates information relating the self to others in ways that\nare semantically prominent across suicide notes. The emotions populating the\nsemantic frame of \"love\" combine joy and trust with anticipation and sadness,\nwhich connects with psychological theories about meaning-making and narrative\npsychology. Our results open new ways for understanding the structure of\ngenuine suicide notes informing future research for suicide prevention.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2007.12603v1",
            "date": "2020-07-24 16:02:14+00:00",
            "title": "IR-BERT: Leveraging BERT for Semantic Search in Background Linking for News Articles",
            "authors": [
                "Anup Anand Deshmukh",
                "Udhav Sethi"
            ],
            "abstract": "This work describes our two approaches for the background linking task of\nTREC 2020 News Track. The main objective of this task is to recommend a list of\nrelevant articles that the reader should refer to in order to understand the\ncontext and gain background information of the query article. Our first\napproach focuses on building an effective search query by combining weighted\nkeywords extracted from the query document and uses BM25 for retrieval. The\nsecond approach leverages the capability of SBERT (Nils Reimers et al.) to\nlearn contextual representations of the query in order to perform semantic\nsearch over the corpus. We empirically show that employing a language model\nbenefits our approach in understanding the context as well as the background of\nthe query article. The proposed approaches are evaluated on the TREC 2018\nWashington Post dataset and our best model outperforms the TREC median as well\nas the highest scoring model of 2018 in terms of the nDCG@5 metric. We further\npropose a diversity measure to evaluate the effectiveness of the various\napproaches in retrieving a diverse set of documents. This would potentially\nmotivate researchers to work on introducing diversity in their recommended\nlist. We have open sourced our implementation on Github and plan to submit our\nruns for the background linking task in TREC 2020.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.12720v1",
            "date": "2020-07-10 22:52:14+00:00",
            "title": "MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines",
            "authors": [
                "Xiaoxue Zang",
                "Abhinav Rastogi",
                "Srinivas Sunkara",
                "Raghav Gupta",
                "Jianguo Zhang",
                "Jindong Chen"
            ],
            "abstract": "MultiWOZ is a well-known task-oriented dialogue dataset containing over\n10,000 annotated dialogues spanning 8 domains. It is extensively used as a\nbenchmark for dialogue state tracking. However, recent works have reported\npresence of substantial noise in the dialogue state annotations. MultiWOZ 2.1\nidentified and fixed many of these erroneous annotations and user utterances,\nresulting in an improved version of this dataset. This work introduces MultiWOZ\n2.2, which is a yet another improved version of this dataset. Firstly, we\nidentify and fix dialogue state annotation errors across 17.3% of the\nutterances on top of MultiWOZ 2.1. Secondly, we redefine the ontology by\ndisallowing vocabularies of slots with a large number of possible values (e.g.,\nrestaurant name, time of booking). In addition, we introduce slot span\nannotations for these slots to standardize them across recent models, which\npreviously used custom string matching heuristics to generate them. We also\nbenchmark a few state of the art dialogue state tracking models on the\ncorrected dataset to facilitate comparison for future work. In the end, we\ndiscuss best practices for dialogue data collection that can help avoid\nannotation errors.",
            "journal ref": "Proceedings of the 2nd Workshop on Natural Language Processing for\n  Conversational AI (2020) 109-117",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.12969v1",
            "date": "2020-07-25 16:29:24+00:00",
            "title": "Constructing a Testbed for Psychometric Natural Language Processing",
            "authors": [
                "Ahmed Abbasi",
                "David G. Dobolyi",
                "Richard G. Netemeyer"
            ],
            "abstract": "Psychometric measures of ability, attitudes, perceptions, and beliefs are\ncrucial for understanding user behaviors in various contexts including health,\nsecurity, e-commerce, and finance. Traditionally, psychometric dimensions have\nbeen measured and collected using survey-based methods. Inferring such\nconstructs from user-generated text could afford opportunities for timely,\nunobtrusive, collection and analysis. In this paper, we describe our efforts to\nconstruct a corpus for psychometric natural language processing (NLP). We\ndiscuss our multi-step process to align user text with their survey-based\nresponse items and provide an overview of the resulting testbed which\nencompasses survey-based psychometric measures and accompanying user-generated\ntext from over 8,500 respondents. We report preliminary results on the use of\nthe text to categorize/predict users' survey response labels. We also discuss\nthe important implications of our work and resulting testbed for future\npsychometric NLP research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13069v1",
            "date": "2020-07-26 07:13:32+00:00",
            "title": "A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges",
            "authors": [
                "Bin Fu",
                "Yunqi Qiu",
                "Chengguang Tang",
                "Yang Li",
                "Haiyang Yu",
                "Jian Sun"
            ],
            "abstract": "Question Answering (QA) over Knowledge Base (KB) aims to automatically answer\nnatural language questions via well-structured relation information between\nentities stored in knowledge bases. In order to make KBQA more applicable in\nactual scenarios, researchers have shifted their attention from simple\nquestions to complex questions, which require more KB triples and constraint\ninference. In this paper, we introduce the recent advances in complex QA.\nBesides traditional methods relying on templates and rules, the research is\ncategorized into a taxonomy that contains two main branches, namely Information\nRetrieval-based and Neural Semantic Parsing-based. After describing the methods\nof these branches, we analyze directions for future research and introduce the\nmodels proposed by the Alime team.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13306v1",
            "date": "2020-07-27 04:31:18+00:00",
            "title": "Public Sentiment Toward Solar Energy: Opinion Mining of Twitter Using a Transformer-Based Language Model",
            "authors": [
                "Serena Y. Kim",
                "Koushik Ganesan",
                "Princess Dickens",
                "Soumya Panda"
            ],
            "abstract": "Public acceptance and support for renewable energy are important determinants\nof renewable energy policies and market conditions. This paper examines public\nsentiment toward solar energy in the United States using data from Twitter, a\nmicro-blogging platform in which people post messages, known as tweets. We\nfiltered tweets specific to solar energy and performed a classification task\nusing Robustly optimized Bidirectional Encoder Representations from\nTransformers (RoBERTa). Analyzing 71,262 tweets during the period of late\nJanuary to early July 2020, we find public sentiment varies significantly\nacross states. Within the study period, the Northeastern U.S. region shows more\npositive sentiment toward solar energy than did the Southern U.S. region. Solar\nradiation does not correlate to variation in solar sentiment across states. We\nalso find that public sentiment toward solar correlates to renewable energy\npolicy and market conditions, specifically, Renewable Portfolio Standards (RPS)\ntargets, customer-friendly net metering policies, and a mature solar market.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13339v1",
            "date": "2020-07-27 07:44:00+00:00",
            "title": "NAYEL at SemEval-2020 Task 12: TF/IDF-Based Approach for Automatic Offensive Language Detection in Arabic Tweets",
            "authors": [
                "Hamada A. Nayel"
            ],
            "abstract": "In this paper, we present the system submitted to \"SemEval-2020 Task 12\". The\nproposed system aims at automatically identify the Offensive Language in Arabic\nTweets. A machine learning based approach has been used to design our system.\nWe implemented a linear classifier with Stochastic Gradient Descent (SGD) as\noptimization algorithm. Our model reported 84.20%, 81.82% f1-score on\ndevelopment set and test set respectively. The best performed system and the\nsystem in the last rank reported 90.17% and 44.51% f1-score on test set\nrespectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13626v1",
            "date": "2020-07-17 16:45:22+00:00",
            "title": "Neural Named Entity Recognition for Kazakh",
            "authors": [
                "Gulmira Tolegen",
                "Alymzhan Toleu",
                "Orken Mamyrbayev",
                "Rustam Mussabayev"
            ],
            "abstract": "We present several neural networks to address the task of named entity\nrecognition for morphologically complex languages (MCL). Kazakh is a\nmorphologically complex language in which each root/stem can produce hundreds\nor thousands of variant word forms. This nature of the language could lead to a\nserious data sparsity problem, which may prevent the deep learning models from\nbeing well trained for under-resourced MCLs. In order to model the MCLs' words\neffectively, we introduce root and entity tag embedding plus tensor layer to\nthe neural networks. The effects of those are significant for improving NER\nmodel performance of MCLs. The proposed models outperform state-of-the-art\nincluding character-based approaches, and can be potentially applied to other\nmorphologically complex languages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2007.13826v1",
            "date": "2020-07-27 19:42:42+00:00",
            "title": "Large Scale Subject Category Classification of Scholarly Papers with Deep Attentive Neural Networks",
            "authors": [
                "Bharath Kandimalla",
                "Shaurya Rohatgi",
                "Jian Wu",
                "C Lee Giles"
            ],
            "abstract": "Subject categories of scholarly papers generally refer to the knowledge\ndomain(s) to which the papers belong, examples being computer science or\nphysics. Subject category information can be used for building faceted search\nfor digital library search engines. This can significantly assist users in\nnarrowing down their search space of relevant documents. Unfortunately, many\nacademic papers do not have such information as part of their metadata.\nExisting methods for solving this task usually focus on unsupervised learning\nthat often relies on citation networks. However, a complete list of papers\nciting the current paper may not be readily available. In particular, new\npapers that have few or no citations cannot be classified using such methods.\nHere, we propose a deep attentive neural network (DANN) that classifies\nscholarly papers using only their abstracts. The network is trained using 9\nmillion abstracts from Web of Science (WoS). We also use the WoS schema that\ncovers 104 subject categories. The proposed network consists of two\nbi-directional recurrent neural networks followed by an attention layer. We\ncompare our model against baselines by varying the architecture and text\nrepresentation. Our best model achieves micro-F1 measure of 0.76 with F1 of\nindividual subject categories ranging from 0.50-0.95. The results showed the\nimportance of retraining word embedding models to maximize the vocabulary\noverlap and the effectiveness of the attention mechanism. The combination of\nword vectors with TFIDF outperforms character and sentence level embedding\nmodels. We discuss imbalanced samples and overlapping categories and suggest\npossible strategies for mitigation. We also determine the subject category\ndistribution in CiteSeerX by classifying a random sample of one million\nacademic papers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13840v3",
            "date": "2020-07-27 20:12:30+00:00",
            "title": "Characterizing the Effect of Sentence Context on Word Meanings: Mapping Brain to Behavior",
            "authors": [
                "N. Aguirre-Celis",
                "R. Miikkulainen"
            ],
            "abstract": "Semantic feature models have become a popular tool for prediction and\ninterpretation of fMRI data. In particular, prior work has shown that\ndifferences in the fMRI patterns in sentence reading can be explained by\ncontext-dependent changes in the semantic feature representations of the words.\nHowever, whether the subjects are aware of such changes and agree with them has\nbeen an open question. This paper aims to answer this question through a\nhuman-subject study. Subjects were asked to judge how the word change from\ntheir generic meaning when the words were used in specific sentences. The\njudgements were consistent with the model predictions well above chance. Thus,\nthe results support the hypothesis that word meaning change systematically\ndepending on sentence context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.13974v1",
            "date": "2020-07-28 03:47:26+00:00",
            "title": "SalamNET at SemEval-2020 Task12: Deep Learning Approach for Arabic Offensive Language Detection",
            "authors": [
                "Fatemah Husain",
                "Jooyeon Lee",
                "Samuel Henry",
                "Ozlem Uzuner"
            ],
            "abstract": "This paper describes SalamNET, an Arabic offensive language detection system\nthat has been submitted to SemEval 2020 shared task 12: Multilingual Offensive\nLanguage Identification in Social Media. Our approach focuses on applying\nmultiple deep learning models and conducting in depth error analysis of results\nto provide system implications for future development considerations. To pursue\nour goal, a Recurrent Neural Network (RNN), a Gated Recurrent Unit (GRU), and\nLong-Short Term Memory (LSTM) models with different design architectures have\nbeen developed and evaluated. The SalamNET, a Bi-directional Gated Recurrent\nUnit (Bi-GRU) based model, reports a macro-F1 score of 0.83.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14626v1",
            "date": "2020-07-29 06:32:18+00:00",
            "title": "Object-and-Action Aware Model for Visual Language Navigation",
            "authors": [
                "Yuankai Qi",
                "Zizheng Pan",
                "Shengping Zhang",
                "Anton van den Hengel",
                "Qi Wu"
            ],
            "abstract": "Vision-and-Language Navigation (VLN) is unique in that it requires turning\nrelatively general natural-language instructions into robot agent actions, on\nthe basis of the visible environment. This requires to extract value from two\nvery different types of natural-language information. The first is object\ndescription (e.g., 'table', 'door'), each presenting as a tip for the agent to\ndetermine the next action by finding the item visible in the environment, and\nthe second is action specification (e.g., 'go straight', 'turn left') which\nallows the robot to directly predict the next movements without relying on\nvisual perceptions. However, most existing methods pay few attention to\ndistinguish these information from each other during instruction encoding and\nmix together the matching between textual object/action encoding and visual\nperception/orientation features of candidate viewpoints. In this paper, we\npropose an Object-and-Action Aware Model (OAAM) that processes these two\ndifferent forms of natural language based instruction separately. This enables\neach process to match object-centered/action-centered instruction to their own\ncounterpart visual perception/action orientation flexibly. However, one\nside-issue caused by above solution is that an object mentioned in instructions\nmay be observed in the direction of two or more candidate viewpoints, thus the\nOAAM may not predict the viewpoint on the shortest path as the next action. To\nhandle this problem, we design a simple but effective path loss to penalize\ntrajectories deviating from the ground truth path. Experimental results\ndemonstrate the effectiveness of the proposed model and path loss, and the\nsuperiority of their combination with a 50% SPL score on the R2R dataset and a\n40% CLS score on the R4R dataset in unseen environments, outperforming the\nprevious state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.14682v1",
            "date": "2020-07-29 08:58:52+00:00",
            "title": "Enriching Video Captions With Contextual Text",
            "authors": [
                "Philipp Rimle",
                "Pelin Dogan",
                "Markus Gross"
            ],
            "abstract": "Understanding video content and generating caption with context is an\nimportant and challenging task. Unlike prior methods that typically attempt to\ngenerate generic video captions without context, our architecture\ncontextualizes captioning by infusing extracted information from relevant text\ndata. We propose an end-to-end sequence-to-sequence model which generates video\ncaptions based on visual input, and mines relevant knowledge such as names and\nlocations from contextual text. In contrast to previous approaches, we do not\npreprocess the text further, and let the model directly learn to attend over\nit. Guided by the visual input, the model is able to copy words from the\ncontextual text via a pointer-generator network, allowing to produce more\nspecific video captions. We show competitive performance on the News Video\nDataset and, through ablation studies, validate the efficacy of contextual\nvideo captioning as well as individual design choices in our model\narchitecture.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2007.15135v1",
            "date": "2020-07-29 22:12:49+00:00",
            "title": "The Return of Lexical Dependencies: Neural Lexicalized PCFGs",
            "authors": [
                "Hao Zhu",
                "Yonatan Bisk",
                "Graham Neubig"
            ],
            "abstract": "In this paper we demonstrate that $\\textit{context free grammar (CFG) based\nmethods for grammar induction benefit from modeling lexical dependencies}$.\nThis contrasts to the most popular current methods for grammar induction, which\nfocus on discovering $\\textit{either}$ constituents $\\textit{or}$ dependencies.\nPrevious approaches to marry these two disparate syntactic formalisms (e.g.\nlexicalized PCFGs) have been plagued by sparsity, making them unsuitable for\nunsupervised grammar induction. However, in this work, we present novel neural\nmodels of lexicalized PCFGs which allow us to overcome sparsity problems and\neffectively induce both constituents and dependencies within a single model.\nExperiments demonstrate that this unified framework results in stronger results\non both representations than achieved when modeling either formalism alone.\nCode is available at https://github.com/neulab/neural-lpcfg.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15779v4",
            "date": "2020-07-31 00:04:15+00:00",
            "title": "Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing",
            "authors": [
                "Yu Gu",
                "Robert Tinn",
                "Hao Cheng",
                "Michael Lucas",
                "Naoto Usuyama",
                "Xiaodong Liu",
                "Tristan Naumann",
                "Jianfeng Gao",
                "Hoifung Poon"
            ],
            "abstract": "Pretraining large neural language models, such as BERT, has led to impressive\ngains on many natural language processing (NLP) tasks. However, most\npretraining efforts focus on general domain corpora, such as newswire and Web.\nA prevailing assumption is that even domain-specific pretraining can benefit by\nstarting from general-domain language models. In this paper, we challenge this\nassumption by showing that for domains with abundant unlabeled text, such as\nbiomedicine, pretraining language models from scratch results in substantial\ngains over continual pretraining of general-domain language models. To\nfacilitate this investigation, we compile a comprehensive biomedical NLP\nbenchmark from publicly-available datasets. Our experiments show that\ndomain-specific pretraining serves as a solid foundation for a wide range of\nbiomedical NLP tasks, leading to new state-of-the-art results across the board.\nFurther, in conducting a thorough evaluation of modeling choices, both for\npretraining and task-specific fine-tuning, we discover that some common\npractices are unnecessary with BERT models, such as using complex tagging\nschemes in named entity recognition (NER). To help accelerate research in\nbiomedical NLP, we have released our state-of-the-art pretrained and\ntask-specific models for the community, and created a leaderboard featuring our\nBLURB benchmark (short for Biomedical Language Understanding & Reasoning\nBenchmark) at https://aka.ms/BLURB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15871v1",
            "date": "2020-07-31 07:00:34+00:00",
            "title": "Improving NER's Performance with Massive financial corpus",
            "authors": [
                "Han Zhang"
            ],
            "abstract": "Training large deep neural networks needs massive high quality annotation\ndata, but the time and labor costs are too expensive for small business. We\nstart a company-name recognition task with a small scale and low quality\ntraining data, then using skills to enhanced model training speed and\npredicting performance with minimum labor cost. The methods we use involve\npre-training a lite language model such as Albert-small or Electra-small in\nfinancial corpus, knowledge of distillation and multi-stage learning. The\nresult is that we raised the recall rate by nearly 20 points and get 4 times as\nfast as BERT-CRF model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.15916v1",
            "date": "2020-07-31 09:21:13+00:00",
            "title": "Evaluating Automatically Generated Phoneme Captions for Images",
            "authors": [
                "Justin van der Hout",
                "Zolt\u00e1n D'Haese",
                "Mark Hasegawa-Johnson",
                "Odette Scharenborg"
            ],
            "abstract": "Image2Speech is the relatively new task of generating a spoken description of\nan image. This paper presents an investigation into the evaluation of this\ntask. For this, first an Image2Speech system was implemented which generates\nimage captions consisting of phoneme sequences. This system outperformed the\noriginal Image2Speech system on the Flickr8k corpus. Subsequently, these\nphoneme captions were converted into sentences of words. The captions were\nrated by human evaluators for their goodness of describing the image. Finally,\nseveral objective metric scores of the results were correlated with these human\nratings. Although BLEU4 does not perfectly correlate with human ratings, it\nobtained the highest correlation among the investigated metrics, and is the\nbest currently existing metric for the Image2Speech task. Current metrics are\nlimited by the fact that they assume their input to be words. A more\nappropriate metric for the Image2Speech task should assume its input to be\nparts of words, i.e. phonemes, instead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16007v2",
            "date": "2020-07-23 08:51:09+00:00",
            "title": "Exploring Swedish & English fastText Embeddings for NER with the Transformer",
            "authors": [
                "Tosin P. Adewumi",
                "Foteini Liwicki",
                "Marcus Liwicki"
            ],
            "abstract": "In this paper, our main contributions are that embeddings from relatively\nsmaller corpora can outperform ones from larger corpora and we make the new\nSwedish analogy test set publicly available. To achieve a good network\nperformance in natural language processing (NLP) downstream tasks, several\nfactors play important roles: dataset size, the right hyper-parameters, and\nwell-trained embeddings. We show that, with the right set of hyper-parameters,\ngood network performance can be reached even on smaller datasets. We evaluate\nthe embeddings at both the intrinsic and extrinsic levels. The embeddings are\ndeployed with the Transformer in named entity recognition (NER) task and\nsignificance tests conducted. This is done for both Swedish and English. We\nobtain better performance in both languages on the downstream task with smaller\ntraining data, compared to recently released, Common Crawl versions; and\ncharacter n-grams appear useful for Swedish, a morphologically rich language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16008v1",
            "date": "2020-07-22 13:44:57+00:00",
            "title": "Multi-task learning for natural language processing in the 2020s: where are we going?",
            "authors": [
                "Joseph Worsham",
                "Jugal Kalita"
            ],
            "abstract": "Multi-task learning (MTL) significantly pre-dates the deep learning era, and\nit has seen a resurgence in the past few years as researchers have been\napplying MTL to deep learning solutions for natural language tasks. While\nsteady MTL research has always been present, there is a growing interest driven\nby the impressive successes published in the related fields of transfer\nlearning and pre-training, such as BERT, and the release of new challenge\nproblems, such as GLUE and the NLP Decathlon (decaNLP). These efforts place\nmore focus on how weights are shared across networks, evaluate the re-usability\nof network components and identify use cases where MTL can significantly\noutperform single-task solutions. This paper strives to provide a comprehensive\nsurvey of the numerous recent MTL contributions to the field of natural\nlanguage processing and provide a forum to focus efforts on the hardest\nunsolved problems in the next decade. While novel models that improve\nperformance on NLP benchmarks are continually produced, lasting MTL challenges\nremain unsolved which could hold the key to better language understanding,\nknowledge discovery and natural language interfaces.",
            "journal ref": "Pattern Recognition Letters 136 (2020) 120-126",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16009v1",
            "date": "2020-07-17 17:51:29+00:00",
            "title": "Toward Givenness Hierarchy Theoretic Natural Language Generation",
            "authors": [
                "Poulomi Pal",
                "Tom Williams"
            ],
            "abstract": "Language-capable interactive robots participating in dialogues with human\ninterlocutors must be able to naturally and efficiently communicate about the\nentities in their environment. A key aspect of such communication is the use of\nanaphoric language. The linguistic theory of the Givenness Hierarchy(GH)\nsuggests that humans use anaphora based on the cognitive statuses their\nreferents have in the minds of their interlocutors. In previous work,\nresearchers presented GH-theoretic approaches to robot anaphora understanding.\nIn this paper we describe how the GH might need to be used quite differently to\nfacilitate robot anaphora generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2007.16127v1",
            "date": "2020-07-31 15:14:05+00:00",
            "title": "Robust Benchmarking for Machine Learning of Clinical Entity Extraction",
            "authors": [
                "Monica Agrawal",
                "Chloe O'Connell",
                "Yasmin Fatemi",
                "Ariel Levy",
                "David Sontag"
            ],
            "abstract": "Clinical studies often require understanding elements of a patient's\nnarrative that exist only in free text clinical notes. To transform notes into\nstructured data for downstream use, these elements are commonly extracted and\nnormalized to medical vocabularies. In this work, we audit the performance of\nand indicate areas of improvement for state-of-the-art systems. We find that\nhigh task accuracies for clinical entity normalization systems on the 2019 n2c2\nShared Task are misleading, and underlying performance is still brittle.\nNormalization accuracy is high for common concepts (95.3%), but much lower for\nconcepts unseen in training data (69.3%). We demonstrate that current\napproaches are hindered in part by inconsistencies in medical vocabularies,\nlimitations of existing labeling schemas, and narrow evaluation techniques. We\nreformulate the annotation framework for clinical entity extraction to factor\nin these issues to allow for robust end-to-end system benchmarking. We evaluate\nconcordance of annotations from our new framework between two annotators and\nachieve a Jaccard similarity of 0.73 for entity recognition and an agreement of\n0.83 for entity normalization. We propose a path forward to address the\ndemonstrated need for the creation of a reference standard to spur method\ndevelopment in entity recognition and normalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00036v2",
            "date": "2020-07-31 19:01:13+00:00",
            "title": "TweepFake: about Detecting Deepfake Tweets",
            "authors": [
                "Tiziano Fagni",
                "Fabrizio Falchi",
                "Margherita Gambini",
                "Antonio Martella",
                "Maurizio Tesconi"
            ],
            "abstract": "The recent advances in language modeling significantly improved the\ngenerative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a\npre-trained language model that can autonomously generate coherent, non-trivial\nand human-like text samples. Since then, ever more powerful text generative\nmodels have been developed. Adversaries can exploit these tremendous generative\ncapabilities to enhance social bots that will have the ability to write\nplausible deepfake messages, hoping to contaminate public debate. To prevent\nthis, it is crucial to develop deepfake social media messages detection\nsystems. However, to the best of our knowledge no one has ever addressed the\ndetection of machine-generated texts on social networks like Twitter or\nFacebook. With the aim of helping the research in this detection field, we\ncollected the first dataset of \\real deepfake tweets, TweepFake. It is real in\nthe sense that each deepfake tweet was actually posted on Twitter. We collected\ntweets from a total of 23 bots, imitating 17 human accounts. The bots are based\non various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM,\nGPT-2. We also randomly selected tweets from the humans imitated by the bots to\nhave an overall balanced dataset of 25,572 tweets (half human and half bots\ngenerated). The dataset is publicly available on Kaggle. Lastly, we evaluated\n13 deepfake text detection methods (based on various state-of-the-art\napproaches) to both demonstrate the challenges that Tweepfake poses and create\na solid baseline of detection techniques. We hope that TweepFake can offer the\nopportunity to tackle the deepfake detection on social media messages as well.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00140v1",
            "date": "2020-08-01 01:05:55+00:00",
            "title": "Experiments in Extractive Summarization: Integer Linear Programming, Term/Sentence Scoring, and Title-driven Models",
            "authors": [
                "Daniel Lee",
                "Rakesh Verma",
                "Avisha Das",
                "Arjun Mukherjee"
            ],
            "abstract": "In this paper, we revisit the challenging problem of unsupervised\nsingle-document summarization and study the following aspects: Integer linear\nprogramming (ILP) based algorithms, Parameterized normalization of term and\nsentence scores, and Title-driven approaches for summarization. We describe a\nnew framework, NewsSumm, that includes many existing and new approaches for\nsummarization including ILP and title-driven approaches. NewsSumm's flexibility\nallows to combine different algorithms and sentence scoring schemes seamlessly.\nOur results combining sentence scoring with ILP and normalization are in\ncontrast to previous work on this topic, showing the importance of a broader\nsearch for optimal parameters. We also show that the new title-driven reduction\nidea leads to improvement in performance for both unsupervised and supervised\napproaches considered.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.00279v1",
            "date": "2020-08-01 15:10:11+00:00",
            "title": "An Empirical Study of Clarifying Question-Based Systems",
            "authors": [
                "Jie Zou",
                "Evangelos Kanoulas",
                "Yiqun Liu"
            ],
            "abstract": "Search and recommender systems that take the initiative to ask clarifying\nquestions to better understand users' information needs are receiving\nincreasing attention from the research community. However, to the best of our\nknowledge, there is no empirical study to quantify whether and to what extent\nusers are willing or able to answer these questions. In this work, we conduct\nan online experiment by deploying an experimental system, which interacts with\nusers by asking clarifying questions against a product repository. We collect\nboth implicit interaction behavior data and explicit feedback from users\nshowing that: (a) users are willing to answer a good number of clarifying\nquestions (11-21 on average), but not many more than that; (b) most users\nanswer questions until they reach the target product, but also a fraction of\nthem stops due to fatigue or due to receiving irrelevant questions; (c) part of\nthe users' answers (12-17%) are actually opposite to the description of the\ntarget product; while (d) most of the users (66-84%) find the question-based\nsystem helpful towards completing their tasks. Some of the findings of the\nstudy contradict current assumptions on simulated evaluations in the field,\nwhile they point towards improvements in the evaluation framework and can\ninspire future interactive search/recommender system designs.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.00304v1",
            "date": "2020-08-01 17:34:37+00:00",
            "title": "SemEval-2020 Task 7: Assessing Humor in Edited News Headlines",
            "authors": [
                "Nabil Hossain",
                "John Krumm",
                "Michael Gamon",
                "Henry Kautz"
            ],
            "abstract": "This paper describes the SemEval-2020 shared task \"Assessing Humor in Edited\nNews Headlines.\" The task's dataset contains news headlines in which short\nedits were applied to make them funny, and the funniness of these edited\nheadlines was rated using crowdsourcing. This task includes two subtasks, the\nfirst of which is to estimate the funniness of headlines on a humor scale in\nthe interval 0-3. The second subtask is to predict, for a pair of edited\nversions of the same original headline, which is the funnier version. To date,\nthis task is the most popular shared computational humor task, attracting 48\nteams for the first subtask and 31 teams for the second.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00545v2",
            "date": "2020-08-02 19:30:39+00:00",
            "title": "Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages",
            "authors": [
                "Badr M. Abdullah",
                "Tania Avgustinova",
                "Bernd M\u00f6bius",
                "Dietrich Klakow"
            ],
            "abstract": "State-of-the-art spoken language identification (LID) systems, which are\nbased on end-to-end deep neural networks, have shown remarkable success not\nonly in discriminating between distant languages but also between\nclosely-related languages or even different spoken varieties of the same\nlanguage. However, it is still unclear to what extent neural LID models\ngeneralize to speech samples with different acoustic conditions due to domain\nshift. In this paper, we present a set of experiments to investigate the impact\nof domain mismatch on the performance of neural LID systems for a subset of six\nSlavic languages across two domains (read speech and radio broadcast) and\nexamine two low-level signal descriptors (spectral and cepstral features) for\nthis task. Our experiments show that (1) out-of-domain speech samples severely\nhinder the performance of neural LID models, and (2) while both spectral and\ncepstral features show comparable performance within-domain, spectral features\nshow more robustness under domain mismatch. Moreover, we apply unsupervised\ndomain adaptation to minimize the discrepancy between the two domains in our\nstudy. We achieve relative accuracy improvements that range from 9% to 77%\ndepending on the diversity of acoustic conditions in the source domain.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.00623v2",
            "date": "2020-08-03 03:08:29+00:00",
            "title": "DeLighT: Deep and Light-weight Transformer",
            "authors": [
                "Sachin Mehta",
                "Marjan Ghazvininejad",
                "Srinivasan Iyer",
                "Luke Zettlemoyer",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "We introduce a deep and light-weight transformer, DeLighT, that delivers\nsimilar or better performance than standard transformer-based models with\nsignificantly fewer parameters. DeLighT more efficiently allocates parameters\nboth (1) within each Transformer block using the DeLighT transformation, a deep\nand light-weight transformation, and (2) across blocks using block-wise\nscaling, which allows for shallower and narrower DeLighT blocks near the input\nand wider and deeper DeLighT blocks near the output. Overall, DeLighT networks\nare 2.5 to 4 times deeper than standard transformer models and yet have fewer\nparameters and operations. Experiments on benchmark machine translation and\nlanguage modeling tasks show that DeLighT matches or improves the performance\nof baseline Transformers with 2 to 3 times fewer parameters on average. Our\nsource code is available at: \\url{https://github.com/sacmehta/delight}",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2008.00702v1",
            "date": "2020-08-03 08:13:09+00:00",
            "title": "Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speech",
            "authors": [
                "Monica Sunkara",
                "Srikanth Ronanki",
                "Dhanush Bekal",
                "Sravan Bodapati",
                "Katrin Kirchhoff"
            ],
            "abstract": "In this work, we explore a multimodal semi-supervised learning approach for\npunctuation prediction by learning representations from large amounts of\nunlabelled audio and text data. Conventional approaches in speech processing\ntypically use forced alignment to encoder per frame acoustic features to word\nlevel features and perform multimodal fusion of the resulting acoustic and\nlexical representations. As an alternative, we explore attention based\nmultimodal fusion and compare its performance with forced alignment based\nfusion. Experiments conducted on the Fisher corpus show that our proposed\napproach achieves ~6-9% and ~3-4% absolute improvement (F1 score) over the\nbaseline BLSTM model on reference transcripts and ASR outputs respectively. We\nfurther improve the model robustness to ASR errors by performing data\naugmentation with N-best lists which achieves up to an additional ~2-6%\nimprovement on ASR outputs. We also demonstrate the effectiveness of\nsemi-supervised learning approach by performing ablation study on various sizes\nof the corpus. When trained on 1 hour of speech and text data, the proposed\nmodel achieved ~9-18% absolute improvement over baseline model.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.00731v1",
            "date": "2020-08-03 09:09:12+00:00",
            "title": "Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metrics",
            "authors": [
                "Okko R\u00e4s\u00e4nen",
                "Mar\u00eda Andrea Cruz Bland\u00f3n"
            ],
            "abstract": "Unsupervised spoken term discovery (UTD) aims at finding recurring segments\nof speech from a corpus of acoustic speech data. One potential approach to this\nproblem is to use dynamic time warping (DTW) to find well-aligning patterns\nfrom the speech data. However, automatic selection of initial candidate\nsegments for the DTW-alignment and detection of \"sufficiently good\" alignments\namong those require some type of pre-defined criteria, often operationalized as\nthreshold parameters for pair-wise distance metrics between signal\nrepresentations. In the existing UTD systems, the optimal hyperparameters may\ndiffer across datasets, limiting their applicability to new corpora and truly\nlow-resource scenarios. In this paper, we propose a novel probabilistic\napproach to DTW-based UTD named as PDTW. In PDTW, distributional\ncharacteristics of the processed corpus are utilized for adaptive evaluation of\nalignment quality, thereby enabling systematic discovery of pattern pairs that\nhave similarity what would be expected by coincidence. We test PDTW on Zero\nResource Speech Challenge 2017 datasets as a part of 2020 implementation of the\nchallenge. The results show that the system performs consistently on all five\ntested languages using fixed hyperparameters, clearly outperforming the earlier\nDTW-based system in terms of coverage of the detected patterns.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.00774v3",
            "date": "2020-08-03 10:51:10+00:00",
            "title": "Elsevier OA CC-By Corpus",
            "authors": [
                "Daniel Kershaw",
                "Rob Koeling"
            ],
            "abstract": "We introduce the Elsevier OA CC-BY corpus. This is the first open corpus of\nScientific Research papers which has a representative sample from across\nscientific disciplines. This corpus not only includes the full text of the\narticle, but also the metadata of the documents, along with the bibliographic\ninformation for each reference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.00791v4",
            "date": "2020-08-03 11:44:22+00:00",
            "title": "Characterizing COVID-19 Misinformation Communities Using a Novel Twitter Dataset",
            "authors": [
                "Shahan Ali Memon",
                "Kathleen M. Carley"
            ],
            "abstract": "From conspiracy theories to fake cures and fake treatments, COVID-19 has\nbecome a hot-bed for the spread of misinformation online. It is more important\nthan ever to identify methods to debunk and correct false information online.\nIn this paper, we present a methodology and analyses to characterize the two\ncompeting COVID-19 misinformation communities online: (i) misinformed users or\nusers who are actively posting misinformation, and (ii) informed users or users\nwho are actively spreading true information, or calling out misinformation. The\ngoals of this study are two-fold: (i) collecting a diverse set of annotated\nCOVID-19 Twitter dataset that can be used by the research community to conduct\nmeaningful analysis; and (ii) characterizing the two target communities in\nterms of their network structure, linguistic patterns, and their membership in\nother communities. Our analyses show that COVID-19 misinformed communities are\ndenser, and more organized than informed communities, with a possibility of a\nhigh volume of the misinformation being part of disinformation campaigns. Our\nanalyses also suggest that a large majority of misinformed users may be\nanti-vaxxers. Finally, our sociolinguistic analyses suggest that COVID-19\ninformed users tend to use more narratives than misinformed users.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2008.00920v1",
            "date": "2020-08-03 14:54:09+00:00",
            "title": "On The Plurality of Graphs",
            "authors": [
                "Nicole Fitzgerald",
                "Jacopo Tagliabue"
            ],
            "abstract": "We conduct a series of experiments designed to empirically demonstrate the\neffects of varying the structural features of a multi-agent emergent\ncommunication game framework. Specifically, we model the interactions (edges)\nbetween individual agents (nodes)as the structure of a graph generated\naccording to a series of known random graph generating algorithms. Confirming\nthe hypothesis proposed in [10], we show that the two factors of variation\ninduced in this work, namely 1) the graph-generating process and 2) the\ncentrality measure according to which edges are sampled, in fact play a\nsignificant role in determining the dynamics of language emergence within the\npopulation at hand.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2008.00956v1",
            "date": "2020-07-31 03:29:49+00:00",
            "title": "Interactive Text Graph Mining with a Prolog-based Dialog Engine",
            "authors": [
                "Paul Tarau",
                "Eduardo Blanco"
            ],
            "abstract": "On top of a neural network-based dependency parser and a graph-based natural\nlanguage processing module we design a Prolog-based dialog engine that explores\ninteractively a ranked fact database extracted from a text document.\n  We reorganize dependency graphs to focus on the most relevant content\nelements of a sentence and integrate sentence identifiers as graph nodes.\n  Additionally, after ranking the graph we take advantage of the implicit\nsemantic information that dependency links and WordNet bring in the form of\nsubject-verb-object, is-a and part-of relations.\n  Working on the Prolog facts and their inferred consequences, the dialog\nengine specializes the text graph with respect to a query and reveals\ninteractively the document's most relevant content elements.\n  The open-source code of the integrated system is available at\nhttps://github.com/ptarau/DeepRank .\n  Under consideration in Theory and Practice of Logic Programming (TPLP).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01297v2",
            "date": "2020-08-04 03:01:23+00:00",
            "title": "An improved Bayesian TRIE based model for SMS text normalization",
            "authors": [
                "Abhinava Sikdar",
                "Niladri Chatterjee"
            ],
            "abstract": "Normalization of SMS text, commonly known as texting language, is being\npursued for more than a decade. A probabilistic approach based on the Trie data\nstructure was proposed in literature which was found to be better performing\nthan HMM based approaches proposed earlier in predicting the correct\nalternative for an out-of-lexicon word. However, success of the Trie based\napproach depends largely on how correctly the underlying probabilities of word\noccurrences are estimated. In this work we propose a structural modification to\nthe existing Trie-based model along with a novel training algorithm and\nprobability generation scheme. We prove two theorems on statistical properties\nof the proposed Trie and use them to claim that is an unbiased and consistent\nestimator of the occurrence probabilities of the words. We further fuse our\nmodel into the paradigm of noisy channel based error correction and provide a\nheuristic to go beyond a Damerau Levenshtein distance of one. We also run\nsimulations to support our claims and show superiority of the proposed scheme\nover previous works.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01515v1",
            "date": "2020-07-29 22:12:26+00:00",
            "title": "Predicting Multiple ICD-10 Codes from Brazilian-Portuguese Clinical Notes",
            "authors": [
                "Arthur D. Reys",
                "Danilo Silva",
                "Daniel Severo",
                "Saulo Pedro",
                "Marcia M. de Souza e S\u00e1",
                "Guilherme A. C. Salgado"
            ],
            "abstract": "ICD coding from electronic clinical records is a manual, time-consuming and\nexpensive process. Code assignment is, however, an important task for billing\npurposes and database organization. While many works have studied the problem\nof automated ICD coding from free text using machine learning techniques, most\nuse records in the English language, especially from the MIMIC-III public\ndataset. This work presents results for a dataset with Brazilian Portuguese\nclinical notes. We develop and optimize a Logistic Regression model, a\nConvolutional Neural Network (CNN), a Gated Recurrent Unit Neural Network and a\nCNN with Attention (CNN-Att) for prediction of diagnosis ICD codes. We also\nreport our results for the MIMIC-III dataset, which outperform previous work\namong models of the same families, as well as the state of the art. Compared to\nMIMIC-III, the Brazilian Portuguese dataset contains far fewer words per\ndocument, when only discharge summaries are used. We experiment concatenating\nadditional documents available in this dataset, achieving a great boost in\nperformance. The CNN-Att model achieves the best results on both datasets, with\nmicro-averaged F1 score of 0.537 on MIMIC-III and 0.485 on our dataset with\nadditional documents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01526v1",
            "date": "2020-07-17 21:05:44+00:00",
            "title": "Multi-Perspective Semantic Information Retrieval in the Biomedical Domain",
            "authors": [
                "Samarth Rawal"
            ],
            "abstract": "Information Retrieval (IR) is the task of obtaining pieces of data (such as\ndocuments) that are relevant to a particular query or need from a large\nrepository of information. IR is a valuable component of several downstream\nNatural Language Processing (NLP) tasks. Practically, IR is at the heart of\nmany widely-used technologies like search engines. While probabilistic ranking\nfunctions like the Okapi BM25 function have been utilized in IR systems since\nthe 1970's, modern neural approaches pose certain advantages compared to their\nclassical counterparts. In particular, the release of BERT (Bidirectional\nEncoder Representations from Transformers) has had a significant impact in the\nNLP community by demonstrating how the use of a Masked Language Model trained\non a large corpus of data can improve a variety of downstream NLP tasks,\nincluding sentence classification and passage re-ranking. IR Systems are also\nimportant in the biomedical and clinical domains. Given the increasing amount\nof scientific literature across biomedical domain, the ability find answers to\nspecific clinical queries from a repository of millions of articles is a matter\nof practical value to medical professionals. Moreover, there are\ndomain-specific challenges present, including handling clinical jargon and\nevaluating the similarity or relatedness of various medical symptoms when\ndetermining the relevance between a query and a sentence. This work presents\ncontributions to several aspects of the Biomedical Semantic Information\nRetrieval domain. First, it introduces Multi-Perspective Sentence Relevance, a\nnovel methodology of utilizing BERT-based models for contextual IR. The system\nis evaluated using the BioASQ Biomedical IR Challenge. Finally, practical\ncontributions in the form of a live IR system for medics and a proposed\nchallenge on the Living Systematic Review clinical task are provided.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.01533v2",
            "date": "2020-07-31 00:23:18+00:00",
            "title": "Writer Identification Using Microblogging Texts for Social Media Forensics",
            "authors": [
                "Fernando Alonso-Fernandez",
                "Nicole Mariah Sharon Belvisi",
                "Kevin Hernandez-Diaz",
                "Naveed Muhammad",
                "Josef Bigun"
            ],
            "abstract": "Establishing authorship of online texts is fundamental to combat cybercrimes.\nUnfortunately, text length is limited on some platforms, making the challenge\nharder. We aim at identifying the authorship of Twitter messages limited to 140\ncharacters. We evaluate popular stylometric features, widely used in literary\nanalysis, and specific Twitter features like URLs, hashtags, replies or quotes.\nWe use two databases with 93 and 3957 authors, respectively. We test varying\nsized author sets and varying amounts of training/test texts per author.\nPerformance is further improved by feature combination via automatic selection.\nWith a large number of training Tweets (>500), a good accuracy (Rank-5>80%) is\nachievable with only a few dozens of test Tweets, even with several thousands\nof authors. With smaller sample sizes (10-20 training Tweets), the search space\ncan be diminished by 9-15% while keeping a high chance that the correct author\nis retrieved among the candidates. In such cases, automatic attribution can\nprovide significant time savings to experts in suspect search. For\ncompleteness, we report verification results. With few training/test Tweets,\nthe EER is above 20-25%, which is reduced to < 15% if hundreds of training\nTweets are available. We also quantify the computational complexity and time\npermanence of the employed features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01547v2",
            "date": "2020-07-28 13:42:02+00:00",
            "title": "TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling",
            "authors": [
                "Shuai Zhang",
                "Peng Zhang",
                "Xindian Ma",
                "Junqiu Wei",
                "Ningning Wang",
                "Qun Liu"
            ],
            "abstract": "Transformer has been widely-used in many Natural Language Processing (NLP)\ntasks and the scaled dot-product attention between tokens is a core module of\nTransformer. This attention is a token-wise design and its complexity is\nquadratic to the length of sequence, limiting its application potential for\nlong sequence tasks. In this paper, we propose a dimension-wise attention\nmechanism based on which a novel language modeling approach (namely\nTensorCoder) can be developed. The dimension-wise attention can reduce the\nattention complexity from the original $O(N^2d)$ to $O(Nd^2)$, where $N$ is the\nlength of the sequence and $d$ is the dimensionality of head. We verify\nTensorCoder on two tasks including masked language modeling and neural machine\ntranslation. Compared with the original Transformer, TensorCoder not only\ngreatly reduces the calculation of the original model but also obtains improved\nperformance on masked language modeling task (in PTB dataset) and comparable\nperformance on machine translation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01548v1",
            "date": "2020-07-28 04:11:10+00:00",
            "title": "Defining and Evaluating Fair Natural Language Generation",
            "authors": [
                "Catherine Yeo",
                "Alyssa Chen"
            ],
            "abstract": "Our work focuses on the biases that emerge in the natural language generation\n(NLG) task of sentence completion. In this paper, we introduce a framework of\nfairness for NLG followed by an evaluation of gender biases in two\nstate-of-the-art language models. Our analysis provides a theoretical\nformulation for biases in NLG and empirical evidence that existing language\ngeneration models embed gender bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01551v1",
            "date": "2020-07-26 04:50:47+00:00",
            "title": "To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection",
            "authors": [
                "Aparna Balagopalan",
                "Benjamin Eyre",
                "Frank Rudzicz",
                "Jekaterina Novikova"
            ],
            "abstract": "Research related to automatically detecting Alzheimer's disease (AD) is\nimportant, given the high prevalence of AD and the high cost of traditional\nmethods. Since AD significantly affects the content and acoustics of\nspontaneous speech, natural language processing and machine learning provide\npromising techniques for reliably detecting AD. We compare and contrast the\nperformance of two such approaches for AD detection on the recent ADReSS\nchallenge dataset: 1) using domain knowledge-based hand-crafted features that\ncapture linguistic and acoustic phenomena, and 2) fine-tuning Bidirectional\nEncoder Representations from Transformer (BERT)-based sequence classification\nmodels. We also compare multiple feature-based regression models for a\nneuropsychological score task in the challenge. We observe that fine-tuned BERT\nmodels, given the relative importance of linguistics in cognitive impairment\ndetection, outperform feature-based approaches on the AD detection task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01564v1",
            "date": "2020-08-01 11:55:03+00:00",
            "title": "LXPER Index: a curriculum-specific text readability assessment model for EFL students in Korea",
            "authors": [
                "Bruce W. Lee",
                "Jason Hyung-Jong Lee"
            ],
            "abstract": "Automatic readability assessment is one of the most important applications of\nNatural Language Processing (NLP) in education. Since automatic readability\nassessment allows the fast selection of appropriate reading material for\nreaders at all levels of proficiency, it can be particularly useful for the\nEnglish education of English as Foreign Language (EFL) students around the\nworld. Most readability assessment models are developed for the native readers\nof English and have low accuracy for texts in the non-native English Language\nTraining (ELT) curriculum. We introduce LXPER Index, which is a readability\nassessment model for non-native EFL readers in the ELT curriculum of Korea. Our\nexperiments show that our new model, trained with CoKEC-text (Text Corpus of\nthe Korean ELT Curriculum), significantly improves the accuracy of automatic\nreadability assessment for texts in the Korean ELT curriculum.",
            "journal ref": "International Journal of Advanced Computer Science and\n  Applications, 11(8), 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01663v3",
            "date": "2020-08-02 17:22:33+00:00",
            "title": "Efficient Urdu Caption Generation using Attention based LSTM",
            "authors": [
                "Inaam Ilahi",
                "Hafiz Muhammad Abdullah Zia",
                "Muhammad Ahtazaz Ahsan",
                "Rauf Tabassam",
                "Armaghan Ahmed",
                "Mohsen Ali"
            ],
            "abstract": "Recent advancements in deep learning have created many opportunities to solve\nreal-world problems that remained unsolved for more than a decade. Automatic\ncaption generation is a major research field, and the research community has\ndone a lot of work on it in most common languages like English. Urdu is the\nnational language of Pakistan and also much spoken and understood in the\nsub-continent region of Pakistan-India, and yet no work has been done for Urdu\nlanguage caption generation. Our research aims to fill this gap by developing\nan attention-based deep learning model using techniques of sequence modeling\nspecialized for the Urdu language. We have prepared a dataset in the Urdu\nlanguage by translating a subset of the \"Flickr8k\" dataset containing 700 'man'\nimages. We evaluate our proposed technique on this dataset and show that it can\nachieve a BLEU score of 0.83 in the Urdu language. We improve on the previous\nstate-of-the-art by using better CNN architectures and optimization techniques.\nFurthermore, we provide a discussion on how the generated captions can be made\ncorrect grammar-wise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01937v2",
            "date": "2020-08-05 05:14:11+00:00",
            "title": "Antibody Watch: Text Mining Antibody Specificity from the Literature",
            "authors": [
                "Chun-Nan Hsu",
                "Chia-Hui Chang",
                "Thamolwan Poopradubsil",
                "Amanda Lo",
                "Karen A. William",
                "Ko-Wei Lin",
                "Anita Bandrowski",
                "Ibrahim Burak Ozyurt",
                "Jeffrey S. Grethe",
                "Maryann E. Martone"
            ],
            "abstract": "Antibodies are widely used reagents to test for expression of proteins and\nother antigens. However, they might not always reliably produce results when\nthey do not specifically bind to the target proteins that their providers\ndesigned them for, leading to unreliable research results. While many proposals\nhave been developed to deal with the problem of antibody specificity, it is\nstill challenging to cover the millions of antibodies that are available to\nresearchers. In this study, we investigate the feasibility of automatically\ngenerating alerts to users of problematic antibodies by extracting statements\nabout antibody specificity reported in the literature. The extracted alerts can\nbe used to construct an \"Antibody Watch\" knowledge base containing supporting\nstatements of problematic antibodies. We developed a deep neural network system\nand tested its performance with a corpus of more than two thousand articles\nthat reported uses of antibodies. We divided the problem into two tasks. Given\nan input article, the first task is to identify snippets about antibody\nspecificity and classify if the snippets report that any antibody exhibits\nnon-specificity, and thus is problematic. The second task is to link each of\nthese snippets to one or more antibodies mentioned in the snippet. The\nexperimental evaluation shows that our system can accurately perform both\nclassification and linking tasks with weighted F-scores over 0.925 and 0.923,\nrespectively, and 0.914 overall when combined to complete the joint task. We\nleveraged Research Resource Identifiers (RRID) to precisely identify antibodies\nlinked to the extracted specificity snippets. The result shows that it is\nfeasible to construct a reliable knowledge base about problematic antibodies by\ntext mining.",
            "journal ref": "PLOS Computational Biology, 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01972v2",
            "date": "2020-08-05 07:42:09+00:00",
            "title": "Ontology-driven weak supervision for clinical entity classification in electronic health records",
            "authors": [
                "Jason A. Fries",
                "Ethan Steinberg",
                "Saelig Khattar",
                "Scott L. Fleming",
                "Jose Posada",
                "Alison Callahan",
                "Nigam H. Shah"
            ],
            "abstract": "In the electronic health record, using clinical notes to identify entities\nsuch as disorders and their temporality (e.g. the order of an event relative to\na time index) can inform many important analyses. However, creating training\ndata for clinical entity tasks is time consuming and sharing labeled data is\nchallenging due to privacy concerns. The information needs of the COVID-19\npandemic highlight the need for agile methods of training machine learning\nmodels for clinical notes. We present Trove, a framework for weakly supervised\nentity classification using medical ontologies and expert-generated rules. Our\napproach, unlike hand-labeled notes, is easy to share and modify, while\noffering performance comparable to learning from manually labeled training\ndata. In this work, we validate our framework on six benchmark tasks and\ndemonstrate Trove's ability to analyze the records of patients visiting the\nemergency department at Stanford Health Care for COVID-19 presenting symptoms\nand risk factors.",
            "journal ref": "Nature Communications 12.1 (2021): 1-11",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.01994v1",
            "date": "2020-08-05 08:32:15+00:00",
            "title": "Improving End-to-End Speech-to-Intent Classification with Reptile",
            "authors": [
                "Yusheng Tian",
                "Philip John Gorinski"
            ],
            "abstract": "End-to-end spoken language understanding (SLU) systems have many advantages\nover conventional pipeline systems, but collecting in-domain speech data to\ntrain an end-to-end system is costly and time consuming. One question arises\nfrom this: how to train an end-to-end SLU with limited amounts of data? Many\nresearchers have explored approaches that make use of other related data\nresources, typically by pre-training parts of the model on high-resource speech\nrecognition. In this paper, we suggest improving the generalization performance\nof SLU models with a non-standard learning algorithm, Reptile. Though Reptile\nwas originally proposed for model-agnostic meta learning, we argue that it can\nalso be used to directly learn a target task and result in better\ngeneralization than conventional gradient descent. In this work, we employ\nReptile to the task of end-to-end spoken intent classification. Experiments on\nfour datasets of different languages and domains show improvement of intent\nprediction accuracy, both when Reptile is used alone and used in addition to\npre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02239v4",
            "date": "2020-08-05 17:09:58+00:00",
            "title": "Glushkov's construction for functional subsequential transducers",
            "authors": [
                "Aleksander Mendoza-Drosik"
            ],
            "abstract": "Glushkov's construction has many interesting properties and they become even\nmore evident when applied to transducers. This article strives to show the wast\nrange of possible extensions and optimisations for this algorithm. Special\nflavour of regular expressions is introduced, which can be efficiently\nconverted to $\\epsilon$-free functional subsequential weighted finite state\ntransducers. Produced automata are very compact, as they contain only one state\nfor each symbol (from input alphabet) of original expression and only one\ntransition for each range of symbols, no matter how large. Such compactified\nranges of transitions allow for efficient binary search lookup during automaton\nevaluation. All the methods and algorithms presented here were used to\nimplement open-source compiler of regular expressions for multitape\ntransducers.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02460v1",
            "date": "2020-08-06 05:12:11+00:00",
            "title": "DeText: A Deep Text Ranking Framework with BERT",
            "authors": [
                "Weiwei Guo",
                "Xiaowei Liu",
                "Sida Wang",
                "Huiji Gao",
                "Ananth Sankar",
                "Zimeng Yang",
                "Qi Guo",
                "Liang Zhang",
                "Bo Long",
                "Bee-Chung Chen",
                "Deepak Agarwal"
            ],
            "abstract": "Ranking is the most important component in a search system. Mostsearch\nsystems deal with large amounts of natural language data,hence an effective\nranking system requires a deep understandingof text semantics. Recently, deep\nlearning based natural languageprocessing (deep NLP) models have generated\npromising results onranking systems. BERT is one of the most successful models\nthatlearn contextual embedding, which has been applied to capturecomplex\nquery-document relations for search ranking. However,this is generally done by\nexhaustively interacting each query wordwith each document word, which is\ninefficient for online servingin search product systems. In this paper, we\ninvestigate how tobuild an efficient BERT-based ranking model for industry use\ncases.The solution is further extended to a general ranking framework,DeText,\nthat is open sourced and can be applied to various rankingproductions. Offline\nand online experiments of DeText on threereal-world search systems present\nsignificant improvement overstate-of-the-art approaches.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.02637v1",
            "date": "2020-08-06 13:17:43+00:00",
            "title": "Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets",
            "authors": [
                "Patrick Lewis",
                "Pontus Stenetorp",
                "Sebastian Riedel"
            ],
            "abstract": "Ideally Open-Domain Question Answering models should exhibit a number of\ncompetencies, ranging from simply memorizing questions seen at training time,\nto answering novel question formulations with answers seen during training, to\ngeneralizing to completely novel questions with novel answers. However, single\naggregated test set scores do not show the full picture of what capabilities\nmodels truly have. In this work, we perform a detailed study of the test sets\nof three popular open-domain benchmark datasets with respect to these\ncompetencies. We find that 60-70% of test-time answers are also present\nsomewhere in the training sets. We also find that 30% of test-set questions\nhave a near-duplicate paraphrase in their corresponding training sets. Using\nthese findings, we evaluate a variety of popular open-domain models to obtain\ngreater insight into what extent they can actually generalize, and what drives\ntheir overall performance. We find that all models perform dramatically worse\non questions that cannot be memorized from training sets, with a mean absolute\nperformance difference of 63% between repeated and non-repeated data. Finally\nwe show that simple nearest-neighbor models out-perform a BART closed-book QA\nmodel, further highlighting the role that training set memorization plays in\nthese benchmarks",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02687v1",
            "date": "2020-07-24 14:50:10+00:00",
            "title": "Personalised Visual Art Recommendation by Learning Latent Semantic Representations",
            "authors": [
                "Bereket Abera Yilma",
                "Najib Aghenda",
                "Marcelo Romero",
                "Yannick Naudet",
                "Herve Panetto"
            ],
            "abstract": "In Recommender systems, data representation techniques play a great role as\nthey have the power to entangle, hide and reveal explanatory factors embedded\nwithin datasets. Hence, they influence the quality of recommendations.\nSpecifically, in Visual Art (VA) recommendations the complexity of the concepts\nembodied within paintings, makes the task of capturing semantics by machines\nfar from trivial. In VA recommendation, prominent works commonly use manually\ncurated metadata to drive recommendations. Recent works in this domain aim at\nleveraging visual features extracted using Deep Neural Networks (DNN). However,\nsuch data representation approaches are resource demanding and do not have a\ndirect interpretation, hindering user acceptance. To address these limitations,\nwe introduce an approach for Personalised Recommendation of Visual arts based\non learning latent semantic representation of paintings. Specifically, we\ntrained a Latent Dirichlet Allocation (LDA) model on textual descriptions of\npaintings. Our LDA model manages to successfully uncover non-obvious semantic\nrelationships between paintings whilst being able to offer explainable\nrecommendations. Experimental evaluations demonstrate that our method tends to\nperform better than exploiting visual features extracted using pre-trained Deep\nNeural Networks.",
            "journal ref": "SMAP 2020 15th International Workshop on Semantic and Social Media\n  Adaptation & Personalization",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.02878v1",
            "date": "2020-08-06 21:26:43+00:00",
            "title": "A Multilingual Neural Machine Translation Model for Biomedical Data",
            "authors": [
                "Alexandre B\u00e9rard",
                "Zae Myung Kim",
                "Vassilina Nikoulina",
                "Eunjeong L. Park",
                "Matthias Gall\u00e9"
            ],
            "abstract": "We release a multilingual neural machine translation model, which can be used\nto translate text in the biomedical domain. The model can translate from 5\nlanguages (French, German, Italian, Korean and Spanish) into English. It is\ntrained with large amounts of generic and biomedical data, using domain tags.\nOur benchmarks show that it performs near state-of-the-art both on news\n(generic domain) and biomedical test sets, and that it outperforms the existing\npublicly released models. We believe that this release will help the\nlarge-scale multilingual analysis of the digital content of the COVID-19 crisis\nand of its effects on society, economy, and healthcare policies.\n  We also release a test set of biomedical text for Korean-English. It consists\nof 758 sentences from official guidelines and recent papers, all about\nCOVID-19.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.02976v2",
            "date": "2020-08-07 03:30:14+00:00",
            "title": "Data Weighted Training Strategies for Grammatical Error Correction",
            "authors": [
                "Jared Lichtarge",
                "Chris Alberti",
                "Shankar Kumar"
            ],
            "abstract": "Recent progress in the task of Grammatical Error Correction (GEC) has been\ndriven by addressing data sparsity, both through new methods for generating\nlarge and noisy pretraining data and through the publication of small and\nhigher-quality finetuning data in the BEA-2019 shared task. Building upon\nrecent work in Neural Machine Translation (NMT), we make use of both kinds of\ndata by deriving example-level scores on our large pretraining data based on a\nsmaller, higher-quality dataset. In this work, we perform an empirical study to\ndiscover how to best incorporate delta-log-perplexity, a type of example\nscoring, into a training schedule for GEC. In doing so, we perform experiments\nthat shed light on the function and applicability of delta-log-perplexity.\nModels trained on scored data achieve state-of-the-art results on common GEC\ntest sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03082v2",
            "date": "2020-08-07 10:48:40+00:00",
            "title": "Perception Score, A Learned Metric for Open-ended Text Generation Evaluation",
            "authors": [
                "Jing Gu",
                "Qingyang Wu",
                "Zhou Yu"
            ],
            "abstract": "Automatic evaluation for open-ended natural language generation tasks remains\na challenge. Existing metrics such as BLEU show a low correlation with human\njudgment. We propose a novel and powerful learning-based evaluation metric:\nPerception Score. The method measures the overall quality of the generation and\nscores holistically instead of only focusing on one evaluation criteria, such\nas word overlapping. Moreover, it also shows the amount of uncertainty about\nits evaluation result. By connecting the uncertainty, Perception Score gives a\nmore accurate evaluation for the generation system. Perception Score provides\nstate-of-the-art results on two conditional generation tasks and two\nunconditional generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03274v1",
            "date": "2020-08-07 17:24:53+00:00",
            "title": "SemEval-2020 Task 10: Emphasis Selection for Written Text in Visual Media",
            "authors": [
                "Amirreza Shirani",
                "Franck Dernoncourt",
                "Nedim Lipka",
                "Paul Asente",
                "Jose Echevarria",
                "Thamar Solorio"
            ],
            "abstract": "In this paper, we present the main findings and compare the results of\nSemEval-2020 Task 10, Emphasis Selection for Written Text in Visual Media. The\ngoal of this shared task is to design automatic methods for emphasis selection,\ni.e. choosing candidates for emphasis in textual content to enable automated\ndesign assistance in authoring. The main focus is on short text instances for\nsocial media, with a variety of examples, from social media posts to\ninspirational quotes. Participants were asked to model emphasis using plain\ntext with no additional context from the user or other design considerations.\nSemEval-2020 Emphasis Selection shared task attracted 197 participants in the\nearly phase and a total of 31 teams made submissions to this task. The\nhighest-ranked submission achieved 0.823 Matchm score. The analysis of systems\nsubmitted to the task indicates that BERT and RoBERTa were the most common\nchoice of pre-trained models used, and part of speech tag (POS) was the most\nuseful feature. Full results can be found on the task's website.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03391v1",
            "date": "2020-08-07 22:25:36+00:00",
            "title": "Diversifying Task-oriented Dialogue Response Generation with Prototype Guided Paraphrasing",
            "authors": [
                "Phillip Lippe",
                "Pengjie Ren",
                "Hinda Haned",
                "Bart Voorn",
                "Maarten de Rijke"
            ],
            "abstract": "Existing methods for Dialogue Response Generation (DRG) in Task-oriented\nDialogue Systems (TDSs) can be grouped into two categories: template-based and\ncorpus-based. The former prepare a collection of response templates in advance\nand fill the slots with system actions to produce system responses at runtime.\nThe latter generate system responses token by token by taking system actions\ninto account. While template-based DRG provides high precision and highly\npredictable responses, they usually lack in terms of generating diverse and\nnatural responses when compared to (neural) corpus-based approaches.\nConversely, while corpus-based DRG methods are able to generate natural\nresponses, we cannot guarantee their precision or predictability. Moreover, the\ndiversity of responses produced by today's corpus-based DRG methods is still\nlimited. We propose to combine the merits of template-based and corpus-based\nDRGs by introducing a prototype-based, paraphrasing neural network, called\nP2-Net, which aims to enhance quality of the responses in terms of both\nprecision and diversity. Instead of generating a response from scratch, P2-Net\ngenerates system responses by paraphrasing template-based responses. To\nguarantee the precision of responses, P2-Net learns to separate a response into\nits semantics, context influence, and paraphrasing noise, and to keep the\nsemantics unchanged during paraphrasing. To introduce diversity, P2-Net\nrandomly samples previous conversational utterances as prototypes, from which\nthe model can then extract speaking style information. We conduct extensive\nexperiments on the MultiWOZ dataset with both automatic and human evaluations.\nThe results show that P2-Net achieves a significant improvement in diversity\nwhile preserving the semantics of responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03709v4",
            "date": "2020-08-09 11:02:06+00:00",
            "title": "Adversarial Training with Fast Gradient Projection Method against Synonym Substitution based Text Attacks",
            "authors": [
                "Xiaosen Wang",
                "Yichen Yang",
                "Yihe Deng",
                "Kun He"
            ],
            "abstract": "Adversarial training is the most empirically successful approach in improving\nthe robustness of deep neural networks for image classification.For text\nclassification, however, existing synonym substitution based adversarial\nattacks are effective but not efficient to be incorporated into practical text\nadversarial training. Gradient-based attacks, which are very efficient for\nimages, are hard to be implemented for synonym substitution based text attacks\ndue to the lexical, grammatical and semantic constraints and the discrete text\ninput space. Thereby, we propose a fast text adversarial attack method called\nFast Gradient Projection Method (FGPM) based on synonym substitution, which is\nabout 20 times faster than existing text attack methods and could achieve\nsimilar attack performance. We then incorporate FGPM with adversarial training\nand propose a text defense method called Adversarial Training with FGPM\nenhanced by Logit pairing (ATFL). Experiments show that ATFL could\nsignificantly improve the model robustness and block the transferability of\nadversarial examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03822v1",
            "date": "2020-08-09 21:48:02+00:00",
            "title": "Distilling the Knowledge of BERT for Sequence-to-Sequence ASR",
            "authors": [
                "Hayato Futami",
                "Hirofumi Inaguma",
                "Sei Ueno",
                "Masato Mimura",
                "Shinsuke Sakai",
                "Tatsuya Kawahara"
            ],
            "abstract": "Attention-based sequence-to-sequence (seq2seq) models have achieved promising\nresults in automatic speech recognition (ASR). However, as these models decode\nin a left-to-right way, they do not have access to context on the right. We\nleverage both left and right context by applying BERT as an external language\nmodel to seq2seq ASR through knowledge distillation. In our proposed method,\nBERT generates soft labels to guide the training of seq2seq ASR. Furthermore,\nwe leverage context beyond the current utterance as input to BERT. Experimental\nevaluations show that our method significantly improves the ASR performance\nfrom the seq2seq baseline on the Corpus of Spontaneous Japanese (CSJ).\nKnowledge distillation from BERT outperforms that from a transformer LM that\nonly looks at left context. We also show the effectiveness of leveraging\ncontext beyond the current utterance. Our method outperforms other LM\napplication approaches such as n-best rescoring and shallow fusion, while it\ndoes not require extra inference cost.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03843v1",
            "date": "2020-08-10 00:18:32+00:00",
            "title": "Question Identification in Arabic Language Using Emotional Based Features",
            "authors": [
                "Ahmed Ramzy",
                "Ahmed Elazab"
            ],
            "abstract": "With the growth of content on social media networks, enterprises and services\nproviders have become interested in identifying the questions of their\ncustomers. Tracking these questions become very challenging with the growth of\ntext that grows directly proportional to the increase of Arabic users thus\nmaking it very difficult to be tracked manually. By automatic identifying the\nquestions seeking answers on the social media networks and defining their\ncategory, we can automatically answer them by finding an existing answer or\neven routing them to those responsible for answering those questions in the\ncustomer service. This will result in saving the time and the effort and\nenhancing the customer feedback and improving the business. In this paper, we\nhave implemented a binary classifier to classify Arabic text to either question\nseeking answer or not. We have added emotional based features to the state of\nthe art features. Experimental evaluation has done and showed that these\nemotional features have improved the accuracy of the classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.03923v1",
            "date": "2020-08-10 07:00:08+00:00",
            "title": "Knowledge Distillation and Data Selection for Semi-Supervised Learning in CTC Acoustic Models",
            "authors": [
                "Prakhar Swarup",
                "Debmalya Chakrabarty",
                "Ashtosh Sapru",
                "Hitesh Tulsiani",
                "Harish Arsikere",
                "Sri Garimella"
            ],
            "abstract": "Semi-supervised learning (SSL) is an active area of research which aims to\nutilize unlabelled data in order to improve the accuracy of speech recognition\nsystems. The current study proposes a methodology for integration of two key\nideas: 1) SSL using connectionist temporal classification (CTC) objective and\nteacher-student based learning 2) Designing effective data-selection mechanisms\nfor leveraging unlabelled data to boost performance of student models. Our aim\nis to establish the importance of good criteria in selecting samples from a\nlarge pool of unlabelled data based on attributes like confidence measure,\nspeaker and content variability. The question we try to answer is: Is it\npossible to design a data selection mechanism which reduces dependence on a\nlarge set of randomly selected unlabelled samples without compromising on Word\nError Rate (WER)? We perform empirical investigations of different data\nselection methods to answer this question and quantify the effect of different\nsampling strategies. On a semi-supervised ASR setting with 40000 hours of\ncarefully selected unlabelled data, our CTC-SSL approach gives 17% relative WER\nimprovement over a baseline CTC system trained with labelled data. It also\nachieves on-par performance with CTC-SSL system trained on order of magnitude\nlarger unlabeled data based on random sampling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04200v1",
            "date": "2020-08-10 15:40:05+00:00",
            "title": "Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach",
            "authors": [
                "Yahui Liu",
                "Marco De Nadai",
                "Deng Cai",
                "Huayang Li",
                "Xavier Alameda-Pineda",
                "Nicu Sebe",
                "Bruno Lepri"
            ],
            "abstract": "Manipulating visual attributes of images through human-written text is a very\nchallenging task. On the one hand, models have to learn the manipulation\nwithout the ground truth of the desired output. On the other hand, models have\nto deal with the inherent ambiguity of natural language. Previous research\nusually requires either the user to describe all the characteristics of the\ndesired image or to use richly-annotated image captioning datasets. In this\nwork, we propose a novel unsupervised approach, based on image-to-image\ntranslation, that alters the attributes of a given image through a command-like\nsentence such as \"change the hair color to black\". Contrarily to\nstate-of-the-art approaches, our model does not require a human-annotated\ndataset nor a textual description of all the attributes of the desired image,\nbut only those that have to be modified. Our proposed model disentangles the\nimage content from the visual attributes, and it learns to modify the latter\nusing the textual description, before generating a new image from the content\nand the modified attribute representation. Because text might be inherently\nambiguous (blond hair may refer to different shadows of blond, e.g. golden,\nicy, sandy), our method generates multiple stochastic versions of the same\ntranslation. Experiments show that the proposed model achieves promising\nperformances on two large-scale public datasets: CelebA and CUB. We believe our\napproach will pave the way to new avenues of research combining textual and\nspeech commands with visual attributes.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.04203v1",
            "date": "2020-08-10 15:43:28+00:00",
            "title": "FireBERT: Hardening BERT-based classifiers against adversarial attack",
            "authors": [
                "Gunnar Mein",
                "Kevin Hartman",
                "Andrew Morris"
            ],
            "abstract": "We present FireBERT, a set of three proof-of-concept NLP classifiers hardened\nagainst TextFooler-style word-perturbation by producing diverse alternatives to\noriginal samples. In one approach, we co-tune BERT against the training data\nand synthetic adversarial samples. In a second approach, we generate the\nsynthetic samples at evaluation time through substitution of words and\nperturbation of embedding vectors. The diversified evaluation results are then\ncombined by voting. A third approach replaces evaluation-time word substitution\nwith perturbation of embedding vectors. We evaluate FireBERT for MNLI and IMDB\nMovie Review datasets, in the original and on adversarial examples generated by\nTextFooler. We also test whether TextFooler is less successful in creating new\nadversarial samples when manipulating FireBERT, compared to working on\nunhardened classifiers. We show that it is possible to improve the accuracy of\nBERT-based models in the face of adversarial attacks without significantly\nreducing the accuracy for regular benchmark samples. We present co-tuning with\na synthetic data generator as a highly effective method to protect against 95%\nof pre-manufactured adversarial samples while maintaining 98% of original\nbenchmark performance. We also demonstrate evaluation-time perturbation as a\npromising direction for further research, restoring accuracy up to 75% of\nbenchmark performance for pre-made adversarials, and up to 65% (from a baseline\nof 75% orig. / 12% attack) under active attack by TextFooler.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04545v1",
            "date": "2020-08-11 06:41:53+00:00",
            "title": "Context Reinforced Neural Topic Modeling over Short Texts",
            "authors": [
                "Jiachun Feng",
                "Zusheng Zhang",
                "Cheng Ding",
                "Yanghui Rao",
                "Haoran Xie"
            ],
            "abstract": "As one of the prevalent topic mining tools, neural topic modeling has\nattracted a lot of interests for the advantages of high efficiency in training\nand strong generalisation abilities. However, due to the lack of context in\neach short text, the existing neural topic models may suffer from feature\nsparsity on such documents. To alleviate this issue, we propose a Context\nReinforced Neural Topic Model (CRNTM), whose characteristics can be summarized\nas follows. Firstly, by assuming that each short text covers only a few salient\ntopics, CRNTM infers the topic for each word in a narrow range. Secondly, our\nmodel exploits pre-trained word embeddings by treating topics as multivariate\nGaussian distributions or Gaussian mixture distributions in the embedding\nspace. Extensive experiments on two benchmark datasets validate the\neffectiveness of the proposed model on both topic discovery and text\nclassification.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.04636v1",
            "date": "2020-08-11 11:41:53+00:00",
            "title": "A Comparison of Synthetic Oversampling Methods for Multi-class Text Classification",
            "authors": [
                "Anna Glazkova"
            ],
            "abstract": "The authors compared oversampling methods for the problem of multi-class\ntopic classification. The SMOTE algorithm underlies one of the most popular\noversampling methods. It consists in choosing two examples of a minority class\nand generating a new example based on them. In the paper, the authors compared\nthe basic SMOTE method with its two modifications (Borderline SMOTE and ADASYN)\nand random oversampling technique on the example of one of text classification\ntasks. The paper discusses the k-nearest neighbor algorithm, the support vector\nmachine algorithm and three types of neural networks (feedforward network, long\nshort-term memory (LSTM) and bidirectional LSTM). The authors combine these\nmachine learning algorithms with different text representations and compared\nsynthetic oversampling methods. In most cases, the use of oversampling\ntechniques can significantly improve the quality of classification. The authors\nconclude that for this task, the quality of the KNN and SVM algorithms is more\ninfluenced by class imbalance than neural networks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.04637v2",
            "date": "2020-08-11 11:42:03+00:00",
            "title": "Real-Time Sign Language Detection using Human Pose Estimation",
            "authors": [
                "Amit Moryossef",
                "Ioannis Tsochantaridis",
                "Roee Aharoni",
                "Sarah Ebling",
                "Srini Narayanan"
            ],
            "abstract": "We propose a lightweight real-time sign language detection model, as we\nidentify the need for such a case in videoconferencing. We extract optical flow\nfeatures based on human pose estimation and, using a linear classifier, show\nthese features are meaningful with an accuracy of 80%, evaluated on the DGS\nCorpus. Using a recurrent model directly on the input, we see improvements of\nup to 91% accuracy, while still working under 4ms. We describe a demo\napplication to sign language detection in the browser in order to demonstrate\nits usage possibility in videoconferencing applications.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.05049v1",
            "date": "2020-08-12 00:58:39+00:00",
            "title": "Distantly Supervised Relation Extraction in Federated Settings",
            "authors": [
                "Dianbo Sui",
                "Yubo Chen",
                "Kang Liu",
                "Jun Zhao"
            ],
            "abstract": "This paper investigates distantly supervised relation extraction in federated\nsettings. Previous studies focus on distant supervision under the assumption of\ncentralized training, which requires collecting texts from different platforms\nand storing them on one machine. However, centralized training is challenged by\ntwo issues, namely, data barriers and privacy protection, which make it almost\nimpossible or cost-prohibitive to centralize data from multiple platforms.\nTherefore, it is worthy to investigate distant supervision in the federated\nlearning paradigm, which decouples the model training from the need for direct\naccess to the raw data. Overcoming label noise of distant supervision, however,\nbecomes more difficult in federated settings, since the sentences containing\nthe same entity pair may scatter around different platforms. In this paper, we\npropose a federated denoising framework to suppress label noise in federated\nsettings. The core of this framework is a multiple instance learning based\ndenoising method that is able to select reliable instances via cross-platform\ncollaboration. Various experimental results on New York Times dataset and miRNA\ngene regulation relation dataset demonstrate the effectiveness of the proposed\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05190v3",
            "date": "2020-08-12 09:12:22+00:00",
            "title": "Evaluating the Impact of Knowledge Graph Context on Entity Disambiguation Models",
            "authors": [
                "Isaiah Onando Mulang'",
                "Kuldeep Singh",
                "Chaitali Prabhu",
                "Abhishek Nadgeri",
                "Johannes Hoffart",
                "Jens Lehmann"
            ],
            "abstract": "Pretrained Transformer models have emerged as state-of-the-art approaches\nthat learn contextual information from text to improve the performance of\nseveral NLP tasks. These models, albeit powerful, still require specialized\nknowledge in specific scenarios. In this paper, we argue that context derived\nfrom a knowledge graph (in our case: Wikidata) provides enough signals to\ninform pretrained transformer models and improve their performance for named\nentity disambiguation (NED) on Wikidata KG. We further hypothesize that our\nproposed KG context can be standardized for Wikipedia, and we evaluate the\nimpact of KG context on state-of-the-art NED model for the Wikipedia knowledge\nbase. Our empirical results validate that the proposed KG context can be\ngeneralized (for Wikipedia), and providing KG context in transformer\narchitectures considerably outperforms the existing baselines, including the\nvanilla transformer models.",
            "journal ref": "CIKM 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05282v1",
            "date": "2020-08-12 13:02:48+00:00",
            "title": "Text Classification based on Multi-granularity Attention Hybrid Neural Network",
            "authors": [
                "Zhenyu Liu",
                "Chaohong Lu",
                "Haiwei Huang",
                "Shengfei Lyu",
                "Zhenchao Tao"
            ],
            "abstract": "Neural network-based approaches have become the driven forces for Natural\nLanguage Processing (NLP) tasks. Conventionally, there are two mainstream\nneural architectures for NLP tasks: the recurrent neural network (RNN) and the\nconvolution neural network (ConvNet). RNNs are good at modeling long-term\ndependencies over input texts, but preclude parallel computation. ConvNets do\nnot have memory capability and it has to model sequential data as un-ordered\nfeatures. Therefore, ConvNets fail to learn sequential dependencies over the\ninput texts, but it is able to carry out high-efficient parallel computation.\nAs each neural architecture, such as RNN and ConvNets, has its own pro and con,\nintegration of different architectures is assumed to be able to enrich the\nsemantic representation of texts, thus enhance the performance of NLP tasks.\nHowever, few investigation explores the reconciliation of these seemingly\nincompatible architectures. To address this issue, we propose a hybrid\narchitecture based on a novel hierarchical multi-granularity attention\nmechanism, named Multi-granularity Attention-based Hybrid Neural Network\n(MahNN). The attention mechanism is to assign different weights to different\nparts of the input sequence to increase the computation efficiency and\nperformance of neural models. In MahNN, two types of attentions are introduced:\nthe syntactical attention and the semantical attention. The syntactical\nattention computes the importance of the syntactic elements (such as words or\nsentence) at the lower symbolic level and the semantical attention is used to\ncompute the importance of the embedded space dimension corresponding to the\nupper latent semantics. We adopt the text classification as an exemplifying way\nto illustrate the ability of MahNN to understand texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05333v2",
            "date": "2020-08-12 14:12:32+00:00",
            "title": "Variance-reduced Language Pretraining via a Mask Proposal Network",
            "authors": [
                "Liang Chen"
            ],
            "abstract": "Self-supervised learning, a.k.a., pretraining, is important in natural\nlanguage processing. Most of the pretraining methods first randomly mask some\npositions in a sentence and then train a model to recover the tokens at the\nmasked positions. In such a way, the model can be trained without human\nlabeling, and the massive data can be used with billion parameters. Therefore,\nthe optimization efficiency becomes critical. In this paper, we tackle the\nproblem from the view of gradient variance reduction. In particular, we first\npropose a principled gradient variance decomposition theorem, which shows that\nthe variance of the stochastic gradient of the language pretraining can be\nnaturally decomposed into two terms: the variance that arises from the sample\nof data in a batch, and the variance that arises from the sampling of the mask.\nThe second term is the key difference between selfsupervised learning and\nsupervised learning, which makes the pretraining slower. In order to reduce the\nvariance of the second part, we leverage the importance sampling strategy,\nwhich aims at sampling the masks according to a proposal distribution instead\nof the uniform distribution. It can be shown that if the proposal distribution\nis proportional to the gradient norm, the variance of the sampling is reduced.\nTo improve efficiency, we introduced a MAsk Proposal Network (MAPNet), which\napproximates the optimal mask proposal distribution and is trained end-to-end\nalong with the model. According to the experimental result, our model converges\nmuch faster and achieves higher performance than the baseline BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05363v1",
            "date": "2020-08-12 14:59:50+00:00",
            "title": "Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering",
            "authors": [
                "Sebastian Hofst\u00e4tter",
                "Markus Zlabinger",
                "Mete Sertkan",
                "Michael Schr\u00f6der",
                "Allan Hanbury"
            ],
            "abstract": "There are many existing retrieval and question answering datasets. However,\nmost of them either focus on ranked list evaluation or single-candidate\nquestion answering. This divide makes it challenging to properly evaluate\napproaches concerned with ranking documents and providing snippets or answers\nfor a given query. In this work, we present FiRA: a novel dataset of\nFine-Grained Relevance Annotations. We extend the ranked retrieval annotations\nof the Deep Learning track of TREC 2019 with passage and word level graded\nrelevance annotations for all relevant documents. We use our newly created data\nto study the distribution of relevance in long documents, as well as the\nattention of annotators to specific positions of the text. As an example, we\nevaluate the recently introduced TKL document ranking model. We find that\nalthough TKL exhibits state-of-the-art retrieval results for long documents, it\nmisses many relevant passages.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.05536v2",
            "date": "2020-08-12 19:17:46+00:00",
            "title": "Model Robustness with Text Classification: Semantic-preserving adversarial attacks",
            "authors": [
                "Rahul Singh",
                "Tarun Joshi",
                "Vijayan N. Nair",
                "Agus Sudjianto"
            ],
            "abstract": "We propose algorithms to create adversarial attacks to assess model\nrobustness in text classification problems. They can be used to create white\nbox attacks and black box attacks while at the same time preserving the\nsemantics and syntax of the original text. The attacks cause significant number\nof flips in white-box setting and same rule based can be used in black-box\nsetting. In a black-box setting, the attacks created are able to reverse\ndecisions of transformer based architectures.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05701v1",
            "date": "2020-08-13 05:53:24+00:00",
            "title": "The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?",
            "authors": [
                "Kevin Roitero",
                "Michael Soprano",
                "Beatrice Portelli",
                "Damiano Spina",
                "Vincenzo Della Mea",
                "Giuseppe Serra",
                "Stefano Mizzaro",
                "Gianluca Demartini"
            ],
            "abstract": "Misinformation is an ever increasing problem that is difficult to solve for\nthe research community and has a negative impact on the society at large. Very\nrecently, the problem has been addressed with a crowdsourcing-based approach to\nscale up labeling efforts: to assess the truthfulness of a statement, instead\nof relying on a few experts, a crowd of (non-expert) judges is exploited. We\nfollow the same approach to study whether crowdsourcing is an effective and\nreliable method to assess statements truthfulness during a pandemic. We\nspecifically target statements related to the COVID-19 health emergency, that\nis still ongoing at the time of the study and has arguably caused an increase\nof the amount of misinformation that is spreading online (a phenomenon for\nwhich the term \"infodemic\" has been used). By doing so, we are able to address\n(mis)information that is both related to a sensitive and personal issue like\nhealth and very recent as compared to when the judgment is done: two issues\nthat have not been analyzed in related work. In our experiment, crowd workers\nare asked to assess the truthfulness of statements, as well as to provide\nevidence for the assessments as a URL and a text justification. Besides showing\nthat the crowd is able to accurately judge the truthfulness of the statements,\nwe also report results on many different aspects, including: agreement among\nworkers, the effect of different aggregation functions, of scales\ntransformations, and of workers background / bias. We also analyze workers\nbehavior, in terms of queries submitted, URLs found / selected, text\njustifications, and other behavioral data like clicks and mouse actions\ncollected by means of an ad hoc logger.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.05713v1",
            "date": "2020-08-13 06:29:24+00:00",
            "title": "Exploration of Gender Differences in COVID-19 Discourse on Reddit",
            "authors": [
                "Jai Aggarwal",
                "Ella Rabinovich",
                "Suzanne Stevenson"
            ],
            "abstract": "Decades of research on differences in the language of men and women have\nestablished postulates about preferences in lexical, topical, and emotional\nexpression between the two genders, along with their sociological\nunderpinnings. Using a novel dataset of male and female linguistic productions\ncollected from the Reddit discussion platform, we further confirm existing\nassumptions about gender-linked affective distinctions, and demonstrate that\nthese distinctions are amplified in social media postings involving\nemotionally-charged discourse related to COVID-19. Our analysis also confirms\nconsiderable differences in topical preferences between male and female authors\nin spontaneous pandemic-related discussions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05759v1",
            "date": "2020-08-13 08:56:40+00:00",
            "title": "MICE: Mining Idioms with Contextual Embeddings",
            "authors": [
                "Tadej \u0160kvorc",
                "Polona Gantar",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "Idiomatic expressions can be problematic for natural language processing\napplications as their meaning cannot be inferred from their constituting words.\nA lack of successful methodological approaches and sufficiently large datasets\nprevents the development of machine learning approaches for detecting idioms,\nespecially for expressions that do not occur in the training set. We present an\napproach, called MICE, that uses contextual embeddings for that purpose. We\npresent a new dataset of multi-word expressions with literal and idiomatic\nmeanings and use it to train a classifier based on two state-of-the-art\ncontextual word embeddings: ELMo and BERT. We show that deep neural networks\nusing both embeddings perform much better than existing approaches, and are\ncapable of detecting idiomatic word use, even for expressions that were not\npresent in the training set. We demonstrate cross-lingual transfer of developed\nmodels and analyze the size of the required dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05760v1",
            "date": "2020-08-13 08:57:16+00:00",
            "title": "MASRI-HEADSET: A Maltese Corpus for Speech Recognition",
            "authors": [
                "Carlos Mena",
                "Albert Gatt",
                "Andrea DeMarco",
                "Claudia Borg",
                "Lonneke van der Plas",
                "Amanda Muscat",
                "Ian Padovani"
            ],
            "abstract": "Maltese, the national language of Malta, is spoken by approximately 500,000\npeople. Speech processing for Maltese is still in its early stages of\ndevelopment. In this paper, we present the first spoken Maltese corpus designed\npurposely for Automatic Speech Recognition (ASR). The MASRI-HEADSET corpus was\ndeveloped by the MASRI project at the University of Malta. It consists of 8\nhours of speech paired with text, recorded by using short text snippets in a\nlaboratory environment. The speakers were recruited from different geographical\nlocations all over the Maltese islands, and were roughly evenly distributed by\ngender. This paper also presents some initial results achieved in baseline\nexperiments for Maltese ASR using Sphinx and Kaldi. The MASRI-HEADSET Corpus is\npublicly available for research/academic purposes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05773v2",
            "date": "2020-08-13 09:36:05+00:00",
            "title": "Continuous Speech Separation with Conformer",
            "authors": [
                "Sanyuan Chen",
                "Yu Wu",
                "Zhuo Chen",
                "Jian Wu",
                "Jinyu Li",
                "Takuya Yoshioka",
                "Chengyi Wang",
                "Shujie Liu",
                "Ming Zhou"
            ],
            "abstract": "Continuous speech separation plays a vital role in complicated speech related\ntasks such as conversation transcription. The separation model extracts a\nsingle speaker signal from a mixed speech. In this paper, we use transformer\nand conformer in lieu of recurrent neural networks in the separation system, as\nwe believe capturing global information with the self-attention based method is\ncrucial for the speech separation. Evaluating on the LibriCSS dataset, the\nconformer separation model achieves state of the art results, with a relative\n23.5% word error rate (WER) reduction from bi-directional LSTM (BLSTM) in the\nutterance-wise evaluation and a 15.4% WER reduction in the continuous\nevaluation.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.05828v1",
            "date": "2020-08-13 11:32:47+00:00",
            "title": "On the Importance of Local Information in Transformer Based Models",
            "authors": [
                "Madhura Pande",
                "Aakriti Budhraja",
                "Preksha Nema",
                "Pratyush Kumar",
                "Mitesh M. Khapra"
            ],
            "abstract": "The self-attention module is a key component of Transformer-based models,\nwherein each token pays attention to every other token. Recent studies have\nshown that these heads exhibit syntactic, semantic, or local behaviour. Some\nstudies have also identified promise in restricting this attention to be local,\ni.e., a token attending to other tokens only in a small neighbourhood around\nit. However, no conclusive evidence exists that such local attention alone is\nsufficient to achieve high accuracy on multiple NLP tasks. In this work, we\nsystematically analyse the role of locality information in learnt models and\ncontrast it with the role of syntactic information. More specifically, we first\ndo a sensitivity analysis and show that, at every layer, the representation of\na token is much more sensitive to tokens in a small neighborhood around it than\nto tokens which are syntactically related to it. We then define an attention\nbias metric to determine whether a head pays more attention to local tokens or\nto syntactically related tokens. We show that a larger fraction of heads have a\nlocality bias as compared to a syntactic bias. Having established the\nimportance of local attention heads, we train and evaluate models where varying\nfractions of the attention heads are constrained to be local. Such models would\nbe more efficient as they would have fewer computations in the attention layer.\nWe evaluate these models on 4 GLUE datasets (QQP, SST-2, MRPC, QNLI) and 2 MT\ndatasets (En-De, En-Ru) and clearly demonstrate that such constrained models\nhave comparable performance to the unconstrained models. Through this\nsystematic evaluation we establish that attention in Transformer-based models\ncan be constrained to be local without affecting performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.05925v1",
            "date": "2020-08-13 14:13:30+00:00",
            "title": "Commonsense Knowledge Graph Reasoning by Selection or Generation? Why?",
            "authors": [
                "Cunxiang Wang",
                "Jinhang Wu",
                "Luxin Liu",
                "Yue Zhang"
            ],
            "abstract": "Commonsense knowledge graph reasoning(CKGR) is the task of predicting a\nmissing entity given one existing and the relation in a commonsense knowledge\ngraph (CKG). Existing methods can be classified into two categories generation\nmethod and selection method. Each method has its own advantage. We\ntheoretically and empirically compare the two methods, finding the selection\nmethod is more suitable than the generation method in CKGR. Given the\nobservation, we further combine the structure of neural Text Encoder and\nKnowledge Graph Embedding models to solve the selection method's two problems,\nachieving competitive results. We provide a basic framework and baseline model\nfor subsequent CKGR tasks by selection methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06173v1",
            "date": "2020-08-14 02:43:57+00:00",
            "title": "Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfaces",
            "authors": [
                "Milind Rao",
                "Anirudh Raju",
                "Pranav Dheram",
                "Bach Bui",
                "Ariya Rastrow"
            ],
            "abstract": "We consider the problem of spoken language understanding (SLU) of extracting\nnatural language intents and associated slot arguments or named entities from\nspeech that is primarily directed at voice assistants. Such a system subsumes\nboth automatic speech recognition (ASR) as well as natural language\nunderstanding (NLU). An end-to-end joint SLU model can be built to a required\nspecification opening up the opportunity to deploy on hardware constrained\nscenarios like devices enabling voice assistants to work offline, in a privacy\npreserving manner, whilst also reducing server costs.\n  We first present models that extract utterance intent directly from speech\nwithout intermediate text output. We then present a compositional model, which\ngenerates the transcript using the Listen Attend Spell ASR system and then\nextracts interpretation using a neural NLU model. Finally, we contrast these\nmethods to a jointly trained end-to-end joint SLU model, consisting of ASR and\nNLU subsystems which are connected by a neural network based interface instead\nof text, that produces transcripts as well as NLU interpretation. We show that\nthe jointly trained model shows improvements to ASR incorporating semantic\ninformation from NLU and also improves NLU by exposing it to ASR confusion\nencoded in the hidden layer.",
            "journal ref": "Proc. Interspeech 2020, 876-880 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06176v1",
            "date": "2020-08-14 03:23:09+00:00",
            "title": "A Hybrid BERT and LightGBM based Model for Predicting Emotion GIF Categories on Twitter",
            "authors": [
                "Ye Bi",
                "Shuo Wang",
                "Zhongrui Fan"
            ],
            "abstract": "The animated Graphical Interchange Format (GIF) images have been widely used\non social media as an intuitive way of expression emotion. Given their\nexpressiveness, GIFs offer a more nuanced and precise way to convey emotions.\nIn this paper, we present our solution for the EmotionGIF 2020 challenge, the\nshared task of SocialNLP 2020. To recommend GIF categories for unlabeled\ntweets, we regarded this problem as a kind of matching tasks and proposed a\nlearning to rank framework based on Bidirectional Encoder Representations from\nTransformer (BERT) and LightGBM. Our team won the 4th place with a Mean Average\nPrecision @ 6 (MAP@6) score of 0.5394 on the round 1 leaderboard.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.06222v1",
            "date": "2020-08-14 07:39:21+00:00",
            "title": "Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis",
            "authors": [
                "Stavros Assimakopoulos",
                "Rebecca Vella Muskat",
                "Lonneke van der Plas",
                "Albert Gatt"
            ],
            "abstract": "This paper presents a novel scheme for the annotation of hate speech in\ncorpora of Web 2.0 commentary. The proposed scheme is motivated by the critical\nanalysis of posts made in reaction to news reports on the Mediterranean\nmigration crisis and LGBTIQ+ matters in Malta, which was conducted under the\nauspices of the EU-funded C.O.N.T.A.C.T. project. Based on the realization that\nhate speech is not a clear-cut category to begin with, appears to belong to a\ncontinuum of discriminatory discourse and is often realized through the use of\nindirect linguistic means, it is argued that annotation schemes for its\ndetection should refrain from directly including the label 'hate speech,' as\ndifferent annotators might have different thresholds as to what constitutes\nhate speech and what not. In view of this, we suggest a multi-layer annotation\nscheme, which is pilot-tested against a binary +/- hate speech classification\nand appears to yield higher inter-annotator agreement. Motivating the\npostulation of our scheme, we then present the MaNeCo corpus on which it will\neventually be used; a substantial corpus of on-line newspaper comments spanning\n10 years.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2008.06239v2",
            "date": "2020-08-14 08:23:21+00:00",
            "title": "Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems",
            "authors": [
                "Andrea Madotto",
                "Zihan Liu",
                "Zhaojiang Lin",
                "Pascale Fung"
            ],
            "abstract": "Task-oriented dialogue systems use four connected modules, namely, Natural\nLanguage Understanding (NLU), a Dialogue State Tracking (DST), Dialogue Policy\n(DP) and Natural Language Generation (NLG). A research challenge is to learn\neach module with the least amount of samples (i.e., few-shots) given the high\ncost related to the data collection. The most common and effective technique to\nsolve this problem is transfer learning, where large language models, either\npre-trained on text or task-specific data, are fine-tuned on the few samples.\nThese methods require fine-tuning steps and a set of parameters for each task.\nDifferently, language models, such as GPT-2 (Radford et al., 2019) and GPT-3\n(Brown et al., 2020), allow few-shot learning by priming the model with few\nexamples. In this paper, we evaluate the priming few-shot ability of language\nmodels in the NLU, DST, DP and NLG tasks. Importantly, we highlight the current\nlimitations of this approach, and we discuss the possible implication for\nfuture work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06268v1",
            "date": "2020-08-14 09:48:58+00:00",
            "title": "An Efficient Model Inference Algorithm for Learning-based Testing of Reactive Systems",
            "authors": [
                "Muddassar A. Sindhu"
            ],
            "abstract": "Learning-based testing (LBT) is an emerging methodology to automate iterative\nblack-box requirements testing of software systems. The methodology involves\ncombining model inference with model checking techniques. However, a variety of\noptimisations on model inference are necessary in order to achieve scalable\ntesting for large systems. In this paper we describe the IKL learning algorithm\nwhich is an active incremental learning algorithm for deterministic Kripke\nstructures. We formally prove the correctness of IKL. We discuss the\noptimisations it incorporates to achieve scalability of testing. We also\nevaluate a black box heuristic for test termination based on convergence of IKL\nlearning.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06274v4",
            "date": "2020-08-14 10:01:34+00:00",
            "title": "Graph-based Modeling of Online Communities for Fake News Detection",
            "authors": [
                "Shantanu Chandra",
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Madhav Nimishakavi",
                "Marzieh Saeidi",
                "Ekaterina Shutova"
            ],
            "abstract": "Over the past few years, there has been a substantial effort towards\nautomated detection of fake news on social media platforms. Existing research\nhas modeled the structure, style, content, and patterns in dissemination of\nonline posts, as well as the demographic traits of users who interact with\nthem. However, no attention has been directed towards modeling the properties\nof online communities that interact with the posts. In this work, we propose a\nnovel social context-aware fake news detection framework, SAFER, based on graph\nneural networks (GNNs). The proposed framework aggregates information with\nrespect to: 1) the nature of the content disseminated, 2) content-sharing\nbehavior of users, and 3) the social network of those users. We furthermore\nperform a systematic comparison of several GNN models for this task and\nintroduce novel methods based on relational and hyperbolic GNNs, which have not\nbeen previously used for user or community modeling within NLP. We empirically\ndemonstrate that our framework yields significant improvements over existing\ntext-based techniques and achieves state-of-the-art results on fake news\ndatasets from two different domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06452v1",
            "date": "2020-08-14 16:30:07+00:00",
            "title": "Predicting Event Time by Classifying Sub-Level Temporal Relations Induced from a Unified Representation of Time Anchors",
            "authors": [
                "Fei Cheng",
                "Yusuke Miyao"
            ],
            "abstract": "Extracting event time from news articles is a challenging but attractive\ntask. In contrast to the most existing pair-wised temporal link annotation,\nReimers et al.(2016) proposed to annotate the time anchor (a.k.a. the exact\ntime) of each event. Their work represents time anchors with discrete\nrepresentations of Single-Day/Multi-Day and Certain/Uncertain. This increases\nthe complexity of modeling the temporal relations between two time anchors,\nwhich cannot be categorized into the relations of Allen's interval algebra\n(Allen, 1990).\n  In this paper, we propose an effective method to decompose such complex\ntemporal relations into sub-level relations by introducing a unified quadruple\nrepresentation for both Single-Day/Multi-Day and Certain/Uncertain time\nanchors. The temporal relation classifiers are trained in a multi-label\nclassification manner. The system structure of our approach is much simpler\nthan the existing decision tree model (Reimers et al., 2018), which is composed\nby a dozen of node classifiers. Another contribution of this work is to\nconstruct a larger event time corpus (256 news documents) with a reasonable\nInter-Annotator Agreement (IAA), for the purpose of overcoming the data\nshortage of the existing event time corpus (36 news documents). The empirical\nresults show our approach outperforms the state-of-the-art decision tree model\nand the increase of data size obtained a significant improvement of\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.06884v2",
            "date": "2020-08-16 11:09:22+00:00",
            "title": "DeVLBert: Learning Deconfounded Visio-Linguistic Representations",
            "authors": [
                "Shengyu Zhang",
                "Tan Jiang",
                "Tan Wang",
                "Kun Kuang",
                "Zhou Zhao",
                "Jianke Zhu",
                "Jin Yu",
                "Hongxia Yang",
                "Fei Wu"
            ],
            "abstract": "In this paper, we propose to investigate the problem of out-of-domain\nvisio-linguistic pretraining, where the pretraining data distribution differs\nfrom that of downstream data on which the pretrained model will be fine-tuned.\nExisting methods for this problem are purely likelihood-based, leading to the\nspurious correlations and hurt the generalization ability when transferred to\nout-of-domain downstream tasks. By spurious correlation, we mean that the\nconditional probability of one token (object or word) given another one can be\nhigh (due to the dataset biases) without robust (causal) relationships between\nthem. To mitigate such dataset biases, we propose a Deconfounded\nVisio-Linguistic Bert framework, abbreviated as DeVLBert, to perform\nintervention-based learning. We borrow the idea of the backdoor adjustment from\nthe research field of causality and propose several neural-network based\narchitectures for Bert-style out-of-domain pretraining. The quantitative\nresults on three downstream tasks, Image Retrieval (IR), Zero-shot IR, and\nVisual Question Answering, show the effectiveness of DeVLBert by boosting\ngeneralization ability.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.06995v1",
            "date": "2020-08-16 20:51:17+00:00",
            "title": "Efficient Knowledge Graph Validation via Cross-Graph Representation Learning",
            "authors": [
                "Yaqing Wang",
                "Fenglong Ma",
                "Jing Gao"
            ],
            "abstract": "Recent advances in information extraction have motivated the automatic\nconstruction of huge Knowledge Graphs (KGs) by mining from large-scale text\ncorpus. However, noisy facts are unavoidably introduced into KGs that could be\ncaused by automatic extraction. To validate the correctness of facts (i.e.,\ntriplets) inside a KG, one possible approach is to map the triplets into vector\nrepresentations by capturing the semantic meanings of facts. Although many\nrepresentation learning approaches have been developed for knowledge graphs,\nthese methods are not effective for validation. They usually assume that facts\nare correct, and thus may overfit noisy facts and fail to detect such facts.\nTowards effective KG validation, we propose to leverage an external\nhuman-curated KG as auxiliary information source to help detect the errors in a\ntarget KG. The external KG is built upon human-curated knowledge repositories\nand tends to have high precision. On the other hand, although the target KG\nbuilt by information extraction from texts has low precision, it can cover new\nor domain-specific facts that are not in any human-curated repositories. To\ntackle this challenging task, we propose a cross-graph representation learning\nframework, i.e., CrossVal, which can leverage an external KG to validate the\nfacts in the target KG efficiently. This is achieved by embedding triplets\nbased on their semantic meanings, drawing cross-KG negative samples and\nestimating a confidence score for each triplet based on its degree of\ncorrectness. We evaluate the proposed framework on datasets across different\ndomains. Experimental results show that the proposed framework achieves the\nbest performance compared with the state-of-the-art methods on large-scale KGs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07267v1",
            "date": "2020-08-17 12:53:20+00:00",
            "title": "A Survey of Active Learning for Text Classification using Deep Neural Networks",
            "authors": [
                "Christopher Schr\u00f6der",
                "Andreas Niekler"
            ],
            "abstract": "Natural language processing (NLP) and neural networks (NNs) have both\nundergone significant changes in recent years. For active learning (AL)\npurposes, NNs are, however, less commonly used -- despite their current\npopularity. By using the superior text classification performance of NNs for\nAL, we can either increase a model's performance using the same amount of data\nor reduce the data and therefore the required annotation efforts while keeping\nthe same performance. We review AL for text classification using deep neural\nnetworks (DNNs) and elaborate on two main causes which used to hinder the\nadoption: (a) the inability of NNs to provide reliable uncertainty estimates,\non which the most commonly used query strategies rely, and (b) the challenge of\ntraining DNNs on small data. To investigate the former, we construct a taxonomy\nof query strategies, which distinguishes between data-based, model-based, and\nprediction-based instance selection, and investigate the prevalence of these\nclasses in recent research. Moreover, we review recent NN-based advances in NLP\nlike word embeddings or language models in the context of (D)NNs, survey the\ncurrent state-of-the-art at the intersection of AL, text classification, and\nDNNs and relate recent advances in NLP to AL. Finally, we analyze recent work\nin AL for text classification, connect the respective query strategies to the\ntaxonomy, and outline commonalities and shortcomings. As a result, we highlight\ngaps in current research and present open research questions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07291v1",
            "date": "2020-08-17 13:16:12+00:00",
            "title": "Evaluating for Diversity in Question Generation over Text",
            "authors": [
                "Michael Sejr Schlichtkrull",
                "Weiwei Cheng"
            ],
            "abstract": "Generating diverse and relevant questions over text is a task with widespread\napplications. We argue that commonly-used evaluation metrics such as BLEU and\nMETEOR are not suitable for this task due to the inherent diversity of\nreference questions, and propose a scheme for extending conventional metrics to\nreflect diversity. We furthermore propose a variational encoder-decoder model\nfor this task. We show through automatic and human evaluation that our\nvariational model improves diversity without loss of quality, and demonstrate\nhow our evaluation scheme reflects this improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.07302v1",
            "date": "2020-08-03 18:14:04+00:00",
            "title": "Lanfrica: A Participatory Approach to Documenting Machine Translation Research on African Languages",
            "authors": [
                "Chris C. Emezue",
                "Bonaventure F. P. Dossou"
            ],
            "abstract": "Over the years, there have been campaigns to include the African languages in\nthe growing research on machine translation (MT) in particular, and natural\nlanguage processing (NLP) in general. Africa has the highest language\ndiversity, with 1500-2000 documented languages and many more undocumented or\nextinct languages(Lewis, 2009; Bendor-Samuel, 2017). This makes it hard to keep\ntrack of the MT research, models and dataset that have been developed for some\nof them. As the internet and social media make up the daily lives of more than\nhalf of the world(Lin, 2020), as well as over 40% of Africans(Campbell, 2019),\nonline platforms can be useful in creating accessibility to researches,\nbenchmarks and datasets in these African languages, thereby improving\nreproducibility and sharing of existing research and their results. In this\npaper, we introduce Lanfrica, a novel, on-going framework that employs a\nparticipatory approach to documenting researches, projects, benchmarks and\ndataset on African languages.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2008.07880v1",
            "date": "2020-08-18 12:14:36+00:00",
            "title": "COVID-SEE: Scientific Evidence Explorer for COVID-19 Related Research",
            "authors": [
                "Karin Verspoor",
                "Simon \u0160uster",
                "Yulia Otmakhova",
                "Shevon Mendis",
                "Zenan Zhai",
                "Biaoyan Fang",
                "Jey Han Lau",
                "Timothy Baldwin",
                "Antonio Jimeno Yepes",
                "David Martinez"
            ],
            "abstract": "We present COVID-SEE, a system for medical literature discovery based on the\nconcept of information exploration, which builds on several distinct text\nanalysis and natural language processing methods to structure and organise\ninformation in publications, and augments search by providing a visual overview\nsupporting exploration of a collection to identify key articles of interest. We\ndeveloped this system over COVID-19 literature to help medical professionals\nand researchers explore the literature evidence, and improve findability of\nrelevant information. COVID-SEE is available at http://covid-see.com.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.08076v2",
            "date": "2020-08-18 17:57:26+00:00",
            "title": "Deploying Lifelong Open-Domain Dialogue Learning",
            "authors": [
                "Kurt Shuster",
                "Jack Urbanek",
                "Emily Dinan",
                "Arthur Szlam",
                "Jason Weston"
            ],
            "abstract": "Much of NLP research has focused on crowdsourced static datasets and the\nsupervised learning paradigm of training once and then evaluating test\nperformance. As argued in de Vries et al. (2020), crowdsourced data has the\nissues of lack of naturalness and relevance to real-world use cases, while the\nstatic dataset paradigm does not allow for a model to learn from its\nexperiences of using language (Silver et al., 2013). In contrast, one might\nhope for machine learning systems that become more useful as they interact with\npeople. In this work, we build and deploy a role-playing game, whereby human\nplayers converse with learning agents situated in an open-domain fantasy world.\nWe show that by training models on the conversations they have with humans in\nthe game the models progressively improve, as measured by automatic metrics and\nonline engagement scores. This learning is shown to be more efficient than\ncrowdsourced data when applied to conversations with real users, as well as\nbeing far cheaper to collect.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2008.08428v1",
            "date": "2020-08-19 13:31:07+00:00",
            "title": "Generating Categories for Sets of Entities",
            "authors": [
                "Shuo Zhang",
                "Krisztian Balog",
                "Jamie Callan"
            ],
            "abstract": "Category systems are central components of knowledge bases, as they provide a\nhierarchical grouping of semantically related concepts and entities. They are a\nunique and valuable resource that is utilized in a broad range of information\naccess tasks. To aid knowledge editors in the manual process of expanding a\ncategory system, this paper presents a method of generating categories for sets\nof entities. First, we employ neural abstractive summarization models to\ngenerate candidate categories. Next, the location within the hierarchy is\nidentified for each candidate. Finally, structure-, content-, and\nhierarchy-based features are used to rank candidates to identify by the most\npromising ones (measured in terms of specificity, hierarchy, and importance).\nWe develop a test collection based on Wikipedia categories and demonstrate the\neffectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.08567v2",
            "date": "2020-08-19 17:51:30+00:00",
            "title": "Transformer based Multilingual document Embedding model",
            "authors": [
                "Wei Li",
                "Brian Mak"
            ],
            "abstract": "One of the current state-of-the-art multilingual document embedding model\nLASER is based on the bidirectional LSTM neural machine translation model. This\npaper presents a transformer-based sentence/document embedding model, T-LASER,\nwhich makes three significant improvements. Firstly, the BiLSTM layers is\nreplaced by the attention-based transformer layers, which is more capable of\nlearning sequential patterns in longer texts. Secondly, due to the absence of\nrecurrence, T-LASER enables faster parallel computations in the encoder to\ngenerate the text embedding. Thirdly, we augment the NMT translation loss\nfunction with an additional novel distance constraint loss. This distance\nconstraint loss would further bring the embeddings of parallel sentences close\ntogether in the vector space; we call the T-LASER model trained with distance\nconstraint, cT-LASER. Our cT-LASER model significantly outperforms both\nBiLSTM-based LASER and the simpler transformer-based T-LASER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09035v1",
            "date": "2020-08-20 15:39:05+00:00",
            "title": "How Have We Reacted To The COVID-19 Pandemic? Analyzing Changing Indian Emotions Through The Lens of Twitter",
            "authors": [
                "Rajdeep Mukherjee",
                "Sriyash Poddar",
                "Atharva Naik",
                "Soham Dasgupta"
            ],
            "abstract": "Since its outbreak, the ongoing COVID-19 pandemic has caused unprecedented\nlosses to human lives and economies around the world. As of 18th July 2020, the\nWorld Health Organization (WHO) has reported more than 13 million confirmed\ncases including close to 600,000 deaths across 216 countries and territories.\nDespite several government measures, India has gradually moved up the ranks to\nbecome the third worst-hit nation by the pandemic after the US and Brazil, thus\ncausing widespread anxiety and fear among her citizens. As majority of the\nworld's population continues to remain confined to their homes, more and more\npeople have started relying on social media platforms such as Twitter for\nexpressing their feelings and attitudes towards various aspects of the\npandemic. With rising concerns of mental well-being, it becomes imperative to\nanalyze the dynamics of public affect in order to anticipate any potential\nthreats and take precautionary measures. Since affective states of human mind\nare more nuanced than meager binary sentiments, here we propose a deep\nlearning-based system to identify people's emotions from their tweets. We\nachieve competitive results on two benchmark datasets for multi-label emotion\nclassification. We then use our system to analyze the evolution of emotional\nresponses among Indians as the pandemic continues to spread its wings. We also\nstudy the development of salient factors contributing towards the changes in\nattitudes over time. Finally, we discuss directions to further improve our work\nand hope that our analysis can aid in better public health monitoring.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2008.09049v1",
            "date": "2020-08-20 16:03:51+00:00",
            "title": "Discovering Useful Sentence Representations from Large Pretrained Language Models",
            "authors": [
                "Nishant Subramani",
                "Nivedita Suresh"
            ],
            "abstract": "Despite the extensive success of pretrained language models as encoders for\nbuilding NLP systems, they haven't seen prominence as decoders for sequence\ngeneration tasks. We explore the question of whether these models can be\nadapted to be used as universal decoders. To be considered \"universal,\" a\ndecoder must have an implicit representation for any target sentence $s$, such\nthat it can recover that sentence exactly when conditioned on its\nrepresentation. For large transformer-based language models trained on vast\namounts of English text, we investigate whether such representations can be\neasily discovered using standard optimization methods. We present and compare\nthree representation injection techniques for transformer-based models and\nthree accompanying methods which map sentences to and from this representation\nspace. Experiments show that not only do representations exist for sentences\nfrom a variety of genres. More importantly, without needing complex\noptimization algorithms, our methods recover these sentences almost perfectly\nwithout fine-tuning the underlying language model at all.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09075v2",
            "date": "2020-08-20 17:02:37+00:00",
            "title": "Controlling Dialogue Generation with Semantic Exemplars",
            "authors": [
                "Prakhar Gupta",
                "Jeffrey P. Bigham",
                "Yulia Tsvetkov",
                "Amy Pavel"
            ],
            "abstract": "Dialogue systems pretrained with large language models generate locally\ncoherent responses, but lack the fine-grained control over responses necessary\nto achieve specific goals. A promising method to control response generation is\nexemplar-based generation, in which models edit exemplar responses that are\nretrieved from training data, or hand-written to strategically address\ndiscourse-level goals, to fit new dialogue contexts. But, current\nexemplar-based approaches often excessively copy words from the exemplar\nresponses, leading to incoherent replies. We present an Exemplar-based Dialogue\nGeneration model, EDGE, that uses the semantic frames present in exemplar\nresponses to guide generation. We show that controlling dialogue generation\nbased on the semantic frames of exemplars, rather than words in the exemplar\nitself, improves the coherence of generated responses, while preserving\nsemantic meaning and conversation goals present in exemplar responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09105v1",
            "date": "2020-08-07 02:12:56+00:00",
            "title": "Location-aware Graph Convolutional Networks for Video Question Answering",
            "authors": [
                "Deng Huang",
                "Peihao Chen",
                "Runhao Zeng",
                "Qing Du",
                "Mingkui Tan",
                "Chuang Gan"
            ],
            "abstract": "We addressed the challenging task of video question answering, which requires\nmachines to answer questions about videos in a natural language form. Previous\nstate-of-the-art methods attempt to apply spatio-temporal attention mechanism\non video frame features without explicitly modeling the location and relations\namong object interaction occurred in videos. However, the relations between\nobject interaction and their location information are very critical for both\naction recognition and question reasoning. In this work, we propose to\nrepresent the contents in the video as a location-aware graph by incorporating\nthe location information of an object into the graph construction. Here, each\nnode is associated with an object represented by its appearance and location\nfeatures. Based on the constructed graph, we propose to use graph convolution\nto infer both the category and temporal locations of an action. As the graph is\nbuilt on objects, our method is able to focus on the foreground action contents\nfor better video question answering. Lastly, we leverage an attention mechanism\nto combine the output of graph convolution and encoded question features for\nfinal answer reasoning. Extensive experiments demonstrate the effectiveness of\nthe proposed methods. Specifically, our method significantly outperforms\nstate-of-the-art methods on TGIF-QA, Youtube2Text-QA, and MSVD-QA datasets.\nCode and pre-trained models are publicly available at:\nhttps://github.com/SunDoge/L-GCN",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.09151v1",
            "date": "2020-08-20 18:31:25+00:00",
            "title": "Multi-modal Cooking Workflow Construction for Food Recipes",
            "authors": [
                "Liangming Pan",
                "Jingjing Chen",
                "Jianlong Wu",
                "Shaoteng Liu",
                "Chong-Wah Ngo",
                "Min-Yen Kan",
                "Yu-Gang Jiang",
                "Tat-Seng Chua"
            ],
            "abstract": "Understanding food recipe requires anticipating the implicit causal effects\nof cooking actions, such that the recipe can be converted into a graph\ndescribing the temporal workflow of the recipe. This is a non-trivial task that\ninvolves common-sense reasoning. However, existing efforts rely on hand-crafted\nfeatures to extract the workflow graph from recipes due to the lack of\nlarge-scale labeled datasets. Moreover, they fail to utilize the cooking\nimages, which constitute an important part of food recipes. In this paper, we\nbuild MM-ReS, the first large-scale dataset for cooking workflow construction,\nconsisting of 9,850 recipes with human-labeled workflow graphs. Cooking steps\nare multi-modal, featuring both text instructions and cooking images. We then\npropose a neural encoder-decoder model that utilizes both visual and textual\ninformation to construct the cooking workflow, which achieved over 20%\nperformance gain over existing hand-crafted baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09152v1",
            "date": "2020-08-20 18:31:52+00:00",
            "title": "ImagiFilter: A resource to enable the semi-automatic mining of images at scale",
            "authors": [
                "Houda Alberts",
                "Iacer Calixto"
            ],
            "abstract": "Datasets (semi-)automatically collected from the web can easily scale to\nmillions of entries, but a dataset's usefulness is directly related to how\nclean and high-quality its examples are. In this paper, we describe and\npublicly release an image dataset along with pretrained models designed to\n(semi-)automatically filter out undesirable images from very large image\ncollections, possibly obtained from the web. Our dataset focusses on\nphotographic and/or natural images, a very common use-case in computer vision\nresearch. We provide annotations for coarse prediction, i.e. photographic vs.\nnon-photographic, and smaller fine-grained prediction tasks where we further\nbreak down the non-photographic class into five classes: maps, drawings,\ngraphs, icons, and sketches. Results on held out validation data show that a\nmodel architecture with reduced memory footprint achieves over 96% accuracy on\ncoarse-prediction. Our best model achieves 88% accuracy on the hardest\nfine-grained classification task available. Dataset and pretrained models are\navailable at: https://github.com/houda96/imagi-filter.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.09333v1",
            "date": "2020-08-21 06:56:57+00:00",
            "title": "Tweet to News Conversion: An Investigation into Unsupervised Controllable Text Generation",
            "authors": [
                "Zishan Ahmad",
                "Mukuntha N S",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "Text generator systems have become extremely popular with the advent of\nrecent deep learning models such as encoder-decoder. Controlling the\ninformation and style of the generated output without supervision is an\nimportant and challenging Natural Language Processing (NLP) task. In this\npaper, we define the task of constructing a coherent paragraph from a set of\ndisaster domain tweets, without any parallel data. We tackle the problem by\nbuilding two systems in pipeline. The first system focuses on unsupervised\nstyle transfer and converts the individual tweets into news sentences. The\nsecond system stitches together the outputs from the first system to form a\ncoherent news paragraph. We also propose a novel training mechanism, by\nsplitting the sentences into propositions and training the second system to\nmerge the sentences. We create a validation and test set consisting of\ntweet-sets and their equivalent news paragraphs to perform empirical\nevaluation. In a completely unsupervised setting, our model was able to achieve\na BLEU score of 19.32, while successfully transferring styles and joining\ntweets to form a meaningful news paragraph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09335v2",
            "date": "2020-08-21 07:02:11+00:00",
            "title": "MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark",
            "authors": [
                "Haoran Li",
                "Abhinav Arora",
                "Shuohui Chen",
                "Anchit Gupta",
                "Sonal Gupta",
                "Yashar Mehdad"
            ],
            "abstract": "Scaling semantic parsing models for task-oriented dialog systems to new\nlanguages is often expensive and time-consuming due to the lack of available\ndatasets. Available datasets suffer from several shortcomings: a) they contain\nfew languages b) they contain small amounts of labeled examples per language c)\nthey are based on the simple intent and slot detection paradigm for\nnon-compositional queries. In this paper, we present a new multilingual\ndataset, called MTOP, comprising of 100k annotated utterances in 6 languages\nacross 11 domains. We use this dataset and other publicly available datasets to\nconduct a comprehensive benchmarking study on using various state-of-the-art\nmultilingual pre-trained models for task-oriented semantic parsing. We achieve\nan average improvement of +6.3 points on Slot F1 for the two existing\nmultilingual datasets, over best results reported in their experiments.\nFurthermore, we demonstrate strong zero-shot performance using pre-trained\nmodels combined with automatic translation and alignment, and a proposed\ndistant supervision method to reduce the noise in slot label projection.",
            "journal ref": "EACL 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09371v1",
            "date": "2020-08-21 08:46:36+00:00",
            "title": "It's better to say \"I can't answer\" than answering incorrectly: Towards Safety critical NLP systems",
            "authors": [
                "Neeraj Varshney",
                "Swaroop Mishra",
                "Chitta Baral"
            ],
            "abstract": "In order to make AI systems more reliable and their adoption in safety\ncritical applications possible, it is essential to impart the capability to\nabstain from answering when their prediction is likely to be incorrect and seek\nhuman intervention. Recently proposed \"selective answering\" techniques model\ncalibration as a binary classification task. We argue that, not all incorrectly\nanswered questions are incorrect to the same extent and the same is true for\ncorrectly answered questions. Hence, treating all correct predictions equally\nand all incorrect predictions equally constraints calibration. In this work, we\npropose a methodology that incorporates the degree of correctness, shifting\naway from classification labels as it directly tries to predict the probability\nof model's prediction being correct. We show the efficacy of the proposed\nmethod on existing Natural Language Inference (NLI) datasets by training on\nSNLI and evaluating on MNLI mismatched and matched datasets. Our approach\nimproves area under the curve (AUC) of risk-coverage plot by 10.22\\% and 8.06\\%\nover maxProb with respect to the maximum possible improvement on MNLI\nmismatched and matched set respectively. In order to evaluate our method on Out\nof Distribution (OOD) datasets, we propose a novel setup involving questions\nwith a variety of reasoning skills. Our setup includes a test set for each of\nthe five reasoning skills: numerical, logical, qualitative, abductive and\ncommonsense. We select confidence threshold for each of the approaches where\nthe in-domain accuracy (SNLI) is 99\\%. Our results show that, the proposed\nmethod outperforms existing approaches by abstaining on 2.6\\% more OOD\nquestions at respective confidence thresholds.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09378v1",
            "date": "2020-08-21 08:59:29+00:00",
            "title": "EmoGraph: Capturing Emotion Correlations using Graph Networks",
            "authors": [
                "Peng Xu",
                "Zihan Liu",
                "Genta Indra Winata",
                "Zhaojiang Lin",
                "Pascale Fung"
            ],
            "abstract": "Most emotion recognition methods tackle the emotion understanding task by\nconsidering individual emotion independently while ignoring their fuzziness\nnature and the interconnections among them. In this paper, we explore how\nemotion correlations can be captured and help different classification tasks.\nWe propose EmoGraph that captures the dependencies among different emotions\nthrough graph networks. These graphs are constructed by leveraging the\nco-occurrence statistics among different emotion categories. Empirical results\non two multi-label classification datasets demonstrate that EmoGraph\noutperforms strong baselines, especially for macro-F1. An additional experiment\nillustrates the captured emotion correlations can also benefit a single-label\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09606v1",
            "date": "2020-08-21 17:59:01+00:00",
            "title": "Howl: A Deployed, Open-Source Wake Word Detection System",
            "authors": [
                "Raphael Tang",
                "Jaejun Lee",
                "Afsaneh Razi",
                "Julia Cambre",
                "Ian Bicking",
                "Jofish Kaye",
                "Jimmy Lin"
            ],
            "abstract": "We describe Howl, an open-source wake word detection toolkit with native\nsupport for open speech datasets, like Mozilla Common Voice and Google Speech\nCommands. We report benchmark results on Speech Commands and our own freely\navailable wake word detection dataset, built from MCV. We operationalize our\nsystem for Firefox Voice, a plugin enabling speech interactivity for the\nFirefox web browser. Howl represents, to the best of our knowledge, the first\nfully productionized yet open-source wake word detection toolkit with a web\nbrowser deployment target. Our codebase is at\nhttps://github.com/castorini/howl.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09676v3",
            "date": "2020-08-21 20:59:34+00:00",
            "title": "Abstractive Summarization of Spoken and Written Instructions with BERT",
            "authors": [
                "Alexandra Savelieva",
                "Bryan Au-Yeung",
                "Vasanth Ramani"
            ],
            "abstract": "Summarization of speech is a difficult problem due to the spontaneity of the\nflow, disfluencies, and other issues that are not usually encountered in\nwritten texts. Our work presents the first application of the BERTSum model to\nconversational language. We generate abstractive summaries of narrated\ninstructional videos across a wide variety of topics, from gardening and\ncooking to software configuration and sports. In order to enrich the\nvocabulary, we use transfer learning and pretrain the model on a few large\ncross-domain datasets in both written and spoken English. We also do\npreprocessing of transcripts to restore sentence segmentation and punctuation\nin the output of an ASR system. The results are evaluated with ROUGE and\nContent-F1 scoring for the How2 and WikiHow datasets. We engage human judges to\nscore a set of summaries randomly selected from a dataset curated from\nHowTo100M and YouTube. Based on blind evaluation, we achieve a level of textual\nfluency and utility close to that of summaries written by human content\ncreators. The model beats current SOTA when applied to WikiHow articles that\nvary widely in style and topic, while showing no performance regression on the\ncanonical CNN/DailyMail dataset. Due to the high generalizability of the model\nacross different styles and domains, it has great potential to improve\naccessibility and discoverability of internet content. We envision this\nintegrated as a feature in intelligent virtual assistants, enabling them to\nsummarize both written and spoken instructional content upon request.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09689v1",
            "date": "2020-08-21 21:48:00+00:00",
            "title": "Fine-tune BERT for E-commerce Non-Default Search Ranking",
            "authors": [
                "Yunjiang Jiang",
                "Yue Shang",
                "Hongwei Shen",
                "Wen-Yun Yang",
                "Yun Xiao"
            ],
            "abstract": "The quality of non-default ranking on e-commerce platforms, such as based on\nascending item price or descending historical sales volume, often suffers from\nacute relevance problems, since the irrelevant items are much easier to be\nexposed at the top of the ranking results. In this work, we propose a two-stage\nranking scheme, which first recalls wide range of candidate items through\nrefined query/title keyword matching, and then classifies the recalled items\nusing BERT-Large fine-tuned on human label data. We also implemented parallel\nprediction on multiple GPU hosts and a C++ tokenization custom op of\nTensorflow. In this data challenge, our model won the 1st place in the\nsupervised phase (based on overall F1 score) and 2nd place in the final phase\n(based on average per query F1 score).",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2008.09706v1",
            "date": "2020-08-21 22:43:27+00:00",
            "title": "Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology",
            "authors": [
                "Yangjun Zhang",
                "Pengjie Ren",
                "Maarten de Rijke"
            ],
            "abstract": "Conversational interfaces are increasingly popular as a way of connecting\npeople to information. Corpus-based conversational interfaces are able to\ngenerate more diverse and natural responses than template-based or\nretrieval-based agents. With their increased generative capacity of corpusbased\nconversational agents comes the need to classify and filter out malevolent\nresponses that are inappropriate in terms of content and dialogue acts.\nPrevious studies on the topic of recognizing and classifying inappropriate\ncontent are mostly focused on a certain category of malevolence or on single\nsentences instead of an entire dialogue. In this paper, we define the task of\nMalevolent Dialogue Response Detection and Classification (MDRDC). We make\nthree contributions to advance research on this task. First, we present a\nHierarchical Malevolent Dialogue Taxonomy (HMDT). Second, we create a labelled\nmulti-turn dialogue dataset and formulate the MDRDC task as a hierarchical\nclassification task over this taxonomy. Third, we apply stateof-the-art text\nclassification methods to the MDRDC task and report on extensive experiments\naimed at assessing the performance of these approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09820v1",
            "date": "2020-08-22 12:01:44+00:00",
            "title": "HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection",
            "authors": [
                "Meghana Bhange",
                "Nirant Kasliwal"
            ],
            "abstract": "Sentiment analysis for code-mixed social media text continues to be an\nunder-explored area. This work adds two common approaches: fine-tuning large\ntransformer models and sample efficient methods like ULMFiT. Prior work\ndemonstrates the efficacy of classical ML methods for polarity detection.\nFine-tuned general-purpose language representation models, such as those of the\nBERT family are benchmarked along with classical machine learning and ensemble\nmethods. We show that NB-SVM beats RoBERTa by 6.2% (relative) F1. The best\nperforming model is a majority-vote ensemble which achieves an F1 of 0.707. The\nleaderboard submission was made under the codalab username nirantk, with F1 of\n0.689.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09859v1",
            "date": "2020-08-22 15:51:16+00:00",
            "title": "CyberWallE at SemEval-2020 Task 11: An Analysis of Feature Engineering for Ensemble Models for Propaganda Detection",
            "authors": [
                "Verena Blaschke",
                "Maxim Korniyenko",
                "Sam Tureski"
            ],
            "abstract": "This paper describes our participation in the SemEval-2020 task Detection of\nPropaganda Techniques in News Articles. We participate in both subtasks: Span\nIdentification (SI) and Technique Classification (TC). We use a bi-LSTM\narchitecture in the SI subtask and train a complex ensemble model for the TC\nsubtask. Our architectures are built using embeddings from BERT in combination\nwith additional lexical features and extensive label post-processing. Our\nsystems achieve a rank of 8 out of 35 teams in the SI subtask (F1-score:\n43.86%) and 8 out of 31 teams in the TC subtask (F1-score: 57.37%).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09869v1",
            "date": "2020-08-22 16:31:01+00:00",
            "title": "UTMN at SemEval-2020 Task 11: A Kitchen Solution to Automatic Propaganda Detection",
            "authors": [
                "Elena Mikhalkova",
                "Nadezhda Ganzherli",
                "Anna Glazkova",
                "Yuliya Bidulya"
            ],
            "abstract": "The article describes a fast solution to propaganda detection at SemEval-2020\nTask 11, based onfeature adjustment. We use per-token vectorization of features\nand a simple Logistic Regressionclassifier to quickly test different hypotheses\nabout our data. We come up with what seems to usthe best solution, however, we\nare unable to align it with the result of the metric suggested by theorganizers\nof the task. We test how our system handles class and feature imbalance by\nvarying thenumber of samples of two classes (Propaganda and None) in the\ntraining set, the size of a contextwindow in which a token is vectorized and\ncombination of vectorization means. The result of oursystem at SemEval2020 Task\n11 is F-score=0.37.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09961v1",
            "date": "2020-08-23 05:14:38+00:00",
            "title": "An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, Pizzagate and storytelling on the web",
            "authors": [
                "Timothy R. Tangherlini",
                "Shadi Shahsavari",
                "Behnam Shahbazi",
                "Ehsan Ebrahimzadeh",
                "Vwani Roychowdhury"
            ],
            "abstract": "Although a great deal of attention has been paid to how conspiracy theories\ncirculate on social media and their factual counterpart conspiracies, there has\nbeen little computational work done on describing their narrative structures.\nWe present an automated pipeline for the discovery and description of the\ngenerative narrative frameworks of conspiracy theories on social media, and\nactual conspiracies reported in the news media. We base this work on two\nseparate repositories of posts and news articles describing the well-known\nconspiracy theory Pizzagate from 2016, and the New Jersey conspiracy Bridgegate\nfrom 2013. We formulate a graphical generative machine learning model where\nnodes represent actors/actants, and multi-edges and self-loops among nodes\ncapture context-specific relationships. Posts and news items are viewed as\nsamples of subgraphs of the hidden narrative network. The problem of\nreconstructing the underlying structure is posed as a latent model estimation\nproblem. We automatically extract and aggregate the actants and their\nrelationships from the posts and articles. We capture context specific actants\nand interactant relationships by developing a system of supernodes and\nsubnodes. We use these to construct a network, which constitutes the underlying\nnarrative framework. We show how the Pizzagate framework relies on the\nconspiracy theorists' interpretation of \"hidden knowledge\" to link otherwise\nunlinked domains of human interaction, and hypothesize that this multi-domain\nfocus is an important feature of conspiracy theories. While Pizzagate relies on\nthe alignment of multiple domains, Bridgegate remains firmly rooted in the\nsingle domain of New Jersey politics. We hypothesize that the narrative\nframework of a conspiracy theory might stabilize quickly in contrast to the\nnarrative framework of an actual one, which may develop more slowly as\nrevelations come to light.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.09976v1",
            "date": "2020-08-23 06:34:05+00:00",
            "title": "Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing",
            "authors": [
                "Cuiyun Gao",
                "Jichuan Zeng",
                "Zhiyuan Wen",
                "David Lo",
                "Xin Xia",
                "Irwin King",
                "Michael R. Lyu"
            ],
            "abstract": "Millions of mobile apps are available in app stores, such as Apple's App\nStore and Google Play. For a mobile app, it would be increasingly challenging\nto stand out from the enormous competitors and become prevalent among users.\nGood user experience and well-designed functionalities are the keys to a\nsuccessful app. To achieve this, popular apps usually schedule their updates\nfrequently. If we can capture the critical app issues faced by users in a\ntimely and accurate manner, developers can make timely updates, and good user\nexperience can be ensured. There exist prior studies on analyzing reviews for\ndetecting emerging app issues. These studies are usually based on topic\nmodeling or clustering techniques. However, the short-length characteristics\nand sentiment of user reviews have not been considered. In this paper, we\npropose a novel emerging issue detection approach named MERIT to take into\nconsideration the two aforementioned characteristics. Specifically, we propose\nan Adaptive Online Biterm Sentiment-Topic (AOBST) model for jointly modeling\ntopics and corresponding sentiments that takes into consideration app versions.\nBased on the AOBST model, we infer the topics negatively reflected in user\nreviews for one app version, and automatically interpret the meaning of the\ntopics with most relevant phrases and sentences. Experiments on popular apps\nfrom Google Play and Apple's App Store demonstrate the effectiveness of MERIT\nin identifying emerging app issues, improving the state-of-the-art method by\n22.3% in terms of F1-score. In terms of efficiency, MERIT can return results\nwithin acceptable time.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2008.10105v1",
            "date": "2020-08-23 21:00:33+00:00",
            "title": "Deep Bayes Factor Scoring for Authorship Verification",
            "authors": [
                "Benedikt Boenninghoff",
                "Julian Rupp",
                "Robert M. Nickel",
                "Dorothea Kolossa"
            ],
            "abstract": "The PAN 2020 authorship verification (AV) challenge focuses on a\ncross-topic/closed-set AV task over a collection of fanfiction texts.\nFanfiction is a fan-written extension of a storyline in which a so-called\nfandom topic describes the principal subject of the document. The data provided\nin the PAN 2020 AV task is quite challenging because authors of texts across\nmultiple/different fandom topics are included. In this work, we present a\nhierarchical fusion of two well-known approaches into a single end-to-end\nlearning procedure: A deep metric learning framework at the bottom aims to\nlearn a pseudo-metric that maps a document of variable length onto a\nfixed-sized feature vector. At the top, we incorporate a probabilistic layer to\nperform Bayes factor scoring in the learned metric space. We also provide text\npreprocessing strategies to deal with the cross-topic issue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10166v2",
            "date": "2020-08-24 02:42:12+00:00",
            "title": "YNU-HPCC at SemEval-2020 Task 11: LSTM Network for Detection of Propaganda Techniques in News Articles",
            "authors": [
                "Jiaxu Dao",
                "Jin Wang",
                "Xuejie Zhang"
            ],
            "abstract": "This paper summarizes our studies on propaganda detection techniques for news\narticles in the SemEval-2020 task 11. This task is divided into the SI and TC\nsubtasks. We implemented the GloVe word representation, the BERT pretraining\nmodel, and the LSTM model architecture to accomplish this task. Our approach\nachieved good results for both the SI and TC subtasks. The macro-F1-score for\nthe SI subtask is 0.406, and the micro-F1-score for the TC subtask is 0.505.\nOur method significantly outperforms the officially released baseline method,\nand the SI and TC subtasks rank 17th and 22nd, respectively, for the test set.\nThis paper also compares the performances of different deep learning model\narchitectures, such as the Bi-LSTM, LSTM, BERT, and XGBoost models, on the\ndetection of news promotion techniques. The code of this paper is availabled\nat: https://github.com/daojiaxu/semeval_11.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10427v1",
            "date": "2020-08-24 13:28:35+00:00",
            "title": "How To Evaluate Your Dialogue System: Probe Tasks as an Alternative for Token-level Evaluation Metrics",
            "authors": [
                "Prasanna Parthasarathi",
                "Joelle Pineau",
                "Sarath Chandar"
            ],
            "abstract": "Though generative dialogue modeling is widely seen as a language modeling\ntask, the task demands an agent to have a complex natural language\nunderstanding of its input text to carry a meaningful interaction with an user.\nThe automatic metrics used evaluate the quality of the generated text as a\nproxy to the holistic interaction of the agent. Such metrics were earlier shown\nto not correlate with the human judgement. In this work, we observe that human\nevaluation of dialogue agents can be inconclusive due to the lack of sufficient\ninformation for appropriate evaluation. The automatic metrics are deterministic\nyet shallow and human evaluation can be relevant yet inconclusive. To bridge\nthis gap in evaluation, we propose designing a set of probing tasks to evaluate\ndialogue models. The hand-crafted tasks are aimed at quantitatively evaluating\na generative dialogue model's understanding beyond the token-level evaluation\non the generated text. The probing tasks are deterministic like automatic\nmetrics and requires human judgement in their designing; benefiting from the\nbest of both worlds. With experiments on probe tasks we observe that, unlike\nRNN based architectures, transformer model may not be learning to comprehend\nthe input text despite its generated text having higher overlap with the target\ntext.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10492v1",
            "date": "2020-08-24 14:53:21+00:00",
            "title": "Prediction of ICD Codes with Clinical BERT Embeddings and Text Augmentation with Label Balancing using MIMIC-III",
            "authors": [
                "Brent Biseda",
                "Gaurav Desai",
                "Haifeng Lin",
                "Anish Philip"
            ],
            "abstract": "This paper achieves state of the art results for the ICD code prediction task\nusing the MIMIC-III dataset. This was achieved through the use of Clinical BERT\n(Alsentzer et al., 2019). embeddings and text augmentation and label balancing\nto improve F1 scores for both ICD Chapter as well as ICD disease codes. We\nattribute the improved performance mainly to the use of novel text augmentation\nto shuffle the order of sentences during training. In comparison to the Top-32\nICD code prediction (Keyang Xu, et. al.) with an F1 score of 0.76, we achieve a\nfinal F1 score of 0.75 but on a total of the top 50 ICD codes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10522v1",
            "date": "2020-08-24 15:49:54+00:00",
            "title": "Machine Semiotics",
            "authors": [
                "Peter Klimczak",
                "G\u00fcnther Wirsching",
                "Peter beim Graben"
            ],
            "abstract": "Despite their satisfactory speech recognition capabilities, current speech\nassistive devices still lack suitable automatic semantic analysis capabilities\nas well as useful representation of pragmatic world knowledge. Instead, current\ntechnologies require users to learn keywords necessary to effectively operate\nand work with a machine. Such a machine-centered approach can be frustrating\nfor users. However, recognizing a basic difference between the semiotics of\nhumans and machines presents a possibility to overcome this shortcoming: For\nthe machine, the meaning of a (human) utterance is defined by its own scope of\nactions. Machines, thus, do not need to understand the meanings of individual\nwords, nor the meaning of phrasal and sentence semantics that combine\nindividual word meanings with additional implicit world knowledge. For speech\nassistive devices, the learning of machine specific meanings of human\nutterances by trial and error should be sufficient. Using the trivial example\nof a cognitive heating device, we show that -- based on dynamic semantics --\nthis process can be formalized as the learning of utterance-meaning pairs\n(UMP). This is followed by a detailed semiotic contextualization of the\npreviously generated signs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10570v1",
            "date": "2020-08-24 17:18:24+00:00",
            "title": "Example-Based Named Entity Recognition",
            "authors": [
                "Morteza Ziyadi",
                "Yuting Sun",
                "Abhishek Goswami",
                "Jade Huang",
                "Weizhu Chen"
            ],
            "abstract": "We present a novel approach to named entity recognition (NER) in the presence\nof scarce data that we call example-based NER. Our train-free few-shot learning\napproach takes inspiration from question-answering to identify entity spans in\na new and unseen domain. In comparison with the current state-of-the-art, the\nproposed method performs significantly better, especially when using a low\nnumber of support examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.10835v1",
            "date": "2020-08-25 06:03:14+00:00",
            "title": "Complicating the Social Networks for Better Storytelling: An Empirical Study of Chinese Historical Text and Novel",
            "authors": [
                "Chenhan Zhang"
            ],
            "abstract": "Digital humanities is an important subject because it enables developments in\nhistory, literature, and films. In this paper, we perform an empirical study of\na Chinese historical text, Records of the Three Kingdoms (\\textit{Records}),\nand a historical novel of the same story, Romance of the Three Kingdoms\n(\\textit{Romance}). We employ natural language processing techniques to extract\ncharacters and their relationships. Then, we characterize the social networks\nand sentiments of the main characters in the historical text and the historical\nnovel. We find that the social network in \\textit{Romance} is more complex and\ndynamic than that of \\textit{Records}, and the influence of the main characters\ndiffers. These findings shed light on the different styles of storytelling in\nthe two literary genres and how the historical novel complicates the social\nnetworks of characters to enrich the literariness of the story.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2008.10873v1",
            "date": "2020-08-25 08:15:55+00:00",
            "title": "Is this sentence valid? An Arabic Dataset for Commonsense Validation",
            "authors": [
                "Saja Tawalbeh",
                "Mohammad AL-Smadi"
            ],
            "abstract": "The commonsense understanding and validation remains a challenging task in\nthe field of natural language understanding. Therefore, several research papers\nhave been published that studied the capability of proposed systems to evaluate\nthe models ability to validate commonsense in text. In this paper, we present a\nbenchmark Arabic dataset for commonsense understanding and validation as well\nas a baseline research and models trained using the same dataset. To the best\nof our knowledge, this dataset is considered as the first in the field of\nArabic text commonsense validation. The dataset is distributed under the\nCreative Commons BY-SA 4.0 license and can be found on GitHub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11228v1",
            "date": "2020-08-25 18:31:08+00:00",
            "title": "A simple method for domain adaptation of sentence embeddings",
            "authors": [
                "Anna Kruspe"
            ],
            "abstract": "Pre-trained sentence embeddings have been shown to be very useful for a\nvariety of NLP tasks. Due to the fact that training such embeddings requires a\nlarge amount of data, they are commonly trained on a variety of text data. An\nadaptation to specific domains could improve results in many cases, but such a\nfinetuning is usually problem-dependent and poses the risk of over-adapting to\nthe data used for adaptation. In this paper, we present a simple universal\nmethod for finetuning Google's Universal Sentence Encoder (USE) using a Siamese\narchitecture. We demonstrate how to use this approach for a variety of data\nsets and present results on different data sets representing similar problems.\nThe approach is also compared to traditional finetuning on these data sets. As\na further advantage, the approach can be used for combining data sets with\ndifferent annotations. We also present an embedding finetuned on all data sets\nin parallel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11649v2",
            "date": "2020-08-26 16:15:18+00:00",
            "title": "Discrete Word Embedding for Logical Natural Language Understanding",
            "authors": [
                "Masataro Asai",
                "Zilu Tang"
            ],
            "abstract": "We propose an unsupervised neural model for learning a discrete embedding of\nwords. Unlike existing discrete embeddings, our binary embedding supports\nvector arithmetic operations similar to continuous embeddings. Our embedding\nrepresents each word as a set of propositional statements describing a\ntransition rule in classical/STRIPS planning formalism. This makes the\nembedding directly compatible with symbolic, state of the art classical\nplanning solvers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11825v1",
            "date": "2020-08-26 21:28:41+00:00",
            "title": "SHAP values for Explaining CNN-based Text Classification Models",
            "authors": [
                "Wei Zhao",
                "Tarun Joshi",
                "Vijayan N. Nair",
                "Agus Sudjianto"
            ],
            "abstract": "Deep neural networks are increasingly used in natural language processing\n(NLP) models. However, the need to interpret and explain the results from\ncomplex algorithms are limiting their widespread adoption in regulated\nindustries such as banking. There has been recent work on interpretability of\nmachine learning algorithms with structured data. But there are only limited\ntechniques for NLP applications where the problem is more challenging due to\nthe size of the vocabulary, high-dimensional nature, and the need to consider\ntextual coherence and language structure. This paper develops a methodology to\ncompute SHAP values for local explainability of CNN-based text classification\nmodels. The approach is also extended to compute global scores to assess the\nimportance of features. The results are illustrated on sentiment analysis of\nAmazon Electronic Review data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11869v4",
            "date": "2020-08-27 00:23:48+00:00",
            "title": "AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization",
            "authors": [
                "Xinsong Zhang",
                "Pengshuai Li",
                "Hang Li"
            ],
            "abstract": "Pre-trained language models such as BERT have exhibited remarkable\nperformances in many tasks in natural language understanding (NLU). The tokens\nin the models are usually fine-grained in the sense that for languages like\nEnglish they are words or sub-words and for languages like Chinese they are\ncharacters. In English, for example, there are multi-word expressions which\nform natural lexical units and thus the use of coarse-grained tokenization also\nappears to be reasonable. In fact, both fine-grained and coarse-grained\ntokenizations have advantages and disadvantages for learning of pre-trained\nlanguage models. In this paper, we propose a novel pre-trained language model,\nreferred to as AMBERT (A Multi-grained BERT), on the basis of both fine-grained\nand coarse-grained tokenizations. For English, AMBERT takes both the sequence\nof words (fine-grained tokens) and the sequence of phrases (coarse-grained\ntokens) as input after tokenization, employs one encoder for processing the\nsequence of words and the other encoder for processing the sequence of the\nphrases, utilizes shared parameters between the two encoders, and finally\ncreates a sequence of contextualized representations of the words and a\nsequence of contextualized representations of the phrases. Experiments have\nbeen conducted on benchmark datasets for Chinese and English, including CLUE,\nGLUE, SQuAD and RACE. The results show that AMBERT can outperform BERT in all\ncases, particularly the improvements are significant for Chinese. We also\ndevelop a method to improve the efficiency of AMBERT in inference, which still\nperforms better than BERT with the same computational cost as BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11897v1",
            "date": "2020-08-27 03:15:40+00:00",
            "title": "Automatic Speech Summarisation: A Scoping Review",
            "authors": [
                "Dana Rezazadegan",
                "Shlomo Berkovsky",
                "Juan C. Quiroz",
                "A. Baki Kocaballi",
                "Ying Wang",
                "Liliana Laranjo",
                "Enrico Coiera"
            ],
            "abstract": "Speech summarisation techniques take human speech as input and then output an\nabridged version as text or speech. Speech summarisation has applications in\nmany domains from information technology to health care, for example improving\nspeech archives or reducing clinical documentation burden. This scoping review\nmaps the speech summarisation literature, with no restrictions on time frame,\nlanguage summarised, research method, or paper type. We reviewed a total of 110\npapers out of a set of 153 found through a literature search and extracted\nspeech features used, methods, scope, and training corpora. Most studies employ\none of four speech summarisation architectures: (1) Sentence extraction and\ncompaction; (2) Feature extraction and classification or rank-based sentence\nselection; (3) Sentence compression and compression summarisation; and (4)\nLanguage modelling. We also discuss the strengths and weaknesses of these\ndifferent methods and speech features. Overall, supervised methods (e.g. Hidden\nMarkov support vector machines, Ranking support vector machines, Conditional\nrandom fields) performed better than unsupervised methods. As supervised\nmethods require manually annotated training data which can be costly, there was\nmore interest in unsupervised methods. Recent research into unsupervised\nmethods focusses on extending language modelling, for example by combining\nUni-gram modelling with deep neural networks. Protocol registration: The\nprotocol for this scoping review is registered at https://osf.io.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.11972v2",
            "date": "2020-08-27 07:54:45+00:00",
            "title": "Opinion-aware Answer Generation for Review-driven Question Answering in E-Commerce",
            "authors": [
                "Yang Deng",
                "Wenxuan Zhang",
                "Wai Lam"
            ],
            "abstract": "Product-related question answering (QA) is an important but challenging task\nin E-Commerce. It leads to a great demand on automatic review-driven QA, which\naims at providing instant responses towards user-posted questions based on\ndiverse product reviews. Nevertheless, the rich information about personal\nopinions in product reviews, which is essential to answer those\nproduct-specific questions, is underutilized in current generation-based\nreview-driven QA studies. There are two main challenges when exploiting the\nopinion information from the reviews to facilitate the opinion-aware answer\ngeneration: (i) jointly modeling opinionated and interrelated information\nbetween the question and reviews to capture important information for answer\ngeneration, (ii) aggregating diverse opinion information to uncover the common\nopinion towards the given question. In this paper, we tackle opinion-aware\nanswer generation by jointly learning answer generation and opinion mining\ntasks with a unified model. Two kinds of opinion fusion strategies, namely,\nstatic and dynamic fusion, are proposed to distill and aggregate important\nopinion information learned from the opinion mining task into the answer\ngeneration process. Then a multi-view pointer-generator network is employed to\ngenerate opinion-aware answers for a given product-related question.\nExperimental results show that our method achieves superior performance in\nreal-world E-Commerce QA datasets, and effectively generate opinionated and\ninformative answers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12348v2",
            "date": "2020-08-27 19:37:27+00:00",
            "title": "Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations",
            "authors": [
                "Ashwin Paranjape",
                "Abigail See",
                "Kathleen Kenealy",
                "Haojun Li",
                "Amelia Hardy",
                "Peng Qi",
                "Kaushik Ram Sadagopan",
                "Nguyet Minh Phu",
                "Dilara Soylu",
                "Christopher D. Manning"
            ],
            "abstract": "We present Chirpy Cardinal, an open-domain dialogue agent, as a research\nplatform for the 2019 Alexa Prize competition. Building an open-domain\nsocialbot that talks to real people is challenging - such a system must meet\nmultiple user expectations such as broad world knowledge, conversational style,\nand emotional connection. Our socialbot engages users on their terms -\nprioritizing their interests, feelings and autonomy. As a result, our socialbot\nprovides a responsive, personalized user experience, capable of talking\nknowledgeably about a wide variety of topics, as well as chatting\nempathetically about ordinary life. Neural generation plays a key role in\nachieving these goals, providing the backbone for our conversational and\nemotional tone. At the end of the competition, Chirpy Cardinal progressed to\nthe finals with an average rating of 3.6/5.0, a median conversation duration of\n2 minutes 16 seconds, and a 90th percentile duration of over 12 minutes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12520v1",
            "date": "2020-08-28 07:33:30+00:00",
            "title": "A Dataset and Baselines for Visual Question Answering on Art",
            "authors": [
                "Noa Garcia",
                "Chentao Ye",
                "Zihua Liu",
                "Qingtao Hu",
                "Mayu Otani",
                "Chenhui Chu",
                "Yuta Nakashima",
                "Teruko Mitamura"
            ],
            "abstract": "Answering questions related to art pieces (paintings) is a difficult task, as\nit implies the understanding of not only the visual information that is shown\nin the picture, but also the contextual knowledge that is acquired through the\nstudy of the history of art. In this work, we introduce our first attempt\ntowards building a new dataset, coined AQUA (Art QUestion Answering). The\nquestion-answer (QA) pairs are automatically generated using state-of-the-art\nquestion generation methods based on paintings and comments provided in an\nexisting art understanding dataset. The QA pairs are cleansed by crowdsourcing\nworkers with respect to their grammatical correctness, answerability, and\nanswers' correctness. Our dataset inherently consists of visual\n(painting-based) and knowledge (comment-based) questions. We also present a\ntwo-branch model as baseline, where the visual and knowledge questions are\nhandled independently. We extensively compare our baseline model against the\nstate-of-the-art models for question answering, and we provide a comprehensive\nstudy about the challenges and potential future directions for visual question\nanswering on art.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2008.12552v2",
            "date": "2020-08-28 09:37:39+00:00",
            "title": "Temporal Random Indexing of Context Vectors Applied to Event Detection",
            "authors": [
                "Yashank Singh",
                "Niladri Chatterjee"
            ],
            "abstract": "In this paper we explore new representations for encoding language data. The\ngeneral method of one-hot encoding grows linearly with the size of the word\ncorpus in space-complexity. We address this by using Random Indexing(RI) of\ncontext vectors with non-zero entries. We propose a novel RI representation\nwhere we exploit the effect imposing a probability distribution on the number\nof randomized entries which leads to a class of RI representations. We also\npropose an algorithm that is log linear in the size of word corpus to track the\nsemantic relationship of the query word to other words for suggesting the\nevents that are relevant to the word in question. Finally we run simulations on\nthe novel RI representations using the proposed algorithms for tweets relevant\nto the word \"iPhone\" and present results. The RI representation is shown to be\nfaster and space efficient as compared to BoW embeddings.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2008.12579v2",
            "date": "2020-08-28 10:59:31+00:00",
            "title": "The Adapter-Bot: All-In-One Controllable Conversational Model",
            "authors": [
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Yejin Bang",
                "Pascale Fung"
            ],
            "abstract": "Considerable progress has been made towards conversational models that\ngenerate coherent and fluent responses by training large language models on\nlarge dialogue datasets. These models have little or no control of the\ngenerated responses and miss two important features: continuous dialogue skills\nintegration and seamlessly leveraging diverse knowledge sources. In this paper,\nwe propose the Adapter-Bot, a dialogue model that uses a fixed backbone\nconversational model such as DialGPT (Zhang et al., 2019) and triggers\non-demand dialogue skills (e.g., emphatic response, weather information, movie\nrecommendation) via different adapters (Houlsby et al., 2019). Each adapter can\nbe trained independently, thus allowing a continual integration of skills\nwithout retraining the entire model. Depending on the skills, the model is able\nto process multiple knowledge types, such as text, tables, and graphs, in a\nseamless manner. The dialogue skills can be triggered automatically via a\ndialogue manager, or manually, thus allowing high-level control of the\ngenerated responses. At the current stage, we have implemented 12 response\nstyles (e.g., positive, negative etc.), 8 goal-oriented skills (e.g. weather\ninformation, movie recommendation, etc.), and personalized and emphatic\nresponses. We evaluate our model using automatic evaluation by comparing it\nwith existing state-of-the-art conversational models, and we have released an\ninteractive system at adapter.bot.ust.hk.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12813v1",
            "date": "2020-08-28 18:58:15+00:00",
            "title": "HittER: Hierarchical Transformers for Knowledge Graph Embeddings",
            "authors": [
                "Sanxing Chen",
                "Xiaodong Liu",
                "Jianfeng Gao",
                "Jian Jiao",
                "Ruofei Zhang",
                "Yangfeng Ji"
            ],
            "abstract": "This paper examines the challenging problem of learning representations of\nentities and relations in a complex multi-relational knowledge graph. We\npropose HittER, a Hierarchical Transformer model to jointly learn\nEntity-relation composition and Relational contextualization based on a source\nentity's neighborhood. Our proposed model consists of two different Transformer\nblocks: the bottom block extracts features of each entity-relation pair in the\nlocal neighborhood of the source entity and the top block aggregates the\nrelational information from the outputs of the bottom block. We further design\na masked entity prediction task to balance information from the relational\ncontext and the source entity itself. Evaluated on the task of link prediction,\nour approach achieves new state-of-the-art results on two standard benchmark\ndatasets FB15K-237 and WN18RR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12854v1",
            "date": "2020-08-28 21:27:42+00:00",
            "title": "TATL at W-NUT 2020 Task 2: A Transformer-based Baseline System for Identification of Informative COVID-19 English Tweets",
            "authors": [
                "Anh Tuan Nguyen"
            ],
            "abstract": "As the COVID-19 outbreak continues to spread throughout the world, more and\nmore information about the pandemic has been shared publicly on social media.\nFor example, there are a huge number of COVID-19 English Tweets daily on\nTwitter. However, the majority of those Tweets are uninformative, and hence it\nis important to be able to automatically select only the informative ones for\ndownstream applications. In this short paper, we present our participation in\nthe W-NUT 2020 Shared Task 2: Identification of Informative COVID-19 English\nTweets. Inspired by the recent advances in pretrained Transformer language\nmodels, we propose a simple yet effective baseline for the task. Despite its\nsimplicity, our proposed approach shows very competitive results in the\nleaderboard as we ranked 8 over 56 teams participated in total.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.12914v1",
            "date": "2020-08-29 05:32:32+00:00",
            "title": "Data augmentation using prosody and false starts to recognize non-native children's speech",
            "authors": [
                "Hemant Kathania",
                "Mittul Singh",
                "Tam\u00e1s Gr\u00f3sz",
                "Mikko Kurimo"
            ],
            "abstract": "This paper describes AaltoASR's speech recognition system for the INTERSPEECH\n2020 shared task on Automatic Speech Recognition (ASR) for non-native\nchildren's speech. The task is to recognize non-native speech from children of\nvarious age groups given a limited amount of speech. Moreover, the speech being\nspontaneous has false starts transcribed as partial words, which in the test\ntranscriptions leads to unseen partial words. To cope with these two\nchallenges, we investigate a data augmentation-based approach. Firstly, we\napply the prosody-based data augmentation to supplement the audio data.\nSecondly, we simulate false starts by introducing partial-word noise in the\nlanguage modeling corpora creating new words. Acoustic models trained on\nprosody-based augmented data outperform the models using the baseline recipe or\nthe SpecAugment-based augmentation. The partial-word noise also helps to\nimprove the baseline language model. Our ASR system, a combination of these\nschemes, is placed third in the evaluation period and achieves the word error\nrate of 18.71%. Post-evaluation period, we observe that increasing the amounts\nof prosody-based augmented data leads to better performance. Furthermore,\nremoving low-confidence-score words from hypotheses can lead to further gains.\nThese two improvements lower the ASR error rate to 17.99%.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.13012v1",
            "date": "2020-08-29 16:55:29+00:00",
            "title": "SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda using Sentence-Level Emotional Salience Features",
            "authors": [
                "Gangeshwar Krishnamurthy",
                "Raj Kumar Gupta",
                "Yinping Yang"
            ],
            "abstract": "This paper describes a system developed for detecting propaganda techniques\nfrom news articles. We focus on examining how emotional salience features\nextracted from a news segment can help to characterize and predict the presence\nof propaganda techniques. Correlation analyses surfaced interesting patterns\nthat, for instance, the \"loaded language\" and \"slogan\" techniques are\nnegatively associated with valence and joy intensity but are positively\nassociated with anger, fear and sadness intensity. In contrast, \"flag waving\"\nand \"appeal to fear-prejudice\" have the exact opposite pattern. Through\npredictive experiments, results further indicate that whereas BERT-only\nfeatures obtained F1-score of 0.548, emotion intensity features and BERT hybrid\nfeatures were able to obtain F1-score of 0.570, when a simple feedforward\nnetwork was used as the classifier in both settings. On gold test data, our\nsystem obtained micro-averaged F1-score of 0.558 on overall detection efficacy\nover fourteen propaganda techniques. It performed relatively well in detecting\n\"loaded language\" (F1 = 0.772), \"name calling and labeling\" (F1 = 0.673),\n\"doubt\" (F1 = 0.604) and \"flag waving\" (F1 = 0.543).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13093v3",
            "date": "2020-08-30 05:17:31+00:00",
            "title": "Parallel Rescoring with Transformer for Streaming On-Device Speech Recognition",
            "authors": [
                "Wei Li",
                "James Qin",
                "Chung-Cheng Chiu",
                "Ruoming Pang",
                "Yanzhang He"
            ],
            "abstract": "Recent advances of end-to-end models have outperformed conventional models\nthrough employing a two-pass model. The two-pass model provides better\nspeed-quality trade-offs for on-device speech recognition, where a 1st-pass\nmodel generates hypotheses in a streaming fashion, and a 2nd-pass model\nre-scores the hypotheses with full audio sequence context. The 2nd-pass model\nplays a key role in the quality improvement of the end-to-end model to surpass\nthe conventional model. One main challenge of the two-pass model is the\ncomputation latency introduced by the 2nd-pass model. Specifically, the\noriginal design of the two-pass model uses LSTMs for the 2nd-pass model, which\nare subject to long latency as they are constrained by the recurrent nature and\nhave to run inference sequentially. In this work we explore replacing the LSTM\nlayers in the 2nd-pass rescorer with Transformer layers, which can process the\nentire hypothesis sequences in parallel and can therefore utilize the on-device\ncomputation resources more efficiently. Compared with an LSTM-based baseline,\nour proposed Transformer rescorer achieves more than 50% latency reduction with\nquality improvement.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2008.13173v1",
            "date": "2020-08-30 13:52:24+00:00",
            "title": "LIMSI_UPV at SemEval-2020 Task 9: Recurrent Convolutional Neural Network for Code-mixed Sentiment Analysis",
            "authors": [
                "Somnath Banerjee",
                "Sahar Ghannay",
                "Sophie Rosset",
                "Anne Vilnat",
                "Paolo Rosso"
            ],
            "abstract": "This paper describes the participation of LIMSI UPV team in SemEval-2020 Task\n9: Sentiment Analysis for Code-Mixed Social Media Text. The proposed approach\ncompeted in SentiMix Hindi-English subtask, that addresses the problem of\npredicting the sentiment of a given Hindi-English code-mixed tweet. We propose\nRecurrent Convolutional Neural Network that combines both the recurrent neural\nnetwork and the convolutional network to better capture the semantics of the\ntext, for code-mixed sentiment analysis. The proposed system obtained 0.69\n(best run) in terms of F1 score on the given test data and achieved the 9th\nplace (Codalab username: somban) in the SentiMix Hindi-English subtask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13549v1",
            "date": "2020-08-09 00:46:26+00:00",
            "title": "C1 at SemEval-2020 Task 9: SentiMix: Sentiment Analysis for Code-Mixed Social Media Text using Feature Engineering",
            "authors": [
                "Laksh Advani",
                "Clement Lu",
                "Suraj Maharjan"
            ],
            "abstract": "In today's interconnected and multilingual world, code-mixing of languages on\nsocial media is a common occurrence. While many Natural Language Processing\n(NLP) tasks like sentiment analysis are mature and well designed for\nmonolingual text, techniques to apply these tasks to code-mixed text still\nwarrant exploration. This paper describes our feature engineering approach to\nsentiment analysis in code-mixed social media text for SemEval-2020 Task 9:\nSentiMix. We tackle this problem by leveraging a set of hand-engineered\nlexical, sentiment, and metadata features to design a classifier that can\ndisambiguate between \"positive\", \"negative\" and \"neutral\" sentiment. With this\nmodel, we are able to obtain a weighted F1 score of 0.65 for the \"Hinglish\"\ntask and 0.63 for the \"Spanglish\" tasks",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2008.13597v2",
            "date": "2020-08-31 13:39:04+00:00",
            "title": "Classifier Combination Approach for Question Classification for Bengali Question Answering System",
            "authors": [
                "Somnath Banerjee",
                "Sudip Kumar Naskar",
                "Paolo Rosso",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Question classification (QC) is a prime constituent of automated question\nanswering system. The work presented here demonstrates that the combination of\nmultiple models achieve better classification performance than those obtained\nwith existing individual models for the question classification task in\nBengali. We have exploited state-of-the-art multiple model combination\ntechniques, i.e., ensemble, stacking and voting, to increase QC accuracy.\nLexical, syntactic and semantic features of Bengali questions are used for four\nwell-known classifiers, namely Na\\\"{\\i}ve Bayes, kernel Na\\\"{\\i}ve Bayes, Rule\nInduction, and Decision Tree, which serve as our base learners. Single-layer\nquestion-class taxonomy with 8 coarse-grained classes is extended to two-layer\ntaxonomy by adding 69 fine-grained classes. We carried out the experiments both\non single-layer and two-layer taxonomies. Experimental results confirmed that\nclassifier combination approaches outperform single classifier classification\napproaches by 4.02% for coarse-grained question classes. Overall, the stacking\napproach produces the best results for fine-grained classification and achieves\n87.79% of accuracy. The approach presented here could be used in other\nIndo-Aryan or Indic languages to develop a question answering system.",
            "journal ref": "Sadhana, Springer, 2019",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00106v1",
            "date": "2020-08-31 21:15:28+00:00",
            "title": "PNEL: Pointer Network based End-To-End Entity Linking over Knowledge Graphs",
            "authors": [
                "Debayan Banerjee",
                "Debanjan Chaudhuri",
                "Mohnish Dubey",
                "Jens Lehmann"
            ],
            "abstract": "Question Answering systems are generally modelled as a pipeline consisting of\na sequence of steps. In such a pipeline, Entity Linking (EL) is often the first\nstep. Several EL models first perform span detection and then entity\ndisambiguation. In such models errors from the span detection phase cascade to\nlater steps and result in a drop of overall accuracy. Moreover, lack of gold\nentity spans in training data is a limiting factor for span detector training.\nHence the movement towards end-to-end EL models began where no separate span\ndetection step is involved. In this work we present a novel approach to\nend-to-end EL by applying the popular Pointer Network model, which achieves\ncompetitive performance. We demonstrate this in our evaluation over three\ndatasets on the Wikidata Knowledge Graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00234v1",
            "date": "2020-08-06 11:59:00+00:00",
            "title": "Semantic Sentiment Analysis Based on Probabilistic Graphical Models and Recurrent Neural Network",
            "authors": [
                "Ukachi Osisiogu"
            ],
            "abstract": "Sentiment Analysis is the task of classifying documents based on the\nsentiments expressed in textual form, this can be achieved by using lexical and\nsemantic methods. The purpose of this study is to investigate the use of\nsemantics to perform sentiment analysis based on probabilistic graphical models\nand recurrent neural networks. In the empirical evaluation, the classification\nperformance of the graphical models was compared with some traditional machine\nlearning classifiers and a recurrent neural network. The datasets used for the\nexperiments were IMDB movie reviews, Amazon Consumer Product reviews, and\nTwitter Review datasets. After this empirical study, we conclude that the\ninclusion of semantics for sentiment analysis tasks can greatly improve the\nperformance of a classifier, as the semantic feature extraction methods reduce\nuncertainties in classification resulting in more accurate predictions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00429v1",
            "date": "2020-09-01 13:58:51+00:00",
            "title": "Hearings and mishearings: decrypting the spoken word",
            "authors": [
                "Anita Mehta",
                "Jean-Marc Luck"
            ],
            "abstract": "We propose a model of the speech perception of individual words in the\npresence of mishearings. This phenomenological approach is based on concepts\nused in linguistics, and provides a formalism that is universal across\nlanguages. We put forward an efficient two-parameter form for the word length\ndistribution, and introduce a simple representation of mishearings, which we\nuse in our subsequent modelling of word recognition. In a context-free\nscenario, word recognition often occurs via anticipation when, part-way into a\nword, we can correctly guess its full form. We give a quantitative estimate of\nthis anticipation threshold when no mishearings occur, in terms of model\nparameters. As might be expected, the whole anticipation effect disappears when\nthere are sufficiently many mishearings. Our global approach to the problem of\nspeech perception is in the spirit of an optimisation problem. We show for\ninstance that speech perception is easy when the word length is less than a\nthreshold, to be identified with a static transition, and hard otherwise. We\nextend this to the dynamics of word recognition, proposing an intuitive\napproach highlighting the distinction between individual, isolated mishearings\nand clusters of contiguous mishearings. At least in some parameter range, a\ndynamical transition is manifest well before the static transition is reached,\nas is the case for many other examples of complex systems.",
            "journal ref": "Adv. Complex Systems 23, 2050008 (2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00596v2",
            "date": "2020-09-01 17:37:39+00:00",
            "title": "Twitter Corpus of the #BlackLivesMatter Movement And Counter Protests: 2013 to 2020",
            "authors": [
                "Salvatore Giorgi",
                "Sharath Chandra Guntuku",
                "Muhammad Rahman",
                "McKenzie Himelein-Wachowiak",
                "Amy Kwarteng",
                "Brenda Curtis"
            ],
            "abstract": "Black Lives Matter (BLM) is a grassroots movement protesting violence towards\nBlack individuals and communities with a focus on police brutality. The\nmovement has gained significant media and political attention following the\nkillings of Ahmaud Arbery, Breonna Taylor, and George Floyd and the shooting of\nJacob Blake in 2020. Due to its decentralized nature, the #BlackLivesMatter\nsocial media hashtag has come to both represent the movement and been used as a\ncall to action. Similar hashtags have appeared to counter the BLM movement,\nsuch as #AllLivesMatter and #BlueLivesMatter. We introduce a data set of 41.8\nmillion tweets from 10 million users which contain one of the following\nkeywords: BlackLivesMatter, AllLivesMatter and BlueLivesMatter. This data set\ncontains all currently available tweets from the beginning of the BLM movement\nin 2013 to June 2020. We summarize the data set and show temporal trends in use\nof both the BlackLivesMatter keyword and keywords associated with counter\nmovements. In the past, similarly themed, though much smaller in scope, BLM\ndata sets have been used for studying discourse in protest and counter protest\nmovements, predicting retweets, examining the role of social media in protest\nmovements and exploring narrative agency. This paper open-sources a large-scale\ndata set to facilitate research in the areas of computational social science,\ncommunications, political science, natural language processing, and machine\nlearning.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2009.00672v1",
            "date": "2020-09-01 19:28:51+00:00",
            "title": "Document Similarity from Vector Space Densities",
            "authors": [
                "Ilia Rushkin"
            ],
            "abstract": "We propose a computationally light method for estimating similarities between\ntext documents, which we call the density similarity (DS) method. The method is\nbased on a word embedding in a high-dimensional Euclidean space and on kernel\nregression, and takes into account semantic relations among words. We find that\nthe accuracy of this method is virtually the same as that of a state-of-the-art\nmethod, while the gain in speed is very substantial. Additionally, we introduce\ngeneralized versions of the top-k accuracy metric and of the Jaccard metric of\nagreement between similarity models.",
            "journal ref": "In: Arai K., Kapoor S., Bhatia R. (eds) Intelligent Systems and\n  Applications. IntelliSys 2020. Advances in Intelligent Systems and Computing,\n  vol 1251. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00751v2",
            "date": "2020-09-01 23:45:42+00:00",
            "title": "Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models",
            "authors": [
                "Tushar Khot",
                "Daniel Khashabi",
                "Kyle Richardson",
                "Peter Clark",
                "Ashish Sabharwal"
            ],
            "abstract": "We propose a general framework called Text Modular Networks(TMNs) for\nbuilding interpretable systems that learn to solve complex tasks by decomposing\nthem into simpler ones solvable by existing models. To ensure solvability of\nsimpler tasks, TMNs learn the textual input-output behavior (i.e., language) of\nexisting models through their datasets. This differs from prior\ndecomposition-based approaches which, besides being designed specifically for\neach complex task, produce decompositions independent of existing sub-models.\nSpecifically, we focus on Question Answering (QA) and show how to train a\nnext-question generator to sequentially produce sub-questions targeting\nappropriate sub-models, without additional human annotation. These\nsub-questions and answers provide a faithful natural language explanation of\nthe model's reasoning. We use this framework to build ModularQA, a system that\ncan answer multi-hop reasoning questions by decomposing them into sub-questions\nanswerable by a neural factoid single-span QA model and a symbolic calculator.\nOur experiments show that ModularQA is more versatile than existing explainable\nsystems for DROP and HotpotQA datasets, is more robust than state-of-the-art\nblackbox (uninterpretable) systems, and generates more understandable and\ntrustworthy explanations compared to prior work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00829v2",
            "date": "2020-09-02 05:37:03+00:00",
            "title": "Automated Storytelling via Causal, Commonsense Plot Ordering",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Wesley Cheung",
                "William Broniec",
                "Mark O. Riedl"
            ],
            "abstract": "Automated story plot generation is the task of generating a coherent sequence\nof plot events. Causal relations between plot events are believed to increase\nthe perception of story and plot coherence. In this work, we introduce the\nconcept of soft causal relations as causal relations inferred from commonsense\nreasoning. We demonstrate C2PO, an approach to narrative generation that\noperationalizes this concept through Causal, Commonsense Plot Ordering. Using\nhuman-participant protocols, we evaluate our system against baseline systems\nwith different commonsense reasoning reasoning and inductive biases to\ndetermine the role of soft causal relations in perceived story quality. Through\nthese studies we also probe the interplay of how changes in commonsense norms\nacross storytelling genres affect perceptions of story quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00901v2",
            "date": "2020-09-02 08:41:46+00:00",
            "title": "A Practical Chinese Dependency Parser Based on A Large-scale Dataset",
            "authors": [
                "Shuai Zhang",
                "Lijie Wang",
                "Ke Sun",
                "Xinyan Xiao"
            ],
            "abstract": "Dependency parsing is a longstanding natural language processing task, with\nits outputs crucial to various downstream tasks. Recently, neural network based\n(NN-based) dependency parsing has achieved significant progress and obtained\nthe state-of-the-art results. As we all know, NN-based approaches require\nmassive amounts of labeled training data, which is very expensive because it\nrequires human annotation by experts. Thus few industrial-oriented dependency\nparser tools are publicly available. In this report, we present Baidu\nDependency Parser (DDParser), a new Chinese dependency parser trained on a\nlarge-scale manually labeled dataset called Baidu Chinese Treebank (DuCTB).\nDuCTB consists of about one million annotated sentences from multiple sources\nincluding search logs, Chinese newswire, various forum discourses, and\nconversation programs. DDParser is extended on the graph-based biaffine parser\nto accommodate to the characteristics of Chinese dataset. We conduct\nexperiments on two test sets: the standard test set with the same distribution\nas the training set and the random test set sampled from other sources, and the\nlabeled attachment scores (LAS) of them are 92.9% and 86.9% respectively.\nDDParser achieves the state-of-the-art results, and is released at\nhttps://github.com/baidu/DDParser.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.00914v1",
            "date": "2020-09-02 09:34:14+00:00",
            "title": "Revisiting the Open-Domain Question Answering Pipeline",
            "authors": [
                "Sina J. Semnani",
                "Manish Pandey"
            ],
            "abstract": "Open-domain question answering (QA) is the tasl of identifying answers to\nnatural questions from a large corpus of documents. The typical open-domain QA\nsystem starts with information retrieval to select a subset of documents from\nthe corpus, which are then processed by a machine reader to select the answer\nspans. This paper describes Mindstone, an open-domain QA system that consists\nof a new multi-stage pipeline that employs a traditional BM25-based information\nretriever, RM3-based neural relevance feedback, neural ranker, and a machine\nreading comprehension stage. This paper establishes a new baseline for\nend-to-end performance on question answering for Wikipedia/SQuAD dataset\n(EM=58.1, F1=65.8), with substantial gains over the previous state of the art\n(Yang et al., 2019b). We also show how the new pipeline enables the use of\nlow-resolution labels, and can be easily tuned to meet various timing\nrequirements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01040v1",
            "date": "2020-09-02 13:15:11+00:00",
            "title": "Defeating Author Gender Identification with Text Style Transfer",
            "authors": [
                "Reza Khan Mohammadi",
                "Seyed Abolghasem Mirroshandel"
            ],
            "abstract": "Text Style Transfer can be named as one of the most important Natural\nLanguage Processing tasks. Up until now, there have been several approaches and\nmethods experimented for this purpose. In this work, we introduce PGST, a novel\npolyglot text style transfer approach in gender domain composed of different\nbuilding blocks. If they become fulfilled with required elements, our method\ncan be applied in multiple languages. We have proceeded with a pre-trained word\nembedding for token replacement purposes, a character-based token classifier\nfor gender exchange purposes, and the beam search algorithm for extracting the\nmost fluent combination among all suggestions. Since different approaches are\nintroduced in our research, we determine a trade-off value for evaluating\ndifferent models' success in faking our gender identification model with\ntransferred text. To demonstrate our method's multilingual applicability, we\napplied our method on both English and Persian corpora and finally ended up\ndefeating our proposed gender identification model by 45.6% and 39.2%,\nrespectively, and obtained highly competitive evaluation results in an analogy\namong English state of the art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01041v1",
            "date": "2020-09-02 13:15:25+00:00",
            "title": "ASTRAL: Adversarial Trained LSTM-CNN for Named Entity Recognition",
            "authors": [
                "Jiuniu Wang",
                "Wenjia Xu",
                "Xingyu Fu",
                "Guangluan Xu",
                "Yirong Wu"
            ],
            "abstract": "Named Entity Recognition (NER) is a challenging task that extracts named\nentities from unstructured text data, including news, articles, social\ncomments, etc. The NER system has been studied for decades. Recently, the\ndevelopment of Deep Neural Networks and the progress of pre-trained word\nembedding have become a driving force for NER. Under such circumstances, how to\nmake full use of the information extracted by word embedding requires more\nin-depth research. In this paper, we propose an Adversarial Trained LSTM-CNN\n(ASTRAL) system to improve the current NER method from both the model structure\nand the training process. In order to make use of the spatial information\nbetween adjacent words, Gated-CNN is introduced to fuse the information of\nadjacent words. Besides, a specific Adversarial training method is proposed to\ndeal with the overfitting problem in NER. We add perturbation to variables in\nthe network during the training process, making the variables more diverse,\nimproving the generalization and robustness of the model. Our model is\nevaluated on three benchmarks, CoNLL-03, OntoNotes 5.0, and WNUT-17, achieving\nstate-of-the-art results. Ablation study and case study also show that our\nsystem can converge faster and is less prone to overfitting.",
            "journal ref": "Knowledge-Based Systems, Volume 197, 7 June 2020, 105842",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01188v2",
            "date": "2020-08-19 21:58:58+00:00",
            "title": "A Stance Data Set on Polarized Conversations on Twitter about the Efficacy of Hydroxychloroquine as a Treatment for COVID-19",
            "authors": [
                "Ece \u00c7i\u011fdem Mutlu",
                "Toktam A. Oghaz",
                "Jasser Jasser",
                "Ege T\u00fct\u00fcnc\u00fcler",
                "Amirarsalan Rajabi",
                "Aida Tayebi",
                "Ozlem Ozmen",
                "Ivan Garibay"
            ],
            "abstract": "At the time of this study, the SARS-CoV-2 virus that caused the COVID-19\npandemic has spread significantly across the world. Considering the uncertainty\nabout policies, health risks, financial difficulties, etc. the online media,\nspecially the Twitter platform, is experiencing a high volume of activity\nrelated to this pandemic. Among the hot topics, the polarized debates about\nunconfirmed medicines for the treatment and prevention of the disease have\nattracted significant attention from online media users. In this work, we\npresent a stance data set, COVID-CQ, of user-generated content on Twitter in\nthe context of COVID-19. We investigated more than 14 thousand tweets and\nmanually annotated the opinions of the tweet initiators regarding the use of\n\"chloroquine\" and \"hydroxychloroquine\" for the treatment or prevention of\nCOVID-19. To the best of our knowledge, COVID-CQ is the first data set of\nTwitter users' stances in the context of the COVID-19 pandemic, and the largest\nTwitter data set on users' stances towards a claim, in any domain. We have made\nthis data set available to the research community via GitHub. We expect this\ndata set to be useful for many research purposes, including stance detection,\nevolution and dynamics of opinions regarding this outbreak, and changes in\nopinions in response to the exogenous shocks such as policy decisions and\nevents.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2009.01630v1",
            "date": "2020-09-02 13:16:24+00:00",
            "title": "SRQA: Synthetic Reader for Factoid Question Answering",
            "authors": [
                "Jiuniu Wang",
                "Wenjia Xu",
                "Xingyu Fu",
                "Yang Wei",
                "Li Jin",
                "Ziyan Chen",
                "Guangluan Xu",
                "Yirong Wu"
            ],
            "abstract": "The question answering system can answer questions from various fields and\nforms with deep neural networks, but it still lacks effective ways when facing\nmultiple evidences. We introduce a new model called SRQA, which means Synthetic\nReader for Factoid Question Answering. This model enhances the question\nanswering system in the multi-document scenario from three aspects: model\nstructure, optimization goal, and training method, corresponding to Multilayer\nAttention (MA), Cross Evidence (CE), and Adversarial Training (AT)\nrespectively. First, we propose a multilayer attention network to obtain a\nbetter representation of the evidences. The multilayer attention mechanism\nconducts interaction between the question and the passage within each layer,\nmaking the token representation of evidences in each layer takes the\nrequirement of the question into account. Second, we design a cross evidence\nstrategy to choose the answer span within more evidences. We improve the\noptimization goal, considering all the answers' locations in multiple evidences\nas training targets, which leads the model to reason among multiple evidences.\nThird, adversarial training is employed to high-level variables besides the\nword embedding in our model. A new normalization method is also proposed for\nadversarial perturbations so that we can jointly add perturbations to several\ntarget variables. As an effective regularization method, adversarial training\nenhances the model's ability to process noisy data. Combining these three\nstrategies, we enhance the contextual representation and locating ability of\nour model, which could synthetically extract the answer span from several\nevidences. We perform SRQA on the WebQA dataset, and experiments show that our\nmodel outperforms the state-of-the-art models (the best fuzzy score of our\nmodel is up to 78.56%, with an improvement of about 2%).",
            "journal ref": "Knowledge-Based Systems, Volume 193, 6 April 2020, 105415",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01719v4",
            "date": "2020-09-03 14:52:03+00:00",
            "title": "Grounded Language Learning Fast and Slow",
            "authors": [
                "Felix Hill",
                "Olivier Tieleman",
                "Tamara von Glehn",
                "Nathaniel Wong",
                "Hamza Merzic",
                "Stephen Clark"
            ],
            "abstract": "Recent work has shown that large text-based neural language models, trained\nwith conventional supervised learning objectives, acquire a surprising\npropensity for few- and one-shot learning. Here, we show that an embodied agent\nsituated in a simulated 3D world, and endowed with a novel dual-coding external\nmemory, can exhibit similar one-shot word learning when trained with\nconventional reinforcement learning algorithms. After a single introduction to\na novel object via continuous visual perception and a language prompt (\"This is\na dax\"), the agent can re-identify the object and manipulate it as instructed\n(\"Put the dax on the bed\"). In doing so, it seamlessly integrates short-term,\nwithin-episode knowledge of the appropriate referent for the word \"dax\" with\nlong-term lexical and motor knowledge acquired across episodes (i.e. \"bed\" and\n\"putting\"). We find that, under certain training conditions and with a\nparticular memory writing mechanism, the agent's one-shot word-object binding\ngeneralizes to novel exemplars within the same ShapeNet category, and is\neffective in settings with unfamiliar numbers of objects. We further show how\ndual-coding memory can be exploited as a signal for intrinsic motivation,\nstimulating the agent to seek names for objects that may be useful for later\nexecuting instructions. Together, the results demonstrate that deep neural\nnetworks can exploit meta-learning, episodic memory and an explicitly\nmulti-modal environment to account for 'fast-mapping', a fundamental pillar of\nhuman cognitive development and a potentially transformative capacity for\nagents that interact with human users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.01938v1",
            "date": "2020-09-03 21:56:38+00:00",
            "title": "Multi-Perspective Semantic Information Retrieval",
            "authors": [
                "Samarth Rawal",
                "Chitta Baral"
            ],
            "abstract": "Information Retrieval (IR) is the task of obtaining pieces of data (such as\ndocuments or snippets of text) that are relevant to a particular query or need\nfrom a large repository of information. While a combination of traditional\nkeyword- and modern BERT-based approaches have been shown to be effective in\nrecent work, there are often nuances in identifying what information is\n\"relevant\" to a particular query, which can be difficult to properly capture\nusing these systems. This work introduces the concept of a Multi-Perspective IR\nsystem, a novel methodology that combines multiple deep learning and\ntraditional IR models to better predict the relevance of a query-sentence pair,\nalong with a standardized framework for tuning this system. This work is\nevaluated on the BioASQ Biomedical IR + QA challenges.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2009.01989v1",
            "date": "2020-09-04 02:53:20+00:00",
            "title": "A Comprehensive Analysis of Information Leakage in Deep Transfer Learning",
            "authors": [
                "Cen Chen",
                "Bingzhe Wu",
                "Minghui Qiu",
                "Li Wang",
                "Jun Zhou"
            ],
            "abstract": "Transfer learning is widely used for transferring knowledge from a source\ndomain to the target domain where the labeled data is scarce. Recently, deep\ntransfer learning has achieved remarkable progress in various applications.\nHowever, the source and target datasets usually belong to two different\norganizations in many real-world scenarios, potential privacy issues in deep\ntransfer learning are posed. In this study, to thoroughly analyze the potential\nprivacy leakage in deep transfer learning, we first divide previous methods\ninto three categories. Based on that, we demonstrate specific threats that lead\nto unintentional privacy leakage in each category. Additionally, we also\nprovide some solutions to prevent these threats. To the best of our knowledge,\nour study is the first to provide a thorough analysis of the information\nleakage issues in deep transfer learning methods and provide potential\nsolutions to the issue. Extensive experiments on two public datasets and an\nindustry dataset are conducted to show the privacy leakage under different deep\ntransfer learning settings and defense solution effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02016v1",
            "date": "2020-09-04 06:18:24+00:00",
            "title": "Dynamic Context-guided Capsule Network for Multimodal Machine Translation",
            "authors": [
                "Huan Lin",
                "Fandong Meng",
                "Jinsong Su",
                "Yongjing Yin",
                "Zhengyuan Yang",
                "Yubin Ge",
                "Jie Zhou",
                "Jiebo Luo"
            ],
            "abstract": "Multimodal machine translation (MMT), which mainly focuses on enhancing\ntext-only translation with visual features, has attracted considerable\nattention from both computer vision and natural language processing\ncommunities. Most current MMT models resort to attention mechanism, global\ncontext modeling or multimodal joint representation learning to utilize visual\nfeatures. However, the attention mechanism lacks sufficient semantic\ninteractions between modalities while the other two provide fixed visual\ncontext, which is unsuitable for modeling the observed variability when\ngenerating translation. To address the above issues, in this paper, we propose\na novel Dynamic Context-guided Capsule Network (DCCN) for MMT. Specifically, at\neach timestep of decoding, we first employ the conventional source-target\nattention to produce a timestep-specific source-side context vector. Next, DCCN\ntakes this vector as input and uses it to guide the iterative extraction of\nrelated visual features via a context-guided dynamic routing mechanism.\nParticularly, we represent the input image with global and regional visual\nfeatures, we introduce two parallel DCCNs to model multimodal context vectors\nwith visual features at different granularities. Finally, we obtain two\nmultimodal context vectors, which are fused and incorporated into the decoder\nfor the prediction of the target word. Experimental results on the Multi30K\ndataset of English-to-German and English-to-French translation demonstrate the\nsuperiority of DCCN. Our code is available on\nhttps://github.com/DeepLearnXMU/MM-DCCN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02035v1",
            "date": "2020-09-04 07:30:57+00:00",
            "title": "What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS",
            "authors": [
                "Brooke Stephenson",
                "Laurent Besacier",
                "Laurent Girin",
                "Thomas Hueber"
            ],
            "abstract": "In incremental text to speech synthesis (iTTS), the synthesizer produces an\naudio output before it has access to the entire input sentence. In this paper,\nwe study the behavior of a neural sequence-to-sequence TTS system when used in\nan incremental mode, i.e. when generating speech output for token n, the system\nhas access to n + k tokens from the text sequence. We first analyze the impact\nof this incremental policy on the evolution of the encoder representations of\ntoken n for different values of k (the lookahead parameter). The results show\nthat, on average, tokens travel 88% of the way to their full context\nrepresentation with a one-word lookahead and 94% after 2 words. We then\ninvestigate which text features are the most influential on the evolution\ntowards the final representation using a random forest analysis. The results\nshow that the most salient factors are related to token length. We finally\nevaluate the effects of lookahead k at the decoder level, using a MUSHRA\nlistening test. This test shows results that contrast with the above high\nfigures: speech synthesis quality obtained with 2 word-lookahead is\nsignificantly lower than the one obtained with the full sentence.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2009.02113v1",
            "date": "2020-09-04 11:17:46+00:00",
            "title": "Going Beyond T-SNE: Exposing \\texttt{whatlies} in Text Embeddings",
            "authors": [
                "Vincent D. Warmerdam",
                "Thomas Kober",
                "Rachael Tatman"
            ],
            "abstract": "We introduce whatlies, an open source toolkit for visually inspecting word\nand sentence embeddings. The project offers a unified and extensible API with\ncurrent support for a range of popular embedding backends including spaCy,\ntfhub, huggingface transformers, gensim, fastText and BytePair embeddings. The\npackage combines a domain specific language for vector arithmetic with\nvisualisation tools that make exploring word embeddings more intuitive and\nconcise. It offers support for many popular dimensionality reduction techniques\nas well as many interactive visualisations that can either be statically\nexported or shared via Jupyter notebooks. The project documentation is\navailable from https://rasahq.github.io/whatlies/.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02358v1",
            "date": "2020-09-04 18:50:13+00:00",
            "title": "Recent Trends in the Use of Deep Learning Models for Grammar Error Handling",
            "authors": [
                "Mina Naghshnejad",
                "Tarun Joshi",
                "Vijayan N. Nair"
            ],
            "abstract": "Grammar error handling (GEH) is an important topic in natural language\nprocessing (NLP). GEH includes both grammar error detection and grammar error\ncorrection. Recent advances in computation systems have promoted the use of\ndeep learning (DL) models for NLP problems such as GEH. In this survey we focus\non two main DL approaches for GEH: neural machine translation models and editor\nmodels. We describe the three main stages of the pipeline for these models:\ndata preparation, training, and inference. Additionally, we discuss different\ntechniques to improve the performance of these models at each stage of the\npipeline. We compare the performance of different models and conclude with\nproposed future directions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02459v2",
            "date": "2020-09-05 04:44:15+00:00",
            "title": "Bio-inspired Structure Identification in Language Embeddings",
            "authors": [
                "Hongwei",
                "Zhou",
                "Oskar Elek",
                "Pranav Anand",
                "Angus G. Forbes"
            ],
            "abstract": "Word embeddings are a popular way to improve downstream performances in\ncontemporary language modeling. However, the underlying geometric structure of\nthe embedding space is not well understood. We present a series of explorations\nusing bio-inspired methodology to traverse and visualize word embeddings,\ndemonstrating evidence of discernible structure. Moreover, our model also\nproduces word similarity rankings that are plausible yet very different from\ncommon similarity metrics, mainly cosine similarity and Euclidean distance. We\nshow that our bio-inspired model can be used to investigate how different word\nembedding techniques result in different semantic outputs, which can emphasize\nor obscure particular interpretations in textual data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02461v1",
            "date": "2020-09-05 04:50:27+00:00",
            "title": "Profiling US Restaurants from Billions of Payment Card Transactions",
            "authors": [
                "Himel Dev",
                "Hossein Hamooni"
            ],
            "abstract": "A payment card (such as debit or credit) is one of the most convenient\npayment methods for purchasing goods and services. Hundreds of millions of card\ntransactions take place across the globe every day, generating a massive volume\nof transaction data. The data render a holistic view of cardholder-merchant\ninteractions, containing insights that can benefit various applications, such\nas payment fraud detection and merchant recommendation. However, utilizing\nthese insights often requires additional information about merchants missing\nfrom the data owner's (i.e., payment company's) perspective. For example,\npayment companies do not know the exact type of product a merchant serves.\nCollecting merchant attributes from external sources for commercial purposes\ncan be expensive. Motivated by this limitation, we aim to infer latent merchant\nattributes from transaction data. As proof of concept, we concentrate on\nrestaurants and infer the cuisine types of restaurants from transactions. To\nthis end, we present a framework for inferring the cuisine types of restaurants\nfrom transaction data. Our proposed framework consists of three steps. In the\nfirst step, we generate cuisine labels for a limited number of restaurants via\nweak supervision. In the second step, we extract a wide variety of statistical\nfeatures and neural embeddings from the restaurant transactions. In the third\nstep, we use deep neural networks (DNNs) to infer the remaining restaurants'\ncuisine types. The proposed framework achieved a 76.2% accuracy in classifying\nthe US restaurants. To the best of our knowledge, this is the first framework\nto infer the cuisine types of restaurants by analyzing transaction data as the\nonly source.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2009.02554v1",
            "date": "2020-09-05 15:40:51+00:00",
            "title": "Visually Analyzing Contextualized Embeddings",
            "authors": [
                "Matthew Berger"
            ],
            "abstract": "In this paper we introduce a method for visually analyzing contextualized\nembeddings produced by deep neural network-based language models. Our approach\nis inspired by linguistic probes for natural language processing, where tasks\nare designed to probe language models for linguistic structure, such as\nparts-of-speech and named entities. These approaches are largely confirmatory,\nhowever, only enabling a user to test for information known a priori. In this\nwork, we eschew supervised probing tasks, and advocate for unsupervised probes,\ncoupled with visual exploration techniques, to assess what is learned by\nlanguage models. Specifically, we cluster contextualized embeddings produced\nfrom a large text corpus, and introduce a visualization design based on this\nclustering and textual structure - cluster co-occurrences, cluster spans, and\ncluster-word membership - to help elicit the functionality of, and relationship\nbetween, individual clusters. User feedback highlights the benefits of our\ndesign in discovering different types of linguistic structures.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2009.02645v1",
            "date": "2020-09-06 05:12:50+00:00",
            "title": "QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation system based on ensemble of language model",
            "authors": [
                "Pai Liu"
            ],
            "abstract": "In this paper, we present language model system submitted to SemEval-2020\nTask 4 competition: \"Commonsense Validation and Explanation\". We participate in\ntwo subtasks for subtask A: validation and subtask B: Explanation. We\nimplemented with transfer learning using pretrained language models (BERT,\nXLNet, RoBERTa, and ALBERT) and fine-tune them on this task. Then we compared\ntheir characteristics in this task to help future researchers understand and\nuse these models more properly. The ensembled model better solves this problem,\nmaking the model's accuracy reached 95.9% on subtask A, which just worse than\nhuman's by only 3% accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02671v2",
            "date": "2020-09-06 08:24:55+00:00",
            "title": "BANANA at WNUT-2020 Task 2: Identifying COVID-19 Information on Twitter by Combining Deep Learning and Transfer Learning Models",
            "authors": [
                "Tin Van Huynh",
                "Luan Thanh Nguyen",
                "Son T. Luu"
            ],
            "abstract": "The outbreak COVID-19 virus caused a significant impact on the health of\npeople all over the world. Therefore, it is essential to have a piece of\nconstant and accurate information about the disease with everyone. This paper\ndescribes our prediction system for WNUT-2020 Task 2: Identification of\nInformative COVID-19 English Tweets. The dataset for this task contains size\n10,000 tweets in English labeled by humans. The ensemble model from our three\ntransformer and deep learning models is used for the final prediction. The\nexperimental result indicates that we have achieved F1 for the INFORMATIVE\nlabel on our systems at 88.81% on the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02696v1",
            "date": "2020-09-06 10:05:43+00:00",
            "title": "SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles",
            "authors": [
                "G. Da San Martino",
                "A. Barr\u00f3n-Cede\u00f1o",
                "H. Wachsmuth",
                "R. Petrov",
                "P. Nakov"
            ],
            "abstract": "We present the results and the main findings of SemEval-2020 Task 11 on\nDetection of Propaganda Techniques in News Articles. The task featured two\nsubtasks. Subtask SI is about Span Identification: given a plain-text document,\nspot the specific text fragments containing propaganda. Subtask TC is about\nTechnique Classification: given a specific text fragment, in the context of a\nfull document, determine the propaganda technique it uses, choosing from an\ninventory of 14 possible propaganda techniques. The task attracted a large\nnumber of participants: 250 teams signed up to participate and 44 made a\nsubmission on the test set. In this paper, we present the task, analyze the\nresults, and discuss the system submissions and the methods they used. For both\nsubtasks, the best systems used pre-trained Transformers and ensembles.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02743v1",
            "date": "2020-09-06 14:20:35+00:00",
            "title": "Romanian Diacritics Restoration Using Recurrent Neural Networks",
            "authors": [
                "Stefan Ruseti",
                "Teodor-Mihai Cotet",
                "Mihai Dascalu"
            ],
            "abstract": "Diacritics restoration is a mandatory step for adequately processing Romanian\ntexts, and not a trivial one, as you generally need context in order to\nproperly restore a character. Most previous methods which were experimented for\nRomanian restoration of diacritics do not use neural networks. Among those that\ndo, there are no solutions specifically optimized for this particular language\n(i.e., they were generally designed to work on many different languages).\nTherefore we propose a novel neural architecture based on recurrent neural\nnetworks that can attend information at different levels of abstractions in\norder to restore diacritics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02963v1",
            "date": "2020-09-07 09:21:34+00:00",
            "title": "TorchKGE: Knowledge Graph Embedding in Python and PyTorch",
            "authors": [
                "Armand Boschin"
            ],
            "abstract": "TorchKGE is a Python module for knowledge graph (KG) embedding relying solely\non PyTorch. This package provides researchers and engineers with a clean and\nefficient API to design and test new models. It features a KG data structure,\nsimple model interfaces and modules for negative sampling and model evaluation.\nIts main strength is a very fast evaluation module for the link prediction\ntask, a central application of KG embedding. Various KG embedding models are\nalso already implemented. Special attention has been paid to code efficiency\nand simplicity, documentation and API consistency. It is distributed using PyPI\nunder BSD license. Source code and pointers to documentation and deployment can\nbe found at https://github.com/torchkge-team/torchkge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.02985v2",
            "date": "2020-09-07 10:08:24+00:00",
            "title": "Ambiguity Hierarchy of Regular Infinite Tree Languages",
            "authors": [
                "Alexander Rabinovich",
                "Doron Tiferet"
            ],
            "abstract": "An automaton is unambiguous if for every input it has at most one accepting\ncomputation. An automaton is k-ambiguous (for k>0) if for every input it has at\nmost k accepting computations. An automaton is boundedly ambiguous if there is\nk, such that for every input it has at most k accepting computations. An\nautomaton is finitely (respectively, countably) ambiguous if for every input it\nhas at most finitely (respectively, countably) many accepting computations.\n  The degree of ambiguity of a regular language is defined in a natural way. A\nlanguage is k-ambiguous (respectively, boundedly, finitely, countably\nambiguous) if it is accepted by a k-ambiguous (respectively, boundedly,\nfinitely, countably ambiguous) automaton. Over finite words, every regular\nlanguage is accepted by a deterministic automaton. Over finite trees, every\nregular language is accepted by an unambiguous automaton. Over $\\omega$-words\nevery regular language is accepted by an unambiguous B\\\"uchi automaton and by a\ndeterministic parity automaton. Over infinite trees, Carayol et al. showed that\nthere are ambiguous languages.\n  We show that over infinite trees there is a hierarchy of degrees of\nambiguity: For every k>1 there are k-ambiguous languages which are not k-1\nambiguous; and there are finitely (respectively countably, uncountably)\nambiguous languages which are not boundedly (respectively finitely, countably)\nambiguous.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/2009.03068v1",
            "date": "2020-09-07 12:48:36+00:00",
            "title": "Uncovering the Corona Virus Map Using Deep Entities and Relationship Models",
            "authors": [
                "Kuldeep Singh",
                "Puneet Singla",
                "Ketan Sarode",
                "Anurag Chandrakar",
                "Chetan Nichkawde"
            ],
            "abstract": "We extract entities and relationships related to COVID-19 from a corpus of\narticles related to Corona virus by employing a novel entities and relationship\nmodel. The entity recognition and relationship discovery models are trained\nwith a multi-task learning objective on a large annotated corpus. We employ a\nconcept masking paradigm to prevent the evolution of neural networks\nfunctioning as an associative memory and induce right inductive bias guiding\nthe network to make inference using only the context. We uncover several import\nsubnetworks, highlight important terms and concepts and elucidate several\ntreatment modalities employed in related ailments in the past.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03095v1",
            "date": "2020-09-07 13:32:07+00:00",
            "title": "Robust Spoken Language Understanding with RL-based Value Error Recovery",
            "authors": [
                "Chen Liu",
                "Su Zhu",
                "Lu Chen",
                "Kai Yu"
            ],
            "abstract": "Spoken Language Understanding (SLU) aims to extract structured semantic\nrepresentations (e.g., slot-value pairs) from speech recognized texts, which\nsuffers from errors of Automatic Speech Recognition (ASR). To alleviate the\nproblem caused by ASR-errors, previous works may apply input adaptations to the\nspeech recognized texts, or correct ASR errors in predicted values by searching\nthe most similar candidates in pronunciation. However, these two methods are\napplied separately and independently. In this work, we propose a new robust SLU\nframework to guide the SLU input adaptation with a rule-based value error\nrecovery module. The framework consists of a slot tagging model and a\nrule-based value error recovery module. We pursue on an adapted slot tagging\nmodel which can extract potential slot-value pairs mentioned in ASR hypotheses\nand is suitable for the existing value error recovery module. After the value\nerror recovery, we can achieve a supervision signal (reward) by comparing\nrefined slot-value pairs with annotations. Since operations of the value error\nrecovery are non-differentiable, we exploit policy gradient based Reinforcement\nLearning (RL) to optimize the SLU model. Extensive experiments on the public\nCATSLU dataset show the effectiveness of our proposed approach, which can\nimprove the robustness of SLU and outperform the baselines by significant\nmargins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03397v1",
            "date": "2020-09-07 19:57:09+00:00",
            "title": "NLP-CIC at SemEval-2020 Task 9: Analysing sentiment in code-switching language using a simple deep-learning classifier",
            "authors": [
                "Jason Angel",
                "Segun Taofeek Aroyehun",
                "Antonio Tamayo",
                "Alexander Gelbukh"
            ],
            "abstract": "Code-switching is a phenomenon in which two or more languages are used in the\nsame message. Nowadays, it is quite common to find messages with languages\nmixed in social media. This phenomenon presents a challenge for sentiment\nanalysis. In this paper, we use a standard convolutional neural network model\nto predict the sentiment of tweets in a blend of Spanish and English languages.\nOur simple approach achieved a F1-score of 0.71 on test set on the competition.\nWe analyze our best model capabilities and perform error analysis to expose\nimportant difficulties for classifying sentiment in a code-switching setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03457v1",
            "date": "2020-09-07 23:49:28+00:00",
            "title": "Robust Conversational AI with Grounded Text Generation",
            "authors": [
                "Jianfeng Gao",
                "Baolin Peng",
                "Chunyuan Li",
                "Jinchao Li",
                "Shahin Shayandeh",
                "Lars Liden",
                "Heung-Yeung Shum"
            ],
            "abstract": "This article presents a hybrid approach based on a Grounded Text Generation\n(GTG) model to building robust task bots at scale. GTG is a hybrid model which\nuses a large-scale Transformer neural network as its backbone, combined with\nsymbol-manipulation modules for knowledge base inference and prior knowledge\nencoding, to generate responses grounded in dialog belief state and real-world\nknowledge for task completion. GTG is pre-trained on large amounts of raw text\nand human conversational data, and can be fine-tuned to complete a wide range\nof tasks.\n  The hybrid approach and its variants are being developed simultaneously by\nmultiple research teams. The primary results reported on task-oriented dialog\nbenchmarks are very promising, demonstrating the big potential of this\napproach. This article provides an overview of this progress and discusses\nrelated methods and technologies that can be incorporated for building robust\nconversational AI systems.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.03849v1",
            "date": "2020-09-08 16:29:25+00:00",
            "title": "LynyrdSkynyrd at WNUT-2020 Task 2: Semi-Supervised Learning for Identification of Informative COVID-19 English Tweets",
            "authors": [
                "Abhilasha Sancheti",
                "Kushal Chawla",
                "Gaurav Verma"
            ],
            "abstract": "We describe our system for WNUT-2020 shared task on the identification of\ninformative COVID-19 English tweets. Our system is an ensemble of various\nmachine learning methods, leveraging both traditional feature-based classifiers\nas well as recent advances in pre-trained language models that help in\ncapturing the syntactic, semantic, and contextual features from the tweets. We\nfurther employ pseudo-labelling to incorporate the unlabelled Twitter data\nreleased on the pandemic. Our best performing model achieves an F1-score of\n0.9179 on the provided validation set and 0.8805 on the blind test-set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03947v3",
            "date": "2020-09-08 19:00:38+00:00",
            "title": "Covid-Transformer: Detecting COVID-19 Trending Topics on Twitter Using Universal Sentence Encoder",
            "authors": [
                "Meysam Asgari-Chenaghlu",
                "Narjes Nikzad-Khasmakhi",
                "Shervin Minaee"
            ],
            "abstract": "The novel corona-virus disease (also known as COVID-19) has led to a\npandemic, impacting more than 200 countries across the globe. With its global\nimpact, COVID-19 has become a major concern of people almost everywhere, and\ntherefore there are a large number of tweets coming out from every corner of\nthe world, about COVID-19 related topics. In this work, we try to analyze the\ntweets and detect the trending topics and major concerns of people on Twitter,\nwhich can enable us to better understand the situation, and devise better\nplanning. More specifically we propose a model based on the universal sentence\nencoder to detect the main topics of Tweets in recent months. We used universal\nsentence encoder in order to derive the semantic representation and the\nsimilarity of tweets. We then used the sentence similarity and their\nembeddings, and feed them to K-means clustering algorithm to group similar\ntweets (in semantic sense). After that, the cluster summary is obtained using a\ntext summarization algorithm based on deep learning, which can uncover the\nunderlying topics of each cluster. Through experimental results, we show that\nour model can detect very informative topics, by processing a large number of\ntweets on sentence level (which can preserve the overall meaning of the\ntweets). Since this framework has no restriction on specific data distribution,\nit can be used to detect trending topics from any other social media and any\nother context rather than COVID-19. Experimental results show superiority of\nour proposed approach to other baselines, including TF-IDF, and latent\nDirichlet allocation (LDA).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.03954v1",
            "date": "2020-09-08 19:12:06+00:00",
            "title": "Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modeling",
            "authors": [
                "Yiding Hao",
                "Simon Mendelsohn",
                "Rachel Sterneck",
                "Randi Martinez",
                "Robert Frank"
            ],
            "abstract": "By positing a relationship between naturalistic reading times and\ninformation-theoretic surprisal, surprisal theory (Hale, 2001; Levy, 2008)\nprovides a natural interface between language models and psycholinguistic\nmodels. This paper re-evaluates a claim due to Goodkind and Bicknell (2018)\nthat a language model's ability to model reading times is a linear function of\nits perplexity. By extending Goodkind and Bicknell's analysis to modern neural\narchitectures, we show that the proposed relation does not always hold for Long\nShort-Term Memory networks, Transformers, and pre-trained models. We introduce\nan alternate measure of language modeling performance called predictability\nnorm correlation based on Cloze probabilities measured from human subjects. Our\nnew metric yields a more robust relationship between language model quality and\npsycholinguistic modeling performance that allows for comparison between models\nwith different training configurations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04008v1",
            "date": "2020-09-08 21:59:08+00:00",
            "title": "Quantifying the Effects of COVID-19 on Mental Health Support Forums",
            "authors": [
                "Laura Biester",
                "Katie Matton",
                "Janarthanan Rajendran",
                "Emily Mower Provost",
                "Rada Mihalcea"
            ],
            "abstract": "The COVID-19 pandemic, like many of the disease outbreaks that have preceded\nit, is likely to have a profound effect on mental health. Understanding its\nimpact can inform strategies for mitigating negative consequences. In this\nwork, we seek to better understand the effects of COVID-19 on mental health by\nexamining discussions within mental health support communities on Reddit.\nFirst, we quantify the rate at which COVID-19 is discussed in each community,\nor subreddit, in order to understand levels of preoccupation with the pandemic.\nNext, we examine the volume of activity in order to determine whether the\nquantity of people seeking online mental health support has risen. Finally, we\nanalyze how COVID-19 has influenced language use and topics of discussion\nwithin each subreddit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04530v1",
            "date": "2020-09-09 19:31:04+00:00",
            "title": "Discovering Textual Structures: Generative Grammar Induction using Template Trees",
            "authors": [
                "Thomas Winters",
                "Luc De Raedt"
            ],
            "abstract": "Natural language generation provides designers with methods for automatically\ngenerating text, e.g. for creating summaries, chatbots and game content. In\npractise, text generators are often either learned and hard to interpret, or\ncreated by hand using techniques such as grammars and templates. In this paper,\nwe introduce a novel grammar induction algorithm for learning interpretable\ngrammars for generative purposes, called Gitta. We also introduce the novel\nnotion of template trees to discover latent templates in corpora to derive\nthese generative grammars. By using existing human-created grammars, we found\nthat the algorithm can reasonably approximate these grammars using only a few\nexamples. These results indicate that Gitta could be used to automatically\nlearn interpretable and easily modifiable grammars, and thus provide a stepping\nstone for human-machine co-creation of generative models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04534v3",
            "date": "2020-09-09 19:39:15+00:00",
            "title": "Pay Attention when Required",
            "authors": [
                "Swetha Mandava",
                "Szymon Migacz",
                "Alex Fit Florea"
            ],
            "abstract": "Transformer-based models consist of interleaved feed-forward blocks - that\ncapture content meaning, and relatively more expensive self-attention blocks -\nthat capture context meaning. In this paper, we explored trade-offs and\nordering of the blocks to improve upon the current Transformer architecture and\nproposed PAR Transformer. It needs 35% lower compute time than Transformer-XL\nachieved by replacing ~63% of the self-attention blocks with feed-forward\nblocks, and retains the perplexity on WikiText-103 language modelling\nbenchmark. We further validated our results on text8 and enwiki8 datasets, as\nwell as on the BERT model.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2009.04617v1",
            "date": "2020-09-10 00:42:59+00:00",
            "title": "Emora: An Inquisitive Social Chatbot Who Cares For You",
            "authors": [
                "Sarah E. Finch",
                "James D. Finch",
                "Ali Ahmadvand",
                "Ingyu",
                "Choi",
                "Xiangjue Dong",
                "Ruixiang Qi",
                "Harshita Sahijwani",
                "Sergey Volokhin",
                "Zihan Wang",
                "Zihao Wang",
                "Jinho D. Choi"
            ],
            "abstract": "Inspired by studies on the overwhelming presence of experience-sharing in\nhuman-human conversations, Emora, the social chatbot developed by Emory\nUniversity, aims to bring such experience-focused interaction to the current\nfield of conversational AI. The traditional approach of information-sharing\ntopic handlers is balanced with a focus on opinion-oriented exchanges that\nEmora delivers, and new conversational abilities are developed that support\ndialogues that consist of a collaborative understanding and learning process of\nthe partner's life experiences. We present a curated dialogue system that\nleverages highly expressive natural language templates, powerful intent\nclassification, and ontology resources to provide an engaging and interesting\nconversational experience to every user.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04703v2",
            "date": "2020-09-10 07:39:05+00:00",
            "title": "Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection",
            "authors": [
                "Taesun Whang",
                "Dongyub Lee",
                "Dongsuk Oh",
                "Chanhee Lee",
                "Kijong Han",
                "Dong-hun Lee",
                "Saebyeok Lee"
            ],
            "abstract": "In this paper, we study the task of selecting the optimal response given a\nuser and system utterance history in retrieval-based multi-turn dialog systems.\nRecently, pre-trained language models (e.g., BERT, RoBERTa, and ELECTRA) showed\nsignificant improvements in various natural language processing tasks. This and\nsimilar response selection tasks can also be solved using such language models\nby formulating the tasks as dialog--response binary classification tasks.\nAlthough existing works using this approach successfully obtained\nstate-of-the-art results, we observe that language models trained in this\nmanner tend to make predictions based on the relatedness of history and\ncandidates, ignoring the sequential nature of multi-turn dialog systems. This\nsuggests that the response selection task alone is insufficient for learning\ntemporal dependencies between utterances. To this end, we propose utterance\nmanipulation strategies (UMS) to address this problem. Specifically, UMS\nconsist of several strategies (i.e., insertion, deletion, and search), which\naid the response selection model towards maintaining dialog coherence. Further,\nUMS are self-supervised methods that do not require additional annotation and\nthus can be easily incorporated into existing approaches. Extensive evaluation\nacross multiple languages and models shows that UMS are highly effective in\nteaching dialog consistency, which leads to models pushing the state-of-the-art\nwith significant margins on multiple public benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04891v1",
            "date": "2020-09-10 14:36:38+00:00",
            "title": "Meta-Learning with Sparse Experience Replay for Lifelong Language Learning",
            "authors": [
                "Nithin Holla",
                "Pushkar Mishra",
                "Helen Yannakoudakis",
                "Ekaterina Shutova"
            ],
            "abstract": "Lifelong learning requires models that can continuously learn from sequential\nstreams of data without suffering catastrophic forgetting due to shifts in data\ndistributions. Deep learning models have thrived in the non-sequential learning\nparadigm; however, when used to learn a sequence of tasks, they fail to retain\npast knowledge and learn incrementally. We propose a novel approach to lifelong\nlearning of language tasks based on meta-learning with sparse experience replay\nthat directly optimizes to prevent forgetting. We show that under the realistic\nsetting of performing a single pass on a stream of tasks and without any task\nidentifiers, our method obtains state-of-the-art results on lifelong text\nclassification and relation extraction. We analyze the effectiveness of our\napproach and further demonstrate its low computational and space complexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04968v1",
            "date": "2020-09-10 16:17:10+00:00",
            "title": "Modern Methods for Text Generation",
            "authors": [
                "Dimas Munoz Montesinos"
            ],
            "abstract": "Synthetic text generation is challenging and has limited success. Recently, a\nnew architecture, called Transformers, allow machine learning models to\nunderstand better sequential data, such as translation or summarization. BERT\nand GPT-2, using Transformers in their cores, have shown a great performance in\ntasks such as text classification, translation and NLI tasks. In this article,\nwe analyse both algorithms and compare their output quality in text generation\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.04984v1",
            "date": "2020-09-10 16:46:46+00:00",
            "title": "Task-specific Objectives of Pre-trained Language Models for Dialogue Adaptation",
            "authors": [
                "Junlong Li",
                "Zhuosheng Zhang",
                "Hai Zhao",
                "Xi Zhou",
                "Xiang Zhou"
            ],
            "abstract": "Pre-trained Language Models (PrLMs) have been widely used as backbones in\nlots of Natural Language Processing (NLP) tasks. The common process of\nutilizing PrLMs is first pre-training on large-scale general corpora with\ntask-independent LM training objectives, then fine-tuning on task datasets with\ntask-specific training objectives. Pre-training in a task-independent way\nenables the models to learn language representations, which is universal to\nsome extent, but fails to capture crucial task-specific features in the\nmeantime. This will lead to an incompatibility between pre-training and\nfine-tuning. To address this issue, we introduce task-specific pre-training on\nin-domain task-related corpora with task-specific objectives. This procedure is\nplaced between the original two stages to enhance the model understanding\ncapacity of specific tasks. In this work, we focus on Dialogue-related Natural\nLanguage Processing (DrNLP) tasks and design a Dialogue-Adaptive Pre-training\nObjective (DAPO) based on some important qualities for assessing dialogues\nwhich are usually ignored by general LM pre-training objectives. PrLMs with\nDAPO on a large in-domain dialogue corpus are then fine-tuned for downstream\nDrNLP tasks. Experimental results show that models with DAPO surpass those with\ngeneral LM pre-training objectives and other strong baselines on downstream\nDrNLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05019v1",
            "date": "2020-09-10 17:33:16+00:00",
            "title": "Multi-modal embeddings using multi-task learning for emotion recognition",
            "authors": [
                "Aparna Khare",
                "Srinivas Parthasarathy",
                "Shiva Sundaram"
            ],
            "abstract": "General embeddings like word2vec, GloVe and ELMo have shown a lot of success\nin natural language tasks. The embeddings are typically extracted from models\nthat are built on general tasks such as skip-gram models and natural language\ngeneration. In this paper, we extend the work from natural language\nunderstanding to multi-modal architectures that use audio, visual and textual\ninformation for machine learning tasks. The embeddings in our network are\nextracted using the encoder of a transformer model trained using multi-task\ntraining. We use person identification and automatic speech recognition as the\ntasks in our embedding generation framework. We tune and evaluate the\nembeddings on the downstream task of emotion recognition and demonstrate that\non the CMU-MOSEI dataset, the embeddings can be used to improve over previous\nstate of the art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05032v1",
            "date": "2020-09-10 17:53:19+00:00",
            "title": "GeoSPARQL+: Syntax, Semantics and System for Integrated Querying of Graph, Raster and Vector Data -- Technical Report",
            "authors": [
                "Timo Homburg",
                "Steffen Staab",
                "Daniel Janke"
            ],
            "abstract": "We introduce an approach to semantically represent and query raster data in a\nSemantic Web graph. We extend the GeoSPARQL vocabulary and query language to\nsupport raster data as a new type of geospatial data. We define new filter\nfunctions and illustrate our approach using several use cases on real-world\ndata sets. Finally, we describe a prototypical implementation and validate the\nfeasibility of our approach.",
            "journal ref": "None",
            "category": "cs.DB"
        },
        {
            "url": "http://arxiv.org/abs/2009.05121v1",
            "date": "2020-09-10 19:40:41+00:00",
            "title": "Patient Cohort Retrieval using Transformer Language Models",
            "authors": [
                "Sarvesh Soni",
                "Kirk Roberts"
            ],
            "abstract": "We apply deep learning-based language models to the task of patient cohort\nretrieval (CR) with the aim to assess their efficacy. The task of CR requires\nthe extraction of relevant documents from the electronic health records (EHRs)\non the basis of a given query. Given the recent advancements in the field of\ndocument retrieval, we map the task of CR to a document retrieval task and\napply various deep neural models implemented for the general domain tasks. In\nthis paper, we propose a framework for retrieving patient cohorts using neural\nlanguage models without the need of explicit feature engineering and domain\nexpertise. We find that a majority of our models outperform the BM25 baseline\nmethod on various evaluation metrics.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2009.05160v2",
            "date": "2020-09-10 22:18:57+00:00",
            "title": "Rank over Class: The Untapped Potential of Ranking in Natural Language Processing",
            "authors": [
                "Amir Atapour-Abarghouei",
                "Stephen Bonner",
                "Andrew Stephen McGough"
            ],
            "abstract": "Text classification has long been a staple in natural language processing\nwith applications spanning across sentiment analysis, online content tagging,\nrecommender systems and spam detection. However, text classification, by\nnature, suffers from a variety of issues stemming from dataset imbalance, text\nambiguity, subjectivity and the lack of linguistic context in the data. In this\npaper, we explore the use of text ranking, commonly used in information\nretrieval, to carry out challenging classification-based tasks. We propose a\nnovel end-to-end ranking approach consisting of a Transformer network\nresponsible for producing representations for a pair of text sequences, which\nare in turn passed into a context aggregating network outputting ranking scores\nused to determine an ordering to the sequences based on some notion of\nrelevance. We perform numerous experiments on publicly-available datasets and\ninvestigate the possibility of applying our ranking approach to certain\nproblems often addressed using classification. In an experiment on a\nheavily-skewed sentiment analysis dataset, converting ranking results to\nclassification labels yields an approximately 22% improvement over\nstate-of-the-art text classification, demonstrating the efficacy of text\nranking over text classification in certain scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05169v3",
            "date": "2020-09-10 22:49:39+00:00",
            "title": "Sparsifying Transformer Models with Trainable Representation Pooling",
            "authors": [
                "Micha\u0142 Pietruszka",
                "\u0141ukasz Borchmann",
                "\u0141ukasz Garncarek"
            ],
            "abstract": "We propose a novel method to sparsify attention in the Transformer model by\nlearning to select the most-informative token representations during the\ntraining process, thus focusing on task-specific parts of the input. A\nreduction of quadratic time and memory complexity to sublinear was achieved due\nto a robust trainable top-k operator. For example, our experiments on a\nchallenging summarization task of long documents show that our method is over 3\ntimes faster and up to 16 times more memory efficient while significantly\noutperforming both dense and state-of-the-art sparse transformer models. The\nmethod can be effortlessly applied to many models used in NLP and CV,\nsimultaneously with other improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05175v2",
            "date": "2020-09-10 23:31:38+00:00",
            "title": "Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection Models",
            "authors": [
                "Khyathi Raghavi Chandu",
                "Piyush Sharma",
                "Soravit Changpinyo",
                "Ashish Thapliyal",
                "Radu Soricut"
            ],
            "abstract": "Training large-scale image captioning (IC) models demands access to a rich\nand diverse set of training examples, gathered from the wild, often from noisy\nalt-text data. However, recent modeling approaches to IC often fall short in\nterms of performance in this case, because they assume a clean annotated\ndataset (as opposed to the noisier alt-text--based annotations), and employ an\nend-to-end generation approach, which often lacks both controllability and\ninterpretability. We address these problems by breaking down the task into two\nsimpler, more controllable tasks -- skeleton prediction and skeleton-based\ncaption generation. Specifically, we show that selecting content words as\nskeletons} helps in generating improved and denoised captions when leveraging\nrich yet noisy alt-text--based uncurated datasets. We also show that the\npredicted English skeletons can be further cross-lingually leveraged to\ngenerate non-English captions, and present experimental results covering\ncaption generation in French, Italian, German, Spanish and Hindi. We also show\nthat skeleton-based prediction allows for better control of certain caption\nproperties, such as length, content, and gender expression, providing a handle\nto perform human-in-the-loop semi-automatic corrections.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05451v1",
            "date": "2020-09-11 14:01:14+00:00",
            "title": "A Comparison of LSTM and BERT for Small Corpus",
            "authors": [
                "Aysu Ezen-Can"
            ],
            "abstract": "Recent advancements in the NLP field showed that transfer learning helps with\nachieving state-of-the-art results for new tasks by tuning pre-trained models\ninstead of starting from scratch. Transformers have made a significant\nimprovement in creating new state-of-the-art results for many NLP tasks\nincluding but not limited to text classification, text generation, and sequence\nlabeling. Most of these success stories were based on large datasets. In this\npaper we focus on a real-life scenario that scientists in academia and industry\nface frequently: given a small dataset, can we use a large pre-trained model\nlike BERT and get better results than simple models? To answer this question,\nwe use a small dataset for intent classification collected for building\nchatbots and compare the performance of a simple bidirectional LSTM model with\na pre-trained BERT model. Our experimental results show that bidirectional LSTM\nmodels can achieve significantly higher results than a BERT model for a small\ndataset and these simple models get trained in much less time than tuning the\npre-trained counterparts. We conclude that the performance of a model is\ndependent on the task and the data, and therefore before making a model choice,\nthese factors should be taken into consideration instead of directly choosing\nthe most popular model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05456v1",
            "date": "2020-09-11 14:10:03+00:00",
            "title": "WOLI at SemEval-2020 Task 12: Arabic Offensive Language Identification on Different Twitter Datasets",
            "authors": [
                "Yasser Otiefy",
                "Ahmed Abdelmalek",
                "Islam El Hosary"
            ],
            "abstract": "Communicating through social platforms has become one of the principal means\nof personal communications and interactions. Unfortunately, healthy\ncommunication is often interfered by offensive language that can have damaging\neffects on the users. A key to fight offensive language on social media is the\nexistence of an automatic offensive language detection system. This paper\npresents the results and the main findings of SemEval-2020, Task 12 OffensEval\nSub-task A Zampieri et al. (2020), on Identifying and categorising Offensive\nLanguage in Social Media. The task was based on the Arabic OffensEval dataset\nMubarak et al. (2020). In this paper, we describe the system submitted by\nWideBot AI Lab for the shared task which ranked 10th out of 52 participants\nwith Macro-F1 86.9% on the golden dataset under CodaLab username\n\"yasserotiefy\". We experimented with various models and the best model is a\nlinear SVM in which we use a combination of both character and word n-grams. We\nalso introduced a neural network approach that enhanced the predictive ability\nof our system that includes CNN, highway network, Bi-LSTM, and attention\nlayers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05552v2",
            "date": "2020-09-11 17:35:05+00:00",
            "title": "Systematic Generalization on gSCAN with Language Conditioned Embedding",
            "authors": [
                "Tong Gao",
                "Qi Huang",
                "Raymond J. Mooney"
            ],
            "abstract": "Systematic Generalization refers to a learning algorithm's ability to\nextrapolate learned behavior to unseen situations that are distinct but\nsemantically similar to its training data. As shown in recent work,\nstate-of-the-art deep learning models fail dramatically even on tasks for which\nthey are designed when the test set is systematically different from the\ntraining data. We hypothesize that explicitly modeling the relations between\nobjects in their contexts while learning their representations will help\nachieve systematic generalization. Therefore, we propose a novel method that\nlearns objects' contextualized embeddings with dynamic message passing\nconditioned on the input natural language and end-to-end trainable with other\ndownstream deep learning modules. To our knowledge, this model is the first one\nthat significantly outperforms the provided baseline and reaches\nstate-of-the-art performance on grounded-SCAN (gSCAN), a grounded natural\nlanguage navigation dataset designed to require systematic generalization in\nits test splits.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.05664v3",
            "date": "2020-09-11 21:24:33+00:00",
            "title": "Towards an Atlas of Cultural Commonsense for Machine Reasoning",
            "authors": [
                "Anurag Acharya",
                "Kartik Talamadupula",
                "Mark A Finlayson"
            ],
            "abstract": "Existing commonsense reasoning datasets for AI and NLP tasks fail to address\nan important aspect of human life: cultural differences. We introduce an\napproach that extends prior work on crowdsourcing commonsense knowledge by\nincorporating differences in knowledge that are attributable to cultural or\nnational groups. We demonstrate the technique by collecting commonsense\nknowledge that surrounds six fairly universal rituals -- birth, coming-of-age,\nmarriage, funerals, new year, and birthdays -- across two national groups: the\nUnited States and India. Our study expands the different types of relationships\nidentified by existing work in the field of commonsense reasoning for\ncommonplace events, and uses these new types to gather information that\ndistinguish the identity of the groups providing the knowledge. It also moves\nus a step closer towards building a machine that doesn't assume a rigid\nframework of universal (and likely Western-biased) commonsense knowledge, but\nrather has the ability to reason in a contextually and culturally sensitive\nway. Our hope is that cultural knowledge of this sort will lead to more\nhuman-like performance in NLP tasks such as question answering (QA) and text\nunderstanding and generation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.05675v1",
            "date": "2020-09-11 22:21:19+00:00",
            "title": "Coreference Resolution System for Indonesian Text with Mention Pair Method and Singleton Exclusion using Convolutional Neural Network",
            "authors": [
                "Turfa Auliarachman",
                "Ayu Purwarianti"
            ],
            "abstract": "Neural network has shown promising performance on coreference resolution\nsystems that uses mention pair method. With deep neural network, it can learn\nhidden and deep relations between two mentions. However, there is no work on\ncoreference resolution for Indonesian text that uses this learning technique.\nThe state-of-the-art system for Indonesian text only states the use of lexical\nand syntactic features can improve the existing coreference resolution system.\nIn this paper, we propose a new coreference resolution system for Indonesian\ntext with mention pair method that uses deep neural network to learn the\nrelations of the two mentions. In addition to lexical and syntactic features,\nin order to learn the representation of the mentions words and context, we use\nword embeddings and feed them to Convolutional Neural Network (CNN).\nFurthermore, we do singleton exclusion using singleton classifier component to\nprevent singleton mentions entering any entity clusters at the end. Achieving\n67.37% without singleton exclusion, 63.27% with trained singleton classifier,\nand 75.95% with gold singleton classifier on CoNLL average F1 score, our\nproposed system outperforms the state-of-the-art system.",
            "journal ref": "2019 International Conference of Advanced Informatics: Concepts,\n  Theory and Applications (ICAICTA)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05687v1",
            "date": "2020-09-11 23:54:31+00:00",
            "title": "Investigating Bi-LSTM and CRF with POS Tag Embedding for Indonesian Named Entity Tagger",
            "authors": [
                "Devin Hoesen",
                "Ayu Purwarianti"
            ],
            "abstract": "Researches on Indonesian named entity (NE) tagger have been conducted since\nyears ago. However, most did not use deep learning and instead employed\ntraditional machine learning algorithms such as association rule, support\nvector machine, random forest, na\\\"ive bayes, etc. In those researches, word\nlists as gazetteers or clue words were provided to enhance the accuracy. Here,\nwe attempt to employ deep learning in our Indonesian NE tagger. We use long\nshort-term memory (LSTM) as the topology since it is the state-of-the-art of NE\ntagger. By using LSTM, we do not need a word list in order to enhance the\naccuracy. Basically, there are two main things that we investigate. The first\nis the output layer of the network: Softmax vs conditional random field (CRF).\nThe second is the usage of part of speech (POS) tag embedding input layer.\nUsing 8400 sentences as the training data and 97 sentences as the evaluation\ndata, we find that using POS tag embedding as additional input improves the\nperformance of our Indonesian NE tagger. As for the comparison between Softmax\nand CRF, we find that both architectures have a weakness in classifying an NE\ntag.",
            "journal ref": "2018 International Conference on Asian Language Processing (IALP)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05698v1",
            "date": "2020-09-12 01:45:08+00:00",
            "title": "Relation Detection for Indonesian Language using Deep Neural Network -- Support Vector Machine",
            "authors": [
                "Ramos Janoah Hasudungan",
                "Ayu Purwarianti"
            ],
            "abstract": "Relation Detection is a task to determine whether two entities are related or\nnot. In this paper, we employ neural network to do relation detection between\ntwo named entities for Indonesian Language. We used feature such as word\nembedding, position embedding, POS-Tag embedding, and character embedding. For\nthe model, we divide the model into two parts: Front-part classifier\n(Convolutional layer or LSTM layer) and Back-part classifier (Dense layer or\nSVM). We did grid search method of neural network hyper parameter and SVM. We\nused 6000 Indonesian sentences for training process and 1,125 for testing. The\nbest result is 0.8083 on F1-Score using Convolutional Layer as front-part and\nSVM as back-part.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05713v1",
            "date": "2020-09-12 03:16:25+00:00",
            "title": "Improving Indonesian Text Classification Using Multilingual Language Model",
            "authors": [
                "Ilham Firdausi Putra",
                "Ayu Purwarianti"
            ],
            "abstract": "Compared to English, the amount of labeled data for Indonesian text\nclassification tasks is very small. Recently developed multilingual language\nmodels have shown its ability to create multilingual representations\neffectively. This paper investigates the effect of combining English and\nIndonesian data on building Indonesian text classification (e.g., sentiment\nanalysis and hate speech) using multilingual language models. Using the\nfeature-based approach, we observe its performance on various data sizes and\ntotal added English data. The experiment showed that the addition of English\ndata, especially if the amount of Indonesian data is small, improves\nperformance. Using the fine-tuning approach, we further showed its\neffectiveness in utilizing the English language to build Indonesian text\nclassification models.",
            "journal ref": "2020 International Conference on Advanced Informatics: Concept,\n  Theory and Application (ICAICTA)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05720v1",
            "date": "2020-09-12 03:43:30+00:00",
            "title": "Improving Bi-LSTM Performance for Indonesian Sentiment Analysis Using Paragraph Vector",
            "authors": [
                "Ayu Purwarianti",
                "Ida Ayu Putu Ari Crisdayanti"
            ],
            "abstract": "Bidirectional Long Short-Term Memory Network (Bi-LSTM) has shown promising\nperformance in sentiment classification task. It processes inputs as sequence\nof information. Due to this behavior, sentiment predictions by Bi-LSTM were\ninfluenced by words sequence and the first or last phrases of the texts tend to\nhave stronger features than other phrases. Meanwhile, in the problem scope of\nIndonesian sentiment analysis, phrases that express the sentiment of a document\nmight not appear in the first or last part of the document that can lead to\nincorrect sentiment classification. To this end, we propose the using of an\nexisting document representation method called paragraph vector as additional\ninput features for Bi-LSTM. This vector provides information context of the\ndocument for each sequence processing. The paragraph vector is simply\nconcatenated to each word vector of the document. This representation also\nhelps to differentiate ambiguous Indonesian words. Bi-LSTM and paragraph vector\nwere previously used as separate methods. Combining the two methods has shown a\nsignificant performance improvement of Indonesian sentiment analysis model.\nSeveral case studies on testing data showed that the proposed method can handle\nthe sentiment phrases position problem encountered by Bi-LSTM.",
            "journal ref": "2019 International Conference of Advanced Informatics: Concepts,\n  Theory and Applications (ICAICTA)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05834v1",
            "date": "2020-09-12 17:36:53+00:00",
            "title": "Exploring the Hierarchy in Relation Labels for Scene Graph Generation",
            "authors": [
                "Yi Zhou",
                "Shuyang Sun",
                "Chao Zhang",
                "Yikang Li",
                "Wanli Ouyang"
            ],
            "abstract": "By assigning each relationship a single label, current approaches formulate\nthe relationship detection as a classification problem. Under this formulation,\npredicate categories are treated as completely different classes. However,\ndifferent from the object labels where different classes have explicit\nboundaries, predicates usually have overlaps in their semantic meanings. For\nexample, sit\\_on and stand\\_on have common meanings in vertical relationships\nbut different details of how these two objects are vertically placed. In order\nto leverage the inherent structures of the predicate categories, we propose to\nfirst build the language hierarchy and then utilize the Hierarchy Guided\nFeature Learning (HGFL) strategy to learn better region features of both the\ncoarse-grained level and the fine-grained level. Besides, we also propose the\nHierarchy Guided Module (HGM) to utilize the coarse-grained level to guide the\nlearning of fine-grained level features. Experiments show that the proposed\nsimple yet effective method can improve several state-of-the-art baselines by a\nlarge margin (up to $33\\%$ relative gain) in terms of Recall@50 on the task of\nScene Graph Generation in different datasets.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2009.05836v1",
            "date": "2020-09-12 17:46:24+00:00",
            "title": "Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication",
            "authors": [
                "Haihua Chen",
                "Huyen Nguyen"
            ],
            "abstract": "Citation function and citation sentiment are two essential aspects of\ncitation content analysis (CCA), which are useful for influence analysis, the\nrecommendation of scientific publications. However, existing studies are mostly\ntraditional machine learning methods, although deep learning techniques have\nalso been explored, the improvement of the performance seems not significant\ndue to insufficient training data, which brings difficulties to applications.\nIn this paper, we propose to fine-tune pre-trained contextual embeddings\nULMFiT, BERT, and XLNet for the task. Experiments on three public datasets show\nthat our strategy outperforms all the baselines in terms of the F1 score. For\ncitation function identification, the XLNet model achieves 87.2%, 86.90%, and\n81.6% on DFKI, UMICH, and TKDE2019 datasets respectively, while it achieves\n91.72% and 91.56% on DFKI and UMICH in term of citation sentiment\nidentification. Our method can be used to enhance the influence analysis of\nscholars and scholarly publications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.05959v1",
            "date": "2020-09-13 09:07:14+00:00",
            "title": "BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks",
            "authors": [
                "Tongwen Huang",
                "Qingyun She",
                "Junlin Zhang"
            ],
            "abstract": "As a pre-trained Transformer model, BERT (Bidirectional Encoder\nRepresentations from Transformers) has achieved ground-breaking performance on\nmultiple NLP tasks. On the other hand, Boosting is a popular ensemble learning\ntechnique which combines many base classifiers and has been demonstrated to\nyield better generalization performance in many machine learning tasks. Some\nworks have indicated that ensemble of BERT can further improve the application\nperformance. However, current ensemble approaches focus on bagging or stacking\nand there has not been much effort on exploring the boosting. In this work, we\nproposed a novel Boosting BERT model to integrate multi-class boosting into the\nBERT. Our proposed model uses the pre-trained Transformer as the base\nclassifier to choose harder training sets to fine-tune and gains the benefits\nof both the pre-training language knowledge and boosting ensemble in NLP tasks.\nWe evaluate the proposed model on the GLUE dataset and 3 popular Chinese NLU\nbenchmarks. Experimental results demonstrate that our proposed model\nsignificantly outperforms BERT on all datasets and proves its effectiveness in\nmany NLP tasks. Replacing the BERT base with RoBERTa as base classifier,\nBoostingBERT achieves new state-of-the-art results in several NLP Tasks. We\nalso use knowledge distillation within the \"teacher-student\" framework to\nreduce the computational overhead and model storage of BoostingBERT while\nkeeping its performance for practical application.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06066v1",
            "date": "2020-09-13 19:35:43+00:00",
            "title": "Cosine meets Softmax: A tough-to-beat baseline for visual grounding",
            "authors": [
                "Nivedita Rufus",
                "Unni Krishnan R Nair",
                "K. Madhava Krishna",
                "Vineet Gandhi"
            ],
            "abstract": "In this paper, we present a simple baseline for visual grounding for\nautonomous driving which outperforms the state of the art methods, while\nretaining minimal design choices. Our framework minimizes the cross-entropy\nloss over the cosine distance between multiple image ROI features with a text\nembedding (representing the give sentence/phrase). We use pre-trained networks\nfor obtaining the initial embeddings and learn a transformation layer on top of\nthe text embedding. We perform experiments on the Talk2Car dataset and achieve\n68.7% AP50 accuracy, improving upon the previous state of the art by 8.6%. Our\ninvestigation suggests reconsideration towards more approaches employing\nsophisticated attention mechanisms or multi-stage reasoning or complex metric\nlearning loss functions by showing promise in simpler alternatives.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2009.06110v1",
            "date": "2020-09-13 23:12:49+00:00",
            "title": "Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication",
            "authors": [
                "Ga\u0161per Begu\u0161"
            ],
            "abstract": "Identity-based patterns for which a computational model needs to output some\nfeature together with a copy of that feature are computationally challenging,\nbut pose no problems to human learners and are common in world's languages. In\nthis paper, we test whether a neural network can learn an identity-based\npattern in speech called reduplication. To our knowledge, this is the first\nattempt to test identity-based patterns in deep convolutional networks trained\non raw continuous data. Unlike existing proposals, we test learning in an\nunsupervised manner and we train the network on raw acoustic data. We use the\nciwGAN architecture (Begu\\v{s} 2020; arXiv:2006.02951) in which learning of\nmeaningful representations in speech emerges from a requirement that the deep\nconvolutional network generates informative data. Based on four generative\ntests, we argue that a deep convolutional network learns to represent an\nidentity-based pattern in its latent space; by manipulating only two\ncategorical variables in the latent space, we can actively turn an\nunreduplicated form into a reduplicated form with no other changes to the\noutput in the majority of cases. We also argue that the network extends the\nidentity-based pattern to unobserved data: when reduplication is forced in the\noutput with the proposed technique for latent space manipulation, the network\ngenerates reduplicated data (e.g., it copies an [s] e.g. in [si-siju] for\n[siju] although it never sees any reduplicated forms containing an [s] in the\ninput). Comparison with human outputs of reduplication show a high degree of\nsimilarity. Exploration of how meaningful representations of identity-based\npatterns emerge and how the latent space variables outside of the training\nrange correlate with identity-based patterns in the output has general\nimplications for neural network interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06257v1",
            "date": "2020-09-14 08:30:24+00:00",
            "title": "A Comparison of Two Fluctuation Analyses for Natural Language Clustering Phenomena: Taylor and Ebeling & Neiman Methods",
            "authors": [
                "Kumiko Tanaka-Ishii",
                "Shuntaro Takahashi"
            ],
            "abstract": "This article considers the fluctuation analysis methods of Taylor and Ebeling\n& Neiman. While both have been applied to various phenomena in the statistical\nmechanics domain, their similarities and differences have not been clarified.\nAfter considering their analytical aspects, this article presents a large-scale\napplication of these methods to text. It is found that both methods can\ndistinguish real text from independently and identically distributed (i.i.d.)\nsequences. Furthermore, it is found that the Taylor exponents acquired from\nwords can roughly distinguish text categories; this is also the case for\nEbeling and Neiman exponents, but to a lesser extent. Additionally, both\nmethods show some possibility of capturing script kinds.",
            "journal ref": "Fractals, in 2021, No.2.\n  https://www.worldscientific.com/toc/fractals/0/ja",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06354v1",
            "date": "2020-09-08 23:34:18+00:00",
            "title": "QED: A Framework and Dataset for Explanations in Question Answering",
            "authors": [
                "Matthew Lamm",
                "Jennimaria Palomaki",
                "Chris Alberti",
                "Daniel Andor",
                "Eunsol Choi",
                "Livio Baldini Soares",
                "Michael Collins"
            ],
            "abstract": "A question answering system that in addition to providing an answer provides\nan explanation of the reasoning that leads to that answer has potential\nadvantages in terms of debuggability, extensibility and trust. To this end, we\npropose QED, a linguistically informed, extensible framework for explanations\nin question answering. A QED explanation specifies the relationship between a\nquestion and answer according to formal semantic notions such as referential\nequality, sentencehood, and entailment. We describe and publicly release an\nexpert-annotated dataset of QED explanations built upon a subset of the Google\nNatural Questions dataset, and report baseline models on two tasks -- post-hoc\nexplanation generation given an answer, and joint question answering and\nexplanation generation. In the joint setting, a promising result suggests that\ntraining on a relatively small amount of QED data can improve question\nanswering. In addition to describing the formal, language-theoretic motivations\nfor the QED approach, we describe a large user study showing that the presence\nof QED explanations significantly improves the ability of untrained raters to\nspot errors made by a strong neural QA baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06367v2",
            "date": "2020-09-14 17:45:36+00:00",
            "title": "GeDi: Generative Discriminator Guided Sequence Generation",
            "authors": [
                "Ben Krause",
                "Akhilesh Deepak Gotmare",
                "Bryan McCann",
                "Nitish Shirish Keskar",
                "Shafiq Joty",
                "Richard Socher",
                "Nazneen Fatema Rajani"
            ],
            "abstract": "While large-scale language models (LMs) are able to imitate the distribution\nof natural language well enough to generate realistic text, it is difficult to\ncontrol which regions of the distribution they generate. This is especially\nproblematic because datasets used for training large LMs usually contain\nsignificant toxicity, hate, bias, and negativity. We propose GeDi as an\nefficient method for using smaller LMs as generative discriminators to guide\ngeneration from large LMs to make them safer and more controllable. GeDi guides\ngeneration at each step by computing classification probabilities for all\npossible next tokens via Bayes rule by normalizing over two class-conditional\ndistributions; one conditioned on the desired attribute, or control code, and\nanother conditioned on the undesired attribute, or anti control code. We find\nthat GeDi gives stronger controllability than the state of the art method while\nalso achieving generation speeds more than 30 times faster. Additionally,\ntraining GeDi on only four topics allows us to controllably generate new topics\nzero-shot from just a keyword, unlocking a new capability that previous\ncontrollable generation methods do not have. Lastly, we show that GeDi can make\nGPT-2 (1.5B parameters) significantly less toxic without sacrificing linguistic\nquality, making it by far the most practical existing method for detoxifying\nlarge language models while maintaining a fast generation speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06485v1",
            "date": "2020-09-14 14:44:08+00:00",
            "title": "A Finitist's Manifesto: Do we need to Reformulate the Foundations of Mathematics?",
            "authors": [
                "Jonathan Lenchner"
            ],
            "abstract": "There is a problem with the foundations of classical mathematics, and\npotentially even with the foundations of computer science, that mathematicians\nhave by-and-large ignored. This essay is a call for practicing mathematicians\nwho have been sleep-walking in their infinitary mathematical paradise to take\nheed. Much of mathematics relies upon either (i) the \"existence'\" of objects\nthat contain an infinite number of elements, (ii) our ability, \"in theory\", to\ncompute with an arbitrary level of precision, or (iii) our ability, \"in\ntheory\", to compute for an arbitrarily large number of time steps. All of\ncalculus relies on the notion of a limit. The monumental results of real and\ncomplex analysis rely on a seamless notion of the \"continuum\" of real numbers,\nwhich extends in the plane to the complex numbers and gives us, among other\nthings, \"rigorous\" definitions of continuity, the derivative, various different\nintegrals, as well as the fundamental theorems of calculus and of algebra --\nthe former of which says that the derivative and integral can be viewed as\ninverse operations, and the latter of which says that every polynomial over\n$\\mathbb{C}$ has a complex root. This essay is an inquiry into whether there is\nany way to assign meaning to the notions of \"existence\" and \"in theory'\" in (i)\nto (iii) above.",
            "journal ref": "None",
            "category": "math.LO"
        },
        {
            "url": "http://arxiv.org/abs/2009.06510v1",
            "date": "2020-09-14 15:16:25+00:00",
            "title": "At your Command! An Empirical Study on How LaypersonsTeach Robots New Functions",
            "authors": [
                "Sebastian Weigelt",
                "Vanessa Steurer",
                "Walter F. Tichy"
            ],
            "abstract": "Even though intelligent systems such as Siri or Google Assistant are\nenjoyable (and useful) dialog partners, users can only access predefined\nfunctionality. Enabling end-users to extend the functionality of intelligent\nsystems will be the next big thing. To promote research in this area we carried\nout an empirical study on how laypersons teach robots new functions by means of\nnatural language instructions. The result is a labeled corpus consisting of\n3168 submissions given by 870 subjects. The analysis of the dataset revealed\nthat many participants used certain wordings to express their wish to teach new\nfunctionality; two corresponding trigrams are among the most frequent. On the\ncontrary, more than one third (36.93%) did not verbalize the teaching intent at\nall. We labeled the semantic constituents in the utterances: declaration\n(including the name of the function) and intermediate steps. The full corpus is\npublicly available: http://dx.doi.org/10.21227/zecn-6c61",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06704v1",
            "date": "2020-09-14 19:36:58+00:00",
            "title": "Food safety risk prediction with Deep Learning models using categorical embeddings on European Union data",
            "authors": [
                "Alberto Nogales",
                "Rodrigo D\u00edaz Mor\u00f3n",
                "\u00c1lvaro J. Garc\u00eda-Tejedor"
            ],
            "abstract": "The world is becoming more globalized every day and people can buy products\nfrom almost every country in the world in their local stores. Given the\ndifferent food and feed safety laws from country to country, the European Union\nbegan to register in 1977 all irregularities related to traded products to\nensure cross-border monitoring of information and a quick reaction when risks\nto public health are detected in the food chain. This information has also an\nenormous potential as a preventive tool, in order to warn actors involved in\nfood safety and optimize their resources. In this paper, a set of data related\nto food issues was scraped and analysed with Machine Learning techniques to\npredict some features of future notifications, so that pre-emptive measures can\nbe taken. The novelty of the work relies on two points: the use of categorical\nembeddings with Deep Learning models (Multilayer Perceptron and 1-Dimension\nConvolutional Neural Networks) and its application to solve the problem of\npredicting food issues in the European Union. The models allow several features\nto be predicted: product category, hazard category and finally the proper\naction to be taken. Results show that the system can predict these features\nwith an accuracy ranging from 74.08% to 93.06%.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2009.06810v1",
            "date": "2020-09-15 01:18:21+00:00",
            "title": "Using Known Words to Learn More Words: A Distributional Analysis of Child Vocabulary Development",
            "authors": [
                "Andrew Z. Flores",
                "Jessica Montag",
                "Jon Willits"
            ],
            "abstract": "Why do children learn some words before others? Understanding individual\nvariability across children and also variability across words, may be\ninformative of the learning processes that underlie language learning. We\ninvestigated item-based variability in vocabulary development using lexical\nproperties of distributional statistics derived from a large corpus of\nchild-directed speech. Unlike previous analyses, we predicted word trajectories\ncross-sectionally, shedding light on trends in vocabulary development that may\nnot have been evident at a single time point. We also show that whether one\nlooks at a single age group or across ages as a whole, the best distributional\npredictor of whether a child knows a word is the number of other known words\nwith which that word tends to co-occur. Keywords: age of acquisition;\nvocabulary development; lexical diversity; child-directed speech;",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06823v2",
            "date": "2020-09-15 01:59:17+00:00",
            "title": "Real-Time Execution of Large-scale Language Models on Mobile",
            "authors": [
                "Wei Niu",
                "Zhenglun Kong",
                "Geng Yuan",
                "Weiwen Jiang",
                "Jiexiong Guan",
                "Caiwen Ding",
                "Pu Zhao",
                "Sijia Liu",
                "Bin Ren",
                "Yanzhi Wang"
            ],
            "abstract": "Pre-trained large-scale language models have increasingly demonstrated high\naccuracy on many natural language processing (NLP) tasks. However, the limited\nweight storage and computational speed on hardware platforms have impeded the\npopularity of pre-trained models, especially in the era of edge computing. In\nthis paper, we seek to find the best model structure of BERT for a given\ncomputation size to match specific devices. We propose the first compiler-aware\nneural architecture optimization framework. Our framework can guarantee the\nidentified model to meet both resource and real-time specifications of mobile\ndevices, thus achieving real-time execution of large transformer-based models\nlike BERT variants. We evaluate our model on several NLP tasks, achieving\ncompetitive results on well-known benchmarks with lower latency on mobile\ndevices. Specifically, our model is 5.2x faster on CPU and 4.1x faster on GPU\nwith 0.5-2% accuracy loss compared with BERT-base. Our overall framework\nachieves up to 7.8x speedup compared with TensorFlow-Lite with only minor\naccuracy loss.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.06857v1",
            "date": "2020-09-15 04:04:20+00:00",
            "title": "Current Limitations of Language Models: What You Need is Retrieval",
            "authors": [
                "Aran Komatsuzaki"
            ],
            "abstract": "We classify and re-examine some of the current approaches to improve the\nperformance-computes trade-off of language models, including (1) non-causal\nmodels (such as masked language models), (2) extension of batch length with\nefficient attention, (3) recurrence, (4) conditional computation and (5)\nretrieval. We identify some limitations (1) - (4) suffer from. For example, (1)\ncurrently struggles with open-ended text generation with the output loosely\nconstrained by the input as well as performing general textual tasks like\nGPT-2/3 due to its need for a specific fine-tuning dataset. (2) and (3) do not\nimprove the prediction of the first $\\sim 10^3$ tokens. Scaling up a model size\n(e.g. efficiently with (4)) still results in poor performance scaling for some\ntasks. We argue (5) would resolve many of these limitations, and it can (a)\nreduce the amount of supervision and (b) efficiently extend the context over\nthe entire training dataset and the entire past of the current sample. We\nspeculate how to modify MARGE to perform unsupervised causal modeling that\nachieves (b) with the retriever jointly trained.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07053v1",
            "date": "2020-09-03 19:56:30+00:00",
            "title": "Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models",
            "authors": [
                "Joseph F DeRose",
                "Jiayao Wang",
                "Matthew Berger"
            ],
            "abstract": "Advances in language modeling have led to the development of deep\nattention-based models that are performant across a wide variety of natural\nlanguage processing (NLP) problems. These language models are typified by a\npre-training process on large unlabeled text corpora and subsequently\nfine-tuned for specific tasks. Although considerable work has been devoted to\nunderstanding the attention mechanisms of pre-trained models, it is less\nunderstood how a model's attention mechanisms change when trained for a target\nNLP task. In this paper, we propose a visual analytics approach to\nunderstanding fine-tuning in attention-based language models. Our\nvisualization, Attention Flows, is designed to support users in querying,\ntracing, and comparing attention within layers, across layers, and amongst\nattention heads in Transformer-based language models. To help users gain\ninsight on how a classification decision is made, our design is centered on\ndepicting classification-based attention at the deepest layer and how attention\nfrom prior layers flows throughout words in the input. Attention Flows supports\nthe analysis of a single model, as well as the visual comparison between\npre-trained and fine-tuned models via their similarities and differences. We\nuse Attention Flows to study attention mechanisms in various sentence\nunderstanding tasks and highlight how attention evolves to address the nuances\nof solving these tasks.",
            "journal ref": "None",
            "category": "cs.HC"
        },
        {
            "url": "http://arxiv.org/abs/2009.07058v1",
            "date": "2020-09-15 13:11:13+00:00",
            "title": "MLMLM: Link Prediction with Mean Likelihood Masked Language Model",
            "authors": [
                "Louis Clouatre",
                "Philippe Trempe",
                "Amal Zouaq",
                "Sarath Chandar"
            ],
            "abstract": "Knowledge Bases (KBs) are easy to query, verifiable, and interpretable. They\nhowever scale with man-hours and high-quality data. Masked Language Models\n(MLMs), such as BERT, scale with computing power as well as unstructured raw\ntext data. The knowledge contained within those models is however not directly\ninterpretable. We propose to perform link prediction with MLMs to address both\nthe KBs scalability issues and the MLMs interpretability issues. To do that we\nintroduce MLMLM, Mean Likelihood Masked Language Model, an approach comparing\nthe mean likelihood of generating the different entities to perform link\nprediction in a tractable manner. We obtain State of the Art (SotA) results on\nthe WN18RR dataset and the best non-entity-embedding based results on the\nFB15k-237 dataset. We also obtain convincing results on link prediction on\npreviously unseen entities, making MLMLM a suitable approach to introducing new\nentities to a KB.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07185v2",
            "date": "2020-09-15 15:49:19+00:00",
            "title": "Critical Thinking for Language Models",
            "authors": [
                "Gregor Betz",
                "Christian Voigt",
                "Kyle Richardson"
            ],
            "abstract": "This paper takes a first step towards a critical thinking curriculum for\nneural auto-regressive language models. We introduce a synthetic corpus of\ndeductively valid arguments, and generate artificial argumentative texts to\ntrain and evaluate GPT-2. Significant transfer learning effects can be\nobserved: Training a model on three simple core schemes allows it to accurately\ncomplete conclusions of different, and more complex types of arguments, too.\nThe language models generalize the core argument schemes in a correct way.\nMoreover, we obtain consistent and promising results for NLU benchmarks. In\nparticular, pre-training on the argument schemes raises zero-shot accuracy on\nthe GLUE diagnostics by up to 15 percentage points. The findings suggest that\nintermediary pre-training on texts that exemplify basic reasoning abilities\n(such as typically covered in critical thinking textbooks) might help language\nmodels to acquire a broad range of reasoning skills. The synthetic\nargumentative texts presented in this paper are a promising starting point for\nbuilding such a \"critical thinking curriculum for language models.\"",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07253v2",
            "date": "2020-09-15 17:43:02+00:00",
            "title": "Autoregressive Knowledge Distillation through Imitation Learning",
            "authors": [
                "Alexander Lin",
                "Jeremy Wohlwend",
                "Howard Chen",
                "Tao Lei"
            ],
            "abstract": "The performance of autoregressive models on natural language generation tasks\nhas dramatically improved due to the adoption of deep, self-attentive\narchitectures. However, these gains have come at the cost of hindering\ninference speed, making state-of-the-art models cumbersome to deploy in\nreal-world, time-sensitive settings. We develop a compression technique for\nautoregressive models that is driven by an imitation learning perspective on\nknowledge distillation. The algorithm is designed to address the exposure bias\nproblem. On prototypical language generation tasks such as translation and\nsummarization, our method consistently outperforms other distillation\nalgorithms, such as sequence-level knowledge distillation. Student models\ntrained with our method attain 1.4 to 4.8 BLEU/ROUGE points higher than those\ntrained from scratch, while increasing inference speed by up to 14 times in\ncomparison to the teacher model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07258v2",
            "date": "2020-09-15 17:50:09+00:00",
            "title": "BERT-QE: Contextualized Query Expansion for Document Re-ranking",
            "authors": [
                "Zhi Zheng",
                "Kai Hui",
                "Ben He",
                "Xianpei Han",
                "Le Sun",
                "Andrew Yates"
            ],
            "abstract": "Query expansion aims to mitigate the mismatch between the language used in a\nquery and in a document. However, query expansion methods can suffer from\nintroducing non-relevant information when expanding the query. To bridge this\ngap, inspired by recent advances in applying contextualized models like BERT to\nthe document retrieval task, this paper proposes a novel query expansion model\nthat leverages the strength of the BERT model to select relevant document\nchunks for expansion. In evaluation on the standard TREC Robust04 and GOV2 test\ncollections, the proposed BERT-QE model significantly outperforms BERT-Large\nmodels.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2009.07373v2",
            "date": "2020-09-15 22:20:27+00:00",
            "title": "Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction",
            "authors": [
                "Rujun Han",
                "Yichao Zhou",
                "Nanyun Peng"
            ],
            "abstract": "Extracting event temporal relations is a critical task for information\nextraction and plays an important role in natural language understanding. Prior\nsystems leverage deep learning and pre-trained language models to improve the\nperformance of the task. However, these systems often suffer from two\nshort-comings: 1) when performing maximum a posteriori (MAP) inference based on\nneural models, previous systems only used structured knowledge that are assumed\nto be absolutely correct, i.e., hard constraints; 2) biased predictions on\ndominant temporal relations when training with a limited amount of data. To\naddress these issues, we propose a framework that enhances deep neural network\nwith distributional constraints constructed by probabilistic domain knowledge.\nWe solve the constrained inference problem via Lagrangian Relaxation and apply\nit on end-to-end event temporal relation extraction tasks. Experimental results\nshow our framework is able to improve the baseline neural network models with\nstrong statistical significance on two widely used datasets in news and\nclinical domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07402v1",
            "date": "2020-09-16 00:30:49+00:00",
            "title": "Asking Complex Questions with Multi-hop Answer-focused Reasoning",
            "authors": [
                "Xiyao Ma",
                "Qile Zhu",
                "Yanlin Zhou",
                "Xiaolin Li",
                "Dapeng Wu"
            ],
            "abstract": "Asking questions from natural language text has attracted increasing\nattention recently, and several schemes have been proposed with promising\nresults by asking the right question words and copy relevant words from the\ninput to the question. However, most state-of-the-art methods focus on asking\nsimple questions involving single-hop relations. In this paper, we propose a\nnew task called multihop question generation that asks complex and semantically\nrelevant questions by additionally discovering and modeling the multiple\nentities and their semantic relations given a collection of documents and the\ncorresponding answer 1. To solve the problem, we propose multi-hop\nanswer-focused reasoning on the grounded answer-centric entity graph to include\ndifferent granularity levels of semantic information including the word-level\nand document-level semantics of the entities and their semantic relations.\nThrough extensive experiments on the HOTPOTQA dataset, we demonstrate the\nsuperiority and effectiveness of our proposed model that serves as a baseline\nto motivate future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07406v1",
            "date": "2020-09-16 00:58:13+00:00",
            "title": "Tag and Correct: Question aware Open Information Extraction with Two-stage Decoding",
            "authors": [
                "Martin Kuo",
                "Yaobo Liang",
                "Lei Ji",
                "Nan Duan",
                "Linjun Shou",
                "Ming Gong",
                "Peng Chen"
            ],
            "abstract": "Question Aware Open Information Extraction (Question aware Open IE) takes\nquestion and passage as inputs, outputting an answer tuple which contains a\nsubject, a predicate, and one or more arguments. Each field of answer is a\nnatural language word sequence and is extracted from the passage. The\nsemi-structured answer has two advantages which are more readable and\nfalsifiable compared to span answer. There are two approaches to solve this\nproblem. One is an extractive method which extracts candidate answers from the\npassage with the Open IE model, and ranks them by matching with questions. It\nfully uses the passage information at the extraction step, but the extraction\nis independent to the question. The other one is the generative method which\nuses a sequence to sequence model to generate answers directly. It combines the\nquestion and passage as input at the same time, but it generates the answer\nfrom scratch, which does not use the facts that most of the answer words come\nfrom in the passage. To guide the generation by passage, we present a two-stage\ndecoding model which contains a tagging decoder and a correction decoder. At\nthe first stage, the tagging decoder will tag keywords from the passage. At the\nsecond stage, the correction decoder will generate answers based on tagged\nkeywords. Our model could be trained end-to-end although it has two stages.\nCompared to previous generative models, we generate better answers by\ngenerating coarse to fine. We evaluate our model on WebAssertions (Yan et al.,\n2018) which is a Question aware Open IE dataset. Our model achieves a BLEU\nscore of 59.32, which is better than previous generative methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07473v1",
            "date": "2020-09-16 05:00:40+00:00",
            "title": "Solomon at SemEval-2020 Task 11: Ensemble Architecture for Fine-Tuned Propaganda Detection in News Articles",
            "authors": [
                "Mayank Raj",
                "Ajay Jaiswal",
                "Rohit R. R",
                "Ankita Gupta",
                "Sudeep Kumar Sahoo",
                "Vertika Srivastava",
                "Yeon Hyang Kim"
            ],
            "abstract": "This paper describes our system (Solomon) details and results of\nparticipation in the SemEval 2020 Task 11 \"Detection of Propaganda Techniques\nin News Articles\"\\cite{DaSanMartinoSemeval20task11}. We participated in Task\n\"Technique Classification\" (TC) which is a multi-class classification task. To\naddress the TC task, we used RoBERTa based transformer architecture for\nfine-tuning on the propaganda dataset. The predictions of RoBERTa were further\nfine-tuned by class-dependent-minority-class classifiers. A special classifier,\nwhich employs dynamically adapted Least Common Sub-sequence algorithm, is used\nto adapt to the intricacies of repetition class. Compared to the other\nparticipating systems, our submission is ranked 4th on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07494v1",
            "date": "2020-09-16 06:38:03+00:00",
            "title": "Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretability",
            "authors": [
                "Ninghao Liu",
                "Yunsong Meng",
                "Xia Hu",
                "Tie Wang",
                "Bo Long"
            ],
            "abstract": "Recent years have witnessed an increasing number of interpretation methods\nbeing developed for improving transparency of NLP models. Meanwhile,\nresearchers also try to answer the question that whether the obtained\ninterpretation is faithful in explaining mechanisms behind model prediction?\nSpecifically, (Jain and Wallace, 2019) proposes that \"attention is not\nexplanation\" by comparing attention interpretation with gradient alternatives.\nHowever, it raises a new question that can we safely pick one interpretation\nmethod as the ground-truth? If not, on what basis can we compare different\ninterpretation methods? In this work, we propose that it is crucial to have a\nconcrete definition of interpretation before we could evaluate faithfulness of\nan interpretation. The definition will affect both the algorithm to obtain\ninterpretation and, more importantly, the metric used in evaluation. Through\nboth theoretical and experimental analysis, we find that although\ninterpretation methods perform differently under a certain evaluation metric,\nsuch a difference may not result from interpretation quality or faithfulness,\nbut rather the inherent bias of the evaluation metric.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07543v2",
            "date": "2020-09-16 08:28:30+00:00",
            "title": "Group-wise Contrastive Learning for Neural Dialogue Generation",
            "authors": [
                "Hengyi Cai",
                "Hongshen Chen",
                "Yonghao Song",
                "Zhuoye Ding",
                "Yongjun Bao",
                "Weipeng Yan",
                "Xiaofang Zhao"
            ],
            "abstract": "Neural dialogue response generation has gained much popularity in recent\nyears. Maximum Likelihood Estimation (MLE) objective is widely adopted in\nexisting dialogue model learning. However, models trained with MLE objective\nfunction are plagued by the low-diversity issue when it comes to the\nopen-domain conversational setting. Inspired by the observation that humans not\nonly learn from the positive signals but also benefit from correcting behaviors\nof undesirable actions, in this work, we introduce contrastive learning into\ndialogue generation, where the model explicitly perceives the difference\nbetween the well-chosen positive and negative utterances. Specifically, we\nemploy a pretrained baseline model as a reference. During contrastive learning,\nthe target dialogue model is trained to give higher conditional probabilities\nfor the positive samples, and lower conditional probabilities for those\nnegative samples, compared to the reference model. To manage the multi-mapping\nrelations prevailed in human conversation, we augment contrastive dialogue\nlearning with group-wise dual sampling. Extensive experimental results show\nthat the proposed group-wise contrastive learning framework is suited for\ntraining a wide range of neural dialogue generation models with very favorable\nperformance over the baseline training approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07659v2",
            "date": "2020-09-16 12:58:31+00:00",
            "title": "RDF2Vec Light -- A Lightweight Approach for Knowledge Graph Embeddings",
            "authors": [
                "Jan Portisch",
                "Michael Hladik",
                "Heiko Paulheim"
            ],
            "abstract": "Knowledge graph embedding approaches represent nodes and edges of graphs as\nmathematical vectors. Current approaches focus on embedding complete knowledge\ngraphs, i.e. all nodes and edges. This leads to very high computational\nrequirements on large graphs such as DBpedia or Wikidata. However, for most\ndownstream application scenarios, only a small subset of concepts is of actual\ninterest. In this paper, we present RDF2Vec Light, a lightweight embedding\napproach based on RDF2Vec which generates vectors for only a subset of\nentities. To that end, RDF2Vec Light only traverses and processes a subgraph of\nthe knowledge graph. Our method allows the application of embeddings of very\nlarge knowledge graphs in scenarios where such embeddings were not possible\nbefore due to a significantly lower runtime and significantly reduced hardware\nrequirements.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.07661v1",
            "date": "2020-09-16 13:01:05+00:00",
            "title": "Adoption of Twitter's New Length Limit: Is 280 the New 140?",
            "authors": [
                "Kristina Gligori\u0107",
                "Ashton Anderson",
                "Robert West"
            ],
            "abstract": "In November 2017, Twitter doubled the maximum allowed tweet length from 140\nto 280 characters, a drastic switch on one of the world's most influential\nsocial media platforms. In the first long-term study of how the new length\nlimit was adopted by Twitter users, we ask: Does the effect of the new length\nlimit resemble that of the old one? Or did the doubling of the limit\nfundamentally change how Twitter is shaped by the limited length of posted\ncontent? By analyzing Twitter's publicly available 1% sample over a period of\naround 3 years, we find that, when the length limit was raised from 140 to 280\ncharacters, the prevalence of tweets around 140 characters dropped immediately,\nwhile the prevalence of tweets around 280 characters rose steadily for about 6\nmonths. Despite this rise, tweets approaching the length limit have been far\nless frequent after than before the switch. We find widely different adoption\nrates across languages and client-device types. The prevalence of tweets around\n140 characters before the switch in a given language is strongly correlated\nwith the prevalence of tweets around 280 characters after the switch in the\nsame language, and very long tweets are vastly more popular on Web clients than\non mobile clients. Moreover, tweets of around 280 characters after the switch\nare syntactically and semantically similar to tweets of around 140 characters\nbefore the switch, manifesting patterns of message squeezing in both cases.\nTaken together, these findings suggest that the new 280-character limit\nconstitutes a new, less intrusive version of the old 140-character limit. The\nlength limit remains an important factor that should be considered in all\nstudies using Twitter data.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2009.07726v1",
            "date": "2020-09-16 14:56:11+00:00",
            "title": "Leveraging Semantic Parsing for Relation Linking over Knowledge Bases",
            "authors": [
                "Nandana Mihindukulasooriya",
                "Gaetano Rossiello",
                "Pavan Kapanipathi",
                "Ibrahim Abdelaziz",
                "Srinivas Ravishankar",
                "Mo Yu",
                "Alfio Gliozzo",
                "Salim Roukos",
                "Alexander Gray"
            ],
            "abstract": "Knowledgebase question answering systems are heavily dependent on relation\nextraction and linking modules. However, the task of extracting and linking\nrelations from text to knowledgebases faces two primary challenges; the\nambiguity of natural language and lack of training data. To overcome these\nchallenges, we present SLING, a relation linking framework which leverages\nsemantic parsing using Abstract Meaning Representation (AMR) and distant\nsupervision. SLING integrates multiple relation linking approaches that capture\ncomplementary signals such as linguistic cues, rich semantic representation,\nand information from the knowledgebase. The experiments on relation linking\nusing three KBQA datasets; QALD-7, QALD-9, and LC-QuAD 1.0 demonstrate that the\nproposed approach achieves state-of-the-art performance on all benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07758v2",
            "date": "2020-09-16 15:41:21+00:00",
            "title": "GLUCOSE: GeneraLized and COntextualized Story Explanations",
            "authors": [
                "Nasrin Mostafazadeh",
                "Aditya Kalyanpur",
                "Lori Moon",
                "David Buchanan",
                "Lauren Berkowitz",
                "Or Biran",
                "Jennifer Chu-Carroll"
            ],
            "abstract": "When humans read or listen, they make implicit commonsense inferences that\nframe their understanding of what happened and why. As a step toward AI systems\nthat can build similar mental models, we introduce GLUCOSE, a large-scale\ndataset of implicit commonsense causal knowledge, encoded as causal\nmini-theories about the world, each grounded in a narrative context. To\nconstruct GLUCOSE, we drew on cognitive psychology to identify ten dimensions\nof causal explanation, focusing on events, states, motivations, and emotions.\nEach GLUCOSE entry includes a story-specific causal statement paired with an\ninference rule generalized from the statement. This paper details two concrete\ncontributions. First, we present our platform for effectively crowdsourcing\nGLUCOSE data at scale, which uses semi-structured templates to elicit causal\nexplanations. Using this platform, we collected a total of ~670K specific\nstatements and general rules that capture implicit commonsense knowledge about\neveryday situations. Second, we show that existing knowledge resources and\npretrained language models do not include or readily predict GLUCOSE's rich\ninferential content. However, when state-of-the-art neural models are trained\non this knowledge, they can start to make commonsense inferences on unseen\nstories that match humans' mental models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.07780v2",
            "date": "2020-09-16 16:18:04+00:00",
            "title": "Deep Learning Approaches for Extracting Adverse Events and Indications of Dietary Supplements from Clinical Text",
            "authors": [
                "Yadan Fan",
                "Sicheng Zhou",
                "Yifan Li",
                "Rui Zhang"
            ],
            "abstract": "The objective of our work is to demonstrate the feasibility of utilizing deep\nlearning models to extract safety signals related to the use of dietary\nsupplements (DS) in clinical text. Two tasks were performed in this study. For\nthe named entity recognition (NER) task, Bi-LSTM-CRF (Bidirectional\nLong-Short-Term-Memory Conditional Random Fields) and BERT (Bidirectional\nEncoder Representations from Transformers) models were trained and compared\nwith CRF model as a baseline to recognize the named entities of DS and Events\nfrom clinical notes. In the relation extraction (RE) task, two deep learning\nmodels, including attention-based Bi-LSTM and CNN (Convolutional Neural\nNetwork), and a random forest model were trained to extract the relations\nbetween DS and Events, which were categorized into three classes: positive\n(i.e., indication), negative (i.e., adverse events), and not related. The best\nperformed NER and RE models were further applied on clinical notes mentioning\n88 DS for discovering DS adverse events and indications, which were compared\nwith a DS knowledge base. For the NER task, deep learning models achieved a\nbetter performance than CRF, with F1 scores above 0.860. The attention-based\nBi-LSTM model performed the best in the relation extraction task, with the F1\nscore of 0.893. When comparing DS event pairs generated by the deep learning\nmodels with the knowledge base for DS and Event, we found both known and\nunknown pairs. Deep learning models can detect adverse events and indication of\nDS in clinical notes, which hold great potential for monitoring the safety of\nDS use.",
            "journal ref": "J Am Med Inform Assoc. 2020 Nov 5:ocaa218. PMID: 33150942",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2009.07839v2",
            "date": "2020-09-16 17:58:37+00:00",
            "title": "Text Generation by Learning from Demonstrations",
            "authors": [
                "Richard Yuanzhe Pang",
                "He He"
            ],
            "abstract": "Current approaches to text generation largely rely on autoregressive models\nand maximum likelihood estimation. This paradigm leads to (i) diverse but\nlow-quality samples due to mismatched learning objective and evaluation metric\n(likelihood vs. quality) and (ii) exposure bias due to mismatched history\ndistributions (gold vs. model-generated). To alleviate these problems, we frame\ntext generation as an offline reinforcement learning (RL) problem with expert\ndemonstrations (i.e., the reference), where the goal is to maximize quality\ngiven model-generated histories. We propose GOLD (generation by off-policy\nlearning from demonstrations): an easy-to-optimize algorithm that learns from\nthe demonstrations by importance weighting. Intuitively, GOLD upweights\nconfident tokens and downweights unconfident ones in the reference during\ntraining, avoiding optimization issues faced by prior RL approaches that rely\non online data collection. According to both automatic and human evaluation,\nmodels trained by GOLD outperform those trained by MLE and policy gradient on\nsummarization, question generation, and machine translation. Further, our\nmodels are less sensitive to decoding algorithms and alleviate exposure bias.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08018v1",
            "date": "2020-09-17 02:13:14+00:00",
            "title": "Multi-modal Summarization for Video-containing Documents",
            "authors": [
                "Xiyan Fu",
                "Jun Wang",
                "Zhenglu Yang"
            ],
            "abstract": "Summarization of multimedia data becomes increasingly significant as it is\nthe basis for many real-world applications, such as question answering, Web\nsearch, and so forth. Most existing multi-modal summarization works however\nhave used visual complementary features extracted from images rather than\nvideos, thereby losing abundant information. Hence, we propose a novel\nmulti-modal summarization task to summarize from a document and its associated\nvideo. In this work, we also build a baseline general model with effective\nstrategies, i.e., bi-hop attention and improved late fusion mechanisms to\nbridge the gap between different modalities, and a bi-stream summarization\nstrategy to employ text and video summarization simultaneously. Comprehensive\nexperiments show that the proposed model is beneficial for multi-modal\nsummarization and superior to existing methods. Moreover, we collect a novel\ndataset and it provides a new resource for future study that results from\ndocuments and videos.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08100v2",
            "date": "2020-09-17 06:39:49+00:00",
            "title": "How-to Present News on Social Media: A Causal Analysis of Editing News Headlines for Boosting User Engagement",
            "authors": [
                "Kunwoo Park",
                "Haewoon Kwak",
                "Jisun An",
                "Sanjay Chawla"
            ],
            "abstract": "To reach a broader audience and optimize traffic toward news articles, media\noutlets commonly run social media accounts and share their content with a short\ntext summary. Despite its importance of writing a compelling message in sharing\narticles, the research community does not own a sufficient understanding of\nwhat kinds of editing strategies effectively promote audience engagement. In\nthis study, we aim to fill the gap by analyzing media outlets' current\npractices using a data-driven approach. We first build a parallel corpus of\noriginal news articles and their corresponding tweets that eight media outlets\nshared. Then, we explore how those media edited tweets against original\nheadlines and the effects of such changes. To estimate the effects of editing\nnews headlines for social media sharing in audience engagement, we present a\nsystematic analysis that incorporates a causal inference technique with deep\nlearning; using propensity score matching, it allows for estimating potential\n(dis-)advantages of an editing style compared to counterfactual cases where a\nsimilar news article is shared with a different style. According to the\nanalyses of various editing styles, we report common and differing effects of\nthe styles across the outlets. To understand the effects of various editing\nstyles, media outlets could apply our easy-to-use tool by themselves.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2009.08115v3",
            "date": "2020-09-17 07:26:37+00:00",
            "title": "A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning",
            "authors": [
                "Yichi Zhang",
                "Zhijian Ou",
                "Huixin Wang",
                "Junlan Feng"
            ],
            "abstract": "Structured belief states are crucial for user goal tracking and database\nquery in task-oriented dialog systems. However, training belief trackers often\nrequires expensive turn-level annotations of every user utterance. In this\npaper we aim at alleviating the reliance on belief state labels in building\nend-to-end dialog systems, by leveraging unlabeled dialog data towards\nsemi-supervised learning. We propose a probabilistic dialog model, called the\nLAtent BElief State (LABES) model, where belief states are represented as\ndiscrete latent variables and jointly modeled with system responses given user\ninputs. Such latent variable modeling enables us to develop semi-supervised\nlearning under the principled variational learning framework. Furthermore, we\nintroduce LABES-S2S, which is a copy-augmented Seq2Seq model instantiation of\nLABES. In supervised experiments, LABES-S2S obtains strong results on three\nbenchmark datasets of different scales. In utilizing unlabeled dialog data,\nsemi-supervised LABES-S2S significantly outperforms both supervised-only and\nsemi-supervised baselines. Remarkably, we can reduce the annotation demands to\n50% without performance loss on MultiWOZ.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08128v2",
            "date": "2020-09-17 08:03:42+00:00",
            "title": "Multi$^2$OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT",
            "authors": [
                "Youngbin Ro",
                "Yukyung Lee",
                "Pilsung Kang"
            ],
            "abstract": "In this paper, we propose Multi$^2$OIE, which performs open information\nextraction (open IE) by combining BERT with multi-head attention. Our model is\na sequence-labeling system with an efficient and effective argument extraction\nmethod. We use a query, key, and value setting inspired by the Multimodal\nTransformer to replace the previously used bidirectional long short-term memory\narchitecture with multi-head attention. Multi$^2$OIE outperforms existing\nsequence-labeling systems with high computational efficiency on two benchmark\nevaluation datasets, Re-OIE2016 and CaRB. Additionally, we apply the proposed\nmethod to multilingual open IE using multilingual BERT. Experimental results on\nnew benchmark datasets introduced for two languages (Spanish and Portuguese)\ndemonstrate that our model outperforms other multilingual systems without\ntraining data for the target languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08138v3",
            "date": "2020-09-17 08:17:12+00:00",
            "title": "FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding",
            "authors": [
                "Yutai Hou",
                "Jiafeng Mao",
                "Yongkui Lai",
                "Cheng Chen",
                "Wanxiang Che",
                "Zhigang Chen",
                "Ting Liu"
            ],
            "abstract": "Few-shot learning (FSL) is one of the key future steps in machine learning\nand has raised a lot of attention. However, in contrast to the rapid\ndevelopment in other domains, such as Computer Vision, the progress of FSL in\nNature Language Processing (NLP) is much slower. One of the key reasons for\nthis is the lacking of public benchmarks. NLP FSL researches always report new\nresults on their own constructed few-shot datasets, which is pretty inefficient\nin results comparison and thus impedes cumulative progress. In this paper, we\npresent FewJoint, a novel Few-Shot Learning benchmark for NLP. Different from\nmost NLP FSL research that only focus on simple N-classification problems, our\nbenchmark introduces few-shot joint dialogue language understanding, which\nadditionally covers the structure prediction and multi-task reliance problems.\nThis allows our benchmark to reflect the real-word NLP complexity beyond simple\nN-classification. Our benchmark is used in the few-shot learning contest of\nSMP2020-ECDT task-1. We also provide a compatible FSL platform to ease\nexperiment set-up.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08205v1",
            "date": "2020-09-17 10:50:42+00:00",
            "title": "Generating Label Cohesive and Well-Formed Adversarial Claims",
            "authors": [
                "Pepa Atanasova",
                "Dustin Wright",
                "Isabelle Augenstein"
            ],
            "abstract": "Adversarial attacks reveal important vulnerabilities and flaws of trained\nmodels. One potent type of attack are universal adversarial triggers, which are\nindividual n-grams that, when appended to instances of a class under attack,\ncan trick a model into predicting a target class. However, for inference tasks\nsuch as fact checking, these triggers often inadvertently invert the meaning of\ninstances they are inserted in. In addition, such attacks produce semantically\nnonsensical inputs, as they simply concatenate triggers to existing samples.\nHere, we investigate how to generate adversarial attacks against fact checking\nsystems that preserve the ground truth meaning and are semantically valid. We\nextend the HotFlip attack algorithm used for universal trigger generation by\njointly minimising the target class loss of a fact checking model and the\nentailment class loss of an auxiliary natural language inference model. We then\ntrain a conditional language model to generate semantically valid statements,\nwhich include the found universal triggers. We find that the generated attacks\nmaintain the directionality and semantic validity of the claim better than\nprevious work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08229v2",
            "date": "2020-09-17 12:18:43+00:00",
            "title": "AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network",
            "authors": [
                "Xinyu Wang",
                "Yong Jiang",
                "Nguyen Bach",
                "Tao Wang",
                "Zhongqiang Huang",
                "Fei Huang",
                "Kewei Tu"
            ],
            "abstract": "The linear-chain Conditional Random Field (CRF) model is one of the most\nwidely-used neural sequence labeling approaches. Exact probabilistic inference\nalgorithms such as the forward-backward and Viterbi algorithms are typically\napplied in training and prediction stages of the CRF model. However, these\nalgorithms require sequential computation that makes parallelization\nimpossible. In this paper, we propose to employ a parallelizable approximate\nvariational inference algorithm for the CRF model. Based on this algorithm, we\ndesign an approximate inference network that can be connected with the encoder\nof the neural CRF model to form an end-to-end network, which is amenable to\nparallelization for faster training and prediction. The empirical results show\nthat our proposed approaches achieve a 12.7-fold improvement in decoding speed\nwith long sentences and a competitive accuracy compared with the traditional\nCRF approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08257v1",
            "date": "2020-09-17 13:00:13+00:00",
            "title": "Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA",
            "authors": [
                "Ieva Stali\u016bnait\u0117",
                "Ignacio Iacobacci"
            ],
            "abstract": "Many NLP tasks have benefited from transferring knowledge from contextualized\nword embeddings, however the picture of what type of knowledge is transferred\nis incomplete. This paper studies the types of linguistic phenomena accounted\nfor by language models in the context of a Conversational Question Answering\n(CoQA) task. We identify the problematic areas for the finetuned RoBERTa, BERT\nand DistilBERT models through systematic error analysis - basic arithmetic\n(counting phrases), compositional semantics (negation and Semantic Role\nLabeling), and lexical semantics (surprisal and antonymy). When enhanced with\nthe relevant linguistic knowledge through multitask learning, the models\nimprove in performance. Ensembles of the enhanced models yield a boost between\n2.2 and 2.7 points in F1 score overall, and up to 42.1 points in F1 on the\nhardest question classes. The results show differences in ability to represent\ncompositional and lexical information between RoBERTa, BERT and DistilBERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08330v3",
            "date": "2020-09-17 14:28:27+00:00",
            "title": "More Embeddings, Better Sequence Labelers?",
            "authors": [
                "Xinyu Wang",
                "Yong Jiang",
                "Nguyen Bach",
                "Tao Wang",
                "Zhongqiang Huang",
                "Fei Huang",
                "Kewei Tu"
            ],
            "abstract": "Recent work proposes a family of contextual embeddings that significantly\nimproves the accuracy of sequence labelers over non-contextual embeddings.\nHowever, there is no definite conclusion on whether we can build better\nsequence labelers by combining different kinds of embeddings in various\nsettings. In this paper, we conduct extensive experiments on 3 tasks over 18\ndatasets and 8 languages to study the accuracy of sequence labeling with\nvarious embedding concatenations and make three observations: (1) concatenating\nmore embedding variants leads to better accuracy in rich-resource and\ncross-domain settings and some conditions of low-resource settings; (2)\nconcatenating additional contextual sub-word embeddings with contextual\ncharacter embeddings hurts the accuracy in extremely low-resource settings; (3)\nbased on the conclusion of (1), concatenating additional similar contextual\nembeddings cannot lead to further improvements. We hope these conclusions can\nhelp people build stronger sequence labelers in various settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08366v3",
            "date": "2020-09-17 15:25:56+00:00",
            "title": "GraphCodeBERT: Pre-training Code Representations with Data Flow",
            "authors": [
                "Daya Guo",
                "Shuo Ren",
                "Shuai Lu",
                "Zhangyin Feng",
                "Duyu Tang",
                "Shujie Liu",
                "Long Zhou",
                "Nan Duan",
                "Alexey Svyatkovskiy",
                "Shengyu Fu",
                "Michele Tufano",
                "Shao Kun Deng",
                "Colin Clement",
                "Dawn Drain",
                "Neel Sundaresan",
                "Jian Yin",
                "Daxin Jiang",
                "Ming Zhou"
            ],
            "abstract": "Pre-trained models for programming language have achieved dramatic empirical\nimprovements on a variety of code-related tasks such as code search, code\ncompletion, code summarization, etc. However, existing pre-trained models\nregard a code snippet as a sequence of tokens, while ignoring the inherent\nstructure of code, which provides crucial code semantics and would enhance the\ncode understanding process. We present GraphCodeBERT, a pre-trained model for\nprogramming language that considers the inherent structure of code. Instead of\ntaking syntactic-level structure of code like abstract syntax tree (AST), we\nuse data flow in the pre-training stage, which is a semantic-level structure of\ncode that encodes the relation of \"where-the-value-comes-from\" between\nvariables. Such a semantic-level structure is neat and does not bring an\nunnecessarily deep hierarchy of AST, the property of which makes the model more\nefficient. We develop GraphCodeBERT based on Transformer. In addition to using\nthe task of masked language modeling, we introduce two structure-aware\npre-training tasks. One is to predict code structure edges, and the other is to\nalign representations between source code and code structure. We implement the\nmodel in an efficient way with a graph-guided masked attention function to\nincorporate the code structure. We evaluate our model on four tasks, including\ncode search, clone detection, code translation, and code refinement. Results\nshow that code structure and newly introduced pre-training tasks can improve\nGraphCodeBERT and achieves state-of-the-art performance on the four downstream\ntasks. We further show that the model prefers structure-level attentions over\ntoken-level attentions in the task of code search.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2009.08441v1",
            "date": "2020-09-17 17:47:00+00:00",
            "title": "A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support",
            "authors": [
                "Ashish Sharma",
                "Adam S. Miner",
                "David C. Atkins",
                "Tim Althoff"
            ],
            "abstract": "Empathy is critical to successful mental health support. Empathy measurement\nhas predominantly occurred in synchronous, face-to-face settings, and may not\ntranslate to asynchronous, text-based contexts. Because millions of people use\ntext-based platforms for mental health support, understanding empathy in these\ncontexts is crucial. In this work, we present a computational approach to\nunderstanding how empathy is expressed in online mental health platforms. We\ndevelop a novel unifying theoretically-grounded framework for characterizing\nthe communication of empathy in text-based conversations. We collect and share\na corpus of 10k (post, response) pairs annotated using this empathy framework\nwith supporting evidence for annotations (rationales). We develop a multi-task\nRoBERTa-based bi-encoder model for identifying empathy in conversations and\nextracting rationales underlying its predictions. Experiments demonstrate that\nour approach can effectively identify empathic conversations. We further apply\nthis model to analyze 235k mental health interactions and show that users do\nnot self-learn empathy over time, revealing opportunities for empathy training\nand feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08445v2",
            "date": "2020-09-17 17:53:59+00:00",
            "title": "Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks",
            "authors": [
                "Trapit Bansal",
                "Rishikesh Jha",
                "Tsendsuren Munkhdalai",
                "Andrew McCallum"
            ],
            "abstract": "Self-supervised pre-training of transformer models has revolutionized NLP\napplications. Such pre-training with language modeling objectives provides a\nuseful initial point for parameters that generalize well to new tasks with\nfine-tuning. However, fine-tuning is still data inefficient -- when there are\nfew labeled examples, accuracy can be low. Data efficiency can be improved by\noptimizing pre-training directly for future fine-tuning with few examples; this\ncan be treated as a meta-learning problem. However, standard meta-learning\ntechniques require many training tasks in order to generalize; unfortunately,\nfinding a diverse set of such supervised tasks is usually difficult. This paper\nproposes a self-supervised approach to generate a large, rich, meta-learning\ntask distribution from unlabeled text. This is achieved using a cloze-style\nobjective, but creating separate multi-class classification tasks by gathering\ntokens-to-be blanked from among only a handful of vocabulary terms. This yields\nas many unique meta-training tasks as the number of subsets of vocabulary\nterms. We meta-train a transformer model on this distribution of tasks using a\nrecent meta-learning framework. On 17 NLP tasks, we show that this\nmeta-training leads to better few-shot generalization than language-model\npre-training followed by finetuning. Furthermore, we show how the\nself-supervised tasks can be combined with supervised tasks for meta-learning,\nproviding substantial accuracy gains over previous supervised meta-learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08552v2",
            "date": "2020-09-17 23:07:03+00:00",
            "title": "Structured Attention for Unsupervised Dialogue Structure Induction",
            "authors": [
                "Liang Qiu",
                "Yizhou Zhao",
                "Weiyan Shi",
                "Yuan Liang",
                "Feng Shi",
                "Tao Yuan",
                "Zhou Yu",
                "Song-Chun Zhu"
            ],
            "abstract": "Inducing a meaningful structural representation from one or a set of\ndialogues is a crucial but challenging task in computational linguistics.\nAdvancement made in this area is critical for dialogue system design and\ndiscourse analysis. It can also be extended to solve grammatical inference. In\nthis work, we propose to incorporate structured attention layers into a\nVariational Recurrent Neural Network (VRNN) model with discrete latent states\nto learn dialogue structure in an unsupervised fashion. Compared to a vanilla\nVRNN, structured attention enables a model to focus on different parts of the\nsource sentence embeddings while enforcing a structural inductive bias.\nExperiments show that on two-party dialogue datasets, VRNN with structured\nattention learns semantic structures that are similar to templates used to\ngenerate this dialogue corpus. While on multi-party dialogue datasets, our\nmodel learns an interactive structure demonstrating its capability of\ndistinguishing speakers or addresses, automatically disentangling dialogues\nwithout explicit human annotation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08553v3",
            "date": "2020-09-17 23:08:01+00:00",
            "title": "Generation-Augmented Retrieval for Open-domain Question Answering",
            "authors": [
                "Yuning Mao",
                "Pengcheng He",
                "Xiaodong Liu",
                "Yelong Shen",
                "Jianfeng Gao",
                "Jiawei Han",
                "Weizhu Chen"
            ],
            "abstract": "We propose Generation-Augmented Retrieval (GAR) for answering open-domain\nquestions, which augments a query through text generation of heuristically\ndiscovered relevant contexts without external resources as supervision. We\ndemonstrate that the generated contexts substantially enrich the semantics of\nthe queries and GAR with sparse representations (BM25) achieves comparable or\nbetter performance than state-of-the-art dense retrieval methods such as DPR.\nWe show that generating diverse contexts for a query is beneficial as fusing\ntheir results consistently yields better retrieval accuracy. Moreover, as\nsparse and dense representations are often complementary, GAR can be easily\ncombined with DPR to achieve even better performance. GAR achieves\nstate-of-the-art performance on Natural Questions and TriviaQA datasets under\nthe extractive QA setup when equipped with an extractive reader, and\nconsistently outperforms other retrieval methods when the same generative\nreader is used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08566v2",
            "date": "2020-09-18 00:22:54+00:00",
            "title": "MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering",
            "authors": [
                "Tejas Gokhale",
                "Pratyay Banerjee",
                "Chitta Baral",
                "Yezhou Yang"
            ],
            "abstract": "While progress has been made on the visual question answering leaderboards,\nmodels often utilize spurious correlations and priors in datasets under the\ni.i.d. setting. As such, evaluation on out-of-distribution (OOD) test samples\nhas emerged as a proxy for generalization. In this paper, we present MUTANT, a\ntraining paradigm that exposes the model to perceptually similar, yet\nsemantically distinct mutations of the input, to improve OOD generalization,\nsuch as the VQA-CP challenge. Under this paradigm, models utilize a\nconsistency-constrained training objective to understand the effect of semantic\nchanges in input (question-image pair) on the output (answer). Unlike existing\nmethods on VQA-CP, MUTANT does not rely on the knowledge about the nature of\ntrain and test answer distributions. MUTANT establishes a new state-of-the-art\naccuracy on VQA-CP with a $10.57\\%$ improvement. Our work opens up avenues for\nthe use of semantic input mutations for OOD generalization in question\nanswering.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2009.08590v1",
            "date": "2020-09-18 02:16:49+00:00",
            "title": "NEU at WNUT-2020 Task 2: Data Augmentation To Tell BERT That Death Is Not Necessarily Informative",
            "authors": [
                "Kumud Chauhan"
            ],
            "abstract": "Millions of people around the world are sharing COVID-19 related information\non social media platforms. Since not all the information shared on the social\nmedia is useful, a machine learning system to identify informative posts can\nhelp users in finding relevant information. In this paper, we present a BERT\nclassifier system for W-NUT2020 Shared Task 2: Identification of Informative\nCOVID-19 English Tweets. Further, we show that BERT exploits some easy signals\nto identify informative tweets, and adding simple patterns to uninformative\ntweets drastically degrades BERT performance. In particular, simply adding 10\ndeaths to tweets in dev set, reduces BERT F1- score from 92.63 to 7.28. We also\npropose a simple data augmentation technique that helps in improving the\nrobustness and generalization ability of the BERT classifier.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08603v1",
            "date": "2020-09-18 03:12:13+00:00",
            "title": "Towards Full-line Code Completion with Neural Language Models",
            "authors": [
                "Wenhan Wang",
                "Sijie Shen",
                "Ge Li",
                "Zhi Jin"
            ],
            "abstract": "A code completion system suggests future code elements to developers given a\npartially-complete code snippet. Code completion is one of the most useful\nfeatures in Integrated Development Environments (IDEs). Currently, most code\ncompletion techniques predict a single token at a time. In this paper, we take\na further step and discuss the probability of directly completing a whole line\nof code instead of a single token. We believe suggesting longer code sequences\ncan further improve the efficiency of developers. Recently neural language\nmodels have been adopted as a preferred approach for code completion, and we\nbelieve these models can still be applied to full-line code completion with a\nfew improvements. We conduct our experiments on two real-world python corpora\nand evaluate existing neural models based on source code tokens or syntactical\nactions. The results show that neural language models can achieve acceptable\nresults on our tasks, with significant room for improvements.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2009.08694v2",
            "date": "2020-09-18 09:02:31+00:00",
            "title": "RECON: Relation Extraction using Knowledge Graph Context in a Graph Neural Network",
            "authors": [
                "Anson Bastos",
                "Abhishek Nadgeri",
                "Kuldeep Singh",
                "Isaiah Onando Mulang'",
                "Saeedeh Shekarpour",
                "Johannes Hoffart",
                "Manohar Kaul"
            ],
            "abstract": "In this paper, we present a novel method named RECON, that automatically\nidentifies relations in a sentence (sentential relation extraction) and aligns\nto a knowledge graph (KG). RECON uses a graph neural network to learn\nrepresentations of both the sentence as well as facts stored in a KG, improving\nthe overall extraction quality. These facts, including entity attributes\n(label, alias, description, instance-of) and factual triples, have not been\ncollectively used in the state of the art methods. We evaluate the effect of\nvarious forms of representing the KG context on the performance of RECON. The\nempirical evaluation on two standard relation extraction datasets shows that\nRECON significantly outperforms all state of the art methods on NYT Freebase\nand Wikidata datasets. RECON reports 87.23 F1 score (Vs 82.29 baseline) on\nWikidata dataset whereas on NYT Freebase, reported values are 87.5(P@10) and\n74.1(P@30) compared to the previous baseline scores of 81.3(P@10) and\n63.1(P@30).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08859v1",
            "date": "2020-09-18 14:28:32+00:00",
            "title": "Principal Components of the Meaning",
            "authors": [
                "Neslihan Suzen",
                "Alexander Gorban",
                "Jeremy Levesley",
                "Evgeny Mirkes"
            ],
            "abstract": "In this paper we argue that (lexical) meaning in science can be represented\nin a 13 dimension Meaning Space. This space is constructed using principal\ncomponent analysis (singular decomposition) on the matrix of word category\nrelative information gains, where the categories are those used by the Web of\nScience, and the words are taken from a reduced word set from texts in the Web\nof Science. We show that this reduced word set plausibly represents all texts\nin the corpus, so that the principal component analysis has some objective\nmeaning with respect to the corpus. We argue that 13 dimensions is adequate to\ndescribe the meaning of scientific texts, and hypothesise about the qualitative\nmeaning of the principal components.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.08899v1",
            "date": "2020-09-18 15:47:25+00:00",
            "title": "Image Captioning with Attention for Smart Local Tourism using EfficientNet",
            "authors": [
                "Dhomas Hatta Fudholi",
                "Yurio Windiatmoko",
                "Nurdi Afrianto",
                "Prastyo Eko Susanto",
                "Magfirah Suyuti",
                "Ahmad Fathan Hidayatullah",
                "Ridho Rahmadi"
            ],
            "abstract": "Smart systems have been massively developed to help humans in various tasks.\nDeep Learning technologies push even further in creating accurate assistant\nsystems due to the explosion of data lakes. One of the smart system tasks is to\ndisseminate users needed information. This is crucial in the tourism sector to\npromote local tourism destinations. In this research, we design a model of\nlocal tourism specific image captioning, which later will support the\ndevelopment of AI-powered systems that assist various users. The model is\ndeveloped using a visual Attention mechanism and uses the state-of-the-art\nfeature extractor architecture EfficientNet. A local tourism dataset is\ncollected and is used in the research, along with two different kinds of\ncaptions. Captions that describe the image literally and captions that\nrepresent human logical responses when seeing the image. This is done to make\nthe captioning model more humane when implemented in the assistance system. We\ncompared the performance of two different models using EfficientNet\narchitectures (B0 and B4) with other well known VGG16 and InceptionV3. The best\nBLEU scores we get are 73.39 and 24.51 for the training set and the validation\nset respectively, using EfficientNetB0. The captioning result using the\ndeveloped model shows that the model can produce logical caption for local\ntourism-related images",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2009.08928v1",
            "date": "2020-09-15 02:24:16+00:00",
            "title": "A Study of Genetic Algorithms for Hyperparameter Optimization of Neural Networks in Machine Translation",
            "authors": [
                "Keshav Ganapathy"
            ],
            "abstract": "With neural networks having demonstrated their versatility and benefits, the\nneed for their optimal performance is as prevalent as ever. A defining\ncharacteristic, hyperparameters, can greatly affect its performance. Thus\nengineers go through a process, tuning, to identify and implement optimal\nhyperparameters. That being said, excess amounts of manual effort are required\nfor tuning network architectures, training configurations, and preprocessing\nsettings such as Byte Pair Encoding (BPE). In this study, we propose an\nautomatic tuning method modeled after Darwin's Survival of the Fittest Theory\nvia a Genetic Algorithm (GA). Research results show that the proposed method, a\nGA, outperforms a random selection of hyperparameters.",
            "journal ref": "None",
            "category": "cs.NE"
        },
        {
            "url": "http://arxiv.org/abs/2009.08942v2",
            "date": "2020-09-18 17:37:13+00:00",
            "title": "Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation",
            "authors": [
                "Tuhin Chakrabarty",
                "Smaranda Muresan",
                "Nanyun Peng"
            ],
            "abstract": "Literary tropes, from poetry to stories, are at the crux of human imagination\nand communication. Figurative language such as a simile go beyond plain\nexpressions to give readers new insights and inspirations. In this paper, we\ntackle the problem of simile generation. Generating a simile requires proper\nunderstanding for effective mapping of properties between two concepts. To this\nend, we first propose a method to automatically construct a parallel corpus by\ntransforming a large number of similes collected from Reddit to their literal\ncounterpart using structured common sense knowledge. We then propose to\nfine-tune a pretrained sequence to sequence model, BART~\\cite{lewis2019bart},\non the literal-simile pairs to gain generalizability, so that we can generate\nnovel similes given a literal sentence. Experiments show that our approach\ngenerates $88\\%$ novel similes that do not share properties with the training\ndata. Human evaluation on an independent set of literal statements shows that\nour model generates similes better than two literary experts\n\\textit{37\\%}\\footnote{We average 32.6\\% and 41.3\\% for 2 humans.} of the\ntimes, and three baseline systems including a recent metaphor generation model\n\\textit{71\\%}\\footnote{We average 82\\% ,63\\% and 68\\% for three baselines.} of\nthe times when compared pairwise.\\footnote{The simile in the title is generated\nby our best model. Input: Generating similes effortlessly, output: Generating\nsimiles \\textit{like a Pro}.} We also show how replacing literal sentences with\nsimiles from our best model in machine generated stories improves evocativeness\nand leads to better acceptance by human judges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09107v2",
            "date": "2020-09-18 22:13:49+00:00",
            "title": "A Simple and Effective Self-Supervised Contrastive Learning Framework for Aspect Detection",
            "authors": [
                "Tian Shi",
                "Liuqing Li",
                "Ping Wang",
                "Chandan K. Reddy"
            ],
            "abstract": "Unsupervised aspect detection (UAD) aims at automatically extracting\ninterpretable aspects and identifying aspect-specific segments (such as\nsentences) from online reviews. However, recent deep learning-based topic\nmodels, specifically aspect-based autoencoder, suffer from several problems,\nsuch as extracting noisy aspects and poorly mapping aspects discovered by\nmodels to the aspects of interest. To tackle these challenges, in this paper,\nwe first propose a self-supervised contrastive learning framework and an\nattention-based model equipped with a novel smooth self-attention (SSA) module\nfor the UAD task in order to learn better representations for aspects and\nreview segments. Secondly, we introduce a high-resolution selective mapping\n(HRSMap) method to efficiently assign aspects discovered by the model to\naspects of interest. We also propose using a knowledge distilling technique to\nfurther improve the aspect detection performance. Our methods outperform\nseveral recent unsupervised and weakly supervised approaches on publicly\navailable benchmark user review datasets. Aspect interpretation results show\nthat extracted aspects are meaningful, have good coverage, and can be easily\nmapped to aspects of interest. Ablation studies and attention weight\nvisualization also demonstrate the effectiveness of SSA and the knowledge\ndistilling method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09112v2",
            "date": "2020-09-18 22:32:39+00:00",
            "title": "An Interpretable and Uncertainty Aware Multi-Task Framework for Multi-Aspect Sentiment Analysis",
            "authors": [
                "Tian Shi",
                "Ping Wang",
                "Chandan K. Reddy"
            ],
            "abstract": "In recent years, several online platforms have seen a rapid increase in the\nnumber of review systems that request users to provide aspect-level feedback.\nDocument-level Multi-aspect Sentiment Classification (DMSC), where the goal is\nto predict the ratings/sentiment from a review at an individual aspect level,\nhas become a challenging and imminent problem. To tackle this challenge, we\npropose a deliberate self-attention-based deep neural network model, namely\nFEDAR, for the DMSC problem, which can achieve competitive performance while\nalso being able to interpret the predictions made. FEDAR is equipped with a\nhighway word embedding layer to transfer knowledge from pre-trained word\nembeddings, an RNN encoder layer with output features enriched by pooling and\nfactorization techniques, and a deliberate self-attention layer. In addition,\nwe also propose an Attention-driven Keywords Ranking (AKR) method, which can\nautomatically discover aspect keywords and aspect-level opinion keywords from\nthe review corpus based on the attention weights. These keywords are\nsignificant for rating predictions by FEDAR. Since crowdsourcing annotation can\nbe an alternate way to recover missing ratings of reviews, we propose a\nLEcture-AuDience (LEAD) strategy to estimate model uncertainty in the context\nof multi-task learning, so that valuable human resources can focus on the most\nuncertain predictions. Our extensive set of experiments on five different\nopen-domain DMSC datasets demonstrate the superiority of the proposed FEDAR and\nLEAD models. We further introduce two new DMSC datasets in the healthcare\ndomain and benchmark different baseline models and our models on them.\nAttention weights visualization results and visualization of aspect and opinion\nkeywords demonstrate the interpretability of our model and the effectiveness of\nour AKR method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09123v1",
            "date": "2020-09-18 23:59:15+00:00",
            "title": "Will it Unblend?",
            "authors": [
                "Yuval Pinter",
                "Cassandra L. Jacobs",
                "Jacob Eisenstein"
            ],
            "abstract": "Natural language processing systems often struggle with out-of-vocabulary\n(OOV) terms, which do not appear in training data. Blends, such as\n\"innoventor\", are one particularly challenging class of OOV, as they are formed\nby fusing together two or more bases that relate to the intended meaning in\nunpredictable manners and degrees. In this work, we run experiments on a novel\ndataset of English OOV blends to quantify the difficulty of interpreting the\nmeanings of blends by large-scale contextual language models such as BERT. We\nfirst show that BERT's processing of these blends does not fully access the\ncomponent meanings, leaving their contextual representations semantically\nimpoverished. We find this is mostly due to the loss of characters resulting\nfrom blend formation. Then, we assess how easily different models can recognize\nthe structure and recover the origin of blends, and find that context-aware\nembedding systems outperform character-level and context-free embeddings,\nalthough their results are still far from satisfactory.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09126v2",
            "date": "2020-09-19 00:29:00+00:00",
            "title": "Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing",
            "authors": [
                "Jiayi Wang",
                "Ke Wang",
                "Niyu Ge",
                "Yangbing Shi",
                "Yu Zhao",
                "Kai Fan"
            ],
            "abstract": "With the advent of neural machine translation, there has been a marked shift\ntowards leveraging and consuming the machine translation results. However, the\ngap between machine translation systems and human translators needs to be\nmanually closed by post-editing. In this paper, we propose an end-to-end deep\nlearning framework of the quality estimation and automatic post-editing of the\nmachine translation output. Our goal is to provide error correction suggestions\nand to further relieve the burden of human translators through an interpretable\nmodel. To imitate the behavior of human translators, we design three efficient\ndelegation modules -- quality estimation, generative post-editing, and atomic\noperation post-editing and construct a hierarchical model based on them. We\nexamine this approach with the English--German dataset from WMT 2017 APE shared\ntask and our experimental results can achieve the state-of-the-art performance.\nWe also verify that the certified translators can significantly expedite their\npost-editing processing with our model in human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09127v1",
            "date": "2020-09-19 00:29:51+00:00",
            "title": "Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation",
            "authors": [
                "Pei Zhang",
                "Boxing Chen",
                "Niyu Ge",
                "Kai Fan"
            ],
            "abstract": "Many document-level neural machine translation (NMT) systems have explored\nthe utility of context-aware architecture, usually requiring an increasing\nnumber of parameters and computational complexity. However, few attention is\npaid to the baseline model. In this paper, we research extensively the pros and\ncons of the standard transformer in document-level translation, and find that\nthe auto-regressive property can simultaneously bring both the advantage of the\nconsistency and the disadvantage of error accumulation. Therefore, we propose a\nsurprisingly simple long-short term masking self-attention on top of the\nstandard transformer to both effectively capture the long-range dependence and\nreduce the propagation of errors. We examine our approach on the two publicly\navailable document-level datasets. We can achieve a strong result in BLEU and\ncapture discourse phenomena.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09226v1",
            "date": "2020-09-19 13:06:27+00:00",
            "title": "Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect",
            "authors": [
                "Zheni Zeng",
                "Chaojun Xiao",
                "Yuan Yao",
                "Ruobing Xie",
                "Zhiyuan Liu",
                "Fen Lin",
                "Leyu Lin",
                "Maosong Sun"
            ],
            "abstract": "Recommender systems aim to provide item recommendations for users, and are\nusually faced with data sparsity problem (e.g., cold start) in real-world\nscenarios. Recently pre-trained models have shown their effectiveness in\nknowledge transfer between domains and tasks, which can potentially alleviate\nthe data sparsity problem in recommender systems. In this survey, we first\nprovide a review of recommender systems with pre-training. In addition, we show\nthe benefits of pre-training to recommender systems through experiments.\nFinally, we discuss several promising directions for future research for\nrecommender systems with pre-training.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2009.09335v3",
            "date": "2020-09-20 02:25:05+00:00",
            "title": "Biomedical Event Extraction with Hierarchical Knowledge Graphs",
            "authors": [
                "Kung-Hsiang Huang",
                "Mu Yang",
                "Nanyun Peng"
            ],
            "abstract": "Biomedical event extraction is critical in understanding biomolecular\ninteractions described in scientific corpus. One of the main challenges is to\nidentify nested structured events that are associated with non-indicative\ntrigger words. We propose to incorporate domain knowledge from Unified Medical\nLanguage System (UMLS) to a pre-trained language model via Graph\nEdge-conditioned Attention Networks (GEANet) and hierarchical graph\nrepresentation. To better recognize the trigger words, each sentence is first\ngrounded to a sentence graph based on a jointly modeled hierarchical knowledge\ngraph from UMLS. The grounded graphs are then propagated by GEANet, a novel\ngraph neural networks for enhanced capabilities in inferring complex events. On\nBioNLP 2011 GENIA Event Extraction task, our approach achieved 1.41% F1 and\n3.19% F1 improvements on all events and complex events, respectively. Ablation\nstudies confirm the importance of GEANet and hierarchical KG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09372v1",
            "date": "2020-09-20 07:06:22+00:00",
            "title": "Softmax Tempering for Training Neural Machine Translation Models",
            "authors": [
                "Raj Dabre",
                "Atsushi Fujita"
            ],
            "abstract": "Neural machine translation (NMT) models are typically trained using a softmax\ncross-entropy loss where the softmax distribution is compared against smoothed\ngold labels. In low-resource scenarios, NMT models tend to over-fit because the\nsoftmax distribution quickly approaches the gold label distribution. To address\nthis issue, we propose to divide the logits by a temperature coefficient, prior\nto applying softmax, during training. In our experiments on 11 language pairs\nin the Asian Language Treebank dataset and the WMT 2019 English-to-German\ntranslation task, we observed significant improvements in translation quality\nby up to 3.9 BLEU points. Furthermore, softmax tempering makes the greedy\nsearch to be as good as beam search decoding in terms of translation quality,\nenabling 1.5 to 3.5 times speed-up. We also study the impact of softmax\ntempering on multilingual NMT and recurrently stacked NMT, both of which aim to\nreduce the NMT model size by parameter sharing thereby verifying the utility of\ntemperature in developing compact NMT models. Finally, an analysis of softmax\nentropies and gradients reveal the impact of our method on the internal\nbehavior of NMT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09378v1",
            "date": "2020-09-20 07:47:26+00:00",
            "title": "Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation",
            "authors": [
                "Chujie Zheng",
                "Yunbo Cao",
                "Daxin Jiang",
                "Minlie Huang"
            ],
            "abstract": "In a multi-turn knowledge-grounded dialog, the difference between the\nknowledge selected at different turns usually provides potential clues to\nknowledge selection, which has been largely neglected in previous research. In\nthis paper, we propose a difference-aware knowledge selection method. It first\ncomputes the difference between the candidate knowledge sentences provided at\nthe current turn and those chosen in the previous turns. Then, the differential\ninformation is fused with or disentangled from the contextual information to\nfacilitate final knowledge selection. Automatic, human observational, and\ninteractive evaluation shows that our method is able to select knowledge more\naccurately and generate more informative responses, significantly outperforming\nthe state-of-the-art baselines. The codes are available at\nhttps://github.com/chujiezheng/DiffKS.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09417v2",
            "date": "2020-09-20 12:03:58+00:00",
            "title": "F^2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax",
            "authors": [
                "Byung-Ju Choi",
                "Jimin Hong",
                "David Keetae Park",
                "Sang Wan Lee"
            ],
            "abstract": "Despite recent advances in neural text generation, encoding the rich\ndiversity in human language remains elusive. We argue that the sub-optimal text\ngeneration is mainly attributable to the imbalanced token distribution, which\nparticularly misdirects the learning model when trained with the\nmaximum-likelihood objective. As a simple yet effective remedy, we propose two\nnovel methods, F^2-Softmax and MefMax, for a balanced training even with the\nskewed frequency distribution. MefMax assigns tokens uniquely to frequency\nclasses, trying to group tokens with similar frequencies and equalize frequency\nmass between the classes. F^2-Softmax then decomposes a probability\ndistribution of the target token into a product of two conditional\nprobabilities of (i) frequency class, and (ii) token from the target frequency\nclass. Models learn more uniform probability distributions because they are\nconfined to subsets of vocabularies. Significant performance gains on seven\nrelevant metrics suggest the supremacy of our approach in improving not only\nthe diversity but also the quality of generated texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09474v2",
            "date": "2020-09-20 17:01:43+00:00",
            "title": "Persian Ezafe Recognition Using Transformers and Its Role in Part-Of-Speech Tagging",
            "authors": [
                "Ehsan Doostmohammadi",
                "Minoo Nassajian",
                "Adel Rahimi"
            ],
            "abstract": "Ezafe is a grammatical particle in some Iranian languages that links two\nwords together. Regardless of the important information it conveys, it is\nalmost always not indicated in Persian script, resulting in mistakes in reading\ncomplex sentences and errors in natural language processing tasks. In this\npaper, we experiment with different machine learning methods to achieve\nstate-of-the-art results in the task of ezafe recognition. Transformer-based\nmethods, BERT and XLMRoBERTa, achieve the best results, the latter achieving\n2.68% F1-score more than the previous state-of-the-art. We, moreover, use ezafe\ninformation to improve Persian part-of-speech tagging results and show that\nsuch information will not be useful to transformer-based methods and explain\nwhy that might be the case.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09654v2",
            "date": "2020-09-21 07:44:04+00:00",
            "title": "Generative Imagination Elevates Machine Translation",
            "authors": [
                "Quanyu Long",
                "Mingxuan Wang",
                "Lei Li"
            ],
            "abstract": "There are common semantics shared across text and images. Given a sentence in\na source language, whether depicting the visual scene helps translation into a\ntarget language? Existing multimodal neural machine translation methods (MNMT)\nrequire triplets of bilingual sentence - image for training and tuples of\nsource sentence - image for inference. In this paper, we propose ImagiT, a\nnovel machine translation method via visual imagination. ImagiT first learns to\ngenerate visual representation from the source sentence, and then utilizes both\nsource sentence and the \"imagined representation\" to produce a target\ntranslation. Unlike previous methods, it only needs the source sentence at the\ninference time. Experiments demonstrate that ImagiT benefits from visual\nimagination and significantly outperforms the text-only neural machine\ntranslation baselines. Further analysis reveals that the imagination process in\nImagiT helps fill in missing information when performing the degradation\nstrategy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09679v1",
            "date": "2020-09-21 08:38:21+00:00",
            "title": "Accent Estimation of Japanese Words from Their Surfaces and Romanizations for Building Large Vocabulary Accent Dictionaries",
            "authors": [
                "Hideyuki Tachibana",
                "Yotaro Katayama"
            ],
            "abstract": "In Japanese text-to-speech (TTS), it is necessary to add accent information\nto the input sentence. However, there are a limited number of publicly\navailable accent dictionaries, and those dictionaries e.g. UniDic, do not\ncontain many compound words, proper nouns, etc., which are required in a\npractical TTS system. In order to build a large scale accent dictionary that\ncontains those words, the authors developed an accent estimation technique that\npredicts the accent of a word from its limited information, namely the surface\n(e.g. kanji) and the yomi (simplified phonetic information). It is\nexperimentally shown that the technique can estimate accents with high\naccuracies, especially for some categories of words. The authors applied this\ntechnique to an existing large vocabulary Japanese dictionary NEologd, and\nobtained a large vocabulary Japanese accent dictionary. Many cases have been\nobserved in which the use of this dictionary yields more appropriate phonetic\ninformation than UniDic.",
            "journal ref": "Proc. ICASSP (2020) 8059-8063",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09704v3",
            "date": "2020-09-21 09:19:07+00:00",
            "title": "\"Listen, Understand and Translate\": Triple Supervision Decouples End-to-end Speech-to-text Translation",
            "authors": [
                "Qianqian Dong",
                "Rong Ye",
                "Mingxuan Wang",
                "Hao Zhou",
                "Shuang Xu",
                "Bo Xu",
                "Lei Li"
            ],
            "abstract": "An end-to-end speech-to-text translation (ST) takes audio in a source\nlanguage and outputs the text in a target language. Existing methods are\nlimited by the amount of parallel corpus. Can we build a system to fully\nutilize signals in a parallel ST corpus? We are inspired by human understanding\nsystem which is composed of auditory perception and cognitive processing. In\nthis paper, we propose Listen-Understand-Translate, (LUT), a unified framework\nwith triple supervision signals to decouple the end-to-end speech-to-text\ntranslation task. LUT is able to guide the acoustic encoder to extract as much\ninformation from the auditory input. In addition, LUT utilizes a pre-trained\nBERT model to enforce the upper encoder to produce as much semantic information\nas possible, without extra data. We perform experiments on a diverse set of\nspeech translation benchmarks, including Librispeech English-French, IWSLT\nEnglish-German and TED English-Chinese. Our results demonstrate LUT achieves\nthe state-of-the-art performance, outperforming previous methods. The code is\navailable at https://github.com/dqqcasia/st.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09737v3",
            "date": "2020-09-21 10:10:45+00:00",
            "title": "Consecutive Decoding for Speech-to-text Translation",
            "authors": [
                "Qianqian Dong",
                "Mingxuan Wang",
                "Hao Zhou",
                "Shuang Xu",
                "Bo Xu",
                "Lei Li"
            ],
            "abstract": "Speech-to-text translation (ST), which directly translates the source\nlanguage speech to the target language text, has attracted intensive attention\nrecently. However, the combination of speech recognition and machine\ntranslation in a single model poses a heavy burden on the direct cross-modal\ncross-lingual mapping. To reduce the learning difficulty, we propose\nCOnSecutive Transcription and Translation (COSTT), an integral approach for\nspeech-to-text translation. The key idea is to generate source transcript and\ntarget translation text with a single decoder. It benefits the model training\nso that additional large parallel text corpus can be fully exploited to enhance\nthe speech translation training. Our method is verified on three mainstream\ndatasets, including Augmented LibriSpeech English-French dataset, TED\nEnglish-German dataset, and TED English-Chinese dataset. Experiments show that\nour proposed COSTT outperforms the previous state-of-the-art methods. The code\nis available at https://github.com/dqqcasia/st.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09781v1",
            "date": "2020-09-21 12:04:18+00:00",
            "title": "Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems",
            "authors": [
                "Ziming Li",
                "Julia Kiseleva",
                "Maarten de Rijke"
            ],
            "abstract": "Dialogue policy learning for task-oriented dialogue systems has enjoyed great\nprogress recently mostly through employing reinforcement learning methods.\nHowever, these approaches have become very sophisticated. It is time to\nre-evaluate it. Are we really making progress developing dialogue agents only\nbased on reinforcement learning? We demonstrate how (1)~traditional supervised\nlearning together with (2)~a simulator-free adversarial learning method can be\nused to achieve performance comparable to state-of-the-art RL-based methods.\nFirst, we introduce a simple dialogue action decoder to predict the appropriate\nactions. Then, the traditional multi-label classification solution for dialogue\npolicy learning is extended by adding dense layers to improve the dialogue\nagent performance. Finally, we employ the Gumbel-Softmax estimator to\nalternatively train the dialogue agent and the dialogue reward model without\nusing reinforcement learning. Based on our extensive experimentation, we can\nconclude the proposed methods can achieve more stable and higher performance\nwith fewer efforts, such as the domain knowledge required to design a user\nsimulator and the intractable parameter tuning in reinforcement learning. Our\nmain goal is not to beat reinforcement learning with supervised learning, but\nto demonstrate the value of rethinking the role of reinforcement learning and\nsupervised learning in optimizing task-oriented dialogue systems.",
            "journal ref": "Findings of EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09870v2",
            "date": "2020-09-21 13:41:32+00:00",
            "title": "Content Planning for Neural Story Generation with Aristotelian Rescoring",
            "authors": [
                "Seraphina Goldfarb-Tarrant",
                "Tuhin Chakrabarty",
                "Ralph Weischedel",
                "Nanyun Peng"
            ],
            "abstract": "Long-form narrative text generated from large language models manages a\nfluent impersonation of human writing, but only at the local sentence level,\nand lacks structure or global cohesion. We posit that many of the problems of\nstory generation can be addressed via high-quality content planning, and\npresent a system that focuses on how to learn good plot structures to guide\nstory generation. We utilize a plot-generation language model along with an\nensemble of rescoring models that each implement an aspect of good\nstory-writing as detailed in Aristotle's Poetics. We find that stories written\nwith our more principled plot-structure are both more relevant to a given\nprompt and higher quality than baselines that do not content plan, or that plan\nin an unprincipled way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.09879v1",
            "date": "2020-09-21 13:59:24+00:00",
            "title": "WESSA at SemEval-2020 Task 9: Code-Mixed Sentiment Analysis using Transformers",
            "authors": [
                "Ahmed Sultan",
                "Mahmoud Salim",
                "Amina Gaber",
                "Islam El Hosary"
            ],
            "abstract": "In this paper, we describe our system submitted for SemEval 2020 Task 9,\nSentiment Analysis for Code-Mixed Social Media Text alongside other\nexperiments. Our best performing system is a Transfer Learning-based model that\nfine-tunes \"XLM-RoBERTa\", a transformer-based multilingual masked language\nmodel, on monolingual English and Spanish data and Spanish-English code-mixed\ndata. Our system outperforms the official task baseline by achieving a 70.1%\naverage F1-Score on the official leaderboard using the test set. For later\nsubmissions, our system manages to achieve a 75.9% average F1-Score on the test\nset using CodaLab username \"ahmed0sultan\".",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10297v2",
            "date": "2020-09-22 03:10:49+00:00",
            "title": "CodeBLEU: a Method for Automatic Evaluation of Code Synthesis",
            "authors": [
                "Shuo Ren",
                "Daya Guo",
                "Shuai Lu",
                "Long Zhou",
                "Shujie Liu",
                "Duyu Tang",
                "Neel Sundaresan",
                "Ming Zhou",
                "Ambrosio Blanco",
                "Shuai Ma"
            ],
            "abstract": "Evaluation metrics play a vital role in the growth of an area as it defines\nthe standard of distinguishing between good and bad models. In the area of code\nsynthesis, the commonly used evaluation metric is BLEU or perfect accuracy, but\nthey are not suitable enough to evaluate codes, because BLEU is originally\ndesigned to evaluate the natural language, neglecting important syntactic and\nsemantic features of codes, and perfect accuracy is too strict thus it\nunderestimates different outputs with the same semantic logic. To remedy this,\nwe introduce a new automatic evaluation metric, dubbed CodeBLEU. It absorbs the\nstrength of BLEU in the n-gram match and further injects code syntax via\nabstract syntax trees (AST) and code semantics via data-flow. We conduct\nexperiments by evaluating the correlation coefficient between CodeBLEU and\nquality scores assigned by the programmers on three code synthesis tasks, i.e.,\ntext-to-code, code translation, and code refinement. Experimental results show\nthat our proposed CodeBLEU can achieve a better correlation with programmer\nassigned scores compared with BLEU and accuracy.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2009.10359v1",
            "date": "2020-09-22 07:30:19+00:00",
            "title": "Global-to-Local Neural Networks for Document-Level Relation Extraction",
            "authors": [
                "Difeng Wang",
                "Wei Hu",
                "Ermei Cao",
                "Weijian Sun"
            ],
            "abstract": "Relation extraction (RE) aims to identify the semantic relations between\nnamed entities in text. Recent years have witnessed it raised to the document\nlevel, which requires complex reasoning with entities and mentions throughout\nan entire document. In this paper, we propose a novel model to document-level\nRE, by encoding the document information in terms of entity global and local\nrepresentations as well as context relation representations. Entity global\nrepresentations model the semantic information of all entities in the document,\nentity local representations aggregate the contextual information of multiple\nmentions of specific entities, and context relation representations encode the\ntopic information of other relations. Experimental results demonstrate that our\nmodel achieves superior performance on two public datasets for document-level\nRE. It is particularly effective in extracting relations between entities of\nlong distance and having multiple mentions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10387v1",
            "date": "2020-09-22 08:26:14+00:00",
            "title": "Logical foundations for hybrid type-logical grammars",
            "authors": [
                "Richard Moot",
                "Symon Stevens-Guille"
            ],
            "abstract": "This paper explores proof-theoretic aspects of hybrid type-logical grammars ,\na logic combining Lambek grammars with lambda grammars. We prove some basic\nproperties of the calculus, such as normalisation and the subformula property\nand also present both a sequent and a proof net calculus for hybrid\ntype-logical grammars. In addition to clarifying the logical foundations of\nhybrid type-logical grammars, the current study opens the way to variants and\nextensions of the original system, including but not limited to a\nnon-associative version and a multimodal version incorporating structural rules\nand unary modes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.10938v3",
            "date": "2020-09-23 06:18:25+00:00",
            "title": "LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Network",
            "authors": [
                "Xinyi Zhang",
                "Jiahao Xu",
                "Charlie Soh",
                "Lihui Chen"
            ],
            "abstract": "Hierarchical multi-label text classification (HMTC) has been gaining\npopularity in recent years thanks to its applicability to a plethora of\nreal-world applications. The existing HMTC algorithms largely focus on the\ndesign of classifiers, such as the local, global, or a combination of them.\nHowever, very few studies have focused on hierarchical feature extraction and\nexplore the association between the hierarchical labels and the text. In this\npaper, we propose a Label-based Attention for Hierarchical Mutlti-label Text\nClassification Neural Network (LA-HCN), where the novel label-based attention\nmodule is designed to hierarchically extract important information from the\ntext based on the labels from different hierarchy levels. Besides, hierarchical\ninformation is shared across levels while preserving the hierarchical\nlabel-based information. Separate local and global document embeddings are\nobtained and used to facilitate the respective local and global\nclassifications. In our experiments, LA-HCN outperforms other state-of-the-art\nneural network-based HMTC algorithms on four public HMTC datasets. The ablation\nstudy also demonstrates the effectiveness of the proposed label-based attention\nmodule as well as the novel local and global embeddings and classifications. By\nvisualizing the learned attention (words), we find that LA-HCN is able to\nextract meaningful information corresponding to the different labels which\nprovides explainability that may be helpful for the human analyst.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11138v1",
            "date": "2020-09-23 13:32:39+00:00",
            "title": "Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer",
            "authors": [
                "Sheng Zhang",
                "Xin Zhang",
                "Weiming Zhang",
                "Anders S\u00f8gaard"
            ],
            "abstract": "Multi-task transfer learning based on pre-trained language encoders achieves\nstate-of-the-art performance across a range of tasks. Standard approaches\nimplicitly assume the tasks, for which we have training data, are equally\nrepresentative of the tasks we are interested in, an assumption which is often\nhard to justify. This paper presents a more agnostic approach to multi-task\ntransfer learning, which uses automated curriculum learning to minimize a new\nfamily of worst-case-aware losses across tasks. Not only do these losses lead\nto better performance on outlier tasks; they also lead to better performance in\nzero-shot and few-shot transfer settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11152v3",
            "date": "2020-09-23 13:54:57+00:00",
            "title": "Hierarchical Pre-training for Sequence Labelling in Spoken Dialog",
            "authors": [
                "Emile Chapuis",
                "Pierre Colombo",
                "Matteo Manica",
                "Matthieu Labeau",
                "Chloe Clavel"
            ],
            "abstract": "Sequence labelling tasks like Dialog Act and Emotion/Sentiment identification\nare a key component of spoken dialog systems. In this work, we propose a new\napproach to learn generic representations adapted to spoken dialog, which we\nevaluate on a new benchmark we call Sequence labellIng evaLuatIon benChmark fOr\nspoken laNguagE benchmark (\\texttt{SILICONE}). \\texttt{SILICONE} is\nmodel-agnostic and contains 10 different datasets of various sizes. We obtain\nour representations with a hierarchical encoder based on transformer\narchitectures, for which we extend two well-known pre-training objectives.\nPre-training is performed on OpenSubtitles: a large corpus of spoken dialog\ncontaining over $2.3$ billion of tokens. We demonstrate how hierarchical\nencoders achieve competitive results with consistently fewer parameters\ncompared to state-of-the-art models and we show their importance for both\npre-training and fine-tuning.",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11207v2",
            "date": "2020-09-23 15:19:27+00:00",
            "title": "Crosslingual Topic Modeling with WikiPDA",
            "authors": [
                "Tiziano Piccardi",
                "Robert West"
            ],
            "abstract": "We present Wikipedia-based Polyglot Dirichlet Allocation (WikiPDA), a\ncrosslingual topic model that learns to represent Wikipedia articles written in\nany language as distributions over a common set of language-independent topics.\nIt leverages the fact that Wikipedia articles link to each other and are mapped\nto concepts in the Wikidata knowledge base, such that, when represented as bags\nof links, articles are inherently language-independent. WikiPDA works in two\nsteps, by first densifying bags of links using matrix completion and then\ntraining a standard monolingual topic model. A human evaluation shows that\nWikiPDA produces more coherent topics than monolingual text-based LDA, thus\noffering crosslinguality at no cost. We demonstrate WikiPDA's utility in two\napplications: a study of topical biases in 28 Wikipedia editions, and\ncrosslingual supervised classification. Finally, we highlight WikiPDA's\ncapacity for zero-shot language transfer, where a model is reused for new\nlanguages without any fine-tuning. Researchers can benefit from WikiPDA as a\npractical tool for studying Wikipedia's content across its 299 language\neditions in interpretable ways, via an easy-to-use library publicly available\nat https://github.com/epfl-dlab/WikiPDA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11226v1",
            "date": "2020-09-23 15:45:32+00:00",
            "title": "A Comparative Study on Structural and Semantic Properties of Sentence Embeddings",
            "authors": [
                "Alexander Kalinowski",
                "Yuan An"
            ],
            "abstract": "Sentence embeddings encode natural language sentences as low-dimensional\ndense vectors. A great deal of effort has been put into using sentence\nembeddings to improve several important natural language processing tasks.\nRelation extraction is such an NLP task that aims at identifying structured\nrelations defined in a knowledge base from unstructured text. A promising and\nmore efficient approach would be to embed both the text and structured\nknowledge in low-dimensional spaces and discover semantic alignments or\nmappings between them. Although a number of techniques have been proposed in\nthe literature for embedding both sentences and knowledge graphs, little is\nknown about the structural and semantic properties of these embedding spaces in\nterms of relation extraction. In this paper, we investigate the aforementioned\nproperties by evaluating the extent to which sentences carrying similar senses\nare embedded in close proximity sub-spaces, and if we can exploit that\nstructure to align sentences to a knowledge graph. We propose a set of\nexperiments using a widely-used large-scale data set for relation extraction\nand focusing on a set of key sentence embedding methods. We additionally\nprovide the code for reproducing these experiments at\nhttps://github.com/akalino/semantic-structural-sentences. These embedding\nmethods cover a wide variety of techniques ranging from simple word embedding\ncombination to transformer-based BERT-style model. Our experimental results\nshow that different embedding spaces have different degrees of strength for the\nstructural and semantic properties. These results provide useful information\nfor developing embedding-based relation extraction methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11264v2",
            "date": "2020-09-23 17:21:33+00:00",
            "title": "On the Ability and Limitations of Transformers to Recognize Formal Languages",
            "authors": [
                "Satwik Bhattamishra",
                "Kabir Ahuja",
                "Navin Goyal"
            ],
            "abstract": "Transformers have supplanted recurrent models in a large number of NLP tasks.\nHowever, the differences in their abilities to model different syntactic\nproperties remain largely unknown. Past works suggest that LSTMs generalize\nvery well on regular languages and have close connections with counter\nlanguages. In this work, we systematically study the ability of Transformers to\nmodel such languages as well as the role of its individual components in doing\nso. We first provide a construction of Transformers for a subclass of counter\nlanguages, including well-studied languages such as n-ary Boolean Expressions,\nDyck-1, and its generalizations. In experiments, we find that Transformers do\nwell on this subclass, and their learned mechanism strongly correlates with our\nconstruction. Perhaps surprisingly, in contrast to LSTMs, Transformers do well\nonly on a subset of regular languages with degrading performance as we make\nlanguages more complex according to a well-known measure of complexity. Our\nanalysis also provides insights on the role of self-attention mechanism in\nmodeling certain behaviors and the influence of positional encoding schemes on\nthe learning and generalization abilities of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11352v1",
            "date": "2020-09-23 19:48:02+00:00",
            "title": "ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)",
            "authors": [
                "Mohammad Aliannejadi",
                "Julia Kiseleva",
                "Aleksandr Chuklin",
                "Jeff Dalton",
                "Mikhail Burtsev"
            ],
            "abstract": "This document presents a detailed description of the challenge on clarifying\nquestions for dialogue systems (ClariQ). The challenge is organized as part of\nthe Conversational AI challenge series (ConvAI3) at Search Oriented\nConversational AI (SCAI) EMNLP workshop in 2020. The main aim of the\nconversational systems is to return an appropriate answer in response to the\nuser requests. However, some user requests might be ambiguous. In IR settings\nsuch a situation is handled mainly thought the diversification of the search\nresult page. It is however much more challenging in dialogue settings with\nlimited bandwidth. Therefore, in this challenge, we provide a common evaluation\nframework to evaluate mixed-initiative conversations. Participants are asked to\nrank clarifying questions in an information-seeking conversations. The\nchallenge is organized in two stages where in Stage 1 we evaluate the\nsubmissions in an offline setting and single-turn conversations. Top\nparticipants of Stage 1 get the chance to have their model tested by human\nannotators.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11485v1",
            "date": "2020-09-24 04:39:38+00:00",
            "title": "CogniFNN: A Fuzzy Neural Network Framework for Cognitive Word Embedding Evaluation",
            "authors": [
                "Xinping Liu",
                "Zehong Cao",
                "Son Tran"
            ],
            "abstract": "Word embeddings can reflect the semantic representations, and the embedding\nqualities can be comprehensively evaluated with human natural reading-related\ncognitive data sources. In this paper, we proposed the CogniFNN framework,\nwhich is the first attempt at using fuzzy neural networks to extract non-linear\nand non-stationary characteristics for evaluations of English word embeddings\nagainst the corresponding cognitive datasets. In our experiment, we used 15\nhuman cognitive datasets across three modalities: EEG, fMRI, and eye-tracking,\nand selected the mean square error and multiple hypotheses testing as metrics\nto evaluate our proposed CogniFNN framework. Compared to the recent pioneer\nframework, our proposed CogniFNN showed smaller prediction errors of both\ncontext-independent (GloVe) and context-sensitive (BERT) word embeddings, and\nachieved higher significant ratios with randomly generated word embeddings. Our\nfindings suggested that the CogniFNN framework could provide a more accurate\nand comprehensive evaluation of cognitive word embeddings. It will potentially\nbe beneficial to the further word embeddings evaluation on extrinsic natural\nlanguage processing tasks.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.11684v1",
            "date": "2020-09-24 13:40:18+00:00",
            "title": "AliMe KG: Domain Knowledge Graph Construction and Application in E-commerce",
            "authors": [
                "Feng-Lin Li",
                "Hehong Chen",
                "Guohai Xu",
                "Tian Qiu",
                "Feng Ji",
                "Ji Zhang",
                "Haiqing Chen"
            ],
            "abstract": "Pre-sales customer service is of importance to E-commerce platforms as it\ncontributes to optimizing customers' buying process. To better serve users, we\npropose AliMe KG, a domain knowledge graph in E-commerce that captures user\nproblems, points of interests (POI), item information and relations thereof. It\nhelps to understand user needs, answer pre-sales questions and generate\nexplanation texts. We applied AliMe KG to several online business scenarios\nsuch as shopping guide, question answering over properties and recommendation\nreason generation, and gained positive results. In the paper, we systematically\nintroduce how we construct domain knowledge graph from free text, and\ndemonstrate its business value with several applications. Our experience shows\nthat mining structured knowledge from free text in vertical domain is\npracticable, and can be of substantial value in industrial settings.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.11795v2",
            "date": "2020-09-24 16:37:04+00:00",
            "title": "Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences",
            "authors": [
                "Boon Peng Yap",
                "Andrew Koh",
                "Eng Siong Chng"
            ],
            "abstract": "Domain adaptation or transfer learning using pre-trained language models such\nas BERT has proven to be an effective approach for many natural language\nprocessing tasks. In this work, we propose to formulate word sense\ndisambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair\nranking task to select the most probable sense definition given a context\nsentence and a list of candidate sense definitions. We also introduce a data\naugmentation technique for WSD using existing example sentences from WordNet.\nUsing the proposed training objective and data augmentation technique, our\nmodels are able to achieve state-of-the-art results on the English all-words\nbenchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11963v1",
            "date": "2020-09-24 21:56:02+00:00",
            "title": "Toward a Thermodynamics of Meaning",
            "authors": [
                "Jonathan Scott Enderle"
            ],
            "abstract": "As language models such as GPT-3 become increasingly successful at generating\nrealistic text, questions about what purely text-based modeling can learn about\nthe world have become more urgent. Is text purely syntactic, as skeptics argue?\nOr does it in fact contain some semantic information that a sufficiently\nsophisticated language model could use to learn about the world without any\nadditional inputs? This paper describes a new model that suggests some\nqualified answers to those questions. By theorizing the relationship between\ntext and the world it describes as an equilibrium relationship between a\nthermodynamic system and a much larger reservoir, this paper argues that even\nvery simple language models do learn structural facts about the world, while\nalso proposing relatively precise limits on the nature and extent of those\nfacts. This perspective promises not only to answer questions about what\nlanguage models actually learn, but also to explain the consistent and\nsurprising success of cooccurrence prediction as a meaning-making strategy in\nAI.",
            "journal ref": "Proceedings of the Workshop on Computational Humanities Research\n  CEUR Vol-2723 (2020) 191-201",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.11982v2",
            "date": "2020-09-24 23:47:18+00:00",
            "title": "Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias",
            "authors": [
                "Ana Valeria Gonzalez",
                "Maria Barrett",
                "Rasmus Hvingelby",
                "Kellie Webster",
                "Anders S\u00f8gaard"
            ],
            "abstract": "The one-sided focus on English in previous studies of gender bias in NLP\nmisses out on opportunities in other languages: English challenge datasets such\nas GAP and WinoGender highlight model preferences that are \"hallucinatory\",\ne.g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors.\nWe show that for languages with type B reflexivization, e.g., Swedish and\nRussian, we can construct multi-task challenge datasets for detecting gender\nbias that lead to unambiguously wrong model predictions: In these languages,\nthe direct translation of 'the doctor removed his mask' is not ambiguous\nbetween a coreferential reading and a disjoint reading. Instead, the\ncoreferential reading requires a non-gendered pronoun, and the gendered,\npossessive pronouns are anti-reflexive. We present a multilingual, multi-task\nchallenge dataset, which spans four languages and four NLP tasks and focuses\nonly on this phenomenon. We find evidence for gender bias across all\ntask-language combinations and correlate model bias with national labor market\nstatistics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12005v2",
            "date": "2020-09-25 02:19:13+00:00",
            "title": "MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems",
            "authors": [
                "Zhaojiang Lin",
                "Andrea Madotto",
                "Genta Indra Winata",
                "Pascale Fung"
            ],
            "abstract": "In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify\nthe system design process of task-oriented dialogue systems and alleviate the\nover-dependency on annotated data. MinTL is a simple yet effective transfer\nlearning framework, which allows us to plug-and-play pre-trained seq2seq\nmodels, and jointly learn dialogue state tracking and dialogue response\ngeneration. Unlike previous approaches, which use a copy mechanism to\n\"carryover\" the old dialogue states to the new one, we introduce Levenshtein\nbelief spans (Lev), that allows efficient dialogue state tracking with a\nminimal generation length. We instantiate our learning framework with two\npre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive\nexperiments demonstrate that: 1) our systems establish new state-of-the-art\nresults on end-to-end response generation, 2) MinTL-based systems are more\nrobust than baseline methods in the low resource setting, and they achieve\ncompetitive results with only 20\\% training data, and 3) Lev greatly improves\nthe inference efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12013v2",
            "date": "2020-09-25 03:28:07+00:00",
            "title": "Revealing the Myth of Higher-Order Inference in Coreference Resolution",
            "authors": [
                "Liyan Xu",
                "Jinho D. Choi"
            ],
            "abstract": "This paper analyzes the impact of higher-order inference (HOI) on the task of\ncoreference resolution. HOI has been adapted by almost all recent coreference\nresolution models without taking much investigation on its true effectiveness\nover representation learning. To make a comprehensive analysis, we implement an\nend-to-end coreference system as well as four HOI approaches, attended\nantecedent, entity equalization, span clustering, and cluster merging, where\nthe latter two are our original methods. We find that given a high-performing\nencoder such as SpanBERT, the impact of HOI is negative to marginal, providing\na new perspective of HOI to this task. Our best model using cluster merging\nshows the Avg-F1 of 80.2 on the CoNLL 2012 shared task dataset in English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12030v2",
            "date": "2020-09-25 04:27:35+00:00",
            "title": "AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding",
            "authors": [
                "Guanglin Niu",
                "Bo Li",
                "Yongfei Zhang",
                "Shiliang Pu",
                "Jingyang Li"
            ],
            "abstract": "Recent advances in Knowledge Graph Embedding (KGE) allow for representing\nentities and relations in continuous vector spaces. Some traditional KGE models\nleveraging additional type information can improve the representation of\nentities which however totally rely on the explicit types or neglect the\ndiverse type representations specific to various relations. Besides, none of\nthe existing methods is capable of inferring all the relation patterns of\nsymmetry, inversion and composition as well as the complex properties of 1-N,\nN-1 and N-N relations, simultaneously. To explore the type information for any\nKG, we develop a novel KGE framework with Automated Entity TypE Representation\n(AutoETER), which learns the latent type embedding of each entity by regarding\neach relation as a translation operation between the types of two entities with\na relation-aware projection mechanism. Particularly, our designed automated\ntype representation learning mechanism is a pluggable module which can be\neasily incorporated with any KGE model. Besides, our approach could model and\ninfer all the relation patterns and complex relations. Experiments on four\ndatasets demonstrate the superior performance of our model compared to\nstate-of-the-art baselines on link prediction tasks, and the visualization of\ntype clustering provides clearly the explanation of type embeddings and\nverifies the effectiveness of our model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12056v2",
            "date": "2020-09-25 06:57:52+00:00",
            "title": "No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension",
            "authors": [
                "Xuguang Wang",
                "Linjun Shou",
                "Ming Gong",
                "Nan Duan",
                "Daxin Jiang"
            ],
            "abstract": "The Natural Questions (NQ) benchmark set brings new challenges to Machine\nReading Comprehension: the answers are not only at different levels of\ngranularity (long and short), but also of richer types (including no-answer,\nyes/no, single-span and multi-span). In this paper, we target at this challenge\nand handle all answer types systematically. In particular, we propose a novel\napproach called Reflection Net which leverages a two-step training procedure to\nidentify the no-answer and wrong-answer cases. Extensive experiments are\nconducted to verify the effectiveness of our approach. At the time of paper\nwriting (May.~20,~2020), our approach achieved the top 1 on both long and short\nanswer leaderboard, with F1 scores of 77.2 and 64.1, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12061v2",
            "date": "2020-09-25 07:16:51+00:00",
            "title": "An Unsupervised Sentence Embedding Method by Mutual Information Maximization",
            "authors": [
                "Yan Zhang",
                "Ruidan He",
                "Zuozhu Liu",
                "Kwan Hui Lim",
                "Lidong Bing"
            ],
            "abstract": "BERT is inefficient for sentence-pair tasks such as clustering or semantic\nsearch as it needs to evaluate combinatorially many sentence pairs which is\nvery time-consuming. Sentence BERT (SBERT) attempted to solve this challenge by\nlearning semantically meaningful representations of single sentences, such that\nsimilarity comparison can be easily accessed. However, SBERT is trained on\ncorpus with high-quality labeled sentence pairs, which limits its application\nto tasks where labeled data is extremely scarce. In this paper, we propose a\nlightweight extension on top of BERT and a novel self-supervised learning\nobjective based on mutual information maximization strategies to derive\nmeaningful sentence embeddings in an unsupervised manner. Unlike SBERT, our\nmethod is not restricted by the availability of labeled data, such that it can\nbe applied on different domain-specific corpus. Experimental results show that\nthe proposed method significantly outperforms other unsupervised sentence\nembedding baselines on common semantic textual similarity (STS) tasks and\ndownstream supervised tasks. It also outperforms SBERT in a setting where\nin-domain labeled data is not available, and achieves performance competitive\nwith supervised methods on various tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12081v2",
            "date": "2020-09-25 08:13:07+00:00",
            "title": "The algebra of non-deterministic programs: demonic operators, orders and axioms",
            "authors": [
                "Robin Hirsch",
                "Szabolcs Mikul\u00e1s",
                "Tim Stokes"
            ],
            "abstract": "Demonic composition, demonic refinement and demonic union are alternatives to\nthe usual \"angelic\" composition, angelic refinement (inclusion) and angelic\n(usual) union defined on binary relations. We first motivate both the angelic\nand demonic via an analysis of the behaviour of non-deterministic programs,\nwith the angelic associated with partial correctness and demonic with total\ncorrectness, both cases emerging from a richer algebraic model of\nnon-deterministic programs incorporating both aspects. Zareckii has shown that\nthe isomorphism class of algebras of binary relations under angelic composition\nand inclusion is finitely axiomatised as the class of ordered semigroups. The\nproof can be used to establish that the same axiomatisation applies to binary\nrelations under demonic composition and refinement, and a further modification\nof the proof can be used to incorporate a zero element representing the empty\nrelation in the angelic case and the full relation in the demonic case. For the\nsignature of angelic composition and union, it is known that no finite\naxiomatisation exists, and we show the analogous result for demonic composition\nand demonic union by showing that the same axiomatisation holds for both. We\nshow that the isomorphism class of algebras of binary relations with the\n\"mixed\" signature of demonic composition and angelic inclusion has no finite\naxiomatisation. As a contrast, we show that the isomorphism class of partial\nalgebras of binary relations with the partial operation of constellation\nproduct and inclusion (also a \"mixed\" signature) is finitely axiomatisable.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/2009.12240v1",
            "date": "2020-09-25 13:56:20+00:00",
            "title": "Weird AI Yankovic: Generating Parody Lyrics",
            "authors": [
                "Mark Riedl"
            ],
            "abstract": "Lyrics parody swaps one set of words that accompany a melody with a new set\nof words, preserving the number of syllables per line and the rhyme scheme.\nLyrics parody generation is a challenge for controllable text generation. We\nshow how a specialized sampling procedure, combined with backward text\ngeneration with XLNet can produce parody lyrics that reliably meet the syllable\nand rhyme scheme constraints.We introduce the Weird AI Yankovic system and\nprovide a case study evaluation. We conclude with societal implications of\nneural lyric parody generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12313v2",
            "date": "2020-09-25 16:09:08+00:00",
            "title": "Are scene graphs good enough to improve Image Captioning?",
            "authors": [
                "Victor Milewski",
                "Marie-Francine Moens",
                "Iacer Calixto"
            ],
            "abstract": "Many top-performing image captioning models rely solely on object features\ncomputed with an object detection model to generate image descriptions.\nHowever, recent studies propose to directly use scene graphs to introduce\ninformation about object relations into captioning, hoping to better describe\ninteractions between objects. In this work, we thoroughly investigate the use\nof scene graphs in image captioning. We empirically study whether using\nadditional scene graph encoders can lead to better image descriptions and\npropose a conditional graph attention network (C-GAT), where the image\ncaptioning decoder state is used to condition the graph updates. Finally, we\ndetermine to what extent noise in the predicted scene graphs influence caption\nquality. Overall, we find no significant difference between models that use\nscene graph features and models that only use object detection features across\ndifferent captioning metrics, which suggests that existing scene graph\ngeneration models are still too noisy to be useful in image captioning.\nMoreover, although the quality of predicted scene graphs is very low in\ngeneral, when using high quality scene graphs we obtain gains of up to 3.3\nCIDEr compared to a strong Bottom-Up Top-Down baseline. We open source code to\nreproduce all our experiments in\nhttps://github.com/iacercalixto/butd-image-captioning.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2009.12404v1",
            "date": "2020-09-25 19:07:00+00:00",
            "title": "Visually Grounded Compound PCFGs",
            "authors": [
                "Yanpeng Zhao",
                "Ivan Titov"
            ],
            "abstract": "Exploiting visual groundings for language understanding has recently been\ndrawing much attention. In this work, we study visually grounded grammar\ninduction and learn a constituency parser from both unlabeled text and its\nvisual groundings. Existing work on this task (Shi et al., 2019) optimizes a\nparser via Reinforce and derives the learning signal only from the alignment of\nimages and sentences. While their model is relatively accurate overall, its\nerror distribution is very uneven, with low performance on certain constituents\ntypes (e.g., 26.2% recall on verb phrases, VPs) and high on others (e.g., 79.6%\nrecall on noun phrases, NPs). This is not surprising as the learning signal is\nlikely insufficient for deriving all aspects of phrase-structure syntax and\ngradient estimates are noisy. We show that using an extension of probabilistic\ncontext-free grammar model we can do fully-differentiable end-to-end visually\ngrounded learning. Additionally, this enables us to complement the image-text\nalignment loss with a language modeling objective. On the MSCOCO test captions,\nour model establishes a new state of the art, outperforming its non-grounded\nversion and, thus, confirming the effectiveness of visual groundings in\nconstituency grammar induction. It also substantially outperforms the previous\ngrounded model, with largest improvements on more `abstract' categories (e.g.,\n+55.1% recall on VPs).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12421v2",
            "date": "2020-09-25 20:08:32+00:00",
            "title": "Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders",
            "authors": [
                "Victor Prokhorov",
                "Yingzhen Li",
                "Ehsan Shareghi",
                "Nigel Collier"
            ],
            "abstract": "It has been long known that sparsity is an effective inductive bias for\nlearning efficient representation of data in vectors with fixed dimensionality,\nand it has been explored in many areas of representation learning. Of\nparticular interest to this work is the investigation of the sparsity within\nthe VAE framework which has been explored a lot in the image domain, but has\nbeen lacking even a basic level of exploration in NLP. Additionally, NLP is\nalso lagging behind in terms of learning sparse representations of large units\nof text e.g., sentences. We use the VAEs that induce sparse latent\nrepresentations of large units of text to address the aforementioned\nshortcomings. First, we move in this direction by measuring the success of\nunsupervised state-of-the-art (SOTA) and other strong VAE-based sparsification\nbaselines for text and propose a hierarchical sparse VAE model to address the\nstability issue of SOTA. Then, we look at the implications of sparsity on text\nclassification across 3 datasets, and highlight a link between performance of\nsparse latent representations on downstream tasks and its ability to encode\ntask-related information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12496v1",
            "date": "2020-09-26 01:25:42+00:00",
            "title": "Modeling Dyadic Conversations for Personality Inference",
            "authors": [
                "Qiang Liu"
            ],
            "abstract": "Nowadays, automatical personality inference is drawing extensive attention\nfrom both academia and industry. Conventional methods are mainly based on user\ngenerated contents, e.g., profiles, likes, and texts of an individual, on\nsocial media, which are actually not very reliable. In contrast, dyadic\nconversations between individuals can not only capture how one expresses\noneself, but also reflect how one reacts to different situations. Rich\ncontextual information in dyadic conversation can explain an individual's\nresponse during his or her conversation. In this paper, we propose a novel\naugmented Gated Recurrent Unit (GRU) model for learning unsupervised Personal\nConversational Embeddings (PCE) based on dyadic conversations between\nindividuals. We adjust the formulation of each layer of a conventional GRU with\nsequence to sequence learning and personal information of both sides of the\nconversation. Based on the learned PCE, we can infer the personality of each\nindividual. We conduct experiments on the Movie Script dataset, which is\ncollected from conversations between characters in movie scripts. We find that\nmodeling dyadic conversations between individuals can significantly improve\npersonality inference accuracy. Experimental results illustrate the successful\nperformance of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12565v1",
            "date": "2020-09-26 11:00:35+00:00",
            "title": "Metaphor Detection using Deep Contextualized Word Embeddings",
            "authors": [
                "Shashwat Aggarwal",
                "Ramesh Singh"
            ],
            "abstract": "Metaphors are ubiquitous in natural language, and their detection plays an\nessential role in many natural language processing tasks, such as language\nunderstanding, sentiment analysis, etc. Most existing approaches for metaphor\ndetection rely on complex, hand-crafted and fine-tuned feature pipelines, which\ngreatly limit their applicability. In this work, we present an end-to-end\nmethod composed of deep contextualized word embeddings, bidirectional LSTMs and\nmulti-head attention mechanism to address the task of automatic metaphor\ndetection. Our method, unlike many other existing approaches, requires only the\nraw text sequences as input features to detect the metaphoricity of a phrase.\nWe compare the performance of our method against the existing baselines on two\nbenchmark datasets, TroFi, and MOH-X respectively. Experimental evaluations\nconfirm the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12622v1",
            "date": "2020-09-26 15:33:16+00:00",
            "title": "Automatic Arabic Dialect Identification Systems for Written Texts: A Survey",
            "authors": [
                "Maha J. Althobaiti"
            ],
            "abstract": "Arabic dialect identification is a specific task of natural language\nprocessing, aiming to automatically predict the Arabic dialect of a given text.\nArabic dialect identification is the first step in various natural language\nprocessing applications such as machine translation, multilingual\ntext-to-speech synthesis, and cross-language text generation. Therefore, in the\nlast decade, interest has increased in addressing the problem of Arabic dialect\nidentification. In this paper, we present a comprehensive survey of Arabic\ndialect identification research in written texts. We first define the problem\nand its challenges. Then, the survey extensively discusses in a critical manner\nmany aspects related to Arabic dialect identification task. So, we review the\ntraditional machine learning methods, deep learning architectures, and complex\nlearning approaches to Arabic dialect identification. We also detail the\nfeatures and techniques for feature representations used to train the proposed\nsystems. Moreover, we illustrate the taxonomy of Arabic dialects studied in the\nliterature, the various levels of text processing at which Arabic dialect\nidentification are conducted (e.g., token, sentence, and document level), as\nwell as the available annotated resources, including evaluation benchmark\ncorpora. Open challenges and issues are discussed at the end of the survey.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12677v2",
            "date": "2020-09-26 19:57:49+00:00",
            "title": "KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning",
            "authors": [
                "Ye Liu",
                "Yao Wan",
                "Lifang He",
                "Hao Peng",
                "Philip S. Yu"
            ],
            "abstract": "Generative commonsense reasoning which aims to empower machines to generate\nsentences with the capacity of reasoning over a set of concepts is a critical\nbottleneck for text generation. Even the state-of-the-art pre-trained language\ngeneration models struggle at this task and often produce implausible and\nanomalous sentences. One reason is that they rarely consider incorporating the\nknowledge graph which can provide rich relational information among the\ncommonsense concepts. To promote the ability of commonsense reasoning for text\ngeneration, we propose a novel knowledge graph augmented pre-trained language\ngeneration model KG-BART, which encompasses the complex relations of concepts\nthrough the knowledge graph and produces more logical and natural sentences as\noutput. Moreover, KG-BART can leverage the graph attention to aggregate the\nrich concept semantics that enhances the model generalization on unseen concept\nsets. Experiments on benchmark CommonGen dataset verify the effectiveness of\nour proposed approach by comparing with several strong pre-trained language\ngeneration models, particularly KG-BART outperforms BART by 5.80, 4.60, in\nterms of BLEU-3, 4. Moreover, we also show that the generated context by our\nmodel can work as background scenarios to benefit downstream commonsense QA\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12711v1",
            "date": "2020-09-27 00:02:34+00:00",
            "title": "Local and non-local dependency learning and emergence of rule-like representations in speech data by Deep Convolutional Generative Adversarial Networks",
            "authors": [
                "Ga\u0161per Begu\u0161"
            ],
            "abstract": "This paper argues that training GANs on local and non-local dependencies in\nspeech data offers insights into how deep neural networks discretize continuous\ndata and how symbolic-like rule-based morphophonological processes emerge in a\ndeep convolutional architecture. Acquisition of speech has recently been\nmodeled as a dependency between latent space and data generated by GANs in\nBegu\\v{s} (arXiv:2006.03965), who models learning of a simple local allophonic\ndistribution. We extend this approach to test learning of local and non-local\nphonological processes that include approximations of morphological processes.\nWe further parallel outputs of the model to results of a behavioral experiment\nwhere human subjects are trained on the data used for training the GAN network.\nFour main conclusions emerge: (i) the networks provide useful information for\ncomputational models of language acquisition even if trained on a comparatively\nsmall dataset of an artificial grammar learning experiment; (ii) local\nprocesses are easier to learn than non-local processes, which matches both\nbehavioral data in human subjects and typology in the world's languages. This\npaper also proposes (iii) how we can actively observe the network's progress in\nlearning and explore the effect of training steps on learning representations\nby keeping latent space constant across different training steps. Finally, this\npaper shows that (iv) the network learns to encode the presence of a prefix\nwith a single latent variable; by interpolating this variable, we can actively\nobserve the operation of a non-local phonological process. The proposed\ntechnique for retrieving learning representations has general implications for\nour understanding of how GANs discretize continuous speech data and suggests\nthat rule-like generalizations in the training data are represented as an\ninteraction between variables in the network's latent space.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12727v2",
            "date": "2020-09-27 02:13:38+00:00",
            "title": "Multi-timescale Representation Learning in LSTM Language Models",
            "authors": [
                "Shivangi Mahto",
                "Vy A. Vo",
                "Javier S. Turek",
                "Alexander G. Huth"
            ],
            "abstract": "Language models must capture statistical dependencies between words at\ntimescales ranging from very short to very long. Earlier work has demonstrated\nthat dependencies in natural language tend to decay with distance between words\naccording to a power law. However, it is unclear how this knowledge can be used\nfor analyzing or designing neural network language models. In this work, we\nderived a theory for how the memory gating mechanism in long short-term memory\n(LSTM) language models can capture power law decay. We found that unit\ntimescales within an LSTM, which are determined by the forget gate bias, should\nfollow an Inverse Gamma distribution. Experiments then showed that LSTM\nlanguage models trained on natural English text learn to approximate this\ntheoretical distribution. Further, we found that explicitly imposing the\ntheoretical distribution upon the model during training yielded better language\nmodel perplexity overall, with particular improvements for predicting\nlow-frequency (rare) words. Moreover, the explicit multi-timescale model\nselectively routes information about different types of words through units\nwith different timescales, potentially improving model interpretability. These\nresults demonstrate the importance of careful, theoretically-motivated analysis\nof memory and timescale in language models.",
            "journal ref": "International Conference on Learning Representations 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.12997v1",
            "date": "2020-09-28 01:05:08+00:00",
            "title": "Fancy Man Lauches Zippo at WNUT 2020 Shared Task-1: A Bert Case Model for Wet Lab Entity Extraction",
            "authors": [
                "Haoding Meng",
                "Qingcheng Zeng",
                "Xiaoyang Fang",
                "Zhexin Liang"
            ],
            "abstract": "Automatic or semi-automatic conversion of protocols specifying steps in\nperforming a lab procedure into machine-readable format benefits biological\nresearch a lot. These noisy, dense, and domain-specific lab protocols\nprocessing draws more and more interests with the development of deep learning.\nThis paper presents our teamwork on WNUT 2020 shared task-1: wet lab entity\nextract, that we conducted studies in several models, including a BiLSTM CRF\nmodel and a Bert case model which can be used to complete wet lab entity\nextraction. And we mainly discussed the performance differences of \\textbf{Bert\ncase} under different situations such as \\emph{transformers} versions, case\nsensitivity that may don't get enough attention before.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13013v1",
            "date": "2020-09-28 02:11:02+00:00",
            "title": "SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval",
            "authors": [
                "Tiancheng Zhao",
                "Xiaopeng Lu",
                "Kyusong Lee"
            ],
            "abstract": "We introduce SPARTA, a novel neural retrieval method that shows great promise\nin performance, generalization, and interpretability for open-domain question\nanswering. Unlike many neural ranking methods that use dense vector nearest\nneighbor search, SPARTA learns a sparse representation that can be efficiently\nimplemented as an Inverted Index. The resulting representation enables scalable\nneural retrieval that does not require expensive approximate vector search and\nleads to better performance than its dense counterpart. We validated our\napproaches on 4 open-domain question answering (OpenQA) tasks and 11 retrieval\nquestion answering (ReQA) tasks. SPARTA achieves new state-of-the-art results\nacross a variety of open-domain question answering tasks in both English and\nChinese datasets, including open SQuAD, Natuarl Question, CMRC and etc.\nAnalysis also confirms that the proposed method creates human interpretable\nrepresentation and allows flexible control over the trade-off between\nperformance and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13080v1",
            "date": "2020-09-28 05:04:22+00:00",
            "title": "Reactive Supervision: A New Method for Collecting Sarcasm Data",
            "authors": [
                "Boaz Shmueli",
                "Lun-Wei Ku",
                "Soumya Ray"
            ],
            "abstract": "Sarcasm detection is an important task in affective computing, requiring\nlarge amounts of labeled data. We introduce reactive supervision, a novel data\ncollection method that utilizes the dynamics of online conversations to\novercome the limitations of existing data collection techniques. We use the new\nmethod to create and release a first-of-its-kind large dataset of tweets with\nsarcasm perspective labels and new contextual features. The dataset is expected\nto advance sarcasm detection research. Our method can be adapted to other\naffective computing domains, thus opening up new research opportunities.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13081v1",
            "date": "2020-09-28 05:07:51+00:00",
            "title": "What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams",
            "authors": [
                "Di Jin",
                "Eileen Pan",
                "Nassim Oufattole",
                "Wei-Hung Weng",
                "Hanyi Fang",
                "Peter Szolovits"
            ],
            "abstract": "Open domain question answering (OpenQA) tasks have been recently attracting\nmore and more attention from the natural language processing (NLP) community.\nIn this work, we present the first free-form multiple-choice OpenQA dataset for\nsolving medical problems, MedQA, collected from the professional medical board\nexams. It covers three languages: English, simplified Chinese, and traditional\nChinese, and contains 12,723, 34,251, and 14,123 questions for the three\nlanguages, respectively. We implement both rule-based and popular neural\nmethods by sequentially combining a document retriever and a machine\ncomprehension model. Through experiments, we find that even the current best\nmethod can only achieve 36.7\\%, 42.0\\%, and 70.1\\% of test accuracy on the\nEnglish, traditional Chinese, and simplified Chinese questions, respectively.\nWe expect MedQA to present great challenges to existing OpenQA systems and hope\nthat it can serve as a platform to promote much stronger OpenQA models from the\nNLP community in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13102v2",
            "date": "2020-09-28 07:13:23+00:00",
            "title": "Deep Transformers with Latent Depth",
            "authors": [
                "Xian Li",
                "Asa Cooper Stickland",
                "Yuqing Tang",
                "Xiang Kong"
            ],
            "abstract": "The Transformer model has achieved state-of-the-art performance in many\nsequence modeling tasks. However, how to leverage model capacity with large or\nvariable depths is still an open challenge. We present a probabilistic\nframework to automatically learn which layer(s) to use by learning the\nposterior distributions of layer selection. As an extension of this framework,\nwe propose a novel method to train one shared Transformer network for\nmultilingual machine translation with different layer selection posteriors for\neach language pair. The proposed method alleviates the vanishing gradient issue\nand enables stable training of deep Transformers (e.g. 100 layers). We evaluate\non WMT English-German machine translation and masked language modeling tasks,\nwhere our method outperforms existing approaches for training deeper\nTransformers. Experiments on multilingual machine translation demonstrate that\nthis approach can effectively leverage increased model capacity and bring\nuniversal improvement for both many-to-one and one-to-many translation with\ndiverse language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13116v1",
            "date": "2020-09-28 07:51:03+00:00",
            "title": "Neural Baselines for Word Alignment",
            "authors": [
                "Anh Khoa Ngo Ho",
                "Fran\u00e7ois Yvon"
            ],
            "abstract": "Word alignments identify translational correspondences between words in a\nparallel sentence pair and is used, for instance, to learn bilingual\ndictionaries, to train statistical machine translation systems , or to perform\nquality estimation. In most areas of natural language processing, neural\nnetwork models nowadays constitute the preferred approach, a situation that\nmight also apply to word alignment models. In this work, we study and\ncomprehensively evaluate neural models for unsupervised word alignment for four\nlanguage pairs, contrasting several variants of neural models. We show that in\nmost settings, neural versions of the IBM-1 and hidden Markov models vastly\noutperform their discrete counterparts. We also analyze typical alignment\nerrors of the baselines that our models overcome to illustrate the benefits-and\nthe limitations-of these new models for morphologically rich languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13117v1",
            "date": "2020-09-28 07:54:09+00:00",
            "title": "Generative latent neural models for automatic word alignment",
            "authors": [
                "Anh Khoa Ngo Ho",
                "Fran\u00e7ois Yvon"
            ],
            "abstract": "Word alignments identify translational correspondences between words in a\nparallel sentence pair and are used, for instance, to learn bilingual\ndictionaries, to train statistical machine translation systems or to perform\nquality estimation. Variational autoencoders have been recently used in various\nof natural language processing to learn in an unsupervised way latent\nrepresentations that are useful for language generation tasks. In this paper,\nwe study these models for the task of word alignment and propose and assess\nseveral evolutions of a vanilla variational autoencoders. We demonstrate that\nthese techniques can yield competitive results as compared to Giza++ and to a\nstrong neural network alignment system for two language pairs.",
            "journal ref": "The Association for Machine Translation in the Americas, Oct 2020,\n  Florida, United States",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13166v1",
            "date": "2020-09-28 09:29:49+00:00",
            "title": "Incomplete Utterance Rewriting as Semantic Segmentation",
            "authors": [
                "Qian Liu",
                "Bei Chen",
                "Jian-Guang Lou",
                "Bin Zhou",
                "Dongmei Zhang"
            ],
            "abstract": "Recent years the task of incomplete utterance rewriting has raised a large\nattention. Previous works usually shape it as a machine translation task and\nemploy sequence to sequence based architecture with copy mechanism. In this\npaper, we present a novel and extensive approach, which formulates it as a\nsemantic segmentation task. Instead of generating from scratch, such a\nformulation introduces edit operations and shapes the problem as prediction of\na word-level edit matrix. Benefiting from being able to capture both local and\nglobal information, our approach achieves state-of-the-art performance on\nseveral public datasets. Furthermore, our approach is four times faster than\nthe standard approach in inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13199v2",
            "date": "2020-09-28 10:28:40+00:00",
            "title": "Knowledge-Aware Procedural Text Understanding with Multi-Stage Training",
            "authors": [
                "Zhihan Zhang",
                "Xiubo Geng",
                "Tao Qin",
                "Yunfang Wu",
                "Daxin Jiang"
            ],
            "abstract": "Procedural text describes dynamic state changes during a step-by-step natural\nprocess (e.g., photosynthesis). In this work, we focus on the task of\nprocedural text understanding, which aims to comprehend such documents and\ntrack entities' states and locations during a process. Although recent\napproaches have achieved substantial progress, their results are far behind\nhuman performance. Two challenges, the difficulty of commonsense reasoning and\ndata insufficiency, still remain unsolved, which require the incorporation of\nexternal knowledge bases. Previous works on external knowledge injection\nusually rely on noisy web mining tools and heuristic rules with limited\napplicable scenarios. In this paper, we propose a novel KnOwledge-Aware\nproceduraL text understAnding (KOALA) model, which effectively leverages\nmultiple forms of external knowledge in this task. Specifically, we retrieve\ninformative knowledge triples from ConceptNet and perform knowledge-aware\nreasoning while tracking the entities. Besides, we employ a multi-stage\ntraining schema which fine-tunes the BERT model over unlabeled data collected\nfrom Wikipedia before further fine-tuning it on the final model. Experimental\nresults on two procedural text datasets, ProPara and Recipes, verify the\neffectiveness of the proposed methods, in which our model achieves\nstate-of-the-art performance in comparison to various baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13275v1",
            "date": "2020-09-17 18:14:25+00:00",
            "title": "Zero-shot Multi-Domain Dialog State Tracking Using Descriptive Rules",
            "authors": [
                "Edgar Altszyler",
                "Pablo Brusco",
                "Nikoletta Basiou",
                "John Byrnes",
                "Dimitra Vergyri"
            ],
            "abstract": "In this work, we present a framework for incorporating descriptive logical\nrules in state-of-the-art neural networks, enabling them to learn how to handle\nunseen labels without the introduction of any new training data. The rules are\nintegrated into existing networks without modifying their architecture, through\nan additional term in the network's loss function that penalizes states of the\nnetwork that do not obey the designed rules. As a case of study, the framework\nis applied to an existing neural-based Dialog State Tracker. Our experiments\ndemonstrate that the inclusion of logical rules allows the prediction of unseen\nlabels, without deteriorating the predictive capacity of the original system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13284v3",
            "date": "2020-09-28 12:49:07+00:00",
            "title": "Pchatbot: A Large-Scale Dataset for Personalized Chatbot",
            "authors": [
                "Hongjin Qian",
                "Xiaohe Li",
                "Hanxun Zhong",
                "Yu Guo",
                "Yueyuan Ma",
                "Yutao Zhu",
                "Zhanliang Liu",
                "Zhicheng Dou",
                "Ji-Rong Wen"
            ],
            "abstract": "Natural language dialogue systems raise great attention recently. As many\ndialogue models are data-driven, high-quality datasets are essential to these\nsystems. In this paper, we introduce Pchatbot, a large-scale dialogue dataset\nthat contains two subsets collected from Weibo and Judicial forums\nrespectively. To adapt the raw dataset to dialogue systems, we elaborately\nnormalize the raw dataset via processes such as anonymization, deduplication,\nsegmentation, and filtering. The scale of Pchatbot is significantly larger than\nexisting Chinese datasets, which might benefit the data-driven models. Besides,\ncurrent dialogue datasets for personalized chatbot usually contain several\npersona sentences or attributes. Different from existing datasets, Pchatbot\nprovides anonymized user IDs and timestamps for both posts and responses. This\nenables the development of personalized dialogue models that directly learn\nimplicit user personality from the user's dialogue history. Our preliminary\nexperimental study benchmarks several state-of-the-art dialogue models to\nprovide a comparison for future work. The dataset can be publicly accessed at\nGithub.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13295v1",
            "date": "2020-09-25 12:01:53+00:00",
            "title": "A Diagnostic Study of Explainability Techniques for Text Classification",
            "authors": [
                "Pepa Atanasova",
                "Jakob Grue Simonsen",
                "Christina Lioma",
                "Isabelle Augenstein"
            ],
            "abstract": "Recent developments in machine learning have introduced models that approach\nhuman performance at the cost of increased architectural complexity. Efforts to\nmake the rationales behind the models' predictions transparent have inspired an\nabundance of new explainability techniques. Provided with an already trained\nmodel, they compute saliency scores for the words of an input instance.\nHowever, there exists no definitive guide on (i) how to choose such a technique\ngiven a particular application task and model architecture, and (ii) the\nbenefits and drawbacks of using each such technique. In this paper, we develop\na comprehensive list of diagnostic properties for evaluating existing\nexplainability techniques. We then employ the proposed list to compare a set of\ndiverse explainability techniques on downstream text classification tasks and\nneural network architectures. We also compare the saliency scores assigned by\nthe explainability techniques with human annotations of salient input regions\nto find relations between a model's performance and the agreement of its\nrationales with human ones. Overall, we find that the gradient-based\nexplanations perform best across tasks and model architectures, and we present\nfurther insights into the properties of the reviewed explainability techniques.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13367v2",
            "date": "2020-09-28 14:35:31+00:00",
            "title": "Similarity Detection Pipeline for Crawling a Topic Related Fake News Corpus",
            "authors": [
                "Inna Vogel",
                "Jeong-Eun Choi",
                "Meghana Meghana"
            ],
            "abstract": "Fake news detection is a challenging task aiming to reduce human time and\neffort to check the truthfulness of news. Automated approaches to combat fake\nnews, however, are limited by the lack of labeled benchmark datasets,\nespecially in languages other than English. Moreover, many publicly available\ncorpora have specific limitations that make them difficult to use. To address\nthis problem, our contribution is threefold. First, we propose a new, publicly\navailable German topic related corpus for fake news detection. To the best of\nour knowledge, this is the first corpus of its kind. In this regard, we\ndeveloped a pipeline for crawling similar news articles. As our third\ncontribution, we conduct different learning experiments to detect fake news.\nThe best performance was achieved using sentence level embeddings from SBERT in\ncombination with a Bi-LSTM (k=0.88).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13401v2",
            "date": "2020-09-28 15:19:28+00:00",
            "title": "Injecting Entity Types into Entity-Guided Text Generation",
            "authors": [
                "Xiangyu Dong",
                "Wenhao Yu",
                "Chenguang Zhu",
                "Meng Jiang"
            ],
            "abstract": "Recent successes in deep generative modeling have led to significant advances\nin natural language generation (NLG). Incorporating entities into neural\ngeneration models has demonstrated great improvements by assisting to infer the\nsummary topic and to generate coherent content. In order to enhance the role of\nentity in NLG, in this paper, we aim to model the entity type in the decoding\nphase to generate contextual words accurately. We develop a novel NLG model to\nproduce a target sequence (i.e., a news article) based on a given list of\nentities. The generation quality depends significantly on whether the input\nentities are logically connected and expressed in the output. Our model has a\nmulti-step decoder that injects the entity types into the process of entity\nmention generation. It first predicts the token of being a contextual word or\nan entity, then if an entity, predicts the entity mention. It effectively\nembeds the entity's meaning into hidden states, making the generated words\nprecise. Experiments on two public datasets demonstrate type injection performs\nbetter than type embedding concatenation baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13570v2",
            "date": "2020-09-28 18:36:23+00:00",
            "title": "DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue",
            "authors": [
                "Shikib Mehri",
                "Mihail Eric",
                "Dilek Hakkani-Tur"
            ],
            "abstract": "A long-standing goal of task-oriented dialogue research is the ability to\nflexibly adapt dialogue models to new domains. To progress research in this\ndirection, we introduce DialoGLUE (Dialogue Language Understanding Evaluation),\na public benchmark consisting of 7 task-oriented dialogue datasets covering 4\ndistinct natural language understanding tasks, designed to encourage dialogue\nresearch in representation-based transfer, domain adaptation, and\nsample-efficient task learning. We release several strong baseline models,\ndemonstrating performance improvements over a vanilla BERT architecture and\nstate-of-the-art results on 5 out of 7 tasks, by pre-training on a large\nopen-domain dialogue corpus and task-adaptive self-supervised training. Through\nthe DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we\nhope to facilitate progress towards the goal of developing more general\ntask-oriented dialogue models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13602v1",
            "date": "2020-09-10 11:37:00+00:00",
            "title": "Non-Pharmaceutical Intervention Discovery with Topic Modeling",
            "authors": [
                "Jonathan Smith",
                "Borna Ghotbi",
                "Seungeun Yi",
                "Mahboobeh Parsapoor"
            ],
            "abstract": "We consider the task of discovering categories of non-pharmaceutical\ninterventions during the evolving COVID-19 pandemic. We explore topic modeling\non two corpora with national and international scope. These models discover\nexisting categories when compared with human intervention labels while reduced\nhuman effort needed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13603v2",
            "date": "2020-09-28 20:09:40+00:00",
            "title": "Visual Pivoting for (Unsupervised) Entity Alignment",
            "authors": [
                "Fangyu Liu",
                "Muhao Chen",
                "Dan Roth",
                "Nigel Collier"
            ],
            "abstract": "This work studies the use of visual semantic representations to align\nentities in heterogeneous knowledge graphs (KGs). Images are natural components\nof many existing KGs. By combining visual knowledge with other auxiliary\ninformation, we show that the proposed new approach, EVA, creates a holistic\nentity representation that provides strong signals for cross-graph entity\nalignment. Besides, previous entity alignment methods require human labelled\nseed alignment, restricting availability. EVA provides a completely\nunsupervised solution by leveraging the visual similarity of entities to create\nan initial seed dictionary (visual pivots). Experiments on benchmark data sets\nDBP15k and DWY15k show that EVA offers state-of-the-art performance on both\nmonolingual and cross-lingual entity alignment tasks. Furthermore, we discover\nthat images are particularly useful to align long-tail KG entities, which\ninherently lack the structural contexts necessary for capturing the\ncorrespondences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13613v2",
            "date": "2020-09-28 20:35:00+00:00",
            "title": "Joint Spatio-Textual Reasoning for Answering Tourism Questions",
            "authors": [
                "Danish Contractor",
                "Shashank Goel",
                "Mausam",
                "Parag Singla"
            ],
            "abstract": "Our goal is to answer real-world tourism questions that seek\nPoints-of-Interest (POI) recommendations. Such questions express various kinds\nof spatial and non-spatial constraints, necessitating a combination of textual\nand spatial reasoning. In response, we develop the first joint spatio-textual\nreasoning model, which combines geo-spatial knowledge with information in\ntextual corpora to answer questions. We first develop a modular\nspatial-reasoning network that uses geo-coordinates of location names mentioned\nin a question, and of candidate answer POIs, to reason over only spatial\nconstraints. We then combine our spatial-reasoner with a textual reasoner in a\njoint model and present experiments on a real world POI recommendation task. We\nreport substantial improvements over existing models with-out joint\nspatio-textual reasoning.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.13655v1",
            "date": "2020-09-28 22:08:00+00:00",
            "title": "Conversational Semantic Parsing",
            "authors": [
                "Armen Aghajanyan",
                "Jean Maillard",
                "Akshat Shrivastava",
                "Keith Diedrick",
                "Mike Haeger",
                "Haoran Li",
                "Yashar Mehdad",
                "Ves Stoyanov",
                "Anuj Kumar",
                "Mike Lewis",
                "Sonal Gupta"
            ],
            "abstract": "The structured representation for semantic parsing in task-oriented assistant\nsystems is geared towards simple understanding of one-turn queries. Due to the\nlimitations of the representation, the session-based properties such as\nco-reference resolution and context carryover are processed downstream in a\npipelined system. In this paper, we propose a semantic representation for such\ntask-oriented conversational systems that can represent concepts such as\nco-reference and context carryover, enabling comprehensive understanding of\nqueries in a session. We release a new session-based, compositional\ntask-oriented parsing dataset of 20k sessions consisting of 60k utterances.\nUnlike Dialog State Tracking Challenges, the queries in the dataset have\ncompositional forms. We propose a new family of Seq2Seq models for the\nsession-based parsing above, which achieve better or comparable performance to\nthe current state-of-the-art on ATIS, SNIPS, TOP and DSTC2. Notably, we improve\nthe best known results on DSTC2 by up to 5 points for slot-carryover.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13656v1",
            "date": "2020-09-28 22:13:54+00:00",
            "title": "Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems",
            "authors": [
                "Andrea Madotto",
                "Samuel Cahyawijaya",
                "Genta Indra Winata",
                "Yan Xu",
                "Zihan Liu",
                "Zhaojiang Lin",
                "Pascale Fung"
            ],
            "abstract": "Task-oriented dialogue systems are either modularized with separate dialogue\nstate tracking (DST) and management steps or end-to-end trainable. In either\ncase, the knowledge base (KB) plays an essential role in fulfilling user\nrequests. Modularized systems rely on DST to interact with the KB, which is\nexpensive in terms of annotation and inference time. End-to-end systems use the\nKB directly as input, but they cannot scale when the KB is larger than a few\nhundred entries. In this paper, we propose a method to embed the KB, of any\nsize, directly into the model parameters. The resulting model does not require\nany DST or template responses, nor the KB as input, and it can dynamically\nupdate its KB via fine-tuning. We evaluate our solution in five task-oriented\ndialogue datasets with small, medium, and large KB size. Our experiments show\nthat end-to-end models can effectively embed knowledge bases in their\nparameters and achieve competitive performance in all evaluated datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13818v2",
            "date": "2020-09-29 07:08:35+00:00",
            "title": "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation",
            "authors": [
                "Dinghan Shen",
                "Mingzhi Zheng",
                "Yelong Shen",
                "Yanru Qu",
                "Weizhu Chen"
            ],
            "abstract": "Adversarial training has been shown effective at endowing the learned\nrepresentations with stronger generalization ability. However, it typically\nrequires expensive computation to determine the direction of the injected\nperturbations. In this paper, we introduce a set of simple yet effective data\naugmentation strategies dubbed cutoff, where part of the information within an\ninput sentence is erased to yield its restricted views (during the fine-tuning\nstage). Notably, this process relies merely on stochastic sampling and thus\nadds little computational overhead. A Jensen-Shannon Divergence consistency\nloss is further utilized to incorporate these augmented samples into the\ntraining objective in a principled manner. To verify the effectiveness of the\nproposed strategies, we apply cutoff to both natural language understanding and\ngeneration problems. On the GLUE benchmark, it is demonstrated that cutoff, in\nspite of its simplicity, performs on par or better than several competitive\nadversarial-based approaches. We further extend cutoff to machine translation\nand observe significant gains in BLEU scores (based upon the Transformer Base\nmodel). Moreover, cutoff consistently outperforms adversarial training and\nachieves state-of-the-art results on the IWSLT2014 German-English dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13827v1",
            "date": "2020-09-29 07:32:17+00:00",
            "title": "SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery",
            "authors": [
                "Jiaming Shen",
                "Wenda Qiu",
                "Jingbo Shang",
                "Michelle Vanni",
                "Xiang Ren",
                "Jiawei Han"
            ],
            "abstract": "Entity set expansion and synonym discovery are two critical NLP tasks.\nPrevious studies accomplish them separately, without exploring their\ninterdependencies. In this work, we hypothesize that these two tasks are\ntightly coupled because two synonymous entities tend to have similar\nlikelihoods of belonging to various semantic classes. This motivates us to\ndesign SynSetExpan, a novel framework that enables two tasks to mutually\nenhance each other. SynSetExpan uses a synonym discovery model to include\npopular entities' infrequent synonyms into the set, which boosts the set\nexpansion recall. Meanwhile, the set expansion model, being able to determine\nwhether an entity belongs to a semantic class, can generate pseudo training\ndata to fine-tune the synonym discovery model towards better accuracy. To\nfacilitate the research on studying the interplays of these two tasks, we\ncreate the first large-scale Synonym-Enhanced Set Expansion (SE2) dataset via\ncrowdsourcing. Extensive experiments on the SE2 dataset and previous benchmarks\ndemonstrate the effectiveness of SynSetExpan for both entity set expansion and\nsynonym discovery tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13845v2",
            "date": "2020-09-29 08:17:58+00:00",
            "title": "GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing",
            "authors": [
                "Tao Yu",
                "Chien-Sheng Wu",
                "Xi Victoria Lin",
                "Bailin Wang",
                "Yi Chern Tan",
                "Xinyi Yang",
                "Dragomir Radev",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "We present GraPPa, an effective pre-training approach for table semantic\nparsing that learns a compositional inductive bias in the joint representations\nof textual and tabular data. We construct synthetic question-SQL pairs over\nhigh-quality tables via a synchronous context-free grammar (SCFG) induced from\nexisting text-to-SQL datasets. We pre-train our model on the synthetic data\nusing a novel text-schema linking objective that predicts the syntactic role of\na table field in the SQL for each question-SQL pair. To maintain the model's\nability to represent real-world data, we also include masked language modeling\n(MLM) over several existing table-and-language datasets to regularize the\npre-training process. On four popular fully supervised and weakly supervised\ntable semantic parsing benchmarks, GraPPa significantly outperforms\nRoBERTa-large as the feature representation layers and establishes new\nstate-of-the-art results on all of them.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.13964v4",
            "date": "2020-09-29 12:29:04+00:00",
            "title": "CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models",
            "authors": [
                "Yusheng Su",
                "Xu Han",
                "Zhengyan Zhang",
                "Peng Li",
                "Zhiyuan Liu",
                "Yankai Lin",
                "Jie Zhou",
                "Maosong Sun"
            ],
            "abstract": "Several recent efforts have been devoted to enhancing pre-trained language\nmodels (PLMs) by utilizing extra heterogeneous knowledge in knowledge graphs\n(KGs) and achieved consistent improvements on various knowledge-driven NLP\ntasks. However, most of these knowledge-enhanced PLMs embed static sub-graphs\nof KGs (\"knowledge context\"), regardless of that the knowledge required by PLMs\nmay change dynamically according to specific text (\"textual context\"). In this\npaper, we propose a novel framework named Coke to dynamically select contextual\nknowledge and embed knowledge context according to textual context for PLMs,\nwhich can avoid the effect of redundant and ambiguous knowledge in KGs that\ncannot match the input text. Our experimental results show that Coke\noutperforms various baselines on typical knowledge-driven NLP tasks, indicating\nthe effectiveness of utilizing dynamic knowledge context for language\nunderstanding. Besides the performance improvements, the dynamically selected\nknowledge in Coke can describe the semantics of text-related knowledge in a\nmore interpretable form than the conventional PLMs. Our source code and\ndatasets will be available to provide more details for Coke.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14116v1",
            "date": "2020-09-29 16:03:13+00:00",
            "title": "A Survey on Semantic Parsing from the perspective of Compositionality",
            "authors": [
                "Pawan Kumar",
                "Srikanta Bedathur"
            ],
            "abstract": "Different from previous surveys in semantic parsing (Kamath and Das, 2018)\nand knowledge base question answering(KBQA)(Chakraborty et al., 2019; Zhu et\nal., 2019; Hoffner et al., 2017) we try to takes a different perspective on the\nstudy of semantic parsing. Specifically, we will focus on (a)meaning\ncomposition from syntactical structure(Partee, 1975), and (b) the ability of\nsemantic parsers to handle lexical variation given the context of a knowledge\nbase (KB). In the following section after an introduction of the field of\nsemantic parsing and its uses in KBQA, we will describe meaning representation\nusing grammar formalism CCG (Steedman, 1996). We will discuss semantic\ncomposition using formal languages in Section 2. In section 3 we will consider\nsystems that uses formal languages e.g. $\\lambda$-calculus (Steedman, 1996),\n$\\lambda$-DCS (Liang, 2013). Section 4 and 5 consider semantic parser using\nstructured-language for logical form. Section 6 is on different benchmark\ndatasets ComplexQuestions (Bao et al.,2016) and GraphQuestions (Su et al.,\n2016) that can be used to evaluate semantic parser on their ability to answer\ncomplex questions that are highly compositional in nature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14167v1",
            "date": "2020-09-29 17:31:43+00:00",
            "title": "Contrastive Distillation on Intermediate Representations for Language Model Compression",
            "authors": [
                "Siqi Sun",
                "Zhe Gan",
                "Yu Cheng",
                "Yuwei Fang",
                "Shuohang Wang",
                "Jingjing Liu"
            ],
            "abstract": "Existing language model compression methods mostly use a simple L2 loss to\ndistill knowledge in the intermediate representations of a large BERT model to\na smaller one. Although widely used, this objective by design assumes that all\nthe dimensions of hidden representations are independent, failing to capture\nimportant structural knowledge in the intermediate layers of the teacher\nnetwork. To achieve better distillation efficacy, we propose Contrastive\nDistillation on Intermediate Representations (CoDIR), a principled knowledge\ndistillation framework where the student is trained to distill knowledge\nthrough intermediate layers of the teacher via a contrastive objective. By\nlearning to distinguish positive sample from a large set of negative samples,\nCoDIR facilitates the student's exploitation of rich information in teacher's\nhidden layers. CoDIR can be readily applied to compress large-scale language\nmodels in both pre-training and finetuning stages, and achieves superb\nperformance on the GLUE benchmark, outperforming state-of-the-art compression\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14259v2",
            "date": "2020-09-29 18:52:39+00:00",
            "title": "Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions",
            "authors": [
                "Peter A. Jansen"
            ],
            "abstract": "The recently proposed ALFRED challenge task aims for a virtual robotic agent\nto complete complex multi-step everyday tasks in a virtual home environment\nfrom high-level natural language directives, such as \"put a hot piece of bread\non a plate\". Currently, the best-performing models are able to complete less\nthan 5% of these tasks successfully. In this work we focus on modeling the\ntranslation problem of converting natural language directives into detailed\nmulti-step sequences of actions that accomplish those goals in the virtual\nenvironment. We empirically demonstrate that it is possible to generate gold\nmulti-step plans from language directives alone without any visual input in 26%\nof unseen cases. When a small amount of visual information is incorporated,\nnamely the starting location in the virtual environment, our best-performing\nGPT-2 model successfully generates gold command sequences in 58% of cases. Our\nresults suggest that contextualized language models may provide strong visual\nsemantic planning modules for grounded virtual agents.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14261v1",
            "date": "2020-09-26 07:38:11+00:00",
            "title": "Abusive Language Detection and Characterization of Twitter Behavior",
            "authors": [
                "Dincy Davis",
                "Reena Murali",
                "Remesh Babu"
            ],
            "abstract": "In this work, abusive language detection in online content is performed using\nBidirectional Recurrent Neural Network (BiRNN) method. Here the main objective\nis to focus on various forms of abusive behaviors on Twitter and to detect\nwhether a speech is abusive or not. The results are compared for various\nabusive behaviors in social media, with Convolutional Neural Netwrok (CNN) and\nRecurrent Neural Network (RNN) methods and proved that the proposed BiRNN is a\nbetter deep learning model for automatic abusive speech detection.",
            "journal ref": "International Journal of Computer Sciences and Engineering, Vol.8,\n  Issue.7, July 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14361v1",
            "date": "2020-09-30 00:57:33+00:00",
            "title": "Ethically Collecting Multi-Modal Spontaneous Conversations with People that have Cognitive Impairments",
            "authors": [
                "Angus Addlesee",
                "Pierre Albert"
            ],
            "abstract": "In order to make spoken dialogue systems (such as Amazon Alexa or Google\nAssistant) more accessible and naturally interactive for people with cognitive\nimpairments, appropriate data must be obtainable. Recordings of multi-modal\nspontaneous conversations with vulnerable user groups are scarce however and\nthis valuable data is challenging to collect. Researchers that call for this\ndata are commonly inexperienced in ethical and legal issues around working with\nvulnerable participants. Additionally, standard recording equipment is insecure\nand should not be used to capture sensitive data. We spent a year consulting\nexperts on how to ethically capture and share recordings of multi-modal\nspontaneous conversations with vulnerable user groups. In this paper we provide\nguidance, collated from these experts, on how to ethically collect such data\nand we present a new system - \"CUSCO\" - to capture, transport and exchange\nsensitive data securely. This framework is intended to be easily followed and\nimplemented to encourage further publications of similar corpora. Using this\nguide and secure recording system, researchers can review and refine their\nethical measures.",
            "journal ref": "LREC Workshop on Legal and Ethical Issues in Human Language\n  Technologies (2020) 15-20",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14445v1",
            "date": "2020-09-30 05:17:42+00:00",
            "title": "Towards Improved Model Design for Authorship Identification: A Survey on Writing Style Understanding",
            "authors": [
                "Weicheng Ma",
                "Ruibo Liu",
                "Lili Wang",
                "Soroush Vosoughi"
            ],
            "abstract": "Authorship identification tasks, which rely heavily on linguistic styles,\nhave always been an important part of Natural Language Understanding (NLU)\nresearch. While other tasks based on linguistic style understanding benefit\nfrom deep learning methods, these methods have not behaved as well as\ntraditional machine learning methods in many authorship-based tasks. With these\ntasks becoming more and more challenging, however, traditional machine learning\nmethods based on handcrafted feature sets are already approaching their\nperformance limits. Thus, in order to inspire future applications of deep\nlearning methods in authorship-based tasks in ways that benefit the extraction\nof stylistic features, we survey authorship-based tasks and other tasks related\nto writing style understanding. We first describe our survey results on the\ncurrent state of research in both sets of tasks and summarize existing\nachievements and problems in authorship-related tasks. We then describe\noutstanding methods in style-related tasks in general and analyze how they are\nused in combination in the top-performing models. We are optimistic about the\napplicability of these models to authorship-based tasks and hope our survey\nwill help advance research in this field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14459v1",
            "date": "2020-09-30 05:50:08+00:00",
            "title": "LEBANONUPRISING: a thorough study of Lebanese tweets",
            "authors": [
                "Reda Khalaf",
                "Mireille Makary"
            ],
            "abstract": "Recent studies showed a huge interest in social networks sentiment analysis.\nTwitter, which is a microblogging service, can be a great source of information\non how the users feel about a certain topic, or what their opinion is regarding\na social, economic and even political matter. On October 17, Lebanon witnessed\nthe start of a revolution; the LebanonUprising hashtag became viral on Twitter.\nA dataset consisting of a 100,0000 tweets was collected between 18 and 21\nOctober. In this paper, we conducted a sentiment analysis study for the tweets\nin spoken Lebanese Arabic related to the LebanonUprising hashtag using\ndifferent machine learning algorithms. The dataset was manually annotated to\nmeasure the precision and recall metrics and to compare between the different\nalgorithms. Furthermore, the work completed in this paper provides two more\ncontributions. The first is related to building a Lebanese to Modern Standard\nArabic mapping dictionary that was used for the preprocessing of the tweets and\nthe second is an attempt to move from sentiment analysis to emotion detection\nusing emojis, and the two emotions we tried to predict were the \"sarcastic\" and\n\"funny\" emotions. We built a training set from the tweets collected in October\n2019 and then we used this set to predict sentiments and emotions of the tweets\nwe collected between May and August 2020. The analysis we conducted shows the\nvariation in sentiments, emotions and users between the two datasets. The\nresults we obtained seem satisfactory especially considering that there was no\nprevious or similar work done involving Lebanese Arabic tweets, to our\nknowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14505v3",
            "date": "2020-09-30 08:45:25+00:00",
            "title": "TaxiNLI: Taking a Ride up the NLU Hill",
            "authors": [
                "Pratik Joshi",
                "Somak Aditya",
                "Aalok Sathe",
                "Monojit Choudhury"
            ],
            "abstract": "Pre-trained Transformer-based neural architectures have consistently achieved\nstate-of-the-art performance in the Natural Language Inference (NLI) task.\nSince NLI examples encompass a variety of linguistic, logical, and reasoning\nphenomena, it remains unclear as to which specific concepts are learnt by the\ntrained systems and where they can achieve strong generalization. To\ninvestigate this question, we propose a taxonomic hierarchy of categories that\nare relevant for the NLI task. We introduce TAXINLI, a new dataset, that has\n10k examples from the MNLI dataset (Williams et al., 2018) with these taxonomic\nlabels. Through various experiments on TAXINLI, we observe that whereas for\ncertain taxonomic categories SOTA neural models have achieved near perfect\naccuracies - a large jump over the previous models - some categories still\nremain difficult. Our work adds to the growing body of literature that shows\nthe gaps in the current NLI systems and datasets through a systematic\npresentation and analysis of reasoning categories.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.14510v1",
            "date": "2020-09-30 08:56:53+00:00",
            "title": "Cross-lingual Spoken Language Understanding with Regularized Representation Alignment",
            "authors": [
                "Zihan Liu",
                "Genta Indra Winata",
                "Peng Xu",
                "Zhaojiang Lin",
                "Pascale Fung"
            ],
            "abstract": "Despite the promising results of current cross-lingual models for spoken\nlanguage understanding systems, they still suffer from imperfect cross-lingual\nrepresentation alignments between the source and target languages, which makes\nthe performance sub-optimal. To cope with this issue, we propose a\nregularization approach to further align word-level and sentence-level\nrepresentations across languages without any external resource. First, we\nregularize the representation of user utterances based on their corresponding\nlabels. Second, we regularize the latent variable model (Liu et al., 2019) by\nleveraging adversarial training to disentangle the latent variables.\nExperiments on the cross-lingual spoken language understanding task show that\nour model outperforms current state-of-the-art methods in both few-shot and\nzero-shot scenarios, and our model, trained on a few-shot setting with only 3\\%\nof the target language training data, achieves comparable performance to the\nsupervised training with all the training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14539v1",
            "date": "2020-09-30 09:50:39+00:00",
            "title": "Explainable Natural Language Reasoning via Conceptual Unification",
            "authors": [
                "Marco Valentino",
                "Mokanarangan Thayaparan",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "This paper presents an abductive framework for multi-hop and interpretable\ntextual inference. The reasoning process is guided by the notions unification\npower and plausibility of an explanation, computed through the interaction of\ntwo major architectural components: (a) An analogical reasoning model that\nranks explanatory facts by leveraging unification patterns in a corpus of\nexplanations; (b) An abductive reasoning model that performs a search for the\nbest explanation, which is realised via conceptual abstraction and subsequent\nunification. We demonstrate that the Step-wise Conceptual Unification can be\neffective for unsupervised question answering, and as an explanation extractor\nin combination with state-of-the-art Transformers. An empirical evaluation on\nthe Worldtree corpus and the ARC Challenge resulted in the following\nconclusions: (1) The question answering model outperforms competitive neural\nand multi-hop baselines without requiring any explicit training on answer\nprediction; (2) When used as an explanation extractor, the proposed model\nsignificantly improves the performance of Transformers, leading to\nstate-of-the-art results on the Worldtree corpus; (3) Analogical and abductive\nreasoning are highly complementary for achieving sound explanatory inference, a\nfeature that demonstrates the impact of the unification patterns on performance\nand interpretability.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2009.14578v1",
            "date": "2020-09-30 11:55:58+00:00",
            "title": "Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text",
            "authors": [
                "Shaoxiong Ji",
                "Erik Cambria",
                "Pekka Marttinen"
            ],
            "abstract": "Medical code assignment, which predicts medical codes from clinical texts, is\na fundamental task of intelligent medical information systems. The emergence of\ndeep models in natural language processing has boosted the development of\nautomatic assignment methods. However, recent advanced neural architectures\nwith flat convolutions or multi-channel feature concatenation ignore the\nsequential causal constraint within a text sequence and may not learn\nmeaningful clinical text representations, especially for lengthy clinical notes\nwith long-term sequential dependency. This paper proposes a Dilated\nConvolutional Attention Network (DCAN), integrating dilated convolutions,\nresidual connections, and label attention, for medical code assignment. It\nadopts dilated convolutions to capture complex medical patterns with a\nreceptive field which increases exponentially with dilation size. Experiments\non a real-world clinical dataset empirically show that our model improves the\nstate of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14722v1",
            "date": "2020-09-30 14:59:09+00:00",
            "title": "RDSGAN: Rank-based Distant Supervision Relation Extraction with Generative Adversarial Framework",
            "authors": [
                "Guoqing Luo",
                "Jiaxin Pan",
                "Min Peng"
            ],
            "abstract": "Distant supervision has been widely used for relation extraction but suffers\nfrom noise labeling problem. Neural network models are proposed to denoise with\nattention mechanism but cannot eliminate noisy data due to its non-zero\nweights. Hard decision is proposed to remove wrongly-labeled instances from the\npositive set though causes loss of useful information contained in removed\ninstances. In this paper, we propose a novel generative neural framework named\nRDSGAN (Rank-based Distant Supervision GAN) which automatically generates valid\ninstances for distant supervision relation extraction. Our framework combines\nsoft attention and hard decision to learn the distribution of true positive\ninstances via adversarial training and selects valid instances conforming to\nthe distribution via rank-based distant supervision, which addresses the false\npositive problem. Experimental results show the superiority of our framework\nover strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2009.14734v2",
            "date": "2020-09-30 15:21:19+00:00",
            "title": "Point-of-Interest Type Inference from Social Media Text",
            "authors": [
                "Danae S\u00e1nchez Villegas",
                "Daniel Preo\u0163iuc-Pietro",
                "Nikolaos Aletras"
            ],
            "abstract": "Physical places help shape how we perceive the experiences we have there. For\nthe first time, we study the relationship between social media text and the\ntype of the place from where it was posted, whether a park, restaurant, or\nsomeplace else. To facilitate this, we introduce a novel data set of\n$\\sim$200,000 English tweets published from 2,761 different points-of-interest\nin the U.S., enriched with place type information. We train classifiers to\npredict the type of the location a tweet was sent from that reach a macro F1 of\n43.67 across eight classes and uncover the linguistic markers associated with\neach type of place. The ability to predict semantic place information from a\ntweet has applications in recommendation systems, personalization services and\ncultural geography.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00038v2",
            "date": "2020-09-30 18:12:50+00:00",
            "title": "AbuseAnalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts",
            "authors": [
                "Mohit Chandra",
                "Ashwin Pathak",
                "Eesha Dutta",
                "Paryul Jain",
                "Manish Gupta",
                "Manish Shrivastava",
                "Ponnurangam Kumaraguru"
            ],
            "abstract": "While extensive popularity of online social media platforms has made\ninformation dissemination faster, it has also resulted in widespread online\nabuse of different types like hate speech, offensive language, sexist and\nracist opinions, etc. Detection and curtailment of such abusive content is\ncritical for avoiding its psychological impact on victim communities, and\nthereby preventing hate crimes. Previous works have focused on classifying user\nposts into various forms of abusive behavior. But there has hardly been any\nfocus on estimating the severity of abuse and the target. In this paper, we\npresent a first of the kind dataset with 7601 posts from Gab which looks at\nonline abuse from the perspective of presence of abuse, severity and target of\nabusive behavior. We also propose a system to address these tasks, obtaining an\naccuracy of ~80% for abuse presence, ~82% for abuse target prediction, and ~65%\nfor abuse severity prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00117v1",
            "date": "2020-09-30 21:50:46+00:00",
            "title": "Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning",
            "authors": [
                "Yuning Mao",
                "Yanru Qu",
                "Yiqing Xie",
                "Xiang Ren",
                "Jiawei Han"
            ],
            "abstract": "While neural sequence learning methods have made significant progress in\nsingle-document summarization (SDS), they produce unsatisfactory results on\nmulti-document summarization (MDS). We observe two major challenges when\nadapting SDS advances to MDS: (1) MDS involves larger search space and yet more\nlimited training data, setting obstacles for neural methods to learn adequate\nrepresentations; (2) MDS needs to resolve higher information redundancy among\nthe source documents, which SDS methods are less effective to handle. To close\nthe gap, we present RL-MMR, Maximal Margin Relevance-guided Reinforcement\nLearning for MDS, which unifies advanced neural SDS methods and statistical\nmeasures used in classical MDS. RL-MMR casts MMR guidance on fewer promising\ncandidates, which restrains the search space and thus leads to better\nrepresentation learning. Additionally, the explicit redundancy measure in MMR\nhelps the neural representation of the summary to better capture redundancy.\nExtensive experiments demonstrate that RL-MMR achieves state-of-the-art\nperformance on benchmark MDS datasets. In particular, we show the benefits of\nincorporating MMR into end-to-end learning when adapting SDS to MDS in terms of\nboth learning effectiveness and efficiency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00121v1",
            "date": "2020-09-30 21:54:22+00:00",
            "title": "Interactive Re-Fitting as a Technique for Improving Word Embeddings",
            "authors": [
                "James Powell",
                "Kari Sentz"
            ],
            "abstract": "Word embeddings are a fixed, distributional representation of the context of\nwords in a corpus learned from word co-occurrences. While word embeddings have\nproven to have many practical uses in natural language processing tasks, they\nreflect the attributes of the corpus upon which they are trained. Recent work\nhas demonstrated that post-processing of word embeddings to apply information\nfound in lexical dictionaries can improve their quality. We build on this\npost-processing technique by making it interactive. Our approach makes it\npossible for humans to adjust portions of a word embedding space by moving sets\nof words closer to one another. One motivating use case for this capability is\nto enable users to identify and reduce the presence of bias in word embeddings.\nOur approach allows users to trigger selective post-processing as they interact\nwith and assess potential bias in word embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00133v1",
            "date": "2020-09-30 22:38:40+00:00",
            "title": "CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models",
            "authors": [
                "Nikita Nangia",
                "Clara Vania",
                "Rasika Bhalerao",
                "Samuel R. Bowman"
            ],
            "abstract": "Pretrained language models, especially masked language models (MLMs) have\nseen success across many NLP tasks. However, there is ample evidence that they\nuse the cultural biases that are undoubtedly present in the corpora they are\ntrained on, implicitly creating harm with biased representations. To measure\nsome forms of social bias in language models against protected demographic\ngroups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark\n(CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing\nwith nine types of bias, like race, religion, and age. In CrowS-Pairs a model\nis presented with two sentences: one that is more stereotyping and another that\nis less stereotyping. The data focuses on stereotypes about historically\ndisadvantaged groups and contrasts them with advantaged groups. We find that\nall three of the widely-used MLMs we evaluate substantially favor sentences\nthat express stereotypes in every category in CrowS-Pairs. As work on building\nless biased models advances, this dataset can be used as a benchmark to\nevaluate progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00200v1",
            "date": "2020-10-01 05:27:51+00:00",
            "title": "RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble",
            "authors": [
                "Michael Bendersky",
                "Honglei Zhuang",
                "Ji Ma",
                "Shuguang Han",
                "Keith Hall",
                "Ryan McDonald"
            ],
            "abstract": "In this paper, we report the results of our participation in the TREC-COVID\nchallenge. To meet the challenge of building a search engine for rapidly\nevolving biomedical collection, we propose a simple yet effective weighted\nhierarchical rank fusion approach, that ensembles together 102 runs from (a)\nlexical and semantic retrieval systems, (b) pre-trained and fine-tuned BERT\nrankers, and (c) relevance feedback runs. Our ablation studies demonstrate the\ncontributions of each of these systems to the overall ensemble. The submitted\nensemble runs achieved state-of-the-art performance in rounds 4 and 5 of the\nTREC-COVID challenge.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.00309v1",
            "date": "2020-10-01 11:39:32+00:00",
            "title": "CoLAKE: Contextualized Language and Knowledge Embedding",
            "authors": [
                "Tianxiang Sun",
                "Yunfan Shao",
                "Xipeng Qiu",
                "Qipeng Guo",
                "Yaru Hu",
                "Xuanjing Huang",
                "Zheng Zhang"
            ],
            "abstract": "With the emerging branch of incorporating factual knowledge into pre-trained\nlanguage models such as BERT, most existing models consider shallow, static,\nand separately pre-trained entity embeddings, which limits the performance\ngains of these models. Few works explore the potential of deep contextualized\nknowledge representation when injecting knowledge. In this paper, we propose\nthe Contextualized Language and Knowledge Embedding (CoLAKE), which jointly\nlearns contextualized representation for both language and knowledge with the\nextended MLM objective. Instead of injecting only entity embeddings, CoLAKE\nextracts the knowledge context of an entity from large-scale knowledge bases.\nTo handle the heterogeneity of knowledge context and language context, we\nintegrate them in a unified data structure, word-knowledge graph (WK graph).\nCoLAKE is pre-trained on large-scale WK graphs with the modified Transformer\nencoder. We conduct experiments on knowledge-driven tasks, knowledge probing\ntasks, and language understanding tasks. Experimental results show that CoLAKE\noutperforms previous counterparts on most of the tasks. Besides, CoLAKE\nachieves surprisingly high performance on our synthetic task called\nword-knowledge graph completion, which shows the superiority of simultaneously\ncontextualizing language and knowledge representation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00357v1",
            "date": "2020-10-01 12:44:24+00:00",
            "title": "Detecting White Supremacist Hate Speech using Domain Specific Word Embedding with Deep Learning and BERT",
            "authors": [
                "Hind Saleh Alatawi",
                "Areej Maatog Alhothali",
                "Kawthar Mustafa Moria"
            ],
            "abstract": "White supremacists embrace a radical ideology that considers white people\nsuperior to people of other races. The critical influence of these groups is no\nlonger limited to social media; they also have a significant effect on society\nin many ways by promoting racial hatred and violence. White supremacist hate\nspeech is one of the most recently observed harmful content on social\nmedia.Traditional channels of reporting hate speech have proved inadequate due\nto the tremendous explosion of information, and therefore, it is necessary to\nfind an automatic way to detect such speech in a timely manner. This research\ninvestigates the viability of automatically detecting white supremacist hate\nspeech on Twitter by using deep learning and natural language processing\ntechniques. Through our experiments, we used two approaches, the first approach\nis by using domain-specific embeddings which are extracted from white\nsupremacist corpus in order to catch the meaning of this white supremacist\nslang with bidirectional Long Short-Term Memory (LSTM) deep learning model,\nthis approach reached a 0.74890 F1-score. The second approach is by using the\none of the most recent language model which is BERT, BERT model provides the\nstate of the art of most NLP tasks. It reached to a 0.79605 F1-score. Both\napproaches are tested on a balanced dataset given that our experiments were\nbased on textual data only. The dataset was combined from dataset created from\nTwitter and a Stormfront dataset compiled from that white supremacist forum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00389v1",
            "date": "2020-10-01 13:26:58+00:00",
            "title": "A Survey on Explainability in Machine Reading Comprehension",
            "authors": [
                "Mokanarangan Thayaparan",
                "Marco Valentino",
                "Andr\u00e9 Freitas"
            ],
            "abstract": "This paper presents a systematic review of benchmarks and approaches for\nexplainability in Machine Reading Comprehension (MRC). We present how the\nrepresentation and inference challenges evolved and the steps which were taken\nto tackle these challenges. We also present the evaluation methodologies to\nassess the performance of explainable systems. In addition, we identify\npersisting open research questions and highlight critical directions for future\nwork.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00514v1",
            "date": "2020-10-01 16:02:30+00:00",
            "title": "Referring Image Segmentation via Cross-Modal Progressive Comprehension",
            "authors": [
                "Shaofei Huang",
                "Tianrui Hui",
                "Si Liu",
                "Guanbin Li",
                "Yunchao Wei",
                "Jizhong Han",
                "Luoqi Liu",
                "Bo Li"
            ],
            "abstract": "Referring image segmentation aims at segmenting the foreground masks of the\nentities that can well match the description given in the natural language\nexpression. Previous approaches tackle this problem using implicit feature\ninteraction and fusion between visual and linguistic modalities, but usually\nfail to explore informative words of the expression to well align features from\nthe two modalities for accurately identifying the referred entity. In this\npaper, we propose a Cross-Modal Progressive Comprehension (CMPC) module and a\nText-Guided Feature Exchange (TGFE) module to effectively address the\nchallenging task. Concretely, the CMPC module first employs entity and\nattribute words to perceive all the related entities that might be considered\nby the expression. Then, the relational words are adopted to highlight the\ncorrect entity as well as suppress other irrelevant ones by multimodal graph\nreasoning. In addition to the CMPC module, we further leverage a simple yet\neffective TGFE module to integrate the reasoned multimodal features from\ndifferent levels with the guidance of textual information. In this way,\nfeatures from multi-levels could communicate with each other and be refined\nbased on the textual context. We conduct extensive experiments on four popular\nreferring segmentation benchmarks and achieve new state-of-the-art\nperformances.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.00515v3",
            "date": "2020-10-01 16:03:51+00:00",
            "title": "Linguistic Structure Guided Context Modeling for Referring Image Segmentation",
            "authors": [
                "Tianrui Hui",
                "Si Liu",
                "Shaofei Huang",
                "Guanbin Li",
                "Sansi Yu",
                "Faxi Zhang",
                "Jizhong Han"
            ],
            "abstract": "Referring image segmentation aims to predict the foreground mask of the\nobject referred by a natural language sentence. Multimodal context of the\nsentence is crucial to distinguish the referent from the background. Existing\nmethods either insufficiently or redundantly model the multimodal context. To\ntackle this problem, we propose a \"gather-propagate-distribute\" scheme to model\nmultimodal context by cross-modal interaction and implement this scheme as a\nnovel Linguistic Structure guided Context Modeling (LSCM) module. Our LSCM\nmodule builds a Dependency Parsing Tree suppressed Word Graph (DPT-WG) which\nguides all the words to include valid multimodal context of the sentence while\nexcluding disturbing ones through three steps over the multimodal feature,\ni.e., gathering, constrained propagation and distributing. Extensive\nexperiments on four benchmarks demonstrate that our method outperforms all the\nprevious state-of-the-arts.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.00656v2",
            "date": "2020-10-01 19:33:27+00:00",
            "title": "Predicting User Engagement Status for Online Evaluation of Intelligent Assistants",
            "authors": [
                "Rui Meng",
                "Zhen Yue",
                "Alyssa Glass"
            ],
            "abstract": "Evaluation of intelligent assistants in large-scale and online settings\nremains an open challenge. User behavior-based online evaluation metrics have\ndemonstrated great effectiveness for monitoring large-scale web search and\nrecommender systems. Therefore, we consider predicting user engagement status\nas the very first and critical step to online evaluation for intelligent\nassistants. In this work, we first proposed a novel framework for classifying\nuser engagement status into four categories -- fulfillment, continuation,\nreformulation and abandonment. We then demonstrated how to design simple but\nindicative metrics based on the framework to quantify user engagement levels.\nWe also aim for automating user engagement prediction with machine learning\nmethods. We compare various models and features for predicting engagement\nstatus using four real-world datasets. We conducted detailed analyses on\nfeatures and failure cases to discuss the performance of current models as well\nas challenges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00667v3",
            "date": "2020-10-01 20:02:43+00:00",
            "title": "Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers",
            "authors": [
                "Hanjie Chen",
                "Yangfeng Ji"
            ],
            "abstract": "To build an interpretable neural text classifier, most of the prior work has\nfocused on designing inherently interpretable models or finding faithful\nexplanations. A new line of work on improving model interpretability has just\nstarted, and many existing methods require either prior information or human\nannotations as additional inputs in training. To address this limitation, we\npropose the variational word mask (VMASK) method to automatically learn\ntask-specific important words and reduce irrelevant information on\nclassification, which ultimately improves the interpretability of model\npredictions. The proposed method is evaluated with three neural text\nclassifiers (CNN, LSTM, and BERT) on seven benchmark text classification\ndatasets. Experiments show the effectiveness of VMASK in improving both model\nprediction accuracy and interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00677v1",
            "date": "2020-10-01 20:40:23+00:00",
            "title": "Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding",
            "authors": [
                "Jiaming Shen",
                "Heng Ji",
                "Jiawei Han"
            ],
            "abstract": "Linguistic steganography studies how to hide secret messages in natural\nlanguage cover texts. Traditional methods aim to transform a secret message\ninto an innocent text via lexical substitution or syntactical modification.\nRecently, advances in neural language models (LMs) enable us to directly\ngenerate cover text conditioned on the secret message. In this study, we\npresent a new linguistic steganography method which encodes secret messages\nusing self-adjusting arithmetic coding based on a neural language model. We\nformally analyze the statistical imperceptibility of this method and\nempirically show it outperforms the previous state-of-the-art methods on four\ndatasets by 15.3% and 38.9% in terms of bits/word and KL metrics, respectively.\nFinally, human evaluations show that 51% of generated cover texts can indeed\nfool eavesdroppers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00678v1",
            "date": "2020-10-01 20:48:37+00:00",
            "title": "Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling",
            "authors": [
                "Yan Shvartzshnaider",
                "Ananth Balashankar",
                "Vikas Patidar",
                "Thomas Wies",
                "Lakshminarayanan Subramanian"
            ],
            "abstract": "This paper formulates a new task of extracting privacy parameters from a\nprivacy policy, through the lens of Contextual Integrity, an established social\ntheory framework for reasoning about privacy norms. Privacy policies, written\nby lawyers, are lengthy and often comprise incomplete and vague statements. In\nthis paper, we show that traditional NLP tasks, including the recently proposed\nQuestion-Answering based solutions, are insufficient to address the privacy\nparameter extraction problem and provide poor precision and recall. We describe\n4 different types of conventional methods that can be partially adapted to\naddress the parameter extraction task with varying degrees of success: Hidden\nMarkov Models, BERT fine-tuned models, Dependency Type Parsing (DP) and\nSemantic Role Labeling (SRL). Based on a detailed evaluation across 36\nreal-world privacy policies of major enterprises, we demonstrate that a\nsolution combining syntactic DP coupled with type-specific SRL tasks provides\nthe highest accuracy for retrieving contextual privacy parameters from privacy\nstatements. We also observe that incorporating domain-specific knowledge is\ncritical to achieving high precision and recall, thus inspiring new NLP\nresearch to address this important problem in the privacy domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00685v3",
            "date": "2020-10-01 21:06:21+00:00",
            "title": "How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds",
            "authors": [
                "Prithviraj Ammanabrolu",
                "Jack Urbanek",
                "Margaret Li",
                "Arthur Szlam",
                "Tim Rockt\u00e4schel",
                "Jason Weston"
            ],
            "abstract": "We seek to create agents that both act and communicate with other agents in\npursuit of a goal. Towards this end, we extend LIGHT (Urbanek et al. 2019) -- a\nlarge-scale crowd-sourced fantasy text-game -- with a dataset of quests. These\ncontain natural language motivations paired with in-game goals and human\ndemonstrations; completing a quest might require dialogue or actions (or both).\nWe introduce a reinforcement learning system that (1) incorporates large-scale\nlanguage modeling-based and commonsense reasoning-based pre-training to imbue\nthe agent with relevant priors; and (2) leverages a factorized action space of\naction commands and dialogue, balancing between the two. We conduct zero-shot\nevaluations using held-out human expert demonstrations, showing that our agents\nare able to act consistently and talk naturally with respect to their\nmotivations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00854v1",
            "date": "2020-10-02 08:34:34+00:00",
            "title": "Which *BERT? A Survey Organizing Contextualized Encoders",
            "authors": [
                "Patrick Xia",
                "Shijie Wu",
                "Benjamin Van Durme"
            ],
            "abstract": "Pretrained contextualized text encoders are now a staple of the NLP\ncommunity. We present a survey on language representation learning with the aim\nof consolidating a series of shared lessons learned across a variety of recent\nefforts. While significant advancements continue at a rapid pace, we find that\nenough has now been discovered, in different directions, that we can begin to\norganize advances according to common themes. Through this organization, we\nhighlight important considerations when interpreting recent contributions and\nchoosing which model to use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00857v1",
            "date": "2020-10-02 08:38:40+00:00",
            "title": "SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces",
            "authors": [
                "K Vani",
                "Sandra Mitrovic",
                "Alessandro Antonucci",
                "Fabio Rinaldi"
            ],
            "abstract": "Lexical semantic change detection (also known as semantic shift tracing) is a\ntask of identifying words that have changed their meaning over time.\nUnsupervised semantic shift tracing, focal point of SemEval2020, is\nparticularly challenging. Given the unsupervised setup, in this work, we\npropose to identify clusters among different occurrences of each target word,\nconsidering these as representatives of different word meanings. As such,\ndisagreements in obtained clusters naturally allow to quantify the level of\nsemantic shift per each target word in four target languages. To leverage this\nidea, clustering is performed on contextualized (BERT-based) embeddings of word\noccurrences. The obtained results show that our approach performs well both\nmeasured separately (per language) and overall, where we surpass all provided\nSemEval baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00910v1",
            "date": "2020-10-02 10:32:29+00:00",
            "title": "Continual Learning for Natural Language Generation in Task-oriented Dialog Systems",
            "authors": [
                "Fei Mi",
                "Liangwei Chen",
                "Mengjie Zhao",
                "Minlie Huang",
                "Boi Faltings"
            ],
            "abstract": "Natural language generation (NLG) is an essential component of task-oriented\ndialog systems. Despite the recent success of neural approaches for NLG, they\nare typically developed in an offline manner for particular domains. To better\nfit real-life applications where new data come in a stream, we study NLG in a\n\"continual learning\" setting to expand its knowledge to new domains or\nfunctionalities incrementally. The major challenge towards this goal is\ncatastrophic forgetting, meaning that a continually trained model tends to\nforget the knowledge it has learned before. To this end, we propose a method\ncalled ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying\nprioritized historical exemplars, together with an adaptive regularization\ntechnique based on ElasticWeight Consolidation. Extensive experiments to\ncontinually learn new domains and intents are conducted on MultiWoZ-2.0 to\nbenchmark ARPER with a wide range of techniques. Empirical results demonstrate\nthat ARPER significantly outperforms other methods by effectively mitigating\nthe detrimental catastrophic forgetting issue.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.00980v1",
            "date": "2020-10-02 13:22:12+00:00",
            "title": "MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale",
            "authors": [
                "Andreas R\u00fcckl\u00e9",
                "Jonas Pfeiffer",
                "Iryna Gurevych"
            ],
            "abstract": "We study the zero-shot transfer capabilities of text matching models on a\nmassive scale, by self-supervised training on 140 source domains from community\nquestion answering forums in English. We investigate the model performances on\nnine benchmarks of answer selection and question similarity tasks, and show\nthat all 140 models transfer surprisingly well, where the large majority of\nmodels substantially outperforms common IR baselines. We also demonstrate that\nconsidering a broad selection of source domains is crucial for obtaining the\nbest zero-shot transfer performances, which contrasts the standard procedure\nthat merely relies on the largest and most similar domains. In addition, we\nextensively study how to best combine multiple source domains. We propose to\nincorporate self-supervised with supervised multi-task learning on all\navailable source domains. Our best zero-shot transfer model considerably\noutperforms in-domain BERT and the previous state of the art on six benchmarks.\nFine-tuning of our model with in-domain data results in additional large gains\nand achieves the new state of the art on all nine benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01057v1",
            "date": "2020-10-02 15:38:03+00:00",
            "title": "LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention",
            "authors": [
                "Ikuya Yamada",
                "Akari Asai",
                "Hiroyuki Shindo",
                "Hideaki Takeda",
                "Yuji Matsumoto"
            ],
            "abstract": "Entity representations are useful in natural language tasks involving\nentities. In this paper, we propose new pretrained contextualized\nrepresentations of words and entities based on the bidirectional transformer.\nThe proposed model treats words and entities in a given text as independent\ntokens, and outputs contextualized representations of them. Our model is\ntrained using a new pretraining task based on the masked language model of\nBERT. The task involves predicting randomly masked words and entities in a\nlarge entity-annotated corpus retrieved from Wikipedia. We also propose an\nentity-aware self-attention mechanism that is an extension of the\nself-attention mechanism of the transformer, and considers the types of tokens\n(words or entities) when computing attention scores. The proposed model\nachieves impressive empirical performance on a wide range of entity-related\ntasks. In particular, it obtains state-of-the-art results on five well-known\ndatasets: Open Entity (entity typing), TACRED (relation classification),\nCoNLL-2003 (named entity recognition), ReCoRD (cloze-style question answering),\nand SQuAD 1.1 (extractive question answering). Our source code and pretrained\nrepresentations are available at https://github.com/studio-ousia/luke.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01061v4",
            "date": "2020-10-02 15:41:57+00:00",
            "title": "Data-Efficient Pretraining via Contrastive Self-Supervision",
            "authors": [
                "Nils Rethmeier",
                "Isabelle Augenstein"
            ],
            "abstract": "For natural language processing `text-to-text' tasks, the prevailing\napproaches heavily rely on pretraining large self-supervised models on\nincreasingly larger `task-external' data. Transfer learning from high-resource\npretraining works well, but research has focused on settings with very large\ndata and compute requirements, while the potential of efficient low-resource\nlearning, without large `task-external' pretraining, remains under-explored. In\nthis work, we evaluate against three core challenges for resource efficient\nlearning. Namely, we analyze: (1) pretraining data ($X$) efficiency; (2) zero\nto few-shot label ($Y$) efficiency; and (3) long-tail generalization, since\nlong-tail preservation has been linked to algorithmic fairness and because data\nin the tail is limited by definition. To address these challenges, we propose a\ndata and compute efficient self-supervised, contrastive text encoder,\npretrained on 60MB of `task-internal' text data, and compare it to RoBERTa,\nwhich was pretrained on 160GB of `task-external' text. We find our method\noutperforms RoBERTa, while pretraining and fine-tuning in a 1/5th of RoBERTa's\nfine-tuning time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01082v1",
            "date": "2020-10-02 16:20:39+00:00",
            "title": "Multi-Modal Open-Domain Dialogue",
            "authors": [
                "Kurt Shuster",
                "Eric Michael Smith",
                "Da Ju",
                "Jason Weston"
            ],
            "abstract": "Recent work in open-domain conversational agents has demonstrated that\nsignificant improvements in model engagingness and humanness metrics can be\nachieved via massive scaling in both pre-training data and model size\n(Adiwardana et al., 2020; Roller et al., 2020). However, if we want to build\nagents with human-like abilities, we must expand beyond handling just text. A\nparticularly important topic is the ability to see images and communicate about\nwhat is perceived. With the goal of engaging humans in multi-modal dialogue, we\ninvestigate combining components from state-of-the-art open-domain dialogue\nagents with those from state-of-the-art vision models. We study incorporating\ndifferent image fusion schemes and domain-adaptive pre-training and fine-tuning\nstrategies, and show that our best resulting model outperforms strong existing\nmodels in multi-modal dialogue while simultaneously performing as well as its\npredecessor (text-only) BlenderBot (Roller et al., 2020) in text-based\nconversation. We additionally investigate and incorporate safety components in\nour final model, and show that such efforts do not diminish model performance\nwith respect to engagingness metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01410v1",
            "date": "2020-10-03 18:57:26+00:00",
            "title": "Code to Comment \"Translation\": Data, Metrics, Baselining & Evaluation",
            "authors": [
                "David Gros",
                "Hariharan Sezhiyan",
                "Prem Devanbu",
                "Zhou Yu"
            ],
            "abstract": "The relationship of comments to code, and in particular, the task of\ngenerating useful comments given the code, has long been of interest. The\nearliest approaches have been based on strong syntactic theories of\ncomment-structures, and relied on textual templates. More recently, researchers\nhave applied deep learning methods to this task, and specifically, trainable\ngenerative translation models which are known to work very well for Natural\nLanguage translation (e.g., from German to English). We carefully examine the\nunderlying assumption here: that the task of generating comments sufficiently\nresembles the task of translating between natural languages, and so similar\nmodels and evaluation metrics could be used. We analyze several recent\ncode-comment datasets for this task: CodeNN, DeepCom, FunCom, and DocString. We\ncompare them with WMT19, a standard dataset frequently used to train state of\nthe art natural language translators. We found some interesting differences\nbetween the code-comment data and the WMT19 natural language data. Next, we\ndescribe and conduct some studies to calibrate BLEU (which is commonly used as\na measure of comment quality). using \"affinity pairs\" of methods, from\ndifferent projects, in the same project, in the same class, etc; Our study\nsuggests that the current performance on some datasets might need to be\nimproved substantially. We also argue that fairly naive information retrieval\n(IR) methods do well enough at this task to be considered a reasonable\nbaseline. Finally, we make some suggestions on how our findings might be used\nin future research in this area.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2010.01417v1",
            "date": "2020-10-03 19:50:04+00:00",
            "title": "Semantic Role Labeling Guided Multi-turn Dialogue ReWriter",
            "authors": [
                "Kun Xu",
                "Haochen Tan",
                "Linfeng Song",
                "Han Wu",
                "Haisong Zhang",
                "Linqi Song",
                "Dong Yu"
            ],
            "abstract": "For multi-turn dialogue rewriting, the capacity of effectively modeling the\nlinguistic knowledge in dialog context and getting rid of the noises is\nessential to improve its performance. Existing attentive models attend to all\nwords without prior focus, which results in inaccurate concentration on some\ndispensable words. In this paper, we propose to use semantic role labeling\n(SRL), which highlights the core semantic information of who did what to whom,\nto provide additional guidance for the rewriter model. Experiments show that\nthis information significantly improves a RoBERTa-based model that already\noutperforms previous state-of-the-art systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01429v1",
            "date": "2020-10-03 21:51:47+00:00",
            "title": "Aspect-Based Sentiment Analysis in Education Domain",
            "authors": [
                "Rinor Hajrizi",
                "Krenare Pireva Nu\u00e7i"
            ],
            "abstract": "Analysis of a large amount of data has always brought value to institutions\nand organizations. Lately, people's opinions expressed through text have become\na very important aspect of this analysis. In response to this challenge, a\nnatural language processing technique known as Aspect-Based Sentiment Analysis\n(ABSA) has emerged. Having the ability to extract the polarity for each aspect\nof opinions separately, ABSA has found itself useful in a wide range of\ndomains. Education is one of the domains in which ABSA can be successfully\nutilized. Being able to understand and find out what students like and don't\nlike most about a course, professor, or teaching methodology can be of great\nimportance for the respective institutions. While this task represents a unique\nNLP challenge, many studies have proposed different approaches to tackle the\nproblem. In this work, we present a comprehensive review of the existing work\nin ABSA with a focus in the education domain. A wide range of methodologies are\ndiscussed and conclusions are drawn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01447v1",
            "date": "2020-10-04 00:04:40+00:00",
            "title": "GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems",
            "authors": [
                "Shiquan Yang",
                "Rui Zhang",
                "Sarah Erfani"
            ],
            "abstract": "End-to-end task-oriented dialogue systems aim to generate system responses\ndirectly from plain text inputs. There are two challenges for such systems: one\nis how to effectively incorporate external knowledge bases (KBs) into the\nlearning framework; the other is how to accurately capture the semantics of\ndialogue history. In this paper, we address these two challenges by exploiting\nthe graph structural information in the knowledge base and in the dependency\nparsing tree of the dialogue. To effectively leverage the structural\ninformation in dialogue history, we propose a new recurrent cell architecture\nwhich allows representation learning on graphs. To exploit the relations\nbetween entities in KBs, the model combines multi-hop reasoning ability based\non the graph structure. Experimental results show that the proposed model\nachieves consistent improvement over state-of-the-art models on two different\ntask-oriented dialogue datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01480v1",
            "date": "2020-10-04 03:54:40+00:00",
            "title": "Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network",
            "authors": [
                "Junyi Li",
                "Siqing Li",
                "Wayne Xin Zhao",
                "Gaole He",
                "Zhicheng Wei",
                "Nicholas Jing Yuan",
                "Ji-Rong Wen"
            ],
            "abstract": "Personalized review generation (PRG) aims to automatically produce review\ntext reflecting user preference, which is a challenging natural language\ngeneration task. Most of previous studies do not explicitly model factual\ndescription of products, tending to generate uninformative content. Moreover,\nthey mainly focus on word-level generation, but cannot accurately reflect more\nabstractive user preference in multiple aspects. To address the above issues,\nwe propose a novel knowledge-enhanced PRG model based on capsule graph neural\nnetwork~(Caps-GNN). We first construct a heterogeneous knowledge graph (HKG)\nfor utilizing rich item attributes. We adopt Caps-GNN to learn graph capsules\nfor encoding underlying characteristics from the HKG. Our generation process\ncontains two major steps, namely aspect sequence generation and sentence\ngeneration. First, based on graph capsules, we adaptively learn aspect capsules\nfor inferring the aspect sequence. Then, conditioned on the inferred aspect\nlabel, we design a graph-based copy mechanism to generate sentences by\nincorporating related entities or words from HKG. To our knowledge, we are the\nfirst to utilize knowledge graph for the PRG task. The incorporated KG\ninformation is able to enhance user preference at both aspect and word levels.\nExtensive experiments on three real-world datasets have demonstrated the\neffectiveness of our model on the PRG task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01486v2",
            "date": "2020-10-04 05:24:12+00:00",
            "title": "Paragraph-level Commonsense Transformers with Recurrent Memory",
            "authors": [
                "Saadia Gabriel",
                "Chandra Bhagavatula",
                "Vered Shwartz",
                "Ronan Le Bras",
                "Maxwell Forbes",
                "Yejin Choi"
            ],
            "abstract": "Human understanding of narrative texts requires making commonsense inferences\nbeyond what is stated explicitly in the text. A recent model, COMET, can\ngenerate such implicit commonsense inferences along several dimensions such as\npre- and post-conditions, motivations, and mental states of the participants.\nHowever, COMET was trained on commonsense inferences of short phrases, and is\ntherefore discourse-agnostic. When presented with each sentence of a\nmulti-sentence narrative, it might generate inferences that are inconsistent\nwith the rest of the narrative.\n  We present the task of discourse-aware commonsense inference. Given a\nsentence within a narrative, the goal is to generate commonsense inferences\nalong predefined dimensions, while maintaining coherence with the rest of the\nnarrative. Such large-scale paragraph-level annotation is hard to get and\ncostly, so we use available sentence-level annotations to efficiently and\nautomatically construct a distantly supervised corpus.\n  Using this corpus, we train PARA-COMET, a discourse-aware model that\nincorporates paragraph-level information to generate coherent commonsense\ninferences from narratives. PARA-COMET captures both semantic knowledge\npertaining to prior world knowledge, and episodic knowledge involving how\ncurrent events relate to prior and future events in a narrative. Our results\nshow that PARA-COMET outperforms the sentence-level baselines, particularly in\ngenerating inferences that are both coherent and novel.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01496v1",
            "date": "2020-10-04 07:23:13+00:00",
            "title": "Explaining Deep Neural Networks",
            "authors": [
                "Oana-Maria Camburu"
            ],
            "abstract": "Deep neural networks are becoming more and more popular due to their\nrevolutionary success in diverse areas, such as computer vision, natural\nlanguage processing, and speech recognition. However, the decision-making\nprocesses of these models are generally not interpretable to users. In various\ndomains, such as healthcare, finance, or law, it is critical to know the\nreasons behind a decision made by an artificial intelligence system. Therefore,\nseveral directions for explaining neural models have recently been explored.\n  In this thesis, I investigate two major directions for explaining deep neural\nnetworks. The first direction consists of feature-based post-hoc explanatory\nmethods, that is, methods that aim to explain an already trained and fixed\nmodel (post-hoc), and that provide explanations in terms of input features,\nsuch as tokens for text and superpixels for images (feature-based). The second\ndirection consists of self-explanatory neural models that generate natural\nlanguage explanations, that is, models that have a built-in module that\ngenerates explanations for the predictions of the model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01526v1",
            "date": "2020-10-04 09:47:40+00:00",
            "title": "NLP Service APIs and Models for Efficient Registration of New Clients",
            "authors": [
                "Sahil Shah",
                "Vihari Piratla",
                "Soumen Chakrabarti",
                "Sunita Sarawagi"
            ],
            "abstract": "State-of-the-art NLP inference uses enormous neural architectures and models\ntrained for GPU-months, well beyond the reach of most consumers of NLP. This\nhas led to one-size-fits-all public API-based NLP service models by major AI\ncompanies, serving large numbers of clients. Neither (hardware deficient)\nclients nor (heavily subscribed) servers can afford traditional fine tuning.\nMany clients own little or no labeled data. We initiate a study of adaptation\nof centralized NLP services to clients, and present one practical and\nlightweight approach. Each client uses an unsupervised, corpus-based sketch to\nregister to the service. The server uses an auxiliary network to map the sketch\nto an abstract vector representation, which then informs the main labeling\nnetwork. When a new client registers with its sketch, it gets immediate\naccuracy benefits. We demonstrate the success of the proposed architecture\nusing sentiment labeling, NER, and predictive language modeling",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.01549v2",
            "date": "2020-10-04 11:31:21+00:00",
            "title": "Static and Animated 3D Scene Generation from Free-form Text Descriptions",
            "authors": [
                "Faria Huq",
                "Nafees Ahmed",
                "Anindya Iqbal"
            ],
            "abstract": "Generating coherent and useful image/video scenes from a free-form textual\ndescription is technically a very difficult problem to handle. Textual\ndescription of the same scene can vary greatly from person to person, or\nsometimes even for the same person from time to time. As the choice of words\nand syntax vary while preparing a textual description, it is challenging for\nthe system to reliably produce a consistently desirable output from different\nforms of language input. The prior works of scene generation have been mostly\nconfined to rigorous sentence structures of text input which restrict the\nfreedom of users to write description. In our work, we study a new pipeline\nthat aims to generate static as well as animated 3D scenes from different types\nof free-form textual scene description without any major restriction. In\nparticular, to keep our study practical and tractable, we focus on a small\nsubspace of all possible 3D scenes, containing various combinations of cube,\ncylinder and sphere. We design a two-stage pipeline. In the first stage, we\nencode the free-form text using an encoder-decoder neural architecture. In the\nsecond stage, we generate a 3D scene based on the generated encoding. Our\nneural architecture exploits state-of-the-art language model as encoder to\nleverage rich contextual encoding and a new multi-head decoder to predict\nmultiple features of an object in the scene simultaneously. For our\nexperiments, we generate a large synthetic data-set which contains 13,00,000\nand 14,00,000 samples of unique static and animated scene descriptions,\nrespectively. We achieve 98.427% accuracy on test data set in detecting the 3D\nobjects features successfully. Our work shows a proof of concept of one\napproach towards solving the problem, and we believe with enough training data,\nthe same pipeline can be expanded to handle even broader set of 3D scene\ngeneration problems.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.01678v1",
            "date": "2020-10-04 20:51:21+00:00",
            "title": "Optimal Neural Program Synthesis from Multimodal Specifications",
            "authors": [
                "Xi Ye",
                "Qiaochu Chen",
                "Isil Dillig",
                "Greg Durrett"
            ],
            "abstract": "Multimodal program synthesis, which leverages different types of user input\nto synthesize a desired program, is an attractive way to scale program\nsynthesis to challenging settings; however, it requires integrating noisy\nsignals from the user (like natural language) with hard constraints on the\nprogram's behavior. This paper proposes an optimal neural synthesis approach\nwhere the goal is to find a program that satisfies user-provided constraints\nwhile also maximizing the program's score with respect to a neural model.\nSpecifically, we focus on multimodal synthesis tasks in which the user intent\nis expressed using combination of natural language (NL) and input-output\nexamples. At the core of our method is a top-down recurrent neural model that\nplaces distributions over abstract syntax trees conditioned on the NL input.\nThis model not only allows for efficient search over the space of syntactically\nvalid programs, but it allows us to leverage automated program analysis\ntechniques for pruning the search space based on infeasibility of partial\nprograms with respect to the user's constraints. The experimental results on a\nmultimodal synthesis dataset (StructuredRegex) show that our method\nsubstantially outperforms prior state-of-the-art techniques in terms of\naccuracy %, finds model-optimal programs more frequently, and explores fewer\nstates during search.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01694v1",
            "date": "2020-10-04 21:44:15+00:00",
            "title": "On Losses for Modern Language Models",
            "authors": [
                "Stephane Aroca-Ouellette",
                "Frank Rudzicz"
            ],
            "abstract": "BERT set many state-of-the-art results over varied NLU benchmarks by\npre-training over two tasks: masked language modelling (MLM) and next sentence\nprediction (NSP), the latter of which has been highly criticized. In this\npaper, we 1) clarify NSP's effect on BERT pre-training, 2) explore fourteen\npossible auxiliary pre-training tasks, of which seven are novel to modern\nlanguage models, and 3) investigate different ways to include multiple tasks\ninto pre-training. We show that NSP is detrimental to training due to its\ncontext splitting and shallow semantic signal. We also identify six auxiliary\npre-training tasks -- sentence ordering, adjacent sentence prediction, TF\nprediction, TF-IDF prediction, a FastSent variant, and a Quick Thoughts variant\n-- that outperform a pure MLM baseline. Finally, we demonstrate that using\nmultiple tasks in a multi-task pre-training framework provides better results\nthan using any single auxiliary task. Using these methods, we outperform BERT\nBase on the GLUE benchmark using fewer than a quarter of the training tokens.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01717v1",
            "date": "2020-10-04 23:26:09+00:00",
            "title": "STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation",
            "authors": [
                "Nader Akoury",
                "Shufan Wang",
                "Josh Whiting",
                "Stephen Hood",
                "Nanyun Peng",
                "Mohit Iyyer"
            ],
            "abstract": "Systems for story generation are asked to produce plausible and enjoyable\nstories given an input context. This task is underspecified, as a vast number\nof diverse stories can originate from a single input. The large output space\nmakes it difficult to build and evaluate story generation models, as (1)\nexisting datasets lack rich enough contexts to meaningfully guide models, and\n(2) existing evaluations (both crowdsourced and automatic) are unreliable for\nassessing long-form creative text. To address these issues, we introduce a\ndataset and evaluation platform built from STORIUM, an online collaborative\nstorytelling community. Our author-generated dataset contains 6K lengthy\nstories (125M tokens) with fine-grained natural language annotations (e.g.,\ncharacter goals and attributes) interspersed throughout each narrative, forming\na robust source for guiding models. We evaluate language models fine-tuned on\nour dataset by integrating them onto STORIUM, where real authors can query a\nmodel for suggested story continuations and then edit them. Automatic metrics\ncomputed over these edits correlate well with both user ratings of generated\nstories and qualitative feedback from semi-structured user interviews. We\nrelease both the STORIUM dataset and evaluation platform to spur more\nprincipled research into story generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01735v1",
            "date": "2020-10-05 01:32:20+00:00",
            "title": "MCMH: Learning Multi-Chain Multi-Hop Rules for Knowledge Graph Reasoning",
            "authors": [
                "Lu Zhang",
                "Mo Yu",
                "Tian Gao",
                "Yue Yu"
            ],
            "abstract": "Multi-hop reasoning approaches over knowledge graphs infer a missing\nrelationship between entities with a multi-hop rule, which corresponds to a\nchain of relationships. We extend existing works to consider a generalized form\nof multi-hop rules, where each rule is a set of relation chains. To learn such\ngeneralized rules efficiently, we propose a two-step approach that first\nselects a small set of relation chains as a rule and then evaluates the\nconfidence of the target relationship by jointly scoring the selected chains. A\ngame-theoretical framework is proposed to this end to simultaneously optimize\nthe rule selection and prediction steps. Empirical results show that our\nmulti-chain multi-hop (MCMH) rules result in superior results compared to the\nstandard single-chain approaches, justifying both our formulation of\ngeneralized rules and the effectiveness of the proposed learning framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01745v1",
            "date": "2020-10-05 02:14:13+00:00",
            "title": "On the Effects of Knowledge-Augmented Data in Word Embeddings",
            "authors": [
                "Diego Ramirez-Echavarria",
                "Antonis Bikakis",
                "Luke Dickens",
                "Rob Miller",
                "Andreas Vlachidis"
            ],
            "abstract": "This paper investigates techniques for knowledge injection into word\nembeddings learned from large corpora of unannotated data. These\nrepresentations are trained with word cooccurrence statistics and do not\ncommonly exploit syntactic and semantic information from linguistic knowledge\nbases, which potentially limits their transferability to domains with differing\nlanguage distributions or usages. We propose a novel approach for linguistic\nknowledge injection through data augmentation to learn word embeddings that\nenforce semantic relationships from the data, and systematically evaluate the\nimpact it has on the resulting representations. We show our knowledge\naugmentation approach improves the intrinsic characteristics of the learned\nembeddings while not significantly altering their results on a downstream text\nclassification task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01786v1",
            "date": "2020-10-05 05:25:43+00:00",
            "title": "Corpora Evaluation and System Bias Detection in Multi-document Summarization",
            "authors": [
                "Alvin Dey",
                "Tanya Chowdhury",
                "Yash Kumar Atri",
                "Tanmoy Chakraborty"
            ],
            "abstract": "Multi-document summarization (MDS) is the task of reflecting key points from\nany set of documents into a concise text paragraph. In the past, it has been\nused to aggregate news, tweets, product reviews, etc. from various sources.\nOwing to no standard definition of the task, we encounter a plethora of\ndatasets with varying levels of overlap and conflict between participating\ndocuments. There is also no standard regarding what constitutes summary\ninformation in MDS. Adding to the challenge is the fact that new systems report\nresults on a set of chosen datasets, which might not correlate with their\nperformance on the other datasets. In this paper, we study this heterogeneous\ntask with the help of a few widely used MDS corpora and a suite of\nstate-of-the-art models. We make an attempt to quantify the quality of\nsummarization corpus and prescribe a list of points to consider while proposing\na new MDS corpus. Next, we analyze the reason behind the absence of an MDS\nsystem which achieves superior performance across all corpora. We then observe\nthe extent to which system metrics are influenced, and bias is propagated due\nto corpus properties. The scripts to reproduce the experiments in this work are\navailable at https://github.com/LCS2-IIITD/summarization_bias.git.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01869v3",
            "date": "2020-10-05 09:09:01+00:00",
            "title": "Linguistic Profiling of a Neural Language Model",
            "authors": [
                "Alessio Miaschi",
                "Dominique Brunato",
                "Felice Dell'Orletta",
                "Giulia Venturi"
            ],
            "abstract": "In this paper we investigate the linguistic knowledge learned by a Neural\nLanguage Model (NLM) before and after a fine-tuning process and how this\nknowledge affects its predictions during several classification problems. We\nuse a wide set of probing tasks, each of which corresponds to a distinct\nsentence-level feature extracted from different levels of linguistic\nannotation. We show that BERT is able to encode a wide range of linguistic\ncharacteristics, but it tends to lose this information when trained on specific\ndownstream tasks. We also find that BERT's capacity to encode different kind of\nlinguistic properties has a positive influence on its predictions: the more it\nstores readable linguistic information of a sentence, the higher will be its\ncapacity of predicting the expected label assigned to that sentence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01891v1",
            "date": "2020-10-05 09:54:51+00:00",
            "title": "A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese",
            "authors": [
                "Anh Tuan Nguyen",
                "Mai Hoang Dao",
                "Dat Quoc Nguyen"
            ],
            "abstract": "Semantic parsing is an important NLP task. However, Vietnamese is a\nlow-resource language in this research area. In this paper, we present the\nfirst public large-scale Text-to-SQL semantic parsing dataset for Vietnamese.\nWe extend and evaluate two strong semantic parsing baselines EditSQL (Zhang et\nal., 2019) and IRNet (Guo et al., 2019) on our dataset. We compare the two\nbaselines with key configurations and find that: automatic Vietnamese word\nsegmentation improves the parsing results of both baselines; the normalized\npointwise mutual information (NPMI) score (Bouma, 2009) is useful for schema\nlinking; latent syntactic features extracted from a neural dependency parser\nfor Vietnamese also improve the results; and the monolingual language model\nPhoBERT for Vietnamese (Nguyen and Nguyen, 2020) helps produce higher\nperformances than the recent best multilingual language model XLM-R (Conneau et\nal., 2020).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.01898v1",
            "date": "2020-10-05 10:26:36+00:00",
            "title": "Exploring Semantic Capacity of Terms",
            "authors": [
                "Jie Huang",
                "Zilong Wang",
                "Kevin Chen-Chuan Chang",
                "Wen-mei Hwu",
                "Jinjun Xiong"
            ],
            "abstract": "We introduce and study semantic capacity of terms. For example, the semantic\ncapacity of artificial intelligence is higher than that of linear regression\nsince artificial intelligence possesses a broader meaning scope. Understanding\nsemantic capacity of terms will help many downstream tasks in natural language\nprocessing. For this purpose, we propose a two-step model to investigate\nsemantic capacity of terms, which takes a large text corpus as input and can\nevaluate semantic capacity of terms if the text corpus can provide enough\nco-occurrence information of terms. Extensive experiments in three fields\ndemonstrate the effectiveness and rationality of our model compared with\nwell-designed baselines and human-level evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02010v2",
            "date": "2020-09-29 00:28:30+00:00",
            "title": "Zero-Shot Clinical Acronym Expansion via Latent Meaning Cells",
            "authors": [
                "Griffin Adams",
                "Mert Ketenci",
                "Shreyas Bhave",
                "Adler Perotte",
                "No\u00e9mie Elhadad"
            ],
            "abstract": "We introduce Latent Meaning Cells, a deep latent variable model which learns\ncontextualized representations of words by combining local lexical context and\nmetadata. Metadata can refer to granular context, such as section type, or to\nmore global context, such as unique document ids. Reliance on metadata for\ncontextualized representation learning is apropos in the clinical domain where\ntext is semi-structured and expresses high variation in topics. We evaluate the\nLMC model on the task of zero-shot clinical acronym expansion across three\ndatasets. The LMC significantly outperforms a diverse set of baselines at a\nfraction of the pre-training cost and learns clinically coherent\nrepresentations. We demonstrate that not only is metadata itself very helpful\nfor the task, but that the LMC inference algorithm provides an additional large\nbenefit.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02069v2",
            "date": "2020-10-05 15:06:27+00:00",
            "title": "The Grammar of Emergent Languages",
            "authors": [
                "Oskar van der Wal",
                "Silvan de Boer",
                "Elia Bruni",
                "Dieuwke Hupkes"
            ],
            "abstract": "In this paper, we consider the syntactic properties of languages emerged in\nreferential games, using unsupervised grammar induction (UGI) techniques\noriginally designed to analyse natural language. We show that the considered\nUGI techniques are appropriate to analyse emergent languages and we then study\nif the languages that emerge in a typical referential game setup exhibit\nsyntactic structure, and to what extent this depends on the maximum message\nlength and number of symbols that the agents are allowed to use. Our\nexperiments demonstrate that a certain message length and vocabulary size are\nrequired for structure to emerge, but they also illustrate that more\nsophisticated game scenarios are required to obtain syntactic properties more\nakin to those observed in human language. We argue that UGI techniques should\nbe part of the standard toolkit for analysing emergent languages and release a\ncomprehensive library to facilitate such analysis for future researchers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02140v1",
            "date": "2020-10-05 16:37:52+00:00",
            "title": "Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems",
            "authors": [
                "Jan Deriu",
                "Don Tuggener",
                "Pius von D\u00e4niken",
                "Jon Ander Campos",
                "Alvaro Rodrigo",
                "Thiziri Belkacem",
                "Aitor Soroa",
                "Eneko Agirre",
                "Mark Cieliebak"
            ],
            "abstract": "The lack of time-efficient and reliable evaluation methods hamper the\ndevelopment of conversational dialogue systems (chatbots). Evaluations\nrequiring humans to converse with chatbots are time and cost-intensive, put\nhigh cognitive demands on the human judges, and yield low-quality results. In\nthis work, we introduce \\emph{Spot The Bot}, a cost-efficient and robust\nevaluation framework that replaces human-bot conversations with conversations\nbetween bots. Human judges then only annotate for each entity in a conversation\nwhether they think it is human or not (assuming there are humans participants\nin these conversations). These annotations then allow us to rank chatbots\nregarding their ability to mimic the conversational behavior of humans. Since\nwe expect that all bots are eventually recognized as such, we incorporate a\nmetric that measures which chatbot can uphold human-like behavior the longest,\ni.e., \\emph{Survival Analysis}. This metric has the ability to correlate a\nbot's performance to certain of its characteristics (e.g., \\ fluency or\nsensibleness), yielding interpretable results. The comparably low cost of our\nframework allows for frequent evaluations of chatbots during their evaluation\ncycle. We empirically validate our claims by applying \\emph{Spot The Bot} to\nthree domains, evaluating several state-of-the-art chatbots, and drawing\ncomparisons to related work. The framework is released as a ready-to-use tool.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.02180v2",
            "date": "2020-10-05 17:27:31+00:00",
            "title": "Pareto Probing: Trading Off Accuracy for Complexity",
            "authors": [
                "Tiago Pimentel",
                "Naomi Saphra",
                "Adina Williams",
                "Ryan Cotterell"
            ],
            "abstract": "The question of how to probe contextual word representations for linguistic\nstructure in a way that is both principled and useful has seen significant\nattention recently in the NLP literature. In our contribution to this\ndiscussion, we argue for a probe metric that reflects the fundamental trade-off\nbetween probe complexity and performance: the Pareto hypervolume. To measure\ncomplexity, we present a number of parametric and non-parametric metrics. Our\nexperiments using Pareto hypervolume as an evaluation metric show that probes\noften do not conform to our expectations---e.g., why should the non-contextual\nfastText representations encode more morpho-syntactic information than the\ncontextual BERT representations? These results suggest that common, simplistic\nprobing tasks, such as part-of-speech labeling and dependency arc labeling, are\ninadequate to evaluate the linguistic structure encoded in contextual word\nrepresentations. This leads us to propose full dependency parsing as a probing\ntask. In support of our suggestion that harder probing tasks are necessary, our\nexperiments with dependency parsing reveal a wide gap in syntactic knowledge\nbetween contextual and non-contextual representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02229v1",
            "date": "2020-10-05 18:00:03+00:00",
            "title": "Learning to Generalize for Sequential Decision Making",
            "authors": [
                "Xusen Yin",
                "Ralph Weischedel",
                "Jonathan May"
            ],
            "abstract": "We consider problems of making sequences of decisions to accomplish tasks,\ninteracting via the medium of language. These problems are often tackled with\nreinforcement learning approaches. We find that these models do not generalize\nwell when applied to novel task domains. However, the large amount of\ncomputation necessary to adequately train and explore the search space of\nsequential decision making, under a reinforcement learning paradigm, precludes\nthe inclusion of large contextualized language models, which might otherwise\nenable the desired generalization ability. We introduce a teacher-student\nimitation learning methodology and a means of converting a reinforcement\nlearning model into a natural language understanding model. Together, these\nmethodologies enable the introduction of contextualized language models into\nthe sequential decision making problem space. We show that models can learn\nfaster and generalize more, leveraging both the imitation learning and the\nreformulation. Our models exceed teacher performance on various held-out\ndecision problems, by up to 7% on in-domain problems and 24% on out-of-domain\nproblems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02246v2",
            "date": "2020-10-05 18:01:38+00:00",
            "title": "MedFilter: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge",
            "authors": [
                "Sopan Khosla",
                "Shikhar Vashishth",
                "Jill Fain Lehman",
                "Carolyn Rose"
            ],
            "abstract": "Information extraction from conversational data is particularly challenging\nbecause the task-centric nature of conversation allows for effective\ncommunication of implicit information by humans, but is challenging for\nmachines. The challenges may differ between utterances depending on the role of\nthe speaker within the conversation, especially when relevant expertise is\ndistributed asymmetrically across roles. Further, the challenges may also\nincrease over the conversation as more shared context is built up through\ninformation communicated implicitly earlier in the dialogue. In this paper, we\npropose the novel modeling approach MedFilter, which addresses these insights\nin order to increase performance at identifying and categorizing task-relevant\nutterances, and in so doing, positively impacts performance at a downstream\ninformation extraction task. We evaluate this approach on a corpus of nearly\n7,000 doctor-patient conversations where MedFilter is used to identify\nmedically relevant contributions to the discussion (achieving a 10% improvement\nover SOTA baselines in terms of area under the PR curve). Identifying\ntask-relevant utterances benefits downstream medical processing, achieving\nimprovements of 15%, 105%, and 23% respectively for the extraction of symptoms,\nmedications, and complaints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02256v2",
            "date": "2020-10-05 18:11:23+00:00",
            "title": "An Ensemble Approach for Automatic Structuring of Radiology Reports",
            "authors": [
                "Morteza Pourreza Shahri",
                "Amir Tahmasebi",
                "Bingyang Ye",
                "Henghui Zhu",
                "Javed Aslam",
                "Timothy Ferris"
            ],
            "abstract": "Automatic structuring of electronic medical records is of high demand for\nclinical workflow solutions to facilitate extraction, storage, and querying of\npatient care information. However, developing a scalable solution is extremely\nchallenging, specifically for radiology reports, as most healthcare institutes\nuse either no template or department/institute specific templates. Moreover,\nradiologists' reporting style varies from one to another as sentences are\ntelegraphic and do not follow general English grammar rules. We present an\nensemble method that consolidates the predictions of three models, capturing\nvarious attributes of textual information for automatic labeling of sentences\nwith section labels. These three models are: 1) Focus Sentence model, capturing\ncontext of the target sentence; 2) Surrounding Context model, capturing the\nneighboring context of the target sentence; and finally, 3) Formatting/Layout\nmodel, aimed at learning report formatting cues. We utilize Bi-directional\nLSTMs, followed by sentence encoders, to acquire the context. Furthermore, we\ndefine several features that incorporate the structure of reports. We compare\nour proposed approach against multiple baselines and state-of-the-art\napproaches on a proprietary dataset as well as 100 manually annotated radiology\nnotes from the MIMIC-III dataset, which we are making publicly available. Our\nproposed approach significantly outperforms other approaches by achieving 97.1%\naccuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02260v1",
            "date": "2020-10-05 18:13:45+00:00",
            "title": "Effects of Naturalistic Variation in Goal-Oriented Dialog",
            "authors": [
                "Jatin Ganhotra",
                "Robert Moore",
                "Sachindra Joshi",
                "Kahini Wadhawan"
            ],
            "abstract": "Existing benchmarks used to evaluate the performance of end-to-end neural\ndialog systems lack a key component: natural variation present in human\nconversations. Most datasets are constructed through crowdsourcing, where the\ncrowd workers follow a fixed template of instructions while enacting the role\nof a user/agent. This results in straight-forward, somewhat routine, and mostly\ntrouble-free conversations, as crowd workers do not think to represent the full\nrange of actions that occur naturally with real users. In this work, we\ninvestigate the impact of naturalistic variation on two goal-oriented datasets:\nbAbI dialog task and Stanford Multi-Domain Dataset (SMD). We also propose new\nand more effective testbeds for both datasets, by introducing naturalistic\nvariation by the user. We observe that there is a significant drop in\nperformance (more than 60% in Ent. F1 on SMD and 85% in per-dialog accuracy on\nbAbI task) of recent state-of-the-art end-to-end neural methods such as BossNet\nand GLMP on both datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02307v2",
            "date": "2020-10-05 19:59:05+00:00",
            "title": "KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation",
            "authors": [
                "Wenhu Chen",
                "Yu Su",
                "Xifeng Yan",
                "William Yang Wang"
            ],
            "abstract": "Data-to-text generation has recently attracted substantial interests due to\nits wide applications. Existing methods have shown impressive performance on an\narray of tasks. However, they rely on a significant amount of labeled data for\neach task, which is costly to acquire and thus limits their application to new\ntasks and domains. In this paper, we propose to leverage pre-training and\ntransfer learning to address this issue. We propose a knowledge-grounded\npre-training (KGPT), which consists of two parts, 1) a general\nknowledge-grounded generation model to generate knowledge-enriched text. 2) a\npre-training paradigm on a massive knowledge-grounded text corpus crawled from\nthe web. The pre-trained model can be fine-tuned on various data-to-text\ngeneration tasks to generate task-specific text. We adopt three settings,\nnamely fully-supervised, zero-shot, few-shot to evaluate its effectiveness.\nUnder the fully-supervised setting, our model can achieve remarkable gains over\nthe known baselines. Under zero-shot setting, our model without seeing any\nexamples achieves over 30 ROUGE-L on WebNLG while all other baselines fail.\nUnder the few-shot setting, our model only needs about one-fifteenth as many\nlabeled examples to achieve the same level of performance as baseline models.\nThese experiments consistently prove the strong generalization ability of our\nproposed framework https://github.com/wenhuchen/KGPT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02322v1",
            "date": "2020-10-05 20:27:14+00:00",
            "title": "SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup",
            "authors": [
                "Rongzhi Zhang",
                "Yue Yu",
                "Chao Zhang"
            ],
            "abstract": "Active learning is an important technique for low-resource sequence labeling\ntasks. However, current active sequence labeling methods use the queried\nsamples alone in each iteration, which is an inefficient way of leveraging\nhuman annotations. We propose a simple but effective data augmentation method\nto improve the label efficiency of active sequence labeling. Our method,\nSeqMix, simply augments the queried samples by generating extra labeled\nsequences in each iteration. The key difficulty is to generate plausible\nsequences along with token-level labels. In SeqMix, we address this challenge\nby performing mixup for both sequences and token-level labels of the queried\nsamples. Furthermore, we design a discriminator during sequence mixup, which\njudges whether the generated sequences are plausible or not. Our experiments on\nNamed Entity Recognition and Event Detection tasks show that SeqMix can improve\nthe standard active sequence labeling method by $2.27\\%$--$3.75\\%$ in terms of\n$F_1$ scores. The code and data for SeqMix can be found at\nhttps://github.com/rz-zhang/SeqMix",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02339v2",
            "date": "2020-10-05 21:16:30+00:00",
            "title": "We Don't Speak the Same Language: Interpreting Polarization through Machine Translation",
            "authors": [
                "Ashiqur R. KhudaBukhsh",
                "Rupak Sarkar",
                "Mark S. Kamlet",
                "Tom M. Mitchell"
            ],
            "abstract": "Polarization among US political parties, media and elites is a widely studied\ntopic. Prominent lines of prior research across multiple disciplines have\nobserved and analyzed growing polarization in social media. In this paper, we\npresent a new methodology that offers a fresh perspective on interpreting\npolarization through the lens of machine translation. With a novel proposition\nthat two sub-communities are speaking in two different \\emph{languages}, we\ndemonstrate that modern machine translation methods can provide a simple yet\npowerful and interpretable framework to understand the differences between two\n(or more) large-scale social media discussion data sets at the granularity of\nwords. Via a substantial corpus of 86.6 million comments by 6.5 million users\non over 200,000 news videos hosted by YouTube channels of four prominent US\nnews networks, we demonstrate that simple word-level and phrase-level\ntranslation pairs can reveal deep insights into the current political divide --\nwhat is \\emph{black lives matter} to one can be \\emph{all lives matter} to the\nother.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02357v1",
            "date": "2020-10-05 21:56:00+00:00",
            "title": "Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning",
            "authors": [
                "Tsvetomila Mihaylova",
                "Vlad Niculae",
                "Andr\u00e9 F. T. Martins"
            ],
            "abstract": "Latent structure models are a powerful tool for modeling language data: they\ncan mitigate the error propagation and annotation bottleneck in pipeline\nsystems, while simultaneously uncovering linguistic insights about the data.\nOne challenge with end-to-end training of these models is the argmax operation,\nwhich has null gradient. In this paper, we focus on surrogate gradients, a\npopular strategy to deal with this problem. We explore latent structure\nlearning through the angle of pulling back the downstream learning objective.\nIn this paradigm, we discover a principled motivation for both the\nstraight-through estimator (STE) as well as the recently-proposed SPIGOT - a\nvariant of STE for structured models. Our perspective leads to new algorithms\nin the same family. We empirically compare the known and the novel pulled-back\nestimators against the popular alternatives, yielding new insight for\npractitioners and revealing intriguing failure cases.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02386v1",
            "date": "2020-10-05 23:09:20+00:00",
            "title": "Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning",
            "authors": [
                "Xiaoxiao Guo",
                "Mo Yu",
                "Yupeng Gao",
                "Chuang Gan",
                "Murray Campbell",
                "Shiyu Chang"
            ],
            "abstract": "Interactive Fiction (IF) games with real human-written natural language texts\nprovide a new natural evaluation for language understanding techniques. In\ncontrast to previous text games with mostly synthetic texts, IF games pose\nlanguage understanding challenges on the human-written textual descriptions of\ndiverse and sophisticated game worlds and language generation challenges on the\naction command generation from less restricted combinatorial space. We take a\nnovel perspective of IF game solving and re-formulate it as Multi-Passage\nReading Comprehension (MPRC) tasks. Our approaches utilize the context-query\nattention mechanisms and the structured prediction in MPRC to efficiently\ngenerate and evaluate action outputs and apply an object-centric historical\nobservation retrieval strategy to mitigate the partial observability of the\ntextual observations. Extensive experiments on the recent IF benchmark\n(Jericho) demonstrate clear advantages of our approaches achieving high winning\nrates and low data requirements compared to all previous approaches. Our source\ncode is available at: https://github.com/XiaoxiaoGuo/rcdqn.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02394v2",
            "date": "2020-10-05 23:37:30+00:00",
            "title": "Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks",
            "authors": [
                "Lichao Sun",
                "Congying Xia",
                "Wenpeng Yin",
                "Tingting Liang",
                "Philip S. Yu",
                "Lifang He"
            ],
            "abstract": "Mixup is the latest data augmentation technique that linearly interpolates\ninput examples and the corresponding labels. It has shown strong effectiveness\nin image classification by interpolating images at the pixel level. Inspired by\nthis line of research, in this paper, we explore i) how to apply mixup to\nnatural language processing tasks since text data can hardly be mixed in the\nraw format; ii) if mixup is still effective in transformer-based learning\nmodels, e.g., BERT. To achieve the goal, we incorporate mixup to\ntransformer-based pre-trained architecture, named \"mixup-transformer\", for a\nwide range of NLP tasks while keeping the whole end-to-end training system. We\nevaluate the proposed framework by running extensive experiments on the GLUE\nbenchmark. Furthermore, we also examine the performance of mixup-transformer in\nlow-resource scenarios by reducing the training data with a certain ratio. Our\nstudies show that mixup is a domain-independent data augmentation technique to\npre-trained language models, resulting in significant performance improvement\nfor transformer-based models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02395v1",
            "date": "2020-10-05 23:41:04+00:00",
            "title": "A Generalized Constraint Approach to Bilingual Dictionary Induction for Low-Resource Language Families",
            "authors": [
                "Arbi Haza Nasution",
                "Yohei Murakami",
                "Toru Ishida"
            ],
            "abstract": "The lack or absence of parallel and comparable corpora makes bilingual\nlexicon extraction a difficult task for low-resource languages. The pivot\nlanguage and cognate recognition approaches have been proven useful for\ninducing bilingual lexicons for such languages. We propose constraint-based\nbilingual lexicon induction for closely-related languages by extending\nconstraints from the recent pivot-based induction technique and further\nenabling multiple symmetry assumption cycles to reach many more cognates in the\ntransgraph. We further identify cognate synonyms to obtain many-to-many\ntranslation pairs. This paper utilizes four datasets: one Austronesian\nlow-resource language and three Indo-European high-resource languages. We use\nthree constraint-based methods from our previous work, the Inverse Consultation\nmethod and translation pairs generated from the Cartesian product of input\ndictionaries as baselines. We evaluate our result using the metrics of\nprecision, recall and F-score. Our customizable approach allows the user to\nconduct cross-validation to predict the optimal hyperparameters (cognate\nthreshold and cognate synonym threshold) with various combinations of\nheuristics and the number of symmetry assumption cycles to gain the highest\nF-score. Our proposed methods have statistically significant improvement of\nprecision and F-score compared to our previous constraint-based methods. The\nresults show that our method demonstrates the potential to complement other\nbilingual dictionary creation methods like word alignment models using parallel\ncorpora for high-resource languages while well handling low-resource languages.",
            "journal ref": "ACM Trans. Asian Low-Resour. Lang. Inf. Process. 17, 2, Article 9\n  (November 2017), 29 pages",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02413v1",
            "date": "2020-10-06 01:14:10+00:00",
            "title": "Efficient One-Pass End-to-End Entity Linking for Questions",
            "authors": [
                "Belinda Z. Li",
                "Sewon Min",
                "Srinivasan Iyer",
                "Yashar Mehdad",
                "Wen-tau Yih"
            ],
            "abstract": "We present ELQ, a fast end-to-end entity linking model for questions, which\nuses a biencoder to jointly perform mention detection and linking in one pass.\nEvaluated on WebQSP and GraphQuestions with extended annotations that cover\nmultiple entities per question, ELQ outperforms the previous state of the art\nby a large margin of +12.7% and +19.6% F1, respectively. With a very fast\ninference time (1.57 examples/s on a single CPU), ELQ can be useful for\ndownstream question answering systems. In a proof-of-concept experiment, we\ndemonstrate that using ELQ significantly improves the downstream QA performance\nof GraphRetriever (arXiv:1911.03868). Code and data available at\nhttps://github.com/facebookresearch/BLINK/tree/master/elq",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02416v2",
            "date": "2020-10-06 01:21:11+00:00",
            "title": "Efficient Inference For Neural Machine Translation",
            "authors": [
                "Yi-Te Hsu",
                "Sarthak Garg",
                "Yi-Hsiu Liao",
                "Ilya Chatsviorkin"
            ],
            "abstract": "Large Transformer models have achieved state-of-the-art results in neural\nmachine translation and have become standard in the field. In this work, we\nlook for the optimal combination of known techniques to optimize inference\nspeed without sacrificing translation quality. We conduct an empirical study\nthat stacks various approaches and demonstrates that combination of replacing\ndecoder self-attention with simplified recurrent units, adopting a deep encoder\nand a shallow decoder architecture and multi-head attention pruning can achieve\nup to 109% and 84% speedup on CPU and GPU respectively and reduce the number of\nparameters by 25% while maintaining the same translation quality in terms of\nBLEU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02467v1",
            "date": "2020-10-06 04:18:18+00:00",
            "title": "Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays",
            "authors": [
                "Jianmo Ni",
                "Chun-Nan Hsu",
                "Amilcare Gentili",
                "Julian McAuley"
            ],
            "abstract": "Automatic medical image report generation has drawn growing attention due to\nits potential to alleviate radiologists' workload. Existing work on report\ngeneration often trains encoder-decoder networks to generate complete reports.\nHowever, such models are affected by data bias (e.g.~label imbalance) and face\ncommon issues inherent in text generation models (e.g.~repetition). In this\nwork, we focus on reporting abnormal findings on radiology images; instead of\ntraining on complete radiology reports, we propose a method to identify\nabnormal findings from the reports in addition to grouping them with\nunsupervised clustering and minimal rules. We formulate the task as cross-modal\nretrieval and propose Conditional Visual-Semantic Embeddings to align images\nand fine-grained abnormal findings in a joint embedding space. We demonstrate\nthat our method is able to retrieve abnormal findings and outperforms existing\ngeneration models on both clinical correctness and text generation metrics.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.02481v2",
            "date": "2020-10-06 05:16:38+00:00",
            "title": "Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection",
            "authors": [
                "Hoang Nguyen",
                "Chenwei Zhang",
                "Congying Xia",
                "Philip S. Yu"
            ],
            "abstract": "Few-shot Intent Detection is challenging due to the scarcity of available\nannotated utterances. Although recent works demonstrate that multi-level\nmatching plays an important role in transferring learned knowledge from seen\ntraining classes to novel testing classes, they rely on a static similarity\nmeasure and overly fine-grained matching components. These limitations inhibit\ngeneralizing capability towards Generalized Few-shot Learning settings where\nboth seen and novel classes are co-existent. In this paper, we propose a novel\nSemantic Matching and Aggregation Network where semantic components are\ndistilled from utterances via multi-head self-attention with additional dynamic\nregularization constraints. These semantic components capture high-level\ninformation, resulting in more effective matching between instances. Our\nmulti-perspective matching method provides a comprehensive matching measure to\nenhance representations of both labeled and unlabeled instances. We also\npropose a more challenging evaluation setting that considers classification on\nthe joint all-class label space. Extensive experimental results demonstrate the\neffectiveness of our method. Our code and data are publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02500v1",
            "date": "2020-10-06 06:08:07+00:00",
            "title": "Efficient Meta Lifelong-Learning with Limited Memory",
            "authors": [
                "Zirui Wang",
                "Sanket Vaibhav Mehta",
                "Barnab\u00e1s P\u00f3czos",
                "Jaime Carbonell"
            ],
            "abstract": "Current natural language processing models work well on a single task, yet\nthey often fail to continuously learn new tasks without forgetting previous\nones as they are re-trained throughout their lifetime, a challenge known as\nlifelong learning. State-of-the-art lifelong language learning methods store\npast examples in episodic memory and replay them at both training and inference\ntime. However, as we show later in our experiments, there are three significant\nimpediments: (1) needing unrealistically large memory module to achieve good\nperformance, (2) suffering from negative transfer, (3) requiring multiple local\nadaptation steps for each test example that significantly slows down the\ninference speed. In this paper, we identify three common principles of lifelong\nlearning methods and propose an efficient meta-lifelong framework that combines\nthem in a synergistic fashion. To achieve sample efficiency, our method trains\nthe model in a manner that it learns a better initialization for local\nadaptation. Extensive experiments on text classification and question answering\nbenchmarks demonstrate the effectiveness of our framework by achieving\nstate-of-the-art performance using merely 1% memory size and narrowing the gap\nwith multi-task learning. We further show that our method alleviates both\ncatastrophic forgetting and negative transfer at the same time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02510v2",
            "date": "2020-10-06 06:27:02+00:00",
            "title": "Investigating African-American Vernacular English in Transformer-Based Text Generation",
            "authors": [
                "Sophie Groenwold",
                "Lily Ou",
                "Aesha Parekh",
                "Samhita Honnavalli",
                "Sharon Levy",
                "Diba Mirza",
                "William Yang Wang"
            ],
            "abstract": "The growth of social media has encouraged the written use of African American\nVernacular English (AAVE), which has traditionally been used only in oral\ncontexts. However, NLP models have historically been developed using dominant\nEnglish varieties, such as Standard American English (SAE), due to text corpora\navailability. We investigate the performance of GPT-2 on AAVE text by creating\na dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating\nsyntactic structure and AAVE- or SAE-specific language for each pair. We\nevaluate each sample and its GPT-2 generated text with pretrained sentiment\nclassifiers and find that while AAVE text results in more classifications of\nnegative sentiment than SAE, the use of GPT-2 generally increases occurrences\nof positive sentiment for both. Additionally, we conduct human evaluation of\nAAVE and SAE text generated with GPT-2 to compare contextual rigor and overall\nquality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02523v1",
            "date": "2020-10-06 06:54:12+00:00",
            "title": "Multi-task Learning for Multilingual Neural Machine Translation",
            "authors": [
                "Yiren Wang",
                "ChengXiang Zhai",
                "Hany Hassan Awadalla"
            ],
            "abstract": "While monolingual data has been shown to be useful in improving bilingual\nneural machine translation (NMT), effectively and efficiently leveraging\nmonolingual data for Multilingual NMT (MNMT) systems is a less explored area.\nIn this work, we propose a multi-task learning (MTL) framework that jointly\ntrains the model with the translation task on bitext data and two denoising\ntasks on the monolingual data. We conduct extensive empirical studies on MNMT\nsystems with 10 language pairs from WMT datasets. We show that the proposed\napproach can effectively improve the translation quality for both high-resource\nand low-resource languages with large margin, achieving significantly better\nresults than the individual bilingual models. We also demonstrate the efficacy\nof the proposed approach in the zero-shot setup for language pairs without\nbitext training data. Furthermore, we show the effectiveness of MTL over\npre-training approaches for both NMT and cross-lingual transfer learning NLU\ntasks; the proposed approach outperforms massive scale models trained on single\ntask.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02582v1",
            "date": "2020-10-06 09:46:20+00:00",
            "title": "Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering",
            "authors": [
                "Wei Han",
                "Hantao Huang",
                "Tao Han"
            ],
            "abstract": "Image text carries essential information to understand the scene and perform\nreasoning. Text-based visual question answering (text VQA) task focuses on\nvisual questions that require reading text in images. Existing text VQA systems\ngenerate an answer by selecting from optical character recognition (OCR) texts\nor a fixed vocabulary. Positional information of text is underused and there is\na lack of evidence for the generated answer. As such, this paper proposes a\nlocalization-aware answer prediction network (LaAP-Net) to address this\nchallenge. Our LaAP-Net not only generates the answer to the question but also\npredicts a bounding box as evidence of the generated answer. Moreover, a\ncontext-enriched OCR representation (COR) for multimodal fusion is proposed to\nfacilitate the localization task. Our proposed LaAP-Net outperforms existing\napproaches on three benchmark datasets for the text VQA task by a noticeable\nmargin.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.02586v2",
            "date": "2020-10-06 09:51:04+00:00",
            "title": "Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles",
            "authors": [
                "Carel van Niekerk",
                "Michael Heck",
                "Christian Geishauser",
                "Hsien-Chin Lin",
                "Nurul Lubis",
                "Marco Moresi",
                "Milica Ga\u0161i\u0107"
            ],
            "abstract": "The ability to accurately track what happens during a conversation is\nessential for the performance of a dialogue system. Current state-of-the-art\nmulti-domain dialogue state trackers achieve just over 55% accuracy on the\ncurrent go-to benchmark, which means that in almost every second dialogue turn\nthey place full confidence in an incorrect dialogue state. Belief trackers, on\nthe other hand, maintain a distribution over possible dialogue states. However,\nthey lack in performance compared to dialogue state trackers, and do not\nproduce well calibrated distributions. In this work we present state-of-the-art\nperformance in calibration for multi-domain dialogue belief trackers using a\ncalibrated ensemble of models. Our resulting dialogue belief tracker also\noutperforms previous dialogue belief tracking models in terms of accuracy.",
            "journal ref": "Proceedings of the Findings of the Association for Computational\n  Linguistics: EMNLP 2020, Pages 3096-3102; Association for Computational\n  Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02598v1",
            "date": "2020-10-06 10:17:49+00:00",
            "title": "Embedding Words in Non-Vector Space with Unsupervised Graph Learning",
            "authors": [
                "Max Ryabinin",
                "Sergei Popov",
                "Liudmila Prokhorenkova",
                "Elena Voita"
            ],
            "abstract": "It has become a de-facto standard to represent words as elements of a vector\nspace (word2vec, GloVe). While this approach is convenient, it is unnatural for\nlanguage: words form a graph with a latent hierarchical structure, and this\nstructure has to be revealed and encoded by word embeddings. We introduce\nGraphGlove: unsupervised graph word representations which are learned\nend-to-end. In our setting, each word is a node in a weighted graph and the\ndistance between words is the shortest path distance between the corresponding\nnodes. We adopt a recent method learning a representation of data in the form\nof a differentiable weighted graph and use it to modify the GloVe training\nalgorithm. We show that our graph-based representations substantially\noutperform vector-based methods on word similarity and analogy tasks. Our\nanalysis reveals that the structure of the learned graphs is hierarchical and\nsimilar to that of WordNet, the geometry is highly non-trivial and contains\nsubgraphs with different local topology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02600v2",
            "date": "2020-10-06 10:19:39+00:00",
            "title": "Converting the Point of View of Messages Spoken to Virtual Assistants",
            "authors": [
                "Isabelle G. Lee",
                "Vera Zu",
                "Sai Srujana Buddi",
                "Dennis Liang",
                "Purva Kulkarni",
                "Jack G. M. Fitzgerald"
            ],
            "abstract": "Virtual Assistants can be quite literal at times. If the user says \"tell Bob\nI love him,\" most virtual assistants will extract the message \"I love him\" and\nsend it to the user's contact named Bob, rather than properly converting the\nmessage to \"I love you.\" We designed a system to allow virtual assistants to\ntake a voice message from one user, convert the point of view of the message,\nand then deliver the result to its target user. We developed a rule-based\nmodel, which integrates a linear text classification model, part-of-speech\ntagging, and constituency parsing with rule-based transformation methods. We\nalso investigated Neural Machine Translation (NMT) approaches, including LSTMs,\nCopyNet, and T5. We explored 5 metrics to gauge both naturalness and\nfaithfulness automatically, and we chose to use BLEU plus METEOR for\nfaithfulness and relative perplexity using a separately trained language model\n(GPT) for naturalness. Transformer-Copynet and T5 performed similarly on\nfaithfulness metrics, with T5 achieving slight edge, a BLEU score of 63.8 and a\nMETEOR score of 83.0. CopyNet was the most natural, with a relative perplexity\nof 1.59. CopyNet also has 37 times fewer parameters than T5. We have publicly\nreleased our dataset, which is composed of 46,565 crowd-sourced samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02602v1",
            "date": "2020-10-06 10:24:45+00:00",
            "title": "Joint Semantics and Data-Driven Path Representation for Knowledge Graph Inference",
            "authors": [
                "Guanglin Niu",
                "Bo Li",
                "Yongfei Zhang",
                "Yongpan Sheng",
                "Chuan Shi",
                "Jingyang Li",
                "Shiliang Pu"
            ],
            "abstract": "Inference on a large-scale knowledge graph (KG) is of great importance for KG\napplications like question answering. The path-based reasoning models can\nleverage much information over paths other than pure triples in the KG, which\nface several challenges: all the existing path-based methods are data-driven,\nlacking explainability for path representation. Besides, some methods either\nconsider only relational paths or ignore the heterogeneity between entities and\nrelations both contained in paths, which cannot capture the rich semantics of\npaths well. To address the above challenges, in this work, we propose a novel\njoint semantics and data-driven path representation that balances\nexplainability and generalization in the framework of KG embedding. More\nspecifically, we inject horn rules to obtain the condensed paths by the\ntransparent and explainable path composition procedure. The entity converter is\ndesigned to transform the entities along paths into the representations in the\nsemantic level similar to relations for reducing the heterogeneity between\nentities and relations, in which the KGs both with and without type information\nare considered. Our proposed model is evaluated on two classes of tasks: link\nprediction and path query answering task. The experimental results show that it\nhas a significant performance gain over several different state-of-the-art\nbaselines.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.02605v2",
            "date": "2020-10-06 10:30:48+00:00",
            "title": "DaNetQA: a yes/no Question Answering Dataset for the Russian Language",
            "authors": [
                "Taisia Glushkova",
                "Alexey Machnev",
                "Alena Fenogenova",
                "Tatiana Shavrina",
                "Ekaterina Artemova",
                "Dmitry I. Ignatov"
            ],
            "abstract": "DaNetQA, a new question-answering corpus, follows (Clark et. al, 2019)\ndesign: it comprises natural yes/no questions. Each question is paired with a\nparagraph from Wikipedia and an answer, derived from the paragraph. The task is\nto take both the question and a paragraph as input and come up with a yes/no\nanswer, i.e. to produce a binary output. In this paper, we present a\nreproducible approach to DaNetQA creation and investigate transfer learning\nmethods for task and language transferring. For task transferring we leverage\nthree similar sentence modelling tasks: 1) a corpus of paraphrases,\nParaphraser, 2) an NLI task, for which we use the Russian part of XNLI, 3)\nanother question answering task, SberQUAD. For language transferring we use\nEnglish to Russian translation together with multilingual language fine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02616v1",
            "date": "2020-10-06 10:54:00+00:00",
            "title": "On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers",
            "authors": [
                "Marius Mosbach",
                "Anna Khokhlova",
                "Michael A. Hedderich",
                "Dietrich Klakow"
            ],
            "abstract": "Fine-tuning pre-trained contextualized embedding models has become an\nintegral part of the NLP pipeline. At the same time, probing has emerged as a\nway to investigate the linguistic knowledge captured by pre-trained models.\nVery little is, however, understood about how fine-tuning affects the\nrepresentations of pre-trained models and thereby the linguistic knowledge they\nencode. This paper contributes towards closing this gap. We study three\ndifferent pre-trained models: BERT, RoBERTa, and ALBERT, and investigate\nthrough sentence-level probing how fine-tuning affects their representations.\nWe find that for some probing tasks fine-tuning leads to substantial changes in\naccuracy, possibly suggesting that fine-tuning introduces or even removes\nlinguistic knowledge from a pre-trained model. These changes, however, vary\ngreatly across different models, fine-tuning and probing tasks. Our analysis\nreveals that while fine-tuning indeed changes the representations of a\npre-trained model and these changes are typically larger for higher layers,\nonly in very few cases, fine-tuning has a positive effect on probing accuracy\nthat is larger than just using the pre-trained model with a strong pooling\nmethod. Based on our findings, we argue that both positive and negative effects\nof fine-tuning on probing require a careful interpretation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02648v1",
            "date": "2020-10-06 11:50:54+00:00",
            "title": "On the Sub-Layer Functionalities of Transformer Decoder",
            "authors": [
                "Yilin Yang",
                "Longyue Wang",
                "Shuming Shi",
                "Prasad Tadepalli",
                "Stefan Lee",
                "Zhaopeng Tu"
            ],
            "abstract": "There have been significant efforts to interpret the encoder of\nTransformer-based encoder-decoder architectures for neural machine translation\n(NMT); meanwhile, the decoder remains largely unexamined despite its critical\nrole. During translation, the decoder must predict output tokens by considering\nboth the source-language text from the encoder and the target-language prefix\nproduced in previous steps. In this work, we study how Transformer-based\ndecoders leverage information from the source and target languages --\ndeveloping a universal probe task to assess how information is propagated\nthrough each module of each decoder layer. We perform extensive experiments on\nthree major translation datasets (WMT En-De, En-Fr, and En-Zh). Our analysis\nprovides insight on when and where decoders leverage different sources. Based\non these insights, we demonstrate that the residual feed-forward module in each\nTransformer decoder layer can be dropped with minimal loss of performance -- a\nsignificant reduction in computation and number of parameters, and consequently\na significant boost to both training and inference speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02667v1",
            "date": "2020-10-06 12:38:02+00:00",
            "title": "Incorporating Behavioral Hypotheses for Query Generation",
            "authors": [
                "Ruey-Cheng Chen",
                "Chia-Jung Lee"
            ],
            "abstract": "Generative neural networks have been shown effective on query suggestion.\nCommonly posed as a conditional generation problem, the task aims to leverage\nearlier inputs from users in a search session to predict queries that they will\nlikely issue at a later time. User inputs come in various forms such as\nquerying and clicking, each of which can imply different semantic signals\nchanneled through the corresponding behavioral patterns. This paper induces\nthese behavioral biases as hypotheses for query generation, where a generic\nencoder-decoder Transformer framework is presented to aggregate arbitrary\nhypotheses of choice. Our experimental results show that the proposed approach\nleads to significant improvements on top-$k$ word error rate and Bert F1 Score\ncompared to a recent BART model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02784v1",
            "date": "2020-10-06 14:52:54+00:00",
            "title": "A Multi-Task Incremental Learning Framework with Category Name Embedding for Aspect-Category Sentiment Analysis",
            "authors": [
                "Zehui Dai",
                "Cheng Peng",
                "Huajie Chen",
                "Yadong Ding"
            ],
            "abstract": "(T)ACSA tasks, including aspect-category sentiment analysis (ACSA) and\ntargeted aspect-category sentiment analysis (TACSA), aims at identifying\nsentiment polarity on predefined categories. Incremental learning on new\ncategories is necessary for (T)ACSA real applications. Though current\nmulti-task learning models achieve good performance in (T)ACSA tasks, they\nsuffer from catastrophic forgetting problems in (T)ACSA incremental learning\ntasks. In this paper, to make multi-task learning feasible for incremental\nlearning, we proposed Category Name Embedding network (CNE-net). We set both\nencoder and decoder shared among all categories to weaken the catastrophic\nforgetting problem. Besides the origin input sentence, we applied another input\nfeature, i.e., category name, for task discrimination. Our model achieved\nstate-of-the-art on two (T)ACSA benchmark datasets. Furthermore, we proposed a\ndataset for (T)ACSA incremental learning and achieved the best performance\ncompared with other strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02807v3",
            "date": "2020-10-06 15:16:31+00:00",
            "title": "Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks",
            "authors": [
                "Shubham Toshniwal",
                "Sam Wiseman",
                "Allyson Ettinger",
                "Karen Livescu",
                "Kevin Gimpel"
            ],
            "abstract": "Long document coreference resolution remains a challenging task due to the\nlarge memory and runtime requirements of current models. Recent work doing\nincremental coreference resolution using just the global representation of\nentities shows practical benefits but requires keeping all entities in memory,\nwhich can be impractical for long documents. We argue that keeping all entities\nin memory is unnecessary, and we propose a memory-augmented neural network that\ntracks only a small bounded number of entities at a time, thus guaranteeing a\nlinear runtime in length of document. We show that (a) the model remains\ncompetitive with models with high memory and computational requirements on\nOntoNotes and LitBank, and (b) the model learns an efficient memory management\nstrategy easily outperforming a rule-based strategy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02810v1",
            "date": "2020-10-06 15:18:21+00:00",
            "title": "Swiss Parliaments Corpus, an Automatically Aligned Swiss German Speech to Standard German Text Corpus",
            "authors": [
                "Michel Pl\u00fcss",
                "Lukas Neukom",
                "Manfred Vogel"
            ],
            "abstract": "We present a forced sentence alignment procedure for Swiss German speech and\nStandard German text. It is able to create a speech-to-text corpus in a fully\nautomatic fashion, given an audio recording and the corresponding unaligned\ntranscript. Compared to a manual alignment, it achieves a mean IoU of 0.8401\nwith a sentence recall of 0.9491. When applying our IoU estimate filter, the\nmean IoU can be further improved to 0.9271 at the cost of a lower sentence\nrecall of 0.4881. Using this procedure, we created the Swiss Parliaments\nCorpus, an automatically aligned Swiss German speech to Standard German text\ncorpus. 65 % of the raw data could be transformed to sentence-level\naudio-text-pairs, resulting in 293 hours of training data. We have made the\ncorpus freely available for download.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02840v1",
            "date": "2020-10-06 16:04:12+00:00",
            "title": "Semantic Evaluation for Text-to-SQL with Distilled Test Suites",
            "authors": [
                "Ruiqi Zhong",
                "Tao Yu",
                "Dan Klein"
            ],
            "abstract": "We propose test suite accuracy to approximate semantic accuracy for\nText-to-SQL models. Our method distills a small test suite of databases that\nachieves high code coverage for the gold query from a large number of randomly\ngenerated databases. At evaluation time, it computes the denotation accuracy of\nthe predicted queries on the distilled test suite, hence calculating a tight\nupper-bound for semantic accuracy efficiently. We use our proposed method to\nevaluate 21 models submitted to the Spider leader board and manually verify\nthat our method is always correct on 100 examples. In contrast, the current\nSpider metric leads to a 2.5% false negative rate on average and 8.1% in the\nworst case, indicating that test suite accuracy is needed. Our implementation,\nalong with distilled test suites for eleven Text-to-SQL datasets, is publicly\navailable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.02983v2",
            "date": "2020-10-06 19:18:06+00:00",
            "title": "Plug and Play Autoencoders for Conditional Text Generation",
            "authors": [
                "Florian Mai",
                "Nikolaos Pappas",
                "Ivan Montero",
                "Noah A. Smith",
                "James Henderson"
            ],
            "abstract": "Text autoencoders are commonly used for conditional generation tasks such as\nstyle transfer. We propose methods which are plug and play, where any\npretrained autoencoder can be used, and only require learning a mapping within\nthe autoencoder's embedding space, training embedding-to-embedding (Emb2Emb).\nThis reduces the need for labeled training data for the task and makes the\ntraining procedure more efficient. Crucial to the success of this method is a\nloss term for keeping the mapped embedding on the manifold of the autoencoder\nand a mapping which is trained to navigate the manifold by learning offset\nvectors. Evaluations on style transfer tasks both with and without\nsequence-to-sequence supervision show that our method performs better than or\ncomparable to strong baselines while being up to four times faster.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03017v1",
            "date": "2020-10-06 20:48:58+00:00",
            "title": "On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment",
            "authors": [
                "Zirui Wang",
                "Zachary C. Lipton",
                "Yulia Tsvetkov"
            ],
            "abstract": "Modern multilingual models are trained on concatenated text from multiple\nlanguages in hopes of conferring benefits to each (positive transfer), with the\nmost pronounced benefits accruing to low-resource languages. However, recent\nwork has shown that this approach can degrade performance on high-resource\nlanguages, a phenomenon known as negative interference. In this paper, we\npresent the first systematic study of negative interference. We show that,\ncontrary to previous belief, negative interference also impacts low-resource\nlanguages. While parameters are maximally shared to learn language-universal\nstructures, we demonstrate that language-specific parameters do exist in\nmultilingual models and they are a potential cause of negative interference.\nMotivated by these observations, we also present a meta-learning algorithm that\nobtains better cross-lingual transferability and alleviates negative\ninterference, by adding language-specific layers as meta-parameters and\ntraining them in a manner that explicitly improves shared layers'\ngeneralization on all languages. Overall, our results show that negative\ninterference is more common than previously known, suggesting new directions\nfor improving multilingual representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03073v1",
            "date": "2020-10-06 22:56:31+00:00",
            "title": "Beyond [CLS] through Ranking by Generation",
            "authors": [
                "Cicero Nogueira dos Santos",
                "Xiaofei Ma",
                "Ramesh Nallapati",
                "Zhiheng Huang",
                "Bing Xiang"
            ],
            "abstract": "Generative models for Information Retrieval, where ranking of documents is\nviewed as the task of generating a query from a document's language model, were\nvery successful in various IR tasks in the past. However, with the advent of\nmodern deep neural networks, attention has shifted to discriminative ranking\nfunctions that model the semantic similarity of documents and queries instead.\nRecently, deep generative models such as GPT2 and BART have been shown to be\nexcellent text generators, but their effectiveness as rankers have not been\ndemonstrated yet. In this work, we revisit the generative framework for\ninformation retrieval and show that our generative approaches are as effective\nas state-of-the-art semantic similarity-based discriminative models for the\nanswer selection task. Additionally, we demonstrate the effectiveness of\nunlikelihood losses for IR.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03096v1",
            "date": "2020-10-07 00:58:10+00:00",
            "title": "Knowledge-aware Method for Confusing Charge Prediction",
            "authors": [
                "Xiya Cheng",
                "Sheng Bi",
                "Guilin Qi",
                "Yongzhen Wang"
            ],
            "abstract": "Automatic charge prediction task aims to determine the final charges based on\nfact descriptions of criminal cases, which is a vital application of legal\nassistant systems. Conventional works usually depend on fact descriptions to\npredict charges while ignoring the legal schematic knowledge, which makes it\ndifficult to distinguish confusing charges. In this paper, we propose a\nknowledge-attentive neural network model, which introduces legal schematic\nknowledge about charges and exploit the knowledge hierarchical representation\nas the discriminative features to differentiate confusing charges. Our model\ntakes the textual fact description as the input and learns fact representation\nthrough a graph convolutional network. A legal schematic knowledge transformer\nis utilized to generate crucial knowledge representations oriented to the legal\nschematic knowledge at both the schema and charge levels. We apply a knowledge\nmatching network for effectively incorporating charge information into the fact\nto learn knowledge-aware fact representation. Finally, we use the\nknowledge-aware fact representation for charge prediction. We create two\nreal-world datasets and experimental results show that our proposed model can\noutperform other state-of-the-art baselines on accuracy and F1 score,\nespecially on dealing with confusing charges.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03124v1",
            "date": "2020-10-07 02:48:44+00:00",
            "title": "VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling",
            "authors": [
                "Machel Reid",
                "Edison Marrese-Taylor",
                "Yutaka Matsuo"
            ],
            "abstract": "In this paper, we tackle the task of definition modeling, where the goal is\nto learn to generate definitions of words and phrases. Existing approaches for\nthis task are discriminative, combining distributional and lexical semantics in\nan implicit rather than direct way. To tackle this issue we propose a\ngenerative model for the task, introducing a continuous latent variable to\nexplicitly model the underlying relationship between a phrase used within a\ncontext and its definition. We rely on variational inference for estimation and\nleverage contextualized word embeddings for improved performance. Our approach\nis evaluated on four existing challenging benchmarks with the addition of two\nnew datasets, \"Cambridge\" and the first non-English corpus \"Robert\", which we\nrelease to complement our empirical study. Our Variational Contextual\nDefinition Modeler (VCDM) achieves state-of-the-art performance in terms of\nautomatic and human evaluation metrics, demonstrating the effectiveness of our\napproach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03127v1",
            "date": "2020-10-07 02:50:38+00:00",
            "title": "A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions",
            "authors": [
                "Takuma Udagawa",
                "Takato Yamazaki",
                "Akiko Aizawa"
            ],
            "abstract": "Recent models achieve promising results in visually grounded dialogues.\nHowever, existing datasets often contain undesirable biases and lack\nsophisticated linguistic analyses, which make it difficult to understand how\nwell current models recognize their precise linguistic structures. To address\nthis problem, we make two design choices: first, we focus on OneCommon Corpus\n\\citep{udagawa2019natural,udagawa2020annotated}, a simple yet challenging\ncommon grounding dataset which contains minimal bias by design. Second, we\nanalyze their linguistic structures based on \\textit{spatial expressions} and\nprovide comprehensive and reliable annotation for 600 dialogues. We show that\nour annotation captures important linguistic structures including\npredicate-argument structure, modification and ellipsis. In our experiments, we\nassess the model's understanding of these structures through reference\nresolution. We demonstrate that our annotation can reveal both the strengths\nand weaknesses of baseline models in essential levels of detail. Overall, we\npropose a novel framework and resource for investigating fine-grained language\nunderstanding in visually grounded dialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03138v1",
            "date": "2020-10-07 03:40:49+00:00",
            "title": "Improving Context Modeling in Neural Topic Segmentation",
            "authors": [
                "Linzi Xing",
                "Brad Hackinen",
                "Giuseppe Carenini",
                "Francesco Trebbi"
            ],
            "abstract": "Topic segmentation is critical in key NLP tasks and recent works favor highly\neffective neural supervised approaches. However, current neural solutions are\narguably limited in how they model context. In this paper, we enhance a\nsegmenter based on a hierarchical attention BiLSTM network to better model\ncontext, by adding a coherence-related auxiliary task and restricted\nself-attention. Our optimized segmenter outperforms SOTA approaches when\ntrained and tested on three datasets. We also the robustness of our proposed\nmodel in domain transfer setting by training a model on a large-scale dataset\nand testing it on four challenging real-world benchmarks. Furthermore, we apply\nour proposed strategy to two other languages (German and Chinese), and show its\neffectiveness in multilingual scenarios.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03146v1",
            "date": "2020-10-07 04:05:01+00:00",
            "title": "Unsupervised Parsing via Constituency Tests",
            "authors": [
                "Steven Cao",
                "Nikita Kitaev",
                "Dan Klein"
            ],
            "abstract": "We propose a method for unsupervised parsing based on the linguistic notion\nof a constituency test. One type of constituency test involves modifying the\nsentence via some transformation (e.g. replacing the span with a pronoun) and\nthen judging the result (e.g. checking if it is grammatical). Motivated by this\nidea, we design an unsupervised parser by specifying a set of transformations\nand using an unsupervised neural acceptability model to make grammaticality\ndecisions. To produce a tree given a sentence, we score each span by\naggregating its constituency test judgments, and we choose the binary tree with\nthe highest total score. While this approach already achieves performance in\nthe range of current methods, we further improve accuracy by fine-tuning the\ngrammaticality model through a refinement procedure, where we alternate between\nimproving the estimated trees and improving the grammaticality model. The\nrefined model achieves 62.8 F1 on the Penn Treebank test set, an absolute\nimprovement of 7.6 points over the previous best published result.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03157v3",
            "date": "2020-10-07 04:49:48+00:00",
            "title": "Knowledge-enriched, Type-constrained and Grammar-guided Question Generation over Knowledge Bases",
            "authors": [
                "Sheng Bi",
                "Xiya Cheng",
                "Yuan-Fang Li",
                "Yongzhen Wang",
                "Guilin Qi"
            ],
            "abstract": "Question generation over knowledge bases (KBQG) aims at generating\nnatural-language questions about a subgraph, i.e. a set of (connected) triples.\nTwo main challenges still face the current crop of encoder-decoder-based\nmethods, especially on small subgraphs: (1) low diversity and poor fluency due\nto the limited information contained in the subgraphs, and (2) semantic drift\ndue to the decoder's oblivion of the semantics of the answer entity. We propose\nan innovative knowledge-enriched, type-constrained and grammar-guided KBQG\nmodel, named KTG, to addresses the above challenges. In our model, the encoder\nis equipped with auxiliary information from the KB, and the decoder is\nconstrained with word types during QG. Specifically, entity domain and\ndescription, as well as relation hierarchy information are considered to\nconstruct question contexts, while a conditional copy mechanism is incorporated\nto modulate question semantics according to current word types. Besides, a\nnovel reward function featuring grammatical similarity is designed to improve\nboth generative richness and syntactic correctness via reinforcement learning.\nExtensive experiments show that our proposed model outperforms existing methods\nby a significant margin on two widely-used benchmark datasets SimpleQuestion\nand PathQuestion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03179v1",
            "date": "2020-10-07 05:23:27+00:00",
            "title": "Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages",
            "authors": [
                "Michael A. Hedderich",
                "David Adelani",
                "Dawei Zhu",
                "Jesujoba Alabi",
                "Udia Markus",
                "Dietrich Klakow"
            ],
            "abstract": "Multilingual transformer models like mBERT and XLM-RoBERTa have obtained\ngreat improvements for many NLP tasks on a variety of languages. However,\nrecent works also showed that results from high-resource languages could not be\neasily transferred to realistic, low-resource scenarios. In this work, we study\ntrends in performance for different amounts of available resources for the\nthree African languages Hausa, isiXhosa and Yor\\`ub\\'a on both NER and topic\nclassification. We show that in combination with transfer learning or distant\nsupervision, these models can achieve with as little as 10 or 100 labeled\nsentences the same performance as baselines with much more supervised training\ndata. However, we also find settings where this does not hold. Our discussions\nand additional experiments on assumptions such as time and hardware\nrestrictions highlight challenges and opportunities in low-resource learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03189v2",
            "date": "2020-10-07 05:40:25+00:00",
            "title": "Theedhum Nandrum@Dravidian-CodeMix-FIRE2020: A Sentiment Polarity Classifier for YouTube Comments with Code-switching between Tamil, Malayalam and English",
            "authors": [
                "BalaSundaraRaman Lakshmanan",
                "Sanjeeth Kumar Ravindranath"
            ],
            "abstract": "Theedhum Nandrum is a sentiment polarity detection system using two\napproaches--a Stochastic Gradient Descent (SGD) based classifier and a Long\nShort-term Memory (LSTM) based Classifier. Our approach utilises language\nfeatures like use of emoji, choice of scripts and code mixing which appeared\nquite marked in the datasets specified for the Dravidian Codemix - FIRE 2020\ntask. The hyperparameters for the SGD were tuned using GridSearchCV. Our system\nwas ranked 4th in Tamil-English with a weighted average F1 score of 0.62 and\n9th in Malayalam-English with a score of 0.65. We achieved a weighted average\nF1 score of 0.77 for Tamil-English using a Logistic Regression based model\nafter the task deadline. This performance betters the top ranked classifier on\nthis dataset by a wide margin. Our use of language-specific Soundex to\nharmonise the spelling variants in code-mixed data appears to be a novel\napplication of Soundex. Our complete code is published in github at\nhttps://github.com/oligoglot/theedhum-nandrum.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03205v1",
            "date": "2020-10-07 06:25:39+00:00",
            "title": "Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions",
            "authors": [
                "Bodhisattwa Prasad Majumder",
                "Harsh Jhamtani",
                "Taylor Berg-Kirkpatrick",
                "Julian McAuley"
            ],
            "abstract": "Existing persona-grounded dialog models often fail to capture simple\nimplications of given persona descriptions, something which humans are able to\ndo seamlessly. For example, state-of-the-art models cannot infer that interest\nin hiking might imply love for nature or longing for a break. In this paper, we\npropose to expand available persona sentences using existing commonsense\nknowledge bases and paraphrasing resources to imbue dialog models with access\nto an expanded and richer set of persona descriptions. Additionally, we\nintroduce fine-grained grounding on personas by encouraging the model to make a\ndiscrete choice among persona sentences while synthesizing a dialog response.\nSince such a choice is not observed in the data, we model it using a discrete\nlatent random variable and use variational learning to sample from hundreds of\npersona expansions. Our model outperforms competitive baselines on the\nPersonaChat dataset in terms of dialog quality and diversity while achieving\npersona-consistent and controllable dialog generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03222v1",
            "date": "2020-10-07 07:03:30+00:00",
            "title": "Unsupervised Evaluation for Question Answering with Transformers",
            "authors": [
                "Lukas Muttenthaler",
                "Isabelle Augenstein",
                "Johannes Bjerva"
            ],
            "abstract": "It is challenging to automatically evaluate the answer of a QA model at\ninference time. Although many models provide confidence scores, and simple\nheuristics can go a long way towards indicating answer correctness, such\nmeasures are heavily dataset-dependent and are unlikely to generalize. In this\nwork, we begin by investigating the hidden representations of questions,\nanswers, and contexts in transformer-based QA architectures. We observe a\nconsistent pattern in the answer representations, which we show can be used to\nautomatically evaluate whether or not a predicted answer span is correct. Our\nmethod does not require any labeled data and outperforms strong heuristic\nbaselines, across 2 datasets and 7 domains. We are able to predict whether or\nnot a model's answer is correct with 91.37% accuracy on SQuAD, and 80.7%\naccuracy on SubjQA. We expect that this method will have broad applications,\ne.g., in the semi-automatic development of QA datasets",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03272v1",
            "date": "2020-10-07 08:45:37+00:00",
            "title": "Narrative Text Generation with a Latent Discrete Plan",
            "authors": [
                "Harsh Jhamtani",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "Past work on story generation has demonstrated the usefulness of conditioning\non a generation plan to generate coherent stories. However, these approaches\nhave used heuristics or off-the-shelf models to first tag training stories with\nthe desired type of plan, and then train generation models in a supervised\nfashion. In this paper, we propose a deep latent variable model that first\nsamples a sequence of anchor words, one per sentence in the story, as part of\nits generative process. During training, our model treats the sequence of\nanchor words as a latent variable and attempts to induce anchoring sequences\nthat help guide generation in an unsupervised fashion. We conduct experiments\nwith several types of sentence decoder distributions: left-to-right and\nnon-monotonic, with different degrees of restriction. Further, since we use\namortized variational inference to train our model, we introduce two\ncorresponding types of inference network for predicting the posterior on anchor\nwords. We conduct human evaluations which demonstrate that the stories produced\nby our model are rated better in comparison with baselines which do not\nconsider story plans, and are similar or better in quality relative to\nbaselines which use external supervision for plans. Additionally, the proposed\nmodel gets favorable scores when evaluated on perplexity, diversity, and\ncontrol of story via discrete plan.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03274v1",
            "date": "2020-10-07 08:46:02+00:00",
            "title": "Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering",
            "authors": [
                "Harsh Jhamtani",
                "Peter Clark"
            ],
            "abstract": "Despite the rapid progress in multihop question-answering (QA), models still\nhave trouble explaining why an answer is correct, with limited explanation\ntraining data available to learn from. To address this, we introduce three\nexplanation datasets in which explanations formed from corpus facts are\nannotated. Our first dataset, eQASC, contains over 98K explanation annotations\nfor the multihop question answering dataset QASC, and is the first that\nannotates multiple candidate explanations for each answer. The second dataset\neQASC-perturbed is constructed by crowd-sourcing perturbations (while\npreserving their validity) of a subset of explanations in QASC, to test\nconsistency and generalization of explanation prediction models. The third\ndataset eOBQA is constructed by adding explanation annotations to the OBQA\ndataset to test generalization of models trained on eQASC. We show that this\ndata can be used to significantly improve explanation quality (+14% absolute F1\nover a strong retrieval baseline) using a BERT-based classifier, but still\nbehind the upper bound, offering a new challenge for future research. We also\nexplore a delexicalized chain representation in which repeated noun phrases are\nreplaced by variables, thus turning them into generalized reasoning chains (for\nexample: \"X is a Y\" AND \"Y has Z\" IMPLIES \"X has Z\"). We find that generalized\nchains maintain performance while also being more robust to certain\nperturbations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03343v1",
            "date": "2020-10-07 11:40:49+00:00",
            "title": "Slice-Aware Neural Ranking",
            "authors": [
                "Gustavo Penha",
                "Claudia Hauff"
            ],
            "abstract": "Understanding when and why neural ranking models fail for an IR task via\nerror analysis is an important part of the research cycle. Here we focus on the\nchallenges of (i) identifying categories of difficult instances (a pair of\nquestion and response candidates) for which a neural ranker is ineffective and\n(ii) improving neural ranking for such instances. To address both challenges we\nresort to slice-based learning for which the goal is to improve effectiveness\nof neural models for slices (subsets) of data. We address challenge (i) by\nproposing different slicing functions (SFs) that select slices of the\ndataset---based on prior work we heuristically capture different failures of\nneural rankers. Then, for challenge (ii) we adapt a neural ranking model to\nlearn slice-aware representations, i.e. the adapted model learns to represent\nthe question and responses differently based on the model's prediction of which\nslices they belong to. Our experimental results (the source code and data are\navailable at https://github.com/Guzpenha/slice_based_learning) across three\ndifferent ranking tasks and four corpora show that slice-based learning\nimproves the effectiveness by an average of 2% over a neural ranker that is not\nslice-aware.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.03369v1",
            "date": "2020-10-07 12:30:30+00:00",
            "title": "Toward Stance-based Personas for Opinionated Dialogues",
            "authors": [
                "Thomas Scialom",
                "Serra Sinem Tekiroglu",
                "Jacopo Staiano",
                "Marco Guerini"
            ],
            "abstract": "In the context of chit-chat dialogues it has been shown that endowing systems\nwith a persona profile is important to produce more coherent and meaningful\nconversations. Still, the representation of such personas has thus far been\nlimited to a fact-based representation (e.g. \"I have two cats.\"). We argue that\nthese representations remain superficial w.r.t. the complexity of human\npersonality. In this work, we propose to make a step forward and investigate\nstance-based persona, trying to grasp more profound characteristics, such as\nopinions, values, and beliefs to drive language generation. To this end, we\nintroduce a novel dataset allowing to explore different stance-based persona\nrepresentations and their impact on claim generation, showing that they are\nable to grasp abstract and profound aspects of the author persona.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03412v1",
            "date": "2020-10-07 13:40:32+00:00",
            "title": "Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation",
            "authors": [
                "Weijia Xu",
                "Xing Niu",
                "Marine Carpuat"
            ],
            "abstract": "While Iterative Back-Translation and Dual Learning effectively incorporate\nmonolingual training data in neural machine translation, they use different\nobjectives and heuristic gradient approximation strategies, and have not been\nextensively compared. We introduce a novel dual reconstruction objective that\nprovides a unified view of Iterative Back-Translation and Dual Learning. It\nmotivates a theoretical analysis and controlled empirical study on\nGerman-English and Turkish-English tasks, which both suggest that Iterative\nBack-Translation is more effective than Dual Learning despite its relative\nsimplicity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03446v1",
            "date": "2020-10-07 14:38:35+00:00",
            "title": "Analogies minus analogy test: measuring regularities in word embeddings",
            "authors": [
                "Louis Fournier",
                "Emmanuel Dupoux",
                "Ewan Dunbar"
            ],
            "abstract": "Vector space models of words have long been claimed to capture linguistic\nregularities as simple vector translations, but problems have been raised with\nthis claim. We decompose and empirically analyze the classic arithmetic word\nanalogy test, to motivate two new metrics that address the issues with the\nstandard test, and which distinguish between class-wise offset concentration\n(similar directions between pairs of words drawn from different broad classes,\nsuch as France--London, China--Ottawa, ...) and pairing consistency (the\nexistence of a regular transformation between correctly-matched pairs such as\nFrance:Paris::China:Beijing). We show that, while the standard analogy test is\nflawed, several popular word embeddings do nevertheless encode linguistic\nregularities.",
            "journal ref": "Proceedings of CoNLL 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03636v2",
            "date": "2020-10-07 20:22:54+00:00",
            "title": "MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics",
            "authors": [
                "Anthony Chen",
                "Gabriel Stanovsky",
                "Sameer Singh",
                "Matt Gardner"
            ],
            "abstract": "Posing reading comprehension as a generation problem provides a great deal of\nflexibility, allowing for open-ended questions with few restrictions on\npossible answers. However, progress is impeded by existing generation metrics,\nwhich rely on token overlap and are agnostic to the nuances of reading\ncomprehension. To address this, we introduce a benchmark for training and\nevaluating generative reading comprehension metrics: MOdeling Correctness with\nHuman Annotations. MOCHA contains 40K human judgement scores on model outputs\nfrom 6 diverse question answering datasets and an additional set of minimal\npairs for evaluation. Using MOCHA, we train a Learned Evaluation metric for\nReading Comprehension, LERC, to mimic human judgement scores. LERC outperforms\nbaseline metrics by 10 to 36 absolute Pearson points on held-out annotations.\nWhen we evaluate robustness on minimal pairs, LERC achieves 80% accuracy,\noutperforming baselines by 14 to 26 absolute percentage points while leaving\nsignificant room for improvement. MOCHA presents a challenging problem for\ndeveloping accurate and robust generative reading comprehension metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03706v5",
            "date": "2020-10-08 00:36:33+00:00",
            "title": "Learning to Recombine and Resample Data for Compositional Generalization",
            "authors": [
                "Ekin Aky\u00fcrek",
                "Afra Feyza Aky\u00fcrek",
                "Jacob Andreas"
            ],
            "abstract": "Flexible neural sequence models outperform grammar- and automaton-based\ncounterparts on a variety of tasks. However, neural models perform poorly in\nsettings requiring compositional generalization beyond the training data --\nparticularly to rare or unseen subsequences. Past work has found symbolic\nscaffolding (e.g. grammars or automata) essential in these settings. We\ndescribe R&R, a learned data augmentation scheme that enables a large category\nof compositional generalizations without appeal to latent symbolic structure.\nR&R has two components: recombination of original training examples via a\nprototype-based generative model and resampling of generated examples to\nencourage extrapolation. Training an ordinary neural sequence model on a\ndataset augmented with recombined and resampled examples significantly improves\ngeneralization in two language processing problems -- instruction following\n(SCAN) and morphological analysis (SIGMORPHON 2018) -- where R&R enables\nlearning of new constructions and tenses from as few as eight initial examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03755v1",
            "date": "2020-10-08 04:37:22+00:00",
            "title": "Generalizable and Explainable Dialogue Generation via Explicit Action Learning",
            "authors": [
                "Xinting Huang",
                "Jianzhong Qi",
                "Yu Sun",
                "Rui Zhang"
            ],
            "abstract": "Response generation for task-oriented dialogues implicitly optimizes two\nobjectives at the same time: task completion and language quality. Conditioned\nresponse generation serves as an effective approach to separately and better\noptimize these two objectives. Such an approach relies on system action\nannotations which are expensive to obtain. To alleviate the need of action\nannotations, latent action learning is introduced to map each utterance to a\nlatent representation. However, this approach is prone to over-dependence on\nthe training data, and the generalization capability is thus restricted. To\naddress this issue, we propose to learn natural language actions that represent\nutterances as a span of words. This explicit action representation promotes\ngeneralization via the compositional structure of language. It also enables an\nexplainable generation process. Our proposed unsupervised approach learns a\nmemory component to summarize system utterances into a short span of words. To\nfurther promote a compact action representation, we propose an auxiliary task\nthat restores state annotations as the summarized dialogue context using the\nmemory component. Our proposed approach outperforms latent action baselines on\nMultiWOZ, a benchmark multi-domain dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03760v1",
            "date": "2020-10-08 04:44:00+00:00",
            "title": "Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference",
            "authors": [
                "Xiaoan Ding",
                "Tianyu Liu",
                "Baobao Chang",
                "Zhifang Sui",
                "Kevin Gimpel"
            ],
            "abstract": "While discriminative neural network classifiers are generally preferred,\nrecent work has shown advantages of generative classifiers in term of data\nefficiency and robustness. In this paper, we focus on natural language\ninference (NLI). We propose GenNLI, a generative classifier for NLI tasks, and\nempirically characterize its performance by comparing it to five baselines,\nincluding discriminative models and large-scale pretrained language\nrepresentation models like BERT. We explore training objectives for\ndiscriminative fine-tuning of our generative classifiers, showing improvements\nover log loss fine-tuning from prior work . In particular, we find strong\nresults with a simple unbounded modification to log loss, which we call the\n\"infinilog loss\". Our experiments show that GenNLI outperforms both\ndiscriminative and pretrained baselines across several challenging NLI\nexperimental settings, including small training sets, imbalanced label\ndistributions, and label noise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03777v2",
            "date": "2020-10-08 05:40:45+00:00",
            "title": "An Empirical Study on Model-agnostic Debiasing Strategies for Robust Natural Language Inference",
            "authors": [
                "Tianyu Liu",
                "Xin Zheng",
                "Xiaoan Ding",
                "Baobao Chang",
                "Zhifang Sui"
            ],
            "abstract": "The prior work on natural language inference (NLI) debiasing mainly targets\nat one or few known biases while not necessarily making the models more robust.\nIn this paper, we focus on the model-agnostic debiasing strategies and explore\nhow to (or is it possible to) make the NLI models robust to multiple distinct\nadversarial attacks while keeping or even strengthening the models'\ngeneralization power. We firstly benchmark prevailing neural NLI models\nincluding pretrained ones on various adversarial datasets. We then try to\ncombat distinct known biases by modifying a mixture of experts (MoE) ensemble\nmethod and show that it's nontrivial to mitigate multiple NLI biases at the\nsame time, and that model-level ensemble method outperforms MoE ensemble\nmethod. We also perform data augmentation including text swap, word\nsubstitution and paraphrase and prove its efficiency in combating various\n(though not all) adversarial attacks at the same time. Finally, we investigate\nseveral methods to merge heterogeneous training data (1.35M) and perform model\nensembling, which are straightforward but effective to strengthen NLI models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03802v2",
            "date": "2020-10-08 07:06:38+00:00",
            "title": "TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling",
            "authors": [
                "Parker Riley",
                "Noah Constant",
                "Mandy Guo",
                "Girish Kumar",
                "David Uthus",
                "Zarana Parekh"
            ],
            "abstract": "We present a novel approach to the problem of text style transfer. Unlike\nprevious approaches that use parallel or non-parallel labeled data, our\ntechnique removes the need for labels entirely, relying instead on the implicit\nconnection in style between adjacent sentences in unlabeled text. We show that\nT5 (Raffel et al., 2020), a strong pretrained text-to-text model, can be\nadapted to extract a style vector from arbitrary text and use this vector to\ncondition the decoder to perform style transfer. As the resulting learned style\nvector space encodes many facets of textual style, we recast transfers as\n\"targeted restyling\" vector operations that adjust specific attributes of the\ninput text while preserving others. When trained over unlabeled Amazon reviews\ndata, our resulting TextSETTR model is competitive on sentiment transfer, even\nwhen given only four exemplars of each class. Furthermore, we demonstrate that\na single model trained on unlabeled Common Crawl data is capable of\ntransferring along multiple dimensions including dialect, emotiveness,\nformality, politeness, and sentiment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03813v2",
            "date": "2020-10-08 07:40:21+00:00",
            "title": "On the importance of pre-training data volume for compact language models",
            "authors": [
                "Vincent Micheli",
                "Martin d'Hoffschmidt",
                "Fran\u00e7ois Fleuret"
            ],
            "abstract": "Recent advances in language modeling have led to computationally intensive\nand resource-demanding state-of-the-art models. In an effort towards\nsustainable practices, we study the impact of pre-training data volume on\ncompact language models. Multiple BERT-based models are trained on gradually\nincreasing amounts of French text. Through fine-tuning on the French Question\nAnswering Dataset (FQuAD), we observe that well-performing models are obtained\nwith as little as 100 MB of text. In addition, we show that past critically low\namounts of pre-training data, an intermediate pre-training step on the\ntask-specific corpus does not yield substantial improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03863v1",
            "date": "2020-10-08 09:32:21+00:00",
            "title": "What Can We Do to Improve Peer Review in NLP?",
            "authors": [
                "Anna Rogers",
                "Isabelle Augenstein"
            ],
            "abstract": "Peer review is our best tool for judging the quality of conference\nsubmissions, but it is becoming increasingly spurious. We argue that a part of\nthe problem is that the reviewers and area chairs face a poorly defined task\nforcing apples-to-oranges comparisons. There are several potential ways\nforward, but the key difficulty is creating the incentives and mechanisms for\ntheir consistent implementation in the NLP community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.03920v1",
            "date": "2020-10-08 12:05:48+00:00",
            "title": "Predicting Typological Features in WALS using Language Embeddings and Conditional Probabilities: \u00daFAL Submission to the SIGTYP 2020 Shared Task",
            "authors": [
                "Martin Vastl",
                "Daniel Zeman",
                "Rudolf Rosa"
            ],
            "abstract": "We present our submission to the SIGTYP 2020 Shared Task on the prediction of\ntypological features. We submit a constrained system, predicting typological\nfeatures only based on the WALS database. We investigate two approaches. The\nsimpler of the two is a system based on estimating correlation of feature\nvalues within languages by computing conditional probabilities and mutual\ninformation. The second approach is to train a neural predictor operating on\nprecomputed language embeddings based on WALS features. Our submitted system\ncombines the two approaches based on their self-estimated confidence scores. We\nreach the accuracy of 70.7% on the test data and rank first in the shared task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04098v2",
            "date": "2020-10-08 16:27:03+00:00",
            "title": "BERTering RAMS: What and How Much does BERT Already Know About Event Arguments? -- A Study on the RAMS Dataset",
            "authors": [
                "Varun Gangal",
                "Eduard Hovy"
            ],
            "abstract": "Using the attention map based probing frame-work from (Clark et al., 2019),\nwe observe that, on the RAMS dataset (Ebner et al., 2020), BERT's attention\nheads have modest but well above-chance ability to spot event arguments sans\nany training or domain finetuning, vary-ing from a low of 17.77% for Place to a\nhigh of 51.61% for Artifact. Next, we find that linear combinations of these\nheads, estimated with approx 11% of available total event argument detection\nsupervision, can push performance well-higher for some roles - highest two\nbeing Victim (68.29% Accuracy) and Artifact(58.82% Accuracy). Furthermore, we\ninvestigate how well our methods do for cross-sentence event arguments. We\npropose a procedure to isolate \"best heads\" for cross-sentence argument\ndetection separately of those for intra-sentence arguments. The heads thus\nestimated have superior cross-sentence performance compared to their jointly\nestimated equivalents, albeit only under the unrealistic assumption that we\nalready know the argument is present in an-other sentence. Lastly, we seek to\nisolate to what extent our numbers stem from lexical frequency based\nassociations between gold arguments and roles. We propose NONCE, a scheme to\ncreate adversarial test examples by replacing gold arguments with randomly\ngenerated \"nonce\" words. We find that learnt linear combinations are robust to\nNONCE, though individual best heads can be more sensitive.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04260v1",
            "date": "2020-10-08 21:16:30+00:00",
            "title": "Fake Reviews Detection through Analysis of Linguistic Features",
            "authors": [
                "Faranak Abri",
                "Luis Felipe Gutierrez",
                "Akbar Siami Namin",
                "Keith S. Jones",
                "David R. W. Sears"
            ],
            "abstract": "Online reviews play an integral part for success or failure of businesses.\nPrior to purchasing services or goods, customers first review the online\ncomments submitted by previous customers. However, it is possible to\nsuperficially boost or hinder some businesses through posting counterfeit and\nfake reviews. This paper explores a natural language processing approach to\nidentify fake reviews. We present a detailed analysis of linguistic features\nfor distinguishing fake and trustworthy online reviews. We study 15 linguistic\nfeatures and measure their significance and importance towards the\nclassification schemes employed in this study. Our results indicate that fake\nreviews tend to include more redundant terms and pauses, and generally contain\nlonger sentences. The application of several machine learning classification\nalgorithms revealed that we were able to discriminate fake from real reviews\nwith high accuracy using these linguistic features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04301v4",
            "date": "2020-10-08 23:41:39+00:00",
            "title": "Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling",
            "authors": [
                "Jonathan Shen",
                "Ye Jia",
                "Mike Chrzanowski",
                "Yu Zhang",
                "Isaac Elias",
                "Heiga Zen",
                "Yonghui Wu"
            ],
            "abstract": "This paper presents Non-Attentive Tacotron based on the Tacotron 2\ntext-to-speech model, replacing the attention mechanism with an explicit\nduration predictor. This improves robustness significantly as measured by\nunaligned duration ratio and word deletion rate, two metrics introduced in this\npaper for large-scale robustness evaluation using a pre-trained speech\nrecognition model. With the use of Gaussian upsampling, Non-Attentive Tacotron\nachieves a 5-scale mean opinion score for naturalness of 4.41, slightly\noutperforming Tacotron 2. The duration predictor enables both utterance-wide\nand per-phoneme control of duration at inference time. When accurate target\ndurations are scarce or unavailable in the training data, we propose a method\nusing a fine-grained variational auto-encoder to train the duration predictor\nin a semi-supervised or unsupervised manner, with results almost as good as\nsupervised training.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2010.04302v1",
            "date": "2020-10-08 23:58:57+00:00",
            "title": "Masked ELMo: An evolution of ELMo towards fully contextual RNN language models",
            "authors": [
                "Gregory Senay",
                "Emmanuelle Salin"
            ],
            "abstract": "This paper presents Masked ELMo, a new RNN-based model for language model\npre-training, evolved from the ELMo language model. Contrary to ELMo which only\nuses independent left-to-right and right-to-left contexts, Masked ELMo learns\nfully bidirectional word representations. To achieve this, we use the same\nMasked language model objective as BERT. Additionally, thanks to optimizations\non the LSTM neuron, the integration of mask accumulation and bidirectional\ntruncated backpropagation through time, we have increased the training speed of\nthe model substantially. All these improvements make it possible to pre-train a\nbetter language model than ELMo while maintaining a low computational cost. We\nevaluate Masked ELMo by comparing it to ELMo within the same protocol on the\nGLUE benchmark, where our model outperforms significantly ELMo and is\ncompetitive with transformer approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04344v1",
            "date": "2020-10-09 03:17:51+00:00",
            "title": "Plug-and-Play Conversational Models",
            "authors": [
                "Andrea Madotto",
                "Etsuko Ishii",
                "Zhaojiang Lin",
                "Sumanth Dathathri",
                "Pascale Fung"
            ],
            "abstract": "There has been considerable progress made towards conversational models that\ngenerate coherent and fluent responses; however, this often involves training\nlarge language models on large dialogue datasets, such as Reddit. These large\nconversational models provide little control over the generated responses, and\nthis control is further limited in the absence of annotated conversational\ndatasets for attribute specific generation that can be used for fine-tuning the\nmodel. In this paper, we first propose and evaluate plug-and-play methods for\ncontrollable response generation, which does not require dialogue specific\ndatasets and does not rely on fine-tuning a large model. While effective, the\ndecoding procedure induces considerable computational overhead, rendering the\nconversational model unsuitable for interactive usage. To overcome this, we\nintroduce an approach that does not require further computation at decoding\ntime, while also does not require any fine-tuning of a large language model. We\ndemonstrate, through extensive automatic and human evaluation, a high degree of\ncontrol over the generated conversational responses with regard to multiple\ndesired attributes, while being fluent.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04361v3",
            "date": "2020-10-09 04:05:49+00:00",
            "title": "Event Representation with Sequential, Semi-Supervised Discrete Variables",
            "authors": [
                "Mehdi Rezaee",
                "Francis Ferraro"
            ],
            "abstract": "Within the context of event modeling and understanding, we propose a new\nmethod for neural sequence modeling that takes partially-observed sequences of\ndiscrete, external knowledge into account. We construct a sequential neural\nvariational autoencoder, which uses Gumbel-Softmax reparametrization within a\ncarefully defined encoder, to allow for successful backpropagation during\ntraining. The core idea is to allow semi-supervised external discrete knowledge\nto guide, but not restrict, the variational latent parameters during training.\nOur experiments indicate that our approach not only outperforms multiple\nbaselines and the state-of-the-art in narrative script induction, but also\nconverges more quickly.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.04372v1",
            "date": "2020-10-09 04:54:54+00:00",
            "title": "Pragmatically Informative Color Generation by Grounding Contextual Modifiers",
            "authors": [
                "Zhengxuan Wu",
                "Desmond C. Ong"
            ],
            "abstract": "Grounding language in contextual information is crucial for fine-grained\nnatural language understanding. One important task that involves grounding\ncontextual modifiers is color generation. Given a reference color \"green\", and\na modifier \"bluey\", how does one generate a color that could represent \"bluey\ngreen\"? We propose a computational pragmatics model that formulates this color\ngeneration task as a recursive game between speakers and listeners. In our\nmodel, a pragmatic speaker reasons about the inferences that a listener would\nmake, and thus generates a modified color that is maximally informative to help\nthe listener recover the original referents. In this paper, we show that\nincorporating pragmatic information provides significant improvements in\nperformance compared with other state-of-the-art deep learning models where\npragmatic inference and flexibility in representing colors from a large\ncontinuous space are lacking. Our model has an absolute 98% increase in\nperformance for the test cases where the reference colors are unseen during\ntraining, and an absolute 40% increase in performance for the test cases where\nboth the reference colors and the modifiers are unseen during training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04505v2",
            "date": "2020-10-09 11:33:16+00:00",
            "title": "Self-Paced Learning for Neural Machine Translation",
            "authors": [
                "Yu Wan",
                "Baosong Yang",
                "Derek F. Wong",
                "Yikai Zhou",
                "Lidia S. Chao",
                "Haibo Zhang",
                "Boxing Chen"
            ],
            "abstract": "Recent studies have proven that the training of neural machine translation\n(NMT) can be facilitated by mimicking the learning process of humans.\nNevertheless, achievements of such kind of curriculum learning rely on the\nquality of artificial schedule drawn up with the handcrafted features, e.g.\nsentence length or word rarity. We ameliorate this procedure with a more\nflexible manner by proposing self-paced learning, where NMT model is allowed to\n1) automatically quantify the learning confidence over training examples; and\n2) flexibly govern its learning via regulating the loss in each iteration step.\nExperimental results over multiple translation tasks demonstrate that the\nproposed model yields better performance than strong baselines and those models\ntrained with human-designed curricula on both translation quality and\nconvergence speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04532v1",
            "date": "2020-10-09 12:45:27+00:00",
            "title": "Measuring What Counts: The case of Rumour Stance Classification",
            "authors": [
                "Carolina Scarton",
                "Diego F. Silva",
                "Kalina Bontcheva"
            ],
            "abstract": "Stance classification can be a powerful tool for understanding whether and\nwhich users believe in online rumours. The task aims to automatically predict\nthe stance of replies towards a given rumour, namely support, deny, question,\nor comment. Numerous methods have been proposed and their performance compared\nin the RumourEval shared tasks in 2017 and 2019. Results demonstrated that this\nis a challenging problem since naturally occurring rumour stance data is highly\nimbalanced. This paper specifically questions the evaluation metrics used in\nthese shared tasks. We re-evaluate the systems submitted to the two RumourEval\ntasks and show that the two widely adopted metrics -- accuracy and macro-F1 --\nare not robust for the four-class imbalanced task of rumour stance\nclassification, as they wrongly favour systems with highly skewed accuracy\ntowards the majority class. To overcome this problem, we propose new evaluation\nmetrics for rumour stance detection. These are not only robust to imbalanced\ndata but also score higher systems that are capable of recognising the two most\ninformative minority classes (support and deny).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04650v1",
            "date": "2020-10-06 13:00:32+00:00",
            "title": "LSTMs Compose (and Learn) Bottom-Up",
            "authors": [
                "Naomi Saphra",
                "Adam Lopez"
            ],
            "abstract": "Recent work in NLP shows that LSTM language models capture hierarchical\nstructure in language data. In contrast to existing work, we consider the\n\\textit{learning} process that leads to their compositional behavior. For a\ncloser look at how an LSTM's sequential representations are composed\nhierarchically, we present a related measure of Decompositional Interdependence\n(DI) between word meanings in an LSTM, based on their gate interactions. We\nconnect this measure to syntax with experiments on English language data, where\nDI is higher on pairs of words with lower syntactic distance. To explore the\ninductive biases that cause these compositional representations to arise during\ntraining, we conduct simple experiments on synthetic data. These synthetic\nexperiments support a specific hypothesis about how hierarchical structures are\ndiscovered over the course of training: that LSTM constituent representations\nare learned bottom-up, relying on effective representations of their shorter\nchildren, rather than learning the longer-range relations independently from\nchildren.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04665v1",
            "date": "2020-10-09 16:19:42+00:00",
            "title": "Scaling Systematic Literature Reviews with Machine Learning Pipelines",
            "authors": [
                "Seraphina Goldfarb-Tarrant",
                "Alexander Robertson",
                "Jasmina Lazic",
                "Theodora Tsouloufi",
                "Louise Donnison",
                "Karen Smyth"
            ],
            "abstract": "Systematic reviews, which entail the extraction of data from large numbers of\nscientific documents, are an ideal avenue for the application of machine\nlearning. They are vital to many fields of science and philanthropy, but are\nvery time-consuming and require experts. Yet the three main stages of a\nsystematic review are easily done automatically: searching for documents can be\ndone via APIs and scrapers, selection of relevant documents can be done via\nbinary classification, and extraction of data can be done via\nsequence-labelling classification. Despite the promise of automation for this\nfield, little research exists that examines the various ways to automate each\nof these tasks. We construct a pipeline that automates each of these aspects,\nand experiment with many human-time vs. system quality trade-offs. We test the\nability of classifiers to work well on small amounts of data and to generalise\nto data from countries not represented in the training data. We test different\ntypes of data extraction with varying difficulty in annotation, and five\ndifferent neural architectures to do the extraction. We find that we can get\nsurprising accuracy and generalisability of the whole pipeline system with only\n2 weeks of human-expert annotation, which is only 15% of the time it takes to\ndo the whole review manually and can be repeated and extended to new data with\nno additional effort.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04704v1",
            "date": "2020-10-09 17:47:16+00:00",
            "title": "Recursive Top-Down Production for Sentence Generation with Latent Trees",
            "authors": [
                "Shawn Tan",
                "Yikang Shen",
                "Timothy J. O'Donnell",
                "Alessandro Sordoni",
                "Aaron Courville"
            ],
            "abstract": "We model the recursive production property of context-free grammars for\nnatural and synthetic languages. To this end, we present a dynamic programming\nalgorithm that marginalises over latent binary tree structures with $N$ leaves,\nallowing us to compute the likelihood of a sequence of $N$ tokens under a\nlatent tree model, which we maximise to train a recursive neural function. We\ndemonstrate performance on two synthetic tasks: SCAN (Lake and Baroni, 2017),\nwhere it outperforms previous models on the LENGTH split, and English question\nformation (McCoy et al., 2020), where it performs comparably to decoders with\nthe ground-truth tree structure. We also present experimental results on\nGerman-English translation on the Multi30k dataset (Elliott et al., 2016), and\nqualitatively analyse the induced tree structures our model learns for the SCAN\ntasks and the German-English translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04791v1",
            "date": "2020-10-09 20:28:06+00:00",
            "title": "ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization",
            "authors": [
                "Shiyue Zhang",
                "Benjamin Frey",
                "Mohit Bansal"
            ],
            "abstract": "Cherokee is a highly endangered Native American language spoken by the\nCherokee people. The Cherokee culture is deeply embedded in its language.\nHowever, there are approximately only 2,000 fluent first language Cherokee\nspeakers remaining in the world, and the number is declining every year. To\nhelp save this endangered language, we introduce ChrEn, a Cherokee-English\nparallel dataset, to facilitate machine translation research between Cherokee\nand English. Compared to some popular machine translation language pairs, ChrEn\nis extremely low-resource, only containing 14k sentence pairs in total. We\nsplit our parallel data in ways that facilitate both in-domain and\nout-of-domain evaluation. We also collect 5k Cherokee monolingual data to\nenable semi-supervised learning. Besides these datasets, we propose several\nCherokee-English and English-Cherokee machine translation systems. We compare\nSMT (phrase-based) versus NMT (RNN-based and Transformer-based) systems;\nsupervised versus semi-supervised (via language model, back-translation, and\nBERT/Multilingual-BERT) methods; as well as transfer learning versus\nmultilingual joint training with 4 other languages. Our best results are\n15.8/12.7 BLEU for in-domain and 6.5/5.0 BLEU for out-of-domain Chr-En/EnChr\ntranslations, respectively, and we hope that our dataset and systems will\nencourage future work by the community for Cherokee language revitalization.\nOur data, code, and demo will be publicly available at\nhttps://github.com/ZhangShiyue/ChrEn",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04826v1",
            "date": "2020-10-09 22:10:03+00:00",
            "title": "On Task-Level Dialogue Composition of Generative Transformer Model",
            "authors": [
                "Prasanna Parthasarathi",
                "Arvind Neelakantan",
                "Sharan Narang"
            ],
            "abstract": "Task-oriented dialogue systems help users accomplish tasks such as booking a\nmovie ticket and ordering food via conversation. Generative models\nparameterized by a deep neural network are widely used for next turn response\ngeneration in such systems. It is natural for users of the system to want to\naccomplish multiple tasks within the same conversation, but the ability of\ngenerative models to compose multiple tasks is not well studied. In this work,\nwe begin by studying the effect of training human-human task-oriented dialogues\ntowards improving the ability to compose multiple tasks on Transformer\ngenerative models. To that end, we propose and explore two solutions: (1)\ncreating synthetic multiple task dialogue data for training from human-human\nsingle task dialogue and (2) forcing the encoder representation to be invariant\nto single and multiple task dialogues using an auxiliary loss. The results from\nour experiments highlight the difficulty of even the sophisticated variant of\ntransformer model in learning to compose multiple tasks from single task\ndialogues.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04842v1",
            "date": "2020-10-09 23:06:57+00:00",
            "title": "Conformal retrofitting via Riemannian manifolds: distilling task-specific graphs into pretrained embeddings",
            "authors": [
                "Justin Dieter",
                "Arun Tejasvi Chaganty"
            ],
            "abstract": "Pretrained (language) embeddings are versatile, task-agnostic feature\nrepresentations of entities, like words, that are central to many machine\nlearning applications. These representations can be enriched through\nretrofitting, a class of methods that incorporate task-specific domain\nknowledge encoded as a graph over a subset of these entities. However, existing\nretrofitting algorithms face two limitations: they overfit the observed graph\nby failing to represent relationships with missing entities; and they underfit\nthe observed graph by only learning embeddings in Euclidean manifolds, which\ncannot faithfully represent even simple tree-structured or cyclic graphs. We\naddress these problems with two key contributions: (i) we propose a novel\nregularizer, a conformality regularizer, that preserves local geometry from the\npretrained embeddings---enabling generalization to missing entities and (ii) a\nnew Riemannian feedforward layer that learns to map pre-trained embeddings onto\na non-Euclidean manifold that can better represent the entire graph. Through\nexperiments on WordNet, we demonstrate that the conformality regularizer\nprevents even existing (Euclidean-only) methods from overfitting on link\nprediction for missing entities, and---together with the Riemannian feedforward\nlayer---learns non-Euclidean embeddings that outperform them.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.04863v1",
            "date": "2020-10-10 01:30:30+00:00",
            "title": "RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion",
            "authors": [
                "Hao Huang",
                "Guodong Long",
                "Tao Shen",
                "Jing Jiang",
                "Chengqi Zhang"
            ],
            "abstract": "Many graph embedding approaches have been proposed for knowledge graph\ncompletion via link prediction. Among those, translating embedding approaches\nenjoy the advantages of light-weight structure, high efficiency and great\ninterpretability. Especially when extended to complex vector space, they show\nthe capability in handling various relation patterns including symmetry,\nantisymmetry, inversion and composition. However, previous translating\nembedding approaches defined in complex vector space suffer from two main\nissues: 1) representing and modeling capacities of the model are limited by the\ntranslation function with rigorous multiplication of two complex numbers; and\n2) embedding ambiguity caused by one-to-many relations is not explicitly\nalleviated. In this paper, we propose a relation-adaptive translation function\nbuilt upon a novel weighted product in complex space, where the weights are\nlearnable, relation-specific and independent to embedding size. The translation\nfunction only requires eight more scalar parameters each relation, but improves\nexpressive power and alleviates embedding ambiguity problem. Based on the\nfunction, we then present our Relation-adaptive translating Embedding (RatE)\napproach to score each graph triple. Moreover, a novel negative sampling method\nis proposed to utilize both prior knowledge and self-adversarial learning for\neffective optimization. Experiments verify RatE achieves state-of-the-art\nperformance on four link prediction benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04883v1",
            "date": "2020-10-10 02:46:06+00:00",
            "title": "Adversarial Self-Supervised Data-Free Distillation for Text Classification",
            "authors": [
                "Xinyin Ma",
                "Yongliang Shen",
                "Gongfan Fang",
                "Chen Chen",
                "Chenghao Jia",
                "Weiming Lu"
            ],
            "abstract": "Large pre-trained transformer-based language models have achieved impressive\nresults on a wide range of NLP tasks. In the past few years, Knowledge\nDistillation(KD) has become a popular paradigm to compress a computationally\nexpensive model to a resource-efficient lightweight model. However, most KD\nalgorithms, especially in NLP, rely on the accessibility of the original\ntraining dataset, which may be unavailable due to privacy issues. To tackle\nthis problem, we propose a novel two-stage data-free distillation method, named\nAdversarial self-Supervised Data-Free Distillation (AS-DFD), which is designed\nfor compressing large-scale transformer-based models (e.g., BERT). To avoid\ntext generation in discrete space, we introduce a Plug & Play Embedding\nGuessing method to craft pseudo embeddings from the teacher's hidden knowledge.\nMeanwhile, with a self-supervised module to quantify the student's ability, we\nadapt the difficulty of pseudo embeddings in an adversarial training manner. To\nthe best of our knowledge, our framework is the first data-free distillation\nframework designed for NLP tasks. We verify the effectiveness of our method on\nseveral text classification datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04898v3",
            "date": "2020-10-10 04:28:42+00:00",
            "title": "Open-Domain Question Answering Goes Conversational via Question Rewriting",
            "authors": [
                "Raviteja Anantha",
                "Svitlana Vakulenko",
                "Zhucheng Tu",
                "Shayne Longpre",
                "Stephen Pulman",
                "Srinivas Chappidi"
            ],
            "abstract": "We introduce a new dataset for Question Rewriting in Conversational Context\n(QReCC), which contains 14K conversations with 80K question-answer pairs. The\ntask in QReCC is to find answers to conversational questions within a\ncollection of 10M web pages (split into 54M passages). Answers to questions in\nthe same conversation may be distributed across several web pages. QReCC\nprovides annotations that allow us to train and evaluate individual subtasks of\nquestion rewriting, passage retrieval and reading comprehension required for\nthe end-to-end conversational question answering (QA) task. We report the\neffectiveness of a strong baseline approach that combines the state-of-the-art\nmodel for question rewriting, and competitive models for open-domain QA. Our\nresults set the first baseline for the QReCC dataset with F1 of 19.10, compared\nto the human upper bound of 75.45, indicating the difficulty of the setup and a\nlarge room for improvement.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.04900v2",
            "date": "2020-10-10 04:35:16+00:00",
            "title": "Toward Micro-Dialect Identification in Diaglossic and Code-Switched Environments",
            "authors": [
                "Muhammad Abdul-Mageed",
                "Chiyu Zhang",
                "AbdelRahim Elmadany",
                "Lyle Ungar"
            ],
            "abstract": "Although the prediction of dialects is an important language processing task,\nwith a wide range of applications, existing work is largely limited to\ncoarse-grained varieties. Inspired by geolocation research, we propose the\nnovel task of Micro-Dialect Identification (MDI) and introduce MARBERT, a new\nlanguage model with striking abilities to predict a fine-grained variety (as\nsmall as that of a city) given a single, short message. For modeling, we offer\na range of novel spatially and linguistically-motivated multi-task learning\nmodels. To showcase the utility of our models, we introduce a new, large-scale\ndataset of Arabic micro-varieties (low-resource) suited to our tasks. MARBERT\npredicts micro-dialects with 9.9% F1, ~76X better than a majority class\nbaseline. Our new language model also establishes new state-of-the-art on\nseveral external tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04903v1",
            "date": "2020-10-10 05:03:14+00:00",
            "title": "What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding",
            "authors": [
                "Yu-An Wang",
                "Yun-Nung Chen"
            ],
            "abstract": "In recent years, pre-trained Transformers have dominated the majority of NLP\nbenchmark tasks. Many variants of pre-trained Transformers have kept breaking\nout, and most focus on designing different pre-training objectives or variants\nof self-attention. Embedding the position information in the self-attention\nmechanism is also an indispensable factor in Transformers however is often\ndiscussed at will. Therefore, this paper carries out an empirical study on\nposition embeddings of mainstream pre-trained Transformers, which mainly\nfocuses on two questions: 1) Do position embeddings really learn the meaning of\npositions? 2) How do these different learned position embeddings affect\nTransformers for NLP tasks? This paper focuses on providing a new insight of\npre-trained position embeddings through feature-level analysis and empirical\nexperiments on most of iconic NLP tasks. It is believed that our experimental\nresults can guide the future work to choose the suitable positional encoding\nfunction for specific tasks given the application property.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04922v1",
            "date": "2020-10-10 06:49:25+00:00",
            "title": "Structured Self-Attention Weights Encode Semantics in Sentiment Analysis",
            "authors": [
                "Zhengxuan Wu",
                "Thanh-Son Nguyen",
                "Desmond C. Ong"
            ],
            "abstract": "Neural attention, especially the self-attention made popular by the\nTransformer, has become the workhorse of state-of-the-art natural language\nprocessing (NLP) models. Very recent work suggests that the self-attention in\nthe Transformer encodes syntactic information; Here, we show that\nself-attention scores encode semantics by considering sentiment analysis tasks.\nIn contrast to gradient-based feature attribution methods, we propose a simple\nand effective Layer-wise Attention Tracing (LAT) method to analyze structured\nattention weights. We apply our method to Transformer models trained on two\ntasks that have surface dissimilarities, but share common semantics---sentiment\nanalysis of movie reviews and time-series valence prediction in life story\nnarratives. Across both tasks, words with high aggregated attention weights\nwere rich in emotional semantics, as quantitatively validated by an emotion\nlexicon labeled by human annotators. Our results show that structured attention\nweights encode rich semantics in sentiment analysis, and match human\ninterpretations of semantics.",
            "journal ref": "BlackBoxNLP Workshop at EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04927v1",
            "date": "2020-10-10 07:13:32+00:00",
            "title": "Cue-word Driven Neural Response Generation with a Shrinking Vocabulary",
            "authors": [
                "Qiansheng Wang",
                "Yuxin Liu",
                "Chengguo Lv",
                "Zhen Wang",
                "Guohong Fu"
            ],
            "abstract": "Open-domain response generation is the task of generating sensible and\ninformative re-sponses to the source sentence. However, neural models tend to\ngenerate safe and mean-ingless responses. While cue-word introducing approaches\nencourage responses with concrete semantics and have shown tremendous\npotential, they still fail to explore di-verse responses during decoding. In\nthis paper, we propose a novel but natural approach that can produce multiple\ncue-words during decoding, and then uses the produced cue-words to drive\ndecoding and shrinks the decoding vocabulary. Thus the neural genera-tion model\ncan explore the full space of responses and discover informative ones with\nefficiency. Experimental results show that our approach significantly\noutperforms several strong baseline models with much lower decoding complexity.\nEspecially, our approach can converge to concrete semantics more efficiently\nduring decoding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04935v1",
            "date": "2020-10-10 08:02:15+00:00",
            "title": "HPCC-YNU at SemEval-2020 Task 9: A Bilingual Vector Gating Mechanism for Sentiment Analysis of Code-Mixed Text",
            "authors": [
                "Jun Kong",
                "Jin Wang",
                "Xuejie Zhang"
            ],
            "abstract": "It is fairly common to use code-mixing on a social media platform to express\nopinions and emotions in multilingual societies. The purpose of this task is to\ndetect the sentiment of code-mixed social media text. Code-mixed text poses a\ngreat challenge for the traditional NLP system, which currently uses\nmonolingual resources to deal with the problem of multilingual mixing. This\ntask has been solved in the past using lexicon lookup in respective sentiment\ndictionaries and using a long short-term memory (LSTM) neural network for\nmonolingual resources. In this paper, we (my codalab username is kongjun)\npresent a system that uses a bilingual vector gating mechanism for bilingual\nresources to complete the task. The model consists of two main parts: the\nvector gating mechanism, which combines the character and word levels, and the\nattention mechanism, which extracts the important emotional parts of the text.\nThe results show that the proposed system outperforms the baseline algorithm.\nWe achieved fifth place in Spanglish and 19th place in Hinglish.The code of\nthis paper is availabled at : https://github.com/JunKong5/Semveal2020-task9",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04971v1",
            "date": "2020-10-10 10:52:22+00:00",
            "title": "Tag Recommendation for Online Q&A Communities based on BERT Pre-Training Technique",
            "authors": [
                "Navid Khezrian",
                "Jafar Habibi",
                "Issa Annamoradnejad"
            ],
            "abstract": "Online Q&A and open source communities use tags and keywords to index,\ncategorize, and search for specific content. The most obvious advantage of tag\nrecommendation is the correct classification of information. In this study, we\nused the BERT pre-training technique in tag recommendation task for online Q&A\nand open-source communities for the first time. Our evaluation on freecode\ndatasets show that the proposed method, called TagBERT, is more accurate\ncompared to deep learning and other baseline methods. Moreover, our model\nachieved a high stability by solving the problem of previous researches, where\nincreasing the number of tag recommendations significantly reduced model\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04976v2",
            "date": "2020-10-10 11:30:58+00:00",
            "title": "Can RNNs trained on harder subject-verb agreement instances still perform well on easier ones?",
            "authors": [
                "Hritik Bansal",
                "Gantavya Bhatt",
                "Sumeet Agarwal"
            ],
            "abstract": "Previous work suggests that RNNs trained on natural language corpora can\ncapture number agreement well for simple sentences but perform less well when\nsentences contain agreement attractors: intervening nouns between the verb and\nthe main subject with grammatical number opposite to the latter. This suggests\nthese models may not learn the actual syntax of agreement, but rather infer\nshallower heuristics such as `agree with the recent noun'. In this work, we\ninvestigate RNN models with varying inductive biases trained on selectively\nchosen `hard' agreement instances, i.e., sentences with at least one agreement\nattractor. For these the verb number cannot be predicted using a simple linear\nheuristic, and hence they might help provide the model additional cues for\nhierarchical syntax. If RNNs can learn the underlying agreement rules when\ntrained on such hard instances, then they should generalize well to other\nsentences, including simpler ones. However, we observe that several RNN types,\nincluding the ONLSTM which has a soft structural inductive bias, surprisingly\nfail to perform well on sentences without attractors when trained solely on\nsentences with attractors. We analyze how these selectively trained RNNs\ncompare to the baseline (training on a natural distribution of agreement\nattractors) along the dimensions of number agreement accuracy, representational\nsimilarity, and performance across different syntactic constructions. Our\nfindings suggest that RNNs trained on our hard agreement instances still do not\ncapture the underlying syntax of agreement, but rather tend to overfit the\ntraining distribution in a way which leads them to perform poorly on `easy'\nout-of-distribution instances. Thus, while RNNs are powerful models which can\npick up non-trivial dependency patterns, inducing them to do so at the level of\nsyntax rather than surface remains a challenge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.04980v1",
            "date": "2020-10-10 12:25:18+00:00",
            "title": "An Empirical Investigation of Beam-Aware Training in Supertagging",
            "authors": [
                "Renato Negrinho",
                "Matthew R. Gormley",
                "Geoffrey J. Gordon"
            ],
            "abstract": "Structured prediction is often approached by training a locally normalized\nmodel with maximum likelihood and decoding approximately with beam search. This\napproach leads to mismatches as, during training, the model is not exposed to\nits mistakes and does not use beam search. Beam-aware training aims to address\nthese problems, but unfortunately, it is not yet widely used due to a lack of\nunderstanding about how it impacts performance, when it is most useful, and\nwhether it is stable. Recently, Negrinho et al. (2018) proposed a\nmeta-algorithm that captures beam-aware training algorithms and suggests new\nones, but unfortunately did not provide empirical results. In this paper, we\nbegin an empirical investigation: we train the supertagging model of Vaswani et\nal. (2016) and a simpler model with instantiations of the meta-algorithm. We\nexplore the influence of various design choices and make recommendations for\nchoosing them. We observe that beam-aware training improves performance for\nboth models, with large improvements for the simpler model which must\neffectively manage uncertainty during decoding. Our results suggest that a\nmodel must be learned with search to maximize its effectiveness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05001v1",
            "date": "2020-10-10 13:47:13+00:00",
            "title": "Beyond Language: Learning Commonsense from Images for Reasoning",
            "authors": [
                "Wanqing Cui",
                "Yanyan Lan",
                "Liang Pang",
                "Jiafeng Guo",
                "Xueqi Cheng"
            ],
            "abstract": "This paper proposes a novel approach to learn commonsense from images,\ninstead of limited raw texts or costly constructed knowledge bases, for the\ncommonsense reasoning problem in NLP. Our motivation comes from the fact that\nan image is worth a thousand words, where richer scene information could be\nleveraged to help distill the commonsense knowledge, which is often hidden in\nlanguages. Our approach, namely Loire, consists of two stages. In the first\nstage, a bi-modal sequence-to-sequence approach is utilized to conduct the\nscene layout generation task, based on a text representation model ViBERT. In\nthis way, the required visual scene knowledge, such as spatial relations, will\nbe encoded in ViBERT by the supervised learning process with some bi-modal data\nlike COCO. Then ViBERT is concatenated with a pre-trained language model to\nperform the downstream commonsense reasoning tasks. Experimental results on two\ncommonsense reasoning problems, i.e. commonsense question answering and pronoun\nresolution, demonstrate that Loire outperforms traditional language-based\nmethods. We also give some case studies to show what knowledge is learned from\nimages and explain how the generated scene layout helps the commonsense\nreasoning process.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05003v2",
            "date": "2020-10-10 13:49:46+00:00",
            "title": "Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training",
            "authors": [
                "Xinyu Wang",
                "Kewei Tu"
            ],
            "abstract": "In this paper, we propose second-order graph-based neural dependency parsing\nusing message passing and end-to-end neural networks. We empirically show that\nour approaches match the accuracy of very recent state-of-the-art second-order\ngraph-based neural dependency parsers and have significantly faster speed in\nboth training and testing. We also empirically show the advantage of\nsecond-order parsing over first-order parsing and observe that the usefulness\nof the head-selection structured constraint vanishes when using BERT embedding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05103v1",
            "date": "2020-10-10 21:56:27+00:00",
            "title": "On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks",
            "authors": [
                "Stephen Mussmann",
                "Robin Jia",
                "Percy Liang"
            ],
            "abstract": "Many pairwise classification tasks, such as paraphrase detection and\nopen-domain question answering, naturally have extreme label imbalance (e.g.,\n$99.99\\%$ of examples are negatives). In contrast, many recent datasets\nheuristically choose examples to ensure label balance. We show that these\nheuristics lead to trained models that generalize poorly: State-of-the art\nmodels trained on QQP and WikiQA each have only $2.4\\%$ average precision when\nevaluated on realistically imbalanced test data. We instead collect training\ndata with active learning, using a BERT-based embedding model to efficiently\nretrieve uncertain points from a very large pool of unlabeled utterance pairs.\nBy creating balanced training data with more informative negative examples,\nactive learning greatly improves average precision to $32.5\\%$ on QQP and\n$20.1\\%$ on WikiQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05106v1",
            "date": "2020-10-10 22:03:58+00:00",
            "title": "Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation",
            "authors": [
                "Mehrad Moradshahi",
                "Giovanni Campagna",
                "Sina J. Semnani",
                "Silei Xu",
                "Monica S. Lam"
            ],
            "abstract": "We propose Semantic Parser Localizer (SPL), a toolkit that leverages Neural\nMachine Translation (NMT) systems to localize a semantic parser for a new\nlanguage. Our methodology is to (1) generate training data automatically in the\ntarget language by augmenting machine-translated datasets with local entities\nscraped from public websites, (2) add a few-shot boost of human-translated\nsentences and train a novel XLMR-LSTM semantic parser, and (3) test the model\non natural utterances curated using human translators.\n  We assess the effectiveness of our approach by extending the current\ncapabilities of Schema2QA, a system for English Question Answering (QA) on the\nopen web, to 10 new languages for the restaurants and hotels domains. Our\nmodels achieve an overall test accuracy ranging between 61% and 69% for the\nhotels domain and between 64% and 78% for restaurants domain, which compares\nfavorably to 69% and 80% obtained for English parser trained on gold English\ndata and a few examples from validation set. We show our approach outperforms\nthe previous state-of-the-art methodology by more than 30% for hotels and 40%\nfor restaurants with localized ontologies for the subset of languages tested.\n  Our methodology enables any software developer to add a new language\ncapability to a QA system for a new domain, leveraging machine translation, in\nless than 24 hours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05171v1",
            "date": "2020-10-11 05:36:54+00:00",
            "title": "fairseq S2T: Fast Speech-to-Text Modeling with fairseq",
            "authors": [
                "Changhan Wang",
                "Yun Tang",
                "Xutai Ma",
                "Anne Wu",
                "Dmytro Okhonko",
                "Juan Pino"
            ],
            "abstract": "We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T)\nmodeling tasks such as end-to-end speech recognition and speech-to-text\ntranslation. It follows fairseq's careful design for scalability and\nextensibility. We provide end-to-end workflows from data pre-processing, model\ntraining to offline (online) inference. We implement state-of-the-art RNN-based\nas well as Transformer-based models and open-source detailed training recipes.\nFairseq's machine translation models and language models can be seamlessly\nintegrated into S2T workflows for multi-task learning or transfer learning.\nFairseq S2T documentation and examples are available at\nhttps://github.com/pytorch/fairseq/tree/master/examples/speech_to_text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05193v1",
            "date": "2020-10-11 08:39:02+00:00",
            "title": "Lexically Cohesive Neural Machine Translation with Copy Mechanism",
            "authors": [
                "Vipul Mishra",
                "Chenhui Chu",
                "Yuki Arase"
            ],
            "abstract": "Lexically cohesive translations preserve consistency in word choices in\ndocument-level translation. We employ a copy mechanism into a context-aware\nneural machine translation model to allow copying words from previous\ntranslation outputs. Different from previous context-aware neural machine\ntranslation models that handle all the discourse phenomena implicitly, our\nmodel explicitly addresses the lexical cohesion problem by boosting the\nprobabilities to output words consistently. We conduct experiments on Japanese\nto English translation using an evaluation dataset for discourse translation.\nThe results showed that the proposed model significantly improved lexical\ncohesion compared to previous context-aware models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05202v1",
            "date": "2020-10-11 09:28:52+00:00",
            "title": "Connecting the Dots Between Fact Verification and Fake News Detection",
            "authors": [
                "Qifei Li",
                "Wangchunshu Zhou"
            ],
            "abstract": "Fact verification models have enjoyed a fast advancement in the last two\nyears with the development of pre-trained language models like BERT and the\nrelease of large scale datasets such as FEVER. However, the challenging problem\nof fake news detection has not benefited from the improvement of fact\nverification models, which is closely related to fake news detection. In this\npaper, we propose a simple yet effective approach to connect the dots between\nfact verification and fake news detection. Our approach first employs a text\nsummarization model pre-trained on news corpora to summarize the long news\narticle into a short claim. Then we use a fact verification model pre-trained\non the FEVER dataset to detect whether the input news article is real or fake.\nOur approach makes use of the recent success of fact verification models and\nenables zero-shot fake news detection, alleviating the need of large-scale\ntraining data to train fake news detection models. Experimental results on\nFakenewsNet, a benchmark dataset for fake news detection, demonstrate the\neffectiveness of our proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05223v1",
            "date": "2020-10-11 11:21:53+00:00",
            "title": "End to End Binarized Neural Networks for Text Classification",
            "authors": [
                "Harshil Jain",
                "Akshat Agarwal",
                "Kumar Shridhar",
                "Denis Kleyko"
            ],
            "abstract": "Deep neural networks have demonstrated their superior performance in almost\nevery Natural Language Processing task, however, their increasing complexity\nraises concerns. In particular, these networks require high expenses on\ncomputational hardware, and training budget is a concern for many. Even for a\ntrained network, the inference phase can be too demanding for\nresource-constrained devices, thus limiting its applicability. The\nstate-of-the-art transformer models are a vivid example. Simplifying the\ncomputations performed by a network is one way of relaxing the complexity\nrequirements. In this paper, we propose an end to end binarized neural network\narchitecture for the intent classification task. In order to fully utilize the\npotential of end to end binarization, both input representations (vector\nembeddings of tokens statistics) and the classifier are binarized. We\ndemonstrate the efficiency of such architecture on the intent classification of\nshort texts over three datasets and for text classification with a larger\ndataset. The proposed architecture achieves comparable to the state-of-the-art\nresults on standard intent classification datasets while utilizing ~ 20-40%\nlesser memory and training time. Furthermore, the individual components of the\narchitecture, such as binarized vector embeddings of documents or binarized\nclassifiers, can be used separately with not necessarily fully binary\narchitectures.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.05243v3",
            "date": "2020-10-11 13:18:46+00:00",
            "title": "Data Agnostic RoBERTa-based Natural Language to SQL Query Generation",
            "authors": [
                "Debaditya Pal",
                "Harsh Sharma",
                "Kaustubh Chaudhari"
            ],
            "abstract": "Relational databases are among the most widely used architectures to store\nmassive amounts of data in the modern world. However, there is a barrier\nbetween these databases and the average user. The user often lacks the\nknowledge of a query language such as SQL required to interact with the\ndatabase. The NL2SQL task aims at finding deep learning approaches to solve\nthis problem by converting natural language questions into valid SQL queries.\nGiven the sensitive nature of some databases and the growing need for data\nprivacy, we have presented an approach with data privacy at its core. We have\npassed RoBERTa embeddings and data-agnostic knowledge vectors into LSTM based\nsubmodels to predict the final query. Although we have not achieved state of\nthe art results, we have eliminated the need for the table data, right from the\ntraining of the model, and have achieved a test set execution accuracy of\n76.7%. By eliminating the table data dependency while training we have created\na model capable of zero shot learning based on the natural language question\nand table schema alone.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.05256v1",
            "date": "2020-10-11 14:42:18+00:00",
            "title": "Few-shot Learning for Multi-label Intent Detection",
            "authors": [
                "Yutai Hou",
                "Yongkui Lai",
                "Yushan Wu",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "In this paper, we study the few-shot multi-label classification for user\nintent detection. For multi-label intent detection, state-of-the-art work\nestimates label-instance relevance scores and uses a threshold to select\nmultiple associated intent labels. To determine appropriate thresholds with\nonly a few examples, we first learn universal thresholding experience on\ndata-rich domains, and then adapt the thresholds to certain few-shot domains\nwith a calibration based on nonparametric learning. For better calculation of\nlabel-instance relevance score, we introduce label name embedding as anchor\npoints in representation space, which refines representations of different\nclasses to be well-separated from each other. Experiments on two datasets show\nthat the proposed model significantly outperforms strong baselines in both\none-shot and five-shot settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05265v2",
            "date": "2020-10-11 15:13:18+00:00",
            "title": "Unsupervised Distillation of Syntactic Information from Contextualized Word Representations",
            "authors": [
                "Shauli Ravfogel",
                "Yanai Elazar",
                "Jacob Goldberger",
                "Yoav Goldberg"
            ],
            "abstract": "Contextualized word representations, such as ELMo and BERT, were shown to\nperform well on various semantic and syntactic tasks. In this work, we tackle\nthe task of unsupervised disentanglement between semantics and structure in\nneural language representations: we aim to learn a transformation of the\ncontextualized vectors, that discards the lexical semantics, but keeps the\nstructural information. To this end, we automatically generate groups of\nsentences which are structurally similar but semantically different, and use\nmetric-learning approach to learn a transformation that emphasizes the\nstructural component that is encoded in the vectors. We demonstrate that our\ntransformation clusters vectors in space by structural properties, rather than\nby lexical semantics. Finally, we demonstrate the utility of our distilled\nrepresentations by showing that they outperform the original contextualized\nrepresentations in a few-shot parsing setting.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05324v1",
            "date": "2020-10-11 19:17:24+00:00",
            "title": "Multilingual Offensive Language Identification with Cross-lingual Embeddings",
            "authors": [
                "Tharindu Ranasinghe",
                "Marcos Zampieri"
            ],
            "abstract": "Offensive content is pervasive in social media and a reason for concern to\ncompanies and government organizations. Several studies have been recently\npublished investigating methods to detect the various forms of such content\n(e.g. hate speech, cyberbulling, and cyberaggression). The clear majority of\nthese studies deal with English partially because most annotated datasets\navailable contain English data. In this paper, we take advantage of English\ndata available by applying cross-lingual contextual word embeddings and\ntransfer learning to make predictions in languages with less resources. We\nproject predictions on comparable data in Bengali, Hindi, and Spanish and we\nreport results of 0.8415 F1 macro for Bengali, 0.8568 F1 macro for Hindi, and\n0.7513 F1 macro for Spanish. Finally, we show that our approach compares\nfavorably to the best systems submitted to recent shared tasks on these three\nlanguages, confirming the robustness of cross-lingual contextual embeddings and\ntransfer learning for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05327v1",
            "date": "2020-10-11 19:31:18+00:00",
            "title": "InfoMiner at WNUT-2020 Task 2: Transformer-based Covid-19 Informative Tweet Extraction",
            "authors": [
                "Hansi Hettiarachchi",
                "Tharindu Ranasinghe"
            ],
            "abstract": "Identifying informative tweets is an important step when building information\nextraction systems based on social media. WNUT-2020 Task 2 was organised to\nrecognise informative tweets from noise tweets. In this paper, we present our\napproach to tackle the task objective using transformers. Overall, our approach\nachieves 10th place in the final rankings scoring 0.9004 F1 score for the test\nset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05349v2",
            "date": "2020-10-11 21:19:19+00:00",
            "title": "ComStreamClust: a communicative multi-agent approach to text clustering in streaming data",
            "authors": [
                "Ali Najafi",
                "Araz Gholipour-Shilabin",
                "Rahim Dehkharghani",
                "Ali Mohammadpur-Fard",
                "Meysam Asgari-Chenaghlu"
            ],
            "abstract": "Topic detection is the task of determining and tracking hot topics in social\nmedia. Twitter is arguably the most popular platform for people to share their\nideas with others about different issues. One such prevalent issue is the\nCOVID-19 pandemic. Detecting and tracking topics on these kinds of issues would\nhelp governments and healthcare companies deal with this phenomenon. In this\npaper, we propose a novel, multi-agent, communicative clustering approach,\nso-called ComStreamClust for clustering sub-topics inside a broader topic,\ne.g., COVID-19. The proposed approach is parallelizable, and can simultaneously\nhandle several data-point. The LaBSE sentence embedding is used to measure the\nsemantic similarity between two tweets. ComStreamClust has been evaluated on\ntwo datasets: the COVID-19 and the FA CUP. The results obtained from\nComStreamClust approve the effectiveness of the proposed approach when compared\nto existing methods.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.05357v1",
            "date": "2020-10-11 22:08:43+00:00",
            "title": "A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining",
            "authors": [
                "Jiahua Chen",
                "Shuai Wang",
                "Sahisnu Mazumder",
                "Bing Liu"
            ],
            "abstract": "Classifying and resolving coreferences of objects (e.g., product names) and\nattributes (e.g., product aspects) in opinionated reviews is crucial for\nimproving the opinion mining performance. However, the task is challenging as\none often needs to consider domain-specific knowledge (e.g., iPad is a tablet\nand has aspect resolution) to identify coreferences in opinionated reviews.\nAlso, compiling a handcrafted and curated domain-specific knowledge base for\neach domain is very time consuming and arduous. This paper proposes an approach\nto automatically mine and leverage domain-specific knowledge for classifying\nobjects and attribute coreferences. The approach extracts domain-specific\nknowledge from unlabeled review data and trains a knowledgeaware neural\ncoreference classification model to leverage (useful) domain knowledge together\nwith general commonsense knowledge for the task. Experimental evaluation on\nrealworld datasets involving five domains (product types) shows the\neffectiveness of the approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05384v1",
            "date": "2020-10-12 01:22:29+00:00",
            "title": "A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies",
            "authors": [
                "Ho-Lam Chung",
                "Ying-Hong Chan",
                "Yao-Chung Fan"
            ],
            "abstract": "In this paper, we investigate the following two limitations for the existing\ndistractor generation (DG) methods. First, the quality of the existing DG\nmethods are still far from practical use. There is still room for DG quality\nimprovement. Second, the existing DG designs are mainly for single distractor\ngeneration. However, for practical MCQ preparation, multiple distractors are\ndesired. Aiming at these goals, in this paper, we present a new distractor\ngeneration scheme with multi-tasking and negative answer training strategies\nfor effectively generating \\textit{multiple} distractors. The experimental\nresults show that (1) our model advances the state-of-the-art result from 28.65\nto 39.81 (BLEU 1 score) and (2) the generated multiple distractors are diverse\nand show strong distracting power for multiple choice question.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05406v1",
            "date": "2020-10-12 02:19:16+00:00",
            "title": "VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles",
            "authors": [
                "Mingzhe Li",
                "Xiuying Chen",
                "Shen Gao",
                "Zhangming Chan",
                "Dongyan Zhao",
                "Rui Yan"
            ],
            "abstract": "A popular multimedia news format nowadays is providing users with a lively\nvideo and a corresponding news article, which is employed by influential news\nmedia including CNN, BBC, and social media including Twitter and Weibo. In such\na case, automatically choosing a proper cover frame of the video and generating\nan appropriate textual summary of the article can help editors save time, and\nreaders make the decision more effectively. Hence, in this paper, we propose\nthe task of Video-based Multimodal Summarization with Multimodal Output (VMSMO)\nto tackle such a problem. The main challenge in this task is to jointly model\nthe temporal dependency of video with semantic meaning of article. To this end,\nwe propose a Dual-Interaction-based Multimodal Summarizer (DIMS), consisting of\na dual interaction module and multimodal generator. In the dual interaction\nmodule, we propose a conditional self-attention mechanism that captures local\nsemantic information within video and a global-attention mechanism that handles\nthe semantic relationship between news text and video from a high level.\nExtensive experiments conducted on a large-scale real-world VMSMO dataset show\nthat DIMS achieves the state-of-the-art performance in terms of both automatic\nmetrics and human evaluations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05419v1",
            "date": "2020-10-12 02:54:22+00:00",
            "title": "Gradient-based Analysis of NLP Models is Manipulable",
            "authors": [
                "Junlin Wang",
                "Jens Tuyls",
                "Eric Wallace",
                "Sameer Singh"
            ],
            "abstract": "Gradient-based analysis methods, such as saliency map visualizations and\nadversarial input perturbations, have found widespread use in interpreting\nneural NLP models due to their simplicity, flexibility, and most importantly,\ntheir faithfulness. In this paper, however, we demonstrate that the gradients\nof a model are easily manipulable, and thus bring into question the reliability\nof gradient-based analyses. In particular, we merge the layers of a target\nmodel with a Facade that overwhelms the gradients without affecting the\npredictions. This Facade can be trained to have gradients that are misleading\nand irrelevant to the task, such as focusing only on the stop words in the\ninput. On a variety of NLP tasks (text classification, NLI, and QA), we show\nthat our method can manipulate numerous gradient-based analysis techniques:\nsaliency maps, input reduction, and adversarial perturbations all identify\nunimportant or targeted tokens as being highly important. The code and a\ntutorial of this paper is available at http://ucinlp.github.io/facade.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05432v1",
            "date": "2020-10-12 03:34:44+00:00",
            "title": "It's not a Non-Issue: Negation as a Source of Error in Machine Translation",
            "authors": [
                "Md Mosharaf Hossain",
                "Antonios Anastasopoulos",
                "Eduardo Blanco",
                "Alexis Palmer"
            ],
            "abstract": "As machine translation (MT) systems progress at a rapid pace, questions of\ntheir adequacy linger. In this study we focus on negation, a universal, core\nproperty of human language that significantly affects the semantics of an\nutterance. We investigate whether translating negation is an issue for modern\nMT systems using 17 translation directions as test bed. Through thorough\nanalysis, we find that indeed the presence of negation can significantly impact\ndownstream quality, in some cases resulting in quality reductions of more than\n60%. We also provide a linguistically motivated analysis that directly explains\nthe majority of our findings. We release our annotations and code to replicate\nour analysis here: https://github.com/mosharafhossain/negation-mt.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05471v1",
            "date": "2020-10-12 06:12:18+00:00",
            "title": "Unseen Target Stance Detection with Adversarial Domain Generalization",
            "authors": [
                "Zhen Wang",
                "Qiansheng Wang",
                "Chengguo Lv",
                "Xue Cao",
                "Guohong Fu"
            ],
            "abstract": "Although stance detection has made great progress in the past few years, it\nis still facing the problem of unseen targets. In this study, we investigate\nthe domain difference between targets and thus incorporate attention-based\nconditional encoding with adversarial domain generalization to perform unseen\ntarget stance detection. Experimental results show that our approach achieves\nnew state-of-the-art performance on the SemEval-2016 dataset, demonstrating the\nimportance of domain difference between targets in unseen target stance\ndetection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05496v2",
            "date": "2020-10-12 07:43:01+00:00",
            "title": "Feature Extraction of Text for Deep Learning Algorithms: Application on Fake News Detection",
            "authors": [
                "HyeonJun Kim"
            ],
            "abstract": "Feature extraction is an important process of machine learning and deep\nlearning, as the process make algorithms function more efficiently, and also\naccurate. In natural language processing used in deception detection such as\nfake news detection, several ways of feature extraction in statistical aspect\nhad been introduced (e.g. N-gram). In this research, it will be shown that by\nusing deep learning algorithms and alphabet frequencies of the original text of\na news without any information about the sequence of the alphabet can actually\nbe used to classify fake news and trustworthy ones in high accuracy (85\\%). As\nthis pre-processing method makes the data notably compact but also include the\nfeature that is needed for the classifier, it seems that alphabet frequencies\ncontains some useful features for understanding complex context or meaning of\nthe original text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05639v1",
            "date": "2020-10-12 12:25:41+00:00",
            "title": "Predicting Clinical Trial Results by Implicit Evidence Integration",
            "authors": [
                "Qiao Jin",
                "Chuanqi Tan",
                "Mosha Chen",
                "Xiaozhong Liu",
                "Songfang Huang"
            ],
            "abstract": "Clinical trials provide essential guidance for practicing Evidence-Based\nMedicine, though often accompanying with unendurable costs and risks. To\noptimize the design of clinical trials, we introduce a novel Clinical Trial\nResult Prediction (CTRP) task. In the CTRP framework, a model takes a\nPICO-formatted clinical trial proposal with its background as input and\npredicts the result, i.e. how the Intervention group compares with the\nComparison group in terms of the measured Outcome in the studied Population.\nWhile structured clinical evidence is prohibitively expensive for manual\ncollection, we exploit large-scale unstructured sentences from medical\nliterature that implicitly contain PICOs and results as evidence. Specifically,\nwe pre-train a model to predict the disentangled results from such implicit\nevidence and fine-tune the model with limited data on the downstream datasets.\nExperiments on the benchmark Evidence Integration dataset show that the\nproposed model outperforms the baselines by large margins, e.g., with a 10.7%\nrelative gain over BioBERT in macro-F1. Moreover, the performance improvement\nis also validated on another dataset composed of clinical trials related to\nCOVID-19.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05710v1",
            "date": "2020-10-12 13:41:32+00:00",
            "title": "HUJI-KU at MRP~2020: Two Transition-based Neural Parsers",
            "authors": [
                "Ofir Arviv",
                "Ruixiang Cui",
                "Daniel Hershcovich"
            ],
            "abstract": "This paper describes the HUJI-KU system submission to the shared task on\nCross-Framework Meaning Representation Parsing (MRP) at the 2020 Conference for\nComputational Language Learning (CoNLL), employing TUPA and the HIT-SCIR\nparser, which were, respectively, the baseline system and winning system in the\n2019 MRP shared task. Both are transition-based parsers using BERT\ncontextualized embeddings. We generalized TUPA to support the newly-added MRP\nframeworks and languages, and experimented with multitask learning with the\nHIT-SCIR parser. We reached 4th place in both the cross-framework and\ncross-lingual tracks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05732v1",
            "date": "2020-10-12 14:26:48+00:00",
            "title": "On the Complementary Nature of Knowledge Graph Embedding, Fine Grain Entity Types, and Language Modeling",
            "authors": [
                "Rajat Patel",
                "Francis Ferraro"
            ],
            "abstract": "We demonstrate the complementary natures of neural knowledge graph embedding,\nfine-grain entity type prediction, and neural language modeling. We show that a\nlanguage model-inspired knowledge graph embedding approach yields both improved\nknowledge graph embeddings and fine-grain entity type representations. Our work\nalso shows that jointly modeling both structured knowledge tuples and language\nimproves both.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05757v1",
            "date": "2020-10-12 14:53:32+00:00",
            "title": "Extracting Angina Symptoms from Clinical Notes Using Pre-Trained Transformer Architectures",
            "authors": [
                "Aaron S. Eisman",
                "Nishant R. Shah",
                "Carsten Eickhoff",
                "George Zerveas",
                "Elizabeth S. Chen",
                "Wen-Chih Wu",
                "Indra Neil Sarkar"
            ],
            "abstract": "Anginal symptoms can connote increased cardiac risk and a need for change in\ncardiovascular management. This study evaluated the potential to extract these\nsymptoms from physician notes using the Bidirectional Encoder from Transformers\nlanguage model fine-tuned on a domain-specific corpus. The history of present\nillness section of 459 expert annotated primary care physician notes from\nconsecutive patients referred for cardiac testing without known atherosclerotic\ncardiovascular disease were included. Notes were annotated for positive and\nnegative mentions of chest pain and shortness of breath characterization. The\nresults demonstrate high sensitivity and specificity for the detection of chest\npain or discomfort, substernal chest pain, shortness of breath, and dyspnea on\nexertion. Small sample size limited extracting factors related to provocation\nand palliation of chest pain. This study provides a promising starting point\nfor the natural language processing of physician notes to characterize\nclinically actionable anginal symptoms.",
            "journal ref": "AMIA Annual Symposium 2020",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.05848v1",
            "date": "2020-10-12 16:53:00+00:00",
            "title": "Human-centric Dialog Training via Offline Reinforcement Learning",
            "authors": [
                "Natasha Jaques",
                "Judy Hanwen Shen",
                "Asma Ghandeharioun",
                "Craig Ferguson",
                "Agata Lapedriza",
                "Noah Jones",
                "Shixiang Shane Gu",
                "Rosalind Picard"
            ],
            "abstract": "How can we train a dialog model to produce better conversations by learning\nfrom human feedback, without the risk of humans teaching it harmful chat\nbehaviors? We start by hosting models online, and gather human feedback from\nreal-time, open-ended conversations, which we then use to train and improve the\nmodels using offline reinforcement learning (RL). We identify implicit\nconversational cues including language similarity, elicitation of laughter,\nsentiment, and more, which indicate positive human feedback, and embed these in\nmultiple reward functions. A well-known challenge is that learning an RL policy\nin an offline setting usually fails due to the lack of ability to explore and\nthe tendency to make over-optimistic estimates of future reward. These problems\nbecome even harder when using RL for language models, which can easily have a\n20,000 action vocabulary and many possible reward functions. We solve the\nchallenge by developing a novel class of offline RL algorithms. These\nalgorithms use KL-control to penalize divergence from a pre-trained prior\nlanguage model, and use a new strategy to make the algorithm pessimistic,\ninstead of optimistic, in the face of uncertainty. We test the resulting dialog\nmodel with ratings from 80 users in an open-domain setting and find it achieves\nsignificant improvements over existing deep offline RL approaches. The novel\noffline RL method is viable for improving any existing generative dialog model\nusing a static dataset of human feedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05874v1",
            "date": "2020-10-12 17:26:34+00:00",
            "title": "Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models",
            "authors": [
                "Zirui Wang",
                "Yulia Tsvetkov",
                "Orhan Firat",
                "Yuan Cao"
            ],
            "abstract": "Massively multilingual models subsuming tens or even hundreds of languages\npose great challenges to multi-task optimization. While it is a common practice\nto apply a language-agnostic procedure optimizing a joint multilingual task\nobjective, how to properly characterize and take advantage of its underlying\nproblem structure for improving optimization efficiency remains under-explored.\nIn this paper, we attempt to peek into the black-box of multilingual\noptimization through the lens of loss function geometry. We find that gradient\nsimilarity measured along the optimization trajectory is an important signal,\nwhich correlates well with not only language proximity but also the overall\nmodel performance. Such observation helps us to identify a critical limitation\nof existing gradient-based multi-task learning methods, and thus we derive a\nsimple and scalable optimization procedure, named Gradient Vaccine, which\nencourages more geometrically aligned parameter updates for close tasks.\nEmpirically, our method obtains significant model performance gains on\nmultilingual machine translation and XTREME benchmark tasks for multilingual\nlanguage models. Our work reveals the importance of properly measuring and\nutilizing language proximity in multilingual optimization, and has broader\nimplications for multi-task learning beyond multilingual modeling.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05961v1",
            "date": "2020-10-12 18:40:08+00:00",
            "title": "Perceptimatic: A human speech perception benchmark for unsupervised subword modelling",
            "authors": [
                "Juliette Millet",
                "Ewan Dunbar"
            ],
            "abstract": "In this paper, we present a data set and methods to compare speech processing\nmodels and human behaviour on a phone discrimination task. We provide\nPerceptimatic, an open data set which consists of French and English speech\nstimuli, as well as the results of 91 English- and 93 French-speaking\nlisteners. The stimuli test a wide range of French and English contrasts, and\nare extracted directly from corpora of natural running read speech, used for\nthe 2017 Zero Resource Speech Challenge. We provide a method to compare humans'\nperceptual space with models' representational space, and we apply it to models\npreviously submitted to the Challenge. We show that, unlike unsupervised models\nand supervised multilingual models, a standard supervised monolingual HMM-GMM\nphone recognition system, while good at discriminating phones, yields a\nrepresentational space very different from that of human native listeners.",
            "journal ref": "Proceedings of Interspeech 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05967v1",
            "date": "2020-10-12 18:56:48+00:00",
            "title": "The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units",
            "authors": [
                "Ewan Dunbar",
                "Julien Karadayi",
                "Mathieu Bernard",
                "Xuan-Nga Cao",
                "Robin Algayres",
                "Lucas Ondel",
                "Laurent Besacier",
                "Sakriani Sakti",
                "Emmanuel Dupoux"
            ],
            "abstract": "We present the Zero Resource Speech Challenge 2020, which aims at learning\nspeech representations from raw audio signals without any labels. It combines\nthe data sets and metrics from two previous benchmarks (2017 and 2019) and\nfeatures two tasks which tap into two levels of speech representation. The\nfirst task is to discover low bit-rate subword representations that optimize\nthe quality of speech synthesis; the second one is to discover word-like units\nfrom unsegmented raw speech. We present the results of the twenty submitted\nmodels and discuss the implications of the main findings for unsupervised\nspeech learning.",
            "journal ref": "Proceedings of Interspeech 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05985v1",
            "date": "2020-10-12 19:25:43+00:00",
            "title": "NEMO: Frequentist Inference Approach to Constrained Linguistic Typology Feature Prediction in SIGTYP 2020 Shared Task",
            "authors": [
                "Alexander Gutkin",
                "Richard Sproat"
            ],
            "abstract": "This paper describes the NEMO submission to SIGTYP 2020 shared task which\ndeals with prediction of linguistic typological features for multiple languages\nusing the data derived from World Atlas of Language Structures (WALS). We\nemploy frequentist inference to represent correlations between typological\nfeatures and use this representation to train simple multi-class estimators\nthat predict individual features. We describe two submitted ridge\nregression-based configurations which ranked second and third overall in the\nconstrained task. Our best configuration achieved the micro-averaged accuracy\nscore of 0.66 on 149 test languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05987v1",
            "date": "2020-10-12 19:28:29+00:00",
            "title": "SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search",
            "authors": [
                "Sean MacAvaney",
                "Arman Cohan",
                "Nazli Goharian"
            ],
            "abstract": "With worldwide concerns surrounding the Severe Acute Respiratory Syndrome\nCoronavirus 2 (SARS-CoV-2), there is a rapidly growing body of scientific\nliterature on the virus. Clinicians, researchers, and policy-makers need to be\nable to search these articles effectively. In this work, we present a zero-shot\nranking algorithm that adapts to COVID-related scientific literature. Our\napproach filters training data from another collection down to medical-related\nqueries, uses a neural re-ranking model pre-trained on scientific text\n(SciBERT), and filters the target document collection. This approach ranks top\namong zero-shot methods on the TREC COVID Round 1 leaderboard, and exhibits a\nP@5 of 0.80 and an nDCG@10 of 0.68 when evaluated on both Round 1 and 2\njudgments. Despite not relying on TREC-COVID data, our method outperforms\nmodels that do. As one of the first search methods to thoroughly evaluate\nCOVID-19 search, we hope that this serves as a strong baseline and helps in the\nglobal crisis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05990v2",
            "date": "2020-10-12 19:37:18+00:00",
            "title": "Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification",
            "authors": [
                "Jordan J. Bird",
                "Anik\u00f3 Ek\u00e1rt",
                "Diego R. Faria"
            ],
            "abstract": "In this work, we present the Chatbot Interaction with Artificial Intelligence\n(CI-AI) framework as an approach to the training of deep learning chatbots for\ntask classification. The intelligent system augments human-sourced data via\nartificial paraphrasing in order to generate a large set of training data for\nfurther classical, attention, and language transformation-based learning\napproaches for Natural Language Processing. Human beings are asked to\nparaphrase commands and questions for task identification for further execution\nof a machine. The commands and questions are split into training and validation\nsets. A total of 483 responses were recorded. Secondly, the training set is\nparaphrased by the T5 model in order to augment it with further data. Seven\nstate-of-the-art transformer-based text classification algorithms (BERT,\nDistilBERT, RoBERTa, DistilRoBERTa, XLM, XLM-RoBERTa, and XLNet) are\nbenchmarked for both sets after fine-tuning on the training data for two\nepochs. We find that all models are improved when training data is augmented by\nthe T5 model, with an average increase of classification accuracy by 4.01%. The\nbest result was the RoBERTa model trained on T5 augmented data which achieved\n98.96% classification accuracy. Finally, we found that an ensemble of the five\nbest-performing transformer models via Logistic Regression of output label\npredictions led to an accuracy of 99.59% on the dataset of human responses. A\nhighly-performing model allows the intelligent system to interpret human\ncommands at the social-interaction level through a chatbot-like interface (e.g.\n\"Robot, can we have a conversation?\") and allows for better accessibility to AI\nby non-technical users.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05993v1",
            "date": "2020-10-12 19:42:22+00:00",
            "title": "Vulgaris: Analysis of a Corpus for Middle-Age Varieties of Italian Language",
            "authors": [
                "Andrea Zugarini",
                "Matteo Tiezzi",
                "Marco Maggini"
            ],
            "abstract": "Italian is a Romance language that has its roots in Vulgar Latin. The birth\nof the modern Italian started in Tuscany around the 14th century, and it is\nmainly attributed to the works of Dante Alighieri, Francesco Petrarca and\nGiovanni Boccaccio, who are among the most acclaimed authors of the medieval\nage in Tuscany. However, Italy has been characterized by a high variety of\ndialects, which are often loosely related to each other, due to the past\nfragmentation of the territory. Italian has absorbed influences from many of\nthese dialects, as also from other languages due to dominion of portions of the\ncountry by other nations, such as Spain and France. In this work we present\nVulgaris, a project aimed at studying a corpus of Italian textual resources\nfrom authors of different regions, ranging in a time period between 1200 and\n1600. Each composition is associated to its author, and authors are also\ngrouped in families, i.e. sharing similar stylistic/chronological\ncharacteristics. Hence, the dataset is not only a valuable resource for\nstudying the diachronic evolution of Italian and the differences between its\ndialects, but it is also useful to investigate stylistic aspects between single\nauthors. We provide a detailed statistical analysis of the data, and a\ncorpus-driven study in dialectology and diachronic varieties.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.05994v1",
            "date": "2020-10-12 19:42:25+00:00",
            "title": "Improving Text Generation with Student-Forcing Optimal Transport",
            "authors": [
                "Guoyin Wang",
                "Chunyuan Li",
                "Jianqiao Li",
                "Hao Fu",
                "Yuh-Chen Lin",
                "Liqun Chen",
                "Yizhe Zhang",
                "Chenyang Tao",
                "Ruiyi Zhang",
                "Wenlin Wang",
                "Dinghan Shen",
                "Qian Yang",
                "Lawrence Carin"
            ],
            "abstract": "Neural language models are often trained with maximum likelihood estimation\n(MLE), where the next word is generated conditioned on the ground-truth word\ntokens. During testing, however, the model is instead conditioned on previously\ngenerated tokens, resulting in what is termed exposure bias. To reduce this gap\nbetween training and testing, we propose using optimal transport (OT) to match\nthe sequences generated in these two modes. An extension is further proposed to\nimprove the OT learning, based on the structural and contextual information of\nthe text sequences. The effectiveness of the proposed method is validated on\nmachine translation, text summarization, and text generation tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06000v1",
            "date": "2020-10-12 19:56:08+00:00",
            "title": "MedICaT: A Dataset of Medical Images, Captions, and Textual References",
            "authors": [
                "Sanjay Subramanian",
                "Lucy Lu Wang",
                "Sachin Mehta",
                "Ben Bogin",
                "Madeleine van Zuylen",
                "Sravanthi Parasa",
                "Sameer Singh",
                "Matt Gardner",
                "Hannaneh Hajishirzi"
            ],
            "abstract": "Understanding the relationship between figures and text is key to scientific\ndocument understanding. Medical figures in particular are quite complex, often\nconsisting of several subfigures (75% of figures in our dataset), with detailed\ntext describing their content. Previous work studying figures in scientific\npapers focused on classifying figure content rather than understanding how\nimages relate to the text. To address challenges in figure retrieval and\nfigure-to-text alignment, we introduce MedICaT, a dataset of medical images in\ncontext. MedICaT consists of 217K images from 131K open access biomedical\npapers, and includes captions, inline references for 74% of figures, and\nmanually annotated subfigures and subcaptions for a subset of figures. Using\nMedICaT, we introduce the task of subfigure to subcaption alignment in compound\nfigures and demonstrate the utility of inline references in image-text\nmatching. Our data and code can be accessed at\nhttps://github.com/allenai/medicat.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.06040v2",
            "date": "2020-10-12 21:28:14+00:00",
            "title": "Improving Self-supervised Pre-training via a Fully-Explored Masked Language Model",
            "authors": [
                "Mingzhi Zheng",
                "Dinghan Shen",
                "Yelong Shen",
                "Weizhu Chen",
                "Lin Xiao"
            ],
            "abstract": "Masked Language Model (MLM) framework has been widely adopted for\nself-supervised language pre-training. In this paper, we argue that randomly\nsampled masks in MLM would lead to undesirably large gradient variance. Thus,\nwe theoretically quantify the gradient variance via correlating the gradient\ncovariance with the Hamming distance between two different masks (given a\ncertain text sequence). To reduce the variance due to the sampling of masks, we\npropose a fully-explored masking strategy, where a text sequence is divided\ninto a certain number of non-overlapping segments. Thereafter, the tokens\nwithin one segment are masked for training. We prove, from a theoretical\nperspective, that the gradients derived from this new masking schema have a\nsmaller variance and can lead to more efficient self-supervised training. We\nconduct extensive experiments on both continual pre-training and general\npre-training from scratch. Empirical results confirm that this new masking\nstrategy can consistently outperform standard random masking. Detailed\nefficiency analysis and ablation studies further validate the advantages of our\nfully-explored masking strategy under the MLM framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06065v1",
            "date": "2020-10-12 22:59:18+00:00",
            "title": "Zero-shot Entity Linking with Efficient Long Range Sequence Modeling",
            "authors": [
                "Zonghai Yao",
                "Liangliang Cao",
                "Huapu Pan"
            ],
            "abstract": "This paper considers the problem of zero-shot entity linking, in which a link\nin the test time may not present in training. Following the prevailing\nBERT-based research efforts, we find a simple yet effective way is to expand\nthe long-range sequence modeling. Unlike many previous methods, our method does\nnot require expensive pre-training of BERT with long position embedding.\nInstead, we propose an efficient position embeddings initialization method\ncalled Embedding-repeat, which initializes larger position embeddings based on\nBERT-Base. On Wikia's zero-shot EL dataset, our method improves the SOTA from\n76.06% to 79.08%, and for its long data, the corresponding improvement is from\n74.57% to 82.14%. Our experiments suggest the effectiveness of long-range\nsequence modeling without retraining the BERT model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06119v3",
            "date": "2020-10-13 02:17:58+00:00",
            "title": "ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis",
            "authors": [
                "Qingyun Wang",
                "Qi Zeng",
                "Lifu Huang",
                "Kevin Knight",
                "Heng Ji",
                "Nazneen Fatema Rajani"
            ],
            "abstract": "To assist human review process, we build a novel ReviewRobot to automatically\nassign a review score and write comments for multiple categories such as\nnovelty and meaningful comparison. A good review needs to be knowledgeable,\nnamely that the comments should be constructive and informative to help improve\nthe paper; and explainable by providing detailed evidence. ReviewRobot achieves\nthese goals via three steps: (1) We perform domain-specific Information\nExtraction to construct a knowledge graph (KG) from the target paper under\nreview, a related work KG from the papers cited by the target paper, and a\nbackground KG from a large collection of previous papers in the domain. (2) By\ncomparing these three KGs, we predict a review score and detailed structured\nknowledge as evidence for each review category. (3) We carefully select and\ngeneralize human review sentences into templates, and apply these templates to\ntransform the review scores and evidence into natural language comments.\nExperimental results show that our review score predictor reaches 71.4%-100%\naccuracy. Human assessment by domain experts shows that 41.7%-70.5% of the\ncomments generated by ReviewRobot are valid and constructive, and better than\nhuman-written ones for 20% of the time. Thus, ReviewRobot can serve as an\nassistant for paper reviewers, program chairs and authors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06127v1",
            "date": "2020-10-13 02:36:48+00:00",
            "title": "Model Selection for Cross-Lingual Transfer using a Learned Scoring Function",
            "authors": [
                "Yang Chen",
                "Alan Ritter"
            ],
            "abstract": "Transformers that are pre-trained on multilingual text corpora, such as,\nmBERT and XLM-RoBERTa, have achieved impressive cross-lingual transfer learning\nresults. In the zero-shot cross-lingual transfer setting, only English training\ndata is assumed, and the fine-tuned model is evaluated on another target\nlanguage. No target-language validation data is assumed in this setting,\nhowever substantial variance has been observed in target language performance\nbetween different fine-tuning runs. Prior work has relied on English\nvalidation/development data to select among models that are fine-tuned with\ndifferent learning rates, number of steps and other hyperparameters, often\nresulting in suboptimal choices. To address this challenge, we propose a\nmeta-learning approach to model selection that uses the fine-tuned model's own\ninternal representations to predict its cross-lingual capabilities. In\nextensive experiments we find that our approach consistently selects better\nmodels than English validation data across five languages and five well-studied\nNLP tasks, achieving results that are comparable to small amounts of target\nlanguage development data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06133v1",
            "date": "2020-10-13 02:53:52+00:00",
            "title": "BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance",
            "authors": [
                "Jianquan Li",
                "Xiaokang Liu",
                "Honghong Zhao",
                "Ruifeng Xu",
                "Min Yang",
                "Yaohong Jin"
            ],
            "abstract": "Pre-trained language models (e.g., BERT) have achieved significant success in\nvarious natural language processing (NLP) tasks. However, high storage and\ncomputational costs obstruct pre-trained language models to be effectively\ndeployed on resource-constrained devices. In this paper, we propose a novel\nBERT distillation method based on many-to-many layer mapping, which allows each\nintermediate student layer to learn from any intermediate teacher layers. In\nthis way, our model can learn from different teacher layers adaptively for\nvarious NLP tasks. %motivated by the intuition that different NLP tasks require\ndifferent levels of linguistic knowledge contained in the intermediate layers\nof BERT. In addition, we leverage Earth Mover's Distance (EMD) to compute the\nminimum cumulative cost that must be paid to transform knowledge from teacher\nnetwork to student network. EMD enables the effective matching for many-to-many\nlayer mapping. %EMD can be applied to network layers with different sizes and\neffectively measures semantic distance between the teacher network and student\nnetwork. Furthermore, we propose a cost attention mechanism to learn the layer\nweights used in EMD automatically, which is supposed to further improve the\nmodel's performance and accelerate convergence time. Extensive experiments on\nGLUE benchmark demonstrate that our model achieves competitive performance\ncompared to strong competitors in terms of both accuracy and model compression.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06150v1",
            "date": "2020-10-13 03:46:25+00:00",
            "title": "Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance",
            "authors": [
                "Xi Chen",
                "Nan Ding",
                "Tomer Levinboim",
                "Radu Soricut"
            ],
            "abstract": "Recent advances in automatic evaluation metrics for text have shown that deep\ncontextualized word representations, such as those generated by BERT encoders,\nare helpful for designing metrics that correlate well with human judgements. At\nthe same time, it has been argued that contextualized word representations\nexhibit sub-optimal statistical properties for encoding the true similarity\nbetween words or sentences. In this paper, we present two techniques for\nimproving encoding representations for similarity metrics: a batch-mean\ncentering strategy that improves statistical properties; and a computationally\nefficient tempered Word Mover Distance, for better fusion of the information in\nthe contextualized word representations. We conduct numerical experiments that\ndemonstrate the robustness of our techniques, reporting results over various\nBERT-backbone learned metrics and achieving state of the art correlation with\nhuman ratings on several benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06196v2",
            "date": "2020-10-13 06:31:53+00:00",
            "title": "Mathematical Word Problem Generation from Commonsense Knowledge Graph and Equations",
            "authors": [
                "Tianqiao Liu",
                "Qian Fang",
                "Wenbiao Ding",
                "Zitao Liu"
            ],
            "abstract": "There is an increasing interest in the use of automatic mathematical word\nproblem (MWP) generation in educational assessment. Different from standard\nnatural question generation, MWP generation needs to maintain the underlying\nmathematical operations between quantities and variables, while at the same\ntime ensuring the relevance between the output and the given topic. To address\nabove problem, we develop an end-to-end neural model to generate diverse MWPs\nin real-world scenarios from commonsense knowledge graph and equations. The\nproposed model (1) learns both representations from edge-enhanced Levi graphs\nof symbolic equations and commonsense knowledge; (2) automatically fuses\nequation and commonsense knowledge information via a self-planning module when\ngenerating the MWPs. Experiments on an educational gold-standard set and a\nlarge-scale generated MWP set show that our approach is superior on the MWP\ngeneration task, and it outperforms the state-of-the-art models in terms of\nboth automatic evaluation metrics, i.e., BLEU-4, ROUGE-L, Self-BLEU, and human\nevaluation metrics, i.e., equation relevance, topic relevance, and language\ncoherence. To encourage reproducible results, we make our code and MWP dataset\npublic available at \\url{https://tinyurl.com/4ppldug7}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06253v1",
            "date": "2020-10-13 09:30:04+00:00",
            "title": "Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks",
            "authors": [
                "Peng Cui",
                "Le Hu",
                "Yuanchao Liu"
            ],
            "abstract": "Text summarization aims to compress a textual document to a short summary\nwhile keeping salient information. Extractive approaches are widely used in\ntext summarization because of their fluency and efficiency. However, most of\nexisting extractive models hardly capture inter-sentence relationships,\nparticularly in long documents. They also often ignore the effect of topical\ninformation on capturing important contents. To address these issues, this\npaper proposes a graph neural network (GNN)-based extractive summarization\nmodel, enabling to capture inter-sentence relationships efficiently via\ngraph-structured document representation. Moreover, our model integrates a\njoint neural topic model (NTM) to discover latent topics, which can provide\ndocument-level features for sentence selection. The experimental results\ndemonstrate that our model not only substantially achieves state-of-the-art\nresults on CNN/DM and NYT datasets but also considerably outperforms existing\napproaches on scientific paper datasets consisting of much longer documents,\nindicating its better robustness in document genres and lengths. Further\ndiscussions show that topical information can help the model preselect salient\ncontents from an entire document, which interprets its effectiveness in long\ndocument summarization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06269v2",
            "date": "2020-10-13 10:25:18+00:00",
            "title": "BRUMS at SemEval-2020 Task 3: Contextualised Embeddings for Predicting the (Graded) Effect of Context in Word Similarity",
            "authors": [
                "Hansi Hettiarachchi",
                "Tharindu Ranasinghe"
            ],
            "abstract": "This paper presents the team BRUMS submission to SemEval-2020 Task 3: Graded\nWord Similarity in Context. The system utilises state-of-the-art contextualised\nword embeddings, which have some task-specific adaptations, including stacked\nembeddings and average embeddings. Overall, the approach achieves good\nevaluation scores across all the languages, while maintaining simplicity.\nFollowing the final rankings, our approach is ranked within the top 5 solutions\nof each language while preserving the 1st position of Finnish subtask 2.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06278v1",
            "date": "2020-10-13 10:39:14+00:00",
            "title": "BRUMS at SemEval-2020 Task 12 : Transformer based Multilingual Offensive Language Identification in Social Media",
            "authors": [
                "Tharindu Ranasinghe",
                "Hansi Hettiarachchi"
            ],
            "abstract": "In this paper, we describe the team \\textit{BRUMS} entry to OffensEval 2:\nMultilingual Offensive Language Identification in Social Media in SemEval-2020.\nThe OffensEval organizers provided participants with annotated datasets\ncontaining posts from social media in Arabic, Danish, English, Greek and\nTurkish. We present a multilingual deep learning model to identify offensive\nlanguage in social media. Overall, the approach achieves acceptable evaluation\nscores, while maintaining flexibility between languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06325v2",
            "date": "2020-10-13 12:20:32+00:00",
            "title": "Modeling the Music Genre Perception across Language-Bound Cultures",
            "authors": [
                "Elena V. Epure",
                "Guillaume Salha",
                "Manuel Moussallam",
                "Romain Hennequin"
            ],
            "abstract": "The music genre perception expressed through human annotations of artists or\nalbums varies significantly across language-bound cultures. These variations\ncannot be modeled as mere translations since we also need to account for\ncultural differences in the music genre perception. In this work, we study the\nfeasibility of obtaining relevant cross-lingual, culture-specific music genre\nannotations based only on language-specific semantic representations, namely\ndistributed concept embeddings and ontologies. Our study, focused on six\nlanguages, shows that unsupervised cross-lingual music genre annotation is\nfeasible with high accuracy, especially when combining both types of\nrepresentations. This approach of studying music genres is the most extensive\nto date and has many implications in musicology and music information\nretrieval. Besides, we introduce a new, domain-dependent cross-lingual corpus\nto benchmark state of the art multilingual pre-trained embedding models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06395v1",
            "date": "2020-10-13 13:51:21+00:00",
            "title": "Aspect-based Document Similarity for Research Papers",
            "authors": [
                "Malte Ostendorff",
                "Terry Ruas",
                "Till Blume",
                "Bela Gipp",
                "Georg Rehm"
            ],
            "abstract": "Traditional document similarity measures provide a coarse-grained distinction\nbetween similar and dissimilar documents. Typically, they do not consider in\nwhat aspects two documents are similar. This limits the granularity of\napplications like recommender systems that rely on document similarity. In this\npaper, we extend similarity with aspect information by performing a pairwise\ndocument classification task. We evaluate our aspect-based document similarity\nfor research papers. Paper citations indicate the aspect-based similarity,\ni.e., the section title in which a citation occurs acts as a label for the pair\nof citing and cited paper. We apply a series of Transformer models such as\nRoBERTa, ELECTRA, XLNet, and BERT variations and compare them to an LSTM\nbaseline. We perform our experiments on two newly constructed datasets of\n172,073 research paper pairs from the ACL Anthology and CORD-19 corpus. Our\nresults show SciBERT as the best performing system. A qualitative examination\nvalidates our quantitative results. Our findings motivate future research of\naspect-based document similarity and the development of a recommender system\nbased on the evaluated techniques. We make our datasets, code, and trained\nmodels publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06467v1",
            "date": "2020-10-13 15:20:32+00:00",
            "title": "Pretrained Transformers for Text Ranking: BERT and Beyond",
            "authors": [
                "Jimmy Lin",
                "Rodrigo Nogueira",
                "Andrew Yates"
            ],
            "abstract": "The goal of text ranking is to generate an ordered list of texts retrieved\nfrom a corpus in response to a query. Although the most common formulation of\ntext ranking is search, instances of the task can also be found in many natural\nlanguage processing applications. This survey provides an overview of text\nranking with neural network architectures known as transformers, of which BERT\nis the best-known example. The combination of transformers and self-supervised\npretraining has, without exaggeration, revolutionized the fields of natural\nlanguage processing (NLP), information retrieval (IR), and beyond. In this\nsurvey, we provide a synthesis of existing work as a single point of entry for\npractitioners who wish to gain a better understanding of how to apply\ntransformers to text ranking problems and researchers who wish to pursue work\nin this area. We cover a wide range of modern techniques, grouped into two\nhigh-level categories: transformer models that perform reranking in multi-stage\nranking architectures and learned dense representations that attempt to perform\nranking directly. There are two themes that pervade our survey: techniques for\nhandling long documents, beyond the typical sentence-by-sentence processing\napproaches used in NLP, and techniques for addressing the tradeoff between\neffectiveness (result quality) and efficiency (query latency). Although\ntransformer architectures and pretraining techniques are recent innovations,\nmany aspects of how they are applied to text ranking are relatively well\nunderstood and represent mature techniques. However, there remain many open\nresearch questions, and thus in addition to laying out the foundations of\npretrained transformers for text ranking, this survey also attempts to\nprognosticate where the field is heading.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.06521v1",
            "date": "2020-10-13 16:26:57+00:00",
            "title": "Autotuning Search Space for Loop Transformations",
            "authors": [
                "Michael Kruse",
                "Hal Finkel",
                "Xingfu Wu"
            ],
            "abstract": "One of the challenges for optimizing compilers is to predict whether applying\nan optimization will improve its execution speed. Programmers may override the\ncompiler's profitability heuristic using optimization directives such as\npragmas in the source code. Machine learning in the form of autotuning can\nassist users in finding the best optimizations for each platform.\n  In this paper we propose a loop transformation search space that takes the\nform of a tree, in contrast to previous approaches that usually use vector\nspaces to represent loop optimization configurations. We implemented a simple\nautotuner exploring the search space and applied it to a selected set of\nPolyBench kernels. While the autotuner is capable of representing every\npossible sequence of loop transformations and their relations, the results\nmotivate the use of better search strategies such as Monte Carlo tree search to\nfind sophisticated loop transformations such as multilevel tiling.",
            "journal ref": "None",
            "category": "cs.DC"
        },
        {
            "url": "http://arxiv.org/abs/2010.06549v2",
            "date": "2020-10-13 17:01:40+00:00",
            "title": "Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting",
            "authors": [
                "Ghazi Felhi",
                "Joseph Leroux",
                "Djam\u00e9 Seddah"
            ],
            "abstract": "Even though Variational Autoencoders (VAEs) are widely used for\nsemi-supervised learning, the reason why they work remains unclear. In fact,\nthe addition of the unsupervised objective is most often vaguely described as a\nregularization. The strength of this regularization is controlled by\ndown-weighting the objective on the unlabeled part of the training set. Through\nan analysis of the objective of semi-supervised VAEs, we observe that they use\nthe posterior of the learned generative model to guide the inference model in\nlearning the partially observed latent variable. We show that given this\nobservation, it is possible to gain finer control on the effect of the\nunsupervised objective on the training procedure. Using importance weighting,\nwe derive two novel objectives that prioritize either one of the partially\nobserved latent variable, or the unobserved latent variable. Experiments on the\nIMDB english sentiment analysis dataset and on the AG News topic classification\ndataset show the improvements brought by our prioritization mechanism and\nexhibit a behavior that is inline with our description of the inner working of\nSemi-Supervised VAEs.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.06572v1",
            "date": "2020-10-13 17:45:28+00:00",
            "title": "Does my multimodal model learn cross-modal interactions? It's harder to tell than you might think!",
            "authors": [
                "Jack Hessel",
                "Lillian Lee"
            ],
            "abstract": "Modeling expressive cross-modal interactions seems crucial in multimodal\ntasks, such as visual question answering. However, sometimes high-performing\nblack-box algorithms turn out to be mostly exploiting unimodal signals in the\ndata. We propose a new diagnostic tool, empirical multimodally-additive\nfunction projection (EMAP), for isolating whether or not cross-modal\ninteractions improve performance for a given model on a given task. This\nfunction projection modifies model predictions so that cross-modal interactions\nare eliminated, isolating the additive, unimodal structure. For seven\nimage+text classification tasks (on each of which we set new state-of-the-art\nbenchmarks), we find that, in many cases, removing cross-modal interactions\nresults in little to no performance degradation. Surprisingly, this holds even\nwhen expressive models, with capacity to consider interactions, otherwise\noutperform less expressive models; thus, performance improvements, even when\npresent, often cannot be attributed to consideration of cross-modal feature\ninteractions. We hence recommend that researchers in multimodal machine\nlearning report the performance not only of unimodal baselines, but also the\nEMAP of their best-performing model.",
            "journal ref": "Published in EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06579v1",
            "date": "2020-10-13 17:57:18+00:00",
            "title": "Fantastic Features and Where to Find Them: Detecting Cognitive Impairment with a Subsequence Classification Guided Approach",
            "authors": [
                "Benjamin Eyre",
                "Aparna Balagopalan",
                "Jekaterina Novikova"
            ],
            "abstract": "Despite the widely reported success of embedding-based machine learning\nmethods on natural language processing tasks, the use of more easily\ninterpreted engineered features remains common in fields such as cognitive\nimpairment (CI) detection. Manually engineering features from noisy text is\ntime and resource consuming, and can potentially result in features that do not\nenhance model performance. To combat this, we describe a new approach to\nfeature engineering that leverages sequential machine learning models and\ndomain knowledge to predict which features help enhance performance. We provide\na concrete example of this method on a standard data set of CI speech and\ndemonstrate that CI classification accuracy improves by 2.3% over a strong\nbaseline when using features produced by this method. This demonstration\nprovides an ex-ample of how this method can be used to assist classification in\nfields where interpretability is important, such as health care.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.06710v1",
            "date": "2020-10-13 21:51:14+00:00",
            "title": "Language Networks: a Practical Approach",
            "authors": [
                "Jorge A. V. Tohalino",
                "Diego R. Amancio"
            ],
            "abstract": "This manuscript provides a short and practical introduction to the topic of\nlanguage networks. This text aims at assisting researchers with no practical\nexperience in text and/or network analysis. We provide a practical tutorial on\nhow to model and characterize texts using network-based features. In this\ntutorial, we also include examples of pre-processing and network\nrepresentations. A brief description of the main tasks allying network science\nand text analysis is also provided. A further development of this text shall\ninclude a practical description of network classification via machine learning\nmethods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06714v1",
            "date": "2020-10-13 22:00:31+00:00",
            "title": "CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring",
            "authors": [
                "Jiaxin Huang",
                "Yiqing Xie",
                "Yu Meng",
                "Yunyi Zhang",
                "Jiawei Han"
            ],
            "abstract": "Taxonomy is not only a fundamental form of knowledge representation, but also\ncrucial to vast knowledge-rich applications, such as question answering and web\nsearch. Most existing taxonomy construction methods extract hypernym-hyponym\nentity pairs to organize a \"universal\" taxonomy. However, these generic\ntaxonomies cannot satisfy user's specific interest in certain areas and\nrelations. Moreover, the nature of instance taxonomy treats each node as a\nsingle word, which has low semantic coverage. In this paper, we propose a\nmethod for seed-guided topical taxonomy construction, which takes a corpus and\na seed taxonomy described by concept names as input, and constructs a more\ncomplete taxonomy based on user's interest, wherein each node is represented by\na cluster of coherent terms. Our framework, CoRel, has two modules to fulfill\nthis goal. A relation transferring module learns and transfers the user's\ninterested relation along multiple paths to expand the seed taxonomy structure\nin width and depth. A concept learning module enriches the semantics of each\nconcept node by jointly embedding the taxonomy and text. Comprehensive\nexperiments conducted on real-world datasets show that Corel generates\nhigh-quality topical taxonomies and outperforms all the baselines\nsignificantly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06792v2",
            "date": "2020-10-14 03:20:46+00:00",
            "title": "Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach",
            "authors": [
                "Bowen Tan",
                "Lianhui Qin",
                "Eric P. Xing",
                "Zhiting Hu"
            ],
            "abstract": "Given a document and a target aspect (e.g., a topic of interest),\naspect-based abstractive summarization attempts to generate a summary with\nrespect to the aspect. Previous studies usually assume a small pre-defined set\nof aspects and fall short of summarizing on other diverse topics. In this work,\nwe study summarizing on arbitrary aspects relevant to the document, which\nsignificantly expands the application of the task in practice. Due to the lack\nof supervision data, we develop a new weak supervision construction method and\nan aspect modeling scheme, both of which integrate rich external knowledge\nsources such as ConceptNet and Wikipedia. Experiments show our approach\nachieves performance boosts on summarizing both real and synthetic documents\ngiven pre-defined or arbitrary aspects.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06801v1",
            "date": "2020-10-14 04:01:54+00:00",
            "title": "A Graph Representation of Semi-structured Data for Web Question Answering",
            "authors": [
                "Xingyao Zhang",
                "Linjun Shou",
                "Jian Pei",
                "Ming Gong",
                "Lijie Wen",
                "Daxin Jiang"
            ],
            "abstract": "The abundant semi-structured data on the Web, such as HTML-based tables and\nlists, provide commercial search engines a rich information source for question\nanswering (QA). Different from plain text passages in Web documents, Web tables\nand lists have inherent structures, which carry semantic correlations among\nvarious elements in tables and lists. Many existing studies treat tables and\nlists as flat documents with pieces of text and do not make good use of\nsemantic information hidden in structures. In this paper, we propose a novel\ngraph representation of Web tables and lists based on a systematic\ncategorization of the components in semi-structured data as well as their\nrelations. We also develop pre-training and reasoning techniques on the graph\nmodel for the QA task. Extensive experiments on several real datasets collected\nfrom a commercial engine verify the effectiveness of our approach. Our method\nimproves F1 score by 3.90 points over the state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06822v2",
            "date": "2020-10-14 06:24:25+00:00",
            "title": "Modeling Protagonist Emotions for Emotion-Aware Storytelling",
            "authors": [
                "Faeze Brahman",
                "Snigdha Chaturvedi"
            ],
            "abstract": "Emotions and their evolution play a central role in creating a captivating\nstory. In this paper, we present the first study on modeling the emotional\ntrajectory of the protagonist in neural storytelling. We design methods that\ngenerate stories that adhere to given story titles and desired emotion arcs for\nthe protagonist. Our models include Emotion Supervision (EmoSup) and two\nEmotion-Reinforced (EmoRL) models. The EmoRL models use special rewards\ndesigned to regularize the story generation process through reinforcement\nlearning. Our automatic and manual evaluations demonstrate that these models\nare significantly better at generating stories that follow the desired emotion\narcs compared to baseline methods, without sacrificing story quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06943v2",
            "date": "2020-10-14 10:45:01+00:00",
            "title": "Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretability",
            "authors": [
                "Yuxian Meng",
                "Chun Fan",
                "Zijun Sun",
                "Eduard Hovy",
                "Fei Wu",
                "Jiwei Li"
            ],
            "abstract": "Any prediction from a model is made by a combination of learning history and\ntest stimuli. This provides significant insights for improving model\ninterpretability: {\\it because of which part(s) of which training example(s),\nthe model attends to which part(s) of a test example}. Unfortunately, existing\nmethods to interpret a model's predictions are only able to capture a single\naspect of either test stimuli or learning history, and evidences from both are\nnever combined or integrated. In this paper, we propose an efficient and\ndifferentiable approach to make it feasible to interpret a model's prediction\nby jointly examining training history and test stimuli. Test stimuli is first\nidentified by gradient-based methods, signifying {\\it the part of a test\nexample that the model attends to}. The gradient-based saliency scores are then\npropagated to training examples using influence functions to identify {\\it\nwhich part(s) of which training example(s)} make the model attends to the test\nstimuli. The system is differentiable and time efficient: the adoption of\nsaliency scores from gradient-based methods allows us to efficiently trace a\nmodel's prediction through test stimuli, and then back to training examples\nthrough influence functions. We demonstrate that the proposed methodology\noffers clear explanations about neural model decisions, along with being useful\nfor performing error analysis, crafting adversarial examples and fixing\nerroneously classified examples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.06975v2",
            "date": "2020-10-14 11:37:24+00:00",
            "title": "Medical Code Assignment with Gated Convolution and Note-Code Interaction",
            "authors": [
                "Shaoxiong Ji",
                "Shirui Pan",
                "Pekka Marttinen"
            ],
            "abstract": "Medical code assignment from clinical text is a fundamental task in clinical\ninformation system management. As medical notes are typically lengthy and the\nmedical coding system's code space is large, this task is a long-standing\nchallenge. Recent work applies deep neural network models to encode the medical\nnotes and assign medical codes to clinical documents. However, these methods\nare still ineffective as they do not fully encode and capture the lengthy and\nrich semantic information of medical notes nor explicitly exploit the\ninteractions between the notes and codes. We propose a novel method, gated\nconvolutional neural networks, and a note-code interaction (GatedCNN-NCI), for\nautomatic medical code assignment to overcome these challenges. Our methods\ncapture the rich semantic information of the lengthy clinical text for better\nrepresentation by utilizing embedding injection and gated information\npropagation in the medical note encoding module. With a novel note-code\ninteraction design and a graph message passing mechanism, we explicitly capture\nthe underlying dependency between notes and codes, enabling effective code\nprediction. A weight sharing scheme is further designed to decrease the number\nof trainable parameters. Empirical experiments on real-world clinical datasets\nshow that our proposed model outperforms state-of-the-art models in most cases,\nand our model size is on par with light-weighted baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07003v1",
            "date": "2020-10-14 12:28:08+00:00",
            "title": "Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search",
            "authors": [
                "Gyuwan Kim",
                "Kyunghyun Cho"
            ],
            "abstract": "Although transformers have achieved impressive accuracies in various tasks in\nnatural language processing, they often come with a prohibitive computational\ncost, that prevents their use in scenarios with limited computational resources\nfor inference. This need for computational efficiency in inference has been\naddressed by for instance PoWER-BERT (Goyal et al., 2020) which gradually\ndecreases the length of a sequence as it is passed through layers. These\napproaches however often assume that the target computational complexity is\nknown in advance at the time of training. This implies that a separate model\nmust be trained for each inference scenario with its distinct computational\nbudget. In this paper, we extend PoWER-BERT to address this issue of\ninefficiency and redundancy. The proposed extension enables us to train a\nlarge-scale transformer, called Length-Adaptive Transformer, once and uses it\nfor various inference scenarios without re-training it. To do so, we train a\ntransformer with LengthDrop, a structural variant of dropout, which\nstochastically determines the length of a sequence at each layer. We then use a\nmulti-objective evolutionary search to find a length configuration that\nmaximizes the accuracy and minimizes the computational complexity under any\ngiven computational budget. Additionally, we significantly extend the\napplicability of PoWER-BERT beyond sequence-level classification into\ntoken-level classification such as span-based question-answering, by\nintroducing the idea of Drop-and-Restore. With Drop-and-Restore, word-vectors\nare dropped temporarily in intermediate layers and restored at the last layer\nif necessary. We empirically verify the utility of the proposed approach by\ndemonstrating the superior accuracy-efficiency trade-off under various setups,\nincluding SQuAD 1.1, MNLI-m, and SST-2. Code is available at\nhttps://github.com/clovaai/length-adaptive-transformer.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07075v1",
            "date": "2020-10-14 13:24:43+00:00",
            "title": "AutoADR: Automatic Model Design for Ad Relevance",
            "authors": [
                "Yiren Chen",
                "Yaming Yang",
                "Hong Sun",
                "Yujing Wang",
                "Yu Xu",
                "Wei Shen",
                "Rong Zhou",
                "Yunhai Tong",
                "Jing Bai",
                "Ruofei Zhang"
            ],
            "abstract": "Large-scale pre-trained models have attracted extensive attention in the\nresearch community and shown promising results on various tasks of natural\nlanguage processing. However, these pre-trained models are memory and\ncomputation intensive, hindering their deployment into industrial online\nsystems like Ad Relevance. Meanwhile, how to design an effective yet efficient\nmodel architecture is another challenging problem in online Ad Relevance.\nRecently, AutoML shed new lights on architecture design, but how to integrate\nit with pre-trained language models remains unsettled. In this paper, we\npropose AutoADR (Automatic model design for AD Relevance) -- a novel end-to-end\nframework to address this challenge, and share our experience to ship these\ncutting-edge techniques into online Ad Relevance system at Microsoft Bing.\nSpecifically, AutoADR leverages a one-shot neural architecture search algorithm\nto find a tailored network architecture for Ad Relevance. The search process is\nsimultaneously guided by knowledge distillation from a large pre-trained\nteacher model (e.g. BERT), while taking the online serving constraints (e.g.\nmemory and latency) into consideration. We add the model designed by AutoADR as\na sub-model into the production Ad Relevance model. This additional sub-model\nimproves the Precision-Recall AUC (PR AUC) on top of the original Ad Relevance\nmodel by 2.65X of the normalized shipping bar. More importantly, adding this\nautomatically designed sub-model leads to a statistically significant 4.6%\nBad-Ad ratio reduction in online A/B testing. This model has been shipped into\nMicrosoft Bing Ad Relevance Production model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07079v2",
            "date": "2020-10-14 13:26:39+00:00",
            "title": "Recipes for Safety in Open-domain Chatbots",
            "authors": [
                "Jing Xu",
                "Da Ju",
                "Margaret Li",
                "Y-Lan Boureau",
                "Jason Weston",
                "Emily Dinan"
            ],
            "abstract": "Models trained on large unlabeled corpora of human interactions will learn\npatterns and mimic behaviors therein, which include offensive or otherwise\ntoxic behavior and unwanted biases. We investigate a variety of methods to\nmitigate these issues in the context of open-domain generative dialogue models.\nWe introduce a new human-and-model-in-the-loop framework for both training\nsafer models and for evaluating them, as well as a novel method to distill\nsafety considerations inside generative models without the use of an external\nclassifier at deployment time. We conduct experiments comparing these methods\nand find our new techniques are (i) safer than existing models as measured by\nautomatic and human evaluations while (ii) maintaining usability metrics such\nas engagingness relative to the state of the art. We then discuss the\nlimitations of this work by analyzing failure cases of our models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07095v1",
            "date": "2020-10-14 13:53:08+00:00",
            "title": "A Relaxed Matching Procedure for Unsupervised BLI",
            "authors": [
                "Xu Zhao",
                "Zihao Wang",
                "Hao Wu",
                "Yong Zhang"
            ],
            "abstract": "Recently unsupervised Bilingual Lexicon Induction (BLI) without any parallel\ncorpus has attracted much research interest. One of the crucial parts in\nmethods for the BLI task is the matching procedure. Previous works impose a too\nstrong constraint on the matching and lead to many counterintuitive translation\npairings. Thus, We propose a relaxed matching procedure to find a more precise\nmatching between two languages. We also find that aligning source and target\nlanguage embedding space bidirectionally will bring significant improvement. We\nfollow the previous iterative framework to conduct experiments. Results on\nstandard benchmark demonstrate the effectiveness of our proposed method, which\nsubstantially outperforms previous unsupervised methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07212v2",
            "date": "2020-10-14 16:26:13+00:00",
            "title": "Geometry matters: Exploring language examples at the decision boundary",
            "authors": [
                "Debajyoti Datta",
                "Shashwat Kumar",
                "Laura Barnes",
                "Tom Fletcher"
            ],
            "abstract": "A growing body of recent evidence has highlighted the limitations of natural\nlanguage processing (NLP) datasets and classifiers. These include the presence\nof annotation artifacts in datasets, classifiers relying on shallow features\nlike a single word (e.g., if a movie review has the word \"romantic\", the review\ntends to be positive), or unnecessary words (e.g., learning a proper noun to\nclassify a movie as positive or negative). The presence of such artifacts has\nsubsequently led to the development of challenging datasets to force the model\nto generalize better. While a variety of heuristic strategies, such as\ncounterfactual examples and contrast sets, have been proposed, the theoretical\njustification about what makes these examples difficult for the classifier is\noften lacking or unclear. In this paper, using tools from information geometry,\nwe propose a theoretical way to quantify the difficulty of an example in NLP.\nUsing our approach, we explore difficult examples for several deep learning\narchitectures. We discover that both BERT, CNN and fasttext are susceptible to\nword substitutions in high difficulty examples. These classifiers tend to\nperform poorly on the FIM test set. (generated by sampling and perturbing\ndifficult examples, with accuracy dropping below 50%). We replicate our\nexperiments on 5 NLP datasets (YelpReviewPolarity, AGNEWS, SogouNews,\nYelpReviewFull and Yahoo Answers). On YelpReviewPolarity we observe a\ncorrelation coefficient of -0.4 between resilience to perturbations and the\ndifficulty score. Similarly we observe a correlation of 0.35 between the\ndifficulty score and the empirical success probability of random substitutions.\nOur approach is simple, architecture agnostic and can be used to study the\nfragilities of text classification models. All the code used will be made\npublicly available, including a tool to explore the difficult examples for\nother datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07245v1",
            "date": "2020-10-14 17:06:41+00:00",
            "title": "Text Classification Using Label Names Only: A Language Model Self-Training Approach",
            "authors": [
                "Yu Meng",
                "Yunyi Zhang",
                "Jiaxin Huang",
                "Chenyan Xiong",
                "Heng Ji",
                "Chao Zhang",
                "Jiawei Han"
            ],
            "abstract": "Current text classification methods typically require a good number of\nhuman-labeled documents as training data, which can be costly and difficult to\nobtain in real applications. Humans can perform classification without seeing\nany labeled examples but only based on a small set of words describing the\ncategories to be classified. In this paper, we explore the potential of only\nusing the label name of each class to train classification models on unlabeled\ndata, without using any labeled documents. We use pre-trained neural language\nmodels both as general linguistic knowledge sources for category understanding\nand as representation learning models for document classification. Our method\n(1) associates semantically related words with the label names, (2) finds\ncategory-indicative words and trains the model to predict their implied\ncategories, and (3) generalizes the model via self-training. We show that our\nmodel achieves around 90% accuracy on four benchmark datasets including topic\nand sentiment classification without using any labeled documents but learning\nfrom unlabeled data supervised by at most 3 words (1 in most cases) per class\nas the label name.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07292v2",
            "date": "2020-10-14 21:33:36+00:00",
            "title": "My Team Will Go On: Differentiating High and Low Viability Teams through Team Interaction",
            "authors": [
                "Hancheng Cao",
                "Vivian Yang",
                "Victor Chen",
                "Yu Jin Lee",
                "Lydia Stone",
                "N'godjigui Junior Diarrassouba",
                "Mark E. Whiting",
                "Michael S. Bernstein"
            ],
            "abstract": "Understanding team viability -- a team's capacity for sustained and future\nsuccess -- is essential for building effective teams. In this study, we\naggregate features drawn from the organizational behavior literature to train a\nviability classification model over a dataset of 669 10-minute text\nconversations of online teams. We train classifiers to identify teams at the\ntop decile (most viable teams), 50th percentile (above a median split), and\nbottom decile (least viable teams), then characterize the attributes of teams\nat each of these viability levels. We find that a lasso regression model\nachieves an accuracy of .74--.92 AUC ROC under different thresholds of\nclassifying viability scores. From these models, we identify the use of\nexclusive language such as `but' and `except', and the use of second person\npronouns, as the most predictive features for detecting the most viable teams,\nsuggesting that active engagement with others' ideas is a crucial signal of a\nviable team. Only a small fraction of the 10-minute discussion, as little as 70\nseconds, is required for predicting the viability of team interaction. This\nwork suggests opportunities for teams to assess, track, and visualize their own\nviability in real time as they collaborate.",
            "journal ref": "Proc. ACM Hum.-Comput. Interact. 4, CSCW3, Article 230 (December\n  2020)",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2010.07410v1",
            "date": "2020-10-14 21:28:06+00:00",
            "title": "Six Attributes of Unhealthy Conversation",
            "authors": [
                "Ilan Price",
                "Jordan Gifford-Moore",
                "Jory Fleming",
                "Saul Musker",
                "Maayan Roichman",
                "Guillaume Sylvain",
                "Nithum Thain",
                "Lucas Dixon",
                "Jeffrey Sorensen"
            ],
            "abstract": "We present a new dataset of approximately 44000 comments labeled by\ncrowdworkers. Each comment is labelled as either 'healthy' or 'unhealthy', in\naddition to binary labels for the presence of six potentially 'unhealthy'\nsub-attributes: (1) hostile; (2) antagonistic, insulting, provocative or\ntrolling; (3) dismissive; (4) condescending or patronising; (5) sarcastic;\nand/or (6) an unfair generalisation. Each label also has an associated\nconfidence score. We argue that there is a need for datasets which enable\nresearch based on a broad notion of 'unhealthy online conversation'. We build\nthis typology to encompass a substantial proportion of the individual comments\nwhich contribute to unhealthy online conversation. For some of these\nattributes, this is the first publicly available dataset of this scale. We\nexplore the quality of the dataset, present some summary statistics and initial\nmodels to illustrate the utility of this data, and highlight limitations and\ndirections for further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07414v3",
            "date": "2020-10-14 21:47:03+00:00",
            "title": "On Cross-Dataset Generalization in Automatic Detection of Online Abuse",
            "authors": [
                "Isar Nejadgholi",
                "Svetlana Kiritchenko"
            ],
            "abstract": "NLP research has attained high performances in abusive language detection as\na supervised classification task. While in research settings, training and test\ndatasets are usually obtained from similar data samples, in practice systems\nare often applied on data that are different from the training set in topic and\nclass distributions. Also, the ambiguity in class definitions inherited in this\ntask aggravates the discrepancies between source and target datasets. We\nexplore the topic bias and the task formulation bias in cross-dataset\ngeneralization. We show that the benign examples in the Wikipedia Detox dataset\nare biased towards platform-specific topics. We identify these examples using\nunsupervised topic modeling and manual inspection of topics' keywords. Removing\nthese topics increases cross-dataset generalization, without reducing in-domain\nclassification performance. For a robust dataset design, we suggest applying\ninexpensive unsupervised methods to inspect the collected data and downsize the\nnon-generalizable content before manually annotating for class labels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07435v1",
            "date": "2020-10-14 23:26:28+00:00",
            "title": "From Language to Language-ish: How Brain-Like is an LSTM's Representation of Nonsensical Language Stimuli?",
            "authors": [
                "Maryam Hashemzadeh",
                "Greta Kaufeld",
                "Martha White",
                "Andrea E. Martin",
                "Alona Fyshe"
            ],
            "abstract": "The representations generated by many models of language (word embeddings,\nrecurrent neural networks and transformers) correlate to brain activity\nrecorded while people read. However, these decoding results are usually based\non the brain's reaction to syntactically and semantically sound language\nstimuli. In this study, we asked: how does an LSTM (long short term memory)\nlanguage model, trained (by and large) on semantically and syntactically intact\nlanguage, represent a language sample with degraded semantic or syntactic\ninformation? Does the LSTM representation still resemble the brain's reaction?\nWe found that, even for some kinds of nonsensical language, there is a\nstatistically significant relationship between the brain's activity and the\nrepresentations of an LSTM. This indicates that, at least in some instances,\nLSTMs and the human brain handle nonsensical data similarly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07447v1",
            "date": "2020-10-15 00:31:15+00:00",
            "title": "Semantic Label Smoothing for Sequence to Sequence Problems",
            "authors": [
                "Michal Lukasik",
                "Himanshu Jain",
                "Aditya Krishna Menon",
                "Seungyeon Kim",
                "Srinadh Bhojanapalli",
                "Felix Yu",
                "Sanjiv Kumar"
            ],
            "abstract": "Label smoothing has been shown to be an effective regularization strategy in\nclassification, that prevents overfitting and helps in label de-noising.\nHowever, extending such methods directly to seq2seq settings, such as Machine\nTranslation, is challenging: the large target output space of such problems\nmakes it intractable to apply label smoothing over all possible outputs. Most\nexisting approaches for seq2seq settings either do token level smoothing, or\nsmooth over sequences generated by randomly substituting tokens in the target\nsequence. Unlike these works, in this paper, we propose a technique that\nsmooths over \\emph{well formed} relevant sequences that not only have\nsufficient n-gram overlap with the target sequence, but are also\n\\emph{semantically similar}. Our method shows a consistent and significant\nimprovement over the state-of-the-art techniques on different datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07466v1",
            "date": "2020-10-15 01:33:05+00:00",
            "title": "The Language of Food during the Pandemic: Hints about the Dietary Effects of Covid-19",
            "authors": [
                "Hoang Van",
                "Ahmad Musa",
                "Mihai Surdeanu",
                "Stephen Kobourov"
            ],
            "abstract": "We study the language of food on Twitter during the pandemic lockdown in the\nUnited States, focusing on the two month period of March 15 to May 15, 2020.\nSpecifically, we analyze over770,000 tweets published during the lockdown and\nthe equivalent period in the five previous years and highlight several worrying\ntrends. First, we observe that during the lockdown there was a notable shift\nfrom mentions of healthy foods to unhealthy foods. Second, we show an increased\npointwise mutual information of depression hashtags with food-related tweets\nposted during the lockdown and an increased association between depression\nhashtags and unhealthy foods, tobacco, and alcohol during the lockdown.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07497v1",
            "date": "2020-10-15 03:34:33+00:00",
            "title": "MedDG: A Large-scale Medical Consultation Dataset for Building Medical Dialogue System",
            "authors": [
                "Wenge Liu",
                "Jianheng Tang",
                "Jinghui Qin",
                "Lin Xu",
                "Zhen Li",
                "Xiaodan Liang"
            ],
            "abstract": "Developing conversational agents to interact with patients and provide\nprimary clinical advice has attracted increasing attention due to its huge\napplication potential, especially in the time of COVID-19 Pandemic. However,\nthe training of end-to-end neural-based medical dialogue system is restricted\nby an insufficient quantity of medical dialogue corpus. In this work, we make\nthe first attempt to build and release a large-scale high-quality Medical\nDialogue dataset related to 12 types of common Gastrointestinal diseases named\nMedDG, with more than 17K conversations collected from the online health\nconsultation community. Five different categories of entities, including\ndiseases, symptoms, attributes, tests, and medicines, are annotated in each\nconversation of MedDG as additional labels. To push forward the future research\non building expert-sensitive medical dialogue system, we proposes two kinds of\nmedical dialogue tasks based on MedDG dataset. One is the next entity\nprediction and the other is the doctor response generation. To acquire a clear\ncomprehension on these two medical dialogue tasks, we implement several\nstate-of-the-art benchmarks, as well as design two dialogue models with a\nfurther consideration on the predicted entities. Experimental results show that\nthe pre-train language models and other baselines struggle on both tasks with\npoor performance in our dataset, and the response quality can be enhanced with\nthe help of auxiliary entity information. From human evaluation, the simple\nretrieval model outperforms several state-of-the-art generative models,\nindicating that there still remains a large room for improvement on generating\nmedically meaningful responses.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07523v2",
            "date": "2020-10-15 05:01:20+00:00",
            "title": "Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis",
            "authors": [
                "Zhengxuan Wu",
                "Desmond C. Ong"
            ],
            "abstract": "Aspect-based sentiment analysis (ABSA) and Targeted ASBA (TABSA) allow\nfiner-grained inferences about sentiment to be drawn from the same text,\ndepending on context. For example, a given text can have different targets\n(e.g., neighborhoods) and different aspects (e.g., price or safety), with\ndifferent sentiment associated with each target-aspect pair. In this paper, we\ninvestigate whether adding context to self-attention models improves\nperformance on (T)ABSA. We propose two variants of Context-Guided BERT\n(CG-BERT) that learn to distribute attention under different contexts. We first\nadapt a context-aware Transformer to produce a CG-BERT that uses context-guided\nsoftmax-attention. Next, we propose an improved Quasi-Attention CG-BERT model\nthat learns a compositional attention that supports subtractive attention. We\ntrain both models with pretrained BERT on two (T)ABSA datasets: SentiHood and\nSemEval-2014 (Task 4). Both models achieve new state-of-the-art results with\nour QACG-BERT model having the best performance. Furthermore, we provide\nanalyses of the impact of context in the our proposed models. Our work provides\nmore evidence for the utility of adding context-dependencies to pretrained\nself-attention-based language models for context-based natural language tasks.",
            "journal ref": "AAAI 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07526v1",
            "date": "2020-10-15 05:08:56+00:00",
            "title": "Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs",
            "authors": [
                "Ana Marasovi\u0107",
                "Chandra Bhagavatula",
                "Jae Sung Park",
                "Ronan Le Bras",
                "Noah A. Smith",
                "Yejin Choi"
            ],
            "abstract": "Natural language rationales could provide intuitive, higher-level\nexplanations that are easily understandable by humans, complementing the more\nbroadly studied lower-level explanations based on gradients or attention\nweights. We present the first study focused on generating natural language\nrationales across several complex visual reasoning tasks: visual commonsense\nreasoning, visual-textual entailment, and visual question answering. The key\nchallenge of accurate rationalization is comprehensive image understanding at\nall levels: not just their explicit content at the pixel level, but their\ncontextual contents at the semantic and pragmatic levels. We present\nRationale^VT Transformer, an integrated model that learns to generate free-text\nrationales by combining pretrained language models with object recognition,\ngrounded visual semantic frames, and visual commonsense graphs. Our experiments\nshow that the base pretrained language model benefits from visual adaptation\nand that free-text rationalization is a promising research direction to\ncomplement model interpretability for complex visual-textual reasoning tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07620v2",
            "date": "2020-10-15 09:30:46+00:00",
            "title": "GMH: A General Multi-hop Reasoning Model for KG Completion",
            "authors": [
                "Yao Zhang",
                "Xu Zhang",
                "Jun Wang",
                "Hongru Liang",
                "Adam Jatowt",
                "Wenqiang Lei",
                "Zhenglu Yang"
            ],
            "abstract": "Knowledge graphs are essential for numerous downstream natural language\nprocessing applications, but are typically incomplete with many facts missing.\nThis results in research efforts on multi-hop reasoning task, which can be\nformulated as a search process and current models typically perform short\ndistance reasoning. However, the long-distance reasoning is also vital with the\nability to connect the superficially unrelated entities. To the best of our\nknowledge, there lacks a general framework that approaches multi-hop reasoning\nin both short and long scenarios. We argue that there are two key issues for\nlong distance reasoning: i) which edge to select, and ii) when to stop the\nsearch. In this work, we propose a general model which resolves the issues with\nthree modules: 1) the local-global knowledge module to estimate the possible\npaths, 2) the differentiated action dropout module to explore a diverse set of\npaths, and 3) the adaptive stopping search module to avoid over searching. The\ncomprehensive results on three datasets demonstrate the superiority of our\nmodel with significant improvements against baselines in both short and long\ndistance reasoning scenarios.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.07637v1",
            "date": "2020-10-15 10:10:41+00:00",
            "title": "DialogueTRM: Exploring the Intra- and Inter-Modal Emotional Behaviors in the Conversation",
            "authors": [
                "Yuzhao Mao",
                "Qi Sun",
                "Guang Liu",
                "Xiaojie Wang",
                "Weiguo Gao",
                "Xuan Li",
                "Jianping Shen"
            ],
            "abstract": "Emotion Recognition in Conversations (ERC) is essential for building\nempathetic human-machine systems. Existing studies on ERC primarily focus on\nsummarizing the context information in a conversation, however, ignoring the\ndifferentiated emotional behaviors within and across different modalities.\nDesigning appropriate strategies that fit the differentiated multi-modal\nemotional behaviors can produce more accurate emotional predictions. Thus, we\npropose the DialogueTransformer to explore the differentiated emotional\nbehaviors from the intra- and inter-modal perspectives. For intra-modal, we\nconstruct a novel Hierarchical Transformer that can easily switch between\nsequential and feed-forward structures according to the differentiated context\npreference within each modality. For inter-modal, we constitute a novel\nMulti-Grained Interactive Fusion that applies both neuron- and vector-grained\nfeature interactions to learn the differentiated contributions across all\nmodalities. Experimental results show that DialogueTRM outperforms the\nstate-of-the-art by a significant margin on three benchmark datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07668v1",
            "date": "2020-10-15 11:25:54+00:00",
            "title": "Inducing Alignment Structure with Gated Graph Attention Networks for Sentence Matching",
            "authors": [
                "Peng Cui",
                "Le Hu",
                "Yuanchao Liu"
            ],
            "abstract": "Sentence matching is a fundamental task of natural language processing with\nvarious applications. Most recent approaches adopt attention-based neural\nmodels to build word- or phrase-level alignment between two sentences. However,\nthese models usually ignore the inherent structure within the sentences and\nfail to consider various dependency relationships among text units. To address\nthese issues, this paper proposes a graph-based approach for sentence matching.\nFirst, we represent a sentence pair as a graph with several carefully design\nstrategies. We then employ a novel gated graph attention network to encode the\nconstructed graph for sentence matching. Experimental results demonstrate that\nour method substantially achieves state-of-the-art performance on two datasets\nacross tasks of natural language and paraphrase identification. Further\ndiscussions show that our model can learn meaningful graph structure,\nindicating its superiority on improved interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07676v1",
            "date": "2020-10-15 11:50:12+00:00",
            "title": "Reliable Evaluations for Natural Language Inference based on a Unified Cross-dataset Benchmark",
            "authors": [
                "Guanhua Zhang",
                "Bing Bai",
                "Jian Liang",
                "Kun Bai",
                "Conghui Zhu",
                "Tiejun Zhao"
            ],
            "abstract": "Recent studies show that crowd-sourced Natural Language Inference (NLI)\ndatasets may suffer from significant biases like annotation artifacts. Models\nutilizing these superficial clues gain mirage advantages on the in-domain\ntesting set, which makes the evaluation results over-estimated. The lack of\ntrustworthy evaluation settings and benchmarks stalls the progress of NLI\nresearch. In this paper, we propose to assess a model's trustworthy\ngeneralization performance with cross-datasets evaluation. We present a new\nunified cross-datasets benchmark with 14 NLI datasets, and re-evaluate 9\nwidely-used neural network-based NLI models as well as 5 recently proposed\ndebiasing methods for annotation artifacts. Our proposed evaluation scheme and\nexperimental baselines could provide a basis to inspire future reliable NLI\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07717v2",
            "date": "2020-10-15 12:52:09+00:00",
            "title": "Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains",
            "authors": [
                "Weijie Yu",
                "Chen Xu",
                "Jun Xu",
                "Liang Pang",
                "Xiaopeng Gao",
                "Xiaozhao Wang",
                "Ji-Rong Wen"
            ],
            "abstract": "One approach to matching texts from asymmetrical domains is projecting the\ninput sequences into a common semantic space as feature vectors upon which the\nmatching function can be readily defined and learned. In real-world matching\npractices, it is often observed that with the training goes on, the feature\nvectors projected from different domains tend to be indistinguishable. The\nphenomenon, however, is often overlooked in existing matching models. As a\nresult, the feature vectors are constructed without any regularization, which\ninevitably increases the difficulty of learning the downstream matching\nfunctions. In this paper, we propose a novel match method tailored for text\nmatching in asymmetrical domains, called WD-Match. In WD-Match, a Wasserstein\ndistance-based regularizer is defined to regularize the features vectors\nprojected from different domains. As a result, the method enforces the feature\nprojection function to generate vectors such that those correspond to different\ndomains cannot be easily discriminated. The training process of WD-Match\namounts to a game that minimizes the matching loss regularized by the\nWasserstein distance. WD-Match can be used to improve different text matching\nmethods, by using the method as its underlying matching model. Four popular\ntext matching methods have been exploited in the paper. Experimental results\nbased on four publicly available benchmarks showed that WD-Match consistently\noutperformed the underlying methods and the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07761v1",
            "date": "2020-10-15 14:04:03+00:00",
            "title": "Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings",
            "authors": [
                "Phillip Keung",
                "Julian Salazar",
                "Yichao Lu",
                "Noah A. Smith"
            ],
            "abstract": "We describe an unsupervised method to create pseudo-parallel corpora for\nmachine translation (MT) from unaligned text. We use multilingual BERT to\ncreate source and target sentence embeddings for nearest-neighbor search and\nadapt the model via self-training. We validate our technique by extracting\nparallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a\n24.5 point increase (absolute) in F1 scores over previous unsupervised methods.\nWe then improve an XLM-based unsupervised neural MT system pre-trained on\nWikipedia by supplementing it with pseudo-parallel text mined from the same\ncorpus, boosting unsupervised translation performance by up to 3.5 BLEU on the\nWMT'14 French-English and WMT'16 German-English tasks and outperforming the\nprevious state-of-the-art. Finally, we enrich the IWSLT'15 English-Vietnamese\ncorpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU\nimprovement on the low-resource MT task. We demonstrate that unsupervised\nbitext mining is an effective way of augmenting MT datasets and complements\nexisting techniques like initializing with pre-trained contextual embeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07785v1",
            "date": "2020-10-15 14:21:38+00:00",
            "title": "Response Selection for Multi-Party Conversations with Dynamic Topic Tracking",
            "authors": [
                "Weishi Wang",
                "Shafiq Joty",
                "Steven C. H. Hoi"
            ],
            "abstract": "While participants in a multi-party multi-turn conversation simultaneously\nengage in multiple conversation topics, existing response selection methods are\ndeveloped mainly focusing on a two-party single-conversation scenario. Hence,\nthe prolongation and transition of conversation topics are ignored by current\nmethods. In this work, we frame response selection as a dynamic topic tracking\ntask to match the topic between the response and relevant conversation context.\nWith this new formulation, we propose a novel multi-task learning framework\nthat supports efficient encoding through large pretrained models with only two\nutterances at once to perform dynamic topic disentanglement and response\nselection. We also propose Topic-BERT an essential pretraining step to embed\ntopic information into BERT with self-supervised learning. Experimental results\non the DSTC-8 Ubuntu IRC dataset show state-of-the-art results in response\nselection and topic disentanglement tasks outperforming existing methods by a\ngood margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07792v1",
            "date": "2020-10-15 14:34:26+00:00",
            "title": "Hierarchical Poset Decoding for Compositional Generalization in Language",
            "authors": [
                "Yinuo Guo",
                "Zeqi Lin",
                "Jian-Guang Lou",
                "Dongmei Zhang"
            ],
            "abstract": "We formalize human language understanding as a structured prediction task\nwhere the output is a partially ordered set (poset). Current encoder-decoder\narchitectures do not take the poset structure of semantics into account\nproperly, thus suffering from poor compositional generalization ability. In\nthis paper, we propose a novel hierarchical poset decoding paradigm for\ncompositional generalization in language. Intuitively: (1) the proposed\nparadigm enforces partial permutation invariance in semantics, thus avoiding\noverfitting to bias ordering information; (2) the hierarchical mechanism allows\nto capture high-level structures of posets. We evaluate our proposed decoder on\nCompositional Freebase Questions (CFQ), a large and realistic natural language\nquestion answering dataset that is specifically designed to measure\ncompositional generalization. Results show that it outperforms current\ndecoders.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07816v1",
            "date": "2020-10-15 15:11:22+00:00",
            "title": "Where's the Question? A Multi-channel Deep Convolutional Neural Network for Question Identification in Textual Data",
            "authors": [
                "George Michalopoulos",
                "Helen Chen",
                "Alexander Wong"
            ],
            "abstract": "In most clinical practice settings, there is no rigorous reviewing of the\nclinical documentation, resulting in inaccurate information captured in the\npatient medical records. The gold standard in clinical data capturing is\nachieved via \"expert-review\", where clinicians can have a dialogue with a\ndomain expert (reviewers) and ask them questions about data entry rules.\nAutomatically identifying \"real questions\" in these dialogues could uncover\nambiguities or common problems in data capturing in a given clinical setting.\n  In this study, we proposed a novel multi-channel deep convolutional neural\nnetwork architecture, namely Quest-CNN, for the purpose of separating real\nquestions that expect an answer (information or help) about an issue from\nsentences that are not questions, as well as from questions referring to an\nissue mentioned in a nearby sentence (e.g., can you clarify this?), which we\nwill refer as \"c-questions\". We conducted a comprehensive performance\ncomparison analysis of the proposed multi-channel deep convolutional neural\nnetwork against other deep neural networks. Furthermore, we evaluated the\nperformance of traditional rule-based and learning-based methods for detecting\nquestion sentences. The proposed Quest-CNN achieved the best F1 score both on a\ndataset of data entry-review dialogue in a dialysis care setting, and on a\ngeneral domain dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07835v3",
            "date": "2020-10-15 15:55:08+00:00",
            "title": "Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach",
            "authors": [
                "Yue Yu",
                "Simiao Zuo",
                "Haoming Jiang",
                "Wendi Ren",
                "Tuo Zhao",
                "Chao Zhang"
            ],
            "abstract": "Fine-tuned pre-trained language models (LMs) have achieved enormous success\nin many natural language processing (NLP) tasks, but they still require\nexcessive labeled data in the fine-tuning stage. We study the problem of\nfine-tuning pre-trained LMs using only weak supervision, without any labeled\ndata. This problem is challenging because the high capacity of LMs makes them\nprone to overfitting the noisy labels generated by weak supervision. To address\nthis problem, we develop a contrastive self-training framework, COSINE, to\nenable fine-tuning LMs with weak supervision. Underpinned by contrastive\nregularization and confidence-based reweighting, this contrastive self-training\nframework can gradually improve model fitting while effectively suppressing\nerror propagation. Experiments on sequence, token, and sentence pair\nclassification tasks show that our model outperforms the strongest baseline by\nlarge margins on 7 benchmarks in 6 tasks, and achieves competitive performance\nwith fully-supervised fine-tuning methods.",
            "journal ref": "NAACL-HLT 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07865v2",
            "date": "2020-10-15 16:37:41+00:00",
            "title": "Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time",
            "authors": [
                "Vladislav Lialin",
                "Rahul Goel",
                "Andrey Simanovsky",
                "Anna Rumshisky",
                "Rushin Shah"
            ],
            "abstract": "Currently used semantic parsing systems deployed in voice assistants can\nrequire weeks to train. Datasets for these models often receive small and\nfrequent updates, data patches. Each patch requires training a new model. To\nreduce training time, one can fine-tune the previously trained model on each\npatch, but naive fine-tuning exhibits catastrophic forgetting - degradation of\nthe model performance on the data not represented in the data patch. In this\nwork, we propose a simple method that alleviates catastrophic forgetting and\nshow that it is possible to match the performance of a model trained from\nscratch in less than 10% of a time via fine-tuning. The key to achieving this\nis supersampling and EWC regularization. We demonstrate the effectiveness of\nour method on multiple splits of the Facebook TOP and SNIPS datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07891v2",
            "date": "2020-10-15 17:14:09+00:00",
            "title": "Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention",
            "authors": [
                "Ekta Sood",
                "Simon Tannert",
                "Philipp Mueller",
                "Andreas Bulling"
            ],
            "abstract": "A lack of corpora has so far limited advances in integrating human gaze data\nas a supervisory signal in neural attention mechanisms for natural language\nprocessing(NLP). We propose a novel hybrid text saliency model(TSM) that, for\nthe first time, combines a cognitive model of reading with explicit human gaze\nsupervision in a single machine learning framework. On four different corpora\nwe demonstrate that our hybrid TSM duration predictions are highly correlated\nwith human gaze ground truth. We further propose a novel joint modeling\napproach to integrate TSM predictions into the attention layer of a network\ndesigned for a specific upstream NLP task without the need for any\ntask-specific human gaze data. We demonstrate that our joint model outperforms\nthe state of the art in paraphrase generation on the Quora Question Pairs\ncorpus by more than 10% in BLEU-4 and achieves state of the art performance for\nsentence compression on the challenging Google Sentence Compression corpus. As\nsuch, our work introduces a practical approach for bridging between data-driven\nand cognitive models and demonstrates a new way to integrate human gaze-guided\nneural attention into NLP tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.07972v2",
            "date": "2020-10-15 18:34:13+00:00",
            "title": "Explicit Alignment Objectives for Multilingual Bidirectional Encoders",
            "authors": [
                "Junjie Hu",
                "Melvin Johnson",
                "Orhan Firat",
                "Aditya Siddhant",
                "Graham Neubig"
            ],
            "abstract": "Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) and\nXLMR (Conneau et al., 2020) have proven to be impressively effective at\nenabling transfer-learning of NLP systems from high-resource languages to\nlow-resource languages. This success comes despite the fact that there is no\nexplicit objective to align the contextual embeddings of words/sentences with\nsimilar meanings across languages together in the same space. In this paper, we\npresent a new method for learning multilingual encoders, AMBER (Aligned\nMultilingual Bidirectional EncodeR). AMBER is trained on additional parallel\ndata using two explicit alignment objectives that align the multilingual\nrepresentations at different granularities. We conduct experiments on zero-shot\ncross-lingual transfer learning for different tasks including sequence tagging,\nsentence retrieval and sentence classification. Experimental results show that\nAMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to\n27.3 average accuracy on retrieval over the XLMR-large model which has 3.2x the\nparameters of AMBER. Our code and models are available at\nhttp://github.com/junjiehu/amber.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08178v1",
            "date": "2020-10-16 05:50:00+00:00",
            "title": "Generating Diverse Translation from Model Distribution with Dropout",
            "authors": [
                "Xuanfu Wu",
                "Yang Feng",
                "Chenze Shao"
            ],
            "abstract": "Despite the improvement of translation quality, neural machine translation\n(NMT) often suffers from the lack of diversity in its generation. In this\npaper, we propose to generate diverse translations by deriving a large number\nof possible models with Bayesian modelling and sampling models from them for\ninference. The possible models are obtained by applying concrete dropout to the\nNMT model and each of them has specific confidence for its prediction, which\ncorresponds to a posterior model distribution under specific training data in\nthe principle of Bayesian modeling. With variational inference, the posterior\nmodel distribution can be approximated with a variational distribution, from\nwhich the final models for inference are sampled. We conducted experiments on\nChinese-English and English-German translation tasks and the results shows that\nour method makes a better trade-off between diversity and accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08185v1",
            "date": "2020-10-16 06:25:48+00:00",
            "title": "DiDi's Machine Translation System for WMT2020",
            "authors": [
                "Tanfang Chen",
                "Weiwei Wang",
                "Wenyang Wei",
                "Xing Shi",
                "Xiangang Li",
                "Jieping Ye",
                "Kevin Knight"
            ],
            "abstract": "This paper describes DiDi AI Labs' submission to the WMT2020 news translation\nshared task. We participate in the translation direction of Chinese->English.\nIn this direction, we use the Transformer as our baseline model, and integrate\nseveral techniques for model enhancement, including data filtering, data\nselection, back-translation, fine-tuning, model ensembling, and re-ranking. As\na result, our submission achieves a BLEU score of $36.6$ in Chinese->English.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08191v2",
            "date": "2020-10-16 06:54:05+00:00",
            "title": "RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering",
            "authors": [
                "Yingqi Qu",
                "Yuchen Ding",
                "Jing Liu",
                "Kai Liu",
                "Ruiyang Ren",
                "Wayne Xin Zhao",
                "Daxiang Dong",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "In open-domain question answering, dense passage retrieval has become a new\nparadigm to retrieve relevant passages for finding answers. Typically, the\ndual-encoder architecture is adopted to learn dense representations of\nquestions and passages for semantic matching. However, it is difficult to\neffectively train a dual-encoder due to the challenges including the\ndiscrepancy between training and inference, the existence of unlabeled\npositives and limited training data. To address these challenges, we propose an\noptimized training approach, called RocketQA, to improving dense passage\nretrieval. We make three major technical contributions in RocketQA, namely\ncross-batch negatives, denoised hard negatives and data augmentation. The\nexperiment results show that RocketQA significantly outperforms previous\nstate-of-the-art models on both MSMARCO and Natural Questions. We also conduct\nextensive experiments to examine the effectiveness of the three strategies in\nRocketQA. Besides, we demonstrate that the performance of end-to-end QA can be\nimproved based on our RocketQA retriever.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08200v1",
            "date": "2020-10-16 07:12:53+00:00",
            "title": "Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning",
            "authors": [
                "Wanyun Cui",
                "Guangyu Zheng",
                "Wei Wang"
            ],
            "abstract": "We propose to solve the natural language inference problem without any\nsupervision from the inference labels via task-agnostic multimodal pretraining.\nAlthough recent studies of multimodal self-supervised learning also represent\nthe linguistic and visual context, their encoders for different modalities are\ncoupled. Thus they cannot incorporate visual information when encoding plain\ntext alone. In this paper, we propose Multimodal Aligned Contrastive Decoupled\nlearning (MACD) network. MACD forces the decoupled text encoder to represent\nthe visual information via contrastive learning. Therefore, it embeds visual\nknowledge even for plain text inference. We conducted comprehensive experiments\nover plain text inference datasets (i.e. SNLI and STS-B). The unsupervised MACD\neven outperforms the fully-supervised BiLSTM and BiLSTM+ELMO on STS-B.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08265v1",
            "date": "2020-10-16 09:37:27+00:00",
            "title": "Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation",
            "authors": [
                "Qiang Wang",
                "Tong Xiao",
                "Jingbo Zhu"
            ],
            "abstract": "The standard neural machine translation model can only decode with the same\ndepth configuration as training. Restricted by this feature, we have to deploy\nmodels of various sizes to maintain the same translation latency, because the\nhardware conditions on different terminal devices (e.g., mobile phones) may\nvary greatly. Such individual training leads to increased model maintenance\ncosts and slower model iterations, especially for the industry. In this work,\nwe propose to use multi-task learning to train a flexible depth model that can\nadapt to different depth configurations during inference. Experimental results\nshow that our approach can simultaneously support decoding in 24 depth\nconfigurations and is superior to the individual training and another flexible\ndepth model training method -- LayerDrop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08346v1",
            "date": "2020-10-16 12:21:01+00:00",
            "title": "From Talk to Action with Accountability: Monitoring the Public Discussion of Finnish Decision-Makers with Deep Neural Networks and Topic Modelling",
            "authors": [
                "Vili H\u00e4t\u00f6nen",
                "Fiona Melzer"
            ],
            "abstract": "Decades of research on climate have provided a consensus that human activity\nhas changed the climate and we are currently heading into a climate crisis.\nMany tools and methods, some of which utilize machine learning, have been\ndeveloped to monitor, evaluate, and predict the changing climate and its\neffects on societies. However, the mere existence of tools and increased\nawareness have not led to swift action to reduce emissions and mitigate climate\nchange. Politicians and other policy makers lack the initiative to move from\ntalking about the climate to concrete climate action. In this work, we\ncontribute to the efforts of holding decision makers accountable by describing\na system which digests politicians' speeches and statements into a topic\nsummary. We propose a multi-source hybrid latent Dirichlet allocation model\nwhich can process the large number of publicly available reports, social media\nposts, speeches, and other documents of Finnish politicians, providing\ntransparency and accountability towards the general public.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08412v1",
            "date": "2020-10-06 16:54:08+00:00",
            "title": "Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications",
            "authors": [
                "Matthew Khoury",
                "Rumen Dangovski",
                "Longwu Ou",
                "Preslav Nakov",
                "Yichen Shen",
                "Li Jing"
            ],
            "abstract": "Deep neural networks have become the standard approach to building reliable\nNatural Language Processing (NLP) applications, ranging from Neural Machine\nTranslation (NMT) to dialogue systems. However, improving accuracy by\nincreasing the model size requires a large number of hardware computations,\nwhich can slow down NLP applications significantly at inference time. To\naddress this issue, we propose a novel vector-vector-matrix architecture\n(VVMA), which greatly reduces the latency at inference time for NMT. This\narchitecture takes advantage of specialized hardware that has low-latency\nvector-vector operations and higher-latency vector-matrix operations. It also\nreduces the number of parameters and FLOPs for virtually all models that rely\non efficient matrix multipliers without significantly impacting accuracy. We\npresent empirical results suggesting that our framework can reduce the latency\nof sequence-to-sequence and Transformer models used for NMT by a factor of\nfour. Finally, we show evidence suggesting that our VVMA extends to other\ndomains, and we discuss novel hardware for its efficient use.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08433v2",
            "date": "2020-10-16 14:54:29+00:00",
            "title": "An efficient representation of chronological events in medical texts",
            "authors": [
                "Andrey Kormilitzin",
                "Nemanja Vaci",
                "Qiang Liu",
                "Hao Ni",
                "Goran Nenadic",
                "Alejo Nevado-Holgado"
            ],
            "abstract": "In this work we addressed the problem of capturing sequential information\ncontained in longitudinal electronic health records (EHRs). Clinical notes,\nwhich is a particular type of EHR data, are a rich source of information and\npractitioners often develop clever solutions how to maximise the sequential\ninformation contained in free-texts. We proposed a systematic methodology for\nlearning from chronological events available in clinical notes. The proposed\nmethodological {\\it path signature} framework creates a non-parametric\nhierarchical representation of sequential events of any type and can be used as\nfeatures for downstream statistical learning tasks. The methodology was\ndeveloped and externally validated using the largest in the UK secondary care\nmental health EHR data on a specific task of predicting survival risk of\npatients diagnosed with Alzheimer's disease. The signature-based model was\ncompared to a common survival random forest model. Our results showed a\n15.4$\\%$ increase of risk prediction AUC at the time point of 20 months after\nthe first admission to a specialist memory clinic and the signature method\noutperformed the baseline mixed-effects model by 13.2 $\\%$.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08525v1",
            "date": "2020-10-16 17:35:40+00:00",
            "title": "Analogous Process Structure Induction for Sub-event Sequence Prediction",
            "authors": [
                "Hongming Zhang",
                "Muhao Chen",
                "Haoyu Wang",
                "Yangqiu Song",
                "Dan Roth"
            ],
            "abstract": "Computational and cognitive studies of event understanding suggest that\nidentifying, comprehending, and predicting events depend on having structured\nrepresentations of a sequence of events and on conceptualizing (abstracting)\nits components into (soft) event categories. Thus, knowledge about a known\nprocess such as \"buying a car\" can be used in the context of a new but\nanalogous process such as \"buying a house\". Nevertheless, most event\nunderstanding work in NLP is still at the ground level and does not consider\nabstraction. In this paper, we propose an Analogous Process Structure Induction\nAPSI framework, which leverages analogies among processes and conceptualization\nof sub-event instances to predict the whole sub-event sequence of previously\nunseen open-domain processes. As our experiments and analysis indicate, APSI\nsupports the generation of meaningful sub-event sequences for unseen processes\nand can help predict missing events.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.08606v1",
            "date": "2020-10-16 19:55:32+00:00",
            "title": "Evaluating Attribution Methods using White-Box LSTMs",
            "authors": [
                "Yiding Hao"
            ],
            "abstract": "Interpretability methods for neural networks are difficult to evaluate\nbecause we do not understand the black-box models typically used to test them.\nThis paper proposes a framework in which interpretability methods are evaluated\nusing manually constructed networks, which we call white-box networks, whose\nbehavior is understood a priori. We evaluate five methods for producing\nattribution heatmaps by applying them to white-box LSTM classifiers for tasks\nbased on formal languages. Although our white-box classifiers solve their tasks\nperfectly and transparently, we find that all five attribution methods fail to\nproduce the expected model explanations.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.08684v2",
            "date": "2020-10-17 01:03:06+00:00",
            "title": "Example-Driven Intent Prediction with Observers",
            "authors": [
                "Shikib Mehri",
                "Mihail Eric"
            ],
            "abstract": "A key challenge of dialog systems research is to effectively and efficiently\nadapt to new domains. A scalable paradigm for adaptation necessitates the\ndevelopment of generalizable models that perform well in few-shot settings. In\nthis paper, we focus on the intent classification problem which aims to\nidentify user intents given utterances addressed to the dialog system. We\npropose two approaches for improving the generalizability of utterance\nclassification models: (1) observers and (2) example-driven training. Prior\nwork has shown that BERT-like models tend to attribute a significant amount of\nattention to the [CLS] token, which we hypothesize results in diluted\nrepresentations. Observers are tokens that are not attended to, and are an\nalternative to the [CLS] token as a semantic representation of utterances.\nExample-driven training learns to classify utterances by comparing to examples,\nthereby using the underlying encoder as a sentence similarity model. These\nmethods are complementary; improving the representation through observers\nallows the example-driven model to better measure sentence similarities. When\ncombined, the proposed methods attain state-of-the-art results on three intent\nprediction datasets (\\textsc{banking77}, \\textsc{clinc150}, \\textsc{hwu64}) in\nboth the full data and few-shot (10 examples per intent) settings. Furthermore,\nwe demonstrate that the proposed approach can transfer to new intents and\nacross datasets without any additional training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08708v1",
            "date": "2020-10-17 03:37:16+00:00",
            "title": "Answer-checking in Context: A Multi-modal FullyAttention Network for Visual Question Answering",
            "authors": [
                "Hantao Huang",
                "Tao Han",
                "Wei Han",
                "Deep Yap",
                "Cheng-Ming Chiang"
            ],
            "abstract": "Visual Question Answering (VQA) is challenging due to the complex cross-modal\nrelations. It has received extensive attention from the research community.\nFrom the human perspective, to answer a visual question, one needs to read the\nquestion and then refer to the image to generate an answer. This answer will\nthen be checked against the question and image again for the final\nconfirmation. In this paper, we mimic this process and propose a fully\nattention based VQA architecture. Moreover, an answer-checking module is\nproposed to perform a unified attention on the jointly answer, question and\nimage representation to update the answer. This mimics the human answer\nchecking process to consider the answer in the context. With answer-checking\nmodules and transferred BERT layers, our model achieves the state-of-the-art\naccuracy 71.57\\% using fewer parameters on VQA-v2.0 test-standard split.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.08712v2",
            "date": "2020-10-17 04:24:16+00:00",
            "title": "Factual Error Correction for Abstractive Summarization Models",
            "authors": [
                "Meng Cao",
                "Yue Dong",
                "Jiapeng Wu",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Neural abstractive summarization systems have achieved promising progress,\nthanks to the availability of large-scale datasets and models pre-trained with\nself-supervised methods. However, ensuring the factual consistency of the\ngenerated summaries for abstractive summarization systems is a challenge. We\npropose a post-editing corrector module to address this issue by identifying\nand correcting factual errors in generated summaries. The neural corrector\nmodel is pre-trained on artificial examples that are created by applying a\nseries of heuristic transformations on reference summaries. These\ntransformations are inspired by an error analysis of state-of-the-art\nsummarization model outputs. Experimental results show that our model is able\nto correct factual errors in summaries generated by other neural summarization\nmodels and outperforms previous models on factual consistency evaluation on the\nCNN/DailyMail dataset. We also find that transferring from artificial error\ncorrection to downstream settings is still very challenging.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08777v1",
            "date": "2020-10-17 12:29:09+00:00",
            "title": "Active Testing: An Unbiased Evaluation Method for Distantly Supervised Relation Extraction",
            "authors": [
                "Pengshuai Li",
                "Xinsong Zhang",
                "Weijia Jia",
                "Wei Zhao"
            ],
            "abstract": "Distant supervision has been a widely used method for neural relation\nextraction for its convenience of automatically labeling datasets. However,\nexisting works on distantly supervised relation extraction suffer from the low\nquality of test set, which leads to considerable biased performance evaluation.\nThese biases not only result in unfair evaluations but also mislead the\noptimization of neural relation extraction. To mitigate this problem, we\npropose a novel evaluation method named active testing through utilizing both\nthe noisy test set and a few manual annotations. Experiments on a widely used\nbenchmark show that our proposed approach can yield approximately unbiased\nevaluations for distantly supervised relation extractors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08883v1",
            "date": "2020-10-17 22:59:34+00:00",
            "title": "Question Answering over Knowledge Base using Language Model Embeddings",
            "authors": [
                "Sai Sharath Japa",
                "Rekabdar Banafsheh"
            ],
            "abstract": "Knowledge Base, represents facts about the world, often in some form of\nsubsumption ontology, rather than implicitly, embedded in procedural code, the\nway a conventional computer program does. While there is a rapid growth in\nknowledge bases, it poses a challenge of retrieving information from them.\nKnowledge Base Question Answering is one of the promising approaches for\nextracting substantial knowledge from Knowledge Bases. Unlike web search,\nQuestion Answering over a knowledge base gives accurate and concise results,\nprovided that natural language questions can be understood and mapped precisely\nto an answer in the knowledge base. However, some of the existing\nembedding-based methods for knowledge base question answering systems ignore\nthe subtle correlation between the question and the Knowledge Base (e.g.,\nentity types, relation paths, and context) and suffer from the Out Of\nVocabulary problem. In this paper, we focused on using a pre-trained language\nmodel for the Knowledge Base Question Answering task. Firstly, we used Bert\nbase uncased for the initial experiments. We further fine-tuned these\nembeddings with a two-way attention mechanism from the knowledge base to the\nasked question and from the asked question to the knowledge base answer\naspects. Our method is based on a simple Convolutional Neural Network\narchitecture with a Multi-Head Attention mechanism to represent the asked\nquestion dynamically in multiple aspects. Our experimental results show the\neffectiveness and the superiority of the Bert pre-trained language model\nembeddings for question answering systems on knowledge bases over other\nwell-known embedding methods.",
            "journal ref": "2020 International Joint Conference on Neural Networks (IJCNN)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.08983v1",
            "date": "2020-10-18 13:33:49+00:00",
            "title": "Towards Interpreting BERT for Reading Comprehension Based QA",
            "authors": [
                "Sahana Ramnath",
                "Preksha Nema",
                "Deep Sahni",
                "Mitesh M. Khapra"
            ],
            "abstract": "BERT and its variants have achieved state-of-the-art performance in various\nNLP tasks. Since then, various works have been proposed to analyze the\nlinguistic information being captured in BERT. However, the current works do\nnot provide an insight into how BERT is able to achieve near human-level\nperformance on the task of Reading Comprehension based Question Answering. In\nthis work, we attempt to interpret BERT for RCQA. Since BERT layers do not have\npredefined roles, we define a layer's role or functionality using Integrated\nGradients. Based on the defined roles, we perform a preliminary analysis across\nall layers. We observed that the initial layers focus on query-passage\ninteraction, whereas later layers focus more on contextual understanding and\nenhancing the answer prediction. Specifically for quantifier questions (how\nmuch/how many), we notice that BERT focuses on confusing words (i.e., on other\nnumerical quantities in the passage) in the later layers, but still manages to\npredict the answer correctly. The fine-tuning and analysis scripts will be\npublicly available at https://github.com/iitmnlp/BERT-Analysis-RCQA .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09030v1",
            "date": "2020-10-18 16:55:25+00:00",
            "title": "Explaining and Improving Model Behavior with k Nearest Neighbor Representations",
            "authors": [
                "Nazneen Fatema Rajani",
                "Ben Krause",
                "Wengpeng Yin",
                "Tong Niu",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Interpretability techniques in NLP have mainly focused on understanding\nindividual predictions using attention visualization or gradient-based saliency\nmaps over tokens. We propose using k nearest neighbor (kNN) representations to\nidentify training examples responsible for a model's predictions and obtain a\ncorpus-level understanding of the model's behavior. Apart from\ninterpretability, we show that kNN representations are effective at uncovering\nlearned spurious associations, identifying mislabeled examples, and improving\nthe fine-tuned model's performance. We focus on Natural Language Inference\n(NLI) as a case study and experiment with multiple datasets. Our method deploys\nbackoff to kNN for BERT and RoBERTa on examples with low model confidence\nwithout any update to the model parameters. Our results indicate that the kNN\napproach makes the finetuned model more robust to adversarial inputs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09142v2",
            "date": "2020-10-18 23:57:33+00:00",
            "title": "Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model",
            "authors": [
                "Jason Obeid",
                "Enamul Hoque"
            ],
            "abstract": "Information visualizations such as bar charts and line charts are very\npopular for exploring data and communicating insights. Interpreting and making\nsense of such visualizations can be challenging for some people, such as those\nwho are visually impaired or have low visualization literacy. In this work, we\nintroduce a new dataset and present a neural model for automatically generating\nnatural language summaries for charts. The generated summaries provide an\ninterpretation of the chart and convey the key insights found within that\nchart. Our neural model is developed by extending the state-of-the-art model\nfor the data-to-text generation task, which utilizes a transformer-based\nencoder-decoder architecture. We found that our approach outperforms the base\nmodel on a content selection metric by a wide margin (55.42% vs. 8.49%) and\ngenerates more informative, concise, and coherent summaries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09189v1",
            "date": "2020-10-19 03:28:27+00:00",
            "title": "Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning",
            "authors": [
                "Ye Liu",
                "Sheng Zhang",
                "Rui Song",
                "Suo Feng",
                "Yanghua Xiao"
            ],
            "abstract": "Open attribute value extraction for emerging entities is an important but\nchallenging task. A lot of previous works formulate the problem as a\n\\textit{question-answering} (QA) task. While the collections of articles from\nweb corpus provide updated information about the emerging entities, the\nretrieved texts can be noisy, irrelevant, thus leading to inaccurate answers.\nEffectively filtering out noisy articles as well as bad answers is the key to\nimproving extraction accuracy. Knowledge graph (KG), which contains rich, well\norganized information about entities, provides a good resource to address the\nchallenge. In this work, we propose a knowledge-guided reinforcement learning\n(RL) framework for open attribute value extraction. Informed by relevant\nknowledge in KG, we trained a deep Q-network to sequentially compare extracted\nanswers to improve extraction accuracy. The proposed framework is applicable to\ndifferent information extraction system. Our experimental results show that our\nmethod outperforms the baselines by 16.5 - 27.8\\%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09240v2",
            "date": "2020-10-19 06:15:36+00:00",
            "title": "Multi-hop Question Generation with Graph Convolutional Network",
            "authors": [
                "Dan Su",
                "Yan Xu",
                "Wenliang Dai",
                "Ziwei Ji",
                "Tiezheng Yu",
                "Pascale Fung"
            ],
            "abstract": "Multi-hop Question Generation (QG) aims to generate answer-related questions\nby aggregating and reasoning over multiple scattered evidence from different\nparagraphs. It is a more challenging yet under-explored task compared to\nconventional single-hop QG, where the questions are generated from the sentence\ncontaining the answer or nearby sentences in the same paragraph without complex\nreasoning. To address the additional challenges in multi-hop QG, we propose\nMulti-Hop Encoding Fusion Network for Question Generation (MulQG), which does\ncontext encoding in multiple hops with Graph Convolutional Network and encoding\nfusion via an Encoder Reasoning Gate. To the best of our knowledge, we are the\nfirst to tackle the challenge of multi-hop reasoning over paragraphs without\nany sentence-level information. Empirical results on HotpotQA dataset\ndemonstrate the effectiveness of our method, in comparison with baselines on\nautomatic evaluation metrics. Moreover, from the human evaluation, our proposed\nmodel is able to generate fluent questions with high completeness and\noutperforms the strongest baseline by 20.8% in the multi-hop evaluation. The\ncode is publicly available at\nhttps://github.com/HLTCHKUST/MulQG}{https://github.com/HLTCHKUST/MulQG .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09254v1",
            "date": "2020-10-19 06:48:40+00:00",
            "title": "Query-aware Tip Generation for Vertical Search",
            "authors": [
                "Yang Yang",
                "Junmei Hao",
                "Canjia Li",
                "Zili Wang",
                "Jingang Wang",
                "Fuzheng Zhang",
                "Rao Fu",
                "Peixu Hou",
                "Gong Zhang",
                "Zhongyuan Wang"
            ],
            "abstract": "As a concise form of user reviews, tips have unique advantages to explain the\nsearch results, assist users' decision making, and further improve user\nexperience in vertical search scenarios. Existing work on tip generation does\nnot take query into consideration, which limits the impact of tips in search\nscenarios. To address this issue, this paper proposes a query-aware tip\ngeneration framework, integrating query information into encoding and\nsubsequent decoding processes. Two specific adaptations of Transformer and\nRecurrent Neural Network (RNN) are proposed. For Transformer, the query impact\nis incorporated into the self-attention computation of both the encoder and the\ndecoder. As for RNN, the query-aware encoder adopts a selective network to\ndistill query-relevant information from the review, while the query-aware\ndecoder integrates the query information into the attention computation during\ndecoding. The framework consistently outperforms the competing methods on both\npublic and real-world industrial datasets. Last but not least, online\ndeployment experiments on Dianping demonstrate the advantage of the proposed\nframework for tip generation as well as its online business values.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09270v1",
            "date": "2020-10-19 07:27:15+00:00",
            "title": "Global Attention for Name Tagging",
            "authors": [
                "Boliang Zhang",
                "Spencer Whitehead",
                "Lifu Huang",
                "Heng Ji"
            ],
            "abstract": "Many name tagging approaches use local contextual information with much\nsuccess, but fail when the local context is ambiguous or limited. We present a\nnew framework to improve name tagging by utilizing local, document-level, and\ncorpus-level contextual information. We retrieve document-level context from\nother sentences within the same document and corpus-level context from\nsentences in other topically related documents. We propose a model that learns\nto incorporate document-level and corpus-level contextual information alongside\nlocal contextual information via global attentions, which dynamically weight\ntheir respective contextual information, and gating mechanisms, which determine\nthe influence of this information. Extensive experiments on benchmark datasets\nshow the effectiveness of our approach, which achieves state-of-the-art results\nfor Dutch, German, and Spanish on the CoNLL-2002 and CoNLL-2003 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09313v1",
            "date": "2020-10-19 08:46:30+00:00",
            "title": "BERTnesia: Investigating the capture and forgetting of knowledge in BERT",
            "authors": [
                "Jonas Wallat",
                "Jaspreet Singh",
                "Avishek Anand"
            ],
            "abstract": "Probing complex language models has recently revealed several insights into\nlinguistic and semantic patterns found in the learned representations. In this\npaper, we probe BERT specifically to understand and measure the relational\nknowledge it captures. We utilize knowledge base completion tasks to probe\nevery layer of pre-trained as well as fine-tuned BERT (ranking, question\nanswering, NER). Our findings show that knowledge is not just contained in\nBERT's final layers. Intermediate layers contribute a significant amount\n(17-60%) to the total knowledge found. Probing intermediate layers also reveals\nhow different types of knowledge emerge at varying rates. When BERT is\nfine-tuned, relational knowledge is forgotten but the extent of forgetting is\nimpacted by the fine-tuning objective but not the size of the dataset. We found\nthat ranking models forget the least and retain more knowledge in their final\nlayer. We release our code on github to repeat the experiments.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09366v1",
            "date": "2020-10-19 10:22:16+00:00",
            "title": "Understanding Unnatural Questions Improves Reasoning over Text",
            "authors": [
                "Xiao-Yu Guo",
                "Yuan-Fang Li",
                "Gholamreza Haffari"
            ],
            "abstract": "Complex question answering (CQA) over raw text is a challenging task. A\nprominent approach to this task is based on the programmer-interpreter\nframework, where the programmer maps the question into a sequence of reasoning\nactions which is then executed on the raw text by the interpreter. Learning an\neffective CQA model requires large amounts of human-annotated data,consisting\nof the ground-truth sequence of reasoning actions, which is time-consuming and\nexpensive to collect at scale. In this paper, we address the challenge of\nlearning a high-quality programmer (parser) by projecting natural\nhuman-generated questions into unnatural machine-generated questions which are\nmore convenient to parse. We firstly generate synthetic (question,action\nsequence) pairs by a data generator, and train a semantic parser that\nassociates synthetic questions with their corresponding action sequences. To\ncapture the diversity when applied tonatural questions, we learn a projection\nmodel to map natural questions into their most similar unnatural questions for\nwhich the parser can work well. Without any natural training data, our\nprojection model provides high-quality action sequences for the CQA task.\nExperimental results show that the QA model trained exclusively with synthetic\ndata generated by our method outperforms its state-of-the-art counterpart\ntrained on human-labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09402v1",
            "date": "2020-10-19 11:51:04+00:00",
            "title": "Revisiting Modularized Multilingual NMT to Meet Industrial Demands",
            "authors": [
                "Sungwon Lyu",
                "Bokyung Son",
                "Kichang Yang",
                "Jaekyoung Bae"
            ],
            "abstract": "The complete sharing of parameters for multilingual translation (1-1) has\nbeen the mainstream approach in current research. However, degraded performance\ndue to the capacity bottleneck and low maintainability hinders its extensive\nadoption in industries. In this study, we revisit the multilingual neural\nmachine translation model that only share modules among the same languages (M2)\nas a practical alternative to 1-1 to satisfy industrial requirements. Through\ncomprehensive experiments, we identify the benefits of multi-way training and\ndemonstrate that the M2 can enjoy these benefits without suffering from the\ncapacity bottleneck. Furthermore, the interlingual space of the M2 allows\nconvenient modification of the model. By leveraging trained modules, we find\nthat incrementally added modules exhibit better performance than singly trained\nmodels. The zero-shot performance of the added modules is even comparable to\nsupervised models. Our findings suggest that the M2 can be a competent\ncandidate for multilingual translation in industries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09413v2",
            "date": "2020-10-19 12:21:38+00:00",
            "title": "Image Captioning with Visual Object Representations Grounded in the Textual Modality",
            "authors": [
                "Du\u0161an Vari\u0161",
                "Katsuhito Sudoh",
                "Satoshi Nakamura"
            ],
            "abstract": "We present our work in progress exploring the possibilities of a shared\nembedding space between textual and visual modality. Leveraging the textual\nnature of object detection labels and the hypothetical expressiveness of\nextracted visual object representations, we propose an approach opposite to the\ncurrent trend, grounding of the representations in the word embedding space of\nthe captioning system instead of grounding words or sentences in their\nassociated images. Based on the previous work, we apply additional grounding\nlosses to the image captioning training objective aiming to force visual object\nrepresentations to create more heterogeneous clusters based on their class\nlabel and copy a semantic structure of the word embedding space. In addition,\nwe provide an analysis of the learned object vector space projection and its\nimpact on the IC system performance. With only slight change in performance,\ngrounded models reach the stopping criterion during training faster than the\nunconstrained model, needing about two to three times less training updates.\nAdditionally, an improvement in structural correlation between the word\nembeddings and both original and projected object vectors suggests that the\ngrounding is actually mutual.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.09482v1",
            "date": "2020-10-19 13:23:12+00:00",
            "title": "Diving Deep into Context-Aware Neural Machine Translation",
            "authors": [
                "Jingjing Huo",
                "Christian Herold",
                "Yingbo Gao",
                "Leonard Dahlmann",
                "Shahram Khadivi",
                "Hermann Ney"
            ],
            "abstract": "Context-aware neural machine translation (NMT) is a promising direction to\nimprove the translation quality by making use of the additional context, e.g.,\ndocument-level translation, or having meta-information. Although there exist\nvarious architectures and analyses, the effectiveness of different\ncontext-aware NMT models is not well explored yet. This paper analyzes the\nperformance of document-level NMT models on four diverse domains with a varied\namount of parallel document-level bilingual data. We conduct a comprehensive\nset of experiments to investigate the impact of document-level NMT. We find\nthat there is no single best approach to document-level NMT, but rather that\ndifferent architectures come out on top on different tasks. Looking at\ntask-specific problems, such as pronoun resolution or headline translation, we\nfind improvements in the context-aware systems, even in cases where the\ncorpus-level metrics like BLEU show no significant improvement. We also show\nthat document-level back-translation significantly helps to compensate for the\nlack of document-level bi-texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09522v2",
            "date": "2020-10-19 13:55:10+00:00",
            "title": "Multimodal Research in Vision and Language: A Review of Current and Emerging Trends",
            "authors": [
                "Shagun Uppal",
                "Sarthak Bhagat",
                "Devamanyu Hazarika",
                "Navonil Majumdar",
                "Soujanya Poria",
                "Roger Zimmermann",
                "Amir Zadeh"
            ],
            "abstract": "Deep Learning and its applications have cascaded impactful research and\ndevelopment with a diverse range of modalities present in the real-world data.\nMore recently, this has enhanced research interests in the intersection of the\nVision and Language arena with its numerous applications and fast-paced growth.\nIn this paper, we present a detailed overview of the latest trends in research\npertaining to visual and language modalities. We look at its applications in\ntheir task formulations and how to solve various problems related to semantic\nperception and content generation. We also address task-specific trends, along\nwith their evaluation strategies and upcoming challenges. Moreover, we shed\nsome light on multi-disciplinary patterns and insights that have emerged in the\nrecent past, directing this field towards more modular and transparent\nintelligent systems. This survey identifies key trends gravitating recent\nliterature in VisLang research and attempts to unearth directions that the\nfield is heading towards.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.09535v2",
            "date": "2020-10-19 14:09:17+00:00",
            "title": "Cold-start Active Learning through Self-supervised Language Modeling",
            "authors": [
                "Michelle Yuan",
                "Hsuan-Tien Lin",
                "Jordan Boyd-Graber"
            ],
            "abstract": "Active learning strives to reduce annotation costs by choosing the most\ncritical examples to label. Typically, the active learning strategy is\ncontingent on the classification model. For instance, uncertainty sampling\ndepends on poorly calibrated model confidence scores. In the cold-start\nsetting, active learning is impractical because of model instability and data\nscarcity. Fortunately, modern NLP provides an additional source of information:\npre-trained language models. The pre-training loss can find examples that\nsurprise the model and should be labeled for efficient fine-tuning. Therefore,\nwe treat the language modeling loss as a proxy for classification uncertainty.\nWith BERT, we develop a simple strategy based on the masked language modeling\nloss that minimizes labeling costs for text classification. Compared to other\nbaselines, our approach reaches higher accuracy within less sampling iterations\nand computation time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09600v2",
            "date": "2020-10-19 15:30:51+00:00",
            "title": "Drug Repurposing for COVID-19 via Knowledge Graph Completion",
            "authors": [
                "Rui Zhang",
                "Dimitar Hristovski",
                "Dalton Schutte",
                "Andrej Kastrin",
                "Marcelo Fiszman",
                "Halil Kilicoglu"
            ],
            "abstract": "Objective: To discover candidate drugs to repurpose for COVID-19 using\nliterature-derived knowledge and knowledge graph completion methods. Methods:\nWe propose a novel, integrative, and neural network-based literature-based\ndiscovery (LBD) approach to identify drug candidates from both PubMed and\nCOVID-19-focused research literature. Our approach relies on semantic triples\nextracted using SemRep (via SemMedDB). We identified an informative subset of\nsemantic triples using filtering rules and an accuracy classifier developed on\na BERT variant, and used this subset to construct a knowledge graph. Five SOTA,\nneural knowledge graph completion algorithms were used to predict drug\nrepurposing candidates. The models were trained and assessed using a time\nslicing approach and the predicted drugs were compared with a list of drugs\nreported in the literature and evaluated in clinical trials. These models were\ncomplemented by a discovery pattern-based approach. Results: Accuracy\nclassifier based on PubMedBERT achieved the best performance (F1= 0.854) in\nclassifying semantic predications. Among five knowledge graph completion\nmodels, TransE outperformed others (MR = 0.923, Hits@1=0.417). Some known drugs\nlinked to COVID-19 in the literature were identified, as well as some candidate\ndrugs that have not yet been studied. Discovery patterns enabled generation of\nplausible hypotheses regarding the relationships between the candidate drugs\nand COVID-19. Among them, five highly ranked and novel drugs (paclitaxel, SB\n203580, alpha 2-antiplasmin, pyrrolidine dithiocarbamate, and butylated\nhydroxytoluene) with their mechanistic explanations were further discussed.\nConclusion: We show that an LBD approach can be feasible for discovering drug\ncandidates for COVID-19, and for generating mechanistic explanations. Our\napproach can be generalized to other diseases as well as to other clinical\nquestions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09657v1",
            "date": "2020-10-19 16:56:03+00:00",
            "title": "PySBD: Pragmatic Sentence Boundary Disambiguation",
            "authors": [
                "Nipun Sadvilkar",
                "Mark Neumann"
            ],
            "abstract": "In this paper, we present a rule-based sentence boundary disambiguation\nPython package that works out-of-the-box for 22 languages. We aim to provide a\nrealistic segmenter which can provide logical sentences even when the format\nand domain of the input text is unknown. In our work, we adapt the Golden Rules\nSet (a language-specific set of sentence boundary exemplars) originally\nimplemented as a ruby gem - pragmatic_segmenter - which we ported to Python\nwith additional improvements and functionality. PySBD passes 97.92% of the\nGolden Rule Set exemplars for English, an improvement of 25% over the next best\nopen-source Python tool.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09697v2",
            "date": "2020-10-19 17:40:38+00:00",
            "title": "Parameter Norm Growth During Training of Transformers",
            "authors": [
                "William Merrill",
                "Vivek Ramanujan",
                "Yoav Goldberg",
                "Roy Schwartz",
                "Noah Smith"
            ],
            "abstract": "The capacity of neural networks like the widely adopted transformer is known\nto be very high. Evidence is emerging that they learn successfully due to\ninductive bias in the training routine, typically some variant of gradient\ndescent (GD). To better understand this bias, we study the tendency of\ntransformer parameters to grow in magnitude during training. We find, both\ntheoretically and empirically, that, in certain contexts, GD increases the\nparameter $L_2$ norm up to a threshold that itself increases with training-set\naccuracy. This means increasing training accuracy over time enables the norm to\nincrease. Empirically, we show that the norm grows continuously over\npretraining for T5 (Raffel et al., 2019). We show that pretrained T5\napproximates a semi-discretized network with saturated activation functions.\nSuch \"saturated\" networks are known to have a reduced capacity compared to the\noriginal network family that can be described in automata-theoretic terms. This\nsuggests saturation is a new characterization of an inductive bias implicit in\nGD that is of particular interest for NLP. While our experiments focus on\ntransformers, our theoretical analysis extends to other architectures with\nsimilar formal properties, such as feedforward ReLU networks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.09780v2",
            "date": "2020-10-19 18:39:30+00:00",
            "title": "Technical Question Answering across Tasks and Domains",
            "authors": [
                "Wenhao Yu",
                "Lingfei Wu",
                "Yu Deng",
                "Qingkai Zeng",
                "Ruchi Mahindru",
                "Sinem Guven",
                "Meng Jiang"
            ],
            "abstract": "Building automatic technical support system is an important yet challenge\ntask. Conceptually, to answer a user question on a technical forum, a human\nexpert has to first retrieve relevant documents, and then read them carefully\nto identify the answer snippet. Despite huge success the researchers have\nachieved in coping with general domain question answering (QA), much less\nattentions have been paid for investigating technical QA. Specifically,\nexisting methods suffer from several unique challenges (i) the question and\nanswer rarely overlaps substantially and (ii) very limited data size. In this\npaper, we propose a novel framework of deep transfer learning to effectively\naddress technical QA across tasks and domains. To this end, we present an\nadjustable joint learning approach for document retrieval and reading\ncomprehension tasks. Our experiments on the TechQA demonstrates superior\nperformance compared with state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09788v1",
            "date": "2020-10-19 19:02:34+00:00",
            "title": "Deriving Commonsense Inference Tasks from Interactive Fictions",
            "authors": [
                "Mo Yu",
                "Xiaoxiao Guo",
                "Yufei Feng",
                "Xiaodan Zhu",
                "Michael Greenspan",
                "Murray Campbell"
            ],
            "abstract": "Commonsense reasoning simulates the human ability to make presumptions about\nour physical world, and it is an indispensable cornerstone in building general\nAI systems. We propose a new commonsense reasoning dataset based on human's\ninteractive fiction game playings as human players demonstrate plentiful and\ndiverse commonsense reasoning. The new dataset mitigates several limitations of\nthe prior art. Experiments show that our task is solvable to human experts with\nsufficient commonsense knowledge but poses challenges to existing machine\nreading models, with a big performance gap of more than 30%.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.09926v1",
            "date": "2020-10-19 23:51:33+00:00",
            "title": "Explainable Automated Fact-Checking for Public Health Claims",
            "authors": [
                "Neema Kotonya",
                "Francesca Toni"
            ],
            "abstract": "Fact-checking is the task of verifying the veracity of claims by assessing\ntheir assertions against credible evidence. The vast majority of fact-checking\nstudies focus exclusively on political claims. Very little research explores\nfact-checking for other topics, specifically subject matters for which\nexpertise is required. We present the first study of explainable fact-checking\nfor claims which require specific expertise. For our case study we choose the\nsetting of public health. To support this case study we construct a new dataset\nPUBHEALTH of 11.8K claims accompanied by journalist crafted, gold standard\nexplanations (i.e., judgments) to support the fact-check labels for claims. We\nexplore two tasks: veracity prediction and explanation generation. We also\ndefine and evaluate, with humans and computationally, three coherence\nproperties of explanation quality. Our results indicate that, by training on\nin-domain data, gains can be made in explainable, automated fact-checking for\nclaims which require specific expertise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.09997v1",
            "date": "2020-10-20 04:08:03+00:00",
            "title": "Word Shape Matters: Robust Machine Translation with Visual Embedding",
            "authors": [
                "Haohan Wang",
                "Peiyan Zhang",
                "Eric P. Xing"
            ],
            "abstract": "Neural machine translation has achieved remarkable empirical performance over\nstandard benchmark datasets, yet recent evidence suggests that the models can\nstill fail easily dealing with substandard inputs such as misspelled words, To\novercome this issue, we introduce a new encoding heuristic of the input symbols\nfor character-level NLP models: it encodes the shape of each character through\nthe images depicting the letters when printed. We name this new strategy visual\nembedding and it is expected to improve the robustness of NLP models because\nhumans also process the corpus visually through printed letters, instead of\nmachinery one-hot vectors. Empirically, our method improves models' robustness\nagainst substandard inputs, even in the test scenario where the models are\ntested with the noises that are beyond what is available during the training\nphase.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10041v3",
            "date": "2020-10-20 05:41:35+00:00",
            "title": "Language Representation in Multilingual BERT and its applications to improve Cross-lingual Generalization",
            "authors": [
                "Chi-Liang Liu",
                "Tsung-Yuan Hsu",
                "Yung-Sung Chuang",
                "Chung-Yi Li",
                "Hung-yi Lee"
            ],
            "abstract": "A token embedding in multilingual BERT (m-BERT) contains both language and\nsemantic information. We find that representation of a language can be obtained\nby simply averaging the embeddings of the tokens of the language. With the\nlanguage representation, we can control the output languages of multilingual\nBERT by manipulating the token embeddings and achieve unsupervised token\ntranslation. We further propose a computationally cheap but effective approach\nto improve the cross-lingual ability of m-BERT based on the observation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10048v2",
            "date": "2020-10-20 06:02:15+00:00",
            "title": "Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training",
            "authors": [
                "Renjie Zheng",
                "Mingbo Ma",
                "Baigong Zheng",
                "Kaibo Liu",
                "Jiahong Yuan",
                "Kenneth Church",
                "Liang Huang"
            ],
            "abstract": "Simultaneous speech-to-speech translation is widely useful but extremely\nchallenging, since it needs to generate target-language speech concurrently\nwith the source-language speech, with only a few seconds delay. In addition, it\nneeds to continuously translate a stream of sentences, but all recent solutions\nmerely focus on the single-sentence scenario. As a result, current approaches\naccumulate latencies progressively when the speaker talks faster, and introduce\nunnatural pauses when the speaker talks slower. To overcome these issues, we\npropose Self-Adaptive Translation (SAT) which flexibly adjusts the length of\ntranslations to accommodate different source speech rates. At similar levels of\ntranslation quality (as measured by BLEU), our method generates more fluent\ntarget speech (as measured by the naturalness metric MOS) with substantially\nlower latency than the baseline, in both Zh <-> En directions.",
            "journal ref": "Findings of EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10156v1",
            "date": "2020-10-20 09:47:52+00:00",
            "title": "Extracting Procedural Knowledge from Technical Documents",
            "authors": [
                "Shivali Agarwal",
                "Shubham Atreja",
                "Vikas Agarwal"
            ],
            "abstract": "Procedures are an important knowledge component of documents that can be\nleveraged by cognitive assistants for automation, question-answering or driving\na conversation. It is a challenging problem to parse big dense documents like\nproduct manuals, user guides to automatically understand which parts are\ntalking about procedures and subsequently extract them. Most of the existing\nresearch has focused on extracting flows in given procedures or understanding\nthe procedures in order to answer conceptual questions. Identifying and\nextracting multiple procedures automatically from documents of diverse formats\nremains a relatively less addressed problem. In this work, we cover some of\nthis ground by -- 1) Providing insights on how structural and linguistic\nproperties of documents can be grouped to define types of procedures, 2)\nAnalyzing documents to extract the relevant linguistic and structural\nproperties, and 3) Formulating procedure identification as a classification\nproblem that leverages the features of the document derived from the above\nanalysis. We first implemented and deployed unsupervised techniques which were\nused in different use cases. Based on the evaluation in different use cases, we\nfigured out the weaknesses of the unsupervised approach. We then designed an\nimproved version which was supervised. We demonstrate that our technique is\neffective in identifying procedures from big and complex documents alike by\nachieving accuracy of 89%.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.10176v1",
            "date": "2020-10-20 10:18:20+00:00",
            "title": "Individual corpora predict fast memory retrieval during reading",
            "authors": [
                "Markus J. Hofmann",
                "Lara M\u00fcller",
                "Andre R\u00f6lke",
                "Ralph Radach",
                "Chris Biemann"
            ],
            "abstract": "The corpus, from which a predictive language model is trained, can be\nconsidered the experience of a semantic system. We recorded everyday reading of\ntwo participants for two months on a tablet, generating individual corpus\nsamples of 300/500K tokens. Then we trained word2vec models from individual\ncorpora and a 70 million-sentence newspaper corpus to obtain individual and\nnorm-based long-term memory structure. To test whether individual corpora can\nmake better predictions for a cognitive task of long-term memory retrieval, we\ngenerated stimulus materials consisting of 134 sentences with uncorrelated\nindividual and norm-based word probabilities. For the subsequent eye tracking\nstudy 1-2 months later, our regression analyses revealed that individual, but\nnot norm-corpus-based word probabilities can account for first-fixation\nduration and first-pass gaze duration. Word length additionally affected gaze\nduration and total viewing duration. The results suggest that corpora\nrepresentative for an individual's longterm memory structure can better explain\nreading performance than a norm corpus, and that recently acquired information\nis lexically accessed rapidly.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10216v1",
            "date": "2020-10-20 12:04:19+00:00",
            "title": "Simulated Chats for Task-oriented Dialog: Learning to Generate Conversations from Instructions",
            "authors": [
                "Biswesh Mohapatra",
                "Gaurav Pandey",
                "Danish Contractor",
                "Sachindra Joshi"
            ],
            "abstract": "Popular task-oriented dialog data sets such as MultiWOZ (Budzianowski et al.\n2018) are created by providing crowd-sourced workers a goal instruction,\nexpressed in natural language, that describes the task to be accomplished.\nCrowd-sourced workers play the role of a user and an agent to generate dialogs\nto accomplish tasks involving booking restaurant tables, making train\nreservations, calling a taxi etc. However, creating large crowd-sourced\ndatasets can be time consuming and expensive. To reduce the cost associated\nwith generating such dialog datasets, recent work has explored methods to\nautomatically create larger datasets from small samples.In this paper, we\npresent a data creation strategy that uses the pre-trained language model, GPT2\n(Radford et al. 2018), to simulate the interaction between crowd-sourced\nworkers by creating a user bot and an agent bot. We train the simulators using\na smaller percentage of actual crowd-generated conversations and their\ncorresponding goal instructions. We demonstrate that by using the simulated\ndata, we achieve significant improvements in both low-resource setting as well\nas in over-all task performance. To the best of our knowledge we are the first\nto present a model for generating entire conversations by simulating the\ncrowd-sourced data collection process",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10239v1",
            "date": "2020-10-20 13:03:48+00:00",
            "title": "Complete Multilingual Neural Machine Translation",
            "authors": [
                "Markus Freitag",
                "Orhan Firat"
            ],
            "abstract": "Multilingual Neural Machine Translation (MNMT) models are commonly trained on\na joint set of bilingual corpora which is acutely English-centric (i.e. English\neither as the source or target language). While direct data between two\nlanguages that are non-English is explicitly available at times, its use is not\ncommon. In this paper, we first take a step back and look at the commonly used\nbilingual corpora (WMT), and resurface the existence and importance of implicit\nstructure that existed in it: multi-way alignment across examples (the same\nsentence in more than two languages). We set out to study the use of multi-way\naligned examples to enrich the original English-centric parallel corpora. We\nreintroduce this direct parallel data from multi-way aligned corpora between\nall source and target languages. By doing so, the English-centric graph expands\ninto a complete graph, every language pair being connected. We call MNMT with\nsuch connectivity pattern complete Multilingual Neural Machine Translation\n(cMNMT) and demonstrate its utility and efficacy with a series of experiments\nand analysis. In combination with a novel training data sampling strategy that\nis conditioned on the target language only, cMNMT yields competitive\ntranslation quality for all language pairs. We further study the size effect of\nmulti-way aligned data, its transfer learning capabilities and how it eases\nadding a new language in MNMT. Finally, we stress test cMNMT at scale and\ndemonstrate that we can train a cMNMT model with up to 111*112=12,432 language\npairs that provides competitive translation quality for all language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10245v1",
            "date": "2020-10-20 13:14:57+00:00",
            "title": "Human-Paraphrased References Improve Neural Machine Translation",
            "authors": [
                "Markus Freitag",
                "George Foster",
                "David Grangier",
                "Colin Cherry"
            ],
            "abstract": "Automatic evaluation comparing candidate translations to human-generated\nparaphrases of reference translations has recently been proposed by Freitag et\nal. When used in place of original references, the paraphrased versions produce\nmetric scores that correlate better with human judgment. This effect holds for\na variety of different automatic metrics, and tends to favor natural\nformulations over more literal (translationese) ones. In this paper we compare\nthe results of performing end-to-end system development using standard and\nparaphrased references. With state-of-the-art English-German NMT components, we\nshow that tuning to paraphrased references produces a system that is\nsignificantly better according to human judgment, but 5 BLEU points worse when\ntested on standard references. Our work confirms the finding that paraphrased\nreferences yield metric scores that correlate better with human judgment, and\ndemonstrates for the first time that using these scores for system development\ncan lead to significant improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10267v2",
            "date": "2020-10-20 13:39:58+00:00",
            "title": "Text Classification of Manifestos and COVID-19 Press Briefings using BERT and Convolutional Neural Networks",
            "authors": [
                "Kakia Chatsiou"
            ],
            "abstract": "We build a sentence-level political discourse classifier using existing human\nexpert annotated corpora of political manifestos from the Manifestos Project\n(Volkens et al., 2020a) and applying them to a corpus ofCOVID-19Press Briefings\n(Chatsiou, 2020). We use manually annotated political manifestos as training\ndata to train a local topic ConvolutionalNeural Network (CNN) classifier; then\napply it to the COVID-19PressBriefings Corpus to automatically classify\nsentences in the test corpus.We report on a series of experiments with CNN\ntrained on top of pre-trained embeddings for sentence-level classification\ntasks. We show thatCNN combined with transformers like BERT outperforms CNN\ncombined with other embeddings (Word2Vec, Glove, ELMo) and that it is possible\nto use a pre-trained classifier to conduct automatic classification on\ndifferent political texts without additional training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10333v1",
            "date": "2020-10-20 14:53:22+00:00",
            "title": "Bridging the Gap between Conversational Reasoning and Interactive Recommendation",
            "authors": [
                "Wenchang Ma",
                "Ryuichi Takanobu",
                "Minghao Tu",
                "Minlie Huang"
            ],
            "abstract": "There have been growing interests in building a conversational recommender\nsystem, where the system simultaneously interacts with the user and explores\nthe user's preference throughout conversational interactions. Recommendation\nand conversation were usually treated as two separate modules with limited\ninformation exchange in existing works, which hinders the capability of both\nsystems: (1) dialog merely incorporated recommendation entities without being\nguided by an explicit recommendation-oriented policy; (2) recommendation\nutilized dialog only as a form of interaction instead of improving\nrecommendation effectively. To address the above issues, we propose a novel\nrecommender dialog model: CR-Walker. In order to view the two separate systems\nwithin a unified framework, we seek high-level mapping between hierarchical\ndialog acts and multi-hop knowledge graph reasoning. The model walks on a\nlarge-scale knowledge graph to form a reasoning tree at each turn, then mapped\nto dialog acts to guide response generation. With such a mapping mechanism as a\nbridge between recommendation and conversation, our framework maximizes the\nmutual benefit between two systems: dialog as an enhancement to recommendation\nquality and explainability, recommendation as a goal and enrichment to dialog\nsemantics. Quantitative evaluation shows that our model excels in conversation\ninformativeness and recommendation effectiveness, at the same time explainable\non the policy level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10386v1",
            "date": "2020-10-20 15:50:50+00:00",
            "title": "A Benchmark for Lease Contract Review",
            "authors": [
                "Spyretta Leivaditi",
                "Julien Rossi",
                "Evangelos Kanoulas"
            ],
            "abstract": "Extracting entities and other useful information from legal contracts is an\nimportant task whose automation can help legal professionals perform contract\nreviews more efficiently and reduce relevant risks. In this paper, we tackle\nthe problem of detecting two different types of elements that play an important\nrole in a contract review, namely entities and red flags. The latter are terms\nor sentences that indicate that there is some danger or other potentially\nproblematic situation for one or more of the signing parties. We focus on\nsupporting the review of lease agreements, a contract type that has received\nlittle attention in the legal information extraction literature, and we define\nthe types of entities and red flags needed for that task. We release a new\nbenchmark dataset of 179 lease agreement documents that we have manually\nannotated with the entities and red flags they contain, and which can be used\nto train and test relevant extraction algorithms. Finally, we release a new\nlanguage model, called ALeaseBERT, pre-trained on this dataset and fine-tuned\nfor the detection of the aforementioned elements, providing a baseline for\nfurther research",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.10418v2",
            "date": "2020-10-20 16:29:13+00:00",
            "title": "ConjNLI: Natural Language Inference Over Conjunctive Sentences",
            "authors": [
                "Swarnadeep Saha",
                "Yixin Nie",
                "Mohit Bansal"
            ],
            "abstract": "Reasoning about conjuncts in conjunctive sentences is important for a deeper\nunderstanding of conjunctions in English and also how their usages and\nsemantics differ from conjunctive and disjunctive boolean logic. Existing NLI\nstress tests do not consider non-boolean usages of conjunctions and use\ntemplates for testing such model knowledge. Hence, we introduce ConjNLI, a\nchallenge stress-test for natural language inference over conjunctive\nsentences, where the premise differs from the hypothesis by conjuncts removed,\nadded, or replaced. These sentences contain single and multiple instances of\ncoordinating conjunctions (\"and\", \"or\", \"but\", \"nor\") with quantifiers,\nnegations, and requiring diverse boolean and non-boolean inferences over\nconjuncts. We find that large-scale pre-trained language models like RoBERTa do\nnot understand conjunctive semantics well and resort to shallow heuristics to\nmake inferences over such sentences. As some initial solutions, we first\npresent an iterative adversarial fine-tuning method that uses synthetically\ncreated training data based on boolean and non-boolean heuristics. We also\npropose a direct model advancement by making RoBERTa aware of predicate\nsemantic roles. While we observe some performance gains, ConjNLI is still\nchallenging for current methods, thus encouraging interesting future work for\nbetter understanding of conjunctions. Our data and code are publicly available\nat: https://github.com/swarnaHub/ConjNLI",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10439v2",
            "date": "2020-10-20 16:48:14+00:00",
            "title": "Open Question Answering over Tables and Text",
            "authors": [
                "Wenhu Chen",
                "Ming-Wei Chang",
                "Eva Schlinger",
                "William Wang",
                "William W. Cohen"
            ],
            "abstract": "In open question answering (QA), the answer to a question is produced by\nretrieving and then analyzing documents that might contain answers to the\nquestion. Most open QA systems have considered only retrieving information from\nunstructured text. Here we consider for the first time open QA over both\ntabular and textual data and present a new large-scale dataset Open\nTable-and-Text Question Answering (OTT-QA) to evaluate performance on this\ntask. Most questions in OTT-QA require multi-hop inference across tabular data\nand unstructured text, and the evidence required to answer a question can be\ndistributed in different ways over these two types of input, making evidence\nretrieval challenging -- our baseline model using an iterative retriever and\nBERT-based reader achieves an exact match score less than 10%. We then propose\ntwo novel techniques to address the challenge of retrieving and aggregating\nevidence for OTT-QA. The first technique is to use \"early fusion\" to group\nmultiple highly relevant tabular and textual units into a fused block, which\nprovides more context for the retriever to search for. The second technique is\nto use a cross-block reader to model the cross-dependency between multiple\nretrieved evidence with global-local sparse attention. Combining these two\ntechniques improves the score significantly, to above 27%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10453v1",
            "date": "2020-10-20 17:09:35+00:00",
            "title": "Modeling Content and Context with Deep Relational Learning",
            "authors": [
                "Maria Leonor Pacheco",
                "Dan Goldwasser"
            ],
            "abstract": "Building models for realistic natural language tasks requires dealing with\nlong texts and accounting for complicated structural dependencies.\nNeural-symbolic representations have emerged as a way to combine the reasoning\ncapabilities of symbolic methods, with the expressiveness of neural networks.\nHowever, most of the existing frameworks for combining neural and symbolic\nrepresentations have been designed for classic relational learning tasks that\nwork over a universe of symbolic entities and relations. In this paper, we\npresent DRaiL, an open-source declarative framework for specifying deep\nrelational models, designed to support a variety of NLP scenarios. Our\nframework supports easy integration with expressive language encoders, and\nprovides an interface to study the interactions between representation,\ninference and learning.",
            "journal ref": "Transactions of the Association for Computational Linguistics,\n  2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10499v2",
            "date": "2020-10-20 17:53:01+00:00",
            "title": "Optimal Subarchitecture Extraction For BERT",
            "authors": [
                "Adrian de Wynter",
                "Daniel J. Perry"
            ],
            "abstract": "We extract an optimal subset of architectural parameters for the BERT\narchitecture from Devlin et al. (2018) by applying recent breakthroughs in\nalgorithms for neural architecture search. This optimal subset, which we refer\nto as \"Bort\", is demonstrably smaller, having an effective (that is, not\ncounting the embedding layer) size of $5.5\\%$ the original BERT-large\narchitecture, and $16\\%$ of the net size. Bort is also able to be pretrained in\n$288$ GPU hours, which is $1.2\\%$ of the time required to pretrain the\nhighest-performing BERT parametric architectural variant, RoBERTa-large (Liu et\nal., 2019), and about $33\\%$ of that of the world-record, in GPU hours,\nrequired to train BERT-large on the same hardware. It is also $7.9$x faster on\na CPU, as well as being better performing than other compressed variants of the\narchitecture, and some of the non-compressed variants: it obtains performance\nimprovements of between $0.3\\%$ and $31\\%$, absolute, with respect to\nBERT-large, on multiple public natural language understanding (NLU) benchmarks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10597v2",
            "date": "2020-10-20 20:13:09+00:00",
            "title": "SKATE: A Natural Language Interface for Encoding Structured Knowledge",
            "authors": [
                "Clifton McFate",
                "Aditya Kalyanpur",
                "Dave Ferrucci",
                "Andrea Bradshaw",
                "Ariel Diertani",
                "David Melville",
                "Lori Moon"
            ],
            "abstract": "In Natural Language (NL) applications, there is often a mismatch between what\nthe NL interface is capable of interpreting and what a lay user knows how to\nexpress. This work describes a novel natural language interface that reduces\nthis mismatch by refining natural language input through successive,\nautomatically generated semi-structured templates. In this paper we describe\nhow our approach, called SKATE, uses a neural semantic parser to parse NL input\nand suggest semi-structured templates, which are recursively filled to produce\nfully structured interpretations. We also show how SKATE integrates with a\nneural rule-generation model to interactively suggest and acquire commonsense\nknowledge. We provide a preliminary coverage analysis of SKATE for the task of\nstory understanding, and then describe a current business use-case of the tool\nin a specific domain: COVID-19 policy design.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10699v2",
            "date": "2020-10-21 01:22:37+00:00",
            "title": "A Weighted Heterogeneous Graph Based Dialogue System",
            "authors": [
                "Xinyan Zhao",
                "Liangwei Chen",
                "Huanhuan Chen"
            ],
            "abstract": "Knowledge based dialogue systems have attracted increasing research interest\nin diverse applications. However, for disease diagnosis, the widely used\nknowledge graph is hard to represent the symptom-symptom relations and\nsymptom-disease relations since the edges of traditional knowledge graph are\nunweighted. Most research on disease diagnosis dialogue systems highly rely on\ndata-driven methods and statistical features, lacking profound comprehension of\nsymptom-disease relations and symptom-symptom relations. To tackle this issue,\nthis work presents a weighted heterogeneous graph based dialogue system for\ndisease diagnosis. Specifically, we build a weighted heterogeneous graph based\non symptom co-occurrence and a proposed symptom frequency-inverse disease\nfrequency. Then this work proposes a graph based deep Q-network (Graph-DQN) for\ndialogue management. By combining Graph Convolutional Network (GCN) with DQN to\nlearn the embeddings of diseases and symptoms from both the structural and\nattribute information in the weighted heterogeneous graph, Graph-DQN could\ncapture the symptom-disease relations and symptom-symptom relations better.\nExperimental results show that the proposed dialogue system rivals the\nstate-of-the-art models. More importantly, the proposed dialogue system can\ncomplete the task with less dialogue turns and possess a better distinguishing\ncapability on diseases with similar symptoms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10755v1",
            "date": "2020-10-21 04:20:13+00:00",
            "title": "FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents",
            "authors": [
                "Bill Yuchen Lin",
                "Ying Sheng",
                "Nguyen Vo",
                "Sandeep Tata"
            ],
            "abstract": "Extracting structured data from HTML documents is a long-studied problem with\na broad range of applications like augmenting knowledge bases, supporting\nfaceted search, and providing domain-specific experiences for key verticals\nlike shopping and movies. Previous approaches have either required a small\nnumber of examples for each target site or relied on carefully handcrafted\nheuristics built over visual renderings of websites. In this paper, we present\na novel two-stage neural approach, named FreeDOM, which overcomes both these\nlimitations. The first stage learns a representation for each DOM node in the\npage by combining both the text and markup information. The second stage\ncaptures longer range distance and semantic relatedness using a relational\nneural network. By combining these stages, FreeDOM is able to generalize to\nunseen sites after training on a small number of seed sites from that vertical\nwithout requiring expensive hand-crafted features over visual renderings of the\npage. Through experiments on a public dataset with 8 different verticals, we\nshow that FreeDOM beats the previous state of the art by nearly 3.7 F1 points\non average without requiring features over rendered pages or expensive\nhand-crafted features.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10789v1",
            "date": "2020-10-21 07:03:20+00:00",
            "title": "ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine",
            "authors": [
                "Weizhen Qi",
                "Yeyun Gong",
                "Yu Yan",
                "Jian Jiao",
                "Bo Shao",
                "Ruofei Zhang",
                "Houqiang Li",
                "Nan Duan",
                "Ming Zhou"
            ],
            "abstract": "In a sponsored search engine, generative retrieval models are recently\nproposed to mine relevant advertisement keywords for users' input queries.\nGenerative retrieval models generate outputs token by token on a path of the\ntarget library prefix tree (Trie), which guarantees all of the generated\noutputs are legal and covered by the target library. In actual use, we found\nseveral typical problems caused by Trie-constrained searching length. In this\npaper, we analyze these problems and propose a looking ahead strategy for\ngenerative retrieval models named ProphetNet-Ads. ProphetNet-Ads improves the\nretrieval ability by directly optimizing the Trie-constrained searching space.\nWe build a dataset from a real-word sponsored search engine and carry out\nexperiments to analyze different generative retrieval models. Compared with\nTrie-based LSTM generative retrieval model proposed recently, our single model\nresult and integrated result improve the recall by 15.58\\% and 18.8\\%\nrespectively with beam size 5. Case studies further demonstrate how these\nproblems are alleviated by ProphetNet-Ads clearly.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.10813v1",
            "date": "2020-10-21 08:11:08+00:00",
            "title": "PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding",
            "authors": [
                "Zhao Jinman",
                "Shawn Zhong",
                "Xiaomin Zhang",
                "Yingyu Liang"
            ],
            "abstract": "We look into the task of \\emph{generalizing} word embeddings: given a set of\npre-trained word vectors over a finite vocabulary, the goal is to predict\nembedding vectors for out-of-vocabulary words, \\emph{without} extra contextual\ninformation. We rely solely on the spellings of words and propose a model,\nalong with an efficient algorithm, that simultaneously models subword\nsegmentation and computes subword-based compositional word embedding. We call\nthe model probabilistic bag-of-subwords (PBoS), as it applies bag-of-subwords\nfor all possible segmentations based on their likelihood. Inspections and affix\nprediction experiment show that PBoS is able to produce meaningful subword\nsegmentations and subword rankings without any source of explicit morphological\nknowledge. Word similarity and POS tagging experiments show clear advantages of\nPBoS over previous subword-level models in the quality of generated word\nembeddings across languages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10873v1",
            "date": "2020-10-21 09:58:00+00:00",
            "title": "Explaining black-box text classifiers for disease-treatment information extraction",
            "authors": [
                "Milad Moradi",
                "Matthias Samwald"
            ],
            "abstract": "Deep neural networks and other intricate Artificial Intelligence (AI) models\nhave reached high levels of accuracy on many biomedical natural language\nprocessing tasks. However, their applicability in real-world use cases may be\nlimited due to their vague inner working and decision logic. A post-hoc\nexplanation method can approximate the behavior of a black-box AI model by\nextracting relationships between feature values and outcomes. In this paper, we\nintroduce a post-hoc explanation method that utilizes confident itemsets to\napproximate the behavior of black-box classifiers for medical information\nextraction. Incorporating medical concepts and semantics into the explanation\nprocess, our explanator finds semantic relations between inputs and outputs in\ndifferent parts of the decision space of a black-box classifier. The\nexperimental results show that our explanation method can outperform\nperturbation and decision set based explanators in terms of fidelity and\ninterpretability of explanations produced for predictions on a\ndisease-treatment information extraction task.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.10874v1",
            "date": "2020-10-21 09:58:39+00:00",
            "title": "TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog",
            "authors": [
                "Erik Ekstedt",
                "Gabriel Skantze"
            ],
            "abstract": "Syntactic and pragmatic completeness is known to be important for turn-taking\nprediction, but so far machine learning models of turn-taking have used such\nlinguistic information in a limited way. In this paper, we introduce TurnGPT, a\ntransformer-based language model for predicting turn-shifts in spoken dialog.\nThe model has been trained and evaluated on a variety of written and spoken\ndialog datasets. We show that the model outperforms two baselines used in prior\nwork. We also report on an ablation study, as well as attention and gradient\nanalyses, which show that the model is able to utilize the dialog context and\npragmatic completeness for turn-taking prediction. Finally, we explore the\nmodel's potential in not only detecting, but also projecting, turn-completions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10906v4",
            "date": "2020-10-21 11:28:23+00:00",
            "title": "German's Next Language Model",
            "authors": [
                "Branden Chan",
                "Stefan Schweter",
                "Timo M\u00f6ller"
            ],
            "abstract": "In this work we present the experiments which lead to the creation of our\nBERT and ELECTRA based German language models, GBERT and GELECTRA. By varying\nthe input training data, model size, and the presence of Whole Word Masking\n(WWM) we were able to attain SoTA performance across a set of document\nclassification and named entity recognition (NER) tasks for both models of base\nand large size. We adopt an evaluation driven approach in training these models\nand our results indicate that both adding more data and utilizing WWM improve\nmodel performance. By benchmarking against existing German models, we show that\nthese models are the best German models to date. Our trained models will be\nmade publicly available to the research community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10938v1",
            "date": "2020-10-20 05:41:56+00:00",
            "title": "What makes multilingual BERT multilingual?",
            "authors": [
                "Chi-Liang Liu",
                "Tsung-Yuan Hsu",
                "Yung-Sung Chuang",
                "Hung-yi Lee"
            ],
            "abstract": "Recently, multilingual BERT works remarkably well on cross-lingual transfer\ntasks, superior to static non-contextualized word embeddings. In this work, we\nprovide an in-depth experimental study to supplement the existing literature of\ncross-lingual ability. We compare the cross-lingual ability of\nnon-contextualized and contextualized representation model with the same data.\nWe found that datasize and context window size are crucial factors to the\ntransferability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.10998v2",
            "date": "2020-10-21 13:38:04+00:00",
            "title": "Open-Domain Frame Semantic Parsing Using Transformers",
            "authors": [
                "Aditya Kalyanpur",
                "Or Biran",
                "Tom Breloff",
                "Jennifer Chu-Carroll",
                "Ariel Diertani",
                "Owen Rambow",
                "Mark Sammons"
            ],
            "abstract": "Frame semantic parsing is a complex problem which includes multiple\nunderlying subtasks. Recent approaches have employed joint learning of subtasks\n(such as predicate and argument detection), and multi-task learning of related\ntasks (such as syntactic and semantic parsing). In this paper, we explore\nmulti-task learning of all subtasks with transformer-based models. We show that\na purely generative encoder-decoder architecture handily beats the previous\nstate of the art in FrameNet 1.7 parsing, and that a mixed decoding multi-task\napproach achieves even better performance. Finally, we show that the multi-task\nmodel also outperforms recent state of the art systems for PropBank SRL parsing\non the CoNLL 2012 benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11003v1",
            "date": "2020-10-21 13:44:35+00:00",
            "title": "Unsupervised Deep Learning based Multiple Choices Question Answering: Start Learning from Basic Knowledge",
            "authors": [
                "Chi-Liang Liu",
                "Hung-yi Lee"
            ],
            "abstract": "In this paper, we study the possibility of almost unsupervised Multiple\nChoices Question Answering (MCQA). Starting from very basic knowledge, MCQA\nmodel knows that some choices have higher probabilities of being correct than\nthe others. The information, though very noisy, guides the training of an MCQA\nmodel. The proposed method is shown to outperform the baseline approaches on\nRACE and even comparable with some supervised learning approaches on MC500.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11092v1",
            "date": "2020-10-21 16:00:09+00:00",
            "title": "Stacking Neural Network Models for Automatic Short Answer Scoring",
            "authors": [
                "Rian Adam Rajagede",
                "Rochana Prih Hastuti"
            ],
            "abstract": "Automatic short answer scoring is one of the text classification problems to\nassess students' answers during exams automatically. Several challenges can\narise in making an automatic short answer scoring system, one of which is the\nquantity and quality of the data. The data labeling process is not easy because\nit requires a human annotator who is an expert in their field. Further, the\ndata imbalance process is also a challenge because the number of labels for\ncorrect answers is always much less than the wrong answers. In this paper, we\npropose the use of a stacking model based on neural network and XGBoost for\nclassification process with sentence embedding feature. We also propose to use\ndata upsampling method to handle imbalance classes and hyperparameters\noptimization algorithm to find a robust model automatically. We use Ukara 1.0\nChallenge dataset and our best model obtained an F1-score of 0.821 exceeding\nthe previous work at the same dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11125v1",
            "date": "2020-10-21 17:01:23+00:00",
            "title": "Beyond English-Centric Multilingual Machine Translation",
            "authors": [
                "Angela Fan",
                "Shruti Bhosale",
                "Holger Schwenk",
                "Zhiyi Ma",
                "Ahmed El-Kishky",
                "Siddharth Goyal",
                "Mandeep Baines",
                "Onur Celebi",
                "Guillaume Wenzek",
                "Vishrav Chaudhary",
                "Naman Goyal",
                "Tom Birch",
                "Vitaliy Liptchinsky",
                "Sergey Edunov",
                "Edouard Grave",
                "Michael Auli",
                "Armand Joulin"
            ],
            "abstract": "Existing work in translation demonstrated the potential of massively\nmultilingual machine translation by training a single model able to translate\nbetween any pair of languages. However, much of this work is English-Centric by\ntraining only on data which was translated from or to English. While this is\nsupported by large sources of training data, it does not reflect translation\nneeds worldwide. In this work, we create a true Many-to-Many multilingual\ntranslation model that can translate directly between any pair of 100\nlanguages. We build and open source a training dataset that covers thousands of\nlanguage directions with supervised data, created through large-scale mining.\nThen, we explore how to effectively increase model capacity through a\ncombination of dense scaling and language-specific sparse parameters to create\nhigh quality models. Our focus on non-English-Centric models brings gains of\nmore than 10 BLEU when directly translating between non-English directions\nwhile performing competitively to the best single systems of WMT. We\nopen-source our scripts so that others may reproduce the data, evaluation, and\nfinal M2M-100 model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11140v2",
            "date": "2020-10-21 16:56:49+00:00",
            "title": "A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation",
            "authors": [
                "Yan Zeng",
                "Jian-Yun Nie"
            ],
            "abstract": "Conditioned dialogue generation suffers from the scarcity of labeled\nresponses. In this work, we exploit labeled non-dialogue text data related to\nthe condition, which are much easier to collect. We propose a multi-task\nlearning approach to leverage both labeled dialogue and text data. The 3 tasks\njointly optimize the same pre-trained Transformer -- conditioned dialogue\ngeneration task on the labeled dialogue data, conditioned language encoding\ntask and conditioned language generation task on the labeled text data.\nExperimental results show that our approach outperforms the state-of-the-art\nmodels by leveraging the labeled texts, and it also obtains larger improvement\nin performance comparing to the previous methods to leverage text data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11170v1",
            "date": "2020-10-21 17:46:11+00:00",
            "title": "Semantic Role Labeling as Syntactic Dependency Parsing",
            "authors": [
                "Tianze Shi",
                "Igor Malioutov",
                "Ozan \u0130rsoy"
            ],
            "abstract": "We reduce the task of (span-based) PropBank-style semantic role labeling\n(SRL) to syntactic dependency parsing. Our approach is motivated by our\nempirical analysis that shows three common syntactic patterns account for over\n98% of the SRL annotations for both English and Chinese data. Based on this\nobservation, we present a conversion scheme that packs SRL annotations into\ndependency tree representations through joint labels that permit highly\naccurate recovery back to the original format. This representation allows us to\ntrain statistical dependency parsers to tackle SRL and achieve competitive\nperformance with the current state of the art. Our findings show the promise of\nsyntactic dependency trees in encoding semantic role relations within their\nsyntactic domain of locality, and point to potential further integration of\nsyntactic methods into semantic role labeling in the future.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11230v2",
            "date": "2020-10-21 18:10:58+00:00",
            "title": "Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents",
            "authors": [
                "Mohammad Kachuee",
                "Hao Yuan",
                "Young-Bum Kim",
                "Sungjin Lee"
            ],
            "abstract": "Turn-level user satisfaction is one of the most important performance metrics\nfor conversational agents. It can be used to monitor the agent's performance\nand provide insights about defective user experiences. Moreover, a powerful\nsatisfaction model can be used as an objective function that a conversational\nagent continuously optimizes for. While end-to-end deep learning has shown\npromising results, having access to a large number of reliable annotated\nsamples required by these methods remains challenging. In a large-scale\nconversational system, there is a growing number of newly developed skills,\nmaking the traditional data collection, annotation, and modeling process\nimpractical due to the required annotation costs as well as the turnaround\ntimes. In this paper, we suggest a self-supervised contrastive learning\napproach that leverages the pool of unlabeled data to learn user-agent\ninteractions. We show that the pre-trained models using the self-supervised\nobjective are transferable to the user satisfaction prediction. In addition, we\npropose a novel few-shot transfer learning approach that ensures better\ntransferability for very small sample sizes. The suggested few-shot method does\nnot require any inner loop optimization process and is scalable to very large\ndatasets and complex models. Based on our experiments using real-world data\nfrom a large-scale commercial system, the suggested approach is able to\nsignificantly reduce the required number of annotations, while improving the\ngeneralization on unseen out-of-domain skills.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.11238v1",
            "date": "2020-10-21 18:43:13+00:00",
            "title": "Detection of COVID-19 informative tweets using RoBERTa",
            "authors": [
                "Sirigireddy Dhanalaxmi",
                "Rohit Agarwal",
                "Aman Sinha"
            ],
            "abstract": "Social media such as Twitter is a hotspot of user-generated information. In\nthis ongoing Covid-19 pandemic, there has been an abundance of data on social\nmedia which can be classified as informative and uninformative content. In this\npaper, we present our work to detect informative Covid-19 English tweets using\nRoBERTa model as a part of the W-NUT workshop 2020. We show the efficacy of our\nmodel on a public dataset with an F1-score of 0.89 on the validation dataset\nand 0.87 on the leaderboard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11246v1",
            "date": "2020-10-21 19:01:00+00:00",
            "title": "On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries",
            "authors": [
                "Tianze Shi",
                "Chen Zhao",
                "Jordan Boyd-Graber",
                "Hal Daum\u00e9 III",
                "Lillian Lee"
            ],
            "abstract": "Large-scale semantic parsing datasets annotated with logical forms have\nenabled major advances in supervised approaches. But can richer supervision\nhelp even more? To explore the utility of fine-grained, lexical-level\nsupervision, we introduce Squall, a dataset that enriches 11,276\nWikiTableQuestions English-language questions with manually created SQL\nequivalents plus alignments between SQL and question fragments. Our annotation\nenables new training possibilities for encoder-decoder models, including\napproaches from machine translation previously precluded by the absence of\nalignments. We propose and test two methods: (1) supervised attention; (2)\nadopting an auxiliary objective of disambiguating references in the input\nqueries to table columns. In 5-fold cross validation, these strategies improve\nover strong baselines by 4.4% execution accuracy. Oracle experiments suggest\nthat annotated alignments can support further accuracy gains of up to 23.9%.",
            "journal ref": "Findings of ACL: EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11322v1",
            "date": "2020-10-21 21:45:44+00:00",
            "title": "Learning to Summarize Long Texts with Memory Compression and Transfer",
            "authors": [
                "Jaehong Park",
                "Jonathan Pilault",
                "Christopher Pal"
            ],
            "abstract": "We introduce Mem2Mem, a memory-to-memory mechanism for hierarchical recurrent\nneural network based encoder decoder architectures and we explore its use for\nabstractive document summarization. Mem2Mem transfers \"memories\" via\nreadable/writable external memory modules that augment both the encoder and\ndecoder. Our memory regularization compresses an encoded input article into a\nmore compact set of sentence representations. Most importantly, the memory\ncompression step performs implicit extraction without labels, sidestepping\nissues with suboptimal ground-truth data and exposure bias of hybrid\nextractive-abstractive summarization techniques. By allowing the decoder to\nread/write over the encoded input memory, the model learns to read salient\ninformation about the input article while keeping track of what has been\ngenerated. Our Mem2Mem approach yields results that are competitive with state\nof the art transformer based summarization methods, but with 16 times fewer\nparameters",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11334v3",
            "date": "2020-10-21 22:14:28+00:00",
            "title": "NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task",
            "authors": [
                "Muhammad Abdul-Mageed",
                "Chiyu Zhang",
                "Houda Bouamor",
                "Nizar Habash"
            ],
            "abstract": "We present the results and findings of the First Nuanced Arabic Dialect\nIdentification Shared Task (NADI). This Shared Task includes two subtasks:\ncountry-level dialect identification (Subtask 1) and province-level sub-dialect\nidentification (Subtask 2). The data for the shared task covers a total of 100\nprovinces from 21 Arab countries and are collected from the Twitter domain. As\nsuch, NADI is the first shared task to target naturally-occurring fine-grained\ndialectal text at the sub-country level. A total of 61 teams from 25 countries\nregistered to participate in the tasks, thus reflecting the interest of the\ncommunity in this area. We received 47 submissions for Subtask 1 from 18 teams\nand 9 submissions for Subtask 2 from 9 teams.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11358v1",
            "date": "2020-10-22 00:48:24+00:00",
            "title": "N-ODE Transformer: A Depth-Adaptive Variant of the Transformer Using Neural Ordinary Differential Equations",
            "authors": [
                "Aaron Baier-Reinio",
                "Hans De Sterck"
            ],
            "abstract": "We use neural ordinary differential equations to formulate a variant of the\nTransformer that is depth-adaptive in the sense that an input-dependent number\nof time steps is taken by the ordinary differential equation solver. Our goal\nin proposing the N-ODE Transformer is to investigate whether its\ndepth-adaptivity may aid in overcoming some specific known theoretical\nlimitations of the Transformer in handling nonlocal effects. Specifically, we\nconsider the simple problem of determining the parity of a binary sequence, for\nwhich the standard Transformer has known limitations that can only be overcome\nby using a sufficiently large number of layers or attention heads. We find,\nhowever, that the depth-adaptivity of the N-ODE Transformer does not provide a\nremedy for the inherently nonlocal nature of the parity problem, and provide\nexplanations for why this is so. Next, we pursue regularization of the N-ODE\nTransformer by penalizing the arclength of the ODE trajectories, but find that\nthis fails to improve the accuracy or efficiency of the N-ODE Transformer on\nthe challenging parity problem. We suggest future avenues of research for\nmodifications and extensions of the N-ODE Transformer that may lead to improved\naccuracy and efficiency for sequence modelling tasks such as neural machine\ntranslation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.11374v1",
            "date": "2020-10-22 01:51:09+00:00",
            "title": "Stronger Transformers for Neural Multi-Hop Question Generation",
            "authors": [
                "Devendra Singh Sachan",
                "Lingfei Wu",
                "Mrinmaya Sachan",
                "William Hamilton"
            ],
            "abstract": "Prior work on automated question generation has almost exclusively focused on\ngenerating simple questions whose answers can be extracted from a single\ndocument. However, there is an increasing interest in developing systems that\nare capable of more complex multi-hop question generation, where answering the\nquestions requires reasoning over multiple documents. In this work, we\nintroduce a series of strong transformer models for multi-hop question\ngeneration, including a graph-augmented transformer that leverages relations\nbetween entities in the text. While prior work has emphasized the importance of\ngraph-based models, we show that we can substantially outperform the\nstate-of-the-art by 5 BLEU points using a standard transformer architecture. We\nfurther demonstrate that graph-based augmentations can provide complimentary\nimprovements on top of this foundation. Interestingly, we find that several\nimportant factors--such as the inclusion of an auxiliary contrastive objective\nand data filtering could have larger impacts on performance. We hope that our\nstronger baselines and analysis provide a constructive foundation for future\nwork in this area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11386v1",
            "date": "2020-10-22 02:26:01+00:00",
            "title": "Distilling Dense Representations for Ranking using Tightly-Coupled Teachers",
            "authors": [
                "Sheng-Chieh Lin",
                "Jheng-Hong Yang",
                "Jimmy Lin"
            ],
            "abstract": "We present an approach to ranking with dense representations that applies\nknowledge distillation to improve the recently proposed late-interaction\nColBERT model. Specifically, we distill the knowledge from ColBERT's expressive\nMaxSim operator for computing relevance scores into a simple dot product, thus\nenabling single-step ANN search. Our key insight is that during distillation,\ntight coupling between the teacher model and the student model enables more\nflexible distillation strategies and yields better learned representations. We\nempirically show that our approach improves query latency and greatly reduces\nthe onerous storage requirements of ColBERT, while only making modest\nsacrifices in terms of effectiveness. By combining our dense representations\nwith sparse representations derived from document expansion, we are able to\napproach the effectiveness of a standard cross-encoder reranker using BERT that\nis orders of magnitude slower.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2010.11395v3",
            "date": "2020-10-22 03:01:21+00:00",
            "title": "Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset",
            "authors": [
                "Xie Chen",
                "Yu Wu",
                "Zhenghao Wang",
                "Shujie Liu",
                "Jinyu Li"
            ],
            "abstract": "Recently, Transformer based end-to-end models have achieved great success in\nmany areas including speech recognition. However, compared to LSTM models, the\nheavy computational cost of the Transformer during inference is a key issue to\nprevent their applications. In this work, we explored the potential of\nTransformer Transducer (T-T) models for the fist pass decoding with low latency\nand fast speed on a large-scale dataset. We combine the idea of Transformer-XL\nand chunk-wise streaming processing to design a streamable Transformer\nTransducer model. We demonstrate that T-T outperforms the hybrid model, RNN\nTransducer (RNN-T), and streamable Transformer attention-based encoder-decoder\nmodel in the streaming scenario. Furthermore, the runtime cost and latency can\nbe optimized with a relatively small look-ahead.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11524v3",
            "date": "2020-10-22 08:36:33+00:00",
            "title": "slimIPL: Language-Model-Free Iterative Pseudo-Labeling",
            "authors": [
                "Tatiana Likhomanenko",
                "Qiantong Xu",
                "Jacob Kahn",
                "Gabriel Synnaeve",
                "Ronan Collobert"
            ],
            "abstract": "Recent results in end-to-end automatic speech recognition have demonstrated\nthe efficacy of pseudo-labeling for semi-supervised models trained both with\nConnectionist Temporal Classification (CTC) and Sequence-to-Sequence (seq2seq)\nlosses. Iterative Pseudo-Labeling (IPL), which continuously trains a single\nmodel using pseudo-labels iteratively re-generated as the model learns, has\nbeen shown to further improve performance in ASR. We improve upon the IPL\nalgorithm: as the model learns, we propose to iteratively re-generate\ntranscriptions with hard labels (the most probable tokens), that is, without a\nlanguage model. We call this approach Language-Model-Free IPL (slimIPL) and\ngive a resultant training setup for low-resource settings with CTC-based\nmodels. slimIPL features a dynamic cache for pseudo-labels which reduces\nsensitivity to changes in relabeling hyperparameters and results in improves\ntraining stability. slimIPL is also highly-efficient and requires 3.5-4x fewer\ncomputational resources to converge than other state-of-the-art\nsemi/self-supervised approaches. With only 10 hours of labeled audio, slimIPL\nis competitive with self-supervised approaches, and is state-of-the-art with\n100 hours of labeled audio without the use of a language model both at test\ntime and during pseudo-label generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11562v1",
            "date": "2020-10-22 09:38:08+00:00",
            "title": "Bilinear Fusion of Commonsense Knowledge with Attention-Based NLI Models",
            "authors": [
                "Amit Gajbhiye",
                "Thomas Winterbottom",
                "Noura Al Moubayed",
                "Steven Bradley"
            ],
            "abstract": "We consider the task of incorporating real-world commonsense knowledge into\ndeep Natural Language Inference (NLI) models. Existing external knowledge\nincorporation methods are limited to lexical level knowledge and lack\ngeneralization across NLI models, datasets, and commonsense knowledge sources.\nTo address these issues, we propose a novel NLI model-independent neural\nframework, BiCAM. BiCAM incorporates real-world commonsense knowledge into NLI\nmodels. Combined with convolutional feature detectors and bilinear feature\nfusion, BiCAM provides a conceptually simple mechanism that generalizes well.\nQuantitative evaluations with two state-of-the-art NLI baselines on SNLI and\nSciTail datasets in conjunction with ConceptNet and Aristo Tuple KGs show that\nBiCAM considerably improves the accuracy the incorporated NLI baselines. For\nexample, our BiECAM model, an instance of BiCAM, on the challenging SciTail\ndataset, improves the accuracy of incorporated baselines by 7.0% with\nConceptNet, and 8.0% with Aristo Tuple KG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11593v1",
            "date": "2020-10-22 10:52:31+00:00",
            "title": "A Technical Report: BUT Speech Translation Systems",
            "authors": [
                "Hari Krishna Vydana",
                "Lukas Burget",
                "Jan Cernocky"
            ],
            "abstract": "The paper describes the BUT's speech translation systems. The systems are\nEnglish$\\longrightarrow$German offline speech translation systems. The systems\nare based on our previous works \\cite{Jointly_trained_transformers}. Though\nEnd-to-End and cascade~(ASR-MT) spoken language translation~(SLT) systems are\nreaching comparable performances, a large degradation is observed when\ntranslating ASR hypothesis compared to the oracle input text. To reduce this\nperformance degradation, we have jointly-trained ASR and MT modules with ASR\nobjective as an auxiliary loss. Both the networks are connected through the\nneural hidden representations. This model has an End-to-End differentiable path\nwith respect to the final objective function and also utilizes the ASR\nobjective for better optimization. During the inference both the modules(i.e.,\nASR and MT) are connected through the hidden representations corresponding to\nthe n-best hypotheses. Ensembling with independently trained ASR and MT models\nhave further improved the performance of the system.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11666v1",
            "date": "2020-10-22 12:54:14+00:00",
            "title": "Reducing Unintended Identity Bias in Russian Hate Speech Detection",
            "authors": [
                "Nadezhda Zueva",
                "Madina Kabirova",
                "Pavel Kalaidin"
            ],
            "abstract": "Toxicity has become a grave problem for many online communities and has been\ngrowing across many languages, including Russian. Hate speech creates an\nenvironment of intimidation, discrimination, and may even incite some\nreal-world violence. Both researchers and social platforms have been focused on\ndeveloping models to detect toxicity in online communication for a while now. A\ncommon problem of these models is the presence of bias towards some words (e.g.\nwoman, black, jew) that are not toxic, but serve as triggers for the classifier\ndue to model caveats. In this paper, we describe our efforts towards\nclassifying hate speech in Russian, and propose simple techniques of reducing\nunintended bias, such as generating training data with language models using\nterms and words related to protected identities as context and applying word\ndropout to such words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11701v1",
            "date": "2020-09-29 16:04:08+00:00",
            "title": "Spatial Attention as an Interface for Image Captioning Models",
            "authors": [
                "Philipp Sadler"
            ],
            "abstract": "The internal workings of modern deep learning models stay often unclear to an\nexternal observer, although spatial attention mechanisms are involved. The idea\nof this work is to translate these spatial attentions into natural language to\nprovide a simpler access to the model's function. Thus, I took a neural image\ncaptioning model and measured the reactions to external modification in its\nspatial attention for three different interface methods: a fixation over the\nwhole generation process, a fixation for the first time-steps and an addition\nto the generator's attention. The experimental results for bounding box based\nspatial attention vectors have shown that the captioning model reacts to method\ndependent changes in up to 52.65% and includes in 9.00% of the cases object\ncategories, which were otherwise unmentioned. Afterwards, I established such a\nlink to a hierarchical co-attention network for visual question answering by\nextraction of its word, phrase and question level spatial attentions. Here,\ngenerated captions for the word level included details of the question-answer\npairs in up to 55.20% of the cases. This work indicates that spatial attention\nseen as an external interface for image caption generators is an useful method\nto access visual functions in natural language.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.11731v2",
            "date": "2020-10-22 13:52:18+00:00",
            "title": "Improving BERT Performance for Aspect-Based Sentiment Analysis",
            "authors": [
                "Akbar Karimi",
                "Leonardo Rossi",
                "Andrea Prati"
            ],
            "abstract": "Aspect-Based Sentiment Analysis (ABSA) studies the consumer opinion on the\nmarket products. It involves examining the type of sentiments as well as\nsentiment targets expressed in product reviews. Analyzing the language used in\na review is a difficult task that requires a deep understanding of the\nlanguage. In recent years, deep language models, such as BERT\n\\cite{devlin2019bert}, have shown great progress in this regard. In this work,\nwe propose two simple modules called Parallel Aggregation and Hierarchical\nAggregation to be utilized on top of BERT for two main ABSA tasks namely Aspect\nExtraction (AE) and Aspect Sentiment Classification (ASC) in order to improve\nthe model's performance. We show that applying the proposed models eliminates\nthe need for further training of the BERT model. The source code is available\non the Web for further research and reproduction of the results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11918v1",
            "date": "2020-10-22 17:49:42+00:00",
            "title": "AdapterDrop: On the Efficiency of Adapters in Transformers",
            "authors": [
                "Andreas R\u00fcckl\u00e9",
                "Gregor Geigle",
                "Max Glockner",
                "Tilman Beck",
                "Jonas Pfeiffer",
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "Massively pre-trained transformer models are computationally expensive to\nfine-tune, slow for inference, and have large storage requirements. Recent\napproaches tackle these shortcomings by training smaller models, dynamically\nreducing the model size, and by training light-weight adapters. In this paper,\nwe propose AdapterDrop, removing adapters from lower transformer layers during\ntraining and inference, which incorporates concepts from all three directions.\nWe show that AdapterDrop can dynamically reduce the computational overhead when\nperforming inference over multiple tasks simultaneously, with minimal decrease\nin task performances. We further prune adapters from AdapterFusion, which\nimproves the inference efficiency while maintaining the task performances\nentirely.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.11930v1",
            "date": "2020-10-22 17:56:33+00:00",
            "title": "Scientific Claim Verification with VERT5ERINI",
            "authors": [
                "Ronak Pradeep",
                "Xueguang Ma",
                "Rodrigo Nogueira",
                "Jimmy Lin"
            ],
            "abstract": "This work describes the adaptation of a pretrained sequence-to-sequence model\nto the task of scientific claim verification in the biomedical domain. We\npropose VERT5ERINI that exploits T5 for abstract retrieval, sentence selection\nand label prediction, which are three critical sub-tasks of claim verification.\nWe evaluate our pipeline on SCIFACT, a newly curated dataset that requires\nmodels to not just predict the veracity of claims but also provide relevant\nsentences from a corpus of scientific literature that support this decision.\nEmpirically, our pipeline outperforms a strong baseline in each of the three\nsteps. Finally, we show VERT5ERINI's ability to generalize to two new datasets\nof COVID-19 claims using evidence from the ever-expanding CORD-19 corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11936v1",
            "date": "2020-10-22 17:58:44+00:00",
            "title": "UniCase -- Rethinking Casing in Language Models",
            "authors": [
                "Rafal Powalski",
                "Tomasz Stanislawek"
            ],
            "abstract": "In this paper, we introduce a new approach to dealing with the problem of\ncase-sensitiveness in Language Modelling (LM). We propose simple architecture\nmodification to the RoBERTa language model, accompanied by a new tokenization\nstrategy, which we named Unified Case LM (UniCase). We tested our solution on\nthe GLUE benchmark, which led to increased performance by 0.42 points.\nMoreover, we prove that the UniCase model works much better when we have to\ndeal with text data, where all tokens are uppercased (+5.88 point).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11966v1",
            "date": "2020-10-22 18:01:51+00:00",
            "title": "Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data",
            "authors": [
                "David Lowell",
                "Brian E. Howard",
                "Zachary C. Lipton",
                "Byron C. Wallace"
            ],
            "abstract": "Unsupervised Data Augmentation (UDA) is a semi-supervised technique that\napplies a consistency loss to penalize differences between a model's\npredictions on (a) observed (unlabeled) examples; and (b) corresponding\n'noised' examples produced via data augmentation. While UDA has gained\npopularity for text classification, open questions linger over which design\ndecisions are necessary and over how to extend the method to sequence labeling\ntasks. This method has recently gained traction for text classification. In\nthis paper, we re-examine UDA and demonstrate its efficacy on several\nsequential tasks. Our main contribution is an empirical study of UDA to\nestablish which components of the algorithm confer benefits in NLP. Notably,\nalthough prior work has emphasized the use of clever augmentation techniques\nincluding back-translation, we find that enforcing consistency between\npredictions assigned to observed and randomly substituted words often yields\ncomparable (or greater) benefits compared to these complex perturbation models.\nFurthermore, we find that applying its consistency loss affords meaningful\ngains without any unlabeled data at all, i.e., in a standard supervised\nsetting. In short: UDA need not be unsupervised, and does not require complex\ndata augmentation to be effective.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.11980v1",
            "date": "2020-10-22 18:36:31+00:00",
            "title": "A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents",
            "authors": [
                "Tuan Manh Lai",
                "Trung Bui",
                "Doo Soon Kim",
                "Quan Hung Tran"
            ],
            "abstract": "Keyphrase extraction is the task of extracting a small set of phrases that\nbest describe a document. Most existing benchmark datasets for the task\ntypically have limited numbers of annotated documents, making it challenging to\ntrain increasingly complex neural networks. In contrast, digital libraries\nstore millions of scientific articles online, covering a wide range of topics.\nWhile a significant portion of these articles contain keyphrases provided by\ntheir authors, most other articles lack such kind of annotations. Therefore, to\neffectively utilize these large amounts of unlabeled articles, we propose a\nsimple and efficient joint learning approach based on the idea of\nself-distillation. Experimental results show that our approach consistently\nimproves the performance of baseline models for keyphrase extraction.\nFurthermore, our best models outperform previous methods for the task,\nachieving new state-of-the-art results on two public benchmarks: Inspec and\nSemEval-2017.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12148v2",
            "date": "2020-10-23 03:42:20+00:00",
            "title": "ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding",
            "authors": [
                "Dongling Xiao",
                "Yu-Kun Li",
                "Han Zhang",
                "Yu Sun",
                "Hao Tian",
                "Hua Wu",
                "Haifeng Wang"
            ],
            "abstract": "Coarse-grained linguistic information, such as named entities or phrases,\nfacilitates adequately representation learning in pre-training. Previous works\nmainly focus on extending the objective of BERT's Masked Language Modeling\n(MLM) from masking individual tokens to contiguous sequences of n tokens. We\nargue that such contiguously masking method neglects to model the\nintra-dependencies and inter-relation of coarse-grained linguistic information.\nAs an alternative, we propose ERNIE-Gram, an explicitly n-gram masking method\nto enhance the integration of coarse-grained information into pre-training. In\nERNIE-Gram, n-grams are masked and predicted directly using explicit n-gram\nidentities rather than contiguous sequences of n tokens. Furthermore,\nERNIE-Gram employs a generator model to sample plausible n-gram identities as\noptional n-gram masks and predict them in both coarse-grained and fine-grained\nmanners to enable comprehensive n-gram prediction and relation modeling. We\npre-train ERNIE-Gram on English and Chinese text corpora and fine-tune on 19\ndownstream tasks. Experimental results show that ERNIE-Gram outperforms\nprevious pre-training models like XLNet and RoBERTa by a large margin, and\nachieves comparable results with state-of-the-art methods. The source codes and\npre-trained models have been released at https://github.com/PaddlePaddle/ERNIE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12267v2",
            "date": "2020-10-23 09:53:01+00:00",
            "title": "Show and Speak: Directly Synthesize Spoken Description of Images",
            "authors": [
                "Xinsheng Wang",
                "Siyuan Feng",
                "Jihua Zhu",
                "Mark Hasegawa-Johnson",
                "Odette Scharenborg"
            ],
            "abstract": "This paper proposes a new model, referred to as the show and speak (SAS)\nmodel that, for the first time, is able to directly synthesize spoken\ndescriptions of images, bypassing the need for any text or phonemes. The basic\nstructure of SAS is an encoder-decoder architecture that takes an image as\ninput and predicts the spectrogram of speech that describes this image. The\nfinal speech audio is obtained from the predicted spectrogram via WaveNet.\nExtensive experiments on the public benchmark database Flickr8k demonstrate\nthat the proposed SAS is able to synthesize natural spoken descriptions for\nimages, indicating that synthesizing spoken descriptions for images while\nbypassing text and phonemes is feasible.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2010.12283v2",
            "date": "2020-10-23 10:28:20+00:00",
            "title": "ST-BERT: Cross-modal Language Model Pre-training For End-to-end Spoken Language Understanding",
            "authors": [
                "Minjeong Kim",
                "Gyuwan Kim",
                "Sang-Woo Lee",
                "Jung-Woo Ha"
            ],
            "abstract": "Language model pre-training has shown promising results in various downstream\ntasks. In this context, we introduce a cross-modal pre-trained language model,\ncalled Speech-Text BERT (ST-BERT), to tackle end-to-end spoken language\nunderstanding (E2E SLU) tasks. Taking phoneme posterior and subword-level text\nas an input, ST-BERT learns a contextualized cross-modal alignment via our two\nproposed pre-training tasks: Cross-modal Masked Language Modeling (CM-MLM) and\nCross-modal Conditioned Language Modeling (CM-CLM). Experimental results on\nthree benchmarks present that our approach is effective for various SLU\ndatasets and shows a surprisingly marginal performance degradation even when 1%\nof the training data are available. Also, our method shows further SLU\nperformance gain via domain-adaptive pre-training with domain-specific\nspeech-text pair data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12305v1",
            "date": "2020-10-23 11:16:53+00:00",
            "title": "Adversarial Learning of Feature-based Meta-Embeddings",
            "authors": [
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen",
                "Dietrich Klakow"
            ],
            "abstract": "Certain embedding types outperform others in different scenarios, e.g.,\nsubword-based embeddings can model rare words well and domain-specific\nembeddings can better represent in-domain terms. Therefore, recent works\nconsider attention-based meta-embeddings to combine different embedding types.\nWe demonstrate that these methods have two shortcomings: First, the attention\nweights are calculated without knowledge of word properties. Second, the\ndifferent embedding types can form clusters in the common embedding space,\npreventing the computation of a meaningful average of different embeddings and\nthus, reducing performance. We propose to solve these problems by using\nfeature-based meta-embeddings learned with adversarial training. Our\nexperiments and analysis on sentence classification and sequence tagging tasks\nshow that our approach is effective. We set the new state of the art on various\ndatasets across languages and domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12309v3",
            "date": "2020-10-23 11:22:01+00:00",
            "title": "A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios",
            "authors": [
                "Michael A. Hedderich",
                "Lukas Lange",
                "Heike Adel",
                "Jannik Str\u00f6tgen",
                "Dietrich Klakow"
            ],
            "abstract": "Deep neural networks and huge language models are becoming omnipresent in\nnatural language applications. As they are known for requiring large amounts of\ntraining data, there is a growing body of work to improve the performance in\nlow-resource settings. Motivated by the recent fundamental changes towards\nneural models and the popular pre-train and fine-tune paradigm, we survey\npromising approaches for low-resource natural language processing. After a\ndiscussion about the different dimensions of data availability, we give a\nstructured overview of methods that enable learning when training data is\nsparse. This includes mechanisms to create additional labeled data like data\naugmentation and distant supervision as well as transfer learning settings that\nreduce the need for target supervision. A goal of our survey is to explain how\nthese methods differ in their requirements as understanding them is essential\nfor choosing a technique suited for a specific low-resource setting. Further\nkey aspects of this work are to highlight open issues and to outline promising\ndirections for future research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12322v1",
            "date": "2020-10-23 11:59:28+00:00",
            "title": "NLNDE at CANTEMIST: Neural Sequence Labeling and Parsing Approaches for Clinical Concept Extraction",
            "authors": [
                "Lukas Lange",
                "Xiang Dai",
                "Heike Adel",
                "Jannik Str\u00f6tgen"
            ],
            "abstract": "The recognition and normalization of clinical information, such as tumor\nmorphology mentions, is an important, but complex process consisting of\nmultiple subtasks. In this paper, we describe our system for the CANTEMIST\nshared task, which is able to extract, normalize and rank ICD codes from\nSpanish electronic health records using neural sequence labeling and parsing\napproaches with context-aware embeddings. Our best system achieves 85.3 F1,\n76.7 F1, and 77.0 MAP for the three tasks, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12421v2",
            "date": "2020-10-23 14:11:04+00:00",
            "title": "TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification",
            "authors": [
                "Francesco Barbieri",
                "Jose Camacho-Collados",
                "Leonardo Neves",
                "Luis Espinosa-Anke"
            ],
            "abstract": "The experimental landscape in natural language processing for social media is\ntoo fragmented. Each year, new shared tasks and datasets are proposed, ranging\nfrom classics like sentiment analysis to irony detection or emoji prediction.\nTherefore, it is unclear what the current state of the art is, as there is no\nstandardized evaluation protocol, neither a strong set of baselines trained on\nsuch domain-specific data. In this paper, we propose a new evaluation framework\n(TweetEval) consisting of seven heterogeneous Twitter-specific classification\ntasks. We also provide a strong set of baselines as starting point, and compare\ndifferent language modeling pre-training strategies. Our initial experiments\nshow the effectiveness of starting off with existing pre-trained generic\nlanguage models, and continue training them on Twitter corpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12512v1",
            "date": "2020-10-23 16:29:26+00:00",
            "title": "Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classification",
            "authors": [
                "Linyi Yang",
                "Eoin M. Kenny",
                "Tin Lok James Ng",
                "Yi Yang",
                "Barry Smyth",
                "Ruihai Dong"
            ],
            "abstract": "Corporate mergers and acquisitions (M&A) account for billions of dollars of\ninvestment globally every year, and offer an interesting and challenging domain\nfor artificial intelligence. However, in these highly sensitive domains, it is\ncrucial to not only have a highly robust and accurate model, but be able to\ngenerate useful explanations to garner a user's trust in the automated system.\nRegrettably, the recent research regarding eXplainable AI (XAI) in financial\ntext classification has received little to no attention, and many current\nmethods for generating textual-based explanations result in highly implausible\nexplanations, which damage a user's trust in the system. To address these\nissues, this paper proposes a novel methodology for producing plausible\ncounterfactual explanations, whilst exploring the regularization benefits of\nadversarial training on language models in the domain of FinTech. Exhaustive\nquantitative experiments demonstrate that not only does this approach improve\nthe model accuracy when compared to the current state-of-the-art and human\nperformance, but it also generates counterfactual explanations which are\nsignificantly more plausible based on human trials.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12562v2",
            "date": "2020-10-23 17:44:59+00:00",
            "title": "On the Transformer Growth for Progressive BERT Training",
            "authors": [
                "Xiaotao Gu",
                "Liyuan Liu",
                "Hongkun Yu",
                "Jing Li",
                "Chen Chen",
                "Jiawei Han"
            ],
            "abstract": "Due to the excessive cost of large-scale language model pre-training,\nconsiderable efforts have been made to train BERT progressively -- start from\nan inferior but low-cost model and gradually grow the model to increase the\ncomputational complexity. Our objective is to advance the understanding of\nTransformer growth and discover principles that guide progressive training.\nFirst, we find that similar to network architecture search, Transformer growth\nalso favors compound scaling. Specifically, while existing methods only conduct\nnetwork growth in a single dimension, we observe that it is beneficial to use\ncompound growth operators and balance multiple dimensions (e.g., depth, width,\nand input length of the model). Moreover, we explore alternative growth\noperators in each dimension via controlled comparison to give operator\nselection practical guidance. In light of our analyses, the proposed method\nspeeds up BERT pre-training by 73.6% and 82.2% for the base and large models\nrespectively, while achieving comparable performances",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12623v2",
            "date": "2020-10-23 19:13:47+00:00",
            "title": "Unsupervised Multi-hop Question Answering by Question Generation",
            "authors": [
                "Liangming Pan",
                "Wenhu Chen",
                "Wenhan Xiong",
                "Min-Yen Kan",
                "William Yang Wang"
            ],
            "abstract": "Obtaining training data for multi-hop question answering (QA) is\ntime-consuming and resource-intensive. We explore the possibility to train a\nwell-performed multi-hop QA model without referencing any human-labeled\nmulti-hop question-answer pairs, i.e., unsupervised multi-hop QA. We propose\nMQA-QG, an unsupervised framework that can generate human-like multi-hop\ntraining data from both homogeneous and heterogeneous data sources. MQA-QG\ngenerates questions by first selecting/generating relevant information from\neach data source and then integrating the multiple information to form a\nmulti-hop question. Using only generated training data, we can train a\ncompetent multi-hop QA which achieves 61% and 83% of the supervised learning\nperformance for the HybridQA and the HotpotQA dataset, respectively. We also\nshow that pretraining the QA system with the generated data would greatly\nreduce the demand for human-annotated training data. Our codes are publicly\navailable at https://github.com/teacherpeterpan/Unsupervised-Multi-hop-QA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12637v2",
            "date": "2020-10-23 19:55:11+00:00",
            "title": "Domain Specific Complex Sentence (DSCS) Semantic Similarity Dataset",
            "authors": [
                "Dhivya Chandrasekaran",
                "Vijay Mago"
            ],
            "abstract": "Semantic textual similarity is one of the open research challenges in the\nfield of Natural Language Processing. Extensive research has been carried out\nin this field and near-perfect results are achieved by recent transformed based\nmodels in existing benchmark datasets like STS dataset and SICK dataset. In\nthis paper, we study the sentences in these datasets and analyze the\nsensitivity of various word embeddings with respect to the complexity of the\nsentences. We propose a new benchmark dataset -- the Domain Specific Complex\nSentences (DSCS) dataset comprising of 50 sentence pairs with associated\nsemantic similarity values provided by 15 human annotators. Readability\nanalysis is performed to highlight the increase in complexity of the sentences\nin the existing benchmark datasets and those in the proposed dataset. Further,\nwe perform a comparative analysis of the performance of various word embeddings\nand the results justify the hypothesis that the performance of the word\nembeddings decrease with an increase in complexity of the sentences.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12673v3",
            "date": "2020-10-23 21:16:30+00:00",
            "title": "On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer",
            "authors": [
                "Liang Lu",
                "Zhong Meng",
                "Naoyuki Kanda",
                "Jinyu Li",
                "Yifan Gong"
            ],
            "abstract": "Hybrid Autoregressive Transducer (HAT) is a recently proposed end-to-end\nacoustic model that extends the standard Recurrent Neural Network Transducer\n(RNN-T) for the purpose of the external language model (LM) fusion. In HAT, the\nblank probability and the label probability are estimated using two separate\nprobability distributions, which provides a more accurate solution for internal\nLM score estimation, and thus works better when combining with an external LM.\nPrevious work mainly focuses on HAT model training with the negative\nlog-likelihood loss, while in this paper, we study the minimum word error rate\n(MWER) training of HAT -- a criterion that is closer to the evaluation metric\nfor speech recognition, and has been successfully applied to other types of\nend-to-end models such as sequence-to-sequence (S2S) and RNN-T models. From\nexperiments with around 30,000 hours of training data, we show that MWER\ntraining can improve the accuracy of HAT models, while at the same time,\nimproving the robustness of the model against the decoding hyper-parameters\nsuch as length normalization and decoding beam during inference.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12675v1",
            "date": "2020-10-23 21:19:03+00:00",
            "title": "Overcoming Conflicting Data for Model Updates",
            "authors": [
                "David Gaddy",
                "Alex Kouzemtchenko",
                "Pavan Kumar Reddy",
                "Prateek Kolhar",
                "Rushin Shah"
            ],
            "abstract": "In this paper, we explore how to use a small amount of new data to update a\nmodel when the desired output for some examples has changed. When making\nupdates in this way, one potential problem that arises is the presence of\nconflicting data, or out-of-date labels in the original training set. To\nevaluate the impact of this problem, we propose an experimental setup for\nsimulating changes to a neural semantic parser. We show that the presence of\nconflicting data greatly hinders learning of an update, then explore several\nmethods to mitigate its effect. Our methods lead to large improvements in model\naccuracy compared to a naive mixing strategy, and our best method closes 86% of\nthe accuracy gap between this baseline and an oracle upper bound.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12676v1",
            "date": "2020-10-23 21:22:50+00:00",
            "title": "A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsing",
            "authors": [
                "Chunchuan Lyu",
                "Shay B. Cohen",
                "Ivan Titov"
            ],
            "abstract": "Abstract Meaning Representations (AMR) are a broad-coverage semantic\nformalism which represents sentence meaning as a directed acyclic graph. To\ntrain most AMR parsers, one needs to segment the graph into subgraphs and align\neach such subgraph to a word in a sentence; this is normally done at\npreprocessing, relying on hand-crafted rules. In contrast, we treat both\nalignment and segmentation as latent variables in our model and induce them as\npart of end-to-end training.\n  As marginalizing over the structured latent variables is infeasible, we use\nthe variational autoencoding framework.\n  To ensure end-to-end differentiable optimization, we introduce a continuous\ndifferentiable relaxation of the segmentation and alignment problems. We\nobserve that inducing segmentation yields substantial gains over using a\n`greedy' segmentation heuristic. The performance of our method also approaches\nthat of a model that relies on \\citet{Lyu2018AMRPA}'s segmentation rules, which\nwere hand-crafted to handle individual AMR constructions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12683v1",
            "date": "2020-10-23 21:57:56+00:00",
            "title": "Long Document Ranking with Query-Directed Sparse Transformer",
            "authors": [
                "Jyun-Yu Jiang",
                "Chenyan Xiong",
                "Chia-Jung Lee",
                "Wei Wang"
            ],
            "abstract": "The computing cost of transformer self-attention often necessitates breaking\nlong documents to fit in pretrained models in document ranking tasks. In this\npaper, we design Query-Directed Sparse attention that induces IR-axiomatic\nstructures in transformer self-attention. Our model, QDS-Transformer, enforces\nthe principle properties desired in ranking: local contextualization,\nhierarchical representation, and query-oriented proximity matching, while it\nalso enjoys efficiency from sparsity. Experiments on one fully supervised and\nthree few-shot TREC document ranking benchmarks demonstrate the consistent and\nrobust advantage of QDS-Transformer over previous approaches, as they either\nretrofit long documents into BERT or use sparse attention without emphasizing\nIR principles. We further quantify the computing complexity and demonstrates\nthat our sparse attention with TVM implementation is twice more efficient than\nthe fully-connected self-attention. All source codes, trained model, and\npredictions of this work are available at\nhttps://github.com/hallogameboy/QDS-Transformer.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.12755v2",
            "date": "2020-10-24 03:17:31+00:00",
            "title": "Effective Distant Supervision for Temporal Relation Extraction",
            "authors": [
                "Xinyu Zhao",
                "Shih-ting Lin",
                "Greg Durrett"
            ],
            "abstract": "A principal barrier to training temporal relation extraction models in new\ndomains is the lack of varied, high quality examples and the challenge of\ncollecting more. We present a method of automatically collecting\ndistantly-supervised examples of temporal relations. We scrape and\nautomatically label event pairs where the temporal relations are made explicit\nin text, then mask out those explicit cues, forcing a model trained on this\ndata to learn other signals. We demonstrate that a pre-trained Transformer\nmodel is able to transfer from the weakly labeled examples to human-annotated\nbenchmarks in both zero-shot and few-shot settings, and that the masking scheme\nis important in improving generalization.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12773v1",
            "date": "2020-10-24 04:35:35+00:00",
            "title": "Structure-Grounded Pretraining for Text-to-SQL",
            "authors": [
                "Xiang Deng",
                "Ahmed Hassan Awadallah",
                "Christopher Meek",
                "Oleksandr Polozov",
                "Huan Sun",
                "Matthew Richardson"
            ],
            "abstract": "Learning to capture text-table alignment is essential for table related tasks\nlike text-to-SQL. The model needs to correctly recognize natural language\nreferences to columns and values and to ground them in the given database\nschema. In this paper, we present a novel weakly supervised Structure-Grounded\npretraining framework (StruG) for text-to-SQL that can effectively learn to\ncapture text-table alignment based on a parallel text-table corpus. We identify\na set of novel prediction tasks: column grounding, value grounding and\ncolumn-value mapping, and train them using weak supervision without requiring\ncomplex SQL annotation. Additionally, to evaluate the model under a more\nrealistic setting, we create a new evaluation set Spider-Realistic based on\nSpider with explicit mentions of column names removed, and adopt two existing\nsingle-database text-to-SQL datasets. StruG significantly outperforms\nBERT-LARGE on Spider and the realistic evaluation sets, while bringing\nconsistent improvement on the large-scale WikiSQL benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12777v1",
            "date": "2020-10-24 04:49:15+00:00",
            "title": "Improving Multilingual Models with Language-Clustered Vocabularies",
            "authors": [
                "Hyung Won Chung",
                "Dan Garrette",
                "Kiat Chuan Tan",
                "Jason Riesa"
            ],
            "abstract": "State-of-the-art multilingual models depend on vocabularies that cover all of\nthe languages the model will expect to see at inference time, but the standard\nmethods for generating those vocabularies are not ideal for massively\nmultilingual applications. In this work, we introduce a novel procedure for\nmultilingual vocabulary generation that combines the separately trained\nvocabularies of several automatically derived language clusters, thus balancing\nthe trade-off between cross-lingual subword sharing and language-specific\nvocabularies. Our experiments show improvements across languages on key\nmultilingual benchmark tasks TyDi QA (+2.9 F1), XNLI (+2.1\\%), and WikiAnn NER\n(+2.8 F1) and factor of 8 reduction in out-of-vocabulary rate, all without\nincreasing the size of the model or data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12780v1",
            "date": "2020-10-24 04:52:28+00:00",
            "title": "Open-Domain Dialogue Generation Based on Pre-trained Language Models",
            "authors": [
                "Yan Zeng",
                "Jian-Yun Nie"
            ],
            "abstract": "Pre-trained language models have been successfully used in response\ngeneration for open-domain dialogue. Four main frameworks have been proposed:\n(1) Transformer-ED using Transformer encoder and decoder separately for source\nand target sentences; (2) Transformer-Dec using Transformer decoder for both\nsource and target sentences; (3) Transformer-MLM using Transformer decoder that\napplies bi-directional attention on the source side and left-to-right attention\non the target side with masked language model objective; and (4) Transformer-AR\nthat uses auto-regressive objective instead. In this study, we compare these\nframeworks on 3 datasets, and our comparison reveals that the best framework\nuses bidirectional attention on the source side and does not separate encoder\nand decoder. We also examine model discrepancy, and our experiments confirm\nthat the performance of a model is directly impacted by the underlying\ndiscrepancies. We then propose two correction methods to reduce the\ndiscrepancies, and both improve the model performance. These results show that\ndiscrepancies is an important factor to consider when we use a pre-trained\nmodel, and a reduction in discrepancies can lead to improved performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12800v1",
            "date": "2020-10-24 06:30:59+00:00",
            "title": "COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval",
            "authors": [
                "Xinliang Frederick Zhang",
                "Heming Sun",
                "Xiang Yue",
                "Emmett Jesrani",
                "Simon Lin",
                "Huan Sun"
            ],
            "abstract": "We present a large challenging dataset, COUGH, for COVID-19 FAQ retrieval.\nSpecifically, similar to a standard FAQ dataset, COUGH consists of three parts:\nFAQ Bank, User Query Bank and Annotated Relevance Set. FAQ Bank contains ~16K\nFAQ items scraped from 55 credible websites (e.g., CDC and WHO). For\nevaluation, we introduce User Query Bank and Annotated Relevance Set, where the\nformer contains 1201 human-paraphrased queries while the latter contains ~32\nhuman-annotated FAQ items for each query. We analyze COUGH by testing different\nFAQ retrieval models built on top of BM25 and BERT, among which the best model\nachieves 0.29 under P@5, indicating that the dataset presents a great challenge\nfor future research. Our dataset is freely available at\nhttps://github.com/sunlab-osu/covid-faq.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12821v1",
            "date": "2020-10-24 07:43:00+00:00",
            "title": "Rethinking embedding coupling in pre-trained language models",
            "authors": [
                "Hyung Won Chung",
                "Thibault F\u00e9vry",
                "Henry Tsai",
                "Melvin Johnson",
                "Sebastian Ruder"
            ],
            "abstract": "We re-evaluate the standard practice of sharing weights between input and\noutput embeddings in state-of-the-art pre-trained language models. We show that\ndecoupled embeddings provide increased modeling flexibility, allowing us to\nsignificantly improve the efficiency of parameter allocation in the input\nembedding of multilingual models. By reallocating the input embedding\nparameters in the Transformer layers, we achieve dramatically better\nperformance on standard natural language understanding tasks with the same\nnumber of parameters during fine-tuning. We also show that allocating\nadditional capacity to the output embedding provides benefits to the model that\npersist through the fine-tuning stage even though the output embedding is\ndiscarded after pre-training. Our analysis shows that larger output embeddings\nprevent the model's last layers from overspecializing to the pre-training task\nand encourage Transformer representations to be more general and more\ntransferable to other tasks and languages. Harnessing these findings, we are\nable to train models that achieve strong performance on the XTREME benchmark\nwithout increasing the number of parameters at the fine-tuning stage.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12828v1",
            "date": "2020-10-24 08:11:23+00:00",
            "title": "Keyphrase Extraction with Dynamic Graph Convolutional Networks and Diversified Inference",
            "authors": [
                "Haoyu Zhang",
                "Dingkun Long",
                "Guangwei Xu",
                "Pengjun Xie",
                "Fei Huang",
                "Ji Wang"
            ],
            "abstract": "Keyphrase extraction (KE) aims to summarize a set of phrases that accurately\nexpress a concept or a topic covered in a given document. Recently,\nSequence-to-Sequence (Seq2Seq) based generative framework is widely used in KE\ntask, and it has obtained competitive performance on various benchmarks. The\nmain challenges of Seq2Seq methods lie in acquiring informative latent document\nrepresentation and better modeling the compositionality of the target\nkeyphrases set, which will directly affect the quality of generated keyphrases.\nIn this paper, we propose to adopt the Dynamic Graph Convolutional Networks\n(DGCN) to solve the above two problems simultaneously. Concretely, we explore\nto integrate dependency trees with GCN for latent representation learning.\nMoreover, the graph structure in our model is dynamically modified during the\nlearning process according to the generated keyphrases. To this end, our\napproach is able to explicitly learn the relations within the keyphrases\ncollection and guarantee the information interchange between encoder and\ndecoder in both directions. Extensive experiments on various KE benchmark\ndatasets demonstrate the effectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12844v2",
            "date": "2020-10-24 09:11:26+00:00",
            "title": "FLIN: A Flexible Natural Language Interface for Web Navigation",
            "authors": [
                "Sahisnu Mazumder",
                "Oriana Riva"
            ],
            "abstract": "AI assistants can now carry out tasks for users by directly interacting with\nwebsite UIs. Current semantic parsing and slot-filling techniques cannot\nflexibly adapt to many different websites without being constantly re-trained.\nWe propose FLIN, a natural language interface for web navigation that maps user\ncommands to concept-level actions (rather than low-level UI actions), thus\nbeing able to flexibly adapt to different websites and handle their transient\nnature. We frame this as a ranking problem: given a user command and a webpage,\nFLIN learns to score the most relevant navigation instruction (involving action\nand parameter values). To train and evaluate FLIN, we collect a dataset using\nnine popular websites from three domains. Our results show that FLIN was able\nto adapt to new websites in a given domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12854v1",
            "date": "2020-10-24 09:53:16+00:00",
            "title": "ReadOnce Transformers: Reusable Representations of Text for Transformers",
            "authors": [
                "Shih-Ting Lin",
                "Ashish Sabharwal",
                "Tushar Khot"
            ],
            "abstract": "While large-scale language models are extremely effective when directly\nfine-tuned on many end-tasks, such models learn to extract information and\nsolve the task simultaneously from end-task supervision. This is wasteful, as\nthe general problem of gathering information from a document is mostly\ntask-independent and need not be re-learned from scratch each time. Moreover,\nonce the information has been captured in a computable representation, it can\nnow be re-used across examples, leading to faster training and evaluation of\nmodels. We present a transformer-based approach, ReadOnce Transformers, that is\ntrained to build such information-capturing representations of text. Our model\ncompresses the document into a variable-length task-independent representation\nthat can now be re-used in different examples and tasks, thereby requiring a\ndocument to only be read once. Additionally, we extend standard text-to-text\nmodels to consume our ReadOnce Representations along with text to solve\nmultiple downstream tasks. We show our task-independent representations can be\nused for multi-hop QA, abstractive QA, and summarization. We observe 2x-5x\nspeedups compared to standard text-to-text models, while also being able to\nhandle long documents that would normally exceed the length limit of current\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12864v2",
            "date": "2020-10-24 10:36:11+00:00",
            "title": "On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning",
            "authors": [
                "Xisen Jin",
                "Francesco Barbieri",
                "Brendan Kennedy",
                "Aida Mostafazadeh Davani",
                "Leonardo Neves",
                "Xiang Ren"
            ],
            "abstract": "Fine-tuned language models have been shown to exhibit biases against\nprotected groups in a host of modeling tasks such as text classification and\ncoreference resolution. Previous works focus on detecting these biases,\nreducing bias in data representations, and using auxiliary training objectives\nto mitigate bias during fine-tuning. Although these techniques achieve bias\nreduction for the task and domain at hand, the effects of bias mitigation may\nnot directly transfer to new tasks, requiring additional data collection and\ncustomized annotation of sensitive attributes, and re-evaluation of appropriate\nfairness metrics. We explore the feasibility and benefits of upstream bias\nmitigation (UBM) for reducing bias on downstream tasks, by first applying bias\nmitigation to an upstream model through fine-tuning and subsequently using it\nfor downstream fine-tuning. We find, in extensive experiments across hate\nspeech detection, toxicity detection, occupation prediction, and coreference\nresolution tasks over various bias factors, that the effects of UBM are indeed\ntransferable to new downstream tasks or domains via fine-tuning, creating less\nbiased downstream models than directly fine-tuning on the downstream task or\ntransferring from a vanilla upstream model. Though challenges remain, we show\nthat UBM promises more efficient and accessible bias mitigation in LM\nfine-tuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12871v1",
            "date": "2020-10-24 11:03:01+00:00",
            "title": "Large Scale Legal Text Classification Using Transformer Models",
            "authors": [
                "Zein Shaheen",
                "Gerhard Wohlgenannt",
                "Erwin Filtz"
            ],
            "abstract": "Large multi-label text classification is a challenging Natural Language\nProcessing (NLP) problem that is concerned with text classification for\ndatasets with thousands of labels. We tackle this problem in the legal domain,\nwhere datasets, such as JRC-Acquis and EURLEX57K labeled with the EuroVoc\nvocabulary were created within the legal information systems of the European\nUnion. The EuroVoc taxonomy includes around 7000 concepts. In this work, we\nstudy the performance of various recent transformer-based models in combination\nwith strategies such as generative pretraining, gradual unfreezing and\ndiscriminative learning rates in order to reach competitive classification\nperformance, and present new state-of-the-art results of 0.661 (F1) for\nJRC-Acquis and 0.754 for EURLEX57K. Furthermore, we quantify the impact of\nindividual steps, such as language model fine-tuning or gradual unfreezing in\nan ablation study, and provide reference dataset splits created with an\niterative stratification algorithm.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12912v1",
            "date": "2020-10-24 15:03:20+00:00",
            "title": "Word Embeddings for Chemical Patent Natural Language Processing",
            "authors": [
                "Camilo Thorne",
                "Saber Akhondi"
            ],
            "abstract": "We evaluate chemical patent word embeddings against known biomedical\nembeddings and show that they outperform the latter extrinsically and\nintrinsically. We also show that using contextualized embeddings can induce\npredictive models of reasonable performance for this domain over a relatively\nsmall gold standard.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12919v4",
            "date": "2020-10-24 15:43:37+00:00",
            "title": "Causal Effects of Linguistic Properties",
            "authors": [
                "Reid Pryzant",
                "Dallas Card",
                "Dan Jurafsky",
                "Victor Veitch",
                "Dhanya Sridhar"
            ],
            "abstract": "We consider the problem of using observational data to estimate the causal\neffects of linguistic properties. For example, does writing a complaint\npolitely lead to a faster response time? How much will a positive product\nreview increase sales? This paper addresses two technical challenges related to\nthe problem before developing a practical method. First, we formalize the\ncausal quantity of interest as the effect of a writer's intent, and establish\nthe assumptions necessary to identify this from observational data. Second, in\npractice, we only have access to noisy proxies for the linguistic properties of\ninterest -- e.g., predictions from classifiers and lexicons. We propose an\nestimator for this setting and prove that its bias is bounded when we perform\nan adjustment for the text. Based on these results, we introduce TextCause, an\nalgorithm for estimating causal effects of linguistic properties. The method\nleverages (1) distant supervision to improve the quality of noisy proxies, and\n(2) a pre-trained language model (BERT) to adjust for the text. We show that\nthe proposed method outperforms related approaches when estimating the effect\nof Amazon review sentiment on semi-simulated sales figures. Finally, we present\nan applied case study investigating the effects of complaint politeness on\nbureaucratic response times.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.12925v1",
            "date": "2020-10-24 16:25:05+00:00",
            "title": "Disease Normalization with Graph Embeddings",
            "authors": [
                "Dhruba Pujary",
                "Camilo Thorne",
                "Wilker Aziz"
            ],
            "abstract": "The detection and normalization of diseases in biomedical texts are key\nbiomedical natural language processing tasks. Disease names need not only be\nidentified, but also normalized or linked to clinical taxonomies describing\ndiseases such as MeSH. In this paper we describe deep learning methods that\ntackle both tasks. We train and test our methods on the known NCBI disease\nbenchmark corpus. We propose to represent disease names by leveraging MeSH's\ngraphical structure together with the lexical information available in the\ntaxonomy using graph embeddings. We also show that combining neural named\nentity recognition models with our graph-based entity linking methods via\nmultitask learning leads to improved disease recognition in the NCBI corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13002v2",
            "date": "2020-10-24 23:15:43+00:00",
            "title": "Pre-trained Summarization Distillation",
            "authors": [
                "Sam Shleifer",
                "Alexander M. Rush"
            ],
            "abstract": "Recent state-of-the-art approaches to summarization utilize large pre-trained\nTransformer models. Distilling these models to smaller student models has\nbecome critically important for practical use; however there are many different\ndistillation methods proposed by the NLP literature. Recent work on distilling\nBERT for classification and regression tasks shows strong performance using\ndirect knowledge distillation. Alternatively, machine translation practitioners\ndistill using pseudo-labeling, where a small model is trained on the\ntranslations of a larger model. A third, simpler approach is to 'shrink and\nfine-tune' (SFT), which avoids any explicit distillation by copying parameters\nto a smaller student model and then fine-tuning. We compare these three\napproaches for distillation of Pegasus and BART, the current and former state\nof the art, pre-trained summarization models, and find that SFT outperforms\nknowledge distillation and pseudo-labeling on the CNN/DailyMail dataset, but\nunder-performs pseudo-labeling on the more abstractive XSUM dataset. PyTorch\nCode and checkpoints of different sizes are available through Hugging Face\ntransformers here http://tiny.cc/4iy0tz.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13009v1",
            "date": "2020-10-25 00:39:32+00:00",
            "title": "Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference",
            "authors": [
                "Jian-Guo Zhang",
                "Kazuma Hashimoto",
                "Wenhao Liu",
                "Chien-Sheng Wu",
                "Yao Wan",
                "Philip S. Yu",
                "Richard Socher",
                "Caiming Xiong"
            ],
            "abstract": "Intent detection is one of the core components of goal-oriented dialog\nsystems, and detecting out-of-scope (OOS) intents is also a practically\nimportant skill. Few-shot learning is attracting much attention to mitigate\ndata scarcity, but OOS detection becomes even more challenging. In this paper,\nwe present a simple yet effective approach, discriminative nearest neighbor\nclassification with deep self-attention. Unlike softmax classifiers, we\nleverage BERT-style pairwise encoding to train a binary classifier that\nestimates the best matched training example for a user input. We propose to\nboost the discriminative ability by transferring a natural language inference\n(NLI) model. Our extensive experiments on a large-scale multi-domain intent\ndetection task show that our method achieves more stable and accurate in-domain\nand OOS detection accuracy than RoBERTa-based classifiers and embedding-based\nnearest neighbor approaches. More notably, the NLI transfer enables our 10-shot\nmodel to perform competitively with 50-shot or even full-shot classifiers,\nwhile we can keep the inference time constant by leveraging a faster embedding\nretrieval model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13168v1",
            "date": "2020-10-25 17:31:12+00:00",
            "title": "Fair Embedding Engine: A Library for Analyzing and Mitigating Gender Bias in Word Embeddings",
            "authors": [
                "Vaibhav Kumar",
                "Tenzin Singhay Bhotia",
                "Vaibhav Kumar"
            ],
            "abstract": "Non-contextual word embedding models have been shown to inherit human-like\nstereotypical biases of gender, race and religion from the training corpora. To\ncounter this issue, a large body of research has emerged which aims to mitigate\nthese biases while keeping the syntactic and semantic utility of embeddings\nintact. This paper describes Fair Embedding Engine (FEE), a library for\nanalysing and mitigating gender bias in word embeddings. FEE combines various\nstate of the art techniques for quantifying, visualising and mitigating gender\nbias in word embeddings under a standard abstraction. FEE will aid\npractitioners in fast track analysis of existing debiasing methods on their\nembedding models. Further, it will allow rapid prototyping of new methods by\nevaluating their performance on a suite of standard metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13374v4",
            "date": "2020-10-26 07:03:14+00:00",
            "title": "LXPER Index 2.0: Improving Text Readability Assessment Model for L2 English Students in Korea",
            "authors": [
                "Bruce W. Lee",
                "Jason Lee"
            ],
            "abstract": "Developing a text readability assessment model specifically for texts in a\nforeign English Language Training (ELT) curriculum has never had much attention\nin the field of Natural Language Processing. Hence, most developed models show\nextremely low accuracy for L2 English texts, up to the point where not many\neven serve as a fair comparison. In this paper, we investigate a text\nreadability assessment model for L2 English learners in Korea. In accordance,\nwe improve and expand the Text Corpus of the Korean ELT curriculum\n(CoKEC-text). Each text is labeled with its target grade level. We train our\nmodel with CoKEC-text and significantly improve the accuracy of readability\nassessment for texts in the Korean ELT curriculum.",
            "journal ref": "Proceedings of the 6th Workshop on Natural Language Processing\n  Techniques for Educational Applications, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13404v3",
            "date": "2020-10-26 08:00:48+00:00",
            "title": "Robust and Consistent Estimation of Word Embedding for Bangla Language by fine-tuning Word2Vec Model",
            "authors": [
                "Rifat Rahman"
            ],
            "abstract": "Word embedding or vector representation of word holds syntactical and\nsemantic characteristics of a word which can be an informative feature for any\nmachine learning-based models of natural language processing. There are several\ndeep learning-based models for the vectorization of words like word2vec,\nfasttext, gensim, glove, etc. In this study, we analyze word2vec model for\nlearning word vectors by tuning different hyper-parameters and present the most\neffective word embedding for Bangla language. For testing the performances of\ndifferent word embeddings generated by fine-tuning of word2vec model, we\nperform both intrinsic and extrinsic evaluations. We cluster the word vectors\nto examine the relational similarity of words for intrinsic evaluation and also\nuse different word embeddings as the feature of news article classifier for\nextrinsic evaluation. From our experiment, we discover that the word vectors\nwith 300 dimensions, generated from \"skip-gram\" method of word2vec model using\nthe sliding window size of 4, are giving the most robust vector representations\nfor Bangla language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13556v2",
            "date": "2020-10-26 13:07:56+00:00",
            "title": "Hierarchical Metadata-Aware Document Categorization under Weak Supervision",
            "authors": [
                "Yu Zhang",
                "Xiusi Chen",
                "Yu Meng",
                "Jiawei Han"
            ],
            "abstract": "Categorizing documents into a given label hierarchy is intuitively appealing\ndue to the ubiquity of hierarchical topic structures in massive text corpora.\nAlthough related studies have achieved satisfying performance in fully\nsupervised hierarchical document classification, they usually require massive\nhuman-annotated training data and only utilize text information. However, in\nmany domains, (1) annotations are quite expensive where very few training\nsamples can be acquired; (2) documents are accompanied by metadata information.\nHence, this paper studies how to integrate the label hierarchy, metadata, and\ntext signals for document categorization under weak supervision. We develop\nHiMeCat, an embedding-based generative framework for our task. Specifically, we\npropose a novel joint representation learning module that allows simultaneous\nmodeling of category dependencies, metadata information and textual semantics,\nand we introduce a data augmentation module that hierarchically synthesizes\ntraining documents to complement the original, small-scale training set. Our\nexperiments demonstrate a consistent improvement of HiMeCat over competitive\nbaselines and validate the contribution of our representation learning and data\naugmentation modules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13609v2",
            "date": "2020-10-26 14:28:29+00:00",
            "title": "UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models",
            "authors": [
                "Mircea-Adrian Tanase",
                "Dumitru-Clementin Cercel",
                "Costin-Gabriel Chiru"
            ],
            "abstract": "Offensive language detection is one of the most challenging problem in the\nnatural language processing field, being imposed by the rising presence of this\nphenomenon in online social media. This paper describes our Transformer-based\nsolutions for identifying offensive language on Twitter in five languages\n(i.e., English, Arabic, Danish, Greek, and Turkish), which was employed in\nSubtask A of the Offenseval 2020 shared task. Several neural architectures\n(i.e., BERT, mBERT, Roberta, XLM-Roberta, and ALBERT), pre-trained using both\nsingle-language and multilingual corpora, were fine-tuned and compared using\nmultiple combinations of datasets. Finally, the highest-scoring models were\nused for our submissions in the competition, which ranked our team 21st of 85,\n28th of 53, 19th of 39, 16th of 37, and 10th of 46 for English, Arabic, Danish,\nGreek, and Turkish, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13652v1",
            "date": "2020-10-26 15:15:10+00:00",
            "title": "Dutch Humor Detection by Generating Negative Examples",
            "authors": [
                "Thomas Winters",
                "Pieter Delobelle"
            ],
            "abstract": "Detecting if a text is humorous is a hard task to do computationally, as it\nusually requires linguistic and common sense insights. In machine learning,\nhumor detection is usually modeled as a binary classification task, trained to\npredict if the given text is a joke or another type of text. Rather than using\ncompletely different non-humorous texts, we propose using text generation\nalgorithms for imitating the original joke dataset to increase the difficulty\nfor the learning algorithm. We constructed several different joke and non-joke\ndatasets to test the humor detection abilities of different language\ntechnologies. In particular, we compare the humor detection capabilities of\nclassic neural network approaches with the state-of-the-art Dutch language\nmodel RobBERT. In doing so, we create and compare the first Dutch humor\ndetection systems. We found that while other language models perform well when\nthe non-jokes came from completely different domains, RobBERT was the only one\nthat was able to distinguish jokes from generated negative examples. This\nperformance illustrates the usefulness of using text generation to create\nnegative datasets for humor recognition, and also shows that transformer models\nare a large step forward in humor detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13658v1",
            "date": "2020-10-26 15:27:51+00:00",
            "title": "Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval",
            "authors": [
                "Tianchi Bi",
                "Liang Yao",
                "Baosong Yang",
                "Haibo Zhang",
                "Weihua Luo",
                "Boxing Chen"
            ],
            "abstract": "Query translation (QT) is a key component in cross-lingual information\nretrieval system (CLIR). With the help of deep learning, neural machine\ntranslation (NMT) has shown promising results on various tasks. However, NMT is\ngenerally trained with large-scale out-of-domain data rather than in-domain\nquery translation pairs. Besides, the translation model lacks a mechanism at\nthe inference time to guarantee the generated words to match the search index.\nThe two shortages of QT result in readable texts for human but inadequate\ncandidates for the downstream retrieval task. In this paper, we propose a novel\napproach to alleviate these problems by limiting the open target vocabulary\nsearch space of QT to a set of important words mined from search index\ndatabase. The constraint translation candidates are employed at both of\ntraining and inference time, thus guiding the translation model to learn and\ngenerate well performing target queries. The proposed methods are exploited and\nexamined in a real-word CLIR system--Aliexpress e-Commerce search engine.\nExperimental results demonstrate that our approach yields better performance on\nboth translation quality and retrieval accuracy than the strong NMT baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13659v1",
            "date": "2020-10-26 15:28:19+00:00",
            "title": "Exploiting Neural Query Translation into Cross Lingual Information Retrieval",
            "authors": [
                "Liang Yao",
                "Baosong Yang",
                "Haibo Zhang",
                "Weihua Luo",
                "Boxing Chen"
            ],
            "abstract": "As a crucial role in cross-language information retrieval (CLIR), query\ntranslation has three main challenges: 1) the adequacy of translation; 2) the\nlack of in-domain parallel training data; and 3) the requisite of low latency.\nTo this end, existing CLIR systems mainly exploit statistical-based machine\ntranslation (SMT) rather than the advanced neural machine translation (NMT),\nlimiting the further improvements on both translation and retrieval quality. In\nthis paper, we investigate how to exploit neural query translation model into\nCLIR system. Specifically, we propose a novel data augmentation method that\nextracts query translation pairs according to user clickthrough data, thus to\nalleviate the problem of domain-adaptation in NMT. Then, we introduce an\nasynchronous strategy which is able to leverage the advantages of the real-time\nin SMT and the veracity in NMT. Experimental results reveal that the proposed\napproach yields better retrieval quality than strong baselines and can be well\napplied into a real-world CLIR system, i.e. Aliexpress e-Commerce search\nengine. Readers can examine and test their cases on our website:\nhttps://aliexpress.com .",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13674v1",
            "date": "2020-10-26 15:52:12+00:00",
            "title": "A Corpus for Argumentative Writing Support in German",
            "authors": [
                "Thiemo Wambsganss",
                "Christina Niklaus",
                "Matthias S\u00f6llner",
                "Siegfried Handschuh",
                "Jan Marco Leimeister"
            ],
            "abstract": "In this paper, we present a novel annotation approach to capture claims and\npremises of arguments and their relations in student-written persuasive peer\nreviews on business models in German language. We propose an annotation scheme\nbased on annotation guidelines that allows to model claims and premises as well\nas support and attack relations for capturing the structure of argumentative\ndiscourse in student-written peer reviews. We conduct an annotation study with\nthree annotators on 50 persuasive essays to evaluate our annotation scheme. The\nobtained inter-rater agreement of $\\alpha=0.57$ for argument components and\n$\\alpha=0.49$ for argumentative relations indicates that the proposed\nannotation scheme successfully guides annotators to moderate agreement.\nFinally, we present our freely available corpus of 1,000 persuasive\nstudent-written peer reviews on business models and our annotation guidelines\nto encourage future research on the design and development of argumentative\nwriting support systems for students.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13688v1",
            "date": "2020-10-26 16:08:13+00:00",
            "title": "A Survey of Embedding Space Alignment Methods for Language and Knowledge Graphs",
            "authors": [
                "Alexander Kalinowski",
                "Yuan An"
            ],
            "abstract": "Neural embedding approaches have become a staple in the fields of computer\nvision, natural language processing, and more recently, graph analytics. Given\nthe pervasive nature of these algorithms, the natural question becomes how to\nexploit the embedding spaces to map, or align, embeddings of different data\nsources. To this end, we survey the current research landscape on word,\nsentence and knowledge graph embedding algorithms. We provide a classification\nof the relevant alignment techniques and discuss benchmark datasets used in\nthis field of research. By gathering these diverse approaches into a singular\nsurvey, we hope to further motivate research into alignment of embedding spaces\nof varied data types and sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13816v1",
            "date": "2020-10-26 18:05:48+00:00",
            "title": "PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction",
            "authors": [
                "Xinyao Ma",
                "Maarten Sap",
                "Hannah Rashkin",
                "Yejin Choi"
            ],
            "abstract": "Unconscious biases continue to be prevalent in modern text and media, calling\nfor algorithms that can assist writers with bias correction. For example, a\nfemale character in a story is often portrayed as passive and powerless (\"She\ndaydreams about being a doctor\") while a man is portrayed as more proactive and\npowerful (\"He pursues his dream of being a doctor\").\n  We formulate *Controllable Debiasing*, a new revision task that aims to\nrewrite a given text to correct the implicit and potentially undesirable bias\nin character portrayals. We then introduce PowerTransformer as an approach that\ndebiases text through the lens of connotation frames (Sap et al., 2017), which\nencode pragmatic knowledge of implied power dynamics with respect to verb\npredicates. One key challenge of our task is the lack of parallel corpora. To\naddress this challenge, we adopt an unsupervised approach using auxiliary\nsupervision with related tasks such as paraphrasing and self-supervision based\non a reconstruction loss, building on pretrained language models.\n  Through comprehensive experiments based on automatic and human evaluations,\nwe demonstrate that our approach outperforms ablations and existing methods\nfrom related tasks. Furthermore, we demonstrate the use of PowerTransformer as\na step toward mitigating the well-documented gender bias in character portrayal\nin movie scripts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13839v1",
            "date": "2020-10-26 18:51:02+00:00",
            "title": "VisualHints: A Visual-Lingual Environment for Multimodal Reinforcement Learning",
            "authors": [
                "Thomas Carta",
                "Subhajit Chaudhury",
                "Kartik Talamadupula",
                "Michiaki Tatsubori"
            ],
            "abstract": "We present VisualHints, a novel environment for multimodal reinforcement\nlearning (RL) involving text-based interactions along with visual hints\n(obtained from the environment). Real-life problems often demand that agents\ninteract with the environment using both natural language information and\nvisual perception towards solving a goal. However, most traditional RL\nenvironments either solve pure vision-based tasks like Atari games or\nvideo-based robotic manipulation; or entirely use natural language as a mode of\ninteraction, like Text-based games and dialog systems. In this work, we aim to\nbridge this gap and unify these two approaches in a single environment for\nmultimodal RL. We introduce an extension of the TextWorld cooking environment\nwith the addition of visual clues interspersed throughout the environment. The\ngoal is to force an RL agent to use both text and visual features to predict\nnatural language action commands for solving the final task of cooking a meal.\nWe enable variations and difficulties in our environment to emulate various\ninteractive real-world scenarios. We present a baseline multimodal agent for\nsolving such problems using CNN-based feature extraction from visual hints and\nLSTMs for textual feature extraction. We believe that our proposed\nvisual-lingual environment will facilitate novel problem settings for the RL\ncommunity.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2010.13856v1",
            "date": "2020-10-26 19:20:29+00:00",
            "title": "Data Troubles in Sentence Level Confidence Estimation for Machine Translation",
            "authors": [
                "Ciprian Chelba",
                "Junpei Zhou",
                "Yuezhang",
                "Li",
                "Hideto Kazawa",
                "Jeff Klingner",
                "Mengmeng Niu"
            ],
            "abstract": "The paper investigates the feasibility of confidence estimation for neural\nmachine translation models operating at the high end of the performance\nspectrum. As a side product of the data annotation process necessary for\nbuilding such models we propose sentence level accuracy $SACC$ as a simple,\nself-explanatory evaluation metric for quality of translation.\n  Experiments on two different annotator pools, one comprised of non-expert\n(crowd-sourced) and one of expert (professional) translators show that $SACC$\ncan vary greatly depending on the translation proficiency of the annotators,\ndespite the fact that both pools are about equally reliable according to\nKrippendorff's alpha metric; the relatively low values of inter-annotator\nagreement confirm the expectation that sentence-level binary labeling $good$ /\n$needs\\ work$ for translation out of context is very hard.\n  For an English-Spanish translation model operating at $SACC = 0.89$ according\nto a non-expert annotator pool we can derive a confidence estimate that labels\n0.5-0.6 of the $good$ translations in an \"in-domain\" test set with 0.95\nPrecision. Switching to an expert annotator pool decreases $SACC$ dramatically:\n$0.61$ for English-Spanish, measured on the exact same data as above. This\nforces us to lower the CE model operating point to 0.9 Precision while labeling\ncorrectly about 0.20-0.25 of the $good$ translations in the data.\n  We find surprising the extent to which CE depends on the level of proficiency\nof the annotator pool used for labeling the data. This leads to an important\nrecommendation we wish to make when tackling CE modeling in practice: it is\ncritical to match the end-user expectation for translation quality in the\ndesired domain with the demands of annotators assigning binary quality labels\nto CE training data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13912v1",
            "date": "2020-10-26 21:34:39+00:00",
            "title": "Probing Task-Oriented Dialogue Representation from Language Models",
            "authors": [
                "Chien-Sheng Wu",
                "Caiming Xiong"
            ],
            "abstract": "This paper investigates pre-trained language models to find out which model\nintrinsically carries the most informative representation for task-oriented\ndialogue tasks. We approach the problem from two aspects: supervised classifier\nprobe and unsupervised mutual information probe. We fine-tune a feed-forward\nlayer as the classifier probe on top of a fixed pre-trained language model with\nannotated labels in a supervised way. Meanwhile, we propose an unsupervised\nmutual information probe to evaluate the mutual dependence between a real\nclustering and a representation clustering. The goals of this empirical paper\nare to 1) investigate probing techniques, especially from the unsupervised\nmutual information aspect, 2) provide guidelines of pre-trained language model\nselection for the dialogue research community, 3) find insights of pre-training\nfactors for dialogue application that may be the key to success.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13920v1",
            "date": "2020-10-26 21:57:42+00:00",
            "title": "Improving Limited Labeled Dialogue State Tracking with Self-Supervision",
            "authors": [
                "Chien-Sheng Wu",
                "Steven Hoi",
                "Caiming Xiong"
            ],
            "abstract": "Existing dialogue state tracking (DST) models require plenty of labeled data.\nHowever, collecting high-quality labels is costly, especially when the number\nof domains increases. In this paper, we address a practical DST problem that is\nrarely discussed, i.e., learning efficiently with limited labeled data. We\npresent and investigate two self-supervised objectives: preserving latent\nconsistency and modeling conversational behavior. We encourage a DST model to\nhave consistent latent distributions given a perturbed input, making it more\nrobust to an unseen scenario. We also add an auxiliary utterance generation\ntask, modeling a potential correlation between conversational behavior and\ndialogue states. The experimental results show that our proposed\nself-supervised signals can improve joint goal accuracy by 8.95\\% when only 1\\%\nlabeled data is used on the MultiWOZ dataset. We can achieve an additional\n1.76\\% improvement if some unlabeled data is jointly trained as semi-supervised\nlearning. We analyze and visualize how our proposed self-supervised signals\nhelp the DST task and hope to stimulate future data-efficient DST research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13982v2",
            "date": "2020-10-27 01:31:42+00:00",
            "title": "Predict and Use Latent Patterns for Short-Text Conversation",
            "authors": [
                "Hung-Ting Chen",
                "Yu-Chieh Chao",
                "Ta-Hsuan Chao",
                "Wei-Yun Ma"
            ],
            "abstract": "Many neural network models nowadays have achieved promising performances in\nChit-chat settings. The majority of them rely on an encoder for understanding\nthe post and a decoder for generating the response. Without given assigned\nsemantics, the models lack the fine-grained control over responses as the\nsemantic mapping between posts and responses is hidden on the fly within the\nend-to-end manners. Some previous works utilize sampled latent words as a\ncontrollable semantic form to drive the generated response around the work, but\nfew works attempt to use more complex semantic patterns to guide the\ngeneration. In this paper, we propose to use more detailed semantic forms,\nincluding latent responses and part-of-speech sequences sampled from the\ncorresponding distributions, as the controllable semantics to guide the\ngeneration. Our results show that the richer semantics are not only able to\nprovide informative and diverse responses, but also increase the overall\nperformance of response quality, including fluency and coherence.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.13984v1",
            "date": "2020-10-27 01:40:41+00:00",
            "title": "Interpretation of NLP models through input marginalization",
            "authors": [
                "Siwon Kim",
                "Jihun Yi",
                "Eunji Kim",
                "Sungroh Yoon"
            ],
            "abstract": "To demystify the \"black box\" property of deep neural networks for natural\nlanguage processing (NLP), several methods have been proposed to interpret\ntheir predictions by measuring the change in prediction probability after\nerasing each token of an input. Since existing methods replace each token with\na predefined value (i.e., zero), the resulting sentence lies out of the\ntraining data distribution, yielding misleading interpretations. In this study,\nwe raise the out-of-distribution problem induced by the existing interpretation\nmethods and present a remedy; we propose to marginalize each token out. We\ninterpret various NLP models trained for sentiment analysis and natural\nlanguage inference using the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14061v2",
            "date": "2020-10-24 04:54:52+00:00",
            "title": "Jointly Optimizing State Operation Prediction and Value Generation for Dialogue State Tracking",
            "authors": [
                "Yan Zeng",
                "Jian-Yun Nie"
            ],
            "abstract": "We investigate the problem of multi-domain Dialogue State Tracking (DST) with\nopen vocabulary. Existing approaches exploit BERT encoder and copy-based RNN\ndecoder, where the encoder predicts the state operation, and the decoder\ngenerates new slot values. However, in such a stacked encoder-decoder\nstructure, the operation prediction objective only affects the BERT encoder and\nthe value generation objective mainly affects the RNN decoder. In this paper,\nwe propose a purely Transformer-based framework, where a single BERT works as\nboth the encoder and the decoder. In so doing, the operation prediction\nobjective and the value generation objective can jointly optimize this BERT for\nDST. At the decoding step, we re-use the hidden states of the encoder in the\nself-attention mechanism of the corresponding decoder layers to construct a\nflat encoder-decoder architecture for effective parameter updating.\nExperimental results show that our approach substantially outperforms the\nexisting state-of-the-art framework, and it also achieves very competitive\nperformance to the best ontology-based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14235v1",
            "date": "2020-10-27 12:10:19+00:00",
            "title": "Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles",
            "authors": [
                "Yao Lu",
                "Yue Dong",
                "Laurent Charlin"
            ],
            "abstract": "Multi-document summarization is a challenging task for which there exists\nlittle large-scale datasets. We propose Multi-XScience, a large-scale\nmulti-document summarization dataset created from scientific articles.\nMulti-XScience introduces a challenging multi-document summarization task:\nwriting the related-work section of a paper based on its abstract and the\narticles it references. Our work is inspired by extreme summarization, a\ndataset construction protocol that favours abstractive modeling approaches.\nDescriptive statistics and empirical results---using several state-of-the-art\nmodels trained on the Multi-XScience dataset---reveal that Multi-XScience is\nwell suited for abstractive models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14255v2",
            "date": "2020-10-27 12:50:27+00:00",
            "title": "RH-Net: Improving Neural Relation Extraction via Reinforcement Learning and Hierarchical Relational Searching",
            "authors": [
                "Jianing Wang"
            ],
            "abstract": "Distant supervision (DS) aims to generate large-scale heuristic labeling\ncorpus, which is widely used for neural relation extraction currently. However,\nit heavily suffers from noisy labeling and long-tail distributions problem.\nMany advanced approaches usually separately address two problems, which ignore\ntheir mutual interactions. In this paper, we propose a novel framework named\nRH-Net, which utilizes Reinforcement learning and Hierarchical relational\nsearching module to improve relation extraction. We leverage reinforcement\nlearning to instruct the model to select high-quality instances. We then\npropose the hierarchical relational searching module to share the semantics\nfrom correlative instances between data-rich and data-poor classes. During the\niterative process, the two modules keep interacting to alleviate the noisy and\nlong-tail problem simultaneously. Extensive experiments on widely used NYT data\nset clearly show that our method significant improvements over state-of-the-art\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14271v1",
            "date": "2020-10-27 13:12:17+00:00",
            "title": "Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation",
            "authors": [
                "Junhao Liu",
                "Linjun Shou",
                "Jian Pei",
                "Ming Gong",
                "Min Yang",
                "Daxin Jiang"
            ],
            "abstract": "Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging\nproblem due to the lack of large-scale annotated datasets in low-source\nlanguages, such as Arabic, Hindi, and Vietnamese. Many previous approaches use\ntranslation data by translating from a rich-source language, such as English,\nto low-source languages as auxiliary supervision. However, how to effectively\nleverage translation data and reduce the impact of noise introduced by\ntranslation remains onerous. In this paper, we tackle this challenge and\nenhance the cross-lingual transferring performance by a novel augmentation\napproach named Language Branch Machine Reading Comprehension (LBMRC). A\nlanguage branch is a group of passages in one single language paired with\nquestions in all target languages. We train multiple machine reading\ncomprehension (MRC) models proficient in individual language based on LBMRC.\nThen, we devise a multilingual distillation approach to amalgamate knowledge\nfrom multiple language branch models to a single model for all target\nlanguages. Combining the LBMRC and multilingual distillation can be more robust\nto the data noises, therefore, improving the model's cross-lingual ability.\nMeanwhile, the produced single multilingual model is applicable to all target\nlanguages, which saves the cost of training, inference, and maintenance for\nmultiple models. Extensive experiments on two CLMRC benchmarks clearly show the\neffectiveness of our proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14481v1",
            "date": "2020-10-27 17:38:51+00:00",
            "title": "Fast Interleaved Bidirectional Sequence Generation",
            "authors": [
                "Biao Zhang",
                "Ivan Titov",
                "Rico Sennrich"
            ],
            "abstract": "Independence assumptions during sequence generation can speed up inference,\nbut parallel generation of highly inter-dependent tokens comes at a cost in\nquality. Instead of assuming independence between neighbouring tokens\n(semi-autoregressive decoding, SA), we take inspiration from bidirectional\nsequence generation and introduce a decoder that generates target words from\nthe left-to-right and right-to-left directions simultaneously. We show that we\ncan easily convert a standard architecture for unidirectional decoding into a\nbidirectional decoder by simply interleaving the two directions and adapting\nthe word positions and self-attention masks. Our interleaved bidirectional\ndecoder (IBDecoder) retains the model simplicity and training efficiency of the\nstandard Transformer, and on five machine translation tasks and two document\nsummarization tasks, achieves a decoding speedup of ~2X compared to\nautoregressive decoding with comparable quality. Notably, it outperforms\nleft-to-right SA because the independence assumptions in IBDecoder are more\nfelicitous. To achieve even higher speedups, we explore hybrid models where we\neither simultaneously predict multiple neighbouring tokens per direction, or\nperform multi-directional decoding by partitioning the target sequence. These\nmethods achieve speedups to 4X-11X across different tasks at the cost of <1\nBLEU or <0.5 ROUGE (on average). Source code is released at\nhttps://github.com/bzhangGo/zero.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14557v1",
            "date": "2020-10-27 18:54:51+00:00",
            "title": "DGST: a Dual-Generator Network for Text Style Transfer",
            "authors": [
                "Xiao Li",
                "Guanyi Chen",
                "Chenghua Lin",
                "Ruizhe Li"
            ],
            "abstract": "We propose DGST, a novel and simple Dual-Generator network architecture for\ntext Style Transfer. Our model employs two generators only, and does not rely\non any discriminators or parallel corpus for training. Both quantitative and\nqualitative experiments on the Yelp and IMDb datasets show that our model gives\ncompetitive performance compared to several strong baselines with more\ncomplicated architecture designs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14571v2",
            "date": "2020-10-27 19:29:17+00:00",
            "title": "Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus",
            "authors": [
                "Isaac Caswell",
                "Theresa Breiner",
                "Daan van Esch",
                "Ankur Bapna"
            ],
            "abstract": "Large text corpora are increasingly important for a wide variety of Natural\nLanguage Processing (NLP) tasks, and automatic language identification (LangID)\nis a core technology needed to collect such datasets in a multilingual context.\nLangID is largely treated as solved in the literature, with models reported\nthat achieve over 90% average F1 on as many as 1,366 languages. We train LangID\nmodels on up to 1,629 languages with comparable quality on held-out test sets,\nbut find that human-judged LangID accuracy for web-crawl text corpora created\nusing these models is only around 5% for many lower-resource languages,\nsuggesting a need for more robust evaluation. Further analysis revealed a\nvariety of error modes, arising from domain mismatch, class imbalance, language\nsimilarity, and insufficiently expressive models. We propose two classes of\ntechniques to mitigate these errors: wordlist-based tunable-precision filters\n(for which we release curated lists in about 500 languages) and\ntransformer-based semi-supervised LangID models, which increase median dataset\nprecision from 5.5% to 71.2%. These techniques enable us to create an initial\ndata set covering 100K or more relatively clean sentences in each of 500+\nlanguages, paving the way towards a 1,000-language web text corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14588v1",
            "date": "2020-10-27 19:51:53+00:00",
            "title": "A Comprehensive Dictionary and Term Variation Analysis for COVID-19 and SARS-CoV-2",
            "authors": [
                "Robert Leaman",
                "Zhiyong Lu"
            ],
            "abstract": "The number of unique terms in the scientific literature used to refer to\neither SARS-CoV-2 or COVID-19 is remarkably large and has continued to increase\nrapidly despite well-established standardized terms. This high degree of term\nvariation makes high recall identification of these important entities\ndifficult. In this manuscript we present an extensive dictionary of terms used\nin the literature to refer to SARS-CoV-2 and COVID-19. We use a rule-based\napproach to iteratively generate new term variants, then locate these variants\nin a large text corpus. We compare our dictionary to an extensive collection of\nterminological resources, demonstrating that our resource provides a\nsubstantial number of additional terms. We use our dictionary to analyze the\nusage of SARS-CoV-2 and COVID-19 terms over time and show that the number of\nunique terms continues to grow rapidly. Our dictionary is freely available at\nhttps://github.com/ncbi-nlp/CovidTermVar.",
            "journal ref": "None",
            "category": "cs.DL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14660v1",
            "date": "2020-10-27 22:56:18+00:00",
            "title": "DualTKB: A Dual Learning Bridge between Text and Knowledge Base",
            "authors": [
                "Pierre L. Dognin",
                "Igor Melnyk",
                "Inkit Padhi",
                "Cicero Nogueira dos Santos",
                "Payel Das"
            ],
            "abstract": "In this work, we present a dual learning approach for unsupervised text to\npath and path to text transfers in Commonsense Knowledge Bases (KBs). We\ninvestigate the impact of weak supervision by creating a weakly supervised\ndataset and show that even a slight amount of supervision can significantly\nimprove the model performance and enable better-quality transfers. We examine\ndifferent model architectures, and evaluation metrics, proposing a novel\nCommonsense KB completion metric tailored for generative models. Extensive\nexperimental results show that the proposed method compares very favorably to\nthe existing baselines. This approach is a viable step towards a more advanced\nsystem for automatic KB construction/expansion and the reverse operation of KB\nconversion to coherent textual descriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14665v2",
            "date": "2020-10-27 23:04:21+00:00",
            "title": "Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applications",
            "authors": [
                "Yongqiang Wang",
                "Yangyang Shi",
                "Frank Zhang",
                "Chunyang Wu",
                "Julian Chan",
                "Ching-Feng Yeh",
                "Alex Xiao"
            ],
            "abstract": "In this paper, we summarize the application of transformer and its streamable\nvariant, Emformer based acoustic model for large scale speech recognition\napplications. We compare the transformer based acoustic models with their LSTM\ncounterparts on industrial scale tasks. Specifically, we compare Emformer with\nlatency-controlled BLSTM (LCBLSTM) on medium latency tasks and LSTM on low\nlatency tasks. On a low latency voice assistant task, Emformer gets 24% to 26%\nrelative word error rate reductions (WERRs). For medium latency scenarios,\ncomparing with LCBLSTM with similar model size and latency, Emformer gets\nsignificant WERR across four languages in video captioning datasets with 2-3\ntimes inference real-time factors reduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14707v1",
            "date": "2020-10-28 02:33:41+00:00",
            "title": "TopicModel4J: A Java Package for Topic Models",
            "authors": [
                "Yang Qian",
                "Yuanchun Jiang",
                "Yidong Chai",
                "Yezheng Liu",
                "Jiansha Sun"
            ],
            "abstract": "Topic models provide a flexible and principled framework for exploring hidden\nstructure in high-dimensional co-occurrence data and are commonly used natural\nlanguage processing (NLP) of text. In this paper, we design and implement a\nJava package, TopicModel4J, which contains 13 kinds of representative\nalgorithms for fitting topic models. The TopicModel4J in the Java programming\nenvironment provides an easy-to-use interface for data analysts to run the\nalgorithms, and allow to easily input and output data. In addition, this\npackage provides a few unstructured text preprocessing techniques, such as\nsplitting textual data into words, lowercasing the words, preforming\nlemmatization and removing the useless characters, URLs and stop words.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14730v2",
            "date": "2020-10-28 03:45:19+00:00",
            "title": "DisenE: Disentangling Knowledge Graph Embeddings",
            "authors": [
                "Xiaoyu Kou",
                "Yankai Lin",
                "Yuntao Li",
                "Jiahao Xu",
                "Peng Li",
                "Jie Zhou",
                "Yan Zhang"
            ],
            "abstract": "Knowledge graph embedding (KGE), aiming to embed entities and relations into\nlow-dimensional vectors, has attracted wide attention recently. However, the\nexisting research is mainly based on the black-box neural models, which makes\nit difficult to interpret the learned representation. In this paper, we\nintroduce DisenE, an end-to-end framework to learn disentangled knowledge graph\nembeddings. Specially, we introduce an attention-based mechanism that enables\nthe model to explicitly focus on relevant components of entity embeddings\naccording to a given relation. Furthermore, we introduce two novel regularizers\nto encourage each component of the entity representation to independently\nreflect an isolated semantic aspect. Experimental results demonstrate that our\nproposed DisenE investigates a perspective to address the interpretability of\nKGE and is proved to be an effective way to improve the performance of link\nprediction tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14784v1",
            "date": "2020-10-28 06:32:41+00:00",
            "title": "A Chinese Text Classification Method With Low Hardware Requirement Based on Improved Model Concatenation",
            "authors": [
                "Yuanhao Zhuo"
            ],
            "abstract": "In order to improve the accuracy performance of Chinese text classification\nmodels with low hardware requirements, an improved concatenation-based model is\ndesigned in this paper, which is a concatenation of 5 different sub-models,\nincluding TextCNN, LSTM, and Bi-LSTM. Compared with the existing ensemble\nlearning method, for a text classification mission, this model's accuracy is 2%\nhigher. Meanwhile, the hardware requirements of this model are much lower than\nthe BERT-based model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14872v1",
            "date": "2020-10-28 10:42:04+00:00",
            "title": "Bayesian Methods for Semi-supervised Text Annotation",
            "authors": [
                "Kristian Miok",
                "Gregor Pirs",
                "Marko Robnik-Sikonja"
            ],
            "abstract": "Human annotations are an important source of information in the development\nof natural language understanding approaches. As under the pressure of\nproductivity annotators can assign different labels to a given text, the\nquality of produced annotations frequently varies. This is especially the case\nif decisions are difficult, with high cognitive load, requires awareness of\nbroader context, or careful consideration of background knowledge. To alleviate\nthe problem, we propose two semi-supervised methods to guide the annotation\nprocess: a Bayesian deep learning model and a Bayesian ensemble method. Using a\nBayesian deep learning method, we can discover annotations that cannot be\ntrusted and might require reannotation. A recently proposed Bayesian ensemble\nmethod helps us to combine the annotators' labels with predictions of trained\nmodels. According to the results obtained from three hate speech detection\nexperiments, the proposed Bayesian methods can improve the annotations and\nprediction performance of BERT models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.14891v2",
            "date": "2020-10-28 11:19:53+00:00",
            "title": "A Cyclic Proof System for HFLN",
            "authors": [
                "Mayuko Kori",
                "Takeshi Tsukada",
                "Naoki Kobayashi"
            ],
            "abstract": "A cyclic proof system allows us to perform inductive reasoning without\nexplicit inductions. We propose a cyclic proof system for HFLN, which is a\nhigher-order predicate logic with natural numbers and alternating fixed-points.\nOurs is the first cyclic proof system for a higher-order logic, to our\nknowledge. Due to the presence of higher-order predicates and alternating\nfixed-points, our cyclic proof system requires a more delicate global condition\non cyclic proofs than the original system of Brotherston and Simpson. We prove\nthe decidability of checking the global condition and soundness of this system,\nand also prove a restricted form of standard completeness for an infinitary\nvariant of our cyclic proof system. A potential application of our cyclic proof\nsystem is semi-automated verification of higher-order programs, based on\nKobayashi et al.'s recent work on reductions from program verification to HFLN\nvalidity checking.",
            "journal ref": "None",
            "category": "cs.LO"
        },
        {
            "url": "http://arxiv.org/abs/2010.15090v1",
            "date": "2020-10-28 17:05:24+00:00",
            "title": "Handling Class Imbalance in Low-Resource Dialogue Systems by Combining Few-Shot Classification and Interpolation",
            "authors": [
                "Vishal Sunder",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Utterance classification performance in low-resource dialogue systems is\nconstrained by an inevitably high degree of data imbalance in class labels. We\npresent a new end-to-end pairwise learning framework that is designed\nspecifically to tackle this phenomenon by inducing a few-shot classification\ncapability in the utterance representations and augmenting data through an\ninterpolation of utterance representations. Our approach is a general purpose\ntraining methodology, agnostic to the neural architecture used for encoding\nutterances. We show significant improvements in macro-F1 score over standard\ncross-entropy training for three different neural architectures, demonstrating\nimprovements on a Virtual Patient dialogue dataset as well as a low-resourced\nemulation of the Switchboard dialogue act classification dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15149v2",
            "date": "2020-10-28 18:01:02+00:00",
            "title": "Detecting Stance in Media on Global Warming",
            "authors": [
                "Yiwei Luo",
                "Dallas Card",
                "Dan Jurafsky"
            ],
            "abstract": "Citing opinions is a powerful yet understudied strategy in argumentation. For\nexample, an environmental activist might say, \"Leading scientists agree that\nglobal warming is a serious concern,\" framing a clause which affirms their own\nstance (\"that global warming is serious\") as an opinion endorsed (\"[scientists]\nagree\") by a reputable source (\"leading\"). In contrast, a global warming denier\nmight frame the same clause as the opinion of an untrustworthy source with a\npredicate connoting doubt: \"Mistaken scientists claim [...].\" Our work studies\nopinion-framing in the global warming (GW) debate, an increasingly partisan\nissue that has received little attention in NLP. We introduce Global Warming\nStance Dataset (GWSD), a dataset of stance-labeled GW sentences, and train a\nBERT classifier to study novel aspects of argumentation in how different sides\nof a debate represent their own and each other's opinions. From 56K news\narticles, we find that similar linguistic devices for self-affirming and\nopponent-doubting discourse are used across GW-accepting and skeptic media,\nthough GW-skeptical media shows more opponent-doubt. We also find that authors\noften characterize sources as hypocritical, by ascribing opinions expressing\nthe author's own view to source entities known to publicly endorse the opposing\nview. We release our stance dataset, model, and lexicons of framing devices for\nfuture work on opinion-framing and the automatic detection of GW stance.",
            "journal ref": "Findings of ACL: EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15266v1",
            "date": "2020-10-28 22:45:16+00:00",
            "title": "CopyNext: Explicit Span Copying and Alignment in Sequence to Sequence Models",
            "authors": [
                "Abhinav Singh",
                "Patrick Xia",
                "Guanghui Qin",
                "Mahsa Yarmohammadi",
                "Benjamin Van Durme"
            ],
            "abstract": "Copy mechanisms are employed in sequence to sequence models (seq2seq) to\ngenerate reproductions of words from the input to the output. These frameworks,\noperating at the lexical type level, fail to provide an explicit alignment that\nrecords where each token was copied from. Further, they require contiguous\ntoken sequences from the input (spans) to be copied individually. We present a\nmodel with an explicit token-level copy operation and extend it to copying\nentire spans. Our model provides hard alignments between spans in the input and\noutput, allowing for nontraditional applications of seq2seq, like information\nextraction. We demonstrate the approach on Nested Named Entity Recognition,\nachieving near state-of-the-art accuracy with an order of magnitude increase in\ndecoding speed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15316v1",
            "date": "2020-10-29 02:15:23+00:00",
            "title": "Multiple Sclerosis Severity Classification From Clinical Text",
            "authors": [
                "Alister D Costa",
                "Stefan Denkovski",
                "Michal Malyska",
                "Sae Young Moon",
                "Brandon Rufino",
                "Zhen Yang",
                "Taylor Killian",
                "Marzyeh Ghassemi"
            ],
            "abstract": "Multiple Sclerosis (MS) is a chronic, inflammatory and degenerative\nneurological disease, which is monitored by a specialist using the Expanded\nDisability Status Scale (EDSS) and recorded in unstructured text in the form of\na neurology consult note. An EDSS measurement contains an overall \"EDSS\" score\nand several functional subscores. Typically, expert knowledge is required to\ninterpret consult notes and generate these scores. Previous approaches used\nlimited context length Word2Vec embeddings and keyword searches to predict\nscores given a consult note, but often failed when scores were not explicitly\nstated. In this work, we present MS-BERT, the first publicly available\ntransformer model trained on real clinical data other than MIMIC. Next, we\npresent MSBC, a classifier that applies MS-BERT to generate embeddings and\npredict EDSS and functional subscores. Lastly, we explore combining MSBC with\nother models through the use of Snorkel to generate scores for unlabelled\nconsult notes. MSBC achieves state-of-the-art performance on all metrics and\nprediction tasks and outperforms the models generated from the Snorkel\nensemble. We improve Macro-F1 by 0.12 (to 0.88) for predicting EDSS and on\naverage by 0.29 (to 0.63) for predicting functional subscores over previous\nWord2Vec CNN and rule-based approaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15602v1",
            "date": "2020-10-26 07:03:49+00:00",
            "title": "Designing learning experiences for online teaching and learning",
            "authors": [
                "Nachamma Sockalingam",
                "Junhua Liu"
            ],
            "abstract": "Teaching is about constantly innovating strategies, ways and means to engage\ndiverse students in active and meaningful learning. In line with this, SUTD\nadopts various student-centric teaching and learning teaching methods and\napproaches. This means that our graduate/undergraduate instructors have to be\nready to teach using these student student-centric teaching and learning\npedagogies. In this article, I share my experiences of redesigning this\nteaching course that is typically conducted face-to-face to a synchronous\nonline course and also invite one of the participant in this course to reflect\non his experience as a student.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2010.15728v3",
            "date": "2020-10-29 16:21:26+00:00",
            "title": "Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation",
            "authors": [
                "Hang Dong",
                "V\u00edctor Su\u00e1rez-Paniagua",
                "William Whiteley",
                "Honghan Wu"
            ],
            "abstract": "Diagnostic or procedural coding of clinical notes aims to derive a coded\nsummary of disease-related information about patients. Such coding is usually\ndone manually in hospitals but could potentially be automated to improve the\nefficiency and accuracy of medical coding. Recent studies on deep learning for\nautomated medical coding achieved promising performances. However, the\nexplainability of these models is usually poor, preventing them to be used\nconfidently in supporting clinical practice. Another limitation is that these\nmodels mostly assume independence among labels, ignoring the complex\ncorrelation among medical codes which can potentially be exploited to improve\nthe performance. We propose a Hierarchical Label-wise Attention Network (HLAN),\nwhich aimed to interpret the model by quantifying importance (as attention\nweights) of words and sentences related to each of the labels. Secondly, we\npropose to enhance the major deep learning models with a label embedding (LE)\ninitialisation approach, which learns a dense, continuous vector representation\nand then injects the representation into the final layers and the label-wise\nattention layers in the models. We evaluated the methods using three settings\non the MIMIC-III discharge summaries: full codes, top-50 codes, and the UK NHS\nCOVID-19 shielding codes. Experiments were conducted to compare HLAN and LE\ninitialisation to the state-of-the-art neural network based methods. HLAN\nachieved the best Micro-level AUC and $F_1$ on the top-50 code prediction and\ncomparable results on the NHS COVID-19 shielding code prediction to other\nmodels. By highlighting the most salient words and sentences for each label,\nHLAN showed more meaningful and comprehensive model interpretation compared to\nits downgraded baselines and the CNN-based models. LE initialisation\nconsistently boosted most deep learning models for automated medical coding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15875v1",
            "date": "2020-10-29 18:28:16+00:00",
            "title": "Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning",
            "authors": [
                "Yuncheng Hua",
                "Yuan-Fang Li",
                "Gholamreza Haffari",
                "Guilin Qi",
                "Wei Wu"
            ],
            "abstract": "A compelling approach to complex question answering is to convert the\nquestion to a sequence of actions, which can then be executed on the knowledge\nbase to yield the answer, aka the programmer-interpreter approach. Use similar\ntraining questions to the test question, meta-learning enables the programmer\nto adapt to unseen questions to tackle potential distributional biases quickly.\nHowever, this comes at the cost of manually labeling similar questions to learn\na retrieval model, which is tedious and expensive. In this paper, we present a\nnovel method that automatically learns a retrieval model alternately with the\nprogrammer from weak supervision, i.e., the system's performance with respect\nto the produced answers. To the best of our knowledge, this is the first\nattempt to train the retrieval model with the programmer jointly. Our system\nleads to state-of-the-art performance on a large-scale task for complex\nquestion answering over knowledge bases. We have released our code at\nhttps://github.com/DevinJake/MARL.",
            "journal ref": "IJCAI 2020: 3679-3686",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2010.15877v1",
            "date": "2020-10-29 18:34:55+00:00",
            "title": "Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning",
            "authors": [
                "Yuncheng Hua",
                "Yuan-Fang Li",
                "Gholamreza Haffari",
                "Guilin Qi",
                "Tongtong Wu"
            ],
            "abstract": "Complex question-answering (CQA) involves answering complex natural-language\nquestions on a knowledge base (KB). However, the conventional neural program\ninduction (NPI) approach exhibits uneven performance when the questions have\ndifferent types, harboring inherently different characteristics, e.g.,\ndifficulty level. This paper proposes a meta-reinforcement learning approach to\nprogram induction in CQA to tackle the potential distributional bias in\nquestions. Our method quickly and effectively adapts the meta-learned\nprogrammer to new questions based on the most similar questions retrieved from\nthe training data. The meta-learned policy is then used to learn a good\nprogramming policy, utilizing the trial trajectories and their rewards for\nsimilar questions in the support set. Our method achieves state-of-the-art\nperformance on the CQA dataset (Saha et al., 2018) while using only five trial\ntrajectories for the top-5 retrieved questions in each support set, and\nmetatraining on tasks constructed from only 1% of the training set. We have\nreleased our code at https://github.com/DevinJake/MRL-CQA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15881v1",
            "date": "2020-10-29 18:42:44+00:00",
            "title": "Less is More: Data-Efficient Complex Question Answering over Knowledge Bases",
            "authors": [
                "Yuncheng Hua",
                "Yuan-Fang Li",
                "Guilin Qi",
                "Wei Wu",
                "Jingyao Zhang",
                "Daiqing Qi"
            ],
            "abstract": "Question answering is an effective method for obtaining information from\nknowledge bases (KB). In this paper, we propose the Neural-Symbolic Complex\nQuestion Answering (NS-CQA) model, a data-efficient reinforcement learning\nframework for complex question answering by using only a modest number of\ntraining samples. Our framework consists of a neural generator and a symbolic\nexecutor that, respectively, transforms a natural-language question into a\nsequence of primitive actions, and executes them over the knowledge base to\ncompute the answer. We carefully formulate a set of primitive symbolic actions\nthat allows us to not only simplify our neural network design but also\naccelerate model convergence. To reduce search space, we employ the copy and\nmasking mechanisms in our encoder-decoder architecture to drastically reduce\nthe decoder output vocabulary and improve model generalizability. We equip our\nmodel with a memory buffer that stores high-reward promising programs. Besides,\nwe propose an adaptive reward function. By comparing the generated trial with\nthe trials stored in the memory buffer, we derive the curriculum-guided reward\nbonus, i.e., the proximity and the novelty. To mitigate the sparse reward\nproblem, we combine the adaptive reward and the reward bonus, reshaping the\nsparse reward into dense feedback. Also, we encourage the model to generate new\ntrials to avoid imitating the spurious trials while making the model remember\nthe past high-reward trials to improve data efficiency. Our NS-CQA model is\nevaluated on two datasets: CQA, a recent large-scale complex question answering\ndataset, and WebQuestionsSP, a multi-hop question answering dataset. On both\ndatasets, our model outperforms the state-of-the-art models. Notably, on CQA,\nNS-CQA performs well on questions with higher complexity, while only using\napproximately 1% of the total training samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15884v1",
            "date": "2020-10-29 18:49:54+00:00",
            "title": "Systolic Computing on GPUs for Productive Performance",
            "authors": [
                "Hongbo Rong",
                "Xiaochen Hao",
                "Yun Liang",
                "Lidong Xu",
                "Hong H Jiang",
                "Pradeep Dubey"
            ],
            "abstract": "We propose a language and compiler to productively build high-performance\n{\\it software systolic arrays} that run on GPUs. Based on a rigorous\nmathematical foundation (uniform recurrence equations and space-time\ntransform), our language has a high abstraction level and covers a wide range\nof applications. A programmer {\\it specifies} a projection of a dataflow\ncompute onto a linear systolic array, while leaving the detailed implementation\nof the projection to a compiler; the compiler implements the specified\nprojection and maps the linear systolic array to the SIMD execution units and\nvector registers of GPUs. In this way, both productivity and performance are\nachieved in the same time. This approach neatly combines loop transformations,\ndata shuffling, and vector register allocation into a single framework.\nMeanwhile, many other optimizations can be applied as well; the compiler\ncomposes the optimizations together to generate efficient code.\n  We implemented the approach on Intel GPUs. This is the first system that\nallows productive construction of systolic arrays on GPUs. We allow multiple\nprojections, arbitrary projection directions and linear schedules, which can\nexpress most, if not all, systolic arrays in practice. Experiments with 1- and\n2-D convolution on an Intel GEN9.5 GPU have demonstrated the generality of the\napproach, and its productivity in expressing various systolic designs for\nfinding the best candidate. Although our systolic arrays are purely software\nrunning on generic SIMD hardware, compared with the GPU's specialized, hardware\nsamplers that perform the same convolutions, some of our best designs are up to\n59\\% faster. Overall, this approach holds promise for productive\nhigh-performance computing on GPUs.",
            "journal ref": "None",
            "category": "cs.PL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15925v2",
            "date": "2020-10-29 20:31:39+00:00",
            "title": "RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark",
            "authors": [
                "Tatiana Shavrina",
                "Alena Fenogenova",
                "Anton Emelyanov",
                "Denis Shevelev",
                "Ekaterina Artemova",
                "Valentin Malykh",
                "Vladislav Mikhailov",
                "Maria Tikhonova",
                "Andrey Chertok",
                "Andrey Evlampiev"
            ],
            "abstract": "In this paper, we introduce an advanced Russian general language\nunderstanding evaluation benchmark -- RussianGLUE. Recent advances in the field\nof universal language models and transformers require the development of a\nmethodology for their broad diagnostics and testing for general intellectual\nskills - detection of natural language inference, commonsense reasoning,\nability to perform simple logical operations regardless of text subject or\nlexicon. For the first time, a benchmark of nine tasks, collected and organized\nanalogically to the SuperGLUE methodology, was developed from scratch for the\nRussian language. We provide baselines, human level evaluation, an open-source\nframework for evaluating models\n(https://github.com/RussianNLP/RussianSuperGLUE), and an overall leaderboard of\ntransformer models for the Russian language. Besides, we present the first\nresults of comparing multilingual models in the adapted diagnostic test set and\noffer the first steps to further expanding or assessing state-of-the-art models\nindependently of language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15939v1",
            "date": "2020-10-29 20:56:15+00:00",
            "title": "RuREBus: a Case Study of Joint Named Entity Recognition and Relation Extraction from e-Government Domain",
            "authors": [
                "Vitaly Ivanin",
                "Ekaterina Artemova",
                "Tatiana Batura",
                "Vladimir Ivanov",
                "Veronika Sarkisyan",
                "Elena Tutubalina",
                "Ivan Smurov"
            ],
            "abstract": "We show-case an application of information extraction methods, such as named\nentity recognition (NER) and relation extraction (RE) to a novel corpus,\nconsisting of documents, issued by a state agency. The main challenges of this\ncorpus are: 1) the annotation scheme differs greatly from the one used for the\ngeneral domain corpora, and 2) the documents are written in a language other\nthan English. Unlike expectations, the state-of-the-art transformer-based\nmodels show modest performance for both tasks, either when approached\nsequentially, or in an end-to-end fashion. Our experiments have demonstrated\nthat fine-tuning on a large unlabeled corpora does not automatically yield\nsignificant improvement and thus we may conclude that more sophisticated\nstrategies of leveraging unlabelled texts are demanded. In this paper, we\ndescribe the whole developed pipeline, starting from text annotation, baseline\ndevelopment, and designing a shared task in hopes of improving the baseline.\nEventually, we realize that the current NER and RE technologies are far from\nbeing mature and do not overcome so far challenges like ours.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.15980v2",
            "date": "2020-10-29 22:54:00+00:00",
            "title": "AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts",
            "authors": [
                "Taylor Shin",
                "Yasaman Razeghi",
                "Robert L. Logan IV",
                "Eric Wallace",
                "Sameer Singh"
            ],
            "abstract": "The remarkable success of pretrained language models has motivated the study\nof what kinds of knowledge these models learn during pretraining. Reformulating\ntasks as fill-in-the-blanks problems (e.g., cloze tests) is a natural approach\nfor gauging such knowledge, however, its usage is limited by the manual effort\nand guesswork required to write suitable prompts. To address this, we develop\nAutoPrompt, an automated method to create prompts for a diverse set of tasks,\nbased on a gradient-guided search. Using AutoPrompt, we show that masked\nlanguage models (MLMs) have an inherent capability to perform sentiment\nanalysis and natural language inference without additional parameters or\nfinetuning, sometimes achieving performance on par with recent state-of-the-art\nsupervised models. We also show that our prompts elicit more accurate factual\nknowledge from MLMs than the manually created prompts on the LAMA benchmark,\nand that MLMs can be used as relation extractors more effectively than\nsupervised relation extraction models. These results demonstrate that\nautomatically generated prompts are a viable parameter-free alternative to\nexisting probing methods, and as pretrained LMs become more sophisticated and\ncapable, potentially a replacement for finetuning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16088v2",
            "date": "2020-10-30 06:12:01+00:00",
            "title": "Cross-Domain Sentiment Classification with Contrastive Learning and Mutual Information Maximization",
            "authors": [
                "Tian Li",
                "Xiang Chen",
                "Shanghang Zhang",
                "Zhen Dong",
                "Kurt Keutzer"
            ],
            "abstract": "Contrastive learning (CL) has been successful as a powerful representation\nlearning method. In this work we propose CLIM: Contrastive Learning with mutual\nInformation Maximization, to explore the potential of CL on cross-domain\nsentiment classification. To the best of our knowledge, CLIM is the first to\nadopt contrastive learning for natural language processing (NLP) tasks across\ndomains. Due to scarcity of labels on the target domain, we introduce mutual\ninformation maximization (MIM) apart from CL to exploit the features that best\nsupport the final prediction. Furthermore, MIM is able to maintain a relatively\nbalanced distribution of the model's prediction, and enlarges the margin\nbetween classes on the target domain. The larger margin increases our model's\nrobustness and enables the same classifier to be optimal across domains.\nConsequently, we achieve new state-of-the-art results on the Amazon-review\ndataset as well as the airlines dataset, showing the efficacy of our proposed\nmethod CLIM.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16131v2",
            "date": "2020-10-30 09:07:37+00:00",
            "title": "Comparison of Speaker Role Recognition and Speaker Enrollment Protocol for conversational Clinical Interviews",
            "authors": [
                "Rachid Riad",
                "Hadrien Titeux",
                "Laurie Lemoine",
                "Justine Montillot",
                "Agnes Sliwinski",
                "Jennifer Hamet Bagnou",
                "Xuan Nga Cao",
                "Anne-Catherine Bachoud-L\u00e9vi",
                "Emmanuel Dupoux"
            ],
            "abstract": "Conversations between a clinician and a patient, in natural conditions, are\nvaluable sources of information for medical follow-up. The automatic analysis\nof these dialogues could help extract new language markers and speed-up the\nclinicians' reports. Yet, it is not clear which speech processing pipeline is\nthe most performing to detect and identify the speaker turns, especially for\nindividuals with speech and language disorders. Here, we proposed a split of\nthe data that allows conducting a comparative evaluation of speaker role\nrecognition and speaker enrollment methods to solve this task. We trained\nend-to-end neural network architectures to adapt to each task and evaluate each\napproach under the same metric. Experimental results are reported on\nnaturalistic clinical conversations between Neuropsychologist and Interviewees,\nat different stages of Huntington's disease. We found that our Speaker Role\nRecognition model gave the best performances. In addition, our study underlined\nthe importance of retraining models with in-domain data. Finally, we observed\nthat results do not depend on the demographics of the Interviewee, highlighting\nthe clinical relevance of our methods.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2010.16143v2",
            "date": "2020-10-30 09:30:54+00:00",
            "title": "HyperText: Endowing FastText with Hyperbolic Geometry",
            "authors": [
                "Yudong Zhu",
                "Di Zhou",
                "Jinghui Xiao",
                "Xin Jiang",
                "Xiao Chen",
                "Qun Liu"
            ],
            "abstract": "Natural language data exhibit tree-like hierarchical structures such as the\nhypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text\nclassifier based on shallow neural network in Euclidean space, may not model\nsuch hierarchies precisely with limited representation capacity. Considering\nthat hyperbolic space is naturally suitable for modeling tree-like hierarchical\ndata, we propose a new model named HyperText for efficient text classification\nby endowing FastText with hyperbolic geometry. Empirically, we show that\nHyperText outperforms FastText on a range of text classification tasks with\nmuch reduced parameters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16249v1",
            "date": "2020-10-30 13:33:41+00:00",
            "title": "SLM: Learning a Discourse Language Representation with Sentence Unshuffling",
            "authors": [
                "Haejun Lee",
                "Drew A. Hudson",
                "Kangwook Lee",
                "Christopher D. Manning"
            ],
            "abstract": "We introduce Sentence-level Language Modeling, a new pre-training objective\nfor learning a discourse language representation in a fully self-supervised\nmanner. Recent pre-training methods in NLP focus on learning either bottom or\ntop-level language representations: contextualized word representations derived\nfrom language model objectives at one extreme and a whole sequence\nrepresentation learned by order classification of two given textual segments at\nthe other. However, these models are not directly encouraged to capture\nrepresentations of intermediate-size structures that exist in natural languages\nsuch as sentences and the relationships among them. To that end, we propose a\nnew approach to encourage learning of a contextualized sentence-level\nrepresentation by shuffling the sequence of input sentences and training a\nhierarchical transformer model to reconstruct the original ordering. Through\nexperiments on downstream tasks such as GLUE, SQuAD, and DiscoEval, we show\nthat this feature of our model improves the performance of the original BERT by\nlarge margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16314v2",
            "date": "2020-10-30 15:09:19+00:00",
            "title": "A Critical Assessment of State-of-the-Art in Entity Alignment",
            "authors": [
                "Max Berrendorf",
                "Ludwig Wacker",
                "Evgeniy Faerman"
            ],
            "abstract": "In this work, we perform an extensive investigation of two state-of-the-art\n(SotA) methods for the task of Entity Alignment in Knowledge Graphs. Therefore,\nwe first carefully examine the benchmarking process and identify several\nshortcomings, which make the results reported in the original works not always\ncomparable. Furthermore, we suspect that it is a common practice in the\ncommunity to make the hyperparameter optimization directly on a test set,\nreducing the informative value of reported performance. Thus, we select a\nrepresentative sample of benchmarking datasets and describe their properties.\nWe also examine different initializations for entity representations since they\nare a decisive factor for model performance. Furthermore, we use a shared\ntrain/validation/test split for a fair evaluation setting in which we evaluate\nall methods on all datasets. In our evaluation, we make several interesting\nfindings. While we observe that most of the time SotA approaches perform better\nthan baselines, they have difficulties when the dataset contains noise, which\nis the case in most real-life applications. Moreover, we find out in our\nablation study that often different features of SotA methods are crucial for\ngood performance than previously assumed. The code is available at\nhttps://github.com/mberr/ea-sota-comparison.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16324v1",
            "date": "2020-10-30 15:29:16+00:00",
            "title": "Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach",
            "authors": [
                "Ahmadreza Mosallanezhad",
                "Kai Shu",
                "Huan Liu"
            ],
            "abstract": "Nowadays, there exist powerful language models such as OpenAI's GPT-2 that\ncan generate readable text and can be fine-tuned to generate text for a\nspecific domain. Considering GPT-2, it cannot directly generate synthetic news\nwith respect to a given topic and the output of the language model cannot be\nexplicitly controlled. In this paper, we study the novel problem of\ntopic-preserving synthetic news generation. We propose a novel deep\nreinforcement learning-based method to control the output of GPT-2 with respect\nto a given news topic. When generating text using GPT-2, by default, the most\nprobable word is selected from the vocabulary. Instead of selecting the best\nword each time from GPT-2's output, an RL agent tries to select words that\noptimize the matching of a given topic. In addition, using a fake news detector\nas an adversary, we investigate generating realistic news using our proposed\nmethod. In this paper, we consider realistic news as news that cannot be easily\ndetected by a fake news classifier. Experimental results demonstrate the\neffectiveness of the proposed framework on generating topic-preserving news\ncontent than state-of-the-art baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16363v1",
            "date": "2020-10-30 16:39:49+00:00",
            "title": "Domain-Specific Lexical Grounding in Noisy Visual-Textual Documents",
            "authors": [
                "Gregory Yauney",
                "Jack Hessel",
                "David Mimno"
            ],
            "abstract": "Images can give us insights into the contextual meanings of words, but\ncurrent image-text grounding approaches require detailed annotations. Such\ngranular annotation is rare, expensive, and unavailable in most domain-specific\ncontexts. In contrast, unlabeled multi-image, multi-sentence documents are\nabundant. Can lexical grounding be learned from such documents, even though\nthey have significant lexical and visual overlap? Working with a case study\ndataset of real estate listings, we demonstrate the challenge of distinguishing\nhighly correlated grounded terms, such as \"kitchen\" and \"bedroom\", and\nintroduce metrics to assess this document similarity. We present a simple\nunsupervised clustering-based method that increases precision and recall beyond\nobject detection and image tagging baselines when evaluated on labeled subsets\nof the dataset. The proposed method is particularly effective for local\ncontextual meanings of a word, for example associating \"granite\" with\ncountertops in the real estate dataset and with rocky landscapes in a Wikipedia\ndataset.",
            "journal ref": "Published in EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16368v4",
            "date": "2020-10-30 16:53:29+00:00",
            "title": "Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition",
            "authors": [
                "Wei Zhou",
                "Simon Berger",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "To join the advantages of classical and end-to-end approaches for speech\nrecognition, we present a simple, novel and competitive approach for\nphoneme-based neural transducer modeling. Different alignment label topologies\nare compared and word-end-based phoneme label augmentation is proposed to\nimprove performance. Utilizing the local dependency of phonemes, we adopt a\nsimplified neural network structure and a straightforward integration with the\nexternal word-level language model to preserve the consistency of seq-to-seq\nmodeling. We also present a simple, stable and efficient training procedure\nusing frame-wise cross-entropy loss. A phonetic context size of one is shown to\nbe sufficient for the best performance. A simplified scheduled sampling\napproach is applied for further improvement and different decoding approaches\nare briefly compared. The overall performance of our best model is comparable\nto state-of-the-art (SOTA) results for the TED-LIUM Release 2 and Switchboard\ncorpora.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16407v1",
            "date": "2020-10-15 00:56:54+00:00",
            "title": "TopicBERT for Energy Efficient Document Classification",
            "authors": [
                "Yatin Chaudhary",
                "Pankaj Gupta",
                "Khushbu Saxena",
                "Vivek Kulkarni",
                "Thomas Runkler",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Prior research notes that BERT's computational cost grows quadratically with\nsequence length thus leading to longer training times, higher GPU memory\nconstraints and carbon emissions. While recent work seeks to address these\nscalability issues at pre-training, these issues are also prominent in\nfine-tuning especially for long sequence tasks like document classification.\nOur work thus focuses on optimizing the computational cost of fine-tuning for\ndocument classification. We achieve this by complementary learning of both\ntopic and language models in a unified framework, named TopicBERT. This\nsignificantly reduces the number of self-attention operations - a main\nperformance bottleneck. Consequently, our model achieves a 1.4x ($\\sim40\\%$)\nspeedup with $\\sim40\\%$ reduction in $CO_2$ emission while retaining $99.9\\%$\nperformance over 5 datasets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2010.16410v1",
            "date": "2020-10-06 03:54:11+00:00",
            "title": "Semi-supervised Relation Extraction via Incremental Meta Self-Training",
            "authors": [
                "Xuming Hu",
                "Fukun Ma",
                "Chenyao Liu",
                "Chenwei Zhang",
                "Lijie Wen",
                "Philip S. Yu"
            ],
            "abstract": "To alleviate human efforts from obtaining large-scale annotations,\nSemi-Supervised Relation Extraction methods aim to leverage unlabeled data in\naddition to learning from limited samples. Existing self-training methods\nsuffer from the gradual drift problem, where noisy pseudo labels on unlabeled\ndata are incorporated during training. To alleviate the noise in pseudo labels,\nwe propose a method called MetaSRE, where a Relation Label Generation Network\ngenerates accurate quality assessment on pseudo labels by (meta) learning from\nthe successful and failed attempts on Relation Classification as an additional\nmeta-objective. To reduce the influence of noisy pseudo labels, MetaSRE adopts\na pseudo label selection and exploitation scheme which assesses pseudo label\nquality on unlabeled samples and only exploits high-quality pseudo labels in a\nself-training fashion to incrementally augment labeled samples for both\nrobustness and accuracy. Experimental results on two public datasets\ndemonstrate the effectiveness of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00057v1",
            "date": "2020-10-30 19:09:48+00:00",
            "title": "A Sui Generis QA Approach using RoBERTa for Adverse Drug Event Identification",
            "authors": [
                "Harshit Jain",
                "Nishant Raj",
                "Suyash Mishra"
            ],
            "abstract": "Extraction of adverse drug events from biomedical literature and other\ntextual data is an important component to monitor drug-safety and this has\nattracted attention of many researchers in healthcare. Existing works are more\npivoted around entity-relation extraction using bidirectional long short term\nmemory networks (Bi-LSTM) which does not attain the best feature\nrepresentations. In this paper, we introduce a question answering framework\nthat exploits the robustness, masking and dynamic attention capabilities of\nRoBERTa by a technique of domain adaptation and attempt to overcome the\naforementioned limitations. Our model outperforms the prior work by 9.53%\nF1-Score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00061v1",
            "date": "2020-10-30 19:12:25+00:00",
            "title": "A New Neural Search and Insights Platform for Navigating and Organizing AI Research",
            "authors": [
                "Marzieh Fadaee",
                "Olga Gureenkova",
                "Fernando Rejon Barrera",
                "Carsten Schnober",
                "Wouter Weerkamp",
                "Jakub Zavrel"
            ],
            "abstract": "To provide AI researchers with modern tools for dealing with the explosive\ngrowth of the research literature in their field, we introduce a new platform,\nAI Research Navigator, that combines classical keyword search with neural\nretrieval to discover and organize relevant literature. The system provides\nsearch at multiple levels of textual granularity, from sentences to\naggregations across documents, both in natural language and through navigation\nin a domain-specific Knowledge Graph. We give an overview of the overall\narchitecture of the system and of the components for document analysis,\nquestion answering, search, analytics, expert search, and recommendations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00080v1",
            "date": "2020-10-30 20:01:56+00:00",
            "title": "Dynamic Data Selection for Curriculum Learning via Ability Estimation",
            "authors": [
                "John P. Lalor",
                "Hong Yu"
            ],
            "abstract": "Curriculum learning methods typically rely on heuristics to estimate the\ndifficulty of training examples or the ability of the model. In this work, we\npropose replacing difficulty heuristics with learned difficulty parameters. We\nalso propose Dynamic Data selection for Curriculum Learning via Ability\nEstimation (DDaCLAE), a strategy that probes model ability at each training\nepoch to select the best training examples at that point. We show that models\nusing learned difficulty and/or ability outperform heuristic-based curriculum\nlearning models on the GLUE classification tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00105v1",
            "date": "2020-10-30 21:01:22+00:00",
            "title": "Learning Structured Representations of Entity Names using Active Learning and Weak Supervision",
            "authors": [
                "Kun Qian",
                "Poornima Chozhiyath Raman",
                "Yunyao Li",
                "Lucian Popa"
            ],
            "abstract": "Structured representations of entity names are useful for many entity-related\ntasks such as entity normalization and variant generation. Learning the\nimplicit structured representations of entity names without context and\nexternal knowledge is particularly challenging. In this paper, we present a\nnovel learning framework that combines active learning and weak supervision to\nsolve this problem. Our experimental evaluation show that this framework\nenables the learning of high-quality models from merely a dozen or so labeled\nexamples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00109v1",
            "date": "2020-10-30 21:18:19+00:00",
            "title": "Semantic similarity-based approach to enhance supervised classification learning accuracy",
            "authors": [
                "Houcemeddine Turki",
                "Mohamed Ali Hadj Taieb",
                "Mohamed Ben Aouicha"
            ],
            "abstract": "This brief communication discusses the usefulness of semantic similarity\nmeasures for the evaluation and amelioration of the accuracy of supervised\nclassification learning. It proposes a semantic similarity-based method to\nenhance the choice of adequate labels for the classification algorithm as well\nas two metrics (SS-Score and TD-Score) and a curve (SA-Curve) that can be\ncoupled to statistical evaluation measures of supervised classification\nlearning to take into consideration the impact of the semantic aspect of the\nlabels on the classification accuracy.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.00136v1",
            "date": "2020-10-30 23:04:56+00:00",
            "title": "Improving Dialogue Breakdown Detection with Semi-Supervised Learning",
            "authors": [
                "Nathan Ng",
                "Marzyeh Ghassemi",
                "Narendran Thangarajan",
                "Jiacheng Pan",
                "Qi Guo"
            ],
            "abstract": "Building user trust in dialogue agents requires smooth and consistent\ndialogue exchanges. However, agents can easily lose conversational context and\ngenerate irrelevant utterances. These situations are called dialogue breakdown,\nwhere agent utterances prevent users from continuing the conversation. Building\nsystems to detect dialogue breakdown allows agents to recover appropriately or\navoid breakdown entirely. In this paper we investigate the use of\nsemi-supervised learning methods to improve dialogue breakdown detection,\nincluding continued pre-training on the Reddit dataset and a manifold-based\ndata augmentation method. We demonstrate the effectiveness of these methods on\nthe Dialogue Breakdown Detection Challenge (DBDC) English shared task. Our\nsubmissions to the 2020 DBDC5 shared task place first, beating baselines and\nother submissions by over 12\\% accuracy. In ablations on DBDC4 data from 2019,\nour semi-supervised learning methods improve the performance of a baseline BERT\nmodel by 2\\% accuracy. These methods are applicable generally to any dialogue\ntask and provide a simple way to improve model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00192v1",
            "date": "2020-10-31 05:26:49+00:00",
            "title": "Personalized Multimodal Feedback Generation in Education",
            "authors": [
                "Haochen Liu",
                "Zitao Liu",
                "Zhongqin Wu",
                "Jiliang Tang"
            ],
            "abstract": "The automatic evaluation for school assignments is an important application\nof AI in the education field. In this work, we focus on the task of\npersonalized multimodal feedback generation, which aims to generate\npersonalized feedback for various teachers to evaluate students' assignments\ninvolving multimodal inputs such as images, audios, and texts. This task\ninvolves the representation and fusion of multimodal information and natural\nlanguage generation, which presents the challenges from three aspects: 1) how\nto encode and integrate multimodal inputs; 2) how to generate feedback specific\nto each modality; and 3) how to realize personalized feedback generation. In\nthis paper, we propose a novel Personalized Multimodal Feedback Generation\nNetwork (PMFGN) armed with a modality gate mechanism and a personalized bias\nmechanism to address these challenges. The extensive experiments on real-world\nK-12 education data show that our model significantly outperforms several\nbaselines by generating more accurate and diverse feedback. In addition,\ndetailed ablation experiments are conducted to deepen our understanding of the\nproposed framework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00345v1",
            "date": "2020-10-31 19:37:22+00:00",
            "title": "Aspectuality Across Genre: A Distributional Semantics Approach",
            "authors": [
                "Thomas Kober",
                "Malihe Alikhani",
                "Matthew Stone",
                "Mark Steedman"
            ],
            "abstract": "The interpretation of the lexical aspect of verbs in English plays a crucial\nrole for recognizing textual entailment and learning discourse-level\ninferences. We show that two elementary dimensions of aspectual class, states\nvs. events, and telic vs. atelic events, can be modelled effectively with\ndistributional semantics. We find that a verb's local context is most\nindicative of its aspectual class, and demonstrate that closed class words tend\nto be stronger discriminating contexts than content words. Our approach\noutperforms previous work on three datasets. Lastly, we contribute a dataset of\nhuman--human conversations annotated with lexical aspect and present\nexperiments that show the correlation of telicity with genre and discourse\ngoals.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00425v1",
            "date": "2020-11-01 04:52:56+00:00",
            "title": "Analyzing the Effect of Multi-task Learning for Biomedical Named Entity Recognition",
            "authors": [
                "Arda Akdemir",
                "Tetsuo Shibuya"
            ],
            "abstract": "Developing high-performing systems for detecting biomedical named entities\nhas major implications. State-of-the-art deep-learning based solutions for\nentity recognition often require large annotated datasets, which is not\navailable in the biomedical domain. Transfer learning and multi-task learning\nhave been shown to improve performance for low-resource domains. However, the\napplications of these methods are relatively scarce in the biomedical domain,\nand a theoretical understanding of why these methods improve the performance is\nlacking. In this study, we performed an extensive analysis to understand the\ntransferability between different biomedical entity datasets. We found useful\nmeasures to predict transferability between these datasets. Besides, we propose\ncombining transfer learning and multi-task learning to improve the performance\nof biomedical named entity recognition systems, which is not applied before to\nthe best of our knowledge.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00483v1",
            "date": "2020-11-01 11:34:50+00:00",
            "title": "Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems",
            "authors": [
                "Vitou Phy",
                "Yang Zhao",
                "Akiko Aizawa"
            ],
            "abstract": "Many automatic evaluation metrics have been proposed to score the overall\nquality of a response in open-domain dialogue. Generally, the overall quality\nis comprised of various aspects, such as relevancy, specificity, and empathy,\nand the importance of each aspect differs according to the task. For instance,\nspecificity is mandatory in a food-ordering dialogue task, whereas fluency is\npreferred in a language-teaching dialogue system. However, existing metrics are\nnot designed to cope with such flexibility. For example, BLEU score\nfundamentally relies only on word overlapping, whereas BERTScore relies on\nsemantic similarity between reference and candidate response. Thus, they are\nnot guaranteed to capture the required aspects, i.e., specificity. To design a\nmetric that is flexible to a task, we first propose making these qualities\nmanageable by grouping them into three groups: understandability, sensibleness,\nand likability, where likability is a combination of qualities that are\nessential for a task. We also propose a simple method to composite metrics of\neach aspect to obtain a single metric called USL-H, which stands for\nUnderstandability, Sensibleness, and Likability in Hierarchy. We demonstrated\nthat USL-H score achieves good correlations with human judgment and maintains\nits configurability towards different aspects and metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00538v1",
            "date": "2020-11-01 15:33:43+00:00",
            "title": "Deep Diacritization: Efficient Hierarchical Recurrence for Improved Arabic Diacritization",
            "authors": [
                "Badr AlKhamissi",
                "Muhammad N. ElNokrashy",
                "Mohamed Gabr"
            ],
            "abstract": "We propose a novel architecture for labelling character sequences that\nachieves state-of-the-art results on the Tashkeela Arabic diacritization\nbenchmark. The core is a two-level recurrence hierarchy that operates on the\nword and character levels separately---enabling faster training and inference\nthan comparable traditional models. A cross-level attention module further\nconnects the two, and opens the door for network interpretability. The task\nmodule is a softmax classifier that enumerates valid combinations of\ndiacritics. This architecture can be extended with a recurrent decoder that\noptionally accepts priors from partially diacritized text, which improves\nresults. We employ extra tricks such as sentence dropout and majority voting to\nfurther boost the final result. Our best model achieves a WER of 5.34%,\noutperforming the previous state-of-the-art with a 30.56% relative error\nreduction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00578v3",
            "date": "2020-11-01 18:03:22+00:00",
            "title": "ASAD: A Twitter-based Benchmark Arabic Sentiment Analysis Dataset",
            "authors": [
                "Basma Alharbi",
                "Hind Alamro",
                "Manal Alshehri",
                "Zuhair Khayyat",
                "Manal Kalkatawi",
                "Inji Ibrahim Jaber",
                "Xiangliang Zhang"
            ],
            "abstract": "This paper provides a detailed description of a new Twitter-based benchmark\ndataset for Arabic Sentiment Analysis (ASAD), which is launched in a\ncompetition3, sponsored by KAUST for awarding 10000 USD, 5000 USD and 2000 USD\nto the first, second and third place winners, respectively. Compared to other\npublicly released Arabic datasets, ASAD is a large, high-quality annotated\ndataset(including 95K tweets), with three-class sentiment labels (positive,\nnegative and neutral). We presents the details of the data collection process\nand annotation process. In addition, we implement several baseline models for\nthe competition task and report the results as a reference for the participants\nto the competition.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00584v1",
            "date": "2020-11-01 18:25:15+00:00",
            "title": "A Unifying Theory of Transition-based and Sequence Labeling Parsing",
            "authors": [
                "Carlos G\u00f3mez-Rodr\u00edguez",
                "Michalina Strzyz",
                "David Vilares"
            ],
            "abstract": "We define a mapping from transition-based parsing algorithms that read\nsentences from left to right to sequence labeling encodings of syntactic trees.\nThis not only establishes a theoretical relation between transition-based\nparsing and sequence-labeling parsing, but also provides a method to obtain new\nencodings for fast and simple sequence labeling parsing from the many existing\ntransition-based parsers for different formalisms. Applying it to dependency\nparsing, we implement sequence labeling versions of four algorithms, showing\nthat they are learnable and obtain comparable performance to existing\nencodings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00593v2",
            "date": "2020-11-01 18:47:51+00:00",
            "title": "MixKD: Towards Efficient Distillation of Large-scale Language Models",
            "authors": [
                "Kevin J Liang",
                "Weituo Hao",
                "Dinghan Shen",
                "Yufan Zhou",
                "Weizhu Chen",
                "Changyou Chen",
                "Lawrence Carin"
            ],
            "abstract": "Large-scale language models have recently demonstrated impressive empirical\nperformance. Nevertheless, the improved results are attained at the price of\nbigger models, more power consumption, and slower inference, which hinder their\napplicability to low-resource (both memory and computation) platforms.\nKnowledge distillation (KD) has been demonstrated as an effective framework for\ncompressing such big models. However, large-scale neural network systems are\nprone to memorize training instances, and thus tend to make inconsistent\npredictions when the data distribution is altered slightly. Moreover, the\nstudent model has few opportunities to request useful information from the\nteacher model when there is limited task-specific data available. To address\nthese issues, we propose MixKD, a data-agnostic distillation framework that\nleverages mixup, a simple yet efficient data augmentation approach, to endow\nthe resulting model with stronger generalization ability. Concretely, in\naddition to the original training examples, the student model is encouraged to\nmimic the teacher's behavior on the linear interpolation of example pairs as\nwell. We prove from a theoretical perspective that under reasonable conditions\nMixKD gives rise to a smaller gap between the generalization error and the\nempirical error. To verify its effectiveness, we conduct experiments on the\nGLUE benchmark, where MixKD consistently leads to significant gains over the\nstandard KD training, and outperforms several competitive baselines.\nExperiments under a limited-data setting and ablation studies further\ndemonstrate the advantages of the proposed approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00620v2",
            "date": "2020-11-01 20:16:45+00:00",
            "title": "Social Chemistry 101: Learning to Reason about Social and Moral Norms",
            "authors": [
                "Maxwell Forbes",
                "Jena D. Hwang",
                "Vered Shwartz",
                "Maarten Sap",
                "Yejin Choi"
            ],
            "abstract": "Social norms -- the unspoken commonsense rules about acceptable social\nbehavior -- are crucial in understanding the underlying causes and intents of\npeople's actions in narratives. For example, underlying an action such as\n\"wanting to call cops on my neighbors\" are social norms that inform our\nconduct, such as \"It is expected that you report crimes.\"\n  We present Social Chemistry, a new conceptual formalism to study people's\neveryday social norms and moral judgments over a rich spectrum of real life\nsituations described in natural language. We introduce Social-Chem-101, a\nlarge-scale corpus that catalogs 292k rules-of-thumb such as \"it is rude to run\na blender at 5am\" as the basic conceptual units. Each rule-of-thumb is further\nbroken down with 12 different dimensions of people's judgments, including\nsocial judgments of good and bad, moral foundations, expected cultural\npressure, and assumed legality, which together amount to over 4.5 million\nannotations of categorical labels and free-text descriptions.\n  Comprehensive empirical results based on state-of-the-art neural models\ndemonstrate that computational modeling of social norms is a promising research\ndirection. Our model framework, Neural Norm Transformer, learns and generalizes\nSocial-Chem-101 to successfully reason about previously unseen situations,\ngenerating relevant (and potentially novel) attribute-aware social\nrules-of-thumb.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00675v2",
            "date": "2020-11-02 01:52:46+00:00",
            "title": "A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoning",
            "authors": [
                "Chang Xu",
                "Jun Wang",
                "Yuqing Tang",
                "Francisco Guzman",
                "Benjamin I. P. Rubinstein",
                "Trevor Cohn"
            ],
            "abstract": "As modern neural machine translation (NMT) systems have been widely deployed,\ntheir security vulnerabilities require close scrutiny. Most recently, NMT\nsystems have been found vulnerable to targeted attacks which cause them to\nproduce specific, unsolicited, and even harmful translations. These attacks are\nusually exploited in a white-box setting, where adversarial inputs causing\ntargeted translations are discovered for a known target system. However, this\napproach is less viable when the target system is black-box and unknown to the\nadversary (e.g., secured commercial systems). In this paper, we show that\ntargeted attacks on black-box NMT systems are feasible, based on poisoning a\nsmall fraction of their parallel training data. We show that this attack can be\nrealised practically via targeted corruption of web documents crawled to form\nthe system's training data. We then analyse the effectiveness of the targeted\npoisoning in two common NMT training scenarios: the from-scratch training and\nthe pre-train & fine-tune paradigm. Our results are alarming: even on the\nstate-of-the-art systems trained with massive parallel data (tens of millions),\nthe attacks are still successful (over 50% success rate) under surprisingly low\npoisoning budgets (e.g., 0.006%). Lastly, we discuss potential defences to\ncounter such attacks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00678v3",
            "date": "2020-11-02 01:55:06+00:00",
            "title": "Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation",
            "authors": [
                "Shuhao Gu",
                "Yang Feng"
            ],
            "abstract": "Neural machine translation (NMT) models usually suffer from catastrophic\nforgetting during continual training where the models tend to gradually forget\npreviously learned knowledge and swing to fit the newly added data which may\nhave a different distribution, e.g. a different domain. Although many methods\nhave been proposed to solve this problem, we cannot get to know what causes\nthis phenomenon yet. Under the background of domain adaptation, we investigate\nthe cause of catastrophic forgetting from the perspectives of modules and\nparameters (neurons). The investigation on the modules of the NMT model shows\nthat some modules have tight relation with the general-domain knowledge while\nsome other modules are more essential in the domain adaptation. And the\ninvestigation on the parameters shows that some parameters are important for\nboth the general-domain and in-domain translation and the great change of them\nduring continual training brings about the performance decline in\ngeneral-domain. We conduct experiments across different language pairs and\ndomains to ensure the validity and reliability of our findings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00681v1",
            "date": "2020-11-02 02:03:07+00:00",
            "title": "Event-Related Bias Removal for Real-time Disaster Events",
            "authors": [
                "Evangelia Spiliopoulou",
                "Salvador Medina Maza",
                "Eduard Hovy",
                "Alexander Hauptmann"
            ],
            "abstract": "Social media has become an important tool to share information about crisis\nevents such as natural disasters and mass attacks. Detecting actionable posts\nthat contain useful information requires rapid analysis of huge volume of data\nin real-time. This poses a complex problem due to the large amount of posts\nthat do not contain any actionable information. Furthermore, the classification\nof information in real-time systems requires training on out-of-domain data, as\nwe do not have any data from a new emerging crisis. Prior work focuses on\nmodels pre-trained on similar event types. However, those models capture\nunnecessary event-specific biases, like the location of the event, which affect\nthe generalizability and performance of the classifiers on new unseen data from\nan emerging new event. In our work, we train an adversarial neural model to\nremove latent event-specific biases and improve the performance on tweet\nimportance classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00696v1",
            "date": "2020-11-02 03:07:38+00:00",
            "title": "ABNIRML: Analyzing the Behavior of Neural IR Models",
            "authors": [
                "Sean MacAvaney",
                "Sergey Feldman",
                "Nazli Goharian",
                "Doug Downey",
                "Arman Cohan"
            ],
            "abstract": "Numerous studies have demonstrated the effectiveness of pretrained\ncontextualized language models such as BERT and T5 for ad-hoc search. However,\nit is not well-understood why these methods are so effective, what makes some\nvariants more effective than others, and what pitfalls they may have. We\npresent a new comprehensive framework for Analyzing the Behavior of Neural IR\nModeLs (ABNIRML), which includes new types of diagnostic tests that allow us to\nprobe several characteristics---such as sensitivity to word order---that are\nnot addressed by previous techniques. To demonstrate the value of the\nframework, we conduct an extensive empirical study that yields insights into\nthe factors that contribute to the neural model's gains, and identify potential\nunintended biases the models exhibit. We find evidence that recent neural\nranking models have fundamentally different characteristics from prior ranking\nmodels. For instance, these models can be highly influenced by altered document\nword order, sentence order and inflectional endings. They can also exhibit\nunexpected behaviors when additional content is added to documents, or when\ndocuments are expressed with different levels of fluency or formality. We find\nthat these differences can depend on the architecture and not just the\nunderlying language model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00704v1",
            "date": "2020-11-02 03:21:39+00:00",
            "title": "Semi-supervised Autoencoding Projective Dependency Parsing",
            "authors": [
                "Xiao Zhang",
                "Dan Goldwasser"
            ],
            "abstract": "We describe two end-to-end autoencoding models for semi-supervised\ngraph-based projective dependency parsing. The first model is a Locally\nAutoencoding Parser (LAP) encoding the input using continuous latent variables\nin a sequential manner; The second model is a Globally Autoencoding Parser\n(GAP) encoding the input into dependency trees as latent variables, with exact\ninference. Both models consist of two parts: an encoder enhanced by deep neural\nnetworks (DNN) that can utilize the contextual information to encode the input\ninto latent variables, and a decoder which is a generative model able to\nreconstruct the input. Both LAP and GAP admit a unified structure with\ndifferent loss functions for labeled and unlabeled data with shared parameters.\nWe conducted experiments on WSJ and UD dependency parsing data sets, showing\nthat our models can exploit the unlabeled data to improve the performance given\na limited amount of labeled data, and outperform a previously proposed\nsemi-supervised model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00766v2",
            "date": "2020-11-02 06:22:01+00:00",
            "title": "I Know What You Asked: Graph Path Learning using AMR for Commonsense Reasoning",
            "authors": [
                "Jungwoo Lim",
                "Dongsuk Oh",
                "Yoonna Jang",
                "Kisu Yang",
                "Heuiseok Lim"
            ],
            "abstract": "CommonsenseQA is a task in which a correct answer is predicted through\ncommonsense reasoning with pre-defined knowledge. Most previous works have\naimed to improve the performance with distributed representation without\nconsidering the process of predicting the answer from the semantic\nrepresentation of the question. To shed light upon the semantic interpretation\nof the question, we propose an AMR-ConceptNet-Pruned (ACP) graph. The ACP graph\nis pruned from a full integrated graph encompassing Abstract Meaning\nRepresentation (AMR) graph generated from input questions and an external\ncommonsense knowledge graph, ConceptNet (CN). Then the ACP graph is exploited\nto interpret the reasoning path as well as to predict the correct answer on the\nCommonsenseQA task. This paper presents the manner in which the commonsense\nreasoning process can be interpreted with the relations and concepts provided\nby the ACP graph. Moreover, ACP-based models are shown to outperform the\nbaselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00780v1",
            "date": "2020-11-02 07:14:34+00:00",
            "title": "Adapting Pretrained Transformer to Lattices for Spoken Language Understanding",
            "authors": [
                "Chao-Wei Huang",
                "Yun-Nung Chen"
            ],
            "abstract": "Lattices are compact representations that encode multiple hypotheses, such as\nspeech recognition results or different word segmentations. It is shown that\nencoding lattices as opposed to 1-best results generated by automatic speech\nrecognizer (ASR) boosts the performance of spoken language understanding (SLU).\nRecently, pretrained language models with the transformer architecture have\nachieved the state-of-the-art results on natural language understanding, but\ntheir ability of encoding lattices has not been explored. Therefore, this paper\naims at adapting pretrained transformers to lattice inputs in order to perform\nunderstanding tasks specifically for spoken language. Our experiments on the\nbenchmark ATIS dataset show that fine-tuning pretrained transformers with\nlattice inputs yields clear improvement over fine-tuning with 1-best results.\nFurther evaluation demonstrates the effectiveness of our methods under\ndifferent acoustic conditions. Our code is available at\nhttps://github.com/MiuLab/Lattice-SLU",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00802v1",
            "date": "2020-11-02 08:07:50+00:00",
            "title": "Hierarchical Bi-Directional Self-Attention Networks for Paper Review Rating Recommendation",
            "authors": [
                "Zhongfen Deng",
                "Hao Peng",
                "Congying Xia",
                "Jianxin Li",
                "Lifang He",
                "Philip S. Yu"
            ],
            "abstract": "Review rating prediction of text reviews is a rapidly growing technology with\na wide range of applications in natural language processing. However, most\nexisting methods either use hand-crafted features or learn features using deep\nlearning with simple text corpus as input for review rating prediction,\nignoring the hierarchies among data. In this paper, we propose a Hierarchical\nbi-directional self-attention Network framework (HabNet) for paper review\nrating prediction and recommendation, which can serve as an effective\ndecision-making tool for the academic paper review process. Specifically, we\nleverage the hierarchical structure of the paper reviews with three levels of\nencoders: sentence encoder (level one), intra-review encoder (level two) and\ninter-review encoder (level three). Each encoder first derives contextual\nrepresentation of each level, then generates a higher-level representation, and\nafter the learning process, we are able to identify useful predictors to make\nthe final acceptance decision, as well as to help discover the inconsistency\nbetween numerical review ratings and text sentiment conveyed by reviewers.\nFurthermore, we introduce two new metrics to evaluate models in data imbalance\nsituations. Extensive experiments on a publicly available dataset (PeerRead)\nand our own collected dataset (OpenReview) demonstrate the superiority of the\nproposed approach compared with state-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.00860v1",
            "date": "2020-11-02 10:06:59+00:00",
            "title": "Learning from Non-Binary Constituency Trees via Tensor Decomposition",
            "authors": [
                "Daniele Castellana",
                "Davide Bacciu"
            ],
            "abstract": "Processing sentence constituency trees in binarised form is a common and\npopular approach in literature. However, constituency trees are non-binary by\nnature. The binarisation procedure changes deeply the structure, furthering\nconstituents that instead are close. In this work, we introduce a new approach\nto deal with non-binary constituency trees which leverages tensor-based models.\nIn particular, we show how a powerful composition function based on the\ncanonical tensor decomposition can exploit such a rich structure. A key point\nof our approach is the weight sharing constraint imposed on the factor\nmatrices, which allows limiting the number of model parameters. Finally, we\nintroduce a Tree-LSTM model which takes advantage of this composition function\nand we experimentally assess its performance on different NLP tasks.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.00905v2",
            "date": "2020-11-02 11:37:17+00:00",
            "title": "Advanced Semantics for Commonsense Knowledge Extraction",
            "authors": [
                "Tuan-Phong Nguyen",
                "Simon Razniewski",
                "Gerhard Weikum"
            ],
            "abstract": "Commonsense knowledge (CSK) about concepts and their properties is useful for\nAI applications such as robust chatbots. Prior works like ConceptNet, TupleKB\nand others compiled large CSK collections, but are restricted in their\nexpressiveness to subject-predicate-object (SPO) triples with simple concepts\nfor S and monolithic strings for P and O. Also, these projects have either\nprioritized precision or recall, but hardly reconcile these complementary\ngoals. This paper presents a methodology, called Ascent, to automatically build\na large-scale knowledge base (KB) of CSK assertions, with advanced\nexpressiveness and both better precision and recall than prior works. Ascent\ngoes beyond triples by capturing composite concepts with subgroups and aspects,\nand by refining assertions with semantic facets. The latter are important to\nexpress temporal and spatial validity of assertions and further qualifiers.\nAscent combines open information extraction with judicious cleaning using\nlanguage models. Intrinsic evaluation shows the superior size and quality of\nthe Ascent KB, and an extrinsic evaluation for QA-support tasks underlines the\nbenefits of Ascent.",
            "journal ref": "WWW 2021",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2011.01007v1",
            "date": "2020-11-02 14:28:16+00:00",
            "title": "The 2020s Political Economy of Machine Translation",
            "authors": [
                "Steven Weber"
            ],
            "abstract": "This paper explores the hypothesis that the diversity of human languages,\nright now a barrier to interoperability in communication and trade, will become\nsignificantly less of a barrier as machine translation technologies are\ndeployed over the next several years.But this new boundary-breaking technology\ndoes not reduce all boundaries equally, and it creates new challenges for the\ndistribution of ideas and thus for innovation and economic growth.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2011.01102v1",
            "date": "2020-11-02 16:37:30+00:00",
            "title": "Exploring Question-Specific Rewards for Generating Deep Questions",
            "authors": [
                "Yuxi Xie",
                "Liangming Pan",
                "Dongzhe Wang",
                "Min-Yen Kan",
                "Yansong Feng"
            ],
            "abstract": "Recent question generation (QG) approaches often utilize the\nsequence-to-sequence framework (Seq2Seq) to optimize the log-likelihood of\nground-truth questions using teacher forcing. However, this training objective\nis inconsistent with actual question quality, which is often reflected by\ncertain global properties such as whether the question can be answered by the\ndocument. As such, we directly optimize for QG-specific objectives via\nreinforcement learning to improve question quality. We design three different\nrewards that target to improve the fluency, relevance, and answerability of\ngenerated questions. We conduct both automatic and human evaluations in\naddition to a thorough analysis to explore the effect of each QG-specific\nreward. We find that optimizing question-specific rewards generally leads to\nbetter performance in automatic evaluation metrics. However, only the rewards\nthat correlate well with human judgement (e.g., relevance) lead to real\nimprovement in question quality. Optimizing for the others, especially\nanswerability, introduces incorrect bias to the model, resulting in poor\nquestion quality. Our code is publicly available at\nhttps://github.com/YuxiXie/RL-for-Question-Generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01130v1",
            "date": "2020-11-02 17:07:17+00:00",
            "title": "Speaker anonymisation using the McAdams coefficient",
            "authors": [
                "Jose Patino",
                "Natalia Tomashenko",
                "Massimiliano Todisco",
                "Andreas Nautsch",
                "Nicholas Evans"
            ],
            "abstract": "Anonymisation has the goal of manipulating speech signals in order to degrade\nthe reliability of automatic approaches to speaker recognition, while\npreserving other aspects of speech, such as those relating to intelligibility\nand naturalness. This paper reports an approach to anonymisation that, unlike\nother current approaches, requires no training data, is based upon well-known\nsignal processing techniques and is both efficient and effective. The proposed\nsolution uses the McAdams coefficient to transform the spectral envelope of\nspeech signals. Results derived using common VoicePrivacy 2020 databases and\nprotocols show that random, optimised transformations can outperform competing\nsolutions in terms of anonymisation while causing only modest, additional\ndegradations to intelligibility, even in the case of a semi-informed privacy\nadversary.",
            "journal ref": "None",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2011.01136v2",
            "date": "2020-11-02 17:20:56+00:00",
            "title": "Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation",
            "authors": [
                "Ruizhe Li",
                "Xiao Li",
                "Guanyi Chen",
                "Chenghua Lin"
            ],
            "abstract": "The Variational Autoencoder (VAE) is a popular and powerful model applied to\ntext modelling to generate diverse sentences. However, an issue known as\nposterior collapse (or KL loss vanishing) happens when the VAE is used in text\nmodelling, where the approximate posterior collapses to the prior, and the\nmodel will totally ignore the latent variables and be degraded to a plain\nlanguage model during text generation. Such an issue is particularly prevalent\nwhen RNN-based VAE models are employed for text modelling. In this paper, we\npropose a simple, generic architecture called Timestep-Wise Regularisation VAE\n(TWR-VAE), which can effectively avoid posterior collapse and can be applied to\nany RNN-based VAE models. The effectiveness and versatility of our model are\ndemonstrated in different tasks, including language modelling and dialogue\nresponse generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01154v1",
            "date": "2020-11-02 17:48:25+00:00",
            "title": "Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets",
            "authors": [
                "Seid Muhie Yimam",
                "Abinew Ali Ayele",
                "Gopalakrishnan Venkatesh",
                "Chris Biemann"
            ],
            "abstract": "The availability of different pre-trained semantic models enabled the quick\ndevelopment of machine learning components for downstream applications. Despite\nthe availability of abundant text data for low resource languages, only a few\nsemantic models are publicly available. Publicly available pre-trained models\nare usually built as a multilingual version of semantic models that can not fit\nwell for each language due to context variations. In this work, we introduce\ndifferent semantic models for Amharic. After we experiment with the existing\npre-trained semantic models, we trained and fine-tuned nine new different\nmodels using a monolingual text corpus. The models are build using word2Vec\nembeddings, distributional thesaurus (DT), contextual embeddings, and DT\nembeddings obtained via network embedding algorithms. Moreover, we employ these\nmodels for different NLP tasks and investigate their impact. We find that newly\ntrained models perform better than pre-trained multilingual models.\nFurthermore, models based on contextual embeddings from RoBERTA perform better\nthan the word2Vec models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01285v1",
            "date": "2020-11-02 20:01:39+00:00",
            "title": "Exemplar Guided Active Learning",
            "authors": [
                "Jason Hartford",
                "Kevin Leyton-Brown",
                "Hadas Raviv",
                "Dan Padnos",
                "Shahar Lev",
                "Barak Lenz"
            ],
            "abstract": "We consider the problem of wisely using a limited budget to label a small\nsubset of a large unlabeled dataset. We are motivated by the NLP problem of\nword sense disambiguation. For any word, we have a set of candidate labels from\na knowledge base, but the label set is not necessarily representative of what\noccurs in the data: there may exist labels in the knowledge base that very\nrarely occur in the corpus because the sense is rare in modern English; and\nconversely there may exist true labels that do not exist in our knowledge base.\nOur aim is to obtain a classifier that performs as well as possible on examples\nof each \"common class\" that occurs with frequency above a given threshold in\nthe unlabeled set while annotating as few examples as possible from \"rare\nclasses\" whose labels occur with less than this frequency. The challenge is\nthat we are not informed which labels are common and which are rare, and the\ntrue label distribution may exhibit extreme skew. We describe an active\nlearning approach that (1) explicitly searches for rare classes by leveraging\nthe contextual embedding spaces provided by modern language models, and (2)\nincorporates a stopping rule that ignores classes once we prove that they occur\nbelow our target threshold with high probability. We prove that our algorithm\nonly costs logarithmically more than a hypothetical approach that knows all\ntrue label frequencies and show experimentally that incorporating automated\nsearch can significantly reduce the number of samples needed to reach target\naccuracy levels.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.01314v1",
            "date": "2020-11-02 20:59:26+00:00",
            "title": "Automatic Detection of Machine Generated Text: A Critical Survey",
            "authors": [
                "Ganesh Jawahar",
                "Muhammad Abdul-Mageed",
                "Laks V. S. Lakshmanan"
            ],
            "abstract": "Text generative models (TGMs) excel in producing text that matches the style\nof human language reasonably well. Such TGMs can be misused by adversaries,\ne.g., by automatically generating fake news and fake product reviews that can\nlook authentic and fool humans. Detectors that can distinguish text generated\nby TGM from human written text play a vital role in mitigating such misuse of\nTGMs. Recently, there has been a flurry of works from both natural language\nprocessing (NLP) and machine learning (ML) communities to build accurate\ndetectors for English. Despite the importance of this problem, there is\ncurrently no work that surveys this fast-growing literature and introduces\nnewcomers to important research challenges. In this work, we fill this void by\nproviding a critical survey and review of this literature to facilitate a\ncomprehensive understanding of this problem. We conduct an in-depth error\nanalysis of the state-of-the-art detector and discuss research directions to\nguide future work in this exciting area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01403v3",
            "date": "2020-11-03 01:10:39+00:00",
            "title": "Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning",
            "authors": [
                "Beliz Gunel",
                "Jingfei Du",
                "Alexis Conneau",
                "Ves Stoyanov"
            ],
            "abstract": "State-of-the-art natural language understanding classification models follow\ntwo-stages: pre-training a large language model on an auxiliary task, and then\nfine-tuning the model on a task-specific labeled dataset using cross-entropy\nloss. However, the cross-entropy loss has several shortcomings that can lead to\nsub-optimal generalization and instability. Driven by the intuition that good\ngeneralization requires capturing the similarity between examples in one class\nand contrasting them with examples in other classes, we propose a supervised\ncontrastive learning (SCL) objective for the fine-tuning stage. Combined with\ncross-entropy, our proposed SCL loss obtains significant improvements over a\nstrong RoBERTa-Large baseline on multiple datasets of the GLUE benchmark in\nfew-shot learning settings, without requiring specialized architecture, data\naugmentations, memory banks, or additional unsupervised data. Our proposed\nfine-tuning objective leads to models that are more robust to different levels\nof noise in the fine-tuning training data, and can generalize better to related\ntasks with limited labeled data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01421v1",
            "date": "2020-11-03 02:02:55+00:00",
            "title": "WSL-DS: Weakly Supervised Learning with Distant Supervision for Query Focused Multi-Document Abstractive Summarization",
            "authors": [
                "Md Tahmid Rahman Laskar",
                "Enamul Hoque",
                "Jimmy Xiangji Huang"
            ],
            "abstract": "In the Query Focused Multi-Document Summarization (QF-MDS) task, a set of\ndocuments and a query are given where the goal is to generate a summary from\nthese documents based on the given query. However, one major challenge for this\ntask is the lack of availability of labeled training datasets. To overcome this\nissue, in this paper, we propose a novel weakly supervised learning approach\nvia utilizing distant supervision. In particular, we use datasets similar to\nthe target dataset as the training data where we leverage pre-trained sentence\nsimilarity models to generate the weak reference summary of each individual\ndocument in a document set from the multi-document gold reference summaries.\nThen, we iteratively train our summarization model on each single-document to\nalleviate the computational complexity issue that occurs while training neural\nsummarization models in multiple documents (i.e., long sequences) at once.\nExperimental results in Document Understanding Conferences (DUC) datasets show\nthat our proposed approach sets a new state-of-the-art result in terms of\nvarious evaluation metrics.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01459v1",
            "date": "2020-11-03 04:05:00+00:00",
            "title": "Weakly- and Semi-supervised Evidence Extraction",
            "authors": [
                "Danish Pruthi",
                "Bhuwan Dhingra",
                "Graham Neubig",
                "Zachary C. Lipton"
            ],
            "abstract": "For many prediction tasks, stakeholders desire not only predictions but also\nsupporting evidence that a human can use to verify its correctness. However, in\npractice, additional annotations marking supporting evidence may only be\navailable for a minority of training examples (if available at all). In this\npaper, we propose new methods to combine few evidence annotations (strong\nsemi-supervision) with abundant document-level labels (weak supervision) for\nthe task of evidence extraction. Evaluating on two classification tasks that\nfeature evidence annotations, we find that our methods outperform baselines\nadapted from the interpretability literature to our task. Our approach yields\nsubstantial gains with as few as hundred evidence annotations. Code and\ndatasets to reproduce our work are available at\nhttps://github.com/danishpruthi/evidence-extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01504v1",
            "date": "2020-11-03 06:46:45+00:00",
            "title": "BioNerFlair: biomedical named entity recognition using flair embedding and sequence tagger",
            "authors": [
                "Harsh Patel"
            ],
            "abstract": "Motivation: The proliferation of Biomedical research articles has made the\ntask of information retrieval more important than ever. Scientists and\nResearchers are having difficulty in finding articles that contain information\nrelevant to them. Proper extraction of biomedical entities like Disease,\nDrug/chem, Species, Gene/protein, can considerably improve the filtering of\narticles resulting in better extraction of relevant information. Performance on\nBioNer benchmarks has progressively improved because of progression in\ntransformers-based models like BERT, XLNet, OpenAI, GPT2, etc. These models\ngive excellent results; however, they are computationally expensive and we can\nachieve better scores for domain-specific tasks using other contextual\nstring-based models and LSTM-CRF based sequence tagger. Results: We introduce\nBioNerFlair, a method to train models for biomedical named entity recognition\nusing Flair plus GloVe embeddings and Bidirectional LSTM-CRF based sequence\ntagger. With almost the same generic architecture widely used for named entity\nrecognition, BioNerFlair outperforms previous state-of-the-art models. I\nperformed experiments on 8 benchmarks datasets for biomedical named entity\nrecognition. Compared to current state-of-the-art models, BioNerFlair achieves\nthe best F1-score of 90.17 beyond 84.72 on the BioCreative II gene mention\n(BC2GM) corpus, best F1-score of 94.03 beyond 92.36 on the BioCreative IV\nchemical and drug (BC4CHEMD) corpus, best F1-score of 88.73 beyond 78.58 on the\nJNLPBA corpus, best F1-score of 91.1 beyond 89.71 on the NCBI disease corpus,\nbest F1-score of 85.48 beyond 78.98 on the Species-800 corpus, while near best\nresults was observed on BC5CDR-chem, BC3CDR-disease, and LINNAEUS corpus.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01549v1",
            "date": "2020-11-03 07:49:15+00:00",
            "title": "DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks",
            "authors": [
                "Bosheng Ding",
                "Linlin Liu",
                "Lidong Bing",
                "Canasai Kruengkrai",
                "Thien Hai Nguyen",
                "Shafiq Joty",
                "Luo Si",
                "Chunyan Miao"
            ],
            "abstract": "Data augmentation techniques have been widely used to improve machine\nlearning performance as they enhance the generalization capability of models.\nIn this work, to generate high quality synthetic data for low-resource tagging\ntasks, we propose a novel augmentation method with language models trained on\nthe linearized labeled sentences. Our method is applicable to both supervised\nand semi-supervised settings. For the supervised settings, we conduct extensive\nexperiments on named entity recognition (NER), part of speech (POS) tagging and\nend-to-end target based sentiment analysis (E2E-TBSA) tasks. For the\nsemi-supervised settings, we evaluate our method on the NER task under the\nconditions of given unlabeled data only and unlabeled data plus a knowledge\nbase. The results show that our method can consistently outperform the\nbaselines, particularly when the given gold training data are less.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01565v1",
            "date": "2020-11-03 08:44:18+00:00",
            "title": "Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings",
            "authors": [
                "Yue Wang",
                "Jing Li",
                "Michael R. Lyu",
                "Irwin King"
            ],
            "abstract": "Social media produces large amounts of contents every day. To help users\nquickly capture what they need, keyphrase prediction is receiving a growing\nattention. Nevertheless, most prior efforts focus on text modeling, largely\nignoring the rich features embedded in the matching images. In this work, we\nexplore the joint effects of texts and images in predicting the keyphrases for\na multimedia post. To better align social media style texts and images, we\npropose: (1) a novel Multi-Modality Multi-Head Attention (M3H-Att) to capture\nthe intricate cross-media interactions; (2) image wordings, in forms of optical\ncharacters and image attributes, to bridge the two modalities. Moreover, we\ndesign a unified framework to leverage the outputs of keyphrase classification\nand generation and couple their advantages. Extensive experiments on a\nlarge-scale dataset newly collected from Twitter show that our model\nsignificantly outperforms the previous state of the art based on traditional\nattention networks. Further analyses show that our multi-head attention is able\nto attend information from various aspects and boost classification or\ngeneration in diverse scenarios.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.01580v1",
            "date": "2020-11-03 09:10:48+00:00",
            "title": "CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search",
            "authors": [
                "Chenyan Xiong",
                "Zhenghao Liu",
                "Si Sun",
                "Zhuyun Dai",
                "Kaitao Zhang",
                "Shi Yu",
                "Zhiyuan Liu",
                "Hoifung Poon",
                "Jianfeng Gao",
                "Paul Bennett"
            ],
            "abstract": "Neural rankers based on deep pretrained language models (LMs) have been shown\nto improve many information retrieval benchmarks. However, these methods are\naffected by their the correlation between pretraining domain and target domain\nand rely on massive fine-tuning relevance labels. Directly applying pretraining\nmethods to specific domains may result in suboptimal search quality because\nspecific domains may have domain adaption problems, such as the COVID domain.\nThis paper presents a search system to alleviate the special domain adaption\nproblem. The system utilizes the domain-adaptive pretraining and few-shot\nlearning technologies to help neural rankers mitigate the domain discrepancy\nand label scarcity problems. Besides, we also integrate dense retrieval to\nalleviate traditional sparse retrieval's vocabulary mismatch obstacle. Our\nsystem performs the best among the non-manual runs in Round 2 of the TREC-COVID\ntask, which aims to retrieve useful information from scientific literature\nrelated to COVID-19. Our code is publicly available at\nhttps://github.com/thunlp/OpenMatch.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2011.01861v1",
            "date": "2020-11-03 17:32:50+00:00",
            "title": "DeL-haTE: A Deep Learning Tunable Ensemble for Hate Speech Detection",
            "authors": [
                "Joshua Melton",
                "Arunkumar Bagavathi",
                "Siddharth Krishnan"
            ],
            "abstract": "Online hate speech on social media has become a fast-growing problem in\nrecent times. Nefarious groups have developed large content delivery networks\nacross several main-stream (Twitter and Facebook) and fringe (Gab, 4chan,\n8chan, etc.) outlets to deliver cascades of hate messages directed both at\nindividuals and communities. Thus addressing these issues has become a top\npriority for large-scale social media outlets. Three key challenges in\nautomated detection and classification of hateful content are the lack of\nclearly labeled data, evolving vocabulary and lexicon - hashtags, emojis, etc.\n- and the lack of baseline models for fringe outlets such as Gab. In this work,\nwe propose a novel framework with three major contributions. (a) We engineer an\nensemble of deep learning models that combines the strengths of\nstate-of-the-art approaches, (b) we incorporate a tuning factor into this\nframework that leverages transfer learning to conduct automated hate speech\nclassification on unlabeled datasets, like Gab, and (c) we develop a weak\nsupervised learning methodology that allows our framework to train on unlabeled\ndata. Our ensemble models achieve an 83% hate recall on the HON dataset,\nsurpassing the performance of the state-of-the-art deep models. We demonstrate\nthat weak supervised training in combination with classifier tuning\nsignificantly increases model performance on unlabeled data from Gab, achieving\na hate recall of 67%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01900v1",
            "date": "2020-11-03 18:26:28+00:00",
            "title": "Warped Language Models for Noise Robust Language Understanding",
            "authors": [
                "Mahdi Namazifar",
                "Gokhan Tur",
                "Dilek Hakkani T\u00fcr"
            ],
            "abstract": "Masked Language Models (MLM) are self-supervised neural networks trained to\nfill in the blanks in a given sentence with masked tokens. Despite the\ntremendous success of MLMs for various text based tasks, they are not robust\nfor spoken language understanding, especially for spontaneous conversational\nspeech recognition noise. In this work we introduce Warped Language Models\n(WLM) in which input sentences at training time go through the same\nmodifications as in MLM, plus two additional modifications, namely inserting\nand dropping random tokens. These two modifications extend and contract the\nsentence in addition to the modifications in MLMs, hence the word \"warped\" in\nthe name. The insertion and drop modification of the input text during training\nof WLM resemble the types of noise due to Automatic Speech Recognition (ASR)\nerrors, and as a result WLMs are likely to be more robust to ASR noise. Through\ncomputational results we show that natural language understanding systems built\non top of WLMs perform better compared to those built based on MLMs, especially\nin the presence of ASR errors.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.01993v1",
            "date": "2020-11-03 20:15:46+00:00",
            "title": "Sound Natural: Content Rephrasing in Dialog Systems",
            "authors": [
                "Arash Einolghozati",
                "Anchit Gupta",
                "Keith Diedrick",
                "Sonal Gupta"
            ],
            "abstract": "We introduce a new task of rephrasing for a more natural virtual assistant.\nCurrently, virtual assistants work in the paradigm of intent slot tagging and\nthe slot values are directly passed as-is to the execution engine. However,\nthis setup fails in some scenarios such as messaging when the query given by\nthe user needs to be changed before repeating it or sending it to another user.\nFor example, for queries like 'ask my wife if she can pick up the kids' or\n'remind me to take my pills', we need to rephrase the content to 'can you pick\nup the kids' and 'take your pills' In this paper, we study the problem of\nrephrasing with messaging as a use case and release a dataset of 3000 pairs of\noriginal query and rephrased query. We show that BART, a pre-trained\ntransformers-based masked language model with auto-regressive decoding, is a\nstrong baseline for the task, and show improvements by adding a copy-pointer\nand copy loss to it. We analyze different tradeoffs of BART-based and\nLSTM-based seq2seq models, and propose a distilled LSTM-based seq2seq as the\nbest practical model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02068v1",
            "date": "2020-11-03 23:49:42+00:00",
            "title": "Exhaustive Entity Recognition for Coptic: Challenges and Solutions",
            "authors": [
                "Amir Zeldes",
                "Lance Martin",
                "Sichang Tu"
            ],
            "abstract": "Entity recognition provides semantic access to ancient materials in the\nDigital Humanities: itexposes people and places of interest in texts that\ncannot be read exhaustively, facilitates linkingresources and can provide a\nwindow into text contents, even for texts with no translations. Inthis paper we\npresent entity recognition for Coptic, the language of Hellenistic era Egypt.\nWeevaluate NLP approaches to the task and lay out difficulties in applying them\nto a low-resource,morphologically complex language. We present solutions for\nnamed and non-named nested en-tity recognition and semi-automatic entity\nlinking to Wikipedia, relying on robust dependencyparsing, feature-based CRF\nmodels, and hand-crafted knowledge base resources, enabling highaccuracy NER\nwith orders of magnitude less data than those used for high resource\nlanguages.The results suggest avenues for research on other languages in\nsimilar settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02160v1",
            "date": "2020-11-04 07:12:44+00:00",
            "title": "Data Augmentation for End-to-end Code-switching Speech Recognition",
            "authors": [
                "Chenpeng Du",
                "Hao Li",
                "Yizhou Lu",
                "Lan Wang",
                "Yanmin Qian"
            ],
            "abstract": "Training a code-switching end-to-end automatic speech recognition (ASR) model\nnormally requires a large amount of data, while code-switching data is often\nlimited. In this paper, three novel approaches are proposed for code-switching\ndata augmentation. Specifically, they are audio splicing with the existing\ncode-switching data, and TTS with new code-switching texts generated by word\ntranslation or word insertion. Our experiments on 200 hours Mandarin-English\ncode-switching dataset show that all the three proposed approaches yield\nsignificant improvements on code-switching ASR individually. Moreover, all the\nproposed approaches can be combined with recent popular SpecAugment, and an\naddition gain can be obtained. WER is significantly reduced by relative 24.0%\ncompared to the system without any data augmentation, and still relative 13.0%\ngain compared to the system with only SpecAugment",
            "journal ref": "2021 IEEE Spoken Language Technology Workshop (SLT)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02164v2",
            "date": "2020-11-04 07:34:54+00:00",
            "title": "An Improved Attention for Visual Question Answering",
            "authors": [
                "Tanzila Rahman",
                "Shih-Han Chou",
                "Leonid Sigal",
                "Giuseppe Carenini"
            ],
            "abstract": "We consider the problem of Visual Question Answering (VQA). Given an image\nand a free-form, open-ended, question, expressed in natural language, the goal\nof VQA system is to provide accurate answer to this question with respect to\nthe image. The task is challenging because it requires simultaneous and\nintricate understanding of both visual and textual information. Attention,\nwhich captures intra- and inter-modal dependencies, has emerged as perhaps the\nmost widely used mechanism for addressing these challenges. In this paper, we\npropose an improved attention-based architecture to solve VQA. We incorporate\nan Attention on Attention (AoA) module within encoder-decoder framework, which\nis able to determine the relation between attention results and queries.\nAttention module generates weighted average for each query. On the other hand,\nAoA module first generates an information vector and an attention gate using\nattention results and current context; and then adds another attention to\ngenerate final attended information by multiplying the two. We also propose\nmultimodal fusion module to combine both visual and textual information. The\ngoal of this fusion module is to dynamically decide how much information should\nbe considered from each modality. Extensive experiments on VQA-v2 benchmark\ndataset show that our method achieves the state-of-the-art performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.02177v4",
            "date": "2020-11-04 08:37:44+00:00",
            "title": "Diversity Aware Relevance Learning for Argument Search",
            "authors": [
                "Michael Fromm",
                "Max Berrendorf",
                "Sandra Obermeier",
                "Thomas Seidl",
                "Evgeniy Faerman"
            ],
            "abstract": "In this work, we focus on the problem of retrieving relevant arguments for a\nquery claim covering diverse aspects. State-of-the-art methods rely on explicit\nmappings between claims and premises, and thus are unable to utilize large\navailable collections of premises without laborious and costly manual\nannotation. Their diversity approach relies on removing duplicates via\nclustering which does not directly ensure that the selected premises cover all\naspects. This work introduces a new multi-step approach for the argument\nretrieval problem. Rather than relying on ground-truth assignments, our\napproach employs a machine learning model to capture semantic relationships\nbetween arguments. Beyond that, it aims to cover diverse facets of the query,\ninstead of trying to identify duplicates explicitly. Our empirical evaluation\ndemonstrates that our approach leads to a significant improvement in the\nargument retrieval task even though it requires less data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2011.02243v1",
            "date": "2020-11-04 12:03:12+00:00",
            "title": "Hybrid Supervised Reinforced Model for Dialogue Systems",
            "authors": [
                "Carlos Miranda",
                "Yacine Kessaci"
            ],
            "abstract": "This paper presents a recurrent hybrid model and training procedure for\ntask-oriented dialogue systems based on Deep Recurrent Q-Networks (DRQN). The\nmodel copes with both tasks required for Dialogue Management: State Tracking\nand Decision Making. It is based on modeling Human-Machine interaction into a\nlatent representation embedding an interaction context to guide the discussion.\nThe model achieves greater performance, learning speed and robustness than a\nnon-recurrent baseline. Moreover, results allow interpreting and validating the\npolicy evolution and the latent representations information-wise.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02266v1",
            "date": "2020-11-04 13:12:29+00:00",
            "title": "Optimizing Transformer for Low-Resource Neural Machine Translation",
            "authors": [
                "Ali Araabi",
                "Christof Monz"
            ],
            "abstract": "Language pairs with limited amounts of parallel data, also known as\nlow-resource languages, remain a challenge for neural machine translation.\nWhile the Transformer model has achieved significant improvements for many\nlanguage pairs and has become the de facto mainstream architecture, its\ncapability under low-resource conditions has not been fully investigated yet.\nOur experiments on different subsets of the IWSLT14 training data show that the\neffectiveness of Transformer under low-resource conditions is highly dependent\non the hyper-parameter settings. Our experiments show that using an optimized\nTransformer for low-resource conditions improves the translation quality up to\n7.3 BLEU points compared to using the Transformer default settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02320v3",
            "date": "2020-11-04 14:39:41+00:00",
            "title": "PCP Theorems, SETH and More: Towards Proving Sub-linear Time Inapproximability",
            "authors": [
                "Hengzhao Ma",
                "Jianzhong Li"
            ],
            "abstract": "In this paper we propose the PCP-like theorem for sub-linear time\ninapproximability. Abboud et al. have devised the distributed PCP framework for\nsub-quadratic time inapproximability. We show that the distributed PCP theorem\ncan be generalized for proving arbitrary polynomial time inapproximability, but\nfails in the linear case. We prove the sub-linear PCP theorem by adapting from\nan MA-protocol for the Set Containment problem, and show how to use the theorem\nto prove both existing and new inapproximability results, exhibiting the power\nof the sub-linear PCP theorem. Considering the emerging research works on\nsub-linear time algorithms, the sub-linear PCP theorem is important in guiding\nthe research in sub-linear time approximation algorithms.",
            "journal ref": "None",
            "category": "cs.CC"
        },
        {
            "url": "http://arxiv.org/abs/2011.02323v1",
            "date": "2020-11-04 14:43:43+00:00",
            "title": "Indic-Transformers: An Analysis of Transformer Language Models for Indian Languages",
            "authors": [
                "Kushal Jain",
                "Adwait Deshpande",
                "Kumar Shridhar",
                "Felix Laumann",
                "Ayushman Dash"
            ],
            "abstract": "Language models based on the Transformer architecture have achieved\nstate-of-the-art performance on a wide range of NLP tasks such as text\nclassification, question-answering, and token classification. However, this\nperformance is usually tested and reported on high-resource languages, like\nEnglish, French, Spanish, and German. Indian languages, on the other hand, are\nunderrepresented in such benchmarks. Despite some Indian languages being\nincluded in training multilingual Transformer models, they have not been the\nprimary focus of such work. In order to evaluate the performance on Indian\nlanguages specifically, we analyze these language models through extensive\nexperiments on multiple downstream tasks in Hindi, Bengali, and Telugu\nlanguage. Here, we compare the efficacy of fine-tuning model parameters of\npre-trained models against that of training a language model from scratch.\nMoreover, we empirically argue against the strict dependency between the\ndataset size and model performance, but rather encourage task-specific model\nand method selection. We achieve state-of-the-art performance on Hindi and\nBengali languages for text classification task. Finally, we present effective\nstrategies for handling the modeling of Indian languages and we release our\nmodel checkpoints for the community :\nhttps://huggingface.co/neuralspace-reverie.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02417v1",
            "date": "2020-11-04 17:17:49+00:00",
            "title": "Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization",
            "authors": [
                "Tristan Thrush",
                "Ethan Wilcox",
                "Roger Levy"
            ],
            "abstract": "Previous studies investigating the syntactic abilities of deep learning\nmodels have not targeted the relationship between the strength of the\ngrammatical generalization and the amount of evidence to which the model is\nexposed during training. We address this issue by deploying a novel\nword-learning paradigm to test BERT's few-shot learning capabilities for two\naspects of English verbs: alternations and classes of selectional preferences.\nFor the former, we fine-tune BERT on a single frame in a verbal-alternation\npair and ask whether the model expects the novel verb to occur in its sister\nframe. For the latter, we fine-tune BERT on an incomplete selectional network\nof verbal objects and ask whether it expects unattested but plausible\nverb/object pairs. We find that BERT makes robust grammatical generalizations\nafter just one or two instances of a novel word in fine-tuning. For the verbal\nalternation tests, we find that the model displays behavior that is consistent\nwith a transitivity bias: verbs seen few times are expected to take direct\nobjects, but verbs seen with direct objects are not expected to occur\nintransitively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02511v2",
            "date": "2020-11-04 19:30:46+00:00",
            "title": "Learning from Human Feedback: Challenges for Real-World Reinforcement Learning in NLP",
            "authors": [
                "Julia Kreutzer",
                "Stefan Riezler",
                "Carolin Lawrence"
            ],
            "abstract": "Large volumes of interaction logs can be collected from NLP systems that are\ndeployed in the real world. How can this wealth of information be leveraged?\nUsing such interaction logs in an offline reinforcement learning (RL) setting\nis a promising approach. However, due to the nature of NLP tasks and the\nconstraints of production systems, a series of challenges arise. We present a\nconcise overview of these challenges and discuss possible solutions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02593v3",
            "date": "2020-11-05 00:18:53+00:00",
            "title": "Detecting Hallucinated Content in Conditional Neural Sequence Generation",
            "authors": [
                "Chunting Zhou",
                "Graham Neubig",
                "Jiatao Gu",
                "Mona Diab",
                "Paco Guzman",
                "Luke Zettlemoyer",
                "Marjan Ghazvininejad"
            ],
            "abstract": "Neural sequence models can generate highly fluent sentences, but recent\nstudies have also shown that they are also prone to hallucinate additional\ncontent not supported by the input. These variety of fluent but wrong outputs\nare particularly problematic, as it will not be possible for users to tell they\nare being presented incorrect content. To detect these errors, we propose a\ntask to predict whether each token in the output sequence is hallucinated (not\ncontained in the input) and collect new manually annotated evaluation sets for\nthis task. We also introduce a method for learning to detect hallucinations\nusing pretrained language models fine tuned on synthetic data that includes\nautomatically inserted hallucinations Experiments on machine translation (MT)\nand abstractive summarization demonstrate that our proposed approach\nconsistently outperforms strong baselines on all benchmark datasets. We further\ndemonstrate how to use the token-level hallucination labels to define a\nfine-grained loss over the target sequence in low-resource MT and achieve\nsignificant improvements over strong baseline methods. We also apply our method\nto word-level quality estimation for MT and show its effectiveness in both\nsupervised and unsupervised settings. Codes and data available at\nhttps://github.com/violet-zct/fairseq-detect-hallucination.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02610v1",
            "date": "2020-11-05 01:52:11+00:00",
            "title": "Improving Event Duration Prediction via Time-aware Pre-training",
            "authors": [
                "Zonglin Yang",
                "Xinya Du",
                "Alexander Rush",
                "Claire Cardie"
            ],
            "abstract": "End-to-end models in NLP rarely encode external world knowledge about length\nof time. We introduce two effective models for duration prediction, which\nincorporate external knowledge by reading temporal-related news sentences\n(time-aware pre-training). Specifically, one model predicts the range/unit\nwhere the duration value falls in (R-pred); and the other predicts the exact\nduration value E-pred. Our best model -- E-pred, substantially outperforms\nprevious work, and captures duration information more accurately than R-pred.\nWe also demonstrate our models are capable of duration prediction in the\nunsupervised setting, outperforming the baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02690v1",
            "date": "2020-11-05 07:28:35+00:00",
            "title": "Entity Linking in 100 Languages",
            "authors": [
                "Jan A. Botha",
                "Zifei Shan",
                "Daniel Gillick"
            ],
            "abstract": "We propose a new formulation for multilingual entity linking, where\nlanguage-specific mentions resolve to a language-agnostic Knowledge Base. We\ntrain a dual encoder in this new setting, building on prior work with improved\nfeature representation, negative mining, and an auxiliary entity-pairing task,\nto obtain a single entity retrieval model that covers 100+ languages and 20\nmillion entities. The model outperforms state-of-the-art results from a far\nmore limited cross-lingual linking task. Rare entities and low-resource\nlanguages pose challenges at this large-scale, so we advocate for an increased\nfocus on zero- and few-shot evaluation. To this end, we provide Mewsli-9, a\nlarge new multilingual dataset (http://goo.gle/mewsli-dataset) matched to our\nsetting, and show how frequency-based analysis provided key insights for our\nmodel and training enhancements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.02705v1",
            "date": "2020-11-05 08:50:43+00:00",
            "title": "Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources",
            "authors": [
                "Qianglong Chen",
                "Feng Ji",
                "Haiqing Chen",
                "Yin Zhang"
            ],
            "abstract": "In order to facilitate natural language understanding, the key is to engage\ncommonsense or background knowledge. However, how to engage commonsense\neffectively in question answering systems is still under exploration in both\nresearch academia and industry. In this paper, we propose a novel\nquestion-answering method by integrating multiple knowledge sources, i.e.\nConceptNet, Wikipedia, and the Cambridge Dictionary, to boost the performance.\nMore concretely, we first introduce a novel graph-based iterative knowledge\nretrieval module, which iteratively retrieves concepts and entities related to\nthe given question and its choices from multiple knowledge sources. Afterward,\nwe use a pre-trained language model to encode the question, retrieved knowledge\nand choices, and propose an answer choice-aware attention mechanism to fuse all\nhidden representations of the previous modules. Finally, the linear classifier\nfor specific tasks is used to predict the answer. Experimental results on the\nCommonsenseQA dataset show that our method significantly outperforms other\ncompetitive methods and achieves the new state-of-the-art. In addition, further\nablation studies demonstrate the effectiveness of our graph-based iterative\nknowledge retrieval module and the answer choice-aware attention module in\nretrieving and synthesizing background knowledge from multiple knowledge\nsources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03023v1",
            "date": "2020-11-05 18:31:22+00:00",
            "title": "Language Model is All You Need: Natural Language Understanding as Question Answering",
            "authors": [
                "Mahdi Namazifar",
                "Alexandros Papangelis",
                "Gokhan Tur",
                "Dilek Hakkani-T\u00fcr"
            ],
            "abstract": "Different flavors of transfer learning have shown tremendous impact in\nadvancing research and applications of machine learning. In this work we study\nthe use of a specific family of transfer learning, where the target domain is\nmapped to the source domain. Specifically we map Natural Language Understanding\n(NLU) problems to QuestionAnswering (QA) problems and we show that in low data\nregimes this approach offers significant improvements compared to other\napproaches to NLU. Moreover we show that these gains could be increased through\nsequential transfer learning across NLU problems from different domains. We\nshow that our approach could reduce the amount of required data for the same\nperformance by up to a factor of 10.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03088v2",
            "date": "2020-11-05 20:33:11+00:00",
            "title": "HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification",
            "authors": [
                "Yichen Jiang",
                "Shikha Bordia",
                "Zheng Zhong",
                "Charles Dognin",
                "Maneesh Singh",
                "Mohit Bansal"
            ],
            "abstract": "We introduce HoVer (HOppy VERification), a dataset for many-hop evidence\nextraction and fact verification. It challenges models to extract facts from\nseveral Wikipedia articles that are relevant to a claim and classify whether\nthe claim is Supported or Not-Supported by the facts. In HoVer, the claims\nrequire evidence to be extracted from as many as four English Wikipedia\narticles and embody reasoning graphs of diverse shapes. Moreover, most of the\n3/4-hop claims are written in multiple sentences, which adds to the complexity\nof understanding long-range dependency relations such as coreference. We show\nthat the performance of an existing state-of-the-art semantic-matching model\ndegrades significantly on our dataset as the number of reasoning hops\nincreases, hence demonstrating the necessity of many-hop reasoning to achieve\nstrong results. We hope that the introduction of this challenging dataset and\nthe accompanying evaluation task will encourage research in many-hop fact\nretrieval and information verification. We make the HoVer dataset publicly\navailable at https://hover-nlp.github.io",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03092v1",
            "date": "2020-11-05 20:50:22+00:00",
            "title": "Machine Generation and Detection of Arabic Manipulated and Fake News",
            "authors": [
                "El Moatez Billah Nagoudi",
                "AbdelRahim Elmadany",
                "Muhammad Abdul-Mageed",
                "Tariq Alhindi",
                "Hasan Cavusoglu"
            ],
            "abstract": "Fake news and deceptive machine-generated text are serious problems\nthreatening modern societies, including in the Arab world. This motivates work\non detecting false and manipulated stories online. However, a bottleneck for\nthis research is lack of sufficient data to train detection models. We present\na novel method for automatically generating Arabic manipulated (and potentially\nfake) news stories. Our method is simple and only depends on availability of\ntrue stories, which are abundant online, and a part of speech tagger (POS). To\nfacilitate future work, we dispense with both of these requirements altogether\nby providing AraNews, a novel and large POS-tagged news dataset that can be\nused off-the-shelf. Using stories generated based on AraNews, we carry out a\nhuman annotation study that casts light on the effects of machine manipulation\non text veracity. The study also measures human ability to detect Arabic\nmachine manipulated text generated by our method. Finally, we develop the first\nmodels for detecting manipulated Arabic news and achieve state-of-the-art\nresults on Arabic fake news detection (macro F1=70.06). Our models and data are\npublicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03096v1",
            "date": "2020-11-05 21:18:21+00:00",
            "title": "Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering",
            "authors": [
                "Quan Tran",
                "Nhan Dam",
                "Tuan Lai",
                "Franck Dernoncourt",
                "Trung Le",
                "Nham Le",
                "Dinh Phung"
            ],
            "abstract": "Interpretability and explainability of deep neural networks are challenging\ndue to their scale, complexity, and the agreeable notions on which the\nexplaining process rests. Previous work, in particular, has focused on\nrepresenting internal components of neural networks through human-friendly\nvisuals and concepts. On the other hand, in real life, when making a decision,\nhuman tends to rely on similar situations and/or associations in the past.\nHence arguably, a promising approach to make the model transparent is to design\nit in a way such that the model explicitly connects the current sample with the\nseen ones, and bases its decision on these samples. Grounded on that principle,\nwe propose in this paper an explainable, evidence-based memory network\narchitecture, which learns to summarize the dataset and extract supporting\nevidences to make its decision. Our model achieves state-of-the-art performance\non two popular question answering datasets (i.e. TrecQA and WikiQA). Via\nfurther analysis, we show that this model can reliably trace the errors it has\nmade in the validation step to the training instances that might have caused\nthese errors. We believe that this error-tracing capability provides\nsignificant benefit in improving dataset quality in many applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03138v1",
            "date": "2020-11-05 23:31:00+00:00",
            "title": "Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities",
            "authors": [
                "Hao Zhang",
                "Jae Ro",
                "Richard Sproat"
            ],
            "abstract": "Breaking domain names such as openresearch into component words open and\nresearch is important for applications like Text-to-Speech synthesis and web\nsearch. We link this problem to the classic problem of Chinese word\nsegmentation and show the effectiveness of a tagging model based on Recurrent\nNeural Networks (RNNs) using characters as input. To compensate for the lack of\ntraining data, we propose a pre-training method on concatenated entity names in\na large knowledge database. Pre-training improves the model by 33% and brings\nthe sequence accuracy to 85%.",
            "journal ref": "Proceedings of the 28th International Conference on Computational\n  Linguistics (2020) 4667--4675",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03228v1",
            "date": "2020-11-06 08:22:12+00:00",
            "title": "From Dataset Recycling to Multi-Property Extraction and Beyond",
            "authors": [
                "Tomasz Dwojak",
                "Micha\u0142 Pietruszka",
                "\u0141ukasz Borchmann",
                "Jakub Ch\u0142\u0119dowski",
                "Filip Grali\u0144ski"
            ],
            "abstract": "This paper investigates various Transformer architectures on the WikiReading\nInformation Extraction and Machine Reading Comprehension dataset. The proposed\ndual-source model outperforms the current state-of-the-art by a large margin.\nNext, we introduce WikiReading Recycled-a newly developed public dataset and\nthe task of multiple property extraction. It uses the same data as WikiReading\nbut does not inherit its predecessor's identified disadvantages. In addition,\nwe provide a human-annotated test set with diagnostic subsets for a detailed\nanalysis of model performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03281v1",
            "date": "2020-11-06 11:00:47+00:00",
            "title": "Corpora Compared: The Case of the Swedish Gigaword & Wikipedia Corpora",
            "authors": [
                "Tosin P. Adewumi",
                "Foteini Liwicki",
                "Marcus Liwicki"
            ],
            "abstract": "In this work, we show that the difference in performance of embeddings from\ndifferently sourced data for a given language can be due to other factors\nbesides data size. Natural language processing (NLP) tasks usually perform\nbetter with embeddings from bigger corpora. However, broadness of covered\ndomain and noise can play important roles. We evaluate embeddings based on two\nSwedish corpora: The Gigaword and Wikipedia, in analogy (intrinsic) tests and\ndiscover that the embeddings from the Wikipedia corpus generally outperform\nthose from the Gigaword corpus, which is a bigger corpus. Downstream tests will\nbe required to have a definite evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03502v1",
            "date": "2020-11-06 18:19:48+00:00",
            "title": "An Unsupervised method for OCR Post-Correction and Spelling Normalisation for Finnish",
            "authors": [
                "Quan Duong",
                "Mika H\u00e4m\u00e4l\u00e4inen",
                "Simon Hengchen"
            ],
            "abstract": "Historical corpora are known to contain errors introduced by OCR (optical\ncharacter recognition) methods used in the digitization process, often said to\nbe degrading the performance of NLP systems. Correcting these errors manually\nis a time-consuming process and a great part of the automatic approaches have\nbeen relying on rules or supervised machine learning. We build on previous work\non fully automatic unsupervised extraction of parallel data to train a\ncharacter-based sequence-to-sequence NMT (neural machine translation) model to\nconduct OCR error correction designed for English, and adapt it to Finnish by\nproposing solutions that take the rich morphology of the language into account.\nOur new method shows increased performance while remaining fully unsupervised,\nwith the added benefit of spelling normalisation. The source code and models\nare available on GitHub and Zenodo.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03646v2",
            "date": "2020-11-07 00:35:31+00:00",
            "title": "Acoustics Based Intent Recognition Using Discovered Phonetic Units for Low Resource Languages",
            "authors": [
                "Akshat Gupta",
                "Xinjian Li",
                "Sai Krishna Rallabandi",
                "Alan W Black"
            ],
            "abstract": "With recent advancements in language technologies, humans are now speaking to\ndevices. Increasing the reach of spoken language technologies requires building\nsystems in local languages. A major bottleneck here are the underlying\ndata-intensive parts that make up such systems, including automatic speech\nrecognition (ASR) systems that require large amounts of labelled data. With the\naim of aiding development of spoken dialog systems in low resourced languages,\nwe propose a novel acoustics based intent recognition system that uses\ndiscovered phonetic units for intent classification. The system is made up of\ntwo blocks - the first block is a universal phone recognition system that\ngenerates a transcript of discovered phonetic units for the input audio, and\nthe second block performs intent classification from the generated phonetic\ntranscripts. We propose a CNN+LSTM based architecture and present results for\ntwo languages families - Indic languages and Romance languages, for two\ndifferent intent recognition tasks. We also perform multilingual training of\nour intent classifier and show improved cross-lingual transfer and zero-shot\nperformance on an unknown language within the same language family.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03755v1",
            "date": "2020-11-07 11:27:18+00:00",
            "title": "NLP-CIC @ DIACR-Ita: POS and Neighbor Based Distributional Models for Lexical Semantic Change in Diachronic Italian Corpora",
            "authors": [
                "Jason Angel",
                "Carlos A. Rodriguez-Diaz",
                "Alexander Gelbukh",
                "Sergio Jimenez"
            ],
            "abstract": "We present our systems and findings on unsupervised lexical semantic change\nfor the Italian language in the DIACR-Ita shared-task at EVALITA 2020. The task\nis to determine whether a target word has evolved its meaning with time, only\nrelying on raw-text from two time-specific datasets. We propose two models\nrepresenting the target words across the periods to predict the changing words\nusing threshold and voting schemes. Our first model solely relies on\npart-of-speech usage and an ensemble of distance measures. The second model\nuses word embedding representation to extract the neighbor's relative distances\nacross spaces and propose \"the average of absolute differences\" to estimate\nlexical semantic change. Our models achieved competent results, ranking third\nin the DIACR-Ita competition. Furthermore, we experiment with the k_neighbor\nparameter of our second model to compare the impact of using \"the average of\nabsolute differences\" versus the cosine distance used in Hamilton et al.\n(2016).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03760v1",
            "date": "2020-11-07 12:13:09+00:00",
            "title": "NLP-CIC @ PRELEARN: Mastering prerequisites relations, from handcrafted features to embeddings",
            "authors": [
                "Jason Angel",
                "Segun Taofeek Aroyehun",
                "Alexander Gelbukh"
            ],
            "abstract": "We present our systems and findings for the prerequisite relation learning\ntask (PRELEARN) at EVALITA 2020. The task aims to classify whether a pair of\nconcepts hold a prerequisite relation or not. We model the problem using\nhandcrafted features and embedding representations for in-domain and\ncross-domain scenarios. Our submissions ranked first place in both scenarios\nwith average F1 score of 0.887 and 0.690 respectively across domains on the\ntest sets. We made our code is freely available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03783v1",
            "date": "2020-11-07 14:28:54+00:00",
            "title": "AlphaMWE: Construction of Multilingual Parallel Corpora with MWE Annotations",
            "authors": [
                "Lifeng Han",
                "Gareth Jones",
                "Alan Smeaton"
            ],
            "abstract": "In this work, we present the construction of multilingual parallel corpora\nwith annotation of multiword expressions (MWEs). MWEs include verbal MWEs\n(vMWEs) defined in the PARSEME shared task that have a verb as the head of the\nstudied terms. The annotated vMWEs are also bilingually and multilingually\naligned manually. The languages covered include English, Chinese, Polish, and\nGerman. Our original English corpus is taken from the PARSEME shared task in\n2018. We performed machine translation of this source corpus followed by human\npost editing and annotation of target MWEs. Strict quality control was applied\nfor error limitation, i.e., each MT output sentence received first manual post\nediting and annotation plus second manual quality rechecking. One of our\nfindings during corpora preparation is that accurate translation of MWEs\npresents challenges to MT systems. To facilitate further MT research, we\npresent a categorisation of the error types encountered by MT systems in\nperforming MWE related translation. To acquire a broader view of MT issues, we\nselected four popular state-of-the-art MT models for comparisons namely:\nMicrosoft Bing Translator, GoogleMT, Baidu Fanyi and DeepL MT. Because of the\nnoise removal, translation post editing and MWE annotation by human\nprofessionals, we believe our AlphaMWE dataset will be an asset for\ncross-lingual and multilingual research, such as MT and information extraction.\nOur multilingual corpora are available as open access at\ngithub.com/poethan/AlphaMWE.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03798v3",
            "date": "2020-11-07 16:09:03+00:00",
            "title": "PairRE: Knowledge Graph Embeddings via Paired Relation Vectors",
            "authors": [
                "Linlin Chao",
                "Jianshan He",
                "Taifeng Wang",
                "Wei Chu"
            ],
            "abstract": "Distance based knowledge graph embedding methods show promising results on\nlink prediction task, on which two topics have been widely studied: one is the\nability to handle complex relations, such as N-to-1, 1-to-N and N-to-N, the\nother is to encode various relation patterns, such as symmetry/antisymmetry.\nHowever, the existing methods fail to solve these two problems at the same\ntime, which leads to unsatisfactory results. To mitigate this problem, we\npropose PairRE, a model with paired vectors for each relation representation.\nThe paired vectors enable an adaptive adjustment of the margin in loss function\nto fit for complex relations. Besides, PairRE is capable of encoding three\nimportant relation patterns, symmetry/antisymmetry, inverse and composition.\nGiven simple constraints on relation representations, PairRE can encode\nsubrelation further. Experiments on link prediction benchmarks demonstrate the\nproposed key capabilities of PairRE. Moreover, We set a new state-of-the-art on\ntwo knowledge graph datasets of the challenging Open Graph Benchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03863v2",
            "date": "2020-11-07 22:52:21+00:00",
            "title": "Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering",
            "authors": [
                "Kaixin Ma",
                "Filip Ilievski",
                "Jonathan Francis",
                "Yonatan Bisk",
                "Eric Nyberg",
                "Alessandro Oltramari"
            ],
            "abstract": "Recent developments in pre-trained neural language modeling have led to leaps\nin accuracy on commonsense question-answering benchmarks. However, there is\nincreasing concern that models overfit to specific tasks, without learning to\nutilize external knowledge or perform general semantic reasoning. In contrast,\nzero-shot evaluations have shown promise as a more robust measure of a model's\ngeneral reasoning abilities. In this paper, we propose a novel neuro-symbolic\nframework for zero-shot question answering across commonsense tasks. Guided by\na set of hypotheses, the framework studies how to transform various\npre-existing knowledge resources into a form that is most effective for\npre-training models. We vary the set of language models, training regimes,\nknowledge sources, and data generation strategies, and measure their impact\nacross tasks. Extending on prior work, we devise and compare four constrained\ndistractor-sampling strategies. We provide empirical results across five\ncommonsense question-answering tasks with data generated from five external\nknowledge resources. We show that, while an individual knowledge graph is\nbetter suited for specific tasks, a global knowledge graph brings consistent\ngains across different tasks. In addition, both preserving the structure of the\ntask as well as generating fair and informative questions help language models\nlearn more effectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03870v1",
            "date": "2020-11-07 23:56:02+00:00",
            "title": "Explainable Automated Fact-Checking: A Survey",
            "authors": [
                "Neema Kotonya",
                "Francesca Toni"
            ],
            "abstract": "A number of exciting advances have been made in automated fact-checking\nthanks to increasingly larger datasets and more powerful systems, leading to\nimprovements in the complexity of claims which can be accurately fact-checked.\nHowever, despite these advances, there are still desirable functionalities\nmissing from the fact-checking pipeline. In this survey, we focus on the\nexplanation functionality -- that is fact-checking systems providing reasons\nfor their predictions. We summarize existing methods for explaining the\npredictions of fact-checking systems and we explore trends in this topic.\nFurther, we consider what makes for good explanations in this specific domain\nthrough a comparative analysis of existing fact-checking explanations against\nsome desirable properties. Finally, we propose further research directions for\ngenerating fact-checking explanations, and describe how these may lead to\nimprovements in the research area.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.03965v1",
            "date": "2020-11-08 12:15:31+00:00",
            "title": "On the Practical Ability of Recurrent Neural Networks to Recognize Hierarchical Languages",
            "authors": [
                "Satwik Bhattamishra",
                "Kabir Ahuja",
                "Navin Goyal"
            ],
            "abstract": "While recurrent models have been effective in NLP tasks, their performance on\ncontext-free languages (CFLs) has been found to be quite weak. Given that CFLs\nare believed to capture important phenomena such as hierarchical structure in\nnatural languages, this discrepancy in performance calls for an explanation. We\nstudy the performance of recurrent models on Dyck-n languages, a particularly\nimportant and well-studied class of CFLs. We find that while recurrent models\ngeneralize nearly perfectly if the lengths of the training and test strings are\nfrom the same range, they perform poorly if the test strings are longer. At the\nsame time, we observe that recurrent models are expressive enough to recognize\nDyck words of arbitrary lengths in finite precision if their depths are\nbounded. Hence, we evaluate our models on samples generated from Dyck languages\nwith bounded depth and find that they are indeed able to generalize to much\nhigher lengths. Since natural language datasets have nested dependencies of\nbounded depth, this may help explain why they perform well in modeling\nhierarchical dependencies in natural language data despite prior works\nindicating poor generalization performance on Dyck languages. We perform\nprobing studies to support our results and provide comparisons with\nTransformers.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04044v1",
            "date": "2020-11-08 18:18:15+00:00",
            "title": "Exploring End-to-End Differentiable Natural Logic Modeling",
            "authors": [
                "Yufei Feng",
                "Zi'ou Zheng",
                "Quan Liu",
                "Michael Greenspan",
                "Xiaodan Zhu"
            ],
            "abstract": "We explore end-to-end trained differentiable models that integrate natural\nlogic with neural networks, aiming to keep the backbone of natural language\nreasoning based on the natural logic formalism while introducing subsymbolic\nvector representations and neural components. The proposed model adapts module\nnetworks to model natural logic operations, which is enhanced with a memory\ncomponent to model contextual information. Experiments show that the proposed\nframework can effectively model monotonicity-based reasoning, compared to the\nbaseline neural network models without built-in inductive bias for\nmonotonicity-based reasoning. Our proposed model shows to be robust when\ntransferred from upward to downward inference. We perform further analyses on\nthe performance of the proposed model on aggregation, showing the effectiveness\nof the proposed subcomponents on helping achieve better intermediate\naggregation performance.",
            "journal ref": "COLING 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04096v1",
            "date": "2020-11-08 22:26:06+00:00",
            "title": "Metrics also Disagree in the Low Scoring Range: Revisiting Summarization Evaluation Metrics",
            "authors": [
                "Manik Bhandari",
                "Pranav Gour",
                "Atabak Ashfaq",
                "Pengfei Liu"
            ],
            "abstract": "In text summarization, evaluating the efficacy of automatic metrics without\nhuman judgments has become recently popular. One exemplar work concludes that\nautomatic metrics strongly disagree when ranking high-scoring summaries. In\nthis paper, we revisit their experiments and find that their observations stem\nfrom the fact that metrics disagree in ranking summaries from any narrow\nscoring range. We hypothesize that this may be because summaries are similar to\neach other in a narrow scoring range and are thus, difficult to rank. Apart\nfrom the width of the scoring range of summaries, we analyze three other\nproperties that impact inter-metric agreement - Ease of Summarization,\nAbstractiveness, and Coverage. To encourage reproducible research, we make all\nour analysis code and data publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04242v1",
            "date": "2020-11-09 08:17:22+00:00",
            "title": "AI Stories: An Interactive Narrative System for Children",
            "authors": [
                "Ben Burtenshaw"
            ],
            "abstract": "AI Stories is a proposed interactive dialogue system, that lets children\nco-create narrative worlds through conversation. Over the next three years this\nsystem will be developed and tested within pediatric wards, where it offers a\nuseful resource between the gap of education and play. Telling and making\nstories is a fundamental part of language play, and its chatty and nonsensical\nqualities are important; therefore, the prologued usage an automated system\noffers is a benefit to children. In this paper I will present the current state\nof this project, in its more experimental and general guise. Conceptually\nstory-telling through dialogue relates to the preprint interpretation of story,\nbeyond the static and linear medium, where stories were performative, temporal,\nand social.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04264v1",
            "date": "2020-11-09 09:23:55+00:00",
            "title": "CapWAP: Captioning with a Purpose",
            "authors": [
                "Adam Fisch",
                "Kenton Lee",
                "Ming-Wei Chang",
                "Jonathan H. Clark",
                "Regina Barzilay"
            ],
            "abstract": "The traditional image captioning task uses generic reference captions to\nprovide textual information about images. Different user populations, however,\nwill care about different visual aspects of images. In this paper, we propose a\nnew task, Captioning with a Purpose (CapWAP). Our goal is to develop systems\nthat can be tailored to be useful for the information needs of an intended\npopulation, rather than merely provide generic information about an image. In\nthis task, we use question-answer (QA) pairs---a natural expression of\ninformation need---from users, instead of reference captions, for both training\nand post-inference evaluation. We show that it is possible to use reinforcement\nlearning to directly optimize for the intended information need, by rewarding\noutputs that allow a question answering model to provide correct answers to\nsampled user questions. We convert several visual question answering datasets\ninto CapWAP datasets, and demonstrate that under a variety of scenarios our\npurposeful captioning system learns to anticipate and fulfill specific\ninformation needs better than its generic counterparts, as measured by QA\nperformance on user questions from unseen images, when using the caption alone\nas context.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04446v1",
            "date": "2020-11-09 14:12:07+00:00",
            "title": "Bangla Text Classification using Transformers",
            "authors": [
                "Tanvirul Alam",
                "Akib Khan",
                "Firoj Alam"
            ],
            "abstract": "Text classification has been one of the earliest problems in NLP. Over time\nthe scope of application areas has broadened and the difficulty of dealing with\nnew areas (e.g., noisy social media content) has increased. The problem-solving\nstrategy switched from classical machine learning to deep learning algorithms.\nOne of the recent deep neural network architecture is the Transformer. Models\ndesigned with this type of network and its variants recently showed their\nsuccess in many downstream natural language processing tasks, especially for\nresource-rich languages, e.g., English. However, these models have not been\nexplored fully for Bangla text classification tasks. In this work, we fine-tune\nmultilingual transformer models for Bangla text classification tasks in\ndifferent domains, including sentiment analysis, emotion detection, news\ncategorization, and authorship attribution. We obtain the state of the art\nresults on six benchmark datasets, improving upon the previous results by 5-29%\naccuracy across different tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04451v1",
            "date": "2020-10-17 09:23:04+00:00",
            "title": "Hierarchical Multitask Learning Approach for BERT",
            "authors": [
                "\u00c7a\u011fla Aksoy",
                "Alper Ahmeto\u011flu",
                "Tunga G\u00fcng\u00f6r"
            ],
            "abstract": "Recent works show that learning contextualized embeddings for words is\nbeneficial for downstream tasks. BERT is one successful example of this\napproach. It learns embeddings by solving two tasks, which are masked language\nmodel (masked LM) and the next sentence prediction (NSP). The pre-training of\nBERT can also be framed as a multitask learning problem. In this work, we adopt\nhierarchical multitask learning approaches for BERT pre-training. Pre-training\ntasks are solved at different layers instead of the last layer, and information\nfrom the NSP task is transferred to the masked LM task. Also, we propose a new\npre-training task bigram shift to encode word order information. We choose two\ndownstream tasks, one of which requires sentence-level embeddings (textual\nentailment), and the other requires contextualized embeddings of words\n(question answering). Due to computational restrictions, we use the downstream\ntask data instead of a large dataset for the pre-training to see the\nperformance of proposed models when given a restricted dataset. We test their\nperformance on several probing tasks to analyze learned embeddings. Our results\nshow that imposing a task hierarchy in pre-training improves the performance of\nembeddings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04512v2",
            "date": "2020-10-24 02:51:17+00:00",
            "title": "Auxiliary Sequence Labeling Tasks for Disfluency Detection",
            "authors": [
                "Dongyub Lee",
                "Byeongil Ko",
                "Myeong Cheol Shin",
                "Taesun Whang",
                "Daniel Lee",
                "Eun Hwa Kim",
                "EungGyun Kim",
                "Jaechoon Jo"
            ],
            "abstract": "Detecting disfluencies in spontaneous speech is an important preprocessing\nstep in natural language processing and speech recognition applications.\nExisting works for disfluency detection have focused on designing a single\nobjective only for disfluency detection, while auxiliary objectives utilizing\nlinguistic information of a word such as named entity or part-of-speech\ninformation can be effective. In this paper, we focus on detecting disfluencies\non spoken transcripts and propose a method utilizing named entity recognition\n(NER) and part-of-speech (POS) as auxiliary sequence labeling (SL) tasks for\ndisfluency detection. First, we investigate cases that utilizing linguistic\ninformation of a word can prevent mispredicting important words and can be\nhelpful for the correct detection of disfluencies. Second, we show that\ntraining a disfluency detection model with auxiliary SL tasks can improve its\nF-score in disfluency detection. Then, we analyze which auxiliary SL tasks are\ninfluential depending on baseline models. Experimental results on the widely\nused English Switchboard dataset show that our method outperforms the previous\nstate-of-the-art in disfluency detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04521v1",
            "date": "2020-11-09 15:57:53+00:00",
            "title": "Automated Discovery of Mathematical Definitions in Text with Deep Neural Networks",
            "authors": [
                "Natalia Vanetik",
                "Marina Litvak",
                "Sergey Shevchuk",
                "Lior Reznik"
            ],
            "abstract": "Automatic definition extraction from texts is an important task that has\nnumerous applications in several natural language processing fields such as\nsummarization, analysis of scientific texts, automatic taxonomy generation,\nontology generation, concept identification, and question answering. For\ndefinitions that are contained within a single sentence, this problem can be\nviewed as a binary classification of sentences into definitions and\nnon-definitions. In this paper, we focus on automatic detection of one-sentence\ndefinitions in mathematical texts, which are difficult to separate from\nsurrounding text. We experiment with several data representations, which\ninclude sentence syntactic structure and word embeddings, and apply deep\nlearning methods such as the Convolutional Neural Network (CNN) and the Long\nShort-Term Memory network (LSTM), in order to identify mathematical\ndefinitions. Our experiments demonstrate the superiority of CNN and its\ncombination with LSTM, when applied on the syntactically-enriched input\nrepresentation. We also present a new dataset for definition extraction from\nmathematical texts. We demonstrate that this dataset is beneficial for training\nsupervised models aimed at extraction of mathematical definitions. Our\nexperiments with different domains demonstrate that mathematical definitions\nrequire special treatment, and that using cross-domain learning is inefficient\nfor that task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04554v1",
            "date": "2020-11-09 16:53:54+00:00",
            "title": "Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts",
            "authors": [
                "Ece Takmaz",
                "Mario Giulianelli",
                "Sandro Pezzelle",
                "Arabella Sinclair",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "Dialogue participants often refer to entities or situations repeatedly within\na conversation, which contributes to its cohesiveness. Subsequent references\nexploit the common ground accumulated by the interlocutors and hence have\nseveral interesting properties, namely, they tend to be shorter and reuse\nexpressions that were effective in previous mentions. In this paper, we tackle\nthe generation of first and subsequent references in visually grounded\ndialogue. We propose a generation model that produces referring utterances\ngrounded in both the visual and the conversational context. To assess the\nreferring effectiveness of its output, we also implement a reference resolution\nsystem. Our experiments and analyses show that the model produces better, more\neffective referring utterances than a model not grounded in the dialogue\ncontext, and generates subsequent references that exhibit linguistic patterns\nakin to humans.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04592v1",
            "date": "2020-11-09 17:45:32+00:00",
            "title": "Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze",
            "authors": [
                "Ece Takmaz",
                "Sandro Pezzelle",
                "Lisa Beinborn",
                "Raquel Fern\u00e1ndez"
            ],
            "abstract": "When speakers describe an image, they tend to look at objects before\nmentioning them. In this paper, we investigate such sequential cross-modal\nalignment by modelling the image description generation process\ncomputationally. We take as our starting point a state-of-the-art image\ncaptioning system and develop several model variants that exploit information\nfrom human gaze patterns recorded during language production. In particular, we\npropose the first approach to image description generation where visual\nprocessing is modelled $\\textit{sequentially}$. Our experiments and analyses\nconfirm that better descriptions can be obtained by exploiting gaze-driven\nattention and shed light on human cognitive processes by comparing different\nways of aligning the gaze modality with language production. We find that\nprocessing gaze data sequentially leads to descriptions that are better aligned\nto those produced by speakers, more diverse, and more natural${-}$particularly\nwhen gaze is encoded with a dedicated recurrent component.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04640v1",
            "date": "2020-11-09 18:51:55+00:00",
            "title": "Scaling Hidden Markov Language Models",
            "authors": [
                "Justin T. Chiu",
                "Alexander M. Rush"
            ],
            "abstract": "The hidden Markov model (HMM) is a fundamental tool for sequence modeling\nthat cleanly separates the hidden state from the emission structure. However,\nthis separation makes it difficult to fit HMMs to large datasets in modern NLP,\nand they have fallen out of use due to very poor performance compared to fully\nobserved models. This work revisits the challenge of scaling HMMs to language\nmodeling datasets, taking ideas from recent approaches to neural modeling. We\npropose methods for scaling HMMs to massive state spaces while maintaining\nefficient exact inference, a compact parameterization, and effective\nregularization. Experiments show that this approach leads to models that are\nmore accurate than previous HMM and n-gram-based methods, making progress\ntowards the performance of state-of-the-art neural models.",
            "journal ref": "EMNLP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04743v1",
            "date": "2020-11-09 20:42:01+00:00",
            "title": "Adversarial Semantic Collisions",
            "authors": [
                "Congzheng Song",
                "Alexander M. Rush",
                "Vitaly Shmatikov"
            ],
            "abstract": "We study semantic collisions: texts that are semantically unrelated but\njudged as similar by NLP models. We develop gradient-based approaches for\ngenerating semantic collisions and demonstrate that state-of-the-art models for\nmany tasks which rely on analyzing the meaning and similarity of texts--\nincluding paraphrase identification, document retrieval, response suggestion,\nand extractive summarization-- are vulnerable to semantic collisions. For\nexample, given a target query, inserting a crafted collision into an irrelevant\ndocument can shift its retrieval rank from 1000 to top 3. We show how to\ngenerate semantic collisions that evade perplexity-based filtering and discuss\nother potential mitigations. Our code is available at\nhttps://github.com/csong27/collision-bert.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04823v1",
            "date": "2020-11-09 23:17:43+00:00",
            "title": "Language Through a Prism: A Spectral Approach for Multiscale Language Representations",
            "authors": [
                "Alex Tamkin",
                "Dan Jurafsky",
                "Noah Goodman"
            ],
            "abstract": "Language exhibits structure at different scales, ranging from subwords to\nwords, sentences, paragraphs, and documents. To what extent do deep models\ncapture information at these scales, and can we force them to better capture\nstructure across this hierarchy? We approach this question by focusing on\nindividual neurons, analyzing the behavior of their activations at different\ntimescales. We show that signal processing provides a natural framework for\nseparating structure across scales, enabling us to 1) disentangle\nscale-specific information in existing embeddings and 2) train models to learn\nmore about particular scales. Concretely, we apply spectral filters to the\nactivations of a neuron across an input, producing filtered embeddings that\nperform well on part of speech tagging (word-level), dialog speech acts\nclassification (utterance-level), or topic classification (document-level),\nwhile performing poorly on the other tasks. We also present a prism layer for\ntraining models, which uses spectral filters to constrain different neurons to\nmodel structure at different scales. Our proposed BERT + Prism model can better\npredict masked tokens using long-range context and produces multiscale\nrepresentations that perform better at utterance- and document-level tasks. Our\nmethods are general and readily applicable to other domains besides language,\nsuch as images, audio, and video.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04839v1",
            "date": "2020-11-10 00:19:04+00:00",
            "title": "Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis",
            "authors": [
                "Erica Cooper",
                "Xin Wang",
                "Yi Zhao",
                "Yusuke Yasuda",
                "Junichi Yamagishi"
            ],
            "abstract": "We explore pretraining strategies including choice of base corpus with the\naim of choosing the best strategy for zero-shot multi-speaker end-to-end\nsynthesis. We also examine choice of neural vocoder for waveform synthesis, as\nwell as acoustic configurations used for mel spectrograms and final audio\noutput. We find that fine-tuning a multi-speaker model from found audiobook\ndata that has passed a simple quality threshold can improve naturalness and\nsimilarity to unseen target speakers of synthetic speech. Additionally, we find\nthat listeners can discern between a 16kHz and 24kHz sampling rate, and that\nWaveRNN produces output waveforms of a comparable quality to WaveNet, with a\nfaster inference time.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2011.04843v2",
            "date": "2020-11-10 00:35:46+00:00",
            "title": "Multi-document Summarization via Deep Learning Techniques: A Survey",
            "authors": [
                "Congbo Ma",
                "Wei Emma Zhang",
                "Mingyu Guo",
                "Hu Wang",
                "Quan Z. Sheng"
            ],
            "abstract": "Multi-document summarization (MDS) is an effective tool for information\naggregation which generates an informative and concise summary from a cluster\nof topic-related documents. Our survey structurally overviews the recent deep\nlearning based multi-document summarization models via a proposed taxonomy and\nit is the first of its kind. Particularly, we propose a novel mechanism to\nsummarize the design strategies of neural networks and conduct a comprehensive\nsummary of the state-of-the-art. We highlight the differences among various\nobjective functions which are rarely discussed in the existing literature.\nFinally, we propose several future directions pertaining to this new and\nexciting development of the field.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04867v1",
            "date": "2020-11-10 02:56:18+00:00",
            "title": "A Transfer Learning Approach for Dialogue Act Classification of GitHub Issue Comments",
            "authors": [
                "Ayesha Enayet",
                "Gita Sukthankar"
            ],
            "abstract": "Social coding platforms, such as GitHub, serve as laboratories for studying\ncollaborative problem solving in open source software development; a key\nfeature is their ability to support issue reporting which is used by teams to\ndiscuss tasks and ideas. Analyzing the dialogue between team members, as\nexpressed in issue comments, can yield important insights about the performance\nof virtual teams. This paper presents a transfer learning approach for\nperforming dialogue act classification on issue comments. Since no large\nlabeled corpus of GitHub issue comments exists, employing transfer learning\nenables us to leverage standard dialogue act datasets in combination with our\nown GitHub comment dataset. We compare the performance of several word and\nsentence level encoding models including Global Vectors for Word\nRepresentations (GloVe), Universal Sentence Encoder (USE), and Bidirectional\nEncoder Representations from Transformers (BERT). Being able to map the issue\ncomments to dialogue acts is a useful stepping stone towards understanding\ncognitive team processes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.04883v1",
            "date": "2020-11-10 04:11:44+00:00",
            "title": "Determining Question-Answer Plausibility in Crowdsourced Datasets Using Multi-Task Learning",
            "authors": [
                "Rachel Gardner",
                "Maya Varma",
                "Clare Zhu",
                "Ranjay Krishna"
            ],
            "abstract": "Datasets extracted from social networks and online forums are often prone to\nthe pitfalls of natural language, namely the presence of unstructured and noisy\ndata. In this work, we seek to enable the collection of high-quality\nquestion-answer datasets from social media by proposing a novel task for\nautomated quality analysis and data cleaning: question-answer (QA)\nplausibility. Given a machine or user-generated question and a crowd-sourced\nresponse from a social media user, we determine if the question and response\nare valid; if so, we identify the answer within the free-form response. We\ndesign BERT-based models to perform the QA plausibility task, and we evaluate\nthe ability of our models to generate a clean, usable question-answer dataset.\nOur highest-performing approach consists of a single-task model which\ndetermines the plausibility of the question, followed by a multi-task model\nwhich evaluates the plausibility of the response as well as extracts answers\n(Question Plausibility AUROC=0.75, Response Plausibility AUROC=0.78, Answer\nExtraction F1=0.665).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05007v1",
            "date": "2020-11-10 09:59:24+00:00",
            "title": "To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding?",
            "authors": [
                "Quynh Do",
                "Judith Gaspers",
                "Tobias Roding",
                "Melanie Bradford"
            ],
            "abstract": "This paper addresses the question as to what degree a BERT-based multilingual\nSpoken Language Understanding (SLU) model can transfer knowledge across\nlanguages. Through experiments we will show that, although it works\nsubstantially well even on distant language groups, there is still a gap to the\nideal multilingual performance. In addition, we propose a novel BERT-based\nadversarial model architecture to learn language-shared and language-specific\nrepresentations for multilingual SLU. Our experimental results prove that the\nproposed model is capable of narrowing the gap to the ideal multilingual\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05037v1",
            "date": "2020-11-10 10:58:38+00:00",
            "title": "Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformers",
            "authors": [
                "Ife Adebara",
                "El Moatez Billah Nagoudi",
                "Muhammad Abdul Mageed"
            ],
            "abstract": "We investigate different approaches to translate between similar languages\nunder low resource conditions, as part of our contribution to the WMT 2020\nSimilar Languages Translation Shared Task. We submitted Transformer-based\nbilingual and multilingual systems for all language pairs, in the two\ndirections. We also leverage back-translation for one of the language pairs,\nacquiring an improvement of more than 3 BLEU points. We interpret our results\nin light of the degree of mutual intelligibility (based on Jaccard similarity)\nbetween each pair, finding a positive correlation between mutual\nintelligibility and model performance. Our Spanish-Catalan model has the best\nperformance of all the five language pairs. Except for the case of\nHindi-Marathi, our bilingual models achieve better performance than the\nmultilingual models on all pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05188v2",
            "date": "2020-11-10 15:38:42+00:00",
            "title": "Biomedical Information Extraction for Disease Gene Prioritization",
            "authors": [
                "Jupinder Parmar",
                "William Koehler",
                "Martin Bringmann",
                "Katharina Sophia Volz",
                "Berk Kapicioglu"
            ],
            "abstract": "We introduce a biomedical information extraction (IE) pipeline that extracts\nbiological relationships from text and demonstrate that its components, such as\nnamed entity recognition (NER) and relation extraction (RE), outperform\nstate-of-the-art in BioNLP. We apply it to tens of millions of PubMed abstracts\nto extract protein-protein interactions (PPIs) and augment these extractions to\na biomedical knowledge graph that already contains PPIs extracted from STRING,\nthe leading structured PPI database. We show that, despite already containing\nPPIs from an established structured source, augmenting our own IE-based\nextractions to the graph allows us to predict novel disease-gene associations\nwith a 20% relative increase in hit@30, an important step towards developing\ndrug targets for uncured diseases.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.05197v1",
            "date": "2020-11-10 15:50:37+00:00",
            "title": "UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations",
            "authors": [
                "Gabriele Sarti"
            ],
            "abstract": "This work describes a self-supervised data augmentation approach used to\nimprove learning models' performances when only a moderate amount of labeled\ndata is available. Multiple copies of the original model are initially trained\non the downstream task. Their predictions are then used to annotate a large set\nof unlabeled examples. Finally, multi-task training is performed on the\nparallel annotations of the resulting training set, and final scores are\nobtained by averaging annotator-specific head predictions. Neural language\nmodels are fine-tuned using this procedure in the context of the AcCompl-it\nshared task at EVALITA 2020, obtaining considerable improvements in prediction\nquality.",
            "journal ref": "Proceedings of the Seventh Evaluation Campaign of Natural Language\n  Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05268v1",
            "date": "2020-10-24 02:05:56+00:00",
            "title": "Towards Interpretable Natural Language Understanding with Explanations as Latent Variables",
            "authors": [
                "Wangchunshu Zhou",
                "Jinyi Hu",
                "Hanlin Zhang",
                "Xiaodan Liang",
                "Maosong Sun",
                "Chenyan Xiong",
                "Jian Tang"
            ],
            "abstract": "Recently generating natural language explanations has shown very promising\nresults in not only offering interpretable explanations but also providing\nadditional information and supervision for prediction. However, existing\napproaches usually require a large set of human annotated explanations for\ntraining while collecting a large set of explanations is not only time\nconsuming but also expensive. In this paper, we develop a general framework for\ninterpretable natural language understanding that requires only a small set of\nhuman annotated explanations for training. Our framework treats natural\nlanguage explanations as latent variables that model the underlying reasoning\nprocess of a neural model. We develop a variational EM framework for\noptimization where an explanation generation module and an\nexplanation-augmented prediction module are alternatively optimized and\nmutually enhance each other. Moreover, we further propose an explanation-based\nself-training method under this framework for semi-supervised learning. It\nalternates between assigning pseudo-labels to unlabeled data and generating new\nexplanations to iteratively improve each other. Experiments on two natural\nlanguage understanding tasks demonstrate that our framework can not only make\neffective predictions in both supervised and semi-supervised settings, but also\ngenerate good natural language explanation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05431v1",
            "date": "2020-11-10 22:28:00+00:00",
            "title": "E.T.: Entity-Transformers. Coreference augmented Neural Language Model for richer mention representations via Entity-Transformer blocks",
            "authors": [
                "Nikolaos Stylianou",
                "Ioannis Vlahavas"
            ],
            "abstract": "In the last decade, the field of Neural Language Modelling has witnessed\nenormous changes, with the development of novel models through the use of\nTransformer architectures. However, even these models struggle to model long\nsequences due to memory constraints and increasing computational complexity.\nCoreference annotations over the training data can provide context far beyond\nthe modelling limitations of such language models. In this paper we present an\nextension over the Transformer-block architecture used in neural language\nmodels, specifically in GPT2, in order to incorporate entity annotations during\ntraining. Our model, GPT2E, extends the Transformer layers architecture of GPT2\nto Entity-Transformers, an architecture designed to handle coreference\ninformation when present. To that end, we achieve richer representations for\nentity mentions, with insignificant training cost. We show the comparative\nmodel performance between GPT2 and GPT2E in terms of Perplexity on the CoNLL\n2012 and LAMBADA datasets as well as the key differences in the entity\nrepresentations and their effects in downstream tasks such as Named Entity\nRecognition. Furthermore, our approach can be adopted by the majority of\nTransformer-based language models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05457v1",
            "date": "2020-11-10 23:28:00+00:00",
            "title": "Resource Constrained Dialog Policy Learning via Differentiable Inductive Logic Programming",
            "authors": [
                "Zhenpeng Zhou",
                "Ahmad Beirami",
                "Paul Crook",
                "Pararth Shah",
                "Rajen Subba",
                "Alborz Geramifard"
            ],
            "abstract": "Motivated by the needs of resource constrained dialog policy learning, we\nintroduce dialog policy via differentiable inductive logic (DILOG). We explore\nthe tasks of one-shot learning and zero-shot domain transfer with DILOG on\nSimDial and MultiWoZ. Using a single representative dialog from the restaurant\ndomain, we train DILOG on the SimDial dataset and obtain 99+% in-domain test\naccuracy. We also show that the trained DILOG zero-shot transfers to all other\ndomains with 99+% accuracy, proving the suitability of DILOG to slot-filling\ndialogs. We further extend our study to the MultiWoZ dataset achieving 90+%\ninform and success metrics. We also observe that these metrics are not\ncapturing some of the shortcomings of DILOG in terms of false positives,\nprompting us to measure an auxiliary Action F1 score. We show that DILOG is\n100x more data efficient than state-of-the-art neural approaches on MultiWoZ\nwhile achieving similar performance metrics. We conclude with a discussion on\nthe strengths and weaknesses of DILOG.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05604v1",
            "date": "2020-11-11 07:32:18+00:00",
            "title": "An Investigation of Potential Function Designs for Neural CRF",
            "authors": [
                "Zechuan Hu",
                "Yong Jiang",
                "Nguyen Bach",
                "Tao Wang",
                "Zhongqiang Huang",
                "Fei Huang",
                "Kewei Tu"
            ],
            "abstract": "The neural linear-chain CRF model is one of the most widely-used approach to\nsequence labeling. In this paper, we investigate a series of increasingly\nexpressive potential functions for neural CRF models, which not only integrate\nthe emission and transition functions, but also explicitly take the\nrepresentations of the contextual words as input. Our extensive experiments\nshow that the decomposed quadrilinear potential function based on the vector\nrepresentations of two neighboring labels and two neighboring words\nconsistently achieves the best performance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05675v2",
            "date": "2020-11-11 10:07:14+00:00",
            "title": "Rule-Based Approach for Party-Based Sentiment Analysis in Legal Opinion Texts",
            "authors": [
                "Isanka Rajapaksha",
                "Chanika Ruchini Mudalige",
                "Dilini Karunarathna",
                "Nisansa de Silva",
                "Gathika Ratnayaka",
                "Amal Shehan Perera"
            ],
            "abstract": "A document which elaborates opinions and arguments related to the previous\ncourt cases is known as a legal opinion text. Lawyers and legal officials have\nto spend considerable effort and time to obtain the required information\nmanually from those documents when dealing with new legal cases. Hence, it\nprovides much convenience to those individuals if there is a way to automate\nthe process of extracting information from legal opinion texts. Party-based\nsentiment analysis will play a key role in the automation system by identifying\nopinion values with respect to each legal parties in legal texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05723v1",
            "date": "2020-11-11 11:59:49+00:00",
            "title": "CalibreNet: Calibration Networks for Multilingual Sequence Labeling",
            "authors": [
                "Shining Liang",
                "Linjun Shou",
                "Jian Pei",
                "Ming Gong",
                "Wanli Zuo",
                "Daxin Jiang"
            ],
            "abstract": "Lack of training data in low-resource languages presents huge challenges to\nsequence labeling tasks such as named entity recognition (NER) and machine\nreading comprehension (MRC). One major obstacle is the errors on the boundary\nof predicted answers. To tackle this problem, we propose CalibreNet, which\npredicts answers in two steps. In the first step, any existing sequence\nlabeling method can be adopted as a base model to generate an initial answer.\nIn the second step, CalibreNet refines the boundary of the initial answer. To\ntackle the challenge of lack of training data in low-resource languages, we\ndedicatedly develop a novel unsupervised phrase boundary recovery pre-training\ntask to enhance the multilingual boundary detection capability of CalibreNet.\nExperiments on two cross-lingual benchmark datasets show that the proposed\napproach achieves SOTA results on zero-shot cross-lingual NER and MRC tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05864v1",
            "date": "2020-11-02 13:14:57+00:00",
            "title": "On the Sentence Embeddings from Pre-trained Language Models",
            "authors": [
                "Bohan Li",
                "Hao Zhou",
                "Junxian He",
                "Mingxuan Wang",
                "Yiming Yang",
                "Lei Li"
            ],
            "abstract": "Pre-trained contextual representations like BERT have achieved great success\nin natural language processing. However, the sentence embeddings from the\npre-trained language models without fine-tuning have been found to poorly\ncapture semantic meaning of sentences. In this paper, we argue that the\nsemantic information in the BERT embeddings is not fully exploited. We first\nreveal the theoretical connection between the masked language model\npre-training objective and the semantic similarity task theoretically, and then\nanalyze the BERT sentence embeddings empirically. We find that BERT always\ninduces a non-smooth anisotropic semantic space of sentences, which harms its\nperformance of semantic similarity. To address this issue, we propose to\ntransform the anisotropic sentence embedding distribution to a smooth and\nisotropic Gaussian distribution through normalizing flows that are learned with\nan unsupervised objective. Experimental results show that our proposed\nBERT-flow method obtains significant performance gains over the\nstate-of-the-art sentence embeddings on a variety of semantic textual\nsimilarity tasks. The code is available at\nhttps://github.com/bohanli/BERT-flow.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05910v1",
            "date": "2020-11-11 17:02:01+00:00",
            "title": "Audrey: A Personalized Open-Domain Conversational Bot",
            "authors": [
                "Chung Hoon Hong",
                "Yuan Liang",
                "Sagnik Sinha Roy",
                "Arushi Jain",
                "Vihang Agarwal",
                "Ryan Draves",
                "Zhizhuo Zhou",
                "William Chen",
                "Yujian Liu",
                "Martha Miracky",
                "Lily Ge",
                "Nikola Banovic",
                "David Jurgens"
            ],
            "abstract": "Conversational Intelligence requires that a person engage on informational,\npersonal and relational levels. Advances in Natural Language Understanding have\nhelped recent chatbots succeed at dialog on the informational level. However,\ncurrent techniques still lag for conversing with humans on a personal level and\nfully relating to them. The University of Michigan's submission to the Alexa\nPrize Grand Challenge 3, Audrey, is an open-domain conversational chat-bot that\naims to engage customers on these levels through interest driven conversations\nguided by customers' personalities and emotions. Audrey is built from\nsocially-aware models such as Emotion Detection and a Personal Understanding\nModule to grasp a deeper understanding of users' interests and desires. Our\narchitecture interacts with customers using a hybrid approach balanced between\nknowledge-driven response generators and context-driven neural response\ngenerators to cater to all three levels of conversations. During the\nsemi-finals period, we achieved an average cumulative rating of 3.25 on a 1-5\nLikert scale.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.05958v1",
            "date": "2020-11-11 18:25:07+00:00",
            "title": "On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments",
            "authors": [
                "Jisi Zhang",
                "Catalin Zorila",
                "Rama Doddipatla",
                "Jon Barker"
            ],
            "abstract": "This paper introduces a new method for multi-channel time domain speech\nseparation in reverberant environments. A fully-convolutional neural network\nstructure has been used to directly separate speech from multiple microphone\nrecordings, with no need of conventional spatial feature extraction. To reduce\nthe influence of reverberation on spatial feature extraction, a dereverberation\npre-processing method has been applied to further improve the separation\nperformance. A spatialized version of wsj0-2mix dataset has been simulated to\nevaluate the proposed system. Both source separation and speech recognition\nperformance of the separated signals have been evaluated objectively.\nExperiments show that the proposed fully-convolutional network improves the\nsource separation metric and the word error rate (WER) by more than 13% and 50%\nrelative, respectively, over a reference system with conventional features.\nApplying dereverberation as pre-processing to the proposed system can further\nreduce the WER by 29% relative using an acoustic model trained on clean and\nreverberated data.",
            "journal ref": "Proc. ICASSP (2020) 6389-6393",
            "category": "eess.AS"
        },
        {
            "url": "http://arxiv.org/abs/2011.05978v1",
            "date": "2020-11-11 18:50:18+00:00",
            "title": "The Impact of Text Presentation on Translator Performance",
            "authors": [
                "Samuel L\u00e4ubli",
                "Patrick Simianer",
                "Joern Wuebker",
                "Geza Kovacs",
                "Rico Sennrich",
                "Spence Green"
            ],
            "abstract": "Widely used computer-aided translation (CAT) tools divide documents into\nsegments such as sentences and arrange them in a side-by-side, spreadsheet-like\nview. We present the first controlled evaluation of these design choices on\ntranslator performance, measuring speed and accuracy in three experimental text\nprocessing tasks. We find significant evidence that sentence-by-sentence\npresentation enables faster text reproduction and within-sentence error\nidentification compared to unsegmented text, and that a top-and-bottom\narrangement of source and target sentences enables faster text reproduction\ncompared to a side-by-side arrangement. For revision, on the other hand, our\nresults suggest that presenting unsegmented text results in the highest\naccuracy and time efficiency. Our findings have direct implications for best\npractices in designing CAT tools.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06128v1",
            "date": "2020-11-12 00:03:26+00:00",
            "title": "Author's Sentiment Prediction",
            "authors": [
                "Mohaddeseh Bastan",
                "Mahnaz Koupaee",
                "Youngseo Son",
                "Richard Sicoli",
                "Niranjan Balasubramanian"
            ],
            "abstract": "We introduce PerSenT, a dataset of crowd-sourced annotations of the sentiment\nexpressed by the authors towards the main entities in news articles. The\ndataset also includes paragraph-level sentiment annotations to provide more\nfine-grained supervision for the task. Our benchmarks of multiple strong\nbaselines show that this is a difficult classification task. The results also\nsuggest that simply fine-tuning document-level representations from BERT isn't\nadequate for this task. Making paragraph-level decisions and aggregating them\nover the entire document is also ineffective. We present empirical and\nqualitative analyses that illustrate the specific challenges posed by this\ndataset. We release this dataset with 5.3k documents and 38k paragraphs\ncovering 3.2k unique entities as a challenge in entity sentiment analysis.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06153v1",
            "date": "2020-11-12 01:32:41+00:00",
            "title": "Augmenting BERT Carefully with Underrepresented Linguistic Features",
            "authors": [
                "Aparna Balagopalan",
                "Jekaterina Novikova"
            ],
            "abstract": "Fine-tuned Bidirectional Encoder Representations from Transformers\n(BERT)-based sequence classification models have proven to be effective for\ndetecting Alzheimer's Disease (AD) from transcripts of human speech. However,\nprevious research shows it is possible to improve BERT's performance on various\ntasks by augmenting the model with additional information. In this work, we use\nprobing tasks as introspection techniques to identify linguistic information\nnot well-represented in various layers of BERT, but important for the AD\ndetection task. We supplement these linguistic features in which\nrepresentations from BERT are found to be insufficient with hand-crafted\nfeatures externally, and show that jointly fine-tuning BERT in combination with\nthese features improves the performance of AD classification by upto 5\\% over\nfine-tuned BERT alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06326v1",
            "date": "2020-11-12 11:45:47+00:00",
            "title": "SigmaLaw-ABSA: Dataset for Aspect-Based Sentiment Analysis in Legal Opinion Texts",
            "authors": [
                "Chanika Ruchini Mudalige",
                "Dilini Karunarathna",
                "Isanka Rajapaksha",
                "Nisansa de Silva",
                "Gathika Ratnayaka",
                "Amal Shehan Perera",
                "Ramesh Pathirana"
            ],
            "abstract": "Aspect-Based Sentiment Analysis (ABSA) has been prominent and ongoing\nresearch over many different domains, but it is not widely discussed in the\nlegal domain. A number of publicly available datasets for a wide range of\ndomains usually fulfill the needs of researchers to perform their studies in\nthe field of ABSA. To the best of our knowledge, there is no publicly available\ndataset for the Aspect (Party) Based Sentiment Analysis for legal opinion\ntexts. Therefore, creating a publicly available dataset for the research of\nABSA for the legal domain can be considered as a task with significant\nimportance. In this study, we introduce a manually annotated legal opinion text\ndataset (SigmaLaw-ABSA) intended towards facilitating researchers for ABSA\ntasks in the legal domain. SigmaLaw-ABSA consists of legal opinion texts in the\nEnglish language which have been annotated by human judges. This study\ndiscusses the sub-tasks of ABSA relevant to the legal domain and how to use the\ndataset to perform them. This paper also describes the statistics of the\ndataset and as a baseline, we present some results on the performance of some\nexisting deep learning based systems on the SigmaLaw-ABSA dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06382v1",
            "date": "2020-11-12 13:50:00+00:00",
            "title": "Towards A Sentiment Analyzer for Low-Resource Languages",
            "authors": [
                "Dian Indriani",
                "Arbi Haza Nasution",
                "Winda Monika",
                "Salhazan Nasution"
            ],
            "abstract": "Twitter is one of the top influenced social media which has a million number\nof active users. It is commonly used for microblogging that allows users to\nshare messages, ideas, thoughts and many more. Thus, millions interaction such\nas short messages or tweets are flowing around among the twitter users\ndiscussing various topics that has been happening world-wide. This research\naims to analyse a sentiment of the users towards a particular trending topic\nthat has been actively and massively discussed at that time. We chose a hashtag\n\\textit{\\#kpujangancurang} that was the trending topic during the Indonesia\npresidential election in 2019. We use the hashtag to obtain a set of data from\nTwitter to analyse and investigate further the positive or the negative\nsentiment of the users from their tweets. This research utilizes rapid miner\ntool to generate the twitter data and comparing Naive Bayes, K-Nearest\nNeighbor, Decision Tree, and Multi-Layer Perceptron classification methods to\nclassify the sentiment of the twitter data. There are overall 200 labeled data\nin this experiment. Overall, Naive Bayes and Multi-Layer Perceptron\nclassification outperformed the other two methods on 11 experiments with\ndifferent size of training-testing data split. The two classifiers are\npotential to be used in creating sentiment analyzer for low-resource languages\nwith small corpus.",
            "journal ref": "Proceedings of International Conference on Smart Computing and\n  Cyber Security: Strategic Foresight, Security Challenges and Innovation\n  (SMARTCYBER 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06395v1",
            "date": "2020-10-25 19:36:23+00:00",
            "title": "Turn-level Dialog Evaluation with Dialog-level Weak Signals for Bot-Human Hybrid Customer Service Systems",
            "authors": [
                "Ruofeng Wen"
            ],
            "abstract": "We developed a machine learning approach that quantifies multiple aspects of\nthe success or values in Customer Service contacts, at anytime during the\ninteraction. Specifically, the value/reward function regarding to the\nturn-level behaviors across human agents, chatbots and other hybrid dialog\nsystems is characterized by the incremental information and confidence gain\nbetween sentences, based on the token-level predictions from a multi-task\nneural network trained with only weak signals in dialog-level\nattributes/states. The resulting model, named Value Profiler, serves as a\ngoal-oriented dialog manager that enhances conversations by regulating\nautomated decisions with its reward and state predictions. It supports both\nreal-time monitoring and scalable offline customer experience evaluation, for\nboth bot- and human-handled contacts. We show how it improves Amazon customer\nservice quality in several applications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06504v1",
            "date": "2020-11-12 17:14:32+00:00",
            "title": "An Interpretable End-to-end Fine-tuning Approach for Long Clinical Text",
            "authors": [
                "Kexin Huang",
                "Sankeerth Garapati",
                "Alexander S. Rich"
            ],
            "abstract": "Unstructured clinical text in EHRs contains crucial information for\napplications including decision support, trial matching, and retrospective\nresearch. Recent work has applied BERT-based models to clinical information\nextraction and text classification, given these models' state-of-the-art\nperformance in other NLP domains. However, BERT is difficult to apply to\nclinical notes because it doesn't scale well to long sequences of text. In this\nwork, we propose a novel fine-tuning approach called SnipBERT. Instead of using\nentire notes, SnipBERT identifies crucial snippets and then feeds them into a\ntruncated BERT-based model in a hierarchical manner. Empirically, SnipBERT not\nonly has significant predictive performance gain across three tasks but also\nprovides improved interpretability, as the model can identify key pieces of\ntext that led to its prediction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06727v1",
            "date": "2020-11-13 02:29:50+00:00",
            "title": "A Survey on Recent Advances in Sequence Labeling from Deep Learning Models",
            "authors": [
                "Zhiyong He",
                "Zanbo Wang",
                "Wei Wei",
                "Shanshan Feng",
                "Xianling Mao",
                "Sheng Jiang"
            ],
            "abstract": "Sequence labeling (SL) is a fundamental research problem encompassing a\nvariety of tasks, e.g., part-of-speech (POS) tagging, named entity recognition\n(NER), text chunking, etc. Though prevalent and effective in many downstream\napplications (e.g., information retrieval, question answering, and knowledge\ngraph embedding), conventional sequence labeling approaches heavily rely on\nhand-crafted or language-specific features. Recently, deep learning has been\nemployed for sequence labeling tasks due to its powerful capability in\nautomatically learning complex features of instances and effectively yielding\nthe stat-of-the-art performances. In this paper, we aim to present a\ncomprehensive review of existing deep learning-based sequence labeling models,\nwhich consists of three related tasks, e.g., part-of-speech tagging, named\nentity recognition, and text chunking. Then, we systematically present the\nexisting approaches base on a scientific taxonomy, as well as the widely-used\nexperimental datasets and popularly-adopted evaluation metrics in the SL\ndomain. Furthermore, we also present an in-depth analysis of different SL\nmodels on the factors that may affect the performance and future directions in\nthe SL domain.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06819v1",
            "date": "2020-11-13 09:19:48+00:00",
            "title": "diagNNose: A Library for Neural Activation Analysis",
            "authors": [
                "Jaap Jumelet"
            ],
            "abstract": "In this paper we introduce diagNNose, an open source library for analysing\nthe activations of deep neural networks. diagNNose contains a wide array of\ninterpretability techniques that provide fundamental insights into the inner\nworkings of neural networks. We demonstrate the functionality of diagNNose with\na case study on subject-verb agreement within language models. diagNNose is\navailable at https://github.com/i-machine-think/diagnnose.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06868v2",
            "date": "2020-11-13 11:47:28+00:00",
            "title": "EDITOR: an Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints",
            "authors": [
                "Weijia Xu",
                "Marine Carpuat"
            ],
            "abstract": "We introduce an Edit-Based Transformer with Repositioning (EDITOR), which\nmakes sequence generation flexible by seamlessly allowing users to specify\npreferences in output lexical choice. Building on recent models for\nnon-autoregressive sequence generation (Gu et al., 2019), EDITOR generates new\nsequences by iteratively editing hypotheses. It relies on a novel reposition\noperation designed to disentangle lexical choice from word positioning\ndecisions, while enabling efficient oracles for imitation learning and parallel\nedits at decoding time. Empirically, EDITOR uses soft lexical constraints more\neffectively than the Levenshtein Transformer (Gu et al., 2019) while speeding\nup decoding dramatically compared to constrained beam search (Post and Vilar,\n2018). EDITOR also achieves comparable or better translation quality with\nfaster decoding speed than the Levenshtein Transformer on standard\nRomanian-English, English-German, and English-Japanese machine translation\ntasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06874v2",
            "date": "2020-11-12 05:26:56+00:00",
            "title": "Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions",
            "authors": [
                "Ali Mottaghi",
                "Prathusha K Sarma",
                "Xavier Amatriain",
                "Serena Yeung",
                "Anitha Kannan"
            ],
            "abstract": "We study the problem of medical symptoms recognition from patient text, for\nthe purposes of gathering pertinent information from the patient (known as\nhistory-taking). A typical patient text is often descriptive of the symptoms\nthe patient is experiencing and a single instance of such a text can be\n\"labeled\" with multiple symptoms. This makes learning a medical symptoms\nrecognizer challenging on account of i) the lack of availability of voluminous\nannotated data as well as ii) the large unknown universe of multiple symptoms\nthat a single text can map to. Furthermore, patient text is often characterized\nby a long tail in the data (i.e., some labels/symptoms occur more frequently\nthan others for e.g \"fever\" vs \"hematochezia\"). In this paper, we introduce an\nactive learning method that leverages underlying structure of a continually\nrefined, learned latent space to select the most informative examples to label.\nThis enables the selection of the most informative examples that progressively\nincreases the coverage on the universe of symptoms via the learned model,\ndespite the long tail in data distribution.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06949v1",
            "date": "2020-11-13 14:50:08+00:00",
            "title": "Learning language variations in news corpora through differential embeddings",
            "authors": [
                "Carlos Selmo",
                "Julian F. Martinez",
                "Mariano G. Beir\u00f3",
                "J. Ignacio Alvarez-Hamelin"
            ],
            "abstract": "There is an increasing interest in the NLP community in capturing variations\nin the usage of language, either through time (i.e., semantic drift), across\nregions (as dialects or variants) or in different social contexts (i.e.,\nprofessional or media technolects). Several successful dynamical embeddings\nhave been proposed that can track semantic change through time. Here we show\nthat a model with a central word representation and a slice-dependent\ncontribution can learn word embeddings from different corpora simultaneously.\nThis model is based on a star-like representation of the slices. We apply it to\nThe New York Times and The Guardian newspapers, and we show that it can capture\nboth temporal dynamics in the yearly slices of each corpus, and language\nvariations between US and UK English in a curated multi-source corpus. We\nprovide an extensive evaluation of this methodology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.06977v1",
            "date": "2020-11-13 15:52:51+00:00",
            "title": "Arabic Dialect Identification Using BERT-Based Domain Adaptation",
            "authors": [
                "Ahmad Beltagy",
                "Abdelrahman Wael",
                "Omar ElSherief"
            ],
            "abstract": "Arabic is one of the most important and growing languages in the world. With\nthe rise of social media platforms such as Twitter, Arabic spoken dialects have\nbecome more in use. In this paper, we describe our approach on the NADI Shared\nTask 1 that requires us to build a system to differentiate between different 21\nArabic dialects, we introduce a deep learning semi-supervised fashion approach\nalong with pre-processing that was reported on NADI shared Task 1 Corpus. Our\nsystem ranks 4th in NADI's shared task competition achieving a 23.09% F1 macro\naverage score with a simple yet efficient approach to differentiating between\n21 Arabic Dialects given tweets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07013v1",
            "date": "2020-11-12 14:23:35+00:00",
            "title": "Deconstructing word embedding algorithms",
            "authors": [
                "Kian Kenyon-Dean",
                "Edward Newell",
                "Jackie Chi Kit Cheung"
            ],
            "abstract": "Word embeddings are reliable feature representations of words used to obtain\nhigh quality results for various NLP applications. Uncontextualized word\nembeddings are used in many NLP tasks today, especially in resource-limited\nsettings where high memory capacity and GPUs are not available. Given the\nhistorical success of word embeddings in NLP, we propose a retrospective on\nsome of the most well-known word embedding algorithms. In this work, we\ndeconstruct Word2vec, GloVe, and others, into a common form, unveiling some of\nthe common conditions that seem to be required for making performant word\nembeddings. We believe that the theoretical findings in this paper can provide\na basis for more informed development of future models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07126v1",
            "date": "2020-11-13 20:57:53+00:00",
            "title": "Zero-shot Learning for Relation Extraction",
            "authors": [
                "Jiaying Gong",
                "Hoda Eldardiry"
            ],
            "abstract": "Most existing supervised and few-shot learning relation extraction methods\nhave relied on labeled training data. However, in real-world scenarios, there\nexist many relations for which there is no available training data. We address\nthis issue from the perspective of zero-shot learning (ZSL) which is similar to\nthe way humans learn and recognize new concepts with no prior knowledge. We\npropose a zero-shot learning relation extraction (ZSLRE) framework, which\nfocuses on recognizing novel relations that have no corresponding labeled data\navailable for training. Our proposed ZSLRE model aims to recognize new\nrelations based on prototypical networks that are modified to utilize side\n(auxiliary) information. The additional use of side information allows those\nmodified prototype networks to recognize novel relations in addition to\nrecognized previously known relations. We construct side information from\nlabels and their synonyms, hypernyms of name entities, and keywords. We build\nan automatic hypernym extraction framework to help get hypernyms of various\nname entities directly from the web. We demonstrate using extensive experiments\non two public datasets (NYT and FewRel) that our proposed model significantly\noutperforms state-of-the-art methods on supervised learning, few-shot learning,\nand zero-shot learning tasks. Our experimental results also demonstrate the\neffectiveness and robustness of our proposed model in a combination scenario.\nOnce accepted for publication, we will publish ZSLRE's source code and datasets\nto enable reproducibility and encourage further research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07208v1",
            "date": "2020-11-14 03:15:26+00:00",
            "title": "Utilizing Bidirectional Encoder Representations from Transformers for Answer Selection",
            "authors": [
                "Md Tahmid Rahman Laskar",
                "Enamul Hoque",
                "Jimmy Xiangji Huang"
            ],
            "abstract": "Pre-training a transformer-based model for the language modeling task in a\nlarge dataset and then fine-tuning it for downstream tasks has been found very\nuseful in recent years. One major advantage of such pre-trained language models\nis that they can effectively absorb the context of each word in a sentence.\nHowever, for tasks such as the answer selection task, the pre-trained language\nmodels have not been extensively used yet. To investigate their effectiveness\nin such tasks, in this paper, we adopt the pre-trained Bidirectional Encoder\nRepresentations from Transformer (BERT) language model and fine-tune it on two\nQuestion Answering (QA) datasets and three Community Question Answering (CQA)\ndatasets for the answer selection task. We find that fine-tuning the BERT model\nfor the answer selection task is very effective and observe a maximum\nimprovement of 13.1% in the QA datasets and 18.7% in the CQA datasets compared\nto the previous state-of-the-art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07280v1",
            "date": "2020-11-14 12:02:30+00:00",
            "title": "Sentiment Analysis for Sinhala Language using Deep Learning Techniques",
            "authors": [
                "Lahiru Senevirathne",
                "Piyumal Demotte",
                "Binod Karunanayake",
                "Udyogi Munasinghe",
                "Surangika Ranathunga"
            ],
            "abstract": "Due to the high impact of the fast-evolving fields of machine learning and\ndeep learning, Natural Language Processing (NLP) tasks have further obtained\ncomprehensive performances for highly resourced languages such as English and\nChinese. However Sinhala, which is an under-resourced language with a rich\nmorphology, has not experienced these advancements. For sentiment analysis,\nthere exists only two previous research with deep learning approaches, which\nfocused only on document-level sentiment analysis for the binary case. They\nexperimented with only three types of deep learning models. In contrast, this\npaper presents a much comprehensive study on the use of standard sequence\nmodels such as RNN, LSTM, Bi-LSTM, as well as more recent state-of-the-art\nmodels such as hierarchical attention hybrid neural networks, and capsule\nnetworks. Classification is done at document-level but with more granularity by\nconsidering POSITIVE, NEGATIVE, NEUTRAL, and CONFLICT classes. A data set of\n15059 Sinhala news comments, annotated with these four classes and a corpus\nconsists of 9.48 million tokens are publicly released. This is the largest\nsentiment annotated data set for Sinhala so far.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07347v1",
            "date": "2020-11-14 17:45:11+00:00",
            "title": "Conditioned Natural Language Generation using only Unconditioned Language Model: An Exploration",
            "authors": [
                "Fan-Keng Sun",
                "Cheng-I Lai"
            ],
            "abstract": "Transformer-based language models have shown to be very powerful for natural\nlanguage generation (NLG). However, text generation conditioned on some user\ninputs, such as topics or attributes, is non-trivial. Past approach relies on\neither modifying the original LM architecture, re-training the LM on corpora\nwith attribute labels, or having separately trained `guidance models' to guide\ntext generation in decoding. We argued that the above approaches are not\nnecessary, and the original unconditioned LM is sufficient for conditioned NLG.\nWe evaluated our approaches by the samples' fluency and diversity with\nautomated and human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07403v2",
            "date": "2020-11-14 22:18:45+00:00",
            "title": "A Hybrid Approach for Improved Low Resource Neural Machine Translation using Monolingual Data",
            "authors": [
                "Idris Abdulmumin",
                "Bashir Shehu Galadanci",
                "Abubakar Isa",
                "Ismaila Idris Sinan"
            ],
            "abstract": "Many language pairs are low resource, meaning the amount and/or quality of\navailable parallel data is not sufficient to train a neural machine translation\n(NMT) model which can reach an acceptable standard of accuracy. Many works have\nexplored using the readily available monolingual data in either or both of the\nlanguages to improve the standard of translation models in low, and even high,\nresource languages. One of the most successful of such works is the\nback-translation that utilizes the translations of the target language\nmonolingual data to increase the amount of the training data. The quality of\nthe backward model which is trained on the available parallel data has been\nshown to determine the performance of the back-translation approach. Despite\nthis, only the forward model is improved on the monolingual target data in\nstandard back-translation. A previous study proposed an iterative\nback-translation approach for improving both models over several iterations.\nBut unlike in the traditional back-translation, it relied on both the target\nand source monolingual data. This work, therefore, proposes a novel approach\nthat enables both the backward and forward models to benefit from the\nmonolingual target data through a hybrid of self-learning and back-translation\nrespectively. Experimental results have shown the superiority of the proposed\napproach over the traditional back-translation method on English-German low\nresource neural machine translation. We also proposed an iterative\nself-learning approach that outperforms the iterative back-translation while\nalso relying only on the monolingual target data and require the training of\nless models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07432v2",
            "date": "2020-11-15 01:55:37+00:00",
            "title": "Target Guided Emotion Aware Chat Machine",
            "authors": [
                "Wei Wei",
                "Jiayi Liu",
                "Xianling Mao",
                "Guibin Guo",
                "Feida Zhu",
                "Pan Zhou",
                "Yuchong Hu",
                "Shanshan Feng"
            ],
            "abstract": "The consistency of a response to a given post at semantic-level and\nemotional-level is essential for a dialogue system to deliver human-like\ninteractions. However, this challenge is not well addressed in the literature,\nsince most of the approaches neglect the emotional information conveyed by a\npost while generating responses. This article addresses this problem by\nproposing a unifed end-to-end neural architecture, which is capable of\nsimultaneously encoding the semantics and the emotions in a post and leverage\ntarget information for generating more intelligent responses with appropriately\nexpressed emotions. Extensive experiments on real-world data demonstrate that\nthe proposed method outperforms the state-of-the-art methods in terms of both\ncontent coherence and emotion appropriateness.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07605v1",
            "date": "2020-11-15 19:02:46+00:00",
            "title": "The Challenge of Diacritics in Yoruba Embeddings",
            "authors": [
                "Tosin P. Adewumi",
                "Foteini Liwicki",
                "Marcus Liwicki"
            ],
            "abstract": "The major contributions of this work include the empirical establishment of a\nbetter performance for Yoruba embeddings from undiacritized (normalized)\ndataset and provision of new analogy sets for evaluation. The Yoruba language,\nbeing a tonal language, utilizes diacritics (tonal marks) in written form. We\nshow that this affects embedding performance by creating embeddings from\nexactly the same Wikipedia dataset but with the second one normalized to be\nundiacritized. We further compare average intrinsic performance with two other\nwork (using analogy test set & WordSim) and we obtain the best performance in\nWordSim and corresponding Spearman correlation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07658v1",
            "date": "2020-11-15 23:27:43+00:00",
            "title": "Deep multi-modal networks for book genre classification based on its cover",
            "authors": [
                "Chandra Kundu",
                "Lukun Zheng"
            ],
            "abstract": "Book covers are usually the very first impression to its readers and they\noften convey important information about the content of the book. Book genre\nclassification based on its cover would be utterly beneficial to many modern\nretrieval systems, considering that the complete digitization of books is an\nextremely expensive task. At the same time, it is also an extremely challenging\ntask due to the following reasons: First, there exists a wide variety of book\ngenres, many of which are not concretely defined. Second, book covers, as\ngraphic designs, vary in many different ways such as colors, styles, textual\ninformation, etc, even for books of the same genre. Third, book cover designs\nmay vary due to many external factors such as country, culture, target reader\npopulations, etc. With the growing competitiveness in the book industry, the\nbook cover designers and typographers push the cover designs to its limit in\nthe hope of attracting sales. The cover-based book classification systems\nbecome a particularly exciting research topic in recent years. In this paper,\nwe propose a multi-modal deep learning framework to solve this problem. The\ncontribution of this paper is four-fold. First, our method adds an extra\nmodality by extracting texts automatically from the book covers. Second,\nimage-based and text-based, state-of-the-art models are evaluated thoroughly\nfor the task of book cover classification. Third, we develop an efficient and\nsalable multi-modal framework based on the images and texts shown on the covers\nonly. Fourth, a thorough analysis of the experimental results is given and\nfuture works to improve the performance is suggested. The results show that the\nmulti-modal framework significantly outperforms the current state-of-the-art\nimage-based models. However, more efforts and resources are needed for this\nclassification task in order to reach a satisfactory level.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.07754v1",
            "date": "2020-11-16 07:13:58+00:00",
            "title": "Deep Shallow Fusion for RNN-T Personalization",
            "authors": [
                "Duc Le",
                "Gil Keren",
                "Julian Chan",
                "Jay Mahadeokar",
                "Christian Fuegen",
                "Michael L. Seltzer"
            ],
            "abstract": "End-to-end models in general, and Recurrent Neural Network Transducer (RNN-T)\nin particular, have gained significant traction in the automatic speech\nrecognition community in the last few years due to their simplicity,\ncompactness, and excellent performance on generic transcription tasks. However,\nthese models are more challenging to personalize compared to traditional hybrid\nsystems due to the lack of external language models and difficulties in\nrecognizing rare long-tail words, specifically entity names. In this work, we\npresent novel techniques to improve RNN-T's ability to model rare WordPieces,\ninfuse extra information into the encoder, enable the use of alternative\ngraphemic pronunciations, and perform deep fusion with personalized language\nmodels for more robust biasing. We show that these combined techniques result\nin 15.4%-34.5% relative Word Error Rate improvement compared to a strong RNN-T\nbaseline which uses shallow fusion and text-to-speech augmentation. Our work\nhelps push the boundary of RNN-T personalization and close the gap with hybrid\nsystems on use cases where biasing and entity recognition are crucial.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07916v1",
            "date": "2020-11-16 13:08:48+00:00",
            "title": "Text Information Aggregation with Centrality Attention",
            "authors": [
                "Jingjing Gong",
                "Hang Yan",
                "Yining Zheng",
                "Xipeng Qiu",
                "Xuanjing Huang"
            ],
            "abstract": "A lot of natural language processing problems need to encode the text\nsequence as a fix-length vector, which usually involves aggregation process of\ncombining the representations of all the words, such as pooling or\nself-attention. However, these widely used aggregation approaches did not take\nhigher-order relationship among the words into consideration. Hence we propose\na new way of obtaining aggregation weights, called eigen-centrality\nself-attention. More specifically, we build a fully-connected graph for all the\nwords in a sentence, then compute the eigen-centrality as the attention score\nof each word.\n  The explicit modeling of relationships as a graph is able to capture some\nhigher-order dependency among words, which helps us achieve better results in 5\ntext classification tasks and one SNLI task than baseline models such as\npooling, self-attention and dynamic routing. Besides, in order to compute the\ndominant eigenvector of the graph, we adopt power method algorithm to get the\neigen-centrality measure. Moreover, we also derive an iterative approach to get\nthe gradient for the power method process to reduce both memory consumption and\ncomputation requirement.}",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07960v2",
            "date": "2020-10-21 17:39:15+00:00",
            "title": "Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle",
            "authors": [
                "Yikang Shen",
                "Shawn Tan",
                "Alessandro Sordoni",
                "Siva Reddy",
                "Aaron Courville"
            ],
            "abstract": "Syntax is fundamental to our thinking about language. Failing to capture the\nstructure of input language could lead to generalization problems and\nover-parametrization. In the present work, we propose a new syntax-aware\nlanguage model: Syntactic Ordered Memory (SOM). The model explicitly models the\nstructure with an incremental parser and maintains the conditional probability\nsetting of a standard language model (left-to-right). To train the incremental\nparser and avoid exposure bias, we also propose a novel dynamic oracle, so that\nSOM is more robust to wrong parsing decisions. Experiments show that SOM can\nachieve strong results in language modeling, incremental parsing and syntactic\ngeneralization tests, while using fewer parameters than other models.",
            "journal ref": "NAACL 2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07961v1",
            "date": "2020-10-20 23:12:23+00:00",
            "title": "An Empirical Investigation of Contextualized Number Prediction",
            "authors": [
                "Daniel Spokoyny",
                "Taylor Berg-Kirkpatrick"
            ],
            "abstract": "We conduct a large scale empirical investigation of contextualized number\nprediction in running text. Specifically, we consider two tasks: (1)masked\nnumber prediction-predicting a missing numerical value within a sentence, and\n(2)numerical anomaly detection-detecting an errorful numeric value within a\nsentence. We experiment with novel combinations of contextual encoders and\noutput distributions over the real number line. Specifically, we introduce a\nsuite of output distribution parameterizations that incorporate latent\nvariables to add expressivity and better fit the natural distribution of\nnumeric values in running text, and combine them with both recurrent and\ntransformer-based encoder architectures. We evaluate these models on two\nnumeric datasets in the financial and scientific domain. Our findings show that\noutput distributions that incorporate discrete latent variables and allow for\nmultiple modes outperform simple flow-based counterparts on all datasets,\nyielding more accurate numerical prediction and anomaly detection. We also show\nthat our models effectively utilize textual con-text and benefit from\ngeneral-purpose unsupervised pretraining.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.07962v1",
            "date": "2020-10-20 08:12:56+00:00",
            "title": "Performance of Transfer Learning Model vs. Traditional Neural Network in Low System Resource Environment",
            "authors": [
                "William Hui"
            ],
            "abstract": "Recently, the use of pre-trained model to build neural network based on\ntransfer learning methodology is increasingly popular. These pre-trained models\npresent the benefit of using less computing resources to train model with\nsmaller amount of training data. The rise of state-of-the-art models such as\nBERT, XLNet and GPT boost accuracy and benefit as a base model for transfer\nleanring. However, these models are still too complex and consume many\ncomputing resource to train for transfer learning with low GPU memory. We will\ncompare the performance and cost between lighter transfer learning model and\npurposely built neural network for NLP application of text classification and\nNER model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08035v1",
            "date": "2020-11-16 15:41:45+00:00",
            "title": "\"What is on your mind?\" Automated Scoring of Mindreading in Childhood and Early Adolescence",
            "authors": [
                "Venelin Kovatchev",
                "Phillip Smith",
                "Mark Lee",
                "Imogen Grumley Traynor",
                "Irene Luque Aguilera",
                "Rory T. Devine"
            ],
            "abstract": "In this paper we present the first work on the automated scoring of\nmindreading ability in middle childhood and early adolescence. We create\nMIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066\nchildren aged 7 to 14. We perform machine learning experiments and carry out\nextensive quantitative and qualitative evaluation. We obtain promising results,\ndemonstrating the applicability of state-of-the-art NLP solutions to a new\ndomain and task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08073v2",
            "date": "2020-11-03 21:22:42+00:00",
            "title": "Analyzing Sustainability Reports Using Natural Language Processing",
            "authors": [
                "Alexandra Luccioni",
                "Emily Baylor",
                "Nicolas Duchene"
            ],
            "abstract": "Climate change is a far-reaching, global phenomenon that will impact many\naspects of our society, including the global stock market\n\\cite{dietz2016climate}. In recent years, companies have increasingly been\naiming to both mitigate their environmental impact and adapt to the changing\nclimate context. This is reported via increasingly exhaustive reports, which\ncover many types of climate risks and exposures under the umbrella of\nEnvironmental, Social, and Governance (ESG). However, given this abundance of\ndata, sustainability analysts are obliged to comb through hundreds of pages of\nreports in order to find relevant information. We leveraged recent progress in\nNatural Language Processing (NLP) to create a custom model, ClimateQA, which\nallows the analysis of financial reports in order to identify climate-relevant\nsections based on a question answering approach. We present this tool and the\nmethodology that we used to develop it in the present article.",
            "journal ref": "Tackling Climate Change with Machine Learning workshop at NeurIPS\n  2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08272v1",
            "date": "2020-11-16 20:58:35+00:00",
            "title": "NLPGym -- A toolkit for evaluating RL agents on Natural Language Processing Tasks",
            "authors": [
                "Rajkumar Ramamurthy",
                "Rafet Sifa",
                "Christian Bauckhage"
            ],
            "abstract": "Reinforcement learning (RL) has recently shown impressive performance in\ncomplex game AI and robotics tasks. To a large extent, this is thanks to the\navailability of simulated environments such as OpenAI Gym, Atari Learning\nEnvironment, or Malmo which allow agents to learn complex tasks through\ninteraction with virtual environments. While RL is also increasingly applied to\nnatural language processing (NLP), there are no simulated textual environments\navailable for researchers to apply and consistently benchmark RL on NLP tasks.\nWith the work reported here, we therefore release NLPGym, an open-source Python\ntoolkit that provides interactive textual environments for standard NLP tasks\nsuch as sequence tagging, multi-label classification, and question answering.\nWe also present experimental results for 6 tasks using different RL algorithms\nwhich serve as baselines for further research. The toolkit is published at\nhttps://github.com/rajcscw/nlp-gym",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08277v1",
            "date": "2020-11-16 21:09:43+00:00",
            "title": "Where Are You? Localization from Embodied Dialog",
            "authors": [
                "Meera Hahn",
                "Jacob Krantz",
                "Dhruv Batra",
                "Devi Parikh",
                "James M. Rehg",
                "Stefan Lee",
                "Peter Anderson"
            ],
            "abstract": "We present Where Are You? (WAY), a dataset of ~6k dialogs in which two humans\n-- an Observer and a Locator -- complete a cooperative localization task. The\nObserver is spawned at random in a 3D environment and can navigate from\nfirst-person views while answering questions from the Locator. The Locator must\nlocalize the Observer in a detailed top-down map by asking questions and giving\ninstructions. Based on this dataset, we define three challenging tasks:\nLocalization from Embodied Dialog or LED (localizing the Observer from dialog\nhistory), Embodied Visual Dialog (modeling the Observer), and Cooperative\nLocalization (modeling both agents). In this paper, we focus on the LED task --\nproviding a strong baseline model with detailed ablations characterizing both\ndataset biases and the importance of various modeling choices. Our best model\nachieves 32.7% success at identifying the Observer's location within 3m in\nunseen buildings, vs. 70.4% for human Locators.",
            "journal ref": "EMNLP 2020",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.08523v2",
            "date": "2020-11-17 09:18:47+00:00",
            "title": "Self-supervised Document Clustering Based on BERT with Data Augment",
            "authors": [
                "Haoxiang Shi",
                "Cen Wang"
            ],
            "abstract": "Contrastive learning is a good way to pursue discriminative unsupervised\nlearning, which can inherit advantages and experiences of well-studied deep\nmodels without complexly novel model designing. In this paper, we propose two\nlearning method for document clustering, the one is a partial contrastive\nlearning with unsupervised data augment, and the other is a self-supervised\ncontrastive learning. Both methods achieve state-of-the-art results in\nclustering accuracy when compared to recently proposed unsupervised clustering\napproaches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08558v2",
            "date": "2020-11-17 10:45:05+00:00",
            "title": "Generating universal language adversarial examples by understanding and enhancing the transferability across neural models",
            "authors": [
                "Liping Yuan",
                "Xiaoqing Zheng",
                "Yi Zhou",
                "Cho-Jui Hsieh",
                "Kai-wei Chang",
                "Xuanjing Huang"
            ],
            "abstract": "Deep neural network models are vulnerable to adversarial attacks. In many\ncases, malicious inputs intentionally crafted for one model can fool another\nmodel in the black-box attack setting. However, there is a lack of systematic\nstudies on the transferability of adversarial examples and how to generate\nuniversal adversarial examples. In this paper, we systematically study the\ntransferability of adversarial attacks for text classification models. In\nparticular, we conduct extensive experiments to investigate how various\nfactors, such as network architecture, input format, word embedding, and model\ncapacity, affect the transferability of adversarial attacks. Based on these\nstudies, we then propose universal black-box attack algorithms that can induce\nadversarial examples to attack almost all existing models. These universal\nadversarial examples reflect the defects of the learning process and the bias\nin the training dataset. Finally, we generalize these adversarial examples into\nuniversal word replacement rules that can be used for model diagnostics.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.08678v2",
            "date": "2020-11-17 14:50:55+00:00",
            "title": "Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources",
            "authors": [
                "Sicheng Zhao",
                "Yang Xiao",
                "Jiang Guo",
                "Xiangyu Yue",
                "Jufeng Yang",
                "Ravi Krishna",
                "Pengfei Xu",
                "Kurt Keutzer"
            ],
            "abstract": "Sentiment analysis of user-generated reviews or comments on products and\nservices in social networks can help enterprises to analyze the feedback from\ncustomers and take corresponding actions for improvement. To mitigate\nlarge-scale annotations on the target domain, domain adaptation (DA) provides\nan alternate solution by learning a transferable model from other labeled\nsource domains. Existing multi-source domain adaptation (MDA) methods either\nfail to extract some discriminative features in the target domain that are\nrelated to sentiment, neglect the correlations of different sources and the\ndistribution difference among different sub-domains even in the same source, or\ncannot reflect the varying optimal weighting during different training stages.\nIn this paper, we propose a novel instance-level MDA framework, named\ncurriculum cycle-consistent generative adversarial network (C-CycleGAN), to\naddress the above issues. Specifically, C-CycleGAN consists of three\ncomponents: (1) pre-trained text encoder which encodes textual input from\ndifferent domains into a continuous representation space, (2) intermediate\ndomain generator with curriculum instance-level adaptation which bridges the\ngap across source and target domains, and (3) task classifier trained on the\nintermediate domain for final sentiment classification. C-CycleGAN transfers\nsource samples at instance-level to an intermediate domain that is closer to\nthe target domain with sentiment semantics preserved and without losing\ndiscriminative features. Further, our dynamic instance-level weighting\nmechanisms can assign the optimal weights to different source samples in each\ntraining stage. We conduct extensive experiments on three benchmark datasets\nand achieve substantial gains over state-of-the-art DA approaches. Our source\ncode is released at: https://github.com/WArushrush/Curriculum-CycleGAN.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08772v2",
            "date": "2020-11-17 16:57:41+00:00",
            "title": "KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System",
            "authors": [
                "Hongru Wang",
                "Min Li",
                "Zimo Zhou",
                "Gabriel Pui Cheong Fung",
                "Kam-Fai Wong"
            ],
            "abstract": "Compared with CrossWOZ (Chinese) and MultiWOZ (English) dataset which have\ncoarse-grained information, there is no dataset which handle fine-grained and\nhierarchical level information properly. In this paper, we publish a first\nCantonese knowledge-driven Dialogue Dataset for REStaurant (KddRES) in Hong\nKong, which grounds the information in multi-turn conversations to one specific\nrestaurant. Our corpus contains 0.8k conversations which derive from 10\nrestaurants with various styles in different regions. In addition to that, we\ndesigned fine-grained slots and intents to better capture semantic information.\nThe benchmark experiments and data statistic analysis show the diversity and\nrich annotations of our dataset. We believe the publish of KddRES can be a\nnecessary supplement of current dialogue datasets and more suitable and\nvaluable for small and middle enterprises (SMEs) of society, such as build a\ncustomized dialogue system for each restaurant. The corpus and benchmark models\nare publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08906v2",
            "date": "2020-11-17 19:52:32+00:00",
            "title": "Gunrock 2.0: A User Adaptive Social Conversational System",
            "authors": [
                "Kaihui Liang",
                "Austin Chau",
                "Yu Li",
                "Xueyuan Lu",
                "Dian Yu",
                "Mingyang Zhou",
                "Ishan Jain",
                "Sam Davidson",
                "Josh Arnold",
                "Minh Nguyen",
                "Zhou Yu"
            ],
            "abstract": "Gunrock 2.0 is built on top of Gunrock with an emphasis on user adaptation.\nGunrock 2.0 combines various neural natural language understanding modules,\nincluding named entity detection, linking, and dialog act prediction, to\nimprove user understanding. Its dialog management is a hierarchical model that\nhandles various topics, such as movies, music, and sports. The system-level\ndialog manager can handle question detection, acknowledgment, error handling,\nand additional functions, making downstream modules much easier to design and\nimplement. The dialog manager also adapts its topic selection to accommodate\ndifferent users' profile information, such as inferred gender and personality.\nThe generation model is a mix of templates and neural generation models.\nGunrock 2.0 is able to achieve an average rating of 3.73 at its latest build\nfrom May 29th to June 4th.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.08927v2",
            "date": "2020-11-16 18:32:22+00:00",
            "title": "A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits",
            "authors": [
                "Arda Mavi"
            ],
            "abstract": "According to interviews with people who work with speech impaired persons,\nspeech impaired people have difficulties in communicating with other people\naround them who do not know the sign language, and this situation may cause\nthem to isolate themselves from society and lose their sense of independence.\nWith this paper, to increase the quality of life of individuals with\nfacilitating communication between individuals who use sign language and who do\nnot know this language, a new American Sign Language (ASL) digits dataset that\ncan help to create machine learning algorithms which need to large and varied\ndata to be successful created and published as Sign Language Digits Dataset on\nKaggle Datasets web page, a proposal Convolutional Neural Network (CNN)\narchitecture that can get 98% test accuracy on our dataset presented, and\ncompared with the existing popular CNN models.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.08952v1",
            "date": "2020-11-17 21:23:58+00:00",
            "title": "Argumentative Topology: Finding Loop(holes) in Logic",
            "authors": [
                "Sarah Tymochko",
                "Zachary New",
                "Lucius Bynum",
                "Emilie Purvine",
                "Timothy Doster",
                "Julien Chaput",
                "Tegan Emerson"
            ],
            "abstract": "Advances in natural language processing have resulted in increased\ncapabilities with respect to multiple tasks. One of the possible causes of the\nobserved performance gains is the introduction of increasingly sophisticated\ntext representations. While many of the new word embedding techniques can be\nshown to capture particular notions of sentiment or associative structures, we\nexplore the ability of two different word embeddings to uncover or capture the\nnotion of logical shape in text. To this end we present a novel framework that\nwe call Topological Word Embeddings which leverages mathematical techniques in\ndynamical system analysis and data driven shape extraction (i.e. topological\ndata analysis). In this preliminary work we show that using a topological delay\nembedding we are able to capture and extract a different, shape-based notion of\nlogic aimed at answering the question \"Can we find a circle in a circular\nargument?\"",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09039v1",
            "date": "2020-11-18 02:18:04+00:00",
            "title": "Sequence-Level Mixed Sample Data Augmentation",
            "authors": [
                "Demi Guo",
                "Yoon Kim",
                "Alexander M. Rush"
            ],
            "abstract": "Despite their empirical success, neural networks still have difficulty\ncapturing compositional aspects of natural language. This work proposes a\nsimple data augmentation approach to encourage compositional behavior in neural\nmodels for sequence-to-sequence problems. Our approach, SeqMix, creates new\nsynthetic examples by softly combining input/output sequences from the training\nset. We connect this approach to existing techniques such as SwitchOut and word\ndropout, and show that these techniques are all approximating variants of a\nsingle objective. SeqMix consistently yields approximately 1.0 BLEU improvement\non five different translation datasets over strong Transformer baselines. On\ntasks that require strong compositional generalization such as SCAN and\nsemantic parsing, SeqMix also offers further improvements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09046v2",
            "date": "2020-11-18 02:42:36+00:00",
            "title": "A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus",
            "authors": [
                "Bowen Zhang",
                "Hexiang Hu",
                "Joonseok Lee",
                "Ming Zhao",
                "Sheide Chammas",
                "Vihan Jain",
                "Eugene Ie",
                "Fei Sha"
            ],
            "abstract": "Identifying a short segment in a long video that semantically matches a text\nquery is a challenging task that has important application potentials in\nlanguage-based video search, browsing, and navigation. Typical retrieval\nsystems respond to a query with either a whole video or a pre-defined video\nsegment, but it is challenging to localize undefined segments in untrimmed and\nunsegmented videos where exhaustively searching over all possible segments is\nintractable. The outstanding challenge is that the representation of a video\nmust account for different levels of granularity in the temporal domain. To\ntackle this problem, we propose the HierArchical Multi-Modal EncodeR (HAMMER)\nthat encodes a video at both the coarse-grained clip level and the fine-grained\nframe level to extract information at different scales based on multiple\nsubtasks, namely, video retrieval, segment temporal localization, and masked\nlanguage modeling. We conduct extensive experiments to evaluate our model on\nmoment localization in video corpus on ActivityNet Captions and TVR datasets.\nOur approach outperforms the previous methods as well as strong baselines,\nestablishing new state-of-the-art for this task.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.09159v1",
            "date": "2020-11-18 08:52:49+00:00",
            "title": "Do Fine-tuned Commonsense Language Models Really Generalize?",
            "authors": [
                "Mayank Kejriwal",
                "Ke Shen"
            ],
            "abstract": "Recently, transformer-based methods such as RoBERTa and GPT-3 have led to\nsignificant experimental advances in natural language processing tasks such as\nquestion answering and commonsense reasoning. The latter is typically evaluated\nthrough multiple benchmarks framed as multiple-choice instances of the former.\nAccording to influential leaderboards hosted by the Allen Institute (evaluating\nstate-of-the-art performance on commonsense reasoning benchmarks), models based\non such transformer methods are approaching human-like performance and have\naverage accuracy well over 80% on many benchmarks. Since these are commonsense\nbenchmarks, a model that generalizes on commonsense reasoning should not\nexperience much performance loss across multiple commonsense benchmarks. In\nthis paper, we study the generalization issue in detail by designing and\nconducting a rigorous scientific study. Using five common benchmarks, multiple\ncontrols and statistical analysis, we find clear evidence that fine-tuned\ncommonsense language models still do not generalize well, even with moderate\nchanges to the experimental setup, and may, in fact, be susceptible to dataset\nbias. We also perform selective studies, including qualitative and consistency\nanalyses, to gain deeper insight into the problem.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09351v1",
            "date": "2020-11-16 07:20:02+00:00",
            "title": "Learning Regular Expressions for Interpretable Medical Text Classification Using a Pool-based Simulated Annealing and Word-vector Models",
            "authors": [
                "Chaofan Tu",
                "Ruibin Bai",
                "Zheng Lu",
                "Uwe Aickelin",
                "Peiming Ge",
                "Jianshuang Zhao"
            ],
            "abstract": "In this paper, we propose a rule-based engine composed of high quality and\ninterpretable regular expressions for medical text classification. The regular\nexpressions are auto generated by a constructive heuristic method and optimized\nusing a Pool-based Simulated Annealing (PSA) approach. Although existing Deep\nNeural Network (DNN) methods present high quality performance in most Natural\nLanguage Processing (NLP) applications, the solutions are regarded as\nuninterpretable black boxes to humans. Therefore, rule-based methods are often\nintroduced when interpretable solutions are needed, especially in the medical\nfield. However, the construction of regular expressions can be extremely\nlabor-intensive for large data sets. This research aims to reduce the manual\nefforts while maintaining high-quality solutions",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09410v3",
            "date": "2020-11-18 17:10:02+00:00",
            "title": "A Definition and a Test for Human-Level Artificial Intelligence",
            "authors": [
                "Md Ashaduzzaman Rubel Mondol",
                "Aishwarya Pothula",
                "Deokgun Park"
            ],
            "abstract": "Despite recent advances in many application-specific domains, we do not know\nhow to build a human-level artificial intelligence (HLAI). We conjecture that\nlearning from others' experience with the language is the essential\ncharacteristic that distinguishes human intelligence from the rest. Humans can\nupdate the action-value function with the verbal description as if they\nexperience states, actions, and corresponding rewards sequences firsthand. In\nthis paper, we present a classification of intelligence according to how\nindividual agents learn and propose a definition and a test for HLAI. The main\nidea is that language acquisition without explicit rewards can be a sufficient\ntest for HLAI.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2011.09448v1",
            "date": "2020-11-18 18:25:58+00:00",
            "title": "Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification",
            "authors": [
                "Daniel Palomino",
                "Jose Ochoa-Luna"
            ],
            "abstract": "We present a transfer learning system to perform a mixed Spanish-English\nsentiment classification task. Our proposal uses the state-of-the-art language\nmodel BERT and embed it within a ULMFiT transfer learning pipeline. This\ncombination allows us to predict the polarity detection of code-mixed\n(English-Spanish) tweets. Thus, among 29 submitted systems, our approach\n(referred to as dplominop) is ranked 4th on the Sentimix Spanglish test set of\nSemEval 2020 Task 9. In fact, our system yields the weighted-F1 score value of\n0.755 which can be easily reproduced -- the source code and implementation\ndetails are made available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09625v1",
            "date": "2020-11-19 03:11:24+00:00",
            "title": "Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP",
            "authors": [
                "John Chen",
                "Ian Berlot-Atwell",
                "Safwan Hossain",
                "Xindi Wang",
                "Frank Rudzicz"
            ],
            "abstract": "Clinical machine learning is increasingly multimodal, collected in both\nstructured tabular formats and unstructured forms such as freetext. We propose\na novel task of exploring fairness on a multimodal clinical dataset, adopting\nequalized odds for the downstream medical prediction tasks. To this end, we\ninvestigate a modality-agnostic fairness algorithm - equalized odds post\nprocessing - and compare it to a text-specific fairness algorithm: debiased\nclinical word embeddings. Despite the fact that debiased word embeddings do not\nexplicitly address equalized odds of protected groups, we show that a\ntext-specific approach to fairness may simultaneously achieve a good balance of\nperformance and classical notions of fairness. We hope that our paper inspires\nfuture contributions at the critical intersection of clinical NLP and fairness.\nThe full source code is available here:\nhttps://github.com/johntiger1/multimodal_fairness",
            "journal ref": "Proceedings of the 3rd Clinical Natural Language Processing\n  Workshop (2020), pages 301--312",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09658v1",
            "date": "2020-11-19 05:19:46+00:00",
            "title": "Relation Extraction with Contextualized Relation Embedding (CRE)",
            "authors": [
                "Xiaoyu Chen",
                "Rohan Badlani"
            ],
            "abstract": "Relation extraction is the task of identifying relation instance between two\nentities given a corpus whereas Knowledge base modeling is the task of\nrepresenting a knowledge base, in terms of relations between entities. This\npaper proposes an architecture for the relation extraction task that integrates\nsemantic information with knowledge base modeling in a novel manner. Existing\napproaches for relation extraction either do not utilize knowledge base\nmodelling or use separately trained KB models for the RE task. We present a\nmodel architecture that internalizes KB modeling in relation extraction. This\nmodel applies a novel approach to encode sentences into contextualized relation\nembeddings, which can then be used together with parameterized entity\nembeddings to score relation instances. The proposed CRE model achieves state\nof the art performance on datasets derived from The New York Times Annotated\nCorpus and FreeBase. The source code has been made available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09754v2",
            "date": "2020-11-19 10:18:47+00:00",
            "title": "An Integrated Approach for Improving Brand Consistency of Web Content: Modeling, Analysis and Recommendation",
            "authors": [
                "Soumyadeep Roy",
                "Shamik Sural",
                "Niyati Chhaya",
                "Anandhavelu Natarajan",
                "Niloy Ganguly"
            ],
            "abstract": "A consumer-dependent (business-to-consumer) organization tends to present\nitself as possessing a set of human qualities, which is termed as the brand\npersonality of the company. The perception is impressed upon the consumer\nthrough the content, be it in the form of advertisement, blogs or magazines,\nproduced by the organization. A consistent brand will generate trust and retain\ncustomers over time as they develop an affinity towards regularity and common\npatterns. However, maintaining a consistent messaging tone for a brand has\nbecome more challenging with the virtual explosion in the amount of content\nwhich needs to be authored and pushed to the Internet to maintain an edge in\nthe era of digital marketing. To understand the depth of the problem, we\ncollect around 300K web page content from around 650 companies. We develop\ntrait-specific classification models by considering the linguistic features of\nthe content. The classifier automatically identifies the web articles which are\nnot consistent with the mission and vision of a company and further helps us to\ndiscover the conditions under which the consistency cannot be maintained. To\naddress the brand inconsistency issue, we then develop a sentence ranking\nsystem that outputs the top three sentences that need to be changed for making\na web article more consistent with the company's brand personality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.09954v2",
            "date": "2020-11-19 16:52:43+00:00",
            "title": "Persuasive Dialogue Understanding: the Baselines and Negative Results",
            "authors": [
                "Hui Chen",
                "Deepanway Ghosal",
                "Navonil Majumder",
                "Amir Hussain",
                "Soujanya Poria"
            ],
            "abstract": "Persuasion aims at forming one's opinion and action via a series of\npersuasive messages containing persuader's strategies. Due to its potential\napplication in persuasive dialogue systems, the task of persuasive strategy\nrecognition has gained much attention lately. Previous methods on user intent\nrecognition in dialogue systems adopt recurrent neural network (RNN) or\nconvolutional neural network (CNN) to model context in conversational history,\nneglecting the tactic history and intra-speaker relation. In this paper, we\ndemonstrate the limitations of a Transformer-based approach coupled with\nConditional Random Field (CRF) for the task of persuasive strategy recognition.\nIn this model, we leverage inter- and intra-speaker contextual semantic\nfeatures, as well as label dependencies to improve the recognition. Despite\nextensive hyper-parameter optimizations, this architecture fails to outperform\nthe baseline methods. We observe two negative results. Firstly, CRF cannot\ncapture persuasive label dependencies, possibly as strategies in persuasive\ndialogues do not follow any strict grammar or rules as the cases in Named\nEntity Recognition (NER) or part-of-speech (POS) tagging. Secondly, the\nTransformer encoder trained from scratch is less capable of capturing\nsequential information in persuasive dialogues than Long Short-Term Memory\n(LSTM). We attribute this to the reason that the vanilla Transformer encoder\ndoes not efficiently consider relative position information of sequence\nelements.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10132v1",
            "date": "2020-11-19 22:32:03+00:00",
            "title": "VLG-Net: Video-Language Graph Matching Network for Video Grounding",
            "authors": [
                "Sisi Qu",
                "Mattia Soldan",
                "Mengmeng Xu",
                "Jesper Tegner",
                "Bernard Ghanem"
            ],
            "abstract": "Grounding language queries in videos aims at identifying the time interval\n(or moment) semantically relevant to a language query. The solution to this\nchallenging task demands the understanding of videos' and queries' semantic\ncontent and the fine-grained reasoning about their multi-modal interactions.\nOur key idea is to recast this challenge into an algorithmic graph matching\nproblem. Fueled by recent advances in Graph Neural Networks, we propose to\nleverage Graph Convolutional Networks to model video and textual information as\nwell as their semantic alignment. To enable the mutual exchange of information\nacross the domains, we design a novel Video-Language Graph Matching Network\n(VLG-Net) to match video and query graphs. Core ingredients include\nrepresentation graphs, built on top of video snippets and query tokens\nseparately, which are used for modeling the intra-modality relationships. A\nGraph Matching layer is adopted for cross-modal context modeling and\nmulti-modal fusion. Finally, moment candidates are created using masked moment\nattention pooling by fusing the moment's enriched snippet features. We\ndemonstrate superior performance over state-of-the-art grounding methods on\nthree widely used datasets for temporal localization of moments in videos with\nnatural language queries: ActivityNet-Captions, TACoS, and DiDeMo.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.10285v1",
            "date": "2020-11-20 08:56:31+00:00",
            "title": "Learning Informative Representations of Biomedical Relations with Latent Variable Models",
            "authors": [
                "Harshil Shah",
                "Julien Fauqueur"
            ],
            "abstract": "Extracting biomedical relations from large corpora of scientific documents is\na challenging natural language processing task. Existing approaches usually\nfocus on identifying a relation either in a single sentence (mention-level) or\nacross an entire corpus (pair-level). In both cases, recent methods have\nachieved strong results by learning a point estimate to represent the relation;\nthis is then used as the input to a relation classifier. However, the relation\nexpressed in text between a pair of biomedical entities is often more complex\nthan can be captured by a point estimate. To address this issue, we propose a\nlatent variable model with an arbitrarily flexible distribution to represent\nthe relation between an entity pair. Additionally, our model provides a unified\narchitecture for both mention-level and pair-level relation extraction. We\ndemonstrate that our model achieves results competitive with strong baselines\nfor both tasks while having fewer parameters and being significantly faster to\ntrain. We make our code publicly available.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10358v1",
            "date": "2020-11-20 11:59:16+00:00",
            "title": "A Deep Language-independent Network to analyze the impact of COVID-19 on the World via Sentiment Analysis",
            "authors": [
                "Ashima Yadav",
                "Dinesh Kumar Vishwakarma"
            ],
            "abstract": "Towards the end of 2019, Wuhan experienced an outbreak of novel coronavirus,\nwhich soon spread all over the world, resulting in a deadly pandemic that\ninfected millions of people around the globe. The government and public health\nagencies followed many strategies to counter the fatal virus. However, the\nvirus severely affected the social and economic lives of the people. In this\npaper, we extract and study the opinion of people from the top five worst\naffected countries by the virus, namely USA, Brazil, India, Russia, and South\nAfrica. We propose a deep language-independent Multilevel Attention-based\nConv-BiGRU network (MACBiG-Net), which includes embedding layer, word-level\nencoded attention, and sentence-level encoded attention mechanism to extract\nthe positive, negative, and neutral sentiments. The embedding layer encodes the\nsentence sequence into a real-valued vector. The word-level and sentence-level\nencoding is performed by a 1D Conv-BiGRU based mechanism, followed by\nword-level and sentence-level attention, respectively. We further develop a\nCOVID-19 Sentiment Dataset by crawling the tweets from Twitter. Extensive\nexperiments on our proposed dataset demonstrate the effectiveness of the\nproposed MACBiG-Net. Also, attention-weights visualization and in-depth results\nanalysis shows that the proposed network has effectively captured the\nsentiments of the people.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10369v1",
            "date": "2020-11-20 12:17:21+00:00",
            "title": "ONION: A Simple and Effective Defense Against Textual Backdoor Attacks",
            "authors": [
                "Fanchao Qi",
                "Yangyi Chen",
                "Mukai Li",
                "Zhiyuan Liu",
                "Maosong Sun"
            ],
            "abstract": "Backdoor attacks, which are a kind of emergent training-time threat to deep\nneural networks (DNNS). They can manipulate the output of DNNs and posses high\ninsidiousness. In the field of natural language processing, some attack methods\nhave been proposed and achieve very high attack success rates on multiple\npopular models. Nevertheless, the studies on defending textual backdoor defense\nare little conducted. In this paper, we propose a simple and effective textual\nbackdoor defense named ONION, which is based on outlier word detection and\nmight be the first method that can handle all the attack situations.\nExperiments demonstrate the effectiveness of our model when blocking two latest\nbackdoor attack methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10426v1",
            "date": "2020-11-20 14:45:46+00:00",
            "title": "Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews",
            "authors": [
                "Quoc Thai Nguyen",
                "Thoai Linh Nguyen",
                "Ngoc Hoang Luong",
                "Quoc Hung Ngo"
            ],
            "abstract": "Sentiment analysis is an important task in the field ofNature Language\nProcessing (NLP), in which users' feedbackdata on a specific issue are\nevaluated and analyzed. Manydeep learning models have been proposed to tackle\nthis task, including the recently-introduced Bidirectional Encoder\nRep-resentations from Transformers (BERT) model. In this paper,we experiment\nwith two BERT fine-tuning methods for thesentiment analysis task on datasets of\nVietnamese reviews: 1) a method that uses only the [CLS] token as the input for\nanattached feed-forward neural network, and 2) another methodin which all BERT\noutput vectors are used as the input forclassification. Experimental results on\ntwo datasets show thatmodels using BERT slightly outperform other models\nusingGloVe and FastText. Also, regarding the datasets employed inthis study,\nour proposed BERT fine-tuning method produces amodel with better performance\nthan the original BERT fine-tuning method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10647v1",
            "date": "2020-11-20 21:27:10+00:00",
            "title": "What do we expect from Multiple-choice QA Systems?",
            "authors": [
                "Krunal Shah",
                "Nitish Gupta",
                "Dan Roth"
            ],
            "abstract": "The recent success of machine learning systems on various QA datasets could\nbe interpreted as a significant improvement in models' language understanding\nabilities. However, using various perturbations, multiple recent works have\nshown that good performance on a dataset might not indicate performance that\ncorrelates well with human's expectations from models that \"understand\"\nlanguage. In this work we consider a top performing model on several Multiple\nChoice Question Answering (MCQA) datasets, and evaluate it against a set of\nexpectations one might have from such a model, using a series of\nzero-information perturbations of the model's inputs. Our results show that the\nmodel clearly falls short of our expectations, and motivates a modified\ntraining approach that forces the model to better attend to the inputs. We show\nthat the new training paradigm leads to a model that performs on par with the\noriginal model while better satisfying our expectations.",
            "journal ref": "Findings of the Association for Computational Linguistics: EMNLP\n  2020 pg. 3547-3553",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.10652v1",
            "date": "2020-11-20 21:38:34+00:00",
            "title": "Self-Supervised learning with cross-modal transformers for emotion recognition",
            "authors": [
                "Aparna Khare",
                "Srinivas Parthasarathy",
                "Shiva Sundaram"
            ],
            "abstract": "Emotion recognition is a challenging task due to limited availability of\nin-the-wild labeled datasets. Self-supervised learning has shown improvements\non tasks with limited labeled datasets in domains like speech and natural\nlanguage. Models such as BERT learn to incorporate context in word embeddings,\nwhich translates to improved performance in downstream tasks like question\nanswering. In this work, we extend self-supervised training to multi-modal\napplications. We learn multi-modal representations using a transformer trained\non the masked language modeling task with audio, visual and text features. This\nmodel is fine-tuned on the downstream task of emotion recognition. Our results\non the CMU-MOSEI dataset show that this pre-training technique can improve the\nemotion recognition performance by up to 3% compared to the baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11074v1",
            "date": "2020-11-22 17:39:43+00:00",
            "title": "Standardizing linguistic data: method and tools for annotating (pre-orthographic) French",
            "authors": [
                "Simon Gabay",
                "Thibault Cl\u00e9rice",
                "Jean-Baptiste Camps",
                "Jean-Baptiste Tanguy",
                "Matthias Gille-Levenson"
            ],
            "abstract": "With the development of big corpora of various periods, it becomes crucial to\nstandardise linguistic annotation (e.g. lemmas, POS tags, morphological\nannotation) to increase the interoperability of the data produced, despite\ndiachronic variations. In the present paper, we describe both methodologically\n(by proposing annotation principles) and technically (by creating the required\ntraining data and the relevant models) the production of a linguistic tagger\nfor (early) modern French (16-18th c.), taking as much as possible into account\nalready existing standards for contemporary and, especially, medieval French.",
            "journal ref": "Proceedings of the 2nd International Digital Tools & Uses Congress\n  (DTUC '20), Oct 2020, Hammamet, Tunisia",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11263v2",
            "date": "2020-11-23 08:08:09+00:00",
            "title": "Evaluating Input Representation for Language Identification in Hindi-English Code Mixed Text",
            "authors": [
                "Ramchandra Joshi",
                "Raviraj Joshi"
            ],
            "abstract": "Natural language processing (NLP) techniques have become mainstream in the\nrecent decade. Most of these advances are attributed to the processing of a\nsingle language. More recently, with the extensive growth of social media\nplatforms focus has shifted to code-mixed text. The code-mixed text comprises\ntext written in more than one language. People naturally tend to combine local\nlanguage with global languages like English. To process such texts, current NLP\ntechniques are not sufficient. As a first step, the text is processed to\nidentify the language of the words in the text. In this work, we focus on\nlanguage identification in code-mixed sentences for Hindi-English mixed text.\nThe task of language identification is formulated as a token classification\ntask. In the supervised setting, each word in the sentence has an associated\nlanguage label. We evaluate different deep learning models and input\nrepresentation combinations for this task. Mainly, character, sub-word, and\nword embeddings are considered in combination with CNN and LSTM based models.\nWe show that sub-word representation along with the LSTM model gives the best\nresults. In general sub-word representations perform significantly better than\nother input representations. We report the best accuracy of 94.52% using a\nsingle layer LSTM model on the standard SAIL ICON 2017 test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11551v1",
            "date": "2020-11-23 17:04:33+00:00",
            "title": "Conformance Checking of Mixed-paradigm Process Models",
            "authors": [
                "Boudewijn van Dongen",
                "Johannes De Smedt",
                "Claudio Di Ciccio",
                "Jan Mendling"
            ],
            "abstract": "Mixed-paradigm process models integrate strengths of procedural and\ndeclarative representations like Petri nets and Declare. They are specifically\ninteresting for process mining because they allow capturing complex behaviour\nin a compact way. A key research challenge for the proliferation of\nmixed-paradigm models for process mining is the lack of corresponding\nconformance checking techniques. In this paper, we address this problem by\ndevising the first approach that works with intertwined state spaces of\nmixed-paradigm models. More specifically, our approach uses an alignment-based\nreplay to explore the state space and compute trace fitness in a procedural\nway. In every state, the declarative constraints are separately updated, such\nthat violations disable the corresponding activities. Our technique provides\nfor an efficient replay towards an optimal alignment by respecting all\northogonal Declare constraints. We have implemented our technique in ProM and\ndemonstrate its performance in an evaluation with real-world event logs.",
            "journal ref": "None",
            "category": "cs.FL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11851v1",
            "date": "2020-11-24 02:35:24+00:00",
            "title": "Dual Supervision Framework for Relation Extraction with Distant Supervision and Human Annotation",
            "authors": [
                "Woohwan Jung",
                "Kyuseok Shim"
            ],
            "abstract": "Relation extraction (RE) has been extensively studied due to its importance\nin real-world applications such as knowledge base construction and question\nanswering. Most of the existing works train the models on either distantly\nsupervised data or human-annotated data. To take advantage of the high accuracy\nof human annotation and the cheap cost of distant supervision, we propose the\ndual supervision framework which effectively utilizes both types of data.\nHowever, simply combining the two types of data to train a RE model may\ndecrease the prediction accuracy since distant supervision has labeling bias.\nWe employ two separate prediction networks HA-Net and DS-Net to predict the\nlabels by human annotation and distant supervision, respectively, to prevent\nthe degradation of accuracy by the incorrect labeling of distant supervision.\nFurthermore, we propose an additional loss term called disagreement penalty to\nenable HA-Net to learn from distantly supervised labels. In addition, we\nexploit additional networks to adaptively assess the labeling bias by\nconsidering contextual information. Our performance study on sentence-level and\ndocument-level REs confirms the effectiveness of the dual supervision\nframework.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.11855v1",
            "date": "2020-11-24 02:46:02+00:00",
            "title": "A Robotic Dating Coaching System Leveraging Online Communities Posts",
            "authors": [
                "Sihyeon Jo",
                "Donghwi Jung",
                "Keonwoo Kim",
                "Eun Gyo Joung",
                "Giulia Nespoli",
                "Seungryong Yoo",
                "Minseob So",
                "Seung-Woo Seo",
                "Seong-Woo Kim"
            ],
            "abstract": "Can a robot be a personal dating coach? Even with the increasing amount of\nconversational data on the internet, the implementation of conversational\nrobots remains a challenge. In particular, a detailed and professional\ncounseling log is expensive and not publicly accessible. In this paper, we\ndevelop a robot dating coaching system leveraging corpus from online\ncommunities. We examine people's perceptions of the dating coaching robot with\na dialogue module. 97 participants joined to have a conversation with the\nrobot, and 30 of them evaluated the robot. The results indicate that\nparticipants thought the robot could become a dating coach while considering\nthe robot is entertaining rather than helpful.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/2011.12014v1",
            "date": "2020-11-24 10:39:44+00:00",
            "title": "Argument from Old Man's View: Assessing Social Bias in Argumentation",
            "authors": [
                "Maximilian Splieth\u00f6ver",
                "Henning Wachsmuth"
            ],
            "abstract": "Social bias in language - towards genders, ethnicities, ages, and other\nsocial groups - poses a problem with ethical impact for many NLP applications.\nRecent research has shown that machine learning models trained on respective\ndata may not only adopt, but even amplify the bias. So far, however, little\nattention has been paid to bias in computational argumentation. In this paper,\nwe study the existence of social biases in large English debate portals. In\nparticular, we train word embedding models on portal-specific corpora and\nsystematically evaluate their bias using WEAT, an existing metric to measure\nbias in word embeddings. In a word co-occurrence analysis, we then investigate\ncauses of bias. The results suggest that all tested debate corpora contain\nunbalanced and biased data, mostly in favor of male people with\nEuropean-American names. Our empirical insights contribute towards an\nunderstanding of bias in argumentative data sources.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12075v1",
            "date": "2020-11-24 13:22:34+00:00",
            "title": "Fuzzy Stochastic Timed Petri Nets for Causal properties representation",
            "authors": [
                "Alejandro Sobrino",
                "Eduardo C. Garrido-Merchan",
                "Cristina Puente"
            ],
            "abstract": "Imagery is frequently used to model, represent and communicate knowledge. In\nparticular, graphs are one of the most powerful tools, being able to represent\nrelations between objects. Causal relations are frequently represented by\ndirected graphs, with nodes denoting causes and links denoting causal\ninfluence. A causal graph is a skeletal picture, showing causal associations\nand impact between entities. Common methods used for graphically representing\ncausal scenarios are neurons, truth tables, causal Bayesian networks, cognitive\nmaps and Petri Nets. Causality is often defined in terms of precedence (the\ncause precedes the effect), concurrency (often, an effect is provoked\nsimultaneously by two or more causes), circularity (a cause provokes the effect\nand the effect reinforces the cause) and imprecision (the presence of the cause\nfavors the effect, but not necessarily causes it). We will show that, even\nthough the traditional graphical models are able to represent separately some\nof the properties aforementioned, they fail trying to illustrate indistinctly\nall of them. To approach that gap, we will introduce Fuzzy Stochastic Timed\nPetri Nets as a graphical tool able to represent time, co-occurrence, looping\nand imprecision in causal flow.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2011.12081v1",
            "date": "2020-11-24 13:34:38+00:00",
            "title": "Tackling Domain-Specific Winograd Schemas with Knowledge-Based Reasoning and Machine Learning",
            "authors": [
                "Suk Joon Hong",
                "Brandon Bennett"
            ],
            "abstract": "The Winograd Schema Challenge (WSC) is a common-sense reasoning task that\nrequires background knowledge. In this paper, we contribute to tackling WSC in\nfour ways. Firstly, we suggest a keyword method to define a restricted domain\nwhere distinctive high-level semantic patterns can be found. A thanking domain\nwas defined by key-words, and the data set in this domain is used in our\nexperiments. Secondly, we develop a high-level knowledge-based reasoning method\nusing semantic roles which is based on the method of Sharma [2019]. Thirdly, we\npropose an ensemble method to combine knowledge-based reasoning and machine\nlearning which shows the best performance in our experiments. As a machine\nlearning method, we used Bidirectional Encoder Representations from\nTransformers (BERT) [Kocijan et al., 2019]. Lastly, in terms of evaluation, we\nsuggest a \"robust\" accuracy measurement by modifying that of Trichelair et al.\n[2018]. As with their switching method, we evaluate a model by considering its\nperformance on trivial variants of each sentence in the test set.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12096v1",
            "date": "2020-11-24 14:02:49+00:00",
            "title": "Gender bias in magazines oriented to men and women: a computational approach",
            "authors": [
                "Diego Kozlowski",
                "Gabriela Lozano",
                "Carla M. Felcher",
                "Fernando Gonzalez",
                "Edgar Altszyler"
            ],
            "abstract": "Cultural products are a source to acquire individual values and behaviours.\nTherefore, the differences in the content of the magazines aimed specifically\nat women or men are a means to create and reproduce gender stereotypes. In this\nstudy, we compare the content of a women-oriented magazine with that of a\nmen-oriented one, both produced by the same editorial group, over a decade\n(2008-2018). With Topic Modelling techniques we identify the main themes\ndiscussed in the magazines and quantify how much the presence of these topics\ndiffers between magazines over time. Then, we performed a word-frequency\nanalysis to validate this methodology and extend the analysis to other subjects\nthat did not emerge automatically. Our results show that the frequency of\nappearance of the topics Family, Business and Women as sex objects, present an\ninitial bias that tends to disappear over time. Conversely, in Fashion and\nScience topics, the initial differences between both magazines are maintained.\nBesides, we show that in 2012, the content associated with horoscope increased\nin the women-oriented magazine, generating a new gap that remained open over\ntime. Also, we show a strong increase in the use of words associated with\nfeminism since 2015 and specifically the word abortion in 2018. Overall, these\ncomputational tools allowed us to analyse more than 24,000 articles. Up to our\nknowledge, this is the first study to compare magazines in such a large\ndataset, a task that would have been prohibitive using manual content analysis\nmethodologies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12165v1",
            "date": "2020-11-24 15:42:32+00:00",
            "title": "Two-Way Neural Machine Translation: A Proof of Concept for Bidirectional Translation Modeling using a Two-Dimensional Grid",
            "authors": [
                "Parnia Bahar",
                "Christopher Brix",
                "Hermann Ney"
            ],
            "abstract": "Neural translation models have proven to be effective in capturing sufficient\ninformation from a source sentence and generating a high-quality target\nsentence. However, it is not easy to get the best effect for bidirectional\ntranslation, i.e., both source-to-target and target-to-source translation using\na single model. If we exclude some pioneering attempts, such as multilingual\nsystems, all other bidirectional translation approaches are required to train\ntwo individual models. This paper proposes to build a single end-to-end\nbidirectional translation model using a two-dimensional grid, where the\nleft-to-right decoding generates source-to-target, and the bottom-to-up\ndecoding creates target-to-source output. Instead of training two models\nindependently, our approach encourages a single network to jointly learn to\ntranslate in both directions. Experiments on the WMT 2018\nGerman$\\leftrightarrow$English and Turkish$\\leftrightarrow$English translation\ntasks show that the proposed model is capable of generating a good translation\nquality and has sufficient potential to direct the research.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12167v1",
            "date": "2020-11-24 15:43:49+00:00",
            "title": "Tight Integrated End-to-End Training for Cascaded Speech Translation",
            "authors": [
                "Parnia Bahar",
                "Tobias Bieschke",
                "Ralf Schl\u00fcter",
                "Hermann Ney"
            ],
            "abstract": "A cascaded speech translation model relies on discrete and non-differentiable\ntranscription, which provides a supervision signal from the source side and\nhelps the transformation between source speech and target text. Such modeling\nsuffers from error propagation between ASR and MT models. Direct speech\ntranslation is an alternative method to avoid error propagation; however, its\nperformance is often behind the cascade system. To use an intermediate\nrepresentation and preserve the end-to-end trainability, previous studies have\nproposed using two-stage models by passing the hidden vectors of the recognizer\ninto the decoder of the MT model and ignoring the MT encoder. This work\nexplores the feasibility of collapsing the entire cascade components into a\nsingle end-to-end trainable model by optimizing all parameters of ASR and MT\nmodels jointly without ignoring any learned parameters. It is a tightly\nintegrated method that passes renormalized source word posterior distributions\nas a soft decision instead of one-hot vectors and enables backpropagation.\nTherefore, it provides both transcriptions and translations and achieves strong\nconsistency between them. Our experiments on four tasks with different data\nscenarios show that the model outperforms cascade models up to 1.8% in BLEU and\n2.0% in TER and is superior compared to direct models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12334v2",
            "date": "2020-11-24 19:21:00+00:00",
            "title": "Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo Approach",
            "authors": [
                "Maosen Zhang",
                "Nan Jiang",
                "Lei Li",
                "Yexiang Xue"
            ],
            "abstract": "Generating natural language under complex constraints is a principled\nformulation towards controllable text generation. We present a framework to\nallow specification of combinatorial constraints for sentence generation. We\npropose TSMH, an efficient method to generate high likelihood sentences with\nrespect to a pre-trained language model while satisfying the constraints. Our\napproach is highly flexible, requires no task-specific training, and leverages\nefficient constraint satisfaction solving techniques. To better handle the\ncombinatorial constraints, a tree search algorithm is embedded into the\nproposal process of the Markov chain Monte Carlo (MCMC) to explore candidates\nthat satisfy more constraints. Compared to existing MCMC approaches, our\nsampling approach has a better mixing performance. Experiments show that TSMH\nachieves consistent and significant improvement on multiple language generation\ntasks.",
            "journal ref": "Findings of the Association for Computational Linguistics: EMNLP\n  2020, pages 1286-1298. November 16 - 20, 2020. 2020 Association for\n  Computational Linguistics",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12380v1",
            "date": "2020-11-24 20:56:47+00:00",
            "title": "Experiments on transfer learning architectures for biomedical relation extraction",
            "authors": [
                "Walid Hafiane",
                "Joel Legrand",
                "Yannick Toussaint",
                "Adrien Coulet"
            ],
            "abstract": "Relation extraction (RE) consists in identifying and structuring\nautomatically relations of interest from texts. Recently, BERT improved the top\nperformances for several NLP tasks, including RE. However, the best way to use\nBERT, within a machine learning architecture, and within a transfer learning\nstrategy is still an open question since it is highly dependent on each\nspecific task and domain. Here, we explore various BERT-based architectures and\ntransfer learning strategies (i.e., frozen or fine-tuned) for the task of\nbiomedical RE on two corpora. Among tested architectures and strategies, our\n*BERT-segMCNN with finetuning reaches performances higher than the\nstate-of-the-art on the two corpora (1.73 % and 32.77 % absolute improvement on\nChemProt and PGxCorpus corpora respectively). More generally, our experiments\nillustrate the expected interest of fine-tuning with BERT, but also the\nunexplored advantage of using structural information (with sentence\nsegmentation), in addition to the context classically leveraged by BERT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12649v1",
            "date": "2020-11-25 11:19:12+00:00",
            "title": "Neural Representations for Modeling Variation in English Speech",
            "authors": [
                "Martijn Bartelds",
                "Wietse de Vries",
                "Faraz Sanal",
                "Caitlin Richter",
                "Mark Liberman",
                "Martijn Wieling"
            ],
            "abstract": "Variation in speech is often represented and investigated using phonetic\ntranscriptions, but transcribing speech is time-consuming and error prone. To\ncreate reliable representations of speech independent from phonetic\ntranscriptions, we investigate the extraction of acoustic embeddings from\nseveral self-supervised neural models. We use these representations to compute\nword-based pronunciation differences between non-native and native speakers of\nEnglish, and evaluate these differences by comparing them with human\nnative-likeness judgments. We show that Transformer-based speech\nrepresentations lead to significant performance gains over the use of phonetic\ntranscriptions, and find that feature-based use of Transformer models is most\neffective with one or more middle layers instead of the final layer. We also\ndemonstrate that these neural speech representations not only capture segmental\ndifferences, but also intonational and durational differences that cannot be\nrepresented by a set of discrete symbols used in phonetic transcriptions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.12662v3",
            "date": "2020-11-25 11:44:12+00:00",
            "title": "XTQA: Span-Level Explanations of the Textbook Question Answering",
            "authors": [
                "Jie Ma",
                "Jun Liu",
                "Junjun Li",
                "Qinghua Zheng",
                "Qingyu Yin",
                "Jianlong Zhou",
                "Yi Huang"
            ],
            "abstract": "Textbook Question Answering (TQA) is a task that one should answer a\ndiagram/non-diagram question given a large multi-modal context consisting of\nabundant essays and diagrams. We argue that the explainability of this task\nshould place students as a key aspect to be considered. To address this issue,\nwe devise a novel architecture towards span-level eXplanations of the TQA\n(XTQA) based on our proposed coarse-to-fine grained algorithm, which can\nprovide not only the answers but also the span-level evidences to choose them\nfor students. This algorithm first coarsely chooses top $M$ paragraphs relevant\nto questions using the TF-IDF method, and then chooses top $K$ evidence spans\nfinely from all candidate spans within these paragraphs by computing the\ninformation gain of each span to questions. Experimental results shows that\nXTQA significantly improves the state-of-the-art performance compared with\nbaselines. The source code is available at\nhttps://github.com/keep-smile-001/opentqa",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13087v2",
            "date": "2020-11-26 01:43:29+00:00",
            "title": "Text Analytics for Resilience-Enabled Extreme Events Reconnaissance",
            "authors": [
                "Alicia Y. Tsai",
                "Selim Gunay",
                "Minjune Hwang",
                "Pengyuan Zhai",
                "Chenglong Li",
                "Laurent El Ghaoui",
                "Khalid M. Mosalam"
            ],
            "abstract": "Post-hazard reconnaissance for natural disasters (e.g., earthquakes) is\nimportant for understanding the performance of the built environment, speeding\nup the recovery, enhancing resilience and making informed decisions related to\ncurrent and future hazards. Natural language processing (NLP) is used in this\nstudy for the purposes of increasing the accuracy and efficiency of natural\nhazard reconnaissance through automation. The study particularly focuses on (1)\nautomated data (news and social media) collection hosted by the Pacific\nEarthquake Engineering Research (PEER) Center server, (2) automatic generation\nof reconnaissance reports, and (3) use of social media to extract post-hazard\ninformation such as the recovery time. Obtained results are encouraging for\nfurther development and wider usage of various NLP methods in natural hazard\nreconnaissance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13200v1",
            "date": "2020-11-26 09:51:29+00:00",
            "title": "Unsupervised Word Translation Pairing using Refinement based Point Set Registration",
            "authors": [
                "Silviu Oprea",
                "Sourav Dutta",
                "Haytham Assem"
            ],
            "abstract": "Cross-lingual alignment of word embeddings play an important role in\nknowledge transfer across languages, for improving machine translation and\nother multi-lingual applications. Current unsupervised approaches rely on\nsimilarities in geometric structure of word embedding spaces across languages,\nto learn structure-preserving linear transformations using adversarial networks\nand refinement strategies. However, such techniques, in practice, tend to\nsuffer from instability and convergence issues, requiring tedious fine-tuning\nfor precise parameter setting. This paper proposes BioSpere, a novel framework\nfor unsupervised mapping of bi-lingual word embeddings onto a shared vector\nspace, by combining adversarial initialization and refinement procedure with\npoint set registration algorithm used in image processing. We show that our\nframework alleviates the shortcomings of existing methodologies, and is\nrelatively invariant to variable adversarial learning performance, depicting\nrobustness in terms of parameter choices and training losses. Experimental\nevaluation on parallel dictionary induction task demonstrates state-of-the-art\nresults for our framework on diverse language pairs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13205v1",
            "date": "2020-11-26 09:58:20+00:00",
            "title": "SLURP: A Spoken Language Understanding Resource Package",
            "authors": [
                "Emanuele Bastianelli",
                "Andrea Vanzo",
                "Pawel Swietojanski",
                "Verena Rieser"
            ],
            "abstract": "Spoken Language Understanding infers semantic meaning directly from audio\ndata, and thus promises to reduce error propagation and misunderstandings in\nend-user applications. However, publicly available SLU resources are limited.\nIn this paper, we release SLURP, a new SLU package containing the following:\n(1) A new challenging dataset in English spanning 18 domains, which is\nsubstantially bigger and linguistically more diverse than existing datasets;\n(2) Competitive baselines based on state-of-the-art NLU and ASR systems; (3) A\nnew transparent metric for entity labelling which enables a detailed error\nanalysis for identifying potential areas of improvement. SLURP is available at\nhttps: //github.com/pswietojanski/slurp.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13210v1",
            "date": "2020-11-26 10:10:57+00:00",
            "title": "Encoding Syntactic Constituency Paths for Frame-Semantic Parsing with Graph Convolutional Networks",
            "authors": [
                "Emanuele Bastianelli",
                "Andrea Vanzo",
                "Oliver Lemon"
            ],
            "abstract": "We study the problem of integrating syntactic information from constituency\ntrees into a neural model in Frame-semantic parsing sub-tasks, namely Target\nIdentification (TI), FrameIdentification (FI), and Semantic Role Labeling\n(SRL). We use a Graph Convolutional Network to learn specific representations\nof constituents, such that each constituent is profiled as the production\ngrammar rule it corresponds to. We leverage these representations to build\nsyntactic features for each word in a sentence, computed as the sum of all the\nconstituents on the path between a word and a task-specific node in the tree,\ne.g. the target predicate for SRL. Our approach improves state-of-the-art\nresults on the TI and SRL of ~1%and~3.5% points, respectively (+2.5% additional\npoints are gained with BERT as input), when tested on FrameNet 1.5, while\nyielding comparable results on the CoNLL05 dataset to other syntax-aware\nsystems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13220v1",
            "date": "2020-11-26 10:39:03+00:00",
            "title": "Unigram-Normalized Perplexity as a Language Model Performance Measure with Different Vocabulary Sizes",
            "authors": [
                "Jihyeon Roh",
                "Sang-Hoon Oh",
                "Soo-Young Lee"
            ],
            "abstract": "Although Perplexity is a widely used performance metric for language models,\nthe values are highly dependent upon the number of words in the corpus and is\nuseful to compare performance of the same corpus only. In this paper, we\npropose a new metric that can be used to evaluate language model performance\nwith different vocabulary sizes. The proposed unigram-normalized Perplexity\nactually presents the performance improvement of the language models from that\nof simple unigram model, and is robust on the vocabulary size. Both theoretical\nanalysis and computational experiments are reported.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13231v1",
            "date": "2020-11-26 10:59:23+00:00",
            "title": "NLPStatTest: A Toolkit for Comparing NLP System Performance",
            "authors": [
                "Haotian Zhu",
                "Denise Mak",
                "Jesse Gioannini",
                "Fei Xia"
            ],
            "abstract": "Statistical significance testing centered on p-values is commonly used to\ncompare NLP system performance, but p-values alone are insufficient because\nstatistical significance differs from practical significance. The latter can be\nmeasured by estimating effect size. In this paper, we propose a three-stage\nprocedure for comparing NLP system performance and provide a toolkit,\nNLPStatTest, that automates the process. Users can upload NLP system evaluation\nscores and the toolkit will analyze these scores, run appropriate significance\ntests, estimate effect size, and conduct power analysis to estimate Type II\nerror. The toolkit provides a convenient and systematic way to compare NLP\nsystem performance that goes beyond statistical significance testing",
            "journal ref": "Proceedings of the 1st Conference of the Asia-Pacific Chapter of\n  the Association for Computational Linguistics and the 10th International\n  Joint Conference on Natural Language Processing: System Demonstrations (2020)\n  40-46",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13238v1",
            "date": "2020-11-26 11:11:44+00:00",
            "title": "Towards Interpretable Multilingual Detection of Hate Speech against Immigrants and Women in Twitter at SemEval-2019 Task 5",
            "authors": [
                "Alvi Md Ishmam"
            ],
            "abstract": "his paper describes our techniques to detect hate speech against women and\nimmigrants on Twitter in multilingual contexts, particularly in English and\nSpanish. The challenge was designed by SemEval-2019 Task 5, where the\nparticipants need to design algorithms to detect hate speech in English and\nSpanish language with a given target (e.g., women or immigrants). Here, we have\ndeveloped two deep neural networks (Bidirectional Gated Recurrent Unit (GRU),\nCharacter-level Convolutional Neural Network (CNN)), and one machine learning\nmodel by exploiting the linguistic features. Our proposed model obtained 57 and\n75 F1 scores for Task A in English and Spanish language respectively. For Task\nB, the F1 scores are 67 for English and 75.33 for Spanish. In the case of task\nA (Spanish) and task B (both English and Spanish), the F1 scores are improved\nby 2, 10, and 5 points respectively. Besides, we present visually interpretable\nmodels that can address the generalizability issues of the custom-designed\nmachine learning architecture by investigating the annotated dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13284v1",
            "date": "2020-11-26 13:33:47+00:00",
            "title": "A question-answering system for aircraft pilots' documentation",
            "authors": [
                "Alexandre Arnold",
                "G\u00e9rard Dupont",
                "F\u00e9lix Furger",
                "Catherine Kobus",
                "Fran\u00e7ois Lancelot"
            ],
            "abstract": "The aerospace industry relies on massive collections of complex and technical\ndocuments covering system descriptions, manuals or procedures. This paper\npresents a question answering (QA) system that would help aircraft pilots\naccess information in this documentation by naturally interacting with the\nsystem and asking questions in natural language. After describing each module\nof the dialog system, we present a multi-task based approach for the QA module\nwhich enables performance improvement on a Flight Crew Operating Manual (FCOM)\ndataset. A method to combine scores from the retriever and the QA modules is\nalso presented.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13384v2",
            "date": "2020-11-26 16:52:48+00:00",
            "title": "Automatic coding of students' writing via Contrastive Representation Learning in the Wasserstein space",
            "authors": [
                "Ruijie Jiang",
                "Julia Gouvea",
                "David Hammer",
                "Eric Miller",
                "Shuchin Aeron"
            ],
            "abstract": "Qualitative analysis of verbal data is of central importance in the learning\nsciences. It is labor-intensive and time-consuming, however, which limits the\namount of data researchers can include in studies. This work is a step towards\nbuilding a statistical machine learning (ML) method for achieving an automated\nsupport for qualitative analyses of students' writing, here specifically in\nscore laboratory reports in introductory biology for sophistication of\nargumentation and reasoning. We start with a set of lab reports from an\nundergraduate biology course, scored by a four-level scheme that considers the\ncomplexity of argument structure, the scope of evidence, and the care and\nnuance of conclusions. Using this set of labeled data, we show that a popular\nnatural language modeling processing pipeline, namely vector representation of\nwords, a.k.a word embeddings, followed by Long Short Term Memory (LSTM) model\nfor capturing language generation as a state-space model, is able to\nquantitatively capture the scoring, with a high Quadratic Weighted Kappa (QWK)\nprediction score, when trained in via a novel contrastive learning set-up. We\nshow that the ML algorithm approached the inter-rater reliability of human\nanalysis. Ultimately, we conclude, that machine learning (ML) for natural\nlanguage processing (NLP) holds promise for assisting learning sciences\nresearchers in conducting qualitative studies at much larger scales than is\ncurrently possible.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2011.13470v1",
            "date": "2020-11-26 20:51:57+00:00",
            "title": "AutoNLU: An On-demand Cloud-based Natural Language Understanding System for Enterprises",
            "authors": [
                "Nham Le",
                "Tuan Lai",
                "Trung Bui",
                "Doo Soon Kim"
            ],
            "abstract": "With the renaissance of deep learning, neural networks have achieved\npromising results on many natural language understanding (NLU) tasks. Even\nthough the source codes of many neural network models are publicly available,\nthere is still a large gap from open-sourced models to solving real-world\nproblems in enterprises. Therefore, to fill this gap, we introduce AutoNLU, an\non-demand cloud-based system with an easy-to-use interface that covers all\ncommon use-cases and steps in developing an NLU model. AutoNLU has supported\nmany product teams within Adobe with different use-cases and datasets, quickly\ndelivering them working models. To demonstrate the effectiveness of AutoNLU, we\npresent two case studies. i) We build a practical NLU model for handling\nvarious image-editing requests in Photoshop. ii) We build powerful keyphrase\nextraction models that achieve state-of-the-art results on two public\nbenchmarks. In both cases, end users only need to write a small amount of code\nto convert their datasets into a common format used by AutoNLU.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13477v1",
            "date": "2020-11-26 21:09:38+00:00",
            "title": "Decoding and Diversity in Machine Translation",
            "authors": [
                "Nicholas Roberts",
                "Davis Liang",
                "Graham Neubig",
                "Zachary C. Lipton"
            ],
            "abstract": "Neural Machine Translation (NMT) systems are typically evaluated using\nautomated metrics that assess the agreement between generated translations and\nground truth candidates. To improve systems with respect to these metrics, NLP\nresearchers employ a variety of heuristic techniques, including searching for\nthe conditional mode (vs. sampling) and incorporating various training\nheuristics (e.g., label smoothing). While search strategies significantly\nimprove BLEU score, they yield deterministic outputs that lack the diversity of\nhuman translations. Moreover, search tends to bias the distribution of\ntranslated gender pronouns. This makes human-level BLEU a misleading benchmark\nin that modern MT systems cannot approach human-level BLEU while simultaneously\nmaintaining human-level translation diversity. In this paper, we characterize\ndistributional differences between generated and real translations, examining\nthe cost in diversity paid for the BLEU scores enjoyed by NMT. Moreover, our\nstudy implicates search as a salient source of known bias when translating\ngender pronouns.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13527v1",
            "date": "2020-11-27 02:26:15+00:00",
            "title": "TaylorGAN: Neighbor-Augmented Policy Update for Sample-Efficient Natural Language Generation",
            "authors": [
                "Chun-Hsing Lin",
                "Siang-Ruei Wu",
                "Hung-Yi Lee",
                "Yun-Nung Chen"
            ],
            "abstract": "Score function-based natural language generation (NLG) approaches such as\nREINFORCE, in general, suffer from low sample efficiency and training\ninstability problems. This is mainly due to the non-differentiable nature of\nthe discrete space sampling and thus these methods have to treat the\ndiscriminator as a black box and ignore the gradient information. To improve\nthe sample efficiency and reduce the variance of REINFORCE, we propose a novel\napproach, TaylorGAN, which augments the gradient estimation by off-policy\nupdate and the first-order Taylor expansion. This approach enables us to train\nNLG models from scratch with smaller batch size -- without maximum likelihood\npre-training, and outperforms existing GAN-based methods on multiple metrics of\nquality and diversity. The source code and data are available at\nhttps://github.com/MiuLab/TaylorGAN",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13565v1",
            "date": "2020-11-27 05:47:26+00:00",
            "title": "Joint Extraction of Entity and Relation with Information Redundancy Elimination",
            "authors": [
                "Yuanhao Shen",
                "Jungang Han"
            ],
            "abstract": "To solve the problem of redundant information and overlapping relations of\nthe entity and relation extraction model, we propose a joint extraction model.\nThis model can directly extract multiple pairs of related entities without\ngenerating unrelated redundant information. We also propose a recurrent neural\nnetwork named Encoder-LSTM that enhances the ability of recurrent units to\nmodel sentences. Specifically, the joint model includes three sub-modules: the\nNamed Entity Recognition sub-module consisted of a pre-trained language model\nand an LSTM decoder layer, the Entity Pair Extraction sub-module which uses\nEncoder-LSTM network to model the order relationship between related entity\npairs, and the Relation Classification sub-module including Attention\nmechanism. We conducted experiments on the public datasets ADE and CoNLL04 to\nevaluate the effectiveness of our model. The results show that the proposed\nmodel achieves good performance in the task of entity and relation extraction\nand can greatly reduce the amount of redundant information.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13574v1",
            "date": "2020-11-27 06:21:12+00:00",
            "title": "Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction",
            "authors": [
                "Yixin Cao",
                "Jun Kuang",
                "Ming Gao",
                "Aoying Zhou",
                "Yonggang Wen",
                "Tat-Seng Chua"
            ],
            "abstract": "Relation Extraction (RE) is a vital step to complete Knowledge Graph (KG) by\nextracting entity relations from texts.However, it usually suffers from the\nlong-tail issue. The training data mainly concentrates on a few types of\nrelations, leading to the lackof sufficient annotations for the remaining types\nof relations. In this paper, we propose a general approach to learn relation\nprototypesfrom unlabeled texts, to facilitate the long-tail relation extraction\nby transferring knowledge from the relation types with sufficient trainingdata.\nWe learn relation prototypes as an implicit factor between entities, which\nreflects the meanings of relations as well as theirproximities for transfer\nlearning. Specifically, we construct a co-occurrence graph from texts, and\ncapture both first-order andsecond-order entity proximities for embedding\nlearning. Based on this, we further optimize the distance from entity pairs\ntocorresponding prototypes, which can be easily adapted to almost arbitrary RE\nframeworks. Thus, the learning of infrequent or evenunseen relation types will\nbenefit from semantically proximate relations through pairs of entities and\nlarge-scale textual information.We have conducted extensive experiments on two\npublicly available datasets: New York Times and Google Distant\nSupervision.Compared with eight state-of-the-art baselines, our proposed model\nachieves significant improvements (4.1% F1 on average). Furtherresults on\nlong-tail relations demonstrate the effectiveness of the learned relation\nprototypes. We further conduct an ablation study toinvestigate the impacts of\nvarying components, and apply it to four basic relation extraction models to\nverify the generalization ability.Finally, we analyze several example cases to\ngive intuitive impressions as qualitative analysis. Our codes will be released\nlater.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.13978v2",
            "date": "2020-11-27 20:02:59+00:00",
            "title": "Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health",
            "authors": [
                "Denis Newman-Griffis",
                "Eric Fosler-Lussier"
            ],
            "abstract": "Linking clinical narratives to standardized vocabularies and coding systems\nis a key component of unlocking the information in medical text for analysis.\nHowever, many domains of medical concepts lack well-developed terminologies\nthat can support effective coding of medical text. We present a framework for\ndeveloping natural language processing (NLP) technologies for automated coding\nof under-studied types of medical information, and demonstrate its\napplicability via a case study on physical mobility function. Mobility is a\ncomponent of many health measures, from post-acute care and surgical outcomes\nto chronic frailty and disability, and is coded in the International\nClassification of Functioning, Disability, and Health (ICF). However, mobility\nand other types of functional activity remain under-studied in medical\ninformatics, and neither the ICF nor commonly-used medical terminologies\ncapture functional status terminology in practice. We investigated two\ndata-driven paradigms, classification and candidate selection, to link\nnarrative observations of mobility to standardized ICF codes, using a dataset\nof clinical narratives from physical therapy encounters. Recent advances in\nlanguage modeling and word embedding were used as features for established\nmachine learning models and a novel deep learning approach, achieving a macro\nF-1 score of 84% on linking mobility activity reports to ICF codes. Both\nclassification and candidate selection approaches present distinct strengths\nfor automated coding in under-studied domains, and we highlight that the\ncombination of (i) a small annotated data set; (ii) expert definitions of codes\nof interest; and (iii) a representative text corpus is sufficient to produce\nhigh-performing automated coding systems. This study has implications for the\nongoing growth of NLP tools for a variety of specialized applications in\nclinical care and research.",
            "journal ref": "Frontiers in Digital Health, 3:620828 (2021)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14084v2",
            "date": "2020-11-28 08:08:25+00:00",
            "title": "A Data-Driven Study of Commonsense Knowledge using the ConceptNet Knowledge Base",
            "authors": [
                "Ke Shen",
                "Mayank Kejriwal"
            ],
            "abstract": "Acquiring commonsense knowledge and reasoning is recognized as an important\nfrontier in achieving general Artificial Intelligence (AI). Recent research in\nthe Natural Language Processing (NLP) community has demonstrated significant\nprogress in this problem setting. Despite this progress, which is mainly on\nmultiple-choice question answering tasks in limited settings, there is still a\nlack of understanding (especially at scale) of the nature of commonsense\nknowledge itself. In this paper, we propose and conduct a systematic study to\nenable a deeper understanding of commonsense knowledge by doing an empirical\nand structural analysis of the ConceptNet knowledge base. ConceptNet is a\nfreely available knowledge base containing millions of commonsense assertions\npresented in natural language. Detailed experimental results on three carefully\ndesigned research questions, using state-of-the-art unsupervised graph\nrepresentation learning ('embedding') and clustering techniques, reveal deep\nsubstructures in ConceptNet relations, allowing us to make data-driven and\ncomputational claims about the meaning of phenomena such as 'context' that are\ntraditionally discussed only in qualitative terms. Furthermore, our methodology\nprovides a case study in how to use data-science and computational\nmethodologies for understanding the nature of an everyday (yet complex)\npsychological phenomenon that is an essential feature of human intelligence.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2011.14203v4",
            "date": "2020-11-28 19:21:47+00:00",
            "title": "EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference",
            "authors": [
                "Thierry Tambe",
                "Coleman Hooper",
                "Lillian Pentecost",
                "Tianyu Jia",
                "En-Yu Yang",
                "Marco Donato",
                "Victor Sanh",
                "Paul Whatmough",
                "Alexander M. Rush",
                "David Brooks",
                "Gu-Yeon Wei"
            ],
            "abstract": "Transformer-based language models such as BERT provide significant accuracy\nimprovement for a multitude of natural language processing (NLP) tasks.\nHowever, their hefty computational and memory demands make them challenging to\ndeploy to resource-constrained edge platforms with strict latency requirements.\nWe present EdgeBERT, an in-depth algorithm-hardware co-design for latency-aware\nenergy optimization for multi-task NLP. EdgeBERT employs entropy-based early\nexit predication in order to perform dynamic voltage-frequency scaling (DVFS),\nat a sentence granularity, for minimal energy consumption while adhering to a\nprescribed target latency. Computation and memory footprint overheads are\nfurther alleviated by employing a calibrated combination of adaptive attention\nspan, selective network pruning, and floating-point quantization. Furthermore,\nin order to maximize the synergistic benefits of these algorithms in always-on\nand intermediate edge computing settings, we specialize a 12nm scalable\nhardware accelerator system, integrating a fast-switching low-dropout voltage\nregulator (LDO), an all-digital phase-locked loop (ADPLL), as well as,\nhigh-density embedded non-volatile memories (eNVMs) wherein the sparse\nfloating-point bit encodings of the shared multi-task parameters are carefully\nstored. Altogether, latency-aware multi-task NLP inference acceleration on the\nEdgeBERT hardware system generates up to 7x, 2.5x, and 53x lower energy\ncompared to the conventional inference without early stopping, the\nlatency-unbounded early exit approach, and CUDA adaptations on an Nvidia Jetson\nTegra X2 mobile GPU, respectively.",
            "journal ref": "None",
            "category": "cs.AR"
        },
        {
            "url": "http://arxiv.org/abs/2011.14330v2",
            "date": "2020-11-29 10:04:38+00:00",
            "title": "A Boundary Regression Model for Nested Named Entity Recognition",
            "authors": [
                "Yanping Chen",
                "Lefei Wu",
                "Liyuan Deng",
                "Yongbin Qing",
                "Ruizhang Huang",
                "Qinghua Zheng",
                "Ping Chen"
            ],
            "abstract": "Recognizing named entities (NEs) is commonly conducted as a classification\nproblem that predicts a class tag for an NE candidate in a sentence. In shallow\nstructures, categorized features are weighted to support the prediction. Recent\ndevelopments in neural networks have adopted deep structures that map\ncategorized features into continuous representations. This approach unfolds a\ndense space saturated with high-order abstract semantic information, where the\nprediction is based on distributed feature representations. In this paper, the\nregression operation is introduced to locate NEs in a sentence. In this\napproach, a deep network is first designed to transform an input sentence into\nrecurrent feature maps. Bounding boxes are generated from the feature maps,\nwhere a box is an abstract representation of an NE candidate. In addition to\nthe class tag, each bounding box has two parameters denoting the start position\nand the length of an NE candidate. In the training process, the location offset\nbetween a bounding box and a true NE are learned to minimize the location loss.\nBased on this motivation, a multiobjective learning framework is designed to\nsimultaneously locate entities and predict the class probability. By sharing\nparameters for locating and predicting, the framework can take full advantage\nof annotated data and enable more potent nonlinear function approximators to\nenhance model discriminability. Experiments demonstrate state-of-the-art\nperformance for nested named entities\\footnote{Our codes will be available at:\n\\url{https://github.com/wuyuefei3/BR}}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.14752v1",
            "date": "2020-11-30 13:08:45+00:00",
            "title": "A Comprehensive Review on Recent Methods and Challenges of Video Description",
            "authors": [
                "Alok Singh",
                "Thoudam Doren Singh",
                "Sivaji Bandyopadhyay"
            ],
            "abstract": "Video description involves the generation of the natural language description\nof actions, events, and objects in the video. There are various applications of\nvideo description by filling the gap between languages and vision for visually\nimpaired people, generating automatic title suggestion based on content,\nbrowsing of the video based on the content and video-guided machine translation\n[86] etc.In the past decade, several works had been done in this field in terms\nof approaches/methods for video description, evaluation metrics,and datasets.\nFor analyzing the progress in the video description task, a comprehensive\nsurvey is needed that covers all the phases of video description approaches\nwith a special focus on recent deep learning approaches. In this work, we\nreport a comprehensive survey on the phases of video description approaches,\nthe dataset for video description, evaluation metrics, open competitions for\nmotivating the research on the video description, open challenges in this\nfield, and future research directions. In this survey, we cover the\nstate-of-the-art approaches proposed for each and every dataset with their pros\nand cons. For the growth of this research domain,the availability of numerous\nbenchmark dataset is a basic need. Further, we categorize all the dataset into\ntwo classes: open domain dataset and domain-specific dataset. From our survey,\nwe observe that the work in this field is in fast-paced development since the\ntask of video description falls in the intersection of computer vision and\nnatural language processing. But still, the work in the video description is\nfar from saturation stage due to various challenges like the redundancy due to\nsimilar frames which affect the quality of visual features, the availability of\ndataset containing more diverse content and availability of an effective\nevaluation metric.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2011.15023v2",
            "date": "2020-11-30 17:27:41+00:00",
            "title": "Transformer-Transducers for Code-Switched Speech Recognition",
            "authors": [
                "Siddharth Dalmia",
                "Yuzong Liu",
                "Srikanth Ronanki",
                "Katrin Kirchhoff"
            ],
            "abstract": "We live in a world where 60% of the population can speak two or more\nlanguages fluently. Members of these communities constantly switch between\nlanguages when having a conversation. As automatic speech recognition (ASR)\nsystems are being deployed to the real-world, there is a need for practical\nsystems that can handle multiple languages both within an utterance or across\nutterances. In this paper, we present an end-to-end ASR system using a\ntransformer-transducer model architecture for code-switched speech recognition.\nWe propose three modifications over the vanilla model in order to handle\nvarious aspects of code-switching. First, we introduce two auxiliary loss\nfunctions to handle the low-resource scenario of code-switching. Second, we\npropose a novel mask-based training strategy with language ID information to\nimprove the label encoder training towards intra-sentential code-switching.\nFinally, we propose a multi-label/multi-audio encoder structure to leverage the\nvast monolingual speech corpora towards code-switching. We demonstrate the\nefficacy of our proposed approaches on the SEAME dataset, a public\nMandarin-English code-switching corpus, achieving a mixed error rate of 18.5%\nand 26.3% on test_man and test_sge sets respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2011.15124v2",
            "date": "2020-11-30 18:55:24+00:00",
            "title": "Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs",
            "authors": [
                "Emanuele Bugliarello",
                "Ryan Cotterell",
                "Naoaki Okazaki",
                "Desmond Elliott"
            ],
            "abstract": "Large-scale pretraining and task-specific fine-tuning is now the standard\nmethodology for many tasks in computer vision and natural language processing.\nRecently, a multitude of methods have been proposed for pretraining vision and\nlanguage BERTs to tackle challenges at the intersection of these two key areas\nof AI. These models can be categorised into either single-stream or dual-stream\nencoders. We study the differences between these two categories, and show how\nthey can be unified under a single theoretical framework. We then conduct\ncontrolled experiments to discern the empirical differences between five V&L\nBERTs. Our experiments show that training data and hyperparameters are\nresponsible for most of the differences between the reported results, but they\nalso reveal that the embedding layer plays a crucial role in these massive\nmodels.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00096v1",
            "date": "2020-11-30 21:18:17+00:00",
            "title": "Multi-Modal Detection of Alzheimer's Disease from Speech and Text",
            "authors": [
                "Amish Mittal",
                "Sourav Sahoo",
                "Arnhav Datar",
                "Juned Kadiwala",
                "Hrithwik Shalu",
                "Jimson Mathew"
            ],
            "abstract": "Reliable detection of the prodromal stages of Alzheimer's disease (AD)\nremains difficult even today because, unlike other neurocognitive impairments,\nthere is no definitive diagnosis of AD in vivo. In this context, existing\nresearch has shown that patients often develop language impairment even in mild\nAD conditions. We propose a multimodal deep learning method that utilizes\nspeech and the corresponding transcript simultaneously to detect AD. For audio\nsignals, the proposed audio-based network, a convolutional neural network (CNN)\nbased model, predicts the diagnosis for multiple speech segments, which are\ncombined for the final prediction. Similarly, we use contextual embedding\nextracted from BERT concatenated with a CNN-generated embedding for classifying\nthe transcript. The individual predictions of the two models are then combined\nto make the final classification. We also perform experiments to analyze the\nmodel performance when Automated Speech Recognition (ASR) system generated\ntranscripts are used instead of manual transcription in the text-based model.\nThe proposed method achieves 85.3% 10-fold cross-validation accuracy when\ntrained and evaluated on the Dementiabank Pitt corpus.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.00187v1",
            "date": "2020-12-01 00:48:27+00:00",
            "title": "Statistical patterns of word frequency suggesting the probabilistic nature of human languages",
            "authors": [
                "Shuiyuan Yu",
                "Chunshan Xu",
                "Haitao Liu"
            ],
            "abstract": "Traditional linguistic theories have largely regard language as a formal\nsystem composed of rigid rules. However, their failures in processing real\nlanguage, the recent successes in statistical natural language processing, and\nthe findings of many psychological experiments have suggested that language may\nbe more a probabilistic system than a formal system, and thus cannot be\nfaithfully modeled with the either/or rules of formal linguistic theory. The\npresent study, based on authentic language data, confirmed that those important\nlinguistic issues, such as linguistic universal, diachronic drift, and language\nvariations can be translated into probability and frequency patterns in parole.\nThese findings suggest that human language may well be probabilistic systems by\nnature, and that statistical may well make inherent properties of human\nlanguages.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00363v1",
            "date": "2020-12-01 09:39:13+00:00",
            "title": "Modifying Memories in Transformer Models",
            "authors": [
                "Chen Zhu",
                "Ankit Singh Rawat",
                "Manzil Zaheer",
                "Srinadh Bhojanapalli",
                "Daliang Li",
                "Felix Yu",
                "Sanjiv Kumar"
            ],
            "abstract": "Large Transformer models have achieved impressive performance in many natural\nlanguage tasks. In particular, Transformer based language models have been\nshown to have great capabilities in encoding factual knowledge in their vast\namount of parameters. While the tasks of improving the memorization and\ngeneralization of Transformers have been widely studied, it is not well known\nhow to make transformers forget specific old facts and memorize new ones. In\nthis paper, we propose a new task of \\emph{explicitly modifying specific\nfactual knowledge in Transformer models while ensuring the model performance\ndoes not degrade on the unmodified facts}. This task is useful in many\nscenarios, such as updating stale knowledge, protecting privacy, and\neliminating unintended biases stored in the models. We benchmarked several\napproaches that provide natural baseline performances on this task. This leads\nto the discovery of key components of a Transformer model that are especially\neffective for knowledge modifications. The work also provides insights into the\nrole that different training phases (such as pretraining and fine-tuning) play\ntowards memorization and knowledge modification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00483v2",
            "date": "2020-12-01 13:42:37+00:00",
            "title": "ClimaText: A Dataset for Climate Change Topic Detection",
            "authors": [
                "Francesco S. Varini",
                "Jordan Boyd-Graber",
                "Massimiliano Ciaramita",
                "Markus Leippold"
            ],
            "abstract": "Climate change communication in the mass media and other textual sources may\naffect and shape public perception. Extracting climate change information from\nthese sources is an important task, e.g., for filtering content and\ne-discovery, sentiment analysis, automatic summarization, question-answering,\nand fact-checking. However, automating this process is a challenge, as climate\nchange is a complex, fast-moving, and often ambiguous topic with scarce\nresources for popular text-based AI tasks. In this paper, we introduce\n\\textsc{ClimaText}, a dataset for sentence-based climate change topic\ndetection, which we make publicly available. We explore different approaches to\nidentify the climate change topic in various text sources. We find that popular\nkeyword-based models are not adequate for such a complex and evolving task.\nContext-based algorithms like BERT \\cite{devlin2018bert} can detect, in\naddition to many trivial cases, a variety of complex and implicit topic\npatterns. Nevertheless, our analysis reveals a great potential for improvement\nin several directions, such as, e.g., capturing the discussion on indirect\neffects of climate change. Hence, we hope this work can serve as a good\nstarting point for further research on this topic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00584v1",
            "date": "2020-12-01 15:53:44+00:00",
            "title": "Neural language models for text classification in evidence-based medicine",
            "authors": [
                "Andres Carvallo",
                "Denis Parra",
                "Gabriel Rada",
                "Daniel Perez",
                "Juan Ignacio Vasquez",
                "Camilo Vergara"
            ],
            "abstract": "The COVID-19 has brought about a significant challenge to the whole of\nhumanity, but with a special burden upon the medical community. Clinicians must\nkeep updated continuously about symptoms, diagnoses, and effectiveness of\nemergent treatments under a never-ending flood of scientific literature. In\nthis context, the role of evidence-based medicine (EBM) for curating the most\nsubstantial evidence to support public health and clinical practice turns\nessential but is being challenged as never before due to the high volume of\nresearch articles published and pre-prints posted daily. Artificial\nIntelligence can have a crucial role in this situation. In this article, we\nreport the results of an applied research project to classify scientific\narticles to support Epistemonikos, one of the most active foundations worldwide\nconducting EBM. We test several methods, and the best one, based on the XLNet\nneural language model, improves the current approach by 93\\% on average\nF1-score, saving valuable time from physicians who volunteer to curate COVID-19\nresearch articles manually.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00614v2",
            "date": "2020-12-01 16:32:54+00:00",
            "title": "CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims",
            "authors": [
                "Thomas Diggelmann",
                "Jordan Boyd-Graber",
                "Jannis Bulian",
                "Massimiliano Ciaramita",
                "Markus Leippold"
            ],
            "abstract": "We introduce CLIMATE-FEVER, a new publicly available dataset for verification\nof climate change-related claims. By providing a dataset for the research\ncommunity, we aim to facilitate and encourage work on improving algorithms for\nretrieving evidential support for climate-specific claims, addressing the\nunderlying language understanding challenges, and ultimately help alleviate the\nimpact of misinformation on climate change. We adapt the methodology of FEVER\n[1], the largest dataset of artificially designed claims, to real-life claims\ncollected from the Internet. While during this process, we could rely on the\nexpertise of renowned climate scientists, it turned out to be no easy task. We\ndiscuss the surprising, subtle complexity of modeling real-world\nclimate-related claims within the \\textsc{fever} framework, which we believe\nprovides a valuable challenge for general natural language understanding. We\nhope that our work will mark the beginning of a new exciting long-term joint\neffort by the climate science and AI community.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00876v1",
            "date": "2020-12-01 22:44:42+00:00",
            "title": "Automatically Identifying Language Family from Acoustic Examples in Low Resource Scenarios",
            "authors": [
                "Peter Wu",
                "Yifan Zhong",
                "Alan W Black"
            ],
            "abstract": "Existing multilingual speech NLP works focus on a relatively small subset of\nlanguages, and thus current linguistic understanding of languages predominantly\nstems from classical approaches. In this work, we propose a method to analyze\nlanguage similarity using deep learning. Namely, we train a model on the\nWilderness dataset and investigate how its latent space compares with classical\nlanguage family findings. Our approach provides a new direction for\ncross-lingual data augmentation in any speech-based NLP task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00893v1",
            "date": "2020-12-01 23:40:21+00:00",
            "title": "Evaluating Explanations: How much do explanations from the teacher aid students?",
            "authors": [
                "Danish Pruthi",
                "Bhuwan Dhingra",
                "Livio Baldini Soares",
                "Michael Collins",
                "Zachary C. Lipton",
                "Graham Neubig",
                "William W. Cohen"
            ],
            "abstract": "While many methods purport to explain predictions by highlighting salient\nfeatures, what precise aims these explanations serve and how to evaluate their\nutility are often unstated. In this work, we formalize the value of\nexplanations using a student-teacher paradigm that measures the extent to which\nexplanations improve student models in learning to simulate the teacher model\non unseen examples for which explanations are unavailable. Student models\nincorporate explanations in training (but not prediction) procedures. Unlike\nmany prior proposals to evaluate explanations, our approach cannot be easily\ngamed, enabling principled, scalable, and automatic evaluation of attributions.\nUsing our framework, we compare multiple attribution methods and observe\nconsistent and quantitative differences amongst them across multiple learning\nstrategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00898v1",
            "date": "2020-12-01 23:54:41+00:00",
            "title": "Federated Marginal Personalization for ASR Rescoring",
            "authors": [
                "Zhe Liu",
                "Fuchun Peng"
            ],
            "abstract": "We introduce federated marginal personalization (FMP), a novel method for\ncontinuously updating personalized neural network language models (NNLMs) on\nprivate devices using federated learning (FL). Instead of fine-tuning the\nparameters of NNLMs on personal data, FMP regularly estimates global and\npersonalized marginal distributions of words, and adjusts the probabilities\nfrom NNLMs by an adaptation factor that is specific to each word. Our presented\napproach can overcome the limitations of federated fine-tuning and efficiently\nlearn personalized NNLMs on devices. We study the application of FMP on\nsecond-pass ASR rescoring tasks. Experiments on two speech evaluation datasets\nshow modest word error rate (WER) reductions. We also demonstrate that FMP\ncould offer reasonable privacy with only a negligible cost in speech\nrecognition accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.00974v2",
            "date": "2020-12-02 05:25:02+00:00",
            "title": "Extracting COVID-19 Diagnoses and Symptoms From Clinical Text: A New Annotated Corpus and Neural Event Extraction Framework",
            "authors": [
                "Kevin Lybarger",
                "Mari Ostendorf",
                "Matthew Thompson",
                "Meliha Yetisgen"
            ],
            "abstract": "Coronavirus disease 2019 (COVID-19) is a global pandemic. Although much has\nbeen learned about the novel coronavirus since its emergence, there are many\nopen questions related to tracking its spread, describing symptomology,\npredicting the severity of infection, and forecasting healthcare utilization.\nFree-text clinical notes contain critical information for resolving these\nquestions. Data-driven, automatic information extraction models are needed to\nuse this text-encoded information in large-scale studies. This work presents a\nnew clinical corpus, referred to as the COVID-19 Annotated Clinical Text (CACT)\nCorpus, which comprises 1,472 notes with detailed annotations characterizing\nCOVID-19 diagnoses, testing, and clinical presentation. We introduce a\nspan-based event extraction model that jointly extracts all annotated\nphenomena, achieving high performance in identifying COVID-19 and symptom\nevents with associated assertion values (0.83-0.97 F1 for events and 0.73-0.79\nF1 for assertions). In a secondary use application, we explored the prediction\nof COVID-19 test results using structured patient data (e.g. vital signs and\nlaboratory results) and automatically extracted symptom information. The\nautomatically extracted symptoms improve prediction performance, beyond\nstructured data alone.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01002v1",
            "date": "2020-12-02 07:38:26+00:00",
            "title": "Classification of Multimodal Hate Speech -- The Winning Solution of Hateful Memes Challenge",
            "authors": [
                "Xiayu Zhong"
            ],
            "abstract": "Hateful Memes is a new challenge set for multimodal classification, focusing\non detecting hate speech in multimodal memes. Difficult examples are added to\nthe dataset to make it hard to rely on unimodal signals, which means only\nmultimodal models can succeed. According to Kiela,the state-of-the-art methods\nperform poorly compared to humans (64.73% vs. 84.7% accuracy) on Hateful Memes.\nI propose a new model that combined multimodal with rules, which achieve the\nfirst ranking of accuracy and AUROC of 86.8% and 0.923 respectively. These\nrules are extracted from training set, and focus on improving the\nclassification accuracy of difficult samples.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01031v1",
            "date": "2020-12-02 08:55:07+00:00",
            "title": "Biomedical Knowledge Graph Refinement with Embedding and Logic Rules",
            "authors": [
                "Sendong Zhao",
                "Bing Qin",
                "Ting Liu",
                "Fei Wang"
            ],
            "abstract": "Currently, there is a rapidly increasing need for high-quality biomedical\nknowledge graphs (BioKG) that provide direct and precise biomedical knowledge.\nIn the context of COVID-19, this issue is even more necessary to be\nhighlighted. However, most BioKG construction inevitably includes numerous\nconflicts and noises deriving from incorrect knowledge descriptions in\nliterature and defective information extraction techniques. Many studies have\ndemonstrated that reasoning upon the knowledge graph is effective in\neliminating such conflicts and noises. This paper proposes a method BioGRER to\nimprove the BioKG's quality, which comprehensively combines the knowledge graph\nembedding and logic rules that support and negate triplets in the BioKG. In the\nproposed model, the BioKG refinement problem is formulated as the probability\nestimation for triplets in the BioKG. We employ the variational EM algorithm to\noptimize knowledge graph embedding and logic rule inference alternately. In\nthis way, our model could combine efforts from both the knowledge graph\nembedding and logic rules, leading to better results than using them alone. We\nevaluate our model over a COVID-19 knowledge graph and obtain competitive\nresults.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.01133v1",
            "date": "2020-11-16 20:47:54+00:00",
            "title": "It's a Thin Line Between Love and Hate: Using the Echo in Modeling Dynamics of Racist Online Communities",
            "authors": [
                "Eyal Arviv",
                "Simo Hanouna",
                "Oren Tsur"
            ],
            "abstract": "The (((echo))) symbol -- triple parenthesis surrounding a name, made it to\nmainstream social networks in early 2016, with the intensification of the U.S.\nPresidential race. It was used by members of the alt-right, white supremacists\nand internet trolls to tag people of Jewish heritage -- a modern incarnation of\nthe infamous yellow badge (Judenstern) used in Nazi-Germany. Tracking this\ntrending meme, its meaning, and its function has proved elusive for its\nsemantic ambiguity (e.g., a symbol for a virtual hug).\n  In this paper we report of the construction of an appropriate dataset\nallowing the reconstruction of networks of racist communities and the way they\nare embedded in the broader community. We combine natural language processing\nand structural network analysis to study communities promoting hate. In order\nto overcome dog-whistling and linguistic ambiguity, we propose a multi-modal\nneural architecture based on a BERT transformer and a BiLSTM network on the\ntweet level, while also taking into account the users ego-network and meta\nfeatures. Our multi-modal neural architecture outperforms a set of strong\nbaselines. We further show how the the use of language and network structure in\ntandem allows the detection of the leaders of the hate communities. We further\nstudy the ``intersectionality'' of hate and show that the antisemitic echo\ncorrelates with hate speech that targets other minority and protected groups.\nFinally, we analyze the role IRA trolls assumed in this network as part of the\nRussian interference campaign. Our findings allow a better understanding of\nrecent manifestations of racism and the dynamics that facilitate it.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01295v1",
            "date": "2020-12-02 16:07:32+00:00",
            "title": "Generating Descriptions for Sequential Images with Local-Object Attention and Global Semantic Context Modelling",
            "authors": [
                "Jing Su",
                "Chenghua Lin",
                "Mian Zhou",
                "Qingyun Dai",
                "Haoyu Lv"
            ],
            "abstract": "In this paper, we propose an end-to-end CNN-LSTM model for generating\ndescriptions for sequential images with a local-object attention mechanism. To\ngenerate coherent descriptions, we capture global semantic context using a\nmulti-layer perceptron, which learns the dependencies between sequential\nimages. A paralleled LSTM network is exploited for decoding the sequence\ndescriptions. Experimental results show that our model outperforms the baseline\nacross three different evaluation metrics on the datasets published by\nMicrosoft.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01300v1",
            "date": "2020-12-02 16:10:54+00:00",
            "title": "Learning from others' mistakes: Avoiding dataset biases without modeling them",
            "authors": [
                "Victor Sanh",
                "Thomas Wolf",
                "Yonatan Belinkov",
                "Alexander M. Rush"
            ],
            "abstract": "State-of-the-art natural language processing (NLP) models often learn to\nmodel dataset biases and surface form correlations instead of features that\ntarget the intended underlying task. Previous work has demonstrated effective\nmethods to circumvent these issues when knowledge of the bias is available. We\nconsider cases where the bias issues may not be explicitly identified, and show\na method for training models that learn to ignore these problematic\ncorrelations. Our approach relies on the observation that models with limited\ncapacity primarily learn to exploit biases in the dataset. We can leverage the\nerrors of such limited capacity models to train a more robust model in a\nproduct of experts, thus bypassing the need to hand-craft a biased model. We\nshow the effectiveness of this method to retain improvements in\nout-of-distribution settings even if no particular bias is targeted by the\nbiased model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01462v3",
            "date": "2020-12-02 19:05:25+00:00",
            "title": "ArCorona: Analyzing Arabic Tweets in the Early Days of Coronavirus (COVID-19) Pandemic",
            "authors": [
                "Hamdy Mubarak",
                "Sabit Hassan"
            ],
            "abstract": "Over the past few months, there were huge numbers of circulating tweets and\ndiscussions about Coronavirus (COVID-19) in the Arab region. It is important\nfor policy makers and many people to identify types of shared tweets to better\nunderstand public behavior, topics of interest, requests from governments,\nsources of tweets, etc. It is also crucial to prevent spreading of rumors and\nmisinformation about the virus or bad cures. To this end, we present the\nlargest manually annotated dataset of Arabic tweets related to COVID-19. We\ndescribe annotation guidelines, analyze our dataset and build effective machine\nlearning and transformer based models for classification.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01603v1",
            "date": "2020-12-02 23:56:34+00:00",
            "title": "SChME at SemEval-2020 Task 1: A Model Ensemble for Detecting Lexical Semantic Change",
            "authors": [
                "Maur\u00edcio Gruppi",
                "Sibel Adali",
                "Pin-Yu Chen"
            ],
            "abstract": "This paper describes SChME (Semantic Change Detection with Model Ensemble), a\nmethod usedin SemEval-2020 Task 1 on unsupervised detection of lexical semantic\nchange. SChME usesa model ensemble combining signals of distributional models\n(word embeddings) and wordfrequency models where each model casts a vote\nindicating the probability that a word sufferedsemantic change according to\nthat feature. More specifically, we combine cosine distance of wordvectors\ncombined with a neighborhood-based metric we named Mapped Neighborhood\nDistance(MAP), and a word frequency differential metric as input signals to our\nmodel. Additionally,we explore alignment-based methods to investigate the\nimportance of the landmarks used in thisprocess. Our results show evidence that\nthe number of landmarks used for alignment has a directimpact on the predictive\nperformance of the model. Moreover, we show that languages that sufferless\nsemantic change tend to benefit from using a large number of landmarks, whereas\nlanguageswith more semantic change benefit from a more careful choice of\nlandmark number for alignment.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01631v1",
            "date": "2020-12-03 01:48:37+00:00",
            "title": "Circles are like Ellipses, or Ellipses are like Circles? Measuring the Degree of Asymmetry of Static and Contextual Embeddings and the Implications to Representation Learning",
            "authors": [
                "Wei Zhang",
                "Murray Campbell",
                "Yang Yu",
                "Sadhana Kumaravel"
            ],
            "abstract": "Human judgments of word similarity have been a popular method of evaluating\nthe quality of word embedding. But it fails to measure the geometry properties\nsuch as asymmetry. For example, it is more natural to say \"Ellipses are like\nCircles\" than \"Circles are like Ellipses\". Such asymmetry has been observed\nfrom a psychoanalysis test called word evocation experiment, where one word is\nused to recall another. Although useful, such experimental data have been\nsignificantly understudied for measuring embedding quality. In this paper, we\nuse three well-known evocation datasets to gain insights into asymmetry\nencoding of embedding. We study both static embedding as well as contextual\nembedding, such as BERT. Evaluating asymmetry for BERT is generally hard due to\nthe dynamic nature of embedding. Thus, we probe BERT's conditional\nprobabilities (as a language model) using a large number of Wikipedia contexts\nto derive a theoretically justifiable Bayesian asymmetry score. The result\nshows that contextual embedding shows randomness than static embedding on\nsimilarity judgments while performing well on asymmetry judgment, which aligns\nwith its strong performance on \"extrinsic evaluations\" such as text\nclassification. The asymmetry judgment and the Bayesian approach provides a new\nperspective to evaluate contextual embedding on intrinsic evaluation, and its\ncomparison to similarity evaluation concludes our work with a discussion on the\ncurrent state and the future of representation learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01707v2",
            "date": "2020-12-03 05:17:55+00:00",
            "title": "Leveraging Abstract Meaning Representation for Knowledge Base Question Answering",
            "authors": [
                "Pavan Kapanipathi",
                "Ibrahim Abdelaziz",
                "Srinivas Ravishankar",
                "Salim Roukos",
                "Alexander Gray",
                "Ramon Astudillo",
                "Maria Chang",
                "Cristina Cornelio",
                "Saswati Dana",
                "Achille Fokoue",
                "Dinesh Garg",
                "Alfio Gliozzo",
                "Sairam Gurajada",
                "Hima Karanam",
                "Naweed Khan",
                "Dinesh Khandelwal",
                "Young-Suk Lee",
                "Yunyao Li",
                "Francois Luus",
                "Ndivhuwo Makondo",
                "Nandana Mihindukulasooriya",
                "Tahira Naseem",
                "Sumit Neelam",
                "Lucian Popa",
                "Revanth Reddy",
                "Ryan Riegel",
                "Gaetano Rossiello",
                "Udit Sharma",
                "G P Shrivatsa Bhargav",
                "Mo Yu"
            ],
            "abstract": "Knowledge base question answering (KBQA)is an important task in Natural\nLanguage Processing. Existing approaches face significant challenges including\ncomplex question understanding, necessity for reasoning, and lack of large\nend-to-end training datasets. In this work, we propose Neuro-Symbolic Question\nAnswering (NSQA), a modular KBQA system, that leverages (1) Abstract Meaning\nRepresentation (AMR) parses for task-independent question understanding; (2) a\nsimple yet effective graph transformation approach to convert AMR parses into\ncandidate logical queries that are aligned to the KB; (3) a pipeline-based\napproach which integrates multiple, reusable modules that are trained\nspecifically for their individual tasks (semantic parser, entity\nandrelationship linkers, and neuro-symbolic reasoner) and do not require\nend-to-end training data. NSQA achieves state-of-the-art performance on two\nprominent KBQA datasets based on DBpedia (QALD-9 and LC-QuAD1.0). Furthermore,\nour analysis emphasizes that AMR is a powerful tool for KBQA systems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01721v1",
            "date": "2020-12-03 06:41:09+00:00",
            "title": "Learning Disentangled Intent Representations for Zero-shot Intent Detection",
            "authors": [
                "Qingyi Si",
                "Yuanxin Liu",
                "Peng Fu",
                "Jiangnan Li",
                "Zheng Lin",
                "Weiping Wang"
            ],
            "abstract": "Zero-shot intent detection (ZSID) aims to deal with the continuously emerging\nintents without annotated training data. However, existing ZSID systems suffer\nfrom two limitations: 1) They are not good at modeling the relationship between\nseen and unseen intents, when the label names are given in the form of raw\nphrases or sentences. 2) They cannot effectively recognize unseen intents under\nthe generalized intent detection (GZSID) setting. A critical factor behind\nthese limitations is the representations of unseen intents, which cannot be\nlearned in the training stage. To address this problem, we propose a\nclass-transductive framework that utilizes unseen class labels to learn\nDisentangled Intent Representations (DIR). Specifically, we allow the model to\npredict unseen intents in the training stage, with the corresponding label\nnames serving as input utterances. Under this framework, we introduce a\nmulti-task learning objective, which encourages the model to learn the\ndistinctions among intents, and a similarity scorer, which estimates the\nconnections among intents more accurately based on the learned intent\nrepresentations. Since the purpose of DIR is to provide better intent\nrepresentations, it can be easily integrated with existing ZSID and GZSID\nmethods. Experiments on two real-world datasets show that the proposed\nframework brings consistent improvement to the baseline systems, regardless of\nthe model architectures or zero-shot learning strategies.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01747v1",
            "date": "2020-12-03 08:17:31+00:00",
            "title": "Bengali Abstractive News Summarization(BANS): A Neural Attention Approach",
            "authors": [
                "Prithwiraj Bhattacharjee",
                "Avi Mallick",
                "Md Saiful Islam",
                "Marium-E-Jannat"
            ],
            "abstract": "Abstractive summarization is the process of generating novel sentences based\non the information extracted from the original text document while retaining\nthe context. Due to abstractive summarization's underlying complexities, most\nof the past research work has been done on the extractive summarization\napproach. Nevertheless, with the triumph of the sequence-to-sequence (seq2seq)\nmodel, abstractive summarization becomes more viable. Although a significant\nnumber of notable research has been done in the English language based on\nabstractive summarization, only a couple of works have been done on Bengali\nabstractive news summarization (BANS). In this article, we presented a seq2seq\nbased Long Short-Term Memory (LSTM) network model with attention at\nencoder-decoder. Our proposed system deploys a local attention-based model that\nproduces a long sequence of words with lucid and human-like generated sentences\nwith noteworthy information of the original document. We also prepared a\ndataset of more than 19k articles and corresponding human-written summaries\ncollected from bangla.bdnews24.com1 which is till now the most extensive\ndataset for Bengali news document summarization and publicly published in\nKaggle2. We evaluated our model qualitatively and quantitatively and compared\nit with other published results. It showed significant improvement in terms of\nhuman evaluation scores with state-of-the-art approaches for BANS.",
            "journal ref": "2nd International Conference on Trends in Computational and\n  Cognitive Engineering, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01815v1",
            "date": "2020-12-03 10:42:56+00:00",
            "title": "SemMT: A Semantic-based Testing Approach for Machine Translation Systems",
            "authors": [
                "Jialun Cao",
                "Meiziniu Li",
                "Yeting Li",
                "Ming Wen",
                "Shing-Chi Cheung"
            ],
            "abstract": "Machine translation has wide applications in daily life. In mission-critical\napplications such as translating official documents, incorrect translation can\nhave unpleasant or sometimes catastrophic consequences. This motivates recent\nresearch on testing methodologies for machine translation systems. Existing\nmethodologies mostly rely on metamorphic relations designed at the textual\nlevel (e.g., Levenshtein distance) or syntactic level (e.g., the distance\nbetween grammar structures) to determine the correctness of translation\nresults. However, these metamorphic relations do not consider whether the\noriginal and translated sentences have the same meaning (i.e., Semantic\nsimilarity). Therefore, in this paper, we propose SemMT, an automatic testing\napproach for machine translation systems based on semantic similarity checking.\nSemMT applies round-trip translation and measures the semantic similarity\nbetween the original and translated sentences. Our insight is that the\nsemantics expressed by the logic and numeric constraint in sentences can be\ncaptured using regular expressions (or deterministic finite automata) where\nefficient equivalence/similarity checking algorithms are available. Leveraging\nthe insight, we propose three semantic similarity metrics and implement them in\nSemMT. The experiment result reveals SemMT can achieve higher effectiveness\ncompared with state-of-the-art works, achieving an increase of 21% and 23% on\naccuracy and F-Score, respectively. We also explore potential improvements that\ncan be achieved when proper combinations of metrics are adopted. Finally, we\ndiscuss a solution to locate the suspicious trip in round-trip translation,\nwhich may shed lights on further exploration.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2012.01936v1",
            "date": "2020-12-03 14:14:30+00:00",
            "title": "CUT: Controllable Unsupervised Text Simplification",
            "authors": [
                "Oleg Kariuk",
                "Dima Karamshuk"
            ],
            "abstract": "In this paper, we focus on the challenge of learning controllable text\nsimplifications in unsupervised settings. While this problem has been\npreviously discussed for supervised learning algorithms, the literature on the\nanalogies in unsupervised methods is scarse. We propose two unsupervised\nmechanisms for controlling the output complexity of the generated texts,\nnamely, back translation with control tokens (a learning-based approach) and\nsimplicity-aware beam search (decoding-based approach). We show that by nudging\na back-translation algorithm to understand the relative simplicity of a text in\ncomparison to its noisy translation, the algorithm self-supervises itself to\nproduce the output of the desired complexity. This approach achieves\ncompetitive performance on well-established benchmarks: SARI score of 46.88%\nand FKGL of 3.65% on the Newsela dataset.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.01941v1",
            "date": "2020-12-02 17:35:49+00:00",
            "title": "On Extending NLP Techniques from the Categorical to the Latent Space: KL Divergence, Zipf's Law, and Similarity Search",
            "authors": [
                "Adam Hare",
                "Yu Chen",
                "Yinan Liu",
                "Zhenming Liu",
                "Christopher G. Brinton"
            ],
            "abstract": "Despite the recent successes of deep learning in natural language processing\n(NLP), there remains widespread usage of and demand for techniques that do not\nrely on machine learning. The advantage of these techniques is their\ninterpretability and low cost when compared to frequently opaque and expensive\nmachine learning models. Although they may not be be as performant in all\ncases, they are often sufficient for common and relatively simple problems. In\nthis paper, we aim to modernize these older methods while retaining their\nadvantages by extending approaches from categorical or bag-of-words\nrepresentations to word embeddings representations in the latent space. First,\nwe show that entropy and Kullback-Leibler divergence can be efficiently\nestimated using word embeddings and use this estimation to compare text across\nseveral categories. Next, we recast the heavy-tailed distribution known as\nZipf's law that is frequently observed in the categorical space to the latent\nspace. Finally, we look to improve the Jaccard similarity measure for sentence\nsuggestion by introducing a new method of identifying similar sentences based\non the set cover problem. We compare the performance of this algorithm against\nseveral baselines including Word Mover's Distance and the Levenshtein distance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02029v2",
            "date": "2020-11-19 02:24:11+00:00",
            "title": "Predicting Early Indicators of Cognitive Decline from Verbal Utterances",
            "authors": [
                "Swati Padhee",
                "Anurag Illendula",
                "Megan Sadler",
                "Valerie L. Shalin",
                "Tanvi Banerjee",
                "Krishnaprasad Thirunarayan",
                "William L. Romine"
            ],
            "abstract": "Dementia is a group of irreversible, chronic, and progressive\nneurodegenerative disorders resulting in impaired memory, communication, and\nthought processes. In recent years, clinical research advances in brain aging\nhave focused on the earliest clinically detectable stage of incipient dementia,\ncommonly known as mild cognitive impairment (MCI). Currently, these disorders\nare diagnosed using a manual analysis of neuropsychological examinations. We\nmeasure the feasibility of using the linguistic characteristics of verbal\nutterances elicited during neuropsychological exams of elderly subjects to\ndistinguish between elderly control groups, people with MCI, people diagnosed\nwith possible Alzheimer's disease (AD), and probable AD. We investigated the\nperformance of both theory-driven psycholinguistic features and data-driven\ncontextual language embeddings in identifying different clinically diagnosed\ngroups. Our experiments show that a combination of contextual and\npsycholinguistic features extracted by a Support Vector Machine improved\ndistinguishing the verbal utterances of elderly controls, people with MCI,\npossible AD, and probable AD. This is the first work to identify four clinical\ndiagnosis groups of dementia in a highly imbalanced dataset. Our work shows\nthat machine learning algorithms built on contextual and psycholinguistic\nfeatures can learn the linguistic biomarkers from verbal utterances and assist\nclinical diagnosis of different stages and types of dementia, even with limited\ndata.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02110v1",
            "date": "2020-12-03 17:45:03+00:00",
            "title": "GottBERT: a pure German Language Model",
            "authors": [
                "Raphael Scheible",
                "Fabian Thomczyk",
                "Patric Tippmann",
                "Victor Jaravine",
                "Martin Boeker"
            ],
            "abstract": "Lately, pre-trained language models advanced the field of natural language\nprocessing (NLP). The introduction of Bidirectional Encoders for Transformers\n(BERT) and its optimized version RoBERTa have had significant impact and\nincreased the relevance of pre-trained models. First, research in this field\nmainly started on English data followed by models trained with multilingual\ntext corpora. However, current research shows that multilingual models are\ninferior to monolingual models. Currently, no German single language RoBERTa\nmodel is yet published, which we introduce in this work (GottBERT). The German\nportion of the OSCAR data set was used as text corpus. In an evaluation we\ncompare its performance on the two Named Entity Recognition (NER) tasks Conll\n2003 and GermEval 2014 as well as on the text classification tasks GermEval\n2018 (fine and coarse) and GNAD with existing German single language BERT\nmodels and two multilingual ones. GottBERT was pre-trained related to the\noriginal RoBERTa model using fairseq. All downstream tasks were trained using\nhyperparameter presets taken from the benchmark of German BERT. The experiments\nwere setup utilizing FARM. Performance was measured by the $F_{1}$ score.\nGottBERT was successfully pre-trained on a 256 core TPU pod using the RoBERTa\nBASE architecture. Even without extensive hyper-parameter optimization, in all\nNER and one text classification task, GottBERT already outperformed all other\ntested German and multilingual models. In order to support the German NLP\nfield, we publish GottBERT under the AGPLv3 license.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02128v1",
            "date": "2020-12-03 18:07:28+00:00",
            "title": "BERT-hLSTMs: BERT and Hierarchical LSTMs for Visual Storytelling",
            "authors": [
                "Jing Su",
                "Qingyun Dai",
                "Frank Guerin",
                "Mian Zhou"
            ],
            "abstract": "Visual storytelling is a creative and challenging task, aiming to\nautomatically generate a story-like description for a sequence of images. The\ndescriptions generated by previous visual storytelling approaches lack\ncoherence because they use word-level sequence generation methods and do not\nadequately consider sentence-level dependencies. To tackle this problem, we\npropose a novel hierarchical visual storytelling framework which separately\nmodels sentence-level and word-level semantics. We use the transformer-based\nBERT to obtain embeddings for sentences and words. We then employ a\nhierarchical LSTM network: the bottom LSTM receives as input the sentence\nvector representation from BERT, to learn the dependencies between the\nsentences corresponding to images, and the top LSTM is responsible for\ngenerating the corresponding word vector representations, taking input from the\nbottom LSTM. Experimental results demonstrate that our model outperforms most\nclosely related baselines under automatic evaluation metrics BLEU and CIDEr,\nand also show the effectiveness of our method with human evaluation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02197v2",
            "date": "2020-12-03 18:53:57+00:00",
            "title": "Addressing machine learning concept drift reveals declining vaccine sentiment during the COVID-19 pandemic",
            "authors": [
                "Martin M\u00fcller",
                "Marcel Salath\u00e9"
            ],
            "abstract": "Social media analysis has become a common approach to assess public opinion\non various topics, including those about health, in near real-time. The growing\nvolume of social media posts has led to an increased usage of modern machine\nlearning methods in natural language processing. While the rapid dynamics of\nsocial media can capture underlying trends quickly, it also poses a technical\nproblem: algorithms trained on annotated data in the past may underperform when\napplied to contemporary data. This phenomenon, known as concept drift, can be\nparticularly problematic when rapid shifts occur either in the topic of\ninterest itself, or in the way the topic is discussed. Here, we explore the\neffect of machine learning concept drift by focussing on vaccine sentiments\nexpressed on Twitter, a topic of central importance especially during the\nCOVID-19 pandemic. We show that while vaccine sentiment has declined\nconsiderably during the COVID-19 pandemic in 2020, algorithms trained on\npre-pandemic data would have largely missed this decline due to concept drift.\nOur results suggest that social media analysis systems must address concept\ndrift in a continuous fashion in order to avoid the risk of systematic\nmisclassification of data, which is particularly likely during a crisis when\nthe underlying data can change suddenly and rapidly.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2012.02339v1",
            "date": "2020-12-04 00:05:02+00:00",
            "title": "Understanding Guided Image Captioning Performance across Domains",
            "authors": [
                "Edwin G. Ng",
                "Bo Pang",
                "Piyush Sharma",
                "Radu Soricut"
            ],
            "abstract": "Image captioning models generally lack the capability to take into account\nuser interest, and usually default to global descriptions that try to balance\nreadability, informativeness, and information overload. On the other hand, VQA\nmodels generally lack the ability to provide long descriptive answers, while\nexpecting the textual question to be quite precise. We present a method to\ncontrol the concepts that an image caption should focus on, using an additional\ninput called the guiding text that refers to either groundable or ungroundable\nconcepts in the image. Our model consists of a Transformer-based multimodal\nencoder that uses the guiding text together with global and object-level image\nfeatures to derive early-fusion representations used to generate the guided\ncaption. While models trained on Visual Genome data have an in-domain advantage\nof fitting well when guided with automatic object labels, we find that guided\ncaptioning models trained on Conceptual Captions generalize better on\nout-of-domain images and guiding texts. Our human-evaluation results indicate\nthat attempting in-the-wild guided image captioning requires access to large,\nunrestricted-domain training datasets, and that increased style diversity (even\nwithout increasing vocabulary size) is a key factor for improved performance.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.02356v2",
            "date": "2020-12-04 01:22:05+00:00",
            "title": "WeaQA: Weak Supervision via Captions for Visual Question Answering",
            "authors": [
                "Pratyay Banerjee",
                "Tejas Gokhale",
                "Yezhou Yang",
                "Chitta Baral"
            ],
            "abstract": "Methodologies for training visual question answering (VQA) models assume the\navailability of datasets with human-annotated \\textit{Image-Question-Answer}\n(I-Q-A) triplets. This has led to heavy reliance on datasets and a lack of\ngeneralization to new types of questions and scenes. Linguistic priors along\nwith biases and errors due to annotator subjectivity have been shown to\npercolate into VQA models trained on such samples. We study whether models can\nbe trained without any human-annotated Q-A pairs, but only with images and\ntheir associated textual descriptions or captions. We present a method to train\nmodels with synthetic Q-A pairs generated procedurally from captions.\nAdditionally, we demonstrate the efficacy of spatial-pyramid image patches as a\nsimple but effective alternative to dense and costly object bounding box\nannotations used in existing VQA models. Our experiments on three VQA\nbenchmarks demonstrate the efficacy of this weakly-supervised approach,\nespecially on the VQA-CP challenge, which tests performance under changing\nlinguistic priors.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.02420v1",
            "date": "2020-12-04 06:09:02+00:00",
            "title": "A Benchmark Dataset for Understandable Medical Language Translation",
            "authors": [
                "Junyu Luo",
                "Zifei Zheng",
                "Hanzhong Ye",
                "Muchao Ye",
                "Yaqing Wang",
                "Quanzeng You",
                "Cao Xiao",
                "Fenglong Ma"
            ],
            "abstract": "In this paper, we introduce MedLane -- a new human-annotated Medical Language\ntranslation dataset, to align professional medical sentences with\nlayperson-understandable expressions. The dataset contains 12,801 training\nsamples, 1,015 validation samples, and 1,016 testing samples. We then evaluate\none naive and six deep learning-based approaches on the MedLane dataset,\nincluding directly copying, a statistical machine translation approach Moses,\nfour neural machine translation approaches (i.e., the proposed PMBERT-MT model,\nSeq2Seq and its two variants), and a modified text summarization model\nPointerNet. To compare the results, we utilize eleven metrics, including three\nnew measures specifically designed for this task. Finally, we discuss the\nlimitations of MedLane and baselines, and point out possible research\ndirections for this task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02446v2",
            "date": "2020-12-04 07:55:15+00:00",
            "title": "Spread Mechanism and Influence Measurement of Online Rumors in China During the COVID-19 Pandemic",
            "authors": [
                "Yiou Lin",
                "Hang Lei",
                "Yu Deng"
            ],
            "abstract": "In early 2020, the Corona Virus Disease 2019 (COVID-19) pandemic swept the\nworld.In China, COVID-19 has caused severe consequences. Moreover, online\nrumors during the COVID-19 pandemic increased people's panic about public\nhealth and social stability. At present, understanding and curbing the spread\nof online rumors is an urgent task. Therefore, we analyzed the rumor spreading\nmechanism and propose a method to quantify a rumors' influence by the speed of\nnew insiders. The search frequency of the rumor is used as an observation\nvariable of new insiders. The peak coefficient and the attenuation coefficient\nare calculated for the search frequency, which conforms to the exponential\ndistribution. We designed several rumor features and used the above two\ncoefficients as predictable labels. A 5-fold cross-validation experiment using\nthe mean square error (MSE) as the loss function showed that the decision tree\nwas suitable for predicting the peak coefficient, and the linear regression\nmodel was ideal for predicting the attenuation coefficient. Our feature\nanalysis showed that precursor features were the most important for the\noutbreak coefficient, while location information and rumor entity information\nwere the most important for the attenuation coefficient. Meanwhile, features\nthat were conducive to the outbreak were usually harmful to the continued\nspread of rumors. At the same time, anxiety was a crucial rumor causing factor.\nFinally, we discuss how to use deep learning technology to reduce the forecast\nloss by using the Bidirectional Encoder Representations from Transformers\n(BERT) model.",
            "journal ref": "None",
            "category": "cs.SI"
        },
        {
            "url": "http://arxiv.org/abs/2012.02505v1",
            "date": "2020-12-04 10:17:42+00:00",
            "title": "Financial Document Causality Detection Shared Task (FinCausal 2020)",
            "authors": [
                "Dominique Mariko",
                "Hanna Abi Akl",
                "Estelle Labidurie",
                "St\u00e9phane Durfort",
                "Hugues de Mazancourt",
                "Mahmoud El-Haj"
            ],
            "abstract": "We present the FinCausal 2020 Shared Task on Causality Detection in Financial\nDocuments and the associated FinCausal dataset, and discuss the participating\nsystems and results. Two sub-tasks are proposed: a binary classification task\n(Task 1) and a relation extraction task (Task 2). A total of 16 teams submitted\nruns across the two Tasks and 13 of them contributed with a system description\npaper. This workshop is associated to the Joint Workshop on Financial Narrative\nProcessing and MultiLing Financial Summarisation (FNP-FNS 2020), held at The\n28th International Conference on Computational Linguistics (COLING'2020),\nBarcelona, Spain on September 12, 2020.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02705v1",
            "date": "2020-12-04 16:27:59+00:00",
            "title": "Spatial Language Understanding for Object Search in Partially Observed Cityscale Environments",
            "authors": [
                "Kaiyu Zheng",
                "Deniz Bayazit",
                "Rebecca Mathew",
                "Ellie Pavlick",
                "Stefanie Tellex"
            ],
            "abstract": "We present a system that enables robots to interpret spatial language as a\ndistribution over object locations for effective search in partially observable\ncityscale environments. We introduce the spatial language observation space and\nformulate a stochastic observation model under the framework of Partially\nObservable Markov Decision Process (POMDP) which incorporates information\nextracted from the spatial language into the robot's belief. To interpret\nambiguous, context-dependent prepositions (e.g.~front), we propose a\nconvolutional neural network model that learns to predict the language\nprovider's relative frame of reference (FoR) given environment context. We\ndemonstrate the generalizability of our FoR prediction model and object search\nsystem through cross-validation over areas of five cities, each with a\n40,000m$^2$ footprint. End-to-end experiments in simulation show that our\nsystem achieves faster search and higher success rate compared to a\nkeyword-based baseline without spatial preposition understanding.",
            "journal ref": "None",
            "category": "cs.RO"
        },
        {
            "url": "http://arxiv.org/abs/2012.02757v1",
            "date": "2020-12-04 18:22:59+00:00",
            "title": "Playing Text-Based Games with Common Sense",
            "authors": [
                "Sahith Dambekodi",
                "Spencer Frazier",
                "Prithviraj Ammanabrolu",
                "Mark O. Riedl"
            ],
            "abstract": "Text based games are simulations in which an agent interacts with the world\npurely through natural language. They typically consist of a number of puzzles\ninterspersed with interactions with common everyday objects and locations. Deep\nreinforcement learning agents can learn to solve these puzzles. However, the\neveryday interactions with the environment, while trivial for human players,\npresent as additional puzzles to agents. We explore two techniques for\nincorporating commonsense knowledge into agents. Inferring possibly hidden\naspects of the world state with either a commonsense inference model COMET, or\na language model BERT. Biasing an agents exploration according to common\npatterns recognized by a language model. We test our technique in the 9to05\ngame, which is an extreme version of a text based game that requires numerous\ninteractions with common, everyday objects in common, everyday scenarios. We\nconclude that agents that augment their beliefs about the world state with\ncommonsense inferences are more robust to observational errors and omissions of\ncommon elements from text descriptions.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.02821v1",
            "date": "2020-12-04 19:51:31+00:00",
            "title": "MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs",
            "authors": [
                "Fangda Han",
                "Guoyao Hao",
                "Ricardo Guerrero",
                "Vladimir Pavlovic"
            ],
            "abstract": "Multilabel conditional image generation is a challenging problem in computer\nvision. In this work we propose Multi-ingredient Pizza Generator (MPG), a\nconditional Generative Neural Network (GAN) framework for synthesizing\nmultilabel images. We design MPG based on a state-of-the-art GAN structure\ncalled StyleGAN2, in which we develop a new conditioning technique by enforcing\nintermediate feature maps to learn scalewise label information. Because of the\ncomplex nature of the multilabel image generation problem, we also regularize\nsynthetic image by predicting the corresponding ingredients as well as\nencourage the discriminator to distinguish between matched image and mismatched\nimage. To verify the efficacy of MPG, we test it on Pizza10, which is a\ncarefully annotated multi-ingredient pizza image dataset. MPG can successfully\ngenerate photo-realist pizza images with desired ingredients. The framework can\nbe easily extend to other multilabel image generation scenarios.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.02943v1",
            "date": "2020-12-05 03:48:32+00:00",
            "title": "Cross-Domain Sentiment Classification with In-Domain Contrastive Learning",
            "authors": [
                "Tian Li",
                "Xiang Chen",
                "Shanghang Zhang",
                "Zhen Dong",
                "Kurt Keutzer"
            ],
            "abstract": "Contrastive learning (CL) has been successful as a powerful representation\nlearning method. In this paper, we propose a contrastive learning framework for\ncross-domain sentiment classification. We aim to induce domain invariant\noptimal classifiers rather than distribution matching. To this end, we\nintroduce in-domain contrastive learning and entropy minimization. Also, we\nfind through ablation studies that these two techniques behaviour differently\nin case of large label distribution shift and conclude that the best practice\nis to choose one of them adaptively according to label distribution shift. The\nnew state-of-the-art results our model achieves on standard benchmarks show the\nefficacy of the proposed method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02952v1",
            "date": "2020-12-05 05:21:57+00:00",
            "title": "Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation",
            "authors": [
                "Ruibo Liu",
                "Guangxuan Xu",
                "Chenyan Jia",
                "Weicheng Ma",
                "Lili Wang",
                "Soroush Vosoughi"
            ],
            "abstract": "Data augmentation is proven to be effective in many NLU tasks, especially for\nthose suffering from data scarcity. In this paper, we present a powerful and\neasy to deploy text augmentation framework, Data Boost, which augments data\nthrough reinforcement learning guided conditional generation. We evaluate Data\nBoost on three diverse text classification tasks under five different\nclassifier architectures. The result shows that Data Boost can boost the\nperformance of classifiers especially in low-resource data scenarios. For\ninstance, Data Boost improves F1 for the three tasks by 8.7% on average when\ngiven only 10% of the whole data for training. We also compare Data Boost with\nsix prior text augmentation methods. Through human evaluations (N=178), we\nconfirm that Data Boost augmentation has comparable quality as the original\ndata with respect to readability and class consistency.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02954v1",
            "date": "2020-12-05 05:45:16+00:00",
            "title": "Enhanced Offensive Language Detection Through Data Augmentation",
            "authors": [
                "Ruibo Liu",
                "Guangxuan Xu",
                "Soroush Vosoughi"
            ],
            "abstract": "Detecting offensive language on social media is an important task. The\nICWSM-2020 Data Challenge Task 2 is aimed at identifying offensive content\nusing a crowd-sourced dataset containing 100k labelled tweets. The dataset,\nhowever, suffers from class imbalance, where certain labels are extremely rare\ncompared with other classes (e.g, the hateful class is only 5% of the data). In\nthis work, we present Dager (Data Augmenter), a generation-based data\naugmentation method, that improves the performance of classification on\nimbalanced and low-resource data such as the offensive language dataset. Dager\nextracts the lexical features of a given class, and uses these features to\nguide the generation of a conditional generator built on GPT-2. The generated\ntext can then be added to the training set as augmentation data. We show that\napplying Dager can increase the F1 score of the data challenge by 11% when we\nuse 1% of the whole dataset for training (using BERT for classification);\nmoreover, the generated data also preserves the original labels very well. We\ntest Dager on four different classifiers (BERT, CNN, Bi-LSTM with attention,\nand Transformer), observing universal improvement on the detection, indicating\nour method is effective and classifier-agnostic.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02957v1",
            "date": "2020-12-05 06:10:56+00:00",
            "title": "A Sequence-Oblivious Generation Method for Context-Aware Hashtag Recommendation",
            "authors": [
                "Junmo Kang",
                "Jeonghwan Kim",
                "Suwon Shin",
                "Sung-Hyon Myaeng"
            ],
            "abstract": "Like search, a recommendation task accepts an input query or cue and provides\ndesirable items, often based on a ranking function. Such a ranking approach\nrarely considers explicit dependency among the recommended items. In this work,\nwe propose a generative approach to tag recommendation, where semantic tags are\nselected one at a time conditioned on the previously generated tags to model\ninter-dependency among the generated tags. We apply this tag recommendation\napproach to an Instagram data set where an array of context feature types\n(image, location, time, and text) are available for posts. To exploit the\ninter-dependency among the distinct types of features, we adopt a simple yet\neffective architecture using self-attention, making deep interactions possible.\nEmpirical results show that our method is significantly superior to not only\nthe usual ranking schemes but also autoregressive models for tag\nrecommendation. They indicate that it is critical to fuse mutually supporting\nfeatures at an early stage to induce extensive and comprehensive view on\ninter-context interaction in generating tags in a recurrent feedback loop.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.02975v1",
            "date": "2020-12-05 08:23:13+00:00",
            "title": "Reciprocal Supervised Learning Improves Neural Machine Translation",
            "authors": [
                "Minkai Xu",
                "Mingxuan Wang",
                "Zhouhan Lin",
                "Hao Zhou",
                "Weinan Zhang",
                "Lei Li"
            ],
            "abstract": "Despite the recent success on image classification, self-training has only\nachieved limited gains on structured prediction tasks such as neural machine\ntranslation (NMT). This is mainly due to the compositionality of the target\nspace, where the far-away prediction hypotheses lead to the notorious\nreinforced mistake problem. In this paper, we revisit the utilization of\nmultiple diverse models and present a simple yet effective approach named\nReciprocal-Supervised Learning (RSL). RSL first exploits individual models to\ngenerate pseudo parallel data, and then cooperatively trains each model on the\ncombined synthetic corpus. RSL leverages the fact that different parameterized\nmodels have different inductive biases, and better predictions can be made by\njointly exploiting the agreement among each other. Unlike the previous\nknowledge distillation methods built upon a much stronger teacher, RSL is\ncapable of boosting the accuracy of one model by introducing other comparable\nor even weaker models. RSL can also be viewed as a more efficient alternative\nto ensemble. Extensive experiments demonstrate the superior performance of RSL\non several benchmarks with significant margins.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03084v1",
            "date": "2020-12-05 17:37:41+00:00",
            "title": "Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks",
            "authors": [
                "Modestas Filipavicius",
                "Matteo Manica",
                "Joris Cadow",
                "Maria Rodriguez Martinez"
            ],
            "abstract": "Less than 1% of protein sequences are structurally and functionally\nannotated. Natural Language Processing (NLP) community has recently embraced\nself-supervised learning as a powerful approach to learn representations from\nunlabeled text, in large part due to the attention-based context-aware\nTransformer models. In this work we present a modification to the RoBERTa model\nby inputting during pre-training a mixture of binding and non-binding protein\nsequences (from STRING database). However, the sequence pairs have no label to\nindicate their binding status, as the model relies solely on Masked Language\nModeling (MLM) objective during pre-training. After fine-tuning, such approach\nsurpasses models trained on single protein sequences for protein-protein\nbinding prediction, TCR-epitope binding prediction, cellular-localization and\nremote homology classification tasks. We suggest that the Transformer's\nattention mechanism contributes to protein binding site discovery. Furthermore,\nwe compress protein sequences by 64% with the Byte Pair Encoding (BPE)\nvocabulary consisting of 10K subwords, each around 3-4 amino acids long.\nFinally, to expand the model input space to even larger proteins and\nmulti-protein assemblies, we pre-train Longformer models that support 2,048\ntokens. Further work in token-level classification for secondary structure\nprediction is needed. Code available at:\nhttps://github.com/PaccMann/paccmann_proteomics",
            "journal ref": "None",
            "category": "q-bio.BM"
        },
        {
            "url": "http://arxiv.org/abs/2012.03091v1",
            "date": "2020-12-05 17:59:59+00:00",
            "title": "Over a Decade of Social Opinion Mining",
            "authors": [
                "Keith Cortis",
                "Brian Davis"
            ],
            "abstract": "Social media popularity and importance is on the increase, due to people\nusing it for various types of social interaction across multiple channels. This\nsocial interaction by online users includes submission of feedback, opinions\nand recommendations about various individuals, entities, topics, and events.\nThis systematic review focuses on the evolving research area of Social Opinion\nMining, tasked with the identification of multiple opinion dimensions, such as\nsubjectivity, sentiment polarity, emotion, affect, sarcasm and irony, from\nuser-generated content represented across multiple social media platforms and\nin various media formats, like text, image, video and audio. Therefore, through\nSocial Opinion Mining, natural language can be understood in terms of the\ndifferent opinion dimensions, as expressed by humans. This contributes towards\nthe evolution of Artificial Intelligence, which in turn helps the advancement\nof several real-world use cases, such as customer service and decision making.\nA thorough systematic review was carried out on Social Opinion Mining research\nwhich totals 485 studies and spans a period of twelve years between 2007 and\n2018. The in-depth analysis focuses on the social media platforms, techniques,\nsocial datasets, language, modality, tools and technologies, natural language\nprocessing tasks and other aspects derived from the published studies. Such\nmulti-source information fusion plays a fundamental role in mining of people's\nsocial opinions from social media platforms. These can be utilised in many\napplication areas, ranging from marketing, advertising and sales for\nproduct/service management, and in multiple domains and industries, such as\npolitics, technology, finance, healthcare, sports and government. Future\nresearch directions are presented, whereas further research and development has\nthe potential of leaving a wider academic and societal impact.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03370v1",
            "date": "2020-12-06 20:32:56+00:00",
            "title": "Competition in Cross-situational Word Learning: A Computational Study",
            "authors": [
                "Aida Nematzadeh",
                "Zahra Shekarchi",
                "Thomas L. Griffiths",
                "Suzanne Stevenson"
            ],
            "abstract": "Children learn word meanings by tapping into the commonalities across\ndifferent situations in which words are used and overcome the high level of\nuncertainty involved in early word learning experiences. In a set of\ncomputational studies, we show that to successfully learn word meanings in the\nface of uncertainty, a learner needs to use two types of competition: words\ncompeting for association to a referent when learning from an observation and\nreferents competing for a word when the word is used.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03418v1",
            "date": "2020-12-07 02:18:49+00:00",
            "title": "From syntactic structure to semantic relationship: hypernym extraction from definitions by recurrent neural networks using the part of speech information",
            "authors": [
                "Yixin Tan",
                "Xiaomeng Wang",
                "Tao Jia"
            ],
            "abstract": "The hyponym-hypernym relation is an essential element in the semantic\nnetwork. Identifying the hypernym from a definition is an important task in\nnatural language processing and semantic analysis. While a public dictionary\nsuch as WordNet works for common words, its application in domain-specific\nscenarios is limited. Existing tools for hypernym extraction either rely on\nspecific semantic patterns or focus on the word representation, which all\ndemonstrate certain limitations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03468v1",
            "date": "2020-12-07 06:14:13+00:00",
            "title": "An Empirical Survey of Unsupervised Text Representation Methods on Twitter Data",
            "authors": [
                "Lili Wang",
                "Chongyang Gao",
                "Jason Wei",
                "Weicheng Ma",
                "Ruibo Liu",
                "Soroush Vosoughi"
            ],
            "abstract": "The field of NLP has seen unprecedented achievements in recent years. Most\nnotably, with the advent of large-scale pre-trained Transformer-based language\nmodels, such as BERT, there has been a noticeable improvement in text\nrepresentation. It is, however, unclear whether these improvements translate to\nnoisy user-generated text, such as tweets. In this paper, we present an\nexperimental survey of a wide range of well-known text representation\ntechniques for the task of text clustering on noisy Twitter data. Our results\nindicate that the more advanced models do not necessarily work best on tweets\nand that more exploration in this area is needed.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03551v1",
            "date": "2020-12-07 09:39:25+00:00",
            "title": "KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning",
            "authors": [
                "Bin He",
                "Xin Jiang",
                "Jinghui Xiao",
                "Qun Liu"
            ],
            "abstract": "Recent studies on pre-trained language models have demonstrated their ability\nto capture factual knowledge and applications in knowledge-aware downstream\ntasks. In this work, we present a language model pre-training framework guided\nby factual knowledge completion and verification, and use the generative and\ndiscriminative approaches cooperatively to learn the model. Particularly, we\ninvestigate two learning schemes, named two-tower scheme and pipeline scheme,\nin training the generator and discriminator with shared parameter. Experimental\nresults on LAMA, a set of zero-shot cloze-style question answering tasks, show\nthat our model contains richer factual knowledge than the conventional\npre-trained language models. Furthermore, when fine-tuned and evaluated on the\nMRQA shared tasks which consists of several machine reading comprehension\ndatasets, our model achieves the state-of-the-art performance, and gains large\nimprovements on NewsQA (+1.26 F1) and TriviaQA (+1.56 F1) over RoBERTa.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03573v1",
            "date": "2020-12-07 10:29:30+00:00",
            "title": "PPKE: Knowledge Representation Learning by Path-based Pre-training",
            "authors": [
                "Bin He",
                "Di Zhou",
                "Jing Xie",
                "Jinghui Xiao",
                "Xin Jiang",
                "Qun Liu"
            ],
            "abstract": "Entities may have complex interactions in a knowledge graph (KG), such as\nmulti-step relationships, which can be viewed as graph contextual information\nof the entities. Traditional knowledge representation learning (KRL) methods\nusually treat a single triple as a training unit, and neglect most of the graph\ncontextual information exists in the topological structure of KGs. In this\nstudy, we propose a Path-based Pre-training model to learn Knowledge\nEmbeddings, called PPKE, which aims to integrate more graph contextual\ninformation between entities into the KRL model. Experiments demonstrate that\nour model achieves state-of-the-art results on several benchmark datasets for\nlink prediction and relation prediction tasks, indicating that our model\nprovides a feasible way to take advantage of graph contextual information in\nKGs.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03656v2",
            "date": "2020-12-07 13:16:01+00:00",
            "title": "An Enhanced MeanSum Method For Generating Hotel Multi-Review Summarizations",
            "authors": [
                "Saibo Geng",
                "Diego Antognini"
            ],
            "abstract": "Multi-document summaritazion is the process of taking multiple texts as input\nand producing a short summary text based on the content of input texts. Up\nuntil recently, multi-document summarizers are mostly supervised extractive.\nHowever, supervised methods require datasets of large, paired document-summary\nexamples which are rare and expensive to produce. In 2018, an unsupervised\nmulti-document abstractive summarization method(Meansum) was proposed by Chu\nand Liu, and demonstrated competitive performances comparing to extractive\nmethods. Despite good evaluation results on automatic metrics, Meansum has\nmultiple limitations, notably the inability of dealing with multiple aspects.\nThe aim of this work was to use Multi-Aspect Masker(MAM) as content selector to\naddress the issue with multi-aspect. Moreover, we propose a regularizer to\ncontrol the length of the generated summaries. Through a series of experiments\non the hotel dataset from Trip Advisor, we validate our assumption and show\nthat our improved model achieves higher ROUGE, Sentiment Accuracy than the\noriginal Meansum method and also beats/ comprarable/close to the supervised\nbaseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03662v3",
            "date": "2020-12-07 13:20:12+00:00",
            "title": "Confidence-aware Non-repetitive Multimodal Transformers for TextCaps",
            "authors": [
                "Zhaokai Wang",
                "Renda Bao",
                "Qi Wu",
                "Si Liu"
            ],
            "abstract": "When describing an image, reading text in the visual scene is crucial to\nunderstand the key information. Recent work explores the TextCaps task, i.e.\nimage captioning with reading Optical Character Recognition (OCR) tokens, which\nrequires models to read text and cover them in generated captions. Existing\napproaches fail to generate accurate descriptions because of their (1) poor\nreading ability; (2) inability to choose the crucial words among all extracted\nOCR tokens; (3) repetition of words in predicted captions. To this end, we\npropose a Confidence-aware Non-repetitive Multimodal Transformers (CNMT) to\ntackle the above challenges. Our CNMT consists of a reading, a reasoning and a\ngeneration modules, in which Reading Module employs better OCR systems to\nenhance text reading ability and a confidence embedding to select the most\nnoteworthy tokens. To address the issue of word redundancy in captions, our\nGeneration Module includes a repetition mask to avoid predicting repeated word\nin captions. Our model outperforms state-of-the-art models on TextCaps dataset,\nimproving from 81.0 to 93.0 in CIDEr. Our source code is publicly available.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.03709v1",
            "date": "2020-12-07 14:11:33+00:00",
            "title": "Reference Knowledgeable Network for Machine Reading Comprehension",
            "authors": [
                "Yilin Zhao",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "Multi-choice Machine Reading Comprehension (MRC) is a major and challenging\nform of MRC tasks that requires model to select the most appropriate answer\nfrom a set of candidates given passage and question. Most of the existing\nresearches focus on the modeling of the task datasets without explicitly\nreferring to external fine-grained commonsense sources, which is a well-known\nchallenge in multi-choice tasks. Thus we propose a novel reference-based\nknowledge enhancement model based on span extraction called Reference\nKnowledgeable Network (RekNet), which simulates human reading strategy to\nrefine critical information from the passage and quote external knowledge in\nnecessity. In detail, RekNet refines fine-grained critical information and\ndefines it as Reference Span, then quotes external knowledge quadruples by the\nco-occurrence information of Reference Span and answer options. Our proposed\nmethod is evaluated on two multi-choice MRC benchmarks: RACE and DREAM, which\nshows remarkable performance improvement with observable statistical\nsignificance level over strong baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.03755v1",
            "date": "2020-12-07 14:49:33+00:00",
            "title": "Foundations for Near-Term Quantum Natural Language Processing",
            "authors": [
                "Bob Coecke",
                "Giovanni de Felice",
                "Konstantinos Meichanetzidis",
                "Alexis Toumi"
            ],
            "abstract": "We provide conceptual and mathematical foundations for near-term quantum\nnatural language processing (QNLP), and do so in quantum computer scientist\nfriendly terms. We opted for an expository presentation style, and provide\nreferences for supporting empirical evidence and formal statements concerning\nmathematical generality.\n  We recall how the quantum model for natural language that we employ\ncanonically combines linguistic meanings with rich linguistic structure, most\nnotably grammar. In particular, the fact that it takes a quantum-like model to\ncombine meaning and structure, establishes QNLP as quantum-native, on par with\nsimulation of quantum systems. Moreover, the now leading Noisy\nIntermediate-Scale Quantum (NISQ) paradigm for encoding classical data on\nquantum hardware, variational quantum circuits, makes NISQ exceptionally\nQNLP-friendly: linguistic structure can be encoded as a free lunch, in contrast\nto the apparently exponentially expensive classical encoding of grammar.\n  Quantum speed-up for QNLP tasks has already been established in previous work\nwith Will Zeng. Here we provide a broader range of tasks which all enjoy the\nsame advantage.\n  Diagrammatic reasoning is at the heart of QNLP. Firstly, the quantum model\ninterprets language as quantum processes via the diagrammatic formalism of\ncategorical quantum mechanics. Secondly, these diagrams are via ZX-calculus\ntranslated into quantum circuits. Parameterisations of meanings then become the\ncircuit variables to be learned.\n  Our encoding of linguistic structure within quantum circuits also embodies a\nnovel approach for establishing word-meanings that goes beyond the current\nstandards in mainstream AI, by placing linguistic structure at the heart of\nWittgenstein's meaning-is-context.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/2012.03756v1",
            "date": "2020-12-07 14:49:34+00:00",
            "title": "Grammar-Aware Question-Answering on Quantum Computers",
            "authors": [
                "Konstantinos Meichanetzidis",
                "Alexis Toumi",
                "Giovanni de Felice",
                "Bob Coecke"
            ],
            "abstract": "Natural language processing (NLP) is at the forefront of great advances in\ncontemporary AI, and it is arguably one of the most challenging areas of the\nfield. At the same time, with the steady growth of quantum hardware and notable\nimprovements towards implementations of quantum algorithms, we are approaching\nan era when quantum computers perform tasks that cannot be done on classical\ncomputers with a reasonable amount of resources. This provides a new range of\nopportunities for AI, and for NLP specifically. Earlier work has already\ndemonstrated a potential quantum advantage for NLP in a number of manners: (i)\nalgorithmic speedups for search-related or classification tasks, which are the\nmost dominant tasks within NLP, (ii) exponentially large quantum state spaces\nallow for accommodating complex linguistic structures, (iii) novel models of\nmeaning employing density matrices naturally model linguistic phenomena such as\nhyponymy and linguistic ambiguity, among others. In this work, we perform the\nfirst implementation of an NLP task on noisy intermediate-scale quantum (NISQ)\nhardware. Sentences are instantiated as parameterised quantum circuits. We\nencode word-meanings in quantum states and we explicitly account for\ngrammatical structure, which even in mainstream NLP is not commonplace, by\nfaithfully hard-wiring it as entangling operations. This makes our approach to\nquantum natural language processing (QNLP) particularly NISQ-friendly. Our\nnovel QNLP model shows concrete promise for scalability as the quality of the\nquantum hardware improves in the near future.",
            "journal ref": "None",
            "category": "quant-ph"
        },
        {
            "url": "http://arxiv.org/abs/2012.03864v1",
            "date": "2020-12-07 17:14:52+00:00",
            "title": "Evaluating Cross-Lingual Transfer Learning Approaches in Multilingual Conversational Agent Models",
            "authors": [
                "Lizhen Tan",
                "Olga Golovneva"
            ],
            "abstract": "With the recent explosion in popularity of voice assistant devices, there is\na growing interest in making them available to user populations in additional\ncountries and languages. However, to provide the highest accuracy and best\nperformance for specific user populations, most existing voice assistant models\nare developed individually for each region or language, which requires linear\ninvestment of effort. In this paper, we propose a general multilingual model\nframework for Natural Language Understanding (NLU) models, which can help\nbootstrap new language models faster and reduce the amount of effort required\nto develop each language separately. We explore how different deep learning\narchitectures affect multilingual NLU model performance. Our experimental\nresults show that these multilingual models can reach same or better\nperformance compared to monolingual models across language-specific test data\nwhile require less effort in creating features and model maintenance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04005v1",
            "date": "2020-12-07 19:17:05+00:00",
            "title": "Improving Clinical Document Understanding on COVID-19 Research with Spark NLP",
            "authors": [
                "Veysel Kocaman",
                "David Talby"
            ],
            "abstract": "Following the global COVID-19 pandemic, the number of scientific papers\nstudying the virus has grown massively, leading to increased interest in\nautomated literate review. We present a clinical text mining system that\nimproves on previous efforts in three ways. First, it can recognize over 100\ndifferent entity types including social determinants of health, anatomy, risk\nfactors, and adverse events in addition to other commonly used clinical and\nbiomedical entities. Second, the text processing pipeline includes assertion\nstatus detection, to distinguish between clinical facts that are present,\nabsent, conditional, or about someone other than the patient. Third, the deep\nlearning models used are more accurate than previously available, leveraging an\nintegrated pipeline of state-of-the-art pretrained named entity recognition\nmodels, and improving on the previous best performing benchmarks for assertion\nstatus detection. We illustrate extracting trends and insights, e.g. most\nfrequent disorders and symptoms, and most common vital signs and EKG findings,\nfrom the COVID-19 Open Research Dataset (CORD-19). The system is built using\nthe Spark NLP library which natively supports scaling to use distributed\nclusters, leveraging GPUs, configurable and reusable NLP pipelines, healthcare\nspecific embeddings, and the ability to train models to support new entity\ntypes or human languages with no code changes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04056v1",
            "date": "2020-12-07 21:00:42+00:00",
            "title": "Semantics Altering Modifications for Evaluating Comprehension in Machine Reading",
            "authors": [
                "Viktor Schlegel",
                "Goran Nenadic",
                "Riza Batista-Navarro"
            ],
            "abstract": "Advances in NLP have yielded impressive results for the task of machine\nreading comprehension (MRC), with approaches having been reported to achieve\nperformance comparable to that of humans. In this paper, we investigate whether\nstate-of-the-art MRC models are able to correctly process Semantics Altering\nModifications (SAM): linguistically-motivated phenomena that alter the\nsemantics of a sentence while preserving most of its lexical surface form. We\npresent a method to automatically generate and align challenge sets featuring\noriginal and altered examples. We further propose a novel evaluation\nmethodology to correctly assess the capability of MRC systems to process these\nexamples independent of the data they were optimised on, by discounting for\neffects introduced by domain shift. In a large-scale empirical study, we apply\nthe methodology in order to evaluate extractive MRC models with regard to their\ncapability to correctly process SAM-enriched data. We comprehensively cover 12\ndifferent state-of-the-art neural architecture configurations and four training\ndatasets and find that -- despite their well-known remarkable performance --\noptimised models consistently struggle to correctly process semantically\naltered data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04099v2",
            "date": "2020-12-07 22:37:38+00:00",
            "title": "Using multiple ASR hypotheses to boost i18n NLU performance",
            "authors": [
                "Charith Peris",
                "Gokmen Oz",
                "Khadige Abboud",
                "Venkata sai Varada",
                "Prashan Wanigasekara",
                "Haidar Khan"
            ],
            "abstract": "Current voice assistants typically use the best hypothesis yielded by their\nAutomatic Speech Recognition (ASR) module as input to their Natural Language\nUnderstanding (NLU) module, thereby losing helpful information that might be\nstored in lower-ranked ASR hypotheses. We explore the change in performance of\nNLU associated tasks when utilizing five-best ASR hypotheses when compared to\nstatus quo for two language datasets, German and Portuguese. To harvest\ninformation from the ASR five-best, we leverage extractive summarization and\njoint extractive-abstractive summarization models for Domain Classification\n(DC) experiments while using a sequence-to-sequence model with a pointer\ngenerator network for Intent Classification (IC) and Named Entity Recognition\n(NER) multi-task experiments. For the DC full test set, we observe significant\nimprovements of up to 7.2% and 15.5% in micro-averaged F1 scores, for German\nand Portuguese, respectively. In cases where the best ASR hypothesis was not an\nexact match to the transcribed utterance (mismatched test set), we see\nimprovements of up to 6.7% and 8.8% micro-averaged F1 scores, for German and\nPortuguese, respectively. For IC and NER multi-task experiments, when\nevaluating on the mismatched test set, we see improvements across all domains\nin German and in 17 out of 19 domains in Portuguese (improvements based on\nchange in SeMER scores). Our results suggest that the use of multiple ASR\nhypotheses, as opposed to one, can lead to significant performance improvements\nin the DC task for these non-English datasets. In addition, it could lead to\nsignificant improvement in the performance of IC and NER tasks in cases where\nthe ASR model makes mistakes.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04188v1",
            "date": "2020-12-08 03:14:28+00:00",
            "title": "Learning to Represent Programs with Heterogeneous Graphs",
            "authors": [
                "Wenhan Wang",
                "Kechi Zhang",
                "Ge Li",
                "Zhi Jin"
            ],
            "abstract": "Program source code contains complex structure information, which can be\nrepresented in structured data forms like trees or graphs. To acquire the\nstructural information in source code, most existing researches use abstract\nsyntax trees (AST). A group of works add additional edges to ASTs to convert\nsource code into graphs and use graph neural networks to learn representations\nfor program graphs. Although these works provide additional control or data\nflow information to ASTs for downstream tasks, they neglect an important aspect\nof structure information in AST itself: the different types of nodes and edges.\nIn ASTs, different nodes contain different kinds of information like variables\nor control flow, and the relation between a node and all its children can also\nbe different.\n  To address the information of node and edge types, we bring the idea of\nheterogeneous graphs to learning on source code and present a new formula of\nbuilding heterogeneous program graphs from ASTs with additional type\ninformation for nodes and edges. We use the ASDL grammar of programming\nlanguage to define the node and edge types of program graphs. Then we use\nheterogeneous graph neural networks to learn on these graphs. We evaluate our\napproach on two tasks: code comment generation and method naming. Both tasks\nrequire reasoning on the semantics of complete code snippets. Experiment\nresults show that our approach outperforms baseline models, including\nhomogeneous graph-based models, showing that leveraging the type information of\nnodes and edges in program graphs can help in learning program semantics.",
            "journal ref": "None",
            "category": "cs.SE"
        },
        {
            "url": "http://arxiv.org/abs/2012.04203v1",
            "date": "2020-12-08 04:21:40+00:00",
            "title": "A Topological Method for Comparing Document Semantics",
            "authors": [
                "Yuqi Kong",
                "Fanchao Meng",
                "Benjamin Carterette"
            ],
            "abstract": "Comparing document semantics is one of the toughest tasks in both Natural\nLanguage Processing and Information Retrieval. To date, on one hand, the tools\nfor this task are still rare. On the other hand, most relevant methods are\ndevised from the statistic or the vector space model perspectives but nearly\nnone from a topological perspective. In this paper, we hope to make a different\nsound. A novel algorithm based on topological persistence for comparing\nsemantics similarity between two documents is proposed. Our experiments are\nconducted on a document dataset with human judges' results. A collection of\nstate-of-the-art methods are selected for comparison. The experimental results\nshow that our algorithm can produce highly human-consistent results, and also\nbeats most state-of-the-art methods though ties with NLTK.",
            "journal ref": "pp. 143-151, 2020. CS & IT - CSCP 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04233v2",
            "date": "2020-12-08 05:53:33+00:00",
            "title": "Early Detection of Fake News by Utilizing the Credibility of News, Publishers, and Users Based on Weakly Supervised Learning",
            "authors": [
                "Chunyuan Yuan",
                "Qianwen Ma",
                "Wei Zhou",
                "Jizhong Han",
                "Songlin Hu"
            ],
            "abstract": "The dissemination of fake news significantly affects personal reputation and\npublic trust. Recently, fake news detection has attracted tremendous attention,\nand previous studies mainly focused on finding clues from news content or\ndiffusion path. However, the required features of previous models are often\nunavailable or insufficient in early detection scenarios, resulting in poor\nperformance. Thus, early fake news detection remains a tough challenge.\nIntuitively, the news from trusted and authoritative sources or shared by many\nusers with a good reputation is more reliable than other news. Using the\ncredibility of publishers and users as prior weakly supervised information, we\ncan quickly locate fake news in massive news and detect them in the early\nstages of dissemination.\n  In this paper, we propose a novel Structure-aware Multi-head Attention\nNetwork (SMAN), which combines the news content, publishing, and reposting\nrelations of publishers and users, to jointly optimize the fake news detection\nand credibility prediction tasks. In this way, we can explicitly exploit the\ncredibility of publishers and users for early fake news detection. We conducted\nexperiments on three real-world datasets, and the results show that SMAN can\ndetect fake news in 4 hours with an accuracy of over 91%, which is much faster\nthan the state-of-the-art models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04276v1",
            "date": "2020-12-08 08:43:13+00:00",
            "title": "Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization",
            "authors": [
                "Yinuo Guo",
                "Hualei Zhu",
                "Zeqi Lin",
                "Bei Chen",
                "Jian-Guang Lou",
                "Dongmei Zhang"
            ],
            "abstract": "Human intelligence exhibits compositional generalization (i.e., the capacity\nto understand and produce unseen combinations of seen components), but current\nneural seq2seq models lack such ability. In this paper, we revisit iterative\nback-translation, a simple yet effective semi-supervised method, to investigate\nwhether and how it can improve compositional generalization. In this work: (1)\nWe first empirically show that iterative back-translation substantially\nimproves the performance on compositional generalization benchmarks (CFQ and\nSCAN). (2) To understand why iterative back-translation is useful, we carefully\nexamine the performance gains and find that iterative back-translation can\nincreasingly correct errors in pseudo-parallel data. (3) To further encourage\nthis mechanism, we propose curriculum iterative back-translation, which better\nimproves the quality of pseudo-parallel data, thus further improving the\nperformance.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04307v1",
            "date": "2020-12-08 09:30:38+00:00",
            "title": "Cross-lingual Approach to Abstractive Summarization",
            "authors": [
                "Ale\u0161 \u017dagar",
                "Marko Robnik-\u0160ikonja"
            ],
            "abstract": "Automatic text summarization extracts important information from texts and\npresents the information in the form of a summary. Abstractive summarization\napproaches progressed significantly by switching to deep neural networks, but\nresults are not yet satisfactory, especially for languages where large training\nsets do not exist. In several natural language processing tasks, cross-lingual\nmodel transfers are successfully applied in low-resource languages. For\nsummarization such cross-lingual model transfer was so far not attempted due to\na non-reusable decoder side of neural models. In our work, we used a pretrained\nEnglish summarization model based on deep neural networks and\nsequence-to-sequence architecture to summarize Slovene news articles. We solved\nthe problem of inadequate decoder by using an additional language model for\ntarget language evaluation. We developed several models with different\nproportions of target language data for fine-tuning. The results were assessed\nwith automatic evaluation measures and with small-scale human evaluation. The\nresults show that summaries of cross-lingual models fine-tuned with relatively\nsmall amount of target language data are useful and of similar quality to an\nabstractive summarizer trained with much more data in the target language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04373v2",
            "date": "2020-12-08 11:31:55+00:00",
            "title": "CrossNER: Evaluating Cross-Domain Named Entity Recognition",
            "authors": [
                "Zihan Liu",
                "Yan Xu",
                "Tiezheng Yu",
                "Wenliang Dai",
                "Ziwei Ji",
                "Samuel Cahyawijaya",
                "Andrea Madotto",
                "Pascale Fung"
            ],
            "abstract": "Cross-domain named entity recognition (NER) models are able to cope with the\nscarcity issue of NER samples in target domains. However, most of the existing\nNER benchmarks lack domain-specialized entity types or do not focus on a\ncertain domain, leading to a less effective cross-domain evaluation. To address\nthese obstacles, we introduce a cross-domain NER dataset (CrossNER), a\nfully-labeled collection of NER data spanning over five diverse domains with\nspecialized entity categories for different domains. Additionally, we also\nprovide a domain-related corpus since using it to continue pre-training\nlanguage models (domain-adaptive pre-training) is effective for the domain\nadaptation. We then conduct comprehensive experiments to explore the\neffectiveness of leveraging different levels of the domain corpus and\npre-training strategies to do domain-adaptive pre-training for the cross-domain\ntask. Results show that focusing on the fractional corpus containing\ndomain-specialized entities and utilizing a more challenging pre-training\nstrategy in domain-adaptive pre-training are beneficial for the NER domain\nadaptation, and our proposed method can consistently outperform existing\ncross-domain NER baselines. Nevertheless, experiments also illustrate the\nchallenge of this cross-domain NER task. We hope that our dataset and baselines\nwill catalyze research in the NER domain adaptation area. The code and data are\navailable at https://github.com/zliucr/CrossNER.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04380v1",
            "date": "2020-12-08 11:52:14+00:00",
            "title": "Combining Machine Learning and Human Experts to Predict Match Outcomes in Football: A Baseline Model",
            "authors": [
                "Ryan Beal",
                "Stuart E. Middleton",
                "Timothy J. Norman",
                "Sarvapali D. Ramchurn"
            ],
            "abstract": "In this paper, we present a new application-focused benchmark dataset and\nresults from a set of baseline Natural Language Processing and Machine Learning\nmodels for prediction of match outcomes for games of football (soccer). By\ndoing so we give a baseline for the prediction accuracy that can be achieved\nexploiting both statistical match data and contextual articles from human\nsports journalists. Our dataset is focuses on a representative time-period over\n6 seasons of the English Premier League, and includes newspaper match previews\nfrom The Guardian. The models presented in this paper achieve an accuracy of\n63.18% showing a 6.9% boost on the traditional statistical methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04539v1",
            "date": "2020-12-07 07:55:31+00:00",
            "title": "Dartmouth CS at WNUT-2020 Task 2: Informative COVID-19 Tweet Classification Using BERT",
            "authors": [
                "Dylan Whang",
                "Soroush Vosoughi"
            ],
            "abstract": "We describe the systems developed for the WNUT-2020 shared task 2,\nidentification of informative COVID-19 English Tweets. BERT is a highly\nperformant model for Natural Language Processing tasks. We increased BERT's\nperformance in this classification task by fine-tuning BERT and concatenating\nits embeddings with Tweet-specific features and training a Support Vector\nMachine (SVM) for classification (henceforth called BERT+). We compared its\nperformance to a suite of machine learning models. We used a Twitter specific\ndata cleaning pipeline and word-level TF-IDF to extract features for the\nnon-BERT models. BERT+ was the top performing model with an F1-score of 0.8713.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04540v1",
            "date": "2020-12-07 08:17:42+00:00",
            "title": "Improvements and Extensions on Metaphor Detection",
            "authors": [
                "Weicheng Ma",
                "Ruibo Liu",
                "Lili Wang",
                "Soroush Vosoughi"
            ],
            "abstract": "Metaphors are ubiquitous in human language. The metaphor detection task (MD)\naims at detecting and interpreting metaphors from written language, which is\ncrucial in natural language understanding (NLU) research. In this paper, we\nintroduce a pre-trained Transformer-based model into MD. Our model outperforms\nthe previous state-of-the-art models by large margins in our evaluations, with\nrelative improvements on the F-1 score from 5.33% to 28.39%. Second, we extend\nMD to a classification task about the metaphoricity of an entire piece of text\nto make MD applicable in more general NLU scenes. Finally, we clean up the\nimproper or outdated annotations in one of the MD benchmark datasets and\nre-benchmark it with our Transformer-based model. This approach could be\napplied to other existing MD datasets as well, since the metaphoricity\nannotations in these benchmark datasets may be outdated. Future research\nefforts are also necessary to build an up-to-date and well-annotated dataset\nconsisting of longer and more complex texts.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04584v1",
            "date": "2020-12-08 17:36:34+00:00",
            "title": "Distilling Knowledge from Reader to Retriever for Question Answering",
            "authors": [
                "Gautier Izacard",
                "Edouard Grave"
            ],
            "abstract": "The task of information retrieval is an important component of many natural\nlanguage processing systems, such as open domain question answering. While\ntraditional methods were based on hand-crafted features, continuous\nrepresentations based on neural networks recently obtained competitive results.\nA challenge of using such methods is to obtain supervised data to train the\nretriever model, corresponding to pairs of query and support documents. In this\npaper, we propose a technique to learn retriever models for downstream tasks,\ninspired by knowledge distillation, and which does not require annotated pairs\nof query and documents. Our approach leverages attention scores of a reader\nmodel, used to solve the task based on retrieved documents, to obtain synthetic\nlabels for the retriever. We evaluate our method on question answering,\nobtaining state-of-the-art results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04585v1",
            "date": "2020-12-08 17:36:39+00:00",
            "title": "Discourse Parsing of Contentious, Non-Convergent Online Discussions",
            "authors": [
                "Stepan Zakharov",
                "Omri Hadar",
                "Tovit Hakak",
                "Dina Grossman",
                "Yifat Ben-David Kolikant",
                "Oren Tsur"
            ],
            "abstract": "Online discourse is often perceived as polarized and unproductive. While some\nconversational discourse parsing frameworks are available, they do not\nnaturally lend themselves to the analysis of contentious and polarizing\ndiscussions. Inspired by the Bakhtinian theory of Dialogism, we propose a novel\ntheoretical and computational framework, better suited for non-convergent\ndiscussions. We redefine the measure of a successful discussion, and develop a\nnovel discourse annotation schema which reflects a hierarchy of discursive\nstrategies. We consider an array of classification models -- from Logistic\nRegression to BERT. We also consider various feature types and representations,\ne.g., LIWC categories, standard embeddings, conversational sequences, and\nnon-conversational discourse markers learnt separately. Given the 31 labels in\nthe tagset, an average F-Score of 0.61 is achieved if we allow a different\nmodel for each tag, and 0.526 with a single model. The promising results\nachieved in annotating discussions according to the proposed schema paves the\nway for a number of downstream tasks and applications such as early detection\nof discussion trajectories, active moderation of open discussions, and\nteacher-assistive bots. Finally, we share the first labeled dataset of\ncontentious non-convergent online discussions.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04726v1",
            "date": "2020-12-08 20:30:43+00:00",
            "title": "Edited Media Understanding: Reasoning About Implications of Manipulated Images",
            "authors": [
                "Jeff Da",
                "Maxwell Forbes",
                "Rowan Zellers",
                "Anthony Zheng",
                "Jena D. Hwang",
                "Antoine Bosselut",
                "Yejin Choi"
            ],
            "abstract": "Multimodal disinformation, from `deepfakes' to simple edits that deceive, is\nan important societal problem. Yet at the same time, the vast majority of media\nedits are harmless -- such as a filtered vacation photo. The difference between\nthis example, and harmful edits that spread disinformation, is one of intent.\nRecognizing and describing this intent is a major challenge for today's AI\nsystems.\n  We present the task of Edited Media Understanding, requiring models to answer\nopen-ended questions that capture the intent and implications of an image edit.\nWe introduce a dataset for our task, EMU, with 48k question-answer pairs\nwritten in rich natural language. We evaluate a wide variety of\nvision-and-language models for our task, and introduce a new model PELICAN,\nwhich builds upon recent progress in pretrained multimodal representations. Our\nmodel obtains promising results on our dataset, with humans rating its answers\nas accurate 40.35% of the time. At the same time, there is still much work to\nbe done -- humans prefer human-annotated captions 93.56% of the time -- and we\nprovide analysis that highlights areas for further progress.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.04987v1",
            "date": "2020-12-09 11:34:35+00:00",
            "title": "Label Confusion Learning to Enhance Text Classification Models",
            "authors": [
                "Biyang Guo",
                "Songqiao Han",
                "Xiao Han",
                "Hailiang Huang",
                "Ting Lu"
            ],
            "abstract": "Representing a true label as a one-hot vector is a common practice in\ntraining text classification models. However, the one-hot representation may\nnot adequately reflect the relation between the instances and labels, as labels\nare often not completely independent and instances may relate to multiple\nlabels in practice. The inadequate one-hot representations tend to train the\nmodel to be over-confident, which may result in arbitrary prediction and model\noverfitting, especially for confused datasets (datasets with very similar\nlabels) or noisy datasets (datasets with labeling errors). While training\nmodels with label smoothing (LS) can ease this problem in some degree, it still\nfails to capture the realistic relation among labels. In this paper, we propose\na novel Label Confusion Model (LCM) as an enhancement component to current\npopular text classification models. LCM can learn label confusion to capture\nsemantic overlap among labels by calculating the similarity between instances\nand labels during training and generate a better label distribution to replace\nthe original one-hot label vector, thus improving the final classification\nperformance. Extensive experiments on five text classification benchmark\ndatasets reveal the effectiveness of LCM for several widely used deep learning\nclassification models. Further experiments also verify that LCM is especially\nhelpful for confused or noisy datasets and superior to the label smoothing\nmethod.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05011v3",
            "date": "2020-12-09 12:47:20+00:00",
            "title": "Infinite use of finite means: Zero-Shot Generalization using Compositional Emergent Protocols",
            "authors": [
                "Rishi Hazra",
                "Sonu Dixit",
                "Sayambhu Sen"
            ],
            "abstract": "Human language has been described as a system that makes \\textit{use of\nfinite means to express an unlimited array of thoughts}. Of particular interest\nis the aspect of compositionality, whereby, the meaning of a compound language\nexpression can be deduced from the meaning of its constituent parts. If\nartificial agents can develop compositional communication protocols akin to\nhuman language, they can be made to seamlessly generalize to unseen\ncombinations. However, the real question is, how do we induce compositionality\nin emergent communication? Studies have recognized the role of curiosity in\nenabling linguistic development in children. It is this same intrinsic urge\nthat drives us to master complex tasks with decreasing amounts of explicit\nreward. In this paper, we seek to use this intrinsic feedback in inducing a\nsystematic and unambiguous protolanguage in artificial agents. We show how\nthese rewards can be leveraged in training agents to induce compositionality in\nabsence of any external feedback. Additionally, we introduce gComm, an\nenvironment for investigating grounded language acquisition in 2D-grid\nenvironments. Using this, we demonstrate how compositionality can enable agents\nto not only interact with unseen objects but also transfer skills from one task\nto another in a zero-shot setting: \\textit{Can an agent, trained to `pull' and\n`push twice', `pull twice'?}.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05302v1",
            "date": "2020-12-09 20:38:17+00:00",
            "title": "Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU",
            "authors": [
                "Olga Golovneva",
                "Charith Peris"
            ],
            "abstract": "Data sparsity is one of the key challenges associated with model development\nin Natural Language Understanding (NLU) for conversational agents. The\nchallenge is made more complex by the demand for high quality annotated\nutterances commonly required for supervised learning, usually resulting in\nweeks of manual labor and high cost. In this paper, we present our results on\nboosting NLU model performance through training data augmentation using a\nsequential generative adversarial network (GAN). We explore data generation in\nthe context of two tasks, the bootstrapping of a new language and the handling\nof low resource features. For both tasks we explore three sequential GAN\narchitectures, one with a token-level reward function, another with our own\nimplementation of a token-level Monte Carlo rollout reward, and a third with\nsentence-level reward. We evaluate the performance of these feedback models\nacross several sampling methodologies and compare our results to upsampling the\noriginal data to the same scale. We further improve the GAN model performance\nthrough the transfer learning of the pretrained embeddings. Our experiments\nreveal synthetic data generated using the sequential generative adversarial\nnetwork provides significant performance boosts across multiple metrics and can\nbe a major benefit to the NLU tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05444v1",
            "date": "2020-12-10 04:06:25+00:00",
            "title": "A Framework for Generating Annotated Social Media Corpora with Demographics, Stance, Civility, and Topicality",
            "authors": [
                "Shubhanshu Mishra",
                "Daniel Collier"
            ],
            "abstract": "In this paper we introduce a framework for annotating a social media text\ncorpora for various categories. Since, social media data is generated via\nindividuals, it is important to annotate the text for the individuals\ndemographic attributes to enable a socio-technical analysis of the corpora.\nFurthermore, when analyzing a large data-set we can often annotate a small\nsample of data and then train a prediction model using this sample to annotate\nthe full data for the relevant categories. We use a case study of a Facebook\ncomment corpora on student loan discussion which was annotated for gender,\nmilitary affiliation, age-group, political leaning, race, stance, topicalilty,\nneoliberlistic views and civility of the comment. We release three datasets of\nFacebook comments for further research at:\nhttps://github.com/socialmediaie/StudentDebtFbComments",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05453v2",
            "date": "2020-12-10 04:59:12+00:00",
            "title": "Causal BERT : Language models for causality detection between events expressed in text",
            "authors": [
                "Vivek Khetan",
                "Roshni Ramnani",
                "Mayuresh Anand",
                "Shubhashis Sengupta",
                "Andrew E. Fano"
            ],
            "abstract": "Causality understanding between events is a critical natural language\nprocessing task that is helpful in many areas, including health care, business\nrisk management and finance. On close examination, one can find a huge amount\nof textual content both in the form of formal documents or in content arising\nfrom social media like Twitter, dedicated to communicating and exploring\nvarious types of causality in the real world. Recognizing these \"Cause-Effect\"\nrelationships between natural language events continues to remain a challenge\nsimply because it is often expressed implicitly. Implicit causality is hard to\ndetect through most of the techniques employed in literature and can also, at\ntimes be perceived as ambiguous or vague. Also, although well-known datasets do\nexist for this problem, the examples in them are limited in the range and\ncomplexity of the causal relationships they depict especially when related to\nimplicit relationships. Most of the contemporary methods are either based on\nlexico-semantic pattern matching or are feature-driven supervised methods.\nTherefore, as expected these methods are more geared towards handling explicit\ncausal relationships leading to limited coverage for implicit relationships and\nare hard to generalize. In this paper, we investigate the language model's\ncapabilities for causal association among events expressed in natural language\ntext using sentence context combined with event information, and by leveraging\nmasked event context with in-domain and out-of-domain data distribution. Our\nproposed methods achieve the state-of-art performance in three different data\ndistributions and can be leveraged for extraction of a causal diagram and/or\nbuilding a chain of events from unstructured text.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05489v1",
            "date": "2020-12-10 07:23:02+00:00",
            "title": "AI Driven Knowledge Extraction from Clinical Practice Guidelines: Turning Research into Practice",
            "authors": [
                "Musarrat Hussain",
                "Jamil Hussain",
                "Taqdir Ali",
                "Fahad Ahmed Satti",
                "Sungyoung Lee"
            ],
            "abstract": "Background and Objectives: Clinical Practice Guidelines (CPGs) represent the\nforemost methodology for sharing state-of-the-art research findings in the\nhealthcare domain with medical practitioners to limit practice variations,\nreduce clinical cost, improve the quality of care, and provide evidence based\ntreatment. However, extracting relevant knowledge from the plethora of CPGs is\nnot feasible for already burdened healthcare professionals, leading to large\ngaps between clinical findings and real practices. It is therefore imperative\nthat state-of-the-art Computing research, especially machine learning is used\nto provide artificial intelligence based solution for extracting the knowledge\nfrom CPGs and reducing the gap between healthcare research/guidelines and\npractice. Methods: This research presents a novel methodology for knowledge\nextraction from CPGs to reduce the gap and turn the latest research findings\ninto clinical practice. First, our system classifies the CPG sentences into\nfour classes such as condition-action, condition-consequences, action, and\nnot-applicable based on the information presented in a sentence. We use deep\nlearning with state-of-the-art word embedding, improved word vectors technique\nin classification process. Second, it identifies qualifier terms in the\nclassified sentences, which assist in recognizing the condition and action\nphrases in a sentence. Finally, the condition and action phrase are processed\nand transformed into plain rule If Condition(s) Then Action format. Results: We\nevaluate the methodology on three different domains guidelines including\nHypertension, Rhinosinusitis, and Asthma. The deep learning model classifies\nthe CPG sentences with an accuracy of 95%. While rule extraction was validated\nby user-centric approach, which achieved a Jaccard coefficient of 0.6, 0.7, and\n0.4 with three human experts extracted rules, respectively.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.05491v2",
            "date": "2020-12-10 07:31:07+00:00",
            "title": "An Event Correlation Filtering Method for Fake News Detection",
            "authors": [
                "Hao Li",
                "Huan Wang",
                "Guanghua Liu"
            ],
            "abstract": "Nowadays, social network platforms have been the prime source for people to\nexperience news and events due to their capacities to spread information\nrapidly, which inevitably provides a fertile ground for the dissemination of\nfake news. Thus, it is significant to detect fake news otherwise it could cause\npublic misleading and panic. Existing deep learning models have achieved great\nprogress to tackle the problem of fake news detection. However, training an\neffective deep learning model usually requires a large amount of labeled news,\nwhile it is expensive and time-consuming to provide sufficient labeled news in\nactual applications. To improve the detection performance of fake news, we take\nadvantage of the event correlations of news and propose an event correlation\nfiltering method (ECFM) for fake news detection, mainly consisting of the news\ncharacterizer, the pseudo label annotator, the event credibility updater, and\nthe news entropy selector. The news characterizer is responsible for extracting\ntextual features from news, which cooperates with the pseudo label annotator to\nassign pseudo labels for unlabeled news by fully exploiting the event\ncorrelations of news. In addition, the event credibility updater employs\nadaptive Kalman filter to weaken the credibility fluctuations of events. To\nfurther improve the detection performance, the news entropy selector\nautomatically discovers high-quality samples from pseudo labeled news by\nquantifying their news entropy. Finally, ECFM is proposed to integrate them to\ndetect fake news in an event correlation filtering manner. Extensive\nexperiments prove that the explainable introduction of the event correlations\nof news is beneficial to improve the detection performance of fake news.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05541v1",
            "date": "2020-12-10 09:31:59+00:00",
            "title": "Approches quantitatives de l'analyse des pr{\u00e9}dictions en traduction automatique neuronale (TAN)",
            "authors": [
                "Maria Zimina-Poirot",
                "Nicolas Ballier",
                "Jean-Baptiste Yun\u00e8s"
            ],
            "abstract": "As part of a larger project on optimal learning conditions in neural machine\ntranslation, we investigate characteristic training phases of translation\nengines. All our experiments are carried out using OpenNMT-Py: the\npre-processing step is implemented using the Europarl training corpus and the\nINTERSECT corpus is used for validation. Longitudinal analyses of training\nphases suggest that the progression of translations is not always linear.\nFollowing the results of textometric explorations, we identify the importance\nof the phenomena related to chronological progression, in order to map\ndifferent processes at work in neural machine translation (NMT).",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05742v2",
            "date": "2020-12-10 15:25:16+00:00",
            "title": "Longitudinal Citation Prediction using Temporal Graph Neural Networks",
            "authors": [
                "Andreas Nugaard Holm",
                "Barbara Plank",
                "Dustin Wright",
                "Isabelle Augenstein"
            ],
            "abstract": "Citation count prediction is the task of predicting the number of citations a\npaper has gained after a period of time. Prior work viewed this as a static\nprediction task. As papers and their citations evolve over time, considering\nthe dynamics of the number of citations a paper will receive would seem\nlogical. Here, we introduce the task of sequence citation prediction. The goal\nis to accurately predict the trajectory of the number of citations a scholarly\nwork receives over time. We propose to view papers as a structured network of\ncitations, allowing us to use topological information as a learning signal.\nAdditionally, we learn how this dynamic citation network changes over time and\nthe impact of paper meta-data such as authors, venues and abstracts. To\napproach the new task, we derive a dynamic citation network from Semantic\nScholar spanning over 42 years. We present a model which exploits topological\nand temporal information using graph convolution networks paired with sequence\nprediction, and compare it against multiple baselines, testing the importance\nof topological and temporal information and analyzing model performance. Our\nexperiments show that leveraging both the temporal and topological information\ngreatly increases the performance of predicting citation counts over time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.05818v1",
            "date": "2020-12-10 16:46:55+00:00",
            "title": "Bew: Towards Answering Business-Entity-Related Web Questions",
            "authors": [
                "Qingqing Cao",
                "Oriana Riva",
                "Aruna Balasubramanian",
                "Niranjan Balasubramanian"
            ],
            "abstract": "We present BewQA, a system specifically designed to answer a class of\nquestions that we call Bew questions. Bew questions are related to\nbusinesses/services such as restaurants, hotels, and movie theaters; for\nexample, \"Until what time is happy hour?\". These questions are challenging to\nanswer because the answers are found in open-domain Web, are present in short\nsentences without surrounding context, and are dynamic since the webpage\ninformation can be updated frequently. Under these conditions, existing QA\nsystems perform poorly. We present a practical approach, called BewQA, that can\nanswer Bew queries by mining a template of the business-related webpages and\nusing the template to guide the search. We show how we can extract the template\nautomatically by leveraging aggregator websites that aggregate information\nabout business entities in a domain (e.g., restaurants). We answer a given\nquestion by identifying the section from the extracted template that is most\nlikely to contain the answer. By doing so we can extract the answers even when\nthe answer span does not have sufficient context. Importantly, BewQA does not\nrequire any training. We crowdsource a new dataset of 1066 Bew questions and\nground-truth answers in the restaurant domain. Compared to state-of-the-art QA\nmodels, BewQA has a 27 percent point improvement in F1 score. Compared to a\ncommercial search engine, BewQA answered correctly 29% more Bew questions.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.05983v2",
            "date": "2020-12-10 21:17:04+00:00",
            "title": "Towards Neural Programming Interfaces",
            "authors": [
                "Zachary C. Brown",
                "Nathaniel Robinson",
                "David Wingate",
                "Nancy Fulda"
            ],
            "abstract": "It is notoriously difficult to control the behavior of artificial neural\nnetworks such as generative neural language models. We recast the problem of\ncontrolling natural language generation as that of learning to interface with a\npretrained language model, just as Application Programming Interfaces (APIs)\ncontrol the behavior of programs by altering hyperparameters. In this new\nparadigm, a specialized neural network (called a Neural Programming Interface\nor NPI) learns to interface with a pretrained language model by manipulating\nthe hidden activations of the pretrained model to produce desired outputs.\nImportantly, no permanent changes are made to the weights of the original\nmodel, allowing us to re-purpose pretrained models for new tasks without\noverwriting any aspect of the language model. We also contribute a new data set\nconstruction algorithm and GAN-inspired loss function that allows us to train\nNPI models to control outputs of autoregressive transformers. In experiments\nagainst other state-of-the-art approaches, we demonstrate the efficacy of our\nmethods using OpenAI's GPT-2 model, successfully controlling noun selection,\ntopic aversion, offensive speech filtering, and other aspects of language while\nlargely maintaining the controlled model's fluency under deterministic\nsettings.",
            "journal ref": "Neural Information Processing Systems 33 (2020) 17416-17428",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06048v2",
            "date": "2020-12-11 08:56:39+00:00",
            "title": "Reinforced Multi-Teacher Selection for Knowledge Distillation",
            "authors": [
                "Fei Yuan",
                "Linjun Shou",
                "Jian Pei",
                "Wutao Lin",
                "Ming Gong",
                "Yan Fu",
                "Daxin Jiang"
            ],
            "abstract": "In natural language processing (NLP) tasks, slow inference speed and huge\nfootprints in GPU usage remain the bottleneck of applying pre-trained deep\nmodels in production. As a popular method for model compression, knowledge\ndistillation transfers knowledge from one or multiple large (teacher) models to\na small (student) model. When multiple teacher models are available in\ndistillation, the state-of-the-art methods assign a fixed weight to a teacher\nmodel in the whole distillation. Furthermore, most of the existing methods\nallocate an equal weight to every teacher model. In this paper, we observe\nthat, due to the complexity of training examples and the differences in student\nmodel capability, learning differentially from teacher models can lead to\nbetter performance of student models distilled. We systematically develop a\nreinforced method to dynamically assign weights to teacher models for different\ntraining instances and optimize the performance of student model. Our extensive\nexperimental results on several NLP tasks clearly verify the feasibility and\neffectiveness of our approach.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06154v1",
            "date": "2020-12-11 06:31:42+00:00",
            "title": "ParsiNLU: A Suite of Language Understanding Challenges for Persian",
            "authors": [
                "Daniel Khashabi",
                "Arman Cohan",
                "Siamak Shakeri",
                "Pedram Hosseini",
                "Pouya Pezeshkpour",
                "Malihe Alikhani",
                "Moin Aminnaseri",
                "Marzieh Bitaab",
                "Faeze Brahman",
                "Sarik Ghazarian",
                "Mozhdeh Gheini",
                "Arman Kabiri",
                "Rabeeh Karimi Mahabadi",
                "Omid Memarrast",
                "Ahmadreza Mosallanezhad",
                "Erfan Noury",
                "Shahab Raji",
                "Mohammad Sadegh Rasooli",
                "Sepideh Sadeghi",
                "Erfan Sadeqi Azer",
                "Niloofar Safi Samghabadi",
                "Mahsa Shafaei",
                "Saber Sheybani",
                "Ali Tazarv",
                "Yadollah Yaghoobzadeh"
            ],
            "abstract": "Despite the progress made in recent years in addressing natural language\nunderstanding (NLU) challenges, the majority of this progress remains to be\nconcentrated on resource-rich languages like English. This work focuses on\nPersian language, one of the widely spoken languages in the world, and yet\nthere are few NLU datasets available for this rich language. The availability\nof high-quality evaluation datasets is a necessity for reliable assessment of\nthe progress on different NLU tasks and domains. We introduce ParsiNLU, the\nfirst benchmark in Persian language that includes a range of high-level tasks\n-- Reading Comprehension, Textual Entailment, etc. These datasets are collected\nin a multitude of ways, often involving manual annotations by native speakers.\nThis results in over 14.5$k$ new instances across 6 distinct NLU tasks.\nBesides, we present the first results on state-of-the-art monolingual and\nmulti-lingual pre-trained language-models on this benchmark and compare them\nwith human performance, which provides valuable insights into our ability to\ntackle natural language understanding challenges in Persian. We hope ParsiNLU\nfosters further research and advances in Persian language understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06259v1",
            "date": "2020-12-11 11:47:13+00:00",
            "title": "Improved Robustness to Disfluencies in RNN-Transducer Based Speech Recognition",
            "authors": [
                "Valentin Mendelev",
                "Tina Raissi",
                "Guglielmo Camporese",
                "Manuel Giollo"
            ],
            "abstract": "Automatic Speech Recognition (ASR) based on Recurrent Neural Network\nTransducers (RNN-T) is gaining interest in the speech community. We investigate\ndata selection and preparation choices aiming for improved robustness of RNN-T\nASR to speech disfluencies with a focus on partial words. For evaluation we use\nclean data, data with disfluencies and a separate dataset with speech affected\nby stuttering. We show that after including a small amount of data with\ndisfluencies in the training set the recognition accuracy on the tests with\ndisfluencies and stuttering improves. Increasing the amount of training data\nwith disfluencies gives additional gains without degradation on the clean data.\nWe also show that replacing partial words with a dedicated token helps to get\neven better accuracy on utterances with disfluencies and stutter. The\nevaluation of our best model shows 22.5% and 16.4% relative WER reduction on\nthose two evaluation sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.06340v2",
            "date": "2020-12-08 06:41:52+00:00",
            "title": "Control Flow Obfuscation for FJ using Continuation Passing",
            "authors": [
                "Kenny Zhuo Ming Lu"
            ],
            "abstract": "Control flow obfuscation deters software reverse engineering attempts by\naltering the program's control flow transfer. The alternation should not affect\nthe software's run-time behaviour. In this paper, we propose a control flow\nobfuscation approach for FJ with exception handling. The approach is based on a\nsource to source transformation using continuation passing style (CPS). We\nargue that the proposed CPS transformation causes malicious attacks using\ncontext insensitive static analysis and context sensitive analysis with fixed\ncall string to lose precision.",
            "journal ref": "None",
            "category": "cs.CR"
        },
        {
            "url": "http://arxiv.org/abs/2012.06836v1",
            "date": "2020-12-12 15:12:03+00:00",
            "title": "Source Code Classification for Energy Efficiency in Parallel Ultra Low-Power Microcontrollers",
            "authors": [
                "Emanuele Parisi",
                "Francesco Barchi",
                "Andrea Bartolini",
                "Giuseppe Tagliavini",
                "Andrea Acquaviva"
            ],
            "abstract": "The analysis of source code through machine learning techniques is an\nincreasingly explored research topic aiming at increasing smartness in the\nsoftware toolchain to exploit modern architectures in the best possible way. In\nthe case of low-power, parallel embedded architectures, this means finding the\nconfiguration, for instance in terms of the number of cores, leading to minimum\nenergy consumption. Depending on the kernel to be executed, the energy optimal\nscaling configuration is not trivial. While recent work has focused on\ngeneral-purpose systems to learn and predict the best execution target in terms\nof the execution time of a snippet of code or kernel (e.g. offload OpenCL\nkernel on multicore CPU or GPU), in this work we focus on static compile-time\nfeatures to assess if they can be successfully used to predict the minimum\nenergy configuration on PULP, an ultra-low-power architecture featuring an\non-chip cluster of RISC-V processors. Experiments show that using machine\nlearning models on the source code to select the best energy scaling\nconfiguration automatically is viable and has the potential to be used in the\ncontext of automatic system configuration for energy minimisation.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.06943v1",
            "date": "2020-12-13 02:34:56+00:00",
            "title": "Discriminative Pre-training for Low Resource Title Compression in Conversational Grocery",
            "authors": [
                "Snehasish Mukherjee",
                "Phaniram Sayapaneni",
                "Shankar Subramanya"
            ],
            "abstract": "The ubiquity of smart voice assistants has made conversational shopping\ncommonplace. This is especially true for low consideration segments like\ngrocery. A central problem in conversational grocery is the automatic\ngeneration of short product titles that can be read out fast during a\nconversation. Several supervised models have been proposed in the literature\nthat leverage manually labeled datasets and additional product features to\ngenerate short titles automatically. However, obtaining large amounts of\nlabeled data is expensive and most grocery item pages are not as feature-rich\nas other categories. To address this problem we propose a pre-training based\nsolution that makes use of unlabeled data to learn contextual product\nrepresentations which can then be fine-tuned to obtain better title compression\neven in a low resource setting. We use a self-attentive BiLSTM encoder network\nwith a time distributed softmax layer for the title compression task. We\novercome the vocabulary mismatch problem by using a hybrid embedding layer that\ncombines pre-trained word embeddings with trainable character level\nconvolutions. We pre-train this network as a discriminator on a replaced-token\ndetection task over a large number of unlabeled grocery product titles.\nFinally, we fine tune this network, without any modifications, with a small\nlabeled dataset for the title compression task. Experiments on Walmart's online\ngrocery catalog show our model achieves performance comparable to\nstate-of-the-art models like BERT and XLNet. When fine tuned on all of the\navailable training data our model attains an F1 score of 0.8558 which lags the\nbest performing model, BERT-Base, by 2.78% and XLNet by 0.28% only, while using\n55 times lesser parameters than both. Further, when allowed to fine tune on 5%\nof the training data only, our model outperforms BERT-Base by 24.3% in F1\nscore.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07000v1",
            "date": "2020-12-13 08:22:33+00:00",
            "title": "KVL-BERT: Knowledge Enhanced Visual-and-Linguistic BERT for Visual Commonsense Reasoning",
            "authors": [
                "Dandan Song",
                "Siyi Ma",
                "Zhanchen Sun",
                "Sicheng Yang",
                "Lejian Liao"
            ],
            "abstract": "Reasoning is a critical ability towards complete visual understanding. To\ndevelop machine with cognition-level visual understanding and reasoning\nabilities, the visual commonsense reasoning (VCR) task has been introduced. In\nVCR, given a challenging question about an image, a machine must answer\ncorrectly and then provide a rationale justifying its answer. The methods\nadopting the powerful BERT model as the backbone for learning joint\nrepresentation of image content and natural language have shown promising\nimprovements on VCR. However, none of the existing methods have utilized\ncommonsense knowledge in visual commonsense reasoning, which we believe will be\ngreatly helpful in this task. With the support of commonsense knowledge,\ncomplex questions even if the required information is not depicted in the image\ncan be answered with cognitive reasoning. Therefore, we incorporate commonsense\nknowledge into the cross-modal BERT, and propose a novel Knowledge Enhanced\nVisual-and-Linguistic BERT (KVL-BERT for short) model. Besides taking visual\nand linguistic contents as input, external commonsense knowledge extracted from\nConceptNet is integrated into the multi-layer Transformer. In order to reserve\nthe structural information and semantic representation of the original\nsentence, we propose using relative position embedding and mask-self-attention\nto weaken the effect between the injected commonsense knowledge and other\nunrelated components in the input sequence. Compared to other task-specific\nmodels and general task-agnostic pre-training models, our KVL-BERT outperforms\nthem by a large margin.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.07004v1",
            "date": "2020-12-13 08:35:37+00:00",
            "title": "C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Filling",
            "authors": [
                "Yutai Hou",
                "Sanyuan Chen",
                "Wanxiang Che",
                "Cheng Chen",
                "Ting Liu"
            ],
            "abstract": "Slot filling, a fundamental module of spoken language understanding, often\nsuffers from insufficient quantity and diversity of training data. To remedy\nthis, we propose a novel Cluster-to-Cluster generation framework for Data\nAugmentation (DA), named C2C-GenDA. It enlarges the training set by\nreconstructing existing utterances into alternative expressions while keeping\nsemantic. Different from previous DA works that reconstruct utterances one by\none independently, C2C-GenDA jointly encodes multiple existing utterances of\nthe same semantics and simultaneously decodes multiple unseen expressions.\nJointly generating multiple new utterances allows to consider the relations\nbetween generated instances and encourages diversity. Besides, encoding\nmultiple existing utterances endows C2C with a wider view of existing\nexpressions, helping to reduce generation that duplicates existing data.\nExperiments on ATIS and Snips datasets show that instances augmented by\nC2C-GenDA improve slot filling by 7.99 (11.9%) and 5.76 (13.6%) F-scores\nrespectively, when there are only hundreds of training utterances.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07011v1",
            "date": "2020-12-13 09:20:42+00:00",
            "title": "Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion",
            "authors": [
                "Ziyue Qiao",
                "Zhiyuan Ning",
                "Yi Du",
                "Yuanchun Zhou"
            ],
            "abstract": "Most researches for knowledge graph completion learn representations of\nentities and relations to predict missing links in incomplete knowledge graphs.\nHowever, these methods fail to take full advantage of both the contextual\ninformation of entity and relation. Here, we extract contexts of entities and\nrelations from the triplets which they compose. We propose a model named AggrE,\nwhich conducts efficient aggregations respectively on entity context and\nrelation context in multi-hops, and learns context-enhanced entity and relation\nembeddings for knowledge graph completion. The experiment results show that\nAggrE is competitive to existing models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07335v1",
            "date": "2020-12-14 08:39:38+00:00",
            "title": "LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding",
            "authors": [
                "Hao Fu",
                "Shaojun Zhou",
                "Qihong Yang",
                "Junjie Tang",
                "Guiquan Liu",
                "Kaikui Liu",
                "Xiaolong Li"
            ],
            "abstract": "The pre-training models such as BERT have achieved great results in various\nnatural language processing problems. However, a large number of parameters\nneed significant amounts of memory and the consumption of inference time, which\nmakes it difficult to deploy them on edge devices. In this work, we propose a\nknowledge distillation method LRC-BERT based on contrastive learning to fit the\noutput of the intermediate layer from the angular distance aspect, which is not\nconsidered by the existing distillation methods. Furthermore, we introduce a\ngradient perturbation-based training architecture in the training phase to\nincrease the robustness of LRC-BERT, which is the first attempt in knowledge\ndistillation. Additionally, in order to better capture the distribution\ncharacteristics of the intermediate layer, we design a two-stage training\nmethod for the total distillation loss. Finally, by verifying 8 datasets on the\nGeneral Language Understanding Evaluation (GLUE) benchmark, the performance of\nthe proposed LRC-BERT exceeds the existing state-of-the-art methods, which\nproves the effectiveness of our method.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07387v1",
            "date": "2020-12-14 10:17:25+00:00",
            "title": "A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings",
            "authors": [
                "Lisa van Staden",
                "Herman Kamper"
            ],
            "abstract": "Many speech processing tasks involve measuring the acoustic similarity\nbetween speech segments. Acoustic word embeddings (AWE) allow for efficient\ncomparisons by mapping speech segments of arbitrary duration to\nfixed-dimensional vectors. For zero-resource speech processing, where\nunlabelled speech is the only available resource, some of the best AWE\napproaches rely on weak top-down constraints in the form of automatically\ndiscovered word-like segments. Rather than learning embeddings at the segment\nlevel, another line of zero-resource research has looked at representation\nlearning at the short-time frame level. Recent approaches include\nself-supervised predictive coding and correspondence autoencoder (CAE) models.\nIn this paper we consider whether these frame-level features are beneficial\nwhen used as inputs for training to an unsupervised AWE model. We compare\nframe-level features from contrastive predictive coding (CPC), autoregressive\npredictive coding and a CAE to conventional MFCCs. These are used as inputs to\na recurrent CAE-based AWE model. In a word discrimination task on English and\nXitsonga data, all three representation learning approaches outperform MFCCs,\nwith CPC consistently showing the biggest improvement. In cross-lingual\nexperiments we find that CPC features trained on English can also be\ntransferred to Xitsonga.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07396v1",
            "date": "2020-12-14 10:30:51+00:00",
            "title": "Towards localisation of keywords in speech using weak supervision",
            "authors": [
                "Kayode Olaleye",
                "Benjamin van Niekerk",
                "Herman Kamper"
            ],
            "abstract": "Developments in weakly supervised and self-supervised models could enable\nspeech technology in low-resource settings where full transcriptions are not\navailable. We consider whether keyword localisation is possible using two forms\nof weak supervision where location information is not provided explicitly. In\nthe first, only the presence or absence of a word is indicated, i.e. a\nbag-of-words (BoW) labelling. In the second, visual context is provided in the\nform of an image paired with an unlabelled utterance; a model then needs to be\ntrained in a self-supervised fashion using the paired data. For keyword\nlocalisation, we adapt a saliency-based method typically used in the vision\ndomain. We compare this to an existing technique that performs localisation as\na part of the network architecture. While the saliency-based method is more\nflexible (it can be applied without architectural restrictions), we identify a\ncritical limitation when using it for keyword localisation. Of the two forms of\nsupervision, the visually trained model performs worse than the BoW-trained\nmodel. We show qualitatively that the visually trained model sometimes locate\nsemantically related words, but this is not consistent. While our results show\nthat there is some signal allowing for localisation, it also calls for other\nlocalisation methods better matched to these forms of weak supervision.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07463v1",
            "date": "2020-12-14 12:34:01+00:00",
            "title": "Parameter-Efficient Transfer Learning with Diff Pruning",
            "authors": [
                "Demi Guo",
                "Alexander M. Rush",
                "Yoon Kim"
            ],
            "abstract": "While task-specific finetuning of pretrained networks has led to significant\nempirical advances in NLP, the large size of networks makes finetuning\ndifficult to deploy in multi-task, memory-constrained settings. We propose diff\npruning as a simple approach to enable parameter-efficient transfer learning\nwithin the pretrain-finetune framework. This approach views finetuning as\nlearning a task-specific diff vector that is applied on top of the pretrained\nparameter vector, which remains fixed and is shared across different tasks. The\ndiff vector is adaptively pruned during training with a differentiable\napproximation to the L0-norm penalty to encourage sparsity. Diff pruning\nbecomes parameter-efficient as the number of tasks increases, as it requires\nstoring only the nonzero positions and weights of the diff vector for each\ntask, while the cost of storing the shared pretrained model remains constant.\nIt further does not require access to all tasks during training, which makes it\nattractive in settings where tasks arrive in stream or the set of tasks is\nunknown. We find that models finetuned with diff pruning can match the\nperformance of fully finetuned baselines on the GLUE benchmark while only\nmodifying 0.5% of the pretrained model's parameters per task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07490v1",
            "date": "2020-11-27 08:42:33+00:00",
            "title": "Machine Learning to study the impact of gender-based violence in the news media",
            "authors": [
                "Hugo J. Bello",
                "Nora Palomar",
                "Elisa Gallego",
                "Lourdes Jim\u00e9nez Navascu\u00e9s",
                "Celia Lozano"
            ],
            "abstract": "While it remains a taboo topic, gender-based violence (GBV) undermines the\nhealth, dignity, security and autonomy of its victims. Many factors have been\nstudied to generate or maintain this kind of violence, however, the influence\nof the media is still uncertain. Here, we use Machine Learning tools to\nextrapolate the effect of the news in GBV. By feeding neural networks with\nnews, the topic information associated with each article can be recovered. Our\nfindings show a relationship between GBV news and public awareness, the effect\nof mediatic GBV cases, and the intrinsic thematic relationship of GBV news.\nBecause the used neural model can be easily adjusted, this also allows us to\nextend our approach to other media sources or topics",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07515v1",
            "date": "2020-12-04 03:44:46+00:00",
            "title": "Data-Driven Regular Expressions Evolution for Medical Text Classification Using Genetic Programming",
            "authors": [
                "J Liu",
                "R Bai",
                "Z Lu",
                "P Ge",
                "D Liu",
                "Uwe Aickelin"
            ],
            "abstract": "In medical fields, text classification is one of the most important tasks\nthat can significantly reduce human workload through structured information\ndigitization and intelligent decision support. Despite the popularity of\nlearning-based text classification techniques, it is hard for human to\nunderstand or manually fine-tune the classification results for better\nprecision and recall, due to the black box nature of learning. This study\nproposes a novel regular expression-based text classification method making use\nof genetic programming (GP) approaches to evolve regular expressions that can\nclassify a given medical text inquiry with satisfactory precision and recall\nwhile allow human to read the classifier and fine-tune accordingly if\nnecessary. Given a seed population of regular expressions (can be randomly\ninitialized or manually constructed by experts), our method evolves a\npopulation of regular expressions according to chosen fitness function, using a\nnovel regular expression syntax and a series of carefully chosen reproduction\noperators. Our method is evaluated with real-life medical text inquiries from\nan online healthcare provider and shows promising performance. More\nimportantly, our method generates classifiers that can be fully understood,\nchecked and updated by medical doctors, which are fundamentally crucial for\nmedical related practices.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07534v1",
            "date": "2020-11-23 02:43:45+00:00",
            "title": "Effect of Word Embedding Models on Hate and Offensive Speech Detection",
            "authors": [
                "Safa Alsafari",
                "Samira Sadaoui",
                "Malek Mouhoub"
            ],
            "abstract": "Deep neural networks have been adopted successfully in hate speech detection\nproblems. Nevertheless, the effect of the word embedding models on the neural\nnetwork's performance has not been appropriately examined in the literature. In\nour study, through different detection tasks, 2-class, 3-class, and 6-class\nclassification, we investigate the impact of both word embedding models and\nneural network architectures on the predictive accuracy. Our focus is on the\nArabic language. We first train several word embedding models on a large-scale\nunlabelled Arabic text corpus. Next, based on a dataset of Arabic hate and\noffensive speech, for each detection task, we train several neural network\nclassifiers using the pre-trained word embedding models. This task yields a\nlarge number of various learned models, which allows conducting an exhaustive\ncomparison. The empirical analysis demonstrates, on the one hand, the\nsuperiority of the skip-gram models and, on the other hand, the superiority of\nthe CNN network across the three detection tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07536v1",
            "date": "2020-12-14 13:54:34+00:00",
            "title": "Movie Summarization via Sparse Graph Construction",
            "authors": [
                "Pinelopi Papalampidi",
                "Frank Keller",
                "Mirella Lapata"
            ],
            "abstract": "We summarize full-length movies by creating shorter videos containing their\nmost informative scenes. We explore the hypothesis that a summary can be\ncreated by assembling scenes which are turning points (TPs), i.e., key events\nin a movie that describe its storyline. We propose a model that identifies TP\nscenes by building a sparse movie graph that represents relations between\nscenes and is constructed using multimodal information. According to human\njudges, the summaries created by our approach are more informative and\ncomplete, and receive higher ratings, than the outputs of sequence-based models\nand general-purpose summarization algorithms. The induced graphs are\ninterpretable, displaying different topology for different movie genres.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07551v1",
            "date": "2020-12-14 14:17:33+00:00",
            "title": "Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks",
            "authors": [
                "Herman Kamper",
                "Benjamin van Niekerk"
            ],
            "abstract": "We investigate segmenting and clustering speech into low-bitrate phone-like\nsequences without supervision. We specifically constrain pretrained\nself-supervised vector-quantized (VQ) neural networks so that blocks of\ncontiguous feature vectors are assigned to the same code, thereby giving a\nvariable-rate segmentation of the speech into discrete units. Two segmentation\nmethods are considered. In the first, features are greedily merged until a\nprespecified number of segments are reached. The second uses dynamic\nprogramming to optimize a squared error with a penalty term to encourage fewer\nbut longer segments. We show that these VQ segmentation methods can be used\nwithout alteration across a wide range of tasks: unsupervised phone\nsegmentation, ABX phone discrimination, same-different word discrimination, and\nas inputs to a symbolic word segmentation algorithm. The penalized method\ngenerally performs best. While results are only comparable to the\nstate-of-the-art in some cases, in all tasks a reasonable competing approach is\noutperformed at a substantially lower bitrate.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07553v1",
            "date": "2020-12-11 04:58:13+00:00",
            "title": "An End-to-End Solution for Named Entity Recognition in eCommerce Search",
            "authors": [
                "Xiang Cheng",
                "Mitchell Bowden",
                "Bhushan Ramesh Bhange",
                "Priyanka Goyal",
                "Thomas Packer",
                "Faizan Javed"
            ],
            "abstract": "Named entity recognition (NER) is a critical step in modern search query\nunderstanding. In the domain of eCommerce, identifying the key entities, such\nas brand and product type, can help a search engine retrieve relevant products\nand therefore offer an engaging shopping experience. Recent research shows\npromising results on shared benchmark NER tasks using deep learning methods,\nbut there are still unique challenges in the industry regarding domain\nknowledge, training data, and model production. This paper demonstrates an\nend-to-end solution to address these challenges. The core of our solution is a\nnovel model training framework \"TripleLearn\" which iteratively learns from\nthree separate training datasets, instead of one training set as is\ntraditionally done. Using this approach, the best model lifts the F1 score from\n69.5 to 93.3 on the holdout test data. In our offline experiments, TripleLearn\nimproved the model performance compared to traditional training approaches\nwhich use a single set of training data. Moreover, in the online A/B test, we\nsee significant improvements in user engagement and revenue conversion. The\nmodel has been live on homedepot.com for more than 9 months, boosting search\nconversions and revenue. Beyond our application, this TripleLearn framework, as\nwell as the end-to-end process, is model-independent and problem-independent,\nso it can be generalized to more industrial applications, especially to the\neCommerce industry which has similar data foundations and problems.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07563v1",
            "date": "2020-12-10 06:51:13+00:00",
            "title": "A Practical Approach towards Causality Mining in Clinical Text using Active Transfer Learning",
            "authors": [
                "Musarrat Hussain",
                "Fahad Ahmed Satti",
                "Jamil Hussain",
                "Taqdir Ali",
                "Syed Imran Ali",
                "Hafiz Syed Muhammad Bilal",
                "Gwang Hoon Park",
                "Sungyoung Lee"
            ],
            "abstract": "Objective: Causality mining is an active research area, which requires the\napplication of state-of-the-art natural language processing techniques. In the\nhealthcare domain, medical experts create clinical text to overcome the\nlimitation of well-defined and schema driven information systems. The objective\nof this research work is to create a framework, which can convert clinical text\ninto causal knowledge. Methods: A practical approach based on term expansion,\nphrase generation, BERT based phrase embedding and semantic matching, semantic\nenrichment, expert verification, and model evolution has been used to construct\na comprehensive causality mining framework. This active transfer learning based\nframework along with its supplementary services, is able to extract and enrich,\ncausal relationships and their corresponding entities from clinical text.\nResults: The multi-model transfer learning technique when applied over multiple\niterations, gains performance improvements in terms of its accuracy and recall\nwhile keeping the precision constant. We also present a comparative analysis of\nthe presented techniques with their common alternatives, which demonstrate the\ncorrectness of our approach and its ability to capture most causal\nrelationships. Conclusion: The presented framework has provided cutting-edge\nresults in the healthcare domain. However, the framework can be tweaked to\nprovide causality detection in other domains, as well. Significance: The\npresented framework is generic enough to be utilized in any domain, healthcare\nservices can gain massive benefits due to the voluminous and various nature of\nits data. This causal knowledge extraction framework can be used to summarize\nclinical text, create personas, discover medical knowledge, and provide\nevidence to clinical decision making.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07575v2",
            "date": "2020-12-08 20:14:54+00:00",
            "title": "Large-scale Quantitative Evidence of Media Impact on Public Opinion toward China",
            "authors": [
                "Junming Huang",
                "Gavin Cook",
                "Yu Xie"
            ],
            "abstract": "Do mass media influence people's opinion of other countries? Using BERT, a\ndeep neural network-based natural language processing model, we analyze a large\ncorpus of 267,907 China-related articles published by The New York Times since\n1970. We then compare our output from The New York Times to a longitudinal data\nset constructed from 101 cross-sectional surveys of the American public's views\non China. We find that the reporting of The New York Times on China in one year\nexplains 54% of the variance in American public opinion on China in the next.\nOur result confirms hypothesized links between media and public opinion and\nhelps shed light on how mass media can influence public opinion of foreign\ncountries.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07580v2",
            "date": "2020-12-04 14:03:03+00:00",
            "title": "Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings",
            "authors": [
                "Na Li",
                "Zied Bouraoui",
                "Jose Camacho Collados",
                "Luis Espinosa-Anke",
                "Qing Gu",
                "Steven Schockaert"
            ],
            "abstract": "While the success of pre-trained language models has largely eliminated the\nneed for high-quality static word vectors in many NLP applications, such\nvectors continue to play an important role in tasks where words need to be\nmodelled in the absence of linguistic context. In this paper, we explore how\nthe contextualised embeddings predicted by BERT can be used to produce\nhigh-quality word vectors for such domains, in particular related to knowledge\nbase completion, where our focus is on capturing the semantic properties of\nnouns. We find that a simple strategy of averaging the contextualised\nembeddings of masked word mentions leads to vectors that outperform the static\nword vectors learned by BERT, as well as those from standard word embedding\nmodels, in property induction tasks. We notice in particular that masking\ntarget words is critical to achieve this strong performance, as the resulting\nvectors focus less on idiosyncratic properties and more on general semantic\nproperties. Inspired by this view, we propose a filtering strategy which is\naimed at removing the most idiosyncratic mention vectors, allowing us to obtain\nfurther performance gains in property induction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07589v1",
            "date": "2020-12-08 13:37:40+00:00",
            "title": "Incorporating Domain Knowledge To Improve Topic Segmentation Of Long MOOC Lecture Videos",
            "authors": [
                "Ananda Das",
                "Partha Pratim Das"
            ],
            "abstract": "Topical Segmentation poses a great role in reducing search space of the\ntopics taught in a lecture video specially when the video metadata lacks topic\nwise segmentation information. This segmentation information eases user efforts\nof searching, locating and browsing a topic inside a lecture video. In this\nwork we propose an algorithm, that combines state-of-the art language model and\ndomain knowledge graph for automatically detecting different coherent topics\npresent inside a long lecture video. We use the language model on\nspeech-to-text transcription to capture the implicit meaning of the whole video\nwhile the knowledge graph provides us the domain specific dependencies between\ndifferent concepts of that subjects. Also leveraging the domain knowledge we\ncan capture the way instructor binds and connects different concepts while\nteaching, which helps us in achieving better segmentation accuracy. We tested\nour approach on NPTEL lecture videos and holistic evaluation shows that it out\nperforms the other methods described in the literature.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07609v1",
            "date": "2020-12-14 14:59:08+00:00",
            "title": "Clickbait in Hindi News Media : A Preliminary Study",
            "authors": [
                "Vivek Kaushal",
                "Kavita Vemuri"
            ],
            "abstract": "A corpus of Hindi news headlines shared on Twitter was created by collecting\ntweets of 5 mainstream Hindi news sources for a period of 4 months. 7\nindependent annotators were recruited to mark the 20 most retweeted news posts\nby each of the 5 news sources on its clickbait nature. The clickbait score\nhence generated was assessed for its correlation with interactions on the\nplatform (retweets, favorites, reader replies), tweet word count, and\nnormalized POS (part-of-speech) tag counts in tweets. A positive correlation\nwas observed between readers' interactions with tweets and tweets' clickbait\nscore. Significant correlations were also observed for POS tag counts and\nclickbait score. The prevalence of clickbait in mainstream Hindi news media was\nfound to be similar to its prevalence in English news media. We hope that our\nobservations would provide a platform for discussions on clickbait in\nmainstream Hindi news media.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07610v1",
            "date": "2020-12-14 15:02:08+00:00",
            "title": "Time to Transfer: Predicting and Evaluating Machine-Human Chatting Handoff",
            "authors": [
                "Jiawei Liu",
                "Zhe Gao",
                "Yangyang Kang",
                "Zhuoren Jiang",
                "Guoxiu He",
                "Changlong Sun",
                "Xiaozhong Liu",
                "Wei Lu"
            ],
            "abstract": "Is chatbot able to completely replace the human agent? The short answer could\nbe - \"it depends...\". For some challenging cases, e.g., dialogue's topical\nspectrum spreads beyond the training corpus coverage, the chatbot may\nmalfunction and return unsatisfied utterances. This problem can be addressed by\nintroducing the Machine-Human Chatting Handoff (MHCH), which enables\nhuman-algorithm collaboration. To detect the normal/transferable utterances, we\npropose a Difficulty-Assisted Matching Inference (DAMI) network, utilizing\ndifficulty-assisted encoding to enhance the representations of utterances.\nMoreover, a matching inference mechanism is introduced to capture the\ncontextual matching features. A new evaluation metric, Golden Transfer within\nTolerance (GT-T), is proposed to assess the performance by considering the\ntolerance property of the MHCH. To provide insights into the task and validate\nthe proposed model, we collect two new datasets. Extensive experimental results\nare presented and contrasted against a series of baseline models to demonstrate\nthe efficacy of our model on MHCH.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07701v1",
            "date": "2020-12-09 01:41:35+00:00",
            "title": "Simple or Complex? Learning to Predict Readability of Bengali Texts",
            "authors": [
                "Susmoy Chakraborty",
                "Mir Tafseer Nayeem",
                "Wasi Uddin Ahmad"
            ],
            "abstract": "Determining the readability of a text is the first step to its\nsimplification. In this paper, we present a readability analysis tool capable\nof analyzing text written in the Bengali language to provide in-depth\ninformation on its readability and complexity. Despite being the 7th most\nspoken language in the world with 230 million native speakers, Bengali suffers\nfrom a lack of fundamental resources for natural language processing.\nReadability related research of the Bengali language so far can be considered\nto be narrow and sometimes faulty due to the lack of resources. Therefore, we\ncorrectly adopt document-level readability formulas traditionally used for U.S.\nbased education system to the Bengali language with a proper age-to-age\ncomparison. Due to the unavailability of large-scale human-annotated corpora,\nwe further divide the document-level task into sentence-level and experiment\nwith neural architectures, which will serve as a baseline for the future works\nof Bengali readability prediction. During the process, we present several\nhuman-annotated corpora and dictionaries such as a document-level dataset\ncomprising 618 documents with 12 different grade levels, a large-scale\nsentence-level dataset comprising more than 96K sentences with simple and\ncomplex labels, a consonant conjunct count algorithm and a corpus of 341 words\nto validate the effectiveness of the algorithm, a list of 3,396 easy words, and\nan updated pronunciation dictionary with more than 67K words. These resources\ncan be useful for several other tasks of this low-resource language. We make\nour Code & Dataset publicly available at\nhttps://github.com/tafseer-nayeem/BengaliReadability} for reproduciblity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.07749v1",
            "date": "2020-12-02 14:55:22+00:00",
            "title": "Towards Fairness in Classifying Medical Conversations into SOAP Sections",
            "authors": [
                "Elisa Ferracane",
                "Sandeep Konam"
            ],
            "abstract": "As machine learning algorithms are more widely deployed in healthcare, the\nquestion of algorithmic fairness becomes more critical to examine. Our work\nseeks to identify and understand disparities in a deployed model that\nclassifies doctor-patient conversations into sections of a medical SOAP note.\nWe employ several metrics to measure disparities in the classifier performance,\nand find small differences in a portion of the disadvantaged groups. A deeper\nanalysis of the language in these conversations and further stratifying the\ngroups suggests these differences are related to and often attributable to the\ntype of medical appointment (e.g., psychiatric vs. internist). Our findings\nstress the importance of understanding the disparities that may exist in the\ndata itself and how that affects a model's ability to equally distribute\nbenefits.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2012.07974v2",
            "date": "2020-12-14 22:18:08+00:00",
            "title": "A review of on-device fully neural end-to-end automatic speech recognition algorithms",
            "authors": [
                "Chanwoo Kim",
                "Dhananjaya Gowda",
                "Dongsoo Lee",
                "Jiyeon Kim",
                "Ankur Kumar",
                "Sungsoo Kim",
                "Abhinav Garg",
                "Changwoo Han"
            ],
            "abstract": "In this paper, we review various end-to-end automatic speech recognition\nalgorithms and their optimization techniques for on-device applications.\nConventional speech recognition systems comprise a large number of discrete\ncomponents such as an acoustic model, a language model, a pronunciation model,\na text-normalizer, an inverse-text normalizer, a decoder based on a Weighted\nFinite State Transducer (WFST), and so on. To obtain sufficiently high speech\nrecognition accuracy with such conventional speech recognition systems, a very\nlarge language model (up to 100 GB) is usually needed. Hence, the corresponding\nWFST size becomes enormous, which prohibits their on-device implementation.\nRecently, fully neural network end-to-end speech recognition algorithms have\nbeen proposed. Examples include speech recognition systems based on\nConnectionist Temporal Classification (CTC), Recurrent Neural Network\nTransducer (RNN-T), Attention-based Encoder-Decoder models (AED), Monotonic\nChunk-wise Attention (MoChA), transformer-based speech recognition systems, and\nso on. These fully neural network-based systems require much smaller memory\nfootprints compared to conventional algorithms, therefore their on-device\nimplementation has become feasible. In this paper, we review such end-to-end\nspeech recognition models. We extensively discuss their structures,\nperformance, and advantages compared to conventional algorithms.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.08013v2",
            "date": "2020-12-14 23:59:05+00:00",
            "title": "Primer AI's Systems for Acronym Identification and Disambiguation",
            "authors": [
                "Nicholas Egan",
                "John Bohannon"
            ],
            "abstract": "The prevalence of ambiguous acronyms make scientific documents harder to\nunderstand for humans and machines alike, presenting a need for models that can\nautomatically identify acronyms in text and disambiguate their meaning. We\nintroduce new methods for acronym identification and disambiguation: our\nacronym identification model projects learned token embeddings onto tag\npredictions, and our acronym disambiguation model finds training examples with\nsimilar sentence embeddings as test examples. Both of our systems achieve\nsignificant performance gains over previously suggested methods, and perform\ncompetitively on the SDU@AAAI-21 shared task leaderboard. Our models were\ntrained in part on new distantly-supervised datasets for these tasks which we\ncall AuxAI and AuxAD. We also identified a duplication conflict issue in the\nSciAD dataset, and formed a deduplicated version of SciAD that we call\nSciAD-dedupe. We publicly released all three of these datasets, and hope that\nthey help the community make further strides in scientific document\nunderstanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08020v3",
            "date": "2020-12-15 00:35:41+00:00",
            "title": "Traditional IR rivals neural models on the MS MARCO Document Ranking Leaderboard",
            "authors": [
                "Leonid Boytsov"
            ],
            "abstract": "This short document describes a traditional IR system that achieved MRR@100\nequal to 0.298 on the MS MARCO Document Ranking leaderboard (on 2020-12-06).\nAlthough inferior to most BERT-based models, it outperformed several neural\nruns (as well as all non-neural ones), including two submissions that used a\nlarge pretrained Transformer model for re-ranking. We provide software and data\nto reproduce our results.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08113v1",
            "date": "2020-12-15 06:31:38+00:00",
            "title": "Enriched Annotations for Tumor Attribute Classification from Pathology Reports with Limited Labeled Data",
            "authors": [
                "Nick Altieri",
                "Briton Park",
                "Mara Olson",
                "John DeNero",
                "Anobel Odisho",
                "Bin Yu"
            ],
            "abstract": "Precision medicine has the potential to revolutionize healthcare, but much of\nthe data for patients is locked away in unstructured free-text, limiting\nresearch and delivery of effective personalized treatments. Generating large\nannotated datasets for information extraction from clinical notes is often\nchallenging and expensive due to the high level of expertise needed for high\nquality annotations. To enable natural language processing for small dataset\nsizes, we develop a novel enriched hierarchical annotation scheme and\nalgorithm, Supervised Line Attention (SLA), and apply this algorithm to\npredicting categorical tumor attributes from kidney and colon cancer pathology\nreports from the University of California San Francisco (UCSF). Whereas\nprevious work only annotated document level labels, we in addition ask the\nannotators to enrich the traditional label by asking them to also highlight the\nrelevant line or potentially lines for the final label, which leads to a 20%\nincrease of annotation time required per document. With the enriched\nannotations, we develop a simple and interpretable machine learning algorithm\nthat first predicts the relevant lines in the document and then predicts the\ntumor attribute. Our results show across the small dataset sizes of 32, 64,\n128, and 186 labeled documents per cancer, SLA only requires half the number of\nlabeled documents as state-of-the-art methods to achieve similar or better\nmicro-f1 and macro-f1 scores for the vast majority of comparisons that we made.\nAccounting for the increased annotation time, this leads to a 40% reduction in\ntotal annotation time over the state of the art.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08128v1",
            "date": "2020-12-15 07:22:39+00:00",
            "title": "Relation-Aware Neighborhood Matching Model for Entity Alignment",
            "authors": [
                "Yao Zhu",
                "Hongzhi Liu",
                "Zhonghai Wu",
                "Yingpeng Du"
            ],
            "abstract": "Entity alignment which aims at linking entities with the same meaning from\ndifferent knowledge graphs (KGs) is a vital step for knowledge fusion. Existing\nresearch focused on learning embeddings of entities by utilizing structural\ninformation of KGs for entity alignment. These methods can aggregate\ninformation from neighboring nodes but may also bring noise from neighbors.\nMost recently, several researchers attempted to compare neighboring nodes in\npairs to enhance the entity alignment. However, they ignored the relations\nbetween entities which are also important for neighborhood matching. In\naddition, existing methods paid less attention to the positive interactions\nbetween the entity alignment and the relation alignment. To deal with these\nissues, we propose a novel Relation-aware Neighborhood Matching model named RNM\nfor entity alignment. Specifically, we propose to utilize the neighborhood\nmatching to enhance the entity alignment. Besides comparing neighbor nodes when\nmatching neighborhood, we also try to explore useful information from the\nconnected relations. Moreover, an iterative framework is designed to leverage\nthe positive interactions between the entity alignment and the relation\nalignment in a semi-supervised manner. Experimental results on three real-world\ndatasets demonstrate that the proposed model RNM performs better than\nstate-of-the-art methods.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.08148v1",
            "date": "2020-12-15 08:35:58+00:00",
            "title": "A Response Retrieval Approach for Dialogue Using a Multi-Attentive Transformer",
            "authors": [
                "Matteo A. Senese",
                "Alberto Benincasa",
                "Barbara Caputo",
                "Giuseppe Rizzo"
            ],
            "abstract": "This paper presents our work for the ninth edition of the Dialogue System\nTechnology Challenge (DSTC9). Our solution addresses the track number four:\nSimulated Interactive MultiModal Conversations. The task consists in providing\nan algorithm able to simulate a shopping assistant that supports the user with\nhis/her requests. We address the task of response retrieval, that is the task\nof retrieving the most appropriate agent response from a pool of response\ncandidates. Our approach makes use of a neural architecture based on\ntransformer with a multi-attentive structure that conditions the response of\nthe agent on the request made by the user and on the product the user is\nreferring to. Final experiments on the SIMMC Fashion Dataset show that our\napproach achieves the second best scores on all the retrieval metrics defined\nby the organizers. The source code is available at\nhttps://github.com/D2KLab/dstc9-SIMMC.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08206v1",
            "date": "2020-12-15 10:52:19+00:00",
            "title": "Efficient Clustering from Distributions over Topics",
            "authors": [
                "Carlos Badenes-Olmedo",
                "Jose-Luis Redondo Garc\u00eda",
                "Oscar Corcho"
            ],
            "abstract": "There are many scenarios where we may want to find pairs of textually similar\ndocuments in a large corpus (e.g. a researcher doing literature review, or an\nR&D project manager analyzing project proposals). To programmatically discover\nthose connections can help experts to achieve those goals, but brute-force\npairwise comparisons are not computationally adequate when the size of the\ndocument corpus is too large. Some algorithms in the literature divide the\nsearch space into regions containing potentially similar documents, which are\nlater processed separately from the rest in order to reduce the number of pairs\ncompared. However, this kind of unsupervised methods still incur in high\ntemporal costs. In this paper, we present an approach that relies on the\nresults of a topic modeling algorithm over the documents in a collection, as a\nmeans to identify smaller subsets of documents where the similarity function\ncan then be computed. This approach has proved to obtain promising results when\nidentifying similar documents in the domain of scientific publications. We have\ncompared our approach against state of the art clustering techniques and with\ndifferent configurations for the topic modeling algorithm. Results suggest that\nour approach outperforms (> 0.5) the other analyzed techniques in terms of\nefficiency.",
            "journal ref": "ACM Proceedings of the Knowledge Capture Conference, article 17,\n  K-CAP 2017",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08266v1",
            "date": "2020-12-15 13:01:26+00:00",
            "title": "*-CFQ: Analyzing the Scalability of Machine Learning on a Compositional Task",
            "authors": [
                "Dmitry Tsarkov",
                "Tibor Tihon",
                "Nathan Scales",
                "Nikola Momchev",
                "Danila Sinopalnikov",
                "Nathanael Sch\u00e4rli"
            ],
            "abstract": "We present *-CFQ (\"star-CFQ\"): a suite of large-scale datasets of varying\nscope based on the CFQ semantic parsing benchmark, designed for principled\ninvestigation of the scalability of machine learning systems in a realistic\ncompositional task setting. Using this suite, we conduct a series of\nexperiments investigating the ability of Transformers to benefit from increased\ntraining size under conditions of fixed computational cost. We show that\ncompositional generalization remains a challenge at all training sizes, and we\nshow that increasing the scope of natural language leads to consistently higher\nerror rates, which are only partially offset by increased training data. We\nfurther show that while additional training data from a related domain improves\nthe accuracy in data-starved situations, this improvement is limited and\ndiminishes as the distance from the related domain to the target domain\nincreases.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.08290v1",
            "date": "2020-12-15 13:57:21+00:00",
            "title": "Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution",
            "authors": [
                "Ron Zhu"
            ],
            "abstract": "Hateful meme detection is a new research area recently brought out that\nrequires both visual, linguistic understanding of the meme and some background\nknowledge to performing well on the task. This technical report summarises the\nfirst place solution of the Hateful Meme Detection Challenge 2020, which\nextending state-of-the-art visual-linguistic transformers to tackle this\nproblem. At the end of the report, we also point out the shortcomings and\npossible directions for improving the current methodology.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08377v2",
            "date": "2020-12-15 15:47:30+00:00",
            "title": "CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts",
            "authors": [
                "Peixiang Zhong",
                "Di Wang",
                "Pengfei Li",
                "Chen Zhang",
                "Hao Wang",
                "Chunyan Miao"
            ],
            "abstract": "Rationality and emotion are two fundamental elements of humans. Endowing\nagents with rationality and emotion has been one of the major milestones in AI.\nHowever, in the field of conversational AI, most existing models only\nspecialize in one aspect and neglect the other, which often leads to dull or\nunrelated responses. In this paper, we hypothesize that combining rationality\nand emotion into conversational agents can improve response quality. To test\nthe hypothesis, we focus on one fundamental aspect of rationality, i.e.,\ncommonsense, and propose CARE, a novel model for commonsense-aware emotional\nresponse generation. Specifically, we first propose a framework to learn and\nconstruct commonsense-aware emotional latent concepts of the response given an\ninput message and a desired emotion. We then propose three methods to\ncollaboratively incorporate the latent concepts into response generation.\nExperimental results on two large-scale datasets support our hypothesis and\nshow that our model can produce more accurate and commonsense-aware emotional\nresponses and achieve better human ratings than state-of-the-art models that\nonly specialize in one aspect.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08383v3",
            "date": "2020-12-15 15:55:32+00:00",
            "title": "Keyword-Guided Neural Conversational Model",
            "authors": [
                "Peixiang Zhong",
                "Yong Liu",
                "Hao Wang",
                "Chunyan Miao"
            ],
            "abstract": "We study the problem of imposing conversational goals/keywords on open-domain\nconversational agents, where the agent is required to lead the conversation to\na target keyword smoothly and fast. Solving this problem enables the\napplication of conversational agents in many real-world scenarios, e.g.,\nrecommendation and psychotherapy. The dominant paradigm for tackling this\nproblem is to 1) train a next-turn keyword classifier, and 2) train a\nkeyword-augmented response retrieval model. However, existing approaches in\nthis paradigm have two limitations: 1) the training and evaluation datasets for\nnext-turn keyword classification are directly extracted from conversations\nwithout human annotations, thus, they are noisy and have low correlation with\nhuman judgements, and 2) during keyword transition, the agents solely rely on\nthe similarities between word embeddings to move closer to the target keyword,\nwhich may not reflect how humans converse. In this paper, we assume that human\nconversations are grounded on commonsense and propose a keyword-guided neural\nconversational model that can leverage external commonsense knowledge graphs\n(CKG) for both keyword transition and response retrieval. Automatic evaluations\nsuggest that commonsense improves the performance of both next-turn keyword\nprediction and keyword-augmented response retrieval. In addition, both\nself-play and human evaluations show that our model produces responses with\nsmoother keyword transition and reaches the target keyword faster than\ncompetitive baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08407v1",
            "date": "2020-12-15 16:34:36+00:00",
            "title": "Multi-Aspect Sentiment Analysis with Latent Sentiment-Aspect Attribution",
            "authors": [
                "Yifan Zhang",
                "Fan Yang",
                "Marjan Hosseinia",
                "Arjun Mukherjee"
            ],
            "abstract": "In this paper, we introduce a new framework called the sentiment-aspect\nattribution module (SAAM). SAAM works on top of traditional neural networks and\nis designed to address the problem of multi-aspect sentiment classification and\nsentiment regression. The framework works by exploiting the correlations\nbetween sentence-level embedding features and variations of document-level\naspect rating scores. We demonstrate several variations of our framework on top\nof CNN and RNN based models. Experiments on a hotel review dataset and a beer\nreview dataset have shown SAAM can improve sentiment analysis performance over\ncorresponding base models. Moreover, because of the way our framework\nintuitively combines sentence-level scores into document-level scores, it is\nable to provide a deeper insight into data (e.g., semi-supervised sentence\naspect labeling). Hence, we end the paper with a detailed analysis that shows\nthe potential of our models for other applications such as sentiment snippet\nextraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08673v2",
            "date": "2020-12-15 23:41:42+00:00",
            "title": "A Closer Look at the Robustness of Vision-and-Language Pre-trained Models",
            "authors": [
                "Linjie Li",
                "Zhe Gan",
                "Jingjing Liu"
            ],
            "abstract": "Large-scale pre-trained multimodal transformers, such as ViLBERT and UNITER,\nhave propelled the state of the art in vision-and-language (V+L) research to a\nnew level. Although achieving impressive performance on standard tasks, to\ndate, it still remains unclear how robust these pre-trained models are. To\ninvestigate, we conduct a host of thorough evaluations on existing pre-trained\nmodels over 4 different types of V+L specific model robustness: (i) Linguistic\nVariation; (ii) Logical Reasoning; (iii) Visual Content Manipulation; and (iv)\nAnswer Distribution Shift. Interestingly, by standard model finetuning,\npre-trained V+L models already exhibit better robustness than many\ntask-specific state-of-the-art methods. To further enhance model robustness, we\npropose Mango, a generic and efficient approach that learns a Multimodal\nAdversarial Noise GeneratOr in the embedding space to fool pre-trained V+L\nmodels. Differing from previous studies focused on one specific type of\nrobustness, Mango is task-agnostic, and enables universal performance lift for\npre-trained models over diverse tasks designed to evaluate broad aspects of\nrobustness. Comprehensive experiments demonstrate that Mango achieves new state\nof the art on 7 out of 9 robustness benchmarks, surpassing existing methods by\na significant margin. As the first comprehensive study on V+L robustness, this\nwork puts robustness of pre-trained models into sharper focus, pointing new\ndirections for future study.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.08743v1",
            "date": "2020-12-16 04:43:43+00:00",
            "title": "Improving Multilingual Neural Machine Translation For Low-Resource Languages: French-, English- Vietnamese",
            "authors": [
                "Thi-Vinh Ngo",
                "Phuong-Thai Nguyen",
                "Thanh-Le Ha",
                "Khac-Quy Dinh",
                "Le-Minh Nguyen"
            ],
            "abstract": "Prior works have demonstrated that a low-resource language pair can benefit\nfrom multilingual machine translation (MT) systems, which rely on many language\npairs' joint training. This paper proposes two simple strategies to address the\nrare word issue in multilingual MT systems for two low-resource language pairs:\nFrench-Vietnamese and English-Vietnamese. The first strategy is about dynamical\nlearning word similarity of tokens in the shared space among source languages\nwhile another one attempts to augment the translation ability of rare words\nthrough updating their embeddings during the training. Besides, we leverage\nmonolingual data for multilingual MT systems to increase the amount of\nsynthetic parallel corpora while dealing with the data sparsity problem. We\nhave shown significant improvements of up to +1.62 and +2.54 BLEU points over\nthe bilingual baseline systems for both language pairs and released our\ndatasets for the research community.",
            "journal ref": "The 3rd Workshop on Technologies for MT of Low Resource Languages\n  (LoResMT 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08773v1",
            "date": "2020-12-16 07:26:43+00:00",
            "title": "Building domain specific lexicon based on TikTok comment dataset",
            "authors": [
                "Hao Jiaxiang"
            ],
            "abstract": "In the sentiment analysis task, predicting the sentiment tendency of a\nsentence is an important branch. Previous research focused more on sentiment\nanalysis in English, for example, analyzing the sentiment tendency of sentences\nbased on Valence, Arousal, Dominance of sentences. the emotional tendency is\ndifferent between the two languages. For example, the sentence order between\nChinese and English may present different emotions. This paper tried a method\nthat builds a domain-specific lexicon. In this way, the model can classify\nChinese words with emotional tendency. In this approach, based on the [13], an\nultra-dense space embedding table is trained through word embedding of Chinese\nTikTok review and emotional lexicon sources(seed words). The result of the\nmodel is a domain-specific lexicon, which presents the emotional tendency of\nwords. I collected Chinese TikTok comments as training data. By comparing The\ntraining results with the PCA method to evaluate the performance of the model\nin Chinese sentiment classification, the results show that the model has done\nwell in Chinese. The source code has released on\ngithub:https://github.com/h2222/douyin_comment_dataset",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08790v1",
            "date": "2020-12-16 08:23:03+00:00",
            "title": "Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference",
            "authors": [
                "Yichao Zhou",
                "Yu Yan",
                "Rujun Han",
                "J. Harry Caufield",
                "Kai-Wei Chang",
                "Yizhou Sun",
                "Peipei Ping",
                "Wei Wang"
            ],
            "abstract": "There has been a steady need in the medical community to precisely extract\nthe temporal relations between clinical events. In particular, temporal\ninformation can facilitate a variety of downstream applications such as case\nreport retrieval and medical question answering. Existing methods either\nrequire expensive feature engineering or are incapable of modeling the global\nrelational dependencies among the events. In this paper, we propose a novel\nmethod, Clinical Temporal ReLation Exaction with Probabilistic Soft Logic\nRegularization and Global Inference (CTRL-PG) to tackle the problem at the\ndocument level. Extensive experiments on two benchmark datasets, I2B2-2012 and\nTB-Dense, demonstrate that CTRL-PG significantly outperforms baseline methods\nfor temporal relation extraction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08920v1",
            "date": "2020-12-16 13:11:30+00:00",
            "title": "R$^2$-Net: Relation of Relation Learning Network for Sentence Semantic Matching",
            "authors": [
                "Kun Zhang",
                "Le Wu",
                "Guangyi Lv",
                "Meng Wang",
                "Enhong Chen",
                "Shulan Ruan"
            ],
            "abstract": "Sentence semantic matching is one of the fundamental tasks in natural\nlanguage processing, which requires an agent to determine the semantic relation\namong input sentences. Recently, deep neural networks have achieved impressive\nperformance in this area, especially BERT. Despite the effectiveness of these\nmodels, most of them treat output labels as meaningless one-hot vectors,\nunderestimating the semantic information and guidance of relations that these\nlabels reveal, especially for tasks with a small number of labels. To address\nthis problem, we propose a Relation of Relation Learning Network (R2-Net) for\nsentence semantic matching. Specifically, we first employ BERT to encode the\ninput sentences from a global perspective. Then a CNN-based encoder is designed\nto capture keywords and phrase information from a local perspective. To fully\nleverage labels for better relation information extraction, we introduce a\nself-supervised relation of relation classification task for guiding R2-Net to\nconsider more about labels. Meanwhile, a triplet loss is employed to\ndistinguish the intra-class and inter-class relations in a finer granularity.\nEmpirical experiments on two sentence semantic matching tasks demonstrate the\nsuperiority of our proposed model. As a byproduct, we have released the codes\nto facilitate other researches.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.08987v7",
            "date": "2020-12-16 14:32:06+00:00",
            "title": "Discovering New Intents with Deep Aligned Clustering",
            "authors": [
                "Hanlei Zhang",
                "Hua Xu",
                "Ting-En Lin",
                "Rui Lyu"
            ],
            "abstract": "Discovering new intents is a crucial task in dialogue systems. Most existing\nmethods are limited in transferring the prior knowledge from known intents to\nnew intents. They also have difficulties in providing high-quality supervised\nsignals to learn clustering-friendly features for grouping unlabeled intents.\nIn this work, we propose an effective method, Deep Aligned Clustering, to\ndiscover new intents with the aid of the limited known intent data. Firstly, we\nleverage a few labeled known intent samples as prior knowledge to pre-train the\nmodel. Then, we perform k-means to produce cluster assignments as\npseudo-labels. Moreover, we propose an alignment strategy to tackle the label\ninconsistency problem during clustering assignments. Finally, we learn the\nintent representations under the supervision of the aligned pseudo-labels. With\nan unknown number of new intents, we predict the number of intent categories by\neliminating low-confidence intent-wise clusters. Extensive experiments on two\nbenchmark datasets show that our method is more robust and achieves substantial\nimprovements over the state-of-the-art methods. The codes are released at\nhttps://github.com/thuiar/DeepAligned-Clustering.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09005v1",
            "date": "2020-12-16 15:04:50+00:00",
            "title": "Using Meta-Knowledge Mined from Identifiers to Improve Intent Recognition in Neuro-Symbolic Algorithms",
            "authors": [
                "Claudio Pinhanez",
                "Paulo Cavalin",
                "Victor Ribeiro",
                "Heloisa Candello",
                "Julio Nogima",
                "Ana Appel",
                "Mauro Pichiliani",
                "Maira Gatti de Bayser",
                "Melina Guerra",
                "Henrique Ferreira",
                "Gabriel Malfatti"
            ],
            "abstract": "In this paper we explore the use of meta-knowledge embedded in intent\nidentifiers to improve intent recognition in conversational systems. As\nevidenced by the analysis of thousands of real-world chatbots and in interviews\nwith professional chatbot curators, developers and domain experts tend to\norganize the set of chatbot intents by identifying them using proto-taxonomies,\ni.e., meta-knowledge connecting high-level, symbolic concepts shared across\ndifferent intents. By using neuro-symbolic algorithms able to incorporate such\nproto-taxonomies to expand intent representation, we show that such mined\nmeta-knowledge can improve accuracy in intent recognition. In a dataset with\nintents and example utterances from hundreds of professional chatbots, we saw\nimprovements of more than 10% in the equal error rate (EER) in almost a third\nof the chatbots when we apply those algorithms in comparison to a baseline of\nthe same algorithms without the meta-knowledge. The meta-knowledge proved to be\neven more relevant in detecting out-of-scope utterances, decreasing the false\nacceptance rate (FAR) in more than 20\\% in about half of the chatbots. The\nexperiments demonstrate that such symbolic meta-knowledge structures can be\neffectively mined and used by neuro-symbolic algorithms, apparently by\nincorporating into the learning process higher-level structures of the problem\nbeing solved. Based on these results, we also discuss how the use of mined\nmeta-knowledge can be an answer for the challenge of knowledge acquisition in\nneuro-symbolic algorithms.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09123v1",
            "date": "2020-12-16 18:09:32+00:00",
            "title": "Building and Using Personal Knowledge Graph to Improve Suicidal Ideation Detection on Social Media",
            "authors": [
                "Lei Cao",
                "Huijun Zhang",
                "Ling Feng"
            ],
            "abstract": "A large number of individuals are suffering from suicidal ideation in the\nworld. There are a number of causes behind why an individual might suffer from\nsuicidal ideation. As the most popular platform for self-expression, emotion\nrelease, and personal interaction, individuals may exhibit a number of symptoms\nof suicidal ideation on social media. Nevertheless, challenges from both data\nand knowledge aspects remain as obstacles, constraining the social media-based\ndetection performance. Data implicitness and sparsity make it difficult to\ndiscover the inner true intentions of individuals based on their posts.\nInspired by psychological studies, we build and unify a high-level\nsuicide-oriented knowledge graph with deep neural networks for suicidal\nideation detection on social media. We further design a two-layered attention\nmechanism to explicitly reason and establish key risk factors to individual's\nsuicidal ideation. The performance study on microblog and Reddit shows that: 1)\nwith the constructed personal knowledge graph, the social media-based suicidal\nideation detection can achieve over 93% accuracy; and 2) among the six\ncategories of personal factors, post, personality, and experience are the top-3\nkey indicators. Under these categories, posted text, stress level, stress\nduration, posted image, and ruminant thinking contribute to one's suicidal\nideation detection.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09157v1",
            "date": "2020-12-16 18:49:29+00:00",
            "title": "LIREx: Augmenting Language Inference with Relevant Explanation",
            "authors": [
                "Xinyan Zhao",
                "V. G. Vinod Vydiswaran"
            ],
            "abstract": "Natural language explanations (NLEs) are a special form of data annotation in\nwhich annotators identify rationales (most significant text tokens) when\nassigning labels to data instances, and write out explanations for the labels\nin natural language based on the rationales. NLEs have been shown to capture\nhuman reasoning better, but not as beneficial for natural language inference\n(NLI). In this paper, we analyze two primary flaws in the way NLEs are\ncurrently used to train explanation generators for language inference tasks. We\nfind that the explanation generators do not take into account the variability\ninherent in human explanation of labels, and that the current explanation\ngeneration models generate spurious explanations. To overcome these\nlimitations, we propose a novel framework, LIREx, that incorporates both a\nrationale-enabled explanation generator and an instance selector to select only\nrelevant, plausible NLEs to augment NLI models. When evaluated on the\nstandardized SNLI data set, LIREx achieved an accuracy of 91.87%, an\nimprovement of 0.32 over the baseline and matching the best-reported\nperformance on the data set. It also achieves significantly better performance\nthan previous studies when transferred to the out-of-domain MultiNLI data set.\nQualitative analysis shows that LIREx generates flexible, faithful, and\nrelevant NLEs that allow the model to be more robust to spurious explanations.\nThe code is available at https://github.com/zhaoxy92/LIREx.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09216v1",
            "date": "2020-12-16 19:11:36+00:00",
            "title": "MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification",
            "authors": [
                "Te-Lin Wu",
                "Shikhar Singh",
                "Sayan Paul",
                "Gully Burns",
                "Nanyun Peng"
            ],
            "abstract": "We introduce a new dataset, MELINDA, for Multimodal biomEdicaL experImeNt\nmethoD clAssification. The dataset is collected in a fully automated distant\nsupervision manner, where the labels are obtained from an existing curated\ndatabase, and the actual contents are extracted from papers associated with\neach of the records in the database. We benchmark various state-of-the-art NLP\nand computer vision models, including unimodal models which only take either\ncaption texts or images as inputs, and multimodal models. Extensive experiments\nand analysis show that multimodal models, despite outperforming unimodal ones,\nstill need improvements especially on a less-supervised way of grounding visual\nconcepts with languages, and better transferability to low resource domains. We\nrelease our dataset and the benchmarks to facilitate future research in\nmultimodal learning, especially to motivate targeted improvements for\napplications in scientific domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09355v1",
            "date": "2020-12-17 02:01:32+00:00",
            "title": "Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization",
            "authors": [
                "Jiho Noh",
                "Ramakanth Kavuluru"
            ],
            "abstract": "Information retrieval (IR) for precision medicine (PM) often involves looking\nfor multiple pieces of evidence that characterize a patient case. This\ntypically includes at least the name of a condition and a genetic variation\nthat applies to the patient. Other factors such as demographic attributes,\ncomorbidities, and social determinants may also be pertinent. As such, the\nretrieval problem is often formulated as ad hoc search but with multiple facets\n(e.g., disease, mutation) that may need to be incorporated. In this paper, we\npresent a document reranking approach that combines neural query-document\nmatching and text summarization toward such retrieval scenarios. Our\narchitecture builds on the basic BERT model with three specific components for\nreranking: (a). document-query matching (b). keyword extraction and (c).\nfacet-conditioned abstractive summarization. The outcomes of (b) and (c) are\nused to essentially transform a candidate document into a concise summary that\ncan be compared with the query at hand to compute a relevance score. Component\n(a) directly generates a matching score of a candidate document for a query.\nThe full architecture benefits from the complementary potential of\ndocument-query matching and the novel document transformation approach based on\nsummarization along PM facets. Evaluations using NIST's TREC-PM track datasets\n(2017--2019) show that our model achieves state-of-the-art performance. To\nfoster reproducibility, our code is made available here:\nhttps://github.com/bionlproc/text-summ-for-doc-retrieval.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09392v1",
            "date": "2020-12-17 04:54:16+00:00",
            "title": "MASKER: Masked Keyword Regularization for Reliable Text Classification",
            "authors": [
                "Seung Jun Moon",
                "Sangwoo Mo",
                "Kimin Lee",
                "Jaeho Lee",
                "Jinwoo Shin"
            ],
            "abstract": "Pre-trained language models have achieved state-of-the-art accuracies on\nvarious text classification tasks, e.g., sentiment analysis, natural language\ninference, and semantic textual similarity. However, the reliability of the\nfine-tuned text classifiers is an often underlooked performance criterion. For\ninstance, one may desire a model that can detect out-of-distribution (OOD)\nsamples (drawn far from training distribution) or be robust against domain\nshifts. We claim that one central obstacle to the reliability is the\nover-reliance of the model on a limited number of keywords, instead of looking\nat the whole context. In particular, we find that (a) OOD samples often contain\nin-distribution keywords, while (b) cross-domain samples may not always contain\nkeywords; over-relying on the keywords can be problematic for both cases. In\nlight of this observation, we propose a simple yet effective fine-tuning\nmethod, coined masked keyword regularization (MASKER), that facilitates\ncontext-based prediction. MASKER regularizes the model to reconstruct the\nkeywords from the rest of the words and make low-confidence predictions without\nenough context. When applied to various pre-trained language models (e.g.,\nBERT, RoBERTa, and ALBERT), we demonstrate that MASKER improves OOD detection\nand cross-domain generalization without degrading classification accuracy. Code\nis available at https://github.com/alinlab/MASKER.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.09446v1",
            "date": "2020-12-17 08:40:34+00:00",
            "title": "Unsupervised Learning of Discourse Structures using a Tree Autoencoder",
            "authors": [
                "Patrick Huber",
                "Giuseppe Carenini"
            ],
            "abstract": "Discourse information, as postulated by popular discourse theories, such as\nRST and PDTB, has been shown to improve an increasing number of downstream NLP\ntasks, showing positive effects and synergies of discourse with important\nreal-world applications. While methods for incorporating discourse become more\nand more sophisticated, the growing need for robust and general discourse\nstructures has not been sufficiently met by current discourse parsers, usually\ntrained on small scale datasets in a strictly limited number of domains. This\nmakes the prediction for arbitrary tasks noisy and unreliable. The overall\nresulting lack of high-quality, high-quantity discourse trees poses a severe\nlimitation to further progress. In order the alleviate this shortcoming, we\npropose a new strategy to generate tree structures in a task-agnostic,\nunsupervised fashion by extending a latent tree induction framework with an\nauto-encoding objective. The proposed approach can be applied to any\ntree-structured objective, such as syntactic parsing, discourse parsing and\nothers. However, due to the especially difficult annotation process to generate\ndiscourse trees, we initially develop a method to generate larger and more\ndiverse discourse treebanks. In this paper we are inferring general tree\nstructures of natural text in multiple domains, showing promising results on a\ndiverse set of tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09486v1",
            "date": "2020-12-17 10:22:15+00:00",
            "title": "ReferentialGym: A Nomenclature and Framework for Language Emergence & Grounding in (Visual) Referential Games",
            "authors": [
                "Kevin Denamgana\u00ef",
                "James Alfred Walker"
            ],
            "abstract": "Natural languages are powerful tools wielded by human beings to communicate\ninformation and co-operate towards common goals. Their values lie in some main\nproperties like compositionality, hierarchy and recurrent syntax, which\ncomputational linguists have been researching the emergence of in artificial\nlanguages induced by language games. Only relatively recently, the AI community\nhas started to investigate language emergence and grounding working towards\nbetter human-machine interfaces. For instance, interactive/conversational AI\nassistants that are able to relate their vision to the ongoing conversation.\n  This paper provides two contributions to this research field. Firstly, a\nnomenclature is proposed to understand the main initiatives in studying\nlanguage emergence and grounding, accounting for the variations in assumptions\nand constraints. Secondly, a PyTorch based deep learning framework is\nintroduced, entitled ReferentialGym, which is dedicated to furthering the\nexploration of language emergence and grounding. By providing baseline\nimplementations of major algorithms and metrics, in addition to many different\nfeatures and approaches, ReferentialGym attempts to ease the entry barrier to\nthe field and provide the community with common implementations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09807v1",
            "date": "2020-12-17 18:18:03+00:00",
            "title": "BERT Goes Shopping: Comparing Distributional Models for Product Representations",
            "authors": [
                "Federico Bianchi",
                "Bingqing Yu",
                "Jacopo Tagliabue"
            ],
            "abstract": "Word embeddings (e.g., word2vec) have been applied successfully to eCommerce\nproducts through prod2vec. Inspired by the recent performance improvements on\nseveral NLP tasks brought by contextualized embeddings, we propose to transfer\nBERT-like architectures to eCommerce: our model -- ProdBERT -- is trained to\ngenerate representations of products through masked session modeling. Through\nextensive experiments over multiple shops, different tasks, and a range of\ndesign choices, we systematically compare the accuracy of ProdBERT and prod2vec\nembeddings: while ProdBERT is found to be superior to traditional methods in\nseveral scenarios, we highlight the importance of resources and hyperparameters\nin the best performing models. Finally, we conclude by providing guidelines for\ntraining embeddings under a variety of computational and data constraints.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.09938v1",
            "date": "2020-12-17 21:12:58+00:00",
            "title": "Can Transformers Reason About Effects of Actions?",
            "authors": [
                "Pratyay Banerjee",
                "Chitta Baral",
                "Man Luo",
                "Arindam Mitra",
                "Kuntal Pal",
                "Tran C. Son",
                "Neeraj Varshney"
            ],
            "abstract": "A recent work has shown that transformers are able to \"reason\" with facts and\nrules in a limited setting where the rules are natural language expressions of\nconjunctions of conditions implying a conclusion. Since this suggests that\ntransformers may be used for reasoning with knowledge given in natural\nlanguage, we do a rigorous evaluation of this with respect to a common form of\nknowledge and its corresponding reasoning -- the reasoning about effects of\nactions. Reasoning about action and change has been a top focus in the\nknowledge representation subfield of AI from the early days of AI and more\nrecently it has been a highlight aspect in common sense question answering. We\nconsider four action domains (Blocks World, Logistics, Dock-Worker-Robots and a\nGeneric Domain) in natural language and create QA datasets that involve\nreasoning about the effects of actions in these domains. We investigate the\nability of transformers to (a) learn to reason in these domains and (b)\ntransfer that learning from the generic domains to the other domains.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10052v1",
            "date": "2020-12-18 04:49:32+00:00",
            "title": "Leveraging Event Specific and Chunk Span features to Extract COVID Events from tweets",
            "authors": [
                "Ayush Kaushal",
                "Tejas Vaidhya"
            ],
            "abstract": "Twitter has acted as an important source of information during disasters and\npandemic, especially during the times of COVID-19. In this paper, we describe\nour system entry for WNUT 2020 Shared Task-3. The task was aimed at automating\nthe extraction of a variety of COVID-19 related events from Twitter, such as\nindividuals who recently contracted the virus, someone with symptoms who were\ndenied testing and believed remedies against the infection. The system consists\nof separate multi-task models for slot-filling subtasks and\nsentence-classification subtasks while leveraging the useful sentence-level\ninformation for the corresponding event. The system uses COVID-Twitter-Bert\nwith attention-weighted pooling of candidate slot-chunk features to capture the\nuseful information chunks. The system ranks 1st at the leader-board with F1 of\n0.6598, without using any ensembles or additional datasets. The code and\ntrained models are available at this https URL.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10063v1",
            "date": "2020-12-18 05:55:52+00:00",
            "title": "Attention-Based LSTM Network for COVID-19 Clinical Trial Parsing",
            "authors": [
                "Xiong Liu",
                "Luca A. Finelli",
                "Greg L. Hersch",
                "Iya Khalil"
            ],
            "abstract": "COVID-19 clinical trial design is a critical task in developing therapeutics\nfor the prevention and treatment of COVID-19. In this study, we apply a deep\nlearning approach to extract eligibility criteria variables from COVID-19\ntrials to enable quantitative analysis of trial design and optimization.\nSpecifically, we train attention-based bidirectional Long Short-Term Memory\n(Att-BiLSTM) models and use the optimal model to extract entities (i.e.,\nvariables) from the eligibility criteria of COVID-19 trials. We compare the\nperformance of Att-BiLSTM with traditional ontology-based method. The result on\na benchmark dataset shows that Att-BiLSTM outperforms the ontology model.\nAtt-BiLSTM achieves a precision of 0.942, recall of 0.810, and F1 of 0.871,\nwhile the ontology model only achieves a precision of 0.715, recall of 0.659,\nand F1 of 0.686. Our analyses demonstrate that Att-BiLSTM is an effective\napproach for characterizing patient populations in COVID-19 clinical trials.",
            "journal ref": "2020 IEEE International Conference on Big Data (IEEE BigData 2020)",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10074v1",
            "date": "2020-12-18 06:51:23+00:00",
            "title": "Mention Extraction and Linking for SQL Query Generation",
            "authors": [
                "Jianqiang Ma",
                "Zeyu Yan",
                "Shuai Pang",
                "Yang Zhang",
                "Jianping Shen"
            ],
            "abstract": "On the WikiSQL benchmark, state-of-the-art text-to-SQL systems typically take\na slot-filling approach by building several dedicated models for each type of\nslots. Such modularized systems are not only complex butalso of limited\ncapacity for capturing inter-dependencies among SQL clauses. To solve these\nproblems, this paper proposes a novel extraction-linking approach, where a\nunified extractor recognizes all types of slot mentions appearing in the\nquestion sentence before a linker maps the recognized columns to the table\nschema to generate executable SQL queries. Trained with automatically generated\nannotations, the proposed method achieves the first place on the WikiSQL\nbenchmark.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10226v1",
            "date": "2020-12-18 13:43:13+00:00",
            "title": "Should I visit this place? Inclusion and Exclusion Phrase Mining from Reviews",
            "authors": [
                "Omkar Gurjar",
                "Manish Gupta"
            ],
            "abstract": "Although several automatic itinerary generation services have made travel\nplanning easy, often times travellers find themselves in unique situations\nwhere they cannot make the best out of their trip. Visitors differ in terms of\nmany factors such as suffering from a disability, being of a particular dietary\npreference, travelling with a toddler, etc. While most tourist spots are\nuniversal, others may not be inclusive for all. In this paper, we focus on the\nproblem of mining inclusion and exclusion phrases associated with 11 such\nfactors, from reviews related to a tourist spot. While existing work on tourism\ndata mining mainly focuses on structured extraction of trip related\ninformation, personalized sentiment analysis, and automatic itinerary\ngeneration, to the best of our knowledge this is the first work on\ninclusion/exclusion phrase mining from tourism reviews. Using a dataset of 2000\nreviews related to 1000 tourist spots, our broad level classifier provides a\nbinary overlap F1 of $\\sim$80 and $\\sim$82 to classify a phrase as inclusion or\nexclusion respectively. Further, our inclusion/exclusion classifier provides an\nF1 of $\\sim$98 and $\\sim$97 for 11-class inclusion and exclusion classification\nrespectively. We believe that our work can significantly improve the quality of\nan automatic itinerary generation service.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.10271v1",
            "date": "2020-12-18 14:47:47+00:00",
            "title": "Understood in Translation, Transformers for Domain Understanding",
            "authors": [
                "Dimitrios Christofidellis",
                "Matteo Manica",
                "Leonidas Georgopoulos",
                "Hans Vandierendonck"
            ],
            "abstract": "Knowledge acquisition is the essential first step of any Knowledge Graph (KG)\napplication. This knowledge can be extracted from a given corpus (KG generation\nprocess) or specified from an existing KG (KG specification process). Focusing\non domain specific solutions, knowledge acquisition is a labor intensive task\nusually orchestrated and supervised by subject matter experts. Specifically,\nthe domain of interest is usually manually defined and then the needed\ngeneration or extraction tools are utilized to produce the KG. Herein, we\npropose a supervised machine learning method, based on Transformers, for domain\ndefinition of a corpus. We argue why such automated definition of the domain's\nstructure is beneficial both in terms of construction time and quality of the\ngenerated graph. The proposed method is extensively validated on three public\ndatasets (WebNLG, NYT and DocRED) by comparing it with two reference methods\nbased on CNNs and RNNs models. The evaluation shows the efficiency of our model\nin this task. Focusing on scientific document understanding, we present a new\nhealth domain dataset based on publications extracted from PubMed and we\nsuccessfully utilize our method on this. Lastly, we demonstrate how this work\nlays the foundation for fully automated and unsupervised KG generation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10285v1",
            "date": "2020-12-18 15:01:50+00:00",
            "title": "Trying Bilinear Pooling in Video-QA",
            "authors": [
                "Thomas Winterbottom",
                "Sarah Xiao",
                "Alistair McLean",
                "Noura Al Moubayed"
            ],
            "abstract": "Bilinear pooling (BLP) refers to a family of operations recently developed\nfor fusing features from different modalities predominantly developed for VQA\nmodels. A bilinear (outer-product) expansion is thought to encourage models to\nlearn interactions between two feature spaces and has experimentally\noutperformed `simpler' vector operations (concatenation and\nelement-wise-addition/multiplication) on VQA benchmarks. Successive BLP\ntechniques have yielded higher performance with lower computational expense and\nare often implemented alongside attention mechanisms. However, despite\nsignificant progress in VQA, BLP methods have not been widely applied to more\nrecently explored video question answering (video-QA) tasks. In this paper, we\nbegin to bridge this research gap by applying BLP techniques to various\nvideo-QA benchmarks, namely: TVQA, TGIF-QA, Ego-VQA and MSVD-QA. We share our\nresults on the TVQA baseline model, and the recently proposed\nheterogeneous-memory-enchanced multimodal attention (HME) model. Our\nexperiments include both simply replacing feature concatenation in the existing\nmodels with BLP, and a modified version of the TVQA baseline to accommodate BLP\nwe name the `dual-stream' model. We find that our relatively simple integration\nof BLP does not increase, and mostly harms, performance on these video-QA\nbenchmarks. Using recently proposed theoretical multimodal fusion taxonomies,\nwe offer insight into why BLP-driven performance gain for video-QA benchmarks\nmay be more difficult to achieve than in earlier VQA models. We suggest a few\nadditional `best-practices' to consider when applying BLP to video-QA. We\nstress that video-QA models should carefully consider where the complex\nrepresentational potential from BLP is actually needed to avoid computational\nexpense on `redundant' fusion.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.10586v2",
            "date": "2020-12-19 03:33:27+00:00",
            "title": "Finding Sparse Structures for Domain Specific Neural Machine Translation",
            "authors": [
                "Jianze Liang",
                "Chengqi Zhao",
                "Mingxuan Wang",
                "Xipeng Qiu",
                "Lei Li"
            ],
            "abstract": "Neural machine translation often adopts the fine-tuning approach to adapt to\nspecific domains. However, nonrestricted fine-tuning can easily degrade on the\ngeneral domain and over-fit to the target domain. To mitigate the issue, we\npropose Prune-Tune, a novel domain adaptation method via gradual pruning. It\nlearns tiny domain-specific sub-networks during fine-tuning on new domains.\nPrune-Tune alleviates the over-fitting and the degradation problem without\nmodel modification. Furthermore, Prune-Tune is able to sequentially learn a\nsingle network with multiple disjoint domain-specific sub-networks for multiple\ndomains. Empirical experiment results show that Prune-Tune outperforms several\nstrong competitors in the target domain test set without sacrificing the\nquality on the general domain in both single and multi-domain settings. The\nsource code and data are available at https://github.com/ohlionel/Prune-Tune.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10821v1",
            "date": "2020-12-20 01:07:30+00:00",
            "title": "Transductive Visual Verb Sense Disambiguation",
            "authors": [
                "Sebastiano Vascon",
                "Sinem Aslan",
                "Gianluca Bigaglia",
                "Lorenzo Giudice",
                "Marcello Pelillo"
            ],
            "abstract": "Verb Sense Disambiguation is a well-known task in NLP, the aim is to find the\ncorrect sense of a verb in a sentence. Recently, this problem has been extended\nin a multimodal scenario, by exploiting both textual and visual features of\nambiguous verbs leading to a new problem, the Visual Verb Sense Disambiguation\n(VVSD). Here, the sense of a verb is assigned considering the content of an\nimage paired with it rather than a sentence in which the verb appears.\nAnnotating a dataset for this task is more complex than textual disambiguation,\nbecause assigning the correct sense to a pair of $<$image, verb$>$ requires\nboth non-trivial linguistic and visual skills. In this work, differently from\nthe literature, the VVSD task will be performed in a transductive\nsemi-supervised learning (SSL) setting, in which only a small amount of labeled\ninformation is required, reducing tremendously the need for annotated data. The\ndisambiguation process is based on a graph-based label propagation method which\ntakes into account mono or multimodal representations for $<$image, verb$>$\npairs. Experiments have been carried out on the recently published dataset\nVerSe, the only available dataset for this task. The achieved results\noutperform the current state-of-the-art by a large margin while using only a\nsmall fraction of labeled samples per sense. Code available:\nhttps://github.com/GiBg1aN/TVVSD.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.10824v1",
            "date": "2020-12-20 01:30:07+00:00",
            "title": "A hybrid deep-learning approach for complex biochemical named entity recognition",
            "authors": [
                "Jian Liu",
                "Lei Gao",
                "Sujie Guo",
                "Rui Ding",
                "Xin Huang",
                "Long Ye",
                "Qinghua Meng",
                "Asef Nazari",
                "Dhananjay Thiruvady"
            ],
            "abstract": "Named entity recognition (NER) of chemicals and drugs is a critical domain of\ninformation extraction in biochemical research. NER provides support for text\nmining in biochemical reactions, including entity relation extraction,\nattribute extraction, and metabolic response relationship extraction. However,\nthe existence of complex naming characteristics in the biomedical field, such\nas polysemy and special characters, make the NER task very challenging. Here,\nwe propose a hybrid deep learning approach to improve the recognition accuracy\nof NER. Specifically, our approach applies the Bidirectional Encoder\nRepresentations from Transformers (BERT) model to extract the underlying\nfeatures of the text, learns a representation of the context of the text\nthrough Bi-directional Long Short-Term Memory (BILSTM), and incorporates the\nmulti-head attention (MHATT) mechanism to extract chapter-level features. In\nthis approach, the MHATT mechanism aims to improve the recognition accuracy of\nabbreviations to efficiently deal with the problem of inconsistency in\nfull-text labels. Moreover, conditional random field (CRF) is used to label\nsequence tags because this probabilistic method does not need strict\nindependence assumptions and can accommodate arbitrary context information. The\nexperimental evaluation on a publicly-available dataset shows that the proposed\nhybrid approach achieves the best recognition performance; in particular, it\nsubstantially improves performance in recognizing abbreviations, polysemes, and\nlow-frequency entities, compared with the state-of-the-art approaches. For\ninstance, compared with the recognition accuracies for low-frequency entities\nproduced by the BILSTM-CRF algorithm, those produced by the hybrid approach on\ntwo entity datasets (MULTIPLE and IDENTIFIER) have been increased by 80% and\n21.69%, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.10877v2",
            "date": "2020-12-20 09:31:35+00:00",
            "title": "Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension",
            "authors": [
                "Nuo Chen",
                "Fenglin Liu",
                "Chenyu You",
                "Peilin Zhou",
                "Yuexian Zou"
            ],
            "abstract": "Recently, the attention-enhanced multi-layer encoder, such as Transformer,\nhas been extensively studied in Machine Reading Comprehension (MRC). To predict\nthe answer, it is common practice to employ a predictor to draw information\nonly from the final encoder layer which generates the \\textit{coarse-grained}\nrepresentations of the source sequences, i.e., passage and question. Previous\nstudies have shown that the representation of source sequence becomes more\n\\textit{coarse-grained} from \\textit{fine-grained} as the encoding layer\nincreases. It is generally believed that with the growing number of layers in\ndeep neural networks, the encoding process will gather relevant information for\neach location increasingly, resulting in more \\textit{coarse-grained}\nrepresentations, which adds the likelihood of similarity to other locations\n(referring to homogeneity). Such a phenomenon will mislead the model to make\nwrong judgments so as to degrade the performance. To this end, we propose a\nnovel approach called Adaptive Bidirectional Attention, which adaptively\nexploits the source representations of different levels to the predictor.\nExperimental results on the benchmark dataset, SQuAD 2.0 demonstrate the\neffectiveness of our approach, and the results are better than the previous\nstate-of-the-art model by 2.5$\\%$ EM and 2.3$\\%$ F1 scores.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11014v1",
            "date": "2020-12-20 20:13:02+00:00",
            "title": "KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA",
            "authors": [
                "Kenneth Marino",
                "Xinlei Chen",
                "Devi Parikh",
                "Abhinav Gupta",
                "Marcus Rohrbach"
            ],
            "abstract": "One of the most challenging question types in VQA is when answering the\nquestion requires outside knowledge not present in the image. In this work we\nstudy open-domain knowledge, the setting when the knowledge required to answer\na question is not given/annotated, neither at training nor test time. We tap\ninto two types of knowledge representations and reasoning. First, implicit\nknowledge which can be learned effectively from unsupervised language\npre-training and supervised training data with transformer-based models.\nSecond, explicit, symbolic knowledge encoded in knowledge bases. Our approach\ncombines both - exploiting the powerful implicit reasoning of transformer\nmodels for answer prediction, and integrating symbolic representations from a\nknowledge graph, while never losing their explicit semantics to an implicit\nembedding. We combine diverse sources of knowledge to cover the wide variety of\nknowledge needed to solve knowledge-based questions. We show our approach,\nKRISP (Knowledge Reasoning with Implicit and Symbolic rePresentations),\nsignificantly outperforms state-of-the-art on OK-VQA, the largest available\ndataset for open-domain knowledge-based VQA. We show with extensive ablations\nthat while our model successfully exploits implicit knowledge reasoning, the\nsymbolic answer module which explicitly connects the knowledge graph to the\nanswer vocabulary is critical to the performance of our method and generalizes\nto rare answers.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.11099v2",
            "date": "2020-12-21 03:38:29+00:00",
            "title": "A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training",
            "authors": [
                "Yongkang Liu",
                "Shi Feng",
                "Daling Wang",
                "Kaisong Song",
                "Feiliang Ren",
                "Yifei Zhang"
            ],
            "abstract": "We investigate response selection for multi-turn conversation in\nretrieval-based chatbots. Existing studies pay more attention to the matching\nbetween utterances and responses by calculating the matching score based on\nlearned features, leading to insufficient model reasoning ability. In this\npaper, we propose a graph-reasoning network (GRN) to address the problem. GRN\nfirst conducts pre-training based on ALBERT using next utterance prediction and\nutterance order prediction tasks specifically devised for response selection.\nThese two customized pre-training tasks can endow our model with the ability of\ncapturing semantical and chronological dependency between utterances. We then\nfine-tune the model on an integrated network with sequence reasoning and graph\nreasoning structures. The sequence reasoning module conducts inference based on\nthe highly summarized context vector of utterance-response pairs from the\nglobal perspective. The graph reasoning module conducts the reasoning on the\nutterance-level graph neural network from the local perspective. Experiments on\ntwo conversational reasoning datasets show that our model can dramatically\noutperform the strong baseline methods and can achieve performance which is\nclose to human-level.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11142v1",
            "date": "2020-12-21 06:44:32+00:00",
            "title": "Towards Incorporating Entity-specific Knowledge Graph Information in Predicting Drug-Drug Interactions",
            "authors": [
                "Ishani Mondal"
            ],
            "abstract": "Off-the-shelf biomedical embeddings obtained from the recently released\nvarious pre-trained language models (such as BERT, XLNET) have demonstrated\nstate-of-the-art results (in terms of accuracy) for the various natural\nlanguage understanding tasks (NLU) in the biomedical domain. Relation\nClassification (RC) falls into one of the most critical tasks. In this paper,\nwe explore how to incorporate domain knowledge of the biomedical entities (such\nas drug, disease, genes), obtained from Knowledge Graph (KG) Embeddings, for\npredicting Drug-Drug Interaction from textual corpus. We propose a new method,\nBERTKG-DDI, to combine drug embeddings obtained from its interaction with other\nbiomedical entities along with domain-specific BioBERT embedding-based RC\narchitecture. Experiments conducted on the DDIExtraction 2013 corpus clearly\nindicate that this strategy improves other baselines architectures by 4.1%\nmacro F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11157v2",
            "date": "2020-12-21 07:18:08+00:00",
            "title": "Narrative Incoherence Detection",
            "authors": [
                "Deng Cai",
                "Yizhe Zhang",
                "Yichen Huang",
                "Wai Lam",
                "Bill Dolan"
            ],
            "abstract": "We propose the task of narrative incoherence detection as a new arena for\ninter-sentential semantic understanding: Given a multi-sentence narrative,\ndecide whether there exist any semantic discrepancies in the narrative flow.\nSpecifically, we focus on the missing sentence and discordant sentence\ndetection. Despite its simple setup, this task is challenging as the model\nneeds to understand and analyze a multi-sentence narrative, and predict\nincoherence at the sentence level. As an initial step towards this task, we\nimplement several baselines either directly analyzing the raw text\n(\\textit{token-level}) or analyzing learned sentence representations\n(\\textit{sentence-level}). We observe that while token-level modeling has\nbetter performance when the input contains fewer sentences, sentence-level\nmodeling performs better on longer narratives and possesses an advantage in\nefficiency and flexibility. Pre-training on large-scale data and auxiliary\nsentence prediction training objective further boost the detection performance\nof the sentence-level model.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11163v1",
            "date": "2020-12-21 07:43:31+00:00",
            "title": "MT-Teql: Evaluating and Augmenting Consistency of Text-to-SQL Models with Metamorphic Testing",
            "authors": [
                "Pingchuan Ma",
                "Shuai Wang"
            ],
            "abstract": "Text-to-SQL is a task to generate SQL queries from human utterances. However,\ndue to the variation of natural language, two semantically equivalent\nutterances may appear differently in the lexical level. Likewise, user\npreferences (e.g., the choice of normal forms) can lead to dramatic changes in\ntable structures when expressing conceptually identical schemas. Envisioning\nthe general difficulty for text-to-SQL models to preserve prediction\nconsistency against linguistic and schema variations, we propose MT-Teql, a\nMetamorphic Testing-based framework for systematically evaluating and\naugmenting the consistency of TExt-to-SQL models. Inspired by the principles of\nsoftware metamorphic testing, MT-Teql delivers a model-agnostic framework which\nimplements a comprehensive set of metamorphic relations (MRs) to conduct\nsemantics-preserving transformations toward utterances and schemas. Model\nInconsistency can be exposed when the original and transformed inputs induce\ndifferent SQL queries. In addition, we leverage the transformed inputs to\nretrain models for further model robustness boost. Our experiments show that\nour framework exposes thousands of prediction errors from SOTA models and\nenriches existing datasets by order of magnitude, eliminating over 40%\ninconsistency errors without compromising standard accuracy.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11164v1",
            "date": "2020-12-21 07:44:37+00:00",
            "title": "Medical Entity Linking using Triplet Network",
            "authors": [
                "Ishani Mondal",
                "Sukannya Purkayastha",
                "Sudeshna Sarkar",
                "Pawan Goyal",
                "Jitesh Pillai",
                "Amitava Bhattacharyya",
                "Mahanandeeshwar Gattu"
            ],
            "abstract": "Entity linking (or Normalization) is an essential task in text mining that\nmaps the entity mentions in the medical text to standard entities in a given\nKnowledge Base (KB). This task is of great importance in the medical domain. It\ncan also be used for merging different medical and clinical ontologies. In this\npaper, we center around the problem of disease linking or normalization. This\ntask is executed in two phases: candidate generation and candidate scoring. In\nthis paper, we present an approach to rank the candidate Knowledge Base entries\nbased on their similarity with disease mention. We make use of the Triplet\nNetwork for candidate ranking. While the existing methods have used carefully\ngenerated sieves and external resources for candidate generation, we introduce\na robust and portable candidate generation scheme that does not make use of the\nhand-crafted rules. Experimental results on the standard benchmark NCBI disease\ndataset demonstrate that our system outperforms the prior methods by a\nsignificant margin.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11213v2",
            "date": "2020-12-21 09:48:58+00:00",
            "title": "Self-Supervised Learning for Visual Summary Identification in Scientific Publications",
            "authors": [
                "Shintaro Yamamoto",
                "Anne Lauscher",
                "Simone Paolo Ponzetto",
                "Goran Glava\u0161",
                "Shigeo Morishima"
            ],
            "abstract": "Providing visual summaries of scientific publications can increase\ninformation access for readers and thereby help deal with the exponential\ngrowth in the number of scientific publications. Nonetheless, efforts in\nproviding visual publication summaries have been few and far apart, primarily\nfocusing on the biomedical domain. This is primarily because of the limited\navailability of annotated gold standards, which hampers the application of\nrobust and high-performing supervised learning techniques. To address these\nproblems we create a new benchmark dataset for selecting figures to serve as\nvisual summaries of publications based on their abstracts, covering several\ndomains in computer science. Moreover, we develop a self-supervised learning\napproach, based on heuristic matching of inline references to figures with\nfigure captions. Experiments in both biomedical and computer science domains\nshow that our model is able to outperform the state of the art despite being\nself-supervised and therefore not relying on any annotated training data.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.11599v1",
            "date": "2020-12-21 07:13:52+00:00",
            "title": "BERTChem-DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information",
            "authors": [
                "Ishani Mondal"
            ],
            "abstract": "Traditional biomedical version of embeddings obtained from pre-trained\nlanguage models have recently shown state-of-the-art results for relation\nextraction (RE) tasks in the medical domain. In this paper, we explore how to\nincorporate domain knowledge, available in the form of molecular structure of\ndrugs, for predicting Drug-Drug Interaction from textual corpus. We propose a\nmethod, BERTChem-DDI, to efficiently combine drug embeddings obtained from the\nrich chemical structure of drugs along with off-the-shelf domain-specific\nBioBERT embedding-based RE architecture. Experiments conducted on the\nDDIExtraction 2013 corpus clearly indicate that this strategy improves other\nstrong baselines architectures by 3.4\\% macro F1-score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11740v1",
            "date": "2020-12-21 23:14:18+00:00",
            "title": "SChuBERT: Scholarly Document Chunks with BERT-encoding boost Citation Count Prediction",
            "authors": [
                "Thomas van Dongen",
                "Gideon Maillette de Buy Wenniger",
                "Lambert Schomaker"
            ],
            "abstract": "Predicting the number of citations of scholarly documents is an upcoming task\nin scholarly document processing. Besides the intrinsic merit of this\ninformation, it also has a wider use as an imperfect proxy for quality which\nhas the advantage of being cheaply available for large volumes of scholarly\ndocuments. Previous work has dealt with number of citations prediction with\nrelatively small training data sets, or larger datasets but with short,\nincomplete input text. In this work we leverage the open access ACL Anthology\ncollection in combination with the Semantic Scholar bibliometric database to\ncreate a large corpus of scholarly documents with associated citation\ninformation and we propose a new citation prediction model called SChuBERT. In\nour experiments we compare SChuBERT with several state-of-the-art citation\nprediction models and show that it outperforms previous methods by a large\nmargin. We also show the merit of using more training data and longer input for\nnumber of citations prediction.",
            "journal ref": "Proceedings of the First Workshop on Scholarly Document\n  Processing. Association for Computational Linguistics. (2020) 158-167.\n  EMNLP|SDP 2020 https://www.aclweb.org/anthology/2020.sdp-1.17/",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11805v1",
            "date": "2020-12-22 02:55:04+00:00",
            "title": "Semi-Supervised Disentangled Framework for Transferable Named Entity Recognition",
            "authors": [
                "Zhifeng Hao",
                "Di Lv",
                "Zijian Li",
                "Ruichu Cai",
                "Wen Wen",
                "Boyan Xu"
            ],
            "abstract": "Named entity recognition (NER) for identifying proper nouns in unstructured\ntext is one of the most important and fundamental tasks in natural language\nprocessing. However, despite the widespread use of NER models, they still\nrequire a large-scale labeled data set, which incurs a heavy burden due to\nmanual annotation. Domain adaptation is one of the most promising solutions to\nthis problem, where rich labeled data from the relevant source domain are\nutilized to strengthen the generalizability of a model based on the target\ndomain. However, the mainstream cross-domain NER models are still affected by\nthe following two challenges (1) Extracting domain-invariant information such\nas syntactic information for cross-domain transfer. (2) Integrating\ndomain-specific information such as semantic information into the model to\nimprove the performance of NER. In this study, we present a semi-supervised\nframework for transferable NER, which disentangles the domain-invariant latent\nvariables and domain-specific latent variables. In the proposed framework, the\ndomain-specific information is integrated with the domain-specific latent\nvariables by using a domain predictor. The domain-specific and domain-invariant\nlatent variables are disentangled using three mutual information regularization\nterms, i.e., maximizing the mutual information between the domain-specific\nlatent variables and the original embedding, maximizing the mutual information\nbetween the domain-invariant latent variables and the original embedding, and\nminimizing the mutual information between the domain-specific and\ndomain-invariant latent variables. Extensive experiments demonstrated that our\nmodel can obtain state-of-the-art performance with cross-domain and\ncross-lingual NER benchmark data sets.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11881v1",
            "date": "2020-12-22 08:46:14+00:00",
            "title": "Undivided Attention: Are Intermediate Layers Necessary for BERT?",
            "authors": [
                "Sharath Nittur Sridhar",
                "Anthony Sarah"
            ],
            "abstract": "In recent times, BERT-based models have been extremely successful in solving\na variety of natural language processing (NLP) tasks such as reading\ncomprehension, natural language inference, sentiment analysis, etc. All\nBERT-based architectures have a self-attention block followed by a block of\nintermediate layers as the basic building component. However, a strong\njustification for the inclusion of these intermediate layers remains missing in\nthe literature. In this work we investigate the importance of intermediate\nlayers on the overall network performance of downstream tasks. We show that\nreducing the number of intermediate layers and modifying the architecture for\nBERT-Base results in minimal loss in fine-tuning accuracy for downstream tasks\nwhile decreasing the number of parameters and training time of the model.\nAdditionally, we use the central kernel alignment (CKA) similarity metric and\nprobing classifiers to demonstrate that removing intermediate layers has little\nimpact on the learned self-attention representations.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11926v1",
            "date": "2020-12-22 10:53:07+00:00",
            "title": "Few-Shot Text Generation with Pattern-Exploiting Training",
            "authors": [
                "Timo Schick",
                "Hinrich Sch\u00fctze"
            ],
            "abstract": "Providing pretrained language models with simple task descriptions or prompts\nin natural language yields impressive few-shot results for a wide range of text\nclassification tasks when combined with gradient-based learning from examples.\nIn this paper, we show that the underlying idea can also be applied to text\ngeneration tasks: We adapt Pattern-Exploiting Training (PET), a recently\nproposed few-shot approach, for finetuning generative language models on text\ngeneration tasks. On several text summarization and headline generation\ndatasets, our proposed variant of PET gives consistent improvements over a\nstrong baseline in few-shot settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11960v1",
            "date": "2020-12-22 12:27:45+00:00",
            "title": "A Hierarchical Reasoning Graph Neural Network for The Automatic Scoring of Answer Transcriptions in Video Job Interviews",
            "authors": [
                "Kai Chen",
                "Meng Niu",
                "Qingcai Chen"
            ],
            "abstract": "We address the task of automatically scoring the competency of candidates\nbased on textual features, from the automatic speech recognition (ASR)\ntranscriptions in the asynchronous video job interview (AVI). The key challenge\nis how to construct the dependency relation between questions and answers, and\nconduct the semantic level interaction for each question-answer (QA) pair.\nHowever, most of the recent studies in AVI focus on how to represent questions\nand answers better, but ignore the dependency information and interaction\nbetween them, which is critical for QA evaluation. In this work, we propose a\nHierarchical Reasoning Graph Neural Network (HRGNN) for the automatic\nassessment of question-answer pairs. Specifically, we construct a\nsentence-level relational graph neural network to capture the dependency\ninformation of sentences in or between the question and the answer. Based on\nthese graphs, we employ a semantic-level reasoning graph attention network to\nmodel the interaction states of the current QA session. Finally, we propose a\ngated recurrent unit encoder to represent the temporal question-answer pairs\nfor the final prediction. Empirical results conducted on CHNAT (a real-world\ndataset) validate that our proposed model significantly outperforms\ntext-matching based benchmark models. Ablation studies and experimental results\nwith 10 random seeds also show the effectiveness and stability of our models.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.11988v1",
            "date": "2020-12-22 13:20:23+00:00",
            "title": "Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation",
            "authors": [
                "Shuai Lin",
                "Pan Zhou",
                "Xiaodan Liang",
                "Jianheng Tang",
                "Ruihui Zhao",
                "Ziliang Chen",
                "Liang Lin"
            ],
            "abstract": "Human doctors with well-structured medical knowledge can diagnose a disease\nmerely via a few conversations with patients about symptoms. In contrast,\nexisting knowledge-grounded dialogue systems often require a large number of\ndialogue instances to learn as they fail to capture the correlations between\ndifferent diseases and neglect the diagnostic experience shared among them. To\naddress this issue, we propose a more natural and practical paradigm, i.e.,\nlow-resource medical dialogue generation, which can transfer the diagnostic\nexperience from source diseases to target ones with a handful of data for\nadaptation. It is capitalized on a commonsense knowledge graph to characterize\nthe prior disease-symptom relations. Besides, we develop a Graph-Evolving\nMeta-Learning (GEML) framework that learns to evolve the commonsense graph for\nreasoning disease-symptom correlations in a new disease, which effectively\nalleviates the needs of a large number of dialogues. More importantly, by\ndynamically evolving disease-symptom graphs, GEML also well addresses the\nreal-world challenges that the disease-symptom correlations of each disease may\nvary or evolve along with more diagnostic cases. Extensive experiment results\non the CMDD dataset and our newly-collected Chunyu dataset testify the\nsuperiority of our approach over state-of-the-art approaches. Besides, our GEML\ncan generate an enriched dialogue-sensitive knowledge graph in an online\nmanner, which could benefit other tasks grounded on knowledge graph.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12007v1",
            "date": "2020-12-22 13:48:09+00:00",
            "title": "Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition",
            "authors": [
                "Yubo Xie",
                "Junze Li",
                "Pearl Pu"
            ],
            "abstract": "Humor recognition has been widely studied as a text classification problem\nusing data-driven approaches. However, most existing work does not examine the\nactual joke mechanism to understand humor. We break down any joke into two\ndistinct components: the set-up and the punchline, and further explore the\nspecial relationship between them. Inspired by the incongruity theory of humor,\nwe model the set-up as the part developing semantic uncertainty, and the\npunchline disrupting audience expectations. With increasingly powerful language\nmodels, we were able to feed the set-up along with the punchline into the GPT-2\nlanguage model, and calculate the uncertainty and surprisal values of the\njokes. By conducting experiments on the SemEval 2021 Task 7 dataset, we found\nthat these two features have better capabilities of telling jokes from\nnon-jokes, compared with existing baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12065v1",
            "date": "2020-12-22 14:56:54+00:00",
            "title": "Event-Driven Query Expansion",
            "authors": [
                "Guy D. Rosin",
                "Ido Guy",
                "Kira Radinsky"
            ],
            "abstract": "A significant number of event-related queries are issued in Web search. In\nthis paper, we seek to improve retrieval performance by leveraging events and\nspecifically target the classic task of query expansion. We propose a method to\nexpand an event-related query by first detecting the events related to it.\nThen, we derive the candidates for expansion as terms semantically related to\nboth the query and the events. To identify the candidates, we utilize a novel\nmechanism to simultaneously embed words and events in the same vector space. We\nshow that our proposed method of leveraging events improves query expansion\nperformance significantly compared with state-of-the-art methods on various\nnewswire TREC datasets.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.12112v2",
            "date": "2020-12-22 15:46:40+00:00",
            "title": "Domain Adaptation of NMT models for English-Hindi Machine Translation Task at AdapMT ICON 2020",
            "authors": [
                "Ramchandra Joshi",
                "Rushabh Karnavat",
                "Kaustubh Jirapure",
                "Raviraj Joshi"
            ],
            "abstract": "Recent advancements in Neural Machine Translation (NMT) models have proved to\nproduce a state of the art results on machine translation for low resource\nIndian languages. This paper describes the neural machine translation systems\nfor the English-Hindi language presented in AdapMT Shared Task ICON 2020. The\nshared task aims to build a translation system for Indian languages in specific\ndomains like Artificial Intelligence (AI) and Chemistry using a small in-domain\nparallel corpus. We evaluated the effectiveness of two popular NMT models i.e,\nLSTM, and Transformer architectures for the English-Hindi machine translation\ntask based on BLEU scores. We train these models primarily using the out of\ndomain data and employ simple domain adaptation techniques based on the\ncharacteristics of the in-domain dataset. The fine-tuning and mixed-domain data\napproaches are used for domain adaptation. Our team was ranked first in the\nchemistry and general domain En-Hi translation task and second in the AI domain\nEn-Hi translation task.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12184v1",
            "date": "2020-12-17 12:58:06+00:00",
            "title": "COVID-19 Emotion Monitoring as a Tool to Increase Preparedness for Disease Outbreaks in Developing Regions",
            "authors": [
                "Santiago Cortes",
                "Juan Mu\u00f1oz",
                "David Betancur",
                "Mauricio Toro"
            ],
            "abstract": "The COVID-19 pandemic brought many challenges, from hospital-occupation\nmanagement to lock-down mental-health repercussions such as anxiety or\ndepression. In this work, we present a solution for the later problem by\ndeveloping a Twitter emotion-monitor system based on a state-of-the-art\nnatural-language processing model. The system monitors six different emotions\non accounts in cities, as well as politicians and health-authorities Twitter\naccounts. With an anonymous use of the emotion monitor, health authorities and\nprivate health-insurance companies can develop strategies to tackle problems\nsuch as suicide and clinical depression. The model chosen for such a task is a\nBidirectional-Encoder Representations from Transformers (BERT) pre-trained on a\nSpanish corpus (BETO). The model performed well on a validation dataset. The\nsystem is deployed online as part of a web application for simulation and data\nanalysis of COVID-19, in Colombia, available at\nhttps://epidemiologia-matematica.org.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12350v2",
            "date": "2020-12-22 20:49:52+00:00",
            "title": "ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces",
            "authors": [
                "Zecheng He",
                "Srinivas Sunkara",
                "Xiaoxue Zang",
                "Ying Xu",
                "Lijuan Liu",
                "Nevan Wichers",
                "Gabriel Schubiner",
                "Ruby Lee",
                "Jindong Chen",
                "Blaise Ag\u00fcera y Arcas"
            ],
            "abstract": "As mobile devices are becoming ubiquitous, regularly interacting with a\nvariety of user interfaces (UIs) is a common aspect of daily life for many\npeople. To improve the accessibility of these devices and to enable their usage\nin a variety of settings, building models that can assist users and accomplish\ntasks through the UI is vitally important. However, there are several\nchallenges to achieve this. First, UI components of similar appearance can have\ndifferent functionalities, making understanding their function more important\nthan just analyzing their appearance. Second, domain-specific features like\nDocument Object Model (DOM) in web pages and View Hierarchy (VH) in mobile\napplications provide important signals about the semantics of UI elements, but\nthese features are not in a natural language format. Third, owing to a large\ndiversity in UIs and absence of standard DOM or VH representations, building a\nUI understanding model with high coverage requires large amounts of training\ndata.\n  Inspired by the success of pre-training based approaches in NLP for tackling\na variety of problems in a data-efficient way, we introduce a new pre-trained\nUI representation model called ActionBert. Our methodology is designed to\nleverage visual, linguistic and domain-specific features in user interaction\ntraces to pre-train generic feature representations of UIs and their\ncomponents. Our key intuition is that user actions, e.g., a sequence of clicks\non different UI components, reveals important information about their\nfunctionality. We evaluate the proposed model on a wide variety of downstream\ntasks, ranging from icon classification to UI component retrieval based on its\nnatural language description. Experiments show that the proposed ActionBert\nmodel outperforms multi-modal baselines across all downstream tasks by up to\n15.5%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12352v3",
            "date": "2020-12-22 21:01:44+00:00",
            "title": "Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks",
            "authors": [
                "Letitia Parcalabescu",
                "Albert Gatt",
                "Anette Frank",
                "Iacer Calixto"
            ],
            "abstract": "We investigate the reasoning ability of pretrained vision and language (V&L)\nmodels in two tasks that require multimodal integration: (1) discriminating a\ncorrect image-sentence pair from an incorrect one, and (2) counting entities in\nan image. We evaluate three pretrained V&L models on these tasks: ViLBERT,\nViLBERT 12-in-1 and LXMERT, in zero-shot and finetuned settings. Our results\nshow that models solve task (1) very well, as expected, since all models are\npretrained on task (1). However, none of the pretrained V&L models is able to\nadequately solve task (2), our counting probe, and they cannot generalise to\nout-of-distribution quantities. We propose a number of explanations for these\nfindings: LXMERT (and to some extent ViLBERT 12-in-1) show some evidence of\ncatastrophic forgetting on task (1). Concerning our results on the counting\nprobe, we find evidence that all models are impacted by dataset bias, and also\nfail to individuate entities in the visual input. While a selling point of\npretrained V&L models is their ability to solve complex tasks, our findings\nsuggest that understanding their reasoning and grounding capabilities requires\nmore targeted investigations on specific phenomena.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2012.12465v1",
            "date": "2020-12-23 03:04:49+00:00",
            "title": "Future-Guided Incremental Transformer for Simultaneous Translation",
            "authors": [
                "Shaolei Zhang",
                "Yang Feng",
                "Liangyou Li"
            ],
            "abstract": "Simultaneous translation (ST) starts translations synchronously while reading\nsource sentences, and is used in many online scenarios. The previous wait-k\npolicy is concise and achieved good results in ST. However, wait-k policy faces\ntwo weaknesses: low training speed caused by the recalculation of hidden states\nand lack of future source information to guide training. For the low training\nspeed, we propose an incremental Transformer with an average embedding layer\n(AEL) to accelerate the speed of calculation of the hidden states during\ntraining. For future-guided training, we propose a conventional Transformer as\nthe teacher of the incremental Transformer, and try to invisibly embed some\nfuture information in the model through knowledge distillation. We conducted\nexperiments on Chinese-English and German-English simultaneous translation\ntasks and compared with the wait-k policy to evaluate the proposed method. Our\nmethod can effectively increase the training speed by about 28 times on average\nat different k and implicitly embed some predictive abilities in the model,\nachieving better translation quality than wait-k baseline.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12573v1",
            "date": "2020-12-23 10:01:18+00:00",
            "title": "Automated Lay Language Summarization of Biomedical Scientific Reviews",
            "authors": [
                "Yue Guo",
                "Wei Qiu",
                "Yizhong Wang",
                "Trevor Cohen"
            ],
            "abstract": "Health literacy has emerged as a crucial factor in making appropriate health\ndecisions and ensuring treatment outcomes. However, medical jargon and the\ncomplex structure of professional language in this domain make health\ninformation especially hard to interpret. Thus, there is an urgent unmet need\nfor automated methods to enhance the accessibility of the biomedical literature\nto the general population. This problem can be framed as a type of translation\nproblem between the language of healthcare professionals, and that of the\ngeneral public. In this paper, we introduce the novel task of automated\ngeneration of lay language summaries of biomedical scientific reviews, and\nconstruct a dataset to support the development and evaluation of automated\nmethods through which to enhance the accessibility of the biomedical\nliterature. We conduct analyses of the various challenges in solving this task,\nincluding not only summarization of the key points but also explanation of\nbackground knowledge and simplification of professional language. We experiment\nwith state-of-the-art summarization models as well as several data augmentation\ntechniques, and evaluate their performance using both automated metrics and\nhuman assessment. Results indicate that automatically generated summaries\nproduced using contemporary neural architectures can achieve promising quality\nand readability as compared with reference summaries developed for the lay\npublic by experts (best ROUGE-L of 50.24 and Flesch-Kincaid readability score\nof 13.30). We also discuss the limitations of the current attempt, providing\ninsights and directions for future work.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12756v1",
            "date": "2020-12-23 15:52:27+00:00",
            "title": "EmotionGIF-IITP-AINLPML: Ensemble-based Automated Deep Neural System for predicting category(ies) of a GIF response",
            "authors": [
                "Soumitra Ghosh",
                "Arkaprava Roy",
                "Asif Ekbal",
                "Pushpak Bhattacharyya"
            ],
            "abstract": "In this paper, we describe the systems submitted by our IITP-AINLPML team in\nthe shared task of SocialNLP 2020, EmotionGIF 2020, on predicting the\ncategory(ies) of a GIF response for a given unlabelled tweet. For the round 1\nphase of the task, we propose an attention-based Bi-directional GRU network\ntrained on both the tweet (text) and their replies (text wherever available)\nand the given category(ies) for its GIF response. In the round 2 phase, we\nbuild several deep neural-based classifiers for the task and report the final\npredictions through a majority voting based ensemble technique. Our proposed\nmodels attain the best Mean Recall (MR) scores of 52.92% and 53.80% in round 1\nand round 2, respectively.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.12871v2",
            "date": "2020-12-23 18:37:11+00:00",
            "title": "A Multimodal Framework for the Detection of Hateful Memes",
            "authors": [
                "Phillip Lippe",
                "Nithin Holla",
                "Shantanu Chandra",
                "Santhosh Rajamanickam",
                "Georgios Antoniou",
                "Ekaterina Shutova",
                "Helen Yannakoudakis"
            ],
            "abstract": "An increasingly common expression of online hate speech is multimodal in\nnature and comes in the form of memes. Designing systems to automatically\ndetect hateful content is of paramount importance if we are to mitigate its\nundesirable effects on the society at large. The detection of multimodal hate\nspeech is an intrinsically difficult and open problem: memes convey a message\nusing both images and text and, hence, require multimodal reasoning and joint\nvisual and language understanding. In this work, we seek to advance this line\nof research and develop a multimodal framework for the detection of hateful\nmemes. We improve the performance of existing multimodal approaches beyond\nsimple fine-tuning and, among others, show the effectiveness of upsampling of\ncontrastive examples to encourage multimodality and ensemble learning based on\ncross-validation to improve robustness. We furthermore analyze model\nmisclassifications and discuss a number of hypothesis-driven augmentations and\ntheir effects on performance, presenting important implications for future\nresearch in the field. Our best approach comprises an ensemble of UNITER-based\nmodels and achieves an AUROC score of 80.53, placing us 4th on phase 2 of the\n2020 Hateful Memes Challenge organized by Facebook.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13031v2",
            "date": "2020-12-24 00:01:40+00:00",
            "title": "Disentangling semantics in language through VAEs and a certain architectural choice",
            "authors": [
                "Ghazi Felhi",
                "Joseph Le Roux",
                "Djam\u00e9 Seddah"
            ],
            "abstract": "We present an unsupervised method to obtain disentangled representations of\nsentences that single out semantic content. Using modified Transformers as\nbuilding blocks, we train a Variational Autoencoder to translate the sentence\nto a fixed number of hierarchically structured latent variables. We study the\ninfluence of each latent variable in generation on the dependency structure of\nsentences, and on the predicate structure it yields when passed through an Open\nInformation Extraction model. Our model could separate verbs, subjects, direct\nobjects, and prepositional objects into latent variables we identified. We show\nthat varying the corresponding latent variables results in varying these\nelements in sentences, and that swapping them between couples of sentences\nleads to the expected partial semantic swap.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13048v1",
            "date": "2020-12-24 00:55:46+00:00",
            "title": "ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language",
            "authors": [
                "Oyvind Tafjord",
                "Bhavana Dalvi Mishra",
                "Peter Clark"
            ],
            "abstract": "Transformers have been shown to emulate logical deduction over natural\nlanguage theories (logical rules expressed in natural language), reliably\nassigning true/false labels to candidate implications. However, their ability\nto generate implications of a theory has not yet been demonstrated, and methods\nfor reconstructing proofs of answers are imperfect. In this work we show that a\ngenerative model, called ProofWriter, can reliably generate both implications\nof a theory and the natural language proof(s) that support them. In particular,\niterating a 1-step implication generator results in proofs that are highly\nreliable, and represent actual model decisions (rather than post-hoc\nrationalizations). On the RuleTaker dataset, the accuracy of ProofWriter's\nproofs exceed previous methods by +9% absolute, and in a way that generalizes\nto proof depths unseen in training and on out-of-domain problems. We also show\nthat generative techniques can perform a type of abduction with high precision:\nGiven a theory and an unprovable conclusion, identify a missing fact that\nallows the conclusion to be proved, along with a proof. These results\nsignificantly improve the viability of neural methods for systematically\nreasoning over natural language.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13185v3",
            "date": "2020-12-24 10:07:32+00:00",
            "title": "REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement",
            "authors": [
                "Yinya Huang",
                "Meng Fang",
                "Xunlin Zhan",
                "Qingxing Cao",
                "Xiaodan Liang",
                "Liang Lin"
            ],
            "abstract": "When answering a question, people often draw upon their rich world knowledge\nin addition to the particular context. While recent works retrieve supporting\nfacts/evidence from commonsense knowledge bases to supply additional\ninformation to each question, there is still ample opportunity to advance it on\nthe quality of the evidence. It is crucial since the quality of the evidence is\nthe key to answering commonsense questions, and even determines the upper bound\non the QA systems performance. In this paper, we propose a recursive erasure\nmemory network (REM-Net) to cope with the quality improvement of evidence. To\naddress this, REM-Net is equipped with a module to refine the evidence by\nrecursively erasing the low-quality evidence that does not explain the question\nanswering. Besides, instead of retrieving evidence from existing knowledge\nbases, REM-Net leverages a pre-trained generative model to generate candidate\nevidence customized for the question. We conduct experiments on two commonsense\nquestion answering datasets, WIQA and CosmosQA. The results demonstrate the\nperformance of REM-Net and show that the refined evidence is explainable.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13189v2",
            "date": "2020-12-24 10:32:41+00:00",
            "title": "Sentence-Based Model Agnostic NLP Interpretability",
            "authors": [
                "Yves Rychener",
                "Xavier Renard",
                "Djam\u00e9 Seddah",
                "Pascal Frossard",
                "Marcin Detyniecki"
            ],
            "abstract": "Today, interpretability of Black-Box Natural Language Processing (NLP) models\nbased on surrogates, like LIME or SHAP, uses word-based sampling to build the\nexplanations. In this paper we explore the use of sentences to tackle NLP\ninterpretability. While this choice may seem straight forward, we show that,\nwhen using complex classifiers like BERT, the word-based approach raises issues\nnot only of computational complexity, but also of an out of distribution\nsampling, eventually leading to non founded explanations. By using sentences,\nthe altered text remains in-distribution and the dimensionality of the problem\nis reduced for better fidelity to the black-box at comparable computational\ncomplexity.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13190v2",
            "date": "2020-12-24 10:43:20+00:00",
            "title": "QUACKIE: A NLP Classification Task With Ground Truth Explanations",
            "authors": [
                "Yves Rychener",
                "Xavier Renard",
                "Djam\u00e9 Seddah",
                "Pascal Frossard",
                "Marcin Detyniecki"
            ],
            "abstract": "NLP Interpretability aims to increase trust in model predictions. This makes\nevaluating interpretability approaches a pressing issue. There are multiple\ndatasets for evaluating NLP Interpretability, but their dependence on human\nprovided ground truths raises questions about their unbiasedness. In this work,\nwe take a different approach and formulate a specific classification task by\ndiverting question-answering datasets. For this custom classification task, the\ninterpretability ground-truth arises directly from the definition of the\nclassification problem. We use this method to propose a benchmark and lay the\ngroundwork for future research in NLP interpretability by evaluating a wide\nrange of current state of the art methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13255v1",
            "date": "2020-12-22 07:42:30+00:00",
            "title": "Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning",
            "authors": [
                "Armen Aghajanyan",
                "Luke Zettlemoyer",
                "Sonal Gupta"
            ],
            "abstract": "Although pretrained language models can be fine-tuned to produce\nstate-of-the-art results for a very wide range of language understanding tasks,\nthe dynamics of this process are not well understood, especially in the low\ndata regime. Why can we use relatively vanilla gradient descent algorithms\n(e.g., without strong regularization) to tune a model with hundreds of millions\nof parameters on datasets with only hundreds or thousands of labeled examples?\nIn this paper, we argue that analyzing fine-tuning through the lens of\nintrinsic dimension provides us with empirical and theoretical intuitions to\nexplain this remarkable phenomenon. We empirically show that common pre-trained\nmodels have a very low intrinsic dimension; in other words, there exists a low\ndimension reparameterization that is as effective for fine-tuning as the full\nparameter space. For example, by optimizing only 200 trainable parameters\nrandomly projected back into the full space, we can tune a RoBERTa model to\nachieve 90\\% of the full parameter performance levels on MRPC. Furthermore, we\nempirically show that pre-training implicitly minimizes intrinsic dimension\nand, perhaps surprisingly, larger models tend to have lower intrinsic dimension\nafter a fixed number of pre-training updates, at least in part explaining their\nextreme effectiveness. Lastly, we connect intrinsic dimensionality with low\ndimensional task representations and compression based generalization bounds to\nprovide intrinsic-dimension-based generalization bounds that are independent of\nthe full parameter count.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2012.13568v2",
            "date": "2020-12-25 12:27:44+00:00",
            "title": "Towards a Universal Continuous Knowledge Base",
            "authors": [
                "Gang Chen",
                "Maosong Sun",
                "Yang Liu"
            ],
            "abstract": "In artificial intelligence (AI), knowledge is the information required by an\nintelligent system to accomplish tasks. While traditional knowledge bases use\ndiscrete, symbolic representations, detecting knowledge encoded in the\ncontinuous representations learned from data has received increasing attention\nrecently. In this work, we propose a method for building a continuous knowledge\nbase (CKB) that can store knowledge imported from multiple, diverse neural\nnetworks. The key idea of our approach is to define an interface for each\nneural network and cast knowledge transferring as a function simulation\nproblem. Experiments on text classification show promising results: the CKB\nimports knowledge from a single model and then exports the knowledge to a new\nmodel, achieving comparable performance with the original model. More\ninteresting, we import the knowledge from multiple models to the knowledge\nbase, from which the fused knowledge is exported back to a single model,\nachieving a higher accuracy than the original model. With the CKB, it is also\neasy to achieve knowledge distillation and transfer learning. Our work opens\nthe door to building a universal continuous knowledge base to collect, store,\nand organize all continuous knowledge encoded in various neural networks\ntrained for different AI tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13575v1",
            "date": "2020-12-25 13:50:03+00:00",
            "title": "Contextual Temperature for Language Modeling",
            "authors": [
                "Pei-Hsin Wang",
                "Sheng-Iou Hsieh",
                "Shih-Chieh Chang",
                "Yu-Ting Chen",
                "Jia-Yu Pan",
                "Wei Wei",
                "Da-Chang Juan"
            ],
            "abstract": "Temperature scaling has been widely used as an effective approach to control\nthe smoothness of a distribution, which helps the model performance in various\ntasks. Current practices to apply temperature scaling assume either a fixed, or\na manually-crafted dynamically changing schedule. However, our studies indicate\nthat the individual optimal trajectory for each class can change with the\ncontext. To this end, we propose contextual temperature, a generalized approach\nthat learns an optimal temperature trajectory for each vocabulary over the\ncontext. Experimental results confirm that the proposed method significantly\nimproves state-of-the-art language models, achieving a perplexity of 55.31 and\n62.89 on the test set of Penn Treebank and WikiText-2, respectively. In-depth\nanalyses show that the behaviour of the learned temperature schedules varies\ndramatically by vocabulary, and that the optimal schedules help in controlling\nthe uncertainties. These evidences further justify the need for the proposed\nmethod and its advantages over fixed temperature schedules.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13577v1",
            "date": "2020-12-25 13:57:04+00:00",
            "title": "LOREN: Logic Enhanced Neural Reasoning for Fact Verification",
            "authors": [
                "Jiangjie Chen",
                "Qiaoben Bao",
                "Jiaze Chen",
                "Changzhi Sun",
                "Hao Zhou",
                "Yanghua Xiao",
                "Lei Li"
            ],
            "abstract": "Given a natural language statement, how to verify whether it is supported,\nrefuted, or unknown according to a large-scale knowledge source like Wikipedia?\nExisting neural-network-based methods often regard a sentence as a whole. While\nwe argue that it is beneficial to decompose a statement into multiple\nverifiable logical points. In this paper, we propose LOREN, a novel approach\nfor fact verification that integrates both Logic guided Reasoning and Neural\ninference. The key insight of LOREN is that it decomposes a statement into\nmultiple reasoning units around the central phrases. Instead of directly\nvalidating a single reasoning unit, LOREN turns it into a question-answering\ntask and calculates the confidence of every single hypothesis using neural\nnetworks in the embedding space. They are aggregated to make a final prediction\nusing a neural joint reasoner guided by a set of three-valued logic rules.\nLOREN enjoys the additional merit of interpretability -- it is easy to explain\nhow it reaches certain results with intermediate results and why it makes\nmistakes. We evaluate LOREN on FEVER, a public benchmark for fact verification.\nExperiments show that our proposed LOREN outperforms other previously published\nmethods and achieves 73.43% of the FEVER score.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13838v1",
            "date": "2020-12-27 00:35:43+00:00",
            "title": "Inserting Information Bottlenecks for Attribution in Transformers",
            "authors": [
                "Zhiying Jiang",
                "Raphael Tang",
                "Ji Xin",
                "Jimmy Lin"
            ],
            "abstract": "Pretrained transformers achieve the state of the art across tasks in natural\nlanguage processing, motivating researchers to investigate their inner\nmechanisms. One common direction is to understand what features are important\nfor prediction. In this paper, we apply information bottlenecks to analyze the\nattribution of each feature for prediction on a black-box model. We use BERT as\nthe example and evaluate our approach both quantitatively and qualitatively. We\nshow the effectiveness of our method in terms of attribution and the ability to\nprovide insight into how information flows through layers. We demonstrate that\nour technique outperforms two competitive methods in degradation tests on four\ndatasets. Code is available at https://github.com/bazingagin/IBA.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.13985v1",
            "date": "2020-12-27 18:06:26+00:00",
            "title": "Explaining NLP Models via Minimal Contrastive Editing (MiCE)",
            "authors": [
                "Alexis Ross",
                "Ana Marasovi\u0107",
                "Matthew E. Peters"
            ],
            "abstract": "Humans give contrastive explanations that explain why an observed event\nhappened rather than some other counterfactual event (the contrast case).\nDespite the important role that contrastivity plays in how people generate and\nevaluate explanations, this property is largely missing from current methods\nfor explaining NLP models. We present Minimal Contrastive Editing (MiCE), a\nmethod for generating contrastive explanations of model predictions in the form\nof edits to inputs that change model outputs to the contrast case. Our\nexperiments across three tasks -- binary sentiment classification, topic\nclassification, and multiple-choice question answering -- show that MiCE is\nable to produce edits that are not only contrastive, but also minimal and\nfluent, consistent with human contrastive edits. We demonstrate how MiCE edits\ncan be used for two use cases in NLP system development -- uncovering dataset\nartifacts and debugging incorrect model predictions -- and thereby illustrate\nthat generating contrastive explanations is a promising research direction for\nmodel interpretability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14011v1",
            "date": "2020-12-27 21:03:40+00:00",
            "title": "SMART: A Situation Model for Algebra Story Problems via Attributed Grammar",
            "authors": [
                "Yining Hong",
                "Qing Li",
                "Ran Gong",
                "Daniel Ciao",
                "Siyuan Huang",
                "Song-Chun Zhu"
            ],
            "abstract": "Solving algebra story problems remains a challenging task in artificial\nintelligence, which requires a detailed understanding of real-world situations\nand a strong mathematical reasoning capability. Previous neural solvers of math\nword problems directly translate problem texts into equations, lacking an\nexplicit interpretation of the situations, and often fail to handle more\nsophisticated situations. To address such limits of neural solvers, we\nintroduce the concept of a \\emph{situation model}, which originates from\npsychology studies to represent the mental states of humans in problem-solving,\nand propose \\emph{SMART}, which adopts attributed grammar as the representation\nof situation models for algebra story problems. Specifically, we first train an\ninformation extraction module to extract nodes, attributes, and relations from\nproblem texts and then generate a parse graph based on a pre-defined attributed\ngrammar. An iterative learning strategy is also proposed to improve the\nperformance of SMART further. To rigorously study this task, we carefully\ncurate a new dataset named \\emph{ASP6.6k}. Experimental results on ASP6.6k show\nthat the proposed model outperforms all previous neural solvers by a large\nmargin while preserving much better interpretability. To test these models'\ngeneralization capability, we also design an out-of-distribution (OOD)\nevaluation, in which problems are more complex than those in the training set.\nOur model exceeds state-of-the-art models by 17\\% in the OOD evaluation,\ndemonstrating its superior generalization ability.",
            "journal ref": "AAAI2021",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14072v1",
            "date": "2020-12-28 02:44:49+00:00",
            "title": "Automatic Curriculum Learning With Over-repetition Penalty for Dialogue Policy Learning",
            "authors": [
                "Yangyang Zhao",
                "Zhenyu Wang",
                "Zhenhua Huang"
            ],
            "abstract": "Dialogue policy learning based on reinforcement learning is difficult to be\napplied to real users to train dialogue agents from scratch because of the high\ncost. User simulators, which choose random user goals for the dialogue agent to\ntrain on, have been considered as an affordable substitute for real users.\nHowever, this random sampling method ignores the law of human learning, making\nthe learned dialogue policy inefficient and unstable. We propose a novel\nframework, Automatic Curriculum Learning-based Deep Q-Network (ACL-DQN), which\nreplaces the traditional random sampling method with a teacher policy model to\nrealize the dialogue policy for automatic curriculum learning. The teacher\nmodel arranges a meaningful ordered curriculum and automatically adjusts it by\nmonitoring the learning progress of the dialogue agent and the over-repetition\npenalty without any requirement of prior knowledge. The learning progress of\nthe dialogue agent reflects the relationship between the dialogue agent's\nability and the sampled goals' difficulty for sample efficiency. The\nover-repetition penalty guarantees the sampled diversity. Experiments show that\nthe ACL-DQN significantly improves the effectiveness and stability of dialogue\ntasks with a statistically significant margin. Furthermore, the framework can\nbe further improved by equipping with different curriculum schedules, which\ndemonstrates that the framework has strong generalizability.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14124v1",
            "date": "2020-12-28 07:25:10+00:00",
            "title": "Neural Text Generation with Artificial Negative Examples",
            "authors": [
                "Keisuke Shirai",
                "Kazuma Hashimoto",
                "Akiko Eriguchi",
                "Takashi Ninomiya",
                "Shinsuke Mori"
            ],
            "abstract": "Neural text generation models conditioning on given input (e.g. machine\ntranslation and image captioning) are usually trained by maximum likelihood\nestimation of target text. However, the trained models suffer from various\ntypes of errors at inference time. In this paper, we propose to suppress an\narbitrary type of errors by training the text generation model in a\nreinforcement learning framework, where we use a trainable reward function that\nis capable of discriminating between references and sentences containing the\ntargeted type of errors. We create such negative examples by artificially\ninjecting the targeted errors to the references. In experiments, we focus on\ntwo error types, repeated and dropped tokens in model-generated text. The\nexperimental results show that our method can suppress the generation errors\nand achieve significant improvements on two machine translation and two image\ncaptioning tasks.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14210v1",
            "date": "2020-12-28 12:25:25+00:00",
            "title": "The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes",
            "authors": [
                "Nils Reimers",
                "Iryna Gurevych"
            ],
            "abstract": "Information Retrieval using dense low-dimensional representations recently\nbecame popular and showed out-performance to traditional sparse-representations\nlike BM25. However, no previous work investigated how dense representations\nperform with large index sizes. We show theoretically and empirically that the\nperformance for dense representations decreases quicker than sparse\nrepresentations for increasing index sizes. In extreme cases, this can even\nlead to a tipping point where at a certain index size sparse representations\noutperform dense representations. We show that this behavior is tightly\nconnected to the number of dimensions of the representations: The lower the\ndimension, the higher the chance for false positives, i.e. returning irrelevant\ndocuments.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.14353v2",
            "date": "2020-12-28 16:46:03+00:00",
            "title": "DeepHateExplainer: Explainable Hate Speech Detection in Under-resourced Bengali Language",
            "authors": [
                "Md. Rezaul Karim",
                "Sumon Kanti Dey",
                "Bharathi Raja Chakravarthi"
            ],
            "abstract": "The exponential growths of social media and micro-blogging sites not only\nprovide platforms for empowering freedom of expression and individual voices,\nbut also enables people to express anti-social behavior like online harassment,\ncyberbullying, and hate speech. Numerous works have been proposed to utilize\nthe textual data for social and anti-social behavior analysis, by predicting\nthe contexts mostly for highly-resourced languages like English. However, some\nlanguages are under-resourced, e.g., South Asian languages like Bengali, that\nlack computational resources for accurate natural language processing (NLP). In\nthis paper, we propose an explainable approach for hate speech detection from\nthe under-resourced Bengali language, which we called DeepHateExplainer. In our\napproach, Bengali texts are first comprehensively preprocessed, before\nclassifying them into political, personal, geopolitical, and religious hates,\nby employing the neural ensemble method of different transformer-based neural\narchitectures (i.e., monolingual Bangla BERT-base, multilingual\nBERT-cased/uncased, and XLM-RoBERTa). Subsequently, important (most and least)\nterms are identified with sensitivity analysis and layer-wise relevance\npropagation (LRP), before providing human-interpretable explanations. Finally,\nto measure the quality of the explanation (i.e., faithfulness), we compute the\ncomprehensiveness and sufficiency. Evaluations against machine learning (linear\nand tree-based models) and deep neural networks (i.e., CNN, Bi-LSTM, and\nConv-LSTM with word embeddings) baselines yield F1 scores of 84%, 90%, 88%, and\n88%, for political, personal, geopolitical, and religious hates, respectively,\noutperforming both ML and DNN baselines.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14653v1",
            "date": "2020-12-29 08:22:48+00:00",
            "title": "Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents",
            "authors": [
                "Yi-Chia Wang",
                "Alexandros Papangelis",
                "Runze Wang",
                "Zhaleh Feizollahi",
                "Gokhan Tur",
                "Robert Kraut"
            ],
            "abstract": "Goal-oriented conversational agents are becoming prevalent in our daily\nlives. For these systems to engage users and achieve their goals, they need to\nexhibit appropriate social behavior as well as provide informative replies that\nguide users through tasks. The first component of the research in this paper\napplies statistical modeling techniques to understand conversations between\nusers and human agents for customer service. Analyses show that social language\nused by human agents is associated with greater users' responsiveness and task\ncompletion. The second component of the research is the construction of a\nconversational agent model capable of injecting social language into an agent's\nresponses while still preserving content. The model uses a sequence-to-sequence\ndeep learning architecture, extended with a social language understanding\nelement. Evaluation in terms of content preservation and social language level\nusing both human judgment and automatic linguistic measures shows that the\nmodel can generate responses that enable agents to address users' issues in a\nmore socially appropriate way.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14666v1",
            "date": "2020-12-29 08:58:49+00:00",
            "title": "RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems",
            "authors": [
                "Baolin Peng",
                "Chunyuan Li",
                "Zhu Zhang",
                "Chenguang Zhu",
                "Jinchao Li",
                "Jianfeng Gao"
            ],
            "abstract": "For task-oriented dialog systems to be maximally useful, it must be able to\nprocess conversations in a way that is (1) generalizable with a small number of\ntraining examples for new task domains, and (2) robust to user input in various\nstyles, modalities or domains. In pursuit of these goals, we introduce the\nRADDLE benchmark, a collection of corpora and tools for evaluating the\nperformance of models across a diverse set of domains. By including tasks with\nlimited training data, RADDLE is designed to favor and encourage models with a\nstrong generalization ability. RADDLE also includes a diagnostic checklist that\nfacilitates detailed robustness analysis in aspects such as language\nvariations, speech errors, unseen entities, and out-of-domain utterances. We\nevaluate recent state-of-the-art systems based on pre-training and fine-tuning,\nand find that grounded pre-training on heterogeneous dialog corpora performs\nbetter than training a separate model per domain. Overall, existing models are\nless than satisfactory in robustness evaluation, which suggests opportunities\nfor future improvement.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14681v2",
            "date": "2020-12-29 09:43:27+00:00",
            "title": "Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation",
            "authors": [
                "Hyojung Han",
                "Sathish Indurthi",
                "Mohd Abbas Zaidi",
                "Nikhil Kumar Lakumarapu",
                "Beomseok Lee",
                "Sangha Kim",
                "Chanwoo Kim",
                "Inchul Hwang"
            ],
            "abstract": "Recently, simultaneous translation has gathered a lot of attention since it\nenables compelling applications such as subtitle translation for a live event\nor real-time video-call translation. Some of these translation applications\nallow editing of partial translation giving rise to re-translation approaches.\nThe current re-translation approaches are based on autoregressive sequence\ngeneration models (ReTA), which generate tar-get tokens in the (partial)\ntranslation sequentially. The multiple re-translations with sequential\ngeneration inReTAmodelslead to an increased inference time gap between the\nincoming source input and the corresponding target output as the source input\ngrows. Besides, due to the large number of inference operations involved, the\nReTA models are not favorable for resource-constrained devices. In this work,\nwe propose a faster re-translation system based on a non-autoregressive\nsequence generation model (FReTNA) to overcome the aforementioned limitations.\nWe evaluate the proposed model on multiple translation tasks and our model\nreduces the inference times by several orders and achieves a competitive\nBLEUscore compared to the ReTA and streaming (Wait-k) models.The proposed model\nreduces the average computation time by a factor of 20 when compared to the\nReTA model by incurring a small drop in the translation quality. It also\noutperforms the streaming-based Wait-k model both in terms of computation time\n(1.5 times lower) and translation quality.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14768v2",
            "date": "2020-12-29 14:26:59+00:00",
            "title": "Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning",
            "authors": [
                "Xuebo Liu",
                "Longyue Wang",
                "Derek F. Wong",
                "Liang Ding",
                "Lidia S. Chao",
                "Zhaopeng Tu"
            ],
            "abstract": "Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder\nlayers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq)\nmodels, which has proven effective on various NLP tasks. However, it is still\nnot entirely clear why and when EncoderFusion should work. In this paper, our\nmain contribution is to take a step further in understanding EncoderFusion.\nMany of previous studies believe that the success of EncoderFusion comes from\nexploiting surface and syntactic information embedded in lower encoder layers.\nUnlike them, we find that the encoder embedding layer is more important than\nother intermediate encoder layers. In addition, the uppermost decoder layer\nconsistently pays more attention to the encoder embedding layer across NLP\ntasks. Based on this observation, we propose a simple fusion method,\nSurfaceFusion, by fusing only the encoder embedding layer for the softmax\nlayer. Experimental results show that SurfaceFusion outperforms EncoderFusion\non several NLP benchmarks, including machine translation, text summarization,\nand grammatical error correction. It obtains the state-of-the-art performance\non WMT16 Romanian-English and WMT14 English-French translation tasks. Extensive\nanalyses reveal that SurfaceFusion learns more expressive bilingual word\nembeddings by building a closer relationship between relevant source and target\nembedding. Source code is freely available at\nhttps://github.com/SunbowLiu/SurfaceFusion.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14778v3",
            "date": "2020-12-29 14:44:13+00:00",
            "title": "Combining Semilattices and Semimodules",
            "authors": [
                "Filippo Bonchi",
                "Alessio Santamaria"
            ],
            "abstract": "We describe the canonical weak distributive law $\\delta \\colon \\mathcal S\n\\mathcal P \\to \\mathcal P \\mathcal S$ of the powerset monad $\\mathcal P$ over\nthe $S$-left-semimodule monad $\\mathcal S$, for a class of semirings $S$. We\nshow that the composition of $\\mathcal P$ with $\\mathcal S$ by means of such\n$\\delta$ yields almost the monad of convex subsets previously introduced by\nJacobs: the only difference consists in the absence in Jacobs's monad of the\nempty convex set. We provide a handy characterisation of the canonical weak\nlifting of $\\mathcal P$ to $\\mathbb{EM}(\\mathcal S)$ as well as an algebraic\ntheory for the resulting composed monad. Finally, we restrict the composed\nmonad to finitely generated convex subsets and we show that it is presented by\nan algebraic theory combining semimodules and semilattices with bottom, which\nare the algebras for the finite powerset monad $\\mathcal P_f$.",
            "journal ref": "Foundations of Software Science and Computation Structures.\n  FOSSACS 2021. Lecture Notes in Computer Science, vol 12650 (2021), pp\n  102-123. Springer, Cham",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14827v3",
            "date": "2020-12-29 16:08:36+00:00",
            "title": "Dialogue Graph Modeling for Conversational Machine Reading",
            "authors": [
                "Siru Ouyang",
                "Zhuosheng Zhang",
                "Hai Zhao"
            ],
            "abstract": "Conversational Machine Reading (CMR) aims at answering questions in a\ncomplicated manner. Machine needs to answer questions through interactions with\nusers based on given rule document, user scenario and dialogue history, and ask\nquestions to clarify if necessary. In this paper, we propose a dialogue graph\nmodeling framework to improve the understanding and reasoning ability of\nmachine on CMR task. There are three types of graph in total. Specifically,\nDiscourse Graph is designed to learn explicitly and extract the discourse\nrelation among rule texts as well as the extra knowledge of scenario;\nDecoupling Graph is used for understanding local and contextualized connection\nwithin rule texts. And finally a global graph for fusing the information\ntogether and reply to the user with our final decision being either\n\"Yes/No/Irrelevant\" or to ask a follow-up question to clarify.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.14862v2",
            "date": "2020-12-29 17:28:53+00:00",
            "title": "Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision",
            "authors": [
                "Si Sun",
                "Yingzhuo Qian",
                "Zhenghao Liu",
                "Chenyan Xiong",
                "Kaitao Zhang",
                "Jie Bao",
                "Zhiyuan Liu",
                "Paul Bennett"
            ],
            "abstract": "The effectiveness of Neural Information Retrieval (Neu-IR) often depends on a\nlarge scale of in-domain relevance training signals, which are not always\navailable in real-world ranking scenarios. To democratize the benefits of\nNeu-IR, this paper presents MetaAdaptRank, a domain adaptive learning method\nthat generalizes Neu-IR models from label-rich source domains to few-shot\ntarget domains. Drawing on source-domain massive relevance supervision,\nMetaAdaptRank contrastively synthesizes a large number of weak supervision\nsignals for target domains and meta-learns to reweight these synthetic \"weak\"\ndata based on their benefits to the target-domain ranking accuracy of Neu-IR\nmodels. Experiments on three TREC benchmarks in the web, news, and biomedical\ndomains show that MetaAdaptRank significantly improves the few-shot ranking\naccuracy of Neu-IR models. Further analyses indicate that MetaAdaptRank thrives\nfrom both its contrastive weak data synthesis and meta-reweighted data\nselection. The code and data of this paper can be obtained from\nhttps://github.com/thunlp/MetaAdaptRank.",
            "journal ref": "None",
            "category": "cs.IR"
        },
        {
            "url": "http://arxiv.org/abs/2012.15022v2",
            "date": "2020-12-30 03:35:22+00:00",
            "title": "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning",
            "authors": [
                "Yujia Qin",
                "Yankai Lin",
                "Ryuichi Takanobu",
                "Zhiyuan Liu",
                "Peng Li",
                "Heng Ji",
                "Minlie Huang",
                "Maosong Sun",
                "Jie Zhou"
            ],
            "abstract": "Pre-trained Language Models (PLMs) have shown superior performance on various\ndownstream Natural Language Processing (NLP) tasks. However, conventional\npre-training objectives do not explicitly model relational facts in text, which\nare crucial for textual understanding. To address this issue, we propose a\nnovel contrastive learning framework ERICA to obtain a deep understanding of\nthe entities and their relations in text. Specifically, we define two novel\npre-training tasks to better understand entities and relations: (1) the entity\ndiscrimination task to distinguish which tail entity can be inferred by the\ngiven head entity and relation; (2) the relation discrimination task to\ndistinguish whether two relations are close or not semantically, which involves\ncomplex relational reasoning. Experimental results demonstrate that ERICA can\nimprove typical PLMs (BERT and RoBERTa) on several language understanding\ntasks, including relation extraction, entity typing and question answering,\nespecially under low-resource settings.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15079v1",
            "date": "2020-12-30 08:31:31+00:00",
            "title": "A Subword Guided Neural Word Segmentation Model for Sindhi",
            "authors": [
                "Wazir Ali",
                "Jay Kumar",
                "Zenglin Xu",
                "Congjian Luo",
                "Junyu Lu",
                "Junming Shao",
                "Rajesh Kumar",
                "Yazhou Ren"
            ],
            "abstract": "Deep neural networks employ multiple processing layers for learning text\nrepresentations to alleviate the burden of manual feature engineering in\nNatural Language Processing (NLP). Such text representations are widely used to\nextract features from unlabeled data. The word segmentation is a fundamental\nand inevitable prerequisite for many languages. Sindhi is an under-resourced\nlanguage, whose segmentation is challenging as it exhibits space omission,\nspace insertion issues, and lacks the labeled corpus for segmentation. In this\npaper, we investigate supervised Sindhi Word Segmentation (SWS) using unlabeled\ndata with a Subword Guided Neural Word Segmenter (SGNWS) for Sindhi. In order\nto learn text representations, we incorporate subword representations to\nrecurrent neural architecture to capture word information at morphemic-level,\nwhich takes advantage of Bidirectional Long-Short Term Memory (BiLSTM),\nself-attention mechanism, and Conditional Random Field (CRF). Our proposed\nSGNWS model achieves an F1 value of 98.51% without relying on feature\nengineering. The empirical results demonstrate the benefits of the proposed\nmodel over the existing Sindhi word segmenters.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15178v1",
            "date": "2020-12-30 14:52:15+00:00",
            "title": "Synthetic Source Language Augmentation for Colloquial Neural Machine Translation",
            "authors": [
                "Asrul Sani Ariesandy",
                "Mukhlis Amien",
                "Alham Fikri Aji",
                "Radityo Eko Prasojo"
            ],
            "abstract": "Neural machine translation (NMT) is typically domain-dependent and\nstyle-dependent, and it requires lots of training data. State-of-the-art NMT\nmodels often fall short in handling colloquial variations of its source\nlanguage and the lack of parallel data in this regard is a challenging hurdle\nin systematically improving the existing models. In this work, we develop a\nnovel colloquial Indonesian-English test-set collected from YouTube transcript\nand Twitter. We perform synthetic style augmentation to the source of formal\nIndonesian language and show that it improves the baseline Id-En models (in\nBLEU) over the new test data.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15197v1",
            "date": "2020-12-30 15:38:26+00:00",
            "title": "SemGloVe: Semantic Co-occurrences for GloVe from BERT",
            "authors": [
                "Leilei Gan",
                "Zhiyang Teng",
                "Yue Zhang",
                "Linchao Zhu",
                "Fei Wu",
                "Yi Yang"
            ],
            "abstract": "GloVe learns word embeddings by leveraging statistical information from word\nco-occurrence matrices. However, word pairs in the matrices are extracted from\na predefined local context window, which might lead to limited word pairs and\npotentially semantic irrelevant word pairs. In this paper, we propose SemGloVe,\nwhich distills semantic co-occurrences from BERT into static GloVe word\nembeddings. Particularly, we propose two models to extract co-occurrence\nstatistics based on either the masked language model or the multi-head\nattention weights of BERT. Our methods can extract word pairs without limiting\nby the local window assumption and can define the co-occurrence weights by\ndirectly considering the semantic distance between word pairs. Experiments on\nseveral word similarity datasets and four external tasks show that SemGloVe can\noutperform GloVe.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15262v2",
            "date": "2020-12-30 18:18:47+00:00",
            "title": "Robustness Testing of Language Understanding in Task-Oriented Dialog",
            "authors": [
                "Jiexi Liu",
                "Ryuichi Takanobu",
                "Jiaxin Wen",
                "Dazhen Wan",
                "Hongguang Li",
                "Weiran Nie",
                "Cheng Li",
                "Wei Peng",
                "Minlie Huang"
            ],
            "abstract": "Most language understanding models in task-oriented dialog systems are\ntrained on a small amount of annotated training data, and evaluated in a small\nset from the same distribution. However, these models can lead to system\nfailure or undesirable output when being exposed to natural language\nperturbation or variation in practice. In this paper, we conduct comprehensive\nevaluation and analysis with respect to the robustness of natural language\nunderstanding models, and introduce three important aspects related to language\nunderstanding in real-world dialog systems, namely, language variety, speech\ncharacteristics, and noise perturbation. We propose a model-agnostic toolkit\nLAUG to approximate natural language perturbations for testing the robustness\nissues in task-oriented dialog. Four data augmentation approaches covering the\nthree aspects are assembled in LAUG, which reveals critical robustness issues\nin state-of-the-art models. The augmented dataset through LAUG can be used to\nfacilitate future research on the robustness testing of language understanding\nin task-oriented dialog.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15329v3",
            "date": "2020-12-30 21:22:04+00:00",
            "title": "Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem",
            "authors": [
                "Raphael Schumann",
                "Stefan Riezler"
            ],
            "abstract": "Car-focused navigation services are based on turns and distances of named\nstreets, whereas navigation instructions naturally used by humans are centered\naround physical objects called landmarks. We present a neural model that takes\nOpenStreetMap representations as input and learns to generate navigation\ninstructions that contain visible and salient landmarks from human natural\nlanguage instructions. Routes on the map are encoded in a location- and\nrotation-invariant graph representation that is decoded into natural language\ninstructions. Our work is based on a novel dataset of 7,672 crowd-sourced\ninstances that have been verified by human navigation in Street View. Our\nevaluation shows that the navigation instructions generated by our system have\nsimilar properties as human-generated instructions, and lead to successful\nhuman navigation in Street View.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15332v1",
            "date": "2020-12-30 21:37:28+00:00",
            "title": "k\u014dan: A Corrected CBOW Implementation",
            "authors": [
                "Ozan \u0130rsoy",
                "Adrian Benton",
                "Karl Stratos"
            ],
            "abstract": "It is a common belief in the NLP community that continuous bag-of-words\n(CBOW) word embeddings tend to underperform skip-gram (SG) embeddings. We find\nthat this belief is founded less on theoretical differences in their training\nobjectives but more on faulty CBOW implementations in standard software\nlibraries such as the official implementation word2vec.c and Gensim. We show\nthat our correct implementation of CBOW yields word embeddings that are fully\ncompetitive with SG on various intrinsic and extrinsic tasks while being more\nthan three times as fast to train. We release our implementation, k\\=oan, at\nhttps://github.com/bloomberg/koan.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15355v4",
            "date": "2020-12-30 22:53:49+00:00",
            "title": "Optimizing Deeper Transformers on Small Datasets",
            "authors": [
                "Peng Xu",
                "Dhruv Kumar",
                "Wei Yang",
                "Wenjie Zi",
                "Keyi Tang",
                "Chenyang Huang",
                "Jackie Chi Kit Cheung",
                "Simon J. D. Prince",
                "Yanshuai Cao"
            ],
            "abstract": "It is a common belief that training deep transformers from scratch requires\nlarge datasets. Consequently, for small datasets, people usually use shallow\nand simple additional layers on top of pre-trained models during fine-tuning.\nThis work shows that this does not always need to be the case: with proper\ninitialization and optimization, the benefits of very deep transformers can\ncarry over to challenging tasks with small datasets, including Text-to-SQL\nsemantic parsing and logical reading comprehension. In particular, we\nsuccessfully train $48$ layers of transformers, comprising $24$ fine-tuned\nlayers from pre-trained RoBERTa and $24$ relation-aware layers trained from\nscratch. With fewer training steps and no task-specific pre-training, we obtain\nthe state-of-the-art performance on the challenging cross-domain Text-to-SQL\nparsing benchmark Spider. We achieve this by deriving a novel Data-dependent\nTransformer Fixed-update initialization scheme (DT-Fixup), inspired by the\nprior T-Fixup work. Further error analysis shows that increasing depth can help\nimprove generalization on small datasets for hard cases that require reasoning\nand structural understanding.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15375v1",
            "date": "2020-12-31 00:02:51+00:00",
            "title": "Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration",
            "authors": [
                "Weiyan Shi",
                "Yu Li",
                "Saurav Sahay",
                "Zhou Yu"
            ],
            "abstract": "Despite the recent success of large-scale language models on various\ndownstream NLP tasks, the repetition and inconsistency problems still persist\nin dialogue response generation. Previous approaches have attempted to avoid\nrepetition by penalizing the language model's undesirable behaviors in the loss\nfunction. However, these methods focus on token-level information and can lead\nto incoherent responses and uninterpretable behaviors. To alleviate these\nissues, we propose to apply reinforcement learning to refine an MLE-based\nlanguage model without user simulators, and distill sentence-level information\nabout repetition, inconsistency and task relevance through rewards. In\naddition, to better accomplish the dialogue task, the model learns from human\ndemonstration to imitate intellectual activities such as persuasion, and\nselects the most persuasive responses. Experiments show that our model\noutperforms previous state-of-the-art dialogue models on both automatic metrics\nand human evaluation results on a donation persuasion task, and generates more\ndiverse, consistent and persuasive conversations according to the user\nfeedback.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15404v1",
            "date": "2020-12-31 02:34:57+00:00",
            "title": "Unified Mandarin TTS Front-end Based on Distilled BERT Model",
            "authors": [
                "Yang Zhang",
                "Liqun Deng",
                "Yasheng Wang"
            ],
            "abstract": "The front-end module in a typical Mandarin text-to-speech system (TTS) is\ncomposed of a long pipeline of text processing components, which requires\nextensive efforts to build and is prone to large accumulative model size and\ncascade errors. In this paper, a pre-trained language model (PLM) based model\nis proposed to simultaneously tackle the two most important tasks in TTS\nfront-end, i.e., prosodic structure prediction (PSP) and grapheme-to-phoneme\n(G2P) conversion. We use a pre-trained Chinese BERT[1] as the text encoder and\nemploy multi-task learning technique to adapt it to the two TTS front-end\ntasks. Then, the BERT encoder is distilled into a smaller model by employing a\nknowledge distillation technique called TinyBERT[2], making the whole model\nsize 25% of that of benchmark pipeline models while maintaining competitive\nperformance on both tasks. With the proposed the methods, we are able to run\nthe whole TTS front-end module in a light and unified manner, which is more\nfriendly to deployment on mobile devices.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2012.15419v1",
            "date": "2020-12-31 03:09:38+00:00",
            "title": "An Experimental Evaluation of Transformer-based Language Models in the Biomedical Domain",
            "authors": [
                "Paul Grouchy",
                "Shobhit Jain",
                "Michael Liu",
                "Kuhan Wang",
                "Max Tian",
                "Nidhi Arora",
                "Hillary Ngai",
                "Faiza Khan Khattak",
                "Elham Dolatabadi",
                "Sedef Akinli Kocak"
            ],
            "abstract": "With the growing amount of text in health data, there have been rapid\nadvances in large pre-trained models that can be applied to a wide variety of\nbiomedical tasks with minimal task-specific modifications. Emphasizing the cost\nof these models, which renders technical replication challenging, this paper\nsummarizes experiments conducted in replicating BioBERT and further\npre-training and careful fine-tuning in the biomedical domain. We also\ninvestigate the effectiveness of domain-specific and domain-agnostic\npre-trained models across downstream biomedical NLP tasks. Our finding confirms\nthat pre-trained models can be impactful in some downstream NLP tasks (QA and\nNER) in the biomedical domain; however, this improvement may not justify the\nhigh cost of domain-specific pre-training.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15484v1",
            "date": "2020-12-31 07:24:55+00:00",
            "title": "Seeing is Knowing! Fact-based Visual Question Answering using Knowledge Graph Embeddings",
            "authors": [
                "Kiran Ramnath",
                "Mark Hasegawa-Johnson"
            ],
            "abstract": "Fact-based Visual Question Answering (FVQA), a challenging variant of VQA,\nrequires a QA-system to include facts from a diverse knowledge graph (KG) in\nits reasoning process to produce an answer. Large KGs, especially common-sense\nKGs, are known to be incomplete, i.e. not all non-existent facts are always\nincorrect. Therefore, being able to reason over incomplete KGs for QA is a\ncritical requirement in real-world applications that has not been addressed\nextensively in the literature. We develop a novel QA architecture that allows\nus to reason over incomplete KGs, something current FVQA state-of-the-art\n(SOTA) approaches lack.We use KG Embeddings, a technique widely used for KG\ncompletion, for the downstream task of FVQA. We also employ a new image\nrepresentation technique we call \"Image-as-Knowledge\" to enable this\ncapability, alongside a simple one-step co-Attention mechanism to attend to\ntext and image during QA. Our FVQA architecture is faster during inference\ntime, being O(m), as opposed to existing FVQA SOTA methods which are O(N logN),\nwhere m is number of vertices, N is number of edges (which is O(m^2)). We\nobserve that our architecture performs comparably in the standard\nanswer-retrieval baseline with existing methods; while for missing-edge\nreasoning, our KG representation outperforms the SOTA representation by 25%,\nand image representation outperforms the SOTA representation by 2.6%.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15495v1",
            "date": "2020-12-31 08:16:29+00:00",
            "title": "Towards Zero-Shot Knowledge Distillation for Natural Language Processing",
            "authors": [
                "Ahmad Rashid",
                "Vasileios Lioutas",
                "Abbas Ghaddar",
                "Mehdi Rezagholizadeh"
            ],
            "abstract": "Knowledge Distillation (KD) is a common knowledge transfer algorithm used for\nmodel compression across a variety of deep learning based natural language\nprocessing (NLP) solutions. In its regular manifestations, KD requires access\nto the teacher's training data for knowledge transfer to the student network.\nHowever, privacy concerns, data regulations and proprietary reasons may prevent\naccess to such data. We present, to the best of our knowledge, the first work\non Zero-Shot Knowledge Distillation for NLP, where the student learns from the\nmuch larger teacher without any task specific data. Our solution combines out\nof domain data and adversarial training to learn the teacher's output\ndistribution. We investigate six tasks from the GLUE benchmark and demonstrate\nthat we can achieve between 75% and 92% of the teacher's classification score\n(accuracy or F1) while compressing the model 30 times.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15504v1",
            "date": "2020-12-31 08:44:25+00:00",
            "title": "Continual Learning in Task-Oriented Dialogue Systems",
            "authors": [
                "Andrea Madotto",
                "Zhaojiang Lin",
                "Zhenpeng Zhou",
                "Seungwhan Moon",
                "Paul Crook",
                "Bing Liu",
                "Zhou Yu",
                "Eunjoon Cho",
                "Zhiguang Wang"
            ],
            "abstract": "Continual learning in task-oriented dialogue systems can allow us to add new\ndomains and functionalities through time without incurring the high cost of a\nwhole system retraining. In this paper, we propose a continual learning\nbenchmark for task-oriented dialogue systems with 37 domains to be learned\ncontinuously in four settings, such as intent recognition, state tracking,\nnatural language generation, and end-to-end. Moreover, we implement and compare\nmultiple existing continual learning baselines, and we propose a simple yet\neffective architectural method based on residual adapters. Our experiments\ndemonstrate that the proposed architectural method and a simple replay-based\nstrategy perform comparably well but they both achieve inferior performance to\nthe multi-task learning baseline, in where all the data are shown at once,\nshowing that continual learning in task-oriented dialogue systems is a\nchallenging task. Furthermore, we reveal several trade-offs between different\ncontinual learning methods in term of parameter usage and memory size, which\nare important in the design of a task-oriented dialogue system. The proposed\nbenchmark is released together with several baselines to promote more research\nin this direction.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15543v1",
            "date": "2020-12-31 10:58:37+00:00",
            "title": "Discovering Dialog Structure Graph for Open-Domain Dialog Generation",
            "authors": [
                "Jun Xu",
                "Zeyang Lei",
                "Haifeng Wang",
                "Zheng-Yu Niu",
                "Hua Wu",
                "Wanxiang Che",
                "Ting Liu"
            ],
            "abstract": "Learning interpretable dialog structure from human-human dialogs yields basic\ninsights into the structure of conversation, and also provides background\nknowledge to facilitate dialog generation. In this paper, we conduct\nunsupervised discovery of dialog structure from chitchat corpora, and then\nleverage it to facilitate dialog generation in downstream systems. To this end,\nwe present a Discrete Variational Auto-Encoder with Graph Neural Network\n(DVAE-GNN), to discover a unified human-readable dialog structure. The\nstructure is a two-layer directed graph that contains session-level semantics\nin the upper-layer vertices, utterance-level semantics in the lower-layer\nvertices, and edges among these semantic vertices. In particular, we integrate\nGNN into DVAE to fine-tune utterance-level semantics for more effective\nrecognition of session-level semantic vertex. Furthermore, to alleviate the\ndifficulty of discovering a large number of utterance-level semantics, we\ndesign a coupling mechanism that binds each utterance-level semantic vertex\nwith a distinct phrase to provide prior semantics. Experimental results on two\nbenchmark corpora confirm that DVAE-GNN can discover meaningful dialog\nstructure, and the use of dialog structure graph as background knowledge can\nfacilitate a graph grounded conversational system to conduct coherent\nmulti-turn dialog generation.",
            "journal ref": "None",
            "category": "cs.AI"
        },
        {
            "url": "http://arxiv.org/abs/2012.15723v2",
            "date": "2020-12-31 17:21:26+00:00",
            "title": "Making Pre-trained Language Models Better Few-shot Learners",
            "authors": [
                "Tianyu Gao",
                "Adam Fisch",
                "Danqi Chen"
            ],
            "abstract": "The recent GPT-3 model (Brown et al., 2020) achieves remarkable few-shot\nperformance solely by leveraging a natural-language prompt and a few task\ndemonstrations as input context. Inspired by their findings, we study few-shot\nlearning in a more practical scenario, where we use smaller language models for\nwhich fine-tuning is computationally efficient. We present LM-BFF--better\nfew-shot fine-tuning of language models--a suite of simple and complementary\ntechniques for fine-tuning language models on a small number of annotated\nexamples. Our approach includes (1) prompt-based fine-tuning together with a\nnovel pipeline for automating prompt generation; and (2) a refined strategy for\ndynamically and selectively incorporating demonstrations into each context.\nFinally, we present a systematic evaluation for analyzing few-shot performance\non a range of NLP tasks, including classification and regression. Our\nexperiments demonstrate that our methods combine to dramatically outperform\nstandard fine-tuning procedures in this low resource setting, achieving up to\n30% absolute improvement, and 11% on average across all tasks. Our approach\nmakes minimal assumptions on task resources and domain expertise, and hence\nconstitutes a strong task-agnostic method for few-shot learning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15738v1",
            "date": "2020-12-31 17:28:01+00:00",
            "title": "Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences",
            "authors": [
                "Denis Emelin",
                "Ronan Le Bras",
                "Jena D. Hwang",
                "Maxwell Forbes",
                "Yejin Choi"
            ],
            "abstract": "In social settings, much of human behavior is governed by unspoken rules of\nconduct. For artificial systems to be fully integrated into social\nenvironments, adherence to such norms is a central prerequisite. We investigate\nwhether contemporary NLG models can function as behavioral priors for systems\ndeployed in social settings by generating action hypotheses that achieve\npredefined goals under moral constraints. Moreover, we examine if models can\nanticipate likely consequences of (im)moral actions, or explain why certain\nactions are preferable by generating relevant norms. For this purpose, we\nintroduce 'Moral Stories', a crowd-sourced dataset of structured, branching\nnarratives for the study of grounded, goal-oriented social reasoning. Finally,\nwe propose decoding strategies that effectively combine multiple expert models\nto significantly improve the quality of generated actions, consequences, and\nnorms compared to strong baselines, e.g. though abductive reasoning.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15761v2",
            "date": "2020-12-31 17:36:48+00:00",
            "title": "Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection",
            "authors": [
                "Bertie Vidgen",
                "Tristan Thrush",
                "Zeerak Waseem",
                "Douwe Kiela"
            ],
            "abstract": "We present a human-and-model-in-the-loop process for dynamically generating\ndatasets and training better performing and more robust hate detection models.\nWe provide a new dataset of ~40,000 entries, generated and labelled by trained\nannotators over four rounds of dynamic data creation. It includes ~15,000\nchallenging perturbations and each hateful entry has fine-grained labels for\nthe type and target of hate. Hateful entries make up 54% of the dataset, which\nis substantially higher than comparable datasets. We show that model\nperformance is substantially improved using this approach. Models trained on\nlater rounds of data collection perform better on test sets and are harder for\nannotators to trick. They also perform better on HateCheck, a suite of\nfunctional tests for online hate detection. We provide the code, dataset and\nannotation guidelines for other researchers to use. Accepted at ACL 2021.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15788v1",
            "date": "2020-12-31 18:11:26+00:00",
            "title": "Factual Error Correction of Claims",
            "authors": [
                "James Thorne",
                "Andreas Vlachos"
            ],
            "abstract": "This paper introduces the task of factual error correction: performing edits\nto a claim so that the generated rewrite is supported by evidence. This serves\ntwo purposes: firstly this provides a mechanism to correct written texts that\ncontain misinformation, and secondly, this acts as an inherent explanation for\nclaims already partially supported by evidence. We demonstrate that factual\nerror correction is possible without the need for any additional training data\nusing distant-supervision and retrieved evidence. We release a dataset of\n65,000 instances, based on a recent fact verification dataset, to compare our\ndistantly-supervised method to a fully supervised ceiling system. Our manual\nevaluation indicates which automated evaluation metrics best correlate with\nhuman judgements of factuality and whether errors were actually corrected.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2012.15856v2",
            "date": "2020-12-31 18:59:08+00:00",
            "title": "Studying Strategically: Learning to Mask for Closed-book QA",
            "authors": [
                "Qinyuan Ye",
                "Belinda Z. Li",
                "Sinong Wang",
                "Benjamin Bolte",
                "Hao Ma",
                "Wen-tau Yih",
                "Xiang Ren",
                "Madian Khabsa"
            ],
            "abstract": "Closed-book question-answering (QA) is a challenging task that requires a\nmodel to directly answer questions without access to external knowledge. It has\nbeen shown that directly fine-tuning pre-trained language models with\n(question, answer) examples yields surprisingly competitive performance, which\nis further improved upon through adding an intermediate pre-training stage\nbetween general pre-training and fine-tuning. Prior work used a heuristic\nduring this intermediate stage, whereby named entities and dates are masked,\nand the model is trained to recover these tokens. In this paper, we aim to\nlearn the optimal masking strategy for the intermediate pre-training stage. We\nfirst train our masking policy to extract spans that are likely to be tested,\nusing supervision from the downstream task itself, then deploy the learned\npolicy during intermediate pre-training. Thus, our policy packs task-relevant\nknowledge into the parameters of a language model. Our approach is particularly\neffective on TriviaQA, outperforming strong heuristics when used to pre-train\nBART.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00010v1",
            "date": "2020-12-30 20:40:48+00:00",
            "title": "Unnatural Language Inference",
            "authors": [
                "Koustuv Sinha",
                "Prasanna Parthasarathi",
                "Joelle Pineau",
                "Adina Williams"
            ],
            "abstract": "Natural Language Understanding has witnessed a watershed moment with the\nintroduction of large pre-trained Transformer networks. These models achieve\nstate-of-the-art on various tasks, notably including Natural Language Inference\n(NLI). Many studies have shown that the large representation space imbibed by\nthe models encodes some syntactic and semantic information. However, to really\n\"know syntax\", a model must recognize when its input violates syntactic rules\nand calculate inferences accordingly. In this work, we find that\nstate-of-the-art NLI models, such as RoBERTa and BART are invariant to, and\nsometimes even perform better on, examples with randomly reordered words. With\niterative search, we are able to construct randomized versions of NLI test\nsets, which contain permuted hypothesis-premise pairs with the same words as\nthe original, yet are classified with perfect accuracy by large pre-trained\nmodels, as well as pre-Transformer state-of-the-art encoders. We find the issue\nto be language and model invariant, and hence investigate the root cause. To\npartially alleviate this effect, we propose a simple training methodology. Our\nfindings call into question the idea that our natural language understanding\nmodels, and the tasks used for measuring their progress, genuinely require a\nhuman-like understanding of syntax.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.00063v1",
            "date": "2020-12-31 20:38:20+00:00",
            "title": "EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets",
            "authors": [
                "Xiaohan Chen",
                "Yu Cheng",
                "Shuohang Wang",
                "Zhe Gan",
                "Zhangyang Wang",
                "Jingjing Liu"
            ],
            "abstract": "Deep, heavily overparameterized language models such as BERT, XLNet and T5\nhave achieved impressive success in many NLP tasks. However, their high model\ncomplexity requires enormous computation resources and extremely long training\ntime for both pre-training and fine-tuning. Many works have studied model\ncompression on large NLP models, but only focus on reducing inference\ncost/time, while still requiring expensive training process. Other works use\nextremely large batch sizes to shorten the pre-training time at the expense of\nhigh demand for computation resources. In this paper, inspired by the\nEarly-Bird Lottery Tickets studied for computer vision tasks, we propose\nEarlyBERT, a general computationally-efficient training algorithm applicable to\nboth pre-training and fine-tuning of large-scale language models. We are the\nfirst to identify structured winning tickets in the early stage of BERT\ntraining, and use them for efficient training. Comprehensive pre-training and\nfine-tuning experiments on GLUE and SQuAD downstream tasks show that EarlyBERT\neasily achieves comparable performance to standard BERT with 35~45% less\ntraining time.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.01142v1",
            "date": "2020-12-28 13:07:42+00:00",
            "title": "Advanced Machine Learning Techniques for Fake News (Online Disinformation) Detection: A Systematic Mapping Study",
            "authors": [
                "Michal Choras",
                "Konstantinos Demestichas",
                "Agata Gielczyk",
                "Alvaro Herrero",
                "Pawel Ksieniewicz",
                "Konstantina Remoundou",
                "Daniel Urda",
                "Michal Wozniak"
            ],
            "abstract": "Fake news has now grown into a big problem for societies and also a major\nchallenge for people fighting disinformation. This phenomenon plagues\ndemocratic elections, reputations of individual persons or organizations, and\nhas negatively impacted citizens, (e.g., during the COVID-19 pandemic in the US\nor Brazil). Hence, developing effective tools to fight this phenomenon by\nemploying advanced Machine Learning (ML) methods poses a significant challenge.\nThe following paper displays the present body of knowledge on the application\nof such intelligent tools in the fight against disinformation. It starts by\nshowing the historical perspective and the current role of fake news in the\ninformation war. Proposed solutions based solely on the work of experts are\nanalysed and the most important directions of the application of intelligent\nsystems in the detection of misinformation sources are pointed out.\nAdditionally, the paper presents some useful resources (mainly datasets useful\nwhen assessing ML solutions for fake news detection) and provides a short\noverview of the most important R&D projects related to this subject. The main\npurpose of this work is to analyse the current state of knowledge in detecting\nfake news; on the one hand to show possible solutions, and on the other hand to\nidentify the main challenges and methodological gaps to motivate future\nresearch.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.03024v2",
            "date": "2020-12-15 19:15:54+00:00",
            "title": "LiteMuL: A Lightweight On-Device Sequence Tagger using Multi-task Learning",
            "authors": [
                "Sonal Kumari",
                "Vibhav Agarwal",
                "Bharath Challa",
                "Kranti Chalamalasetti",
                "Sourav Ghosh",
                "Harshavardhana",
                "Barath Raj Kandur Raja"
            ],
            "abstract": "Named entity detection and Parts-of-speech tagging are the key tasks for many\nNLP applications. Although the current state of the art methods achieved near\nperfection for long, formal, structured text there are hindrances in deploying\nthese models on memory-constrained devices such as mobile phones. Furthermore,\nthe performance of these models is degraded when they encounter short,\ninformal, and casual conversations. To overcome these difficulties, we present\nLiteMuL - a lightweight on-device sequence tagger that can efficiently process\nthe user conversations using a Multi-Task Learning (MTL) approach. To the best\nof our knowledge, the proposed model is the first on-device MTL neural model\nfor sequence tagging. Our LiteMuL model is about 2.39 MB in size and achieved\nan accuracy of 0.9433 (for NER), 0.9090 (for POS) on the CoNLL 2003 dataset.\nThe proposed LiteMuL not only outperforms the current state of the art results\nbut also surpasses the results of our proposed on-device task-specific models,\nwith accuracy gains of up to 11% and model-size reduction by 50%-56%. Our model\nis competitive with other MTL approaches for NER and POS tasks while outshines\nthem with a low memory footprint. We also evaluated our model on custom-curated\nuser conversations and observed impressive results.",
            "journal ref": "2021 IEEE 15th International Conference on Semantic Computing\n  (ICSC), Laguna Hills, CA, USA, 2021, pp. 1-8",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.03025v1",
            "date": "2020-12-15 19:00:44+00:00",
            "title": "EmpLite: A Lightweight Sequence Labeling Model for Emphasis Selection of Short Texts",
            "authors": [
                "Vibhav Agarwal",
                "Sourav Ghosh",
                "Kranti Chalamalasetti",
                "Bharath Challa",
                "Sonal Kumari",
                "Harshavardhana",
                "Barath Raj Kandur Raja"
            ],
            "abstract": "Word emphasis in textual content aims at conveying the desired intention by\nchanging the size, color, typeface, style (bold, italic, etc.), and other\ntypographical features. The emphasized words are extremely helpful in drawing\nthe readers' attention to specific information that the authors wish to\nemphasize. However, performing such emphasis using a soft keyboard for social\nmedia interactions is time-consuming and has an associated learning curve. In\nthis paper, we propose a novel approach to automate the emphasis word detection\non short written texts. To the best of our knowledge, this work presents the\nfirst lightweight deep learning approach for smartphone deployment of emphasis\nselection. Experimental results show that our approach achieves comparable\naccuracy at a much lower model size than existing models. Our best lightweight\nmodel has a memory footprint of 2.82 MB with a matching score of 0.716 on\nSemEval-2020 public benchmark dataset.",
            "journal ref": "17th International Conference on Natural Language Processing\n  (ICON), Patna, India, December 18 - 21, 2020, pages 19-26, ACL Anthology:\n  2020.icon-1.3",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.03029v1",
            "date": "2020-12-10 19:07:31+00:00",
            "title": "\"Let's Eat Grandma\": When Punctuation Matters in Sentence Representation for Sentiment Analysis",
            "authors": [
                "Mansooreh Karami",
                "Ahmadreza Mosallanezhad",
                "Michelle V Mancenido",
                "Huan Liu"
            ],
            "abstract": "Neural network-based embeddings have been the mainstream approach for\ncreating a vector representation of the text to capture lexical and semantic\nsimilarities and dissimilarities. In general, existing encoding methods dismiss\nthe punctuation as insignificant information; consequently, they are routinely\neliminated in the pre-processing phase as they are shown to improve task\nperformance. In this paper, we hypothesize that punctuation could play a\nsignificant role in sentiment analysis and propose a novel representation model\nto improve syntactic and contextual performance. We corroborate our findings by\nconducting experiments on publicly available datasets and verify that our model\ncan identify the sentiments more accurately over other state-of-the-art\nbaseline methods.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2101.06111v1",
            "date": "2020-12-15 16:53:28+00:00",
            "title": "Knowledge Graphs and Natural-Language Processing",
            "authors": [
                "Andreas L Opdahl"
            ],
            "abstract": "Emergency-relevant data comes in many varieties. It can be high volume and\nhigh velocity, and reaction times are critical, calling for efficient and\npowerful techniques for data analysis and management. Knowledge graphs\nrepresent data in a rich, flexible, and uniform way that is well matched with\nthe needs of emergency management. They build on existing standards, resources,\ntechniques, and tools for semantic data and computing. This chapter explains\nthe most important semantic technologies and how they support knowledge graphs.\nWe proceed to discuss their benefits and challenges and give examples of\nrelevant semantic data sources and vocabularies. Natural-language texts -- in\nparticular those collected from social media such as Twitter -- is a type of\ndata source that poses particular analysis challenges. We therefore include an\noverview of techniques for processing natural-language texts.",
            "journal ref": "None",
            "category": "cs.CY"
        },
        {
            "url": "http://arxiv.org/abs/2104.11642v1",
            "date": "2020-08-21 13:30:44+00:00",
            "title": "Turkish Text Classification: From Lexicon Analysis to Bidirectional Transformer",
            "authors": [
                "Deniz Kavi"
            ],
            "abstract": "Text classification has seen an increased use in both academic and industry\nsettings. Though rule based methods have been fairly successful, supervised\nmachine learning has been shown to be most successful for most languages, where\nmost research was done on English. In this article, the success of lexicon\nanalysis, support vector machines, and extreme gradient boosting for the task\nof text classification and sentiment analysis are evaluated in Turkish and a\npretrained transformer based classifier is proposed, outperforming previous\nmethods for Turkish text classification. In the context of text classification,\nall machine learning models proposed in the article are domain-independent and\ndo not require any task-specific modifications.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00372v1",
            "date": "2020-01-02 09:51:51+00:00",
            "title": "Phase-based Information for Voice Pathology Detection",
            "authors": [
                "Thomas Drugman",
                "Thomas Dubuisson",
                "Thierry Dutoit"
            ],
            "abstract": "In most current approaches of speech processing, information is extracted\nfrom the magnitude spectrum. However recent perceptual studies have underlined\nthe importance of the phase component. The goal of this paper is to investigate\nthe potential of using phase-based features for automatically detecting voice\ndisorders. It is shown that group delay functions are appropriate for\ncharacterizing irregularities in the phonation. Besides the respect of the\nmixed-phase model of speech is discussed. The proposed phase-based features are\nevaluated and compared to other parameters derived from the magnitude spectrum.\nBoth streams are shown to be interestingly complementary. Furthermore\nphase-based features turn out to convey a great amount of relevant information,\nleading to high discrimination performance.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00571v1",
            "date": "2020-01-03 00:16:46+00:00",
            "title": "Question Type Classification Methods Comparison",
            "authors": [
                "Tamirlan Seidakhmetov"
            ],
            "abstract": "The paper presents a comparative study of state-of-the-art approaches for\nquestion classification task: Logistic Regression, Convolutional Neural\nNetworks (CNN), Long Short-Term Memory Network (LSTM) and Quasi-Recurrent\nNeural Networks (QRNN). All models use pre-trained GLoVe word embeddings and\ntrained on human-labeled data. The best accuracy is achieved using CNN model\nwith five convolutional layers and various kernel sizes stacked in parallel,\nfollowed by one fully connected layer. The model reached 90.7% accuracy on TREC\n10 test set. All the model architectures in this paper were developed from\nscratch on PyTorch, in few cases based on reliable open-source implementation.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00579v1",
            "date": "2020-01-02 09:25:30+00:00",
            "title": "A Comparative Evaluation of Pitch Modification Techniques",
            "authors": [
                "Thomas Drugman",
                "Thierry Dutoit"
            ],
            "abstract": "This paper addresses the problem of pitch modification, as an important\nmodule for an efficient voice transformation system. The Deterministic plus\nStochastic Model of the residual signal we proposed in a previous work is\ncompared to TDPSOLA, HNM and STRAIGHT. The four methods are compared through an\nimportant subjective test. The influence of the speaker gender and of the pitch\nmodification ratio is analyzed. Despite its higher compression level, the DSM\ntechnique is shown to give similar or better results than other methods,\nespecially for male speakers and important ratios of modification. The DSM\nturns out to be only outperformed by STRAIGHT for female voices.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00581v1",
            "date": "2020-01-02 09:39:07+00:00",
            "title": "Eigenresiduals for improved Parametric Speech Synthesis",
            "authors": [
                "Thomas Drugman",
                "Geoffrey Wilfart",
                "Thierry Dutoit"
            ],
            "abstract": "Statistical parametric speech synthesizers have recently shown their ability\nto produce natural-sounding and flexible voices. Unfortunately the delivered\nquality suffers from a typical buzziness due to the fact that speech is\nvocoded. This paper proposes a new excitation model in order to reduce this\nundesirable effect. This model is based on the decomposition of\npitch-synchronous residual frames on an orthonormal basis obtained by Principal\nComponent Analysis. This basis contains a limited number of eigenresiduals and\nis computed on a relatively small speech database. A stream of PCA-based\ncoefficients is added to our HMM-based synthesizer and allows to generate the\nvoiced excitation during the synthesis. An improvement compared to the\ntraditional excitation is reported while the synthesis engine footprint remains\nunder about 1Mb.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00582v1",
            "date": "2020-01-02 09:44:52+00:00",
            "title": "Excitation-based Voice Quality Analysis and Modification",
            "authors": [
                "Thomas Drugman",
                "Thierry Dutoit",
                "Baris Bozkurt"
            ],
            "abstract": "This paper investigates the differences occuring in the excitation for\ndifferent voice qualities. Its goal is two-fold. First a large corpus\ncontaining three voice qualities (modal, soft and loud) uttered by the same\nspeaker is analyzed and significant differences in characteristics extracted\nfrom the excitation are observed. Secondly rules of modification derived from\nthe analysis are used to build a voice quality transformation system applied as\na post-process to HMM-based speech synthesis. The system is shown to\neffectively achieve the transformations while maintaining the delivered\nquality.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00583v1",
            "date": "2020-01-02 10:04:37+00:00",
            "title": "On the Mutual Information between Source and Filter Contributions for Voice Pathology Detection",
            "authors": [
                "Thomas Drugman",
                "Thomas Dubuisson",
                "Thierry Dutoit"
            ],
            "abstract": "This paper addresses the problem of automatic detection of voice pathologies\ndirectly from the speech signal. For this, we investigate the use of the\nglottal source estimation as a means to detect voice disorders. Three sets of\nfeatures are proposed, depending on whether they are related to the speech or\nthe glottal signal, or to prosody. The relevancy of these features is assessed\nthrough mutual information-based measures. This allows an intuitive\ninterpretation in terms of discrimation power and redundancy between the\nfeatures, independently of any subsequent classifier. It is discussed which\ncharacteristics are interestingly informative or complementary for detecting\nvoice pathologies.",
            "journal ref": "None",
            "category": "cs.SD"
        },
        {
            "url": "http://arxiv.org/abs/2001.00781v1",
            "date": "2020-01-03 10:53:35+00:00",
            "title": "On the comparability of Pre-trained Language Models",
            "authors": [
                "Matthias A\u00dfenmacher",
                "Christian Heumann"
            ],
            "abstract": "Recent developments in unsupervised representation learning have successfully\nestablished the concept of transfer learning in NLP. Mainly three forces are\ndriving the improvements in this area of research: More elaborated\narchitectures are making better use of contextual information. Instead of\nsimply plugging in static pre-trained representations, these are learned based\non surrounding context in end-to-end trainable models with more intelligently\ndesigned language modelling objectives. Along with this, larger corpora are\nused as resources for pre-training large language models in a self-supervised\nfashion which are afterwards fine-tuned on supervised tasks. Advances in\nparallel computing as well as in cloud computing, made it possible to train\nthese models with growing capacities in the same or even in shorter time than\npreviously established models. These three developments agglomerate in new\nstate-of-the-art (SOTA) results being revealed in a higher and higher\nfrequency. It is not always obvious where these improvements originate from, as\nit is not possible to completely disentangle the contributions of the three\ndriving forces. We set ourselves to providing a clear and concise overview on\nseveral large pre-trained language models, which achieved SOTA results in the\nlast two years, with respect to their use of new architectures and resources.\nWe want to clarify for the reader where the differences between the models are\nand we furthermore attempt to gain some insight into the single contributions\nof lexical/computational improvements as well as of architectural changes. We\nexplicitly do not intend to quantify these contributions, but rather see our\nwork as an overview in order to identify potential starting points for\nbenchmark comparisons. Furthermore, we tentatively want to point at potential\npossibilities for improvement in the field of open-sourcing and reproducible\nresearch.",
            "journal ref": "Proceedings of the 5th Swiss Text Analytics Conference (SwissText)\n  & 16th Conference on Natural Language Processing (KONVENS), Zurich,\n  Switzerland, June 23-25, 2020",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.00926v1",
            "date": "2020-01-03 18:40:35+00:00",
            "title": "Learning Accurate Integer Transformer Machine-Translation Models",
            "authors": [
                "Ephrem Wu"
            ],
            "abstract": "We describe a method for training accurate Transformer machine-translation\nmodels to run inference using 8-bit integer (INT8) hardware matrix multipliers,\nas opposed to the more costly single-precision floating-point (FP32) hardware.\nUnlike previous work, which converted only 85 Transformer matrix\nmultiplications to INT8, leaving 48 out of 133 of them in FP32 because of\nunacceptable accuracy loss, we convert them all to INT8 without compromising\naccuracy. Tested on the newstest2014 English-to-German translation task, our\nINT8 Transformer Base and Transformer Big models yield BLEU scores that are\n99.3% to 100% relative to those of the corresponding FP32 models. Our approach\nconverts all matrix-multiplication tensors from an existing FP32 model into\nINT8 tensors by automatically making range-precision trade-offs during\ntraining. To demonstrate the robustness of this approach, we also include\nresults from INT6 Transformer models.",
            "journal ref": "None",
            "category": "cs.LG"
        },
        {
            "url": "http://arxiv.org/abs/2001.01037v4",
            "date": "2020-01-04 05:15:11+00:00",
            "title": "Understanding Image Captioning Models beyond Visualizing Attention",
            "authors": [
                "Jiamei Sun",
                "Sebastian Lapuschkin",
                "Wojciech Samek",
                "Alexander Binder"
            ],
            "abstract": "This paper interprets the predictions of image captioning models with\nattention mechanisms beyond visualizing the attention itself. In this paper, we\ndevelop variants of layer-wise relevance propagation (LRP) and gradient-based\nexplanation methods, tailored to image captioning models with attention\nmechanisms. We compare the interpretability of attention heatmaps\nsystematically against the explanations computed with explanation methods such\nas LRP, Grad-CAM, and Guided Grad-CAM. We show that explanation methods provide\nsimultaneously pixel-wise image explanation (supporting and opposing pixels of\nthe input image) and linguistic explanation (supporting and opposing words of\nthe preceding sequence) for each word in the predicted captions. We demonstrate\nwith extensive experiments that explanation methods can 1) reveal more related\nevidence used by the model to make decisions than attention; 2) correlate to\nobject locations with high precision; 3) is helpful to `debug' the model such\nas analyzing the reasons for hallucinated object words. With the observed\nproperties of explanations, we further design an LRP-inference fine-tuning\nstrategy that can alleviate the object hallucination of image captioning\nmodels, meanwhile, maintain the sentence fluency. We conduct experiments with\ntwo widely used attention mechanisms: the adaptive attention mechanism\ncalculated with the additive attention and the multi-head attention calculated\nwith the scaled dot product.",
            "journal ref": "None",
            "category": "cs.CV"
        },
        {
            "url": "http://arxiv.org/abs/2001.01115v2",
            "date": "2020-01-04 19:38:00+00:00",
            "title": "A Comprehensive Survey of Multilingual Neural Machine Translation",
            "authors": [
                "Raj Dabre",
                "Chenhui Chu",
                "Anoop Kunchukuttan"
            ],
            "abstract": "We present a survey on multilingual neural machine translation (MNMT), which\nhas gained a lot of traction in the recent years. MNMT has been useful in\nimproving translation quality as a result of translation knowledge transfer\n(transfer learning). MNMT is more promising and interesting than its\nstatistical machine translation counterpart because end-to-end modeling and\ndistributed representations open new avenues for research on machine\ntranslation. Many approaches have been proposed in order to exploit\nmultilingual parallel corpora for improving translation quality. However, the\nlack of a comprehensive survey makes it difficult to determine which approaches\nare promising and hence deserve further exploration. In this paper, we present\nan in-depth survey of existing literature on MNMT. We first categorize various\napproaches based on their central use-case and then further categorize them\nbased on resource scenarios, underlying modeling principles, core-issues and\nchallenges. Wherever possible we address the strengths and weaknesses of\nseveral techniques by comparing them with each other. We also discuss the\nfuture directions that MNMT research might take. This paper is aimed towards\nboth, beginners and experts in NMT. We hope this paper will serve as a starting\npoint as well as a source of new ideas for researchers and engineers interested\nin MNMT.",
            "journal ref": "None",
            "category": "cs.CL"
        },
        {
            "url": "http://arxiv.org/abs/2001.01126v1",
            "date": "2020-01-04 20:56:21+00:00",
            "title": "Can x2vec Save Lives? Integrating Graph and Language Embeddings for Automatic Mental Health Classification",
            "authors": [
                "Alexander Ruch"
            ],
            "abstract": "Graph and language embedding models ar